2024-05-27T07:02:15.650 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-27T07:02:15.651 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649 --verbose --name vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/ffsb}} -- /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649/orig.config.yaml 2024-05-27T07:02:15.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T07:02:15.801 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649 branch: wip-vshankar-testing-20240521.063558-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/ffsb}} email: vshankar@redhat.com first_in_suite: false job_id: '7727649' kernel: flavor: default kdb: 1 sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240521.063558-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240521.063558-debug sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 owner: scheduled_vshankar@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2499 sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 sleep_before_teardown: 0 subset: 573/1024 suite: fs suite_branch: wip-vshankar-testing-20240521.063558-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKV6y3epIlh036WPmR/vmgglKRaGHAgF2PyzHTqQW2+svsoRe4/3T4/gN35sspmsog6SZmKeHp8qJ2L7Fg1yLW4= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsIZNzlpAU2MmnrTU/pE6QBL7T+FuLIzUmzSEUeRLvGfixIeVypvTU/QW1LpeoJKOAtGrbinw5gz+Ta/JHcW+4= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAGo9qb19VVzS8hqfR9Ry2dzjnGzb1dqiCGvY4j7yIhY4gw3Us+oIxOxSMY5xKYQR/qTputGWj4dMz2lJzBv7zE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-27_05:52:49 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-27T07:02:15.801 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa; will attempt to use it 2024-05-27T07:02:15.802 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks 2024-05-27T07:02:15.802 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-27T07:02:15.804 INFO:teuthology.task.internal:Checking packages... 2024-05-27T07:02:15.824 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '38f9d900c83c63799fbdbe61acc9a11b0d3554a6' 2024-05-27T07:02:15.824 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-27T07:02:15.824 INFO:teuthology.packaging:ref: None 2024-05-27T07:02:15.824 INFO:teuthology.packaging:tag: None 2024-05-27T07:02:15.824 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240521.063558-debug 2024-05-27T07:02:15.824 INFO:teuthology.packaging:sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:02:15.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240521.063558-debug 2024-05-27T07:02:16.059 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3872.g38f9d900 2024-05-27T07:02:16.060 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-27T07:02:16.067 INFO:teuthology.task.internal:no buildpackages task found 2024-05-27T07:02:16.067 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-27T07:02:16.086 INFO:teuthology.task.internal:Saving configuration 2024-05-27T07:02:16.106 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-27T07:02:16.113 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-27T07:02:16.138 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 06:57:20.950057', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKV6y3epIlh036WPmR/vmgglKRaGHAgF2PyzHTqQW2+svsoRe4/3T4/gN35sspmsog6SZmKeHp8qJ2L7Fg1yLW4='} 2024-05-27T07:02:16.161 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 06:57:20.948283', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEsIZNzlpAU2MmnrTU/pE6QBL7T+FuLIzUmzSEUeRLvGfixIeVypvTU/QW1LpeoJKOAtGrbinw5gz+Ta/JHcW+4='} 2024-05-27T07:02:16.181 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 06:57:20.946290', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAGo9qb19VVzS8hqfR9Ry2dzjnGzb1dqiCGvY4j7yIhY4gw3Us+oIxOxSMY5xKYQR/qTputGWj4dMz2lJzBv7zE='} 2024-05-27T07:02:16.181 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-27T07:02:16.188 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-27T07:02:16.188 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-27T07:02:16.188 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-27T07:02:16.188 INFO:teuthology.run_tasks:Running task console_log... 2024-05-27T07:02:16.275 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f925f885090>, signals=[15]) 2024-05-27T07:02:16.276 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-27T07:02:16.282 INFO:teuthology.task.internal:Opening connections... 2024-05-27T07:02:16.282 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T07:02:16.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:02:16.362 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-05-27T07:02:16.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:02:16.442 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-05-27T07:02:16.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:02:16.518 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-27T07:02:16.524 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-05-27T07:02:16.542 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-05-27T07:02:16.543 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-05-27T07:02:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-05-27T07:02:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-05-27T07:02:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-05-27T07:02:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-05-27T07:02:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-05-27T07:02:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-05-27T07:02:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T07:02:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-05-27T07:02:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-05-27T07:02:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T07:02:16.602 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-05-27T07:02:16.602 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T07:02:16.602 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T07:02:16.602 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T07:02:16.602 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-05-27T07:02:16.617 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-05-27T07:02:16.635 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-05-27T07:02:16.636 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-05-27T07:02:16.692 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-05-27T07:02:16.692 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-05-27T07:02:16.692 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-05-27T07:02:16.692 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-05-27T07:02:16.692 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-05-27T07:02:16.693 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-05-27T07:02:16.693 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T07:02:16.693 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-05-27T07:02:16.693 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-05-27T07:02:16.693 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T07:02:16.693 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-05-27T07:02:16.693 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T07:02:16.693 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T07:02:16.693 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T07:02:16.693 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-05-27T07:02:16.717 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-05-27T07:02:16.735 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-05-27T07:02:16.735 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-05-27T07:02:16.792 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-05-27T07:02:16.792 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T07:02:16.793 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T07:02:16.794 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-05-27T07:02:16.811 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-27T07:02:16.821 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-27T07:02:16.831 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-27T07:02:16.831 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-05-27T07:02:16.833 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-05-27T07:02:16.835 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-05-27T07:02:16.851 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-27T07:02:16.856 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-27T07:02:16.857 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-05-27T07:02:16.891 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-05-27T07:02:16.893 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-05-27T07:02:16.937 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-27T07:02:17.073 INFO:teuthology.run_tasks:Running task kernel... 2024-05-27T07:02:17.093 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'} 2024-05-27T07:02:17.093 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.c': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, timeout 300 2024-05-27T07:02:17.093 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T07:02:17.094 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T07:02:17.095 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T07:02:17.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:02:17.111 DEBUG:teuthology.task.kernel:sha1 for mon.c is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:17.112 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T07:02:17.112 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-05-27T07:02:17.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:02:17.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:02:17.113 DEBUG:teuthology.task.kernel:sha1 for mon.a is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:17.113 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T07:02:17.114 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-27T07:02:17.114 DEBUG:teuthology.task.kernel:sha1 for mon.b is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:17.114 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T07:02:17.114 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-05-27T07:02:17.167 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-437.el9.x86_64 2024-05-27T07:02:17.168 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:17.168 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T07:02:17.168 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:17.173 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-437.el9.x86_64 2024-05-27T07:02:17.173 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-437.el9.x86_64 2024-05-27T07:02:17.174 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:17.174 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T07:02:17.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:17.175 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:17.175 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T07:02:17.175 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:17.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum upgrade -y linux-firmware 2024-05-27T07:02:17.328 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum upgrade -y linux-firmware 2024-05-27T07:02:17.417 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum upgrade -y linux-firmware 2024-05-27T07:02:19.501 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-27T07:02:20.542 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.1 MB 00:01 2024-05-27T07:02:22.890 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.1 MB 00:04 2024-05-27T07:02:24.101 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-05-27T07:02:26.324 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 19 MB 00:02 2024-05-27T07:02:28.901 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-05-27T07:02:30.366 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.2 MB 00:00 2024-05-27T07:02:31.461 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 17 kB 00:01 2024-05-27T07:02:32.050 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 17 kB 00:00 2024-05-27T07:02:32.645 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-27T07:02:33.460 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-27T07:02:37.325 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-05-27T07:02:38.037 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-27T07:02:38.635 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-27T07:02:40.133 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-27T07:02:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-05-27T07:02:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-27T07:02:40.237 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-27T07:02:40.237 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:40.536 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T07:02:40.536 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T07:02:40.539 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi063.front.sepia.ceph.com to complete... 2024-05-27T07:02:40.649 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-05-27T07:02:40.650 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-05-27T07:02:40.650 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-05-27T07:02:40.766 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-27T07:02:40.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:40.903 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T07:02:40.903 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T07:02:40.906 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-05-27T07:02:42.167 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.2 MB 00:01 2024-05-27T07:02:43.526 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 17 kB 00:00 2024-05-27T07:02:49.177 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 3.8 MB/s | 21 MB 00:05 2024-05-27T07:02:54.444 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-27T07:02:56.467 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-27T07:02:56.468 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-05-27T07:02:56.468 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-27T07:02:56.589 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.c... 2024-05-27T07:02:56.589 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:02:56.814 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T07:02:56.814 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T07:02:56.817 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi149.front.sepia.ceph.com to complete... 2024-05-27T07:03:20.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27 07:03:20 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T07:03:20.475 INFO:teuthology.orchestra.run.smithi063.stderr:FINISHED --2024-05-27 07:03:20-- 2024-05-27T07:03:20.476 INFO:teuthology.orchestra.run.smithi063.stderr:Total wall clock time: 40s 2024-05-27T07:03:20.476 INFO:teuthology.orchestra.run.smithi063.stderr:Downloaded: 1 files, 340M in 39s (8.64 MB/s) 2024-05-27T07:03:20.479 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-27T07:03:20.479 DEBUG:teuthology.orchestra.run.smithi063:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T07:03:22.954 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying... ######################################## 2024-05-27T07:03:23.071 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing... ######################################## 2024-05-27T07:03:23.182 INFO:teuthology.orchestra.run.smithi063.stdout:Updating / installing... 2024-05-27T07:03:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27 07:03:30 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T07:03:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:FINISHED --2024-05-27 07:03:30-- 2024-05-27T07:03:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:Total wall clock time: 49s 2024-05-27T07:03:30.031 INFO:teuthology.orchestra.run.smithi086.stderr:Downloaded: 1 files, 340M in 48s (7.02 MB/s) 2024-05-27T07:03:30.033 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-27T07:03:30.034 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T07:03:31.469 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T07:03:32.026 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27 07:03:32 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T07:03:32.026 INFO:teuthology.orchestra.run.smithi149.stderr:FINISHED --2024-05-27 07:03:32-- 2024-05-27T07:03:32.026 INFO:teuthology.orchestra.run.smithi149.stderr:Total wall clock time: 35s 2024-05-27T07:03:32.026 INFO:teuthology.orchestra.run.smithi149.stderr:Downloaded: 1 files, 340M in 35s (9.83 MB/s) 2024-05-27T07:03:32.029 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.c... 2024-05-27T07:03:32.029 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T07:03:32.119 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying... ######################################## 2024-05-27T07:03:32.336 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing... ######################################## 2024-05-27T07:03:32.451 INFO:teuthology.orchestra.run.smithi086.stdout:Updating / installing... 2024-05-27T07:03:34.121 INFO:teuthology.orchestra.run.smithi149.stdout:Verifying... ######################################## 2024-05-27T07:03:34.331 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing... ######################################## 2024-05-27T07:03:34.434 INFO:teuthology.orchestra.run.smithi149.stdout:Updating / installing... 2024-05-27T07:03:40.451 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T07:03:42.482 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T07:04:09.196 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi063.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T07:04:09.196 DEBUG:teuthology.orchestra.run.smithi063:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T07:04:09.274 INFO:teuthology.orchestra.run.smithi063.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T07:04:09.274 INFO:teuthology.orchestra.run.smithi063.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T07:04:09.274 INFO:teuthology.orchestra.run.smithi063.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T07:04:09.274 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T07:04:09.274 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T07:04:09.274 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T07:04:09.274 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T07:04:09.275 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T07:04:09.276 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T07:04:09.277 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T07:04:09.278 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T07:04:09.279 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T07:04:09.280 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T07:04:09.281 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T07:04:09.282 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T07:04:09.283 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T07:04:09.284 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T07:04:09.285 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T07:04:09.286 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T07:04:09.287 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T07:04:09.288 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T07:04:09.289 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T07:04:09.290 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T07:04:09.291 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T07:04:09.292 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T07:04:09.293 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T07:04:09.294 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T07:04:09.295 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T07:04:09.296 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T07:04:09.297 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T07:04:09.298 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T07:04:09.299 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T07:04:09.300 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T07:04:09.301 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T07:04:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T07:04:09.303 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T07:04:09.304 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T07:04:09.304 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T07:04:09.304 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T07:04:09.304 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T07:04:09.304 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T07:04:09.304 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T07:04:09.304 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T07:04:09.304 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T07:04:09.304 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T07:04:09.304 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T07:04:09.305 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T07:04:09.306 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T07:04:09.307 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T07:04:09.308 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T07:04:09.309 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T07:04:09.310 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T07:04:09.311 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T07:04:09.312 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T07:04:09.313 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T07:04:09.314 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T07:04:09.315 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T07:04:09.316 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T07:04:09.317 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T07:04:09.318 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T07:04:09.319 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T07:04:09.320 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T07:04:09.321 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T07:04:09.322 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T07:04:09.323 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T07:04:09.324 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T07:04:09.325 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T07:04:09.326 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T07:04:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T07:04:09.328 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T07:04:09.329 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T07:04:09.330 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T07:04:09.331 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T07:04:09.332 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T07:04:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T07:04:09.334 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T07:04:09.335 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T07:04:09.336 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T07:04:09.337 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T07:04:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T07:04:09.339 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T07:04:09.340 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T07:04:09.341 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T07:04:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T07:04:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T07:04:09.344 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T07:04:09.345 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T07:04:09.346 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T07:04:09.347 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T07:04:09.348 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T07:04:09.349 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T07:04:09.350 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T07:04:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T07:04:09.352 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T07:04:09.353 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T07:04:09.354 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T07:04:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T07:04:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T07:04:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T07:04:09.358 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T07:04:09.359 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T07:04:09.360 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T07:04:09.361 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T07:04:09.362 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T07:04:09.363 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T07:04:09.364 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T07:04:09.365 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T07:04:09.366 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T07:04:09.367 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T07:04:09.368 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T07:04:09.369 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T07:04:09.370 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T07:04:09.371 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T07:04:09.372 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T07:04:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T07:04:09.374 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T07:04:09.375 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T07:04:09.376 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T07:04:09.377 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T07:04:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T07:04:09.379 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T07:04:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T07:04:09.381 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T07:04:09.382 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T07:04:09.383 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T07:04:09.384 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T07:04:09.385 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T07:04:09.386 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T07:04:09.387 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T07:04:09.388 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T07:04:09.389 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T07:04:09.390 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T07:04:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T07:04:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T07:04:09.393 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T07:04:09.394 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T07:04:09.395 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T07:04:09.396 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T07:04:09.397 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T07:04:09.398 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T07:04:09.399 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T07:04:09.400 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T07:04:09.401 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T07:04:09.402 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T07:04:09.403 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T07:04:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T07:04:09.405 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T07:04:09.406 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T07:04:09.407 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T07:04:09.408 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T07:04:09.409 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T07:04:09.410 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T07:04:09.411 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T07:04:09.412 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T07:04:09.413 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T07:04:09.414 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T07:04:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T07:04:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T07:04:09.417 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T07:04:09.418 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T07:04:09.419 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T07:04:09.420 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T07:04:09.421 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T07:04:09.421 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T07:04:09.421 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T07:04:09.421 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T07:04:09.421 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T07:04:09.421 INFO:teuthology.orchestra.run.smithi063.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:04:09.425 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T07:04:09.425 DEBUG:teuthology.orchestra.run.smithi063:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout:fi 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout:for file in vmlinuz System.map config; do 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: fi 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout:done 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T07:04:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T07:04:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:04:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:04:09.451 INFO:teuthology.orchestra.run.smithi063.stdout:fi 2024-05-27T07:04:09.451 INFO:teuthology.orchestra.run.smithi063.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T07:04:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T07:04:09.451 INFO:teuthology.orchestra.run.smithi063.stdout:fi 2024-05-27T07:04:09.451 INFO:teuthology.orchestra.run.smithi063.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T07:04:09.451 INFO:teuthology.orchestra.run.smithi063.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:04:09.451 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T07:04:09.451 INFO:teuthology.orchestra.run.smithi063.stdout:fi 2024-05-27T07:04:09.452 DEBUG:teuthology.orchestra.run.smithi063:> sudo rpm -qi grub2-tools 2024-05-27T07:04:09.537 INFO:teuthology.orchestra.run.smithi063.stdout:Name : grub2-tools 2024-05-27T07:04:09.537 INFO:teuthology.orchestra.run.smithi063.stdout:Epoch : 1 2024-05-27T07:04:09.537 INFO:teuthology.orchestra.run.smithi063.stdout:Version : 2.06 2024-05-27T07:04:09.537 INFO:teuthology.orchestra.run.smithi063.stdout:Release : 68.el9 2024-05-27T07:04:09.537 INFO:teuthology.orchestra.run.smithi063.stdout:Architecture: x86_64 2024-05-27T07:04:09.537 INFO:teuthology.orchestra.run.smithi063.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T07:04:09.537 INFO:teuthology.orchestra.run.smithi063.stdout:Group : Unspecified 2024-05-27T07:04:09.537 INFO:teuthology.orchestra.run.smithi063.stdout:Size : 8264254 2024-05-27T07:04:09.537 INFO:teuthology.orchestra.run.smithi063.stdout:License : GPLv3+ 2024-05-27T07:04:09.537 INFO:teuthology.orchestra.run.smithi063.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:Packager : builder@centos.org 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:Vendor : CentOS 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:Summary : Support tools for GRUB. 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:Description : 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T07:04:09.538 INFO:teuthology.orchestra.run.smithi063.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T07:04:09.539 INFO:teuthology.orchestra.run.smithi063.stdout:hardware devices. 2024-05-27T07:04:09.539 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:04:09.539 INFO:teuthology.orchestra.run.smithi063.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T07:04:09.539 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T07:04:09.540 INFO:teuthology.task.kernel:Updating grub on smithi063 to boot 6.9.0-gb9459a26b088 2024-05-27T07:04:09.540 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T07:04:10.129 INFO:teuthology.orchestra.run.smithi063.stderr:Generating grub configuration file ... 2024-05-27T07:04:11.529 INFO:teuthology.orchestra.run.smithi063.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T07:04:11.560 INFO:teuthology.orchestra.run.smithi063.stderr:done 2024-05-27T07:04:11.563 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-27T07:04:11.583 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.lCkSA0ZlwA 2024-05-27T07:04:11.584 DEBUG:teuthology.orchestra.run.smithi063:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lCkSA0ZlwA 2024-05-27T07:04:11.650 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.lCkSA0ZlwA 2024-05-27T07:04:11.794 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.lCkSA0ZlwA is 6KB 2024-05-27T07:04:11.843 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.lCkSA0ZlwA 2024-05-27T07:04:11.858 DEBUG:teuthology.orchestra.run.smithi063:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T07:04:11.921 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T07:04:11.921 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T07:04:11.921 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T07:04:11.921 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T07:04:11.923 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T07:04:12.070 DEBUG:teuthology.orchestra.run.smithi063:> sudo shutdown -r now 2024-05-27T07:04:17.979 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T07:04:17.980 DEBUG:teuthology.orchestra.run.smithi086:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T07:04:18.020 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi149.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T07:04:18.020 DEBUG:teuthology.orchestra.run.smithi149:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T07:04:18.116 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T07:04:18.117 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T07:04:18.117 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T07:04:18.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T07:04:18.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T07:04:18.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T07:04:18.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T07:04:18.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T07:04:18.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T07:04:18.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T07:04:18.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T07:04:18.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T07:04:18.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T07:04:18.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T07:04:18.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T07:04:18.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T07:04:18.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T07:04:18.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T07:04:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T07:04:18.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T07:04:18.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T07:04:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T07:04:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T07:04:18.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T07:04:18.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T07:04:18.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T07:04:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T07:04:18.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T07:04:18.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T07:04:18.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T07:04:18.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T07:04:18.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T07:04:18.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T07:04:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T07:04:18.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T07:04:18.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T07:04:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T07:04:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T07:04:18.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T07:04:18.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T07:04:18.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T07:04:18.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T07:04:18.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T07:04:18.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T07:04:18.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T07:04:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T07:04:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T07:04:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T07:04:18.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T07:04:18.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T07:04:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T07:04:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T07:04:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T07:04:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T07:04:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T07:04:18.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T07:04:18.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T07:04:18.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T07:04:18.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T07:04:18.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T07:04:18.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T07:04:18.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T07:04:18.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T07:04:18.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T07:04:18.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T07:04:18.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T07:04:18.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T07:04:18.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T07:04:18.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T07:04:18.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T07:04:18.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T07:04:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T07:04:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T07:04:18.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T07:04:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T07:04:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T07:04:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T07:04:18.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T07:04:18.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T07:04:18.186 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T07:04:18.187 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T07:04:18.188 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T07:04:18.189 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T07:04:18.190 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T07:04:18.191 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T07:04:18.192 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T07:04:18.193 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T07:04:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T07:04:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T07:04:18.196 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T07:04:18.197 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T07:04:18.198 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T07:04:18.199 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T07:04:18.200 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T07:04:18.201 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T07:04:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T07:04:18.203 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T07:04:18.204 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T07:04:18.205 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T07:04:18.206 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T07:04:18.207 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T07:04:18.208 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T07:04:18.209 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T07:04:18.210 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T07:04:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T07:04:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T07:04:18.213 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T07:04:18.214 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T07:04:18.215 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T07:04:18.216 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T07:04:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T07:04:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T07:04:18.219 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T07:04:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T07:04:18.221 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T07:04:18.222 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T07:04:18.223 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T07:04:18.224 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T07:04:18.225 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T07:04:18.226 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T07:04:18.227 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T07:04:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T07:04:18.229 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T07:04:18.230 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T07:04:18.231 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T07:04:18.232 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T07:04:18.233 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T07:04:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T07:04:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T07:04:18.236 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T07:04:18.237 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T07:04:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T07:04:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T07:04:18.240 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T07:04:18.241 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T07:04:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T07:04:18.243 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T07:04:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T07:04:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T07:04:18.246 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T07:04:18.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T07:04:18.247 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T07:04:18.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T07:04:18.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T07:04:18.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T07:04:18.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T07:04:18.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T07:04:18.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T07:04:18.250 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T07:04:18.251 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T07:04:18.252 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T07:04:18.253 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T07:04:18.254 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T07:04:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T07:04:18.256 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T07:04:18.257 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T07:04:18.258 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T07:04:18.259 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T07:04:18.260 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T07:04:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T07:04:18.262 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T07:04:18.263 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T07:04:18.264 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T07:04:18.265 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T07:04:18.266 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T07:04:18.267 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T07:04:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T07:04:18.269 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T07:04:18.270 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T07:04:18.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T07:04:18.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T07:04:18.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T07:04:18.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T07:04:18.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T07:04:18.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T07:04:18.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T07:04:18.271 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:04:18.275 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T07:04:18.275 DEBUG:teuthology.orchestra.run.smithi086:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T07:04:18.289 INFO:teuthology.orchestra.run.smithi149.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T07:04:18.289 INFO:teuthology.orchestra.run.smithi149.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T07:04:18.290 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T07:04:18.291 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T07:04:18.292 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T07:04:18.293 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T07:04:18.294 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T07:04:18.295 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T07:04:18.296 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T07:04:18.297 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T07:04:18.298 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T07:04:18.299 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T07:04:18.300 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T07:04:18.301 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T07:04:18.302 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T07:04:18.303 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T07:04:18.304 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T07:04:18.305 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T07:04:18.306 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T07:04:18.307 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T07:04:18.308 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T07:04:18.309 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T07:04:18.310 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T07:04:18.311 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T07:04:18.312 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T07:04:18.313 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T07:04:18.314 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T07:04:18.315 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T07:04:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T07:04:18.317 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T07:04:18.318 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T07:04:18.319 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T07:04:18.320 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T07:04:18.321 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T07:04:18.322 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T07:04:18.323 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T07:04:18.324 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T07:04:18.325 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T07:04:18.326 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T07:04:18.327 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T07:04:18.328 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T07:04:18.329 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T07:04:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T07:04:18.331 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T07:04:18.332 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T07:04:18.333 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T07:04:18.334 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T07:04:18.335 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T07:04:18.336 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T07:04:18.337 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T07:04:18.338 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T07:04:18.339 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T07:04:18.340 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T07:04:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T07:04:18.342 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T07:04:18.343 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T07:04:18.344 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T07:04:18.345 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T07:04:18.346 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T07:04:18.347 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T07:04:18.348 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T07:04:18.349 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T07:04:18.350 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T07:04:18.351 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T07:04:18.352 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T07:04:18.353 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T07:04:18.354 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T07:04:18.355 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T07:04:18.356 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T07:04:18.357 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T07:04:18.358 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T07:04:18.359 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T07:04:18.360 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T07:04:18.361 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T07:04:18.362 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T07:04:18.363 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T07:04:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T07:04:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T07:04:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T07:04:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T07:04:18.368 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T07:04:18.369 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T07:04:18.370 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T07:04:18.371 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T07:04:18.372 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T07:04:18.373 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T07:04:18.374 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T07:04:18.375 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T07:04:18.376 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T07:04:18.377 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T07:04:18.378 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T07:04:18.379 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T07:04:18.380 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T07:04:18.381 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T07:04:18.382 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T07:04:18.383 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T07:04:18.384 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T07:04:18.385 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T07:04:18.386 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T07:04:18.387 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T07:04:18.388 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T07:04:18.389 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T07:04:18.390 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T07:04:18.391 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T07:04:18.392 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T07:04:18.393 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T07:04:18.394 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T07:04:18.395 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T07:04:18.396 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T07:04:18.397 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T07:04:18.398 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T07:04:18.399 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T07:04:18.400 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T07:04:18.401 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T07:04:18.402 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T07:04:18.403 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T07:04:18.404 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T07:04:18.405 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T07:04:18.406 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T07:04:18.407 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T07:04:18.408 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T07:04:18.409 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T07:04:18.410 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T07:04:18.411 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T07:04:18.412 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T07:04:18.413 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T07:04:18.414 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T07:04:18.415 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T07:04:18.416 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T07:04:18.417 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T07:04:18.418 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T07:04:18.419 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T07:04:18.420 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T07:04:18.421 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T07:04:18.422 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T07:04:18.423 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T07:04:18.424 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T07:04:18.425 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T07:04:18.426 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T07:04:18.427 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T07:04:18.428 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T07:04:18.429 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T07:04:18.430 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T07:04:18.431 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T07:04:18.432 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T07:04:18.433 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T07:04:18.434 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T07:04:18.435 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T07:04:18.436 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T07:04:18.437 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T07:04:18.438 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T07:04:18.439 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T07:04:18.440 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:04:18.443 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T07:04:18.443 DEBUG:teuthology.orchestra.run.smithi149:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout:for file in vmlinuz System.map config; do 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout: fi 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout:done 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T07:04:18.445 INFO:teuthology.orchestra.run.smithi086.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T07:04:18.446 INFO:teuthology.orchestra.run.smithi086.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:04:18.446 INFO:teuthology.orchestra.run.smithi086.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:04:18.446 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-05-27T07:04:18.446 INFO:teuthology.orchestra.run.smithi086.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T07:04:18.446 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T07:04:18.446 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-05-27T07:04:18.446 INFO:teuthology.orchestra.run.smithi086.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T07:04:18.446 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:04:18.446 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T07:04:18.446 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-05-27T07:04:18.446 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -qi grub2-tools 2024-05-27T07:04:18.467 INFO:teuthology.orchestra.run.smithi149.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout:for file in vmlinuz System.map config; do 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout: fi 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout:done 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:04:18.468 INFO:teuthology.orchestra.run.smithi149.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:04:18.469 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-05-27T07:04:18.469 INFO:teuthology.orchestra.run.smithi149.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T07:04:18.469 INFO:teuthology.orchestra.run.smithi149.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T07:04:18.469 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-05-27T07:04:18.469 INFO:teuthology.orchestra.run.smithi149.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T07:04:18.469 INFO:teuthology.orchestra.run.smithi149.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:04:18.469 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T07:04:18.469 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-05-27T07:04:18.470 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -qi grub2-tools 2024-05-27T07:04:18.516 INFO:teuthology.orchestra.run.smithi086.stdout:Name : grub2-tools 2024-05-27T07:04:18.516 INFO:teuthology.orchestra.run.smithi086.stdout:Epoch : 1 2024-05-27T07:04:18.516 INFO:teuthology.orchestra.run.smithi086.stdout:Version : 2.06 2024-05-27T07:04:18.516 INFO:teuthology.orchestra.run.smithi086.stdout:Release : 68.el9 2024-05-27T07:04:18.516 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: x86_64 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Group : Unspecified 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Size : 8264254 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:License : GPLv3+ 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Packager : builder@centos.org 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Vendor : CentOS 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Summary : Support tools for GRUB. 2024-05-27T07:04:18.517 INFO:teuthology.orchestra.run.smithi086.stdout:Description : 2024-05-27T07:04:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:04:18.518 INFO:teuthology.orchestra.run.smithi086.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T07:04:18.518 INFO:teuthology.orchestra.run.smithi086.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T07:04:18.518 INFO:teuthology.orchestra.run.smithi086.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T07:04:18.518 INFO:teuthology.orchestra.run.smithi086.stdout:hardware devices. 2024-05-27T07:04:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:04:18.518 INFO:teuthology.orchestra.run.smithi086.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T07:04:18.519 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T07:04:18.519 INFO:teuthology.task.kernel:Updating grub on smithi086 to boot 6.9.0-gb9459a26b088 2024-05-27T07:04:18.519 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T07:04:18.544 INFO:teuthology.orchestra.run.smithi149.stdout:Name : grub2-tools 2024-05-27T07:04:18.544 INFO:teuthology.orchestra.run.smithi149.stdout:Epoch : 1 2024-05-27T07:04:18.544 INFO:teuthology.orchestra.run.smithi149.stdout:Version : 2.06 2024-05-27T07:04:18.544 INFO:teuthology.orchestra.run.smithi149.stdout:Release : 68.el9 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Architecture: x86_64 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Group : Unspecified 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Size : 8264254 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:License : GPLv3+ 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Packager : builder@centos.org 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Vendor : CentOS 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Summary : Support tools for GRUB. 2024-05-27T07:04:18.545 INFO:teuthology.orchestra.run.smithi149.stdout:Description : 2024-05-27T07:04:18.546 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:04:18.546 INFO:teuthology.orchestra.run.smithi149.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T07:04:18.546 INFO:teuthology.orchestra.run.smithi149.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T07:04:18.546 INFO:teuthology.orchestra.run.smithi149.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T07:04:18.546 INFO:teuthology.orchestra.run.smithi149.stdout:hardware devices. 2024-05-27T07:04:18.546 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:04:18.546 INFO:teuthology.orchestra.run.smithi149.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T07:04:18.547 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T07:04:18.547 INFO:teuthology.task.kernel:Updating grub on smithi149 to boot 6.9.0-gb9459a26b088 2024-05-27T07:04:18.547 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T07:04:19.040 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-05-27T07:04:19.131 INFO:teuthology.orchestra.run.smithi149.stderr:Generating grub configuration file ... 2024-05-27T07:04:20.459 INFO:teuthology.orchestra.run.smithi149.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T07:04:20.474 INFO:teuthology.orchestra.run.smithi086.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T07:04:20.495 INFO:teuthology.orchestra.run.smithi149.stderr:done 2024-05-27T07:04:20.497 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-05-27T07:04:20.517 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.iAAGYhuMlg 2024-05-27T07:04:20.517 DEBUG:teuthology.orchestra.run.smithi149:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.iAAGYhuMlg 2024-05-27T07:04:20.520 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-05-27T07:04:20.522 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-05-27T07:04:20.537 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.Gz6uUmJ224 2024-05-27T07:04:20.537 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Gz6uUmJ224 2024-05-27T07:04:20.584 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.iAAGYhuMlg 2024-05-27T07:04:20.641 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.Gz6uUmJ224 2024-05-27T07:04:20.719 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.iAAGYhuMlg is 6KB 2024-05-27T07:04:20.744 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.Gz6uUmJ224 is 6KB 2024-05-27T07:04:20.768 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.iAAGYhuMlg 2024-05-27T07:04:20.782 DEBUG:teuthology.orchestra.run.smithi149:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T07:04:20.792 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.Gz6uUmJ224 2024-05-27T07:04:20.806 DEBUG:teuthology.orchestra.run.smithi086:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T07:04:20.844 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T07:04:20.844 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T07:04:20.844 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T07:04:20.844 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T07:04:20.845 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T07:04:20.869 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T07:04:20.870 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T07:04:20.870 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T07:04:20.870 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T07:04:20.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T07:04:20.995 DEBUG:teuthology.orchestra.run.smithi149:> sudo shutdown -r now 2024-05-27T07:04:21.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-05-27T07:04:42.090 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-27T07:04:42.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-27T07:04:42.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:04:51.006 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-27T07:04:51.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-27T07:04:51.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:04:51.014 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-27T07:04:51.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-05-27T07:04:51.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:05:16.575 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-27T07:05:25.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-05-27T07:05:25.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-27T07:05:25.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:05:34.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-27T07:05:34.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:05:43.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-27T07:05:51.022 DEBUG:teuthology.orchestra.remote:timed out 2024-05-27T07:05:52.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-05-27T07:05:55.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-27T07:05:55.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:05:56.445 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-27T07:05:57.107 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-27T07:05:57.107 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-gb9459a26b088"... 2024-05-27T07:05:57.108 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-27T07:05:57.149 INFO:teuthology.orchestra.run.smithi063.stdout:6.9.0-gb9459a26b088 2024-05-27T07:05:57.149 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T07:05:57.149 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T07:05:57.149 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T07:05:58.151 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-27T07:05:58.151 DEBUG:teuthology.orchestra.run.smithi063:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T07:05:58.317 INFO:teuthology.orchestra.run.smithi063.stdout:ttyS1 2024-05-27T07:05:58.372 DEBUG:teuthology.parallel:result is None 2024-05-27T07:06:00.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-05-27T07:06:00.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:06:03.497 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-05-27T07:06:04.076 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-05-27T07:06:04.076 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-gb9459a26b088"... 2024-05-27T07:06:04.077 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-05-27T07:06:04.114 INFO:teuthology.orchestra.run.smithi086.stdout:6.9.0-gb9459a26b088 2024-05-27T07:06:04.114 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T07:06:04.115 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T07:06:04.115 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T07:06:04.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-27T07:06:04.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:06:05.119 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-27T07:06:05.119 DEBUG:teuthology.orchestra.run.smithi086:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T07:06:05.239 INFO:teuthology.orchestra.run.smithi086.stdout:ttyS1 2024-05-27T07:06:05.311 DEBUG:teuthology.parallel:result is None 2024-05-27T07:06:08.561 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-05-27T07:06:09.108 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-27T07:06:09.108 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-gb9459a26b088"... 2024-05-27T07:06:09.109 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-05-27T07:06:09.148 INFO:teuthology.orchestra.run.smithi149.stdout:6.9.0-gb9459a26b088 2024-05-27T07:06:09.148 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T07:06:09.148 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T07:06:09.148 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T07:06:10.149 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-27T07:06:10.150 DEBUG:teuthology.orchestra.run.smithi149:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T07:06:10.286 INFO:teuthology.orchestra.run.smithi149.stdout:ttyS1 2024-05-27T07:06:10.347 DEBUG:teuthology.parallel:result is None 2024-05-27T07:06:10.348 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-27T07:06:10.359 INFO:teuthology.task.internal:Creating test directory... 2024-05-27T07:06:10.359 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T07:06:10.365 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T07:06:10.373 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T07:06:10.428 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-27T07:06:10.487 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-27T07:06:10.534 INFO:teuthology.task.internal:Creating archive directory... 2024-05-27T07:06:10.535 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T07:06:10.542 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T07:06:10.548 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T07:06:10.772 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-27T07:06:10.831 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-27T07:06:10.831 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T07:06:10.838 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T07:06:10.843 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T07:06:10.912 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:06:10.916 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:06:10.938 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:06:10.943 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:06:10.947 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:06:10.973 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:06:10.978 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-27T07:06:10.986 INFO:teuthology.task.internal:Configuring sudo... 2024-05-27T07:06:10.986 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T07:06:10.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T07:06:10.995 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T07:06:11.065 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-27T07:06:11.087 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-27T07:06:11.087 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T07:06:11.092 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T07:06:11.096 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T07:06:11.139 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:06:11.210 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:06:11.283 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:06:11.361 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:06:11.436 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:06:11.436 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T07:06:11.511 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:06:11.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:06:11.657 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:06:11.725 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:06:11.813 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:06:11.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T07:06:11.874 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:06:11.944 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:06:12.032 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:06:12.104 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:06:12.167 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:06:12.168 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T07:06:12.229 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-05-27T07:06:12.241 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-05-27T07:06:12.248 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-05-27T07:06:12.332 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T07:06:12.381 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T07:06:12.398 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T07:06:12.705 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-27T07:06:12.713 INFO:teuthology.task.internal:Starting timer... 2024-05-27T07:06:12.713 INFO:teuthology.run_tasks:Running task pcp... 2024-05-27T07:06:12.755 INFO:teuthology.run_tasks:Running task selinux... 2024-05-27T07:06:12.794 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-27T07:06:12.795 DEBUG:teuthology.orchestra.run.smithi063:> sudo service auditd rotate 2024-05-27T07:06:13.002 INFO:teuthology.orchestra.run.smithi063.stdout:Rotating logs: 2024-05-27T07:06:13.007 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-05-27T07:06:13.130 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-05-27T07:06:13.134 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-05-27T07:06:13.294 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-05-27T07:06:13.299 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-27T07:06:13.299 DEBUG:teuthology.orchestra.run.smithi063:> /usr/sbin/getenforce 2024-05-27T07:06:13.389 INFO:teuthology.orchestra.run.smithi063.stdout:Permissive 2024-05-27T07:06:13.389 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-05-27T07:06:13.450 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-05-27T07:06:13.450 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-05-27T07:06:13.504 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-05-27T07:06:13.504 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi063.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2024-05-27T07:06:13.504 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T07:06:13.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:06:13.612 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T07:06:13.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:06:13.671 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T07:06:13.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:06:13.745 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-27T07:06:13.745 DEBUG:teuthology.orchestra.run.smithi063:> sudo /usr/sbin/setenforce permissive 2024-05-27T07:06:13.867 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-05-27T07:06:13.935 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-05-27T07:06:14.003 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-27T07:06:14.017 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-27T07:06:14.031 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-27T07:06:14.032 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi063.front.sepia.ceph.com,smithi086.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-27T07:12:55.121 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi063.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-05-27T07:12:55.122 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-27T07:12:55.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:12:55.240 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-27T07:12:55.402 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-27T07:12:55.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-05-27T07:12:55.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:12:55.510 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-05-27T07:12:55.656 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-05-27T07:12:55.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-27T07:12:55.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:12:55.765 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-05-27T07:12:55.919 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-27T07:12:55.920 INFO:teuthology.run_tasks:Running task clock... 2024-05-27T07:12:55.931 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-27T07:12:55.932 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T07:12:55.933 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T07:12:55.941 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T07:12:55.941 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T07:12:55.948 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T07:12:55.949 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T07:12:56.026 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T07:12:56.031 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T07:12:56.044 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T07:12:56.067 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T07:12:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T07:12:56.086 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T07:12:56.140 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: ntpd: command not found 2024-05-27T07:12:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-05-27T07:12:56.156 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-05-27T07:12:56.174 INFO:teuthology.orchestra.run.smithi063.stdout:506 Cannot talk to daemon 2024-05-27T07:12:56.178 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-05-27T07:12:56.189 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-05-27T07:12:56.220 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T07:12:56.220 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T07:12:56.232 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T07:12:56.264 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T07:12:56.267 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T07:12:56.275 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T07:12:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-05-27T07:12:56.415 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T07:12:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-05-27T07:12:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.416 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.419 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-05-27T07:12:56.420 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-05-27T07:12:56.428 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T07:12:56.428 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-05-27T07:12:56.428 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.428 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.428 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.429 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.430 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T07:12:56.430 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-05-27T07:12:56.430 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.430 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.430 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.430 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:12:56.432 INFO:teuthology.run_tasks:Running task install... 2024-05-27T07:12:56.440 DEBUG:teuthology.task.install:project ceph 2024-05-27T07:12:56.440 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6'}} 2024-05-27T07:12:56.440 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6'} 2024-05-27T07:12:56.440 INFO:teuthology.task.install:Using flavor: default 2024-05-27T07:12:56.446 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-27T07:12:56.446 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-27T07:12:56.446 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'tag': None, 'wait_for_package': False} 2024-05-27T07:12:56.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:12:56.449 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'tag': None, 'wait_for_package': False} 2024-05-27T07:12:56.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:12:56.450 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'tag': None, 'wait_for_package': False} 2024-05-27T07:12:56.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:12:56.602 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/ 2024-05-27T07:12:56.602 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3872.g38f9d900 2024-05-27T07:12:56.689 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/ 2024-05-27T07:12:56.689 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3872.g38f9d900 2024-05-27T07:12:56.695 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/ 2024-05-27T07:12:56.695 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3872.g38f9d900 2024-05-27T07:12:56.715 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T07:12:56.715 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:12:56.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T07:12:56.802 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T07:12:56.802 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:12:56.802 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T07:12:56.806 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T07:12:56.806 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T07:12:56.867 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T07:12:56.867 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:12:56.867 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T07:12:56.892 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T07:12:56.893 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T07:12:56.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T07:12:56.955 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T07:12:56.955 DEBUG:teuthology.orchestra.run.smithi063:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T07:12:57.020 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T07:12:57.045 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T07:12:57.091 DEBUG:teuthology.orchestra.run.smithi063:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T07:12:57.120 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-05-27T07:12:57.124 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-05-27T07:12:57.129 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T07:12:57.209 DEBUG:teuthology.orchestra.run.smithi063:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T07:12:57.210 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-05-27T07:12:57.215 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-05-27T07:12:57.296 INFO:teuthology.orchestra.run.smithi063.stdout:check_obsoletes = 1 2024-05-27T07:12:57.300 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum clean all 2024-05-27T07:12:57.434 INFO:teuthology.orchestra.run.smithi086.stdout:45 files removed 2024-05-27T07:12:57.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T07:12:57.543 INFO:teuthology.orchestra.run.smithi149.stdout:45 files removed 2024-05-27T07:12:57.582 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T07:12:57.621 INFO:teuthology.orchestra.run.smithi063.stdout:45 files removed 2024-05-27T07:12:57.660 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T07:12:58.310 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 209 kB/s | 84 kB 00:00 2024-05-27T07:12:58.524 INFO:teuthology.orchestra.run.smithi063.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-05-27T07:12:58.564 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 151 kB/s | 84 kB 00:00 2024-05-27T07:12:58.910 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-05-27T07:12:59.348 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 17 kB/s | 12 kB 00:00 2024-05-27T07:12:59.366 INFO:teuthology.orchestra.run.smithi063.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-05-27T07:12:59.769 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-05-27T07:13:00.174 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-05-27T07:13:00.258 INFO:teuthology.orchestra.run.smithi063.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-05-27T07:13:00.805 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-27T07:13:00.823 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-27T07:13:01.267 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.1 MB 00:01 2024-05-27T07:13:02.757 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-27T07:13:02.798 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-27T07:13:03.493 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-05-27T07:13:07.491 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.2 MB 00:00 2024-05-27T07:13:09.050 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 17 kB 00:00 2024-05-27T07:13:09.594 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-05-27T07:13:09.615 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-05-27T07:13:11.007 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-27T07:13:11.128 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 17 kB 00:00 2024-05-27T07:13:11.307 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 17 kB 00:00 2024-05-27T07:13:12.547 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-27T07:13:17.427 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-27T07:13:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-27T07:13:20.083 INFO:teuthology.orchestra.run.smithi063.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:13:20.085 INFO:teuthology.orchestra.run.smithi063.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:13:20.108 INFO:teuthology.orchestra.run.smithi063.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T07:13:20.109 INFO:teuthology.orchestra.run.smithi063.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T07:13:20.110 INFO:teuthology.orchestra.run.smithi063.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T07:13:20.112 INFO:teuthology.orchestra.run.smithi063.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T07:13:20.115 INFO:teuthology.orchestra.run.smithi063.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T07:13:20.118 INFO:teuthology.orchestra.run.smithi063.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T07:13:20.120 INFO:teuthology.orchestra.run.smithi063.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T07:13:20.200 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-27T07:13:20.215 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-27T07:13:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-27T07:13:20.215 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-27T07:13:20.215 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-27T07:13:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T07:13:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: ceph x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 6.6 k 2024-05-27T07:13:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.4 M 2024-05-27T07:13:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 863 k 2024-05-27T07:13:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 147 k 2024-05-27T07:13:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 1.5 M 2024-05-27T07:13:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 143 k 2024-05-27T07:13:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 3.5 M 2024-05-27T07:13:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 7.4 M 2024-05-27T07:13:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 49 k 2024-05-27T07:13:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 10 M 2024-05-27T07:13:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 43 M 2024-05-27T07:13:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 273 k 2024-05-27T07:13:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 764 k 2024-05-27T07:13:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 224 k 2024-05-27T07:13:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-shell x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 27 k 2024-05-27T07:13:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-top noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 18 k 2024-05-27T07:13:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T07:13:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T07:13:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T07:13:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 32 k 2024-05-27T07:13:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 755 k 2024-05-27T07:13:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 127 k 2024-05-27T07:13:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T07:13:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T07:13:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 161 k 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 320 k 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 300 k 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 99 k 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 86 k 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.0 M 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 169 k 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrading: 2024-05-27T07:13:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: librados2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.3 M 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.1 M 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-27T07:13:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 20 M 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 34 k 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 2.3 M 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 274 k 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 4.9 M 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 18 M 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 16 k 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 25 k 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T07:13:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T07:13:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T07:13:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T07:13:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T07:13:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T07:13:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T07:13:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T07:13:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 157 k 2024-05-27T07:13:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T07:13:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T07:13:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T07:13:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T07:13:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T07:13:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 498 k 2024-05-27T07:13:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T07:13:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.2 M 2024-05-27T07:13:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T07:13:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 45 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 136 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T07:13:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T07:13:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T07:13:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T07:13:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T07:13:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T07:13:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T07:13:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T07:13:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T07:13:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T07:13:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T07:13:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T07:13:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T07:13:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T07:13:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T07:13:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T07:13:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T07:13:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T07:13:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T07:13:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T07:13:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T07:13:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T07:13:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T07:13:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T07:13:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T07:13:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T07:13:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T07:13:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout:Installing weak dependencies: 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T07:13:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:13:20.232 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-27T07:13:20.232 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-27T07:13:20.232 INFO:teuthology.orchestra.run.smithi063.stdout:Install 148 Packages 2024-05-27T07:13:20.232 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrade 11 Packages 2024-05-27T07:13:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:13:20.233 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 219 M 2024-05-27T07:13:20.233 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-27T07:13:21.098 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:13:21.099 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:13:21.118 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T07:13:21.119 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T07:13:21.120 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T07:13:21.121 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T07:13:21.124 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T07:13:21.127 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T07:13:21.128 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T07:13:21.198 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 6.6 k 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.4 M 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 863 k 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 147 k 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 1.5 M 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 143 k 2024-05-27T07:13:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 3.5 M 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 7.4 M 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 49 k 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 10 M 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 43 M 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 273 k 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 764 k 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 224 k 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 27 k 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 18 k 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T07:13:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 32 k 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 755 k 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 127 k 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 161 k 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 320 k 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 300 k 2024-05-27T07:13:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 99 k 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 86 k 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.0 M 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 169 k 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.3 M 2024-05-27T07:13:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.1 M 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 20 M 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 34 k 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 2.3 M 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 274 k 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 4.9 M 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 18 M 2024-05-27T07:13:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 16 k 2024-05-27T07:13:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 25 k 2024-05-27T07:13:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T07:13:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T07:13:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T07:13:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T07:13:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T07:13:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T07:13:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T07:13:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T07:13:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 157 k 2024-05-27T07:13:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T07:13:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T07:13:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T07:13:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T07:13:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T07:13:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 498 k 2024-05-27T07:13:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T07:13:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.2 M 2024-05-27T07:13:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T07:13:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 45 k 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 136 k 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T07:13:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T07:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T07:13:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T07:13:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T07:13:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout:Install 148 Packages 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 11 Packages 2024-05-27T07:13:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:13:21.243 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 219 M 2024-05-27T07:13:21.243 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-05-27T07:13:21.662 INFO:teuthology.orchestra.run.smithi063.stdout:(1/159): ceph-19.0.0-3872.g38f9d900.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-05-27T07:13:22.440 INFO:teuthology.orchestra.run.smithi086.stdout:(1/159): ceph-19.0.0-3872.g38f9d900.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-05-27T07:13:22.745 INFO:teuthology.orchestra.run.smithi063.stdout:(2/159): ceph-fuse-19.0.0-3872.g38f9d900.el9.x8 798 kB/s | 863 kB 00:01 2024-05-27T07:13:22.970 INFO:teuthology.orchestra.run.smithi063.stdout:(3/159): ceph-immutable-object-cache-19.0.0-387 657 kB/s | 147 kB 00:00 2024-05-27T07:13:23.590 INFO:teuthology.orchestra.run.smithi086.stdout:(2/159): ceph-fuse-19.0.0-3872.g38f9d900.el9.x8 752 kB/s | 863 kB 00:01 2024-05-27T07:13:23.823 INFO:teuthology.orchestra.run.smithi086.stdout:(3/159): ceph-immutable-object-cache-19.0.0-387 632 kB/s | 147 kB 00:00 2024-05-27T07:13:24.379 INFO:teuthology.orchestra.run.smithi063.stdout:(4/159): ceph-mds-19.0.0-3872.g38f9d900.el9.x86 1.6 MB/s | 2.3 MB 00:01 2024-05-27T07:13:25.021 INFO:teuthology.orchestra.run.smithi063.stdout:(5/159): ceph-mgr-19.0.0-3872.g38f9d900.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-05-27T07:13:25.182 INFO:teuthology.orchestra.run.smithi086.stdout:(4/159): ceph-mds-19.0.0-3872.g38f9d900.el9.x86 1.7 MB/s | 2.3 MB 00:01 2024-05-27T07:13:26.115 INFO:teuthology.orchestra.run.smithi086.stdout:(5/159): ceph-mgr-19.0.0-3872.g38f9d900.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-05-27T07:13:26.738 INFO:teuthology.orchestra.run.smithi063.stdout:(6/159): ceph-base-19.0.0-3872.g38f9d900.el9.x8 975 kB/s | 5.4 MB 00:05 2024-05-27T07:13:26.963 INFO:teuthology.orchestra.run.smithi063.stdout:(7/159): ceph-mon-19.0.0-3872.g38f9d900.el9.x86 2.5 MB/s | 4.9 MB 00:01 2024-05-27T07:13:28.838 INFO:teuthology.orchestra.run.smithi063.stdout:(8/159): ceph-common-19.0.0-3872.g38f9d900.el9. 2.5 MB/s | 20 MB 00:07 2024-05-27T07:13:28.897 INFO:teuthology.orchestra.run.smithi063.stdout:(9/159): ceph-selinux-19.0.0-3872.g38f9d900.el9 440 kB/s | 25 kB 00:00 2024-05-27T07:13:29.082 INFO:teuthology.orchestra.run.smithi086.stdout:(6/159): ceph-mon-19.0.0-3872.g38f9d900.el9.x86 1.6 MB/s | 4.9 MB 00:02 2024-05-27T07:13:30.291 INFO:teuthology.orchestra.run.smithi086.stdout:(7/159): ceph-base-19.0.0-3872.g38f9d900.el9.x8 656 kB/s | 5.4 MB 00:08 2024-05-27T07:13:31.313 INFO:teuthology.orchestra.run.smithi063.stdout:(10/159): ceph-radosgw-19.0.0-3872.g38f9d900.el 2.3 MB/s | 10 MB 00:04 2024-05-27T07:13:31.530 INFO:teuthology.orchestra.run.smithi063.stdout:(11/159): cephfs-mirror-19.0.0-3872.g38f9d900.e 1.0 MB/s | 224 kB 00:00 2024-05-27T07:13:31.739 INFO:teuthology.orchestra.run.smithi063.stdout:(12/159): cephfs-shell-19.0.0-3872.g38f9d900.el 130 kB/s | 27 kB 00:00 2024-05-27T07:13:31.847 INFO:teuthology.orchestra.run.smithi063.stdout:(13/159): libcephfs-devel-19.0.0-3872.g38f9d900 295 kB/s | 32 kB 00:00 2024-05-27T07:13:32.281 INFO:teuthology.orchestra.run.smithi063.stdout:(14/159): libcephfs2-19.0.0-3872.g38f9d900.el9. 1.7 MB/s | 755 kB 00:00 2024-05-27T07:13:32.398 INFO:teuthology.orchestra.run.smithi063.stdout:(15/159): libcephsqlite-19.0.0-3872.g38f9d900.e 1.3 MB/s | 157 kB 00:00 2024-05-27T07:13:32.498 INFO:teuthology.orchestra.run.smithi063.stdout:(16/159): librados-devel-19.0.0-3872.g38f9d900. 1.3 MB/s | 127 kB 00:00 2024-05-27T07:13:33.010 INFO:teuthology.orchestra.run.smithi063.stdout:(17/159): libradosstriper1-19.0.0-3872.g38f9d90 974 kB/s | 498 kB 00:00 2024-05-27T07:13:33.324 INFO:teuthology.orchestra.run.smithi086.stdout:(8/159): ceph-common-19.0.0-3872.g38f9d900.el9. 1.7 MB/s | 20 MB 00:11 2024-05-27T07:13:33.357 INFO:teuthology.orchestra.run.smithi063.stdout:(18/159): ceph-osd-19.0.0-3872.g38f9d900.el9.x8 2.7 MB/s | 18 MB 00:06 2024-05-27T07:13:33.399 INFO:teuthology.orchestra.run.smithi086.stdout:(9/159): ceph-selinux-19.0.0-3872.g38f9d900.el9 340 kB/s | 25 kB 00:00 2024-05-27T07:13:33.432 INFO:teuthology.orchestra.run.smithi063.stdout:(19/159): python3-ceph-argparse-19.0.0-3872.g38 610 kB/s | 45 kB 00:00 2024-05-27T07:13:33.641 INFO:teuthology.orchestra.run.smithi063.stdout:(20/159): python3-ceph-common-19.0.0-3872.g38f9 653 kB/s | 136 kB 00:00 2024-05-27T07:13:33.741 INFO:teuthology.orchestra.run.smithi063.stdout:(21/159): python3-cephfs-19.0.0-3872.g38f9d900. 1.6 MB/s | 161 kB 00:00 2024-05-27T07:13:34.350 INFO:teuthology.orchestra.run.smithi063.stdout:(22/159): python3-rados-19.0.0-3872.g38f9d900.e 527 kB/s | 320 kB 00:00 2024-05-27T07:13:34.533 INFO:teuthology.orchestra.run.smithi063.stdout:(23/159): python3-rbd-19.0.0-3872.g38f9d900.el9 1.6 MB/s | 300 kB 00:00 2024-05-27T07:13:34.634 INFO:teuthology.orchestra.run.smithi063.stdout:(24/159): python3-rgw-19.0.0-3872.g38f9d900.el9 1.0 MB/s | 99 kB 00:00 2024-05-27T07:13:34.734 INFO:teuthology.orchestra.run.smithi063.stdout:(25/159): rbd-fuse-19.0.0-3872.g38f9d900.el9.x8 865 kB/s | 86 kB 00:00 2024-05-27T07:13:34.863 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 929 kB/s | 21 MB 00:23 2024-05-27T07:13:35.751 INFO:teuthology.orchestra.run.smithi063.stdout:(26/159): librgw2-19.0.0-3872.g38f9d900.el9.x86 1.9 MB/s | 5.2 MB 00:02 2024-05-27T07:13:35.835 INFO:teuthology.orchestra.run.smithi063.stdout:(27/159): rbd-mirror-19.0.0-3872.g38f9d900.el9. 2.8 MB/s | 3.0 MB 00:01 2024-05-27T07:13:35.918 INFO:teuthology.orchestra.run.smithi063.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3872.g 416 kB/s | 34 kB 00:00 2024-05-27T07:13:35.943 INFO:teuthology.orchestra.run.smithi063.stdout:(29/159): rbd-nbd-19.0.0-3872.g38f9d900.el9.x86 886 kB/s | 169 kB 00:00 2024-05-27T07:13:36.027 INFO:teuthology.orchestra.run.smithi063.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3872.g38f9d90 1.3 MB/s | 143 kB 00:00 2024-05-27T07:13:37.011 INFO:teuthology.orchestra.run.smithi063.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3872.g38f9d 3.3 MB/s | 3.5 MB 00:01 2024-05-27T07:13:37.178 INFO:teuthology.orchestra.run.smithi063.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3872.g38 1.6 MB/s | 274 kB 00:00 2024-05-27T07:13:37.344 INFO:teuthology.orchestra.run.smithi063.stdout:(33/159): ceph-mgr-rook-19.0.0-3872.g38f9d900.e 297 kB/s | 49 kB 00:00 2024-05-27T07:13:37.436 INFO:teuthology.orchestra.run.smithi063.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3872.g3 181 kB/s | 16 kB 00:00 2024-05-27T07:13:37.612 INFO:teuthology.orchestra.run.smithi063.stdout:(35/159): ceph-volume-19.0.0-3872.g38f9d900.el9 1.5 MB/s | 273 kB 00:00 2024-05-27T07:13:37.862 INFO:teuthology.orchestra.run.smithi063.stdout:(36/159): cephadm-19.0.0-3872.g38f9d900.el9.noa 3.0 MB/s | 764 kB 00:00 2024-05-27T07:13:37.954 INFO:teuthology.orchestra.run.smithi063.stdout:(37/159): cephfs-top-19.0.0-3872.g38f9d900.el9. 200 kB/s | 18 kB 00:00 2024-05-27T07:13:38.138 INFO:teuthology.orchestra.run.smithi063.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 440 kB/s | 80 kB 00:00 2024-05-27T07:13:38.157 INFO:teuthology.orchestra.run.smithi086.stdout:(10/159): ceph-radosgw-19.0.0-3872.g38f9d900.el 1.3 MB/s | 10 MB 00:07 2024-05-27T07:13:38.263 INFO:teuthology.orchestra.run.smithi063.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-05-27T07:13:38.313 INFO:teuthology.orchestra.run.smithi063.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-05-27T07:13:38.333 INFO:teuthology.orchestra.run.smithi086.stdout:(11/159): cephfs-mirror-19.0.0-3872.g38f9d900.e 1.3 MB/s | 224 kB 00:00 2024-05-27T07:13:38.372 INFO:teuthology.orchestra.run.smithi063.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-05-27T07:13:38.433 INFO:teuthology.orchestra.run.smithi086.stdout:(12/159): cephfs-shell-19.0.0-3872.g38f9d900.el 273 kB/s | 27 kB 00:00 2024-05-27T07:13:38.455 INFO:teuthology.orchestra.run.smithi063.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-05-27T07:13:38.497 INFO:teuthology.orchestra.run.smithi063.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.6 MB/s | 106 kB 00:00 2024-05-27T07:13:38.525 INFO:teuthology.orchestra.run.smithi086.stdout:(13/159): libcephfs-devel-19.0.0-3872.g38f9d900 349 kB/s | 32 kB 00:00 2024-05-27T07:13:38.539 INFO:teuthology.orchestra.run.smithi063.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-05-27T07:13:38.581 INFO:teuthology.orchestra.run.smithi063.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 3.0 MB/s | 125 kB 00:00 2024-05-27T07:13:38.631 INFO:teuthology.orchestra.run.smithi063.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.3 MB/s | 215 kB 00:00 2024-05-27T07:13:38.817 INFO:teuthology.orchestra.run.smithi086.stdout:(14/159): ceph-osd-19.0.0-3872.g38f9d900.el9.x8 1.8 MB/s | 18 MB 00:09 2024-05-27T07:13:38.857 INFO:teuthology.orchestra.run.smithi063.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-05-27T07:13:39.007 INFO:teuthology.orchestra.run.smithi063.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 6.2 MB/s | 943 kB 00:00 2024-05-27T07:13:39.009 INFO:teuthology.orchestra.run.smithi086.stdout:(15/159): libcephfs2-19.0.0-3872.g38f9d900.el9. 1.5 MB/s | 755 kB 00:00 2024-05-27T07:13:39.042 INFO:teuthology.orchestra.run.smithi086.stdout:(16/159): libcephsqlite-19.0.0-3872.g38f9d900.e 703 kB/s | 157 kB 00:00 2024-05-27T07:13:39.083 INFO:teuthology.orchestra.run.smithi063.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-05-27T07:13:39.109 INFO:teuthology.orchestra.run.smithi086.stdout:(17/159): librados-devel-19.0.0-3872.g38f9d900. 1.2 MB/s | 127 kB 00:00 2024-05-27T07:13:39.133 INFO:teuthology.orchestra.run.smithi063.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 442 kB/s | 22 kB 00:00 2024-05-27T07:13:39.208 INFO:teuthology.orchestra.run.smithi063.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-05-27T07:13:39.259 INFO:teuthology.orchestra.run.smithi063.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 661 kB/s | 33 kB 00:00 2024-05-27T07:13:39.285 INFO:teuthology.orchestra.run.smithi086.stdout:(18/159): libradosstriper1-19.0.0-3872.g38f9d90 2.0 MB/s | 498 kB 00:00 2024-05-27T07:13:39.385 INFO:teuthology.orchestra.run.smithi086.stdout:(19/159): python3-ceph-argparse-19.0.0-3872.g38 455 kB/s | 45 kB 00:00 2024-05-27T07:13:39.434 INFO:teuthology.orchestra.run.smithi063.stdout:(53/159): ceph-mgr-diskprediction-local-19.0.0- 2.2 MB/s | 7.4 MB 00:03 2024-05-27T07:13:39.485 INFO:teuthology.orchestra.run.smithi086.stdout:(20/159): python3-ceph-common-19.0.0-3872.g38f9 1.3 MB/s | 136 kB 00:00 2024-05-27T07:13:39.526 INFO:teuthology.orchestra.run.smithi063.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 201 kB/s | 18 kB 00:00 2024-05-27T07:13:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 226 kB/s | 11 kB 00:00 2024-05-27T07:13:39.602 INFO:teuthology.orchestra.run.smithi086.stdout:(21/159): python3-cephfs-19.0.0-3872.g38f9d900. 1.4 MB/s | 161 kB 00:00 2024-05-27T07:13:39.685 INFO:teuthology.orchestra.run.smithi063.stdout:(56/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.0 MB/s | 3.0 MB 00:00 2024-05-27T07:13:39.735 INFO:teuthology.orchestra.run.smithi063.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-05-27T07:13:39.760 INFO:teuthology.orchestra.run.smithi063.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 610 kB/s | 45 kB 00:00 2024-05-27T07:13:39.786 INFO:teuthology.orchestra.run.smithi086.stdout:(22/159): python3-rados-19.0.0-3872.g38f9d900.e 1.7 MB/s | 320 kB 00:00 2024-05-27T07:13:39.836 INFO:teuthology.orchestra.run.smithi063.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-05-27T07:13:39.869 INFO:teuthology.orchestra.run.smithi063.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-05-27T07:13:39.894 INFO:teuthology.orchestra.run.smithi063.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 442 kB/s | 25 kB 00:00 2024-05-27T07:13:39.920 INFO:teuthology.orchestra.run.smithi063.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 959 kB/s | 47 kB 00:00 2024-05-27T07:13:39.970 INFO:teuthology.orchestra.run.smithi063.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-27T07:13:39.978 INFO:teuthology.orchestra.run.smithi086.stdout:(23/159): python3-rbd-19.0.0-3872.g38f9d900.el9 1.5 MB/s | 300 kB 00:00 2024-05-27T07:13:39.995 INFO:teuthology.orchestra.run.smithi063.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-05-27T07:13:40.021 INFO:teuthology.orchestra.run.smithi063.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 712 kB/s | 35 kB 00:00 2024-05-27T07:13:40.078 INFO:teuthology.orchestra.run.smithi086.stdout:(24/159): python3-rgw-19.0.0-3872.g38f9d900.el9 1.0 MB/s | 99 kB 00:00 2024-05-27T07:13:40.170 INFO:teuthology.orchestra.run.smithi086.stdout:(25/159): rbd-fuse-19.0.0-3872.g38f9d900.el9.x8 947 kB/s | 86 kB 00:00 2024-05-27T07:13:40.212 INFO:teuthology.orchestra.run.smithi063.stdout:(66/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-05-27T07:13:40.305 INFO:teuthology.orchestra.run.smithi063.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-27T07:13:40.430 INFO:teuthology.orchestra.run.smithi063.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-05-27T07:13:40.605 INFO:teuthology.orchestra.run.smithi063.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 911 kB 00:00 2024-05-27T07:13:40.639 INFO:teuthology.orchestra.run.smithi063.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 642 kB/s | 133 kB 00:00 2024-05-27T07:13:40.822 INFO:teuthology.orchestra.run.smithi063.stdout:(71/159): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-05-27T07:13:40.881 INFO:teuthology.orchestra.run.smithi063.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-05-27T07:13:40.906 INFO:teuthology.orchestra.run.smithi063.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-05-27T07:13:40.957 INFO:teuthology.orchestra.run.smithi063.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-05-27T07:13:41.015 INFO:teuthology.orchestra.run.smithi063.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.2 MB/s | 244 kB 00:00 2024-05-27T07:13:41.031 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-05-27T07:13:41.082 INFO:teuthology.orchestra.run.smithi063.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-05-27T07:13:41.132 INFO:teuthology.orchestra.run.smithi063.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 521 kB/s | 26 kB 00:00 2024-05-27T07:13:41.174 INFO:teuthology.orchestra.run.smithi063.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 470 kB/s | 19 kB 00:00 2024-05-27T07:13:41.408 INFO:teuthology.orchestra.run.smithi063.stdout:(79/159): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-05-27T07:13:41.441 INFO:teuthology.orchestra.run.smithi063.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 647 kB/s | 172 kB 00:00 2024-05-27T07:13:41.492 INFO:teuthology.orchestra.run.smithi063.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 419 kB/s | 35 kB 00:00 2024-05-27T07:13:41.551 INFO:teuthology.orchestra.run.smithi063.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-05-27T07:13:41.609 INFO:teuthology.orchestra.run.smithi063.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-05-27T07:13:41.659 INFO:teuthology.orchestra.run.smithi063.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-27T07:13:41.777 INFO:teuthology.orchestra.run.smithi063.stdout:(85/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-05-27T07:13:41.778 INFO:teuthology.orchestra.run.smithi086.stdout:(26/159): rbd-mirror-19.0.0-3872.g38f9d900.el9. 1.9 MB/s | 3.0 MB 00:01 2024-05-27T07:13:41.802 INFO:teuthology.orchestra.run.smithi063.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 299 kB/s | 42 kB 00:00 2024-05-27T07:13:41.835 INFO:teuthology.orchestra.run.smithi063.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-05-27T07:13:41.869 INFO:teuthology.orchestra.run.smithi063.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-27T07:13:41.879 INFO:teuthology.orchestra.run.smithi086.stdout:(27/159): librgw2-19.0.0-3872.g38f9d900.el9.x86 1.9 MB/s | 5.2 MB 00:02 2024-05-27T07:13:41.902 INFO:teuthology.orchestra.run.smithi063.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-27T07:13:41.913 INFO:teuthology.orchestra.run.smithi086.stdout:(28/159): rbd-nbd-19.0.0-3872.g38f9d900.el9.x86 1.2 MB/s | 169 kB 00:00 2024-05-27T07:13:41.928 INFO:teuthology.orchestra.run.smithi063.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 885 kB/s | 51 kB 00:00 2024-05-27T07:13:41.953 INFO:teuthology.orchestra.run.smithi063.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-27T07:13:41.971 INFO:teuthology.orchestra.run.smithi086.stdout:(29/159): ceph-grafana-dashboards-19.0.0-3872.g 377 kB/s | 34 kB 00:00 2024-05-27T07:13:42.003 INFO:teuthology.orchestra.run.smithi063.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 850 kB/s | 42 kB 00:00 2024-05-27T07:13:42.021 INFO:teuthology.orchestra.run.smithi086.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3872.g38f9d90 1.3 MB/s | 143 kB 00:00 2024-05-27T07:13:42.054 INFO:teuthology.orchestra.run.smithi063.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 900 kB/s | 44 kB 00:00 2024-05-27T07:13:42.112 INFO:teuthology.orchestra.run.smithi063.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-27T07:13:42.163 INFO:teuthology.orchestra.run.smithi063.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 732 kB/s | 36 kB 00:00 2024-05-27T07:13:42.213 INFO:teuthology.orchestra.run.smithi063.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 966 kB/s | 47 kB 00:00 2024-05-27T07:13:42.272 INFO:teuthology.orchestra.run.smithi063.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-05-27T07:13:42.322 INFO:teuthology.orchestra.run.smithi063.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 1.2 MB/s | 58 kB 00:00 2024-05-27T07:13:42.372 INFO:teuthology.orchestra.run.smithi063.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-05-27T07:13:42.422 INFO:teuthology.orchestra.run.smithi063.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-05-27T07:13:42.472 INFO:teuthology.orchestra.run.smithi063.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-05-27T07:13:42.531 INFO:teuthology.orchestra.run.smithi063.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 387 kB/s | 22 kB 00:00 2024-05-27T07:13:42.748 INFO:teuthology.orchestra.run.smithi063.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-05-27T07:13:42.807 INFO:teuthology.orchestra.run.smithi063.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-05-27T07:13:43.374 INFO:teuthology.orchestra.run.smithi063.stdout:(105/159): ceph-test-19.0.0-3872.g38f9d900.el9. 3.0 MB/s | 43 MB 00:14 2024-05-27T07:13:43.458 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:13:43.459 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:13:43.480 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T07:13:43.481 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T07:13:43.483 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T07:13:43.484 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T07:13:43.486 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T07:13:43.490 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T07:13:43.491 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T07:13:43.563 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-27T07:13:43.588 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-27T07:13:43.588 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-27T07:13:43.588 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-27T07:13:43.588 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-27T07:13:43.588 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 6.6 k 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.4 M 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 863 k 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 147 k 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 1.5 M 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 143 k 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 3.5 M 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 7.4 M 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 49 k 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 10 M 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 43 M 2024-05-27T07:13:43.589 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 273 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 764 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 224 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 27 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 18 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 32 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 755 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 127 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T07:13:43.590 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 161 k 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 320 k 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 300 k 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 99 k 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 86 k 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.0 M 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 169 k 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T07:13:43.591 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.3 M 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.1 M 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T07:13:43.592 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T07:13:43.593 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-27T07:13:43.593 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T07:13:43.593 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T07:13:43.593 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T07:13:43.593 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 20 M 2024-05-27T07:13:43.593 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 34 k 2024-05-27T07:13:43.593 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 2.3 M 2024-05-27T07:13:43.593 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 274 k 2024-05-27T07:13:43.593 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 4.9 M 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 18 M 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 16 k 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 25 k 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T07:13:43.594 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 157 k 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 498 k 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.2 M 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T07:13:43.595 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T07:13:43.596 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T07:13:43.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T07:13:43.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T07:13:43.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T07:13:43.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 45 k 2024-05-27T07:13:43.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 136 k 2024-05-27T07:13:43.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T07:13:43.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T07:13:43.597 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T07:13:43.598 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T07:13:43.599 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T07:13:43.600 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T07:13:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T07:13:43.602 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T07:13:43.603 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T07:13:43.603 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-05-27T07:13:43.603 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T07:13:43.603 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:13:43.603 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-27T07:13:43.603 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-27T07:13:43.603 INFO:teuthology.orchestra.run.smithi149.stdout:Install 148 Packages 2024-05-27T07:13:43.603 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 11 Packages 2024-05-27T07:13:43.603 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:13:43.604 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 219 M 2024-05-27T07:13:43.604 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-27T07:13:44.005 INFO:teuthology.orchestra.run.smithi086.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3872.g38f9d 1.7 MB/s | 3.5 MB 00:02 2024-05-27T07:13:44.007 INFO:teuthology.orchestra.run.smithi063.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 39 kB/s | 19 kB 00:00 2024-05-27T07:13:44.141 INFO:teuthology.orchestra.run.smithi063.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 394 kB/s | 548 kB 00:01 2024-05-27T07:13:44.175 INFO:teuthology.orchestra.run.smithi063.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 226 kB/s | 309 kB 00:01 2024-05-27T07:13:44.206 INFO:teuthology.orchestra.run.smithi086.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3872.g38 1.3 MB/s | 274 kB 00:00 2024-05-27T07:13:44.314 INFO:teuthology.orchestra.run.smithi086.stdout:(33/159): ceph-mgr-rook-19.0.0-3872.g38f9d900.e 459 kB/s | 49 kB 00:00 2024-05-27T07:13:44.406 INFO:teuthology.orchestra.run.smithi086.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3872.g3 181 kB/s | 16 kB 00:00 2024-05-27T07:13:44.435 INFO:teuthology.orchestra.run.smithi063.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 89 kB/s | 26 kB 00:00 2024-05-27T07:13:44.492 INFO:teuthology.orchestra.run.smithi063.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 153 kB/s | 49 kB 00:00 2024-05-27T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stdout:(35/159): ceph-volume-19.0.0-3872.g38f9d900.el9 1.5 MB/s | 273 kB 00:00 2024-05-27T07:13:44.617 INFO:teuthology.orchestra.run.smithi063.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 372 kB/s | 67 kB 00:00 2024-05-27T07:13:44.726 INFO:teuthology.orchestra.run.smithi063.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 650 kB/s | 151 kB 00:00 2024-05-27T07:13:44.842 INFO:teuthology.orchestra.run.smithi149.stdout:(1/159): ceph-19.0.0-3872.g38f9d900.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-05-27T07:13:44.934 INFO:teuthology.orchestra.run.smithi063.stdout:(113/159): libarrow-9.0.0-11.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2024-05-27T07:13:44.977 INFO:teuthology.orchestra.run.smithi063.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.3 MB/s | 839 kB 00:00 2024-05-27T07:13:45.018 INFO:teuthology.orchestra.run.smithi063.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-05-27T07:13:45.060 INFO:teuthology.orchestra.run.smithi063.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 349 kB/s | 43 kB 00:00 2024-05-27T07:13:45.073 INFO:teuthology.orchestra.run.smithi086.stdout:(36/159): cephadm-19.0.0-3872.g38f9d900.el9.noa 1.5 MB/s | 764 kB 00:00 2024-05-27T07:13:45.212 INFO:teuthology.orchestra.run.smithi086.stdout:(37/159): cephfs-top-19.0.0-3872.g38f9d900.el9. 133 kB/s | 18 kB 00:00 2024-05-27T07:13:45.244 INFO:teuthology.orchestra.run.smithi063.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 940 kB/s | 172 kB 00:00 2024-05-27T07:13:45.291 INFO:teuthology.orchestra.run.smithi086.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 80 kB 00:00 2024-05-27T07:13:45.383 INFO:teuthology.orchestra.run.smithi086.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 8.7 MB/s | 803 kB 00:00 2024-05-27T07:13:45.419 INFO:teuthology.orchestra.run.smithi063.stdout:(118/159): python3-cherrypy-18.6.1-2.el9.noarch 2.0 MB/s | 358 kB 00:00 2024-05-27T07:13:45.508 INFO:teuthology.orchestra.run.smithi086.stdout:(40/159): ceph-mgr-diskprediction-local-19.0.0- 2.1 MB/s | 7.4 MB 00:03 2024-05-27T07:13:45.542 INFO:teuthology.orchestra.run.smithi086.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-05-27T07:13:45.552 INFO:teuthology.orchestra.run.smithi063.stdout:(119/159): python3-cachetools-4.2.4-1.el9.noarc 56 kB/s | 32 kB 00:00 2024-05-27T07:13:45.578 INFO:teuthology.orchestra.run.smithi063.stdout:(120/159): python3-certifi-2021.10.8-2.el9.noar 27 kB/s | 15 kB 00:00 2024-05-27T07:13:45.617 INFO:teuthology.orchestra.run.smithi086.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.2 MB 00:00 2024-05-27T07:13:45.653 INFO:teuthology.orchestra.run.smithi063.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.3 MB/s | 303 kB 00:00 2024-05-27T07:13:45.659 INFO:teuthology.orchestra.run.smithi086.stdout:(43/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-05-27T07:13:45.684 INFO:teuthology.orchestra.run.smithi086.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-27T07:13:45.718 INFO:teuthology.orchestra.run.smithi086.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-27T07:13:45.745 INFO:teuthology.orchestra.run.smithi063.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.0 MB/s | 206 kB 00:00 2024-05-27T07:13:45.751 INFO:teuthology.orchestra.run.smithi086.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-27T07:13:45.767 INFO:teuthology.orchestra.run.smithi149.stdout:(2/159): ceph-fuse-19.0.0-3872.g38f9d900.el9.x8 935 kB/s | 863 kB 00:00 2024-05-27T07:13:45.785 INFO:teuthology.orchestra.run.smithi086.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-05-27T07:13:45.820 INFO:teuthology.orchestra.run.smithi063.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 869 kB/s | 144 kB 00:00 2024-05-27T07:13:45.863 INFO:teuthology.orchestra.run.smithi063.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-05-27T07:13:45.929 INFO:teuthology.orchestra.run.smithi063.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-05-27T07:13:45.942 INFO:teuthology.orchestra.run.smithi149.stdout:(3/159): ceph-immutable-object-cache-19.0.0-387 845 kB/s | 147 kB 00:00 2024-05-27T07:13:45.966 INFO:teuthology.orchestra.run.smithi063.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-05-27T07:13:45.970 INFO:teuthology.orchestra.run.smithi086.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-05-27T07:13:46.019 INFO:teuthology.orchestra.run.smithi086.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 4.0 MB/s | 943 kB 00:00 2024-05-27T07:13:46.021 INFO:teuthology.orchestra.run.smithi063.stdout:(127/159): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-05-27T07:13:46.047 INFO:teuthology.orchestra.run.smithi063.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-05-27T07:13:46.053 INFO:teuthology.orchestra.run.smithi086.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-05-27T07:13:46.072 INFO:teuthology.orchestra.run.smithi063.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-05-27T07:13:46.078 INFO:teuthology.orchestra.run.smithi086.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 376 kB/s | 22 kB 00:00 2024-05-27T07:13:46.120 INFO:teuthology.orchestra.run.smithi086.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-05-27T07:13:46.145 INFO:teuthology.orchestra.run.smithi086.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 494 kB/s | 33 kB 00:00 2024-05-27T07:13:46.156 INFO:teuthology.orchestra.run.smithi063.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 429 kB/s | 46 kB 00:00 2024-05-27T07:13:46.189 INFO:teuthology.orchestra.run.smithi063.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 677 kB/s | 79 kB 00:00 2024-05-27T07:13:46.229 INFO:teuthology.orchestra.run.smithi086.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 220 kB/s | 18 kB 00:00 2024-05-27T07:13:46.256 INFO:teuthology.orchestra.run.smithi063.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-05-27T07:13:46.281 INFO:teuthology.orchestra.run.smithi063.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 464 kB/s | 58 kB 00:00 2024-05-27T07:13:46.315 INFO:teuthology.orchestra.run.smithi063.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-05-27T07:13:46.346 INFO:teuthology.orchestra.run.smithi086.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 96 kB/s | 11 kB 00:00 2024-05-27T07:13:46.365 INFO:teuthology.orchestra.run.smithi063.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-05-27T07:13:46.391 INFO:teuthology.orchestra.run.smithi063.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 832 kB/s | 90 kB 00:00 2024-05-27T07:13:46.396 INFO:teuthology.orchestra.run.smithi086.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 160 kB 00:00 2024-05-27T07:13:46.440 INFO:teuthology.orchestra.run.smithi063.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 226 kB/s | 28 kB 00:00 2024-05-27T07:13:46.446 INFO:teuthology.orchestra.run.smithi086.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 920 kB/s | 45 kB 00:00 2024-05-27T07:13:46.491 INFO:teuthology.orchestra.run.smithi063.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 247 kB/s | 31 kB 00:00 2024-05-27T07:13:46.514 INFO:teuthology.orchestra.run.smithi086.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 662 kB 00:00 2024-05-27T07:13:46.524 INFO:teuthology.orchestra.run.smithi063.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-05-27T07:13:46.566 INFO:teuthology.orchestra.run.smithi063.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-05-27T07:13:46.600 INFO:teuthology.orchestra.run.smithi063.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-05-27T07:13:46.634 INFO:teuthology.orchestra.run.smithi063.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-05-27T07:13:46.684 INFO:teuthology.orchestra.run.smithi063.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-27T07:13:46.718 INFO:teuthology.orchestra.run.smithi063.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 769 kB/s | 90 kB 00:00 2024-05-27T07:13:46.759 INFO:teuthology.orchestra.run.smithi063.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-27T07:13:46.793 INFO:teuthology.orchestra.run.smithi063.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-05-27T07:13:46.843 INFO:teuthology.orchestra.run.smithi063.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-27T07:13:46.944 INFO:teuthology.orchestra.run.smithi063.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-05-27T07:13:46.948 INFO:teuthology.orchestra.run.smithi086.stdout:(59/159): ceph-test-19.0.0-3872.g38f9d900.el9.x 3.2 MB/s | 43 MB 00:13 2024-05-27T07:13:46.984 INFO:teuthology.orchestra.run.smithi149.stdout:(4/159): ceph-mds-19.0.0-3872.g38f9d900.el9.x86 2.2 MB/s | 2.3 MB 00:01 2024-05-27T07:13:46.989 INFO:teuthology.orchestra.run.smithi086.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 1.2 MB/s | 582 kB 00:00 2024-05-27T07:13:47.040 INFO:teuthology.orchestra.run.smithi086.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 967 kB/s | 47 kB 00:00 2024-05-27T07:13:47.065 INFO:teuthology.orchestra.run.smithi086.stdout:(62/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 220 kB/s | 25 kB 00:00 2024-05-27T07:13:47.099 INFO:teuthology.orchestra.run.smithi086.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-05-27T07:13:47.161 INFO:teuthology.orchestra.run.smithi063.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-27T07:13:47.174 INFO:teuthology.orchestra.run.smithi086.stdout:(64/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-05-27T07:13:47.199 INFO:teuthology.orchestra.run.smithi086.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 354 kB/s | 35 kB 00:00 2024-05-27T07:13:47.233 INFO:teuthology.orchestra.run.smithi086.stdout:(66/159): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-05-27T07:13:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-05-27T07:13:47.294 INFO:teuthology.orchestra.run.smithi063.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-27T07:13:47.345 INFO:teuthology.orchestra.run.smithi063.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.9 MB/s | 248 kB 00:00 2024-05-27T07:13:47.387 INFO:teuthology.orchestra.run.smithi063.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 904 kB/s | 37 kB 00:00 2024-05-27T07:13:47.416 INFO:teuthology.orchestra.run.smithi086.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-27T07:13:47.437 INFO:teuthology.orchestra.run.smithi063.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.1 MB/s | 207 kB 00:00 2024-05-27T07:13:47.483 INFO:teuthology.orchestra.run.smithi086.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 6.4 MB/s | 1.6 MB 00:00 2024-05-27T07:13:47.487 INFO:teuthology.orchestra.run.smithi063.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.2 MB/s | 260 kB 00:00 2024-05-27T07:13:47.542 INFO:teuthology.orchestra.run.smithi086.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 7.2 MB/s | 911 kB 00:00 2024-05-27T07:13:47.567 INFO:teuthology.orchestra.run.smithi086.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-05-27T07:13:47.579 INFO:teuthology.orchestra.run.smithi063.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-05-27T07:13:47.626 INFO:teuthology.orchestra.run.smithi149.stdout:(5/159): ceph-mgr-19.0.0-3872.g38f9d900.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-05-27T07:13:47.676 INFO:teuthology.orchestra.run.smithi086.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 9.4 MB/s | 1.0 MB 00:00 2024-05-27T07:13:47.688 INFO:teuthology.orchestra.run.smithi063.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-05-27T07:13:47.726 INFO:teuthology.orchestra.run.smithi086.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-27T07:13:47.734 INFO:teuthology.orchestra.run.smithi149.stdout:(6/159): ceph-base-19.0.0-3872.g38f9d900.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-05-27T07:13:47.902 INFO:teuthology.orchestra.run.smithi086.stdout:(73/159): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-05-27T07:13:48.136 INFO:teuthology.orchestra.run.smithi086.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 262 kB/s | 61 kB 00:00 2024-05-27T07:13:48.252 INFO:teuthology.orchestra.run.smithi086.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.1 MB/s | 244 kB 00:00 2024-05-27T07:13:48.302 INFO:teuthology.orchestra.run.smithi086.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-27T07:13:48.344 INFO:teuthology.orchestra.run.smithi086.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 626 kB/s | 26 kB 00:00 2024-05-27T07:13:48.478 INFO:teuthology.orchestra.run.smithi086.stdout:(78/159): python3-babel-2.9.1-2.el9.noarch.rpm 7.9 MB/s | 6.0 MB 00:00 2024-05-27T07:13:48.503 INFO:teuthology.orchestra.run.smithi086.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 121 kB/s | 19 kB 00:00 2024-05-27T07:13:48.530 INFO:teuthology.orchestra.run.smithi063.stdout:(158/159): librados2-19.0.0-3872.g38f9d900.el9. 1.9 MB/s | 3.3 MB 00:01 2024-05-27T07:13:48.537 INFO:teuthology.orchestra.run.smithi086.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-27T07:13:48.579 INFO:teuthology.orchestra.run.smithi086.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 467 kB/s | 35 kB 00:00 2024-05-27T07:13:48.654 INFO:teuthology.orchestra.run.smithi086.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-27T07:13:48.755 INFO:teuthology.orchestra.run.smithi086.stdout:(83/159): openblas-openmp-0.3.21-2.el9.x86_64.r 3.0 MB/s | 4.7 MB 00:01 2024-05-27T07:13:48.788 INFO:teuthology.orchestra.run.smithi086.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-05-27T07:13:48.814 INFO:teuthology.orchestra.run.smithi086.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-27T07:13:48.839 INFO:teuthology.orchestra.run.smithi086.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 851 kB/s | 42 kB 00:00 2024-05-27T07:13:48.873 INFO:teuthology.orchestra.run.smithi086.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-05-27T07:13:48.981 INFO:teuthology.orchestra.run.smithi086.stdout:(88/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-05-27T07:13:49.015 INFO:teuthology.orchestra.run.smithi086.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 908 kB/s | 159 kB 00:00 2024-05-27T07:13:49.042 INFO:teuthology.orchestra.run.smithi086.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 885 kB/s | 51 kB 00:00 2024-05-27T07:13:49.065 INFO:teuthology.orchestra.run.smithi086.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-27T07:13:49.130 INFO:teuthology.orchestra.run.smithi086.stdout:(92/159): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 279 kB 00:00 2024-05-27T07:13:49.158 INFO:teuthology.orchestra.run.smithi086.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 457 kB/s | 42 kB 00:00 2024-05-27T07:13:49.183 INFO:teuthology.orchestra.run.smithi086.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 854 kB/s | 44 kB 00:00 2024-05-27T07:13:49.217 INFO:teuthology.orchestra.run.smithi086.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-27T07:13:49.242 INFO:teuthology.orchestra.run.smithi086.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 622 kB/s | 36 kB 00:00 2024-05-27T07:13:49.267 INFO:teuthology.orchestra.run.smithi086.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-05-27T07:13:49.334 INFO:teuthology.orchestra.run.smithi086.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 876 kB/s | 58 kB 00:00 2024-05-27T07:13:49.384 INFO:teuthology.orchestra.run.smithi086.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-05-27T07:13:49.418 INFO:teuthology.orchestra.run.smithi086.stdout:(100/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 305 kB 00:00 2024-05-27T07:13:49.452 INFO:teuthology.orchestra.run.smithi086.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 965 kB/s | 64 kB 00:00 2024-05-27T07:13:49.459 INFO:teuthology.orchestra.run.smithi149.stdout:(7/159): ceph-mon-19.0.0-3872.g38f9d900.el9.x86 2.7 MB/s | 4.9 MB 00:01 2024-05-27T07:13:49.477 INFO:teuthology.orchestra.run.smithi086.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-05-27T07:13:49.552 INFO:teuthology.orchestra.run.smithi086.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 225 kB/s | 22 kB 00:00 2024-05-27T07:13:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-27T07:13:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-05-27T07:13:50.163 INFO:teuthology.orchestra.run.smithi063.stdout:(159/159): librbd1-19.0.0-3872.g38f9d900.el9.x8 954 kB/s | 3.1 MB 00:03 2024-05-27T07:13:50.177 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-27T07:13:50.178 INFO:teuthology.orchestra.run.smithi063.stdout:Total 7.3 MB/s | 219 MB 00:29 2024-05-27T07:13:50.311 INFO:teuthology.orchestra.run.smithi086.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 46 kB/s | 19 kB 00:00 2024-05-27T07:13:50.335 INFO:teuthology.orchestra.run.smithi149.stdout:(8/159): ceph-osd-19.0.0-3872.g38f9d900.el9.x86 6.8 MB/s | 18 MB 00:02 2024-05-27T07:13:50.520 INFO:teuthology.orchestra.run.smithi086.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 464 kB/s | 309 kB 00:00 2024-05-27T07:13:50.585 INFO:teuthology.orchestra.run.smithi149.stdout:(9/159): ceph-common-19.0.0-3872.g38f9d900.el9. 3.2 MB/s | 20 MB 00:06 2024-05-27T07:13:50.595 INFO:teuthology.orchestra.run.smithi086.stdout:(108/159): abseil-cpp-20211102.0-3.el9.x86_64.r 526 kB/s | 548 kB 00:01 2024-05-27T07:13:50.610 INFO:teuthology.orchestra.run.smithi149.stdout:(10/159): ceph-selinux-19.0.0-3872.g38f9d900.el 92 kB/s | 25 kB 00:00 2024-05-27T07:13:50.721 INFO:teuthology.orchestra.run.smithi086.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-05-27T07:13:50.735 INFO:teuthology.orchestra.run.smithi149.stdout:(11/159): cephfs-mirror-19.0.0-3872.g38f9d900.e 1.8 MB/s | 224 kB 00:00 2024-05-27T07:13:50.754 INFO:teuthology.orchestra.run.smithi086.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 309 kB/s | 49 kB 00:00 2024-05-27T07:13:50.810 INFO:teuthology.orchestra.run.smithi149.stdout:(12/159): cephfs-shell-19.0.0-3872.g38f9d900.el 366 kB/s | 27 kB 00:00 2024-05-27T07:13:50.894 INFO:teuthology.orchestra.run.smithi149.stdout:(13/159): libcephfs-devel-19.0.0-3872.g38f9d900 385 kB/s | 32 kB 00:00 2024-05-27T07:13:50.904 INFO:teuthology.orchestra.run.smithi086.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 4.4 MB 00:00 2024-05-27T07:13:50.929 INFO:teuthology.orchestra.run.smithi086.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 340 kB/s | 67 kB 00:00 2024-05-27T07:13:50.955 INFO:teuthology.orchestra.run.smithi086.stdout:(113/159): luarocks-3.9.2-5.el9.noarch.rpm 777 kB/s | 151 kB 00:00 2024-05-27T07:13:51.047 INFO:teuthology.orchestra.run.smithi086.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.9 MB/s | 839 kB 00:00 2024-05-27T07:13:51.061 INFO:teuthology.orchestra.run.smithi149.stdout:(14/159): libcephfs2-19.0.0-3872.g38f9d900.el9. 4.5 MB/s | 755 kB 00:00 2024-05-27T07:13:51.110 INFO:teuthology.orchestra.run.smithi086.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-05-27T07:13:51.144 INFO:teuthology.orchestra.run.smithi149.stdout:(15/159): libcephsqlite-19.0.0-3872.g38f9d900.e 1.9 MB/s | 157 kB 00:00 2024-05-27T07:13:51.147 INFO:teuthology.orchestra.run.smithi086.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 227 kB/s | 43 kB 00:00 2024-05-27T07:13:51.189 INFO:teuthology.orchestra.run.smithi086.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 237 kB/s | 32 kB 00:00 2024-05-27T07:13:51.237 INFO:teuthology.orchestra.run.smithi149.stdout:(16/159): librados-devel-19.0.0-3872.g38f9d900. 1.4 MB/s | 127 kB 00:00 2024-05-27T07:13:51.256 INFO:teuthology.orchestra.run.smithi086.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 140 kB/s | 15 kB 00:00 2024-05-27T07:13:51.282 INFO:teuthology.orchestra.run.smithi086.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-27T07:13:51.315 INFO:teuthology.orchestra.run.smithi086.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-27T07:13:51.345 INFO:teuthology.orchestra.run.smithi149.stdout:(17/159): libradosstriper1-19.0.0-3872.g38f9d90 4.5 MB/s | 498 kB 00:00 2024-05-27T07:13:51.374 INFO:teuthology.orchestra.run.smithi086.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.6 MB/s | 303 kB 00:00 2024-05-27T07:13:51.408 INFO:teuthology.orchestra.run.smithi086.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-27T07:13:51.508 INFO:teuthology.orchestra.run.smithi086.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-27T07:13:51.533 INFO:teuthology.orchestra.run.smithi086.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 924 kB/s | 144 kB 00:00 2024-05-27T07:13:51.558 INFO:teuthology.orchestra.run.smithi086.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 77 kB/s | 11 kB 00:00 2024-05-27T07:13:51.617 INFO:teuthology.orchestra.run.smithi086.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 166 kB/s | 18 kB 00:00 2024-05-27T07:13:51.667 INFO:teuthology.orchestra.run.smithi086.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-05-27T07:13:51.692 INFO:teuthology.orchestra.run.smithi086.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 146 kB/s | 19 kB 00:00 2024-05-27T07:13:51.726 INFO:teuthology.orchestra.run.smithi086.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-05-27T07:13:51.818 INFO:teuthology.orchestra.run.smithi086.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-05-27T07:13:51.843 INFO:teuthology.orchestra.run.smithi086.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 308 kB/s | 46 kB 00:00 2024-05-27T07:13:51.868 INFO:teuthology.orchestra.run.smithi086.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 557 kB/s | 79 kB 00:00 2024-05-27T07:13:51.927 INFO:teuthology.orchestra.run.smithi086.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 540 kB/s | 58 kB 00:00 2024-05-27T07:13:51.986 INFO:teuthology.orchestra.run.smithi086.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-27T07:13:52.011 INFO:teuthology.orchestra.run.smithi086.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-05-27T07:13:52.036 INFO:teuthology.orchestra.run.smithi086.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 831 kB/s | 90 kB 00:00 2024-05-27T07:13:52.104 INFO:teuthology.orchestra.run.smithi149.stdout:(18/159): librgw2-19.0.0-3872.g38f9d900.el9.x86 6.8 MB/s | 5.2 MB 00:00 2024-05-27T07:13:52.144 INFO:teuthology.orchestra.run.smithi086.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 214 kB/s | 28 kB 00:00 2024-05-27T07:13:52.170 INFO:teuthology.orchestra.run.smithi086.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 196 kB/s | 31 kB 00:00 2024-05-27T07:13:52.203 INFO:teuthology.orchestra.run.smithi086.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-05-27T07:13:52.279 INFO:teuthology.orchestra.run.smithi086.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 558 kB/s | 59 kB 00:00 2024-05-27T07:13:52.287 INFO:teuthology.orchestra.run.smithi149.stdout:(19/159): ceph-radosgw-19.0.0-3872.g38f9d900.el 3.6 MB/s | 10 MB 00:02 2024-05-27T07:13:52.304 INFO:teuthology.orchestra.run.smithi086.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 273 kB/s | 36 kB 00:00 2024-05-27T07:13:52.313 INFO:teuthology.orchestra.run.smithi149.stdout:(20/159): python3-ceph-argparse-19.0.0-3872.g38 217 kB/s | 45 kB 00:00 2024-05-27T07:13:52.329 INFO:teuthology.orchestra.run.smithi086.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 408 kB/s | 51 kB 00:00 2024-05-27T07:13:52.363 INFO:teuthology.orchestra.run.smithi149.stdout:(21/159): python3-ceph-common-19.0.0-3872.g38f9 1.8 MB/s | 136 kB 00:00 2024-05-27T07:13:52.396 INFO:teuthology.orchestra.run.smithi086.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-27T07:13:52.405 INFO:teuthology.orchestra.run.smithi149.stdout:(22/159): python3-cephfs-19.0.0-3872.g38f9d900. 1.7 MB/s | 161 kB 00:00 2024-05-27T07:13:52.421 INFO:teuthology.orchestra.run.smithi086.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 773 kB/s | 90 kB 00:00 2024-05-27T07:13:52.456 INFO:teuthology.orchestra.run.smithi086.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-27T07:13:52.497 INFO:teuthology.orchestra.run.smithi086.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 201 kB/s | 20 kB 00:00 2024-05-27T07:13:52.505 INFO:teuthology.orchestra.run.smithi149.stdout:(23/159): python3-rbd-19.0.0-3872.g38f9d900.el9 2.9 MB/s | 300 kB 00:00 2024-05-27T07:13:52.535 INFO:teuthology.orchestra.run.smithi149.stdout:(24/159): python3-rados-19.0.0-3872.g38f9d900.e 1.9 MB/s | 320 kB 00:00 2024-05-27T07:13:52.540 INFO:teuthology.orchestra.run.smithi086.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-27T07:13:52.589 INFO:teuthology.orchestra.run.smithi149.stdout:(25/159): python3-rgw-19.0.0-3872.g38f9d900.el9 1.2 MB/s | 99 kB 00:00 2024-05-27T07:13:52.614 INFO:teuthology.orchestra.run.smithi149.stdout:(26/159): rbd-fuse-19.0.0-3872.g38f9d900.el9.x8 1.0 MB/s | 86 kB 00:00 2024-05-27T07:13:52.615 INFO:teuthology.orchestra.run.smithi086.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-27T07:13:52.715 INFO:teuthology.orchestra.run.smithi149.stdout:(27/159): rbd-nbd-19.0.0-3872.g38f9d900.el9.x86 1.7 MB/s | 169 kB 00:00 2024-05-27T07:13:52.798 INFO:teuthology.orchestra.run.smithi086.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-27T07:13:52.823 INFO:teuthology.orchestra.run.smithi149.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3872.g 318 kB/s | 34 kB 00:00 2024-05-27T07:13:52.882 INFO:teuthology.orchestra.run.smithi086.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 22 MB/s | 1.8 MB 00:00 2024-05-27T07:13:52.923 INFO:teuthology.orchestra.run.smithi086.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-27T07:13:52.966 INFO:teuthology.orchestra.run.smithi086.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 5.9 MB/s | 248 kB 00:00 2024-05-27T07:13:53.008 INFO:teuthology.orchestra.run.smithi086.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 898 kB/s | 37 kB 00:00 2024-05-27T07:13:53.049 INFO:teuthology.orchestra.run.smithi086.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 5.0 MB/s | 207 kB 00:00 2024-05-27T07:13:53.300 INFO:teuthology.orchestra.run.smithi086.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.0 MB/s | 260 kB 00:00 2024-05-27T07:13:53.407 INFO:teuthology.orchestra.run.smithi149.stdout:(29/159): rbd-mirror-19.0.0-3872.g38f9d900.el9. 3.7 MB/s | 3.0 MB 00:00 2024-05-27T07:13:53.409 INFO:teuthology.orchestra.run.smithi086.stdout:(156/159): librbd1-19.0.0-3872.g38f9d900.el9.x8 3.6 MB/s | 3.1 MB 00:00 2024-05-27T07:13:53.467 INFO:teuthology.orchestra.run.smithi086.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-27T07:13:53.559 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-27T07:13:53.592 INFO:teuthology.orchestra.run.smithi086.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-05-27T07:13:53.607 INFO:teuthology.orchestra.run.smithi149.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3872.g38f9d90 183 kB/s | 143 kB 00:00 2024-05-27T07:13:53.828 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-27T07:13:53.828 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-27T07:13:53.966 INFO:teuthology.orchestra.run.smithi149.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3872.g38f9d 6.3 MB/s | 3.5 MB 00:00 2024-05-27T07:13:54.092 INFO:teuthology.orchestra.run.smithi149.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3872.g38 2.2 MB/s | 274 kB 00:00 2024-05-27T07:13:54.192 INFO:teuthology.orchestra.run.smithi149.stdout:(33/159): ceph-mgr-rook-19.0.0-3872.g38f9d900.e 500 kB/s | 49 kB 00:00 2024-05-27T07:13:54.209 INFO:teuthology.orchestra.run.smithi086.stdout:(159/159): librados2-19.0.0-3872.g38f9d900.el9. 1.9 MB/s | 3.3 MB 00:01 2024-05-27T07:13:54.222 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-05-27T07:13:54.222 INFO:teuthology.orchestra.run.smithi086.stdout:Total 6.6 MB/s | 219 MB 00:32 2024-05-27T07:13:54.300 INFO:teuthology.orchestra.run.smithi149.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3872.g3 153 kB/s | 16 kB 00:00 2024-05-27T07:13:54.409 INFO:teuthology.orchestra.run.smithi149.stdout:(35/159): ceph-volume-19.0.0-3872.g38f9d900.el9 2.5 MB/s | 273 kB 00:00 2024-05-27T07:13:54.593 INFO:teuthology.orchestra.run.smithi149.stdout:(36/159): cephadm-19.0.0-3872.g38f9d900.el9.noa 4.1 MB/s | 764 kB 00:00 2024-05-27T07:13:54.684 INFO:teuthology.orchestra.run.smithi149.stdout:(37/159): cephfs-top-19.0.0-3872.g38f9d900.el9. 201 kB/s | 18 kB 00:00 2024-05-27T07:13:54.801 INFO:teuthology.orchestra.run.smithi149.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 695 kB/s | 80 kB 00:00 2024-05-27T07:13:54.943 INFO:teuthology.orchestra.run.smithi149.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 5.6 MB/s | 803 kB 00:00 2024-05-27T07:13:55.002 INFO:teuthology.orchestra.run.smithi149.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-05-27T07:13:55.060 INFO:teuthology.orchestra.run.smithi149.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-05-27T07:13:55.169 INFO:teuthology.orchestra.run.smithi149.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-05-27T07:13:55.228 INFO:teuthology.orchestra.run.smithi149.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-27T07:13:55.279 INFO:teuthology.orchestra.run.smithi149.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-05-27T07:13:55.337 INFO:teuthology.orchestra.run.smithi149.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-27T07:13:55.396 INFO:teuthology.orchestra.run.smithi149.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 215 kB 00:00 2024-05-27T07:13:55.671 INFO:teuthology.orchestra.run.smithi149.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-05-27T07:13:55.838 INFO:teuthology.orchestra.run.smithi149.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 5.6 MB/s | 943 kB 00:00 2024-05-27T07:13:55.896 INFO:teuthology.orchestra.run.smithi149.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-27T07:13:55.997 INFO:teuthology.orchestra.run.smithi149.stdout:(50/159): ceph-mgr-diskprediction-local-19.0.0- 3.1 MB/s | 7.4 MB 00:02 2024-05-27T07:13:56.022 INFO:teuthology.orchestra.run.smithi149.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 174 kB/s | 22 kB 00:00 2024-05-27T07:13:56.081 INFO:teuthology.orchestra.run.smithi149.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 570 kB/s | 33 kB 00:00 2024-05-27T07:13:56.214 INFO:teuthology.orchestra.run.smithi149.stdout:(53/159): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 313 kB 00:00 2024-05-27T07:13:56.273 INFO:teuthology.orchestra.run.smithi149.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 317 kB/s | 18 kB 00:00 2024-05-27T07:13:56.332 INFO:teuthology.orchestra.run.smithi149.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 194 kB/s | 11 kB 00:00 2024-05-27T07:13:56.407 INFO:teuthology.orchestra.run.smithi149.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-27T07:13:56.466 INFO:teuthology.orchestra.run.smithi149.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 789 kB/s | 45 kB 00:00 2024-05-27T07:13:56.535 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-27T07:13:56.535 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-27T07:13:56.541 INFO:teuthology.orchestra.run.smithi149.stdout:(58/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.5 MB/s | 3.0 MB 00:00 2024-05-27T07:13:56.625 INFO:teuthology.orchestra.run.smithi149.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-05-27T07:13:56.666 INFO:teuthology.orchestra.run.smithi149.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 582 kB 00:00 2024-05-27T07:13:56.692 INFO:teuthology.orchestra.run.smithi149.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 385 kB/s | 25 kB 00:00 2024-05-27T07:13:56.726 INFO:teuthology.orchestra.run.smithi149.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 823 kB/s | 47 kB 00:00 2024-05-27T07:13:56.759 INFO:teuthology.orchestra.run.smithi149.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-05-27T07:13:56.792 INFO:teuthology.orchestra.run.smithi149.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-05-27T07:13:56.818 INFO:teuthology.orchestra.run.smithi149.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-05-27T07:13:57.335 INFO:teuthology.orchestra.run.smithi149.stdout:(66/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-05-27T07:13:57.429 INFO:teuthology.orchestra.run.smithi149.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 7.5 MB/s | 4.7 MB 00:00 2024-05-27T07:13:57.439 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-05-27T07:13:57.668 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-05-27T07:13:57.668 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-05-27T07:13:57.947 INFO:teuthology.orchestra.run.smithi149.stdout:(68/159): ceph-test-19.0.0-3872.g38f9d900.el9.x 5.9 MB/s | 43 MB 00:07 2024-05-27T07:13:58.005 INFO:teuthology.orchestra.run.smithi149.stdout:(69/159): postgresql-13.14-1.el9.x86_64.rpm 2.4 MB/s | 1.6 MB 00:00 2024-05-27T07:13:58.047 INFO:teuthology.orchestra.run.smithi149.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 1.4 MB/s | 911 kB 00:00 2024-05-27T07:13:58.198 INFO:teuthology.orchestra.run.smithi149.stdout:(71/159): postgresql-private-libs-13.14-1.el9.x 535 kB/s | 133 kB 00:00 2024-05-27T07:13:58.256 INFO:teuthology.orchestra.run.smithi149.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-27T07:13:58.448 INFO:teuthology.orchestra.run.smithi149.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 2.5 MB/s | 1.0 MB 00:00 2024-05-27T07:13:58.507 INFO:teuthology.orchestra.run.smithi149.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-05-27T07:13:58.632 INFO:teuthology.orchestra.run.smithi149.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-05-27T07:13:58.775 INFO:teuthology.orchestra.run.smithi149.stdout:(76/159): postgresql-server-13.14-1.el9.x86_64. 8.0 MB/s | 5.8 MB 00:00 2024-05-27T07:13:58.808 INFO:teuthology.orchestra.run.smithi149.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-05-27T07:13:58.841 INFO:teuthology.orchestra.run.smithi149.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 389 kB/s | 26 kB 00:00 2024-05-27T07:13:58.867 INFO:teuthology.orchestra.run.smithi149.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-05-27T07:13:58.908 INFO:teuthology.orchestra.run.smithi149.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-05-27T07:13:58.951 INFO:teuthology.orchestra.run.smithi149.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 419 kB/s | 35 kB 00:00 2024-05-27T07:13:59.076 INFO:teuthology.orchestra.run.smithi149.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 7.3 MB/s | 6.0 MB 00:00 2024-05-27T07:13:59.109 INFO:teuthology.orchestra.run.smithi149.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-05-27T07:13:59.143 INFO:teuthology.orchestra.run.smithi149.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-27T07:13:59.168 INFO:teuthology.orchestra.run.smithi149.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-27T07:13:59.202 INFO:teuthology.orchestra.run.smithi149.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 733 kB/s | 42 kB 00:00 2024-05-27T07:13:59.269 INFO:teuthology.orchestra.run.smithi149.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-05-27T07:13:59.294 INFO:teuthology.orchestra.run.smithi149.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-05-27T07:13:59.345 INFO:teuthology.orchestra.run.smithi149.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-27T07:13:59.403 INFO:teuthology.orchestra.run.smithi149.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-05-27T07:13:59.420 INFO:teuthology.orchestra.run.smithi149.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 721 kB/s | 54 kB 00:00 2024-05-27T07:13:59.529 INFO:teuthology.orchestra.run.smithi149.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.2 MB/s | 5.1 MB 00:00 2024-05-27T07:13:59.554 INFO:teuthology.orchestra.run.smithi149.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 313 kB/s | 42 kB 00:00 2024-05-27T07:13:59.579 INFO:teuthology.orchestra.run.smithi149.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 897 kB/s | 44 kB 00:00 2024-05-27T07:13:59.613 INFO:teuthology.orchestra.run.smithi149.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-27T07:13:59.638 INFO:teuthology.orchestra.run.smithi149.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 619 kB/s | 36 kB 00:00 2024-05-27T07:13:59.681 INFO:teuthology.orchestra.run.smithi149.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 716 kB/s | 47 kB 00:00 2024-05-27T07:13:59.714 INFO:teuthology.orchestra.run.smithi149.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-05-27T07:13:59.739 INFO:teuthology.orchestra.run.smithi149.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-05-27T07:13:59.773 INFO:teuthology.orchestra.run.smithi149.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-05-27T07:13:59.790 INFO:teuthology.orchestra.run.smithi149.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-05-27T07:13:59.824 INFO:teuthology.orchestra.run.smithi149.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-27T07:14:00.327 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-05-27T07:14:00.327 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-05-27T07:14:00.407 INFO:teuthology.orchestra.run.smithi149.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 36 kB/s | 22 kB 00:00 2024-05-27T07:14:00.903 INFO:teuthology.orchestra.run.smithi149.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 800 kB/s | 863 kB 00:01 2024-05-27T07:14:01.016 INFO:teuthology.orchestra.run.smithi149.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 903 kB/s | 548 kB 00:00 2024-05-27T07:14:01.116 INFO:teuthology.orchestra.run.smithi149.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 196 kB/s | 19 kB 00:00 2024-05-27T07:14:01.425 INFO:teuthology.orchestra.run.smithi149.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 594 kB/s | 309 kB 00:00 2024-05-27T07:14:01.542 INFO:teuthology.orchestra.run.smithi149.stdout:(108/159): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-05-27T07:14:01.567 INFO:teuthology.orchestra.run.smithi149.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 184 kB/s | 26 kB 00:00 2024-05-27T07:14:01.642 INFO:teuthology.orchestra.run.smithi149.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 490 kB/s | 49 kB 00:00 2024-05-27T07:14:01.668 INFO:teuthology.orchestra.run.smithi149.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 678 kB/s | 67 kB 00:00 2024-05-27T07:14:01.743 INFO:teuthology.orchestra.run.smithi149.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-05-27T07:14:01.889 INFO:teuthology.orchestra.run.smithi149.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 3.7 MB/s | 548 kB 00:00 2024-05-27T07:14:01.936 INFO:teuthology.orchestra.run.smithi149.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.1 MB/s | 839 kB 00:00 2024-05-27T07:14:01.986 INFO:teuthology.orchestra.run.smithi149.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 453 kB/s | 43 kB 00:00 2024-05-27T07:14:02.036 INFO:teuthology.orchestra.run.smithi149.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 324 kB/s | 32 kB 00:00 2024-05-27T07:14:02.262 INFO:teuthology.orchestra.run.smithi149.stdout:(117/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.7 MB/s | 16 MB 00:02 2024-05-27T07:14:02.287 INFO:teuthology.orchestra.run.smithi149.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 50 kB/s | 15 kB 00:00 2024-05-27T07:14:02.320 INFO:teuthology.orchestra.run.smithi149.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 606 kB/s | 172 kB 00:00 2024-05-27T07:14:02.387 INFO:teuthology.orchestra.run.smithi149.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-27T07:14:02.437 INFO:teuthology.orchestra.run.smithi149.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-27T07:14:02.563 INFO:teuthology.orchestra.run.smithi149.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-27T07:14:02.688 INFO:teuthology.orchestra.run.smithi149.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 86 kB/s | 11 kB 00:00 2024-05-27T07:14:02.849 INFO:teuthology.orchestra.run.smithi149.stdout:(124/159): python3-grpcio-1.46.7-10.el9.x86_64. 4.4 MB/s | 2.0 MB 00:00 2024-05-27T07:14:02.880 INFO:teuthology.orchestra.run.smithi149.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 92 kB/s | 18 kB 00:00 2024-05-27T07:14:02.914 INFO:teuthology.orchestra.run.smithi149.stdout:(126/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 484 kB/s | 303 kB 00:00 2024-05-27T07:14:02.947 INFO:teuthology.orchestra.run.smithi149.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 239 kB/s | 23 kB 00:00 2024-05-27T07:14:02.981 INFO:teuthology.orchestra.run.smithi149.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-05-27T07:14:03.015 INFO:teuthology.orchestra.run.smithi149.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-05-27T07:14:03.082 INFO:teuthology.orchestra.run.smithi149.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-05-27T07:14:03.107 INFO:teuthology.orchestra.run.smithi149.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-05-27T07:14:03.132 INFO:teuthology.orchestra.run.smithi149.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 676 kB/s | 79 kB 00:00 2024-05-27T07:14:03.182 INFO:teuthology.orchestra.run.smithi149.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-05-27T07:14:03.250 INFO:teuthology.orchestra.run.smithi149.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-27T07:14:03.250 INFO:teuthology.orchestra.run.smithi149.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-05-27T07:14:03.262 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-27T07:14:03.283 INFO:teuthology.orchestra.run.smithi149.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 899 kB/s | 90 kB 00:00 2024-05-27T07:14:03.309 INFO:teuthology.orchestra.run.smithi149.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 309 kB/s | 28 kB 00:00 2024-05-27T07:14:03.326 INFO:teuthology.orchestra.run.smithi149.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 336 kB/s | 31 kB 00:00 2024-05-27T07:14:03.369 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T07:14:03.393 INFO:teuthology.orchestra.run.smithi149.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-27T07:14:03.418 INFO:teuthology.orchestra.run.smithi149.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 546 kB/s | 59 kB 00:00 2024-05-27T07:14:03.435 INFO:teuthology.orchestra.run.smithi149.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-05-27T07:14:03.455 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T07:14:03.510 INFO:teuthology.orchestra.run.smithi149.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 439 kB/s | 51 kB 00:00 2024-05-27T07:14:03.544 INFO:teuthology.orchestra.run.smithi149.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-05-27T07:14:03.565 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T07:14:03.569 INFO:teuthology.orchestra.run.smithi149.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 672 kB/s | 90 kB 00:00 2024-05-27T07:14:03.625 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T07:14:03.636 INFO:teuthology.orchestra.run.smithi149.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-27T07:14:03.662 INFO:teuthology.orchestra.run.smithi149.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-05-27T07:14:03.664 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T07:14:03.695 INFO:teuthology.orchestra.run.smithi149.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-27T07:14:03.762 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T07:14:04.204 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T07:14:04.311 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:14:04.496 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:14:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:14:04.661 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:14:04.695 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:14:04.712 INFO:teuthology.orchestra.run.smithi149.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-05-27T07:14:04.784 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T07:14:04.920 INFO:teuthology.orchestra.run.smithi149.stdout:(149/159): librbd1-19.0.0-3872.g38f9d900.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-05-27T07:14:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T07:14:05.210 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T07:14:05.229 INFO:teuthology.orchestra.run.smithi149.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 4.5 MB/s | 2.3 MB 00:00 2024-05-27T07:14:05.234 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:14:05.296 INFO:teuthology.orchestra.run.smithi149.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 632 kB/s | 41 kB 00:00 2024-05-27T07:14:05.371 INFO:teuthology.orchestra.run.smithi149.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 3.3 MB/s | 248 kB 00:00 2024-05-27T07:14:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:14:05.427 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:14:05.446 INFO:teuthology.orchestra.run.smithi149.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 496 kB/s | 37 kB 00:00 2024-05-27T07:14:05.522 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:14:05.530 INFO:teuthology.orchestra.run.smithi149.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 2.4 MB/s | 207 kB 00:00 2024-05-27T07:14:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:14:05.605 INFO:teuthology.orchestra.run.smithi149.stdout:(155/159): librados2-19.0.0-3872.g38f9d900.el9. 1.7 MB/s | 3.3 MB 00:01 2024-05-27T07:14:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:14:05.639 INFO:teuthology.orchestra.run.smithi149.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.4 MB/s | 260 kB 00:00 2024-05-27T07:14:05.739 INFO:teuthology.orchestra.run.smithi149.stdout:(157/159): NetworkManager-libnm-1.47.90-1.el9.x 2.2 MB/s | 1.8 MB 00:00 2024-05-27T07:14:05.906 INFO:teuthology.orchestra.run.smithi149.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-05-27T07:14:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:14:06.040 INFO:teuthology.orchestra.run.smithi149.stdout:(159/159): openssl-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 1.3 MB 00:00 2024-05-27T07:14:06.046 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-27T07:14:06.047 INFO:teuthology.orchestra.run.smithi149.stdout:Total 9.8 MB/s | 219 MB 00:22 2024-05-27T07:14:06.282 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:14:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:14:06.573 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:14:06.726 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:14:06.834 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-05-27T07:14:06.843 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:14:06.925 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T07:14:07.000 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T07:14:07.092 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T07:14:07.143 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T07:14:07.176 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T07:14:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:14:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T07:14:07.392 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T07:14:07.526 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T07:14:07.595 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T07:14:07.712 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T07:14:07.730 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T07:14:07.786 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T07:14:07.824 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:14:07.874 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T07:14:07.940 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T07:14:07.977 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T07:14:08.007 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:14:08.034 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:14:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T07:14:08.158 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T07:14:08.159 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:14:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:14:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T07:14:08.248 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T07:14:08.277 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T07:14:08.361 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T07:14:08.396 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 33/170 2024-05-27T07:14:08.450 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:14:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 35/170 2024-05-27T07:14:08.612 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T07:14:08.628 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T07:14:08.693 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T07:14:08.698 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T07:14:08.723 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:14:08.809 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T07:14:08.854 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:14:08.890 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:14:08.959 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:14:08.983 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:14:08.989 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T07:14:09.045 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:14:09.059 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T07:14:09.241 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-27T07:14:09.352 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:14:09.492 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-27T07:14:09.492 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-27T07:14:09.694 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:14:09.758 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T07:14:09.870 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T07:14:09.890 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:14:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T07:14:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T07:14:09.978 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:14:10.012 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T07:14:10.060 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 46/170 2024-05-27T07:14:10.136 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T07:14:10.137 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:14:10.251 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:14:10.350 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T07:14:10.439 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T07:14:10.574 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T07:14:10.589 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:14:10.646 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T07:14:10.743 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T07:14:10.781 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T07:14:10.785 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T07:14:10.820 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T07:14:10.823 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T07:14:10.876 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T07:14:10.886 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T07:14:10.929 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T07:14:10.982 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T07:14:10.990 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T07:14:11.035 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T07:14:11.041 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T07:14:11.099 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T07:14:11.148 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T07:14:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T07:14:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T07:14:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T07:14:11.305 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T07:14:11.367 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T07:14:11.377 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T07:14:11.414 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T07:14:11.469 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T07:14:11.500 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T07:14:11.546 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T07:14:11.588 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T07:14:11.592 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T07:14:11.626 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 33/170 2024-05-27T07:14:11.628 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T07:14:11.661 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T07:14:11.675 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:14:11.708 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 35/170 2024-05-27T07:14:11.824 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T07:14:11.878 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T07:14:11.878 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T07:14:11.974 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T07:14:12.064 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T07:14:12.088 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-27T07:14:12.089 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-27T07:14:12.101 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T07:14:12.141 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T07:14:12.162 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T07:14:12.196 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T07:14:12.231 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T07:14:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T07:14:12.378 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T07:14:12.424 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T07:14:12.470 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T07:14:12.519 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T07:14:12.596 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T07:14:12.649 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T07:14:12.813 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T07:14:12.880 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T07:14:12.909 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T07:14:12.924 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T07:14:12.997 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T07:14:13.002 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T07:14:13.040 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T07:14:13.084 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T07:14:13.144 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T07:14:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T07:14:13.191 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 46/170 2024-05-27T07:14:13.229 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T07:14:13.264 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T07:14:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T07:14:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T07:14:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T07:14:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T07:14:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T07:14:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T07:14:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T07:14:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T07:14:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T07:14:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T07:14:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T07:14:14.228 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T07:14:14.286 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T07:14:14.319 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T07:14:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T07:14:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T07:14:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T07:14:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T07:14:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T07:14:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T07:14:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T07:14:14.678 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T07:14:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T07:14:14.730 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T07:14:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T07:14:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T07:14:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T07:14:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T07:14:14.996 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T07:14:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T07:14:15.128 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T07:14:15.167 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T07:14:15.213 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T07:14:15.251 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T07:14:15.312 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T07:14:15.373 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T07:14:15.424 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T07:14:15.466 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T07:14:15.512 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T07:14:15.562 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T07:14:15.637 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T07:14:15.684 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T07:14:15.831 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T07:14:15.891 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T07:14:15.934 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T07:14:15.990 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T07:14:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T07:14:16.128 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T07:14:16.163 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T07:14:16.198 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T07:14:17.113 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T07:14:17.165 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T07:14:17.198 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T07:14:17.319 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T07:14:17.373 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T07:14:17.426 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T07:14:17.525 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T07:14:17.567 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T07:14:17.803 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T07:14:17.839 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T07:14:17.884 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T07:14:17.946 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T07:14:17.984 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T07:14:18.019 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T07:14:18.053 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T07:14:18.445 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T07:14:18.512 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:14:18.639 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:14:18.670 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 104/170 2024-05-27T07:14:18.731 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T07:14:18.738 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-27T07:14:18.781 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T07:14:18.859 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T07:14:18.926 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T07:14:18.929 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T07:14:19.042 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T07:14:19.098 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T07:14:19.131 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T07:14:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T07:14:19.346 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T07:14:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:14:19.500 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:14:19.534 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 108/170 2024-05-27T07:14:19.798 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T07:14:19.913 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:14:20.095 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:14:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:14:20.266 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:14:20.355 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:14:20.439 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T07:14:20.515 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T07:14:20.620 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T07:14:20.656 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T07:14:20.691 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T07:14:20.794 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T07:14:20.861 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T07:14:20.885 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:14:21.037 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:14:21.076 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T07:14:21.098 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:14:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:14:21.189 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:14:21.221 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:14:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:14:21.286 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:14:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 104/170 2024-05-27T07:14:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T07:14:21.581 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:14:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T07:14:21.942 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:14:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:14:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:14:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 108/170 2024-05-27T07:14:22.143 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:14:22.232 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:14:22.318 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:14:22.356 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:14:22.402 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:14:22.505 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:14:22.968 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:14:23.071 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:14:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:14:23.262 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:14:23.262 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T07:14:23.262 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T07:14:23.262 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:23.293 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:14:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T07:14:23.436 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T07:14:23.494 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T07:14:23.608 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T07:14:23.684 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T07:14:23.761 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T07:14:23.814 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T07:14:23.860 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T07:14:23.980 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T07:14:24.092 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T07:14:24.131 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T07:14:24.211 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T07:14:24.322 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T07:14:24.365 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 33/170 2024-05-27T07:14:24.420 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:14:24.455 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 35/170 2024-05-27T07:14:24.574 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T07:14:24.641 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T07:14:24.648 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:14:24.740 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T07:14:24.759 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:14:24.918 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T07:14:24.987 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T07:14:25.453 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:14:25.504 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:14:25.675 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T07:14:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:14:25.688 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T07:14:25.688 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T07:14:25.688 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:25.730 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:14:25.783 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T07:14:25.836 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T07:14:25.876 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T07:14:25.933 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T07:14:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 46/170 2024-05-27T07:14:26.061 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T07:14:26.308 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T07:14:26.381 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T07:14:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T07:14:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T07:14:26.673 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T07:14:26.717 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T07:14:26.755 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T07:14:26.837 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T07:14:26.881 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T07:14:26.942 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T07:14:27.008 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T07:14:27.239 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T07:14:27.290 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T07:14:27.343 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T07:14:27.468 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T07:14:27.506 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T07:14:27.552 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T07:14:27.588 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T07:14:27.620 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T07:14:27.783 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T07:14:27.964 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T07:14:28.036 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T07:14:28.080 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T07:14:28.122 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T07:14:28.180 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T07:14:28.231 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T07:14:28.272 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T07:14:28.326 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T07:14:28.376 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T07:14:28.445 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T07:14:28.490 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T07:14:28.633 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T07:14:28.696 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T07:14:28.740 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T07:14:28.808 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T07:14:28.959 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T07:14:29.029 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T07:14:29.990 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T07:14:30.044 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T07:14:30.078 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T07:14:30.199 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T07:14:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T07:14:30.316 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T07:14:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T07:14:30.471 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T07:14:30.702 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T07:14:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T07:14:30.863 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T07:14:31.699 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T07:14:31.833 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T07:14:33.442 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T07:14:33.541 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T07:14:33.578 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T07:14:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T07:14:33.971 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T07:14:34.020 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:14:34.152 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:14:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 104/170 2024-05-27T07:14:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T07:14:34.818 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T07:14:34.848 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:14:34.939 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:14:34.966 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 108/170 2024-05-27T07:14:36.699 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:14:36.699 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /sys 2024-05-27T07:14:36.699 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /proc 2024-05-27T07:14:36.700 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /mnt 2024-05-27T07:14:36.700 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /var/tmp 2024-05-27T07:14:36.700 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /home 2024-05-27T07:14:36.700 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /root 2024-05-27T07:14:36.700 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /tmp 2024-05-27T07:14:36.700 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:36.944 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:14:36.995 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:14:36.995 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:36.995 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T07:14:36.995 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:37.554 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:14:37.622 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:14:38.315 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:14:38.359 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:14:38.521 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:14:38.521 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T07:14:38.522 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T07:14:38.522 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:14:38.576 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:14:38.606 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:14:38.606 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-05-27T07:14:38.606 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-05-27T07:14:38.606 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-05-27T07:14:38.606 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-05-27T07:14:38.606 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-05-27T07:14:38.606 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-05-27T07:14:38.606 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-05-27T07:14:38.606 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:38.813 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:14:38.837 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:14:38.885 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:14:38.886 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:38.886 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T07:14:38.886 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:38.924 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:14:38.925 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:38.925 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T07:14:38.925 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:14:38.925 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:14:38.925 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:14:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:14:39.289 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:39.289 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T07:14:39.289 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:14:39.289 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:14:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:14:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:14:39.927 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:39.927 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T07:14:39.927 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:14:39.927 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:14:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T07:14:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T07:14:40.204 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T07:14:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T07:14:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T07:14:40.614 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:14:40.644 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T07:14:40.694 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:14:40.694 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:40.694 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T07:14:40.695 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:14:40.695 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:14:40.695 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:40.753 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T07:14:40.921 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T07:14:40.977 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T07:14:40.981 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:14:41.018 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T07:14:41.048 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:14:41.049 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:41.049 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T07:14:41.049 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:14:41.049 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:14:41.049 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T07:14:41.599 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:14:41.665 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:14:41.666 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:41.666 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T07:14:41.666 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:14:41.666 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:14:41.666 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:41.699 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:14:41.737 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:14:41.784 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 128/170 2024-05-27T07:14:41.799 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T07:14:41.861 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T07:14:41.897 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 129/170 2024-05-27T07:14:41.941 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:14:41.956 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T07:14:42.195 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T07:14:42.270 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T07:14:42.368 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T07:14:42.470 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T07:14:42.631 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T07:14:42.687 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T07:14:42.727 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T07:14:42.760 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T07:14:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:14:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:14:43.419 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:14:43.452 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:14:43.507 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 128/170 2024-05-27T07:14:43.615 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 129/170 2024-05-27T07:14:43.645 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:14:44.508 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:14:44.532 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:14:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:14:44.894 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 133/170 2024-05-27T07:14:44.926 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:14:44.969 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:14:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:14:45.029 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:45.029 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T07:14:45.029 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:14:45.029 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:14:45.029 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:45.030 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:14:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:14:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:14:45.283 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 136/170 2024-05-27T07:14:45.357 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 137/170 2024-05-27T07:14:46.130 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:14:46.163 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:14:46.320 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:14:46.476 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T07:14:46.509 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 133/170 2024-05-27T07:14:46.512 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:14:46.540 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:14:46.609 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:14:46.609 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:46.610 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T07:14:46.610 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:14:46.610 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:14:46.610 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:46.634 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:14:46.634 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:46.634 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T07:14:46.635 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:14:46.635 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:14:46.635 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:46.663 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:14:46.706 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:14:46.746 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:14:46.765 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:14:46.766 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:46.766 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T07:14:46.766 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:46.847 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:14:46.881 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 136/170 2024-05-27T07:14:46.930 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:14:46.931 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:46.931 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T07:14:46.931 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:14:46.931 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:14:46.931 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:46.945 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 137/170 2024-05-27T07:14:47.330 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:14:47.408 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:14:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T07:14:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:14:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:14:47.408 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:47.985 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T07:14:48.019 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:14:48.114 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:14:48.115 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:48.115 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T07:14:48.115 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:14:48.115 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:14:48.115 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:48.161 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:14:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:14:48.259 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:48.259 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T07:14:48.259 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:14:48.410 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:14:48.410 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:48.410 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T07:14:48.411 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:14:48.411 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:14:48.411 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:48.787 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:14:48.855 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:14:48.855 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:48.855 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T07:14:48.855 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:14:48.856 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:14:48.856 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:51.496 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:14:51.497 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-05-27T07:14:51.497 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-05-27T07:14:51.497 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-05-27T07:14:51.497 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-05-27T07:14:51.497 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-05-27T07:14:51.497 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-05-27T07:14:51.497 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-05-27T07:14:51.497 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:14:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:14:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:14:51.797 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:51.798 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T07:14:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:14:52.336 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 143/170 2024-05-27T07:14:52.389 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 144/170 2024-05-27T07:14:52.444 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 145/170 2024-05-27T07:14:52.476 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T07:14:52.514 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T07:14:52.806 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 148/170 2024-05-27T07:14:52.881 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T07:14:52.916 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 150/170 2024-05-27T07:14:52.955 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T07:14:53.007 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T07:14:53.064 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:14:53.518 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 143/170 2024-05-27T07:14:53.568 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:14:53.575 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 144/170 2024-05-27T07:14:53.637 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 145/170 2024-05-27T07:14:53.646 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:14:53.646 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:53.646 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T07:14:53.646 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:14:53.646 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:14:53.646 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:14:53.676 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T07:14:53.714 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T07:14:53.919 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:14:54.007 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 148/170 2024-05-27T07:14:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:14:54.009 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:54.009 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T07:14:54.010 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:14:54.010 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:14:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:14:54.037 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:14:54.055 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T07:14:54.084 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:14:54.088 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 150/170 2024-05-27T07:14:54.120 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T07:14:54.153 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:14:54.168 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T07:14:54.221 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:14:54.527 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:14:54.594 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:14:54.594 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:54.594 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T07:14:54.594 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:14:54.594 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:14:54.594 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:14:54.728 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T07:14:54.813 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T07:14:54.891 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T07:14:55.123 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T07:14:55.141 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:14:55.182 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T07:14:55.183 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:14:55.236 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:14:55.274 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T07:14:55.383 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T07:14:55.561 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T07:14:55.615 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T07:14:55.623 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 155/170 2024-05-27T07:14:55.655 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T07:14:55.718 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T07:14:55.740 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T07:14:55.798 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T07:14:55.955 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T07:14:55.992 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T07:14:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:14:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T07:14:56.101 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T07:14:56.101 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:14:56.234 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:14:56.279 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T07:14:56.280 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:14:56.383 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:14:56.417 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:14:56.425 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:14:56.464 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T07:14:56.478 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 128/170 2024-05-27T07:14:56.499 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:14:56.499 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:14:56.533 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 155/170 2024-05-27T07:14:56.590 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 129/170 2024-05-27T07:14:56.619 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:14:56.639 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T07:14:56.698 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T07:14:56.843 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T07:14:56.843 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:14:56.844 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:14:56.882 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T07:14:56.915 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:14:56.950 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T07:14:56.989 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:14:56.990 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T07:14:56.991 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:14:57.033 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T07:14:57.125 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:14:57.156 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:14:57.170 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T07:14:57.171 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:14:57.307 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:14:57.363 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T07:14:57.396 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:14:57.397 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:14:57.423 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:14:57.423 INFO:teuthology.orchestra.run.smithi063.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T07:14:57.424 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:14:57.717 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:14:57.717 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:14:57.863 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:14:57.894 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:14:57.908 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T07:14:57.938 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:14:58.026 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:14:58.286 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:14:58.287 INFO:teuthology.orchestra.run.smithi086.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T07:14:58.287 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:14:59.052 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:14:59.087 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:14:59.248 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:14:59.434 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 133/170 2024-05-27T07:14:59.465 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:14:59.541 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:14:59.541 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:14:59.541 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T07:14:59.541 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:14:59.541 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:14:59.541 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:14:59.599 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:14:59.637 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:14:59.751 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 136/170 2024-05-27T07:14:59.818 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 137/170 2024-05-27T07:15:00.840 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T07:15:00.876 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:15:00.961 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:15:00.961 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:15:00.961 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T07:15:00.961 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:15:00.961 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:15:00.961 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:15:01.010 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:15:01.082 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:15:01.082 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:15:01.082 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T07:15:01.082 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:15:01.144 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:15:01.227 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:15:01.227 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:15:01.227 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T07:15:01.227 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:15:01.227 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:15:01.227 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:15:01.603 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:15:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:15:01.676 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:15:01.676 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T07:15:01.676 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:15:01.676 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:15:01.676 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:15:03.189 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 1/170 2024-05-27T07:15:03.189 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 3/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 4/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 5/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 6/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 7/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 11/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 12/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 13/170 2024-05-27T07:15:03.190 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 15/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 16/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 17/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 18/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 19/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 20/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 21/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 22/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 23/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 24/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 25/170 2024-05-27T07:15:03.191 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 26/170 2024-05-27T07:15:03.192 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 27/170 2024-05-27T07:15:03.192 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 28/170 2024-05-27T07:15:03.204 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 29/170 2024-05-27T07:15:03.204 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 30/170 2024-05-27T07:15:03.204 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 31/170 2024-05-27T07:15:03.204 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 32/170 2024-05-27T07:15:03.204 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 33/170 2024-05-27T07:15:03.204 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 35/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 36/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 37/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 38/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 39/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T07:15:03.205 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T07:15:03.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T07:15:03.207 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T07:15:03.208 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T07:15:03.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T07:15:03.210 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T07:15:03.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T07:15:03.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T07:15:03.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 149/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 151/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:15:03.214 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T07:15:03.215 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T07:15:03.215 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T07:15:03.215 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T07:15:03.215 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T07:15:03.215 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T07:15:03.215 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T07:15:03.215 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T07:15:03.215 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T07:15:04.158 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 1/170 2024-05-27T07:15:04.158 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2/170 2024-05-27T07:15:04.158 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 3/170 2024-05-27T07:15:04.158 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 4/170 2024-05-27T07:15:04.158 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 5/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 6/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 7/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 11/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 12/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 13/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 15/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 16/170 2024-05-27T07:15:04.159 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 17/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 18/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 19/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 20/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 21/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 22/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 23/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 24/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 25/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 26/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 27/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 28/170 2024-05-27T07:15:04.160 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 29/170 2024-05-27T07:15:04.172 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 30/170 2024-05-27T07:15:04.173 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 31/170 2024-05-27T07:15:04.173 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 32/170 2024-05-27T07:15:04.173 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 33/170 2024-05-27T07:15:04.173 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:15:04.173 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 35/170 2024-05-27T07:15:04.173 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 36/170 2024-05-27T07:15:04.173 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 37/170 2024-05-27T07:15:04.173 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 38/170 2024-05-27T07:15:04.173 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 39/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T07:15:04.174 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T07:15:04.175 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T07:15:04.176 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T07:15:04.177 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T07:15:04.178 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T07:15:04.179 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T07:15:04.180 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T07:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 149/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 151/170 2024-05-27T07:15:04.182 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T07:15:04.183 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T07:15:04.184 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T07:15:04.184 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T07:15:04.184 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T07:15:04.184 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T07:15:04.184 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T07:15:04.184 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T07:15:05.454 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T07:15:05.454 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:15:05.454 INFO:teuthology.orchestra.run.smithi063.stdout:Upgraded: 2024-05-27T07:15:05.454 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:05.454 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:05.454 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:05.454 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:05.454 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T07:15:05.454 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.455 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.456 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T07:15:05.457 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T07:15:05.458 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T07:15:05.459 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T07:15:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T07:15:05.461 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T07:15:05.462 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T07:15:05.463 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T07:15:05.464 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T07:15:05.465 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T07:15:05.466 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.466 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.466 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:05.466 INFO:teuthology.orchestra.run.smithi063.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T07:15:05.466 INFO:teuthology.orchestra.run.smithi063.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T07:15:05.466 INFO:teuthology.orchestra.run.smithi063.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T07:15:05.466 INFO:teuthology.orchestra.run.smithi063.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T07:15:05.466 INFO:teuthology.orchestra.run.smithi063.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T07:15:05.466 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T07:15:05.467 INFO:teuthology.orchestra.run.smithi063.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T07:15:05.467 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:15:05.467 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-27T07:15:06.128 DEBUG:teuthology.parallel:result is None 2024-05-27T07:15:06.385 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 143/170 2024-05-27T07:15:06.437 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 144/170 2024-05-27T07:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T07:15:06.445 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.446 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T07:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T07:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-05-27T07:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T07:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T07:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T07:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T07:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.447 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.448 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T07:15:06.449 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.450 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T07:15:06.451 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T07:15:06.452 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.453 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T07:15:06.454 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T07:15:06.455 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T07:15:06.455 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T07:15:06.455 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T07:15:06.455 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T07:15:06.455 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T07:15:06.455 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T07:15:06.455 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T07:15:06.455 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T07:15:06.455 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T07:15:06.456 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T07:15:06.457 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T07:15:06.457 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T07:15:06.457 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T07:15:06.457 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T07:15:06.457 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T07:15:06.457 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T07:15:06.457 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.457 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.457 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T07:15:06.458 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T07:15:06.458 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T07:15:06.458 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.458 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T07:15:06.458 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T07:15:06.458 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T07:15:06.458 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.459 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.460 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:06.460 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T07:15:06.460 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T07:15:06.460 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T07:15:06.460 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T07:15:06.460 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T07:15:06.460 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T07:15:06.460 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T07:15:06.460 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-05-27T07:15:06.460 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-05-27T07:15:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 145/170 2024-05-27T07:15:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T07:15:06.570 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T07:15:06.854 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 148/170 2024-05-27T07:15:06.913 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T07:15:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 150/170 2024-05-27T07:15:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T07:15:07.036 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T07:15:07.080 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:15:07.083 DEBUG:teuthology.parallel:result is None 2024-05-27T07:15:08.034 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:15:08.076 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:15:08.139 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:15:09.539 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 155/170 2024-05-27T07:15:09.650 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T07:15:09.708 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T07:15:09.857 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T07:15:09.902 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T07:15:09.935 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:15:09.969 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T07:15:10.011 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T07:15:10.011 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:15:10.169 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:15:10.223 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T07:15:10.224 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:15:10.341 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:15:10.374 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T07:15:10.409 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:15:10.409 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:15:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:15:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:15:10.874 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:15:10.919 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T07:15:11.039 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:15:11.309 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:15:11.310 INFO:teuthology.orchestra.run.smithi149.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T07:15:11.310 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:15:17.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 1/170 2024-05-27T07:15:17.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2/170 2024-05-27T07:15:17.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 3/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 4/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 5/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 6/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 7/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 11/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 12/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 13/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:15:17.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 15/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 16/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 17/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 18/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 19/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 20/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 21/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 22/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 23/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 24/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 25/170 2024-05-27T07:15:17.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 26/170 2024-05-27T07:15:17.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 27/170 2024-05-27T07:15:17.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 28/170 2024-05-27T07:15:17.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 29/170 2024-05-27T07:15:17.485 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 30/170 2024-05-27T07:15:17.485 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 31/170 2024-05-27T07:15:17.485 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 32/170 2024-05-27T07:15:17.485 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 33/170 2024-05-27T07:15:17.485 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:15:17.485 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 35/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 36/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 37/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 38/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 39/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T07:15:17.486 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T07:15:17.487 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T07:15:17.488 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T07:15:17.489 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T07:15:17.490 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T07:15:17.491 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T07:15:17.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T07:15:17.493 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T07:15:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T07:15:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T07:15:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T07:15:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T07:15:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T07:15:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T07:15:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T07:15:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T07:15:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T07:15:17.494 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 149/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 151/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T07:15:17.495 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T07:15:17.496 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T07:15:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T07:15:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T07:15:19.646 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T07:15:19.646 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:15:19.646 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-05-27T07:15:19.646 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:19.646 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:19.646 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T07:15:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:19.648 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T07:15:19.649 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T07:15:19.650 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T07:15:19.651 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T07:15:19.652 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T07:15:19.653 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T07:15:19.654 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T07:15:19.655 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T07:15:19.656 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T07:15:19.657 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.658 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:15:19.659 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T07:15:19.659 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T07:15:19.659 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T07:15:19.659 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T07:15:19.659 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T07:15:19.659 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T07:15:19.659 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T07:15:19.659 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:15:19.659 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-27T07:15:20.300 DEBUG:teuthology.parallel:result is None 2024-05-27T07:15:20.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:15:20.451 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T07:15:20.531 INFO:teuthology.orchestra.run.smithi063.stdout:19.0.0-3872.g38f9d900.el9 2024-05-27T07:15:20.531 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3872.g38f9d900.el9 2024-05-27T07:15:20.532 INFO:teuthology.task.install:The correct ceph version 19.0.0-3872.g38f9d900 is installed. 2024-05-27T07:15:20.533 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:15:20.809 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T07:15:20.882 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-3872.g38f9d900.el9 2024-05-27T07:15:20.882 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3872.g38f9d900.el9 2024-05-27T07:15:20.883 INFO:teuthology.task.install:The correct ceph version 19.0.0-3872.g38f9d900 is installed. 2024-05-27T07:15:20.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:15:21.197 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T07:15:21.254 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-3872.g38f9d900.el9 2024-05-27T07:15:21.255 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3872.g38f9d900.el9 2024-05-27T07:15:21.255 INFO:teuthology.task.install:The correct ceph version 19.0.0-3872.g38f9d900 is installed. 2024-05-27T07:15:21.256 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-27T07:15:21.257 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:15:21.257 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T07:15:21.354 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:15:21.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T07:15:21.447 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:15:21.447 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T07:15:21.509 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-27T07:15:21.509 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:15:21.510 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T07:15:21.606 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T07:15:21.743 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:15:21.743 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T07:15:21.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T07:15:21.959 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:15:21.959 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T07:15:22.026 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T07:15:22.130 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-27T07:15:22.130 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:15:22.130 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T07:15:22.224 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T07:15:22.355 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:15:22.355 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T07:15:22.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T07:15:22.570 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:15:22.570 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T07:15:22.642 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T07:15:22.750 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-27T07:15:22.750 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:15:22.750 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T07:15:22.846 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T07:15:22.936 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:15:22.936 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T07:15:23.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T07:15:23.111 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:15:23.112 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T07:15:23.196 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T07:15:23.261 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-27T07:15:23.345 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'subvols': {'create': 2}} 2024-05-27T07:15:23.346 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:15:23.346 INFO:tasks.cephadm:Cluster fsid is e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:15:23.346 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-27T07:15:23.346 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.63', 'mon.b': '172.21.15.86', 'mon.c': '172.21.15.149'} 2024-05-27T07:15:23.346 INFO:tasks.cephadm:First mon is mon.a on smithi063 2024-05-27T07:15:23.347 INFO:tasks.cephadm:First mgr is x 2024-05-27T07:15:23.347 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-27T07:15:23.347 DEBUG:teuthology.orchestra.run.smithi063:> sudo hostname $(hostname -s) 2024-05-27T07:15:23.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-05-27T07:15:23.535 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-05-27T07:15:23.622 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-27T07:15:23.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:15:23.842 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79587/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3872-g38f9d900', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3872.g38f9d900'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-21 08:08:15.155937', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240521.063558-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79583/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3872-g38f9d900', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3872.g38f9d900'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-21 07:54:08.730060', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240521.063558-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-27T07:15:23.982 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-vshankar-testing-20240521.063558-debug, sha1 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:15:23.983 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm 2024-05-27T07:15:23.984 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm 2024-05-27T07:15:23.984 DEBUG:teuthology.orchestra.run.smithi063:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T07:15:24.654 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r--. 1 ubuntu ubuntu 782786 May 27 07:15 /home/ubuntu/cephtest/cephadm 2024-05-27T07:15:24.654 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T07:15:25.196 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r--. 1 ubuntu ubuntu 782786 May 27 07:15 /home/ubuntu/cephtest/cephadm 2024-05-27T07:15:25.197 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T07:15:25.897 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 782786 May 27 07:15 /home/ubuntu/cephtest/cephadm 2024-05-27T07:15:25.898 DEBUG:teuthology.orchestra.run.smithi063:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T07:15:25.962 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T07:15:26.022 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T07:15:26.098 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 on all hosts... 2024-05-27T07:15:26.098 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 pull 2024-05-27T07:15:26.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 pull 2024-05-27T07:15:26.113 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 pull 2024-05-27T07:15:27.233 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T07:15:27.271 INFO:teuthology.orchestra.run.smithi063.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T07:15:27.312 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T07:16:21.654 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-27T07:16:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:16:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799", 2024-05-27T07:16:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-05-27T07:16:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6" 2024-05-27T07:16:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-27T07:16:21.655 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-27T07:16:23.868 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-05-27T07:16:23.868 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:16:23.868 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799", 2024-05-27T07:16:23.868 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-05-27T07:16:23.868 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6" 2024-05-27T07:16:23.868 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-05-27T07:16:23.868 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-05-27T07:16:32.207 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:16:32.207 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:16:32.207 INFO:teuthology.orchestra.run.smithi063.stdout: "image_id": "e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799", 2024-05-27T07:16:32.207 INFO:teuthology.orchestra.run.smithi063.stdout: "repo_digests": [ 2024-05-27T07:16:32.207 INFO:teuthology.orchestra.run.smithi063.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6" 2024-05-27T07:16:32.207 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T07:16:32.207 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:16:32.250 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph 2024-05-27T07:16:32.340 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-05-27T07:16:32.436 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-05-27T07:16:32.532 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /etc/ceph 2024-05-27T07:16:32.628 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-05-27T07:16:32.711 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-05-27T07:16:32.799 INFO:tasks.cephadm:Writing seed config... 2024-05-27T07:16:32.800 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-27T07:16:32.800 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-27T07:16:32.800 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-27T07:16:32.800 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-27T07:16:32.800 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-27T07:16:32.801 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-27T07:16:32.802 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-27T07:16:32.803 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-27T07:16:32.804 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-27T07:16:32.804 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:16:32.804 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-27T07:16:32.864 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e24014b4-1bf8-11ef-bc9b-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-27T07:16:32.865 DEBUG:teuthology.orchestra.run.smithi063:mon.a> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.a.service 2024-05-27T07:16:32.912 DEBUG:teuthology.orchestra.run.smithi063:mgr.x> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.x.service 2024-05-27T07:16:32.959 INFO:tasks.cephadm:Bootstrapping... 2024-05-27T07:16:32.959 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 -v bootstrap --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.63 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:16:33.233 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-27T07:16:33.233 INFO:teuthology.orchestra.run.smithi063.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6', '-v', 'bootstrap', '--fsid', 'e24014b4-1bf8-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.63', '--skip-admin-label'] 2024-05-27T07:16:33.233 INFO:teuthology.orchestra.run.smithi063.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-27T07:16:33.234 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying podman|docker is present... 2024-05-27T07:16:33.295 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 5.0.2 2024-05-27T07:16:33.296 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying lvm2 is present... 2024-05-27T07:16:33.296 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying time synchronization is in place... 2024-05-27T07:16:33.313 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T07:16:33.313 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T07:16:33.329 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T07:16:33.329 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-27T07:16:33.349 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout enabled 2024-05-27T07:16:33.368 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout active 2024-05-27T07:16:33.368 INFO:teuthology.orchestra.run.smithi063.stdout:Unit chronyd.service is enabled and running 2024-05-27T07:16:33.368 INFO:teuthology.orchestra.run.smithi063.stdout:Repeating the final host check... 2024-05-27T07:16:33.428 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 5.0.2 2024-05-27T07:16:33.428 INFO:teuthology.orchestra.run.smithi063.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-27T07:16:33.428 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl is present 2024-05-27T07:16:33.428 INFO:teuthology.orchestra.run.smithi063.stdout:lvcreate is present 2024-05-27T07:16:33.447 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T07:16:33.447 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T07:16:33.466 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T07:16:33.466 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-27T07:16:33.489 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout enabled 2024-05-27T07:16:33.509 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout active 2024-05-27T07:16:33.509 INFO:teuthology.orchestra.run.smithi063.stdout:Unit chronyd.service is enabled and running 2024-05-27T07:16:33.509 INFO:teuthology.orchestra.run.smithi063.stdout:Host looks OK 2024-05-27T07:16:33.509 INFO:teuthology.orchestra.run.smithi063.stdout:Cluster fsid: e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:16:33.510 INFO:teuthology.orchestra.run.smithi063.stdout:Acquiring lock 139915013133072 on /run/cephadm/e24014b4-1bf8-11ef-bc9b-c7b262605968.lock 2024-05-27T07:16:33.510 INFO:teuthology.orchestra.run.smithi063.stdout:Lock 139915013133072 acquired on /run/cephadm/e24014b4-1bf8-11ef-bc9b-c7b262605968.lock 2024-05-27T07:16:33.510 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying IP 172.21.15.63 port 3300 ... 2024-05-27T07:16:33.511 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying IP 172.21.15.63 port 6789 ... 2024-05-27T07:16:33.511 INFO:teuthology.orchestra.run.smithi063.stdout:Base mon IP(s) is [172.21.15.63:3300, 172.21.15.63:6789], mon addrv is [v2:172.21.15.63:3300,v1:172.21.15.63:6789] 2024-05-27T07:16:33.521 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.63 metric 100 2024-05-27T07:16:33.521 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.63 metric 100 2024-05-27T07:16:33.530 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-27T07:16:33.530 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-27T07:16:33.530 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-05-27T07:16:33.538 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-27T07:16:33.539 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-27T07:16:33.539 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T07:16:33.539 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-27T07:16:33.539 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-27T07:16:33.539 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T07:16:33.540 INFO:teuthology.orchestra.run.smithi063.stdout:Mon IP `172.21.15.63` is in CIDR network `172.21.0.0/20` 2024-05-27T07:16:33.540 INFO:teuthology.orchestra.run.smithi063.stdout:Mon IP `172.21.15.63` is in CIDR network `172.21.0.0/20` 2024-05-27T07:16:33.540 INFO:teuthology.orchestra.run.smithi063.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-27T07:16:33.542 INFO:teuthology.orchestra.run.smithi063.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-27T07:16:33.543 INFO:teuthology.orchestra.run.smithi063.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T07:16:34.187 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 2024-05-27T07:16:34.187 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T07:16:34.187 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Getting image source signatures 2024-05-27T07:16:34.187 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying blob sha256:6ac36e7c57f227110df5cd65cfdca1cc05e0b17df6748637a6aa04c5c80f89c6 2024-05-27T07:16:34.187 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying blob sha256:cf46015854c6cdc81d3273f73f8f001fe4a1edc6ed77e464da8c9d5e01e9b108 2024-05-27T07:16:34.188 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying config sha256:e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 2024-05-27T07:16:34.188 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-27T07:16:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:ceph: stdout ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:16:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:Ceph version: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:16:35.614 INFO:teuthology.orchestra.run.smithi063.stdout:Extracting ceph user uid/gid from container image... 2024-05-27T07:16:36.794 INFO:teuthology.orchestra.run.smithi063.stdout:stat: stdout 167 167 2024-05-27T07:16:36.794 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial keys... 2024-05-27T07:16:38.006 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQBVM1Rmd8H3FRAAuqhlRCy48kNDv0n8AMXPfw== 2024-05-27T07:16:39.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQBWM1Rm0OYcKBAAmUX/HxKGlYXkbo1z/KVndQ== 2024-05-27T07:16:40.399 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQBXM1Rm8MLTLxAAkhx33b6AnXkgFo7Ub0/rag== 2024-05-27T07:16:40.401 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial monmap... 2024-05-27T07:16:41.614 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T07:16:41.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-27T07:16:41.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:16:41.615 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T07:16:41.616 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool for a [v2:172.21.15.63:3300,v1:172.21.15.63:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T07:16:41.616 INFO:teuthology.orchestra.run.smithi063.stdout:setting min_mon_release = quincy 2024-05-27T07:16:41.616 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: set fsid to e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:16:41.616 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T07:16:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:16:41.616 INFO:teuthology.orchestra.run.smithi063.stdout:Creating mon... 2024-05-27T07:16:42.902 INFO:teuthology.orchestra.run.smithi063.stdout:create mon.a on 2024-05-27T07:16:43.289 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-27T07:16:43.877 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-27T07:16:45.946 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968.target. 2024-05-27T07:16:45.946 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968.target. 2024-05-27T07:16:49.486 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.a 2024-05-27T07:16:49.486 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-05-27T07:16:50.121 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968.target.wants/ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@.service. 2024-05-27T07:16:50.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:50 smithi063 podman[31228]: 2024-05-27 07:16:50.436404467 +0000 UTC m=+0.033698867 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:16:52.116 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:51 smithi063 podman[31228]: 2024-05-27 07:16:51.867198001 +0000 UTC m=+1.464492438 container create 788ec44b7fc2da24b57e37385d7c659c014ab8a615b1e11c1fb69daf3768332b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:16:52.116 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:52 smithi063 podman[31228]: 2024-05-27 07:16:52.112764578 +0000 UTC m=+1.710059042 container init 788ec44b7fc2da24b57e37385d7c659c014ab8a615b1e11c1fb69daf3768332b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True) 2024-05-27T07:16:52.116 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:52 smithi063 podman[31228]: 2024-05-27 07:16:52.125646395 +0000 UTC m=+1.722940839 container start 788ec44b7fc2da24b57e37385d7c659c014ab8a615b1e11c1fb69daf3768332b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-27T07:16:52.239 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T07:16:52.239 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-27T07:16:52.259 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T07:16:52.259 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-27T07:16:52.259 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-27T07:16:52.259 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T07:16:52.259 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mon to start... 2024-05-27T07:16:52.260 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mon... 2024-05-27T07:16:52.409 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:52 smithi063 bash[31228]: 788ec44b7fc2da24b57e37385d7c659c014ab8a615b1e11c1fb69daf3768332b 2024-05-27T07:16:52.409 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:52 smithi063 systemd[1]: Started Ceph mon.a for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:16:52.409 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:52 smithi063 ceph-mon[31247]: mkfs e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:16:52.410 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:52 smithi063 ceph-mon[31247]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T07:16:53.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T07:16:53.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: monmap epoch 1 2024-05-27T07:16:53.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:16:53.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: last_changed 2024-05-27T07:16:40.949861+0000 2024-05-27T07:16:53.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: created 2024-05-27T07:16:40.949861+0000 2024-05-27T07:16:53.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: min_mon_release 19 (squid) 2024-05-27T07:16:53.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: election_strategy: 1 2024-05-27T07:16:53.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-27T07:16:53.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: fsmap 2024-05-27T07:16:53.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T07:16:53.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: mgrmap e1: no daemons active 2024-05-27T07:16:53.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:53 smithi063 ceph-mon[31247]: from='client.? 172.21.15.63:0/3458888880' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-27T07:16:53.735 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout cluster: 2024-05-27T07:16:53.736 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout id: e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:16:53.737 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-27T07:16:53.737 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:16:53.737 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout services: 2024-05-27T07:16:53.737 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.809623s) 2024-05-27T07:16:53.737 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-27T07:16:53.739 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-27T07:16:53.739 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:16:53.739 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout data: 2024-05-27T07:16:53.739 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout pgs: 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.994+0000 7f2841800700 1 Processor -- start 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.994+0000 7f2841800700 1 -- start start 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.995+0000 7f2841800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.995+0000 7f2841800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f283c10a2b0 con 0x7f283c1098b0 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.996+0000 7f283be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.996+0000 7f283be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42826/0 (socket says 172.21.15.63:42826) 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.996+0000 7f283be00700 1 -- 172.21.15.63:0/1613041955 learned_addr learned my addr 172.21.15.63:0/1613041955 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:16:53.740 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.997+0000 7f283be00700 1 -- 172.21.15.63:0/1613041955 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c10aa80 con 0x7f283c1098b0 2024-05-27T07:16:53.741 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.997+0000 7f283be00700 1 --2- 172.21.15.63:0/1613041955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c109cc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f282c0098a0 tx=0x7f282c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4184613dae3f83d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:16:53.741 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.997+0000 7f283aa00700 1 -- 172.21.15.63:0/1613041955 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282c004160 con 0x7f283c1098b0 2024-05-27T07:16:53.741 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.997+0000 7f283aa00700 1 -- 172.21.15.63:0/1613041955 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c004300 con 0x7f283c1098b0 2024-05-27T07:16:53.741 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.997+0000 7f283aa00700 1 -- 172.21.15.63:0/1613041955 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282c0036a0 con 0x7f283c1098b0 2024-05-27T07:16:53.741 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.998+0000 7f2841800700 1 -- 172.21.15.63:0/1613041955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 msgr2=0x7f283c109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:16:53.741 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.998+0000 7f2841800700 1 --2- 172.21.15.63:0/1613041955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c109cc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f282c0098a0 tx=0x7f282c009bb0 comp rx=0 tx=0).stop 2024-05-27T07:16:53.741 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.998+0000 7f2841800700 1 -- 172.21.15.63:0/1613041955 shutdown_connections 2024-05-27T07:16:53.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.998+0000 7f2841800700 1 --2- 172.21.15.63:0/1613041955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c109cc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:16:53.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.998+0000 7f2841800700 1 -- 172.21.15.63:0/1613041955 >> 172.21.15.63:0/1613041955 conn(0x7f283c104f30 msgr2=0x7f283c1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.999+0000 7f2841800700 1 -- 172.21.15.63:0/1613041955 shutdown_connections 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:52.999+0000 7f2841800700 1 -- 172.21.15.63:0/1613041955 wait complete. 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.000+0000 7f2841800700 1 Processor -- start 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.000+0000 7f2841800700 1 -- start start 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.001+0000 7f2841800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.001+0000 7f2841800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f283c10afb0 con 0x7f283c1098b0 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.001+0000 7f283be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.001+0000 7f283be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c19f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42842/0 (socket says 172.21.15.63:42842) 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.001+0000 7f283be00700 1 -- 172.21.15.63:0/3458888880 learned_addr learned my addr 172.21.15.63:0/3458888880 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.001+0000 7f283be00700 1 -- 172.21.15.63:0/3458888880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c19fa60 con 0x7f283c1098b0 2024-05-27T07:16:53.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.002+0000 7f283be00700 1 --2- 172.21.15.63:0/3458888880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c19f520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f282c000c00 tx=0x7f282c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.002+0000 7f2838c00700 1 -- 172.21.15.63:0/3458888880 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282c003db0 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.002+0000 7f2838c00700 1 -- 172.21.15.63:0/3458888880 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c011630 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.002+0000 7f2838c00700 1 -- 172.21.15.63:0/3458888880 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282c011950 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.002+0000 7f2841800700 1 -- 172.21.15.63:0/3458888880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c19fcf0 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.002+0000 7f2841800700 1 -- 172.21.15.63:0/3458888880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c1a2900 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.003+0000 7f2838c00700 1 -- 172.21.15.63:0/3458888880 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f282c011af0 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.003+0000 7f2838c00700 1 -- 172.21.15.63:0/3458888880 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f282c01c450 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.004+0000 7f2841800700 1 -- 172.21.15.63:0/3458888880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c064e10 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.008+0000 7f2838c00700 1 -- 172.21.15.63:0/3458888880 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f282c032020 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.089+0000 7f2841800700 1 -- 172.21.15.63:0/3458888880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f283c1a2ad0 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.089+0000 7f2838c00700 1 -- 172.21.15.63:0/3458888880 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f283c1a2ad0 con 0x7f283c1098b0 2024-05-27T07:16:53.746 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.091+0000 7f2841800700 1 -- 172.21.15.63:0/3458888880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 msgr2=0x7f283c19f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:16:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.091+0000 7f2841800700 1 --2- 172.21.15.63:0/3458888880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c19f520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f282c000c00 tx=0x7f282c004750 comp rx=0 tx=0).stop 2024-05-27T07:16:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.091+0000 7f2841800700 1 -- 172.21.15.63:0/3458888880 shutdown_connections 2024-05-27T07:16:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.091+0000 7f2841800700 1 --2- 172.21.15.63:0/3458888880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f283c1098b0 0x7f283c19f520 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:16:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.091+0000 7f2841800700 1 -- 172.21.15.63:0/3458888880 >> 172.21.15.63:0/3458888880 conn(0x7f283c104f30 msgr2=0x7f283c07dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:16:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.092+0000 7f2841800700 1 -- 172.21.15.63:0/3458888880 shutdown_connections 2024-05-27T07:16:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:53.092+0000 7f2841800700 1 -- 172.21.15.63:0/3458888880 wait complete. 2024-05-27T07:16:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:mon is available 2024-05-27T07:16:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:Assimilating anything we can from ceph.conf... 2024-05-27T07:16:54.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:54 smithi063 ceph-mon[31247]: from='client.? 172.21.15.63:0/1049015558' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T07:16:54.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:54 smithi063 ceph-mon[31247]: from='client.? 172.21.15.63:0/1049015558' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-27T07:16:55.578 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:16:55.578 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T07:16:55.579 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout fsid = e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:16:55.579 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-27T07:16:55.579 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.63:3300,v1:172.21.15.63:6789] 2024-05-27T07:16:55.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T07:16:55.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T07:16:55.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T07:16:55.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T07:16:55.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:16:55.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T07:16:55.580 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T07:16:55.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:16:55.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T07:16:55.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-05-27T07:16:55.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T07:16:55.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T07:16:55.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T07:16:55.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T07:16:55.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.451+0000 7f0395800700 1 Processor -- start 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.452+0000 7f0395800700 1 -- start start 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.452+0000 7f0395800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f03901085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.452+0000 7f0395800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f039007d820 con 0x7f03901081e0 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.453+0000 7f038fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f03901085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.453+0000 7f038fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f03901085f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42852/0 (socket says 172.21.15.63:42852) 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.453+0000 7f038fe00700 1 -- 172.21.15.63:0/4286911924 learned_addr learned my addr 172.21.15.63:0/4286911924 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.454+0000 7f038fe00700 1 -- 172.21.15.63:0/4286911924 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0390108b30 con 0x7f03901081e0 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.454+0000 7f038fe00700 1 --2- 172.21.15.63:0/4286911924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f03901085f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f03800098a0 tx=0x7f0380009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd9ddd9f38266ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.454+0000 7f038ea00700 1 -- 172.21.15.63:0/4286911924 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0380004160 con 0x7f03901081e0 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.455+0000 7f038ea00700 1 -- 172.21.15.63:0/4286911924 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0380004300 con 0x7f03901081e0 2024-05-27T07:16:55.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.455+0000 7f038ea00700 1 -- 172.21.15.63:0/4286911924 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03800036a0 con 0x7f03901081e0 2024-05-27T07:16:55.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.455+0000 7f0395800700 1 -- 172.21.15.63:0/4286911924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 msgr2=0x7f03901085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:16:55.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.455+0000 7f0395800700 1 --2- 172.21.15.63:0/4286911924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f03901085f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f03800098a0 tx=0x7f0380009bb0 comp rx=0 tx=0).stop 2024-05-27T07:16:55.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.455+0000 7f0395800700 1 -- 172.21.15.63:0/4286911924 shutdown_connections 2024-05-27T07:16:55.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.455+0000 7f0395800700 1 --2- 172.21.15.63:0/4286911924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f03901085f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:16:55.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.455+0000 7f0395800700 1 -- 172.21.15.63:0/4286911924 >> 172.21.15.63:0/4286911924 conn(0x7f0390103dd0 msgr2=0x7f0390106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:16:55.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.456+0000 7f0395800700 1 -- 172.21.15.63:0/4286911924 shutdown_connections 2024-05-27T07:16:55.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.456+0000 7f0395800700 1 -- 172.21.15.63:0/4286911924 wait complete. 2024-05-27T07:16:55.585 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.457+0000 7f0395800700 1 Processor -- start 2024-05-27T07:16:55.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.457+0000 7f0395800700 1 -- start start 2024-05-27T07:16:55.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.457+0000 7f0395800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f039019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:16:55.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.457+0000 7f0395800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0390109060 con 0x7f03901081e0 2024-05-27T07:16:55.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.458+0000 7f038fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f039019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.458+0000 7f038fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f039019f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42864/0 (socket says 172.21.15.63:42864) 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.458+0000 7f038fe00700 1 -- 172.21.15.63:0/1049015558 learned_addr learned my addr 172.21.15.63:0/1049015558 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.458+0000 7f038fe00700 1 -- 172.21.15.63:0/1049015558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039019fa50 con 0x7f03901081e0 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.459+0000 7f038fe00700 1 --2- 172.21.15.63:0/1049015558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f039019f510 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0380000c00 tx=0x7f0380004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.459+0000 7f038cc00700 1 -- 172.21.15.63:0/1049015558 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0380003db0 con 0x7f03901081e0 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.459+0000 7f038cc00700 1 -- 172.21.15.63:0/1049015558 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0380011630 con 0x7f03901081e0 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.459+0000 7f0395800700 1 -- 172.21.15.63:0/1049015558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039019fce0 con 0x7f03901081e0 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.459+0000 7f038cc00700 1 -- 172.21.15.63:0/1049015558 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0380011950 con 0x7f03901081e0 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.459+0000 7f0395800700 1 -- 172.21.15.63:0/1049015558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03901a28f0 con 0x7f03901081e0 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.459+0000 7f038cc00700 1 -- 172.21.15.63:0/1049015558 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0380011af0 con 0x7f03901081e0 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.459+0000 7f038cc00700 1 -- 172.21.15.63:0/1049015558 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f038001c790 con 0x7f03901081e0 2024-05-27T07:16:55.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.460+0000 7f0395800700 1 -- 172.21.15.63:0/1049015558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0390064e10 con 0x7f03901081e0 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.463+0000 7f038cc00700 1 -- 172.21.15.63:0/1049015558 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0380032020 con 0x7f03901081e0 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.542+0000 7f0395800700 1 -- 172.21.15.63:0/1049015558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f03901a2ac0 con 0x7f03901081e0 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.557+0000 7f038cc00700 1 -- 172.21.15.63:0/1049015558 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+490 (secure 0 0 0) 0x7f0380004260 con 0x7f03901081e0 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.559+0000 7f0395800700 1 -- 172.21.15.63:0/1049015558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 msgr2=0x7f039019f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.559+0000 7f0395800700 1 --2- 172.21.15.63:0/1049015558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f039019f510 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0380000c00 tx=0x7f0380004750 comp rx=0 tx=0).stop 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.559+0000 7f038cc00700 1 -- 172.21.15.63:0/1049015558 <== mon.0 v2:172.21.15.63:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f038001ce00 con 0x7f03901081e0 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.559+0000 7f0395800700 1 -- 172.21.15.63:0/1049015558 shutdown_connections 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.559+0000 7f0395800700 1 --2- 172.21.15.63:0/1049015558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03901081e0 0x7f039019f510 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.559+0000 7f0395800700 1 -- 172.21.15.63:0/1049015558 >> 172.21.15.63:0/1049015558 conn(0x7f0390103dd0 msgr2=0x7f03901957e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.559+0000 7f0395800700 1 -- 172.21.15.63:0/1049015558 shutdown_connections 2024-05-27T07:16:55.590 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:54.559+0000 7f0395800700 1 -- 172.21.15.63:0/1049015558 wait complete. 2024-05-27T07:16:55.591 INFO:teuthology.orchestra.run.smithi063.stdout:Generating new minimal ceph.conf... 2024-05-27T07:16:56.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:56 smithi063 ceph-mon[31247]: from='client.? 172.21.15.63:0/1440412803' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:16:57.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.251+0000 7f8427200700 1 Processor -- start 2024-05-27T07:16:57.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.252+0000 7f8427200700 1 -- start start 2024-05-27T07:16:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.252+0000 7f8427200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f8428112b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:16:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.252+0000 7f8427200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8428113110 con 0x7f8428110720 2024-05-27T07:16:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.252+0000 7f8425e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f8428112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:16:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.253+0000 7f8425e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f8428112b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42872/0 (socket says 172.21.15.63:42872) 2024-05-27T07:16:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.253+0000 7f8425e00700 1 -- 172.21.15.63:0/2732816504 learned_addr learned my addr 172.21.15.63:0/2732816504 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:16:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.253+0000 7f8425e00700 1 -- 172.21.15.63:0/2732816504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8428113290 con 0x7f8428110720 2024-05-27T07:16:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.253+0000 7f8425e00700 1 --2- 172.21.15.63:0/2732816504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f8428112b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f84180098a0 tx=0x7f8418009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232c2309c0ece9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:16:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.254+0000 7f8424a00700 1 -- 172.21.15.63:0/2732816504 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8418004160 con 0x7f8428110720 2024-05-27T07:16:57.231 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.254+0000 7f8424a00700 1 -- 172.21.15.63:0/2732816504 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8418004300 con 0x7f8428110720 2024-05-27T07:16:57.231 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.254+0000 7f8424a00700 1 -- 172.21.15.63:0/2732816504 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84180119e0 con 0x7f8428110720 2024-05-27T07:16:57.231 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.254+0000 7f8427200700 1 -- 172.21.15.63:0/2732816504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 msgr2=0x7f8428112b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:16:57.231 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.254+0000 7f8427200700 1 --2- 172.21.15.63:0/2732816504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f8428112b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f84180098a0 tx=0x7f8418009bb0 comp rx=0 tx=0).stop 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.255+0000 7f8427200700 1 -- 172.21.15.63:0/2732816504 shutdown_connections 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.255+0000 7f8427200700 1 --2- 172.21.15.63:0/2732816504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f8428112b40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.255+0000 7f8427200700 1 -- 172.21.15.63:0/2732816504 >> 172.21.15.63:0/2732816504 conn(0x7f842807f0b0 msgr2=0x7f84280814d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.255+0000 7f8427200700 1 -- 172.21.15.63:0/2732816504 shutdown_connections 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.255+0000 7f8427200700 1 -- 172.21.15.63:0/2732816504 wait complete. 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.256+0000 7f8427200700 1 Processor -- start 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.256+0000 7f8427200700 1 -- start start 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.256+0000 7f8427200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f84281a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.256+0000 7f8427200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84281141e0 con 0x7f8428110720 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.257+0000 7f8425e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f84281a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:16:57.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.257+0000 7f8425e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f84281a40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42878/0 (socket says 172.21.15.63:42878) 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.257+0000 7f8425e00700 1 -- 172.21.15.63:0/1440412803 learned_addr learned my addr 172.21.15.63:0/1440412803 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.257+0000 7f8425e00700 1 -- 172.21.15.63:0/1440412803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84281a4630 con 0x7f8428110720 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.258+0000 7f8425e00700 1 --2- 172.21.15.63:0/1440412803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f84281a40f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84180116a0 tx=0x7f8418011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.258+0000 7f841ea00700 1 -- 172.21.15.63:0/1440412803 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8418003970 con 0x7f8428110720 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.258+0000 7f841ea00700 1 -- 172.21.15.63:0/1440412803 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f8418003b10 con 0x7f8428110720 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.258+0000 7f8427200700 1 -- 172.21.15.63:0/1440412803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84281a4920 con 0x7f8428110720 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.258+0000 7f841ea00700 1 -- 172.21.15.63:0/1440412803 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8418019550 con 0x7f8428110720 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.258+0000 7f8427200700 1 -- 172.21.15.63:0/1440412803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84281a74d0 con 0x7f8428110720 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.258+0000 7f841ea00700 1 -- 172.21.15.63:0/1440412803 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8418019b50 con 0x7f8428110720 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.258+0000 7f841ea00700 1 -- 172.21.15.63:0/1440412803 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f841800b720 con 0x7f8428110720 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.260+0000 7f8427200700 1 -- 172.21.15.63:0/1440412803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8428064e10 con 0x7f8428110720 2024-05-27T07:16:57.233 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.263+0000 7f841ea00700 1 -- 172.21.15.63:0/1440412803 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f841801aa50 con 0x7f8428110720 2024-05-27T07:16:57.234 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.340+0000 7f8427200700 1 -- 172.21.15.63:0/1440412803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f84281a77c0 con 0x7f8428110720 2024-05-27T07:16:57.234 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.341+0000 7f841ea00700 1 -- 172.21.15.63:0/1440412803 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f841801a430 con 0x7f8428110720 2024-05-27T07:16:57.234 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.342+0000 7f8427200700 1 -- 172.21.15.63:0/1440412803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 msgr2=0x7f84281a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:16:57.234 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.342+0000 7f8427200700 1 --2- 172.21.15.63:0/1440412803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f84281a40f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84180116a0 tx=0x7f8418011a60 comp rx=0 tx=0).stop 2024-05-27T07:16:57.234 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.342+0000 7f8427200700 1 -- 172.21.15.63:0/1440412803 shutdown_connections 2024-05-27T07:16:57.234 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.342+0000 7f8427200700 1 --2- 172.21.15.63:0/1440412803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8428110720 0x7f84281a40f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:16:57.234 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.342+0000 7f8427200700 1 -- 172.21.15.63:0/1440412803 >> 172.21.15.63:0/1440412803 conn(0x7f842807f0b0 msgr2=0x7f8428080830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:16:57.234 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.342+0000 7f8427200700 1 -- 172.21.15.63:0/1440412803 shutdown_connections 2024-05-27T07:16:57.234 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:16:56.343+0000 7f8427200700 1 -- 172.21.15.63:0/1440412803 wait complete. 2024-05-27T07:16:57.235 INFO:teuthology.orchestra.run.smithi063.stdout:Restarting the monitor... 2024-05-27T07:16:57.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:57 smithi063 systemd[1]: Stopping Ceph mon.a for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:16:58.084 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:57 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a[31243]: 2024-05-27T07:16:57.769+0000 7f9fd2a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:16:58.084 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:57 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a[31243]: 2024-05-27T07:16:57.769+0000 7f9fd2a00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-27T07:16:58.084 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:57 smithi063 podman[31490]: 2024-05-27 07:16:57.811043377 +0000 UTC m=+0.170016423 container died 788ec44b7fc2da24b57e37385d7c659c014ab8a615b1e11c1fb69daf3768332b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-27T07:16:58.412 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:58 smithi063 podman[31490]: 2024-05-27 07:16:58.245863887 +0000 UTC m=+0.604836935 container cleanup 788ec44b7fc2da24b57e37385d7c659c014ab8a615b1e11c1fb69daf3768332b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-27T07:16:58.412 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:58 smithi063 bash[31490]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a 2024-05-27T07:16:58.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:58 smithi063 podman[31504]: 2024-05-27 07:16:58.421538705 +0000 UTC m=+0.592469453 container remove 788ec44b7fc2da24b57e37385d7c659c014ab8a615b1e11c1fb69daf3768332b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:16:59.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:58 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-27T07:16:59.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:58 smithi063 systemd[1]: Stopped Ceph mon.a for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:16:59.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:58 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.a.service: Consumed 1.982s CPU time. 2024-05-27T07:16:59.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:58 smithi063 systemd[1]: Starting Ceph mon.a for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:16:59.551 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 podman[31594]: 2024-05-27 07:16:59.210814494 +0000 UTC m=+0.034398867 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:16:59.551 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 podman[31594]: 2024-05-27 07:16:59.322307868 +0000 UTC m=+0.145892278 container create 07e5df3f996bf0093f4354423418f98e8af659f9fec288bce0b1e9d8dd875baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:16:59.552 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 podman[31594]: 2024-05-27 07:16:59.550682181 +0000 UTC m=+0.374266655 container init 07e5df3f996bf0093f4354423418f98e8af659f9fec288bce0b1e9d8dd875baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-27T07:16:59.653 INFO:teuthology.orchestra.run.smithi063.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-27T07:16:59.807 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 podman[31594]: 2024-05-27 07:16:59.561494186 +0000 UTC m=+0.385078579 container start 07e5df3f996bf0093f4354423418f98e8af659f9fec288bce0b1e9d8dd875baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:16:59.807 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 bash[31594]: 07e5df3f996bf0093f4354423418f98e8af659f9fec288bce0b1e9d8dd875baf 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 systemd[1]: Started Ceph mon.a for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev), process ceph-mon, pid 2 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: pidfile_write: ignore empty --pid-file 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: load: jerasure load: lrc 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: RocksDB version: 7.9.2 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Git sha 0 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Compile date 2024-05-21 07:00:12 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: DB SUMMARY 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: DB Session ID: I3RTN3LY6PIZZF5WXW52 2024-05-27T07:16:59.808 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: CURRENT file: CURRENT 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104460 ; 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.error_if_exists: 0 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.create_if_missing: 0 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.paranoid_checks: 1 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.env: 0x556db9e46500 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T07:16:59.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.info_log: 0x556dbc40a040 2024-05-27T07:16:59.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T07:16:59.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.statistics: (nil) 2024-05-27T07:16:59.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.use_fsync: 0 2024-05-27T07:16:59.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_log_file_size: 0 2024-05-27T07:16:59.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T07:16:59.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T07:16:59.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T07:16:59.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T07:16:59.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.allow_fallocate: 1 2024-05-27T07:16:59.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T07:16:59.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T07:16:59.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.use_direct_reads: 0 2024-05-27T07:16:59.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.db_log_dir: 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.wal_dir: 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T07:16:59.812 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.write_buffer_manager: 0x556dbc2d48c0 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.unordered_write: 0 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T07:16:59.813 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.row_cache: None 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.wal_filter: None 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.two_write_queues: 0 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.wal_compression: 0 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.atomic_flush: 0 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T07:16:59.814 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.log_readahead_size: 0 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_background_jobs: 2 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_background_compactions: -1 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_subcompactions: 1 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T07:16:59.815 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_open_files: -1 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T07:16:59.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_background_flushes: -1 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Compression algorithms supported: 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: kZSTD supported: 0 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: kXpressCompression supported: 0 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: kZlibCompression supported: 1 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: kSnappyCompression supported: 1 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: kLZ4Compression supported: 1 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: kBZip2Compression supported: 0 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-27T07:16:59.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.merge_operator: 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_filter: None 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556dbc40a200) 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout: cache_index_and_filter_blocks: 1 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T07:16:59.818 INFO:journalctl@ceph.mon.a.smithi063.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: pin_top_level_index_and_filter: 1 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: index_type: 0 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: data_block_index_type: 0 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: index_shortening: 1 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: checksum: 4 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: no_block_cache: 0 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_cache: 0x556dbc3071f0 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_cache_name: BinnedLRUCache 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_cache_options: 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: capacity : 536870912 2024-05-27T07:16:59.819 INFO:journalctl@ceph.mon.a.smithi063.stdout: num_shard_bits : 4 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: strict_capacity_limit : 0 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: high_pri_pool_ratio: 0.000 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_cache_compressed: (nil) 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: persistent_cache: (nil) 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_size: 4096 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_size_deviation: 10 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_restart_interval: 16 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: index_block_restart_interval: 1 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: metadata_block_size: 4096 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: partition_filters: 0 2024-05-27T07:16:59.820 INFO:journalctl@ceph.mon.a.smithi063.stdout: use_delta_encoding: 1 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: filter_policy: bloomfilter 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: whole_key_filtering: 1 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: verify_compression: 0 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: read_amp_bytes_per_bit: 0 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: format_version: 5 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: enable_index_compression: 1 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: block_align: 0 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: max_auto_readahead_size: 262144 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: prepopulate_block_cache: 0 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: initial_auto_readahead_size: 8192 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T07:16:59.821 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T07:16:59.822 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compression: NoCompression 2024-05-27T07:16:59.822 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T07:16:59.822 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T07:16:59.822 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T07:16:59.822 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.num_levels: 7 2024-05-27T07:16:59.822 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T07:16:59.822 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T07:16:59.822 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T07:16:59.823 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T07:16:59.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T07:16:59.825 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T07:16:59.825 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T07:16:59.825 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T07:16:59.825 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T07:16:59.825 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T07:16:59.825 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T07:16:59.825 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T07:16:59.825 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T07:16:59.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T07:16:59.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T07:16:59.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T07:16:59.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T07:16:59.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T07:16:59.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T07:16:59.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T07:16:59.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.inplace_update_support: 0 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T07:16:59.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.bloom_locality: 0 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.max_successive_merges: 0 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.ttl: 2592000 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.enable_blob_files: false 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.min_blob_size: 0 2024-05-27T07:16:59.828 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 44b0eb7f-230a-433a-b05b-969063596ed5 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794219696385, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-27T07:16:59.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794219698361, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99717, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97599, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14020, "raw_average_key_size": 49, "raw_value_size": 89935, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716794219, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "44b0eb7f-230a-433a-b05b-969063596ed5", "db_session_id": "I3RTN3LY6PIZZF5WXW52", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794219698635, "job": 1, "event": "recovery_finished"} 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556dbc2ebc00 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: DB pointer 0x556dc10de000 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout: ** DB Stats ** 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:16:59.830 INFO:journalctl@ceph.mon.a.smithi063.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: ** Compaction Stats [default] ** 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: L0 2/0 99.24 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 84.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: Sum 2/0 99.24 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 84.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 84.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:16:59.831 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: ** Compaction Stats [default] ** 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 84.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T07:16:59.832 INFO:journalctl@ceph.mon.a.smithi063.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout: Cumulative compaction: 0.00 GB write, 13.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout: Interval compaction: 0.00 GB write, 13.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout: Block cache BinnedLRUCache@0x556dbc3071f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: starting mon.a rank 0 at public addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] at bind addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: mon.a@-1(???) e1 preinit fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: mon.a@-1(???).mds e1 new map 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: mon.a@-1(???).mds e1 print_map 2024-05-27T07:16:59.833 INFO:journalctl@ceph.mon.a.smithi063.stdout: e1 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout: legacy client fscid: -1 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout: 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout: No filesystems configured 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: monmap epoch 1 2024-05-27T07:16:59.834 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:16:59.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: last_changed 2024-05-27T07:16:40.949861+0000 2024-05-27T07:16:59.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: created 2024-05-27T07:16:40.949861+0000 2024-05-27T07:16:59.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: min_mon_release 19 (squid) 2024-05-27T07:16:59.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: election_strategy: 1 2024-05-27T07:16:59.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-27T07:16:59.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: fsmap 2024-05-27T07:16:59.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T07:16:59.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:16:59 smithi063 ceph-mon[31613]: mgrmap e1: no daemons active 2024-05-27T07:17:01.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.463+0000 7f5f76800700 1 Processor -- start 2024-05-27T07:17:01.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.463+0000 7f5f76800700 1 -- start start 2024-05-27T07:17:01.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.464+0000 7f5f76800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7010c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:01.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.464+0000 7f5f76800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7010c690 con 0x7f5f7010bc90 2024-05-27T07:17:01.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.465+0000 7f5f6f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7010c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:01.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.465+0000 7f5f6f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7010c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42438/0 (socket says 172.21.15.63:42438) 2024-05-27T07:17:01.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.465+0000 7f5f6f400700 1 -- 172.21.15.63:0/631687728 learned_addr learned my addr 172.21.15.63:0/631687728 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:01.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.466+0000 7f5f6f400700 1 -- 172.21.15.63:0/631687728 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7010ce60 con 0x7f5f7010bc90 2024-05-27T07:17:01.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.466+0000 7f5f6f400700 1 --2- 172.21.15.63:0/631687728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7010c0a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009e80 tx=0x7f5f6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5749a2f5e9e4bac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:01.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.466+0000 7f5f6ea00700 1 -- 172.21.15.63:0/631687728 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f60017070 con 0x7f5f7010bc90 2024-05-27T07:17:01.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.466+0000 7f5f6ea00700 1 -- 172.21.15.63:0/631687728 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5f6000b8c0 con 0x7f5f7010bc90 2024-05-27T07:17:01.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.466+0000 7f5f6ea00700 1 -- 172.21.15.63:0/631687728 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f60004420 con 0x7f5f7010bc90 2024-05-27T07:17:01.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.467+0000 7f5f76800700 1 -- 172.21.15.63:0/631687728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 msgr2=0x7f5f7010c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:01.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.467+0000 7f5f76800700 1 --2- 172.21.15.63:0/631687728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7010c0a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009e80 tx=0x7f5f6000b330 comp rx=0 tx=0).stop 2024-05-27T07:17:01.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.467+0000 7f5f76800700 1 -- 172.21.15.63:0/631687728 shutdown_connections 2024-05-27T07:17:01.210 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.467+0000 7f5f76800700 1 --2- 172.21.15.63:0/631687728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7010c0a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:01.210 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.467+0000 7f5f76800700 1 -- 172.21.15.63:0/631687728 >> 172.21.15.63:0/631687728 conn(0x7f5f701074b0 msgr2=0x7f5f701098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:01.210 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.468+0000 7f5f76800700 1 -- 172.21.15.63:0/631687728 shutdown_connections 2024-05-27T07:17:01.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.468+0000 7f5f76800700 1 -- 172.21.15.63:0/631687728 wait complete. 2024-05-27T07:17:01.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.469+0000 7f5f76800700 1 Processor -- start 2024-05-27T07:17:01.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.469+0000 7f5f76800700 1 -- start start 2024-05-27T07:17:01.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.469+0000 7f5f76800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7019ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:01.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.469+0000 7f5f76800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5f70110210 con 0x7f5f7010bc90 2024-05-27T07:17:01.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.469+0000 7f5f6f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7019ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:01.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.469+0000 7f5f6f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7019ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42446/0 (socket says 172.21.15.63:42446) 2024-05-27T07:17:01.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.469+0000 7f5f6f400700 1 -- 172.21.15.63:0/4193716146 learned_addr learned my addr 172.21.15.63:0/4193716146 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:01.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.470+0000 7f5f6f400700 1 -- 172.21.15.63:0/4193716146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f701a0500 con 0x7f5f7010bc90 2024-05-27T07:17:01.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.470+0000 7f5f6f400700 1 --2- 172.21.15.63:0/4193716146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7019ffc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5f60004110 tx=0x7f5f60009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:01.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.471+0000 7f5f6cc00700 1 -- 172.21.15.63:0/4193716146 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f60020020 con 0x7f5f7010bc90 2024-05-27T07:17:01.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.471+0000 7f5f76800700 1 -- 172.21.15.63:0/4193716146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f701a0790 con 0x7f5f7010bc90 2024-05-27T07:17:01.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.471+0000 7f5f6cc00700 1 -- 172.21.15.63:0/4193716146 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5f60003df0 con 0x7f5f7010bc90 2024-05-27T07:17:01.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.471+0000 7f5f6cc00700 1 -- 172.21.15.63:0/4193716146 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f60017040 con 0x7f5f7010bc90 2024-05-27T07:17:01.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.471+0000 7f5f76800700 1 -- 172.21.15.63:0/4193716146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f701a33a0 con 0x7f5f7010bc90 2024-05-27T07:17:01.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.471+0000 7f5f6cc00700 1 -- 172.21.15.63:0/4193716146 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5f60023020 con 0x7f5f7010bc90 2024-05-27T07:17:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.471+0000 7f5f6cc00700 1 -- 172.21.15.63:0/4193716146 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f6000f050 con 0x7f5f7010bc90 2024-05-27T07:17:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.472+0000 7f5f76800700 1 -- 172.21.15.63:0/4193716146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f70064e10 con 0x7f5f7010bc90 2024-05-27T07:17:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.476+0000 7f5f6cc00700 1 -- 172.21.15.63:0/4193716146 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f5f6002a020 con 0x7f5f7010bc90 2024-05-27T07:17:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.555+0000 7f5f76800700 1 -- 172.21.15.63:0/4193716146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f5f701a3570 con 0x7f5f7010bc90 2024-05-27T07:17:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.560+0000 7f5f6cc00700 1 -- 172.21.15.63:0/4193716146 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f5f60012b30 con 0x7f5f7010bc90 2024-05-27T07:17:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.561+0000 7f5f76800700 1 -- 172.21.15.63:0/4193716146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 msgr2=0x7f5f7019ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.561+0000 7f5f76800700 1 --2- 172.21.15.63:0/4193716146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7019ffc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5f60004110 tx=0x7f5f60009390 comp rx=0 tx=0).stop 2024-05-27T07:17:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.562+0000 7f5f76800700 1 -- 172.21.15.63:0/4193716146 shutdown_connections 2024-05-27T07:17:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.562+0000 7f5f76800700 1 --2- 172.21.15.63:0/4193716146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f7010bc90 0x7f5f7019ffc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.562+0000 7f5f76800700 1 -- 172.21.15.63:0/4193716146 >> 172.21.15.63:0/4193716146 conn(0x7f5f701074b0 msgr2=0x7f5f70108d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:01.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.562+0000 7f5f76800700 1 -- 172.21.15.63:0/4193716146 shutdown_connections 2024-05-27T07:17:01.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:00.562+0000 7f5f76800700 1 -- 172.21.15.63:0/4193716146 wait complete. 2024-05-27T07:17:01.216 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-27T07:17:01.255 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:17:01.255 INFO:teuthology.orchestra.run.smithi063.stdout:Creating mgr... 2024-05-27T07:17:01.256 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-27T07:17:01.256 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-27T07:17:01.696 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.x 2024-05-27T07:17:01.696 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-05-27T07:17:01.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:01 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4193716146' entity='client.admin' 2024-05-27T07:17:02.031 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968.target.wants/ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@.service. 2024-05-27T07:17:02.679 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:02 smithi063 podman[31837]: 2024-05-27 07:17:02.434318727 +0000 UTC m=+0.132367030 container create 806b8430f4eba2e34aa114f6aa324bd722553db93281543fa4b6127642aadb06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:17:02.680 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:02 smithi063 podman[31837]: 2024-05-27 07:17:02.68805536 +0000 UTC m=+0.386103686 container init 806b8430f4eba2e34aa114f6aa324bd722553db93281543fa4b6127642aadb06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS) 2024-05-27T07:17:02.820 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T07:17:02.820 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-27T07:17:02.837 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T07:17:02.837 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-27T07:17:02.837 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-27T07:17:02.837 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T07:17:02.855 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T07:17:02.856 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-27T07:17:02.872 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T07:17:02.872 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-27T07:17:02.872 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-27T07:17:02.872 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-27T07:17:02.872 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr to start... 2024-05-27T07:17:02.873 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr... 2024-05-27T07:17:02.956 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:02 smithi063 podman[31837]: 2024-05-27 07:17:02.697549795 +0000 UTC m=+0.395598083 container start 806b8430f4eba2e34aa114f6aa324bd722553db93281543fa4b6127642aadb06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-27T07:17:02.956 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:02 smithi063 bash[31837]: 806b8430f4eba2e34aa114f6aa324bd722553db93281543fa4b6127642aadb06 2024-05-27T07:17:02.956 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:02 smithi063 systemd[1]: Started Ceph mgr.x for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:17:03.253 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:03 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:03.001+0000 7f243e7b8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:17:03.253 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:03 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:03.261+0000 7f243e7b8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:17:03.549 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:03 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:03.400+0000 7f243e7b8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:17:03.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:03 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4092937151' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T07:17:04.113 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:03 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:03.845+0000 7f243e7b8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:17:04.113 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:04 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:04.001+0000 7f243e7b8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:17:04.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:17:04.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:17:04.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:17:04.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T07:17:04.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T07:17:04.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T07:17:04.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T07:17:04.396 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T07:17:04.397 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:04.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T07:17:04.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:04.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T07:17:04.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T07:17:04.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T07:17:04.405 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T07:17:04.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T07:17:04.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T07:16:52.282619+0000", 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T07:17:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.558+0000 7fef2e400700 1 Processor -- start 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.558+0000 7fef2e400700 1 -- start start 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.558+0000 7fef2e400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef28108d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.558+0000 7fef2e400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef28109380 con 0x7fef28108980 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.559+0000 7fef27400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef28108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.559+0000 7fef27400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef28108d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42480/0 (socket says 172.21.15.63:42480) 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.559+0000 7fef27400700 1 -- 172.21.15.63:0/3474845290 learned_addr learned my addr 172.21.15.63:0/3474845290 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.560+0000 7fef27400700 1 -- 172.21.15.63:0/3474845290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef28109b50 con 0x7fef28108980 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.560+0000 7fef27400700 1 --2- 172.21.15.63:0/3474845290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef28108d90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fef1800bb40 tx=0x7fef1800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e8643ae07aa65f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.560+0000 7fef26a00700 1 -- 172.21.15.63:0/3474845290 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef1800dbe0 con 0x7fef28108980 2024-05-27T07:17:04.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.560+0000 7fef26a00700 1 -- 172.21.15.63:0/3474845290 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fef180044a0 con 0x7fef28108980 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.560+0000 7fef26a00700 1 -- 172.21.15.63:0/3474845290 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef18003e30 con 0x7fef28108980 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.561+0000 7fef2e400700 1 -- 172.21.15.63:0/3474845290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 msgr2=0x7fef28108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.561+0000 7fef2e400700 1 --2- 172.21.15.63:0/3474845290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef28108d90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fef1800bb40 tx=0x7fef1800be50 comp rx=0 tx=0).stop 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.561+0000 7fef2e400700 1 -- 172.21.15.63:0/3474845290 shutdown_connections 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.561+0000 7fef2e400700 1 --2- 172.21.15.63:0/3474845290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef28108d90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.561+0000 7fef2e400700 1 -- 172.21.15.63:0/3474845290 >> 172.21.15.63:0/3474845290 conn(0x7fef28104060 msgr2=0x7fef281064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.562+0000 7fef2e400700 1 -- 172.21.15.63:0/3474845290 shutdown_connections 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.562+0000 7fef2e400700 1 -- 172.21.15.63:0/3474845290 wait complete. 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.563+0000 7fef2e400700 1 Processor -- start 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.563+0000 7fef2e400700 1 -- start start 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.563+0000 7fef2e400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef2819ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.564+0000 7fef2e400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef2807bfc0 con 0x7fef28108980 2024-05-27T07:17:04.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.564+0000 7fef27400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef2819ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.564+0000 7fef27400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef2819ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42482/0 (socket says 172.21.15.63:42482) 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.564+0000 7fef27400700 1 -- 172.21.15.63:0/4092937151 learned_addr learned my addr 172.21.15.63:0/4092937151 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.564+0000 7fef27400700 1 -- 172.21.15.63:0/4092937151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef281a04a0 con 0x7fef28108980 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.565+0000 7fef27400700 1 --2- 172.21.15.63:0/4092937151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef2819ff60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fef1800b340 tx=0x7fef18003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.565+0000 7fef24c00700 1 -- 172.21.15.63:0/4092937151 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef18010580 con 0x7fef28108980 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.565+0000 7fef24c00700 1 -- 172.21.15.63:0/4092937151 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fef1801d070 con 0x7fef28108980 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.565+0000 7fef2e400700 1 -- 172.21.15.63:0/4092937151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef281a0730 con 0x7fef28108980 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.565+0000 7fef24c00700 1 -- 172.21.15.63:0/4092937151 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef180219b0 con 0x7fef28108980 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.565+0000 7fef2e400700 1 -- 172.21.15.63:0/4092937151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef281a3340 con 0x7fef28108980 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.566+0000 7fef24c00700 1 -- 172.21.15.63:0/4092937151 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fef1801f030 con 0x7fef28108980 2024-05-27T07:17:04.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.566+0000 7fef24c00700 1 -- 172.21.15.63:0/4092937151 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fef180193f0 con 0x7fef28108980 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.567+0000 7fef2e400700 1 -- 172.21.15.63:0/4092937151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef28064e10 con 0x7fef28108980 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.570+0000 7fef24c00700 1 -- 172.21.15.63:0/4092937151 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fef18015070 con 0x7fef28108980 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.650+0000 7fef2e400700 1 -- 172.21.15.63:0/4092937151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fef28194db0 con 0x7fef28108980 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.651+0000 7fef24c00700 1 -- 172.21.15.63:0/4092937151 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fef18021ce0 con 0x7fef28108980 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.653+0000 7fef2e400700 1 -- 172.21.15.63:0/4092937151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 msgr2=0x7fef2819ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.653+0000 7fef2e400700 1 --2- 172.21.15.63:0/4092937151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef2819ff60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fef1800b340 tx=0x7fef18003f80 comp rx=0 tx=0).stop 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.653+0000 7fef2e400700 1 -- 172.21.15.63:0/4092937151 shutdown_connections 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.653+0000 7fef2e400700 1 --2- 172.21.15.63:0/4092937151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef28108980 0x7fef2819ff60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.653+0000 7fef2e400700 1 -- 172.21.15.63:0/4092937151 >> 172.21.15.63:0/4092937151 conn(0x7fef28104060 msgr2=0x7fef28191e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.653+0000 7fef2e400700 1 -- 172.21.15.63:0/4092937151 shutdown_connections 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:03.653+0000 7fef2e400700 1 -- 172.21.15.63:0/4092937151 wait complete. 2024-05-27T07:17:04.412 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (1/15)... 2024-05-27T07:17:05.956 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:05 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:05.666+0000 7f243e7b8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:17:05.956 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:05 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:05.902+0000 7f243e7b8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:17:06.455 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:06 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:06.121+0000 7f243e7b8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:17:06.814 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:06 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:06.519+0000 7f243e7b8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:17:06.814 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:06 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:06.648+0000 7f243e7b8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:17:07.134 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:06 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:06.918+0000 7f243e7b8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:17:07.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:07 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1015386540' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T07:17:07.456 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:07 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:07.202+0000 7f243e7b8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:17:07.775 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-27T07:17:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T07:17:07.777 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T07:17:07.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T07:17:07.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T07:17:07.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T07:17:07.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T07:17:07.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:07.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:07.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T07:16:52.282619+0000", 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.051+0000 7f57cb800700 1 Processor -- start 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.051+0000 7f57cb800700 1 -- start start 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.052+0000 7f57cb800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c4109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.052+0000 7f57cb800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57c410a4a0 con 0x7f57c4109aa0 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.052+0000 7f57c8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c4109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.053+0000 7f57c8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c4109eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42496/0 (socket says 172.21.15.63:42496) 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.053+0000 7f57c8e00700 1 -- 172.21.15.63:0/3207858482 learned_addr learned my addr 172.21.15.63:0/3207858482 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:07.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.053+0000 7f57c8e00700 1 -- 172.21.15.63:0/3207858482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c410ac70 con 0x7f57c4109aa0 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.053+0000 7f57c8e00700 1 --2- 172.21.15.63:0/3207858482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c4109eb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f57b80098a0 tx=0x7f57b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b38a8271c6e988f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.054+0000 7f57c3400700 1 -- 172.21.15.63:0/3207858482 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57b8004160 con 0x7f57c4109aa0 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.054+0000 7f57c3400700 1 -- 172.21.15.63:0/3207858482 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f57b8004300 con 0x7f57c4109aa0 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.054+0000 7f57c3400700 1 -- 172.21.15.63:0/3207858482 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57b80119e0 con 0x7f57c4109aa0 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.055+0000 7f57cb800700 1 -- 172.21.15.63:0/3207858482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 msgr2=0x7f57c4109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.055+0000 7f57cb800700 1 --2- 172.21.15.63:0/3207858482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c4109eb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f57b80098a0 tx=0x7f57b8009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.055+0000 7f57cb800700 1 -- 172.21.15.63:0/3207858482 shutdown_connections 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.055+0000 7f57cb800700 1 --2- 172.21.15.63:0/3207858482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c4109eb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.055+0000 7f57cb800700 1 -- 172.21.15.63:0/3207858482 >> 172.21.15.63:0/3207858482 conn(0x7f57c4105120 msgr2=0x7f57c4107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.055+0000 7f57cb800700 1 -- 172.21.15.63:0/3207858482 shutdown_connections 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.055+0000 7f57cb800700 1 -- 172.21.15.63:0/3207858482 wait complete. 2024-05-27T07:17:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.056+0000 7f57cb800700 1 Processor -- start 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.056+0000 7f57cb800700 1 -- start start 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.057+0000 7f57cb800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.057+0000 7f57cb800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57c410e020 con 0x7f57c4109aa0 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.057+0000 7f57c8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.057+0000 7f57c8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42502/0 (socket says 172.21.15.63:42502) 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.057+0000 7f57c8e00700 1 -- 172.21.15.63:0/1015386540 learned_addr learned my addr 172.21.15.63:0/1015386540 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.057+0000 7f57c8e00700 1 -- 172.21.15.63:0/1015386540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c419e340 con 0x7f57c4109aa0 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.058+0000 7f57c8e00700 1 --2- 172.21.15.63:0/1015386540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c419de00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57b80119b0 tx=0x7f57b8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.058+0000 7f57c1600700 1 -- 172.21.15.63:0/1015386540 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57b8003970 con 0x7f57c4109aa0 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.058+0000 7f57c1600700 1 -- 172.21.15.63:0/1015386540 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f57b8003b10 con 0x7f57c4109aa0 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.058+0000 7f57cb800700 1 -- 172.21.15.63:0/1015386540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c419e5d0 con 0x7f57c4109aa0 2024-05-27T07:17:07.790 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.058+0000 7f57c1600700 1 -- 172.21.15.63:0/1015386540 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57b8024610 con 0x7f57c4109aa0 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.058+0000 7f57cb800700 1 -- 172.21.15.63:0/1015386540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c41a11e0 con 0x7f57c4109aa0 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.060+0000 7f57c1600700 1 -- 172.21.15.63:0/1015386540 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f57b80248d0 con 0x7f57c4109aa0 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.060+0000 7f57c1600700 1 -- 172.21.15.63:0/1015386540 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f57b802ddb0 con 0x7f57c4109aa0 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.060+0000 7f57cb800700 1 -- 172.21.15.63:0/1015386540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c4064e10 con 0x7f57c4109aa0 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.062+0000 7f57c1600700 1 -- 172.21.15.63:0/1015386540 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f57b801a7c0 con 0x7f57c4109aa0 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.142+0000 7f57cb800700 1 -- 172.21.15.63:0/1015386540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f57c41a13b0 con 0x7f57c4109aa0 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.143+0000 7f57c1600700 1 -- 172.21.15.63:0/1015386540 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f57b801aae0 con 0x7f57c4109aa0 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.144+0000 7f57cb800700 1 -- 172.21.15.63:0/1015386540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 msgr2=0x7f57c419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.144+0000 7f57cb800700 1 --2- 172.21.15.63:0/1015386540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c419de00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57b80119b0 tx=0x7f57b8011a90 comp rx=0 tx=0).stop 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.145+0000 7f57cb800700 1 -- 172.21.15.63:0/1015386540 shutdown_connections 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.145+0000 7f57cb800700 1 --2- 172.21.15.63:0/1015386540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c4109aa0 0x7f57c419de00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.145+0000 7f57cb800700 1 -- 172.21.15.63:0/1015386540 >> 172.21.15.63:0/1015386540 conn(0x7f57c4105120 msgr2=0x7f57c41059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:07.791 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.145+0000 7f57cb800700 1 -- 172.21.15.63:0/1015386540 shutdown_connections 2024-05-27T07:17:07.792 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:07.145+0000 7f57cb800700 1 -- 172.21.15.63:0/1015386540 wait complete. 2024-05-27T07:17:07.792 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (2/15)... 2024-05-27T07:17:08.326 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:08 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:08.035+0000 7f243e7b8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:17:08.326 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:08 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:08.334+0000 7f243e7b8200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:17:09.206 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:08 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:08.912+0000 7f243e7b8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:17:09.206 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:09 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:09.044+0000 7f243e7b8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:17:09.942 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:09 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:09.544+0000 7f243e7b8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:17:10.206 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:10 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:10.033+0000 7f243e7b8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:17:10.206 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:10 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:10.165+0000 7f243e7b8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:17:10.833 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:10 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2510100910' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T07:17:10.833 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:10 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:10.506+0000 7f243e7b8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:17:10.833 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:10 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:10.708+0000 7f243e7b8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:17:10.833 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:10 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:10.840+0000 7f243e7b8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:17:11.260 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:17:11.260 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:17:11.260 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:11.261 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-27T07:17:11.262 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T07:17:11.262 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:11.262 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T07:17:11.262 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T07:17:11.262 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:11.262 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T07:17:11.262 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:11.262 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T07:17:11.262 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T07:17:11.262 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T07:17:11.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T07:17:11.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T07:17:11.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:11.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:11.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T07:16:52.282619+0000", 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.471+0000 7fb459600700 1 Processor -- start 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.471+0000 7fb459600700 1 -- start start 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.471+0000 7fb459600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb45407d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.471+0000 7fb459600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb45407eff0 con 0x7fb45407d250 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.472+0000 7fb452a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb45407d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:11.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.472+0000 7fb452a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb45407d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52182/0 (socket says 172.21.15.63:52182) 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.472+0000 7fb452a00700 1 -- 172.21.15.63:0/697755961 learned_addr learned my addr 172.21.15.63:0/697755961 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.472+0000 7fb452a00700 1 -- 172.21.15.63:0/697755961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45407dba0 con 0x7fb45407d250 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.472+0000 7fb452a00700 1 --2- 172.21.15.63:0/697755961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb45407d660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb4440098a0 tx=0x7fb444009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61d086fffb8885e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.473+0000 7fb451600700 1 -- 172.21.15.63:0/697755961 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb444004160 con 0x7fb45407d250 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.473+0000 7fb451600700 1 -- 172.21.15.63:0/697755961 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb444004300 con 0x7fb45407d250 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.473+0000 7fb451600700 1 -- 172.21.15.63:0/697755961 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4440119e0 con 0x7fb45407d250 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.474+0000 7fb459600700 1 -- 172.21.15.63:0/697755961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 msgr2=0x7fb45407d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.474+0000 7fb459600700 1 --2- 172.21.15.63:0/697755961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb45407d660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb4440098a0 tx=0x7fb444009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.474+0000 7fb459600700 1 -- 172.21.15.63:0/697755961 shutdown_connections 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.474+0000 7fb459600700 1 --2- 172.21.15.63:0/697755961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb45407d660 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.474+0000 7fb459600700 1 -- 172.21.15.63:0/697755961 >> 172.21.15.63:0/697755961 conn(0x7fb4541050e0 msgr2=0x7fb4541075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.474+0000 7fb459600700 1 -- 172.21.15.63:0/697755961 shutdown_connections 2024-05-27T07:17:11.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.474+0000 7fb459600700 1 -- 172.21.15.63:0/697755961 wait complete. 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.475+0000 7fb459600700 1 Processor -- start 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.476+0000 7fb459600700 1 -- start start 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.476+0000 7fb459600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb4541a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.476+0000 7fb459600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb454110140 con 0x7fb45407d250 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.476+0000 7fb452a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb4541a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.476+0000 7fb452a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb4541a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52196/0 (socket says 172.21.15.63:52196) 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.476+0000 7fb452a00700 1 -- 172.21.15.63:0/2510100910 learned_addr learned my addr 172.21.15.63:0/2510100910 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.477+0000 7fb452a00700 1 -- 172.21.15.63:0/2510100910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4541a05b0 con 0x7fb45407d250 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.477+0000 7fb452a00700 1 --2- 172.21.15.63:0/2510100910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb4541a0070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4440116a0 tx=0x7fb444011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.478+0000 7fb44b400700 1 -- 172.21.15.63:0/2510100910 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4440039f0 con 0x7fb45407d250 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.478+0000 7fb459600700 1 -- 172.21.15.63:0/2510100910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4541a0840 con 0x7fb45407d250 2024-05-27T07:17:11.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.478+0000 7fb44b400700 1 -- 172.21.15.63:0/2510100910 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb444003b90 con 0x7fb45407d250 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.478+0000 7fb44b400700 1 -- 172.21.15.63:0/2510100910 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb44401a600 con 0x7fb45407d250 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.478+0000 7fb459600700 1 -- 172.21.15.63:0/2510100910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4541a3450 con 0x7fb45407d250 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.478+0000 7fb44b400700 1 -- 172.21.15.63:0/2510100910 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb44401ac60 con 0x7fb45407d250 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.478+0000 7fb44b400700 1 -- 172.21.15.63:0/2510100910 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb444019700 con 0x7fb45407d250 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.479+0000 7fb459600700 1 -- 172.21.15.63:0/2510100910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb454064e10 con 0x7fb45407d250 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.482+0000 7fb44b400700 1 -- 172.21.15.63:0/2510100910 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb444024a50 con 0x7fb45407d250 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.560+0000 7fb459600700 1 -- 172.21.15.63:0/2510100910 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb4541a39f0 con 0x7fb45407d250 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.561+0000 7fb44b400700 1 -- 172.21.15.63:0/2510100910 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fb444024430 con 0x7fb45407d250 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.563+0000 7fb459600700 1 -- 172.21.15.63:0/2510100910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 msgr2=0x7fb4541a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.563+0000 7fb459600700 1 --2- 172.21.15.63:0/2510100910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb4541a0070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4440116a0 tx=0x7fb444011a60 comp rx=0 tx=0).stop 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.563+0000 7fb459600700 1 -- 172.21.15.63:0/2510100910 shutdown_connections 2024-05-27T07:17:11.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.563+0000 7fb459600700 1 --2- 172.21.15.63:0/2510100910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb45407d250 0x7fb4541a0070 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.563+0000 7fb459600700 1 -- 172.21.15.63:0/2510100910 >> 172.21.15.63:0/2510100910 conn(0x7fb4541050e0 msgr2=0x7fb454105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.563+0000 7fb459600700 1 -- 172.21.15.63:0/2510100910 shutdown_connections 2024-05-27T07:17:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:10.563+0000 7fb459600700 1 -- 172.21.15.63:0/2510100910 wait complete. 2024-05-27T07:17:11.276 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (3/15)... 2024-05-27T07:17:11.706 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:11.254+0000 7f243e7b8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:17:11.706 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:11.385+0000 7f243e7b8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:17:12.456 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:12 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:12.176+0000 7f243e7b8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:17:12.956 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:12 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:12.478+0000 7f243e7b8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:17:12.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: Activating manager daemon x 2024-05-27T07:17:12.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: mgrmap e2: x(active, starting, since 0.0127593s) 2024-05-27T07:17:12.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: from='mgr.14100 172.21.15.63:0/843024709' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:17:12.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: from='mgr.14100 172.21.15.63:0/843024709' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:17:12.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: from='mgr.14100 172.21.15.63:0/843024709' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:17:12.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: from='mgr.14100 172.21.15.63:0/843024709' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:17:12.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: from='mgr.14100 172.21.15.63:0/843024709' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:17:12.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: Manager daemon x is now available 2024-05-27T07:17:12.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: from='mgr.14100 172.21.15.63:0/843024709' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:17:12.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: from='mgr.14100 172.21.15.63:0/843024709' entity='mgr.x' 2024-05-27T07:17:12.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:12 smithi063 ceph-mon[31613]: from='mgr.14100 172.21.15.63:0/843024709' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:17:13.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:13 smithi063 ceph-mon[31613]: from='mgr.14100 172.21.15.63:0/843024709' entity='mgr.x' 2024-05-27T07:17:13.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:13 smithi063 ceph-mon[31613]: from='mgr.14100 172.21.15.63:0/843024709' entity='mgr.x' 2024-05-27T07:17:13.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:13 smithi063 ceph-mon[31613]: mgrmap e3: x(active, since 1.0274s) 2024-05-27T07:17:14.649 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:14 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3081420835' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T07:17:14.967 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:17:14.967 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:17:14.967 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:17:14.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T07:17:14.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T07:17:14.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T07:17:14.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T07:17:14.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:14.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T07:17:14.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T07:17:14.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-05-27T07:17:14.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:14.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T07:17:14.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T07:17:14.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:14.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-27T07:17:14.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T07:17:14.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:14.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T07:17:14.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T07:17:14.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:14.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T07:17:14.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:14.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T07:17:14.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T07:17:14.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T07:17:14.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T07:17:14.974 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T07:17:14.974 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T07:17:14.974 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:14.974 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T07:17:14.974 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T07:17:14.974 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T07:17:14.974 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T07:17:14.975 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T07:17:14.975 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T07:17:14.975 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T07:17:14.975 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T07:17:14.975 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T07:17:14.975 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:14.975 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T07:17:14.975 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:14.975 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T07:17:14.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T07:17:14.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:14.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T07:17:14.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T07:17:14.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T07:17:14.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T07:17:14.977 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T07:16:52.282619+0000", 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T07:17:14.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:17:14.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.967+0000 7fc61f200700 1 Processor -- start 2024-05-27T07:17:14.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.967+0000 7fc61f200700 1 -- start start 2024-05-27T07:17:14.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.968+0000 7fc61f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62007d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:14.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.968+0000 7fc61f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc62007d750 con 0x7fc62007ede0 2024-05-27T07:17:14.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.968+0000 7fc61de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62007d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:14.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.968+0000 7fc61de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62007d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52288/0 (socket says 172.21.15.63:52288) 2024-05-27T07:17:14.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.968+0000 7fc61de00700 1 -- 172.21.15.63:0/2935363334 learned_addr learned my addr 172.21.15.63:0/2935363334 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:14.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.969+0000 7fc61de00700 1 -- 172.21.15.63:0/2935363334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62007d8d0 con 0x7fc62007ede0 2024-05-27T07:17:14.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.969+0000 7fc61de00700 1 --2- 172.21.15.63:0/2935363334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62007d210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc6100098a0 tx=0x7fc610009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344b38e09bda0b55 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:14.981 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.969+0000 7fc61ca00700 1 -- 172.21.15.63:0/2935363334 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc610004160 con 0x7fc62007ede0 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.969+0000 7fc61ca00700 1 -- 172.21.15.63:0/2935363334 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc610004300 con 0x7fc62007ede0 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.969+0000 7fc61ca00700 1 -- 172.21.15.63:0/2935363334 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6100119e0 con 0x7fc62007ede0 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.970+0000 7fc61f200700 1 -- 172.21.15.63:0/2935363334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 msgr2=0x7fc62007d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.970+0000 7fc61f200700 1 --2- 172.21.15.63:0/2935363334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62007d210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc6100098a0 tx=0x7fc610009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.970+0000 7fc61f200700 1 -- 172.21.15.63:0/2935363334 shutdown_connections 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.970+0000 7fc61f200700 1 --2- 172.21.15.63:0/2935363334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62007d210 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.970+0000 7fc61f200700 1 -- 172.21.15.63:0/2935363334 >> 172.21.15.63:0/2935363334 conn(0x7fc620104f40 msgr2=0x7fc620107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.970+0000 7fc61f200700 1 -- 172.21.15.63:0/2935363334 shutdown_connections 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.970+0000 7fc61f200700 1 -- 172.21.15.63:0/2935363334 wait complete. 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.972+0000 7fc61f200700 1 Processor -- start 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.972+0000 7fc61f200700 1 -- start start 2024-05-27T07:17:14.982 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.972+0000 7fc61f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:14.983 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.972+0000 7fc61f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010fe40 con 0x7fc62007ede0 2024-05-27T07:17:14.985 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.972+0000 7fc61de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:14.985 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.972+0000 7fc61de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52298/0 (socket says 172.21.15.63:52298) 2024-05-27T07:17:14.985 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.972+0000 7fc61de00700 1 -- 172.21.15.63:0/3081420835 learned_addr learned my addr 172.21.15.63:0/3081420835 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:14.985 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.973+0000 7fc61de00700 1 -- 172.21.15.63:0/3081420835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6201a0230 con 0x7fc62007ede0 2024-05-27T07:17:14.985 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.973+0000 7fc61de00700 1 --2- 172.21.15.63:0/3081420835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62019fcf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6100116a0 tx=0x7fc610011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:14.985 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.973+0000 7fc616a00700 1 -- 172.21.15.63:0/3081420835 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc610003970 con 0x7fc62007ede0 2024-05-27T07:17:14.985 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.973+0000 7fc616a00700 1 -- 172.21.15.63:0/3081420835 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc610003b10 con 0x7fc62007ede0 2024-05-27T07:17:14.985 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.973+0000 7fc616a00700 1 -- 172.21.15.63:0/3081420835 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc610019550 con 0x7fc62007ede0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.973+0000 7fc61f200700 1 -- 172.21.15.63:0/3081420835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6201a04c0 con 0x7fc62007ede0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.974+0000 7fc61f200700 1 -- 172.21.15.63:0/3081420835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201a30d0 con 0x7fc62007ede0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.975+0000 7fc61f200700 1 -- 172.21.15.63:0/3081420835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc620064e10 con 0x7fc62007ede0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.975+0000 7fc616a00700 1 -- 172.21.15.63:0/3081420835 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7fc61001a430 con 0x7fc62007ede0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.976+0000 7fc616a00700 1 --2- 172.21.15.63:0/3081420835 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fc60403e410 0x7fc604040920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.976+0000 7fc616a00700 1 -- 172.21.15.63:0/3081420835 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc610053580 con 0x7fc62007ede0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.977+0000 7fc61d400700 1 --2- 172.21.15.63:0/3081420835 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fc60403e410 0x7fc604040920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.978+0000 7fc61d400700 1 --2- 172.21.15.63:0/3081420835 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fc60403e410 0x7fc604040920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc60c006fd0 tx=0x7fc60c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:13.983+0000 7fc616a00700 1 -- 172.21.15.63:0/3081420835 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc61000bcc0 con 0x7fc62007ede0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.312+0000 7fc61f200700 1 -- 172.21.15.63:0/3081420835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc6201a32f0 con 0x7fc62007ede0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.313+0000 7fc616a00700 1 -- 172.21.15.63:0/3081420835 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc6201a32f0 con 0x7fc62007ede0 2024-05-27T07:17:14.986 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.316+0000 7fc61f200700 1 -- 172.21.15.63:0/3081420835 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fc60403e410 msgr2=0x7fc604040920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:14.987 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.316+0000 7fc61f200700 1 --2- 172.21.15.63:0/3081420835 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fc60403e410 0x7fc604040920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc60c006fd0 tx=0x7fc60c006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:14.987 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.316+0000 7fc61f200700 1 -- 172.21.15.63:0/3081420835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 msgr2=0x7fc62019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:14.987 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.316+0000 7fc61f200700 1 --2- 172.21.15.63:0/3081420835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62019fcf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6100116a0 tx=0x7fc610011a60 comp rx=0 tx=0).stop 2024-05-27T07:17:14.987 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.317+0000 7fc61f200700 1 -- 172.21.15.63:0/3081420835 shutdown_connections 2024-05-27T07:17:14.987 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.317+0000 7fc61f200700 1 --2- 172.21.15.63:0/3081420835 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fc60403e410 0x7fc604040920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:14.988 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.317+0000 7fc61f200700 1 --2- 172.21.15.63:0/3081420835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc62007ede0 0x7fc62019fcf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:14.988 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.317+0000 7fc61f200700 1 -- 172.21.15.63:0/3081420835 >> 172.21.15.63:0/3081420835 conn(0x7fc620104f40 msgr2=0x7fc620105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:14.988 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.317+0000 7fc61f200700 1 -- 172.21.15.63:0/3081420835 shutdown_connections 2024-05-27T07:17:14.988 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:14.317+0000 7fc61f200700 1 -- 172.21.15.63:0/3081420835 wait complete. 2024-05-27T07:17:14.988 INFO:teuthology.orchestra.run.smithi063.stdout:mgr is available 2024-05-27T07:17:15.668 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:15 smithi063 ceph-mon[31613]: mgrmap e4: x(active, since 2s) 2024-05-27T07:17:16.575 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:16 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/631380820' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T07:17:16.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:17:16.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T07:17:16.583 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout fsid = e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:17:16.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-27T07:17:16.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T07:17:16.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T07:17:16.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T07:17:16.584 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T07:17:16.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:17:16.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T07:17:16.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T07:17:16.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-05-27T07:17:16.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T07:17:16.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-05-27T07:17:16.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T07:17:16.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T07:17:16.586 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T07:17:16.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T07:17:16.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T07:17:16.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.677+0000 7fa5d8a00700 1 Processor -- start 2024-05-27T07:17:16.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.677+0000 7fa5d8a00700 1 -- start start 2024-05-27T07:17:16.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.677+0000 7fa5d8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d4109e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:16.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.677+0000 7fa5d8a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d410a480 con 0x7fa5d4109a80 2024-05-27T07:17:16.588 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.678+0000 7fa5d1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d4109e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:16.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.678+0000 7fa5d1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d4109e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52312/0 (socket says 172.21.15.63:52312) 2024-05-27T07:17:16.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.678+0000 7fa5d1e00700 1 -- 172.21.15.63:0/189683632 learned_addr learned my addr 172.21.15.63:0/189683632 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:16.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.678+0000 7fa5d1e00700 1 -- 172.21.15.63:0/189683632 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d410ac50 con 0x7fa5d4109a80 2024-05-27T07:17:16.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.679+0000 7fa5d1e00700 1 --2- 172.21.15.63:0/189683632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d4109e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40098a0 tx=0x7fa5c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a0318d1bad44c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:16.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.679+0000 7fa5d1400700 1 -- 172.21.15.63:0/189683632 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5c4004160 con 0x7fa5d4109a80 2024-05-27T07:17:16.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.679+0000 7fa5d1400700 1 -- 172.21.15.63:0/189683632 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa5c4004300 con 0x7fa5d4109a80 2024-05-27T07:17:16.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.679+0000 7fa5d1400700 1 -- 172.21.15.63:0/189683632 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5c40119e0 con 0x7fa5d4109a80 2024-05-27T07:17:16.589 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.680+0000 7fa5d8a00700 1 -- 172.21.15.63:0/189683632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 msgr2=0x7fa5d4109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:16.591 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.680+0000 7fa5d8a00700 1 --2- 172.21.15.63:0/189683632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d4109e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40098a0 tx=0x7fa5c4009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:16.591 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.680+0000 7fa5d8a00700 1 -- 172.21.15.63:0/189683632 shutdown_connections 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.680+0000 7fa5d8a00700 1 --2- 172.21.15.63:0/189683632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d4109e90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.680+0000 7fa5d8a00700 1 -- 172.21.15.63:0/189683632 >> 172.21.15.63:0/189683632 conn(0x7fa5d41050c0 msgr2=0x7fa5d4107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.680+0000 7fa5d8a00700 1 -- 172.21.15.63:0/189683632 shutdown_connections 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.680+0000 7fa5d8a00700 1 -- 172.21.15.63:0/189683632 wait complete. 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.681+0000 7fa5d8a00700 1 Processor -- start 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.681+0000 7fa5d8a00700 1 -- start start 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.682+0000 7fa5d8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d419ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.682+0000 7fa5d8a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d410e000 con 0x7fa5d4109a80 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.682+0000 7fa5d1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d419ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.682+0000 7fa5d1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d419ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52328/0 (socket says 172.21.15.63:52328) 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.682+0000 7fa5d1e00700 1 -- 172.21.15.63:0/631380820 learned_addr learned my addr 172.21.15.63:0/631380820 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.683+0000 7fa5d1e00700 1 -- 172.21.15.63:0/631380820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d41a04e0 con 0x7fa5d4109a80 2024-05-27T07:17:16.593 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.683+0000 7fa5d1e00700 1 --2- 172.21.15.63:0/631380820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d419ffa0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009fb0 tx=0x7fa5c4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:16.593 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.683+0000 7fa5caa00700 1 -- 172.21.15.63:0/631380820 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5c40038b0 con 0x7fa5d4109a80 2024-05-27T07:17:16.593 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.683+0000 7fa5caa00700 1 -- 172.21.15.63:0/631380820 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa5c4003a50 con 0x7fa5d4109a80 2024-05-27T07:17:16.593 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.683+0000 7fa5caa00700 1 -- 172.21.15.63:0/631380820 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5c4019550 con 0x7fa5d4109a80 2024-05-27T07:17:16.593 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.683+0000 7fa5d8a00700 1 -- 172.21.15.63:0/631380820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d41a0770 con 0x7fa5d4109a80 2024-05-27T07:17:16.593 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.683+0000 7fa5d8a00700 1 -- 172.21.15.63:0/631380820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d41a3380 con 0x7fa5d4109a80 2024-05-27T07:17:16.593 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.685+0000 7fa5d8a00700 1 -- 172.21.15.63:0/631380820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d4064e10 con 0x7fa5d4109a80 2024-05-27T07:17:16.593 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.685+0000 7fa5caa00700 1 -- 172.21.15.63:0/631380820 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fa5c401a430 con 0x7fa5d4109a80 2024-05-27T07:17:16.595 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.686+0000 7fa5caa00700 1 --2- 172.21.15.63:0/631380820 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fa5b803e4e0 0x7fa5b80409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:16.595 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.686+0000 7fa5caa00700 1 -- 172.21.15.63:0/631380820 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa5c4053aa0 con 0x7fa5d4109a80 2024-05-27T07:17:16.595 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.686+0000 7fa5cbe00700 1 --2- 172.21.15.63:0/631380820 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fa5b803e4e0 0x7fa5b80409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:16.595 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.687+0000 7fa5cbe00700 1 --2- 172.21.15.63:0/631380820 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fa5b803e4e0 0x7fa5b80409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc006fd0 tx=0x7fa5bc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:16.595 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.692+0000 7fa5caa00700 1 -- 172.21.15.63:0/631380820 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa5c40534c0 con 0x7fa5d4109a80 2024-05-27T07:17:16.595 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.928+0000 7fa5d8a00700 1 -- 172.21.15.63:0/631380820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa5d402e8e0 con 0x7fa5d4109a80 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.933+0000 7fa5caa00700 1 -- 172.21.15.63:0/631380820 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+434 (secure 0 0 0) 0x7fa5c40196f0 con 0x7fa5d4109a80 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.937+0000 7fa5d8a00700 1 -- 172.21.15.63:0/631380820 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fa5b803e4e0 msgr2=0x7fa5b80409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.937+0000 7fa5d8a00700 1 --2- 172.21.15.63:0/631380820 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fa5b803e4e0 0x7fa5b80409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc006fd0 tx=0x7fa5bc006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.937+0000 7fa5d8a00700 1 -- 172.21.15.63:0/631380820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 msgr2=0x7fa5d419ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.937+0000 7fa5d8a00700 1 --2- 172.21.15.63:0/631380820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d419ffa0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009fb0 tx=0x7fa5c4011a60 comp rx=0 tx=0).stop 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.937+0000 7fa5d8a00700 1 -- 172.21.15.63:0/631380820 shutdown_connections 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.937+0000 7fa5d8a00700 1 --2- 172.21.15.63:0/631380820 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fa5b803e4e0 0x7fa5b80409f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.937+0000 7fa5d8a00700 1 --2- 172.21.15.63:0/631380820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d4109a80 0x7fa5d419ffa0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.937+0000 7fa5d8a00700 1 -- 172.21.15.63:0/631380820 >> 172.21.15.63:0/631380820 conn(0x7fa5d41050c0 msgr2=0x7fa5d407daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.937+0000 7fa5d8a00700 1 -- 172.21.15.63:0/631380820 shutdown_connections 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:15.938+0000 7fa5d8a00700 1 -- 172.21.15.63:0/631380820 wait complete. 2024-05-27T07:17:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling cephadm module... 2024-05-27T07:17:17.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:17 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2984075155' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-27T07:17:17.956 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:17 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: ignoring --setuser ceph since I am not root 2024-05-27T07:17:17.956 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:17 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: ignoring --setgroup ceph since I am not root 2024-05-27T07:17:18.323 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:18 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:18.069+0000 7ff281931200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:17:18.323 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:18 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:18.331+0000 7ff281931200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:17:18.379 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.329+0000 7fe3e4e00700 1 Processor -- start 2024-05-27T07:17:18.379 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.329+0000 7fe3e4e00700 1 -- start start 2024-05-27T07:17:18.379 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.330+0000 7fe3e4e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e007d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:18.379 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.330+0000 7fe3e4e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e007f030 con 0x7fe3e007d290 2024-05-27T07:17:18.379 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.330+0000 7fe3dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e007d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.331+0000 7fe3dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e007d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52334/0 (socket says 172.21.15.63:52334) 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.331+0000 7fe3dde00700 1 -- 172.21.15.63:0/511967029 learned_addr learned my addr 172.21.15.63:0/511967029 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.331+0000 7fe3dde00700 1 -- 172.21.15.63:0/511967029 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e007dbe0 con 0x7fe3e007d290 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.331+0000 7fe3dde00700 1 --2- 172.21.15.63:0/511967029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e007d6a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00098a0 tx=0x7fe3d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f0b22916c1e8d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.332+0000 7fe3dd400700 1 -- 172.21.15.63:0/511967029 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3d0004160 con 0x7fe3e007d290 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.332+0000 7fe3dd400700 1 -- 172.21.15.63:0/511967029 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe3d0004300 con 0x7fe3e007d290 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.332+0000 7fe3dd400700 1 -- 172.21.15.63:0/511967029 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3d00119e0 con 0x7fe3e007d290 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.333+0000 7fe3e4e00700 1 -- 172.21.15.63:0/511967029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 msgr2=0x7fe3e007d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.333+0000 7fe3e4e00700 1 --2- 172.21.15.63:0/511967029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e007d6a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00098a0 tx=0x7fe3d0009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.333+0000 7fe3e4e00700 1 -- 172.21.15.63:0/511967029 shutdown_connections 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.333+0000 7fe3e4e00700 1 --2- 172.21.15.63:0/511967029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e007d6a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:18.380 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.333+0000 7fe3e4e00700 1 -- 172.21.15.63:0/511967029 >> 172.21.15.63:0/511967029 conn(0x7fe3e0105100 msgr2=0x7fe3e0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.333+0000 7fe3e4e00700 1 -- 172.21.15.63:0/511967029 shutdown_connections 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.333+0000 7fe3e4e00700 1 -- 172.21.15.63:0/511967029 wait complete. 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.334+0000 7fe3e4e00700 1 Processor -- start 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.335+0000 7fe3e4e00700 1 -- start start 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.335+0000 7fe3e4e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e01a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.335+0000 7fe3e4e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e010bd20 con 0x7fe3e007d290 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.336+0000 7fe3dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e01a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.336+0000 7fe3dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e01a4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52340/0 (socket says 172.21.15.63:52340) 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.336+0000 7fe3dde00700 1 -- 172.21.15.63:0/2984075155 learned_addr learned my addr 172.21.15.63:0/2984075155 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.336+0000 7fe3dde00700 1 -- 172.21.15.63:0/2984075155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e01a49a0 con 0x7fe3e007d290 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.336+0000 7fe3dde00700 1 --2- 172.21.15.63:0/2984075155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e01a4460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00119b0 tx=0x7fe3d0011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.337+0000 7fe3d6a00700 1 -- 172.21.15.63:0/2984075155 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3d0003af0 con 0x7fe3e007d290 2024-05-27T07:17:18.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.337+0000 7fe3d6a00700 1 -- 172.21.15.63:0/2984075155 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fe3d001a8a0 con 0x7fe3e007d290 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.337+0000 7fe3d6a00700 1 -- 172.21.15.63:0/2984075155 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3d0024830 con 0x7fe3e007d290 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.337+0000 7fe3e4e00700 1 -- 172.21.15.63:0/2984075155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e01a4c30 con 0x7fe3e007d290 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.337+0000 7fe3e4e00700 1 -- 172.21.15.63:0/2984075155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e01a7840 con 0x7fe3e007d290 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.338+0000 7fe3e4e00700 1 -- 172.21.15.63:0/2984075155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e0064e10 con 0x7fe3e007d290 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.338+0000 7fe3d6a00700 1 -- 172.21.15.63:0/2984075155 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fe3d0022020 con 0x7fe3e007d290 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.339+0000 7fe3d6a00700 1 --2- 172.21.15.63:0/2984075155 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fe3c403e530 0x7fe3c4040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.339+0000 7fe3d6a00700 1 -- 172.21.15.63:0/2984075155 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe3d0053510 con 0x7fe3e007d290 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.339+0000 7fe3d7e00700 1 --2- 172.21.15.63:0/2984075155 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fe3c403e530 0x7fe3c4040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.340+0000 7fe3d7e00700 1 --2- 172.21.15.63:0/2984075155 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fe3c403e530 0x7fe3c4040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8006fd0 tx=0x7fe3c8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.344+0000 7fe3d6a00700 1 -- 172.21.15.63:0/2984075155 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3d00212c0 con 0x7fe3e007d290 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.640+0000 7fe3e4e00700 1 -- 172.21.15.63:0/2984075155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fe3e002e7f0 con 0x7fe3e007d290 2024-05-27T07:17:18.382 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.694+0000 7fe3d6a00700 1 -- 172.21.15.63:0/2984075155 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fe3d0053270 con 0x7fe3e007d290 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.698+0000 7fe3e4e00700 1 -- 172.21.15.63:0/2984075155 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fe3c403e530 msgr2=0x7fe3c4040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.698+0000 7fe3e4e00700 1 --2- 172.21.15.63:0/2984075155 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fe3c403e530 0x7fe3c4040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8006fd0 tx=0x7fe3c8006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.698+0000 7fe3e4e00700 1 -- 172.21.15.63:0/2984075155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 msgr2=0x7fe3e01a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.698+0000 7fe3d6a00700 1 -- 172.21.15.63:0/2984075155 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fe3d000f2e0 con 0x7fe3e007d290 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.698+0000 7fe3e4e00700 1 --2- 172.21.15.63:0/2984075155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e01a4460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00119b0 tx=0x7fe3d0011a90 comp rx=0 tx=0).stop 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.700+0000 7fe3e4e00700 1 -- 172.21.15.63:0/2984075155 shutdown_connections 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.700+0000 7fe3e4e00700 1 --2- 172.21.15.63:0/2984075155 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fe3c403e530 0x7fe3c4040a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.700+0000 7fe3e4e00700 1 --2- 172.21.15.63:0/2984075155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3e007d290 0x7fe3e01a4460 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.700+0000 7fe3e4e00700 1 -- 172.21.15.63:0/2984075155 >> 172.21.15.63:0/2984075155 conn(0x7fe3e0105100 msgr2=0x7fe3e019ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.700+0000 7fe3e4e00700 1 -- 172.21.15.63:0/2984075155 shutdown_connections 2024-05-27T07:17:18.383 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:17.700+0000 7fe3e4e00700 1 -- 172.21.15.63:0/2984075155 wait complete. 2024-05-27T07:17:18.680 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:18 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:18.472+0000 7ff281931200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:17:18.942 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:18 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:18.890+0000 7ff281931200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:17:18.942 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:18 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2984075155' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-27T07:17:18.942 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:18 smithi063 ceph-mon[31613]: mgrmap e5: x(active, since 5s) 2024-05-27T07:17:19.205 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:19 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:19.013+0000 7ff281931200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:17:19.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:19 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1551211736' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T07:17:20.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:17:20.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-27T07:17:20.075 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.089+0000 7f5b37200700 1 Processor -- start 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.089+0000 7f5b37200700 1 -- start start 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.090+0000 7f5b37200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b38109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.090+0000 7f5b37200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3810a2a0 con 0x7f5b381098a0 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.090+0000 7f5b35e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b38109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.090+0000 7f5b35e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b38109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42110/0 (socket says 172.21.15.63:42110) 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.090+0000 7f5b35e00700 1 -- 172.21.15.63:0/1655823422 learned_addr learned my addr 172.21.15.63:0/1655823422 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.091+0000 7f5b35e00700 1 -- 172.21.15.63:0/1655823422 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3810aa70 con 0x7f5b381098a0 2024-05-27T07:17:20.076 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.091+0000 7f5b35e00700 1 --2- 172.21.15.63:0/1655823422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b38109cb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5b280098a0 tx=0x7f5b28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e110e593b4609022 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.092+0000 7f5b34a00700 1 -- 172.21.15.63:0/1655823422 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b28004160 con 0x7f5b381098a0 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.092+0000 7f5b34a00700 1 -- 172.21.15.63:0/1655823422 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5b28004300 con 0x7f5b381098a0 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.092+0000 7f5b34a00700 1 -- 172.21.15.63:0/1655823422 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b280119e0 con 0x7f5b381098a0 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.093+0000 7f5b37200700 1 -- 172.21.15.63:0/1655823422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 msgr2=0x7f5b38109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.093+0000 7f5b37200700 1 --2- 172.21.15.63:0/1655823422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b38109cb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5b280098a0 tx=0x7f5b28009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.093+0000 7f5b37200700 1 -- 172.21.15.63:0/1655823422 shutdown_connections 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.093+0000 7f5b37200700 1 --2- 172.21.15.63:0/1655823422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b38109cb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.093+0000 7f5b37200700 1 -- 172.21.15.63:0/1655823422 >> 172.21.15.63:0/1655823422 conn(0x7f5b38104f40 msgr2=0x7f5b38107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.093+0000 7f5b37200700 1 -- 172.21.15.63:0/1655823422 shutdown_connections 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.093+0000 7f5b37200700 1 -- 172.21.15.63:0/1655823422 wait complete. 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.094+0000 7f5b37200700 1 Processor -- start 2024-05-27T07:17:20.077 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.094+0000 7f5b37200700 1 -- start start 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.095+0000 7f5b37200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b3819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.095+0000 7f5b37200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3810de20 con 0x7f5b381098a0 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.095+0000 7f5b35e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b3819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.095+0000 7f5b35e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b3819fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42116/0 (socket says 172.21.15.63:42116) 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.095+0000 7f5b35e00700 1 -- 172.21.15.63:0/1551211736 learned_addr learned my addr 172.21.15.63:0/1551211736 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.096+0000 7f5b35e00700 1 -- 172.21.15.63:0/1551211736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b381a02d0 con 0x7f5b381098a0 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.096+0000 7f5b35e00700 1 --2- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b3819fd90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009fb0 tx=0x7f5b28011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.096+0000 7f5b2ea00700 1 -- 172.21.15.63:0/1551211736 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b280038b0 con 0x7f5b381098a0 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.096+0000 7f5b2ea00700 1 -- 172.21.15.63:0/1551211736 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5b28003a50 con 0x7f5b381098a0 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.096+0000 7f5b2ea00700 1 -- 172.21.15.63:0/1551211736 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b2801a5c0 con 0x7f5b381098a0 2024-05-27T07:17:20.078 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.096+0000 7f5b37200700 1 -- 172.21.15.63:0/1551211736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b381a0560 con 0x7f5b381098a0 2024-05-27T07:17:20.081 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.097+0000 7f5b37200700 1 -- 172.21.15.63:0/1551211736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b381a3170 con 0x7f5b381098a0 2024-05-27T07:17:20.081 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.098+0000 7f5b37200700 1 -- 172.21.15.63:0/1551211736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b38064e10 con 0x7f5b381098a0 2024-05-27T07:17:20.081 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.098+0000 7f5b2ea00700 1 -- 172.21.15.63:0/1551211736 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f5b2801a870 con 0x7f5b381098a0 2024-05-27T07:17:20.081 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.098+0000 7f5b2ea00700 1 --2- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7f5b1c03e530 0x7f5b1c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:20.081 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.098+0000 7f5b2ea00700 1 -- 172.21.15.63:0/1551211736 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5b28053ae0 con 0x7f5b381098a0 2024-05-27T07:17:20.081 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.098+0000 7f5b35400700 1 -- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7f5b1c03e530 msgr2=0x7f5b1c040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3806249841 2024-05-27T07:17:20.081 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.098+0000 7f5b35400700 1 --2- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7f5b1c03e530 0x7f5b1c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:17:20.081 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.106+0000 7f5b2ea00700 1 -- 172.21.15.63:0/1551211736 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b28017e90 con 0x7f5b381098a0 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.299+0000 7f5b35400700 1 -- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7f5b1c03e530 msgr2=0x7f5b1c040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3806249841 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.299+0000 7f5b35400700 1 --2- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7f5b1c03e530 0x7f5b1c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.397+0000 7f5b37200700 1 -- 172.21.15.63:0/1551211736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5b3802e8e0 con 0x7f5b381098a0 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.398+0000 7f5b2ea00700 1 -- 172.21.15.63:0/1551211736 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f5b2802c350 con 0x7f5b381098a0 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.402+0000 7f5b37200700 1 -- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7f5b1c03e530 msgr2=0x7f5b1c040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.402+0000 7f5b37200700 1 --2- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7f5b1c03e530 0x7f5b1c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.402+0000 7f5b37200700 1 -- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 msgr2=0x7f5b3819fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.402+0000 7f5b37200700 1 --2- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b3819fd90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009fb0 tx=0x7f5b28011a60 comp rx=0 tx=0).stop 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.402+0000 7f5b37200700 1 -- 172.21.15.63:0/1551211736 shutdown_connections 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.402+0000 7f5b37200700 1 --2- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7f5b1c03e530 0x7f5b1c040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.402+0000 7f5b37200700 1 --2- 172.21.15.63:0/1551211736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b381098a0 0x7f5b3819fd90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:20.082 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.402+0000 7f5b37200700 1 -- 172.21.15.63:0/1551211736 >> 172.21.15.63:0/1551211736 conn(0x7f5b38104f40 msgr2=0x7f5b3807d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:20.083 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.402+0000 7f5b37200700 1 -- 172.21.15.63:0/1551211736 shutdown_connections 2024-05-27T07:17:20.083 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:19.402+0000 7f5b37200700 1 -- 172.21.15.63:0/1551211736 wait complete. 2024-05-27T07:17:20.083 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for the mgr to restart... 2024-05-27T07:17:20.083 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr epoch 5... 2024-05-27T07:17:20.891 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:20 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:20.660+0000 7ff281931200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:17:20.891 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:20 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:20.899+0000 7ff281931200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:17:21.205 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:21 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:21.122+0000 7ff281931200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:17:21.641 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:21 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:21.521+0000 7ff281931200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:17:21.642 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:21 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:21.649+0000 7ff281931200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:17:22.171 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:21 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:21.911+0000 7ff281931200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:17:22.171 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:22 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:22.179+0000 7ff281931200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:17:23.300 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:23 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:23.004+0000 7ff281931200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:17:23.300 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:23 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:23.308+0000 7ff281931200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:17:24.012 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:23 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:23.887+0000 7ff281931200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:17:24.455 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:24 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:24.021+0000 7ff281931200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:17:24.955 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:24 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:24.501+0000 7ff281931200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:17:25.441 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:24 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:24.977+0000 7ff281931200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:17:25.441 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:25.108+0000 7ff281931200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:17:25.441 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:25.449+0000 7ff281931200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:17:25.706 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:25.639+0000 7ff281931200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:17:26.205 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:25.775+0000 7ff281931200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:17:26.706 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:26 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:26.217+0000 7ff281931200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:17:26.706 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:26 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:26.348+0000 7ff281931200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:17:27.206 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:27 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:27.151+0000 7ff281931200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:17:27.706 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:27 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:27.454+0000 7ff281931200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:17:27.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:27 smithi063 ceph-mon[31613]: Active manager daemon x restarted 2024-05-27T07:17:27.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:27 smithi063 ceph-mon[31613]: Activating manager daemon x 2024-05-27T07:17:27.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:27 smithi063 ceph-mon[31613]: osdmap e2: 0 total, 0 up, 0 in 2024-05-27T07:17:27.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:27 smithi063 ceph-mon[31613]: mgrmap e6: x(active, starting, since 0.00757648s) 2024-05-27T07:17:27.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:27 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:17:27.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:27 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:17:27.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:27 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:17:27.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:27 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:17:27.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:27 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:17:27.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:27 smithi063 ceph-mon[31613]: Manager daemon x is now available 2024-05-27T07:17:28.907 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:28 smithi063 ceph-mon[31613]: Found migration_current of "None". Setting to last migration. 2024-05-27T07:17:28.907 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:28 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:28.908 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:28 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:28.908 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:28 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:17:28.908 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:28 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:17:28.908 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:28 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:17:28.908 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:28 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:17:28.908 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:28 smithi063 ceph-mon[31613]: mgrmap e7: x(active, since 1.01981s) 2024-05-27T07:17:29.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:17:29.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-27T07:17:29.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T07:17:29.208 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:17:29.209 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.732+0000 7fd1f6400700 1 Processor -- start 2024-05-27T07:17:29.211 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.733+0000 7fd1f6400700 1 -- start start 2024-05-27T07:17:29.211 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.733+0000 7fd1f6400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f007d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:29.211 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.733+0000 7fd1f6400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f007eff0 con 0x7fd1f007d250 2024-05-27T07:17:29.211 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.734+0000 7fd1ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f007d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:29.211 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.734+0000 7fd1ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f007d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42118/0 (socket says 172.21.15.63:42118) 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.734+0000 7fd1ef400700 1 -- 172.21.15.63:0/304564497 learned_addr learned my addr 172.21.15.63:0/304564497 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.734+0000 7fd1ef400700 1 -- 172.21.15.63:0/304564497 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f007dba0 con 0x7fd1f007d250 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.735+0000 7fd1ef400700 1 --2- 172.21.15.63:0/304564497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f007d660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00098a0 tx=0x7fd1e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7024de8e488cfbf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.735+0000 7fd1ee000700 1 -- 172.21.15.63:0/304564497 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e0004160 con 0x7fd1f007d250 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.735+0000 7fd1ee000700 1 -- 172.21.15.63:0/304564497 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd1e0004300 con 0x7fd1f007d250 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.735+0000 7fd1ee000700 1 -- 172.21.15.63:0/304564497 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e00119e0 con 0x7fd1f007d250 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.736+0000 7fd1f6400700 1 -- 172.21.15.63:0/304564497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 msgr2=0x7fd1f007d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.736+0000 7fd1f6400700 1 --2- 172.21.15.63:0/304564497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f007d660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00098a0 tx=0x7fd1e0009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.736+0000 7fd1f6400700 1 -- 172.21.15.63:0/304564497 shutdown_connections 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.736+0000 7fd1f6400700 1 --2- 172.21.15.63:0/304564497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f007d660 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.736+0000 7fd1f6400700 1 -- 172.21.15.63:0/304564497 >> 172.21.15.63:0/304564497 conn(0x7fd1f01050e0 msgr2=0x7fd1f01075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.736+0000 7fd1f6400700 1 -- 172.21.15.63:0/304564497 shutdown_connections 2024-05-27T07:17:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.736+0000 7fd1f6400700 1 -- 172.21.15.63:0/304564497 wait complete. 2024-05-27T07:17:29.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.737+0000 7fd1f6400700 1 Processor -- start 2024-05-27T07:17:29.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.737+0000 7fd1f6400700 1 -- start start 2024-05-27T07:17:29.213 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.738+0000 7fd1f6400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f01a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:29.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.738+0000 7fd1f6400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0110140 con 0x7fd1f007d250 2024-05-27T07:17:29.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.738+0000 7fd1ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f01a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.738+0000 7fd1ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f01a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42132/0 (socket says 172.21.15.63:42132) 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.738+0000 7fd1ef400700 1 -- 172.21.15.63:0/2376892072 learned_addr learned my addr 172.21.15.63:0/2376892072 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.738+0000 7fd1ef400700 1 -- 172.21.15.63:0/2376892072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f01a05b0 con 0x7fd1f007d250 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.739+0000 7fd1ef400700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f01a0070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00119b0 tx=0x7fd1e0011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.739+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e0003af0 con 0x7fd1f007d250 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.739+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd1e001a8a0 con 0x7fd1f007d250 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.739+0000 7fd1f6400700 1 -- 172.21.15.63:0/2376892072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f01a0840 con 0x7fd1f007d250 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.739+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e0024830 con 0x7fd1f007d250 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.739+0000 7fd1f6400700 1 -- 172.21.15.63:0/2376892072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f01a3450 con 0x7fd1f007d250 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.741+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fd1e0022020 con 0x7fd1f007d250 2024-05-27T07:17:29.218 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.741+0000 7fd1e7e00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 0x7fd1d4040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.741+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 --> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd1d40411a0 con 0x7fd1d403e580 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.741+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd1e00537e0 con 0x7fd1f007d250 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.741+0000 7fd1eea00700 1 -- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 msgr2=0x7fd1d4040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3806249841 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.741+0000 7fd1eea00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 0x7fd1d4040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.942+0000 7fd1eea00700 1 -- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 msgr2=0x7fd1d4040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3806249841 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:20.942+0000 7fd1eea00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 0x7fd1d4040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:21.343+0000 7fd1eea00700 1 -- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 msgr2=0x7fd1d4040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3806249841 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:21.343+0000 7fd1eea00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 0x7fd1d4040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:22.144+0000 7fd1eea00700 1 -- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 msgr2=0x7fd1d4040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3806249841 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:22.144+0000 7fd1eea00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 0x7fd1d4040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:23.745+0000 7fd1eea00700 1 -- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 msgr2=0x7fd1d4040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3806249841 2024-05-27T07:17:29.219 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:23.745+0000 7fd1eea00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 0x7fd1d4040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T07:17:29.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:26.948+0000 7fd1eea00700 1 -- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 msgr2=0x7fd1d4040a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/3806249841 2024-05-27T07:17:29.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:26.948+0000 7fd1eea00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 0x7fd1d4040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-27T07:17:29.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:27.464+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fd1e001aae0 con 0x7fd1f007d250 2024-05-27T07:17:29.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:27.465+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 msgr2=0x7fd1d4040a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:17:29.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:27.465+0000 7fd1e7e00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 0x7fd1d4040a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:29.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.472+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fd1e00552c0 con 0x7fd1f007d250 2024-05-27T07:17:29.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.472+0000 7fd1e7e00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fd1d4041ec0 0x7fd1d40442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:29.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.472+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 --> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd1e0023ec0 con 0x7fd1d4041ec0 2024-05-27T07:17:29.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.474+0000 7fd1eea00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fd1d4041ec0 0x7fd1d40442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:29.220 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.479+0000 7fd1eea00700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fd1d4041ec0 0x7fd1d40442e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc003a10 tx=0x7fd1dc0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.481+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 <== mgr.14120 v2:172.21.15.63:6800/1689640179 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd1e0023ec0 con 0x7fd1d4041ec0 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.490+0000 7fd1f6400700 1 -- 172.21.15.63:0/2376892072 --> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd1f0064e10 con 0x7fd1d4041ec0 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.491+0000 7fd1e7e00700 1 -- 172.21.15.63:0/2376892072 <== mgr.14120 v2:172.21.15.63:6800/1689640179 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fd1f0064e10 con 0x7fd1d4041ec0 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 -- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fd1d4041ec0 msgr2=0x7fd1d40442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fd1d4041ec0 0x7fd1d40442e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc003a10 tx=0x7fd1dc0092b0 comp rx=0 tx=0).stop 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 -- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 msgr2=0x7fd1f01a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f01a0070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00119b0 tx=0x7fd1e0011a90 comp rx=0 tx=0).stop 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 -- 172.21.15.63:0/2376892072 shutdown_connections 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1f007d250 0x7fd1f01a0070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fd1d4041ec0 0x7fd1d40442e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:29.221 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 --2- 172.21.15.63:0/2376892072 >> [v2:172.21.15.63:6800/3806249841,v1:172.21.15.63:6801/3806249841] conn(0x7fd1d403e580 0x7fd1d4040a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:29.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 -- 172.21.15.63:0/2376892072 >> 172.21.15.63:0/2376892072 conn(0x7fd1f01050e0 msgr2=0x7fd1f0105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:29.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 -- 172.21.15.63:0/2376892072 shutdown_connections 2024-05-27T07:17:29.222 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:28.492+0000 7fd1f6400700 1 -- 172.21.15.63:0/2376892072 wait complete. 2024-05-27T07:17:29.223 INFO:teuthology.orchestra.run.smithi063.stdout:mgr epoch 5 is available 2024-05-27T07:17:29.223 INFO:teuthology.orchestra.run.smithi063.stdout:Setting orchestrator backend to cephadm... 2024-05-27T07:17:29.911 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:29 smithi063 ceph-mon[31613]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T07:17:29.911 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:29 smithi063 ceph-mon[31613]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T07:17:29.911 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:29 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:29.911 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:29 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:29.912 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:29 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:17:30.847 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:30 smithi063 ceph-mon[31613]: [27/May/2024:07:17:29] ENGINE Bus STARTING 2024-05-27T07:17:30.847 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:30 smithi063 ceph-mon[31613]: [27/May/2024:07:17:29] ENGINE Serving on http://172.21.15.63:8765 2024-05-27T07:17:30.847 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:30 smithi063 ceph-mon[31613]: [27/May/2024:07:17:29] ENGINE Serving on https://172.21.15.63:7150 2024-05-27T07:17:30.847 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:30 smithi063 ceph-mon[31613]: [27/May/2024:07:17:29] ENGINE Bus STARTED 2024-05-27T07:17:30.847 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:30 smithi063 ceph-mon[31613]: mgrmap e8: x(active, since 2s) 2024-05-27T07:17:30.847 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:30 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:30.848 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:30 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:17:30.895 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.919+0000 7f7186c00700 1 Processor -- start 2024-05-27T07:17:30.895 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.920+0000 7f7186c00700 1 -- start start 2024-05-27T07:17:30.895 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.920+0000 7f7186c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718010bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:30.895 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.920+0000 7f7186c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f718010c550 con 0x7f718010bb50 2024-05-27T07:17:30.895 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.921+0000 7f7185800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718010bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:30.895 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.921+0000 7f7185800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718010bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54836/0 (socket says 172.21.15.63:54836) 2024-05-27T07:17:30.895 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.921+0000 7f7185800700 1 -- 172.21.15.63:0/1381037206 learned_addr learned my addr 172.21.15.63:0/1381037206 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:30.896 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.921+0000 7f7185800700 1 -- 172.21.15.63:0/1381037206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718010cc80 con 0x7f718010bb50 2024-05-27T07:17:30.896 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.921+0000 7f7185800700 1 --2- 172.21.15.63:0/1381037206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718010bf60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71740098a0 tx=0x7f7174009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab0398224602d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:30.896 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.922+0000 7f717fe00700 1 -- 172.21.15.63:0/1381037206 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7174004160 con 0x7f718010bb50 2024-05-27T07:17:30.896 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.922+0000 7f717fe00700 1 -- 172.21.15.63:0/1381037206 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7174004300 con 0x7f718010bb50 2024-05-27T07:17:30.896 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.922+0000 7f717fe00700 1 -- 172.21.15.63:0/1381037206 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71740119e0 con 0x7f718010bb50 2024-05-27T07:17:30.896 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.923+0000 7f7186c00700 1 -- 172.21.15.63:0/1381037206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 msgr2=0x7f718010bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:30.896 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.923+0000 7f7186c00700 1 --2- 172.21.15.63:0/1381037206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718010bf60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71740098a0 tx=0x7f7174009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:30.896 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.923+0000 7f7186c00700 1 -- 172.21.15.63:0/1381037206 shutdown_connections 2024-05-27T07:17:30.896 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.923+0000 7f7186c00700 1 --2- 172.21.15.63:0/1381037206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718010bf60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:30.896 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.923+0000 7f7186c00700 1 -- 172.21.15.63:0/1381037206 >> 172.21.15.63:0/1381037206 conn(0x7f71801074b0 msgr2=0x7f71801098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:30.897 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.923+0000 7f7186c00700 1 -- 172.21.15.63:0/1381037206 shutdown_connections 2024-05-27T07:17:30.897 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.923+0000 7f7186c00700 1 -- 172.21.15.63:0/1381037206 wait complete. 2024-05-27T07:17:30.897 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.924+0000 7f7186c00700 1 Processor -- start 2024-05-27T07:17:30.897 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.924+0000 7f7186c00700 1 -- start start 2024-05-27T07:17:30.897 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.924+0000 7f7186c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:30.897 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.924+0000 7f7186c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7180110030 con 0x7f718010bb50 2024-05-27T07:17:30.897 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.925+0000 7f7185800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:30.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.925+0000 7f7185800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718019fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54840/0 (socket says 172.21.15.63:54840) 2024-05-27T07:17:30.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.925+0000 7f7185800700 1 -- 172.21.15.63:0/3728337654 learned_addr learned my addr 172.21.15.63:0/3728337654 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:30.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.925+0000 7f7185800700 1 -- 172.21.15.63:0/3728337654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71801a02c0 con 0x7f718010bb50 2024-05-27T07:17:30.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.925+0000 7f7185800700 1 --2- 172.21.15.63:0/3728337654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718019fd80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71740046e0 tx=0x7f7174011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:30.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.926+0000 7f717e000700 1 -- 172.21.15.63:0/3728337654 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71740039e0 con 0x7f718010bb50 2024-05-27T07:17:30.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.926+0000 7f717e000700 1 -- 172.21.15.63:0/3728337654 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7174003b80 con 0x7f718010bb50 2024-05-27T07:17:30.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.926+0000 7f717e000700 1 -- 172.21.15.63:0/3728337654 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7174024700 con 0x7f718010bb50 2024-05-27T07:17:30.900 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.926+0000 7f7186c00700 1 -- 172.21.15.63:0/3728337654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71801a0550 con 0x7f718010bb50 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.926+0000 7f7186c00700 1 -- 172.21.15.63:0/3728337654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71801a3160 con 0x7f718010bb50 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.927+0000 7f7186c00700 1 -- 172.21.15.63:0/3728337654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7180064e10 con 0x7f718010bb50 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.927+0000 7f717e000700 1 -- 172.21.15.63:0/3728337654 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7174022020 con 0x7f718010bb50 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.928+0000 7f717e000700 1 --2- 172.21.15.63:0/3728337654 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f716803e580 0x7f7168040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.928+0000 7f717e000700 1 -- 172.21.15.63:0/3728337654 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f71740539b0 con 0x7f718010bb50 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.928+0000 7f7184e00700 1 --2- 172.21.15.63:0/3728337654 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f716803e580 0x7f7168040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.929+0000 7f7184e00700 1 --2- 172.21.15.63:0/3728337654 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f716803e580 0x7f7168040a90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f716c006fd0 tx=0x7f716c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:29.935+0000 7f717e000700 1 -- 172.21.15.63:0/3728337654 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f717400b3e0 con 0x7f718010bb50 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.177+0000 7f7186c00700 1 -- 172.21.15.63:0/3728337654 --> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f7180109370 con 0x7f716803e580 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.190+0000 7f717e000700 1 -- 172.21.15.63:0/3728337654 <== mgr.14120 v2:172.21.15.63:6800/1689640179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7180109370 con 0x7f716803e580 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.193+0000 7f7186c00700 1 -- 172.21.15.63:0/3728337654 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f716803e580 msgr2=0x7f7168040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:30.901 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.193+0000 7f7186c00700 1 --2- 172.21.15.63:0/3728337654 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f716803e580 0x7f7168040a90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f716c006fd0 tx=0x7f716c006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:30.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.193+0000 7f7186c00700 1 -- 172.21.15.63:0/3728337654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 msgr2=0x7f718019fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:30.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.193+0000 7f7186c00700 1 --2- 172.21.15.63:0/3728337654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718019fd80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71740046e0 tx=0x7f7174011a60 comp rx=0 tx=0).stop 2024-05-27T07:17:30.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.193+0000 7f7186c00700 1 -- 172.21.15.63:0/3728337654 shutdown_connections 2024-05-27T07:17:30.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.193+0000 7f7186c00700 1 --2- 172.21.15.63:0/3728337654 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f716803e580 0x7f7168040a90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:30.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.193+0000 7f7186c00700 1 --2- 172.21.15.63:0/3728337654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f718010bb50 0x7f718019fd80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:30.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.194+0000 7f7186c00700 1 -- 172.21.15.63:0/3728337654 >> 172.21.15.63:0/3728337654 conn(0x7f71801074b0 msgr2=0x7f7180108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:30.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.194+0000 7f7186c00700 1 -- 172.21.15.63:0/3728337654 shutdown_connections 2024-05-27T07:17:30.902 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:30.194+0000 7f7186c00700 1 -- 172.21.15.63:0/3728337654 wait complete. 2024-05-27T07:17:31.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:31 smithi063 ceph-mon[31613]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:17:32.515 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-27T07:17:32.515 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.583+0000 7f7388c00700 1 Processor -- start 2024-05-27T07:17:32.515 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.583+0000 7f7388c00700 1 -- start start 2024-05-27T07:17:32.515 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.584+0000 7f7388c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f7384109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:32.515 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.584+0000 7f7388c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f738410a2a0 con 0x7f73841098a0 2024-05-27T07:17:32.516 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.584+0000 7f7383400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f7384109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:32.516 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.584+0000 7f7383400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f7384109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54852/0 (socket says 172.21.15.63:54852) 2024-05-27T07:17:32.516 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.584+0000 7f7383400700 1 -- 172.21.15.63:0/1534199439 learned_addr learned my addr 172.21.15.63:0/1534199439 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:32.516 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.585+0000 7f7383400700 1 -- 172.21.15.63:0/1534199439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738410aa70 con 0x7f73841098a0 2024-05-27T07:17:32.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.585+0000 7f7383400700 1 --2- 172.21.15.63:0/1534199439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f7384109cb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f73740098a0 tx=0x7f7374009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b5f6039b7f381b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:32.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.586+0000 7f7382000700 1 -- 172.21.15.63:0/1534199439 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7374004160 con 0x7f73841098a0 2024-05-27T07:17:32.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.586+0000 7f7382000700 1 -- 172.21.15.63:0/1534199439 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7374004300 con 0x7f73841098a0 2024-05-27T07:17:32.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.586+0000 7f7382000700 1 -- 172.21.15.63:0/1534199439 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73740119e0 con 0x7f73841098a0 2024-05-27T07:17:32.520 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.587+0000 7f7388c00700 1 -- 172.21.15.63:0/1534199439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 msgr2=0x7f7384109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.587+0000 7f7388c00700 1 --2- 172.21.15.63:0/1534199439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f7384109cb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f73740098a0 tx=0x7f7374009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.587+0000 7f7388c00700 1 -- 172.21.15.63:0/1534199439 shutdown_connections 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.587+0000 7f7388c00700 1 --2- 172.21.15.63:0/1534199439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f7384109cb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.587+0000 7f7388c00700 1 -- 172.21.15.63:0/1534199439 >> 172.21.15.63:0/1534199439 conn(0x7f7384104f40 msgr2=0x7f7384107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.587+0000 7f7388c00700 1 -- 172.21.15.63:0/1534199439 shutdown_connections 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.587+0000 7f7388c00700 1 -- 172.21.15.63:0/1534199439 wait complete. 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.588+0000 7f7388c00700 1 Processor -- start 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.588+0000 7f7388c00700 1 -- start start 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.589+0000 7f7388c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f738419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.589+0000 7f7388c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f738410de20 con 0x7f73841098a0 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.589+0000 7f7383400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f738419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:32.521 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.589+0000 7f7383400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f738419fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54866/0 (socket says 172.21.15.63:54866) 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.589+0000 7f7383400700 1 -- 172.21.15.63:0/1800793281 learned_addr learned my addr 172.21.15.63:0/1800793281 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.589+0000 7f7383400700 1 -- 172.21.15.63:0/1800793281 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73841a02d0 con 0x7f73841098a0 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.590+0000 7f7383400700 1 --2- 172.21.15.63:0/1800793281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f738419fd90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7374009fb0 tx=0x7f7374011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.590+0000 7f737be00700 1 -- 172.21.15.63:0/1800793281 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73740038b0 con 0x7f73841098a0 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.590+0000 7f737be00700 1 -- 172.21.15.63:0/1800793281 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7374003a50 con 0x7f73841098a0 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.590+0000 7f737be00700 1 -- 172.21.15.63:0/1800793281 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f737401a5c0 con 0x7f73841098a0 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.590+0000 7f7388c00700 1 -- 172.21.15.63:0/1800793281 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73841a0560 con 0x7f73841098a0 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.590+0000 7f7388c00700 1 -- 172.21.15.63:0/1800793281 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73841a3170 con 0x7f73841098a0 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.591+0000 7f7388c00700 1 -- 172.21.15.63:0/1800793281 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7384064e10 con 0x7f73841098a0 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.592+0000 7f737be00700 1 -- 172.21.15.63:0/1800793281 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7374024a60 con 0x7f73841098a0 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.592+0000 7f737be00700 1 --2- 172.21.15.63:0/1800793281 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f736803e4e0 0x7f73680409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:32.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.592+0000 7f737be00700 1 -- 172.21.15.63:0/1800793281 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7374053be0 con 0x7f73841098a0 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.593+0000 7f7382a00700 1 --2- 172.21.15.63:0/1800793281 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f736803e4e0 0x7f73680409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.593+0000 7f7382a00700 1 --2- 172.21.15.63:0/1800793281 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f736803e4e0 0x7f73680409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7370006fd0 tx=0x7f7370006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.602+0000 7f737be00700 1 -- 172.21.15.63:0/1800793281 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73740212a0 con 0x7f73841098a0 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.837+0000 7f7388c00700 1 -- 172.21.15.63:0/1800793281 --> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f738402e900 con 0x7f736803e4e0 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.839+0000 7f737be00700 1 -- 172.21.15.63:0/1800793281 <== mgr.14120 v2:172.21.15.63:6800/1689640179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f738402e900 con 0x7f736803e4e0 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.841+0000 7f7388c00700 1 -- 172.21.15.63:0/1800793281 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f736803e4e0 msgr2=0x7f73680409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.841+0000 7f7388c00700 1 --2- 172.21.15.63:0/1800793281 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f736803e4e0 0x7f73680409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7370006fd0 tx=0x7f7370006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.841+0000 7f7388c00700 1 -- 172.21.15.63:0/1800793281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 msgr2=0x7f738419fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.841+0000 7f7388c00700 1 --2- 172.21.15.63:0/1800793281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f738419fd90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7374009fb0 tx=0x7f7374011a60 comp rx=0 tx=0).stop 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.842+0000 7f7388c00700 1 -- 172.21.15.63:0/1800793281 shutdown_connections 2024-05-27T07:17:32.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.842+0000 7f7388c00700 1 --2- 172.21.15.63:0/1800793281 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f736803e4e0 0x7f73680409f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:32.525 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.842+0000 7f7388c00700 1 --2- 172.21.15.63:0/1800793281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73841098a0 0x7f738419fd90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:32.525 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.842+0000 7f7388c00700 1 -- 172.21.15.63:0/1800793281 >> 172.21.15.63:0/1800793281 conn(0x7f7384104f40 msgr2=0x7f738407d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:32.525 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.842+0000 7f7388c00700 1 -- 172.21.15.63:0/1800793281 shutdown_connections 2024-05-27T07:17:32.525 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:31.842+0000 7f7388c00700 1 -- 172.21.15.63:0/1800793281 wait complete. 2024-05-27T07:17:32.525 INFO:teuthology.orchestra.run.smithi063.stdout:Generating ssh key... 2024-05-27T07:17:32.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:32 smithi063 ceph-mon[31613]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:17:33.732 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: Generating public/private rsa key pair. 2024-05-27T07:17:33.732 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: Your identification has been saved in /tmp/tmpse4vjg6u/key. 2024-05-27T07:17:33.732 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: Your public key has been saved in /tmp/tmpse4vjg6u/key.pub. 2024-05-27T07:17:33.732 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: The key fingerprint is: 2024-05-27T07:17:33.732 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: SHA256:8fIHE7ww/Fe/nNunqApwwhYPDwTYxxT11HiLQdsxQYs ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: The key's randomart image is: 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: +---[RSA 3072]----+ 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: | o.++o..o+=. | 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: |. ..o +o=o+ | 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: | .+ E+=. . | 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: | . * .*.o . . | 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: | = + S * . .| 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: | . + o + . o| 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: | . . . + | 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: | . . . +| 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: | ..... .oo| 2024-05-27T07:17:33.733 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: +----[SHA256]-----+ 2024-05-27T07:17:34.398 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.231+0000 7f0139c00700 1 Processor -- start 2024-05-27T07:17:34.399 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.231+0000 7f0139c00700 1 -- start start 2024-05-27T07:17:34.399 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.232+0000 7f0139c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013410bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:34.399 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.232+0000 7f0139c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f013410c550 con 0x7f013410bb50 2024-05-27T07:17:34.400 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.232+0000 7f0133e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013410bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:34.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.232+0000 7f0133e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013410bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54868/0 (socket says 172.21.15.63:54868) 2024-05-27T07:17:34.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.232+0000 7f0133e00700 1 -- 172.21.15.63:0/1886033607 learned_addr learned my addr 172.21.15.63:0/1886033607 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:34.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.233+0000 7f0133e00700 1 -- 172.21.15.63:0/1886033607 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013410cc80 con 0x7f013410bb50 2024-05-27T07:17:34.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.233+0000 7f0133e00700 1 --2- 172.21.15.63:0/1886033607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013410bf60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f01240098a0 tx=0x7f0124009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3964257ad64a8c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:34.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.234+0000 7f0132a00700 1 -- 172.21.15.63:0/1886033607 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0124004160 con 0x7f013410bb50 2024-05-27T07:17:34.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.234+0000 7f0132a00700 1 -- 172.21.15.63:0/1886033607 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0124004300 con 0x7f013410bb50 2024-05-27T07:17:34.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.234+0000 7f0132a00700 1 -- 172.21.15.63:0/1886033607 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01240119e0 con 0x7f013410bb50 2024-05-27T07:17:34.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.235+0000 7f0139c00700 1 -- 172.21.15.63:0/1886033607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 msgr2=0x7f013410bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:34.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.235+0000 7f0139c00700 1 --2- 172.21.15.63:0/1886033607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013410bf60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f01240098a0 tx=0x7f0124009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:34.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.235+0000 7f0139c00700 1 -- 172.21.15.63:0/1886033607 shutdown_connections 2024-05-27T07:17:34.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.235+0000 7f0139c00700 1 --2- 172.21.15.63:0/1886033607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013410bf60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:34.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.235+0000 7f0139c00700 1 -- 172.21.15.63:0/1886033607 >> 172.21.15.63:0/1886033607 conn(0x7f01341074b0 msgr2=0x7f01341098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:34.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.235+0000 7f0139c00700 1 -- 172.21.15.63:0/1886033607 shutdown_connections 2024-05-27T07:17:34.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.235+0000 7f0139c00700 1 -- 172.21.15.63:0/1886033607 wait complete. 2024-05-27T07:17:34.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.236+0000 7f0139c00700 1 Processor -- start 2024-05-27T07:17:34.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.237+0000 7f0139c00700 1 -- start start 2024-05-27T07:17:34.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.237+0000 7f0139c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.237+0000 7f0139c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0134110030 con 0x7f013410bb50 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.238+0000 7f0133e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.238+0000 7f0133e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54884/0 (socket says 172.21.15.63:54884) 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.238+0000 7f0133e00700 1 -- 172.21.15.63:0/3719103358 learned_addr learned my addr 172.21.15.63:0/3719103358 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.238+0000 7f0133e00700 1 -- 172.21.15.63:0/3719103358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01341a02c0 con 0x7f013410bb50 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.238+0000 7f0133e00700 1 --2- 172.21.15.63:0/3719103358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013419fd80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01240046e0 tx=0x7f0124011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.239+0000 7f0130c00700 1 -- 172.21.15.63:0/3719103358 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01240039e0 con 0x7f013410bb50 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.239+0000 7f0130c00700 1 -- 172.21.15.63:0/3719103358 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0124003b80 con 0x7f013410bb50 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.239+0000 7f0139c00700 1 -- 172.21.15.63:0/3719103358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01341a0550 con 0x7f013410bb50 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.239+0000 7f0130c00700 1 -- 172.21.15.63:0/3719103358 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0124024700 con 0x7f013410bb50 2024-05-27T07:17:34.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.239+0000 7f0139c00700 1 -- 172.21.15.63:0/3719103358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01341a3160 con 0x7f013410bb50 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.240+0000 7f0139c00700 1 -- 172.21.15.63:0/3719103358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0134064e10 con 0x7f013410bb50 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.240+0000 7f0130c00700 1 -- 172.21.15.63:0/3719103358 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0124022020 con 0x7f013410bb50 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.241+0000 7f0130c00700 1 --2- 172.21.15.63:0/3719103358 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f011803e580 0x7f0118040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.241+0000 7f0130c00700 1 -- 172.21.15.63:0/3719103358 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0124053900 con 0x7f013410bb50 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.241+0000 7f0133400700 1 --2- 172.21.15.63:0/3719103358 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f011803e580 0x7f0118040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.242+0000 7f0133400700 1 --2- 172.21.15.63:0/3719103358 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f011803e580 0x7f0118040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0120006fd0 tx=0x7f0120006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.247+0000 7f0130c00700 1 -- 172.21.15.63:0/3719103358 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0124016260 con 0x7f013410bb50 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.488+0000 7f0139c00700 1 -- 172.21.15.63:0/3719103358 --> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0134109370 con 0x7f011803e580 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.710+0000 7f0130c00700 1 -- 172.21.15.63:0/3719103358 <== mgr.14120 v2:172.21.15.63:6800/1689640179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0134109370 con 0x7f011803e580 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.714+0000 7f0139c00700 1 -- 172.21.15.63:0/3719103358 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f011803e580 msgr2=0x7f0118040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.714+0000 7f0139c00700 1 --2- 172.21.15.63:0/3719103358 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f011803e580 0x7f0118040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0120006fd0 tx=0x7f0120006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.714+0000 7f0139c00700 1 -- 172.21.15.63:0/3719103358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 msgr2=0x7f013419fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:34.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.714+0000 7f0139c00700 1 --2- 172.21.15.63:0/3719103358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013419fd80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01240046e0 tx=0x7f0124011a60 comp rx=0 tx=0).stop 2024-05-27T07:17:34.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.714+0000 7f0139c00700 1 -- 172.21.15.63:0/3719103358 shutdown_connections 2024-05-27T07:17:34.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.714+0000 7f0139c00700 1 --2- 172.21.15.63:0/3719103358 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f011803e580 0x7f0118040a90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:34.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.714+0000 7f0139c00700 1 --2- 172.21.15.63:0/3719103358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f013410bb50 0x7f013419fd80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:34.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.714+0000 7f0139c00700 1 -- 172.21.15.63:0/3719103358 >> 172.21.15.63:0/3719103358 conn(0x7f01341074b0 msgr2=0x7f0134108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:34.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.714+0000 7f0139c00700 1 -- 172.21.15.63:0/3719103358 shutdown_connections 2024-05-27T07:17:34.412 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:33.714+0000 7f0139c00700 1 -- 172.21.15.63:0/3719103358 wait complete. 2024-05-27T07:17:34.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:34 smithi063 ceph-mon[31613]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:17:34.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:34 smithi063 ceph-mon[31613]: Generating ssh key... 2024-05-27T07:17:34.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:34 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:34.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:34 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:36.077 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:35 smithi063 ceph-mon[31613]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:17:36.087 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4yzyjof97XttYidMe34GG5EeEt7krKWnfplm+1S/56RMy4HVyPIO+/PCVuooOu9hYV3JKXh7+XSJ/gTycYmiIpBWJipsVzzBi7mWK1koWNbFLzPPIg1mS4plj2XDQNoeef8MlCZ3phf82mh3TXBXZMWiP//psjtsmHBbSFFEXWqFEEH0BveMEbFiBiL9y7eMaoPNod6LJ6WfIcXwecItAm5JdpolkhD6n6RjXOmtjOu2UyCvNuVPSPrxKdGc5/304gjeUEUFUBFtnlP/At/Fbl2zY8Rs40u1poamzyoD16ytRqoV9J8hWYmPhk5XX8EbCvUQnZ8n1yaLNK6FxA4/vuBA+ZBGKsBzfuRVATvkgFzmfMas9FHFIn85HIQ8BZrDxdiVsDheHF4ix+ISN/mqmmkEMlV7lw2XZahHo3r1/aoGL5H3RShCxm3VhQb+Gb2ccx/PTgBhamIMxyDEMBR8G923Zx9qI82mJA8wDpTxJPLP/y5/GxqHx1wNLX0KPhRk= ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:17:36.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.092+0000 7f578cc00700 1 Processor -- start 2024-05-27T07:17:36.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.092+0000 7f578cc00700 1 -- start start 2024-05-27T07:17:36.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.093+0000 7f578cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578810c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:36.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.093+0000 7f578cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f578807ce70 con 0x7f578810a560 2024-05-27T07:17:36.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.093+0000 7f5785e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578810c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:36.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.093+0000 7f5785e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578810c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54894/0 (socket says 172.21.15.63:54894) 2024-05-27T07:17:36.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.093+0000 7f5785e00700 1 -- 172.21.15.63:0/1015635883 learned_addr learned my addr 172.21.15.63:0/1015635883 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:36.090 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.094+0000 7f5785e00700 1 -- 172.21.15.63:0/1015635883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578810cec0 con 0x7f578810a560 2024-05-27T07:17:36.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.094+0000 7f5785e00700 1 --2- 172.21.15.63:0/1015635883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578810c980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57780098a0 tx=0x7f5778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebe42657efaec17 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:36.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.094+0000 7f5784a00700 1 -- 172.21.15.63:0/1015635883 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5778004160 con 0x7f578810a560 2024-05-27T07:17:36.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.094+0000 7f5784a00700 1 -- 172.21.15.63:0/1015635883 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5778004300 con 0x7f578810a560 2024-05-27T07:17:36.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.094+0000 7f5784a00700 1 -- 172.21.15.63:0/1015635883 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57780119e0 con 0x7f578810a560 2024-05-27T07:17:36.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.095+0000 7f578cc00700 1 -- 172.21.15.63:0/1015635883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 msgr2=0x7f578810c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:36.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.095+0000 7f578cc00700 1 --2- 172.21.15.63:0/1015635883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578810c980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57780098a0 tx=0x7f5778009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:36.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.095+0000 7f578cc00700 1 -- 172.21.15.63:0/1015635883 shutdown_connections 2024-05-27T07:17:36.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.095+0000 7f578cc00700 1 --2- 172.21.15.63:0/1015635883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578810c980 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:36.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.095+0000 7f578cc00700 1 -- 172.21.15.63:0/1015635883 >> 172.21.15.63:0/1015635883 conn(0x7f5788104060 msgr2=0x7f57881064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:36.091 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.095+0000 7f578cc00700 1 -- 172.21.15.63:0/1015635883 shutdown_connections 2024-05-27T07:17:36.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.096+0000 7f578cc00700 1 -- 172.21.15.63:0/1015635883 wait complete. 2024-05-27T07:17:36.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.097+0000 7f578cc00700 1 Processor -- start 2024-05-27T07:17:36.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.097+0000 7f578cc00700 1 -- start start 2024-05-27T07:17:36.092 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.097+0000 7f578cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:36.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.097+0000 7f578cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f578810de20 con 0x7f578810a560 2024-05-27T07:17:36.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.097+0000 7f5785e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:36.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.097+0000 7f5785e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578819dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54902/0 (socket says 172.21.15.63:54902) 2024-05-27T07:17:36.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.097+0000 7f5785e00700 1 -- 172.21.15.63:0/3879652923 learned_addr learned my addr 172.21.15.63:0/3879652923 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:36.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.098+0000 7f5785e00700 1 -- 172.21.15.63:0/3879652923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578819e280 con 0x7f578810a560 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.098+0000 7f5785e00700 1 --2- 172.21.15.63:0/3879652923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578819dd40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f57780119b0 tx=0x7f5778011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.098+0000 7f577ea00700 1 -- 172.21.15.63:0/3879652923 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5778003af0 con 0x7f578810a560 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.098+0000 7f578cc00700 1 -- 172.21.15.63:0/3879652923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578819e510 con 0x7f578810a560 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.098+0000 7f577ea00700 1 -- 172.21.15.63:0/3879652923 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f577801a8a0 con 0x7f578810a560 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.098+0000 7f577ea00700 1 -- 172.21.15.63:0/3879652923 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5778024830 con 0x7f578810a560 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.099+0000 7f578cc00700 1 -- 172.21.15.63:0/3879652923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57881a1110 con 0x7f578810a560 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.100+0000 7f578cc00700 1 -- 172.21.15.63:0/3879652923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5788064e10 con 0x7f578810a560 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.100+0000 7f577ea00700 1 -- 172.21.15.63:0/3879652923 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5778022020 con 0x7f578810a560 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.100+0000 7f577ea00700 1 --2- 172.21.15.63:0/3879652923 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f576c03e530 0x7f576c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.100+0000 7f577ea00700 1 -- 172.21.15.63:0/3879652923 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5778053b70 con 0x7f578810a560 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.101+0000 7f5785400700 1 --2- 172.21.15.63:0/3879652923 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f576c03e530 0x7f576c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.101+0000 7f5785400700 1 --2- 172.21.15.63:0/3879652923 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f576c03e530 0x7f576c040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5774006fd0 tx=0x7f5774006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.106+0000 7f577ea00700 1 -- 172.21.15.63:0/3879652923 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f57780094f0 con 0x7f578810a560 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.349+0000 7f578cc00700 1 -- 172.21.15.63:0/3879652923 --> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f5788106040 con 0x7f576c03e530 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.350+0000 7f577ea00700 1 -- 172.21.15.63:0/3879652923 <== mgr.14120 v2:172.21.15.63:6800/1689640179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f5788106040 con 0x7f576c03e530 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.353+0000 7f578cc00700 1 -- 172.21.15.63:0/3879652923 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f576c03e530 msgr2=0x7f576c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.353+0000 7f578cc00700 1 --2- 172.21.15.63:0/3879652923 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f576c03e530 0x7f576c040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5774006fd0 tx=0x7f5774006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.353+0000 7f578cc00700 1 -- 172.21.15.63:0/3879652923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 msgr2=0x7f578819dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.353+0000 7f578cc00700 1 --2- 172.21.15.63:0/3879652923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578819dd40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f57780119b0 tx=0x7f5778011a90 comp rx=0 tx=0).stop 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.354+0000 7f578cc00700 1 -- 172.21.15.63:0/3879652923 shutdown_connections 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.354+0000 7f578cc00700 1 --2- 172.21.15.63:0/3879652923 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f576c03e530 0x7f576c040a40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.354+0000 7f578cc00700 1 --2- 172.21.15.63:0/3879652923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f578810a560 0x7f578819dd40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.354+0000 7f578cc00700 1 -- 172.21.15.63:0/3879652923 >> 172.21.15.63:0/3879652923 conn(0x7f5788104060 msgr2=0x7f5788105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.354+0000 7f578cc00700 1 -- 172.21.15.63:0/3879652923 shutdown_connections 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:35.354+0000 7f578cc00700 1 -- 172.21.15.63:0/3879652923 wait complete. 2024-05-27T07:17:36.097 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-27T07:17:36.098 INFO:teuthology.orchestra.run.smithi063.stdout:Adding key to root@localhost authorized_keys... 2024-05-27T07:17:36.098 INFO:teuthology.orchestra.run.smithi063.stdout:Adding host smithi063... 2024-05-27T07:17:38.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:37 smithi063 ceph-mon[31613]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi063", "addr": "172.21.15.63", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:17:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:38 smithi063 ceph-mon[31613]: Deploying cephadm binary to smithi063 2024-05-27T07:17:43.997 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Added host 'smithi063' with addr '172.21.15.63' 2024-05-27T07:17:43.997 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.781+0000 7f0119800700 1 Processor -- start 2024-05-27T07:17:43.997 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.781+0000 7f0119800700 1 -- start start 2024-05-27T07:17:43.997 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.781+0000 7f0119800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f011407d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:43.997 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.781+0000 7f0119800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f011407efb0 con 0x7f011407d210 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.782+0000 7f0112a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f011407d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.782+0000 7f0112a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f011407d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54910/0 (socket says 172.21.15.63:54910) 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.782+0000 7f0112a00700 1 -- 172.21.15.63:0/2408958861 learned_addr learned my addr 172.21.15.63:0/2408958861 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.782+0000 7f0112a00700 1 -- 172.21.15.63:0/2408958861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011407db60 con 0x7f011407d210 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.783+0000 7f0112a00700 1 --2- 172.21.15.63:0/2408958861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f011407d620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01040098a0 tx=0x7f0104009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2cc553ce029b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.783+0000 7f0112000700 1 -- 172.21.15.63:0/2408958861 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0104004160 con 0x7f011407d210 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.783+0000 7f0112000700 1 -- 172.21.15.63:0/2408958861 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0104004300 con 0x7f011407d210 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.783+0000 7f0112000700 1 -- 172.21.15.63:0/2408958861 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01040119e0 con 0x7f011407d210 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.784+0000 7f0119800700 1 -- 172.21.15.63:0/2408958861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 msgr2=0x7f011407d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.784+0000 7f0119800700 1 --2- 172.21.15.63:0/2408958861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f011407d620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01040098a0 tx=0x7f0104009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:43.998 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.784+0000 7f0119800700 1 -- 172.21.15.63:0/2408958861 shutdown_connections 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.784+0000 7f0119800700 1 --2- 172.21.15.63:0/2408958861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f011407d620 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.784+0000 7f0119800700 1 -- 172.21.15.63:0/2408958861 >> 172.21.15.63:0/2408958861 conn(0x7f01141050e0 msgr2=0x7f01141075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.784+0000 7f0119800700 1 -- 172.21.15.63:0/2408958861 shutdown_connections 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.784+0000 7f0119800700 1 -- 172.21.15.63:0/2408958861 wait complete. 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.786+0000 7f0119800700 1 Processor -- start 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.786+0000 7f0119800700 1 -- start start 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.786+0000 7f0119800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f01141a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.786+0000 7f0119800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0114110170 con 0x7f011407d210 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.786+0000 7f0112a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f01141a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.786+0000 7f0112a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f01141a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54920/0 (socket says 172.21.15.63:54920) 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.786+0000 7f0112a00700 1 -- 172.21.15.63:0/2166438892 learned_addr learned my addr 172.21.15.63:0/2166438892 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:43.999 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.787+0000 7f0112a00700 1 -- 172.21.15.63:0/2166438892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01141a05b0 con 0x7f011407d210 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.787+0000 7f0112a00700 1 --2- 172.21.15.63:0/2166438892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f01141a0070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01040119b0 tx=0x7f0104011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.787+0000 7f010be00700 1 -- 172.21.15.63:0/2166438892 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0104003af0 con 0x7f011407d210 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.787+0000 7f010be00700 1 -- 172.21.15.63:0/2166438892 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f010401a8a0 con 0x7f011407d210 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.788+0000 7f0119800700 1 -- 172.21.15.63:0/2166438892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01141a0840 con 0x7f011407d210 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.788+0000 7f010be00700 1 -- 172.21.15.63:0/2166438892 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0104024830 con 0x7f011407d210 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.788+0000 7f0119800700 1 -- 172.21.15.63:0/2166438892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01141a3450 con 0x7f011407d210 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.789+0000 7f0119800700 1 -- 172.21.15.63:0/2166438892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0114064e10 con 0x7f011407d210 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.789+0000 7f010be00700 1 -- 172.21.15.63:0/2166438892 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0104022020 con 0x7f011407d210 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.789+0000 7f010be00700 1 --2- 172.21.15.63:0/2166438892 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f00f803e580 0x7f00f8040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.790+0000 7f010be00700 1 -- 172.21.15.63:0/2166438892 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0104053980 con 0x7f011407d210 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.790+0000 7f010a000700 1 --2- 172.21.15.63:0/2166438892 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f00f803e580 0x7f00f8040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.791+0000 7f010a000700 1 --2- 172.21.15.63:0/2166438892 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f00f803e580 0x7f00f8040a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f00fc006fd0 tx=0x7f00fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:36.795+0000 7f010be00700 1 -- 172.21.15.63:0/2166438892 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f01040532d0 con 0x7f011407d210 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:37.038+0000 7f0119800700 1 -- 172.21.15.63:0/2166438892 --> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi063", "addr": "172.21.15.63", "target": ["mon-mgr", ""]}) -- 0x7f01141060a0 con 0x7f00f803e580 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.278+0000 7f010be00700 1 -- 172.21.15.63:0/2166438892 <== mgr.14120 v2:172.21.15.63:6800/1689640179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f01141060a0 con 0x7f00f803e580 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.283+0000 7f0119800700 1 -- 172.21.15.63:0/2166438892 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f00f803e580 msgr2=0x7f00f8040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.283+0000 7f0119800700 1 --2- 172.21.15.63:0/2166438892 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f00f803e580 0x7f00f8040a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f00fc006fd0 tx=0x7f00fc006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.283+0000 7f0119800700 1 -- 172.21.15.63:0/2166438892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 msgr2=0x7f01141a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.283+0000 7f0119800700 1 --2- 172.21.15.63:0/2166438892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f01141a0070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01040119b0 tx=0x7f0104011a90 comp rx=0 tx=0).stop 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.283+0000 7f0119800700 1 -- 172.21.15.63:0/2166438892 shutdown_connections 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.283+0000 7f0119800700 1 --2- 172.21.15.63:0/2166438892 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f00f803e580 0x7f00f8040a90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.283+0000 7f0119800700 1 --2- 172.21.15.63:0/2166438892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f011407d210 0x7f01141a0070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.283+0000 7f0119800700 1 -- 172.21.15.63:0/2166438892 >> 172.21.15.63:0/2166438892 conn(0x7f01141050e0 msgr2=0x7f0114105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:44.001 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.283+0000 7f0119800700 1 -- 172.21.15.63:0/2166438892 shutdown_connections 2024-05-27T07:17:44.002 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:43.283+0000 7f0119800700 1 -- 172.21.15.63:0/2166438892 wait complete. 2024-05-27T07:17:44.002 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying unmanaged mon service... 2024-05-27T07:17:44.626 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:44 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:44.627 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:44 smithi063 ceph-mon[31613]: Added host smithi063 2024-05-27T07:17:44.627 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:44 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:17:46.160 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-27T07:17:46.160 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.741+0000 7f019aa00700 1 Processor -- start 2024-05-27T07:17:46.160 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.742+0000 7f019aa00700 1 -- start start 2024-05-27T07:17:46.160 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.742+0000 7f019aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019410bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:46.160 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.742+0000 7f019aa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f019410c4c0 con 0x7f019410bac0 2024-05-27T07:17:46.160 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.743+0000 7f0199600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019410bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:46.160 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.743+0000 7f0199600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019410bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59382/0 (socket says 172.21.15.63:59382) 2024-05-27T07:17:46.160 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.743+0000 7f0199600700 1 -- 172.21.15.63:0/920324015 learned_addr learned my addr 172.21.15.63:0/920324015 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:46.160 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.743+0000 7f0199600700 1 -- 172.21.15.63:0/920324015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019410cc90 con 0x7f019410bac0 2024-05-27T07:17:46.160 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.743+0000 7f0199600700 1 --2- 172.21.15.63:0/920324015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019410bed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01880098a0 tx=0x7f0188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943b603abb6a65a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.744+0000 7f0193e00700 1 -- 172.21.15.63:0/920324015 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0188004160 con 0x7f019410bac0 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.744+0000 7f0193e00700 1 -- 172.21.15.63:0/920324015 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0188004300 con 0x7f019410bac0 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.744+0000 7f0193e00700 1 -- 172.21.15.63:0/920324015 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01880119e0 con 0x7f019410bac0 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.745+0000 7f019aa00700 1 -- 172.21.15.63:0/920324015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 msgr2=0x7f019410bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.745+0000 7f019aa00700 1 --2- 172.21.15.63:0/920324015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019410bed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01880098a0 tx=0x7f0188009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.745+0000 7f019aa00700 1 -- 172.21.15.63:0/920324015 shutdown_connections 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.745+0000 7f019aa00700 1 --2- 172.21.15.63:0/920324015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019410bed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.745+0000 7f019aa00700 1 -- 172.21.15.63:0/920324015 >> 172.21.15.63:0/920324015 conn(0x7f019407f0b0 msgr2=0x7f0194109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.745+0000 7f019aa00700 1 -- 172.21.15.63:0/920324015 shutdown_connections 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.745+0000 7f019aa00700 1 -- 172.21.15.63:0/920324015 wait complete. 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.746+0000 7f019aa00700 1 Processor -- start 2024-05-27T07:17:46.161 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.746+0000 7f019aa00700 1 -- start start 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.747+0000 7f019aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.747+0000 7f019aa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0194110040 con 0x7f019410bac0 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.747+0000 7f0199600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.747+0000 7f0199600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019419fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59388/0 (socket says 172.21.15.63:59388) 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.747+0000 7f0199600700 1 -- 172.21.15.63:0/2934946477 learned_addr learned my addr 172.21.15.63:0/2934946477 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.747+0000 7f0199600700 1 -- 172.21.15.63:0/2934946477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01941a0290 con 0x7f019410bac0 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.748+0000 7f0199600700 1 --2- 172.21.15.63:0/2934946477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019419fd50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f01880046e0 tx=0x7f0188011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.748+0000 7f0192000700 1 -- 172.21.15.63:0/2934946477 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01880039e0 con 0x7f019410bac0 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.748+0000 7f0192000700 1 -- 172.21.15.63:0/2934946477 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0188003b80 con 0x7f019410bac0 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.748+0000 7f0192000700 1 -- 172.21.15.63:0/2934946477 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0188024700 con 0x7f019410bac0 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.748+0000 7f019aa00700 1 -- 172.21.15.63:0/2934946477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01941a0520 con 0x7f019410bac0 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.748+0000 7f019aa00700 1 -- 172.21.15.63:0/2934946477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01941a3120 con 0x7f019410bac0 2024-05-27T07:17:46.162 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.749+0000 7f019aa00700 1 -- 172.21.15.63:0/2934946477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0194064e10 con 0x7f019410bac0 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.749+0000 7f0192000700 1 -- 172.21.15.63:0/2934946477 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0188022020 con 0x7f019410bac0 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.750+0000 7f0192000700 1 --2- 172.21.15.63:0/2934946477 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f017c03e580 0x7f017c040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.750+0000 7f0192000700 1 -- 172.21.15.63:0/2934946477 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0188053900 con 0x7f019410bac0 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.750+0000 7f0198c00700 1 --2- 172.21.15.63:0/2934946477 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f017c03e580 0x7f017c040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.751+0000 7f0198c00700 1 --2- 172.21.15.63:0/2934946477 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f017c03e580 0x7f017c040a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0180006fd0 tx=0x7f0180006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:44.756+0000 7f0192000700 1 -- 172.21.15.63:0/2934946477 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0188016260 con 0x7f019410bac0 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.035+0000 7f019aa00700 1 -- 172.21.15.63:0/2934946477 --> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0194109990 con 0x7f017c03e580 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.043+0000 7f0192000700 1 -- 172.21.15.63:0/2934946477 <== mgr.14120 v2:172.21.15.63:6800/1689640179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0194109990 con 0x7f017c03e580 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.046+0000 7f019aa00700 1 -- 172.21.15.63:0/2934946477 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f017c03e580 msgr2=0x7f017c040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.046+0000 7f019aa00700 1 --2- 172.21.15.63:0/2934946477 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f017c03e580 0x7f017c040a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0180006fd0 tx=0x7f0180006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.047+0000 7f019aa00700 1 -- 172.21.15.63:0/2934946477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 msgr2=0x7f019419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:46.163 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.047+0000 7f019aa00700 1 --2- 172.21.15.63:0/2934946477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019419fd50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f01880046e0 tx=0x7f0188011a60 comp rx=0 tx=0).stop 2024-05-27T07:17:46.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.047+0000 7f019aa00700 1 -- 172.21.15.63:0/2934946477 shutdown_connections 2024-05-27T07:17:46.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.047+0000 7f019aa00700 1 --2- 172.21.15.63:0/2934946477 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f017c03e580 0x7f017c040a90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:46.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.047+0000 7f019aa00700 1 --2- 172.21.15.63:0/2934946477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019410bac0 0x7f019419fd50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:46.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.047+0000 7f019aa00700 1 -- 172.21.15.63:0/2934946477 >> 172.21.15.63:0/2934946477 conn(0x7f019407f0b0 msgr2=0x7f0194109280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:46.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.047+0000 7f019aa00700 1 -- 172.21.15.63:0/2934946477 shutdown_connections 2024-05-27T07:17:46.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:45.047+0000 7f019aa00700 1 -- 172.21.15.63:0/2934946477 wait complete. 2024-05-27T07:17:46.164 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying unmanaged mgr service... 2024-05-27T07:17:46.316 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:46 smithi063 ceph-mon[31613]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:17:46.316 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:46 smithi063 ceph-mon[31613]: Saving service mon spec with placement count:5 2024-05-27T07:17:46.316 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:46 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:47.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:47 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:47.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:47 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:47.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:47 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:48.365 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-27T07:17:48.365 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.357+0000 7f5466e00700 1 Processor -- start 2024-05-27T07:17:48.365 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.357+0000 7f5466e00700 1 -- start start 2024-05-27T07:17:48.365 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.358+0000 7f5466e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f546007d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:48.365 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.358+0000 7f5466e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f546007efb0 con 0x7f546007d210 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.358+0000 7f545fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f546007d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.359+0000 7f545fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f546007d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59396/0 (socket says 172.21.15.63:59396) 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.359+0000 7f545fe00700 1 -- 172.21.15.63:0/3330250586 learned_addr learned my addr 172.21.15.63:0/3330250586 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.359+0000 7f545fe00700 1 -- 172.21.15.63:0/3330250586 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546007db60 con 0x7f546007d210 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.359+0000 7f545fe00700 1 --2- 172.21.15.63:0/3330250586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f546007d620 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54500098a0 tx=0x7f5450009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc67913cb55f4a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.360+0000 7f545ea00700 1 -- 172.21.15.63:0/3330250586 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5450004160 con 0x7f546007d210 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.360+0000 7f545ea00700 1 -- 172.21.15.63:0/3330250586 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5450004300 con 0x7f546007d210 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.360+0000 7f545ea00700 1 -- 172.21.15.63:0/3330250586 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5450011960 con 0x7f546007d210 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.361+0000 7f5466e00700 1 -- 172.21.15.63:0/3330250586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 msgr2=0x7f546007d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.361+0000 7f5466e00700 1 --2- 172.21.15.63:0/3330250586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f546007d620 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54500098a0 tx=0x7f5450009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:48.366 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.361+0000 7f5466e00700 1 -- 172.21.15.63:0/3330250586 shutdown_connections 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.361+0000 7f5466e00700 1 --2- 172.21.15.63:0/3330250586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f546007d620 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.361+0000 7f5466e00700 1 -- 172.21.15.63:0/3330250586 >> 172.21.15.63:0/3330250586 conn(0x7f54601050e0 msgr2=0x7f54601075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.361+0000 7f5466e00700 1 -- 172.21.15.63:0/3330250586 shutdown_connections 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.361+0000 7f5466e00700 1 -- 172.21.15.63:0/3330250586 wait complete. 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.362+0000 7f5466e00700 1 Processor -- start 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.362+0000 7f5466e00700 1 -- start start 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.363+0000 7f5466e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f54601a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.363+0000 7f5466e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f54601101c0 con 0x7f546007d210 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.363+0000 7f545fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f54601a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.363+0000 7f545fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f54601a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59408/0 (socket says 172.21.15.63:59408) 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.363+0000 7f545fe00700 1 -- 172.21.15.63:0/3153955533 learned_addr learned my addr 172.21.15.63:0/3153955533 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:48.367 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.363+0000 7f545fe00700 1 -- 172.21.15.63:0/3153955533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54601a05e0 con 0x7f546007d210 2024-05-27T07:17:48.368 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.364+0000 7f545fe00700 1 --2- 172.21.15.63:0/3153955533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f54601a00a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5450011670 tx=0x7f5450011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:48.368 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.364+0000 7f545cc00700 1 -- 172.21.15.63:0/3153955533 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5450003a60 con 0x7f546007d210 2024-05-27T07:17:48.368 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.364+0000 7f545cc00700 1 -- 172.21.15.63:0/3153955533 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f545001b8d0 con 0x7f546007d210 2024-05-27T07:17:48.368 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.364+0000 7f5466e00700 1 -- 172.21.15.63:0/3153955533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54601a0870 con 0x7f546007d210 2024-05-27T07:17:48.368 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.364+0000 7f545cc00700 1 -- 172.21.15.63:0/3153955533 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5450023790 con 0x7f546007d210 2024-05-27T07:17:48.368 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.364+0000 7f5466e00700 1 -- 172.21.15.63:0/3153955533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54601a3480 con 0x7f546007d210 2024-05-27T07:17:48.368 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.366+0000 7f5466e00700 1 -- 172.21.15.63:0/3153955533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5460064e10 con 0x7f546007d210 2024-05-27T07:17:48.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.366+0000 7f545cc00700 1 -- 172.21.15.63:0/3153955533 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5450023a40 con 0x7f546007d210 2024-05-27T07:17:48.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.366+0000 7f545cc00700 1 --2- 172.21.15.63:0/3153955533 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f544403e580 0x7f5444040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:48.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.366+0000 7f545cc00700 1 -- 172.21.15.63:0/3153955533 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5450054180 con 0x7f546007d210 2024-05-27T07:17:48.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.367+0000 7f545f400700 1 --2- 172.21.15.63:0/3153955533 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f544403e580 0x7f5444040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:48.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.368+0000 7f545f400700 1 --2- 172.21.15.63:0/3153955533 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f544403e580 0x7f5444040a90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f544c006fd0 tx=0x7f544c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:48.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.372+0000 7f545cc00700 1 -- 172.21.15.63:0/3153955533 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f545001f030 con 0x7f546007d210 2024-05-27T07:17:48.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.643+0000 7f5466e00700 1 -- 172.21.15.63:0/3153955533 --> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f54601060a0 con 0x7f544403e580 2024-05-27T07:17:48.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.654+0000 7f545cc00700 1 -- 172.21.15.63:0/3153955533 <== mgr.14120 v2:172.21.15.63:6800/1689640179 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f54601060a0 con 0x7f544403e580 2024-05-27T07:17:48.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.657+0000 7f5466e00700 1 -- 172.21.15.63:0/3153955533 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f544403e580 msgr2=0x7f5444040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:48.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.657+0000 7f5466e00700 1 --2- 172.21.15.63:0/3153955533 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f544403e580 0x7f5444040a90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f544c006fd0 tx=0x7f544c006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:48.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.657+0000 7f5466e00700 1 -- 172.21.15.63:0/3153955533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 msgr2=0x7f54601a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:48.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.657+0000 7f5466e00700 1 --2- 172.21.15.63:0/3153955533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f54601a00a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5450011670 tx=0x7f5450011a30 comp rx=0 tx=0).stop 2024-05-27T07:17:48.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.657+0000 7f5466e00700 1 -- 172.21.15.63:0/3153955533 shutdown_connections 2024-05-27T07:17:48.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.657+0000 7f5466e00700 1 --2- 172.21.15.63:0/3153955533 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f544403e580 0x7f5444040a90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:48.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.657+0000 7f5466e00700 1 --2- 172.21.15.63:0/3153955533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546007d210 0x7f54601a00a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:48.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.657+0000 7f5466e00700 1 -- 172.21.15.63:0/3153955533 >> 172.21.15.63:0/3153955533 conn(0x7f54601050e0 msgr2=0x7f5460105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:48.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.657+0000 7f5466e00700 1 -- 172.21.15.63:0/3153955533 shutdown_connections 2024-05-27T07:17:48.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:47.657+0000 7f5466e00700 1 -- 172.21.15.63:0/3153955533 wait complete. 2024-05-27T07:17:48.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:48 smithi063 ceph-mon[31613]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:17:48.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:48 smithi063 ceph-mon[31613]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:17:48.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:48 smithi063 ceph-mon[31613]: Saving service mgr spec with placement count:2 2024-05-27T07:17:50.419 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.073+0000 7ff7dbc00700 1 Processor -- start 2024-05-27T07:17:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.073+0000 7ff7dbc00700 1 -- start start 2024-05-27T07:17:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.073+0000 7ff7dbc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d407d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.073+0000 7ff7dbc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d407efd0 con 0x7ff7d407d230 2024-05-27T07:17:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.074+0000 7ff7d9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d407d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.074+0000 7ff7d9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d407d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43930/0 (socket says 172.21.15.63:43930) 2024-05-27T07:17:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.074+0000 7ff7d9200700 1 -- 172.21.15.63:0/818952809 learned_addr learned my addr 172.21.15.63:0/818952809 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.074+0000 7ff7d9200700 1 -- 172.21.15.63:0/818952809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d407db80 con 0x7ff7d407d230 2024-05-27T07:17:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.075+0000 7ff7d9200700 1 --2- 172.21.15.63:0/818952809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d407d640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80098a0 tx=0x7ff7c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91649083fd07b2d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.075+0000 7ff7d3400700 1 -- 172.21.15.63:0/818952809 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7c8004160 con 0x7ff7d407d230 2024-05-27T07:17:50.420 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.075+0000 7ff7d3400700 1 -- 172.21.15.63:0/818952809 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7c8004300 con 0x7ff7d407d230 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.075+0000 7ff7d3400700 1 -- 172.21.15.63:0/818952809 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7c8011960 con 0x7ff7d407d230 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.076+0000 7ff7dbc00700 1 -- 172.21.15.63:0/818952809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 msgr2=0x7ff7d407d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.076+0000 7ff7dbc00700 1 --2- 172.21.15.63:0/818952809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d407d640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80098a0 tx=0x7ff7c8009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.076+0000 7ff7dbc00700 1 -- 172.21.15.63:0/818952809 shutdown_connections 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.076+0000 7ff7dbc00700 1 --2- 172.21.15.63:0/818952809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d407d640 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.076+0000 7ff7dbc00700 1 -- 172.21.15.63:0/818952809 >> 172.21.15.63:0/818952809 conn(0x7ff7d41050e0 msgr2=0x7ff7d41075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.076+0000 7ff7dbc00700 1 -- 172.21.15.63:0/818952809 shutdown_connections 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.076+0000 7ff7dbc00700 1 -- 172.21.15.63:0/818952809 wait complete. 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.077+0000 7ff7dbc00700 1 Processor -- start 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.077+0000 7ff7dbc00700 1 -- start start 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.078+0000 7ff7dbc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d41a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.078+0000 7ff7dbc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d41101a0 con 0x7ff7d407d230 2024-05-27T07:17:50.421 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.078+0000 7ff7d9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d41a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.078+0000 7ff7d9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d41a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43946/0 (socket says 172.21.15.63:43946) 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.078+0000 7ff7d9200700 1 -- 172.21.15.63:0/1134940807 learned_addr learned my addr 172.21.15.63:0/1134940807 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.079+0000 7ff7d9200700 1 -- 172.21.15.63:0/1134940807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d41a05d0 con 0x7ff7d407d230 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.079+0000 7ff7d9200700 1 --2- 172.21.15.63:0/1134940807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d41a0090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8011670 tx=0x7ff7c8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.079+0000 7ff7d1600700 1 -- 172.21.15.63:0/1134940807 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7c8003a60 con 0x7ff7d407d230 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.079+0000 7ff7d1600700 1 -- 172.21.15.63:0/1134940807 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7c801b8d0 con 0x7ff7d407d230 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.079+0000 7ff7d1600700 1 -- 172.21.15.63:0/1134940807 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7c8023790 con 0x7ff7d407d230 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.079+0000 7ff7dbc00700 1 -- 172.21.15.63:0/1134940807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d41a0860 con 0x7ff7d407d230 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.080+0000 7ff7dbc00700 1 -- 172.21.15.63:0/1134940807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d41a3470 con 0x7ff7d407d230 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.081+0000 7ff7dbc00700 1 -- 172.21.15.63:0/1134940807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d4064e10 con 0x7ff7d407d230 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.081+0000 7ff7d1600700 1 -- 172.21.15.63:0/1134940807 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ff7c801b440 con 0x7ff7d407d230 2024-05-27T07:17:50.422 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.082+0000 7ff7d1600700 1 --2- 172.21.15.63:0/1134940807 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7ff7bc03e530 0x7ff7bc040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.082+0000 7ff7d1600700 1 -- 172.21.15.63:0/1134940807 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff7c8054150 con 0x7ff7d407d230 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.082+0000 7ff7d3e00700 1 --2- 172.21.15.63:0/1134940807 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7ff7bc03e530 0x7ff7bc040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.083+0000 7ff7d3e00700 1 --2- 172.21.15.63:0/1134940807 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7ff7bc03e530 0x7ff7bc040a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0006fd0 tx=0x7ff7c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.087+0000 7ff7d1600700 1 -- 172.21.15.63:0/1134940807 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7c801f030 con 0x7ff7d407d230 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.323+0000 7ff7dbc00700 1 -- 172.21.15.63:0/1134940807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7ff7d41a3970 con 0x7ff7d407d230 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.332+0000 7ff7d1600700 1 -- 172.21.15.63:0/1134940807 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7ff7c80202b0 con 0x7ff7d407d230 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.336+0000 7ff7dbc00700 1 -- 172.21.15.63:0/1134940807 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7ff7bc03e530 msgr2=0x7ff7bc040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.336+0000 7ff7dbc00700 1 --2- 172.21.15.63:0/1134940807 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7ff7bc03e530 0x7ff7bc040a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0006fd0 tx=0x7ff7c0006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.336+0000 7ff7dbc00700 1 -- 172.21.15.63:0/1134940807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 msgr2=0x7ff7d41a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.336+0000 7ff7dbc00700 1 --2- 172.21.15.63:0/1134940807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d41a0090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8011670 tx=0x7ff7c8011a30 comp rx=0 tx=0).stop 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.336+0000 7ff7dbc00700 1 -- 172.21.15.63:0/1134940807 shutdown_connections 2024-05-27T07:17:50.423 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.336+0000 7ff7dbc00700 1 --2- 172.21.15.63:0/1134940807 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7ff7bc03e530 0x7ff7bc040a40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:50.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.336+0000 7ff7dbc00700 1 --2- 172.21.15.63:0/1134940807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7d407d230 0x7ff7d41a0090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:50.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.336+0000 7ff7dbc00700 1 -- 172.21.15.63:0/1134940807 >> 172.21.15.63:0/1134940807 conn(0x7ff7d41050e0 msgr2=0x7ff7d4105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:50.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.336+0000 7ff7dbc00700 1 -- 172.21.15.63:0/1134940807 shutdown_connections 2024-05-27T07:17:50.424 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:49.337+0000 7ff7dbc00700 1 -- 172.21.15.63:0/1134940807 wait complete. 2024-05-27T07:17:50.601 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:50 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1134940807' entity='client.admin' 2024-05-27T07:17:50.601 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:50 smithi063 ceph-mon[31613]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:17:51.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:51 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:51.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:51 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3847721216' entity='client.admin' 2024-05-27T07:17:52.101 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.099+0000 7f8a88e00700 1 Processor -- start 2024-05-27T07:17:52.102 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.100+0000 7f8a88e00700 1 -- start start 2024-05-27T07:17:52.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.100+0000 7f8a88e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8410b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:52.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.100+0000 7f8a88e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a841096b0 con 0x7f8a8410b1a0 2024-05-27T07:17:52.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.100+0000 7f8a81e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8410b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:52.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.101+0000 7f8a81e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8410b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43956/0 (socket says 172.21.15.63:43956) 2024-05-27T07:17:52.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.101+0000 7f8a81e00700 1 -- 172.21.15.63:0/2589477483 learned_addr learned my addr 172.21.15.63:0/2589477483 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:52.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.101+0000 7f8a81e00700 1 -- 172.21.15.63:0/2589477483 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a84109830 con 0x7f8a8410b1a0 2024-05-27T07:17:52.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.101+0000 7f8a81e00700 1 --2- 172.21.15.63:0/2589477483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8410b5b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a740098a0 tx=0x7f8a74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce41aea73b5f5554 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:52.103 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.102+0000 7f8a80a00700 1 -- 172.21.15.63:0/2589477483 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a74004160 con 0x7f8a8410b1a0 2024-05-27T07:17:52.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.102+0000 7f8a80a00700 1 -- 172.21.15.63:0/2589477483 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a74004300 con 0x7f8a8410b1a0 2024-05-27T07:17:52.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.102+0000 7f8a80a00700 1 -- 172.21.15.63:0/2589477483 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a74011960 con 0x7f8a8410b1a0 2024-05-27T07:17:52.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.103+0000 7f8a88e00700 1 -- 172.21.15.63:0/2589477483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 msgr2=0x7f8a8410b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:52.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.103+0000 7f8a88e00700 1 --2- 172.21.15.63:0/2589477483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8410b5b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a740098a0 tx=0x7f8a74009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:52.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.103+0000 7f8a88e00700 1 -- 172.21.15.63:0/2589477483 shutdown_connections 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.103+0000 7f8a88e00700 1 --2- 172.21.15.63:0/2589477483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8410b5b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.103+0000 7f8a88e00700 1 -- 172.21.15.63:0/2589477483 >> 172.21.15.63:0/2589477483 conn(0x7f8a841050e0 msgr2=0x7f8a841075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.103+0000 7f8a88e00700 1 -- 172.21.15.63:0/2589477483 shutdown_connections 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.103+0000 7f8a88e00700 1 -- 172.21.15.63:0/2589477483 wait complete. 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.105+0000 7f8a88e00700 1 Processor -- start 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.105+0000 7f8a88e00700 1 -- start start 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.105+0000 7f8a88e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.106+0000 7f8a88e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8410e040 con 0x7f8a8410b1a0 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.106+0000 7f8a81e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.106+0000 7f8a81e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43960/0 (socket says 172.21.15.63:43960) 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.106+0000 7f8a81e00700 1 -- 172.21.15.63:0/3847721216 learned_addr learned my addr 172.21.15.63:0/3847721216 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.106+0000 7f8a81e00700 1 -- 172.21.15.63:0/3847721216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8419e3f0 con 0x7f8a8410b1a0 2024-05-27T07:17:52.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.107+0000 7f8a81e00700 1 --2- 172.21.15.63:0/3847721216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8419deb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8a74000c00 tx=0x7f8a74011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:52.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.107+0000 7f8a7aa00700 1 -- 172.21.15.63:0/3847721216 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a74003a60 con 0x7f8a8410b1a0 2024-05-27T07:17:52.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.107+0000 7f8a7aa00700 1 -- 172.21.15.63:0/3847721216 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a7401b8d0 con 0x7f8a8410b1a0 2024-05-27T07:17:52.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.107+0000 7f8a7aa00700 1 -- 172.21.15.63:0/3847721216 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a74019780 con 0x7f8a8410b1a0 2024-05-27T07:17:52.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.107+0000 7f8a88e00700 1 -- 172.21.15.63:0/3847721216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8419e680 con 0x7f8a8410b1a0 2024-05-27T07:17:52.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.107+0000 7f8a88e00700 1 -- 172.21.15.63:0/3847721216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a841a1290 con 0x7f8a8410b1a0 2024-05-27T07:17:52.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.109+0000 7f8a88e00700 1 -- 172.21.15.63:0/3847721216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a84064e10 con 0x7f8a8410b1a0 2024-05-27T07:17:52.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.109+0000 7f8a7aa00700 1 -- 172.21.15.63:0/3847721216 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8a7401b440 con 0x7f8a8410b1a0 2024-05-27T07:17:52.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.109+0000 7f8a7aa00700 1 --2- 172.21.15.63:0/3847721216 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f8a6803e580 0x7f8a68040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:52.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.110+0000 7f8a7aa00700 1 -- 172.21.15.63:0/3847721216 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8a74053e20 con 0x7f8a8410b1a0 2024-05-27T07:17:52.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.110+0000 7f8a81400700 1 --2- 172.21.15.63:0/3847721216 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f8a6803e580 0x7f8a68040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:52.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.111+0000 7f8a81400700 1 --2- 172.21.15.63:0/3847721216 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f8a6803e580 0x7f8a68040a90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006fd0 tx=0x7f8a70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:52.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.116+0000 7f8a7aa00700 1 -- 172.21.15.63:0/3847721216 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a7401f030 con 0x7f8a8410b1a0 2024-05-27T07:17:52.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.356+0000 7f8a88e00700 1 -- 172.21.15.63:0/3847721216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f8a841a1460 con 0x7f8a8410b1a0 2024-05-27T07:17:52.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.362+0000 7f8a7aa00700 1 -- 172.21.15.63:0/3847721216 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f8a7401ba80 con 0x7f8a8410b1a0 2024-05-27T07:17:52.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.366+0000 7f8a88e00700 1 -- 172.21.15.63:0/3847721216 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f8a6803e580 msgr2=0x7f8a68040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:52.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.366+0000 7f8a88e00700 1 --2- 172.21.15.63:0/3847721216 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f8a6803e580 0x7f8a68040a90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006fd0 tx=0x7f8a70006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:52.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.366+0000 7f8a88e00700 1 -- 172.21.15.63:0/3847721216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 msgr2=0x7f8a8419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:52.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.366+0000 7f8a88e00700 1 --2- 172.21.15.63:0/3847721216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8419deb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8a74000c00 tx=0x7f8a74011a30 comp rx=0 tx=0).stop 2024-05-27T07:17:52.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.366+0000 7f8a88e00700 1 -- 172.21.15.63:0/3847721216 shutdown_connections 2024-05-27T07:17:52.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.366+0000 7f8a88e00700 1 --2- 172.21.15.63:0/3847721216 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f8a6803e580 0x7f8a68040a90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:52.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.366+0000 7f8a88e00700 1 --2- 172.21.15.63:0/3847721216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a8410b1a0 0x7f8a8419deb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:52.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.366+0000 7f8a88e00700 1 -- 172.21.15.63:0/3847721216 >> 172.21.15.63:0/3847721216 conn(0x7f8a841050e0 msgr2=0x7f8a841059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:52.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.366+0000 7f8a88e00700 1 -- 172.21.15.63:0/3847721216 shutdown_connections 2024-05-27T07:17:52.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:51.366+0000 7f8a88e00700 1 -- 172.21.15.63:0/3847721216 wait complete. 2024-05-27T07:17:52.111 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling the dashboard module... 2024-05-27T07:17:52.668 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:52 smithi063 ceph-mon[31613]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:17:53.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:53 smithi063 ceph-mon[31613]: from='mgr.14120 172.21.15.63:0/1151530939' entity='mgr.x' 2024-05-27T07:17:53.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:53 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3230519547' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-27T07:17:54.077 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:53 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: ignoring --setuser ceph since I am not root 2024-05-27T07:17:54.077 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:53 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: ignoring --setgroup ceph since I am not root 2024-05-27T07:17:54.078 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:54 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:54.085+0000 7f16e2f44200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:17:54.381 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.811+0000 7f9613200700 1 Processor -- start 2024-05-27T07:17:54.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.811+0000 7f9613200700 1 -- start start 2024-05-27T07:17:54.384 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.812+0000 7f9613200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c07d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.812+0000 7f9613200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f960c07d820 con 0x7f960c07ee20 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.812+0000 7f9611e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c07d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.812+0000 7f9611e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c07d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43968/0 (socket says 172.21.15.63:43968) 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.812+0000 7f9611e00700 1 -- 172.21.15.63:0/2104484171 learned_addr learned my addr 172.21.15.63:0/2104484171 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.813+0000 7f9611e00700 1 -- 172.21.15.63:0/2104484171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c07d9a0 con 0x7f960c07ee20 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.813+0000 7f9611e00700 1 --2- 172.21.15.63:0/2104484171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c07d250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96000098a0 tx=0x7f9600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b14cb5f5851bbd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.813+0000 7f9610a00700 1 -- 172.21.15.63:0/2104484171 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9600004160 con 0x7f960c07ee20 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.813+0000 7f9610a00700 1 -- 172.21.15.63:0/2104484171 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9600004300 con 0x7f960c07ee20 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.813+0000 7f9610a00700 1 -- 172.21.15.63:0/2104484171 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9600011960 con 0x7f960c07ee20 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.814+0000 7f9613200700 1 -- 172.21.15.63:0/2104484171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 msgr2=0x7f960c07d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.814+0000 7f9613200700 1 --2- 172.21.15.63:0/2104484171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c07d250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96000098a0 tx=0x7f9600009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.814+0000 7f9613200700 1 -- 172.21.15.63:0/2104484171 shutdown_connections 2024-05-27T07:17:54.385 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.814+0000 7f9613200700 1 --2- 172.21.15.63:0/2104484171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c07d250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.814+0000 7f9613200700 1 -- 172.21.15.63:0/2104484171 >> 172.21.15.63:0/2104484171 conn(0x7f960c104f40 msgr2=0x7f960c107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.815+0000 7f9613200700 1 -- 172.21.15.63:0/2104484171 shutdown_connections 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.815+0000 7f9613200700 1 -- 172.21.15.63:0/2104484171 wait complete. 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.816+0000 7f9613200700 1 Processor -- start 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.816+0000 7f9613200700 1 -- start start 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.817+0000 7f9613200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.817+0000 7f9613200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10fea0 con 0x7f960c07ee20 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.817+0000 7f9611e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.817+0000 7f9611e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43978/0 (socket says 172.21.15.63:43978) 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.817+0000 7f9611e00700 1 -- 172.21.15.63:0/3230519547 learned_addr learned my addr 172.21.15.63:0/3230519547 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:54.386 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.818+0000 7f9611e00700 1 -- 172.21.15.63:0/3230519547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c1a0240 con 0x7f960c07ee20 2024-05-27T07:17:54.389 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.818+0000 7f9611e00700 1 --2- 172.21.15.63:0/3230519547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c19fd00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9600011670 tx=0x7f9600011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:54.389 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.818+0000 7f9606a00700 1 -- 172.21.15.63:0/3230519547 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f960001f070 con 0x7f960c07ee20 2024-05-27T07:17:54.389 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.818+0000 7f9606a00700 1 -- 172.21.15.63:0/3230519547 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9600003b20 con 0x7f960c07ee20 2024-05-27T07:17:54.389 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.818+0000 7f9613200700 1 -- 172.21.15.63:0/3230519547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c1a04d0 con 0x7f960c07ee20 2024-05-27T07:17:54.389 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.818+0000 7f9606a00700 1 -- 172.21.15.63:0/3230519547 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9600019580 con 0x7f960c07ee20 2024-05-27T07:17:54.389 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.819+0000 7f9613200700 1 -- 172.21.15.63:0/3230519547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c1a30e0 con 0x7f960c07ee20 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.820+0000 7f9606a00700 1 -- 172.21.15.63:0/3230519547 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f9600022070 con 0x7f960c07ee20 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.820+0000 7f9613200700 1 -- 172.21.15.63:0/3230519547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f0005360 con 0x7f960c07ee20 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.820+0000 7f9606a00700 1 --2- 172.21.15.63:0/3230519547 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f95f403e580 0x7f95f4040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.821+0000 7f9606a00700 1 -- 172.21.15.63:0/3230519547 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f96000549a0 con 0x7f960c07ee20 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.821+0000 7f9611400700 1 --2- 172.21.15.63:0/3230519547 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f95f403e580 0x7f95f4040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.821+0000 7f9611400700 1 --2- 172.21.15.63:0/3230519547 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f95f403e580 0x7f95f4040a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f95fc006fd0 tx=0x7f95fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:52.826+0000 7f9606a00700 1 -- 172.21.15.63:0/3230519547 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9600017e90 con 0x7f960c07ee20 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.135+0000 7f9613200700 1 -- 172.21.15.63:0/3230519547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f95f0005650 con 0x7f960c07ee20 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.662+0000 7f9606a00700 1 -- 172.21.15.63:0/3230519547 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f96000542f0 con 0x7f960c07ee20 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.666+0000 7f9606a00700 1 -- 172.21.15.63:0/3230519547 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f9600019cc0 con 0x7f960c07ee20 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.667+0000 7f9613200700 1 -- 172.21.15.63:0/3230519547 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f95f403e580 msgr2=0x7f95f4040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:54.390 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.667+0000 7f9613200700 1 --2- 172.21.15.63:0/3230519547 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f95f403e580 0x7f95f4040a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f95fc006fd0 tx=0x7f95fc006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:54.391 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.667+0000 7f9613200700 1 -- 172.21.15.63:0/3230519547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 msgr2=0x7f960c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:54.391 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.667+0000 7f9613200700 1 --2- 172.21.15.63:0/3230519547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c19fd00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9600011670 tx=0x7f9600011a30 comp rx=0 tx=0).stop 2024-05-27T07:17:54.391 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.667+0000 7f9613200700 1 -- 172.21.15.63:0/3230519547 shutdown_connections 2024-05-27T07:17:54.391 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.667+0000 7f9613200700 1 --2- 172.21.15.63:0/3230519547 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f95f403e580 0x7f95f4040a90 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f95fc006fd0 tx=0x7f95fc006e40 comp rx=0 tx=0).stop 2024-05-27T07:17:54.391 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.667+0000 7f9613200700 1 --2- 172.21.15.63:0/3230519547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f960c07ee20 0x7f960c19fd00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:54.391 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.667+0000 7f9613200700 1 -- 172.21.15.63:0/3230519547 >> 172.21.15.63:0/3230519547 conn(0x7f960c104f40 msgr2=0x7f960c105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:54.391 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.671+0000 7f9613200700 1 -- 172.21.15.63:0/3230519547 shutdown_connections 2024-05-27T07:17:54.391 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:53.671+0000 7f9613200700 1 -- 172.21.15.63:0/3230519547 wait complete. 2024-05-27T07:17:54.625 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:54 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:54.340+0000 7f16e2f44200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:17:54.626 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:54 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:54.482+0000 7f16e2f44200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:17:54.898 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:54 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3230519547' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-27T07:17:54.898 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:54 smithi063 ceph-mon[31613]: mgrmap e9: x(active, since 26s) 2024-05-27T07:17:55.206 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:54 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:54.906+0000 7f16e2f44200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:17:55.206 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:55 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:55.041+0000 7f16e2f44200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:17:56.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:17:56 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1017226737' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T07:17:56.809 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:56 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:56.701+0000 7f16e2f44200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:17:56.825 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:17:56.825 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-27T07:17:56.825 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T07:17:56.827 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-27T07:17:56.827 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T07:17:56.827 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:17:56.827 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.866+0000 7fe37ce00700 1 Processor -- start 2024-05-27T07:17:56.827 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.866+0000 7fe37ce00700 1 -- start start 2024-05-27T07:17:56.827 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.867+0000 7fe37ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37810c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:56.827 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.867+0000 7fe37ce00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe37810ca70 con 0x7fe37807ccc0 2024-05-27T07:17:56.828 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.867+0000 7fe377400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37810c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:56.828 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.867+0000 7fe377400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37810c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44010/0 (socket says 172.21.15.63:44010) 2024-05-27T07:17:56.828 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.867+0000 7fe377400700 1 -- 172.21.15.63:0/3753834021 learned_addr learned my addr 172.21.15.63:0/3753834021 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:56.828 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.868+0000 7fe377400700 1 -- 172.21.15.63:0/3753834021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37810cbf0 con 0x7fe37807ccc0 2024-05-27T07:17:56.828 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.868+0000 7fe377400700 1 --2- 172.21.15.63:0/3753834021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37810c460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3680098a0 tx=0x7fe368009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a964808f64dba1d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:56.828 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.869+0000 7fe376000700 1 -- 172.21.15.63:0/3753834021 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe368004160 con 0x7fe37807ccc0 2024-05-27T07:17:56.828 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.869+0000 7fe376000700 1 -- 172.21.15.63:0/3753834021 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe368004300 con 0x7fe37807ccc0 2024-05-27T07:17:56.830 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.869+0000 7fe376000700 1 -- 172.21.15.63:0/3753834021 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe368011960 con 0x7fe37807ccc0 2024-05-27T07:17:56.830 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.870+0000 7fe37ce00700 1 -- 172.21.15.63:0/3753834021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 msgr2=0x7fe37810c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:56.830 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.870+0000 7fe37ce00700 1 --2- 172.21.15.63:0/3753834021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37810c460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3680098a0 tx=0x7fe368009bb0 comp rx=0 tx=0).stop 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.870+0000 7fe37ce00700 1 -- 172.21.15.63:0/3753834021 shutdown_connections 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.870+0000 7fe37ce00700 1 --2- 172.21.15.63:0/3753834021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37810c460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.870+0000 7fe37ce00700 1 -- 172.21.15.63:0/3753834021 >> 172.21.15.63:0/3753834021 conn(0x7fe378103e70 msgr2=0x7fe378106320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.870+0000 7fe37ce00700 1 -- 172.21.15.63:0/3753834021 shutdown_connections 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.870+0000 7fe37ce00700 1 -- 172.21.15.63:0/3753834021 wait complete. 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.871+0000 7fe37ce00700 1 Processor -- start 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.871+0000 7fe37ce00700 1 -- start start 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.872+0000 7fe37ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.872+0000 7fe37ce00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe37810dc50 con 0x7fe37807ccc0 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.872+0000 7fe377400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:17:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.872+0000 7fe377400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37819db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44024/0 (socket says 172.21.15.63:44024) 2024-05-27T07:17:56.832 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.872+0000 7fe377400700 1 -- 172.21.15.63:0/1017226737 learned_addr learned my addr 172.21.15.63:0/1017226737 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:17:56.832 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.873+0000 7fe377400700 1 -- 172.21.15.63:0/1017226737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37819e040 con 0x7fe37807ccc0 2024-05-27T07:17:56.832 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.873+0000 7fe377400700 1 --2- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37819db00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe368011670 tx=0x7fe368011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:17:56.832 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.873+0000 7fe36fe00700 1 -- 172.21.15.63:0/1017226737 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe368003aa0 con 0x7fe37807ccc0 2024-05-27T07:17:56.832 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.873+0000 7fe36fe00700 1 -- 172.21.15.63:0/1017226737 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe36801b8d0 con 0x7fe37807ccc0 2024-05-27T07:17:56.832 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.873+0000 7fe37ce00700 1 -- 172.21.15.63:0/1017226737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37819e2d0 con 0x7fe37807ccc0 2024-05-27T07:17:56.832 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.873+0000 7fe37ce00700 1 -- 172.21.15.63:0/1017226737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3781a0ee0 con 0x7fe37807ccc0 2024-05-27T07:17:56.832 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.873+0000 7fe36fe00700 1 -- 172.21.15.63:0/1017226737 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3680197f0 con 0x7fe37807ccc0 2024-05-27T07:17:56.834 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.874+0000 7fe37ce00700 1 -- 172.21.15.63:0/1017226737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe378064e10 con 0x7fe37807ccc0 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.875+0000 7fe36fe00700 1 -- 172.21.15.63:0/1017226737 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fe3680199d0 con 0x7fe37807ccc0 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.875+0000 7fe36fe00700 1 --2- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fe35c03e530 0x7fe35c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.875+0000 7fe36fe00700 1 -- 172.21.15.63:0/1017226737 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe368053f60 con 0x7fe37807ccc0 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.875+0000 7fe376a00700 1 -- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fe35c03e530 msgr2=0x7fe35c040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1689640179 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.875+0000 7fe376a00700 1 --2- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fe35c03e530 0x7fe35c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:55.881+0000 7fe36fe00700 1 -- 172.21.15.63:0/1017226737 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe36801f030 con 0x7fe37807ccc0 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.076+0000 7fe376a00700 1 -- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fe35c03e530 msgr2=0x7fe35c040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1689640179 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.076+0000 7fe376a00700 1 --2- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fe35c03e530 0x7fe35c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.171+0000 7fe37ce00700 1 -- 172.21.15.63:0/1017226737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe3781a10b0 con 0x7fe37807ccc0 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.172+0000 7fe36fe00700 1 -- 172.21.15.63:0/1017226737 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fe368020d70 con 0x7fe37807ccc0 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.177+0000 7fe37ce00700 1 -- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fe35c03e530 msgr2=0x7fe35c040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:17:56.835 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.177+0000 7fe37ce00700 1 --2- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fe35c03e530 0x7fe35c040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:56.836 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.177+0000 7fe37ce00700 1 -- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 msgr2=0x7fe37819db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:17:56.836 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.177+0000 7fe37ce00700 1 --2- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37819db00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe368011670 tx=0x7fe368011a30 comp rx=0 tx=0).stop 2024-05-27T07:17:56.836 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.177+0000 7fe37ce00700 1 -- 172.21.15.63:0/1017226737 shutdown_connections 2024-05-27T07:17:56.836 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.177+0000 7fe37ce00700 1 --2- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7fe35c03e530 0x7fe35c040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:56.836 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.177+0000 7fe37ce00700 1 --2- 172.21.15.63:0/1017226737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37807ccc0 0x7fe37819db00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:17:56.836 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.177+0000 7fe37ce00700 1 -- 172.21.15.63:0/1017226737 >> 172.21.15.63:0/1017226737 conn(0x7fe378103e70 msgr2=0x7fe378104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:17:56.836 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.177+0000 7fe37ce00700 1 -- 172.21.15.63:0/1017226737 shutdown_connections 2024-05-27T07:17:56.836 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:56.177+0000 7fe37ce00700 1 -- 172.21.15.63:0/1017226737 wait complete. 2024-05-27T07:17:56.836 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for the mgr to restart... 2024-05-27T07:17:56.836 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr epoch 9... 2024-05-27T07:17:57.145 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:56 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:56.949+0000 7f16e2f44200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:17:57.404 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:57 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:57.176+0000 7f16e2f44200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:17:57.705 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:57 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:57.588+0000 7f16e2f44200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:17:57.972 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:57 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:57.718+0000 7f16e2f44200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:17:58.241 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:57 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:57.980+0000 7f16e2f44200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:17:58.241 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:58 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:58.249+0000 7f16e2f44200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:17:59.206 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:59 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:59.083+0000 7f16e2f44200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:17:59.705 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:59 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:59.388+0000 7f16e2f44200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:18:00.456 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:17:59 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:17:59.975+0000 7f16e2f44200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:18:00.456 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:00 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:00.111+0000 7f16e2f44200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:18:00.956 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:00 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:00.597+0000 7f16e2f44200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:18:01.455 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:01.079+0000 7f16e2f44200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:18:01.456 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:01.212+0000 7f16e2f44200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:18:01.741 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:01.557+0000 7f16e2f44200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:18:02.154 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:01.749+0000 7f16e2f44200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:18:02.154 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:01.888+0000 7f16e2f44200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:18:02.434 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:02 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:02.308+0000 7f16e2f44200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:18:02.435 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:02 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:02.442+0000 7f16e2f44200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:18:03.456 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:03 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:03.252+0000 7f16e2f44200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:18:03.956 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:18:03 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:18:03.555+0000 7f16e2f44200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:18:03.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:03 smithi063 ceph-mon[31613]: Active manager daemon x restarted 2024-05-27T07:18:03.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:03 smithi063 ceph-mon[31613]: Activating manager daemon x 2024-05-27T07:18:03.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:03 smithi063 ceph-mon[31613]: osdmap e3: 0 total, 0 up, 0 in 2024-05-27T07:18:03.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:03 smithi063 ceph-mon[31613]: mgrmap e10: x(active, starting, since 0.0111772s) 2024-05-27T07:18:03.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:18:03.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:18:03.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:18:03.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:18:03.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:18:03.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:03 smithi063 ceph-mon[31613]: Manager daemon x is now available 2024-05-27T07:18:04.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:18:04.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:18:04.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:18:04.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:04 smithi063 ceph-mon[31613]: mgrmap e11: x(active, since 1.01331s) 2024-05-27T07:18:05.268 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.528+0000 7f776b000700 1 Processor -- start 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.528+0000 7f776b000700 1 -- start start 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.528+0000 7f776b000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f776407d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.529+0000 7f776b000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f776407efd0 con 0x7f776407d230 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.529+0000 7f7763e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f776407d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.529+0000 7f7763e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f776407d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44036/0 (socket says 172.21.15.63:44036) 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.529+0000 7f7763e00700 1 -- 172.21.15.63:0/732384233 learned_addr learned my addr 172.21.15.63:0/732384233 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.530+0000 7f7763e00700 1 -- 172.21.15.63:0/732384233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776407db80 con 0x7f776407d230 2024-05-27T07:18:05.269 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.530+0000 7f7763e00700 1 --2- 172.21.15.63:0/732384233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f776407d640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77540098a0 tx=0x7f7754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4812beb46d02ce34 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.530+0000 7f7763400700 1 -- 172.21.15.63:0/732384233 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7754004160 con 0x7f776407d230 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.531+0000 7f7763400700 1 -- 172.21.15.63:0/732384233 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7754004300 con 0x7f776407d230 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.531+0000 7f7763400700 1 -- 172.21.15.63:0/732384233 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7754011960 con 0x7f776407d230 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.532+0000 7f776b000700 1 -- 172.21.15.63:0/732384233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 msgr2=0x7f776407d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.532+0000 7f776b000700 1 --2- 172.21.15.63:0/732384233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f776407d640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77540098a0 tx=0x7f7754009bb0 comp rx=0 tx=0).stop 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.532+0000 7f776b000700 1 -- 172.21.15.63:0/732384233 shutdown_connections 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.532+0000 7f776b000700 1 --2- 172.21.15.63:0/732384233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f776407d640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.532+0000 7f776b000700 1 -- 172.21.15.63:0/732384233 >> 172.21.15.63:0/732384233 conn(0x7f77641050e0 msgr2=0x7f77641075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.532+0000 7f776b000700 1 -- 172.21.15.63:0/732384233 shutdown_connections 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.532+0000 7f776b000700 1 -- 172.21.15.63:0/732384233 wait complete. 2024-05-27T07:18:05.270 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.534+0000 7f776b000700 1 Processor -- start 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.534+0000 7f776b000700 1 -- start start 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.534+0000 7f776b000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f77641a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.534+0000 7f776b000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77641101a0 con 0x7f776407d230 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.535+0000 7f7763e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f77641a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.535+0000 7f7763e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f77641a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44052/0 (socket says 172.21.15.63:44052) 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.535+0000 7f7763e00700 1 -- 172.21.15.63:0/1697530129 learned_addr learned my addr 172.21.15.63:0/1697530129 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.535+0000 7f7763e00700 1 -- 172.21.15.63:0/1697530129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77641a05a0 con 0x7f776407d230 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.536+0000 7f7763e00700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f77641a0060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7754011670 tx=0x7f7754011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.536+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7754003a60 con 0x7f776407d230 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.536+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f775401b8d0 con 0x7f776407d230 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.536+0000 7f776b000700 1 -- 172.21.15.63:0/1697530129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77641a0830 con 0x7f776407d230 2024-05-27T07:18:05.271 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.536+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7754023750 con 0x7f776407d230 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.536+0000 7f776b000700 1 -- 172.21.15.63:0/1697530129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77641a3440 con 0x7f776407d230 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.538+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f7754023a50 con 0x7f776407d230 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.538+0000 7f7761600700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 0x7f7748040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.538+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 --> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7748041150 con 0x7f774803e530 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.538+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7754054250 con 0x7f776407d230 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.538+0000 7f775b400700 1 -- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 msgr2=0x7f7748040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1689640179 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.538+0000 7f775b400700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 0x7f7748040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.739+0000 7f775b400700 1 -- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 msgr2=0x7f7748040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1689640179 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:57.739+0000 7f775b400700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 0x7f7748040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:58.140+0000 7f775b400700 1 -- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 msgr2=0x7f7748040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1689640179 2024-05-27T07:18:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:58.140+0000 7f775b400700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 0x7f7748040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:58.941+0000 7f775b400700 1 -- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 msgr2=0x7f7748040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1689640179 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:17:58.941+0000 7f775b400700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 0x7f7748040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:00.543+0000 7f775b400700 1 -- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 msgr2=0x7f7748040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/1689640179 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:00.543+0000 7f775b400700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 0x7f7748040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:03.566+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f775402c3f0 con 0x7f776407d230 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:03.567+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 msgr2=0x7f7748040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:03.567+0000 7f7761600700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 0x7f7748040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.569+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f775402ba40 con 0x7f776407d230 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.570+0000 7f7761600700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f7754053de0 0x7f7748043fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.570+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7748041150 con 0x7f7754053de0 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.572+0000 7f775b400700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f7754053de0 0x7f7748043fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:05.273 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.575+0000 7f775b400700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f7754053de0 0x7f7748043fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f774c003a10 tx=0x7f774c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.577+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7748041150 con 0x7f7754053de0 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.586+0000 7f776b000700 1 -- 172.21.15.63:0/1697530129 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7764064e10 con 0x7f7754053de0 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.587+0000 7f7761600700 1 -- 172.21.15.63:0/1697530129 <== mgr.14152 v2:172.21.15.63:6800/675305989 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f7764064e10 con 0x7f7754053de0 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 -- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f7754053de0 msgr2=0x7f7748043fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f7754053de0 0x7f7748043fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f774c003a10 tx=0x7f774c0092b0 comp rx=0 tx=0).stop 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 -- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 msgr2=0x7f77641a0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f77641a0060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7754011670 tx=0x7f7754011a30 comp rx=0 tx=0).stop 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 -- 172.21.15.63:0/1697530129 shutdown_connections 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f7754053de0 0x7f7748043fc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776407d230 0x7f77641a0060 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 --2- 172.21.15.63:0/1697530129 >> [v2:172.21.15.63:6800/1689640179,v1:172.21.15.63:6801/1689640179] conn(0x7f774803e530 0x7f7748040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 -- 172.21.15.63:0/1697530129 >> 172.21.15.63:0/1697530129 conn(0x7f77641050e0 msgr2=0x7f77641058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:05.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 -- 172.21.15.63:0/1697530129 shutdown_connections 2024-05-27T07:18:05.275 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:04.588+0000 7f776b000700 1 -- 172.21.15.63:0/1697530129 wait complete. 2024-05-27T07:18:05.275 INFO:teuthology.orchestra.run.smithi063.stdout:mgr epoch 9 is available 2024-05-27T07:18:05.275 INFO:teuthology.orchestra.run.smithi063.stdout:Generating a dashboard self-signed certificate... 2024-05-27T07:18:05.913 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:05 smithi063 ceph-mon[31613]: [27/May/2024:07:18:03] ENGINE Bus STARTING 2024-05-27T07:18:05.914 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:05 smithi063 ceph-mon[31613]: [27/May/2024:07:18:04] ENGINE Serving on https://172.21.15.63:7150 2024-05-27T07:18:05.914 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:05 smithi063 ceph-mon[31613]: [27/May/2024:07:18:04] ENGINE Serving on http://172.21.15.63:8765 2024-05-27T07:18:05.914 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:05 smithi063 ceph-mon[31613]: [27/May/2024:07:18:04] ENGINE Bus STARTED 2024-05-27T07:18:05.914 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:05 smithi063 ceph-mon[31613]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T07:18:05.914 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:05 smithi063 ceph-mon[31613]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T07:18:06.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:06 smithi063 ceph-mon[31613]: mgrmap e12: x(active, since 2s) 2024-05-27T07:18:06.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:06.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:06.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:07.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-27T07:18:07.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.166+0000 7f8250800700 1 Processor -- start 2024-05-27T07:18:07.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.167+0000 7f8250800700 1 -- start start 2024-05-27T07:18:07.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.167+0000 7f8250800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f8248112c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:07.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.167+0000 7f8250800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8248113250 con 0x7f82481107e0 2024-05-27T07:18:07.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.167+0000 7f824de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f8248112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:07.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.167+0000 7f824de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f8248112c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38830/0 (socket says 172.21.15.63:38830) 2024-05-27T07:18:07.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.167+0000 7f824de00700 1 -- 172.21.15.63:0/4177750005 learned_addr learned my addr 172.21.15.63:0/4177750005 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:07.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.168+0000 7f824de00700 1 -- 172.21.15.63:0/4177750005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481133d0 con 0x7f82481107e0 2024-05-27T07:18:07.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.168+0000 7f824de00700 1 --2- 172.21.15.63:0/4177750005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f8248112c40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f823c0098a0 tx=0x7f823c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc14d23445a3968 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:07.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.169+0000 7f824ca00700 1 -- 172.21.15.63:0/4177750005 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823c004160 con 0x7f82481107e0 2024-05-27T07:18:07.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.169+0000 7f824ca00700 1 -- 172.21.15.63:0/4177750005 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f823c004300 con 0x7f82481107e0 2024-05-27T07:18:07.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.169+0000 7f824ca00700 1 -- 172.21.15.63:0/4177750005 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823c011960 con 0x7f82481107e0 2024-05-27T07:18:07.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.169+0000 7f8250800700 1 -- 172.21.15.63:0/4177750005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 msgr2=0x7f8248112c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:07.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.169+0000 7f8250800700 1 --2- 172.21.15.63:0/4177750005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f8248112c40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f823c0098a0 tx=0x7f823c009bb0 comp rx=0 tx=0).stop 2024-05-27T07:18:07.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.170+0000 7f8250800700 1 -- 172.21.15.63:0/4177750005 shutdown_connections 2024-05-27T07:18:07.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.170+0000 7f8250800700 1 --2- 172.21.15.63:0/4177750005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f8248112c40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:07.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.170+0000 7f8250800700 1 -- 172.21.15.63:0/4177750005 >> 172.21.15.63:0/4177750005 conn(0x7f824807f0f0 msgr2=0x7f82480815a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:07.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.170+0000 7f8250800700 1 -- 172.21.15.63:0/4177750005 shutdown_connections 2024-05-27T07:18:07.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.170+0000 7f8250800700 1 -- 172.21.15.63:0/4177750005 wait complete. 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.171+0000 7f8250800700 1 Processor -- start 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.171+0000 7f8250800700 1 -- start start 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.172+0000 7f8250800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f82481a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.172+0000 7f8250800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8248114430 con 0x7f82481107e0 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.172+0000 7f824de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f82481a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.172+0000 7f824de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f82481a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38836/0 (socket says 172.21.15.63:38836) 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.172+0000 7f824de00700 1 -- 172.21.15.63:0/2816447805 learned_addr learned my addr 172.21.15.63:0/2816447805 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.172+0000 7f824de00700 1 -- 172.21.15.63:0/2816447805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481a4840 con 0x7f82481107e0 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.173+0000 7f824de00700 1 --2- 172.21.15.63:0/2816447805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f82481a4300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f823c011670 tx=0x7f823c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.173+0000 7f8242a00700 1 -- 172.21.15.63:0/2816447805 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823c003a60 con 0x7f82481107e0 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.173+0000 7f8242a00700 1 -- 172.21.15.63:0/2816447805 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f823c01b8d0 con 0x7f82481107e0 2024-05-27T07:18:07.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.173+0000 7f8242a00700 1 -- 172.21.15.63:0/2816447805 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823c023790 con 0x7f82481107e0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.173+0000 7f8250800700 1 -- 172.21.15.63:0/2816447805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82481a4ad0 con 0x7f82481107e0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.174+0000 7f8250800700 1 -- 172.21.15.63:0/2816447805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82481a76e0 con 0x7f82481107e0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.175+0000 7f8250800700 1 -- 172.21.15.63:0/2816447805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8248064e10 con 0x7f82481107e0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.175+0000 7f8242a00700 1 -- 172.21.15.63:0/2816447805 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f823c01b440 con 0x7f82481107e0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.176+0000 7f8242a00700 1 --2- 172.21.15.63:0/2816447805 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f823003e5e0 0x7f8230040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.176+0000 7f8242a00700 1 -- 172.21.15.63:0/2816447805 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f823c054100 con 0x7f82481107e0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.176+0000 7f824d400700 1 --2- 172.21.15.63:0/2816447805 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f823003e5e0 0x7f8230040af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.177+0000 7f824d400700 1 --2- 172.21.15.63:0/2816447805 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f823003e5e0 0x7f8230040af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8238006fd0 tx=0x7f8238006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.183+0000 7f8242a00700 1 -- 172.21.15.63:0/2816447805 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f823c01f030 con 0x7f82481107e0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.429+0000 7f8250800700 1 -- 172.21.15.63:0/2816447805 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f8248080060 con 0x7f823003e5e0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.577+0000 7f8242a00700 1 -- 172.21.15.63:0/2816447805 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f8248080060 con 0x7f823003e5e0 2024-05-27T07:18:07.206 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.581+0000 7f8250800700 1 -- 172.21.15.63:0/2816447805 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f823003e5e0 msgr2=0x7f8230040af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:07.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.581+0000 7f8250800700 1 --2- 172.21.15.63:0/2816447805 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f823003e5e0 0x7f8230040af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8238006fd0 tx=0x7f8238006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:07.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.581+0000 7f8250800700 1 -- 172.21.15.63:0/2816447805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 msgr2=0x7f82481a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:07.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.581+0000 7f8250800700 1 --2- 172.21.15.63:0/2816447805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f82481a4300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f823c011670 tx=0x7f823c011a30 comp rx=0 tx=0).stop 2024-05-27T07:18:07.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.581+0000 7f8250800700 1 -- 172.21.15.63:0/2816447805 shutdown_connections 2024-05-27T07:18:07.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.582+0000 7f8250800700 1 --2- 172.21.15.63:0/2816447805 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f823003e5e0 0x7f8230040af0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:07.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.582+0000 7f8250800700 1 --2- 172.21.15.63:0/2816447805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82481107e0 0x7f82481a4300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:07.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.582+0000 7f8250800700 1 -- 172.21.15.63:0/2816447805 >> 172.21.15.63:0/2816447805 conn(0x7f824807f0f0 msgr2=0x7f824807f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:07.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.582+0000 7f8250800700 1 -- 172.21.15.63:0/2816447805 shutdown_connections 2024-05-27T07:18:07.207 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:06.582+0000 7f8250800700 1 -- 172.21.15.63:0/2816447805 wait complete. 2024-05-27T07:18:07.207 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial admin user... 2024-05-27T07:18:07.879 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:07 smithi063 ceph-mon[31613]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:18:09.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Ab.k4ptp7k/8/U2cwfVTa.TJT.y9GUAFqHZEABL71nJi9e93usdMG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716794288, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-27T07:18:09.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.888+0000 7f280cc00700 1 Processor -- start 2024-05-27T07:18:09.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.888+0000 7f280cc00700 1 -- start start 2024-05-27T07:18:09.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.889+0000 7f280cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280810c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:09.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.889+0000 7f280cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f280807ce70 con 0x7f280810a560 2024-05-27T07:18:09.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.889+0000 7f2805e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280810c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:09.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.890+0000 7f2805e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280810c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38842/0 (socket says 172.21.15.63:38842) 2024-05-27T07:18:09.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.890+0000 7f2805e00700 1 -- 172.21.15.63:0/2139372385 learned_addr learned my addr 172.21.15.63:0/2139372385 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:09.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.890+0000 7f2805e00700 1 -- 172.21.15.63:0/2139372385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280810cec0 con 0x7f280810a560 2024-05-27T07:18:09.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.890+0000 7f2805e00700 1 --2- 172.21.15.63:0/2139372385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280810c980 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27f80098a0 tx=0x7f27f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e18ba2dba6c705b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.891+0000 7f2804a00700 1 -- 172.21.15.63:0/2139372385 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8004160 con 0x7f280810a560 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.891+0000 7f2804a00700 1 -- 172.21.15.63:0/2139372385 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27f8004300 con 0x7f280810a560 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.891+0000 7f2804a00700 1 -- 172.21.15.63:0/2139372385 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8011960 con 0x7f280810a560 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.892+0000 7f280cc00700 1 -- 172.21.15.63:0/2139372385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 msgr2=0x7f280810c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.892+0000 7f280cc00700 1 --2- 172.21.15.63:0/2139372385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280810c980 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27f80098a0 tx=0x7f27f8009bb0 comp rx=0 tx=0).stop 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.892+0000 7f280cc00700 1 -- 172.21.15.63:0/2139372385 shutdown_connections 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.892+0000 7f280cc00700 1 --2- 172.21.15.63:0/2139372385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280810c980 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.892+0000 7f280cc00700 1 -- 172.21.15.63:0/2139372385 >> 172.21.15.63:0/2139372385 conn(0x7f2808104060 msgr2=0x7f28081064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.892+0000 7f280cc00700 1 -- 172.21.15.63:0/2139372385 shutdown_connections 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.892+0000 7f280cc00700 1 -- 172.21.15.63:0/2139372385 wait complete. 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.894+0000 7f280cc00700 1 Processor -- start 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.894+0000 7f280cc00700 1 -- start start 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.895+0000 7f280cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.895+0000 7f280cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f280810de60 con 0x7f280810a560 2024-05-27T07:18:09.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.895+0000 7f2805e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.895+0000 7f2805e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280819dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38854/0 (socket says 172.21.15.63:38854) 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.895+0000 7f2805e00700 1 -- 172.21.15.63:0/589922521 learned_addr learned my addr 172.21.15.63:0/589922521 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.896+0000 7f2805e00700 1 -- 172.21.15.63:0/589922521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280819e2a0 con 0x7f280810a560 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.896+0000 7f2805e00700 1 --2- 172.21.15.63:0/589922521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280819dd60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27f8011670 tx=0x7f27f8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.896+0000 7f27fea00700 1 -- 172.21.15.63:0/589922521 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8003a60 con 0x7f280810a560 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.896+0000 7f27fea00700 1 -- 172.21.15.63:0/589922521 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27f801b8d0 con 0x7f280810a560 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.896+0000 7f27fea00700 1 -- 172.21.15.63:0/589922521 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8023790 con 0x7f280810a560 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.896+0000 7f280cc00700 1 -- 172.21.15.63:0/589922521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280819e530 con 0x7f280810a560 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.897+0000 7f280cc00700 1 -- 172.21.15.63:0/589922521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28081a1130 con 0x7f280810a560 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.898+0000 7f27fea00700 1 -- 172.21.15.63:0/589922521 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f27f801b440 con 0x7f280810a560 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.898+0000 7f280cc00700 1 -- 172.21.15.63:0/589922521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2808064e10 con 0x7f280810a560 2024-05-27T07:18:09.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.899+0000 7f27fea00700 1 --2- 172.21.15.63:0/589922521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27ec03e5e0 0x7f27ec040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.899+0000 7f27fea00700 1 -- 172.21.15.63:0/589922521 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27f8054050 con 0x7f280810a560 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.899+0000 7f2805400700 1 --2- 172.21.15.63:0/589922521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27ec03e5e0 0x7f27ec040af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.900+0000 7f2805400700 1 --2- 172.21.15.63:0/589922521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27ec03e5e0 0x7f27ec040af0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27f4006fd0 tx=0x7f27f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:07.905+0000 7f27fea00700 1 -- 172.21.15.63:0/589922521 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27f801f030 con 0x7f280810a560 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.163+0000 7f280cc00700 1 -- 172.21.15.63:0/589922521 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f28081a13c0 con 0x7f27ec03e5e0 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.446+0000 7f27fea00700 1 -- 172.21.15.63:0/589922521 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f28081a13c0 con 0x7f27ec03e5e0 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.449+0000 7f280cc00700 1 -- 172.21.15.63:0/589922521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27ec03e5e0 msgr2=0x7f27ec040af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.449+0000 7f280cc00700 1 --2- 172.21.15.63:0/589922521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27ec03e5e0 0x7f27ec040af0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27f4006fd0 tx=0x7f27f4006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.449+0000 7f280cc00700 1 -- 172.21.15.63:0/589922521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 msgr2=0x7f280819dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.449+0000 7f280cc00700 1 --2- 172.21.15.63:0/589922521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280819dd60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27f8011670 tx=0x7f27f8011a30 comp rx=0 tx=0).stop 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.449+0000 7f280cc00700 1 -- 172.21.15.63:0/589922521 shutdown_connections 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.449+0000 7f280cc00700 1 --2- 172.21.15.63:0/589922521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27ec03e5e0 0x7f27ec040af0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.449+0000 7f280cc00700 1 --2- 172.21.15.63:0/589922521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280810a560 0x7f280819dd60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:09.139 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.449+0000 7f280cc00700 1 -- 172.21.15.63:0/589922521 >> 172.21.15.63:0/589922521 conn(0x7f2808104060 msgr2=0x7f2808105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:09.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.449+0000 7f280cc00700 1 -- 172.21.15.63:0/589922521 shutdown_connections 2024-05-27T07:18:09.140 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:08.450+0000 7f280cc00700 1 -- 172.21.15.63:0/589922521 wait complete. 2024-05-27T07:18:09.140 INFO:teuthology.orchestra.run.smithi063.stdout:Fetching dashboard port number... 2024-05-27T07:18:09.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:09 smithi063 ceph-mon[31613]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:18:09.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:09 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:09.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:09 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:10.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:10 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2297294080' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-27T07:18:11.515 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 8443 2024-05-27T07:18:11.515 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.848+0000 7f59f7400700 1 Processor -- start 2024-05-27T07:18:11.515 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.848+0000 7f59f7400700 1 -- start start 2024-05-27T07:18:11.515 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.848+0000 7f59f7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f0112c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:11.517 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.848+0000 7f59f7400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0113250 con 0x7f59f01107e0 2024-05-27T07:18:11.517 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.848+0000 7f59f4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f0112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:11.517 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.848+0000 7f59f4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f0112c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52174/0 (socket says 172.21.15.63:52174) 2024-05-27T07:18:11.517 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.849+0000 7f59f4a00700 1 -- 172.21.15.63:0/1192577944 learned_addr learned my addr 172.21.15.63:0/1192577944 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:11.517 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.849+0000 7f59f4a00700 1 -- 172.21.15.63:0/1192577944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f01133d0 con 0x7f59f01107e0 2024-05-27T07:18:11.517 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.849+0000 7f59f4a00700 1 --2- 172.21.15.63:0/1192577944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f0112c40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f59e40098a0 tx=0x7f59e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a810fabb455f5bf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:11.517 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.850+0000 7f59ef400700 1 -- 172.21.15.63:0/1192577944 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e4004160 con 0x7f59f01107e0 2024-05-27T07:18:11.517 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.850+0000 7f59ef400700 1 -- 172.21.15.63:0/1192577944 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59e4004300 con 0x7f59f01107e0 2024-05-27T07:18:11.518 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.850+0000 7f59ef400700 1 -- 172.21.15.63:0/1192577944 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e4011960 con 0x7f59f01107e0 2024-05-27T07:18:11.518 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.850+0000 7f59f7400700 1 -- 172.21.15.63:0/1192577944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 msgr2=0x7f59f0112c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:11.518 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.850+0000 7f59f7400700 1 --2- 172.21.15.63:0/1192577944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f0112c40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f59e40098a0 tx=0x7f59e4009bb0 comp rx=0 tx=0).stop 2024-05-27T07:18:11.518 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.851+0000 7f59f7400700 1 -- 172.21.15.63:0/1192577944 shutdown_connections 2024-05-27T07:18:11.518 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.851+0000 7f59f7400700 1 --2- 172.21.15.63:0/1192577944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f0112c40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.851+0000 7f59f7400700 1 -- 172.21.15.63:0/1192577944 >> 172.21.15.63:0/1192577944 conn(0x7f59f007f0f0 msgr2=0x7f59f00815a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.851+0000 7f59f7400700 1 -- 172.21.15.63:0/1192577944 shutdown_connections 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.851+0000 7f59f7400700 1 -- 172.21.15.63:0/1192577944 wait complete. 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.852+0000 7f59f7400700 1 Processor -- start 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.852+0000 7f59f7400700 1 -- start start 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.852+0000 7f59f7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f01a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.852+0000 7f59f7400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0114430 con 0x7f59f01107e0 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.853+0000 7f59f4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f01a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.853+0000 7f59f4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f01a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52190/0 (socket says 172.21.15.63:52190) 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.853+0000 7f59f4a00700 1 -- 172.21.15.63:0/2297294080 learned_addr learned my addr 172.21.15.63:0/2297294080 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.853+0000 7f59f4a00700 1 -- 172.21.15.63:0/2297294080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f01a4840 con 0x7f59f01107e0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.853+0000 7f59f4a00700 1 --2- 172.21.15.63:0/2297294080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f01a4300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f59e4011670 tx=0x7f59e4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.854+0000 7f59ed600700 1 -- 172.21.15.63:0/2297294080 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e4003a60 con 0x7f59f01107e0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.854+0000 7f59ed600700 1 -- 172.21.15.63:0/2297294080 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59e401b8d0 con 0x7f59f01107e0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.854+0000 7f59ed600700 1 -- 172.21.15.63:0/2297294080 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e4023790 con 0x7f59f01107e0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.854+0000 7f59f7400700 1 -- 172.21.15.63:0/2297294080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f01a4ad0 con 0x7f59f01107e0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.854+0000 7f59f7400700 1 -- 172.21.15.63:0/2297294080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f01a76e0 con 0x7f59f01107e0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.855+0000 7f59f7400700 1 -- 172.21.15.63:0/2297294080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f0064e10 con 0x7f59f01107e0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.855+0000 7f59ed600700 1 -- 172.21.15.63:0/2297294080 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f59e401b440 con 0x7f59f01107e0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.856+0000 7f59ed600700 1 --2- 172.21.15.63:0/2297294080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f59d803e5e0 0x7f59d8040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.856+0000 7f59ed600700 1 -- 172.21.15.63:0/2297294080 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f59e40541b0 con 0x7f59f01107e0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.856+0000 7f59efe00700 1 --2- 172.21.15.63:0/2297294080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f59d803e5e0 0x7f59d8040af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:11.523 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.856+0000 7f59efe00700 1 --2- 172.21.15.63:0/2297294080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f59d803e5e0 0x7f59d8040af0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f59dc006fd0 tx=0x7f59dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:09.861+0000 7f59ed600700 1 -- 172.21.15.63:0/2297294080 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f59e401f030 con 0x7f59f01107e0 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.089+0000 7f59f7400700 1 -- 172.21.15.63:0/2297294080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f59f01a78b0 con 0x7f59f01107e0 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.090+0000 7f59ed600700 1 -- 172.21.15.63:0/2297294080 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f59e4016260 con 0x7f59f01107e0 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.093+0000 7f59f7400700 1 -- 172.21.15.63:0/2297294080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f59d803e5e0 msgr2=0x7f59d8040af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.093+0000 7f59f7400700 1 --2- 172.21.15.63:0/2297294080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f59d803e5e0 0x7f59d8040af0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f59dc006fd0 tx=0x7f59dc006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.094+0000 7f59f7400700 1 -- 172.21.15.63:0/2297294080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 msgr2=0x7f59f01a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.094+0000 7f59f7400700 1 --2- 172.21.15.63:0/2297294080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f01a4300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f59e4011670 tx=0x7f59e4011a30 comp rx=0 tx=0).stop 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.094+0000 7f59f7400700 1 -- 172.21.15.63:0/2297294080 shutdown_connections 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.094+0000 7f59f7400700 1 --2- 172.21.15.63:0/2297294080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f59d803e5e0 0x7f59d8040af0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.094+0000 7f59f7400700 1 --2- 172.21.15.63:0/2297294080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59f01107e0 0x7f59f01a4300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.094+0000 7f59f7400700 1 -- 172.21.15.63:0/2297294080 >> 172.21.15.63:0/2297294080 conn(0x7f59f007f0f0 msgr2=0x7f59f007f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.094+0000 7f59f7400700 1 -- 172.21.15.63:0/2297294080 shutdown_connections 2024-05-27T07:18:11.525 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:10.094+0000 7f59f7400700 1 -- 172.21.15.63:0/2297294080 wait complete. 2024-05-27T07:18:11.549 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T07:18:11.549 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-05-27T07:18:11.572 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T07:18:11.572 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-05-27T07:18:11.573 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-05-27T07:18:11.573 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-27T07:18:11.581 INFO:teuthology.orchestra.run.smithi063.stdout:Ceph Dashboard is now available at: 2024-05-27T07:18:11.581 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:11.581 INFO:teuthology.orchestra.run.smithi063.stdout: URL: https://smithi063.front.sepia.ceph.com:8443/ 2024-05-27T07:18:11.581 INFO:teuthology.orchestra.run.smithi063.stdout: User: admin 2024-05-27T07:18:11.582 INFO:teuthology.orchestra.run.smithi063.stdout: Password: fnzryn7mt0 2024-05-27T07:18:11.582 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:11.582 INFO:teuthology.orchestra.run.smithi063.stdout:Saving cluster configuration to /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config directory 2024-05-27T07:18:11.584 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling autotune for osd_memory_target 2024-05-27T07:18:11.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:11 smithi063 ceph-mon[31613]: mgrmap e13: x(active, since 6s) 2024-05-27T07:18:13.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.294+0000 7ff2ea600700 1 Processor -- start 2024-05-27T07:18:13.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.295+0000 7ff2ea600700 1 -- start start 2024-05-27T07:18:13.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.295+0000 7ff2ea600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e407d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:13.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.295+0000 7ff2ea600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e407d860 con 0x7ff2e407ee60 2024-05-27T07:18:13.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.296+0000 7ff2e9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:13.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.296+0000 7ff2e9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e407d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52196/0 (socket says 172.21.15.63:52196) 2024-05-27T07:18:13.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.296+0000 7ff2e9200700 1 -- 172.21.15.63:0/37465919 learned_addr learned my addr 172.21.15.63:0/37465919 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:13.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.296+0000 7ff2e9200700 1 -- 172.21.15.63:0/37465919 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e407d9e0 con 0x7ff2e407ee60 2024-05-27T07:18:13.238 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.297+0000 7ff2e9200700 1 --2- 172.21.15.63:0/37465919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e407d290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009e80 tx=0x7ff2d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50754cdd487588d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:13.239 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.297+0000 7ff2e3e00700 1 -- 172.21.15.63:0/37465919 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2d4017070 con 0x7ff2e407ee60 2024-05-27T07:18:13.239 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.297+0000 7ff2e3e00700 1 -- 172.21.15.63:0/37465919 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2d400b8c0 con 0x7ff2e407ee60 2024-05-27T07:18:13.239 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.297+0000 7ff2e3e00700 1 -- 172.21.15.63:0/37465919 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2d4004420 con 0x7ff2e407ee60 2024-05-27T07:18:13.243 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.298+0000 7ff2ea600700 1 -- 172.21.15.63:0/37465919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 msgr2=0x7ff2e407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:13.243 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.298+0000 7ff2ea600700 1 --2- 172.21.15.63:0/37465919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e407d290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009e80 tx=0x7ff2d400b330 comp rx=0 tx=0).stop 2024-05-27T07:18:13.243 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.298+0000 7ff2ea600700 1 -- 172.21.15.63:0/37465919 shutdown_connections 2024-05-27T07:18:13.243 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.298+0000 7ff2ea600700 1 --2- 172.21.15.63:0/37465919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e407d290 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:13.243 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.298+0000 7ff2ea600700 1 -- 172.21.15.63:0/37465919 >> 172.21.15.63:0/37465919 conn(0x7ff2e4104f40 msgr2=0x7ff2e4107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:13.243 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.298+0000 7ff2ea600700 1 -- 172.21.15.63:0/37465919 shutdown_connections 2024-05-27T07:18:13.243 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.298+0000 7ff2ea600700 1 -- 172.21.15.63:0/37465919 wait complete. 2024-05-27T07:18:13.243 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.299+0000 7ff2ea600700 1 Processor -- start 2024-05-27T07:18:13.243 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.300+0000 7ff2ea600700 1 -- start start 2024-05-27T07:18:13.243 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.300+0000 7ff2ea600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e41a4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.300+0000 7ff2ea600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e410bab0 con 0x7ff2e407ee60 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.300+0000 7ff2e9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e41a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.300+0000 7ff2e9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e41a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52206/0 (socket says 172.21.15.63:52206) 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.300+0000 7ff2e9200700 1 -- 172.21.15.63:0/2761916040 learned_addr learned my addr 172.21.15.63:0/2761916040 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.300+0000 7ff2e9200700 1 -- 172.21.15.63:0/2761916040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e41a46d0 con 0x7ff2e407ee60 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.301+0000 7ff2e9200700 1 --2- 172.21.15.63:0/2761916040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e41a4190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4004000 tx=0x7ff2d400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.301+0000 7ff2e2000700 1 -- 172.21.15.63:0/2761916040 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2d4020020 con 0x7ff2e407ee60 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.301+0000 7ff2e2000700 1 -- 172.21.15.63:0/2761916040 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2d40128c0 con 0x7ff2e407ee60 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.301+0000 7ff2ea600700 1 -- 172.21.15.63:0/2761916040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e41a4960 con 0x7ff2e407ee60 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.301+0000 7ff2e2000700 1 -- 172.21.15.63:0/2761916040 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2d4017040 con 0x7ff2e407ee60 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.301+0000 7ff2ea600700 1 -- 172.21.15.63:0/2761916040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e41a7570 con 0x7ff2e407ee60 2024-05-27T07:18:13.244 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.303+0000 7ff2e2000700 1 -- 172.21.15.63:0/2761916040 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff2d4023020 con 0x7ff2e407ee60 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.303+0000 7ff2ea600700 1 -- 172.21.15.63:0/2761916040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e4064e10 con 0x7ff2e407ee60 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.303+0000 7ff2e2000700 1 --2- 172.21.15.63:0/2761916040 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff2c803e620 0x7ff2c8040b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.303+0000 7ff2e2000700 1 -- 172.21.15.63:0/2761916040 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff2d400f050 con 0x7ff2e407ee60 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.304+0000 7ff2dbe00700 1 --2- 172.21.15.63:0/2761916040 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff2c803e620 0x7ff2c8040b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.304+0000 7ff2dbe00700 1 --2- 172.21.15.63:0/2761916040 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff2c803e620 0x7ff2c8040b30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc006fd0 tx=0x7ff2cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.309+0000 7ff2e2000700 1 -- 172.21.15.63:0/2761916040 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2d402a020 con 0x7ff2e407ee60 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.547+0000 7ff2ea600700 1 -- 172.21.15.63:0/2761916040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff2e402e910 con 0x7ff2e407ee60 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.548+0000 7ff2e2000700 1 -- 172.21.15.63:0/2761916040 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff2d40119d0 con 0x7ff2e407ee60 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.552+0000 7ff2ea600700 1 -- 172.21.15.63:0/2761916040 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff2c803e620 msgr2=0x7ff2c8040b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.552+0000 7ff2ea600700 1 --2- 172.21.15.63:0/2761916040 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff2c803e620 0x7ff2c8040b30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc006fd0 tx=0x7ff2cc006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.552+0000 7ff2ea600700 1 -- 172.21.15.63:0/2761916040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 msgr2=0x7ff2e41a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.552+0000 7ff2ea600700 1 --2- 172.21.15.63:0/2761916040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e41a4190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4004000 tx=0x7ff2d400bd50 comp rx=0 tx=0).stop 2024-05-27T07:18:13.245 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.553+0000 7ff2ea600700 1 -- 172.21.15.63:0/2761916040 shutdown_connections 2024-05-27T07:18:13.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.553+0000 7ff2ea600700 1 --2- 172.21.15.63:0/2761916040 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff2c803e620 0x7ff2c8040b30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:13.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.553+0000 7ff2ea600700 1 --2- 172.21.15.63:0/2761916040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e407ee60 0x7ff2e41a4190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:13.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.553+0000 7ff2ea600700 1 -- 172.21.15.63:0/2761916040 >> 172.21.15.63:0/2761916040 conn(0x7ff2e4104f40 msgr2=0x7ff2e419ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:13.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.553+0000 7ff2ea600700 1 -- 172.21.15.63:0/2761916040 shutdown_connections 2024-05-27T07:18:13.246 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:12.553+0000 7ff2ea600700 1 -- 172.21.15.63:0/2761916040 wait complete. 2024-05-27T07:18:15.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.914+0000 7fc781c00700 1 Processor -- start 2024-05-27T07:18:15.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.915+0000 7fc781c00700 1 -- start start 2024-05-27T07:18:15.035 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.915+0000 7fc781c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c10bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:15.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.915+0000 7fc781c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c10c4c0 con 0x7fc77c10bac0 2024-05-27T07:18:15.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.916+0000 7fc77be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c10bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:15.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.916+0000 7fc77be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c10bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52220/0 (socket says 172.21.15.63:52220) 2024-05-27T07:18:15.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.916+0000 7fc77be00700 1 -- 172.21.15.63:0/4233394286 learned_addr learned my addr 172.21.15.63:0/4233394286 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:15.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.916+0000 7fc77be00700 1 -- 172.21.15.63:0/4233394286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c10cc90 con 0x7fc77c10bac0 2024-05-27T07:18:15.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.917+0000 7fc77be00700 1 --2- 172.21.15.63:0/4233394286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c10bed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0098a0 tx=0x7fc76c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362c984c7ad3e67b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:15.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.917+0000 7fc77aa00700 1 -- 172.21.15.63:0/4233394286 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc76c004160 con 0x7fc77c10bac0 2024-05-27T07:18:15.036 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.917+0000 7fc77aa00700 1 -- 172.21.15.63:0/4233394286 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc76c004300 con 0x7fc77c10bac0 2024-05-27T07:18:15.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.917+0000 7fc77aa00700 1 -- 172.21.15.63:0/4233394286 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc76c011960 con 0x7fc77c10bac0 2024-05-27T07:18:15.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.918+0000 7fc781c00700 1 -- 172.21.15.63:0/4233394286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 msgr2=0x7fc77c10bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:15.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.918+0000 7fc781c00700 1 --2- 172.21.15.63:0/4233394286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c10bed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0098a0 tx=0x7fc76c009bb0 comp rx=0 tx=0).stop 2024-05-27T07:18:15.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.919+0000 7fc781c00700 1 -- 172.21.15.63:0/4233394286 shutdown_connections 2024-05-27T07:18:15.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.919+0000 7fc781c00700 1 --2- 172.21.15.63:0/4233394286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c10bed0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:15.040 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.919+0000 7fc781c00700 1 -- 172.21.15.63:0/4233394286 >> 172.21.15.63:0/4233394286 conn(0x7fc77c07f0b0 msgr2=0x7fc77c109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.919+0000 7fc781c00700 1 -- 172.21.15.63:0/4233394286 shutdown_connections 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.919+0000 7fc781c00700 1 -- 172.21.15.63:0/4233394286 wait complete. 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.920+0000 7fc781c00700 1 Processor -- start 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.920+0000 7fc781c00700 1 -- start start 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.921+0000 7fc781c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.921+0000 7fc781c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c110040 con 0x7fc77c10bac0 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.921+0000 7fc77be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.921+0000 7fc77be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c19fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52230/0 (socket says 172.21.15.63:52230) 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.921+0000 7fc77be00700 1 -- 172.21.15.63:0/348762912 learned_addr learned my addr 172.21.15.63:0/348762912 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.922+0000 7fc77be00700 1 -- 172.21.15.63:0/348762912 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c1a02b0 con 0x7fc77c10bac0 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.922+0000 7fc77be00700 1 --2- 172.21.15.63:0/348762912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c19fd70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc76c011980 tx=0x7fc76c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.922+0000 7fc778c00700 1 -- 172.21.15.63:0/348762912 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc76c01f070 con 0x7fc77c10bac0 2024-05-27T07:18:15.041 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.922+0000 7fc778c00700 1 -- 172.21.15.63:0/348762912 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc76c003aa0 con 0x7fc77c10bac0 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.922+0000 7fc778c00700 1 -- 172.21.15.63:0/348762912 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc76c019560 con 0x7fc77c10bac0 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.923+0000 7fc781c00700 1 -- 172.21.15.63:0/348762912 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77c1a0540 con 0x7fc77c10bac0 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.923+0000 7fc781c00700 1 -- 172.21.15.63:0/348762912 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77c1a3150 con 0x7fc77c10bac0 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.924+0000 7fc781c00700 1 -- 172.21.15.63:0/348762912 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77c064e10 con 0x7fc77c10bac0 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.924+0000 7fc778c00700 1 -- 172.21.15.63:0/348762912 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc76c022070 con 0x7fc77c10bac0 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.925+0000 7fc778c00700 1 --2- 172.21.15.63:0/348762912 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc76003e670 0x7fc760040b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.925+0000 7fc778c00700 1 -- 172.21.15.63:0/348762912 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc76c053b40 con 0x7fc77c10bac0 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.925+0000 7fc77b400700 1 --2- 172.21.15.63:0/348762912 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc76003e670 0x7fc760040b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.926+0000 7fc77b400700 1 --2- 172.21.15.63:0/348762912 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc76003e670 0x7fc760040b80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc768006fd0 tx=0x7fc768006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:13.930+0000 7fc778c00700 1 -- 172.21.15.63:0/348762912 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc76c023aa0 con 0x7fc77c10bac0 2024-05-27T07:18:15.042 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.299+0000 7fc781c00700 1 -- 172.21.15.63:0/348762912 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc77c1a3320 con 0x7fc77c10bac0 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.306+0000 7fc778c00700 1 -- 172.21.15.63:0/348762912 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fc76c01ba80 con 0x7fc77c10bac0 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.309+0000 7fc781c00700 1 -- 172.21.15.63:0/348762912 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc76003e670 msgr2=0x7fc760040b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.309+0000 7fc781c00700 1 --2- 172.21.15.63:0/348762912 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc76003e670 0x7fc760040b80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc768006fd0 tx=0x7fc768006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.309+0000 7fc781c00700 1 -- 172.21.15.63:0/348762912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 msgr2=0x7fc77c19fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.309+0000 7fc781c00700 1 --2- 172.21.15.63:0/348762912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c19fd70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc76c011980 tx=0x7fc76c011a60 comp rx=0 tx=0).stop 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.309+0000 7fc781c00700 1 -- 172.21.15.63:0/348762912 shutdown_connections 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.309+0000 7fc781c00700 1 --2- 172.21.15.63:0/348762912 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc76003e670 0x7fc760040b80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.309+0000 7fc781c00700 1 --2- 172.21.15.63:0/348762912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc77c10bac0 0x7fc77c19fd70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.309+0000 7fc781c00700 1 -- 172.21.15.63:0/348762912 >> 172.21.15.63:0/348762912 conn(0x7fc77c07f0b0 msgr2=0x7fc77c109280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.310+0000 7fc781c00700 1 -- 172.21.15.63:0/348762912 shutdown_connections 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-05-27T07:18:14.310+0000 7fc781c00700 1 -- 172.21.15.63:0/348762912 wait complete. 2024-05-27T07:18:15.043 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-27T07:18:15.044 INFO:teuthology.orchestra.run.smithi063.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-27T07:18:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:18:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:15.044 INFO:teuthology.orchestra.run.smithi063.stdout:Or, if you are only running a single cluster on this host: 2024-05-27T07:18:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: ceph telemetry on 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout:For more information see: 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:15.045 INFO:teuthology.orchestra.run.smithi063.stdout:Bootstrap complete. 2024-05-27T07:18:15.137 INFO:tasks.cephadm:Fetching config... 2024-05-27T07:18:15.138 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:18:15.138 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-27T07:18:15.188 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-27T07:18:15.188 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:18:15.188 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-27T07:18:15.236 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-27T07:18:15.236 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:18:15.236 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-27T07:18:15.350 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-27T07:18:15.350 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:18:15.350 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-27T07:18:15.446 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-27T07:18:15.446 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4yzyjof97XttYidMe34GG5EeEt7krKWnfplm+1S/56RMy4HVyPIO+/PCVuooOu9hYV3JKXh7+XSJ/gTycYmiIpBWJipsVzzBi7mWK1koWNbFLzPPIg1mS4plj2XDQNoeef8MlCZ3phf82mh3TXBXZMWiP//psjtsmHBbSFFEXWqFEEH0BveMEbFiBiL9y7eMaoPNod6LJ6WfIcXwecItAm5JdpolkhD6n6RjXOmtjOu2UyCvNuVPSPrxKdGc5/304gjeUEUFUBFtnlP/At/Fbl2zY8Rs40u1poamzyoD16ytRqoV9J8hWYmPhk5XX8EbCvUQnZ8n1yaLNK6FxA4/vuBA+ZBGKsBzfuRVATvkgFzmfMas9FHFIn85HIQ8BZrDxdiVsDheHF4ix+ISN/mqmmkEMlV7lw2XZahHo3r1/aoGL5H3RShCxm3VhQb+Gb2ccx/PTgBhamIMxyDEMBR8G923Zx9qI82mJA8wDpTxJPLP/y5/GxqHx1wNLX0KPhRk= ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T07:18:15.556 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:15 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/348762912' entity='client.admin' 2024-05-27T07:18:15.568 INFO:teuthology.orchestra.run.smithi063.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4yzyjof97XttYidMe34GG5EeEt7krKWnfplm+1S/56RMy4HVyPIO+/PCVuooOu9hYV3JKXh7+XSJ/gTycYmiIpBWJipsVzzBi7mWK1koWNbFLzPPIg1mS4plj2XDQNoeef8MlCZ3phf82mh3TXBXZMWiP//psjtsmHBbSFFEXWqFEEH0BveMEbFiBiL9y7eMaoPNod6LJ6WfIcXwecItAm5JdpolkhD6n6RjXOmtjOu2UyCvNuVPSPrxKdGc5/304gjeUEUFUBFtnlP/At/Fbl2zY8Rs40u1poamzyoD16ytRqoV9J8hWYmPhk5XX8EbCvUQnZ8n1yaLNK6FxA4/vuBA+ZBGKsBzfuRVATvkgFzmfMas9FHFIn85HIQ8BZrDxdiVsDheHF4ix+ISN/mqmmkEMlV7lw2XZahHo3r1/aoGL5H3RShCxm3VhQb+Gb2ccx/PTgBhamIMxyDEMBR8G923Zx9qI82mJA8wDpTxJPLP/y5/GxqHx1wNLX0KPhRk= ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:18:15.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4yzyjof97XttYidMe34GG5EeEt7krKWnfplm+1S/56RMy4HVyPIO+/PCVuooOu9hYV3JKXh7+XSJ/gTycYmiIpBWJipsVzzBi7mWK1koWNbFLzPPIg1mS4plj2XDQNoeef8MlCZ3phf82mh3TXBXZMWiP//psjtsmHBbSFFEXWqFEEH0BveMEbFiBiL9y7eMaoPNod6LJ6WfIcXwecItAm5JdpolkhD6n6RjXOmtjOu2UyCvNuVPSPrxKdGc5/304gjeUEUFUBFtnlP/At/Fbl2zY8Rs40u1poamzyoD16ytRqoV9J8hWYmPhk5XX8EbCvUQnZ8n1yaLNK6FxA4/vuBA+ZBGKsBzfuRVATvkgFzmfMas9FHFIn85HIQ8BZrDxdiVsDheHF4ix+ISN/mqmmkEMlV7lw2XZahHo3r1/aoGL5H3RShCxm3VhQb+Gb2ccx/PTgBhamIMxyDEMBR8G923Zx9qI82mJA8wDpTxJPLP/y5/GxqHx1wNLX0KPhRk= ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T07:18:15.719 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4yzyjof97XttYidMe34GG5EeEt7krKWnfplm+1S/56RMy4HVyPIO+/PCVuooOu9hYV3JKXh7+XSJ/gTycYmiIpBWJipsVzzBi7mWK1koWNbFLzPPIg1mS4plj2XDQNoeef8MlCZ3phf82mh3TXBXZMWiP//psjtsmHBbSFFEXWqFEEH0BveMEbFiBiL9y7eMaoPNod6LJ6WfIcXwecItAm5JdpolkhD6n6RjXOmtjOu2UyCvNuVPSPrxKdGc5/304gjeUEUFUBFtnlP/At/Fbl2zY8Rs40u1poamzyoD16ytRqoV9J8hWYmPhk5XX8EbCvUQnZ8n1yaLNK6FxA4/vuBA+ZBGKsBzfuRVATvkgFzmfMas9FHFIn85HIQ8BZrDxdiVsDheHF4ix+ISN/mqmmkEMlV7lw2XZahHo3r1/aoGL5H3RShCxm3VhQb+Gb2ccx/PTgBhamIMxyDEMBR8G923Zx9qI82mJA8wDpTxJPLP/y5/GxqHx1wNLX0KPhRk= ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:18:15.757 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4yzyjof97XttYidMe34GG5EeEt7krKWnfplm+1S/56RMy4HVyPIO+/PCVuooOu9hYV3JKXh7+XSJ/gTycYmiIpBWJipsVzzBi7mWK1koWNbFLzPPIg1mS4plj2XDQNoeef8MlCZ3phf82mh3TXBXZMWiP//psjtsmHBbSFFEXWqFEEH0BveMEbFiBiL9y7eMaoPNod6LJ6WfIcXwecItAm5JdpolkhD6n6RjXOmtjOu2UyCvNuVPSPrxKdGc5/304gjeUEUFUBFtnlP/At/Fbl2zY8Rs40u1poamzyoD16ytRqoV9J8hWYmPhk5XX8EbCvUQnZ8n1yaLNK6FxA4/vuBA+ZBGKsBzfuRVATvkgFzmfMas9FHFIn85HIQ8BZrDxdiVsDheHF4ix+ISN/mqmmkEMlV7lw2XZahHo3r1/aoGL5H3RShCxm3VhQb+Gb2ccx/PTgBhamIMxyDEMBR8G923Zx9qI82mJA8wDpTxJPLP/y5/GxqHx1wNLX0KPhRk= ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T07:18:15.866 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4yzyjof97XttYidMe34GG5EeEt7krKWnfplm+1S/56RMy4HVyPIO+/PCVuooOu9hYV3JKXh7+XSJ/gTycYmiIpBWJipsVzzBi7mWK1koWNbFLzPPIg1mS4plj2XDQNoeef8MlCZ3phf82mh3TXBXZMWiP//psjtsmHBbSFFEXWqFEEH0BveMEbFiBiL9y7eMaoPNod6LJ6WfIcXwecItAm5JdpolkhD6n6RjXOmtjOu2UyCvNuVPSPrxKdGc5/304gjeUEUFUBFtnlP/At/Fbl2zY8Rs40u1poamzyoD16ytRqoV9J8hWYmPhk5XX8EbCvUQnZ8n1yaLNK6FxA4/vuBA+ZBGKsBzfuRVATvkgFzmfMas9FHFIn85HIQ8BZrDxdiVsDheHF4ix+ISN/mqmmkEMlV7lw2XZahHo3r1/aoGL5H3RShCxm3VhQb+Gb2ccx/PTgBhamIMxyDEMBR8G923Zx9qI82mJA8wDpTxJPLP/y5/GxqHx1wNLX0KPhRk= ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:18:15.920 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-27T07:18:16.361 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:18:19.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.078+0000 7f427fe00700 1 -- 172.21.15.63:0/1569088458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4278077710 msgr2=0x7f4278075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:19.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.078+0000 7f427fe00700 1 --2- 172.21.15.63:0/1569088458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4278077710 0x7f4278075b40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f426c009910 tx=0x7f426c009c20 comp rx=0 tx=0).stop 2024-05-27T07:18:19.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.078+0000 7f427fe00700 1 -- 172.21.15.63:0/1569088458 shutdown_connections 2024-05-27T07:18:19.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.078+0000 7f427fe00700 1 --2- 172.21.15.63:0/1569088458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4278077710 0x7f4278075b40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:19.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.078+0000 7f427fe00700 1 -- 172.21.15.63:0/1569088458 >> 172.21.15.63:0/1569088458 conn(0x7f42780ff210 msgr2=0x7f42781016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:19.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.078+0000 7f427fe00700 1 -- 172.21.15.63:0/1569088458 shutdown_connections 2024-05-27T07:18:19.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.078+0000 7f427fe00700 1 -- 172.21.15.63:0/1569088458 wait complete. 2024-05-27T07:18:19.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.079+0000 7f427fe00700 1 Processor -- start 2024-05-27T07:18:19.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.080+0000 7f427fe00700 1 -- start start 2024-05-27T07:18:19.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.080+0000 7f427fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4278077710 0x7f427819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:19.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.080+0000 7f427fe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f427810bc30 con 0x7f4278077710 2024-05-27T07:18:19.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.080+0000 7f427d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4278077710 0x7f427819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:19.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.080+0000 7f427d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4278077710 0x7f427819b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56676/0 (socket says 172.21.15.63:56676) 2024-05-27T07:18:19.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.080+0000 7f427d400700 1 -- 172.21.15.63:0/1495310347 learned_addr learned my addr 172.21.15.63:0/1495310347 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:19.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.081+0000 7f427d400700 1 -- 172.21.15.63:0/1495310347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427819bd40 con 0x7f4278077710 2024-05-27T07:18:19.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.081+0000 7f427d400700 1 --2- 172.21.15.63:0/1495310347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4278077710 0x7f427819b800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f426c005120 tx=0x7f426c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:19.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.082+0000 7f4276000700 1 -- 172.21.15.63:0/1495310347 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f426c027070 con 0x7f4278077710 2024-05-27T07:18:19.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.082+0000 7f427fe00700 1 -- 172.21.15.63:0/1495310347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427819bfd0 con 0x7f4278077710 2024-05-27T07:18:19.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.082+0000 7f4276000700 1 -- 172.21.15.63:0/1495310347 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f426c00bdf0 con 0x7f4278077710 2024-05-27T07:18:19.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.082+0000 7f4276000700 1 -- 172.21.15.63:0/1495310347 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f426c00fd70 con 0x7f4278077710 2024-05-27T07:18:19.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.082+0000 7f427fe00700 1 -- 172.21.15.63:0/1495310347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427819ebe0 con 0x7f4278077710 2024-05-27T07:18:19.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.083+0000 7f4276000700 1 -- 172.21.15.63:0/1495310347 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f426c02a070 con 0x7f4278077710 2024-05-27T07:18:19.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.083+0000 7f427fe00700 1 -- 172.21.15.63:0/1495310347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427810c3b0 con 0x7f4278077710 2024-05-27T07:18:19.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.084+0000 7f4276000700 1 --2- 172.21.15.63:0/1495310347 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f426003e510 0x7f4260040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:19.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.084+0000 7f4276000700 1 -- 172.21.15.63:0/1495310347 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f426c053be0 con 0x7f4278077710 2024-05-27T07:18:19.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.084+0000 7f427ca00700 1 --2- 172.21.15.63:0/1495310347 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f426003e510 0x7f4260040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:19.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.085+0000 7f427ca00700 1 --2- 172.21.15.63:0/1495310347 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f426003e510 0x7f4260040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4264006fd0 tx=0x7f4264006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:19.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.090+0000 7f4276000700 1 -- 172.21.15.63:0/1495310347 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f426c013070 con 0x7f4278077710 2024-05-27T07:18:19.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.329+0000 7f427fe00700 1 -- 172.21.15.63:0/1495310347 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f4278194a80 con 0x7f4278077710 2024-05-27T07:18:19.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.337+0000 7f4276000700 1 -- 172.21.15.63:0/1495310347 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f4278194a80 con 0x7f4278077710 2024-05-27T07:18:19.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.340+0000 7f427fe00700 1 -- 172.21.15.63:0/1495310347 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f426003e510 msgr2=0x7f4260040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:19.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.340+0000 7f427fe00700 1 --2- 172.21.15.63:0/1495310347 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f426003e510 0x7f4260040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4264006fd0 tx=0x7f4264006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:19.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.340+0000 7f427fe00700 1 -- 172.21.15.63:0/1495310347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4278077710 msgr2=0x7f427819b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:19.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.340+0000 7f427fe00700 1 --2- 172.21.15.63:0/1495310347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4278077710 0x7f427819b800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f426c005120 tx=0x7f426c005810 comp rx=0 tx=0).stop 2024-05-27T07:18:19.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.341+0000 7f427fe00700 1 -- 172.21.15.63:0/1495310347 shutdown_connections 2024-05-27T07:18:19.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.341+0000 7f427fe00700 1 --2- 172.21.15.63:0/1495310347 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f426003e510 0x7f4260040a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:19.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.341+0000 7f427fe00700 1 --2- 172.21.15.63:0/1495310347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4278077710 0x7f427819b800 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:19.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.341+0000 7f427fe00700 1 -- 172.21.15.63:0/1495310347 >> 172.21.15.63:0/1495310347 conn(0x7f42780ff210 msgr2=0x7f42780fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:19.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.341+0000 7f427fe00700 1 -- 172.21.15.63:0/1495310347 shutdown_connections 2024-05-27T07:18:19.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:19.341+0000 7f427fe00700 1 -- 172.21.15.63:0/1495310347 wait complete. 2024-05-27T07:18:19.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:19.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:19.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:18:19.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:19.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:18:19.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:19.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:19 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1495310347' entity='client.admin' 2024-05-27T07:18:20.123 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-27T07:18:20.124 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-27T07:18:20.545 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:18:20.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:18:20.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:23.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.272+0000 7f3a5e800700 1 -- 172.21.15.63:0/3432618674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a58105dd0 msgr2=0x7f3a581061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:23.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.272+0000 7f3a5e800700 1 --2- 172.21.15.63:0/3432618674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a58105dd0 0x7f3a581061e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009910 tx=0x7f3a48009c20 comp rx=0 tx=0).stop 2024-05-27T07:18:23.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.272+0000 7f3a5e800700 1 -- 172.21.15.63:0/3432618674 shutdown_connections 2024-05-27T07:18:23.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.272+0000 7f3a5e800700 1 --2- 172.21.15.63:0/3432618674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a58105dd0 0x7f3a581061e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:23.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.272+0000 7f3a5e800700 1 -- 172.21.15.63:0/3432618674 >> 172.21.15.63:0/3432618674 conn(0x7f3a58101410 msgr2=0x7f3a581038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:23.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.272+0000 7f3a5e800700 1 -- 172.21.15.63:0/3432618674 shutdown_connections 2024-05-27T07:18:23.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.272+0000 7f3a5e800700 1 -- 172.21.15.63:0/3432618674 wait complete. 2024-05-27T07:18:23.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.274+0000 7f3a5e800700 1 Processor -- start 2024-05-27T07:18:23.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.274+0000 7f3a5e800700 1 -- start start 2024-05-27T07:18:23.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.274+0000 7f3a5e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a58105dd0 0x7f3a5819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:23.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.274+0000 7f3a5e800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5810bc40 con 0x7f3a58105dd0 2024-05-27T07:18:23.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.274+0000 7f3a57400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a58105dd0 0x7f3a5819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:23.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.274+0000 7f3a57400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a58105dd0 0x7f3a5819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56692/0 (socket says 172.21.15.63:56692) 2024-05-27T07:18:23.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.274+0000 7f3a57400700 1 -- 172.21.15.63:0/3402482776 learned_addr learned my addr 172.21.15.63:0/3402482776 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:23.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.275+0000 7f3a57400700 1 -- 172.21.15.63:0/3402482776 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5819bd90 con 0x7f3a58105dd0 2024-05-27T07:18:23.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.275+0000 7f3a57400700 1 --2- 172.21.15.63:0/3402482776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a58105dd0 0x7f3a5819b850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a48005950 tx=0x7f3a48005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:23.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.276+0000 7f3a4fe00700 1 -- 172.21.15.63:0/3402482776 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a48027070 con 0x7f3a58105dd0 2024-05-27T07:18:23.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.276+0000 7f3a4fe00700 1 -- 172.21.15.63:0/3402482776 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a4800bdf0 con 0x7f3a58105dd0 2024-05-27T07:18:23.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.276+0000 7f3a4fe00700 1 -- 172.21.15.63:0/3402482776 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a4800fdb0 con 0x7f3a58105dd0 2024-05-27T07:18:23.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.276+0000 7f3a5e800700 1 -- 172.21.15.63:0/3402482776 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5819c020 con 0x7f3a58105dd0 2024-05-27T07:18:23.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.276+0000 7f3a5e800700 1 -- 172.21.15.63:0/3402482776 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5819ec30 con 0x7f3a58105dd0 2024-05-27T07:18:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.277+0000 7f3a5e800700 1 -- 172.21.15.63:0/3402482776 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a58194a90 con 0x7f3a58105dd0 2024-05-27T07:18:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.277+0000 7f3a4fe00700 1 -- 172.21.15.63:0/3402482776 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3a4802a070 con 0x7f3a58105dd0 2024-05-27T07:18:23.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.278+0000 7f3a4fe00700 1 --2- 172.21.15.63:0/3402482776 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a3c03e570 0x7f3a3c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:23.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.278+0000 7f3a4fe00700 1 -- 172.21.15.63:0/3402482776 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a48053950 con 0x7f3a58105dd0 2024-05-27T07:18:23.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.278+0000 7f3a56a00700 1 --2- 172.21.15.63:0/3402482776 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a3c03e570 0x7f3a3c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:23.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.279+0000 7f3a56a00700 1 --2- 172.21.15.63:0/3402482776 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a3c03e570 0x7f3a3c040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a44006fd0 tx=0x7f3a44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:23.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.284+0000 7f3a4fe00700 1 -- 172.21.15.63:0/3402482776 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a48013070 con 0x7f3a58105dd0 2024-05-27T07:18:23.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.527+0000 7f3a5e800700 1 -- 172.21.15.63:0/3402482776 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f3a5802e6f0 con 0x7f3a3c03e570 2024-05-27T07:18:23.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.533+0000 7f3a4fe00700 1 -- 172.21.15.63:0/3402482776 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3a5802e6f0 con 0x7f3a3c03e570 2024-05-27T07:18:23.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.536+0000 7f3a5e800700 1 -- 172.21.15.63:0/3402482776 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a3c03e570 msgr2=0x7f3a3c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:23.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.536+0000 7f3a5e800700 1 --2- 172.21.15.63:0/3402482776 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a3c03e570 0x7f3a3c040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a44006fd0 tx=0x7f3a44006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:23.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.536+0000 7f3a5e800700 1 -- 172.21.15.63:0/3402482776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a58105dd0 msgr2=0x7f3a5819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:23.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.536+0000 7f3a5e800700 1 --2- 172.21.15.63:0/3402482776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a58105dd0 0x7f3a5819b850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a48005950 tx=0x7f3a48005810 comp rx=0 tx=0).stop 2024-05-27T07:18:23.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.537+0000 7f3a5e800700 1 -- 172.21.15.63:0/3402482776 shutdown_connections 2024-05-27T07:18:23.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.537+0000 7f3a5e800700 1 --2- 172.21.15.63:0/3402482776 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a3c03e570 0x7f3a3c040a80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:23.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.537+0000 7f3a5e800700 1 --2- 172.21.15.63:0/3402482776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a58105dd0 0x7f3a5819b850 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:23.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.537+0000 7f3a5e800700 1 -- 172.21.15.63:0/3402482776 >> 172.21.15.63:0/3402482776 conn(0x7f3a58101410 msgr2=0x7f3a58102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:23.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.537+0000 7f3a5e800700 1 -- 172.21.15.63:0/3402482776 shutdown_connections 2024-05-27T07:18:23.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:23.537+0000 7f3a5e800700 1 -- 172.21.15.63:0/3402482776 wait complete. 2024-05-27T07:18:24.242 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi086 2024-05-27T07:18:24.243 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:18:24.243 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.conf 2024-05-27T07:18:24.301 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:18:24.301 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:18:24.397 INFO:tasks.cephadm:Adding host smithi086 to orchestrator... 2024-05-27T07:18:24.398 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch host add smithi086 2024-05-27T07:18:24.820 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:18:24.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:24 smithi063 ceph-mon[31613]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:18:24.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:24.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:24 smithi063 ceph-mon[31613]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:24.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:18:24.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:18:24.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:18:24.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:24 smithi063 ceph-mon[31613]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-27T07:18:25.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:25 smithi063 ceph-mon[31613]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:18:26.640 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:26 smithi063 ceph-mon[31613]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:18:26.640 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:26 smithi063 ceph-mon[31613]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:26.640 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:26 smithi063 ceph-mon[31613]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:18:26.640 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:26 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:26.640 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:26 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:26.640 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:26 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:26.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.900+0000 7fd2e0200700 1 -- 172.21.15.63:0/2555704625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8105ff0 msgr2=0x7fd2d8106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:26.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.900+0000 7fd2e0200700 1 --2- 172.21.15.63:0/2555704625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8105ff0 0x7fd2d8106400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009910 tx=0x7fd2cc009c20 comp rx=0 tx=0).stop 2024-05-27T07:18:26.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.900+0000 7fd2e0200700 1 -- 172.21.15.63:0/2555704625 shutdown_connections 2024-05-27T07:18:26.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.900+0000 7fd2e0200700 1 --2- 172.21.15.63:0/2555704625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8105ff0 0x7fd2d8106400 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:26.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.900+0000 7fd2e0200700 1 -- 172.21.15.63:0/2555704625 >> 172.21.15.63:0/2555704625 conn(0x7fd2d81018a0 msgr2=0x7fd2d8103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:26.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.900+0000 7fd2e0200700 1 -- 172.21.15.63:0/2555704625 shutdown_connections 2024-05-27T07:18:26.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.900+0000 7fd2e0200700 1 -- 172.21.15.63:0/2555704625 wait complete. 2024-05-27T07:18:26.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.901+0000 7fd2e0200700 1 Processor -- start 2024-05-27T07:18:26.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.902+0000 7fd2e0200700 1 -- start start 2024-05-27T07:18:26.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.902+0000 7fd2e0200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8105ff0 0x7fd2d81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:26.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.902+0000 7fd2e0200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d8110190 con 0x7fd2d8105ff0 2024-05-27T07:18:26.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.902+0000 7fd2dd800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8105ff0 0x7fd2d81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:26.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.903+0000 7fd2dd800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8105ff0 0x7fd2d81a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56722/0 (socket says 172.21.15.63:56722) 2024-05-27T07:18:26.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.903+0000 7fd2dd800700 1 -- 172.21.15.63:0/589230305 learned_addr learned my addr 172.21.15.63:0/589230305 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:26.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.903+0000 7fd2dd800700 1 -- 172.21.15.63:0/589230305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d81a1ba0 con 0x7fd2d8105ff0 2024-05-27T07:18:26.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.903+0000 7fd2dd800700 1 --2- 172.21.15.63:0/589230305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8105ff0 0x7fd2d81a1660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc006010 tx=0x7fd2cc005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:26.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.904+0000 7fd2d6000700 1 -- 172.21.15.63:0/589230305 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2cc026070 con 0x7fd2d8105ff0 2024-05-27T07:18:26.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.904+0000 7fd2d6000700 1 -- 172.21.15.63:0/589230305 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2cc00b6f0 con 0x7fd2d8105ff0 2024-05-27T07:18:26.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.904+0000 7fd2e0200700 1 -- 172.21.15.63:0/589230305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d81a1e90 con 0x7fd2d8105ff0 2024-05-27T07:18:26.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.904+0000 7fd2d6000700 1 -- 172.21.15.63:0/589230305 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2cc003810 con 0x7fd2d8105ff0 2024-05-27T07:18:26.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.904+0000 7fd2e0200700 1 -- 172.21.15.63:0/589230305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d819b880 con 0x7fd2d8105ff0 2024-05-27T07:18:26.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.905+0000 7fd2e0200700 1 -- 172.21.15.63:0/589230305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d81074a0 con 0x7fd2d8105ff0 2024-05-27T07:18:26.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.905+0000 7fd2d6000700 1 -- 172.21.15.63:0/589230305 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd2cc029070 con 0x7fd2d8105ff0 2024-05-27T07:18:26.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.906+0000 7fd2d6000700 1 --2- 172.21.15.63:0/589230305 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd2c003e510 0x7fd2c0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:26.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.906+0000 7fd2d6000700 1 -- 172.21.15.63:0/589230305 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd2cc01e070 con 0x7fd2d8105ff0 2024-05-27T07:18:26.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.906+0000 7fd2dce00700 1 --2- 172.21.15.63:0/589230305 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd2c003e510 0x7fd2c0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:26.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.907+0000 7fd2dce00700 1 --2- 172.21.15.63:0/589230305 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd2c003e510 0x7fd2c0040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4006fd0 tx=0x7fd2c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:26.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:26.912+0000 7fd2d6000700 1 -- 172.21.15.63:0/589230305 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2cc013070 con 0x7fd2d8105ff0 2024-05-27T07:18:27.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:27.152+0000 7fd2e0200700 1 -- 172.21.15.63:0/589230305 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}) -- 0x7fd2d819ce20 con 0x7fd2c003e510 2024-05-27T07:18:28.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:28 smithi063 ceph-mon[31613]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:18:28.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:28 smithi063 ceph-mon[31613]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:29.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:29 smithi063 ceph-mon[31613]: Deploying cephadm binary to smithi086 2024-05-27T07:18:30.892 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:30 smithi063 ceph-mon[31613]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:32.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:32 smithi063 ceph-mon[31613]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:33.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.479+0000 7fd2d6000700 1 -- 172.21.15.63:0/589230305 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fd2d819ce20 con 0x7fd2c003e510 2024-05-27T07:18:33.470 INFO:teuthology.orchestra.run.smithi063.stdout:Added host 'smithi086' with addr '172.21.15.86' 2024-05-27T07:18:33.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.483+0000 7fd2e0200700 1 -- 172.21.15.63:0/589230305 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd2c003e510 msgr2=0x7fd2c0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:33.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.483+0000 7fd2e0200700 1 --2- 172.21.15.63:0/589230305 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd2c003e510 0x7fd2c0040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4006fd0 tx=0x7fd2c4006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:33.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.483+0000 7fd2e0200700 1 -- 172.21.15.63:0/589230305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8105ff0 msgr2=0x7fd2d81a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:33.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.483+0000 7fd2e0200700 1 --2- 172.21.15.63:0/589230305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8105ff0 0x7fd2d81a1660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc006010 tx=0x7fd2cc005710 comp rx=0 tx=0).stop 2024-05-27T07:18:33.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.484+0000 7fd2e0200700 1 -- 172.21.15.63:0/589230305 shutdown_connections 2024-05-27T07:18:33.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.484+0000 7fd2e0200700 1 --2- 172.21.15.63:0/589230305 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd2c003e510 0x7fd2c0040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:33.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.484+0000 7fd2e0200700 1 --2- 172.21.15.63:0/589230305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8105ff0 0x7fd2d81a1660 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:33.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.484+0000 7fd2e0200700 1 -- 172.21.15.63:0/589230305 >> 172.21.15.63:0/589230305 conn(0x7fd2d81018a0 msgr2=0x7fd2d8103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:33.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.484+0000 7fd2e0200700 1 -- 172.21.15.63:0/589230305 shutdown_connections 2024-05-27T07:18:33.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:33.484+0000 7fd2e0200700 1 -- 172.21.15.63:0/589230305 wait complete. 2024-05-27T07:18:34.230 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T07:18:34.682 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:18:34.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:34 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:34.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:34 smithi063 ceph-mon[31613]: Added host smithi086 2024-05-27T07:18:34.836 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:34 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:18:34.836 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:34 smithi063 ceph-mon[31613]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:34.836 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:34 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:36.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:36.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:36 smithi063 ceph-mon[31613]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:36.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.736+0000 7f6904e00700 1 -- 172.21.15.63:0/2994708187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900105e00 msgr2=0x7f6900106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:36.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.736+0000 7f6904e00700 1 --2- 172.21.15.63:0/2994708187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900105e00 0x7f6900106210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009910 tx=0x7f68f0009c20 comp rx=0 tx=0).stop 2024-05-27T07:18:36.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.736+0000 7f6904e00700 1 -- 172.21.15.63:0/2994708187 shutdown_connections 2024-05-27T07:18:36.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.737+0000 7f6904e00700 1 --2- 172.21.15.63:0/2994708187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900105e00 0x7f6900106210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:36.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.737+0000 7f6904e00700 1 -- 172.21.15.63:0/2994708187 >> 172.21.15.63:0/2994708187 conn(0x7f69001016d0 msgr2=0x7f6900103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:36.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.737+0000 7f6904e00700 1 -- 172.21.15.63:0/2994708187 shutdown_connections 2024-05-27T07:18:36.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.737+0000 7f6904e00700 1 -- 172.21.15.63:0/2994708187 wait complete. 2024-05-27T07:18:36.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.738+0000 7f6904e00700 1 Processor -- start 2024-05-27T07:18:36.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.738+0000 7f6904e00700 1 -- start start 2024-05-27T07:18:36.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.739+0000 7f6904e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900105e00 0x7f69001a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:36.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.739+0000 7f6904e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f690010ffa0 con 0x7f6900105e00 2024-05-27T07:18:36.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.739+0000 7f68ff400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900105e00 0x7f69001a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:36.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.739+0000 7f68ff400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900105e00 0x7f69001a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33258/0 (socket says 172.21.15.63:33258) 2024-05-27T07:18:36.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.739+0000 7f68ff400700 1 -- 172.21.15.63:0/2066277840 learned_addr learned my addr 172.21.15.63:0/2066277840 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:36.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.739+0000 7f68ff400700 1 -- 172.21.15.63:0/2066277840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69001a1a60 con 0x7f6900105e00 2024-05-27T07:18:36.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.740+0000 7f68ff400700 1 --2- 172.21.15.63:0/2066277840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900105e00 0x7f69001a1520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f68f0005660 tx=0x7f68f0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:36.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.740+0000 7f68f7e00700 1 -- 172.21.15.63:0/2066277840 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68f0026070 con 0x7f6900105e00 2024-05-27T07:18:36.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.740+0000 7f68f7e00700 1 -- 172.21.15.63:0/2066277840 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68f000b6f0 con 0x7f6900105e00 2024-05-27T07:18:36.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.740+0000 7f6904e00700 1 -- 172.21.15.63:0/2066277840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69001a1d50 con 0x7f6900105e00 2024-05-27T07:18:36.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.740+0000 7f68f7e00700 1 -- 172.21.15.63:0/2066277840 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68f00037b0 con 0x7f6900105e00 2024-05-27T07:18:36.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.740+0000 7f6904e00700 1 -- 172.21.15.63:0/2066277840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690019b880 con 0x7f6900105e00 2024-05-27T07:18:36.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.741+0000 7f6904e00700 1 -- 172.21.15.63:0/2066277840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69001072b0 con 0x7f6900105e00 2024-05-27T07:18:36.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.742+0000 7f68f7e00700 1 -- 172.21.15.63:0/2066277840 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f68f0029070 con 0x7f6900105e00 2024-05-27T07:18:36.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.742+0000 7f68f7e00700 1 --2- 172.21.15.63:0/2066277840 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f68e403e570 0x7f68e4040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:36.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.742+0000 7f68f7e00700 1 -- 172.21.15.63:0/2066277840 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f68f001e070 con 0x7f6900105e00 2024-05-27T07:18:36.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.742+0000 7f68fea00700 1 --2- 172.21.15.63:0/2066277840 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f68e403e570 0x7f68e4040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:36.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.743+0000 7f68fea00700 1 --2- 172.21.15.63:0/2066277840 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f68e403e570 0x7f68e4040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006fd0 tx=0x7f68ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:36.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:36.748+0000 7f68f7e00700 1 -- 172.21.15.63:0/2066277840 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f68f0013070 con 0x7f6900105e00 2024-05-27T07:18:36.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.000+0000 7f6904e00700 1 -- 172.21.15.63:0/2066277840 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f690019cbc0 con 0x7f68e403e570 2024-05-27T07:18:36.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.002+0000 7f68f7e00700 1 -- 172.21.15.63:0/2066277840 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f690019cbc0 con 0x7f68e403e570 2024-05-27T07:18:36.993 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:36.993 INFO:teuthology.orchestra.run.smithi063.stdout:[{"addr": "172.21.15.63", "hostname": "smithi063", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}] 2024-05-27T07:18:36.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.005+0000 7f6904e00700 1 -- 172.21.15.63:0/2066277840 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f68e403e570 msgr2=0x7f68e4040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:36.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.006+0000 7f6904e00700 1 --2- 172.21.15.63:0/2066277840 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f68e403e570 0x7f68e4040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006fd0 tx=0x7f68ec006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:36.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.006+0000 7f6904e00700 1 -- 172.21.15.63:0/2066277840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900105e00 msgr2=0x7f69001a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:36.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.006+0000 7f6904e00700 1 --2- 172.21.15.63:0/2066277840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900105e00 0x7f69001a1520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f68f0005660 tx=0x7f68f0005740 comp rx=0 tx=0).stop 2024-05-27T07:18:36.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.006+0000 7f6904e00700 1 -- 172.21.15.63:0/2066277840 shutdown_connections 2024-05-27T07:18:36.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.006+0000 7f6904e00700 1 --2- 172.21.15.63:0/2066277840 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f68e403e570 0x7f68e4040a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:36.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.006+0000 7f6904e00700 1 --2- 172.21.15.63:0/2066277840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6900105e00 0x7f69001a1520 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:36.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.006+0000 7f6904e00700 1 -- 172.21.15.63:0/2066277840 >> 172.21.15.63:0/2066277840 conn(0x7f69001016d0 msgr2=0x7f6900103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:36.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.006+0000 7f6904e00700 1 -- 172.21.15.63:0/2066277840 shutdown_connections 2024-05-27T07:18:36.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:37.006+0000 7f6904e00700 1 -- 172.21.15.63:0/2066277840 wait complete. 2024-05-27T07:18:37.747 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-05-27T07:18:37.748 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:18:37.748 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-05-27T07:18:37.806 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:18:37.806 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:18:37.903 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-05-27T07:18:37.903 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch host add smithi149 2024-05-27T07:18:38.373 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:18:38.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:38 smithi063 ceph-mon[31613]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:18:38.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:38 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:38.702 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:38 smithi063 ceph-mon[31613]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.446+0000 7f0a97800700 1 -- 172.21.15.63:0/1813469335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a90105fb0 msgr2=0x7f0a901063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.446+0000 7f0a97800700 1 --2- 172.21.15.63:0/1813469335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a90105fb0 0x7f0a901063c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009910 tx=0x7f0a84009c20 comp rx=0 tx=0).stop 2024-05-27T07:18:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.446+0000 7f0a97800700 1 -- 172.21.15.63:0/1813469335 shutdown_connections 2024-05-27T07:18:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.446+0000 7f0a97800700 1 --2- 172.21.15.63:0/1813469335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a90105fb0 0x7f0a901063c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.446+0000 7f0a97800700 1 -- 172.21.15.63:0/1813469335 >> 172.21.15.63:0/1813469335 conn(0x7f0a90101840 msgr2=0x7f0a90103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.446+0000 7f0a97800700 1 -- 172.21.15.63:0/1813469335 shutdown_connections 2024-05-27T07:18:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.446+0000 7f0a97800700 1 -- 172.21.15.63:0/1813469335 wait complete. 2024-05-27T07:18:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.447+0000 7f0a97800700 1 Processor -- start 2024-05-27T07:18:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.447+0000 7f0a97800700 1 -- start start 2024-05-27T07:18:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.448+0000 7f0a97800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a90105fb0 0x7f0a901a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.448+0000 7f0a97800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90110150 con 0x7f0a90105fb0 2024-05-27T07:18:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.448+0000 7f0a94e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a90105fb0 0x7f0a901a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.448+0000 7f0a94e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a90105fb0 0x7f0a901a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49610/0 (socket says 172.21.15.63:49610) 2024-05-27T07:18:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.448+0000 7f0a94e00700 1 -- 172.21.15.63:0/1388544238 learned_addr learned my addr 172.21.15.63:0/1388544238 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.449+0000 7f0a94e00700 1 -- 172.21.15.63:0/1388544238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a901a1bc0 con 0x7f0a90105fb0 2024-05-27T07:18:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.449+0000 7f0a94e00700 1 --2- 172.21.15.63:0/1388544238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a90105fb0 0x7f0a901a1680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0a84006010 tx=0x7f0a84005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.449+0000 7f0a8d600700 1 -- 172.21.15.63:0/1388544238 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a84026070 con 0x7f0a90105fb0 2024-05-27T07:18:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.449+0000 7f0a8d600700 1 -- 172.21.15.63:0/1388544238 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a8400b6f0 con 0x7f0a90105fb0 2024-05-27T07:18:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.449+0000 7f0a97800700 1 -- 172.21.15.63:0/1388544238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a901a1eb0 con 0x7f0a90105fb0 2024-05-27T07:18:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.449+0000 7f0a8d600700 1 -- 172.21.15.63:0/1388544238 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a84003810 con 0x7f0a90105fb0 2024-05-27T07:18:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.450+0000 7f0a97800700 1 -- 172.21.15.63:0/1388544238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9019b8a0 con 0x7f0a90105fb0 2024-05-27T07:18:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.451+0000 7f0a97800700 1 -- 172.21.15.63:0/1388544238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a90107460 con 0x7f0a90105fb0 2024-05-27T07:18:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.451+0000 7f0a8d600700 1 -- 172.21.15.63:0/1388544238 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0a84029070 con 0x7f0a90105fb0 2024-05-27T07:18:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.452+0000 7f0a8d600700 1 --2- 172.21.15.63:0/1388544238 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a7803e510 0x7f0a78040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.452+0000 7f0a8d600700 1 -- 172.21.15.63:0/1388544238 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a8401e070 con 0x7f0a90105fb0 2024-05-27T07:18:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.452+0000 7f0a8fe00700 1 --2- 172.21.15.63:0/1388544238 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a7803e510 0x7f0a78040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.453+0000 7f0a8fe00700 1 --2- 172.21.15.63:0/1388544238 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a7803e510 0x7f0a78040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c006fd0 tx=0x7f0a7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.458+0000 7f0a8d600700 1 -- 172.21.15.63:0/1388544238 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a84013070 con 0x7f0a90105fb0 2024-05-27T07:18:40.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:40.701+0000 7f0a97800700 1 -- 172.21.15.63:0/1388544238 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) -- 0x7f0a9019cdd0 con 0x7f0a7803e510 2024-05-27T07:18:40.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:40 smithi063 ceph-mon[31613]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:41.929 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:41 smithi063 ceph-mon[31613]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:18:42.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:42 smithi063 ceph-mon[31613]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:42.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:42 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:42.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:42 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:42.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:42 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:42.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:42 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:42.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:42 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:18:42.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:42 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:18:42.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:42 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:18:43.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:43 smithi063 ceph-mon[31613]: Deploying cephadm binary to smithi149 2024-05-27T07:18:43.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:43 smithi063 ceph-mon[31613]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-27T07:18:44.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:44 smithi063 ceph-mon[31613]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:18:44.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:44 smithi063 ceph-mon[31613]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:46.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:46 smithi063 ceph-mon[31613]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:47.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.095+0000 7f0a8d600700 1 -- 172.21.15.63:0/1388544238 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0a9019cdd0 con 0x7f0a7803e510 2024-05-27T07:18:47.086 INFO:teuthology.orchestra.run.smithi063.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-05-27T07:18:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.099+0000 7f0a97800700 1 -- 172.21.15.63:0/1388544238 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a7803e510 msgr2=0x7f0a78040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.099+0000 7f0a97800700 1 --2- 172.21.15.63:0/1388544238 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a7803e510 0x7f0a78040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c006fd0 tx=0x7f0a7c006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.099+0000 7f0a97800700 1 -- 172.21.15.63:0/1388544238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a90105fb0 msgr2=0x7f0a901a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.099+0000 7f0a97800700 1 --2- 172.21.15.63:0/1388544238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a90105fb0 0x7f0a901a1680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0a84006010 tx=0x7f0a84005710 comp rx=0 tx=0).stop 2024-05-27T07:18:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.099+0000 7f0a97800700 1 -- 172.21.15.63:0/1388544238 shutdown_connections 2024-05-27T07:18:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.099+0000 7f0a97800700 1 --2- 172.21.15.63:0/1388544238 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a7803e510 0x7f0a78040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.099+0000 7f0a97800700 1 --2- 172.21.15.63:0/1388544238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a90105fb0 0x7f0a901a1680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.099+0000 7f0a97800700 1 -- 172.21.15.63:0/1388544238 >> 172.21.15.63:0/1388544238 conn(0x7f0a90101840 msgr2=0x7f0a9010bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.099+0000 7f0a97800700 1 -- 172.21.15.63:0/1388544238 shutdown_connections 2024-05-27T07:18:47.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:47.100+0000 7f0a97800700 1 -- 172.21.15.63:0/1388544238 wait complete. 2024-05-27T07:18:47.894 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:47 smithi063 ceph-mon[31613]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:18:47.894 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:47 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:47.992 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T07:18:48.444 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:18:48.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:48 smithi063 ceph-mon[31613]: Added host smithi149 2024-05-27T07:18:48.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:48 smithi063 ceph-mon[31613]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:18:48.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:48 smithi063 ceph-mon[31613]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:48.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:48.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:48.877 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:48.877 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:18:50.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.399+0000 7f6f1a400700 1 -- 172.21.15.63:0/3507646736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f14075d20 msgr2=0x7f6f14076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:50.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.399+0000 7f6f1a400700 1 --2- 172.21.15.63:0/3507646736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f14075d20 0x7f6f14076130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009910 tx=0x7f6f04009c20 comp rx=0 tx=0).stop 2024-05-27T07:18:50.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:50 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:50.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:50 smithi063 ceph-mon[31613]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:50.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:50 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:50.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.399+0000 7f6f1a400700 1 -- 172.21.15.63:0/3507646736 shutdown_connections 2024-05-27T07:18:50.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.399+0000 7f6f1a400700 1 --2- 172.21.15.63:0/3507646736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f14075d20 0x7f6f14076130 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:50.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.399+0000 7f6f1a400700 1 -- 172.21.15.63:0/3507646736 >> 172.21.15.63:0/3507646736 conn(0x7f6f140ff690 msgr2=0x7f6f14101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:50.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.399+0000 7f6f1a400700 1 -- 172.21.15.63:0/3507646736 shutdown_connections 2024-05-27T07:18:50.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.399+0000 7f6f1a400700 1 -- 172.21.15.63:0/3507646736 wait complete. 2024-05-27T07:18:50.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.400+0000 7f6f1a400700 1 Processor -- start 2024-05-27T07:18:50.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.400+0000 7f6f1a400700 1 -- start start 2024-05-27T07:18:50.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.401+0000 7f6f1a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f14075d20 0x7f6f14103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:50.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.401+0000 7f6f1a400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141101e0 con 0x7f6f14075d20 2024-05-27T07:18:50.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.401+0000 7f6f13400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f14075d20 0x7f6f14103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:50.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.401+0000 7f6f13400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f14075d20 0x7f6f14103ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42410/0 (socket says 172.21.15.63:42410) 2024-05-27T07:18:50.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.401+0000 7f6f13400700 1 -- 172.21.15.63:0/121964358 learned_addr learned my addr 172.21.15.63:0/121964358 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:50.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.401+0000 7f6f13400700 1 -- 172.21.15.63:0/121964358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f14107920 con 0x7f6f14075d20 2024-05-27T07:18:50.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.402+0000 7f6f13400700 1 --2- 172.21.15.63:0/121964358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f14075d20 0x7f6f14103ab0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f04005060 tx=0x7f6f04005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:50.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.402+0000 7f6f0be00700 1 -- 172.21.15.63:0/121964358 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f04026070 con 0x7f6f14075d20 2024-05-27T07:18:50.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.402+0000 7f6f0be00700 1 -- 172.21.15.63:0/121964358 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f0400b750 con 0x7f6f14075d20 2024-05-27T07:18:50.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.402+0000 7f6f1a400700 1 -- 172.21.15.63:0/121964358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f14107bb0 con 0x7f6f14075d20 2024-05-27T07:18:50.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.402+0000 7f6f0be00700 1 -- 172.21.15.63:0/121964358 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f04003850 con 0x7f6f14075d20 2024-05-27T07:18:50.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.402+0000 7f6f1a400700 1 -- 172.21.15.63:0/121964358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f141041c0 con 0x7f6f14075d20 2024-05-27T07:18:50.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.403+0000 7f6f1a400700 1 -- 172.21.15.63:0/121964358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f14077290 con 0x7f6f14075d20 2024-05-27T07:18:50.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.404+0000 7f6f0be00700 1 -- 172.21.15.63:0/121964358 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6f04029070 con 0x7f6f14075d20 2024-05-27T07:18:50.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.404+0000 7f6f0be00700 1 --2- 172.21.15.63:0/121964358 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6ef803e570 0x7f6ef8040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:50.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.404+0000 7f6f0be00700 1 -- 172.21.15.63:0/121964358 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f0401e070 con 0x7f6f14075d20 2024-05-27T07:18:50.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.405+0000 7f6f12a00700 1 --2- 172.21.15.63:0/121964358 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6ef803e570 0x7f6ef8040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:50.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.405+0000 7f6f12a00700 1 --2- 172.21.15.63:0/121964358 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6ef803e570 0x7f6ef8040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6f00006fd0 tx=0x7f6f00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:50.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.410+0000 7f6f0be00700 1 -- 172.21.15.63:0/121964358 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f04013070 con 0x7f6f14075d20 2024-05-27T07:18:50.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.652+0000 7f6f1a400700 1 -- 172.21.15.63:0/121964358 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f14104f90 con 0x7f6ef803e570 2024-05-27T07:18:50.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.654+0000 7f6f0be00700 1 -- 172.21.15.63:0/121964358 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f6f14104f90 con 0x7f6ef803e570 2024-05-27T07:18:50.645 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:18:50.645 INFO:teuthology.orchestra.run.smithi063.stdout:[{"addr": "172.21.15.63", "hostname": "smithi063", "labels": [], "status": ""}, {"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-05-27T07:18:50.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.657+0000 7f6f1a400700 1 -- 172.21.15.63:0/121964358 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6ef803e570 msgr2=0x7f6ef8040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:50.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.657+0000 7f6f1a400700 1 --2- 172.21.15.63:0/121964358 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6ef803e570 0x7f6ef8040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6f00006fd0 tx=0x7f6f00006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.657+0000 7f6f1a400700 1 -- 172.21.15.63:0/121964358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f14075d20 msgr2=0x7f6f14103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.657+0000 7f6f1a400700 1 --2- 172.21.15.63:0/121964358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f14075d20 0x7f6f14103ab0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f04005060 tx=0x7f6f04005710 comp rx=0 tx=0).stop 2024-05-27T07:18:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.657+0000 7f6f1a400700 1 -- 172.21.15.63:0/121964358 shutdown_connections 2024-05-27T07:18:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.657+0000 7f6f1a400700 1 --2- 172.21.15.63:0/121964358 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6ef803e570 0x7f6ef8040a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.657+0000 7f6f1a400700 1 --2- 172.21.15.63:0/121964358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f14075d20 0x7f6f14103ab0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.657+0000 7f6f1a400700 1 -- 172.21.15.63:0/121964358 >> 172.21.15.63:0/121964358 conn(0x7f6f140ff690 msgr2=0x7f6f1410c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.657+0000 7f6f1a400700 1 -- 172.21.15.63:0/121964358 shutdown_connections 2024-05-27T07:18:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:50.658+0000 7f6f1a400700 1 -- 172.21.15.63:0/121964358 wait complete. 2024-05-27T07:18:51.341 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-27T07:18:51.341 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-05-27T07:18:51.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:51 smithi063 ceph-mon[31613]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:18:51.794 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:18:52.635 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:52 smithi063 ceph-mon[31613]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:52.635 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:52 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:53.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.777+0000 7f25c2400700 1 -- 172.21.15.63:0/3341836352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25bc109c20 msgr2=0x7f25bc10a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.777+0000 7f25c2400700 1 --2- 172.21.15.63:0/3341836352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25bc109c20 0x7f25bc10a030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009910 tx=0x7f25ac009c20 comp rx=0 tx=0).stop 2024-05-27T07:18:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.777+0000 7f25c2400700 1 -- 172.21.15.63:0/3341836352 shutdown_connections 2024-05-27T07:18:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.777+0000 7f25c2400700 1 --2- 172.21.15.63:0/3341836352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25bc109c20 0x7f25bc10a030 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.777+0000 7f25c2400700 1 -- 172.21.15.63:0/3341836352 >> 172.21.15.63:0/3341836352 conn(0x7f25bc0ff660 msgr2=0x7f25bc101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.777+0000 7f25c2400700 1 -- 172.21.15.63:0/3341836352 shutdown_connections 2024-05-27T07:18:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.777+0000 7f25c2400700 1 -- 172.21.15.63:0/3341836352 wait complete. 2024-05-27T07:18:53.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.778+0000 7f25c2400700 1 Processor -- start 2024-05-27T07:18:53.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.779+0000 7f25c2400700 1 -- start start 2024-05-27T07:18:53.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.779+0000 7f25c2400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25bc109c20 0x7f25bc1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:53.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.779+0000 7f25c2400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f25bc10dfb0 con 0x7f25bc109c20 2024-05-27T07:18:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.780+0000 7f25bb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25bc109c20 0x7f25bc1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.780+0000 7f25bb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25bc109c20 0x7f25bc1a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42418/0 (socket says 172.21.15.63:42418) 2024-05-27T07:18:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.780+0000 7f25bb400700 1 -- 172.21.15.63:0/3073461193 learned_addr learned my addr 172.21.15.63:0/3073461193 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:18:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.780+0000 7f25bb400700 1 -- 172.21.15.63:0/3073461193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25bc1a1b90 con 0x7f25bc109c20 2024-05-27T07:18:53.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.781+0000 7f25bb400700 1 --2- 172.21.15.63:0/3073461193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25bc109c20 0x7f25bc1a1650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25ac005660 tx=0x7f25ac005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:53.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.781+0000 7f25b8c00700 1 -- 172.21.15.63:0/3073461193 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25ac026070 con 0x7f25bc109c20 2024-05-27T07:18:53.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.781+0000 7f25b8c00700 1 -- 172.21.15.63:0/3073461193 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25ac00b750 con 0x7f25bc109c20 2024-05-27T07:18:53.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.781+0000 7f25b8c00700 1 -- 172.21.15.63:0/3073461193 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25ac003810 con 0x7f25bc109c20 2024-05-27T07:18:53.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.781+0000 7f25c2400700 1 -- 172.21.15.63:0/3073461193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25bc1a1e20 con 0x7f25bc109c20 2024-05-27T07:18:53.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.782+0000 7f25c2400700 1 -- 172.21.15.63:0/3073461193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25bc19b870 con 0x7f25bc109c20 2024-05-27T07:18:53.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.783+0000 7f25c2400700 1 -- 172.21.15.63:0/3073461193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25bc1051f0 con 0x7f25bc109c20 2024-05-27T07:18:53.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.783+0000 7f25b8c00700 1 -- 172.21.15.63:0/3073461193 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f25ac029070 con 0x7f25bc109c20 2024-05-27T07:18:53.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.784+0000 7f25b8c00700 1 --2- 172.21.15.63:0/3073461193 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25a003e570 0x7f25a0040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:53.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.784+0000 7f25b8c00700 1 -- 172.21.15.63:0/3073461193 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f25ac01e070 con 0x7f25bc109c20 2024-05-27T07:18:53.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.784+0000 7f25b3e00700 1 --2- 172.21.15.63:0/3073461193 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25a003e570 0x7f25a0040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:53.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.785+0000 7f25b3e00700 1 --2- 172.21.15.63:0/3073461193 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25a003e570 0x7f25a0040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f25a4006fd0 tx=0x7f25a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:53.790+0000 7f25b8c00700 1 -- 172.21.15.63:0/3073461193 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25ac013070 con 0x7f25bc109c20 2024-05-27T07:18:54.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.034+0000 7f25c2400700 1 -- 172.21.15.63:0/3073461193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f25bc19cda0 con 0x7f25bc109c20 2024-05-27T07:18:54.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.639+0000 7f25b8c00700 1 -- 172.21.15.63:0/3073461193 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f25ac004790 con 0x7f25bc109c20 2024-05-27T07:18:54.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.643+0000 7f25c2400700 1 -- 172.21.15.63:0/3073461193 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25a003e570 msgr2=0x7f25a0040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:54.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.643+0000 7f25c2400700 1 --2- 172.21.15.63:0/3073461193 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25a003e570 0x7f25a0040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f25a4006fd0 tx=0x7f25a4006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:54.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.643+0000 7f25c2400700 1 -- 172.21.15.63:0/3073461193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25bc109c20 msgr2=0x7f25bc1a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:54.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.643+0000 7f25c2400700 1 --2- 172.21.15.63:0/3073461193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25bc109c20 0x7f25bc1a1650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25ac005660 tx=0x7f25ac005740 comp rx=0 tx=0).stop 2024-05-27T07:18:54.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.643+0000 7f25c2400700 1 -- 172.21.15.63:0/3073461193 shutdown_connections 2024-05-27T07:18:54.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.643+0000 7f25c2400700 1 --2- 172.21.15.63:0/3073461193 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25a003e570 0x7f25a0040a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:54.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.643+0000 7f25c2400700 1 --2- 172.21.15.63:0/3073461193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f25bc109c20 0x7f25bc1a1650 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:54.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.643+0000 7f25c2400700 1 -- 172.21.15.63:0/3073461193 >> 172.21.15.63:0/3073461193 conn(0x7f25bc0ff660 msgr2=0x7f25bc0ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:54.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.643+0000 7f25c2400700 1 -- 172.21.15.63:0/3073461193 shutdown_connections 2024-05-27T07:18:54.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:18:54.644+0000 7f25c2400700 1 -- 172.21.15.63:0/3073461193 wait complete. 2024-05-27T07:18:54.636 INFO:teuthology.orchestra.run.smithi063.stderr:adjusted tunables profile to default 2024-05-27T07:18:54.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:54 smithi063 ceph-mon[31613]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:54.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:54 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3073461193' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T07:18:55.360 INFO:tasks.cephadm:Adding mon.a on smithi063 2024-05-27T07:18:55.360 INFO:tasks.cephadm:Adding mon.b on smithi086 2024-05-27T07:18:55.360 INFO:tasks.cephadm:Adding mon.c on smithi149 2024-05-27T07:18:55.360 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi063:172.21.15.63=a;smithi086:172.21.15.86=b;smithi149:172.21.15.149=c' 2024-05-27T07:18:55.765 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T07:18:55.868 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T07:18:55.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:55 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3073461193' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T07:18:55.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:55 smithi063 ceph-mon[31613]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:18:56.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:56 smithi063 ceph-mon[31613]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:58.438 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.447+0000 7f9985a00700 1 -- 172.21.15.149:0/1935393242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99801046c0 msgr2=0x7f9980106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:58.438 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.447+0000 7f9985a00700 1 --2- 172.21.15.149:0/1935393242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99801046c0 0x7f9980106ae0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9970009910 tx=0x7f9970009c20 comp rx=0 tx=0).stop 2024-05-27T07:18:58.438 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.447+0000 7f9985a00700 1 -- 172.21.15.149:0/1935393242 shutdown_connections 2024-05-27T07:18:58.438 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.447+0000 7f9985a00700 1 --2- 172.21.15.149:0/1935393242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99801046c0 0x7f9980106ae0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:58.438 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.447+0000 7f9985a00700 1 -- 172.21.15.149:0/1935393242 >> 172.21.15.149:0/1935393242 conn(0x7f99800fe100 msgr2=0x7f99801005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:58.438 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.447+0000 7f9985a00700 1 -- 172.21.15.149:0/1935393242 shutdown_connections 2024-05-27T07:18:58.438 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.447+0000 7f9985a00700 1 -- 172.21.15.149:0/1935393242 wait complete. 2024-05-27T07:18:58.439 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.448+0000 7f9985a00700 1 Processor -- start 2024-05-27T07:18:58.439 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.449+0000 7f9985a00700 1 -- start start 2024-05-27T07:18:58.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.449+0000 7f9985a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99801046c0 0x7f9980199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:58.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.449+0000 7f9985a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9980109a30 con 0x7f99801046c0 2024-05-27T07:18:58.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.450+0000 7f997ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99801046c0 0x7f9980199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:58.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.450+0000 7f997ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99801046c0 0x7f9980199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.149:48936/0 (socket says 172.21.15.149:48936) 2024-05-27T07:18:58.440 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.450+0000 7f997ea00700 1 -- 172.21.15.149:0/1325051978 learned_addr learned my addr 172.21.15.149:0/1325051978 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:18:58.441 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.450+0000 7f997ea00700 1 -- 172.21.15.149:0/1325051978 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9980199b60 con 0x7f99801046c0 2024-05-27T07:18:58.441 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.451+0000 7f997ea00700 1 --2- 172.21.15.149:0/1325051978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99801046c0 0x7f9980199620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9970005790 tx=0x7f9970004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:58.442 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.451+0000 7f9977400700 1 -- 172.21.15.149:0/1325051978 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9970027070 con 0x7f99801046c0 2024-05-27T07:18:58.442 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.451+0000 7f9985a00700 1 -- 172.21.15.149:0/1325051978 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9980199df0 con 0x7f99801046c0 2024-05-27T07:18:58.442 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.451+0000 7f9985a00700 1 -- 172.21.15.149:0/1325051978 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998019ca00 con 0x7f99801046c0 2024-05-27T07:18:58.442 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.452+0000 7f9977400700 1 -- 172.21.15.149:0/1325051978 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9970004830 con 0x7f99801046c0 2024-05-27T07:18:58.442 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.452+0000 7f9977400700 1 -- 172.21.15.149:0/1325051978 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f997000fab0 con 0x7f99801046c0 2024-05-27T07:18:58.443 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.453+0000 7f9985a00700 1 -- 172.21.15.149:0/1325051978 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995c005360 con 0x7f99801046c0 2024-05-27T07:18:58.443 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.453+0000 7f9977400700 1 -- 172.21.15.149:0/1325051978 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f997002a070 con 0x7f99801046c0 2024-05-27T07:18:58.444 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.454+0000 7f9977400700 1 --2- 172.21.15.149:0/1325051978 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f996403e570 0x7f9964040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:18:58.444 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.454+0000 7f9977400700 1 -- 172.21.15.149:0/1325051978 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9970053ae0 con 0x7f99801046c0 2024-05-27T07:18:58.444 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.455+0000 7f9977e00700 1 --2- 172.21.15.149:0/1325051978 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f996403e570 0x7f9964040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:18:58.446 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.456+0000 7f9977e00700 1 --2- 172.21.15.149:0/1325051978 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f996403e570 0x7f9964040a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9968006fd0 tx=0x7f9968006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:18:58.462 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.472+0000 7f9977400700 1 -- 172.21.15.149:0/1325051978 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9970013070 con 0x7f99801046c0 2024-05-27T07:18:58.709 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.718+0000 7f9985a00700 1 -- 172.21.15.149:0/1325051978 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi063:172.21.15.63=a;smithi086:172.21.15.86=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}) -- 0x7f995c000cb0 con 0x7f996403e570 2024-05-27T07:18:58.726 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.736+0000 7f9977400700 1 -- 172.21.15.149:0/1325051978 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f995c000cb0 con 0x7f996403e570 2024-05-27T07:18:58.727 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mon update... 2024-05-27T07:18:58.730 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.739+0000 7f9985a00700 1 -- 172.21.15.149:0/1325051978 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f996403e570 msgr2=0x7f9964040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:58.730 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.739+0000 7f9985a00700 1 --2- 172.21.15.149:0/1325051978 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f996403e570 0x7f9964040a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9968006fd0 tx=0x7f9968006e40 comp rx=0 tx=0).stop 2024-05-27T07:18:58.730 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.739+0000 7f9985a00700 1 -- 172.21.15.149:0/1325051978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99801046c0 msgr2=0x7f9980199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:18:58.730 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.739+0000 7f9985a00700 1 --2- 172.21.15.149:0/1325051978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99801046c0 0x7f9980199620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9970005790 tx=0x7f9970004f60 comp rx=0 tx=0).stop 2024-05-27T07:18:58.730 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.740+0000 7f9985a00700 1 -- 172.21.15.149:0/1325051978 shutdown_connections 2024-05-27T07:18:58.730 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.740+0000 7f9985a00700 1 --2- 172.21.15.149:0/1325051978 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f996403e570 0x7f9964040a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:58.730 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.740+0000 7f9985a00700 1 --2- 172.21.15.149:0/1325051978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99801046c0 0x7f9980199620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:18:58.730 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.740+0000 7f9985a00700 1 -- 172.21.15.149:0/1325051978 >> 172.21.15.149:0/1325051978 conn(0x7f99800fe100 msgr2=0x7f99800feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:18:58.731 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.740+0000 7f9985a00700 1 -- 172.21.15.149:0/1325051978 shutdown_connections 2024-05-27T07:18:58.731 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:18:58.740+0000 7f9985a00700 1 -- 172.21.15.149:0/1325051978 wait complete. 2024-05-27T07:18:59.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:58 smithi063 ceph-mon[31613]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:18:59.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:59.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:59.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:59.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:59.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:18:59.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:18:59.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:18:59.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:18:59.479 DEBUG:teuthology.orchestra.run.smithi086:mon.b> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.b.service 2024-05-27T07:18:59.486 DEBUG:teuthology.orchestra.run.smithi149:mon.c> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.c.service 2024-05-27T07:18:59.491 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T07:18:59.491 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T07:18:59.908 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:00.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:59 smithi063 ceph-mon[31613]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-27T07:19:00.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:59 smithi063 ceph-mon[31613]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:00.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:59 smithi063 ceph-mon[31613]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi063:172.21.15.63=a;smithi086:172.21.15.86=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:00.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:18:59 smithi063 ceph-mon[31613]: Saving service mon spec with placement smithi063:172.21.15.63=a;smithi086:172.21.15.86=b;smithi149:172.21.15.149=c;count:3 2024-05-27T07:19:01.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:00 smithi063 ceph-mon[31613]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:19:01.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:00 smithi063 ceph-mon[31613]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:01.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:01.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:01.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:01.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:01.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:01.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.935+0000 7f6558800700 1 -- 172.21.15.149:0/2562624944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6550075d20 msgr2=0x7f6550076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:01.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.935+0000 7f6558800700 1 --2- 172.21.15.149:0/2562624944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6550075d20 0x7f6550076130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6544009910 tx=0x7f6544009c20 comp rx=0 tx=0).stop 2024-05-27T07:19:01.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.935+0000 7f6558800700 1 -- 172.21.15.149:0/2562624944 shutdown_connections 2024-05-27T07:19:01.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.935+0000 7f6558800700 1 --2- 172.21.15.149:0/2562624944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6550075d20 0x7f6550076130 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:01.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.935+0000 7f6558800700 1 -- 172.21.15.149:0/2562624944 >> 172.21.15.149:0/2562624944 conn(0x7f65500ff690 msgr2=0x7f6550101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:01.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.935+0000 7f6558800700 1 -- 172.21.15.149:0/2562624944 shutdown_connections 2024-05-27T07:19:01.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.936+0000 7f6558800700 1 -- 172.21.15.149:0/2562624944 wait complete. 2024-05-27T07:19:01.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.937+0000 7f6558800700 1 Processor -- start 2024-05-27T07:19:01.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.937+0000 7f6558800700 1 -- start start 2024-05-27T07:19:01.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.937+0000 7f6558800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6550075d20 0x7f65501a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:01.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.937+0000 7f6558800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65501101e0 con 0x7f6550075d20 2024-05-27T07:19:01.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.938+0000 7f6555e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6550075d20 0x7f65501a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:01.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.938+0000 7f6555e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6550075d20 0x7f65501a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.149:51058/0 (socket says 172.21.15.149:51058) 2024-05-27T07:19:01.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.938+0000 7f6555e00700 1 -- 172.21.15.149:0/3816085876 learned_addr learned my addr 172.21.15.149:0/3816085876 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:19:01.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.939+0000 7f6555e00700 1 -- 172.21.15.149:0/3816085876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65501a1c10 con 0x7f6550075d20 2024-05-27T07:19:01.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.939+0000 7f6555e00700 1 --2- 172.21.15.149:0/3816085876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6550075d20 0x7f65501a16d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6544006010 tx=0x7f6544005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:01.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.940+0000 7f654aa00700 1 -- 172.21.15.149:0/3816085876 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6544026070 con 0x7f6550075d20 2024-05-27T07:19:01.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.940+0000 7f6558800700 1 -- 172.21.15.149:0/3816085876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65501a1ea0 con 0x7f6550075d20 2024-05-27T07:19:01.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.940+0000 7f654aa00700 1 -- 172.21.15.149:0/3816085876 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f654400b750 con 0x7f6550075d20 2024-05-27T07:19:01.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.940+0000 7f654aa00700 1 -- 172.21.15.149:0/3816085876 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65440038a0 con 0x7f6550075d20 2024-05-27T07:19:01.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.940+0000 7f6558800700 1 -- 172.21.15.149:0/3816085876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655019b8f0 con 0x7f6550075d20 2024-05-27T07:19:01.931 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.941+0000 7f6558800700 1 -- 172.21.15.149:0/3816085876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6550077290 con 0x7f6550075d20 2024-05-27T07:19:01.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.942+0000 7f654aa00700 1 -- 172.21.15.149:0/3816085876 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6544029070 con 0x7f6550075d20 2024-05-27T07:19:01.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.942+0000 7f654aa00700 1 --2- 172.21.15.149:0/3816085876 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f653803e510 0x7f6538040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:01.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.942+0000 7f654aa00700 1 -- 172.21.15.149:0/3816085876 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f654401e070 con 0x7f6550075d20 2024-05-27T07:19:01.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.943+0000 7f6555400700 1 --2- 172.21.15.149:0/3816085876 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f653803e510 0x7f6538040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:01.934 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.944+0000 7f6555400700 1 --2- 172.21.15.149:0/3816085876 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f653803e510 0x7f6538040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f6540006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:01.949 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:01.959+0000 7f654aa00700 1 -- 172.21.15.149:0/3816085876 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6544013070 con 0x7f6550075d20 2024-05-27T07:19:02.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:01 smithi063 ceph-mon[31613]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:19:02.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:01 smithi063 ceph-mon[31613]: Deploying daemon mon.c on smithi149 2024-05-27T07:19:02.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.288+0000 7f6558800700 1 -- 172.21.15.149:0/3816085876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6550064e10 con 0x7f6550075d20 2024-05-27T07:19:02.280 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.289+0000 7f654aa00700 1 -- 172.21.15.149:0/3816085876 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f6544030090 con 0x7f6550075d20 2024-05-27T07:19:02.280 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:19:02.280 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","modified":"2024-05-27T07:16:40.949861Z","created":"2024-05-27T07:16:40.949861Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T07:19:02.283 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.292+0000 7f6558800700 1 -- 172.21.15.149:0/3816085876 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f653803e510 msgr2=0x7f6538040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:02.284 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.292+0000 7f6558800700 1 --2- 172.21.15.149:0/3816085876 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f653803e510 0x7f6538040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f6540006e40 comp rx=0 tx=0).stop 2024-05-27T07:19:02.284 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.293+0000 7f6558800700 1 -- 172.21.15.149:0/3816085876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6550075d20 msgr2=0x7f65501a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:02.284 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.293+0000 7f6558800700 1 --2- 172.21.15.149:0/3816085876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6550075d20 0x7f65501a16d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6544006010 tx=0x7f6544005710 comp rx=0 tx=0).stop 2024-05-27T07:19:02.284 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.293+0000 7f6558800700 1 -- 172.21.15.149:0/3816085876 shutdown_connections 2024-05-27T07:19:02.284 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.293+0000 7f6558800700 1 --2- 172.21.15.149:0/3816085876 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f653803e510 0x7f6538040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:02.284 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.293+0000 7f6558800700 1 --2- 172.21.15.149:0/3816085876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6550075d20 0x7f65501a16d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:02.284 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.293+0000 7f6558800700 1 -- 172.21.15.149:0/3816085876 >> 172.21.15.149:0/3816085876 conn(0x7f65500ff690 msgr2=0x7f655010c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:02.285 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.293+0000 7f6558800700 1 -- 172.21.15.149:0/3816085876 shutdown_connections 2024-05-27T07:19:02.285 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:02.293+0000 7f6558800700 1 -- 172.21.15.149:0/3816085876 wait complete. 2024-05-27T07:19:02.285 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-05-27T07:19:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:02 smithi063 ceph-mon[31613]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:02 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/3816085876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:19:04.155 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T07:19:04.155 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T07:19:04.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:03 smithi063 ceph-mon[31613]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:05.030 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:19:06.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:06 smithi063 ceph-mon[31613]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:07.736 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.745+0000 7efe20400700 1 -- 172.21.15.149:0/467938197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe18104db0 msgr2=0x7efe181051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:07.737 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.745+0000 7efe20400700 1 --2- 172.21.15.149:0/467938197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe18104db0 0x7efe181051c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009910 tx=0x7efe0c009c20 comp rx=0 tx=0).stop 2024-05-27T07:19:07.737 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.745+0000 7efe20400700 1 -- 172.21.15.149:0/467938197 shutdown_connections 2024-05-27T07:19:07.737 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.745+0000 7efe20400700 1 --2- 172.21.15.149:0/467938197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe18104db0 0x7efe181051c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:07.737 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.745+0000 7efe20400700 1 -- 172.21.15.149:0/467938197 >> 172.21.15.149:0/467938197 conn(0x7efe180fe660 msgr2=0x7efe18100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:07.737 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.745+0000 7efe20400700 1 -- 172.21.15.149:0/467938197 shutdown_connections 2024-05-27T07:19:07.737 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.745+0000 7efe20400700 1 -- 172.21.15.149:0/467938197 wait complete. 2024-05-27T07:19:07.737 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.746+0000 7efe20400700 1 Processor -- start 2024-05-27T07:19:07.737 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.746+0000 7efe20400700 1 -- start start 2024-05-27T07:19:07.738 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.747+0000 7efe20400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe18104db0 0x7efe181a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:07.738 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.747+0000 7efe20400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe1810e0c0 con 0x7efe18104db0 2024-05-27T07:19:07.738 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.748+0000 7efe1da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe18104db0 0x7efe181a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:07.738 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.748+0000 7efe1da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe18104db0 0x7efe181a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.149:51088/0 (socket says 172.21.15.149:51088) 2024-05-27T07:19:07.738 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.748+0000 7efe1da00700 1 -- 172.21.15.149:0/360023579 learned_addr learned my addr 172.21.15.149:0/360023579 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:19:07.739 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.748+0000 7efe1da00700 1 -- 172.21.15.149:0/360023579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe181a1d90 con 0x7efe18104db0 2024-05-27T07:19:07.739 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.749+0000 7efe1da00700 1 --2- 172.21.15.149:0/360023579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe18104db0 0x7efe181a1850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efe0c005060 tx=0x7efe0c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:07.739 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.749+0000 7efe16000700 1 -- 172.21.15.149:0/360023579 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe0c026070 con 0x7efe18104db0 2024-05-27T07:19:07.739 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.749+0000 7efe20400700 1 -- 172.21.15.149:0/360023579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe181a2020 con 0x7efe18104db0 2024-05-27T07:19:07.740 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.749+0000 7efe16000700 1 -- 172.21.15.149:0/360023579 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe0c00b750 con 0x7efe18104db0 2024-05-27T07:19:07.740 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.749+0000 7efe20400700 1 -- 172.21.15.149:0/360023579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1819ba70 con 0x7efe18104db0 2024-05-27T07:19:07.740 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.749+0000 7efe16000700 1 -- 172.21.15.149:0/360023579 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe0c003810 con 0x7efe18104db0 2024-05-27T07:19:07.740 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.750+0000 7efe20400700 1 -- 172.21.15.149:0/360023579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1806bb20 con 0x7efe18104db0 2024-05-27T07:19:07.741 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.751+0000 7efe16000700 1 -- 172.21.15.149:0/360023579 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7efe0c029070 con 0x7efe18104db0 2024-05-27T07:19:07.742 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.752+0000 7efe16000700 1 --2- 172.21.15.149:0/360023579 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efe0003e570 0x7efe00040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:07.742 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.752+0000 7efe16000700 1 -- 172.21.15.149:0/360023579 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe0c01e070 con 0x7efe18104db0 2024-05-27T07:19:07.742 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.752+0000 7efe1d000700 1 --2- 172.21.15.149:0/360023579 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efe0003e570 0x7efe00040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:07.743 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.753+0000 7efe1d000700 1 --2- 172.21.15.149:0/360023579 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efe0003e570 0x7efe00040a80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efe04006fd0 tx=0x7efe04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:07.759 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:07.769+0000 7efe16000700 1 -- 172.21.15.149:0/360023579 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe0c013070 con 0x7efe18104db0 2024-05-27T07:19:08.027 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:07 smithi149 podman[35366]: 2024-05-27 07:19:07.874828466 +0000 UTC m=+0.159187863 container create f1a4fd65cc3f30fbec0745952818fbda3fe9dc5b4a890dba070ca479f8310bb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-c, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:19:08.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.100+0000 7efe20400700 1 -- 172.21.15.149:0/360023579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efe18064e10 con 0x7efe18104db0 2024-05-27T07:19:08.092 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.102+0000 7efe16000700 1 -- 172.21.15.149:0/360023579 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7efe0c017440 con 0x7efe18104db0 2024-05-27T07:19:08.093 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:19:08.093 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","modified":"2024-05-27T07:16:40.949861Z","created":"2024-05-27T07:16:40.949861Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T07:19:08.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.105+0000 7efe20400700 1 -- 172.21.15.149:0/360023579 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efe0003e570 msgr2=0x7efe00040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:08.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.105+0000 7efe20400700 1 --2- 172.21.15.149:0/360023579 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efe0003e570 0x7efe00040a80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efe04006fd0 tx=0x7efe04006e40 comp rx=0 tx=0).stop 2024-05-27T07:19:08.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.105+0000 7efe20400700 1 -- 172.21.15.149:0/360023579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe18104db0 msgr2=0x7efe181a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:08.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.105+0000 7efe20400700 1 --2- 172.21.15.149:0/360023579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe18104db0 0x7efe181a1850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efe0c005060 tx=0x7efe0c005710 comp rx=0 tx=0).stop 2024-05-27T07:19:08.095 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.105+0000 7efe20400700 1 -- 172.21.15.149:0/360023579 shutdown_connections 2024-05-27T07:19:08.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.105+0000 7efe20400700 1 --2- 172.21.15.149:0/360023579 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efe0003e570 0x7efe00040a80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:08.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.105+0000 7efe20400700 1 --2- 172.21.15.149:0/360023579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe18104db0 0x7efe181a1850 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:08.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.105+0000 7efe20400700 1 -- 172.21.15.149:0/360023579 >> 172.21.15.149:0/360023579 conn(0x7efe180fe660 msgr2=0x7efe1810a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:08.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.105+0000 7efe20400700 1 -- 172.21.15.149:0/360023579 shutdown_connections 2024-05-27T07:19:08.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:08.106+0000 7efe20400700 1 -- 172.21.15.149:0/360023579 wait complete. 2024-05-27T07:19:08.097 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-05-27T07:19:08.417 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 podman[35366]: 2024-05-27 07:19:08.12715551 +0000 UTC m=+0.411514914 container init f1a4fd65cc3f30fbec0745952818fbda3fe9dc5b4a890dba070ca479f8310bb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-27T07:19:08.417 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 podman[35366]: 2024-05-27 07:19:08.137827847 +0000 UTC m=+0.422187280 container start f1a4fd65cc3f30fbec0745952818fbda3fe9dc5b4a890dba070ca479f8310bb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-c, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-27T07:19:08.417 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T07:19:08.417 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev), process ceph-mon, pid 2 2024-05-27T07:19:08.417 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pidfile_write: ignore empty --pid-file 2024-05-27T07:19:08.417 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: load: jerasure load: lrc 2024-05-27T07:19:08.417 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: RocksDB version: 7.9.2 2024-05-27T07:19:08.417 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Git sha 0 2024-05-27T07:19:08.417 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Compile date 2024-05-21 07:00:12 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: DB SUMMARY 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: DB Session ID: CVHJFT9JMZ82MFC5EU4I 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: CURRENT file: CURRENT 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.error_if_exists: 0 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.create_if_missing: 0 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.paranoid_checks: 1 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T07:19:08.418 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T07:19:08.419 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.env: 0x564bb9062500 2024-05-27T07:19:08.419 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T07:19:08.419 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.info_log: 0x564bd74fc040 2024-05-27T07:19:08.419 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.statistics: (nil) 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.use_fsync: 0 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_log_file_size: 0 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.allow_fallocate: 1 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.use_direct_reads: 0 2024-05-27T07:19:08.420 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.db_log_dir: 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.wal_dir: 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T07:19:08.421 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.write_buffer_manager: 0x564bd73c68c0 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.unordered_write: 0 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T07:19:08.422 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.row_cache: None 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.wal_filter: None 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.two_write_queues: 0 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.wal_compression: 0 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.atomic_flush: 0 2024-05-27T07:19:08.423 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.log_readahead_size: 0 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_background_jobs: 2 2024-05-27T07:19:08.424 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_background_compactions: -1 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_subcompactions: 1 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_open_files: -1 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T07:19:08.425 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_background_flushes: -1 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Compression algorithms supported: 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: kZSTD supported: 0 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: kXpressCompression supported: 0 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: kZlibCompression supported: 1 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: kSnappyCompression supported: 1 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: kLZ4Compression supported: 1 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: kBZip2Compression supported: 0 2024-05-27T07:19:08.426 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.merge_operator: 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_filter: None 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564bd74fc200) 2024-05-27T07:19:08.427 INFO:journalctl@ceph.mon.c.smithi149.stdout: cache_index_and_filter_blocks: 1 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: pin_top_level_index_and_filter: 1 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_type: 0 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: data_block_index_type: 0 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_shortening: 1 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: checksum: 4 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: no_block_cache: 0 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache: 0x564bd73f91f0 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_name: BinnedLRUCache 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_options: 2024-05-27T07:19:08.428 INFO:journalctl@ceph.mon.c.smithi149.stdout: capacity : 536870912 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: num_shard_bits : 4 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: strict_capacity_limit : 0 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: high_pri_pool_ratio: 0.000 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_compressed: (nil) 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: persistent_cache: (nil) 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_size: 4096 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_size_deviation: 10 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_restart_interval: 16 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_block_restart_interval: 1 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: metadata_block_size: 4096 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: partition_filters: 0 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: use_delta_encoding: 1 2024-05-27T07:19:08.429 INFO:journalctl@ceph.mon.c.smithi149.stdout: filter_policy: bloomfilter 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout: whole_key_filtering: 1 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout: verify_compression: 0 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout: read_amp_bytes_per_bit: 0 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout: format_version: 5 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout: enable_index_compression: 1 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_align: 0 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout: max_auto_readahead_size: 262144 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout: prepopulate_block_cache: 0 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout: initial_auto_readahead_size: 8192 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T07:19:08.430 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compression: NoCompression 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.num_levels: 7 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T07:19:08.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T07:19:08.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T07:19:08.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T07:19:08.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T07:19:08.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T07:19:08.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T07:19:08.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T07:19:08.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T07:19:08.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T07:19:08.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T07:19:08.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T07:19:08.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T07:19:08.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T07:19:08.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T07:19:08.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T07:19:08.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T07:19:08.435 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.inplace_update_support: 0 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.bloom_locality: 0 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.max_successive_merges: 0 2024-05-27T07:19:08.436 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.ttl: 2592000 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.enable_blob_files: false 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.min_blob_size: 0 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T07:19:08.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e4ec0588-1efc-4e40-87e9-b82d21d74c8e 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794348265436, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794348266409, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716794348, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e4ec0588-1efc-4e40-87e9-b82d21d74c8e", "db_session_id": "CVHJFT9JMZ82MFC5EU4I", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794348266804, "job": 1, "event": "recovery_finished"} 2024-05-27T07:19:08.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564bd73ddc00 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: DB pointer 0x564bdc1d0000 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** DB Stats ** 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T07:19:08.439 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** Compaction Stats [default] ** 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** Compaction Stats [default] ** 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T07:19:08.440 INFO:journalctl@ceph.mon.c.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: Block cache BinnedLRUCache@0x564bd73f91f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-27T07:19:08.441 INFO:journalctl@ceph.mon.c.smithi149.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: using public_addr v2:172.21.15.149:0/0 -> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: starting mon.c rank -1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(???) e0 preinit fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).mds e1 new map 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout: e1 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout: legacy client fscid: -1 2024-05-27T07:19:08.442 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout: No filesystems configured 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:19:08.443 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Deploying cephadm binary to smithi086 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Added host smithi086 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.444 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.445 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Deploying cephadm binary to smithi149 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Added host smithi149 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:19:08.446 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3073461193' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3073461193' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi063:172.21.15.63=a;smithi086:172.21.15.86=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Saving service mon spec with placement smithi063:172.21.15.63=a;smithi086:172.21.15.86=b;smithi149:172.21.15.149=c;count:3 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:19:08.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: Deploying daemon mon.c on smithi149 2024-05-27T07:19:08.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/3816085876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:19:08.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:08.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 ceph-mon[35408]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-27T07:19:08.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 bash[35366]: f1a4fd65cc3f30fbec0745952818fbda3fe9dc5b4a890dba070ca479f8310bb0 2024-05-27T07:19:08.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:08 smithi149 systemd[1]: Started Ceph mon.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:19:09.972 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T07:19:09.972 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T07:19:10.389 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:19:13.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:19:13.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: mon.a calling monitor election 2024-05-27T07:19:13.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:13.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: mon.c calling monitor election 2024-05-27T07:19:13.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:13.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: monmap epoch 2 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: last_changed 2024-05-27T07:19:08.318261+0000 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: created 2024-05-27T07:16:40.949861+0000 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: min_mon_release 19 (squid) 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: election_strategy: 1 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: fsmap 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: mgrmap e13: x(active, since 69s) 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: overall HEALTH_OK 2024-05-27T07:19:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:13.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:13.708 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:19:13.708 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: mon.a calling monitor election 2024-05-27T07:19:13.708 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.708 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.708 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:13.708 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.708 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: mon.c calling monitor election 2024-05-27T07:19:13.708 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.708 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: monmap epoch 2 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: last_changed 2024-05-27T07:19:08.318261+0000 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: created 2024-05-27T07:16:40.949861+0000 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: min_mon_release 19 (squid) 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: election_strategy: 1 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: fsmap 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:19:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: mgrmap e13: x(active, since 69s) 2024-05-27T07:19:13.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: overall HEALTH_OK 2024-05-27T07:19:13.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:13.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:13.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.825+0000 7fae23a00700 1 -- 172.21.15.149:0/1664764437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae1c075ca0 msgr2=0x7fae1c0760b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:13.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.825+0000 7fae23a00700 1 --2- 172.21.15.149:0/1664764437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae1c075ca0 0x7fae1c0760b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae10005fa0 tx=0x7fae1000fe80 comp rx=0 tx=0).stop 2024-05-27T07:19:13.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.826+0000 7fae23a00700 1 -- 172.21.15.149:0/1664764437 shutdown_connections 2024-05-27T07:19:13.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.826+0000 7fae23a00700 1 --2- 172.21.15.149:0/1664764437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae1c075ca0 0x7fae1c0760b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:13.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.826+0000 7fae23a00700 1 -- 172.21.15.149:0/1664764437 >> 172.21.15.149:0/1664764437 conn(0x7fae1c0ff760 msgr2=0x7fae1c101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:13.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.826+0000 7fae23a00700 1 -- 172.21.15.149:0/1664764437 shutdown_connections 2024-05-27T07:19:13.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.826+0000 7fae23a00700 1 -- 172.21.15.149:0/1664764437 wait complete. 2024-05-27T07:19:13.819 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.829+0000 7fae23a00700 1 Processor -- start 2024-05-27T07:19:13.820 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.830+0000 7fae23a00700 1 -- start start 2024-05-27T07:19:13.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.831+0000 7fae23a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae1c075ca0 0x7fae1c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:13.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.831+0000 7fae23a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c1a1de0 0x7fae1c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:13.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.831+0000 7fae23a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c110290 con 0x7fae1c075ca0 2024-05-27T07:19:13.821 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.831+0000 7fae23a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c110110 con 0x7fae1c1a1de0 2024-05-27T07:19:13.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.832+0000 7fae21000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae1c075ca0 0x7fae1c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:13.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.832+0000 7fae21000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae1c075ca0 0x7fae1c1a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.149:36944/0 (socket says 172.21.15.149:36944) 2024-05-27T07:19:13.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.832+0000 7fae21000700 1 -- 172.21.15.149:0/2715678117 learned_addr learned my addr 172.21.15.149:0/2715678117 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:19:13.822 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.832+0000 7fae1be00700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c1a1de0 0x7fae1c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:13.823 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.832+0000 7fae1be00700 1 -- 172.21.15.149:0/2715678117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c1a1de0 msgr2=0x7fae1c19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-27T07:19:13.823 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.832+0000 7fae1be00700 1 -- 172.21.15.149:0/2715678117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c1a1de0 msgr2=0x7fae1c19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T07:19:13.823 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.832+0000 7fae1be00700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c1a1de0 0x7fae1c19bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T07:19:13.823 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.832+0000 7fae1be00700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c1a1de0 0x7fae1c19bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:19:13.823 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.833+0000 7fae21000700 1 -- 172.21.15.149:0/2715678117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c1a1de0 msgr2=0x7fae1c19bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:19:13.823 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.833+0000 7fae21000700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c1a1de0 0x7fae1c19bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:13.823 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.833+0000 7fae21000700 1 -- 172.21.15.149:0/2715678117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c1a23d0 con 0x7fae1c075ca0 2024-05-27T07:19:13.824 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.834+0000 7fae21000700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae1c075ca0 0x7fae1c1a18a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fae100116c0 tx=0x7fae10003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:13.825 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.834+0000 7fae19600700 1 -- 172.21.15.149:0/2715678117 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fae100045a0 con 0x7fae1c075ca0 2024-05-27T07:19:13.825 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.835+0000 7fae23a00700 1 -- 172.21.15.149:0/2715678117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1c19c020 con 0x7fae1c075ca0 2024-05-27T07:19:13.825 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.835+0000 7fae19600700 1 -- 172.21.15.149:0/2715678117 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae10004740 con 0x7fae1c075ca0 2024-05-27T07:19:13.825 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.835+0000 7fae19600700 1 -- 172.21.15.149:0/2715678117 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fae1001c620 con 0x7fae1c075ca0 2024-05-27T07:19:13.825 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.835+0000 7fae23a00700 1 -- 172.21.15.149:0/2715678117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1c19c5b0 con 0x7fae1c075ca0 2024-05-27T07:19:13.827 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.836+0000 7fae23a00700 1 -- 172.21.15.149:0/2715678117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae1c19c920 con 0x7fae1c075ca0 2024-05-27T07:19:13.828 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.838+0000 7fae19600700 1 -- 172.21.15.149:0/2715678117 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fae10012970 con 0x7fae1c075ca0 2024-05-27T07:19:13.828 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.838+0000 7fae19600700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae0403e320 0x7fae04040830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:13.828 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.838+0000 7fae19600700 1 -- 172.21.15.149:0/2715678117 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fae10022070 con 0x7fae1c075ca0 2024-05-27T07:19:13.829 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.839+0000 7fae1be00700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae0403e320 0x7fae04040830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:13.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.841+0000 7fae1be00700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae0403e320 0x7fae04040830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae1c109340 tx=0x7fae08008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:13.844 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:13.854+0000 7fae19600700 1 -- 172.21.15.149:0/2715678117 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae1001b3d0 con 0x7fae1c075ca0 2024-05-27T07:19:14.181 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.191+0000 7fae23a00700 1 -- 172.21.15.149:0/2715678117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fae1c19cfd0 con 0x7fae1c075ca0 2024-05-27T07:19:14.182 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.192+0000 7fae19600700 1 -- 172.21.15.149:0/2715678117 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fae100056f0 con 0x7fae1c075ca0 2024-05-27T07:19:14.183 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:19:14.183 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","modified":"2024-05-27T07:19:08.318261Z","created":"2024-05-27T07:16:40.949861Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-27T07:19:14.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 -- 172.21.15.149:0/2715678117 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae0403e320 msgr2=0x7fae04040830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:14.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae0403e320 0x7fae04040830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae1c109340 tx=0x7fae08008040 comp rx=0 tx=0).stop 2024-05-27T07:19:14.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 -- 172.21.15.149:0/2715678117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae1c075ca0 msgr2=0x7fae1c1a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:14.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae1c075ca0 0x7fae1c1a18a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fae100116c0 tx=0x7fae10003f50 comp rx=0 tx=0).stop 2024-05-27T07:19:14.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 -- 172.21.15.149:0/2715678117 shutdown_connections 2024-05-27T07:19:14.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae0403e320 0x7fae04040830 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:14.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae1c075ca0 0x7fae1c1a18a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:14.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 --2- 172.21.15.149:0/2715678117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae1c1a1de0 0x7fae1c19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:14.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 -- 172.21.15.149:0/2715678117 >> 172.21.15.149:0/2715678117 conn(0x7fae1c0ff760 msgr2=0x7fae1c10c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:14.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 -- 172.21.15.149:0/2715678117 shutdown_connections 2024-05-27T07:19:14.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:14.196+0000 7fae23a00700 1 -- 172.21.15.149:0/2715678117 wait complete. 2024-05-27T07:19:14.188 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2024-05-27T07:19:14.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:14.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:14.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:14.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:14 smithi149 ceph-mon[35408]: Deploying daemon mon.b on smithi086 2024-05-27T07:19:14.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:14 smithi149 ceph-mon[35408]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:14.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:14 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/2715678117' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:19:14.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:14.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:14.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:14.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:14.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:14 smithi063 ceph-mon[31613]: Deploying daemon mon.b on smithi086 2024-05-27T07:19:14.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:14 smithi063 ceph-mon[31613]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:14.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:14 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/2715678117' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:19:14.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:15.905 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T07:19:15.906 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T07:19:16.367 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:19:16.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:16 smithi149 ceph-mon[35408]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:16.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:16 smithi063 ceph-mon[31613]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:18.360 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.369+0000 7f053b800700 1 -- 172.21.15.149:0/1743895436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534103eb0 msgr2=0x7f05341042c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:18.360 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.369+0000 7f053b800700 1 --2- 172.21.15.149:0/1743895436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534103eb0 0x7f05341042c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0528009910 tx=0x7f0528009c20 comp rx=0 tx=0).stop 2024-05-27T07:19:18.360 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.369+0000 7f053b800700 1 -- 172.21.15.149:0/1743895436 shutdown_connections 2024-05-27T07:19:18.360 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.369+0000 7f053b800700 1 --2- 172.21.15.149:0/1743895436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534103eb0 0x7f05341042c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:18.360 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.369+0000 7f053b800700 1 -- 172.21.15.149:0/1743895436 >> 172.21.15.149:0/1743895436 conn(0x7f05340ff760 msgr2=0x7f0534101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:18.360 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.369+0000 7f053b800700 1 -- 172.21.15.149:0/1743895436 shutdown_connections 2024-05-27T07:19:18.360 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.369+0000 7f053b800700 1 -- 172.21.15.149:0/1743895436 wait complete. 2024-05-27T07:19:18.361 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.371+0000 7f053b800700 1 Processor -- start 2024-05-27T07:19:18.361 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.371+0000 7f053b800700 1 -- start start 2024-05-27T07:19:18.362 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.372+0000 7f053b800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534103eb0 0x7f053419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:18.362 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.372+0000 7f053b800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053419fbf0 0x7f05341998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:18.362 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.372+0000 7f053b800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f053410e050 con 0x7f0534103eb0 2024-05-27T07:19:18.362 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.372+0000 7f053b800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f053410ded0 con 0x7f053419fbf0 2024-05-27T07:19:18.363 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.372+0000 7f0533e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053419fbf0 0x7f05341998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:18.363 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.372+0000 7f0533e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053419fbf0 0x7f05341998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:58734/0 (socket says 172.21.15.149:58734) 2024-05-27T07:19:18.363 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.372+0000 7f0533e00700 1 -- 172.21.15.149:0/1130461130 learned_addr learned my addr 172.21.15.149:0/1130461130 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:19:18.363 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.372+0000 7f0538e00700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534103eb0 0x7f053419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:18.363 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.373+0000 7f0533e00700 1 -- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534103eb0 msgr2=0x7f053419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:18.363 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.373+0000 7f0533e00700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534103eb0 0x7f053419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:18.364 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.373+0000 7f0533e00700 1 -- 172.21.15.149:0/1130461130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05341a01c0 con 0x7f053419fbf0 2024-05-27T07:19:18.364 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.373+0000 7f0538e00700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534103eb0 0x7f053419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:19:18.364 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.373+0000 7f0533e00700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053419fbf0 0x7f05341998d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f052000b700 tx=0x7f052000bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:18.364 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.373+0000 7f0531600700 1 -- 172.21.15.149:0/1130461130 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0520011630 con 0x7f053419fbf0 2024-05-27T07:19:18.364 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.373+0000 7f0531600700 1 -- 172.21.15.149:0/1130461130 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0520011c60 con 0x7f053419fbf0 2024-05-27T07:19:18.364 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.373+0000 7f0531600700 1 -- 172.21.15.149:0/1130461130 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0520010c90 con 0x7f053419fbf0 2024-05-27T07:19:18.365 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.373+0000 7f053b800700 1 -- 172.21.15.149:0/1130461130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0534199e70 con 0x7f053419fbf0 2024-05-27T07:19:18.365 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.374+0000 7f053b800700 1 -- 172.21.15.149:0/1130461130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053419a400 con 0x7f053419fbf0 2024-05-27T07:19:18.365 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.375+0000 7f053b800700 1 -- 172.21.15.149:0/1130461130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053419a710 con 0x7f053419fbf0 2024-05-27T07:19:18.365 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.375+0000 7f0531600700 1 -- 172.21.15.149:0/1130461130 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0520018020 con 0x7f053419fbf0 2024-05-27T07:19:18.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.376+0000 7f0531600700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f051c03e2b0 0x7f051c0407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:18.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.376+0000 7f0531600700 1 -- 172.21.15.149:0/1130461130 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f05200522e0 con 0x7f053419fbf0 2024-05-27T07:19:18.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.376+0000 7f0538e00700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f051c03e2b0 0x7f051c0407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:18.367 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.377+0000 7f0538e00700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f051c03e2b0 0x7f051c0407c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0528006010 tx=0x7f052801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:18.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.382+0000 7f0531600700 1 -- 172.21.15.149:0/1130461130 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f052001f020 con 0x7f053419fbf0 2024-05-27T07:19:18.699 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.708+0000 7f053b800700 1 -- 172.21.15.149:0/1130461130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f053419ae60 con 0x7f053419fbf0 2024-05-27T07:19:18.699 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.709+0000 7f0531600700 1 -- 172.21.15.149:0/1130461130 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f0520018310 con 0x7f053419fbf0 2024-05-27T07:19:18.700 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:19:18.700 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","modified":"2024-05-27T07:19:08.318261Z","created":"2024-05-27T07:16:40.949861Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-27T07:19:18.703 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.712+0000 7f053b800700 1 -- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f051c03e2b0 msgr2=0x7f051c0407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:18.703 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.712+0000 7f053b800700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f051c03e2b0 0x7f051c0407c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0528006010 tx=0x7f052801a040 comp rx=0 tx=0).stop 2024-05-27T07:19:18.703 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.712+0000 7f053b800700 1 -- 172.21.15.149:0/1130461130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053419fbf0 msgr2=0x7f05341998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:18.703 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.712+0000 7f053b800700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053419fbf0 0x7f05341998d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f052000b700 tx=0x7f052000bac0 comp rx=0 tx=0).stop 2024-05-27T07:19:18.703 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.712+0000 7f053b800700 1 -- 172.21.15.149:0/1130461130 shutdown_connections 2024-05-27T07:19:18.703 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.712+0000 7f053b800700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f051c03e2b0 0x7f051c0407c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:18.703 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.713+0000 7f053b800700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0534103eb0 0x7f053419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:18.704 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.713+0000 7f053b800700 1 --2- 172.21.15.149:0/1130461130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053419fbf0 0x7f05341998d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:18.704 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.713+0000 7f053b800700 1 -- 172.21.15.149:0/1130461130 >> 172.21.15.149:0/1130461130 conn(0x7f05340ff760 msgr2=0x7f053410b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:18.704 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.713+0000 7f053b800700 1 -- 172.21.15.149:0/1130461130 shutdown_connections 2024-05-27T07:19:18.704 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:18.713+0000 7f053b800700 1 -- 172.21.15.149:0/1130461130 wait complete. 2024-05-27T07:19:18.704 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2024-05-27T07:19:18.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:18 smithi149 ceph-mon[35408]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:18.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:18 smithi063 ceph-mon[31613]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.177 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 podman[34669]: 2024-05-27 07:19:19.175690097 +0000 UTC m=+0.372139463 container init 4467d0c1d437a4c1ac627761ead9cde45b76e5d1bb99fc4a7c1252a1c9700c04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:19:19.178 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 podman[34669]: 2024-05-27 07:19:19.185626302 +0000 UTC m=+0.382075649 container start 4467d0c1d437a4c1ac627761ead9cde45b76e5d1bb99fc4a7c1252a1c9700c04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-27T07:19:19.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 bash[34669]: 4467d0c1d437a4c1ac627761ead9cde45b76e5d1bb99fc4a7c1252a1c9700c04 2024-05-27T07:19:19.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 systemd[1]: Started Ceph mon.b for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:19:19.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T07:19:19.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev), process ceph-mon, pid 2 2024-05-27T07:19:19.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pidfile_write: ignore empty --pid-file 2024-05-27T07:19:19.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: load: jerasure load: lrc 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: RocksDB version: 7.9.2 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Git sha 0 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Compile date 2024-05-21 07:00:12 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: DB SUMMARY 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: DB Session ID: F8PQGRTHDRNCJR3OTPSH 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: CURRENT file: CURRENT 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-27T07:19:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.error_if_exists: 0 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.create_if_missing: 0 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.paranoid_checks: 1 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.env: 0x55beef35d500 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.info_log: 0x55bf14a98040 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.statistics: (nil) 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.use_fsync: 0 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_log_file_size: 0 2024-05-27T07:19:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T07:19:19.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T07:19:19.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T07:19:19.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T07:19:19.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.allow_fallocate: 1 2024-05-27T07:19:19.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T07:19:19.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T07:19:19.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.use_direct_reads: 0 2024-05-27T07:19:19.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T07:19:19.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T07:19:19.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.db_log_dir: 2024-05-27T07:19:19.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.wal_dir: 2024-05-27T07:19:19.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T07:19:19.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T07:19:19.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T07:19:19.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T07:19:19.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T07:19:19.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.write_buffer_manager: 0x55bf149628c0 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T07:19:19.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.unordered_write: 0 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.row_cache: None 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.wal_filter: None 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.two_write_queues: 0 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T07:19:19.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.wal_compression: 0 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.atomic_flush: 0 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.log_readahead_size: 0 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T07:19:19.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_background_jobs: 2 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_background_compactions: -1 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_subcompactions: 1 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T07:19:19.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_open_files: -1 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_background_flushes: -1 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Compression algorithms supported: 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: kZSTD supported: 0 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: kXpressCompression supported: 0 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: kZlibCompression supported: 1 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: kSnappyCompression supported: 1 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: kLZ4Compression supported: 1 2024-05-27T07:19:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: kBZip2Compression supported: 0 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.merge_operator: 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_filter: None 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bf14a98200) 2024-05-27T07:19:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_type: 0 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: data_block_index_type: 0 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_shortening: 1 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: checksum: 4 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: no_block_cache: 0 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache: 0x55bf149951f0 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-05-27T07:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_options: 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: capacity : 536870912 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: num_shard_bits : 4 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: strict_capacity_limit : 0 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_cache_compressed: (nil) 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: persistent_cache: (nil) 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_size: 4096 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_size_deviation: 10 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_restart_interval: 16 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: index_block_restart_interval: 1 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: metadata_block_size: 4096 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: partition_filters: 0 2024-05-27T07:19:19.462 INFO:journalctl@ceph.mon.b.smithi086.stdout: use_delta_encoding: 1 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: filter_policy: bloomfilter 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: whole_key_filtering: 1 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: verify_compression: 0 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: format_version: 5 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: enable_index_compression: 1 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: block_align: 0 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: max_auto_readahead_size: 262144 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: prepopulate_block_cache: 0 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: initial_auto_readahead_size: 8192 2024-05-27T07:19:19.463 INFO:journalctl@ceph.mon.b.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compression: NoCompression 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.num_levels: 7 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T07:19:19.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T07:19:19.465 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T07:19:19.467 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:19:19.467 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T07:19:19.467 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T07:19:19.467 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:19:19.467 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:19:19.468 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T07:19:19.468 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T07:19:19.468 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T07:19:19.468 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T07:19:19.468 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:19:19.468 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:19:19.468 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T07:19:19.469 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T07:19:19.470 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T07:19:19.471 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.inplace_update_support: 0 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.bloom_locality: 0 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.max_successive_merges: 0 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.ttl: 2592000 2024-05-27T07:19:19.472 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.enable_blob_files: false 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.min_blob_size: 0 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-27T07:19:19.473 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f38b1b1b-64c8-4e8b-be77-45853a6dda91 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794359318376, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794359319711, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716794359, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f38b1b1b-64c8-4e8b-be77-45853a6dda91", "db_session_id": "F8PQGRTHDRNCJR3OTPSH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794359320333, "job": 1, "event": "recovery_finished"} 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bf14979c00 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: DB pointer 0x55bf1976c000 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T07:19:19.474 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** DB Stats ** 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** Compaction Stats [default] ** 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:19:19.475 INFO:journalctl@ceph.mon.b.smithi086.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** Compaction Stats [default] ** 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T07:19:19.476 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout: Cumulative compaction: 0.00 GB write, 0.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout: Interval compaction: 0.00 GB write, 0.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout: Block cache BinnedLRUCache@0x55bf149951f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: using public_addr v2:172.21.15.86:0/0 -> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: starting mon.b rank -1 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:19.477 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(???) e0 preinit fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).mds e1 new map 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout: e1 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout: legacy client fscid: -1 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout: 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout: No filesystems configured 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-27T07:19:19.478 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-27T07:19:19.479 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-27T07:19:19.479 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T07:19:19.479 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:19:19.479 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:19:19.479 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:19:19.479 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:19:19.479 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.479 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Deploying cephadm binary to smithi086 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Added host smithi086 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:19.480 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.481 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Deploying cephadm binary to smithi149 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Added host smithi149 2024-05-27T07:19:19.482 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.483 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3073461193' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T07:19:19.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3073461193' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T07:19:19.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:19:19.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:19:19.484 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi063:172.21.15.63=a;smithi086:172.21.15.86=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Saving service mon spec with placement smithi063:172.21.15.63=a;smithi086:172.21.15.86=b;smithi149:172.21.15.149=c;count:3 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.485 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Deploying daemon mon.c on smithi149 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/3816085876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.a calling monitor election 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:19.486 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.c calling monitor election 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: monmap epoch 2 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: last_changed 2024-05-27T07:19:08.318261+0000 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: created 2024-05-27T07:16:40.949861+0000 2024-05-27T07:19:19.487 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: min_mon_release 19 (squid) 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: election_strategy: 1 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: fsmap 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mgrmap e13: x(active, since 69s) 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: overall HEALTH_OK 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:19.488 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:19.489 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:19.489 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: Deploying daemon mon.b on smithi086 2024-05-27T07:19:19.489 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.489 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/2715678117' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:19:19.489 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:19.489 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.489 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:19.489 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:19 smithi086 ceph-mon[34687]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-05-27T07:19:20.423 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T07:19:20.424 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T07:19:20.853 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:19:24.909 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:19:24.909 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: mon.a calling monitor election 2024-05-27T07:19:24.909 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.909 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:24.909 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: mon.c calling monitor election 2024-05-27T07:19:24.909 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:24.909 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.909 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.909 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: mon.b calling monitor election 2024-05-27T07:19:24.909 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: monmap epoch 3 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: last_changed 2024-05-27T07:19:19.387981+0000 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: created 2024-05-27T07:16:40.949861+0000 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: min_mon_release 19 (squid) 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: election_strategy: 1 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-27T07:19:24.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-27T07:19:24.911 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-05-27T07:19:24.911 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: fsmap 2024-05-27T07:19:24.911 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:19:24.911 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: mgrmap e13: x(active, since 80s) 2024-05-27T07:19:24.911 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: overall HEALTH_OK 2024-05-27T07:19:24.911 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:19:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: mon.a calling monitor election 2024-05-27T07:19:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: mon.c calling monitor election 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: mon.b calling monitor election 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: monmap epoch 3 2024-05-27T07:19:24.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:24.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: last_changed 2024-05-27T07:19:19.387981+0000 2024-05-27T07:19:24.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: created 2024-05-27T07:16:40.949861+0000 2024-05-27T07:19:24.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: min_mon_release 19 (squid) 2024-05-27T07:19:24.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: election_strategy: 1 2024-05-27T07:19:24.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-27T07:19:24.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-27T07:19:24.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-05-27T07:19:24.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: fsmap 2024-05-27T07:19:24.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:19:24.956 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: mgrmap e13: x(active, since 80s) 2024-05-27T07:19:24.956 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: overall HEALTH_OK 2024-05-27T07:19:24.956 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:24.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:19:24.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: mon.a calling monitor election 2024-05-27T07:19:24.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:19:24.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: mon.c calling monitor election 2024-05-27T07:19:24.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:24.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: mon.b calling monitor election 2024-05-27T07:19:24.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: monmap epoch 3 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: last_changed 2024-05-27T07:19:19.387981+0000 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: created 2024-05-27T07:16:40.949861+0000 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: min_mon_release 19 (squid) 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: election_strategy: 1 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: 0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-27T07:19:24.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-05-27T07:19:24.960 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: 2: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-05-27T07:19:24.960 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: fsmap 2024-05-27T07:19:24.960 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:19:24.960 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: mgrmap e13: x(active, since 80s) 2024-05-27T07:19:24.960 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: overall HEALTH_OK 2024-05-27T07:19:24.960 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:25.626 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:25 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:25.627 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:25 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:25.627 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:25 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:25.627 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:25 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:25.887 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:25 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:25.887 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:25 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:25.887 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:25 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:25.887 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:25 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:25.888 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.896+0000 7ff61f400700 1 -- 172.21.15.149:0/1234833835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6181060f0 msgr2=0x7ff600005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:25.888 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.896+0000 7ff61f400700 1 --2- 172.21.15.149:0/1234833835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6181060f0 0x7ff600005610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff60c005f40 tx=0x7ff60c011f90 comp rx=0 tx=0).stop 2024-05-27T07:19:25.888 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.896+0000 7ff61f400700 1 -- 172.21.15.149:0/1234833835 shutdown_connections 2024-05-27T07:19:25.888 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.896+0000 7ff61f400700 1 --2- 172.21.15.149:0/1234833835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6181060f0 0x7ff600005610 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:25.888 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.896+0000 7ff61f400700 1 -- 172.21.15.149:0/1234833835 >> 172.21.15.149:0/1234833835 conn(0x7ff618101980 msgr2=0x7ff618103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:25.888 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.896+0000 7ff61f400700 1 -- 172.21.15.149:0/1234833835 shutdown_connections 2024-05-27T07:19:25.888 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.896+0000 7ff61f400700 1 -- 172.21.15.149:0/1234833835 wait complete. 2024-05-27T07:19:25.889 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.898+0000 7ff61f400700 1 Processor -- start 2024-05-27T07:19:25.889 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.899+0000 7ff61f400700 1 -- start start 2024-05-27T07:19:25.890 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.899+0000 7ff61f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181060f0 0x7ff61819bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:25.890 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.900+0000 7ff61f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff61819c2d0 0x7ff6181a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:25.890 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.900+0000 7ff61f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6181a0b00 0x7ff6181a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:25.890 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.900+0000 7ff61f400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff61810dce0 con 0x7ff6181a0b00 2024-05-27T07:19:25.890 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.900+0000 7ff61f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff61810db60 con 0x7ff6181060f0 2024-05-27T07:19:25.890 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.900+0000 7ff61f400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff61810de60 con 0x7ff61819c2d0 2024-05-27T07:19:25.891 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.900+0000 7ff61ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181060f0 0x7ff61819bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:25.891 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.900+0000 7ff61d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6181a0b00 0x7ff6181a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:25.891 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.900+0000 7ff61ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181060f0 0x7ff61819bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.149:54934/0 (socket says 172.21.15.149:54934) 2024-05-27T07:19:25.891 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.900+0000 7ff61ca00700 1 -- 172.21.15.149:0/470023119 learned_addr learned my addr 172.21.15.149:0/470023119 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:19:25.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.901+0000 7ff617e00700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff61819c2d0 0x7ff6181a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:25.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.901+0000 7ff61ca00700 1 -- 172.21.15.149:0/470023119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181060f0 msgr2=0x7ff61819bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-27T07:19:25.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.901+0000 7ff61ca00700 1 -- 172.21.15.149:0/470023119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181060f0 msgr2=0x7ff61819bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T07:19:25.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.901+0000 7ff61ca00700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181060f0 0x7ff61819bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T07:19:25.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.901+0000 7ff61ca00700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181060f0 0x7ff61819bd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:19:25.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.901+0000 7ff61d400700 1 -- 172.21.15.149:0/470023119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff61819c2d0 msgr2=0x7ff6181a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:25.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.901+0000 7ff61d400700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff61819c2d0 0x7ff6181a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:25.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.901+0000 7ff61d400700 1 -- 172.21.15.149:0/470023119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181060f0 msgr2=0x7ff61819bd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:19:25.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.901+0000 7ff61d400700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181060f0 0x7ff61819bd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:25.892 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.901+0000 7ff61d400700 1 -- 172.21.15.149:0/470023119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6181a8870 con 0x7ff6181a0b00 2024-05-27T07:19:25.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.902+0000 7ff61d400700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6181a0b00 0x7ff6181a0f10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff61000bd50 tx=0x7ff61000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:25.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.902+0000 7ff615600700 1 -- 172.21.15.149:0/470023119 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610019070 con 0x7ff6181a0b00 2024-05-27T07:19:25.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.902+0000 7ff61f400700 1 -- 172.21.15.149:0/470023119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6181a8aa0 con 0x7ff6181a0b00 2024-05-27T07:19:25.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.902+0000 7ff615600700 1 -- 172.21.15.149:0/470023119 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff61000cc90 con 0x7ff6181a0b00 2024-05-27T07:19:25.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.902+0000 7ff615600700 1 -- 172.21.15.149:0/470023119 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100125a0 con 0x7ff6181a0b00 2024-05-27T07:19:25.893 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.902+0000 7ff61f400700 1 -- 172.21.15.149:0/470023119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6181a8f90 con 0x7ff6181a0b00 2024-05-27T07:19:25.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.904+0000 7ff61f400700 1 -- 172.21.15.149:0/470023119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff618064e10 con 0x7ff6181a0b00 2024-05-27T07:19:25.894 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.904+0000 7ff615600700 1 -- 172.21.15.149:0/470023119 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff61001c070 con 0x7ff6181a0b00 2024-05-27T07:19:25.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.905+0000 7ff615600700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5f803e9a0 0x7ff5f8040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:25.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.905+0000 7ff615600700 1 -- 172.21.15.149:0/470023119 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff610052110 con 0x7ff6181a0b00 2024-05-27T07:19:25.895 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.905+0000 7ff61ca00700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5f803e9a0 0x7ff5f8040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:25.896 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.906+0000 7ff61ca00700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5f803e9a0 0x7ff5f8040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff60c011af0 tx=0x7ff60c005a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:25.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:25.915+0000 7ff615600700 1 -- 172.21.15.149:0/470023119 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff61001c360 con 0x7ff6181a0b00 2024-05-27T07:19:25.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:25 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:25.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:25 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:25.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:25 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:25.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:25 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:19:26.235 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.244+0000 7ff61f400700 1 -- 172.21.15.149:0/470023119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff61802e7a0 con 0x7ff6181a0b00 2024-05-27T07:19:26.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.246+0000 7ff615600700 1 -- 172.21.15.149:0/470023119 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7ff610013e10 con 0x7ff6181a0b00 2024-05-27T07:19:26.236 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-27T07:19:26.237 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":3,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","modified":"2024-05-27T07:19:19.387981Z","created":"2024-05-27T07:16:40.949861Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-27T07:19:26.239 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 -- 172.21.15.149:0/470023119 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5f803e9a0 msgr2=0x7ff5f8040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:26.239 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5f803e9a0 0x7ff5f8040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff60c011af0 tx=0x7ff60c005a30 comp rx=0 tx=0).stop 2024-05-27T07:19:26.239 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 -- 172.21.15.149:0/470023119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6181a0b00 msgr2=0x7ff6181a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:26.240 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6181a0b00 0x7ff6181a0f10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff61000bd50 tx=0x7ff61000bd80 comp rx=0 tx=0).stop 2024-05-27T07:19:26.240 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 -- 172.21.15.149:0/470023119 shutdown_connections 2024-05-27T07:19:26.240 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5f803e9a0 0x7ff5f8040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:26.240 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6181a0b00 0x7ff6181a0f10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:26.240 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181060f0 0x7ff61819bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:26.240 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 --2- 172.21.15.149:0/470023119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff61819c2d0 0x7ff6181a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:26.240 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 -- 172.21.15.149:0/470023119 >> 172.21.15.149:0/470023119 conn(0x7ff618101980 msgr2=0x7ff6181098f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:26.240 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 -- 172.21.15.149:0/470023119 shutdown_connections 2024-05-27T07:19:26.240 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:26.249+0000 7ff61f400700 1 -- 172.21.15.149:0/470023119 wait complete. 2024-05-27T07:19:26.241 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 3 2024-05-27T07:19:26.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:26 smithi086 ceph-mon[34687]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:26.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:26 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/470023119' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:19:26.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:26 smithi149 ceph-mon[35408]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:26.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:26 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/470023119' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:19:26.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:26 smithi063 ceph-mon[31613]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:26.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:26 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/470023119' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:19:27.524 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-27T07:19:27.524 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-05-27T07:19:27.950 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:19:27.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:27 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:27.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:27 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:27.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:27 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:27.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:27 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:27.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:27 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:27.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:27 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:28.020 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:27 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.020 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:27 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.020 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:27 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.020 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:27 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.020 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:27 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:28.021 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:27 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:28.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:27 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:27 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:27 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:27 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:27 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:28.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:27 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:28.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-27T07:19:28.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-27T07:19:28.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.157 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-27T07:19:29.158 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-27T07:19:29.158 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-27T07:19:29.158 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:29.158 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:29.158 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:29.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:29.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.159 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-27T07:19:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-27T07:19:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-27T07:19:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.059+0000 7f2811600700 1 -- 172.21.15.63:0/4033073537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280c10a3c0 msgr2=0x7f280c10a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.059+0000 7f2811600700 1 --2- 172.21.15.63:0/4033073537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280c10a3c0 0x7f280c10a7d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009910 tx=0x7f27fc009c20 comp rx=0 tx=0).stop 2024-05-27T07:19:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.059+0000 7f2811600700 1 -- 172.21.15.63:0/4033073537 shutdown_connections 2024-05-27T07:19:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.059+0000 7f2811600700 1 --2- 172.21.15.63:0/4033073537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280c10a3c0 0x7f280c10a7d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.059+0000 7f2811600700 1 -- 172.21.15.63:0/4033073537 >> 172.21.15.63:0/4033073537 conn(0x7f280c0799e0 msgr2=0x7f280c1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.059+0000 7f2811600700 1 -- 172.21.15.63:0/4033073537 shutdown_connections 2024-05-27T07:19:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.059+0000 7f2811600700 1 -- 172.21.15.63:0/4033073537 wait complete. 2024-05-27T07:19:30.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.060+0000 7f2811600700 1 Processor -- start 2024-05-27T07:19:30.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.060+0000 7f2811600700 1 -- start start 2024-05-27T07:19:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.060+0000 7f2811600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f280c10a3c0 0x7f280c1a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.060+0000 7f2811600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280c1a0660 0x7f280c1a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.060+0000 7f2811600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f280c1a4e90 0x7f280c1a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2811600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f280c114500 con 0x7f280c1a0660 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2811600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f280c114380 con 0x7f280c10a3c0 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2811600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f280c114680 con 0x7f280c1a4e90 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2803e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280c1a0660 0x7f280c1a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2803e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280c1a0660 0x7f280c1a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44046/0 (socket says 172.21.15.63:44046) 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2803e00700 1 -- 172.21.15.63:0/2874892141 learned_addr learned my addr 172.21.15.63:0/2874892141 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f280aa00700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f280c10a3c0 0x7f280c1a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f280b400700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f280c1a4e90 0x7f280c1a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2803e00700 1 -- 172.21.15.63:0/2874892141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f280c1a4e90 msgr2=0x7f280c1a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2803e00700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f280c1a4e90 0x7f280c1a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2803e00700 1 -- 172.21.15.63:0/2874892141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f280c10a3c0 msgr2=0x7f280c1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:30.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2803e00700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f280c10a3c0 0x7f280c1a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:30.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.061+0000 7f2803e00700 1 -- 172.21.15.63:0/2874892141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c1acc00 con 0x7f280c1a0660 2024-05-27T07:19:30.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.062+0000 7f280b400700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f280c1a4e90 0x7f280c1a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:19:30.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.062+0000 7f2803e00700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280c1a0660 0x7f280c1a4950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f27f400b780 tx=0x7f27f400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:30.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.062+0000 7f2803400700 1 -- 172.21.15.63:0/2874892141 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4004230 con 0x7f280c1a0660 2024-05-27T07:19:30.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.062+0000 7f2803400700 1 -- 172.21.15.63:0/2874892141 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27f40043d0 con 0x7f280c1a0660 2024-05-27T07:19:30.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.062+0000 7f2803400700 1 -- 172.21.15.63:0/2874892141 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4011ac0 con 0x7f280c1a0660 2024-05-27T07:19:30.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.062+0000 7f2811600700 1 -- 172.21.15.63:0/2874892141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c1ace30 con 0x7f280c1a0660 2024-05-27T07:19:30.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.062+0000 7f2811600700 1 -- 172.21.15.63:0/2874892141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c1ad230 con 0x7f280c1a0660 2024-05-27T07:19:30.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.064+0000 7f2811600700 1 -- 172.21.15.63:0/2874892141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c064e10 con 0x7f280c1a0660 2024-05-27T07:19:30.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.064+0000 7f2803400700 1 -- 172.21.15.63:0/2874892141 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f27f4011ca0 con 0x7f280c1a0660 2024-05-27T07:19:30.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.064+0000 7f2803400700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27f003e9a0 0x7f27f0040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:30.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.064+0000 7f2803400700 1 -- 172.21.15.63:0/2874892141 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f27f4052410 con 0x7f280c1a0660 2024-05-27T07:19:30.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.065+0000 7f280aa00700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27f003e9a0 0x7f27f0040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:30.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.065+0000 7f280aa00700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27f003e9a0 0x7f27f0040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b5c0 tx=0x7f27fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:30.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.071+0000 7f2803400700 1 -- 172.21.15.63:0/2874892141 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27f400d280 con 0x7f280c1a0660 2024-05-27T07:19:30.315 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.315 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.315 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.315 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.315 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:30 smithi063 ceph-mon[31613]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T07:19:30.315 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:30.316 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:19:30.316 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:30.316 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:30 smithi063 ceph-mon[31613]: Reconfiguring daemon mon.a on smithi063 2024-05-27T07:19:30.316 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:30 smithi063 ceph-mon[31613]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.329+0000 7f2811600700 1 -- 172.21.15.63:0/2874892141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f280c1ad6f0 con 0x7f280c1a0660 2024-05-27T07:19:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.330+0000 7f2803400700 1 -- 172.21.15.63:0/2874892141 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f27f4019e20 con 0x7f280c1a0660 2024-05-27T07:19:30.322 INFO:teuthology.orchestra.run.smithi063.stdout:# minimal ceph.conf for e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:30.322 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-27T07:19:30.322 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:19:30.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon_host = [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-05-27T07:19:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.333+0000 7f2811600700 1 -- 172.21.15.63:0/2874892141 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27f003e9a0 msgr2=0x7f27f0040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.333+0000 7f2811600700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27f003e9a0 0x7f27f0040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00b5c0 tx=0x7f27fc005fb0 comp rx=0 tx=0).stop 2024-05-27T07:19:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.333+0000 7f2811600700 1 -- 172.21.15.63:0/2874892141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280c1a0660 msgr2=0x7f280c1a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.333+0000 7f2811600700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280c1a0660 0x7f280c1a4950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f27f400b780 tx=0x7f27f400bb40 comp rx=0 tx=0).stop 2024-05-27T07:19:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.334+0000 7f2811600700 1 -- 172.21.15.63:0/2874892141 shutdown_connections 2024-05-27T07:19:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.334+0000 7f2811600700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f280c1a4e90 0x7f280c1a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.334+0000 7f2811600700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f280c10a3c0 0x7f280c1a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:30.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.334+0000 7f2811600700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f27f003e9a0 0x7f27f0040eb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.334+0000 7f2811600700 1 --2- 172.21.15.63:0/2874892141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f280c1a0660 0x7f280c1a4950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.334+0000 7f2811600700 1 -- 172.21.15.63:0/2874892141 >> 172.21.15.63:0/2874892141 conn(0x7f280c0799e0 msgr2=0x7f280c10bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.334+0000 7f2811600700 1 -- 172.21.15.63:0/2874892141 shutdown_connections 2024-05-27T07:19:30.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:30.334+0000 7f2811600700 1 -- 172.21.15.63:0/2874892141 wait complete. 2024-05-27T07:19:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:30 smithi086 ceph-mon[34687]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T07:19:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:19:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:30.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:30 smithi086 ceph-mon[34687]: Reconfiguring daemon mon.a on smithi063 2024-05-27T07:19:30.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:30 smithi086 ceph-mon[34687]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:30.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:30.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:30 smithi149 ceph-mon[35408]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T07:19:30.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:30.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:19:30.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:30.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:30 smithi149 ceph-mon[35408]: Reconfiguring daemon mon.a on smithi063 2024-05-27T07:19:30.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:30 smithi149 ceph-mon[35408]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:31.163 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:31 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2874892141' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:31.366 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-27T07:19:31.366 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:19:31.366 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T07:19:31.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:31 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2874892141' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:31.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:31 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2874892141' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:31.455 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:19:31.455 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:19:31.573 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:19:31.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T07:19:31.667 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:19:31.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:19:31.793 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:19:31.793 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T07:19:31.881 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:19:31.881 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:19:32.008 INFO:tasks.cephadm:Adding mgr.x on smithi063 2024-05-27T07:19:32.008 INFO:tasks.cephadm:Adding mgr.y on smithi086 2024-05-27T07:19:32.008 INFO:tasks.cephadm:Adding mgr.z on smithi149 2024-05-27T07:19:32.008 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi063=x;smithi086=y;smithi149=z' 2024-05-27T07:19:32.477 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:19:32.948 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:32 smithi149 ceph-mon[35408]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:32.948 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:32.948 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:32.949 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:32 smithi149 ceph-mon[35408]: Reconfiguring mon.b (monmap changed)... 2024-05-27T07:19:32.949 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:32.949 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:19:32.949 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:32.949 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:32 smithi149 ceph-mon[35408]: Reconfiguring daemon mon.b on smithi086 2024-05-27T07:19:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:32 smithi086 ceph-mon[34687]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:32 smithi086 ceph-mon[34687]: Reconfiguring mon.b (monmap changed)... 2024-05-27T07:19:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:19:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:33.005 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:32 smithi086 ceph-mon[34687]: Reconfiguring daemon mon.b on smithi086 2024-05-27T07:19:33.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:32 smithi063 ceph-mon[31613]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:33.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:33.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:33.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:32 smithi063 ceph-mon[31613]: Reconfiguring mon.b (monmap changed)... 2024-05-27T07:19:33.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:33.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:19:33.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:33.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:32 smithi063 ceph-mon[31613]: Reconfiguring daemon mon.b on smithi086 2024-05-27T07:19:34.519 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.527+0000 7f749be00700 1 -- 172.21.15.149:0/2749547744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f749c104660 msgr2=0x7f749c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:34.519 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.527+0000 7f749be00700 1 --2- 172.21.15.149:0/2749547744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f749c104660 0x7f749c106a80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f748c009910 tx=0x7f748c009c20 comp rx=0 tx=0).stop 2024-05-27T07:19:34.519 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.527+0000 7f749be00700 1 -- 172.21.15.149:0/2749547744 shutdown_connections 2024-05-27T07:19:34.519 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.527+0000 7f749be00700 1 --2- 172.21.15.149:0/2749547744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f749c104660 0x7f749c106a80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:34.519 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.527+0000 7f749be00700 1 -- 172.21.15.149:0/2749547744 >> 172.21.15.149:0/2749547744 conn(0x7f749c0fe040 msgr2=0x7f749c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:34.519 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.527+0000 7f749be00700 1 -- 172.21.15.149:0/2749547744 shutdown_connections 2024-05-27T07:19:34.519 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.527+0000 7f749be00700 1 -- 172.21.15.149:0/2749547744 wait complete. 2024-05-27T07:19:34.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.528+0000 7f749be00700 1 Processor -- start 2024-05-27T07:19:34.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.529+0000 7f749be00700 1 -- start start 2024-05-27T07:19:34.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.529+0000 7f749be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749c104660 0x7f749c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:34.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.529+0000 7f749be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f749c19be10 0x7f749c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:34.520 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.529+0000 7f749be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f749c1a12e0 0x7f749c1a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:34.521 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.529+0000 7f749be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f749c109900 con 0x7f749c19be10 2024-05-27T07:19:34.521 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.529+0000 7f749be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f749c109780 con 0x7f749c104660 2024-05-27T07:19:34.521 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.529+0000 7f749be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f749c109a80 con 0x7f749c1a12e0 2024-05-27T07:19:34.522 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.530+0000 7f749b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f749c1a12e0 0x7f749c1a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:34.522 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.530+0000 7f749b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f749c1a12e0 0x7f749c1a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:59612/0 (socket says 172.21.15.149:59612) 2024-05-27T07:19:34.522 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.530+0000 7f749b400700 1 -- 172.21.15.149:0/2042051137 learned_addr learned my addr 172.21.15.149:0/2042051137 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:19:34.522 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.530+0000 7f749a000700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f749c19be10 0x7f749c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:34.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.530+0000 7f749aa00700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749c104660 0x7f749c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:34.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.530+0000 7f749b400700 1 -- 172.21.15.149:0/2042051137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749c104660 msgr2=0x7f749c19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:34.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.530+0000 7f749b400700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749c104660 0x7f749c19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:34.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.530+0000 7f749b400700 1 -- 172.21.15.149:0/2042051137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f749c19be10 msgr2=0x7f749c1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:34.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.530+0000 7f749b400700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f749c19be10 0x7f749c1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:34.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.530+0000 7f749b400700 1 -- 172.21.15.149:0/2042051137 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c1a3c70 con 0x7f749c1a12e0 2024-05-27T07:19:34.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.531+0000 7f749b400700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f749c1a12e0 0x7f749c1a3700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f749400ba50 tx=0x7f749400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:34.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.531+0000 7f7493400700 1 -- 172.21.15.149:0/2042051137 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494019070 con 0x7f749c1a12e0 2024-05-27T07:19:34.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.531+0000 7f7493400700 1 -- 172.21.15.149:0/2042051137 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f749400ce40 con 0x7f749c1a12e0 2024-05-27T07:19:34.523 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.531+0000 7f749be00700 1 -- 172.21.15.149:0/2042051137 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749c1a3f60 con 0x7f749c1a12e0 2024-05-27T07:19:34.524 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.531+0000 7f7493400700 1 -- 172.21.15.149:0/2042051137 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494007350 con 0x7f749c1a12e0 2024-05-27T07:19:34.524 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.531+0000 7f749be00700 1 -- 172.21.15.149:0/2042051137 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749c1a4440 con 0x7f749c1a12e0 2024-05-27T07:19:34.524 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.533+0000 7f7490c00700 1 -- 172.21.15.149:0/2042051137 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749c064e40 con 0x7f749c1a12e0 2024-05-27T07:19:34.524 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.533+0000 7f7493400700 1 -- 172.21.15.149:0/2042051137 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f749401c070 con 0x7f749c1a12e0 2024-05-27T07:19:34.525 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.534+0000 7f7493400700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f748003e9a0 0x7f7480040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:34.525 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.534+0000 7f7493400700 1 -- 172.21.15.149:0/2042051137 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7494051d50 con 0x7f749c1a12e0 2024-05-27T07:19:34.525 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.535+0000 7f749aa00700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f748003e9a0 0x7f7480040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:34.527 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.536+0000 7f749aa00700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f748003e9a0 0x7f7480040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f748c00b5c0 tx=0x7f748c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:34.531 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.540+0000 7f7493400700 1 -- 172.21.15.149:0/2042051137 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f749c064e40 con 0x7f749c1a12e0 2024-05-27T07:19:34.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.823+0000 7f7490c00700 1 -- 172.21.15.149:0/2042051137 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi063=x;smithi086=y;smithi149=z", "target": ["mon-mgr", ""]}) -- 0x7f749c064820 con 0x7f748003e9a0 2024-05-27T07:19:34.831 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.840+0000 7f7493400700 1 -- 172.21.15.149:0/2042051137 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f749c064820 con 0x7f748003e9a0 2024-05-27T07:19:34.831 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mgr update... 2024-05-27T07:19:34.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 -- 172.21.15.149:0/2042051137 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f748003e9a0 msgr2=0x7f7480040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f748003e9a0 0x7f7480040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f748c00b5c0 tx=0x7f748c01a040 comp rx=0 tx=0).stop 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 -- 172.21.15.149:0/2042051137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f749c1a12e0 msgr2=0x7f749c1a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f749c1a12e0 0x7f749c1a3700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f749400ba50 tx=0x7f749400bd60 comp rx=0 tx=0).stop 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 -- 172.21.15.149:0/2042051137 shutdown_connections 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f749c1a12e0 0x7f749c1a3700 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749c104660 0x7f749c19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f748003e9a0 0x7f7480040eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 --2- 172.21.15.149:0/2042051137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f749c19be10 0x7f749c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 -- 172.21.15.149:0/2042051137 >> 172.21.15.149:0/2042051137 conn(0x7f749c0fe040 msgr2=0x7f749c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 -- 172.21.15.149:0/2042051137 shutdown_connections 2024-05-27T07:19:34.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:19:34.843+0000 7f7490c00700 1 -- 172.21.15.149:0/2042051137 wait complete. 2024-05-27T07:19:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:34 smithi086 ceph-mon[34687]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:34 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:34 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:34 smithi086 ceph-mon[34687]: Reconfiguring mon.c (monmap changed)... 2024-05-27T07:19:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:34 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:34 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:19:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:34 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:35.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:34 smithi086 ceph-mon[34687]: Reconfiguring daemon mon.c on smithi149 2024-05-27T07:19:35.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:34 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:34 smithi149 ceph-mon[35408]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:34 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:34 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:34 smithi149 ceph-mon[35408]: Reconfiguring mon.c (monmap changed)... 2024-05-27T07:19:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:34 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:34 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:19:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:34 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:34 smithi149 ceph-mon[35408]: Reconfiguring daemon mon.c on smithi149 2024-05-27T07:19:35.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:34 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:35.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:34 smithi063 ceph-mon[31613]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:35.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:34 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:35.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:34 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:35.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:34 smithi063 ceph-mon[31613]: Reconfiguring mon.c (monmap changed)... 2024-05-27T07:19:35.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:34 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:19:35.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:34 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:19:35.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:34 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:35.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:34 smithi063 ceph-mon[31613]: Reconfiguring daemon mon.c on smithi149 2024-05-27T07:19:35.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:34 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:35.548 DEBUG:teuthology.orchestra.run.smithi086:mgr.y> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.y.service 2024-05-27T07:19:35.556 DEBUG:teuthology.orchestra.run.smithi149:mgr.z> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.z.service 2024-05-27T07:19:35.563 INFO:tasks.cephadm:Deploying OSDs... 2024-05-27T07:19:35.563 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:19:35.563 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T07:19:35.624 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T07:19:35.624 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-05-27T07:19:35.680 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T07:19:35.681 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:35.681 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-27T07:19:35.681 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:35.681 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:35.681 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-27 07:18:20.156285924 +0000 2024-05-27T07:19:35.681 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-27 07:11:56.261916947 +0000 2024-05-27T07:19:35.681 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-27 07:11:56.261916947 +0000 2024-05-27T07:19:35.681 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-27 07:11:56.261916947 +0000 2024-05-27T07:19:35.681 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T07:19:35.767 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-27T07:19:35.767 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-27T07:19:35.767 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000405259 s, 1.3 MB/s 2024-05-27T07:19:35.771 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T07:19:35.830 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-05-27T07:19:35.882 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T07:19:35.883 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-27T07:19:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-27 07:18:20.214285380 +0000 2024-05-27T07:19:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-27 07:11:56.862911227 +0000 2024-05-27T07:19:35.883 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-27 07:11:56.862911227 +0000 2024-05-27T07:19:35.883 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-27 07:11:56.862911227 +0000 2024-05-27T07:19:35.883 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T07:19:35.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:35 smithi086 ceph-mon[34687]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi063=x;smithi086=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:35.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:35 smithi086 ceph-mon[34687]: Saving service mgr spec with placement smithi063=x;smithi086=y;smithi149=z;count:3 2024-05-27T07:19:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-27T07:19:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-27T07:19:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000336903 s, 1.5 MB/s 2024-05-27T07:19:35.959 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T07:19:36.012 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-05-27T07:19:36.060 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T07:19:36.061 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:36.061 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-27T07:19:36.061 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:36.061 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:36.061 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-27 07:18:20.265284902 +0000 2024-05-27T07:19:36.061 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-27 07:11:57.468905459 +0000 2024-05-27T07:19:36.061 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-27 07:11:57.468905459 +0000 2024-05-27T07:19:36.061 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-27 07:11:57.468905459 +0000 2024-05-27T07:19:36.061 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T07:19:36.134 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:35 smithi063 ceph-mon[31613]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi063=x;smithi086=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:36.134 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:35 smithi063 ceph-mon[31613]: Saving service mgr spec with placement smithi063=x;smithi086=y;smithi149=z;count:3 2024-05-27T07:19:36.142 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-27T07:19:36.142 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-27T07:19:36.142 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000376897 s, 1.4 MB/s 2024-05-27T07:19:36.147 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T07:19:36.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:35 smithi149 ceph-mon[35408]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi063=x;smithi086=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:36.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:35 smithi149 ceph-mon[35408]: Saving service mgr spec with placement smithi063=x;smithi086=y;smithi149=z;count:3 2024-05-27T07:19:36.207 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-05-27T07:19:36.261 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T07:19:36.261 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:36.261 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-27T07:19:36.261 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:36.261 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:36.261 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-27 07:18:20.316284424 +0000 2024-05-27T07:19:36.262 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-27 07:11:58.093899510 +0000 2024-05-27T07:19:36.262 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-27 07:11:58.093899510 +0000 2024-05-27T07:19:36.262 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-27 07:11:58.093899510 +0000 2024-05-27T07:19:36.262 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T07:19:36.345 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-27T07:19:36.345 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-27T07:19:36.345 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000377635 s, 1.4 MB/s 2024-05-27T07:19:36.349 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T07:19:36.411 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-05-27T07:19:36.411 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T07:19:36.467 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T07:19:36.467 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-05-27T07:19:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T07:19:36.519 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:36.519 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-27T07:19:36.519 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:36.519 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:36.519 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-05-27 07:19:18.417451110 +0000 2024-05-27T07:19:36.520 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-05-27 07:11:49.274760493 +0000 2024-05-27T07:19:36.520 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-05-27 07:11:49.274760493 +0000 2024-05-27T07:19:36.520 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-05-27 07:11:49.274760493 +0000 2024-05-27T07:19:36.520 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T07:19:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-05-27T07:19:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-05-27T07:19:36.598 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000385292 s, 1.3 MB/s 2024-05-27T07:19:36.601 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T07:19:36.654 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-05-27T07:19:36.699 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T07:19:36.699 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:36.699 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-05-27T07:19:36.699 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:36.700 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:36.700 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-05-27 07:19:18.418451089 +0000 2024-05-27T07:19:36.700 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-05-27 07:11:49.853748363 +0000 2024-05-27T07:19:36.700 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-05-27 07:11:49.853748363 +0000 2024-05-27T07:19:36.700 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-05-27 07:11:49.853748363 +0000 2024-05-27T07:19:36.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T07:19:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-05-27T07:19:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-05-27T07:19:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.00037331 s, 1.4 MB/s 2024-05-27T07:19:36.821 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T07:19:36.874 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-05-27T07:19:36.922 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T07:19:36.923 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:36.923 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-05-27T07:19:36.923 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:36.923 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:36.923 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-05-27 07:19:18.419451068 +0000 2024-05-27T07:19:36.923 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-05-27 07:11:50.446735940 +0000 2024-05-27T07:19:36.923 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-05-27 07:11:50.446735940 +0000 2024-05-27T07:19:36.923 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-05-27 07:11:50.446735940 +0000 2024-05-27T07:19:36.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T07:19:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-05-27T07:19:36.995 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-05-27T07:19:36.996 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000382571 s, 1.3 MB/s 2024-05-27T07:19:36.999 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T07:19:37.051 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-05-27T07:19:37.097 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T07:19:37.097 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:37.097 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-05-27T07:19:37.097 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:37.097 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:37.097 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-05-27 07:19:18.420451047 +0000 2024-05-27T07:19:37.097 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-05-27 07:11:51.015724020 +0000 2024-05-27T07:19:37.097 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-05-27 07:11:51.015724020 +0000 2024-05-27T07:19:37.097 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-05-27 07:11:51.015724020 +0000 2024-05-27T07:19:37.098 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T07:19:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:19:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:19:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:19:37.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:19:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:19:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:19:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:37.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:37.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:37.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:19:37.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:19:37.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:19:37.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:37.219 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-05-27T07:19:37.219 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-05-27T07:19:37.220 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000340402 s, 1.5 MB/s 2024-05-27T07:19:37.226 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T07:19:37.282 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-27T07:19:37.282 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T07:19:37.324 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T07:19:37.324 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-05-27T07:19:37.413 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T07:19:37.413 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:37.413 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-05-27T07:19:37.413 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:37.413 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:37.413 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-27 07:19:07.146386708 +0000 2024-05-27T07:19:37.413 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-27 07:11:50.322172758 +0000 2024-05-27T07:19:37.413 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-27 07:11:50.322172758 +0000 2024-05-27T07:19:37.414 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-27 07:11:50.322172758 +0000 2024-05-27T07:19:37.414 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T07:19:37.485 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-27T07:19:37.486 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-27T07:19:37.486 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000399697 s, 1.3 MB/s 2024-05-27T07:19:37.489 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T07:19:37.540 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-05-27T07:19:37.580 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T07:19:37.580 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:37.580 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-27T07:19:37.580 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:37.580 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:37.580 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-27 07:19:07.147386688 +0000 2024-05-27T07:19:37.580 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-27 07:11:50.914161041 +0000 2024-05-27T07:19:37.580 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-27 07:11:50.914161041 +0000 2024-05-27T07:19:37.580 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-27 07:11:50.914161041 +0000 2024-05-27T07:19:37.581 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T07:19:37.697 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-27T07:19:37.697 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-27T07:19:37.697 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000400106 s, 1.3 MB/s 2024-05-27T07:19:37.701 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T07:19:37.757 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-05-27T07:19:37.804 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T07:19:37.804 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:37.804 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-05-27T07:19:37.804 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:37.805 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:37.805 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-27 07:19:07.147386688 +0000 2024-05-27T07:19:37.805 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-27 07:11:51.499149462 +0000 2024-05-27T07:19:37.805 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-27 07:11:51.499149462 +0000 2024-05-27T07:19:37.805 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-27 07:11:51.499149462 +0000 2024-05-27T07:19:37.805 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T07:19:37.864 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-27T07:19:37.864 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-27T07:19:37.864 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000285977 s, 1.8 MB/s 2024-05-27T07:19:37.867 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T07:19:37.909 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-05-27T07:19:37.956 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:37 smithi149 ceph-mon[35408]: Deploying daemon mgr.z on smithi149 2024-05-27T07:19:37.999 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T07:19:38.000 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:19:38.000 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-27T07:19:38.000 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:19:38.000 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:19:38.000 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-27 07:19:07.148386669 +0000 2024-05-27T07:19:38.000 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-27 07:11:52.071138140 +0000 2024-05-27T07:19:38.000 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-27 07:11:52.071138140 +0000 2024-05-27T07:19:38.000 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-27 07:11:52.071138140 +0000 2024-05-27T07:19:38.000 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T07:19:38.079 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-27T07:19:38.079 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-27T07:19:38.079 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000297255 s, 1.7 MB/s 2024-05-27T07:19:38.083 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T07:19:38.130 INFO:tasks.cephadm:Deploying osd.0 on smithi063 with /dev/vg_nvme/lv_4... 2024-05-27T07:19:38.131 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T07:19:38.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:37 smithi086 ceph-mon[34687]: Deploying daemon mgr.z on smithi149 2024-05-27T07:19:38.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:37 smithi063 ceph-mon[31613]: Deploying daemon mgr.z on smithi149 2024-05-27T07:19:38.583 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:19:39.054 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:38 smithi063 ceph-mon[31613]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:39.191 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:38 smithi149 ceph-mon[35408]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:38 smithi086 ceph-mon[34687]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:39.452 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:39 smithi149 systemd[1]: Starting Ceph mgr.z for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:19:39.883 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:39 smithi149 podman[37400]: 2024-05-27 07:19:39.516174674 +0000 UTC m=+0.034516279 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:19:39.883 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:39 smithi149 podman[37400]: 2024-05-27 07:19:39.654623726 +0000 UTC m=+0.172965383 container create d58d6f862e235a38c820f7da4fea0c038fb789beb929beff261e3d24ae1a62a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:19:39.883 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:39 smithi149 podman[37400]: 2024-05-27 07:19:39.891554077 +0000 UTC m=+0.409895699 container init d58d6f862e235a38c820f7da4fea0c038fb789beb929beff261e3d24ae1a62a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521) 2024-05-27T07:19:40.203 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:39 smithi149 podman[37400]: 2024-05-27 07:19:39.901494124 +0000 UTC m=+0.419835748 container start d58d6f862e235a38c820f7da4fea0c038fb789beb929beff261e3d24ae1a62a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-27T07:19:40.203 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:39 smithi149 bash[37400]: d58d6f862e235a38c820f7da4fea0c038fb789beb929beff261e3d24ae1a62a8 2024-05-27T07:19:40.204 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:40 smithi149 systemd[1]: Started Ceph mgr.z for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:19:40.204 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:40 smithi149 ceph-mgr[37419]: -- 172.21.15.149:0/1571492035 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e68a525a0 con 0x555e6d7dc000 2024-05-27T07:19:40.466 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:40 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:40.222+0000 7f78af3b1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:19:40.467 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:40 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:40.474+0000 7f78af3b1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:19:40.875 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:40 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:40.610+0000 7f78af3b1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:19:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:40 smithi086 ceph-mon[34687]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:41.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:19:41.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:19:41.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:19:41.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:41.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:40 smithi149 ceph-mon[35408]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:41.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:41.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:41.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:41.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:19:41.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:19:41.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:19:41.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:41.204 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:41 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:41.015+0000 7f78af3b1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:19:41.204 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:41 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:41.132+0000 7f78af3b1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:19:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:40 smithi063 ceph-mon[31613]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:19:41.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:19:41.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:19:41.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:41.492 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T07:19:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T07:19:41.493 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-27T07:19:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-27T07:19:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0406084 s, 258 MB/s 2024-05-27T07:19:41.493 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-27T07:19:42.137 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:41 smithi063 ceph-mon[31613]: Deploying daemon mgr.y on smithi086 2024-05-27T07:19:42.170 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:41 smithi086 ceph-mon[34687]: Deploying daemon mgr.y on smithi086 2024-05-27T07:19:42.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:41 smithi149 ceph-mon[35408]: Deploying daemon mgr.y on smithi086 2024-05-27T07:19:42.210 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_4 2024-05-27T07:19:42.669 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:19:42.941 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:42 smithi086 ceph-mon[34687]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:43.099 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:42 smithi063 ceph-mon[31613]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:43.179 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:42 smithi149 ceph-mon[35408]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:43.179 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:42.735+0000 7f78af3b1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:19:43.179 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:42.967+0000 7f78af3b1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:19:43.179 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:43.188+0000 7f78af3b1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:19:43.199 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:42 smithi086 systemd[1]: Starting Ceph mgr.y for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:19:43.616 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:43 smithi086 podman[36048]: 2024-05-27 07:19:43.213874766 +0000 UTC m=+0.033595662 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:19:43.616 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:43 smithi086 podman[36048]: 2024-05-27 07:19:43.322764149 +0000 UTC m=+0.142485072 container create df98b87f3baf9416f705d97a779f493a68373cf765c3865be68182c5c3f883d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-27T07:19:43.616 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:43 smithi086 podman[36048]: 2024-05-27 07:19:43.542781765 +0000 UTC m=+0.362502672 container init df98b87f3baf9416f705d97a779f493a68373cf765c3865be68182c5c3f883d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:19:43.616 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:43 smithi086 podman[36048]: 2024-05-27 07:19:43.552999477 +0000 UTC m=+0.372720395 container start df98b87f3baf9416f705d97a779f493a68373cf765c3865be68182c5c3f883d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521) 2024-05-27T07:19:43.616 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:43 smithi086 bash[36048]: df98b87f3baf9416f705d97a779f493a68373cf765c3865be68182c5c3f883d3 2024-05-27T07:19:43.949 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:43 smithi086 systemd[1]: Started Ceph mgr.y for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:19:43.949 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:43 smithi086 ceph-mgr[36068]: -- 172.21.15.86:0/3951286492 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642fa9ca5a0 con 0x5642ff754900 2024-05-27T07:19:43.949 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:43 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:43.872+0000 7fd9d1555200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:19:43.953 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:43.574+0000 7f78af3b1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:19:43.953 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:43.698+0000 7f78af3b1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:19:44.213 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:43.962+0000 7f78af3b1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:19:44.213 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:44.223+0000 7f78af3b1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:19:44.448 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:44 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:44.136+0000 7fd9d1555200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:19:44.448 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:44 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:44.279+0000 7fd9d1555200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:19:44.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.703+0000 7f289be00700 1 -- 172.21.15.63:0/2635953916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28940776d0 msgr2=0x7f2894075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:44.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.703+0000 7f289be00700 1 --2- 172.21.15.63:0/2635953916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28940776d0 0x7f2894075ba0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f28880099c0 tx=0x7f2888009cd0 comp rx=0 tx=0).stop 2024-05-27T07:19:44.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.703+0000 7f289be00700 1 -- 172.21.15.63:0/2635953916 shutdown_connections 2024-05-27T07:19:44.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.703+0000 7f289be00700 1 --2- 172.21.15.63:0/2635953916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2894110980 0x7f2894112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:44.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.703+0000 7f289be00700 1 --2- 172.21.15.63:0/2635953916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28940776d0 0x7f2894075ba0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:44.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.703+0000 7f289be00700 1 --2- 172.21.15.63:0/2635953916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28940760e0 0x7f2894076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:44.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.703+0000 7f289be00700 1 -- 172.21.15.63:0/2635953916 >> 172.21.15.63:0/2635953916 conn(0x7f28940ff9b0 msgr2=0x7f2894101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:19:44.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.703+0000 7f289be00700 1 -- 172.21.15.63:0/2635953916 shutdown_connections 2024-05-27T07:19:44.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.704+0000 7f289be00700 1 -- 172.21.15.63:0/2635953916 wait complete. 2024-05-27T07:19:44.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.705+0000 7f289be00700 1 Processor -- start 2024-05-27T07:19:44.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.705+0000 7f289be00700 1 -- start start 2024-05-27T07:19:44.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.705+0000 7f289be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28940760e0 0x7f28941a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:44.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.705+0000 7f289be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28940776d0 0x7f28941a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:44.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.705+0000 7f289be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2894110980 0x7f28941a8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:44.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.705+0000 7f289be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2894116270 con 0x7f28940776d0 2024-05-27T07:19:44.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.705+0000 7f289be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28941160f0 con 0x7f28940760e0 2024-05-27T07:19:44.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.705+0000 7f289be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f28941163f0 con 0x7f2894110980 2024-05-27T07:19:44.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.705+0000 7f2898a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28940776d0 0x7f28941a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:44.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.706+0000 7f2898a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28940776d0 0x7f28941a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33080/0 (socket says 172.21.15.63:33080) 2024-05-27T07:19:44.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.706+0000 7f2898a00700 1 -- 172.21.15.63:0/1505820469 learned_addr learned my addr 172.21.15.63:0/1505820469 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:19:44.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.706+0000 7f2899400700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28940760e0 0x7f28941a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:44.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.706+0000 7f2899e00700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2894110980 0x7f28941a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:44.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.706+0000 7f2898a00700 1 -- 172.21.15.63:0/1505820469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2894110980 msgr2=0x7f28941a8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:44.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.706+0000 7f2898a00700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2894110980 0x7f28941a8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:44.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.706+0000 7f2898a00700 1 -- 172.21.15.63:0/1505820469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28940760e0 msgr2=0x7f28941a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:19:44.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.706+0000 7f2898a00700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28940760e0 0x7f28941a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:19:44.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.706+0000 7f2898a00700 1 -- 172.21.15.63:0/1505820469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28941a89d0 con 0x7f28940776d0 2024-05-27T07:19:44.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.707+0000 7f2898a00700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28940776d0 0x7f28941a13b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f288000e9a0 tx=0x7f288000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:44.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.707+0000 7f2892000700 1 -- 172.21.15.63:0/1505820469 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288000ca10 con 0x7f28940776d0 2024-05-27T07:19:44.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.707+0000 7f2892000700 1 -- 172.21.15.63:0/1505820469 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f288000cbb0 con 0x7f28940776d0 2024-05-27T07:19:44.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.707+0000 7f2892000700 1 -- 172.21.15.63:0/1505820469 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880018680 con 0x7f28940776d0 2024-05-27T07:19:44.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.707+0000 7f289be00700 1 -- 172.21.15.63:0/1505820469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28941a8cc0 con 0x7f28940776d0 2024-05-27T07:19:44.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.707+0000 7f289be00700 1 -- 172.21.15.63:0/1505820469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941a91a0 con 0x7f28940776d0 2024-05-27T07:19:44.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.709+0000 7f289be00700 1 -- 172.21.15.63:0/1505820469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28940512b0 con 0x7f28940776d0 2024-05-27T07:19:44.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.709+0000 7f2892000700 1 -- 172.21.15.63:0/1505820469 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2880010430 con 0x7f28940776d0 2024-05-27T07:19:44.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.709+0000 7f2892000700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f287c03e750 0x7f287c040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:19:44.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.709+0000 7f2892000700 1 -- 172.21.15.63:0/1505820469 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2880052b50 con 0x7f28940776d0 2024-05-27T07:19:44.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.710+0000 7f2899400700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f287c03e750 0x7f287c040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:19:44.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.710+0000 7f2899400700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f287c03e750 0x7f287c040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2888000c00 tx=0x7f288801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:19:44.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.716+0000 7f2892000700 1 -- 172.21.15.63:0/1505820469 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2880054e20 con 0x7f28940776d0 2024-05-27T07:19:44.948 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:44 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:44.706+0000 7fd9d1555200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:19:44.948 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:44 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:44.825+0000 7fd9d1555200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:19:44.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:44 smithi086 ceph-mon[34687]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:44.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:44.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:44.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:44.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:44.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:44.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:44 smithi063 ceph-mon[31613]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:44.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:44.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:44.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:44.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:44.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:44.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:44.966+0000 7f289be00700 1 -- 172.21.15.63:0/1505820469 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2894063bf0 con 0x7f287c03e750 2024-05-27T07:19:45.094 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:44 smithi149 ceph-mon[35408]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:45.094 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:45.094 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:45.094 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:45.094 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:45.094 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:45.405 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:45 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:45.103+0000 7f78af3b1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:19:45.703 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:45 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:45.414+0000 7f78af3b1200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:19:46.072 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:19:46.072 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:19:46.072 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:46.120 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:45 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:45.992+0000 7f78af3b1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:19:46.120 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:19:46.120 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:19:46.120 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:46.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:19:46.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:19:46.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:46.452 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:46 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:46.129+0000 7f78af3b1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:19:46.781 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:46 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:46.506+0000 7fd9d1555200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:19:46.781 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:46 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:46.740+0000 7fd9d1555200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:19:46.781 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:46 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:46.629+0000 7f78af3b1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:19:46.927 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:46 smithi063 ceph-mon[31613]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:46.927 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:46 smithi063 ceph-mon[31613]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:46.927 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:46 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:46.927 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:46 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:46.927 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:46 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:46.927 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:46 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.092 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:46 smithi149 ceph-mon[35408]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:47.093 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:46 smithi149 ceph-mon[35408]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:47.093 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:46 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.093 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:46 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.093 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:46 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.093 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:46 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.093 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:47.102+0000 7f78af3b1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:19:47.198 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:46 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:46.960+0000 7fd9d1555200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:19:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:46 smithi086 ceph-mon[34687]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:19:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:46 smithi086 ceph-mon[34687]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:46 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:46 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:46 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:46 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.453 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:47.230+0000 7f78af3b1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:19:47.698 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:47 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:47.353+0000 7fd9d1555200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:19:47.699 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:47 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:47.477+0000 7fd9d1555200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:19:47.874 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:47 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:47.874 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:47 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:47.874 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:47 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.874 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:47 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:19:47.874 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:47 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:19:47.874 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:47 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:47.875 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:47.562+0000 7f78af3b1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:19:47.875 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:47.753+0000 7f78af3b1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:19:47.875 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:47.883+0000 7f78af3b1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:19:47.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:47 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:47.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:47 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:47.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:47 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:47 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:19:47.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:47 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:19:47.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:47 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:47.988 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:47 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:47.737+0000 7fd9d1555200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:19:47.989 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:47 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:47.997+0000 7fd9d1555200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:19:47.989 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:47 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:47.989 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:47 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:47.989 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:47 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:47.989 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:47 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:19:47.989 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:47 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:19:47.989 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:47 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:48.702 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:48 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:48.293+0000 7f78af3b1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:19:48.703 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:48 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:48.417+0000 7f78af3b1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:19:49.086 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:48 smithi086 ceph-mon[34687]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:49.087 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:48 smithi086 ceph-mon[34687]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T07:19:49.087 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:48 smithi086 ceph-mon[34687]: Reconfiguring daemon mgr.x on smithi063 2024-05-27T07:19:49.087 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:48 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:49.087 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:48 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:48.804+0000 7fd9d1555200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:19:49.087 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:49.095+0000 7fd9d1555200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:19:49.166 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:48 smithi149 ceph-mon[35408]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:49.166 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:48 smithi149 ceph-mon[35408]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T07:19:49.167 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:48 smithi149 ceph-mon[35408]: Reconfiguring daemon mgr.x on smithi063 2024-05-27T07:19:49.167 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:48 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:49.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:48 smithi063 ceph-mon[31613]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:49.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:48 smithi063 ceph-mon[31613]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T07:19:49.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:48 smithi063 ceph-mon[31613]: Reconfiguring daemon mgr.x on smithi063 2024-05-27T07:19:49.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:49.453 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:49 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:49.176+0000 7f78af3b1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:19:49.792 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:19:49 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:19:49.467+0000 7f78af3b1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:19:49.949 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:49.645+0000 7fd9d1555200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:19:49.949 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:49.772+0000 7fd9d1555200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:19:49.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:49 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3295752742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "032421af-e923-4b4a-beb0-2d42713538bf"}]: dispatch 2024-05-27T07:19:49.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:49 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3295752742' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "032421af-e923-4b4a-beb0-2d42713538bf"}]': finished 2024-05-27T07:19:49.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:49 smithi086 ceph-mon[34687]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T07:19:49.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:19:49.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:49 smithi086 ceph-mon[34687]: Standby manager daemon z started 2024-05-27T07:19:49.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:49 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:19:49.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:49 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:19:49.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:49 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:19:49.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:49 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:19:49.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:49 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3295752742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "032421af-e923-4b4a-beb0-2d42713538bf"}]: dispatch 2024-05-27T07:19:49.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:49 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3295752742' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "032421af-e923-4b4a-beb0-2d42713538bf"}]': finished 2024-05-27T07:19:49.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:49 smithi063 ceph-mon[31613]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T07:19:49.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:19:49.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:49 smithi063 ceph-mon[31613]: Standby manager daemon z started 2024-05-27T07:19:49.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:49 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:19:49.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:49 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:19:49.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:49 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:19:49.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:49 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:19:50.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:50.053+0000 7f2892000700 1 -- 172.21.15.63:0/1505820469 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f288001cd30 con 0x7f28940776d0 2024-05-27T07:19:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:49 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3295752742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "032421af-e923-4b4a-beb0-2d42713538bf"}]: dispatch 2024-05-27T07:19:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:49 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3295752742' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "032421af-e923-4b4a-beb0-2d42713538bf"}]': finished 2024-05-27T07:19:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:49 smithi149 ceph-mon[35408]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T07:19:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:19:50.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:49 smithi149 ceph-mon[35408]: Standby manager daemon z started 2024-05-27T07:19:50.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:49 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:19:50.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:49 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:19:50.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:49 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:19:50.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:49 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.149:0/212651564' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:19:50.688 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:50 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:50.247+0000 7fd9d1555200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:19:50.688 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:50 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:50.696+0000 7fd9d1555200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:19:50.949 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:50 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:50.823+0000 7fd9d1555200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:19:50.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:50 smithi086 ceph-mon[34687]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:50.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:50 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2352014088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:19:50.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:50 smithi086 ceph-mon[34687]: mgrmap e14: x(active, since 106s), standbys: z 2024-05-27T07:19:50.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:50 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:19:51.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:50 smithi149 ceph-mon[35408]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:51.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:50 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2352014088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:19:51.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:50 smithi149 ceph-mon[35408]: mgrmap e14: x(active, since 106s), standbys: z 2024-05-27T07:19:51.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:50 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:19:51.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:50 smithi063 ceph-mon[31613]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:51.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:50 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2352014088' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:19:51.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:50 smithi063 ceph-mon[31613]: mgrmap e14: x(active, since 106s), standbys: z 2024-05-27T07:19:51.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:50 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:19:51.441 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:51 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:51.144+0000 7fd9d1555200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:19:51.441 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:51 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:51.322+0000 7fd9d1555200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:19:51.441 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:51 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:51.449+0000 7fd9d1555200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:19:52.199 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:51 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:51.849+0000 7fd9d1555200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:19:52.199 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:51 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:51.973+0000 7fd9d1555200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:19:53.004 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:52 smithi086 ceph-mon[34687]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:53.004 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:52 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:52.725+0000 7fd9d1555200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:19:53.005 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:19:53 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:19:53.013+0000 7fd9d1555200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:19:53.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:52 smithi149 ceph-mon[35408]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:53.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:52 smithi063 ceph-mon[31613]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:53.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:19:53.852+0000 7f2892000700 1 -- 172.21.15.63:0/1505820469 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f288001d210 con 0x7f28940776d0 2024-05-27T07:19:54.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:53 smithi086 ceph-mon[34687]: Standby manager daemon y started 2024-05-27T07:19:54.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:53 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:19:54.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:53 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:19:54.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:53 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:19:54.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:53 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:19:54.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:53 smithi149 ceph-mon[35408]: Standby manager daemon y started 2024-05-27T07:19:54.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:53 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:19:54.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:53 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:19:54.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:53 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:19:54.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:53 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:19:54.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:53 smithi063 ceph-mon[31613]: Standby manager daemon y started 2024-05-27T07:19:54.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:53 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:19:54.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:53 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:19:54.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:53 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:19:54.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:53 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.86:0/1300787999' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:19:54.852 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:54 smithi063 ceph-mon[31613]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:54.852 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:54 smithi063 ceph-mon[31613]: mgrmap e15: x(active, since 110s), standbys: z, y 2024-05-27T07:19:54.852 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:19:54.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:54 smithi086 ceph-mon[34687]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:54.910 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:54 smithi086 ceph-mon[34687]: mgrmap e15: x(active, since 110s), standbys: z, y 2024-05-27T07:19:54.911 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:19:55.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:54 smithi149 ceph-mon[35408]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:55.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:54 smithi149 ceph-mon[35408]: mgrmap e15: x(active, since 110s), standbys: z, y 2024-05-27T07:19:55.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:19:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:56 smithi063 ceph-mon[31613]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:57.151 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:56 smithi149 ceph-mon[35408]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:57.151 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:57.151 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:57.151 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:57.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:56 smithi086 ceph-mon[34687]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:57.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:57.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:57.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:19:57.908 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:57 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:57.908 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:57 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:57.908 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:57 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:58.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:57 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:58.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:57 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:58.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:57 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:58.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:57 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:19:58.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:57 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:19:58.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:57 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:19:59.111 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:58 smithi063 ceph-mon[31613]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:58 smithi086 ceph-mon[34687]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:19:59.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:58 smithi149 ceph-mon[35408]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:00.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:59 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T07:20:00.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:19:59 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:00.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:59 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T07:20:00.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:19:59 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:00.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:59 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T07:20:00.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:19:59 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:00 smithi086 ceph-mon[34687]: Deploying daemon osd.0 on smithi063 2024-05-27T07:20:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:00 smithi086 ceph-mon[34687]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:00 smithi086 ceph-mon[34687]: overall HEALTH_OK 2024-05-27T07:20:01.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:00 smithi149 ceph-mon[35408]: Deploying daemon osd.0 on smithi063 2024-05-27T07:20:01.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:00 smithi149 ceph-mon[35408]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:01.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:00 smithi149 ceph-mon[35408]: overall HEALTH_OK 2024-05-27T07:20:01.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:00 smithi063 ceph-mon[31613]: Deploying daemon osd.0 on smithi063 2024-05-27T07:20:01.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:00 smithi063 ceph-mon[31613]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:01.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:00 smithi063 ceph-mon[31613]: overall HEALTH_OK 2024-05-27T07:20:03.099 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:02 smithi063 ceph-mon[31613]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:03.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:02 smithi086 ceph-mon[34687]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:03.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:02 smithi149 ceph-mon[35408]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:04 smithi086 ceph-mon[34687]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:05.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:04 smithi149 ceph-mon[35408]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:05.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:04 smithi063 ceph-mon[31613]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:06 smithi086 ceph-mon[34687]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:06 smithi149 ceph-mon[35408]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:06 smithi063 ceph-mon[31613]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:08 smithi086 ceph-mon[34687]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:09.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:08 smithi149 ceph-mon[35408]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:09.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:09.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:09.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:09.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:08 smithi063 ceph-mon[31613]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:09.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:09.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:09.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:11.033 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:10 smithi063 ceph-mon[31613]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:11.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:10 smithi086 ceph-mon[34687]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:10 smithi149 ceph-mon[35408]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:12.943 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:12 smithi063 ceph-mon[31613]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:12.944 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:12.944 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:12.944 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:12.944 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:12.944 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:12.944 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:12 smithi063 ceph-mon[31613]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T07:20:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:12 smithi086 ceph-mon[34687]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:12 smithi086 ceph-mon[34687]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T07:20:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:12 smithi149 ceph-mon[35408]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:12 smithi149 ceph-mon[35408]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T07:20:13.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.919+0000 7f2892000700 1 -- 172.21.15.63:0/1505820469 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2894063bf0 con 0x7f287c03e750 2024-05-27T07:20:13.911 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 0 on host 'smithi063' 2024-05-27T07:20:13.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.924+0000 7f289be00700 1 -- 172.21.15.63:0/1505820469 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f287c03e750 msgr2=0x7f287c040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:20:13.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.924+0000 7f289be00700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f287c03e750 0x7f287c040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2888000c00 tx=0x7f288801a040 comp rx=0 tx=0).stop 2024-05-27T07:20:13.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.924+0000 7f289be00700 1 -- 172.21.15.63:0/1505820469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28940776d0 msgr2=0x7f28941a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:20:13.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.924+0000 7f289be00700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28940776d0 0x7f28941a13b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f288000e9a0 tx=0x7f288000ecb0 comp rx=0 tx=0).stop 2024-05-27T07:20:13.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.925+0000 7f289be00700 1 -- 172.21.15.63:0/1505820469 shutdown_connections 2024-05-27T07:20:13.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.925+0000 7f289be00700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2894110980 0x7f28941a8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:13.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.925+0000 7f289be00700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28940760e0 0x7f28941a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:13.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.925+0000 7f289be00700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f287c03e750 0x7f287c040c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:13.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.925+0000 7f289be00700 1 --2- 172.21.15.63:0/1505820469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28940776d0 0x7f28941a13b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:13.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.925+0000 7f289be00700 1 -- 172.21.15.63:0/1505820469 >> 172.21.15.63:0/1505820469 conn(0x7f28940ff9b0 msgr2=0x7f2894103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:20:13.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.925+0000 7f289be00700 1 -- 172.21.15.63:0/1505820469 shutdown_connections 2024-05-27T07:20:13.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:20:13.925+0000 7f289be00700 1 -- 172.21.15.63:0/1505820469 wait complete. 2024-05-27T07:20:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:13 smithi086 ceph-mon[34687]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T07:20:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:13 smithi086 ceph-mon[34687]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T07:20:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:13 smithi086 ceph-mon[34687]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:20:14.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:13 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:14.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:13 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:14.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:13 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:14.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:13 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:14.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:13 smithi149 ceph-mon[35408]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T07:20:14.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:13 smithi149 ceph-mon[35408]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T07:20:14.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:13 smithi149 ceph-mon[35408]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:20:14.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:14.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:14.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:14.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:13 smithi063 ceph-mon[31613]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T07:20:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:13 smithi063 ceph-mon[31613]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T07:20:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:13 smithi063 ceph-mon[31613]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:20:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:14.728 DEBUG:teuthology.orchestra.run.smithi063:osd.0> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.0.service 2024-05-27T07:20:14.732 INFO:tasks.cephadm:Deploying osd.1 on smithi086 with /dev/vg_nvme/lv_4... 2024-05-27T07:20:14.732 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T07:20:14.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:14 smithi063 ceph-mon[31613]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:14.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:14 smithi063 ceph-mon[31613]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:20:14.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:14 smithi063 ceph-mon[31613]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T07:20:14.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:14.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:15.148 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:20:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:14 smithi086 ceph-mon[34687]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:14 smithi086 ceph-mon[34687]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:20:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:14 smithi086 ceph-mon[34687]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T07:20:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:14 smithi149 ceph-mon[35408]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:15.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:14 smithi149 ceph-mon[35408]: from='osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:20:15.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:14 smithi149 ceph-mon[35408]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T07:20:15.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:15.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:16.030 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:15 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:20:16.030 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:15 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:20:16.030 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:15 smithi063 ceph-mon[31613]: osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] boot 2024-05-27T07:20:16.031 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:15 smithi063 ceph-mon[31613]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T07:20:16.031 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:15 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:16.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:15 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:20:16.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:15 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:20:16.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:15 smithi086 ceph-mon[34687]: osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] boot 2024-05-27T07:20:16.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:15 smithi086 ceph-mon[34687]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T07:20:16.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:15 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:16.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:15 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:20:16.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:15 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:20:16.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:15 smithi149 ceph-mon[35408]: osd.0 [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] boot 2024-05-27T07:20:16.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:15 smithi149 ceph-mon[35408]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T07:20:16.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:15 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:20:17.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:16 smithi086 ceph-mon[34687]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:17.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:16 smithi086 ceph-mon[34687]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T07:20:17.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:16 smithi149 ceph-mon[35408]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:17.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:16 smithi149 ceph-mon[35408]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T07:20:17.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:16 smithi063 ceph-mon[31613]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:20:17.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:16 smithi063 ceph-mon[31613]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T07:20:17.838 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T07:20:17.839 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T07:20:17.839 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-05-27T07:20:17.839 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-05-27T07:20:17.839 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0402498 s, 261 MB/s 2024-05-27T07:20:17.839 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-05-27T07:20:18.184 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:17 smithi086 ceph-mon[34687]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:18.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:17 smithi149 ceph-mon[35408]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:18.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:17 smithi063 ceph-mon[31613]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:18.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-05-27T07:20:19.053 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:20:20.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:20 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi063 2024-05-27T07:20:20.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:20:20.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:20 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi063 to 17089M 2024-05-27T07:20:20.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:20.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:20.815 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.816 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:20 smithi086 ceph-mon[34687]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:20 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi063 2024-05-27T07:20:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:20:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:20 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi063 to 17089M 2024-05-27T07:20:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:20.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:20 smithi149 ceph-mon[35408]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:20.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:20 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi063 2024-05-27T07:20:20.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:20:20.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:20 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi063 to 17089M 2024-05-27T07:20:20.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:20.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:20.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:20.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:20 smithi063 ceph-mon[31613]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.069+0000 7ff8a4600700 1 -- 172.21.15.86:0/3345364443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff89c107ef0 msgr2=0x7ff89c108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:20:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.069+0000 7ff8a4600700 1 --2- 172.21.15.86:0/3345364443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff89c107ef0 0x7ff89c108300 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff89400b0a0 tx=0x7ff89400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:20:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.069+0000 7ff8a4600700 1 -- 172.21.15.86:0/3345364443 shutdown_connections 2024-05-27T07:20:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.069+0000 7ff8a4600700 1 --2- 172.21.15.86:0/3345364443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff89c107ef0 0x7ff89c108300 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.069+0000 7ff8a4600700 1 --2- 172.21.15.86:0/3345364443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff89c102c30 0x7ff89c105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.069+0000 7ff8a4600700 1 --2- 172.21.15.86:0/3345364443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff89c105590 0x7ff89c1079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.069+0000 7ff8a4600700 1 -- 172.21.15.86:0/3345364443 >> 172.21.15.86:0/3345364443 conn(0x7ff89c0fe880 msgr2=0x7ff89c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:20:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.069+0000 7ff8a4600700 1 -- 172.21.15.86:0/3345364443 shutdown_connections 2024-05-27T07:20:21.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.069+0000 7ff8a4600700 1 -- 172.21.15.86:0/3345364443 wait complete. 2024-05-27T07:20:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.071+0000 7ff8a4600700 1 Processor -- start 2024-05-27T07:20:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.071+0000 7ff8a4600700 1 -- start start 2024-05-27T07:20:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.071+0000 7ff8a4600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff89c102c30 0x7ff89c1a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:20:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.071+0000 7ff8a4600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff89c105590 0x7ff89c1a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:20:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.071+0000 7ff8a4600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff89c107ef0 0x7ff89c1a8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:20:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.071+0000 7ff8a4600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1162a0 con 0x7ff89c105590 2024-05-27T07:20:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.071+0000 7ff8a4600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c116120 con 0x7ff89c107ef0 2024-05-27T07:20:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.071+0000 7ff8a4600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c116420 con 0x7ff89c102c30 2024-05-27T07:20:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a2600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff89c107ef0 0x7ff89c1a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:20:21.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a2600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff89c107ef0 0x7ff89c1a8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57036/0 (socket says 172.21.15.86:57036) 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a2600700 1 -- 172.21.15.86:0/1076242195 learned_addr learned my addr 172.21.15.86:0/1076242195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a1c00700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff89c102c30 0x7ff89c1a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a1200700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff89c105590 0x7ff89c1a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a2600700 1 -- 172.21.15.86:0/1076242195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff89c102c30 msgr2=0x7ff89c1a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a2600700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff89c102c30 0x7ff89c1a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a2600700 1 -- 172.21.15.86:0/1076242195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff89c105590 msgr2=0x7ff89c1a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a2600700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff89c105590 0x7ff89c1a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a2600700 1 -- 172.21.15.86:0/1076242195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c1a8950 con 0x7ff89c107ef0 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.072+0000 7ff8a2600700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff89c107ef0 0x7ff89c1a8280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff89400b830 tx=0x7ff89400bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.073+0000 7ff89a000700 1 -- 172.21.15.86:0/1076242195 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894021070 con 0x7ff89c107ef0 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.073+0000 7ff89a000700 1 -- 172.21.15.86:0/1076242195 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff894007c80 con 0x7ff89c107ef0 2024-05-27T07:20:21.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.073+0000 7ff8a4600700 1 -- 172.21.15.86:0/1076242195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89c1a8be0 con 0x7ff89c107ef0 2024-05-27T07:20:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.073+0000 7ff89a000700 1 -- 172.21.15.86:0/1076242195 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89401b7b0 con 0x7ff89c107ef0 2024-05-27T07:20:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.073+0000 7ff8a4600700 1 -- 172.21.15.86:0/1076242195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89c1a9030 con 0x7ff89c107ef0 2024-05-27T07:20:21.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.074+0000 7ff8a4600700 1 -- 172.21.15.86:0/1076242195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89c0512b0 con 0x7ff89c107ef0 2024-05-27T07:20:21.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.077+0000 7ff89a000700 1 -- 172.21.15.86:0/1076242195 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7ff894024070 con 0x7ff89c107ef0 2024-05-27T07:20:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.079+0000 7ff89a000700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8840b27f0 0x7ff8840b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:20:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.079+0000 7ff89a000700 1 -- 172.21.15.86:0/1076242195 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff89400f020 con 0x7ff89c107ef0 2024-05-27T07:20:21.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.079+0000 7ff8a1c00700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8840b27f0 0x7ff8840b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:20:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.081+0000 7ff8a1c00700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8840b27f0 0x7ff8840b4d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff890005950 tx=0x7ff8900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:20:21.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.082+0000 7ff89a000700 1 -- 172.21.15.86:0/1076242195 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff89402b020 con 0x7ff89c107ef0 2024-05-27T07:20:21.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:21.323+0000 7ff8a4600700 1 -- 172.21.15.86:0/1076242195 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff89c063bf0 con 0x7ff8840b27f0 2024-05-27T07:20:21.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:21 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:20:21.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:21 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:20:21.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:21 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:21.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:21 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:20:21.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:21 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:20:21.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:21 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:21.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:21 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:20:21.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:21 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:20:21.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:21 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:22.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:22 smithi086 ceph-mon[34687]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:20:22.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:22 smithi086 ceph-mon[34687]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:22.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:22 smithi149 ceph-mon[35408]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:20:22.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:22 smithi149 ceph-mon[35408]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:22.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:22 smithi063 ceph-mon[31613]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:20:22.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:22 smithi063 ceph-mon[31613]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:24 smithi086 ceph-mon[34687]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:24 smithi149 ceph-mon[35408]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:24.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:24 smithi063 ceph-mon[31613]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:25.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:25 smithi086 ceph-mon[34687]: from='client.? 172.21.15.86:0/329752940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f57b3d89-2b35-4381-99a6-7db08bef64a4"}]: dispatch 2024-05-27T07:20:25.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:25 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f57b3d89-2b35-4381-99a6-7db08bef64a4"}]: dispatch 2024-05-27T07:20:25.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:25 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f57b3d89-2b35-4381-99a6-7db08bef64a4"}]': finished 2024-05-27T07:20:25.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:25 smithi086 ceph-mon[34687]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T07:20:25.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:25 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:25.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:25 smithi149 ceph-mon[35408]: from='client.? 172.21.15.86:0/329752940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f57b3d89-2b35-4381-99a6-7db08bef64a4"}]: dispatch 2024-05-27T07:20:25.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:25 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f57b3d89-2b35-4381-99a6-7db08bef64a4"}]: dispatch 2024-05-27T07:20:25.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:25 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f57b3d89-2b35-4381-99a6-7db08bef64a4"}]': finished 2024-05-27T07:20:25.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:25 smithi149 ceph-mon[35408]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T07:20:25.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:25 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:25.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:25 smithi063 ceph-mon[31613]: from='client.? 172.21.15.86:0/329752940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f57b3d89-2b35-4381-99a6-7db08bef64a4"}]: dispatch 2024-05-27T07:20:25.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:25 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f57b3d89-2b35-4381-99a6-7db08bef64a4"}]: dispatch 2024-05-27T07:20:25.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:25 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f57b3d89-2b35-4381-99a6-7db08bef64a4"}]': finished 2024-05-27T07:20:25.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:25 smithi063 ceph-mon[31613]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T07:20:25.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:25 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:26.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:26 smithi086 ceph-mon[34687]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:26.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:26 smithi086 ceph-mon[34687]: from='client.? 172.21.15.86:0/593840854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:20:26.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:26 smithi149 ceph-mon[35408]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:26.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:26 smithi149 ceph-mon[35408]: from='client.? 172.21.15.86:0/593840854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:20:26.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:26 smithi063 ceph-mon[31613]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:26.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:26 smithi063 ceph-mon[31613]: from='client.? 172.21.15.86:0/593840854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:20:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:28 smithi086 ceph-mon[34687]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:28.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:28 smithi149 ceph-mon[35408]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:28.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:28 smithi063 ceph-mon[31613]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:30.791 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:30 smithi086 ceph-mon[34687]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:30.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:30 smithi149 ceph-mon[35408]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:30.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:30 smithi063 ceph-mon[31613]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:32.899 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:32 smithi086 ceph-mon[34687]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:32.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:32 smithi149 ceph-mon[35408]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:32.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:32 smithi063 ceph-mon[31613]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:34.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:34 smithi086 ceph-mon[34687]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:34.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:34 smithi149 ceph-mon[35408]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:34.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:34 smithi063 ceph-mon[31613]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:35.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:35 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T07:20:35.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:35 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:35.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:35 smithi149 ceph-mon[35408]: Deploying daemon osd.1 on smithi086 2024-05-27T07:20:35.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:35 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T07:20:35.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:35 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:35.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:35 smithi063 ceph-mon[31613]: Deploying daemon osd.1 on smithi086 2024-05-27T07:20:35.968 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:35 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T07:20:35.968 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:35 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:35.969 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:35 smithi086 ceph-mon[34687]: Deploying daemon osd.1 on smithi086 2024-05-27T07:20:36.813 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:36 smithi086 ceph-mon[34687]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:36.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:36 smithi149 ceph-mon[35408]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:36.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:36 smithi063 ceph-mon[31613]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:38.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:38 smithi086 ceph-mon[34687]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:38 smithi149 ceph-mon[35408]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:38 smithi063 ceph-mon[31613]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:41.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:40 smithi149 ceph-mon[35408]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:41.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:40 smithi086 ceph-mon[34687]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:41.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:40 smithi063 ceph-mon[31613]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:42.832 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:42 smithi086 ceph-mon[34687]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:43.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:42 smithi149 ceph-mon[35408]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:43.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:42 smithi063 ceph-mon[31613]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:44.904 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:44 smithi086 ceph-mon[34687]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:44.905 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:44.905 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:44.905 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:44.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:44 smithi149 ceph-mon[35408]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:44.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:44.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:44.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:44.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:44 smithi063 ceph-mon[31613]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:44.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:44.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:44.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:46.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:46 smithi086 ceph-mon[34687]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:46.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:46 smithi149 ceph-mon[35408]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:46.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:46 smithi063 ceph-mon[31613]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:48.388 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:48 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:48.388 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:48 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:48.388 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:48 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:48.388 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:48 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:48.389 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:48 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:48.389 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:48 smithi086 ceph-mon[34687]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:48.389 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:48 smithi086 ceph-mon[34687]: from='osd.1 [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:20:48.389 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:48 smithi086 ceph-mon[34687]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:20:48.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:48 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:48.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:48 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:48.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:48 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:48.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:48 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:48.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:48 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:48.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:48 smithi149 ceph-mon[35408]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:48.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:48 smithi149 ceph-mon[35408]: from='osd.1 [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:20:48.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:48 smithi149 ceph-mon[35408]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:20:48.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:48.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:48.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:48.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:48.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:48.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:48 smithi063 ceph-mon[31613]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:48.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:48 smithi063 ceph-mon[31613]: from='osd.1 [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:20:48.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:48 smithi063 ceph-mon[31613]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:20:48.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.837+0000 7ff89a000700 1 -- 172.21.15.86:0/1076242195 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff89c063bf0 con 0x7ff8840b27f0 2024-05-27T07:20:48.828 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 1 on host 'smithi086' 2024-05-27T07:20:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 -- 172.21.15.86:0/1076242195 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8840b27f0 msgr2=0x7ff8840b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:20:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8840b27f0 0x7ff8840b4d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff890005950 tx=0x7ff8900058e0 comp rx=0 tx=0).stop 2024-05-27T07:20:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 -- 172.21.15.86:0/1076242195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff89c107ef0 msgr2=0x7ff89c1a8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:20:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff89c107ef0 0x7ff89c1a8280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff89400b830 tx=0x7ff89400bd70 comp rx=0 tx=0).stop 2024-05-27T07:20:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 -- 172.21.15.86:0/1076242195 shutdown_connections 2024-05-27T07:20:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff89c107ef0 0x7ff89c1a8280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff89c102c30 0x7ff89c1a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8840b27f0 0x7ff8840b4d00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 --2- 172.21.15.86:0/1076242195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff89c105590 0x7ff89c1a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 -- 172.21.15.86:0/1076242195 >> 172.21.15.86:0/1076242195 conn(0x7ff89c0fe880 msgr2=0x7ff89c103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:20:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.841+0000 7ff8a4600700 1 -- 172.21.15.86:0/1076242195 shutdown_connections 2024-05-27T07:20:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:20:48.842+0000 7ff8a4600700 1 -- 172.21.15.86:0/1076242195 wait complete. 2024-05-27T07:20:49.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:49 smithi086 ceph-mon[34687]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T07:20:49.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:49 smithi086 ceph-mon[34687]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T07:20:49.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:49.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:49 smithi086 ceph-mon[34687]: from='osd.1 [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:20:49.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:49 smithi086 ceph-mon[34687]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:20:49.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:49.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:49.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:49.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:49 smithi149 ceph-mon[35408]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T07:20:49.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:49 smithi149 ceph-mon[35408]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T07:20:49.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:49.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:49 smithi149 ceph-mon[35408]: from='osd.1 [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:20:49.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:49 smithi149 ceph-mon[35408]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:20:49.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:49.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:49.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:49.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:49 smithi063 ceph-mon[31613]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T07:20:49.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:49 smithi063 ceph-mon[31613]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T07:20:49.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:49.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:49 smithi063 ceph-mon[31613]: from='osd.1 [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:20:49.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:49 smithi063 ceph-mon[31613]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:20:49.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:49.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:49.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:20:49.633 DEBUG:teuthology.orchestra.run.smithi086:osd.1> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.1.service 2024-05-27T07:20:49.637 INFO:tasks.cephadm:Deploying osd.2 on smithi149 with /dev/vg_nvme/lv_4... 2024-05-27T07:20:49.637 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T07:20:50.071 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:20:50.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:50 smithi149 ceph-mon[35408]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:20:50.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:50 smithi149 ceph-mon[35408]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T07:20:50.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:50 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:50.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:50 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:50.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:50 smithi149 ceph-mon[35408]: pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:50.434 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:50 smithi149 ceph-mon[35408]: from='osd.1 ' entity='osd.1' 2024-05-27T07:20:50.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:50 smithi086 ceph-mon[34687]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:20:50.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:50 smithi086 ceph-mon[34687]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T07:20:50.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:50 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:50.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:50 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:50.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:50 smithi086 ceph-mon[34687]: pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:50.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:50 smithi086 ceph-mon[34687]: from='osd.1 ' entity='osd.1' 2024-05-27T07:20:50.449 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 07:20:50 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1[39249]: 2024-05-27T07:20:50.026+0000 7f6f63400700 -1 osd.1 0 waiting for initial osdmap 2024-05-27T07:20:50.450 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 07:20:50 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1[39249]: 2024-05-27T07:20:50.043+0000 7f6f58800700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T07:20:50.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:50 smithi063 ceph-mon[31613]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:20:50.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:50 smithi063 ceph-mon[31613]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T07:20:50.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:50 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:50.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:50 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:50.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:50 smithi063 ceph-mon[31613]: pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:20:50.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:50 smithi063 ceph-mon[31613]: from='osd.1 ' entity='osd.1' 2024-05-27T07:20:51.276 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:51 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:20:51.277 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:51 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:20:51.277 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:51.277 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:51 smithi149 ceph-mon[35408]: osd.1 [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] boot 2024-05-27T07:20:51.277 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:51 smithi149 ceph-mon[35408]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T07:20:51.277 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:51.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:51 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:20:51.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:51 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:20:51.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:51.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:51 smithi086 ceph-mon[34687]: osd.1 [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] boot 2024-05-27T07:20:51.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:51 smithi086 ceph-mon[34687]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T07:20:51.408 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:51.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:51 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:20:51.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:51 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:20:51.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:51.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:51 smithi063 ceph-mon[31613]: osd.1 [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] boot 2024-05-27T07:20:51.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:51 smithi063 ceph-mon[31613]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T07:20:51.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:20:52.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:52 smithi086 ceph-mon[34687]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:52.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:52 smithi149 ceph-mon[35408]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:52.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:52 smithi063 ceph-mon[31613]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:52.819 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T07:20:52.819 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T07:20:52.819 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-27T07:20:52.819 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-27T07:20:52.820 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0342328 s, 306 MB/s 2024-05-27T07:20:52.820 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-27T07:20:53.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:53 smithi149 ceph-mon[35408]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T07:20:53.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:53 smithi063 ceph-mon[31613]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T07:20:53.529 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-05-27T07:20:53.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:53 smithi086 ceph-mon[34687]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T07:20:53.925 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:20:54.343 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:54 smithi149 ceph-mon[35408]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:54.600 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:54 smithi086 ceph-mon[34687]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:54.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:54 smithi063 ceph-mon[31613]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:56.004 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.013+0000 7f61a0c00700 1 -- 172.21.15.149:0/2775908007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f619c109fe0 msgr2=0x7f619c10c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:20:56.004 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.013+0000 7f61a0c00700 1 --2- 172.21.15.149:0/2775908007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f619c109fe0 0x7f619c10c4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f619400b0a0 tx=0x7f619400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:20:56.004 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.013+0000 7f61a0c00700 1 -- 172.21.15.149:0/2775908007 shutdown_connections 2024-05-27T07:20:56.004 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.013+0000 7f61a0c00700 1 --2- 172.21.15.149:0/2775908007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f619c109fe0 0x7f619c10c4f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:56.004 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.013+0000 7f61a0c00700 1 --2- 172.21.15.149:0/2775908007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c06c2c0 0x7f619c106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:56.004 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.013+0000 7f61a0c00700 1 --2- 172.21.15.149:0/2775908007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c107470 0x7f619c109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:56.004 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.013+0000 7f61a0c00700 1 -- 172.21.15.149:0/2775908007 >> 172.21.15.149:0/2775908007 conn(0x7f619c0fe870 msgr2=0x7f619c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:20:56.004 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.013+0000 7f61a0c00700 1 -- 172.21.15.149:0/2775908007 shutdown_connections 2024-05-27T07:20:56.004 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.013+0000 7f61a0c00700 1 -- 172.21.15.149:0/2775908007 wait complete. 2024-05-27T07:20:56.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.014+0000 7f61a0c00700 1 Processor -- start 2024-05-27T07:20:56.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.014+0000 7f61a0c00700 1 -- start start 2024-05-27T07:20:56.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.015+0000 7f61a0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c06c2c0 0x7f619c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:20:56.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.015+0000 7f61a0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c107470 0x7f619c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:20:56.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.015+0000 7f61a0c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f619c109fe0 0x7f619c1a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:20:56.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.015+0000 7f61a0c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f619c10fcd0 con 0x7f619c107470 2024-05-27T07:20:56.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.015+0000 7f61a0c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f619c10fb50 con 0x7f619c06c2c0 2024-05-27T07:20:56.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.015+0000 7f61a0c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f619c10fe50 con 0x7f619c109fe0 2024-05-27T07:20:56.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.015+0000 7f619a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f619c109fe0 0x7f619c1a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:20:56.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.015+0000 7f619a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f619c109fe0 0x7f619c1a3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:34542/0 (socket says 172.21.15.149:34542) 2024-05-27T07:20:56.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.015+0000 7f619a800700 1 -- 172.21.15.149:0/3080636249 learned_addr learned my addr 172.21.15.149:0/3080636249 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:20:56.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.015+0000 7f6199400700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c107470 0x7f619c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:20:56.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.016+0000 7f6199e00700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c06c2c0 0x7f619c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:20:56.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.016+0000 7f619a800700 1 -- 172.21.15.149:0/3080636249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c06c2c0 msgr2=0x7f619c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:20:56.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.016+0000 7f619a800700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c06c2c0 0x7f619c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:56.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.016+0000 7f619a800700 1 -- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c107470 msgr2=0x7f619c19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:20:56.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.016+0000 7f619a800700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c107470 0x7f619c19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:20:56.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.016+0000 7f619a800700 1 -- 172.21.15.149:0/3080636249 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c1a4500 con 0x7f619c109fe0 2024-05-27T07:20:56.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.016+0000 7f6199400700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c107470 0x7f619c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:20:56.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.016+0000 7f619a800700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f619c109fe0 0x7f619c1a3f10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f61940062a0 tx=0x7f619400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:20:56.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.017+0000 7f6192a00700 1 -- 172.21.15.149:0/3080636249 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619401f030 con 0x7f619c109fe0 2024-05-27T07:20:56.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.017+0000 7f6192a00700 1 -- 172.21.15.149:0/3080636249 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6194007ac0 con 0x7f619c109fe0 2024-05-27T07:20:56.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.017+0000 7f6192a00700 1 -- 172.21.15.149:0/3080636249 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194004840 con 0x7f619c109fe0 2024-05-27T07:20:56.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.017+0000 7f61a0c00700 1 -- 172.21.15.149:0/3080636249 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619c1a4790 con 0x7f619c109fe0 2024-05-27T07:20:56.008 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.017+0000 7f61a0c00700 1 -- 172.21.15.149:0/3080636249 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619c1a4d10 con 0x7f619c109fe0 2024-05-27T07:20:56.009 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.018+0000 7f61a0c00700 1 -- 172.21.15.149:0/3080636249 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619c0512b0 con 0x7f619c109fe0 2024-05-27T07:20:56.012 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.023+0000 7f6192a00700 1 -- 172.21.15.149:0/3080636249 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f6194003a00 con 0x7f619c109fe0 2024-05-27T07:20:56.015 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.025+0000 7f6192a00700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f61800b2870 0x7f61800b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:20:56.015 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.025+0000 7f6192a00700 1 -- 172.21.15.149:0/3080636249 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f619400f020 con 0x7f619c109fe0 2024-05-27T07:20:56.015 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.026+0000 7f6199e00700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f61800b2870 0x7f61800b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:20:56.016 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.026+0000 7f6192a00700 1 -- 172.21.15.149:0/3080636249 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6194029020 con 0x7f619c109fe0 2024-05-27T07:20:56.016 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.027+0000 7f6199e00700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f61800b2870 0x7f61800b4d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f618c006fd0 tx=0x7f618c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:20:56.255 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:20:56.265+0000 7f61a0c00700 1 -- 172.21.15.149:0/3080636249 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f619c10aab0 con 0x7f61800b2870 2024-05-27T07:20:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi086 2024-05-27T07:20:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:20:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi086 to 17089M 2024-05-27T07:20:56.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:56.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:56.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:20:56.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:20:56.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:56.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi086 2024-05-27T07:20:56.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:20:56.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:56.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi086 to 17089M 2024-05-27T07:20:56.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:56.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:56.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:20:56.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:20:56.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi086 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi086 to 17089M 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:20:56.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:20:56.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:20:56.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:20:57.866 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:57 smithi149 ceph-mon[35408]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:20:57.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:57 smithi086 ceph-mon[34687]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:20:57.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:57 smithi063 ceph-mon[31613]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:20:58.811 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:20:58 smithi149 ceph-mon[35408]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:58.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:20:58 smithi086 ceph-mon[34687]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:20:58.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:20:58 smithi063 ceph-mon[31613]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:00.887 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:00 smithi149 ceph-mon[35408]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:00.888 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:00 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/1680837458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "025e966c-8833-4c12-a136-64d91cb0f4a4"}]: dispatch 2024-05-27T07:21:00.888 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:00 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "025e966c-8833-4c12-a136-64d91cb0f4a4"}]: dispatch 2024-05-27T07:21:00.888 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:00 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "025e966c-8833-4c12-a136-64d91cb0f4a4"}]': finished 2024-05-27T07:21:00.888 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:00 smithi149 ceph-mon[35408]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T07:21:00.888 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:00.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:00 smithi086 ceph-mon[34687]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:00.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:00 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/1680837458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "025e966c-8833-4c12-a136-64d91cb0f4a4"}]: dispatch 2024-05-27T07:21:00.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:00 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "025e966c-8833-4c12-a136-64d91cb0f4a4"}]: dispatch 2024-05-27T07:21:00.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:00 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "025e966c-8833-4c12-a136-64d91cb0f4a4"}]': finished 2024-05-27T07:21:00.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:00 smithi086 ceph-mon[34687]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T07:21:00.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:00.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:00 smithi063 ceph-mon[31613]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:00.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:00 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/1680837458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "025e966c-8833-4c12-a136-64d91cb0f4a4"}]: dispatch 2024-05-27T07:21:00.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:00 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "025e966c-8833-4c12-a136-64d91cb0f4a4"}]: dispatch 2024-05-27T07:21:00.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:00 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "025e966c-8833-4c12-a136-64d91cb0f4a4"}]': finished 2024-05-27T07:21:00.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:00 smithi063 ceph-mon[31613]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T07:21:00.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:01.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:01 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/2772708275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:21:01.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:01 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/2772708275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:21:01.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:01 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/2772708275' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:21:02.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:02 smithi086 ceph-mon[34687]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:02.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:02 smithi149 ceph-mon[35408]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:02.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:02 smithi063 ceph-mon[31613]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:04.789 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:04 smithi149 ceph-mon[35408]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:04.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:04 smithi086 ceph-mon[34687]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:04.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:04 smithi063 ceph-mon[31613]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:06.783 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:06 smithi149 ceph-mon[35408]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:06.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:06 smithi086 ceph-mon[34687]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:06.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:06 smithi063 ceph-mon[31613]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:08.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:08 smithi086 ceph-mon[34687]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:08.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:08 smithi149 ceph-mon[35408]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:08.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:08 smithi063 ceph-mon[31613]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:09.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:09 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T07:21:09.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:09 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:09.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:09 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T07:21:09.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:09 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:09.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:09 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T07:21:09.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:09 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:10 smithi086 ceph-mon[34687]: Deploying daemon osd.2 on smithi149 2024-05-27T07:21:10.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:10 smithi086 ceph-mon[34687]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:10.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:10 smithi149 ceph-mon[35408]: Deploying daemon osd.2 on smithi149 2024-05-27T07:21:10.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:10 smithi149 ceph-mon[35408]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:10.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:10 smithi063 ceph-mon[31613]: Deploying daemon osd.2 on smithi149 2024-05-27T07:21:10.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:10 smithi063 ceph-mon[31613]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:12.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:12 smithi086 ceph-mon[34687]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:12.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:12 smithi149 ceph-mon[35408]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:12.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:12 smithi063 ceph-mon[31613]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:14.919 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:14 smithi149 ceph-mon[35408]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:14.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:14 smithi086 ceph-mon[34687]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:14.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:14 smithi063 ceph-mon[31613]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:16.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:16 smithi086 ceph-mon[34687]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:16.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:16 smithi149 ceph-mon[35408]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:16.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:16 smithi063 ceph-mon[31613]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:18 smithi086 ceph-mon[34687]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:18.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:21:18.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:18 smithi149 ceph-mon[35408]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:18.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:18.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:18.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:21:18.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:18 smithi063 ceph-mon[31613]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:18.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:18.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:18.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:21:20.870 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:20 smithi149 ceph-mon[35408]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:20.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:20 smithi063 ceph-mon[31613]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:21.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:20 smithi086 ceph-mon[34687]: pgmap v103: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:22.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:22 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:22.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:22 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:22.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:22 smithi149 ceph-mon[35408]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:22.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:22 smithi149 ceph-mon[35408]: from='osd.2 [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:21:22.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:22 smithi149 ceph-mon[35408]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:21:22.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:22 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:22.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:22 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:22.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:22 smithi086 ceph-mon[34687]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:22.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:22 smithi086 ceph-mon[34687]: from='osd.2 [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:21:22.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:22 smithi086 ceph-mon[34687]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:21:22.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:22 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:22.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:22 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:22.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:22 smithi063 ceph-mon[31613]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:22.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:22 smithi063 ceph-mon[31613]: from='osd.2 [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:21:22.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:22 smithi063 ceph-mon[31613]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:21:23.277 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 2 on host 'smithi149' 2024-05-27T07:21:23.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.286+0000 7f6192a00700 1 -- 172.21.15.149:0/3080636249 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f619c10aab0 con 0x7f61800b2870 2024-05-27T07:21:23.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.290+0000 7f61a0c00700 1 -- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f61800b2870 msgr2=0x7f61800b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:21:23.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.290+0000 7f61a0c00700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f61800b2870 0x7f61800b4d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f618c006fd0 tx=0x7f618c008040 comp rx=0 tx=0).stop 2024-05-27T07:21:23.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.291+0000 7f61a0c00700 1 -- 172.21.15.149:0/3080636249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f619c109fe0 msgr2=0x7f619c1a3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:21:23.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.291+0000 7f61a0c00700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f619c109fe0 0x7f619c1a3f10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f61940062a0 tx=0x7f619400bd70 comp rx=0 tx=0).stop 2024-05-27T07:21:23.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.291+0000 7f61a0c00700 1 -- 172.21.15.149:0/3080636249 shutdown_connections 2024-05-27T07:21:23.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.291+0000 7f61a0c00700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f619c109fe0 0x7f619c1a3f10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:21:23.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.291+0000 7f61a0c00700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c06c2c0 0x7f619c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:21:23.281 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.291+0000 7f61a0c00700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f61800b2870 0x7f61800b4d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:21:23.282 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.291+0000 7f61a0c00700 1 --2- 172.21.15.149:0/3080636249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619c107470 0x7f619c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:21:23.282 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.291+0000 7f61a0c00700 1 -- 172.21.15.149:0/3080636249 >> 172.21.15.149:0/3080636249 conn(0x7f619c0fe870 msgr2=0x7f619c100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:21:23.282 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.291+0000 7f61a0c00700 1 -- 172.21.15.149:0/3080636249 shutdown_connections 2024-05-27T07:21:23.282 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:21:23.291+0000 7f61a0c00700 1 -- 172.21.15.149:0/3080636249 wait complete. 2024-05-27T07:21:23.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T07:21:23.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: from='osd.2 [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:21:23.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T07:21:23.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:23.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:21:23.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:23.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:21:23.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:23.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:23.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:23.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:21:23.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T07:21:23.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: from='osd.2 [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:21:23.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T07:21:23.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:23.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:21:23.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:23.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:21:23.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:23.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:23.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:23.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: from='osd.2 [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:23.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:21:24.051 DEBUG:teuthology.orchestra.run.smithi149:osd.2> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.2.service 2024-05-27T07:21:24.055 INFO:tasks.cephadm:Deploying osd.3 on smithi063 with /dev/vg_nvme/lv_3... 2024-05-27T07:21:24.055 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T07:21:24.522 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:21:24.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:24 smithi063 ceph-mon[31613]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:21:24.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:24 smithi063 ceph-mon[31613]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T07:21:24.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:24.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:24.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:24 smithi063 ceph-mon[31613]: pgmap v107: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:24 smithi086 ceph-mon[34687]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:21:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:24 smithi086 ceph-mon[34687]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T07:21:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:24 smithi086 ceph-mon[34687]: pgmap v107: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:24.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:24 smithi149 ceph-mon[35408]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:21:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:24 smithi149 ceph-mon[35408]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T07:21:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:24 smithi149 ceph-mon[35408]: pgmap v107: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:21:25.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:25 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:21:25.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:25 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:21:25.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:25 smithi063 ceph-mon[31613]: osd.2 [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] boot 2024-05-27T07:21:25.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:25 smithi063 ceph-mon[31613]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T07:21:25.826 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:25 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:25.941 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:25 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:21:25.941 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:25 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:21:25.941 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:25 smithi149 ceph-mon[35408]: osd.2 [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] boot 2024-05-27T07:21:25.941 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:25 smithi149 ceph-mon[35408]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T07:21:25.941 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:25 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:25.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:25 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:21:25.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:25 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:21:25.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:25 smithi086 ceph-mon[34687]: osd.2 [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] boot 2024-05-27T07:21:25.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:25 smithi086 ceph-mon[34687]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T07:21:25.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:25 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:21:26.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:26 smithi086 ceph-mon[34687]: pgmap v109: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:26.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:26 smithi086 ceph-mon[34687]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T07:21:26.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:26 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:21:26.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:26 smithi149 ceph-mon[35408]: pgmap v109: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:26.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:26 smithi149 ceph-mon[35408]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T07:21:26.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:26 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:21:26.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:26 smithi063 ceph-mon[31613]: pgmap v109: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:26.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:26 smithi063 ceph-mon[31613]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T07:21:26.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:26 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:21:27.404 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T07:21:27.404 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T07:21:27.404 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-27T07:21:27.404 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-27T07:21:27.404 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0433111 s, 242 MB/s 2024-05-27T07:21:27.404 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-27T07:21:27.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:27 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T07:21:27.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:27 smithi086 ceph-mon[34687]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T07:21:27.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:27 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:21:27.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:27 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T07:21:27.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:27 smithi149 ceph-mon[35408]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T07:21:27.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:27 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:21:27.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:27 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T07:21:27.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:27 smithi063 ceph-mon[31613]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T07:21:27.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:27 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:21:27.956 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 07:21:27 smithi063 sudo[50602]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:21:27.956 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 07:21:27 smithi063 sudo[50602]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:27.956 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 07:21:27 smithi063 sudo[50602]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:27.956 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 07:21:27 smithi063 sudo[50602]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:27.956 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 07:21:27 smithi063 sudo[50605]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:21:27.956 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 07:21:27 smithi063 sudo[50605]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:27.956 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 07:21:27 smithi063 sudo[50605]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:27.957 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 07:21:27 smithi063 sudo[50605]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.346 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_3 2024-05-27T07:21:28.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 sudo[50619]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:21:28.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 sudo[50619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:28.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 sudo[50619]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:28.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 sudo[50619]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.449 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43397]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:21:28.449 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43397]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:28.449 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43397]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:28.449 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43397]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.449 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43400]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:21:28.449 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43400]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:28.449 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43400]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:28.449 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43400]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.453 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44620]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:21:28.453 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44620]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:28.453 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44620]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:28.453 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44620]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.453 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44623]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:21:28.454 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44623]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:28.454 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44623]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:28.454 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44623]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 sudo[50636]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:21:28.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 sudo[50636]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:28.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 sudo[50636]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:28.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 sudo[50636]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:28.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:21:28.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T07:21:28.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T07:21:28.780 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:21:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43403]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:21:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43403]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43403]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43403]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43406]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:21:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43406]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43406]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 sudo[43406]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:21:28.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:21:28.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:21:28.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:21:28.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:21:28.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:21:28.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:21:28.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:21:28.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:21:28.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:21:28.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44627]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:21:28.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44627]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44627]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44627]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44630]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44630]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44630]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 sudo[44630]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:28.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:21:28.955 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:21:28.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:28.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:28.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:28.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:21:28.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:21:29.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:21:29.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi149 2024-05-27T07:21:29.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:21:29.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T07:21:29.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:21:29.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi149 to 5817M 2024-05-27T07:21:29.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:29.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:21:29.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:21:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi149 2024-05-27T07:21:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:21:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T07:21:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:21:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi149 to 5817M 2024-05-27T07:21:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:29.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:21:29.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:21:29.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi149 2024-05-27T07:21:29.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:21:29.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T07:21:29.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:21:29.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi149 to 5817M 2024-05-27T07:21:29.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:29.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:29.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:21:29.959 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:30.739 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:30 smithi063 ceph-mon[31613]: pgmap v115: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:30.739 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:30 smithi063 ceph-mon[31613]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T07:21:30.739 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:30 smithi063 ceph-mon[31613]: Cluster is now healthy 2024-05-27T07:21:30.739 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:30 smithi063 ceph-mon[31613]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T07:21:30.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:30 smithi086 ceph-mon[34687]: pgmap v115: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:30.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:30 smithi086 ceph-mon[34687]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T07:21:30.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:30 smithi086 ceph-mon[34687]: Cluster is now healthy 2024-05-27T07:21:30.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:30 smithi086 ceph-mon[34687]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T07:21:30.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:30 smithi149 ceph-mon[35408]: pgmap v115: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:30.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:30 smithi149 ceph-mon[35408]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T07:21:30.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:30 smithi149 ceph-mon[35408]: Cluster is now healthy 2024-05-27T07:21:30.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:30 smithi149 ceph-mon[35408]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T07:21:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.137+0000 7f786e000700 1 -- 172.21.15.63:0/2047501509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7868108010 msgr2=0x7f786810e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:21:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.137+0000 7f786e000700 1 --2- 172.21.15.63:0/2047501509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7868108010 0x7f786810e8c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f786000b0a0 tx=0x7f786000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:21:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.137+0000 7f786e000700 1 -- 172.21.15.63:0/2047501509 shutdown_connections 2024-05-27T07:21:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.137+0000 7f786e000700 1 --2- 172.21.15.63:0/2047501509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7868108010 0x7f786810e8c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:21:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.137+0000 7f786e000700 1 --2- 172.21.15.63:0/2047501509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7868106540 0x7f7868106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:21:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.137+0000 7f786e000700 1 --2- 172.21.15.63:0/2047501509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7868107670 0x7f7868107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:21:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.137+0000 7f786e000700 1 -- 172.21.15.63:0/2047501509 >> 172.21.15.63:0/2047501509 conn(0x7f7868101b80 msgr2=0x7f7868104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:21:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.138+0000 7f786e000700 1 -- 172.21.15.63:0/2047501509 shutdown_connections 2024-05-27T07:21:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.138+0000 7f786e000700 1 -- 172.21.15.63:0/2047501509 wait complete. 2024-05-27T07:21:31.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.139+0000 7f786e000700 1 Processor -- start 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.139+0000 7f786e000700 1 -- start start 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.139+0000 7f786e000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7868106540 0x7f786819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.139+0000 7f786e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7868107670 0x7f786819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.139+0000 7f786e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7868108010 0x7f78681a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.139+0000 7f786e000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7868111e50 con 0x7f7868107670 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.139+0000 7f786e000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7868111cd0 con 0x7f7868108010 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.139+0000 7f786e000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7868111fd0 con 0x7f7868106540 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f785ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7868107670 0x7f786819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f785ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7868107670 0x7f786819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41992/0 (socket says 172.21.15.63:41992) 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f785ea00700 1 -- 172.21.15.63:0/3917626967 learned_addr learned my addr 172.21.15.63:0/3917626967 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f7867400700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7868106540 0x7f786819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:21:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f7867e00700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7868108010 0x7f78681a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f785ea00700 1 -- 172.21.15.63:0/3917626967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7868106540 msgr2=0x7f786819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f785ea00700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7868106540 0x7f786819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f785ea00700 1 -- 172.21.15.63:0/3917626967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7868108010 msgr2=0x7f78681a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f785ea00700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7868108010 0x7f78681a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f785ea00700 1 -- 172.21.15.63:0/3917626967 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78681a43d0 con 0x7f7868107670 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.140+0000 7f7867e00700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7868108010 0x7f78681a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.141+0000 7f785ea00700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7868107670 0x7f786819ce40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f785000c9b0 tx=0x7f785000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.141+0000 7f7864c00700 1 -- 172.21.15.63:0/3917626967 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850007b00 con 0x7f7868107670 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.141+0000 7f7864c00700 1 -- 172.21.15.63:0/3917626967 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7850007ca0 con 0x7f7868107670 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.141+0000 7f786e000700 1 -- 172.21.15.63:0/3917626967 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78681a46c0 con 0x7f7868107670 2024-05-27T07:21:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.141+0000 7f7864c00700 1 -- 172.21.15.63:0/3917626967 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850018660 con 0x7f7868107670 2024-05-27T07:21:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.141+0000 7f786e000700 1 -- 172.21.15.63:0/3917626967 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78681a4c00 con 0x7f7868107670 2024-05-27T07:21:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.143+0000 7f786e000700 1 -- 172.21.15.63:0/3917626967 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78680512b0 con 0x7f7868107670 2024-05-27T07:21:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.145+0000 7f7864c00700 1 -- 172.21.15.63:0/3917626967 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f785000f450 con 0x7f7868107670 2024-05-27T07:21:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.148+0000 7f7864c00700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f784c0b2880 0x7f784c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:21:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.148+0000 7f7864c00700 1 -- 172.21.15.63:0/3917626967 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7f78500e1270 con 0x7f7868107670 2024-05-27T07:21:31.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.148+0000 7f7867400700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f784c0b2880 0x7f784c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:21:31.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.148+0000 7f7867400700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f784c0b2880 0x7f784c0b4d90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7858005fd0 tx=0x7f7858005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:21:31.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.150+0000 7f7864c00700 1 -- 172.21.15.63:0/3917626967 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7850014030 con 0x7f7868107670 2024-05-27T07:21:31.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:21:31.402+0000 7f786e000700 1 -- 172.21.15.63:0/3917626967 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7868063bf0 con 0x7f784c0b2880 2024-05-27T07:21:32.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:31 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:21:32.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:31 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:21:32.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:31 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:32.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:31 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:21:32.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:31 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:21:32.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:31 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:32.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:31 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:21:32.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:31 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:21:32.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:31 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:33.177 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:32 smithi063 ceph-mon[31613]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:21:33.177 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:32 smithi063 ceph-mon[31613]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:33.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:32 smithi086 ceph-mon[34687]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:21:33.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:32 smithi086 ceph-mon[34687]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:33.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:32 smithi149 ceph-mon[35408]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:21:33.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:32 smithi149 ceph-mon[35408]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:35.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:34 smithi086 ceph-mon[34687]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:34 smithi149 ceph-mon[35408]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:35.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:34 smithi063 ceph-mon[31613]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:36 smithi086 ceph-mon[34687]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:36 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3027650446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e39fff31-e8c9-47f1-9029-e8763af5abe5"}]: dispatch 2024-05-27T07:21:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:36 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3027650446' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e39fff31-e8c9-47f1-9029-e8763af5abe5"}]': finished 2024-05-27T07:21:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:36 smithi086 ceph-mon[34687]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T07:21:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:21:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:36 smithi149 ceph-mon[35408]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:36 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3027650446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e39fff31-e8c9-47f1-9029-e8763af5abe5"}]: dispatch 2024-05-27T07:21:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:36 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3027650446' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e39fff31-e8c9-47f1-9029-e8763af5abe5"}]': finished 2024-05-27T07:21:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:36 smithi149 ceph-mon[35408]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T07:21:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:21:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:36 smithi063 ceph-mon[31613]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:36 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3027650446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e39fff31-e8c9-47f1-9029-e8763af5abe5"}]: dispatch 2024-05-27T07:21:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:36 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3027650446' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e39fff31-e8c9-47f1-9029-e8763af5abe5"}]': finished 2024-05-27T07:21:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:36 smithi063 ceph-mon[31613]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T07:21:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:21:38.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:37 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2397263209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:21:38.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:37 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2397263209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:21:38.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:37 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2397263209' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:21:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:38 smithi086 ceph-mon[34687]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:38 smithi149 ceph-mon[35408]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:38 smithi063 ceph-mon[31613]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:40.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:40 smithi063 ceph-mon[31613]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:41.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:40 smithi086 ceph-mon[34687]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:41.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:40 smithi149 ceph-mon[35408]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:42.911 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:42 smithi063 ceph-mon[31613]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:42 smithi086 ceph-mon[34687]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:43.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:42 smithi149 ceph-mon[35408]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:45.038 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:44 smithi063 ceph-mon[31613]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:44 smithi086 ceph-mon[34687]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:44 smithi149 ceph-mon[35408]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:47.149 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:46 smithi063 ceph-mon[31613]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:47.149 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:46 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T07:21:47.149 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:46 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:47.149 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:46 smithi063 ceph-mon[31613]: Deploying daemon osd.3 on smithi063 2024-05-27T07:21:47.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:46 smithi086 ceph-mon[34687]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:46 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T07:21:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:46 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:46 smithi086 ceph-mon[34687]: Deploying daemon osd.3 on smithi063 2024-05-27T07:21:47.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:46 smithi149 ceph-mon[35408]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:47.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:46 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T07:21:47.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:46 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:21:47.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:46 smithi149 ceph-mon[35408]: Deploying daemon osd.3 on smithi063 2024-05-27T07:21:48.872 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:48 smithi063 ceph-mon[31613]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:49.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:48 smithi086 ceph-mon[34687]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:49.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:48 smithi149 ceph-mon[35408]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:51.138 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:50 smithi063 ceph-mon[31613]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:50 smithi086 ceph-mon[34687]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:51.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:50 smithi149 ceph-mon[35408]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:53.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:52 smithi086 ceph-mon[34687]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:53.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:52 smithi149 ceph-mon[35408]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:53.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:52 smithi063 ceph-mon[31613]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:55.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:54 smithi063 ceph-mon[31613]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:55.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:54 smithi086 ceph-mon[34687]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:55.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:54 smithi149 ceph-mon[35408]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:56.550 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:21:56.550 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:56.550 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:56 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:56.550 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:56 smithi063 ceph-mon[31613]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:56.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:21:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:56 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:56.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:56 smithi086 ceph-mon[34687]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:56.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:21:56.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:56.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:56 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:56.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:56 smithi149 ceph-mon[35408]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:58.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:58 smithi063 ceph-mon[31613]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:58.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:58 smithi086 ceph-mon[34687]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:58.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:58 smithi149 ceph-mon[35408]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:21:59.954 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:59 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:59.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:59 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:21:59.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:21:59 smithi063 ceph-mon[31613]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T07:22:00.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:59 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:00.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:59 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:00.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:21:59 smithi086 ceph-mon[34687]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T07:22:00.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:59 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:00.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:59 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:00.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:21:59 smithi149 ceph-mon[35408]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T07:22:01.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.064+0000 7f7864c00700 1 -- 172.21.15.63:0/3917626967 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7868063bf0 con 0x7f784c0b2880 2024-05-27T07:22:01.055 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 3 on host 'smithi063' 2024-05-27T07:22:01.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.067+0000 7f786e000700 1 -- 172.21.15.63:0/3917626967 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f784c0b2880 msgr2=0x7f784c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:22:01.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.067+0000 7f786e000700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f784c0b2880 0x7f784c0b4d90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7858005fd0 tx=0x7f7858005e50 comp rx=0 tx=0).stop 2024-05-27T07:22:01.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.067+0000 7f786e000700 1 -- 172.21.15.63:0/3917626967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7868107670 msgr2=0x7f786819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:22:01.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.067+0000 7f786e000700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7868107670 0x7f786819ce40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f785000c9b0 tx=0x7f785000cd70 comp rx=0 tx=0).stop 2024-05-27T07:22:01.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.067+0000 7f786e000700 1 -- 172.21.15.63:0/3917626967 shutdown_connections 2024-05-27T07:22:01.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.067+0000 7f786e000700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7868108010 0x7f78681a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:01.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.067+0000 7f786e000700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7868106540 0x7f786819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:01.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.067+0000 7f786e000700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f784c0b2880 0x7f784c0b4d90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:01.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.067+0000 7f786e000700 1 --2- 172.21.15.63:0/3917626967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7868107670 0x7f786819ce40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:01.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.067+0000 7f786e000700 1 -- 172.21.15.63:0/3917626967 >> 172.21.15.63:0/3917626967 conn(0x7f7868101b80 msgr2=0x7f786810a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:22:01.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.068+0000 7f786e000700 1 -- 172.21.15.63:0/3917626967 shutdown_connections 2024-05-27T07:22:01.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:22:01.068+0000 7f786e000700 1 -- 172.21.15.63:0/3917626967 wait complete. 2024-05-27T07:22:01.086 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:00 smithi063 ceph-mon[31613]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:22:01.086 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:00 smithi063 ceph-mon[31613]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T07:22:01.086 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:00 smithi063 ceph-mon[31613]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T07:22:01.086 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:01.086 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:00 smithi063 ceph-mon[31613]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:22:01.086 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:01.086 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:01.086 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:00 smithi086 ceph-mon[34687]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:22:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:00 smithi086 ceph-mon[34687]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T07:22:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:00 smithi086 ceph-mon[34687]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T07:22:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:00 smithi086 ceph-mon[34687]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:22:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:01.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:01.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:00 smithi149 ceph-mon[35408]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:22:01.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:00 smithi149 ceph-mon[35408]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T07:22:01.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:00 smithi149 ceph-mon[35408]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T07:22:01.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:01.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:00 smithi149 ceph-mon[35408]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:22:01.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:01.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:01.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:01.922 DEBUG:teuthology.orchestra.run.smithi063:osd.3> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.3.service 2024-05-27T07:22:01.927 INFO:tasks.cephadm:Deploying osd.4 on smithi086 with /dev/vg_nvme/lv_3... 2024-05-27T07:22:01.927 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T07:22:02.151 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:22:02.151 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T07:22:02.151 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:02.151 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:02.151 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:22:02.151 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:02.151 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:02.152 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' 2024-05-27T07:22:02.152 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] boot 2024-05-27T07:22:02.152 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T07:22:02.152 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:01 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:02.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:22:02.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T07:22:02.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:02.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:02.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:22:02.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:02.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:02.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' 2024-05-27T07:22:02.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] boot 2024-05-27T07:22:02.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T07:22:02.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:01 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:02.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:22:02.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T07:22:02.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:02.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:02.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:22:02.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:02.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:02.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: from='osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415]' entity='osd.3' 2024-05-27T07:22:02.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: osd.3 [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] boot 2024-05-27T07:22:02.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T07:22:02.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:01 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:22:02.354 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:22:03.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:02 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:22:03.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:02 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:22:03.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:02 smithi086 ceph-mon[34687]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:22:03.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:02 smithi086 ceph-mon[34687]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T07:22:03.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:02 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:22:03.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:02 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:22:03.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:02 smithi149 ceph-mon[35408]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:22:03.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:02 smithi149 ceph-mon[35408]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T07:22:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:02 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:22:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:02 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:22:03.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:02 smithi063 ceph-mon[31613]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:22:03.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:02 smithi063 ceph-mon[31613]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T07:22:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:04 smithi086 ceph-mon[34687]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:04 smithi086 ceph-mon[34687]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T07:22:05.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:04 smithi149 ceph-mon[35408]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:05.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:04 smithi149 ceph-mon[35408]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T07:22:05.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:04 smithi063 ceph-mon[31613]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:05.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:04 smithi063 ceph-mon[31613]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T07:22:05.358 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T07:22:05.359 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T07:22:05.359 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-05-27T07:22:05.359 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-05-27T07:22:05.359 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0367388 s, 285 MB/s 2024-05-27T07:22:05.359 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-05-27T07:22:06.111 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-05-27T07:22:06.541 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:22:06.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:06 smithi086 ceph-mon[34687]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:06.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:06 smithi149 ceph-mon[35408]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:07.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:06 smithi063 ceph-mon[31613]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 516 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:08.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:07 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi063 2024-05-27T07:22:08.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:08.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:08.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:08.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:07 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi063 to 8544M 2024-05-27T07:22:08.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:08.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:08.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:08.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:08.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:07 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi063 2024-05-27T07:22:08.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:08.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:08.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:08.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:07 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi063 to 8544M 2024-05-27T07:22:08.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:08.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:08.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:08.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:08.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:07 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi063 2024-05-27T07:22:08.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:08.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:08.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:08.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:07 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi063 to 8544M 2024-05-27T07:22:08.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:08.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:08.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:08.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.803+0000 7fabce400700 1 -- 172.21.15.86:0/687745183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc810c1e0 msgr2=0x7fabc810e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:22:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.803+0000 7fabce400700 1 --2- 172.21.15.86:0/687745183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc810c1e0 0x7fabc810e6f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fabc000b0a0 tx=0x7fabc000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:22:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.803+0000 7fabce400700 1 -- 172.21.15.86:0/687745183 shutdown_connections 2024-05-27T07:22:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.803+0000 7fabce400700 1 --2- 172.21.15.86:0/687745183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc810c1e0 0x7fabc810e6f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.803+0000 7fabce400700 1 --2- 172.21.15.86:0/687745183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc80776d0 0x7fabc8075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.803+0000 7fabce400700 1 --2- 172.21.15.86:0/687745183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabc80760e0 0x7fabc8076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.803+0000 7fabce400700 1 -- 172.21.15.86:0/687745183 >> 172.21.15.86:0/687745183 conn(0x7fabc80ff9b0 msgr2=0x7fabc8101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:22:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.803+0000 7fabce400700 1 -- 172.21.15.86:0/687745183 shutdown_connections 2024-05-27T07:22:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.803+0000 7fabce400700 1 -- 172.21.15.86:0/687745183 wait complete. 2024-05-27T07:22:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.804+0000 7fabce400700 1 Processor -- start 2024-05-27T07:22:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.805+0000 7fabce400700 1 -- start start 2024-05-27T07:22:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.805+0000 7fabce400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc80760e0 0x7fabc819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:22:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.805+0000 7fabce400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc80776d0 0x7fabc819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:22:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.805+0000 7fabce400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabc810c1e0 0x7fabc81a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:22:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.805+0000 7fabce400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fabc8111e40 con 0x7fabc810c1e0 2024-05-27T07:22:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.805+0000 7fabce400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabc8111cc0 con 0x7fabc80776d0 2024-05-27T07:22:08.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.805+0000 7fabce400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fabc8111fc0 con 0x7fabc80760e0 2024-05-27T07:22:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.805+0000 7fabc6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc80776d0 0x7fabc819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:22:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.805+0000 7fabc6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc80776d0 0x7fabc819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41572/0 (socket says 172.21.15.86:41572) 2024-05-27T07:22:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.805+0000 7fabc6a00700 1 -- 172.21.15.86:0/595150503 learned_addr learned my addr 172.21.15.86:0/595150503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-27T07:22:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.806+0000 7fabc7400700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc80760e0 0x7fabc819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:22:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.806+0000 7fabc7e00700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabc810c1e0 0x7fabc81a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:22:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.806+0000 7fabc6a00700 1 -- 172.21.15.86:0/595150503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc80760e0 msgr2=0x7fabc819c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:22:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.806+0000 7fabc6a00700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc80760e0 0x7fabc819c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.806+0000 7fabc6a00700 1 -- 172.21.15.86:0/595150503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabc810c1e0 msgr2=0x7fabc81a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:22:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.806+0000 7fabc6a00700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabc810c1e0 0x7fabc81a3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.806+0000 7fabc6a00700 1 -- 172.21.15.86:0/595150503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc81a4510 con 0x7fabc80776d0 2024-05-27T07:22:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.806+0000 7fabc6a00700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc80776d0 0x7fabc819cef0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fabb400d950 tx=0x7fabb400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:22:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.807+0000 7fabbfe00700 1 -- 172.21.15.86:0/595150503 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4016070 con 0x7fabc80776d0 2024-05-27T07:22:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.807+0000 7fabbfe00700 1 -- 172.21.15.86:0/595150503 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabb4009dd0 con 0x7fabc80776d0 2024-05-27T07:22:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.807+0000 7fabbfe00700 1 -- 172.21.15.86:0/595150503 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb400f3c0 con 0x7fabc80776d0 2024-05-27T07:22:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.808+0000 7fabce400700 1 -- 172.21.15.86:0/595150503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc81a4800 con 0x7fabc80776d0 2024-05-27T07:22:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.808+0000 7fabce400700 1 -- 172.21.15.86:0/595150503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc81a4cf0 con 0x7fabc80776d0 2024-05-27T07:22:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.809+0000 7fabce400700 1 -- 172.21.15.86:0/595150503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc80512b0 con 0x7fabc80776d0 2024-05-27T07:22:08.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.812+0000 7fabbfe00700 1 -- 172.21.15.86:0/595150503 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fabb4019070 con 0x7fabc80776d0 2024-05-27T07:22:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.813+0000 7fabbfe00700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fabac0b2900 0x7fabac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:22:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.814+0000 7fabbfe00700 1 -- 172.21.15.86:0/595150503 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fabb40e0d20 con 0x7fabc80776d0 2024-05-27T07:22:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.814+0000 7fabc7400700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fabac0b2900 0x7fabac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:22:08.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.815+0000 7fabc7400700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fabac0b2900 0x7fabac0b4e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabb8005950 tx=0x7fabb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:22:08.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:08.816+0000 7fabbfe00700 1 -- 172.21.15.86:0/595150503 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fabb40a9bc0 con 0x7fabc80776d0 2024-05-27T07:22:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:09.061+0000 7fabce400700 1 -- 172.21.15.86:0/595150503 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fabc8063bf0 con 0x7fabac0b2900 2024-05-27T07:22:09.052 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:08 smithi086 ceph-mon[34687]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:22:09.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:08 smithi149 ceph-mon[35408]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:22:09.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:08 smithi063 ceph-mon[31613]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:22:10.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:09 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:22:10.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:09 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:22:10.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:09 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:10.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:09 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:22:10.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:09 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:22:10.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:09 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:10.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:09 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:22:10.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:09 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:22:10.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:09 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:11.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:10 smithi086 ceph-mon[34687]: from='client.34211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:22:11.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:10 smithi086 ceph-mon[34687]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-27T07:22:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:10 smithi149 ceph-mon[35408]: from='client.34211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:22:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:10 smithi149 ceph-mon[35408]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-27T07:22:11.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:10 smithi063 ceph-mon[31613]: from='client.34211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:22:11.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:10 smithi063 ceph-mon[31613]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-27T07:22:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:12 smithi086 ceph-mon[34687]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-27T07:22:13.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:12 smithi149 ceph-mon[35408]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-27T07:22:13.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:12 smithi063 ceph-mon[31613]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-27T07:22:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:13 smithi086 ceph-mon[34687]: from='client.? 172.21.15.86:0/1043626731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8b86405-4ec3-4853-9429-85a69f9a5bc9"}]: dispatch 2024-05-27T07:22:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:13 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8b86405-4ec3-4853-9429-85a69f9a5bc9"}]: dispatch 2024-05-27T07:22:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:13 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8b86405-4ec3-4853-9429-85a69f9a5bc9"}]': finished 2024-05-27T07:22:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:13 smithi086 ceph-mon[34687]: osdmap e29: 5 total, 4 up, 5 in 2024-05-27T07:22:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:13 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:14.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:13 smithi149 ceph-mon[35408]: from='client.? 172.21.15.86:0/1043626731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8b86405-4ec3-4853-9429-85a69f9a5bc9"}]: dispatch 2024-05-27T07:22:14.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:13 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8b86405-4ec3-4853-9429-85a69f9a5bc9"}]: dispatch 2024-05-27T07:22:14.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:13 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8b86405-4ec3-4853-9429-85a69f9a5bc9"}]': finished 2024-05-27T07:22:14.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:13 smithi149 ceph-mon[35408]: osdmap e29: 5 total, 4 up, 5 in 2024-05-27T07:22:14.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:13 smithi063 ceph-mon[31613]: from='client.? 172.21.15.86:0/1043626731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8b86405-4ec3-4853-9429-85a69f9a5bc9"}]: dispatch 2024-05-27T07:22:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:13 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8b86405-4ec3-4853-9429-85a69f9a5bc9"}]: dispatch 2024-05-27T07:22:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:13 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8b86405-4ec3-4853-9429-85a69f9a5bc9"}]': finished 2024-05-27T07:22:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:13 smithi063 ceph-mon[31613]: osdmap e29: 5 total, 4 up, 5 in 2024-05-27T07:22:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:14 smithi086 ceph-mon[34687]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T07:22:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:14 smithi086 ceph-mon[34687]: from='client.? 172.21.15.86:0/262290430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:22:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:14 smithi149 ceph-mon[35408]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T07:22:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:14 smithi149 ceph-mon[35408]: from='client.? 172.21.15.86:0/262290430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:22:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:14 smithi063 ceph-mon[31613]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T07:22:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:14 smithi063 ceph-mon[31613]: from='client.? 172.21.15.86:0/262290430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:22:17.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:16 smithi149 ceph-mon[35408]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:22:17.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:16 smithi063 ceph-mon[31613]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:22:17.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:16 smithi086 ceph-mon[34687]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:22:18.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:17 smithi086 ceph-mon[34687]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:18.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:17 smithi149 ceph-mon[35408]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:18.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:17 smithi063 ceph-mon[31613]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:20.937 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:20 smithi086 ceph-mon[34687]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:20.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:20 smithi149 ceph-mon[35408]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:20.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:20 smithi063 ceph-mon[31613]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:22.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:22 smithi086 ceph-mon[34687]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:22.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:22 smithi149 ceph-mon[35408]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:22.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:22 smithi063 ceph-mon[31613]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:23.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T07:22:23.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:23.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T07:22:23.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:23.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T07:22:23.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:24.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:24 smithi086 ceph-mon[34687]: Deploying daemon osd.4 on smithi086 2024-05-27T07:22:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:24 smithi086 ceph-mon[34687]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:25.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:24 smithi149 ceph-mon[35408]: Deploying daemon osd.4 on smithi086 2024-05-27T07:22:25.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:24 smithi149 ceph-mon[35408]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:25.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:24 smithi063 ceph-mon[31613]: Deploying daemon osd.4 on smithi086 2024-05-27T07:22:25.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:24 smithi063 ceph-mon[31613]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:26.973 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:26 smithi086 ceph-mon[34687]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:27.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:26 smithi149 ceph-mon[35408]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:27.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:26 smithi063 ceph-mon[31613]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:28 smithi086 ceph-mon[34687]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:29.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:28 smithi149 ceph-mon[35408]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:29.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:28 smithi063 ceph-mon[31613]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:31.009 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:30 smithi063 ceph-mon[31613]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:31.130 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:30 smithi086 ceph-mon[34687]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:31.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:30 smithi149 ceph-mon[35408]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:33.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:32 smithi086 ceph-mon[34687]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:33.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:33.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:22:33.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:33.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:32 smithi149 ceph-mon[35408]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:33.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:33.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:22:33.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:33.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:32 smithi063 ceph-mon[31613]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:33.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:33.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:22:33.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:34.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:34 smithi086 ceph-mon[34687]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:35.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:34 smithi149 ceph-mon[35408]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:35.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:34 smithi063 ceph-mon[31613]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:36.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:36 smithi086 ceph-mon[34687]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:36.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:36.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:36.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:36.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:36.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:36.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:36 smithi086 ceph-mon[34687]: from='osd.4 [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:22:36.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:36 smithi086 ceph-mon[34687]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:22:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:36 smithi149 ceph-mon[35408]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:36 smithi149 ceph-mon[35408]: from='osd.4 [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:22:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:36 smithi149 ceph-mon[35408]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:22:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:36 smithi063 ceph-mon[31613]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:36 smithi063 ceph-mon[31613]: from='osd.4 [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:22:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:36 smithi063 ceph-mon[31613]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:22:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.882+0000 7fabbfe00700 1 -- 172.21.15.86:0/595150503 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fabc8063bf0 con 0x7fabac0b2900 2024-05-27T07:22:37.873 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 4 on host 'smithi086' 2024-05-27T07:22:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 -- 172.21.15.86:0/595150503 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fabac0b2900 msgr2=0x7fabac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:22:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fabac0b2900 0x7fabac0b4e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabb8005950 tx=0x7fabb800b410 comp rx=0 tx=0).stop 2024-05-27T07:22:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 -- 172.21.15.86:0/595150503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc80776d0 msgr2=0x7fabc819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:22:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc80776d0 0x7fabc819cef0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fabb400d950 tx=0x7fabb400dc60 comp rx=0 tx=0).stop 2024-05-27T07:22:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 -- 172.21.15.86:0/595150503 shutdown_connections 2024-05-27T07:22:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fabac0b2900 0x7fabac0b4e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabc810c1e0 0x7fabc81a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc80760e0 0x7fabc819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 --2- 172.21.15.86:0/595150503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc80776d0 0x7fabc819cef0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 -- 172.21.15.86:0/595150503 >> 172.21.15.86:0/595150503 conn(0x7fabc80ff9b0 msgr2=0x7fabc8108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:22:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.887+0000 7fabce400700 1 -- 172.21.15.86:0/595150503 shutdown_connections 2024-05-27T07:22:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:22:37.888+0000 7fabce400700 1 -- 172.21.15.86:0/595150503 wait complete. 2024-05-27T07:22:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:38 smithi086 ceph-mon[34687]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T07:22:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:38 smithi086 ceph-mon[34687]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T07:22:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:38 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:38 smithi086 ceph-mon[34687]: from='osd.4 [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:22:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:38 smithi086 ceph-mon[34687]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:22:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:38 smithi086 ceph-mon[34687]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:38 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:38 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:38.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:38 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:22:38.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:38 smithi149 ceph-mon[35408]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T07:22:38.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:38 smithi149 ceph-mon[35408]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T07:22:38.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:38 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:38.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:38 smithi149 ceph-mon[35408]: from='osd.4 [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:22:38.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:38 smithi149 ceph-mon[35408]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:22:38.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:38 smithi149 ceph-mon[35408]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:38.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:38 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:38.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:38 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:38.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:38 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:22:38.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:38 smithi063 ceph-mon[31613]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T07:22:38.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:38 smithi063 ceph-mon[31613]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T07:22:38.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:38 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:38.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:38 smithi063 ceph-mon[31613]: from='osd.4 [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:22:38.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:38 smithi063 ceph-mon[31613]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:22:38.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:38 smithi063 ceph-mon[31613]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:38.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:38 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:38.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:38 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:38.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:38 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:22:38.734 DEBUG:teuthology.orchestra.run.smithi086:osd.4> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.4.service 2024-05-27T07:22:38.739 INFO:tasks.cephadm:Deploying osd.5 on smithi149 with /dev/vg_nvme/lv_3... 2024-05-27T07:22:38.739 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T07:22:39.174 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:22:39.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:39 smithi086 ceph-mon[34687]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:22:39.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:39 smithi086 ceph-mon[34687]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T07:22:39.197 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:39 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:39 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:39 smithi086 ceph-mon[34687]: from='osd.4 ' entity='osd.4' 2024-05-27T07:22:39.198 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 07:22:38 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4[46232]: 2024-05-27T07:22:38.950+0000 7f42bc000700 -1 osd.4 0 waiting for initial osdmap 2024-05-27T07:22:39.198 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 07:22:38 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4[46232]: 2024-05-27T07:22:38.989+0000 7f42b1400700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T07:22:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:39 smithi149 ceph-mon[35408]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:22:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:39 smithi149 ceph-mon[35408]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T07:22:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:39 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:39 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:39 smithi149 ceph-mon[35408]: from='osd.4 ' entity='osd.4' 2024-05-27T07:22:39.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:39 smithi063 ceph-mon[31613]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:22:39.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:39 smithi063 ceph-mon[31613]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T07:22:39.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:39 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:39.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:39 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:39.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:39 smithi063 ceph-mon[31613]: from='osd.4 ' entity='osd.4' 2024-05-27T07:22:40.186 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:40 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:22:40.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:40 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:22:40.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:40.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:40 smithi086 ceph-mon[34687]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:40.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:40 smithi086 ceph-mon[34687]: osd.4 [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] boot 2024-05-27T07:22:40.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:40 smithi086 ceph-mon[34687]: osdmap e32: 5 total, 5 up, 5 in 2024-05-27T07:22:40.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:40.389 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:40 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:22:40.389 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:40 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:22:40.389 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:40.390 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:40 smithi149 ceph-mon[35408]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:40.390 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:40 smithi149 ceph-mon[35408]: osd.4 [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] boot 2024-05-27T07:22:40.390 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:40 smithi149 ceph-mon[35408]: osdmap e32: 5 total, 5 up, 5 in 2024-05-27T07:22:40.390 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:40.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:40 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:22:40.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:40 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:22:40.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:40.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:40 smithi063 ceph-mon[31613]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:22:40.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:40 smithi063 ceph-mon[31613]: osd.4 [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] boot 2024-05-27T07:22:40.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:40 smithi063 ceph-mon[31613]: osdmap e32: 5 total, 5 up, 5 in 2024-05-27T07:22:40.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:22:41.892 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T07:22:41.892 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T07:22:41.892 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-27T07:22:41.892 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-27T07:22:41.892 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0373495 s, 281 MB/s 2024-05-27T07:22:41.892 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-27T07:22:42.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:42 smithi149 ceph-mon[35408]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T07:22:42.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:42 smithi149 ceph-mon[35408]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:42.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:42 smithi086 ceph-mon[34687]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T07:22:42.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:42 smithi086 ceph-mon[34687]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:42.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:42 smithi063 ceph-mon[31613]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T07:22:42.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:42 smithi063 ceph-mon[31613]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:42.687 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-05-27T07:22:43.127 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:22:44.689 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi086 2024-05-27T07:22:44.689 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.689 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.690 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:44.690 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:44.690 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi086 to 8544M 2024-05-27T07:22:44.690 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.690 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:44.690 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:44.690 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.690 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:44 smithi149 ceph-mon[35408]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:44.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi086 2024-05-27T07:22:44.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:44.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:44.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi086 to 8544M 2024-05-27T07:22:44.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:44.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:44.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:44 smithi086 ceph-mon[34687]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:44.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi086 2024-05-27T07:22:44.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:44.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:22:44.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi086 to 8544M 2024-05-27T07:22:44.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:44.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:22:44.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:22:44.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:44 smithi063 ceph-mon[31613]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:45.268 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.278+0000 7f7545000700 1 -- 172.21.15.149:0/3681208481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f754010ab00 msgr2=0x7f754010af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:22:45.268 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.278+0000 7f7545000700 1 --2- 172.21.15.149:0/3681208481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f754010ab00 0x7f754010af10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7530009990 tx=0x7f7530009ca0 comp rx=0 tx=0).stop 2024-05-27T07:22:45.268 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.278+0000 7f7545000700 1 -- 172.21.15.149:0/3681208481 shutdown_connections 2024-05-27T07:22:45.268 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.278+0000 7f7545000700 1 --2- 172.21.15.149:0/3681208481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f754010c410 0x7f7540112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:45.269 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.278+0000 7f7545000700 1 --2- 172.21.15.149:0/3681208481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f754010ab00 0x7f754010af10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:45.269 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.278+0000 7f7545000700 1 --2- 172.21.15.149:0/3681208481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f754010ba70 0x7f754010be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:45.269 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.278+0000 7f7545000700 1 -- 172.21.15.149:0/3681208481 >> 172.21.15.149:0/3681208481 conn(0x7f75400799b0 msgr2=0x7f754007be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:22:45.269 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.278+0000 7f7545000700 1 -- 172.21.15.149:0/3681208481 shutdown_connections 2024-05-27T07:22:45.269 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.278+0000 7f7545000700 1 -- 172.21.15.149:0/3681208481 wait complete. 2024-05-27T07:22:45.270 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.279+0000 7f7545000700 1 Processor -- start 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.279+0000 7f7545000700 1 -- start start 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f7545000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f754010ab00 0x7f75401a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f7545000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f754010ba70 0x7f75401a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f7545000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f754010c410 0x7f75401a8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f7545000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7540116280 con 0x7f754010c410 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f7545000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7540116100 con 0x7f754010ba70 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f7545000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7540116400 con 0x7f754010ab00 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f753de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f754010ab00 0x7f75401a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f753de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f754010ab00 0x7f75401a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:46246/0 (socket says 172.21.15.149:46246) 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f753de00700 1 -- 172.21.15.149:0/40832296 learned_addr learned my addr 172.21.15.149:0/40832296 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f753d400700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f754010ba70 0x7f75401a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:22:45.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f753e800700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f754010c410 0x7f75401a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:22:45.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f753de00700 1 -- 172.21.15.149:0/40832296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f754010ba70 msgr2=0x7f75401a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:22:45.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.280+0000 7f753de00700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f754010ba70 0x7f75401a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:45.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.281+0000 7f753de00700 1 -- 172.21.15.149:0/40832296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f754010c410 msgr2=0x7f75401a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:22:45.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.281+0000 7f753de00700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f754010c410 0x7f75401a8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:22:45.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.281+0000 7f753de00700 1 -- 172.21.15.149:0/40832296 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75401a8990 con 0x7f754010ab00 2024-05-27T07:22:45.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.281+0000 7f753d400700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f754010ba70 0x7f75401a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:22:45.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.281+0000 7f753de00700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f754010ab00 0x7f75401a0e80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7530003820 tx=0x7f75300047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:22:45.273 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.281+0000 7f7536a00700 1 -- 172.21.15.149:0/40832296 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530003c10 con 0x7f754010ab00 2024-05-27T07:22:45.273 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.281+0000 7f7536a00700 1 -- 172.21.15.149:0/40832296 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7530004830 con 0x7f754010ab00 2024-05-27T07:22:45.273 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.281+0000 7f7536a00700 1 -- 172.21.15.149:0/40832296 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530005260 con 0x7f754010ab00 2024-05-27T07:22:45.273 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.281+0000 7f7545000700 1 -- 172.21.15.149:0/40832296 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75401a8c20 con 0x7f754010ab00 2024-05-27T07:22:45.273 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.281+0000 7f7545000700 1 -- 172.21.15.149:0/40832296 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75401a9010 con 0x7f754010ab00 2024-05-27T07:22:45.273 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.283+0000 7f7545000700 1 -- 172.21.15.149:0/40832296 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75400512b0 con 0x7f754010ab00 2024-05-27T07:22:45.275 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.286+0000 7f7536a00700 1 -- 172.21.15.149:0/40832296 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f75300054c0 con 0x7f754010ab00 2024-05-27T07:22:45.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.288+0000 7f7536a00700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75240b2880 0x7f75240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:22:45.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.289+0000 7f7536a00700 1 -- 172.21.15.149:0/40832296 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7f75300e23f0 con 0x7f754010ab00 2024-05-27T07:22:45.279 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.289+0000 7f753d400700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75240b2880 0x7f75240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:22:45.280 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.290+0000 7f753d400700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75240b2880 0x7f75240b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7528005fd0 tx=0x7f7528005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:22:45.282 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.292+0000 7f7536a00700 1 -- 172.21.15.149:0/40832296 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7530013070 con 0x7f754010ab00 2024-05-27T07:22:45.524 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:22:45.534+0000 7f7545000700 1 -- 172.21.15.149:0/40832296 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7540063bf0 con 0x7f75240b2880 2024-05-27T07:22:45.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:22:45.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:22:45.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:45.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:22:45.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:22:45.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:45.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:22:45.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:22:45.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:22:46.750 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:46 smithi149 ceph-mon[35408]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:22:46.751 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:46 smithi149 ceph-mon[35408]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:46.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:46 smithi086 ceph-mon[34687]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:22:46.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:46 smithi086 ceph-mon[34687]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:46.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:46 smithi063 ceph-mon[31613]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:22:46.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:46 smithi063 ceph-mon[31613]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:48.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:48 smithi086 ceph-mon[34687]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:48.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:48 smithi149 ceph-mon[35408]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:48.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:48 smithi063 ceph-mon[31613]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:50.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:50 smithi063 ceph-mon[31613]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:50.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:50 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/3987782421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8d5a42-64bb-40f2-83f4-e22767f857f0"}]: dispatch 2024-05-27T07:22:50.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:50 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8d5a42-64bb-40f2-83f4-e22767f857f0"}]: dispatch 2024-05-27T07:22:50.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:50 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff8d5a42-64bb-40f2-83f4-e22767f857f0"}]': finished 2024-05-27T07:22:50.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:50 smithi063 ceph-mon[31613]: osdmap e34: 6 total, 5 up, 6 in 2024-05-27T07:22:50.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:50 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:22:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:50 smithi086 ceph-mon[34687]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:51.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:50 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/3987782421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8d5a42-64bb-40f2-83f4-e22767f857f0"}]: dispatch 2024-05-27T07:22:51.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:50 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8d5a42-64bb-40f2-83f4-e22767f857f0"}]: dispatch 2024-05-27T07:22:51.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:50 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff8d5a42-64bb-40f2-83f4-e22767f857f0"}]': finished 2024-05-27T07:22:51.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:50 smithi086 ceph-mon[34687]: osdmap e34: 6 total, 5 up, 6 in 2024-05-27T07:22:51.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:50 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:22:51.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:50 smithi149 ceph-mon[35408]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:51.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:50 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/3987782421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8d5a42-64bb-40f2-83f4-e22767f857f0"}]: dispatch 2024-05-27T07:22:51.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:50 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8d5a42-64bb-40f2-83f4-e22767f857f0"}]: dispatch 2024-05-27T07:22:51.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:50 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff8d5a42-64bb-40f2-83f4-e22767f857f0"}]': finished 2024-05-27T07:22:51.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:50 smithi149 ceph-mon[35408]: osdmap e34: 6 total, 5 up, 6 in 2024-05-27T07:22:51.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:50 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:22:52.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:51 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/2371691635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:22:52.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:51 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/2371691635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:22:52.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:51 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/2371691635' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:22:53.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:52 smithi086 ceph-mon[34687]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:53.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:52 smithi149 ceph-mon[35408]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:53.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:52 smithi063 ceph-mon[31613]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:54.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:54 smithi149 ceph-mon[35408]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:55.001 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:54 smithi086 ceph-mon[34687]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:55.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:54 smithi063 ceph-mon[31613]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:56.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:56 smithi086 ceph-mon[34687]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:57.100 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:56 smithi149 ceph-mon[35408]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:57.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:56 smithi063 ceph-mon[31613]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:58.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:58 smithi149 ceph-mon[35408]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:58 smithi086 ceph-mon[34687]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:22:59.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:58 smithi063 ceph-mon[31613]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:00.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:59 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T07:23:00.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:22:59 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:00.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:59 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T07:23:00.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:22:59 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:00.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:59 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T07:23:00.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:22:59 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:00.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:00 smithi149 ceph-mon[35408]: Deploying daemon osd.5 on smithi149 2024-05-27T07:23:00.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:00 smithi149 ceph-mon[35408]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:01.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:00 smithi063 ceph-mon[31613]: Deploying daemon osd.5 on smithi149 2024-05-27T07:23:01.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:00 smithi063 ceph-mon[31613]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:01.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:00 smithi086 ceph-mon[34687]: Deploying daemon osd.5 on smithi149 2024-05-27T07:23:01.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:00 smithi086 ceph-mon[34687]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:02.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:02 smithi149 ceph-mon[35408]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:03.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:02 smithi086 ceph-mon[34687]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:02 smithi063 ceph-mon[31613]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:05.064 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:04 smithi149 ceph-mon[35408]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:04 smithi086 ceph-mon[34687]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:05.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:04 smithi063 ceph-mon[31613]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:06.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:06 smithi086 ceph-mon[34687]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:07.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:06 smithi149 ceph-mon[35408]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:06 smithi063 ceph-mon[31613]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:09.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:08 smithi086 ceph-mon[34687]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:09.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:09.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:08 smithi149 ceph-mon[35408]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:09.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:09.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:09.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:09.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:08 smithi063 ceph-mon[31613]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:09.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:09.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:09.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:11.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:10 smithi086 ceph-mon[34687]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:10 smithi149 ceph-mon[35408]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:11.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:10 smithi063 ceph-mon[31613]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:13.098 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:12 smithi149 ceph-mon[35408]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:13.098 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:13.098 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:13.098 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:13.098 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:23:13.098 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:13.098 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:12 smithi149 ceph-mon[35408]: from='osd.5 [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:23:13.099 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:12 smithi149 ceph-mon[35408]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:23:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:12 smithi086 ceph-mon[34687]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:23:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:12 smithi086 ceph-mon[34687]: from='osd.5 [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:23:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:12 smithi086 ceph-mon[34687]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:23:13.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:12 smithi063 ceph-mon[31613]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:13.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:13.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:13.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:13.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:23:13.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:13.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:12 smithi063 ceph-mon[31613]: from='osd.5 [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:23:13.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:12 smithi063 ceph-mon[31613]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:23:13.849 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.858+0000 7f7536a00700 1 -- 172.21.15.149:0/40832296 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7540063bf0 con 0x7f75240b2880 2024-05-27T07:23:13.850 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 5 on host 'smithi149' 2024-05-27T07:23:13.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 -- 172.21.15.149:0/40832296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75240b2880 msgr2=0x7f75240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:23:13.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75240b2880 0x7f75240b4d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7528005fd0 tx=0x7f7528005f00 comp rx=0 tx=0).stop 2024-05-27T07:23:13.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 -- 172.21.15.149:0/40832296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f754010ab00 msgr2=0x7f75401a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:23:13.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f754010ab00 0x7f75401a0e80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7530003820 tx=0x7f75300047b0 comp rx=0 tx=0).stop 2024-05-27T07:23:13.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 -- 172.21.15.149:0/40832296 shutdown_connections 2024-05-27T07:23:13.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75240b2880 0x7f75240b4d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:13.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f754010c410 0x7f75401a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:13.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f754010ab00 0x7f75401a0e80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:13.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 --2- 172.21.15.149:0/40832296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f754010ba70 0x7f75401a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:13.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 -- 172.21.15.149:0/40832296 >> 172.21.15.149:0/40832296 conn(0x7f75400799b0 msgr2=0x7f754010eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:23:13.853 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 -- 172.21.15.149:0/40832296 shutdown_connections 2024-05-27T07:23:13.853 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:23:13.862+0000 7f7545000700 1 -- 172.21.15.149:0/40832296 wait complete. 2024-05-27T07:23:13.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:13 smithi149 ceph-mon[35408]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T07:23:13.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:13 smithi149 ceph-mon[35408]: from='osd.5 [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:23:13.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:13 smithi149 ceph-mon[35408]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T07:23:13.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:13.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:13 smithi149 ceph-mon[35408]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:23:13.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:13.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:13.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:13 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:14.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:13 smithi086 ceph-mon[34687]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T07:23:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:13 smithi086 ceph-mon[34687]: from='osd.5 [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:23:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:13 smithi086 ceph-mon[34687]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T07:23:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:13 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:13 smithi086 ceph-mon[34687]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:23:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:13 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:13 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:14.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:13 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:13 smithi063 ceph-mon[31613]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T07:23:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:13 smithi063 ceph-mon[31613]: from='osd.5 [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:23:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:13 smithi063 ceph-mon[31613]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T07:23:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:13 smithi063 ceph-mon[31613]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:23:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:14.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:13 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:15.008 DEBUG:teuthology.orchestra.run.smithi149:osd.5> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.5.service 2024-05-27T07:23:15.015 INFO:tasks.cephadm:Deploying osd.6 on smithi063 with /dev/vg_nvme/lv_2... 2024-05-27T07:23:15.015 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T07:23:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:14 smithi086 ceph-mon[34687]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:14 smithi086 ceph-mon[34687]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:23:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:14 smithi086 ceph-mon[34687]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T07:23:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:14 smithi086 ceph-mon[34687]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T07:23:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:14 smithi086 ceph-mon[34687]: from='osd.5 ' entity='osd.5' 2024-05-27T07:23:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:14 smithi149 ceph-mon[35408]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:14 smithi149 ceph-mon[35408]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:23:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:14 smithi149 ceph-mon[35408]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T07:23:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:14 smithi149 ceph-mon[35408]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T07:23:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:14 smithi149 ceph-mon[35408]: from='osd.5 ' entity='osd.5' 2024-05-27T07:23:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:14 smithi063 ceph-mon[31613]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:14 smithi063 ceph-mon[31613]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:23:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:14 smithi063 ceph-mon[31613]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T07:23:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:14 smithi063 ceph-mon[31613]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T07:23:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:15.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:14 smithi063 ceph-mon[31613]: from='osd.5 ' entity='osd.5' 2024-05-27T07:23:15.470 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:23:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:15 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:23:16.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:15 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:23:16.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:15 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:16.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:15 smithi086 ceph-mon[34687]: osd.5 [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] boot 2024-05-27T07:23:16.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:15 smithi086 ceph-mon[34687]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T07:23:16.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:15 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:16.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:15 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:23:16.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:15 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:23:16.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:15 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:16.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:15 smithi149 ceph-mon[35408]: osd.5 [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] boot 2024-05-27T07:23:16.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:15 smithi149 ceph-mon[35408]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T07:23:16.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:15 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:15 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:23:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:15 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:23:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:15 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:16.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:15 smithi063 ceph-mon[31613]: osd.5 [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] boot 2024-05-27T07:23:16.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:15 smithi063 ceph-mon[31613]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T07:23:16.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:15 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:23:16.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:16 smithi149 ceph-mon[35408]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:16.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:16 smithi149 ceph-mon[35408]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T07:23:17.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:16 smithi086 ceph-mon[34687]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:17.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:16 smithi086 ceph-mon[34687]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T07:23:17.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:16 smithi063 ceph-mon[31613]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:23:17.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:16 smithi063 ceph-mon[31613]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T07:23:18.899 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T07:23:18.899 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T07:23:18.899 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-27T07:23:18.899 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-27T07:23:18.899 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0356262 s, 294 MB/s 2024-05-27T07:23:18.900 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-27T07:23:18.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:18 smithi149 ceph-mon[35408]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:18.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:18 smithi149 ceph-mon[35408]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T07:23:19.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:18 smithi086 ceph-mon[34687]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:19.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:18 smithi086 ceph-mon[34687]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T07:23:19.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:18 smithi063 ceph-mon[31613]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:19.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:18 smithi063 ceph-mon[31613]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T07:23:20.009 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_2 2024-05-27T07:23:20.472 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:23:20.862 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi149 2024-05-27T07:23:20.862 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.862 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.862 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:23:20.862 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:23:20.862 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi149 to 2908M 2024-05-27T07:23:20.862 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.863 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:20.863 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:23:20.863 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.863 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:20 smithi063 ceph-mon[31613]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi149 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi149 to 2908M 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:20 smithi086 ceph-mon[34687]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi149 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi149 to 2908M 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:20.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:20 smithi149 ceph-mon[35408]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:22.995 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:22 smithi063 ceph-mon[31613]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:23:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.005+0000 7ff4e0800700 1 -- 172.21.15.63:0/3167354393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d810ba70 msgr2=0x7ff4d810be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:23:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.005+0000 7ff4e0800700 1 --2- 172.21.15.63:0/3167354393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d810ba70 0x7ff4d810be80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009990 tx=0x7ff4c8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:23:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.005+0000 7ff4e0800700 1 -- 172.21.15.63:0/3167354393 shutdown_connections 2024-05-27T07:23:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.005+0000 7ff4e0800700 1 --2- 172.21.15.63:0/3167354393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810c410 0x7ff4d8112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.005+0000 7ff4e0800700 1 --2- 172.21.15.63:0/3167354393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d810ab00 0x7ff4d810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.005+0000 7ff4e0800700 1 --2- 172.21.15.63:0/3167354393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d810ba70 0x7ff4d810be80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.005+0000 7ff4e0800700 1 -- 172.21.15.63:0/3167354393 >> 172.21.15.63:0/3167354393 conn(0x7ff4d80799b0 msgr2=0x7ff4d807be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:23:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.005+0000 7ff4e0800700 1 -- 172.21.15.63:0/3167354393 shutdown_connections 2024-05-27T07:23:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.005+0000 7ff4e0800700 1 -- 172.21.15.63:0/3167354393 wait complete. 2024-05-27T07:23:22.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.006+0000 7ff4e0800700 1 Processor -- start 2024-05-27T07:23:22.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.006+0000 7ff4e0800700 1 -- start start 2024-05-27T07:23:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4e0800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d810ab00 0x7ff4d81a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:23:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4e0800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810ba70 0x7ff4d81a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:23:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4e0800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d810c410 0x7ff4d81a8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:23:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4e0800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81162b0 con 0x7ff4d810c410 2024-05-27T07:23:22.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4e0800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d8116130 con 0x7ff4d810ab00 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4e0800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d8116430 con 0x7ff4d810ba70 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4de800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d810c410 0x7ff4d81a8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4de800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d810c410 0x7ff4d81a8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59440/0 (socket says 172.21.15.63:59440) 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4de800700 1 -- 172.21.15.63:0/107684251 learned_addr learned my addr 172.21.15.63:0/107684251 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4dde00700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d810ab00 0x7ff4d81a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.007+0000 7ff4dd400700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810ba70 0x7ff4d81a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4de800700 1 -- 172.21.15.63:0/107684251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810ba70 msgr2=0x7ff4d81a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4de800700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810ba70 0x7ff4d81a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4de800700 1 -- 172.21.15.63:0/107684251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d810ab00 msgr2=0x7ff4d81a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4de800700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d810ab00 0x7ff4d81a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4de800700 1 -- 172.21.15.63:0/107684251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d81a89b0 con 0x7ff4d810c410 2024-05-27T07:23:22.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4dde00700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d810ab00 0x7ff4d81a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:23:23.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4de800700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d810c410 0x7ff4d81a8330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4007ee0 tx=0x7ff4d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:23:23.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4d2a00700 1 -- 172.21.15.63:0/107684251 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d400dbd0 con 0x7ff4d810c410 2024-05-27T07:23:23.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4e0800700 1 -- 172.21.15.63:0/107684251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d81a8ca0 con 0x7ff4d810c410 2024-05-27T07:23:23.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4d2a00700 1 -- 172.21.15.63:0/107684251 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4d400dd70 con 0x7ff4d810c410 2024-05-27T07:23:23.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4d2a00700 1 -- 172.21.15.63:0/107684251 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40058d0 con 0x7ff4d810c410 2024-05-27T07:23:23.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.008+0000 7ff4e0800700 1 -- 172.21.15.63:0/107684251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d81a9190 con 0x7ff4d810c410 2024-05-27T07:23:23.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.010+0000 7ff4e0800700 1 -- 172.21.15.63:0/107684251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d80512b0 con 0x7ff4d810c410 2024-05-27T07:23:23.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.012+0000 7ff4d2a00700 1 -- 172.21.15.63:0/107684251 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff4d401a020 con 0x7ff4d810c410 2024-05-27T07:23:23.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.014+0000 7ff4d2a00700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff4c00b2880 0x7ff4c00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:23:23.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.014+0000 7ff4d2a00700 1 -- 172.21.15.63:0/107684251 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4533+0+0 (secure 0 0 0) 0x7ff4d40e12f0 con 0x7ff4d810c410 2024-05-27T07:23:23.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.014+0000 7ff4dde00700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff4c00b2880 0x7ff4c00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:23:23.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.015+0000 7ff4dde00700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff4c00b2880 0x7ff4c00b4d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc005950 tx=0x7ff4cc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:23:23.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.017+0000 7ff4d2a00700 1 -- 172.21.15.63:0/107684251 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff4d4010070 con 0x7ff4d810c410 2024-05-27T07:23:23.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:22 smithi086 ceph-mon[34687]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:23:23.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:22 smithi149 ceph-mon[35408]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:23:23.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:23.268+0000 7ff4e0800700 1 -- 172.21.15.63:0/107684251 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff4d8063cb0 con 0x7ff4c00b2880 2024-05-27T07:23:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:23:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:23:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:24.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:23:24.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:23:24.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:24.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:23:24.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:23:24.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:24.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:24 smithi063 ceph-mon[31613]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:23:24.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:24 smithi063 ceph-mon[31613]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-27T07:23:25.100 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:24 smithi086 ceph-mon[34687]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:23:25.100 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:24 smithi086 ceph-mon[34687]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-27T07:23:25.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:24 smithi149 ceph-mon[35408]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:23:25.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:24 smithi149 ceph-mon[35408]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-27T07:23:27.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:26 smithi086 ceph-mon[34687]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T07:23:27.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:26 smithi149 ceph-mon[35408]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T07:23:27.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:26 smithi063 ceph-mon[31613]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T07:23:27.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:27 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2426688975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f496ab56-b7a8-47c0-bd34-fe6f2f82f69d"}]: dispatch 2024-05-27T07:23:28.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:27 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2426688975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f496ab56-b7a8-47c0-bd34-fe6f2f82f69d"}]: dispatch 2024-05-27T07:23:28.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:27 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2426688975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f496ab56-b7a8-47c0-bd34-fe6f2f82f69d"}]: dispatch 2024-05-27T07:23:29.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:28 smithi086 ceph-mon[34687]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:23:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:28 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2426688975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f496ab56-b7a8-47c0-bd34-fe6f2f82f69d"}]': finished 2024-05-27T07:23:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:28 smithi086 ceph-mon[34687]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T07:23:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:28 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1963481492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:23:29.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:28 smithi149 ceph-mon[35408]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:23:29.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:28 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2426688975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f496ab56-b7a8-47c0-bd34-fe6f2f82f69d"}]': finished 2024-05-27T07:23:29.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:28 smithi149 ceph-mon[35408]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T07:23:29.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:29.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:28 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1963481492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:23:29.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:28 smithi063 ceph-mon[31613]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:23:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:28 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2426688975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f496ab56-b7a8-47c0-bd34-fe6f2f82f69d"}]': finished 2024-05-27T07:23:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:28 smithi063 ceph-mon[31613]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T07:23:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:28 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1963481492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:23:31.009 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:30 smithi063 ceph-mon[31613]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:23:31.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:30 smithi086 ceph-mon[34687]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:23:31.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:30 smithi149 ceph-mon[35408]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:23:33.081 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:32 smithi063 ceph-mon[31613]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:33.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:32 smithi086 ceph-mon[34687]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:33.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:32 smithi149 ceph-mon[35408]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:35.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:34 smithi063 ceph-mon[31613]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:35.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:34 smithi086 ceph-mon[34687]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:35.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:34 smithi149 ceph-mon[35408]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:37.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:36 smithi086 ceph-mon[34687]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:37.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:36 smithi063 ceph-mon[31613]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:37.217 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:36 smithi149 ceph-mon[35408]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:38.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:37 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T07:23:38.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:37 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:38.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:37 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T07:23:38.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:37 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:38.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:37 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T07:23:38.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:37 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:38.893 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:38 smithi063 ceph-mon[31613]: Deploying daemon osd.6 on smithi063 2024-05-27T07:23:38.893 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:38 smithi063 ceph-mon[31613]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:38 smithi086 ceph-mon[34687]: Deploying daemon osd.6 on smithi063 2024-05-27T07:23:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:38 smithi086 ceph-mon[34687]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:38 smithi149 ceph-mon[35408]: Deploying daemon osd.6 on smithi063 2024-05-27T07:23:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:38 smithi149 ceph-mon[35408]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:41.094 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:40 smithi063 ceph-mon[31613]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:41.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:40 smithi086 ceph-mon[34687]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:41.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:40 smithi149 ceph-mon[35408]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:43.036 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:42 smithi063 ceph-mon[31613]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:42 smithi086 ceph-mon[34687]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:43.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:42 smithi149 ceph-mon[35408]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:44 smithi086 ceph-mon[34687]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:44 smithi149 ceph-mon[35408]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:45.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:44 smithi063 ceph-mon[31613]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:47.104 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:46 smithi149 ceph-mon[35408]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:47.152 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:46 smithi063 ceph-mon[31613]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:47.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:46 smithi086 ceph-mon[34687]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:48.907 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:48.907 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:48.907 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:48.907 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:48 smithi063 ceph-mon[31613]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:48.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:48 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:48.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:48 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:48.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:48 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:48.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:48 smithi086 ceph-mon[34687]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:48.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:48 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:48.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:48 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:48.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:48 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:48.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:48 smithi149 ceph-mon[35408]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:50.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:50 smithi063 ceph-mon[31613]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:50 smithi086 ceph-mon[34687]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:51.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:50 smithi149 ceph-mon[35408]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:52.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:52 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:52.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:52 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:52.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:52 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:52.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:52 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:23:52.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:52 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:52.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:52 smithi086 ceph-mon[34687]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T07:23:52.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:52 smithi086 ceph-mon[34687]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:52.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:52 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:52.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:52 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:52.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:52 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:52.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:52 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:23:52.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:52 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:52.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:52 smithi149 ceph-mon[35408]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T07:23:52.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:52 smithi149 ceph-mon[35408]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:52.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:52 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:52.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:52 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:52.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:52 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:23:52.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:52 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:23:52.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:52 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:52.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:52 smithi063 ceph-mon[31613]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T07:23:52.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:52 smithi063 ceph-mon[31613]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:53.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.367+0000 7ff4d2a00700 1 -- 172.21.15.63:0/107684251 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff4d8063cb0 con 0x7ff4c00b2880 2024-05-27T07:23:53.359 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 6 on host 'smithi063' 2024-05-27T07:23:53.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.372+0000 7ff4e0800700 1 -- 172.21.15.63:0/107684251 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff4c00b2880 msgr2=0x7ff4c00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:23:53.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.372+0000 7ff4e0800700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff4c00b2880 0x7ff4c00b4d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc005950 tx=0x7ff4cc0058e0 comp rx=0 tx=0).stop 2024-05-27T07:23:53.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.372+0000 7ff4e0800700 1 -- 172.21.15.63:0/107684251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d810c410 msgr2=0x7ff4d81a8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:23:53.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.372+0000 7ff4e0800700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d810c410 0x7ff4d81a8330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4007ee0 tx=0x7ff4d400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:23:53.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.372+0000 7ff4e0800700 1 -- 172.21.15.63:0/107684251 shutdown_connections 2024-05-27T07:23:53.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.372+0000 7ff4e0800700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff4c00b2880 0x7ff4c00b4d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:53.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.372+0000 7ff4e0800700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d810c410 0x7ff4d81a8330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:53.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.372+0000 7ff4e0800700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d810ab00 0x7ff4d81a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:53.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.372+0000 7ff4e0800700 1 --2- 172.21.15.63:0/107684251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810ba70 0x7ff4d81a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:23:53.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.372+0000 7ff4e0800700 1 -- 172.21.15.63:0/107684251 >> 172.21.15.63:0/107684251 conn(0x7ff4d80799b0 msgr2=0x7ff4d810eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:23:53.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.373+0000 7ff4e0800700 1 -- 172.21.15.63:0/107684251 shutdown_connections 2024-05-27T07:23:53.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:23:53.373+0000 7ff4e0800700 1 -- 172.21.15.63:0/107684251 wait complete. 2024-05-27T07:23:53.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:53 smithi063 ceph-mon[31613]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T07:23:53.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:53 smithi063 ceph-mon[31613]: osdmap e42: 7 total, 6 up, 7 in 2024-05-27T07:23:53.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:53 smithi063 ceph-mon[31613]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:23:53.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:53 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:53.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:53 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:53.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:53 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:53.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:53 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:53.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:53 smithi086 ceph-mon[34687]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T07:23:53.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:53 smithi086 ceph-mon[34687]: osdmap e42: 7 total, 6 up, 7 in 2024-05-27T07:23:53.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:53 smithi086 ceph-mon[34687]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:23:53.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:53 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:53.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:53 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:53.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:53 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:53.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:53 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:53.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:53 smithi149 ceph-mon[35408]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T07:23:53.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:53 smithi149 ceph-mon[35408]: osdmap e42: 7 total, 6 up, 7 in 2024-05-27T07:23:53.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:53 smithi149 ceph-mon[35408]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:23:53.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:53 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:53.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:53 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:23:53.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:53 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:53.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:53 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:23:54.172 DEBUG:teuthology.orchestra.run.smithi063:osd.6> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.6.service 2024-05-27T07:23:54.176 INFO:tasks.cephadm:Deploying osd.7 on smithi086 with /dev/vg_nvme/lv_2... 2024-05-27T07:23:54.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T07:23:54.616 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:23:54.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:54 smithi086 ceph-mon[34687]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:23:54.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:54 smithi086 ceph-mon[34687]: osdmap e43: 7 total, 6 up, 7 in 2024-05-27T07:23:54.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:54.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:54.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:54 smithi086 ceph-mon[34687]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:54.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:54 smithi063 ceph-mon[31613]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:23:54.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:54 smithi063 ceph-mon[31613]: osdmap e43: 7 total, 6 up, 7 in 2024-05-27T07:23:54.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:54.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:54.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:54 smithi063 ceph-mon[31613]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:54.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:54 smithi149 ceph-mon[35408]: from='osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:23:54.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:54 smithi149 ceph-mon[35408]: osdmap e43: 7 total, 6 up, 7 in 2024-05-27T07:23:54.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:54.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:54.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:54 smithi149 ceph-mon[35408]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:23:55.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:55 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:23:55.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:55 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:23:55.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:55 smithi086 ceph-mon[34687]: osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] boot 2024-05-27T07:23:55.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:55 smithi086 ceph-mon[34687]: osdmap e44: 7 total, 7 up, 7 in 2024-05-27T07:23:55.792 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:55 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:55.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:55 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:23:55.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:55 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:23:55.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:55 smithi149 ceph-mon[35408]: osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] boot 2024-05-27T07:23:55.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:55 smithi149 ceph-mon[35408]: osdmap e44: 7 total, 7 up, 7 in 2024-05-27T07:23:55.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:55 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:55.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:55 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:23:55.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:55 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:23:55.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:55 smithi063 ceph-mon[31613]: osd.6 [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] boot 2024-05-27T07:23:55.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:55 smithi063 ceph-mon[31613]: osdmap e44: 7 total, 7 up, 7 in 2024-05-27T07:23:55.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:55 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:23:56.685 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:56 smithi086 ceph-mon[34687]: osdmap e45: 7 total, 7 up, 7 in 2024-05-27T07:23:56.685 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:56 smithi086 ceph-mon[34687]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:23:56.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:56 smithi063 ceph-mon[31613]: osdmap e45: 7 total, 7 up, 7 in 2024-05-27T07:23:56.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:56 smithi063 ceph-mon[31613]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:23:56.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:56 smithi149 ceph-mon[35408]: osdmap e45: 7 total, 7 up, 7 in 2024-05-27T07:23:56.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:56 smithi149 ceph-mon[35408]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:23:57.552 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T07:23:57.553 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T07:23:57.553 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-05-27T07:23:57.553 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-05-27T07:23:57.553 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0327793 s, 320 MB/s 2024-05-27T07:23:57.553 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-05-27T07:23:58.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-05-27T07:23:58.843 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:23:58 smithi063 ceph-mon[31613]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:23:58.853 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:23:58.998 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:23:58 smithi086 ceph-mon[34687]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:23:59.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:23:58 smithi149 ceph-mon[35408]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:00.180 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi063 2024-05-27T07:24:00.180 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.180 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.180 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:00.180 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:00.180 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:00.180 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi063 to 5696M 2024-05-27T07:24:00.180 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.180 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:00.181 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:24:00.181 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.181 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:00 smithi086 ceph-mon[34687]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:00.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi063 2024-05-27T07:24:00.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:00.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:00.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:00.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi063 to 5696M 2024-05-27T07:24:00.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:00.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:24:00.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:00 smithi149 ceph-mon[35408]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:00.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi063 2024-05-27T07:24:00.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:00.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:00.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:00.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi063 to 5696M 2024-05-27T07:24:00.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:00.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:24:00.457 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:00.457 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:00 smithi063 ceph-mon[31613]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.076+0000 7f2616000700 1 -- 172.21.15.86:0/4016163209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610106370 msgr2=0x7f2610106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:24:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.076+0000 7f2616000700 1 --2- 172.21.15.86:0/4016163209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610106370 0x7f2610106780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2604009990 tx=0x7f2604009ca0 comp rx=0 tx=0).stop 2024-05-27T07:24:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.076+0000 7f2616000700 1 -- 172.21.15.86:0/4016163209 shutdown_connections 2024-05-27T07:24:01.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.076+0000 7f2616000700 1 --2- 172.21.15.86:0/4016163209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2610107e40 0x7f261010e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:01.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.076+0000 7f2616000700 1 --2- 172.21.15.86:0/4016163209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610106370 0x7f2610106780 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:01.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.076+0000 7f2616000700 1 --2- 172.21.15.86:0/4016163209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26101074a0 0x7f26101078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:01.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.076+0000 7f2616000700 1 -- 172.21.15.86:0/4016163209 >> 172.21.15.86:0/4016163209 conn(0x7f26101019b0 msgr2=0x7f2610103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:24:01.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.076+0000 7f2616000700 1 -- 172.21.15.86:0/4016163209 shutdown_connections 2024-05-27T07:24:01.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.076+0000 7f2616000700 1 -- 172.21.15.86:0/4016163209 wait complete. 2024-05-27T07:24:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.078+0000 7f2616000700 1 Processor -- start 2024-05-27T07:24:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.078+0000 7f2616000700 1 -- start start 2024-05-27T07:24:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.078+0000 7f2616000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2610106370 0x7f261019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:24:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.078+0000 7f2616000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26101074a0 0x7f261019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:24:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.078+0000 7f2616000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610107e40 0x7f26101a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:24:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.078+0000 7f2616000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2610111cc0 con 0x7f2610106370 2024-05-27T07:24:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.078+0000 7f2616000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2610111b40 con 0x7f2610107e40 2024-05-27T07:24:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.078+0000 7f2616000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2610111e40 con 0x7f26101074a0 2024-05-27T07:24:01.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.078+0000 7f2615600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610107e40 0x7f26101a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:24:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f2615600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610107e40 0x7f26101a3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48162/0 (socket says 172.21.15.86:48162) 2024-05-27T07:24:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f2615600700 1 -- 172.21.15.86:0/2555943071 learned_addr learned my addr 172.21.15.86:0/2555943071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-27T07:24:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f2614c00700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2610106370 0x7f261019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:24:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f260fe00700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26101074a0 0x7f261019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:24:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f2615600700 1 -- 172.21.15.86:0/2555943071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26101074a0 msgr2=0x7f261019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:24:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f2615600700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26101074a0 0x7f261019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f2615600700 1 -- 172.21.15.86:0/2555943071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2610106370 msgr2=0x7f261019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:24:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f2615600700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2610106370 0x7f261019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f2615600700 1 -- 172.21.15.86:0/2555943071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26101a4450 con 0x7f2610107e40 2024-05-27T07:24:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f260fe00700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26101074a0 0x7f261019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:24:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.079+0000 7f2615600700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610107e40 0x7f26101a3dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2608007ee0 tx=0x7f260800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:24:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.080+0000 7f260d600700 1 -- 172.21.15.86:0/2555943071 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260800dbd0 con 0x7f2610107e40 2024-05-27T07:24:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.080+0000 7f260d600700 1 -- 172.21.15.86:0/2555943071 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f260800dd70 con 0x7f2610107e40 2024-05-27T07:24:01.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.080+0000 7f2616000700 1 -- 172.21.15.86:0/2555943071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26101a4740 con 0x7f2610107e40 2024-05-27T07:24:01.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.080+0000 7f260d600700 1 -- 172.21.15.86:0/2555943071 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26080058d0 con 0x7f2610107e40 2024-05-27T07:24:01.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.080+0000 7f2616000700 1 -- 172.21.15.86:0/2555943071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101a4c30 con 0x7f2610107e40 2024-05-27T07:24:01.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.081+0000 7f2616000700 1 -- 172.21.15.86:0/2555943071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26100512b0 con 0x7f2610107e40 2024-05-27T07:24:01.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.084+0000 7f260d600700 1 -- 172.21.15.86:0/2555943071 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f260801a020 con 0x7f2610107e40 2024-05-27T07:24:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.085+0000 7f260d600700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25f80b2880 0x7f25f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:24:01.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.085+0000 7f260d600700 1 -- 172.21.15.86:0/2555943071 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f26080e0dc0 con 0x7f2610107e40 2024-05-27T07:24:01.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.086+0000 7f2614c00700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25f80b2880 0x7f25f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:24:01.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.087+0000 7f2614c00700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25f80b2880 0x7f25f80b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2604006010 tx=0x7f260401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:24:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.090+0000 7f260d600700 1 -- 172.21.15.86:0/2555943071 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2608010070 con 0x7f2610107e40 2024-05-27T07:24:01.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:01.333+0000 7f2616000700 1 -- 172.21.15.86:0/2555943071 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2610063bf0 con 0x7f25f80b2880 2024-05-27T07:24:01.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:01 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:24:01.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:01 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:24:01.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:01 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:01.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:01 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:24:01.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:01 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:24:01.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:01 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:01.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:01 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:24:01.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:01 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:24:01.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:01 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:02.532 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:02 smithi086 ceph-mon[34687]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:24:02.532 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:02 smithi086 ceph-mon[34687]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:02.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:02 smithi149 ceph-mon[35408]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:24:02.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:02 smithi149 ceph-mon[35408]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:02.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:02 smithi063 ceph-mon[31613]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:24:02.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:02 smithi063 ceph-mon[31613]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:04 smithi086 ceph-mon[34687]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:05.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:04 smithi149 ceph-mon[35408]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:05.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:04 smithi063 ceph-mon[31613]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:06.185 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:05 smithi086 ceph-mon[34687]: from='client.? 172.21.15.86:0/3649079938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76b47ee5-4a59-4a08-9d10-f8cb104236d5"}]: dispatch 2024-05-27T07:24:06.185 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:05 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76b47ee5-4a59-4a08-9d10-f8cb104236d5"}]: dispatch 2024-05-27T07:24:06.185 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:05 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76b47ee5-4a59-4a08-9d10-f8cb104236d5"}]': finished 2024-05-27T07:24:06.185 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:05 smithi086 ceph-mon[34687]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T07:24:06.185 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:06.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:05 smithi149 ceph-mon[35408]: from='client.? 172.21.15.86:0/3649079938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76b47ee5-4a59-4a08-9d10-f8cb104236d5"}]: dispatch 2024-05-27T07:24:06.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:05 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76b47ee5-4a59-4a08-9d10-f8cb104236d5"}]: dispatch 2024-05-27T07:24:06.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:05 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76b47ee5-4a59-4a08-9d10-f8cb104236d5"}]': finished 2024-05-27T07:24:06.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:05 smithi149 ceph-mon[35408]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T07:24:06.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:06.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:05 smithi063 ceph-mon[31613]: from='client.? 172.21.15.86:0/3649079938' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76b47ee5-4a59-4a08-9d10-f8cb104236d5"}]: dispatch 2024-05-27T07:24:06.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:05 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76b47ee5-4a59-4a08-9d10-f8cb104236d5"}]: dispatch 2024-05-27T07:24:06.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:05 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76b47ee5-4a59-4a08-9d10-f8cb104236d5"}]': finished 2024-05-27T07:24:06.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:05 smithi063 ceph-mon[31613]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T07:24:06.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:06 smithi086 ceph-mon[34687]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:06 smithi086 ceph-mon[34687]: from='client.? 172.21.15.86:0/2935432023' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:24:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:06 smithi149 ceph-mon[35408]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:06 smithi149 ceph-mon[35408]: from='client.? 172.21.15.86:0/2935432023' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:24:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:06 smithi063 ceph-mon[31613]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:06 smithi063 ceph-mon[31613]: from='client.? 172.21.15.86:0/2935432023' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:24:09.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:08 smithi086 ceph-mon[34687]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:09.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:08 smithi149 ceph-mon[35408]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:09.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:08 smithi063 ceph-mon[31613]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:10.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:10 smithi086 ceph-mon[34687]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:10 smithi149 ceph-mon[35408]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:11.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:10 smithi063 ceph-mon[31613]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:13.078 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:12 smithi086 ceph-mon[34687]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:13.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:12 smithi149 ceph-mon[35408]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:13.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:12 smithi063 ceph-mon[31613]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:15.185 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:14 smithi086 ceph-mon[34687]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:14 smithi149 ceph-mon[35408]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:14 smithi063 ceph-mon[31613]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:15 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T07:24:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:15 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:16.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:15 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T07:24:16.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:15 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:15 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T07:24:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:15 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:16.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:16 smithi086 ceph-mon[34687]: Deploying daemon osd.7 on smithi086 2024-05-27T07:24:16.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:16 smithi086 ceph-mon[34687]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:17.105 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:16 smithi149 ceph-mon[35408]: Deploying daemon osd.7 on smithi086 2024-05-27T07:24:17.105 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:16 smithi149 ceph-mon[35408]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:17.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:16 smithi063 ceph-mon[31613]: Deploying daemon osd.7 on smithi086 2024-05-27T07:24:17.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:16 smithi063 ceph-mon[31613]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:19.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:18 smithi086 ceph-mon[34687]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:19.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:18 smithi149 ceph-mon[35408]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:19.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:18 smithi063 ceph-mon[31613]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:21.076 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:20 smithi086 ceph-mon[34687]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:21.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:20 smithi149 ceph-mon[35408]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:21.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:20 smithi063 ceph-mon[31613]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:23.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:22 smithi086 ceph-mon[34687]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:23.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:22 smithi149 ceph-mon[35408]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:23.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:22 smithi063 ceph-mon[31613]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:24 smithi086 ceph-mon[34687]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:24:25.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:24 smithi149 ceph-mon[35408]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:25.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:25.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:25.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:24:25.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:24 smithi063 ceph-mon[31613]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:25.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:25.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:25.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:24:27.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:26 smithi086 ceph-mon[34687]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:27.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:26 smithi149 ceph-mon[35408]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:27.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:26 smithi063 ceph-mon[31613]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:28.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:28 smithi086 ceph-mon[34687]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:28.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:28 smithi086 ceph-mon[34687]: from='osd.7 [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:24:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:28 smithi086 ceph-mon[34687]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:24:29.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:28 smithi149 ceph-mon[35408]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:29.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:29.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:29.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:28 smithi149 ceph-mon[35408]: from='osd.7 [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:24:29.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:28 smithi149 ceph-mon[35408]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:24:29.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:28 smithi063 ceph-mon[31613]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:29.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:29.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:29.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:28 smithi063 ceph-mon[31613]: from='osd.7 [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:24:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:28 smithi063 ceph-mon[31613]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:24:30.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.280+0000 7f260d600700 1 -- 172.21.15.86:0/2555943071 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2610063bf0 con 0x7f25f80b2880 2024-05-27T07:24:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 7 on host 'smithi086' 2024-05-27T07:24:30.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 -- 172.21.15.86:0/2555943071 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25f80b2880 msgr2=0x7f25f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:24:30.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25f80b2880 0x7f25f80b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2604006010 tx=0x7f260401a040 comp rx=0 tx=0).stop 2024-05-27T07:24:30.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 -- 172.21.15.86:0/2555943071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610107e40 msgr2=0x7f26101a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:24:30.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610107e40 0x7f26101a3dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2608007ee0 tx=0x7f260800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:24:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 -- 172.21.15.86:0/2555943071 shutdown_connections 2024-05-27T07:24:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610107e40 0x7f26101a3dd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f25f80b2880 0x7f25f80b4d90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2610106370 0x7f261019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 --2- 172.21.15.86:0/2555943071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26101074a0 0x7f261019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 -- 172.21.15.86:0/2555943071 >> 172.21.15.86:0/2555943071 conn(0x7f26101019b0 msgr2=0x7f261010a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:24:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 -- 172.21.15.86:0/2555943071 shutdown_connections 2024-05-27T07:24:30.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:24:30.284+0000 7f2616000700 1 -- 172.21.15.86:0/2555943071 wait complete. 2024-05-27T07:24:30.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T07:24:30.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: from='osd.7 [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:24:30.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T07:24:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:24:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:24:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:30.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:24:30.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T07:24:30.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: from='osd.7 [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:24:30.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T07:24:30.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:30.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:24:30.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:30.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:24:30.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:30.704 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:30.704 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:30.704 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:30.704 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:24:30.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T07:24:30.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: from='osd.7 [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:24:30.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T07:24:30.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:30.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:24:30.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:30.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:24:30.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:30.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:30.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:30.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:30.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:24:31.155 DEBUG:teuthology.orchestra.run.smithi086:osd.7> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.7.service 2024-05-27T07:24:31.159 INFO:tasks.cephadm:Deploying osd.8 on smithi149 with /dev/vg_nvme/lv_2... 2024-05-27T07:24:31.160 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T07:24:31.601 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:31 smithi086 ceph-mon[34687]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:24:31.601 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:31 smithi086 ceph-mon[34687]: osdmap e48: 8 total, 7 up, 8 in 2024-05-27T07:24:31.601 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:31 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:31.602 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:31 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:31.602 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:31 smithi086 ceph-mon[34687]: from='osd.7 ' entity='osd.7' 2024-05-27T07:24:31.609 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:31 smithi063 ceph-mon[31613]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:24:31.609 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:31 smithi063 ceph-mon[31613]: osdmap e48: 8 total, 7 up, 8 in 2024-05-27T07:24:31.609 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:31 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:31.609 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:31 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:31.609 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:31 smithi063 ceph-mon[31613]: from='osd.7 ' entity='osd.7' 2024-05-27T07:24:31.612 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:24:31.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:31 smithi149 ceph-mon[35408]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:24:31.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:31 smithi149 ceph-mon[35408]: osdmap e48: 8 total, 7 up, 8 in 2024-05-27T07:24:31.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:31 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:31.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:31 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:31.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:31 smithi149 ceph-mon[35408]: from='osd.7 ' entity='osd.7' 2024-05-27T07:24:32.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:32 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:24:32.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:32 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:24:32.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:32.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:32 smithi086 ceph-mon[34687]: osd.7 [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] boot 2024-05-27T07:24:32.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:32 smithi086 ceph-mon[34687]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T07:24:32.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:32.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:32 smithi086 ceph-mon[34687]: pgmap v233: 1 pgs: 1 unknown; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:32.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:32 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:24:32.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:32 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:24:32.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:32.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:32 smithi149 ceph-mon[35408]: osd.7 [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] boot 2024-05-27T07:24:32.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:32 smithi149 ceph-mon[35408]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T07:24:32.704 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:32.704 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:32 smithi149 ceph-mon[35408]: pgmap v233: 1 pgs: 1 unknown; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:32.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:32 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:24:32.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:32 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:24:32.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:32.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:32 smithi063 ceph-mon[31613]: osd.7 [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] boot 2024-05-27T07:24:32.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:32 smithi063 ceph-mon[31613]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T07:24:32.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:24:32.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:32 smithi063 ceph-mon[31613]: pgmap v233: 1 pgs: 1 unknown; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:24:33.569 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:33 smithi149 ceph-mon[35408]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-27T07:24:33.569 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:33 smithi149 ceph-mon[35408]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T07:24:33.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:33 smithi086 ceph-mon[34687]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-27T07:24:33.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:33 smithi086 ceph-mon[34687]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T07:24:33.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:33 smithi063 ceph-mon[31613]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-27T07:24:33.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:33 smithi063 ceph-mon[31613]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T07:24:34.510 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T07:24:34.511 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T07:24:34.511 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-27T07:24:34.511 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-27T07:24:34.511 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0414935 s, 253 MB/s 2024-05-27T07:24:34.511 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-27T07:24:34.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:34 smithi086 ceph-mon[34687]: osdmap e51: 8 total, 8 up, 8 in 2024-05-27T07:24:34.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:34 smithi086 ceph-mon[34687]: pgmap v236: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:34.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:34 smithi149 ceph-mon[35408]: osdmap e51: 8 total, 8 up, 8 in 2024-05-27T07:24:34.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:34 smithi149 ceph-mon[35408]: pgmap v236: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:34.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:34 smithi063 ceph-mon[31613]: osdmap e51: 8 total, 8 up, 8 in 2024-05-27T07:24:34.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:34 smithi063 ceph-mon[31613]: pgmap v236: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:35.450 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-05-27T07:24:35.866 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:24:37.013 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:36 smithi149 ceph-mon[35408]: pgmap v237: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:37.014 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.014 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.014 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:37.014 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:37.014 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:37.014 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.014 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:37.014 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:24:37.014 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:36 smithi086 ceph-mon[34687]: pgmap v237: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:24:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:36 smithi063 ceph-mon[31613]: pgmap v237: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:24:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:24:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:24:37.903 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.911+0000 7fbaa6400700 1 -- 172.21.15.149:0/129555060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaa0078760 msgr2=0x7fbaa0076c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:24:37.903 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.911+0000 7fbaa6400700 1 --2- 172.21.15.149:0/129555060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaa0078760 0x7fbaa0076c30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fba90009990 tx=0x7fba90009ca0 comp rx=0 tx=0).stop 2024-05-27T07:24:37.903 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.912+0000 7fbaa6400700 1 -- 172.21.15.149:0/129555060 shutdown_connections 2024-05-27T07:24:37.903 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.912+0000 7fbaa6400700 1 --2- 172.21.15.149:0/129555060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa010c240 0x7fbaa010e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:37.903 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.912+0000 7fbaa6400700 1 --2- 172.21.15.149:0/129555060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaa0078760 0x7fbaa0076c30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:37.903 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.912+0000 7fbaa6400700 1 --2- 172.21.15.149:0/129555060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbaa0077170 0x7fbaa0077580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:37.903 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.912+0000 7fbaa6400700 1 -- 172.21.15.149:0/129555060 >> 172.21.15.149:0/129555060 conn(0x7fbaa00ff9c0 msgr2=0x7fbaa0101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:24:37.904 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.912+0000 7fbaa6400700 1 -- 172.21.15.149:0/129555060 shutdown_connections 2024-05-27T07:24:37.904 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.912+0000 7fbaa6400700 1 -- 172.21.15.149:0/129555060 wait complete. 2024-05-27T07:24:37.905 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.913+0000 7fbaa6400700 1 Processor -- start 2024-05-27T07:24:37.905 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.913+0000 7fbaa6400700 1 -- start start 2024-05-27T07:24:37.905 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.913+0000 7fbaa6400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaa0077170 0x7fbaa019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:24:37.905 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.913+0000 7fbaa6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa0078760 0x7fbaa019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:24:37.905 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.913+0000 7fbaa6400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbaa010c240 0x7fbaa01a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.913+0000 7fbaa6400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa0111ec0 con 0x7fbaa010c240 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.913+0000 7fbaa6400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa0111d40 con 0x7fbaa0078760 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.913+0000 7fbaa6400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa0112040 con 0x7fbaa0077170 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaa0077170 0x7fbaa019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaa0077170 0x7fbaa019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:47926/0 (socket says 172.21.15.149:47926) 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9f400700 1 -- 172.21.15.149:0/2785809516 learned_addr learned my addr 172.21.15.149:0/2785809516 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9fe00700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbaa010c240 0x7fbaa01a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9f400700 1 -- 172.21.15.149:0/2785809516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa0078760 msgr2=0x7fbaa019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9ea00700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa0078760 0x7fbaa019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9f400700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa0078760 0x7fbaa019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9f400700 1 -- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbaa010c240 msgr2=0x7fbaa01a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:24:37.906 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9f400700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbaa010c240 0x7fbaa01a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:24:37.907 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9f400700 1 -- 172.21.15.149:0/2785809516 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa01a45c0 con 0x7fbaa0077170 2024-05-27T07:24:37.907 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9ea00700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa0078760 0x7fbaa019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:24:37.907 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.914+0000 7fba9fe00700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbaa010c240 0x7fbaa01a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:24:37.907 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.915+0000 7fba9f400700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaa0077170 0x7fbaa019ca60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fba90000c00 tx=0x7fba90009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:24:37.907 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.915+0000 7fba97e00700 1 -- 172.21.15.149:0/2785809516 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90026070 con 0x7fbaa0077170 2024-05-27T07:24:37.908 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.915+0000 7fba97e00700 1 -- 172.21.15.149:0/2785809516 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba900040a0 con 0x7fbaa0077170 2024-05-27T07:24:37.908 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.915+0000 7fba97e00700 1 -- 172.21.15.149:0/2785809516 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9000fd40 con 0x7fbaa0077170 2024-05-27T07:24:37.908 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.915+0000 7fbaa6400700 1 -- 172.21.15.149:0/2785809516 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa01a4850 con 0x7fbaa0077170 2024-05-27T07:24:37.908 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.915+0000 7fbaa6400700 1 -- 172.21.15.149:0/2785809516 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa01a4ca0 con 0x7fbaa0077170 2024-05-27T07:24:37.908 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.916+0000 7fbaa6400700 1 -- 172.21.15.149:0/2785809516 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa00512b0 con 0x7fbaa0077170 2024-05-27T07:24:37.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:37 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi086 2024-05-27T07:24:37.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:37 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi086 to 5696M 2024-05-27T07:24:37.910 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.920+0000 7fba97e00700 1 -- 172.21.15.149:0/2785809516 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fba90029070 con 0x7fbaa0077170 2024-05-27T07:24:37.913 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.922+0000 7fba97e00700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba840b2880 0x7fba840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:24:37.913 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.923+0000 7fba97e00700 1 -- 172.21.15.149:0/2785809516 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5397+0+0 (secure 0 0 0) 0x7fba900e27a0 con 0x7fbaa0077170 2024-05-27T07:24:37.913 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.923+0000 7fba9ea00700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba840b2880 0x7fba840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:24:37.914 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.924+0000 7fba9ea00700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba840b2880 0x7fba840b4d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fba8c005fd0 tx=0x7fba8c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:24:37.916 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:37.926+0000 7fba97e00700 1 -- 172.21.15.149:0/2785809516 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba90013070 con 0x7fbaa0077170 2024-05-27T07:24:38.000 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:37 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi086 2024-05-27T07:24:38.000 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:37 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi086 to 5696M 2024-05-27T07:24:38.157 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:24:38.166+0000 7fbaa6400700 1 -- 172.21.15.149:0/2785809516 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fbaa0063bf0 con 0x7fba840b2880 2024-05-27T07:24:38.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:37 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi086 2024-05-27T07:24:38.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:37 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi086 to 5696M 2024-05-27T07:24:39.117 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:38 smithi149 ceph-mon[35408]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 182 KiB/s, 0 objects/s recovering 2024-05-27T07:24:39.117 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:38 smithi149 ceph-mon[35408]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-27T07:24:39.117 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:38 smithi149 ceph-mon[35408]: Cluster is now healthy 2024-05-27T07:24:39.117 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:38 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:24:39.117 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:38 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:24:39.117 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:38 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:39.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:38 smithi086 ceph-mon[34687]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 182 KiB/s, 0 objects/s recovering 2024-05-27T07:24:39.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:38 smithi086 ceph-mon[34687]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-27T07:24:39.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:38 smithi086 ceph-mon[34687]: Cluster is now healthy 2024-05-27T07:24:39.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:38 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:24:39.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:38 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:24:39.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:38 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:38 smithi063 ceph-mon[31613]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 182 KiB/s, 0 objects/s recovering 2024-05-27T07:24:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:38 smithi063 ceph-mon[31613]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-27T07:24:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:38 smithi063 ceph-mon[31613]: Cluster is now healthy 2024-05-27T07:24:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:38 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:24:39.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:38 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:24:39.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:38 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:40.192 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:39 smithi149 ceph-mon[35408]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:24:40.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:39 smithi086 ceph-mon[34687]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:24:40.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:39 smithi063 ceph-mon[31613]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:24:40.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:40 smithi149 ceph-mon[35408]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-27T07:24:41.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:40 smithi086 ceph-mon[34687]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-27T07:24:41.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:40 smithi063 ceph-mon[31613]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-27T07:24:43.061 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:42 smithi149 ceph-mon[35408]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-27T07:24:43.062 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:42 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/3102765117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63112f81-efa0-4622-8eac-311302d0bca6"}]: dispatch 2024-05-27T07:24:43.062 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:42 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63112f81-efa0-4622-8eac-311302d0bca6"}]: dispatch 2024-05-27T07:24:43.062 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:42 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63112f81-efa0-4622-8eac-311302d0bca6"}]': finished 2024-05-27T07:24:43.062 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:42 smithi149 ceph-mon[35408]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T07:24:43.062 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:42 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:24:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:42 smithi086 ceph-mon[34687]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-27T07:24:43.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:42 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/3102765117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63112f81-efa0-4622-8eac-311302d0bca6"}]: dispatch 2024-05-27T07:24:43.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:42 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63112f81-efa0-4622-8eac-311302d0bca6"}]: dispatch 2024-05-27T07:24:43.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:42 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63112f81-efa0-4622-8eac-311302d0bca6"}]': finished 2024-05-27T07:24:43.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:42 smithi086 ceph-mon[34687]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T07:24:43.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:42 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:24:43.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:42 smithi063 ceph-mon[31613]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-27T07:24:43.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:42 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/3102765117' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63112f81-efa0-4622-8eac-311302d0bca6"}]: dispatch 2024-05-27T07:24:43.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:42 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63112f81-efa0-4622-8eac-311302d0bca6"}]: dispatch 2024-05-27T07:24:43.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:42 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63112f81-efa0-4622-8eac-311302d0bca6"}]': finished 2024-05-27T07:24:43.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:42 smithi063 ceph-mon[31613]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T07:24:43.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:42 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:24:44.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:43 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/1602354129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:24:44.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:43 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/1602354129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:24:44.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:43 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/1602354129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:24:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:44 smithi086 ceph-mon[34687]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:24:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:44 smithi149 ceph-mon[35408]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:24:45.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:44 smithi063 ceph-mon[31613]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:24:47.184 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:46 smithi149 ceph-mon[35408]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:24:47.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:46 smithi086 ceph-mon[34687]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:24:47.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:46 smithi063 ceph-mon[31613]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:24:49.190 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:48 smithi149 ceph-mon[35408]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:49.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:48 smithi086 ceph-mon[34687]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:49.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:48 smithi063 ceph-mon[31613]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:50 smithi086 ceph-mon[34687]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:51.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:50 smithi149 ceph-mon[35408]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:51.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:50 smithi063 ceph-mon[31613]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:53.110 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:52 smithi149 ceph-mon[35408]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:53.110 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:52 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T07:24:53.110 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:52 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:53.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:52 smithi086 ceph-mon[34687]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:53.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:52 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T07:24:53.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:52 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:53.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:52 smithi063 ceph-mon[31613]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:53.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:52 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T07:24:53.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:52 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:24:54.131 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:53 smithi149 ceph-mon[35408]: Deploying daemon osd.8 on smithi149 2024-05-27T07:24:54.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:53 smithi086 ceph-mon[34687]: Deploying daemon osd.8 on smithi149 2024-05-27T07:24:54.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:53 smithi063 ceph-mon[31613]: Deploying daemon osd.8 on smithi149 2024-05-27T07:24:54.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:54 smithi149 ceph-mon[35408]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:55.100 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:54 smithi086 ceph-mon[34687]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:55.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:54 smithi063 ceph-mon[31613]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:56.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:56 smithi086 ceph-mon[34687]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:57.179 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:56 smithi149 ceph-mon[35408]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:57.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:56 smithi063 ceph-mon[31613]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:59.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:24:58 smithi086 ceph-mon[34687]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:24:58 smithi149 ceph-mon[35408]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:24:59.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:24:58 smithi063 ceph-mon[31613]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:01.024 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:00 smithi149 ceph-mon[35408]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:01.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:00 smithi086 ceph-mon[34687]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:01.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:00 smithi063 ceph-mon[31613]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:02.526 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:02 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:02.526 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:02 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:02.526 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:02 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:02.526 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:02 smithi149 ceph-mon[35408]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:02.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:02 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:02.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:02 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:02.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:02 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:02.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:02 smithi086 ceph-mon[34687]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:02.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:02 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:02.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:02 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:02.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:02 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:02.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:02 smithi063 ceph-mon[31613]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:05.107 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:04 smithi149 ceph-mon[35408]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:05.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:04 smithi086 ceph-mon[34687]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:05.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:04 smithi063 ceph-mon[31613]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:06.077 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:06.078 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:06.078 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:05 smithi149 ceph-mon[35408]: from='osd.8 [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:25:06.078 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:05 smithi149 ceph-mon[35408]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:25:06.146 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:06.147 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:06.147 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:05 smithi086 ceph-mon[34687]: from='osd.8 [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:25:06.147 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:05 smithi086 ceph-mon[34687]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:25:06.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:06.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:06.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:05 smithi063 ceph-mon[31613]: from='osd.8 [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:25:06.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:05 smithi063 ceph-mon[31613]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:25:06.824 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 8 on host 'smithi149' 2024-05-27T07:25:06.825 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.834+0000 7fba97e00700 1 -- 172.21.15.149:0/2785809516 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbaa0063bf0 con 0x7fba840b2880 2024-05-27T07:25:06.828 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.838+0000 7fbaa6400700 1 -- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba840b2880 msgr2=0x7fba840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:25:06.828 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.838+0000 7fbaa6400700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba840b2880 0x7fba840b4d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fba8c005fd0 tx=0x7fba8c005f00 comp rx=0 tx=0).stop 2024-05-27T07:25:06.828 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.838+0000 7fbaa6400700 1 -- 172.21.15.149:0/2785809516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaa0077170 msgr2=0x7fbaa019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:25:06.829 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.838+0000 7fbaa6400700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaa0077170 0x7fbaa019ca60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fba90000c00 tx=0x7fba90009580 comp rx=0 tx=0).stop 2024-05-27T07:25:06.829 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.839+0000 7fbaa6400700 1 -- 172.21.15.149:0/2785809516 shutdown_connections 2024-05-27T07:25:06.829 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.839+0000 7fbaa6400700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba840b2880 0x7fba840b4d90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:06.829 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.839+0000 7fbaa6400700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbaa010c240 0x7fbaa01a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:06.829 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.839+0000 7fbaa6400700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaa0077170 0x7fbaa019ca60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:06.829 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.839+0000 7fbaa6400700 1 --2- 172.21.15.149:0/2785809516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa0078760 0x7fbaa019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:06.829 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.839+0000 7fbaa6400700 1 -- 172.21.15.149:0/2785809516 >> 172.21.15.149:0/2785809516 conn(0x7fbaa00ff9c0 msgr2=0x7fbaa0108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:25:06.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.840+0000 7fbaa6400700 1 -- 172.21.15.149:0/2785809516 shutdown_connections 2024-05-27T07:25:06.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:25:06.840+0000 7fbaa6400700 1 -- 172.21.15.149:0/2785809516 wait complete. 2024-05-27T07:25:06.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:06.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T07:25:06.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: from='osd.8 [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:25:06.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T07:25:06.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:06.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:25:06.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:06.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:06.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:06.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:06.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:06.954 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: from='osd.8 [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:07.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: from='osd.8 [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:07.743 DEBUG:teuthology.orchestra.run.smithi149:osd.8> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.8.service 2024-05-27T07:25:07.747 INFO:tasks.cephadm:Deploying osd.9 on smithi063 with /dev/vg_nvme/lv_1... 2024-05-27T07:25:07.748 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T07:25:07.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:07 smithi149 ceph-mon[35408]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:25:07.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:07 smithi149 ceph-mon[35408]: osdmap e54: 9 total, 8 up, 9 in 2024-05-27T07:25:07.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:07.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:07.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:07 smithi149 ceph-mon[35408]: from='osd.8 ' entity='osd.8' 2024-05-27T07:25:08.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:07 smithi086 ceph-mon[34687]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:25:08.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:07 smithi086 ceph-mon[34687]: osdmap e54: 9 total, 8 up, 9 in 2024-05-27T07:25:08.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:08.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:08.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:07 smithi086 ceph-mon[34687]: from='osd.8 ' entity='osd.8' 2024-05-27T07:25:08.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:07 smithi063 ceph-mon[31613]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:25:08.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:07 smithi063 ceph-mon[31613]: osdmap e54: 9 total, 8 up, 9 in 2024-05-27T07:25:08.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:08.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:08.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:07 smithi063 ceph-mon[31613]: from='osd.8 ' entity='osd.8' 2024-05-27T07:25:08.205 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:25:08.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:08 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:25:08.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:08 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:25:08.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:08 smithi149 ceph-mon[35408]: pgmap v256: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:08.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:08.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:08 smithi149 ceph-mon[35408]: osd.8 [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] boot 2024-05-27T07:25:08.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:08 smithi149 ceph-mon[35408]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T07:25:08.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:09.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:08 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:25:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:08 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:25:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:08 smithi086 ceph-mon[34687]: pgmap v256: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:08 smithi086 ceph-mon[34687]: osd.8 [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] boot 2024-05-27T07:25:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:08 smithi086 ceph-mon[34687]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T07:25:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:09.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:08 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:25:09.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:08 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:25:09.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:08 smithi063 ceph-mon[31613]: pgmap v256: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:25:09.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:09.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:08 smithi063 ceph-mon[31613]: osd.8 [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] boot 2024-05-27T07:25:09.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:08 smithi063 ceph-mon[31613]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T07:25:09.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:25:10.075 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:09 smithi063 ceph-mon[31613]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T07:25:10.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:09 smithi086 ceph-mon[34687]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T07:25:10.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:09 smithi149 ceph-mon[35408]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T07:25:10.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:10 smithi063 ceph-mon[31613]: pgmap v259: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:10.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:10 smithi063 ceph-mon[31613]: osdmap e57: 9 total, 9 up, 9 in 2024-05-27T07:25:11.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:10 smithi086 ceph-mon[34687]: pgmap v259: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:11.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:10 smithi086 ceph-mon[34687]: osdmap e57: 9 total, 9 up, 9 in 2024-05-27T07:25:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:10 smithi149 ceph-mon[35408]: pgmap v259: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:10 smithi149 ceph-mon[35408]: osdmap e57: 9 total, 9 up, 9 in 2024-05-27T07:25:11.355 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T07:25:11.355 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T07:25:11.355 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-05-27T07:25:11.355 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-05-27T07:25:11.355 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0353204 s, 297 MB/s 2024-05-27T07:25:11.355 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-05-27T07:25:12.229 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi063:vg_nvme/lv_1 2024-05-27T07:25:12.716 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:25:13.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:12 smithi086 ceph-mon[34687]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:13.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:12 smithi149 ceph-mon[35408]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:13.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.203 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:12 smithi063 ceph-mon[31613]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:13.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:13.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:13.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:13.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:13.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:13.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:13.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:14.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:13 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi149 2024-05-27T07:25:14.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:13 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi149 to 1939M 2024-05-27T07:25:14.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:13 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi149 2024-05-27T07:25:14.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:13 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi149 to 1939M 2024-05-27T07:25:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:13 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi149 2024-05-27T07:25:14.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:13 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi149 to 1939M 2024-05-27T07:25:15.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.050+0000 7f840b200700 1 -- 172.21.15.63:0/1447072143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84040776d0 msgr2=0x7f8404075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:25:15.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.050+0000 7f840b200700 1 --2- 172.21.15.63:0/1447072143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84040776d0 0x7f8404075ba0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f83f80099c0 tx=0x7f83f8009cd0 comp rx=0 tx=0).stop 2024-05-27T07:25:15.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.050+0000 7f840b200700 1 -- 172.21.15.63:0/1447072143 shutdown_connections 2024-05-27T07:25:15.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.050+0000 7f840b200700 1 --2- 172.21.15.63:0/1447072143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8404110790 0x7f8404112bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:15.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.050+0000 7f840b200700 1 --2- 172.21.15.63:0/1447072143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84040776d0 0x7f8404075ba0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:15.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.050+0000 7f840b200700 1 --2- 172.21.15.63:0/1447072143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84040760e0 0x7f8404076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:15.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.050+0000 7f840b200700 1 -- 172.21.15.63:0/1447072143 >> 172.21.15.63:0/1447072143 conn(0x7f84040ff7a0 msgr2=0x7f8404101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:25:15.041 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:14 smithi063 ceph-mon[31613]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:25:15.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.050+0000 7f840b200700 1 -- 172.21.15.63:0/1447072143 shutdown_connections 2024-05-27T07:25:15.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.050+0000 7f840b200700 1 -- 172.21.15.63:0/1447072143 wait complete. 2024-05-27T07:25:15.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.051+0000 7f840b200700 1 Processor -- start 2024-05-27T07:25:15.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.051+0000 7f840b200700 1 -- start start 2024-05-27T07:25:15.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.052+0000 7f840b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84040760e0 0x7f84041a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:25:15.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.052+0000 7f840b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84040776d0 0x7f84041a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:25:15.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.052+0000 7f840b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8404110790 0x7f84041a8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:25:15.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.052+0000 7f840b200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8404116030 con 0x7f8404110790 2024-05-27T07:25:15.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.052+0000 7f840b200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8404115eb0 con 0x7f84040776d0 2024-05-27T07:25:15.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.052+0000 7f840b200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f84041161b0 con 0x7f84040760e0 2024-05-27T07:25:15.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.052+0000 7f840a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8404110790 0x7f84041a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:25:15.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.052+0000 7f840a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8404110790 0x7f84041a8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55018/0 (socket says 172.21.15.63:55018) 2024-05-27T07:25:15.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.052+0000 7f840a800700 1 -- 172.21.15.63:0/686853950 learned_addr learned my addr 172.21.15.63:0/686853950 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:25:15.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.052+0000 7f8409e00700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84040760e0 0x7f84041a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:25:15.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.053+0000 7f8409400700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84040776d0 0x7f84041a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:25:15.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.053+0000 7f840a800700 1 -- 172.21.15.63:0/686853950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84040760e0 msgr2=0x7f84041a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:25:15.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.053+0000 7f840a800700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84040760e0 0x7f84041a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:15.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.053+0000 7f840a800700 1 -- 172.21.15.63:0/686853950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84040776d0 msgr2=0x7f84041a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:25:15.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.053+0000 7f840a800700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84040776d0 0x7f84041a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:15.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.053+0000 7f840a800700 1 -- 172.21.15.63:0/686853950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84041a8760 con 0x7f8404110790 2024-05-27T07:25:15.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.053+0000 7f840a800700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8404110790 0x7f84041a8120 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8400007ee0 tx=0x7f840000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:25:15.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.053+0000 7f83fea00700 1 -- 172.21.15.63:0/686853950 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840000dbd0 con 0x7f8404110790 2024-05-27T07:25:15.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.054+0000 7f83fea00700 1 -- 172.21.15.63:0/686853950 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f840000dd70 con 0x7f8404110790 2024-05-27T07:25:15.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.054+0000 7f840b200700 1 -- 172.21.15.63:0/686853950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84041a8a50 con 0x7f8404110790 2024-05-27T07:25:15.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.054+0000 7f83fea00700 1 -- 172.21.15.63:0/686853950 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84000058d0 con 0x7f8404110790 2024-05-27T07:25:15.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.054+0000 7f840b200700 1 -- 172.21.15.63:0/686853950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84041a8f10 con 0x7f8404110790 2024-05-27T07:25:15.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.055+0000 7f840b200700 1 -- 172.21.15.63:0/686853950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840402e5e0 con 0x7f8404110790 2024-05-27T07:25:15.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.058+0000 7f83fea00700 1 -- 172.21.15.63:0/686853950 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f840001a020 con 0x7f8404110790 2024-05-27T07:25:15.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.060+0000 7f83fea00700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f83ec0b2880 0x7f83ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:25:15.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.061+0000 7f83fea00700 1 -- 172.21.15.63:0/686853950 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7f84000e1750 con 0x7f8404110790 2024-05-27T07:25:15.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.061+0000 7f8409e00700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f83ec0b2880 0x7f83ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:25:15.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.061+0000 7f8409e00700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f83ec0b2880 0x7f83ec0b4d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009990 tx=0x7f83f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:25:15.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.064+0000 7f83fea00700 1 -- 172.21.15.63:0/686853950 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8400010070 con 0x7f8404110790 2024-05-27T07:25:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:14 smithi086 ceph-mon[34687]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:25:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:14 smithi149 ceph-mon[35408]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:25:15.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:15.310+0000 7f840b200700 1 -- 172.21.15.63:0/686853950 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8404063bf0 con 0x7f83ec0b2880 2024-05-27T07:25:16.193 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:15 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:25:16.193 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:15 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:25:16.193 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:15 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:15 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:25:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:15 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:25:16.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:15 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:15 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:25:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:15 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:25:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:15 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:17.177 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:16 smithi063 ceph-mon[31613]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:25:17.178 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:16 smithi063 ceph-mon[31613]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-27T07:25:17.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:16 smithi086 ceph-mon[34687]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:25:17.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:16 smithi086 ceph-mon[34687]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-27T07:25:17.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:16 smithi149 ceph-mon[35408]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:25:17.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:16 smithi149 ceph-mon[35408]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-27T07:25:19.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:18 smithi086 ceph-mon[34687]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T07:25:19.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:18 smithi149 ceph-mon[35408]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T07:25:19.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:18 smithi063 ceph-mon[31613]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T07:25:20.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:19 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3668515377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee506119-9cb2-4271-84da-ddcc6ca6cf09"}]: dispatch 2024-05-27T07:25:20.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:19 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3668515377' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee506119-9cb2-4271-84da-ddcc6ca6cf09"}]': finished 2024-05-27T07:25:20.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:19 smithi086 ceph-mon[34687]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T07:25:20.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:20.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:19 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3668515377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee506119-9cb2-4271-84da-ddcc6ca6cf09"}]: dispatch 2024-05-27T07:25:20.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:19 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3668515377' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee506119-9cb2-4271-84da-ddcc6ca6cf09"}]': finished 2024-05-27T07:25:20.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:19 smithi149 ceph-mon[35408]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T07:25:20.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:19 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:20.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:19 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3668515377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee506119-9cb2-4271-84da-ddcc6ca6cf09"}]: dispatch 2024-05-27T07:25:20.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:19 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3668515377' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee506119-9cb2-4271-84da-ddcc6ca6cf09"}]': finished 2024-05-27T07:25:20.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:19 smithi063 ceph-mon[31613]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T07:25:20.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:21.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:20 smithi086 ceph-mon[34687]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:25:21.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:20 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1199776231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:25:21.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:20 smithi149 ceph-mon[35408]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:25:21.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:20 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1199776231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:25:21.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:20 smithi063 ceph-mon[31613]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:25:21.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:20 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1199776231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:25:23.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:22 smithi063 ceph-mon[31613]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:25:23.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:22 smithi086 ceph-mon[34687]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:25:23.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:22 smithi149 ceph-mon[35408]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:25:25.251 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:24 smithi086 ceph-mon[34687]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:25.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:24 smithi149 ceph-mon[35408]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:25.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:24 smithi063 ceph-mon[31613]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:27.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:26 smithi063 ceph-mon[31613]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:27.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:26 smithi086 ceph-mon[34687]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:27.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:26 smithi149 ceph-mon[35408]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:29.287 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:28 smithi149 ceph-mon[35408]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:29.413 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:28 smithi063 ceph-mon[31613]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:29.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:28 smithi086 ceph-mon[34687]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:30.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T07:25:30.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:30.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T07:25:30.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:30.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T07:25:30.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:31.141 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:30 smithi063 ceph-mon[31613]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:31.141 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:30 smithi063 ceph-mon[31613]: Deploying daemon osd.9 on smithi063 2024-05-27T07:25:31.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:30 smithi086 ceph-mon[34687]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:31.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:30 smithi086 ceph-mon[34687]: Deploying daemon osd.9 on smithi063 2024-05-27T07:25:31.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:30 smithi149 ceph-mon[35408]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:31.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:30 smithi149 ceph-mon[35408]: Deploying daemon osd.9 on smithi063 2024-05-27T07:25:33.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:33 smithi063 ceph-mon[31613]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:33.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:33 smithi086 ceph-mon[34687]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:33.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:33 smithi149 ceph-mon[35408]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:34.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:34 smithi086 ceph-mon[34687]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:34.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:34 smithi149 ceph-mon[35408]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:34.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:34 smithi063 ceph-mon[31613]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:37.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:36 smithi086 ceph-mon[34687]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:36 smithi149 ceph-mon[35408]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:36 smithi063 ceph-mon[31613]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:38.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:38 smithi063 ceph-mon[31613]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:38 smithi086 ceph-mon[34687]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:38 smithi149 ceph-mon[35408]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:40.802 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:40.802 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:40.803 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:40.803 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:40 smithi063 ceph-mon[31613]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:40.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:40.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:40.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:40.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:40 smithi086 ceph-mon[34687]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:40.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:40.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:40.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:40.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:40 smithi149 ceph-mon[35408]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:42.820 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:42 smithi063 ceph-mon[31613]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:42 smithi086 ceph-mon[34687]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:43.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:42 smithi149 ceph-mon[35408]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:44.667 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:44.667 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:44.667 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:44 smithi063 ceph-mon[31613]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:44.668 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:44 smithi063 ceph-mon[31613]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T07:25:44.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:44.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:44.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:44 smithi086 ceph-mon[34687]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:44.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:44 smithi086 ceph-mon[34687]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T07:25:44.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:44.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:44.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:44 smithi149 ceph-mon[35408]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:44.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:44 smithi149 ceph-mon[35408]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T07:25:45.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.131+0000 7f83fea00700 1 -- 172.21.15.63:0/686853950 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8404063bf0 con 0x7f83ec0b2880 2024-05-27T07:25:45.122 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 9 on host 'smithi063' 2024-05-27T07:25:45.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.135+0000 7f840b200700 1 -- 172.21.15.63:0/686853950 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f83ec0b2880 msgr2=0x7f83ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:25:45.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.135+0000 7f840b200700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f83ec0b2880 0x7f83ec0b4d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009990 tx=0x7f83f8005fb0 comp rx=0 tx=0).stop 2024-05-27T07:25:45.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.135+0000 7f840b200700 1 -- 172.21.15.63:0/686853950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8404110790 msgr2=0x7f84041a8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:25:45.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.135+0000 7f840b200700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8404110790 0x7f84041a8120 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8400007ee0 tx=0x7f840000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:25:45.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.135+0000 7f840b200700 1 -- 172.21.15.63:0/686853950 shutdown_connections 2024-05-27T07:25:45.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.135+0000 7f840b200700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f83ec0b2880 0x7f83ec0b4d90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:45.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.135+0000 7f840b200700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8404110790 0x7f84041a8120 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:45.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.135+0000 7f840b200700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84040760e0 0x7f84041a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:45.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.135+0000 7f840b200700 1 --2- 172.21.15.63:0/686853950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84040776d0 0x7f84041a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:45.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.135+0000 7f840b200700 1 -- 172.21.15.63:0/686853950 >> 172.21.15.63:0/686853950 conn(0x7f84040ff7a0 msgr2=0x7f8404103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:25:45.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.136+0000 7f840b200700 1 -- 172.21.15.63:0/686853950 shutdown_connections 2024-05-27T07:25:45.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:25:45.136+0000 7f840b200700 1 -- 172.21.15.63:0/686853950 wait complete. 2024-05-27T07:25:45.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:45 smithi086 ceph-mon[34687]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T07:25:45.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:45 smithi086 ceph-mon[34687]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T07:25:45.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:45 smithi086 ceph-mon[34687]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:25:45.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:45.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:45.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:45.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:45.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:45.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:45.700 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:45.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:45 smithi149 ceph-mon[35408]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T07:25:45.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:45 smithi149 ceph-mon[35408]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T07:25:45.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:45 smithi149 ceph-mon[35408]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:25:45.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:45.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:45.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:45.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:45.704 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:45.704 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:45.704 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:45.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:45 smithi063 ceph-mon[31613]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T07:25:45.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:45 smithi063 ceph-mon[31613]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T07:25:45.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:45 smithi063 ceph-mon[31613]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-05-27T07:25:45.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:45.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:45.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:45.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:45.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:45.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:25:45.707 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:45.948 DEBUG:teuthology.orchestra.run.smithi063:osd.9> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.9.service 2024-05-27T07:25:45.953 INFO:tasks.cephadm:Deploying osd.10 on smithi086 with /dev/vg_nvme/lv_1... 2024-05-27T07:25:45.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T07:25:46.398 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:25:46.488 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:46 smithi063 ceph-mon[31613]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:25:46.488 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:46 smithi063 ceph-mon[31613]: osdmap e60: 10 total, 9 up, 10 in 2024-05-27T07:25:46.488 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:46 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:46.488 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:46 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:46.489 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:46 smithi063 ceph-mon[31613]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:46.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:46 smithi086 ceph-mon[34687]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:25:46.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:46 smithi086 ceph-mon[34687]: osdmap e60: 10 total, 9 up, 10 in 2024-05-27T07:25:46.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:46 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:46.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:46 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:46.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:46 smithi086 ceph-mon[34687]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:46.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:46 smithi149 ceph-mon[35408]: from='osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-05-27T07:25:46.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:46 smithi149 ceph-mon[35408]: osdmap e60: 10 total, 9 up, 10 in 2024-05-27T07:25:46.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:46 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:46.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:46 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:46.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:46 smithi149 ceph-mon[35408]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:25:47.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:47 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:25:47.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:47 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:25:47.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:47 smithi086 ceph-mon[34687]: osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] boot 2024-05-27T07:25:47.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:47 smithi086 ceph-mon[34687]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T07:25:47.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:47 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:47.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:47 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:25:47.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:47 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:25:47.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:47 smithi149 ceph-mon[35408]: osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] boot 2024-05-27T07:25:47.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:47 smithi149 ceph-mon[35408]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T07:25:47.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:47 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:47.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:47 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:25:47.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:47 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:25:47.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:47 smithi063 ceph-mon[31613]: osd.9 [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] boot 2024-05-27T07:25:47.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:47 smithi063 ceph-mon[31613]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T07:25:47.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:47 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:25:48.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:48 smithi086 ceph-mon[34687]: osdmap e62: 10 total, 10 up, 10 in 2024-05-27T07:25:48.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:48 smithi086 ceph-mon[34687]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:48.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:48 smithi149 ceph-mon[35408]: osdmap e62: 10 total, 10 up, 10 in 2024-05-27T07:25:48.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:48 smithi149 ceph-mon[35408]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:48.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:48 smithi063 ceph-mon[31613]: osdmap e62: 10 total, 10 up, 10 in 2024-05-27T07:25:48.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:48 smithi063 ceph-mon[31613]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T07:25:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T07:25:49.383 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-05-27T07:25:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-05-27T07:25:49.383 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0344417 s, 304 MB/s 2024-05-27T07:25:49.384 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-05-27T07:25:50.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-05-27T07:25:50.826 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:25:51.151 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:50 smithi086 ceph-mon[34687]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:51.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:50 smithi149 ceph-mon[35408]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:51.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:50 smithi063 ceph-mon[31613]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:52.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi063 2024-05-27T07:25:52.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi063 to 4272M 2024-05-27T07:25:52.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:52.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:52.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:51 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi063 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi063 to 4272M 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:52.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:51 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi063 2024-05-27T07:25:52.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:25:52.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi063 to 4272M 2024-05-27T07:25:52.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:52.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:52.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:25:52.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:51 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:25:53.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.197+0000 7f0a34200700 1 -- 172.21.15.86:0/1276558994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a2c105490 msgr2=0x7f0a2c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:25:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.197+0000 7f0a34200700 1 --2- 172.21.15.86:0/1276558994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a2c105490 0x7f0a2c1058a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009990 tx=0x7f0a18009ca0 comp rx=0 tx=0).stop 2024-05-27T07:25:53.187 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:52 smithi086 ceph-mon[34687]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.197+0000 7f0a34200700 1 -- 172.21.15.86:0/1276558994 shutdown_connections 2024-05-27T07:25:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.197+0000 7f0a34200700 1 --2- 172.21.15.86:0/1276558994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a2c105e30 0x7f0a2c10c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.197+0000 7f0a34200700 1 --2- 172.21.15.86:0/1276558994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a2c104360 0x7f0a2c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:53.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.197+0000 7f0a34200700 1 --2- 172.21.15.86:0/1276558994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a2c105490 0x7f0a2c1058a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:53.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.197+0000 7f0a34200700 1 -- 172.21.15.86:0/1276558994 >> 172.21.15.86:0/1276558994 conn(0x7f0a2c0ff9c0 msgr2=0x7f0a2c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:25:53.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.197+0000 7f0a34200700 1 -- 172.21.15.86:0/1276558994 shutdown_connections 2024-05-27T07:25:53.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.197+0000 7f0a34200700 1 -- 172.21.15.86:0/1276558994 wait complete. 2024-05-27T07:25:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.199+0000 7f0a34200700 1 Processor -- start 2024-05-27T07:25:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.199+0000 7f0a34200700 1 -- start start 2024-05-27T07:25:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.199+0000 7f0a34200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a2c104360 0x7f0a2c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:25:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.199+0000 7f0a34200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a2c105490 0x7f0a2c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:25:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.199+0000 7f0a34200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a2c105e30 0x7f0a2c1a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:25:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.199+0000 7f0a34200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10fc70 con 0x7f0a2c105490 2024-05-27T07:25:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.199+0000 7f0a34200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10faf0 con 0x7f0a2c105e30 2024-05-27T07:25:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.199+0000 7f0a34200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10fdf0 con 0x7f0a2c104360 2024-05-27T07:25:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a32200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a2c105e30 0x7f0a2c1a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:25:53.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a32200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a2c105e30 0x7f0a2c1a3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33636/0 (socket says 172.21.15.86:33636) 2024-05-27T07:25:53.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a32200700 1 -- 172.21.15.86:0/2543288256 learned_addr learned my addr 172.21.15.86:0/2543288256 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-05-27T07:25:53.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a31800700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a2c104360 0x7f0a2c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a30e00700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a2c105490 0x7f0a2c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a32200700 1 -- 172.21.15.86:0/2543288256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a2c104360 msgr2=0x7f0a2c19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a32200700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a2c104360 0x7f0a2c19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a32200700 1 -- 172.21.15.86:0/2543288256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a2c105490 msgr2=0x7f0a2c19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a32200700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a2c105490 0x7f0a2c19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a32200700 1 -- 172.21.15.86:0/2543288256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c1a4520 con 0x7f0a2c105e30 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.200+0000 7f0a32200700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a2c105e30 0x7f0a2c1a3f30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0a24007d70 tx=0x7f0a2400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.201+0000 7f0a2a000700 1 -- 172.21.15.86:0/2543288256 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24016020 con 0x7f0a2c105e30 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.201+0000 7f0a2a000700 1 -- 172.21.15.86:0/2543288256 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a2400ad40 con 0x7f0a2c105e30 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.201+0000 7f0a2a000700 1 -- 172.21.15.86:0/2543288256 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a240115e0 con 0x7f0a2c105e30 2024-05-27T07:25:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.201+0000 7f0a34200700 1 -- 172.21.15.86:0/2543288256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c1a4810 con 0x7f0a2c105e30 2024-05-27T07:25:53.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.201+0000 7f0a34200700 1 -- 172.21.15.86:0/2543288256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c1a4d00 con 0x7f0a2c105e30 2024-05-27T07:25:53.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.202+0000 7f0a34200700 1 -- 172.21.15.86:0/2543288256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c0512b0 con 0x7f0a2c105e30 2024-05-27T07:25:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.204+0000 7f0a2a000700 1 -- 172.21.15.86:0/2543288256 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0a24019020 con 0x7f0a2c105e30 2024-05-27T07:25:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.206+0000 7f0a2a000700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a140b2880 0x7f0a140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:25:53.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.206+0000 7f0a2a000700 1 -- 172.21.15.86:0/2543288256 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f0a240e1b40 con 0x7f0a2c105e30 2024-05-27T07:25:53.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.207+0000 7f0a31800700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a140b2880 0x7f0a140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:25:53.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.208+0000 7f0a31800700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a140b2880 0x7f0a140b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0a20005fd0 tx=0x7f0a20005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:25:53.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.211+0000 7f0a2a000700 1 -- 172.21.15.86:0/2543288256 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a24020020 con 0x7f0a2c105e30 2024-05-27T07:25:53.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:52 smithi149 ceph-mon[35408]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:53.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:52 smithi063 ceph-mon[31613]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:53.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:25:53.455+0000 7f0a34200700 1 -- 172.21.15.86:0/2543288256 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0a2c1a4fc0 con 0x7f0a140b2880 2024-05-27T07:25:54.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:53 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:25:54.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:53 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:25:54.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:53 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:54.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:53 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:25:54.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:53 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:25:54.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:53 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:54.419 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:53 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:25:54.419 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:53 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:25:54.420 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:53 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:25:55.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:54 smithi086 ceph-mon[34687]: from='client.34355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:25:55.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:54 smithi086 ceph-mon[34687]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:55.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:54 smithi063 ceph-mon[31613]: from='client.34355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:25:55.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:54 smithi063 ceph-mon[31613]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:55.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:54 smithi149 ceph-mon[35408]: from='client.34355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:25:55.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:54 smithi149 ceph-mon[35408]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:57.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:56 smithi086 ceph-mon[34687]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:57.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:56 smithi149 ceph-mon[35408]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:57.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:56 smithi063 ceph-mon[31613]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:58.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:57 smithi086 ceph-mon[34687]: from='client.? 172.21.15.86:0/735030827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "861ae487-365b-4de2-91e9-1d8254d36aaf"}]: dispatch 2024-05-27T07:25:58.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:57 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "861ae487-365b-4de2-91e9-1d8254d36aaf"}]: dispatch 2024-05-27T07:25:58.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:57 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "861ae487-365b-4de2-91e9-1d8254d36aaf"}]': finished 2024-05-27T07:25:58.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:57 smithi086 ceph-mon[34687]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T07:25:58.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:57 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:25:58.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:57 smithi149 ceph-mon[35408]: from='client.? 172.21.15.86:0/735030827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "861ae487-365b-4de2-91e9-1d8254d36aaf"}]: dispatch 2024-05-27T07:25:58.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:57 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "861ae487-365b-4de2-91e9-1d8254d36aaf"}]: dispatch 2024-05-27T07:25:58.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:57 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "861ae487-365b-4de2-91e9-1d8254d36aaf"}]': finished 2024-05-27T07:25:58.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:57 smithi149 ceph-mon[35408]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T07:25:58.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:57 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:25:58.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:57 smithi063 ceph-mon[31613]: from='client.? 172.21.15.86:0/735030827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "861ae487-365b-4de2-91e9-1d8254d36aaf"}]: dispatch 2024-05-27T07:25:58.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:57 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "861ae487-365b-4de2-91e9-1d8254d36aaf"}]: dispatch 2024-05-27T07:25:58.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:57 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "861ae487-365b-4de2-91e9-1d8254d36aaf"}]': finished 2024-05-27T07:25:58.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:57 smithi063 ceph-mon[31613]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T07:25:58.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:57 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:25:59.298 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:58 smithi149 ceph-mon[35408]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:59.298 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:25:58 smithi149 ceph-mon[35408]: from='client.? 172.21.15.86:0/1710657140' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:25:59.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:58 smithi086 ceph-mon[34687]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:59.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:25:58 smithi086 ceph-mon[34687]: from='client.? 172.21.15.86:0/1710657140' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:25:59.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:58 smithi063 ceph-mon[31613]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:25:59.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:25:58 smithi063 ceph-mon[31613]: from='client.? 172.21.15.86:0/1710657140' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:26:01.257 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:00 smithi063 ceph-mon[31613]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:01.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:00 smithi086 ceph-mon[34687]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:01.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:00 smithi149 ceph-mon[35408]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:02 smithi063 ceph-mon[31613]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:03.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:02 smithi086 ceph-mon[34687]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:03.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:02 smithi149 ceph-mon[35408]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:05.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:04 smithi086 ceph-mon[34687]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:05.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:04 smithi149 ceph-mon[35408]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:05.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:04 smithi063 ceph-mon[31613]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:07.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:07 smithi086 ceph-mon[34687]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:07.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:07 smithi149 ceph-mon[35408]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:07.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:07 smithi063 ceph-mon[31613]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:08.154 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T07:26:08.155 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:08.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T07:26:08.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:08.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T07:26:08.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:09.406 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:09 smithi086 ceph-mon[34687]: Deploying daemon osd.10 on smithi086 2024-05-27T07:26:09.406 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:09 smithi086 ceph-mon[34687]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:09.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:09 smithi149 ceph-mon[35408]: Deploying daemon osd.10 on smithi086 2024-05-27T07:26:09.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:09 smithi149 ceph-mon[35408]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:09.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:09 smithi063 ceph-mon[31613]: Deploying daemon osd.10 on smithi086 2024-05-27T07:26:09.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:09 smithi063 ceph-mon[31613]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:10.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:10 smithi086 ceph-mon[34687]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:10.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:10 smithi149 ceph-mon[35408]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:10.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:10 smithi063 ceph-mon[31613]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:13.017 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:12 smithi086 ceph-mon[34687]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:13.022 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:12 smithi063 ceph-mon[31613]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:13.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:12 smithi149 ceph-mon[35408]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:15.030 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:14 smithi086 ceph-mon[34687]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:14 smithi149 ceph-mon[35408]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:14 smithi063 ceph-mon[31613]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:17.061 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:16 smithi086 ceph-mon[34687]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:17.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:16 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:17.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:16 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:17.062 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:16 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:17.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:16 smithi149 ceph-mon[35408]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:17.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:16 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:17.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:16 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:17.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:16 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:17.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:16 smithi063 ceph-mon[31613]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:17.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:16 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:17.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:16 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:17.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:16 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:18.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:18 smithi086 ceph-mon[34687]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:19.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:18 smithi149 ceph-mon[35408]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:19.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:18 smithi063 ceph-mon[31613]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:20.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:20 smithi086 ceph-mon[34687]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:20.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:20.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:20.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:20.947 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:26:20.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:21.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:20 smithi149 ceph-mon[35408]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:21.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:21.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:21.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:21.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:26:21.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:21.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:20 smithi063 ceph-mon[31613]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:21.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:21.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:21.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:21.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:26:21.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:22.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:21 smithi086 ceph-mon[34687]: from='osd.10 [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:26:22.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:21 smithi086 ceph-mon[34687]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:26:22.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:21 smithi149 ceph-mon[35408]: from='osd.10 [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:26:22.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:21 smithi149 ceph-mon[35408]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:26:22.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:21 smithi063 ceph-mon[31613]: from='osd.10 [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:26:22.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:21 smithi063 ceph-mon[31613]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:26:22.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.376+0000 7f0a2a000700 1 -- 172.21.15.86:0/2543288256 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0a2c1a4fc0 con 0x7f0a140b2880 2024-05-27T07:26:22.368 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 10 on host 'smithi086' 2024-05-27T07:26:22.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.380+0000 7f0a34200700 1 -- 172.21.15.86:0/2543288256 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a140b2880 msgr2=0x7f0a140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:26:22.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.380+0000 7f0a34200700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a140b2880 0x7f0a140b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0a20005fd0 tx=0x7f0a20005dd0 comp rx=0 tx=0).stop 2024-05-27T07:26:22.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.380+0000 7f0a34200700 1 -- 172.21.15.86:0/2543288256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a2c105e30 msgr2=0x7f0a2c1a3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:26:22.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.380+0000 7f0a34200700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a2c105e30 0x7f0a2c1a3f30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0a24007d70 tx=0x7f0a2400a340 comp rx=0 tx=0).stop 2024-05-27T07:26:22.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.381+0000 7f0a34200700 1 -- 172.21.15.86:0/2543288256 shutdown_connections 2024-05-27T07:26:22.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.381+0000 7f0a34200700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a2c105e30 0x7f0a2c1a3f30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:22.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.381+0000 7f0a34200700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a2c104360 0x7f0a2c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:22.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.381+0000 7f0a34200700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0a140b2880 0x7f0a140b4d90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:22.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.381+0000 7f0a34200700 1 --2- 172.21.15.86:0/2543288256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a2c105490 0x7f0a2c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:22.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.381+0000 7f0a34200700 1 -- 172.21.15.86:0/2543288256 >> 172.21.15.86:0/2543288256 conn(0x7f0a2c0ff9c0 msgr2=0x7f0a2c076af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:26:22.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.381+0000 7f0a34200700 1 -- 172.21.15.86:0/2543288256 shutdown_connections 2024-05-27T07:26:22.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-05-27T07:26:22.381+0000 7f0a34200700 1 -- 172.21.15.86:0/2543288256 wait complete. 2024-05-27T07:26:23.139 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:22 smithi086 ceph-mon[34687]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:23.140 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:22 smithi086 ceph-mon[34687]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T07:26:23.140 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:22 smithi086 ceph-mon[34687]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T07:26:23.140 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:22 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:23.140 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:22 smithi086 ceph-mon[34687]: from='osd.10 [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:26:23.140 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:22 smithi086 ceph-mon[34687]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:26:23.140 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:22 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:23.140 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:22 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:23.140 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:22 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:23.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:22 smithi149 ceph-mon[35408]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:23.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:22 smithi149 ceph-mon[35408]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T07:26:23.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:22 smithi149 ceph-mon[35408]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T07:26:23.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:22 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:23.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:22 smithi149 ceph-mon[35408]: from='osd.10 [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:26:23.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:22 smithi149 ceph-mon[35408]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:26:23.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:22 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:23.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:22 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:23.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:22 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:23.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:22 smithi063 ceph-mon[31613]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:23.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:22 smithi063 ceph-mon[31613]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T07:26:23.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:22 smithi063 ceph-mon[31613]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T07:26:23.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:22 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:23.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:22 smithi063 ceph-mon[31613]: from='osd.10 [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:26:23.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:22 smithi063 ceph-mon[31613]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-05-27T07:26:23.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:22 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:23.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:22 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:23.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:22 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:23.211 DEBUG:teuthology.orchestra.run.smithi086:osd.10> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.10.service 2024-05-27T07:26:23.217 INFO:tasks.cephadm:Deploying osd.11 on smithi149 with /dev/vg_nvme/lv_1... 2024-05-27T07:26:23.217 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T07:26:23.666 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:26:24.069 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:23 smithi149 ceph-mon[35408]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:26:24.070 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:23 smithi149 ceph-mon[35408]: osdmap e65: 11 total, 10 up, 11 in 2024-05-27T07:26:24.070 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:24.070 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:24.070 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:23 smithi149 ceph-mon[35408]: osd.10 [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] boot 2024-05-27T07:26:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:23 smithi086 ceph-mon[34687]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:26:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:23 smithi086 ceph-mon[34687]: osdmap e65: 11 total, 10 up, 11 in 2024-05-27T07:26:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:23 smithi086 ceph-mon[34687]: osd.10 [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] boot 2024-05-27T07:26:24.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:23 smithi063 ceph-mon[31613]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-05-27T07:26:24.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:23 smithi063 ceph-mon[31613]: osdmap e65: 11 total, 10 up, 11 in 2024-05-27T07:26:24.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:24.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:24.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:23 smithi063 ceph-mon[31613]: osd.10 [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] boot 2024-05-27T07:26:24.857 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:24 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:26:24.857 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:24 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:26:24.857 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:24 smithi149 ceph-mon[35408]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:24.857 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:24 smithi149 ceph-mon[35408]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T07:26:24.858 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:24.858 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:24 smithi149 ceph-mon[35408]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T07:26:24.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:24 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:26:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:24 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:26:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:24 smithi086 ceph-mon[34687]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:24 smithi086 ceph-mon[34687]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T07:26:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:24 smithi086 ceph-mon[34687]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T07:26:25.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:24 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:26:25.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:24 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:26:25.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:24 smithi063 ceph-mon[31613]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:26:25.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:24 smithi063 ceph-mon[31613]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T07:26:25.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:26:25.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:24 smithi063 ceph-mon[31613]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T07:26:26.572 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T07:26:26.573 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T07:26:26.573 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-05-27T07:26:26.573 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-05-27T07:26:26.573 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0339958 s, 308 MB/s 2024-05-27T07:26:26.573 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-05-27T07:26:27.084 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:26 smithi149 ceph-mon[35408]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:27.085 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:26 smithi149 ceph-mon[35408]: osdmap e68: 11 total, 11 up, 11 in 2024-05-27T07:26:27.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:26 smithi086 ceph-mon[34687]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:27.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:26 smithi086 ceph-mon[34687]: osdmap e68: 11 total, 11 up, 11 in 2024-05-27T07:26:27.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:26 smithi063 ceph-mon[31613]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:27.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:26 smithi063 ceph-mon[31613]: osdmap e68: 11 total, 11 up, 11 in 2024-05-27T07:26:27.496 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-05-27T07:26:27.928 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:26:29.091 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-27T07:26:29.091 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi086 2024-05-27T07:26:29.091 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.091 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.091 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.092 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.092 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.092 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.092 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi086 to 4272M 2024-05-27T07:26:29.092 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.092 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:29.092 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:26:29.092 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-27T07:26:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi086 2024-05-27T07:26:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi086 to 4272M 2024-05-27T07:26:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:26:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-27T07:26:29.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi086 2024-05-27T07:26:29.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:26:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi086 to 4272M 2024-05-27T07:26:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:29.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:26:29.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:30.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.193+0000 7fce8bc00700 1 -- 172.21.15.149:0/3515213472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce84104e20 msgr2=0x7fce84107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:26:30.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.193+0000 7fce8bc00700 1 --2- 172.21.15.149:0/3515213472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce84104e20 0x7fce84107240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fce74009990 tx=0x7fce74009ca0 comp rx=0 tx=0).stop 2024-05-27T07:26:30.183 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.193+0000 7fce8bc00700 1 -- 172.21.15.149:0/3515213472 shutdown_connections 2024-05-27T07:26:30.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.193+0000 7fce8bc00700 1 --2- 172.21.15.149:0/3515213472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8410a0e0 0x7fce8410c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:30.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.193+0000 7fce8bc00700 1 --2- 172.21.15.149:0/3515213472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce84104e20 0x7fce84107240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:30.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.193+0000 7fce8bc00700 1 --2- 172.21.15.149:0/3515213472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce84107780 0x7fce84109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:30.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.193+0000 7fce8bc00700 1 -- 172.21.15.149:0/3515213472 >> 172.21.15.149:0/3515213472 conn(0x7fce840fe860 msgr2=0x7fce84100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:26:30.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.193+0000 7fce8bc00700 1 -- 172.21.15.149:0/3515213472 shutdown_connections 2024-05-27T07:26:30.184 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.193+0000 7fce8bc00700 1 -- 172.21.15.149:0/3515213472 wait complete. 2024-05-27T07:26:30.185 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.194+0000 7fce8bc00700 1 Processor -- start 2024-05-27T07:26:30.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce8bc00700 1 -- start start 2024-05-27T07:26:30.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce8bc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce84104e20 0x7fce8419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:26:30.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce8bc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce84107780 0x7fce8419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:26:30.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce8bc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce8410a0e0 0x7fce841a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:26:30.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce8bc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fce8410fc60 con 0x7fce84104e20 2024-05-27T07:26:30.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce8bc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce8410fae0 con 0x7fce84107780 2024-05-27T07:26:30.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce8bc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fce8410fde0 con 0x7fce8410a0e0 2024-05-27T07:26:30.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce89c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce8410a0e0 0x7fce841a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:26:30.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce89c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce8410a0e0 0x7fce841a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:41964/0 (socket says 172.21.15.149:41964) 2024-05-27T07:26:30.186 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce89c00700 1 -- 172.21.15.149:0/3781292611 learned_addr learned my addr 172.21.15.149:0/3781292611 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:26:30.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.195+0000 7fce89200700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce84104e20 0x7fce8419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:26:30.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.196+0000 7fce7be00700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce84107780 0x7fce8419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:26:30.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.196+0000 7fce89c00700 1 -- 172.21.15.149:0/3781292611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce84107780 msgr2=0x7fce8419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:26:30.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.196+0000 7fce89c00700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce84107780 0x7fce8419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:30.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.196+0000 7fce89c00700 1 -- 172.21.15.149:0/3781292611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce84104e20 msgr2=0x7fce8419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:26:30.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.196+0000 7fce89c00700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce84104e20 0x7fce8419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:30.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.196+0000 7fce89c00700 1 -- 172.21.15.149:0/3781292611 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce841a45a0 con 0x7fce8410a0e0 2024-05-27T07:26:30.187 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.196+0000 7fce89c00700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce8410a0e0 0x7fce841a3f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce7c007ee0 tx=0x7fce7c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:26:30.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.197+0000 7fce82000700 1 -- 172.21.15.149:0/3781292611 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c00dbd0 con 0x7fce8410a0e0 2024-05-27T07:26:30.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.197+0000 7fce82000700 1 -- 172.21.15.149:0/3781292611 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce7c00dd70 con 0x7fce8410a0e0 2024-05-27T07:26:30.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.197+0000 7fce8bc00700 1 -- 172.21.15.149:0/3781292611 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce841a4890 con 0x7fce8410a0e0 2024-05-27T07:26:30.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.197+0000 7fce82000700 1 -- 172.21.15.149:0/3781292611 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0058d0 con 0x7fce8410a0e0 2024-05-27T07:26:30.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.197+0000 7fce8bc00700 1 -- 172.21.15.149:0/3781292611 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce841a4dd0 con 0x7fce8410a0e0 2024-05-27T07:26:30.188 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.198+0000 7fce8bc00700 1 -- 172.21.15.149:0/3781292611 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce840512b0 con 0x7fce8410a0e0 2024-05-27T07:26:30.190 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.200+0000 7fce82000700 1 -- 172.21.15.149:0/3781292611 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fce7c01a020 con 0x7fce8410a0e0 2024-05-27T07:26:30.191 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.202+0000 7fce82000700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce680b2900 0x7fce680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:26:30.191 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.202+0000 7fce82000700 1 -- 172.21.15.149:0/3781292611 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7fce7c0e1a50 con 0x7fce8410a0e0 2024-05-27T07:26:30.192 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.203+0000 7fce89200700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce680b2900 0x7fce680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:26:30.193 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.204+0000 7fce89200700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce680b2900 0x7fce680b4e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fce7400b5c0 tx=0x7fce7401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:26:30.194 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.205+0000 7fce82000700 1 -- 172.21.15.149:0/3781292611 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce7c010070 con 0x7fce8410a0e0 2024-05-27T07:26:30.437 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:30.447+0000 7fce8bc00700 1 -- 172.21.15.149:0/3781292611 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fce84103460 con 0x7fce680b2900 2024-05-27T07:26:31.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:30 smithi086 ceph-mon[34687]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:26:31.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:26:31.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:26:31.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:30 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:31.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:30 smithi149 ceph-mon[35408]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:26:31.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:26:31.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:26:31.431 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:30 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:31.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:30 smithi063 ceph-mon[31613]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:26:31.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:26:31.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:26:31.432 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:30 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:31.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:31 smithi086 ceph-mon[34687]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:26:32.175 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:31 smithi149 ceph-mon[35408]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:26:32.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:31 smithi063 ceph-mon[31613]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:26:33.151 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:32 smithi149 ceph-mon[35408]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-27T07:26:33.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:32 smithi086 ceph-mon[34687]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-27T07:26:33.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:32 smithi063 ceph-mon[31613]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-27T07:26:35.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:34 smithi086 ceph-mon[34687]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T07:26:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:34 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/3324334324' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69396a1e-76b9-40c2-a289-392b3f2a5dc3"}]: dispatch 2024-05-27T07:26:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:34 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69396a1e-76b9-40c2-a289-392b3f2a5dc3"}]: dispatch 2024-05-27T07:26:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:34 smithi086 ceph-mon[34687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69396a1e-76b9-40c2-a289-392b3f2a5dc3"}]': finished 2024-05-27T07:26:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:34 smithi086 ceph-mon[34687]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T07:26:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:34 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:26:35.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:34 smithi149 ceph-mon[35408]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T07:26:35.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:34 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/3324334324' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69396a1e-76b9-40c2-a289-392b3f2a5dc3"}]: dispatch 2024-05-27T07:26:35.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:34 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69396a1e-76b9-40c2-a289-392b3f2a5dc3"}]: dispatch 2024-05-27T07:26:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:34 smithi149 ceph-mon[35408]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69396a1e-76b9-40c2-a289-392b3f2a5dc3"}]': finished 2024-05-27T07:26:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:34 smithi149 ceph-mon[35408]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T07:26:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:34 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:26:35.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:34 smithi063 ceph-mon[31613]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-05-27T07:26:35.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:34 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/3324334324' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69396a1e-76b9-40c2-a289-392b3f2a5dc3"}]: dispatch 2024-05-27T07:26:35.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:34 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69396a1e-76b9-40c2-a289-392b3f2a5dc3"}]: dispatch 2024-05-27T07:26:35.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:34 smithi063 ceph-mon[31613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69396a1e-76b9-40c2-a289-392b3f2a5dc3"}]': finished 2024-05-27T07:26:35.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:34 smithi063 ceph-mon[31613]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T07:26:35.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:34 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:26:36.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:35 smithi086 ceph-mon[34687]: from='client.? 172.21.15.149:0/452098671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:26:36.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:35 smithi149 ceph-mon[35408]: from='client.? 172.21.15.149:0/452098671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:26:36.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:35 smithi063 ceph-mon[31613]: from='client.? 172.21.15.149:0/452098671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:26:37.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:36 smithi086 ceph-mon[34687]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:26:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:36 smithi149 ceph-mon[35408]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:26:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:36 smithi063 ceph-mon[31613]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:26:39.090 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:38 smithi149 ceph-mon[35408]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:38 smithi086 ceph-mon[34687]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:38 smithi063 ceph-mon[31613]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:41.108 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:40 smithi149 ceph-mon[35408]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:40 smithi086 ceph-mon[34687]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:41.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:40 smithi063 ceph-mon[31613]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:42.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:42 smithi149 ceph-mon[35408]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:43.128 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:42 smithi063 ceph-mon[31613]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:42 smithi086 ceph-mon[34687]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:44 smithi086 ceph-mon[34687]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T07:26:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:44 smithi149 ceph-mon[35408]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T07:26:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:45.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:44 smithi063 ceph-mon[31613]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:45.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T07:26:45.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:46.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:45 smithi086 ceph-mon[34687]: Deploying daemon osd.11 on smithi149 2024-05-27T07:26:46.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:45 smithi149 ceph-mon[35408]: Deploying daemon osd.11 on smithi149 2024-05-27T07:26:46.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:45 smithi063 ceph-mon[31613]: Deploying daemon osd.11 on smithi149 2024-05-27T07:26:47.040 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:46 smithi149 ceph-mon[35408]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:47.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:46 smithi086 ceph-mon[34687]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:47.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:46 smithi063 ceph-mon[31613]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:49.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:48 smithi086 ceph-mon[34687]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:49.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:48 smithi149 ceph-mon[35408]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:49.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:48 smithi063 ceph-mon[31613]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:51.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:50 smithi086 ceph-mon[34687]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:51.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:50 smithi149 ceph-mon[35408]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:51.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:50 smithi063 ceph-mon[31613]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:53.059 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:52 smithi149 ceph-mon[35408]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:53.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:52 smithi086 ceph-mon[34687]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:53.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:52 smithi063 ceph-mon[31613]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:54.594 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:54.594 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:54.594 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:54.595 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:54 smithi149 ceph-mon[35408]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:54.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:54.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:54.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:54.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:54 smithi086 ceph-mon[34687]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:54.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:54.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:54.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:54.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:54 smithi063 ceph-mon[31613]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:57.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:56 smithi086 ceph-mon[34687]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:57.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:56 smithi149 ceph-mon[35408]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:57.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:56 smithi063 ceph-mon[31613]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:58.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:57 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:58.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:57 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:58.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:57 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:58.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:57 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:26:58.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:57 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:58.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:57 smithi086 ceph-mon[34687]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T07:26:58.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:57 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:58.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:57 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:58.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:57 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:58.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:57 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:26:58.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:57 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:58.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:57 smithi149 ceph-mon[35408]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T07:26:58.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:57 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:58.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:57 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:58.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:57 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:26:58.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:57 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:26:58.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:57 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:58.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:57 smithi063 ceph-mon[31613]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T07:26:58.711 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.720+0000 7fce82000700 1 -- 172.21.15.149:0/3781292611 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fce84103460 con 0x7fce680b2900 2024-05-27T07:26:58.711 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 11 on host 'smithi149' 2024-05-27T07:26:58.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.724+0000 7fce8bc00700 1 -- 172.21.15.149:0/3781292611 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce680b2900 msgr2=0x7fce680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:26:58.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.724+0000 7fce8bc00700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce680b2900 0x7fce680b4e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fce7400b5c0 tx=0x7fce7401a040 comp rx=0 tx=0).stop 2024-05-27T07:26:58.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.724+0000 7fce8bc00700 1 -- 172.21.15.149:0/3781292611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce8410a0e0 msgr2=0x7fce841a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:26:58.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.724+0000 7fce8bc00700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce8410a0e0 0x7fce841a3f20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce7c007ee0 tx=0x7fce7c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:26:58.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.724+0000 7fce8bc00700 1 -- 172.21.15.149:0/3781292611 shutdown_connections 2024-05-27T07:26:58.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.724+0000 7fce8bc00700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce8410a0e0 0x7fce841a3f20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:58.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.724+0000 7fce8bc00700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce680b2900 0x7fce680b4e10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:58.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.724+0000 7fce8bc00700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce84104e20 0x7fce8419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:58.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.724+0000 7fce8bc00700 1 --2- 172.21.15.149:0/3781292611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce84107780 0x7fce8419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:26:58.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.724+0000 7fce8bc00700 1 -- 172.21.15.149:0/3781292611 >> 172.21.15.149:0/3781292611 conn(0x7fce840fe860 msgr2=0x7fce84102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:26:58.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.725+0000 7fce8bc00700 1 -- 172.21.15.149:0/3781292611 shutdown_connections 2024-05-27T07:26:58.716 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:26:58.725+0000 7fce8bc00700 1 -- 172.21.15.149:0/3781292611 wait complete. 2024-05-27T07:26:59.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:58 smithi086 ceph-mon[34687]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:59.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:58 smithi086 ceph-mon[34687]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T07:26:59.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:58 smithi086 ceph-mon[34687]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T07:26:59.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:58 smithi086 ceph-mon[34687]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:26:59.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:26:59.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:59.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:59.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:59.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:58 smithi149 ceph-mon[35408]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:59.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:58 smithi149 ceph-mon[35408]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T07:26:59.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:58 smithi149 ceph-mon[35408]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T07:26:59.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:58 smithi149 ceph-mon[35408]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:26:59.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:26:59.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:59.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:59.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:59.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:58 smithi063 ceph-mon[31613]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:26:59.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:58 smithi063 ceph-mon[31613]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T07:26:59.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:58 smithi063 ceph-mon[31613]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T07:26:59.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:58 smithi063 ceph-mon[31613]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-05-27T07:26:59.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:26:59.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:59.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:26:59.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:26:59.709 DEBUG:teuthology.orchestra.run.smithi149:osd.11> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.11.service 2024-05-27T07:26:59.715 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-27T07:26:59.715 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-05-27T07:26:59.914 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 07:26:59 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T07:26:59.796+0000 7f6cbb000700 -1 osd.11 0 waiting for initial osdmap 2024-05-27T07:26:59.914 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 07:26:59 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T07:26:59.826+0000 7f6cb0e00700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T07:27:00.200 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:27:00.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:59 smithi149 ceph-mon[35408]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:27:00.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:59 smithi149 ceph-mon[35408]: osdmap e71: 12 total, 11 up, 12 in 2024-05-27T07:27:00.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:59 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:00.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:59 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:00.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:26:59 smithi149 ceph-mon[35408]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' 2024-05-27T07:27:00.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:59 smithi063 ceph-mon[31613]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:27:00.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:59 smithi063 ceph-mon[31613]: osdmap e71: 12 total, 11 up, 12 in 2024-05-27T07:27:00.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:59 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:00.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:59 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:00.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:26:59 smithi063 ceph-mon[31613]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' 2024-05-27T07:27:00.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:59 smithi086 ceph-mon[34687]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-05-27T07:27:00.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:59 smithi086 ceph-mon[34687]: osdmap e71: 12 total, 11 up, 12 in 2024-05-27T07:27:00.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:59 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:00.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:59 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:00.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:26:59 smithi086 ceph-mon[34687]: from='osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443]' entity='osd.11' 2024-05-27T07:27:00.985 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:00 smithi063 ceph-mon[31613]: purged_snaps scrub starts 2024-05-27T07:27:00.986 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:00 smithi063 ceph-mon[31613]: purged_snaps scrub ok 2024-05-27T07:27:00.986 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:00 smithi063 ceph-mon[31613]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:27:00.986 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:00.986 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:00 smithi063 ceph-mon[31613]: osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] boot 2024-05-27T07:27:01.099 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:00 smithi149 ceph-mon[35408]: purged_snaps scrub starts 2024-05-27T07:27:01.099 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:00 smithi149 ceph-mon[35408]: purged_snaps scrub ok 2024-05-27T07:27:01.099 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:00 smithi149 ceph-mon[35408]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:27:01.099 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:01.099 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:00 smithi149 ceph-mon[35408]: osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] boot 2024-05-27T07:27:01.099 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:00 smithi149 ceph-mon[35408]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T07:27:01.099 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:00 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:01.293 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:00 smithi063 ceph-mon[31613]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T07:27:01.293 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:00 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:01.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:00 smithi086 ceph-mon[34687]: purged_snaps scrub starts 2024-05-27T07:27:01.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:00 smithi086 ceph-mon[34687]: purged_snaps scrub ok 2024-05-27T07:27:01.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:00 smithi086 ceph-mon[34687]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:27:01.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:01.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:00 smithi086 ceph-mon[34687]: osd.11 [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] boot 2024-05-27T07:27:01.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:00 smithi086 ceph-mon[34687]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T07:27:01.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:00 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:27:02.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.623+0000 7f2c67200700 1 -- 172.21.15.63:0/539355763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c68107e50 msgr2=0x7f2c6810e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:02.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.623+0000 7f2c67200700 1 --2- 172.21.15.63:0/539355763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c68107e50 0x7f2c6810e700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2c6000b0a0 tx=0x7f2c6000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.624+0000 7f2c67200700 1 -- 172.21.15.63:0/539355763 shutdown_connections 2024-05-27T07:27:02.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.624+0000 7f2c67200700 1 --2- 172.21.15.63:0/539355763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c68107e50 0x7f2c6810e700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.624+0000 7f2c67200700 1 --2- 172.21.15.63:0/539355763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c68106380 0x7f2c68106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.624+0000 7f2c67200700 1 --2- 172.21.15.63:0/539355763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c681074b0 0x7f2c681078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.624+0000 7f2c67200700 1 -- 172.21.15.63:0/539355763 >> 172.21.15.63:0/539355763 conn(0x7f2c68101a00 msgr2=0x7f2c68103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:02.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.624+0000 7f2c67200700 1 -- 172.21.15.63:0/539355763 shutdown_connections 2024-05-27T07:27:02.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.624+0000 7f2c67200700 1 -- 172.21.15.63:0/539355763 wait complete. 2024-05-27T07:27:02.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.625+0000 7f2c67200700 1 Processor -- start 2024-05-27T07:27:02.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.625+0000 7f2c67200700 1 -- start start 2024-05-27T07:27:02.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c67200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c68106380 0x7f2c6819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:02.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c67200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c681074b0 0x7f2c6819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:02.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c67200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c68107e50 0x7f2c681a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:02.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c67200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68111ca0 con 0x7f2c68106380 2024-05-27T07:27:02.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c67200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68111b20 con 0x7f2c68107e50 2024-05-27T07:27:02.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c67200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68111e20 con 0x7f2c681074b0 2024-05-27T07:27:02.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c65e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c68106380 0x7f2c6819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:02.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c65e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c68106380 0x7f2c6819c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35172/0 (socket says 172.21.15.63:35172) 2024-05-27T07:27:02.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c65e00700 1 -- 172.21.15.63:0/1220097863 learned_addr learned my addr 172.21.15.63:0/1220097863 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:27:02.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c65400700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c681074b0 0x7f2c6819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:02.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.626+0000 7f2c66800700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c68107e50 0x7f2c681a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:02.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.627+0000 7f2c65e00700 1 -- 172.21.15.63:0/1220097863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c681074b0 msgr2=0x7f2c6819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:02.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.627+0000 7f2c65e00700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c681074b0 0x7f2c6819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.627+0000 7f2c65e00700 1 -- 172.21.15.63:0/1220097863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c68107e50 msgr2=0x7f2c681a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:02.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.627+0000 7f2c65e00700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c68107e50 0x7f2c681a3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.627+0000 7f2c65e00700 1 -- 172.21.15.63:0/1220097863 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c681a4340 con 0x7f2c68106380 2024-05-27T07:27:02.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.627+0000 7f2c65e00700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c68106380 0x7f2c6819c7e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800d950 tx=0x7f2c5800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:02.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.627+0000 7f2c5ea00700 1 -- 172.21.15.63:0/1220097863 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58017070 con 0x7f2c68106380 2024-05-27T07:27:02.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.627+0000 7f2c5ea00700 1 -- 172.21.15.63:0/1220097863 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c5800de30 con 0x7f2c68106380 2024-05-27T07:27:02.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.627+0000 7f2c5ea00700 1 -- 172.21.15.63:0/1220097863 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58003aa0 con 0x7f2c68106380 2024-05-27T07:27:02.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.627+0000 7f2c67200700 1 -- 172.21.15.63:0/1220097863 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c681a4630 con 0x7f2c68106380 2024-05-27T07:27:02.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.628+0000 7f2c67200700 1 -- 172.21.15.63:0/1220097863 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c681a4b20 con 0x7f2c68106380 2024-05-27T07:27:02.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.629+0000 7f2c67200700 1 -- 172.21.15.63:0/1220097863 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c680512b0 con 0x7f2c68106380 2024-05-27T07:27:02.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.631+0000 7f2c5ea00700 1 -- 172.21.15.63:0/1220097863 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2c58003c40 con 0x7f2c68106380 2024-05-27T07:27:02.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.633+0000 7f2c5ea00700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c4c0b2900 0x7f2c4c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:02.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.633+0000 7f2c5ea00700 1 -- 172.21.15.63:0/1220097863 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f2c580e20e0 con 0x7f2c68106380 2024-05-27T07:27:02.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.633+0000 7f2c65400700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c4c0b2900 0x7f2c4c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:02.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.634+0000 7f2c65400700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c4c0b2900 0x7f2c4c0b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2c54005950 tx=0x7f2c540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:02.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.636+0000 7f2c5ea00700 1 -- 172.21.15.63:0/1220097863 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c58020030 con 0x7f2c68106380 2024-05-27T07:27:02.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.874+0000 7f2c67200700 1 -- 172.21.15.63:0/1220097863 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2c6802e5e0 con 0x7f2c68106380 2024-05-27T07:27:02.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.876+0000 7f2c5ea00700 1 -- 172.21.15.63:0/1220097863 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f2c580aa280 con 0x7f2c68106380 2024-05-27T07:27:02.865 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:27:02.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.879+0000 7f2c67200700 1 -- 172.21.15.63:0/1220097863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c4c0b2900 msgr2=0x7f2c4c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:02.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.879+0000 7f2c67200700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c4c0b2900 0x7f2c4c0b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2c54005950 tx=0x7f2c540058e0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.879+0000 7f2c67200700 1 -- 172.21.15.63:0/1220097863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c68106380 msgr2=0x7f2c6819c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:02.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.879+0000 7f2c67200700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c68106380 0x7f2c6819c7e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800d950 tx=0x7f2c5800dd10 comp rx=0 tx=0).stop 2024-05-27T07:27:02.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.879+0000 7f2c67200700 1 -- 172.21.15.63:0/1220097863 shutdown_connections 2024-05-27T07:27:02.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.879+0000 7f2c67200700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c68107e50 0x7f2c681a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.879+0000 7f2c67200700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c4c0b2900 0x7f2c4c0b4e10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.879+0000 7f2c67200700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c68106380 0x7f2c6819c7e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.879+0000 7f2c67200700 1 --2- 172.21.15.63:0/1220097863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c681074b0 0x7f2c6819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:02.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.879+0000 7f2c67200700 1 -- 172.21.15.63:0/1220097863 >> 172.21.15.63:0/1220097863 conn(0x7f2c68101a00 msgr2=0x7f2c6810a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:02.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.880+0000 7f2c67200700 1 -- 172.21.15.63:0/1220097863 shutdown_connections 2024-05-27T07:27:02.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:02.880+0000 7f2c67200700 1 -- 172.21.15.63:0/1220097863 wait complete. 2024-05-27T07:27:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:03 smithi063 ceph-mon[31613]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:03 smithi063 ceph-mon[31613]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T07:27:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:03 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1220097863' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T07:27:03.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:03 smithi086 ceph-mon[34687]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:03.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:03 smithi086 ceph-mon[34687]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T07:27:03.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:03 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1220097863' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T07:27:03.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:03 smithi149 ceph-mon[35408]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:03.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:03 smithi149 ceph-mon[35408]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T07:27:03.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:03 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1220097863' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T07:27:03.737 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1716794820,"num_in_osds":12,"osd_in_since":1716794794,"num_remapped_pgs":0} 2024-05-27T07:27:03.738 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:27:04.199 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:27:05.273 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:05.274 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.274 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.274 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.274 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.274 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.274 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.274 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.274 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:05.275 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:27:05.275 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:05.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:05.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:27:05.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:05.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:27:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:27:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:06.355 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:06 smithi149 ceph-mon[35408]: Detected new or changed devices on smithi149 2024-05-27T07:27:06.356 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:06 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi149 to 1454M 2024-05-27T07:27:06.374 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:06 smithi063 ceph-mon[31613]: Detected new or changed devices on smithi149 2024-05-27T07:27:06.374 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:06 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi149 to 1454M 2024-05-27T07:27:06.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:06 smithi086 ceph-mon[34687]: Detected new or changed devices on smithi149 2024-05-27T07:27:06.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:06 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi149 to 1454M 2024-05-27T07:27:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.633+0000 7ff0f8e00700 1 -- 172.21.15.63:0/1436580344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410b100 msgr2=0x7ff0f4117610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.633+0000 7ff0f8e00700 1 --2- 172.21.15.63:0/1436580344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410b100 0x7ff0f4117610 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00b0a0 tx=0x7ff0ec00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.633+0000 7ff0f8e00700 1 -- 172.21.15.63:0/1436580344 shutdown_connections 2024-05-27T07:27:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.633+0000 7ff0f8e00700 1 --2- 172.21.15.63:0/1436580344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410b100 0x7ff0f4117610 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.633+0000 7ff0f8e00700 1 --2- 172.21.15.63:0/1436580344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410f6b0 0x7ff0f410fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.633+0000 7ff0f8e00700 1 --2- 172.21.15.63:0/1436580344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f410a7b0 0x7ff0f410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.633+0000 7ff0f8e00700 1 -- 172.21.15.63:0/1436580344 >> 172.21.15.63:0/1436580344 conn(0x7ff0f4076af0 msgr2=0x7ff0f4076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:06.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.633+0000 7ff0f8e00700 1 -- 172.21.15.63:0/1436580344 shutdown_connections 2024-05-27T07:27:06.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.633+0000 7ff0f8e00700 1 -- 172.21.15.63:0/1436580344 wait complete. 2024-05-27T07:27:06.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.635+0000 7ff0f8e00700 1 Processor -- start 2024-05-27T07:27:06.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.635+0000 7ff0f8e00700 1 -- start start 2024-05-27T07:27:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.635+0000 7ff0f8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410a7b0 0x7ff0f41a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.635+0000 7ff0f8e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410b100 0x7ff0f41a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.635+0000 7ff0f8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f410f6b0 0x7ff0f41a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.635+0000 7ff0f8e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f411a790 con 0x7ff0f410a7b0 2024-05-27T07:27:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.635+0000 7ff0f8e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f411a610 con 0x7ff0f410f6b0 2024-05-27T07:27:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.635+0000 7ff0f8e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f411a910 con 0x7ff0f410b100 2024-05-27T07:27:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410a7b0 0x7ff0f41a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410a7b0 0x7ff0f41a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35198/0 (socket says 172.21.15.63:35198) 2024-05-27T07:27:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f3400700 1 -- 172.21.15.63:0/758258953 learned_addr learned my addr 172.21.15.63:0/758258953 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:27:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f410f6b0 0x7ff0f41a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f2a00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410b100 0x7ff0f41a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f3400700 1 -- 172.21.15.63:0/758258953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410b100 msgr2=0x7ff0f41a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f3400700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410b100 0x7ff0f41a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f3400700 1 -- 172.21.15.63:0/758258953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f410f6b0 msgr2=0x7ff0f41a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f3400700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f410f6b0 0x7ff0f41a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f3400700 1 -- 172.21.15.63:0/758258953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f41a5fb0 con 0x7ff0f410a7b0 2024-05-27T07:27:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.636+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f410f6b0 0x7ff0f41a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:27:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.637+0000 7ff0f3400700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410a7b0 0x7ff0f41a10e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400d950 tx=0x7ff0e400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.637+0000 7ff0ebe00700 1 -- 172.21.15.63:0/758258953 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4016070 con 0x7ff0f410a7b0 2024-05-27T07:27:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.637+0000 7ff0f8e00700 1 -- 172.21.15.63:0/758258953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f41a62a0 con 0x7ff0f410a7b0 2024-05-27T07:27:06.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.637+0000 7ff0ebe00700 1 -- 172.21.15.63:0/758258953 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0e4010460 con 0x7ff0f410a7b0 2024-05-27T07:27:06.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.637+0000 7ff0ebe00700 1 -- 172.21.15.63:0/758258953 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400f620 con 0x7ff0f410a7b0 2024-05-27T07:27:06.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.637+0000 7ff0f8e00700 1 -- 172.21.15.63:0/758258953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f41adbc0 con 0x7ff0f410a7b0 2024-05-27T07:27:06.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.639+0000 7ff0f8e00700 1 -- 172.21.15.63:0/758258953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f40512b0 con 0x7ff0f410a7b0 2024-05-27T07:27:06.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.641+0000 7ff0ebe00700 1 -- 172.21.15.63:0/758258953 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff0e4019070 con 0x7ff0f410a7b0 2024-05-27T07:27:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.643+0000 7ff0ebe00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2880 0x7ff0d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.643+0000 7ff0ebe00700 1 -- 172.21.15.63:0/758258953 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ff0e40e19b0 con 0x7ff0f410a7b0 2024-05-27T07:27:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.643+0000 7ff0f2a00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2880 0x7ff0d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:06.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.644+0000 7ff0f2a00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2880 0x7ff0d80b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0005950 tx=0x7ff0e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:06.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.646+0000 7ff0ebe00700 1 -- 172.21.15.63:0/758258953 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0e40a9ad0 con 0x7ff0f410a7b0 2024-05-27T07:27:06.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.885+0000 7ff0f8e00700 1 -- 172.21.15.63:0/758258953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0f402e5e0 con 0x7ff0f410a7b0 2024-05-27T07:27:06.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.887+0000 7ff0ebe00700 1 -- 172.21.15.63:0/758258953 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18407 (secure 0 0 0) 0x7ff0e40a93d0 con 0x7ff0f410a7b0 2024-05-27T07:27:06.877 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:27:06.877 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":73,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:27:01.995350+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:27:06.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.890+0000 7ff0f8e00700 1 -- 172.21.15.63:0/758258953 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2880 msgr2=0x7ff0d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:06.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.890+0000 7ff0f8e00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2880 0x7ff0d80b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0005950 tx=0x7ff0e00058e0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.890+0000 7ff0f8e00700 1 -- 172.21.15.63:0/758258953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410a7b0 msgr2=0x7ff0f41a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:06.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.890+0000 7ff0f8e00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410a7b0 0x7ff0f41a10e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400d950 tx=0x7ff0e400dd10 comp rx=0 tx=0).stop 2024-05-27T07:27:06.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.890+0000 7ff0f8e00700 1 -- 172.21.15.63:0/758258953 shutdown_connections 2024-05-27T07:27:06.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.890+0000 7ff0f8e00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f410f6b0 0x7ff0f41a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.890+0000 7ff0f8e00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2880 0x7ff0d80b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.890+0000 7ff0f8e00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410a7b0 0x7ff0f41a10e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.890+0000 7ff0f8e00700 1 --2- 172.21.15.63:0/758258953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410b100 0x7ff0f41a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:06.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.890+0000 7ff0f8e00700 1 -- 172.21.15.63:0/758258953 >> 172.21.15.63:0/758258953 conn(0x7ff0f4076af0 msgr2=0x7ff0f4108850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:06.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.891+0000 7ff0f8e00700 1 -- 172.21.15.63:0/758258953 shutdown_connections 2024-05-27T07:27:06.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:06.891+0000 7ff0f8e00700 1 -- 172.21.15.63:0/758258953 wait complete. 2024-05-27T07:27:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:07 smithi063 ceph-mon[31613]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:07 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/758258953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:27:07.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:07 smithi086 ceph-mon[34687]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:07.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:07 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/758258953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:27:07.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:07 smithi149 ceph-mon[35408]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:07.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:07 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/758258953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:27:07.769 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T07:21:25.773038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T07:27:07.769 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-27T07:27:08.237 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:27:08.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:08 smithi063 ceph-mon[31613]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:08.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:08 smithi086 ceph-mon[34687]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:08.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:08 smithi149 ceph-mon[35408]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:10.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.502+0000 7f1d5cc00700 1 -- 172.21.15.63:0/8487201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58106570 msgr2=0x7f1d58106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:10.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.502+0000 7f1d5cc00700 1 --2- 172.21.15.63:0/8487201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58106570 0x7f1d58106980 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009990 tx=0x7f1d44009ca0 comp rx=0 tx=0).stop 2024-05-27T07:27:10.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.502+0000 7f1d5cc00700 1 -- 172.21.15.63:0/8487201 shutdown_connections 2024-05-27T07:27:10.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.502+0000 7f1d5cc00700 1 --2- 172.21.15.63:0/8487201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d58106ec0 0x7f1d58113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:10.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.502+0000 7f1d5cc00700 1 --2- 172.21.15.63:0/8487201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5810c5b0 0x7f1d5810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:10.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.502+0000 7f1d5cc00700 1 --2- 172.21.15.63:0/8487201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58106570 0x7f1d58106980 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:10.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.502+0000 7f1d5cc00700 1 -- 172.21.15.63:0/8487201 >> 172.21.15.63:0/8487201 conn(0x7f1d581020b0 msgr2=0x7f1d58104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:10.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.502+0000 7f1d5cc00700 1 -- 172.21.15.63:0/8487201 shutdown_connections 2024-05-27T07:27:10.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.502+0000 7f1d5cc00700 1 -- 172.21.15.63:0/8487201 wait complete. 2024-05-27T07:27:10.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.503+0000 7f1d5cc00700 1 Processor -- start 2024-05-27T07:27:10.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.504+0000 7f1d5cc00700 1 -- start start 2024-05-27T07:27:10.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.504+0000 7f1d5cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58106570 0x7f1d5819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:10.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.504+0000 7f1d5cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58106ec0 0x7f1d5819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:10.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.504+0000 7f1d5cc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d5810c5b0 0x7f1d581a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:10.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.504+0000 7f1d5cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d58116550 con 0x7f1d58106570 2024-05-27T07:27:10.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.504+0000 7f1d5cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581163d0 con 0x7f1d58106ec0 2024-05-27T07:27:10.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.504+0000 7f1d5cc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1d581166d0 con 0x7f1d5810c5b0 2024-05-27T07:27:10.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.504+0000 7f1d55e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58106570 0x7f1d5819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:10.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.504+0000 7f1d55e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58106570 0x7f1d5819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34982/0 (socket says 172.21.15.63:34982) 2024-05-27T07:27:10.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.504+0000 7f1d55e00700 1 -- 172.21.15.63:0/930995111 learned_addr learned my addr 172.21.15.63:0/930995111 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:27:10.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.505+0000 7f1d55400700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58106ec0 0x7f1d5819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:10.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.505+0000 7f1d56800700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d5810c5b0 0x7f1d581a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:10.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.505+0000 7f1d55e00700 1 -- 172.21.15.63:0/930995111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d5810c5b0 msgr2=0x7f1d581a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:10.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.505+0000 7f1d55e00700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d5810c5b0 0x7f1d581a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:10.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.505+0000 7f1d55e00700 1 -- 172.21.15.63:0/930995111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58106ec0 msgr2=0x7f1d5819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:10.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.505+0000 7f1d55e00700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58106ec0 0x7f1d5819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:10.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.505+0000 7f1d55e00700 1 -- 172.21.15.63:0/930995111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d581a1db0 con 0x7f1d58106570 2024-05-27T07:27:10.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.505+0000 7f1d56800700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d5810c5b0 0x7f1d581a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:27:10.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.505+0000 7f1d55400700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58106ec0 0x7f1d5819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:27:10.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.505+0000 7f1d55e00700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58106570 0x7f1d5819cee0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800d980 tx=0x7f1d4800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:10.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.506+0000 7f1d4ea00700 1 -- 172.21.15.63:0/930995111 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48017070 con 0x7f1d58106570 2024-05-27T07:27:10.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.506+0000 7f1d4ea00700 1 -- 172.21.15.63:0/930995111 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d48010460 con 0x7f1d58106570 2024-05-27T07:27:10.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.506+0000 7f1d5cc00700 1 -- 172.21.15.63:0/930995111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d581a20a0 con 0x7f1d58106570 2024-05-27T07:27:10.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.506+0000 7f1d4ea00700 1 -- 172.21.15.63:0/930995111 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4800de60 con 0x7f1d58106570 2024-05-27T07:27:10.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.506+0000 7f1d5cc00700 1 -- 172.21.15.63:0/930995111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d581a99c0 con 0x7f1d58106570 2024-05-27T07:27:10.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.507+0000 7f1d5cc00700 1 -- 172.21.15.63:0/930995111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d580512b0 con 0x7f1d58106570 2024-05-27T07:27:10.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.510+0000 7f1d4ea00700 1 -- 172.21.15.63:0/930995111 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1d48010610 con 0x7f1d58106570 2024-05-27T07:27:10.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.512+0000 7f1d4ea00700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2900 0x7f1d3c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:10.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.512+0000 7f1d4ea00700 1 -- 172.21.15.63:0/930995111 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f1d480e18a0 con 0x7f1d58106570 2024-05-27T07:27:10.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.512+0000 7f1d55400700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2900 0x7f1d3c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:10.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.513+0000 7f1d55400700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2900 0x7f1d3c0b4e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1d44006010 tx=0x7f1d44005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:10.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.514+0000 7f1d4ea00700 1 -- 172.21.15.63:0/930995111 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d480a9a40 con 0x7f1d58106570 2024-05-27T07:27:10.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.755+0000 7f1d5cc00700 1 -- 172.21.15.63:0/930995111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1d5819e270 con 0x7f1d58106570 2024-05-27T07:27:10.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.756+0000 7f1d4ea00700 1 -- 172.21.15.63:0/930995111 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f1d5819e270 con 0x7f1d58106570 2024-05-27T07:27:10.746 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-05-27T07:27:10.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 -- 172.21.15.63:0/930995111 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2900 msgr2=0x7f1d3c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:10.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2900 0x7f1d3c0b4e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1d44006010 tx=0x7f1d44005960 comp rx=0 tx=0).stop 2024-05-27T07:27:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 -- 172.21.15.63:0/930995111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58106570 msgr2=0x7f1d5819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58106570 0x7f1d5819cee0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800d980 tx=0x7f1d4800dd40 comp rx=0 tx=0).stop 2024-05-27T07:27:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 -- 172.21.15.63:0/930995111 shutdown_connections 2024-05-27T07:27:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d5810c5b0 0x7f1d581a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2900 0x7f1d3c0b4e10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58106570 0x7f1d5819cee0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 --2- 172.21.15.63:0/930995111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58106ec0 0x7f1d5819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:10.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 -- 172.21.15.63:0/930995111 >> 172.21.15.63:0/930995111 conn(0x7f1d581020b0 msgr2=0x7f1d581049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:10.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 -- 172.21.15.63:0/930995111 shutdown_connections 2024-05-27T07:27:10.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:10.760+0000 7f1d5cc00700 1 -- 172.21.15.63:0/930995111 wait complete. 2024-05-27T07:27:11.144 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:10 smithi063 ceph-mon[31613]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:11.145 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:10 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/930995111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:27:11.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:10 smithi086 ceph-mon[34687]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:11.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:10 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/930995111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:27:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:10 smithi149 ceph-mon[35408]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:10 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/930995111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:27:11.568 INFO:tasks.cephadm:Adding mds.a on smithi063 2024-05-27T07:27:11.568 INFO:tasks.cephadm:Adding mds.d on smithi063 2024-05-27T07:27:11.568 INFO:tasks.cephadm:Adding mds.g on smithi063 2024-05-27T07:27:11.568 INFO:tasks.cephadm:Adding mds.j on smithi063 2024-05-27T07:27:11.568 INFO:tasks.cephadm:Adding mds.b on smithi086 2024-05-27T07:27:11.569 INFO:tasks.cephadm:Adding mds.e on smithi086 2024-05-27T07:27:11.569 INFO:tasks.cephadm:Adding mds.h on smithi086 2024-05-27T07:27:11.569 INFO:tasks.cephadm:Adding mds.k on smithi086 2024-05-27T07:27:11.569 INFO:tasks.cephadm:Adding mds.c on smithi149 2024-05-27T07:27:11.569 INFO:tasks.cephadm:Adding mds.f on smithi149 2024-05-27T07:27:11.569 INFO:tasks.cephadm:Adding mds.i on smithi149 2024-05-27T07:27:11.569 INFO:tasks.cephadm:Adding mds.l on smithi149 2024-05-27T07:27:11.569 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l' 2024-05-27T07:27:12.010 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:27:13.048 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:12 smithi149 ceph-mon[35408]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:13.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:12 smithi086 ceph-mon[34687]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:13.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:12 smithi063 ceph-mon[31613]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:14.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.327+0000 7ffa61400700 1 -- 172.21.15.149:0/1865457256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c10ba70 msgr2=0x7ffa5c10be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:14.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.327+0000 7ffa61400700 1 --2- 172.21.15.149:0/1865457256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c10ba70 0x7ffa5c10be80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009990 tx=0x7ffa48009ca0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.328+0000 7ffa61400700 1 -- 172.21.15.149:0/1865457256 shutdown_connections 2024-05-27T07:27:14.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.328+0000 7ffa61400700 1 --2- 172.21.15.149:0/1865457256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5c10c410 0x7ffa5c112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.328+0000 7ffa61400700 1 --2- 172.21.15.149:0/1865457256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c10ab00 0x7ffa5c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.328+0000 7ffa61400700 1 --2- 172.21.15.149:0/1865457256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c10ba70 0x7ffa5c10be80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.328+0000 7ffa61400700 1 -- 172.21.15.149:0/1865457256 >> 172.21.15.149:0/1865457256 conn(0x7ffa5c0799b0 msgr2=0x7ffa5c07be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:14.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.328+0000 7ffa61400700 1 -- 172.21.15.149:0/1865457256 shutdown_connections 2024-05-27T07:27:14.319 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.328+0000 7ffa61400700 1 -- 172.21.15.149:0/1865457256 wait complete. 2024-05-27T07:27:14.320 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.329+0000 7ffa61400700 1 Processor -- start 2024-05-27T07:27:14.320 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.329+0000 7ffa61400700 1 -- start start 2024-05-27T07:27:14.320 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa61400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5c10ab00 0x7ffa5c1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:14.320 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa61400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c10ba70 0x7ffa5c1a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:14.320 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa61400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c10c410 0x7ffa5c1a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:14.320 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa61400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1162b0 con 0x7ffa5c10ba70 2024-05-27T07:27:14.320 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa61400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c116130 con 0x7ffa5c10ab00 2024-05-27T07:27:14.320 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa61400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c116430 con 0x7ffa5c10c410 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa5b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c10c410 0x7ffa5c1a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa5b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c10c410 0x7ffa5c1a83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:42608/0 (socket says 172.21.15.149:42608) 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa5b200700 1 -- 172.21.15.149:0/3016976939 learned_addr learned my addr 172.21.15.149:0/3016976939 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa5a800700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5c10ab00 0x7ffa5c1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.330+0000 7ffa59e00700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c10ba70 0x7ffa5c1a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa5b200700 1 -- 172.21.15.149:0/3016976939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5c10ab00 msgr2=0x7ffa5c1a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa5b200700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5c10ab00 0x7ffa5c1a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa5b200700 1 -- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c10ba70 msgr2=0x7ffa5c1a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa5b200700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c10ba70 0x7ffa5c1a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa5b200700 1 -- 172.21.15.149:0/3016976939 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c1a8a40 con 0x7ffa5c10c410 2024-05-27T07:27:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa59e00700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c10ba70 0x7ffa5c1a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:27:14.322 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa5b200700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c10c410 0x7ffa5c1a83c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400ef30 tx=0x7ffa5400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:14.322 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa53400700 1 -- 172.21.15.149:0/3016976939 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540092e0 con 0x7ffa5c10c410 2024-05-27T07:27:14.322 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa53400700 1 -- 172.21.15.149:0/3016976939 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa54004bb0 con 0x7ffa5c10c410 2024-05-27T07:27:14.322 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa61400700 1 -- 172.21.15.149:0/3016976939 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c1a8cd0 con 0x7ffa5c10c410 2024-05-27T07:27:14.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa53400700 1 -- 172.21.15.149:0/3016976939 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54004080 con 0x7ffa5c10c410 2024-05-27T07:27:14.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.331+0000 7ffa61400700 1 -- 172.21.15.149:0/3016976939 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1a91c0 con 0x7ffa5c10c410 2024-05-27T07:27:14.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.333+0000 7ffa61400700 1 -- 172.21.15.149:0/3016976939 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5c0512b0 con 0x7ffa5c10c410 2024-05-27T07:27:14.324 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.335+0000 7ffa53400700 1 -- 172.21.15.149:0/3016976939 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ffa5401b030 con 0x7ffa5c10c410 2024-05-27T07:27:14.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.337+0000 7ffa53400700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffa400b2880 0x7ffa400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:14.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.337+0000 7ffa53400700 1 -- 172.21.15.149:0/3016976939 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ffa540e19f0 con 0x7ffa5c10c410 2024-05-27T07:27:14.327 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.337+0000 7ffa5a800700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffa400b2880 0x7ffa400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:14.328 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.339+0000 7ffa5a800700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffa400b2880 0x7ffa400b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005950 tx=0x7ffa4c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:14.331 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.341+0000 7ffa53400700 1 -- 172.21.15.149:0/3016976939 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa54022020 con 0x7ffa5c10c410 2024-05-27T07:27:14.576 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.586+0000 7ffa61400700 1 -- 172.21.15.149:0/3016976939 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}) -- 0x7ffa5c063eb0 con 0x7ffa400b2880 2024-05-27T07:27:14.595 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.605+0000 7ffa53400700 1 -- 172.21.15.149:0/3016976939 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ffa5c063eb0 con 0x7ffa400b2880 2024-05-27T07:27:14.595 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mds.all update... 2024-05-27T07:27:14.598 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 -- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffa400b2880 msgr2=0x7ffa400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:14.598 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffa400b2880 0x7ffa400b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005950 tx=0x7ffa4c0058e0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.599 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 -- 172.21.15.149:0/3016976939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c10c410 msgr2=0x7ffa5c1a83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:14.599 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c10c410 0x7ffa5c1a83c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400ef30 tx=0x7ffa5400c450 comp rx=0 tx=0).stop 2024-05-27T07:27:14.599 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 -- 172.21.15.149:0/3016976939 shutdown_connections 2024-05-27T07:27:14.599 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5c10c410 0x7ffa5c1a83c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.599 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5c10ab00 0x7ffa5c1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.599 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffa400b2880 0x7ffa400b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.599 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 --2- 172.21.15.149:0/3016976939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5c10ba70 0x7ffa5c1a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:14.599 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 -- 172.21.15.149:0/3016976939 >> 172.21.15.149:0/3016976939 conn(0x7ffa5c0799b0 msgr2=0x7ffa5c10eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:14.600 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 -- 172.21.15.149:0/3016976939 shutdown_connections 2024-05-27T07:27:14.600 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-27T07:27:14.609+0000 7ffa61400700 1 -- 172.21.15.149:0/3016976939 wait complete. 2024-05-27T07:27:14.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:14 smithi149 ceph-mon[35408]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:14.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:14.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:27:14.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:14.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:27:14.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:14.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:14.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:14.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:14 smithi086 ceph-mon[34687]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:27:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:27:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:15.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:14 smithi063 ceph-mon[31613]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:27:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:15.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:27:15.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:15.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:15.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:15.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:15.390 DEBUG:teuthology.orchestra.run.smithi063:mds.a> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service 2024-05-27T07:27:15.402 DEBUG:teuthology.orchestra.run.smithi063:mds.d> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.d.service 2024-05-27T07:27:15.454 DEBUG:teuthology.orchestra.run.smithi063:mds.g> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.g.service 2024-05-27T07:27:15.502 DEBUG:teuthology.orchestra.run.smithi063:mds.j> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.j.service 2024-05-27T07:27:15.552 DEBUG:teuthology.orchestra.run.smithi086:mds.b> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.b.service 2024-05-27T07:27:15.562 DEBUG:teuthology.orchestra.run.smithi086:mds.e> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.e.service 2024-05-27T07:27:15.609 DEBUG:teuthology.orchestra.run.smithi086:mds.h> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.h.service 2024-05-27T07:27:15.656 DEBUG:teuthology.orchestra.run.smithi086:mds.k> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service 2024-05-27T07:27:15.703 DEBUG:teuthology.orchestra.run.smithi149:mds.c> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service 2024-05-27T07:27:15.710 DEBUG:teuthology.orchestra.run.smithi149:mds.f> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service 2024-05-27T07:27:15.756 DEBUG:teuthology.orchestra.run.smithi149:mds.i> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service 2024-05-27T07:27:15.802 DEBUG:teuthology.orchestra.run.smithi149:mds.l> sudo journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service 2024-05-27T07:27:15.848 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-27T07:27:15.848 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:27:15.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:15 smithi086 ceph-mon[34687]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:27:15.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:15 smithi086 ceph-mon[34687]: Saving service mds.all spec with placement smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-05-27T07:27:15.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:15 smithi086 ceph-mon[34687]: Deploying daemon mds.l on smithi149 2024-05-27T07:27:15.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:15 smithi149 ceph-mon[35408]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:27:15.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:15 smithi149 ceph-mon[35408]: Saving service mds.all spec with placement smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-05-27T07:27:15.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:15 smithi149 ceph-mon[35408]: Deploying daemon mds.l on smithi149 2024-05-27T07:27:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:15 smithi063 ceph-mon[31613]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:27:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:15 smithi063 ceph-mon[31613]: Saving service mds.all spec with placement smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-05-27T07:27:16.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:15 smithi063 ceph-mon[31613]: Deploying daemon mds.l on smithi149 2024-05-27T07:27:16.322 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:27:17.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:16 smithi086 ceph-mon[34687]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:17.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:16 smithi149 ceph-mon[35408]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:17.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:16 smithi063 ceph-mon[31613]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:17.952 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 07:27:17 smithi149 systemd[1]: Starting Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:27:18.294 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 07:27:18 smithi149 podman[65314]: 2024-05-27 07:27:17.925117505 +0000 UTC m=+0.034853954 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:27:18.294 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 07:27:18 smithi149 podman[65314]: 2024-05-27 07:27:18.041777284 +0000 UTC m=+0.151513735 container create b6ae53a70b5f0a8f915b2526ee86fdd5de62bb618851a39809738f7e2da0dc75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-05-27T07:27:18.294 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 07:27:18 smithi149 podman[65314]: 2024-05-27 07:27:18.304143503 +0000 UTC m=+0.413880013 container init b6ae53a70b5f0a8f915b2526ee86fdd5de62bb618851a39809738f7e2da0dc75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True) 2024-05-27T07:27:18.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.582+0000 7f7558e00700 1 -- 172.21.15.63:0/960505389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75541045e0 msgr2=0x7f75541049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:18.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.582+0000 7f7558e00700 1 --2- 172.21.15.63:0/960505389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75541045e0 0x7f75541049f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f75440099c0 tx=0x7f7544009cd0 comp rx=0 tx=0).stop 2024-05-27T07:27:18.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.582+0000 7f7558e00700 1 -- 172.21.15.63:0/960505389 shutdown_connections 2024-05-27T07:27:18.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.582+0000 7f7558e00700 1 --2- 172.21.15.63:0/960505389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7554109860 0x7f75541111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:18.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.582+0000 7f7558e00700 1 --2- 172.21.15.63:0/960505389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75541045e0 0x7f75541049f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:18.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.582+0000 7f7558e00700 1 --2- 172.21.15.63:0/960505389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554108f10 0x7f7554109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:18.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.582+0000 7f7558e00700 1 -- 172.21.15.63:0/960505389 >> 172.21.15.63:0/960505389 conn(0x7f7554100030 msgr2=0x7f7554102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:18.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.582+0000 7f7558e00700 1 -- 172.21.15.63:0/960505389 shutdown_connections 2024-05-27T07:27:18.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.582+0000 7f7558e00700 1 -- 172.21.15.63:0/960505389 wait complete. 2024-05-27T07:27:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.583+0000 7f7558e00700 1 Processor -- start 2024-05-27T07:27:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.584+0000 7f7558e00700 1 -- start start 2024-05-27T07:27:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.584+0000 7f7558e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75541045e0 0x7f755410e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.584+0000 7f7558e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7554108f10 0x7f755410ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.584+0000 7f7558e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554109860 0x7f755410f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.584+0000 7f7558e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7554077780 con 0x7f7554108f10 2024-05-27T07:27:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.584+0000 7f7558e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7554077600 con 0x7f7554109860 2024-05-27T07:27:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.584+0000 7f7558e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7554077900 con 0x7f75541045e0 2024-05-27T07:27:18.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.584+0000 7f7551400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7554108f10 0x7f755410ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:18.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.584+0000 7f7551400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7554108f10 0x7f755410ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35000/0 (socket says 172.21.15.63:35000) 2024-05-27T07:27:18.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.584+0000 7f7551400700 1 -- 172.21.15.63:0/793189201 learned_addr learned my addr 172.21.15.63:0/793189201 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:27:18.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.585+0000 7f7551e00700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75541045e0 0x7f755410e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:18.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.585+0000 7f7552800700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554109860 0x7f755410f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.585+0000 7f7551400700 1 -- 172.21.15.63:0/793189201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75541045e0 msgr2=0x7f755410e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.585+0000 7f7551400700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75541045e0 0x7f755410e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.585+0000 7f7551400700 1 -- 172.21.15.63:0/793189201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554109860 msgr2=0x7f755410f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.585+0000 7f7551400700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554109860 0x7f755410f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.585+0000 7f7551400700 1 -- 172.21.15.63:0/793189201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75541ad5f0 con 0x7f7554108f10 2024-05-27T07:27:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.585+0000 7f7551400700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7554108f10 0x7f755410ec80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f754000e970 tx=0x7f754000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.586+0000 7f754aa00700 1 -- 172.21.15.63:0/793189201 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754000cbd0 con 0x7f7554108f10 2024-05-27T07:27:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.586+0000 7f754aa00700 1 -- 172.21.15.63:0/793189201 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f754000cd70 con 0x7f7554108f10 2024-05-27T07:27:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.586+0000 7f7558e00700 1 -- 172.21.15.63:0/793189201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75541ad8e0 con 0x7f7554108f10 2024-05-27T07:27:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.586+0000 7f754aa00700 1 -- 172.21.15.63:0/793189201 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540005570 con 0x7f7554108f10 2024-05-27T07:27:18.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.586+0000 7f7558e00700 1 -- 172.21.15.63:0/793189201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75541ade70 con 0x7f7554108f10 2024-05-27T07:27:18.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.587+0000 7f7558e00700 1 -- 172.21.15.63:0/793189201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7554105a90 con 0x7f7554108f10 2024-05-27T07:27:18.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.593+0000 7f754aa00700 1 -- 172.21.15.63:0/793189201 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7540004750 con 0x7f7554108f10 2024-05-27T07:27:18.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.595+0000 7f754aa00700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75380b2900 0x7f75380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:18.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.595+0000 7f754aa00700 1 -- 172.21.15.63:0/793189201 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f75400e21c0 con 0x7f7554108f10 2024-05-27T07:27:18.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.595+0000 7f7551e00700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75380b2900 0x7f75380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:18.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.596+0000 7f7551e00700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75380b2900 0x7f75380b4e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7544000c00 tx=0x7f754401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:18.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.596+0000 7f754aa00700 1 -- 172.21.15.63:0/793189201 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7540054df0 con 0x7f7554108f10 2024-05-27T07:27:18.703 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 07:27:18 smithi149 podman[65314]: 2024-05-27 07:27:18.314726333 +0000 UTC m=+0.424462776 container start b6ae53a70b5f0a8f915b2526ee86fdd5de62bb618851a39809738f7e2da0dc75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-27T07:27:18.703 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 07:27:18 smithi149 bash[65314]: b6ae53a70b5f0a8f915b2526ee86fdd5de62bb618851a39809738f7e2da0dc75 2024-05-27T07:27:18.703 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 07:27:18 smithi149 systemd[1]: Started Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:27:18.703 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mds[65333]: -- 172.21.15.149:0/952319510 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557acadffa40 con 0x557acae47600 2024-05-27T07:27:18.703 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 07:27:18 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: starting mds.l at 2024-05-27T07:27:18.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.884+0000 7f7558e00700 1 -- 172.21.15.63:0/793189201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f755410fc20 con 0x7f7554108f10 2024-05-27T07:27:18.874 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:18 smithi063 ceph-mon[31613]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:18.874 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:18 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6832/1182053542,v1:172.21.15.149:6833/1182053542] up:boot 2024-05-27T07:27:18.875 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:18 smithi063 ceph-mon[31613]: fsmap 1 up:standby 2024-05-27T07:27:18.875 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:27:18.875 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:18.875 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:18.875 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:18.875 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:18.875 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:18.875 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.885+0000 7f754aa00700 1 -- 172.21.15.63:0/793189201 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f75400aa2e0 con 0x7f7554108f10 2024-05-27T07:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-27T07:27:18.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 -- 172.21.15.63:0/793189201 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75380b2900 msgr2=0x7f75380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:18.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75380b2900 0x7f75380b4e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7544000c00 tx=0x7f754401a040 comp rx=0 tx=0).stop 2024-05-27T07:27:18.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 -- 172.21.15.63:0/793189201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7554108f10 msgr2=0x7f755410ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:18.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7554108f10 0x7f755410ec80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f754000e970 tx=0x7f754000ec80 comp rx=0 tx=0).stop 2024-05-27T07:27:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 -- 172.21.15.63:0/793189201 shutdown_connections 2024-05-27T07:27:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7554109860 0x7f755410f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75541045e0 0x7f755410e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75380b2900 0x7f75380b4e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 --2- 172.21.15.63:0/793189201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7554108f10 0x7f755410ec80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 -- 172.21.15.63:0/793189201 >> 172.21.15.63:0/793189201 conn(0x7f7554100030 msgr2=0x7f7554102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.889+0000 7f7558e00700 1 -- 172.21.15.63:0/793189201 shutdown_connections 2024-05-27T07:27:18.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:18.890+0000 7f7558e00700 1 -- 172.21.15.63:0/793189201 wait complete. 2024-05-27T07:27:18.880 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 2 2024-05-27T07:27:19.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:18 smithi086 ceph-mon[34687]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:19.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:18 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6832/1182053542,v1:172.21.15.149:6833/1182053542] up:boot 2024-05-27T07:27:19.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:18 smithi086 ceph-mon[34687]: fsmap 1 up:standby 2024-05-27T07:27:19.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:27:19.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:19.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:19.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:19.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:19.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:19.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:19.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mon[35408]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:19.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6832/1182053542,v1:172.21.15.149:6833/1182053542] up:boot 2024-05-27T07:27:19.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mon[35408]: fsmap 1 up:standby 2024-05-27T07:27:19.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:27:19.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:19.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:19.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:19.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:19.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:19.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:19.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:19 smithi086 ceph-mon[34687]: Deploying daemon mds.k on smithi086 2024-05-27T07:27:19.849 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:19 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/793189201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:19.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:19 smithi063 ceph-mon[31613]: Deploying daemon mds.k on smithi086 2024-05-27T07:27:19.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:19 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/793189201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:20.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:19 smithi149 ceph-mon[35408]: Deploying daemon mds.k on smithi086 2024-05-27T07:27:20.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:19 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/793189201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:21.148 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:20 smithi149 ceph-mon[35408]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:21.150 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:20 smithi086 ceph-mon[34687]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:21.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:20 smithi063 ceph-mon[31613]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:21.705 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:27:22.191 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:27:22.198 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:27:21 smithi086 systemd[1]: Starting Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:27:22.584 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:27:22 smithi086 podman[64157]: 2024-05-27 07:27:22.171998256 +0000 UTC m=+0.033995721 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:27:22.585 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:27:22 smithi086 podman[64157]: 2024-05-27 07:27:22.298898294 +0000 UTC m=+0.160895794 container create b6a4ca6736a2da27ceec249db559e39592a377bac3a2b6858a2b8db0e1ddf142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-27T07:27:22.585 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:27:22 smithi086 podman[64157]: 2024-05-27 07:27:22.594096138 +0000 UTC m=+0.456093685 container init b6a4ca6736a2da27ceec249db559e39592a377bac3a2b6858a2b8db0e1ddf142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-27T07:27:22.842 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:27:22 smithi086 podman[64157]: 2024-05-27 07:27:22.603838515 +0000 UTC m=+0.465835986 container start b6a4ca6736a2da27ceec249db559e39592a377bac3a2b6858a2b8db0e1ddf142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T07:27:22.842 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:27:22 smithi086 ceph-mds[64176]: -- 172.21.15.86:0/4199021767 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564cac60fa40 con 0x564cac657a80 2024-05-27T07:27:22.842 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:27:22 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: starting mds.k at 2024-05-27T07:27:22.842 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:27:22 smithi086 bash[64157]: b6a4ca6736a2da27ceec249db559e39592a377bac3a2b6858a2b8db0e1ddf142 2024-05-27T07:27:22.842 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:27:22 smithi086 systemd[1]: Started Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:27:22.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:22 smithi063 ceph-mon[31613]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:22.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:22 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:23.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:22 smithi086 ceph-mon[34687]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:23.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:22 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:23.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:22 smithi149 ceph-mon[35408]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:23.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:22 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:24.099 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:24.099 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:23 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6832/1769339501,v1:172.21.15.86:6833/1769339501] up:boot 2024-05-27T07:27:24.100 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:23 smithi063 ceph-mon[31613]: fsmap 2 up:standby 2024-05-27T07:27:24.100 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:27:24.100 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:24.100 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:24.100 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:24.100 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:23 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:24.100 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:23 smithi063 ceph-mon[31613]: Deploying daemon mds.j on smithi063 2024-05-27T07:27:24.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:23 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6832/1769339501,v1:172.21.15.86:6833/1769339501] up:boot 2024-05-27T07:27:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:23 smithi086 ceph-mon[34687]: fsmap 2 up:standby 2024-05-27T07:27:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:27:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:23 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:24.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:23 smithi086 ceph-mon[34687]: Deploying daemon mds.j on smithi063 2024-05-27T07:27:24.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:24.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:23 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6832/1769339501,v1:172.21.15.86:6833/1769339501] up:boot 2024-05-27T07:27:24.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:23 smithi149 ceph-mon[35408]: fsmap 2 up:standby 2024-05-27T07:27:24.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:27:24.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:24.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:24.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:24.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:23 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:24.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:23 smithi149 ceph-mon[35408]: Deploying daemon mds.j on smithi063 2024-05-27T07:27:24.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.354+0000 7ff87be00700 1 -- 172.21.15.63:0/866167239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff87c10a7b0 msgr2=0x7ff87c10abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.354+0000 7ff87be00700 1 --2- 172.21.15.63:0/866167239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff87c10a7b0 0x7ff87c10abc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff868009990 tx=0x7ff868009ca0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.354+0000 7ff87be00700 1 -- 172.21.15.63:0/866167239 shutdown_connections 2024-05-27T07:27:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.354+0000 7ff87be00700 1 --2- 172.21.15.63:0/866167239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff87c10b100 0x7ff87c117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.354+0000 7ff87be00700 1 --2- 172.21.15.63:0/866167239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff87c10f6b0 0x7ff87c10fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.354+0000 7ff87be00700 1 --2- 172.21.15.63:0/866167239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff87c10a7b0 0x7ff87c10abc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.354+0000 7ff87be00700 1 -- 172.21.15.63:0/866167239 >> 172.21.15.63:0/866167239 conn(0x7ff87c076af0 msgr2=0x7ff87c076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.354+0000 7ff87be00700 1 -- 172.21.15.63:0/866167239 shutdown_connections 2024-05-27T07:27:24.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.354+0000 7ff87be00700 1 -- 172.21.15.63:0/866167239 wait complete. 2024-05-27T07:27:24.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.355+0000 7ff87be00700 1 Processor -- start 2024-05-27T07:27:24.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.355+0000 7ff87be00700 1 -- start start 2024-05-27T07:27:24.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff87c10a7b0 0x7ff87c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:24.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff87c10b100 0x7ff87c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:24.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff87c10f6b0 0x7ff87c1a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:24.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c11a7a0 con 0x7ff87c10a7b0 2024-05-27T07:27:24.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c11a620 con 0x7ff87c10b100 2024-05-27T07:27:24.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c11a920 con 0x7ff87c10f6b0 2024-05-27T07:27:24.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff87c10a7b0 0x7ff87c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:24.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff87c10a7b0 0x7ff87c1a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59788/0 (socket says 172.21.15.63:59788) 2024-05-27T07:27:24.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87aa00700 1 -- 172.21.15.63:0/307786230 learned_addr learned my addr 172.21.15.63:0/307786230 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:27:24.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87a000700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff87c10b100 0x7ff87c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:24.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.356+0000 7ff87b400700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff87c10f6b0 0x7ff87c1a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:24.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.357+0000 7ff87aa00700 1 -- 172.21.15.63:0/307786230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff87c10f6b0 msgr2=0x7ff87c1a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:24.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.357+0000 7ff87aa00700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff87c10f6b0 0x7ff87c1a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.357+0000 7ff87aa00700 1 -- 172.21.15.63:0/307786230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff87c10b100 msgr2=0x7ff87c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:24.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.357+0000 7ff87aa00700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff87c10b100 0x7ff87c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.357+0000 7ff87aa00700 1 -- 172.21.15.63:0/307786230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c1a6080 con 0x7ff87c10a7b0 2024-05-27T07:27:24.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.357+0000 7ff87aa00700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff87c10a7b0 0x7ff87c1a11b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00c920 tx=0x7ff86c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:24.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.357+0000 7ff873400700 1 -- 172.21.15.63:0/307786230 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c007b00 con 0x7ff87c10a7b0 2024-05-27T07:27:24.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.357+0000 7ff87be00700 1 -- 172.21.15.63:0/307786230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87c1a6370 con 0x7ff87c10a7b0 2024-05-27T07:27:24.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.357+0000 7ff873400700 1 -- 172.21.15.63:0/307786230 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff86c004cc0 con 0x7ff87c10a7b0 2024-05-27T07:27:24.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.358+0000 7ff873400700 1 -- 172.21.15.63:0/307786230 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0054f0 con 0x7ff87c10a7b0 2024-05-27T07:27:24.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.358+0000 7ff87be00700 1 -- 172.21.15.63:0/307786230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87c1adc90 con 0x7ff87c10a7b0 2024-05-27T07:27:24.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.359+0000 7ff87be00700 1 -- 172.21.15.63:0/307786230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87c0512b0 con 0x7ff87c10a7b0 2024-05-27T07:27:24.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.364+0000 7ff873400700 1 -- 172.21.15.63:0/307786230 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff86c007ca0 con 0x7ff87c10a7b0 2024-05-27T07:27:24.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.365+0000 7ff873400700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8600b2880 0x7ff8600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:24.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.365+0000 7ff873400700 1 -- 172.21.15.63:0/307786230 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ff86c0e1dc0 con 0x7ff87c10a7b0 2024-05-27T07:27:24.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.365+0000 7ff87a000700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8600b2880 0x7ff8600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:24.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.366+0000 7ff87a000700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8600b2880 0x7ff8600b4d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff86800b5c0 tx=0x7ff868005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:24.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.368+0000 7ff873400700 1 -- 172.21.15.63:0/307786230 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff86c014030 con 0x7ff87c10a7b0 2024-05-27T07:27:24.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.662+0000 7ff87be00700 1 -- 172.21.15.63:0/307786230 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff87c1a2510 con 0x7ff87c10a7b0 2024-05-27T07:27:24.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.664+0000 7ff873400700 1 -- 172.21.15.63:0/307786230 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7ff86c0a9ee0 con 0x7ff87c10a7b0 2024-05-27T07:27:24.655 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:27:24.655 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34430,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T07:27:24.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.667+0000 7ff87be00700 1 -- 172.21.15.63:0/307786230 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8600b2880 msgr2=0x7ff8600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:24.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.667+0000 7ff87be00700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8600b2880 0x7ff8600b4d90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff86800b5c0 tx=0x7ff868005fb0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.667+0000 7ff87be00700 1 -- 172.21.15.63:0/307786230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff87c10a7b0 msgr2=0x7ff87c1a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:24.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.667+0000 7ff87be00700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff87c10a7b0 0x7ff87c1a11b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00c920 tx=0x7ff86c00cce0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.668+0000 7ff87be00700 1 -- 172.21.15.63:0/307786230 shutdown_connections 2024-05-27T07:27:24.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.668+0000 7ff87be00700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff87c10f6b0 0x7ff87c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.668+0000 7ff87be00700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8600b2880 0x7ff8600b4d90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.668+0000 7ff87be00700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff87c10a7b0 0x7ff87c1a11b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.668+0000 7ff87be00700 1 --2- 172.21.15.63:0/307786230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff87c10b100 0x7ff87c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:24.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.668+0000 7ff87be00700 1 -- 172.21.15.63:0/307786230 >> 172.21.15.63:0/307786230 conn(0x7ff87c076af0 msgr2=0x7ff87c108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:24.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.668+0000 7ff87be00700 1 -- 172.21.15.63:0/307786230 shutdown_connections 2024-05-27T07:27:24.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:24.668+0000 7ff87be00700 1 -- 172.21.15.63:0/307786230 wait complete. 2024-05-27T07:27:24.660 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 3 2024-05-27T07:27:24.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:24 smithi063 ceph-mon[31613]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:24.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:24 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/307786230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:24 smithi086 ceph-mon[34687]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:25.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:24 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/307786230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:25.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:24 smithi149 ceph-mon[35408]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:25.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:24 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/307786230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:26.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:26 smithi063 ceph-mon[31613]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:27.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:26 smithi086 ceph-mon[34687]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:27.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:26 smithi149 ceph-mon[35408]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:27.456 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 07:27:27 smithi063 podman[72022]: 2024-05-27 07:27:27.113213938 +0000 UTC m=+0.175114307 container create 10e29a0321480d0b5ba16156ce20778194001fcf5f9f24e09137cece40c3c973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240521) 2024-05-27T07:27:27.955 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 07:27:27 smithi063 podman[72022]: 2024-05-27 07:27:27.501694773 +0000 UTC m=+0.563595172 container init 10e29a0321480d0b5ba16156ce20778194001fcf5f9f24e09137cece40c3c973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-j, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-27T07:27:27.955 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 07:27:27 smithi063 podman[72022]: 2024-05-27 07:27:27.51259532 +0000 UTC m=+0.574495646 container start 10e29a0321480d0b5ba16156ce20778194001fcf5f9f24e09137cece40c3c973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-j, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:27:27.955 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 07:27:27 smithi063 ceph-mds[72041]: -- 172.21.15.63:0/498464142 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b70b8ba40 con 0x564b70bd3600 2024-05-27T07:27:27.956 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 07:27:27 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-j[72037]: starting mds.j at 2024-05-27T07:27:27.956 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 07:27:27 smithi063 bash[72022]: 10e29a0321480d0b5ba16156ce20778194001fcf5f9f24e09137cece40c3c973 2024-05-27T07:27:27.956 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 07:27:27 smithi063 systemd[1]: Started Ceph mds.j for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:27:28.278 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:27:28.733 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:27:29.023 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:29.024 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:29.024 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:29.024 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:29.024 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:29.024 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:29.024 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:29.024 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: Deploying daemon mds.d on smithi063 2024-05-27T07:27:29.024 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.63:6834/134276953,v1:172.21.15.63:6835/134276953] up:boot 2024-05-27T07:27:29.024 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: fsmap 3 up:standby 2024-05-27T07:27:29.024 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:28 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:27:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:29.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: Deploying daemon mds.d on smithi063 2024-05-27T07:27:29.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.63:6834/134276953,v1:172.21.15.63:6835/134276953] up:boot 2024-05-27T07:27:29.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: fsmap 3 up:standby 2024-05-27T07:27:29.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:28 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:27:29.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:29.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:29.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:29.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:29.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:29.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:29.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:29.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: Deploying daemon mds.d on smithi063 2024-05-27T07:27:29.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.63:6834/134276953,v1:172.21.15.63:6835/134276953] up:boot 2024-05-27T07:27:29.205 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: fsmap 3 up:standby 2024-05-27T07:27:29.205 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:28 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:27:30.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:30 smithi149 ceph-mon[35408]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:31.167 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:30 smithi063 ceph-mon[31613]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:31.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:30 smithi086 ceph-mon[34687]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:32.250 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 27 07:27:32 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-d[72455]: starting mds.d at 2024-05-27T07:27:32.662 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 27 07:27:32 smithi063 bash[72422]: e7462a8dbccd7616158b9942de2ed2b038a319e1d75dcd35998c2eba4240557c 2024-05-27T07:27:32.663 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 27 07:27:32 smithi063 systemd[1]: Started Ceph mds.d for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:27:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.716+0000 7f9c29400700 1 -- 172.21.15.63:0/2738259045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c24107de0 msgr2=0x7f9c241081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.716+0000 7f9c29400700 1 --2- 172.21.15.63:0/2738259045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c24107de0 0x7f9c241081f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c100099c0 tx=0x7f9c10009cd0 comp rx=0 tx=0).stop 2024-05-27T07:27:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.716+0000 7f9c29400700 1 -- 172.21.15.63:0/2738259045 shutdown_connections 2024-05-27T07:27:32.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.716+0000 7f9c29400700 1 --2- 172.21.15.63:0/2738259045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c24108730 0x7f9c24110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.716+0000 7f9c29400700 1 --2- 172.21.15.63:0/2738259045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c241034b0 0x7f9c241038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.716+0000 7f9c29400700 1 --2- 172.21.15.63:0/2738259045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c24107de0 0x7f9c241081f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.716+0000 7f9c29400700 1 -- 172.21.15.63:0/2738259045 >> 172.21.15.63:0/2738259045 conn(0x7f9c240fedb0 msgr2=0x7f9c241011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.716+0000 7f9c29400700 1 -- 172.21.15.63:0/2738259045 shutdown_connections 2024-05-27T07:27:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.716+0000 7f9c29400700 1 -- 172.21.15.63:0/2738259045 wait complete. 2024-05-27T07:27:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.717+0000 7f9c29400700 1 Processor -- start 2024-05-27T07:27:32.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c29400700 1 -- start start 2024-05-27T07:27:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c29400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c241034b0 0x7f9c2419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c29400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c24107de0 0x7f9c2419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c29400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c24108730 0x7f9c2419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c29400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c24113240 con 0x7f9c24108730 2024-05-27T07:27:32.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c29400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c241130c0 con 0x7f9c24107de0 2024-05-27T07:27:32.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c29400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9c241133c0 con 0x7f9c241034b0 2024-05-27T07:27:32.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c23200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c24108730 0x7f9c2419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:32.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c23200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c24108730 0x7f9c2419f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56180/0 (socket says 172.21.15.63:56180) 2024-05-27T07:27:32.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c23200700 1 -- 172.21.15.63:0/3519791080 learned_addr learned my addr 172.21.15.63:0/3519791080 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:27:32.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.718+0000 7f9c21e00700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c24107de0 0x7f9c2419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:32.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.719+0000 7f9c22800700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c241034b0 0x7f9c2419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:32.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.719+0000 7f9c23200700 1 -- 172.21.15.63:0/3519791080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c241034b0 msgr2=0x7f9c2419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:32.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.719+0000 7f9c23200700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c241034b0 0x7f9c2419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:32.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.719+0000 7f9c23200700 1 -- 172.21.15.63:0/3519791080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c24107de0 msgr2=0x7f9c2419b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:32.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.719+0000 7f9c23200700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c24107de0 0x7f9c2419b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:32.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.719+0000 7f9c23200700 1 -- 172.21.15.63:0/3519791080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2419fbf0 con 0x7f9c24108730 2024-05-27T07:27:32.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.719+0000 7f9c21e00700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c24107de0 0x7f9c2419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:27:32.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.719+0000 7f9c23200700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c24108730 0x7f9c2419f550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00ef30 tx=0x7f9c1c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:32.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.720+0000 7f9c1b400700 1 -- 172.21.15.63:0/3519791080 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0092e0 con 0x7f9c24108730 2024-05-27T07:27:32.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.720+0000 7f9c1b400700 1 -- 172.21.15.63:0/3519791080 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c1c004bb0 con 0x7f9c24108730 2024-05-27T07:27:32.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.720+0000 7f9c29400700 1 -- 172.21.15.63:0/3519791080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2419fe80 con 0x7f9c24108730 2024-05-27T07:27:32.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.720+0000 7f9c1b400700 1 -- 172.21.15.63:0/3519791080 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c004020 con 0x7f9c24108730 2024-05-27T07:27:32.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.720+0000 7f9c29400700 1 -- 172.21.15.63:0/3519791080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c241a0340 con 0x7f9c24108730 2024-05-27T07:27:32.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.721+0000 7f9c29400700 1 -- 172.21.15.63:0/3519791080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c240512b0 con 0x7f9c24108730 2024-05-27T07:27:32.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.724+0000 7f9c1b400700 1 -- 172.21.15.63:0/3519791080 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9c1c01b030 con 0x7f9c24108730 2024-05-27T07:27:32.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.725+0000 7f9c1b400700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9c080b2880 0x7f9c080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:32.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.725+0000 7f9c1b400700 1 -- 172.21.15.63:0/3519791080 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f9c1c0e1860 con 0x7f9c24108730 2024-05-27T07:27:32.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.726+0000 7f9c22800700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9c080b2880 0x7f9c080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:32.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.726+0000 7f9c22800700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9c080b2880 0x7f9c080b4d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9c14005950 tx=0x7f9c140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:32.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:32.730+0000 7f9c1b400700 1 -- 172.21.15.63:0/3519791080 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c1c022020 con 0x7f9c24108730 2024-05-27T07:27:32.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:32 smithi063 ceph-mon[31613]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:32.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:32.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:32.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:32.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:32.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:32.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:33.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.027+0000 7f9c29400700 1 -- 172.21.15.63:0/3519791080 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c2402e5e0 con 0x7f9c24108730 2024-05-27T07:27:33.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.028+0000 7f9c1b400700 1 -- 172.21.15.63:0/3519791080 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7f9c1c010040 con 0x7f9c24108730 2024-05-27T07:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34430,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 -- 172.21.15.63:0/3519791080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9c080b2880 msgr2=0x7f9c080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9c080b2880 0x7f9c080b4d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9c14005950 tx=0x7f9c140058e0 comp rx=0 tx=0).stop 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 -- 172.21.15.63:0/3519791080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c24108730 msgr2=0x7f9c2419f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c24108730 0x7f9c2419f550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00ef30 tx=0x7f9c1c00c3a0 comp rx=0 tx=0).stop 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 -- 172.21.15.63:0/3519791080 shutdown_connections 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9c080b2880 0x7f9c080b4d90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c24108730 0x7f9c2419f550 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c241034b0 0x7f9c2419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 --2- 172.21.15.63:0/3519791080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c24107de0 0x7f9c2419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 -- 172.21.15.63:0/3519791080 >> 172.21.15.63:0/3519791080 conn(0x7f9c240fedb0 msgr2=0x7f9c2406b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:33.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 -- 172.21.15.63:0/3519791080 shutdown_connections 2024-05-27T07:27:33.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:33.031+0000 7f9c29400700 1 -- 172.21.15.63:0/3519791080 wait complete. 2024-05-27T07:27:33.023 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 5 2024-05-27T07:27:33.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:32 smithi086 ceph-mon[34687]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:33.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:33.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:33.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:33.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:33.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:33.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:33.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:32 smithi149 ceph-mon[35408]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:33.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:33.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:33.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:33.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:33.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:33.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:34.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:33 smithi086 ceph-mon[34687]: Deploying daemon mds.e on smithi086 2024-05-27T07:27:34.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:33 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.63:6836/2286910897,v1:172.21.15.63:6837/2286910897] up:boot 2024-05-27T07:27:34.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:33 smithi086 ceph-mon[34687]: fsmap 4 up:standby 2024-05-27T07:27:34.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:33 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:27:34.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:33 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3519791080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:34.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:33 smithi149 ceph-mon[35408]: Deploying daemon mds.e on smithi086 2024-05-27T07:27:34.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:33 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.63:6836/2286910897,v1:172.21.15.63:6837/2286910897] up:boot 2024-05-27T07:27:34.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:33 smithi149 ceph-mon[35408]: fsmap 4 up:standby 2024-05-27T07:27:34.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:33 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:27:34.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:33 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3519791080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:34.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:33 smithi063 ceph-mon[31613]: Deploying daemon mds.e on smithi086 2024-05-27T07:27:34.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:33 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.63:6836/2286910897,v1:172.21.15.63:6837/2286910897] up:boot 2024-05-27T07:27:34.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:33 smithi063 ceph-mon[31613]: fsmap 4 up:standby 2024-05-27T07:27:34.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:33 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:27:34.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:33 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3519791080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:35.055 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:34 smithi086 ceph-mon[34687]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:35.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:34 smithi149 ceph-mon[35408]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:35.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:34 smithi063 ceph-mon[31613]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:35.843 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:27:36.108 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 07:27:35 smithi086 systemd[1]: Starting Ceph mds.e for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:27:36.314 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:27:36.406 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 07:27:36 smithi086 podman[64469]: 2024-05-27 07:27:36.022649778 +0000 UTC m=+0.034866554 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:27:36.407 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 07:27:36 smithi086 podman[64469]: 2024-05-27 07:27:36.162396435 +0000 UTC m=+0.174613249 container create 409476ac200c537f2855ce95cb271ac17a8d7660581ccb1398a9d46eb5df985b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:27:36.407 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 07:27:36 smithi086 podman[64469]: 2024-05-27 07:27:36.416051046 +0000 UTC m=+0.428267867 container init 409476ac200c537f2855ce95cb271ac17a8d7660581ccb1398a9d46eb5df985b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-27T07:27:36.699 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 07:27:36 smithi086 podman[64469]: 2024-05-27 07:27:36.426973251 +0000 UTC m=+0.439190036 container start 409476ac200c537f2855ce95cb271ac17a8d7660581ccb1398a9d46eb5df985b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True) 2024-05-27T07:27:36.699 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 07:27:36 smithi086 bash[64469]: 409476ac200c537f2855ce95cb271ac17a8d7660581ccb1398a9d46eb5df985b 2024-05-27T07:27:36.699 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 07:27:36 smithi086 systemd[1]: Started Ceph mds.e for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:27:36.699 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 07:27:36 smithi086 ceph-mds[64488]: -- 172.21.15.86:0/684220982 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed6c247a40 con 0x55ed6c28f180 2024-05-27T07:27:36.699 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 07:27:36 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-e[64484]: starting mds.e at 2024-05-27T07:27:37.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:36 smithi086 ceph-mon[34687]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:36 smithi149 ceph-mon[35408]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:36 smithi063 ceph-mon[31613]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:37.968 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:37 smithi149 ceph-mon[35408]: Deploying daemon mds.f on smithi149 2024-05-27T07:27:37.968 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:37 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] up:boot 2024-05-27T07:27:37.968 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:37 smithi149 ceph-mon[35408]: fsmap 5 up:standby 2024-05-27T07:27:37.969 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:37 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:27:38.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:37 smithi063 ceph-mon[31613]: Deploying daemon mds.f on smithi149 2024-05-27T07:27:38.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:37 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] up:boot 2024-05-27T07:27:38.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:37 smithi063 ceph-mon[31613]: fsmap 5 up:standby 2024-05-27T07:27:38.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:37 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:27:38.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:37 smithi086 ceph-mon[34687]: Deploying daemon mds.f on smithi149 2024-05-27T07:27:38.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:37 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] up:boot 2024-05-27T07:27:38.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:37 smithi086 ceph-mon[34687]: fsmap 5 up:standby 2024-05-27T07:27:38.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:37 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:27:38.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.727+0000 7ff5f1800700 1 -- 172.21.15.63:0/1322349366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5ec10b890 msgr2=0x7ff5ec1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:38.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.727+0000 7ff5f1800700 1 --2- 172.21.15.63:0/1322349366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5ec10b890 0x7ff5ec1131f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400b0d0 tx=0x7ff5e400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:27:38.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.727+0000 7ff5f1800700 1 -- 172.21.15.63:0/1322349366 shutdown_connections 2024-05-27T07:27:38.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.727+0000 7ff5f1800700 1 --2- 172.21.15.63:0/1322349366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5ec10b890 0x7ff5ec1131f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:38.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.727+0000 7ff5f1800700 1 --2- 172.21.15.63:0/1322349366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec106610 0x7ff5ec106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:38.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.727+0000 7ff5f1800700 1 --2- 172.21.15.63:0/1322349366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5ec10af40 0x7ff5ec10b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:38.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.727+0000 7ff5f1800700 1 -- 172.21.15.63:0/1322349366 >> 172.21.15.63:0/1322349366 conn(0x7ff5ec102020 msgr2=0x7ff5ec104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:38.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.727+0000 7ff5f1800700 1 -- 172.21.15.63:0/1322349366 shutdown_connections 2024-05-27T07:27:38.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.727+0000 7ff5f1800700 1 -- 172.21.15.63:0/1322349366 wait complete. 2024-05-27T07:27:38.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.728+0000 7ff5f1800700 1 Processor -- start 2024-05-27T07:27:38.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.728+0000 7ff5f1800700 1 -- start start 2024-05-27T07:27:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.729+0000 7ff5f1800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec106610 0x7ff5ec19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.729+0000 7ff5f1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5ec10af40 0x7ff5ec19d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.729+0000 7ff5f1800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5ec10b890 0x7ff5ec1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.729+0000 7ff5f1800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec116370 con 0x7ff5ec10b890 2024-05-27T07:27:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.729+0000 7ff5f1800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1161f0 con 0x7ff5ec10af40 2024-05-27T07:27:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.729+0000 7ff5f1800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1164f0 con 0x7ff5ec106610 2024-05-27T07:27:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.729+0000 7ff5f0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5ec10b890 0x7ff5ec1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.729+0000 7ff5f0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5ec10b890 0x7ff5ec1a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56194/0 (socket says 172.21.15.63:56194) 2024-05-27T07:27:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.729+0000 7ff5f0e00700 1 -- 172.21.15.63:0/2517609781 learned_addr learned my addr 172.21.15.63:0/2517609781 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:27:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.729+0000 7ff5ebe00700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec106610 0x7ff5ec19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:38.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.730+0000 7ff5e3400700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5ec10af40 0x7ff5ec19d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.730+0000 7ff5f0e00700 1 -- 172.21.15.63:0/2517609781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec106610 msgr2=0x7ff5ec19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.730+0000 7ff5f0e00700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec106610 0x7ff5ec19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.730+0000 7ff5f0e00700 1 -- 172.21.15.63:0/2517609781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5ec10af40 msgr2=0x7ff5ec19d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.730+0000 7ff5f0e00700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5ec10af40 0x7ff5ec19d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.730+0000 7ff5f0e00700 1 -- 172.21.15.63:0/2517609781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec1a1bf0 con 0x7ff5ec10b890 2024-05-27T07:27:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.730+0000 7ff5e3400700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5ec10af40 0x7ff5ec19d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:27:38.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.730+0000 7ff5f0e00700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5ec10b890 0x7ff5ec1a1550 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009850 tx=0x7ff5e4007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:38.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.731+0000 7ff5e9600700 1 -- 172.21.15.63:0/2517609781 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e401f030 con 0x7ff5ec10b890 2024-05-27T07:27:38.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.731+0000 7ff5e9600700 1 -- 172.21.15.63:0/2517609781 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5e401bad0 con 0x7ff5ec10b890 2024-05-27T07:27:38.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.731+0000 7ff5f1800700 1 -- 172.21.15.63:0/2517609781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec1a1e80 con 0x7ff5ec10b890 2024-05-27T07:27:38.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.731+0000 7ff5e9600700 1 -- 172.21.15.63:0/2517609781 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4012bb0 con 0x7ff5ec10b890 2024-05-27T07:27:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.731+0000 7ff5f1800700 1 -- 172.21.15.63:0/2517609781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec1a9800 con 0x7ff5ec10b890 2024-05-27T07:27:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.732+0000 7ff5f1800700 1 -- 172.21.15.63:0/2517609781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec107ac0 con 0x7ff5ec10b890 2024-05-27T07:27:38.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.735+0000 7ff5e9600700 1 -- 172.21.15.63:0/2517609781 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff5e4022030 con 0x7ff5ec10b890 2024-05-27T07:27:38.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.736+0000 7ff5e9600700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5d00b2880 0x7ff5d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:38.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.737+0000 7ff5e9600700 1 -- 172.21.15.63:0/2517609781 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ff5e400f020 con 0x7ff5ec10b890 2024-05-27T07:27:38.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.737+0000 7ff5ebe00700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5d00b2880 0x7ff5d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:38.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.737+0000 7ff5ebe00700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5d00b2880 0x7ff5d00b4d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc005fd0 tx=0x7ff5dc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:38.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:38.742+0000 7ff5e9600700 1 -- 172.21.15.63:0/2517609781 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff5e4029020 con 0x7ff5ec10b890 2024-05-27T07:27:38.925 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:38 smithi149 ceph-mon[35408]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:39.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.028+0000 7ff5f1800700 1 -- 172.21.15.63:0/2517609781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5ec19e080 con 0x7ff5ec10b890 2024-05-27T07:27:39.019 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:38 smithi063 ceph-mon[31613]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:39.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.029+0000 7ff5e9600700 1 -- 172.21.15.63:0/2517609781 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7ff5e40ab900 con 0x7ff5ec10b890 2024-05-27T07:27:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:27:39.020 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34430,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-27T07:27:39.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.032+0000 7ff5f1800700 1 -- 172.21.15.63:0/2517609781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5d00b2880 msgr2=0x7ff5d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:39.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.032+0000 7ff5f1800700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5d00b2880 0x7ff5d00b4d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc005fd0 tx=0x7ff5dc00a560 comp rx=0 tx=0).stop 2024-05-27T07:27:39.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.032+0000 7ff5f1800700 1 -- 172.21.15.63:0/2517609781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5ec10b890 msgr2=0x7ff5ec1a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:39.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.032+0000 7ff5f1800700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5ec10b890 0x7ff5ec1a1550 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009850 tx=0x7ff5e4007e40 comp rx=0 tx=0).stop 2024-05-27T07:27:39.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.033+0000 7ff5f1800700 1 -- 172.21.15.63:0/2517609781 shutdown_connections 2024-05-27T07:27:39.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.033+0000 7ff5f1800700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5d00b2880 0x7ff5d00b4d90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:39.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.033+0000 7ff5f1800700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5ec10b890 0x7ff5ec1a1550 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:39.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.033+0000 7ff5f1800700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5ec106610 0x7ff5ec19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:39.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.033+0000 7ff5f1800700 1 --2- 172.21.15.63:0/2517609781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5ec10af40 0x7ff5ec19d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:39.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.033+0000 7ff5f1800700 1 -- 172.21.15.63:0/2517609781 >> 172.21.15.63:0/2517609781 conn(0x7ff5ec102020 msgr2=0x7ff5ec10a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:39.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.033+0000 7ff5f1800700 1 -- 172.21.15.63:0/2517609781 shutdown_connections 2024-05-27T07:27:39.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:39.033+0000 7ff5f1800700 1 -- 172.21.15.63:0/2517609781 wait complete. 2024-05-27T07:27:39.024 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 6 2024-05-27T07:27:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:38 smithi086 ceph-mon[34687]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:40.162 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:39 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2517609781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:40.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:39 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2517609781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:40.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:39 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2517609781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:40.703 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:27:40 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: starting mds.f at 2024-05-27T07:27:41.018 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:40 smithi149 ceph-mon[35408]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:41.052 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:41.052 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:41.053 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:41.053 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:41.053 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:41.053 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:40 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:40 smithi086 ceph-mon[34687]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:40 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:40 smithi063 ceph-mon[31613]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:41.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:41.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:40 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:41.861 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:27:41.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:41 smithi149 ceph-mon[35408]: Deploying daemon mds.c on smithi149 2024-05-27T07:27:41.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:41 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/2496628912,v1:172.21.15.149:6835/2496628912] up:boot 2024-05-27T07:27:41.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:41 smithi149 ceph-mon[35408]: fsmap 6 up:standby 2024-05-27T07:27:41.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:41 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:27:41.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:41 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:27:42.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:41 smithi086 ceph-mon[34687]: Deploying daemon mds.c on smithi149 2024-05-27T07:27:42.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:41 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/2496628912,v1:172.21.15.149:6835/2496628912] up:boot 2024-05-27T07:27:42.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:41 smithi086 ceph-mon[34687]: fsmap 6 up:standby 2024-05-27T07:27:42.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:41 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:27:42.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:41 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:27:42.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:41 smithi063 ceph-mon[31613]: Deploying daemon mds.c on smithi149 2024-05-27T07:27:42.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:41 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/2496628912,v1:172.21.15.149:6835/2496628912] up:boot 2024-05-27T07:27:42.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:41 smithi063 ceph-mon[31613]: fsmap 6 up:standby 2024-05-27T07:27:42.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:41 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:27:42.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:41 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:27:42.342 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:27:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:42 smithi086 ceph-mon[34687]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:43.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:42 smithi149 ceph-mon[35408]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:43.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:42 smithi063 ceph-mon[31613]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:44.153 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 07:27:43 smithi149 systemd[1]: Starting Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:27:44.453 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 07:27:44 smithi149 podman[65937]: 2024-05-27 07:27:44.06817914 +0000 UTC m=+0.033965066 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:27:44.453 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 07:27:44 smithi149 podman[65937]: 2024-05-27 07:27:44.210965097 +0000 UTC m=+0.176751028 container create f582fedf84be7e9fff0ae7148df20a3d06cb78579dfaa6692f37e4fa79ef556b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-27T07:27:44.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.746+0000 7fc20b800700 1 -- 172.21.15.63:0/1461865153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20410e930 msgr2=0x7fc204110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:44.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.746+0000 7fc20b800700 1 --2- 172.21.15.63:0/1461865153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20410e930 0x7fc204110e00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00b0a0 tx=0x7fc1fc00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:27:44.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.746+0000 7fc20b800700 1 -- 172.21.15.63:0/1461865153 shutdown_connections 2024-05-27T07:27:44.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.746+0000 7fc20b800700 1 --2- 172.21.15.63:0/1461865153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20410e930 0x7fc204110e00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:44.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.746+0000 7fc20b800700 1 --2- 172.21.15.63:0/1461865153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc204105470 0x7fc204105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:44.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.746+0000 7fc20b800700 1 --2- 172.21.15.63:0/1461865153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204105e50 0x7fc20410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:44.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.746+0000 7fc20b800700 1 -- 172.21.15.63:0/1461865153 >> 172.21.15.63:0/1461865153 conn(0x7fc2040fef40 msgr2=0x7fc204101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:44.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.746+0000 7fc20b800700 1 -- 172.21.15.63:0/1461865153 shutdown_connections 2024-05-27T07:27:44.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.746+0000 7fc20b800700 1 -- 172.21.15.63:0/1461865153 wait complete. 2024-05-27T07:27:44.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.748+0000 7fc20b800700 1 Processor -- start 2024-05-27T07:27:44.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.748+0000 7fc20b800700 1 -- start start 2024-05-27T07:27:44.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.749+0000 7fc20b800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204105470 0x7fc20419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:44.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.749+0000 7fc20b800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc204105e50 0x7fc20419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.749+0000 7fc20b800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20410e930 0x7fc20419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.749+0000 7fc20b800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2041143a0 con 0x7fc20410e930 2024-05-27T07:27:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.749+0000 7fc20b800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc204114220 con 0x7fc204105470 2024-05-27T07:27:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.749+0000 7fc20b800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc204114520 con 0x7fc204105e50 2024-05-27T07:27:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.749+0000 7fc209800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20410e930 0x7fc20419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.749+0000 7fc209800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20410e930 0x7fc20419f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59540/0 (socket says 172.21.15.63:59540) 2024-05-27T07:27:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.749+0000 7fc209800700 1 -- 172.21.15.63:0/1514525110 learned_addr learned my addr 172.21.15.63:0/1514525110 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:27:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.749+0000 7fc208e00700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204105470 0x7fc20419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:44.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.750+0000 7fc203e00700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc204105e50 0x7fc20419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.750+0000 7fc209800700 1 -- 172.21.15.63:0/1514525110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc204105e50 msgr2=0x7fc20419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.750+0000 7fc209800700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc204105e50 0x7fc20419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.750+0000 7fc209800700 1 -- 172.21.15.63:0/1514525110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204105470 msgr2=0x7fc20419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.750+0000 7fc209800700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204105470 0x7fc20419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.750+0000 7fc209800700 1 -- 172.21.15.63:0/1514525110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20419fbe0 con 0x7fc20410e930 2024-05-27T07:27:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.750+0000 7fc209800700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20410e930 0x7fc20419f540 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00b070 tx=0x7fc1fc004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:44.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.751+0000 7fc201600700 1 -- 172.21.15.63:0/1514525110 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc021070 con 0x7fc20410e930 2024-05-27T07:27:44.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.751+0000 7fc201600700 1 -- 172.21.15.63:0/1514525110 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1fc007b80 con 0x7fc20410e930 2024-05-27T07:27:44.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.751+0000 7fc201600700 1 -- 172.21.15.63:0/1514525110 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc0046e0 con 0x7fc20410e930 2024-05-27T07:27:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.751+0000 7fc20b800700 1 -- 172.21.15.63:0/1514525110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20419fe70 con 0x7fc20410e930 2024-05-27T07:27:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.751+0000 7fc20b800700 1 -- 172.21.15.63:0/1514525110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc204103090 con 0x7fc20410e930 2024-05-27T07:27:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.753+0000 7fc20b800700 1 -- 172.21.15.63:0/1514525110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2040762c0 con 0x7fc20410e930 2024-05-27T07:27:44.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.757+0000 7fc201600700 1 -- 172.21.15.63:0/1514525110 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc1fc003a00 con 0x7fc20410e930 2024-05-27T07:27:44.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.759+0000 7fc201600700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc1ec0b2900 0x7fc1ec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:44.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.759+0000 7fc201600700 1 -- 172.21.15.63:0/1514525110 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc1fc00f020 con 0x7fc20410e930 2024-05-27T07:27:44.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.759+0000 7fc208e00700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc1ec0b2900 0x7fc1ec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:44.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.760+0000 7fc208e00700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc1ec0b2900 0x7fc1ec0b4e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8007be0 tx=0x7fc1f80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:44.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:44.760+0000 7fc201600700 1 -- 172.21.15.63:0/1514525110 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1fc02a020 con 0x7fc20410e930 2024-05-27T07:27:44.894 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 07:27:44 smithi149 podman[65937]: 2024-05-27 07:27:44.50684828 +0000 UTC m=+0.472634287 container init f582fedf84be7e9fff0ae7148df20a3d06cb78579dfaa6692f37e4fa79ef556b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:27:44.894 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 07:27:44 smithi149 podman[65937]: 2024-05-27 07:27:44.516689175 +0000 UTC m=+0.482475152 container start f582fedf84be7e9fff0ae7148df20a3d06cb78579dfaa6692f37e4fa79ef556b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T07:27:44.894 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 07:27:44 smithi149 bash[65937]: f582fedf84be7e9fff0ae7148df20a3d06cb78579dfaa6692f37e4fa79ef556b 2024-05-27T07:27:44.894 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 07:27:44 smithi149 systemd[1]: Started Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:27:44.894 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 07:27:44 smithi149 ceph-mds[65956]: -- 172.21.15.149:0/2245911639 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56461575fa40 con 0x5646157a7a80 2024-05-27T07:27:44.895 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 07:27:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: starting mds.c at 2024-05-27T07:27:45.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.055+0000 7fc20b800700 1 -- 172.21.15.63:0/1514525110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc20419c070 con 0x7fc20410e930 2024-05-27T07:27:45.046 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:44 smithi063 ceph-mon[31613]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:45.046 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:45.047 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:45.047 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:45.047 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:45.047 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:45.047 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:44 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:45.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.056+0000 7fc201600700 1 -- 172.21.15.63:0/1514525110 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fc1fc0ab840 con 0x7fc20410e930 2024-05-27T07:27:45.048 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:27:45.048 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34430,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-27T07:27:45.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.059+0000 7fc20b800700 1 -- 172.21.15.63:0/1514525110 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc1ec0b2900 msgr2=0x7fc1ec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:45.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.059+0000 7fc20b800700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc1ec0b2900 0x7fc1ec0b4e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8007be0 tx=0x7fc1f80073d0 comp rx=0 tx=0).stop 2024-05-27T07:27:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.059+0000 7fc20b800700 1 -- 172.21.15.63:0/1514525110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20410e930 msgr2=0x7fc20419f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.059+0000 7fc20b800700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20410e930 0x7fc20419f540 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00b070 tx=0x7fc1fc004d50 comp rx=0 tx=0).stop 2024-05-27T07:27:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.060+0000 7fc20b800700 1 -- 172.21.15.63:0/1514525110 shutdown_connections 2024-05-27T07:27:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.060+0000 7fc20b800700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc1ec0b2900 0x7fc1ec0b4e10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.060+0000 7fc20b800700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20410e930 0x7fc20419f540 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.060+0000 7fc20b800700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc204105470 0x7fc20419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.060+0000 7fc20b800700 1 --2- 172.21.15.63:0/1514525110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc204105e50 0x7fc20419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.060+0000 7fc20b800700 1 -- 172.21.15.63:0/1514525110 >> 172.21.15.63:0/1514525110 conn(0x7fc2040fef40 msgr2=0x7fc204100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.060+0000 7fc20b800700 1 -- 172.21.15.63:0/1514525110 shutdown_connections 2024-05-27T07:27:45.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:45.060+0000 7fc20b800700 1 -- 172.21.15.63:0/1514525110 wait complete. 2024-05-27T07:27:45.052 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 8 2024-05-27T07:27:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:44 smithi086 ceph-mon[34687]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:45.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:45.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:44 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:44 smithi149 ceph-mon[35408]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:45.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:45.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:45.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:44 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:46.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:45 smithi086 ceph-mon[34687]: Deploying daemon mds.h on smithi086 2024-05-27T07:27:46.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:45 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6836/3113374312,v1:172.21.15.149:6837/3113374312] up:boot 2024-05-27T07:27:46.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:45 smithi086 ceph-mon[34687]: fsmap 7 up:standby 2024-05-27T07:27:46.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:45 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:27:46.014 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:45 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1514525110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:46.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:45 smithi149 ceph-mon[35408]: Deploying daemon mds.h on smithi086 2024-05-27T07:27:46.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:45 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6836/3113374312,v1:172.21.15.149:6837/3113374312] up:boot 2024-05-27T07:27:46.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:45 smithi149 ceph-mon[35408]: fsmap 7 up:standby 2024-05-27T07:27:46.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:45 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:27:46.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:45 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1514525110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:46.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:45 smithi063 ceph-mon[31613]: Deploying daemon mds.h on smithi086 2024-05-27T07:27:46.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:45 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6836/3113374312,v1:172.21.15.149:6837/3113374312] up:boot 2024-05-27T07:27:46.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:45 smithi063 ceph-mon[31613]: fsmap 7 up:standby 2024-05-27T07:27:46.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:45 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:27:46.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:45 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1514525110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:47.163 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:46 smithi086 ceph-mon[34687]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:47.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:46 smithi149 ceph-mon[35408]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:47.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:46 smithi063 ceph-mon[31613]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:47.945 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:27:48.337 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 07:27:47 smithi086 systemd[1]: Starting Ceph mds.h for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:27:48.419 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:27:48.681 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 07:27:48 smithi086 podman[64776]: 2024-05-27 07:27:48.251564985 +0000 UTC m=+0.031326585 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:27:48.682 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 07:27:48 smithi086 podman[64776]: 2024-05-27 07:27:48.374896186 +0000 UTC m=+0.154657810 container create f61d2b53dbb796ea5538514ad28cb795a9a6604d154f24f75b47d398ceb2b8d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T07:27:48.682 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 07:27:48 smithi086 podman[64776]: 2024-05-27 07:27:48.691130614 +0000 UTC m=+0.470892281 container init f61d2b53dbb796ea5538514ad28cb795a9a6604d154f24f75b47d398ceb2b8d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-27T07:27:48.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:48 smithi086 ceph-mon[34687]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:48.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:48 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:48.949 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 07:27:48 smithi086 podman[64776]: 2024-05-27 07:27:48.701491476 +0000 UTC m=+0.481253107 container start f61d2b53dbb796ea5538514ad28cb795a9a6604d154f24f75b47d398ceb2b8d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-h, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T07:27:48.949 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 07:27:48 smithi086 bash[64776]: f61d2b53dbb796ea5538514ad28cb795a9a6604d154f24f75b47d398ceb2b8d9 2024-05-27T07:27:48.949 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 07:27:48 smithi086 ceph-mds[64795]: -- 172.21.15.86:0/3209233058 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56365e3dfa40 con 0x56365e427600 2024-05-27T07:27:48.949 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 07:27:48 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-h[64791]: starting mds.h at 2024-05-27T07:27:48.949 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 07:27:48 smithi086 systemd[1]: Started Ceph mds.h for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:27:49.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:48 smithi149 ceph-mon[35408]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:49.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:48 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:49.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:48 smithi063 ceph-mon[31613]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:49.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:48 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:50.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:49 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] up:boot 2024-05-27T07:27:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:49 smithi086 ceph-mon[34687]: fsmap 8 up:standby 2024-05-27T07:27:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:27:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:49 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:49 smithi086 ceph-mon[34687]: Deploying daemon mds.i on smithi149 2024-05-27T07:27:50.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:49 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] up:boot 2024-05-27T07:27:50.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:49 smithi149 ceph-mon[35408]: fsmap 8 up:standby 2024-05-27T07:27:50.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:27:50.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:49 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:49 smithi149 ceph-mon[35408]: Deploying daemon mds.i on smithi149 2024-05-27T07:27:50.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:49 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] up:boot 2024-05-27T07:27:50.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:49 smithi063 ceph-mon[31613]: fsmap 8 up:standby 2024-05-27T07:27:50.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:27:50.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:50.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:50.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:50.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:50.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:49 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:50.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:49 smithi063 ceph-mon[31613]: Deploying daemon mds.i on smithi149 2024-05-27T07:27:50.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.914+0000 7f5291200700 1 -- 172.21.15.63:0/221710481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f528c110b90 msgr2=0x7f528c112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:50.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.914+0000 7f5291200700 1 --2- 172.21.15.63:0/221710481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f528c110b90 0x7f528c112ff0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f528400b0a0 tx=0x7f528400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:27:50.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.914+0000 7f5291200700 1 -- 172.21.15.63:0/221710481 shutdown_connections 2024-05-27T07:27:50.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.914+0000 7f5291200700 1 --2- 172.21.15.63:0/221710481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f528c110b90 0x7f528c112ff0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:50.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.915+0000 7f5291200700 1 --2- 172.21.15.63:0/221710481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528c075d20 0x7f528c076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:50.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.915+0000 7f5291200700 1 --2- 172.21.15.63:0/221710481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c076670 0x7f528c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:50.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.915+0000 7f5291200700 1 -- 172.21.15.63:0/221710481 >> 172.21.15.63:0/221710481 conn(0x7f528c100030 msgr2=0x7f528c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:50.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.915+0000 7f5291200700 1 -- 172.21.15.63:0/221710481 shutdown_connections 2024-05-27T07:27:50.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.915+0000 7f5291200700 1 -- 172.21.15.63:0/221710481 wait complete. 2024-05-27T07:27:50.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.916+0000 7f5291200700 1 Processor -- start 2024-05-27T07:27:50.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.916+0000 7f5291200700 1 -- start start 2024-05-27T07:27:50.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f5291200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c075d20 0x7f528c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:50.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f5291200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f528c076670 0x7f528c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:50.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f5291200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528c110b90 0x7f528c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:50.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f5291200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f528c116570 con 0x7f528c076670 2024-05-27T07:27:50.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f5291200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1163f0 con 0x7f528c075d20 2024-05-27T07:27:50.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f5291200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1166f0 con 0x7f528c110b90 2024-05-27T07:27:50.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f5281400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f528c076670 0x7f528c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:50.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f5281400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f528c076670 0x7f528c19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58394/0 (socket says 172.21.15.63:58394) 2024-05-27T07:27:50.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f5281400700 1 -- 172.21.15.63:0/3011113015 learned_addr learned my addr 172.21.15.63:0/3011113015 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:27:50.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f528a800700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528c110b90 0x7f528c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:50.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.917+0000 7f5289e00700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c075d20 0x7f528c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:50.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.918+0000 7f5281400700 1 -- 172.21.15.63:0/3011113015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528c110b90 msgr2=0x7f528c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:50.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.918+0000 7f5281400700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528c110b90 0x7f528c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:50.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.918+0000 7f5281400700 1 -- 172.21.15.63:0/3011113015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c075d20 msgr2=0x7f528c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:50.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.918+0000 7f5281400700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c075d20 0x7f528c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:50.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.918+0000 7f5281400700 1 -- 172.21.15.63:0/3011113015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c1a1df0 con 0x7f528c076670 2024-05-27T07:27:50.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.918+0000 7f5281400700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f528c076670 0x7f528c19d4c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f527400d950 tx=0x7f527400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:50.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.918+0000 7f5283400700 1 -- 172.21.15.63:0/3011113015 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274016070 con 0x7f528c076670 2024-05-27T07:27:50.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.918+0000 7f5283400700 1 -- 172.21.15.63:0/3011113015 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5274010460 con 0x7f528c076670 2024-05-27T07:27:50.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.918+0000 7f5283400700 1 -- 172.21.15.63:0/3011113015 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527400f620 con 0x7f528c076670 2024-05-27T07:27:50.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.919+0000 7f5291200700 1 -- 172.21.15.63:0/3011113015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528c1a20e0 con 0x7f528c076670 2024-05-27T07:27:50.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.919+0000 7f5291200700 1 -- 172.21.15.63:0/3011113015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f528c1a9a00 con 0x7f528c076670 2024-05-27T07:27:50.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.920+0000 7f5291200700 1 -- 172.21.15.63:0/3011113015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f528c0512b0 con 0x7f528c076670 2024-05-27T07:27:50.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.923+0000 7f5283400700 1 -- 172.21.15.63:0/3011113015 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5274019070 con 0x7f528c076670 2024-05-27T07:27:50.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.924+0000 7f5283400700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f52700b2880 0x7f52700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:50.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.924+0000 7f5283400700 1 -- 172.21.15.63:0/3011113015 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f52740e19b0 con 0x7f528c076670 2024-05-27T07:27:50.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.924+0000 7f5289e00700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f52700b2880 0x7f52700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:50.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.925+0000 7f5289e00700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f52700b2880 0x7f52700b4d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f527c009bf0 tx=0x7f527c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:50.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:50.927+0000 7f5283400700 1 -- 172.21.15.63:0/3011113015 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52740a9ad0 con 0x7f528c076670 2024-05-27T07:27:51.033 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:51 smithi149 ceph-mon[35408]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:51.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.212+0000 7f5291200700 1 -- 172.21.15.63:0/3011113015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f528c02e5e0 con 0x7f528c076670 2024-05-27T07:27:51.203 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:51 smithi063 ceph-mon[31613]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:51.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.213+0000 7f5283400700 1 -- 172.21.15.63:0/3011113015 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f52740a93d0 con 0x7f528c076670 2024-05-27T07:27:51.204 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:27:51.204 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34430,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34484,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T07:27:51.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.216+0000 7f5291200700 1 -- 172.21.15.63:0/3011113015 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f52700b2880 msgr2=0x7f52700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:51.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.216+0000 7f5291200700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f52700b2880 0x7f52700b4d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f527c009bf0 tx=0x7f527c009260 comp rx=0 tx=0).stop 2024-05-27T07:27:51.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.216+0000 7f5291200700 1 -- 172.21.15.63:0/3011113015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f528c076670 msgr2=0x7f528c19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:51.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.216+0000 7f5291200700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f528c076670 0x7f528c19d4c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f527400d950 tx=0x7f527400dd10 comp rx=0 tx=0).stop 2024-05-27T07:27:51.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.217+0000 7f5291200700 1 -- 172.21.15.63:0/3011113015 shutdown_connections 2024-05-27T07:27:51.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.217+0000 7f5291200700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528c110b90 0x7f528c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:51.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.217+0000 7f5291200700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c075d20 0x7f528c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:51.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.217+0000 7f5291200700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f52700b2880 0x7f52700b4d90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:51.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.217+0000 7f5291200700 1 --2- 172.21.15.63:0/3011113015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f528c076670 0x7f528c19d4c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:51.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.217+0000 7f5291200700 1 -- 172.21.15.63:0/3011113015 >> 172.21.15.63:0/3011113015 conn(0x7f528c100030 msgr2=0x7f528c1006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:51.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.217+0000 7f5291200700 1 -- 172.21.15.63:0/3011113015 shutdown_connections 2024-05-27T07:27:51.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:51.217+0000 7f5291200700 1 -- 172.21.15.63:0/3011113015 wait complete. 2024-05-27T07:27:51.209 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 9 2024-05-27T07:27:51.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:51 smithi086 ceph-mon[34687]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:52.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:52 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3011113015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:52.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:52 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3011113015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:52.437 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 07:27:52 smithi149 systemd[1]: Starting Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:27:52.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:52 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3011113015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:52.695 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 07:27:52 smithi149 podman[66238]: 2024-05-27 07:27:52.3509712 +0000 UTC m=+0.035738890 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:27:52.695 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 07:27:52 smithi149 podman[66238]: 2024-05-27 07:27:52.475227708 +0000 UTC m=+0.159995437 container create 82900682d407a3cb20594d9258f973e44a61708188c66cdb9c6bea4dc71b6ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-27T07:27:53.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:53 smithi149 ceph-mon[35408]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:53.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:53 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:53.203 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 07:27:52 smithi149 podman[66238]: 2024-05-27 07:27:52.805177531 +0000 UTC m=+0.489945240 container init 82900682d407a3cb20594d9258f973e44a61708188c66cdb9c6bea4dc71b6ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-27T07:27:53.203 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 07:27:52 smithi149 podman[66238]: 2024-05-27 07:27:52.815937246 +0000 UTC m=+0.500704942 container start 82900682d407a3cb20594d9258f973e44a61708188c66cdb9c6bea4dc71b6ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-27T07:27:53.203 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 07:27:52 smithi149 bash[66238]: 82900682d407a3cb20594d9258f973e44a61708188c66cdb9c6bea4dc71b6ea4 2024-05-27T07:27:53.203 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 07:27:52 smithi149 systemd[1]: Started Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:27:53.203 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 07:27:52 smithi149 ceph-mds[66257]: -- 172.21.15.149:0/2200976427 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5580a658fa40 con 0x5580a65d7a80 2024-05-27T07:27:53.203 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 07:27:52 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: starting mds.i at 2024-05-27T07:27:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:53 smithi086 ceph-mon[34687]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:53 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:53.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:53 smithi063 ceph-mon[31613]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:53.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:53 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:53.962 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:27:54.321 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:54.321 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:54 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6838/1574913181,v1:172.21.15.149:6839/1574913181] up:boot 2024-05-27T07:27:54.321 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:54 smithi063 ceph-mon[31613]: fsmap 9 up:standby 2024-05-27T07:27:54.321 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:27:54.321 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:54.321 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:54.322 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:54.322 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:54 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:54.322 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:54 smithi063 ceph-mon[31613]: Deploying daemon mds.g on smithi063 2024-05-27T07:27:54.322 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:54 smithi063 ceph-mon[31613]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:54.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:54.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:54 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6838/1574913181,v1:172.21.15.149:6839/1574913181] up:boot 2024-05-27T07:27:54.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:54 smithi086 ceph-mon[34687]: fsmap 9 up:standby 2024-05-27T07:27:54.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:27:54.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:54.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:54.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:54.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:54 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:54.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:54 smithi086 ceph-mon[34687]: Deploying daemon mds.g on smithi063 2024-05-27T07:27:54.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:54 smithi086 ceph-mon[34687]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:54.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:54.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:54 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6838/1574913181,v1:172.21.15.149:6839/1574913181] up:boot 2024-05-27T07:27:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:54 smithi149 ceph-mon[35408]: fsmap 9 up:standby 2024-05-27T07:27:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:27:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:54 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:54 smithi149 ceph-mon[35408]: Deploying daemon mds.g on smithi063 2024-05-27T07:27:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:54 smithi149 ceph-mon[35408]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:55.341 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:27:56.388 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 07:27:56 smithi063 systemd[1]: Starting Ceph mds.g for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:27:57.163 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:56 smithi086 ceph-mon[34687]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:57.176 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:56 smithi149 ceph-mon[35408]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:57.178 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:56 smithi063 ceph-mon[31613]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:57.178 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 07:27:56 smithi063 podman[73359]: 2024-05-27 07:27:56.886332288 +0000 UTC m=+0.035834629 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:27:57.178 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 07:27:57 smithi063 podman[73359]: 2024-05-27 07:27:57.008697629 +0000 UTC m=+0.158200001 container create 906bc25a3e3b67470fe40387beb456f9bd13629afb8361c96f45f6993661687d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-g, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-27T07:27:57.808 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 07:27:57 smithi063 podman[73359]: 2024-05-27 07:27:57.55732561 +0000 UTC m=+0.706827994 container init 906bc25a3e3b67470fe40387beb456f9bd13629afb8361c96f45f6993661687d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-g, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-27T07:27:57.808 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 07:27:57 smithi063 podman[73359]: 2024-05-27 07:27:57.568820249 +0000 UTC m=+0.718322600 container start 906bc25a3e3b67470fe40387beb456f9bd13629afb8361c96f45f6993661687d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:27:57.809 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 07:27:57 smithi063 bash[73359]: 906bc25a3e3b67470fe40387beb456f9bd13629afb8361c96f45f6993661687d 2024-05-27T07:27:57.809 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 07:27:57 smithi063 ceph-mds[73400]: -- 172.21.15.63:0/2697521422 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5582a6421a40 con 0x5582a6469600 2024-05-27T07:27:57.809 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 07:27:57 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-g[73389]: starting mds.g at 2024-05-27T07:27:57.809 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 07:27:57 smithi063 systemd[1]: Started Ceph mds.g for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:27:58.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.063+0000 7f1b9f000700 1 -- 172.21.15.63:0/3947605380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b98077780 msgr2=0x7f1b98110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:58.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.063+0000 7f1b9f000700 1 --2- 172.21.15.63:0/3947605380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b98077780 0x7f1b98110640 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009990 tx=0x7f1b84009ca0 comp rx=0 tx=0).stop 2024-05-27T07:27:58.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.064+0000 7f1b9f000700 1 -- 172.21.15.63:0/3947605380 shutdown_connections 2024-05-27T07:27:58.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.064+0000 7f1b9f000700 1 --2- 172.21.15.63:0/3947605380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b98110b80 0x7f1b98112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:58.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.064+0000 7f1b9f000700 1 --2- 172.21.15.63:0/3947605380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98076e30 0x7f1b98077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:58.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.064+0000 7f1b9f000700 1 --2- 172.21.15.63:0/3947605380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b98077780 0x7f1b98110640 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:58.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.064+0000 7f1b9f000700 1 -- 172.21.15.63:0/3947605380 >> 172.21.15.63:0/3947605380 conn(0x7f1b980ffee0 msgr2=0x7f1b98102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:58.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.064+0000 7f1b9f000700 1 -- 172.21.15.63:0/3947605380 shutdown_connections 2024-05-27T07:27:58.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.064+0000 7f1b9f000700 1 -- 172.21.15.63:0/3947605380 wait complete. 2024-05-27T07:27:58.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.065+0000 7f1b9f000700 1 Processor -- start 2024-05-27T07:27:58.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.065+0000 7f1b9f000700 1 -- start start 2024-05-27T07:27:58.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.066+0000 7f1b9f000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98076e30 0x7f1b9819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:58.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.066+0000 7f1b9f000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b98077780 0x7f1b9819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:58.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.066+0000 7f1b9f000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b98110b80 0x7f1b981a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:58.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.066+0000 7f1b9f000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98116580 con 0x7f1b98110b80 2024-05-27T07:27:58.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.066+0000 7f1b9f000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98116400 con 0x7f1b98077780 2024-05-27T07:27:58.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.066+0000 7f1b9f000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98116700 con 0x7f1b98076e30 2024-05-27T07:27:58.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.066+0000 7f1b9d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b98110b80 0x7f1b981a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:58.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.066+0000 7f1b9d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b98110b80 0x7f1b981a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58440/0 (socket says 172.21.15.63:58440) 2024-05-27T07:27:58.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.066+0000 7f1b9d000700 1 -- 172.21.15.63:0/3741105564 learned_addr learned my addr 172.21.15.63:0/3741105564 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:27:58.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.066+0000 7f1b97e00700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98076e30 0x7f1b9819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:58.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.067+0000 7f1b97400700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b98077780 0x7f1b9819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:58.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.067+0000 7f1b9d000700 1 -- 172.21.15.63:0/3741105564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98076e30 msgr2=0x7f1b9819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:58.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.067+0000 7f1b9d000700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98076e30 0x7f1b9819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:58.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.067+0000 7f1b9d000700 1 -- 172.21.15.63:0/3741105564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b98077780 msgr2=0x7f1b9819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:58.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.067+0000 7f1b9d000700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b98077780 0x7f1b9819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:58.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.067+0000 7f1b9d000700 1 -- 172.21.15.63:0/3741105564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b981a1e30 con 0x7f1b98110b80 2024-05-27T07:27:58.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.067+0000 7f1b97400700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b98077780 0x7f1b9819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:27:58.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.067+0000 7f1b9d000700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b98110b80 0x7f1b981a1840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1b90007d40 tx=0x7f1b9000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:58.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.068+0000 7f1b94c00700 1 -- 172.21.15.63:0/3741105564 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90016020 con 0x7f1b98110b80 2024-05-27T07:27:58.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.068+0000 7f1b9f000700 1 -- 172.21.15.63:0/3741105564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b981a2120 con 0x7f1b98110b80 2024-05-27T07:27:58.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.068+0000 7f1b94c00700 1 -- 172.21.15.63:0/3741105564 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b9000ad40 con 0x7f1b98110b80 2024-05-27T07:27:58.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.068+0000 7f1b94c00700 1 -- 172.21.15.63:0/3741105564 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90012e60 con 0x7f1b98110b80 2024-05-27T07:27:58.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.068+0000 7f1b9f000700 1 -- 172.21.15.63:0/3741105564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b981a2660 con 0x7f1b98110b80 2024-05-27T07:27:58.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.069+0000 7f1b9f000700 1 -- 172.21.15.63:0/3741105564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b980512b0 con 0x7f1b98110b80 2024-05-27T07:27:58.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.073+0000 7f1b94c00700 1 -- 172.21.15.63:0/3741105564 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1b90019020 con 0x7f1b98110b80 2024-05-27T07:27:58.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.074+0000 7f1b94c00700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1b7c0b2900 0x7f1b7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:27:58.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.074+0000 7f1b94c00700 1 -- 172.21.15.63:0/3741105564 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f1b900e2080 con 0x7f1b98110b80 2024-05-27T07:27:58.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.075+0000 7f1b97e00700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1b7c0b2900 0x7f1b7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:27:58.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.075+0000 7f1b97e00700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1b7c0b2900 0x7f1b7c0b4e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1b88005950 tx=0x7f1b8800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:27:58.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.077+0000 7f1b94c00700 1 -- 172.21.15.63:0/3741105564 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1b90020020 con 0x7f1b98110b80 2024-05-27T07:27:58.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.365+0000 7f1b9f000700 1 -- 172.21.15.63:0/3741105564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b9802e5e0 con 0x7f1b98110b80 2024-05-27T07:27:58.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.367+0000 7f1b94c00700 1 -- 172.21.15.63:0/3741105564 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f1b900aa1a0 con 0x7f1b98110b80 2024-05-27T07:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34430,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34484,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T07:27:58.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.370+0000 7f1b9f000700 1 -- 172.21.15.63:0/3741105564 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1b7c0b2900 msgr2=0x7f1b7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:58.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.370+0000 7f1b9f000700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1b7c0b2900 0x7f1b7c0b4e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1b88005950 tx=0x7f1b8800a300 comp rx=0 tx=0).stop 2024-05-27T07:27:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.370+0000 7f1b9f000700 1 -- 172.21.15.63:0/3741105564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b98110b80 msgr2=0x7f1b981a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:27:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.370+0000 7f1b9f000700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b98110b80 0x7f1b981a1840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1b90007d40 tx=0x7f1b9000a340 comp rx=0 tx=0).stop 2024-05-27T07:27:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.371+0000 7f1b9f000700 1 -- 172.21.15.63:0/3741105564 shutdown_connections 2024-05-27T07:27:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.371+0000 7f1b9f000700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1b7c0b2900 0x7f1b7c0b4e10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.371+0000 7f1b9f000700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b98110b80 0x7f1b981a1840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.371+0000 7f1b9f000700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98076e30 0x7f1b9819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.371+0000 7f1b9f000700 1 --2- 172.21.15.63:0/3741105564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b98077780 0x7f1b9819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:27:58.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.371+0000 7f1b9f000700 1 -- 172.21.15.63:0/3741105564 >> 172.21.15.63:0/3741105564 conn(0x7f1b980ffee0 msgr2=0x7f1b981005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:27:58.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.371+0000 7f1b9f000700 1 -- 172.21.15.63:0/3741105564 shutdown_connections 2024-05-27T07:27:58.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:27:58.371+0000 7f1b9f000700 1 -- 172.21.15.63:0/3741105564 wait complete. 2024-05-27T07:27:58.363 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 11 2024-05-27T07:27:58.856 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:58.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:58.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:58.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.63:6838/3288686403,v1:172.21.15.63:6839/3288686403] up:boot 2024-05-27T07:27:58.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: fsmap 10 up:standby 2024-05-27T07:27:58.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:27:58.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:58.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:58.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:58.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:58.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: Deploying daemon mds.a on smithi063 2024-05-27T07:27:58.858 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:27:58 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3741105564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:59.118 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:59.118 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:59.118 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:59.119 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.63:6838/3288686403,v1:172.21.15.63:6839/3288686403] up:boot 2024-05-27T07:27:59.119 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: fsmap 10 up:standby 2024-05-27T07:27:59.119 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:27:59.119 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:59.119 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:59.119 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:59.119 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:59.119 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: Deploying daemon mds.a on smithi063 2024-05-27T07:27:59.119 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:27:58 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3741105564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:27:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:27:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.63:6838/3288686403,v1:172.21.15.63:6839/3288686403] up:boot 2024-05-27T07:27:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: fsmap 10 up:standby 2024-05-27T07:27:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:27:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:27:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:27:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:27:59.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:27:59.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: Deploying daemon mds.a on smithi063 2024-05-27T07:27:59.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:27:58 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3741105564' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:28:01.181 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:28:01.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:00 smithi086 ceph-mon[34687]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:01.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:00 smithi149 ceph-mon[35408]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:01.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:00 smithi063 ceph-mon[31613]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:01.674 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 07:28:01 smithi063 systemd[1]: Starting Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:28:01.956 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 07:28:01 smithi063 podman[73749]: 2024-05-27 07:28:01.589520268 +0000 UTC m=+0.035312512 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:28:01.956 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 07:28:01 smithi063 podman[73749]: 2024-05-27 07:28:01.719832099 +0000 UTC m=+0.165624380 container create c0c4c4595f4fdc70e441211190d87de97c026b8dcb4c40ba731f7948c104e528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-27T07:28:02.166 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:02.316 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 07:28:02 smithi063 podman[73749]: 2024-05-27 07:28:02.032152212 +0000 UTC m=+0.477944556 container init c0c4c4595f4fdc70e441211190d87de97c026b8dcb4c40ba731f7948c104e528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-27T07:28:02.316 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 07:28:02 smithi063 podman[73749]: 2024-05-27 07:28:02.043046666 +0000 UTC m=+0.488838954 container start c0c4c4595f4fdc70e441211190d87de97c026b8dcb4c40ba731f7948c104e528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:28:02.317 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 07:28:02 smithi063 bash[73749]: c0c4c4595f4fdc70e441211190d87de97c026b8dcb4c40ba731f7948c104e528 2024-05-27T07:28:02.317 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 07:28:02 smithi063 systemd[1]: Started Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:28:02.317 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 07:28:02 smithi063 ceph-mds[73796]: -- 172.21.15.63:0/3964018732 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556aa699da40 con 0x556aa69e5a80 2024-05-27T07:28:02.317 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 07:28:02 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: starting mds.a at 2024-05-27T07:28:02.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:02 smithi063 ceph-mon[31613]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:02.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:02 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:02.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:02 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:02.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:02 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:02.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:02 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:28:02.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:02 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:28:02.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:02 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:28:03.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:02 smithi086 ceph-mon[34687]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:03.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:02 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:03.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:02 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:03.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:02 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:03.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:02 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:28:03.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:02 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:28:03.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:02 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:28:03.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:02 smithi149 ceph-mon[35408]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:03.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:02 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:03.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:02 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:03.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:02 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:03.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:02 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:28:03.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:02 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:28:03.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:02 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:28:03.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:03 smithi086 ceph-mon[34687]: Deploying daemon mds.b on smithi086 2024-05-27T07:28:03.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:03 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.63:6840/80080490,v1:172.21.15.63:6841/80080490] up:boot 2024-05-27T07:28:03.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:03 smithi086 ceph-mon[34687]: fsmap 11 up:standby 2024-05-27T07:28:03.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:28:04.089 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:03 smithi063 ceph-mon[31613]: Deploying daemon mds.b on smithi086 2024-05-27T07:28:04.090 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:03 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.63:6840/80080490,v1:172.21.15.63:6841/80080490] up:boot 2024-05-27T07:28:04.090 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:03 smithi063 ceph-mon[31613]: fsmap 11 up:standby 2024-05-27T07:28:04.090 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:28:04.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:03 smithi149 ceph-mon[35408]: Deploying daemon mds.b on smithi086 2024-05-27T07:28:04.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:03 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.63:6840/80080490,v1:172.21.15.63:6841/80080490] up:boot 2024-05-27T07:28:04.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:03 smithi149 ceph-mon[35408]: fsmap 11 up:standby 2024-05-27T07:28:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:28:04.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.465+0000 7fbbb8400700 1 -- 172.21.15.63:0/183594738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb0106ec0 msgr2=0x7fbbb0113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:04.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.465+0000 7fbbb8400700 1 --2- 172.21.15.63:0/183594738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb0106ec0 0x7fbbb0113350 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009990 tx=0x7fbba8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:28:04.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.465+0000 7fbbb8400700 1 -- 172.21.15.63:0/183594738 shutdown_connections 2024-05-27T07:28:04.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.465+0000 7fbbb8400700 1 --2- 172.21.15.63:0/183594738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb0106ec0 0x7fbbb0113350 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:04.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.465+0000 7fbbb8400700 1 --2- 172.21.15.63:0/183594738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb010c5b0 0x7fbbb010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:04.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.465+0000 7fbbb8400700 1 --2- 172.21.15.63:0/183594738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106570 0x7fbbb0106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:04.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.465+0000 7fbbb8400700 1 -- 172.21.15.63:0/183594738 >> 172.21.15.63:0/183594738 conn(0x7fbbb0102070 msgr2=0x7fbbb01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:04.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.466+0000 7fbbb8400700 1 -- 172.21.15.63:0/183594738 shutdown_connections 2024-05-27T07:28:04.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.466+0000 7fbbb8400700 1 -- 172.21.15.63:0/183594738 wait complete. 2024-05-27T07:28:04.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.467+0000 7fbbb8400700 1 Processor -- start 2024-05-27T07:28:04.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.467+0000 7fbbb8400700 1 -- start start 2024-05-27T07:28:04.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.467+0000 7fbbb8400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106570 0x7fbbb019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:04.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.467+0000 7fbbb8400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb0106ec0 0x7fbbb019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:04.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.467+0000 7fbbb8400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb010c5b0 0x7fbbb01a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:04.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.467+0000 7fbbb8400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01165e0 con 0x7fbbb0106ec0 2024-05-27T07:28:04.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.467+0000 7fbbb8400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0116460 con 0x7fbbb010c5b0 2024-05-27T07:28:04.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.467+0000 7fbbb8400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0116760 con 0x7fbbb0106570 2024-05-27T07:28:04.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.468+0000 7fbbb5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb0106ec0 0x7fbbb019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:04.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.468+0000 7fbbb5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb0106ec0 0x7fbbb019d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56468/0 (socket says 172.21.15.63:56468) 2024-05-27T07:28:04.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.468+0000 7fbbb5000700 1 -- 172.21.15.63:0/2955286348 learned_addr learned my addr 172.21.15.63:0/2955286348 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:04.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.468+0000 7fbbb6400700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb010c5b0 0x7fbbb01a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:04.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.468+0000 7fbbb5a00700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106570 0x7fbbb019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:04.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.468+0000 7fbbb5000700 1 -- 172.21.15.63:0/2955286348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106570 msgr2=0x7fbbb019cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:04.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.468+0000 7fbbb5000700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106570 0x7fbbb019cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:04.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.468+0000 7fbbb5000700 1 -- 172.21.15.63:0/2955286348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb010c5b0 msgr2=0x7fbbb01a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:04.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.468+0000 7fbbb5000700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb010c5b0 0x7fbbb01a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:04.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.468+0000 7fbbb5000700 1 -- 172.21.15.63:0/2955286348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb01a1e80 con 0x7fbbb0106ec0 2024-05-27T07:28:04.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.469+0000 7fbbb5000700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb0106ec0 0x7fbbb019d550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00c8c0 tx=0x7fbb9c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:04.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.469+0000 7fbbae000700 1 -- 172.21.15.63:0/2955286348 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c004220 con 0x7fbbb0106ec0 2024-05-27T07:28:04.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.469+0000 7fbbb8400700 1 -- 172.21.15.63:0/2955286348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb01a2170 con 0x7fbbb0106ec0 2024-05-27T07:28:04.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.469+0000 7fbbae000700 1 -- 172.21.15.63:0/2955286348 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb9c007d90 con 0x7fbbb0106ec0 2024-05-27T07:28:04.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.469+0000 7fbbae000700 1 -- 172.21.15.63:0/2955286348 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c004e30 con 0x7fbbb0106ec0 2024-05-27T07:28:04.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.469+0000 7fbbb8400700 1 -- 172.21.15.63:0/2955286348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb01a9a90 con 0x7fbbb0106ec0 2024-05-27T07:28:04.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.470+0000 7fbbb8400700 1 -- 172.21.15.63:0/2955286348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb00512b0 con 0x7fbbb0106ec0 2024-05-27T07:28:04.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.473+0000 7fbbae000700 1 -- 172.21.15.63:0/2955286348 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbb9c0043c0 con 0x7fbbb0106ec0 2024-05-27T07:28:04.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.474+0000 7fbbae000700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb980b2880 0x7fbb980b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.474+0000 7fbbae000700 1 -- 172.21.15.63:0/2955286348 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fbb9c0e1d50 con 0x7fbbb0106ec0 2024-05-27T07:28:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.474+0000 7fbbb5a00700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb980b2880 0x7fbb980b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.475+0000 7fbbb5a00700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb980b2880 0x7fbb980b4d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbba4007700 tx=0x7fbba4007690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:04.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.477+0000 7fbbae000700 1 -- 172.21.15.63:0/2955286348 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb9c014030 con 0x7fbbb0106ec0 2024-05-27T07:28:04.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.762+0000 7fbbb8400700 1 -- 172.21.15.63:0/2955286348 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbb019e310 con 0x7fbbb0106ec0 2024-05-27T07:28:04.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.763+0000 7fbbae000700 1 -- 172.21.15.63:0/2955286348 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9647 (secure 0 0 0) 0x7fbb9c0a9e70 con 0x7fbbb0106ec0 2024-05-27T07:28:04.754 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:28:04.754 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34430,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34484,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T07:28:04.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.766+0000 7fbbb8400700 1 -- 172.21.15.63:0/2955286348 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb980b2880 msgr2=0x7fbb980b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:04.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.766+0000 7fbbb8400700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb980b2880 0x7fbb980b4d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbba4007700 tx=0x7fbba4007690 comp rx=0 tx=0).stop 2024-05-27T07:28:04.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.766+0000 7fbbb8400700 1 -- 172.21.15.63:0/2955286348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb0106ec0 msgr2=0x7fbbb019d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:04.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.766+0000 7fbbb8400700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb0106ec0 0x7fbbb019d550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00c8c0 tx=0x7fbb9c00cc80 comp rx=0 tx=0).stop 2024-05-27T07:28:04.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.766+0000 7fbbb8400700 1 -- 172.21.15.63:0/2955286348 shutdown_connections 2024-05-27T07:28:04.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.767+0000 7fbbb8400700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb010c5b0 0x7fbbb01a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:04.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.767+0000 7fbbb8400700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106570 0x7fbbb019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:04.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.767+0000 7fbbb8400700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb980b2880 0x7fbb980b4d90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:04.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.767+0000 7fbbb8400700 1 --2- 172.21.15.63:0/2955286348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb0106ec0 0x7fbbb019d550 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:04.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.767+0000 7fbbb8400700 1 -- 172.21.15.63:0/2955286348 >> 172.21.15.63:0/2955286348 conn(0x7fbbb0102070 msgr2=0x7fbbb01047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:04.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.767+0000 7fbbb8400700 1 -- 172.21.15.63:0/2955286348 shutdown_connections 2024-05-27T07:28:04.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:04.767+0000 7fbbb8400700 1 -- 172.21.15.63:0/2955286348 wait complete. 2024-05-27T07:28:04.759 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 12 2024-05-27T07:28:05.081 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:04 smithi086 ceph-mon[34687]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:05.081 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:04 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2955286348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:28:05.147 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:04 smithi063 ceph-mon[31613]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:05.147 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:04 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2955286348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:28:05.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:04 smithi149 ceph-mon[35408]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:05.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:04 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2955286348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:28:05.698 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:05 smithi086 systemd[1]: Starting Ceph mds.b for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:28:06.082 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:05 smithi086 podman[65087]: 2024-05-27 07:28:05.68704492 +0000 UTC m=+0.035662762 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:28:06.082 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:05 smithi086 podman[65087]: 2024-05-27 07:28:05.804728827 +0000 UTC m=+0.153346705 container create db608efef4d2041a0e5f89adf2077e1ec25a948b9a372f4cc4de1a88ea3a91c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T07:28:06.082 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:06 smithi086 podman[65087]: 2024-05-27 07:28:06.091904595 +0000 UTC m=+0.440522503 container init db608efef4d2041a0e5f89adf2077e1ec25a948b9a372f4cc4de1a88ea3a91c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-27T07:28:06.449 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:06 smithi086 podman[65087]: 2024-05-27 07:28:06.104647802 +0000 UTC m=+0.453265641 container start db608efef4d2041a0e5f89adf2077e1ec25a948b9a372f4cc4de1a88ea3a91c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:28:06.449 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:06 smithi086 bash[65087]: db608efef4d2041a0e5f89adf2077e1ec25a948b9a372f4cc4de1a88ea3a91c8 2024-05-27T07:28:06.449 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:06 smithi086 systemd[1]: Started Ceph mds.b for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:28:06.449 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:06 smithi086 ceph-mds[65107]: -- 172.21.15.86:0/2739781353 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56117ac0c4e0 con 0x56117ac57180 2024-05-27T07:28:06.449 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:06 smithi086 ceph-mds[65107]: -- 172.21.15.86:0/2739781353 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56117ac0fa40 con 0x56117ac57180 2024-05-27T07:28:06.449 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:06 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b[65103]: starting mds.b at 2024-05-27T07:28:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:06 smithi086 ceph-mon[34687]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:06 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:06 smithi149 ceph-mon[35408]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:06 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:06 smithi063 ceph-mon[31613]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:06 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.576 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:28:07.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:07 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] up:boot 2024-05-27T07:28:07.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:07 smithi063 ceph-mon[31613]: fsmap 12 up:standby 2024-05-27T07:28:07.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:28:07.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:28:07.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:07.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:07 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:08.048 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:08.169 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:07 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] up:boot 2024-05-27T07:28:08.170 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:07 smithi086 ceph-mon[34687]: fsmap 12 up:standby 2024-05-27T07:28:08.170 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:28:08.170 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:28:08.170 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:08.170 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:07 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:07 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] up:boot 2024-05-27T07:28:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:07 smithi149 ceph-mon[35408]: fsmap 12 up:standby 2024-05-27T07:28:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:28:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:28:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:08.174 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:07 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:09.125 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:08 smithi063 ceph-mon[31613]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:09.125 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:08 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:09.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:08 smithi086 ceph-mon[34687]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:09.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:08 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:09.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:08 smithi149 ceph-mon[35408]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:09.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:08 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:10.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.496+0000 7faa75000700 1 -- 172.21.15.63:0/1435479862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa70107c00 msgr2=0x7faa70108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:10.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.496+0000 7faa75000700 1 --2- 172.21.15.63:0/1435479862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa70107c00 0x7faa70108010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0099c0 tx=0x7faa5c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.496+0000 7faa75000700 1 -- 172.21.15.63:0/1435479862 shutdown_connections 2024-05-27T07:28:10.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.496+0000 7faa75000700 1 --2- 172.21.15.63:0/1435479862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa70108550 0x7faa7010feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.496+0000 7faa75000700 1 --2- 172.21.15.63:0/1435479862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa701032d0 0x7faa701036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.496+0000 7faa75000700 1 --2- 172.21.15.63:0/1435479862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa70107c00 0x7faa70108010 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.496+0000 7faa75000700 1 -- 172.21.15.63:0/1435479862 >> 172.21.15.63:0/1435479862 conn(0x7faa70078ab0 msgr2=0x7faa70078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:10.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.497+0000 7faa75000700 1 -- 172.21.15.63:0/1435479862 shutdown_connections 2024-05-27T07:28:10.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.497+0000 7faa75000700 1 -- 172.21.15.63:0/1435479862 wait complete. 2024-05-27T07:28:10.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.498+0000 7faa75000700 1 Processor -- start 2024-05-27T07:28:10.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.498+0000 7faa75000700 1 -- start start 2024-05-27T07:28:10.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa75000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa701032d0 0x7faa70100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:10.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa75000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa70107c00 0x7faa70100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:10.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa75000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa70108550 0x7faa70100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:10.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa75000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa70113060 con 0x7faa701032d0 2024-05-27T07:28:10.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa75000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa70112ee0 con 0x7faa70107c00 2024-05-27T07:28:10.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa75000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa701131e0 con 0x7faa70108550 2024-05-27T07:28:10.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa6f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa701032d0 0x7faa70100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:10.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa6f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa701032d0 0x7faa70100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60326/0 (socket says 172.21.15.63:60326) 2024-05-27T07:28:10.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa6f400700 1 -- 172.21.15.63:0/1952375768 learned_addr learned my addr 172.21.15.63:0/1952375768 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:10.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa6ea00700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa70107c00 0x7faa70100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:10.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.499+0000 7faa6fe00700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa70108550 0x7faa70100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:10.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.500+0000 7faa6f400700 1 -- 172.21.15.63:0/1952375768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa70108550 msgr2=0x7faa70100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:10.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.500+0000 7faa6f400700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa70108550 0x7faa70100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.500+0000 7faa6f400700 1 -- 172.21.15.63:0/1952375768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa70107c00 msgr2=0x7faa70100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:10.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.500+0000 7faa6f400700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa70107c00 0x7faa70100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.500+0000 7faa6f400700 1 -- 172.21.15.63:0/1952375768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa701a8f00 con 0x7faa701032d0 2024-05-27T07:28:10.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.500+0000 7faa6f400700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa701032d0 0x7faa70100530 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faa600060b0 tx=0x7faa6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:10.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.500+0000 7faa67e00700 1 -- 172.21.15.63:0/1952375768 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6001a070 con 0x7faa701032d0 2024-05-27T07:28:10.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.501+0000 7faa67e00700 1 -- 172.21.15.63:0/1952375768 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa6000dd20 con 0x7faa701032d0 2024-05-27T07:28:10.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.501+0000 7faa75000700 1 -- 172.21.15.63:0/1952375768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa701a91f0 con 0x7faa701032d0 2024-05-27T07:28:10.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.501+0000 7faa67e00700 1 -- 172.21.15.63:0/1952375768 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60014600 con 0x7faa701032d0 2024-05-27T07:28:10.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.501+0000 7faa75000700 1 -- 172.21.15.63:0/1952375768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa701a9750 con 0x7faa701032d0 2024-05-27T07:28:10.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.502+0000 7faa75000700 1 -- 172.21.15.63:0/1952375768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa700512b0 con 0x7faa701032d0 2024-05-27T07:28:10.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.506+0000 7faa67e00700 1 -- 172.21.15.63:0/1952375768 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7faa6001d070 con 0x7faa701032d0 2024-05-27T07:28:10.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.508+0000 7faa67e00700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa540b2900 0x7faa540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:10.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.508+0000 7faa67e00700 1 -- 172.21.15.63:0/1952375768 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7faa600e1710 con 0x7faa701032d0 2024-05-27T07:28:10.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.508+0000 7faa6ea00700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa540b2900 0x7faa540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:10.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.509+0000 7faa6ea00700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa540b2900 0x7faa540b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009990 tx=0x7faa5c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:10.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.511+0000 7faa67e00700 1 -- 172.21.15.63:0/1952375768 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa60010070 con 0x7faa701032d0 2024-05-27T07:28:10.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.814+0000 7faa75000700 1 -- 172.21.15.63:0/1952375768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa7002e5e0 con 0x7faa701032d0 2024-05-27T07:28:10.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.816+0000 7faa67e00700 1 -- 172.21.15.63:0/1952375768 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7faa600a91e0 con 0x7faa701032d0 2024-05-27T07:28:10.806 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:28:10.807 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34430,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34484,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34520,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-27T07:28:10.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 -- 172.21.15.63:0/1952375768 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa540b2900 msgr2=0x7faa540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:10.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa540b2900 0x7faa540b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009990 tx=0x7faa5c005fb0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 -- 172.21.15.63:0/1952375768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa701032d0 msgr2=0x7faa70100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:10.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa701032d0 0x7faa70100530 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faa600060b0 tx=0x7faa6000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 -- 172.21.15.63:0/1952375768 shutdown_connections 2024-05-27T07:28:10.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa70108550 0x7faa70100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa540b2900 0x7faa540b4e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa701032d0 0x7faa70100530 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 --2- 172.21.15.63:0/1952375768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa70107c00 0x7faa70100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:10.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 -- 172.21.15.63:0/1952375768 >> 172.21.15.63:0/1952375768 conn(0x7faa70078ab0 msgr2=0x7faa7006b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:10.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 -- 172.21.15.63:0/1952375768 shutdown_connections 2024-05-27T07:28:10.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:10.819+0000 7faa75000700 1 -- 172.21.15.63:0/1952375768 wait complete. 2024-05-27T07:28:10.811 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 13 2024-05-27T07:28:11.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:10 smithi086 ceph-mon[34687]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:11.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:10 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:10 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:10 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:10 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:10 smithi149 ceph-mon[35408]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:10 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:10 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:10 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:10 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:10 smithi063 ceph-mon[31613]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:11.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:10 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:10 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:10 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:10 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:11.717 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-27T07:28:11.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:11 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1952375768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:28:12.151 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:12.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:11 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1952375768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:28:12.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:11 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1952375768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:28:12.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:12 smithi063 ceph-mon[31613]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:12.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:12.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:12 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:13.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:12 smithi086 ceph-mon[34687]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:13.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:13.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:12 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:13.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:12 smithi149 ceph-mon[35408]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:13.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:13.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:12 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.425+0000 7f292ac00700 1 -- 172.21.15.63:0/1083564645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2924103540 msgr2=0x7f2924103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.425+0000 7f292ac00700 1 --2- 172.21.15.63:0/1083564645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2924103540 0x7f2924103950 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f29140099c0 tx=0x7f2914009cd0 comp rx=0 tx=0).stop 2024-05-27T07:28:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.425+0000 7f292ac00700 1 -- 172.21.15.63:0/1083564645 shutdown_connections 2024-05-27T07:28:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.425+0000 7f292ac00700 1 --2- 172.21.15.63:0/1083564645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924108770 0x7f29241100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.425+0000 7f292ac00700 1 --2- 172.21.15.63:0/1083564645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2924103540 0x7f2924103950 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.425+0000 7f292ac00700 1 --2- 172.21.15.63:0/1083564645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2924107e20 0x7f2924108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.425+0000 7f292ac00700 1 -- 172.21.15.63:0/1083564645 >> 172.21.15.63:0/1083564645 conn(0x7f2924078df0 msgr2=0x7f2924079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.425+0000 7f292ac00700 1 -- 172.21.15.63:0/1083564645 shutdown_connections 2024-05-27T07:28:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.425+0000 7f292ac00700 1 -- 172.21.15.63:0/1083564645 wait complete. 2024-05-27T07:28:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.427+0000 7f292ac00700 1 Processor -- start 2024-05-27T07:28:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.427+0000 7f292ac00700 1 -- start start 2024-05-27T07:28:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f292ac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2924103540 0x7f292419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f292ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924107e20 0x7f292419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f292ac00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2924108770 0x7f2924199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f292ac00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29241132a0 con 0x7f2924108770 2024-05-27T07:28:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f292ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2924113120 con 0x7f2924107e20 2024-05-27T07:28:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f292ac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2924113420 con 0x7f2924103540 2024-05-27T07:28:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f2928c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2924108770 0x7f2924199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f2923e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2924103540 0x7f292419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f2923400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924107e20 0x7f292419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f2923e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2924103540 0x7f292419f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53738/0 (socket says 172.21.15.63:53738) 2024-05-27T07:28:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f2928c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2924108770 0x7f2924199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60334/0 (socket says 172.21.15.63:60334) 2024-05-27T07:28:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.428+0000 7f2923e00700 1 -- 172.21.15.63:0/1941161796 learned_addr learned my addr 172.21.15.63:0/1941161796 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.429+0000 7f2923e00700 1 -- 172.21.15.63:0/1941161796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924107e20 msgr2=0x7f292419fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.429+0000 7f2923e00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924107e20 0x7f292419fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.429+0000 7f2923e00700 1 -- 172.21.15.63:0/1941161796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2924108770 msgr2=0x7f2924199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.429+0000 7f2923e00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2924108770 0x7f2924199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.429+0000 7f2923e00700 1 -- 172.21.15.63:0/1941161796 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292419a1e0 con 0x7f2924103540 2024-05-27T07:28:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.429+0000 7f2928c00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2924108770 0x7f2924199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:28:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.429+0000 7f2923e00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2924103540 0x7f292419f8c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2914006010 tx=0x7f2914005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.429+0000 7f2920c00700 1 -- 172.21.15.63:0/1941161796 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914026070 con 0x7f2924103540 2024-05-27T07:28:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.430+0000 7f292ac00700 1 -- 172.21.15.63:0/1941161796 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292419a470 con 0x7f2924103540 2024-05-27T07:28:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.430+0000 7f2920c00700 1 -- 172.21.15.63:0/1941161796 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2914005680 con 0x7f2924103540 2024-05-27T07:28:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.430+0000 7f2920c00700 1 -- 172.21.15.63:0/1941161796 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291400faa0 con 0x7f2924103540 2024-05-27T07:28:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.430+0000 7f292ac00700 1 -- 172.21.15.63:0/1941161796 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29240759e0 con 0x7f2924103540 2024-05-27T07:28:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.431+0000 7f292ac00700 1 -- 172.21.15.63:0/1941161796 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29241049a0 con 0x7f2924103540 2024-05-27T07:28:14.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.434+0000 7f2920c00700 1 -- 172.21.15.63:0/1941161796 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2914029070 con 0x7f2924103540 2024-05-27T07:28:14.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.436+0000 7f2920c00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f29080b2900 0x7f29080b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:14.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.436+0000 7f2920c00700 1 -- 172.21.15.63:0/1941161796 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f29140e2d40 con 0x7f2924103540 2024-05-27T07:28:14.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.436+0000 7f2923400700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f29080b2900 0x7f29080b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.437+0000 7f2923400700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f29080b2900 0x7f29080b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2924079320 tx=0x7f2910010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:14.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.441+0000 7f2920c00700 1 -- 172.21.15.63:0/1941161796 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2914030080 con 0x7f2924103540 2024-05-27T07:28:14.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.687+0000 7f292ac00700 1 -- 172.21.15.63:0/1941161796 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f29240512b0 con 0x7f2924103540 2024-05-27T07:28:14.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.689+0000 7f2920c00700 1 -- 172.21.15.63:0/1941161796 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f2914013070 con 0x7f2924103540 2024-05-27T07:28:14.679 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:28:14.679 INFO:teuthology.orchestra.run.smithi063.stdout:[ 2024-05-27T07:28:14.679 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T07:28:14.680 INFO:teuthology.orchestra.run.smithi063.stdout: "poolnum": 1, 2024-05-27T07:28:14.680 INFO:teuthology.orchestra.run.smithi063.stdout: "poolname": ".mgr" 2024-05-27T07:28:14.680 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:28:14.680 INFO:teuthology.orchestra.run.smithi063.stdout:] 2024-05-27T07:28:14.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.692+0000 7f292ac00700 1 -- 172.21.15.63:0/1941161796 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f29080b2900 msgr2=0x7f29080b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:14.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.692+0000 7f292ac00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f29080b2900 0x7f29080b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2924079320 tx=0x7f2910010040 comp rx=0 tx=0).stop 2024-05-27T07:28:14.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.692+0000 7f292ac00700 1 -- 172.21.15.63:0/1941161796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2924103540 msgr2=0x7f292419f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:14.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.692+0000 7f292ac00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2924103540 0x7f292419f8c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2914006010 tx=0x7f2914005280 comp rx=0 tx=0).stop 2024-05-27T07:28:14.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.693+0000 7f292ac00700 1 -- 172.21.15.63:0/1941161796 shutdown_connections 2024-05-27T07:28:14.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.693+0000 7f292ac00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f29080b2900 0x7f29080b4e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:14.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.693+0000 7f292ac00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2924108770 0x7f2924199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:14.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.693+0000 7f292ac00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2924103540 0x7f292419f8c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:14.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.693+0000 7f292ac00700 1 --2- 172.21.15.63:0/1941161796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2924107e20 0x7f292419fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:14.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.693+0000 7f292ac00700 1 -- 172.21.15.63:0/1941161796 >> 172.21.15.63:0/1941161796 conn(0x7f2924078df0 msgr2=0x7f2924100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:14.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.693+0000 7f292ac00700 1 -- 172.21.15.63:0/1941161796 shutdown_connections 2024-05-27T07:28:14.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:14.693+0000 7f292ac00700 1 -- 172.21.15.63:0/1941161796 wait complete. 2024-05-27T07:28:14.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:14 smithi063 ceph-mon[31613]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:14.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:28:14.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:28:14.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:14.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:14 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:28:14.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:14 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1941161796' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T07:28:15.069 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:14 smithi086 ceph-mon[34687]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:15.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:28:15.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:28:15.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:15.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:14 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:28:15.070 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:14 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1941161796' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T07:28:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:14 smithi149 ceph-mon[35408]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:28:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:28:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:14 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:28:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:14 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1941161796' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T07:28:15.503 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-27T07:28:15.504 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-27T07:28:16.301 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:16.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:16 smithi063 ceph-mon[31613]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:17.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:16 smithi086 ceph-mon[34687]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:17.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:16 smithi149 ceph-mon[35408]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:18.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:18 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:18 smithi063 ceph-mon[31613]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:18.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:18 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:18 smithi086 ceph-mon[34687]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:18.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:18 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:18.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:18 smithi149 ceph-mon[35408]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:18.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.980+0000 7fbe30600700 1 -- 172.21.15.63:0/1959076490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe281030f0 msgr2=0x7fbe28103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:18.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.980+0000 7fbe30600700 1 --2- 172.21.15.63:0/1959076490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe281030f0 0x7fbe28103500 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009990 tx=0x7fbe1c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:28:18.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.980+0000 7fbe30600700 1 -- 172.21.15.63:0/1959076490 shutdown_connections 2024-05-27T07:28:18.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.980+0000 7fbe30600700 1 --2- 172.21.15.63:0/1959076490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe28110ca0 0x7fbe281130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:18.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.980+0000 7fbe30600700 1 --2- 172.21.15.63:0/1959076490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe281030f0 0x7fbe28103500 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:18.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.980+0000 7fbe30600700 1 --2- 172.21.15.63:0/1959076490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe28103a40 0x7fbe28110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:18.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.980+0000 7fbe30600700 1 -- 172.21.15.63:0/1959076490 >> 172.21.15.63:0/1959076490 conn(0x7fbe280fedb0 msgr2=0x7fbe281011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:18.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.981+0000 7fbe30600700 1 -- 172.21.15.63:0/1959076490 shutdown_connections 2024-05-27T07:28:18.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.981+0000 7fbe30600700 1 -- 172.21.15.63:0/1959076490 wait complete. 2024-05-27T07:28:18.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.982+0000 7fbe30600700 1 Processor -- start 2024-05-27T07:28:18.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.982+0000 7fbe30600700 1 -- start start 2024-05-27T07:28:18.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.982+0000 7fbe30600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe281030f0 0x7fbe2819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:18.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.982+0000 7fbe30600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe28103a40 0x7fbe2819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:18.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.982+0000 7fbe30600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe28110ca0 0x7fbe281a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:18.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.982+0000 7fbe30600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28116500 con 0x7fbe281030f0 2024-05-27T07:28:18.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.982+0000 7fbe30600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28116380 con 0x7fbe28110ca0 2024-05-27T07:28:18.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.982+0000 7fbe30600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28116680 con 0x7fbe28103a40 2024-05-27T07:28:18.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.983+0000 7fbe2dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe281030f0 0x7fbe2819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:18.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.983+0000 7fbe2dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe281030f0 0x7fbe2819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47566/0 (socket says 172.21.15.63:47566) 2024-05-27T07:28:18.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.983+0000 7fbe2dc00700 1 -- 172.21.15.63:0/1927437136 learned_addr learned my addr 172.21.15.63:0/1927437136 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:18.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.983+0000 7fbe2d200700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe28103a40 0x7fbe2819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:18.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.983+0000 7fbe2dc00700 1 -- 172.21.15.63:0/1927437136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe28103a40 msgr2=0x7fbe2819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:18.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.984+0000 7fbe2dc00700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe28103a40 0x7fbe2819d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:18.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.984+0000 7fbe2dc00700 1 -- 172.21.15.63:0/1927437136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe28110ca0 msgr2=0x7fbe281a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:18.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.984+0000 7fbe2dc00700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe28110ca0 0x7fbe281a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:18.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.984+0000 7fbe2dc00700 1 -- 172.21.15.63:0/1927437136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe281a1ce0 con 0x7fbe281030f0 2024-05-27T07:28:18.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.984+0000 7fbe2dc00700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe281030f0 0x7fbe2819cf10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c000c00 tx=0x7fbe1c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:18.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.984+0000 7fbe26000700 1 -- 172.21.15.63:0/1927437136 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c027070 con 0x7fbe281030f0 2024-05-27T07:28:18.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.984+0000 7fbe26000700 1 -- 172.21.15.63:0/1927437136 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe1c00fd00 con 0x7fbe281030f0 2024-05-27T07:28:18.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.984+0000 7fbe30600700 1 -- 172.21.15.63:0/1927437136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe281a1f70 con 0x7fbe281030f0 2024-05-27T07:28:18.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.984+0000 7fbe26000700 1 -- 172.21.15.63:0/1927437136 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c022e20 con 0x7fbe281030f0 2024-05-27T07:28:18.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.985+0000 7fbe30600700 1 -- 172.21.15.63:0/1927437136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe281a2450 con 0x7fbe281030f0 2024-05-27T07:28:18.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.986+0000 7fbe30600700 1 -- 172.21.15.63:0/1927437136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe280512b0 con 0x7fbe281030f0 2024-05-27T07:28:18.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.988+0000 7fbe26000700 1 -- 172.21.15.63:0/1927437136 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbe1c02a070 con 0x7fbe281030f0 2024-05-27T07:28:18.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.990+0000 7fbe26000700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbe100b2880 0x7fbe100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:18.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.990+0000 7fbe26000700 1 -- 172.21.15.63:0/1927437136 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fbe1c0e2c60 con 0x7fbe281030f0 2024-05-27T07:28:18.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.990+0000 7fbe2d200700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbe100b2880 0x7fbe100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:18.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.991+0000 7fbe2d200700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbe100b2880 0x7fbe100b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe28104ac0 tx=0x7fbe1400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:18.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:18.995+0000 7fbe26000700 1 -- 172.21.15.63:0/1927437136 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe1c013070 con 0x7fbe281030f0 2024-05-27T07:28:19.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.238+0000 7fbe30600700 1 -- 172.21.15.63:0/1927437136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbe2802e5e0 con 0x7fbe281030f0 2024-05-27T07:28:19.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.362+0000 7fbe26000700 1 -- 172.21.15.63:0/1927437136 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7fbe1c0aae10 con 0x7fbe281030f0 2024-05-27T07:28:19.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 -- 172.21.15.63:0/1927437136 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbe100b2880 msgr2=0x7fbe100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:19.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbe100b2880 0x7fbe100b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe28104ac0 tx=0x7fbe1400a3b0 comp rx=0 tx=0).stop 2024-05-27T07:28:19.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 -- 172.21.15.63:0/1927437136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe281030f0 msgr2=0x7fbe2819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:19.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe281030f0 0x7fbe2819cf10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c000c00 tx=0x7fbe1c0047b0 comp rx=0 tx=0).stop 2024-05-27T07:28:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 -- 172.21.15.63:0/1927437136 shutdown_connections 2024-05-27T07:28:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe28110ca0 0x7fbe281a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbe100b2880 0x7fbe100b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe281030f0 0x7fbe2819cf10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 --2- 172.21.15.63:0/1927437136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe28103a40 0x7fbe2819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 -- 172.21.15.63:0/1927437136 >> 172.21.15.63:0/1927437136 conn(0x7fbe280fedb0 msgr2=0x7fbe281128d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:19.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 -- 172.21.15.63:0/1927437136 shutdown_connections 2024-05-27T07:28:19.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:19.366+0000 7fbe30600700 1 -- 172.21.15.63:0/1927437136 wait complete. 2024-05-27T07:28:19.359 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'cephfs_metadata' created 2024-05-27T07:28:19.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:19 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1927437136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T07:28:19.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:19.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:19.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:28:19.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:28:19.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:19.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:19 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1927437136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T07:28:19.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:19 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:19.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:19 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:19.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:19 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:28:19.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:19 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:28:19.703 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:19 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:19.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:19 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1927437136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T07:28:19.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:19.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:19.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:28:19.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:28:19.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:28:20.311 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-27T07:28:20.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:20 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1927437136' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T07:28:20.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:20 smithi086 ceph-mon[34687]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T07:28:20.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:20 smithi086 ceph-mon[34687]: pgmap v372: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:20.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:20 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1927437136' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T07:28:20.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:20 smithi149 ceph-mon[35408]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T07:28:20.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:20 smithi149 ceph-mon[35408]: pgmap v372: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:20.704 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:20 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1927437136' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T07:28:20.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:20 smithi063 ceph-mon[31613]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T07:28:20.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:20 smithi063 ceph-mon[31613]: pgmap v372: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:20.780 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:21.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:21 smithi063 ceph-mon[31613]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:28:21.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:21 smithi063 ceph-mon[31613]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T07:28:21.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:21 smithi086 ceph-mon[34687]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:28:21.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:21 smithi086 ceph-mon[34687]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T07:28:21.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:21 smithi149 ceph-mon[35408]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:28:21.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:21 smithi149 ceph-mon[35408]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T07:28:22.668 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:22 smithi063 ceph-mon[31613]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T07:28:22.668 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:22 smithi063 ceph-mon[31613]: pgmap v375: 33 pgs: 6 creating+peering, 13 active+clean, 14 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:22.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:22 smithi086 ceph-mon[34687]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T07:28:22.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:22 smithi086 ceph-mon[34687]: pgmap v375: 33 pgs: 6 creating+peering, 13 active+clean, 14 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:22.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:22 smithi149 ceph-mon[35408]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T07:28:22.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:22 smithi149 ceph-mon[35408]: pgmap v375: 33 pgs: 6 creating+peering, 13 active+clean, 14 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:23.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.072+0000 7f1442800700 1 -- 172.21.15.63:0/2346251143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f143c106590 msgr2=0x7f143c1069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:23.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.072+0000 7f1442800700 1 --2- 172.21.15.63:0/2346251143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f143c106590 0x7f143c1069a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1428009990 tx=0x7f1428009ca0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.073+0000 7f1442800700 1 -- 172.21.15.63:0/2346251143 shutdown_connections 2024-05-27T07:28:23.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.073+0000 7f1442800700 1 --2- 172.21.15.63:0/2346251143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c106ee0 0x7f143c113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.073+0000 7f1442800700 1 --2- 172.21.15.63:0/2346251143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c10c5d0 0x7f143c10c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.073+0000 7f1442800700 1 --2- 172.21.15.63:0/2346251143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f143c106590 0x7f143c1069a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.073+0000 7f1442800700 1 -- 172.21.15.63:0/2346251143 >> 172.21.15.63:0/2346251143 conn(0x7f143c1020b0 msgr2=0x7f143c104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:23.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.073+0000 7f1442800700 1 -- 172.21.15.63:0/2346251143 shutdown_connections 2024-05-27T07:28:23.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.073+0000 7f1442800700 1 -- 172.21.15.63:0/2346251143 wait complete. 2024-05-27T07:28:23.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.074+0000 7f1442800700 1 Processor -- start 2024-05-27T07:28:23.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.074+0000 7f1442800700 1 -- start start 2024-05-27T07:28:23.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.075+0000 7f1442800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c106590 0x7f143c19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:23.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.075+0000 7f1442800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c106ee0 0x7f143c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:23.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.075+0000 7f1442800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f143c10c5d0 0x7f143c1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:23.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.075+0000 7f1442800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1165b0 con 0x7f143c10c5d0 2024-05-27T07:28:23.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.075+0000 7f1442800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f143c116430 con 0x7f143c106590 2024-05-27T07:28:23.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.075+0000 7f1442800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f143c116730 con 0x7f143c106ee0 2024-05-27T07:28:23.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.076+0000 7f143be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f143c10c5d0 0x7f143c1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:23.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.076+0000 7f143b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c106590 0x7f143c19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:23.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.076+0000 7f143be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f143c10c5d0 0x7f143c1a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47588/0 (socket says 172.21.15.63:47588) 2024-05-27T07:28:23.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.076+0000 7f143be00700 1 -- 172.21.15.63:0/1267367578 learned_addr learned my addr 172.21.15.63:0/1267367578 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:23.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.076+0000 7f143aa00700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c106ee0 0x7f143c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:23.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.076+0000 7f143be00700 1 -- 172.21.15.63:0/1267367578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c106ee0 msgr2=0x7f143c19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:23.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.076+0000 7f143be00700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c106ee0 0x7f143c19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.076+0000 7f143be00700 1 -- 172.21.15.63:0/1267367578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c106590 msgr2=0x7f143c19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:23.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.076+0000 7f143be00700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c106590 0x7f143c19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.076+0000 7f143be00700 1 -- 172.21.15.63:0/1267367578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c1a1e70 con 0x7f143c10c5d0 2024-05-27T07:28:23.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.077+0000 7f143be00700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f143c10c5d0 0x7f143c1a17d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f143400ef30 tx=0x7f143400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:23.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.077+0000 7f1433e00700 1 -- 172.21.15.63:0/1267367578 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14340092e0 con 0x7f143c10c5d0 2024-05-27T07:28:23.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.077+0000 7f1442800700 1 -- 172.21.15.63:0/1267367578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143c1a2100 con 0x7f143c10c5d0 2024-05-27T07:28:23.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.077+0000 7f1433e00700 1 -- 172.21.15.63:0/1267367578 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1434004bb0 con 0x7f143c10c5d0 2024-05-27T07:28:23.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.077+0000 7f1433e00700 1 -- 172.21.15.63:0/1267367578 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434004020 con 0x7f143c10c5d0 2024-05-27T07:28:23.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.077+0000 7f1442800700 1 -- 172.21.15.63:0/1267367578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143c1a2610 con 0x7f143c10c5d0 2024-05-27T07:28:23.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.078+0000 7f1442800700 1 -- 172.21.15.63:0/1267367578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f143c0512b0 con 0x7f143c10c5d0 2024-05-27T07:28:23.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.081+0000 7f1433e00700 1 -- 172.21.15.63:0/1267367578 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f143401b030 con 0x7f143c10c5d0 2024-05-27T07:28:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.083+0000 7f1433e00700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f14200b2900 0x7f14200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.083+0000 7f1433e00700 1 -- 172.21.15.63:0/1267367578 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f14340e2450 con 0x7f143c10c5d0 2024-05-27T07:28:23.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.083+0000 7f143b400700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f14200b2900 0x7f14200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:23.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.084+0000 7f143b400700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f14200b2900 0x7f14200b4e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f142c005950 tx=0x7f142c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:23.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.086+0000 7f1433e00700 1 -- 172.21.15.63:0/1267367578 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1434022020 con 0x7f143c10c5d0 2024-05-27T07:28:23.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.323+0000 7f1442800700 1 -- 172.21.15.63:0/1267367578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f143c02e5e0 con 0x7f143c10c5d0 2024-05-27T07:28:23.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.428+0000 7f1433e00700 1 -- 172.21.15.63:0/1267367578 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v77) ==== 171+0+0 (secure 0 0 0) 0x7f1434010040 con 0x7f143c10c5d0 2024-05-27T07:28:23.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.433+0000 7f1442800700 1 -- 172.21.15.63:0/1267367578 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f14200b2900 msgr2=0x7f14200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:23.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.433+0000 7f1442800700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f14200b2900 0x7f14200b4e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f142c005950 tx=0x7f142c00b410 comp rx=0 tx=0).stop 2024-05-27T07:28:23.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.433+0000 7f1442800700 1 -- 172.21.15.63:0/1267367578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f143c10c5d0 msgr2=0x7f143c1a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:23.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.433+0000 7f1442800700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f143c10c5d0 0x7f143c1a17d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f143400ef30 tx=0x7f143400c3a0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.433+0000 7f1442800700 1 -- 172.21.15.63:0/1267367578 shutdown_connections 2024-05-27T07:28:23.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.433+0000 7f1442800700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f14200b2900 0x7f14200b4e10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.433+0000 7f1442800700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f143c10c5d0 0x7f143c1a17d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.433+0000 7f1442800700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143c106590 0x7f143c19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.433+0000 7f1442800700 1 --2- 172.21.15.63:0/1267367578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f143c106ee0 0x7f143c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:23.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.433+0000 7f1442800700 1 -- 172.21.15.63:0/1267367578 >> 172.21.15.63:0/1267367578 conn(0x7f143c1020b0 msgr2=0x7f143c104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:23.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.434+0000 7f1442800700 1 -- 172.21.15.63:0/1267367578 shutdown_connections 2024-05-27T07:28:23.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:23.434+0000 7f1442800700 1 -- 172.21.15.63:0/1267367578 wait complete. 2024-05-27T07:28:23.426 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'cephfs_data' created 2024-05-27T07:28:23.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:23 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1267367578' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-27T07:28:23.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:23 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1267367578' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-27T07:28:23.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:23 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1267367578' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-27T07:28:24.283 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-27T07:28:24.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:24 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1267367578' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-27T07:28:24.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:24 smithi063 ceph-mon[31613]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T07:28:24.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:24 smithi063 ceph-mon[31613]: pgmap v377: 97 pgs: 6 creating+peering, 18 active+clean, 73 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:24.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:24 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1267367578' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-27T07:28:24.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:24 smithi149 ceph-mon[35408]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T07:28:24.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:24 smithi149 ceph-mon[35408]: pgmap v377: 97 pgs: 6 creating+peering, 18 active+clean, 73 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:24.757 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:24.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:24 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1267367578' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-27T07:28:24.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:24 smithi086 ceph-mon[34687]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T07:28:24.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:24 smithi086 ceph-mon[34687]: pgmap v377: 97 pgs: 6 creating+peering, 18 active+clean, 73 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:25.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:25 smithi086 ceph-mon[34687]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T07:28:25.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:25 smithi149 ceph-mon[35408]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T07:28:25.706 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:25 smithi063 ceph-mon[31613]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T07:28:26.646 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:26 smithi063 ceph-mon[31613]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T07:28:26.646 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:26 smithi063 ceph-mon[31613]: pgmap v380: 97 pgs: 8 creating+peering, 45 active+clean, 44 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:26.951 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:26 smithi086 ceph-mon[34687]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T07:28:27.136 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:26 smithi086 ceph-mon[34687]: pgmap v380: 97 pgs: 8 creating+peering, 45 active+clean, 44 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:27.137 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:26 smithi149 ceph-mon[35408]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T07:28:27.137 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:26 smithi149 ceph-mon[35408]: pgmap v380: 97 pgs: 8 creating+peering, 45 active+clean, 44 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.258+0000 7fae93800700 1 -- 172.21.15.63:0/1167107873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae8c1045c0 msgr2=0x7fae8c1049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.258+0000 7fae93800700 1 --2- 172.21.15.63:0/1167107873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae8c1045c0 0x7fae8c1049d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae800099c0 tx=0x7fae80009cd0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.258+0000 7fae93800700 1 -- 172.21.15.63:0/1167107873 shutdown_connections 2024-05-27T07:28:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.258+0000 7fae93800700 1 --2- 172.21.15.63:0/1167107873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c109840 0x7fae8c1111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.258+0000 7fae93800700 1 --2- 172.21.15.63:0/1167107873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae8c1045c0 0x7fae8c1049d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.258+0000 7fae93800700 1 --2- 172.21.15.63:0/1167107873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c108ef0 0x7fae8c109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.258+0000 7fae93800700 1 -- 172.21.15.63:0/1167107873 >> 172.21.15.63:0/1167107873 conn(0x7fae8c0ffe90 msgr2=0x7fae8c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.258+0000 7fae93800700 1 -- 172.21.15.63:0/1167107873 shutdown_connections 2024-05-27T07:28:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.258+0000 7fae93800700 1 -- 172.21.15.63:0/1167107873 wait complete. 2024-05-27T07:28:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.259+0000 7fae93800700 1 Processor -- start 2024-05-27T07:28:27.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.259+0000 7fae93800700 1 -- start start 2024-05-27T07:28:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.260+0000 7fae93800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae8c1045c0 0x7fae8c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.260+0000 7fae93800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108ef0 0x7fae8c19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.260+0000 7fae93800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c109840 0x7fae8c1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:27.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.260+0000 7fae93800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c114320 con 0x7fae8c1045c0 2024-05-27T07:28:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.260+0000 7fae93800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1141a0 con 0x7fae8c109840 2024-05-27T07:28:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.260+0000 7fae93800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1144a0 con 0x7fae8c108ef0 2024-05-27T07:28:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.260+0000 7fae90e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae8c1045c0 0x7fae8c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.260+0000 7fae90e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae8c1045c0 0x7fae8c19cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47608/0 (socket says 172.21.15.63:47608) 2024-05-27T07:28:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.260+0000 7fae90e00700 1 -- 172.21.15.63:0/1939852651 learned_addr learned my addr 172.21.15.63:0/1939852651 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:27.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae8be00700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108ef0 0x7fae8c19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae91800700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c109840 0x7fae8c1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae90e00700 1 -- 172.21.15.63:0/1939852651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108ef0 msgr2=0x7fae8c19d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae90e00700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108ef0 0x7fae8c19d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae90e00700 1 -- 172.21.15.63:0/1939852651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c109840 msgr2=0x7fae8c1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae90e00700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c109840 0x7fae8c1a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae90e00700 1 -- 172.21.15.63:0/1939852651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c1a1dc0 con 0x7fae8c1045c0 2024-05-27T07:28:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae8be00700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108ef0 0x7fae8c19d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:28:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae91800700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c109840 0x7fae8c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:28:27.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae90e00700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae8c1045c0 0x7fae8c19cef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae80005f50 tx=0x7fae8000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:27.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae89600700 1 -- 172.21.15.63:0/1939852651 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80027070 con 0x7fae8c1045c0 2024-05-27T07:28:27.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae89600700 1 -- 172.21.15.63:0/1939852651 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae8000fdd0 con 0x7fae8c1045c0 2024-05-27T07:28:27.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae93800700 1 -- 172.21.15.63:0/1939852651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c1a2050 con 0x7fae8c1045c0 2024-05-27T07:28:27.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.261+0000 7fae89600700 1 -- 172.21.15.63:0/1939852651 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae800173f0 con 0x7fae8c1045c0 2024-05-27T07:28:27.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.262+0000 7fae93800700 1 -- 172.21.15.63:0/1939852651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c1a2530 con 0x7fae8c1045c0 2024-05-27T07:28:27.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.263+0000 7fae93800700 1 -- 172.21.15.63:0/1939852651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8c0512b0 con 0x7fae8c1045c0 2024-05-27T07:28:27.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.265+0000 7fae89600700 1 -- 172.21.15.63:0/1939852651 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fae8002a070 con 0x7fae8c1045c0 2024-05-27T07:28:27.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.267+0000 7fae89600700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae740b2880 0x7fae740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:27.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.267+0000 7fae89600700 1 -- 172.21.15.63:0/1939852651 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7fae800e31d0 con 0x7fae8c1045c0 2024-05-27T07:28:27.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.267+0000 7fae8be00700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae740b2880 0x7fae740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:27.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.268+0000 7fae8be00700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae740b2880 0x7fae740b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae78005fd0 tx=0x7fae78005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:27.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.270+0000 7fae89600700 1 -- 172.21.15.63:0/1939852651 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae80013070 con 0x7fae8c1045c0 2024-05-27T07:28:27.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.555+0000 7fae93800700 1 -- 172.21.15.63:0/1939852651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fae8c1a9a90 con 0x7fae8c1045c0 2024-05-27T07:28:27.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.612+0000 7fae89600700 1 -- 172.21.15.63:0/1939852651 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fae800ab090 con 0x7fae8c1045c0 2024-05-27T07:28:27.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.616+0000 7fae93800700 1 -- 172.21.15.63:0/1939852651 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae740b2880 msgr2=0x7fae740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:27.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.616+0000 7fae93800700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae740b2880 0x7fae740b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae78005fd0 tx=0x7fae78005ee0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.616+0000 7fae93800700 1 -- 172.21.15.63:0/1939852651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae8c1045c0 msgr2=0x7fae8c19cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:27.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.616+0000 7fae93800700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae8c1045c0 0x7fae8c19cef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae80005f50 tx=0x7fae8000f460 comp rx=0 tx=0).stop 2024-05-27T07:28:27.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.618+0000 7fae93800700 1 -- 172.21.15.63:0/1939852651 shutdown_connections 2024-05-27T07:28:27.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.618+0000 7fae93800700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c109840 0x7fae8c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.618+0000 7fae93800700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fae740b2880 0x7fae740b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.618+0000 7fae93800700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae8c1045c0 0x7fae8c19cef0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.618+0000 7fae93800700 1 --2- 172.21.15.63:0/1939852651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108ef0 0x7fae8c19d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:27.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.618+0000 7fae93800700 1 -- 172.21.15.63:0/1939852651 >> 172.21.15.63:0/1939852651 conn(0x7fae8c0ffe90 msgr2=0x7fae8c10a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:27.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.618+0000 7fae93800700 1 -- 172.21.15.63:0/1939852651 shutdown_connections 2024-05-27T07:28:27.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:27.618+0000 7fae93800700 1 -- 172.21.15.63:0/1939852651 wait complete. 2024-05-27T07:28:27.609 INFO:teuthology.orchestra.run.smithi063.stderr:new fs with metadata pool 2 and data pool 3 2024-05-27T07:28:27.872 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:27 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1939852651' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T07:28:27.872 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:27 smithi086 ceph-mon[34687]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T07:28:27.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:27 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1939852651' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T07:28:27.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:27 smithi149 ceph-mon[35408]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T07:28:27.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:27 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1939852651' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T07:28:27.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:27 smithi063 ceph-mon[31613]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T07:28:28.432 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-27T07:28:28.432 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-27T07:28:28.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:28 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1939852651' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T07:28:28.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:28 smithi063 ceph-mon[31613]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T07:28:28.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:28 smithi063 ceph-mon[31613]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T07:28:28.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:28 smithi063 ceph-mon[31613]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T07:28:28.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:28 smithi063 ceph-mon[31613]: pgmap v382: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:28.893 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:28 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1939852651' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T07:28:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:28 smithi086 ceph-mon[34687]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T07:28:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:28 smithi086 ceph-mon[34687]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T07:28:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:28 smithi086 ceph-mon[34687]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T07:28:28.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:28 smithi086 ceph-mon[34687]: pgmap v382: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:28.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:28 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1939852651' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T07:28:28.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:28 smithi149 ceph-mon[35408]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T07:28:28.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:28 smithi149 ceph-mon[35408]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T07:28:28.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:28 smithi149 ceph-mon[35408]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T07:28:28.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:28 smithi149 ceph-mon[35408]: pgmap v382: 97 pgs: 8 creating+peering, 89 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:29 smithi086 ceph-mon[34687]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T07:28:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:29 smithi086 ceph-mon[34687]: Cluster is now healthy 2024-05-27T07:28:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:29 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] up:active 2024-05-27T07:28:29.698 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:29 smithi086 ceph-mon[34687]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:28:29.699 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:29 smithi086 ceph-mon[34687]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:28:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:29 smithi063 ceph-mon[31613]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T07:28:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:29 smithi063 ceph-mon[31613]: Cluster is now healthy 2024-05-27T07:28:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:29 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] up:active 2024-05-27T07:28:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:29 smithi063 ceph-mon[31613]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:28:29.705 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:29 smithi063 ceph-mon[31613]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:28:29.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:29 smithi149 ceph-mon[35408]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T07:28:29.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:29 smithi149 ceph-mon[35408]: Cluster is now healthy 2024-05-27T07:28:29.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:29 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] up:active 2024-05-27T07:28:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:29 smithi149 ceph-mon[35408]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:28:29.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:29 smithi149 ceph-mon[35408]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:28:30.702 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:30 smithi149 ceph-mon[35408]: pgmap v383: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 4 op/s 2024-05-27T07:28:30.923 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:30 smithi063 ceph-mon[31613]: pgmap v383: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 4 op/s 2024-05-27T07:28:30.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:30 smithi086 ceph-mon[34687]: pgmap v383: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 4 op/s 2024-05-27T07:28:31.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.318+0000 7efe02e00700 1 -- 172.21.15.63:0/329917340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdfc10e6f0 msgr2=0x7efdfc110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:31.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.318+0000 7efe02e00700 1 --2- 172.21.15.63:0/329917340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdfc10e6f0 0x7efdfc110bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b0a0 tx=0x7efdf400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:28:31.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.318+0000 7efe02e00700 1 -- 172.21.15.63:0/329917340 shutdown_connections 2024-05-27T07:28:31.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.318+0000 7efe02e00700 1 --2- 172.21.15.63:0/329917340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdfc10e6f0 0x7efdfc110bd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:31.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.318+0000 7efe02e00700 1 --2- 172.21.15.63:0/329917340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdfc105280 0x7efdfc105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:31.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.318+0000 7efe02e00700 1 --2- 172.21.15.63:0/329917340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdfc105bd0 0x7efdfc10e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:31.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.318+0000 7efe02e00700 1 -- 172.21.15.63:0/329917340 >> 172.21.15.63:0/329917340 conn(0x7efdfc078ab0 msgr2=0x7efdfc078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.318+0000 7efe02e00700 1 -- 172.21.15.63:0/329917340 shutdown_connections 2024-05-27T07:28:31.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.318+0000 7efe02e00700 1 -- 172.21.15.63:0/329917340 wait complete. 2024-05-27T07:28:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.319+0000 7efe02e00700 1 Processor -- start 2024-05-27T07:28:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.320+0000 7efe02e00700 1 -- start start 2024-05-27T07:28:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.320+0000 7efe02e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdfc105280 0x7efdfc102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:31.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.320+0000 7efe02e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdfc105bd0 0x7efdfc1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.320+0000 7efe02e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdfc10e6f0 0x7efdfc1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.320+0000 7efe02e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc114150 con 0x7efdfc10e6f0 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.320+0000 7efe02e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc113fd0 con 0x7efdfc105280 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.320+0000 7efe02e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc1142d0 con 0x7efdfc105bd0 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.321+0000 7efe02400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdfc10e6f0 0x7efdfc1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.321+0000 7efe02400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdfc10e6f0 0x7efdfc1039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36610/0 (socket says 172.21.15.63:36610) 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.321+0000 7efe02400700 1 -- 172.21.15.63:0/403684897 learned_addr learned my addr 172.21.15.63:0/403684897 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.321+0000 7efe01a00700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdfc105280 0x7efdfc102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.321+0000 7efe01000700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdfc105bd0 0x7efdfc1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.321+0000 7efe02400700 1 -- 172.21.15.63:0/403684897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdfc105bd0 msgr2=0x7efdfc1034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.321+0000 7efe02400700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdfc105bd0 0x7efdfc1034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.321+0000 7efe02400700 1 -- 172.21.15.63:0/403684897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdfc105280 msgr2=0x7efdfc102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.321+0000 7efe02400700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdfc105280 0x7efdfc102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.321+0000 7efe02400700 1 -- 172.21.15.63:0/403684897 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc1a8ee0 con 0x7efdfc10e6f0 2024-05-27T07:28:31.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.322+0000 7efe02400700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdfc10e6f0 0x7efdfc1039f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b070 tx=0x7efdf4004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.322+0000 7efdfa000700 1 -- 172.21.15.63:0/403684897 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4021070 con 0x7efdfc10e6f0 2024-05-27T07:28:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.322+0000 7efdfa000700 1 -- 172.21.15.63:0/403684897 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdf4007b80 con 0x7efdfc10e6f0 2024-05-27T07:28:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.322+0000 7efe02e00700 1 -- 172.21.15.63:0/403684897 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc1a9170 con 0x7efdfc10e6f0 2024-05-27T07:28:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.322+0000 7efdfa000700 1 -- 172.21.15.63:0/403684897 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4004770 con 0x7efdfc10e6f0 2024-05-27T07:28:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.322+0000 7efe02e00700 1 -- 172.21.15.63:0/403684897 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc1a9620 con 0x7efdfc10e6f0 2024-05-27T07:28:31.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.323+0000 7efe02e00700 1 -- 172.21.15.63:0/403684897 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdfc0512b0 con 0x7efdfc10e6f0 2024-05-27T07:28:31.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.326+0000 7efdfa000700 1 -- 172.21.15.63:0/403684897 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efdf4003a00 con 0x7efdfc10e6f0 2024-05-27T07:28:31.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.328+0000 7efdfa000700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efde40b2900 0x7efde40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.329+0000 7efdfa000700 1 -- 172.21.15.63:0/403684897 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7efdf400f020 con 0x7efdfc10e6f0 2024-05-27T07:28:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.329+0000 7efe01a00700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efde40b2900 0x7efde40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.330+0000 7efe01a00700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efde40b2900 0x7efde40b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efdf0005950 tx=0x7efdf000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:31.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.331+0000 7efdfa000700 1 -- 172.21.15.63:0/403684897 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efdf402a020 con 0x7efdfc10e6f0 2024-05-27T07:28:31.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.579+0000 7efe02e00700 1 -- 172.21.15.63:0/403684897 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7efdfc02e7a0 con 0x7efdfc10e6f0 2024-05-27T07:28:31.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.659+0000 7efdfa000700 1 -- 172.21.15.63:0/403684897 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) ==== 157+0+0 (secure 0 0 0) 0x7efdf40ab890 con 0x7efdfc10e6f0 2024-05-27T07:28:31.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.663+0000 7efe02e00700 1 -- 172.21.15.63:0/403684897 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efde40b2900 msgr2=0x7efde40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:31.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.663+0000 7efe02e00700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efde40b2900 0x7efde40b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efdf0005950 tx=0x7efdf000b410 comp rx=0 tx=0).stop 2024-05-27T07:28:31.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.663+0000 7efe02e00700 1 -- 172.21.15.63:0/403684897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdfc10e6f0 msgr2=0x7efdfc1039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:31.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.663+0000 7efe02e00700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdfc10e6f0 0x7efdfc1039f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b070 tx=0x7efdf4004d50 comp rx=0 tx=0).stop 2024-05-27T07:28:31.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.664+0000 7efe02e00700 1 -- 172.21.15.63:0/403684897 shutdown_connections 2024-05-27T07:28:31.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.664+0000 7efe02e00700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efde40b2900 0x7efde40b4e10 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efdf0005950 tx=0x7efdf000b410 comp rx=0 tx=0).stop 2024-05-27T07:28:31.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.664+0000 7efe02e00700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdfc10e6f0 0x7efdfc1039f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:31.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.664+0000 7efe02e00700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdfc105280 0x7efdfc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:31.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.664+0000 7efe02e00700 1 --2- 172.21.15.63:0/403684897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdfc105bd0 0x7efdfc1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:31.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.664+0000 7efe02e00700 1 -- 172.21.15.63:0/403684897 >> 172.21.15.63:0/403684897 conn(0x7efdfc078ab0 msgr2=0x7efdfc0fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:31.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.665+0000 7efe02e00700 1 -- 172.21.15.63:0/403684897 shutdown_connections 2024-05-27T07:28:31.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:31.665+0000 7efe02e00700 1 -- 172.21.15.63:0/403684897 wait complete. 2024-05-27T07:28:31.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:31 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/403684897' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-27T07:28:31.948 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:28:31 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b[65103]: 2024-05-27T07:28:31.876+0000 7f8aa2600700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-27T07:28:31.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:31 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/403684897' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-27T07:28:31.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:31 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/403684897' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-27T07:28:32.547 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-27T07:28:32.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:32 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/403684897' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-27T07:28:32.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:32 smithi086 ceph-mon[34687]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T07:28:32.948 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:32 smithi086 ceph-mon[34687]: pgmap v385: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-27T07:28:32.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:32 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/403684897' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-27T07:28:32.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:32 smithi149 ceph-mon[35408]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T07:28:32.952 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:32 smithi149 ceph-mon[35408]: pgmap v385: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-27T07:28:32.954 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:32 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/403684897' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-27T07:28:32.954 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:32 smithi063 ceph-mon[31613]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T07:28:32.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:32 smithi063 ceph-mon[31613]: pgmap v385: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-27T07:28:33.000 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:34.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:34 smithi063 ceph-mon[31613]: pgmap v386: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:28:35.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:34 smithi086 ceph-mon[34687]: pgmap v386: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:28:35.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:34 smithi149 ceph-mon[35408]: pgmap v386: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:28:35.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.345+0000 7f7807200700 1 -- 172.21.15.63:0/2480893538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7808109670 msgr2=0x7f7808110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:35.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.345+0000 7f7807200700 1 --2- 172.21.15.63:0/2480893538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7808109670 0x7f7808110fd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f780000b0d0 tx=0x7f780000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:28:35.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.345+0000 7f7807200700 1 -- 172.21.15.63:0/2480893538 shutdown_connections 2024-05-27T07:28:35.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.345+0000 7f7807200700 1 --2- 172.21.15.63:0/2480893538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7808109670 0x7f7808110fd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:35.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.345+0000 7f7807200700 1 --2- 172.21.15.63:0/2480893538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78081043f0 0x7f7808104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:35.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.345+0000 7f7807200700 1 --2- 172.21.15.63:0/2480893538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7808108d20 0x7f7808109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:35.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.345+0000 7f7807200700 1 -- 172.21.15.63:0/2480893538 >> 172.21.15.63:0/2480893538 conn(0x7f7808079d00 msgr2=0x7f780807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:35.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.345+0000 7f7807200700 1 -- 172.21.15.63:0/2480893538 shutdown_connections 2024-05-27T07:28:35.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.346+0000 7f7807200700 1 -- 172.21.15.63:0/2480893538 wait complete. 2024-05-27T07:28:35.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.347+0000 7f7807200700 1 Processor -- start 2024-05-27T07:28:35.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.348+0000 7f7807200700 1 -- start start 2024-05-27T07:28:35.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.348+0000 7f7807200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78081043f0 0x7f780819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:35.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.348+0000 7f7807200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7808108d20 0x7f780819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:35.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.348+0000 7f7807200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7808109670 0x7f78081a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:35.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.348+0000 7f7807200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78081141a0 con 0x7f7808109670 2024-05-27T07:28:35.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.348+0000 7f7807200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7808114020 con 0x7f7808108d20 2024-05-27T07:28:35.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.348+0000 7f7807200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7808114320 con 0x7f78081043f0 2024-05-27T07:28:35.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.349+0000 7f7806800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7808109670 0x7f78081a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:35.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.349+0000 7f7806800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7808109670 0x7f78081a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36624/0 (socket says 172.21.15.63:36624) 2024-05-27T07:28:35.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.349+0000 7f7806800700 1 -- 172.21.15.63:0/1489328986 learned_addr learned my addr 172.21.15.63:0/1489328986 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:35.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.349+0000 7f7805e00700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78081043f0 0x7f780819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:35.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.349+0000 7f77ffe00700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7808108d20 0x7f780819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:35.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.349+0000 7f7806800700 1 -- 172.21.15.63:0/1489328986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78081043f0 msgr2=0x7f780819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:35.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.349+0000 7f7806800700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78081043f0 0x7f780819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:35.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.349+0000 7f7806800700 1 -- 172.21.15.63:0/1489328986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7808108d20 msgr2=0x7f780819d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:35.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.349+0000 7f7806800700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7808108d20 0x7f780819d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:35.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.349+0000 7f7806800700 1 -- 172.21.15.63:0/1489328986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78081a1be0 con 0x7f7808109670 2024-05-27T07:28:35.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.350+0000 7f7806800700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7808109670 0x7f78081a1540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7800007d10 tx=0x7f7800007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:35.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.350+0000 7f77fea00700 1 -- 172.21.15.63:0/1489328986 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780001f070 con 0x7f7808109670 2024-05-27T07:28:35.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.350+0000 7f77fea00700 1 -- 172.21.15.63:0/1489328986 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7800003e10 con 0x7f7808109670 2024-05-27T07:28:35.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.350+0000 7f7807200700 1 -- 172.21.15.63:0/1489328986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78081a1e70 con 0x7f7808109670 2024-05-27T07:28:35.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.350+0000 7f77fea00700 1 -- 172.21.15.63:0/1489328986 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780001b720 con 0x7f7808109670 2024-05-27T07:28:35.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.350+0000 7f7807200700 1 -- 172.21.15.63:0/1489328986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78081a97f0 con 0x7f7808109670 2024-05-27T07:28:35.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.352+0000 7f7807200700 1 -- 172.21.15.63:0/1489328986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78081058a0 con 0x7f7808109670 2024-05-27T07:28:35.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.356+0000 7f77fea00700 1 -- 172.21.15.63:0/1489328986 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7800022070 con 0x7f7808109670 2024-05-27T07:28:35.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.357+0000 7f77fea00700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f77ec0b2900 0x7f77ec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:35.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.358+0000 7f77fea00700 1 -- 172.21.15.63:0/1489328986 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8277+0+0 (secure 0 0 0) 0x7f780000f020 con 0x7f7808109670 2024-05-27T07:28:35.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.358+0000 7f7805e00700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f77ec0b2900 0x7f77ec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:35.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.359+0000 7f7805e00700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f77ec0b2900 0x7f77ec0b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005950 tx=0x7f77f800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:35.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.361+0000 7f77fea00700 1 -- 172.21.15.63:0/1489328986 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7800029020 con 0x7f7808109670 2024-05-27T07:28:35.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:35.601+0000 7f7807200700 1 -- 172.21.15.63:0/1489328986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f780819e070 con 0x7f7808109670 2024-05-27T07:28:35.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:35 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1489328986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T07:28:36.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:35 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1489328986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T07:28:36.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:35 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1489328986' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T07:28:36.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.856+0000 7f77fea00700 1 -- 172.21.15.63:0/1489328986 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) ==== 228+0+0 (secure 0 0 0) 0x7f78000ab780 con 0x7f7808109670 2024-05-27T07:28:36.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.861+0000 7f7807200700 1 -- 172.21.15.63:0/1489328986 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f77ec0b2900 msgr2=0x7f77ec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:36.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.861+0000 7f7807200700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f77ec0b2900 0x7f77ec0b4e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005950 tx=0x7f77f800b410 comp rx=0 tx=0).stop 2024-05-27T07:28:36.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.861+0000 7f7807200700 1 -- 172.21.15.63:0/1489328986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7808109670 msgr2=0x7f78081a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:36.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.861+0000 7f7807200700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7808109670 0x7f78081a1540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7800007d10 tx=0x7f7800007d40 comp rx=0 tx=0).stop 2024-05-27T07:28:36.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.862+0000 7f7807200700 1 -- 172.21.15.63:0/1489328986 shutdown_connections 2024-05-27T07:28:36.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.862+0000 7f7807200700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f77ec0b2900 0x7f77ec0b4e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:36.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.862+0000 7f7807200700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7808109670 0x7f78081a1540 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:36.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.862+0000 7f7807200700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78081043f0 0x7f780819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:36.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.862+0000 7f7807200700 1 --2- 172.21.15.63:0/1489328986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7808108d20 0x7f780819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:36.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.862+0000 7f7807200700 1 -- 172.21.15.63:0/1489328986 >> 172.21.15.63:0/1489328986 conn(0x7f7808079d00 msgr2=0x7f7808110cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:36.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.862+0000 7f7807200700 1 -- 172.21.15.63:0/1489328986 shutdown_connections 2024-05-27T07:28:36.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:36.862+0000 7f7807200700 1 -- 172.21.15.63:0/1489328986 wait complete. 2024-05-27T07:28:36.853 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'cephfs_data_ec' created 2024-05-27T07:28:37.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:36 smithi086 ceph-mon[34687]: pgmap v387: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:28:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:36 smithi086 ceph-mon[34687]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T07:28:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:36 smithi149 ceph-mon[35408]: pgmap v387: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:28:37.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:36 smithi149 ceph-mon[35408]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T07:28:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:36 smithi063 ceph-mon[31613]: pgmap v387: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:28:37.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:36 smithi063 ceph-mon[31613]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T07:28:37.881 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-27T07:28:38.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:37 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1489328986' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T07:28:38.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:37 smithi086 ceph-mon[34687]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T07:28:38.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:37 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1489328986' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T07:28:38.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:37 smithi149 ceph-mon[35408]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T07:28:38.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:37 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1489328986' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T07:28:38.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:37 smithi063 ceph-mon[31613]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T07:28:38.325 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:38 smithi086 ceph-mon[34687]: pgmap v390: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:39.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:38 smithi086 ceph-mon[34687]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:28:39.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:38 smithi086 ceph-mon[34687]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T07:28:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:38 smithi149 ceph-mon[35408]: pgmap v390: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:38 smithi149 ceph-mon[35408]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:28:39.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:38 smithi149 ceph-mon[35408]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T07:28:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:38 smithi063 ceph-mon[31613]: pgmap v390: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:38 smithi063 ceph-mon[31613]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:28:39.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:38 smithi063 ceph-mon[31613]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T07:28:40.104 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:39 smithi063 ceph-mon[31613]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T07:28:40.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:39 smithi086 ceph-mon[34687]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T07:28:40.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:39 smithi149 ceph-mon[35408]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T07:28:40.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.732+0000 7f99a9200700 1 -- 172.21.15.63:0/2628066925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a410b890 msgr2=0x7f99a41131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:40.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.732+0000 7f99a9200700 1 --2- 172.21.15.63:0/2628066925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a410b890 0x7f99a41131f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f999c00b0d0 tx=0x7f999c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:28:40.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.732+0000 7f99a9200700 1 -- 172.21.15.63:0/2628066925 shutdown_connections 2024-05-27T07:28:40.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.732+0000 7f99a9200700 1 --2- 172.21.15.63:0/2628066925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a410b890 0x7f99a41131f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:40.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.732+0000 7f99a9200700 1 --2- 172.21.15.63:0/2628066925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a4106610 0x7f99a4106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:40.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.732+0000 7f99a9200700 1 --2- 172.21.15.63:0/2628066925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a410af40 0x7f99a410b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:40.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.732+0000 7f99a9200700 1 -- 172.21.15.63:0/2628066925 >> 172.21.15.63:0/2628066925 conn(0x7f99a4102020 msgr2=0x7f99a4104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:40.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.732+0000 7f99a9200700 1 -- 172.21.15.63:0/2628066925 shutdown_connections 2024-05-27T07:28:40.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.732+0000 7f99a9200700 1 -- 172.21.15.63:0/2628066925 wait complete. 2024-05-27T07:28:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.734+0000 7f99a9200700 1 Processor -- start 2024-05-27T07:28:40.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.734+0000 7f99a9200700 1 -- start start 2024-05-27T07:28:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.734+0000 7f99a9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a4106610 0x7f99a419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.734+0000 7f99a9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a410af40 0x7f99a419d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.734+0000 7f99a9200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a410b890 0x7f99a41a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a9200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99a41163c0 con 0x7f99a4106610 2024-05-27T07:28:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a9200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4116240 con 0x7f99a410af40 2024-05-27T07:28:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a9200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4116540 con 0x7f99a410b890 2024-05-27T07:28:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a4106610 0x7f99a419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a4106610 0x7f99a419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34890/0 (socket says 172.21.15.63:34890) 2024-05-27T07:28:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a3400700 1 -- 172.21.15.63:0/2894634574 learned_addr learned my addr 172.21.15.63:0/2894634574 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:40.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a2a00700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a410af40 0x7f99a419d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:40.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a3e00700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a410b890 0x7f99a41a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:40.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a3400700 1 -- 172.21.15.63:0/2894634574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a410b890 msgr2=0x7f99a41a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:40.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a3400700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a410b890 0x7f99a41a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:40.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a3400700 1 -- 172.21.15.63:0/2894634574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a410af40 msgr2=0x7f99a419d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:40.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a3400700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a410af40 0x7f99a419d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:40.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.735+0000 7f99a3400700 1 -- 172.21.15.63:0/2894634574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a41a1b30 con 0x7f99a4106610 2024-05-27T07:28:40.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.736+0000 7f99a3400700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a4106610 0x7f99a419cd10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f999400c9b0 tx=0x7f999400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:40.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.736+0000 7f999be00700 1 -- 172.21.15.63:0/2894634574 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994007b00 con 0x7f99a4106610 2024-05-27T07:28:40.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.736+0000 7f999be00700 1 -- 172.21.15.63:0/2894634574 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9994007ca0 con 0x7f99a4106610 2024-05-27T07:28:40.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.736+0000 7f99a9200700 1 -- 172.21.15.63:0/2894634574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a41a1e20 con 0x7f99a4106610 2024-05-27T07:28:40.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.736+0000 7f999be00700 1 -- 172.21.15.63:0/2894634574 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994018660 con 0x7f99a4106610 2024-05-27T07:28:40.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.736+0000 7f99a9200700 1 -- 172.21.15.63:0/2894634574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a41a2360 con 0x7f99a4106610 2024-05-27T07:28:40.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.738+0000 7f99a9200700 1 -- 172.21.15.63:0/2894634574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a40512b0 con 0x7f99a4106610 2024-05-27T07:28:40.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.740+0000 7f999be00700 1 -- 172.21.15.63:0/2894634574 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f999400f450 con 0x7f99a4106610 2024-05-27T07:28:40.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.742+0000 7f999be00700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f99880b2880 0x7f99880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:40.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.742+0000 7f999be00700 1 -- 172.21.15.63:0/2894634574 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f99940e2820 con 0x7f99a4106610 2024-05-27T07:28:40.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.742+0000 7f99a2a00700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f99880b2880 0x7f99880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:40.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.743+0000 7f99a2a00700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f99880b2880 0x7f99880b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9990005950 tx=0x7f99900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:40.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.745+0000 7f999be00700 1 -- 172.21.15.63:0/2894634574 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9994014030 con 0x7f99a4106610 2024-05-27T07:28:40.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:40.981+0000 7f99a9200700 1 -- 172.21.15.63:0/2894634574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f99a402e5e0 con 0x7f99a4106610 2024-05-27T07:28:41.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:40 smithi086 ceph-mon[34687]: pgmap v393: 129 pgs: 3 creating+peering, 24 unknown, 102 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:41.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:40 smithi149 ceph-mon[35408]: pgmap v393: 129 pgs: 3 creating+peering, 24 unknown, 102 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:41.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:40 smithi063 ceph-mon[31613]: pgmap v393: 129 pgs: 3 creating+peering, 24 unknown, 102 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:41.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.930+0000 7f999be00700 1 -- 172.21.15.63:0/2894634574 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) ==== 169+0+0 (secure 0 0 0) 0x7f99940aa350 con 0x7f99a4106610 2024-05-27T07:28:41.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.935+0000 7f99a9200700 1 -- 172.21.15.63:0/2894634574 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f99880b2880 msgr2=0x7f99880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:41.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.935+0000 7f99a9200700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f99880b2880 0x7f99880b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9990005950 tx=0x7f99900058e0 comp rx=0 tx=0).stop 2024-05-27T07:28:41.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.935+0000 7f99a9200700 1 -- 172.21.15.63:0/2894634574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a4106610 msgr2=0x7f99a419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:41.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.935+0000 7f99a9200700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a4106610 0x7f99a419cd10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f999400c9b0 tx=0x7f999400cd70 comp rx=0 tx=0).stop 2024-05-27T07:28:41.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.936+0000 7f99a9200700 1 -- 172.21.15.63:0/2894634574 shutdown_connections 2024-05-27T07:28:41.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.936+0000 7f99a9200700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99a410b890 0x7f99a41a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:41.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.936+0000 7f99a9200700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f99880b2880 0x7f99880b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:41.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.936+0000 7f99a9200700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a4106610 0x7f99a419cd10 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f999400c9b0 tx=0x7f999400cd70 comp rx=0 tx=0).stop 2024-05-27T07:28:41.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.936+0000 7f99a9200700 1 --2- 172.21.15.63:0/2894634574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99a410af40 0x7f99a419d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:41.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.936+0000 7f99a9200700 1 -- 172.21.15.63:0/2894634574 >> 172.21.15.63:0/2894634574 conn(0x7f99a4102020 msgr2=0x7f99a410a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:41.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.937+0000 7f99a9200700 1 -- 172.21.15.63:0/2894634574 shutdown_connections 2024-05-27T07:28:41.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:41.937+0000 7f99a9200700 1 -- 172.21.15.63:0/2894634574 wait complete. 2024-05-27T07:28:41.928 INFO:teuthology.orchestra.run.smithi063.stderr:set pool 4 allow_ec_overwrites to true 2024-05-27T07:28:42.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:41 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2894634574' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-27T07:28:42.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:41 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2894634574' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-27T07:28:42.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:41 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2894634574' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-27T07:28:42.844 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-27T07:28:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:42 smithi086 ceph-mon[34687]: pgmap v394: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:42 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2894634574' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-27T07:28:43.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:42 smithi086 ceph-mon[34687]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T07:28:43.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:42 smithi149 ceph-mon[35408]: pgmap v394: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:43.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:42 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2894634574' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-27T07:28:43.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:42 smithi149 ceph-mon[35408]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T07:28:43.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:42 smithi063 ceph-mon[31613]: pgmap v394: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:43.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:42 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2894634574' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-27T07:28:43.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:42 smithi063 ceph-mon[31613]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T07:28:43.311 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:45.197 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:44 smithi063 ceph-mon[31613]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:45.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:44 smithi086 ceph-mon[34687]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:45.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:44 smithi149 ceph-mon[35408]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:45.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.577+0000 7f4d1c200700 1 -- 172.21.15.63:0/676661189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d14110960 msgr2=0x7f4d14110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:45.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.577+0000 7f4d1c200700 1 --2- 172.21.15.63:0/676661189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d14110960 0x7f4d14110d70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009990 tx=0x7f4d08009ca0 comp rx=0 tx=0).stop 2024-05-27T07:28:45.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.577+0000 7f4d1c200700 1 -- 172.21.15.63:0/676661189 shutdown_connections 2024-05-27T07:28:45.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.577+0000 7f4d1c200700 1 --2- 172.21.15.63:0/676661189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d1410b2d0 0x7f4d14117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:45.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.577+0000 7f4d1c200700 1 --2- 172.21.15.63:0/676661189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d14110960 0x7f4d14110d70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:45.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.577+0000 7f4d1c200700 1 --2- 172.21.15.63:0/676661189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d1410a980 0x7f4d1410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:45.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.577+0000 7f4d1c200700 1 -- 172.21.15.63:0/676661189 >> 172.21.15.63:0/676661189 conn(0x7f4d14077c70 msgr2=0x7f4d140780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:45.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.577+0000 7f4d1c200700 1 -- 172.21.15.63:0/676661189 shutdown_connections 2024-05-27T07:28:45.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.577+0000 7f4d1c200700 1 -- 172.21.15.63:0/676661189 wait complete. 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.578+0000 7f4d1c200700 1 Processor -- start 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.579+0000 7f4d1c200700 1 -- start start 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.579+0000 7f4d1c200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d1410a980 0x7f4d141a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.579+0000 7f4d1c200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d1410b2d0 0x7f4d141a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.579+0000 7f4d1c200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d14110960 0x7f4d141a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.579+0000 7f4d1c200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1411a910 con 0x7f4d1410b2d0 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.579+0000 7f4d1c200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1411a790 con 0x7f4d14110960 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.579+0000 7f4d1c200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1411aa90 con 0x7f4d1410a980 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.579+0000 7f4d18e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d1410b2d0 0x7f4d141a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.579+0000 7f4d18e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d1410b2d0 0x7f4d141a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34902/0 (socket says 172.21.15.63:34902) 2024-05-27T07:28:45.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.579+0000 7f4d18e00700 1 -- 172.21.15.63:0/328016894 learned_addr learned my addr 172.21.15.63:0/328016894 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:45.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.580+0000 7f4d19800700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d1410a980 0x7f4d141a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:45.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.580+0000 7f4d1a200700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d14110960 0x7f4d141a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:45.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.580+0000 7f4d18e00700 1 -- 172.21.15.63:0/328016894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d1410a980 msgr2=0x7f4d141a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.580+0000 7f4d18e00700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d1410a980 0x7f4d141a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.580+0000 7f4d18e00700 1 -- 172.21.15.63:0/328016894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d14110960 msgr2=0x7f4d141a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.580+0000 7f4d18e00700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d14110960 0x7f4d141a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.580+0000 7f4d18e00700 1 -- 172.21.15.63:0/328016894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d141a6230 con 0x7f4d1410b2d0 2024-05-27T07:28:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.580+0000 7f4d18e00700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d1410b2d0 0x7f4d141a1900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000d980 tx=0x7f4d0000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.581+0000 7f4d12000700 1 -- 172.21.15.63:0/328016894 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00017070 con 0x7f4d1410b2d0 2024-05-27T07:28:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.581+0000 7f4d12000700 1 -- 172.21.15.63:0/328016894 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d00010460 con 0x7f4d1410b2d0 2024-05-27T07:28:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.581+0000 7f4d12000700 1 -- 172.21.15.63:0/328016894 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000de60 con 0x7f4d1410b2d0 2024-05-27T07:28:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.581+0000 7f4d1c200700 1 -- 172.21.15.63:0/328016894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d141a6520 con 0x7f4d1410b2d0 2024-05-27T07:28:45.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.581+0000 7f4d1c200700 1 -- 172.21.15.63:0/328016894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d141ade40 con 0x7f4d1410b2d0 2024-05-27T07:28:45.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.582+0000 7f4d1c200700 1 -- 172.21.15.63:0/328016894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d1410c070 con 0x7f4d1410b2d0 2024-05-27T07:28:45.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.585+0000 7f4d12000700 1 -- 172.21.15.63:0/328016894 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4d00010610 con 0x7f4d1410b2d0 2024-05-27T07:28:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.588+0000 7f4d12000700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4cfc0b2900 0x7f4cfc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.588+0000 7f4d19800700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4cfc0b2900 0x7f4cfc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.588+0000 7f4d12000700 1 -- 172.21.15.63:0/328016894 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7f4d000e1f80 con 0x7f4d1410b2d0 2024-05-27T07:28:45.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.589+0000 7f4d19800700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4cfc0b2900 0x7f4cfc0b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4d08006010 tx=0x7f4d0800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:45.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.591+0000 7f4d12000700 1 -- 172.21.15.63:0/328016894 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d000a9aa0 con 0x7f4d1410b2d0 2024-05-27T07:28:45.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.881+0000 7f4d1c200700 1 -- 172.21.15.63:0/328016894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f4d141a26c0 con 0x7f4d1410b2d0 2024-05-27T07:28:45.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.960+0000 7f4d12000700 1 -- 172.21.15.63:0/328016894 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f4d000a93a0 con 0x7f4d1410b2d0 2024-05-27T07:28:45.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 -- 172.21.15.63:0/328016894 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4cfc0b2900 msgr2=0x7f4cfc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:45.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4cfc0b2900 0x7f4cfc0b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4d08006010 tx=0x7f4d0800b540 comp rx=0 tx=0).stop 2024-05-27T07:28:45.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 -- 172.21.15.63:0/328016894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d1410b2d0 msgr2=0x7f4d141a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:45.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d1410b2d0 0x7f4d141a1900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000d980 tx=0x7f4d0000dd40 comp rx=0 tx=0).stop 2024-05-27T07:28:45.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 -- 172.21.15.63:0/328016894 shutdown_connections 2024-05-27T07:28:45.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d14110960 0x7f4d141a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:45.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d1410a980 0x7f4d141a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:45.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4cfc0b2900 0x7f4cfc0b4e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:45.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 --2- 172.21.15.63:0/328016894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d1410b2d0 0x7f4d141a1900 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:45.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 -- 172.21.15.63:0/328016894 >> 172.21.15.63:0/328016894 conn(0x7f4d14077c70 msgr2=0x7f4d1407bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:45.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 -- 172.21.15.63:0/328016894 shutdown_connections 2024-05-27T07:28:45.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:45.964+0000 7f4d1c200700 1 -- 172.21.15.63:0/328016894 wait complete. 2024-05-27T07:28:45.957 INFO:teuthology.orchestra.run.smithi063.stderr:added data pool 4 to fsmap 2024-05-27T07:28:46.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:45 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/328016894' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-27T07:28:46.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:45 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/328016894' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-27T07:28:46.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:45 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/328016894' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-27T07:28:46.851 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:28:47.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:46 smithi086 ceph-mon[34687]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:47.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:46 smithi086 ceph-mon[34687]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T07:28:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:46 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/328016894' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-27T07:28:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:46 smithi086 ceph-mon[34687]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:28:47.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:46 smithi149 ceph-mon[35408]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:47.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:46 smithi149 ceph-mon[35408]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T07:28:47.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:46 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/328016894' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-27T07:28:47.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:46 smithi149 ceph-mon[35408]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:28:47.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:46 smithi063 ceph-mon[31613]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:47.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:46 smithi063 ceph-mon[31613]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T07:28:47.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:46 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/328016894' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-27T07:28:47.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:46 smithi063 ceph-mon[31613]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:28:47.326 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:48.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:48 smithi063 ceph-mon[31613]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T07:28:48.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:48 smithi063 ceph-mon[31613]: Cluster is now healthy 2024-05-27T07:28:48.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:48 smithi086 ceph-mon[34687]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T07:28:48.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:48 smithi086 ceph-mon[34687]: Cluster is now healthy 2024-05-27T07:28:48.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:48 smithi149 ceph-mon[35408]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T07:28:48.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:48 smithi149 ceph-mon[35408]: Cluster is now healthy 2024-05-27T07:28:49.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:49 smithi063 ceph-mon[31613]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:49.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:49 smithi086 ceph-mon[34687]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:49.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:49 smithi149 ceph-mon[35408]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:49.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.523+0000 7f4bfb800700 1 -- 172.21.15.63:0/2254003334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf4109860 msgr2=0x7f4bf41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:49.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.523+0000 7f4bfb800700 1 --2- 172.21.15.63:0/2254003334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf4109860 0x7f4bf41111c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00b0d0 tx=0x7f4bec00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.523+0000 7f4bfb800700 1 -- 172.21.15.63:0/2254003334 shutdown_connections 2024-05-27T07:28:49.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.523+0000 7f4bfb800700 1 --2- 172.21.15.63:0/2254003334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf4109860 0x7f4bf41111c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.523+0000 7f4bfb800700 1 --2- 172.21.15.63:0/2254003334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bf41045e0 0x7f4bf41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.523+0000 7f4bfb800700 1 --2- 172.21.15.63:0/2254003334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bf4108f10 0x7f4bf4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.523+0000 7f4bfb800700 1 -- 172.21.15.63:0/2254003334 >> 172.21.15.63:0/2254003334 conn(0x7f4bf4100030 msgr2=0x7f4bf4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.524+0000 7f4bfb800700 1 -- 172.21.15.63:0/2254003334 shutdown_connections 2024-05-27T07:28:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.524+0000 7f4bfb800700 1 -- 172.21.15.63:0/2254003334 wait complete. 2024-05-27T07:28:49.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.525+0000 7f4bfb800700 1 Processor -- start 2024-05-27T07:28:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.525+0000 7f4bfb800700 1 -- start start 2024-05-27T07:28:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.525+0000 7f4bfb800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bf41045e0 0x7f4bf419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.525+0000 7f4bfb800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bf4108f10 0x7f4bf419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.525+0000 7f4bfb800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf4109860 0x7f4bf41a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bfb800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4114390 con 0x7f4bf4109860 2024-05-27T07:28:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bfb800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4114210 con 0x7f4bf4108f10 2024-05-27T07:28:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bfb800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4114510 con 0x7f4bf41045e0 2024-05-27T07:28:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bf9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf4109860 0x7f4bf41a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bf9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf4109860 0x7f4bf41a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52260/0 (socket says 172.21.15.63:52260) 2024-05-27T07:28:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bf9800700 1 -- 172.21.15.63:0/2419165755 learned_addr learned my addr 172.21.15.63:0/2419165755 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bf8e00700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bf41045e0 0x7f4bf419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bf3e00700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bf4108f10 0x7f4bf419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bf9800700 1 -- 172.21.15.63:0/2419165755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bf41045e0 msgr2=0x7f4bf419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bf9800700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bf41045e0 0x7f4bf419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bf9800700 1 -- 172.21.15.63:0/2419165755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bf4108f10 msgr2=0x7f4bf419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bf9800700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bf4108f10 0x7f4bf419d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.526+0000 7f4bf9800700 1 -- 172.21.15.63:0/2419165755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf41a1e70 con 0x7f4bf4109860 2024-05-27T07:28:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.527+0000 7f4bf9800700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf4109860 0x7f4bf41a17d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4bec007d10 tx=0x7f4bec007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.527+0000 7f4bf1600700 1 -- 172.21.15.63:0/2419165755 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec01f070 con 0x7f4bf4109860 2024-05-27T07:28:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.527+0000 7f4bf1600700 1 -- 172.21.15.63:0/2419165755 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4bec003e10 con 0x7f4bf4109860 2024-05-27T07:28:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.527+0000 7f4bfb800700 1 -- 172.21.15.63:0/2419165755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf41a2100 con 0x7f4bf4109860 2024-05-27T07:28:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.527+0000 7f4bf1600700 1 -- 172.21.15.63:0/2419165755 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec01b820 con 0x7f4bf4109860 2024-05-27T07:28:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.527+0000 7f4bfb800700 1 -- 172.21.15.63:0/2419165755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf41a25e0 con 0x7f4bf4109860 2024-05-27T07:28:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.529+0000 7f4bfb800700 1 -- 172.21.15.63:0/2419165755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf40512b0 con 0x7f4bf4109860 2024-05-27T07:28:49.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.531+0000 7f4bf1600700 1 -- 172.21.15.63:0/2419165755 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4bec022070 con 0x7f4bf4109860 2024-05-27T07:28:49.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.532+0000 7f4bf1600700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4bdc0b2880 0x7f4bdc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:49.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.533+0000 7f4bf1600700 1 -- 172.21.15.63:0/2419165755 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f4bec00f020 con 0x7f4bf4109860 2024-05-27T07:28:49.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.533+0000 7f4bf8e00700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4bdc0b2880 0x7f4bdc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:49.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.534+0000 7f4bf8e00700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4bdc0b2880 0x7f4bdc0b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005950 tx=0x7f4be80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:49.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.538+0000 7f4bf1600700 1 -- 172.21.15.63:0/2419165755 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4bec029020 con 0x7f4bf4109860 2024-05-27T07:28:49.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.831+0000 7f4bfb800700 1 -- 172.21.15.63:0/2419165755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bf41a9a80 con 0x7f4bf4109860 2024-05-27T07:28:49.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.833+0000 7f4bf1600700 1 -- 172.21.15.63:0/2419165755 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11788 (secure 0 0 0) 0x7f4bec0ab560 con 0x7f4bf4109860 2024-05-27T07:28:49.823 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:28:49.823 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34430,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34484,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:28:45.882661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34520,"qdb_cluster":[34520]},"id":1}]} 2024-05-27T07:28:49.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 -- 172.21.15.63:0/2419165755 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4bdc0b2880 msgr2=0x7f4bdc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:49.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4bdc0b2880 0x7f4bdc0b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005950 tx=0x7f4be80058e0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 -- 172.21.15.63:0/2419165755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf4109860 msgr2=0x7f4bf41a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:49.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf4109860 0x7f4bf41a17d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4bec007d10 tx=0x7f4bec007df0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 -- 172.21.15.63:0/2419165755 shutdown_connections 2024-05-27T07:28:49.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f4bdc0b2880 0x7f4bdc0b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf4109860 0x7f4bf41a17d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bf41045e0 0x7f4bf419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 --2- 172.21.15.63:0/2419165755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4bf4108f10 0x7f4bf419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:49.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 -- 172.21.15.63:0/2419165755 >> 172.21.15.63:0/2419165755 conn(0x7f4bf4100030 msgr2=0x7f4bf41086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:49.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.836+0000 7f4bfb800700 1 -- 172.21.15.63:0/2419165755 shutdown_connections 2024-05-27T07:28:49.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:49.837+0000 7f4bfb800700 1 -- 172.21.15.63:0/2419165755 wait complete. 2024-05-27T07:28:49.828 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 17 2024-05-27T07:28:50.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:50 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2419165755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:28:50.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:50 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2419165755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:28:50.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:50 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2419165755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:28:50.717 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:28:51.203 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:51.360 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:51 smithi063 ceph-mon[31613]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:51.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:51 smithi086 ceph-mon[34687]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:51.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:51 smithi149 ceph-mon[35408]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:53.273 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:53 smithi063 ceph-mon[31613]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:53.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:53 smithi086 ceph-mon[34687]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:53.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:53 smithi149 ceph-mon[35408]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:53.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.707+0000 7f333f800700 1 -- 172.21.15.63:0/829520132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3338078630 msgr2=0x7f3338114ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:53.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.707+0000 7f333f800700 1 --2- 172.21.15.63:0/829520132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3338078630 0x7f3338114ca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3324009990 tx=0x7f3324009ca0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.707+0000 7f333f800700 1 -- 172.21.15.63:0/829520132 shutdown_connections 2024-05-27T07:28:53.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.707+0000 7f333f800700 1 --2- 172.21.15.63:0/829520132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33381151e0 0x7f3338117600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.707+0000 7f333f800700 1 --2- 172.21.15.63:0/829520132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3338077ce0 0x7f33380780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.707+0000 7f333f800700 1 --2- 172.21.15.63:0/829520132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3338078630 0x7f3338114ca0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.707+0000 7f333f800700 1 -- 172.21.15.63:0/829520132 >> 172.21.15.63:0/829520132 conn(0x7f3338102080 msgr2=0x7f33381044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:53.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.707+0000 7f333f800700 1 -- 172.21.15.63:0/829520132 shutdown_connections 2024-05-27T07:28:53.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.707+0000 7f333f800700 1 -- 172.21.15.63:0/829520132 wait complete. 2024-05-27T07:28:53.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.708+0000 7f333f800700 1 Processor -- start 2024-05-27T07:28:53.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.708+0000 7f333f800700 1 -- start start 2024-05-27T07:28:53.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.709+0000 7f333f800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3338077ce0 0x7f33381a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:53.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.709+0000 7f333f800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3338078630 0x7f33381a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:53.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.709+0000 7f333f800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33381151e0 0x7f33381a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:53.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.709+0000 7f333f800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f333811a900 con 0x7f33381151e0 2024-05-27T07:28:53.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.709+0000 7f333f800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f333811a780 con 0x7f3338078630 2024-05-27T07:28:53.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.709+0000 7f333f800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f333811aa80 con 0x7f3338077ce0 2024-05-27T07:28:53.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.709+0000 7f333d800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33381151e0 0x7f33381a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:53.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.709+0000 7f333d800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33381151e0 0x7f33381a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52280/0 (socket says 172.21.15.63:52280) 2024-05-27T07:28:53.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.709+0000 7f333d800700 1 -- 172.21.15.63:0/1665900949 learned_addr learned my addr 172.21.15.63:0/1665900949 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:53.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.710+0000 7f333ce00700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3338077ce0 0x7f33381a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:53.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.710+0000 7f3337e00700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3338078630 0x7f33381a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:53.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.710+0000 7f333d800700 1 -- 172.21.15.63:0/1665900949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3338077ce0 msgr2=0x7f33381a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:53.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.710+0000 7f333d800700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3338077ce0 0x7f33381a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.710+0000 7f333d800700 1 -- 172.21.15.63:0/1665900949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3338078630 msgr2=0x7f33381a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:53.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.710+0000 7f333d800700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3338078630 0x7f33381a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.710+0000 7f333d800700 1 -- 172.21.15.63:0/1665900949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33381a6280 con 0x7f33381151e0 2024-05-27T07:28:53.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.710+0000 7f333d800700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33381151e0 0x7f33381a5be0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3330007ee0 tx=0x7f333000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:53.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.711+0000 7f3335600700 1 -- 172.21.15.63:0/1665900949 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330004bb0 con 0x7f33381151e0 2024-05-27T07:28:53.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.711+0000 7f3335600700 1 -- 172.21.15.63:0/1665900949 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3330004d50 con 0x7f33381151e0 2024-05-27T07:28:53.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.711+0000 7f3335600700 1 -- 172.21.15.63:0/1665900949 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33300058b0 con 0x7f33381151e0 2024-05-27T07:28:53.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.711+0000 7f333f800700 1 -- 172.21.15.63:0/1665900949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33381a6570 con 0x7f33381151e0 2024-05-27T07:28:53.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.711+0000 7f333f800700 1 -- 172.21.15.63:0/1665900949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33381ade90 con 0x7f33381151e0 2024-05-27T07:28:53.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.712+0000 7f333f800700 1 -- 172.21.15.63:0/1665900949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3338079420 con 0x7f33381151e0 2024-05-27T07:28:53.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.715+0000 7f3335600700 1 -- 172.21.15.63:0/1665900949 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f333001a020 con 0x7f33381151e0 2024-05-27T07:28:53.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.717+0000 7f3335600700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f33200b2900 0x7f33200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:53.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.717+0000 7f3335600700 1 -- 172.21.15.63:0/1665900949 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f33300e20d0 con 0x7f33381151e0 2024-05-27T07:28:53.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.717+0000 7f333ce00700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f33200b2900 0x7f33200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:53.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.718+0000 7f333ce00700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f33200b2900 0x7f33200b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f332c005950 tx=0x7f332c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:53.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.719+0000 7f3335600700 1 -- 172.21.15.63:0/1665900949 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3330010070 con 0x7f33381151e0 2024-05-27T07:28:53.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.960+0000 7f333f800700 1 -- 172.21.15.63:0/1665900949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f333802e5e0 con 0x7f33381151e0 2024-05-27T07:28:53.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.962+0000 7f3335600700 1 -- 172.21.15.63:0/1665900949 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24079 (secure 0 0 0) 0x7f33300a9b50 con 0x7f33381151e0 2024-05-27T07:28:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:28:53.953 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":87,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:28:45.882492+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:28:53.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.966+0000 7f333f800700 1 -- 172.21.15.63:0/1665900949 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f33200b2900 msgr2=0x7f33200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:53.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.966+0000 7f333f800700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f33200b2900 0x7f33200b4e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f332c005950 tx=0x7f332c00b410 comp rx=0 tx=0).stop 2024-05-27T07:28:53.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.966+0000 7f333f800700 1 -- 172.21.15.63:0/1665900949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33381151e0 msgr2=0x7f33381a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:53.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.966+0000 7f333f800700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33381151e0 0x7f33381a5be0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3330007ee0 tx=0x7f333000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.967+0000 7f333f800700 1 -- 172.21.15.63:0/1665900949 shutdown_connections 2024-05-27T07:28:53.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.967+0000 7f333f800700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f33200b2900 0x7f33200b4e10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.967+0000 7f333f800700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33381151e0 0x7f33381a5be0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.967+0000 7f333f800700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3338077ce0 0x7f33381a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.967+0000 7f333f800700 1 --2- 172.21.15.63:0/1665900949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3338078630 0x7f33381a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:53.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.967+0000 7f333f800700 1 -- 172.21.15.63:0/1665900949 >> 172.21.15.63:0/1665900949 conn(0x7f3338102080 msgr2=0x7f33381044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:53.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.967+0000 7f333f800700 1 -- 172.21.15.63:0/1665900949 shutdown_connections 2024-05-27T07:28:53.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:53.967+0000 7f333f800700 1 -- 172.21.15.63:0/1665900949 wait complete. 2024-05-27T07:28:54.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:54 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1665900949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:28:54.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:54 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1665900949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:28:54.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:54 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1665900949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:28:54.853 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:28:55.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:55 smithi063 ceph-mon[31613]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:55.333 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:28:55.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:55 smithi086 ceph-mon[34687]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:55.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:55 smithi149 ceph-mon[35408]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:57.188 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:57 smithi063 ceph-mon[31613]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:57.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:57 smithi086 ceph-mon[34687]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:57.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:57 smithi149 ceph-mon[35408]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:57.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.845+0000 7f2c77e00700 1 -- 172.21.15.63:0/4168472951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c781043f0 msgr2=0x7f2c78104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:57.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.845+0000 7f2c77e00700 1 --2- 172.21.15.63:0/4168472951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c781043f0 0x7f2c78104800 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2c680099c0 tx=0x7f2c68009cd0 comp rx=0 tx=0).stop 2024-05-27T07:28:57.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.845+0000 7f2c77e00700 1 -- 172.21.15.63:0/4168472951 shutdown_connections 2024-05-27T07:28:57.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.845+0000 7f2c77e00700 1 --2- 172.21.15.63:0/4168472951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c78109670 0x7f2c78110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:57.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.845+0000 7f2c77e00700 1 --2- 172.21.15.63:0/4168472951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c781043f0 0x7f2c78104800 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:57.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.845+0000 7f2c77e00700 1 --2- 172.21.15.63:0/4168472951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c78108d20 0x7f2c78109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:57.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.845+0000 7f2c77e00700 1 -- 172.21.15.63:0/4168472951 >> 172.21.15.63:0/4168472951 conn(0x7f2c78079d00 msgr2=0x7f2c7807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:57.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.845+0000 7f2c77e00700 1 -- 172.21.15.63:0/4168472951 shutdown_connections 2024-05-27T07:28:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.846+0000 7f2c77e00700 1 -- 172.21.15.63:0/4168472951 wait complete. 2024-05-27T07:28:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.847+0000 7f2c77e00700 1 Processor -- start 2024-05-27T07:28:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.847+0000 7f2c77e00700 1 -- start start 2024-05-27T07:28:57.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.847+0000 7f2c77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c781043f0 0x7f2c781a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:57.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.847+0000 7f2c77e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c78108d20 0x7f2c781a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:57.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.847+0000 7f2c77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c78109670 0x7f2c7819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:57.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.847+0000 7f2c77e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c781141a0 con 0x7f2c781043f0 2024-05-27T07:28:57.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.847+0000 7f2c77e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78114020 con 0x7f2c78109670 2024-05-27T07:28:57.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.847+0000 7f2c77e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78114320 con 0x7f2c78108d20 2024-05-27T07:28:57.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c76a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c781043f0 0x7f2c781a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:57.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c76a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c781043f0 0x7f2c781a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52294/0 (socket says 172.21.15.63:52294) 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c76a00700 1 -- 172.21.15.63:0/2436045511 learned_addr learned my addr 172.21.15.63:0/2436045511 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c77400700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c78109670 0x7f2c7819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c6e000700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c78108d20 0x7f2c781a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c76a00700 1 -- 172.21.15.63:0/2436045511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c78108d20 msgr2=0x7f2c781a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c76a00700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c78108d20 0x7f2c781a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c76a00700 1 -- 172.21.15.63:0/2436045511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c78109670 msgr2=0x7f2c7819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c76a00700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c78109670 0x7f2c7819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c76a00700 1 -- 172.21.15.63:0/2436045511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7819b110 con 0x7f2c781043f0 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.848+0000 7f2c77400700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c78109670 0x7f2c7819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.849+0000 7f2c76a00700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c781043f0 0x7f2c781a0840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000c00 tx=0x7f2c6800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.849+0000 7f2c7ca00700 1 -- 172.21.15.63:0/2436045511 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68027070 con 0x7f2c781043f0 2024-05-27T07:28:57.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.849+0000 7f2c77e00700 1 -- 172.21.15.63:0/2436045511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7819b3a0 con 0x7f2c781043f0 2024-05-27T07:28:57.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.849+0000 7f2c7ca00700 1 -- 172.21.15.63:0/2436045511 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c6800fe50 con 0x7f2c781043f0 2024-05-27T07:28:57.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.849+0000 7f2c7ca00700 1 -- 172.21.15.63:0/2436045511 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680173f0 con 0x7f2c781043f0 2024-05-27T07:28:57.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.849+0000 7f2c77e00700 1 -- 172.21.15.63:0/2436045511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c781a76d0 con 0x7f2c781043f0 2024-05-27T07:28:57.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.851+0000 7f2c77e00700 1 -- 172.21.15.63:0/2436045511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c781058a0 con 0x7f2c781043f0 2024-05-27T07:28:57.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.853+0000 7f2c7ca00700 1 -- 172.21.15.63:0/2436045511 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2c6802a070 con 0x7f2c781043f0 2024-05-27T07:28:57.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.854+0000 7f2c7ca00700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c5c0b2900 0x7f2c5c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:28:57.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.855+0000 7f2c7ca00700 1 -- 172.21.15.63:0/2436045511 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f2c680e35f0 con 0x7f2c781043f0 2024-05-27T07:28:57.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.855+0000 7f2c6e000700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c5c0b2900 0x7f2c5c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:28:57.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.855+0000 7f2c6e000700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c5c0b2900 0x7f2c5c0b4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c60005950 tx=0x7f2c600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:28:57.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:57.858+0000 7f2c7ca00700 1 -- 172.21.15.63:0/2436045511 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c68013070 con 0x7f2c781043f0 2024-05-27T07:28:58.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.104+0000 7f2c77e00700 1 -- 172.21.15.63:0/2436045511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c7819bf90 con 0x7f2c781043f0 2024-05-27T07:28:58.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.108+0000 7f2c7ca00700 1 -- 172.21.15.63:0/2436045511 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24079 (secure 0 0 0) 0x7f2c680a6020 con 0x7f2c781043f0 2024-05-27T07:28:58.210 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:28:58.210 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":87,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:28:45.882492+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:28:58.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.111+0000 7f2c77e00700 1 -- 172.21.15.63:0/2436045511 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c5c0b2900 msgr2=0x7f2c5c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:58.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.111+0000 7f2c77e00700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c5c0b2900 0x7f2c5c0b4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c60005950 tx=0x7f2c600058e0 comp rx=0 tx=0).stop 2024-05-27T07:28:58.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.111+0000 7f2c77e00700 1 -- 172.21.15.63:0/2436045511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c781043f0 msgr2=0x7f2c781a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:28:58.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.111+0000 7f2c77e00700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c781043f0 0x7f2c781a0840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000c00 tx=0x7f2c6800f510 comp rx=0 tx=0).stop 2024-05-27T07:28:58.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.111+0000 7f2c77e00700 1 -- 172.21.15.63:0/2436045511 shutdown_connections 2024-05-27T07:28:58.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.111+0000 7f2c77e00700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c78109670 0x7f2c7819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:58.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.111+0000 7f2c77e00700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c5c0b2900 0x7f2c5c0b4e10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:58.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.111+0000 7f2c77e00700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c781043f0 0x7f2c781a0840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:58.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.111+0000 7f2c77e00700 1 --2- 172.21.15.63:0/2436045511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c78108d20 0x7f2c781a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:28:58.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.111+0000 7f2c77e00700 1 -- 172.21.15.63:0/2436045511 >> 172.21.15.63:0/2436045511 conn(0x7f2c78079d00 msgr2=0x7f2c78101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:28:58.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.112+0000 7f2c77e00700 1 -- 172.21.15.63:0/2436045511 shutdown_connections 2024-05-27T07:28:58.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:28:58.112+0000 7f2c77e00700 1 -- 172.21.15.63:0/2436045511 wait complete. 2024-05-27T07:28:58.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:58 smithi086 ceph-mon[34687]: pgmap v404: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:58.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:58 smithi149 ceph-mon[35408]: pgmap v404: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:58.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:58 smithi063 ceph-mon[31613]: pgmap v404: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:28:59.060 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:28:59.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:28:59 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2436045511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:28:59.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:28:59 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2436045511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:28:59.454 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:28:59 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2436045511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:28:59.531 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:00.448 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:00 smithi086 ceph-mon[34687]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:00.452 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:00 smithi149 ceph-mon[35408]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:00.455 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:00 smithi063 ceph-mon[31613]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:01.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.842+0000 7f010ca00700 1 -- 172.21.15.63:0/2490334950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01081045e0 msgr2=0x7f01081049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.842+0000 7f010ca00700 1 --2- 172.21.15.63:0/2490334950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01081045e0 0x7f01081049f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f00f80099c0 tx=0x7f00f8009cd0 comp rx=0 tx=0).stop 2024-05-27T07:29:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.842+0000 7f010ca00700 1 -- 172.21.15.63:0/2490334950 shutdown_connections 2024-05-27T07:29:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.842+0000 7f010ca00700 1 --2- 172.21.15.63:0/2490334950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0108109860 0x7f01081111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.842+0000 7f010ca00700 1 --2- 172.21.15.63:0/2490334950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01081045e0 0x7f01081049f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.842+0000 7f010ca00700 1 --2- 172.21.15.63:0/2490334950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0108108f10 0x7f0108109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.842+0000 7f010ca00700 1 -- 172.21.15.63:0/2490334950 >> 172.21.15.63:0/2490334950 conn(0x7f0108100030 msgr2=0x7f0108102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.843+0000 7f010ca00700 1 -- 172.21.15.63:0/2490334950 shutdown_connections 2024-05-27T07:29:01.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.843+0000 7f010ca00700 1 -- 172.21.15.63:0/2490334950 wait complete. 2024-05-27T07:29:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.844+0000 7f010ca00700 1 Processor -- start 2024-05-27T07:29:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.844+0000 7f010ca00700 1 -- start start 2024-05-27T07:29:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.844+0000 7f010ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01081045e0 0x7f010819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f010ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0108108f10 0x7f010819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f010ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0108109860 0x7f01081a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f010ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0108114340 con 0x7f0108108f10 2024-05-27T07:29:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f010ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01081141c0 con 0x7f01081045e0 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f010ca00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01081144c0 con 0x7f0108109860 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f0105400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0108108f10 0x7f010819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f0105400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0108108f10 0x7f010819d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47218/0 (socket says 172.21.15.63:47218) 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f0105400700 1 -- 172.21.15.63:0/2327295133 learned_addr learned my addr 172.21.15.63:0/2327295133 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f0105e00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01081045e0 0x7f010819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f0106800700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0108109860 0x7f01081a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f0105400700 1 -- 172.21.15.63:0/2327295133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0108109860 msgr2=0x7f01081a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f0105400700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0108109860 0x7f01081a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.845+0000 7f0105400700 1 -- 172.21.15.63:0/2327295133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01081045e0 msgr2=0x7f010819cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.846+0000 7f0105400700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01081045e0 0x7f010819cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:01.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.846+0000 7f0105400700 1 -- 172.21.15.63:0/2327295133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01081a1dd0 con 0x7f0108108f10 2024-05-27T07:29:01.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.846+0000 7f0105e00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01081045e0 0x7f010819cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:29:01.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.846+0000 7f0105400700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0108108f10 0x7f010819d550 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f00f400d980 tx=0x7f00f400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:01.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.846+0000 7f00fea00700 1 -- 172.21.15.63:0/2327295133 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4017070 con 0x7f0108108f10 2024-05-27T07:29:01.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.846+0000 7f00fea00700 1 -- 172.21.15.63:0/2327295133 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00f4010460 con 0x7f0108108f10 2024-05-27T07:29:01.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.846+0000 7f010ca00700 1 -- 172.21.15.63:0/2327295133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01081a20c0 con 0x7f0108108f10 2024-05-27T07:29:01.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.846+0000 7f00fea00700 1 -- 172.21.15.63:0/2327295133 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f400de60 con 0x7f0108108f10 2024-05-27T07:29:01.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.846+0000 7f010ca00700 1 -- 172.21.15.63:0/2327295133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01081a2600 con 0x7f0108108f10 2024-05-27T07:29:01.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.848+0000 7f010ca00700 1 -- 172.21.15.63:0/2327295133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01080512b0 con 0x7f0108108f10 2024-05-27T07:29:01.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.851+0000 7f00fea00700 1 -- 172.21.15.63:0/2327295133 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f00f4010610 con 0x7f0108108f10 2024-05-27T07:29:01.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.853+0000 7f00fea00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f00ec0b2880 0x7f00ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:01.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.853+0000 7f00fea00700 1 -- 172.21.15.63:0/2327295133 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f00f40e1fd0 con 0x7f0108108f10 2024-05-27T07:29:01.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.853+0000 7f0105e00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f00ec0b2880 0x7f00ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:01.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.854+0000 7f0105e00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f00ec0b2880 0x7f00ec0b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f00f8000c00 tx=0x7f00f8005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:01.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:01.854+0000 7f00fea00700 1 -- 172.21.15.63:0/2327295133 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00f40a9a50 con 0x7f0108108f10 2024-05-27T07:29:02.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.159+0000 7f010ca00700 1 -- 172.21.15.63:0/2327295133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01081a9b20 con 0x7f0108108f10 2024-05-27T07:29:02.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.161+0000 7f00fea00700 1 -- 172.21.15.63:0/2327295133 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11788 (secure 0 0 0) 0x7f00f40a9350 con 0x7f0108108f10 2024-05-27T07:29:02.151 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:29:02.151 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34430,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34454,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34484,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:28:45.882661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34520,"qdb_cluster":[34520]},"id":1}]} 2024-05-27T07:29:02.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.165+0000 7f010ca00700 1 -- 172.21.15.63:0/2327295133 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f00ec0b2880 msgr2=0x7f00ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:02.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.165+0000 7f010ca00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f00ec0b2880 0x7f00ec0b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f00f8000c00 tx=0x7f00f8005940 comp rx=0 tx=0).stop 2024-05-27T07:29:02.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.165+0000 7f010ca00700 1 -- 172.21.15.63:0/2327295133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0108108f10 msgr2=0x7f010819d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:02.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.165+0000 7f010ca00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0108108f10 0x7f010819d550 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f00f400d980 tx=0x7f00f400dd40 comp rx=0 tx=0).stop 2024-05-27T07:29:02.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.166+0000 7f010ca00700 1 -- 172.21.15.63:0/2327295133 shutdown_connections 2024-05-27T07:29:02.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.166+0000 7f010ca00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0108109860 0x7f01081a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:02.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.166+0000 7f010ca00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01081045e0 0x7f010819cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:02.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.166+0000 7f010ca00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f00ec0b2880 0x7f00ec0b4d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:02.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.166+0000 7f010ca00700 1 --2- 172.21.15.63:0/2327295133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0108108f10 0x7f010819d550 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:02.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.166+0000 7f010ca00700 1 -- 172.21.15.63:0/2327295133 >> 172.21.15.63:0/2327295133 conn(0x7f0108100030 msgr2=0x7f010810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:02.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.166+0000 7f010ca00700 1 -- 172.21.15.63:0/2327295133 shutdown_connections 2024-05-27T07:29:02.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:02.166+0000 7f010ca00700 1 -- 172.21.15.63:0/2327295133 wait complete. 2024-05-27T07:29:02.157 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 17 2024-05-27T07:29:02.965 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:29:03.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:02 smithi086 ceph-mon[34687]: pgmap v406: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:03.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:02 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2327295133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:29:03.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:02 smithi149 ceph-mon[35408]: pgmap v406: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:03.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:02 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2327295133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:29:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:02 smithi063 ceph-mon[31613]: pgmap v406: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:03.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:02 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2327295133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:29:03.420 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:04.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-27T07:29:04.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-27T07:29:04.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-27T07:29:04.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-27T07:29:04.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-27T07:29:04.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-27T07:29:04.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-05-27T07:29:04.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-27T07:29:04.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-27T07:29:04.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:03 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-27T07:29:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-27T07:29:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-27T07:29:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-27T07:29:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-27T07:29:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-27T07:29:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-27T07:29:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-05-27T07:29:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-27T07:29:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-27T07:29:04.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:03 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-27T07:29:04.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-27T07:29:04.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-27T07:29:04.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-27T07:29:04.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-27T07:29:04.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-27T07:29:04.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-05-27T07:29:04.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-05-27T07:29:04.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-05-27T07:29:04.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-27T07:29:04.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:03 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-27T07:29:05.107 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: pgmap v407: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:05.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-27T07:29:05.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-27T07:29:05.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-27T07:29:05.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-27T07:29:05.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-27T07:29:05.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-27T07:29:05.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-05-27T07:29:05.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-27T07:29:05.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-27T07:29:05.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-27T07:29:05.109 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:04 smithi063 ceph-mon[31613]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T07:29:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: pgmap v407: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-27T07:29:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-27T07:29:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-27T07:29:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-27T07:29:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-27T07:29:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-27T07:29:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-05-27T07:29:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-27T07:29:05.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-27T07:29:05.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-27T07:29:05.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:04 smithi086 ceph-mon[34687]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: pgmap v407: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-27T07:29:05.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:04 smithi149 ceph-mon[35408]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T07:29:05.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.725+0000 7fde19a00700 1 -- 172.21.15.63:0/692891524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde14108f10 msgr2=0x7fde14109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:05.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.725+0000 7fde19a00700 1 --2- 172.21.15.63:0/692891524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde14108f10 0x7fde14109320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fde000099c0 tx=0x7fde00009cd0 comp rx=0 tx=0).stop 2024-05-27T07:29:05.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.725+0000 7fde19a00700 1 -- 172.21.15.63:0/692891524 shutdown_connections 2024-05-27T07:29:05.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.725+0000 7fde19a00700 1 --2- 172.21.15.63:0/692891524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde14109860 0x7fde141111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:05.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.725+0000 7fde19a00700 1 --2- 172.21.15.63:0/692891524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde141045e0 0x7fde141049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:05.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.725+0000 7fde19a00700 1 --2- 172.21.15.63:0/692891524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde14108f10 0x7fde14109320 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:05.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.725+0000 7fde19a00700 1 -- 172.21.15.63:0/692891524 >> 172.21.15.63:0/692891524 conn(0x7fde14100030 msgr2=0x7fde14102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:05.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.725+0000 7fde19a00700 1 -- 172.21.15.63:0/692891524 shutdown_connections 2024-05-27T07:29:05.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.725+0000 7fde19a00700 1 -- 172.21.15.63:0/692891524 wait complete. 2024-05-27T07:29:05.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.727+0000 7fde19a00700 1 Processor -- start 2024-05-27T07:29:05.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.727+0000 7fde19a00700 1 -- start start 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.727+0000 7fde19a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde141045e0 0x7fde1419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.727+0000 7fde19a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde14108f10 0x7fde1419d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.727+0000 7fde19a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde14109860 0x7fde141a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.727+0000 7fde19a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde14114320 con 0x7fde14109860 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.727+0000 7fde19a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde141141a0 con 0x7fde14108f10 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.727+0000 7fde19a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde141144a0 con 0x7fde141045e0 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde13400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde14109860 0x7fde141a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde13400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde14109860 0x7fde141a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47244/0 (socket says 172.21.15.63:47244) 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde13400700 1 -- 172.21.15.63:0/3660591317 learned_addr learned my addr 172.21.15.63:0/3660591317 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde12000700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde14108f10 0x7fde1419d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde12a00700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde141045e0 0x7fde1419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:05.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde13400700 1 -- 172.21.15.63:0/3660591317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde141045e0 msgr2=0x7fde1419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:05.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde13400700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde141045e0 0x7fde1419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:05.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde13400700 1 -- 172.21.15.63:0/3660591317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde14108f10 msgr2=0x7fde1419d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:05.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde13400700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde14108f10 0x7fde1419d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:05.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde13400700 1 -- 172.21.15.63:0/3660591317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde141a1d90 con 0x7fde14109860 2024-05-27T07:29:05.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.728+0000 7fde12000700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde14108f10 0x7fde1419d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:29:05.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.729+0000 7fde13400700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde14109860 0x7fde141a17a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00ada0 tx=0x7fde0c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:05.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.729+0000 7fde0b400700 1 -- 172.21.15.63:0/3660591317 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c017020 con 0x7fde14109860 2024-05-27T07:29:05.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.729+0000 7fde0b400700 1 -- 172.21.15.63:0/3660591317 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde0c00ce10 con 0x7fde14109860 2024-05-27T07:29:05.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.729+0000 7fde19a00700 1 -- 172.21.15.63:0/3660591317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde141a2080 con 0x7fde14109860 2024-05-27T07:29:05.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.729+0000 7fde0b400700 1 -- 172.21.15.63:0/3660591317 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c005440 con 0x7fde14109860 2024-05-27T07:29:05.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.729+0000 7fde19a00700 1 -- 172.21.15.63:0/3660591317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde141a2590 con 0x7fde14109860 2024-05-27T07:29:05.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.730+0000 7fde19a00700 1 -- 172.21.15.63:0/3660591317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde140512b0 con 0x7fde14109860 2024-05-27T07:29:05.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.733+0000 7fde0b400700 1 -- 172.21.15.63:0/3660591317 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fde0c004ad0 con 0x7fde14109860 2024-05-27T07:29:05.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.735+0000 7fde0b400700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fddf80b2900 0x7fddf80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:05.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.735+0000 7fde0b400700 1 -- 172.21.15.63:0/3660591317 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fde0c0e22d0 con 0x7fde14109860 2024-05-27T07:29:05.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.735+0000 7fde12a00700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fddf80b2900 0x7fddf80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:05.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.736+0000 7fde12a00700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fddf80b2900 0x7fddf80b4e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde04005ea0 tx=0x7fde04005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:05.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.737+0000 7fde0b400700 1 -- 172.21.15.63:0/3660591317 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fde0c021020 con 0x7fde14109860 2024-05-27T07:29:05.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.976+0000 7fde19a00700 1 -- 172.21.15.63:0/3660591317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde141a9ab0 con 0x7fde14109860 2024-05-27T07:29:05.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:05 smithi063 ceph-mon[31613]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T07:29:05.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.978+0000 7fde0b400700 1 -- 172.21.15.63:0/3660591317 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24552 (secure 0 0 0) 0x7fde0c0a9cc0 con 0x7fde14109860 2024-05-27T07:29:05.969 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:29:05.970 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":89,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:29:04.850822+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:29:05.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.981+0000 7fde19a00700 1 -- 172.21.15.63:0/3660591317 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fddf80b2900 msgr2=0x7fddf80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:05.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.981+0000 7fde19a00700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fddf80b2900 0x7fddf80b4e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde04005ea0 tx=0x7fde04005e30 comp rx=0 tx=0).stop 2024-05-27T07:29:05.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.981+0000 7fde19a00700 1 -- 172.21.15.63:0/3660591317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde14109860 msgr2=0x7fde141a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:05.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.981+0000 7fde19a00700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde14109860 0x7fde141a17a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00ada0 tx=0x7fde0c00c330 comp rx=0 tx=0).stop 2024-05-27T07:29:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.982+0000 7fde19a00700 1 -- 172.21.15.63:0/3660591317 shutdown_connections 2024-05-27T07:29:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.982+0000 7fde19a00700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fddf80b2900 0x7fddf80b4e10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.982+0000 7fde19a00700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde14109860 0x7fde141a17a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.982+0000 7fde19a00700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde141045e0 0x7fde1419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.982+0000 7fde19a00700 1 --2- 172.21.15.63:0/3660591317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde14108f10 0x7fde1419d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.982+0000 7fde19a00700 1 -- 172.21.15.63:0/3660591317 >> 172.21.15.63:0/3660591317 conn(0x7fde14100030 msgr2=0x7fde1410a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.982+0000 7fde19a00700 1 -- 172.21.15.63:0/3660591317 shutdown_connections 2024-05-27T07:29:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:05.982+0000 7fde19a00700 1 -- 172.21.15.63:0/3660591317 wait complete. 2024-05-27T07:29:06.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:05 smithi086 ceph-mon[34687]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T07:29:06.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:05 smithi149 ceph-mon[35408]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T07:29:06.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-27T07:28:27.557349+0000', 'modified': '2024-05-27T07:28:45.882661+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34520': {'gid': 34520, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.86:6839/953218889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 953218889}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 953218889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34520, 'qdb_cluster': [34520]} 2024-05-27T07:29:06.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-27T07:29:06.809 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T07:29:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:06 smithi086 ceph-mon[34687]: pgmap v410: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:07.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:06 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3660591317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:29:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:06 smithi149 ceph-mon[35408]: pgmap v410: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:07.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:06 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3660591317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:29:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:06 smithi063 ceph-mon[31613]: pgmap v410: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:07.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:06 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3660591317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:29:07.279 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:09.043 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:08 smithi063 ceph-mon[31613]: pgmap v411: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:09.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:08 smithi086 ceph-mon[34687]: pgmap v411: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:09.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:08 smithi149 ceph-mon[35408]: pgmap v411: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:09.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.659+0000 7fd191800700 1 -- 172.21.15.63:0/1766408005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd18c10e8d0 msgr2=0x7fd18c110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:09.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.659+0000 7fd191800700 1 --2- 172.21.15.63:0/1766408005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd18c10e8d0 0x7fd18c110db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b0a0 tx=0x7fd18400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.659+0000 7fd191800700 1 -- 172.21.15.63:0/1766408005 shutdown_connections 2024-05-27T07:29:09.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.659+0000 7fd191800700 1 --2- 172.21.15.63:0/1766408005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd18c10e8d0 0x7fd18c110db0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.659+0000 7fd191800700 1 --2- 172.21.15.63:0/1766408005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd18c1054f0 0x7fd18c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.659+0000 7fd191800700 1 --2- 172.21.15.63:0/1766408005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c105e40 0x7fd18c10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.659+0000 7fd191800700 1 -- 172.21.15.63:0/1766408005 >> 172.21.15.63:0/1766408005 conn(0x7fd18c0fedb0 msgr2=0x7fd18c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:09.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.659+0000 7fd191800700 1 -- 172.21.15.63:0/1766408005 shutdown_connections 2024-05-27T07:29:09.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.659+0000 7fd191800700 1 -- 172.21.15.63:0/1766408005 wait complete. 2024-05-27T07:29:09.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.661+0000 7fd191800700 1 Processor -- start 2024-05-27T07:29:09.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.661+0000 7fd191800700 1 -- start start 2024-05-27T07:29:09.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.661+0000 7fd191800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd18c1054f0 0x7fd18c1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.661+0000 7fd191800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd18c105e40 0x7fd18c103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.661+0000 7fd191800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c10e8d0 0x7fd18c103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.661+0000 7fd191800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c114330 con 0x7fd18c105e40 2024-05-27T07:29:09.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.661+0000 7fd191800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c1141b0 con 0x7fd18c10e8d0 2024-05-27T07:29:09.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.661+0000 7fd191800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c1144b0 con 0x7fd18c1054f0 2024-05-27T07:29:09.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.661+0000 7fd182000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd18c105e40 0x7fd18c103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.662+0000 7fd182000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd18c105e40 0x7fd18c103720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52504/0 (socket says 172.21.15.63:52504) 2024-05-27T07:29:09.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.662+0000 7fd182000700 1 -- 172.21.15.63:0/3080033413 learned_addr learned my addr 172.21.15.63:0/3080033413 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:09.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.662+0000 7fd18b400700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c10e8d0 0x7fd18c103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.662+0000 7fd18aa00700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd18c1054f0 0x7fd18c1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.662+0000 7fd182000700 1 -- 172.21.15.63:0/3080033413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd18c1054f0 msgr2=0x7fd18c1031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:09.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.662+0000 7fd182000700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd18c1054f0 0x7fd18c1031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.662+0000 7fd182000700 1 -- 172.21.15.63:0/3080033413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c10e8d0 msgr2=0x7fd18c103c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:09.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.662+0000 7fd182000700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c10e8d0 0x7fd18c103c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.662+0000 7fd182000700 1 -- 172.21.15.63:0/3080033413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c1a8f90 con 0x7fd18c105e40 2024-05-27T07:29:09.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.662+0000 7fd182000700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd18c105e40 0x7fd18c103720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd17400c9b0 tx=0x7fd17400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:09.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.663+0000 7fd183e00700 1 -- 172.21.15.63:0/3080033413 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174007b00 con 0x7fd18c105e40 2024-05-27T07:29:09.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.663+0000 7fd191800700 1 -- 172.21.15.63:0/3080033413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c1a9280 con 0x7fd18c105e40 2024-05-27T07:29:09.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.663+0000 7fd183e00700 1 -- 172.21.15.63:0/3080033413 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd174007ca0 con 0x7fd18c105e40 2024-05-27T07:29:09.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.663+0000 7fd183e00700 1 -- 172.21.15.63:0/3080033413 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174018660 con 0x7fd18c105e40 2024-05-27T07:29:09.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.663+0000 7fd191800700 1 -- 172.21.15.63:0/3080033413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c1a97c0 con 0x7fd18c105e40 2024-05-27T07:29:09.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.669+0000 7fd183e00700 1 -- 172.21.15.63:0/3080033413 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd17400f450 con 0x7fd18c105e40 2024-05-27T07:29:09.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.669+0000 7fd1a0277b80 1 Processor -- start 2024-05-27T07:29:09.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.669+0000 7fd1a0277b80 1 -- start start 2024-05-27T07:29:09.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.670+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e57c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.670+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e57c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.670+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5890c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.670+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56176e4aadf0 con 0x56176e57ce20 2024-05-27T07:29:09.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.670+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56176e49a990 con 0x56176e3fddf0 2024-05-27T07:29:09.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.670+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56176e49e9d0 con 0x56176e4e99a0 2024-05-27T07:29:09.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.670+0000 7fd182000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.670+0000 7fd182000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5890c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52518/0 (socket says 172.21.15.63:52518) 2024-05-27T07:29:09.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.670+0000 7fd182000700 1 -- 172.21.15.63:0/2336947070 learned_addr learned my addr 172.21.15.63:0/2336947070 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:09.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.671+0000 7fd183e00700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1700b2880 0x7fd1700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.671+0000 7fd18b400700 1 --2- 172.21.15.63:0/2336947070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e57c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.671+0000 7fd18aa00700 1 --2- 172.21.15.63:0/2336947070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e57c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.671+0000 7fd183e00700 1 -- 172.21.15.63:0/3080033413 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd1740e25c0 con 0x7fd18c105e40 2024-05-27T07:29:09.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.671+0000 7fd18b400700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1700b2880 0x7fd1700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.671+0000 7fd182000700 1 -- 172.21.15.63:0/2336947070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 msgr2=0x56176e57c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:29:09.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.671+0000 7fd182000700 1 --2- 172.21.15.63:0/2336947070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e57c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.671+0000 7fd182000700 1 -- 172.21.15.63:0/2336947070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 msgr2=0x56176e57c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:29:09.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.671+0000 7fd182000700 1 --2- 172.21.15.63:0/2336947070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e57c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.671+0000 7fd182000700 1 -- 172.21.15.63:0/2336947070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176e4633d0 con 0x56176e57ce20 2024-05-27T07:29:09.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.672+0000 7fd182000700 1 --2- 172.21.15.63:0/2336947070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5890c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1740a6a90 tx=0x7fd174000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c45fa933b74aa43 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:09.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.672+0000 7fd17be00700 1 -- 172.21.15.63:0/2336947070 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1740a9170 con 0x56176e57ce20 2024-05-27T07:29:09.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.672+0000 7fd17be00700 1 -- 172.21.15.63:0/2336947070 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1740e5e50 con 0x56176e57ce20 2024-05-27T07:29:09.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.672+0000 7fd17be00700 1 -- 172.21.15.63:0/2336947070 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1740a8f90 con 0x56176e57ce20 2024-05-27T07:29:09.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.672+0000 7fd18b400700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1700b2880 0x7fd1700b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x56176e57efa0 tx=0x7fd18400b3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:09.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.673+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2336947070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 msgr2=0x56176e5890c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:09.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.673+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/2336947070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5890c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1740a6a90 tx=0x7fd174000bc0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.673+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2336947070 shutdown_connections 2024-05-27T07:29:09.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.673+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/2336947070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5890c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.673+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/2336947070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e57c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.673+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/2336947070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e57c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.673+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2336947070 >> 172.21.15.63:0/2336947070 conn(0x56176e497340 msgr2=0x56176e578470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:09.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.673+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2336947070 shutdown_connections 2024-05-27T07:29:09.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.673+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2336947070 wait complete. 2024-05-27T07:29:09.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.674+0000 7fd1a0277b80 1 -- start start 2024-05-27T07:29:09.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.674+0000 7fd1a0277b80 1 Processor -- start 2024-05-27T07:29:09.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.675+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e586660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.675+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e586ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.675+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5870e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.675+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56176e3a0f50 con 0x56176e57ce20 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.675+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56176e43b710 con 0x56176e3fddf0 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.675+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56176e589a70 con 0x56176e4e99a0 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.675+0000 7fd1a0277b80 10 client.0 _ll_register_callbacks cb 0x56176e444c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.675+0000 7fd18b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5870e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.675+0000 7fd18b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5870e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52522/0 (socket says 172.21.15.63:52522) 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.675+0000 7fd18b400700 1 -- 172.21.15.63:0/3721573234 learned_addr learned my addr 172.21.15.63:0/3721573234 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.676+0000 7fd182000700 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e586ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.676+0000 7fd18aa00700 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e586660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.676+0000 7fd18b400700 1 -- 172.21.15.63:0/3721573234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 msgr2=0x56176e586ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:29:09.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.676+0000 7fd18b400700 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e586ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.676+0000 7fd18b400700 1 -- 172.21.15.63:0/3721573234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 msgr2=0x56176e586660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.676+0000 7fd18b400700 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e586660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.676+0000 7fd18b400700 1 -- 172.21.15.63:0/3721573234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176e3ea880 con 0x56176e57ce20 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.676+0000 7fd18aa00700 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e586660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.676+0000 7fd18b400700 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5870e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd184009430 tx=0x7fd1840099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0b7adbe531cfb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.676+0000 7fd178c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.677+0000 7fd178c00700 1 -- 172.21.15.63:0/3721573234 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18402f060 con 0x56176e57ce20 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.677+0000 7fd1a0277b80 1 -- 172.21.15.63:0/3721573234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1740e5e50 con 0x56176e57ce20 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.677+0000 7fd178c00700 1 -- 172.21.15.63:0/3721573234 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd184007c50 con 0x56176e57ce20 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.677+0000 7fd178c00700 1 -- 172.21.15.63:0/3721573234 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd184026040 con 0x56176e57ce20 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.677+0000 7fd1a0277b80 10 client.14697 fetch_fsmap learned FSMap version 17 2024-05-27T07:29:09.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.677+0000 7fd1a0277b80 1 -- 172.21.15.63:0/3721573234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56176e46b010 con 0x56176e57ce20 2024-05-27T07:29:09.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.678+0000 7fd178c00700 1 -- 172.21.15.63:0/3721573234 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1740e5e50 con 0x56176e57ce20 2024-05-27T07:29:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.678+0000 7fd178c00700 1 -- 172.21.15.63:0/3721573234 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fd184029070 con 0x56176e57ce20 2024-05-27T07:29:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.678+0000 7fd1a0277b80 10 client.14697 fetch_fsmap finished waiting for FSMap version 17 2024-05-27T07:29:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.678+0000 7fd1a0277b80 10 client.14697 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:29:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.678+0000 7fd1a0277b80 20 client.14697 populate_metadata read hostname 'smithi063' 2024-05-27T07:29:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.678+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56176e596a20 0x56176e597e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.678+0000 7fd1a0277b80 4 client.14697 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:29:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.678+0000 7fd1a0277b80 1 -- 172.21.15.63:0/3721573234 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56176e4acd40 con 0x56176e596a20 2024-05-27T07:29:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.679+0000 7fd18aa00700 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56176e596a20 0x56176e597e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.680+0000 7fd18aa00700 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56176e596a20 0x56176e597e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9041d8d16538352 server_cookie=34f9dd79ac902ce8 in_seq=0 out_seq=0 2024-05-27T07:29:09.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.680+0000 7fd178c00700 10 client.14697 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:29:09.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.682+0000 7fd178c00700 1 -- 172.21.15.63:0/3721573234 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56176e4acd40 con 0x56176e596a20 2024-05-27T07:29:09.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.682+0000 7fd178c00700 10 client.14697 handle_command_reply: tid=0 2024-05-27T07:29:09.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.682+0000 7fd1a0277b80 1 client.14697 shutdown 2024-05-27T07:29:09.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.682+0000 7fd1a0277b80 1 -- 172.21.15.63:0/3721573234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 msgr2=0x56176e5870e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:09.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.682+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5870e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd184009430 tx=0x7fd1840099e0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.683+0000 7fd1a0277b80 1 -- 172.21.15.63:0/3721573234 shutdown_connections 2024-05-27T07:29:09.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.683+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56176e596a20 0x56176e597e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.683+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e5870e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.683+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e586660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.683+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/3721573234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e586ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.683+0000 7fd1a0277b80 1 -- 172.21.15.63:0/3721573234 >> 172.21.15.63:0/3721573234 conn(0x56176e497340 msgr2=0x56176e57c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:09.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.683+0000 7fd178c00700 0 client.14697 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:29:09.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.683+0000 7fd1a0277b80 1 -- 172.21.15.63:0/3721573234 shutdown_connections 2024-05-27T07:29:09.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.683+0000 7fd1a0277b80 1 -- 172.21.15.63:0/3721573234 wait complete. 2024-05-27T07:29:09.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.683+0000 7fd1a0277b80 20 client.14697 trim_cache size 0 max 16384 2024-05-27T07:29:09.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.707+0000 7fd1a0277b80 1 Processor -- start 2024-05-27T07:29:09.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.707+0000 7fd1a0277b80 1 -- start start 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.707+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e577f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.707+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e578440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.707+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e57c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.707+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56176e5961d0 con 0x56176e57ce20 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.707+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56176e5968a0 con 0x56176e3fddf0 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.707+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56176e596f70 con 0x56176e4e99a0 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.708+0000 7fd18b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e57c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.708+0000 7fd18b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e57c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52536/0 (socket says 172.21.15.63:52536) 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.708+0000 7fd18b400700 1 -- 172.21.15.63:0/4243173731 learned_addr learned my addr 172.21.15.63:0/4243173731 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.708+0000 7fd18aa00700 1 --2- 172.21.15.63:0/4243173731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e577f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.708+0000 7fd182000700 1 --2- 172.21.15.63:0/4243173731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e578440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.708+0000 7fd18b400700 1 -- 172.21.15.63:0/4243173731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 msgr2=0x56176e578440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:29:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.708+0000 7fd18b400700 1 --2- 172.21.15.63:0/4243173731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e578440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.708+0000 7fd18b400700 1 -- 172.21.15.63:0/4243173731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 msgr2=0x56176e577f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:29:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.708+0000 7fd18b400700 1 --2- 172.21.15.63:0/4243173731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e577f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.708+0000 7fd18b400700 1 -- 172.21.15.63:0/4243173731 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176e587450 con 0x56176e57ce20 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.709+0000 7fd18b400700 1 --2- 172.21.15.63:0/4243173731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e57c060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b830 tx=0x7fd1840090b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b0d099e9de76d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.709+0000 7fd17b400700 1 -- 172.21.15.63:0/4243173731 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18403e020 con 0x56176e57ce20 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.709+0000 7fd17b400700 1 -- 172.21.15.63:0/4243173731 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd184007d30 con 0x56176e57ce20 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.709+0000 7fd17b400700 1 -- 172.21.15.63:0/4243173731 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18402f040 con 0x56176e57ce20 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.710+0000 7fd1a0277b80 1 -- 172.21.15.63:0/4243173731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 msgr2=0x56176e57c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.710+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/4243173731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e57c060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b830 tx=0x7fd1840090b0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.710+0000 7fd1a0277b80 1 -- 172.21.15.63:0/4243173731 shutdown_connections 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.710+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/4243173731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e57c060 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.710+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/4243173731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e577f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.710+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/4243173731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e578440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.710+0000 7fd1a0277b80 1 -- 172.21.15.63:0/4243173731 >> 172.21.15.63:0/4243173731 conn(0x56176e497340 msgr2=0x56176e584f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:09.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.710+0000 7fd1a0277b80 1 -- 172.21.15.63:0/4243173731 shutdown_connections 2024-05-27T07:29:09.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.710+0000 7fd1a0277b80 1 -- 172.21.15.63:0/4243173731 wait complete. 2024-05-27T07:29:09.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.711+0000 7fd1a0277b80 1 -- start start 2024-05-27T07:29:09.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.711+0000 7fd1a0277b80 1 Processor -- start 2024-05-27T07:29:09.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e5805b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e580af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd1a0277b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e581030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56176e597d10 con 0x56176e57ce20 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56176e597640 con 0x56176e3fddf0 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd1a0277b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56176e5983e0 con 0x56176e4e99a0 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd1a0277b80 10 client.0 _ll_register_callbacks cb 0x56176e444c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd18b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e581030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd18b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e581030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52552/0 (socket says 172.21.15.63:52552) 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd18b400700 1 -- 172.21.15.63:0/2148516726 learned_addr learned my addr 172.21.15.63:0/2148516726 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd182000700 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e580af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.712+0000 7fd18aa00700 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e5805b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.713+0000 7fd18b400700 1 -- 172.21.15.63:0/2148516726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 msgr2=0x56176e580af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.713+0000 7fd18b400700 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e580af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.713+0000 7fd18b400700 1 -- 172.21.15.63:0/2148516726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 msgr2=0x56176e5805b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.713+0000 7fd18b400700 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e5805b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.713+0000 7fd18b400700 1 -- 172.21.15.63:0/2148516726 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176e5a0400 con 0x56176e57ce20 2024-05-27T07:29:09.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.713+0000 7fd18b400700 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e581030 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd184009430 tx=0x7fd184007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf05b3baa2a17630 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:09.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.713+0000 7fd178c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:29:09.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.714+0000 7fd178c00700 1 -- 172.21.15.63:0/2148516726 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18403e070 con 0x56176e57ce20 2024-05-27T07:29:09.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.714+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2148516726 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd184007d30 con 0x56176e57ce20 2024-05-27T07:29:09.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.714+0000 7fd178c00700 1 -- 172.21.15.63:0/2148516726 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd1840055b0 con 0x56176e57ce20 2024-05-27T07:29:09.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.714+0000 7fd178c00700 1 -- 172.21.15.63:0/2148516726 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd1840057f0 con 0x56176e57ce20 2024-05-27T07:29:09.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.714+0000 7fd1a0277b80 10 client.14703 fetch_fsmap learned FSMap version 17 2024-05-27T07:29:09.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.714+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2148516726 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56176e5a1640 con 0x56176e57ce20 2024-05-27T07:29:09.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.714+0000 7fd178c00700 1 -- 172.21.15.63:0/2148516726 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd184007d30 con 0x56176e57ce20 2024-05-27T07:29:09.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.714+0000 7fd178c00700 1 -- 172.21.15.63:0/2148516726 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fd184029050 con 0x56176e57ce20 2024-05-27T07:29:09.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.714+0000 7fd1a0277b80 10 client.14703 fetch_fsmap finished waiting for FSMap version 17 2024-05-27T07:29:09.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.715+0000 7fd1a0277b80 10 client.14703 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:29:09.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.715+0000 7fd1a0277b80 20 client.14703 populate_metadata read hostname 'smithi063' 2024-05-27T07:29:09.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.715+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56176e6035a0 0x56176e604a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:09.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.715+0000 7fd1a0277b80 4 client.14703 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:29:09.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.715+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2148516726 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "status"}) -- 0x56176e4a6520 con 0x56176e6035a0 2024-05-27T07:29:09.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.715+0000 7fd18aa00700 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56176e6035a0 0x56176e604a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:09.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.716+0000 7fd18aa00700 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56176e6035a0 0x56176e604a10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c09713da7ad7a10d server_cookie=638fcc20dd075143 in_seq=0 out_seq=0 2024-05-27T07:29:09.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.716+0000 7fd178c00700 10 client.14703 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:29:09.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd178c00700 1 -- 172.21.15.63:0/2148516726 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56176e4a6520 con 0x56176e6035a0 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd178c00700 10 client.14703 handle_command_reply: tid=0 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 client.14703 shutdown 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2148516726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 msgr2=0x56176e581030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e581030 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd184009430 tx=0x7fd184007ab0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2148516726 shutdown_connections 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56176e6035a0 0x56176e604a10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56176e57ce20 0x56176e581030 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176e3fddf0 0x56176e5805b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 --2- 172.21.15.63:0/2148516726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56176e4e99a0 0x56176e580af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd178c00700 0 client.14703 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2148516726 >> 172.21.15.63:0/2148516726 conn(0x56176e497340 msgr2=0x56176e57c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2148516726 shutdown_connections 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 1 -- 172.21.15.63:0/2148516726 wait complete. 2024-05-27T07:29:09.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.717+0000 7fd1a0277b80 20 client.14703 trim_cache size 0 max 16384 2024-05-27T07:29:09.710 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:29:09.710 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:29:09.710 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 0, 2024-05-27T07:29:09.710 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34520, 2024-05-27T07:29:09.710 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-27T07:29:09.710 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-27T07:29:09.710 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-27T07:29:09.710 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 42.101817507, 2024-05-27T07:29:09.711 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 17, 2024-05-27T07:29:09.711 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 89, 2024-05-27T07:29:09.711 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 87, 2024-05-27T07:29:09.711 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 63.486034904999997 2024-05-27T07:29:09.711 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:29:09.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.718+0000 7fd191800700 1 -- 172.21.15.63:0/3080033413 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1700b2880 msgr2=0x7fd1700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:09.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.718+0000 7fd191800700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1700b2880 0x7fd1700b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x56176e57efa0 tx=0x7fd18400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.718+0000 7fd191800700 1 -- 172.21.15.63:0/3080033413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd18c105e40 msgr2=0x7fd18c103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:09.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.718+0000 7fd191800700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd18c105e40 0x7fd18c103720 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd17400c9b0 tx=0x7fd17400cd70 comp rx=0 tx=0).stop 2024-05-27T07:29:09.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.718+0000 7fd191800700 1 -- 172.21.15.63:0/3080033413 shutdown_connections 2024-05-27T07:29:09.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.719+0000 7fd191800700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd18c10e8d0 0x7fd18c103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.719+0000 7fd191800700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd18c1054f0 0x7fd18c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.719+0000 7fd191800700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1700b2880 0x7fd1700b4d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.719+0000 7fd191800700 1 --2- 172.21.15.63:0/3080033413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd18c105e40 0x7fd18c103720 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:09.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.719+0000 7fd191800700 1 -- 172.21.15.63:0/3080033413 >> 172.21.15.63:0/3080033413 conn(0x7fd18c0fedb0 msgr2=0x7fd18c100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:09.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.719+0000 7fd191800700 1 -- 172.21.15.63:0/3080033413 shutdown_connections 2024-05-27T07:29:09.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:09.719+0000 7fd191800700 1 -- 172.21.15.63:0/3080033413 wait complete. 2024-05-27T07:29:10.595 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T07:29:10.595 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:29:10.595 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-27T07:29:11.054 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:11.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:10 smithi086 ceph-mon[34687]: pgmap v412: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:11.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:10 smithi149 ceph-mon[35408]: pgmap v412: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:11.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:10 smithi063 ceph-mon[31613]: pgmap v412: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:12.994 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:12 smithi063 ceph-mon[31613]: pgmap v413: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:13.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:12 smithi086 ceph-mon[34687]: pgmap v413: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:13.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:12 smithi149 ceph-mon[35408]: pgmap v413: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.381+0000 7fe395a00700 1 -- 172.21.15.63:0/168302536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe39010e8c0 msgr2=0x7fe390110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.381+0000 7fe395a00700 1 --2- 172.21.15.63:0/168302536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe39010e8c0 0x7fe390110da0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe38800b0a0 tx=0x7fe38800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.381+0000 7fe395a00700 1 -- 172.21.15.63:0/168302536 shutdown_connections 2024-05-27T07:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.381+0000 7fe395a00700 1 --2- 172.21.15.63:0/168302536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe39010e8c0 0x7fe390110da0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.381+0000 7fe395a00700 1 --2- 172.21.15.63:0/168302536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe390105450 0x7fe390105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.381+0000 7fe395a00700 1 --2- 172.21.15.63:0/168302536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe390105da0 0x7fe39010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.381+0000 7fe395a00700 1 -- 172.21.15.63:0/168302536 >> 172.21.15.63:0/168302536 conn(0x7fe3900fedb0 msgr2=0x7fe3901011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.381+0000 7fe395a00700 1 -- 172.21.15.63:0/168302536 shutdown_connections 2024-05-27T07:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.381+0000 7fe395a00700 1 -- 172.21.15.63:0/168302536 wait complete. 2024-05-27T07:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.382+0000 7fe395a00700 1 Processor -- start 2024-05-27T07:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.382+0000 7fe395a00700 1 -- start start 2024-05-27T07:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe395a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe390105450 0x7fe390103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe395a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe390105da0 0x7fe390103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe395a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe39010e8c0 0x7fe390103bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe395a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe390114300 con 0x7fe390105da0 2024-05-27T07:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe395a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe390114180 con 0x7fe390105450 2024-05-27T07:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe395a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe390114480 con 0x7fe39010e8c0 2024-05-27T07:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe38e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe390105da0 0x7fe390103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe38e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe390105da0 0x7fe390103680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52566/0 (socket says 172.21.15.63:52566) 2024-05-27T07:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe38e000700 1 -- 172.21.15.63:0/3069421006 learned_addr learned my addr 172.21.15.63:0/3069421006 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe38f400700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe39010e8c0 0x7fe390103bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.383+0000 7fe38ea00700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe390105450 0x7fe390103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.384+0000 7fe38e000700 1 -- 172.21.15.63:0/3069421006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe39010e8c0 msgr2=0x7fe390103bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.384+0000 7fe38e000700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe39010e8c0 0x7fe390103bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.384+0000 7fe38e000700 1 -- 172.21.15.63:0/3069421006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe390105450 msgr2=0x7fe390103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.384+0000 7fe38e000700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe390105450 0x7fe390103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.384+0000 7fe38e000700 1 -- 172.21.15.63:0/3069421006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3901a9030 con 0x7fe390105da0 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.384+0000 7fe38ea00700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe390105450 0x7fe390103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.384+0000 7fe38e000700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe390105da0 0x7fe390103680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00d980 tx=0x7fe37c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.385+0000 7fe387400700 1 -- 172.21.15.63:0/3069421006 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c017070 con 0x7fe390105da0 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.385+0000 7fe395a00700 1 -- 172.21.15.63:0/3069421006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3901a92c0 con 0x7fe390105da0 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.385+0000 7fe387400700 1 -- 172.21.15.63:0/3069421006 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe37c005020 con 0x7fe390105da0 2024-05-27T07:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.385+0000 7fe387400700 1 -- 172.21.15.63:0/3069421006 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c009ab0 con 0x7fe390105da0 2024-05-27T07:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.385+0000 7fe395a00700 1 -- 172.21.15.63:0/3069421006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3901a9800 con 0x7fe390105da0 2024-05-27T07:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.386+0000 7fe395a00700 1 -- 172.21.15.63:0/3069421006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3900512b0 con 0x7fe390105da0 2024-05-27T07:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.392+0000 7fe387400700 1 -- 172.21.15.63:0/3069421006 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe37c0051d0 con 0x7fe390105da0 2024-05-27T07:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.393+0000 7fe387400700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe3740b2880 0x7fe3740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.393+0000 7fe387400700 1 -- 172.21.15.63:0/3069421006 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe37c0e28a0 con 0x7fe390105da0 2024-05-27T07:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.394+0000 7fe38ea00700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe3740b2880 0x7fe3740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:13.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.394+0000 7fe38ea00700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe3740b2880 0x7fe3740b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe380005950 tx=0x7fe3800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:13.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.395+0000 7fe387400700 1 -- 172.21.15.63:0/3069421006 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe37c020020 con 0x7fe390105da0 2024-05-27T07:29:13.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.685+0000 7fe395a00700 1 -- 172.21.15.63:0/3069421006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe39002e5e0 con 0x7fe390105da0 2024-05-27T07:29:13.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.686+0000 7fe387400700 1 -- 172.21.15.63:0/3069421006 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fe37c0aa200 con 0x7fe390105da0 2024-05-27T07:29:13.676 INFO:teuthology.orchestra.run.smithi063.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T07:29:13.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 -- 172.21.15.63:0/3069421006 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe3740b2880 msgr2=0x7fe3740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe3740b2880 0x7fe3740b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe380005950 tx=0x7fe3800058e0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 -- 172.21.15.63:0/3069421006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe390105da0 msgr2=0x7fe390103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe390105da0 0x7fe390103680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00d980 tx=0x7fe37c00dc90 comp rx=0 tx=0).stop 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 -- 172.21.15.63:0/3069421006 shutdown_connections 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe39010e8c0 0x7fe390103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe390105450 0x7fe390103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe3740b2880 0x7fe3740b4d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 --2- 172.21.15.63:0/3069421006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe390105da0 0x7fe390103680 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 -- 172.21.15.63:0/3069421006 >> 172.21.15.63:0/3069421006 conn(0x7fe3900fedb0 msgr2=0x7fe3900ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 -- 172.21.15.63:0/3069421006 shutdown_connections 2024-05-27T07:29:13.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:13.690+0000 7fe395a00700 1 -- 172.21.15.63:0/3069421006 wait complete. 2024-05-27T07:29:13.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:13 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3069421006' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:29:14.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:13 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3069421006' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:29:14.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:13 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3069421006' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:29:14.489 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-27T07:29:14.489 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T07:29:14.489 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-27T07:29:14.489 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-27T07:29:14.489 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:29:14.489 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-27T07:29:14.489 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T07:29:14.490 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T07:29:14.490 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-27T07:29:14.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:14.490 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-27T07:29:14.551 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T07:29:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T07:29:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T07:29:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:29:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-27T07:29:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:29:14.552 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:29:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:29:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-27T07:29:14.552 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:29:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:29:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-27T07:29:14.553 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:29:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:29:14.553 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T07:29:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:29:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T07:29:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 41802sec preferred_lft 41802sec 2024-05-27T07:29:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-27T07:29:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:29:14.555 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T07:29:14.555 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:29:14.555 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-27T07:29:14.555 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add name ceph-brx type bridge 2024-05-27T07:29:14.555 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr flush dev ceph-brx 2024-05-27T07:29:14.555 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set ceph-brx up 2024-05-27T07:29:14.555 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T07:29:14.555 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-27T07:29:14.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:29:14.944 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:14 smithi063 ceph-mon[31613]: pgmap v414: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:15.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:29:15.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:15.057 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T07:29:15.131 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-27T07:29:15.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:15.135 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-27T07:29:15.189 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-27T07:29:15.189 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T07:29:15.189 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T07:29:15.190 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T07:29:15.190 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T07:29:15.190 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:29:15.190 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-27T07:29:15.190 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T07:29:15.190 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T07:29:15.190 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T07:29:15.190 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-27T07:29:15.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:14 smithi086 ceph-mon[34687]: pgmap v414: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:15.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:14 smithi149 ceph-mon[35408]: pgmap v414: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:15.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:29:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:29:15.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:15.940 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-27T07:29:15.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:15.996 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-27T07:29:16.049 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:29:16.049 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-27T07:29:16.049 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:29:16.049 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-27T07:29:16.049 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-27T07:29:16.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:29:16.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:29:16.243 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-27T07:29:16.243 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:29:16.244 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-27T07:29:16.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-27T07:29:16.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T07:29:16.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-27T07:29:16.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-27T07:29:16.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-27T07:29:16.244 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-27T07:29:16.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:29:16.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:29:16.579 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:29:16.579 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-27T07:29:16.579 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.0 up 2024-05-27T07:29:16.579 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T07:29:16.579 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-27T07:29:16.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:29:16.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:29:16.790 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-27T07:29:16.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T07:29:16.790 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:29:16.840 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-27T07:29:16.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T07:29:16.840 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:29:16.937 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-27T07:29:16.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:16 smithi063 ceph-mon[31613]: pgmap v415: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:17.013 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:29:17.116 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-27T07:29:17.116 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:29:17.117 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-27T07:29:17.118 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/29dea1c71f72254dbd38c06f7960bb9abb9f32ebd5c2c16eab1643b7188c5af2/merged 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/9d16fb203407a66c05d9b3d5a493a1d4434dc9cf09cd534e440f7fc09a15706b/merged 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/46f73ca9e7d66f52b311c4f5b060aba62e6d669224020624d26585980c697b13/merged 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2fbdf4f593ac9a3902c7e08243d1db408f4800bb2e9402d0412cfc431cfa7cc3/merged 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/080203433eaec43264a1bfede6815473130384fa7fdd4e942d6a178f0712dc7d/merged 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1bb182d89c236e7f9ab32900f23cef8a891842dcb1573f3a6e706b02621a143b/merged 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/35f9e9a1c864c5fd7d1b53a6016b7f186820d58461be037237931b30dfeb854a/merged 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/fc5228364a0f07c2d16a3bac26cf2c5be6b35931679b5f73919e28de51d6c9ef/merged 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/6509a31a1e961e9c9068d336ed9fe585f994f3b01fabdc9478177855ede51b69/merged 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2faabbd5b4689eb81b028bb408e5256b283410519a885ad5672e4555bcd53948/merged 2024-05-27T07:29:17.119 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-27T07:29:17.120 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:29:17.120 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:29:17.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:17.121 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-27T07:29:17.175 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-27T07:29:17.176 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-27T07:29:17.181 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-27T07:29:17.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:16 smithi086 ceph-mon[34687]: pgmap v415: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:17.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:16 smithi149 ceph-mon[35408]: pgmap v415: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:17.254 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:29:17.297 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-27T07:29:17.297 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-27T07:29:17.297 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-27T07:29:17.297 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-27T07:29:17.297 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-27T07:29:17.297 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-27T07:29:17.297 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-27T07:29:17.297 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-27T07:29:17.297 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-27T07:29:17.297 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:29:17.298 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/29dea1c71f72254dbd38c06f7960bb9abb9f32ebd5c2c16eab1643b7188c5af2/merged 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/9d16fb203407a66c05d9b3d5a493a1d4434dc9cf09cd534e440f7fc09a15706b/merged 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/46f73ca9e7d66f52b311c4f5b060aba62e6d669224020624d26585980c697b13/merged 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2fbdf4f593ac9a3902c7e08243d1db408f4800bb2e9402d0412cfc431cfa7cc3/merged 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/080203433eaec43264a1bfede6815473130384fa7fdd4e942d6a178f0712dc7d/merged 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1bb182d89c236e7f9ab32900f23cef8a891842dcb1573f3a6e706b02621a143b/merged 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/35f9e9a1c864c5fd7d1b53a6016b7f186820d58461be037237931b30dfeb854a/merged 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/fc5228364a0f07c2d16a3bac26cf2c5be6b35931679b5f73919e28de51d6c9ef/merged 2024-05-27T07:29:17.299 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/6509a31a1e961e9c9068d336ed9fe585f994f3b01fabdc9478177855ede51b69/merged 2024-05-27T07:29:17.300 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2faabbd5b4689eb81b028bb408e5256b283410519a885ad5672e4555bcd53948/merged 2024-05-27T07:29:17.300 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-27T07:29:17.300 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:29:17.300 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:29:17.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:17.301 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-27T07:29:17.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi063.stderr:2024-05-27T07:29:17.537+0000 7f09857b8480 -1 init, newargv = 0x7f0978004f20 newargc=15 2024-05-27T07:29:17.528 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi063.stderr:ceph-fuse[77829]: starting ceph client 2024-05-27T07:29:17.546 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi063.stderr:ceph-fuse[77829]: starting fuse 2024-05-27T07:29:18.499 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-27T07:29:18.597 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:29:18.658 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-27T07:29:18.658 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-27T07:29:18.658 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-27T07:29:18.658 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-27T07:29:18.658 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-27T07:29:18.658 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-27T07:29:18.658 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-27T07:29:18.658 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-27T07:29:18.658 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-27T07:29:18.658 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:29:18.659 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/29dea1c71f72254dbd38c06f7960bb9abb9f32ebd5c2c16eab1643b7188c5af2/merged 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/9d16fb203407a66c05d9b3d5a493a1d4434dc9cf09cd534e440f7fc09a15706b/merged 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/46f73ca9e7d66f52b311c4f5b060aba62e6d669224020624d26585980c697b13/merged 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2fbdf4f593ac9a3902c7e08243d1db408f4800bb2e9402d0412cfc431cfa7cc3/merged 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/080203433eaec43264a1bfede6815473130384fa7fdd4e942d6a178f0712dc7d/merged 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1bb182d89c236e7f9ab32900f23cef8a891842dcb1573f3a6e706b02621a143b/merged 2024-05-27T07:29:18.660 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/35f9e9a1c864c5fd7d1b53a6016b7f186820d58461be037237931b30dfeb854a/merged 2024-05-27T07:29:18.661 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/fc5228364a0f07c2d16a3bac26cf2c5be6b35931679b5f73919e28de51d6c9ef/merged 2024-05-27T07:29:18.661 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/6509a31a1e961e9c9068d336ed9fe585f994f3b01fabdc9478177855ede51b69/merged 2024-05-27T07:29:18.661 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2faabbd5b4689eb81b028bb408e5256b283410519a885ad5672e4555bcd53948/merged 2024-05-27T07:29:18.661 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-27T07:29:18.661 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:29:18.661 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:29:18.661 INFO:teuthology.orchestra.run.smithi063.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-27T07:29:18.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:18.662 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-27T07:29:18.720 INFO:teuthology.orchestra.run.smithi063.stdout:119 2024-05-27T07:29:18.720 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-05-27T07:29:18.720 DEBUG:teuthology.orchestra.run.smithi063:> sudo stdin-killer -- python3 -c ' 2024-05-27T07:29:18.720 DEBUG:teuthology.orchestra.run.smithi063:> import glob 2024-05-27T07:29:18.720 DEBUG:teuthology.orchestra.run.smithi063:> import re 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> import os 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> import subprocess 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> def _find_admin_socket(client_name): 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> files = glob.glob(asok_path) 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> # Given a non-glob path, it better be there 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> if "*" not in asok_path: 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> assert(len(files) == 1) 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> return files[0] 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> for f in files: 2024-05-27T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi063:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-27T07:29:18.722 DEBUG:teuthology.orchestra.run.smithi063:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-27T07:29:18.722 DEBUG:teuthology.orchestra.run.smithi063:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-27T07:29:18.722 DEBUG:teuthology.orchestra.run.smithi063:> contents = proc_f.read() 2024-05-27T07:29:18.722 DEBUG:teuthology.orchestra.run.smithi063:> if mountpoint in contents: 2024-05-27T07:29:18.722 DEBUG:teuthology.orchestra.run.smithi063:> return f 2024-05-27T07:29:18.722 DEBUG:teuthology.orchestra.run.smithi063:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-27T07:29:18.722 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-27T07:29:18.722 DEBUG:teuthology.orchestra.run.smithi063:> print(_find_admin_socket("client.admin")) 2024-05-27T07:29:18.722 DEBUG:teuthology.orchestra.run.smithi063:> ' 2024-05-27T07:29:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:/var/run/ceph/ceph-client.admin.77829.asok 2024-05-27T07:29:18.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:29:18.913 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.77829.asok 2024-05-27T07:29:18.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:18.913 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.77829.asok status 2024-05-27T07:29:19.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:18 smithi086 ceph-mon[34687]: pgmap v416: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:19.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:18 smithi149 ceph-mon[35408]: pgmap v416: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:19.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:18 smithi063 ceph-mon[31613]: pgmap v416: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:29:19.218 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:29:19.218 INFO:teuthology.orchestra.run.smithi063.stdout: "metadata": { 2024-05-27T07:29:19.218 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_sha1": "38f9d900c83c63799fbdbe61acc9a11b0d3554a6", 2024-05-27T07:29:19.218 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:29:19.218 INFO:teuthology.orchestra.run.smithi063.stdout: "entity_id": "admin", 2024-05-27T07:29:19.218 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": "smithi063", 2024-05-27T07:29:19.218 INFO:teuthology.orchestra.run.smithi063.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-27T07:29:19.218 INFO:teuthology.orchestra.run.smithi063.stdout: "pid": "77829", 2024-05-27T07:29:19.218 INFO:teuthology.orchestra.run.smithi063.stdout: "root": "/" 2024-05-27T07:29:19.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_count": 0, 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_pinned_count": 0, 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 14715, 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "inst": { 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "name": { 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "client", 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "num": 14715 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3663901675 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:29:19.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3663901675 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "inst_str": "client.14715 192.168.0.1:0/3663901675", 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "addr_str": "192.168.0.1:0/3663901675", 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "inode_count": 1, 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_epoch": 17, 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch": 89, 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch_barrier": 0, 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklisted": false, 2024-05-27T07:29:19.220 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs" 2024-05-27T07:29:19.221 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:29:19.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:19.240 DEBUG:teuthology.orchestra.run.smithi063:> stat --file-system '--printf=%T 2024-05-27T07:29:19.240 DEBUG:teuthology.orchestra.run.smithi063:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:29:19.298 INFO:teuthology.orchestra.run.smithi063.stdout:fuseblk 2024-05-27T07:29:19.299 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:29:19.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:19.299 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:29:19.391 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-27T07:29:19.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T07:29:19.555 INFO:teuthology.orchestra.run.smithi063.stdout:# file: . 2024-05-27T07:29:19.555 INFO:teuthology.orchestra.run.smithi063.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-27T07:29:19.555 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:29:19.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:29:19.566 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:29:19.566 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T07:29:19.624 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T07:29:19.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:19 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:29:20.088 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:20.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:19 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:29:20.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:19 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:29:21.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:20 smithi086 ceph-mon[34687]: pgmap v417: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T07:29:21.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:20 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:20 smithi149 ceph-mon[35408]: pgmap v417: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T07:29:21.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:20 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:20 smithi063 ceph-mon[31613]: pgmap v417: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T07:29:21.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:21.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:20 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:22.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.574+0000 7f1d37600700 1 -- 172.21.15.63:0/882356004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d30106570 msgr2=0x7f1d30106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:22.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.574+0000 7f1d37600700 1 --2- 172.21.15.63:0/882356004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d30106570 0x7f1d30106980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009990 tx=0x7f1d1c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.574+0000 7f1d37600700 1 -- 172.21.15.63:0/882356004 shutdown_connections 2024-05-27T07:29:22.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.574+0000 7f1d37600700 1 --2- 172.21.15.63:0/882356004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d30106ec0 0x7f1d30113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.574+0000 7f1d37600700 1 --2- 172.21.15.63:0/882356004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d3010c5b0 0x7f1d3010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.574+0000 7f1d37600700 1 --2- 172.21.15.63:0/882356004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d30106570 0x7f1d30106980 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.574+0000 7f1d37600700 1 -- 172.21.15.63:0/882356004 >> 172.21.15.63:0/882356004 conn(0x7f1d30102070 msgr2=0x7f1d301044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:22.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.574+0000 7f1d37600700 1 -- 172.21.15.63:0/882356004 shutdown_connections 2024-05-27T07:29:22.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.575+0000 7f1d37600700 1 -- 172.21.15.63:0/882356004 wait complete. 2024-05-27T07:29:22.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.576+0000 7f1d37600700 1 Processor -- start 2024-05-27T07:29:22.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.576+0000 7f1d37600700 1 -- start start 2024-05-27T07:29:22.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d37600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d30106570 0x7f1d3019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:22.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d37600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30106ec0 0x7f1d3019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:22.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d37600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3010c5b0 0x7f1d301a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:22.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d37600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30116590 con 0x7f1d30106570 2024-05-27T07:29:22.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d37600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30116410 con 0x7f1d3010c5b0 2024-05-27T07:29:22.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d37600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30116710 con 0x7f1d30106ec0 2024-05-27T07:29:22.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d34c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d30106570 0x7f1d3019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:22.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d2fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30106ec0 0x7f1d3019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:22.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d34c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d30106570 0x7f1d3019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59486/0 (socket says 172.21.15.63:59486) 2024-05-27T07:29:22.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d2fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30106ec0 0x7f1d3019d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:43394/0 (socket says 172.21.15.63:43394) 2024-05-27T07:29:22.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d34c00700 1 -- 172.21.15.63:0/771080196 learned_addr learned my addr 172.21.15.63:0/771080196 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:22.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.577+0000 7f1d35600700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3010c5b0 0x7f1d301a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:22.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.578+0000 7f1d2fe00700 1 -- 172.21.15.63:0/771080196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3010c5b0 msgr2=0x7f1d301a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:22.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.578+0000 7f1d2fe00700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3010c5b0 0x7f1d301a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.578+0000 7f1d2fe00700 1 -- 172.21.15.63:0/771080196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d30106570 msgr2=0x7f1d3019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:22.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.578+0000 7f1d2fe00700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d30106570 0x7f1d3019cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.578+0000 7f1d2fe00700 1 -- 172.21.15.63:0/771080196 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d301a1e30 con 0x7f1d30106ec0 2024-05-27T07:29:22.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.578+0000 7f1d2fe00700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30106ec0 0x7f1d3019d500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c000c00 tx=0x7f1d1c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:22.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.578+0000 7f1d2d600700 1 -- 172.21.15.63:0/771080196 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c026070 con 0x7f1d30106ec0 2024-05-27T07:29:22.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.578+0000 7f1d2d600700 1 -- 172.21.15.63:0/771080196 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d1c021410 con 0x7f1d30106ec0 2024-05-27T07:29:22.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.578+0000 7f1d37600700 1 -- 172.21.15.63:0/771080196 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d301a20c0 con 0x7f1d30106ec0 2024-05-27T07:29:22.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.579+0000 7f1d2d600700 1 -- 172.21.15.63:0/771080196 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c017650 con 0x7f1d30106ec0 2024-05-27T07:29:22.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.579+0000 7f1d37600700 1 -- 172.21.15.63:0/771080196 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301a25a0 con 0x7f1d30106ec0 2024-05-27T07:29:22.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.580+0000 7f1d37600700 1 -- 172.21.15.63:0/771080196 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d300512b0 con 0x7f1d30106ec0 2024-05-27T07:29:22.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.582+0000 7f1d2d600700 1 -- 172.21.15.63:0/771080196 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1d1c029070 con 0x7f1d30106ec0 2024-05-27T07:29:22.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.584+0000 7f1d2d600700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d180b2880 0x7f1d180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:22.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.584+0000 7f1d2d600700 1 -- 172.21.15.63:0/771080196 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1d1c0e33f0 con 0x7f1d30106ec0 2024-05-27T07:29:22.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.584+0000 7f1d34c00700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d180b2880 0x7f1d180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:22.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.585+0000 7f1d34c00700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d180b2880 0x7f1d180b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009680 tx=0x7f1d24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:22.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.589+0000 7f1d2d600700 1 -- 172.21.15.63:0/771080196 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d1c05edf0 con 0x7f1d30106ec0 2024-05-27T07:29:22.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.848+0000 7f1d37600700 1 -- 172.21.15.63:0/771080196 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1d3002e5e0 con 0x7f1d30106ec0 2024-05-27T07:29:22.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.848+0000 7f1d2d600700 1 -- 172.21.15.63:0/771080196 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+589 (secure 0 0 0) 0x7f1d1c013070 con 0x7f1d30106ec0 2024-05-27T07:29:22.840 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1689640179 2024-05-28T07:18:03.559305+0000 2024-05-27T07:29:22.840 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3946208673 2024-05-28T07:18:03.559305+0000 2024-05-27T07:29:22.840 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/736953168 2024-05-28T07:18:03.559305+0000 2024-05-27T07:29:22.840 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/3806249841 2024-05-28T07:17:27.459562+0000 2024-05-27T07:29:22.840 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1689640179 2024-05-28T07:18:03.559305+0000 2024-05-27T07:29:22.840 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4210215351 2024-05-28T07:17:27.459562+0000 2024-05-27T07:29:22.840 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/3806249841 2024-05-28T07:17:27.459562+0000 2024-05-27T07:29:22.840 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2613878696 2024-05-28T07:17:27.459562+0000 2024-05-27T07:29:22.840 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1318703057 2024-05-28T07:17:27.459562+0000 2024-05-27T07:29:22.840 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/58174792 2024-05-28T07:18:03.559305+0000 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.851+0000 7f1d37600700 1 -- 172.21.15.63:0/771080196 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d180b2880 msgr2=0x7f1d180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d180b2880 0x7f1d180b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009680 tx=0x7f1d24009260 comp rx=0 tx=0).stop 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 -- 172.21.15.63:0/771080196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30106ec0 msgr2=0x7f1d3019d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30106ec0 0x7f1d3019d500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c000c00 tx=0x7f1d1c0057c0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 -- 172.21.15.63:0/771080196 shutdown_connections 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3010c5b0 0x7f1d301a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d180b2880 0x7f1d180b4d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d30106570 0x7f1d3019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 --2- 172.21.15.63:0/771080196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30106ec0 0x7f1d3019d500 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 -- 172.21.15.63:0/771080196 >> 172.21.15.63:0/771080196 conn(0x7f1d30102070 msgr2=0x7f1d30104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:22.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 -- 172.21.15.63:0/771080196 shutdown_connections 2024-05-27T07:29:22.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:22.852+0000 7f1d37600700 1 -- 172.21.15.63:0/771080196 wait complete. 2024-05-27T07:29:22.844 INFO:teuthology.orchestra.run.smithi063.stderr:listed 10 entries 2024-05-27T07:29:23.198 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:22 smithi086 ceph-mon[34687]: pgmap v418: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T07:29:23.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:22 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/771080196' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:29:23.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:22 smithi149 ceph-mon[35408]: pgmap v418: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T07:29:23.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:22 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/771080196' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:29:23.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:22 smithi063 ceph-mon[31613]: pgmap v418: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T07:29:23.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:22 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/771080196' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:29:23.663 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:29:23.663 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T07:29:23.708 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T07:29:24.194 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:24.658 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.658 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.658 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.658 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.658 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:24 smithi063 ceph-mon[31613]: pgmap v419: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T07:29:24.658 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.658 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:24 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.949 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:24 smithi086 ceph-mon[34687]: pgmap v419: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T07:29:24.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.950 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:24 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:24 smithi149 ceph-mon[35408]: pgmap v419: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-27T07:29:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:24.953 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:24 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:26.063 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:25 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:29:26.063 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:25 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:29:26.063 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:25 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:26.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:25 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:29:26.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:25 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:29:26.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:25 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:26.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:25 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:29:26.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:25 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:29:26.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:25 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:29:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.486+0000 7f1e26e00700 1 -- 172.21.15.63:0/2962181450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201084c0 msgr2=0x7f1e20108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.486+0000 7f1e26e00700 1 --2- 172.21.15.63:0/2962181450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201084c0 0x7f1e20108990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800b0a0 tx=0x7f1e1800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.486+0000 7f1e26e00700 1 -- 172.21.15.63:0/2962181450 shutdown_connections 2024-05-27T07:29:26.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.486+0000 7f1e26e00700 1 --2- 172.21.15.63:0/2962181450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201084c0 0x7f1e20108990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.486+0000 7f1e26e00700 1 --2- 172.21.15.63:0/2962181450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e20103070 0x7f1e20103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.486+0000 7f1e26e00700 1 --2- 172.21.15.63:0/2962181450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e201039c0 0x7f1e20107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.486+0000 7f1e26e00700 1 -- 172.21.15.63:0/2962181450 >> 172.21.15.63:0/2962181450 conn(0x7f1e20078af0 msgr2=0x7f1e20078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:26.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.486+0000 7f1e26e00700 1 -- 172.21.15.63:0/2962181450 shutdown_connections 2024-05-27T07:29:26.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.486+0000 7f1e26e00700 1 -- 172.21.15.63:0/2962181450 wait complete. 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.487+0000 7f1e26e00700 1 Processor -- start 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.487+0000 7f1e26e00700 1 -- start start 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.488+0000 7f1e26e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103070 0x7f1e201a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.488+0000 7f1e26e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e201039c0 0x7f1e201a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.488+0000 7f1e26e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201084c0 0x7f1e201a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.488+0000 7f1e26e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2011a700 con 0x7f1e201084c0 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.488+0000 7f1e26e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2011a580 con 0x7f1e20103070 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.488+0000 7f1e26e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2011a880 con 0x7f1e201039c0 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.488+0000 7f1e26400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201084c0 0x7f1e201a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.488+0000 7f1e26400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201084c0 0x7f1e201a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59506/0 (socket says 172.21.15.63:59506) 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.488+0000 7f1e26400700 1 -- 172.21.15.63:0/3566078590 learned_addr learned my addr 172.21.15.63:0/3566078590 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:26.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.488+0000 7f1e25000700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e201039c0 0x7f1e201a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.489+0000 7f1e25a00700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103070 0x7f1e201a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.489+0000 7f1e26400700 1 -- 172.21.15.63:0/3566078590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e201039c0 msgr2=0x7f1e201a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.489+0000 7f1e26400700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e201039c0 0x7f1e201a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.489+0000 7f1e26400700 1 -- 172.21.15.63:0/3566078590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103070 msgr2=0x7f1e201a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.489+0000 7f1e26400700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103070 0x7f1e201a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.489+0000 7f1e26400700 1 -- 172.21.15.63:0/3566078590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e201a5f60 con 0x7f1e201084c0 2024-05-27T07:29:26.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.489+0000 7f1e26400700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201084c0 0x7f1e201a58c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800b830 tx=0x7f1e18004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.489+0000 7f1e1e000700 1 -- 172.21.15.63:0/3566078590 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18021070 con 0x7f1e201084c0 2024-05-27T07:29:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.490+0000 7f1e1e000700 1 -- 172.21.15.63:0/3566078590 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e18007c80 con 0x7f1e201084c0 2024-05-27T07:29:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.490+0000 7f1e26e00700 1 -- 172.21.15.63:0/3566078590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e201a61f0 con 0x7f1e201084c0 2024-05-27T07:29:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.490+0000 7f1e1e000700 1 -- 172.21.15.63:0/3566078590 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18005260 con 0x7f1e201084c0 2024-05-27T07:29:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.490+0000 7f1e26e00700 1 -- 172.21.15.63:0/3566078590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201adb70 con 0x7f1e201084c0 2024-05-27T07:29:26.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.491+0000 7f1e26e00700 1 -- 172.21.15.63:0/3566078590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2006c910 con 0x7f1e201084c0 2024-05-27T07:29:26.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.494+0000 7f1e1e000700 1 -- 172.21.15.63:0/3566078590 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1e18024070 con 0x7f1e201084c0 2024-05-27T07:29:26.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.495+0000 7f1e1e000700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1e080b2900 0x7f1e080b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:26.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.496+0000 7f1e1e000700 1 -- 172.21.15.63:0/3566078590 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1e1800f020 con 0x7f1e201084c0 2024-05-27T07:29:26.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.496+0000 7f1e25a00700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1e080b2900 0x7f1e080b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:26.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.497+0000 7f1e25a00700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1e080b2900 0x7f1e080b4e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1e14005950 tx=0x7f1e140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.498+0000 7f1e1e000700 1 -- 172.21.15.63:0/3566078590 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e1802b020 con 0x7f1e201084c0 2024-05-27T07:29:26.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.752+0000 7f1e26e00700 1 -- 172.21.15.63:0/3566078590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1e200512b0 con 0x7f1e201084c0 2024-05-27T07:29:26.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.759+0000 7f1e1e000700 1 -- 172.21.15.63:0/3566078590 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+589 (secure 0 0 0) 0x7f1e180aba60 con 0x7f1e201084c0 2024-05-27T07:29:26.751 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1689640179 2024-05-28T07:18:03.559305+0000 2024-05-27T07:29:26.751 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3946208673 2024-05-28T07:18:03.559305+0000 2024-05-27T07:29:26.751 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/736953168 2024-05-28T07:18:03.559305+0000 2024-05-27T07:29:26.751 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/3806249841 2024-05-28T07:17:27.459562+0000 2024-05-27T07:29:26.751 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1689640179 2024-05-28T07:18:03.559305+0000 2024-05-27T07:29:26.752 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4210215351 2024-05-28T07:17:27.459562+0000 2024-05-27T07:29:26.752 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/3806249841 2024-05-28T07:17:27.459562+0000 2024-05-27T07:29:26.752 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2613878696 2024-05-28T07:17:27.459562+0000 2024-05-27T07:29:26.752 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1318703057 2024-05-28T07:17:27.459562+0000 2024-05-27T07:29:26.752 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/58174792 2024-05-28T07:18:03.559305+0000 2024-05-27T07:29:26.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.762+0000 7f1e26e00700 1 -- 172.21.15.63:0/3566078590 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1e080b2900 msgr2=0x7f1e080b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:26.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.762+0000 7f1e26e00700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1e080b2900 0x7f1e080b4e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1e14005950 tx=0x7f1e140058e0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.762+0000 7f1e26e00700 1 -- 172.21.15.63:0/3566078590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201084c0 msgr2=0x7f1e201a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:26.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.762+0000 7f1e26e00700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201084c0 0x7f1e201a58c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800b830 tx=0x7f1e18004db0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.763+0000 7f1e26e00700 1 -- 172.21.15.63:0/3566078590 shutdown_connections 2024-05-27T07:29:26.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.763+0000 7f1e26e00700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1e080b2900 0x7f1e080b4e10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.763+0000 7f1e26e00700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e201084c0 0x7f1e201a58c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.763+0000 7f1e26e00700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103070 0x7f1e201a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.763+0000 7f1e26e00700 1 --2- 172.21.15.63:0/3566078590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e201039c0 0x7f1e201a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:26.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.763+0000 7f1e26e00700 1 -- 172.21.15.63:0/3566078590 >> 172.21.15.63:0/3566078590 conn(0x7f1e20078af0 msgr2=0x7f1e20105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:26.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.763+0000 7f1e26e00700 1 -- 172.21.15.63:0/3566078590 shutdown_connections 2024-05-27T07:29:26.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:26.763+0000 7f1e26e00700 1 -- 172.21.15.63:0/3566078590 wait complete. 2024-05-27T07:29:26.757 INFO:teuthology.orchestra.run.smithi063.stderr:listed 10 entries 2024-05-27T07:29:27.100 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:26 smithi086 ceph-mon[34687]: pgmap v420: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T07:29:27.133 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:26 smithi063 ceph-mon[31613]: pgmap v420: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T07:29:27.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:26 smithi149 ceph-mon[35408]: pgmap v420: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T07:29:27.548 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi063.front.sepia.ceph.com... 2024-05-27T07:29:27.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:27.548 DEBUG:teuthology.orchestra.run.smithi063:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:29:27.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi063.stderr:ceph-fuse[77829]: fuse finished with error 0 and tester_r 0 2024-05-27T07:29:27.643 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T07:29:28.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:27 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3566078590' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:29:28.202 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:27 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3566078590' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:29:28.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:27 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3566078590' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:29:29.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:28 smithi086 ceph-mon[34687]: pgmap v421: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T07:29:29.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:28 smithi149 ceph-mon[35408]: pgmap v421: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T07:29:29.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:28 smithi063 ceph-mon[31613]: pgmap v421: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-27T07:29:31.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:30 smithi086 ceph-mon[34687]: pgmap v422: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-27T07:29:31.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:30 smithi149 ceph-mon[35408]: pgmap v422: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-27T07:29:31.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:30 smithi063 ceph-mon[31613]: pgmap v422: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-27T07:29:33.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:32 smithi086 ceph-mon[34687]: pgmap v423: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T07:29:33.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:32 smithi149 ceph-mon[35408]: pgmap v423: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T07:29:33.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:32 smithi063 ceph-mon[31613]: pgmap v423: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T07:29:33.745 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T07:29:33.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:29:33.746 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-27T07:29:33.819 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:29:34.293 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:35.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:34 smithi086 ceph-mon[34687]: pgmap v424: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T07:29:35.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:34 smithi149 ceph-mon[35408]: pgmap v424: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T07:29:35.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:34 smithi063 ceph-mon[31613]: pgmap v424: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T07:29:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.638+0000 7f22d7200700 1 -- 172.21.15.63:0/3929317518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d010e930 msgr2=0x7f22d0110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.638+0000 7f22d7200700 1 --2- 172.21.15.63:0/3929317518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d010e930 0x7f22d0110e00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f22c800b0a0 tx=0x7f22c800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:29:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.638+0000 7f22d7200700 1 -- 172.21.15.63:0/3929317518 shutdown_connections 2024-05-27T07:29:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.638+0000 7f22d7200700 1 --2- 172.21.15.63:0/3929317518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d010e930 0x7f22d0110e00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:36.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.638+0000 7f22d7200700 1 --2- 172.21.15.63:0/3929317518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0105470 0x7f22d0105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:36.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.638+0000 7f22d7200700 1 --2- 172.21.15.63:0/3929317518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0105e50 0x7f22d010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:36.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.638+0000 7f22d7200700 1 -- 172.21.15.63:0/3929317518 >> 172.21.15.63:0/3929317518 conn(0x7f22d00fef40 msgr2=0x7f22d0101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:36.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.638+0000 7f22d7200700 1 -- 172.21.15.63:0/3929317518 shutdown_connections 2024-05-27T07:29:36.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.638+0000 7f22d7200700 1 -- 172.21.15.63:0/3929317518 wait complete. 2024-05-27T07:29:36.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.639+0000 7f22d7200700 1 Processor -- start 2024-05-27T07:29:36.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.639+0000 7f22d7200700 1 -- start start 2024-05-27T07:29:36.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22d7200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0105470 0x7f22d019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:36.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22d7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0105e50 0x7f22d019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:36.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22d7200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d010e930 0x7f22d01a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:36.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22d7200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0114350 con 0x7f22d010e930 2024-05-27T07:29:36.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22d7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01141d0 con 0x7f22d0105e50 2024-05-27T07:29:36.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22d7200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22d01144d0 con 0x7f22d0105470 2024-05-27T07:29:36.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22d5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d010e930 0x7f22d01a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:36.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22d5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d010e930 0x7f22d01a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34098/0 (socket says 172.21.15.63:34098) 2024-05-27T07:29:36.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22d5200700 1 -- 172.21.15.63:0/4044398325 learned_addr learned my addr 172.21.15.63:0/4044398325 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:36.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22cfe00700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0105470 0x7f22d019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:36.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.640+0000 7f22c7400700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0105e50 0x7f22d019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:36.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.641+0000 7f22d5200700 1 -- 172.21.15.63:0/4044398325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0105470 msgr2=0x7f22d019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:36.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.641+0000 7f22d5200700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0105470 0x7f22d019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:36.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.641+0000 7f22d5200700 1 -- 172.21.15.63:0/4044398325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0105e50 msgr2=0x7f22d019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:36.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.641+0000 7f22d5200700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0105e50 0x7f22d019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:36.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.641+0000 7f22d5200700 1 -- 172.21.15.63:0/4044398325 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d01a1d70 con 0x7f22d010e930 2024-05-27T07:29:36.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.641+0000 7f22d5200700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d010e930 0x7f22d01a16d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f22c800b780 tx=0x7f22c8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:36.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.642+0000 7f22cd600700 1 -- 172.21.15.63:0/4044398325 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8021070 con 0x7f22d010e930 2024-05-27T07:29:36.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.642+0000 7f22cd600700 1 -- 172.21.15.63:0/4044398325 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22c8007b80 con 0x7f22d010e930 2024-05-27T07:29:36.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.642+0000 7f22d7200700 1 -- 172.21.15.63:0/4044398325 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d01a2000 con 0x7f22d010e930 2024-05-27T07:29:36.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.642+0000 7f22cd600700 1 -- 172.21.15.63:0/4044398325 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c80052a0 con 0x7f22d010e930 2024-05-27T07:29:36.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.642+0000 7f22d7200700 1 -- 172.21.15.63:0/4044398325 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d01a24e0 con 0x7f22d010e930 2024-05-27T07:29:36.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.643+0000 7f22d7200700 1 -- 172.21.15.63:0/4044398325 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d00512b0 con 0x7f22d010e930 2024-05-27T07:29:36.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.646+0000 7f22cd600700 1 -- 172.21.15.63:0/4044398325 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f22c8024070 con 0x7f22d010e930 2024-05-27T07:29:36.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.648+0000 7f22cd600700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f22b40b2880 0x7f22b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:36.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.648+0000 7f22cd600700 1 -- 172.21.15.63:0/4044398325 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f22c800f020 con 0x7f22d010e930 2024-05-27T07:29:36.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.648+0000 7f22cfe00700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f22b40b2880 0x7f22b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:36.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.649+0000 7f22cfe00700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f22b40b2880 0x7f22b40b4d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005950 tx=0x7f22c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:36.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.654+0000 7f22cd600700 1 -- 172.21.15.63:0/4044398325 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22c802b020 con 0x7f22d010e930 2024-05-27T07:29:36.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.898+0000 7f22d7200700 1 -- 172.21.15.63:0/4044398325 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22d01a9980 con 0x7f22d010e930 2024-05-27T07:29:36.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.902+0000 7f22cd600700 1 -- 172.21.15.63:0/4044398325 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24552 (secure 0 0 0) 0x7f22c80ab510 con 0x7f22d010e930 2024-05-27T07:29:36.894 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:29:36.894 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":89,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:29:04.850822+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:29:36.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 -- 172.21.15.63:0/4044398325 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f22b40b2880 msgr2=0x7f22b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:36.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f22b40b2880 0x7f22b40b4d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005950 tx=0x7f22c000b410 comp rx=0 tx=0).stop 2024-05-27T07:29:36.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 -- 172.21.15.63:0/4044398325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d010e930 msgr2=0x7f22d01a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:36.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d010e930 0x7f22d01a16d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f22c800b780 tx=0x7f22c8004d00 comp rx=0 tx=0).stop 2024-05-27T07:29:36.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 -- 172.21.15.63:0/4044398325 shutdown_connections 2024-05-27T07:29:36.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f22b40b2880 0x7f22b40b4d90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:36.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22d010e930 0x7f22d01a16d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:36.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d0105470 0x7f22d019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:36.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 --2- 172.21.15.63:0/4044398325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22d0105e50 0x7f22d019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:36.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 -- 172.21.15.63:0/4044398325 >> 172.21.15.63:0/4044398325 conn(0x7f22d00fef40 msgr2=0x7f22d01030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:36.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 -- 172.21.15.63:0/4044398325 shutdown_connections 2024-05-27T07:29:36.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:36.906+0000 7f22d7200700 1 -- 172.21.15.63:0/4044398325 wait complete. 2024-05-27T07:29:37.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:36 smithi086 ceph-mon[34687]: pgmap v425: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T07:29:37.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:36 smithi149 ceph-mon[35408]: pgmap v425: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T07:29:37.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:36 smithi063 ceph-mon[31613]: pgmap v425: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-27T07:29:37.754 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:29:37.955 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:37 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4044398325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:29:38.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:37 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4044398325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:29:38.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:37 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4044398325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:29:38.234 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:39.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:38 smithi086 ceph-mon[34687]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:29:39.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:38 smithi149 ceph-mon[35408]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:29:39.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:38 smithi063 ceph-mon[31613]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:29:40.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.562+0000 7f69c9800700 1 -- 172.21.15.63:0/3002554455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c410b890 msgr2=0x7f69c41131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:40.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.562+0000 7f69c9800700 1 --2- 172.21.15.63:0/3002554455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c410b890 0x7f69c41131f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b0d0 tx=0x7f69bc00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:29:40.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.562+0000 7f69c9800700 1 -- 172.21.15.63:0/3002554455 shutdown_connections 2024-05-27T07:29:40.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.562+0000 7f69c9800700 1 --2- 172.21.15.63:0/3002554455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c410b890 0x7f69c41131f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:40.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.562+0000 7f69c9800700 1 --2- 172.21.15.63:0/3002554455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c4106610 0x7f69c4106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:40.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.562+0000 7f69c9800700 1 --2- 172.21.15.63:0/3002554455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c410af40 0x7f69c410b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:40.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.562+0000 7f69c9800700 1 -- 172.21.15.63:0/3002554455 >> 172.21.15.63:0/3002554455 conn(0x7f69c4102020 msgr2=0x7f69c4104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:40.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.562+0000 7f69c9800700 1 -- 172.21.15.63:0/3002554455 shutdown_connections 2024-05-27T07:29:40.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.562+0000 7f69c9800700 1 -- 172.21.15.63:0/3002554455 wait complete. 2024-05-27T07:29:40.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.564+0000 7f69c9800700 1 Processor -- start 2024-05-27T07:29:40.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.564+0000 7f69c9800700 1 -- start start 2024-05-27T07:29:40.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.564+0000 7f69c9800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c4106610 0x7f69c41105e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.564+0000 7f69c9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c410af40 0x7f69c4110b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.564+0000 7f69c9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c410b890 0x7f69c4111060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.564+0000 7f69c9800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69c41163f0 con 0x7f69c410b890 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.564+0000 7f69c9800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4116270 con 0x7f69c410af40 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.564+0000 7f69c9800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4116570 con 0x7f69c4106610 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.565+0000 7f69c8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c410b890 0x7f69c4111060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.565+0000 7f69c8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c410b890 0x7f69c4111060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34114/0 (socket says 172.21.15.63:34114) 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.565+0000 7f69c8e00700 1 -- 172.21.15.63:0/191491418 learned_addr learned my addr 172.21.15.63:0/191491418 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.565+0000 7f69c3e00700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c4106610 0x7f69c41105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.565+0000 7f69c3400700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c410af40 0x7f69c4110b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.565+0000 7f69c8e00700 1 -- 172.21.15.63:0/191491418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c4106610 msgr2=0x7f69c41105e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:40.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.565+0000 7f69c8e00700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c4106610 0x7f69c41105e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:40.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.565+0000 7f69c8e00700 1 -- 172.21.15.63:0/191491418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c410af40 msgr2=0x7f69c4110b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:40.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.565+0000 7f69c8e00700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c410af40 0x7f69c4110b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:40.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.565+0000 7f69c8e00700 1 -- 172.21.15.63:0/191491418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c41ad290 con 0x7f69c410b890 2024-05-27T07:29:40.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.566+0000 7f69c8e00700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c410b890 0x7f69c4111060 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f69bc007d90 tx=0x7f69bc007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:40.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.566+0000 7f69c0c00700 1 -- 172.21.15.63:0/191491418 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc01f030 con 0x7f69c410b890 2024-05-27T07:29:40.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.566+0000 7f69c0c00700 1 -- 172.21.15.63:0/191491418 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69bc01bb90 con 0x7f69c410b890 2024-05-27T07:29:40.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.566+0000 7f69c0c00700 1 -- 172.21.15.63:0/191491418 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc012c90 con 0x7f69c410b890 2024-05-27T07:29:40.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.566+0000 7f69c9800700 1 -- 172.21.15.63:0/191491418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c41ad520 con 0x7f69c410b890 2024-05-27T07:29:40.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.566+0000 7f69c9800700 1 -- 172.21.15.63:0/191491418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c41ad980 con 0x7f69c410b890 2024-05-27T07:29:40.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.568+0000 7f69c9800700 1 -- 172.21.15.63:0/191491418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c4079370 con 0x7f69c410b890 2024-05-27T07:29:40.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.571+0000 7f69c0c00700 1 -- 172.21.15.63:0/191491418 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f69bc022030 con 0x7f69c410b890 2024-05-27T07:29:40.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.572+0000 7f69c0c00700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69a80b2880 0x7f69a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:40.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.572+0000 7f69c0c00700 1 -- 172.21.15.63:0/191491418 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f69bc00f020 con 0x7f69c410b890 2024-05-27T07:29:40.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.573+0000 7f69c3e00700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69a80b2880 0x7f69a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:40.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.573+0000 7f69c3e00700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69a80b2880 0x7f69a80b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006fd0 tx=0x7f69b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:40.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.575+0000 7f69c0c00700 1 -- 172.21.15.63:0/191491418 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69bc029020 con 0x7f69c410b890 2024-05-27T07:29:40.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.812+0000 7f69c9800700 1 -- 172.21.15.63:0/191491418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69c4107ac0 con 0x7f69c410b890 2024-05-27T07:29:40.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.815+0000 7f69c0c00700 1 -- 172.21.15.63:0/191491418 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24552 (secure 0 0 0) 0x7f69c4107ac0 con 0x7f69c410b890 2024-05-27T07:29:40.807 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:29:40.808 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":89,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:29:04.850822+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:29:40.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.819+0000 7f69c9800700 1 -- 172.21.15.63:0/191491418 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69a80b2880 msgr2=0x7f69a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:40.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.819+0000 7f69c9800700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69a80b2880 0x7f69a80b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69b4006fd0 tx=0x7f69b4008040 comp rx=0 tx=0).stop 2024-05-27T07:29:40.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.819+0000 7f69c9800700 1 -- 172.21.15.63:0/191491418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c410b890 msgr2=0x7f69c4111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:40.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.819+0000 7f69c9800700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c410b890 0x7f69c4111060 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f69bc007d90 tx=0x7f69bc007e70 comp rx=0 tx=0).stop 2024-05-27T07:29:40.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.819+0000 7f69c9800700 1 -- 172.21.15.63:0/191491418 shutdown_connections 2024-05-27T07:29:40.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.819+0000 7f69c9800700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69a80b2880 0x7f69a80b4d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:40.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.819+0000 7f69c9800700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69c410b890 0x7f69c4111060 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:40.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.819+0000 7f69c9800700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69c4106610 0x7f69c41105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:40.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.819+0000 7f69c9800700 1 --2- 172.21.15.63:0/191491418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c410af40 0x7f69c4110b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:40.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.819+0000 7f69c9800700 1 -- 172.21.15.63:0/191491418 >> 172.21.15.63:0/191491418 conn(0x7f69c4102020 msgr2=0x7f69c410a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:40.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.820+0000 7f69c9800700 1 -- 172.21.15.63:0/191491418 shutdown_connections 2024-05-27T07:29:40.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:40.820+0000 7f69c9800700 1 -- 172.21.15.63:0/191491418 wait complete. 2024-05-27T07:29:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:40 smithi086 ceph-mon[34687]: pgmap v427: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:29:41.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:40 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/191491418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:29:41.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:40 smithi149 ceph-mon[35408]: pgmap v427: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:29:41.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:40 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/191491418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:29:41.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:40 smithi063 ceph-mon[31613]: pgmap v427: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:29:41.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:40 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/191491418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:29:41.652 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-27T07:29:42.135 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:43.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:42 smithi086 ceph-mon[34687]: pgmap v428: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:43.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:42 smithi149 ceph-mon[35408]: pgmap v428: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:43.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:42 smithi063 ceph-mon[31613]: pgmap v428: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:44.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.563+0000 7f53fe000700 1 -- 172.21.15.63:0/256305007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53f81045e0 msgr2=0x7f53f81049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:44.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.563+0000 7f53fe000700 1 --2- 172.21.15.63:0/256305007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53f81045e0 0x7f53f81049f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53e800b0d0 tx=0x7f53e800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:29:44.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.564+0000 7f53fe000700 1 -- 172.21.15.63:0/256305007 shutdown_connections 2024-05-27T07:29:44.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.564+0000 7f53fe000700 1 --2- 172.21.15.63:0/256305007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f8109860 0x7f53f81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:44.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.564+0000 7f53fe000700 1 --2- 172.21.15.63:0/256305007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53f81045e0 0x7f53f81049f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:44.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.564+0000 7f53fe000700 1 --2- 172.21.15.63:0/256305007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f8108f10 0x7f53f8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:44.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.564+0000 7f53fe000700 1 -- 172.21.15.63:0/256305007 >> 172.21.15.63:0/256305007 conn(0x7f53f8100030 msgr2=0x7f53f8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:44.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.564+0000 7f53fe000700 1 -- 172.21.15.63:0/256305007 shutdown_connections 2024-05-27T07:29:44.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.564+0000 7f53fe000700 1 -- 172.21.15.63:0/256305007 wait complete. 2024-05-27T07:29:44.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.565+0000 7f53fe000700 1 Processor -- start 2024-05-27T07:29:44.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.566+0000 7f53fe000700 1 -- start start 2024-05-27T07:29:44.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.566+0000 7f53fe000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f81045e0 0x7f53f819d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:44.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.566+0000 7f53fe000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f8108f10 0x7f53f819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:44.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.566+0000 7f53fe000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53f8109860 0x7f53f81a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:44.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.566+0000 7f53fe000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8114390 con 0x7f53f8109860 2024-05-27T07:29:44.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.566+0000 7f53fe000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8114210 con 0x7f53f81045e0 2024-05-27T07:29:44.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.566+0000 7f53fe000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8114510 con 0x7f53f8108f10 2024-05-27T07:29:44.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53f7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53f8109860 0x7f53f81a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:44.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53f7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53f8109860 0x7f53f81a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34124/0 (socket says 172.21.15.63:34124) 2024-05-27T07:29:44.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53f7e00700 1 -- 172.21.15.63:0/1371876826 learned_addr learned my addr 172.21.15.63:0/1371876826 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:44.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53eea00700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f8108f10 0x7f53f819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:44.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53f7400700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f81045e0 0x7f53f819d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:44.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53f7e00700 1 -- 172.21.15.63:0/1371876826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f8108f10 msgr2=0x7f53f819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:44.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53f7e00700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f8108f10 0x7f53f819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:44.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53f7e00700 1 -- 172.21.15.63:0/1371876826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f81045e0 msgr2=0x7f53f819d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:44.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53f7e00700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f81045e0 0x7f53f819d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:44.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53f7e00700 1 -- 172.21.15.63:0/1371876826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f81a1ee0 con 0x7f53f8109860 2024-05-27T07:29:44.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.567+0000 7f53f7400700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f81045e0 0x7f53f819d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:29:44.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.568+0000 7f53f7e00700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53f8109860 0x7f53f81a1840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f53f000d980 tx=0x7f53f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:44.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.568+0000 7f53f4c00700 1 -- 172.21.15.63:0/1371876826 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0017070 con 0x7f53f8109860 2024-05-27T07:29:44.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.568+0000 7f53f4c00700 1 -- 172.21.15.63:0/1371876826 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53f0005020 con 0x7f53f8109860 2024-05-27T07:29:44.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.568+0000 7f53fe000700 1 -- 172.21.15.63:0/1371876826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f81a21d0 con 0x7f53f8109860 2024-05-27T07:29:44.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.568+0000 7f53f4c00700 1 -- 172.21.15.63:0/1371876826 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0009990 con 0x7f53f8109860 2024-05-27T07:29:44.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.568+0000 7f53fe000700 1 -- 172.21.15.63:0/1371876826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f81a9af0 con 0x7f53f8109860 2024-05-27T07:29:44.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.570+0000 7f53fe000700 1 -- 172.21.15.63:0/1371876826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f80512b0 con 0x7f53f8109860 2024-05-27T07:29:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.572+0000 7f53f4c00700 1 -- 172.21.15.63:0/1371876826 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f53f0004a30 con 0x7f53f8109860 2024-05-27T07:29:44.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.574+0000 7f53f4c00700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f53dc0b2900 0x7f53dc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:44.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.574+0000 7f53f4c00700 1 -- 172.21.15.63:0/1371876826 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f53f00e2500 con 0x7f53f8109860 2024-05-27T07:29:44.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.574+0000 7f53f7400700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f53dc0b2900 0x7f53dc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:44.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.575+0000 7f53f7400700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f53dc0b2900 0x7f53dc0b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009fd0 tx=0x7f53e800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:44.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.576+0000 7f53f4c00700 1 -- 172.21.15.63:0/1371876826 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53f0020030 con 0x7f53f8109860 2024-05-27T07:29:44.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.865+0000 7f53fe000700 1 -- 172.21.15.63:0/1371876826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f53f81a9cc0 con 0x7f53f8109860 2024-05-27T07:29:44.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.902+0000 7f53f4c00700 1 -- 172.21.15.63:0/1371876826 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f53f00a9ee0 con 0x7f53f8109860 2024-05-27T07:29:44.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 -- 172.21.15.63:0/1371876826 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f53dc0b2900 msgr2=0x7f53dc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:44.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f53dc0b2900 0x7f53dc0b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009fd0 tx=0x7f53e800b040 comp rx=0 tx=0).stop 2024-05-27T07:29:44.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 -- 172.21.15.63:0/1371876826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53f8109860 msgr2=0x7f53f81a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:44.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53f8109860 0x7f53f81a1840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f53f000d980 tx=0x7f53f000dc90 comp rx=0 tx=0).stop 2024-05-27T07:29:44.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 -- 172.21.15.63:0/1371876826 shutdown_connections 2024-05-27T07:29:44.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f53dc0b2900 0x7f53dc0b4e10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:44.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53f8109860 0x7f53f81a1840 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:44.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53f81045e0 0x7f53f819d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:44.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 --2- 172.21.15.63:0/1371876826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f8108f10 0x7f53f819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:44.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 -- 172.21.15.63:0/1371876826 >> 172.21.15.63:0/1371876826 conn(0x7f53f8100030 msgr2=0x7f53f810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:44.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 -- 172.21.15.63:0/1371876826 shutdown_connections 2024-05-27T07:29:44.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:44.906+0000 7f53fe000700 1 -- 172.21.15.63:0/1371876826 wait complete. 2024-05-27T07:29:45.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:44 smithi086 ceph-mon[34687]: pgmap v429: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:45.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:44 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1371876826' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T07:29:45.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:44 smithi149 ceph-mon[35408]: pgmap v429: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:45.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:44 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1371876826' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T07:29:45.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:44 smithi063 ceph-mon[31613]: pgmap v429: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:45.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:44 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1371876826' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T07:29:45.705 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-27T07:29:45.705 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-27T07:29:46.184 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:46.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:45 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1371876826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T07:29:46.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:45 smithi086 ceph-mon[34687]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:29:46.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:45 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1371876826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T07:29:46.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:45 smithi149 ceph-mon[35408]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:29:46.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:45 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1371876826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T07:29:46.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:45 smithi063 ceph-mon[31613]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:29:47.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:46 smithi086 ceph-mon[34687]: pgmap v430: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:47.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:46 smithi149 ceph-mon[35408]: pgmap v430: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:47.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:46 smithi063 ceph-mon[31613]: pgmap v430: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.547+0000 7ff5d7c00700 1 -- 172.21.15.63:0/4191582180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0110bf0 msgr2=0x7ff5d0113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.547+0000 7ff5d7c00700 1 --2- 172.21.15.63:0/4191582180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0110bf0 0x7ff5d0113010 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800b0a0 tx=0x7ff5c800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:29:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.547+0000 7ff5d7c00700 1 -- 172.21.15.63:0/4191582180 shutdown_connections 2024-05-27T07:29:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.547+0000 7ff5d7c00700 1 --2- 172.21.15.63:0/4191582180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0110bf0 0x7ff5d0113010 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:48.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.547+0000 7ff5d7c00700 1 --2- 172.21.15.63:0/4191582180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0075af0 0x7ff5d0075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.547+0000 7ff5d7c00700 1 --2- 172.21.15.63:0/4191582180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5d0076440 0x7ff5d01106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.547+0000 7ff5d7c00700 1 -- 172.21.15.63:0/4191582180 >> 172.21.15.63:0/4191582180 conn(0x7ff5d0100030 msgr2=0x7ff5d0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.547+0000 7ff5d7c00700 1 -- 172.21.15.63:0/4191582180 shutdown_connections 2024-05-27T07:29:48.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.547+0000 7ff5d7c00700 1 -- 172.21.15.63:0/4191582180 wait complete. 2024-05-27T07:29:48.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.548+0000 7ff5d7c00700 1 Processor -- start 2024-05-27T07:29:48.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.549+0000 7ff5d7c00700 1 -- start start 2024-05-27T07:29:48.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.549+0000 7ff5d7c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0075af0 0x7ff5d019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:48.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.549+0000 7ff5d7c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0076440 0x7ff5d019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:48.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.549+0000 7ff5d7c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5d0110bf0 0x7ff5d01a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:48.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.549+0000 7ff5d7c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0116560 con 0x7ff5d0076440 2024-05-27T07:29:48.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.549+0000 7ff5d7c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01163e0 con 0x7ff5d0075af0 2024-05-27T07:29:48.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.549+0000 7ff5d7c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01166e0 con 0x7ff5d0110bf0 2024-05-27T07:29:48.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.549+0000 7ff5cfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0076440 0x7ff5d019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:48.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.550+0000 7ff5cfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0076440 0x7ff5d019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34146/0 (socket says 172.21.15.63:34146) 2024-05-27T07:29:48.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.550+0000 7ff5cfe00700 1 -- 172.21.15.63:0/2918524324 learned_addr learned my addr 172.21.15.63:0/2918524324 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:48.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.550+0000 7ff5d5200700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0075af0 0x7ff5d019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:48.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.550+0000 7ff5d5c00700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5d0110bf0 0x7ff5d01a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.550+0000 7ff5cfe00700 1 -- 172.21.15.63:0/2918524324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5d0110bf0 msgr2=0x7ff5d01a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.550+0000 7ff5cfe00700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5d0110bf0 0x7ff5d01a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.550+0000 7ff5cfe00700 1 -- 172.21.15.63:0/2918524324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0075af0 msgr2=0x7ff5d019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.550+0000 7ff5cfe00700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0075af0 0x7ff5d019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.550+0000 7ff5cfe00700 1 -- 172.21.15.63:0/2918524324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d01a1e00 con 0x7ff5d0076440 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.551+0000 7ff5cfe00700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0076440 0x7ff5d019d4d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00d980 tx=0x7ff5bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.551+0000 7ff5cd600700 1 -- 172.21.15.63:0/2918524324 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc017070 con 0x7ff5d0076440 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.551+0000 7ff5d7c00700 1 -- 172.21.15.63:0/2918524324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d01a20f0 con 0x7ff5d0076440 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.551+0000 7ff5cd600700 1 -- 172.21.15.63:0/2918524324 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5bc009dc0 con 0x7ff5d0076440 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.551+0000 7ff5cd600700 1 -- 172.21.15.63:0/2918524324 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc00f3c0 con 0x7ff5d0076440 2024-05-27T07:29:48.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.551+0000 7ff5d7c00700 1 -- 172.21.15.63:0/2918524324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d01a9a10 con 0x7ff5d0076440 2024-05-27T07:29:48.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.552+0000 7ff5d7c00700 1 -- 172.21.15.63:0/2918524324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d00512b0 con 0x7ff5d0076440 2024-05-27T07:29:48.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.555+0000 7ff5cd600700 1 -- 172.21.15.63:0/2918524324 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff5bc009930 con 0x7ff5d0076440 2024-05-27T07:29:48.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.556+0000 7ff5cd600700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5b80b2880 0x7ff5b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:48.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.557+0000 7ff5cd600700 1 -- 172.21.15.63:0/2918524324 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff5bc0e2060 con 0x7ff5d0076440 2024-05-27T07:29:48.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.557+0000 7ff5d5200700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5b80b2880 0x7ff5b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:48.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.558+0000 7ff5d5200700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5b80b2880 0x7ff5b80b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005950 tx=0x7ff5c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:48.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.561+0000 7ff5cd600700 1 -- 172.21.15.63:0/2918524324 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff5bc0a99c0 con 0x7ff5d0076440 2024-05-27T07:29:48.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.851+0000 7ff5d7c00700 1 -- 172.21.15.63:0/2918524324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ff5d002e5e0 con 0x7ff5d0076440 2024-05-27T07:29:48.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.938+0000 7ff5cd600700 1 -- 172.21.15.63:0/2918524324 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7ff5bc0a92c0 con 0x7ff5d0076440 2024-05-27T07:29:48.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.942+0000 7ff5d7c00700 1 -- 172.21.15.63:0/2918524324 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5b80b2880 msgr2=0x7ff5b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:48.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.942+0000 7ff5d7c00700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5b80b2880 0x7ff5b80b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005950 tx=0x7ff5c400b410 comp rx=0 tx=0).stop 2024-05-27T07:29:48.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.942+0000 7ff5d7c00700 1 -- 172.21.15.63:0/2918524324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0076440 msgr2=0x7ff5d019d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:48.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.942+0000 7ff5d7c00700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0076440 0x7ff5d019d4d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00d980 tx=0x7ff5bc00dc90 comp rx=0 tx=0).stop 2024-05-27T07:29:48.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.943+0000 7ff5d7c00700 1 -- 172.21.15.63:0/2918524324 shutdown_connections 2024-05-27T07:29:48.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.943+0000 7ff5d7c00700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5d0110bf0 0x7ff5d01a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:48.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.943+0000 7ff5d7c00700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0075af0 0x7ff5d019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:48.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.943+0000 7ff5d7c00700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff5b80b2880 0x7ff5b80b4d90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:48.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.943+0000 7ff5d7c00700 1 --2- 172.21.15.63:0/2918524324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5d0076440 0x7ff5d019d4d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:48.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.943+0000 7ff5d7c00700 1 -- 172.21.15.63:0/2918524324 >> 172.21.15.63:0/2918524324 conn(0x7ff5d0100030 msgr2=0x7ff5d01006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:48.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.943+0000 7ff5d7c00700 1 -- 172.21.15.63:0/2918524324 shutdown_connections 2024-05-27T07:29:48.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:48.943+0000 7ff5d7c00700 1 -- 172.21.15.63:0/2918524324 wait complete. 2024-05-27T07:29:49.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:48 smithi086 ceph-mon[34687]: pgmap v431: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:49.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:48 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2918524324' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-27T07:29:49.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:48 smithi149 ceph-mon[35408]: pgmap v431: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:49.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:48 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2918524324' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-27T07:29:49.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:48 smithi063 ceph-mon[31613]: pgmap v431: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:29:49.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:48 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2918524324' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-27T07:29:49.805 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-27T07:29:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:49 smithi086 ceph-mon[34687]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T07:29:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:49 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2918524324' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-27T07:29:50.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:49 smithi086 ceph-mon[34687]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T07:29:50.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:49 smithi086 ceph-mon[34687]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:29:50.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:49 smithi086 ceph-mon[34687]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T07:29:50.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:49 smithi086 ceph-mon[34687]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T07:29:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:49 smithi149 ceph-mon[35408]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T07:29:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:49 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2918524324' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-27T07:29:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:49 smithi149 ceph-mon[35408]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T07:29:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:49 smithi149 ceph-mon[35408]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:29:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:49 smithi149 ceph-mon[35408]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T07:29:50.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:49 smithi149 ceph-mon[35408]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T07:29:50.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:49 smithi063 ceph-mon[31613]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T07:29:50.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:49 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2918524324' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-27T07:29:50.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:49 smithi063 ceph-mon[31613]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T07:29:50.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:49 smithi063 ceph-mon[31613]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:29:50.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:49 smithi063 ceph-mon[31613]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T07:29:50.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:49 smithi063 ceph-mon[31613]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T07:29:50.289 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:51.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:50 smithi063 ceph-mon[31613]: pgmap v432: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-27T07:29:51.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:50 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] up:active 2024-05-27T07:29:51.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:50 smithi063 ceph-mon[31613]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T07:29:51.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:50 smithi063 ceph-mon[31613]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T07:29:51.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:50 smithi063 ceph-mon[31613]: Cluster is now healthy 2024-05-27T07:29:51.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:50 smithi063 ceph-mon[31613]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T07:29:51.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:50 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T07:29:51.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:50 smithi063 ceph-mon[31613]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T07:29:51.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:50 smithi086 ceph-mon[34687]: pgmap v432: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-27T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:50 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] up:active 2024-05-27T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:50 smithi086 ceph-mon[34687]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:50 smithi086 ceph-mon[34687]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:50 smithi086 ceph-mon[34687]: Cluster is now healthy 2024-05-27T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:50 smithi086 ceph-mon[34687]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:50 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:50 smithi086 ceph-mon[34687]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T07:29:51.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:50 smithi149 ceph-mon[35408]: pgmap v432: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-27T07:29:51.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:50 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] up:active 2024-05-27T07:29:51.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:50 smithi149 ceph-mon[35408]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T07:29:51.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:50 smithi149 ceph-mon[35408]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T07:29:51.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:50 smithi149 ceph-mon[35408]: Cluster is now healthy 2024-05-27T07:29:51.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:50 smithi149 ceph-mon[35408]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T07:29:51.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:50 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T07:29:51.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:50 smithi149 ceph-mon[35408]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T07:29:52.199 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:29:51 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b[65103]: 2024-05-27T07:29:51.887+0000 7f8aa2600700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-27T07:29:52.200 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 07:29:51 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b[65103]: 2024-05-27T07:29:51.887+0000 7f8aa2600700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-27T07:29:52.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:51 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] up:active 2024-05-27T07:29:52.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:51 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T07:29:52.362 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:51 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] up:active 2024-05-27T07:29:52.362 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:51 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T07:29:52.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:51 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] up:active 2024-05-27T07:29:52.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:51 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T07:29:52.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.623+0000 7f5ae9400700 1 -- 172.21.15.63:0/1756884000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae41054d0 msgr2=0x7f5ae41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:52.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.623+0000 7f5ae9400700 1 --2- 172.21.15.63:0/1756884000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae41054d0 0x7f5ae41058e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009990 tx=0x7f5ad4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.623+0000 7f5ae9400700 1 -- 172.21.15.63:0/1756884000 shutdown_connections 2024-05-27T07:29:52.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.623+0000 7f5ae9400700 1 --2- 172.21.15.63:0/1756884000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e900 0x7f5ae4110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.623+0000 7f5ae9400700 1 --2- 172.21.15.63:0/1756884000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae41054d0 0x7f5ae41058e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.623+0000 7f5ae9400700 1 --2- 172.21.15.63:0/1756884000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae4105e20 0x7f5ae410e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.623+0000 7f5ae9400700 1 -- 172.21.15.63:0/1756884000 >> 172.21.15.63:0/1756884000 conn(0x7f5ae40fedb0 msgr2=0x7f5ae41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:52.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.623+0000 7f5ae9400700 1 -- 172.21.15.63:0/1756884000 shutdown_connections 2024-05-27T07:29:52.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.623+0000 7f5ae9400700 1 -- 172.21.15.63:0/1756884000 wait complete. 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.624+0000 7f5ae9400700 1 Processor -- start 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.625+0000 7f5ae9400700 1 -- start start 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.625+0000 7f5ae9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae41054d0 0x7f5ae41031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.625+0000 7f5ae9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae4105e20 0x7f5ae4103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.625+0000 7f5ae9400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e900 0x7f5ae4103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.625+0000 7f5ae9400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41143a0 con 0x7f5ae41054d0 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.625+0000 7f5ae9400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4114220 con 0x7f5ae4105e20 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.625+0000 7f5ae9400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4114520 con 0x7f5ae410e900 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.626+0000 7f5ae2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae41054d0 0x7f5ae41031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.626+0000 7f5ae2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae41054d0 0x7f5ae41031c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39784/0 (socket says 172.21.15.63:39784) 2024-05-27T07:29:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.626+0000 7f5ae2800700 1 -- 172.21.15.63:0/3366481727 learned_addr learned my addr 172.21.15.63:0/3366481727 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.626+0000 7f5ae1e00700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae4105e20 0x7f5ae4103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.626+0000 7f5ae3200700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e900 0x7f5ae4103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.626+0000 7f5ae2800700 1 -- 172.21.15.63:0/3366481727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e900 msgr2=0x7f5ae4103c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.626+0000 7f5ae2800700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e900 0x7f5ae4103c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.626+0000 7f5ae2800700 1 -- 172.21.15.63:0/3366481727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae4105e20 msgr2=0x7f5ae4103700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.626+0000 7f5ae2800700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae4105e20 0x7f5ae4103700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.626+0000 7f5ae2800700 1 -- 172.21.15.63:0/3366481727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae41a9190 con 0x7f5ae41054d0 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.627+0000 7f5ae2800700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae41054d0 0x7f5ae41031c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4000c00 tx=0x7f5ad40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.627+0000 7f5adb400700 1 -- 172.21.15.63:0/3366481727 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4027070 con 0x7f5ae41054d0 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.627+0000 7f5adb400700 1 -- 172.21.15.63:0/3366481727 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad400fc00 con 0x7f5ae41054d0 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.627+0000 7f5ae9400700 1 -- 172.21.15.63:0/3366481727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae41a9420 con 0x7f5ae41054d0 2024-05-27T07:29:52.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.627+0000 7f5adb400700 1 -- 172.21.15.63:0/3366481727 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4022ca0 con 0x7f5ae41054d0 2024-05-27T07:29:52.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.627+0000 7f5ae9400700 1 -- 172.21.15.63:0/3366481727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae41a9950 con 0x7f5ae41054d0 2024-05-27T07:29:52.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.628+0000 7f5ae9400700 1 -- 172.21.15.63:0/3366481727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae40512b0 con 0x7f5ae41054d0 2024-05-27T07:29:52.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.631+0000 7f5adb400700 1 -- 172.21.15.63:0/3366481727 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5ad402a070 con 0x7f5ae41054d0 2024-05-27T07:29:52.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.632+0000 7f5adb400700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f5ac80b2900 0x7f5ac80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:52.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.633+0000 7f5adb400700 1 -- 172.21.15.63:0/3366481727 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f5ad40e37c0 con 0x7f5ae41054d0 2024-05-27T07:29:52.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.633+0000 7f5ae1e00700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f5ac80b2900 0x7f5ac80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:52.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.634+0000 7f5ae1e00700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f5ac80b2900 0x7f5ac80b4e10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4102430 tx=0x7f5ad0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:52.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.636+0000 7f5adb400700 1 -- 172.21.15.63:0/3366481727 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ad405fdf0 con 0x7f5ae41054d0 2024-05-27T07:29:52.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:52.931+0000 7f5ae9400700 1 -- 172.21.15.63:0/3366481727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f5ae402e5e0 con 0x7f5ae41054d0 2024-05-27T07:29:52.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.001+0000 7f5adb400700 1 -- 172.21.15.63:0/3366481727 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f5ad4013070 con 0x7f5ae41054d0 2024-05-27T07:29:52.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.005+0000 7f5ae9400700 1 -- 172.21.15.63:0/3366481727 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f5ac80b2900 msgr2=0x7f5ac80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:52.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.005+0000 7f5ae9400700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f5ac80b2900 0x7f5ac80b4e10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4102430 tx=0x7f5ad0009190 comp rx=0 tx=0).stop 2024-05-27T07:29:52.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.005+0000 7f5ae9400700 1 -- 172.21.15.63:0/3366481727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae41054d0 msgr2=0x7f5ae41031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:52.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.005+0000 7f5ae9400700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae41054d0 0x7f5ae41031c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4000c00 tx=0x7f5ad40057c0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.005+0000 7f5ae9400700 1 -- 172.21.15.63:0/3366481727 shutdown_connections 2024-05-27T07:29:52.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.005+0000 7f5ae9400700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e900 0x7f5ae4103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.005+0000 7f5ae9400700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f5ac80b2900 0x7f5ac80b4e10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.005+0000 7f5ae9400700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae41054d0 0x7f5ae41031c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.005+0000 7f5ae9400700 1 --2- 172.21.15.63:0/3366481727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae4105e20 0x7f5ae4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.005+0000 7f5ae9400700 1 -- 172.21.15.63:0/3366481727 >> 172.21.15.63:0/3366481727 conn(0x7f5ae40fedb0 msgr2=0x7f5ae40ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.006+0000 7f5ae9400700 1 -- 172.21.15.63:0/3366481727 shutdown_connections 2024-05-27T07:29:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:53.006+0000 7f5ae9400700 1 -- 172.21.15.63:0/3366481727 wait complete. 2024-05-27T07:29:53.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:53 smithi063 ceph-mon[31613]: pgmap v433: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-27T07:29:53.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:53 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3366481727' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T07:29:53.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:53 smithi086 ceph-mon[34687]: pgmap v433: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-27T07:29:53.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:53 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3366481727' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T07:29:53.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:53 smithi149 ceph-mon[35408]: pgmap v433: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-27T07:29:53.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:53 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3366481727' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T07:29:53.827 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-27T07:29:54.322 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:54.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:54 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3366481727' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T07:29:54.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:54 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T07:29:54.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:54 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:29:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:54 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3366481727' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T07:29:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:54 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T07:29:54.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:54 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:29:54.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:54 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3366481727' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T07:29:54.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:54 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T07:29:54.457 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:54 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:29:55.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:55 smithi063 ceph-mon[31613]: pgmap v434: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-27T07:29:55.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:55 smithi086 ceph-mon[34687]: pgmap v434: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-27T07:29:55.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:55 smithi149 ceph-mon[35408]: pgmap v434: 129 pgs: 129 active+clean; 583 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-05-27T07:29:56.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.624+0000 7fb7bde00700 1 -- 172.21.15.63:0/1321562495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b81043f0 msgr2=0x7fb7b8104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:56.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.624+0000 7fb7bde00700 1 --2- 172.21.15.63:0/1321562495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b81043f0 0x7fb7b8104800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac0099c0 tx=0x7fb7ac009cd0 comp rx=0 tx=0).stop 2024-05-27T07:29:56.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.624+0000 7fb7bde00700 1 -- 172.21.15.63:0/1321562495 shutdown_connections 2024-05-27T07:29:56.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.624+0000 7fb7bde00700 1 --2- 172.21.15.63:0/1321562495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7b8109670 0x7fb7b8110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:56.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.624+0000 7fb7bde00700 1 --2- 172.21.15.63:0/1321562495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b81043f0 0x7fb7b8104800 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:56.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.624+0000 7fb7bde00700 1 --2- 172.21.15.63:0/1321562495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b8108d20 0x7fb7b8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:56.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.624+0000 7fb7bde00700 1 -- 172.21.15.63:0/1321562495 >> 172.21.15.63:0/1321562495 conn(0x7fb7b8079d00 msgr2=0x7fb7b807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:56.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.625+0000 7fb7bde00700 1 -- 172.21.15.63:0/1321562495 shutdown_connections 2024-05-27T07:29:56.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.625+0000 7fb7bde00700 1 -- 172.21.15.63:0/1321562495 wait complete. 2024-05-27T07:29:56.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.626+0000 7fb7bde00700 1 Processor -- start 2024-05-27T07:29:56.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.626+0000 7fb7bde00700 1 -- start start 2024-05-27T07:29:56.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b81043f0 0x7fb7b81a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:56.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bde00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b8108d20 0x7fb7b81a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:56.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7b8109670 0x7fb7b819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:56.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bde00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b81141a0 con 0x7fb7b81043f0 2024-05-27T07:29:56.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bde00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b8114020 con 0x7fb7b8109670 2024-05-27T07:29:56.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bde00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b8114320 con 0x7fb7b8108d20 2024-05-27T07:29:56.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b81043f0 0x7fb7b81a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:56.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b81043f0 0x7fb7b81a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39796/0 (socket says 172.21.15.63:39796) 2024-05-27T07:29:56.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bca00700 1 -- 172.21.15.63:0/3272096019 learned_addr learned my addr 172.21.15.63:0/3272096019 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:29:56.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7b7e00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b8108d20 0x7fb7b81a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:56.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bd400700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7b8109670 0x7fb7b819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:56.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bca00700 1 -- 172.21.15.63:0/3272096019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b8108d20 msgr2=0x7fb7b81a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:56.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bca00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b8108d20 0x7fb7b81a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:56.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bca00700 1 -- 172.21.15.63:0/3272096019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7b8109670 msgr2=0x7fb7b819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:56.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.627+0000 7fb7bca00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7b8109670 0x7fb7b819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:56.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.628+0000 7fb7bca00700 1 -- 172.21.15.63:0/3272096019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b819b030 con 0x7fb7b81043f0 2024-05-27T07:29:56.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.628+0000 7fb7bd400700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7b8109670 0x7fb7b819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:29:56.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.628+0000 7fb7bca00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b81043f0 0x7fb7b81a0760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac005910 tx=0x7fb7ac004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:56.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.628+0000 7fb7b5600700 1 -- 172.21.15.63:0/3272096019 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac026070 con 0x7fb7b81043f0 2024-05-27T07:29:56.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.628+0000 7fb7b5600700 1 -- 172.21.15.63:0/3272096019 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7ac003c50 con 0x7fb7b81043f0 2024-05-27T07:29:56.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.628+0000 7fb7bde00700 1 -- 172.21.15.63:0/3272096019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b819b2c0 con 0x7fb7b81043f0 2024-05-27T07:29:56.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.628+0000 7fb7b5600700 1 -- 172.21.15.63:0/3272096019 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00fb10 con 0x7fb7b81043f0 2024-05-27T07:29:56.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.629+0000 7fb7bde00700 1 -- 172.21.15.63:0/3272096019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b819b7a0 con 0x7fb7b81043f0 2024-05-27T07:29:56.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.630+0000 7fb7bde00700 1 -- 172.21.15.63:0/3272096019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b80512b0 con 0x7fb7b81043f0 2024-05-27T07:29:56.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.632+0000 7fb7b5600700 1 -- 172.21.15.63:0/3272096019 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb7ac029070 con 0x7fb7b81043f0 2024-05-27T07:29:56.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.634+0000 7fb7b5600700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb7a00b2880 0x7fb7a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:56.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.634+0000 7fb7b5600700 1 -- 172.21.15.63:0/3272096019 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb7ac0e3530 con 0x7fb7b81043f0 2024-05-27T07:29:56.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.634+0000 7fb7b7e00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb7a00b2880 0x7fb7a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:56.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.635+0000 7fb7b7e00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb7a00b2880 0x7fb7a00b4d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8100d40 tx=0x7fb7a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:56.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.637+0000 7fb7b5600700 1 -- 172.21.15.63:0/3272096019 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7ac013070 con 0x7fb7b81043f0 2024-05-27T07:29:56.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:56.942+0000 7fb7bde00700 1 -- 172.21.15.63:0/3272096019 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb7b802e5e0 con 0x7fb7b81043f0 2024-05-27T07:29:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.049+0000 7fb7b5600700 1 -- 172.21.15.63:0/3272096019 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fb7ac0aaf20 con 0x7fb7b81043f0 2024-05-27T07:29:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 -- 172.21.15.63:0/3272096019 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb7a00b2880 msgr2=0x7fb7a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb7a00b2880 0x7fb7a00b4d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8100d40 tx=0x7fb7a4009260 comp rx=0 tx=0).stop 2024-05-27T07:29:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 -- 172.21.15.63:0/3272096019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b81043f0 msgr2=0x7fb7b81a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b81043f0 0x7fb7b81a0760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac005910 tx=0x7fb7ac004320 comp rx=0 tx=0).stop 2024-05-27T07:29:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 -- 172.21.15.63:0/3272096019 shutdown_connections 2024-05-27T07:29:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7b8109670 0x7fb7b819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb7a00b2880 0x7fb7a00b4d90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b81043f0 0x7fb7b81a0760 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 --2- 172.21.15.63:0/3272096019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7b8108d20 0x7fb7b81a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 -- 172.21.15.63:0/3272096019 >> 172.21.15.63:0/3272096019 conn(0x7fb7b8079d00 msgr2=0x7fb7b81017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.054+0000 7fb7bde00700 1 -- 172.21.15.63:0/3272096019 shutdown_connections 2024-05-27T07:29:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:29:57.055+0000 7fb7bde00700 1 -- 172.21.15.63:0/3272096019 wait complete. 2024-05-27T07:29:57.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:57 smithi063 ceph-mon[31613]: pgmap v435: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-27T07:29:57.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:57 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3272096019' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T07:29:57.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:57 smithi086 ceph-mon[34687]: pgmap v435: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-27T07:29:57.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:57 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3272096019' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T07:29:57.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:57 smithi149 ceph-mon[35408]: pgmap v435: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-27T07:29:57.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:57 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3272096019' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T07:29:57.923 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-27T07:29:57.923 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-27T07:29:58.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:58 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3272096019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T07:29:58.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:58 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:29:58.417 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:29:58.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:58 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3272096019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T07:29:58.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:58 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:29:58.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:58 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3272096019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T07:29:58.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:58 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:29:59.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:29:59 smithi063 ceph-mon[31613]: pgmap v436: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T07:29:59.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:29:59 smithi086 ceph-mon[34687]: pgmap v436: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T07:29:59.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:29:59 smithi149 ceph-mon[35408]: pgmap v436: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-27T07:30:00.310 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:00 smithi063 ceph-mon[31613]: pgmap v437: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-27T07:30:00.310 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:00 smithi063 ceph-mon[31613]: overall HEALTH_OK 2024-05-27T07:30:00.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:00 smithi086 ceph-mon[34687]: pgmap v437: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-27T07:30:00.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:00 smithi086 ceph-mon[34687]: overall HEALTH_OK 2024-05-27T07:30:00.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:00 smithi149 ceph-mon[35408]: pgmap v437: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-27T07:30:00.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:00 smithi149 ceph-mon[35408]: overall HEALTH_OK 2024-05-27T07:30:00.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.745+0000 7f8c2a200700 1 -- 172.21.15.63:0/3831495746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c24109860 msgr2=0x7f8c241111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:00.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.745+0000 7f8c2a200700 1 --2- 172.21.15.63:0/3831495746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c24109860 0x7f8c241111c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00b0d0 tx=0x7f8c1c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:30:00.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.745+0000 7f8c2a200700 1 -- 172.21.15.63:0/3831495746 shutdown_connections 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.745+0000 7f8c2a200700 1 --2- 172.21.15.63:0/3831495746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c24109860 0x7f8c241111c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.745+0000 7f8c2a200700 1 --2- 172.21.15.63:0/3831495746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c241045e0 0x7f8c241049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.745+0000 7f8c2a200700 1 --2- 172.21.15.63:0/3831495746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c24108f10 0x7f8c24109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.745+0000 7f8c2a200700 1 -- 172.21.15.63:0/3831495746 >> 172.21.15.63:0/3831495746 conn(0x7f8c240fffd0 msgr2=0x7f8c241023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.745+0000 7f8c2a200700 1 -- 172.21.15.63:0/3831495746 shutdown_connections 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.745+0000 7f8c2a200700 1 -- 172.21.15.63:0/3831495746 wait complete. 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.747+0000 7f8c2a200700 1 Processor -- start 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.747+0000 7f8c2a200700 1 -- start start 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.748+0000 7f8c2a200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c241045e0 0x7f8c2419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.748+0000 7f8c2a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c24108f10 0x7f8c2419d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.748+0000 7f8c2a200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c24109860 0x7f8c241a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.748+0000 7f8c2a200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24114390 con 0x7f8c24109860 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.748+0000 7f8c2a200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24114210 con 0x7f8c24108f10 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.748+0000 7f8c2a200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24114510 con 0x7f8c241045e0 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.748+0000 7f8c23e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c24109860 0x7f8c241a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.748+0000 7f8c23400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c241045e0 0x7f8c2419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.748+0000 7f8c23e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c24109860 0x7f8c241a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59690/0 (socket says 172.21.15.63:59690) 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.748+0000 7f8c23e00700 1 -- 172.21.15.63:0/2198009132 learned_addr learned my addr 172.21.15.63:0/2198009132 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.749+0000 7f8c1be00700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c24108f10 0x7f8c2419d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.749+0000 7f8c23e00700 1 -- 172.21.15.63:0/2198009132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c241045e0 msgr2=0x7f8c2419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.749+0000 7f8c23e00700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c241045e0 0x7f8c2419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.749+0000 7f8c23e00700 1 -- 172.21.15.63:0/2198009132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c24108f10 msgr2=0x7f8c2419d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.749+0000 7f8c23e00700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c24108f10 0x7f8c2419d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.749+0000 7f8c23e00700 1 -- 172.21.15.63:0/2198009132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c241a1e10 con 0x7f8c24109860 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.749+0000 7f8c23e00700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c24109860 0x7f8c241a1770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0079a0 tx=0x7f8c1c0079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.750+0000 7f8c20c00700 1 -- 172.21.15.63:0/2198009132 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c01f070 con 0x7f8c24109860 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.750+0000 7f8c20c00700 1 -- 172.21.15.63:0/2198009132 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c1c003c00 con 0x7f8c24109860 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.750+0000 7f8c2a200700 1 -- 172.21.15.63:0/2198009132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c241a20a0 con 0x7f8c24109860 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.750+0000 7f8c2a200700 1 -- 172.21.15.63:0/2198009132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c241a9a20 con 0x7f8c24109860 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.750+0000 7f8c20c00700 1 -- 172.21.15.63:0/2198009132 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c01b430 con 0x7f8c24109860 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.751+0000 7f8c2a200700 1 -- 172.21.15.63:0/2198009132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c00005330 con 0x7f8c24109860 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.754+0000 7f8c20c00700 1 -- 172.21.15.63:0/2198009132 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8c1c022070 con 0x7f8c24109860 2024-05-27T07:30:00.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.755+0000 7f8c20c00700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8c080b2900 0x7f8c080b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:00.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.755+0000 7f8c20c00700 1 -- 172.21.15.63:0/2198009132 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f8c1c00f020 con 0x7f8c24109860 2024-05-27T07:30:00.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.756+0000 7f8c23400700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8c080b2900 0x7f8c080b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:00.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.756+0000 7f8c23400700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8c080b2900 0x7f8c080b4e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c14005950 tx=0x7f8c1400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:00.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:00.758+0000 7f8c20c00700 1 -- 172.21.15.63:0/2198009132 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c1c029020 con 0x7f8c24109860 2024-05-27T07:30:01.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.033+0000 7f8c2a200700 1 -- 172.21.15.63:0/2198009132 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8c00000bc0 con 0x7f8c080b2900 2024-05-27T07:30:01.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.096+0000 7f8c20c00700 1 -- 172.21.15.63:0/2198009132 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8c00000bc0 con 0x7f8c080b2900 2024-05-27T07:30:01.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.099+0000 7f8c2a200700 1 -- 172.21.15.63:0/2198009132 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8c080b2900 msgr2=0x7f8c080b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:01.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.099+0000 7f8c2a200700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8c080b2900 0x7f8c080b4e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c14005950 tx=0x7f8c1400b410 comp rx=0 tx=0).stop 2024-05-27T07:30:01.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.099+0000 7f8c2a200700 1 -- 172.21.15.63:0/2198009132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c24109860 msgr2=0x7f8c241a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:01.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.099+0000 7f8c2a200700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c24109860 0x7f8c241a1770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0079a0 tx=0x7f8c1c0079d0 comp rx=0 tx=0).stop 2024-05-27T07:30:01.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.100+0000 7f8c2a200700 1 -- 172.21.15.63:0/2198009132 shutdown_connections 2024-05-27T07:30:01.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.100+0000 7f8c2a200700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8c080b2900 0x7f8c080b4e10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:01.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.100+0000 7f8c2a200700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c24109860 0x7f8c241a1770 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:01.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.100+0000 7f8c2a200700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c241045e0 0x7f8c2419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:01.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.100+0000 7f8c2a200700 1 --2- 172.21.15.63:0/2198009132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c24108f10 0x7f8c2419d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:01.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.100+0000 7f8c2a200700 1 -- 172.21.15.63:0/2198009132 >> 172.21.15.63:0/2198009132 conn(0x7f8c240fffd0 msgr2=0x7f8c2410fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:01.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.100+0000 7f8c2a200700 1 -- 172.21.15.63:0/2198009132 shutdown_connections 2024-05-27T07:30:01.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:01.100+0000 7f8c2a200700 1 -- 172.21.15.63:0/2198009132 wait complete. 2024-05-27T07:30:01.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:01 smithi086 ceph-mon[34687]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:01.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:01 smithi149 ceph-mon[35408]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:01.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:01 smithi063 ceph-mon[31613]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:01.457 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:30:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:30:01.050+0000 7f1637400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:30:01.457 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:30:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:30:01.050+0000 7f1637400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:30:01.457 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:30:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:30:01.050+0000 7f1637400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:30:01.457 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:30:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:30:01.050+0000 7f1637400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:30:01.457 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:30:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:30:01.050+0000 7f1637400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:30:01.910 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-27T07:30:02.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:02 smithi063 ceph-mon[31613]: pgmap v438: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-27T07:30:02.379 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:02.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:02 smithi086 ceph-mon[34687]: pgmap v438: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-27T07:30:02.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:02 smithi149 ceph-mon[35408]: pgmap v438: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-27T07:30:03.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:03 smithi063 ceph-mon[31613]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-27T07:30:03.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:03 smithi086 ceph-mon[34687]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-27T07:30:03.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:03 smithi149 ceph-mon[35408]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-27T07:30:04.361 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:04 smithi063 ceph-mon[31613]: pgmap v439: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-27T07:30:04.361 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:04 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6, 3, 7, 5, 9]}]: dispatch 2024-05-27T07:30:04.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:04 smithi086 ceph-mon[34687]: pgmap v439: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-27T07:30:04.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:04 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6, 3, 7, 5, 9]}]: dispatch 2024-05-27T07:30:04.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:04 smithi149 ceph-mon[35408]: pgmap v439: 129 pgs: 129 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-27T07:30:04.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:04 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6, 3, 7, 5, 9]}]: dispatch 2024-05-27T07:30:04.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.625+0000 7fe82d800700 1 -- 172.21.15.63:0/1111634403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe828106eb0 msgr2=0x7fe828113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:04.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.625+0000 7fe82d800700 1 --2- 172.21.15.63:0/1111634403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe828106eb0 0x7fe828113340 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe82000b0a0 tx=0x7fe82000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.625+0000 7fe82d800700 1 -- 172.21.15.63:0/1111634403 shutdown_connections 2024-05-27T07:30:04.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.625+0000 7fe82d800700 1 --2- 172.21.15.63:0/1111634403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe828106eb0 0x7fe828113340 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.625+0000 7fe82d800700 1 --2- 172.21.15.63:0/1111634403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82810c5a0 0x7fe82810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.626+0000 7fe82d800700 1 --2- 172.21.15.63:0/1111634403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe828106560 0x7fe828106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.626+0000 7fe82d800700 1 -- 172.21.15.63:0/1111634403 >> 172.21.15.63:0/1111634403 conn(0x7fe828102040 msgr2=0x7fe828104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:04.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.626+0000 7fe82d800700 1 -- 172.21.15.63:0/1111634403 shutdown_connections 2024-05-27T07:30:04.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.626+0000 7fe82d800700 1 -- 172.21.15.63:0/1111634403 wait complete. 2024-05-27T07:30:04.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.627+0000 7fe82d800700 1 Processor -- start 2024-05-27T07:30:04.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.627+0000 7fe82d800700 1 -- start start 2024-05-27T07:30:04.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe82d800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe828106560 0x7fe82819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:04.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe82d800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe828106eb0 0x7fe82819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:04.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe82d800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82810c5a0 0x7fe8281a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe82d800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281165d0 con 0x7fe82810c5a0 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe82d800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe828116450 con 0x7fe828106560 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe82d800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe828116750 con 0x7fe828106eb0 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe827400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82810c5a0 0x7fe8281a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe827400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82810c5a0 0x7fe8281a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59716/0 (socket says 172.21.15.63:59716) 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe827400700 1 -- 172.21.15.63:0/3620324983 learned_addr learned my addr 172.21.15.63:0/3620324983 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe826a00700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe828106560 0x7fe82819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.628+0000 7fe826000700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe828106eb0 0x7fe82819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.629+0000 7fe827400700 1 -- 172.21.15.63:0/3620324983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe828106eb0 msgr2=0x7fe82819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.629+0000 7fe827400700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe828106eb0 0x7fe82819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.629+0000 7fe827400700 1 -- 172.21.15.63:0/3620324983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe828106560 msgr2=0x7fe82819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.629+0000 7fe827400700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe828106560 0x7fe82819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.629+0000 7fe827400700 1 -- 172.21.15.63:0/3620324983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8281a1d90 con 0x7fe82810c5a0 2024-05-27T07:30:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.629+0000 7fe827400700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82810c5a0 0x7fe8281a16f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe82000b070 tx=0x7fe820004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.629+0000 7fe81f400700 1 -- 172.21.15.63:0/3620324983 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820021070 con 0x7fe82810c5a0 2024-05-27T07:30:04.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.630+0000 7fe81f400700 1 -- 172.21.15.63:0/3620324983 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe820007b80 con 0x7fe82810c5a0 2024-05-27T07:30:04.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.630+0000 7fe82d800700 1 -- 172.21.15.63:0/3620324983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8281a2020 con 0x7fe82810c5a0 2024-05-27T07:30:04.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.630+0000 7fe81f400700 1 -- 172.21.15.63:0/3620324983 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820004650 con 0x7fe82810c5a0 2024-05-27T07:30:04.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.630+0000 7fe82d800700 1 -- 172.21.15.63:0/3620324983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8281a99a0 con 0x7fe82810c5a0 2024-05-27T07:30:04.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.631+0000 7fe82d800700 1 -- 172.21.15.63:0/3620324983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe82810e960 con 0x7fe82810c5a0 2024-05-27T07:30:04.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.635+0000 7fe81f400700 1 -- 172.21.15.63:0/3620324983 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe820003a00 con 0x7fe82810c5a0 2024-05-27T07:30:04.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.636+0000 7fe81f400700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe80c0b2990 0x7fe80c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:04.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.637+0000 7fe81f400700 1 -- 172.21.15.63:0/3620324983 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9137+0+0 (secure 0 0 0) 0x7fe82000f020 con 0x7fe82810c5a0 2024-05-27T07:30:04.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.637+0000 7fe826a00700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe80c0b2990 0x7fe80c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:04.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.638+0000 7fe826a00700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe80c0b2990 0x7fe80c0b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8281079d0 tx=0x7fe818009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:04.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.638+0000 7fe81f400700 1 -- 172.21.15.63:0/3620324983 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe82002a020 con 0x7fe82810c5a0 2024-05-27T07:30:04.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.907+0000 7fe82d800700 1 -- 172.21.15.63:0/3620324983 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe82819e220 con 0x7fe80c0b2990 2024-05-27T07:30:04.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.982+0000 7fe81f400700 1 -- 172.21.15.63:0/3620324983 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe82819e220 con 0x7fe80c0b2990 2024-05-27T07:30:04.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.984+0000 7fe82d800700 1 -- 172.21.15.63:0/3620324983 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe80c0b2990 msgr2=0x7fe80c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.984+0000 7fe82d800700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe80c0b2990 0x7fe80c0b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8281079d0 tx=0x7fe818009260 comp rx=0 tx=0).stop 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.984+0000 7fe82d800700 1 -- 172.21.15.63:0/3620324983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82810c5a0 msgr2=0x7fe8281a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.984+0000 7fe82d800700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82810c5a0 0x7fe8281a16f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe82000b070 tx=0x7fe820004c80 comp rx=0 tx=0).stop 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.985+0000 7fe82d800700 1 -- 172.21.15.63:0/3620324983 shutdown_connections 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.985+0000 7fe82d800700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe80c0b2990 0x7fe80c0b4ea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.985+0000 7fe82d800700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe82810c5a0 0x7fe8281a16f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.985+0000 7fe82d800700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe828106560 0x7fe82819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.985+0000 7fe82d800700 1 --2- 172.21.15.63:0/3620324983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe828106eb0 0x7fe82819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.985+0000 7fe82d800700 1 -- 172.21.15.63:0/3620324983 >> 172.21.15.63:0/3620324983 conn(0x7fe828102040 msgr2=0x7fe8281037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.985+0000 7fe82d800700 1 -- 172.21.15.63:0/3620324983 shutdown_connections 2024-05-27T07:30:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:04.985+0000 7fe82d800700 1 -- 172.21.15.63:0/3620324983 wait complete. 2024-05-27T07:30:05.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:05 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6, 3, 7, 5, 9]}]': finished 2024-05-27T07:30:05.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:05 smithi063 ceph-mon[31613]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T07:30:05.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:05 smithi063 ceph-mon[31613]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:05.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:05 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6, 3, 7, 5, 9]}]': finished 2024-05-27T07:30:05.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:05 smithi086 ceph-mon[34687]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T07:30:05.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:05 smithi086 ceph-mon[34687]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:05 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6, 3, 7, 5, 9]}]': finished 2024-05-27T07:30:05.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:05 smithi149 ceph-mon[35408]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T07:30:05.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:05 smithi149 ceph-mon[35408]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:05.858 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-27T07:30:06.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:06 smithi063 ceph-mon[31613]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T07:30:06.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:06 smithi063 ceph-mon[31613]: pgmap v442: 129 pgs: 129 active+clean; 590 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-27T07:30:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:06.449 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:06 smithi086 ceph-mon[34687]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T07:30:06.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:06 smithi086 ceph-mon[34687]: pgmap v442: 129 pgs: 129 active+clean; 590 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-27T07:30:06.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:06 smithi149 ceph-mon[35408]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T07:30:06.453 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:06 smithi149 ceph-mon[35408]: pgmap v442: 129 pgs: 129 active+clean; 590 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-27T07:30:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.782+0000 7f033be00700 1 -- 172.21.15.63:0/1926147515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0334105510 msgr2=0x7f0334105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.782+0000 7f033be00700 1 --2- 172.21.15.63:0/1926147515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0334105510 0x7f0334105920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0328009990 tx=0x7f0328009ca0 comp rx=0 tx=0).stop 2024-05-27T07:30:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.782+0000 7f033be00700 1 -- 172.21.15.63:0/1926147515 shutdown_connections 2024-05-27T07:30:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.782+0000 7f033be00700 1 --2- 172.21.15.63:0/1926147515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f033410e8f0 0x7f0334110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.782+0000 7f033be00700 1 --2- 172.21.15.63:0/1926147515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0334105510 0x7f0334105920 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.782+0000 7f033be00700 1 --2- 172.21.15.63:0/1926147515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0334105e60 0x7f033410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.782+0000 7f033be00700 1 -- 172.21.15.63:0/1926147515 >> 172.21.15.63:0/1926147515 conn(0x7f03340fedb0 msgr2=0x7f03341011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.782+0000 7f033be00700 1 -- 172.21.15.63:0/1926147515 shutdown_connections 2024-05-27T07:30:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.782+0000 7f033be00700 1 -- 172.21.15.63:0/1926147515 wait complete. 2024-05-27T07:30:08.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.784+0000 7f033be00700 1 Processor -- start 2024-05-27T07:30:08.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.784+0000 7f033be00700 1 -- start start 2024-05-27T07:30:08.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.784+0000 7f033be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0334105510 0x7f0334103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:08.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.784+0000 7f033be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0334105e60 0x7f0334103740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:08.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.785+0000 7f033be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f033410e8f0 0x7f0334103c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:08.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.785+0000 7f033be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03341142d0 con 0x7f033410e8f0 2024-05-27T07:30:08.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.785+0000 7f033be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0334114150 con 0x7f0334105e60 2024-05-27T07:30:08.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.785+0000 7f033be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0334114450 con 0x7f0334105510 2024-05-27T07:30:08.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.785+0000 7f0339e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f033410e8f0 0x7f0334103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:08.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.785+0000 7f0339e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f033410e8f0 0x7f0334103c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59734/0 (socket says 172.21.15.63:59734) 2024-05-27T07:30:08.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.785+0000 7f0339e00700 1 -- 172.21.15.63:0/3959677267 learned_addr learned my addr 172.21.15.63:0/3959677267 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:08.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.785+0000 7f0339400700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0334105510 0x7f0334103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:08.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.785+0000 7f0338a00700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0334105e60 0x7f0334103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:08.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.786+0000 7f0339e00700 1 -- 172.21.15.63:0/3959677267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0334105510 msgr2=0x7f0334103200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:08.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.786+0000 7f0339e00700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0334105510 0x7f0334103200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:08.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.786+0000 7f0339e00700 1 -- 172.21.15.63:0/3959677267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0334105e60 msgr2=0x7f0334103740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:08.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.786+0000 7f0339e00700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0334105e60 0x7f0334103740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:08.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.786+0000 7f0339e00700 1 -- 172.21.15.63:0/3959677267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03341a9080 con 0x7f033410e8f0 2024-05-27T07:30:08.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.786+0000 7f0339e00700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f033410e8f0 0x7f0334103c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f032c007ee0 tx=0x7f032c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:08.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.786+0000 7f0332000700 1 -- 172.21.15.63:0/3959677267 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c004bb0 con 0x7f033410e8f0 2024-05-27T07:30:08.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.787+0000 7f0332000700 1 -- 172.21.15.63:0/3959677267 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f032c004d50 con 0x7f033410e8f0 2024-05-27T07:30:08.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.787+0000 7f0332000700 1 -- 172.21.15.63:0/3959677267 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c0058b0 con 0x7f033410e8f0 2024-05-27T07:30:08.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.787+0000 7f033be00700 1 -- 172.21.15.63:0/3959677267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03341a9370 con 0x7f033410e8f0 2024-05-27T07:30:08.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.787+0000 7f033be00700 1 -- 172.21.15.63:0/3959677267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03341a9860 con 0x7f033410e8f0 2024-05-27T07:30:08.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.788+0000 7f033be00700 1 -- 172.21.15.63:0/3959677267 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03340512b0 con 0x7f033410e8f0 2024-05-27T07:30:08.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.790+0000 7f0332000700 1 -- 172.21.15.63:0/3959677267 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f032c01a020 con 0x7f033410e8f0 2024-05-27T07:30:08.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.792+0000 7f0332000700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f031c0b2910 0x7f031c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:08.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.792+0000 7f0332000700 1 -- 172.21.15.63:0/3959677267 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f032c0e1df0 con 0x7f033410e8f0 2024-05-27T07:30:08.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.792+0000 7f0339400700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f031c0b2910 0x7f031c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:08.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.793+0000 7f0339400700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f031c0b2910 0x7f031c0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f032800b5c0 tx=0x7f0328005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:08.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:08.797+0000 7f0332000700 1 -- 172.21.15.63:0/3959677267 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f032c010070 con 0x7f033410e8f0 2024-05-27T07:30:08.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:08 smithi063 ceph-mon[31613]: pgmap v443: 129 pgs: 129 active+clean; 590 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-27T07:30:09.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.075+0000 7f033be00700 1 -- 172.21.15.63:0/3959677267 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0334063bf0 con 0x7f031c0b2910 2024-05-27T07:30:09.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.091+0000 7f0332000700 1 -- 172.21.15.63:0/3959677267 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0334063bf0 con 0x7f031c0b2910 2024-05-27T07:30:09.082 INFO:teuthology.orchestra.run.smithi063.stdout:/volumes/qa/sv_0/58c2b8b2-2562-4ecc-bcb2-fb349cec4fca 2024-05-27T07:30:09.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.095+0000 7f033be00700 1 -- 172.21.15.63:0/3959677267 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f031c0b2910 msgr2=0x7f031c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:09.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.095+0000 7f033be00700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f031c0b2910 0x7f031c0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f032800b5c0 tx=0x7f0328005960 comp rx=0 tx=0).stop 2024-05-27T07:30:09.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.095+0000 7f033be00700 1 -- 172.21.15.63:0/3959677267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f033410e8f0 msgr2=0x7f0334103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:09.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.095+0000 7f033be00700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f033410e8f0 0x7f0334103c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f032c007ee0 tx=0x7f032c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:30:09.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.096+0000 7f033be00700 1 -- 172.21.15.63:0/3959677267 shutdown_connections 2024-05-27T07:30:09.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.096+0000 7f033be00700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f031c0b2910 0x7f031c0b4e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:09.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.096+0000 7f033be00700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f033410e8f0 0x7f0334103c80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:09.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.096+0000 7f033be00700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0334105510 0x7f0334103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:09.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.096+0000 7f033be00700 1 --2- 172.21.15.63:0/3959677267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0334105e60 0x7f0334103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:09.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.096+0000 7f033be00700 1 -- 172.21.15.63:0/3959677267 >> 172.21.15.63:0/3959677267 conn(0x7f03340fedb0 msgr2=0x7f03340ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:09.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.096+0000 7f033be00700 1 -- 172.21.15.63:0/3959677267 shutdown_connections 2024-05-27T07:30:09.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:09.096+0000 7f033be00700 1 -- 172.21.15.63:0/3959677267 wait complete. 2024-05-27T07:30:09.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:08 smithi086 ceph-mon[34687]: pgmap v443: 129 pgs: 129 active+clean; 590 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-27T07:30:09.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:08 smithi149 ceph-mon[35408]: pgmap v443: 129 pgs: 129 active+clean; 590 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-27T07:30:10.049 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-27T07:30:10.527 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:10.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:10 smithi063 ceph-mon[31613]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:10.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:10 smithi063 ceph-mon[31613]: pgmap v444: 129 pgs: 129 active+clean; 610 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-27T07:30:11.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:10 smithi086 ceph-mon[34687]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:11.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:10 smithi086 ceph-mon[34687]: pgmap v444: 129 pgs: 129 active+clean; 610 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-27T07:30:11.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:10 smithi149 ceph-mon[35408]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:11.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:10 smithi149 ceph-mon[35408]: pgmap v444: 129 pgs: 129 active+clean; 610 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-27T07:30:13.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.033+0000 7fb556800700 1 -- 172.21.15.63:0/2813019590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb55010b890 msgr2=0x7fb5501131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:13.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.033+0000 7fb556800700 1 --2- 172.21.15.63:0/2813019590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb55010b890 0x7fb5501131f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b0d0 tx=0x7fb54800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.033+0000 7fb556800700 1 -- 172.21.15.63:0/2813019590 shutdown_connections 2024-05-27T07:30:13.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.033+0000 7fb556800700 1 --2- 172.21.15.63:0/2813019590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb55010b890 0x7fb5501131f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.033+0000 7fb556800700 1 --2- 172.21.15.63:0/2813019590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb550106610 0x7fb550106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.033+0000 7fb556800700 1 --2- 172.21.15.63:0/2813019590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb55010af40 0x7fb55010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.033+0000 7fb556800700 1 -- 172.21.15.63:0/2813019590 >> 172.21.15.63:0/2813019590 conn(0x7fb550102020 msgr2=0x7fb550104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:13.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.033+0000 7fb556800700 1 -- 172.21.15.63:0/2813019590 shutdown_connections 2024-05-27T07:30:13.025 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:12 smithi063 ceph-mon[31613]: pgmap v445: 129 pgs: 129 active+clean; 610 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-05-27T07:30:13.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.034+0000 7fb556800700 1 -- 172.21.15.63:0/2813019590 wait complete. 2024-05-27T07:30:13.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.035+0000 7fb556800700 1 Processor -- start 2024-05-27T07:30:13.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.035+0000 7fb556800700 1 -- start start 2024-05-27T07:30:13.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.035+0000 7fb556800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb550106610 0x7fb55019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:13.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.035+0000 7fb556800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb55010af40 0x7fb55019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:13.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.036+0000 7fb556800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb55010b890 0x7fb5501a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:13.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.036+0000 7fb556800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5501163c0 con 0x7fb55010af40 2024-05-27T07:30:13.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.036+0000 7fb556800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb550116240 con 0x7fb55010b890 2024-05-27T07:30:13.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.036+0000 7fb556800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb550116540 con 0x7fb550106610 2024-05-27T07:30:13.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.036+0000 7fb554a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb55010af40 0x7fb55019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:13.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.036+0000 7fb554a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb55010af40 0x7fb55019d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54840/0 (socket says 172.21.15.63:54840) 2024-05-27T07:30:13.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.036+0000 7fb554a00700 1 -- 172.21.15.63:0/711314721 learned_addr learned my addr 172.21.15.63:0/711314721 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:13.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.036+0000 7fb555400700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb550106610 0x7fb55019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:13.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.036+0000 7fb555e00700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb55010b890 0x7fb5501a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:13.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb554a00700 1 -- 172.21.15.63:0/711314721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb550106610 msgr2=0x7fb55019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:13.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb554a00700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb550106610 0x7fb55019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb554a00700 1 -- 172.21.15.63:0/711314721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb55010b890 msgr2=0x7fb5501a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:13.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb554a00700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb55010b890 0x7fb5501a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb554a00700 1 -- 172.21.15.63:0/711314721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5501a1b30 con 0x7fb55010af40 2024-05-27T07:30:13.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb555400700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb550106610 0x7fb55019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:30:13.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb554a00700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb55010af40 0x7fb55019d2b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00d980 tx=0x7fb53c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:13.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb54e000700 1 -- 172.21.15.63:0/711314721 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c017070 con 0x7fb55010af40 2024-05-27T07:30:13.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb54e000700 1 -- 172.21.15.63:0/711314721 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb53c010460 con 0x7fb55010af40 2024-05-27T07:30:13.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb54e000700 1 -- 172.21.15.63:0/711314721 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00de60 con 0x7fb55010af40 2024-05-27T07:30:13.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.037+0000 7fb556800700 1 -- 172.21.15.63:0/711314721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5501a1e20 con 0x7fb55010af40 2024-05-27T07:30:13.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.038+0000 7fb556800700 1 -- 172.21.15.63:0/711314721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5501a2310 con 0x7fb55010af40 2024-05-27T07:30:13.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.039+0000 7fb556800700 1 -- 172.21.15.63:0/711314721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5500512b0 con 0x7fb55010af40 2024-05-27T07:30:13.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.041+0000 7fb54e000700 1 -- 172.21.15.63:0/711314721 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb53c010610 con 0x7fb55010af40 2024-05-27T07:30:13.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.043+0000 7fb54e000700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb5380b2990 0x7fb5380b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:13.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.043+0000 7fb54e000700 1 -- 172.21.15.63:0/711314721 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fb53c0e21a0 con 0x7fb55010af40 2024-05-27T07:30:13.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.043+0000 7fb555400700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb5380b2990 0x7fb5380b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:13.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.044+0000 7fb555400700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb5380b2990 0x7fb5380b4ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb544005fd0 tx=0x7fb544005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:13.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.046+0000 7fb54e000700 1 -- 172.21.15.63:0/711314721 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb53c0a99f0 con 0x7fb55010af40 2024-05-27T07:30:13.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:12 smithi086 ceph-mon[34687]: pgmap v445: 129 pgs: 129 active+clean; 610 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-05-27T07:30:13.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:12 smithi149 ceph-mon[35408]: pgmap v445: 129 pgs: 129 active+clean; 610 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-05-27T07:30:13.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.321+0000 7fb556800700 1 -- 172.21.15.63:0/711314721 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb550063bf0 con 0x7fb5380b2990 2024-05-27T07:30:13.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.394+0000 7fb54e000700 1 -- 172.21.15.63:0/711314721 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb550063bf0 con 0x7fb5380b2990 2024-05-27T07:30:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.397+0000 7fb556800700 1 -- 172.21.15.63:0/711314721 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb5380b2990 msgr2=0x7fb5380b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.397+0000 7fb556800700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb5380b2990 0x7fb5380b4ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb544005fd0 tx=0x7fb544005e30 comp rx=0 tx=0).stop 2024-05-27T07:30:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.397+0000 7fb556800700 1 -- 172.21.15.63:0/711314721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb55010af40 msgr2=0x7fb55019d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:13.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.397+0000 7fb556800700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb55010af40 0x7fb55019d2b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00d980 tx=0x7fb53c00dd40 comp rx=0 tx=0).stop 2024-05-27T07:30:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.398+0000 7fb556800700 1 -- 172.21.15.63:0/711314721 shutdown_connections 2024-05-27T07:30:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.398+0000 7fb556800700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb55010b890 0x7fb5501a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.398+0000 7fb556800700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb550106610 0x7fb55019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.398+0000 7fb556800700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb5380b2990 0x7fb5380b4ea0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.398+0000 7fb556800700 1 --2- 172.21.15.63:0/711314721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb55010af40 0x7fb55019d2b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.398+0000 7fb556800700 1 -- 172.21.15.63:0/711314721 >> 172.21.15.63:0/711314721 conn(0x7fb550102020 msgr2=0x7fb55010a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.398+0000 7fb556800700 1 -- 172.21.15.63:0/711314721 shutdown_connections 2024-05-27T07:30:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:13.398+0000 7fb556800700 1 -- 172.21.15.63:0/711314721 wait complete. 2024-05-27T07:30:14.344 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-27T07:30:14.831 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:15.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:14 smithi063 ceph-mon[31613]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:15.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:14 smithi063 ceph-mon[31613]: pgmap v446: 129 pgs: 129 active+clean; 610 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-27T07:30:15.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:14 smithi086 ceph-mon[34687]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:15.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:14 smithi086 ceph-mon[34687]: pgmap v446: 129 pgs: 129 active+clean; 610 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-27T07:30:15.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:14 smithi149 ceph-mon[35408]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:15.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:14 smithi149 ceph-mon[35408]: pgmap v446: 129 pgs: 129 active+clean; 610 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-27T07:30:16.910 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:16 smithi063 ceph-mon[31613]: pgmap v447: 129 pgs: 129 active+clean; 628 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-27T07:30:17.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:16 smithi086 ceph-mon[34687]: pgmap v447: 129 pgs: 129 active+clean; 628 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-27T07:30:17.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:16 smithi149 ceph-mon[35408]: pgmap v447: 129 pgs: 129 active+clean; 628 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-27T07:30:17.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.276+0000 7faa7d200700 1 -- 172.21.15.63:0/2944984340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa78108f10 msgr2=0x7faa78109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:17.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.276+0000 7faa7d200700 1 --2- 172.21.15.63:0/2944984340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa78108f10 0x7faa78109320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faa640099c0 tx=0x7faa64009cd0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.276+0000 7faa7d200700 1 -- 172.21.15.63:0/2944984340 shutdown_connections 2024-05-27T07:30:17.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.276+0000 7faa7d200700 1 --2- 172.21.15.63:0/2944984340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78109860 0x7faa781111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.276+0000 7faa7d200700 1 --2- 172.21.15.63:0/2944984340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa781045e0 0x7faa781049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.276+0000 7faa7d200700 1 --2- 172.21.15.63:0/2944984340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa78108f10 0x7faa78109320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.276+0000 7faa7d200700 1 -- 172.21.15.63:0/2944984340 >> 172.21.15.63:0/2944984340 conn(0x7faa78100030 msgr2=0x7faa78102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:17.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.277+0000 7faa7d200700 1 -- 172.21.15.63:0/2944984340 shutdown_connections 2024-05-27T07:30:17.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.277+0000 7faa7d200700 1 -- 172.21.15.63:0/2944984340 wait complete. 2024-05-27T07:30:17.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.279+0000 7faa7d200700 1 Processor -- start 2024-05-27T07:30:17.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.279+0000 7faa7d200700 1 -- start start 2024-05-27T07:30:17.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.279+0000 7faa7d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa781045e0 0x7faa7819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:17.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.279+0000 7faa7d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa78108f10 0x7faa7819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:17.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.280+0000 7faa7d200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78109860 0x7faa781a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:17.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.280+0000 7faa7d200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa78114420 con 0x7faa78108f10 2024-05-27T07:30:17.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.280+0000 7faa7d200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa781142a0 con 0x7faa781045e0 2024-05-27T07:30:17.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.280+0000 7faa7d200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa781145a0 con 0x7faa78109860 2024-05-27T07:30:17.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.280+0000 7faa75400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa78108f10 0x7faa7819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.280+0000 7faa75400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa78108f10 0x7faa7819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54846/0 (socket says 172.21.15.63:54846) 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.280+0000 7faa75400700 1 -- 172.21.15.63:0/3808280375 learned_addr learned my addr 172.21.15.63:0/3808280375 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.280+0000 7faa75e00700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa781045e0 0x7faa7819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.280+0000 7faa76800700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78109860 0x7faa781a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa75400700 1 -- 172.21.15.63:0/3808280375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78109860 msgr2=0x7faa781a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa75400700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78109860 0x7faa781a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa75400700 1 -- 172.21.15.63:0/3808280375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa781045e0 msgr2=0x7faa7819d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa75400700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa781045e0 0x7faa7819d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa75400700 1 -- 172.21.15.63:0/3808280375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa781a1f20 con 0x7faa78108f10 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa76800700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78109860 0x7faa781a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa75e00700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa781045e0 0x7faa7819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:30:17.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa75400700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa78108f10 0x7faa7819d5f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa64006010 tx=0x7faa64005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:17.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa6ea00700 1 -- 172.21.15.63:0/3808280375 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64026070 con 0x7faa78108f10 2024-05-27T07:30:17.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa6ea00700 1 -- 172.21.15.63:0/3808280375 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa640040b0 con 0x7faa78108f10 2024-05-27T07:30:17.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa6ea00700 1 -- 172.21.15.63:0/3808280375 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6400fc70 con 0x7faa78108f10 2024-05-27T07:30:17.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.281+0000 7faa7d200700 1 -- 172.21.15.63:0/3808280375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa781a21b0 con 0x7faa78108f10 2024-05-27T07:30:17.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.282+0000 7faa7d200700 1 -- 172.21.15.63:0/3808280375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa781a9b30 con 0x7faa78108f10 2024-05-27T07:30:17.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.283+0000 7faa7d200700 1 -- 172.21.15.63:0/3808280375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa78105a90 con 0x7faa78108f10 2024-05-27T07:30:17.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.285+0000 7faa6ea00700 1 -- 172.21.15.63:0/3808280375 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7faa64029070 con 0x7faa78108f10 2024-05-27T07:30:17.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.287+0000 7faa6ea00700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa5c0b2990 0x7faa5c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:17.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.287+0000 7faa6ea00700 1 -- 172.21.15.63:0/3808280375 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7faa640e39c0 con 0x7faa78108f10 2024-05-27T07:30:17.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.287+0000 7faa75e00700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa5c0b2990 0x7faa5c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:17.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.288+0000 7faa75e00700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa5c0b2990 0x7faa5c0b4ea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa6800bcf0 tx=0x7faa6800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:17.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.290+0000 7faa6ea00700 1 -- 172.21.15.63:0/3808280375 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa64013070 con 0x7faa78108f10 2024-05-27T07:30:17.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.565+0000 7faa7d200700 1 -- 172.21.15.63:0/3808280375 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7faa7819e3b0 con 0x7faa5c0b2990 2024-05-27T07:30:17.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.575+0000 7faa6ea00700 1 -- 172.21.15.63:0/3808280375 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7faa7819e3b0 con 0x7faa5c0b2990 2024-05-27T07:30:17.566 INFO:teuthology.orchestra.run.smithi063.stdout:/volumes/qa/sv_1/78e6ce10-677d-44b7-93cb-c4cc5c8bf511 2024-05-27T07:30:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.578+0000 7faa7d200700 1 -- 172.21.15.63:0/3808280375 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa5c0b2990 msgr2=0x7faa5c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.578+0000 7faa7d200700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa5c0b2990 0x7faa5c0b4ea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa6800bcf0 tx=0x7faa6800b2c0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.578+0000 7faa7d200700 1 -- 172.21.15.63:0/3808280375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa78108f10 msgr2=0x7faa7819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.578+0000 7faa7d200700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa78108f10 0x7faa7819d5f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa64006010 tx=0x7faa64005f10 comp rx=0 tx=0).stop 2024-05-27T07:30:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.579+0000 7faa7d200700 1 -- 172.21.15.63:0/3808280375 shutdown_connections 2024-05-27T07:30:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.579+0000 7faa7d200700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78109860 0x7faa781a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.579+0000 7faa7d200700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa781045e0 0x7faa7819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.579+0000 7faa7d200700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faa5c0b2990 0x7faa5c0b4ea0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.579+0000 7faa7d200700 1 --2- 172.21.15.63:0/3808280375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa78108f10 0x7faa7819d5f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:17.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.579+0000 7faa7d200700 1 -- 172.21.15.63:0/3808280375 >> 172.21.15.63:0/3808280375 conn(0x7faa78100030 msgr2=0x7faa7810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.579+0000 7faa7d200700 1 -- 172.21.15.63:0/3808280375 shutdown_connections 2024-05-27T07:30:17.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:17.579+0000 7faa7d200700 1 -- 172.21.15.63:0/3808280375 wait complete. 2024-05-27T07:30:18.421 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:30:18.900 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:19.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:18 smithi086 ceph-mon[34687]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:19.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:18 smithi086 ceph-mon[34687]: pgmap v448: 129 pgs: 129 active+clean; 628 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-05-27T07:30:19.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:18 smithi149 ceph-mon[35408]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:19.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:18 smithi149 ceph-mon[35408]: pgmap v448: 129 pgs: 129 active+clean; 628 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-05-27T07:30:19.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:18 smithi063 ceph-mon[31613]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:19.205 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:18 smithi063 ceph-mon[31613]: pgmap v448: 129 pgs: 129 active+clean; 628 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-05-27T07:30:21.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:20 smithi086 ceph-mon[34687]: pgmap v449: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-05-27T07:30:21.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:20 smithi149 ceph-mon[35408]: pgmap v449: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-05-27T07:30:21.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.214+0000 7ff96ba00700 1 -- 172.21.15.63:0/1241278381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff964110bf0 msgr2=0x7ff964113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:21.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.214+0000 7ff96ba00700 1 --2- 172.21.15.63:0/1241278381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff964110bf0 0x7ff964113010 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00b0a0 tx=0x7ff95c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.214+0000 7ff96ba00700 1 -- 172.21.15.63:0/1241278381 shutdown_connections 2024-05-27T07:30:21.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.214+0000 7ff96ba00700 1 --2- 172.21.15.63:0/1241278381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff964110bf0 0x7ff964113010 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.214+0000 7ff96ba00700 1 --2- 172.21.15.63:0/1241278381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff964076c00 0x7ff964077010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.214+0000 7ff96ba00700 1 --2- 172.21.15.63:0/1241278381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff964077550 0x7ff9641106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.214+0000 7ff96ba00700 1 -- 172.21.15.63:0/1241278381 >> 172.21.15.63:0/1241278381 conn(0x7ff9640ffeb0 msgr2=0x7ff964102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:21.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.214+0000 7ff96ba00700 1 -- 172.21.15.63:0/1241278381 shutdown_connections 2024-05-27T07:30:21.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.214+0000 7ff96ba00700 1 -- 172.21.15.63:0/1241278381 wait complete. 2024-05-27T07:30:21.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:20 smithi063 ceph-mon[31613]: pgmap v449: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-05-27T07:30:21.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.216+0000 7ff96ba00700 1 Processor -- start 2024-05-27T07:30:21.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.216+0000 7ff96ba00700 1 -- start start 2024-05-27T07:30:21.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.217+0000 7ff96ba00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff964076c00 0x7ff96419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:21.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.217+0000 7ff96ba00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff964077550 0x7ff96419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:21.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.217+0000 7ff96ba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff964110bf0 0x7ff9641a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:21.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.217+0000 7ff96ba00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9641165b0 con 0x7ff964077550 2024-05-27T07:30:21.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.217+0000 7ff96ba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff964116430 con 0x7ff964110bf0 2024-05-27T07:30:21.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.217+0000 7ff96ba00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff964116730 con 0x7ff964076c00 2024-05-27T07:30:21.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.217+0000 7ff963e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff964077550 0x7ff96419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:21.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.217+0000 7ff963e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff964077550 0x7ff96419d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44234/0 (socket says 172.21.15.63:44234) 2024-05-27T07:30:21.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.217+0000 7ff963e00700 1 -- 172.21.15.63:0/448365202 learned_addr learned my addr 172.21.15.63:0/448365202 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:21.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.217+0000 7ff969a00700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff964110bf0 0x7ff9641a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:21.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.218+0000 7ff963e00700 1 -- 172.21.15.63:0/448365202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff964076c00 msgr2=0x7ff96419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:21.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.218+0000 7ff963e00700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff964076c00 0x7ff96419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.218+0000 7ff963e00700 1 -- 172.21.15.63:0/448365202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff964110bf0 msgr2=0x7ff9641a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:21.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.218+0000 7ff963e00700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff964110bf0 0x7ff9641a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.218+0000 7ff963e00700 1 -- 172.21.15.63:0/448365202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9641a1e20 con 0x7ff964077550 2024-05-27T07:30:21.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.218+0000 7ff963e00700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff964077550 0x7ff96419d540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff95000d980 tx=0x7ff95000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:21.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.218+0000 7ff961600700 1 -- 172.21.15.63:0/448365202 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950017070 con 0x7ff964077550 2024-05-27T07:30:21.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.218+0000 7ff961600700 1 -- 172.21.15.63:0/448365202 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff950010460 con 0x7ff964077550 2024-05-27T07:30:21.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.218+0000 7ff961600700 1 -- 172.21.15.63:0/448365202 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95000de60 con 0x7ff964077550 2024-05-27T07:30:21.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.218+0000 7ff96ba00700 1 -- 172.21.15.63:0/448365202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9641a2110 con 0x7ff964077550 2024-05-27T07:30:21.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.219+0000 7ff96ba00700 1 -- 172.21.15.63:0/448365202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9641a9a30 con 0x7ff964077550 2024-05-27T07:30:21.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.220+0000 7ff96ba00700 1 -- 172.21.15.63:0/448365202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9640512b0 con 0x7ff964077550 2024-05-27T07:30:21.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.223+0000 7ff961600700 1 -- 172.21.15.63:0/448365202 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff950010610 con 0x7ff964077550 2024-05-27T07:30:21.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.224+0000 7ff961600700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff94c0b2910 0x7ff94c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:21.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.224+0000 7ff961600700 1 -- 172.21.15.63:0/448365202 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7ff9500e2220 con 0x7ff964077550 2024-05-27T07:30:21.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.224+0000 7ff969000700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff94c0b2910 0x7ff94c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:21.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.225+0000 7ff969000700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff94c0b2910 0x7ff94c0b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff958005950 tx=0x7ff9580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:21.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.227+0000 7ff961600700 1 -- 172.21.15.63:0/448365202 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9500a99f0 con 0x7ff964077550 2024-05-27T07:30:21.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.522+0000 7ff96ba00700 1 -- 172.21.15.63:0/448365202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff96402e5e0 con 0x7ff964077550 2024-05-27T07:30:21.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.523+0000 7ff961600700 1 -- 172.21.15.63:0/448365202 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11861 (secure 0 0 0) 0x7ff9500a92f0 con 0x7ff964077550 2024-05-27T07:30:21.514 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:30:21.515 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:29:56.944716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T07:30:21.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 -- 172.21.15.63:0/448365202 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff94c0b2910 msgr2=0x7ff94c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:21.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff94c0b2910 0x7ff94c0b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff958005950 tx=0x7ff9580058e0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 -- 172.21.15.63:0/448365202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff964077550 msgr2=0x7ff96419d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:21.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff964077550 0x7ff96419d540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff95000d980 tx=0x7ff95000dd40 comp rx=0 tx=0).stop 2024-05-27T07:30:21.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 -- 172.21.15.63:0/448365202 shutdown_connections 2024-05-27T07:30:21.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff964110bf0 0x7ff9641a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff964076c00 0x7ff96419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff94c0b2910 0x7ff94c0b4e20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 --2- 172.21.15.63:0/448365202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff964077550 0x7ff96419d540 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:21.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 -- 172.21.15.63:0/448365202 >> 172.21.15.63:0/448365202 conn(0x7ff9640ffeb0 msgr2=0x7ff964101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:21.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 -- 172.21.15.63:0/448365202 shutdown_connections 2024-05-27T07:30:21.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:21.527+0000 7ff96ba00700 1 -- 172.21.15.63:0/448365202 wait complete. 2024-05-27T07:30:21.519 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 26 2024-05-27T07:30:22.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:21 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/448365202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:30:22.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:21 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/448365202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:30:22.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:21 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/448365202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:30:22.384 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:30:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:23.194 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:22 smithi063 ceph-mon[31613]: pgmap v450: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-27T07:30:23.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:22 smithi086 ceph-mon[34687]: pgmap v450: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-27T07:30:23.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:22 smithi149 ceph-mon[35408]: pgmap v450: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-27T07:30:25.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:24 smithi086 ceph-mon[34687]: pgmap v451: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-27T07:30:25.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:24 smithi149 ceph-mon[35408]: pgmap v451: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-27T07:30:25.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:24 smithi063 ceph-mon[31613]: pgmap v451: 129 pgs: 129 active+clean; 637 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-05-27T07:30:25.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.264+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1753632732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410af40 msgr2=0x7ff0f410b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.264+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/1753632732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410af40 0x7ff0f410b350 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0099c0 tx=0x7ff0dc009cd0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.264+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1753632732 shutdown_connections 2024-05-27T07:30:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.264+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/1753632732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410b890 0x7ff0f41131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.264+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/1753632732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4106610 0x7ff0f4106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.264+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/1753632732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410af40 0x7ff0f410b350 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.264+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1753632732 >> 172.21.15.63:0/1753632732 conn(0x7ff0f4102020 msgr2=0x7ff0f4104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.265+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1753632732 shutdown_connections 2024-05-27T07:30:25.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.265+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1753632732 wait complete. 2024-05-27T07:30:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.266+0000 7ff0f3e00700 1 Processor -- start 2024-05-27T07:30:25.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.266+0000 7ff0f3e00700 1 -- start start 2024-05-27T07:30:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.266+0000 7ff0f3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4106610 0x7ff0f4077cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.266+0000 7ff0f3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410af40 0x7ff0f4078200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0f3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410b890 0x7ff0f4078740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0f3e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4116350 con 0x7ff0f410af40 2024-05-27T07:30:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0f3e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41161d0 con 0x7ff0f4106610 2024-05-27T07:30:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0f3e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41164d0 con 0x7ff0f410b890 2024-05-27T07:30:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0ebe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410af40 0x7ff0f4078200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0ebe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410af40 0x7ff0f4078200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44260/0 (socket says 172.21.15.63:44260) 2024-05-27T07:30:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0ebe00700 1 -- 172.21.15.63:0/1579522650 learned_addr learned my addr 172.21.15.63:0/1579522650 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:25.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0f2a00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4106610 0x7ff0f4077cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:25.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0ebe00700 1 -- 172.21.15.63:0/1579522650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410b890 msgr2=0x7ff0f4078740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0ebe00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410b890 0x7ff0f4078740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0ebe00700 1 -- 172.21.15.63:0/1579522650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4106610 msgr2=0x7ff0f4077cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0ebe00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4106610 0x7ff0f4077cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0ebe00700 1 -- 172.21.15.63:0/1579522650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f41ad420 con 0x7ff0f410af40 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.267+0000 7ff0f2a00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4106610 0x7ff0f4077cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.268+0000 7ff0ebe00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410af40 0x7ff0f4078200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00b580 tx=0x7ff0dc00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.268+0000 7ff0f8a00700 1 -- 172.21.15.63:0/1579522650 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc027070 con 0x7ff0f410af40 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.268+0000 7ff0f8a00700 1 -- 172.21.15.63:0/1579522650 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0dc00fda0 con 0x7ff0f410af40 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.268+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1579522650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f41ad650 con 0x7ff0f410af40 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.268+0000 7ff0f8a00700 1 -- 172.21.15.63:0/1579522650 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc0173f0 con 0x7ff0f410af40 2024-05-27T07:30:25.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.268+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1579522650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f41adb00 con 0x7ff0f410af40 2024-05-27T07:30:25.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.269+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1579522650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f4107ac0 con 0x7ff0f410af40 2024-05-27T07:30:25.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.272+0000 7ff0f8a00700 1 -- 172.21.15.63:0/1579522650 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff0dc02a070 con 0x7ff0f410af40 2024-05-27T07:30:25.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.274+0000 7ff0f8a00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2990 0x7ff0d80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.274+0000 7ff0f8a00700 1 -- 172.21.15.63:0/1579522650 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7ff0dc0e37b0 con 0x7ff0f410af40 2024-05-27T07:30:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.274+0000 7ff0f2a00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2990 0x7ff0d80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:25.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.275+0000 7ff0f2a00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2990 0x7ff0d80b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400bc10 tx=0x7ff0e400b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:25.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.277+0000 7ff0f8a00700 1 -- 172.21.15.63:0/1579522650 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0dc05fdf0 con 0x7ff0f410af40 2024-05-27T07:30:25.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.518+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1579522650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0f402e5e0 con 0x7ff0f410af40 2024-05-27T07:30:25.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.523+0000 7ff0f8a00700 1 -- 172.21.15.63:0/1579522650 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24635 (secure 0 0 0) 0x7ff0dc013070 con 0x7ff0f410af40 2024-05-27T07:30:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:30:25.515 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":91,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:30:05.109968+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6},{"from":3,"to":7},{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:30:25.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1579522650 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2990 msgr2=0x7ff0d80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:25.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2990 0x7ff0d80b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400bc10 tx=0x7ff0e400b5f0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1579522650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410af40 msgr2=0x7ff0f4078200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:25.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410af40 0x7ff0f4078200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00b580 tx=0x7ff0dc00f460 comp rx=0 tx=0).stop 2024-05-27T07:30:25.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1579522650 shutdown_connections 2024-05-27T07:30:25.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f410b890 0x7ff0f4078740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4106610 0x7ff0f4077cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff0d80b2990 0x7ff0d80b4ea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 --2- 172.21.15.63:0/1579522650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0f410af40 0x7ff0f4078200 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1579522650 >> 172.21.15.63:0/1579522650 conn(0x7ff0f4102020 msgr2=0x7ff0f410a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:25.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1579522650 shutdown_connections 2024-05-27T07:30:25.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:25.526+0000 7ff0f3e00700 1 -- 172.21.15.63:0/1579522650 wait complete. 2024-05-27T07:30:26.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:25 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1579522650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:30:26.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:25 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1579522650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:30:26.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:25 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1579522650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:30:26.318 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T07:30:26.318 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T07:30:26.761 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:27.199 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:26 smithi086 ceph-mon[34687]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-05-27T07:30:27.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:26 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:30:27.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:26 smithi149 ceph-mon[35408]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-05-27T07:30:27.203 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:26 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:30:27.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:26 smithi063 ceph-mon[31613]: pgmap v452: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-05-27T07:30:27.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:26 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:30:29.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:28 smithi063 ceph-mon[31613]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-27T07:30:29.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:28 smithi086 ceph-mon[34687]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-27T07:30:29.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:28 smithi149 ceph-mon[35408]: pgmap v453: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-05-27T07:30:29.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.636+0000 7f167c800700 1 -- 172.21.15.63:0/3849528259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674076670 msgr2=0x7f1674110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.636+0000 7f167c800700 1 --2- 172.21.15.63:0/3849528259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674076670 0x7f1674110650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1664009990 tx=0x7f1664009ca0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.636+0000 7f167c800700 1 -- 172.21.15.63:0/3849528259 shutdown_connections 2024-05-27T07:30:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.636+0000 7f167c800700 1 --2- 172.21.15.63:0/3849528259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1674110b90 0x7f1674112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.636+0000 7f167c800700 1 --2- 172.21.15.63:0/3849528259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1674075d20 0x7f1674076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.636+0000 7f167c800700 1 --2- 172.21.15.63:0/3849528259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674076670 0x7f1674110650 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.636+0000 7f167c800700 1 -- 172.21.15.63:0/3849528259 >> 172.21.15.63:0/3849528259 conn(0x7f1674100030 msgr2=0x7f1674102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.636+0000 7f167c800700 1 -- 172.21.15.63:0/3849528259 shutdown_connections 2024-05-27T07:30:29.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.636+0000 7f167c800700 1 -- 172.21.15.63:0/3849528259 wait complete. 2024-05-27T07:30:29.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.638+0000 7f167c800700 1 Processor -- start 2024-05-27T07:30:29.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.638+0000 7f167c800700 1 -- start start 2024-05-27T07:30:29.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.638+0000 7f167c800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674075d20 0x7f167419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:29.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.638+0000 7f167c800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1674076670 0x7f167419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:29.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.638+0000 7f167c800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1674110b90 0x7f16741a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:29.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.638+0000 7f167c800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1674116540 con 0x7f1674075d20 2024-05-27T07:30:29.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.638+0000 7f167c800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16741163c0 con 0x7f1674110b90 2024-05-27T07:30:29.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.638+0000 7f167c800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16741166c0 con 0x7f1674076670 2024-05-27T07:30:29.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.639+0000 7f1679e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674075d20 0x7f167419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:29.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.639+0000 7f1679e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674075d20 0x7f167419cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57692/0 (socket says 172.21.15.63:57692) 2024-05-27T07:30:29.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.639+0000 7f1679e00700 1 -- 172.21.15.63:0/2376191707 learned_addr learned my addr 172.21.15.63:0/2376191707 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:29.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.639+0000 7f167a800700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1674110b90 0x7f16741a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:29.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.639+0000 7f1679e00700 1 -- 172.21.15.63:0/2376191707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1674076670 msgr2=0x7f167419d550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:30:29.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.639+0000 7f1679e00700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1674076670 0x7f167419d550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.639+0000 7f1679e00700 1 -- 172.21.15.63:0/2376191707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1674110b90 msgr2=0x7f16741a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:29.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.639+0000 7f1679e00700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1674110b90 0x7f16741a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.639+0000 7f1679e00700 1 -- 172.21.15.63:0/2376191707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16741a1dd0 con 0x7f1674075d20 2024-05-27T07:30:29.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.639+0000 7f167a800700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1674110b90 0x7f16741a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:30:29.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.640+0000 7f1679e00700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674075d20 0x7f167419cfb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f166800d950 tx=0x7f166800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:29.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.640+0000 7f166ea00700 1 -- 172.21.15.63:0/2376191707 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668016070 con 0x7f1674075d20 2024-05-27T07:30:29.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.640+0000 7f167c800700 1 -- 172.21.15.63:0/2376191707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16741a20c0 con 0x7f1674075d20 2024-05-27T07:30:29.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.640+0000 7f166ea00700 1 -- 172.21.15.63:0/2376191707 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1668010460 con 0x7f1674075d20 2024-05-27T07:30:29.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.640+0000 7f167c800700 1 -- 172.21.15.63:0/2376191707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16741a2600 con 0x7f1674075d20 2024-05-27T07:30:29.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.640+0000 7f166ea00700 1 -- 172.21.15.63:0/2376191707 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166800f620 con 0x7f1674075d20 2024-05-27T07:30:29.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.644+0000 7f167c800700 1 -- 172.21.15.63:0/2376191707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1658005330 con 0x7f1674075d20 2024-05-27T07:30:29.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.645+0000 7f166ea00700 1 -- 172.21.15.63:0/2376191707 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1668019070 con 0x7f1674075d20 2024-05-27T07:30:29.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.647+0000 7f166ea00700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f165c0b2910 0x7f165c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:29.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.648+0000 7f166ea00700 1 -- 172.21.15.63:0/2376191707 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f16680e26a0 con 0x7f1674075d20 2024-05-27T07:30:29.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.648+0000 7f1679400700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f165c0b2910 0x7f165c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:29.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.649+0000 7f1679400700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f165c0b2910 0x7f165c0b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f166400b5c0 tx=0x7f1664005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:29.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.651+0000 7f166ea00700 1 -- 172.21.15.63:0/2376191707 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16680a9e70 con 0x7f1674075d20 2024-05-27T07:30:29.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.906+0000 7f167c800700 1 -- 172.21.15.63:0/2376191707 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1658000bc0 con 0x7f165c0b2910 2024-05-27T07:30:29.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.926+0000 7f166ea00700 1 -- 172.21.15.63:0/2376191707 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340394 (secure 0 0 0) 0x7f1658000bc0 con 0x7f165c0b2910 2024-05-27T07:30:29.925 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.933+0000 7f167c800700 1 -- 172.21.15.63:0/2376191707 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f165c0b2910 msgr2=0x7f165c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.933+0000 7f167c800700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f165c0b2910 0x7f165c0b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f166400b5c0 tx=0x7f1664005fb0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.933+0000 7f167c800700 1 -- 172.21.15.63:0/2376191707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674075d20 msgr2=0x7f167419cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.933+0000 7f167c800700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674075d20 0x7f167419cfb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f166800d950 tx=0x7f166800dd10 comp rx=0 tx=0).stop 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.933+0000 7f167c800700 1 -- 172.21.15.63:0/2376191707 shutdown_connections 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.933+0000 7f167c800700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1674110b90 0x7f16741a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.933+0000 7f167c800700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f165c0b2910 0x7f165c0b4e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.933+0000 7f167c800700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674075d20 0x7f167419cfb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.933+0000 7f167c800700 1 --2- 172.21.15.63:0/2376191707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1674076670 0x7f167419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.933+0000 7f167c800700 1 -- 172.21.15.63:0/2376191707 >> 172.21.15.63:0/2376191707 conn(0x7f1674100030 msgr2=0x7f1674100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.934+0000 7f167c800700 1 -- 172.21.15.63:0/2376191707 shutdown_connections 2024-05-27T07:30:29.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:29.934+0000 7f167c800700 1 -- 172.21.15.63:0/2376191707 wait complete. 2024-05-27T07:30:29.929 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-27T07:30:30.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:29 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:29 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:29 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:30.921 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-05-27T07:30:29.827408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662031,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":317,"num_read_kb":264,"num_write":294,"num_write_kb":2273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375992,"kb_used_data":14344,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565832,"statfs":{"total":1151940427776,"available":1151555411968,"internally_reserved":0,"allocated":14688256,"data_stored":8227998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18519,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":74,"num_write":12,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007350"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582368+0000","last_change":"2024-05-27T07:28:24.485151+0000","last_active":"2024-05-27T07:30:05.582368+0000","last_peered":"2024-05-27T07:30:05.582368+0000","last_clean":"2024-05-27T07:30:05.582368+0000","last_became_active":"2024-05-27T07:28:24.483965+0000","last_became_peered":"2024-05-27T07:28:24.483965+0000","last_unstale":"2024-05-27T07:30:05.582368+0000","last_undegraded":"2024-05-27T07:30:05.582368+0000","last_fullsized":"2024-05-27T07:30:05.582368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:18.376755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576218+0000","last_change":"2024-05-27T07:28:24.483608+0000","last_active":"2024-05-27T07:30:05.576218+0000","last_peered":"2024-05-27T07:30:05.576218+0000","last_clean":"2024-05-27T07:30:05.576218+0000","last_became_active":"2024-05-27T07:28:24.479875+0000","last_became_peered":"2024-05-27T07:28:24.479875+0000","last_unstale":"2024-05-27T07:30:05.576218+0000","last_undegraded":"2024-05-27T07:30:05.576218+0000","last_fullsized":"2024-05-27T07:30:05.576218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:30.129708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576001+0000","last_change":"2024-05-27T07:28:24.470104+0000","last_active":"2024-05-27T07:30:05.576001+0000","last_peered":"2024-05-27T07:30:05.576001+0000","last_clean":"2024-05-27T07:30:05.576001+0000","last_became_active":"2024-05-27T07:28:24.469756+0000","last_became_peered":"2024-05-27T07:28:24.469756+0000","last_unstale":"2024-05-27T07:30:05.576001+0000","last_undegraded":"2024-05-27T07:30:05.576001+0000","last_fullsized":"2024-05-27T07:30:05.576001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:36.907691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.538072+0000","last_change":"2024-05-27T07:28:24.472381+0000","last_active":"2024-05-27T07:30:05.538072+0000","last_peered":"2024-05-27T07:30:05.538072+0000","last_clean":"2024-05-27T07:30:05.538072+0000","last_became_active":"2024-05-27T07:28:24.469959+0000","last_became_peered":"2024-05-27T07:28:24.469959+0000","last_unstale":"2024-05-27T07:30:05.538072+0000","last_undegraded":"2024-05-27T07:30:05.538072+0000","last_fullsized":"2024-05-27T07:30:05.538072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:15.267554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580431+0000","last_change":"2024-05-27T07:28:24.474210+0000","last_active":"2024-05-27T07:30:05.580431+0000","last_peered":"2024-05-27T07:30:05.580431+0000","last_clean":"2024-05-27T07:30:05.580431+0000","last_became_active":"2024-05-27T07:28:24.473688+0000","last_became_peered":"2024-05-27T07:28:24.473688+0000","last_unstale":"2024-05-27T07:30:05.580431+0000","last_undegraded":"2024-05-27T07:30:05.580431+0000","last_fullsized":"2024-05-27T07:30:05.580431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:38.733536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578117+0000","last_change":"2024-05-27T07:28:24.484679+0000","last_active":"2024-05-27T07:30:05.578117+0000","last_peered":"2024-05-27T07:30:05.578117+0000","last_clean":"2024-05-27T07:30:05.578117+0000","last_became_active":"2024-05-27T07:28:24.484251+0000","last_became_peered":"2024-05-27T07:28:24.484251+0000","last_unstale":"2024-05-27T07:30:05.578117+0000","last_undegraded":"2024-05-27T07:30:05.578117+0000","last_fullsized":"2024-05-27T07:30:05.578117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:28:09.507871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515699+0000","last_change":"2024-05-27T07:28:24.472694+0000","last_active":"2024-05-27T07:30:05.515699+0000","last_peered":"2024-05-27T07:30:05.515699+0000","last_clean":"2024-05-27T07:30:05.515699+0000","last_became_active":"2024-05-27T07:28:24.472100+0000","last_became_peered":"2024-05-27T07:28:24.472100+0000","last_unstale":"2024-05-27T07:30:05.515699+0000","last_undegraded":"2024-05-27T07:30:05.515699+0000","last_fullsized":"2024-05-27T07:30:05.515699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119432+0000","last_change":"2024-05-27T07:28:24.488577+0000","last_active":"2024-05-27T07:30:05.119432+0000","last_peered":"2024-05-27T07:30:05.119432+0000","last_clean":"2024-05-27T07:30:05.119432+0000","last_became_active":"2024-05-27T07:28:24.488180+0000","last_became_peered":"2024-05-27T07:28:24.488180+0000","last_unstale":"2024-05-27T07:30:05.119432+0000","last_undegraded":"2024-05-27T07:30:05.119432+0000","last_fullsized":"2024-05-27T07:30:05.119432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:17.130058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.123384+0000","last_change":"2024-05-27T07:28:24.480109+0000","last_active":"2024-05-27T07:30:05.123384+0000","last_peered":"2024-05-27T07:30:05.123384+0000","last_clean":"2024-05-27T07:30:05.123384+0000","last_became_active":"2024-05-27T07:28:24.479722+0000","last_became_peered":"2024-05-27T07:28:24.479722+0000","last_unstale":"2024-05-27T07:30:05.123384+0000","last_undegraded":"2024-05-27T07:30:05.123384+0000","last_fullsized":"2024-05-27T07:30:05.123384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:27:36.074612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579053+0000","last_change":"2024-05-27T07:28:24.490389+0000","last_active":"2024-05-27T07:30:05.579053+0000","last_peered":"2024-05-27T07:30:05.579053+0000","last_clean":"2024-05-27T07:30:05.579053+0000","last_became_active":"2024-05-27T07:28:24.488081+0000","last_became_peered":"2024-05-27T07:28:24.488081+0000","last_unstale":"2024-05-27T07:30:05.579053+0000","last_undegraded":"2024-05-27T07:30:05.579053+0000","last_fullsized":"2024-05-27T07:30:05.579053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:06.318235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516771+0000","last_change":"2024-05-27T07:28:24.477062+0000","last_active":"2024-05-27T07:30:05.516771+0000","last_peered":"2024-05-27T07:30:05.516771+0000","last_clean":"2024-05-27T07:30:05.516771+0000","last_became_active":"2024-05-27T07:28:24.475322+0000","last_became_peered":"2024-05-27T07:28:24.475322+0000","last_unstale":"2024-05-27T07:30:05.516771+0000","last_undegraded":"2024-05-27T07:30:05.516771+0000","last_fullsized":"2024-05-27T07:30:05.516771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:30.365295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.126261+0000","last_change":"2024-05-27T07:29:04.901715+0000","last_active":"2024-05-27T07:30:05.126261+0000","last_peered":"2024-05-27T07:30:05.126261+0000","last_clean":"2024-05-27T07:30:05.126261+0000","last_became_active":"2024-05-27T07:29:04.900968+0000","last_became_peered":"2024-05-27T07:29:04.900968+0000","last_unstale":"2024-05-27T07:30:05.126261+0000","last_undegraded":"2024-05-27T07:30:05.126261+0000","last_fullsized":"2024-05-27T07:30:05.126261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:16:34.033701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.404676+0000","last_change":"2024-05-27T07:28:24.477504+0000","last_active":"2024-05-27T07:30:05.404676+0000","last_peered":"2024-05-27T07:30:05.404676+0000","last_clean":"2024-05-27T07:30:05.404676+0000","last_became_active":"2024-05-27T07:28:24.476709+0000","last_became_peered":"2024-05-27T07:28:24.476709+0000","last_unstale":"2024-05-27T07:30:05.404676+0000","last_undegraded":"2024-05-27T07:30:05.404676+0000","last_fullsized":"2024-05-27T07:30:05.404676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:05:48.456347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.575872+0000","last_change":"2024-05-27T07:28:24.485567+0000","last_active":"2024-05-27T07:30:05.575872+0000","last_peered":"2024-05-27T07:30:05.575872+0000","last_clean":"2024-05-27T07:30:05.575872+0000","last_became_active":"2024-05-27T07:28:24.485175+0000","last_became_peered":"2024-05-27T07:28:24.485175+0000","last_unstale":"2024-05-27T07:30:05.575872+0000","last_undegraded":"2024-05-27T07:30:05.575872+0000","last_fullsized":"2024-05-27T07:30:05.575872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:25:30.062813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.575698+0000","last_change":"2024-05-27T07:29:04.900657+0000","last_active":"2024-05-27T07:30:05.575698+0000","last_peered":"2024-05-27T07:30:05.575698+0000","last_clean":"2024-05-27T07:30:05.575698+0000","last_became_active":"2024-05-27T07:29:04.900093+0000","last_became_peered":"2024-05-27T07:29:04.900093+0000","last_unstale":"2024-05-27T07:30:05.575698+0000","last_undegraded":"2024-05-27T07:30:05.575698+0000","last_fullsized":"2024-05-27T07:30:05.575698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:28:04.157825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.121911+0000","last_change":"2024-05-27T07:28:24.470503+0000","last_active":"2024-05-27T07:30:05.121911+0000","last_peered":"2024-05-27T07:30:05.121911+0000","last_clean":"2024-05-27T07:30:05.121911+0000","last_became_active":"2024-05-27T07:28:24.470128+0000","last_became_peered":"2024-05-27T07:28:24.470128+0000","last_unstale":"2024-05-27T07:30:05.121911+0000","last_undegraded":"2024-05-27T07:30:05.121911+0000","last_fullsized":"2024-05-27T07:30:05.121911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:46:43.149635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581135+0000","last_change":"2024-05-27T07:28:24.491382+0000","last_active":"2024-05-27T07:30:05.581135+0000","last_peered":"2024-05-27T07:30:05.581135+0000","last_clean":"2024-05-27T07:30:05.581135+0000","last_became_active":"2024-05-27T07:28:24.486155+0000","last_became_peered":"2024-05-27T07:28:24.486155+0000","last_unstale":"2024-05-27T07:30:05.581135+0000","last_undegraded":"2024-05-27T07:30:05.581135+0000","last_fullsized":"2024-05-27T07:30:05.581135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:48.408573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127068+0000","last_change":"2024-05-27T07:28:24.489140+0000","last_active":"2024-05-27T07:30:05.127068+0000","last_peered":"2024-05-27T07:30:05.127068+0000","last_clean":"2024-05-27T07:30:05.127068+0000","last_became_active":"2024-05-27T07:28:24.488733+0000","last_became_peered":"2024-05-27T07:28:24.488733+0000","last_unstale":"2024-05-27T07:30:05.127068+0000","last_undegraded":"2024-05-27T07:30:05.127068+0000","last_fullsized":"2024-05-27T07:30:05.127068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:02:57.711197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120975+0000","last_change":"2024-05-27T07:28:24.486022+0000","last_active":"2024-05-27T07:30:05.120975+0000","last_peered":"2024-05-27T07:30:05.120975+0000","last_clean":"2024-05-27T07:30:05.120975+0000","last_became_active":"2024-05-27T07:28:24.485639+0000","last_became_peered":"2024-05-27T07:28:24.485639+0000","last_unstale":"2024-05-27T07:30:05.120975+0000","last_undegraded":"2024-05-27T07:30:05.120975+0000","last_fullsized":"2024-05-27T07:30:05.120975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:07.629402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119429+0000","last_change":"2024-05-27T07:28:37.898984+0000","last_active":"2024-05-27T07:30:05.119429+0000","last_peered":"2024-05-27T07:30:05.119429+0000","last_clean":"2024-05-27T07:30:05.119429+0000","last_became_active":"2024-05-27T07:28:37.898132+0000","last_became_peered":"2024-05-27T07:28:37.898132+0000","last_unstale":"2024-05-27T07:30:05.119429+0000","last_undegraded":"2024-05-27T07:30:05.119429+0000","last_fullsized":"2024-05-27T07:30:05.119429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:51.987386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127598+0000","last_change":"2024-05-27T07:28:20.402788+0000","last_active":"2024-05-27T07:30:05.127598+0000","last_peered":"2024-05-27T07:30:05.127598+0000","last_clean":"2024-05-27T07:30:05.127598+0000","last_became_active":"2024-05-27T07:28:20.402041+0000","last_became_peered":"2024-05-27T07:28:20.402041+0000","last_unstale":"2024-05-27T07:30:05.127598+0000","last_undegraded":"2024-05-27T07:30:05.127598+0000","last_fullsized":"2024-05-27T07:30:05.127598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:44.421979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.121315+0000","last_change":"2024-05-27T07:28:24.485193+0000","last_active":"2024-05-27T07:30:05.121315+0000","last_peered":"2024-05-27T07:30:05.121315+0000","last_clean":"2024-05-27T07:30:05.121315+0000","last_became_active":"2024-05-27T07:28:24.484086+0000","last_became_peered":"2024-05-27T07:28:24.484086+0000","last_unstale":"2024-05-27T07:30:05.121315+0000","last_undegraded":"2024-05-27T07:30:05.121315+0000","last_fullsized":"2024-05-27T07:30:05.121315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:10.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536402+0000","last_change":"2024-05-27T07:28:37.902602+0000","last_active":"2024-05-27T07:30:05.536402+0000","last_peered":"2024-05-27T07:30:05.536402+0000","last_clean":"2024-05-27T07:30:05.536402+0000","last_became_active":"2024-05-27T07:28:37.902120+0000","last_became_peered":"2024-05-27T07:28:37.902120+0000","last_unstale":"2024-05-27T07:30:05.536402+0000","last_undegraded":"2024-05-27T07:30:05.536402+0000","last_fullsized":"2024-05-27T07:30:05.536402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:21.209851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122003+0000","last_change":"2024-05-27T07:28:20.465500+0000","last_active":"2024-05-27T07:30:05.122003+0000","last_peered":"2024-05-27T07:30:05.122003+0000","last_clean":"2024-05-27T07:30:05.122003+0000","last_became_active":"2024-05-27T07:28:20.464951+0000","last_became_peered":"2024-05-27T07:28:20.464951+0000","last_unstale":"2024-05-27T07:30:05.122003+0000","last_undegraded":"2024-05-27T07:30:05.122003+0000","last_fullsized":"2024-05-27T07:30:05.122003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:23:25.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576522+0000","last_change":"2024-05-27T07:28:24.467655+0000","last_active":"2024-05-27T07:30:05.576522+0000","last_peered":"2024-05-27T07:30:05.576522+0000","last_clean":"2024-05-27T07:30:05.576522+0000","last_became_active":"2024-05-27T07:28:24.467194+0000","last_became_peered":"2024-05-27T07:28:24.467194+0000","last_unstale":"2024-05-27T07:30:05.576522+0000","last_undegraded":"2024-05-27T07:30:05.576522+0000","last_fullsized":"2024-05-27T07:30:05.576522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:38.024514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536716+0000","last_change":"2024-05-27T07:29:04.975679+0000","last_active":"2024-05-27T07:30:05.536716+0000","last_peered":"2024-05-27T07:30:05.536716+0000","last_clean":"2024-05-27T07:30:05.536716+0000","last_became_active":"2024-05-27T07:29:04.974323+0000","last_became_peered":"2024-05-27T07:29:04.974323+0000","last_unstale":"2024-05-27T07:30:05.536716+0000","last_undegraded":"2024-05-27T07:30:05.536716+0000","last_fullsized":"2024-05-27T07:30:05.536716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:50.489737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584247+0000","last_change":"2024-05-27T07:28:20.408117+0000","last_active":"2024-05-27T07:30:05.584247+0000","last_peered":"2024-05-27T07:30:05.584247+0000","last_clean":"2024-05-27T07:30:05.584247+0000","last_became_active":"2024-05-27T07:28:20.407723+0000","last_became_peered":"2024-05-27T07:28:20.407723+0000","last_unstale":"2024-05-27T07:30:05.584247+0000","last_undegraded":"2024-05-27T07:30:05.584247+0000","last_fullsized":"2024-05-27T07:30:05.584247+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:20:31.376656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122481+0000","last_change":"2024-05-27T07:28:24.490269+0000","last_active":"2024-05-27T07:30:05.122481+0000","last_peered":"2024-05-27T07:30:05.122481+0000","last_clean":"2024-05-27T07:30:05.122481+0000","last_became_active":"2024-05-27T07:28:24.488849+0000","last_became_peered":"2024-05-27T07:28:24.488849+0000","last_unstale":"2024-05-27T07:30:05.122481+0000","last_undegraded":"2024-05-27T07:30:05.122481+0000","last_fullsized":"2024-05-27T07:30:05.122481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:00.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536216+0000","last_change":"2024-05-27T07:28:37.908658+0000","last_active":"2024-05-27T07:30:05.536216+0000","last_peered":"2024-05-27T07:30:05.536216+0000","last_clean":"2024-05-27T07:30:05.536216+0000","last_became_active":"2024-05-27T07:28:37.908056+0000","last_became_peered":"2024-05-27T07:28:37.908056+0000","last_unstale":"2024-05-27T07:30:05.536216+0000","last_undegraded":"2024-05-27T07:30:05.536216+0000","last_fullsized":"2024-05-27T07:30:05.536216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:31.853606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583453+0000","last_change":"2024-05-27T07:28:20.410944+0000","last_active":"2024-05-27T07:30:05.583453+0000","last_peered":"2024-05-27T07:30:05.583453+0000","last_clean":"2024-05-27T07:30:05.583453+0000","last_became_active":"2024-05-27T07:28:20.409789+0000","last_became_peered":"2024-05-27T07:28:20.409789+0000","last_unstale":"2024-05-27T07:30:05.583453+0000","last_undegraded":"2024-05-27T07:30:05.583453+0000","last_fullsized":"2024-05-27T07:30:05.583453+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:53:34.962397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583982+0000","last_change":"2024-05-27T07:28:24.482828+0000","last_active":"2024-05-27T07:30:05.583982+0000","last_peered":"2024-05-27T07:30:05.583982+0000","last_clean":"2024-05-27T07:30:05.583982+0000","last_became_active":"2024-05-27T07:28:24.482475+0000","last_became_peered":"2024-05-27T07:28:24.482475+0000","last_unstale":"2024-05-27T07:30:05.583982+0000","last_undegraded":"2024-05-27T07:30:05.583982+0000","last_fullsized":"2024-05-27T07:30:05.583982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:53:32.064441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120474+0000","last_change":"2024-05-27T07:29:04.905734+0000","last_active":"2024-05-27T07:30:05.120474+0000","last_peered":"2024-05-27T07:30:05.120474+0000","last_clean":"2024-05-27T07:30:05.120474+0000","last_became_active":"2024-05-27T07:29:04.902387+0000","last_became_peered":"2024-05-27T07:29:04.902387+0000","last_unstale":"2024-05-27T07:30:05.120474+0000","last_undegraded":"2024-05-27T07:30:05.120474+0000","last_fullsized":"2024-05-27T07:30:05.120474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:35.993193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":84,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:24.130785+0000","last_change":"2024-05-27T07:28:20.412258+0000","last_active":"2024-05-27T07:30:24.130785+0000","last_peered":"2024-05-27T07:30:24.130785+0000","last_clean":"2024-05-27T07:30:24.130785+0000","last_became_active":"2024-05-27T07:28:20.411764+0000","last_became_peered":"2024-05-27T07:28:20.411764+0000","last_unstale":"2024-05-27T07:30:24.130785+0000","last_undegraded":"2024-05-27T07:30:24.130785+0000","last_fullsized":"2024-05-27T07:30:24.130785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:38.462887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581379+0000","last_change":"2024-05-27T07:28:24.482179+0000","last_active":"2024-05-27T07:30:05.581379+0000","last_peered":"2024-05-27T07:30:05.581379+0000","last_clean":"2024-05-27T07:30:05.581379+0000","last_became_active":"2024-05-27T07:28:24.480682+0000","last_became_peered":"2024-05-27T07:28:24.480682+0000","last_unstale":"2024-05-27T07:30:05.581379+0000","last_undegraded":"2024-05-27T07:30:05.581379+0000","last_fullsized":"2024-05-27T07:30:05.581379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:51.403862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119762+0000","last_change":"2024-05-27T07:28:37.903716+0000","last_active":"2024-05-27T07:30:05.119762+0000","last_peered":"2024-05-27T07:30:05.119762+0000","last_clean":"2024-05-27T07:30:05.119762+0000","last_became_active":"2024-05-27T07:28:37.901083+0000","last_became_peered":"2024-05-27T07:28:37.901083+0000","last_unstale":"2024-05-27T07:30:05.119762+0000","last_undegraded":"2024-05-27T07:30:05.119762+0000","last_fullsized":"2024-05-27T07:30:05.119762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:44:11.606711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578100+0000","last_change":"2024-05-27T07:28:20.392441+0000","last_active":"2024-05-27T07:30:05.578100+0000","last_peered":"2024-05-27T07:30:05.578100+0000","last_clean":"2024-05-27T07:30:05.578100+0000","last_became_active":"2024-05-27T07:28:20.390983+0000","last_became_peered":"2024-05-27T07:28:20.390983+0000","last_unstale":"2024-05-27T07:30:05.578100+0000","last_undegraded":"2024-05-27T07:30:05.578100+0000","last_fullsized":"2024-05-27T07:30:05.578100+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:33:30.338043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.520219+0000","last_change":"2024-05-27T07:30:05.518915+0000","last_active":"2024-05-27T07:30:05.520219+0000","last_peered":"2024-05-27T07:30:05.520219+0000","last_clean":"2024-05-27T07:30:05.520219+0000","last_became_active":"2024-05-27T07:30:05.517894+0000","last_became_peered":"2024-05-27T07:30:05.517894+0000","last_unstale":"2024-05-27T07:30:05.520219+0000","last_undegraded":"2024-05-27T07:30:05.520219+0000","last_fullsized":"2024-05-27T07:30:05.520219+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:51.353713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,7],"acting":[9,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580815+0000","last_change":"2024-05-27T07:28:37.913352+0000","last_active":"2024-05-27T07:30:05.580815+0000","last_peered":"2024-05-27T07:30:05.580815+0000","last_clean":"2024-05-27T07:30:05.580815+0000","last_became_active":"2024-05-27T07:28:37.913027+0000","last_became_peered":"2024-05-27T07:28:37.913027+0000","last_unstale":"2024-05-27T07:30:05.580815+0000","last_undegraded":"2024-05-27T07:30:05.580815+0000","last_fullsized":"2024-05-27T07:30:05.580815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:03.485594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.404529+0000","last_change":"2024-05-27T07:28:20.406094+0000","last_active":"2024-05-27T07:30:05.404529+0000","last_peered":"2024-05-27T07:30:05.404529+0000","last_clean":"2024-05-27T07:30:05.404529+0000","last_became_active":"2024-05-27T07:28:20.405641+0000","last_became_peered":"2024-05-27T07:28:20.405641+0000","last_unstale":"2024-05-27T07:30:05.404529+0000","last_undegraded":"2024-05-27T07:30:05.404529+0000","last_fullsized":"2024-05-27T07:30:05.404529+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:16.598743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119613+0000","last_change":"2024-05-27T07:28:24.474696+0000","last_active":"2024-05-27T07:30:05.119613+0000","last_peered":"2024-05-27T07:30:05.119613+0000","last_clean":"2024-05-27T07:30:05.119613+0000","last_became_active":"2024-05-27T07:28:24.474319+0000","last_became_peered":"2024-05-27T07:28:24.474319+0000","last_unstale":"2024-05-27T07:30:05.119613+0000","last_undegraded":"2024-05-27T07:30:05.119613+0000","last_fullsized":"2024-05-27T07:30:05.119613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:53.010898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576072+0000","last_change":"2024-05-27T07:28:37.910386+0000","last_active":"2024-05-27T07:30:05.576072+0000","last_peered":"2024-05-27T07:30:05.576072+0000","last_clean":"2024-05-27T07:30:05.576072+0000","last_became_active":"2024-05-27T07:28:37.909941+0000","last_became_peered":"2024-05-27T07:28:37.909941+0000","last_unstale":"2024-05-27T07:30:05.576072+0000","last_undegraded":"2024-05-27T07:30:05.576072+0000","last_fullsized":"2024-05-27T07:30:05.576072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:15.820318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581844+0000","last_change":"2024-05-27T07:28:20.411855+0000","last_active":"2024-05-27T07:30:05.581844+0000","last_peered":"2024-05-27T07:30:05.581844+0000","last_clean":"2024-05-27T07:30:05.581844+0000","last_became_active":"2024-05-27T07:28:20.411169+0000","last_became_peered":"2024-05-27T07:28:20.411169+0000","last_unstale":"2024-05-27T07:30:05.581844+0000","last_undegraded":"2024-05-27T07:30:05.581844+0000","last_fullsized":"2024-05-27T07:30:05.581844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:37.278507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.125456+0000","last_change":"2024-05-27T07:28:24.485799+0000","last_active":"2024-05-27T07:30:05.125456+0000","last_peered":"2024-05-27T07:30:05.125456+0000","last_clean":"2024-05-27T07:30:05.125456+0000","last_became_active":"2024-05-27T07:28:24.485395+0000","last_became_peered":"2024-05-27T07:28:24.485395+0000","last_unstale":"2024-05-27T07:30:05.125456+0000","last_undegraded":"2024-05-27T07:30:05.125456+0000","last_fullsized":"2024-05-27T07:30:05.125456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:11.178129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119918+0000","last_change":"2024-05-27T07:28:37.898929+0000","last_active":"2024-05-27T07:30:05.119918+0000","last_peered":"2024-05-27T07:30:05.119918+0000","last_clean":"2024-05-27T07:30:05.119918+0000","last_became_active":"2024-05-27T07:28:37.898371+0000","last_became_peered":"2024-05-27T07:28:37.898371+0000","last_unstale":"2024-05-27T07:30:05.119918+0000","last_undegraded":"2024-05-27T07:30:05.119918+0000","last_fullsized":"2024-05-27T07:30:05.119918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:48.867729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":83,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:27.138496+0000","last_change":"2024-05-27T07:28:20.407117+0000","last_active":"2024-05-27T07:30:27.138496+0000","last_peered":"2024-05-27T07:30:27.138496+0000","last_clean":"2024-05-27T07:30:27.138496+0000","last_became_active":"2024-05-27T07:28:20.403562+0000","last_became_peered":"2024-05-27T07:28:20.403562+0000","last_unstale":"2024-05-27T07:30:27.138496+0000","last_undegraded":"2024-05-27T07:30:27.138496+0000","last_fullsized":"2024-05-27T07:30:27.138496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:25.903886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.123411+0000","last_change":"2024-05-27T07:28:24.489771+0000","last_active":"2024-05-27T07:30:05.123411+0000","last_peered":"2024-05-27T07:30:05.123411+0000","last_clean":"2024-05-27T07:30:05.123411+0000","last_became_active":"2024-05-27T07:28:24.489163+0000","last_became_peered":"2024-05-27T07:28:24.489163+0000","last_unstale":"2024-05-27T07:30:05.123411+0000","last_undegraded":"2024-05-27T07:30:05.123411+0000","last_fullsized":"2024-05-27T07:30:05.123411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:08.053634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.123519+0000","last_change":"2024-05-27T07:28:37.914598+0000","last_active":"2024-05-27T07:30:05.123519+0000","last_peered":"2024-05-27T07:30:05.123519+0000","last_clean":"2024-05-27T07:30:05.123519+0000","last_became_active":"2024-05-27T07:28:37.913916+0000","last_became_peered":"2024-05-27T07:28:37.913916+0000","last_unstale":"2024-05-27T07:30:05.123519+0000","last_undegraded":"2024-05-27T07:30:05.123519+0000","last_fullsized":"2024-05-27T07:30:05.123519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:01.475789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516991+0000","last_change":"2024-05-27T07:29:04.901366+0000","last_active":"2024-05-27T07:30:05.516991+0000","last_peered":"2024-05-27T07:30:05.516991+0000","last_clean":"2024-05-27T07:30:05.516991+0000","last_became_active":"2024-05-27T07:29:04.900355+0000","last_became_peered":"2024-05-27T07:29:04.900355+0000","last_unstale":"2024-05-27T07:30:05.516991+0000","last_undegraded":"2024-05-27T07:30:05.516991+0000","last_fullsized":"2024-05-27T07:30:05.516991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:10:03.349345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576630+0000","last_change":"2024-05-27T07:28:24.477771+0000","last_active":"2024-05-27T07:30:05.576630+0000","last_peered":"2024-05-27T07:30:05.576630+0000","last_clean":"2024-05-27T07:30:05.576630+0000","last_became_active":"2024-05-27T07:28:24.474179+0000","last_became_peered":"2024-05-27T07:28:24.474179+0000","last_unstale":"2024-05-27T07:30:05.576630+0000","last_undegraded":"2024-05-27T07:30:05.576630+0000","last_fullsized":"2024-05-27T07:30:05.576630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:08:04.218264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537214+0000","last_change":"2024-05-27T07:29:04.896583+0000","last_active":"2024-05-27T07:30:05.537214+0000","last_peered":"2024-05-27T07:30:05.537214+0000","last_clean":"2024-05-27T07:30:05.537214+0000","last_became_active":"2024-05-27T07:29:04.894118+0000","last_became_peered":"2024-05-27T07:29:04.894118+0000","last_unstale":"2024-05-27T07:30:05.537214+0000","last_undegraded":"2024-05-27T07:30:05.537214+0000","last_fullsized":"2024-05-27T07:30:05.537214+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:17.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583441+0000","last_change":"2024-05-27T07:28:20.466851+0000","last_active":"2024-05-27T07:30:05.583441+0000","last_peered":"2024-05-27T07:30:05.583441+0000","last_clean":"2024-05-27T07:30:05.583441+0000","last_became_active":"2024-05-27T07:28:20.465133+0000","last_became_peered":"2024-05-27T07:28:20.465133+0000","last_unstale":"2024-05-27T07:30:05.583441+0000","last_undegraded":"2024-05-27T07:30:05.583441+0000","last_fullsized":"2024-05-27T07:30:05.583441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:11.025588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.123045+0000","last_change":"2024-05-27T07:28:24.469924+0000","last_active":"2024-05-27T07:30:05.123045+0000","last_peered":"2024-05-27T07:30:05.123045+0000","last_clean":"2024-05-27T07:30:05.123045+0000","last_became_active":"2024-05-27T07:28:24.469559+0000","last_became_peered":"2024-05-27T07:28:24.469559+0000","last_unstale":"2024-05-27T07:30:05.123045+0000","last_undegraded":"2024-05-27T07:30:05.123045+0000","last_fullsized":"2024-05-27T07:30:05.123045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:57.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.124945+0000","last_change":"2024-05-27T07:28:37.904549+0000","last_active":"2024-05-27T07:30:05.124945+0000","last_peered":"2024-05-27T07:30:05.124945+0000","last_clean":"2024-05-27T07:30:05.124945+0000","last_became_active":"2024-05-27T07:28:37.903882+0000","last_became_peered":"2024-05-27T07:28:37.903882+0000","last_unstale":"2024-05-27T07:30:05.124945+0000","last_undegraded":"2024-05-27T07:30:05.124945+0000","last_fullsized":"2024-05-27T07:30:05.124945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:41.607641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":161,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:28.159367+0000","last_change":"2024-05-27T07:28:20.410025+0000","last_active":"2024-05-27T07:30:28.159367+0000","last_peered":"2024-05-27T07:30:28.159367+0000","last_clean":"2024-05-27T07:30:28.159367+0000","last_became_active":"2024-05-27T07:28:20.409115+0000","last_became_peered":"2024-05-27T07:28:20.409115+0000","last_unstale":"2024-05-27T07:30:28.159367+0000","last_undegraded":"2024-05-27T07:30:28.159367+0000","last_fullsized":"2024-05-27T07:30:28.159367+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:56.834059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67819,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":83,"num_read_kb":108,"num_write":35,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584860+0000","last_change":"2024-05-27T07:28:24.469609+0000","last_active":"2024-05-27T07:30:05.584860+0000","last_peered":"2024-05-27T07:30:05.584860+0000","last_clean":"2024-05-27T07:30:05.584860+0000","last_became_active":"2024-05-27T07:28:24.469192+0000","last_became_peered":"2024-05-27T07:28:24.469192+0000","last_unstale":"2024-05-27T07:30:05.584860+0000","last_undegraded":"2024-05-27T07:30:05.584860+0000","last_fullsized":"2024-05-27T07:30:05.584860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:32:59.006298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584929+0000","last_change":"2024-05-27T07:28:37.900745+0000","last_active":"2024-05-27T07:30:05.584929+0000","last_peered":"2024-05-27T07:30:05.584929+0000","last_clean":"2024-05-27T07:30:05.584929+0000","last_became_active":"2024-05-27T07:28:37.900205+0000","last_became_peered":"2024-05-27T07:28:37.900205+0000","last_unstale":"2024-05-27T07:30:05.584929+0000","last_undegraded":"2024-05-27T07:30:05.584929+0000","last_fullsized":"2024-05-27T07:30:05.584929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:52:32.235201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120148+0000","last_change":"2024-05-27T07:28:20.413380+0000","last_active":"2024-05-27T07:30:05.120148+0000","last_peered":"2024-05-27T07:30:05.120148+0000","last_clean":"2024-05-27T07:30:05.120148+0000","last_became_active":"2024-05-27T07:28:20.412839+0000","last_became_peered":"2024-05-27T07:28:20.412839+0000","last_unstale":"2024-05-27T07:30:05.120148+0000","last_undegraded":"2024-05-27T07:30:05.120148+0000","last_fullsized":"2024-05-27T07:30:05.120148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:57.481203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.405393+0000","last_change":"2024-05-27T07:28:24.474228+0000","last_active":"2024-05-27T07:30:05.405393+0000","last_peered":"2024-05-27T07:30:05.405393+0000","last_clean":"2024-05-27T07:30:05.405393+0000","last_became_active":"2024-05-27T07:28:24.473833+0000","last_became_peered":"2024-05-27T07:28:24.473833+0000","last_unstale":"2024-05-27T07:30:05.405393+0000","last_undegraded":"2024-05-27T07:30:05.405393+0000","last_fullsized":"2024-05-27T07:30:05.405393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:52.456730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580276+0000","last_change":"2024-05-27T07:28:37.917630+0000","last_active":"2024-05-27T07:30:05.580276+0000","last_peered":"2024-05-27T07:30:05.580276+0000","last_clean":"2024-05-27T07:30:05.580276+0000","last_became_active":"2024-05-27T07:28:37.917253+0000","last_became_peered":"2024-05-27T07:28:37.917253+0000","last_unstale":"2024-05-27T07:30:05.580276+0000","last_undegraded":"2024-05-27T07:30:05.580276+0000","last_fullsized":"2024-05-27T07:30:05.580276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:05.069946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584581+0000","last_change":"2024-05-27T07:28:20.408988+0000","last_active":"2024-05-27T07:30:05.584581+0000","last_peered":"2024-05-27T07:30:05.584581+0000","last_clean":"2024-05-27T07:30:05.584581+0000","last_became_active":"2024-05-27T07:28:20.408313+0000","last_became_peered":"2024-05-27T07:28:20.408313+0000","last_unstale":"2024-05-27T07:30:05.584581+0000","last_undegraded":"2024-05-27T07:30:05.584581+0000","last_fullsized":"2024-05-27T07:30:05.584581+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:17.480412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.517051+0000","last_change":"2024-05-27T07:28:24.480096+0000","last_active":"2024-05-27T07:30:05.517051+0000","last_peered":"2024-05-27T07:30:05.517051+0000","last_clean":"2024-05-27T07:30:05.517051+0000","last_became_active":"2024-05-27T07:28:24.477988+0000","last_became_peered":"2024-05-27T07:28:24.477988+0000","last_unstale":"2024-05-27T07:30:05.517051+0000","last_undegraded":"2024-05-27T07:30:05.517051+0000","last_fullsized":"2024-05-27T07:30:05.517051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:13:59.860458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127016+0000","last_change":"2024-05-27T07:28:37.917518+0000","last_active":"2024-05-27T07:30:05.127016+0000","last_peered":"2024-05-27T07:30:05.127016+0000","last_clean":"2024-05-27T07:30:05.127016+0000","last_became_active":"2024-05-27T07:28:37.916971+0000","last_became_peered":"2024-05-27T07:28:37.916971+0000","last_unstale":"2024-05-27T07:30:05.127016+0000","last_undegraded":"2024-05-27T07:30:05.127016+0000","last_fullsized":"2024-05-27T07:30:05.127016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:17:53.067747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":82,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:24.133194+0000","last_change":"2024-05-27T07:28:20.416766+0000","last_active":"2024-05-27T07:30:24.133194+0000","last_peered":"2024-05-27T07:30:24.133194+0000","last_clean":"2024-05-27T07:30:24.133194+0000","last_became_active":"2024-05-27T07:28:20.416297+0000","last_became_peered":"2024-05-27T07:28:20.416297+0000","last_unstale":"2024-05-27T07:30:24.133194+0000","last_undegraded":"2024-05-27T07:30:24.133194+0000","last_fullsized":"2024-05-27T07:30:24.133194+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:02.841494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515837+0000","last_change":"2024-05-27T07:28:24.486869+0000","last_active":"2024-05-27T07:30:05.515837+0000","last_peered":"2024-05-27T07:30:05.515837+0000","last_clean":"2024-05-27T07:30:05.515837+0000","last_became_active":"2024-05-27T07:28:24.486451+0000","last_became_peered":"2024-05-27T07:28:24.486451+0000","last_unstale":"2024-05-27T07:30:05.515837+0000","last_undegraded":"2024-05-27T07:30:05.515837+0000","last_fullsized":"2024-05-27T07:30:05.515837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:10.722659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406667+0000","last_change":"2024-05-27T07:29:04.901632+0000","last_active":"2024-05-27T07:30:05.406667+0000","last_peered":"2024-05-27T07:30:05.406667+0000","last_clean":"2024-05-27T07:30:05.406667+0000","last_became_active":"2024-05-27T07:29:04.900203+0000","last_became_peered":"2024-05-27T07:29:04.900203+0000","last_unstale":"2024-05-27T07:30:05.406667+0000","last_undegraded":"2024-05-27T07:30:05.406667+0000","last_fullsized":"2024-05-27T07:30:05.406667+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:57.834889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579936+0000","last_change":"2024-05-27T07:28:20.468162+0000","last_active":"2024-05-27T07:30:05.579936+0000","last_peered":"2024-05-27T07:30:05.579936+0000","last_clean":"2024-05-27T07:30:05.579936+0000","last_became_active":"2024-05-27T07:28:20.465226+0000","last_became_peered":"2024-05-27T07:28:20.465226+0000","last_unstale":"2024-05-27T07:30:05.579936+0000","last_undegraded":"2024-05-27T07:30:05.579936+0000","last_fullsized":"2024-05-27T07:30:05.579936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:21:34.110356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582781+0000","last_change":"2024-05-27T07:28:24.485053+0000","last_active":"2024-05-27T07:30:05.582781+0000","last_peered":"2024-05-27T07:30:05.582781+0000","last_clean":"2024-05-27T07:30:05.582781+0000","last_became_active":"2024-05-27T07:28:24.484391+0000","last_became_peered":"2024-05-27T07:28:24.484391+0000","last_unstale":"2024-05-27T07:30:05.582781+0000","last_undegraded":"2024-05-27T07:30:05.582781+0000","last_fullsized":"2024-05-27T07:30:05.582781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:05.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582670+0000","last_change":"2024-05-27T07:28:37.904731+0000","last_active":"2024-05-27T07:30:05.582670+0000","last_peered":"2024-05-27T07:30:05.582670+0000","last_clean":"2024-05-27T07:30:05.582670+0000","last_became_active":"2024-05-27T07:28:37.904284+0000","last_became_peered":"2024-05-27T07:28:37.904284+0000","last_unstale":"2024-05-27T07:30:05.582670+0000","last_undegraded":"2024-05-27T07:30:05.582670+0000","last_fullsized":"2024-05-27T07:30:05.582670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:23.198465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.585569+0000","last_change":"2024-05-27T07:28:20.406829+0000","last_active":"2024-05-27T07:30:05.585569+0000","last_peered":"2024-05-27T07:30:05.585569+0000","last_clean":"2024-05-27T07:30:05.585569+0000","last_became_active":"2024-05-27T07:28:20.405497+0000","last_became_peered":"2024-05-27T07:28:20.405497+0000","last_unstale":"2024-05-27T07:30:05.585569+0000","last_undegraded":"2024-05-27T07:30:05.585569+0000","last_fullsized":"2024-05-27T07:30:05.585569+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:14.225630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581920+0000","last_change":"2024-05-27T07:28:24.488023+0000","last_active":"2024-05-27T07:30:05.581920+0000","last_peered":"2024-05-27T07:30:05.581920+0000","last_clean":"2024-05-27T07:30:05.581920+0000","last_became_active":"2024-05-27T07:28:24.487652+0000","last_became_peered":"2024-05-27T07:28:24.487652+0000","last_unstale":"2024-05-27T07:30:05.581920+0000","last_undegraded":"2024-05-27T07:30:05.581920+0000","last_fullsized":"2024-05-27T07:30:05.581920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:25.951157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581074+0000","last_change":"2024-05-27T07:28:37.968034+0000","last_active":"2024-05-27T07:30:05.581074+0000","last_peered":"2024-05-27T07:30:05.581074+0000","last_clean":"2024-05-27T07:30:05.581074+0000","last_became_active":"2024-05-27T07:28:37.967127+0000","last_became_peered":"2024-05-27T07:28:37.967127+0000","last_unstale":"2024-05-27T07:30:05.581074+0000","last_undegraded":"2024-05-27T07:30:05.581074+0000","last_fullsized":"2024-05-27T07:30:05.581074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:49.353423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.126685+0000","last_change":"2024-05-27T07:28:20.465782+0000","last_active":"2024-05-27T07:30:05.126685+0000","last_peered":"2024-05-27T07:30:05.126685+0000","last_clean":"2024-05-27T07:30:05.126685+0000","last_became_active":"2024-05-27T07:28:20.465134+0000","last_became_peered":"2024-05-27T07:28:20.465134+0000","last_unstale":"2024-05-27T07:30:05.126685+0000","last_undegraded":"2024-05-27T07:30:05.126685+0000","last_fullsized":"2024-05-27T07:30:05.126685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:17.992092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579036+0000","last_change":"2024-05-27T07:28:24.484691+0000","last_active":"2024-05-27T07:30:05.579036+0000","last_peered":"2024-05-27T07:30:05.579036+0000","last_clean":"2024-05-27T07:30:05.579036+0000","last_became_active":"2024-05-27T07:28:24.484249+0000","last_became_peered":"2024-05-27T07:28:24.484249+0000","last_unstale":"2024-05-27T07:30:05.579036+0000","last_undegraded":"2024-05-27T07:30:05.579036+0000","last_fullsized":"2024-05-27T07:30:05.579036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:25.753276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.405759+0000","last_change":"2024-05-27T07:28:37.907849+0000","last_active":"2024-05-27T07:30:05.405759+0000","last_peered":"2024-05-27T07:30:05.405759+0000","last_clean":"2024-05-27T07:30:05.405759+0000","last_became_active":"2024-05-27T07:28:37.907105+0000","last_became_peered":"2024-05-27T07:28:37.907105+0000","last_unstale":"2024-05-27T07:30:05.405759+0000","last_undegraded":"2024-05-27T07:30:05.405759+0000","last_fullsized":"2024-05-27T07:30:05.405759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:54.075240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516795+0000","last_change":"2024-05-27T07:28:20.411302+0000","last_active":"2024-05-27T07:30:05.516795+0000","last_peered":"2024-05-27T07:30:05.516795+0000","last_clean":"2024-05-27T07:30:05.516795+0000","last_became_active":"2024-05-27T07:28:20.409144+0000","last_became_peered":"2024-05-27T07:28:20.409144+0000","last_unstale":"2024-05-27T07:30:05.516795+0000","last_undegraded":"2024-05-27T07:30:05.516795+0000","last_fullsized":"2024-05-27T07:30:05.516795+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:49.985962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578341+0000","last_change":"2024-05-27T07:28:24.487742+0000","last_active":"2024-05-27T07:30:05.578341+0000","last_peered":"2024-05-27T07:30:05.578341+0000","last_clean":"2024-05-27T07:30:05.578341+0000","last_became_active":"2024-05-27T07:28:24.485279+0000","last_became_peered":"2024-05-27T07:28:24.485279+0000","last_unstale":"2024-05-27T07:30:05.578341+0000","last_undegraded":"2024-05-27T07:30:05.578341+0000","last_fullsized":"2024-05-27T07:30:05.578341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:15.049572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406610+0000","last_change":"2024-05-27T07:28:37.968780+0000","last_active":"2024-05-27T07:30:05.406610+0000","last_peered":"2024-05-27T07:30:05.406610+0000","last_clean":"2024-05-27T07:30:05.406610+0000","last_became_active":"2024-05-27T07:28:37.968052+0000","last_became_peered":"2024-05-27T07:28:37.968052+0000","last_unstale":"2024-05-27T07:30:05.406610+0000","last_undegraded":"2024-05-27T07:30:05.406610+0000","last_fullsized":"2024-05-27T07:30:05.406610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:51:26.787969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583127+0000","last_change":"2024-05-27T07:28:24.485280+0000","last_active":"2024-05-27T07:30:05.583127+0000","last_peered":"2024-05-27T07:30:05.583127+0000","last_clean":"2024-05-27T07:30:05.583127+0000","last_became_active":"2024-05-27T07:28:24.484882+0000","last_became_peered":"2024-05-27T07:28:24.484882+0000","last_unstale":"2024-05-27T07:30:05.583127+0000","last_undegraded":"2024-05-27T07:30:05.583127+0000","last_fullsized":"2024-05-27T07:30:05.583127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:28.643857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":81,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:25.134867+0000","last_change":"2024-05-27T07:28:20.462593+0000","last_active":"2024-05-27T07:30:25.134867+0000","last_peered":"2024-05-27T07:30:25.134867+0000","last_clean":"2024-05-27T07:30:25.134867+0000","last_became_active":"2024-05-27T07:28:20.461128+0000","last_became_peered":"2024-05-27T07:28:20.461128+0000","last_unstale":"2024-05-27T07:30:25.134867+0000","last_undegraded":"2024-05-27T07:30:25.134867+0000","last_fullsized":"2024-05-27T07:30:25.134867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:24:42.013315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516454+0000","last_change":"2024-05-27T07:28:37.911546+0000","last_active":"2024-05-27T07:30:05.516454+0000","last_peered":"2024-05-27T07:30:05.516454+0000","last_clean":"2024-05-27T07:30:05.516454+0000","last_became_active":"2024-05-27T07:28:37.910777+0000","last_became_peered":"2024-05-27T07:28:37.910777+0000","last_unstale":"2024-05-27T07:30:05.516454+0000","last_undegraded":"2024-05-27T07:30:05.516454+0000","last_fullsized":"2024-05-27T07:30:05.516454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:13.344317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120079+0000","last_change":"2024-05-27T07:28:24.483946+0000","last_active":"2024-05-27T07:30:05.120079+0000","last_peered":"2024-05-27T07:30:05.120079+0000","last_clean":"2024-05-27T07:30:05.120079+0000","last_became_active":"2024-05-27T07:28:24.481424+0000","last_became_peered":"2024-05-27T07:28:24.481424+0000","last_unstale":"2024-05-27T07:30:05.120079+0000","last_undegraded":"2024-05-27T07:30:05.120079+0000","last_fullsized":"2024-05-27T07:30:05.120079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:08.254046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536932+0000","last_change":"2024-05-27T07:28:20.414250+0000","last_active":"2024-05-27T07:30:05.536932+0000","last_peered":"2024-05-27T07:30:05.536932+0000","last_clean":"2024-05-27T07:30:05.536932+0000","last_became_active":"2024-05-27T07:28:20.413676+0000","last_became_peered":"2024-05-27T07:28:20.413676+0000","last_unstale":"2024-05-27T07:30:05.536932+0000","last_undegraded":"2024-05-27T07:30:05.536932+0000","last_fullsized":"2024-05-27T07:30:05.536932+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:10.226771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515428+0000","last_change":"2024-05-27T07:26:26.055626+0000","last_active":"2024-05-27T07:30:05.515428+0000","last_peered":"2024-05-27T07:30:05.515428+0000","last_clean":"2024-05-27T07:30:05.515428+0000","last_became_active":"2024-05-27T07:26:26.037376+0000","last_became_peered":"2024-05-27T07:26:26.037376+0000","last_unstale":"2024-05-27T07:30:05.515428+0000","last_undegraded":"2024-05-27T07:30:05.515428+0000","last_fullsized":"2024-05-27T07:30:05.515428+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_clean_scrub_stamp":"2024-05-27T07:21:26.641949+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:16.741856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584257+0000","last_change":"2024-05-27T07:28:37.973078+0000","last_active":"2024-05-27T07:30:05.584257+0000","last_peered":"2024-05-27T07:30:05.584257+0000","last_clean":"2024-05-27T07:30:05.584257+0000","last_became_active":"2024-05-27T07:28:37.972381+0000","last_became_peered":"2024-05-27T07:28:37.972381+0000","last_unstale":"2024-05-27T07:30:05.584257+0000","last_undegraded":"2024-05-27T07:30:05.584257+0000","last_fullsized":"2024-05-27T07:30:05.584257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:18:16.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.404251+0000","last_change":"2024-05-27T07:28:24.483512+0000","last_active":"2024-05-27T07:30:05.404251+0000","last_peered":"2024-05-27T07:30:05.404251+0000","last_clean":"2024-05-27T07:30:05.404251+0000","last_became_active":"2024-05-27T07:28:24.482655+0000","last_became_peered":"2024-05-27T07:28:24.482655+0000","last_unstale":"2024-05-27T07:30:05.404251+0000","last_undegraded":"2024-05-27T07:30:05.404251+0000","last_fullsized":"2024-05-27T07:30:05.404251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:01:07.898504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515986+0000","last_change":"2024-05-27T07:28:20.408953+0000","last_active":"2024-05-27T07:30:05.515986+0000","last_peered":"2024-05-27T07:30:05.515986+0000","last_clean":"2024-05-27T07:30:05.515986+0000","last_became_active":"2024-05-27T07:28:20.407631+0000","last_became_peered":"2024-05-27T07:28:20.407631+0000","last_unstale":"2024-05-27T07:30:05.515986+0000","last_undegraded":"2024-05-27T07:30:05.515986+0000","last_fullsized":"2024-05-27T07:30:05.515986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:43.912338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579924+0000","last_change":"2024-05-27T07:28:37.907441+0000","last_active":"2024-05-27T07:30:05.579924+0000","last_peered":"2024-05-27T07:30:05.579924+0000","last_clean":"2024-05-27T07:30:05.579924+0000","last_became_active":"2024-05-27T07:28:37.906928+0000","last_became_peered":"2024-05-27T07:28:37.906928+0000","last_unstale":"2024-05-27T07:30:05.579924+0000","last_undegraded":"2024-05-27T07:30:05.579924+0000","last_fullsized":"2024-05-27T07:30:05.579924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:28:32.345826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584754+0000","last_change":"2024-05-27T07:28:20.467408+0000","last_active":"2024-05-27T07:30:05.584754+0000","last_peered":"2024-05-27T07:30:05.584754+0000","last_clean":"2024-05-27T07:30:05.584754+0000","last_became_active":"2024-05-27T07:28:20.466064+0000","last_became_peered":"2024-05-27T07:28:20.466064+0000","last_unstale":"2024-05-27T07:30:05.584754+0000","last_undegraded":"2024-05-27T07:30:05.584754+0000","last_fullsized":"2024-05-27T07:30:05.584754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:15:51.449514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406010+0000","last_change":"2024-05-27T07:28:24.478475+0000","last_active":"2024-05-27T07:30:05.406010+0000","last_peered":"2024-05-27T07:30:05.406010+0000","last_clean":"2024-05-27T07:30:05.406010+0000","last_became_active":"2024-05-27T07:28:24.477936+0000","last_became_peered":"2024-05-27T07:28:24.477936+0000","last_unstale":"2024-05-27T07:30:05.406010+0000","last_undegraded":"2024-05-27T07:30:05.406010+0000","last_fullsized":"2024-05-27T07:30:05.406010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:53:17.500040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119635+0000","last_change":"2024-05-27T07:28:37.968192+0000","last_active":"2024-05-27T07:30:05.119635+0000","last_peered":"2024-05-27T07:30:05.119635+0000","last_clean":"2024-05-27T07:30:05.119635+0000","last_became_active":"2024-05-27T07:28:37.967603+0000","last_became_peered":"2024-05-27T07:28:37.967603+0000","last_unstale":"2024-05-27T07:30:05.119635+0000","last_undegraded":"2024-05-27T07:30:05.119635+0000","last_fullsized":"2024-05-27T07:30:05.119635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:53:36.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":65,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:19.659257+0000","last_change":"2024-05-27T07:29:04.901062+0000","last_active":"2024-05-27T07:30:19.659257+0000","last_peered":"2024-05-27T07:30:19.659257+0000","last_clean":"2024-05-27T07:30:19.659257+0000","last_became_active":"2024-05-27T07:29:04.892060+0000","last_became_peered":"2024-05-27T07:29:04.892060+0000","last_unstale":"2024-05-27T07:30:19.659257+0000","last_undegraded":"2024-05-27T07:30:19.659257+0000","last_fullsized":"2024-05-27T07:30:19.659257+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:27:37.894263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.121792+0000","last_change":"2024-05-27T07:28:24.477096+0000","last_active":"2024-05-27T07:30:05.121792+0000","last_peered":"2024-05-27T07:30:05.121792+0000","last_clean":"2024-05-27T07:30:05.121792+0000","last_became_active":"2024-05-27T07:28:24.476712+0000","last_became_peered":"2024-05-27T07:28:24.476712+0000","last_unstale":"2024-05-27T07:30:05.121792+0000","last_undegraded":"2024-05-27T07:30:05.121792+0000","last_fullsized":"2024-05-27T07:30:05.121792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:31:58.010761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119189+0000","last_change":"2024-05-27T07:28:37.894013+0000","last_active":"2024-05-27T07:30:05.119189+0000","last_peered":"2024-05-27T07:30:05.119189+0000","last_clean":"2024-05-27T07:30:05.119189+0000","last_became_active":"2024-05-27T07:28:37.893345+0000","last_became_peered":"2024-05-27T07:28:37.893345+0000","last_unstale":"2024-05-27T07:30:05.119189+0000","last_undegraded":"2024-05-27T07:30:05.119189+0000","last_fullsized":"2024-05-27T07:30:05.119189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:26.917731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537638+0000","last_change":"2024-05-27T07:28:20.744087+0000","last_active":"2024-05-27T07:30:05.537638+0000","last_peered":"2024-05-27T07:30:05.537638+0000","last_clean":"2024-05-27T07:30:05.537638+0000","last_became_active":"2024-05-27T07:28:20.742883+0000","last_became_peered":"2024-05-27T07:28:20.742883+0000","last_unstale":"2024-05-27T07:30:05.537638+0000","last_undegraded":"2024-05-27T07:30:05.537638+0000","last_fullsized":"2024-05-27T07:30:05.537638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:06.083348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576358+0000","last_change":"2024-05-27T07:28:24.480330+0000","last_active":"2024-05-27T07:30:05.576358+0000","last_peered":"2024-05-27T07:30:05.576358+0000","last_clean":"2024-05-27T07:30:05.576358+0000","last_became_active":"2024-05-27T07:28:24.479300+0000","last_became_peered":"2024-05-27T07:28:24.479300+0000","last_unstale":"2024-05-27T07:30:05.576358+0000","last_undegraded":"2024-05-27T07:30:05.576358+0000","last_fullsized":"2024-05-27T07:30:05.576358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:43.802273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537054+0000","last_change":"2024-05-27T07:28:37.973592+0000","last_active":"2024-05-27T07:30:05.537054+0000","last_peered":"2024-05-27T07:30:05.537054+0000","last_clean":"2024-05-27T07:30:05.537054+0000","last_became_active":"2024-05-27T07:28:37.972475+0000","last_became_peered":"2024-05-27T07:28:37.972475+0000","last_unstale":"2024-05-27T07:30:05.537054+0000","last_undegraded":"2024-05-27T07:30:05.537054+0000","last_fullsized":"2024-05-27T07:30:05.537054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:31.662384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122119+0000","last_change":"2024-05-27T07:29:04.890751+0000","last_active":"2024-05-27T07:30:05.122119+0000","last_peered":"2024-05-27T07:30:05.122119+0000","last_clean":"2024-05-27T07:30:05.122119+0000","last_became_active":"2024-05-27T07:29:04.889447+0000","last_became_peered":"2024-05-27T07:29:04.889447+0000","last_unstale":"2024-05-27T07:30:05.122119+0000","last_undegraded":"2024-05-27T07:30:05.122119+0000","last_fullsized":"2024-05-27T07:30:05.122119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:13:34.743691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580514+0000","last_change":"2024-05-27T07:28:24.474298+0000","last_active":"2024-05-27T07:30:05.580514+0000","last_peered":"2024-05-27T07:30:05.580514+0000","last_clean":"2024-05-27T07:30:05.580514+0000","last_became_active":"2024-05-27T07:28:24.473877+0000","last_became_peered":"2024-05-27T07:28:24.473877+0000","last_unstale":"2024-05-27T07:30:05.580514+0000","last_undegraded":"2024-05-27T07:30:05.580514+0000","last_fullsized":"2024-05-27T07:30:05.580514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:31.545883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127438+0000","last_change":"2024-05-27T07:28:37.905417+0000","last_active":"2024-05-27T07:30:05.127438+0000","last_peered":"2024-05-27T07:30:05.127438+0000","last_clean":"2024-05-27T07:30:05.127438+0000","last_became_active":"2024-05-27T07:28:37.904968+0000","last_became_peered":"2024-05-27T07:28:37.904968+0000","last_unstale":"2024-05-27T07:30:05.127438+0000","last_undegraded":"2024-05-27T07:30:05.127438+0000","last_fullsized":"2024-05-27T07:30:05.127438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:02.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120112+0000","last_change":"2024-05-27T07:28:20.392647+0000","last_active":"2024-05-27T07:30:05.120112+0000","last_peered":"2024-05-27T07:30:05.120112+0000","last_clean":"2024-05-27T07:30:05.120112+0000","last_became_active":"2024-05-27T07:28:20.392073+0000","last_became_peered":"2024-05-27T07:28:20.392073+0000","last_unstale":"2024-05-27T07:30:05.120112+0000","last_undegraded":"2024-05-27T07:30:05.120112+0000","last_fullsized":"2024-05-27T07:30:05.120112+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:18.777628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579061+0000","last_change":"2024-05-27T07:28:24.490522+0000","last_active":"2024-05-27T07:30:05.579061+0000","last_peered":"2024-05-27T07:30:05.579061+0000","last_clean":"2024-05-27T07:30:05.579061+0000","last_became_active":"2024-05-27T07:28:24.485228+0000","last_became_peered":"2024-05-27T07:28:24.485228+0000","last_unstale":"2024-05-27T07:30:05.579061+0000","last_undegraded":"2024-05-27T07:30:05.579061+0000","last_fullsized":"2024-05-27T07:30:05.579061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:14.221016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537587+0000","last_change":"2024-05-27T07:28:37.900449+0000","last_active":"2024-05-27T07:30:05.537587+0000","last_peered":"2024-05-27T07:30:05.537587+0000","last_clean":"2024-05-27T07:30:05.537587+0000","last_became_active":"2024-05-27T07:28:37.899526+0000","last_became_peered":"2024-05-27T07:28:37.899526+0000","last_unstale":"2024-05-27T07:30:05.537587+0000","last_undegraded":"2024-05-27T07:30:05.537587+0000","last_fullsized":"2024-05-27T07:30:05.537587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:29:49.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119900+0000","last_change":"2024-05-27T07:28:20.405236+0000","last_active":"2024-05-27T07:30:05.119900+0000","last_peered":"2024-05-27T07:30:05.119900+0000","last_clean":"2024-05-27T07:30:05.119900+0000","last_became_active":"2024-05-27T07:28:20.404619+0000","last_became_peered":"2024-05-27T07:28:20.404619+0000","last_unstale":"2024-05-27T07:30:05.119900+0000","last_undegraded":"2024-05-27T07:30:05.119900+0000","last_fullsized":"2024-05-27T07:30:05.119900+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:17.671250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120712+0000","last_change":"2024-05-27T07:28:24.490661+0000","last_active":"2024-05-27T07:30:05.120712+0000","last_peered":"2024-05-27T07:30:05.120712+0000","last_clean":"2024-05-27T07:30:05.120712+0000","last_became_active":"2024-05-27T07:28:24.487299+0000","last_became_peered":"2024-05-27T07:28:24.487299+0000","last_unstale":"2024-05-27T07:30:05.120712+0000","last_undegraded":"2024-05-27T07:30:05.120712+0000","last_fullsized":"2024-05-27T07:30:05.120712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:50:27.291721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536951+0000","last_change":"2024-05-27T07:28:37.970463+0000","last_active":"2024-05-27T07:30:05.536951+0000","last_peered":"2024-05-27T07:30:05.536951+0000","last_clean":"2024-05-27T07:30:05.536951+0000","last_became_active":"2024-05-27T07:28:37.968068+0000","last_became_peered":"2024-05-27T07:28:37.968068+0000","last_unstale":"2024-05-27T07:30:05.536951+0000","last_undegraded":"2024-05-27T07:30:05.536951+0000","last_fullsized":"2024-05-27T07:30:05.536951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:22:00.825999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580077+0000","last_change":"2024-05-27T07:28:20.406203+0000","last_active":"2024-05-27T07:30:05.580077+0000","last_peered":"2024-05-27T07:30:05.580077+0000","last_clean":"2024-05-27T07:30:05.580077+0000","last_became_active":"2024-05-27T07:28:20.405404+0000","last_became_peered":"2024-05-27T07:28:20.405404+0000","last_unstale":"2024-05-27T07:30:05.580077+0000","last_undegraded":"2024-05-27T07:30:05.580077+0000","last_fullsized":"2024-05-27T07:30:05.580077+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:15.274419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516952+0000","last_change":"2024-05-27T07:28:24.473747+0000","last_active":"2024-05-27T07:30:05.516952+0000","last_peered":"2024-05-27T07:30:05.516952+0000","last_clean":"2024-05-27T07:30:05.516952+0000","last_became_active":"2024-05-27T07:28:24.473313+0000","last_became_peered":"2024-05-27T07:28:24.473313+0000","last_unstale":"2024-05-27T07:30:05.516952+0000","last_undegraded":"2024-05-27T07:30:05.516952+0000","last_fullsized":"2024-05-27T07:30:05.516952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:19:40.107840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.121935+0000","last_change":"2024-05-27T07:28:37.907410+0000","last_active":"2024-05-27T07:30:05.121935+0000","last_peered":"2024-05-27T07:30:05.121935+0000","last_clean":"2024-05-27T07:30:05.121935+0000","last_became_active":"2024-05-27T07:28:37.906753+0000","last_became_peered":"2024-05-27T07:28:37.906753+0000","last_unstale":"2024-05-27T07:30:05.121935+0000","last_undegraded":"2024-05-27T07:30:05.121935+0000","last_fullsized":"2024-05-27T07:30:05.121935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:53.612015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120470+0000","last_change":"2024-05-27T07:28:20.744068+0000","last_active":"2024-05-27T07:30:05.120470+0000","last_peered":"2024-05-27T07:30:05.120470+0000","last_clean":"2024-05-27T07:30:05.120470+0000","last_became_active":"2024-05-27T07:28:20.742686+0000","last_became_peered":"2024-05-27T07:28:20.742686+0000","last_unstale":"2024-05-27T07:30:05.120470+0000","last_undegraded":"2024-05-27T07:30:05.120470+0000","last_fullsized":"2024-05-27T07:30:05.120470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:54.361007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.517087+0000","last_change":"2024-05-27T07:28:24.475743+0000","last_active":"2024-05-27T07:30:05.517087+0000","last_peered":"2024-05-27T07:30:05.517087+0000","last_clean":"2024-05-27T07:30:05.517087+0000","last_became_active":"2024-05-27T07:28:24.474742+0000","last_became_peered":"2024-05-27T07:28:24.474742+0000","last_unstale":"2024-05-27T07:30:05.517087+0000","last_undegraded":"2024-05-27T07:30:05.517087+0000","last_fullsized":"2024-05-27T07:30:05.517087+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:58:47.061573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576220+0000","last_change":"2024-05-27T07:28:37.914935+0000","last_active":"2024-05-27T07:30:05.576220+0000","last_peered":"2024-05-27T07:30:05.576220+0000","last_clean":"2024-05-27T07:30:05.576220+0000","last_became_active":"2024-05-27T07:28:37.911829+0000","last_became_peered":"2024-05-27T07:28:37.911829+0000","last_unstale":"2024-05-27T07:30:05.576220+0000","last_undegraded":"2024-05-27T07:30:05.576220+0000","last_fullsized":"2024-05-27T07:30:05.576220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:29.260539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584299+0000","last_change":"2024-05-27T07:28:20.412554+0000","last_active":"2024-05-27T07:30:05.584299+0000","last_peered":"2024-05-27T07:30:05.584299+0000","last_clean":"2024-05-27T07:30:05.584299+0000","last_became_active":"2024-05-27T07:28:20.412024+0000","last_became_peered":"2024-05-27T07:28:20.412024+0000","last_unstale":"2024-05-27T07:30:05.584299+0000","last_undegraded":"2024-05-27T07:30:05.584299+0000","last_fullsized":"2024-05-27T07:30:05.584299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:29.244093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516559+0000","last_change":"2024-05-27T07:28:24.481966+0000","last_active":"2024-05-27T07:30:05.516559+0000","last_peered":"2024-05-27T07:30:05.516559+0000","last_clean":"2024-05-27T07:30:05.516559+0000","last_became_active":"2024-05-27T07:28:24.480775+0000","last_became_peered":"2024-05-27T07:28:24.480775+0000","last_unstale":"2024-05-27T07:30:05.516559+0000","last_undegraded":"2024-05-27T07:30:05.516559+0000","last_fullsized":"2024-05-27T07:30:05.516559+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:03.018174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578676+0000","last_change":"2024-05-27T07:28:37.902893+0000","last_active":"2024-05-27T07:30:05.578676+0000","last_peered":"2024-05-27T07:30:05.578676+0000","last_clean":"2024-05-27T07:30:05.578676+0000","last_became_active":"2024-05-27T07:28:37.898447+0000","last_became_peered":"2024-05-27T07:28:37.898447+0000","last_unstale":"2024-05-27T07:30:05.578676+0000","last_undegraded":"2024-05-27T07:30:05.578676+0000","last_fullsized":"2024-05-27T07:30:05.578676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:55:49.754803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406914+0000","last_change":"2024-05-27T07:28:20.408072+0000","last_active":"2024-05-27T07:30:05.406914+0000","last_peered":"2024-05-27T07:30:05.406914+0000","last_clean":"2024-05-27T07:30:05.406914+0000","last_became_active":"2024-05-27T07:28:20.407174+0000","last_became_peered":"2024-05-27T07:28:20.407174+0000","last_unstale":"2024-05-27T07:30:05.406914+0000","last_undegraded":"2024-05-27T07:30:05.406914+0000","last_fullsized":"2024-05-27T07:30:05.406914+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:15:52.984967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.517471+0000","last_change":"2024-05-27T07:28:24.475891+0000","last_active":"2024-05-27T07:30:05.517471+0000","last_peered":"2024-05-27T07:30:05.517471+0000","last_clean":"2024-05-27T07:30:05.517471+0000","last_became_active":"2024-05-27T07:28:24.475498+0000","last_became_peered":"2024-05-27T07:28:24.475498+0000","last_unstale":"2024-05-27T07:30:05.517471+0000","last_undegraded":"2024-05-27T07:30:05.517471+0000","last_fullsized":"2024-05-27T07:30:05.517471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:50.669118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579123+0000","last_change":"2024-05-27T07:29:04.891583+0000","last_active":"2024-05-27T07:30:05.579123+0000","last_peered":"2024-05-27T07:30:05.579123+0000","last_clean":"2024-05-27T07:30:05.579123+0000","last_became_active":"2024-05-27T07:29:04.890187+0000","last_became_peered":"2024-05-27T07:29:04.890187+0000","last_unstale":"2024-05-27T07:30:05.579123+0000","last_undegraded":"2024-05-27T07:30:05.579123+0000","last_fullsized":"2024-05-27T07:30:05.579123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:03.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119928+0000","last_change":"2024-05-27T07:28:24.478672+0000","last_active":"2024-05-27T07:30:05.119928+0000","last_peered":"2024-05-27T07:30:05.119928+0000","last_clean":"2024-05-27T07:30:05.119928+0000","last_became_active":"2024-05-27T07:28:24.478331+0000","last_became_peered":"2024-05-27T07:28:24.478331+0000","last_unstale":"2024-05-27T07:30:05.119928+0000","last_undegraded":"2024-05-27T07:30:05.119928+0000","last_fullsized":"2024-05-27T07:30:05.119928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:56:35.112855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119521+0000","last_change":"2024-05-27T07:28:24.487962+0000","last_active":"2024-05-27T07:30:05.119521+0000","last_peered":"2024-05-27T07:30:05.119521+0000","last_clean":"2024-05-27T07:30:05.119521+0000","last_became_active":"2024-05-27T07:28:24.487583+0000","last_became_peered":"2024-05-27T07:28:24.487583+0000","last_unstale":"2024-05-27T07:30:05.119521+0000","last_undegraded":"2024-05-27T07:30:05.119521+0000","last_fullsized":"2024-05-27T07:30:05.119521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:25.758293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576333+0000","last_change":"2024-05-27T07:28:24.466058+0000","last_active":"2024-05-27T07:30:05.576333+0000","last_peered":"2024-05-27T07:30:05.576333+0000","last_clean":"2024-05-27T07:30:05.576333+0000","last_became_active":"2024-05-27T07:28:24.465709+0000","last_became_peered":"2024-05-27T07:28:24.465709+0000","last_unstale":"2024-05-27T07:30:05.576333+0000","last_undegraded":"2024-05-27T07:30:05.576333+0000","last_fullsized":"2024-05-27T07:30:05.576333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:38.064077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406173+0000","last_change":"2024-05-27T07:28:24.471585+0000","last_active":"2024-05-27T07:30:05.406173+0000","last_peered":"2024-05-27T07:30:05.406173+0000","last_clean":"2024-05-27T07:30:05.406173+0000","last_became_active":"2024-05-27T07:28:24.470948+0000","last_became_peered":"2024-05-27T07:28:24.470948+0000","last_unstale":"2024-05-27T07:30:05.406173+0000","last_undegraded":"2024-05-27T07:30:05.406173+0000","last_fullsized":"2024-05-27T07:30:05.406173+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:48.213323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581655+0000","last_change":"2024-05-27T07:28:24.477954+0000","last_active":"2024-05-27T07:30:05.581655+0000","last_peered":"2024-05-27T07:30:05.581655+0000","last_clean":"2024-05-27T07:30:05.581655+0000","last_became_active":"2024-05-27T07:28:24.477495+0000","last_became_peered":"2024-05-27T07:28:24.477495+0000","last_unstale":"2024-05-27T07:30:05.581655+0000","last_undegraded":"2024-05-27T07:30:05.581655+0000","last_fullsized":"2024-05-27T07:30:05.581655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:50.371439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.121046+0000","last_change":"2024-05-27T07:28:24.476422+0000","last_active":"2024-05-27T07:30:05.121046+0000","last_peered":"2024-05-27T07:30:05.121046+0000","last_clean":"2024-05-27T07:30:05.121046+0000","last_became_active":"2024-05-27T07:28:24.476095+0000","last_became_peered":"2024-05-27T07:28:24.476095+0000","last_unstale":"2024-05-27T07:30:05.121046+0000","last_undegraded":"2024-05-27T07:30:05.121046+0000","last_fullsized":"2024-05-27T07:30:05.121046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:51.667169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122064+0000","last_change":"2024-05-27T07:28:24.486431+0000","last_active":"2024-05-27T07:30:05.122064+0000","last_peered":"2024-05-27T07:30:05.122064+0000","last_clean":"2024-05-27T07:30:05.122064+0000","last_became_active":"2024-05-27T07:28:24.482427+0000","last_became_peered":"2024-05-27T07:28:24.482427+0000","last_unstale":"2024-05-27T07:30:05.122064+0000","last_undegraded":"2024-05-27T07:30:05.122064+0000","last_fullsized":"2024-05-27T07:30:05.122064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:44:37.857793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582171+0000","last_change":"2024-05-27T07:28:24.484393+0000","last_active":"2024-05-27T07:30:05.582171+0000","last_peered":"2024-05-27T07:30:05.582171+0000","last_clean":"2024-05-27T07:30:05.582171+0000","last_became_active":"2024-05-27T07:28:24.483130+0000","last_became_peered":"2024-05-27T07:28:24.483130+0000","last_unstale":"2024-05-27T07:30:05.582171+0000","last_undegraded":"2024-05-27T07:30:05.582171+0000","last_fullsized":"2024-05-27T07:30:05.582171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:34.924573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120191+0000","last_change":"2024-05-27T07:28:24.486274+0000","last_active":"2024-05-27T07:30:05.120191+0000","last_peered":"2024-05-27T07:30:05.120191+0000","last_clean":"2024-05-27T07:30:05.120191+0000","last_became_active":"2024-05-27T07:28:24.485880+0000","last_became_peered":"2024-05-27T07:28:24.485880+0000","last_unstale":"2024-05-27T07:30:05.120191+0000","last_undegraded":"2024-05-27T07:30:05.120191+0000","last_fullsized":"2024-05-27T07:30:05.120191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:33.287615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122910+0000","last_change":"2024-05-27T07:28:24.477909+0000","last_active":"2024-05-27T07:30:05.122910+0000","last_peered":"2024-05-27T07:30:05.122910+0000","last_clean":"2024-05-27T07:30:05.122910+0000","last_became_active":"2024-05-27T07:28:24.477345+0000","last_became_peered":"2024-05-27T07:28:24.477345+0000","last_unstale":"2024-05-27T07:30:05.122910+0000","last_undegraded":"2024-05-27T07:30:05.122910+0000","last_fullsized":"2024-05-27T07:30:05.122910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:26:03.958489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.404938+0000","last_change":"2024-05-27T07:28:24.477151+0000","last_active":"2024-05-27T07:30:05.404938+0000","last_peered":"2024-05-27T07:30:05.404938+0000","last_clean":"2024-05-27T07:30:05.404938+0000","last_became_active":"2024-05-27T07:28:24.474036+0000","last_became_peered":"2024-05-27T07:28:24.474036+0000","last_unstale":"2024-05-27T07:30:05.404938+0000","last_undegraded":"2024-05-27T07:30:05.404938+0000","last_fullsized":"2024-05-27T07:30:05.404938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:16.787252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580908+0000","last_change":"2024-05-27T07:28:24.483601+0000","last_active":"2024-05-27T07:30:05.580908+0000","last_peered":"2024-05-27T07:30:05.580908+0000","last_clean":"2024-05-27T07:30:05.580908+0000","last_became_active":"2024-05-27T07:28:24.481080+0000","last_became_peered":"2024-05-27T07:28:24.481080+0000","last_unstale":"2024-05-27T07:30:05.580908+0000","last_undegraded":"2024-05-27T07:30:05.580908+0000","last_fullsized":"2024-05-27T07:30:05.580908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:12.208506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71429,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":236,"num_read_kb":195,"num_write":169,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":214287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645355,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":522520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841586,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1740800,"data_stored":1177607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005114,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":526852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201347,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":522676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397578,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32792,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712360,"statfs":{"total":95995035648,"available":95961456640,"internally_reserved":0,"allocated":1654784,"data_stored":1118194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561104,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":517932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":581396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":519320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":587011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149799,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":518126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953567,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1662976,"data_stored":1113782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:30:30.927 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T07:30:31.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:30 smithi063 ceph-mon[31613]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-27T07:30:31.206 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:30 smithi063 ceph-mon[31613]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:30:31.392 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:31.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:30 smithi086 ceph-mon[34687]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-27T07:30:31.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:30 smithi086 ceph-mon[34687]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:30:31.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:30 smithi149 ceph-mon[35408]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-27T07:30:31.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:30 smithi149 ceph-mon[35408]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:30:32.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:32.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:32 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:32.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:32 smithi063 ceph-mon[31613]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-27T07:30:32.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:32.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:32 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:32.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:32 smithi086 ceph-mon[34687]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-27T07:30:32.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:32.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:32 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:32.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:32 smithi149 ceph-mon[35408]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-27T07:30:33.144 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:33 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:30:33.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:33 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:30:33.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:33 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:30:33.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:33 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:33.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:33 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:30:33.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:33 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:30:33.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:33 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:33.457 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:33 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:30:33.457 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:33 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:30:33.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.947+0000 7f24ea800700 1 -- 172.21.15.63:0/4191263026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24e4110960 msgr2=0x7f24e4110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:33.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.947+0000 7f24ea800700 1 --2- 172.21.15.63:0/4191263026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24e4110960 0x7f24e4110d70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009990 tx=0x7f24d4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:30:33.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.947+0000 7f24ea800700 1 -- 172.21.15.63:0/4191263026 shutdown_connections 2024-05-27T07:30:33.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.947+0000 7f24ea800700 1 --2- 172.21.15.63:0/4191263026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e410b2d0 0x7f24e4117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:33.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.947+0000 7f24ea800700 1 --2- 172.21.15.63:0/4191263026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24e4110960 0x7f24e4110d70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:33.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.947+0000 7f24ea800700 1 --2- 172.21.15.63:0/4191263026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24e410a980 0x7f24e410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:33.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.947+0000 7f24ea800700 1 -- 172.21.15.63:0/4191263026 >> 172.21.15.63:0/4191263026 conn(0x7f24e4077c70 msgr2=0x7f24e40780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:33.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.947+0000 7f24ea800700 1 -- 172.21.15.63:0/4191263026 shutdown_connections 2024-05-27T07:30:33.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.947+0000 7f24ea800700 1 -- 172.21.15.63:0/4191263026 wait complete. 2024-05-27T07:30:33.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.949+0000 7f24ea800700 1 Processor -- start 2024-05-27T07:30:33.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.949+0000 7f24ea800700 1 -- start start 2024-05-27T07:30:33.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24ea800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24e410a980 0x7f24e41a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:33.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24ea800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24e410b2d0 0x7f24e41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:33.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24ea800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e4110960 0x7f24e41a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:33.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24ea800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24e411a960 con 0x7f24e410b2d0 2024-05-27T07:30:33.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24ea800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24e411a7e0 con 0x7f24e4110960 2024-05-27T07:30:33.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24ea800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24e411aae0 con 0x7f24e410a980 2024-05-27T07:30:33.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24e2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24e410b2d0 0x7f24e41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:33.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24e2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24e410b2d0 0x7f24e41a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57702/0 (socket says 172.21.15.63:57702) 2024-05-27T07:30:33.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24e2a00700 1 -- 172.21.15.63:0/1880385368 learned_addr learned my addr 172.21.15.63:0/1880385368 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24e3400700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24e410a980 0x7f24e41a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.950+0000 7f24e3e00700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e4110960 0x7f24e41a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.951+0000 7f24e2a00700 1 -- 172.21.15.63:0/1880385368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24e410a980 msgr2=0x7f24e41a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.951+0000 7f24e2a00700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24e410a980 0x7f24e41a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.951+0000 7f24e2a00700 1 -- 172.21.15.63:0/1880385368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e4110960 msgr2=0x7f24e41a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.951+0000 7f24e2a00700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e4110960 0x7f24e41a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.951+0000 7f24e2a00700 1 -- 172.21.15.63:0/1880385368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e41a6260 con 0x7f24e410b2d0 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.951+0000 7f24e3400700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24e410a980 0x7f24e41a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.951+0000 7f24e2a00700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24e410b2d0 0x7f24e41a1930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f24d000d980 tx=0x7f24d000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.951+0000 7f24dbe00700 1 -- 172.21.15.63:0/1880385368 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0017070 con 0x7f24e410b2d0 2024-05-27T07:30:33.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.951+0000 7f24ea800700 1 -- 172.21.15.63:0/1880385368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e41a6550 con 0x7f24e410b2d0 2024-05-27T07:30:33.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.951+0000 7f24dbe00700 1 -- 172.21.15.63:0/1880385368 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24d0010460 con 0x7f24e410b2d0 2024-05-27T07:30:33.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.952+0000 7f24ea800700 1 -- 172.21.15.63:0/1880385368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e41ade70 con 0x7f24e410b2d0 2024-05-27T07:30:33.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.952+0000 7f24dbe00700 1 -- 172.21.15.63:0/1880385368 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0020060 con 0x7f24e410b2d0 2024-05-27T07:30:33.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.953+0000 7f24ea800700 1 -- 172.21.15.63:0/1880385368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e40512b0 con 0x7f24e410b2d0 2024-05-27T07:30:33.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.956+0000 7f24dbe00700 1 -- 172.21.15.63:0/1880385368 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f24d0010610 con 0x7f24e410b2d0 2024-05-27T07:30:33.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.958+0000 7f24dbe00700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f24c80b2910 0x7f24c80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:33.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.958+0000 7f24dbe00700 1 -- 172.21.15.63:0/1880385368 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f24d00e24e0 con 0x7f24e410b2d0 2024-05-27T07:30:33.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.958+0000 7f24e3400700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f24c80b2910 0x7f24c80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:33.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.959+0000 7f24e3400700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f24c80b2910 0x7f24c80b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f24d4006010 tx=0x7f24d400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:33.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:33.960+0000 7f24dbe00700 1 -- 172.21.15.63:0/1880385368 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f24d00a9cb0 con 0x7f24e410b2d0 2024-05-27T07:30:34.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.196+0000 7f24ea800700 1 -- 172.21.15.63:0/1880385368 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24e41a26f0 con 0x7f24c80b2910 2024-05-27T07:30:34.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.210+0000 7f24dbe00700 1 -- 172.21.15.63:0/1880385368 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340392 (secure 0 0 0) 0x7f24e41a26f0 con 0x7f24c80b2910 2024-05-27T07:30:34.208 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:30:34.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.217+0000 7f24ea800700 1 -- 172.21.15.63:0/1880385368 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f24c80b2910 msgr2=0x7f24c80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:34.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.217+0000 7f24ea800700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f24c80b2910 0x7f24c80b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f24d4006010 tx=0x7f24d400b540 comp rx=0 tx=0).stop 2024-05-27T07:30:34.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.217+0000 7f24ea800700 1 -- 172.21.15.63:0/1880385368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24e410b2d0 msgr2=0x7f24e41a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:34.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.217+0000 7f24ea800700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24e410b2d0 0x7f24e41a1930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f24d000d980 tx=0x7f24d000dd40 comp rx=0 tx=0).stop 2024-05-27T07:30:34.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.218+0000 7f24ea800700 1 -- 172.21.15.63:0/1880385368 shutdown_connections 2024-05-27T07:30:34.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.218+0000 7f24ea800700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24e4110960 0x7f24e41a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.218+0000 7f24ea800700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24e410a980 0x7f24e41a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.218+0000 7f24ea800700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f24c80b2910 0x7f24c80b4e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.218+0000 7f24ea800700 1 --2- 172.21.15.63:0/1880385368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24e410b2d0 0x7f24e41a1930 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.218+0000 7f24ea800700 1 -- 172.21.15.63:0/1880385368 >> 172.21.15.63:0/1880385368 conn(0x7f24e4077c70 msgr2=0x7f24e407bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.218+0000 7f24ea800700 1 -- 172.21.15.63:0/1880385368 shutdown_connections 2024-05-27T07:30:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:34.218+0000 7f24ea800700 1 -- 172.21.15.63:0/1880385368 wait complete. 2024-05-27T07:30:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-27T07:30:34.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:34 smithi086 ceph-mon[34687]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-05-27T07:30:34.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:34 smithi149 ceph-mon[35408]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-05-27T07:30:34.457 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:34 smithi063 ceph-mon[31613]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-05-27T07:30:35.152 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-05-27T07:30:33.828963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662031,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":332,"num_read_kb":269,"num_write":294,"num_write_kb":2273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375996,"kb_used_data":14348,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565828,"statfs":{"total":1151940427776,"available":1151555407872,"internally_reserved":0,"allocated":14692352,"data_stored":8232263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9639,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":41,"num_write":6,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006187"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583117+0000","last_change":"2024-05-27T07:28:24.485151+0000","last_active":"2024-05-27T07:30:05.583117+0000","last_peered":"2024-05-27T07:30:05.583117+0000","last_clean":"2024-05-27T07:30:05.583117+0000","last_became_active":"2024-05-27T07:28:24.483965+0000","last_became_peered":"2024-05-27T07:28:24.483965+0000","last_unstale":"2024-05-27T07:30:05.583117+0000","last_undegraded":"2024-05-27T07:30:05.583117+0000","last_fullsized":"2024-05-27T07:30:05.583117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:18.376755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576274+0000","last_change":"2024-05-27T07:28:24.483608+0000","last_active":"2024-05-27T07:30:05.576274+0000","last_peered":"2024-05-27T07:30:05.576274+0000","last_clean":"2024-05-27T07:30:05.576274+0000","last_became_active":"2024-05-27T07:28:24.479875+0000","last_became_peered":"2024-05-27T07:28:24.479875+0000","last_unstale":"2024-05-27T07:30:05.576274+0000","last_undegraded":"2024-05-27T07:30:05.576274+0000","last_fullsized":"2024-05-27T07:30:05.576274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:30.129708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576154+0000","last_change":"2024-05-27T07:28:24.470104+0000","last_active":"2024-05-27T07:30:05.576154+0000","last_peered":"2024-05-27T07:30:05.576154+0000","last_clean":"2024-05-27T07:30:05.576154+0000","last_became_active":"2024-05-27T07:28:24.469756+0000","last_became_peered":"2024-05-27T07:28:24.469756+0000","last_unstale":"2024-05-27T07:30:05.576154+0000","last_undegraded":"2024-05-27T07:30:05.576154+0000","last_fullsized":"2024-05-27T07:30:05.576154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:36.907691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.538072+0000","last_change":"2024-05-27T07:28:24.472381+0000","last_active":"2024-05-27T07:30:05.538072+0000","last_peered":"2024-05-27T07:30:05.538072+0000","last_clean":"2024-05-27T07:30:05.538072+0000","last_became_active":"2024-05-27T07:28:24.469959+0000","last_became_peered":"2024-05-27T07:28:24.469959+0000","last_unstale":"2024-05-27T07:30:05.538072+0000","last_undegraded":"2024-05-27T07:30:05.538072+0000","last_fullsized":"2024-05-27T07:30:05.538072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:15.267554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580431+0000","last_change":"2024-05-27T07:28:24.474210+0000","last_active":"2024-05-27T07:30:05.580431+0000","last_peered":"2024-05-27T07:30:05.580431+0000","last_clean":"2024-05-27T07:30:05.580431+0000","last_became_active":"2024-05-27T07:28:24.473688+0000","last_became_peered":"2024-05-27T07:28:24.473688+0000","last_unstale":"2024-05-27T07:30:05.580431+0000","last_undegraded":"2024-05-27T07:30:05.580431+0000","last_fullsized":"2024-05-27T07:30:05.580431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:38.733536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578117+0000","last_change":"2024-05-27T07:28:24.484679+0000","last_active":"2024-05-27T07:30:05.578117+0000","last_peered":"2024-05-27T07:30:05.578117+0000","last_clean":"2024-05-27T07:30:05.578117+0000","last_became_active":"2024-05-27T07:28:24.484251+0000","last_became_peered":"2024-05-27T07:28:24.484251+0000","last_unstale":"2024-05-27T07:30:05.578117+0000","last_undegraded":"2024-05-27T07:30:05.578117+0000","last_fullsized":"2024-05-27T07:30:05.578117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:28:09.507871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515699+0000","last_change":"2024-05-27T07:28:24.472694+0000","last_active":"2024-05-27T07:30:05.515699+0000","last_peered":"2024-05-27T07:30:05.515699+0000","last_clean":"2024-05-27T07:30:05.515699+0000","last_became_active":"2024-05-27T07:28:24.472100+0000","last_became_peered":"2024-05-27T07:28:24.472100+0000","last_unstale":"2024-05-27T07:30:05.515699+0000","last_undegraded":"2024-05-27T07:30:05.515699+0000","last_fullsized":"2024-05-27T07:30:05.515699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119432+0000","last_change":"2024-05-27T07:28:24.488577+0000","last_active":"2024-05-27T07:30:05.119432+0000","last_peered":"2024-05-27T07:30:05.119432+0000","last_clean":"2024-05-27T07:30:05.119432+0000","last_became_active":"2024-05-27T07:28:24.488180+0000","last_became_peered":"2024-05-27T07:28:24.488180+0000","last_unstale":"2024-05-27T07:30:05.119432+0000","last_undegraded":"2024-05-27T07:30:05.119432+0000","last_fullsized":"2024-05-27T07:30:05.119432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:17.130058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.123384+0000","last_change":"2024-05-27T07:28:24.480109+0000","last_active":"2024-05-27T07:30:05.123384+0000","last_peered":"2024-05-27T07:30:05.123384+0000","last_clean":"2024-05-27T07:30:05.123384+0000","last_became_active":"2024-05-27T07:28:24.479722+0000","last_became_peered":"2024-05-27T07:28:24.479722+0000","last_unstale":"2024-05-27T07:30:05.123384+0000","last_undegraded":"2024-05-27T07:30:05.123384+0000","last_fullsized":"2024-05-27T07:30:05.123384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:27:36.074612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579053+0000","last_change":"2024-05-27T07:28:24.490389+0000","last_active":"2024-05-27T07:30:05.579053+0000","last_peered":"2024-05-27T07:30:05.579053+0000","last_clean":"2024-05-27T07:30:05.579053+0000","last_became_active":"2024-05-27T07:28:24.488081+0000","last_became_peered":"2024-05-27T07:28:24.488081+0000","last_unstale":"2024-05-27T07:30:05.579053+0000","last_undegraded":"2024-05-27T07:30:05.579053+0000","last_fullsized":"2024-05-27T07:30:05.579053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:06.318235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516771+0000","last_change":"2024-05-27T07:28:24.477062+0000","last_active":"2024-05-27T07:30:05.516771+0000","last_peered":"2024-05-27T07:30:05.516771+0000","last_clean":"2024-05-27T07:30:05.516771+0000","last_became_active":"2024-05-27T07:28:24.475322+0000","last_became_peered":"2024-05-27T07:28:24.475322+0000","last_unstale":"2024-05-27T07:30:05.516771+0000","last_undegraded":"2024-05-27T07:30:05.516771+0000","last_fullsized":"2024-05-27T07:30:05.516771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:30.365295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.126261+0000","last_change":"2024-05-27T07:29:04.901715+0000","last_active":"2024-05-27T07:30:05.126261+0000","last_peered":"2024-05-27T07:30:05.126261+0000","last_clean":"2024-05-27T07:30:05.126261+0000","last_became_active":"2024-05-27T07:29:04.900968+0000","last_became_peered":"2024-05-27T07:29:04.900968+0000","last_unstale":"2024-05-27T07:30:05.126261+0000","last_undegraded":"2024-05-27T07:30:05.126261+0000","last_fullsized":"2024-05-27T07:30:05.126261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:16:34.033701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.404676+0000","last_change":"2024-05-27T07:28:24.477504+0000","last_active":"2024-05-27T07:30:05.404676+0000","last_peered":"2024-05-27T07:30:05.404676+0000","last_clean":"2024-05-27T07:30:05.404676+0000","last_became_active":"2024-05-27T07:28:24.476709+0000","last_became_peered":"2024-05-27T07:28:24.476709+0000","last_unstale":"2024-05-27T07:30:05.404676+0000","last_undegraded":"2024-05-27T07:30:05.404676+0000","last_fullsized":"2024-05-27T07:30:05.404676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:05:48.456347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.575872+0000","last_change":"2024-05-27T07:28:24.485567+0000","last_active":"2024-05-27T07:30:05.575872+0000","last_peered":"2024-05-27T07:30:05.575872+0000","last_clean":"2024-05-27T07:30:05.575872+0000","last_became_active":"2024-05-27T07:28:24.485175+0000","last_became_peered":"2024-05-27T07:28:24.485175+0000","last_unstale":"2024-05-27T07:30:05.575872+0000","last_undegraded":"2024-05-27T07:30:05.575872+0000","last_fullsized":"2024-05-27T07:30:05.575872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:25:30.062813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.575698+0000","last_change":"2024-05-27T07:29:04.900657+0000","last_active":"2024-05-27T07:30:05.575698+0000","last_peered":"2024-05-27T07:30:05.575698+0000","last_clean":"2024-05-27T07:30:05.575698+0000","last_became_active":"2024-05-27T07:29:04.900093+0000","last_became_peered":"2024-05-27T07:29:04.900093+0000","last_unstale":"2024-05-27T07:30:05.575698+0000","last_undegraded":"2024-05-27T07:30:05.575698+0000","last_fullsized":"2024-05-27T07:30:05.575698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:28:04.157825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.121911+0000","last_change":"2024-05-27T07:28:24.470503+0000","last_active":"2024-05-27T07:30:05.121911+0000","last_peered":"2024-05-27T07:30:05.121911+0000","last_clean":"2024-05-27T07:30:05.121911+0000","last_became_active":"2024-05-27T07:28:24.470128+0000","last_became_peered":"2024-05-27T07:28:24.470128+0000","last_unstale":"2024-05-27T07:30:05.121911+0000","last_undegraded":"2024-05-27T07:30:05.121911+0000","last_fullsized":"2024-05-27T07:30:05.121911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:46:43.149635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581135+0000","last_change":"2024-05-27T07:28:24.491382+0000","last_active":"2024-05-27T07:30:05.581135+0000","last_peered":"2024-05-27T07:30:05.581135+0000","last_clean":"2024-05-27T07:30:05.581135+0000","last_became_active":"2024-05-27T07:28:24.486155+0000","last_became_peered":"2024-05-27T07:28:24.486155+0000","last_unstale":"2024-05-27T07:30:05.581135+0000","last_undegraded":"2024-05-27T07:30:05.581135+0000","last_fullsized":"2024-05-27T07:30:05.581135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:48.408573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127068+0000","last_change":"2024-05-27T07:28:24.489140+0000","last_active":"2024-05-27T07:30:05.127068+0000","last_peered":"2024-05-27T07:30:05.127068+0000","last_clean":"2024-05-27T07:30:05.127068+0000","last_became_active":"2024-05-27T07:28:24.488733+0000","last_became_peered":"2024-05-27T07:28:24.488733+0000","last_unstale":"2024-05-27T07:30:05.127068+0000","last_undegraded":"2024-05-27T07:30:05.127068+0000","last_fullsized":"2024-05-27T07:30:05.127068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:02:57.711197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120975+0000","last_change":"2024-05-27T07:28:24.486022+0000","last_active":"2024-05-27T07:30:05.120975+0000","last_peered":"2024-05-27T07:30:05.120975+0000","last_clean":"2024-05-27T07:30:05.120975+0000","last_became_active":"2024-05-27T07:28:24.485639+0000","last_became_peered":"2024-05-27T07:28:24.485639+0000","last_unstale":"2024-05-27T07:30:05.120975+0000","last_undegraded":"2024-05-27T07:30:05.120975+0000","last_fullsized":"2024-05-27T07:30:05.120975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:07.629402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119429+0000","last_change":"2024-05-27T07:28:37.898984+0000","last_active":"2024-05-27T07:30:05.119429+0000","last_peered":"2024-05-27T07:30:05.119429+0000","last_clean":"2024-05-27T07:30:05.119429+0000","last_became_active":"2024-05-27T07:28:37.898132+0000","last_became_peered":"2024-05-27T07:28:37.898132+0000","last_unstale":"2024-05-27T07:30:05.119429+0000","last_undegraded":"2024-05-27T07:30:05.119429+0000","last_fullsized":"2024-05-27T07:30:05.119429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:51.987386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127598+0000","last_change":"2024-05-27T07:28:20.402788+0000","last_active":"2024-05-27T07:30:05.127598+0000","last_peered":"2024-05-27T07:30:05.127598+0000","last_clean":"2024-05-27T07:30:05.127598+0000","last_became_active":"2024-05-27T07:28:20.402041+0000","last_became_peered":"2024-05-27T07:28:20.402041+0000","last_unstale":"2024-05-27T07:30:05.127598+0000","last_undegraded":"2024-05-27T07:30:05.127598+0000","last_fullsized":"2024-05-27T07:30:05.127598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:44.421979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.121315+0000","last_change":"2024-05-27T07:28:24.485193+0000","last_active":"2024-05-27T07:30:05.121315+0000","last_peered":"2024-05-27T07:30:05.121315+0000","last_clean":"2024-05-27T07:30:05.121315+0000","last_became_active":"2024-05-27T07:28:24.484086+0000","last_became_peered":"2024-05-27T07:28:24.484086+0000","last_unstale":"2024-05-27T07:30:05.121315+0000","last_undegraded":"2024-05-27T07:30:05.121315+0000","last_fullsized":"2024-05-27T07:30:05.121315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:10.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536402+0000","last_change":"2024-05-27T07:28:37.902602+0000","last_active":"2024-05-27T07:30:05.536402+0000","last_peered":"2024-05-27T07:30:05.536402+0000","last_clean":"2024-05-27T07:30:05.536402+0000","last_became_active":"2024-05-27T07:28:37.902120+0000","last_became_peered":"2024-05-27T07:28:37.902120+0000","last_unstale":"2024-05-27T07:30:05.536402+0000","last_undegraded":"2024-05-27T07:30:05.536402+0000","last_fullsized":"2024-05-27T07:30:05.536402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:21.209851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122003+0000","last_change":"2024-05-27T07:28:20.465500+0000","last_active":"2024-05-27T07:30:05.122003+0000","last_peered":"2024-05-27T07:30:05.122003+0000","last_clean":"2024-05-27T07:30:05.122003+0000","last_became_active":"2024-05-27T07:28:20.464951+0000","last_became_peered":"2024-05-27T07:28:20.464951+0000","last_unstale":"2024-05-27T07:30:05.122003+0000","last_undegraded":"2024-05-27T07:30:05.122003+0000","last_fullsized":"2024-05-27T07:30:05.122003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:23:25.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576574+0000","last_change":"2024-05-27T07:28:24.467655+0000","last_active":"2024-05-27T07:30:05.576574+0000","last_peered":"2024-05-27T07:30:05.576574+0000","last_clean":"2024-05-27T07:30:05.576574+0000","last_became_active":"2024-05-27T07:28:24.467194+0000","last_became_peered":"2024-05-27T07:28:24.467194+0000","last_unstale":"2024-05-27T07:30:05.576574+0000","last_undegraded":"2024-05-27T07:30:05.576574+0000","last_fullsized":"2024-05-27T07:30:05.576574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:38.024514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536716+0000","last_change":"2024-05-27T07:29:04.975679+0000","last_active":"2024-05-27T07:30:05.536716+0000","last_peered":"2024-05-27T07:30:05.536716+0000","last_clean":"2024-05-27T07:30:05.536716+0000","last_became_active":"2024-05-27T07:29:04.974323+0000","last_became_peered":"2024-05-27T07:29:04.974323+0000","last_unstale":"2024-05-27T07:30:05.536716+0000","last_undegraded":"2024-05-27T07:30:05.536716+0000","last_fullsized":"2024-05-27T07:30:05.536716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:50.489737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584247+0000","last_change":"2024-05-27T07:28:20.408117+0000","last_active":"2024-05-27T07:30:05.584247+0000","last_peered":"2024-05-27T07:30:05.584247+0000","last_clean":"2024-05-27T07:30:05.584247+0000","last_became_active":"2024-05-27T07:28:20.407723+0000","last_became_peered":"2024-05-27T07:28:20.407723+0000","last_unstale":"2024-05-27T07:30:05.584247+0000","last_undegraded":"2024-05-27T07:30:05.584247+0000","last_fullsized":"2024-05-27T07:30:05.584247+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:20:31.376656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122481+0000","last_change":"2024-05-27T07:28:24.490269+0000","last_active":"2024-05-27T07:30:05.122481+0000","last_peered":"2024-05-27T07:30:05.122481+0000","last_clean":"2024-05-27T07:30:05.122481+0000","last_became_active":"2024-05-27T07:28:24.488849+0000","last_became_peered":"2024-05-27T07:28:24.488849+0000","last_unstale":"2024-05-27T07:30:05.122481+0000","last_undegraded":"2024-05-27T07:30:05.122481+0000","last_fullsized":"2024-05-27T07:30:05.122481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:00.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536216+0000","last_change":"2024-05-27T07:28:37.908658+0000","last_active":"2024-05-27T07:30:05.536216+0000","last_peered":"2024-05-27T07:30:05.536216+0000","last_clean":"2024-05-27T07:30:05.536216+0000","last_became_active":"2024-05-27T07:28:37.908056+0000","last_became_peered":"2024-05-27T07:28:37.908056+0000","last_unstale":"2024-05-27T07:30:05.536216+0000","last_undegraded":"2024-05-27T07:30:05.536216+0000","last_fullsized":"2024-05-27T07:30:05.536216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:31.853606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583453+0000","last_change":"2024-05-27T07:28:20.410944+0000","last_active":"2024-05-27T07:30:05.583453+0000","last_peered":"2024-05-27T07:30:05.583453+0000","last_clean":"2024-05-27T07:30:05.583453+0000","last_became_active":"2024-05-27T07:28:20.409789+0000","last_became_peered":"2024-05-27T07:28:20.409789+0000","last_unstale":"2024-05-27T07:30:05.583453+0000","last_undegraded":"2024-05-27T07:30:05.583453+0000","last_fullsized":"2024-05-27T07:30:05.583453+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:53:34.962397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583982+0000","last_change":"2024-05-27T07:28:24.482828+0000","last_active":"2024-05-27T07:30:05.583982+0000","last_peered":"2024-05-27T07:30:05.583982+0000","last_clean":"2024-05-27T07:30:05.583982+0000","last_became_active":"2024-05-27T07:28:24.482475+0000","last_became_peered":"2024-05-27T07:28:24.482475+0000","last_unstale":"2024-05-27T07:30:05.583982+0000","last_undegraded":"2024-05-27T07:30:05.583982+0000","last_fullsized":"2024-05-27T07:30:05.583982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:53:32.064441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120474+0000","last_change":"2024-05-27T07:29:04.905734+0000","last_active":"2024-05-27T07:30:05.120474+0000","last_peered":"2024-05-27T07:30:05.120474+0000","last_clean":"2024-05-27T07:30:05.120474+0000","last_became_active":"2024-05-27T07:29:04.902387+0000","last_became_peered":"2024-05-27T07:29:04.902387+0000","last_unstale":"2024-05-27T07:30:05.120474+0000","last_undegraded":"2024-05-27T07:30:05.120474+0000","last_fullsized":"2024-05-27T07:30:05.120474+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:35.993193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":89,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:29.143146+0000","last_change":"2024-05-27T07:28:20.412258+0000","last_active":"2024-05-27T07:30:29.143146+0000","last_peered":"2024-05-27T07:30:29.143146+0000","last_clean":"2024-05-27T07:30:29.143146+0000","last_became_active":"2024-05-27T07:28:20.411764+0000","last_became_peered":"2024-05-27T07:28:20.411764+0000","last_unstale":"2024-05-27T07:30:29.143146+0000","last_undegraded":"2024-05-27T07:30:29.143146+0000","last_fullsized":"2024-05-27T07:30:29.143146+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:38.462887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581379+0000","last_change":"2024-05-27T07:28:24.482179+0000","last_active":"2024-05-27T07:30:05.581379+0000","last_peered":"2024-05-27T07:30:05.581379+0000","last_clean":"2024-05-27T07:30:05.581379+0000","last_became_active":"2024-05-27T07:28:24.480682+0000","last_became_peered":"2024-05-27T07:28:24.480682+0000","last_unstale":"2024-05-27T07:30:05.581379+0000","last_undegraded":"2024-05-27T07:30:05.581379+0000","last_fullsized":"2024-05-27T07:30:05.581379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:51.403862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119762+0000","last_change":"2024-05-27T07:28:37.903716+0000","last_active":"2024-05-27T07:30:05.119762+0000","last_peered":"2024-05-27T07:30:05.119762+0000","last_clean":"2024-05-27T07:30:05.119762+0000","last_became_active":"2024-05-27T07:28:37.901083+0000","last_became_peered":"2024-05-27T07:28:37.901083+0000","last_unstale":"2024-05-27T07:30:05.119762+0000","last_undegraded":"2024-05-27T07:30:05.119762+0000","last_fullsized":"2024-05-27T07:30:05.119762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:44:11.606711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578100+0000","last_change":"2024-05-27T07:28:20.392441+0000","last_active":"2024-05-27T07:30:05.578100+0000","last_peered":"2024-05-27T07:30:05.578100+0000","last_clean":"2024-05-27T07:30:05.578100+0000","last_became_active":"2024-05-27T07:28:20.390983+0000","last_became_peered":"2024-05-27T07:28:20.390983+0000","last_unstale":"2024-05-27T07:30:05.578100+0000","last_undegraded":"2024-05-27T07:30:05.578100+0000","last_fullsized":"2024-05-27T07:30:05.578100+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:33:30.338043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.520219+0000","last_change":"2024-05-27T07:30:05.518915+0000","last_active":"2024-05-27T07:30:05.520219+0000","last_peered":"2024-05-27T07:30:05.520219+0000","last_clean":"2024-05-27T07:30:05.520219+0000","last_became_active":"2024-05-27T07:30:05.517894+0000","last_became_peered":"2024-05-27T07:30:05.517894+0000","last_unstale":"2024-05-27T07:30:05.520219+0000","last_undegraded":"2024-05-27T07:30:05.520219+0000","last_fullsized":"2024-05-27T07:30:05.520219+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:51.353713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,7],"acting":[9,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580815+0000","last_change":"2024-05-27T07:28:37.913352+0000","last_active":"2024-05-27T07:30:05.580815+0000","last_peered":"2024-05-27T07:30:05.580815+0000","last_clean":"2024-05-27T07:30:05.580815+0000","last_became_active":"2024-05-27T07:28:37.913027+0000","last_became_peered":"2024-05-27T07:28:37.913027+0000","last_unstale":"2024-05-27T07:30:05.580815+0000","last_undegraded":"2024-05-27T07:30:05.580815+0000","last_fullsized":"2024-05-27T07:30:05.580815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:03.485594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.404529+0000","last_change":"2024-05-27T07:28:20.406094+0000","last_active":"2024-05-27T07:30:05.404529+0000","last_peered":"2024-05-27T07:30:05.404529+0000","last_clean":"2024-05-27T07:30:05.404529+0000","last_became_active":"2024-05-27T07:28:20.405641+0000","last_became_peered":"2024-05-27T07:28:20.405641+0000","last_unstale":"2024-05-27T07:30:05.404529+0000","last_undegraded":"2024-05-27T07:30:05.404529+0000","last_fullsized":"2024-05-27T07:30:05.404529+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:16.598743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119613+0000","last_change":"2024-05-27T07:28:24.474696+0000","last_active":"2024-05-27T07:30:05.119613+0000","last_peered":"2024-05-27T07:30:05.119613+0000","last_clean":"2024-05-27T07:30:05.119613+0000","last_became_active":"2024-05-27T07:28:24.474319+0000","last_became_peered":"2024-05-27T07:28:24.474319+0000","last_unstale":"2024-05-27T07:30:05.119613+0000","last_undegraded":"2024-05-27T07:30:05.119613+0000","last_fullsized":"2024-05-27T07:30:05.119613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:53.010898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576072+0000","last_change":"2024-05-27T07:28:37.910386+0000","last_active":"2024-05-27T07:30:05.576072+0000","last_peered":"2024-05-27T07:30:05.576072+0000","last_clean":"2024-05-27T07:30:05.576072+0000","last_became_active":"2024-05-27T07:28:37.909941+0000","last_became_peered":"2024-05-27T07:28:37.909941+0000","last_unstale":"2024-05-27T07:30:05.576072+0000","last_undegraded":"2024-05-27T07:30:05.576072+0000","last_fullsized":"2024-05-27T07:30:05.576072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:15.820318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581844+0000","last_change":"2024-05-27T07:28:20.411855+0000","last_active":"2024-05-27T07:30:05.581844+0000","last_peered":"2024-05-27T07:30:05.581844+0000","last_clean":"2024-05-27T07:30:05.581844+0000","last_became_active":"2024-05-27T07:28:20.411169+0000","last_became_peered":"2024-05-27T07:28:20.411169+0000","last_unstale":"2024-05-27T07:30:05.581844+0000","last_undegraded":"2024-05-27T07:30:05.581844+0000","last_fullsized":"2024-05-27T07:30:05.581844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:37.278507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.125456+0000","last_change":"2024-05-27T07:28:24.485799+0000","last_active":"2024-05-27T07:30:05.125456+0000","last_peered":"2024-05-27T07:30:05.125456+0000","last_clean":"2024-05-27T07:30:05.125456+0000","last_became_active":"2024-05-27T07:28:24.485395+0000","last_became_peered":"2024-05-27T07:28:24.485395+0000","last_unstale":"2024-05-27T07:30:05.125456+0000","last_undegraded":"2024-05-27T07:30:05.125456+0000","last_fullsized":"2024-05-27T07:30:05.125456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:11.178129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119918+0000","last_change":"2024-05-27T07:28:37.898929+0000","last_active":"2024-05-27T07:30:05.119918+0000","last_peered":"2024-05-27T07:30:05.119918+0000","last_clean":"2024-05-27T07:30:05.119918+0000","last_became_active":"2024-05-27T07:28:37.898371+0000","last_became_peered":"2024-05-27T07:28:37.898371+0000","last_unstale":"2024-05-27T07:30:05.119918+0000","last_undegraded":"2024-05-27T07:30:05.119918+0000","last_fullsized":"2024-05-27T07:30:05.119918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:48.867729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":83,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:27.138496+0000","last_change":"2024-05-27T07:28:20.407117+0000","last_active":"2024-05-27T07:30:27.138496+0000","last_peered":"2024-05-27T07:30:27.138496+0000","last_clean":"2024-05-27T07:30:27.138496+0000","last_became_active":"2024-05-27T07:28:20.403562+0000","last_became_peered":"2024-05-27T07:28:20.403562+0000","last_unstale":"2024-05-27T07:30:27.138496+0000","last_undegraded":"2024-05-27T07:30:27.138496+0000","last_fullsized":"2024-05-27T07:30:27.138496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:25.903886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.123411+0000","last_change":"2024-05-27T07:28:24.489771+0000","last_active":"2024-05-27T07:30:05.123411+0000","last_peered":"2024-05-27T07:30:05.123411+0000","last_clean":"2024-05-27T07:30:05.123411+0000","last_became_active":"2024-05-27T07:28:24.489163+0000","last_became_peered":"2024-05-27T07:28:24.489163+0000","last_unstale":"2024-05-27T07:30:05.123411+0000","last_undegraded":"2024-05-27T07:30:05.123411+0000","last_fullsized":"2024-05-27T07:30:05.123411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:08.053634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.123519+0000","last_change":"2024-05-27T07:28:37.914598+0000","last_active":"2024-05-27T07:30:05.123519+0000","last_peered":"2024-05-27T07:30:05.123519+0000","last_clean":"2024-05-27T07:30:05.123519+0000","last_became_active":"2024-05-27T07:28:37.913916+0000","last_became_peered":"2024-05-27T07:28:37.913916+0000","last_unstale":"2024-05-27T07:30:05.123519+0000","last_undegraded":"2024-05-27T07:30:05.123519+0000","last_fullsized":"2024-05-27T07:30:05.123519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:01.475789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516991+0000","last_change":"2024-05-27T07:29:04.901366+0000","last_active":"2024-05-27T07:30:05.516991+0000","last_peered":"2024-05-27T07:30:05.516991+0000","last_clean":"2024-05-27T07:30:05.516991+0000","last_became_active":"2024-05-27T07:29:04.900355+0000","last_became_peered":"2024-05-27T07:29:04.900355+0000","last_unstale":"2024-05-27T07:30:05.516991+0000","last_undegraded":"2024-05-27T07:30:05.516991+0000","last_fullsized":"2024-05-27T07:30:05.516991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:10:03.349345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576630+0000","last_change":"2024-05-27T07:28:24.477771+0000","last_active":"2024-05-27T07:30:05.576630+0000","last_peered":"2024-05-27T07:30:05.576630+0000","last_clean":"2024-05-27T07:30:05.576630+0000","last_became_active":"2024-05-27T07:28:24.474179+0000","last_became_peered":"2024-05-27T07:28:24.474179+0000","last_unstale":"2024-05-27T07:30:05.576630+0000","last_undegraded":"2024-05-27T07:30:05.576630+0000","last_fullsized":"2024-05-27T07:30:05.576630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:08:04.218264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537214+0000","last_change":"2024-05-27T07:29:04.896583+0000","last_active":"2024-05-27T07:30:05.537214+0000","last_peered":"2024-05-27T07:30:05.537214+0000","last_clean":"2024-05-27T07:30:05.537214+0000","last_became_active":"2024-05-27T07:29:04.894118+0000","last_became_peered":"2024-05-27T07:29:04.894118+0000","last_unstale":"2024-05-27T07:30:05.537214+0000","last_undegraded":"2024-05-27T07:30:05.537214+0000","last_fullsized":"2024-05-27T07:30:05.537214+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:17.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583441+0000","last_change":"2024-05-27T07:28:20.466851+0000","last_active":"2024-05-27T07:30:05.583441+0000","last_peered":"2024-05-27T07:30:05.583441+0000","last_clean":"2024-05-27T07:30:05.583441+0000","last_became_active":"2024-05-27T07:28:20.465133+0000","last_became_peered":"2024-05-27T07:28:20.465133+0000","last_unstale":"2024-05-27T07:30:05.583441+0000","last_undegraded":"2024-05-27T07:30:05.583441+0000","last_fullsized":"2024-05-27T07:30:05.583441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:11.025588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.123045+0000","last_change":"2024-05-27T07:28:24.469924+0000","last_active":"2024-05-27T07:30:05.123045+0000","last_peered":"2024-05-27T07:30:05.123045+0000","last_clean":"2024-05-27T07:30:05.123045+0000","last_became_active":"2024-05-27T07:28:24.469559+0000","last_became_peered":"2024-05-27T07:28:24.469559+0000","last_unstale":"2024-05-27T07:30:05.123045+0000","last_undegraded":"2024-05-27T07:30:05.123045+0000","last_fullsized":"2024-05-27T07:30:05.123045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:57.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.124945+0000","last_change":"2024-05-27T07:28:37.904549+0000","last_active":"2024-05-27T07:30:05.124945+0000","last_peered":"2024-05-27T07:30:05.124945+0000","last_clean":"2024-05-27T07:30:05.124945+0000","last_became_active":"2024-05-27T07:28:37.903882+0000","last_became_peered":"2024-05-27T07:28:37.903882+0000","last_unstale":"2024-05-27T07:30:05.124945+0000","last_undegraded":"2024-05-27T07:30:05.124945+0000","last_fullsized":"2024-05-27T07:30:05.124945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:41.607641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":161,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:28.159367+0000","last_change":"2024-05-27T07:28:20.410025+0000","last_active":"2024-05-27T07:30:28.159367+0000","last_peered":"2024-05-27T07:30:28.159367+0000","last_clean":"2024-05-27T07:30:28.159367+0000","last_became_active":"2024-05-27T07:28:20.409115+0000","last_became_peered":"2024-05-27T07:28:20.409115+0000","last_unstale":"2024-05-27T07:30:28.159367+0000","last_undegraded":"2024-05-27T07:30:28.159367+0000","last_fullsized":"2024-05-27T07:30:28.159367+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:56.834059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67819,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":83,"num_read_kb":108,"num_write":35,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584860+0000","last_change":"2024-05-27T07:28:24.469609+0000","last_active":"2024-05-27T07:30:05.584860+0000","last_peered":"2024-05-27T07:30:05.584860+0000","last_clean":"2024-05-27T07:30:05.584860+0000","last_became_active":"2024-05-27T07:28:24.469192+0000","last_became_peered":"2024-05-27T07:28:24.469192+0000","last_unstale":"2024-05-27T07:30:05.584860+0000","last_undegraded":"2024-05-27T07:30:05.584860+0000","last_fullsized":"2024-05-27T07:30:05.584860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:32:59.006298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584929+0000","last_change":"2024-05-27T07:28:37.900745+0000","last_active":"2024-05-27T07:30:05.584929+0000","last_peered":"2024-05-27T07:30:05.584929+0000","last_clean":"2024-05-27T07:30:05.584929+0000","last_became_active":"2024-05-27T07:28:37.900205+0000","last_became_peered":"2024-05-27T07:28:37.900205+0000","last_unstale":"2024-05-27T07:30:05.584929+0000","last_undegraded":"2024-05-27T07:30:05.584929+0000","last_fullsized":"2024-05-27T07:30:05.584929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:52:32.235201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120148+0000","last_change":"2024-05-27T07:28:20.413380+0000","last_active":"2024-05-27T07:30:05.120148+0000","last_peered":"2024-05-27T07:30:05.120148+0000","last_clean":"2024-05-27T07:30:05.120148+0000","last_became_active":"2024-05-27T07:28:20.412839+0000","last_became_peered":"2024-05-27T07:28:20.412839+0000","last_unstale":"2024-05-27T07:30:05.120148+0000","last_undegraded":"2024-05-27T07:30:05.120148+0000","last_fullsized":"2024-05-27T07:30:05.120148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:57.481203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.405393+0000","last_change":"2024-05-27T07:28:24.474228+0000","last_active":"2024-05-27T07:30:05.405393+0000","last_peered":"2024-05-27T07:30:05.405393+0000","last_clean":"2024-05-27T07:30:05.405393+0000","last_became_active":"2024-05-27T07:28:24.473833+0000","last_became_peered":"2024-05-27T07:28:24.473833+0000","last_unstale":"2024-05-27T07:30:05.405393+0000","last_undegraded":"2024-05-27T07:30:05.405393+0000","last_fullsized":"2024-05-27T07:30:05.405393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:52.456730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580276+0000","last_change":"2024-05-27T07:28:37.917630+0000","last_active":"2024-05-27T07:30:05.580276+0000","last_peered":"2024-05-27T07:30:05.580276+0000","last_clean":"2024-05-27T07:30:05.580276+0000","last_became_active":"2024-05-27T07:28:37.917253+0000","last_became_peered":"2024-05-27T07:28:37.917253+0000","last_unstale":"2024-05-27T07:30:05.580276+0000","last_undegraded":"2024-05-27T07:30:05.580276+0000","last_fullsized":"2024-05-27T07:30:05.580276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:05.069946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584581+0000","last_change":"2024-05-27T07:28:20.408988+0000","last_active":"2024-05-27T07:30:05.584581+0000","last_peered":"2024-05-27T07:30:05.584581+0000","last_clean":"2024-05-27T07:30:05.584581+0000","last_became_active":"2024-05-27T07:28:20.408313+0000","last_became_peered":"2024-05-27T07:28:20.408313+0000","last_unstale":"2024-05-27T07:30:05.584581+0000","last_undegraded":"2024-05-27T07:30:05.584581+0000","last_fullsized":"2024-05-27T07:30:05.584581+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:17.480412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.517051+0000","last_change":"2024-05-27T07:28:24.480096+0000","last_active":"2024-05-27T07:30:05.517051+0000","last_peered":"2024-05-27T07:30:05.517051+0000","last_clean":"2024-05-27T07:30:05.517051+0000","last_became_active":"2024-05-27T07:28:24.477988+0000","last_became_peered":"2024-05-27T07:28:24.477988+0000","last_unstale":"2024-05-27T07:30:05.517051+0000","last_undegraded":"2024-05-27T07:30:05.517051+0000","last_fullsized":"2024-05-27T07:30:05.517051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:13:59.860458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127016+0000","last_change":"2024-05-27T07:28:37.917518+0000","last_active":"2024-05-27T07:30:05.127016+0000","last_peered":"2024-05-27T07:30:05.127016+0000","last_clean":"2024-05-27T07:30:05.127016+0000","last_became_active":"2024-05-27T07:28:37.916971+0000","last_became_peered":"2024-05-27T07:28:37.916971+0000","last_unstale":"2024-05-27T07:30:05.127016+0000","last_undegraded":"2024-05-27T07:30:05.127016+0000","last_fullsized":"2024-05-27T07:30:05.127016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:17:53.067747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":87,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:29.146912+0000","last_change":"2024-05-27T07:28:20.416766+0000","last_active":"2024-05-27T07:30:29.146912+0000","last_peered":"2024-05-27T07:30:29.146912+0000","last_clean":"2024-05-27T07:30:29.146912+0000","last_became_active":"2024-05-27T07:28:20.416297+0000","last_became_peered":"2024-05-27T07:28:20.416297+0000","last_unstale":"2024-05-27T07:30:29.146912+0000","last_undegraded":"2024-05-27T07:30:29.146912+0000","last_fullsized":"2024-05-27T07:30:29.146912+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:02.841494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515837+0000","last_change":"2024-05-27T07:28:24.486869+0000","last_active":"2024-05-27T07:30:05.515837+0000","last_peered":"2024-05-27T07:30:05.515837+0000","last_clean":"2024-05-27T07:30:05.515837+0000","last_became_active":"2024-05-27T07:28:24.486451+0000","last_became_peered":"2024-05-27T07:28:24.486451+0000","last_unstale":"2024-05-27T07:30:05.515837+0000","last_undegraded":"2024-05-27T07:30:05.515837+0000","last_fullsized":"2024-05-27T07:30:05.515837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:10.722659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406667+0000","last_change":"2024-05-27T07:29:04.901632+0000","last_active":"2024-05-27T07:30:05.406667+0000","last_peered":"2024-05-27T07:30:05.406667+0000","last_clean":"2024-05-27T07:30:05.406667+0000","last_became_active":"2024-05-27T07:29:04.900203+0000","last_became_peered":"2024-05-27T07:29:04.900203+0000","last_unstale":"2024-05-27T07:30:05.406667+0000","last_undegraded":"2024-05-27T07:30:05.406667+0000","last_fullsized":"2024-05-27T07:30:05.406667+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:57.834889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579936+0000","last_change":"2024-05-27T07:28:20.468162+0000","last_active":"2024-05-27T07:30:05.579936+0000","last_peered":"2024-05-27T07:30:05.579936+0000","last_clean":"2024-05-27T07:30:05.579936+0000","last_became_active":"2024-05-27T07:28:20.465226+0000","last_became_peered":"2024-05-27T07:28:20.465226+0000","last_unstale":"2024-05-27T07:30:05.579936+0000","last_undegraded":"2024-05-27T07:30:05.579936+0000","last_fullsized":"2024-05-27T07:30:05.579936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:21:34.110356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582781+0000","last_change":"2024-05-27T07:28:24.485053+0000","last_active":"2024-05-27T07:30:05.582781+0000","last_peered":"2024-05-27T07:30:05.582781+0000","last_clean":"2024-05-27T07:30:05.582781+0000","last_became_active":"2024-05-27T07:28:24.484391+0000","last_became_peered":"2024-05-27T07:28:24.484391+0000","last_unstale":"2024-05-27T07:30:05.582781+0000","last_undegraded":"2024-05-27T07:30:05.582781+0000","last_fullsized":"2024-05-27T07:30:05.582781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:05.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582670+0000","last_change":"2024-05-27T07:28:37.904731+0000","last_active":"2024-05-27T07:30:05.582670+0000","last_peered":"2024-05-27T07:30:05.582670+0000","last_clean":"2024-05-27T07:30:05.582670+0000","last_became_active":"2024-05-27T07:28:37.904284+0000","last_became_peered":"2024-05-27T07:28:37.904284+0000","last_unstale":"2024-05-27T07:30:05.582670+0000","last_undegraded":"2024-05-27T07:30:05.582670+0000","last_fullsized":"2024-05-27T07:30:05.582670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:23.198465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.585569+0000","last_change":"2024-05-27T07:28:20.406829+0000","last_active":"2024-05-27T07:30:05.585569+0000","last_peered":"2024-05-27T07:30:05.585569+0000","last_clean":"2024-05-27T07:30:05.585569+0000","last_became_active":"2024-05-27T07:28:20.405497+0000","last_became_peered":"2024-05-27T07:28:20.405497+0000","last_unstale":"2024-05-27T07:30:05.585569+0000","last_undegraded":"2024-05-27T07:30:05.585569+0000","last_fullsized":"2024-05-27T07:30:05.585569+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:14.225630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581920+0000","last_change":"2024-05-27T07:28:24.488023+0000","last_active":"2024-05-27T07:30:05.581920+0000","last_peered":"2024-05-27T07:30:05.581920+0000","last_clean":"2024-05-27T07:30:05.581920+0000","last_became_active":"2024-05-27T07:28:24.487652+0000","last_became_peered":"2024-05-27T07:28:24.487652+0000","last_unstale":"2024-05-27T07:30:05.581920+0000","last_undegraded":"2024-05-27T07:30:05.581920+0000","last_fullsized":"2024-05-27T07:30:05.581920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:25.951157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581074+0000","last_change":"2024-05-27T07:28:37.968034+0000","last_active":"2024-05-27T07:30:05.581074+0000","last_peered":"2024-05-27T07:30:05.581074+0000","last_clean":"2024-05-27T07:30:05.581074+0000","last_became_active":"2024-05-27T07:28:37.967127+0000","last_became_peered":"2024-05-27T07:28:37.967127+0000","last_unstale":"2024-05-27T07:30:05.581074+0000","last_undegraded":"2024-05-27T07:30:05.581074+0000","last_fullsized":"2024-05-27T07:30:05.581074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:49.353423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.126685+0000","last_change":"2024-05-27T07:28:20.465782+0000","last_active":"2024-05-27T07:30:05.126685+0000","last_peered":"2024-05-27T07:30:05.126685+0000","last_clean":"2024-05-27T07:30:05.126685+0000","last_became_active":"2024-05-27T07:28:20.465134+0000","last_became_peered":"2024-05-27T07:28:20.465134+0000","last_unstale":"2024-05-27T07:30:05.126685+0000","last_undegraded":"2024-05-27T07:30:05.126685+0000","last_fullsized":"2024-05-27T07:30:05.126685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:17.992092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579036+0000","last_change":"2024-05-27T07:28:24.484691+0000","last_active":"2024-05-27T07:30:05.579036+0000","last_peered":"2024-05-27T07:30:05.579036+0000","last_clean":"2024-05-27T07:30:05.579036+0000","last_became_active":"2024-05-27T07:28:24.484249+0000","last_became_peered":"2024-05-27T07:28:24.484249+0000","last_unstale":"2024-05-27T07:30:05.579036+0000","last_undegraded":"2024-05-27T07:30:05.579036+0000","last_fullsized":"2024-05-27T07:30:05.579036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:25.753276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.405759+0000","last_change":"2024-05-27T07:28:37.907849+0000","last_active":"2024-05-27T07:30:05.405759+0000","last_peered":"2024-05-27T07:30:05.405759+0000","last_clean":"2024-05-27T07:30:05.405759+0000","last_became_active":"2024-05-27T07:28:37.907105+0000","last_became_peered":"2024-05-27T07:28:37.907105+0000","last_unstale":"2024-05-27T07:30:05.405759+0000","last_undegraded":"2024-05-27T07:30:05.405759+0000","last_fullsized":"2024-05-27T07:30:05.405759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:54.075240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516795+0000","last_change":"2024-05-27T07:28:20.411302+0000","last_active":"2024-05-27T07:30:05.516795+0000","last_peered":"2024-05-27T07:30:05.516795+0000","last_clean":"2024-05-27T07:30:05.516795+0000","last_became_active":"2024-05-27T07:28:20.409144+0000","last_became_peered":"2024-05-27T07:28:20.409144+0000","last_unstale":"2024-05-27T07:30:05.516795+0000","last_undegraded":"2024-05-27T07:30:05.516795+0000","last_fullsized":"2024-05-27T07:30:05.516795+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:49.985962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578341+0000","last_change":"2024-05-27T07:28:24.487742+0000","last_active":"2024-05-27T07:30:05.578341+0000","last_peered":"2024-05-27T07:30:05.578341+0000","last_clean":"2024-05-27T07:30:05.578341+0000","last_became_active":"2024-05-27T07:28:24.485279+0000","last_became_peered":"2024-05-27T07:28:24.485279+0000","last_unstale":"2024-05-27T07:30:05.578341+0000","last_undegraded":"2024-05-27T07:30:05.578341+0000","last_fullsized":"2024-05-27T07:30:05.578341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:15.049572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406610+0000","last_change":"2024-05-27T07:28:37.968780+0000","last_active":"2024-05-27T07:30:05.406610+0000","last_peered":"2024-05-27T07:30:05.406610+0000","last_clean":"2024-05-27T07:30:05.406610+0000","last_became_active":"2024-05-27T07:28:37.968052+0000","last_became_peered":"2024-05-27T07:28:37.968052+0000","last_unstale":"2024-05-27T07:30:05.406610+0000","last_undegraded":"2024-05-27T07:30:05.406610+0000","last_fullsized":"2024-05-27T07:30:05.406610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:51:26.787969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583127+0000","last_change":"2024-05-27T07:28:24.485280+0000","last_active":"2024-05-27T07:30:05.583127+0000","last_peered":"2024-05-27T07:30:05.583127+0000","last_clean":"2024-05-27T07:30:05.583127+0000","last_became_active":"2024-05-27T07:28:24.484882+0000","last_became_peered":"2024-05-27T07:28:24.484882+0000","last_unstale":"2024-05-27T07:30:05.583127+0000","last_undegraded":"2024-05-27T07:30:05.583127+0000","last_fullsized":"2024-05-27T07:30:05.583127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:28.643857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":86,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:29.145893+0000","last_change":"2024-05-27T07:28:20.462593+0000","last_active":"2024-05-27T07:30:29.145893+0000","last_peered":"2024-05-27T07:30:29.145893+0000","last_clean":"2024-05-27T07:30:29.145893+0000","last_became_active":"2024-05-27T07:28:20.461128+0000","last_became_peered":"2024-05-27T07:28:20.461128+0000","last_unstale":"2024-05-27T07:30:29.145893+0000","last_undegraded":"2024-05-27T07:30:29.145893+0000","last_fullsized":"2024-05-27T07:30:29.145893+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:24:42.013315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516454+0000","last_change":"2024-05-27T07:28:37.911546+0000","last_active":"2024-05-27T07:30:05.516454+0000","last_peered":"2024-05-27T07:30:05.516454+0000","last_clean":"2024-05-27T07:30:05.516454+0000","last_became_active":"2024-05-27T07:28:37.910777+0000","last_became_peered":"2024-05-27T07:28:37.910777+0000","last_unstale":"2024-05-27T07:30:05.516454+0000","last_undegraded":"2024-05-27T07:30:05.516454+0000","last_fullsized":"2024-05-27T07:30:05.516454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:13.344317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120233+0000","last_change":"2024-05-27T07:28:24.483946+0000","last_active":"2024-05-27T07:30:05.120233+0000","last_peered":"2024-05-27T07:30:05.120233+0000","last_clean":"2024-05-27T07:30:05.120233+0000","last_became_active":"2024-05-27T07:28:24.481424+0000","last_became_peered":"2024-05-27T07:28:24.481424+0000","last_unstale":"2024-05-27T07:30:05.120233+0000","last_undegraded":"2024-05-27T07:30:05.120233+0000","last_fullsized":"2024-05-27T07:30:05.120233+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:08.254046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536932+0000","last_change":"2024-05-27T07:28:20.414250+0000","last_active":"2024-05-27T07:30:05.536932+0000","last_peered":"2024-05-27T07:30:05.536932+0000","last_clean":"2024-05-27T07:30:05.536932+0000","last_became_active":"2024-05-27T07:28:20.413676+0000","last_became_peered":"2024-05-27T07:28:20.413676+0000","last_unstale":"2024-05-27T07:30:05.536932+0000","last_undegraded":"2024-05-27T07:30:05.536932+0000","last_fullsized":"2024-05-27T07:30:05.536932+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:10.226771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515428+0000","last_change":"2024-05-27T07:26:26.055626+0000","last_active":"2024-05-27T07:30:05.515428+0000","last_peered":"2024-05-27T07:30:05.515428+0000","last_clean":"2024-05-27T07:30:05.515428+0000","last_became_active":"2024-05-27T07:26:26.037376+0000","last_became_peered":"2024-05-27T07:26:26.037376+0000","last_unstale":"2024-05-27T07:30:05.515428+0000","last_undegraded":"2024-05-27T07:30:05.515428+0000","last_fullsized":"2024-05-27T07:30:05.515428+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_clean_scrub_stamp":"2024-05-27T07:21:26.641949+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:16.741856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584257+0000","last_change":"2024-05-27T07:28:37.973078+0000","last_active":"2024-05-27T07:30:05.584257+0000","last_peered":"2024-05-27T07:30:05.584257+0000","last_clean":"2024-05-27T07:30:05.584257+0000","last_became_active":"2024-05-27T07:28:37.972381+0000","last_became_peered":"2024-05-27T07:28:37.972381+0000","last_unstale":"2024-05-27T07:30:05.584257+0000","last_undegraded":"2024-05-27T07:30:05.584257+0000","last_fullsized":"2024-05-27T07:30:05.584257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:18:16.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.404315+0000","last_change":"2024-05-27T07:28:24.483512+0000","last_active":"2024-05-27T07:30:05.404315+0000","last_peered":"2024-05-27T07:30:05.404315+0000","last_clean":"2024-05-27T07:30:05.404315+0000","last_became_active":"2024-05-27T07:28:24.482655+0000","last_became_peered":"2024-05-27T07:28:24.482655+0000","last_unstale":"2024-05-27T07:30:05.404315+0000","last_undegraded":"2024-05-27T07:30:05.404315+0000","last_fullsized":"2024-05-27T07:30:05.404315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:01:07.898504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515986+0000","last_change":"2024-05-27T07:28:20.408953+0000","last_active":"2024-05-27T07:30:05.515986+0000","last_peered":"2024-05-27T07:30:05.515986+0000","last_clean":"2024-05-27T07:30:05.515986+0000","last_became_active":"2024-05-27T07:28:20.407631+0000","last_became_peered":"2024-05-27T07:28:20.407631+0000","last_unstale":"2024-05-27T07:30:05.515986+0000","last_undegraded":"2024-05-27T07:30:05.515986+0000","last_fullsized":"2024-05-27T07:30:05.515986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:43.912338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579924+0000","last_change":"2024-05-27T07:28:37.907441+0000","last_active":"2024-05-27T07:30:05.579924+0000","last_peered":"2024-05-27T07:30:05.579924+0000","last_clean":"2024-05-27T07:30:05.579924+0000","last_became_active":"2024-05-27T07:28:37.906928+0000","last_became_peered":"2024-05-27T07:28:37.906928+0000","last_unstale":"2024-05-27T07:30:05.579924+0000","last_undegraded":"2024-05-27T07:30:05.579924+0000","last_fullsized":"2024-05-27T07:30:05.579924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:28:32.345826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584754+0000","last_change":"2024-05-27T07:28:20.467408+0000","last_active":"2024-05-27T07:30:05.584754+0000","last_peered":"2024-05-27T07:30:05.584754+0000","last_clean":"2024-05-27T07:30:05.584754+0000","last_became_active":"2024-05-27T07:28:20.466064+0000","last_became_peered":"2024-05-27T07:28:20.466064+0000","last_unstale":"2024-05-27T07:30:05.584754+0000","last_undegraded":"2024-05-27T07:30:05.584754+0000","last_fullsized":"2024-05-27T07:30:05.584754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:15:51.449514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406118+0000","last_change":"2024-05-27T07:28:24.478475+0000","last_active":"2024-05-27T07:30:05.406118+0000","last_peered":"2024-05-27T07:30:05.406118+0000","last_clean":"2024-05-27T07:30:05.406118+0000","last_became_active":"2024-05-27T07:28:24.477936+0000","last_became_peered":"2024-05-27T07:28:24.477936+0000","last_unstale":"2024-05-27T07:30:05.406118+0000","last_undegraded":"2024-05-27T07:30:05.406118+0000","last_fullsized":"2024-05-27T07:30:05.406118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:53:17.500040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119635+0000","last_change":"2024-05-27T07:28:37.968192+0000","last_active":"2024-05-27T07:30:05.119635+0000","last_peered":"2024-05-27T07:30:05.119635+0000","last_clean":"2024-05-27T07:30:05.119635+0000","last_became_active":"2024-05-27T07:28:37.967603+0000","last_became_peered":"2024-05-27T07:28:37.967603+0000","last_unstale":"2024-05-27T07:30:05.119635+0000","last_undegraded":"2024-05-27T07:30:05.119635+0000","last_fullsized":"2024-05-27T07:30:05.119635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:53:36.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'12","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:23.659869+0000","last_change":"2024-05-27T07:29:04.901062+0000","last_active":"2024-05-27T07:30:23.659869+0000","last_peered":"2024-05-27T07:30:23.659869+0000","last_clean":"2024-05-27T07:30:23.659869+0000","last_became_active":"2024-05-27T07:29:04.892060+0000","last_became_peered":"2024-05-27T07:29:04.892060+0000","last_unstale":"2024-05-27T07:30:23.659869+0000","last_undegraded":"2024-05-27T07:30:23.659869+0000","last_fullsized":"2024-05-27T07:30:23.659869+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:27:37.894263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.121792+0000","last_change":"2024-05-27T07:28:24.477096+0000","last_active":"2024-05-27T07:30:05.121792+0000","last_peered":"2024-05-27T07:30:05.121792+0000","last_clean":"2024-05-27T07:30:05.121792+0000","last_became_active":"2024-05-27T07:28:24.476712+0000","last_became_peered":"2024-05-27T07:28:24.476712+0000","last_unstale":"2024-05-27T07:30:05.121792+0000","last_undegraded":"2024-05-27T07:30:05.121792+0000","last_fullsized":"2024-05-27T07:30:05.121792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:31:58.010761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119189+0000","last_change":"2024-05-27T07:28:37.894013+0000","last_active":"2024-05-27T07:30:05.119189+0000","last_peered":"2024-05-27T07:30:05.119189+0000","last_clean":"2024-05-27T07:30:05.119189+0000","last_became_active":"2024-05-27T07:28:37.893345+0000","last_became_peered":"2024-05-27T07:28:37.893345+0000","last_unstale":"2024-05-27T07:30:05.119189+0000","last_undegraded":"2024-05-27T07:30:05.119189+0000","last_fullsized":"2024-05-27T07:30:05.119189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:26.917731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537802+0000","last_change":"2024-05-27T07:28:20.744087+0000","last_active":"2024-05-27T07:30:05.537802+0000","last_peered":"2024-05-27T07:30:05.537802+0000","last_clean":"2024-05-27T07:30:05.537802+0000","last_became_active":"2024-05-27T07:28:20.742883+0000","last_became_peered":"2024-05-27T07:28:20.742883+0000","last_unstale":"2024-05-27T07:30:05.537802+0000","last_undegraded":"2024-05-27T07:30:05.537802+0000","last_fullsized":"2024-05-27T07:30:05.537802+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:06.083348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576358+0000","last_change":"2024-05-27T07:28:24.480330+0000","last_active":"2024-05-27T07:30:05.576358+0000","last_peered":"2024-05-27T07:30:05.576358+0000","last_clean":"2024-05-27T07:30:05.576358+0000","last_became_active":"2024-05-27T07:28:24.479300+0000","last_became_peered":"2024-05-27T07:28:24.479300+0000","last_unstale":"2024-05-27T07:30:05.576358+0000","last_undegraded":"2024-05-27T07:30:05.576358+0000","last_fullsized":"2024-05-27T07:30:05.576358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:43.802273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537054+0000","last_change":"2024-05-27T07:28:37.973592+0000","last_active":"2024-05-27T07:30:05.537054+0000","last_peered":"2024-05-27T07:30:05.537054+0000","last_clean":"2024-05-27T07:30:05.537054+0000","last_became_active":"2024-05-27T07:28:37.972475+0000","last_became_peered":"2024-05-27T07:28:37.972475+0000","last_unstale":"2024-05-27T07:30:05.537054+0000","last_undegraded":"2024-05-27T07:30:05.537054+0000","last_fullsized":"2024-05-27T07:30:05.537054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:31.662384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122331+0000","last_change":"2024-05-27T07:29:04.890751+0000","last_active":"2024-05-27T07:30:05.122331+0000","last_peered":"2024-05-27T07:30:05.122331+0000","last_clean":"2024-05-27T07:30:05.122331+0000","last_became_active":"2024-05-27T07:29:04.889447+0000","last_became_peered":"2024-05-27T07:29:04.889447+0000","last_unstale":"2024-05-27T07:30:05.122331+0000","last_undegraded":"2024-05-27T07:30:05.122331+0000","last_fullsized":"2024-05-27T07:30:05.122331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:13:34.743691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580514+0000","last_change":"2024-05-27T07:28:24.474298+0000","last_active":"2024-05-27T07:30:05.580514+0000","last_peered":"2024-05-27T07:30:05.580514+0000","last_clean":"2024-05-27T07:30:05.580514+0000","last_became_active":"2024-05-27T07:28:24.473877+0000","last_became_peered":"2024-05-27T07:28:24.473877+0000","last_unstale":"2024-05-27T07:30:05.580514+0000","last_undegraded":"2024-05-27T07:30:05.580514+0000","last_fullsized":"2024-05-27T07:30:05.580514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:31.545883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127438+0000","last_change":"2024-05-27T07:28:37.905417+0000","last_active":"2024-05-27T07:30:05.127438+0000","last_peered":"2024-05-27T07:30:05.127438+0000","last_clean":"2024-05-27T07:30:05.127438+0000","last_became_active":"2024-05-27T07:28:37.904968+0000","last_became_peered":"2024-05-27T07:28:37.904968+0000","last_unstale":"2024-05-27T07:30:05.127438+0000","last_undegraded":"2024-05-27T07:30:05.127438+0000","last_fullsized":"2024-05-27T07:30:05.127438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:02.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120278+0000","last_change":"2024-05-27T07:28:20.392647+0000","last_active":"2024-05-27T07:30:05.120278+0000","last_peered":"2024-05-27T07:30:05.120278+0000","last_clean":"2024-05-27T07:30:05.120278+0000","last_became_active":"2024-05-27T07:28:20.392073+0000","last_became_peered":"2024-05-27T07:28:20.392073+0000","last_unstale":"2024-05-27T07:30:05.120278+0000","last_undegraded":"2024-05-27T07:30:05.120278+0000","last_fullsized":"2024-05-27T07:30:05.120278+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:18.777628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579061+0000","last_change":"2024-05-27T07:28:24.490522+0000","last_active":"2024-05-27T07:30:05.579061+0000","last_peered":"2024-05-27T07:30:05.579061+0000","last_clean":"2024-05-27T07:30:05.579061+0000","last_became_active":"2024-05-27T07:28:24.485228+0000","last_became_peered":"2024-05-27T07:28:24.485228+0000","last_unstale":"2024-05-27T07:30:05.579061+0000","last_undegraded":"2024-05-27T07:30:05.579061+0000","last_fullsized":"2024-05-27T07:30:05.579061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:14.221016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537587+0000","last_change":"2024-05-27T07:28:37.900449+0000","last_active":"2024-05-27T07:30:05.537587+0000","last_peered":"2024-05-27T07:30:05.537587+0000","last_clean":"2024-05-27T07:30:05.537587+0000","last_became_active":"2024-05-27T07:28:37.899526+0000","last_became_peered":"2024-05-27T07:28:37.899526+0000","last_unstale":"2024-05-27T07:30:05.537587+0000","last_undegraded":"2024-05-27T07:30:05.537587+0000","last_fullsized":"2024-05-27T07:30:05.537587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:29:49.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119900+0000","last_change":"2024-05-27T07:28:20.405236+0000","last_active":"2024-05-27T07:30:05.119900+0000","last_peered":"2024-05-27T07:30:05.119900+0000","last_clean":"2024-05-27T07:30:05.119900+0000","last_became_active":"2024-05-27T07:28:20.404619+0000","last_became_peered":"2024-05-27T07:28:20.404619+0000","last_unstale":"2024-05-27T07:30:05.119900+0000","last_undegraded":"2024-05-27T07:30:05.119900+0000","last_fullsized":"2024-05-27T07:30:05.119900+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:17.671250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120712+0000","last_change":"2024-05-27T07:28:24.490661+0000","last_active":"2024-05-27T07:30:05.120712+0000","last_peered":"2024-05-27T07:30:05.120712+0000","last_clean":"2024-05-27T07:30:05.120712+0000","last_became_active":"2024-05-27T07:28:24.487299+0000","last_became_peered":"2024-05-27T07:28:24.487299+0000","last_unstale":"2024-05-27T07:30:05.120712+0000","last_undegraded":"2024-05-27T07:30:05.120712+0000","last_fullsized":"2024-05-27T07:30:05.120712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:50:27.291721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536951+0000","last_change":"2024-05-27T07:28:37.970463+0000","last_active":"2024-05-27T07:30:05.536951+0000","last_peered":"2024-05-27T07:30:05.536951+0000","last_clean":"2024-05-27T07:30:05.536951+0000","last_became_active":"2024-05-27T07:28:37.968068+0000","last_became_peered":"2024-05-27T07:28:37.968068+0000","last_unstale":"2024-05-27T07:30:05.536951+0000","last_undegraded":"2024-05-27T07:30:05.536951+0000","last_fullsized":"2024-05-27T07:30:05.536951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:22:00.825999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580077+0000","last_change":"2024-05-27T07:28:20.406203+0000","last_active":"2024-05-27T07:30:05.580077+0000","last_peered":"2024-05-27T07:30:05.580077+0000","last_clean":"2024-05-27T07:30:05.580077+0000","last_became_active":"2024-05-27T07:28:20.405404+0000","last_became_peered":"2024-05-27T07:28:20.405404+0000","last_unstale":"2024-05-27T07:30:05.580077+0000","last_undegraded":"2024-05-27T07:30:05.580077+0000","last_fullsized":"2024-05-27T07:30:05.580077+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:15.274419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516952+0000","last_change":"2024-05-27T07:28:24.473747+0000","last_active":"2024-05-27T07:30:05.516952+0000","last_peered":"2024-05-27T07:30:05.516952+0000","last_clean":"2024-05-27T07:30:05.516952+0000","last_became_active":"2024-05-27T07:28:24.473313+0000","last_became_peered":"2024-05-27T07:28:24.473313+0000","last_unstale":"2024-05-27T07:30:05.516952+0000","last_undegraded":"2024-05-27T07:30:05.516952+0000","last_fullsized":"2024-05-27T07:30:05.516952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:19:40.107840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.121935+0000","last_change":"2024-05-27T07:28:37.907410+0000","last_active":"2024-05-27T07:30:05.121935+0000","last_peered":"2024-05-27T07:30:05.121935+0000","last_clean":"2024-05-27T07:30:05.121935+0000","last_became_active":"2024-05-27T07:28:37.906753+0000","last_became_peered":"2024-05-27T07:28:37.906753+0000","last_unstale":"2024-05-27T07:30:05.121935+0000","last_undegraded":"2024-05-27T07:30:05.121935+0000","last_fullsized":"2024-05-27T07:30:05.121935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:53.612015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120470+0000","last_change":"2024-05-27T07:28:20.744068+0000","last_active":"2024-05-27T07:30:05.120470+0000","last_peered":"2024-05-27T07:30:05.120470+0000","last_clean":"2024-05-27T07:30:05.120470+0000","last_became_active":"2024-05-27T07:28:20.742686+0000","last_became_peered":"2024-05-27T07:28:20.742686+0000","last_unstale":"2024-05-27T07:30:05.120470+0000","last_undegraded":"2024-05-27T07:30:05.120470+0000","last_fullsized":"2024-05-27T07:30:05.120470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:54.361007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.517087+0000","last_change":"2024-05-27T07:28:24.475743+0000","last_active":"2024-05-27T07:30:05.517087+0000","last_peered":"2024-05-27T07:30:05.517087+0000","last_clean":"2024-05-27T07:30:05.517087+0000","last_became_active":"2024-05-27T07:28:24.474742+0000","last_became_peered":"2024-05-27T07:28:24.474742+0000","last_unstale":"2024-05-27T07:30:05.517087+0000","last_undegraded":"2024-05-27T07:30:05.517087+0000","last_fullsized":"2024-05-27T07:30:05.517087+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:58:47.061573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576220+0000","last_change":"2024-05-27T07:28:37.914935+0000","last_active":"2024-05-27T07:30:05.576220+0000","last_peered":"2024-05-27T07:30:05.576220+0000","last_clean":"2024-05-27T07:30:05.576220+0000","last_became_active":"2024-05-27T07:28:37.911829+0000","last_became_peered":"2024-05-27T07:28:37.911829+0000","last_unstale":"2024-05-27T07:30:05.576220+0000","last_undegraded":"2024-05-27T07:30:05.576220+0000","last_fullsized":"2024-05-27T07:30:05.576220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:29.260539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584299+0000","last_change":"2024-05-27T07:28:20.412554+0000","last_active":"2024-05-27T07:30:05.584299+0000","last_peered":"2024-05-27T07:30:05.584299+0000","last_clean":"2024-05-27T07:30:05.584299+0000","last_became_active":"2024-05-27T07:28:20.412024+0000","last_became_peered":"2024-05-27T07:28:20.412024+0000","last_unstale":"2024-05-27T07:30:05.584299+0000","last_undegraded":"2024-05-27T07:30:05.584299+0000","last_fullsized":"2024-05-27T07:30:05.584299+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:29.244093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516559+0000","last_change":"2024-05-27T07:28:24.481966+0000","last_active":"2024-05-27T07:30:05.516559+0000","last_peered":"2024-05-27T07:30:05.516559+0000","last_clean":"2024-05-27T07:30:05.516559+0000","last_became_active":"2024-05-27T07:28:24.480775+0000","last_became_peered":"2024-05-27T07:28:24.480775+0000","last_unstale":"2024-05-27T07:30:05.516559+0000","last_undegraded":"2024-05-27T07:30:05.516559+0000","last_fullsized":"2024-05-27T07:30:05.516559+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:03.018174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578676+0000","last_change":"2024-05-27T07:28:37.902893+0000","last_active":"2024-05-27T07:30:05.578676+0000","last_peered":"2024-05-27T07:30:05.578676+0000","last_clean":"2024-05-27T07:30:05.578676+0000","last_became_active":"2024-05-27T07:28:37.898447+0000","last_became_peered":"2024-05-27T07:28:37.898447+0000","last_unstale":"2024-05-27T07:30:05.578676+0000","last_undegraded":"2024-05-27T07:30:05.578676+0000","last_fullsized":"2024-05-27T07:30:05.578676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:55:49.754803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406914+0000","last_change":"2024-05-27T07:28:20.408072+0000","last_active":"2024-05-27T07:30:05.406914+0000","last_peered":"2024-05-27T07:30:05.406914+0000","last_clean":"2024-05-27T07:30:05.406914+0000","last_became_active":"2024-05-27T07:28:20.407174+0000","last_became_peered":"2024-05-27T07:28:20.407174+0000","last_unstale":"2024-05-27T07:30:05.406914+0000","last_undegraded":"2024-05-27T07:30:05.406914+0000","last_fullsized":"2024-05-27T07:30:05.406914+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:15:52.984967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.517471+0000","last_change":"2024-05-27T07:28:24.475891+0000","last_active":"2024-05-27T07:30:05.517471+0000","last_peered":"2024-05-27T07:30:05.517471+0000","last_clean":"2024-05-27T07:30:05.517471+0000","last_became_active":"2024-05-27T07:28:24.475498+0000","last_became_peered":"2024-05-27T07:28:24.475498+0000","last_unstale":"2024-05-27T07:30:05.517471+0000","last_undegraded":"2024-05-27T07:30:05.517471+0000","last_fullsized":"2024-05-27T07:30:05.517471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:50.669118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.579269+0000","last_change":"2024-05-27T07:29:04.891583+0000","last_active":"2024-05-27T07:30:05.579269+0000","last_peered":"2024-05-27T07:30:05.579269+0000","last_clean":"2024-05-27T07:30:05.579269+0000","last_became_active":"2024-05-27T07:29:04.890187+0000","last_became_peered":"2024-05-27T07:29:04.890187+0000","last_unstale":"2024-05-27T07:30:05.579269+0000","last_undegraded":"2024-05-27T07:30:05.579269+0000","last_fullsized":"2024-05-27T07:30:05.579269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:03.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119928+0000","last_change":"2024-05-27T07:28:24.478672+0000","last_active":"2024-05-27T07:30:05.119928+0000","last_peered":"2024-05-27T07:30:05.119928+0000","last_clean":"2024-05-27T07:30:05.119928+0000","last_became_active":"2024-05-27T07:28:24.478331+0000","last_became_peered":"2024-05-27T07:28:24.478331+0000","last_unstale":"2024-05-27T07:30:05.119928+0000","last_undegraded":"2024-05-27T07:30:05.119928+0000","last_fullsized":"2024-05-27T07:30:05.119928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:56:35.112855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119521+0000","last_change":"2024-05-27T07:28:24.487962+0000","last_active":"2024-05-27T07:30:05.119521+0000","last_peered":"2024-05-27T07:30:05.119521+0000","last_clean":"2024-05-27T07:30:05.119521+0000","last_became_active":"2024-05-27T07:28:24.487583+0000","last_became_peered":"2024-05-27T07:28:24.487583+0000","last_unstale":"2024-05-27T07:30:05.119521+0000","last_undegraded":"2024-05-27T07:30:05.119521+0000","last_fullsized":"2024-05-27T07:30:05.119521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:25.758293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576506+0000","last_change":"2024-05-27T07:28:24.466058+0000","last_active":"2024-05-27T07:30:05.576506+0000","last_peered":"2024-05-27T07:30:05.576506+0000","last_clean":"2024-05-27T07:30:05.576506+0000","last_became_active":"2024-05-27T07:28:24.465709+0000","last_became_peered":"2024-05-27T07:28:24.465709+0000","last_unstale":"2024-05-27T07:30:05.576506+0000","last_undegraded":"2024-05-27T07:30:05.576506+0000","last_fullsized":"2024-05-27T07:30:05.576506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:38.064077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406173+0000","last_change":"2024-05-27T07:28:24.471585+0000","last_active":"2024-05-27T07:30:05.406173+0000","last_peered":"2024-05-27T07:30:05.406173+0000","last_clean":"2024-05-27T07:30:05.406173+0000","last_became_active":"2024-05-27T07:28:24.470948+0000","last_became_peered":"2024-05-27T07:28:24.470948+0000","last_unstale":"2024-05-27T07:30:05.406173+0000","last_undegraded":"2024-05-27T07:30:05.406173+0000","last_fullsized":"2024-05-27T07:30:05.406173+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:48.213323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581655+0000","last_change":"2024-05-27T07:28:24.477954+0000","last_active":"2024-05-27T07:30:05.581655+0000","last_peered":"2024-05-27T07:30:05.581655+0000","last_clean":"2024-05-27T07:30:05.581655+0000","last_became_active":"2024-05-27T07:28:24.477495+0000","last_became_peered":"2024-05-27T07:28:24.477495+0000","last_unstale":"2024-05-27T07:30:05.581655+0000","last_undegraded":"2024-05-27T07:30:05.581655+0000","last_fullsized":"2024-05-27T07:30:05.581655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:50.371439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.121046+0000","last_change":"2024-05-27T07:28:24.476422+0000","last_active":"2024-05-27T07:30:05.121046+0000","last_peered":"2024-05-27T07:30:05.121046+0000","last_clean":"2024-05-27T07:30:05.121046+0000","last_became_active":"2024-05-27T07:28:24.476095+0000","last_became_peered":"2024-05-27T07:28:24.476095+0000","last_unstale":"2024-05-27T07:30:05.121046+0000","last_undegraded":"2024-05-27T07:30:05.121046+0000","last_fullsized":"2024-05-27T07:30:05.121046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:51.667169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122064+0000","last_change":"2024-05-27T07:28:24.486431+0000","last_active":"2024-05-27T07:30:05.122064+0000","last_peered":"2024-05-27T07:30:05.122064+0000","last_clean":"2024-05-27T07:30:05.122064+0000","last_became_active":"2024-05-27T07:28:24.482427+0000","last_became_peered":"2024-05-27T07:28:24.482427+0000","last_unstale":"2024-05-27T07:30:05.122064+0000","last_undegraded":"2024-05-27T07:30:05.122064+0000","last_fullsized":"2024-05-27T07:30:05.122064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:44:37.857793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582171+0000","last_change":"2024-05-27T07:28:24.484393+0000","last_active":"2024-05-27T07:30:05.582171+0000","last_peered":"2024-05-27T07:30:05.582171+0000","last_clean":"2024-05-27T07:30:05.582171+0000","last_became_active":"2024-05-27T07:28:24.483130+0000","last_became_peered":"2024-05-27T07:28:24.483130+0000","last_unstale":"2024-05-27T07:30:05.582171+0000","last_undegraded":"2024-05-27T07:30:05.582171+0000","last_fullsized":"2024-05-27T07:30:05.582171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:34.924573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120191+0000","last_change":"2024-05-27T07:28:24.486274+0000","last_active":"2024-05-27T07:30:05.120191+0000","last_peered":"2024-05-27T07:30:05.120191+0000","last_clean":"2024-05-27T07:30:05.120191+0000","last_became_active":"2024-05-27T07:28:24.485880+0000","last_became_peered":"2024-05-27T07:28:24.485880+0000","last_unstale":"2024-05-27T07:30:05.120191+0000","last_undegraded":"2024-05-27T07:30:05.120191+0000","last_fullsized":"2024-05-27T07:30:05.120191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:33.287615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122910+0000","last_change":"2024-05-27T07:28:24.477909+0000","last_active":"2024-05-27T07:30:05.122910+0000","last_peered":"2024-05-27T07:30:05.122910+0000","last_clean":"2024-05-27T07:30:05.122910+0000","last_became_active":"2024-05-27T07:28:24.477345+0000","last_became_peered":"2024-05-27T07:28:24.477345+0000","last_unstale":"2024-05-27T07:30:05.122910+0000","last_undegraded":"2024-05-27T07:30:05.122910+0000","last_fullsized":"2024-05-27T07:30:05.122910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:26:03.958489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.404938+0000","last_change":"2024-05-27T07:28:24.477151+0000","last_active":"2024-05-27T07:30:05.404938+0000","last_peered":"2024-05-27T07:30:05.404938+0000","last_clean":"2024-05-27T07:30:05.404938+0000","last_became_active":"2024-05-27T07:28:24.474036+0000","last_became_peered":"2024-05-27T07:28:24.474036+0000","last_unstale":"2024-05-27T07:30:05.404938+0000","last_undegraded":"2024-05-27T07:30:05.404938+0000","last_fullsized":"2024-05-27T07:30:05.404938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:16.787252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580908+0000","last_change":"2024-05-27T07:28:24.483601+0000","last_active":"2024-05-27T07:30:05.580908+0000","last_peered":"2024-05-27T07:30:05.580908+0000","last_clean":"2024-05-27T07:30:05.580908+0000","last_became_active":"2024-05-27T07:28:24.481080+0000","last_became_peered":"2024-05-27T07:28:24.481080+0000","last_unstale":"2024-05-27T07:30:05.580908+0000","last_undegraded":"2024-05-27T07:30:05.580908+0000","last_fullsized":"2024-05-27T07:30:05.580908+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:12.208506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71429,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":251,"num_read_kb":200,"num_write":169,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":215425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645356,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":522520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841586,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1740800,"data_stored":1177607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005115,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":526852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201347,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":522676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397579,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32792,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712360,"statfs":{"total":95995035648,"available":95961456640,"internally_reserved":0,"allocated":1654784,"data_stored":1118194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561105,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":517932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":585661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":519320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":587011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149800,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":518126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953568,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757338,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1662976,"data_stored":1113782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:30:35.157 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T07:30:35.158 INFO:tasks.cephadm:Setting up client nodes... 2024-05-27T07:30:35.158 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-27T07:30:35.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:35 smithi086 ceph-mon[34687]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:30:35.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:35 smithi149 ceph-mon[35408]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:30:35.456 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:35 smithi063 ceph-mon[31613]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:30:35.674 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:36.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:36 smithi086 ceph-mon[34687]: pgmap v457: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-27T07:30:36.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:36 smithi149 ceph-mon[35408]: pgmap v457: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-27T07:30:36.457 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:36 smithi063 ceph-mon[31613]: pgmap v457: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-27T07:30:37.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.977+0000 7f093f200700 1 -- 172.21.15.63:0/1616214544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09401063a0 msgr2=0x7f09401067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:37.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.977+0000 7f093f200700 1 --2- 172.21.15.63:0/1616214544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09401063a0 0x7f09401067b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0930009990 tx=0x7f0930009ca0 comp rx=0 tx=0).stop 2024-05-27T07:30:37.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.978+0000 7f093f200700 1 -- 172.21.15.63:0/1616214544 shutdown_connections 2024-05-27T07:30:37.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.978+0000 7f093f200700 1 --2- 172.21.15.63:0/1616214544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0940107e70 0x7f094010e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:37.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.978+0000 7f093f200700 1 --2- 172.21.15.63:0/1616214544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09401063a0 0x7f09401067b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:37.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.978+0000 7f093f200700 1 --2- 172.21.15.63:0/1616214544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09401074d0 0x7f09401078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:37.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.978+0000 7f093f200700 1 -- 172.21.15.63:0/1616214544 >> 172.21.15.63:0/1616214544 conn(0x7f09401019e0 msgr2=0x7f0940103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:37.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.978+0000 7f093f200700 1 -- 172.21.15.63:0/1616214544 shutdown_connections 2024-05-27T07:30:37.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.978+0000 7f093f200700 1 -- 172.21.15.63:0/1616214544 wait complete. 2024-05-27T07:30:37.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.979+0000 7f093f200700 1 Processor -- start 2024-05-27T07:30:37.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.979+0000 7f093f200700 1 -- start start 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.979+0000 7f093f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09401063a0 0x7f094019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.979+0000 7f093f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09401074d0 0x7f094019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.979+0000 7f093f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0940107e70 0x7f09401a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.979+0000 7f093f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0940111d70 con 0x7f09401063a0 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.979+0000 7f093f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0940111bf0 con 0x7f0940107e70 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.979+0000 7f093f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0940111ef0 con 0x7f09401074d0 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.980+0000 7f093de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09401063a0 0x7f094019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.980+0000 7f093de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09401063a0 0x7f094019ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57726/0 (socket says 172.21.15.63:57726) 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.980+0000 7f093de00700 1 -- 172.21.15.63:0/1598174649 learned_addr learned my addr 172.21.15.63:0/1598174649 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.980+0000 7f093e800700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0940107e70 0x7f09401a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:37.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.980+0000 7f093de00700 1 -- 172.21.15.63:0/1598174649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09401074d0 msgr2=0x7f094019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:37.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.980+0000 7f093de00700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09401074d0 0x7f094019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:37.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.980+0000 7f093de00700 1 -- 172.21.15.63:0/1598174649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0940107e70 msgr2=0x7f09401a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:37.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.980+0000 7f093de00700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0940107e70 0x7f09401a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:37.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.980+0000 7f093de00700 1 -- 172.21.15.63:0/1598174649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09401a4560 con 0x7f09401063a0 2024-05-27T07:30:37.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.980+0000 7f093e800700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0940107e70 0x7f09401a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:30:37.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.981+0000 7f093de00700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09401063a0 0x7f094019ca00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0930003820 tx=0x7f09300047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:37.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.981+0000 7f0936a00700 1 -- 172.21.15.63:0/1598174649 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930003c10 con 0x7f09401063a0 2024-05-27T07:30:37.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.981+0000 7f0936a00700 1 -- 172.21.15.63:0/1598174649 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0930004830 con 0x7f09401063a0 2024-05-27T07:30:37.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.981+0000 7f0936a00700 1 -- 172.21.15.63:0/1598174649 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930005260 con 0x7f09401063a0 2024-05-27T07:30:37.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.981+0000 7f093f200700 1 -- 172.21.15.63:0/1598174649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09401a47f0 con 0x7f09401063a0 2024-05-27T07:30:37.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.981+0000 7f093f200700 1 -- 172.21.15.63:0/1598174649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09401a4be0 con 0x7f09401063a0 2024-05-27T07:30:37.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.982+0000 7f093f200700 1 -- 172.21.15.63:0/1598174649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09400512b0 con 0x7f09401063a0 2024-05-27T07:30:37.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.985+0000 7f0936a00700 1 -- 172.21.15.63:0/1598174649 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f09300054c0 con 0x7f09401063a0 2024-05-27T07:30:37.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.986+0000 7f0936a00700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f09240b2910 0x7f09240b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:37.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.987+0000 7f0936a00700 1 -- 172.21.15.63:0/1598174649 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f09300e38b0 con 0x7f09401063a0 2024-05-27T07:30:37.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.987+0000 7f093d400700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f09240b2910 0x7f09240b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:37.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.988+0000 7f093d400700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f09240b2910 0x7f09240b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f094019dab0 tx=0x7f092c006d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:37.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:37.989+0000 7f0936a00700 1 -- 172.21.15.63:0/1598174649 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0930013070 con 0x7f09401063a0 2024-05-27T07:30:38.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.327+0000 7f093f200700 1 -- 172.21.15.63:0/1598174649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f094002e5e0 con 0x7f09401063a0 2024-05-27T07:30:38.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.334+0000 7f0936a00700 1 -- 172.21.15.63:0/1598174649 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v34) ==== 170+0+59 (secure 0 0 0) 0x7f09300ab110 con 0x7f09401063a0 2024-05-27T07:30:38.326 INFO:teuthology.orchestra.run.smithi063.stdout:[client.0] 2024-05-27T07:30:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQCeNlRmcjygExAAB2uhAXiq+NyxNhOr7xjIkw== 2024-05-27T07:30:38.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.337+0000 7f093f200700 1 -- 172.21.15.63:0/1598174649 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f09240b2910 msgr2=0x7f09240b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:38.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.337+0000 7f093f200700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f09240b2910 0x7f09240b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f094019dab0 tx=0x7f092c006d10 comp rx=0 tx=0).stop 2024-05-27T07:30:38.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.338+0000 7f093f200700 1 -- 172.21.15.63:0/1598174649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09401063a0 msgr2=0x7f094019ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:38.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.338+0000 7f093f200700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09401063a0 0x7f094019ca00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0930003820 tx=0x7f09300047b0 comp rx=0 tx=0).stop 2024-05-27T07:30:38.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.338+0000 7f093f200700 1 -- 172.21.15.63:0/1598174649 shutdown_connections 2024-05-27T07:30:38.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.338+0000 7f093f200700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0940107e70 0x7f09401a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:38.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.338+0000 7f093f200700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f09240b2910 0x7f09240b4e20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:38.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.338+0000 7f093f200700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09401063a0 0x7f094019ca00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:38.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.338+0000 7f093f200700 1 --2- 172.21.15.63:0/1598174649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09401074d0 0x7f094019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:38.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.338+0000 7f093f200700 1 -- 172.21.15.63:0/1598174649 >> 172.21.15.63:0/1598174649 conn(0x7f09401019e0 msgr2=0x7f094010a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:38.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.338+0000 7f093f200700 1 -- 172.21.15.63:0/1598174649 shutdown_connections 2024-05-27T07:30:38.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:38.338+0000 7f093f200700 1 -- 172.21.15.63:0/1598174649 wait complete. 2024-05-27T07:30:39.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:38 smithi086 ceph-mon[34687]: pgmap v458: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-27T07:30:39.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:38 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1598174649' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T07:30:39.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:38 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1598174649' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T07:30:39.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:38 smithi149 ceph-mon[35408]: pgmap v458: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-27T07:30:39.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:38 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1598174649' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T07:30:39.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:38 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1598174649' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T07:30:39.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:38 smithi063 ceph-mon[31613]: pgmap v458: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-27T07:30:39.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:38 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1598174649' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T07:30:39.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:38 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1598174649' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T07:30:39.303 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:30:39.303 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-27T07:30:39.303 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-27T07:30:39.450 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-27T07:30:39.450 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-27T07:30:39.450 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-05-27T07:30:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:41.169 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:40 smithi063 ceph-mon[31613]: pgmap v459: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T07:30:41.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:40 smithi086 ceph-mon[34687]: pgmap v459: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T07:30:41.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:40 smithi149 ceph-mon[35408]: pgmap v459: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-27T07:30:42.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.449+0000 7ff902600700 1 -- 172.21.15.63:0/3241125140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8fc1054d0 msgr2=0x7ff8fc1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:42.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.449+0000 7ff902600700 1 --2- 172.21.15.63:0/3241125140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8fc1054d0 0x7ff8fc1058e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009990 tx=0x7ff8ec009ca0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.449+0000 7ff902600700 1 -- 172.21.15.63:0/3241125140 shutdown_connections 2024-05-27T07:30:42.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.449+0000 7ff902600700 1 --2- 172.21.15.63:0/3241125140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8fc10e900 0x7ff8fc110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.449+0000 7ff902600700 1 --2- 172.21.15.63:0/3241125140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8fc1054d0 0x7ff8fc1058e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.449+0000 7ff902600700 1 --2- 172.21.15.63:0/3241125140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8fc105e20 0x7ff8fc10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.449+0000 7ff902600700 1 -- 172.21.15.63:0/3241125140 >> 172.21.15.63:0/3241125140 conn(0x7ff8fc0fef40 msgr2=0x7ff8fc101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:42.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.449+0000 7ff902600700 1 -- 172.21.15.63:0/3241125140 shutdown_connections 2024-05-27T07:30:42.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.449+0000 7ff902600700 1 -- 172.21.15.63:0/3241125140 wait complete. 2024-05-27T07:30:42.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.451+0000 7ff902600700 1 Processor -- start 2024-05-27T07:30:42.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.451+0000 7ff902600700 1 -- start start 2024-05-27T07:30:42.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.451+0000 7ff902600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8fc1054d0 0x7ff8fc1031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:42.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.451+0000 7ff902600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8fc105e20 0x7ff8fc103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:42.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.451+0000 7ff902600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8fc10e900 0x7ff8fc103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:42.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.451+0000 7ff902600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc114310 con 0x7ff8fc10e900 2024-05-27T07:30:42.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.451+0000 7ff902600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc114190 con 0x7ff8fc1054d0 2024-05-27T07:30:42.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.451+0000 7ff902600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc114490 con 0x7ff8fc105e20 2024-05-27T07:30:42.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.452+0000 7ff8fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8fc10e900 0x7ff8fc103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:42.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.452+0000 7ff8fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8fc10e900 0x7ff8fc103c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39442/0 (socket says 172.21.15.63:39442) 2024-05-27T07:30:42.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.452+0000 7ff8fbe00700 1 -- 172.21.15.63:0/371324696 learned_addr learned my addr 172.21.15.63:0/371324696 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:42.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.452+0000 7ff8fb400700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8fc1054d0 0x7ff8fc1031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.452+0000 7ff8faa00700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8fc105e20 0x7ff8fc103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.452+0000 7ff8fbe00700 1 -- 172.21.15.63:0/371324696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8fc105e20 msgr2=0x7ff8fc103700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.452+0000 7ff8fbe00700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8fc105e20 0x7ff8fc103700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.452+0000 7ff8fbe00700 1 -- 172.21.15.63:0/371324696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8fc1054d0 msgr2=0x7ff8fc1031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.452+0000 7ff8fbe00700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8fc1054d0 0x7ff8fc1031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.453+0000 7ff8fbe00700 1 -- 172.21.15.63:0/371324696 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc1a90e0 con 0x7ff8fc10e900 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.453+0000 7ff8fbe00700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8fc10e900 0x7ff8fc103c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4007ee0 tx=0x7ff8f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.453+0000 7ff8f3e00700 1 -- 172.21.15.63:0/371324696 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4004bb0 con 0x7ff8fc10e900 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.453+0000 7ff8f3e00700 1 -- 172.21.15.63:0/371324696 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8f4004d50 con 0x7ff8fc10e900 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.453+0000 7ff902600700 1 -- 172.21.15.63:0/371324696 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc1a93d0 con 0x7ff8fc10e900 2024-05-27T07:30:42.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.454+0000 7ff8f3e00700 1 -- 172.21.15.63:0/371324696 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40058b0 con 0x7ff8fc10e900 2024-05-27T07:30:42.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.455+0000 7ff902600700 1 -- 172.21.15.63:0/371324696 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc1a9910 con 0x7ff8fc10e900 2024-05-27T07:30:42.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.456+0000 7ff902600700 1 -- 172.21.15.63:0/371324696 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8fc0512b0 con 0x7ff8fc10e900 2024-05-27T07:30:42.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.457+0000 7ff8f3e00700 1 -- 172.21.15.63:0/371324696 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff8f401a020 con 0x7ff8fc10e900 2024-05-27T07:30:42.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.459+0000 7ff8f3e00700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8e00b2990 0x7ff8e00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:42.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.459+0000 7ff8f3e00700 1 -- 172.21.15.63:0/371324696 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7ff8f40e1d60 con 0x7ff8fc10e900 2024-05-27T07:30:42.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.459+0000 7ff8fb400700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8e00b2990 0x7ff8e00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:42.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.460+0000 7ff8fb400700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8e00b2990 0x7ff8e00b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b5c0 tx=0x7ff8ec005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:42.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.464+0000 7ff8f3e00700 1 -- 172.21.15.63:0/371324696 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8f4010070 con 0x7ff8fc10e900 2024-05-27T07:30:42.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.767+0000 7ff902600700 1 -- 172.21.15.63:0/371324696 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff8fc02e5e0 con 0x7ff8fc10e900 2024-05-27T07:30:42.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.775+0000 7ff8f3e00700 1 -- 172.21.15.63:0/371324696 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288644 (secure 0 0 0) 0x7ff8f40a9530 con 0x7ff8fc10e900 2024-05-27T07:30:42.767 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:30:42.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 -- 172.21.15.63:0/371324696 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8e00b2990 msgr2=0x7ff8e00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:42.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8e00b2990 0x7ff8e00b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b5c0 tx=0x7ff8ec005960 comp rx=0 tx=0).stop 2024-05-27T07:30:42.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 -- 172.21.15.63:0/371324696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8fc10e900 msgr2=0x7ff8fc103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:42.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8fc10e900 0x7ff8fc103c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4007ee0 tx=0x7ff8f400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 -- 172.21.15.63:0/371324696 shutdown_connections 2024-05-27T07:30:42.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ff8e00b2990 0x7ff8e00b4ea0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8fc10e900 0x7ff8fc103c40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8fc1054d0 0x7ff8fc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 --2- 172.21.15.63:0/371324696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8fc105e20 0x7ff8fc103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:42.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 -- 172.21.15.63:0/371324696 >> 172.21.15.63:0/371324696 conn(0x7ff8fc0fef40 msgr2=0x7ff8fc0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:42.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.780+0000 7ff902600700 1 -- 172.21.15.63:0/371324696 shutdown_connections 2024-05-27T07:30:42.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:42.781+0000 7ff902600700 1 -- 172.21.15.63:0/371324696 wait complete. 2024-05-27T07:30:43.158 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:42 smithi063 ceph-mon[31613]: pgmap v460: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:30:43.158 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:42 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/371324696' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T07:30:43.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:42 smithi086 ceph-mon[34687]: pgmap v460: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:30:43.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:42 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/371324696' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T07:30:43.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:42 smithi149 ceph-mon[35408]: pgmap v460: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:30:43.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:42 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/371324696' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T07:30:43.685 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":675305989},{"type":"v1","addr":"172.21.15.63:6801","nonce":675305989}]},"active_addr":"172.21.15.63:6801/675305989","active_change":"2024-05-27T07:18:03.559620+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.63:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":3428263920}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":1202958531}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":1558308201}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":1961493183}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":3467232624}]}]} 2024-05-27T07:30:43.694 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-27T07:30:43.694 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-27T07:30:43.694 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:30:44.199 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:45.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:44 smithi086 ceph-mon[34687]: pgmap v461: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:30:45.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:44 smithi149 ceph-mon[35408]: pgmap v461: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:30:45.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:44 smithi063 ceph-mon[31613]: pgmap v461: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:30:46.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.795+0000 7f708d200700 1 -- 172.21.15.63:0/2265942125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7088104640 msgr2=0x7f7088104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:46.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.795+0000 7f708d200700 1 --2- 172.21.15.63:0/2265942125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7088104640 0x7f7088104a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f70780099c0 tx=0x7f7078009cd0 comp rx=0 tx=0).stop 2024-05-27T07:30:46.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.795+0000 7f708d200700 1 -- 172.21.15.63:0/2265942125 shutdown_connections 2024-05-27T07:30:46.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.795+0000 7f708d200700 1 --2- 172.21.15.63:0/2265942125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7088109870 0x7f70881111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:46.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.795+0000 7f708d200700 1 --2- 172.21.15.63:0/2265942125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7088104640 0x7f7088104a50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:46.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.795+0000 7f708d200700 1 --2- 172.21.15.63:0/2265942125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7088108f20 0x7f7088109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:46.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.795+0000 7f708d200700 1 -- 172.21.15.63:0/2265942125 >> 172.21.15.63:0/2265942125 conn(0x7f7088079ef0 msgr2=0x7f708807a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:46.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.795+0000 7f708d200700 1 -- 172.21.15.63:0/2265942125 shutdown_connections 2024-05-27T07:30:46.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.795+0000 7f708d200700 1 -- 172.21.15.63:0/2265942125 wait complete. 2024-05-27T07:30:46.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.796+0000 7f708d200700 1 Processor -- start 2024-05-27T07:30:46.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.796+0000 7f708d200700 1 -- start start 2024-05-27T07:30:46.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.797+0000 7f708d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7088104640 0x7f708819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:46.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.797+0000 7f708d200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7088108f20 0x7f708819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:46.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.797+0000 7f708d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7088109870 0x7f70881a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:46.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.797+0000 7f708d200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70881143a0 con 0x7f7088104640 2024-05-27T07:30:46.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.797+0000 7f708d200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7088114220 con 0x7f7088109870 2024-05-27T07:30:46.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.797+0000 7f708d200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7088114520 con 0x7f7088108f20 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.797+0000 7f7085e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7088104640 0x7f708819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.797+0000 7f7085e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7088104640 0x7f708819cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39466/0 (socket says 172.21.15.63:39466) 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.797+0000 7f7085e00700 1 -- 172.21.15.63:0/315005795 learned_addr learned my addr 172.21.15.63:0/315005795 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.798+0000 7f7085400700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7088108f20 0x7f708819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.798+0000 7f7086800700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7088109870 0x7f70881a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.798+0000 7f7085e00700 1 -- 172.21.15.63:0/315005795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7088108f20 msgr2=0x7f708819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.798+0000 7f7085e00700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7088108f20 0x7f708819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.798+0000 7f7085e00700 1 -- 172.21.15.63:0/315005795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7088109870 msgr2=0x7f70881a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.798+0000 7f7085e00700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7088109870 0x7f70881a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.798+0000 7f7085e00700 1 -- 172.21.15.63:0/315005795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70881a1ec0 con 0x7f7088104640 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.799+0000 7f7085e00700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7088104640 0x7f708819cff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7078000c00 tx=0x7f707800f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:46.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.799+0000 7f707ea00700 1 -- 172.21.15.63:0/315005795 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078027070 con 0x7f7088104640 2024-05-27T07:30:46.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.799+0000 7f707ea00700 1 -- 172.21.15.63:0/315005795 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f707800fda0 con 0x7f7088104640 2024-05-27T07:30:46.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.799+0000 7f707ea00700 1 -- 172.21.15.63:0/315005795 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078022e50 con 0x7f7088104640 2024-05-27T07:30:46.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.799+0000 7f708d200700 1 -- 172.21.15.63:0/315005795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70881a2150 con 0x7f7088104640 2024-05-27T07:30:46.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.799+0000 7f708d200700 1 -- 172.21.15.63:0/315005795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70881a9ad0 con 0x7f7088104640 2024-05-27T07:30:46.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.801+0000 7f708d200700 1 -- 172.21.15.63:0/315005795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70880512b0 con 0x7f7088104640 2024-05-27T07:30:46.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.803+0000 7f707ea00700 1 -- 172.21.15.63:0/315005795 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f707802a070 con 0x7f7088104640 2024-05-27T07:30:46.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.805+0000 7f707ea00700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f706c0b2b30 0x7f706c0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:46.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.805+0000 7f707ea00700 1 -- 172.21.15.63:0/315005795 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f70780e3830 con 0x7f7088104640 2024-05-27T07:30:46.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.805+0000 7f7085400700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f706c0b2b30 0x7f706c0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:46.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.806+0000 7f7085400700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f706c0b2b30 0x7f706c0b5040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7088103e40 tx=0x7f707400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:46.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:46.808+0000 7f707ea00700 1 -- 172.21.15.63:0/315005795 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7078013070 con 0x7f7088104640 2024-05-27T07:30:47.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.049+0000 7f708d200700 1 -- 172.21.15.63:0/315005795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70881a9ca0 con 0x7f7088104640 2024-05-27T07:30:47.042 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:46 smithi063 ceph-mon[31613]: pgmap v462: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:30:47.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.052+0000 7f707ea00700 1 -- 172.21.15.63:0/315005795 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24635 (secure 0 0 0) 0x7f70780ab090 con 0x7f7088104640 2024-05-27T07:30:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:30:47.045 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":91,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:30:05.109968+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6},{"from":3,"to":7},{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:30:47.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.055+0000 7f708d200700 1 -- 172.21.15.63:0/315005795 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f706c0b2b30 msgr2=0x7f706c0b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:47.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.055+0000 7f708d200700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f706c0b2b30 0x7f706c0b5040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7088103e40 tx=0x7f707400b410 comp rx=0 tx=0).stop 2024-05-27T07:30:47.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.055+0000 7f708d200700 1 -- 172.21.15.63:0/315005795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7088104640 msgr2=0x7f708819cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:47.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.055+0000 7f708d200700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7088104640 0x7f708819cff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7078000c00 tx=0x7f707800f460 comp rx=0 tx=0).stop 2024-05-27T07:30:47.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.056+0000 7f708d200700 1 -- 172.21.15.63:0/315005795 shutdown_connections 2024-05-27T07:30:47.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.056+0000 7f708d200700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7088109870 0x7f70881a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:47.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.056+0000 7f708d200700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f706c0b2b30 0x7f706c0b5040 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:47.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.056+0000 7f708d200700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7088104640 0x7f708819cff0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:47.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.056+0000 7f708d200700 1 --2- 172.21.15.63:0/315005795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7088108f20 0x7f708819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:47.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.056+0000 7f708d200700 1 -- 172.21.15.63:0/315005795 >> 172.21.15.63:0/315005795 conn(0x7f7088079ef0 msgr2=0x7f7088101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:47.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.056+0000 7f708d200700 1 -- 172.21.15.63:0/315005795 shutdown_connections 2024-05-27T07:30:47.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:47.056+0000 7f708d200700 1 -- 172.21.15.63:0/315005795 wait complete. 2024-05-27T07:30:47.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:46 smithi086 ceph-mon[34687]: pgmap v462: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:30:47.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:46 smithi149 ceph-mon[35408]: pgmap v462: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:30:47.976 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-27T07:30:47.977 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:30:48.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:47 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/315005795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:30:48.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:47 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/315005795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:30:48.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:47 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/315005795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:30:48.445 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:49.096 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:48 smithi063 ceph-mon[31613]: pgmap v463: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:30:49.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:48 smithi086 ceph-mon[34687]: pgmap v463: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:30:49.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:48 smithi149 ceph-mon[35408]: pgmap v463: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:30:50.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.906+0000 7f1d5d600700 1 -- 172.21.15.63:0/3254588465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d5810e900 msgr2=0x7f1d58110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:50.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.906+0000 7f1d5d600700 1 --2- 172.21.15.63:0/3254588465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d5810e900 0x7f1d58110de0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000b0a0 tx=0x7f1d5000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:30:50.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.906+0000 7f1d5d600700 1 -- 172.21.15.63:0/3254588465 shutdown_connections 2024-05-27T07:30:50.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.906+0000 7f1d5d600700 1 --2- 172.21.15.63:0/3254588465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d5810e900 0x7f1d58110de0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:50.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.906+0000 7f1d5d600700 1 --2- 172.21.15.63:0/3254588465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d581054d0 0x7f1d581058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:50.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.906+0000 7f1d5d600700 1 --2- 172.21.15.63:0/3254588465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d58105e20 0x7f1d5810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:50.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.906+0000 7f1d5d600700 1 -- 172.21.15.63:0/3254588465 >> 172.21.15.63:0/3254588465 conn(0x7f1d580fef20 msgr2=0x7f1d58101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:50.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.906+0000 7f1d5d600700 1 -- 172.21.15.63:0/3254588465 shutdown_connections 2024-05-27T07:30:50.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.906+0000 7f1d5d600700 1 -- 172.21.15.63:0/3254588465 wait complete. 2024-05-27T07:30:50.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.908+0000 7f1d5d600700 1 Processor -- start 2024-05-27T07:30:50.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.908+0000 7f1d5d600700 1 -- start start 2024-05-27T07:30:50.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.908+0000 7f1d5d600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d581054d0 0x7f1d5819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:50.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.908+0000 7f1d5d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58105e20 0x7f1d5819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:50.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.908+0000 7f1d5d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5810e900 0x7f1d581a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:50.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.908+0000 7f1d5d600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d58114380 con 0x7f1d58105e20 2024-05-27T07:30:50.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.908+0000 7f1d5d600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d58114200 con 0x7f1d5810e900 2024-05-27T07:30:50.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.908+0000 7f1d5d600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1d58114500 con 0x7f1d581054d0 2024-05-27T07:30:50.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.909+0000 7f1d56000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58105e20 0x7f1d5819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:50.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.909+0000 7f1d56000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58105e20 0x7f1d5819d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40438/0 (socket says 172.21.15.63:40438) 2024-05-27T07:30:50.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.909+0000 7f1d56000700 1 -- 172.21.15.63:0/804197818 learned_addr learned my addr 172.21.15.63:0/804197818 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:30:50.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.909+0000 7f1d56a00700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d581054d0 0x7f1d5819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:50.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.909+0000 7f1d57400700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5810e900 0x7f1d581a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:50.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.909+0000 7f1d56000700 1 -- 172.21.15.63:0/804197818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d581054d0 msgr2=0x7f1d5819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:50.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.909+0000 7f1d56000700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d581054d0 0x7f1d5819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:50.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.909+0000 7f1d56000700 1 -- 172.21.15.63:0/804197818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5810e900 msgr2=0x7f1d581a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:50.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.909+0000 7f1d56000700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5810e900 0x7f1d581a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:50.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.909+0000 7f1d56000700 1 -- 172.21.15.63:0/804197818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d581a1de0 con 0x7f1d58105e20 2024-05-27T07:30:50.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.910+0000 7f1d56000700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58105e20 0x7f1d5819d4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400e9a0 tx=0x7f1d4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:50.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.910+0000 7f1d4f400700 1 -- 172.21.15.63:0/804197818 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400cbd0 con 0x7f1d58105e20 2024-05-27T07:30:50.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.910+0000 7f1d4f400700 1 -- 172.21.15.63:0/804197818 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d4400cd70 con 0x7f1d58105e20 2024-05-27T07:30:50.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.910+0000 7f1d5d600700 1 -- 172.21.15.63:0/804197818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d581a20d0 con 0x7f1d58105e20 2024-05-27T07:30:50.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.910+0000 7f1d4f400700 1 -- 172.21.15.63:0/804197818 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44018680 con 0x7f1d58105e20 2024-05-27T07:30:50.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.910+0000 7f1d5d600700 1 -- 172.21.15.63:0/804197818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d581a99f0 con 0x7f1d58105e20 2024-05-27T07:30:50.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.912+0000 7f1d5d600700 1 -- 172.21.15.63:0/804197818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d580512b0 con 0x7f1d58105e20 2024-05-27T07:30:50.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.915+0000 7f1d4f400700 1 -- 172.21.15.63:0/804197818 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1d44010430 con 0x7f1d58105e20 2024-05-27T07:30:50.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.917+0000 7f1d4f400700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2990 0x7f1d3c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:50.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.917+0000 7f1d4f400700 1 -- 172.21.15.63:0/804197818 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f1d440e2bf0 con 0x7f1d58105e20 2024-05-27T07:30:50.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.917+0000 7f1d56a00700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2990 0x7f1d3c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:50.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.918+0000 7f1d56a00700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2990 0x7f1d3c0b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009a10 tx=0x7f1d48009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:50.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:50.919+0000 7f1d4f400700 1 -- 172.21.15.63:0/804197818 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d440aa4d0 con 0x7f1d58105e20 2024-05-27T07:30:51.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.157+0000 7f1d5d600700 1 -- 172.21.15.63:0/804197818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d581a9bc0 con 0x7f1d58105e20 2024-05-27T07:30:51.150 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:50 smithi063 ceph-mon[31613]: pgmap v464: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:30:51.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.159+0000 7f1d4f400700 1 -- 172.21.15.63:0/804197818 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24635 (secure 0 0 0) 0x7f1d440a9dd0 con 0x7f1d58105e20 2024-05-27T07:30:51.152 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:30:51.153 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":91,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:30:05.109968+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6},{"from":3,"to":7},{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:30:51.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 -- 172.21.15.63:0/804197818 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2990 msgr2=0x7f1d3c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:51.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2990 0x7f1d3c0b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009a10 tx=0x7f1d48009190 comp rx=0 tx=0).stop 2024-05-27T07:30:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 -- 172.21.15.63:0/804197818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58105e20 msgr2=0x7f1d5819d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58105e20 0x7f1d5819d4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400e9a0 tx=0x7f1d4400ecb0 comp rx=0 tx=0).stop 2024-05-27T07:30:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 -- 172.21.15.63:0/804197818 shutdown_connections 2024-05-27T07:30:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5810e900 0x7f1d581a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d581054d0 0x7f1d5819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1d3c0b2990 0x7f1d3c0b4ea0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 --2- 172.21.15.63:0/804197818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d58105e20 0x7f1d5819d4b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 -- 172.21.15.63:0/804197818 >> 172.21.15.63:0/804197818 conn(0x7f1d580fef20 msgr2=0x7f1d580ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 -- 172.21.15.63:0/804197818 shutdown_connections 2024-05-27T07:30:51.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:30:51.163+0000 7f1d5d600700 1 -- 172.21.15.63:0/804197818 wait complete. 2024-05-27T07:30:51.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:50 smithi086 ceph-mon[34687]: pgmap v464: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:30:51.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:50 smithi149 ceph-mon[35408]: pgmap v464: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:30:52.107 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-27T07:30:52.107 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-27T07:30:52.108 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-27T07:30:52.108 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-27T07:30:52.108 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-27T07:30:52.109 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-27T07:30:52.109 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-27T07:30:52.109 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-27T07:30:52.110 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-27T07:30:52.110 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-27T07:30:52.111 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-27T07:30:52.111 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-27T07:30:52.200 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:51 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/804197818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:30:52.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:51 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/804197818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:30:52.211 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:51 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/804197818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:30:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:53.184 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:53.188 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:53.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:52 smithi149 ceph-mon[35408]: pgmap v465: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:30:53.204 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:53.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:52 smithi063 ceph-mon[31613]: pgmap v465: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:30:53.390 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:53.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:52 smithi086 ceph-mon[34687]: pgmap v465: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:30:53.733 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:53.735 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:53.736 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:54.172 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:55.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:54 smithi063 ceph-mon[31613]: pgmap v466: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:30:55.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:54 smithi086 ceph-mon[34687]: pgmap v466: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:30:55.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:54 smithi149 ceph-mon[35408]: pgmap v466: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:30:55.542 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:55.543 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:30:57.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:56 smithi063 ceph-mon[31613]: pgmap v467: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:30:57.253 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:56 smithi086 ceph-mon[34687]: pgmap v467: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:30:57.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:56 smithi149 ceph-mon[35408]: pgmap v467: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:30:59.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:30:58 smithi063 ceph-mon[31613]: pgmap v468: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:30:59.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:30:58 smithi086 ceph-mon[34687]: pgmap v468: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:30:59.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:30:58 smithi149 ceph-mon[35408]: pgmap v468: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:01.129 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.255+0000 7fa07be00700 1 -- 172.21.15.63:0/3563467480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa074110960 msgr2=0x7fa074110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.255+0000 7fa07be00700 1 --2- 172.21.15.63:0/3563467480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa074110960 0x7fa074110d70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa068009990 tx=0x7fa068009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:01.248 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:00 smithi063 ceph-mon[31613]: pgmap v469: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.255+0000 7fa07be00700 1 -- 172.21.15.63:0/3563467480 shutdown_connections 2024-05-27T07:31:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.255+0000 7fa07be00700 1 --2- 172.21.15.63:0/3563467480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07410b2d0 0x7fa074117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.255+0000 7fa07be00700 1 --2- 172.21.15.63:0/3563467480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa074110960 0x7fa074110d70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.255+0000 7fa07be00700 1 --2- 172.21.15.63:0/3563467480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa07410a980 0x7fa07410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.255+0000 7fa07be00700 1 -- 172.21.15.63:0/3563467480 >> 172.21.15.63:0/3563467480 conn(0x7fa074077c70 msgr2=0x7fa0740780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.255+0000 7fa07be00700 1 -- 172.21.15.63:0/3563467480 shutdown_connections 2024-05-27T07:31:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.255+0000 7fa07be00700 1 -- 172.21.15.63:0/3563467480 wait complete. 2024-05-27T07:31:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.257+0000 7fa07be00700 1 Processor -- start 2024-05-27T07:31:01.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.257+0000 7fa07be00700 1 -- start start 2024-05-27T07:31:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.258+0000 7fa07be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07410a980 0x7fa0741a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.258+0000 7fa07be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa07410b2d0 0x7fa0741a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.258+0000 7fa07be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa074110960 0x7fa0741a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.258+0000 7fa07be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa07411a960 con 0x7fa07410b2d0 2024-05-27T07:31:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.258+0000 7fa07be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa07411a7e0 con 0x7fa07410a980 2024-05-27T07:31:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.258+0000 7fa07be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa07411aae0 con 0x7fa074110960 2024-05-27T07:31:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.258+0000 7fa078a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa07410b2d0 0x7fa0741a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.258+0000 7fa078a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa07410b2d0 0x7fa0741a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38392/0 (socket says 172.21.15.63:38392) 2024-05-27T07:31:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.258+0000 7fa078a00700 1 -- 172.21.15.63:0/2817671521 learned_addr learned my addr 172.21.15.63:0/2817671521 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.258+0000 7fa079400700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07410a980 0x7fa0741a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.259+0000 7fa078a00700 1 -- 172.21.15.63:0/2817671521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa074110960 msgr2=0x7fa0741a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.259+0000 7fa078a00700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa074110960 0x7fa0741a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.259+0000 7fa078a00700 1 -- 172.21.15.63:0/2817671521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07410a980 msgr2=0x7fa0741a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.259+0000 7fa078a00700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07410a980 0x7fa0741a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.259+0000 7fa078a00700 1 -- 172.21.15.63:0/2817671521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0741a6280 con 0x7fa07410b2d0 2024-05-27T07:31:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.259+0000 7fa078a00700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa07410b2d0 0x7fa0741a1950 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa06000d980 tx=0x7fa06000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.259+0000 7fa072000700 1 -- 172.21.15.63:0/2817671521 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060017070 con 0x7fa07410b2d0 2024-05-27T07:31:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.259+0000 7fa072000700 1 -- 172.21.15.63:0/2817671521 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa060009dc0 con 0x7fa07410b2d0 2024-05-27T07:31:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.259+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0741a6570 con 0x7fa07410b2d0 2024-05-27T07:31:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.260+0000 7fa072000700 1 -- 172.21.15.63:0/2817671521 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06000f3c0 con 0x7fa07410b2d0 2024-05-27T07:31:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.260+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0741ade90 con 0x7fa07410b2d0 2024-05-27T07:31:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.261+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa074063f40 con 0x7fa07410b2d0 2024-05-27T07:31:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.264+0000 7fa072000700 1 -- 172.21.15.63:0/2817671521 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa060009930 con 0x7fa07410b2d0 2024-05-27T07:31:01.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.265+0000 7fa072000700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa05c0b2910 0x7fa05c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:01.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.265+0000 7fa072000700 1 -- 172.21.15.63:0/2817671521 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fa0600e2680 con 0x7fa07410b2d0 2024-05-27T07:31:01.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.265+0000 7fa079400700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa05c0b2910 0x7fa05c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:01.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.266+0000 7fa072000700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] conn(0x7fa05c0bdb50 0x7fa05c0c0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.266+0000 7fa072000700 1 -- 172.21.15.63:0/2817671521 --> [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa060016ec0 con 0x7fa05c0bdb50 2024-05-27T07:31:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.266+0000 7fa072000700 1 -- 172.21.15.63:0/2817671521 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fa0600e2a30 con 0x7fa07410b2d0 2024-05-27T07:31:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.266+0000 7fa079400700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa05c0b2910 0x7fa05c0b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa06800b5c0 tx=0x7fa068005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.266+0000 7fa079e00700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] conn(0x7fa05c0bdb50 0x7fa05c0c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:01.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.267+0000 7fa079e00700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] conn(0x7fa05c0bdb50 0x7fa05c0c0040 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:01.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.270+0000 7fa072000700 1 -- 172.21.15.63:0/2817671521 <== osd.1 v2:172.21.15.86:6800/1435942303 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa060016ec0 con 0x7fa05c0bdb50 2024-05-27T07:31:01.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.288+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 --> [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa074064340 con 0x7fa05c0bdb50 2024-05-27T07:31:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.290+0000 7fa072000700 1 -- 172.21.15.63:0/2817671521 <== osd.1 v2:172.21.15.86:6800/1435942303 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa074064340 con 0x7fa05c0bdb50 2024-05-27T07:31:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.290+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 >> [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] conn(0x7fa05c0bdb50 msgr2=0x7fa05c0c0040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.290+0000 7fa07be00700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] conn(0x7fa05c0bdb50 0x7fa05c0c0040 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:01.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.290+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa05c0b2910 msgr2=0x7fa05c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:01.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.290+0000 7fa07be00700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa05c0b2910 0x7fa05c0b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa06800b5c0 tx=0x7fa068005fd0 comp rx=0 tx=0).stop 2024-05-27T07:31:01.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.290+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa07410b2d0 msgr2=0x7fa0741a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:01.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.290+0000 7fa07be00700 1 --2- 172.21.15.63:0/2817671521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa07410b2d0 0x7fa0741a1950 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa06000d980 tx=0x7fa06000dc90 comp rx=0 tx=0).stop 2024-05-27T07:31:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.291+0000 7fa079e00700 1 -- 172.21.15.63:0/2817671521 reap_dead start 2024-05-27T07:31:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.291+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 shutdown_connections 2024-05-27T07:31:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.291+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 >> 172.21.15.63:0/2817671521 conn(0x7fa074077c70 msgr2=0x7fa07407bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.291+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 shutdown_connections 2024-05-27T07:31:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:01.291+0000 7fa07be00700 1 -- 172.21.15.63:0/2817671521 wait complete. 2024-05-27T07:31:01.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:00 smithi086 ceph-mon[34687]: pgmap v469: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:01.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:00 smithi149 ceph-mon[35408]: pgmap v469: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:03.238 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:02 smithi063 ceph-mon[31613]: pgmap v470: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:03.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:02 smithi086 ceph-mon[34687]: pgmap v470: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:03.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:02 smithi149 ceph-mon[35408]: pgmap v470: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:03.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.925+0000 7fd1f1800700 1 -- 172.21.15.63:0/679830710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1ec1045e0 msgr2=0x7fd1ec1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:03.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.925+0000 7fd1f1800700 1 --2- 172.21.15.63:0/679830710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1ec1045e0 0x7fd1ec1049f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0099c0 tx=0x7fd1dc009cd0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.926+0000 7fd1f1800700 1 -- 172.21.15.63:0/679830710 shutdown_connections 2024-05-27T07:31:03.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.926+0000 7fd1f1800700 1 --2- 172.21.15.63:0/679830710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1ec109860 0x7fd1ec1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.926+0000 7fd1f1800700 1 --2- 172.21.15.63:0/679830710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1ec1045e0 0x7fd1ec1049f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.926+0000 7fd1f1800700 1 --2- 172.21.15.63:0/679830710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1ec108f10 0x7fd1ec109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.926+0000 7fd1f1800700 1 -- 172.21.15.63:0/679830710 >> 172.21.15.63:0/679830710 conn(0x7fd1ec0fffd0 msgr2=0x7fd1ec1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:03.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.926+0000 7fd1f1800700 1 -- 172.21.15.63:0/679830710 shutdown_connections 2024-05-27T07:31:03.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.926+0000 7fd1f1800700 1 -- 172.21.15.63:0/679830710 wait complete. 2024-05-27T07:31:03.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.928+0000 7fd1f1800700 1 Processor -- start 2024-05-27T07:31:03.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.928+0000 7fd1f1800700 1 -- start start 2024-05-27T07:31:03.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.928+0000 7fd1f1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1ec1045e0 0x7fd1ec077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:03.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.928+0000 7fd1f1800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1ec108f10 0x7fd1ec075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:03.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.929+0000 7fd1f1800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1ec109860 0x7fd1ec076050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:03.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.929+0000 7fd1f1800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec114400 con 0x7fd1ec109860 2024-05-27T07:31:03.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.929+0000 7fd1f1800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec114280 con 0x7fd1ec1045e0 2024-05-27T07:31:03.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.929+0000 7fd1f1800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec114580 con 0x7fd1ec108f10 2024-05-27T07:31:03.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.929+0000 7fd1eaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1ec1045e0 0x7fd1ec077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.929+0000 7fd1eb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1ec109860 0x7fd1ec076050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.929+0000 7fd1ea000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1ec108f10 0x7fd1ec075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.929+0000 7fd1eaa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1ec1045e0 0x7fd1ec077600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44642/0 (socket says 172.21.15.63:44642) 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.929+0000 7fd1eaa00700 1 -- 172.21.15.63:0/3654239288 learned_addr learned my addr 172.21.15.63:0/3654239288 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.929+0000 7fd1ea000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1ec108f10 0x7fd1ec075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:56250/0 (socket says 172.21.15.63:56250) 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.930+0000 7fd1eb400700 1 -- 172.21.15.63:0/3654239288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1ec108f10 msgr2=0x7fd1ec075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.930+0000 7fd1eb400700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1ec108f10 0x7fd1ec075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.930+0000 7fd1eb400700 1 -- 172.21.15.63:0/3654239288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1ec1045e0 msgr2=0x7fd1ec077600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.930+0000 7fd1eb400700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1ec1045e0 0x7fd1ec077600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.930+0000 7fd1eb400700 1 -- 172.21.15.63:0/3654239288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec0767f0 con 0x7fd1ec109860 2024-05-27T07:31:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.930+0000 7fd1eb400700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1ec109860 0x7fd1ec076050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400cc50 tx=0x7fd1e400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:03.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.931+0000 7fd1e3400700 1 -- 172.21.15.63:0/3654239288 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4016020 con 0x7fd1ec109860 2024-05-27T07:31:03.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.931+0000 7fd1e3400700 1 -- 172.21.15.63:0/3654239288 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1e4012420 con 0x7fd1ec109860 2024-05-27T07:31:03.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.931+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ec1ad900 con 0x7fd1ec109860 2024-05-27T07:31:03.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.931+0000 7fd1e3400700 1 -- 172.21.15.63:0/3654239288 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40115e0 con 0x7fd1ec109860 2024-05-27T07:31:03.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.931+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ec1ade10 con 0x7fd1ec109860 2024-05-27T07:31:03.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.932+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1ec063e80 con 0x7fd1ec109860 2024-05-27T07:31:03.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.935+0000 7fd1e3400700 1 -- 172.21.15.63:0/3654239288 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd1e4019020 con 0x7fd1ec109860 2024-05-27T07:31:03.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.936+0000 7fd1e3400700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1d00b2990 0x7fd1d00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:03.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.937+0000 7fd1e3400700 1 -- 172.21.15.63:0/3654239288 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fd1e40e2980 con 0x7fd1ec109860 2024-05-27T07:31:03.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.937+0000 7fd1eaa00700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1d00b2990 0x7fd1d00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:03.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.937+0000 7fd1e3400700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] conn(0x7fd1d00bdbd0 0x7fd1d00c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:03.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.937+0000 7fd1e3400700 1 -- 172.21.15.63:0/3654239288 --> [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1d00c0790 con 0x7fd1d00bdbd0 2024-05-27T07:31:03.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.937+0000 7fd1e3400700 1 -- 172.21.15.63:0/3654239288 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fd1e40e2d30 con 0x7fd1ec109860 2024-05-27T07:31:03.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.938+0000 7fd1eaa00700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1d00b2990 0x7fd1d00b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009fd0 tx=0x7fd1dc0096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:03.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.938+0000 7fd1ea000700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] conn(0x7fd1d00bdbd0 0x7fd1d00c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:03.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.940+0000 7fd1ea000700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] conn(0x7fd1d00bdbd0 0x7fd1d00c00c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:03.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.942+0000 7fd1e3400700 1 -- 172.21.15.63:0/3654239288 <== osd.5 v2:172.21.15.149:6808/3863528283 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1d00c0790 con 0x7fd1d00bdbd0 2024-05-27T07:31:03.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.962+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 --> [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd1ec0642b0 con 0x7fd1d00bdbd0 2024-05-27T07:31:03.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.963+0000 7fd1e3400700 1 -- 172.21.15.63:0/3654239288 <== osd.5 v2:172.21.15.149:6808/3863528283 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd1ec0642b0 con 0x7fd1d00bdbd0 2024-05-27T07:31:03.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 >> [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] conn(0x7fd1d00bdbd0 msgr2=0x7fd1d00c00c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:03.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1f1800700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] conn(0x7fd1d00bdbd0 0x7fd1d00c00c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1d00b2990 msgr2=0x7fd1d00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:03.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1f1800700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd1d00b2990 0x7fd1d00b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009fd0 tx=0x7fd1dc0096d0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1ec109860 msgr2=0x7fd1ec076050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:03.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1f1800700 1 --2- 172.21.15.63:0/3654239288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1ec109860 0x7fd1ec076050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400cc50 tx=0x7fd1e400cf60 comp rx=0 tx=0).stop 2024-05-27T07:31:03.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1eb400700 1 -- 172.21.15.63:0/3654239288 reap_dead start 2024-05-27T07:31:03.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 shutdown_connections 2024-05-27T07:31:03.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 >> 172.21.15.63:0/3654239288 conn(0x7fd1ec0fffd0 msgr2=0x7fd1ec107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:03.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 shutdown_connections 2024-05-27T07:31:03.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:03.964+0000 7fd1f1800700 1 -- 172.21.15.63:0/3654239288 wait complete. 2024-05-27T07:31:04.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.227+0000 7f28dfe00700 1 -- 172.21.15.63:0/1115486643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28e010af60 msgr2=0x7f28e010b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:04.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.227+0000 7f28dfe00700 1 --2- 172.21.15.63:0/1115486643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28e010af60 0x7f28e010b370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0099c0 tx=0x7f28cc009cd0 comp rx=0 tx=0).stop 2024-05-27T07:31:04.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.227+0000 7f28dfe00700 1 -- 172.21.15.63:0/1115486643 shutdown_connections 2024-05-27T07:31:04.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.227+0000 7f28dfe00700 1 --2- 172.21.15.63:0/1115486643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28e010b8b0 0x7f28e0113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:04.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.227+0000 7f28dfe00700 1 --2- 172.21.15.63:0/1115486643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28e0106630 0x7f28e0106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:04.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.227+0000 7f28dfe00700 1 --2- 172.21.15.63:0/1115486643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28e010af60 0x7f28e010b370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:04.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.227+0000 7f28dfe00700 1 -- 172.21.15.63:0/1115486643 >> 172.21.15.63:0/1115486643 conn(0x7f28e0101ec0 msgr2=0x7f28e0104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:04.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.227+0000 7f28dfe00700 1 -- 172.21.15.63:0/1115486643 shutdown_connections 2024-05-27T07:31:04.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.227+0000 7f28dfe00700 1 -- 172.21.15.63:0/1115486643 wait complete. 2024-05-27T07:31:04.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.229+0000 7f28dfe00700 1 Processor -- start 2024-05-27T07:31:04.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.229+0000 7f28dfe00700 1 -- start start 2024-05-27T07:31:04.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.229+0000 7f28dfe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28e0106630 0x7f28e019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:04.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.229+0000 7f28dfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28e010af60 0x7f28e019d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:04.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.229+0000 7f28dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28e010b8b0 0x7f28e01a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:04.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.229+0000 7f28dfe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01163c0 con 0x7f28e010af60 2024-05-27T07:31:04.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.229+0000 7f28dfe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0116240 con 0x7f28e010b8b0 2024-05-27T07:31:04.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.229+0000 7f28dfe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0116540 con 0x7f28e0106630 2024-05-27T07:31:04.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.230+0000 7f28de000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28e010af60 0x7f28e019d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:04.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.230+0000 7f28de000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28e010af60 0x7f28e019d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38430/0 (socket says 172.21.15.63:38430) 2024-05-27T07:31:04.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.230+0000 7f28de000700 1 -- 172.21.15.63:0/4292892293 learned_addr learned my addr 172.21.15.63:0/4292892293 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:04.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.230+0000 7f28df400700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28e010b8b0 0x7f28e01a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:04.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.230+0000 7f28de000700 1 -- 172.21.15.63:0/4292892293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28e0106630 msgr2=0x7f28e019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:04.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.230+0000 7f28de000700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28e0106630 0x7f28e019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:04.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.230+0000 7f28de000700 1 -- 172.21.15.63:0/4292892293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28e010b8b0 msgr2=0x7f28e01a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:04.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.230+0000 7f28de000700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28e010b8b0 0x7f28e01a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:04.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.230+0000 7f28de000700 1 -- 172.21.15.63:0/4292892293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e01a1bf0 con 0x7f28e010af60 2024-05-27T07:31:04.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.231+0000 7f28de000700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28e010af60 0x7f28e019d370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00b580 tx=0x7f28cc00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:04.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.231+0000 7f28e4a00700 1 -- 172.21.15.63:0/4292892293 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc027070 con 0x7f28e010af60 2024-05-27T07:31:04.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.231+0000 7f28e4a00700 1 -- 172.21.15.63:0/4292892293 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28cc00fda0 con 0x7f28e010af60 2024-05-27T07:31:04.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.231+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e01a1e80 con 0x7f28e010af60 2024-05-27T07:31:04.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.231+0000 7f28e4a00700 1 -- 172.21.15.63:0/4292892293 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc0173f0 con 0x7f28e010af60 2024-05-27T07:31:04.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.231+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e01a23b0 con 0x7f28e010af60 2024-05-27T07:31:04.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.232+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28e0063f00 con 0x7f28e010af60 2024-05-27T07:31:04.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.235+0000 7f28e4a00700 1 -- 172.21.15.63:0/4292892293 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f28cc02a070 con 0x7f28e010af60 2024-05-27T07:31:04.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.237+0000 7f28e4a00700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f28c40b2990 0x7f28c40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:04.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.237+0000 7f28e4a00700 1 -- 172.21.15.63:0/4292892293 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f28cc0e39e0 con 0x7f28e010af60 2024-05-27T07:31:04.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.237+0000 7f28dea00700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f28c40b2990 0x7f28c40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:04.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.238+0000 7f28e4a00700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] conn(0x7f28c40bdbc0 0x7f28c40c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:04.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.238+0000 7f28e4a00700 1 -- 172.21.15.63:0/4292892293 --> [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28c40c0780 con 0x7f28c40bdbc0 2024-05-27T07:31:04.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.238+0000 7f28e4a00700 1 -- 172.21.15.63:0/4292892293 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f28cc0e3e30 con 0x7f28e010af60 2024-05-27T07:31:04.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.238+0000 7f28df400700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] conn(0x7f28c40bdbc0 0x7f28c40c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:04.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.238+0000 7f28dea00700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f28c40b2990 0x7f28c40b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005950 tx=0x7f28d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:04.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.239+0000 7f28df400700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] conn(0x7f28c40bdbc0 0x7f28c40c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:04.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.240+0000 7f28e4a00700 1 -- 172.21.15.63:0/4292892293 <== osd.6 v2:172.21.15.63:6818/3628147489 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28c40c0780 con 0x7f28c40bdbc0 2024-05-27T07:31:04.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.259+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 --> [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f28e0064330 con 0x7f28c40bdbc0 2024-05-27T07:31:04.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.259+0000 7f28e4a00700 1 -- 172.21.15.63:0/4292892293 <== osd.6 v2:172.21.15.63:6818/3628147489 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f28e0064330 con 0x7f28c40bdbc0 2024-05-27T07:31:04.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.260+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] conn(0x7f28c40bdbc0 msgr2=0x7f28c40c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:04.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.260+0000 7f28dfe00700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] conn(0x7f28c40bdbc0 0x7f28c40c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:04.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.260+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f28c40b2990 msgr2=0x7f28c40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:04.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.260+0000 7f28dfe00700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f28c40b2990 0x7f28c40b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005950 tx=0x7f28d00058e0 comp rx=0 tx=0).stop 2024-05-27T07:31:04.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.260+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28e010af60 msgr2=0x7f28e019d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:04.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.260+0000 7f28dfe00700 1 --2- 172.21.15.63:0/4292892293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28e010af60 0x7f28e019d370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00b580 tx=0x7f28cc00f460 comp rx=0 tx=0).stop 2024-05-27T07:31:04.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.260+0000 7f28df400700 1 -- 172.21.15.63:0/4292892293 reap_dead start 2024-05-27T07:31:04.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.261+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 shutdown_connections 2024-05-27T07:31:04.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.261+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 >> 172.21.15.63:0/4292892293 conn(0x7f28e0101ec0 msgr2=0x7f28e010c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:04.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.261+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 shutdown_connections 2024-05-27T07:31:04.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.261+0000 7f28dfe00700 1 -- 172.21.15.63:0/4292892293 wait complete. 2024-05-27T07:31:05.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:04 smithi063 ceph-mon[31613]: pgmap v471: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:05.443 INFO:teuthology.orchestra.run.smithi063.stdout:163208757345 2024-05-27T07:31:05.444 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-27T07:31:05.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:05 smithi086 ceph-mon[34687]: pgmap v471: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:05.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:05 smithi149 ceph-mon[35408]: pgmap v471: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:06.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.958+0000 7fd67f200700 1 -- 172.21.15.63:0/3248077363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd680109670 msgr2=0x7fd680110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:06.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.958+0000 7fd67f200700 1 --2- 172.21.15.63:0/3248077363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd680109670 0x7fd680110fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b0d0 tx=0x7fd67800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.958+0000 7fd67f200700 1 -- 172.21.15.63:0/3248077363 shutdown_connections 2024-05-27T07:31:06.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.958+0000 7fd67f200700 1 --2- 172.21.15.63:0/3248077363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd680109670 0x7fd680110fd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.958+0000 7fd67f200700 1 --2- 172.21.15.63:0/3248077363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6801043f0 0x7fd680104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.958+0000 7fd67f200700 1 --2- 172.21.15.63:0/3248077363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd680108d20 0x7fd680109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.958+0000 7fd67f200700 1 -- 172.21.15.63:0/3248077363 >> 172.21.15.63:0/3248077363 conn(0x7fd680079d00 msgr2=0x7fd68007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:06.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.958+0000 7fd67f200700 1 -- 172.21.15.63:0/3248077363 shutdown_connections 2024-05-27T07:31:06.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.958+0000 7fd67f200700 1 -- 172.21.15.63:0/3248077363 wait complete. 2024-05-27T07:31:06.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.959+0000 7fd67f200700 1 Processor -- start 2024-05-27T07:31:06.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.959+0000 7fd67f200700 1 -- start start 2024-05-27T07:31:06.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.960+0000 7fd67f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6801043f0 0x7fd6801a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:06.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.960+0000 7fd67f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd680108d20 0x7fd6801a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.960+0000 7fd67f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd680109670 0x7fd68019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.960+0000 7fd67f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd6801141c0 con 0x7fd6801043f0 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.960+0000 7fd67f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd680114040 con 0x7fd680109670 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.960+0000 7fd67f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd680114340 con 0x7fd680108d20 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.960+0000 7fd67de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6801043f0 0x7fd6801a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.960+0000 7fd67de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6801043f0 0x7fd6801a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38452/0 (socket says 172.21.15.63:38452) 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.960+0000 7fd67de00700 1 -- 172.21.15.63:0/1277763477 learned_addr learned my addr 172.21.15.63:0/1277763477 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd67e800700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd680109670 0x7fd68019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd67de00700 1 -- 172.21.15.63:0/1277763477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd680108d20 msgr2=0x7fd6801a0c60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd67de00700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd680108d20 0x7fd6801a0c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd67de00700 1 -- 172.21.15.63:0/1277763477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd680109670 msgr2=0x7fd68019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:06.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd67de00700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd680109670 0x7fd68019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd67de00700 1 -- 172.21.15.63:0/1277763477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68019aff0 con 0x7fd6801043f0 2024-05-27T07:31:06.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd67e800700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd680109670 0x7fd68019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:31:06.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd67de00700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6801043f0 0x7fd6801a0720 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd67000d950 tx=0x7fd67000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:06.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd676a00700 1 -- 172.21.15.63:0/1277763477 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670016070 con 0x7fd6801043f0 2024-05-27T07:31:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd676a00700 1 -- 172.21.15.63:0/1277763477 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd670010460 con 0x7fd6801043f0 2024-05-27T07:31:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68019b2e0 con 0x7fd6801043f0 2024-05-27T07:31:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.961+0000 7fd676a00700 1 -- 172.21.15.63:0/1277763477 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67000f620 con 0x7fd6801043f0 2024-05-27T07:31:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.962+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6801a75b0 con 0x7fd6801043f0 2024-05-27T07:31:06.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.962+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd680063f40 con 0x7fd6801043f0 2024-05-27T07:31:06.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.965+0000 7fd676a00700 1 -- 172.21.15.63:0/1277763477 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd670019070 con 0x7fd6801043f0 2024-05-27T07:31:06.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.967+0000 7fd676a00700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd6640b2910 0x7fd6640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.967+0000 7fd676a00700 1 -- 172.21.15.63:0/1277763477 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fd6700e28d0 con 0x7fd6801043f0 2024-05-27T07:31:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.967+0000 7fd67d400700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd6640b2910 0x7fd6640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.967+0000 7fd676a00700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] conn(0x7fd6640bdb50 0x7fd6640c0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.967+0000 7fd676a00700 1 -- 172.21.15.63:0/1277763477 --> [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6640c0710 con 0x7fd6640bdb50 2024-05-27T07:31:06.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.967+0000 7fd676a00700 1 -- 172.21.15.63:0/1277763477 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fd6700e2c80 con 0x7fd6801043f0 2024-05-27T07:31:06.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.968+0000 7fd67d400700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd6640b2910 0x7fd6640b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd680101120 tx=0x7fd66c006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:06.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.968+0000 7fd67e800700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] conn(0x7fd6640bdb50 0x7fd6640c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:06.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.969+0000 7fd67e800700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] conn(0x7fd6640bdb50 0x7fd6640c0040 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:06.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.972+0000 7fd676a00700 1 -- 172.21.15.63:0/1277763477 <== osd.4 v2:172.21.15.86:6808/3874353435 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6640c0710 con 0x7fd6640bdb50 2024-05-27T07:31:06.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.992+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 --> [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd6800643c0 con 0x7fd6640bdb50 2024-05-27T07:31:06.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.993+0000 7fd676a00700 1 -- 172.21.15.63:0/1277763477 <== osd.4 v2:172.21.15.86:6808/3874353435 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd6800643c0 con 0x7fd6640bdb50 2024-05-27T07:31:06.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.994+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 >> [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] conn(0x7fd6640bdb50 msgr2=0x7fd6640c0040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:06.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.994+0000 7fd67f200700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] conn(0x7fd6640bdb50 0x7fd6640c0040 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.994+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd6640b2910 msgr2=0x7fd6640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:06.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.994+0000 7fd67f200700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd6640b2910 0x7fd6640b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd680101120 tx=0x7fd66c006c60 comp rx=0 tx=0).stop 2024-05-27T07:31:06.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.994+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6801043f0 msgr2=0x7fd6801a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:06.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.994+0000 7fd67f200700 1 --2- 172.21.15.63:0/1277763477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6801043f0 0x7fd6801a0720 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd67000d950 tx=0x7fd67000dd10 comp rx=0 tx=0).stop 2024-05-27T07:31:06.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.994+0000 7fd67e800700 1 -- 172.21.15.63:0/1277763477 reap_dead start 2024-05-27T07:31:06.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.994+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 shutdown_connections 2024-05-27T07:31:06.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.994+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 >> 172.21.15.63:0/1277763477 conn(0x7fd680079d00 msgr2=0x7fd6801015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:06.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.995+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 shutdown_connections 2024-05-27T07:31:06.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:06.995+0000 7fd67f200700 1 -- 172.21.15.63:0/1277763477 wait complete. 2024-05-27T07:31:07.021 INFO:teuthology.orchestra.run.smithi063.stdout:188978561112 2024-05-27T07:31:07.021 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-27T07:31:07.083 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:07 smithi063 ceph-mon[31613]: pgmap v472: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:07.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:07 smithi086 ceph-mon[34687]: pgmap v472: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:07.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:07 smithi149 ceph-mon[35408]: pgmap v472: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:07.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.035+0000 7f5207800700 1 -- 172.21.15.63:0/680563314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5200108f10 msgr2=0x7f5200109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.035+0000 7f5207800700 1 --2- 172.21.15.63:0/680563314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5200108f10 0x7f5200109320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0099c0 tx=0x7f51ec009cd0 comp rx=0 tx=0).stop 2024-05-27T07:31:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.035+0000 7f5207800700 1 -- 172.21.15.63:0/680563314 shutdown_connections 2024-05-27T07:31:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.035+0000 7f5207800700 1 --2- 172.21.15.63:0/680563314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200109860 0x7f52001111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.035+0000 7f5207800700 1 --2- 172.21.15.63:0/680563314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52001045e0 0x7f52001049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.035+0000 7f5207800700 1 --2- 172.21.15.63:0/680563314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5200108f10 0x7f5200109320 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.035+0000 7f5207800700 1 -- 172.21.15.63:0/680563314 >> 172.21.15.63:0/680563314 conn(0x7f5200100030 msgr2=0x7f5200102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.035+0000 7f5207800700 1 -- 172.21.15.63:0/680563314 shutdown_connections 2024-05-27T07:31:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.035+0000 7f5207800700 1 -- 172.21.15.63:0/680563314 wait complete. 2024-05-27T07:31:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.037+0000 7f5207800700 1 Processor -- start 2024-05-27T07:31:07.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.037+0000 7f5207800700 1 -- start start 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.037+0000 7f5207800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52001045e0 0x7f5200075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.037+0000 7f5207800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200108f10 0x7f5200075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.038+0000 7f5207800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5200109860 0x7f5200076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.038+0000 7f5207800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5200114320 con 0x7f5200109860 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.038+0000 7f5207800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52001141a0 con 0x7f5200108f10 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.038+0000 7f5207800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f52001144a0 con 0x7f52001045e0 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.038+0000 7f5205800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5200109860 0x7f5200076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.038+0000 7f5205800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5200109860 0x7f5200076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38408/0 (socket says 172.21.15.63:38408) 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.038+0000 7f5205800700 1 -- 172.21.15.63:0/3421019516 learned_addr learned my addr 172.21.15.63:0/3421019516 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.038+0000 7f5204e00700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52001045e0 0x7f5200075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.038+0000 7f51ffe00700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200108f10 0x7f5200075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:08.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.039+0000 7f5205800700 1 -- 172.21.15.63:0/3421019516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52001045e0 msgr2=0x7f5200075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.039+0000 7f5205800700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52001045e0 0x7f5200075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.039+0000 7f5205800700 1 -- 172.21.15.63:0/3421019516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200108f10 msgr2=0x7f5200075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.039+0000 7f5205800700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5200108f10 0x7f5200075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.039+0000 7f5205800700 1 -- 172.21.15.63:0/3421019516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52001a9110 con 0x7f5200109860 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.039+0000 7f5204e00700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52001045e0 0x7f5200075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.039+0000 7f5205800700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5200109860 0x7f5200076530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f51f8007ee0 tx=0x7f51f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.040+0000 7f51fd600700 1 -- 172.21.15.63:0/3421019516 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f800dbd0 con 0x7f5200109860 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.040+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52001a9400 con 0x7f5200109860 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.040+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001a9910 con 0x7f5200109860 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.040+0000 7f51fd600700 1 -- 172.21.15.63:0/3421019516 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51f800dd70 con 0x7f5200109860 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.040+0000 7f51fd600700 1 -- 172.21.15.63:0/3421019516 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f80058d0 con 0x7f5200109860 2024-05-27T07:31:08.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.041+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51e0001030 con 0x7f5200109860 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.046+0000 7f51fd600700 1 -- 172.21.15.63:0/3421019516 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f51f801a020 con 0x7f5200109860 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.047+0000 7f51fd600700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f51e80b2990 0x7f51e80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.048+0000 7f51fd600700 1 -- 172.21.15.63:0/3421019516 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f51f80e1d70 con 0x7f5200109860 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.048+0000 7f5204e00700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f51e80b2990 0x7f51e80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.048+0000 7f51fd600700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] conn(0x7f51e80bdbc0 0x7f51e80c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.048+0000 7f51fd600700 1 -- 172.21.15.63:0/3421019516 --> [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51f8014ec0 con 0x7f51e80bdbc0 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.048+0000 7f51fd600700 1 -- 172.21.15.63:0/3421019516 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f51f80e4390 con 0x7f5200109860 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.048+0000 7f51ffe00700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] conn(0x7f51e80bdbc0 0x7f51e80c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.049+0000 7f5204e00700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f51e80b2990 0x7f51e80b4ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009cb0 tx=0x7f51f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.049+0000 7f51ffe00700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] conn(0x7f51e80bdbc0 0x7f51e80c00b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.050+0000 7f51fd600700 1 -- 172.21.15.63:0/3421019516 <== osd.3 v2:172.21.15.63:6810/3416673415 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51f8014ec0 con 0x7f51e80bdbc0 2024-05-27T07:31:08.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.069+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 --> [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51e0002e60 con 0x7f51e80bdbc0 2024-05-27T07:31:08.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.070+0000 7f51fd600700 1 -- 172.21.15.63:0/3421019516 <== osd.3 v2:172.21.15.63:6810/3416673415 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f51e0002e60 con 0x7f51e80bdbc0 2024-05-27T07:31:08.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.070+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] conn(0x7f51e80bdbc0 msgr2=0x7f51e80c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:08.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.070+0000 7f5207800700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] conn(0x7f51e80bdbc0 0x7f51e80c00b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:08.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.070+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f51e80b2990 msgr2=0x7f51e80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:08.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.070+0000 7f5207800700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f51e80b2990 0x7f51e80b4ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009cb0 tx=0x7f51f4009260 comp rx=0 tx=0).stop 2024-05-27T07:31:08.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.070+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5200109860 msgr2=0x7f5200076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:08.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.070+0000 7f5207800700 1 --2- 172.21.15.63:0/3421019516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5200109860 0x7f5200076530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f51f8007ee0 tx=0x7f51f800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:31:08.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.070+0000 7f5205800700 1 -- 172.21.15.63:0/3421019516 reap_dead start 2024-05-27T07:31:08.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.070+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 shutdown_connections 2024-05-27T07:31:08.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.070+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 >> 172.21.15.63:0/3421019516 conn(0x7f5200100030 msgr2=0x7f520010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:08.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.071+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 shutdown_connections 2024-05-27T07:31:08.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:04.071+0000 7f5207800700 1 -- 172.21.15.63:0/3421019516 wait complete. 2024-05-27T07:31:09.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:09 smithi063 ceph-mon[31613]: pgmap v473: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:09.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:09 smithi086 ceph-mon[34687]: pgmap v473: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:09.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:09 smithi149 ceph-mon[35408]: pgmap v473: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:09.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.682+0000 7f080dc00700 1 -- 172.21.15.63:0/1097102840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f080810a980 msgr2=0x7f080810ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:09.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.682+0000 7f080dc00700 1 --2- 172.21.15.63:0/1097102840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f080810a980 0x7f080810ad90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009990 tx=0x7f07f4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.683+0000 7f080dc00700 1 -- 172.21.15.63:0/1097102840 shutdown_connections 2024-05-27T07:31:09.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.683+0000 7f080dc00700 1 --2- 172.21.15.63:0/1097102840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080810b2d0 0x7f0808117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.683+0000 7f080dc00700 1 --2- 172.21.15.63:0/1097102840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0808110960 0x7f0808110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.683+0000 7f080dc00700 1 --2- 172.21.15.63:0/1097102840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f080810a980 0x7f080810ad90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.683+0000 7f080dc00700 1 -- 172.21.15.63:0/1097102840 >> 172.21.15.63:0/1097102840 conn(0x7f0808077c70 msgr2=0x7f08080780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:09.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.683+0000 7f080dc00700 1 -- 172.21.15.63:0/1097102840 shutdown_connections 2024-05-27T07:31:09.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.683+0000 7f080dc00700 1 -- 172.21.15.63:0/1097102840 wait complete. 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.684+0000 7f080dc00700 1 Processor -- start 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.684+0000 7f080dc00700 1 -- start start 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.685+0000 7f080dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f080810a980 0x7f08081a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.685+0000 7f080dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080810b2d0 0x7f08081a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.685+0000 7f080dc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0808110960 0x7f08081a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.685+0000 7f080dc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f080811a980 con 0x7f080810a980 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.685+0000 7f080dc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f080811a800 con 0x7f080810b2d0 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.685+0000 7f080dc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f080811ab00 con 0x7f0808110960 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.685+0000 7f0806a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f080810a980 0x7f08081a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.685+0000 7f0806a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f080810a980 0x7f08081a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38470/0 (socket says 172.21.15.63:38470) 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.685+0000 7f0806a00700 1 -- 172.21.15.63:0/3404140863 learned_addr learned my addr 172.21.15.63:0/3404140863 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:09.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.686+0000 7f0806000700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080810b2d0 0x7f08081a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:09.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.686+0000 7f0807400700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0808110960 0x7f08081a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:09.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.686+0000 7f0806a00700 1 -- 172.21.15.63:0/3404140863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0808110960 msgr2=0x7f08081a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:09.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.686+0000 7f0806a00700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0808110960 0x7f08081a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.686+0000 7f0806a00700 1 -- 172.21.15.63:0/3404140863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080810b2d0 msgr2=0x7f08081a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:09.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.686+0000 7f0806a00700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080810b2d0 0x7f08081a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.686+0000 7f0806a00700 1 -- 172.21.15.63:0/3404140863 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08081a6280 con 0x7f080810a980 2024-05-27T07:31:09.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.686+0000 7f0806a00700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f080810a980 0x7f08081a13b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f07f800d980 tx=0x7f07f800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.687+0000 7f07ff400700 1 -- 172.21.15.63:0/3404140863 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8017070 con 0x7f080810a980 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.687+0000 7f07ff400700 1 -- 172.21.15.63:0/3404140863 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07f8010460 con 0x7f080810a980 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.687+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08081a6570 con 0x7f080810a980 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.687+0000 7f07ff400700 1 -- 172.21.15.63:0/3404140863 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f800de60 con 0x7f080810a980 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.687+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08081ade90 con 0x7f080810a980 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.688+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0808063ec0 con 0x7f080810a980 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.691+0000 7f07ff400700 1 -- 172.21.15.63:0/3404140863 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f07f8010610 con 0x7f080810a980 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.692+0000 7f07ff400700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f07ec0b2910 0x7f07ec0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.692+0000 7f07ff400700 1 -- 172.21.15.63:0/3404140863 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f07f80e2670 con 0x7f080810a980 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.693+0000 7f0806000700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f07ec0b2910 0x7f07ec0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.693+0000 7f07ff400700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] conn(0x7f07ec0bdb50 0x7f07ec0c0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:09.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.693+0000 7f07ff400700 1 -- 172.21.15.63:0/3404140863 --> [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07f8016ec0 con 0x7f07ec0bdb50 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.693+0000 7f07ff400700 1 -- 172.21.15.63:0/3404140863 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f07f80e2a20 con 0x7f080810a980 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.693+0000 7f0806000700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f07ec0b2910 0x7f07ec0b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0808109a90 tx=0x7f07f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.694+0000 7f0807400700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] conn(0x7f07ec0bdb50 0x7f07ec0c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.695+0000 7f0807400700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] conn(0x7f07ec0bdb50 0x7f07ec0c0040 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.697+0000 7f07ff400700 1 -- 172.21.15.63:0/3404140863 <== osd.7 v2:172.21.15.86:6816/146162920 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07f8016ec0 con 0x7f07ec0bdb50 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.716+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 --> [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0808064340 con 0x7f07ec0bdb50 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.717+0000 7f07ff400700 1 -- 172.21.15.63:0/3404140863 <== osd.7 v2:172.21.15.86:6816/146162920 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0808064340 con 0x7f07ec0bdb50 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.717+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 >> [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] conn(0x7f07ec0bdb50 msgr2=0x7f07ec0c0040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.717+0000 7f080dc00700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] conn(0x7f07ec0bdb50 0x7f07ec0c0040 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.717+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f07ec0b2910 msgr2=0x7f07ec0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.717+0000 7f080dc00700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f07ec0b2910 0x7f07ec0b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0808109a90 tx=0x7f07f4005fb0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.718+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f080810a980 msgr2=0x7f08081a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:09.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.718+0000 7f080dc00700 1 --2- 172.21.15.63:0/3404140863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f080810a980 0x7f08081a13b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f07f800d980 tx=0x7f07f800dd40 comp rx=0 tx=0).stop 2024-05-27T07:31:09.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.718+0000 7f0807400700 1 -- 172.21.15.63:0/3404140863 reap_dead start 2024-05-27T07:31:09.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.718+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 shutdown_connections 2024-05-27T07:31:09.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.718+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 >> 172.21.15.63:0/3404140863 conn(0x7f0808077c70 msgr2=0x7f080807b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:09.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.718+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 shutdown_connections 2024-05-27T07:31:09.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:07.718+0000 7f080dc00700 1 -- 172.21.15.63:0/3404140863 wait complete. 2024-05-27T07:31:10.271 INFO:teuthology.orchestra.run.smithi063.stdout:55834574972 2024-05-27T07:31:10.272 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-27T07:31:11.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:11 smithi063 ceph-mon[31613]: pgmap v474: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:11.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:11 smithi086 ceph-mon[34687]: pgmap v474: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:11.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:11 smithi149 ceph-mon[35408]: pgmap v474: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:13.100 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:13 smithi063 ceph-mon[31613]: pgmap v475: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:13.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:13 smithi086 ceph-mon[34687]: pgmap v475: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:13.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:13 smithi149 ceph-mon[35408]: pgmap v475: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.946+0000 7f01ab200700 1 -- 172.21.15.63:0/1805679809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ac108d20 msgr2=0x7f01ac109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.946+0000 7f01ab200700 1 --2- 172.21.15.63:0/1805679809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ac108d20 0x7f01ac109130 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f01940099c0 tx=0x7f0194009cd0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.946+0000 7f01ab200700 1 -- 172.21.15.63:0/1805679809 shutdown_connections 2024-05-27T07:31:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.946+0000 7f01ab200700 1 --2- 172.21.15.63:0/1805679809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac109670 0x7f01ac110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.946+0000 7f01ab200700 1 --2- 172.21.15.63:0/1805679809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ac1043f0 0x7f01ac104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.946+0000 7f01ab200700 1 --2- 172.21.15.63:0/1805679809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ac108d20 0x7f01ac109130 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.946+0000 7f01ab200700 1 -- 172.21.15.63:0/1805679809 >> 172.21.15.63:0/1805679809 conn(0x7f01ac079d00 msgr2=0x7f01ac07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.946+0000 7f01ab200700 1 -- 172.21.15.63:0/1805679809 shutdown_connections 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.946+0000 7f01ab200700 1 -- 172.21.15.63:0/1805679809 wait complete. 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.947+0000 7f01ab200700 1 Processor -- start 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.948+0000 7f01ab200700 1 -- start start 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.948+0000 7f01ab200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac1043f0 0x7f01ac075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.948+0000 7f01ab200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ac108d20 0x7f01ac075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.948+0000 7f01ab200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ac109670 0x7f01ac076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.948+0000 7f01ab200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1141a0 con 0x7f01ac109670 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.948+0000 7f01ab200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac114020 con 0x7f01ac1043f0 2024-05-27T07:31:14.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.948+0000 7f01ab200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac114320 con 0x7f01ac108d20 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.948+0000 7f01aa800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ac109670 0x7f01ac076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.949+0000 7f01aa800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ac109670 0x7f01ac076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56268/0 (socket says 172.21.15.63:56268) 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.949+0000 7f01aa800700 1 -- 172.21.15.63:0/1074503781 learned_addr learned my addr 172.21.15.63:0/1074503781 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.949+0000 7f01a9e00700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac1043f0 0x7f01ac075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.949+0000 7f01a1400700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ac108d20 0x7f01ac075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.949+0000 7f01aa800700 1 -- 172.21.15.63:0/1074503781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ac108d20 msgr2=0x7f01ac075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.949+0000 7f01aa800700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01ac108d20 0x7f01ac075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.949+0000 7f01aa800700 1 -- 172.21.15.63:0/1074503781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac1043f0 msgr2=0x7f01ac075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.949+0000 7f01aa800700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ac1043f0 0x7f01ac075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.949+0000 7f01aa800700 1 -- 172.21.15.63:0/1074503781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac1a8f70 con 0x7f01ac109670 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.949+0000 7f01aa800700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ac109670 0x7f01ac076530 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f01a4007ee0 tx=0x7f01a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:14.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.950+0000 7f01a3400700 1 -- 172.21.15.63:0/1074503781 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a400dbd0 con 0x7f01ac109670 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.950+0000 7f01a3400700 1 -- 172.21.15.63:0/1074503781 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01a400dd70 con 0x7f01ac109670 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.950+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac1a9260 con 0x7f01ac109670 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.950+0000 7f01a3400700 1 -- 172.21.15.63:0/1074503781 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40058d0 con 0x7f01ac109670 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.950+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac1a9770 con 0x7f01ac109670 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.951+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01ac063ec0 con 0x7f01ac109670 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.954+0000 7f01a3400700 1 -- 172.21.15.63:0/1074503781 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f01a401a020 con 0x7f01ac109670 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.955+0000 7f01a3400700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f01900b2990 0x7f01900b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.955+0000 7f01a3400700 1 -- 172.21.15.63:0/1074503781 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f01a40e2630 con 0x7f01ac109670 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.955+0000 7f01a9e00700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f01900b2990 0x7f01900b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.956+0000 7f01a3400700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] conn(0x7f01900bdbd0 0x7f01900c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.956+0000 7f01a3400700 1 -- 172.21.15.63:0/1074503781 --> [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01900c0790 con 0x7f01900bdbd0 2024-05-27T07:31:14.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.956+0000 7f01a3400700 1 -- 172.21.15.63:0/1074503781 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f01a40e29e0 con 0x7f01ac109670 2024-05-27T07:31:14.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.956+0000 7f01a1400700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] conn(0x7f01900bdbd0 0x7f01900c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:14.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.956+0000 7f01a9e00700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f01900b2990 0x7f01900b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f019c006fd0 tx=0x7f019c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:14.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.957+0000 7f01a1400700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] conn(0x7f01900bdbd0 0x7f01900c00c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:14.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.958+0000 7f01a3400700 1 -- 172.21.15.63:0/1074503781 <== osd.0 v2:172.21.15.63:6802/1698060699 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01900c0790 con 0x7f01900bdbd0 2024-05-27T07:31:14.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.976+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 --> [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f01ac0642c0 con 0x7f01900bdbd0 2024-05-27T07:31:14.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.977+0000 7f01a3400700 1 -- 172.21.15.63:0/1074503781 <== osd.0 v2:172.21.15.63:6802/1698060699 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f01ac0642c0 con 0x7f01900bdbd0 2024-05-27T07:31:14.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.977+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] conn(0x7f01900bdbd0 msgr2=0x7f01900c00c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:14.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.977+0000 7f01ab200700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] conn(0x7f01900bdbd0 0x7f01900c00c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.977+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f01900b2990 msgr2=0x7f01900b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:14.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.977+0000 7f01ab200700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f01900b2990 0x7f01900b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f019c006fd0 tx=0x7f019c008040 comp rx=0 tx=0).stop 2024-05-27T07:31:14.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.977+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ac109670 msgr2=0x7f01ac076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:14.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.977+0000 7f01ab200700 1 --2- 172.21.15.63:0/1074503781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01ac109670 0x7f01ac076530 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f01a4007ee0 tx=0x7f01a400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.977+0000 7f01aa800700 1 -- 172.21.15.63:0/1074503781 reap_dead start 2024-05-27T07:31:14.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.978+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 shutdown_connections 2024-05-27T07:31:14.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.978+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 >> 172.21.15.63:0/1074503781 conn(0x7f01ac079d00 msgr2=0x7f01ac101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:14.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.978+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 shutdown_connections 2024-05-27T07:31:14.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:11.978+0000 7f01ab200700 1 -- 172.21.15.63:0/1074503781 wait complete. 2024-05-27T07:31:14.926 INFO:teuthology.orchestra.run.smithi063.stdout:137438953576 2024-05-27T07:31:14.927 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-27T07:31:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.736+0000 7f87cc600700 1 -- 172.21.15.63:0/2003173848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c4109860 msgr2=0x7f87c41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:15.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.736+0000 7f87cc600700 1 --2- 172.21.15.63:0/2003173848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c4109860 0x7f87c41111c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00b0d0 tx=0x7f87bc00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.736+0000 7f87cc600700 1 -- 172.21.15.63:0/2003173848 shutdown_connections 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.736+0000 7f87cc600700 1 --2- 172.21.15.63:0/2003173848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c4109860 0x7f87c41111c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.736+0000 7f87cc600700 1 --2- 172.21.15.63:0/2003173848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c41045e0 0x7f87c41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.736+0000 7f87cc600700 1 --2- 172.21.15.63:0/2003173848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4108f10 0x7f87c4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.736+0000 7f87cc600700 1 -- 172.21.15.63:0/2003173848 >> 172.21.15.63:0/2003173848 conn(0x7f87c4100030 msgr2=0x7f87c4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.737+0000 7f87cc600700 1 -- 172.21.15.63:0/2003173848 shutdown_connections 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.737+0000 7f87cc600700 1 -- 172.21.15.63:0/2003173848 wait complete. 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.738+0000 7f87cc600700 1 Processor -- start 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.738+0000 7f87cc600700 1 -- start start 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.738+0000 7f87cc600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c41045e0 0x7f87c419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:15.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87cc600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c4108f10 0x7f87c419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87cc600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4109860 0x7f87c41a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87cc600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f87c41143c0 con 0x7f87c4108f10 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87cc600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87c4114240 con 0x7f87c4109860 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87cc600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f87c4114540 con 0x7f87c41045e0 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87c9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c4108f10 0x7f87c419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87c9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c4108f10 0x7f87c419d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56284/0 (socket says 172.21.15.63:56284) 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87c9200700 1 -- 172.21.15.63:0/3262126388 learned_addr learned my addr 172.21.15.63:0/3262126388 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87c9c00700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c41045e0 0x7f87c419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87c9200700 1 -- 172.21.15.63:0/3262126388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c41045e0 msgr2=0x7f87c419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87c9200700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87c41045e0 0x7f87c419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87ca600700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4109860 0x7f87c41a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:15.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87c9200700 1 -- 172.21.15.63:0/3262126388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4109860 msgr2=0x7f87c41a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87c9200700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87c4109860 0x7f87c41a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.739+0000 7f87c9200700 1 -- 172.21.15.63:0/3262126388 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c41a1ea0 con 0x7f87c4108f10 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.740+0000 7f87c9200700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c4108f10 0x7f87c419d570 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f87b000d980 tx=0x7f87b000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.740+0000 7f87c2000700 1 -- 172.21.15.63:0/3262126388 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0017070 con 0x7f87c4108f10 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.740+0000 7f87c2000700 1 -- 172.21.15.63:0/3262126388 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87b0010460 con 0x7f87c4108f10 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.740+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c41a2190 con 0x7f87c4108f10 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.740+0000 7f87c2000700 1 -- 172.21.15.63:0/3262126388 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b000de60 con 0x7f87c4108f10 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.740+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c41a9ab0 con 0x7f87c4108f10 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.741+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f87c4076d40 con 0x7f87c4108f10 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.744+0000 7f87c2000700 1 -- 172.21.15.63:0/3262126388 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f87b0010610 con 0x7f87c4108f10 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.746+0000 7f87c2000700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f87ac0b2910 0x7f87ac0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.746+0000 7f87c2000700 1 -- 172.21.15.63:0/3262126388 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f87b00e2710 con 0x7f87c4108f10 2024-05-27T07:31:15.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.747+0000 7f87c9c00700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f87ac0b2910 0x7f87ac0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.747+0000 7f87c2000700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] conn(0x7f87ac0bdb50 0x7f87ac0c0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.747+0000 7f87c2000700 1 -- 172.21.15.63:0/3262126388 --> [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87b0016ec0 con 0x7f87ac0bdb50 2024-05-27T07:31:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.747+0000 7f87c2000700 1 -- 172.21.15.63:0/3262126388 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f87b00e2ac0 con 0x7f87c4108f10 2024-05-27T07:31:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.747+0000 7f87c9c00700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f87ac0b2910 0x7f87ac0b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87b8006fd0 tx=0x7f87b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.748+0000 7f87ca600700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] conn(0x7f87ac0bdb50 0x7f87ac0c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.749+0000 7f87ca600700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] conn(0x7f87ac0bdb50 0x7f87ac0c0040 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.752+0000 7f87c2000700 1 -- 172.21.15.63:0/3262126388 <== osd.8 v2:172.21.15.149:6816/2466164082 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87b0016ec0 con 0x7f87ac0bdb50 2024-05-27T07:31:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.771+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 --> [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f87c4105a90 con 0x7f87ac0bdb50 2024-05-27T07:31:15.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.772+0000 7f87c2000700 1 -- 172.21.15.63:0/3262126388 <== osd.8 v2:172.21.15.149:6816/2466164082 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f87c4105a90 con 0x7f87ac0bdb50 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.773+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 >> [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] conn(0x7f87ac0bdb50 msgr2=0x7f87ac0c0040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.773+0000 7f87cc600700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] conn(0x7f87ac0bdb50 0x7f87ac0c0040 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.773+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f87ac0b2910 msgr2=0x7f87ac0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.773+0000 7f87cc600700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f87ac0b2910 0x7f87ac0b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87b8006fd0 tx=0x7f87b8008040 comp rx=0 tx=0).stop 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.773+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c4108f10 msgr2=0x7f87c419d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.773+0000 7f87cc600700 1 --2- 172.21.15.63:0/3262126388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87c4108f10 0x7f87c419d570 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f87b000d980 tx=0x7f87b000dd40 comp rx=0 tx=0).stop 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.773+0000 7f87ca600700 1 -- 172.21.15.63:0/3262126388 reap_dead start 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.773+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 shutdown_connections 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.773+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 >> 172.21.15.63:0/3262126388 conn(0x7f87c4100030 msgr2=0x7f87c4110ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.774+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 shutdown_connections 2024-05-27T07:31:15.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:13.774+0000 7f87cc600700 1 -- 172.21.15.63:0/3262126388 wait complete. 2024-05-27T07:31:15.404 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:15 smithi063 ceph-mon[31613]: pgmap v476: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:15.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:15 smithi086 ceph-mon[34687]: pgmap v476: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:15.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:15 smithi149 ceph-mon[35408]: pgmap v476: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:16.012 INFO:teuthology.orchestra.run.smithi063.stdout:111669149807 2024-05-27T07:31:16.012 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-27T07:31:16.921 INFO:teuthology.orchestra.run.smithi063.stdout:236223201356 2024-05-27T07:31:16.921 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-27T07:31:16.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.794+0000 7fbb35200700 1 -- 172.21.15.63:0/123844365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb30075af0 msgr2=0x7fbb30075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:16.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.794+0000 7fbb35200700 1 --2- 172.21.15.63:0/123844365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb30075af0 0x7fbb30075f00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009990 tx=0x7fbb20009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:16.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.794+0000 7fbb35200700 1 -- 172.21.15.63:0/123844365 shutdown_connections 2024-05-27T07:31:16.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.794+0000 7fbb35200700 1 --2- 172.21.15.63:0/123844365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb30110bf0 0x7fbb30113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:16.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.794+0000 7fbb35200700 1 --2- 172.21.15.63:0/123844365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb30075af0 0x7fbb30075f00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:16.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.794+0000 7fbb35200700 1 --2- 172.21.15.63:0/123844365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb30076440 0x7fbb301106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:16.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.794+0000 7fbb35200700 1 -- 172.21.15.63:0/123844365 >> 172.21.15.63:0/123844365 conn(0x7fbb30100030 msgr2=0x7fbb30102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:16.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.794+0000 7fbb35200700 1 -- 172.21.15.63:0/123844365 shutdown_connections 2024-05-27T07:31:16.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.794+0000 7fbb35200700 1 -- 172.21.15.63:0/123844365 wait complete. 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.795+0000 7fbb35200700 1 Processor -- start 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.795+0000 7fbb35200700 1 -- start start 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.795+0000 7fbb35200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb30075af0 0x7fbb3019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb35200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb30076440 0x7fbb3019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb35200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb30110bf0 0x7fbb301a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb35200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb301165d0 con 0x7fbb30075af0 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb35200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb30116450 con 0x7fbb30110bf0 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb35200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbb30116750 con 0x7fbb30076440 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb2de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb30075af0 0x7fbb3019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb2de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb30075af0 0x7fbb3019d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56300/0 (socket says 172.21.15.63:56300) 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb2de00700 1 -- 172.21.15.63:0/3258626901 learned_addr learned my addr 172.21.15.63:0/3258626901 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:16.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb27e00700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb30076440 0x7fbb3019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb2e800700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb30110bf0 0x7fbb301a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb2de00700 1 -- 172.21.15.63:0/3258626901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb30076440 msgr2=0x7fbb3019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb2de00700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb30076440 0x7fbb3019d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb2de00700 1 -- 172.21.15.63:0/3258626901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb30110bf0 msgr2=0x7fbb301a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb2de00700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb30110bf0 0x7fbb301a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb2de00700 1 -- 172.21.15.63:0/3258626901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb301a1e60 con 0x7fbb30075af0 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.796+0000 7fbb2e800700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb30110bf0 0x7fbb301a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.797+0000 7fbb2de00700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb30075af0 0x7fbb3019d040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbb20005910 tx=0x7fbb200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.797+0000 7fbb26a00700 1 -- 172.21.15.63:0/3258626901 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20027070 con 0x7fbb30075af0 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.797+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb301a20f0 con 0x7fbb30075af0 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.797+0000 7fbb26a00700 1 -- 172.21.15.63:0/3258626901 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb2000fc00 con 0x7fbb30075af0 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.797+0000 7fbb26a00700 1 -- 172.21.15.63:0/3258626901 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20022d80 con 0x7fbb30075af0 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.797+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb301a2620 con 0x7fbb30075af0 2024-05-27T07:31:16.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.798+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb30063f40 con 0x7fbb30075af0 2024-05-27T07:31:16.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.801+0000 7fbb26a00700 1 -- 172.21.15.63:0/3258626901 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbb2002a070 con 0x7fbb30075af0 2024-05-27T07:31:16.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.802+0000 7fbb26a00700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb140b2990 0x7fbb140b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:16.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.803+0000 7fbb26a00700 1 -- 172.21.15.63:0/3258626901 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fbb200e3df0 con 0x7fbb30075af0 2024-05-27T07:31:16.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.803+0000 7fbb27e00700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb140b2990 0x7fbb140b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:16.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.803+0000 7fbb26a00700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] conn(0x7fbb140bdbd0 0x7fbb140c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:16.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.803+0000 7fbb26a00700 1 -- 172.21.15.63:0/3258626901 --> [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb140c0790 con 0x7fbb140bdbd0 2024-05-27T07:31:16.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.803+0000 7fbb26a00700 1 -- 172.21.15.63:0/3258626901 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fbb200e41a0 con 0x7fbb30075af0 2024-05-27T07:31:16.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.803+0000 7fbb2e800700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] conn(0x7fbb140bdbd0 0x7fbb140c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:16.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.804+0000 7fbb27e00700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb140b2990 0x7fbb140b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005950 tx=0x7fbb180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.804+0000 7fbb2e800700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] conn(0x7fbb140bdbd0 0x7fbb140c00c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.805+0000 7fbb26a00700 1 -- 172.21.15.63:0/3258626901 <== osd.9 v2:172.21.15.63:6826/269558995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb140c0790 con 0x7fbb140bdbd0 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.824+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 --> [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb30064340 con 0x7fbb140bdbd0 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.824+0000 7fbb26a00700 1 -- 172.21.15.63:0/3258626901 <== osd.9 v2:172.21.15.63:6826/269558995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb30064340 con 0x7fbb140bdbd0 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.824+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] conn(0x7fbb140bdbd0 msgr2=0x7fbb140c00c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.824+0000 7fbb35200700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] conn(0x7fbb140bdbd0 0x7fbb140c00c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.825+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb140b2990 msgr2=0x7fbb140b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.825+0000 7fbb35200700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fbb140b2990 0x7fbb140b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005950 tx=0x7fbb180058e0 comp rx=0 tx=0).stop 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.825+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb30075af0 msgr2=0x7fbb3019d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.825+0000 7fbb35200700 1 --2- 172.21.15.63:0/3258626901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb30075af0 0x7fbb3019d040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbb20005910 tx=0x7fbb200047b0 comp rx=0 tx=0).stop 2024-05-27T07:31:16.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.825+0000 7fbb2e800700 1 -- 172.21.15.63:0/3258626901 reap_dead start 2024-05-27T07:31:16.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.825+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 shutdown_connections 2024-05-27T07:31:16.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.825+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 >> 172.21.15.63:0/3258626901 conn(0x7fbb30100030 msgr2=0x7fbb30100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:16.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.825+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 shutdown_connections 2024-05-27T07:31:16.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:16.825+0000 7fbb35200700 1 -- 172.21.15.63:0/3258626901 wait complete. 2024-05-27T07:31:17.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:17 smithi063 ceph-mon[31613]: pgmap v477: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:17.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:17 smithi086 ceph-mon[34687]: pgmap v477: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:17.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:17 smithi149 ceph-mon[35408]: pgmap v477: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:17.619 INFO:teuthology.orchestra.run.smithi063.stdout:34359738502 2024-05-27T07:31:17.620 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-27T07:31:18.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.862+0000 7f75f7a00700 1 -- 172.21.15.63:0/542405411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f0105ed0 msgr2=0x7f75f010e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:18.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.862+0000 7f75f7a00700 1 --2- 172.21.15.63:0/542405411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f0105ed0 0x7f75f010e420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009990 tx=0x7f75dc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.862+0000 7f75f7a00700 1 -- 172.21.15.63:0/542405411 shutdown_connections 2024-05-27T07:31:18.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.862+0000 7f75f7a00700 1 --2- 172.21.15.63:0/542405411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75f010e960 0x7f75f0110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.862+0000 7f75f7a00700 1 --2- 172.21.15.63:0/542405411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0105580 0x7f75f0105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.862+0000 7f75f7a00700 1 --2- 172.21.15.63:0/542405411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f0105ed0 0x7f75f010e420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.862+0000 7f75f7a00700 1 -- 172.21.15.63:0/542405411 >> 172.21.15.63:0/542405411 conn(0x7f75f0078df0 msgr2=0x7f75f0079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:18.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.862+0000 7f75f7a00700 1 -- 172.21.15.63:0/542405411 shutdown_connections 2024-05-27T07:31:18.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.863+0000 7f75f7a00700 1 -- 172.21.15.63:0/542405411 wait complete. 2024-05-27T07:31:18.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.864+0000 7f75f7a00700 1 Processor -- start 2024-05-27T07:31:18.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.864+0000 7f75f7a00700 1 -- start start 2024-05-27T07:31:18.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75f7a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75f0105580 0x7f75f0103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:18.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75f7a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0105ed0 0x7f75f01037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:18.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75f7a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f010e960 0x7f75f0103cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:18.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75f7a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0114380 con 0x7f75f010e960 2024-05-27T07:31:18.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75f7a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0114200 con 0x7f75f0105ed0 2024-05-27T07:31:18.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75f7a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f75f0114500 con 0x7f75f0105580 2024-05-27T07:31:18.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75f5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f010e960 0x7f75f0103cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:18.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75f5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f010e960 0x7f75f0103cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56328/0 (socket says 172.21.15.63:56328) 2024-05-27T07:31:18.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75f5a00700 1 -- 172.21.15.63:0/1156498384 learned_addr learned my addr 172.21.15.63:0/1156498384 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:18.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75efe00700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0105ed0 0x7f75f01037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:18.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.865+0000 7f75f5000700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75f0105580 0x7f75f0103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:18.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.866+0000 7f75f5a00700 1 -- 172.21.15.63:0/1156498384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75f0105580 msgr2=0x7f75f0103270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:18.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.866+0000 7f75f5a00700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75f0105580 0x7f75f0103270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.866+0000 7f75f5a00700 1 -- 172.21.15.63:0/1156498384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0105ed0 msgr2=0x7f75f01037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:18.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.866+0000 7f75f5a00700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75f0105ed0 0x7f75f01037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.866+0000 7f75f5a00700 1 -- 172.21.15.63:0/1156498384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f01a9150 con 0x7f75f010e960 2024-05-27T07:31:18.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.866+0000 7f75f5a00700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f010e960 0x7f75f0103cf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75e8007ee0 tx=0x7f75e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:18.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.867+0000 7f75ed600700 1 -- 172.21.15.63:0/1156498384 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e800dbd0 con 0x7f75f010e960 2024-05-27T07:31:18.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.867+0000 7f75ed600700 1 -- 172.21.15.63:0/1156498384 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75e800dd70 con 0x7f75f010e960 2024-05-27T07:31:18.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.867+0000 7f75ed600700 1 -- 172.21.15.63:0/1156498384 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e80058d0 con 0x7f75f010e960 2024-05-27T07:31:18.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.867+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f01a9440 con 0x7f75f010e960 2024-05-27T07:31:18.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.867+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f01a9930 con 0x7f75f010e960 2024-05-27T07:31:18.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.868+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75f0063ec0 con 0x7f75f010e960 2024-05-27T07:31:18.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.872+0000 7f75ed600700 1 -- 172.21.15.63:0/1156498384 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f75e801a020 con 0x7f75f010e960 2024-05-27T07:31:18.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.873+0000 7f75ed600700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75d80b2910 0x7f75d80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:18.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.874+0000 7f75ed600700 1 -- 172.21.15.63:0/1156498384 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f75e80e2550 con 0x7f75f010e960 2024-05-27T07:31:18.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.874+0000 7f75ed600700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] conn(0x7f75d80bdb50 0x7f75d80c0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:18.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.874+0000 7f75ed600700 1 -- 172.21.15.63:0/1156498384 --> [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75d80c0710 con 0x7f75d80bdb50 2024-05-27T07:31:18.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.874+0000 7f75ed600700 1 -- 172.21.15.63:0/1156498384 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f75e80e29a0 con 0x7f75f010e960 2024-05-27T07:31:18.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.874+0000 7f75f5000700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75d80b2910 0x7f75d80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:18.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.875+0000 7f75efe00700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] conn(0x7f75d80bdb50 0x7f75d80c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:18.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.875+0000 7f75f5000700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75d80b2910 0x7f75d80b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f75e4005950 tx=0x7f75e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:18.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.876+0000 7f75efe00700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] conn(0x7f75d80bdb50 0x7f75d80c0040 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:18.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.879+0000 7f75ed600700 1 -- 172.21.15.63:0/1156498384 <== osd.10 v2:172.21.15.86:6824/1317761853 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75d80c0710 con 0x7f75d80bdb50 2024-05-27T07:31:18.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.897+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 --> [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75f0064340 con 0x7f75d80bdb50 2024-05-27T07:31:18.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.898+0000 7f75ed600700 1 -- 172.21.15.63:0/1156498384 <== osd.10 v2:172.21.15.86:6824/1317761853 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f75f0064340 con 0x7f75d80bdb50 2024-05-27T07:31:18.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.898+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 >> [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] conn(0x7f75d80bdb50 msgr2=0x7f75d80c0040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:18.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.898+0000 7f75f7a00700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] conn(0x7f75d80bdb50 0x7f75d80c0040 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.898+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75d80b2910 msgr2=0x7f75d80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.898+0000 7f75f7a00700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f75d80b2910 0x7f75d80b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f75e4005950 tx=0x7f75e40058e0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.898+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f010e960 msgr2=0x7f75f0103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.898+0000 7f75f7a00700 1 --2- 172.21.15.63:0/1156498384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f010e960 0x7f75f0103cf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75e8007ee0 tx=0x7f75e800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.898+0000 7f75f5a00700 1 -- 172.21.15.63:0/1156498384 reap_dead start 2024-05-27T07:31:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.899+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 shutdown_connections 2024-05-27T07:31:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.899+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 >> 172.21.15.63:0/1156498384 conn(0x7f75f0078df0 msgr2=0x7f75f0100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.899+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 shutdown_connections 2024-05-27T07:31:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.899+0000 7f75f7a00700 1 -- 172.21.15.63:0/1156498384 wait complete. 2024-05-27T07:31:19.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:19 smithi063 ceph-mon[31613]: pgmap v478: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:19.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.766+0000 7f2144800700 1 -- 172.21.15.63:0/878880779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c1045e0 msgr2=0x7f213c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:19.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.766+0000 7f2144800700 1 --2- 172.21.15.63:0/878880779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c1045e0 0x7f213c1049f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f21300099c0 tx=0x7f2130009cd0 comp rx=0 tx=0).stop 2024-05-27T07:31:19.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.766+0000 7f2144800700 1 -- 172.21.15.63:0/878880779 shutdown_connections 2024-05-27T07:31:19.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.766+0000 7f2144800700 1 --2- 172.21.15.63:0/878880779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c109860 0x7f213c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:19.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.766+0000 7f2144800700 1 --2- 172.21.15.63:0/878880779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c1045e0 0x7f213c1049f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:19.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.766+0000 7f2144800700 1 --2- 172.21.15.63:0/878880779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f213c108f10 0x7f213c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.766+0000 7f2144800700 1 -- 172.21.15.63:0/878880779 >> 172.21.15.63:0/878880779 conn(0x7f213c100030 msgr2=0x7f213c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.766+0000 7f2144800700 1 -- 172.21.15.63:0/878880779 shutdown_connections 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.766+0000 7f2144800700 1 -- 172.21.15.63:0/878880779 wait complete. 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.768+0000 7f2144800700 1 Processor -- start 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.768+0000 7f2144800700 1 -- start start 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2144800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f213c1045e0 0x7f213c075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2144800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c108f10 0x7f213c075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2144800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c109860 0x7f213c076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2144800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f213c114360 con 0x7f213c109860 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2144800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f213c1141e0 con 0x7f213c108f10 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2144800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f213c1144e0 con 0x7f213c1045e0 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2142800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c109860 0x7f213c076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:19.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2142800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c109860 0x7f213c076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56304/0 (socket says 172.21.15.63:56304) 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2142800700 1 -- 172.21.15.63:0/830794623 learned_addr learned my addr 172.21.15.63:0/830794623 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2141e00700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f213c1045e0 0x7f213c075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.769+0000 7f2141400700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c108f10 0x7f213c075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.770+0000 7f2142800700 1 -- 172.21.15.63:0/830794623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f213c1045e0 msgr2=0x7f213c075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.770+0000 7f2142800700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f213c1045e0 0x7f213c075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.770+0000 7f2142800700 1 -- 172.21.15.63:0/830794623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c108f10 msgr2=0x7f213c075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.770+0000 7f2142800700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f213c108f10 0x7f213c075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.770+0000 7f2142800700 1 -- 172.21.15.63:0/830794623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c1a90f0 con 0x7f213c109860 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.770+0000 7f2142800700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c109860 0x7f213c076530 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2138007ee0 tx=0x7f213800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.770+0000 7f2136a00700 1 -- 172.21.15.63:0/830794623 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138004bb0 con 0x7f213c109860 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.770+0000 7f2136a00700 1 -- 172.21.15.63:0/830794623 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2138004d50 con 0x7f213c109860 2024-05-27T07:31:19.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.770+0000 7f2136a00700 1 -- 172.21.15.63:0/830794623 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21380058b0 con 0x7f213c109860 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.770+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213c1a93e0 con 0x7f213c109860 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.771+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213c1a98d0 con 0x7f213c109860 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.772+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f213c063ec0 con 0x7f213c109860 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.774+0000 7f2136a00700 1 -- 172.21.15.63:0/830794623 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f213801a020 con 0x7f213c109860 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.776+0000 7f2136a00700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f21240b2910 0x7f21240b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.776+0000 7f2136a00700 1 -- 172.21.15.63:0/830794623 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f21380e2440 con 0x7f213c109860 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.776+0000 7f2141e00700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f21240b2910 0x7f21240b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.776+0000 7f2136a00700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] conn(0x7f21240bdb50 0x7f21240c0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.776+0000 7f2136a00700 1 -- 172.21.15.63:0/830794623 --> [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2138012ec0 con 0x7f21240bdb50 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.776+0000 7f2136a00700 1 -- 172.21.15.63:0/830794623 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f21380e2890 con 0x7f213c109860 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.777+0000 7f2141400700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] conn(0x7f21240bdb50 0x7f21240c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.777+0000 7f2141e00700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f21240b2910 0x7f21240b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f21300058e0 tx=0x7f2130005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.778+0000 7f2141400700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] conn(0x7f21240bdb50 0x7f21240c0040 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:19.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.781+0000 7f2136a00700 1 -- 172.21.15.63:0/830794623 <== osd.2 v2:172.21.15.149:6800/363230187 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2138012ec0 con 0x7f21240bdb50 2024-05-27T07:31:19.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.799+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 --> [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f213c064340 con 0x7f21240bdb50 2024-05-27T07:31:19.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.800+0000 7f2136a00700 1 -- 172.21.15.63:0/830794623 <== osd.2 v2:172.21.15.149:6800/363230187 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f213c064340 con 0x7f21240bdb50 2024-05-27T07:31:19.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.801+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 >> [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] conn(0x7f21240bdb50 msgr2=0x7f21240c0040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:19.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.801+0000 7f2144800700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] conn(0x7f21240bdb50 0x7f21240c0040 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:19.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.801+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f21240b2910 msgr2=0x7f21240b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:19.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.801+0000 7f2144800700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f21240b2910 0x7f21240b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f21300058e0 tx=0x7f2130005fb0 comp rx=0 tx=0).stop 2024-05-27T07:31:19.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.801+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c109860 msgr2=0x7f213c076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:19.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.801+0000 7f2144800700 1 --2- 172.21.15.63:0/830794623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f213c109860 0x7f213c076530 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2138007ee0 tx=0x7f213800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:31:19.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.801+0000 7f2142800700 1 -- 172.21.15.63:0/830794623 reap_dead start 2024-05-27T07:31:19.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.801+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 shutdown_connections 2024-05-27T07:31:19.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.801+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 >> 172.21.15.63:0/830794623 conn(0x7f213c100030 msgr2=0x7f213c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:19.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.802+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 shutdown_connections 2024-05-27T07:31:19.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:18.802+0000 7f2144800700 1 -- 172.21.15.63:0/830794623 wait complete. 2024-05-27T07:31:19.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:19 smithi086 ceph-mon[34687]: pgmap v478: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:19.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:19 smithi149 ceph-mon[35408]: pgmap v478: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:19.736 INFO:teuthology.orchestra.run.smithi063.stdout:210453397587 2024-05-27T07:31:19.737 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-27T07:31:20.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.362+0000 7f0c39200700 1 -- 172.21.15.63:0/815212290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c341045e0 msgr2=0x7f0c341049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:20.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.363+0000 7f0c39200700 1 --2- 172.21.15.63:0/815212290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c341045e0 0x7f0c341049f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0c240099c0 tx=0x7f0c24009cd0 comp rx=0 tx=0).stop 2024-05-27T07:31:20.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.363+0000 7f0c39200700 1 -- 172.21.15.63:0/815212290 shutdown_connections 2024-05-27T07:31:20.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.363+0000 7f0c39200700 1 --2- 172.21.15.63:0/815212290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34109860 0x7f0c341111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:20.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.363+0000 7f0c39200700 1 --2- 172.21.15.63:0/815212290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c341045e0 0x7f0c341049f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:20.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.363+0000 7f0c39200700 1 --2- 172.21.15.63:0/815212290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c34108f10 0x7f0c34109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:20.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.363+0000 7f0c39200700 1 -- 172.21.15.63:0/815212290 >> 172.21.15.63:0/815212290 conn(0x7f0c34100030 msgr2=0x7f0c34102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:20.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.363+0000 7f0c39200700 1 -- 172.21.15.63:0/815212290 shutdown_connections 2024-05-27T07:31:20.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.363+0000 7f0c39200700 1 -- 172.21.15.63:0/815212290 wait complete. 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.364+0000 7f0c39200700 1 Processor -- start 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.364+0000 7f0c39200700 1 -- start start 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c39200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c341045e0 0x7f0c3419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c39200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c34108f10 0x7f0c3419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c39200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c34109860 0x7f0c341a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c39200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34114390 con 0x7f0c34109860 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c39200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34114210 con 0x7f0c341045e0 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c39200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34114510 con 0x7f0c34108f10 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c32800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c34109860 0x7f0c341a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c32800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c34109860 0x7f0c341a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42030/0 (socket says 172.21.15.63:42030) 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c32800700 1 -- 172.21.15.63:0/520449028 learned_addr learned my addr 172.21.15.63:0/520449028 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c31e00700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c341045e0 0x7f0c3419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:20.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.365+0000 7f0c2be00700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c34108f10 0x7f0c3419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:20.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.366+0000 7f0c32800700 1 -- 172.21.15.63:0/520449028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c34108f10 msgr2=0x7f0c3419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:20.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.366+0000 7f0c32800700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c34108f10 0x7f0c3419d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:20.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.366+0000 7f0c32800700 1 -- 172.21.15.63:0/520449028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c341045e0 msgr2=0x7f0c3419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:20.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.366+0000 7f0c32800700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c341045e0 0x7f0c3419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:20.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.366+0000 7f0c32800700 1 -- 172.21.15.63:0/520449028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c341a1de0 con 0x7f0c34109860 2024-05-27T07:31:20.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.366+0000 7f0c32800700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c34109860 0x7f0c341a17f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c007d70 tx=0x7f0c2c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:20.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.366+0000 7f0c2aa00700 1 -- 172.21.15.63:0/520449028 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c016020 con 0x7f0c34109860 2024-05-27T07:31:20.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.366+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c341a20d0 con 0x7f0c34109860 2024-05-27T07:31:20.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.366+0000 7f0c2aa00700 1 -- 172.21.15.63:0/520449028 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c2c00ad40 con 0x7f0c34109860 2024-05-27T07:31:20.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.366+0000 7f0c2aa00700 1 -- 172.21.15.63:0/520449028 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0115e0 con 0x7f0c34109860 2024-05-27T07:31:20.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.367+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c341a2610 con 0x7f0c34109860 2024-05-27T07:31:20.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.367+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c34110800 con 0x7f0c34109860 2024-05-27T07:31:20.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.370+0000 7f0c2aa00700 1 -- 172.21.15.63:0/520449028 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0c2c019020 con 0x7f0c34109860 2024-05-27T07:31:20.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.372+0000 7f0c2aa00700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0c180b2990 0x7f0c180b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.372+0000 7f0c2aa00700 1 -- 172.21.15.63:0/520449028 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f0c2c0e2b30 con 0x7f0c34109860 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.372+0000 7f0c31e00700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0c180b2990 0x7f0c180b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.372+0000 7f0c2aa00700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] conn(0x7f0c180bdbd0 0x7f0c180c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.372+0000 7f0c2aa00700 1 -- 172.21.15.63:0/520449028 --> [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c180c0790 con 0x7f0c180bdbd0 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.372+0000 7f0c2aa00700 1 -- 172.21.15.63:0/520449028 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f0c2c0e2ee0 con 0x7f0c34109860 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.373+0000 7f0c31e00700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0c180b2990 0x7f0c180b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c24000c00 tx=0x7f0c24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.373+0000 7f0c2be00700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] conn(0x7f0c180bdbd0 0x7f0c180c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.374+0000 7f0c2be00700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] conn(0x7f0c180bdbd0 0x7f0c180c00c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.376+0000 7f0c2aa00700 1 -- 172.21.15.63:0/520449028 <== osd.11 v2:172.21.15.149:6824/4045663443 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c180c0790 con 0x7f0c180bdbd0 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.395+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 --> [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c34105a90 con 0x7f0c180bdbd0 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.397+0000 7f0c2aa00700 1 -- 172.21.15.63:0/520449028 <== osd.11 v2:172.21.15.149:6824/4045663443 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0c34105a90 con 0x7f0c180bdbd0 2024-05-27T07:31:20.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.397+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 >> [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] conn(0x7f0c180bdbd0 msgr2=0x7f0c180c00c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:20.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.397+0000 7f0c39200700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] conn(0x7f0c180bdbd0 0x7f0c180c00c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:20.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.397+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0c180b2990 msgr2=0x7f0c180b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:20.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.397+0000 7f0c39200700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f0c180b2990 0x7f0c180b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c24000c00 tx=0x7f0c24005fb0 comp rx=0 tx=0).stop 2024-05-27T07:31:20.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.397+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c34109860 msgr2=0x7f0c341a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:20.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.397+0000 7f0c39200700 1 --2- 172.21.15.63:0/520449028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c34109860 0x7f0c341a17f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c007d70 tx=0x7f0c2c00a340 comp rx=0 tx=0).stop 2024-05-27T07:31:20.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.397+0000 7f0c32800700 1 -- 172.21.15.63:0/520449028 reap_dead start 2024-05-27T07:31:20.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.398+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 shutdown_connections 2024-05-27T07:31:20.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.398+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 >> 172.21.15.63:0/520449028 conn(0x7f0c34100030 msgr2=0x7f0c3410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:20.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.398+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 shutdown_connections 2024-05-27T07:31:20.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:20.398+0000 7f0c39200700 1 -- 172.21.15.63:0/520449028 wait complete. 2024-05-27T07:31:21.167 INFO:teuthology.orchestra.run.smithi063.stdout:283467841597 2024-05-27T07:31:21.167 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-27T07:31:21.278 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:21.324 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:21 smithi063 ceph-mon[31613]: pgmap v479: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:21.326 INFO:teuthology.orchestra.run.smithi063.stdout:261993005125 2024-05-27T07:31:21.326 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-27T07:31:21.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:21 smithi086 ceph-mon[34687]: pgmap v479: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:21.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:21 smithi149 ceph-mon[35408]: pgmap v479: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:22.069 INFO:teuthology.orchestra.run.smithi063.stdout:77309411449 2024-05-27T07:31:22.069 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-27T07:31:22.201 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:22.205 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:22.207 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:22.207 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:22.248 INFO:teuthology.orchestra.run.smithi063.stdout:309237645366 2024-05-27T07:31:22.248 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-27T07:31:22.449 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:23.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:23 smithi063 ceph-mon[31613]: pgmap v480: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:23.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:23 smithi086 ceph-mon[34687]: pgmap v480: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:23.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:23 smithi149 ceph-mon[35408]: pgmap v480: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:25.292 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:25.339 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:25 smithi063 ceph-mon[31613]: pgmap v481: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:25.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:25 smithi086 ceph-mon[34687]: pgmap v481: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:25.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:25 smithi149 ceph-mon[35408]: pgmap v481: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:26.025 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:27.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:27 smithi063 ceph-mon[31613]: pgmap v482: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:27.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:27 smithi086 ceph-mon[34687]: pgmap v482: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:27.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:27 smithi149 ceph-mon[35408]: pgmap v482: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:28.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:28 smithi086 ceph-mon[34687]: pgmap v483: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:28.454 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:28 smithi149 ceph-mon[35408]: pgmap v483: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:28.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:28 smithi063 ceph-mon[31613]: pgmap v483: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:30.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.041+0000 7f1c33200700 1 -- 172.21.15.63:0/805880084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c34108d20 msgr2=0x7f1c34109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:30.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.041+0000 7f1c33200700 1 --2- 172.21.15.63:0/805880084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c34108d20 0x7f1c34109130 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1c200099c0 tx=0x7f1c20009cd0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.041+0000 7f1c33200700 1 -- 172.21.15.63:0/805880084 shutdown_connections 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.041+0000 7f1c33200700 1 --2- 172.21.15.63:0/805880084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34109670 0x7f1c34110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.041+0000 7f1c33200700 1 --2- 172.21.15.63:0/805880084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c341043f0 0x7f1c34104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.041+0000 7f1c33200700 1 --2- 172.21.15.63:0/805880084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c34108d20 0x7f1c34109130 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.041+0000 7f1c33200700 1 -- 172.21.15.63:0/805880084 >> 172.21.15.63:0/805880084 conn(0x7f1c34079d00 msgr2=0x7f1c3407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.042+0000 7f1c33200700 1 -- 172.21.15.63:0/805880084 shutdown_connections 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.042+0000 7f1c33200700 1 -- 172.21.15.63:0/805880084 wait complete. 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.043+0000 7f1c33200700 1 Processor -- start 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.043+0000 7f1c33200700 1 -- start start 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.043+0000 7f1c33200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c341043f0 0x7f1c3419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:30.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c33200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c34108d20 0x7f1c3419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:30.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c33200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34109670 0x7f1c341a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:30.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c33200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34114180 con 0x7f1c34108d20 2024-05-27T07:31:30.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c33200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34114000 con 0x7f1c341043f0 2024-05-27T07:31:30.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c33200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34114300 con 0x7f1c34109670 2024-05-27T07:31:30.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c31400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c34108d20 0x7f1c3419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:30.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c31400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c34108d20 0x7f1c3419d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44946/0 (socket says 172.21.15.63:44946) 2024-05-27T07:31:30.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c31400700 1 -- 172.21.15.63:0/1500041507 learned_addr learned my addr 172.21.15.63:0/1500041507 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:30.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c31e00700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c341043f0 0x7f1c3419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:30.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c31400700 1 -- 172.21.15.63:0/1500041507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34109670 msgr2=0x7f1c341a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:30.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c31400700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34109670 0x7f1c341a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c31400700 1 -- 172.21.15.63:0/1500041507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c341043f0 msgr2=0x7f1c3419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c31400700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c341043f0 0x7f1c3419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.044+0000 7f1c31400700 1 -- 172.21.15.63:0/1500041507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c341a1ca0 con 0x7f1c34108d20 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.045+0000 7f1c31e00700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c341043f0 0x7f1c3419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.045+0000 7f1c31400700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c34108d20 0x7f1c3419d370 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1c200098e0 tx=0x7f1c2000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.045+0000 7f1c2aa00700 1 -- 172.21.15.63:0/1500041507 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20027070 con 0x7f1c34108d20 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.045+0000 7f1c2aa00700 1 -- 172.21.15.63:0/1500041507 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c2000fdd0 con 0x7f1c34108d20 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.045+0000 7f1c33200700 1 -- 172.21.15.63:0/1500041507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c341a1f30 con 0x7f1c34108d20 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.045+0000 7f1c2aa00700 1 -- 172.21.15.63:0/1500041507 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20022e30 con 0x7f1c34108d20 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.045+0000 7f1c33200700 1 -- 172.21.15.63:0/1500041507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c341a98b0 con 0x7f1c34108d20 2024-05-27T07:31:30.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.047+0000 7f1c33200700 1 -- 172.21.15.63:0/1500041507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c341058a0 con 0x7f1c34108d20 2024-05-27T07:31:30.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.049+0000 7f1c2aa00700 1 -- 172.21.15.63:0/1500041507 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1c2002a070 con 0x7f1c34108d20 2024-05-27T07:31:30.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.050+0000 7f1c2aa00700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1c180b2990 0x7f1c180b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:30.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.051+0000 7f1c2aa00700 1 -- 172.21.15.63:0/1500041507 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f1c200e33f0 con 0x7f1c34108d20 2024-05-27T07:31:30.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.051+0000 7f1c31e00700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1c180b2990 0x7f1c180b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:30.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.052+0000 7f1c31e00700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1c180b2990 0x7f1c180b4ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005950 tx=0x7f1c240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:30.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.056+0000 7f1c2aa00700 1 -- 172.21.15.63:0/1500041507 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c20013070 con 0x7f1c34108d20 2024-05-27T07:31:30.279 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:30.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.319+0000 7f1c33200700 1 -- 172.21.15.63:0/1500041507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1c341a9a80 con 0x7f1c34108d20 2024-05-27T07:31:30.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.320+0000 7f1c2aa00700 1 -- 172.21.15.63:0/1500041507 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1c200aacb0 con 0x7f1c34108d20 2024-05-27T07:31:30.313 INFO:teuthology.orchestra.run.smithi063.stdout:137438953580 2024-05-27T07:31:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 -- 172.21.15.63:0/1500041507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1c180b2990 msgr2=0x7f1c180b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1c180b2990 0x7f1c180b4ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005950 tx=0x7f1c240058e0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 -- 172.21.15.63:0/1500041507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c34108d20 msgr2=0x7f1c3419d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c34108d20 0x7f1c3419d370 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1c200098e0 tx=0x7f1c2000f510 comp rx=0 tx=0).stop 2024-05-27T07:31:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 -- 172.21.15.63:0/1500041507 shutdown_connections 2024-05-27T07:31:30.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c34109670 0x7f1c341a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c341043f0 0x7f1c3419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f1c180b2990 0x7f1c180b4ea0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 --2- 172.21.15.63:0/1500041507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c34108d20 0x7f1c3419d370 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 -- 172.21.15.63:0/1500041507 >> 172.21.15.63:0/1500041507 conn(0x7f1c34079d00 msgr2=0x7f1c3410f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 -- 172.21.15.63:0/1500041507 shutdown_connections 2024-05-27T07:31:30.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:30.324+0000 7f1c33200700 1 -- 172.21.15.63:0/1500041507 wait complete. 2024-05-27T07:31:31.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:30 smithi086 ceph-mon[34687]: pgmap v484: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:31.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:30 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1500041507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T07:31:31.205 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:30 smithi149 ceph-mon[35408]: pgmap v484: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:31.205 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:30 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1500041507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T07:31:31.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:30 smithi063 ceph-mon[31613]: pgmap v484: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:31:31.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:30 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1500041507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T07:31:31.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.105+0000 7fba87a00700 1 -- 172.21.15.63:0/3254520072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8010a950 msgr2=0x7fba8010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.105+0000 7fba87a00700 1 --2- 172.21.15.63:0/3254520072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8010a950 0x7fba8010ad60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009990 tx=0x7fba6c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.105+0000 7fba87a00700 1 -- 172.21.15.63:0/3254520072 shutdown_connections 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.105+0000 7fba87a00700 1 --2- 172.21.15.63:0/3254520072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba8010b2a0 0x7fba80117820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.105+0000 7fba87a00700 1 --2- 172.21.15.63:0/3254520072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba80110980 0x7fba80110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.105+0000 7fba87a00700 1 --2- 172.21.15.63:0/3254520072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8010a950 0x7fba8010ad60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.105+0000 7fba87a00700 1 -- 172.21.15.63:0/3254520072 >> 172.21.15.63:0/3254520072 conn(0x7fba80076d20 msgr2=0x7fba80077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.105+0000 7fba87a00700 1 -- 172.21.15.63:0/3254520072 shutdown_connections 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.105+0000 7fba87a00700 1 -- 172.21.15.63:0/3254520072 wait complete. 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.107+0000 7fba87a00700 1 Processor -- start 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.107+0000 7fba87a00700 1 -- start start 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba87a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba8010a950 0x7fba801a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:31.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba87a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba8010b2a0 0x7fba801a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba87a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba80110980 0x7fba801a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba87a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba8011a9b0 con 0x7fba80110980 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba87a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba8011a830 con 0x7fba8010b2a0 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba87a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba8011ab30 con 0x7fba8010a950 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba85a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba80110980 0x7fba801a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba85a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba80110980 0x7fba801a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44964/0 (socket says 172.21.15.63:44964) 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba85a00700 1 -- 172.21.15.63:0/1082498369 learned_addr learned my addr 172.21.15.63:0/1082498369 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba85000700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba8010a950 0x7fba801a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.108+0000 7fba7fe00700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba8010b2a0 0x7fba801a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.109+0000 7fba85a00700 1 -- 172.21.15.63:0/1082498369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba8010a950 msgr2=0x7fba801a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:31.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.109+0000 7fba85a00700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba8010a950 0x7fba801a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.109+0000 7fba85a00700 1 -- 172.21.15.63:0/1082498369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba8010b2a0 msgr2=0x7fba801a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.109+0000 7fba85a00700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba8010b2a0 0x7fba801a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.109+0000 7fba85a00700 1 -- 172.21.15.63:0/1082498369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba801a6290 con 0x7fba80110980 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.109+0000 7fba85a00700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba80110980 0x7fba801a5bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fba7800b970 tx=0x7fba7800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.109+0000 7fba7d600700 1 -- 172.21.15.63:0/1082498369 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78019070 con 0x7fba80110980 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.109+0000 7fba87a00700 1 -- 172.21.15.63:0/1082498369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba801a6580 con 0x7fba80110980 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.109+0000 7fba7d600700 1 -- 172.21.15.63:0/1082498369 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba7800ce40 con 0x7fba80110980 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.109+0000 7fba7d600700 1 -- 172.21.15.63:0/1082498369 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78007370 con 0x7fba80110980 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.110+0000 7fba87a00700 1 -- 172.21.15.63:0/1082498369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba801adea0 con 0x7fba80110980 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.111+0000 7fba87a00700 1 -- 172.21.15.63:0/1082498369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8010c090 con 0x7fba80110980 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.113+0000 7fba7d600700 1 -- 172.21.15.63:0/1082498369 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fba7801c070 con 0x7fba80110980 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.115+0000 7fba7d600700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba680b2910 0x7fba680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.115+0000 7fba7d600700 1 -- 172.21.15.63:0/1082498369 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fba780e2330 con 0x7fba80110980 2024-05-27T07:31:31.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.115+0000 7fba85000700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba680b2910 0x7fba680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.116+0000 7fba85000700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba680b2910 0x7fba680b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fba74005950 tx=0x7fba740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.117+0000 7fba7d600700 1 -- 172.21.15.63:0/1082498369 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba78023070 con 0x7fba80110980 2024-05-27T07:31:31.384 INFO:teuthology.orchestra.run.smithi063.stdout:111669149812 2024-05-27T07:31:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.363+0000 7fba87a00700 1 -- 172.21.15.63:0/1082498369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fba8002e5e0 con 0x7fba80110980 2024-05-27T07:31:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.364+0000 7fba7d600700 1 -- 172.21.15.63:0/1082498369 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fba780a9ca0 con 0x7fba80110980 2024-05-27T07:31:31.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.367+0000 7fba87a00700 1 -- 172.21.15.63:0/1082498369 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba680b2910 msgr2=0x7fba680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.367+0000 7fba87a00700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba680b2910 0x7fba680b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fba74005950 tx=0x7fba740058e0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.367+0000 7fba87a00700 1 -- 172.21.15.63:0/1082498369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba80110980 msgr2=0x7fba801a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.367+0000 7fba87a00700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba80110980 0x7fba801a5bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fba7800b970 tx=0x7fba7800bd30 comp rx=0 tx=0).stop 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.368+0000 7fba87a00700 1 -- 172.21.15.63:0/1082498369 shutdown_connections 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.368+0000 7fba87a00700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fba680b2910 0x7fba680b4e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.368+0000 7fba87a00700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba80110980 0x7fba801a5bf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.368+0000 7fba87a00700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba8010a950 0x7fba801a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.368+0000 7fba87a00700 1 --2- 172.21.15.63:0/1082498369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba8010b2a0 0x7fba801a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.368+0000 7fba87a00700 1 -- 172.21.15.63:0/1082498369 >> 172.21.15.63:0/1082498369 conn(0x7fba80076d20 msgr2=0x7fba80108930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.368+0000 7fba87a00700 1 -- 172.21.15.63:0/1082498369 shutdown_connections 2024-05-27T07:31:31.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:31.368+0000 7fba87a00700 1 -- 172.21.15.63:0/1082498369 wait complete. 2024-05-27T07:31:32.094 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:31 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1082498369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T07:31:32.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:31 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1082498369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T07:31:32.205 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:31 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1082498369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T07:31:32.957 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:32 smithi063 ceph-mon[31613]: pgmap v485: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:33.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:32 smithi086 ceph-mon[34687]: pgmap v485: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:33.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:32 smithi149 ceph-mon[35408]: pgmap v485: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:33.395 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:34.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.092+0000 7f81a5400700 1 -- 172.21.15.63:0/3042035787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81a0075d20 msgr2=0x7f81a0076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.092+0000 7f81a5400700 1 --2- 172.21.15.63:0/3042035787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81a0075d20 0x7f81a0076130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8190009990 tx=0x7f8190009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.092+0000 7f81a5400700 1 -- 172.21.15.63:0/3042035787 shutdown_connections 2024-05-27T07:31:34.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.092+0000 7f81a5400700 1 --2- 172.21.15.63:0/3042035787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a0110b90 0x7f81a0112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.092+0000 7f81a5400700 1 --2- 172.21.15.63:0/3042035787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81a0075d20 0x7f81a0076130 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.092+0000 7f81a5400700 1 --2- 172.21.15.63:0/3042035787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a0076670 0x7f81a0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.092+0000 7f81a5400700 1 -- 172.21.15.63:0/3042035787 >> 172.21.15.63:0/3042035787 conn(0x7f81a0100030 msgr2=0x7f81a0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:34.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.092+0000 7f81a5400700 1 -- 172.21.15.63:0/3042035787 shutdown_connections 2024-05-27T07:31:34.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.092+0000 7f81a5400700 1 -- 172.21.15.63:0/3042035787 wait complete. 2024-05-27T07:31:34.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.094+0000 7f81a5400700 1 Processor -- start 2024-05-27T07:31:34.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.094+0000 7f81a5400700 1 -- start start 2024-05-27T07:31:34.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.094+0000 7f81a5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81a0075d20 0x7f81a019d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:34.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.094+0000 7f81a5400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a0076670 0x7f81a019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:34.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.094+0000 7f81a5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a0110b90 0x7f81a01a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:34.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.094+0000 7f81a5400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0116600 con 0x7f81a0075d20 2024-05-27T07:31:34.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.094+0000 7f81a5400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0116480 con 0x7f81a0110b90 2024-05-27T07:31:34.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.094+0000 7f81a5400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f81a0116780 con 0x7f81a0076670 2024-05-27T07:31:34.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f819e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81a0075d20 0x7f81a019d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:34.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f819e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81a0075d20 0x7f81a019d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45018/0 (socket says 172.21.15.63:45018) 2024-05-27T07:31:34.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f819e800700 1 -- 172.21.15.63:0/4075263842 learned_addr learned my addr 172.21.15.63:0/4075263842 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:34.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f8197e00700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a0076670 0x7f81a019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:34.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f819f200700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a0110b90 0x7f81a01a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:34.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f819e800700 1 -- 172.21.15.63:0/4075263842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a0076670 msgr2=0x7f81a019d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f819e800700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a0076670 0x7f81a019d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f819e800700 1 -- 172.21.15.63:0/4075263842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a0110b90 msgr2=0x7f81a01a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f819e800700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a0110b90 0x7f81a01a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f819e800700 1 -- 172.21.15.63:0/4075263842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a01a1e90 con 0x7f81a0075d20 2024-05-27T07:31:34.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.095+0000 7f819f200700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a0110b90 0x7f81a01a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:31:34.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.096+0000 7f819e800700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81a0075d20 0x7f81a019d070 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8190005910 tx=0x7f81900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:34.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.096+0000 7f8197400700 1 -- 172.21.15.63:0/4075263842 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190027070 con 0x7f81a0075d20 2024-05-27T07:31:34.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.097+0000 7f8197400700 1 -- 172.21.15.63:0/4075263842 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f819000fc80 con 0x7f81a0075d20 2024-05-27T07:31:34.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.097+0000 7f8197400700 1 -- 172.21.15.63:0/4075263842 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81900173f0 con 0x7f81a0075d20 2024-05-27T07:31:34.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.097+0000 7f81a5400700 1 -- 172.21.15.63:0/4075263842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a01a2120 con 0x7f81a0075d20 2024-05-27T07:31:34.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.097+0000 7f81a5400700 1 -- 172.21.15.63:0/4075263842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a01a2650 con 0x7f81a0075d20 2024-05-27T07:31:34.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.099+0000 7f81a5400700 1 -- 172.21.15.63:0/4075263842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a00512b0 con 0x7f81a0075d20 2024-05-27T07:31:34.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.102+0000 7f8197400700 1 -- 172.21.15.63:0/4075263842 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f819002a070 con 0x7f81a0075d20 2024-05-27T07:31:34.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.104+0000 7f8197400700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f81840b2990 0x7f81840b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:34.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.104+0000 7f8197400700 1 -- 172.21.15.63:0/4075263842 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f81900e3750 con 0x7f81a0075d20 2024-05-27T07:31:34.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.104+0000 7f8197e00700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f81840b2990 0x7f81840b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:34.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.105+0000 7f8197e00700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f81840b2990 0x7f81840b4ea0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f81a0103520 tx=0x7f8188009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:34.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.109+0000 7f8197400700 1 -- 172.21.15.63:0/4075263842 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8190013070 con 0x7f81a0075d20 2024-05-27T07:31:34.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:33 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:31:34.204 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:33 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:31:34.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:33 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:31:34.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.349+0000 7f81a5400700 1 -- 172.21.15.63:0/4075263842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f81a002e5e0 con 0x7f81a0075d20 2024-05-27T07:31:34.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.350+0000 7f8197400700 1 -- 172.21.15.63:0/4075263842 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f81900aafb0 con 0x7f81a0075d20 2024-05-27T07:31:34.344 INFO:teuthology.orchestra.run.smithi063.stdout:55834574979 2024-05-27T07:31:34.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.353+0000 7f81a5400700 1 -- 172.21.15.63:0/4075263842 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f81840b2990 msgr2=0x7f81840b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.353+0000 7f81a5400700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f81840b2990 0x7f81840b4ea0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f81a0103520 tx=0x7f8188009190 comp rx=0 tx=0).stop 2024-05-27T07:31:34.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.353+0000 7f81a5400700 1 -- 172.21.15.63:0/4075263842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81a0075d20 msgr2=0x7f81a019d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.353+0000 7f81a5400700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81a0075d20 0x7f81a019d070 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8190005910 tx=0x7f81900047b0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.354+0000 7f81a5400700 1 -- 172.21.15.63:0/4075263842 shutdown_connections 2024-05-27T07:31:34.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.354+0000 7f81a5400700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81a0110b90 0x7f81a01a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.354+0000 7f81a5400700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f81840b2990 0x7f81840b4ea0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.354+0000 7f81a5400700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81a0075d20 0x7f81a019d070 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.354+0000 7f81a5400700 1 --2- 172.21.15.63:0/4075263842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a0076670 0x7f81a019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.354+0000 7f81a5400700 1 -- 172.21.15.63:0/4075263842 >> 172.21.15.63:0/4075263842 conn(0x7f81a0100030 msgr2=0x7f81a0100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:34.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.354+0000 7f81a5400700 1 -- 172.21.15.63:0/4075263842 shutdown_connections 2024-05-27T07:31:34.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:34.354+0000 7f81a5400700 1 -- 172.21.15.63:0/4075263842 wait complete. 2024-05-27T07:31:34.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.649+0000 7f2c82000700 1 -- 172.21.15.63:0/735652108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c7c077530 msgr2=0x7f2c7c1106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.649+0000 7f2c82000700 1 --2- 172.21.15.63:0/735652108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c7c077530 0x7f2c7c1106b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009990 tx=0x7f2c68009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.649+0000 7f2c82000700 1 -- 172.21.15.63:0/735652108 shutdown_connections 2024-05-27T07:31:34.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.649+0000 7f2c82000700 1 --2- 172.21.15.63:0/735652108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c7c110bf0 0x7f2c7c113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.649+0000 7f2c82000700 1 --2- 172.21.15.63:0/735652108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c7c076be0 0x7f2c7c076ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.649+0000 7f2c82000700 1 --2- 172.21.15.63:0/735652108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c7c077530 0x7f2c7c1106b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.649+0000 7f2c82000700 1 -- 172.21.15.63:0/735652108 >> 172.21.15.63:0/735652108 conn(0x7f2c7c0ffe90 msgr2=0x7f2c7c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.650+0000 7f2c82000700 1 -- 172.21.15.63:0/735652108 shutdown_connections 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.650+0000 7f2c82000700 1 -- 172.21.15.63:0/735652108 wait complete. 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.651+0000 7f2c82000700 1 Processor -- start 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.651+0000 7f2c82000700 1 -- start start 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.651+0000 7f2c82000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c7c076be0 0x7f2c7c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.651+0000 7f2c82000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c7c077530 0x7f2c7c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.651+0000 7f2c82000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c7c110bf0 0x7f2c7c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.651+0000 7f2c82000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c116500 con 0x7f2c7c077530 2024-05-27T07:31:34.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.651+0000 7f2c82000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c116380 con 0x7f2c7c076be0 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.651+0000 7f2c82000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c116680 con 0x7f2c7c110bf0 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.652+0000 7f2c7aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c7c077530 0x7f2c7c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.652+0000 7f2c7b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c7c076be0 0x7f2c7c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.652+0000 7f2c7aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c7c077530 0x7f2c7c19d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44990/0 (socket says 172.21.15.63:44990) 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.652+0000 7f2c7aa00700 1 -- 172.21.15.63:0/1144055087 learned_addr learned my addr 172.21.15.63:0/1144055087 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.652+0000 7f2c7b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c7c076be0 0x7f2c7c19cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:33136/0 (socket says 172.21.15.63:33136) 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.652+0000 7f2c7be00700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c7c110bf0 0x7f2c7c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.652+0000 7f2c7aa00700 1 -- 172.21.15.63:0/1144055087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c7c110bf0 msgr2=0x7f2c7c1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.652+0000 7f2c7aa00700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c7c110bf0 0x7f2c7c1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.652+0000 7f2c7aa00700 1 -- 172.21.15.63:0/1144055087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c7c076be0 msgr2=0x7f2c7c19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.652+0000 7f2c7aa00700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c7c076be0 0x7f2c7c19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.653+0000 7f2c7aa00700 1 -- 172.21.15.63:0/1144055087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c1a1e00 con 0x7f2c7c077530 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.653+0000 7f2c7be00700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c7c110bf0 0x7f2c7c1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.653+0000 7f2c7aa00700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c7c077530 0x7f2c7c19d4d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000c00 tx=0x7f2c68005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.653+0000 7f2c73e00700 1 -- 172.21.15.63:0/1144055087 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68027070 con 0x7f2c7c077530 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.653+0000 7f2c73e00700 1 -- 172.21.15.63:0/1144055087 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c6800fc40 con 0x7f2c7c077530 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.653+0000 7f2c82000700 1 -- 172.21.15.63:0/1144055087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7c1a2090 con 0x7f2c7c077530 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.653+0000 7f2c73e00700 1 -- 172.21.15.63:0/1144055087 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68022c00 con 0x7f2c7c077530 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.654+0000 7f2c82000700 1 -- 172.21.15.63:0/1144055087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7c1a9a10 con 0x7f2c7c077530 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.655+0000 7f2c82000700 1 -- 172.21.15.63:0/1144055087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c0512b0 con 0x7f2c7c077530 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.658+0000 7f2c73e00700 1 -- 172.21.15.63:0/1144055087 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2c68022470 con 0x7f2c7c077530 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.660+0000 7f2c73e00700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c600b2990 0x7f2c600b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.660+0000 7f2c73e00700 1 -- 172.21.15.63:0/1144055087 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f2c680e39d0 con 0x7f2c7c077530 2024-05-27T07:31:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.661+0000 7f2c7b400700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c600b2990 0x7f2c600b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:34.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.661+0000 7f2c7b400700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c600b2990 0x7f2c600b4ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c005950 tx=0x7f2c6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:34.507 INFO:teuthology.orchestra.run.smithi063.stdout:188978561118 2024-05-27T07:31:34.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.663+0000 7f2c73e00700 1 -- 172.21.15.63:0/1144055087 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c68013070 con 0x7f2c7c077530 2024-05-27T07:31:34.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.946+0000 7f2c82000700 1 -- 172.21.15.63:0/1144055087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2c7c02e5e0 con 0x7f2c7c077530 2024-05-27T07:31:34.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.951+0000 7f2c73e00700 1 -- 172.21.15.63:0/1144055087 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2c680ab230 con 0x7f2c7c077530 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 -- 172.21.15.63:0/1144055087 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c600b2990 msgr2=0x7f2c600b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c600b2990 0x7f2c600b4ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c005950 tx=0x7f2c6c00b410 comp rx=0 tx=0).stop 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 -- 172.21.15.63:0/1144055087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c7c077530 msgr2=0x7f2c7c19d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c7c077530 0x7f2c7c19d4d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000c00 tx=0x7f2c68005ef0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 -- 172.21.15.63:0/1144055087 shutdown_connections 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c7c110bf0 0x7f2c7c1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c7c076be0 0x7f2c7c19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f2c600b2990 0x7f2c600b4ea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 --2- 172.21.15.63:0/1144055087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c7c077530 0x7f2c7c19d4d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 -- 172.21.15.63:0/1144055087 >> 172.21.15.63:0/1144055087 conn(0x7f2c7c0ffe90 msgr2=0x7f2c7c100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 -- 172.21.15.63:0/1144055087 shutdown_connections 2024-05-27T07:31:34.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:33.954+0000 7f2c82000700 1 -- 172.21.15.63:0/1144055087 wait complete. 2024-05-27T07:31:35.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:34 smithi063 ceph-mon[31613]: pgmap v486: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:35.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:34 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1144055087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T07:31:35.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:34 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4075263842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T07:31:35.239 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149807 got 111669149812 for osd.3 2024-05-27T07:31:35.239 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:34 smithi086 ceph-mon[34687]: pgmap v486: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:34 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1144055087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T07:31:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:34 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4075263842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T07:31:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:34 smithi149 ceph-mon[35408]: pgmap v486: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:34 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1144055087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T07:31:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:34 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4075263842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T07:31:35.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.888+0000 7fad29200700 1 -- 172.21.15.63:0/438581572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad2410c5b0 msgr2=0x7fad2410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:35.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.888+0000 7fad29200700 1 --2- 172.21.15.63:0/438581572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad2410c5b0 0x7fad2410c9c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad14009990 tx=0x7fad14009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:35.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.888+0000 7fad29200700 1 -- 172.21.15.63:0/438581572 shutdown_connections 2024-05-27T07:31:35.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.888+0000 7fad29200700 1 --2- 172.21.15.63:0/438581572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad24106ec0 0x7fad24113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:35.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.888+0000 7fad29200700 1 --2- 172.21.15.63:0/438581572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad2410c5b0 0x7fad2410c9c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:35.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.888+0000 7fad29200700 1 --2- 172.21.15.63:0/438581572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad24106570 0x7fad24106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:35.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.888+0000 7fad29200700 1 -- 172.21.15.63:0/438581572 >> 172.21.15.63:0/438581572 conn(0x7fad24102070 msgr2=0x7fad241044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:35.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.888+0000 7fad29200700 1 -- 172.21.15.63:0/438581572 shutdown_connections 2024-05-27T07:31:35.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.888+0000 7fad29200700 1 -- 172.21.15.63:0/438581572 wait complete. 2024-05-27T07:31:35.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.890+0000 7fad29200700 1 Processor -- start 2024-05-27T07:31:35.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.890+0000 7fad29200700 1 -- start start 2024-05-27T07:31:35.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.890+0000 7fad29200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad24106570 0x7fad2419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:35.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.890+0000 7fad29200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad24106ec0 0x7fad2419d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.890+0000 7fad29200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad2410c5b0 0x7fad241a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad29200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad241165c0 con 0x7fad24106570 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad29200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad24116440 con 0x7fad24106ec0 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad29200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fad24116740 con 0x7fad2410c5b0 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad21e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad24106570 0x7fad2419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad21e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad24106570 0x7fad2419cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45024/0 (socket says 172.21.15.63:45024) 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad21e00700 1 -- 172.21.15.63:0/947877067 learned_addr learned my addr 172.21.15.63:0/947877067 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad22800700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad2410c5b0 0x7fad241a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad21400700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad24106ec0 0x7fad2419d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad21e00700 1 -- 172.21.15.63:0/947877067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad2410c5b0 msgr2=0x7fad241a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad21e00700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad2410c5b0 0x7fad241a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:35.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad21e00700 1 -- 172.21.15.63:0/947877067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad24106ec0 msgr2=0x7fad2419d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:35.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad21e00700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad24106ec0 0x7fad2419d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:35.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.891+0000 7fad21e00700 1 -- 172.21.15.63:0/947877067 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad241a1e60 con 0x7fad24106570 2024-05-27T07:31:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.892+0000 7fad21e00700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad24106570 0x7fad2419cf90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad1400b5c0 tx=0x7fad140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.892+0000 7fad1aa00700 1 -- 172.21.15.63:0/947877067 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14004830 con 0x7fad24106570 2024-05-27T07:31:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.892+0000 7fad1aa00700 1 -- 172.21.15.63:0/947877067 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad14005020 con 0x7fad24106570 2024-05-27T07:31:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.892+0000 7fad1aa00700 1 -- 172.21.15.63:0/947877067 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1400f950 con 0x7fad24106570 2024-05-27T07:31:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.892+0000 7fad29200700 1 -- 172.21.15.63:0/947877067 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad241a20f0 con 0x7fad24106570 2024-05-27T07:31:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.892+0000 7fad29200700 1 -- 172.21.15.63:0/947877067 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad241a2520 con 0x7fad24106570 2024-05-27T07:31:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.894+0000 7fad29200700 1 -- 172.21.15.63:0/947877067 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad240512b0 con 0x7fad24106570 2024-05-27T07:31:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.896+0000 7fad1aa00700 1 -- 172.21.15.63:0/947877067 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fad140049d0 con 0x7fad24106570 2024-05-27T07:31:35.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.898+0000 7fad1aa00700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fad080b2900 0x7fad080b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:35.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.898+0000 7fad1aa00700 1 -- 172.21.15.63:0/947877067 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fad140e3820 con 0x7fad24106570 2024-05-27T07:31:35.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.898+0000 7fad21400700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fad080b2900 0x7fad080b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:35.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.899+0000 7fad21400700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fad080b2900 0x7fad080b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fad10005950 tx=0x7fad100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:35.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:35.901+0000 7fad1aa00700 1 -- 172.21.15.63:0/947877067 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad1402f020 con 0x7fad24106570 2024-05-27T07:31:36.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.143+0000 7fad29200700 1 -- 172.21.15.63:0/947877067 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fad2402e5e0 con 0x7fad24106570 2024-05-27T07:31:36.136 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:35 smithi063 ceph-mon[31613]: mgrmap e18: x(active, since 13m), standbys: z, y 2024-05-27T07:31:36.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.144+0000 7fad1aa00700 1 -- 172.21.15.63:0/947877067 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fad14013070 con 0x7fad24106570 2024-05-27T07:31:36.137 INFO:teuthology.orchestra.run.smithi063.stdout:163208757351 2024-05-27T07:31:36.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 -- 172.21.15.63:0/947877067 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fad080b2900 msgr2=0x7fad080b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:36.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fad080b2900 0x7fad080b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fad10005950 tx=0x7fad100058e0 comp rx=0 tx=0).stop 2024-05-27T07:31:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 -- 172.21.15.63:0/947877067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad24106570 msgr2=0x7fad2419cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad24106570 0x7fad2419cf90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad1400b5c0 tx=0x7fad140047b0 comp rx=0 tx=0).stop 2024-05-27T07:31:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 -- 172.21.15.63:0/947877067 shutdown_connections 2024-05-27T07:31:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad2410c5b0 0x7fad241a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fad080b2900 0x7fad080b4e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad24106570 0x7fad2419cf90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 --2- 172.21.15.63:0/947877067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad24106ec0 0x7fad2419d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 -- 172.21.15.63:0/947877067 >> 172.21.15.63:0/947877067 conn(0x7fad24102070 msgr2=0x7fad24104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.147+0000 7fad29200700 1 -- 172.21.15.63:0/947877067 shutdown_connections 2024-05-27T07:31:36.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:36.148+0000 7fad29200700 1 -- 172.21.15.63:0/947877067 wait complete. 2024-05-27T07:31:36.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:35 smithi086 ceph-mon[34687]: mgrmap e18: x(active, since 13m), standbys: z, y 2024-05-27T07:31:36.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:35 smithi149 ceph-mon[35408]: mgrmap e18: x(active, since 13m), standbys: z, y 2024-05-27T07:31:36.526 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:36.527 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:37.033 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:36 smithi063 ceph-mon[31613]: pgmap v487: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:31:37.033 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:36 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/947877067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T07:31:37.033 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:31:37.033 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:31:37.034 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:36 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:31:37.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:36 smithi086 ceph-mon[34687]: pgmap v487: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:31:37.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:36 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/947877067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T07:31:37.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:31:37.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:31:37.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:36 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:31:37.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:36 smithi149 ceph-mon[35408]: pgmap v487: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:31:37.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:36 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/947877067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T07:31:37.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:31:37.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:31:37.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:36 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:31:38.621 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561112 got 188978561118 for osd.6 2024-05-27T07:31:38.622 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:39.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:39 smithi063 ceph-mon[31613]: pgmap v488: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:31:39.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:39 smithi086 ceph-mon[34687]: pgmap v488: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:31:39.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:39 smithi149 ceph-mon[35408]: pgmap v488: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:31:40.472 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757345 got 163208757351 for osd.5 2024-05-27T07:31:40.472 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:41.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:41 smithi063 ceph-mon[31613]: pgmap v489: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:31:41.219 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953576 got 137438953580 for osd.4 2024-05-27T07:31:41.220 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:41.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:41 smithi086 ceph-mon[34687]: pgmap v489: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:31:41.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:41 smithi149 ceph-mon[35408]: pgmap v489: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:31:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.760+0000 7f17e8600700 1 -- 172.21.15.63:0/854578616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17e0105e70 msgr2=0x7f17e010e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.760+0000 7f17e8600700 1 --2- 172.21.15.63:0/854578616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17e0105e70 0x7f17e010e3c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009990 tx=0x7f17cc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.760+0000 7f17e8600700 1 -- 172.21.15.63:0/854578616 shutdown_connections 2024-05-27T07:31:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.760+0000 7f17e8600700 1 --2- 172.21.15.63:0/854578616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e010e950 0x7f17e0110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.760+0000 7f17e8600700 1 --2- 172.21.15.63:0/854578616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0105520 0x7f17e0105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.760+0000 7f17e8600700 1 --2- 172.21.15.63:0/854578616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17e0105e70 0x7f17e010e3c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.760+0000 7f17e8600700 1 -- 172.21.15.63:0/854578616 >> 172.21.15.63:0/854578616 conn(0x7f17e0078df0 msgr2=0x7f17e0079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.761+0000 7f17e8600700 1 -- 172.21.15.63:0/854578616 shutdown_connections 2024-05-27T07:31:41.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.761+0000 7f17e8600700 1 -- 172.21.15.63:0/854578616 wait complete. 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.762+0000 7f17e8600700 1 Processor -- start 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.763+0000 7f17e8600700 1 -- start start 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.763+0000 7f17e8600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0105520 0x7f17e019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.763+0000 7f17e8600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17e0105e70 0x7f17e019d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.763+0000 7f17e8600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e010e950 0x7f17e01a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.763+0000 7f17e8600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0114340 con 0x7f17e0105e70 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.763+0000 7f17e8600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01141c0 con 0x7f17e010e950 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.763+0000 7f17e8600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01144c0 con 0x7f17e0105520 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.763+0000 7f17e5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17e0105e70 0x7f17e019d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.763+0000 7f17e5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17e0105e70 0x7f17e019d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42628/0 (socket says 172.21.15.63:42628) 2024-05-27T07:31:41.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.763+0000 7f17e5200700 1 -- 172.21.15.63:0/4187665373 learned_addr learned my addr 172.21.15.63:0/4187665373 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.764+0000 7f17e5c00700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0105520 0x7f17e019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.764+0000 7f17e6600700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e010e950 0x7f17e01a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.764+0000 7f17e5200700 1 -- 172.21.15.63:0/4187665373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0105520 msgr2=0x7f17e019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.764+0000 7f17e5200700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0105520 0x7f17e019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.764+0000 7f17e5200700 1 -- 172.21.15.63:0/4187665373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e010e950 msgr2=0x7f17e01a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.764+0000 7f17e5200700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e010e950 0x7f17e01a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.764+0000 7f17e5200700 1 -- 172.21.15.63:0/4187665373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e01a1e20 con 0x7f17e0105e70 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.764+0000 7f17e5200700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17e0105e70 0x7f17e019d4f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f17cc000c00 tx=0x7f17cc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.765+0000 7f17de000700 1 -- 172.21.15.63:0/4187665373 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc027070 con 0x7f17e0105e70 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.765+0000 7f17de000700 1 -- 172.21.15.63:0/4187665373 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17cc00fc70 con 0x7f17e0105e70 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.765+0000 7f17e8600700 1 -- 172.21.15.63:0/4187665373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e01a20b0 con 0x7f17e0105e70 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.765+0000 7f17de000700 1 -- 172.21.15.63:0/4187665373 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc022d40 con 0x7f17e0105e70 2024-05-27T07:31:41.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.765+0000 7f17e8600700 1 -- 172.21.15.63:0/4187665373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e01a9a30 con 0x7f17e0105e70 2024-05-27T07:31:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.766+0000 7f17e8600700 1 -- 172.21.15.63:0/4187665373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e00512b0 con 0x7f17e0105e70 2024-05-27T07:31:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.770+0000 7f17de000700 1 -- 172.21.15.63:0/4187665373 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f17cc02a070 con 0x7f17e0105e70 2024-05-27T07:31:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.772+0000 7f17de000700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f17c80b2900 0x7f17c80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.772+0000 7f17de000700 1 -- 172.21.15.63:0/4187665373 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f17cc0e39b0 con 0x7f17e0105e70 2024-05-27T07:31:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.773+0000 7f17e5c00700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f17c80b2900 0x7f17c80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.773+0000 7f17e5c00700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f17c80b2900 0x7f17c80b4e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f17d4005950 tx=0x7f17d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:41.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:41.777+0000 7f17de000700 1 -- 172.21.15.63:0/4187665373 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f17cc031020 con 0x7f17e0105e70 2024-05-27T07:31:42.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.024+0000 7f17e8600700 1 -- 172.21.15.63:0/4187665373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f17e01a9c00 con 0x7f17e0105e70 2024-05-27T07:31:42.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.025+0000 7f17de000700 1 -- 172.21.15.63:0/4187665373 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f17cc0b1020 con 0x7f17e0105e70 2024-05-27T07:31:42.017 INFO:teuthology.orchestra.run.smithi063.stdout:210453397593 2024-05-27T07:31:42.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.028+0000 7f17e8600700 1 -- 172.21.15.63:0/4187665373 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f17c80b2900 msgr2=0x7f17c80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:42.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.028+0000 7f17e8600700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f17c80b2900 0x7f17c80b4e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f17d4005950 tx=0x7f17d40058e0 comp rx=0 tx=0).stop 2024-05-27T07:31:42.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.028+0000 7f17e8600700 1 -- 172.21.15.63:0/4187665373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17e0105e70 msgr2=0x7f17e019d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:42.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.028+0000 7f17e8600700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17e0105e70 0x7f17e019d4f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f17cc000c00 tx=0x7f17cc009f90 comp rx=0 tx=0).stop 2024-05-27T07:31:42.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.028+0000 7f17e8600700 1 -- 172.21.15.63:0/4187665373 shutdown_connections 2024-05-27T07:31:42.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.028+0000 7f17e8600700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17e010e950 0x7f17e01a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:42.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.028+0000 7f17e8600700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17e0105520 0x7f17e019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:42.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.028+0000 7f17e8600700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f17c80b2900 0x7f17c80b4e10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:42.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.028+0000 7f17e8600700 1 --2- 172.21.15.63:0/4187665373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17e0105e70 0x7f17e019d4f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:42.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.029+0000 7f17e8600700 1 -- 172.21.15.63:0/4187665373 >> 172.21.15.63:0/4187665373 conn(0x7f17e0078df0 msgr2=0x7f17e0103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:42.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.029+0000 7f17e8600700 1 -- 172.21.15.63:0/4187665373 shutdown_connections 2024-05-27T07:31:42.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:42.029+0000 7f17e8600700 1 -- 172.21.15.63:0/4187665373 wait complete. 2024-05-27T07:31:43.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:43 smithi063 ceph-mon[31613]: pgmap v490: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:31:43.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:43 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4187665373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T07:31:43.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:43 smithi086 ceph-mon[34687]: pgmap v490: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:31:43.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:43 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4187665373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T07:31:43.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:43 smithi149 ceph-mon[35408]: pgmap v490: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:31:43.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:43 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4187665373' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T07:31:44.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:44 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1376110209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T07:31:44.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:44 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1376110209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T07:31:44.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:44 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1376110209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T07:31:44.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.656+0000 7f4502200700 1 -- 172.21.15.63:0/357218257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44fc1027d0 msgr2=0x7f44fc102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.656+0000 7f4502200700 1 --2- 172.21.15.63:0/357218257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44fc1027d0 0x7f44fc102be0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009990 tx=0x7f44e8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.657+0000 7f4502200700 1 -- 172.21.15.63:0/357218257 shutdown_connections 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.657+0000 7f4502200700 1 --2- 172.21.15.63:0/357218257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc103120 0x7f44fc111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.657+0000 7f4502200700 1 --2- 172.21.15.63:0/357218257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44fc10a3d0 0x7f44fc10a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.657+0000 7f4502200700 1 --2- 172.21.15.63:0/357218257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44fc1027d0 0x7f44fc102be0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.657+0000 7f4502200700 1 -- 172.21.15.63:0/357218257 >> 172.21.15.63:0/357218257 conn(0x7f44fc0fe6e0 msgr2=0x7f44fc100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.657+0000 7f4502200700 1 -- 172.21.15.63:0/357218257 shutdown_connections 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.657+0000 7f4502200700 1 -- 172.21.15.63:0/357218257 wait complete. 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.658+0000 7f4502200700 1 Processor -- start 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.658+0000 7f4502200700 1 -- start start 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.658+0000 7f4502200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44fc1027d0 0x7f44fc19d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:44.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f4502200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc103120 0x7f44fc19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f4502200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44fc10a3d0 0x7f44fc1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f4502200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc114460 con 0x7f44fc1027d0 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f4502200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc1142e0 con 0x7f44fc103120 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f4502200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc1145e0 con 0x7f44fc10a3d0 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f44fb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44fc1027d0 0x7f44fc19d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f44fb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44fc1027d0 0x7f44fc19d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42646/0 (socket says 172.21.15.63:42646) 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f44fb400700 1 -- 172.21.15.63:0/1376110209 learned_addr learned my addr 172.21.15.63:0/1376110209 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f44faa00700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc103120 0x7f44fc19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f44fb400700 1 -- 172.21.15.63:0/1376110209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44fc10a3d0 msgr2=0x7f44fc1a1960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f44fb400700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44fc10a3d0 0x7f44fc1a1960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f44fb400700 1 -- 172.21.15.63:0/1376110209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc103120 msgr2=0x7f44fc19d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f44fb400700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc103120 0x7f44fc19d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.659+0000 7f44fb400700 1 -- 172.21.15.63:0/1376110209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc1a2000 con 0x7f44fc1027d0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.660+0000 7f44fb400700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44fc1027d0 0x7f44fc19d130 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00b550 tx=0x7f44ec00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.660+0000 7f44f3e00700 1 -- 172.21.15.63:0/1376110209 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec015020 con 0x7f44fc1027d0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.660+0000 7f44f3e00700 1 -- 172.21.15.63:0/1376110209 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44ec011cc0 con 0x7f44fc1027d0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.660+0000 7f4502200700 1 -- 172.21.15.63:0/1376110209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc1a22f0 con 0x7f44fc1027d0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.660+0000 7f44f3e00700 1 -- 172.21.15.63:0/1376110209 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec010ca0 con 0x7f44fc1027d0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.660+0000 7f4502200700 1 -- 172.21.15.63:0/1376110209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc1a9c10 con 0x7f44fc1027d0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.661+0000 7f4502200700 1 -- 172.21.15.63:0/1376110209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc0512b0 con 0x7f44fc1027d0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.664+0000 7f44f3e00700 1 -- 172.21.15.63:0/1376110209 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f44ec018020 con 0x7f44fc1027d0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.666+0000 7f44f3e00700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f44e00b2880 0x7f44e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.667+0000 7f44f3e00700 1 -- 172.21.15.63:0/1376110209 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f44ec0e1e40 con 0x7f44fc1027d0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.667+0000 7f44faa00700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f44e00b2880 0x7f44e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:44.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.668+0000 7f44faa00700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f44e00b2880 0x7f44e00b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f44fc1019b0 tx=0x7f44e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:44.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.671+0000 7f44f3e00700 1 -- 172.21.15.63:0/1376110209 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44ec01f020 con 0x7f44fc1027d0 2024-05-27T07:31:44.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.923+0000 7f4502200700 1 -- 172.21.15.63:0/1376110209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f44fc1a9de0 con 0x7f44fc1027d0 2024-05-27T07:31:44.962 INFO:teuthology.orchestra.run.smithi063.stdout:77309411454 2024-05-27T07:31:44.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.924+0000 7f44f3e00700 1 -- 172.21.15.63:0/1376110209 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44ec0a9770 con 0x7f44fc1027d0 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 -- 172.21.15.63:0/1376110209 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f44e00b2880 msgr2=0x7f44e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f44e00b2880 0x7f44e00b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f44fc1019b0 tx=0x7f44e801a040 comp rx=0 tx=0).stop 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 -- 172.21.15.63:0/1376110209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44fc1027d0 msgr2=0x7f44fc19d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44fc1027d0 0x7f44fc19d130 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00b550 tx=0x7f44ec00b910 comp rx=0 tx=0).stop 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 -- 172.21.15.63:0/1376110209 shutdown_connections 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44fc10a3d0 0x7f44fc1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f44e00b2880 0x7f44e00b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44fc1027d0 0x7f44fc19d130 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 --2- 172.21.15.63:0/1376110209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44fc103120 0x7f44fc19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 -- 172.21.15.63:0/1376110209 >> 172.21.15.63:0/1376110209 conn(0x7f44fc0fe6e0 msgr2=0x7f44fc10f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 -- 172.21.15.63:0/1376110209 shutdown_connections 2024-05-27T07:31:44.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:43.927+0000 7f4502200700 1 -- 172.21.15.63:0/1376110209 wait complete. 2024-05-27T07:31:45.141 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:45 smithi063 ceph-mon[31613]: pgmap v491: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:31:45.142 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:45 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1285676849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T07:31:45.142 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:45 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3267423884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T07:31:45.303 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574972 got 55834574979 for osd.1 2024-05-27T07:31:45.303 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.972+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2400244911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e181034b0 msgr2=0x7f8e181038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.972+0000 7f8e1ca00700 1 --2- 172.21.15.63:0/2400244911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e181034b0 0x7f8e181038c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8e080099c0 tx=0x7f8e08009cd0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.973+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2400244911 shutdown_connections 2024-05-27T07:31:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.973+0000 7f8e1ca00700 1 --2- 172.21.15.63:0/2400244911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e18108730 0x7f8e18110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.973+0000 7f8e1ca00700 1 --2- 172.21.15.63:0/2400244911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e181034b0 0x7f8e181038c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.973+0000 7f8e1ca00700 1 --2- 172.21.15.63:0/2400244911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e18107de0 0x7f8e181081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.973+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2400244911 >> 172.21.15.63:0/2400244911 conn(0x7f8e180fedb0 msgr2=0x7f8e181011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:45.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.973+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2400244911 shutdown_connections 2024-05-27T07:31:45.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.973+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2400244911 wait complete. 2024-05-27T07:31:45.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.974+0000 7f8e1ca00700 1 Processor -- start 2024-05-27T07:31:45.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.974+0000 7f8e1ca00700 1 -- start start 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.974+0000 7f8e1ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e181034b0 0x7f8e180ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.974+0000 7f8e1ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e18107de0 0x7f8e180ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.975+0000 7f8e1ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e18108730 0x7f8e180ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.975+0000 7f8e1ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18113210 con 0x7f8e181034b0 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.975+0000 7f8e1ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18113090 con 0x7f8e18107de0 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.975+0000 7f8e1ca00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18113390 con 0x7f8e18108730 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.975+0000 7f8e15e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e181034b0 0x7f8e180ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.975+0000 7f8e15e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e181034b0 0x7f8e180ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42714/0 (socket says 172.21.15.63:42714) 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.975+0000 7f8e15e00700 1 -- 172.21.15.63:0/2220297517 learned_addr learned my addr 172.21.15.63:0/2220297517 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.975+0000 7f8e15400700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e18107de0 0x7f8e180ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.975+0000 7f8e16800700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e18108730 0x7f8e180ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e15e00700 1 -- 172.21.15.63:0/2220297517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e18108730 msgr2=0x7f8e180ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e15e00700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e18108730 0x7f8e180ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e15e00700 1 -- 172.21.15.63:0/2220297517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e18107de0 msgr2=0x7f8e180ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e15e00700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e18107de0 0x7f8e180ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e15e00700 1 -- 172.21.15.63:0/2220297517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e181a90a0 con 0x7f8e181034b0 2024-05-27T07:31:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e15400700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e18107de0 0x7f8e180ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:31:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e15e00700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e181034b0 0x7f8e180ff290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e08005f50 tx=0x7f8e080057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e0ea00700 1 -- 172.21.15.63:0/2220297517 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08027070 con 0x7f8e181034b0 2024-05-27T07:31:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e0ea00700 1 -- 172.21.15.63:0/2220297517 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e0800fd60 con 0x7f8e181034b0 2024-05-27T07:31:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2220297517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e181a9330 con 0x7f8e181034b0 2024-05-27T07:31:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.976+0000 7f8e0ea00700 1 -- 172.21.15.63:0/2220297517 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08022e30 con 0x7f8e181034b0 2024-05-27T07:31:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.977+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2220297517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e181a9860 con 0x7f8e181034b0 2024-05-27T07:31:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.978+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2220297517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e180512b0 con 0x7f8e181034b0 2024-05-27T07:31:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.981+0000 7f8e0ea00700 1 -- 172.21.15.63:0/2220297517 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8e0802a070 con 0x7f8e181034b0 2024-05-27T07:31:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.983+0000 7f8e0ea00700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8dfc0b2900 0x7f8dfc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.984+0000 7f8e0ea00700 1 -- 172.21.15.63:0/2220297517 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f8e080e3360 con 0x7f8e181034b0 2024-05-27T07:31:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.985+0000 7f8e15400700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8dfc0b2900 0x7f8dfc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.986+0000 7f8e15400700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8dfc0b2900 0x7f8dfc0b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e04005950 tx=0x7f8e040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.989+0000 7f8e0ea00700 1 -- 172.21.15.63:0/2220297517 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e08013070 con 0x7f8e181034b0 2024-05-27T07:31:45.392 INFO:teuthology.orchestra.run.smithi063.stdout:34359738508 2024-05-27T07:31:45.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.259+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2220297517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8e181a9bd0 con 0x7f8e181034b0 2024-05-27T07:31:45.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.259+0000 7f8e0ea00700 1 -- 172.21.15.63:0/2220297517 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8e080aad20 con 0x7f8e181034b0 2024-05-27T07:31:45.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.262+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2220297517 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8dfc0b2900 msgr2=0x7f8dfc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:45.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.262+0000 7f8e1ca00700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8dfc0b2900 0x7f8dfc0b4e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e04005950 tx=0x7f8e040058e0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.262+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2220297517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e181034b0 msgr2=0x7f8e180ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:45.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.262+0000 7f8e1ca00700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e181034b0 0x7f8e180ff290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e08005f50 tx=0x7f8e080057c0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.262+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2220297517 shutdown_connections 2024-05-27T07:31:45.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.262+0000 7f8e1ca00700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e18108730 0x7f8e180ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.262+0000 7f8e1ca00700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f8dfc0b2900 0x7f8dfc0b4e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.262+0000 7f8e1ca00700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e181034b0 0x7f8e180ff290 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.262+0000 7f8e1ca00700 1 --2- 172.21.15.63:0/2220297517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e18107de0 0x7f8e180ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:45.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.263+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2220297517 >> 172.21.15.63:0/2220297517 conn(0x7f8e180fedb0 msgr2=0x7f8e1806b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:45.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.263+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2220297517 shutdown_connections 2024-05-27T07:31:45.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.263+0000 7f8e1ca00700 1 -- 172.21.15.63:0/2220297517 wait complete. 2024-05-27T07:31:45.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:45 smithi086 ceph-mon[34687]: pgmap v491: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:31:45.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:45 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1285676849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T07:31:45.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:45 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3267423884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T07:31:45.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:45 smithi149 ceph-mon[35408]: pgmap v491: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:31:45.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:45 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1285676849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T07:31:45.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:45 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3267423884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T07:31:45.989 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397587 got 210453397593 for osd.7 2024-05-27T07:31:45.990 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:46.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.869+0000 7ffb02a00700 1 -- 172.21.15.63:0/1954520165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffafc075b50 msgr2=0x7ffafc075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:46.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.869+0000 7ffb02a00700 1 --2- 172.21.15.63:0/1954520165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffafc075b50 0x7ffafc075f60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009990 tx=0x7ffaec009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.869+0000 7ffb02a00700 1 -- 172.21.15.63:0/1954520165 shutdown_connections 2024-05-27T07:31:46.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.869+0000 7ffb02a00700 1 --2- 172.21.15.63:0/1954520165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc115360 0x7ffafc117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.869+0000 7ffb02a00700 1 --2- 172.21.15.63:0/1954520165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffafc075b50 0x7ffafc075f60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.869+0000 7ffb02a00700 1 --2- 172.21.15.63:0/1954520165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffafc0764a0 0x7ffafc114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.869+0000 7ffb02a00700 1 -- 172.21.15.63:0/1954520165 >> 172.21.15.63:0/1954520165 conn(0x7ffafc100030 msgr2=0x7ffafc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.869+0000 7ffb02a00700 1 -- 172.21.15.63:0/1954520165 shutdown_connections 2024-05-27T07:31:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.869+0000 7ffb02a00700 1 -- 172.21.15.63:0/1954520165 wait complete. 2024-05-27T07:31:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.870+0000 7ffb02a00700 1 Processor -- start 2024-05-27T07:31:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.871+0000 7ffb02a00700 1 -- start start 2024-05-27T07:31:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.871+0000 7ffb02a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc075b50 0x7ffafc1a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.871+0000 7ffb02a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffafc0764a0 0x7ffafc1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.871+0000 7ffb02a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffafc115360 0x7ffafc1a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.871+0000 7ffb02a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc11a960 con 0x7ffafc115360 2024-05-27T07:31:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.871+0000 7ffb02a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc11a7e0 con 0x7ffafc075b50 2024-05-27T07:31:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.871+0000 7ffb02a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc11aae0 con 0x7ffafc0764a0 2024-05-27T07:31:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.872+0000 7ffb00a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffafc115360 0x7ffafc1a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:46.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.872+0000 7ffafbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc075b50 0x7ffafc1a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:46.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.872+0000 7ffb00a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffafc115360 0x7ffafc1a5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42702/0 (socket says 172.21.15.63:42702) 2024-05-27T07:31:46.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.872+0000 7ffb00a00700 1 -- 172.21.15.63:0/3173728662 learned_addr learned my addr 172.21.15.63:0/3173728662 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:46.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.873+0000 7ffb00a00700 1 -- 172.21.15.63:0/3173728662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffafc0764a0 msgr2=0x7ffafc1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:46.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.873+0000 7ffb00a00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffafc0764a0 0x7ffafc1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.873+0000 7ffb00a00700 1 -- 172.21.15.63:0/3173728662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc075b50 msgr2=0x7ffafc1a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.873+0000 7ffb00a00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc075b50 0x7ffafc1a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.873+0000 7ffb00a00700 1 -- 172.21.15.63:0/3173728662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc1a6260 con 0x7ffafc115360 2024-05-27T07:31:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.873+0000 7ffb00a00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffafc115360 0x7ffafc1a5bc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4007ee0 tx=0x7ffaf400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.874+0000 7ffaf8c00700 1 -- 172.21.15.63:0/3173728662 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400dbd0 con 0x7ffafc115360 2024-05-27T07:31:46.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.874+0000 7ffaf8c00700 1 -- 172.21.15.63:0/3173728662 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffaf400dd70 con 0x7ffafc115360 2024-05-27T07:31:46.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.874+0000 7ffaf8c00700 1 -- 172.21.15.63:0/3173728662 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40058d0 con 0x7ffafc115360 2024-05-27T07:31:46.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.874+0000 7ffb02a00700 1 -- 172.21.15.63:0/3173728662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffafc1a6550 con 0x7ffafc115360 2024-05-27T07:31:46.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.874+0000 7ffb02a00700 1 -- 172.21.15.63:0/3173728662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffafc1ade70 con 0x7ffafc115360 2024-05-27T07:31:46.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.876+0000 7ffb02a00700 1 -- 172.21.15.63:0/3173728662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffafc077290 con 0x7ffafc115360 2024-05-27T07:31:46.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.880+0000 7ffaf8c00700 1 -- 172.21.15.63:0/3173728662 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffaf401a020 con 0x7ffafc115360 2024-05-27T07:31:46.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.882+0000 7ffaf8c00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffae00b2880 0x7ffae00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:46.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.882+0000 7ffaf8c00700 1 -- 172.21.15.63:0/3173728662 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7ffaf40e2330 con 0x7ffafc115360 2024-05-27T07:31:46.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.882+0000 7ffafbe00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffae00b2880 0x7ffae00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:46.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.883+0000 7ffafbe00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffae00b2880 0x7ffae00b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00b5c0 tx=0x7ffaec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:46.055 INFO:teuthology.orchestra.run.smithi063.stdout:283467841601 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.884+0000 7ffaf8c00700 1 -- 172.21.15.63:0/3173728662 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffaf4010070 con 0x7ffafc115360 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.149+0000 7ffb02a00700 1 -- 172.21.15.63:0/3173728662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ffafc02e5e0 con 0x7ffafc115360 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.150+0000 7ffaf8c00700 1 -- 172.21.15.63:0/3173728662 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ffaf40a9d60 con 0x7ffafc115360 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 -- 172.21.15.63:0/3173728662 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffae00b2880 msgr2=0x7ffae00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffae00b2880 0x7ffae00b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00b5c0 tx=0x7ffaec005fd0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 -- 172.21.15.63:0/3173728662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffafc115360 msgr2=0x7ffafc1a5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffafc115360 0x7ffafc1a5bc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4007ee0 tx=0x7ffaf400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 -- 172.21.15.63:0/3173728662 shutdown_connections 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7ffae00b2880 0x7ffae00b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffafc115360 0x7ffafc1a5bc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffafc075b50 0x7ffafc1a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 --2- 172.21.15.63:0/3173728662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffafc0764a0 0x7ffafc1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 -- 172.21.15.63:0/3173728662 >> 172.21.15.63:0/3173728662 conn(0x7ffafc100030 msgr2=0x7ffafc101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:46.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 -- 172.21.15.63:0/3173728662 shutdown_connections 2024-05-27T07:31:46.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:45.153+0000 7ffb02a00700 1 -- 172.21.15.63:0/3173728662 wait complete. 2024-05-27T07:31:46.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:46 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3173728662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T07:31:46.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:46 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2220297517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T07:31:46.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:46 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3173728662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T07:31:46.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:46 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2220297517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T07:31:46.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:46 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3173728662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T07:31:46.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:46 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2220297517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T07:31:46.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.377+0000 7f64b7200700 1 -- 172.21.15.63:0/2051707282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b0075ad0 msgr2=0x7f64b0075ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.377+0000 7f64b7200700 1 --2- 172.21.15.63:0/2051707282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b0075ad0 0x7f64b0075ee0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009990 tx=0x7f64a0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.377+0000 7f64b7200700 1 -- 172.21.15.63:0/2051707282 shutdown_connections 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.377+0000 7f64b7200700 1 --2- 172.21.15.63:0/2051707282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b0110c10 0x7f64b0113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.377+0000 7f64b7200700 1 --2- 172.21.15.63:0/2051707282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b0075ad0 0x7f64b0075ee0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.377+0000 7f64b7200700 1 --2- 172.21.15.63:0/2051707282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64b0076420 0x7f64b01106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.377+0000 7f64b7200700 1 -- 172.21.15.63:0/2051707282 >> 172.21.15.63:0/2051707282 conn(0x7f64b0100030 msgr2=0x7f64b0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.377+0000 7f64b7200700 1 -- 172.21.15.63:0/2051707282 shutdown_connections 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.377+0000 7f64b7200700 1 -- 172.21.15.63:0/2051707282 wait complete. 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.378+0000 7f64b7200700 1 Processor -- start 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.379+0000 7f64b7200700 1 -- start start 2024-05-27T07:31:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.379+0000 7f64b7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b0075ad0 0x7f64b019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.379+0000 7f64b7200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64b0076420 0x7f64b019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.379+0000 7f64b7200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b0110c10 0x7f64b01a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.379+0000 7f64b7200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64b0116590 con 0x7f64b0110c10 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.379+0000 7f64b7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64b0116410 con 0x7f64b0075ad0 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.379+0000 7f64b7200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f64b0116710 con 0x7f64b0076420 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.379+0000 7f64b5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b0110c10 0x7f64b01a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.380+0000 7f64b5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b0110c10 0x7f64b01a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42666/0 (socket says 172.21.15.63:42666) 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.380+0000 7f64b5200700 1 -- 172.21.15.63:0/1285676849 learned_addr learned my addr 172.21.15.63:0/1285676849 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.380+0000 7f64afe00700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b0075ad0 0x7f64b019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.380+0000 7f64af400700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64b0076420 0x7f64b019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.380+0000 7f64b5200700 1 -- 172.21.15.63:0/1285676849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64b0076420 msgr2=0x7f64b019d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:46.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.380+0000 7f64b5200700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64b0076420 0x7f64b019d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.380+0000 7f64b5200700 1 -- 172.21.15.63:0/1285676849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b0075ad0 msgr2=0x7f64b019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.380+0000 7f64b5200700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b0075ad0 0x7f64b019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.380+0000 7f64b5200700 1 -- 172.21.15.63:0/1285676849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b01a1e80 con 0x7f64b0110c10 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.380+0000 7f64b5200700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b0110c10 0x7f64b01a17e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f64a8007ee0 tx=0x7f64a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.381+0000 7f64acc00700 1 -- 172.21.15.63:0/1285676849 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a800dbd0 con 0x7f64b0110c10 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.381+0000 7f64acc00700 1 -- 172.21.15.63:0/1285676849 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64a800dd70 con 0x7f64b0110c10 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.381+0000 7f64b7200700 1 -- 172.21.15.63:0/1285676849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b01a2170 con 0x7f64b0110c10 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.381+0000 7f64acc00700 1 -- 172.21.15.63:0/1285676849 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a80058d0 con 0x7f64b0110c10 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.381+0000 7f64b7200700 1 -- 172.21.15.63:0/1285676849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b01a9a90 con 0x7f64b0110c10 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.382+0000 7f64b7200700 1 -- 172.21.15.63:0/1285676849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b00512b0 con 0x7f64b0110c10 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.385+0000 7f64acc00700 1 -- 172.21.15.63:0/1285676849 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f64a801a020 con 0x7f64b0110c10 2024-05-27T07:31:46.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.387+0000 7f64acc00700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f64940b2880 0x7f64940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:46.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.387+0000 7f64acc00700 1 -- 172.21.15.63:0/1285676849 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f64a80e24c0 con 0x7f64b0110c10 2024-05-27T07:31:46.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.387+0000 7f64afe00700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f64940b2880 0x7f64940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:46.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.388+0000 7f64afe00700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f64940b2880 0x7f64940b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64a000b5c0 tx=0x7f64a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:46.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.389+0000 7f64acc00700 1 -- 172.21.15.63:0/1285676849 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f64a8010070 con 0x7f64b0110c10 2024-05-27T07:31:46.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.637+0000 7f64b7200700 1 -- 172.21.15.63:0/1285676849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f64b002e5e0 con 0x7f64b0110c10 2024-05-27T07:31:46.965 INFO:teuthology.orchestra.run.smithi063.stdout:261993005130 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.638+0000 7f64acc00700 1 -- 172.21.15.63:0/1285676849 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f64a80b0e20 con 0x7f64b0110c10 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.641+0000 7f64b7200700 1 -- 172.21.15.63:0/1285676849 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f64940b2880 msgr2=0x7f64940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.641+0000 7f64b7200700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f64940b2880 0x7f64940b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64a000b5c0 tx=0x7f64a0005fb0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.641+0000 7f64b7200700 1 -- 172.21.15.63:0/1285676849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b0110c10 msgr2=0x7f64b01a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.641+0000 7f64b7200700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b0110c10 0x7f64b01a17e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f64a8007ee0 tx=0x7f64a800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.641+0000 7f64b7200700 1 -- 172.21.15.63:0/1285676849 shutdown_connections 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.641+0000 7f64b7200700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f64940b2880 0x7f64940b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.641+0000 7f64b7200700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b0110c10 0x7f64b01a17e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.641+0000 7f64b7200700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64b0075ad0 0x7f64b019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.641+0000 7f64b7200700 1 --2- 172.21.15.63:0/1285676849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64b0076420 0x7f64b019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:46.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.641+0000 7f64b7200700 1 -- 172.21.15.63:0/1285676849 >> 172.21.15.63:0/1285676849 conn(0x7f64b0100030 msgr2=0x7f64b0101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:46.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.642+0000 7f64b7200700 1 -- 172.21.15.63:0/1285676849 shutdown_connections 2024-05-27T07:31:46.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.642+0000 7f64b7200700 1 -- 172.21.15.63:0/1285676849 wait complete. 2024-05-27T07:31:47.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:47 smithi086 ceph-mon[34687]: pgmap v492: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:31:47.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:47 smithi149 ceph-mon[35408]: pgmap v492: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:31:47.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:47 smithi063 ceph-mon[31613]: pgmap v492: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:31:47.883 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005125 got 261993005130 for osd.9 2024-05-27T07:31:47.883 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:48.464 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841597 got 283467841601 for osd.10 2024-05-27T07:31:48.464 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:48.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.714+0000 7f3b42a00700 1 -- 172.21.15.63:0/408440718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c108f10 msgr2=0x7f3b3c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:48.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.714+0000 7f3b42a00700 1 --2- 172.21.15.63:0/408440718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c108f10 0x7f3b3c109320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3b280099c0 tx=0x7f3b28009cd0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.715+0000 7f3b42a00700 1 -- 172.21.15.63:0/408440718 shutdown_connections 2024-05-27T07:31:48.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.715+0000 7f3b42a00700 1 --2- 172.21.15.63:0/408440718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c109860 0x7f3b3c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.715+0000 7f3b42a00700 1 --2- 172.21.15.63:0/408440718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b3c1045e0 0x7f3b3c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.715+0000 7f3b42a00700 1 --2- 172.21.15.63:0/408440718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c108f10 0x7f3b3c109320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.715+0000 7f3b42a00700 1 -- 172.21.15.63:0/408440718 >> 172.21.15.63:0/408440718 conn(0x7f3b3c100030 msgr2=0x7f3b3c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.715+0000 7f3b42a00700 1 -- 172.21.15.63:0/408440718 shutdown_connections 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.715+0000 7f3b42a00700 1 -- 172.21.15.63:0/408440718 wait complete. 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.716+0000 7f3b42a00700 1 Processor -- start 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.716+0000 7f3b42a00700 1 -- start start 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b42a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c1045e0 0x7f3b3c075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b42a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b3c108f10 0x7f3b3c075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b42a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c109860 0x7f3b3c076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b42a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c114370 con 0x7f3b3c109860 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b42a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1141f0 con 0x7f3b3c1045e0 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b42a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c1144f0 con 0x7f3b3c108f10 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b40a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c109860 0x7f3b3c076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:48.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b40a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c109860 0x7f3b3c076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42686/0 (socket says 172.21.15.63:42686) 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b40a00700 1 -- 172.21.15.63:0/3267423884 learned_addr learned my addr 172.21.15.63:0/3267423884 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b3b400700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b3c108f10 0x7f3b3c075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.717+0000 7f3b3be00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c1045e0 0x7f3b3c075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.718+0000 7f3b40a00700 1 -- 172.21.15.63:0/3267423884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b3c108f10 msgr2=0x7f3b3c075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.718+0000 7f3b40a00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b3c108f10 0x7f3b3c075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.718+0000 7f3b40a00700 1 -- 172.21.15.63:0/3267423884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c1045e0 msgr2=0x7f3b3c075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.718+0000 7f3b40a00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c1045e0 0x7f3b3c075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.718+0000 7f3b40a00700 1 -- 172.21.15.63:0/3267423884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3c1a9160 con 0x7f3b3c109860 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.718+0000 7f3b3be00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c1045e0 0x7f3b3c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.718+0000 7f3b40a00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c109860 0x7f3b3c076530 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b34007ee0 tx=0x7f3b3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.718+0000 7f3b38c00700 1 -- 172.21.15.63:0/3267423884 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400dbd0 con 0x7f3b3c109860 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.719+0000 7f3b38c00700 1 -- 172.21.15.63:0/3267423884 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b3400dd70 con 0x7f3b3c109860 2024-05-27T07:31:48.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.719+0000 7f3b38c00700 1 -- 172.21.15.63:0/3267423884 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340058d0 con 0x7f3b3c109860 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.719+0000 7f3b42a00700 1 -- 172.21.15.63:0/3267423884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3c1a9450 con 0x7f3b3c109860 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.719+0000 7f3b42a00700 1 -- 172.21.15.63:0/3267423884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3c1a9910 con 0x7f3b3c109860 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.720+0000 7f3b42a00700 1 -- 172.21.15.63:0/3267423884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3c0512b0 con 0x7f3b3c109860 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.724+0000 7f3b38c00700 1 -- 172.21.15.63:0/3267423884 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3b3401a020 con 0x7f3b3c109860 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.726+0000 7f3b38c00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3b200b2900 0x7f3b200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.726+0000 7f3b38c00700 1 -- 172.21.15.63:0/3267423884 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f3b340e2330 con 0x7f3b3c109860 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.726+0000 7f3b3be00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3b200b2900 0x7f3b200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.727+0000 7f3b38c00700 1 -- 172.21.15.63:0/3267423884 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b34010070 con 0x7f3b3c109860 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.727+0000 7f3b3be00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3b200b2900 0x7f3b200b4e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005950 tx=0x7f3b2c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.985+0000 7f3b42a00700 1 -- 172.21.15.63:0/3267423884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f3b3c02e5e0 con 0x7f3b3c109860 2024-05-27T07:31:48.608 INFO:teuthology.orchestra.run.smithi063.stdout:236223201362 2024-05-27T07:31:48.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.989+0000 7f3b38c00700 1 -- 172.21.15.63:0/3267423884 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3b340a9c60 con 0x7f3b3c109860 2024-05-27T07:31:48.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 -- 172.21.15.63:0/3267423884 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3b200b2900 msgr2=0x7f3b200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:48.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3b200b2900 0x7f3b200b4e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c005950 tx=0x7f3b2c0058e0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 -- 172.21.15.63:0/3267423884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c109860 msgr2=0x7f3b3c076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:48.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c109860 0x7f3b3c076530 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b34007ee0 tx=0x7f3b3400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 -- 172.21.15.63:0/3267423884 shutdown_connections 2024-05-27T07:31:48.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3b200b2900 0x7f3b200b4e10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b3c109860 0x7f3b3c076530 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c1045e0 0x7f3b3c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 --2- 172.21.15.63:0/3267423884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b3c108f10 0x7f3b3c075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 -- 172.21.15.63:0/3267423884 >> 172.21.15.63:0/3267423884 conn(0x7f3b3c100030 msgr2=0x7f3b3c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:48.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 -- 172.21.15.63:0/3267423884 shutdown_connections 2024-05-27T07:31:48.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:44.992+0000 7f3b42a00700 1 -- 172.21.15.63:0/3267423884 wait complete. 2024-05-27T07:31:49.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:49 smithi063 ceph-mon[31613]: pgmap v493: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-27T07:31:49.383 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201356 got 236223201362 for osd.8 2024-05-27T07:31:49.383 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:49.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:49 smithi086 ceph-mon[34687]: pgmap v493: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-27T07:31:49.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:49 smithi149 ceph-mon[35408]: pgmap v493: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-27T07:31:49.678 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738502 got 34359738508 for osd.0 2024-05-27T07:31:49.679 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:50.551 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411449 got 77309411454 for osd.2 2024-05-27T07:31:50.551 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:50.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.738+0000 7f38fec00700 1 -- 172.21.15.63:0/3057159564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38f8110980 msgr2=0x7f38f8110d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:50.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.738+0000 7f38fec00700 1 --2- 172.21.15.63:0/3057159564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38f8110980 0x7f38f8110d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009990 tx=0x7f38e8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.738+0000 7f38fec00700 1 -- 172.21.15.63:0/3057159564 shutdown_connections 2024-05-27T07:31:50.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.738+0000 7f38fec00700 1 --2- 172.21.15.63:0/3057159564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f810b2a0 0x7f38f8117820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.738+0000 7f38fec00700 1 --2- 172.21.15.63:0/3057159564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38f8110980 0x7f38f8110d90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.738+0000 7f38fec00700 1 --2- 172.21.15.63:0/3057159564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38f810a950 0x7f38f810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.738+0000 7f38fec00700 1 -- 172.21.15.63:0/3057159564 >> 172.21.15.63:0/3057159564 conn(0x7f38f8076d20 msgr2=0x7f38f8077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:50.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.738+0000 7f38fec00700 1 -- 172.21.15.63:0/3057159564 shutdown_connections 2024-05-27T07:31:50.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.738+0000 7f38fec00700 1 -- 172.21.15.63:0/3057159564 wait complete. 2024-05-27T07:31:50.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.739+0000 7f38fec00700 1 Processor -- start 2024-05-27T07:31:50.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.739+0000 7f38fec00700 1 -- start start 2024-05-27T07:31:50.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.740+0000 7f38fec00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38f810a950 0x7f38f81a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:50.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.740+0000 7f38fec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f810b2a0 0x7f38f81a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:50.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.740+0000 7f38fec00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38f8110980 0x7f38f81a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:50.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.740+0000 7f38fec00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38f811a930 con 0x7f38f8110980 2024-05-27T07:31:50.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.740+0000 7f38fec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38f811a7b0 con 0x7f38f810b2a0 2024-05-27T07:31:50.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.740+0000 7f38fec00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f38f811aab0 con 0x7f38f810a950 2024-05-27T07:31:50.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.740+0000 7f38fcc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38f8110980 0x7f38f81a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:50.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.740+0000 7f38fcc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38f8110980 0x7f38f81a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60584/0 (socket says 172.21.15.63:60584) 2024-05-27T07:31:50.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.740+0000 7f38fcc00700 1 -- 172.21.15.63:0/2198523917 learned_addr learned my addr 172.21.15.63:0/2198523917 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:50.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.740+0000 7f38f7e00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38f810a950 0x7f38f81a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.741+0000 7f38f7400700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f810b2a0 0x7f38f81a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.741+0000 7f38fcc00700 1 -- 172.21.15.63:0/2198523917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38f810a950 msgr2=0x7f38f81a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.741+0000 7f38fcc00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38f810a950 0x7f38f81a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.741+0000 7f38fcc00700 1 -- 172.21.15.63:0/2198523917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f810b2a0 msgr2=0x7f38f81a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.741+0000 7f38fcc00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f810b2a0 0x7f38f81a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.741+0000 7f38fcc00700 1 -- 172.21.15.63:0/2198523917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f81a6250 con 0x7f38f8110980 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.741+0000 7f38f7400700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f810b2a0 0x7f38f81a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.741+0000 7f38fcc00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38f8110980 0x7f38f81a5bb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f38f0007ee0 tx=0x7f38f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.742+0000 7f38f4c00700 1 -- 172.21.15.63:0/2198523917 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f000dbd0 con 0x7f38f8110980 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.742+0000 7f38f4c00700 1 -- 172.21.15.63:0/2198523917 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38f000dd70 con 0x7f38f8110980 2024-05-27T07:31:50.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.742+0000 7f38fec00700 1 -- 172.21.15.63:0/2198523917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f81a6540 con 0x7f38f8110980 2024-05-27T07:31:50.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.742+0000 7f38f4c00700 1 -- 172.21.15.63:0/2198523917 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f00058d0 con 0x7f38f8110980 2024-05-27T07:31:50.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.742+0000 7f38fec00700 1 -- 172.21.15.63:0/2198523917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f81ade60 con 0x7f38f8110980 2024-05-27T07:31:50.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.743+0000 7f38fec00700 1 -- 172.21.15.63:0/2198523917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f810c090 con 0x7f38f8110980 2024-05-27T07:31:50.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.747+0000 7f38f4c00700 1 -- 172.21.15.63:0/2198523917 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f38f001a020 con 0x7f38f8110980 2024-05-27T07:31:50.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.749+0000 7f38f4c00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f38dc0b2900 0x7f38dc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:50.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.749+0000 7f38f4c00700 1 -- 172.21.15.63:0/2198523917 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f38f00e2320 con 0x7f38f8110980 2024-05-27T07:31:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.749+0000 7f38f7e00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f38dc0b2900 0x7f38dc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.750+0000 7f38f7e00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f38dc0b2900 0x7f38dc0b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f38e800b5c0 tx=0x7f38e8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.752+0000 7f38f4c00700 1 -- 172.21.15.63:0/2198523917 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f38f0010070 con 0x7f38f8110980 2024-05-27T07:31:50.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.992+0000 7f38fec00700 1 -- 172.21.15.63:0/2198523917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f38f802e5e0 con 0x7f38f8110980 2024-05-27T07:31:50.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.993+0000 7f38f4c00700 1 -- 172.21.15.63:0/2198523917 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f38f00a9c50 con 0x7f38f8110980 2024-05-27T07:31:50.986 INFO:teuthology.orchestra.run.smithi063.stdout:309237645372 2024-05-27T07:31:50.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.996+0000 7f38fec00700 1 -- 172.21.15.63:0/2198523917 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f38dc0b2900 msgr2=0x7f38dc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:50.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.996+0000 7f38fec00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f38dc0b2900 0x7f38dc0b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f38e800b5c0 tx=0x7f38e8005960 comp rx=0 tx=0).stop 2024-05-27T07:31:50.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.996+0000 7f38fec00700 1 -- 172.21.15.63:0/2198523917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38f8110980 msgr2=0x7f38f81a5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:50.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.996+0000 7f38fec00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38f8110980 0x7f38f81a5bb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f38f0007ee0 tx=0x7f38f000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.997+0000 7f38fec00700 1 -- 172.21.15.63:0/2198523917 shutdown_connections 2024-05-27T07:31:50.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.997+0000 7f38fec00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f38dc0b2900 0x7f38dc0b4e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.997+0000 7f38fec00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38f8110980 0x7f38f81a5bb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.997+0000 7f38fec00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38f810a950 0x7f38f81a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.997+0000 7f38fec00700 1 --2- 172.21.15.63:0/2198523917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f810b2a0 0x7f38f81a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:50.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.997+0000 7f38fec00700 1 -- 172.21.15.63:0/2198523917 >> 172.21.15.63:0/2198523917 conn(0x7f38f8076d20 msgr2=0x7f38f8109320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:50.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.997+0000 7f38fec00700 1 -- 172.21.15.63:0/2198523917 shutdown_connections 2024-05-27T07:31:50.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:50.997+0000 7f38fec00700 1 -- 172.21.15.63:0/2198523917 wait complete. 2024-05-27T07:31:51.360 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:51 smithi063 ceph-mon[31613]: pgmap v494: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-27T07:31:51.360 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:51 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2198523917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T07:31:51.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:51 smithi086 ceph-mon[34687]: pgmap v494: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-27T07:31:51.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:51 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2198523917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T07:31:51.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:51 smithi149 ceph-mon[35408]: pgmap v494: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-27T07:31:51.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:51 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2198523917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T07:31:51.799 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645366 got 309237645372 for osd.11 2024-05-27T07:31:51.799 DEBUG:teuthology.parallel:result is None 2024-05-27T07:31:51.799 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T07:31:51.800 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T07:31:52.283 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:53.329 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:53 smithi063 ceph-mon[31613]: pgmap v495: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:53.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:53 smithi086 ceph-mon[34687]: pgmap v495: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:53.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:53 smithi149 ceph-mon[35408]: pgmap v495: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:31:54.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.659+0000 7f3d47e00700 1 -- 172.21.15.63:0/314736526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d48106610 msgr2=0x7f3d48106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:54.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.659+0000 7f3d47e00700 1 --2- 172.21.15.63:0/314736526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d48106610 0x7f3d48106a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3d380099c0 tx=0x7f3d38009cd0 comp rx=0 tx=0).stop 2024-05-27T07:31:54.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.660+0000 7f3d47e00700 1 -- 172.21.15.63:0/314736526 shutdown_connections 2024-05-27T07:31:54.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.660+0000 7f3d47e00700 1 --2- 172.21.15.63:0/314736526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4810b890 0x7f3d481131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:54.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.660+0000 7f3d47e00700 1 --2- 172.21.15.63:0/314736526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d48106610 0x7f3d48106a20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:54.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.660+0000 7f3d47e00700 1 --2- 172.21.15.63:0/314736526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d4810af40 0x7f3d4810b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:54.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.660+0000 7f3d47e00700 1 -- 172.21.15.63:0/314736526 >> 172.21.15.63:0/314736526 conn(0x7f3d48102020 msgr2=0x7f3d48104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:54.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.660+0000 7f3d47e00700 1 -- 172.21.15.63:0/314736526 shutdown_connections 2024-05-27T07:31:54.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.660+0000 7f3d47e00700 1 -- 172.21.15.63:0/314736526 wait complete. 2024-05-27T07:31:54.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.661+0000 7f3d47e00700 1 Processor -- start 2024-05-27T07:31:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.661+0000 7f3d47e00700 1 -- start start 2024-05-27T07:31:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.662+0000 7f3d47e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d48106610 0x7f3d4819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.662+0000 7f3d47e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d4810af40 0x7f3d4819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.662+0000 7f3d47e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4810b890 0x7f3d481a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.662+0000 7f3d47e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d481163c0 con 0x7f3d48106610 2024-05-27T07:31:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.662+0000 7f3d47e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48116240 con 0x7f3d4810b890 2024-05-27T07:31:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.662+0000 7f3d47e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3d48116540 con 0x7f3d4810af40 2024-05-27T07:31:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.662+0000 7f3d46a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d48106610 0x7f3d4819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.662+0000 7f3d46a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d48106610 0x7f3d4819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60592/0 (socket says 172.21.15.63:60592) 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.662+0000 7f3d46a00700 1 -- 172.21.15.63:0/766112304 learned_addr learned my addr 172.21.15.63:0/766112304 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.662+0000 7f3d3e000700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d4810af40 0x7f3d4819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.663+0000 7f3d47400700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4810b890 0x7f3d481a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.663+0000 7f3d46a00700 1 -- 172.21.15.63:0/766112304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d4810af40 msgr2=0x7f3d4819d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.663+0000 7f3d46a00700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d4810af40 0x7f3d4819d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.663+0000 7f3d46a00700 1 -- 172.21.15.63:0/766112304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4810b890 msgr2=0x7f3d481a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.663+0000 7f3d46a00700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4810b890 0x7f3d481a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.663+0000 7f3d46a00700 1 -- 172.21.15.63:0/766112304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d481a1c60 con 0x7f3d48106610 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.663+0000 7f3d46a00700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d48106610 0x7f3d4819cd90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800b580 tx=0x7f3d3800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.663+0000 7f3d3fe00700 1 -- 172.21.15.63:0/766112304 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38027070 con 0x7f3d48106610 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.664+0000 7f3d3fe00700 1 -- 172.21.15.63:0/766112304 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d3800fdd0 con 0x7f3d48106610 2024-05-27T07:31:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.664+0000 7f3d47e00700 1 -- 172.21.15.63:0/766112304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d481a1ef0 con 0x7f3d48106610 2024-05-27T07:31:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.664+0000 7f3d3fe00700 1 -- 172.21.15.63:0/766112304 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d380173f0 con 0x7f3d48106610 2024-05-27T07:31:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.664+0000 7f3d47e00700 1 -- 172.21.15.63:0/766112304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d481a23d0 con 0x7f3d48106610 2024-05-27T07:31:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.665+0000 7f3d47e00700 1 -- 172.21.15.63:0/766112304 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d480512b0 con 0x7f3d48106610 2024-05-27T07:31:54.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.668+0000 7f3d3fe00700 1 -- 172.21.15.63:0/766112304 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3d3802a070 con 0x7f3d48106610 2024-05-27T07:31:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.669+0000 7f3d3fe00700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3d2c0b2880 0x7f3d2c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.669+0000 7f3d3fe00700 1 -- 172.21.15.63:0/766112304 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f3d380e3bb0 con 0x7f3d48106610 2024-05-27T07:31:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.670+0000 7f3d3e000700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3d2c0b2880 0x7f3d2c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.670+0000 7f3d3e000700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3d2c0b2880 0x7f3d2c0b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3d48109810 tx=0x7f3d30009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.672+0000 7f3d3fe00700 1 -- 172.21.15.63:0/766112304 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d38031020 con 0x7f3d48106610 2024-05-27T07:31:54.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.902+0000 7f3d47e00700 1 -- 172.21.15.63:0/766112304 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d48063bf0 con 0x7f3d2c0b2880 2024-05-27T07:31:54.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.937+0000 7f3d3fe00700 1 -- 172.21.15.63:0/766112304 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340470 (secure 0 0 0) 0x7f3d48063bf0 con 0x7f3d2c0b2880 2024-05-27T07:31:54.936 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:31:54.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.946+0000 7f3d47e00700 1 -- 172.21.15.63:0/766112304 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3d2c0b2880 msgr2=0x7f3d2c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:54.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.946+0000 7f3d47e00700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3d2c0b2880 0x7f3d2c0b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3d48109810 tx=0x7f3d30009040 comp rx=0 tx=0).stop 2024-05-27T07:31:54.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.946+0000 7f3d47e00700 1 -- 172.21.15.63:0/766112304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d48106610 msgr2=0x7f3d4819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:54.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.946+0000 7f3d47e00700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d48106610 0x7f3d4819cd90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800b580 tx=0x7f3d3800f510 comp rx=0 tx=0).stop 2024-05-27T07:31:54.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.946+0000 7f3d47e00700 1 -- 172.21.15.63:0/766112304 shutdown_connections 2024-05-27T07:31:54.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.946+0000 7f3d47e00700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d4810b890 0x7f3d481a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:54.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.946+0000 7f3d47e00700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3d2c0b2880 0x7f3d2c0b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:54.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.946+0000 7f3d47e00700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d48106610 0x7f3d4819cd90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:54.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.946+0000 7f3d47e00700 1 --2- 172.21.15.63:0/766112304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d4810af40 0x7f3d4819d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:54.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.946+0000 7f3d47e00700 1 -- 172.21.15.63:0/766112304 >> 172.21.15.63:0/766112304 conn(0x7f3d48102020 msgr2=0x7f3d4810a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:54.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.947+0000 7f3d47e00700 1 -- 172.21.15.63:0/766112304 shutdown_connections 2024-05-27T07:31:54.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:54.947+0000 7f3d47e00700 1 -- 172.21.15.63:0/766112304 wait complete. 2024-05-27T07:31:54.941 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-27T07:31:55.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:55 smithi063 ceph-mon[31613]: pgmap v496: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:55.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:55 smithi086 ceph-mon[34687]: pgmap v496: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:55.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:55 smithi149 ceph-mon[35408]: pgmap v496: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:31:55.725 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-05-27T07:31:53.879605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663418,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":813,"num_read_kb":512,"num_write":301,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":744,"ondisk_log_size":744,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375980,"kb_used_data":14348,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565844,"statfs":{"total":1151940427776,"available":1151555424256,"internally_reserved":0,"allocated":14692352,"data_stored":8236424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006536"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776902+0000","last_change":"2024-05-27T07:28:24.485151+0000","last_active":"2024-05-27T07:31:33.776902+0000","last_peered":"2024-05-27T07:31:33.776902+0000","last_clean":"2024-05-27T07:31:33.776902+0000","last_became_active":"2024-05-27T07:28:24.483965+0000","last_became_peered":"2024-05-27T07:28:24.483965+0000","last_unstale":"2024-05-27T07:31:33.776902+0000","last_undegraded":"2024-05-27T07:31:33.776902+0000","last_fullsized":"2024-05-27T07:31:33.776902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:18.376755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.752840+0000","last_change":"2024-05-27T07:28:24.483608+0000","last_active":"2024-05-27T07:31:33.752840+0000","last_peered":"2024-05-27T07:31:33.752840+0000","last_clean":"2024-05-27T07:31:33.752840+0000","last_became_active":"2024-05-27T07:28:24.479875+0000","last_became_peered":"2024-05-27T07:28:24.479875+0000","last_unstale":"2024-05-27T07:31:33.752840+0000","last_undegraded":"2024-05-27T07:31:33.752840+0000","last_fullsized":"2024-05-27T07:31:33.752840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:30.129708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.770675+0000","last_change":"2024-05-27T07:28:24.470104+0000","last_active":"2024-05-27T07:31:33.770675+0000","last_peered":"2024-05-27T07:31:33.770675+0000","last_clean":"2024-05-27T07:31:33.770675+0000","last_became_active":"2024-05-27T07:28:24.469756+0000","last_became_peered":"2024-05-27T07:28:24.469756+0000","last_unstale":"2024-05-27T07:31:33.770675+0000","last_undegraded":"2024-05-27T07:31:33.770675+0000","last_fullsized":"2024-05-27T07:31:33.770675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:36.907691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.747804+0000","last_change":"2024-05-27T07:28:24.472381+0000","last_active":"2024-05-27T07:31:33.747804+0000","last_peered":"2024-05-27T07:31:33.747804+0000","last_clean":"2024-05-27T07:31:33.747804+0000","last_became_active":"2024-05-27T07:28:24.469959+0000","last_became_peered":"2024-05-27T07:28:24.469959+0000","last_unstale":"2024-05-27T07:31:33.747804+0000","last_undegraded":"2024-05-27T07:31:33.747804+0000","last_fullsized":"2024-05-27T07:31:33.747804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:15.267554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776719+0000","last_change":"2024-05-27T07:28:24.474210+0000","last_active":"2024-05-27T07:31:33.776719+0000","last_peered":"2024-05-27T07:31:33.776719+0000","last_clean":"2024-05-27T07:31:33.776719+0000","last_became_active":"2024-05-27T07:28:24.473688+0000","last_became_peered":"2024-05-27T07:28:24.473688+0000","last_unstale":"2024-05-27T07:31:33.776719+0000","last_undegraded":"2024-05-27T07:31:33.776719+0000","last_fullsized":"2024-05-27T07:31:33.776719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:38.733536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.760411+0000","last_change":"2024-05-27T07:28:24.484679+0000","last_active":"2024-05-27T07:31:33.760411+0000","last_peered":"2024-05-27T07:31:33.760411+0000","last_clean":"2024-05-27T07:31:33.760411+0000","last_became_active":"2024-05-27T07:28:24.484251+0000","last_became_peered":"2024-05-27T07:28:24.484251+0000","last_unstale":"2024-05-27T07:31:33.760411+0000","last_undegraded":"2024-05-27T07:31:33.760411+0000","last_fullsized":"2024-05-27T07:31:33.760411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:28:09.507871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.759926+0000","last_change":"2024-05-27T07:28:24.472694+0000","last_active":"2024-05-27T07:31:33.759926+0000","last_peered":"2024-05-27T07:31:33.759926+0000","last_clean":"2024-05-27T07:31:33.759926+0000","last_became_active":"2024-05-27T07:28:24.472100+0000","last_became_peered":"2024-05-27T07:28:24.472100+0000","last_unstale":"2024-05-27T07:31:33.759926+0000","last_undegraded":"2024-05-27T07:31:33.759926+0000","last_fullsized":"2024-05-27T07:31:33.759926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.777956+0000","last_change":"2024-05-27T07:28:24.488577+0000","last_active":"2024-05-27T07:31:33.777956+0000","last_peered":"2024-05-27T07:31:33.777956+0000","last_clean":"2024-05-27T07:31:33.777956+0000","last_became_active":"2024-05-27T07:28:24.488180+0000","last_became_peered":"2024-05-27T07:28:24.488180+0000","last_unstale":"2024-05-27T07:31:33.777956+0000","last_undegraded":"2024-05-27T07:31:33.777956+0000","last_fullsized":"2024-05-27T07:31:33.777956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:17.130058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.781727+0000","last_change":"2024-05-27T07:28:24.480109+0000","last_active":"2024-05-27T07:31:33.781727+0000","last_peered":"2024-05-27T07:31:33.781727+0000","last_clean":"2024-05-27T07:31:33.781727+0000","last_became_active":"2024-05-27T07:28:24.479722+0000","last_became_peered":"2024-05-27T07:28:24.479722+0000","last_unstale":"2024-05-27T07:31:33.781727+0000","last_undegraded":"2024-05-27T07:31:33.781727+0000","last_fullsized":"2024-05-27T07:31:33.781727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:27:36.074612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.756788+0000","last_change":"2024-05-27T07:28:24.490389+0000","last_active":"2024-05-27T07:31:33.756788+0000","last_peered":"2024-05-27T07:31:33.756788+0000","last_clean":"2024-05-27T07:31:33.756788+0000","last_became_active":"2024-05-27T07:28:24.488081+0000","last_became_peered":"2024-05-27T07:28:24.488081+0000","last_unstale":"2024-05-27T07:31:33.756788+0000","last_undegraded":"2024-05-27T07:31:33.756788+0000","last_fullsized":"2024-05-27T07:31:33.756788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:06.318235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776147+0000","last_change":"2024-05-27T07:28:24.477062+0000","last_active":"2024-05-27T07:31:33.776147+0000","last_peered":"2024-05-27T07:31:33.776147+0000","last_clean":"2024-05-27T07:31:33.776147+0000","last_became_active":"2024-05-27T07:28:24.475322+0000","last_became_peered":"2024-05-27T07:28:24.475322+0000","last_unstale":"2024-05-27T07:31:33.776147+0000","last_undegraded":"2024-05-27T07:31:33.776147+0000","last_fullsized":"2024-05-27T07:31:33.776147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:30.365295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.757461+0000","last_change":"2024-05-27T07:29:04.901715+0000","last_active":"2024-05-27T07:31:33.757461+0000","last_peered":"2024-05-27T07:31:33.757461+0000","last_clean":"2024-05-27T07:31:33.757461+0000","last_became_active":"2024-05-27T07:29:04.900968+0000","last_became_peered":"2024-05-27T07:29:04.900968+0000","last_unstale":"2024-05-27T07:31:33.757461+0000","last_undegraded":"2024-05-27T07:31:33.757461+0000","last_fullsized":"2024-05-27T07:31:33.757461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:16:34.033701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.775821+0000","last_change":"2024-05-27T07:28:24.477504+0000","last_active":"2024-05-27T07:31:33.775821+0000","last_peered":"2024-05-27T07:31:33.775821+0000","last_clean":"2024-05-27T07:31:33.775821+0000","last_became_active":"2024-05-27T07:28:24.476709+0000","last_became_peered":"2024-05-27T07:28:24.476709+0000","last_unstale":"2024-05-27T07:31:33.775821+0000","last_undegraded":"2024-05-27T07:31:33.775821+0000","last_fullsized":"2024-05-27T07:31:33.775821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:05:48.456347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.751537+0000","last_change":"2024-05-27T07:28:24.485567+0000","last_active":"2024-05-27T07:31:33.751537+0000","last_peered":"2024-05-27T07:31:33.751537+0000","last_clean":"2024-05-27T07:31:33.751537+0000","last_became_active":"2024-05-27T07:28:24.485175+0000","last_became_peered":"2024-05-27T07:28:24.485175+0000","last_unstale":"2024-05-27T07:31:33.751537+0000","last_undegraded":"2024-05-27T07:31:33.751537+0000","last_fullsized":"2024-05-27T07:31:33.751537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:25:30.062813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.764430+0000","last_change":"2024-05-27T07:29:04.900657+0000","last_active":"2024-05-27T07:31:33.764430+0000","last_peered":"2024-05-27T07:31:33.764430+0000","last_clean":"2024-05-27T07:31:33.764430+0000","last_became_active":"2024-05-27T07:29:04.900093+0000","last_became_peered":"2024-05-27T07:29:04.900093+0000","last_unstale":"2024-05-27T07:31:33.764430+0000","last_undegraded":"2024-05-27T07:31:33.764430+0000","last_fullsized":"2024-05-27T07:31:33.764430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:28:04.157825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.766949+0000","last_change":"2024-05-27T07:28:24.470503+0000","last_active":"2024-05-27T07:31:33.766949+0000","last_peered":"2024-05-27T07:31:33.766949+0000","last_clean":"2024-05-27T07:31:33.766949+0000","last_became_active":"2024-05-27T07:28:24.470128+0000","last_became_peered":"2024-05-27T07:28:24.470128+0000","last_unstale":"2024-05-27T07:31:33.766949+0000","last_undegraded":"2024-05-27T07:31:33.766949+0000","last_fullsized":"2024-05-27T07:31:33.766949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:46:43.149635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.780079+0000","last_change":"2024-05-27T07:28:24.491382+0000","last_active":"2024-05-27T07:31:33.780079+0000","last_peered":"2024-05-27T07:31:33.780079+0000","last_clean":"2024-05-27T07:31:33.780079+0000","last_became_active":"2024-05-27T07:28:24.486155+0000","last_became_peered":"2024-05-27T07:28:24.486155+0000","last_unstale":"2024-05-27T07:31:33.780079+0000","last_undegraded":"2024-05-27T07:31:33.780079+0000","last_fullsized":"2024-05-27T07:31:33.780079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:48.408573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.773959+0000","last_change":"2024-05-27T07:28:24.489140+0000","last_active":"2024-05-27T07:31:33.773959+0000","last_peered":"2024-05-27T07:31:33.773959+0000","last_clean":"2024-05-27T07:31:33.773959+0000","last_became_active":"2024-05-27T07:28:24.488733+0000","last_became_peered":"2024-05-27T07:28:24.488733+0000","last_unstale":"2024-05-27T07:31:33.773959+0000","last_undegraded":"2024-05-27T07:31:33.773959+0000","last_fullsized":"2024-05-27T07:31:33.773959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:02:57.711197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776828+0000","last_change":"2024-05-27T07:28:24.486022+0000","last_active":"2024-05-27T07:31:33.776828+0000","last_peered":"2024-05-27T07:31:33.776828+0000","last_clean":"2024-05-27T07:31:33.776828+0000","last_became_active":"2024-05-27T07:28:24.485639+0000","last_became_peered":"2024-05-27T07:28:24.485639+0000","last_unstale":"2024-05-27T07:31:33.776828+0000","last_undegraded":"2024-05-27T07:31:33.776828+0000","last_fullsized":"2024-05-27T07:31:33.776828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:07.629402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119584+0000","last_change":"2024-05-27T07:28:37.898984+0000","last_active":"2024-05-27T07:30:05.119584+0000","last_peered":"2024-05-27T07:30:05.119584+0000","last_clean":"2024-05-27T07:30:05.119584+0000","last_became_active":"2024-05-27T07:28:37.898132+0000","last_became_peered":"2024-05-27T07:28:37.898132+0000","last_unstale":"2024-05-27T07:30:05.119584+0000","last_undegraded":"2024-05-27T07:30:05.119584+0000","last_fullsized":"2024-05-27T07:30:05.119584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:51.987386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127598+0000","last_change":"2024-05-27T07:28:20.402788+0000","last_active":"2024-05-27T07:30:05.127598+0000","last_peered":"2024-05-27T07:30:05.127598+0000","last_clean":"2024-05-27T07:30:05.127598+0000","last_became_active":"2024-05-27T07:28:20.402041+0000","last_became_peered":"2024-05-27T07:28:20.402041+0000","last_unstale":"2024-05-27T07:30:05.127598+0000","last_undegraded":"2024-05-27T07:30:05.127598+0000","last_fullsized":"2024-05-27T07:30:05.127598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:44.421979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.774601+0000","last_change":"2024-05-27T07:28:24.485193+0000","last_active":"2024-05-27T07:31:33.774601+0000","last_peered":"2024-05-27T07:31:33.774601+0000","last_clean":"2024-05-27T07:31:33.774601+0000","last_became_active":"2024-05-27T07:28:24.484086+0000","last_became_peered":"2024-05-27T07:28:24.484086+0000","last_unstale":"2024-05-27T07:31:33.774601+0000","last_undegraded":"2024-05-27T07:31:33.774601+0000","last_fullsized":"2024-05-27T07:31:33.774601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:10.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536567+0000","last_change":"2024-05-27T07:28:37.902602+0000","last_active":"2024-05-27T07:30:05.536567+0000","last_peered":"2024-05-27T07:30:05.536567+0000","last_clean":"2024-05-27T07:30:05.536567+0000","last_became_active":"2024-05-27T07:28:37.902120+0000","last_became_peered":"2024-05-27T07:28:37.902120+0000","last_unstale":"2024-05-27T07:30:05.536567+0000","last_undegraded":"2024-05-27T07:30:05.536567+0000","last_fullsized":"2024-05-27T07:30:05.536567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:21.209851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122003+0000","last_change":"2024-05-27T07:28:20.465500+0000","last_active":"2024-05-27T07:30:05.122003+0000","last_peered":"2024-05-27T07:30:05.122003+0000","last_clean":"2024-05-27T07:30:05.122003+0000","last_became_active":"2024-05-27T07:28:20.464951+0000","last_became_peered":"2024-05-27T07:28:20.464951+0000","last_unstale":"2024-05-27T07:30:05.122003+0000","last_undegraded":"2024-05-27T07:30:05.122003+0000","last_fullsized":"2024-05-27T07:30:05.122003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:23:25.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.780254+0000","last_change":"2024-05-27T07:28:24.467655+0000","last_active":"2024-05-27T07:31:33.780254+0000","last_peered":"2024-05-27T07:31:33.780254+0000","last_clean":"2024-05-27T07:31:33.780254+0000","last_became_active":"2024-05-27T07:28:24.467194+0000","last_became_peered":"2024-05-27T07:28:24.467194+0000","last_unstale":"2024-05-27T07:31:33.780254+0000","last_undegraded":"2024-05-27T07:31:33.780254+0000","last_fullsized":"2024-05-27T07:31:33.780254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:38.024514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536923+0000","last_change":"2024-05-27T07:29:04.975679+0000","last_active":"2024-05-27T07:30:05.536923+0000","last_peered":"2024-05-27T07:30:05.536923+0000","last_clean":"2024-05-27T07:30:05.536923+0000","last_became_active":"2024-05-27T07:29:04.974323+0000","last_became_peered":"2024-05-27T07:29:04.974323+0000","last_unstale":"2024-05-27T07:30:05.536923+0000","last_undegraded":"2024-05-27T07:30:05.536923+0000","last_fullsized":"2024-05-27T07:30:05.536923+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:50.489737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584407+0000","last_change":"2024-05-27T07:28:20.408117+0000","last_active":"2024-05-27T07:30:05.584407+0000","last_peered":"2024-05-27T07:30:05.584407+0000","last_clean":"2024-05-27T07:30:05.584407+0000","last_became_active":"2024-05-27T07:28:20.407723+0000","last_became_peered":"2024-05-27T07:28:20.407723+0000","last_unstale":"2024-05-27T07:30:05.584407+0000","last_undegraded":"2024-05-27T07:30:05.584407+0000","last_fullsized":"2024-05-27T07:30:05.584407+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:20:31.376656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.779839+0000","last_change":"2024-05-27T07:28:24.490269+0000","last_active":"2024-05-27T07:31:33.779839+0000","last_peered":"2024-05-27T07:31:33.779839+0000","last_clean":"2024-05-27T07:31:33.779839+0000","last_became_active":"2024-05-27T07:28:24.488849+0000","last_became_peered":"2024-05-27T07:28:24.488849+0000","last_unstale":"2024-05-27T07:31:33.779839+0000","last_undegraded":"2024-05-27T07:31:33.779839+0000","last_fullsized":"2024-05-27T07:31:33.779839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:00.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536807+0000","last_change":"2024-05-27T07:28:37.908658+0000","last_active":"2024-05-27T07:30:05.536807+0000","last_peered":"2024-05-27T07:30:05.536807+0000","last_clean":"2024-05-27T07:30:05.536807+0000","last_became_active":"2024-05-27T07:28:37.908056+0000","last_became_peered":"2024-05-27T07:28:37.908056+0000","last_unstale":"2024-05-27T07:30:05.536807+0000","last_undegraded":"2024-05-27T07:30:05.536807+0000","last_fullsized":"2024-05-27T07:30:05.536807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:31.853606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583611+0000","last_change":"2024-05-27T07:28:20.410944+0000","last_active":"2024-05-27T07:30:05.583611+0000","last_peered":"2024-05-27T07:30:05.583611+0000","last_clean":"2024-05-27T07:30:05.583611+0000","last_became_active":"2024-05-27T07:28:20.409789+0000","last_became_peered":"2024-05-27T07:28:20.409789+0000","last_unstale":"2024-05-27T07:30:05.583611+0000","last_undegraded":"2024-05-27T07:30:05.583611+0000","last_fullsized":"2024-05-27T07:30:05.583611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:53:34.962397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.767919+0000","last_change":"2024-05-27T07:28:24.482828+0000","last_active":"2024-05-27T07:31:33.767919+0000","last_peered":"2024-05-27T07:31:33.767919+0000","last_clean":"2024-05-27T07:31:33.767919+0000","last_became_active":"2024-05-27T07:28:24.482475+0000","last_became_peered":"2024-05-27T07:28:24.482475+0000","last_unstale":"2024-05-27T07:31:33.767919+0000","last_undegraded":"2024-05-27T07:31:33.767919+0000","last_fullsized":"2024-05-27T07:31:33.767919+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:53:32.064441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120655+0000","last_change":"2024-05-27T07:29:04.905734+0000","last_active":"2024-05-27T07:30:05.120655+0000","last_peered":"2024-05-27T07:30:05.120655+0000","last_clean":"2024-05-27T07:30:05.120655+0000","last_became_active":"2024-05-27T07:29:04.902387+0000","last_became_peered":"2024-05-27T07:29:04.902387+0000","last_unstale":"2024-05-27T07:30:05.120655+0000","last_undegraded":"2024-05-27T07:30:05.120655+0000","last_fullsized":"2024-05-27T07:30:05.120655+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:35.993193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":170,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:49.410649+0000","last_change":"2024-05-27T07:28:20.412258+0000","last_active":"2024-05-27T07:31:49.410649+0000","last_peered":"2024-05-27T07:31:49.410649+0000","last_clean":"2024-05-27T07:31:49.410649+0000","last_became_active":"2024-05-27T07:28:20.411764+0000","last_became_peered":"2024-05-27T07:28:20.411764+0000","last_unstale":"2024-05-27T07:31:49.410649+0000","last_undegraded":"2024-05-27T07:31:49.410649+0000","last_fullsized":"2024-05-27T07:31:49.410649+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:38.462887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.759152+0000","last_change":"2024-05-27T07:28:24.482179+0000","last_active":"2024-05-27T07:31:33.759152+0000","last_peered":"2024-05-27T07:31:33.759152+0000","last_clean":"2024-05-27T07:31:33.759152+0000","last_became_active":"2024-05-27T07:28:24.480682+0000","last_became_peered":"2024-05-27T07:28:24.480682+0000","last_unstale":"2024-05-27T07:31:33.759152+0000","last_undegraded":"2024-05-27T07:31:33.759152+0000","last_fullsized":"2024-05-27T07:31:33.759152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:51.403862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119903+0000","last_change":"2024-05-27T07:28:37.903716+0000","last_active":"2024-05-27T07:30:05.119903+0000","last_peered":"2024-05-27T07:30:05.119903+0000","last_clean":"2024-05-27T07:30:05.119903+0000","last_became_active":"2024-05-27T07:28:37.901083+0000","last_became_peered":"2024-05-27T07:28:37.901083+0000","last_unstale":"2024-05-27T07:30:05.119903+0000","last_undegraded":"2024-05-27T07:30:05.119903+0000","last_fullsized":"2024-05-27T07:30:05.119903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:44:11.606711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578206+0000","last_change":"2024-05-27T07:28:20.392441+0000","last_active":"2024-05-27T07:30:05.578206+0000","last_peered":"2024-05-27T07:30:05.578206+0000","last_clean":"2024-05-27T07:30:05.578206+0000","last_became_active":"2024-05-27T07:28:20.390983+0000","last_became_peered":"2024-05-27T07:28:20.390983+0000","last_unstale":"2024-05-27T07:30:05.578206+0000","last_undegraded":"2024-05-27T07:30:05.578206+0000","last_fullsized":"2024-05-27T07:30:05.578206+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:33:30.338043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.773247+0000","last_change":"2024-05-27T07:30:05.518915+0000","last_active":"2024-05-27T07:31:33.773247+0000","last_peered":"2024-05-27T07:31:33.773247+0000","last_clean":"2024-05-27T07:31:33.773247+0000","last_became_active":"2024-05-27T07:30:05.517894+0000","last_became_peered":"2024-05-27T07:30:05.517894+0000","last_unstale":"2024-05-27T07:31:33.773247+0000","last_undegraded":"2024-05-27T07:31:33.773247+0000","last_fullsized":"2024-05-27T07:31:33.773247+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:51.353713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,7],"acting":[9,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580964+0000","last_change":"2024-05-27T07:28:37.913352+0000","last_active":"2024-05-27T07:30:05.580964+0000","last_peered":"2024-05-27T07:30:05.580964+0000","last_clean":"2024-05-27T07:30:05.580964+0000","last_became_active":"2024-05-27T07:28:37.913027+0000","last_became_peered":"2024-05-27T07:28:37.913027+0000","last_unstale":"2024-05-27T07:30:05.580964+0000","last_undegraded":"2024-05-27T07:30:05.580964+0000","last_fullsized":"2024-05-27T07:30:05.580964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:03.485594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.404589+0000","last_change":"2024-05-27T07:28:20.406094+0000","last_active":"2024-05-27T07:30:05.404589+0000","last_peered":"2024-05-27T07:30:05.404589+0000","last_clean":"2024-05-27T07:30:05.404589+0000","last_became_active":"2024-05-27T07:28:20.405641+0000","last_became_peered":"2024-05-27T07:28:20.405641+0000","last_unstale":"2024-05-27T07:30:05.404589+0000","last_undegraded":"2024-05-27T07:30:05.404589+0000","last_fullsized":"2024-05-27T07:30:05.404589+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:16.598743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.777772+0000","last_change":"2024-05-27T07:28:24.474696+0000","last_active":"2024-05-27T07:31:33.777772+0000","last_peered":"2024-05-27T07:31:33.777772+0000","last_clean":"2024-05-27T07:31:33.777772+0000","last_became_active":"2024-05-27T07:28:24.474319+0000","last_became_peered":"2024-05-27T07:28:24.474319+0000","last_unstale":"2024-05-27T07:31:33.777772+0000","last_undegraded":"2024-05-27T07:31:33.777772+0000","last_fullsized":"2024-05-27T07:31:33.777772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:53.010898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576132+0000","last_change":"2024-05-27T07:28:37.910386+0000","last_active":"2024-05-27T07:30:05.576132+0000","last_peered":"2024-05-27T07:30:05.576132+0000","last_clean":"2024-05-27T07:30:05.576132+0000","last_became_active":"2024-05-27T07:28:37.909941+0000","last_became_peered":"2024-05-27T07:28:37.909941+0000","last_unstale":"2024-05-27T07:30:05.576132+0000","last_undegraded":"2024-05-27T07:30:05.576132+0000","last_fullsized":"2024-05-27T07:30:05.576132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:15.820318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582005+0000","last_change":"2024-05-27T07:28:20.411855+0000","last_active":"2024-05-27T07:30:05.582005+0000","last_peered":"2024-05-27T07:30:05.582005+0000","last_clean":"2024-05-27T07:30:05.582005+0000","last_became_active":"2024-05-27T07:28:20.411169+0000","last_became_peered":"2024-05-27T07:28:20.411169+0000","last_unstale":"2024-05-27T07:30:05.582005+0000","last_undegraded":"2024-05-27T07:30:05.582005+0000","last_fullsized":"2024-05-27T07:30:05.582005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:37.278507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.779523+0000","last_change":"2024-05-27T07:28:24.485799+0000","last_active":"2024-05-27T07:31:33.779523+0000","last_peered":"2024-05-27T07:31:33.779523+0000","last_clean":"2024-05-27T07:31:33.779523+0000","last_became_active":"2024-05-27T07:28:24.485395+0000","last_became_peered":"2024-05-27T07:28:24.485395+0000","last_unstale":"2024-05-27T07:31:33.779523+0000","last_undegraded":"2024-05-27T07:31:33.779523+0000","last_fullsized":"2024-05-27T07:31:33.779523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:11.178129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120055+0000","last_change":"2024-05-27T07:28:37.898929+0000","last_active":"2024-05-27T07:30:05.120055+0000","last_peered":"2024-05-27T07:30:05.120055+0000","last_clean":"2024-05-27T07:30:05.120055+0000","last_became_active":"2024-05-27T07:28:37.898371+0000","last_became_peered":"2024-05-27T07:28:37.898371+0000","last_unstale":"2024-05-27T07:30:05.120055+0000","last_undegraded":"2024-05-27T07:30:05.120055+0000","last_fullsized":"2024-05-27T07:30:05.120055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:48.867729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":164,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:47.403653+0000","last_change":"2024-05-27T07:28:20.407117+0000","last_active":"2024-05-27T07:31:47.403653+0000","last_peered":"2024-05-27T07:31:47.403653+0000","last_clean":"2024-05-27T07:31:47.403653+0000","last_became_active":"2024-05-27T07:28:20.403562+0000","last_became_peered":"2024-05-27T07:28:20.403562+0000","last_unstale":"2024-05-27T07:31:47.403653+0000","last_undegraded":"2024-05-27T07:31:47.403653+0000","last_fullsized":"2024-05-27T07:31:47.403653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:25.903886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776246+0000","last_change":"2024-05-27T07:28:24.489771+0000","last_active":"2024-05-27T07:31:33.776246+0000","last_peered":"2024-05-27T07:31:33.776246+0000","last_clean":"2024-05-27T07:31:33.776246+0000","last_became_active":"2024-05-27T07:28:24.489163+0000","last_became_peered":"2024-05-27T07:28:24.489163+0000","last_unstale":"2024-05-27T07:31:33.776246+0000","last_undegraded":"2024-05-27T07:31:33.776246+0000","last_fullsized":"2024-05-27T07:31:33.776246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:08.053634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.123674+0000","last_change":"2024-05-27T07:28:37.914598+0000","last_active":"2024-05-27T07:30:05.123674+0000","last_peered":"2024-05-27T07:30:05.123674+0000","last_clean":"2024-05-27T07:30:05.123674+0000","last_became_active":"2024-05-27T07:28:37.913916+0000","last_became_peered":"2024-05-27T07:28:37.913916+0000","last_unstale":"2024-05-27T07:30:05.123674+0000","last_undegraded":"2024-05-27T07:30:05.123674+0000","last_fullsized":"2024-05-27T07:30:05.123674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:01.475789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.517157+0000","last_change":"2024-05-27T07:29:04.901366+0000","last_active":"2024-05-27T07:30:05.517157+0000","last_peered":"2024-05-27T07:30:05.517157+0000","last_clean":"2024-05-27T07:30:05.517157+0000","last_became_active":"2024-05-27T07:29:04.900355+0000","last_became_peered":"2024-05-27T07:29:04.900355+0000","last_unstale":"2024-05-27T07:30:05.517157+0000","last_undegraded":"2024-05-27T07:30:05.517157+0000","last_fullsized":"2024-05-27T07:30:05.517157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:10:03.349345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.764329+0000","last_change":"2024-05-27T07:28:24.477771+0000","last_active":"2024-05-27T07:31:33.764329+0000","last_peered":"2024-05-27T07:31:33.764329+0000","last_clean":"2024-05-27T07:31:33.764329+0000","last_became_active":"2024-05-27T07:28:24.474179+0000","last_became_peered":"2024-05-27T07:28:24.474179+0000","last_unstale":"2024-05-27T07:31:33.764329+0000","last_undegraded":"2024-05-27T07:31:33.764329+0000","last_fullsized":"2024-05-27T07:31:33.764329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:08:04.218264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537477+0000","last_change":"2024-05-27T07:29:04.896583+0000","last_active":"2024-05-27T07:30:05.537477+0000","last_peered":"2024-05-27T07:30:05.537477+0000","last_clean":"2024-05-27T07:30:05.537477+0000","last_became_active":"2024-05-27T07:29:04.894118+0000","last_became_peered":"2024-05-27T07:29:04.894118+0000","last_unstale":"2024-05-27T07:30:05.537477+0000","last_undegraded":"2024-05-27T07:30:05.537477+0000","last_fullsized":"2024-05-27T07:30:05.537477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:17.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583611+0000","last_change":"2024-05-27T07:28:20.466851+0000","last_active":"2024-05-27T07:30:05.583611+0000","last_peered":"2024-05-27T07:30:05.583611+0000","last_clean":"2024-05-27T07:30:05.583611+0000","last_became_active":"2024-05-27T07:28:20.465133+0000","last_became_peered":"2024-05-27T07:28:20.465133+0000","last_unstale":"2024-05-27T07:30:05.583611+0000","last_undegraded":"2024-05-27T07:30:05.583611+0000","last_fullsized":"2024-05-27T07:30:05.583611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:11.025588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.779962+0000","last_change":"2024-05-27T07:28:24.469924+0000","last_active":"2024-05-27T07:31:33.779962+0000","last_peered":"2024-05-27T07:31:33.779962+0000","last_clean":"2024-05-27T07:31:33.779962+0000","last_became_active":"2024-05-27T07:28:24.469559+0000","last_became_peered":"2024-05-27T07:28:24.469559+0000","last_unstale":"2024-05-27T07:31:33.779962+0000","last_undegraded":"2024-05-27T07:31:33.779962+0000","last_fullsized":"2024-05-27T07:31:33.779962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:57.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.125161+0000","last_change":"2024-05-27T07:28:37.904549+0000","last_active":"2024-05-27T07:30:05.125161+0000","last_peered":"2024-05-27T07:30:05.125161+0000","last_clean":"2024-05-27T07:30:05.125161+0000","last_became_active":"2024-05-27T07:28:37.903882+0000","last_became_peered":"2024-05-27T07:28:37.903882+0000","last_unstale":"2024-05-27T07:30:05.125161+0000","last_undegraded":"2024-05-27T07:30:05.125161+0000","last_fullsized":"2024-05-27T07:30:05.125161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:41.607641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'39","reported_seq":329,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:48.405029+0000","last_change":"2024-05-27T07:28:20.410025+0000","last_active":"2024-05-27T07:31:48.405029+0000","last_peered":"2024-05-27T07:31:48.405029+0000","last_clean":"2024-05-27T07:31:48.405029+0000","last_became_active":"2024-05-27T07:28:20.409115+0000","last_became_peered":"2024-05-27T07:28:20.409115+0000","last_unstale":"2024-05-27T07:31:48.405029+0000","last_undegraded":"2024-05-27T07:31:48.405029+0000","last_fullsized":"2024-05-27T07:31:48.405029+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:56.834059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69206,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":245,"num_read_kb":191,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.758038+0000","last_change":"2024-05-27T07:28:24.469609+0000","last_active":"2024-05-27T07:31:33.758038+0000","last_peered":"2024-05-27T07:31:33.758038+0000","last_clean":"2024-05-27T07:31:33.758038+0000","last_became_active":"2024-05-27T07:28:24.469192+0000","last_became_peered":"2024-05-27T07:28:24.469192+0000","last_unstale":"2024-05-27T07:31:33.758038+0000","last_undegraded":"2024-05-27T07:31:33.758038+0000","last_fullsized":"2024-05-27T07:31:33.758038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:32:59.006298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.585019+0000","last_change":"2024-05-27T07:28:37.900745+0000","last_active":"2024-05-27T07:30:05.585019+0000","last_peered":"2024-05-27T07:30:05.585019+0000","last_clean":"2024-05-27T07:30:05.585019+0000","last_became_active":"2024-05-27T07:28:37.900205+0000","last_became_peered":"2024-05-27T07:28:37.900205+0000","last_unstale":"2024-05-27T07:30:05.585019+0000","last_undegraded":"2024-05-27T07:30:05.585019+0000","last_fullsized":"2024-05-27T07:30:05.585019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:52:32.235201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120246+0000","last_change":"2024-05-27T07:28:20.413380+0000","last_active":"2024-05-27T07:30:05.120246+0000","last_peered":"2024-05-27T07:30:05.120246+0000","last_clean":"2024-05-27T07:30:05.120246+0000","last_became_active":"2024-05-27T07:28:20.412839+0000","last_became_peered":"2024-05-27T07:28:20.412839+0000","last_unstale":"2024-05-27T07:30:05.120246+0000","last_undegraded":"2024-05-27T07:30:05.120246+0000","last_fullsized":"2024-05-27T07:30:05.120246+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:57.481203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.777363+0000","last_change":"2024-05-27T07:28:24.474228+0000","last_active":"2024-05-27T07:31:33.777363+0000","last_peered":"2024-05-27T07:31:33.777363+0000","last_clean":"2024-05-27T07:31:33.777363+0000","last_became_active":"2024-05-27T07:28:24.473833+0000","last_became_peered":"2024-05-27T07:28:24.473833+0000","last_unstale":"2024-05-27T07:31:33.777363+0000","last_undegraded":"2024-05-27T07:31:33.777363+0000","last_fullsized":"2024-05-27T07:31:33.777363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:52.456730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580416+0000","last_change":"2024-05-27T07:28:37.917630+0000","last_active":"2024-05-27T07:30:05.580416+0000","last_peered":"2024-05-27T07:30:05.580416+0000","last_clean":"2024-05-27T07:30:05.580416+0000","last_became_active":"2024-05-27T07:28:37.917253+0000","last_became_peered":"2024-05-27T07:28:37.917253+0000","last_unstale":"2024-05-27T07:30:05.580416+0000","last_undegraded":"2024-05-27T07:30:05.580416+0000","last_fullsized":"2024-05-27T07:30:05.580416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:05.069946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584721+0000","last_change":"2024-05-27T07:28:20.408988+0000","last_active":"2024-05-27T07:30:05.584721+0000","last_peered":"2024-05-27T07:30:05.584721+0000","last_clean":"2024-05-27T07:30:05.584721+0000","last_became_active":"2024-05-27T07:28:20.408313+0000","last_became_peered":"2024-05-27T07:28:20.408313+0000","last_unstale":"2024-05-27T07:30:05.584721+0000","last_undegraded":"2024-05-27T07:30:05.584721+0000","last_fullsized":"2024-05-27T07:30:05.584721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:17.480412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.770467+0000","last_change":"2024-05-27T07:28:24.480096+0000","last_active":"2024-05-27T07:31:33.770467+0000","last_peered":"2024-05-27T07:31:33.770467+0000","last_clean":"2024-05-27T07:31:33.770467+0000","last_became_active":"2024-05-27T07:28:24.477988+0000","last_became_peered":"2024-05-27T07:28:24.477988+0000","last_unstale":"2024-05-27T07:31:33.770467+0000","last_undegraded":"2024-05-27T07:31:33.770467+0000","last_fullsized":"2024-05-27T07:31:33.770467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:13:59.860458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127148+0000","last_change":"2024-05-27T07:28:37.917518+0000","last_active":"2024-05-27T07:30:05.127148+0000","last_peered":"2024-05-27T07:30:05.127148+0000","last_clean":"2024-05-27T07:30:05.127148+0000","last_became_active":"2024-05-27T07:28:37.916971+0000","last_became_peered":"2024-05-27T07:28:37.916971+0000","last_unstale":"2024-05-27T07:30:05.127148+0000","last_undegraded":"2024-05-27T07:30:05.127148+0000","last_fullsized":"2024-05-27T07:30:05.127148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:17:53.067747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":167,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:48.419462+0000","last_change":"2024-05-27T07:28:20.416766+0000","last_active":"2024-05-27T07:31:48.419462+0000","last_peered":"2024-05-27T07:31:48.419462+0000","last_clean":"2024-05-27T07:31:48.419462+0000","last_became_active":"2024-05-27T07:28:20.416297+0000","last_became_peered":"2024-05-27T07:28:20.416297+0000","last_unstale":"2024-05-27T07:31:48.419462+0000","last_undegraded":"2024-05-27T07:31:48.419462+0000","last_fullsized":"2024-05-27T07:31:48.419462+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:02.841494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.756721+0000","last_change":"2024-05-27T07:28:24.486869+0000","last_active":"2024-05-27T07:31:33.756721+0000","last_peered":"2024-05-27T07:31:33.756721+0000","last_clean":"2024-05-27T07:31:33.756721+0000","last_became_active":"2024-05-27T07:28:24.486451+0000","last_became_peered":"2024-05-27T07:28:24.486451+0000","last_unstale":"2024-05-27T07:31:33.756721+0000","last_undegraded":"2024-05-27T07:31:33.756721+0000","last_fullsized":"2024-05-27T07:31:33.756721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:10.722659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406823+0000","last_change":"2024-05-27T07:29:04.901632+0000","last_active":"2024-05-27T07:30:05.406823+0000","last_peered":"2024-05-27T07:30:05.406823+0000","last_clean":"2024-05-27T07:30:05.406823+0000","last_became_active":"2024-05-27T07:29:04.900203+0000","last_became_peered":"2024-05-27T07:29:04.900203+0000","last_unstale":"2024-05-27T07:30:05.406823+0000","last_undegraded":"2024-05-27T07:30:05.406823+0000","last_fullsized":"2024-05-27T07:30:05.406823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:57.834889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580102+0000","last_change":"2024-05-27T07:28:20.468162+0000","last_active":"2024-05-27T07:30:05.580102+0000","last_peered":"2024-05-27T07:30:05.580102+0000","last_clean":"2024-05-27T07:30:05.580102+0000","last_became_active":"2024-05-27T07:28:20.465226+0000","last_became_peered":"2024-05-27T07:28:20.465226+0000","last_unstale":"2024-05-27T07:30:05.580102+0000","last_undegraded":"2024-05-27T07:30:05.580102+0000","last_fullsized":"2024-05-27T07:30:05.580102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:21:34.110356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.756405+0000","last_change":"2024-05-27T07:28:24.485053+0000","last_active":"2024-05-27T07:31:33.756405+0000","last_peered":"2024-05-27T07:31:33.756405+0000","last_clean":"2024-05-27T07:31:33.756405+0000","last_became_active":"2024-05-27T07:28:24.484391+0000","last_became_peered":"2024-05-27T07:28:24.484391+0000","last_unstale":"2024-05-27T07:31:33.756405+0000","last_undegraded":"2024-05-27T07:31:33.756405+0000","last_fullsized":"2024-05-27T07:31:33.756405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:05.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582857+0000","last_change":"2024-05-27T07:28:37.904731+0000","last_active":"2024-05-27T07:30:05.582857+0000","last_peered":"2024-05-27T07:30:05.582857+0000","last_clean":"2024-05-27T07:30:05.582857+0000","last_became_active":"2024-05-27T07:28:37.904284+0000","last_became_peered":"2024-05-27T07:28:37.904284+0000","last_unstale":"2024-05-27T07:30:05.582857+0000","last_undegraded":"2024-05-27T07:30:05.582857+0000","last_fullsized":"2024-05-27T07:30:05.582857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:23.198465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.585639+0000","last_change":"2024-05-27T07:28:20.406829+0000","last_active":"2024-05-27T07:30:05.585639+0000","last_peered":"2024-05-27T07:30:05.585639+0000","last_clean":"2024-05-27T07:30:05.585639+0000","last_became_active":"2024-05-27T07:28:20.405497+0000","last_became_peered":"2024-05-27T07:28:20.405497+0000","last_unstale":"2024-05-27T07:30:05.585639+0000","last_undegraded":"2024-05-27T07:30:05.585639+0000","last_fullsized":"2024-05-27T07:30:05.585639+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:14.225630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.745746+0000","last_change":"2024-05-27T07:28:24.488023+0000","last_active":"2024-05-27T07:31:33.745746+0000","last_peered":"2024-05-27T07:31:33.745746+0000","last_clean":"2024-05-27T07:31:33.745746+0000","last_became_active":"2024-05-27T07:28:24.487652+0000","last_became_peered":"2024-05-27T07:28:24.487652+0000","last_unstale":"2024-05-27T07:31:33.745746+0000","last_undegraded":"2024-05-27T07:31:33.745746+0000","last_fullsized":"2024-05-27T07:31:33.745746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:25.951157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581217+0000","last_change":"2024-05-27T07:28:37.968034+0000","last_active":"2024-05-27T07:30:05.581217+0000","last_peered":"2024-05-27T07:30:05.581217+0000","last_clean":"2024-05-27T07:30:05.581217+0000","last_became_active":"2024-05-27T07:28:37.967127+0000","last_became_peered":"2024-05-27T07:28:37.967127+0000","last_unstale":"2024-05-27T07:30:05.581217+0000","last_undegraded":"2024-05-27T07:30:05.581217+0000","last_fullsized":"2024-05-27T07:30:05.581217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:49.353423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.126685+0000","last_change":"2024-05-27T07:28:20.465782+0000","last_active":"2024-05-27T07:30:05.126685+0000","last_peered":"2024-05-27T07:30:05.126685+0000","last_clean":"2024-05-27T07:30:05.126685+0000","last_became_active":"2024-05-27T07:28:20.465134+0000","last_became_peered":"2024-05-27T07:28:20.465134+0000","last_unstale":"2024-05-27T07:30:05.126685+0000","last_undegraded":"2024-05-27T07:30:05.126685+0000","last_fullsized":"2024-05-27T07:30:05.126685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:17.992092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.761621+0000","last_change":"2024-05-27T07:28:24.484691+0000","last_active":"2024-05-27T07:31:33.761621+0000","last_peered":"2024-05-27T07:31:33.761621+0000","last_clean":"2024-05-27T07:31:33.761621+0000","last_became_active":"2024-05-27T07:28:24.484249+0000","last_became_peered":"2024-05-27T07:28:24.484249+0000","last_unstale":"2024-05-27T07:31:33.761621+0000","last_undegraded":"2024-05-27T07:31:33.761621+0000","last_fullsized":"2024-05-27T07:31:33.761621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:25.753276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.405878+0000","last_change":"2024-05-27T07:28:37.907849+0000","last_active":"2024-05-27T07:30:05.405878+0000","last_peered":"2024-05-27T07:30:05.405878+0000","last_clean":"2024-05-27T07:30:05.405878+0000","last_became_active":"2024-05-27T07:28:37.907105+0000","last_became_peered":"2024-05-27T07:28:37.907105+0000","last_unstale":"2024-05-27T07:30:05.405878+0000","last_undegraded":"2024-05-27T07:30:05.405878+0000","last_fullsized":"2024-05-27T07:30:05.405878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:54.075240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516795+0000","last_change":"2024-05-27T07:28:20.411302+0000","last_active":"2024-05-27T07:30:05.516795+0000","last_peered":"2024-05-27T07:30:05.516795+0000","last_clean":"2024-05-27T07:30:05.516795+0000","last_became_active":"2024-05-27T07:28:20.409144+0000","last_became_peered":"2024-05-27T07:28:20.409144+0000","last_unstale":"2024-05-27T07:30:05.516795+0000","last_undegraded":"2024-05-27T07:30:05.516795+0000","last_fullsized":"2024-05-27T07:30:05.516795+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:49.985962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.751489+0000","last_change":"2024-05-27T07:28:24.487742+0000","last_active":"2024-05-27T07:31:33.751489+0000","last_peered":"2024-05-27T07:31:33.751489+0000","last_clean":"2024-05-27T07:31:33.751489+0000","last_became_active":"2024-05-27T07:28:24.485279+0000","last_became_peered":"2024-05-27T07:28:24.485279+0000","last_unstale":"2024-05-27T07:31:33.751489+0000","last_undegraded":"2024-05-27T07:31:33.751489+0000","last_fullsized":"2024-05-27T07:31:33.751489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:15.049572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406726+0000","last_change":"2024-05-27T07:28:37.968780+0000","last_active":"2024-05-27T07:30:05.406726+0000","last_peered":"2024-05-27T07:30:05.406726+0000","last_clean":"2024-05-27T07:30:05.406726+0000","last_became_active":"2024-05-27T07:28:37.968052+0000","last_became_peered":"2024-05-27T07:28:37.968052+0000","last_unstale":"2024-05-27T07:30:05.406726+0000","last_undegraded":"2024-05-27T07:30:05.406726+0000","last_fullsized":"2024-05-27T07:30:05.406726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:51:26.787969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.753642+0000","last_change":"2024-05-27T07:28:24.485280+0000","last_active":"2024-05-27T07:31:33.753642+0000","last_peered":"2024-05-27T07:31:33.753642+0000","last_clean":"2024-05-27T07:31:33.753642+0000","last_became_active":"2024-05-27T07:28:24.484882+0000","last_became_peered":"2024-05-27T07:28:24.484882+0000","last_unstale":"2024-05-27T07:31:33.753642+0000","last_undegraded":"2024-05-27T07:31:33.753642+0000","last_fullsized":"2024-05-27T07:31:33.753642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:28.643857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:49.421106+0000","last_change":"2024-05-27T07:28:20.462593+0000","last_active":"2024-05-27T07:31:49.421106+0000","last_peered":"2024-05-27T07:31:49.421106+0000","last_clean":"2024-05-27T07:31:49.421106+0000","last_became_active":"2024-05-27T07:28:20.461128+0000","last_became_peered":"2024-05-27T07:28:20.461128+0000","last_unstale":"2024-05-27T07:31:49.421106+0000","last_undegraded":"2024-05-27T07:31:49.421106+0000","last_fullsized":"2024-05-27T07:31:49.421106+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:24:42.013315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516623+0000","last_change":"2024-05-27T07:28:37.911546+0000","last_active":"2024-05-27T07:30:05.516623+0000","last_peered":"2024-05-27T07:30:05.516623+0000","last_clean":"2024-05-27T07:30:05.516623+0000","last_became_active":"2024-05-27T07:28:37.910777+0000","last_became_peered":"2024-05-27T07:28:37.910777+0000","last_unstale":"2024-05-27T07:30:05.516623+0000","last_undegraded":"2024-05-27T07:30:05.516623+0000","last_fullsized":"2024-05-27T07:30:05.516623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:13.344317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.775196+0000","last_change":"2024-05-27T07:28:24.483946+0000","last_active":"2024-05-27T07:31:33.775196+0000","last_peered":"2024-05-27T07:31:33.775196+0000","last_clean":"2024-05-27T07:31:33.775196+0000","last_became_active":"2024-05-27T07:28:24.481424+0000","last_became_peered":"2024-05-27T07:28:24.481424+0000","last_unstale":"2024-05-27T07:31:33.775196+0000","last_undegraded":"2024-05-27T07:31:33.775196+0000","last_fullsized":"2024-05-27T07:31:33.775196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:08.254046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536932+0000","last_change":"2024-05-27T07:28:20.414250+0000","last_active":"2024-05-27T07:30:05.536932+0000","last_peered":"2024-05-27T07:30:05.536932+0000","last_clean":"2024-05-27T07:30:05.536932+0000","last_became_active":"2024-05-27T07:28:20.413676+0000","last_became_peered":"2024-05-27T07:28:20.413676+0000","last_unstale":"2024-05-27T07:30:05.536932+0000","last_undegraded":"2024-05-27T07:30:05.536932+0000","last_fullsized":"2024-05-27T07:30:05.536932+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:10.226771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515596+0000","last_change":"2024-05-27T07:26:26.055626+0000","last_active":"2024-05-27T07:30:05.515596+0000","last_peered":"2024-05-27T07:30:05.515596+0000","last_clean":"2024-05-27T07:30:05.515596+0000","last_became_active":"2024-05-27T07:26:26.037376+0000","last_became_peered":"2024-05-27T07:26:26.037376+0000","last_unstale":"2024-05-27T07:30:05.515596+0000","last_undegraded":"2024-05-27T07:30:05.515596+0000","last_fullsized":"2024-05-27T07:30:05.515596+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_clean_scrub_stamp":"2024-05-27T07:21:26.641949+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:16.741856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584417+0000","last_change":"2024-05-27T07:28:37.973078+0000","last_active":"2024-05-27T07:30:05.584417+0000","last_peered":"2024-05-27T07:30:05.584417+0000","last_clean":"2024-05-27T07:30:05.584417+0000","last_became_active":"2024-05-27T07:28:37.972381+0000","last_became_peered":"2024-05-27T07:28:37.972381+0000","last_unstale":"2024-05-27T07:30:05.584417+0000","last_undegraded":"2024-05-27T07:30:05.584417+0000","last_fullsized":"2024-05-27T07:30:05.584417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:18:16.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.759844+0000","last_change":"2024-05-27T07:28:24.483512+0000","last_active":"2024-05-27T07:31:33.759844+0000","last_peered":"2024-05-27T07:31:33.759844+0000","last_clean":"2024-05-27T07:31:33.759844+0000","last_became_active":"2024-05-27T07:28:24.482655+0000","last_became_peered":"2024-05-27T07:28:24.482655+0000","last_unstale":"2024-05-27T07:31:33.759844+0000","last_undegraded":"2024-05-27T07:31:33.759844+0000","last_fullsized":"2024-05-27T07:31:33.759844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:01:07.898504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515986+0000","last_change":"2024-05-27T07:28:20.408953+0000","last_active":"2024-05-27T07:30:05.515986+0000","last_peered":"2024-05-27T07:30:05.515986+0000","last_clean":"2024-05-27T07:30:05.515986+0000","last_became_active":"2024-05-27T07:28:20.407631+0000","last_became_peered":"2024-05-27T07:28:20.407631+0000","last_unstale":"2024-05-27T07:30:05.515986+0000","last_undegraded":"2024-05-27T07:30:05.515986+0000","last_fullsized":"2024-05-27T07:30:05.515986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:43.912338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580070+0000","last_change":"2024-05-27T07:28:37.907441+0000","last_active":"2024-05-27T07:30:05.580070+0000","last_peered":"2024-05-27T07:30:05.580070+0000","last_clean":"2024-05-27T07:30:05.580070+0000","last_became_active":"2024-05-27T07:28:37.906928+0000","last_became_peered":"2024-05-27T07:28:37.906928+0000","last_unstale":"2024-05-27T07:30:05.580070+0000","last_undegraded":"2024-05-27T07:30:05.580070+0000","last_fullsized":"2024-05-27T07:30:05.580070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:28:32.345826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584754+0000","last_change":"2024-05-27T07:28:20.467408+0000","last_active":"2024-05-27T07:30:05.584754+0000","last_peered":"2024-05-27T07:30:05.584754+0000","last_clean":"2024-05-27T07:30:05.584754+0000","last_became_active":"2024-05-27T07:28:20.466064+0000","last_became_peered":"2024-05-27T07:28:20.466064+0000","last_unstale":"2024-05-27T07:30:05.584754+0000","last_undegraded":"2024-05-27T07:30:05.584754+0000","last_fullsized":"2024-05-27T07:30:05.584754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:15:51.449514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.767788+0000","last_change":"2024-05-27T07:28:24.478475+0000","last_active":"2024-05-27T07:31:33.767788+0000","last_peered":"2024-05-27T07:31:33.767788+0000","last_clean":"2024-05-27T07:31:33.767788+0000","last_became_active":"2024-05-27T07:28:24.477936+0000","last_became_peered":"2024-05-27T07:28:24.477936+0000","last_unstale":"2024-05-27T07:31:33.767788+0000","last_undegraded":"2024-05-27T07:31:33.767788+0000","last_fullsized":"2024-05-27T07:31:33.767788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:53:17.500040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119717+0000","last_change":"2024-05-27T07:28:37.968192+0000","last_active":"2024-05-27T07:30:05.119717+0000","last_peered":"2024-05-27T07:30:05.119717+0000","last_clean":"2024-05-27T07:30:05.119717+0000","last_became_active":"2024-05-27T07:28:37.967603+0000","last_became_peered":"2024-05-27T07:28:37.967603+0000","last_unstale":"2024-05-27T07:30:05.119717+0000","last_undegraded":"2024-05-27T07:30:05.119717+0000","last_fullsized":"2024-05-27T07:30:05.119717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:53:36.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":67,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:31.459304+0000","last_change":"2024-05-27T07:29:04.901062+0000","last_active":"2024-05-27T07:30:31.459304+0000","last_peered":"2024-05-27T07:30:31.459304+0000","last_clean":"2024-05-27T07:30:31.459304+0000","last_became_active":"2024-05-27T07:29:04.892060+0000","last_became_peered":"2024-05-27T07:29:04.892060+0000","last_unstale":"2024-05-27T07:30:31.459304+0000","last_undegraded":"2024-05-27T07:30:31.459304+0000","last_fullsized":"2024-05-27T07:30:31.459304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:27:37.894263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.778506+0000","last_change":"2024-05-27T07:28:24.477096+0000","last_active":"2024-05-27T07:31:33.778506+0000","last_peered":"2024-05-27T07:31:33.778506+0000","last_clean":"2024-05-27T07:31:33.778506+0000","last_became_active":"2024-05-27T07:28:24.476712+0000","last_became_peered":"2024-05-27T07:28:24.476712+0000","last_unstale":"2024-05-27T07:31:33.778506+0000","last_undegraded":"2024-05-27T07:31:33.778506+0000","last_fullsized":"2024-05-27T07:31:33.778506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:31:58.010761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119351+0000","last_change":"2024-05-27T07:28:37.894013+0000","last_active":"2024-05-27T07:30:05.119351+0000","last_peered":"2024-05-27T07:30:05.119351+0000","last_clean":"2024-05-27T07:30:05.119351+0000","last_became_active":"2024-05-27T07:28:37.893345+0000","last_became_peered":"2024-05-27T07:28:37.893345+0000","last_unstale":"2024-05-27T07:30:05.119351+0000","last_undegraded":"2024-05-27T07:30:05.119351+0000","last_fullsized":"2024-05-27T07:30:05.119351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:26.917731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537802+0000","last_change":"2024-05-27T07:28:20.744087+0000","last_active":"2024-05-27T07:30:05.537802+0000","last_peered":"2024-05-27T07:30:05.537802+0000","last_clean":"2024-05-27T07:30:05.537802+0000","last_became_active":"2024-05-27T07:28:20.742883+0000","last_became_peered":"2024-05-27T07:28:20.742883+0000","last_unstale":"2024-05-27T07:30:05.537802+0000","last_undegraded":"2024-05-27T07:30:05.537802+0000","last_fullsized":"2024-05-27T07:30:05.537802+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:06.083348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776817+0000","last_change":"2024-05-27T07:28:24.480330+0000","last_active":"2024-05-27T07:31:33.776817+0000","last_peered":"2024-05-27T07:31:33.776817+0000","last_clean":"2024-05-27T07:31:33.776817+0000","last_became_active":"2024-05-27T07:28:24.479300+0000","last_became_peered":"2024-05-27T07:28:24.479300+0000","last_unstale":"2024-05-27T07:31:33.776817+0000","last_undegraded":"2024-05-27T07:31:33.776817+0000","last_fullsized":"2024-05-27T07:31:33.776817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:43.802273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537207+0000","last_change":"2024-05-27T07:28:37.973592+0000","last_active":"2024-05-27T07:30:05.537207+0000","last_peered":"2024-05-27T07:30:05.537207+0000","last_clean":"2024-05-27T07:30:05.537207+0000","last_became_active":"2024-05-27T07:28:37.972475+0000","last_became_peered":"2024-05-27T07:28:37.972475+0000","last_unstale":"2024-05-27T07:30:05.537207+0000","last_undegraded":"2024-05-27T07:30:05.537207+0000","last_fullsized":"2024-05-27T07:30:05.537207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:31.662384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122331+0000","last_change":"2024-05-27T07:29:04.890751+0000","last_active":"2024-05-27T07:30:05.122331+0000","last_peered":"2024-05-27T07:30:05.122331+0000","last_clean":"2024-05-27T07:30:05.122331+0000","last_became_active":"2024-05-27T07:29:04.889447+0000","last_became_peered":"2024-05-27T07:29:04.889447+0000","last_unstale":"2024-05-27T07:30:05.122331+0000","last_undegraded":"2024-05-27T07:30:05.122331+0000","last_fullsized":"2024-05-27T07:30:05.122331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:13:34.743691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.703604+0000","last_change":"2024-05-27T07:28:24.474298+0000","last_active":"2024-05-27T07:31:33.703604+0000","last_peered":"2024-05-27T07:31:33.703604+0000","last_clean":"2024-05-27T07:31:33.703604+0000","last_became_active":"2024-05-27T07:28:24.473877+0000","last_became_peered":"2024-05-27T07:28:24.473877+0000","last_unstale":"2024-05-27T07:31:33.703604+0000","last_undegraded":"2024-05-27T07:31:33.703604+0000","last_fullsized":"2024-05-27T07:31:33.703604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:31.545883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127532+0000","last_change":"2024-05-27T07:28:37.905417+0000","last_active":"2024-05-27T07:30:05.127532+0000","last_peered":"2024-05-27T07:30:05.127532+0000","last_clean":"2024-05-27T07:30:05.127532+0000","last_became_active":"2024-05-27T07:28:37.904968+0000","last_became_peered":"2024-05-27T07:28:37.904968+0000","last_unstale":"2024-05-27T07:30:05.127532+0000","last_undegraded":"2024-05-27T07:30:05.127532+0000","last_fullsized":"2024-05-27T07:30:05.127532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:02.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120278+0000","last_change":"2024-05-27T07:28:20.392647+0000","last_active":"2024-05-27T07:30:05.120278+0000","last_peered":"2024-05-27T07:30:05.120278+0000","last_clean":"2024-05-27T07:30:05.120278+0000","last_became_active":"2024-05-27T07:28:20.392073+0000","last_became_peered":"2024-05-27T07:28:20.392073+0000","last_unstale":"2024-05-27T07:30:05.120278+0000","last_undegraded":"2024-05-27T07:30:05.120278+0000","last_fullsized":"2024-05-27T07:30:05.120278+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:18.777628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.757595+0000","last_change":"2024-05-27T07:28:24.490522+0000","last_active":"2024-05-27T07:31:33.757595+0000","last_peered":"2024-05-27T07:31:33.757595+0000","last_clean":"2024-05-27T07:31:33.757595+0000","last_became_active":"2024-05-27T07:28:24.485228+0000","last_became_peered":"2024-05-27T07:28:24.485228+0000","last_unstale":"2024-05-27T07:31:33.757595+0000","last_undegraded":"2024-05-27T07:31:33.757595+0000","last_fullsized":"2024-05-27T07:31:33.757595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:14.221016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:13.384143+0000","last_change":"2024-05-27T07:28:37.900449+0000","last_active":"2024-05-27T07:30:13.384143+0000","last_peered":"2024-05-27T07:30:13.384143+0000","last_clean":"2024-05-27T07:30:13.384143+0000","last_became_active":"2024-05-27T07:28:37.899526+0000","last_became_peered":"2024-05-27T07:28:37.899526+0000","last_unstale":"2024-05-27T07:30:13.384143+0000","last_undegraded":"2024-05-27T07:30:13.384143+0000","last_fullsized":"2024-05-27T07:30:13.384143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:29:49.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119900+0000","last_change":"2024-05-27T07:28:20.405236+0000","last_active":"2024-05-27T07:30:05.119900+0000","last_peered":"2024-05-27T07:30:05.119900+0000","last_clean":"2024-05-27T07:30:05.119900+0000","last_became_active":"2024-05-27T07:28:20.404619+0000","last_became_peered":"2024-05-27T07:28:20.404619+0000","last_unstale":"2024-05-27T07:30:05.119900+0000","last_undegraded":"2024-05-27T07:30:05.119900+0000","last_fullsized":"2024-05-27T07:30:05.119900+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:17.671250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.762832+0000","last_change":"2024-05-27T07:28:24.490661+0000","last_active":"2024-05-27T07:31:33.762832+0000","last_peered":"2024-05-27T07:31:33.762832+0000","last_clean":"2024-05-27T07:31:33.762832+0000","last_became_active":"2024-05-27T07:28:24.487299+0000","last_became_peered":"2024-05-27T07:28:24.487299+0000","last_unstale":"2024-05-27T07:31:33.762832+0000","last_undegraded":"2024-05-27T07:31:33.762832+0000","last_fullsized":"2024-05-27T07:31:33.762832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:50:27.291721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537106+0000","last_change":"2024-05-27T07:28:37.970463+0000","last_active":"2024-05-27T07:30:05.537106+0000","last_peered":"2024-05-27T07:30:05.537106+0000","last_clean":"2024-05-27T07:30:05.537106+0000","last_became_active":"2024-05-27T07:28:37.968068+0000","last_became_peered":"2024-05-27T07:28:37.968068+0000","last_unstale":"2024-05-27T07:30:05.537106+0000","last_undegraded":"2024-05-27T07:30:05.537106+0000","last_fullsized":"2024-05-27T07:30:05.537106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:22:00.825999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580250+0000","last_change":"2024-05-27T07:28:20.406203+0000","last_active":"2024-05-27T07:30:05.580250+0000","last_peered":"2024-05-27T07:30:05.580250+0000","last_clean":"2024-05-27T07:30:05.580250+0000","last_became_active":"2024-05-27T07:28:20.405404+0000","last_became_peered":"2024-05-27T07:28:20.405404+0000","last_unstale":"2024-05-27T07:30:05.580250+0000","last_undegraded":"2024-05-27T07:30:05.580250+0000","last_fullsized":"2024-05-27T07:30:05.580250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:15.274419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.767110+0000","last_change":"2024-05-27T07:28:24.473747+0000","last_active":"2024-05-27T07:31:33.767110+0000","last_peered":"2024-05-27T07:31:33.767110+0000","last_clean":"2024-05-27T07:31:33.767110+0000","last_became_active":"2024-05-27T07:28:24.473313+0000","last_became_peered":"2024-05-27T07:28:24.473313+0000","last_unstale":"2024-05-27T07:31:33.767110+0000","last_undegraded":"2024-05-27T07:31:33.767110+0000","last_fullsized":"2024-05-27T07:31:33.767110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:19:40.107840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122107+0000","last_change":"2024-05-27T07:28:37.907410+0000","last_active":"2024-05-27T07:30:05.122107+0000","last_peered":"2024-05-27T07:30:05.122107+0000","last_clean":"2024-05-27T07:30:05.122107+0000","last_became_active":"2024-05-27T07:28:37.906753+0000","last_became_peered":"2024-05-27T07:28:37.906753+0000","last_unstale":"2024-05-27T07:30:05.122107+0000","last_undegraded":"2024-05-27T07:30:05.122107+0000","last_fullsized":"2024-05-27T07:30:05.122107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:53.612015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120654+0000","last_change":"2024-05-27T07:28:20.744068+0000","last_active":"2024-05-27T07:30:05.120654+0000","last_peered":"2024-05-27T07:30:05.120654+0000","last_clean":"2024-05-27T07:30:05.120654+0000","last_became_active":"2024-05-27T07:28:20.742686+0000","last_became_peered":"2024-05-27T07:28:20.742686+0000","last_unstale":"2024-05-27T07:30:05.120654+0000","last_undegraded":"2024-05-27T07:30:05.120654+0000","last_fullsized":"2024-05-27T07:30:05.120654+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:54.361007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776298+0000","last_change":"2024-05-27T07:28:24.475743+0000","last_active":"2024-05-27T07:31:33.776298+0000","last_peered":"2024-05-27T07:31:33.776298+0000","last_clean":"2024-05-27T07:31:33.776298+0000","last_became_active":"2024-05-27T07:28:24.474742+0000","last_became_peered":"2024-05-27T07:28:24.474742+0000","last_unstale":"2024-05-27T07:31:33.776298+0000","last_undegraded":"2024-05-27T07:31:33.776298+0000","last_fullsized":"2024-05-27T07:31:33.776298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:58:47.061573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576290+0000","last_change":"2024-05-27T07:28:37.914935+0000","last_active":"2024-05-27T07:30:05.576290+0000","last_peered":"2024-05-27T07:30:05.576290+0000","last_clean":"2024-05-27T07:30:05.576290+0000","last_became_active":"2024-05-27T07:28:37.911829+0000","last_became_peered":"2024-05-27T07:28:37.911829+0000","last_unstale":"2024-05-27T07:30:05.576290+0000","last_undegraded":"2024-05-27T07:30:05.576290+0000","last_fullsized":"2024-05-27T07:30:05.576290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:29.260539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584461+0000","last_change":"2024-05-27T07:28:20.412554+0000","last_active":"2024-05-27T07:30:05.584461+0000","last_peered":"2024-05-27T07:30:05.584461+0000","last_clean":"2024-05-27T07:30:05.584461+0000","last_became_active":"2024-05-27T07:28:20.412024+0000","last_became_peered":"2024-05-27T07:28:20.412024+0000","last_unstale":"2024-05-27T07:30:05.584461+0000","last_undegraded":"2024-05-27T07:30:05.584461+0000","last_fullsized":"2024-05-27T07:30:05.584461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:29.244093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.778000+0000","last_change":"2024-05-27T07:28:24.481966+0000","last_active":"2024-05-27T07:31:33.778000+0000","last_peered":"2024-05-27T07:31:33.778000+0000","last_clean":"2024-05-27T07:31:33.778000+0000","last_became_active":"2024-05-27T07:28:24.480775+0000","last_became_peered":"2024-05-27T07:28:24.480775+0000","last_unstale":"2024-05-27T07:31:33.778000+0000","last_undegraded":"2024-05-27T07:31:33.778000+0000","last_fullsized":"2024-05-27T07:31:33.778000+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:03.018174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578887+0000","last_change":"2024-05-27T07:28:37.902893+0000","last_active":"2024-05-27T07:30:05.578887+0000","last_peered":"2024-05-27T07:30:05.578887+0000","last_clean":"2024-05-27T07:30:05.578887+0000","last_became_active":"2024-05-27T07:28:37.898447+0000","last_became_peered":"2024-05-27T07:28:37.898447+0000","last_unstale":"2024-05-27T07:30:05.578887+0000","last_undegraded":"2024-05-27T07:30:05.578887+0000","last_fullsized":"2024-05-27T07:30:05.578887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:55:49.754803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.407029+0000","last_change":"2024-05-27T07:28:20.408072+0000","last_active":"2024-05-27T07:30:05.407029+0000","last_peered":"2024-05-27T07:30:05.407029+0000","last_clean":"2024-05-27T07:30:05.407029+0000","last_became_active":"2024-05-27T07:28:20.407174+0000","last_became_peered":"2024-05-27T07:28:20.407174+0000","last_unstale":"2024-05-27T07:30:05.407029+0000","last_undegraded":"2024-05-27T07:30:05.407029+0000","last_fullsized":"2024-05-27T07:30:05.407029+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:15:52.984967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.761250+0000","last_change":"2024-05-27T07:28:24.475891+0000","last_active":"2024-05-27T07:31:33.761250+0000","last_peered":"2024-05-27T07:31:33.761250+0000","last_clean":"2024-05-27T07:31:33.761250+0000","last_became_active":"2024-05-27T07:28:24.475498+0000","last_became_peered":"2024-05-27T07:28:24.475498+0000","last_unstale":"2024-05-27T07:31:33.761250+0000","last_undegraded":"2024-05-27T07:31:33.761250+0000","last_fullsized":"2024-05-27T07:31:33.761250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:50.669118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.761917+0000","last_change":"2024-05-27T07:29:04.891583+0000","last_active":"2024-05-27T07:31:33.761917+0000","last_peered":"2024-05-27T07:31:33.761917+0000","last_clean":"2024-05-27T07:31:33.761917+0000","last_became_active":"2024-05-27T07:29:04.890187+0000","last_became_peered":"2024-05-27T07:29:04.890187+0000","last_unstale":"2024-05-27T07:31:33.761917+0000","last_undegraded":"2024-05-27T07:31:33.761917+0000","last_fullsized":"2024-05-27T07:31:33.761917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:03.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.753001+0000","last_change":"2024-05-27T07:28:24.478672+0000","last_active":"2024-05-27T07:31:33.753001+0000","last_peered":"2024-05-27T07:31:33.753001+0000","last_clean":"2024-05-27T07:31:33.753001+0000","last_became_active":"2024-05-27T07:28:24.478331+0000","last_became_peered":"2024-05-27T07:28:24.478331+0000","last_unstale":"2024-05-27T07:31:33.753001+0000","last_undegraded":"2024-05-27T07:31:33.753001+0000","last_fullsized":"2024-05-27T07:31:33.753001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:56:35.112855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.775323+0000","last_change":"2024-05-27T07:28:24.487962+0000","last_active":"2024-05-27T07:31:33.775323+0000","last_peered":"2024-05-27T07:31:33.775323+0000","last_clean":"2024-05-27T07:31:33.775323+0000","last_became_active":"2024-05-27T07:28:24.487583+0000","last_became_peered":"2024-05-27T07:28:24.487583+0000","last_unstale":"2024-05-27T07:31:33.775323+0000","last_undegraded":"2024-05-27T07:31:33.775323+0000","last_fullsized":"2024-05-27T07:31:33.775323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:25.758293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.765769+0000","last_change":"2024-05-27T07:28:24.466058+0000","last_active":"2024-05-27T07:31:33.765769+0000","last_peered":"2024-05-27T07:31:33.765769+0000","last_clean":"2024-05-27T07:31:33.765769+0000","last_became_active":"2024-05-27T07:28:24.465709+0000","last_became_peered":"2024-05-27T07:28:24.465709+0000","last_unstale":"2024-05-27T07:31:33.765769+0000","last_undegraded":"2024-05-27T07:31:33.765769+0000","last_fullsized":"2024-05-27T07:31:33.765769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:38.064077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.765176+0000","last_change":"2024-05-27T07:28:24.471585+0000","last_active":"2024-05-27T07:31:33.765176+0000","last_peered":"2024-05-27T07:31:33.765176+0000","last_clean":"2024-05-27T07:31:33.765176+0000","last_became_active":"2024-05-27T07:28:24.470948+0000","last_became_peered":"2024-05-27T07:28:24.470948+0000","last_unstale":"2024-05-27T07:31:33.765176+0000","last_undegraded":"2024-05-27T07:31:33.765176+0000","last_fullsized":"2024-05-27T07:31:33.765176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:48.213323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.745710+0000","last_change":"2024-05-27T07:28:24.477954+0000","last_active":"2024-05-27T07:31:33.745710+0000","last_peered":"2024-05-27T07:31:33.745710+0000","last_clean":"2024-05-27T07:31:33.745710+0000","last_became_active":"2024-05-27T07:28:24.477495+0000","last_became_peered":"2024-05-27T07:28:24.477495+0000","last_unstale":"2024-05-27T07:31:33.745710+0000","last_undegraded":"2024-05-27T07:31:33.745710+0000","last_fullsized":"2024-05-27T07:31:33.745710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:50.371439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.769895+0000","last_change":"2024-05-27T07:28:24.476422+0000","last_active":"2024-05-27T07:31:33.769895+0000","last_peered":"2024-05-27T07:31:33.769895+0000","last_clean":"2024-05-27T07:31:33.769895+0000","last_became_active":"2024-05-27T07:28:24.476095+0000","last_became_peered":"2024-05-27T07:28:24.476095+0000","last_unstale":"2024-05-27T07:31:33.769895+0000","last_undegraded":"2024-05-27T07:31:33.769895+0000","last_fullsized":"2024-05-27T07:31:33.769895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:51.667169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.774521+0000","last_change":"2024-05-27T07:28:24.486431+0000","last_active":"2024-05-27T07:31:33.774521+0000","last_peered":"2024-05-27T07:31:33.774521+0000","last_clean":"2024-05-27T07:31:33.774521+0000","last_became_active":"2024-05-27T07:28:24.482427+0000","last_became_peered":"2024-05-27T07:28:24.482427+0000","last_unstale":"2024-05-27T07:31:33.774521+0000","last_undegraded":"2024-05-27T07:31:33.774521+0000","last_fullsized":"2024-05-27T07:31:33.774521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:44:37.857793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.775665+0000","last_change":"2024-05-27T07:28:24.484393+0000","last_active":"2024-05-27T07:31:33.775665+0000","last_peered":"2024-05-27T07:31:33.775665+0000","last_clean":"2024-05-27T07:31:33.775665+0000","last_became_active":"2024-05-27T07:28:24.483130+0000","last_became_peered":"2024-05-27T07:28:24.483130+0000","last_unstale":"2024-05-27T07:31:33.775665+0000","last_undegraded":"2024-05-27T07:31:33.775665+0000","last_fullsized":"2024-05-27T07:31:33.775665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:34.924573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.779162+0000","last_change":"2024-05-27T07:28:24.486274+0000","last_active":"2024-05-27T07:31:33.779162+0000","last_peered":"2024-05-27T07:31:33.779162+0000","last_clean":"2024-05-27T07:31:33.779162+0000","last_became_active":"2024-05-27T07:28:24.485880+0000","last_became_peered":"2024-05-27T07:28:24.485880+0000","last_unstale":"2024-05-27T07:31:33.779162+0000","last_undegraded":"2024-05-27T07:31:33.779162+0000","last_fullsized":"2024-05-27T07:31:33.779162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:33.287615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.779769+0000","last_change":"2024-05-27T07:28:24.477909+0000","last_active":"2024-05-27T07:31:33.779769+0000","last_peered":"2024-05-27T07:31:33.779769+0000","last_clean":"2024-05-27T07:31:33.779769+0000","last_became_active":"2024-05-27T07:28:24.477345+0000","last_became_peered":"2024-05-27T07:28:24.477345+0000","last_unstale":"2024-05-27T07:31:33.779769+0000","last_undegraded":"2024-05-27T07:31:33.779769+0000","last_fullsized":"2024-05-27T07:31:33.779769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:26:03.958489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.775475+0000","last_change":"2024-05-27T07:28:24.477151+0000","last_active":"2024-05-27T07:31:33.775475+0000","last_peered":"2024-05-27T07:31:33.775475+0000","last_clean":"2024-05-27T07:31:33.775475+0000","last_became_active":"2024-05-27T07:28:24.474036+0000","last_became_peered":"2024-05-27T07:28:24.474036+0000","last_unstale":"2024-05-27T07:31:33.775475+0000","last_undegraded":"2024-05-27T07:31:33.775475+0000","last_fullsized":"2024-05-27T07:31:33.775475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:16.787252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.757111+0000","last_change":"2024-05-27T07:28:24.483601+0000","last_active":"2024-05-27T07:31:33.757111+0000","last_peered":"2024-05-27T07:31:33.757111+0000","last_clean":"2024-05-27T07:31:33.757111+0000","last_became_active":"2024-05-27T07:28:24.481080+0000","last_became_peered":"2024-05-27T07:28:24.481080+0000","last_unstale":"2024-05-27T07:31:33.757111+0000","last_undegraded":"2024-05-27T07:31:33.757111+0000","last_fullsized":"2024-05-27T07:31:33.757111+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:12.208506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72816,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":732,"num_read_kb":443,"num_write":176,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":218448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645373,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":522520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841603,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1740800,"data_stored":1178994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005132,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":526852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201364,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":522676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397596,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32792,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712360,"statfs":{"total":95995035648,"available":95961456640,"internally_reserved":0,"allocated":1654784,"data_stored":1118194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561122,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":517932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":587048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738510,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":519320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":588398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149817,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":518126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953585,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757355,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1113782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:31:55.731 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T07:31:56.258 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:56.407 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:56 smithi063 ceph-mon[31613]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:31:56.407 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:56 smithi063 ceph-mon[31613]: pgmap v497: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:56.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:56 smithi086 ceph-mon[34687]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:31:56.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:56 smithi086 ceph-mon[34687]: pgmap v497: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:56.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:56 smithi149 ceph-mon[35408]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:31:56.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:56 smithi149 ceph-mon[35408]: pgmap v497: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:31:58.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.592+0000 7f6991400700 1 -- 172.21.15.63:0/1092964318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f698c1151e0 msgr2=0x7f698c117600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:58.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.592+0000 7f6991400700 1 --2- 172.21.15.63:0/1092964318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f698c1151e0 0x7f698c117600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f698400b0a0 tx=0x7f698400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:31:58.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.592+0000 7f6991400700 1 -- 172.21.15.63:0/1092964318 shutdown_connections 2024-05-27T07:31:58.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.592+0000 7f6991400700 1 --2- 172.21.15.63:0/1092964318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f698c1151e0 0x7f698c117600 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:58.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.592+0000 7f6991400700 1 --2- 172.21.15.63:0/1092964318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c077ce0 0x7f698c0780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:58.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.592+0000 7f6991400700 1 --2- 172.21.15.63:0/1092964318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f698c078630 0x7f698c114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:58.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.592+0000 7f6991400700 1 -- 172.21.15.63:0/1092964318 >> 172.21.15.63:0/1092964318 conn(0x7f698c102080 msgr2=0x7f698c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:58.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.593+0000 7f6991400700 1 -- 172.21.15.63:0/1092964318 shutdown_connections 2024-05-27T07:31:58.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.593+0000 7f6991400700 1 -- 172.21.15.63:0/1092964318 wait complete. 2024-05-27T07:31:58.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.594+0000 7f6991400700 1 Processor -- start 2024-05-27T07:31:58.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.594+0000 7f6991400700 1 -- start start 2024-05-27T07:31:58.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.594+0000 7f6991400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f698c077ce0 0x7f698c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:58.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.594+0000 7f6991400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f698c078630 0x7f698c1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:58.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.594+0000 7f6991400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c1151e0 0x7f698c1a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:58.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.594+0000 7f6991400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f698c11a970 con 0x7f698c078630 2024-05-27T07:31:58.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.594+0000 7f6991400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f698c11a7f0 con 0x7f698c1151e0 2024-05-27T07:31:58.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.594+0000 7f6991400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f698c11aaf0 con 0x7f698c077ce0 2024-05-27T07:31:58.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.595+0000 7f6989e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f698c078630 0x7f698c1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:58.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.595+0000 7f6989e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f698c078630 0x7f698c1a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60602/0 (socket says 172.21.15.63:60602) 2024-05-27T07:31:58.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.595+0000 7f6989e00700 1 -- 172.21.15.63:0/153738878 learned_addr learned my addr 172.21.15.63:0/153738878 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:31:58.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.595+0000 7f698a800700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f698c077ce0 0x7f698c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:58.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.595+0000 7f698b200700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c1151e0 0x7f698c1a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:58.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.596+0000 7f6989e00700 1 -- 172.21.15.63:0/153738878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f698c077ce0 msgr2=0x7f698c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:58.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.596+0000 7f6989e00700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f698c077ce0 0x7f698c1a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:58.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.596+0000 7f6989e00700 1 -- 172.21.15.63:0/153738878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c1151e0 msgr2=0x7f698c1a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:58.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.596+0000 7f6989e00700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c1151e0 0x7f698c1a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:58.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.596+0000 7f6989e00700 1 -- 172.21.15.63:0/153738878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c1a61f0 con 0x7f698c078630 2024-05-27T07:31:58.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.596+0000 7f6989e00700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f698c078630 0x7f698c1a18c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f697800d980 tx=0x7f697800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:58.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.597+0000 7f6983400700 1 -- 172.21.15.63:0/153738878 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978017070 con 0x7f698c078630 2024-05-27T07:31:58.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.597+0000 7f6983400700 1 -- 172.21.15.63:0/153738878 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f697800fd20 con 0x7f698c078630 2024-05-27T07:31:58.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.597+0000 7f6991400700 1 -- 172.21.15.63:0/153738878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c1a6480 con 0x7f698c078630 2024-05-27T07:31:58.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.597+0000 7f6983400700 1 -- 172.21.15.63:0/153738878 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697800e3b0 con 0x7f698c078630 2024-05-27T07:31:58.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.597+0000 7f6991400700 1 -- 172.21.15.63:0/153738878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c1ade00 con 0x7f698c078630 2024-05-27T07:31:58.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.598+0000 7f6991400700 1 -- 172.21.15.63:0/153738878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c0512b0 con 0x7f698c078630 2024-05-27T07:31:58.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.604+0000 7f6983400700 1 -- 172.21.15.63:0/153738878 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f697800f890 con 0x7f698c078630 2024-05-27T07:31:58.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.606+0000 7f6983400700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69700b2880 0x7f69700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:58.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.606+0000 7f6983400700 1 -- 172.21.15.63:0/153738878 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f69780e1fe0 con 0x7f698c078630 2024-05-27T07:31:58.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.606+0000 7f698a800700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69700b2880 0x7f69700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:58.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.607+0000 7f698a800700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69700b2880 0x7f69700b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f697c005950 tx=0x7f697c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:58.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.608+0000 7f6983400700 1 -- 172.21.15.63:0/153738878 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69780a9930 con 0x7f698c078630 2024-05-27T07:31:58.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.845+0000 7f6991400700 1 -- 172.21.15.63:0/153738878 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f698c1a2680 con 0x7f69700b2880 2024-05-27T07:31:58.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.874+0000 7f6983400700 1 -- 172.21.15.63:0/153738878 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340470 (secure 0 0 0) 0x7f698c1a2680 con 0x7f69700b2880 2024-05-27T07:31:58.873 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:31:58.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 -- 172.21.15.63:0/153738878 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69700b2880 msgr2=0x7f69700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69700b2880 0x7f69700b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f697c005950 tx=0x7f697c00b410 comp rx=0 tx=0).stop 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 -- 172.21.15.63:0/153738878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f698c078630 msgr2=0x7f698c1a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f698c078630 0x7f698c1a18c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f697800d980 tx=0x7f697800dc90 comp rx=0 tx=0).stop 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 -- 172.21.15.63:0/153738878 shutdown_connections 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f698c1151e0 0x7f698c1a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f698c077ce0 0x7f698c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f69700b2880 0x7f69700b4d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 --2- 172.21.15.63:0/153738878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f698c078630 0x7f698c1a18c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 -- 172.21.15.63:0/153738878 >> 172.21.15.63:0/153738878 conn(0x7f698c102080 msgr2=0x7f698c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.882+0000 7f6991400700 1 -- 172.21.15.63:0/153738878 shutdown_connections 2024-05-27T07:31:58.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:31:58.883+0000 7f6991400700 1 -- 172.21.15.63:0/153738878 wait complete. 2024-05-27T07:31:58.877 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-27T07:31:59.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:58 smithi086 ceph-mon[34687]: pgmap v498: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:59.205 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:58 smithi149 ceph-mon[35408]: pgmap v498: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:59.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:58 smithi063 ceph-mon[31613]: pgmap v498: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:31:59.663 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-05-27T07:31:57.881664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663418,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":843,"num_read_kb":527,"num_write":301,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":744,"ondisk_log_size":744,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375980,"kb_used_data":14348,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565844,"statfs":{"total":1151940427776,"available":1151555424256,"internally_reserved":0,"allocated":14692352,"data_stored":8236424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006149"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776902+0000","last_change":"2024-05-27T07:28:24.485151+0000","last_active":"2024-05-27T07:31:33.776902+0000","last_peered":"2024-05-27T07:31:33.776902+0000","last_clean":"2024-05-27T07:31:33.776902+0000","last_became_active":"2024-05-27T07:28:24.483965+0000","last_became_peered":"2024-05-27T07:28:24.483965+0000","last_unstale":"2024-05-27T07:31:33.776902+0000","last_undegraded":"2024-05-27T07:31:33.776902+0000","last_fullsized":"2024-05-27T07:31:33.776902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:18.376755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.752840+0000","last_change":"2024-05-27T07:28:24.483608+0000","last_active":"2024-05-27T07:31:33.752840+0000","last_peered":"2024-05-27T07:31:33.752840+0000","last_clean":"2024-05-27T07:31:33.752840+0000","last_became_active":"2024-05-27T07:28:24.479875+0000","last_became_peered":"2024-05-27T07:28:24.479875+0000","last_unstale":"2024-05-27T07:31:33.752840+0000","last_undegraded":"2024-05-27T07:31:33.752840+0000","last_fullsized":"2024-05-27T07:31:33.752840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:30.129708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.770675+0000","last_change":"2024-05-27T07:28:24.470104+0000","last_active":"2024-05-27T07:31:33.770675+0000","last_peered":"2024-05-27T07:31:33.770675+0000","last_clean":"2024-05-27T07:31:33.770675+0000","last_became_active":"2024-05-27T07:28:24.469756+0000","last_became_peered":"2024-05-27T07:28:24.469756+0000","last_unstale":"2024-05-27T07:31:33.770675+0000","last_undegraded":"2024-05-27T07:31:33.770675+0000","last_fullsized":"2024-05-27T07:31:33.770675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:36.907691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.747804+0000","last_change":"2024-05-27T07:28:24.472381+0000","last_active":"2024-05-27T07:31:33.747804+0000","last_peered":"2024-05-27T07:31:33.747804+0000","last_clean":"2024-05-27T07:31:33.747804+0000","last_became_active":"2024-05-27T07:28:24.469959+0000","last_became_peered":"2024-05-27T07:28:24.469959+0000","last_unstale":"2024-05-27T07:31:33.747804+0000","last_undegraded":"2024-05-27T07:31:33.747804+0000","last_fullsized":"2024-05-27T07:31:33.747804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:15.267554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776719+0000","last_change":"2024-05-27T07:28:24.474210+0000","last_active":"2024-05-27T07:31:33.776719+0000","last_peered":"2024-05-27T07:31:33.776719+0000","last_clean":"2024-05-27T07:31:33.776719+0000","last_became_active":"2024-05-27T07:28:24.473688+0000","last_became_peered":"2024-05-27T07:28:24.473688+0000","last_unstale":"2024-05-27T07:31:33.776719+0000","last_undegraded":"2024-05-27T07:31:33.776719+0000","last_fullsized":"2024-05-27T07:31:33.776719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:38.733536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.760411+0000","last_change":"2024-05-27T07:28:24.484679+0000","last_active":"2024-05-27T07:31:33.760411+0000","last_peered":"2024-05-27T07:31:33.760411+0000","last_clean":"2024-05-27T07:31:33.760411+0000","last_became_active":"2024-05-27T07:28:24.484251+0000","last_became_peered":"2024-05-27T07:28:24.484251+0000","last_unstale":"2024-05-27T07:31:33.760411+0000","last_undegraded":"2024-05-27T07:31:33.760411+0000","last_fullsized":"2024-05-27T07:31:33.760411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:28:09.507871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.759926+0000","last_change":"2024-05-27T07:28:24.472694+0000","last_active":"2024-05-27T07:31:33.759926+0000","last_peered":"2024-05-27T07:31:33.759926+0000","last_clean":"2024-05-27T07:31:33.759926+0000","last_became_active":"2024-05-27T07:28:24.472100+0000","last_became_peered":"2024-05-27T07:28:24.472100+0000","last_unstale":"2024-05-27T07:31:33.759926+0000","last_undegraded":"2024-05-27T07:31:33.759926+0000","last_fullsized":"2024-05-27T07:31:33.759926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.777956+0000","last_change":"2024-05-27T07:28:24.488577+0000","last_active":"2024-05-27T07:31:33.777956+0000","last_peered":"2024-05-27T07:31:33.777956+0000","last_clean":"2024-05-27T07:31:33.777956+0000","last_became_active":"2024-05-27T07:28:24.488180+0000","last_became_peered":"2024-05-27T07:28:24.488180+0000","last_unstale":"2024-05-27T07:31:33.777956+0000","last_undegraded":"2024-05-27T07:31:33.777956+0000","last_fullsized":"2024-05-27T07:31:33.777956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:17.130058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.781727+0000","last_change":"2024-05-27T07:28:24.480109+0000","last_active":"2024-05-27T07:31:33.781727+0000","last_peered":"2024-05-27T07:31:33.781727+0000","last_clean":"2024-05-27T07:31:33.781727+0000","last_became_active":"2024-05-27T07:28:24.479722+0000","last_became_peered":"2024-05-27T07:28:24.479722+0000","last_unstale":"2024-05-27T07:31:33.781727+0000","last_undegraded":"2024-05-27T07:31:33.781727+0000","last_fullsized":"2024-05-27T07:31:33.781727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:27:36.074612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.756788+0000","last_change":"2024-05-27T07:28:24.490389+0000","last_active":"2024-05-27T07:31:33.756788+0000","last_peered":"2024-05-27T07:31:33.756788+0000","last_clean":"2024-05-27T07:31:33.756788+0000","last_became_active":"2024-05-27T07:28:24.488081+0000","last_became_peered":"2024-05-27T07:28:24.488081+0000","last_unstale":"2024-05-27T07:31:33.756788+0000","last_undegraded":"2024-05-27T07:31:33.756788+0000","last_fullsized":"2024-05-27T07:31:33.756788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:06.318235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776147+0000","last_change":"2024-05-27T07:28:24.477062+0000","last_active":"2024-05-27T07:31:33.776147+0000","last_peered":"2024-05-27T07:31:33.776147+0000","last_clean":"2024-05-27T07:31:33.776147+0000","last_became_active":"2024-05-27T07:28:24.475322+0000","last_became_peered":"2024-05-27T07:28:24.475322+0000","last_unstale":"2024-05-27T07:31:33.776147+0000","last_undegraded":"2024-05-27T07:31:33.776147+0000","last_fullsized":"2024-05-27T07:31:33.776147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:30.365295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.757461+0000","last_change":"2024-05-27T07:29:04.901715+0000","last_active":"2024-05-27T07:31:33.757461+0000","last_peered":"2024-05-27T07:31:33.757461+0000","last_clean":"2024-05-27T07:31:33.757461+0000","last_became_active":"2024-05-27T07:29:04.900968+0000","last_became_peered":"2024-05-27T07:29:04.900968+0000","last_unstale":"2024-05-27T07:31:33.757461+0000","last_undegraded":"2024-05-27T07:31:33.757461+0000","last_fullsized":"2024-05-27T07:31:33.757461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:16:34.033701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.775821+0000","last_change":"2024-05-27T07:28:24.477504+0000","last_active":"2024-05-27T07:31:33.775821+0000","last_peered":"2024-05-27T07:31:33.775821+0000","last_clean":"2024-05-27T07:31:33.775821+0000","last_became_active":"2024-05-27T07:28:24.476709+0000","last_became_peered":"2024-05-27T07:28:24.476709+0000","last_unstale":"2024-05-27T07:31:33.775821+0000","last_undegraded":"2024-05-27T07:31:33.775821+0000","last_fullsized":"2024-05-27T07:31:33.775821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:05:48.456347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.751537+0000","last_change":"2024-05-27T07:28:24.485567+0000","last_active":"2024-05-27T07:31:33.751537+0000","last_peered":"2024-05-27T07:31:33.751537+0000","last_clean":"2024-05-27T07:31:33.751537+0000","last_became_active":"2024-05-27T07:28:24.485175+0000","last_became_peered":"2024-05-27T07:28:24.485175+0000","last_unstale":"2024-05-27T07:31:33.751537+0000","last_undegraded":"2024-05-27T07:31:33.751537+0000","last_fullsized":"2024-05-27T07:31:33.751537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:25:30.062813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.764430+0000","last_change":"2024-05-27T07:29:04.900657+0000","last_active":"2024-05-27T07:31:33.764430+0000","last_peered":"2024-05-27T07:31:33.764430+0000","last_clean":"2024-05-27T07:31:33.764430+0000","last_became_active":"2024-05-27T07:29:04.900093+0000","last_became_peered":"2024-05-27T07:29:04.900093+0000","last_unstale":"2024-05-27T07:31:33.764430+0000","last_undegraded":"2024-05-27T07:31:33.764430+0000","last_fullsized":"2024-05-27T07:31:33.764430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:28:04.157825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.766949+0000","last_change":"2024-05-27T07:28:24.470503+0000","last_active":"2024-05-27T07:31:33.766949+0000","last_peered":"2024-05-27T07:31:33.766949+0000","last_clean":"2024-05-27T07:31:33.766949+0000","last_became_active":"2024-05-27T07:28:24.470128+0000","last_became_peered":"2024-05-27T07:28:24.470128+0000","last_unstale":"2024-05-27T07:31:33.766949+0000","last_undegraded":"2024-05-27T07:31:33.766949+0000","last_fullsized":"2024-05-27T07:31:33.766949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:46:43.149635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.780079+0000","last_change":"2024-05-27T07:28:24.491382+0000","last_active":"2024-05-27T07:31:33.780079+0000","last_peered":"2024-05-27T07:31:33.780079+0000","last_clean":"2024-05-27T07:31:33.780079+0000","last_became_active":"2024-05-27T07:28:24.486155+0000","last_became_peered":"2024-05-27T07:28:24.486155+0000","last_unstale":"2024-05-27T07:31:33.780079+0000","last_undegraded":"2024-05-27T07:31:33.780079+0000","last_fullsized":"2024-05-27T07:31:33.780079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:48.408573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.773959+0000","last_change":"2024-05-27T07:28:24.489140+0000","last_active":"2024-05-27T07:31:33.773959+0000","last_peered":"2024-05-27T07:31:33.773959+0000","last_clean":"2024-05-27T07:31:33.773959+0000","last_became_active":"2024-05-27T07:28:24.488733+0000","last_became_peered":"2024-05-27T07:28:24.488733+0000","last_unstale":"2024-05-27T07:31:33.773959+0000","last_undegraded":"2024-05-27T07:31:33.773959+0000","last_fullsized":"2024-05-27T07:31:33.773959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:02:57.711197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776828+0000","last_change":"2024-05-27T07:28:24.486022+0000","last_active":"2024-05-27T07:31:33.776828+0000","last_peered":"2024-05-27T07:31:33.776828+0000","last_clean":"2024-05-27T07:31:33.776828+0000","last_became_active":"2024-05-27T07:28:24.485639+0000","last_became_peered":"2024-05-27T07:28:24.485639+0000","last_unstale":"2024-05-27T07:31:33.776828+0000","last_undegraded":"2024-05-27T07:31:33.776828+0000","last_fullsized":"2024-05-27T07:31:33.776828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:07.629402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119584+0000","last_change":"2024-05-27T07:28:37.898984+0000","last_active":"2024-05-27T07:30:05.119584+0000","last_peered":"2024-05-27T07:30:05.119584+0000","last_clean":"2024-05-27T07:30:05.119584+0000","last_became_active":"2024-05-27T07:28:37.898132+0000","last_became_peered":"2024-05-27T07:28:37.898132+0000","last_unstale":"2024-05-27T07:30:05.119584+0000","last_undegraded":"2024-05-27T07:30:05.119584+0000","last_fullsized":"2024-05-27T07:30:05.119584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:51.987386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127598+0000","last_change":"2024-05-27T07:28:20.402788+0000","last_active":"2024-05-27T07:30:05.127598+0000","last_peered":"2024-05-27T07:30:05.127598+0000","last_clean":"2024-05-27T07:30:05.127598+0000","last_became_active":"2024-05-27T07:28:20.402041+0000","last_became_peered":"2024-05-27T07:28:20.402041+0000","last_unstale":"2024-05-27T07:30:05.127598+0000","last_undegraded":"2024-05-27T07:30:05.127598+0000","last_fullsized":"2024-05-27T07:30:05.127598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:44.421979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.774601+0000","last_change":"2024-05-27T07:28:24.485193+0000","last_active":"2024-05-27T07:31:33.774601+0000","last_peered":"2024-05-27T07:31:33.774601+0000","last_clean":"2024-05-27T07:31:33.774601+0000","last_became_active":"2024-05-27T07:28:24.484086+0000","last_became_peered":"2024-05-27T07:28:24.484086+0000","last_unstale":"2024-05-27T07:31:33.774601+0000","last_undegraded":"2024-05-27T07:31:33.774601+0000","last_fullsized":"2024-05-27T07:31:33.774601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:10.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536567+0000","last_change":"2024-05-27T07:28:37.902602+0000","last_active":"2024-05-27T07:30:05.536567+0000","last_peered":"2024-05-27T07:30:05.536567+0000","last_clean":"2024-05-27T07:30:05.536567+0000","last_became_active":"2024-05-27T07:28:37.902120+0000","last_became_peered":"2024-05-27T07:28:37.902120+0000","last_unstale":"2024-05-27T07:30:05.536567+0000","last_undegraded":"2024-05-27T07:30:05.536567+0000","last_fullsized":"2024-05-27T07:30:05.536567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:21.209851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122003+0000","last_change":"2024-05-27T07:28:20.465500+0000","last_active":"2024-05-27T07:30:05.122003+0000","last_peered":"2024-05-27T07:30:05.122003+0000","last_clean":"2024-05-27T07:30:05.122003+0000","last_became_active":"2024-05-27T07:28:20.464951+0000","last_became_peered":"2024-05-27T07:28:20.464951+0000","last_unstale":"2024-05-27T07:30:05.122003+0000","last_undegraded":"2024-05-27T07:30:05.122003+0000","last_fullsized":"2024-05-27T07:30:05.122003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:23:25.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.780254+0000","last_change":"2024-05-27T07:28:24.467655+0000","last_active":"2024-05-27T07:31:33.780254+0000","last_peered":"2024-05-27T07:31:33.780254+0000","last_clean":"2024-05-27T07:31:33.780254+0000","last_became_active":"2024-05-27T07:28:24.467194+0000","last_became_peered":"2024-05-27T07:28:24.467194+0000","last_unstale":"2024-05-27T07:31:33.780254+0000","last_undegraded":"2024-05-27T07:31:33.780254+0000","last_fullsized":"2024-05-27T07:31:33.780254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:38.024514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536923+0000","last_change":"2024-05-27T07:29:04.975679+0000","last_active":"2024-05-27T07:30:05.536923+0000","last_peered":"2024-05-27T07:30:05.536923+0000","last_clean":"2024-05-27T07:30:05.536923+0000","last_became_active":"2024-05-27T07:29:04.974323+0000","last_became_peered":"2024-05-27T07:29:04.974323+0000","last_unstale":"2024-05-27T07:30:05.536923+0000","last_undegraded":"2024-05-27T07:30:05.536923+0000","last_fullsized":"2024-05-27T07:30:05.536923+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:50.489737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584407+0000","last_change":"2024-05-27T07:28:20.408117+0000","last_active":"2024-05-27T07:30:05.584407+0000","last_peered":"2024-05-27T07:30:05.584407+0000","last_clean":"2024-05-27T07:30:05.584407+0000","last_became_active":"2024-05-27T07:28:20.407723+0000","last_became_peered":"2024-05-27T07:28:20.407723+0000","last_unstale":"2024-05-27T07:30:05.584407+0000","last_undegraded":"2024-05-27T07:30:05.584407+0000","last_fullsized":"2024-05-27T07:30:05.584407+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:20:31.376656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.779839+0000","last_change":"2024-05-27T07:28:24.490269+0000","last_active":"2024-05-27T07:31:33.779839+0000","last_peered":"2024-05-27T07:31:33.779839+0000","last_clean":"2024-05-27T07:31:33.779839+0000","last_became_active":"2024-05-27T07:28:24.488849+0000","last_became_peered":"2024-05-27T07:28:24.488849+0000","last_unstale":"2024-05-27T07:31:33.779839+0000","last_undegraded":"2024-05-27T07:31:33.779839+0000","last_fullsized":"2024-05-27T07:31:33.779839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:00.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536807+0000","last_change":"2024-05-27T07:28:37.908658+0000","last_active":"2024-05-27T07:30:05.536807+0000","last_peered":"2024-05-27T07:30:05.536807+0000","last_clean":"2024-05-27T07:30:05.536807+0000","last_became_active":"2024-05-27T07:28:37.908056+0000","last_became_peered":"2024-05-27T07:28:37.908056+0000","last_unstale":"2024-05-27T07:30:05.536807+0000","last_undegraded":"2024-05-27T07:30:05.536807+0000","last_fullsized":"2024-05-27T07:30:05.536807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:31.853606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583611+0000","last_change":"2024-05-27T07:28:20.410944+0000","last_active":"2024-05-27T07:30:05.583611+0000","last_peered":"2024-05-27T07:30:05.583611+0000","last_clean":"2024-05-27T07:30:05.583611+0000","last_became_active":"2024-05-27T07:28:20.409789+0000","last_became_peered":"2024-05-27T07:28:20.409789+0000","last_unstale":"2024-05-27T07:30:05.583611+0000","last_undegraded":"2024-05-27T07:30:05.583611+0000","last_fullsized":"2024-05-27T07:30:05.583611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:53:34.962397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.767919+0000","last_change":"2024-05-27T07:28:24.482828+0000","last_active":"2024-05-27T07:31:33.767919+0000","last_peered":"2024-05-27T07:31:33.767919+0000","last_clean":"2024-05-27T07:31:33.767919+0000","last_became_active":"2024-05-27T07:28:24.482475+0000","last_became_peered":"2024-05-27T07:28:24.482475+0000","last_unstale":"2024-05-27T07:31:33.767919+0000","last_undegraded":"2024-05-27T07:31:33.767919+0000","last_fullsized":"2024-05-27T07:31:33.767919+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:53:32.064441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120655+0000","last_change":"2024-05-27T07:29:04.905734+0000","last_active":"2024-05-27T07:30:05.120655+0000","last_peered":"2024-05-27T07:30:05.120655+0000","last_clean":"2024-05-27T07:30:05.120655+0000","last_became_active":"2024-05-27T07:29:04.902387+0000","last_became_peered":"2024-05-27T07:29:04.902387+0000","last_unstale":"2024-05-27T07:30:05.120655+0000","last_undegraded":"2024-05-27T07:30:05.120655+0000","last_fullsized":"2024-05-27T07:30:05.120655+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:35.993193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":175,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:54.426056+0000","last_change":"2024-05-27T07:28:20.412258+0000","last_active":"2024-05-27T07:31:54.426056+0000","last_peered":"2024-05-27T07:31:54.426056+0000","last_clean":"2024-05-27T07:31:54.426056+0000","last_became_active":"2024-05-27T07:28:20.411764+0000","last_became_peered":"2024-05-27T07:28:20.411764+0000","last_unstale":"2024-05-27T07:31:54.426056+0000","last_undegraded":"2024-05-27T07:31:54.426056+0000","last_fullsized":"2024-05-27T07:31:54.426056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:38.462887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.759152+0000","last_change":"2024-05-27T07:28:24.482179+0000","last_active":"2024-05-27T07:31:33.759152+0000","last_peered":"2024-05-27T07:31:33.759152+0000","last_clean":"2024-05-27T07:31:33.759152+0000","last_became_active":"2024-05-27T07:28:24.480682+0000","last_became_peered":"2024-05-27T07:28:24.480682+0000","last_unstale":"2024-05-27T07:31:33.759152+0000","last_undegraded":"2024-05-27T07:31:33.759152+0000","last_fullsized":"2024-05-27T07:31:33.759152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:51.403862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119903+0000","last_change":"2024-05-27T07:28:37.903716+0000","last_active":"2024-05-27T07:30:05.119903+0000","last_peered":"2024-05-27T07:30:05.119903+0000","last_clean":"2024-05-27T07:30:05.119903+0000","last_became_active":"2024-05-27T07:28:37.901083+0000","last_became_peered":"2024-05-27T07:28:37.901083+0000","last_unstale":"2024-05-27T07:30:05.119903+0000","last_undegraded":"2024-05-27T07:30:05.119903+0000","last_fullsized":"2024-05-27T07:30:05.119903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:44:11.606711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578206+0000","last_change":"2024-05-27T07:28:20.392441+0000","last_active":"2024-05-27T07:30:05.578206+0000","last_peered":"2024-05-27T07:30:05.578206+0000","last_clean":"2024-05-27T07:30:05.578206+0000","last_became_active":"2024-05-27T07:28:20.390983+0000","last_became_peered":"2024-05-27T07:28:20.390983+0000","last_unstale":"2024-05-27T07:30:05.578206+0000","last_undegraded":"2024-05-27T07:30:05.578206+0000","last_fullsized":"2024-05-27T07:30:05.578206+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:33:30.338043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.773247+0000","last_change":"2024-05-27T07:30:05.518915+0000","last_active":"2024-05-27T07:31:33.773247+0000","last_peered":"2024-05-27T07:31:33.773247+0000","last_clean":"2024-05-27T07:31:33.773247+0000","last_became_active":"2024-05-27T07:30:05.517894+0000","last_became_peered":"2024-05-27T07:30:05.517894+0000","last_unstale":"2024-05-27T07:31:33.773247+0000","last_undegraded":"2024-05-27T07:31:33.773247+0000","last_fullsized":"2024-05-27T07:31:33.773247+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:51.353713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,7],"acting":[9,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580964+0000","last_change":"2024-05-27T07:28:37.913352+0000","last_active":"2024-05-27T07:30:05.580964+0000","last_peered":"2024-05-27T07:30:05.580964+0000","last_clean":"2024-05-27T07:30:05.580964+0000","last_became_active":"2024-05-27T07:28:37.913027+0000","last_became_peered":"2024-05-27T07:28:37.913027+0000","last_unstale":"2024-05-27T07:30:05.580964+0000","last_undegraded":"2024-05-27T07:30:05.580964+0000","last_fullsized":"2024-05-27T07:30:05.580964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:03.485594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.404589+0000","last_change":"2024-05-27T07:28:20.406094+0000","last_active":"2024-05-27T07:30:05.404589+0000","last_peered":"2024-05-27T07:30:05.404589+0000","last_clean":"2024-05-27T07:30:05.404589+0000","last_became_active":"2024-05-27T07:28:20.405641+0000","last_became_peered":"2024-05-27T07:28:20.405641+0000","last_unstale":"2024-05-27T07:30:05.404589+0000","last_undegraded":"2024-05-27T07:30:05.404589+0000","last_fullsized":"2024-05-27T07:30:05.404589+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:16.598743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.777772+0000","last_change":"2024-05-27T07:28:24.474696+0000","last_active":"2024-05-27T07:31:33.777772+0000","last_peered":"2024-05-27T07:31:33.777772+0000","last_clean":"2024-05-27T07:31:33.777772+0000","last_became_active":"2024-05-27T07:28:24.474319+0000","last_became_peered":"2024-05-27T07:28:24.474319+0000","last_unstale":"2024-05-27T07:31:33.777772+0000","last_undegraded":"2024-05-27T07:31:33.777772+0000","last_fullsized":"2024-05-27T07:31:33.777772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:53.010898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576132+0000","last_change":"2024-05-27T07:28:37.910386+0000","last_active":"2024-05-27T07:30:05.576132+0000","last_peered":"2024-05-27T07:30:05.576132+0000","last_clean":"2024-05-27T07:30:05.576132+0000","last_became_active":"2024-05-27T07:28:37.909941+0000","last_became_peered":"2024-05-27T07:28:37.909941+0000","last_unstale":"2024-05-27T07:30:05.576132+0000","last_undegraded":"2024-05-27T07:30:05.576132+0000","last_fullsized":"2024-05-27T07:30:05.576132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:15.820318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582005+0000","last_change":"2024-05-27T07:28:20.411855+0000","last_active":"2024-05-27T07:30:05.582005+0000","last_peered":"2024-05-27T07:30:05.582005+0000","last_clean":"2024-05-27T07:30:05.582005+0000","last_became_active":"2024-05-27T07:28:20.411169+0000","last_became_peered":"2024-05-27T07:28:20.411169+0000","last_unstale":"2024-05-27T07:30:05.582005+0000","last_undegraded":"2024-05-27T07:30:05.582005+0000","last_fullsized":"2024-05-27T07:30:05.582005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:37.278507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.779523+0000","last_change":"2024-05-27T07:28:24.485799+0000","last_active":"2024-05-27T07:31:33.779523+0000","last_peered":"2024-05-27T07:31:33.779523+0000","last_clean":"2024-05-27T07:31:33.779523+0000","last_became_active":"2024-05-27T07:28:24.485395+0000","last_became_peered":"2024-05-27T07:28:24.485395+0000","last_unstale":"2024-05-27T07:31:33.779523+0000","last_undegraded":"2024-05-27T07:31:33.779523+0000","last_fullsized":"2024-05-27T07:31:33.779523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:11.178129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120055+0000","last_change":"2024-05-27T07:28:37.898929+0000","last_active":"2024-05-27T07:30:05.120055+0000","last_peered":"2024-05-27T07:30:05.120055+0000","last_clean":"2024-05-27T07:30:05.120055+0000","last_became_active":"2024-05-27T07:28:37.898371+0000","last_became_peered":"2024-05-27T07:28:37.898371+0000","last_unstale":"2024-05-27T07:30:05.120055+0000","last_undegraded":"2024-05-27T07:30:05.120055+0000","last_fullsized":"2024-05-27T07:30:05.120055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:48.867729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":169,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:52.418628+0000","last_change":"2024-05-27T07:28:20.407117+0000","last_active":"2024-05-27T07:31:52.418628+0000","last_peered":"2024-05-27T07:31:52.418628+0000","last_clean":"2024-05-27T07:31:52.418628+0000","last_became_active":"2024-05-27T07:28:20.403562+0000","last_became_peered":"2024-05-27T07:28:20.403562+0000","last_unstale":"2024-05-27T07:31:52.418628+0000","last_undegraded":"2024-05-27T07:31:52.418628+0000","last_fullsized":"2024-05-27T07:31:52.418628+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:25.903886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776246+0000","last_change":"2024-05-27T07:28:24.489771+0000","last_active":"2024-05-27T07:31:33.776246+0000","last_peered":"2024-05-27T07:31:33.776246+0000","last_clean":"2024-05-27T07:31:33.776246+0000","last_became_active":"2024-05-27T07:28:24.489163+0000","last_became_peered":"2024-05-27T07:28:24.489163+0000","last_unstale":"2024-05-27T07:31:33.776246+0000","last_undegraded":"2024-05-27T07:31:33.776246+0000","last_fullsized":"2024-05-27T07:31:33.776246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:08.053634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.123674+0000","last_change":"2024-05-27T07:28:37.914598+0000","last_active":"2024-05-27T07:30:05.123674+0000","last_peered":"2024-05-27T07:30:05.123674+0000","last_clean":"2024-05-27T07:30:05.123674+0000","last_became_active":"2024-05-27T07:28:37.913916+0000","last_became_peered":"2024-05-27T07:28:37.913916+0000","last_unstale":"2024-05-27T07:30:05.123674+0000","last_undegraded":"2024-05-27T07:30:05.123674+0000","last_fullsized":"2024-05-27T07:30:05.123674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:01.475789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.517157+0000","last_change":"2024-05-27T07:29:04.901366+0000","last_active":"2024-05-27T07:30:05.517157+0000","last_peered":"2024-05-27T07:30:05.517157+0000","last_clean":"2024-05-27T07:30:05.517157+0000","last_became_active":"2024-05-27T07:29:04.900355+0000","last_became_peered":"2024-05-27T07:29:04.900355+0000","last_unstale":"2024-05-27T07:30:05.517157+0000","last_undegraded":"2024-05-27T07:30:05.517157+0000","last_fullsized":"2024-05-27T07:30:05.517157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:10:03.349345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.764329+0000","last_change":"2024-05-27T07:28:24.477771+0000","last_active":"2024-05-27T07:31:33.764329+0000","last_peered":"2024-05-27T07:31:33.764329+0000","last_clean":"2024-05-27T07:31:33.764329+0000","last_became_active":"2024-05-27T07:28:24.474179+0000","last_became_peered":"2024-05-27T07:28:24.474179+0000","last_unstale":"2024-05-27T07:31:33.764329+0000","last_undegraded":"2024-05-27T07:31:33.764329+0000","last_fullsized":"2024-05-27T07:31:33.764329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:08:04.218264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537477+0000","last_change":"2024-05-27T07:29:04.896583+0000","last_active":"2024-05-27T07:30:05.537477+0000","last_peered":"2024-05-27T07:30:05.537477+0000","last_clean":"2024-05-27T07:30:05.537477+0000","last_became_active":"2024-05-27T07:29:04.894118+0000","last_became_peered":"2024-05-27T07:29:04.894118+0000","last_unstale":"2024-05-27T07:30:05.537477+0000","last_undegraded":"2024-05-27T07:30:05.537477+0000","last_fullsized":"2024-05-27T07:30:05.537477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:17.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.583611+0000","last_change":"2024-05-27T07:28:20.466851+0000","last_active":"2024-05-27T07:30:05.583611+0000","last_peered":"2024-05-27T07:30:05.583611+0000","last_clean":"2024-05-27T07:30:05.583611+0000","last_became_active":"2024-05-27T07:28:20.465133+0000","last_became_peered":"2024-05-27T07:28:20.465133+0000","last_unstale":"2024-05-27T07:30:05.583611+0000","last_undegraded":"2024-05-27T07:30:05.583611+0000","last_fullsized":"2024-05-27T07:30:05.583611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:11.025588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.779962+0000","last_change":"2024-05-27T07:28:24.469924+0000","last_active":"2024-05-27T07:31:33.779962+0000","last_peered":"2024-05-27T07:31:33.779962+0000","last_clean":"2024-05-27T07:31:33.779962+0000","last_became_active":"2024-05-27T07:28:24.469559+0000","last_became_peered":"2024-05-27T07:28:24.469559+0000","last_unstale":"2024-05-27T07:31:33.779962+0000","last_undegraded":"2024-05-27T07:31:33.779962+0000","last_fullsized":"2024-05-27T07:31:33.779962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:57.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.125161+0000","last_change":"2024-05-27T07:28:37.904549+0000","last_active":"2024-05-27T07:30:05.125161+0000","last_peered":"2024-05-27T07:30:05.125161+0000","last_clean":"2024-05-27T07:30:05.125161+0000","last_became_active":"2024-05-27T07:28:37.903882+0000","last_became_peered":"2024-05-27T07:28:37.903882+0000","last_unstale":"2024-05-27T07:30:05.125161+0000","last_undegraded":"2024-05-27T07:30:05.125161+0000","last_fullsized":"2024-05-27T07:30:05.125161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:41.607641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'39","reported_seq":339,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:53.419769+0000","last_change":"2024-05-27T07:28:20.410025+0000","last_active":"2024-05-27T07:31:53.419769+0000","last_peered":"2024-05-27T07:31:53.419769+0000","last_clean":"2024-05-27T07:31:53.419769+0000","last_became_active":"2024-05-27T07:28:20.409115+0000","last_became_peered":"2024-05-27T07:28:20.409115+0000","last_unstale":"2024-05-27T07:31:53.419769+0000","last_undegraded":"2024-05-27T07:31:53.419769+0000","last_fullsized":"2024-05-27T07:31:53.419769+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:56.834059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69206,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":255,"num_read_kb":196,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.758038+0000","last_change":"2024-05-27T07:28:24.469609+0000","last_active":"2024-05-27T07:31:33.758038+0000","last_peered":"2024-05-27T07:31:33.758038+0000","last_clean":"2024-05-27T07:31:33.758038+0000","last_became_active":"2024-05-27T07:28:24.469192+0000","last_became_peered":"2024-05-27T07:28:24.469192+0000","last_unstale":"2024-05-27T07:31:33.758038+0000","last_undegraded":"2024-05-27T07:31:33.758038+0000","last_fullsized":"2024-05-27T07:31:33.758038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:32:59.006298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.585019+0000","last_change":"2024-05-27T07:28:37.900745+0000","last_active":"2024-05-27T07:30:05.585019+0000","last_peered":"2024-05-27T07:30:05.585019+0000","last_clean":"2024-05-27T07:30:05.585019+0000","last_became_active":"2024-05-27T07:28:37.900205+0000","last_became_peered":"2024-05-27T07:28:37.900205+0000","last_unstale":"2024-05-27T07:30:05.585019+0000","last_undegraded":"2024-05-27T07:30:05.585019+0000","last_fullsized":"2024-05-27T07:30:05.585019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:52:32.235201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120246+0000","last_change":"2024-05-27T07:28:20.413380+0000","last_active":"2024-05-27T07:30:05.120246+0000","last_peered":"2024-05-27T07:30:05.120246+0000","last_clean":"2024-05-27T07:30:05.120246+0000","last_became_active":"2024-05-27T07:28:20.412839+0000","last_became_peered":"2024-05-27T07:28:20.412839+0000","last_unstale":"2024-05-27T07:30:05.120246+0000","last_undegraded":"2024-05-27T07:30:05.120246+0000","last_fullsized":"2024-05-27T07:30:05.120246+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:57.481203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.777363+0000","last_change":"2024-05-27T07:28:24.474228+0000","last_active":"2024-05-27T07:31:33.777363+0000","last_peered":"2024-05-27T07:31:33.777363+0000","last_clean":"2024-05-27T07:31:33.777363+0000","last_became_active":"2024-05-27T07:28:24.473833+0000","last_became_peered":"2024-05-27T07:28:24.473833+0000","last_unstale":"2024-05-27T07:31:33.777363+0000","last_undegraded":"2024-05-27T07:31:33.777363+0000","last_fullsized":"2024-05-27T07:31:33.777363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:52.456730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580416+0000","last_change":"2024-05-27T07:28:37.917630+0000","last_active":"2024-05-27T07:30:05.580416+0000","last_peered":"2024-05-27T07:30:05.580416+0000","last_clean":"2024-05-27T07:30:05.580416+0000","last_became_active":"2024-05-27T07:28:37.917253+0000","last_became_peered":"2024-05-27T07:28:37.917253+0000","last_unstale":"2024-05-27T07:30:05.580416+0000","last_undegraded":"2024-05-27T07:30:05.580416+0000","last_fullsized":"2024-05-27T07:30:05.580416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:05.069946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584721+0000","last_change":"2024-05-27T07:28:20.408988+0000","last_active":"2024-05-27T07:30:05.584721+0000","last_peered":"2024-05-27T07:30:05.584721+0000","last_clean":"2024-05-27T07:30:05.584721+0000","last_became_active":"2024-05-27T07:28:20.408313+0000","last_became_peered":"2024-05-27T07:28:20.408313+0000","last_unstale":"2024-05-27T07:30:05.584721+0000","last_undegraded":"2024-05-27T07:30:05.584721+0000","last_fullsized":"2024-05-27T07:30:05.584721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:17.480412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.770467+0000","last_change":"2024-05-27T07:28:24.480096+0000","last_active":"2024-05-27T07:31:33.770467+0000","last_peered":"2024-05-27T07:31:33.770467+0000","last_clean":"2024-05-27T07:31:33.770467+0000","last_became_active":"2024-05-27T07:28:24.477988+0000","last_became_peered":"2024-05-27T07:28:24.477988+0000","last_unstale":"2024-05-27T07:31:33.770467+0000","last_undegraded":"2024-05-27T07:31:33.770467+0000","last_fullsized":"2024-05-27T07:31:33.770467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:13:59.860458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127148+0000","last_change":"2024-05-27T07:28:37.917518+0000","last_active":"2024-05-27T07:30:05.127148+0000","last_peered":"2024-05-27T07:30:05.127148+0000","last_clean":"2024-05-27T07:30:05.127148+0000","last_became_active":"2024-05-27T07:28:37.916971+0000","last_became_peered":"2024-05-27T07:28:37.916971+0000","last_unstale":"2024-05-27T07:30:05.127148+0000","last_undegraded":"2024-05-27T07:30:05.127148+0000","last_fullsized":"2024-05-27T07:30:05.127148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:17:53.067747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":172,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:53.435326+0000","last_change":"2024-05-27T07:28:20.416766+0000","last_active":"2024-05-27T07:31:53.435326+0000","last_peered":"2024-05-27T07:31:53.435326+0000","last_clean":"2024-05-27T07:31:53.435326+0000","last_became_active":"2024-05-27T07:28:20.416297+0000","last_became_peered":"2024-05-27T07:28:20.416297+0000","last_unstale":"2024-05-27T07:31:53.435326+0000","last_undegraded":"2024-05-27T07:31:53.435326+0000","last_fullsized":"2024-05-27T07:31:53.435326+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:02.841494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.756721+0000","last_change":"2024-05-27T07:28:24.486869+0000","last_active":"2024-05-27T07:31:33.756721+0000","last_peered":"2024-05-27T07:31:33.756721+0000","last_clean":"2024-05-27T07:31:33.756721+0000","last_became_active":"2024-05-27T07:28:24.486451+0000","last_became_peered":"2024-05-27T07:28:24.486451+0000","last_unstale":"2024-05-27T07:31:33.756721+0000","last_undegraded":"2024-05-27T07:31:33.756721+0000","last_fullsized":"2024-05-27T07:31:33.756721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:10.722659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406823+0000","last_change":"2024-05-27T07:29:04.901632+0000","last_active":"2024-05-27T07:30:05.406823+0000","last_peered":"2024-05-27T07:30:05.406823+0000","last_clean":"2024-05-27T07:30:05.406823+0000","last_became_active":"2024-05-27T07:29:04.900203+0000","last_became_peered":"2024-05-27T07:29:04.900203+0000","last_unstale":"2024-05-27T07:30:05.406823+0000","last_undegraded":"2024-05-27T07:30:05.406823+0000","last_fullsized":"2024-05-27T07:30:05.406823+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:57.834889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580102+0000","last_change":"2024-05-27T07:28:20.468162+0000","last_active":"2024-05-27T07:30:05.580102+0000","last_peered":"2024-05-27T07:30:05.580102+0000","last_clean":"2024-05-27T07:30:05.580102+0000","last_became_active":"2024-05-27T07:28:20.465226+0000","last_became_peered":"2024-05-27T07:28:20.465226+0000","last_unstale":"2024-05-27T07:30:05.580102+0000","last_undegraded":"2024-05-27T07:30:05.580102+0000","last_fullsized":"2024-05-27T07:30:05.580102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:21:34.110356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.756405+0000","last_change":"2024-05-27T07:28:24.485053+0000","last_active":"2024-05-27T07:31:33.756405+0000","last_peered":"2024-05-27T07:31:33.756405+0000","last_clean":"2024-05-27T07:31:33.756405+0000","last_became_active":"2024-05-27T07:28:24.484391+0000","last_became_peered":"2024-05-27T07:28:24.484391+0000","last_unstale":"2024-05-27T07:31:33.756405+0000","last_undegraded":"2024-05-27T07:31:33.756405+0000","last_fullsized":"2024-05-27T07:31:33.756405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:05.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.582857+0000","last_change":"2024-05-27T07:28:37.904731+0000","last_active":"2024-05-27T07:30:05.582857+0000","last_peered":"2024-05-27T07:30:05.582857+0000","last_clean":"2024-05-27T07:30:05.582857+0000","last_became_active":"2024-05-27T07:28:37.904284+0000","last_became_peered":"2024-05-27T07:28:37.904284+0000","last_unstale":"2024-05-27T07:30:05.582857+0000","last_undegraded":"2024-05-27T07:30:05.582857+0000","last_fullsized":"2024-05-27T07:30:05.582857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:23.198465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.585639+0000","last_change":"2024-05-27T07:28:20.406829+0000","last_active":"2024-05-27T07:30:05.585639+0000","last_peered":"2024-05-27T07:30:05.585639+0000","last_clean":"2024-05-27T07:30:05.585639+0000","last_became_active":"2024-05-27T07:28:20.405497+0000","last_became_peered":"2024-05-27T07:28:20.405497+0000","last_unstale":"2024-05-27T07:30:05.585639+0000","last_undegraded":"2024-05-27T07:30:05.585639+0000","last_fullsized":"2024-05-27T07:30:05.585639+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:14.225630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.745746+0000","last_change":"2024-05-27T07:28:24.488023+0000","last_active":"2024-05-27T07:31:33.745746+0000","last_peered":"2024-05-27T07:31:33.745746+0000","last_clean":"2024-05-27T07:31:33.745746+0000","last_became_active":"2024-05-27T07:28:24.487652+0000","last_became_peered":"2024-05-27T07:28:24.487652+0000","last_unstale":"2024-05-27T07:31:33.745746+0000","last_undegraded":"2024-05-27T07:31:33.745746+0000","last_fullsized":"2024-05-27T07:31:33.745746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:25.951157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.581217+0000","last_change":"2024-05-27T07:28:37.968034+0000","last_active":"2024-05-27T07:30:05.581217+0000","last_peered":"2024-05-27T07:30:05.581217+0000","last_clean":"2024-05-27T07:30:05.581217+0000","last_became_active":"2024-05-27T07:28:37.967127+0000","last_became_peered":"2024-05-27T07:28:37.967127+0000","last_unstale":"2024-05-27T07:30:05.581217+0000","last_undegraded":"2024-05-27T07:30:05.581217+0000","last_fullsized":"2024-05-27T07:30:05.581217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:49.353423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.126685+0000","last_change":"2024-05-27T07:28:20.465782+0000","last_active":"2024-05-27T07:30:05.126685+0000","last_peered":"2024-05-27T07:30:05.126685+0000","last_clean":"2024-05-27T07:30:05.126685+0000","last_became_active":"2024-05-27T07:28:20.465134+0000","last_became_peered":"2024-05-27T07:28:20.465134+0000","last_unstale":"2024-05-27T07:30:05.126685+0000","last_undegraded":"2024-05-27T07:30:05.126685+0000","last_fullsized":"2024-05-27T07:30:05.126685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:17.992092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.761621+0000","last_change":"2024-05-27T07:28:24.484691+0000","last_active":"2024-05-27T07:31:33.761621+0000","last_peered":"2024-05-27T07:31:33.761621+0000","last_clean":"2024-05-27T07:31:33.761621+0000","last_became_active":"2024-05-27T07:28:24.484249+0000","last_became_peered":"2024-05-27T07:28:24.484249+0000","last_unstale":"2024-05-27T07:31:33.761621+0000","last_undegraded":"2024-05-27T07:31:33.761621+0000","last_fullsized":"2024-05-27T07:31:33.761621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:25.753276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.405878+0000","last_change":"2024-05-27T07:28:37.907849+0000","last_active":"2024-05-27T07:30:05.405878+0000","last_peered":"2024-05-27T07:30:05.405878+0000","last_clean":"2024-05-27T07:30:05.405878+0000","last_became_active":"2024-05-27T07:28:37.907105+0000","last_became_peered":"2024-05-27T07:28:37.907105+0000","last_unstale":"2024-05-27T07:30:05.405878+0000","last_undegraded":"2024-05-27T07:30:05.405878+0000","last_fullsized":"2024-05-27T07:30:05.405878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:54.075240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516795+0000","last_change":"2024-05-27T07:28:20.411302+0000","last_active":"2024-05-27T07:30:05.516795+0000","last_peered":"2024-05-27T07:30:05.516795+0000","last_clean":"2024-05-27T07:30:05.516795+0000","last_became_active":"2024-05-27T07:28:20.409144+0000","last_became_peered":"2024-05-27T07:28:20.409144+0000","last_unstale":"2024-05-27T07:30:05.516795+0000","last_undegraded":"2024-05-27T07:30:05.516795+0000","last_fullsized":"2024-05-27T07:30:05.516795+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:49.985962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.751489+0000","last_change":"2024-05-27T07:28:24.487742+0000","last_active":"2024-05-27T07:31:33.751489+0000","last_peered":"2024-05-27T07:31:33.751489+0000","last_clean":"2024-05-27T07:31:33.751489+0000","last_became_active":"2024-05-27T07:28:24.485279+0000","last_became_peered":"2024-05-27T07:28:24.485279+0000","last_unstale":"2024-05-27T07:31:33.751489+0000","last_undegraded":"2024-05-27T07:31:33.751489+0000","last_fullsized":"2024-05-27T07:31:33.751489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:15.049572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.406726+0000","last_change":"2024-05-27T07:28:37.968780+0000","last_active":"2024-05-27T07:30:05.406726+0000","last_peered":"2024-05-27T07:30:05.406726+0000","last_clean":"2024-05-27T07:30:05.406726+0000","last_became_active":"2024-05-27T07:28:37.968052+0000","last_became_peered":"2024-05-27T07:28:37.968052+0000","last_unstale":"2024-05-27T07:30:05.406726+0000","last_undegraded":"2024-05-27T07:30:05.406726+0000","last_fullsized":"2024-05-27T07:30:05.406726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:51:26.787969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.753642+0000","last_change":"2024-05-27T07:28:24.485280+0000","last_active":"2024-05-27T07:31:33.753642+0000","last_peered":"2024-05-27T07:31:33.753642+0000","last_clean":"2024-05-27T07:31:33.753642+0000","last_became_active":"2024-05-27T07:28:24.484882+0000","last_became_peered":"2024-05-27T07:28:24.484882+0000","last_unstale":"2024-05-27T07:31:33.753642+0000","last_undegraded":"2024-05-27T07:31:33.753642+0000","last_fullsized":"2024-05-27T07:31:33.753642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:28.643857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":171,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:54.437049+0000","last_change":"2024-05-27T07:28:20.462593+0000","last_active":"2024-05-27T07:31:54.437049+0000","last_peered":"2024-05-27T07:31:54.437049+0000","last_clean":"2024-05-27T07:31:54.437049+0000","last_became_active":"2024-05-27T07:28:20.461128+0000","last_became_peered":"2024-05-27T07:28:20.461128+0000","last_unstale":"2024-05-27T07:31:54.437049+0000","last_undegraded":"2024-05-27T07:31:54.437049+0000","last_fullsized":"2024-05-27T07:31:54.437049+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:24:42.013315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.516623+0000","last_change":"2024-05-27T07:28:37.911546+0000","last_active":"2024-05-27T07:30:05.516623+0000","last_peered":"2024-05-27T07:30:05.516623+0000","last_clean":"2024-05-27T07:30:05.516623+0000","last_became_active":"2024-05-27T07:28:37.910777+0000","last_became_peered":"2024-05-27T07:28:37.910777+0000","last_unstale":"2024-05-27T07:30:05.516623+0000","last_undegraded":"2024-05-27T07:30:05.516623+0000","last_fullsized":"2024-05-27T07:30:05.516623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:13.344317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.775196+0000","last_change":"2024-05-27T07:28:24.483946+0000","last_active":"2024-05-27T07:31:33.775196+0000","last_peered":"2024-05-27T07:31:33.775196+0000","last_clean":"2024-05-27T07:31:33.775196+0000","last_became_active":"2024-05-27T07:28:24.481424+0000","last_became_peered":"2024-05-27T07:28:24.481424+0000","last_unstale":"2024-05-27T07:31:33.775196+0000","last_undegraded":"2024-05-27T07:31:33.775196+0000","last_fullsized":"2024-05-27T07:31:33.775196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:08.254046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.536932+0000","last_change":"2024-05-27T07:28:20.414250+0000","last_active":"2024-05-27T07:30:05.536932+0000","last_peered":"2024-05-27T07:30:05.536932+0000","last_clean":"2024-05-27T07:30:05.536932+0000","last_became_active":"2024-05-27T07:28:20.413676+0000","last_became_peered":"2024-05-27T07:28:20.413676+0000","last_unstale":"2024-05-27T07:30:05.536932+0000","last_undegraded":"2024-05-27T07:30:05.536932+0000","last_fullsized":"2024-05-27T07:30:05.536932+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:10.226771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515596+0000","last_change":"2024-05-27T07:26:26.055626+0000","last_active":"2024-05-27T07:30:05.515596+0000","last_peered":"2024-05-27T07:30:05.515596+0000","last_clean":"2024-05-27T07:30:05.515596+0000","last_became_active":"2024-05-27T07:26:26.037376+0000","last_became_peered":"2024-05-27T07:26:26.037376+0000","last_unstale":"2024-05-27T07:30:05.515596+0000","last_undegraded":"2024-05-27T07:30:05.515596+0000","last_fullsized":"2024-05-27T07:30:05.515596+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_clean_scrub_stamp":"2024-05-27T07:21:26.641949+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:16.741856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584417+0000","last_change":"2024-05-27T07:28:37.973078+0000","last_active":"2024-05-27T07:30:05.584417+0000","last_peered":"2024-05-27T07:30:05.584417+0000","last_clean":"2024-05-27T07:30:05.584417+0000","last_became_active":"2024-05-27T07:28:37.972381+0000","last_became_peered":"2024-05-27T07:28:37.972381+0000","last_unstale":"2024-05-27T07:30:05.584417+0000","last_undegraded":"2024-05-27T07:30:05.584417+0000","last_fullsized":"2024-05-27T07:30:05.584417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:18:16.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.759844+0000","last_change":"2024-05-27T07:28:24.483512+0000","last_active":"2024-05-27T07:31:33.759844+0000","last_peered":"2024-05-27T07:31:33.759844+0000","last_clean":"2024-05-27T07:31:33.759844+0000","last_became_active":"2024-05-27T07:28:24.482655+0000","last_became_peered":"2024-05-27T07:28:24.482655+0000","last_unstale":"2024-05-27T07:31:33.759844+0000","last_undegraded":"2024-05-27T07:31:33.759844+0000","last_fullsized":"2024-05-27T07:31:33.759844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:01:07.898504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.515986+0000","last_change":"2024-05-27T07:28:20.408953+0000","last_active":"2024-05-27T07:30:05.515986+0000","last_peered":"2024-05-27T07:30:05.515986+0000","last_clean":"2024-05-27T07:30:05.515986+0000","last_became_active":"2024-05-27T07:28:20.407631+0000","last_became_peered":"2024-05-27T07:28:20.407631+0000","last_unstale":"2024-05-27T07:30:05.515986+0000","last_undegraded":"2024-05-27T07:30:05.515986+0000","last_fullsized":"2024-05-27T07:30:05.515986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:43.912338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580070+0000","last_change":"2024-05-27T07:28:37.907441+0000","last_active":"2024-05-27T07:30:05.580070+0000","last_peered":"2024-05-27T07:30:05.580070+0000","last_clean":"2024-05-27T07:30:05.580070+0000","last_became_active":"2024-05-27T07:28:37.906928+0000","last_became_peered":"2024-05-27T07:28:37.906928+0000","last_unstale":"2024-05-27T07:30:05.580070+0000","last_undegraded":"2024-05-27T07:30:05.580070+0000","last_fullsized":"2024-05-27T07:30:05.580070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:28:32.345826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584754+0000","last_change":"2024-05-27T07:28:20.467408+0000","last_active":"2024-05-27T07:30:05.584754+0000","last_peered":"2024-05-27T07:30:05.584754+0000","last_clean":"2024-05-27T07:30:05.584754+0000","last_became_active":"2024-05-27T07:28:20.466064+0000","last_became_peered":"2024-05-27T07:28:20.466064+0000","last_unstale":"2024-05-27T07:30:05.584754+0000","last_undegraded":"2024-05-27T07:30:05.584754+0000","last_fullsized":"2024-05-27T07:30:05.584754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:15:51.449514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.767788+0000","last_change":"2024-05-27T07:28:24.478475+0000","last_active":"2024-05-27T07:31:33.767788+0000","last_peered":"2024-05-27T07:31:33.767788+0000","last_clean":"2024-05-27T07:31:33.767788+0000","last_became_active":"2024-05-27T07:28:24.477936+0000","last_became_peered":"2024-05-27T07:28:24.477936+0000","last_unstale":"2024-05-27T07:31:33.767788+0000","last_undegraded":"2024-05-27T07:31:33.767788+0000","last_fullsized":"2024-05-27T07:31:33.767788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:53:17.500040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119717+0000","last_change":"2024-05-27T07:28:37.968192+0000","last_active":"2024-05-27T07:30:05.119717+0000","last_peered":"2024-05-27T07:30:05.119717+0000","last_clean":"2024-05-27T07:30:05.119717+0000","last_became_active":"2024-05-27T07:28:37.967603+0000","last_became_peered":"2024-05-27T07:28:37.967603+0000","last_unstale":"2024-05-27T07:30:05.119717+0000","last_undegraded":"2024-05-27T07:30:05.119717+0000","last_fullsized":"2024-05-27T07:30:05.119717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:53:36.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":67,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:31.459304+0000","last_change":"2024-05-27T07:29:04.901062+0000","last_active":"2024-05-27T07:30:31.459304+0000","last_peered":"2024-05-27T07:30:31.459304+0000","last_clean":"2024-05-27T07:30:31.459304+0000","last_became_active":"2024-05-27T07:29:04.892060+0000","last_became_peered":"2024-05-27T07:29:04.892060+0000","last_unstale":"2024-05-27T07:30:31.459304+0000","last_undegraded":"2024-05-27T07:30:31.459304+0000","last_fullsized":"2024-05-27T07:30:31.459304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:27:37.894263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.778506+0000","last_change":"2024-05-27T07:28:24.477096+0000","last_active":"2024-05-27T07:31:33.778506+0000","last_peered":"2024-05-27T07:31:33.778506+0000","last_clean":"2024-05-27T07:31:33.778506+0000","last_became_active":"2024-05-27T07:28:24.476712+0000","last_became_peered":"2024-05-27T07:28:24.476712+0000","last_unstale":"2024-05-27T07:31:33.778506+0000","last_undegraded":"2024-05-27T07:31:33.778506+0000","last_fullsized":"2024-05-27T07:31:33.778506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:31:58.010761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119351+0000","last_change":"2024-05-27T07:28:37.894013+0000","last_active":"2024-05-27T07:30:05.119351+0000","last_peered":"2024-05-27T07:30:05.119351+0000","last_clean":"2024-05-27T07:30:05.119351+0000","last_became_active":"2024-05-27T07:28:37.893345+0000","last_became_peered":"2024-05-27T07:28:37.893345+0000","last_unstale":"2024-05-27T07:30:05.119351+0000","last_undegraded":"2024-05-27T07:30:05.119351+0000","last_fullsized":"2024-05-27T07:30:05.119351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:26.917731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537802+0000","last_change":"2024-05-27T07:28:20.744087+0000","last_active":"2024-05-27T07:30:05.537802+0000","last_peered":"2024-05-27T07:30:05.537802+0000","last_clean":"2024-05-27T07:30:05.537802+0000","last_became_active":"2024-05-27T07:28:20.742883+0000","last_became_peered":"2024-05-27T07:28:20.742883+0000","last_unstale":"2024-05-27T07:30:05.537802+0000","last_undegraded":"2024-05-27T07:30:05.537802+0000","last_fullsized":"2024-05-27T07:30:05.537802+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:06.083348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776817+0000","last_change":"2024-05-27T07:28:24.480330+0000","last_active":"2024-05-27T07:31:33.776817+0000","last_peered":"2024-05-27T07:31:33.776817+0000","last_clean":"2024-05-27T07:31:33.776817+0000","last_became_active":"2024-05-27T07:28:24.479300+0000","last_became_peered":"2024-05-27T07:28:24.479300+0000","last_unstale":"2024-05-27T07:31:33.776817+0000","last_undegraded":"2024-05-27T07:31:33.776817+0000","last_fullsized":"2024-05-27T07:31:33.776817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:43.802273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537207+0000","last_change":"2024-05-27T07:28:37.973592+0000","last_active":"2024-05-27T07:30:05.537207+0000","last_peered":"2024-05-27T07:30:05.537207+0000","last_clean":"2024-05-27T07:30:05.537207+0000","last_became_active":"2024-05-27T07:28:37.972475+0000","last_became_peered":"2024-05-27T07:28:37.972475+0000","last_unstale":"2024-05-27T07:30:05.537207+0000","last_undegraded":"2024-05-27T07:30:05.537207+0000","last_fullsized":"2024-05-27T07:30:05.537207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:31.662384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122331+0000","last_change":"2024-05-27T07:29:04.890751+0000","last_active":"2024-05-27T07:30:05.122331+0000","last_peered":"2024-05-27T07:30:05.122331+0000","last_clean":"2024-05-27T07:30:05.122331+0000","last_became_active":"2024-05-27T07:29:04.889447+0000","last_became_peered":"2024-05-27T07:29:04.889447+0000","last_unstale":"2024-05-27T07:30:05.122331+0000","last_undegraded":"2024-05-27T07:30:05.122331+0000","last_fullsized":"2024-05-27T07:30:05.122331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:13:34.743691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.703604+0000","last_change":"2024-05-27T07:28:24.474298+0000","last_active":"2024-05-27T07:31:33.703604+0000","last_peered":"2024-05-27T07:31:33.703604+0000","last_clean":"2024-05-27T07:31:33.703604+0000","last_became_active":"2024-05-27T07:28:24.473877+0000","last_became_peered":"2024-05-27T07:28:24.473877+0000","last_unstale":"2024-05-27T07:31:33.703604+0000","last_undegraded":"2024-05-27T07:31:33.703604+0000","last_fullsized":"2024-05-27T07:31:33.703604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:31.545883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.127532+0000","last_change":"2024-05-27T07:28:37.905417+0000","last_active":"2024-05-27T07:30:05.127532+0000","last_peered":"2024-05-27T07:30:05.127532+0000","last_clean":"2024-05-27T07:30:05.127532+0000","last_became_active":"2024-05-27T07:28:37.904968+0000","last_became_peered":"2024-05-27T07:28:37.904968+0000","last_unstale":"2024-05-27T07:30:05.127532+0000","last_undegraded":"2024-05-27T07:30:05.127532+0000","last_fullsized":"2024-05-27T07:30:05.127532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:02.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120278+0000","last_change":"2024-05-27T07:28:20.392647+0000","last_active":"2024-05-27T07:30:05.120278+0000","last_peered":"2024-05-27T07:30:05.120278+0000","last_clean":"2024-05-27T07:30:05.120278+0000","last_became_active":"2024-05-27T07:28:20.392073+0000","last_became_peered":"2024-05-27T07:28:20.392073+0000","last_unstale":"2024-05-27T07:30:05.120278+0000","last_undegraded":"2024-05-27T07:30:05.120278+0000","last_fullsized":"2024-05-27T07:30:05.120278+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:18.777628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.757595+0000","last_change":"2024-05-27T07:28:24.490522+0000","last_active":"2024-05-27T07:31:33.757595+0000","last_peered":"2024-05-27T07:31:33.757595+0000","last_clean":"2024-05-27T07:31:33.757595+0000","last_became_active":"2024-05-27T07:28:24.485228+0000","last_became_peered":"2024-05-27T07:28:24.485228+0000","last_unstale":"2024-05-27T07:31:33.757595+0000","last_undegraded":"2024-05-27T07:31:33.757595+0000","last_fullsized":"2024-05-27T07:31:33.757595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:14.221016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:13.384143+0000","last_change":"2024-05-27T07:28:37.900449+0000","last_active":"2024-05-27T07:30:13.384143+0000","last_peered":"2024-05-27T07:30:13.384143+0000","last_clean":"2024-05-27T07:30:13.384143+0000","last_became_active":"2024-05-27T07:28:37.899526+0000","last_became_peered":"2024-05-27T07:28:37.899526+0000","last_unstale":"2024-05-27T07:30:13.384143+0000","last_undegraded":"2024-05-27T07:30:13.384143+0000","last_fullsized":"2024-05-27T07:30:13.384143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:29:49.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.119900+0000","last_change":"2024-05-27T07:28:20.405236+0000","last_active":"2024-05-27T07:30:05.119900+0000","last_peered":"2024-05-27T07:30:05.119900+0000","last_clean":"2024-05-27T07:30:05.119900+0000","last_became_active":"2024-05-27T07:28:20.404619+0000","last_became_peered":"2024-05-27T07:28:20.404619+0000","last_unstale":"2024-05-27T07:30:05.119900+0000","last_undegraded":"2024-05-27T07:30:05.119900+0000","last_fullsized":"2024-05-27T07:30:05.119900+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:17.671250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.762832+0000","last_change":"2024-05-27T07:28:24.490661+0000","last_active":"2024-05-27T07:31:33.762832+0000","last_peered":"2024-05-27T07:31:33.762832+0000","last_clean":"2024-05-27T07:31:33.762832+0000","last_became_active":"2024-05-27T07:28:24.487299+0000","last_became_peered":"2024-05-27T07:28:24.487299+0000","last_unstale":"2024-05-27T07:31:33.762832+0000","last_undegraded":"2024-05-27T07:31:33.762832+0000","last_fullsized":"2024-05-27T07:31:33.762832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:50:27.291721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.537106+0000","last_change":"2024-05-27T07:28:37.970463+0000","last_active":"2024-05-27T07:30:05.537106+0000","last_peered":"2024-05-27T07:30:05.537106+0000","last_clean":"2024-05-27T07:30:05.537106+0000","last_became_active":"2024-05-27T07:28:37.968068+0000","last_became_peered":"2024-05-27T07:28:37.968068+0000","last_unstale":"2024-05-27T07:30:05.537106+0000","last_undegraded":"2024-05-27T07:30:05.537106+0000","last_fullsized":"2024-05-27T07:30:05.537106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:22:00.825999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.580250+0000","last_change":"2024-05-27T07:28:20.406203+0000","last_active":"2024-05-27T07:30:05.580250+0000","last_peered":"2024-05-27T07:30:05.580250+0000","last_clean":"2024-05-27T07:30:05.580250+0000","last_became_active":"2024-05-27T07:28:20.405404+0000","last_became_peered":"2024-05-27T07:28:20.405404+0000","last_unstale":"2024-05-27T07:30:05.580250+0000","last_undegraded":"2024-05-27T07:30:05.580250+0000","last_fullsized":"2024-05-27T07:30:05.580250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:15.274419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.767110+0000","last_change":"2024-05-27T07:28:24.473747+0000","last_active":"2024-05-27T07:31:33.767110+0000","last_peered":"2024-05-27T07:31:33.767110+0000","last_clean":"2024-05-27T07:31:33.767110+0000","last_became_active":"2024-05-27T07:28:24.473313+0000","last_became_peered":"2024-05-27T07:28:24.473313+0000","last_unstale":"2024-05-27T07:31:33.767110+0000","last_undegraded":"2024-05-27T07:31:33.767110+0000","last_fullsized":"2024-05-27T07:31:33.767110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:19:40.107840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.122107+0000","last_change":"2024-05-27T07:28:37.907410+0000","last_active":"2024-05-27T07:30:05.122107+0000","last_peered":"2024-05-27T07:30:05.122107+0000","last_clean":"2024-05-27T07:30:05.122107+0000","last_became_active":"2024-05-27T07:28:37.906753+0000","last_became_peered":"2024-05-27T07:28:37.906753+0000","last_unstale":"2024-05-27T07:30:05.122107+0000","last_undegraded":"2024-05-27T07:30:05.122107+0000","last_fullsized":"2024-05-27T07:30:05.122107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:53.612015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.120654+0000","last_change":"2024-05-27T07:28:20.744068+0000","last_active":"2024-05-27T07:30:05.120654+0000","last_peered":"2024-05-27T07:30:05.120654+0000","last_clean":"2024-05-27T07:30:05.120654+0000","last_became_active":"2024-05-27T07:28:20.742686+0000","last_became_peered":"2024-05-27T07:28:20.742686+0000","last_unstale":"2024-05-27T07:30:05.120654+0000","last_undegraded":"2024-05-27T07:30:05.120654+0000","last_fullsized":"2024-05-27T07:30:05.120654+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:54.361007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.776298+0000","last_change":"2024-05-27T07:28:24.475743+0000","last_active":"2024-05-27T07:31:33.776298+0000","last_peered":"2024-05-27T07:31:33.776298+0000","last_clean":"2024-05-27T07:31:33.776298+0000","last_became_active":"2024-05-27T07:28:24.474742+0000","last_became_peered":"2024-05-27T07:28:24.474742+0000","last_unstale":"2024-05-27T07:31:33.776298+0000","last_undegraded":"2024-05-27T07:31:33.776298+0000","last_fullsized":"2024-05-27T07:31:33.776298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:58:47.061573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.576290+0000","last_change":"2024-05-27T07:28:37.914935+0000","last_active":"2024-05-27T07:30:05.576290+0000","last_peered":"2024-05-27T07:30:05.576290+0000","last_clean":"2024-05-27T07:30:05.576290+0000","last_became_active":"2024-05-27T07:28:37.911829+0000","last_became_peered":"2024-05-27T07:28:37.911829+0000","last_unstale":"2024-05-27T07:30:05.576290+0000","last_undegraded":"2024-05-27T07:30:05.576290+0000","last_fullsized":"2024-05-27T07:30:05.576290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:29.260539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.584461+0000","last_change":"2024-05-27T07:28:20.412554+0000","last_active":"2024-05-27T07:30:05.584461+0000","last_peered":"2024-05-27T07:30:05.584461+0000","last_clean":"2024-05-27T07:30:05.584461+0000","last_became_active":"2024-05-27T07:28:20.412024+0000","last_became_peered":"2024-05-27T07:28:20.412024+0000","last_unstale":"2024-05-27T07:30:05.584461+0000","last_undegraded":"2024-05-27T07:30:05.584461+0000","last_fullsized":"2024-05-27T07:30:05.584461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:29.244093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.778000+0000","last_change":"2024-05-27T07:28:24.481966+0000","last_active":"2024-05-27T07:31:33.778000+0000","last_peered":"2024-05-27T07:31:33.778000+0000","last_clean":"2024-05-27T07:31:33.778000+0000","last_became_active":"2024-05-27T07:28:24.480775+0000","last_became_peered":"2024-05-27T07:28:24.480775+0000","last_unstale":"2024-05-27T07:31:33.778000+0000","last_undegraded":"2024-05-27T07:31:33.778000+0000","last_fullsized":"2024-05-27T07:31:33.778000+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:03.018174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.578887+0000","last_change":"2024-05-27T07:28:37.902893+0000","last_active":"2024-05-27T07:30:05.578887+0000","last_peered":"2024-05-27T07:30:05.578887+0000","last_clean":"2024-05-27T07:30:05.578887+0000","last_became_active":"2024-05-27T07:28:37.898447+0000","last_became_peered":"2024-05-27T07:28:37.898447+0000","last_unstale":"2024-05-27T07:30:05.578887+0000","last_undegraded":"2024-05-27T07:30:05.578887+0000","last_fullsized":"2024-05-27T07:30:05.578887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:55:49.754803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:30:05.407029+0000","last_change":"2024-05-27T07:28:20.408072+0000","last_active":"2024-05-27T07:30:05.407029+0000","last_peered":"2024-05-27T07:30:05.407029+0000","last_clean":"2024-05-27T07:30:05.407029+0000","last_became_active":"2024-05-27T07:28:20.407174+0000","last_became_peered":"2024-05-27T07:28:20.407174+0000","last_unstale":"2024-05-27T07:30:05.407029+0000","last_undegraded":"2024-05-27T07:30:05.407029+0000","last_fullsized":"2024-05-27T07:30:05.407029+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:15:52.984967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.761250+0000","last_change":"2024-05-27T07:28:24.475891+0000","last_active":"2024-05-27T07:31:33.761250+0000","last_peered":"2024-05-27T07:31:33.761250+0000","last_clean":"2024-05-27T07:31:33.761250+0000","last_became_active":"2024-05-27T07:28:24.475498+0000","last_became_peered":"2024-05-27T07:28:24.475498+0000","last_unstale":"2024-05-27T07:31:33.761250+0000","last_undegraded":"2024-05-27T07:31:33.761250+0000","last_fullsized":"2024-05-27T07:31:33.761250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:50.669118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.761917+0000","last_change":"2024-05-27T07:29:04.891583+0000","last_active":"2024-05-27T07:31:33.761917+0000","last_peered":"2024-05-27T07:31:33.761917+0000","last_clean":"2024-05-27T07:31:33.761917+0000","last_became_active":"2024-05-27T07:29:04.890187+0000","last_became_peered":"2024-05-27T07:29:04.890187+0000","last_unstale":"2024-05-27T07:31:33.761917+0000","last_undegraded":"2024-05-27T07:31:33.761917+0000","last_fullsized":"2024-05-27T07:31:33.761917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:03.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.753001+0000","last_change":"2024-05-27T07:28:24.478672+0000","last_active":"2024-05-27T07:31:33.753001+0000","last_peered":"2024-05-27T07:31:33.753001+0000","last_clean":"2024-05-27T07:31:33.753001+0000","last_became_active":"2024-05-27T07:28:24.478331+0000","last_became_peered":"2024-05-27T07:28:24.478331+0000","last_unstale":"2024-05-27T07:31:33.753001+0000","last_undegraded":"2024-05-27T07:31:33.753001+0000","last_fullsized":"2024-05-27T07:31:33.753001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:56:35.112855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.775323+0000","last_change":"2024-05-27T07:28:24.487962+0000","last_active":"2024-05-27T07:31:33.775323+0000","last_peered":"2024-05-27T07:31:33.775323+0000","last_clean":"2024-05-27T07:31:33.775323+0000","last_became_active":"2024-05-27T07:28:24.487583+0000","last_became_peered":"2024-05-27T07:28:24.487583+0000","last_unstale":"2024-05-27T07:31:33.775323+0000","last_undegraded":"2024-05-27T07:31:33.775323+0000","last_fullsized":"2024-05-27T07:31:33.775323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:25.758293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.765769+0000","last_change":"2024-05-27T07:28:24.466058+0000","last_active":"2024-05-27T07:31:33.765769+0000","last_peered":"2024-05-27T07:31:33.765769+0000","last_clean":"2024-05-27T07:31:33.765769+0000","last_became_active":"2024-05-27T07:28:24.465709+0000","last_became_peered":"2024-05-27T07:28:24.465709+0000","last_unstale":"2024-05-27T07:31:33.765769+0000","last_undegraded":"2024-05-27T07:31:33.765769+0000","last_fullsized":"2024-05-27T07:31:33.765769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:38.064077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.765176+0000","last_change":"2024-05-27T07:28:24.471585+0000","last_active":"2024-05-27T07:31:33.765176+0000","last_peered":"2024-05-27T07:31:33.765176+0000","last_clean":"2024-05-27T07:31:33.765176+0000","last_became_active":"2024-05-27T07:28:24.470948+0000","last_became_peered":"2024-05-27T07:28:24.470948+0000","last_unstale":"2024-05-27T07:31:33.765176+0000","last_undegraded":"2024-05-27T07:31:33.765176+0000","last_fullsized":"2024-05-27T07:31:33.765176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:48.213323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.745710+0000","last_change":"2024-05-27T07:28:24.477954+0000","last_active":"2024-05-27T07:31:33.745710+0000","last_peered":"2024-05-27T07:31:33.745710+0000","last_clean":"2024-05-27T07:31:33.745710+0000","last_became_active":"2024-05-27T07:28:24.477495+0000","last_became_peered":"2024-05-27T07:28:24.477495+0000","last_unstale":"2024-05-27T07:31:33.745710+0000","last_undegraded":"2024-05-27T07:31:33.745710+0000","last_fullsized":"2024-05-27T07:31:33.745710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:50.371439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.769895+0000","last_change":"2024-05-27T07:28:24.476422+0000","last_active":"2024-05-27T07:31:33.769895+0000","last_peered":"2024-05-27T07:31:33.769895+0000","last_clean":"2024-05-27T07:31:33.769895+0000","last_became_active":"2024-05-27T07:28:24.476095+0000","last_became_peered":"2024-05-27T07:28:24.476095+0000","last_unstale":"2024-05-27T07:31:33.769895+0000","last_undegraded":"2024-05-27T07:31:33.769895+0000","last_fullsized":"2024-05-27T07:31:33.769895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:51.667169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.774521+0000","last_change":"2024-05-27T07:28:24.486431+0000","last_active":"2024-05-27T07:31:33.774521+0000","last_peered":"2024-05-27T07:31:33.774521+0000","last_clean":"2024-05-27T07:31:33.774521+0000","last_became_active":"2024-05-27T07:28:24.482427+0000","last_became_peered":"2024-05-27T07:28:24.482427+0000","last_unstale":"2024-05-27T07:31:33.774521+0000","last_undegraded":"2024-05-27T07:31:33.774521+0000","last_fullsized":"2024-05-27T07:31:33.774521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:44:37.857793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.775665+0000","last_change":"2024-05-27T07:28:24.484393+0000","last_active":"2024-05-27T07:31:33.775665+0000","last_peered":"2024-05-27T07:31:33.775665+0000","last_clean":"2024-05-27T07:31:33.775665+0000","last_became_active":"2024-05-27T07:28:24.483130+0000","last_became_peered":"2024-05-27T07:28:24.483130+0000","last_unstale":"2024-05-27T07:31:33.775665+0000","last_undegraded":"2024-05-27T07:31:33.775665+0000","last_fullsized":"2024-05-27T07:31:33.775665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:34.924573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.779162+0000","last_change":"2024-05-27T07:28:24.486274+0000","last_active":"2024-05-27T07:31:33.779162+0000","last_peered":"2024-05-27T07:31:33.779162+0000","last_clean":"2024-05-27T07:31:33.779162+0000","last_became_active":"2024-05-27T07:28:24.485880+0000","last_became_peered":"2024-05-27T07:28:24.485880+0000","last_unstale":"2024-05-27T07:31:33.779162+0000","last_undegraded":"2024-05-27T07:31:33.779162+0000","last_fullsized":"2024-05-27T07:31:33.779162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:33.287615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.779769+0000","last_change":"2024-05-27T07:28:24.477909+0000","last_active":"2024-05-27T07:31:33.779769+0000","last_peered":"2024-05-27T07:31:33.779769+0000","last_clean":"2024-05-27T07:31:33.779769+0000","last_became_active":"2024-05-27T07:28:24.477345+0000","last_became_peered":"2024-05-27T07:28:24.477345+0000","last_unstale":"2024-05-27T07:31:33.779769+0000","last_undegraded":"2024-05-27T07:31:33.779769+0000","last_fullsized":"2024-05-27T07:31:33.779769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:26:03.958489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.775475+0000","last_change":"2024-05-27T07:28:24.477151+0000","last_active":"2024-05-27T07:31:33.775475+0000","last_peered":"2024-05-27T07:31:33.775475+0000","last_clean":"2024-05-27T07:31:33.775475+0000","last_became_active":"2024-05-27T07:28:24.474036+0000","last_became_peered":"2024-05-27T07:28:24.474036+0000","last_unstale":"2024-05-27T07:31:33.775475+0000","last_undegraded":"2024-05-27T07:31:33.775475+0000","last_fullsized":"2024-05-27T07:31:33.775475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:16.787252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:31:33.757111+0000","last_change":"2024-05-27T07:28:24.483601+0000","last_active":"2024-05-27T07:31:33.757111+0000","last_peered":"2024-05-27T07:31:33.757111+0000","last_clean":"2024-05-27T07:31:33.757111+0000","last_became_active":"2024-05-27T07:28:24.481080+0000","last_became_peered":"2024-05-27T07:28:24.481080+0000","last_unstale":"2024-05-27T07:31:33.757111+0000","last_undegraded":"2024-05-27T07:31:33.757111+0000","last_fullsized":"2024-05-27T07:31:33.757111+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:12.208506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72816,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":762,"num_read_kb":458,"num_write":176,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":218448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645374,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":522520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841604,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28716,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716436,"statfs":{"total":95995035648,"available":95965630464,"internally_reserved":0,"allocated":1740800,"data_stored":1178994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005133,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":526852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201365,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":522676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397597,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32792,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712360,"statfs":{"total":95995035648,"available":95961456640,"internally_reserved":0,"allocated":1654784,"data_stored":1118194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561123,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":517932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":587048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738511,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":519320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":588398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149818,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":518126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953586,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757356,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1662976,"data_stored":1113782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:31:59.672 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T07:31:59.672 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-27T07:31:59.672 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-27T07:31:59.672 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-05-27T07:32:00.196 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:00.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:31:59 smithi086 ceph-mon[34687]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:32:00.205 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:31:59 smithi149 ceph-mon[35408]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:32:00.207 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:31:59 smithi063 ceph-mon[31613]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:32:01.201 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:00 smithi086 ceph-mon[34687]: pgmap v499: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:01.205 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:00 smithi149 ceph-mon[35408]: pgmap v499: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:01.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:00 smithi063 ceph-mon[31613]: pgmap v499: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:02.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.644+0000 7f13cc400700 1 -- 172.21.15.63:0/1906442090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c4105e00 msgr2=0x7f13c410e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:02.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.644+0000 7f13cc400700 1 --2- 172.21.15.63:0/1906442090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c4105e00 0x7f13c410e350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009990 tx=0x7f13b0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.644+0000 7f13cc400700 1 -- 172.21.15.63:0/1906442090 shutdown_connections 2024-05-27T07:32:02.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.644+0000 7f13cc400700 1 --2- 172.21.15.63:0/1906442090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c410e8e0 0x7f13c4110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.644+0000 7f13cc400700 1 --2- 172.21.15.63:0/1906442090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c41054b0 0x7f13c41058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.644+0000 7f13cc400700 1 --2- 172.21.15.63:0/1906442090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c4105e00 0x7f13c410e350 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.644+0000 7f13cc400700 1 -- 172.21.15.63:0/1906442090 >> 172.21.15.63:0/1906442090 conn(0x7f13c40fedb0 msgr2=0x7f13c41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:02.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.644+0000 7f13cc400700 1 -- 172.21.15.63:0/1906442090 shutdown_connections 2024-05-27T07:32:02.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.644+0000 7f13cc400700 1 -- 172.21.15.63:0/1906442090 wait complete. 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.645+0000 7f13cc400700 1 Processor -- start 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.646+0000 7f13cc400700 1 -- start start 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.646+0000 7f13cc400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41054b0 0x7f13c419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.646+0000 7f13cc400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c4105e00 0x7f13c419d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.646+0000 7f13cc400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c410e8e0 0x7f13c41a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.646+0000 7f13cc400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13c4114340 con 0x7f13c4105e00 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.646+0000 7f13cc400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13c41141c0 con 0x7f13c41054b0 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.646+0000 7f13cc400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13c41144c0 con 0x7f13c410e8e0 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.646+0000 7f13c9000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c4105e00 0x7f13c419d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.646+0000 7f13c9000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c4105e00 0x7f13c419d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40552/0 (socket says 172.21.15.63:40552) 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.646+0000 7f13c9000700 1 -- 172.21.15.63:0/1254288338 learned_addr learned my addr 172.21.15.63:0/1254288338 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:02.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.647+0000 7f13c9a00700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41054b0 0x7f13c419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:02.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.647+0000 7f13ca400700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c410e8e0 0x7f13c41a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:02.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.647+0000 7f13c9000700 1 -- 172.21.15.63:0/1254288338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c410e8e0 msgr2=0x7f13c41a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:02.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.647+0000 7f13c9000700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c410e8e0 0x7f13c41a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.647+0000 7f13c9000700 1 -- 172.21.15.63:0/1254288338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41054b0 msgr2=0x7f13c419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:02.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.647+0000 7f13c9000700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41054b0 0x7f13c419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.647+0000 7f13c9000700 1 -- 172.21.15.63:0/1254288338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c41a1e40 con 0x7f13c4105e00 2024-05-27T07:32:02.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.647+0000 7f13ca400700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c410e8e0 0x7f13c41a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:02.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.648+0000 7f13c9000700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c4105e00 0x7f13c419d510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f13b0000c00 tx=0x7f13b00057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:02.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.648+0000 7f13c2000700 1 -- 172.21.15.63:0/1254288338 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0027070 con 0x7f13c4105e00 2024-05-27T07:32:02.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.648+0000 7f13c2000700 1 -- 172.21.15.63:0/1254288338 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13b000fb00 con 0x7f13c4105e00 2024-05-27T07:32:02.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.648+0000 7f13cc400700 1 -- 172.21.15.63:0/1254288338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c41a20d0 con 0x7f13c4105e00 2024-05-27T07:32:02.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.648+0000 7f13c2000700 1 -- 172.21.15.63:0/1254288338 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0022c00 con 0x7f13c4105e00 2024-05-27T07:32:02.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.648+0000 7f13cc400700 1 -- 172.21.15.63:0/1254288338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c41a9a50 con 0x7f13c4105e00 2024-05-27T07:32:02.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.650+0000 7f13cc400700 1 -- 172.21.15.63:0/1254288338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c40512b0 con 0x7f13c4105e00 2024-05-27T07:32:02.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.652+0000 7f13c2000700 1 -- 172.21.15.63:0/1254288338 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f13b0022470 con 0x7f13c4105e00 2024-05-27T07:32:02.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.654+0000 7f13c2000700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f13ac0b2900 0x7f13ac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:02.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.654+0000 7f13c2000700 1 -- 172.21.15.63:0/1254288338 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f13b00e33f0 con 0x7f13c4105e00 2024-05-27T07:32:02.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.654+0000 7f13c9a00700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f13ac0b2900 0x7f13ac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:02.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.655+0000 7f13c9a00700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f13ac0b2900 0x7f13ac0b4e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f13b8005950 tx=0x7f13b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:02.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.659+0000 7f13c2000700 1 -- 172.21.15.63:0/1254288338 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f13b0013070 con 0x7f13c4105e00 2024-05-27T07:32:02.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.946+0000 7f13cc400700 1 -- 172.21.15.63:0/1254288338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f13c419e2d0 con 0x7f13c4105e00 2024-05-27T07:32:02.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.947+0000 7f13c2000700 1 -- 172.21.15.63:0/1254288338 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f13b00aae10 con 0x7f13c4105e00 2024-05-27T07:32:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:32:02.940 INFO:teuthology.orchestra.run.smithi063.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-27T07:32:02.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.950+0000 7f13cc400700 1 -- 172.21.15.63:0/1254288338 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f13ac0b2900 msgr2=0x7f13ac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:02.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.950+0000 7f13cc400700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f13ac0b2900 0x7f13ac0b4e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f13b8005950 tx=0x7f13b800b410 comp rx=0 tx=0).stop 2024-05-27T07:32:02.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.950+0000 7f13cc400700 1 -- 172.21.15.63:0/1254288338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c4105e00 msgr2=0x7f13c419d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:02.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.950+0000 7f13cc400700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c4105e00 0x7f13c419d510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f13b0000c00 tx=0x7f13b00057a0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.950+0000 7f13cc400700 1 -- 172.21.15.63:0/1254288338 shutdown_connections 2024-05-27T07:32:02.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.950+0000 7f13cc400700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c410e8e0 0x7f13c41a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.950+0000 7f13cc400700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41054b0 0x7f13c419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.950+0000 7f13cc400700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f13ac0b2900 0x7f13ac0b4e10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.950+0000 7f13cc400700 1 --2- 172.21.15.63:0/1254288338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c4105e00 0x7f13c419d510 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:02.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.950+0000 7f13cc400700 1 -- 172.21.15.63:0/1254288338 >> 172.21.15.63:0/1254288338 conn(0x7f13c40fedb0 msgr2=0x7f13c40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:02.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.951+0000 7f13cc400700 1 -- 172.21.15.63:0/1254288338 shutdown_connections 2024-05-27T07:32:02.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:02.951+0000 7f13cc400700 1 -- 172.21.15.63:0/1254288338 wait complete. 2024-05-27T07:32:03.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:02 smithi063 ceph-mon[31613]: pgmap v500: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:03.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:02 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1254288338' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T07:32:03.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:02 smithi086 ceph-mon[34687]: pgmap v500: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:03.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:02 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1254288338' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T07:32:03.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:02 smithi149 ceph-mon[35408]: pgmap v500: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:03.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:02 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1254288338' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T07:32:03.865 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-27T07:32:03.866 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:32:04.301 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:05.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:04 smithi063 ceph-mon[31613]: pgmap v501: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:05.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:04 smithi086 ceph-mon[34687]: pgmap v501: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:05.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:04 smithi149 ceph-mon[35408]: pgmap v501: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.694+0000 7faf33000700 1 -- 172.21.15.63:0/3712873460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf2c106ec0 msgr2=0x7faf2c113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.694+0000 7faf33000700 1 --2- 172.21.15.63:0/3712873460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf2c106ec0 0x7faf2c113350 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf2400b0a0 tx=0x7faf2400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.694+0000 7faf33000700 1 -- 172.21.15.63:0/3712873460 shutdown_connections 2024-05-27T07:32:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.694+0000 7faf33000700 1 --2- 172.21.15.63:0/3712873460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf2c106ec0 0x7faf2c113350 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.694+0000 7faf33000700 1 --2- 172.21.15.63:0/3712873460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c10c5b0 0x7faf2c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.694+0000 7faf33000700 1 --2- 172.21.15.63:0/3712873460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf2c106570 0x7faf2c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.694+0000 7faf33000700 1 -- 172.21.15.63:0/3712873460 >> 172.21.15.63:0/3712873460 conn(0x7faf2c102070 msgr2=0x7faf2c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.694+0000 7faf33000700 1 -- 172.21.15.63:0/3712873460 shutdown_connections 2024-05-27T07:32:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.694+0000 7faf33000700 1 -- 172.21.15.63:0/3712873460 wait complete. 2024-05-27T07:32:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.696+0000 7faf33000700 1 Processor -- start 2024-05-27T07:32:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.696+0000 7faf33000700 1 -- start start 2024-05-27T07:32:06.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.696+0000 7faf33000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf2c106570 0x7faf2c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:06.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.696+0000 7faf33000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf2c106ec0 0x7faf2c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:06.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.696+0000 7faf33000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c10c5b0 0x7faf2c1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.696+0000 7faf33000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1165e0 con 0x7faf2c106ec0 2024-05-27T07:32:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.696+0000 7faf33000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c116460 con 0x7faf2c106570 2024-05-27T07:32:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.696+0000 7faf33000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c116760 con 0x7faf2c10c5b0 2024-05-27T07:32:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.697+0000 7faf2b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf2c106ec0 0x7faf2c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.697+0000 7faf2b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf2c106ec0 0x7faf2c19d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40556/0 (socket says 172.21.15.63:40556) 2024-05-27T07:32:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.697+0000 7faf2b400700 1 -- 172.21.15.63:0/4225915479 learned_addr learned my addr 172.21.15.63:0/4225915479 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.697+0000 7faf2be00700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf2c106570 0x7faf2c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.697+0000 7faf31000700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c10c5b0 0x7faf2c1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.697+0000 7faf2b400700 1 -- 172.21.15.63:0/4225915479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c10c5b0 msgr2=0x7faf2c1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.697+0000 7faf2b400700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c10c5b0 0x7faf2c1a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.697+0000 7faf2b400700 1 -- 172.21.15.63:0/4225915479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf2c106570 msgr2=0x7faf2c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.697+0000 7faf2b400700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf2c106570 0x7faf2c19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.697+0000 7faf2b400700 1 -- 172.21.15.63:0/4225915479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c1a1de0 con 0x7faf2c106ec0 2024-05-27T07:32:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.698+0000 7faf2be00700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf2c106570 0x7faf2c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.698+0000 7faf31000700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c10c5b0 0x7faf2c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.698+0000 7faf2b400700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf2c106ec0 0x7faf2c19d4b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf1800e940 tx=0x7faf1800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.698+0000 7faf28c00700 1 -- 172.21.15.63:0/4225915479 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1800cbd0 con 0x7faf2c106ec0 2024-05-27T07:32:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.698+0000 7faf28c00700 1 -- 172.21.15.63:0/4225915479 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf1800cd70 con 0x7faf2c106ec0 2024-05-27T07:32:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.698+0000 7faf28c00700 1 -- 172.21.15.63:0/4225915479 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18018680 con 0x7faf2c106ec0 2024-05-27T07:32:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.698+0000 7faf33000700 1 -- 172.21.15.63:0/4225915479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2c1a20d0 con 0x7faf2c106ec0 2024-05-27T07:32:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.699+0000 7faf33000700 1 -- 172.21.15.63:0/4225915479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2c1a99f0 con 0x7faf2c106ec0 2024-05-27T07:32:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.700+0000 7faf33000700 1 -- 172.21.15.63:0/4225915479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf2c0512b0 con 0x7faf2c106ec0 2024-05-27T07:32:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.703+0000 7faf28c00700 1 -- 172.21.15.63:0/4225915479 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7faf18010430 con 0x7faf2c106ec0 2024-05-27T07:32:06.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.705+0000 7faf28c00700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faf100b2900 0x7faf100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:06.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.706+0000 7faf2be00700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faf100b2900 0x7faf100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:06.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.706+0000 7faf28c00700 1 -- 172.21.15.63:0/4225915479 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7faf180e2b80 con 0x7faf2c106ec0 2024-05-27T07:32:06.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.706+0000 7faf2be00700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faf100b2900 0x7faf100b4e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf1c005950 tx=0x7faf1c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:06.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:06.707+0000 7faf28c00700 1 -- 172.21.15.63:0/4225915479 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf180aa4b0 con 0x7faf2c106ec0 2024-05-27T07:32:06.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.000+0000 7faf33000700 1 -- 172.21.15.63:0/4225915479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf2c19e270 con 0x7faf2c106ec0 2024-05-27T07:32:06.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.002+0000 7faf28c00700 1 -- 172.21.15.63:0/4225915479 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11861 (secure 0 0 0) 0x7faf180a9db0 con 0x7faf2c106ec0 2024-05-27T07:32:06.995 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:32:06.996 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:29:56.944716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T07:32:06.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 -- 172.21.15.63:0/4225915479 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faf100b2900 msgr2=0x7faf100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:06.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faf100b2900 0x7faf100b4e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf1c005950 tx=0x7faf1c0058e0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 -- 172.21.15.63:0/4225915479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf2c106ec0 msgr2=0x7faf2c19d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:06.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf2c106ec0 0x7faf2c19d4b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf1800e940 tx=0x7faf1800ed00 comp rx=0 tx=0).stop 2024-05-27T07:32:06.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 -- 172.21.15.63:0/4225915479 shutdown_connections 2024-05-27T07:32:06.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c10c5b0 0x7faf2c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf2c106570 0x7faf2c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7faf100b2900 0x7faf100b4e10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 --2- 172.21.15.63:0/4225915479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf2c106ec0 0x7faf2c19d4b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:06.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 -- 172.21.15.63:0/4225915479 >> 172.21.15.63:0/4225915479 conn(0x7faf2c102070 msgr2=0x7faf2c1046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:07.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 -- 172.21.15.63:0/4225915479 shutdown_connections 2024-05-27T07:32:07.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:07.006+0000 7faf33000700 1 -- 172.21.15.63:0/4225915479 wait complete. 2024-05-27T07:32:07.002 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 26 2024-05-27T07:32:07.365 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:06 smithi063 ceph-mon[31613]: pgmap v502: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:07.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:06 smithi086 ceph-mon[34687]: pgmap v502: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:07.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:06 smithi149 ceph-mon[35408]: pgmap v502: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:07.905 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:32:08.381 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:08.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:07 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4225915479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:32:08.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:07 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4225915479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:32:08.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:07 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4225915479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:32:09.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:08 smithi063 ceph-mon[31613]: pgmap v503: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:09.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:08 smithi086 ceph-mon[34687]: pgmap v503: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:09.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:08 smithi149 ceph-mon[35408]: pgmap v503: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:10.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.821+0000 7f030be00700 1 -- 172.21.15.63:0/819488854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f030c110790 msgr2=0x7f030c110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:10.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.821+0000 7f030be00700 1 --2- 172.21.15.63:0/819488854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f030c110790 0x7f030c110ba0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009990 tx=0x7f02fc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:10.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.821+0000 7f030be00700 1 -- 172.21.15.63:0/819488854 shutdown_connections 2024-05-27T07:32:10.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.821+0000 7f030be00700 1 --2- 172.21.15.63:0/819488854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c10b100 0x7f030c117630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:10.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.821+0000 7f030be00700 1 --2- 172.21.15.63:0/819488854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f030c110790 0x7f030c110ba0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:10.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.821+0000 7f030be00700 1 --2- 172.21.15.63:0/819488854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c10a7b0 0x7f030c10abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:10.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.821+0000 7f030be00700 1 -- 172.21.15.63:0/819488854 >> 172.21.15.63:0/819488854 conn(0x7f030c076af0 msgr2=0x7f030c076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:10.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.821+0000 7f030be00700 1 -- 172.21.15.63:0/819488854 shutdown_connections 2024-05-27T07:32:10.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.821+0000 7f030be00700 1 -- 172.21.15.63:0/819488854 wait complete. 2024-05-27T07:32:10.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.823+0000 7f030be00700 1 Processor -- start 2024-05-27T07:32:10.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.823+0000 7f030be00700 1 -- start start 2024-05-27T07:32:10.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.823+0000 7f030be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f030c10a7b0 0x7f030c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:10.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.823+0000 7f030be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c10b100 0x7f030c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:10.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.823+0000 7f030be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c110790 0x7f030c1a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:10.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.823+0000 7f030be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f030c11a780 con 0x7f030c10a7b0 2024-05-27T07:32:10.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.823+0000 7f030be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f030c11a600 con 0x7f030c10b100 2024-05-27T07:32:10.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.823+0000 7f030be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f030c11a900 con 0x7f030c110790 2024-05-27T07:32:10.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f030aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f030c10a7b0 0x7f030c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:10.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f030aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f030c10a7b0 0x7f030c1a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34254/0 (socket says 172.21.15.63:34254) 2024-05-27T07:32:10.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f030aa00700 1 -- 172.21.15.63:0/2111908507 learned_addr learned my addr 172.21.15.63:0/2111908507 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:10.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f0302000700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c10b100 0x7f030c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:10.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f030b400700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c110790 0x7f030c1a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:10.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f030aa00700 1 -- 172.21.15.63:0/2111908507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c110790 msgr2=0x7f030c1a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:10.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f030aa00700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c110790 0x7f030c1a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:10.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f030aa00700 1 -- 172.21.15.63:0/2111908507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c10b100 msgr2=0x7f030c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:10.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f030aa00700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c10b100 0x7f030c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:10.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f030aa00700 1 -- 172.21.15.63:0/2111908507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c1a6080 con 0x7f030c10a7b0 2024-05-27T07:32:10.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.824+0000 7f030b400700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c110790 0x7f030c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:32:10.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.825+0000 7f030aa00700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f030c10a7b0 0x7f030c1a11b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f02fc005910 tx=0x7f02fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:10.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.825+0000 7f0303e00700 1 -- 172.21.15.63:0/2111908507 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc027070 con 0x7f030c10a7b0 2024-05-27T07:32:10.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.825+0000 7f0303e00700 1 -- 172.21.15.63:0/2111908507 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02fc00fc00 con 0x7f030c10a7b0 2024-05-27T07:32:10.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.825+0000 7f0303e00700 1 -- 172.21.15.63:0/2111908507 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc022d40 con 0x7f030c10a7b0 2024-05-27T07:32:10.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.825+0000 7f030be00700 1 -- 172.21.15.63:0/2111908507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c1a6310 con 0x7f030c10a7b0 2024-05-27T07:32:10.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.825+0000 7f030be00700 1 -- 172.21.15.63:0/2111908507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c1a67a0 con 0x7f030c10a7b0 2024-05-27T07:32:10.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.826+0000 7f030be00700 1 -- 172.21.15.63:0/2111908507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030c0512b0 con 0x7f030c10a7b0 2024-05-27T07:32:10.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.829+0000 7f0303e00700 1 -- 172.21.15.63:0/2111908507 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f02fc02a070 con 0x7f030c10a7b0 2024-05-27T07:32:10.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.830+0000 7f0303e00700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f02f00b2900 0x7f02f00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:10.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.831+0000 7f0303e00700 1 -- 172.21.15.63:0/2111908507 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f02fc0e3ac0 con 0x7f030c10a7b0 2024-05-27T07:32:10.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.831+0000 7f0302000700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f02f00b2900 0x7f02f00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:10.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.832+0000 7f0302000700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f02f00b2900 0x7f02f00b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f02f4005950 tx=0x7f02f40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:10.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:10.833+0000 7f0303e00700 1 -- 172.21.15.63:0/2111908507 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02fc013070 con 0x7f030c10a7b0 2024-05-27T07:32:11.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.120+0000 7f030be00700 1 -- 172.21.15.63:0/2111908507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f030c02e5e0 con 0x7f030c10a7b0 2024-05-27T07:32:11.114 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:10 smithi063 ceph-mon[31613]: pgmap v504: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:11.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.122+0000 7f0303e00700 1 -- 172.21.15.63:0/2111908507 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11861 (secure 0 0 0) 0x7f02fc0ab480 con 0x7f030c10a7b0 2024-05-27T07:32:11.115 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:32:11.115 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:29:56.944716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T07:32:11.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.125+0000 7f030be00700 1 -- 172.21.15.63:0/2111908507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f02f00b2900 msgr2=0x7f02f00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:11.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.125+0000 7f030be00700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f02f00b2900 0x7f02f00b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f02f4005950 tx=0x7f02f40058e0 comp rx=0 tx=0).stop 2024-05-27T07:32:11.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.125+0000 7f030be00700 1 -- 172.21.15.63:0/2111908507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f030c10a7b0 msgr2=0x7f030c1a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:11.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.125+0000 7f030be00700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f030c10a7b0 0x7f030c1a11b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f02fc005910 tx=0x7f02fc0047b0 comp rx=0 tx=0).stop 2024-05-27T07:32:11.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.125+0000 7f030be00700 1 -- 172.21.15.63:0/2111908507 shutdown_connections 2024-05-27T07:32:11.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.125+0000 7f030be00700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f030c110790 0x7f030c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:11.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.126+0000 7f030be00700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f02f00b2900 0x7f02f00b4e10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:11.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.126+0000 7f030be00700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f030c10a7b0 0x7f030c1a11b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:11.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.126+0000 7f030be00700 1 --2- 172.21.15.63:0/2111908507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f030c10b100 0x7f030c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:11.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.126+0000 7f030be00700 1 -- 172.21.15.63:0/2111908507 >> 172.21.15.63:0/2111908507 conn(0x7f030c076af0 msgr2=0x7f030c108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:11.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.126+0000 7f030be00700 1 -- 172.21.15.63:0/2111908507 shutdown_connections 2024-05-27T07:32:11.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:11.126+0000 7f030be00700 1 -- 172.21.15.63:0/2111908507 wait complete. 2024-05-27T07:32:11.120 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 26 2024-05-27T07:32:11.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:10 smithi086 ceph-mon[34687]: pgmap v504: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:11.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:10 smithi149 ceph-mon[35408]: pgmap v504: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:11.958 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:32:12.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:12 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2111908507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:32:12.427 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:12.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:12 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2111908507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:32:12.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:12 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2111908507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:32:13.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:13 smithi063 ceph-mon[31613]: pgmap v505: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:13.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:13 smithi086 ceph-mon[34687]: pgmap v505: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:13.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:13 smithi149 ceph-mon[35408]: pgmap v505: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.747+0000 7fb693200700 1 -- 172.21.15.63:0/3123563604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb694102590 msgr2=0x7fb6941029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.747+0000 7fb693200700 1 --2- 172.21.15.63:0/3123563604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb694102590 0x7fb6941029a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb680009990 tx=0x7fb680009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.747+0000 7fb693200700 1 -- 172.21.15.63:0/3123563604 shutdown_connections 2024-05-27T07:32:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.747+0000 7fb693200700 1 --2- 172.21.15.63:0/3123563604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb694102ee0 0x7fb694110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.747+0000 7fb693200700 1 --2- 172.21.15.63:0/3123563604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb694109100 0x7fb694109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:14.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.747+0000 7fb693200700 1 --2- 172.21.15.63:0/3123563604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb694102590 0x7fb6941029a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.747+0000 7fb693200700 1 -- 172.21.15.63:0/3123563604 >> 172.21.15.63:0/3123563604 conn(0x7fb6940783a0 msgr2=0x7fb6940787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.747+0000 7fb693200700 1 -- 172.21.15.63:0/3123563604 shutdown_connections 2024-05-27T07:32:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.747+0000 7fb693200700 1 -- 172.21.15.63:0/3123563604 wait complete. 2024-05-27T07:32:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.749+0000 7fb693200700 1 Processor -- start 2024-05-27T07:32:14.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.749+0000 7fb693200700 1 -- start start 2024-05-27T07:32:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb693200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb694102590 0x7fb6941a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb693200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb694102ee0 0x7fb6941a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb693200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb694109100 0x7fb69419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb693200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb694114250 con 0x7fb694102ee0 2024-05-27T07:32:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb693200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941140d0 con 0x7fb694109100 2024-05-27T07:32:14.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb693200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941143d0 con 0x7fb694102590 2024-05-27T07:32:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb691400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb694102ee0 0x7fb6941a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb691400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb694102ee0 0x7fb6941a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34272/0 (socket says 172.21.15.63:34272) 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb691400700 1 -- 172.21.15.63:0/183482378 learned_addr learned my addr 172.21.15.63:0/183482378 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb692800700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb694109100 0x7fb69419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb691400700 1 -- 172.21.15.63:0/183482378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb694102590 msgr2=0x7fb6941a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb691400700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb694102590 0x7fb6941a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb691400700 1 -- 172.21.15.63:0/183482378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb694109100 msgr2=0x7fb69419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb691400700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb694109100 0x7fb69419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.750+0000 7fb691400700 1 -- 172.21.15.63:0/183482378 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69419b120 con 0x7fb694102ee0 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.751+0000 7fb692800700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb694109100 0x7fb69419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.751+0000 7fb691400700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb694102ee0 0x7fb6941a0d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb680000c00 tx=0x7fb680005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.751+0000 7fb68aa00700 1 -- 172.21.15.63:0/183482378 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680027070 con 0x7fb694102ee0 2024-05-27T07:32:14.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.751+0000 7fb68aa00700 1 -- 172.21.15.63:0/183482378 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb68000fd40 con 0x7fb694102ee0 2024-05-27T07:32:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.751+0000 7fb693200700 1 -- 172.21.15.63:0/183482378 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69419b3b0 con 0x7fb694102ee0 2024-05-27T07:32:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.751+0000 7fb68aa00700 1 -- 172.21.15.63:0/183482378 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680022ca0 con 0x7fb694102ee0 2024-05-27T07:32:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.752+0000 7fb693200700 1 -- 172.21.15.63:0/183482378 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69419b890 con 0x7fb694102ee0 2024-05-27T07:32:14.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.753+0000 7fb693200700 1 -- 172.21.15.63:0/183482378 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6940512b0 con 0x7fb694102ee0 2024-05-27T07:32:14.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.755+0000 7fb68aa00700 1 -- 172.21.15.63:0/183482378 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb68002a070 con 0x7fb694102ee0 2024-05-27T07:32:14.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.757+0000 7fb68aa00700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb6780b2880 0x7fb6780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:14.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.757+0000 7fb68aa00700 1 -- 172.21.15.63:0/183482378 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fb6800e3630 con 0x7fb694102ee0 2024-05-27T07:32:14.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.757+0000 7fb691e00700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb6780b2880 0x7fb6780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:14.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.758+0000 7fb691e00700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb6780b2880 0x7fb6780b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb69419bbb0 tx=0x7fb684009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:14.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:14.761+0000 7fb68aa00700 1 -- 172.21.15.63:0/183482378 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb680013070 con 0x7fb694102ee0 2024-05-27T07:32:14.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.000+0000 7fb693200700 1 -- 172.21.15.63:0/183482378 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb69402e5e0 con 0x7fb694102ee0 2024-05-27T07:32:14.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.005+0000 7fb68aa00700 1 -- 172.21.15.63:0/183482378 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24635 (secure 0 0 0) 0x7fb6800aaff0 con 0x7fb694102ee0 2024-05-27T07:32:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:32:14.998 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":91,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:30:05.109968+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6},{"from":3,"to":7},{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:32:15.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.008+0000 7fb693200700 1 -- 172.21.15.63:0/183482378 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb6780b2880 msgr2=0x7fb6780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:15.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.008+0000 7fb693200700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb6780b2880 0x7fb6780b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb69419bbb0 tx=0x7fb684009470 comp rx=0 tx=0).stop 2024-05-27T07:32:15.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.008+0000 7fb693200700 1 -- 172.21.15.63:0/183482378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb694102ee0 msgr2=0x7fb6941a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:15.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.008+0000 7fb693200700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb694102ee0 0x7fb6941a0d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb680000c00 tx=0x7fb680005ef0 comp rx=0 tx=0).stop 2024-05-27T07:32:15.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.008+0000 7fb693200700 1 -- 172.21.15.63:0/183482378 shutdown_connections 2024-05-27T07:32:15.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.008+0000 7fb693200700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb694109100 0x7fb69419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:15.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.008+0000 7fb693200700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb694102590 0x7fb6941a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:15.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.008+0000 7fb693200700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fb6780b2880 0x7fb6780b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:15.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.008+0000 7fb693200700 1 --2- 172.21.15.63:0/183482378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb694102ee0 0x7fb6941a0d90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:15.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.008+0000 7fb693200700 1 -- 172.21.15.63:0/183482378 >> 172.21.15.63:0/183482378 conn(0x7fb6940783a0 msgr2=0x7fb6941000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:15.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.009+0000 7fb693200700 1 -- 172.21.15.63:0/183482378 shutdown_connections 2024-05-27T07:32:15.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:15.009+0000 7fb693200700 1 -- 172.21.15.63:0/183482378 wait complete. 2024-05-27T07:32:15.397 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:15 smithi063 ceph-mon[31613]: pgmap v506: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:15.397 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:15 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/183482378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:32:15.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:15 smithi086 ceph-mon[34687]: pgmap v506: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:15.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:15 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/183482378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:32:15.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:15 smithi149 ceph-mon[35408]: pgmap v506: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:15.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:15 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/183482378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:32:15.767 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:32:16.233 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:17.306 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:17 smithi063 ceph-mon[31613]: pgmap v507: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:17.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:17 smithi086 ceph-mon[34687]: pgmap v507: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:17.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:17 smithi149 ceph-mon[35408]: pgmap v507: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:18.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.762+0000 7efca2200700 1 -- 172.21.15.63:0/3477595439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc9c10c5b0 msgr2=0x7efc9c10c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:18.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.762+0000 7efca2200700 1 --2- 172.21.15.63:0/3477595439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc9c10c5b0 0x7efc9c10c9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009990 tx=0x7efc8c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.762+0000 7efca2200700 1 -- 172.21.15.63:0/3477595439 shutdown_connections 2024-05-27T07:32:18.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.762+0000 7efca2200700 1 --2- 172.21.15.63:0/3477595439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc9c106ec0 0x7efc9c113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.762+0000 7efca2200700 1 --2- 172.21.15.63:0/3477595439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc9c10c5b0 0x7efc9c10c9c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.762+0000 7efca2200700 1 --2- 172.21.15.63:0/3477595439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc9c106570 0x7efc9c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.762+0000 7efca2200700 1 -- 172.21.15.63:0/3477595439 >> 172.21.15.63:0/3477595439 conn(0x7efc9c102070 msgr2=0x7efc9c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:18.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.762+0000 7efca2200700 1 -- 172.21.15.63:0/3477595439 shutdown_connections 2024-05-27T07:32:18.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.763+0000 7efca2200700 1 -- 172.21.15.63:0/3477595439 wait complete. 2024-05-27T07:32:18.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.764+0000 7efca2200700 1 Processor -- start 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.764+0000 7efca2200700 1 -- start start 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efca2200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc9c106570 0x7efc9c19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efca2200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc9c106ec0 0x7efc9c19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efca2200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc9c10c5b0 0x7efc9c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efca2200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1166b0 con 0x7efc9c10c5b0 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efca2200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c116530 con 0x7efc9c106570 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efca2200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c116830 con 0x7efc9c106ec0 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efc9be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc9c10c5b0 0x7efc9c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efc9be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc9c10c5b0 0x7efc9c1a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34296/0 (socket says 172.21.15.63:34296) 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efc9be00700 1 -- 172.21.15.63:0/128608210 learned_addr learned my addr 172.21.15.63:0/128608210 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:18.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efc9b400700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc9c106570 0x7efc9c19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:18.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efc9be00700 1 -- 172.21.15.63:0/128608210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc9c106ec0 msgr2=0x7efc9c19d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:18.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.765+0000 7efc9be00700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc9c106ec0 0x7efc9c19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.766+0000 7efc9be00700 1 -- 172.21.15.63:0/128608210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc9c106570 msgr2=0x7efc9c19d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:18.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.766+0000 7efc9be00700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc9c106570 0x7efc9c19d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.766+0000 7efc9be00700 1 -- 172.21.15.63:0/128608210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c1a2070 con 0x7efc9c10c5b0 2024-05-27T07:32:18.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.766+0000 7efc9be00700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc9c10c5b0 0x7efc9c1a19d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efc9400b780 tx=0x7efc9400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:18.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.766+0000 7efc98c00700 1 -- 172.21.15.63:0/128608210 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9400f840 con 0x7efc9c10c5b0 2024-05-27T07:32:18.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.766+0000 7efc98c00700 1 -- 172.21.15.63:0/128608210 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc94017400 con 0x7efc9c10c5b0 2024-05-27T07:32:18.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.766+0000 7efca2200700 1 -- 172.21.15.63:0/128608210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9c1a2360 con 0x7efc9c10c5b0 2024-05-27T07:32:18.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.766+0000 7efc98c00700 1 -- 172.21.15.63:0/128608210 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9400d630 con 0x7efc9c10c5b0 2024-05-27T07:32:18.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.766+0000 7efca2200700 1 -- 172.21.15.63:0/128608210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9c1a9c80 con 0x7efc9c10c5b0 2024-05-27T07:32:18.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.768+0000 7efca2200700 1 -- 172.21.15.63:0/128608210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc9c0512b0 con 0x7efc9c10c5b0 2024-05-27T07:32:18.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.770+0000 7efc98c00700 1 -- 172.21.15.63:0/128608210 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efc940175b0 con 0x7efc9c10c5b0 2024-05-27T07:32:18.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.772+0000 7efc98c00700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efc800b2900 0x7efc800b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:18.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.772+0000 7efc98c00700 1 -- 172.21.15.63:0/128608210 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7efc940e2420 con 0x7efc9c10c5b0 2024-05-27T07:32:18.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.772+0000 7efc9b400700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efc800b2900 0x7efc800b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:18.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.773+0000 7efc9b400700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efc800b2900 0x7efc800b4e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efc8c006010 tx=0x7efc8c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:18.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:18.777+0000 7efc98c00700 1 -- 172.21.15.63:0/128608210 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc94013030 con 0x7efc9c10c5b0 2024-05-27T07:32:19.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.067+0000 7efca2200700 1 -- 172.21.15.63:0/128608210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc9c19e500 con 0x7efc9c10c5b0 2024-05-27T07:32:19.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.068+0000 7efc98c00700 1 -- 172.21.15.63:0/128608210 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11861 (secure 0 0 0) 0x7efc940a9db0 con 0x7efc9c10c5b0 2024-05-27T07:32:19.062 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:32:19.062 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:29:56.944716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T07:32:19.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.071+0000 7efca2200700 1 -- 172.21.15.63:0/128608210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efc800b2900 msgr2=0x7efc800b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:19.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.071+0000 7efca2200700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efc800b2900 0x7efc800b4e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efc8c006010 tx=0x7efc8c00b540 comp rx=0 tx=0).stop 2024-05-27T07:32:19.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.071+0000 7efca2200700 1 -- 172.21.15.63:0/128608210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc9c10c5b0 msgr2=0x7efc9c1a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:19.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.071+0000 7efca2200700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc9c10c5b0 0x7efc9c1a19d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efc9400b780 tx=0x7efc9400bb40 comp rx=0 tx=0).stop 2024-05-27T07:32:19.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.071+0000 7efca2200700 1 -- 172.21.15.63:0/128608210 shutdown_connections 2024-05-27T07:32:19.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.071+0000 7efca2200700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7efc800b2900 0x7efc800b4e10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:19.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.071+0000 7efca2200700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc9c10c5b0 0x7efc9c1a19d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:19.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.071+0000 7efca2200700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc9c106570 0x7efc9c19d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:19.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.071+0000 7efca2200700 1 --2- 172.21.15.63:0/128608210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc9c106ec0 0x7efc9c19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:19.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.072+0000 7efca2200700 1 -- 172.21.15.63:0/128608210 >> 172.21.15.63:0/128608210 conn(0x7efc9c102070 msgr2=0x7efc9c1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:19.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.072+0000 7efca2200700 1 -- 172.21.15.63:0/128608210 shutdown_connections 2024-05-27T07:32:19.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:19.072+0000 7efca2200700 1 -- 172.21.15.63:0/128608210 wait complete. 2024-05-27T07:32:19.066 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 26 2024-05-27T07:32:19.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:19 smithi086 ceph-mon[34687]: pgmap v508: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:19.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:19 smithi149 ceph-mon[35408]: pgmap v508: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:19.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:19 smithi063 ceph-mon[31613]: pgmap v508: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:20.073 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T07:28:27.557349+0000', 'modified': '2024-05-27T07:29:56.944716+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34520, 'mds_1': 34484, 'mds_2': 34454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/3113374312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 3113374312}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 3113374312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6839/1574913181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 1574913181}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 1574913181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34430': {'gid': 34430, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6833/1769339501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6832', 'nonce': 1769339501}, {'type': 'v1', 'addr': '172.21.15.86:6833', 'nonce': 1769339501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34454': {'gid': 34454, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 35, 'addr': '172.21.15.86:6835/1494101824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 1494101824}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 1494101824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34484': {'gid': 34484, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.86:6837/559054291', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 559054291}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 559054291}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34520': {'gid': 34520, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.86:6839/953218889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 953218889}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 953218889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34520, 'qdb_cluster': [34484, 34520, 34454]} 2024-05-27T07:32:20.073 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-27T07:32:20.073 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-05-27T07:32:20.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:20 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/128608210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:32:20.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:20 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/128608210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:32:20.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:20 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/128608210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:32:20.531 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:21.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:21 smithi086 ceph-mon[34687]: pgmap v509: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:21.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:21 smithi149 ceph-mon[35408]: pgmap v509: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:21.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:21 smithi063 ceph-mon[31613]: pgmap v509: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:22.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.801+0000 7f3a96400700 1 -- 172.21.15.63:0/1528244144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a90109840 msgr2=0x7f3a901111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.801+0000 7f3a96400700 1 --2- 172.21.15.63:0/1528244144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a90109840 0x7f3a901111a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800b0d0 tx=0x7f3a8800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.801+0000 7f3a96400700 1 -- 172.21.15.63:0/1528244144 shutdown_connections 2024-05-27T07:32:22.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.801+0000 7f3a96400700 1 --2- 172.21.15.63:0/1528244144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a90109840 0x7f3a901111a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.801+0000 7f3a96400700 1 --2- 172.21.15.63:0/1528244144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a901045c0 0x7f3a901049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.801+0000 7f3a96400700 1 --2- 172.21.15.63:0/1528244144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a90108ef0 0x7f3a90109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.801+0000 7f3a96400700 1 -- 172.21.15.63:0/1528244144 >> 172.21.15.63:0/1528244144 conn(0x7f3a900ffe90 msgr2=0x7f3a901022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:22.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.801+0000 7f3a96400700 1 -- 172.21.15.63:0/1528244144 shutdown_connections 2024-05-27T07:32:22.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.801+0000 7f3a96400700 1 -- 172.21.15.63:0/1528244144 wait complete. 2024-05-27T07:32:22.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.802+0000 7f3a96400700 1 Processor -- start 2024-05-27T07:32:22.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.803+0000 7f3a96400700 1 -- start start 2024-05-27T07:32:22.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.803+0000 7f3a96400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a901045c0 0x7f3a9019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.803+0000 7f3a96400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a90108ef0 0x7f3a9019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.803+0000 7f3a96400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a90109840 0x7f3a901a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.803+0000 7f3a96400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90114320 con 0x7f3a90108ef0 2024-05-27T07:32:22.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.803+0000 7f3a96400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901141a0 con 0x7f3a901045c0 2024-05-27T07:32:22.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.803+0000 7f3a96400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901144a0 con 0x7f3a90109840 2024-05-27T07:32:22.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.804+0000 7f3a8ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a90108ef0 0x7f3a9019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.804+0000 7f3a8ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a90108ef0 0x7f3a9019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50072/0 (socket says 172.21.15.63:50072) 2024-05-27T07:32:22.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.804+0000 7f3a8ea00700 1 -- 172.21.15.63:0/3190544208 learned_addr learned my addr 172.21.15.63:0/3190544208 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:22.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.804+0000 7f3a8f400700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a901045c0 0x7f3a9019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.804+0000 7f3a8fe00700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a90109840 0x7f3a901a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.804+0000 7f3a8ea00700 1 -- 172.21.15.63:0/3190544208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a90109840 msgr2=0x7f3a901a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.804+0000 7f3a8ea00700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a90109840 0x7f3a901a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.804+0000 7f3a8ea00700 1 -- 172.21.15.63:0/3190544208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a901045c0 msgr2=0x7f3a9019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.804+0000 7f3a8ea00700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a901045c0 0x7f3a9019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.804+0000 7f3a8ea00700 1 -- 172.21.15.63:0/3190544208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a901a1d50 con 0x7f3a90108ef0 2024-05-27T07:32:22.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.805+0000 7f3a8ea00700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a90108ef0 0x7f3a9019d4d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00c9e0 tx=0x7f3a7c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:22.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.805+0000 7f3a87e00700 1 -- 172.21.15.63:0/3190544208 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c007aa0 con 0x7f3a90108ef0 2024-05-27T07:32:22.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.805+0000 7f3a87e00700 1 -- 172.21.15.63:0/3190544208 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a7c007c40 con 0x7f3a90108ef0 2024-05-27T07:32:22.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.805+0000 7f3a87e00700 1 -- 172.21.15.63:0/3190544208 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c018660 con 0x7f3a90108ef0 2024-05-27T07:32:22.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.805+0000 7f3a96400700 1 -- 172.21.15.63:0/3190544208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a901a2040 con 0x7f3a90108ef0 2024-05-27T07:32:22.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.805+0000 7f3a96400700 1 -- 172.21.15.63:0/3190544208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a901a2530 con 0x7f3a90108ef0 2024-05-27T07:32:22.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.810+0000 7f3a87e00700 1 -- 172.21.15.63:0/3190544208 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3a7c00f450 con 0x7f3a90108ef0 2024-05-27T07:32:22.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.811+0000 7f3aa5032b80 1 Processor -- start 2024-05-27T07:32:22.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.811+0000 7f3aa5032b80 1 -- start start 2024-05-27T07:32:22.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.811+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a4d5df0 0x55566a65a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.811+0000 7f3a87e00700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a740b2880 0x7f3a740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.811+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a65ace0 0x55566a654820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.811+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a654d60 0x55566a655170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.811+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55566a582df0 con 0x55566a654d60 2024-05-27T07:32:22.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.811+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55566a572990 con 0x55566a65ace0 2024-05-27T07:32:22.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.811+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55566a5769d0 con 0x55566a4d5df0 2024-05-27T07:32:22.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.812+0000 7f3a87e00700 1 -- 172.21.15.63:0/3190544208 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f3a7c0e2520 con 0x7f3a90108ef0 2024-05-27T07:32:22.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.812+0000 7f3a8fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a654d60 0x55566a655170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.812+0000 7f3a8fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a654d60 0x55566a655170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50080/0 (socket says 172.21.15.63:50080) 2024-05-27T07:32:22.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.812+0000 7f3a8fe00700 1 -- 172.21.15.63:0/2511938143 learned_addr learned my addr 172.21.15.63:0/2511938143 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:22.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.812+0000 7f3a8ea00700 1 --2- 172.21.15.63:0/2511938143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a65ace0 0x55566a654820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.812+0000 7f3a8fe00700 1 --2- 172.21.15.63:0/2511938143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a4d5df0 0x55566a65a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.812+0000 7f3a8f400700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a740b2880 0x7f3a740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.813+0000 7f3a8fe00700 1 -- 172.21.15.63:0/2511938143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a4d5df0 msgr2=0x55566a65a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:22.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.813+0000 7f3a8fe00700 1 --2- 172.21.15.63:0/2511938143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a4d5df0 0x55566a65a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.813+0000 7f3a8fe00700 1 -- 172.21.15.63:0/2511938143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a65ace0 msgr2=0x55566a654820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:22.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.813+0000 7f3a8fe00700 1 --2- 172.21.15.63:0/2511938143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a65ace0 0x55566a654820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.813+0000 7f3a8fe00700 1 -- 172.21.15.63:0/2511938143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55566a53b3d0 con 0x55566a654d60 2024-05-27T07:32:22.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.813+0000 7f3a8fe00700 1 --2- 172.21.15.63:0/2511938143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a654d60 0x55566a655170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55566a655ae0 tx=0x7f3a88005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a95cbdd56bcdf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:22.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.813+0000 7f3a8f400700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a740b2880 0x7f3a740b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a80005950 tx=0x7f3a8000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:22.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.813+0000 7f3a84c00700 1 -- 172.21.15.63:0/2511938143 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88004c50 con 0x55566a654d60 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.813+0000 7f3a84c00700 1 -- 172.21.15.63:0/2511938143 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a88004df0 con 0x55566a654d60 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.813+0000 7f3a84c00700 1 -- 172.21.15.63:0/2511938143 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a880052c0 con 0x55566a654d60 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.814+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2511938143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a654d60 msgr2=0x55566a655170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.814+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/2511938143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a654d60 0x55566a655170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55566a655ae0 tx=0x7f3a88005ac0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.815+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2511938143 shutdown_connections 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.815+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/2511938143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a654d60 0x55566a655170 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.815+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/2511938143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a4d5df0 0x55566a65a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.815+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/2511938143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a65ace0 0x55566a654820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.815+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2511938143 >> 172.21.15.63:0/2511938143 conn(0x55566a535f70 msgr2=0x55566a64f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.815+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2511938143 shutdown_connections 2024-05-27T07:32:22.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.815+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2511938143 wait complete. 2024-05-27T07:32:22.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.815+0000 7f3aa5032b80 1 -- start start 2024-05-27T07:32:22.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.816+0000 7f3aa5032b80 1 Processor -- start 2024-05-27T07:32:22.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.816+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a4d5df0 0x55566a65e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.816+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a654d60 0x55566a65ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.816+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a65ace0 0x55566a65ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.816+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55566a478f50 con 0x55566a65ace0 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.816+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55566a513710 con 0x55566a4d5df0 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.816+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55566a661750 con 0x55566a654d60 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.816+0000 7f3aa5032b80 10 client.0 _ll_register_callbacks cb 0x55566a51cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.817+0000 7f3a8fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a65ace0 0x55566a65ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.817+0000 7f3a8fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a65ace0 0x55566a65ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50092/0 (socket says 172.21.15.63:50092) 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.817+0000 7f3a8fe00700 1 -- 172.21.15.63:0/2881657876 learned_addr learned my addr 172.21.15.63:0/2881657876 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.817+0000 7f3a8f400700 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a4d5df0 0x55566a65e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.817+0000 7f3a8ea00700 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a654d60 0x55566a65ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.817+0000 7f3a8fe00700 1 -- 172.21.15.63:0/2881657876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a654d60 msgr2=0x55566a65ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.817+0000 7f3a8fe00700 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a654d60 0x55566a65ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.817+0000 7f3a8fe00700 1 -- 172.21.15.63:0/2881657876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a4d5df0 msgr2=0x55566a65e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.817+0000 7f3a8fe00700 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a4d5df0 0x55566a65e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.817+0000 7f3a8fe00700 1 -- 172.21.15.63:0/2881657876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566a4c2880 con 0x55566a65ace0 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.818+0000 7f3a8fe00700 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a65ace0 0x55566a65ef40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3a88002230 tx=0x7f3a88007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e016c8496f16f818 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.818+0000 7f3a79600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.818+0000 7f3a79600700 1 -- 172.21.15.63:0/2881657876 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88004fa0 con 0x55566a65ace0 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.818+0000 7f3a79600700 1 -- 172.21.15.63:0/2881657876 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f3a88005140 con 0x55566a65ace0 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.818+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2881657876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a88004df0 con 0x55566a65ace0 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.818+0000 7f3a79600700 1 -- 172.21.15.63:0/2881657876 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3a88005340 con 0x55566a65ace0 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.819+0000 7f3aa5032b80 10 client.15045 fetch_fsmap learned FSMap version 26 2024-05-27T07:32:22.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.819+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2881657876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55566a543010 con 0x55566a65ace0 2024-05-27T07:32:22.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.819+0000 7f3a79600700 1 -- 172.21.15.63:0/2881657876 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a88004df0 con 0x55566a65ace0 2024-05-27T07:32:22.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.819+0000 7f3a79600700 1 -- 172.21.15.63:0/2881657876 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3a88032030 con 0x55566a65ace0 2024-05-27T07:32:22.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.819+0000 7f3aa5032b80 10 client.15045 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:32:22.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.819+0000 7f3aa5032b80 10 client.15045 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T07:32:22.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.819+0000 7f3aa5032b80 20 client.15045 populate_metadata read hostname 'smithi063' 2024-05-27T07:32:22.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.819+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55566a66d3d0 0x55566a66f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.819+0000 7f3aa5032b80 4 client.15045 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:32:22.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.819+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2881657876 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55566a584d40 con 0x55566a66d3d0 2024-05-27T07:32:22.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.820+0000 7f3a8f400700 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55566a66d3d0 0x55566a66f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.821+0000 7f3a8f400700 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55566a66d3d0 0x55566a66f5b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88291b7764f26e41 server_cookie=380c2cafee63b7aa in_seq=0 out_seq=0 2024-05-27T07:32:22.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.821+0000 7f3a79600700 10 client.15045 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:32:22.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.823+0000 7f3a79600700 1 -- 172.21.15.63:0/2881657876 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55566a584d40 con 0x55566a66d3d0 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.823+0000 7f3a79600700 10 client.15045 handle_command_reply: tid=0 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3aa5032b80 1 client.15045 shutdown 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2881657876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a65ace0 msgr2=0x55566a65ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a65ace0 0x55566a65ef40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3a88002230 tx=0x7f3a88007870 comp rx=0 tx=0).stop 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2881657876 shutdown_connections 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55566a66d3d0 0x55566a66f5b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a65ace0 0x55566a65ef40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a4d5df0 0x55566a65e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/2881657876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a654d60 0x55566a65ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2881657876 >> 172.21.15.63:0/2881657876 conn(0x55566a535f70 msgr2=0x55566a64f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3a79600700 0 client.15045 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:32:22.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.824+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2881657876 shutdown_connections 2024-05-27T07:32:22.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.825+0000 7f3aa5032b80 1 -- 172.21.15.63:0/2881657876 wait complete. 2024-05-27T07:32:22.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.825+0000 7f3aa5032b80 20 client.15045 trim_cache size 0 max 16384 2024-05-27T07:32:22.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.869+0000 7f3aa5032b80 1 Processor -- start 2024-05-27T07:32:22.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.870+0000 7f3aa5032b80 1 -- start start 2024-05-27T07:32:22.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.870+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a654820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.870+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a654d60 0x55566a659e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.870+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a65ace0 0x55566a660960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.870+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55566a66d8f0 con 0x55566a4d5df0 2024-05-27T07:32:22.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.870+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55566a66dfc0 con 0x55566a654d60 2024-05-27T07:32:22.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.870+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55566a66e690 con 0x55566a65ace0 2024-05-27T07:32:22.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.871+0000 7f3a8f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a654820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.871+0000 7f3a8f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a654820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50102/0 (socket says 172.21.15.63:50102) 2024-05-27T07:32:22.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.871+0000 7f3a8f400700 1 -- 172.21.15.63:0/3401655203 learned_addr learned my addr 172.21.15.63:0/3401655203 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:22.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.871+0000 7f3a8ea00700 1 --2- 172.21.15.63:0/3401655203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a654d60 0x55566a659e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.871+0000 7f3a8f400700 1 -- 172.21.15.63:0/3401655203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a65ace0 msgr2=0x55566a660960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.871+0000 7f3a8f400700 1 --2- 172.21.15.63:0/3401655203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a65ace0 0x55566a660960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.871+0000 7f3a8f400700 1 -- 172.21.15.63:0/3401655203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a654d60 msgr2=0x55566a659e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.871+0000 7f3a8f400700 1 --2- 172.21.15.63:0/3401655203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a654d60 0x55566a659e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.871+0000 7f3a8f400700 1 -- 172.21.15.63:0/3401655203 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55566a65f4d0 con 0x55566a4d5df0 2024-05-27T07:32:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.872+0000 7f3a8f400700 1 --2- 172.21.15.63:0/3401655203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a654820 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55566a65dd40 tx=0x7f3a80013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e36d9a92d3ec345 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.872+0000 7f3a7be00700 1 -- 172.21.15.63:0/3401655203 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80004830 con 0x55566a4d5df0 2024-05-27T07:32:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.872+0000 7f3a7be00700 1 -- 172.21.15.63:0/3401655203 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a800117e0 con 0x55566a4d5df0 2024-05-27T07:32:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.872+0000 7f3a7be00700 1 -- 172.21.15.63:0/3401655203 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80005280 con 0x55566a4d5df0 2024-05-27T07:32:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.873+0000 7f3aa5032b80 1 -- 172.21.15.63:0/3401655203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 msgr2=0x55566a654820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.873+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/3401655203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a654820 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55566a65dd40 tx=0x7f3a80013000 comp rx=0 tx=0).stop 2024-05-27T07:32:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.873+0000 7f3aa5032b80 1 -- 172.21.15.63:0/3401655203 shutdown_connections 2024-05-27T07:32:22.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.873+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/3401655203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a65ace0 0x55566a660960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.873+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/3401655203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a654820 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.873+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/3401655203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a654d60 0x55566a659e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.873+0000 7f3aa5032b80 1 -- 172.21.15.63:0/3401655203 >> 172.21.15.63:0/3401655203 conn(0x55566a535f70 msgr2=0x55566a65eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:22.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.873+0000 7f3aa5032b80 1 -- 172.21.15.63:0/3401655203 shutdown_connections 2024-05-27T07:32:22.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.873+0000 7f3aa5032b80 1 -- 172.21.15.63:0/3401655203 wait complete. 2024-05-27T07:32:22.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.874+0000 7f3aa5032b80 1 -- start start 2024-05-27T07:32:22.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.874+0000 7f3aa5032b80 1 Processor -- start 2024-05-27T07:32:22.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.874+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a678a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.874+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a654d60 0x55566a678f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.874+0000 7f3aa5032b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a65ace0 0x55566a6739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.874+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55566a66f430 con 0x55566a4d5df0 2024-05-27T07:32:22.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.874+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55566a66ed60 con 0x55566a65ace0 2024-05-27T07:32:22.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.874+0000 7f3aa5032b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55566a66fb00 con 0x55566a654d60 2024-05-27T07:32:22.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3aa5032b80 10 client.0 _ll_register_callbacks cb 0x55566a51cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:32:22.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3a8f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a678a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3a8f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a678a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50116/0 (socket says 172.21.15.63:50116) 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3a8f400700 1 -- 172.21.15.63:0/317860901 learned_addr learned my addr 172.21.15.63:0/317860901 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3a8ea00700 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a654d60 0x55566a678f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3a8fe00700 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a65ace0 0x55566a6739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3a8f400700 1 -- 172.21.15.63:0/317860901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a654d60 msgr2=0x55566a678f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3a8f400700 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a654d60 0x55566a678f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3a8f400700 1 -- 172.21.15.63:0/317860901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a65ace0 msgr2=0x55566a6739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3a8f400700 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a65ace0 0x55566a6739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.875+0000 7f3a8f400700 1 -- 172.21.15.63:0/317860901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566a673f20 con 0x55566a4d5df0 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.876+0000 7f3a8f400700 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a678a30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3a800046e0 tx=0x7f3a80011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a704a899aced70 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.876+0000 7f3a79600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.876+0000 7f3a79600700 1 -- 172.21.15.63:0/317860901 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80004ae0 con 0x55566a4d5df0 2024-05-27T07:32:22.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.876+0000 7f3a79600700 1 -- 172.21.15.63:0/317860901 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f3a80004c80 con 0x55566a4d5df0 2024-05-27T07:32:22.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.876+0000 7f3aa5032b80 1 -- 172.21.15.63:0/317860901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a800117e0 con 0x55566a4d5df0 2024-05-27T07:32:22.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.876+0000 7f3a79600700 1 -- 172.21.15.63:0/317860901 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3a8003e050 con 0x55566a4d5df0 2024-05-27T07:32:22.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.876+0000 7f3aa5032b80 10 client.15051 fetch_fsmap learned FSMap version 26 2024-05-27T07:32:22.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.876+0000 7f3aa5032b80 1 -- 172.21.15.63:0/317860901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55566a6d9f50 con 0x55566a4d5df0 2024-05-27T07:32:22.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.877+0000 7f3a79600700 1 -- 172.21.15.63:0/317860901 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a800117e0 con 0x55566a4d5df0 2024-05-27T07:32:22.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.877+0000 7f3a79600700 1 -- 172.21.15.63:0/317860901 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3a800307f0 con 0x55566a4d5df0 2024-05-27T07:32:22.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.877+0000 7f3aa5032b80 10 client.15051 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:32:22.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.877+0000 7f3aa5032b80 10 client.15051 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T07:32:22.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.877+0000 7f3aa5032b80 20 client.15051 populate_metadata read hostname 'smithi063' 2024-05-27T07:32:22.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.877+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55566a6dc010 0x55566a6dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:22.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.877+0000 7f3aa5032b80 4 client.15051 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:32:22.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.877+0000 7f3aa5032b80 1 -- 172.21.15.63:0/317860901 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "status"}) -- 0x55566a57e520 con 0x55566a6dc010 2024-05-27T07:32:22.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.878+0000 7f3a8ea00700 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55566a6dc010 0x55566a6dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:22.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.879+0000 7f3a8ea00700 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55566a6dc010 0x55566a6dd480 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1297f4e7226bb1b server_cookie=59de75a0d080c652 in_seq=0 out_seq=0 2024-05-27T07:32:22.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.879+0000 7f3a79600700 10 client.15051 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:32:22.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.879+0000 7f3a79600700 1 -- 172.21.15.63:0/317860901 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55566a57e520 con 0x55566a6dc010 2024-05-27T07:32:22.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.879+0000 7f3a79600700 10 client.15051 handle_command_reply: tid=0 2024-05-27T07:32:22.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 client.15051 shutdown 2024-05-27T07:32:22.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 -- 172.21.15.63:0/317860901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 msgr2=0x55566a678a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a678a30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3a800046e0 tx=0x7f3a80011e10 comp rx=0 tx=0).stop 2024-05-27T07:32:22.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 -- 172.21.15.63:0/317860901 shutdown_connections 2024-05-27T07:32:22.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55566a65ace0 0x55566a6739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55566a6dc010 0x55566a6dd480 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55566a4d5df0 0x55566a678a30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 --2- 172.21.15.63:0/317860901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55566a654d60 0x55566a678f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 -- 172.21.15.63:0/317860901 >> 172.21.15.63:0/317860901 conn(0x55566a535f70 msgr2=0x55566a6580f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:22.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3a79600700 0 client.15051 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:32:22.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 -- 172.21.15.63:0/317860901 shutdown_connections 2024-05-27T07:32:22.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 1 -- 172.21.15.63:0/317860901 wait complete. 2024-05-27T07:32:22.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.880+0000 7f3aa5032b80 20 client.15051 trim_cache size 0 max 16384 2024-05-27T07:32:22.875 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:32:22.875 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:32:22.875 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 2, 2024-05-27T07:32:22.875 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34454, 2024-05-27T07:32:22.875 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-27T07:32:22.875 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-27T07:32:22.875 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-27T07:32:22.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 152.89775501700001, 2024-05-27T07:32:22.876 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 26, 2024-05-27T07:32:22.876 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-27T07:32:22.876 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:32:22.876 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 286.327479117 2024-05-27T07:32:22.876 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:32:22.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 -- 172.21.15.63:0/3190544208 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a740b2880 msgr2=0x7f3a740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a740b2880 0x7f3a740b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a80005950 tx=0x7f3a8000b410 comp rx=0 tx=0).stop 2024-05-27T07:32:22.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 -- 172.21.15.63:0/3190544208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a90108ef0 msgr2=0x7f3a9019d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a90108ef0 0x7f3a9019d4d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00c9e0 tx=0x7f3a7c00ccf0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 -- 172.21.15.63:0/3190544208 shutdown_connections 2024-05-27T07:32:22.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a90109840 0x7f3a901a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a901045c0 0x7f3a9019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f3a740b2880 0x7f3a740b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 --2- 172.21.15.63:0/3190544208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a90108ef0 0x7f3a9019d4d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 -- 172.21.15.63:0/3190544208 >> 172.21.15.63:0/3190544208 conn(0x7f3a900ffe90 msgr2=0x7f3a9010a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:22.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.881+0000 7f3a96400700 1 -- 172.21.15.63:0/3190544208 shutdown_connections 2024-05-27T07:32:22.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:22.882+0000 7f3a96400700 1 -- 172.21.15.63:0/3190544208 wait complete. 2024-05-27T07:32:23.311 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:23 smithi063 ceph-mon[31613]: pgmap v510: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:23.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:23 smithi086 ceph-mon[34687]: pgmap v510: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:23.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:23 smithi149 ceph-mon[35408]: pgmap v510: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:23.828 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T07:32:24.284 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:25.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:25 smithi086 ceph-mon[34687]: pgmap v511: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:25.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:25 smithi149 ceph-mon[35408]: pgmap v511: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:25.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:25 smithi063 ceph-mon[31613]: pgmap v511: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:26.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.632+0000 7fce72000700 1 -- 172.21.15.63:0/1050426482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce6c110960 msgr2=0x7fce6c110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.632+0000 7fce72000700 1 --2- 172.21.15.63:0/1050426482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce6c110960 0x7fce6c110d70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009990 tx=0x7fce5c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.632+0000 7fce72000700 1 -- 172.21.15.63:0/1050426482 shutdown_connections 2024-05-27T07:32:26.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.632+0000 7fce72000700 1 --2- 172.21.15.63:0/1050426482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce6c10b2d0 0x7fce6c117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.632+0000 7fce72000700 1 --2- 172.21.15.63:0/1050426482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce6c110960 0x7fce6c110d70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.632+0000 7fce72000700 1 --2- 172.21.15.63:0/1050426482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c10a980 0x7fce6c10ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.632+0000 7fce72000700 1 -- 172.21.15.63:0/1050426482 >> 172.21.15.63:0/1050426482 conn(0x7fce6c077c70 msgr2=0x7fce6c0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:26.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.632+0000 7fce72000700 1 -- 172.21.15.63:0/1050426482 shutdown_connections 2024-05-27T07:32:26.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.632+0000 7fce72000700 1 -- 172.21.15.63:0/1050426482 wait complete. 2024-05-27T07:32:26.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.634+0000 7fce72000700 1 Processor -- start 2024-05-27T07:32:26.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.634+0000 7fce72000700 1 -- start start 2024-05-27T07:32:26.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.635+0000 7fce72000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce6c10a980 0x7fce6c1a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.635+0000 7fce72000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c10b2d0 0x7fce6c1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.635+0000 7fce72000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce6c110960 0x7fce6c1a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.635+0000 7fce72000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c11a960 con 0x7fce6c110960 2024-05-27T07:32:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.635+0000 7fce72000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c11a7e0 con 0x7fce6c10b2d0 2024-05-27T07:32:26.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.635+0000 7fce72000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c11aae0 con 0x7fce6c10a980 2024-05-27T07:32:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.635+0000 7fce6be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce6c110960 0x7fce6c1a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.635+0000 7fce6be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce6c110960 0x7fce6c1a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50136/0 (socket says 172.21.15.63:50136) 2024-05-27T07:32:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.635+0000 7fce6be00700 1 -- 172.21.15.63:0/3999891516 learned_addr learned my addr 172.21.15.63:0/3999891516 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.635+0000 7fce6b400700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce6c10a980 0x7fce6c1a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.636+0000 7fce6aa00700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c10b2d0 0x7fce6c1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.636+0000 7fce6be00700 1 -- 172.21.15.63:0/3999891516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce6c10a980 msgr2=0x7fce6c1a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.636+0000 7fce6be00700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce6c10a980 0x7fce6c1a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.636+0000 7fce6be00700 1 -- 172.21.15.63:0/3999891516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c10b2d0 msgr2=0x7fce6c1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.636+0000 7fce6be00700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c10b2d0 0x7fce6c1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.636+0000 7fce6be00700 1 -- 172.21.15.63:0/3999891516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6c1a6220 con 0x7fce6c110960 2024-05-27T07:32:26.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.636+0000 7fce6be00700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce6c110960 0x7fce6c1a5b80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fce6400ef30 tx=0x7fce6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:26.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.637+0000 7fce63e00700 1 -- 172.21.15.63:0/3999891516 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640092e0 con 0x7fce6c110960 2024-05-27T07:32:26.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.637+0000 7fce63e00700 1 -- 172.21.15.63:0/3999891516 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce64004bb0 con 0x7fce6c110960 2024-05-27T07:32:26.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.637+0000 7fce63e00700 1 -- 172.21.15.63:0/3999891516 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64004020 con 0x7fce6c110960 2024-05-27T07:32:26.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.637+0000 7fce72000700 1 -- 172.21.15.63:0/3999891516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6c1a64b0 con 0x7fce6c110960 2024-05-27T07:32:26.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.637+0000 7fce72000700 1 -- 172.21.15.63:0/3999891516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6c1a69a0 con 0x7fce6c110960 2024-05-27T07:32:26.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.641+0000 7fce63e00700 1 -- 172.21.15.63:0/3999891516 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fce6401b030 con 0x7fce6c110960 2024-05-27T07:32:26.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.643+0000 7fce80c20b80 1 Processor -- start 2024-05-27T07:32:26.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.643+0000 7fce80c20b80 1 -- start start 2024-05-27T07:32:26.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.643+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af5ea6df0 0x564af602b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.643+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af602bce0 0x564af6025820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.643+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af6025d60 0x564af6026170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce80c20b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564af5f53df0 con 0x564af6025d60 2024-05-27T07:32:26.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce80c20b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564af5f43990 con 0x564af602bce0 2024-05-27T07:32:26.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce80c20b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564af5f479d0 con 0x564af5ea6df0 2024-05-27T07:32:26.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce63e00700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce500b2900 0x7fce500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce6aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af6025d60 0x564af6026170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce6aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af6025d60 0x564af6026170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50148/0 (socket says 172.21.15.63:50148) 2024-05-27T07:32:26.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce6aa00700 1 -- 172.21.15.63:0/4262503537 learned_addr learned my addr 172.21.15.63:0/4262503537 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:26.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce6b400700 1 --2- 172.21.15.63:0/4262503537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af602bce0 0x564af6025820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce6aa00700 1 --2- 172.21.15.63:0/4262503537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af5ea6df0 0x564af602b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce63e00700 1 -- 172.21.15.63:0/3999891516 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fce640e1fe0 con 0x7fce6c110960 2024-05-27T07:32:26.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.644+0000 7fce6be00700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce500b2900 0x7fce500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.645+0000 7fce6aa00700 1 -- 172.21.15.63:0/4262503537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af5ea6df0 msgr2=0x564af602b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:26.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.645+0000 7fce6aa00700 1 --2- 172.21.15.63:0/4262503537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af5ea6df0 0x564af602b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.645+0000 7fce6aa00700 1 -- 172.21.15.63:0/4262503537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af602bce0 msgr2=0x564af6025820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:26.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.645+0000 7fce6aa00700 1 --2- 172.21.15.63:0/4262503537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af602bce0 0x564af6025820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.645+0000 7fce6aa00700 1 -- 172.21.15.63:0/4262503537 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af5f0c3d0 con 0x564af6025d60 2024-05-27T07:32:26.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.645+0000 7fce6be00700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce500b2900 0x7fce500b4e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce640a5320 tx=0x7fce64009480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:26.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.645+0000 7fce6aa00700 1 --2- 172.21.15.63:0/4262503537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af6025d60 0x564af6026170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fce58005950 tx=0x7fce580058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae384e6aac69ea2e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:26.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.645+0000 7fce60c00700 1 -- 172.21.15.63:0/4262503537 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58004030 con 0x564af6025d60 2024-05-27T07:32:26.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.646+0000 7fce60c00700 1 -- 172.21.15.63:0/4262503537 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce58004830 con 0x564af6025d60 2024-05-27T07:32:26.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.646+0000 7fce60c00700 1 -- 172.21.15.63:0/4262503537 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce580052d0 con 0x564af6025d60 2024-05-27T07:32:26.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.646+0000 7fce80c20b80 1 -- 172.21.15.63:0/4262503537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af6025d60 msgr2=0x564af6026170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.646+0000 7fce80c20b80 1 --2- 172.21.15.63:0/4262503537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af6025d60 0x564af6026170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fce58005950 tx=0x7fce580058e0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.647+0000 7fce80c20b80 1 -- 172.21.15.63:0/4262503537 shutdown_connections 2024-05-27T07:32:26.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.647+0000 7fce80c20b80 1 --2- 172.21.15.63:0/4262503537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af6025d60 0x564af6026170 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.647+0000 7fce80c20b80 1 --2- 172.21.15.63:0/4262503537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af5ea6df0 0x564af602b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.647+0000 7fce80c20b80 1 --2- 172.21.15.63:0/4262503537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af602bce0 0x564af6025820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.647+0000 7fce80c20b80 1 -- 172.21.15.63:0/4262503537 >> 172.21.15.63:0/4262503537 conn(0x564af5f06f70 msgr2=0x564af6020880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:26.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.647+0000 7fce80c20b80 1 -- 172.21.15.63:0/4262503537 shutdown_connections 2024-05-27T07:32:26.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.647+0000 7fce80c20b80 1 -- 172.21.15.63:0/4262503537 wait complete. 2024-05-27T07:32:26.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.648+0000 7fce80c20b80 1 -- start start 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.648+0000 7fce80c20b80 1 Processor -- start 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.648+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af602f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.648+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af6025d60 0x564af602f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.648+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af602bce0 0x564af602fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.648+0000 7fce80c20b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564af5e49f50 con 0x564af5ea6df0 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.648+0000 7fce80c20b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564af5ee4710 con 0x564af6025d60 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.648+0000 7fce80c20b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564af60327b0 con 0x564af602bce0 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.649+0000 7fce80c20b80 10 client.0 _ll_register_callbacks cb 0x564af5eedc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.649+0000 7fce6aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af602f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.649+0000 7fce6aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af602f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50160/0 (socket says 172.21.15.63:50160) 2024-05-27T07:32:26.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.649+0000 7fce6aa00700 1 -- 172.21.15.63:0/307340308 learned_addr learned my addr 172.21.15.63:0/307340308 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:26.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.649+0000 7fce6b400700 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af6025d60 0x564af602f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.649+0000 7fce6aa00700 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af602bce0 0x564af602fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.650+0000 7fce6aa00700 1 -- 172.21.15.63:0/307340308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af602bce0 msgr2=0x564af602fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:26.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.650+0000 7fce6aa00700 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af602bce0 0x564af602fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.650+0000 7fce6aa00700 1 -- 172.21.15.63:0/307340308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af6025d60 msgr2=0x564af602f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:26.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.650+0000 7fce6aa00700 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af6025d60 0x564af602f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.650+0000 7fce6aa00700 1 -- 172.21.15.63:0/307340308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af5e93880 con 0x564af5ea6df0 2024-05-27T07:32:26.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.650+0000 7fce6aa00700 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af602f460 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fce58009340 tx=0x7fce58009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18404e13eac457f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.650+0000 7fce55600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.650+0000 7fce55600700 1 -- 172.21.15.63:0/307340308 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce580054a0 con 0x564af5ea6df0 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.650+0000 7fce55600700 1 -- 172.21.15.63:0/307340308 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fce58005640 con 0x564af5ea6df0 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.650+0000 7fce80c20b80 1 -- 172.21.15.63:0/307340308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce58004830 con 0x564af5ea6df0 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce55600700 1 -- 172.21.15.63:0/307340308 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce5801f440 con 0x564af5ea6df0 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce80c20b80 10 client.15063 fetch_fsmap learned FSMap version 26 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce80c20b80 1 -- 172.21.15.63:0/307340308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564af5f14010 con 0x564af5ea6df0 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce55600700 1 -- 172.21.15.63:0/307340308 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce58004830 con 0x564af5ea6df0 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce55600700 1 -- 172.21.15.63:0/307340308 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fce5800f680 con 0x564af5ea6df0 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce80c20b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce80c20b80 10 client.15063 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T07:32:26.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce80c20b80 20 client.15063 populate_metadata read hostname 'smithi063' 2024-05-27T07:32:26.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce80c20b80 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564af603f310 0x564af6040780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce80c20b80 4 client.15063 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:32:26.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.651+0000 7fce80c20b80 1 -- 172.21.15.63:0/307340308 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564af5f55d40 con 0x564af603f310 2024-05-27T07:32:26.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.652+0000 7fce6b400700 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564af603f310 0x564af6040780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.653+0000 7fce6b400700 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564af603f310 0x564af6040780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=996b1ce6a8c2f8b0 server_cookie=9f14580664ef3419 in_seq=0 out_seq=0 2024-05-27T07:32:26.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.653+0000 7fce55600700 10 client.15063 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:32:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce55600700 1 -- 172.21.15.63:0/307340308 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564af5f55d40 con 0x564af603f310 2024-05-27T07:32:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce55600700 10 client.15063 handle_command_reply: tid=0 2024-05-27T07:32:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 client.15063 shutdown 2024-05-27T07:32:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 -- 172.21.15.63:0/307340308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 msgr2=0x564af602f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af602f460 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fce58009340 tx=0x7fce58009da0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 -- 172.21.15.63:0/307340308 shutdown_connections 2024-05-27T07:32:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af602bce0 0x564af602fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564af603f310 0x564af6040780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af602f460 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 --2- 172.21.15.63:0/307340308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af6025d60 0x564af602f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 -- 172.21.15.63:0/307340308 >> 172.21.15.63:0/307340308 conn(0x564af5f06f70 msgr2=0x564af6020530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:26.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce55600700 0 client.15063 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:32:26.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 -- 172.21.15.63:0/307340308 shutdown_connections 2024-05-27T07:32:26.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.656+0000 7fce80c20b80 1 -- 172.21.15.63:0/307340308 wait complete. 2024-05-27T07:32:26.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.657+0000 7fce80c20b80 20 client.15063 trim_cache size 0 max 16384 2024-05-27T07:32:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.681+0000 7fce80c20b80 1 Processor -- start 2024-05-27T07:32:26.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.681+0000 7fce80c20b80 1 -- start start 2024-05-27T07:32:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.681+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af5ea6df0 0x564af6025820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.681+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af6025d60 0x564af6031900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.681+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af602bce0 0x564af602ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.681+0000 7fce80c20b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564af603eac0 con 0x564af602bce0 2024-05-27T07:32:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.681+0000 7fce80c20b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564af603f190 con 0x564af6025d60 2024-05-27T07:32:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.681+0000 7fce80c20b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564af603f860 con 0x564af5ea6df0 2024-05-27T07:32:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af602bce0 0x564af602ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af602bce0 0x564af602ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50174/0 (socket says 172.21.15.63:50174) 2024-05-27T07:32:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6aa00700 1 -- 172.21.15.63:0/1394603474 learned_addr learned my addr 172.21.15.63:0/1394603474 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6aa00700 1 --2- 172.21.15.63:0/1394603474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af5ea6df0 0x564af6025820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6b400700 1 --2- 172.21.15.63:0/1394603474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af6025d60 0x564af6031900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6aa00700 1 -- 172.21.15.63:0/1394603474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af5ea6df0 msgr2=0x564af6025820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:26.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6aa00700 1 --2- 172.21.15.63:0/1394603474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af5ea6df0 0x564af6025820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6aa00700 1 -- 172.21.15.63:0/1394603474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af6025d60 msgr2=0x564af6031900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6aa00700 1 --2- 172.21.15.63:0/1394603474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af6025d60 0x564af6031900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6aa00700 1 -- 172.21.15.63:0/1394603474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af6032200 con 0x564af602bce0 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.682+0000 7fce6aa00700 1 --2- 172.21.15.63:0/1394603474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af602bce0 0x564af602ad20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fce58005a80 tx=0x7fce58009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201811cb1810a9e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.683+0000 7fce57e00700 1 -- 172.21.15.63:0/1394603474 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58004fe0 con 0x564af602bce0 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.683+0000 7fce57e00700 1 -- 172.21.15.63:0/1394603474 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce5802a040 con 0x564af602bce0 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.683+0000 7fce57e00700 1 -- 172.21.15.63:0/1394603474 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5801f840 con 0x564af602bce0 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.684+0000 7fce80c20b80 1 -- 172.21.15.63:0/1394603474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af602bce0 msgr2=0x564af602ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.684+0000 7fce80c20b80 1 --2- 172.21.15.63:0/1394603474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af602bce0 0x564af602ad20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fce58005a80 tx=0x7fce58009d00 comp rx=0 tx=0).stop 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.684+0000 7fce80c20b80 1 -- 172.21.15.63:0/1394603474 shutdown_connections 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.684+0000 7fce80c20b80 1 --2- 172.21.15.63:0/1394603474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af602bce0 0x564af602ad20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.684+0000 7fce80c20b80 1 --2- 172.21.15.63:0/1394603474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af5ea6df0 0x564af6025820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.684+0000 7fce80c20b80 1 --2- 172.21.15.63:0/1394603474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af6025d60 0x564af6031900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.684+0000 7fce80c20b80 1 -- 172.21.15.63:0/1394603474 >> 172.21.15.63:0/1394603474 conn(0x564af5f06f70 msgr2=0x564af602a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:26.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.684+0000 7fce80c20b80 1 -- 172.21.15.63:0/1394603474 shutdown_connections 2024-05-27T07:32:26.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.684+0000 7fce80c20b80 1 -- 172.21.15.63:0/1394603474 wait complete. 2024-05-27T07:32:26.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.684+0000 7fce80c20b80 1 -- start start 2024-05-27T07:32:26.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.685+0000 7fce80c20b80 1 Processor -- start 2024-05-27T07:32:26.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.685+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af604a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.685+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af6025d60 0x564af6048db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.685+0000 7fce80c20b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af602bce0 0x564af60492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.685+0000 7fce80c20b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564af6040600 con 0x564af5ea6df0 2024-05-27T07:32:26.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.685+0000 7fce80c20b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564af603ff30 con 0x564af602bce0 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.685+0000 7fce80c20b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564af6040cd0 con 0x564af6025d60 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.685+0000 7fce80c20b80 10 client.0 _ll_register_callbacks cb 0x564af5eedc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.686+0000 7fce6aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af604a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.686+0000 7fce6aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af604a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50182/0 (socket says 172.21.15.63:50182) 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.686+0000 7fce6aa00700 1 -- 172.21.15.63:0/3854458283 learned_addr learned my addr 172.21.15.63:0/3854458283 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.686+0000 7fce6b400700 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af6025d60 0x564af6048db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.686+0000 7fce6aa00700 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af602bce0 0x564af60492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.686+0000 7fce6aa00700 1 -- 172.21.15.63:0/3854458283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af6025d60 msgr2=0x564af6048db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.686+0000 7fce6aa00700 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af6025d60 0x564af6048db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.686+0000 7fce6aa00700 1 -- 172.21.15.63:0/3854458283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af602bce0 msgr2=0x564af60492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.686+0000 7fce6aa00700 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af602bce0 0x564af60492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.686+0000 7fce6aa00700 1 -- 172.21.15.63:0/3854458283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af6049830 con 0x564af5ea6df0 2024-05-27T07:32:26.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.687+0000 7fce6aa00700 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af604a050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fce58009ec0 tx=0x7fce58009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c49b39d246f67a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:26.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.687+0000 7fce55600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:32:26.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.687+0000 7fce55600700 1 -- 172.21.15.63:0/3854458283 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce580056b0 con 0x564af5ea6df0 2024-05-27T07:32:26.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.687+0000 7fce55600700 1 -- 172.21.15.63:0/3854458283 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fce5801f5e0 con 0x564af5ea6df0 2024-05-27T07:32:26.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.687+0000 7fce80c20b80 1 -- 172.21.15.63:0/3854458283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5802a040 con 0x564af5ea6df0 2024-05-27T07:32:26.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.687+0000 7fce55600700 1 -- 172.21.15.63:0/3854458283 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce5801f7e0 con 0x564af5ea6df0 2024-05-27T07:32:26.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.688+0000 7fce80c20b80 10 client.15069 fetch_fsmap learned FSMap version 26 2024-05-27T07:32:26.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.688+0000 7fce80c20b80 1 -- 172.21.15.63:0/3854458283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564af6031530 con 0x564af5ea6df0 2024-05-27T07:32:26.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.688+0000 7fce55600700 1 -- 172.21.15.63:0/3854458283 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce5802a040 con 0x564af5ea6df0 2024-05-27T07:32:26.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.688+0000 7fce55600700 1 -- 172.21.15.63:0/3854458283 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fce58003c20 con 0x564af5ea6df0 2024-05-27T07:32:26.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.688+0000 7fce80c20b80 10 client.15069 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:32:26.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.688+0000 7fce80c20b80 10 client.15069 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T07:32:26.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.688+0000 7fce80c20b80 20 client.15069 populate_metadata read hostname 'smithi063' 2024-05-27T07:32:26.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.688+0000 7fce80c20b80 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564af60ac530 0x564af60ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:26.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.688+0000 7fce80c20b80 4 client.15069 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:32:26.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.688+0000 7fce80c20b80 1 -- 172.21.15.63:0/3854458283 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "status"}) -- 0x564af5f4f520 con 0x564af60ac530 2024-05-27T07:32:26.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.689+0000 7fce6b400700 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564af60ac530 0x564af60ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:26.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.690+0000 7fce6b400700 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564af60ac530 0x564af60ad9a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8764edc2c8d99825 server_cookie=eabee0620802c541 in_seq=0 out_seq=0 2024-05-27T07:32:26.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.690+0000 7fce55600700 10 client.15069 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:32:26.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce55600700 1 -- 172.21.15.63:0/3854458283 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564af5f4f520 con 0x564af60ac530 2024-05-27T07:32:26.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce55600700 10 client.15069 handle_command_reply: tid=0 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 client.15069 shutdown 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 -- 172.21.15.63:0/3854458283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 msgr2=0x564af604a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af604a050 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fce58009ec0 tx=0x7fce58009e10 comp rx=0 tx=0).stop 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 -- 172.21.15.63:0/3854458283 shutdown_connections 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564af602bce0 0x564af60492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564af60ac530 0x564af60ad9a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564af5ea6df0 0x564af604a050 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 --2- 172.21.15.63:0/3854458283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564af6025d60 0x564af6048db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 -- 172.21.15.63:0/3854458283 >> 172.21.15.63:0/3854458283 conn(0x564af5f06f70 msgr2=0x564af60291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce55600700 0 client.15069 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 -- 172.21.15.63:0/3854458283 shutdown_connections 2024-05-27T07:32:26.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.691+0000 7fce80c20b80 1 -- 172.21.15.63:0/3854458283 wait complete. 2024-05-27T07:32:26.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce80c20b80 20 client.15069 trim_cache size 0 max 16384 2024-05-27T07:32:26.687 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:32:26.687 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:32:26.687 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 1, 2024-05-27T07:32:26.687 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34484, 2024-05-27T07:32:26.687 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-27T07:32:26.687 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-27T07:32:26.687 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-27T07:32:26.688 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 157.73811417600001, 2024-05-27T07:32:26.688 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 26, 2024-05-27T07:32:26.688 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-27T07:32:26.688 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:32:26.688 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 277.86401169999999 2024-05-27T07:32:26.688 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce72000700 1 -- 172.21.15.63:0/3999891516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce500b2900 msgr2=0x7fce500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce72000700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce500b2900 0x7fce500b4e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce640a5320 tx=0x7fce64009480 comp rx=0 tx=0).stop 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce72000700 1 -- 172.21.15.63:0/3999891516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce6c110960 msgr2=0x7fce6c1a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce72000700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce6c110960 0x7fce6c1a5b80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fce6400ef30 tx=0x7fce6400c3a0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce72000700 1 -- 172.21.15.63:0/3999891516 shutdown_connections 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce72000700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce500b2900 0x7fce500b4e10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce72000700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce6c110960 0x7fce6c1a5b80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce72000700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce6c10a980 0x7fce6c1a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce72000700 1 --2- 172.21.15.63:0/3999891516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce6c10b2d0 0x7fce6c1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.692+0000 7fce72000700 1 -- 172.21.15.63:0/3999891516 >> 172.21.15.63:0/3999891516 conn(0x7fce6c077c70 msgr2=0x7fce6c07bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.693+0000 7fce72000700 1 -- 172.21.15.63:0/3999891516 shutdown_connections 2024-05-27T07:32:26.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:26.693+0000 7fce72000700 1 -- 172.21.15.63:0/3999891516 wait complete. 2024-05-27T07:32:27.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:27 smithi086 ceph-mon[34687]: pgmap v512: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:27.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:27 smithi149 ceph-mon[35408]: pgmap v512: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:27.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:27 smithi063 ceph-mon[31613]: pgmap v512: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:27.587 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T07:32:28.053 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:29.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:29 smithi086 ceph-mon[34687]: pgmap v513: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:29.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:29 smithi149 ceph-mon[35408]: pgmap v513: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:29.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:29 smithi063 ceph-mon[31613]: pgmap v513: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:30.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:30 smithi063 ceph-mon[31613]: pgmap v514: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:30.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:30 smithi086 ceph-mon[34687]: pgmap v514: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:30.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:30 smithi149 ceph-mon[35408]: pgmap v514: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:30.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.504+0000 7f6fdbe00700 1 -- 172.21.15.63:0/1707247228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd410a980 msgr2=0x7f6fd410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:30.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.504+0000 7f6fdbe00700 1 --2- 172.21.15.63:0/1707247228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd410a980 0x7f6fd410ad90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009990 tx=0x7f6fc0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.504+0000 7f6fdbe00700 1 -- 172.21.15.63:0/1707247228 shutdown_connections 2024-05-27T07:32:30.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.504+0000 7f6fdbe00700 1 --2- 172.21.15.63:0/1707247228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd410b2d0 0x7f6fd4117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.504+0000 7f6fdbe00700 1 --2- 172.21.15.63:0/1707247228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd4110960 0x7f6fd4110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.504+0000 7f6fdbe00700 1 --2- 172.21.15.63:0/1707247228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd410a980 0x7f6fd410ad90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.504+0000 7f6fdbe00700 1 -- 172.21.15.63:0/1707247228 >> 172.21.15.63:0/1707247228 conn(0x7f6fd4077c70 msgr2=0x7f6fd40780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:30.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.504+0000 7f6fdbe00700 1 -- 172.21.15.63:0/1707247228 shutdown_connections 2024-05-27T07:32:30.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.504+0000 7f6fdbe00700 1 -- 172.21.15.63:0/1707247228 wait complete. 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.505+0000 7f6fdbe00700 1 Processor -- start 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.506+0000 7f6fdbe00700 1 -- start start 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.506+0000 7f6fdbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd410a980 0x7f6fd41a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.506+0000 7f6fdbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd410b2d0 0x7f6fd41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.506+0000 7f6fdbe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4110960 0x7f6fd41a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.506+0000 7f6fdbe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd411a980 con 0x7f6fd410b2d0 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.506+0000 7f6fdbe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd411a800 con 0x7f6fd410a980 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.506+0000 7f6fdbe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd411ab00 con 0x7f6fd4110960 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.507+0000 7f6fd8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd410b2d0 0x7f6fd41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.507+0000 7f6fd8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd410b2d0 0x7f6fd41a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36288/0 (socket says 172.21.15.63:36288) 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.507+0000 7f6fd8a00700 1 -- 172.21.15.63:0/2627369227 learned_addr learned my addr 172.21.15.63:0/2627369227 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:30.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.507+0000 7f6fd9400700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd410a980 0x7f6fd41a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.507+0000 7f6fd9e00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4110960 0x7f6fd41a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.507+0000 7f6fd8a00700 1 -- 172.21.15.63:0/2627369227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4110960 msgr2=0x7f6fd41a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.507+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4110960 0x7f6fd41a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.507+0000 7f6fd8a00700 1 -- 172.21.15.63:0/2627369227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd410a980 msgr2=0x7f6fd41a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.507+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd410a980 0x7f6fd41a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.507+0000 7f6fd8a00700 1 -- 172.21.15.63:0/2627369227 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd41a61b0 con 0x7f6fd410b2d0 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.508+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd410b2d0 0x7f6fd41a1930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000b5c0 tx=0x7f6fc00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.508+0000 7f6fd2000700 1 -- 172.21.15.63:0/2627369227 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0004830 con 0x7f6fd410b2d0 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.508+0000 7f6fd2000700 1 -- 172.21.15.63:0/2627369227 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fc0005020 con 0x7f6fd410b2d0 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.508+0000 7f6fd2000700 1 -- 172.21.15.63:0/2627369227 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc000f950 con 0x7f6fd410b2d0 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.508+0000 7f6fdbe00700 1 -- 172.21.15.63:0/2627369227 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd41a6440 con 0x7f6fd410b2d0 2024-05-27T07:32:30.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.508+0000 7f6fdbe00700 1 -- 172.21.15.63:0/2627369227 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd41a6870 con 0x7f6fd410b2d0 2024-05-27T07:32:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.513+0000 7f6fea91db80 1 Processor -- start 2024-05-27T07:32:30.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.513+0000 7f6fea91db80 1 -- start start 2024-05-27T07:32:30.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.514+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.514+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 0x5589205b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.514+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589205b8e20 0x5589205c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.514+0000 7f6fea91db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5589204e6df0 con 0x5589205259a0 2024-05-27T07:32:30.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.514+0000 7f6fea91db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589204d6990 con 0x5589205b8e20 2024-05-27T07:32:30.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.514+0000 7f6fea91db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5589204da9d0 con 0x558920439df0 2024-05-27T07:32:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.514+0000 7f6fd9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.514+0000 7f6fd9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205b82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36296/0 (socket says 172.21.15.63:36296) 2024-05-27T07:32:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.514+0000 7f6fd9e00700 1 -- 172.21.15.63:0/178966270 learned_addr learned my addr 172.21.15.63:0/178966270 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.515+0000 7f6fd9400700 1 --2- 172.21.15.63:0/178966270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 0x5589205b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.515+0000 7f6fd9e00700 1 -- 172.21.15.63:0/178966270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 msgr2=0x5589205b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.515+0000 7f6fd9e00700 1 --2- 172.21.15.63:0/178966270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 0x5589205b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.515+0000 7f6fd9e00700 1 -- 172.21.15.63:0/178966270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589205b8e20 msgr2=0x5589205c50c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:32:30.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.515+0000 7f6fd9e00700 1 --2- 172.21.15.63:0/178966270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589205b8e20 0x5589205c50c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.515+0000 7f6fd9e00700 1 -- 172.21.15.63:0/178966270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55892049f3d0 con 0x5589205259a0 2024-05-27T07:32:30.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.515+0000 7f6fd9400700 1 --2- 172.21.15.63:0/178966270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 0x5589205b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:30.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.516+0000 7f6fd9e00700 1 --2- 172.21.15.63:0/178966270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205b82e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009dc0 tx=0x7f6fcc0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafcba4f4067046a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:30.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.516+0000 7f6fc6a00700 1 -- 172.21.15.63:0/178966270 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc014610 con 0x5589205259a0 2024-05-27T07:32:30.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.516+0000 7f6fc6a00700 1 -- 172.21.15.63:0/178966270 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fcc019070 con 0x5589205259a0 2024-05-27T07:32:30.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.516+0000 7f6fc6a00700 1 -- 172.21.15.63:0/178966270 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0138b0 con 0x5589205259a0 2024-05-27T07:32:30.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.517+0000 7f6fea91db80 1 -- 172.21.15.63:0/178966270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 msgr2=0x5589205b82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:30.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.517+0000 7f6fea91db80 1 --2- 172.21.15.63:0/178966270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205b82e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009dc0 tx=0x7f6fcc0077e0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.517+0000 7f6fea91db80 1 -- 172.21.15.63:0/178966270 shutdown_connections 2024-05-27T07:32:30.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.517+0000 7f6fea91db80 1 --2- 172.21.15.63:0/178966270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589205b8e20 0x5589205c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.517+0000 7f6fea91db80 1 --2- 172.21.15.63:0/178966270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205b82e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.517+0000 7f6fea91db80 1 --2- 172.21.15.63:0/178966270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 0x5589205b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.517+0000 7f6fea91db80 1 -- 172.21.15.63:0/178966270 >> 172.21.15.63:0/178966270 conn(0x5589204d3340 msgr2=0x5589205b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.519+0000 7f6fd2000700 1 -- 172.21.15.63:0/2627369227 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6fc00049d0 con 0x7f6fd410b2d0 2024-05-27T07:32:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.519+0000 7f6fea91db80 1 -- 172.21.15.63:0/178966270 shutdown_connections 2024-05-27T07:32:30.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.519+0000 7f6fea91db80 1 -- 172.21.15.63:0/178966270 wait complete. 2024-05-27T07:32:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.519+0000 7f6fea91db80 1 -- start start 2024-05-27T07:32:30.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.520+0000 7f6fea91db80 1 Processor -- start 2024-05-27T07:32:30.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.520+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 0x5589205c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.520+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589205259a0 0x5589205c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.520+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205c30f0 0x5589205ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.520+0000 7f6fea91db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5589203dcf50 con 0x5589205c30f0 2024-05-27T07:32:30.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.520+0000 7f6fea91db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558920477710 con 0x558920439df0 2024-05-27T07:32:30.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.520+0000 7f6fea91db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5589205c5b30 con 0x5589205259a0 2024-05-27T07:32:30.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.520+0000 7f6fd2000700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6fbc0b2810 0x7f6fbc0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.521+0000 7f6fea91db80 10 client.0 _ll_register_callbacks cb 0x558920480c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:32:30.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.521+0000 7f6fd2000700 1 -- 172.21.15.63:0/2627369227 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f6fc00e34d0 con 0x7f6fd410b2d0 2024-05-27T07:32:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.521+0000 7f6fd8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205c30f0 0x5589205ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.521+0000 7f6fd8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205c30f0 0x5589205ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36308/0 (socket says 172.21.15.63:36308) 2024-05-27T07:32:30.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.521+0000 7f6fd8a00700 1 -- 172.21.15.63:0/1179698860 learned_addr learned my addr 172.21.15.63:0/1179698860 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.521+0000 7f6fd9e00700 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 0x5589205c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.521+0000 7f6fd9400700 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589205259a0 0x5589205c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.521+0000 7f6fd9e00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6fbc0b2810 0x7f6fbc0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fd8a00700 1 -- 172.21.15.63:0/1179698860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589205259a0 msgr2=0x5589205c2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589205259a0 0x5589205c2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fd8a00700 1 -- 172.21.15.63:0/1179698860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 msgr2=0x5589205c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 0x5589205c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fd8a00700 1 -- 172.21.15.63:0/1179698860 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558920426880 con 0x5589205c30f0 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205c30f0 0x5589205ceef0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00e0fb0 tx=0x7f6fc0079ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4537b01b81608ae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fbb400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fbb400700 1 -- 172.21.15.63:0/1179698860 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00051d0 con 0x5589205c30f0 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fbb400700 1 -- 172.21.15.63:0/1179698860 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f6fc00e3200 con 0x5589205c30f0 2024-05-27T07:32:30.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fea91db80 1 -- 172.21.15.63:0/1179698860 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fcc019070 con 0x5589205c30f0 2024-05-27T07:32:30.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.522+0000 7f6fd9e00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6fbc0b2810 0x7f6fbc0b4d20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009660 tx=0x7f6fcc000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:30.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fbb400700 1 -- 172.21.15.63:0/1179698860 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6fc0004cc0 con 0x5589205c30f0 2024-05-27T07:32:30.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fea91db80 10 client.15081 fetch_fsmap learned FSMap version 26 2024-05-27T07:32:30.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fea91db80 1 -- 172.21.15.63:0/1179698860 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589204a7010 con 0x5589205c30f0 2024-05-27T07:32:30.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fbb400700 1 -- 172.21.15.63:0/1179698860 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fcc019070 con 0x5589205c30f0 2024-05-27T07:32:30.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fbb400700 1 -- 172.21.15.63:0/1179698860 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6fc00b3130 con 0x5589205c30f0 2024-05-27T07:32:30.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fea91db80 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:32:30.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fea91db80 10 client.15081 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:32:30.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fea91db80 20 client.15081 populate_metadata read hostname 'smithi063' 2024-05-27T07:32:30.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fea91db80 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589205d2f30 0x5589205d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fea91db80 4 client.15081 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:32:30.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.523+0000 7f6fea91db80 1 -- 172.21.15.63:0/1179698860 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589204e8d40 con 0x5589205d2f30 2024-05-27T07:32:30.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.524+0000 7f6fd9400700 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589205d2f30 0x5589205d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.525+0000 7f6fd9400700 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589205d2f30 0x5589205d43a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a33e730624f6485 server_cookie=f532105dfedca2d1 in_seq=0 out_seq=0 2024-05-27T07:32:30.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.525+0000 7f6fbb400700 10 client.15081 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:32:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.527+0000 7f6fbb400700 1 -- 172.21.15.63:0/1179698860 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5589204e8d40 con 0x5589205d2f30 2024-05-27T07:32:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.527+0000 7f6fbb400700 10 client.15081 handle_command_reply: tid=0 2024-05-27T07:32:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.527+0000 7f6fea91db80 1 client.15081 shutdown 2024-05-27T07:32:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 1 -- 172.21.15.63:0/1179698860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205c30f0 msgr2=0x5589205ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:30.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205c30f0 0x5589205ceef0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00e0fb0 tx=0x7f6fc0079ab0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 1 -- 172.21.15.63:0/1179698860 shutdown_connections 2024-05-27T07:32:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589205d2f30 0x5589205d43a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205c30f0 0x5589205ceef0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 0x5589205c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 1 --2- 172.21.15.63:0/1179698860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589205259a0 0x5589205c2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fbb400700 0 client.15081 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:32:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 1 -- 172.21.15.63:0/1179698860 >> 172.21.15.63:0/1179698860 conn(0x5589204d3340 msgr2=0x5589205b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 1 -- 172.21.15.63:0/1179698860 shutdown_connections 2024-05-27T07:32:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 1 -- 172.21.15.63:0/1179698860 wait complete. 2024-05-27T07:32:30.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.528+0000 7f6fea91db80 20 client.15081 trim_cache size 0 max 16384 2024-05-27T07:32:30.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.552+0000 7f6fea91db80 1 Processor -- start 2024-05-27T07:32:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.552+0000 7f6fea91db80 1 -- start start 2024-05-27T07:32:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.552+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 0x5589205bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.552+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.552+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589205c30f0 0x5589205c0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.552+0000 7f6fea91db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5589205d26e0 con 0x5589205259a0 2024-05-27T07:32:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.552+0000 7f6fea91db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589205d2db0 con 0x5589205c30f0 2024-05-27T07:32:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.552+0000 7f6fea91db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5589205d3480 con 0x558920439df0 2024-05-27T07:32:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.552+0000 7f6fd8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205c2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36312/0 (socket says 172.21.15.63:36312) 2024-05-27T07:32:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd8a00700 1 -- 172.21.15.63:0/910279578 learned_addr learned my addr 172.21.15.63:0/910279578 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd9400700 1 --2- 172.21.15.63:0/910279578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 0x5589205bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd9e00700 1 --2- 172.21.15.63:0/910279578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589205c30f0 0x5589205c0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd8a00700 1 -- 172.21.15.63:0/910279578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 msgr2=0x5589205bdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/910279578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 0x5589205bdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd8a00700 1 -- 172.21.15.63:0/910279578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589205c30f0 msgr2=0x5589205c0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/910279578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589205c30f0 0x5589205c0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd8a00700 1 -- 172.21.15.63:0/910279578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589205b4fb0 con 0x5589205259a0 2024-05-27T07:32:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd9e00700 1 --2- 172.21.15.63:0/910279578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589205c30f0 0x5589205c0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:32:30.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.553+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/910279578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205c2b60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00aa4f0 tx=0x7f6fc00aa1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0e1cb0e0c7656 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:30.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.554+0000 7f6fc6000700 1 -- 172.21.15.63:0/910279578 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00051d0 con 0x5589205259a0 2024-05-27T07:32:30.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.554+0000 7f6fc6000700 1 -- 172.21.15.63:0/910279578 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fc00b10f0 con 0x5589205259a0 2024-05-27T07:32:30.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.554+0000 7f6fc6000700 1 -- 172.21.15.63:0/910279578 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00b3130 con 0x5589205259a0 2024-05-27T07:32:30.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.554+0000 7f6fea91db80 1 -- 172.21.15.63:0/910279578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 msgr2=0x5589205c2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:30.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.555+0000 7f6fea91db80 1 --2- 172.21.15.63:0/910279578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205c2b60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00aa4f0 tx=0x7f6fc00aa1f0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.555+0000 7f6fea91db80 1 -- 172.21.15.63:0/910279578 shutdown_connections 2024-05-27T07:32:30.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.555+0000 7f6fea91db80 1 --2- 172.21.15.63:0/910279578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589205c30f0 0x5589205c0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.555+0000 7f6fea91db80 1 --2- 172.21.15.63:0/910279578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558920439df0 0x5589205bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.555+0000 7f6fea91db80 1 --2- 172.21.15.63:0/910279578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205c2b60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.555+0000 7f6fea91db80 1 -- 172.21.15.63:0/910279578 >> 172.21.15.63:0/910279578 conn(0x5589204d3340 msgr2=0x5589205c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:30.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.555+0000 7f6fea91db80 1 -- 172.21.15.63:0/910279578 shutdown_connections 2024-05-27T07:32:30.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.555+0000 7f6fea91db80 1 -- 172.21.15.63:0/910279578 wait complete. 2024-05-27T07:32:30.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.555+0000 7f6fea91db80 1 -- start start 2024-05-27T07:32:30.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.556+0000 7f6fea91db80 1 Processor -- start 2024-05-27T07:32:30.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.556+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 0x5589205ddc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.556+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.556+0000 7f6fea91db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589205c30f0 0x5589205dcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.556+0000 7f6fea91db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5589205d4220 con 0x5589205259a0 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.556+0000 7f6fea91db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589205d3b50 con 0x558920439df0 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.556+0000 7f6fea91db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5589205d48f0 con 0x5589205c30f0 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.556+0000 7f6fea91db80 10 client.0 _ll_register_callbacks cb 0x558920480c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.556+0000 7f6fd8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205dc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205dc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36320/0 (socket says 172.21.15.63:36320) 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd8a00700 1 -- 172.21.15.63:0/3243968058 learned_addr learned my addr 172.21.15.63:0/3243968058 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd9400700 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 0x5589205ddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd9e00700 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589205c30f0 0x5589205dcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd8a00700 1 -- 172.21.15.63:0/3243968058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589205c30f0 msgr2=0x5589205dcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589205c30f0 0x5589205dcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd8a00700 1 -- 172.21.15.63:0/3243968058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 msgr2=0x5589205ddc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 0x5589205ddc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd8a00700 1 -- 172.21.15.63:0/3243968058 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589205dd330 con 0x5589205259a0 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd9400700 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 0x5589205ddc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fd8a00700 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205dc8b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00a6d90 tx=0x7f6fc00aaef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87f4157b33de3f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.557+0000 7f6fbb400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.558+0000 7f6fbb400700 1 -- 172.21.15.63:0/3243968058 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00c9070 con 0x5589205259a0 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.558+0000 7f6fbb400700 1 -- 172.21.15.63:0/3243968058 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f6fc00bfdc0 con 0x5589205259a0 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.558+0000 7f6fea91db80 1 -- 172.21.15.63:0/3243968058 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fc00b10f0 con 0x5589205259a0 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.558+0000 7f6fbb400700 1 -- 172.21.15.63:0/3243968058 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6fc00051d0 con 0x5589205259a0 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.558+0000 7f6fea91db80 10 client.15087 fetch_fsmap learned FSMap version 26 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.558+0000 7f6fea91db80 1 -- 172.21.15.63:0/3243968058 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55892063f190 con 0x5589205259a0 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.558+0000 7f6fbb400700 1 -- 172.21.15.63:0/3243968058 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fc00b10f0 con 0x5589205259a0 2024-05-27T07:32:30.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.559+0000 7f6fbb400700 1 -- 172.21.15.63:0/3243968058 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6fc00abb20 con 0x5589205259a0 2024-05-27T07:32:30.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.559+0000 7f6fea91db80 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:32:30.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.559+0000 7f6fea91db80 10 client.15087 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:32:30.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.559+0000 7f6fea91db80 20 client.15087 populate_metadata read hostname 'smithi063' 2024-05-27T07:32:30.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.559+0000 7f6fea91db80 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558920640f60 0x5589206423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:30.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.559+0000 7f6fea91db80 4 client.15087 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:32:30.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.559+0000 7f6fea91db80 1 -- 172.21.15.63:0/3243968058 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "status"}) -- 0x5589204e2520 con 0x558920640f60 2024-05-27T07:32:30.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.560+0000 7f6fd9400700 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558920640f60 0x5589206423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:30.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.561+0000 7f6fd9400700 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558920640f60 0x5589206423d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b12d202f527b0d server_cookie=a0d0c868d2ea19da in_seq=0 out_seq=0 2024-05-27T07:32:30.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.561+0000 7f6fbb400700 10 client.15087 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fbb400700 1 -- 172.21.15.63:0/3243968058 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5589204e2520 con 0x558920640f60 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fbb400700 10 client.15087 handle_command_reply: tid=0 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 client.15087 shutdown 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 -- 172.21.15.63:0/3243968058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 msgr2=0x5589205dc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205dc8b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00a6d90 tx=0x7f6fc00aaef0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 -- 172.21.15.63:0/3243968058 shutdown_connections 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558920640f60 0x5589206423d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589205c30f0 0x5589205dcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558920439df0 0x5589205ddc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 --2- 172.21.15.63:0/3243968058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589205259a0 0x5589205dc8b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 -- 172.21.15.63:0/3243968058 >> 172.21.15.63:0/3243968058 conn(0x5589204d3340 msgr2=0x5589205b8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fbb400700 0 client.15087 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 -- 172.21.15.63:0/3243968058 shutdown_connections 2024-05-27T07:32:30.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 1 -- 172.21.15.63:0/3243968058 wait complete. 2024-05-27T07:32:30.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.562+0000 7f6fea91db80 20 client.15087 trim_cache size 0 max 16384 2024-05-27T07:32:30.558 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:32:30.558 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:32:30.558 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 0, 2024-05-27T07:32:30.558 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34520, 2024-05-27T07:32:30.558 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-27T07:32:30.558 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-27T07:32:30.558 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-27T07:32:30.558 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 242.94664133200001, 2024-05-27T07:32:30.559 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 26, 2024-05-27T07:32:30.559 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-27T07:32:30.559 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:32:30.559 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 264.33089280799999 2024-05-27T07:32:30.559 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:32:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.563+0000 7f6fdbe00700 1 -- 172.21.15.63:0/2627369227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6fbc0b2810 msgr2=0x7f6fbc0b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:30.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.563+0000 7f6fdbe00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6fbc0b2810 0x7f6fbc0b4d20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009660 tx=0x7f6fcc000f40 comp rx=0 tx=0).stop 2024-05-27T07:32:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.563+0000 7f6fdbe00700 1 -- 172.21.15.63:0/2627369227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd410b2d0 msgr2=0x7f6fd41a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.563+0000 7f6fdbe00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd410b2d0 0x7f6fd41a1930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000b5c0 tx=0x7f6fc00047b0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.563+0000 7f6fdbe00700 1 -- 172.21.15.63:0/2627369227 shutdown_connections 2024-05-27T07:32:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.563+0000 7f6fdbe00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd4110960 0x7f6fd41a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.563+0000 7f6fdbe00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd410a980 0x7f6fd41a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.563+0000 7f6fdbe00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f6fbc0b2810 0x7f6fbc0b4d20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.563+0000 7f6fdbe00700 1 --2- 172.21.15.63:0/2627369227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd410b2d0 0x7f6fd41a1930 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.563+0000 7f6fdbe00700 1 -- 172.21.15.63:0/2627369227 >> 172.21.15.63:0/2627369227 conn(0x7f6fd4077c70 msgr2=0x7f6fd407b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.564+0000 7f6fdbe00700 1 -- 172.21.15.63:0/2627369227 shutdown_connections 2024-05-27T07:32:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:30.564+0000 7f6fdbe00700 1 -- 172.21.15.63:0/2627369227 wait complete. 2024-05-27T07:32:31.515 INFO:tasks.cephadm:Setup complete, yielding 2024-05-27T07:32:31.515 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T07:32:31.525 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T07:32:31.525 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-05-27T07:32:31.994 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:33.141 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:32 smithi063 ceph-mon[31613]: pgmap v515: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:33.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:32 smithi086 ceph-mon[34687]: pgmap v515: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:33.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:32 smithi149 ceph-mon[35408]: pgmap v515: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:34.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.320+0000 7fd9a0c00700 1 -- 172.21.15.63:0/745731032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd99c107680 msgr2=0x7fd99c107a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:34.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.320+0000 7fd9a0c00700 1 --2- 172.21.15.63:0/745731032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd99c107680 0x7fd99c107a90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd988009990 tx=0x7fd988009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.320+0000 7fd9a0c00700 1 -- 172.21.15.63:0/745731032 shutdown_connections 2024-05-27T07:32:34.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.320+0000 7fd9a0c00700 1 --2- 172.21.15.63:0/745731032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c108020 0x7fd99c10e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.320+0000 7fd9a0c00700 1 --2- 172.21.15.63:0/745731032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd99c106550 0x7fd99c106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.320+0000 7fd9a0c00700 1 --2- 172.21.15.63:0/745731032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd99c107680 0x7fd99c107a90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.320+0000 7fd9a0c00700 1 -- 172.21.15.63:0/745731032 >> 172.21.15.63:0/745731032 conn(0x7fd99c101b90 msgr2=0x7fd99c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:34.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.320+0000 7fd9a0c00700 1 -- 172.21.15.63:0/745731032 shutdown_connections 2024-05-27T07:32:34.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.320+0000 7fd9a0c00700 1 -- 172.21.15.63:0/745731032 wait complete. 2024-05-27T07:32:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.321+0000 7fd9a0c00700 1 Processor -- start 2024-05-27T07:32:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.322+0000 7fd9a0c00700 1 -- start start 2024-05-27T07:32:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.322+0000 7fd9a0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd99c106550 0x7fd99c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.322+0000 7fd9a0c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd99c107680 0x7fd99c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.322+0000 7fd9a0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c108020 0x7fd99c1a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.322+0000 7fd9a0c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c111e60 con 0x7fd99c106550 2024-05-27T07:32:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.322+0000 7fd9a0c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c111ce0 con 0x7fd99c108020 2024-05-27T07:32:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.322+0000 7fd9a0c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c111fe0 con 0x7fd99c107680 2024-05-27T07:32:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.322+0000 7fd999e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd99c106550 0x7fd99c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.322+0000 7fd999e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd99c106550 0x7fd99c19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36336/0 (socket says 172.21.15.63:36336) 2024-05-27T07:32:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.322+0000 7fd999e00700 1 -- 172.21.15.63:0/368428104 learned_addr learned my addr 172.21.15.63:0/368428104 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.323+0000 7fd99a800700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c108020 0x7fd99c1a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.323+0000 7fd999e00700 1 -- 172.21.15.63:0/368428104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd99c107680 msgr2=0x7fd99c19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.323+0000 7fd999e00700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd99c107680 0x7fd99c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.323+0000 7fd999e00700 1 -- 172.21.15.63:0/368428104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c108020 msgr2=0x7fd99c1a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.323+0000 7fd999e00700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c108020 0x7fd99c1a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.323+0000 7fd999e00700 1 -- 172.21.15.63:0/368428104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c1a45f0 con 0x7fd99c106550 2024-05-27T07:32:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.323+0000 7fd999e00700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd99c106550 0x7fd99c19ca90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00d950 tx=0x7fd98c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.324+0000 7fd992a00700 1 -- 172.21.15.63:0/368428104 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c017070 con 0x7fd99c106550 2024-05-27T07:32:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.324+0000 7fd992a00700 1 -- 172.21.15.63:0/368428104 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd98c00de30 con 0x7fd99c106550 2024-05-27T07:32:34.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.324+0000 7fd9a0c00700 1 -- 172.21.15.63:0/368428104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c1a48e0 con 0x7fd99c106550 2024-05-27T07:32:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.324+0000 7fd992a00700 1 -- 172.21.15.63:0/368428104 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c003aa0 con 0x7fd99c106550 2024-05-27T07:32:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.324+0000 7fd9a0c00700 1 -- 172.21.15.63:0/368428104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c1a4e20 con 0x7fd99c106550 2024-05-27T07:32:34.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.325+0000 7fd9a0c00700 1 -- 172.21.15.63:0/368428104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99c0512b0 con 0x7fd99c106550 2024-05-27T07:32:34.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.328+0000 7fd992a00700 1 -- 172.21.15.63:0/368428104 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd98c003c40 con 0x7fd99c106550 2024-05-27T07:32:34.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.330+0000 7fd992a00700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd9800b2880 0x7fd9800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:34.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.330+0000 7fd992a00700 1 -- 172.21.15.63:0/368428104 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fd98c0e21b0 con 0x7fd99c106550 2024-05-27T07:32:34.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.330+0000 7fd999400700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd9800b2880 0x7fd9800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:34.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.331+0000 7fd999400700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd9800b2880 0x7fd9800b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd9880098e0 tx=0x7fd988005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:34.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.337+0000 7fd992a00700 1 -- 172.21.15.63:0/368428104 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd98c020030 con 0x7fd99c106550 2024-05-27T07:32:34.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.596+0000 7fd9a0c00700 1 -- 172.21.15.63:0/368428104 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd99c063cb0 con 0x7fd9800b2880 2024-05-27T07:32:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.598+0000 7fd992a00700 1 -- 172.21.15.63:0/368428104 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd99c063cb0 con 0x7fd9800b2880 2024-05-27T07:32:34.592 INFO:teuthology.orchestra.run.smithi063.stdout:Backend: cephadm 2024-05-27T07:32:34.592 INFO:teuthology.orchestra.run.smithi063.stdout:Available: Yes 2024-05-27T07:32:34.592 INFO:teuthology.orchestra.run.smithi063.stdout:Paused: No 2024-05-27T07:32:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.601+0000 7fd9a0c00700 1 -- 172.21.15.63:0/368428104 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd9800b2880 msgr2=0x7fd9800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.601+0000 7fd9a0c00700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd9800b2880 0x7fd9800b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd9880098e0 tx=0x7fd988005fb0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.601+0000 7fd9a0c00700 1 -- 172.21.15.63:0/368428104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd99c106550 msgr2=0x7fd99c19ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.601+0000 7fd9a0c00700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd99c106550 0x7fd99c19ca90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00d950 tx=0x7fd98c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:32:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.602+0000 7fd9a0c00700 1 -- 172.21.15.63:0/368428104 shutdown_connections 2024-05-27T07:32:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.602+0000 7fd9a0c00700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c108020 0x7fd99c1a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.602+0000 7fd9a0c00700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fd9800b2880 0x7fd9800b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.602+0000 7fd9a0c00700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd99c106550 0x7fd99c19ca90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.602+0000 7fd9a0c00700 1 --2- 172.21.15.63:0/368428104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd99c107680 0x7fd99c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.602+0000 7fd9a0c00700 1 -- 172.21.15.63:0/368428104 >> 172.21.15.63:0/368428104 conn(0x7fd99c101b90 msgr2=0x7fd99c10a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.602+0000 7fd9a0c00700 1 -- 172.21.15.63:0/368428104 shutdown_connections 2024-05-27T07:32:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:34.602+0000 7fd9a0c00700 1 -- 172.21.15.63:0/368428104 wait complete. 2024-05-27T07:32:35.382 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:34 smithi063 ceph-mon[31613]: pgmap v516: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:35.444 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-27T07:32:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:34 smithi086 ceph-mon[34687]: pgmap v516: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:34 smithi149 ceph-mon[35408]: pgmap v516: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:35.946 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:36.279 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:35 smithi063 ceph-mon[31613]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:36.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:35 smithi086 ceph-mon[34687]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:36.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:35 smithi149 ceph-mon[35408]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:37.373 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:36 smithi063 ceph-mon[31613]: pgmap v517: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:32:37.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:36 smithi086 ceph-mon[34687]: pgmap v517: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:32:37.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:36 smithi149 ceph-mon[35408]: pgmap v517: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:32:38.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.285+0000 7fe518200700 1 -- 172.21.15.63:0/3622693948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe510105e00 msgr2=0x7fe51010c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:38.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.285+0000 7fe518200700 1 --2- 172.21.15.63:0/3622693948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe510105e00 0x7fe51010c6b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe50800b0a0 tx=0x7fe50800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.285+0000 7fe518200700 1 -- 172.21.15.63:0/3622693948 shutdown_connections 2024-05-27T07:32:38.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.285+0000 7fe518200700 1 --2- 172.21.15.63:0/3622693948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe510105e00 0x7fe51010c6b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.285+0000 7fe518200700 1 --2- 172.21.15.63:0/3622693948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510104330 0x7fe510104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.285+0000 7fe518200700 1 --2- 172.21.15.63:0/3622693948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe510105460 0x7fe510105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.285+0000 7fe518200700 1 -- 172.21.15.63:0/3622693948 >> 172.21.15.63:0/3622693948 conn(0x7fe5100ff9b0 msgr2=0x7fe510101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:38.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.285+0000 7fe518200700 1 -- 172.21.15.63:0/3622693948 shutdown_connections 2024-05-27T07:32:38.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.285+0000 7fe518200700 1 -- 172.21.15.63:0/3622693948 wait complete. 2024-05-27T07:32:38.282 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:37 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:38.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.287+0000 7fe518200700 1 Processor -- start 2024-05-27T07:32:38.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.287+0000 7fe518200700 1 -- start start 2024-05-27T07:32:38.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.287+0000 7fe518200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe510104330 0x7fe51019c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:38.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.287+0000 7fe518200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105460 0x7fe51019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.287+0000 7fe518200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe510105e00 0x7fe5101a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.287+0000 7fe518200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe51010fc50 con 0x7fe510104330 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.287+0000 7fe518200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe51010fad0 con 0x7fe510105460 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.287+0000 7fe518200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe51010fdd0 con 0x7fe510105e00 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe515800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe510104330 0x7fe51019c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe515800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe510104330 0x7fe51019c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36360/0 (socket says 172.21.15.63:36360) 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe515800700 1 -- 172.21.15.63:0/1557512085 learned_addr learned my addr 172.21.15.63:0/1557512085 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe516200700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe510105e00 0x7fe5101a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe514e00700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105460 0x7fe51019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe515800700 1 -- 172.21.15.63:0/1557512085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe510105e00 msgr2=0x7fe5101a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe515800700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe510105e00 0x7fe5101a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe515800700 1 -- 172.21.15.63:0/1557512085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105460 msgr2=0x7fe51019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:38.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe515800700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105460 0x7fe51019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe515800700 1 -- 172.21.15.63:0/1557512085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5101a4530 con 0x7fe510104330 2024-05-27T07:32:38.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe514e00700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105460 0x7fe51019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:38.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.288+0000 7fe516200700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe510105e00 0x7fe5101a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:38.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.289+0000 7fe515800700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe510104330 0x7fe51019c9d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe50400d950 tx=0x7fe50400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:38.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.289+0000 7fe50e000700 1 -- 172.21.15.63:0/1557512085 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504017070 con 0x7fe510104330 2024-05-27T07:32:38.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.289+0000 7fe50e000700 1 -- 172.21.15.63:0/1557512085 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe50400de30 con 0x7fe510104330 2024-05-27T07:32:38.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.289+0000 7fe518200700 1 -- 172.21.15.63:0/1557512085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5101a4820 con 0x7fe510104330 2024-05-27T07:32:38.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.289+0000 7fe50e000700 1 -- 172.21.15.63:0/1557512085 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504003aa0 con 0x7fe510104330 2024-05-27T07:32:38.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.289+0000 7fe518200700 1 -- 172.21.15.63:0/1557512085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5101a4d60 con 0x7fe510104330 2024-05-27T07:32:38.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.291+0000 7fe518200700 1 -- 172.21.15.63:0/1557512085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5100512b0 con 0x7fe510104330 2024-05-27T07:32:38.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.294+0000 7fe50e000700 1 -- 172.21.15.63:0/1557512085 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe504003c40 con 0x7fe510104330 2024-05-27T07:32:38.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.295+0000 7fe50e000700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4f80b2880 0x7fe4f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:38.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.295+0000 7fe50e000700 1 -- 172.21.15.63:0/1557512085 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fe5040e2950 con 0x7fe510104330 2024-05-27T07:32:38.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.295+0000 7fe514e00700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4f80b2880 0x7fe4f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:38.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.296+0000 7fe514e00700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4f80b2880 0x7fe4f80b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005950 tx=0x7fe4fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:38.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.298+0000 7fe50e000700 1 -- 172.21.15.63:0/1557512085 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe504020030 con 0x7fe510104330 2024-05-27T07:32:38.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:37 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:38.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:37 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:38.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.554+0000 7fe518200700 1 -- 172.21.15.63:0/1557512085 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe510077100 con 0x7fe4f80b2880 2024-05-27T07:32:38.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.574+0000 7fe50e000700 1 -- 172.21.15.63:0/1557512085 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fe510077100 con 0x7fe4f80b2880 2024-05-27T07:32:38.568 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T07:32:38.568 INFO:teuthology.orchestra.run.smithi063.stdout:mds.a smithi063 running (4m) 2m ago 4m 80.3M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 c0c4c4595f4f 2024-05-27T07:32:38.568 INFO:teuthology.orchestra.run.smithi063.stdout:mds.b smithi086 running (4m) 2m ago 4m 92.0M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 db608efef4d2 2024-05-27T07:32:38.568 INFO:teuthology.orchestra.run.smithi063.stdout:mds.c smithi149 running (4m) 2m ago 4m 82.5M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 f582fedf84be 2024-05-27T07:32:38.568 INFO:teuthology.orchestra.run.smithi063.stdout:mds.d smithi063 running (5m) 2m ago 5m 80.5M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 e7462a8dbccd 2024-05-27T07:32:38.568 INFO:teuthology.orchestra.run.smithi063.stdout:mds.e smithi086 running (5m) 2m ago 5m 83.5M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 409476ac200c 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mds.f smithi149 running (4m) 2m ago 4m 80.6M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 1e9496991c7b 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mds.g smithi063 running (4m) 2m ago 4m 80.3M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 906bc25a3e3b 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mds.h smithi086 running (4m) 2m ago 4m 83.6M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 f61d2b53dbb7 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mds.i smithi149 running (4m) 2m ago 4m 82.4M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 82900682d407 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mds.j smithi063 running (5m) 2m ago 5m 80.5M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 10e29a032148 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mds.k smithi086 running (5m) 2m ago 5m 83.8M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 b6a4ca6736a2 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mds.l smithi149 running (5m) 2m ago 5m 80.6M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 b6ae53a70b5f 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.x smithi063 *:9283,8765 running (15m) 2m ago 15m 599M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 806b8430f4eb 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.y smithi086 *:8443,8765 running (12m) 2m ago 12m 537M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 df98b87f3baf 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.z smithi149 *:8443,8765 running (12m) 2m ago 12m 534M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 d58d6f862e23 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mon.a smithi063 running (15m) 2m ago 15m 119M 2048M 19.0.0-3872-g38f9d900 e36ea4fc8a58 07e5df3f996b 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mon.b smithi086 running (13m) 2m ago 13m 107M 2048M 19.0.0-3872-g38f9d900 e36ea4fc8a58 4467d0c1d437 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:mon.c smithi149 running (13m) 2m ago 13m 108M 2048M 19.0.0-3872-g38f9d900 e36ea4fc8a58 f1a4fd65cc3f 2024-05-27T07:32:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (12m) 2m ago 12m 112M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 4b6ab480bf52 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi086 running (11m) 2m ago 11m 122M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 6b3cc3ffde21 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi149 running (11m) 2m ago 11m 113M 1454M 19.0.0-3872-g38f9d900 e36ea4fc8a58 bee2e341bc93 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi063 running (10m) 2m ago 10m 118M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 ae5a25c3c5c7 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi086 running (10m) 2m ago 10m 117M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 27054a3ed759 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi149 running (9m) 2m ago 9m 118M 1454M 19.0.0-3872-g38f9d900 e36ea4fc8a58 9a453d27138b 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6 smithi063 running (8m) 2m ago 8m 109M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 7f531da015bb 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7 smithi086 running (8m) 2m ago 8m 119M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 6062f7dcbed3 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.8 smithi149 running (7m) 2m ago 7m 116M 1454M 19.0.0-3872-g38f9d900 e36ea4fc8a58 8a16841b5ae2 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.9 smithi063 running (6m) 2m ago 6m 109M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 b10793112eee 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.10 smithi086 running (6m) 2m ago 6m 110M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 629ec773f6aa 2024-05-27T07:32:38.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.11 smithi149 running (5m) 2m ago 5m 116M 1454M 19.0.0-3872-g38f9d900 e36ea4fc8a58 090f8212b0e2 2024-05-27T07:32:38.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.577+0000 7fe518200700 1 -- 172.21.15.63:0/1557512085 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4f80b2880 msgr2=0x7fe4f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:38.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.577+0000 7fe518200700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4f80b2880 0x7fe4f80b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005950 tx=0x7fe4fc0058e0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.577+0000 7fe518200700 1 -- 172.21.15.63:0/1557512085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe510104330 msgr2=0x7fe51019c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:38.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.577+0000 7fe518200700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe510104330 0x7fe51019c9d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe50400d950 tx=0x7fe50400dd10 comp rx=0 tx=0).stop 2024-05-27T07:32:38.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.577+0000 7fe518200700 1 -- 172.21.15.63:0/1557512085 shutdown_connections 2024-05-27T07:32:38.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.577+0000 7fe518200700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe510105e00 0x7fe5101a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.577+0000 7fe518200700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4f80b2880 0x7fe4f80b4d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.577+0000 7fe518200700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe510104330 0x7fe51019c9d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.577+0000 7fe518200700 1 --2- 172.21.15.63:0/1557512085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105460 0x7fe51019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:38.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.577+0000 7fe518200700 1 -- 172.21.15.63:0/1557512085 >> 172.21.15.63:0/1557512085 conn(0x7fe5100ff9b0 msgr2=0x7fe5100759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:38.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.578+0000 7fe518200700 1 -- 172.21.15.63:0/1557512085 shutdown_connections 2024-05-27T07:32:38.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:38.578+0000 7fe518200700 1 -- 172.21.15.63:0/1557512085 wait complete. 2024-05-27T07:32:39.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:39 smithi063 ceph-mon[31613]: pgmap v518: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-27T07:32:39.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:39 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:39.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:39 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:39.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:39 smithi063 ceph-mon[31613]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:32:39.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:39 smithi086 ceph-mon[34687]: pgmap v518: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-27T07:32:39.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:39 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:39.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:39 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:39.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:39 smithi086 ceph-mon[34687]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:32:39.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:39 smithi149 ceph-mon[35408]: pgmap v518: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-27T07:32:39.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:39 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:39.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:39 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:39.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:39 smithi149 ceph-mon[35408]: from='mgr.14152 172.21.15.63:0/3968546523' entity='mgr.x' 2024-05-27T07:32:39.530 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-27T07:32:39.998 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:40.402 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:40 smithi063 ceph-mon[31613]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:40.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:40 smithi086 ceph-mon[34687]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:40.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:40 smithi149 ceph-mon[35408]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:41.447 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:41 smithi063 ceph-mon[31613]: pgmap v519: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-27T07:32:41.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:41 smithi086 ceph-mon[34687]: pgmap v519: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-27T07:32:41.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:41 smithi149 ceph-mon[35408]: pgmap v519: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-27T07:32:42.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.349+0000 7fa9fb200700 1 -- 172.21.15.63:0/1904625227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9fc107280 msgr2=0x7fa9fc109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.349+0000 7fa9fb200700 1 --2- 172.21.15.63:0/1904625227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9fc107280 0x7fa9fc109760 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009990 tx=0x7fa9e8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.349+0000 7fa9fb200700 1 -- 172.21.15.63:0/1904625227 shutdown_connections 2024-05-27T07:32:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.349+0000 7fa9fb200700 1 --2- 172.21.15.63:0/1904625227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc109df0 0x7fa9fc10c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.349+0000 7fa9fb200700 1 --2- 172.21.15.63:0/1904625227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc06c2c0 0x7fa9fc106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.349+0000 7fa9fb200700 1 --2- 172.21.15.63:0/1904625227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9fc107280 0x7fa9fc109760 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.349+0000 7fa9fb200700 1 -- 172.21.15.63:0/1904625227 >> 172.21.15.63:0/1904625227 conn(0x7fa9fc0fe680 msgr2=0x7fa9fc100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.349+0000 7fa9fb200700 1 -- 172.21.15.63:0/1904625227 shutdown_connections 2024-05-27T07:32:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.349+0000 7fa9fb200700 1 -- 172.21.15.63:0/1904625227 wait complete. 2024-05-27T07:32:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.350+0000 7fa9fb200700 1 Processor -- start 2024-05-27T07:32:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.351+0000 7fa9fb200700 1 -- start start 2024-05-27T07:32:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.351+0000 7fa9fb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9fc06c2c0 0x7fa9fc104500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.351+0000 7fa9fb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc107280 0x7fa9fc102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.351+0000 7fa9fb200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc109df0 0x7fa9fc1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.351+0000 7fa9fb200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc10fa90 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.351+0000 7fa9fb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc10f910 con 0x7fa9fc107280 2024-05-27T07:32:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.351+0000 7fa9fb200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc10fc10 con 0x7fa9fc109df0 2024-05-27T07:32:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.352+0000 7fa9f9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9fc06c2c0 0x7fa9fc104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.352+0000 7fa9f9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9fc06c2c0 0x7fa9fc104500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47944/0 (socket says 172.21.15.63:47944) 2024-05-27T07:32:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.352+0000 7fa9f9e00700 1 -- 172.21.15.63:0/3337996661 learned_addr learned my addr 172.21.15.63:0/3337996661 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.352+0000 7fa9f9400700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc107280 0x7fa9fc102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.352+0000 7fa9f9e00700 1 -- 172.21.15.63:0/3337996661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc109df0 msgr2=0x7fa9fc1030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.352+0000 7fa9f9e00700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc109df0 0x7fa9fc1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.352+0000 7fa9f9e00700 1 -- 172.21.15.63:0/3337996661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc107280 msgr2=0x7fa9fc102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.352+0000 7fa9f9e00700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc107280 0x7fa9fc102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.352+0000 7fa9f9e00700 1 -- 172.21.15.63:0/3337996661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc103850 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.353+0000 7fa9f9e00700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9fc06c2c0 0x7fa9fc104500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00d950 tx=0x7fa9ec00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.353+0000 7fa9f2a00700 1 -- 172.21.15.63:0/3337996661 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec016070 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.353+0000 7fa9f2a00700 1 -- 172.21.15.63:0/3337996661 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9ec010460 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.353+0000 7fa9f2a00700 1 -- 172.21.15.63:0/3337996661 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec00f620 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.353+0000 7fa9fb200700 1 -- 172.21.15.63:0/3337996661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9fc1a4730 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.353+0000 7fa9fb200700 1 -- 172.21.15.63:0/3337996661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9fc1a4bc0 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.355+0000 7fa9fb200700 1 -- 172.21.15.63:0/3337996661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9fc0512b0 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.357+0000 7fa9f2a00700 1 -- 172.21.15.63:0/3337996661 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa9ec019070 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.358+0000 7fa9f2a00700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa9e00b2880 0x7fa9e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:42.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.359+0000 7fa9f2a00700 1 -- 172.21.15.63:0/3337996661 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fa9ec0e2210 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.359+0000 7fa9f9400700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa9e00b2880 0x7fa9e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:42.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.359+0000 7fa9f9400700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa9e00b2880 0x7fa9e00b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:42.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.363+0000 7fa9f2a00700 1 -- 172.21.15.63:0/3337996661 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9ec0a9b40 con 0x7fa9fc06c2c0 2024-05-27T07:32:42.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.611+0000 7fa9fb200700 1 -- 172.21.15.63:0/3337996661 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fa9fc063cb0 con 0x7fa9e00b2880 2024-05-27T07:32:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.629+0000 7fa9f2a00700 1 -- 172.21.15.63:0/3337996661 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fa9fc063cb0 con 0x7fa9e00b2880 2024-05-27T07:32:42.623 INFO:teuthology.orchestra.run.smithi063.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-27T07:32:42.623 INFO:teuthology.orchestra.run.smithi063.stdout:mds.all 12/12 2m ago 5m smithi063=a;smithi063=d;smithi063=g;smithi063=j;smithi086=b;smithi086=e;smithi086=h;smithi086=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-05-27T07:32:42.623 INFO:teuthology.orchestra.run.smithi063.stdout:mgr 3/3 2m ago 13m smithi063=x;smithi086=y;smithi149=z;count:3 2024-05-27T07:32:42.623 INFO:teuthology.orchestra.run.smithi063.stdout:mon 3/3 2m ago 13m smithi063:172.21.15.63=a;smithi086:172.21.15.86=b;smithi149:172.21.15.149=c;count:3 2024-05-27T07:32:42.623 INFO:teuthology.orchestra.run.smithi063.stdout:osd 12 2m ago - 2024-05-27T07:32:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 -- 172.21.15.63:0/3337996661 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa9e00b2880 msgr2=0x7fa9e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa9e00b2880 0x7fa9e00b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e801a040 comp rx=0 tx=0).stop 2024-05-27T07:32:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 -- 172.21.15.63:0/3337996661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9fc06c2c0 msgr2=0x7fa9fc104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9fc06c2c0 0x7fa9fc104500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00d950 tx=0x7fa9ec00dd10 comp rx=0 tx=0).stop 2024-05-27T07:32:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 -- 172.21.15.63:0/3337996661 shutdown_connections 2024-05-27T07:32:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9fc109df0 0x7fa9fc1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa9e00b2880 0x7fa9e00b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9fc06c2c0 0x7fa9fc104500 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 --2- 172.21.15.63:0/3337996661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9fc107280 0x7fa9fc102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 -- 172.21.15.63:0/3337996661 >> 172.21.15.63:0/3337996661 conn(0x7fa9fc0fe680 msgr2=0x7fa9fc1068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 -- 172.21.15.63:0/3337996661 shutdown_connections 2024-05-27T07:32:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:42.632+0000 7fa9fb200700 1 -- 172.21.15.63:0/3337996661 wait complete. 2024-05-27T07:32:43.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:43 smithi086 ceph-mon[34687]: pgmap v520: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-27T07:32:43.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:43 smithi149 ceph-mon[35408]: pgmap v520: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-27T07:32:43.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:43 smithi063 ceph-mon[31613]: pgmap v520: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-05-27T07:32:43.563 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-27T07:32:44.047 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:44.382 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:44 smithi063 ceph-mon[31613]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:44.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:44 smithi086 ceph-mon[34687]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:44.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:44 smithi149 ceph-mon[35408]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:45.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:45 smithi086 ceph-mon[34687]: pgmap v521: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-27T07:32:45.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:45 smithi149 ceph-mon[35408]: pgmap v521: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-27T07:32:45.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:45 smithi063 ceph-mon[31613]: pgmap v521: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-27T07:32:46.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.493+0000 7f90c7000700 1 -- 172.21.15.63:0/3101204457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c00776d0 msgr2=0x7f90c0075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:46.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.493+0000 7f90c7000700 1 --2- 172.21.15.63:0/3101204457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c00776d0 0x7f90c0075ba0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f90b40099c0 tx=0x7f90b4009cd0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.493+0000 7f90c7000700 1 -- 172.21.15.63:0/3101204457 shutdown_connections 2024-05-27T07:32:46.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.493+0000 7f90c7000700 1 --2- 172.21.15.63:0/3101204457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c0110790 0x7f90c0112bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.493+0000 7f90c7000700 1 --2- 172.21.15.63:0/3101204457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c00776d0 0x7f90c0075ba0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.493+0000 7f90c7000700 1 --2- 172.21.15.63:0/3101204457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c00760e0 0x7f90c0076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.493+0000 7f90c7000700 1 -- 172.21.15.63:0/3101204457 >> 172.21.15.63:0/3101204457 conn(0x7f90c00ff7a0 msgr2=0x7f90c0101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:46.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.493+0000 7f90c7000700 1 -- 172.21.15.63:0/3101204457 shutdown_connections 2024-05-27T07:32:46.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.493+0000 7f90c7000700 1 -- 172.21.15.63:0/3101204457 wait complete. 2024-05-27T07:32:46.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.494+0000 7f90c7000700 1 Processor -- start 2024-05-27T07:32:46.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.495+0000 7f90c7000700 1 -- start start 2024-05-27T07:32:46.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.495+0000 7f90c7000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c00760e0 0x7f90c01a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:46.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.495+0000 7f90c7000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c00776d0 0x7f90c01a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:46.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.495+0000 7f90c7000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0110790 0x7f90c01a8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:46.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.495+0000 7f90c7000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0116180 con 0x7f90c00776d0 2024-05-27T07:32:46.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.495+0000 7f90c7000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0116000 con 0x7f90c00760e0 2024-05-27T07:32:46.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.495+0000 7f90c7000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90c0116300 con 0x7f90c0110790 2024-05-27T07:32:46.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.496+0000 7f90c5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c00776d0 0x7f90c01a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:46.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.496+0000 7f90c5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c00776d0 0x7f90c01a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47954/0 (socket says 172.21.15.63:47954) 2024-05-27T07:32:46.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.496+0000 7f90c5200700 1 -- 172.21.15.63:0/1411631427 learned_addr learned my addr 172.21.15.63:0/1411631427 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:46.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.496+0000 7f90c5c00700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c00760e0 0x7f90c01a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:46.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.496+0000 7f90c6600700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0110790 0x7f90c01a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:46.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.496+0000 7f90c5200700 1 -- 172.21.15.63:0/1411631427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0110790 msgr2=0x7f90c01a8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:46.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.496+0000 7f90c5200700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0110790 0x7f90c01a8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.496+0000 7f90c5200700 1 -- 172.21.15.63:0/1411631427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c00760e0 msgr2=0x7f90c01a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:46.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.496+0000 7f90c5200700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c00760e0 0x7f90c01a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.496+0000 7f90c5200700 1 -- 172.21.15.63:0/1411631427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c01a88e0 con 0x7f90c00776d0 2024-05-27T07:32:46.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.497+0000 7f90c5200700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c00776d0 0x7f90c01a12c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b530 tx=0x7f90ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:46.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.497+0000 7f90be000700 1 -- 172.21.15.63:0/1411631427 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac015020 con 0x7f90c00776d0 2024-05-27T07:32:46.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.497+0000 7f90c7000700 1 -- 172.21.15.63:0/1411631427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c01a8bd0 con 0x7f90c00776d0 2024-05-27T07:32:46.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.497+0000 7f90be000700 1 -- 172.21.15.63:0/1411631427 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90ac011cc0 con 0x7f90c00776d0 2024-05-27T07:32:46.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.497+0000 7f90be000700 1 -- 172.21.15.63:0/1411631427 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac010dc0 con 0x7f90c00776d0 2024-05-27T07:32:46.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.497+0000 7f90c7000700 1 -- 172.21.15.63:0/1411631427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c01a9110 con 0x7f90c00776d0 2024-05-27T07:32:46.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.499+0000 7f90c7000700 1 -- 172.21.15.63:0/1411631427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c00512b0 con 0x7f90c00776d0 2024-05-27T07:32:46.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.503+0000 7f90be000700 1 -- 172.21.15.63:0/1411631427 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f90ac018020 con 0x7f90c00776d0 2024-05-27T07:32:46.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.504+0000 7f90be000700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f90a80b2880 0x7f90a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:46.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.505+0000 7f90be000700 1 -- 172.21.15.63:0/1411631427 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f90ac0e2440 con 0x7f90c00776d0 2024-05-27T07:32:46.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.505+0000 7f90c5c00700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f90a80b2880 0x7f90a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:46.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.506+0000 7f90c5c00700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f90a80b2880 0x7f90a80b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90b40098e0 tx=0x7f90b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:46.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.508+0000 7f90be000700 1 -- 172.21.15.63:0/1411631427 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f90ac01f020 con 0x7f90c00776d0 2024-05-27T07:32:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.753+0000 7f90c7000700 1 -- 172.21.15.63:0/1411631427 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f90c0063bf0 con 0x7f90a80b2880 2024-05-27T07:32:46.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.757+0000 7f90be000700 1 -- 172.21.15.63:0/1411631427 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f90c0063bf0 con 0x7f90a80b2880 2024-05-27T07:32:46.751 INFO:teuthology.orchestra.run.smithi063.stdout:HOST ADDR LABELS STATUS 2024-05-27T07:32:46.751 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063 172.21.15.63 2024-05-27T07:32:46.751 INFO:teuthology.orchestra.run.smithi063.stdout:smithi086 172.21.15.86 2024-05-27T07:32:46.751 INFO:teuthology.orchestra.run.smithi063.stdout:smithi149 172.21.15.149 2024-05-27T07:32:46.751 INFO:teuthology.orchestra.run.smithi063.stdout:3 hosts in cluster 2024-05-27T07:32:46.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.760+0000 7f90c7000700 1 -- 172.21.15.63:0/1411631427 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f90a80b2880 msgr2=0x7f90a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:46.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.760+0000 7f90c7000700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f90a80b2880 0x7f90a80b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90b40098e0 tx=0x7f90b4005fb0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.760+0000 7f90c7000700 1 -- 172.21.15.63:0/1411631427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c00776d0 msgr2=0x7f90c01a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.760+0000 7f90c7000700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c00776d0 0x7f90c01a12c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b530 tx=0x7f90ac00b840 comp rx=0 tx=0).stop 2024-05-27T07:32:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.760+0000 7f90c7000700 1 -- 172.21.15.63:0/1411631427 shutdown_connections 2024-05-27T07:32:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.760+0000 7f90c7000700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90c0110790 0x7f90c01a8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.760+0000 7f90c7000700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90c00760e0 0x7f90c01a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.760+0000 7f90c7000700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f90a80b2880 0x7f90a80b4d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.760+0000 7f90c7000700 1 --2- 172.21.15.63:0/1411631427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90c00776d0 0x7f90c01a12c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.760+0000 7f90c7000700 1 -- 172.21.15.63:0/1411631427 >> 172.21.15.63:0/1411631427 conn(0x7f90c00ff7a0 msgr2=0x7f90c0103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.761+0000 7f90c7000700 1 -- 172.21.15.63:0/1411631427 shutdown_connections 2024-05-27T07:32:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:46.761+0000 7f90c7000700 1 -- 172.21.15.63:0/1411631427 wait complete. 2024-05-27T07:32:47.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:47 smithi086 ceph-mon[34687]: pgmap v522: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:32:47.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:47 smithi149 ceph-mon[35408]: pgmap v522: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:32:47.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:47 smithi063 ceph-mon[31613]: pgmap v522: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:32:47.631 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-27T07:32:48.107 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:48.451 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:48 smithi063 ceph-mon[31613]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:48.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:48 smithi086 ceph-mon[34687]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:48.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:48 smithi149 ceph-mon[35408]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:49.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:49 smithi149 ceph-mon[35408]: pgmap v523: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:49.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:49 smithi086 ceph-mon[34687]: pgmap v523: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:49.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:49 smithi063 ceph-mon[31613]: pgmap v523: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:50.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.625+0000 7fce42e00700 1 -- 172.21.15.63:0/1017857604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce3c105460 msgr2=0x7fce3c105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:50.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.625+0000 7fce42e00700 1 --2- 172.21.15.63:0/1017857604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce3c105460 0x7fce3c105870 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fce24009990 tx=0x7fce24009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.625+0000 7fce42e00700 1 -- 172.21.15.63:0/1017857604 shutdown_connections 2024-05-27T07:32:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.625+0000 7fce42e00700 1 --2- 172.21.15.63:0/1017857604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c105e00 0x7fce3c10c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.625+0000 7fce42e00700 1 --2- 172.21.15.63:0/1017857604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce3c104330 0x7fce3c104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.625+0000 7fce42e00700 1 --2- 172.21.15.63:0/1017857604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce3c105460 0x7fce3c105870 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.625+0000 7fce42e00700 1 -- 172.21.15.63:0/1017857604 >> 172.21.15.63:0/1017857604 conn(0x7fce3c0ff9b0 msgr2=0x7fce3c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.625+0000 7fce42e00700 1 -- 172.21.15.63:0/1017857604 shutdown_connections 2024-05-27T07:32:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.625+0000 7fce42e00700 1 -- 172.21.15.63:0/1017857604 wait complete. 2024-05-27T07:32:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.626+0000 7fce42e00700 1 Processor -- start 2024-05-27T07:32:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.626+0000 7fce42e00700 1 -- start start 2024-05-27T07:32:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce42e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce3c104330 0x7fce3c1a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce42e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c105460 0x7fce3c1a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce42e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce3c105e00 0x7fce3c1a8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce42e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c10fd10 con 0x7fce3c105e00 2024-05-27T07:32:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce42e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c10fb90 con 0x7fce3c105460 2024-05-27T07:32:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce42e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c10fe90 con 0x7fce3c104330 2024-05-27T07:32:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce40e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce3c105e00 0x7fce3c1a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce40e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce3c105e00 0x7fce3c1a8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37700/0 (socket says 172.21.15.63:37700) 2024-05-27T07:32:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce40e00700 1 -- 172.21.15.63:0/3285322041 learned_addr learned my addr 172.21.15.63:0/3285322041 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce3be00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce3c104330 0x7fce3c1a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.627+0000 7fce33400700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c105460 0x7fce3c1a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.628+0000 7fce40e00700 1 -- 172.21.15.63:0/3285322041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce3c104330 msgr2=0x7fce3c1a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:50.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.628+0000 7fce40e00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce3c104330 0x7fce3c1a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.628+0000 7fce40e00700 1 -- 172.21.15.63:0/3285322041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c105460 msgr2=0x7fce3c1a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:50.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.628+0000 7fce40e00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c105460 0x7fce3c1a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.628+0000 7fce40e00700 1 -- 172.21.15.63:0/3285322041 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c1a8a90 con 0x7fce3c105e00 2024-05-27T07:32:50.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.628+0000 7fce33400700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c105460 0x7fce3c1a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:50.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.628+0000 7fce3be00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce3c104330 0x7fce3c1a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:50.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.628+0000 7fce40e00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce3c105e00 0x7fce3c1a8410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fce34007ee0 tx=0x7fce3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:50.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.629+0000 7fce39600700 1 -- 172.21.15.63:0/3285322041 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3400dbd0 con 0x7fce3c105e00 2024-05-27T07:32:50.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.629+0000 7fce39600700 1 -- 172.21.15.63:0/3285322041 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce3400dd70 con 0x7fce3c105e00 2024-05-27T07:32:50.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.629+0000 7fce42e00700 1 -- 172.21.15.63:0/3285322041 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3c1a8d80 con 0x7fce3c105e00 2024-05-27T07:32:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.629+0000 7fce39600700 1 -- 172.21.15.63:0/3285322041 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce340058d0 con 0x7fce3c105e00 2024-05-27T07:32:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.629+0000 7fce42e00700 1 -- 172.21.15.63:0/3285322041 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3c1a9230 con 0x7fce3c105e00 2024-05-27T07:32:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.630+0000 7fce42e00700 1 -- 172.21.15.63:0/3285322041 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3c0512b0 con 0x7fce3c105e00 2024-05-27T07:32:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.632+0000 7fce39600700 1 -- 172.21.15.63:0/3285322041 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fce3401a020 con 0x7fce3c105e00 2024-05-27T07:32:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.634+0000 7fce39600700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce200b2880 0x7fce200b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.634+0000 7fce39600700 1 -- 172.21.15.63:0/3285322041 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fce340e1e60 con 0x7fce3c105e00 2024-05-27T07:32:50.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.634+0000 7fce3be00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce200b2880 0x7fce200b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:50.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.635+0000 7fce3be00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce200b2880 0x7fce200b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce3c105360 tx=0x7fce2c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:50.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.639+0000 7fce39600700 1 -- 172.21.15.63:0/3285322041 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce34010070 con 0x7fce3c105e00 2024-05-27T07:32:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.886+0000 7fce42e00700 1 -- 172.21.15.63:0/3285322041 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fce3c063bf0 con 0x7fce200b2880 2024-05-27T07:32:50.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.890+0000 7fce39600700 1 -- 172.21.15.63:0/3285322041 <== mgr.14152 v2:172.21.15.63:6800/675305989 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fce3c063bf0 con 0x7fce200b2880 2024-05-27T07:32:50.884 INFO:teuthology.orchestra.run.smithi063.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-27T07:32:50.885 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5344005B400BGN 372G No 7m ago Has a FileSystem, LVM detected 2024-05-27T07:32:50.885 INFO:teuthology.orchestra.run.smithi063.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-27T07:32:50.885 INFO:teuthology.orchestra.run.smithi063.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-27T07:32:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.893+0000 7fce42e00700 1 -- 172.21.15.63:0/3285322041 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce200b2880 msgr2=0x7fce200b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:50.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce200b2880 0x7fce200b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce3c105360 tx=0x7fce2c008040 comp rx=0 tx=0).stop 2024-05-27T07:32:50.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 -- 172.21.15.63:0/3285322041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce3c105e00 msgr2=0x7fce3c1a8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:50.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce3c105e00 0x7fce3c1a8410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fce34007ee0 tx=0x7fce3400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 -- 172.21.15.63:0/3285322041 shutdown_connections 2024-05-27T07:32:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fce200b2880 0x7fce200b4d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce3c105e00 0x7fce3c1a8410 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce3c104330 0x7fce3c1a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 --2- 172.21.15.63:0/3285322041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c105460 0x7fce3c1a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 -- 172.21.15.63:0/3285322041 >> 172.21.15.63:0/3285322041 conn(0x7fce3c0ff9b0 msgr2=0x7fce3c106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 -- 172.21.15.63:0/3285322041 shutdown_connections 2024-05-27T07:32:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:50.894+0000 7fce42e00700 1 -- 172.21.15.63:0/3285322041 wait complete. 2024-05-27T07:32:51.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:51 smithi063 ceph-mon[31613]: pgmap v524: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:51.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:51 smithi086 ceph-mon[34687]: pgmap v524: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:51.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:51 smithi149 ceph-mon[35408]: pgmap v524: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:32:51.898 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T07:32:51.908 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T07:32:51.908 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-27T07:32:52.395 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:52.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:52 smithi086 ceph-mon[34687]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:52.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:52 smithi086 ceph-mon[34687]: pgmap v525: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:52.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:52 smithi149 ceph-mon[35408]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:52.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:52 smithi149 ceph-mon[35408]: pgmap v525: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:52.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:52 smithi063 ceph-mon[31613]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:52.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:52 smithi063 ceph-mon[31613]: pgmap v525: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:32:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.719+0000 7fc89ca00700 1 -- 172.21.15.63:0/2712916229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8980776d0 msgr2=0x7fc898075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.719+0000 7fc89ca00700 1 --2- 172.21.15.63:0/2712916229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8980776d0 0x7fc898075ba0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc8880099c0 tx=0x7fc888009cd0 comp rx=0 tx=0).stop 2024-05-27T07:32:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.719+0000 7fc89ca00700 1 -- 172.21.15.63:0/2712916229 shutdown_connections 2024-05-27T07:32:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.719+0000 7fc89ca00700 1 --2- 172.21.15.63:0/2712916229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc898110790 0x7fc898112bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.719+0000 7fc89ca00700 1 --2- 172.21.15.63:0/2712916229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8980776d0 0x7fc898075ba0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.719+0000 7fc89ca00700 1 --2- 172.21.15.63:0/2712916229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8980760e0 0x7fc898076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:54.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.719+0000 7fc89ca00700 1 -- 172.21.15.63:0/2712916229 >> 172.21.15.63:0/2712916229 conn(0x7fc8980ff7a0 msgr2=0x7fc898101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:54.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.719+0000 7fc89ca00700 1 -- 172.21.15.63:0/2712916229 shutdown_connections 2024-05-27T07:32:54.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.719+0000 7fc89ca00700 1 -- 172.21.15.63:0/2712916229 wait complete. 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.720+0000 7fc89ca00700 1 Processor -- start 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.721+0000 7fc89ca00700 1 -- start start 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.721+0000 7fc89ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8980760e0 0x7fc8981a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.721+0000 7fc89ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8980776d0 0x7fc8981a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.721+0000 7fc89ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc898110790 0x7fc8981a8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.721+0000 7fc89ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc898116080 con 0x7fc8980760e0 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.721+0000 7fc89ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc898115f00 con 0x7fc898110790 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.721+0000 7fc89ca00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc898116200 con 0x7fc8980776d0 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.721+0000 7fc897400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8980760e0 0x7fc8981a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.721+0000 7fc897400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8980760e0 0x7fc8981a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37714/0 (socket says 172.21.15.63:37714) 2024-05-27T07:32:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.721+0000 7fc897400700 1 -- 172.21.15.63:0/4225388210 learned_addr learned my addr 172.21.15.63:0/4225388210 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.722+0000 7fc896a00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8980776d0 0x7fc8981a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.722+0000 7fc897e00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc898110790 0x7fc8981a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.722+0000 7fc897400700 1 -- 172.21.15.63:0/4225388210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8980776d0 msgr2=0x7fc8981a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.722+0000 7fc897400700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8980776d0 0x7fc8981a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.722+0000 7fc897400700 1 -- 172.21.15.63:0/4225388210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc898110790 msgr2=0x7fc8981a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.722+0000 7fc897400700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc898110790 0x7fc8981a8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.722+0000 7fc897400700 1 -- 172.21.15.63:0/4225388210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8981a8850 con 0x7fc8980760e0 2024-05-27T07:32:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.722+0000 7fc897e00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc898110790 0x7fc8981a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.723+0000 7fc897400700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8980760e0 0x7fc8981a0cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc88800b580 tx=0x7fc88800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:54.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.723+0000 7fc88fe00700 1 -- 172.21.15.63:0/4225388210 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888027070 con 0x7fc8980760e0 2024-05-27T07:32:54.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.723+0000 7fc89ca00700 1 -- 172.21.15.63:0/4225388210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8981a8ae0 con 0x7fc8980760e0 2024-05-27T07:32:54.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.723+0000 7fc88fe00700 1 -- 172.21.15.63:0/4225388210 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc88800bd10 con 0x7fc8980760e0 2024-05-27T07:32:54.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.723+0000 7fc88fe00700 1 -- 172.21.15.63:0/4225388210 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888022d70 con 0x7fc8980760e0 2024-05-27T07:32:54.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.724+0000 7fc89ca00700 1 -- 172.21.15.63:0/4225388210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8981a8f40 con 0x7fc8980760e0 2024-05-27T07:32:54.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.725+0000 7fc89ca00700 1 -- 172.21.15.63:0/4225388210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8980512b0 con 0x7fc8980760e0 2024-05-27T07:32:54.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.728+0000 7fc88fe00700 1 -- 172.21.15.63:0/4225388210 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc88802a070 con 0x7fc8980760e0 2024-05-27T07:32:54.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.729+0000 7fc88fe00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc87c0b2880 0x7fc87c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.730+0000 7fc88fe00700 1 -- 172.21.15.63:0/4225388210 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fc8880e3cc0 con 0x7fc8980760e0 2024-05-27T07:32:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.730+0000 7fc896a00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc87c0b2880 0x7fc87c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:54.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.731+0000 7fc896a00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc87c0b2880 0x7fc87c0b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc8981a1d60 tx=0x7fc88400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:54.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:54.731+0000 7fc88fe00700 1 -- 172.21.15.63:0/4225388210 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc888013070 con 0x7fc8980760e0 2024-05-27T07:32:55.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.016+0000 7fc89ca00700 1 -- 172.21.15.63:0/4225388210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc89802e5e0 con 0x7fc8980760e0 2024-05-27T07:32:55.011 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:54 smithi063 ceph-mon[31613]: pgmap v526: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:55.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.017+0000 7fc88fe00700 1 -- 172.21.15.63:0/4225388210 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3044 (secure 0 0 0) 0x7fc8880ab680 con 0x7fc8980760e0 2024-05-27T07:32:55.013 INFO:teuthology.orchestra.run.smithi063.stdout:e26 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 26 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-27T07:28:27.557349+0000 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-27T07:29:56.944716+0000 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-05-27T07:32:55.014 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 300 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 0 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 3 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1,2 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=34520,1=34484,2=34454} 2024-05-27T07:32:55.015 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-05-27T07:32:55.016 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-05-27T07:32:55.016 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-05-27T07:32:55.016 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3,4] 2024-05-27T07:32:55.016 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-05-27T07:32:55.016 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-05-27T07:32:55.016 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-05-27T07:32:55.016 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-05-27T07:32:55.016 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 0 2024-05-27T07:32:55.016 INFO:teuthology.orchestra.run.smithi063.stdout:qdb_cluster leader: 34520 members: 34484,34520,34454 2024-05-27T07:32:55.016 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b{0:34520} state up:active seq 7 addr [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.k{0:34430} state up:standby-replay seq 1 addr [v2:172.21.15.86:6832/1769339501,v1:172.21.15.86:6833/1769339501] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.h{1:34484} state up:active seq 32 addr [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.i{1:24505} state up:standby-replay seq 1 addr [v2:172.21.15.149:6838/1574913181,v1:172.21.15.149:6839/1574913181] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.e{2:34454} state up:active seq 35 addr [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c{2:24484} state up:standby-replay seq 1 addr [v2:172.21.15.149:6836/3113374312,v1:172.21.15.149:6837/3113374312] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.j{-1:14517} state up:standby seq 1 addr [v2:172.21.15.63:6834/134276953,v1:172.21.15.63:6835/134276953] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d{-1:14523} state up:standby seq 1 addr [v2:172.21.15.63:6836/2286910897,v1:172.21.15.63:6837/2286910897] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.g{-1:14577} state up:standby seq 1 addr [v2:172.21.15.63:6838/3288686403,v1:172.21.15.63:6839/3288686403] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a{-1:14589} state up:standby seq 1 addr [v2:172.21.15.63:6840/80080490,v1:172.21.15.63:6841/80080490] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.l{-1:24436} state up:standby seq 1 addr [v2:172.21.15.149:6832/1182053542,v1:172.21.15.149:6833/1182053542] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.018 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.f{-1:24478} state up:standby seq 1 addr [v2:172.21.15.149:6834/2496628912,v1:172.21.15.149:6835/2496628912] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:32:55.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.021+0000 7fc89ca00700 1 -- 172.21.15.63:0/4225388210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc87c0b2880 msgr2=0x7fc87c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:55.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.021+0000 7fc89ca00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc87c0b2880 0x7fc87c0b4d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc8981a1d60 tx=0x7fc88400b410 comp rx=0 tx=0).stop 2024-05-27T07:32:55.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.021+0000 7fc89ca00700 1 -- 172.21.15.63:0/4225388210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8980760e0 msgr2=0x7fc8981a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:55.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.021+0000 7fc89ca00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8980760e0 0x7fc8981a0cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc88800b580 tx=0x7fc88800f690 comp rx=0 tx=0).stop 2024-05-27T07:32:55.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.022+0000 7fc89ca00700 1 -- 172.21.15.63:0/4225388210 shutdown_connections 2024-05-27T07:32:55.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.022+0000 7fc89ca00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc898110790 0x7fc8981a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:55.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.022+0000 7fc89ca00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc87c0b2880 0x7fc87c0b4d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:55.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.022+0000 7fc89ca00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8980760e0 0x7fc8981a0cb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.022+0000 7fc89ca00700 1 --2- 172.21.15.63:0/4225388210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8980776d0 0x7fc8981a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.022+0000 7fc89ca00700 1 -- 172.21.15.63:0/4225388210 >> 172.21.15.63:0/4225388210 conn(0x7fc8980ff7a0 msgr2=0x7fc898103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.022+0000 7fc89ca00700 1 -- 172.21.15.63:0/4225388210 shutdown_connections 2024-05-27T07:32:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:55.022+0000 7fc89ca00700 1 -- 172.21.15.63:0/4225388210 wait complete. 2024-05-27T07:32:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 26 2024-05-27T07:32:55.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:54 smithi086 ceph-mon[34687]: pgmap v526: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:55.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:54 smithi149 ceph-mon[35408]: pgmap v526: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:32:56.019 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-27T07:32:56.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:55 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4225388210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T07:32:56.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:55 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4225388210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T07:32:56.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:55 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4225388210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T07:32:56.486 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:57.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:57 smithi063 ceph-mon[31613]: pgmap v527: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:57.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:57 smithi086 ceph-mon[34687]: pgmap v527: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:57.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:57 smithi149 ceph-mon[35408]: pgmap v527: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:32:58.400 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:58 smithi063 ceph-mon[31613]: pgmap v528: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:58.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:58 smithi086 ceph-mon[34687]: pgmap v528: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:58.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:58 smithi149 ceph-mon[35408]: pgmap v528: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:32:58.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.821+0000 7f8438200700 1 -- 172.21.15.63:0/1227513312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8430107790 msgr2=0x7f8430109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:58.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.821+0000 7f8438200700 1 --2- 172.21.15.63:0/1227513312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8430107790 0x7f8430109bb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f841c009990 tx=0x7f841c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:58.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.822+0000 7f8438200700 1 -- 172.21.15.63:0/1227513312 shutdown_connections 2024-05-27T07:32:58.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.822+0000 7f8438200700 1 --2- 172.21.15.63:0/1227513312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f843010a0f0 0x7f843010c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:58.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.822+0000 7f8438200700 1 --2- 172.21.15.63:0/1227513312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430104e30 0x7f8430107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:58.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.822+0000 7f8438200700 1 --2- 172.21.15.63:0/1227513312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8430107790 0x7f8430109bb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:58.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.822+0000 7f8438200700 1 -- 172.21.15.63:0/1227513312 >> 172.21.15.63:0/1227513312 conn(0x7f84300fe870 msgr2=0x7f8430100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:58.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.822+0000 7f8438200700 1 -- 172.21.15.63:0/1227513312 shutdown_connections 2024-05-27T07:32:58.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.822+0000 7f8438200700 1 -- 172.21.15.63:0/1227513312 wait complete. 2024-05-27T07:32:58.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.823+0000 7f8438200700 1 Processor -- start 2024-05-27T07:32:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.823+0000 7f8438200700 1 -- start start 2024-05-27T07:32:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.823+0000 7f8438200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430104e30 0x7f843019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8438200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8430107790 0x7f843019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8438200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f843010a0f0 0x7f84301a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8438200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f843010fd00 con 0x7f8430107790 2024-05-27T07:32:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8438200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f843010fb80 con 0x7f843010a0f0 2024-05-27T07:32:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8438200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f843010fe80 con 0x7f8430104e30 2024-05-27T07:32:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8434e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8430107790 0x7f843019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8434e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8430107790 0x7f843019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37728/0 (socket says 172.21.15.63:37728) 2024-05-27T07:32:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8434e00700 1 -- 172.21.15.63:0/1743766277 learned_addr learned my addr 172.21.15.63:0/1743766277 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:32:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8435800700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430104e30 0x7f843019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8436200700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f843010a0f0 0x7f84301a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8434e00700 1 -- 172.21.15.63:0/1743766277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430104e30 msgr2=0x7f843019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.824+0000 7f8434e00700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430104e30 0x7f843019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.825+0000 7f8434e00700 1 -- 172.21.15.63:0/1743766277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f843010a0f0 msgr2=0x7f84301a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.825+0000 7f8434e00700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f843010a0f0 0x7f84301a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.825+0000 7f8434e00700 1 -- 172.21.15.63:0/1743766277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84301a4640 con 0x7f8430107790 2024-05-27T07:32:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.825+0000 7f8436200700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f843010a0f0 0x7f84301a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:32:58.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.825+0000 7f8434e00700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8430107790 0x7f843019d020 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:58.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.825+0000 7f842e000700 1 -- 172.21.15.63:0/1743766277 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c027070 con 0x7f8430107790 2024-05-27T07:32:58.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.825+0000 7f842e000700 1 -- 172.21.15.63:0/1743766277 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f841c0040a0 con 0x7f8430107790 2024-05-27T07:32:58.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.825+0000 7f842e000700 1 -- 172.21.15.63:0/1743766277 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c022d50 con 0x7f8430107790 2024-05-27T07:32:58.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.825+0000 7f8438200700 1 -- 172.21.15.63:0/1743766277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84301a48d0 con 0x7f8430107790 2024-05-27T07:32:58.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.826+0000 7f8438200700 1 -- 172.21.15.63:0/1743766277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84301a4d50 con 0x7f8430107790 2024-05-27T07:32:58.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.827+0000 7f8438200700 1 -- 172.21.15.63:0/1743766277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84300512b0 con 0x7f8430107790 2024-05-27T07:32:58.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.831+0000 7f842e000700 1 -- 172.21.15.63:0/1743766277 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f841c02a070 con 0x7f8430107790 2024-05-27T07:32:58.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.832+0000 7f842e000700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f84180b2900 0x7f84180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:58.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.833+0000 7f842e000700 1 -- 172.21.15.63:0/1743766277 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f841c0e3cb0 con 0x7f8430107790 2024-05-27T07:32:58.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.833+0000 7f8435800700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f84180b2900 0x7f84180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:58.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.834+0000 7f8435800700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f84180b2900 0x7f84180b4e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8424006fd0 tx=0x7f8424008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:58.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:58.835+0000 7f842e000700 1 -- 172.21.15.63:0/1743766277 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f841c031080 con 0x7f8430107790 2024-05-27T07:32:59.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.079+0000 7f8438200700 1 -- 172.21.15.63:0/1743766277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f84301a51d0 con 0x7f8430107790 2024-05-27T07:32:59.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.081+0000 7f842e000700 1 -- 172.21.15.63:0/1743766277 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5621 (secure 0 0 0) 0x7f841c013070 con 0x7f8430107790 2024-05-27T07:32:59.076 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 91 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-27T07:16:52.281756+0000 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-05-27T07:30:05.109968+0000 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:crush_version 27 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:full_ratio 0.95 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:backfillfull_ratio 0.9 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:nearfull_ratio 0.85 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:require_min_compat_client luminous 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:min_compat_client luminous 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:require_osd_release squid 2024-05-27T07:32:59.077 INFO:teuthology.orchestra.run.smithi063.stdout:stretch_mode_enabled false 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:max_osd 12 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6802/1698060699,v1:172.21.15.63:6803/1698060699] [v2:172.21.15.63:6804/1698060699,v1:172.21.15.63:6805/1698060699] exists,up 032421af-e923-4b4a-beb0-2d42713538bf 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6800/1435942303,v1:172.21.15.86:6801/1435942303] [v2:172.21.15.86:6802/1435942303,v1:172.21.15.86:6803/1435942303] exists,up f57b3d89-2b35-4381-99a6-7db08bef64a4 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/363230187,v1:172.21.15.149:6801/363230187] [v2:172.21.15.149:6802/363230187,v1:172.21.15.149:6803/363230187] exists,up 025e966c-8833-4c12-a136-64d91cb0f4a4 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 up in weight 1 up_from 26 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6810/3416673415,v1:172.21.15.63:6811/3416673415] [v2:172.21.15.63:6812/3416673415,v1:172.21.15.63:6813/3416673415] exists,up e39fff31-e8c9-47f1-9029-e8763af5abe5 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 up in weight 1 up_from 32 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6808/3874353435,v1:172.21.15.86:6809/3874353435] [v2:172.21.15.86:6810/3874353435,v1:172.21.15.86:6811/3874353435] exists,up b8b86405-4ec3-4853-9429-85a69f9a5bc9 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/3863528283,v1:172.21.15.149:6809/3863528283] [v2:172.21.15.149:6810/3863528283,v1:172.21.15.149:6811/3863528283] exists,up ff8d5a42-64bb-40f2-83f4-e22767f857f0 2024-05-27T07:32:59.078 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6 up in weight 1 up_from 44 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6818/3628147489,v1:172.21.15.63:6819/3628147489] [v2:172.21.15.63:6820/3628147489,v1:172.21.15.63:6821/3628147489] exists,up f496ab56-b7a8-47c0-bd34-fe6f2f82f69d 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7 up in weight 1 up_from 49 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6816/146162920,v1:172.21.15.86:6817/146162920] [v2:172.21.15.86:6818/146162920,v1:172.21.15.86:6819/146162920] exists,up 76b47ee5-4a59-4a08-9d10-f8cb104236d5 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:osd.8 up in weight 1 up_from 55 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/2466164082,v1:172.21.15.149:6817/2466164082] [v2:172.21.15.149:6818/2466164082,v1:172.21.15.149:6819/2466164082] exists,up 63112f81-efa0-4622-8eac-311302d0bca6 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:osd.9 up in weight 1 up_from 61 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6826/269558995,v1:172.21.15.63:6827/269558995] [v2:172.21.15.63:6828/269558995,v1:172.21.15.63:6829/269558995] exists,up ee506119-9cb2-4271-84da-ddcc6ca6cf09 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6824/1317761853,v1:172.21.15.86:6825/1317761853] [v2:172.21.15.86:6826/1317761853,v1:172.21.15.86:6827/1317761853] exists,up 861ae487-365b-4de2-91e9-1d8254d36aaf 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:osd.11 up in weight 1 up_from 72 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/4045663443,v1:172.21.15.149:6825/4045663443] [v2:172.21.15.149:6826/4045663443,v1:172.21.15.149:6827/4045663443] exists,up 69396a1e-76b9-40c2-a289-392b3f2a5dc3 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 2.6 [8,11] 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 2.8 [2,11] 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 2.16 [2,4] 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 3.1b [4,6,3,7,5,9] 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 3.20 [5,9] 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 3.31 [4,8] 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 3.34 [4,9] 2024-05-27T07:32:59.079 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 4.13 [9,10] 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 4.16 [9,8] 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 4.1b [9,8] 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:pg_upmap_items 4.1d [5,8] 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:6801/1689640179 expires 2024-05-28T07:18:03.559305+0000 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/3946208673 expires 2024-05-28T07:18:03.559305+0000 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/736953168 expires 2024-05-28T07:18:03.559305+0000 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:6801/3806249841 expires 2024-05-28T07:17:27.459562+0000 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:6800/1689640179 expires 2024-05-28T07:18:03.559305+0000 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/4210215351 expires 2024-05-28T07:17:27.459562+0000 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:6800/3806249841 expires 2024-05-28T07:17:27.459562+0000 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/2613878696 expires 2024-05-28T07:17:27.459562+0000 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/1318703057 expires 2024-05-28T07:17:27.459562+0000 2024-05-27T07:32:59.080 INFO:teuthology.orchestra.run.smithi063.stdout:blocklist 172.21.15.63:0/58174792 expires 2024-05-28T07:18:03.559305+0000 2024-05-27T07:32:59.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.086+0000 7f8438200700 1 -- 172.21.15.63:0/1743766277 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f84180b2900 msgr2=0x7f84180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:59.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.086+0000 7f8438200700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f84180b2900 0x7f84180b4e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8424006fd0 tx=0x7f8424008040 comp rx=0 tx=0).stop 2024-05-27T07:32:59.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.086+0000 7f8438200700 1 -- 172.21.15.63:0/1743766277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8430107790 msgr2=0x7f843019d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:59.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.086+0000 7f8438200700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8430107790 0x7f843019d020 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c009580 comp rx=0 tx=0).stop 2024-05-27T07:32:59.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.086+0000 7f8438200700 1 -- 172.21.15.63:0/1743766277 shutdown_connections 2024-05-27T07:32:59.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.086+0000 7f8438200700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f843010a0f0 0x7f84301a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:59.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.086+0000 7f8438200700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430104e30 0x7f843019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:59.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.086+0000 7f8438200700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f84180b2900 0x7f84180b4e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:59.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.086+0000 7f8438200700 1 --2- 172.21.15.63:0/1743766277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8430107790 0x7f843019d020 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:59.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.086+0000 7f8438200700 1 -- 172.21.15.63:0/1743766277 >> 172.21.15.63:0/1743766277 conn(0x7f84300fe870 msgr2=0x7f8430102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:59.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.087+0000 7f8438200700 1 -- 172.21.15.63:0/1743766277 shutdown_connections 2024-05-27T07:32:59.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:32:59.087+0000 7f8438200700 1 -- 172.21.15.63:0/1743766277 wait complete. 2024-05-27T07:32:59.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:32:59 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1743766277' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T07:32:59.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:32:59 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1743766277' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T07:32:59.458 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:32:59 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1743766277' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T07:32:59.975 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-27T07:32:59.993 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:33:00.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:00 smithi063 ceph-mon[31613]: pgmap v529: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:00.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:00 smithi086 ceph-mon[34687]: pgmap v529: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:00.467 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:00 smithi149 ceph-mon[35408]: pgmap v529: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:00.483 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:02.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.817+0000 7fc997000700 1 -- 172.21.15.63:0/3187881645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990109860 msgr2=0x7fc9901111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:02.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.817+0000 7fc997000700 1 --2- 172.21.15.63:0/3187881645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990109860 0x7fc9901111c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc98800b0d0 tx=0x7fc98800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:33:02.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.817+0000 7fc997000700 1 -- 172.21.15.63:0/3187881645 shutdown_connections 2024-05-27T07:33:02.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.817+0000 7fc997000700 1 --2- 172.21.15.63:0/3187881645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990109860 0x7fc9901111c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:02.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.817+0000 7fc997000700 1 --2- 172.21.15.63:0/3187881645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9901045e0 0x7fc9901049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:02.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.817+0000 7fc997000700 1 --2- 172.21.15.63:0/3187881645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc990108f10 0x7fc990109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:02.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.817+0000 7fc997000700 1 -- 172.21.15.63:0/3187881645 >> 172.21.15.63:0/3187881645 conn(0x7fc990100030 msgr2=0x7fc990102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:02.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.817+0000 7fc997000700 1 -- 172.21.15.63:0/3187881645 shutdown_connections 2024-05-27T07:33:02.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.818+0000 7fc997000700 1 -- 172.21.15.63:0/3187881645 wait complete. 2024-05-27T07:33:02.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.819+0000 7fc997000700 1 Processor -- start 2024-05-27T07:33:02.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.819+0000 7fc997000700 1 -- start start 2024-05-27T07:33:02.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.819+0000 7fc997000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9901045e0 0x7fc99019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:02.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.819+0000 7fc997000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc990108f10 0x7fc99019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:02.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.819+0000 7fc997000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990109860 0x7fc9901a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:02.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.819+0000 7fc997000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc990114340 con 0x7fc990109860 2024-05-27T07:33:02.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.819+0000 7fc997000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901141c0 con 0x7fc990108f10 2024-05-27T07:33:02.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.819+0000 7fc997000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901144c0 con 0x7fc9901045e0 2024-05-27T07:33:02.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.820+0000 7fc995000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990109860 0x7fc9901a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:02.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.820+0000 7fc995000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990109860 0x7fc9901a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48590/0 (socket says 172.21.15.63:48590) 2024-05-27T07:33:02.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.820+0000 7fc995000700 1 -- 172.21.15.63:0/2021799296 learned_addr learned my addr 172.21.15.63:0/2021799296 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:02.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.820+0000 7fc98fe00700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9901045e0 0x7fc99019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:02.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.820+0000 7fc98f400700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc990108f10 0x7fc99019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.820+0000 7fc995000700 1 -- 172.21.15.63:0/2021799296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9901045e0 msgr2=0x7fc99019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.820+0000 7fc995000700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9901045e0 0x7fc99019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.820+0000 7fc995000700 1 -- 172.21.15.63:0/2021799296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc990108f10 msgr2=0x7fc99019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.820+0000 7fc995000700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc990108f10 0x7fc99019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.820+0000 7fc995000700 1 -- 172.21.15.63:0/2021799296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9901a1d30 con 0x7fc990109860 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.821+0000 7fc995000700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990109860 0x7fc9901a1740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc988009850 tx=0x7fc988007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.821+0000 7fc98cc00700 1 -- 172.21.15.63:0/2021799296 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98801f030 con 0x7fc990109860 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.821+0000 7fc98cc00700 1 -- 172.21.15.63:0/2021799296 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc98801bad0 con 0x7fc990109860 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.821+0000 7fc98cc00700 1 -- 172.21.15.63:0/2021799296 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988012bb0 con 0x7fc990109860 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.821+0000 7fc997000700 1 -- 172.21.15.63:0/2021799296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9901a1fc0 con 0x7fc990109860 2024-05-27T07:33:02.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.821+0000 7fc997000700 1 -- 172.21.15.63:0/2021799296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a24b0 con 0x7fc990109860 2024-05-27T07:33:02.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.823+0000 7fc997000700 1 -- 172.21.15.63:0/2021799296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9900512b0 con 0x7fc990109860 2024-05-27T07:33:02.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.825+0000 7fc98cc00700 1 -- 172.21.15.63:0/2021799296 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc988022030 con 0x7fc990109860 2024-05-27T07:33:02.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.827+0000 7fc98cc00700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc9740b2880 0x7fc9740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:02.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.827+0000 7fc98cc00700 1 -- 172.21.15.63:0/2021799296 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fc98800f020 con 0x7fc990109860 2024-05-27T07:33:02.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.827+0000 7fc98fe00700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc9740b2880 0x7fc9740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:02.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.828+0000 7fc98fe00700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc9740b2880 0x7fc9740b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc980005ea0 tx=0x7fc980005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:02.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:02.832+0000 7fc98cc00700 1 -- 172.21.15.63:0/2021799296 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc988029020 con 0x7fc990109860 2024-05-27T07:33:03.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.122+0000 7fc997000700 1 -- 172.21.15.63:0/2021799296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9901a99f0 con 0x7fc990109860 2024-05-27T07:33:03.117 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:02 smithi063 ceph-mon[31613]: pgmap v530: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:33:03.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.123+0000 7fc98cc00700 1 -- 172.21.15.63:0/2021799296 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11861 (secure 0 0 0) 0x7fc9880ab270 con 0x7fc990109860 2024-05-27T07:33:03.118 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:33:03.118 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:29:56.944716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T07:33:03.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.126+0000 7fc997000700 1 -- 172.21.15.63:0/2021799296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc9740b2880 msgr2=0x7fc9740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:03.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.126+0000 7fc997000700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc9740b2880 0x7fc9740b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc980005ea0 tx=0x7fc980005e30 comp rx=0 tx=0).stop 2024-05-27T07:33:03.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.126+0000 7fc997000700 1 -- 172.21.15.63:0/2021799296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990109860 msgr2=0x7fc9901a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:03.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.127+0000 7fc997000700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990109860 0x7fc9901a1740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc988009850 tx=0x7fc988007e40 comp rx=0 tx=0).stop 2024-05-27T07:33:03.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.127+0000 7fc997000700 1 -- 172.21.15.63:0/2021799296 shutdown_connections 2024-05-27T07:33:03.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.127+0000 7fc997000700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc9740b2880 0x7fc9740b4d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:03.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.127+0000 7fc997000700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc990109860 0x7fc9901a1740 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:03.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.127+0000 7fc997000700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9901045e0 0x7fc99019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:03.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.127+0000 7fc997000700 1 --2- 172.21.15.63:0/2021799296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc990108f10 0x7fc99019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:03.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.127+0000 7fc997000700 1 -- 172.21.15.63:0/2021799296 >> 172.21.15.63:0/2021799296 conn(0x7fc990100030 msgr2=0x7fc990110e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:03.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.127+0000 7fc997000700 1 -- 172.21.15.63:0/2021799296 shutdown_connections 2024-05-27T07:33:03.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:03.127+0000 7fc997000700 1 -- 172.21.15.63:0/2021799296 wait complete. 2024-05-27T07:33:03.123 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 26 2024-05-27T07:33:03.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:02 smithi086 ceph-mon[34687]: pgmap v530: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:33:03.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:02 smithi149 ceph-mon[35408]: pgmap v530: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:33:04.029 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:33:04.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:03 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2021799296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:04.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:03 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2021799296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:04.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:03 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2021799296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:04.481 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:05.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:04 smithi086 ceph-mon[34687]: pgmap v531: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:33:05.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:04 smithi149 ceph-mon[35408]: pgmap v531: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:33:05.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:04 smithi063 ceph-mon[31613]: pgmap v531: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:33:06.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.888+0000 7f471e600700 1 -- 172.21.15.63:0/77722863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4718108ef0 msgr2=0x7f4718109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:06.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.888+0000 7f471e600700 1 --2- 172.21.15.63:0/77722863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4718108ef0 0x7f4718109300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47040099c0 tx=0x7f4704009cd0 comp rx=0 tx=0).stop 2024-05-27T07:33:06.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.888+0000 7f471e600700 1 -- 172.21.15.63:0/77722863 shutdown_connections 2024-05-27T07:33:06.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.888+0000 7f471e600700 1 --2- 172.21.15.63:0/77722863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718109840 0x7f47181111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:06.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.888+0000 7f471e600700 1 --2- 172.21.15.63:0/77722863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181045c0 0x7f47181049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:06.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.888+0000 7f471e600700 1 --2- 172.21.15.63:0/77722863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4718108ef0 0x7f4718109300 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:06.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.888+0000 7f471e600700 1 -- 172.21.15.63:0/77722863 >> 172.21.15.63:0/77722863 conn(0x7f47180ffe90 msgr2=0x7f47181022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:06.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.888+0000 7f471e600700 1 -- 172.21.15.63:0/77722863 shutdown_connections 2024-05-27T07:33:06.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.888+0000 7f471e600700 1 -- 172.21.15.63:0/77722863 wait complete. 2024-05-27T07:33:06.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.889+0000 7f471e600700 1 Processor -- start 2024-05-27T07:33:06.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.890+0000 7f471e600700 1 -- start start 2024-05-27T07:33:06.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.890+0000 7f471e600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181045c0 0x7f471819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:06.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.890+0000 7f471e600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4718108ef0 0x7f471819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:06.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.890+0000 7f471e600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718109840 0x7f47181a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:06.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.890+0000 7f471e600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4718114350 con 0x7f4718108ef0 2024-05-27T07:33:06.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.890+0000 7f471e600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47181141d0 con 0x7f47181045c0 2024-05-27T07:33:06.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.890+0000 7f471e600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f47181144d0 con 0x7f4718109840 2024-05-27T07:33:06.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.891+0000 7f4716a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4718108ef0 0x7f471819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:06.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.891+0000 7f4716a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4718108ef0 0x7f471819d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48608/0 (socket says 172.21.15.63:48608) 2024-05-27T07:33:06.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.891+0000 7f4716a00700 1 -- 172.21.15.63:0/2541396804 learned_addr learned my addr 172.21.15.63:0/2541396804 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:06.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.891+0000 7f4717400700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181045c0 0x7f471819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:06.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.891+0000 7f4717e00700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718109840 0x7f47181a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.891+0000 7f4716a00700 1 -- 172.21.15.63:0/2541396804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718109840 msgr2=0x7f47181a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.891+0000 7f4716a00700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718109840 0x7f47181a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.891+0000 7f4716a00700 1 -- 172.21.15.63:0/2541396804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181045c0 msgr2=0x7f471819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.891+0000 7f4716a00700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181045c0 0x7f471819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.891+0000 7f4716a00700 1 -- 172.21.15.63:0/2541396804 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47181a1e30 con 0x7f4718108ef0 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.892+0000 7f4716a00700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4718108ef0 0x7f471819d500 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f470400b580 tx=0x7f47040057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.892+0000 7f470fe00700 1 -- 172.21.15.63:0/2541396804 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704027070 con 0x7f4718108ef0 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.892+0000 7f471e600700 1 -- 172.21.15.63:0/2541396804 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47181a20c0 con 0x7f4718108ef0 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.892+0000 7f470fe00700 1 -- 172.21.15.63:0/2541396804 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f470400fc80 con 0x7f4718108ef0 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.892+0000 7f470fe00700 1 -- 172.21.15.63:0/2541396804 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704022d70 con 0x7f4718108ef0 2024-05-27T07:33:06.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.892+0000 7f471e600700 1 -- 172.21.15.63:0/2541396804 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47181a25a0 con 0x7f4718108ef0 2024-05-27T07:33:06.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.894+0000 7f471e600700 1 -- 172.21.15.63:0/2541396804 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47180512b0 con 0x7f4718108ef0 2024-05-27T07:33:06.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.897+0000 7f470fe00700 1 -- 172.21.15.63:0/2541396804 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f470402a070 con 0x7f4718108ef0 2024-05-27T07:33:06.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.898+0000 7f470fe00700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f46fc0b2880 0x7f46fc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:06.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.898+0000 7f470fe00700 1 -- 172.21.15.63:0/2541396804 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f47040e3670 con 0x7f4718108ef0 2024-05-27T07:33:06.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.898+0000 7f4717400700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f46fc0b2880 0x7f46fc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:06.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.899+0000 7f4717400700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f46fc0b2880 0x7f46fc0b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4708005950 tx=0x7f470800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:06.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:06.901+0000 7f470fe00700 1 -- 172.21.15.63:0/2541396804 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4704013070 con 0x7f4718108ef0 2024-05-27T07:33:07.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.187+0000 7f471e600700 1 -- 172.21.15.63:0/2541396804 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47181a9aa0 con 0x7f4718108ef0 2024-05-27T07:33:07.182 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:07 smithi063 ceph-mon[31613]: pgmap v532: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:33:07.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.189+0000 7f470fe00700 1 -- 172.21.15.63:0/2541396804 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11861 (secure 0 0 0) 0x7f47040ab030 con 0x7f4718108ef0 2024-05-27T07:33:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:33:07.184 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:29:56.944716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T07:33:07.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 -- 172.21.15.63:0/2541396804 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f46fc0b2880 msgr2=0x7f46fc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:07.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f46fc0b2880 0x7f46fc0b4d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4708005950 tx=0x7f470800b410 comp rx=0 tx=0).stop 2024-05-27T07:33:07.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 -- 172.21.15.63:0/2541396804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4718108ef0 msgr2=0x7f471819d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:07.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4718108ef0 0x7f471819d500 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f470400b580 tx=0x7f47040057c0 comp rx=0 tx=0).stop 2024-05-27T07:33:07.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 -- 172.21.15.63:0/2541396804 shutdown_connections 2024-05-27T07:33:07.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4718109840 0x7f47181a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:07.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47181045c0 0x7f471819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:07.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f46fc0b2880 0x7f46fc0b4d90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:07.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 --2- 172.21.15.63:0/2541396804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4718108ef0 0x7f471819d500 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:07.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 -- 172.21.15.63:0/2541396804 >> 172.21.15.63:0/2541396804 conn(0x7f47180ffe90 msgr2=0x7f471810a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:07.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.192+0000 7f471e600700 1 -- 172.21.15.63:0/2541396804 shutdown_connections 2024-05-27T07:33:07.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:07.193+0000 7f471e600700 1 -- 172.21.15.63:0/2541396804 wait complete. 2024-05-27T07:33:07.189 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 26 2024-05-27T07:33:07.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:07 smithi086 ceph-mon[34687]: pgmap v532: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:33:07.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:07 smithi149 ceph-mon[35408]: pgmap v532: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:33:08.042 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:33:08.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:08 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2541396804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:08.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:08 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2541396804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:08.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:08 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2541396804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:08.526 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:09.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:09 smithi086 ceph-mon[34687]: pgmap v533: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:33:09.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:09 smithi149 ceph-mon[35408]: pgmap v533: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:33:09.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:09 smithi063 ceph-mon[31613]: pgmap v533: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:33:11.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.038+0000 7f3443000700 1 -- 172.21.15.63:0/588405650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f343c1032e0 msgr2=0x7f343c1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:11.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.038+0000 7f3443000700 1 --2- 172.21.15.63:0/588405650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f343c1032e0 0x7f343c1036f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f34300099c0 tx=0x7f3430009cd0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.038+0000 7f3443000700 1 -- 172.21.15.63:0/588405650 shutdown_connections 2024-05-27T07:33:11.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.038+0000 7f3443000700 1 --2- 172.21.15.63:0/588405650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c108560 0x7f343c10fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.038+0000 7f3443000700 1 --2- 172.21.15.63:0/588405650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f343c1032e0 0x7f343c1036f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.038+0000 7f3443000700 1 --2- 172.21.15.63:0/588405650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c107c10 0x7f343c108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.038+0000 7f3443000700 1 -- 172.21.15.63:0/588405650 >> 172.21.15.63:0/588405650 conn(0x7f343c078af0 msgr2=0x7f343c078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:11.033 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:11 smithi063 ceph-mon[31613]: pgmap v534: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:11.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.038+0000 7f3443000700 1 -- 172.21.15.63:0/588405650 shutdown_connections 2024-05-27T07:33:11.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.040+0000 7f3443000700 1 -- 172.21.15.63:0/588405650 wait complete. 2024-05-27T07:33:11.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.041+0000 7f3443000700 1 Processor -- start 2024-05-27T07:33:11.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.041+0000 7f3443000700 1 -- start start 2024-05-27T07:33:11.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.041+0000 7f3443000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f343c1032e0 0x7f343c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:11.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.041+0000 7f3443000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c107c10 0x7f343c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:11.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.041+0000 7f3443000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c108560 0x7f343c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:11.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.041+0000 7f3443000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f343c1130b0 con 0x7f343c1032e0 2024-05-27T07:33:11.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.041+0000 7f3443000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f343c112f30 con 0x7f343c108560 2024-05-27T07:33:11.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.041+0000 7f3443000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f343c113230 con 0x7f343c107c10 2024-05-27T07:33:11.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.042+0000 7f3441c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f343c1032e0 0x7f343c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:11.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.042+0000 7f3441c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f343c1032e0 0x7f343c1a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48894/0 (socket says 172.21.15.63:48894) 2024-05-27T07:33:11.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.042+0000 7f3441c00700 1 -- 172.21.15.63:0/367549227 learned_addr learned my addr 172.21.15.63:0/367549227 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:11.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.042+0000 7f3442600700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c108560 0x7f343c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:11.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.042+0000 7f3441200700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c107c10 0x7f343c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:11.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.042+0000 7f3441c00700 1 -- 172.21.15.63:0/367549227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c107c10 msgr2=0x7f343c1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:11.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.042+0000 7f3441c00700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c107c10 0x7f343c1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.042+0000 7f3441c00700 1 -- 172.21.15.63:0/367549227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c108560 msgr2=0x7f343c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:11.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.042+0000 7f3441c00700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c108560 0x7f343c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.042+0000 7f3441c00700 1 -- 172.21.15.63:0/367549227 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c19b0b0 con 0x7f343c1032e0 2024-05-27T07:33:11.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.043+0000 7f3441c00700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f343c1032e0 0x7f343c1a07e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f343000b580 tx=0x7f343000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:11.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.043+0000 7f343a000700 1 -- 172.21.15.63:0/367549227 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430027070 con 0x7f343c1032e0 2024-05-27T07:33:11.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.043+0000 7f343a000700 1 -- 172.21.15.63:0/367549227 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f343000fdd0 con 0x7f343c1032e0 2024-05-27T07:33:11.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.043+0000 7f343a000700 1 -- 172.21.15.63:0/367549227 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34300173f0 con 0x7f343c1032e0 2024-05-27T07:33:11.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.043+0000 7f3443000700 1 -- 172.21.15.63:0/367549227 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343c19b340 con 0x7f343c1032e0 2024-05-27T07:33:11.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.043+0000 7f3443000700 1 -- 172.21.15.63:0/367549227 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343c19b7d0 con 0x7f343c1032e0 2024-05-27T07:33:11.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.044+0000 7f3443000700 1 -- 172.21.15.63:0/367549227 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343c104790 con 0x7f343c1032e0 2024-05-27T07:33:11.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.047+0000 7f343a000700 1 -- 172.21.15.63:0/367549227 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f343002a070 con 0x7f343c1032e0 2024-05-27T07:33:11.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.050+0000 7f343a000700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f34240b2900 0x7f34240b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:11.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.050+0000 7f343a000700 1 -- 172.21.15.63:0/367549227 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f34300e3b60 con 0x7f343c1032e0 2024-05-27T07:33:11.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.050+0000 7f3441200700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f34240b2900 0x7f34240b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:11.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.051+0000 7f3441200700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f34240b2900 0x7f34240b4e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3428005950 tx=0x7f34280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:11.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.053+0000 7f343a000700 1 -- 172.21.15.63:0/367549227 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3430031020 con 0x7f343c1032e0 2024-05-27T07:33:11.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.298+0000 7f3443000700 1 -- 172.21.15.63:0/367549227 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f343c02e5e0 con 0x7f343c1032e0 2024-05-27T07:33:11.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.301+0000 7f343a000700 1 -- 172.21.15.63:0/367549227 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24635 (secure 0 0 0) 0x7f3430013070 con 0x7f343c1032e0 2024-05-27T07:33:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:33:11.297 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":91,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:30:05.109968+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6},{"from":3,"to":7},{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:33:11.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.304+0000 7f3443000700 1 -- 172.21.15.63:0/367549227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f34240b2900 msgr2=0x7f34240b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:11.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.304+0000 7f3443000700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f34240b2900 0x7f34240b4e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3428005950 tx=0x7f34280058e0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.304+0000 7f3443000700 1 -- 172.21.15.63:0/367549227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f343c1032e0 msgr2=0x7f343c1a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:11.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.304+0000 7f3443000700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f343c1032e0 0x7f343c1a07e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f343000b580 tx=0x7f343000f510 comp rx=0 tx=0).stop 2024-05-27T07:33:11.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.305+0000 7f3443000700 1 -- 172.21.15.63:0/367549227 shutdown_connections 2024-05-27T07:33:11.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.305+0000 7f3443000700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c108560 0x7f343c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.305+0000 7f3443000700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f34240b2900 0x7f34240b4e10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.305+0000 7f3443000700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f343c1032e0 0x7f343c1a07e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.305+0000 7f3443000700 1 --2- 172.21.15.63:0/367549227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c107c10 0x7f343c1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.305+0000 7f3443000700 1 -- 172.21.15.63:0/367549227 >> 172.21.15.63:0/367549227 conn(0x7f343c078af0 msgr2=0x7f343c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:11.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.305+0000 7f3443000700 1 -- 172.21.15.63:0/367549227 shutdown_connections 2024-05-27T07:33:11.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:11.305+0000 7f3443000700 1 -- 172.21.15.63:0/367549227 wait complete. 2024-05-27T07:33:11.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:11 smithi086 ceph-mon[34687]: pgmap v534: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:11.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:11 smithi149 ceph-mon[35408]: pgmap v534: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:12.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T07:28:27.557349+0000', 'modified': '2024-05-27T07:29:56.944716+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34520, 'mds_1': 34484, 'mds_2': 34454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/3113374312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 3113374312}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 3113374312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6839/1574913181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 1574913181}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 1574913181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34430': {'gid': 34430, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6833/1769339501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6832', 'nonce': 1769339501}, {'type': 'v1', 'addr': '172.21.15.86:6833', 'nonce': 1769339501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34454': {'gid': 34454, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 35, 'addr': '172.21.15.86:6835/1494101824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 1494101824}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 1494101824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34484': {'gid': 34484, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.86:6837/559054291', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 559054291}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 559054291}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34520': {'gid': 34520, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.86:6839/953218889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 953218889}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 953218889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34520, 'qdb_cluster': [34484, 34520, 34454]} 2024-05-27T07:33:12.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-27T07:33:12.239 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-05-27T07:33:12.317 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:12 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/367549227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:33:12.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:12 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/367549227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:33:12.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:12 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/367549227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:33:12.709 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:13.126 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:13 smithi063 ceph-mon[31613]: pgmap v535: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:33:13.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:13 smithi086 ceph-mon[34687]: pgmap v535: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:33:13.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:13 smithi149 ceph-mon[35408]: pgmap v535: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:33:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.154+0000 7fe219600700 1 -- 172.21.15.63:0/4175504593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe21410b2d0 msgr2=0x7fe214117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:15.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.154+0000 7fe219600700 1 --2- 172.21.15.63:0/4175504593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe21410b2d0 0x7fe214117800 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00b0a0 tx=0x7fe20c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.149 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:15 smithi063 ceph-mon[31613]: pgmap v536: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.154+0000 7fe219600700 1 -- 172.21.15.63:0/4175504593 shutdown_connections 2024-05-27T07:33:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.154+0000 7fe219600700 1 --2- 172.21.15.63:0/4175504593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe21410b2d0 0x7fe214117800 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.154+0000 7fe219600700 1 --2- 172.21.15.63:0/4175504593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe214110960 0x7fe214110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.154+0000 7fe219600700 1 --2- 172.21.15.63:0/4175504593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21410a980 0x7fe21410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.154+0000 7fe219600700 1 -- 172.21.15.63:0/4175504593 >> 172.21.15.63:0/4175504593 conn(0x7fe214077c70 msgr2=0x7fe2140780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.154+0000 7fe219600700 1 -- 172.21.15.63:0/4175504593 shutdown_connections 2024-05-27T07:33:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.154+0000 7fe219600700 1 -- 172.21.15.63:0/4175504593 wait complete. 2024-05-27T07:33:15.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.155+0000 7fe219600700 1 Processor -- start 2024-05-27T07:33:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.155+0000 7fe219600700 1 -- start start 2024-05-27T07:33:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.156+0000 7fe219600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21410a980 0x7fe2141a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.156+0000 7fe219600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21410b2d0 0x7fe2141a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.156+0000 7fe219600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe214110960 0x7fe2141a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.156+0000 7fe219600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe21411a930 con 0x7fe214110960 2024-05-27T07:33:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.156+0000 7fe219600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe21411a7b0 con 0x7fe21410a980 2024-05-27T07:33:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.156+0000 7fe219600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe21411aab0 con 0x7fe21410b2d0 2024-05-27T07:33:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.156+0000 7fe213400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe214110960 0x7fe2141a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.156+0000 7fe213400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe214110960 0x7fe2141a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48900/0 (socket says 172.21.15.63:48900) 2024-05-27T07:33:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.156+0000 7fe213400700 1 -- 172.21.15.63:0/2166698349 learned_addr learned my addr 172.21.15.63:0/2166698349 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.157+0000 7fe212a00700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21410a980 0x7fe2141a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.157+0000 7fe212000700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21410b2d0 0x7fe2141a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.157+0000 7fe213400700 1 -- 172.21.15.63:0/2166698349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21410b2d0 msgr2=0x7fe2141a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.157+0000 7fe213400700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21410b2d0 0x7fe2141a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.157+0000 7fe213400700 1 -- 172.21.15.63:0/2166698349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21410a980 msgr2=0x7fe2141a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.157+0000 7fe213400700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21410a980 0x7fe2141a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.157+0000 7fe213400700 1 -- 172.21.15.63:0/2166698349 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2141a6210 con 0x7fe214110960 2024-05-27T07:33:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.157+0000 7fe213400700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe214110960 0x7fe2141a5b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00b830 tx=0x7fe20c004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.158+0000 7fe20b400700 1 -- 172.21.15.63:0/2166698349 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c021070 con 0x7fe214110960 2024-05-27T07:33:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.158+0000 7fe20b400700 1 -- 172.21.15.63:0/2166698349 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe20c007c80 con 0x7fe214110960 2024-05-27T07:33:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.158+0000 7fe219600700 1 -- 172.21.15.63:0/2166698349 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2141a64a0 con 0x7fe214110960 2024-05-27T07:33:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.158+0000 7fe20b400700 1 -- 172.21.15.63:0/2166698349 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c005260 con 0x7fe214110960 2024-05-27T07:33:15.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.158+0000 7fe219600700 1 -- 172.21.15.63:0/2166698349 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2141ade20 con 0x7fe214110960 2024-05-27T07:33:15.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.162+0000 7fe20b400700 1 -- 172.21.15.63:0/2166698349 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe20c024070 con 0x7fe214110960 2024-05-27T07:33:15.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.163+0000 7fe228211b80 1 Processor -- start 2024-05-27T07:33:15.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.163+0000 7fe228211b80 1 -- start start 2024-05-27T07:33:15.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.163+0000 7fe20b400700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe1f80b2900 0x7fe1f80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.163+0000 7fe228211b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 0x5643b27827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.163+0000 7fe20b400700 1 -- 172.21.15.63:0/2166698349 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fe20c00f020 con 0x7fe214110960 2024-05-27T07:33:15.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe228211b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b2782ce0 0x5643b277c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe228211b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 0x5643b277d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe228211b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5643b26aadf0 con 0x5643b2782ce0 2024-05-27T07:33:15.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe228211b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643b269a990 con 0x5643b277cd60 2024-05-27T07:33:15.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe228211b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5643b269e9d0 con 0x5643b25fddf0 2024-05-27T07:33:15.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe212000700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe1f80b2900 0x7fe1f80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe213400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 0x5643b277d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe212a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 0x5643b27827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe212000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b2782ce0 0x5643b277c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe213400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 0x5643b277d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:60350/0 (socket says 172.21.15.63:60350) 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe212a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 0x5643b27827a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:58506/0 (socket says 172.21.15.63:58506) 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.164+0000 7fe213400700 1 -- 172.21.15.63:0/3054897607 learned_addr learned my addr 172.21.15.63:0/3054897607 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.165+0000 7fe213400700 1 -- 172.21.15.63:0/3054897607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 msgr2=0x5643b27827a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.165+0000 7fe213400700 1 --2- 172.21.15.63:0/3054897607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 0x5643b27827a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.165+0000 7fe213400700 1 -- 172.21.15.63:0/3054897607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b2782ce0 msgr2=0x5643b277c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.165+0000 7fe213400700 1 --2- 172.21.15.63:0/3054897607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b2782ce0 0x5643b277c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.165+0000 7fe213400700 1 -- 172.21.15.63:0/3054897607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643b26633d0 con 0x5643b277cd60 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.165+0000 7fe212000700 1 --2- 172.21.15.63:0/3054897607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b2782ce0 0x5643b277c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:33:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.165+0000 7fe212000700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe1f80b2900 0x7fe1f80b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe200005950 tx=0x7fe20000cc00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:15.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.165+0000 7fe213400700 1 --2- 172.21.15.63:0/3054897607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 0x5643b277d170 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0a81b0 tx=0x7fe20c004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a4da791f745cef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:15.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.166+0000 7fe1ffe00700 1 -- 172.21.15.63:0/3054897607 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0aaf40 con 0x5643b277cd60 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.166+0000 7fe1ffe00700 1 -- 172.21.15.63:0/3054897607 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe20c0af2f0 con 0x5643b277cd60 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.166+0000 7fe1ffe00700 1 -- 172.21.15.63:0/3054897607 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0b3880 con 0x5643b277cd60 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.167+0000 7fe228211b80 1 -- 172.21.15.63:0/3054897607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 msgr2=0x5643b277d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.167+0000 7fe228211b80 1 --2- 172.21.15.63:0/3054897607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 0x5643b277d170 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0a81b0 tx=0x7fe20c004f90 comp rx=0 tx=0).stop 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.167+0000 7fe228211b80 1 -- 172.21.15.63:0/3054897607 shutdown_connections 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.167+0000 7fe228211b80 1 --2- 172.21.15.63:0/3054897607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 0x5643b277d170 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.167+0000 7fe228211b80 1 --2- 172.21.15.63:0/3054897607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 0x5643b27827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.167+0000 7fe228211b80 1 --2- 172.21.15.63:0/3054897607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b2782ce0 0x5643b277c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.167+0000 7fe228211b80 1 -- 172.21.15.63:0/3054897607 >> 172.21.15.63:0/3054897607 conn(0x5643b265df70 msgr2=0x5643b2777880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.167+0000 7fe228211b80 1 -- 172.21.15.63:0/3054897607 shutdown_connections 2024-05-27T07:33:15.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.167+0000 7fe228211b80 1 -- 172.21.15.63:0/3054897607 wait complete. 2024-05-27T07:33:15.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.168+0000 7fe228211b80 1 -- start start 2024-05-27T07:33:15.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.168+0000 7fe228211b80 1 Processor -- start 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe228211b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b25fddf0 0x5643b2786400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe228211b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b2786940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe228211b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 0x5643b2786e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe228211b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5643b25a0f50 con 0x5643b277cd60 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe228211b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643b263b710 con 0x5643b25fddf0 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe228211b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5643b2789570 con 0x5643b2782ce0 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe228211b80 10 client.0 _ll_register_callbacks cb 0x5643b2644c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe212000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b2786940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe212000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b2786940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48906/0 (socket says 172.21.15.63:48906) 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe212000700 1 -- 172.21.15.63:0/2836468859 learned_addr learned my addr 172.21.15.63:0/2836468859 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe212a00700 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b25fddf0 0x5643b2786400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.169+0000 7fe213400700 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 0x5643b2786e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe212000700 1 -- 172.21.15.63:0/2836468859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 msgr2=0x5643b2786e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:15.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe212000700 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 0x5643b2786e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe212000700 1 -- 172.21.15.63:0/2836468859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b25fddf0 msgr2=0x5643b2786400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe212000700 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b25fddf0 0x5643b2786400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe212000700 1 -- 172.21.15.63:0/2836468859 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643b25ea880 con 0x5643b277cd60 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe213400700 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 0x5643b2786e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe212000700 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b2786940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe200011c60 tx=0x7fe2000058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb27fc740fc122b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe1fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe1fcc00700 1 -- 172.21.15.63:0/2836468859 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200026070 con 0x5643b277cd60 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe228211b80 1 -- 172.21.15.63:0/2836468859 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe20c0af2f0 con 0x5643b277cd60 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.170+0000 7fe1fcc00700 1 -- 172.21.15.63:0/2836468859 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fe200012c70 con 0x5643b277cd60 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.171+0000 7fe1fcc00700 1 -- 172.21.15.63:0/2836468859 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe200032050 con 0x5643b277cd60 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.171+0000 7fe228211b80 10 client.15159 fetch_fsmap learned FSMap version 26 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.171+0000 7fe228211b80 1 -- 172.21.15.63:0/2836468859 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643b266b010 con 0x5643b277cd60 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.171+0000 7fe1fcc00700 1 -- 172.21.15.63:0/2836468859 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe20c0af2f0 con 0x5643b277cd60 2024-05-27T07:33:15.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.171+0000 7fe1fcc00700 1 -- 172.21.15.63:0/2836468859 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe20000f3d0 con 0x5643b277cd60 2024-05-27T07:33:15.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.172+0000 7fe228211b80 10 client.15159 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:33:15.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.172+0000 7fe228211b80 10 client.15159 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T07:33:15.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.172+0000 7fe228211b80 20 client.15159 populate_metadata read hostname 'smithi063' 2024-05-27T07:33:15.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.172+0000 7fe228211b80 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5643b27963b0 0x5643b2797820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.172+0000 7fe228211b80 4 client.15159 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:33:15.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.172+0000 7fe228211b80 1 -- 172.21.15.63:0/2836468859 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643b26acd40 con 0x5643b27963b0 2024-05-27T07:33:15.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.172+0000 7fe212a00700 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5643b27963b0 0x5643b2797820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.173+0000 7fe212a00700 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5643b27963b0 0x5643b2797820 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc2cdd7a5a5189b7 server_cookie=600808ca5ed39323 in_seq=0 out_seq=0 2024-05-27T07:33:15.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.173+0000 7fe1fcc00700 10 client.15159 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:33:15.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.175+0000 7fe1fcc00700 1 -- 172.21.15.63:0/2836468859 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5643b26acd40 con 0x5643b27963b0 2024-05-27T07:33:15.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.175+0000 7fe1fcc00700 10 client.15159 handle_command_reply: tid=0 2024-05-27T07:33:15.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.175+0000 7fe228211b80 1 client.15159 shutdown 2024-05-27T07:33:15.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 1 -- 172.21.15.63:0/2836468859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 msgr2=0x5643b2786940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:15.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b2786940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe200011c60 tx=0x7fe2000058e0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 1 -- 172.21.15.63:0/2836468859 shutdown_connections 2024-05-27T07:33:15.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 0x5643b2786e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b25fddf0 0x5643b2786400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5643b27963b0 0x5643b2797820 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 1 --2- 172.21.15.63:0/2836468859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b2786940 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 1 -- 172.21.15.63:0/2836468859 >> 172.21.15.63:0/2836468859 conn(0x5643b265df70 msgr2=0x5643b2777530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:15.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe1fcc00700 0 client.15159 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:33:15.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 1 -- 172.21.15.63:0/2836468859 shutdown_connections 2024-05-27T07:33:15.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 1 -- 172.21.15.63:0/2836468859 wait complete. 2024-05-27T07:33:15.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.176+0000 7fe228211b80 20 client.15159 trim_cache size 0 max 16384 2024-05-27T07:33:15.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.200+0000 7fe228211b80 1 Processor -- start 2024-05-27T07:33:15.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.200+0000 7fe228211b80 1 -- start start 2024-05-27T07:33:15.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.200+0000 7fe228211b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b25fddf0 0x5643b277c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.200+0000 7fe228211b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 0x5643b27927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.200+0000 7fe228211b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 0x5643b2785360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.201+0000 7fe228211b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5643b2795b60 con 0x5643b25fddf0 2024-05-27T07:33:15.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.201+0000 7fe228211b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643b2796230 con 0x5643b277cd60 2024-05-27T07:33:15.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.201+0000 7fe228211b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5643b2796900 con 0x5643b2782ce0 2024-05-27T07:33:15.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.201+0000 7fe212a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b25fddf0 0x5643b277c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.201+0000 7fe212a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b25fddf0 0x5643b277c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48922/0 (socket says 172.21.15.63:48922) 2024-05-27T07:33:15.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.201+0000 7fe212a00700 1 -- 172.21.15.63:0/1742975090 learned_addr learned my addr 172.21.15.63:0/1742975090 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:15.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.201+0000 7fe212000700 1 --2- 172.21.15.63:0/1742975090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 0x5643b27927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.201+0000 7fe213400700 1 --2- 172.21.15.63:0/1742975090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 0x5643b2785360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.202+0000 7fe212a00700 1 -- 172.21.15.63:0/1742975090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 msgr2=0x5643b2785360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:15.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.202+0000 7fe212a00700 1 --2- 172.21.15.63:0/1742975090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 0x5643b2785360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.202+0000 7fe212a00700 1 -- 172.21.15.63:0/1742975090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 msgr2=0x5643b27927d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:15.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.202+0000 7fe212a00700 1 --2- 172.21.15.63:0/1742975090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 0x5643b27927d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.202+0000 7fe212a00700 1 -- 172.21.15.63:0/1742975090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643b2784970 con 0x5643b25fddf0 2024-05-27T07:33:15.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.202+0000 7fe213400700 1 --2- 172.21.15.63:0/1742975090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 0x5643b2785360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:15.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.202+0000 7fe212a00700 1 --2- 172.21.15.63:0/1742975090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b25fddf0 0x5643b277c820 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x5643b2798170 tx=0x7fe20400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe35d92109c6cce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:15.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.202+0000 7fe1ff400700 1 -- 172.21.15.63:0/1742975090 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204003ae0 con 0x5643b25fddf0 2024-05-27T07:33:15.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.203+0000 7fe1ff400700 1 -- 172.21.15.63:0/1742975090 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe204003c80 con 0x5643b25fddf0 2024-05-27T07:33:15.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.203+0000 7fe1ff400700 1 -- 172.21.15.63:0/1742975090 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2040117c0 con 0x5643b25fddf0 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.203+0000 7fe228211b80 1 -- 172.21.15.63:0/1742975090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b25fddf0 msgr2=0x5643b277c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.203+0000 7fe228211b80 1 --2- 172.21.15.63:0/1742975090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b25fddf0 0x5643b277c820 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x5643b2798170 tx=0x7fe20400dc40 comp rx=0 tx=0).stop 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.204+0000 7fe228211b80 1 -- 172.21.15.63:0/1742975090 shutdown_connections 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.204+0000 7fe228211b80 1 --2- 172.21.15.63:0/1742975090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b2782ce0 0x5643b2785360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.204+0000 7fe228211b80 1 --2- 172.21.15.63:0/1742975090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b25fddf0 0x5643b277c820 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.204+0000 7fe228211b80 1 --2- 172.21.15.63:0/1742975090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b277cd60 0x5643b27927d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.204+0000 7fe228211b80 1 -- 172.21.15.63:0/1742975090 >> 172.21.15.63:0/1742975090 conn(0x5643b265df70 msgr2=0x5643b27801c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.204+0000 7fe228211b80 1 -- 172.21.15.63:0/1742975090 shutdown_connections 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.204+0000 7fe228211b80 1 -- 172.21.15.63:0/1742975090 wait complete. 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.204+0000 7fe228211b80 1 -- start start 2024-05-27T07:33:15.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.204+0000 7fe228211b80 1 Processor -- start 2024-05-27T07:33:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.205+0000 7fe228211b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 0x5643b27a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.205+0000 7fe228211b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b27a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.205+0000 7fe228211b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b2782ce0 0x5643b279bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.205+0000 7fe228211b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5643b27976a0 con 0x5643b277cd60 2024-05-27T07:33:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.205+0000 7fe228211b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643b2796fd0 con 0x5643b2782ce0 2024-05-27T07:33:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.205+0000 7fe228211b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5643b2797d70 con 0x5643b25fddf0 2024-05-27T07:33:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.205+0000 7fe228211b80 10 client.0 _ll_register_callbacks cb 0x5643b2644c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:33:15.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.205+0000 7fe212000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b27a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.205+0000 7fe212000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b27a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48934/0 (socket says 172.21.15.63:48934) 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.205+0000 7fe212000700 1 -- 172.21.15.63:0/4034890989 learned_addr learned my addr 172.21.15.63:0/4034890989 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe212a00700 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 0x5643b27a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe213400700 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b2782ce0 0x5643b279bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe212000700 1 -- 172.21.15.63:0/4034890989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 msgr2=0x5643b27a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe212000700 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 0x5643b27a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe212000700 1 -- 172.21.15.63:0/4034890989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b2782ce0 msgr2=0x5643b279bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe212000700 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b2782ce0 0x5643b279bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe212000700 1 -- 172.21.15.63:0/4034890989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643b27929c0 con 0x5643b277cd60 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe213400700 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b2782ce0 0x5643b279bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe212000700 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b27a11e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe20000f990 tx=0x7fe20000cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbb7fdefea47270 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe1fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:33:15.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.206+0000 7fe1fcc00700 1 -- 172.21.15.63:0/4034890989 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20003a070 con 0x5643b277cd60 2024-05-27T07:33:15.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.207+0000 7fe1fcc00700 1 -- 172.21.15.63:0/4034890989 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fe200012c70 con 0x5643b277cd60 2024-05-27T07:33:15.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.207+0000 7fe228211b80 1 -- 172.21.15.63:0/4034890989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe204003c80 con 0x5643b277cd60 2024-05-27T07:33:15.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.207+0000 7fe1fcc00700 1 -- 172.21.15.63:0/4034890989 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe200046050 con 0x5643b277cd60 2024-05-27T07:33:15.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.207+0000 7fe228211b80 10 client.15165 fetch_fsmap learned FSMap version 26 2024-05-27T07:33:15.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.207+0000 7fe228211b80 1 -- 172.21.15.63:0/4034890989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643b2801fa0 con 0x5643b277cd60 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.207+0000 7fe1fcc00700 1 -- 172.21.15.63:0/4034890989 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe204003c80 con 0x5643b277cd60 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.207+0000 7fe1fcc00700 1 -- 172.21.15.63:0/4034890989 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe200020980 con 0x5643b277cd60 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.207+0000 7fe228211b80 10 client.15165 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.208+0000 7fe228211b80 10 client.15165 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.208+0000 7fe228211b80 20 client.15165 populate_metadata read hostname 'smithi063' 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.208+0000 7fe228211b80 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5643b28043d0 0x5643b2805840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.208+0000 7fe228211b80 4 client.15165 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.208+0000 7fe228211b80 1 -- 172.21.15.63:0/4034890989 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "status"}) -- 0x5643b26a6520 con 0x5643b28043d0 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.208+0000 7fe212a00700 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5643b28043d0 0x5643b2805840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.209+0000 7fe212a00700 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5643b28043d0 0x5643b2805840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6835b7991c7c105d server_cookie=cf782f15f8d943a9 in_seq=0 out_seq=0 2024-05-27T07:33:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.209+0000 7fe1fcc00700 10 client.15165 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:33:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe1fcc00700 1 -- 172.21.15.63:0/4034890989 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5643b26a6520 con 0x5643b28043d0 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe1fcc00700 10 client.15165 handle_command_reply: tid=0 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe228211b80 1 client.15165 shutdown 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe228211b80 1 -- 172.21.15.63:0/4034890989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 msgr2=0x5643b27a11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe228211b80 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b27a11e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe20000f990 tx=0x7fe20000cec0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe228211b80 1 -- 172.21.15.63:0/4034890989 shutdown_connections 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe228211b80 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643b2782ce0 0x5643b279bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe228211b80 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5643b25fddf0 0x5643b27a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe228211b80 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5643b28043d0 0x5643b2805840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe228211b80 1 --2- 172.21.15.63:0/4034890989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5643b277cd60 0x5643b27a11e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe228211b80 1 -- 172.21.15.63:0/4034890989 >> 172.21.15.63:0/4034890989 conn(0x5643b265df70 msgr2=0x5643b2781d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.210+0000 7fe1fcc00700 0 client.15165 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:33:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.211+0000 7fe228211b80 1 -- 172.21.15.63:0/4034890989 shutdown_connections 2024-05-27T07:33:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.211+0000 7fe228211b80 1 -- 172.21.15.63:0/4034890989 wait complete. 2024-05-27T07:33:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.211+0000 7fe228211b80 20 client.15165 trim_cache size 0 max 16384 2024-05-27T07:33:15.207 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:33:15.207 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:33:15.207 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 2, 2024-05-27T07:33:15.207 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34454, 2024-05-27T07:33:15.207 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-27T07:33:15.207 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-27T07:33:15.207 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-27T07:33:15.207 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 205.229114592, 2024-05-27T07:33:15.208 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 26, 2024-05-27T07:33:15.208 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-27T07:33:15.208 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:33:15.208 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 338.65883924399998 2024-05-27T07:33:15.208 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:33:15.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.211+0000 7fe219600700 1 -- 172.21.15.63:0/2166698349 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe1f80b2900 msgr2=0x7fe1f80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:15.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.211+0000 7fe219600700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe1f80b2900 0x7fe1f80b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe200005950 tx=0x7fe20000cc00 comp rx=0 tx=0).stop 2024-05-27T07:33:15.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.211+0000 7fe219600700 1 -- 172.21.15.63:0/2166698349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe214110960 msgr2=0x7fe2141a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.211+0000 7fe219600700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe214110960 0x7fe2141a5b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00b830 tx=0x7fe20c004db0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.212+0000 7fe219600700 1 -- 172.21.15.63:0/2166698349 shutdown_connections 2024-05-27T07:33:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.212+0000 7fe219600700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe1f80b2900 0x7fe1f80b4e10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.212+0000 7fe219600700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe214110960 0x7fe2141a5b70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.212+0000 7fe219600700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21410a980 0x7fe2141a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.212+0000 7fe219600700 1 --2- 172.21.15.63:0/2166698349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21410b2d0 0x7fe2141a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.212+0000 7fe219600700 1 -- 172.21.15.63:0/2166698349 >> 172.21.15.63:0/2166698349 conn(0x7fe214077c70 msgr2=0x7fe21407b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.212+0000 7fe219600700 1 -- 172.21.15.63:0/2166698349 shutdown_connections 2024-05-27T07:33:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:15.212+0000 7fe219600700 1 -- 172.21.15.63:0/2166698349 wait complete. 2024-05-27T07:33:15.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:15 smithi086 ceph-mon[34687]: pgmap v536: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:15.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:15 smithi149 ceph-mon[35408]: pgmap v536: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:16.194 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T07:33:16.648 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:17.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:17 smithi086 ceph-mon[34687]: pgmap v537: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:33:17.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:17 smithi149 ceph-mon[35408]: pgmap v537: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:33:17.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:17 smithi063 ceph-mon[31613]: pgmap v537: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-27T07:33:18.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.990+0000 7eff4e800700 1 -- 172.21.15.63:0/834484749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff48103a00 msgr2=0x7eff48110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.990+0000 7eff4e800700 1 --2- 172.21.15.63:0/834484749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff48103a00 0x7eff48110720 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7eff30009990 tx=0x7eff30009ca0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.990+0000 7eff4e800700 1 -- 172.21.15.63:0/834484749 shutdown_connections 2024-05-27T07:33:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.990+0000 7eff4e800700 1 --2- 172.21.15.63:0/834484749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff48110c60 0x7eff48113080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.990+0000 7eff4e800700 1 --2- 172.21.15.63:0/834484749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff481030b0 0x7eff481034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.990+0000 7eff4e800700 1 --2- 172.21.15.63:0/834484749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff48103a00 0x7eff48110720 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.990+0000 7eff4e800700 1 -- 172.21.15.63:0/834484749 >> 172.21.15.63:0/834484749 conn(0x7eff480fedb0 msgr2=0x7eff481011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.990+0000 7eff4e800700 1 -- 172.21.15.63:0/834484749 shutdown_connections 2024-05-27T07:33:18.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.990+0000 7eff4e800700 1 -- 172.21.15.63:0/834484749 wait complete. 2024-05-27T07:33:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.992+0000 7eff4e800700 1 Processor -- start 2024-05-27T07:33:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.992+0000 7eff4e800700 1 -- start start 2024-05-27T07:33:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.992+0000 7eff4e800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481030b0 0x7eff4819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.992+0000 7eff4e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff48103a00 0x7eff4819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:18.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.992+0000 7eff4e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff48110c60 0x7eff481a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.992+0000 7eff4e800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff481164f0 con 0x7eff48103a00 2024-05-27T07:33:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.992+0000 7eff4e800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff48116370 con 0x7eff48110c60 2024-05-27T07:33:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.992+0000 7eff4e800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7eff48116670 con 0x7eff481030b0 2024-05-27T07:33:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff48103a00 0x7eff4819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff48103a00 0x7eff4819d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35898/0 (socket says 172.21.15.63:35898) 2024-05-27T07:33:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff3ea00700 1 -- 172.21.15.63:0/742490703 learned_addr learned my addr 172.21.15.63:0/742490703 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff47400700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481030b0 0x7eff4819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:18.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff47e00700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff48110c60 0x7eff481a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:18.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff3ea00700 1 -- 172.21.15.63:0/742490703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481030b0 msgr2=0x7eff4819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:18.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff3ea00700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481030b0 0x7eff4819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff3ea00700 1 -- 172.21.15.63:0/742490703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff48110c60 msgr2=0x7eff481a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:18.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff3ea00700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff48110c60 0x7eff481a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff3ea00700 1 -- 172.21.15.63:0/742490703 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff481a1dd0 con 0x7eff48103a00 2024-05-27T07:33:18.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff47e00700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff48110c60 0x7eff481a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:33:18.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.993+0000 7eff47400700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481030b0 0x7eff4819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:18.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.994+0000 7eff3ea00700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff48103a00 0x7eff4819d4a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7eff30000c00 tx=0x7eff30005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:18.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.994+0000 7eff44c00700 1 -- 172.21.15.63:0/742490703 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30027070 con 0x7eff48103a00 2024-05-27T07:33:18.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.994+0000 7eff4e800700 1 -- 172.21.15.63:0/742490703 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff481a2060 con 0x7eff48103a00 2024-05-27T07:33:18.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.994+0000 7eff44c00700 1 -- 172.21.15.63:0/742490703 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff3000fd40 con 0x7eff48103a00 2024-05-27T07:33:18.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.994+0000 7eff44c00700 1 -- 172.21.15.63:0/742490703 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30022ca0 con 0x7eff48103a00 2024-05-27T07:33:18.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.994+0000 7eff4e800700 1 -- 172.21.15.63:0/742490703 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff481a2540 con 0x7eff48103a00 2024-05-27T07:33:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:18.998+0000 7eff44c00700 1 -- 172.21.15.63:0/742490703 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7eff3002a070 con 0x7eff48103a00 2024-05-27T07:33:18.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.000+0000 7eff5d324b80 1 Processor -- start 2024-05-27T07:33:18.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.000+0000 7eff5d324b80 1 -- start start 2024-05-27T07:33:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f160f5fdf0 0x55f1610e47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610de820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1610ded60 0x55f1610df170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff5d324b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f16100cdf0 con 0x55f1610ded60 2024-05-27T07:33:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff5d324b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f160ffc990 con 0x55f1610e4ce0 2024-05-27T07:33:18.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff5d324b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f1610009d0 con 0x55f160f5fdf0 2024-05-27T07:33:18.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff44c00700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7eff2c0b2880 0x7eff2c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff44c00700 1 -- 172.21.15.63:0/742490703 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7eff300e3560 con 0x7eff48103a00 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1610ded60 0x55f1610df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff3ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1610ded60 0x55f1610df170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35908/0 (socket says 172.21.15.63:35908) 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff47e00700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7eff2c0b2880 0x7eff2c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.001+0000 7eff3ea00700 1 -- 172.21.15.63:0/3640753855 learned_addr learned my addr 172.21.15.63:0/3640753855 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.002+0000 7eff47400700 1 --2- 172.21.15.63:0/3640753855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.002+0000 7eff47e00700 1 --2- 172.21.15.63:0/3640753855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f160f5fdf0 0x55f1610e47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.002+0000 7eff3ea00700 1 -- 172.21.15.63:0/3640753855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f160f5fdf0 msgr2=0x55f1610e47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.002+0000 7eff3ea00700 1 --2- 172.21.15.63:0/3640753855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f160f5fdf0 0x55f1610e47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.002+0000 7eff3ea00700 1 -- 172.21.15.63:0/3640753855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 msgr2=0x55f1610de820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.002+0000 7eff3ea00700 1 --2- 172.21.15.63:0/3640753855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610de820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.002+0000 7eff3ea00700 1 -- 172.21.15.63:0/3640753855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f160fc53d0 con 0x55f1610ded60 2024-05-27T07:33:18.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.002+0000 7eff47e00700 1 --2- 172.21.15.63:0/3640753855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f160f5fdf0 0x55f1610e47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:18.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.002+0000 7eff3ea00700 1 --2- 172.21.15.63:0/3640753855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1610ded60 0x55f1610df170 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7eff300e0c70 tx=0x7eff30009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b8c5c85ca017ad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:18.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.003+0000 7eff47e00700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7eff2c0b2880 0x7eff2c0b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55f1610dfae0 tx=0x7eff4000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:18.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.003+0000 7eff3cc00700 1 -- 172.21.15.63:0/3640753855 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff300aa330 con 0x55f1610ded60 2024-05-27T07:33:18.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.003+0000 7eff3cc00700 1 -- 172.21.15.63:0/3640753855 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff300aaf20 con 0x55f1610ded60 2024-05-27T07:33:18.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.003+0000 7eff3cc00700 1 -- 172.21.15.63:0/3640753855 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30022470 con 0x55f1610ded60 2024-05-27T07:33:18.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.004+0000 7eff5d324b80 1 -- 172.21.15.63:0/3640753855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1610ded60 msgr2=0x55f1610df170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:18.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.004+0000 7eff5d324b80 1 --2- 172.21.15.63:0/3640753855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1610ded60 0x55f1610df170 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7eff300e0c70 tx=0x7eff30009ca0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.004+0000 7eff5d324b80 1 -- 172.21.15.63:0/3640753855 shutdown_connections 2024-05-27T07:33:18.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.004+0000 7eff5d324b80 1 --2- 172.21.15.63:0/3640753855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1610ded60 0x55f1610df170 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.004+0000 7eff5d324b80 1 --2- 172.21.15.63:0/3640753855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f160f5fdf0 0x55f1610e47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.004+0000 7eff5d324b80 1 --2- 172.21.15.63:0/3640753855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:18.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.004+0000 7eff5d324b80 1 -- 172.21.15.63:0/3640753855 >> 172.21.15.63:0/3640753855 conn(0x55f160fbff70 msgr2=0x55f1610d9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:19.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.004+0000 7eff5d324b80 1 -- 172.21.15.63:0/3640753855 shutdown_connections 2024-05-27T07:33:19.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.004+0000 7eff5d324b80 1 -- 172.21.15.63:0/3640753855 wait complete. 2024-05-27T07:33:19.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.005+0000 7eff5d324b80 1 -- start start 2024-05-27T07:33:19.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.005+0000 7eff5d324b80 1 Processor -- start 2024-05-27T07:33:19.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.006+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.006+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610ded60 0x55f1610e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.006+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610e4ce0 0x55f1610e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.006+0000 7eff5d324b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f160f02f50 con 0x55f160f5fdf0 2024-05-27T07:33:19.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.006+0000 7eff5d324b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f160f9d710 con 0x55f1610ded60 2024-05-27T07:33:19.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.006+0000 7eff5d324b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f1610eb570 con 0x55f1610e4ce0 2024-05-27T07:33:19.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.006+0000 7eff5d324b80 10 client.0 _ll_register_callbacks cb 0x55f160fa6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:33:19.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.006+0000 7eff47e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610e4ce0 0x55f1610e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:19.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.006+0000 7eff47e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610e4ce0 0x55f1610e8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41654/0 (socket says 172.21.15.63:41654) 2024-05-27T07:33:19.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.006+0000 7eff47e00700 1 -- 172.21.15.63:0/2900921911 learned_addr learned my addr 172.21.15.63:0/2900921911 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:19.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.007+0000 7eff3ea00700 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610ded60 0x55f1610e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:19.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.007+0000 7eff47400700 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:19.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.007+0000 7eff47400700 1 -- 172.21.15.63:0/2900921911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610e4ce0 msgr2=0x55f1610e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:19.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.007+0000 7eff47400700 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610e4ce0 0x55f1610e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.007+0000 7eff47400700 1 -- 172.21.15.63:0/2900921911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610ded60 msgr2=0x55f1610e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:19.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.007+0000 7eff47400700 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610ded60 0x55f1610e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.007+0000 7eff47400700 1 -- 172.21.15.63:0/2900921911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f160f4c880 con 0x55f160f5fdf0 2024-05-27T07:33:19.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.007+0000 7eff47e00700 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610e4ce0 0x55f1610e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:33:19.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.008+0000 7eff47400700 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610e84c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7eff38007b20 tx=0x7eff38007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b19087e2496941c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:19.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.008+0000 7eff35600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:33:19.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.008+0000 7eff35600700 1 -- 172.21.15.63:0/2900921911 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3801a020 con 0x55f160f5fdf0 2024-05-27T07:33:19.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.008+0000 7eff35600700 1 -- 172.21.15.63:0/2900921911 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7eff38012730 con 0x55f160f5fdf0 2024-05-27T07:33:19.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.008+0000 7eff5d324b80 1 -- 172.21.15.63:0/2900921911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff300aaf20 con 0x55f160f5fdf0 2024-05-27T07:33:19.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.008+0000 7eff35600700 1 -- 172.21.15.63:0/2900921911 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff38025050 con 0x55f160f5fdf0 2024-05-27T07:33:19.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.008+0000 7eff5d324b80 10 client.15177 fetch_fsmap learned FSMap version 26 2024-05-27T07:33:19.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.008+0000 7eff5d324b80 1 -- 172.21.15.63:0/2900921911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f160fcd010 con 0x55f160f5fdf0 2024-05-27T07:33:19.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.009+0000 7eff35600700 1 -- 172.21.15.63:0/2900921911 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff300aaf20 con 0x55f160f5fdf0 2024-05-27T07:33:19.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.009+0000 7eff35600700 1 -- 172.21.15.63:0/2900921911 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7eff380044c0 con 0x55f160f5fdf0 2024-05-27T07:33:19.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.009+0000 7eff5d324b80 10 client.15177 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:33:19.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.009+0000 7eff5d324b80 10 client.15177 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T07:33:19.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.009+0000 7eff5d324b80 20 client.15177 populate_metadata read hostname 'smithi063' 2024-05-27T07:33:19.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.009+0000 7eff5d324b80 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f1610f8440 0x55f1610f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.009+0000 7eff5d324b80 4 client.15177 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:33:19.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.009+0000 7eff5d324b80 1 -- 172.21.15.63:0/2900921911 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f16100ed40 con 0x55f1610f8440 2024-05-27T07:33:19.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.010+0000 7eff3ea00700 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f1610f8440 0x55f1610f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:19.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.011+0000 7eff3ea00700 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f1610f8440 0x55f1610f98b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49ac5520fb61d0b5 server_cookie=e5ee43e6e3a2521d in_seq=0 out_seq=0 2024-05-27T07:33:19.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.011+0000 7eff35600700 10 client.15177 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:33:19.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff35600700 1 -- 172.21.15.63:0/2900921911 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f16100ed40 con 0x55f1610f8440 2024-05-27T07:33:19.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff35600700 10 client.15177 handle_command_reply: tid=0 2024-05-27T07:33:19.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 client.15177 shutdown 2024-05-27T07:33:19.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 -- 172.21.15.63:0/2900921911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 msgr2=0x55f1610e84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:19.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610e84c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7eff38007b20 tx=0x7eff38007e30 comp rx=0 tx=0).stop 2024-05-27T07:33:19.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 -- 172.21.15.63:0/2900921911 shutdown_connections 2024-05-27T07:33:19.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610e4ce0 0x55f1610e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f1610f8440 0x55f1610f98b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610e84c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 --2- 172.21.15.63:0/2900921911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610ded60 0x55f1610e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 -- 172.21.15.63:0/2900921911 >> 172.21.15.63:0/2900921911 conn(0x55f160fbff70 msgr2=0x55f1610d9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:19.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff35600700 0 client.15177 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:33:19.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 -- 172.21.15.63:0/2900921911 shutdown_connections 2024-05-27T07:33:19.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 1 -- 172.21.15.63:0/2900921911 wait complete. 2024-05-27T07:33:19.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.014+0000 7eff5d324b80 20 client.15177 trim_cache size 0 max 16384 2024-05-27T07:33:19.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.038+0000 7eff5d324b80 1 Processor -- start 2024-05-27T07:33:19.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.038+0000 7eff5d324b80 1 -- start start 2024-05-27T07:33:19.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.039+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610de820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.039+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610ded60 0x55f1610e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.039+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.039+0000 7eff5d324b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f1610f7bf0 con 0x55f160f5fdf0 2024-05-27T07:33:19.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.039+0000 7eff5d324b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f1610f82c0 con 0x55f1610e4ce0 2024-05-27T07:33:19.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.039+0000 7eff5d324b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f1610f8990 con 0x55f1610ded60 2024-05-27T07:33:19.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.039+0000 7eff47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:19.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.039+0000 7eff47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610de820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35918/0 (socket says 172.21.15.63:35918) 2024-05-27T07:33:19.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.039+0000 7eff47400700 1 -- 172.21.15.63:0/2531762805 learned_addr learned my addr 172.21.15.63:0/2531762805 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:19.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.040+0000 7eff47e00700 1 --2- 172.21.15.63:0/2531762805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:19.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.040+0000 7eff3ea00700 1 --2- 172.21.15.63:0/2531762805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610ded60 0x55f1610e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:19.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.040+0000 7eff47400700 1 -- 172.21.15.63:0/2531762805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610ded60 msgr2=0x55f1610e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:19.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.040+0000 7eff47400700 1 --2- 172.21.15.63:0/2531762805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610ded60 0x55f1610e77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.040+0000 7eff47400700 1 -- 172.21.15.63:0/2531762805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 msgr2=0x55f1610e3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:19.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.040+0000 7eff47400700 1 --2- 172.21.15.63:0/2531762805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610e3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.040+0000 7eff47400700 1 -- 172.21.15.63:0/2531762805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1610e92b0 con 0x55f160f5fdf0 2024-05-27T07:33:19.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.040+0000 7eff47400700 1 --2- 172.21.15.63:0/2531762805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610de820 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7eff380048e0 tx=0x7eff38016600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c16c1d1c3e786f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:19.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.041+0000 7eff37e00700 1 -- 172.21.15.63:0/2531762805 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3801a030 con 0x55f160f5fdf0 2024-05-27T07:33:19.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.041+0000 7eff37e00700 1 -- 172.21.15.63:0/2531762805 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff38014c20 con 0x55f160f5fdf0 2024-05-27T07:33:19.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.041+0000 7eff37e00700 1 -- 172.21.15.63:0/2531762805 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38014580 con 0x55f160f5fdf0 2024-05-27T07:33:19.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.042+0000 7eff5d324b80 1 -- 172.21.15.63:0/2531762805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 msgr2=0x55f1610de820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:19.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.042+0000 7eff5d324b80 1 --2- 172.21.15.63:0/2531762805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610de820 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7eff380048e0 tx=0x7eff38016600 comp rx=0 tx=0).stop 2024-05-27T07:33:19.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.042+0000 7eff5d324b80 1 -- 172.21.15.63:0/2531762805 shutdown_connections 2024-05-27T07:33:19.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.042+0000 7eff5d324b80 1 --2- 172.21.15.63:0/2531762805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.042+0000 7eff5d324b80 1 --2- 172.21.15.63:0/2531762805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f1610de820 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.042+0000 7eff5d324b80 1 --2- 172.21.15.63:0/2531762805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610ded60 0x55f1610e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.042+0000 7eff5d324b80 1 -- 172.21.15.63:0/2531762805 >> 172.21.15.63:0/2531762805 conn(0x55f160fbff70 msgr2=0x55f1610e2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:19.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.042+0000 7eff5d324b80 1 -- 172.21.15.63:0/2531762805 shutdown_connections 2024-05-27T07:33:19.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.042+0000 7eff5d324b80 1 -- 172.21.15.63:0/2531762805 wait complete. 2024-05-27T07:33:19.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.043+0000 7eff5d324b80 1 -- start start 2024-05-27T07:33:19.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.043+0000 7eff5d324b80 1 Processor -- start 2024-05-27T07:33:19.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.043+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f161103010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.043+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610ded60 0x55f161103550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.043+0000 7eff5d324b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610f3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.043+0000 7eff5d324b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f1610f9730 con 0x55f160f5fdf0 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff5d324b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f1610f9060 con 0x55f1610e4ce0 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff5d324b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f1610f9e00 con 0x55f1610ded60 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff5d324b80 10 client.0 _ll_register_callbacks cb 0x55f160fa6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f161103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f161103010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35932/0 (socket says 172.21.15.63:35932) 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff47400700 1 -- 172.21.15.63:0/724131522 learned_addr learned my addr 172.21.15.63:0/724131522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff47e00700 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff47400700 1 -- 172.21.15.63:0/724131522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610ded60 msgr2=0x55f161103550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff47400700 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610ded60 0x55f161103550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff47400700 1 -- 172.21.15.63:0/724131522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 msgr2=0x55f1610f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:19.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff47400700 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610f3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.044+0000 7eff47400700 1 -- 172.21.15.63:0/724131522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1610f3db0 con 0x55f160f5fdf0 2024-05-27T07:33:19.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.045+0000 7eff47400700 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f161103010 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff38016480 tx=0x7eff38005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdd443727e13e80 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:19.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.045+0000 7eff35600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:33:19.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.045+0000 7eff35600700 1 -- 172.21.15.63:0/724131522 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38016a00 con 0x55f160f5fdf0 2024-05-27T07:33:19.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.045+0000 7eff35600700 1 -- 172.21.15.63:0/724131522 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7eff38016ba0 con 0x55f160f5fdf0 2024-05-27T07:33:19.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.045+0000 7eff5d324b80 1 -- 172.21.15.63:0/724131522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff38014c20 con 0x55f160f5fdf0 2024-05-27T07:33:19.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.045+0000 7eff35600700 1 -- 172.21.15.63:0/724131522 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff3803a050 con 0x55f160f5fdf0 2024-05-27T07:33:19.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.045+0000 7eff5d324b80 10 client.15183 fetch_fsmap learned FSMap version 26 2024-05-27T07:33:19.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.045+0000 7eff5d324b80 1 -- 172.21.15.63:0/724131522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f161102870 con 0x55f160f5fdf0 2024-05-27T07:33:19.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.046+0000 7eff35600700 1 -- 172.21.15.63:0/724131522 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff38014c20 con 0x55f160f5fdf0 2024-05-27T07:33:19.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.046+0000 7eff35600700 1 -- 172.21.15.63:0/724131522 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7eff38012960 con 0x55f160f5fdf0 2024-05-27T07:33:19.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.046+0000 7eff5d324b80 10 client.15183 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:33:19.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.046+0000 7eff5d324b80 10 client.15183 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T07:33:19.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.046+0000 7eff5d324b80 20 client.15183 populate_metadata read hostname 'smithi063' 2024-05-27T07:33:19.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.046+0000 7eff5d324b80 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f161165790 0x55f161166c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:19.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.046+0000 7eff5d324b80 4 client.15183 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:33:19.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.046+0000 7eff5d324b80 1 -- 172.21.15.63:0/724131522 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "status"}) -- 0x55f161008520 con 0x55f161165790 2024-05-27T07:33:19.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.047+0000 7eff3ea00700 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f161165790 0x55f161166c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.047+0000 7eff3ea00700 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f161165790 0x55f161166c00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7215f5f537373516 server_cookie=780ac71db703ac0d in_seq=0 out_seq=0 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.047+0000 7eff35600700 10 client.15183 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff35600700 1 -- 172.21.15.63:0/724131522 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f161008520 con 0x55f161165790 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff35600700 10 client.15183 handle_command_reply: tid=0 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff5d324b80 1 client.15183 shutdown 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff5d324b80 1 -- 172.21.15.63:0/724131522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 msgr2=0x55f161103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff5d324b80 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f161103010 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff38016480 tx=0x7eff38005e10 comp rx=0 tx=0).stop 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff5d324b80 1 -- 172.21.15.63:0/724131522 shutdown_connections 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff5d324b80 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1610e4ce0 0x55f1610f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff5d324b80 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f161165790 0x55f161166c00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff5d324b80 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f160f5fdf0 0x55f161103010 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff5d324b80 1 --2- 172.21.15.63:0/724131522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1610ded60 0x55f161103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff5d324b80 1 -- 172.21.15.63:0/724131522 >> 172.21.15.63:0/724131522 conn(0x55f160fbff70 msgr2=0x55f1610e7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:19.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.048+0000 7eff35600700 0 client.15183 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:33:19.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.049+0000 7eff5d324b80 1 -- 172.21.15.63:0/724131522 shutdown_connections 2024-05-27T07:33:19.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.049+0000 7eff5d324b80 1 -- 172.21.15.63:0/724131522 wait complete. 2024-05-27T07:33:19.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.049+0000 7eff5d324b80 20 client.15183 trim_cache size 0 max 16384 2024-05-27T07:33:19.045 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:33:19.045 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:33:19.045 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 1, 2024-05-27T07:33:19.046 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34484, 2024-05-27T07:33:19.046 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-27T07:33:19.046 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-27T07:33:19.046 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-27T07:33:19.046 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 210.09551832700001, 2024-05-27T07:33:19.046 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 26, 2024-05-27T07:33:19.046 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-27T07:33:19.046 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:33:19.046 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 330.22142558000002 2024-05-27T07:33:19.046 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:33:19.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 -- 172.21.15.63:0/742490703 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7eff2c0b2880 msgr2=0x7eff2c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:19.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7eff2c0b2880 0x7eff2c0b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55f1610dfae0 tx=0x7eff4000b260 comp rx=0 tx=0).stop 2024-05-27T07:33:19.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 -- 172.21.15.63:0/742490703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff48103a00 msgr2=0x7eff4819d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:19.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff48103a00 0x7eff4819d4a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7eff30000c00 tx=0x7eff30005ef0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 -- 172.21.15.63:0/742490703 shutdown_connections 2024-05-27T07:33:19.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff48110c60 0x7eff481a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff481030b0 0x7eff4819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7eff2c0b2880 0x7eff2c0b4d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 --2- 172.21.15.63:0/742490703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff48103a00 0x7eff4819d4a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:19.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 -- 172.21.15.63:0/742490703 >> 172.21.15.63:0/742490703 conn(0x7eff480fedb0 msgr2=0x7eff48112970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:19.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 -- 172.21.15.63:0/742490703 shutdown_connections 2024-05-27T07:33:19.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:19.050+0000 7eff4e800700 1 -- 172.21.15.63:0/742490703 wait complete. 2024-05-27T07:33:19.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:19 smithi086 ceph-mon[34687]: pgmap v538: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:33:19.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:19 smithi149 ceph-mon[35408]: pgmap v538: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:33:19.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:19 smithi063 ceph-mon[31613]: pgmap v538: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:33:20.141 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T07:33:20.620 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:21.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:21 smithi086 ceph-mon[34687]: pgmap v539: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:21.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:21 smithi149 ceph-mon[35408]: pgmap v539: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:21.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:21 smithi063 ceph-mon[31613]: pgmap v539: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.039+0000 7f9e47e00700 1 -- 172.21.15.63:0/1896209212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e48115040 msgr2=0x7f9e48117460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.039+0000 7f9e47e00700 1 --2- 172.21.15.63:0/1896209212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e48115040 0x7f9e48117460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000b0a0 tx=0x7f9e4000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.040+0000 7f9e47e00700 1 -- 172.21.15.63:0/1896209212 shutdown_connections 2024-05-27T07:33:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.040+0000 7f9e47e00700 1 --2- 172.21.15.63:0/1896209212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e48115040 0x7f9e48117460 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.040+0000 7f9e47e00700 1 --2- 172.21.15.63:0/1896209212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e48077ce0 0x7f9e480780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.040+0000 7f9e47e00700 1 --2- 172.21.15.63:0/1896209212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e48078630 0x7f9e48114b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.040+0000 7f9e47e00700 1 -- 172.21.15.63:0/1896209212 >> 172.21.15.63:0/1896209212 conn(0x7f9e48102020 msgr2=0x7f9e48104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.040+0000 7f9e47e00700 1 -- 172.21.15.63:0/1896209212 shutdown_connections 2024-05-27T07:33:23.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.040+0000 7f9e47e00700 1 -- 172.21.15.63:0/1896209212 wait complete. 2024-05-27T07:33:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.041+0000 7f9e47e00700 1 Processor -- start 2024-05-27T07:33:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.041+0000 7f9e47e00700 1 -- start start 2024-05-27T07:33:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e47e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e48077ce0 0x7f9e481a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e47e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e48078630 0x7f9e481a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e47e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e48115040 0x7f9e481a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e47e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4811a790 con 0x7f9e48115040 2024-05-27T07:33:23.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e47e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4811a610 con 0x7f9e48077ce0 2024-05-27T07:33:23.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e47e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4811a910 con 0x7f9e48078630 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e48115040 0x7f9e481a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e48115040 0x7f9e481a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35936/0 (socket says 172.21.15.63:35936) 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e47400700 1 -- 172.21.15.63:0/56686797 learned_addr learned my addr 172.21.15.63:0/56686797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e48078630 0x7f9e481a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.042+0000 7f9e46a00700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e48077ce0 0x7f9e481a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.043+0000 7f9e47400700 1 -- 172.21.15.63:0/56686797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e48078630 msgr2=0x7f9e481a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.043+0000 7f9e47400700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e48078630 0x7f9e481a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.043+0000 7f9e47400700 1 -- 172.21.15.63:0/56686797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e48077ce0 msgr2=0x7f9e481a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.043+0000 7f9e47400700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e48077ce0 0x7f9e481a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.043+0000 7f9e47400700 1 -- 172.21.15.63:0/56686797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e481a6010 con 0x7f9e48115040 2024-05-27T07:33:23.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.043+0000 7f9e47400700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e48115040 0x7f9e481a5970 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000b070 tx=0x7f9e40004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.044+0000 7f9e3f400700 1 -- 172.21.15.63:0/56686797 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40021070 con 0x7f9e48115040 2024-05-27T07:33:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.044+0000 7f9e3f400700 1 -- 172.21.15.63:0/56686797 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e40007b80 con 0x7f9e48115040 2024-05-27T07:33:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.044+0000 7f9e3f400700 1 -- 172.21.15.63:0/56686797 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e400046f0 con 0x7f9e48115040 2024-05-27T07:33:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.044+0000 7f9e47e00700 1 -- 172.21.15.63:0/56686797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e481a62a0 con 0x7f9e48115040 2024-05-27T07:33:23.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.044+0000 7f9e47e00700 1 -- 172.21.15.63:0/56686797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e481a6780 con 0x7f9e48115040 2024-05-27T07:33:23.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.048+0000 7f9e3f400700 1 -- 172.21.15.63:0/56686797 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9e40003a00 con 0x7f9e48115040 2024-05-27T07:33:23.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.049+0000 7f9e5c0c4b80 1 Processor -- start 2024-05-27T07:33:23.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.049+0000 7f9e5c0c4b80 1 -- start start 2024-05-27T07:33:23.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.049+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bbb4df0 0x55f40bd32e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.049+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd333c0 0x55f40bd3da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd3ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f40bc61df0 con 0x55f40bd337d0 2024-05-27T07:33:23.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f40bc51990 con 0x55f40bbb4df0 2024-05-27T07:33:23.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f40bc559d0 con 0x55f40bd333c0 2024-05-27T07:33:23.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e3f400700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9e2c0b2880 0x7f9e2c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e3fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd3ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e3f400700 1 -- 172.21.15.63:0/56686797 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f9e4000f020 con 0x7f9e48115040 2024-05-27T07:33:23.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e3fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd3ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35948/0 (socket says 172.21.15.63:35948) 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e3fe00700 1 -- 172.21.15.63:0/3120680449 learned_addr learned my addr 172.21.15.63:0/3120680449 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e46a00700 1 --2- 172.21.15.63:0/3120680449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd333c0 0x55f40bd3da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/3120680449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bbb4df0 0x55f40bd32e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.050+0000 7f9e47400700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9e2c0b2880 0x7f9e2c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.051+0000 7f9e3fe00700 1 -- 172.21.15.63:0/3120680449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd333c0 msgr2=0x55f40bd3da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.051+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/3120680449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd333c0 0x55f40bd3da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.051+0000 7f9e3fe00700 1 -- 172.21.15.63:0/3120680449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bbb4df0 msgr2=0x55f40bd32e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.051+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/3120680449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bbb4df0 0x55f40bd32e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.051+0000 7f9e3fe00700 1 -- 172.21.15.63:0/3120680449 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f40bc1a3d0 con 0x55f40bd337d0 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.051+0000 7f9e47400700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9e2c0b2880 0x7f9e2c0b4d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e400aaa20 tx=0x7f9e40009b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:23.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.051+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/3120680449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd3ffd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9e30007fd0 tx=0x7f9e30005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eead5e31b559fa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:23.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.052+0000 7f9e37e00700 1 -- 172.21.15.63:0/3120680449 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30012cb0 con 0x55f40bd337d0 2024-05-27T07:33:23.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.052+0000 7f9e37e00700 1 -- 172.21.15.63:0/3120680449 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e30012e50 con 0x55f40bd337d0 2024-05-27T07:33:23.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.052+0000 7f9e37e00700 1 -- 172.21.15.63:0/3120680449 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3001e6b0 con 0x55f40bd337d0 2024-05-27T07:33:23.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.053+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/3120680449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 msgr2=0x55f40bd3ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:23.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.053+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/3120680449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd3ffd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9e30007fd0 tx=0x7f9e30005f60 comp rx=0 tx=0).stop 2024-05-27T07:33:23.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.053+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/3120680449 shutdown_connections 2024-05-27T07:33:23.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.053+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/3120680449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd3ffd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.053+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/3120680449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bbb4df0 0x55f40bd32e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.053+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/3120680449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd333c0 0x55f40bd3da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.053+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/3120680449 >> 172.21.15.63:0/3120680449 conn(0x55f40bc14f70 msgr2=0x55f40bd2ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:23.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.053+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/3120680449 shutdown_connections 2024-05-27T07:33:23.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.053+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/3120680449 wait complete. 2024-05-27T07:33:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.054+0000 7f9e5c0c4b80 1 -- start start 2024-05-27T07:33:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.054+0000 7f9e5c0c4b80 1 Processor -- start 2024-05-27T07:33:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.054+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bbb4df0 0x55f40bd3d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.054+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd333c0 0x55f40bd3da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.054+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd337d0 0x55f40bd3dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.054+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f40bb57f50 con 0x55f40bbb4df0 2024-05-27T07:33:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.054+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f40bbf2710 con 0x55f40bd333c0 2024-05-27T07:33:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.054+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f40bd40940 con 0x55f40bd337d0 2024-05-27T07:33:23.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e5c0c4b80 10 client.0 _ll_register_callbacks cb 0x55f40bbfbc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:33:23.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e3fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bbb4df0 0x55f40bd3d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e3fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bbb4df0 0x55f40bd3d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35960/0 (socket says 172.21.15.63:35960) 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e3fe00700 1 -- 172.21.15.63:0/1814943027 learned_addr learned my addr 172.21.15.63:0/1814943027 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e46a00700 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd333c0 0x55f40bd3da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e3fe00700 1 -- 172.21.15.63:0/1814943027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd337d0 msgr2=0x55f40bd3dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd337d0 0x55f40bd3dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e3fe00700 1 -- 172.21.15.63:0/1814943027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd333c0 msgr2=0x55f40bd3da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd333c0 0x55f40bd3da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e3fe00700 1 -- 172.21.15.63:0/1814943027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f40bba1880 con 0x55f40bbb4df0 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.055+0000 7f9e46a00700 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd333c0 0x55f40bd3da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.056+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bbb4df0 0x55f40bd3d530 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9e30010f00 tx=0x7f9e30010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb848bf50817121 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:23.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.056+0000 7f9e34c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:33:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.056+0000 7f9e34c00700 1 -- 172.21.15.63:0/1814943027 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30005480 con 0x55f40bbb4df0 2024-05-27T07:33:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.056+0000 7f9e34c00700 1 -- 172.21.15.63:0/1814943027 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f9e30005620 con 0x55f40bbb4df0 2024-05-27T07:33:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.056+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/1814943027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e30012e50 con 0x55f40bbb4df0 2024-05-27T07:33:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.056+0000 7f9e34c00700 1 -- 172.21.15.63:0/1814943027 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9e30031050 con 0x55f40bbb4df0 2024-05-27T07:33:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.056+0000 7f9e5c0c4b80 10 client.15195 fetch_fsmap learned FSMap version 26 2024-05-27T07:33:23.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.056+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/1814943027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f40bc22010 con 0x55f40bbb4df0 2024-05-27T07:33:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.057+0000 7f9e34c00700 1 -- 172.21.15.63:0/1814943027 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e30012e50 con 0x55f40bbb4df0 2024-05-27T07:33:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.057+0000 7f9e34c00700 1 -- 172.21.15.63:0/1814943027 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9e30004a50 con 0x55f40bbb4df0 2024-05-27T07:33:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.057+0000 7f9e5c0c4b80 10 client.15195 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:33:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.057+0000 7f9e5c0c4b80 10 client.15195 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:33:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.057+0000 7f9e5c0c4b80 20 client.15195 populate_metadata read hostname 'smithi063' 2024-05-27T07:33:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.057+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f40bd4d5a0 0x55f40bd4ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.057+0000 7f9e5c0c4b80 4 client.15195 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:33:23.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.057+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/1814943027 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f40bc63d40 con 0x55f40bd4d5a0 2024-05-27T07:33:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.058+0000 7f9e46a00700 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f40bd4d5a0 0x55f40bd4ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.059+0000 7f9e46a00700 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f40bd4d5a0 0x55f40bd4ea10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5acf212e3e12e92 server_cookie=86b5a3ee63668284 in_seq=0 out_seq=0 2024-05-27T07:33:23.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.059+0000 7f9e34c00700 10 client.15195 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:33:23.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e34c00700 1 -- 172.21.15.63:0/1814943027 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f40bc63d40 con 0x55f40bd4d5a0 2024-05-27T07:33:23.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e34c00700 10 client.15195 handle_command_reply: tid=0 2024-05-27T07:33:23.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e5c0c4b80 1 client.15195 shutdown 2024-05-27T07:33:23.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/1814943027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bbb4df0 msgr2=0x55f40bd3d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:23.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bbb4df0 0x55f40bd3d530 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9e30010f00 tx=0x7f9e30010dd0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/1814943027 shutdown_connections 2024-05-27T07:33:23.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f40bd4d5a0 0x55f40bd4ea10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd337d0 0x55f40bd3dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bbb4df0 0x55f40bd3d530 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/1814943027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd333c0 0x55f40bd3da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e34c00700 0 client.15195 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:33:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.061+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/1814943027 >> 172.21.15.63:0/1814943027 conn(0x55f40bc14f70 msgr2=0x55f40bd2dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.062+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/1814943027 shutdown_connections 2024-05-27T07:33:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.062+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/1814943027 wait complete. 2024-05-27T07:33:23.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.062+0000 7f9e5c0c4b80 20 client.15195 trim_cache size 0 max 16384 2024-05-27T07:33:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.086+0000 7f9e5c0c4b80 1 Processor -- start 2024-05-27T07:33:23.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.086+0000 7f9e5c0c4b80 1 -- start start 2024-05-27T07:33:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.086+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bbb4df0 0x55f40bd27c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.086+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd333c0 0x55f40bd38d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.086+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd337d0 0x55f40bd49c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.086+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f40bd4cd50 con 0x55f40bd333c0 2024-05-27T07:33:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.086+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f40bd4d420 con 0x55f40bd337d0 2024-05-27T07:33:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.086+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f40bd4daf0 con 0x55f40bbb4df0 2024-05-27T07:33:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.087+0000 7f9e46a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd333c0 0x55f40bd38d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.087+0000 7f9e46a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd333c0 0x55f40bd38d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35972/0 (socket says 172.21.15.63:35972) 2024-05-27T07:33:23.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.087+0000 7f9e46a00700 1 -- 172.21.15.63:0/795224683 learned_addr learned my addr 172.21.15.63:0/795224683 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.087+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/795224683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd337d0 0x55f40bd49c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.087+0000 7f9e46a00700 1 -- 172.21.15.63:0/795224683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bbb4df0 msgr2=0x55f40bd27c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:33:23.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.087+0000 7f9e46a00700 1 --2- 172.21.15.63:0/795224683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bbb4df0 0x55f40bd27c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.087+0000 7f9e46a00700 1 -- 172.21.15.63:0/795224683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd337d0 msgr2=0x55f40bd49c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.087+0000 7f9e46a00700 1 --2- 172.21.15.63:0/795224683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd337d0 0x55f40bd49c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.087+0000 7f9e46a00700 1 -- 172.21.15.63:0/795224683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f40bd30300 con 0x55f40bd333c0 2024-05-27T07:33:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.087+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/795224683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd337d0 0x55f40bd49c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:33:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.088+0000 7f9e46a00700 1 --2- 172.21.15.63:0/795224683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd333c0 0x55f40bd38d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55f40bd343e0 tx=0x7f9e38009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717e37cce50bc4eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.088+0000 7f9e37400700 1 -- 172.21.15.63:0/795224683 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38004050 con 0x55f40bd333c0 2024-05-27T07:33:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.088+0000 7f9e37400700 1 -- 172.21.15.63:0/795224683 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e38012e50 con 0x55f40bd333c0 2024-05-27T07:33:23.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.088+0000 7f9e37400700 1 -- 172.21.15.63:0/795224683 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38011440 con 0x55f40bd333c0 2024-05-27T07:33:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.089+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/795224683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd333c0 msgr2=0x55f40bd38d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.089+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/795224683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd333c0 0x55f40bd38d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55f40bd343e0 tx=0x7f9e38009260 comp rx=0 tx=0).stop 2024-05-27T07:33:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.089+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/795224683 shutdown_connections 2024-05-27T07:33:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.089+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/795224683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bd337d0 0x55f40bd49c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.089+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/795224683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bbb4df0 0x55f40bd27c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.089+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/795224683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd333c0 0x55f40bd38d40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.089+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/795224683 >> 172.21.15.63:0/795224683 conn(0x55f40bc14f70 msgr2=0x55f40bd3dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.089+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/795224683 shutdown_connections 2024-05-27T07:33:23.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.089+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/795224683 wait complete. 2024-05-27T07:33:23.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.090+0000 7f9e5c0c4b80 1 -- start start 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.090+0000 7f9e5c0c4b80 1 Processor -- start 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bbb4df0 0x55f40bd584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd333c0 0x55f40bd51870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e5c0c4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd51db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f40bd4e890 con 0x55f40bd337d0 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f40bd4e1c0 con 0x55f40bbb4df0 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e5c0c4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f40bd4ef60 con 0x55f40bd333c0 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e5c0c4b80 10 client.0 _ll_register_callbacks cb 0x55f40bbfbc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e3fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd51db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e3fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd51db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35984/0 (socket says 172.21.15.63:35984) 2024-05-27T07:33:23.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e3fe00700 1 -- 172.21.15.63:0/2180147451 learned_addr learned my addr 172.21.15.63:0/2180147451 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.091+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bbb4df0 0x55f40bd584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.092+0000 7f9e3fe00700 1 -- 172.21.15.63:0/2180147451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd333c0 msgr2=0x55f40bd51870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:33:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.092+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd333c0 0x55f40bd51870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.092+0000 7f9e3fe00700 1 -- 172.21.15.63:0/2180147451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bbb4df0 msgr2=0x55f40bd584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.092+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bbb4df0 0x55f40bd584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.092+0000 7f9e3fe00700 1 -- 172.21.15.63:0/2180147451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f40bd52530 con 0x55f40bd337d0 2024-05-27T07:33:23.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.092+0000 7f9e3fe00700 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd51db0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9e300059d0 tx=0x7f9e30001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80998d0b3f91b5a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:23.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.092+0000 7f9e34c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:33:23.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.092+0000 7f9e34c00700 1 -- 172.21.15.63:0/2180147451 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30005480 con 0x55f40bd337d0 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.092+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/2180147451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e38012e50 con 0x55f40bd337d0 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.092+0000 7f9e34c00700 1 -- 172.21.15.63:0/2180147451 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7f9e30005620 con 0x55f40bd337d0 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e34c00700 1 -- 172.21.15.63:0/2180147451 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9e30012030 con 0x55f40bd337d0 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e5c0c4b80 10 client.15201 fetch_fsmap learned FSMap version 26 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/2180147451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f40bdb8d30 con 0x55f40bd337d0 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e34c00700 1 -- 172.21.15.63:0/2180147451 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e38012e50 con 0x55f40bd337d0 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e34c00700 1 -- 172.21.15.63:0/2180147451 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9e300106e0 con 0x55f40bd337d0 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e5c0c4b80 10 client.15201 fetch_fsmap finished waiting for FSMap version 26 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e5c0c4b80 10 client.15201 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e5c0c4b80 20 client.15201 populate_metadata read hostname 'smithi063' 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f40bdbae80 0x55f40bdbc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e5c0c4b80 4 client.15201 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:33:23.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.093+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/2180147451 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "status"}) -- 0x55f40bc5d520 con 0x55f40bdbae80 2024-05-27T07:33:23.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.094+0000 7f9e46a00700 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f40bdbae80 0x55f40bdbc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:23.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.095+0000 7f9e46a00700 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f40bdbae80 0x55f40bdbc2f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c92be7139e13b66 server_cookie=e18a423d71e0161d in_seq=0 out_seq=0 2024-05-27T07:33:23.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.095+0000 7f9e34c00700 10 client.15201 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:33:23.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.095+0000 7f9e34c00700 1 -- 172.21.15.63:0/2180147451 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f40bc5d520 con 0x55f40bdbae80 2024-05-27T07:33:23.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.095+0000 7f9e34c00700 10 client.15201 handle_command_reply: tid=0 2024-05-27T07:33:23.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.095+0000 7f9e5c0c4b80 1 client.15201 shutdown 2024-05-27T07:33:23.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/2180147451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 msgr2=0x55f40bd51db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:23.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd51db0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9e300059d0 tx=0x7f9e30001e50 comp rx=0 tx=0).stop 2024-05-27T07:33:23.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/2180147451 shutdown_connections 2024-05-27T07:33:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f40bdbae80 0x55f40bdbc2f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f40bd337d0 0x55f40bd51db0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f40bbb4df0 0x55f40bd584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 1 --2- 172.21.15.63:0/2180147451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f40bd333c0 0x55f40bd51870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e34c00700 0 client.15201 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:33:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/2180147451 >> 172.21.15.63:0/2180147451 conn(0x55f40bc14f70 msgr2=0x55f40bd3daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/2180147451 shutdown_connections 2024-05-27T07:33:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 1 -- 172.21.15.63:0/2180147451 wait complete. 2024-05-27T07:33:23.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.096+0000 7f9e5c0c4b80 20 client.15201 trim_cache size 0 max 16384 2024-05-27T07:33:23.092 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:33:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_fsid": "e24014b4-1bf8-11ef-bc9b-c7b262605968", 2024-05-27T07:33:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "whoami": 0, 2024-05-27T07:33:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 34520, 2024-05-27T07:33:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "want_state": "up:active", 2024-05-27T07:33:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "state": "up:active", 2024-05-27T07:33:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs", 2024-05-27T07:33:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "rank_uptime": 295.47953734100003, 2024-05-27T07:33:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "mdsmap_epoch": 26, 2024-05-27T07:33:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch": 91, 2024-05-27T07:33:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:33:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "uptime": 316.86378880199999 2024-05-27T07:33:23.093 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:33:23.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 -- 172.21.15.63:0/56686797 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9e2c0b2880 msgr2=0x7f9e2c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:23.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9e2c0b2880 0x7f9e2c0b4d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e400aaa20 tx=0x7f9e40009b80 comp rx=0 tx=0).stop 2024-05-27T07:33:23.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 -- 172.21.15.63:0/56686797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e48115040 msgr2=0x7f9e481a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:23.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e48115040 0x7f9e481a5970 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000b070 tx=0x7f9e40004c80 comp rx=0 tx=0).stop 2024-05-27T07:33:23.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 -- 172.21.15.63:0/56686797 shutdown_connections 2024-05-27T07:33:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7f9e2c0b2880 0x7f9e2c0b4d90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e48115040 0x7f9e481a5970 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e48077ce0 0x7f9e481a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 --2- 172.21.15.63:0/56686797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e48078630 0x7f9e481a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 -- 172.21.15.63:0/56686797 >> 172.21.15.63:0/56686797 conn(0x7f9e48102020 msgr2=0x7f9e4807ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 -- 172.21.15.63:0/56686797 shutdown_connections 2024-05-27T07:33:23.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:23.097+0000 7f9e47e00700 1 -- 172.21.15.63:0/56686797 wait complete. 2024-05-27T07:33:23.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:23 smithi086 ceph-mon[34687]: pgmap v540: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:33:23.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:23 smithi149 ceph-mon[35408]: pgmap v540: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:33:23.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:23 smithi063 ceph-mon[31613]: pgmap v540: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-27T07:33:24.132 INFO:teuthology.run_tasks:Running task sequential... 2024-05-27T07:33:24.141 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-27T07:33:24.141 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-27T07:33:24.142 INFO:teuthology.task.print:Enabling mgr modules 2024-05-27T07:33:24.142 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-27T07:33:24.143 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:33:24.143 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T07:33:24.143 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-27T07:33:24.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.439+0000 7fac3b200640 1 -- 172.21.15.63:0/3579286842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac34110060 msgr2=0x7fac34112480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:24.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.439+0000 7fac3b200640 1 --2- 172.21.15.63:0/3579286842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac34110060 0x7fac34112480 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fac30009ca0 tx=0x7fac3002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:33:24.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.440+0000 7fac3b200640 1 -- 172.21.15.63:0/3579286842 shutdown_connections 2024-05-27T07:33:24.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.440+0000 7fac3b200640 1 --2- 172.21.15.63:0/3579286842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac34110060 0x7fac34112480 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:24.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.440+0000 7fac3b200640 1 --2- 172.21.15.63:0/3579286842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3410b360 0x7fac3410b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:24.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.440+0000 7fac3b200640 1 --2- 172.21.15.63:0/3579286842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac3410a3f0 0x7fac3410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:24.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.440+0000 7fac3b200640 1 -- 172.21.15.63:0/3579286842 >> 172.21.15.63:0/3579286842 conn(0x7fac34078d20 msgr2=0x7fac3407b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:24.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.440+0000 7fac3b200640 1 -- 172.21.15.63:0/3579286842 shutdown_connections 2024-05-27T07:33:24.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.440+0000 7fac3b200640 1 -- 172.21.15.63:0/3579286842 wait complete. 2024-05-27T07:33:24.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.441+0000 7fac3b200640 1 Processor -- start 2024-05-27T07:33:24.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.441+0000 7fac3b200640 1 -- start start 2024-05-27T07:33:24.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac3b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3410a3f0 0x7fac341a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:24.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac3b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3410b360 0x7fac341a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:24.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac3b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac34110060 0x7fac341a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:24.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac3b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fac34115aa0 con 0x7fac3410b360 2024-05-27T07:33:24.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac3b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac34115920 con 0x7fac3410a3f0 2024-05-27T07:33:24.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac3b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fac34115c20 con 0x7fac34110060 2024-05-27T07:33:24.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac39400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3410b360 0x7fac341a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:24.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac39400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3410b360 0x7fac341a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36004/0 (socket says 172.21.15.63:36004) 2024-05-27T07:33:24.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac39400640 1 -- 172.21.15.63:0/2981067728 learned_addr learned my addr 172.21.15.63:0/2981067728 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:24.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac39e00640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3410a3f0 0x7fac341a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:24.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.442+0000 7fac39400640 1 -- 172.21.15.63:0/2981067728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac34110060 msgr2=0x7fac341a7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:24.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.443+0000 7fac39400640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac34110060 0x7fac341a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:24.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.443+0000 7fac39400640 1 -- 172.21.15.63:0/2981067728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3410a3f0 msgr2=0x7fac341a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:24.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.443+0000 7fac39400640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3410a3f0 0x7fac341a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:24.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.443+0000 7fac39400640 1 -- 172.21.15.63:0/2981067728 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac341a8190 con 0x7fac3410b360 2024-05-27T07:33:24.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.443+0000 7fac39400640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3410b360 0x7fac341a0b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac2400d950 tx=0x7fac2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:24.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.443+0000 7fac2ea00640 1 -- 172.21.15.63:0/2981067728 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24017070 con 0x7fac3410b360 2024-05-27T07:33:24.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.443+0000 7fac2ea00640 1 -- 172.21.15.63:0/2981067728 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac24009c30 con 0x7fac3410b360 2024-05-27T07:33:24.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.443+0000 7fac3b200640 1 -- 172.21.15.63:0/2981067728 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac341a8480 con 0x7fac3410b360 2024-05-27T07:33:24.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.443+0000 7fac2ea00640 1 -- 172.21.15.63:0/2981067728 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2400f620 con 0x7fac3410b360 2024-05-27T07:33:24.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.443+0000 7fac3b200640 1 -- 172.21.15.63:0/2981067728 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac341a8a20 con 0x7fac3410b360 2024-05-27T07:33:24.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.445+0000 7fac3b200640 1 -- 172.21.15.63:0/2981067728 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3410ee70 con 0x7fac3410b360 2024-05-27T07:33:24.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.447+0000 7fac2ea00640 1 -- 172.21.15.63:0/2981067728 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fac24010460 con 0x7fac3410b360 2024-05-27T07:33:24.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.449+0000 7fac2ea00640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fac1c0b2860 0x7fac1c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:24.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.449+0000 7fac2ea00640 1 -- 172.21.15.63:0/2981067728 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fac240e2260 con 0x7fac3410b360 2024-05-27T07:33:24.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.449+0000 7fac39e00640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fac1c0b2860 0x7fac1c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:24.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.450+0000 7fac39e00640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fac1c0b2860 0x7fac1c0b4d70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac28005fd0 tx=0x7fac28005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:24.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.454+0000 7fac2ea00640 1 -- 172.21.15.63:0/2981067728 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac240a9b90 con 0x7fac3410b360 2024-05-27T07:33:24.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:24.718+0000 7fac3b200640 1 -- 172.21.15.63:0/2981067728 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fac340638c0 con 0x7fac3410b360 2024-05-27T07:33:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.092+0000 7fac2ea00640 1 -- 172.21.15.63:0/2981067728 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fac24009de0 con 0x7fac3410b360 2024-05-27T07:33:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 -- 172.21.15.63:0/2981067728 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fac1c0b2860 msgr2=0x7fac1c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fac1c0b2860 0x7fac1c0b4d70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac28005fd0 tx=0x7fac28005850 comp rx=0 tx=0).stop 2024-05-27T07:33:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 -- 172.21.15.63:0/2981067728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3410b360 msgr2=0x7fac341a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3410b360 0x7fac341a0b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac2400d950 tx=0x7fac2400de10 comp rx=0 tx=0).stop 2024-05-27T07:33:25.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 -- 172.21.15.63:0/2981067728 shutdown_connections 2024-05-27T07:33:25.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fac1c0b2860 0x7fac1c0b4d70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:25.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac34110060 0x7fac341a7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:25.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac3410b360 0x7fac341a0b90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:25.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 --2- 172.21.15.63:0/2981067728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac3410a3f0 0x7fac341a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:25.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 -- 172.21.15.63:0/2981067728 >> 172.21.15.63:0/2981067728 conn(0x7fac34078d20 msgr2=0x7fac3410c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:25.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 -- 172.21.15.63:0/2981067728 shutdown_connections 2024-05-27T07:33:25.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:25.097+0000 7fac3b200640 1 -- 172.21.15.63:0/2981067728 wait complete. 2024-05-27T07:33:25.133 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-27T07:33:25.143 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-27T07:33:25.143 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-27T07:33:25.144 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-27T07:33:25.144 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T07:33:25.144 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:25.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:25.144 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-27T07:33:25.186 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-27T07:33:25.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:25.187 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:33:25.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:25.307 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-27T07:33:25.368 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:25 smithi063 ceph-mon[31613]: pgmap v541: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:25.369 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:25 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2981067728' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T07:33:25.369 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: ignoring --setuser ceph since I am not root 2024-05-27T07:33:25.369 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: ignoring --setgroup ceph since I am not root 2024-05-27T07:33:25.404 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-27T07:33:25.405 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-27T07:33:25.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:25 smithi086 ceph-mon[34687]: pgmap v541: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:25.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:25 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2981067728' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T07:33:25.453 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:25 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: ignoring --setuser ceph since I am not root 2024-05-27T07:33:25.453 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:25 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: ignoring --setgroup ceph since I am not root 2024-05-27T07:33:25.453 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:25 smithi086 ceph-mgr[36068]: -- 172.21.15.86:0/3471386930 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561740f205a0 con 0x561745caa480 2024-05-27T07:33:25.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:25 smithi149 ceph-mon[35408]: pgmap v541: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-27T07:33:25.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:25 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2981067728' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T07:33:25.457 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:25 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: ignoring --setuser ceph since I am not root 2024-05-27T07:33:25.457 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:25 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: ignoring --setgroup ceph since I am not root 2024-05-27T07:33:25.457 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:25 smithi149 ceph-mgr[37419]: -- 172.21.15.149:0/2426788108 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3b81185a0 con 0x55b3bcea2480 2024-05-27T07:33:25.709 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:25.555+0000 7f8698ac9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:33:25.757 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:25 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:25.506+0000 7fbd666c7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:33:25.760 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:25 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:25.507+0000 7f4569e0b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:33:25.761 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:25 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:25.765+0000 7f4569e0b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:33:25.841 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:26.116 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:25.830+0000 7f8698ac9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:33:26.116 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:25.976+0000 7f8698ac9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:33:26.117 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:25 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:25.900+0000 7f4569e0b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:33:26.117 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:25 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:25.761+0000 7fbd666c7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:33:26.117 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:25 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:25.897+0000 7fbd666c7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:33:26.395 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:26 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2981067728' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T07:33:26.395 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:26 smithi063 ceph-mon[31613]: mgrmap e19: x(active, since 15m), standbys: z, y 2024-05-27T07:33:26.407 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:26 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2981067728' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T07:33:26.407 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:26 smithi086 ceph-mon[34687]: mgrmap e19: x(active, since 15m), standbys: z, y 2024-05-27T07:33:26.407 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:26 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:26.298+0000 7f4569e0b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:33:26.418 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:26.306+0000 7fbd666c7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:33:26.419 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:26 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2981067728' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T07:33:26.419 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:26 smithi149 ceph-mon[35408]: mgrmap e19: x(active, since 15m), standbys: z, y 2024-05-27T07:33:26.702 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:26 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:26.412+0000 7f4569e0b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:33:26.706 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:26.423+0000 7fbd666c7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:33:26.709 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:26 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:26.401+0000 7f8698ac9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:33:26.709 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:26 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:26.522+0000 7f8698ac9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:33:28.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.152+0000 7fa906e00700 1 -- 172.21.15.63:0/1534518279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa900075b50 msgr2=0x7fa900075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:28.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.152+0000 7fa906e00700 1 --2- 172.21.15.63:0/1534518279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa900075b50 0x7fa900075f60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009990 tx=0x7fa8f4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.152+0000 7fa906e00700 1 -- 172.21.15.63:0/1534518279 shutdown_connections 2024-05-27T07:33:28.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.152+0000 7fa906e00700 1 --2- 172.21.15.63:0/1534518279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa900115170 0x7fa900117590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.152+0000 7fa906e00700 1 --2- 172.21.15.63:0/1534518279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa900075b50 0x7fa900075f60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.152+0000 7fa906e00700 1 --2- 172.21.15.63:0/1534518279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9000764a0 0x7fa900114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.152+0000 7fa906e00700 1 -- 172.21.15.63:0/1534518279 >> 172.21.15.63:0/1534518279 conn(0x7fa900079d00 msgr2=0x7fa90007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:28.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.152+0000 7fa906e00700 1 -- 172.21.15.63:0/1534518279 shutdown_connections 2024-05-27T07:33:28.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.152+0000 7fa906e00700 1 -- 172.21.15.63:0/1534518279 wait complete. 2024-05-27T07:33:28.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.154+0000 7fa906e00700 1 Processor -- start 2024-05-27T07:33:28.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.154+0000 7fa906e00700 1 -- start start 2024-05-27T07:33:28.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.154+0000 7fa906e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa900075b50 0x7fa9001a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:28.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.154+0000 7fa906e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9000764a0 0x7fa9001a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:28.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.155+0000 7fa906e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa900115170 0x7fa9001a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:28.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.155+0000 7fa906e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa90011a800 con 0x7fa900115170 2024-05-27T07:33:28.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.155+0000 7fa906e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa90011a680 con 0x7fa900075b50 2024-05-27T07:33:28.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.155+0000 7fa906e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa90011a980 con 0x7fa9000764a0 2024-05-27T07:33:28.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.155+0000 7fa906400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa900115170 0x7fa9001a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:28.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.155+0000 7fa905a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa900075b50 0x7fa9001a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:28.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.155+0000 7fa906400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa900115170 0x7fa9001a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36042/0 (socket says 172.21.15.63:36042) 2024-05-27T07:33:28.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.155+0000 7fa906400700 1 -- 172.21.15.63:0/4152447298 learned_addr learned my addr 172.21.15.63:0/4152447298 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:28.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa906400700 1 -- 172.21.15.63:0/4152447298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9000764a0 msgr2=0x7fa9001a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:28.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa906400700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9000764a0 0x7fa9001a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa906400700 1 -- 172.21.15.63:0/4152447298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa900075b50 msgr2=0x7fa9001a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:28.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa906400700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa900075b50 0x7fa9001a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa906400700 1 -- 172.21.15.63:0/4152447298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9001a60e0 con 0x7fa900115170 2024-05-27T07:33:28.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa905a00700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa900075b50 0x7fa9001a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:33:28.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa906400700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa900115170 0x7fa9001a5a40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800b970 tx=0x7fa8f800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:28.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa8fe000700 1 -- 172.21.15.63:0/4152447298 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8019070 con 0x7fa900115170 2024-05-27T07:33:28.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa8fe000700 1 -- 172.21.15.63:0/4152447298 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8f800cdc0 con 0x7fa900115170 2024-05-27T07:33:28.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa906e00700 1 -- 172.21.15.63:0/4152447298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9001a63d0 con 0x7fa900115170 2024-05-27T07:33:28.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.156+0000 7fa8fe000700 1 -- 172.21.15.63:0/4152447298 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8014e30 con 0x7fa900115170 2024-05-27T07:33:28.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.157+0000 7fa906e00700 1 -- 172.21.15.63:0/4152447298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9001adcf0 con 0x7fa900115170 2024-05-27T07:33:28.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.158+0000 7fa906e00700 1 -- 172.21.15.63:0/4152447298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e0005330 con 0x7fa900115170 2024-05-27T07:33:28.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.160+0000 7fa8fe000700 1 -- 172.21.15.63:0/4152447298 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fa8f801c070 con 0x7fa900115170 2024-05-27T07:33:28.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.163+0000 7fa8fe000700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa8e80b28d0 0x7fa8e80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:28.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.163+0000 7fa905a00700 1 -- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa8e80b28d0 msgr2=0x7fa8e80b4de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/675305989 2024-05-27T07:33:28.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.163+0000 7fa8fe000700 1 -- 172.21.15.63:0/4152447298 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fa8f80e1900 con 0x7fa900115170 2024-05-27T07:33:28.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.163+0000 7fa905a00700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa8e80b28d0 0x7fa8e80b4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:33:28.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.166+0000 7fa8fe000700 1 -- 172.21.15.63:0/4152447298 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8f80a9290 con 0x7fa900115170 2024-05-27T07:33:28.160 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:28 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:28.152+0000 7f8698ac9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:33:28.202 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:27 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:27.944+0000 7f4569e0b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:33:28.203 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:28 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:28.173+0000 7f4569e0b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:33:28.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.364+0000 7fa905a00700 1 -- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa8e80b28d0 msgr2=0x7fa8e80b4de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/675305989 2024-05-27T07:33:28.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.364+0000 7fa905a00700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa8e80b28d0 0x7fa8e80b4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:33:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.447+0000 7fa906e00700 1 -- 172.21.15.63:0/4152447298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa8e0005e30 con 0x7fa900115170 2024-05-27T07:33:28.442 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:28 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:28.399+0000 7f8698ac9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:33:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.448+0000 7fa8fe000700 1 -- 172.21.15.63:0/4152447298 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7fa8f80b1310 con 0x7fa900115170 2024-05-27T07:33:28.442 INFO:teuthology.orchestra.run.smithi063.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T07:33:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 -- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa8e80b28d0 msgr2=0x7fa8e80b4de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:33:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa8e80b28d0 0x7fa8e80b4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 -- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa900115170 msgr2=0x7fa9001a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa900115170 0x7fa9001a5a40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800b970 tx=0x7fa8f800bc80 comp rx=0 tx=0).stop 2024-05-27T07:33:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 -- 172.21.15.63:0/4152447298 shutdown_connections 2024-05-27T07:33:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fa8e80b28d0 0x7fa8e80b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa900115170 0x7fa9001a5a40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa900075b50 0x7fa9001a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 --2- 172.21.15.63:0/4152447298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9000764a0 0x7fa9001a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 -- 172.21.15.63:0/4152447298 >> 172.21.15.63:0/4152447298 conn(0x7fa900079d00 msgr2=0x7fa900101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 -- 172.21.15.63:0/4152447298 shutdown_connections 2024-05-27T07:33:28.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:28.452+0000 7fa906e00700 1 -- 172.21.15.63:0/4152447298 wait complete. 2024-05-27T07:33:28.448 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:27 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:27.995+0000 7fbd666c7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:33:28.448 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:28.231+0000 7fbd666c7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:33:28.448 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:28.453+0000 7fbd666c7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:33:28.702 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:28 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:28.389+0000 7f4569e0b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:33:28.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:28 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4152447298' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:33:28.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:28 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4152447298' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:33:28.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:28 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4152447298' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:33:28.709 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:28 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:28.629+0000 7f8698ac9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:33:29.150 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:28 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:28.774+0000 7f4569e0b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:33:29.150 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:28 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:28.899+0000 7f4569e0b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:33:29.164 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:29 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:29.038+0000 7f8698ac9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:33:29.206 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:28.846+0000 7fbd666c7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:33:29.206 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:28.975+0000 7fbd666c7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:33:29.381 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T07:33:29.381 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T07:33:29.381 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-27T07:33:29.381 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T07:33:29.381 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:29.381 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T07:33:29.381 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T07:33:29.381 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T07:33:29.381 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T07:33:29.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:29.382 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-27T07:33:29.408 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:29 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:29.155+0000 7f4569e0b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:33:29.434 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T07:33:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T07:33:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T07:33:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:33:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-27T07:33:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:33:29.435 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:33:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:33:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-27T07:33:29.436 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:33:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:33:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-27T07:33:29.436 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:33:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:33:29.436 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T07:33:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:33:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T07:33:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 41547sec preferred_lft 41547sec 2024-05-27T07:33:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-27T07:33:29.437 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:33:29.437 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:29 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:29.170+0000 7f8698ac9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:33:29.437 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:29 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:29.441+0000 7f8698ac9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:33:29.438 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T07:33:29.439 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:33:29.439 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-27T07:33:29.439 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add name ceph-brx type bridge 2024-05-27T07:33:29.439 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr flush dev ceph-brx 2024-05-27T07:33:29.439 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set ceph-brx up 2024-05-27T07:33:29.439 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T07:33:29.439 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-27T07:33:29.493 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:29.238+0000 7fbd666c7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:33:29.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:33:29.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:33:29.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:29.669 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T07:33:29.702 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:29 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:29.413+0000 7f4569e0b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:33:29.730 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-27T07:33:29.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:29.734 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-27T07:33:29.780 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-27T07:33:29.781 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T07:33:29.781 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T07:33:29.781 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T07:33:29.781 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T07:33:29.782 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:33:29.782 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-27T07:33:29.782 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T07:33:29.782 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T07:33:29.782 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T07:33:29.783 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-27T07:33:29.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:33:29.957 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:29.498+0000 7fbd666c7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:33:29.979 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:29 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:29.721+0000 7f8698ac9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:33:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:33:30.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:30.031 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-27T07:33:30.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:30.075 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-27T07:33:30.170 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:33:30.171 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-27T07:33:30.171 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:33:30.171 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-27T07:33:30.171 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-27T07:33:30.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:33:30.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:33:30.355 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-27T07:33:30.355 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:33:30.355 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-27T07:33:30.355 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-27T07:33:30.356 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T07:33:30.356 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-27T07:33:30.356 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-27T07:33:30.356 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-27T07:33:30.356 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-27T07:33:30.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:33:30.452 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:30 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:30.212+0000 7f4569e0b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:33:30.610 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:30.315+0000 7fbd666c7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:33:30.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:33:30.619 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:33:30.619 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-27T07:33:30.619 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.0 up 2024-05-27T07:33:30.619 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T07:33:30.619 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-27T07:33:30.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:33:30.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:33:30.810 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-27T07:33:30.810 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T07:33:30.811 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:30.840 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:30 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:30.514+0000 7f4569e0b200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:33:30.854 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-27T07:33:30.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T07:33:30.855 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:30.887 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:30 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:30.579+0000 7f8698ac9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:33:30.887 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:30 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:30.893+0000 7f8698ac9200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:33:30.938 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-27T07:33:30.956 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:30.614+0000 7fbd666c7200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:33:31.011 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:33:31.058 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-27T07:33:31.058 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-27T07:33:31.058 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-27T07:33:31.058 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-27T07:33:31.058 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-27T07:33:31.059 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/29dea1c71f72254dbd38c06f7960bb9abb9f32ebd5c2c16eab1643b7188c5af2/merged 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/9d16fb203407a66c05d9b3d5a493a1d4434dc9cf09cd534e440f7fc09a15706b/merged 2024-05-27T07:33:31.060 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/46f73ca9e7d66f52b311c4f5b060aba62e6d669224020624d26585980c697b13/merged 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2fbdf4f593ac9a3902c7e08243d1db408f4800bb2e9402d0412cfc431cfa7cc3/merged 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/080203433eaec43264a1bfede6815473130384fa7fdd4e942d6a178f0712dc7d/merged 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1bb182d89c236e7f9ab32900f23cef8a891842dcb1573f3a6e706b02621a143b/merged 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/35f9e9a1c864c5fd7d1b53a6016b7f186820d58461be037237931b30dfeb854a/merged 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/fc5228364a0f07c2d16a3bac26cf2c5be6b35931679b5f73919e28de51d6c9ef/merged 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/6509a31a1e961e9c9068d336ed9fe585f994f3b01fabdc9478177855ede51b69/merged 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2faabbd5b4689eb81b028bb408e5256b283410519a885ad5672e4555bcd53948/merged 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:33:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:33:31.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:31.062 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-27T07:33:31.107 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-27T07:33:31.108 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/58c2b8b2-2562-4ecc-bcb2-fb349cec4fca --client_fs=cephfs) 2024-05-27T07:33:31.156 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-27T07:33:31.225 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:33:31.271 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-27T07:33:31.271 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-27T07:33:31.271 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-27T07:33:31.271 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-27T07:33:31.271 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-27T07:33:31.272 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/29dea1c71f72254dbd38c06f7960bb9abb9f32ebd5c2c16eab1643b7188c5af2/merged 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/9d16fb203407a66c05d9b3d5a493a1d4434dc9cf09cd534e440f7fc09a15706b/merged 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/46f73ca9e7d66f52b311c4f5b060aba62e6d669224020624d26585980c697b13/merged 2024-05-27T07:33:31.273 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2fbdf4f593ac9a3902c7e08243d1db408f4800bb2e9402d0412cfc431cfa7cc3/merged 2024-05-27T07:33:31.274 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/080203433eaec43264a1bfede6815473130384fa7fdd4e942d6a178f0712dc7d/merged 2024-05-27T07:33:31.274 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1bb182d89c236e7f9ab32900f23cef8a891842dcb1573f3a6e706b02621a143b/merged 2024-05-27T07:33:31.274 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/35f9e9a1c864c5fd7d1b53a6016b7f186820d58461be037237931b30dfeb854a/merged 2024-05-27T07:33:31.274 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/fc5228364a0f07c2d16a3bac26cf2c5be6b35931679b5f73919e28de51d6c9ef/merged 2024-05-27T07:33:31.274 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/6509a31a1e961e9c9068d336ed9fe585f994f3b01fabdc9478177855ede51b69/merged 2024-05-27T07:33:31.274 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2faabbd5b4689eb81b028bb408e5256b283410519a885ad5672e4555bcd53948/merged 2024-05-27T07:33:31.274 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-27T07:33:31.274 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:33:31.274 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:33:31.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:31.275 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-27T07:33:31.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:2024-05-27T07:33:31.413+0000 7fd57ac12480 -1 init, newargv = 0x7fd564054370 newargc=15 2024-05-27T07:33:31.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[90751]: starting ceph client 2024-05-27T07:33:31.441 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[90751]: starting fuse 2024-05-27T07:33:31.456 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:31.183+0000 7fbd666c7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:33:31.456 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:31.313+0000 7fbd666c7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:33:31.458 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:31 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:31.103+0000 7f4569e0b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:33:31.459 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:31 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:31.237+0000 7f4569e0b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:33:31.891 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:31 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:31.505+0000 7f8698ac9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:33:31.891 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:31 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:31.646+0000 7f8698ac9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:33:32.177 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:31 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:31.725+0000 7f4569e0b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:33:32.177 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:32 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:32.181+0000 7f4569e0b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:33:32.206 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:31.799+0000 7fbd666c7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:33:32.452 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:32 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:32.308+0000 7f4569e0b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:33:32.459 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:32 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:32.182+0000 7f8698ac9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:33:32.466 DEBUG:teuthology.orchestra.run.smithi063:> sudo modprobe fuse 2024-05-27T07:33:32.547 DEBUG:teuthology.orchestra.run.smithi063:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:33:32.596 INFO:teuthology.orchestra.run.smithi063.stdout:/proc 2024-05-27T07:33:32.596 INFO:teuthology.orchestra.run.smithi063.stdout:/sys 2024-05-27T07:33:32.596 INFO:teuthology.orchestra.run.smithi063.stdout:/dev 2024-05-27T07:33:32.596 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/security 2024-05-27T07:33:32.596 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/shm 2024-05-27T07:33:32.596 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/pts 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/run 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/cgroup 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/pstore 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/bpf 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/ 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/selinux 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/mqueue 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/dev/hugepages 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/debug 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/tracing 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/fs/fuse/connections 2024-05-27T07:33:32.597 INFO:teuthology.orchestra.run.smithi063.stdout:/sys/kernel/config 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/1000 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/ceph 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/29dea1c71f72254dbd38c06f7960bb9abb9f32ebd5c2c16eab1643b7188c5af2/merged 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/9d16fb203407a66c05d9b3d5a493a1d4434dc9cf09cd534e440f7fc09a15706b/merged 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/run/user/0 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/46f73ca9e7d66f52b311c4f5b060aba62e6d669224020624d26585980c697b13/merged 2024-05-27T07:33:32.598 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2fbdf4f593ac9a3902c7e08243d1db408f4800bb2e9402d0412cfc431cfa7cc3/merged 2024-05-27T07:33:32.599 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/080203433eaec43264a1bfede6815473130384fa7fdd4e942d6a178f0712dc7d/merged 2024-05-27T07:33:32.599 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/1bb182d89c236e7f9ab32900f23cef8a891842dcb1573f3a6e706b02621a143b/merged 2024-05-27T07:33:32.599 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/35f9e9a1c864c5fd7d1b53a6016b7f186820d58461be037237931b30dfeb854a/merged 2024-05-27T07:33:32.599 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/fc5228364a0f07c2d16a3bac26cf2c5be6b35931679b5f73919e28de51d6c9ef/merged 2024-05-27T07:33:32.599 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/6509a31a1e961e9c9068d336ed9fe585f994f3b01fabdc9478177855ede51b69/merged 2024-05-27T07:33:32.599 INFO:teuthology.orchestra.run.smithi063.stdout:/var/lib/containers/storage/overlay/2faabbd5b4689eb81b028bb408e5256b283410519a885ad5672e4555bcd53948/merged 2024-05-27T07:33:32.599 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns 2024-05-27T07:33:32.599 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:33:32.599 INFO:teuthology.orchestra.run.smithi063.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T07:33:32.599 INFO:teuthology.orchestra.run.smithi063.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:32.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:32.600 DEBUG:teuthology.orchestra.run.smithi063:> ls /sys/fs/fuse/connections 2024-05-27T07:33:32.695 INFO:teuthology.orchestra.run.smithi063.stdout:119 2024-05-27T07:33:32.695 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-05-27T07:33:32.696 DEBUG:teuthology.orchestra.run.smithi063:> sudo stdin-killer -- python3 -c ' 2024-05-27T07:33:32.696 DEBUG:teuthology.orchestra.run.smithi063:> import glob 2024-05-27T07:33:32.696 DEBUG:teuthology.orchestra.run.smithi063:> import re 2024-05-27T07:33:32.696 DEBUG:teuthology.orchestra.run.smithi063:> import os 2024-05-27T07:33:32.696 DEBUG:teuthology.orchestra.run.smithi063:> import subprocess 2024-05-27T07:33:32.696 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-27T07:33:32.696 DEBUG:teuthology.orchestra.run.smithi063:> def _find_admin_socket(client_name): 2024-05-27T07:33:32.696 DEBUG:teuthology.orchestra.run.smithi063:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-27T07:33:32.696 DEBUG:teuthology.orchestra.run.smithi063:> files = glob.glob(asok_path) 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> # Given a non-glob path, it better be there 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> if "*" not in asok_path: 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> assert(len(files) == 1) 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> return files[0] 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> for f in files: 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> contents = proc_f.read() 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> if mountpoint in contents: 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> return f 2024-05-27T07:33:32.697 DEBUG:teuthology.orchestra.run.smithi063:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-27T07:33:32.698 DEBUG:teuthology.orchestra.run.smithi063:> 2024-05-27T07:33:32.698 DEBUG:teuthology.orchestra.run.smithi063:> print(_find_admin_socket("client.0")) 2024-05-27T07:33:32.698 DEBUG:teuthology.orchestra.run.smithi063:> ' 2024-05-27T07:33:32.706 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:32 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:32.266+0000 7fbd666c7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:33:32.706 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:32 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:32.395+0000 7fbd666c7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:33:32.812 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:32 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:32.675+0000 7f8698ac9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:33:32.843 INFO:teuthology.orchestra.run.smithi063.stdout:/var/run/ceph/ceph-client.0.90751.asok 2024-05-27T07:33:32.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:33:32.864 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.90751.asok 2024-05-27T07:33:32.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:32.865 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.90751.asok status 2024-05-27T07:33:32.944 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:32 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:32.637+0000 7f4569e0b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:33:32.944 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:32 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:32.816+0000 7f4569e0b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:33:32.944 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:32 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:32.948+0000 7f4569e0b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:33:33.044 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:32 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:32.729+0000 7fbd666c7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:33:33.044 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:32 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:32.915+0000 7fbd666c7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:33:33.044 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:33 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:33.048+0000 7fbd666c7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:33:33.138 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:33:33.138 INFO:teuthology.orchestra.run.smithi063.stdout: "metadata": { 2024-05-27T07:33:33.138 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_sha1": "38f9d900c83c63799fbdbe61acc9a11b0d3554a6", 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "entity_id": "0", 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": "smithi063", 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "pid": "90751", 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "root": "/volumes/qa/sv_0/58c2b8b2-2562-4ecc-bcb2-fb349cec4fca" 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_count": 0, 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "dentry_pinned_count": 0, 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "id": 15228, 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "inst": { 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "name": { 2024-05-27T07:33:33.139 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "client", 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: "num": 15228 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2876187493 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": { 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "192.168.0.1:0", 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2876187493 2024-05-27T07:33:33.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T07:33:33.141 INFO:teuthology.orchestra.run.smithi063.stdout: "inst_str": "client.15228 192.168.0.1:0/2876187493", 2024-05-27T07:33:33.141 INFO:teuthology.orchestra.run.smithi063.stdout: "addr_str": "192.168.0.1:0/2876187493", 2024-05-27T07:33:33.141 INFO:teuthology.orchestra.run.smithi063.stdout: "inode_count": 5, 2024-05-27T07:33:33.141 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_epoch": 26, 2024-05-27T07:33:33.141 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch": 91, 2024-05-27T07:33:33.141 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_epoch_barrier": 0, 2024-05-27T07:33:33.141 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklisted": false, 2024-05-27T07:33:33.141 INFO:teuthology.orchestra.run.smithi063.stdout: "fs_name": "cephfs" 2024-05-27T07:33:33.141 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:33:33.155 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:32 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:32.818+0000 7f8698ac9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:33:33.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:33.158 DEBUG:teuthology.orchestra.run.smithi063:> stat --file-system '--printf=%T 2024-05-27T07:33:33.158 DEBUG:teuthology.orchestra.run.smithi063:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:33.204 INFO:teuthology.orchestra.run.smithi063.stdout:fuseblk 2024-05-27T07:33:33.204 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:33.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:33:33.204 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:33.325 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T07:33:33.334 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:33:33.334 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T07:33:33.334 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-27T07:33:33.406 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:33.181+0000 7f8698ac9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:33:33.406 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:33.374+0000 7f8698ac9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:33:33.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.610+0000 7fc6bb200640 1 -- 172.21.15.63:0/373926692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6bc106b70 msgr2=0x7fc6bc109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:33.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.610+0000 7fc6bb200640 1 --2- 172.21.15.63:0/373926692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6bc106b70 0x7fc6bc109050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009900 tx=0x7fc6a402a410 comp rx=0 tx=0).stop 2024-05-27T07:33:33.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.610+0000 7fc6bb200640 1 -- 172.21.15.63:0/373926692 shutdown_connections 2024-05-27T07:33:33.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.610+0000 7fc6bb200640 1 --2- 172.21.15.63:0/373926692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6bc1096e0 0x7fc6bc10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:33.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.610+0000 7fc6bb200640 1 --2- 172.21.15.63:0/373926692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6bc106b70 0x7fc6bc109050 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:33.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.610+0000 7fc6bb200640 1 --2- 172.21.15.63:0/373926692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc104180 0x7fc6bc1065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:33.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.610+0000 7fc6bb200640 1 -- 172.21.15.63:0/373926692 >> 172.21.15.63:0/373926692 conn(0x7fc6bc0fdb40 msgr2=0x7fc6bc0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:33.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.610+0000 7fc6bb200640 1 -- 172.21.15.63:0/373926692 shutdown_connections 2024-05-27T07:33:33.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.610+0000 7fc6bb200640 1 -- 172.21.15.63:0/373926692 wait complete. 2024-05-27T07:33:33.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.611+0000 7fc6bb200640 1 Processor -- start 2024-05-27T07:33:33.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.611+0000 7fc6bb200640 1 -- start start 2024-05-27T07:33:33.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6bb200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc104180 0x7fc6bc19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:33.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6bb200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6bc106b70 0x7fc6bc19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:33.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6bb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6bc1096e0 0x7fc6bc1a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6bb200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10f400 con 0x7fc6bc106b70 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6bb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10f280 con 0x7fc6bc1096e0 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6bb200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10f580 con 0x7fc6bc104180 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6bc106b70 0x7fc6bc19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6bc106b70 0x7fc6bc19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38934/0 (socket says 172.21.15.63:38934) 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6b3e00640 1 -- 172.21.15.63:0/2163607907 learned_addr learned my addr 172.21.15.63:0/2163607907 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6b9e00640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc104180 0x7fc6bc19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6ba800640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6bc1096e0 0x7fc6bc1a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6b3e00640 1 -- 172.21.15.63:0/2163607907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc104180 msgr2=0x7fc6bc19c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6b3e00640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc104180 0x7fc6bc19c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6b3e00640 1 -- 172.21.15.63:0/2163607907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6bc1096e0 msgr2=0x7fc6bc1a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.612+0000 7fc6b3e00640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6bc1096e0 0x7fc6bc1a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.613+0000 7fc6b3e00640 1 -- 172.21.15.63:0/2163607907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6bc1a3e50 con 0x7fc6bc106b70 2024-05-27T07:33:33.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.613+0000 7fc6ba800640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6bc1096e0 0x7fc6bc1a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:33.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.613+0000 7fc6b3e00640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6bc106b70 0x7fc6bc19c850 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009230 tx=0x7fc6a4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:33.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.613+0000 7fc6b2a00640 1 -- 172.21.15.63:0/2163607907 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a403f030 con 0x7fc6bc106b70 2024-05-27T07:33:33.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.613+0000 7fc6b2a00640 1 -- 172.21.15.63:0/2163607907 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6a40043d0 con 0x7fc6bc106b70 2024-05-27T07:33:33.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.613+0000 7fc6b2a00640 1 -- 172.21.15.63:0/2163607907 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4035070 con 0x7fc6bc106b70 2024-05-27T07:33:33.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.613+0000 7fc6bb200640 1 -- 172.21.15.63:0/2163607907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6bc1a40e0 con 0x7fc6bc106b70 2024-05-27T07:33:33.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.613+0000 7fc6bb200640 1 -- 172.21.15.63:0/2163607907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6bc1a4620 con 0x7fc6bc106b70 2024-05-27T07:33:33.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.614+0000 7fc6bb200640 1 -- 172.21.15.63:0/2163607907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6bc1a4880 con 0x7fc6bc106b70 2024-05-27T07:33:33.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.617+0000 7fc6b2a00640 1 -- 172.21.15.63:0/2163607907 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fc6a4042030 con 0x7fc6bc106b70 2024-05-27T07:33:33.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.620+0000 7fc6b2a00640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc6a00b28b0 0x7fc6a00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:33.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.620+0000 7fc6b9e00640 1 -- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc6a00b28b0 msgr2=0x7fc6a00b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/675305989 2024-05-27T07:33:33.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.620+0000 7fc6b2a00640 1 -- 172.21.15.63:0/2163607907 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fc6a4102ee0 con 0x7fc6bc106b70 2024-05-27T07:33:33.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.620+0000 7fc6b9e00640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc6a00b28b0 0x7fc6a00b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:33:33.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.624+0000 7fc6b2a00640 1 -- 172.21.15.63:0/2163607907 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6a40ca810 con 0x7fc6bc106b70 2024-05-27T07:33:33.702 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:33 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:33.350+0000 7f4569e0b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:33:33.703 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:33 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:33.476+0000 7f4569e0b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:33:33.709 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:33.517+0000 7f8698ac9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:33:33.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.821+0000 7fc6b9e00640 1 -- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc6a00b28b0 msgr2=0x7fc6a00b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/675305989 2024-05-27T07:33:33.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.821+0000 7fc6b9e00640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc6a00b28b0 0x7fc6a00b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:33:33.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.870+0000 7fc6bb200640 1 -- 172.21.15.63:0/2163607907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fc6bc1a4a50 con 0x7fc6bc106b70 2024-05-27T07:33:33.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.925+0000 7fc6b2a00640 1 -- 172.21.15.63:0/2163607907 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7fc6a40ca110 con 0x7fc6bc106b70 2024-05-27T07:33:33.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.928+0000 7fc6bb200640 1 -- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc6a00b28b0 msgr2=0x7fc6a00b4dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:33:33.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.928+0000 7fc6bb200640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc6a00b28b0 0x7fc6a00b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:33.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.928+0000 7fc6bb200640 1 -- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6bc106b70 msgr2=0x7fc6bc19c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:33.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.928+0000 7fc6bb200640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6bc106b70 0x7fc6bc19c850 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009230 tx=0x7fc6a4005e80 comp rx=0 tx=0).stop 2024-05-27T07:33:33.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.928+0000 7fc6bb200640 1 -- 172.21.15.63:0/2163607907 shutdown_connections 2024-05-27T07:33:33.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.928+0000 7fc6bb200640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fc6a00b28b0 0x7fc6a00b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:33.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.928+0000 7fc6bb200640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6bc1096e0 0x7fc6bc1a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:33.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.928+0000 7fc6bb200640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6bc106b70 0x7fc6bc19c850 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:33.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.928+0000 7fc6bb200640 1 --2- 172.21.15.63:0/2163607907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6bc104180 0x7fc6bc19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:33.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.928+0000 7fc6bb200640 1 -- 172.21.15.63:0/2163607907 >> 172.21.15.63:0/2163607907 conn(0x7fc6bc0fdb40 msgr2=0x7fc6bc102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:33.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.929+0000 7fc6bb200640 1 -- 172.21.15.63:0/2163607907 shutdown_connections 2024-05-27T07:33:33.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:33.929+0000 7fc6bb200640 1 -- 172.21.15.63:0/2163607907 wait complete. 2024-05-27T07:33:33.929 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:33 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:33.465+0000 7fbd666c7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:33:33.929 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:33 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:33.593+0000 7fbd666c7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:33:33.947 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-27T07:33:34.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:33 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2163607907' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T07:33:34.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.213+0000 7fe4af400640 1 -- 172.21.15.63:0/2792969297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a80750b0 msgr2=0x7fe4a80754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:34.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.213+0000 7fe4af400640 1 --2- 172.21.15.63:0/2792969297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a80750b0 0x7fe4a80754c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009900 tx=0x7fe49c02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:33:34.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.213+0000 7fe4af400640 1 -- 172.21.15.63:0/2792969297 shutdown_connections 2024-05-27T07:33:34.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.213+0000 7fe4af400640 1 --2- 172.21.15.63:0/2792969297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a810bbc0 0x7fe4a810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:34.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.213+0000 7fe4af400640 1 --2- 172.21.15.63:0/2792969297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4a8075a00 0x7fe4a810b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:34.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.213+0000 7fe4af400640 1 --2- 172.21.15.63:0/2792969297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a80750b0 0x7fe4a80754c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:34.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.213+0000 7fe4af400640 1 -- 172.21.15.63:0/2792969297 >> 172.21.15.63:0/2792969297 conn(0x7fe4a80fee20 msgr2=0x7fe4a81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:34.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.213+0000 7fe4af400640 1 -- 172.21.15.63:0/2792969297 shutdown_connections 2024-05-27T07:33:34.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.213+0000 7fe4af400640 1 -- 172.21.15.63:0/2792969297 wait complete. 2024-05-27T07:33:34.208 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:33 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2163607907' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T07:33:34.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.214+0000 7fe4af400640 1 Processor -- start 2024-05-27T07:33:34.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.214+0000 7fe4af400640 1 -- start start 2024-05-27T07:33:34.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.214+0000 7fe4af400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a80750b0 0x7fe4a819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:34.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.214+0000 7fe4af400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a8075a00 0x7fe4a819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:34.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.214+0000 7fe4af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4a810bbc0 0x7fe4a81a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:34.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.214+0000 7fe4af400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8111800 con 0x7fe4a80750b0 2024-05-27T07:33:34.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.214+0000 7fe4af400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8111680 con 0x7fe4a810bbc0 2024-05-27T07:33:34.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.214+0000 7fe4af400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8111980 con 0x7fe4a8075a00 2024-05-27T07:33:34.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4aca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a80750b0 0x7fe4a819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:34.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4aca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a80750b0 0x7fe4a819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38948/0 (socket says 172.21.15.63:38948) 2024-05-27T07:33:34.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4aca00640 1 -- 172.21.15.63:0/62942336 learned_addr learned my addr 172.21.15.63:0/62942336 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:34.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4a7e00640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a8075a00 0x7fe4a819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:34.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4ad400640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4a810bbc0 0x7fe4a81a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:34.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4aca00640 1 -- 172.21.15.63:0/62942336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a8075a00 msgr2=0x7fe4a819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:34.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4aca00640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a8075a00 0x7fe4a819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:34.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4aca00640 1 -- 172.21.15.63:0/62942336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4a810bbc0 msgr2=0x7fe4a81a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:34.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4aca00640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4a810bbc0 0x7fe4a81a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:34.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4aca00640 1 -- 172.21.15.63:0/62942336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a81a3f50 con 0x7fe4a80750b0 2024-05-27T07:33:34.211 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:33 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2163607907' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T07:33:34.211 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:33.942+0000 7f8698ac9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:33:34.211 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:34 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:34.078+0000 7f8698ac9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:33:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.215+0000 7fe4a7e00640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4a8075a00 0x7fe4a819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.216+0000 7fe4aca00640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a80750b0 0x7fe4a819c4c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe49c031460 tx=0x7fe49c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.216+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c03f030 con 0x7fe4a80750b0 2024-05-27T07:33:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.216+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe49c0044d0 con 0x7fe4a80750b0 2024-05-27T07:33:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.216+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c035070 con 0x7fe4a80750b0 2024-05-27T07:33:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.216+0000 7fe4af400640 1 -- 172.21.15.63:0/62942336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a81a41e0 con 0x7fe4a80750b0 2024-05-27T07:33:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.216+0000 7fe4af400640 1 -- 172.21.15.63:0/62942336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a81a4680 con 0x7fe4a80750b0 2024-05-27T07:33:34.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.218+0000 7fe4af400640 1 -- 172.21.15.63:0/62942336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a810ad80 con 0x7fe4a80750b0 2024-05-27T07:33:34.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.220+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fe49c042030 con 0x7fe4a80750b0 2024-05-27T07:33:34.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.221+0000 7fe4a5600640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4900b28b0 0x7fe4900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:34.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.222+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9137+0+0 (secure 0 0 0) 0x7fe49c102c50 con 0x7fe4a80750b0 2024-05-27T07:33:34.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.222+0000 7fe4a7e00640 1 -- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4900b28b0 msgr2=0x7fe4900b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/675305989 2024-05-27T07:33:34.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.222+0000 7fe4a7e00640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4900b28b0 0x7fe4900b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:33:34.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.226+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe49c108020 con 0x7fe4a80750b0 2024-05-27T07:33:34.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.423+0000 7fe4a7e00640 1 -- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4900b28b0 msgr2=0x7fe4900b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/675305989 2024-05-27T07:33:34.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.423+0000 7fe4a7e00640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4900b28b0 0x7fe4900b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:33:34.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.461+0000 7fe4af400640 1 -- 172.21.15.63:0/62942336 --> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fe4a80631a0 con 0x7fe4900b28b0 2024-05-27T07:33:34.533 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:34 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:34.250+0000 7f4569e0b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:33:34.669 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:34.379+0000 7fbd666c7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:33:34.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.824+0000 7fe4a7e00640 1 -- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4900b28b0 msgr2=0x7fe4900b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/675305989 2024-05-27T07:33:34.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.824+0000 7fe4a7e00640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4900b28b0 0x7fe4900b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T07:33:34.954 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 07:33:34 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y[36064]: 2024-05-27T07:33:34.538+0000 7f4569e0b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:33:34.958 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 07:33:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z[37415]: 2024-05-27T07:33:34.673+0000 7fbd666c7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:33:34.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:34.967+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fe49c0c6170 con 0x7fe4a80750b0 2024-05-27T07:33:35.209 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:34 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:34.901+0000 7f8698ac9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:33:35.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2163607907' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T07:33:35.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:33:35.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: Standby manager daemon y restarted 2024-05-27T07:33:35.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: Standby manager daemon y started 2024-05-27T07:33:35.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:33:35.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:33:35.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:33:35.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:33:35.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: Standby manager daemon z restarted 2024-05-27T07:33:35.211 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: Standby manager daemon z started 2024-05-27T07:33:35.211 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:33:35.211 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:33:35.211 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:33:35.211 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:34 smithi063 ceph-mon[31613]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:33:35.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:35.252+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7fe49c03bb10 con 0x7fe4a80750b0 2024-05-27T07:33:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:35.253+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4900b28b0 msgr2=0x7fe4900b4dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:33:35.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:35.253+0000 7fe4a5600640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/675305989,v1:172.21.15.63:6801/675305989] conn(0x7fe4900b28b0 0x7fe4900b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:35.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2163607907' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T07:33:35.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:33:35.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: Standby manager daemon y restarted 2024-05-27T07:33:35.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: Standby manager daemon y started 2024-05-27T07:33:35.450 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:33:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:33:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:33:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:33:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: Standby manager daemon z restarted 2024-05-27T07:33:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: Standby manager daemon z started 2024-05-27T07:33:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:33:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:33:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:33:35.451 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:34 smithi086 ceph-mon[34687]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2163607907' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: Standby manager daemon y restarted 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: Standby manager daemon y started 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.86:0/799287893' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: Standby manager daemon z restarted 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: Standby manager daemon z started 2024-05-27T07:33:35.455 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:33:35.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:33:35.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:33:35.456 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:34 smithi149 ceph-mon[35408]: from='mgr.? 172.21.15.149:0/1574544565' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:33:35.478 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.217+0000 7f8698ac9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:33:36.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-27T07:33:36.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: Active manager daemon x restarted 2024-05-27T07:33:36.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: Activating manager daemon x 2024-05-27T07:33:36.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T07:33:36.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: mgrmap e21: x(active, starting, since 0.0362016s), standbys: z, y 2024-05-27T07:33:36.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:33:36.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:33:36.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:33:36.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:33:36.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:33:36.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:33:36.206 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:33:36.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:36.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:36.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:33:36.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:33:36.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:33:36.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:33:36.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:33:36.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:33:36.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:33:36.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:33:36.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:33:36.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:33:36.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:33:36.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:33:36.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:33:36.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: Manager daemon x is now available 2024-05-27T07:33:36.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:36.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:33:36.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:33:36.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-27T07:33:36.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: Active manager daemon x restarted 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: Activating manager daemon x 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: mgrmap e21: x(active, starting, since 0.0362016s), standbys: z, y 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:33:36.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:33:36.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: Manager daemon x is now available 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:36.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:33:36.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:33:36.213 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.811+0000 7f85ed000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.213 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.811+0000 7f85ed000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.213 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.811+0000 7f85ed000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.213 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.811+0000 7f85ed000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.213 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.811+0000 7f85ed000700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.213 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.813+0000 7f85e9e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.214 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.813+0000 7f85e9e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.214 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.813+0000 7f85e9e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.214 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.813+0000 7f85e9e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.214 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:35.813+0000 7f85e9e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-27T07:33:36.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: Active manager daemon x restarted 2024-05-27T07:33:36.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: Activating manager daemon x 2024-05-27T07:33:36.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T07:33:36.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: mgrmap e21: x(active, starting, since 0.0362016s), standbys: z, y 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:33:36.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:33:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: Manager daemon x is now available 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:33:36.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:33:36.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.261+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 <== mon.0 v2:172.21.15.63:3300/0 9 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fe49c08ac30 con 0x7fe4a80750b0 2024-05-27T07:33:36.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.262+0000 7fe4a5600640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe490039f10 0x7fe49003a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:36.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.262+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 --> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fe49c0c89e0 con 0x7fe490039f10 2024-05-27T07:33:36.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.266+0000 7fe4a7e00640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe490039f10 0x7fe49003a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:36.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.274+0000 7fe4a7e00640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe490039f10 0x7fe49003a320 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe4a819d560 tx=0x7fe494005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:36.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.314+0000 7fe4a5600640 1 -- 172.21.15.63:0/62942336 <== mgr.15216 v2:172.21.15.63:6800/1343315489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fe49c0c89e0 con 0x7fe490039f10 2024-05-27T07:33:36.308 INFO:teuthology.orchestra.run.smithi063.stdout:{} 2024-05-27T07:33:36.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.317+0000 7fe4af400640 1 -- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe490039f10 msgr2=0x7fe49003a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:36.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.317+0000 7fe4af400640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe490039f10 0x7fe49003a320 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe4a819d560 tx=0x7fe494005f90 comp rx=0 tx=0).stop 2024-05-27T07:33:36.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.317+0000 7fe4af400640 1 -- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a80750b0 msgr2=0x7fe4a819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:36.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.317+0000 7fe4af400640 1 --2- 172.21.15.63:0/62942336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4a80750b0 0x7fe4a819c4c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe49c031460 tx=0x7fe49c0042e0 comp rx=0 tx=0).stop 2024-05-27T07:33:36.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.317+0000 7fe4ad400640 1 -- 172.21.15.63:0/62942336 reap_dead start 2024-05-27T07:33:36.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.317+0000 7fe4af400640 1 -- 172.21.15.63:0/62942336 shutdown_connections 2024-05-27T07:33:36.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.317+0000 7fe4af400640 1 -- 172.21.15.63:0/62942336 >> 172.21.15.63:0/62942336 conn(0x7fe4a80fee20 msgr2=0x7fe4a80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:36.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.317+0000 7fe4af400640 1 -- 172.21.15.63:0/62942336 shutdown_connections 2024-05-27T07:33:36.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.317+0000 7fe4af400640 1 -- 172.21.15.63:0/62942336 wait complete. 2024-05-27T07:33:36.334 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-27T07:33:36.345 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-27T07:33:36.410 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T07:33:36.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.696+0000 7f731a600640 1 -- 172.21.15.63:0/459681927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7314103d60 msgr2=0x7f7314104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:36.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.696+0000 7f731a600640 1 --2- 172.21.15.63:0/459681927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7314103d60 0x7f7314104170 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7300009900 tx=0x7f730002a410 comp rx=0 tx=0).stop 2024-05-27T07:33:36.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.696+0000 7f731a600640 1 -- 172.21.15.63:0/459681927 shutdown_connections 2024-05-27T07:33:36.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.696+0000 7f731a600640 1 --2- 172.21.15.63:0/459681927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7314108a60 0x7f731410ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:36.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.696+0000 7f731a600640 1 --2- 172.21.15.63:0/459681927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7314103d60 0x7f7314104170 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:36.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.696+0000 7f731a600640 1 --2- 172.21.15.63:0/459681927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7314102b20 0x7f7314103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:36.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.696+0000 7f731a600640 1 -- 172.21.15.63:0/459681927 >> 172.21.15.63:0/459681927 conn(0x7f73140fdd10 msgr2=0x7f73141001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:36.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.696+0000 7f731a600640 1 -- 172.21.15.63:0/459681927 shutdown_connections 2024-05-27T07:33:36.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.696+0000 7f731a600640 1 -- 172.21.15.63:0/459681927 wait complete. 2024-05-27T07:33:36.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.697+0000 7f731a600640 1 Processor -- start 2024-05-27T07:33:36.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.697+0000 7f731a600640 1 -- start start 2024-05-27T07:33:36.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.697+0000 7f731a600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7314102b20 0x7f731419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:36.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.697+0000 7f731a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7314103d60 0x7f731419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:36.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f731a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7314108a60 0x7f73141a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:36.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f731a600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f731410e4d0 con 0x7f7314108a60 2024-05-27T07:33:36.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f731a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f731410e350 con 0x7f7314103d60 2024-05-27T07:33:36.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f731a600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f731410e650 con 0x7f7314102b20 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7313e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7314108a60 0x7f73141a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7313400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7314102b20 0x7f731419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7313e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7314108a60 0x7f73141a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39106/0 (socket says 172.21.15.63:39106) 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7313e00640 1 -- 172.21.15.63:0/1675127738 learned_addr learned my addr 172.21.15.63:0/1675127738 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7312a00640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7314103d60 0x7f731419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7313e00640 1 -- 172.21.15.63:0/1675127738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7314102b20 msgr2=0x7f731419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7313e00640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7314102b20 0x7f731419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7313e00640 1 -- 172.21.15.63:0/1675127738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7314103d60 msgr2=0x7f731419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7313e00640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7314103d60 0x7f731419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7313e00640 1 -- 172.21.15.63:0/1675127738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73141a1dc0 con 0x7f7314108a60 2024-05-27T07:33:36.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7313400640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7314102b20 0x7f731419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:33:36.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.698+0000 7f7312a00640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7314103d60 0x7f731419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:33:36.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.699+0000 7f7313e00640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7314108a60 0x7f73141a1750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f730c009fd0 tx=0x7f730c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:36.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.699+0000 7f730be00640 1 -- 172.21.15.63:0/1675127738 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c017070 con 0x7f7314108a60 2024-05-27T07:33:36.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.699+0000 7f730be00640 1 -- 172.21.15.63:0/1675127738 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f730c003e00 con 0x7f7314108a60 2024-05-27T07:33:36.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.699+0000 7f731a600640 1 -- 172.21.15.63:0/1675127738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73141a20b0 con 0x7f7314108a60 2024-05-27T07:33:36.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.699+0000 7f730be00640 1 -- 172.21.15.63:0/1675127738 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c010460 con 0x7f7314108a60 2024-05-27T07:33:36.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.699+0000 7f731a600640 1 -- 172.21.15.63:0/1675127738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73141a25b0 con 0x7f7314108a60 2024-05-27T07:33:36.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.700+0000 7f731a600640 1 -- 172.21.15.63:0/1675127738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73141a2810 con 0x7f7314108a60 2024-05-27T07:33:36.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.703+0000 7f730be00640 1 -- 172.21.15.63:0/1675127738 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f730c003970 con 0x7f7314108a60 2024-05-27T07:33:36.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.704+0000 7f730be00640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b25e0 0x7f72f80b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:36.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.704+0000 7f730be00640 1 -- 172.21.15.63:0/1675127738 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f730c0e22e0 con 0x7f7314108a60 2024-05-27T07:33:36.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.704+0000 7f7313400640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b25e0 0x7f72f80b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:36.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.705+0000 7f7313400640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b25e0 0x7f72f80b4af0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7314103720 tx=0x7f7304005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:36.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.709+0000 7f730be00640 1 -- 172.21.15.63:0/1675127738 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f730c0a9810 con 0x7f7314108a60 2024-05-27T07:33:36.709 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:36 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:36.288+0000 7f85e6200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.709 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:36 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:36.288+0000 7f85e6200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.709 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:36 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:36.288+0000 7f85e6200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.709 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:36 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:36.288+0000 7f85e6200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.709 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:33:36 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:33:36.288+0000 7f85e6200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:33:36.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.935+0000 7f731a600640 1 -- 172.21.15.63:0/1675127738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73141a29e0 con 0x7f7314108a60 2024-05-27T07:33:36.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.939+0000 7f730be00640 1 -- 172.21.15.63:0/1675127738 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+25011 (secure 0 0 0) 0x7f730c0adb10 con 0x7f7314108a60 2024-05-27T07:33:36.933 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:33:36.933 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":92,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:33:35.220889+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6},{"from":3,"to":7},{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/1961493183":"2024-05-28T07:33:35.220779+0000","172.21.15.63:6801/675305989":"2024-05-28T07:33:35.220779+0000","172.21.15.63:0/1202958531":"2024-05-28T07:33:35.220779+0000","172.21.15.63:6800/675305989":"2024-05-28T07:33:35.220779+0000","172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3428263920":"2024-05-28T07:33:35.220779+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1558308201":"2024-05-28T07:33:35.220779+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:33:36.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.942+0000 7f731a600640 1 -- 172.21.15.63:0/1675127738 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b25e0 msgr2=0x7f72f80b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:36.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.942+0000 7f731a600640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b25e0 0x7f72f80b4af0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7314103720 tx=0x7f7304005f90 comp rx=0 tx=0).stop 2024-05-27T07:33:36.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.942+0000 7f731a600640 1 -- 172.21.15.63:0/1675127738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7314108a60 msgr2=0x7f73141a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:36.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.942+0000 7f731a600640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7314108a60 0x7f73141a1750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f730c009fd0 tx=0x7f730c00b260 comp rx=0 tx=0).stop 2024-05-27T07:33:36.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.942+0000 7f731a600640 1 -- 172.21.15.63:0/1675127738 shutdown_connections 2024-05-27T07:33:36.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.942+0000 7f731a600640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b25e0 0x7f72f80b4af0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:36.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.942+0000 7f731a600640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7314108a60 0x7f73141a1750 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:36.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.942+0000 7f731a600640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7314103d60 0x7f731419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:36.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.942+0000 7f731a600640 1 --2- 172.21.15.63:0/1675127738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7314102b20 0x7f731419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:36.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.942+0000 7f731a600640 1 -- 172.21.15.63:0/1675127738 >> 172.21.15.63:0/1675127738 conn(0x7f73140fdd10 msgr2=0x7f731410a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:36.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.943+0000 7f731a600640 1 -- 172.21.15.63:0/1675127738 shutdown_connections 2024-05-27T07:33:36.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:36.943+0000 7f731a600640 1 -- 172.21.15.63:0/1675127738 wait complete. 2024-05-27T07:33:36.963 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T07:21:25.773038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T07:28:19.257874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T07:28:23.339839+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T07:28:35.867485+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T07:33:36.963 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T07:33:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.241+0000 7f8416a00640 1 -- 172.21.15.63:0/672775902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8410103d40 msgr2=0x7f8410104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.241+0000 7f8416a00640 1 --2- 172.21.15.63:0/672775902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8410103d40 0x7f8410104150 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009900 tx=0x7f83fc02a410 comp rx=0 tx=0).stop 2024-05-27T07:33:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.241+0000 7f8416a00640 1 -- 172.21.15.63:0/672775902 shutdown_connections 2024-05-27T07:33:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.241+0000 7f8416a00640 1 --2- 172.21.15.63:0/672775902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8410108a40 0x7f841010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.241+0000 7f8416a00640 1 --2- 172.21.15.63:0/672775902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8410103d40 0x7f8410104150 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.241+0000 7f8416a00640 1 --2- 172.21.15.63:0/672775902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8410102b00 0x7f8410103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.241+0000 7f8416a00640 1 -- 172.21.15.63:0/672775902 >> 172.21.15.63:0/672775902 conn(0x7f84100fdcf0 msgr2=0x7f84101001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.242+0000 7f8416a00640 1 -- 172.21.15.63:0/672775902 shutdown_connections 2024-05-27T07:33:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.242+0000 7f8416a00640 1 -- 172.21.15.63:0/672775902 wait complete. 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f8416a00640 1 Processor -- start 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f8416a00640 1 -- start start 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f8416a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8410102b00 0x7f841019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f8416a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8410103d40 0x7f841019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f8416a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8410108a40 0x7f84101a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f8416a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f841010f640 con 0x7f8410103d40 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f8416a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f841010f4c0 con 0x7f8410102b00 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f8416a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f841010f7c0 con 0x7f8410108a40 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f840f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8410103d40 0x7f841019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f840f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8410103d40 0x7f841019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39136/0 (socket says 172.21.15.63:39136) 2024-05-27T07:33:37.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.243+0000 7f840f400640 1 -- 172.21.15.63:0/1956604901 learned_addr learned my addr 172.21.15.63:0/1956604901 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:37.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.244+0000 7f840fe00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8410102b00 0x7f841019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:37.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.244+0000 7f8414a00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8410108a40 0x7f84101a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:37.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.244+0000 7f840f400640 1 -- 172.21.15.63:0/1956604901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8410108a40 msgr2=0x7f84101a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:37.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.244+0000 7f840f400640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8410108a40 0x7f84101a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.244+0000 7f840f400640 1 -- 172.21.15.63:0/1956604901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8410102b00 msgr2=0x7f841019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:37.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.244+0000 7f840f400640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8410102b00 0x7f841019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.244+0000 7f840f400640 1 -- 172.21.15.63:0/1956604901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84101a1da0 con 0x7f8410103d40 2024-05-27T07:33:37.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.244+0000 7f840f400640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8410103d40 0x7f841019a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f83fc005760 tx=0x7f83fc003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.245+0000 7f840cc00640 1 -- 172.21.15.63:0/1956604901 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc03f070 con 0x7f8410103d40 2024-05-27T07:33:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.245+0000 7f840cc00640 1 -- 172.21.15.63:0/1956604901 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83fc031da0 con 0x7f8410103d40 2024-05-27T07:33:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.245+0000 7f8416a00640 1 -- 172.21.15.63:0/1956604901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84101a2030 con 0x7f8410103d40 2024-05-27T07:33:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.245+0000 7f840cc00640 1 -- 172.21.15.63:0/1956604901 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc035030 con 0x7f8410103d40 2024-05-27T07:33:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.245+0000 7f8416a00640 1 -- 172.21.15.63:0/1956604901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84101a2520 con 0x7f8410103d40 2024-05-27T07:33:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.246+0000 7f8416a00640 1 -- 172.21.15.63:0/1956604901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8410107850 con 0x7f8410103d40 2024-05-27T07:33:37.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.249+0000 7f840cc00640 1 -- 172.21.15.63:0/1956604901 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f83fc042070 con 0x7f8410103d40 2024-05-27T07:33:37.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.250+0000 7f840cc00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83f40b25e0 0x7f83f40b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:37.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.250+0000 7f840cc00640 1 -- 172.21.15.63:0/1956604901 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f83fc103300 con 0x7f8410103d40 2024-05-27T07:33:37.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.250+0000 7f840fe00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83f40b25e0 0x7f83f40b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:37.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.251+0000 7f840fe00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83f40b25e0 0x7f83f40b4af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8410103700 tx=0x7f840000b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:37.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.254+0000 7f840cc00640 1 -- 172.21.15.63:0/1956604901 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83fc0ca830 con 0x7f8410103d40 2024-05-27T07:33:37.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.291+0000 7f840cc00640 1 -- 172.21.15.63:0/1956604901 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7f83fc0c6ac0 con 0x7f8410103d40 2024-05-27T07:33:37.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:37 smithi086 ceph-mon[34687]: [27/May/2024:07:33:36] ENGINE Bus STARTING 2024-05-27T07:33:37.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:37 smithi086 ceph-mon[34687]: [27/May/2024:07:33:36] ENGINE Serving on https://172.21.15.63:7150 2024-05-27T07:33:37.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:37 smithi086 ceph-mon[34687]: mgrmap e22: x(active, since 1.0465s), standbys: z, y 2024-05-27T07:33:37.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:37 smithi086 ceph-mon[34687]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:37.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:37 smithi086 ceph-mon[34687]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:37.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:37 smithi086 ceph-mon[34687]: [27/May/2024:07:33:36] ENGINE Serving on http://172.21.15.63:8765 2024-05-27T07:33:37.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:37 smithi086 ceph-mon[34687]: [27/May/2024:07:33:36] ENGINE Bus STARTED 2024-05-27T07:33:37.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:37 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1675127738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:33:37.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.496+0000 7f8416a00640 1 -- 172.21.15.63:0/1956604901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f84100638c0 con 0x7f8410103d40 2024-05-27T07:33:37.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.497+0000 7f840cc00640 1 -- 172.21.15.63:0/1956604901 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f83fc0bf2b0 con 0x7f8410103d40 2024-05-27T07:33:37.490 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-05-27T07:33:37.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 -- 172.21.15.63:0/1956604901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83f40b25e0 msgr2=0x7f83f40b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:37.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83f40b25e0 0x7f83f40b4af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8410103700 tx=0x7f840000b390 comp rx=0 tx=0).stop 2024-05-27T07:33:37.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 -- 172.21.15.63:0/1956604901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8410103d40 msgr2=0x7f841019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:37.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8410103d40 0x7f841019a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f83fc005760 tx=0x7f83fc003d50 comp rx=0 tx=0).stop 2024-05-27T07:33:37.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 -- 172.21.15.63:0/1956604901 shutdown_connections 2024-05-27T07:33:37.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83f40b25e0 0x7f83f40b4af0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8410108a40 0x7f84101a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8410103d40 0x7f841019a7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 --2- 172.21.15.63:0/1956604901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8410102b00 0x7f841019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 -- 172.21.15.63:0/1956604901 >> 172.21.15.63:0/1956604901 conn(0x7f84100fdcf0 msgr2=0x7f8410104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:37.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.499+0000 7f8416a00640 1 -- 172.21.15.63:0/1956604901 shutdown_connections 2024-05-27T07:33:37.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.500+0000 7f8416a00640 1 -- 172.21.15.63:0/1956604901 wait complete. 2024-05-27T07:33:37.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:37 smithi063 ceph-mon[31613]: [27/May/2024:07:33:36] ENGINE Bus STARTING 2024-05-27T07:33:37.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:37 smithi063 ceph-mon[31613]: [27/May/2024:07:33:36] ENGINE Serving on https://172.21.15.63:7150 2024-05-27T07:33:37.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:37 smithi063 ceph-mon[31613]: mgrmap e22: x(active, since 1.0465s), standbys: z, y 2024-05-27T07:33:37.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:37 smithi063 ceph-mon[31613]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:37.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:37 smithi063 ceph-mon[31613]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:37.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:37 smithi063 ceph-mon[31613]: [27/May/2024:07:33:36] ENGINE Serving on http://172.21.15.63:8765 2024-05-27T07:33:37.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:37 smithi063 ceph-mon[31613]: [27/May/2024:07:33:36] ENGINE Bus STARTED 2024-05-27T07:33:37.517 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:37 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1675127738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:33:37.520 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T07:33:37.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:37 smithi149 ceph-mon[35408]: [27/May/2024:07:33:36] ENGINE Bus STARTING 2024-05-27T07:33:37.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:37 smithi149 ceph-mon[35408]: [27/May/2024:07:33:36] ENGINE Serving on https://172.21.15.63:7150 2024-05-27T07:33:37.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:37 smithi149 ceph-mon[35408]: mgrmap e22: x(active, since 1.0465s), standbys: z, y 2024-05-27T07:33:37.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:37 smithi149 ceph-mon[35408]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:37.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:37 smithi149 ceph-mon[35408]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:37.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:37 smithi149 ceph-mon[35408]: [27/May/2024:07:33:36] ENGINE Serving on http://172.21.15.63:8765 2024-05-27T07:33:37.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:37 smithi149 ceph-mon[35408]: [27/May/2024:07:33:36] ENGINE Bus STARTED 2024-05-27T07:33:37.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:37 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1675127738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:33:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.798+0000 7ff955a00640 1 -- 172.21.15.63:0/493444928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff950075a00 msgr2=0x7ff95010b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.798+0000 7ff955a00640 1 --2- 172.21.15.63:0/493444928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff950075a00 0x7ff95010b630 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009900 tx=0x7ff93c02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.798+0000 7ff955a00640 1 -- 172.21.15.63:0/493444928 shutdown_connections 2024-05-27T07:33:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.798+0000 7ff955a00640 1 --2- 172.21.15.63:0/493444928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff95010bb70 0x7ff95010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.798+0000 7ff955a00640 1 --2- 172.21.15.63:0/493444928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff950075a00 0x7ff95010b630 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.798+0000 7ff955a00640 1 --2- 172.21.15.63:0/493444928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9500750b0 0x7ff9500754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.798+0000 7ff955a00640 1 -- 172.21.15.63:0/493444928 >> 172.21.15.63:0/493444928 conn(0x7ff9500fedf0 msgr2=0x7ff9501012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.798+0000 7ff955a00640 1 -- 172.21.15.63:0/493444928 shutdown_connections 2024-05-27T07:33:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.798+0000 7ff955a00640 1 -- 172.21.15.63:0/493444928 wait complete. 2024-05-27T07:33:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.799+0000 7ff955a00640 1 Processor -- start 2024-05-27T07:33:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff955a00640 1 -- start start 2024-05-27T07:33:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff955a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9500750b0 0x7ff95019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff955a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff950075a00 0x7ff95019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff955a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff95010bb70 0x7ff9501a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff955a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff950111850 con 0x7ff9500750b0 2024-05-27T07:33:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff955a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501116d0 con 0x7ff950075a00 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff955a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501119d0 con 0x7ff95010bb70 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff94ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9500750b0 0x7ff95019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff94ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9500750b0 0x7ff95019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39180/0 (socket says 172.21.15.63:39180) 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff94ea00640 1 -- 172.21.15.63:0/62101710 learned_addr learned my addr 172.21.15.63:0/62101710 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff94e000640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff950075a00 0x7ff95019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.800+0000 7ff94f400640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff95010bb70 0x7ff9501a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.801+0000 7ff94ea00640 1 -- 172.21.15.63:0/62101710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff95010bb70 msgr2=0x7ff9501a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.801+0000 7ff94ea00640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff95010bb70 0x7ff9501a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.801+0000 7ff94ea00640 1 -- 172.21.15.63:0/62101710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff950075a00 msgr2=0x7ff95019c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.801+0000 7ff94ea00640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff950075a00 0x7ff95019c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.801+0000 7ff94ea00640 1 -- 172.21.15.63:0/62101710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9501a3fa0 con 0x7ff9500750b0 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.801+0000 7ff94e000640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff950075a00 0x7ff95019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.801+0000 7ff94ea00640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9500750b0 0x7ff95019c4b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff94000d770 tx=0x7ff94000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.801+0000 7ff947400640 1 -- 172.21.15.63:0/62101710 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940017070 con 0x7ff9500750b0 2024-05-27T07:33:37.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.801+0000 7ff947400640 1 -- 172.21.15.63:0/62101710 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff940009c30 con 0x7ff9500750b0 2024-05-27T07:33:37.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.801+0000 7ff955a00640 1 -- 172.21.15.63:0/62101710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9501a4290 con 0x7ff9500750b0 2024-05-27T07:33:37.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.802+0000 7ff947400640 1 -- 172.21.15.63:0/62101710 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94000f620 con 0x7ff9500750b0 2024-05-27T07:33:37.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.802+0000 7ff955a00640 1 -- 172.21.15.63:0/62101710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9501a47b0 con 0x7ff9500750b0 2024-05-27T07:33:37.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.803+0000 7ff955a00640 1 -- 172.21.15.63:0/62101710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff95010a440 con 0x7ff9500750b0 2024-05-27T07:33:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.805+0000 7ff947400640 1 -- 172.21.15.63:0/62101710 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7ff94001a070 con 0x7ff9500750b0 2024-05-27T07:33:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.807+0000 7ff947400640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9340b29f0 0x7ff9340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.808+0000 7ff947400640 1 -- 172.21.15.63:0/62101710 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7ff9400e25c0 con 0x7ff9500750b0 2024-05-27T07:33:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.808+0000 7ff94e000640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9340b29f0 0x7ff9340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.809+0000 7ff94e000640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9340b29f0 0x7ff9340b4f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff93c005760 tx=0x7ff93c031400 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:37.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:37.811+0000 7ff947400640 1 -- 172.21.15.63:0/62101710 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff940009de0 con 0x7ff9500750b0 2024-05-27T07:33:38.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.018+0000 7ff955a00640 1 -- 172.21.15.63:0/62101710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9500638c0 con 0x7ff9500750b0 2024-05-27T07:33:38.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.019+0000 7ff947400640 1 -- 172.21.15.63:0/62101710 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7ff9400a9410 con 0x7ff9500750b0 2024-05-27T07:33:38.013 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-27T07:33:38.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.021+0000 7ff955a00640 1 -- 172.21.15.63:0/62101710 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9340b29f0 msgr2=0x7ff9340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:38.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.021+0000 7ff955a00640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9340b29f0 0x7ff9340b4f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff93c005760 tx=0x7ff93c031400 comp rx=0 tx=0).stop 2024-05-27T07:33:38.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.021+0000 7ff955a00640 1 -- 172.21.15.63:0/62101710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9500750b0 msgr2=0x7ff95019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:38.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.021+0000 7ff955a00640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9500750b0 0x7ff95019c4b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff94000d770 tx=0x7ff94000dc30 comp rx=0 tx=0).stop 2024-05-27T07:33:38.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.021+0000 7ff955a00640 1 -- 172.21.15.63:0/62101710 shutdown_connections 2024-05-27T07:33:38.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.021+0000 7ff955a00640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9340b29f0 0x7ff9340b4f00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.021+0000 7ff955a00640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff95010bb70 0x7ff9501a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.021+0000 7ff955a00640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff950075a00 0x7ff95019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.021+0000 7ff955a00640 1 --2- 172.21.15.63:0/62101710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9500750b0 0x7ff95019c4b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.021+0000 7ff955a00640 1 -- 172.21.15.63:0/62101710 >> 172.21.15.63:0/62101710 conn(0x7ff9500fedf0 msgr2=0x7ff950107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:38.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.022+0000 7ff955a00640 1 -- 172.21.15.63:0/62101710 shutdown_connections 2024-05-27T07:33:38.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.022+0000 7ff955a00640 1 -- 172.21.15.63:0/62101710 wait complete. 2024-05-27T07:33:38.042 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T07:33:38.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.322+0000 7efd97200640 1 -- 172.21.15.63:0/272884645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd90109b40 msgr2=0x7efd9010bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:38.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.322+0000 7efd97200640 1 --2- 172.21.15.63:0/272884645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd90109b40 0x7efd9010bf60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efd88009ca0 tx=0x7efd8802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.322+0000 7efd97200640 1 -- 172.21.15.63:0/272884645 shutdown_connections 2024-05-27T07:33:38.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.322+0000 7efd97200640 1 --2- 172.21.15.63:0/272884645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd90109b40 0x7efd9010bf60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.322+0000 7efd97200640 1 --2- 172.21.15.63:0/272884645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90104e40 0x7efd90105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.322+0000 7efd97200640 1 --2- 172.21.15.63:0/272884645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd90103c00 0x7efd90104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.322+0000 7efd97200640 1 -- 172.21.15.63:0/272884645 >> 172.21.15.63:0/272884645 conn(0x7efd900fedf0 msgr2=0x7efd901012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:38.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.322+0000 7efd97200640 1 -- 172.21.15.63:0/272884645 shutdown_connections 2024-05-27T07:33:38.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.322+0000 7efd97200640 1 -- 172.21.15.63:0/272884645 wait complete. 2024-05-27T07:33:38.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd97200640 1 Processor -- start 2024-05-27T07:33:38.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd97200640 1 -- start start 2024-05-27T07:33:38.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd97200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90103c00 0x7efd9019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:38.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd97200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd90104e40 0x7efd9019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:38.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd97200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd90109b40 0x7efd901a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:38.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd97200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd9010f610 con 0x7efd90109b40 2024-05-27T07:33:38.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd97200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd9010f490 con 0x7efd90103c00 2024-05-27T07:33:38.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd97200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efd9010f790 con 0x7efd90104e40 2024-05-27T07:33:38.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd95200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd90109b40 0x7efd901a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:38.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd95200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd90109b40 0x7efd901a38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39204/0 (socket says 172.21.15.63:39204) 2024-05-27T07:33:38.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd95200640 1 -- 172.21.15.63:0/3427580653 learned_addr learned my addr 172.21.15.63:0/3427580653 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:38.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.324+0000 7efd87e00640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd90104e40 0x7efd9019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:38.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.325+0000 7efd8fe00640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90103c00 0x7efd9019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:38.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.325+0000 7efd95200640 1 -- 172.21.15.63:0/3427580653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd90104e40 msgr2=0x7efd9019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:38.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.325+0000 7efd95200640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd90104e40 0x7efd9019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.325+0000 7efd95200640 1 -- 172.21.15.63:0/3427580653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90103c00 msgr2=0x7efd9019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:38.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.325+0000 7efd95200640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90103c00 0x7efd9019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.325+0000 7efd95200640 1 -- 172.21.15.63:0/3427580653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd901a3f40 con 0x7efd90109b40 2024-05-27T07:33:38.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.325+0000 7efd95200640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd90109b40 0x7efd901a38d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efd88031040 tx=0x7efd88003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:38.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.326+0000 7efd8d600640 1 -- 172.21.15.63:0/3427580653 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88003d60 con 0x7efd90109b40 2024-05-27T07:33:38.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.326+0000 7efd8d600640 1 -- 172.21.15.63:0/3427580653 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd880044d0 con 0x7efd90109b40 2024-05-27T07:33:38.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.326+0000 7efd97200640 1 -- 172.21.15.63:0/3427580653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd901a41d0 con 0x7efd90109b40 2024-05-27T07:33:38.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.326+0000 7efd8d600640 1 -- 172.21.15.63:0/3427580653 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88030c50 con 0x7efd90109b40 2024-05-27T07:33:38.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.326+0000 7efd97200640 1 -- 172.21.15.63:0/3427580653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd901a4720 con 0x7efd90109b40 2024-05-27T07:33:38.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.327+0000 7efd97200640 1 -- 172.21.15.63:0/3427580653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd901a4980 con 0x7efd90109b40 2024-05-27T07:33:38.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.331+0000 7efd8d600640 1 -- 172.21.15.63:0/3427580653 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7efd88004040 con 0x7efd90109b40 2024-05-27T07:33:38.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.334+0000 7efd8d600640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd740b29f0 0x7efd740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:38.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.334+0000 7efd8d600640 1 -- 172.21.15.63:0/3427580653 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7efd88103fd0 con 0x7efd90109b40 2024-05-27T07:33:38.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.335+0000 7efd8fe00640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd740b29f0 0x7efd740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:38.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.335+0000 7efd8fe00640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd740b29f0 0x7efd740b4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd90104800 tx=0x7efd80005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:38.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.336+0000 7efd8d600640 1 -- 172.21.15.63:0/3427580653 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd880cb580 con 0x7efd90109b40 2024-05-27T07:33:38.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:38 smithi063 ceph-mon[31613]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:38.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:38 smithi063 ceph-mon[31613]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T07:33:38.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:38 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1956604901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:33:38.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:38 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/62101710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:33:38.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.571+0000 7efd97200640 1 -- 172.21.15.63:0/3427580653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd901a4b50 con 0x7efd90109b40 2024-05-27T07:33:38.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.571+0000 7efd8d600640 1 -- 172.21.15.63:0/3427580653 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7efd880cae80 con 0x7efd90109b40 2024-05-27T07:33:38.565 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 64 2024-05-27T07:33:38.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 -- 172.21.15.63:0/3427580653 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd740b29f0 msgr2=0x7efd740b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:38.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd740b29f0 0x7efd740b4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd90104800 tx=0x7efd80005f90 comp rx=0 tx=0).stop 2024-05-27T07:33:38.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 -- 172.21.15.63:0/3427580653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd90109b40 msgr2=0x7efd901a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:38.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd90109b40 0x7efd901a38d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efd88031040 tx=0x7efd88003970 comp rx=0 tx=0).stop 2024-05-27T07:33:38.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 -- 172.21.15.63:0/3427580653 shutdown_connections 2024-05-27T07:33:38.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd740b29f0 0x7efd740b4f00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd90109b40 0x7efd901a38d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd90104e40 0x7efd9019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 --2- 172.21.15.63:0/3427580653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90103c00 0x7efd9019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 -- 172.21.15.63:0/3427580653 >> 172.21.15.63:0/3427580653 conn(0x7efd900fedf0 msgr2=0x7efd900ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:38.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 -- 172.21.15.63:0/3427580653 shutdown_connections 2024-05-27T07:33:38.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.574+0000 7efd97200640 1 -- 172.21.15.63:0/3427580653 wait complete. 2024-05-27T07:33:38.594 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T07:33:38.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:38 smithi086 ceph-mon[34687]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:38.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:38 smithi086 ceph-mon[34687]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T07:33:38.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:38 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1956604901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:33:38.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:38 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/62101710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:33:38.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:38 smithi149 ceph-mon[35408]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:38.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:38 smithi149 ceph-mon[35408]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T07:33:38.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:38 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1956604901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:33:38.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:38 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/62101710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:33:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.872+0000 7f08a7200640 1 -- 172.21.15.63:0/2918016078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a81096b0 msgr2=0x7f08a810bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.872+0000 7f08a7200640 1 --2- 172.21.15.63:0/2918016078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a81096b0 0x7f08a810bbc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009ca0 tx=0x7f08a002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.872+0000 7f08a7200640 1 -- 172.21.15.63:0/2918016078 shutdown_connections 2024-05-27T07:33:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.872+0000 7f08a7200640 1 --2- 172.21.15.63:0/2918016078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a81096b0 0x7f08a810bbc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.872+0000 7f08a7200640 1 --2- 172.21.15.63:0/2918016078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8106b40 0x7f08a8109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.872+0000 7f08a7200640 1 --2- 172.21.15.63:0/2918016078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08a8104150 0x7f08a8106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.872+0000 7f08a7200640 1 -- 172.21.15.63:0/2918016078 >> 172.21.15.63:0/2918016078 conn(0x7f08a80fdb10 msgr2=0x7f08a80fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:38.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.872+0000 7f08a7200640 1 -- 172.21.15.63:0/2918016078 shutdown_connections 2024-05-27T07:33:38.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.872+0000 7f08a7200640 1 -- 172.21.15.63:0/2918016078 wait complete. 2024-05-27T07:33:38.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.874+0000 7f08a7200640 1 Processor -- start 2024-05-27T07:33:38.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.874+0000 7f08a7200640 1 -- start start 2024-05-27T07:33:38.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.874+0000 7f08a7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8104150 0x7f08a81039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:38.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.874+0000 7f08a7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a8106b40 0x7f08a8102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:38.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.874+0000 7f08a7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08a81096b0 0x7f08a81025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:38.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.874+0000 7f08a7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810f420 con 0x7f08a8106b40 2024-05-27T07:33:38.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.874+0000 7f08a7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810f2a0 con 0x7f08a8104150 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.874+0000 7f08a7200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810f5a0 con 0x7f08a81096b0 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a8106b40 0x7f08a8102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a8106b40 0x7f08a8102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39222/0 (socket says 172.21.15.63:39222) 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5400640 1 -- 172.21.15.63:0/3169296456 learned_addr learned my addr 172.21.15.63:0/3169296456 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5e00640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8104150 0x7f08a81039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a6800640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08a81096b0 0x7f08a81025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5400640 1 -- 172.21.15.63:0/3169296456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08a81096b0 msgr2=0x7f08a81025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5400640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08a81096b0 0x7f08a81025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5400640 1 -- 172.21.15.63:0/3169296456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8104150 msgr2=0x7f08a81039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5400640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8104150 0x7f08a81039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5400640 1 -- 172.21.15.63:0/3169296456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a8102cd0 con 0x7f08a8106b40 2024-05-27T07:33:38.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5e00640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8104150 0x7f08a81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:33:38.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.875+0000 7f08a5400640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a8106b40 0x7f08a8102060 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f089400d950 tx=0x7f089400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:38.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.876+0000 7f089ea00640 1 -- 172.21.15.63:0/3169296456 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894017030 con 0x7f08a8106b40 2024-05-27T07:33:38.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.876+0000 7f089ea00640 1 -- 172.21.15.63:0/3169296456 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0894004020 con 0x7f08a8106b40 2024-05-27T07:33:38.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.876+0000 7f08a7200640 1 -- 172.21.15.63:0/3169296456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a81a40c0 con 0x7f08a8106b40 2024-05-27T07:33:38.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.876+0000 7f089ea00640 1 -- 172.21.15.63:0/3169296456 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08940055d0 con 0x7f08a8106b40 2024-05-27T07:33:38.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.876+0000 7f08a7200640 1 -- 172.21.15.63:0/3169296456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a81a4580 con 0x7f08a8106b40 2024-05-27T07:33:38.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.878+0000 7f08a7200640 1 -- 172.21.15.63:0/3169296456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a80ff630 con 0x7f08a8106b40 2024-05-27T07:33:38.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.879+0000 7f089ea00640 1 -- 172.21.15.63:0/3169296456 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7f0894009a30 con 0x7f08a8106b40 2024-05-27T07:33:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.881+0000 7f089ea00640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f088c0b29e0 0x7f088c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:38.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.881+0000 7f089ea00640 1 -- 172.21.15.63:0/3169296456 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f08940e24c0 con 0x7f08a8106b40 2024-05-27T07:33:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.881+0000 7f08a5e00640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f088c0b29e0 0x7f088c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:38.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.882+0000 7f08a5e00640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f088c0b29e0 0x7f088c0b4ef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08a806a970 tx=0x7f0898005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:38.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:38.886+0000 7f089ea00640 1 -- 172.21.15.63:0/3169296456 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f08940a9960 con 0x7f08a8106b40 2024-05-27T07:33:39.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.096+0000 7f08a7200640 1 -- 172.21.15.63:0/3169296456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f08a802ddb0 con 0x7f08a8106b40 2024-05-27T07:33:39.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.097+0000 7f089ea00640 1 -- 172.21.15.63:0/3169296456 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f08a802ddb0 con 0x7f08a8106b40 2024-05-27T07:33:39.091 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-27T07:33:39.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 -- 172.21.15.63:0/3169296456 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f088c0b29e0 msgr2=0x7f088c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:39.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f088c0b29e0 0x7f088c0b4ef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f08a806a970 tx=0x7f0898005f90 comp rx=0 tx=0).stop 2024-05-27T07:33:39.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 -- 172.21.15.63:0/3169296456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a8106b40 msgr2=0x7f08a8102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:39.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a8106b40 0x7f08a8102060 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f089400d950 tx=0x7f089400de10 comp rx=0 tx=0).stop 2024-05-27T07:33:39.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 -- 172.21.15.63:0/3169296456 shutdown_connections 2024-05-27T07:33:39.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f088c0b29e0 0x7f088c0b4ef0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:39.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08a81096b0 0x7f08a81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:39.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a8106b40 0x7f08a8102060 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:39.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 --2- 172.21.15.63:0/3169296456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08a8104150 0x7f08a81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:39.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 -- 172.21.15.63:0/3169296456 >> 172.21.15.63:0/3169296456 conn(0x7f08a80fdb10 msgr2=0x7f08a8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:39.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 -- 172.21.15.63:0/3169296456 shutdown_connections 2024-05-27T07:33:39.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:39.100+0000 7f08a7200640 1 -- 172.21.15.63:0/3169296456 wait complete. 2024-05-27T07:33:39.121 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-27T07:33:39.122 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:33:39.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:39 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3427580653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:33:39.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:39 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:39 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:39 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:39 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:39 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:39 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:39 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3169296456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:33:39.579 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:39.702 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:39 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3427580653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:33:39.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:39 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:39 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:39 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:39 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:39 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:39 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:39 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3169296456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:33:39.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:39 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3427580653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:33:39.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:39 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:39 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:39 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:39 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:39 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:39 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:39.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:39 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3169296456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:33:40.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:40 smithi063 ceph-mon[31613]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:40.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:40 smithi063 ceph-mon[31613]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T07:33:40.702 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:40 smithi086 ceph-mon[34687]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:40.702 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:40 smithi086 ceph-mon[34687]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T07:33:40.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:40 smithi149 ceph-mon[35408]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:40.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:40 smithi149 ceph-mon[35408]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T07:33:42.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.183+0000 7f81ec600700 1 -- 172.21.15.63:0/848300421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e41045e0 msgr2=0x7f81e41049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:42.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.183+0000 7f81ec600700 1 --2- 172.21.15.63:0/848300421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e41045e0 0x7f81e41049f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009920 tx=0x7f81d8009c30 comp rx=0 tx=0).stop 2024-05-27T07:33:42.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.183+0000 7f81ec600700 1 -- 172.21.15.63:0/848300421 shutdown_connections 2024-05-27T07:33:42.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.183+0000 7f81ec600700 1 --2- 172.21.15.63:0/848300421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81e4109860 0x7f81e41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:42.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.183+0000 7f81ec600700 1 --2- 172.21.15.63:0/848300421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e41045e0 0x7f81e41049f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:42.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.183+0000 7f81ec600700 1 --2- 172.21.15.63:0/848300421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81e4108f10 0x7f81e4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:42.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.183+0000 7f81ec600700 1 -- 172.21.15.63:0/848300421 >> 172.21.15.63:0/848300421 conn(0x7f81e4100030 msgr2=0x7f81e4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:42.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.183+0000 7f81ec600700 1 -- 172.21.15.63:0/848300421 shutdown_connections 2024-05-27T07:33:42.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.183+0000 7f81ec600700 1 -- 172.21.15.63:0/848300421 wait complete. 2024-05-27T07:33:42.178 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.178 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.178 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.178 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.180 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.180 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.180 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:42.180 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:41 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.185+0000 7f81ec600700 1 Processor -- start 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.185+0000 7f81ec600700 1 -- start start 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.186+0000 7f81ec600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e41045e0 0x7f81e419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.186+0000 7f81ec600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81e4108f10 0x7f81e419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.186+0000 7f81ec600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81e4109860 0x7f81e41a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.186+0000 7f81ec600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81e4114340 con 0x7f81e41045e0 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.186+0000 7f81ec600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81e41141c0 con 0x7f81e4109860 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.186+0000 7f81ec600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f81e41144c0 con 0x7f81e4108f10 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.186+0000 7f81e9c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e41045e0 0x7f81e419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.186+0000 7f81e9c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e41045e0 0x7f81e419d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42336/0 (socket says 172.21.15.63:42336) 2024-05-27T07:33:42.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.187+0000 7f81e9c00700 1 -- 172.21.15.63:0/715130912 learned_addr learned my addr 172.21.15.63:0/715130912 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:42.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.187+0000 7f81ea600700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81e4109860 0x7f81e41a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:42.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.187+0000 7f81e9200700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81e4108f10 0x7f81e419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:42.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.187+0000 7f81ea600700 1 -- 172.21.15.63:0/715130912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81e4108f10 msgr2=0x7f81e419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:42.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.187+0000 7f81ea600700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81e4108f10 0x7f81e419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:42.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.187+0000 7f81ea600700 1 -- 172.21.15.63:0/715130912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e41045e0 msgr2=0x7f81e419d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:42.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.187+0000 7f81ea600700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e41045e0 0x7f81e419d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:42.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.187+0000 7f81ea600700 1 -- 172.21.15.63:0/715130912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e41a1fa0 con 0x7f81e4109860 2024-05-27T07:33:42.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.188+0000 7f81ea600700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81e4109860 0x7f81e41a1900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009cd0 tx=0x7f81dc00c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:42.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.188+0000 7f81e2000700 1 -- 172.21.15.63:0/715130912 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00cd10 con 0x7f81e4109860 2024-05-27T07:33:42.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.188+0000 7f81e2000700 1 -- 172.21.15.63:0/715130912 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f81dc003a00 con 0x7f81e4109860 2024-05-27T07:33:42.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.188+0000 7f81ec600700 1 -- 172.21.15.63:0/715130912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e41a2230 con 0x7f81e4109860 2024-05-27T07:33:42.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.188+0000 7f81e2000700 1 -- 172.21.15.63:0/715130912 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0043e0 con 0x7f81e4109860 2024-05-27T07:33:42.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.188+0000 7f81ec600700 1 -- 172.21.15.63:0/715130912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e41a2720 con 0x7f81e4109860 2024-05-27T07:33:42.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.190+0000 7f81ec600700 1 -- 172.21.15.63:0/715130912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c4005330 con 0x7f81e4109860 2024-05-27T07:33:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.193+0000 7f81e2000700 1 -- 172.21.15.63:0/715130912 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f81dc01a020 con 0x7f81e4109860 2024-05-27T07:33:42.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.194+0000 7f81e2000700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f81cc0b2b10 0x7f81cc0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:42.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.194+0000 7f81e2000700 1 -- 172.21.15.63:0/715130912 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f81dc0e2890 con 0x7f81e4109860 2024-05-27T07:33:42.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.195+0000 7f81e9c00700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f81cc0b2b10 0x7f81cc0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:42.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.195+0000 7f81e9c00700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f81cc0b2b10 0x7f81cc0b5020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009680 tx=0x7f81d8019040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:42.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.200+0000 7f81e2000700 1 -- 172.21.15.63:0/715130912 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81dc010040 con 0x7f81e4109860 2024-05-27T07:33:42.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:42.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:41 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:42.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:42.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.208 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.208 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.208 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:42.208 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:42.208 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:41 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:42.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.503+0000 7f81ec600700 1 -- 172.21.15.63:0/715130912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81c4005620 con 0x7f81e4109860 2024-05-27T07:33:42.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.504+0000 7f81e2000700 1 -- 172.21.15.63:0/715130912 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11861 (secure 0 0 0) 0x7f81dc0a9c50 con 0x7f81e4109860 2024-05-27T07:33:42.498 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:33:42.498 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:33:33.871810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:33:42.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.507+0000 7f81ec600700 1 -- 172.21.15.63:0/715130912 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f81cc0b2b10 msgr2=0x7f81cc0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:42.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.507+0000 7f81ec600700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f81cc0b2b10 0x7f81cc0b5020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009680 tx=0x7f81d8019040 comp rx=0 tx=0).stop 2024-05-27T07:33:42.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.507+0000 7f81ec600700 1 -- 172.21.15.63:0/715130912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81e4109860 msgr2=0x7f81e41a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:42.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.507+0000 7f81ec600700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81e4109860 0x7f81e41a1900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009cd0 tx=0x7f81dc00c620 comp rx=0 tx=0).stop 2024-05-27T07:33:42.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.507+0000 7f81ec600700 1 -- 172.21.15.63:0/715130912 shutdown_connections 2024-05-27T07:33:42.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.507+0000 7f81ec600700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f81cc0b2b10 0x7f81cc0b5020 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:42.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.507+0000 7f81ec600700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81e4109860 0x7f81e41a1900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:42.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.507+0000 7f81ec600700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e41045e0 0x7f81e419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:42.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.507+0000 7f81ec600700 1 --2- 172.21.15.63:0/715130912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81e4108f10 0x7f81e419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:42.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.507+0000 7f81ec600700 1 -- 172.21.15.63:0/715130912 >> 172.21.15.63:0/715130912 conn(0x7f81e4100030 msgr2=0x7f81e410a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:42.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.508+0000 7f81ec600700 1 -- 172.21.15.63:0/715130912 shutdown_connections 2024-05-27T07:33:42.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:42.508+0000 7f81ec600700 1 -- 172.21.15.63:0/715130912 wait complete. 2024-05-27T07:33:42.503 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 27 2024-05-27T07:33:43.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:42 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T07:33:43.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:42 smithi086 ceph-mon[34687]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T07:33:43.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:42 smithi086 ceph-mon[34687]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-27T07:33:43.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:42 smithi086 ceph-mon[34687]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-27T07:33:43.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:42 smithi086 ceph-mon[34687]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-27T07:33:43.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:42 smithi086 ceph-mon[34687]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:33:43.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:42 smithi086 ceph-mon[34687]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:33:43.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:42 smithi086 ceph-mon[34687]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:33:43.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:42 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/715130912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:43.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:42 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T07:33:43.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:42 smithi149 ceph-mon[35408]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T07:33:43.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:42 smithi149 ceph-mon[35408]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-27T07:33:43.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:42 smithi149 ceph-mon[35408]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-27T07:33:43.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:42 smithi149 ceph-mon[35408]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-27T07:33:43.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:42 smithi149 ceph-mon[35408]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:33:43.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:42 smithi149 ceph-mon[35408]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:33:43.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:42 smithi149 ceph-mon[35408]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:33:43.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:42 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/715130912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:43.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:42 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T07:33:43.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:42 smithi063 ceph-mon[31613]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T07:33:43.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:42 smithi063 ceph-mon[31613]: Updating smithi063:/etc/ceph/ceph.conf 2024-05-27T07:33:43.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:42 smithi063 ceph-mon[31613]: Updating smithi086:/etc/ceph/ceph.conf 2024-05-27T07:33:43.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:42 smithi063 ceph-mon[31613]: Updating smithi149:/etc/ceph/ceph.conf 2024-05-27T07:33:43.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:42 smithi063 ceph-mon[31613]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:33:43.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:42 smithi063 ceph-mon[31613]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:33:43.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:42 smithi063 ceph-mon[31613]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:33:43.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:42 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/715130912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:43.429 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-27T07:33:43.429 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-27T07:33:43.429 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T07:33:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.701+0000 7f37d2000640 1 -- 172.21.15.63:0/1368180018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37cc102930 msgr2=0x7f37cc102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.701+0000 7f37d2000640 1 --2- 172.21.15.63:0/1368180018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37cc102930 0x7f37cc102d40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009900 tx=0x7f37bc02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.701+0000 7f37d2000640 1 -- 172.21.15.63:0/1368180018 shutdown_connections 2024-05-27T07:33:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.701+0000 7f37d2000640 1 --2- 172.21.15.63:0/1368180018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc1104d0 0x7f37cc1128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.701+0000 7f37d2000640 1 --2- 172.21.15.63:0/1368180018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37cc10db70 0x7f37cc10ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.701+0000 7f37d2000640 1 --2- 172.21.15.63:0/1368180018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37cc102930 0x7f37cc102d40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.701+0000 7f37d2000640 1 -- 172.21.15.63:0/1368180018 >> 172.21.15.63:0/1368180018 conn(0x7f37cc078260 msgr2=0x7f37cc0786a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:43.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.701+0000 7f37d2000640 1 -- 172.21.15.63:0/1368180018 shutdown_connections 2024-05-27T07:33:43.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.701+0000 7f37d2000640 1 -- 172.21.15.63:0/1368180018 wait complete. 2024-05-27T07:33:43.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.702+0000 7f37d2000640 1 Processor -- start 2024-05-27T07:33:43.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.702+0000 7f37d2000640 1 -- start start 2024-05-27T07:33:43.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.703+0000 7f37d2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37cc102930 0x7f37cc1a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:43.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.703+0000 7f37d2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37cc10db70 0x7f37cc1a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:43.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.703+0000 7f37d2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc1104d0 0x7f37cc19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:43.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.703+0000 7f37d2000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc115eb0 con 0x7f37cc102930 2024-05-27T07:33:43.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.703+0000 7f37d2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc115d30 con 0x7f37cc1104d0 2024-05-27T07:33:43.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.703+0000 7f37d2000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc116030 con 0x7f37cc10db70 2024-05-27T07:33:43.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.703+0000 7f37cb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37cc102930 0x7f37cc1a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:43.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.703+0000 7f37cb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37cc102930 0x7f37cc1a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42358/0 (socket says 172.21.15.63:42358) 2024-05-27T07:33:43.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.703+0000 7f37cb400640 1 -- 172.21.15.63:0/2169039704 learned_addr learned my addr 172.21.15.63:0/2169039704 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:43.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.704+0000 7f37caa00640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37cc10db70 0x7f37cc1a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:43.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.704+0000 7f37cbe00640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc1104d0 0x7f37cc19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:43.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.704+0000 7f37cb400640 1 -- 172.21.15.63:0/2169039704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37cc10db70 msgr2=0x7f37cc1a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:43.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.704+0000 7f37cb400640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37cc10db70 0x7f37cc1a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.704+0000 7f37cb400640 1 -- 172.21.15.63:0/2169039704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc1104d0 msgr2=0x7f37cc19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:43.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.704+0000 7f37cb400640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc1104d0 0x7f37cc19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.704+0000 7f37cb400640 1 -- 172.21.15.63:0/2169039704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc19cdb0 con 0x7f37cc102930 2024-05-27T07:33:43.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.704+0000 7f37cbe00640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc1104d0 0x7f37cc19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:43.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.704+0000 7f37caa00640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37cc10db70 0x7f37cc1a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:43.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.704+0000 7f37cb400640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37cc102930 0x7f37cc1a25f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f37bc02a910 tx=0x7f37bc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:43.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.705+0000 7f37c3e00640 1 -- 172.21.15.63:0/2169039704 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc03f070 con 0x7f37cc102930 2024-05-27T07:33:43.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.705+0000 7f37d2000640 1 -- 172.21.15.63:0/2169039704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc19d040 con 0x7f37cc102930 2024-05-27T07:33:43.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.705+0000 7f37c3e00640 1 -- 172.21.15.63:0/2169039704 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37bc003e40 con 0x7f37cc102930 2024-05-27T07:33:43.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.705+0000 7f37c3e00640 1 -- 172.21.15.63:0/2169039704 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc035030 con 0x7f37cc102930 2024-05-27T07:33:43.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.705+0000 7f37d2000640 1 -- 172.21.15.63:0/2169039704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc19d530 con 0x7f37cc102930 2024-05-27T07:33:43.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.706+0000 7f37d2000640 1 -- 172.21.15.63:0/2169039704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37cc0757d0 con 0x7f37cc102930 2024-05-27T07:33:43.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.709+0000 7f37c3e00640 1 -- 172.21.15.63:0/2169039704 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f37bc042070 con 0x7f37cc102930 2024-05-27T07:33:43.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.710+0000 7f37c3e00640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37b00b2a80 0x7f37b00b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:43.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.711+0000 7f37c3e00640 1 -- 172.21.15.63:0/2169039704 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f37bc1038c0 con 0x7f37cc102930 2024-05-27T07:33:43.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.711+0000 7f37caa00640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37b00b2a80 0x7f37b00b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:43.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.712+0000 7f37caa00640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37b00b2a80 0x7f37b00b4f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f37cc19df10 tx=0x7f37b800a5e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:43.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.713+0000 7f37c3e00640 1 -- 172.21.15.63:0/2169039704 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37bc0cad50 con 0x7f37cc102930 2024-05-27T07:33:43.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.955+0000 7f37d2000640 1 -- 172.21.15.63:0/2169039704 --> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37cc1018f0 con 0x7f37b00b2a80 2024-05-27T07:33:43.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.972+0000 7f37c3e00640 1 -- 172.21.15.63:0/2169039704 <== mgr.15216 v2:172.21.15.63:6800/1343315489 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339430 (secure 0 0 0) 0x7f37cc1018f0 con 0x7f37b00b2a80 2024-05-27T07:33:43.973 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:33:43.976 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-27T07:33:43.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 -- 172.21.15.63:0/2169039704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37b00b2a80 msgr2=0x7f37b00b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:43.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37b00b2a80 0x7f37b00b4f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f37cc19df10 tx=0x7f37b800a5e0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 -- 172.21.15.63:0/2169039704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37cc102930 msgr2=0x7f37cc1a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:43.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37cc102930 0x7f37cc1a25f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f37bc02a910 tx=0x7f37bc003c20 comp rx=0 tx=0).stop 2024-05-27T07:33:43.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 -- 172.21.15.63:0/2169039704 shutdown_connections 2024-05-27T07:33:43.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37b00b2a80 0x7f37b00b4f90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37cc1104d0 0x7f37cc19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37cc10db70 0x7f37cc1a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 --2- 172.21.15.63:0/2169039704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37cc102930 0x7f37cc1a25f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:43.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 -- 172.21.15.63:0/2169039704 >> 172.21.15.63:0/2169039704 conn(0x7f37cc078260 msgr2=0x7f37cc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:43.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 -- 172.21.15.63:0/2169039704 shutdown_connections 2024-05-27T07:33:43.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:43.978+0000 7f37d2000640 1 -- 172.21.15.63:0/2169039704 wait complete. 2024-05-27T07:33:44.000 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T07:33:43.267988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1198502,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1640,"num_read_kb":1114,"num_write":441,"num_write_kb":3607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377776,"kb_used_data":16112,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124564048,"statfs":{"total":1151940427776,"available":1151553585152,"internally_reserved":0,"allocated":16498688,"data_stored":9952352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3302,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":201,"num_write":17,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.976226"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385739+0000","last_change":"2024-05-27T07:28:24.485151+0000","last_active":"2024-05-27T07:33:35.385739+0000","last_peered":"2024-05-27T07:33:35.385739+0000","last_clean":"2024-05-27T07:33:35.385739+0000","last_became_active":"2024-05-27T07:28:24.483965+0000","last_became_peered":"2024-05-27T07:28:24.483965+0000","last_unstale":"2024-05-27T07:33:35.385739+0000","last_undegraded":"2024-05-27T07:33:35.385739+0000","last_fullsized":"2024-05-27T07:33:35.385739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:18.376755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431746+0000","last_change":"2024-05-27T07:28:24.483608+0000","last_active":"2024-05-27T07:33:35.431746+0000","last_peered":"2024-05-27T07:33:35.431746+0000","last_clean":"2024-05-27T07:33:35.431746+0000","last_became_active":"2024-05-27T07:28:24.479875+0000","last_became_peered":"2024-05-27T07:28:24.479875+0000","last_unstale":"2024-05-27T07:33:35.431746+0000","last_undegraded":"2024-05-27T07:33:35.431746+0000","last_fullsized":"2024-05-27T07:33:35.431746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:30.129708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431562+0000","last_change":"2024-05-27T07:28:24.470104+0000","last_active":"2024-05-27T07:33:35.431562+0000","last_peered":"2024-05-27T07:33:35.431562+0000","last_clean":"2024-05-27T07:33:35.431562+0000","last_became_active":"2024-05-27T07:28:24.469756+0000","last_became_peered":"2024-05-27T07:28:24.469756+0000","last_unstale":"2024-05-27T07:33:35.431562+0000","last_undegraded":"2024-05-27T07:33:35.431562+0000","last_fullsized":"2024-05-27T07:33:35.431562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:36.907691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243760+0000","last_change":"2024-05-27T07:28:24.472381+0000","last_active":"2024-05-27T07:33:35.243760+0000","last_peered":"2024-05-27T07:33:35.243760+0000","last_clean":"2024-05-27T07:33:35.243760+0000","last_became_active":"2024-05-27T07:28:24.469959+0000","last_became_peered":"2024-05-27T07:28:24.469959+0000","last_unstale":"2024-05-27T07:33:35.243760+0000","last_undegraded":"2024-05-27T07:33:35.243760+0000","last_fullsized":"2024-05-27T07:33:35.243760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:15.267554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434585+0000","last_change":"2024-05-27T07:28:24.474210+0000","last_active":"2024-05-27T07:33:35.434585+0000","last_peered":"2024-05-27T07:33:35.434585+0000","last_clean":"2024-05-27T07:33:35.434585+0000","last_became_active":"2024-05-27T07:28:24.473688+0000","last_became_peered":"2024-05-27T07:28:24.473688+0000","last_unstale":"2024-05-27T07:33:35.434585+0000","last_undegraded":"2024-05-27T07:33:35.434585+0000","last_fullsized":"2024-05-27T07:33:35.434585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:38.733536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401818+0000","last_change":"2024-05-27T07:28:24.484679+0000","last_active":"2024-05-27T07:33:35.401818+0000","last_peered":"2024-05-27T07:33:35.401818+0000","last_clean":"2024-05-27T07:33:35.401818+0000","last_became_active":"2024-05-27T07:28:24.484251+0000","last_became_peered":"2024-05-27T07:28:24.484251+0000","last_unstale":"2024-05-27T07:33:35.401818+0000","last_undegraded":"2024-05-27T07:33:35.401818+0000","last_fullsized":"2024-05-27T07:33:35.401818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:28:09.507871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434697+0000","last_change":"2024-05-27T07:28:24.472694+0000","last_active":"2024-05-27T07:33:35.434697+0000","last_peered":"2024-05-27T07:33:35.434697+0000","last_clean":"2024-05-27T07:33:35.434697+0000","last_became_active":"2024-05-27T07:28:24.472100+0000","last_became_peered":"2024-05-27T07:28:24.472100+0000","last_unstale":"2024-05-27T07:33:35.434697+0000","last_undegraded":"2024-05-27T07:33:35.434697+0000","last_fullsized":"2024-05-27T07:33:35.434697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336904+0000","last_change":"2024-05-27T07:28:24.488577+0000","last_active":"2024-05-27T07:33:35.336904+0000","last_peered":"2024-05-27T07:33:35.336904+0000","last_clean":"2024-05-27T07:33:35.336904+0000","last_became_active":"2024-05-27T07:28:24.488180+0000","last_became_peered":"2024-05-27T07:28:24.488180+0000","last_unstale":"2024-05-27T07:33:35.336904+0000","last_undegraded":"2024-05-27T07:33:35.336904+0000","last_fullsized":"2024-05-27T07:33:35.336904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:17.130058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437122+0000","last_change":"2024-05-27T07:28:24.480109+0000","last_active":"2024-05-27T07:33:35.437122+0000","last_peered":"2024-05-27T07:33:35.437122+0000","last_clean":"2024-05-27T07:33:35.437122+0000","last_became_active":"2024-05-27T07:28:24.479722+0000","last_became_peered":"2024-05-27T07:28:24.479722+0000","last_unstale":"2024-05-27T07:33:35.437122+0000","last_undegraded":"2024-05-27T07:33:35.437122+0000","last_fullsized":"2024-05-27T07:33:35.437122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:27:36.074612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.402840+0000","last_change":"2024-05-27T07:28:24.490389+0000","last_active":"2024-05-27T07:33:35.402840+0000","last_peered":"2024-05-27T07:33:35.402840+0000","last_clean":"2024-05-27T07:33:35.402840+0000","last_became_active":"2024-05-27T07:28:24.488081+0000","last_became_peered":"2024-05-27T07:28:24.488081+0000","last_unstale":"2024-05-27T07:33:35.402840+0000","last_undegraded":"2024-05-27T07:33:35.402840+0000","last_fullsized":"2024-05-27T07:33:35.402840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:06.318235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436339+0000","last_change":"2024-05-27T07:28:24.477062+0000","last_active":"2024-05-27T07:33:35.436339+0000","last_peered":"2024-05-27T07:33:35.436339+0000","last_clean":"2024-05-27T07:33:35.436339+0000","last_became_active":"2024-05-27T07:28:24.475322+0000","last_became_peered":"2024-05-27T07:28:24.475322+0000","last_unstale":"2024-05-27T07:33:35.436339+0000","last_undegraded":"2024-05-27T07:33:35.436339+0000","last_fullsized":"2024-05-27T07:33:35.436339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:30.365295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.238628+0000","last_change":"2024-05-27T07:29:04.901715+0000","last_active":"2024-05-27T07:33:35.238628+0000","last_peered":"2024-05-27T07:33:35.238628+0000","last_clean":"2024-05-27T07:33:35.238628+0000","last_became_active":"2024-05-27T07:29:04.900968+0000","last_became_peered":"2024-05-27T07:29:04.900968+0000","last_unstale":"2024-05-27T07:33:35.238628+0000","last_undegraded":"2024-05-27T07:33:35.238628+0000","last_fullsized":"2024-05-27T07:33:35.238628+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:16:34.033701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370124+0000","last_change":"2024-05-27T07:28:24.477504+0000","last_active":"2024-05-27T07:33:35.370124+0000","last_peered":"2024-05-27T07:33:35.370124+0000","last_clean":"2024-05-27T07:33:35.370124+0000","last_became_active":"2024-05-27T07:28:24.476709+0000","last_became_peered":"2024-05-27T07:28:24.476709+0000","last_unstale":"2024-05-27T07:33:35.370124+0000","last_undegraded":"2024-05-27T07:33:35.370124+0000","last_fullsized":"2024-05-27T07:33:35.370124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:05:48.456347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.432633+0000","last_change":"2024-05-27T07:28:24.485567+0000","last_active":"2024-05-27T07:33:35.432633+0000","last_peered":"2024-05-27T07:33:35.432633+0000","last_clean":"2024-05-27T07:33:35.432633+0000","last_became_active":"2024-05-27T07:28:24.485175+0000","last_became_peered":"2024-05-27T07:28:24.485175+0000","last_unstale":"2024-05-27T07:33:35.432633+0000","last_undegraded":"2024-05-27T07:33:35.432633+0000","last_fullsized":"2024-05-27T07:33:35.432633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:25:30.062813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431453+0000","last_change":"2024-05-27T07:29:04.900657+0000","last_active":"2024-05-27T07:33:35.431453+0000","last_peered":"2024-05-27T07:33:35.431453+0000","last_clean":"2024-05-27T07:33:35.431453+0000","last_became_active":"2024-05-27T07:29:04.900093+0000","last_became_peered":"2024-05-27T07:29:04.900093+0000","last_unstale":"2024-05-27T07:33:35.431453+0000","last_undegraded":"2024-05-27T07:33:35.431453+0000","last_fullsized":"2024-05-27T07:33:35.431453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:28:04.157825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434350+0000","last_change":"2024-05-27T07:28:24.470503+0000","last_active":"2024-05-27T07:33:35.434350+0000","last_peered":"2024-05-27T07:33:35.434350+0000","last_clean":"2024-05-27T07:33:35.434350+0000","last_became_active":"2024-05-27T07:28:24.470128+0000","last_became_peered":"2024-05-27T07:28:24.470128+0000","last_unstale":"2024-05-27T07:33:35.434350+0000","last_undegraded":"2024-05-27T07:33:35.434350+0000","last_fullsized":"2024-05-27T07:33:35.434350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:46:43.149635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385138+0000","last_change":"2024-05-27T07:28:24.491382+0000","last_active":"2024-05-27T07:33:35.385138+0000","last_peered":"2024-05-27T07:33:35.385138+0000","last_clean":"2024-05-27T07:33:35.385138+0000","last_became_active":"2024-05-27T07:28:24.486155+0000","last_became_peered":"2024-05-27T07:28:24.486155+0000","last_unstale":"2024-05-27T07:33:35.385138+0000","last_undegraded":"2024-05-27T07:33:35.385138+0000","last_fullsized":"2024-05-27T07:33:35.385138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:48.408573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.239172+0000","last_change":"2024-05-27T07:28:24.489140+0000","last_active":"2024-05-27T07:33:35.239172+0000","last_peered":"2024-05-27T07:33:35.239172+0000","last_clean":"2024-05-27T07:33:35.239172+0000","last_became_active":"2024-05-27T07:28:24.488733+0000","last_became_peered":"2024-05-27T07:28:24.488733+0000","last_unstale":"2024-05-27T07:33:35.239172+0000","last_undegraded":"2024-05-27T07:33:35.239172+0000","last_fullsized":"2024-05-27T07:33:35.239172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:02:57.711197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437255+0000","last_change":"2024-05-27T07:28:24.486022+0000","last_active":"2024-05-27T07:33:35.437255+0000","last_peered":"2024-05-27T07:33:35.437255+0000","last_clean":"2024-05-27T07:33:35.437255+0000","last_became_active":"2024-05-27T07:28:24.485639+0000","last_became_peered":"2024-05-27T07:28:24.485639+0000","last_unstale":"2024-05-27T07:33:35.437255+0000","last_undegraded":"2024-05-27T07:33:35.437255+0000","last_fullsized":"2024-05-27T07:33:35.437255+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:07.629402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435201+0000","last_change":"2024-05-27T07:28:37.898984+0000","last_active":"2024-05-27T07:33:35.435201+0000","last_peered":"2024-05-27T07:33:35.435201+0000","last_clean":"2024-05-27T07:33:35.435201+0000","last_became_active":"2024-05-27T07:28:37.898132+0000","last_became_peered":"2024-05-27T07:28:37.898132+0000","last_unstale":"2024-05-27T07:33:35.435201+0000","last_undegraded":"2024-05-27T07:33:35.435201+0000","last_fullsized":"2024-05-27T07:33:35.435201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:51.987386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.239432+0000","last_change":"2024-05-27T07:28:20.402788+0000","last_active":"2024-05-27T07:33:35.239432+0000","last_peered":"2024-05-27T07:33:35.239432+0000","last_clean":"2024-05-27T07:33:35.239432+0000","last_became_active":"2024-05-27T07:28:20.402041+0000","last_became_peered":"2024-05-27T07:28:20.402041+0000","last_unstale":"2024-05-27T07:33:35.239432+0000","last_undegraded":"2024-05-27T07:33:35.239432+0000","last_fullsized":"2024-05-27T07:33:35.239432+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:44.421979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337479+0000","last_change":"2024-05-27T07:28:24.485193+0000","last_active":"2024-05-27T07:33:35.337479+0000","last_peered":"2024-05-27T07:33:35.337479+0000","last_clean":"2024-05-27T07:33:35.337479+0000","last_became_active":"2024-05-27T07:28:24.484086+0000","last_became_peered":"2024-05-27T07:28:24.484086+0000","last_unstale":"2024-05-27T07:33:35.337479+0000","last_undegraded":"2024-05-27T07:33:35.337479+0000","last_fullsized":"2024-05-27T07:33:35.337479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:10.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242950+0000","last_change":"2024-05-27T07:28:37.902602+0000","last_active":"2024-05-27T07:33:35.242950+0000","last_peered":"2024-05-27T07:33:35.242950+0000","last_clean":"2024-05-27T07:33:35.242950+0000","last_became_active":"2024-05-27T07:28:37.902120+0000","last_became_peered":"2024-05-27T07:28:37.902120+0000","last_unstale":"2024-05-27T07:33:35.242950+0000","last_undegraded":"2024-05-27T07:33:35.242950+0000","last_fullsized":"2024-05-27T07:33:35.242950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:21.209851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337842+0000","last_change":"2024-05-27T07:28:20.465500+0000","last_active":"2024-05-27T07:33:35.337842+0000","last_peered":"2024-05-27T07:33:35.337842+0000","last_clean":"2024-05-27T07:33:35.337842+0000","last_became_active":"2024-05-27T07:28:20.464951+0000","last_became_peered":"2024-05-27T07:28:20.464951+0000","last_unstale":"2024-05-27T07:33:35.337842+0000","last_undegraded":"2024-05-27T07:33:35.337842+0000","last_fullsized":"2024-05-27T07:33:35.337842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:23:25.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.432047+0000","last_change":"2024-05-27T07:28:24.467655+0000","last_active":"2024-05-27T07:33:35.432047+0000","last_peered":"2024-05-27T07:33:35.432047+0000","last_clean":"2024-05-27T07:33:35.432047+0000","last_became_active":"2024-05-27T07:28:24.467194+0000","last_became_peered":"2024-05-27T07:28:24.467194+0000","last_unstale":"2024-05-27T07:33:35.432047+0000","last_undegraded":"2024-05-27T07:33:35.432047+0000","last_fullsized":"2024-05-27T07:33:35.432047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:38.024514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243715+0000","last_change":"2024-05-27T07:29:04.975679+0000","last_active":"2024-05-27T07:33:35.243715+0000","last_peered":"2024-05-27T07:33:35.243715+0000","last_clean":"2024-05-27T07:33:35.243715+0000","last_became_active":"2024-05-27T07:29:04.974323+0000","last_became_peered":"2024-05-27T07:29:04.974323+0000","last_unstale":"2024-05-27T07:33:35.243715+0000","last_undegraded":"2024-05-27T07:33:35.243715+0000","last_fullsized":"2024-05-27T07:33:35.243715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:50.489737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435362+0000","last_change":"2024-05-27T07:28:20.408117+0000","last_active":"2024-05-27T07:33:35.435362+0000","last_peered":"2024-05-27T07:33:35.435362+0000","last_clean":"2024-05-27T07:33:35.435362+0000","last_became_active":"2024-05-27T07:28:20.407723+0000","last_became_peered":"2024-05-27T07:28:20.407723+0000","last_unstale":"2024-05-27T07:33:35.435362+0000","last_undegraded":"2024-05-27T07:33:35.435362+0000","last_fullsized":"2024-05-27T07:33:35.435362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:20:31.376656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437235+0000","last_change":"2024-05-27T07:28:24.490269+0000","last_active":"2024-05-27T07:33:35.437235+0000","last_peered":"2024-05-27T07:33:35.437235+0000","last_clean":"2024-05-27T07:33:35.437235+0000","last_became_active":"2024-05-27T07:28:24.488849+0000","last_became_peered":"2024-05-27T07:28:24.488849+0000","last_unstale":"2024-05-27T07:33:35.437235+0000","last_undegraded":"2024-05-27T07:33:35.437235+0000","last_fullsized":"2024-05-27T07:33:35.437235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:00.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242985+0000","last_change":"2024-05-27T07:28:37.908658+0000","last_active":"2024-05-27T07:33:35.242985+0000","last_peered":"2024-05-27T07:33:35.242985+0000","last_clean":"2024-05-27T07:33:35.242985+0000","last_became_active":"2024-05-27T07:28:37.908056+0000","last_became_peered":"2024-05-27T07:28:37.908056+0000","last_unstale":"2024-05-27T07:33:35.242985+0000","last_undegraded":"2024-05-27T07:33:35.242985+0000","last_fullsized":"2024-05-27T07:33:35.242985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:31.853606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436498+0000","last_change":"2024-05-27T07:28:20.410944+0000","last_active":"2024-05-27T07:33:35.436498+0000","last_peered":"2024-05-27T07:33:35.436498+0000","last_clean":"2024-05-27T07:33:35.436498+0000","last_became_active":"2024-05-27T07:28:20.409789+0000","last_became_peered":"2024-05-27T07:28:20.409789+0000","last_unstale":"2024-05-27T07:33:35.436498+0000","last_undegraded":"2024-05-27T07:33:35.436498+0000","last_fullsized":"2024-05-27T07:33:35.436498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:53:34.962397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244941+0000","last_change":"2024-05-27T07:28:24.482828+0000","last_active":"2024-05-27T07:33:35.244941+0000","last_peered":"2024-05-27T07:33:35.244941+0000","last_clean":"2024-05-27T07:33:35.244941+0000","last_became_active":"2024-05-27T07:28:24.482475+0000","last_became_peered":"2024-05-27T07:28:24.482475+0000","last_unstale":"2024-05-27T07:33:35.244941+0000","last_undegraded":"2024-05-27T07:33:35.244941+0000","last_fullsized":"2024-05-27T07:33:35.244941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:53:32.064441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436828+0000","last_change":"2024-05-27T07:29:04.905734+0000","last_active":"2024-05-27T07:33:35.436828+0000","last_peered":"2024-05-27T07:33:35.436828+0000","last_clean":"2024-05-27T07:33:35.436828+0000","last_became_active":"2024-05-27T07:29:04.902387+0000","last_became_peered":"2024-05-27T07:29:04.902387+0000","last_unstale":"2024-05-27T07:33:35.436828+0000","last_undegraded":"2024-05-27T07:33:35.436828+0000","last_fullsized":"2024-05-27T07:33:35.436828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:35.993193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":285,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.741281+0000","last_change":"2024-05-27T07:28:20.412258+0000","last_active":"2024-05-27T07:33:39.741281+0000","last_peered":"2024-05-27T07:33:39.741281+0000","last_clean":"2024-05-27T07:33:39.741281+0000","last_became_active":"2024-05-27T07:28:20.411764+0000","last_became_peered":"2024-05-27T07:28:20.411764+0000","last_unstale":"2024-05-27T07:33:39.741281+0000","last_undegraded":"2024-05-27T07:33:39.741281+0000","last_fullsized":"2024-05-27T07:33:39.741281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:38.462887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":231,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434849+0000","last_change":"2024-05-27T07:28:24.482179+0000","last_active":"2024-05-27T07:33:35.434849+0000","last_peered":"2024-05-27T07:33:35.434849+0000","last_clean":"2024-05-27T07:33:35.434849+0000","last_became_active":"2024-05-27T07:28:24.480682+0000","last_became_peered":"2024-05-27T07:28:24.480682+0000","last_unstale":"2024-05-27T07:33:35.434849+0000","last_undegraded":"2024-05-27T07:33:35.434849+0000","last_fullsized":"2024-05-27T07:33:35.434849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:51.403862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436757+0000","last_change":"2024-05-27T07:28:37.903716+0000","last_active":"2024-05-27T07:33:35.436757+0000","last_peered":"2024-05-27T07:33:35.436757+0000","last_clean":"2024-05-27T07:33:35.436757+0000","last_became_active":"2024-05-27T07:28:37.901083+0000","last_became_peered":"2024-05-27T07:28:37.901083+0000","last_unstale":"2024-05-27T07:33:35.436757+0000","last_undegraded":"2024-05-27T07:33:35.436757+0000","last_fullsized":"2024-05-27T07:33:35.436757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:44:11.606711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401548+0000","last_change":"2024-05-27T07:28:20.392441+0000","last_active":"2024-05-27T07:33:35.401548+0000","last_peered":"2024-05-27T07:33:35.401548+0000","last_clean":"2024-05-27T07:33:35.401548+0000","last_became_active":"2024-05-27T07:28:20.390983+0000","last_became_peered":"2024-05-27T07:28:20.390983+0000","last_unstale":"2024-05-27T07:33:35.401548+0000","last_undegraded":"2024-05-27T07:33:35.401548+0000","last_fullsized":"2024-05-27T07:33:35.401548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:33:30.338043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237689+0000","last_change":"2024-05-27T07:30:05.518915+0000","last_active":"2024-05-27T07:33:35.237689+0000","last_peered":"2024-05-27T07:33:35.237689+0000","last_clean":"2024-05-27T07:33:35.237689+0000","last_became_active":"2024-05-27T07:30:05.517894+0000","last_became_peered":"2024-05-27T07:30:05.517894+0000","last_unstale":"2024-05-27T07:33:35.237689+0000","last_undegraded":"2024-05-27T07:33:35.237689+0000","last_fullsized":"2024-05-27T07:33:35.237689+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:51.353713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,7],"acting":[9,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384762+0000","last_change":"2024-05-27T07:28:37.913352+0000","last_active":"2024-05-27T07:33:35.384762+0000","last_peered":"2024-05-27T07:33:35.384762+0000","last_clean":"2024-05-27T07:33:35.384762+0000","last_became_active":"2024-05-27T07:28:37.913027+0000","last_became_peered":"2024-05-27T07:28:37.913027+0000","last_unstale":"2024-05-27T07:33:35.384762+0000","last_undegraded":"2024-05-27T07:33:35.384762+0000","last_fullsized":"2024-05-27T07:33:35.384762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:03.485594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369823+0000","last_change":"2024-05-27T07:28:20.406094+0000","last_active":"2024-05-27T07:33:35.369823+0000","last_peered":"2024-05-27T07:33:35.369823+0000","last_clean":"2024-05-27T07:33:35.369823+0000","last_became_active":"2024-05-27T07:28:20.405641+0000","last_became_peered":"2024-05-27T07:28:20.405641+0000","last_unstale":"2024-05-27T07:33:35.369823+0000","last_undegraded":"2024-05-27T07:33:35.369823+0000","last_fullsized":"2024-05-27T07:33:35.369823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:16.598743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337048+0000","last_change":"2024-05-27T07:28:24.474696+0000","last_active":"2024-05-27T07:33:35.337048+0000","last_peered":"2024-05-27T07:33:35.337048+0000","last_clean":"2024-05-27T07:33:35.337048+0000","last_became_active":"2024-05-27T07:28:24.474319+0000","last_became_peered":"2024-05-27T07:28:24.474319+0000","last_unstale":"2024-05-27T07:33:35.337048+0000","last_undegraded":"2024-05-27T07:33:35.337048+0000","last_fullsized":"2024-05-27T07:33:35.337048+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:53.010898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431719+0000","last_change":"2024-05-27T07:28:37.910386+0000","last_active":"2024-05-27T07:33:35.431719+0000","last_peered":"2024-05-27T07:33:35.431719+0000","last_clean":"2024-05-27T07:33:35.431719+0000","last_became_active":"2024-05-27T07:28:37.909941+0000","last_became_peered":"2024-05-27T07:28:37.909941+0000","last_unstale":"2024-05-27T07:33:35.431719+0000","last_undegraded":"2024-05-27T07:33:35.431719+0000","last_fullsized":"2024-05-27T07:33:35.431719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:15.820318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242969+0000","last_change":"2024-05-27T07:28:20.411855+0000","last_active":"2024-05-27T07:33:35.242969+0000","last_peered":"2024-05-27T07:33:35.242969+0000","last_clean":"2024-05-27T07:33:35.242969+0000","last_became_active":"2024-05-27T07:28:20.411169+0000","last_became_peered":"2024-05-27T07:28:20.411169+0000","last_unstale":"2024-05-27T07:33:35.242969+0000","last_undegraded":"2024-05-27T07:33:35.242969+0000","last_fullsized":"2024-05-27T07:33:35.242969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:37.278507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237559+0000","last_change":"2024-05-27T07:28:24.485799+0000","last_active":"2024-05-27T07:33:35.237559+0000","last_peered":"2024-05-27T07:33:35.237559+0000","last_clean":"2024-05-27T07:33:35.237559+0000","last_became_active":"2024-05-27T07:28:24.485395+0000","last_became_peered":"2024-05-27T07:28:24.485395+0000","last_unstale":"2024-05-27T07:33:35.237559+0000","last_undegraded":"2024-05-27T07:33:35.237559+0000","last_fullsized":"2024-05-27T07:33:35.237559+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:11.178129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336979+0000","last_change":"2024-05-27T07:28:37.898929+0000","last_active":"2024-05-27T07:33:35.336979+0000","last_peered":"2024-05-27T07:33:35.336979+0000","last_clean":"2024-05-27T07:33:35.336979+0000","last_became_active":"2024-05-27T07:28:37.898371+0000","last_became_peered":"2024-05-27T07:28:37.898371+0000","last_unstale":"2024-05-27T07:33:35.336979+0000","last_undegraded":"2024-05-27T07:33:35.336979+0000","last_fullsized":"2024-05-27T07:33:35.336979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:48.867729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":280,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.740411+0000","last_change":"2024-05-27T07:28:20.407117+0000","last_active":"2024-05-27T07:33:39.740411+0000","last_peered":"2024-05-27T07:33:39.740411+0000","last_clean":"2024-05-27T07:33:39.740411+0000","last_became_active":"2024-05-27T07:28:20.403562+0000","last_became_peered":"2024-05-27T07:28:20.403562+0000","last_unstale":"2024-05-27T07:33:39.740411+0000","last_undegraded":"2024-05-27T07:33:39.740411+0000","last_fullsized":"2024-05-27T07:33:39.740411+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:25.903886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.438353+0000","last_change":"2024-05-27T07:28:24.489771+0000","last_active":"2024-05-27T07:33:35.438353+0000","last_peered":"2024-05-27T07:33:35.438353+0000","last_clean":"2024-05-27T07:33:35.438353+0000","last_became_active":"2024-05-27T07:28:24.489163+0000","last_became_peered":"2024-05-27T07:28:24.489163+0000","last_unstale":"2024-05-27T07:33:35.438353+0000","last_undegraded":"2024-05-27T07:33:35.438353+0000","last_fullsized":"2024-05-27T07:33:35.438353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:08.053634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237277+0000","last_change":"2024-05-27T07:28:37.914598+0000","last_active":"2024-05-27T07:33:35.237277+0000","last_peered":"2024-05-27T07:33:35.237277+0000","last_clean":"2024-05-27T07:33:35.237277+0000","last_became_active":"2024-05-27T07:28:37.913916+0000","last_became_peered":"2024-05-27T07:28:37.913916+0000","last_unstale":"2024-05-27T07:33:35.237277+0000","last_undegraded":"2024-05-27T07:33:35.237277+0000","last_fullsized":"2024-05-27T07:33:35.237277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:01.475789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436369+0000","last_change":"2024-05-27T07:29:04.901366+0000","last_active":"2024-05-27T07:33:35.436369+0000","last_peered":"2024-05-27T07:33:35.436369+0000","last_clean":"2024-05-27T07:33:35.436369+0000","last_became_active":"2024-05-27T07:29:04.900355+0000","last_became_peered":"2024-05-27T07:29:04.900355+0000","last_unstale":"2024-05-27T07:33:35.436369+0000","last_undegraded":"2024-05-27T07:33:35.436369+0000","last_fullsized":"2024-05-27T07:33:35.436369+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:10:03.349345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.432117+0000","last_change":"2024-05-27T07:28:24.477771+0000","last_active":"2024-05-27T07:33:35.432117+0000","last_peered":"2024-05-27T07:33:35.432117+0000","last_clean":"2024-05-27T07:33:35.432117+0000","last_became_active":"2024-05-27T07:28:24.474179+0000","last_became_peered":"2024-05-27T07:28:24.474179+0000","last_unstale":"2024-05-27T07:33:35.432117+0000","last_undegraded":"2024-05-27T07:33:35.432117+0000","last_fullsized":"2024-05-27T07:33:35.432117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:08:04.218264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243888+0000","last_change":"2024-05-27T07:29:04.896583+0000","last_active":"2024-05-27T07:33:35.243888+0000","last_peered":"2024-05-27T07:33:35.243888+0000","last_clean":"2024-05-27T07:33:35.243888+0000","last_became_active":"2024-05-27T07:29:04.894118+0000","last_became_peered":"2024-05-27T07:29:04.894118+0000","last_unstale":"2024-05-27T07:33:35.243888+0000","last_undegraded":"2024-05-27T07:33:35.243888+0000","last_fullsized":"2024-05-27T07:33:35.243888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:17.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244297+0000","last_change":"2024-05-27T07:28:20.466851+0000","last_active":"2024-05-27T07:33:35.244297+0000","last_peered":"2024-05-27T07:33:35.244297+0000","last_clean":"2024-05-27T07:33:35.244297+0000","last_became_active":"2024-05-27T07:28:20.465133+0000","last_became_peered":"2024-05-27T07:28:20.465133+0000","last_unstale":"2024-05-27T07:33:35.244297+0000","last_undegraded":"2024-05-27T07:33:35.244297+0000","last_fullsized":"2024-05-27T07:33:35.244297+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:11.025588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437815+0000","last_change":"2024-05-27T07:28:24.469924+0000","last_active":"2024-05-27T07:33:35.437815+0000","last_peered":"2024-05-27T07:33:35.437815+0000","last_clean":"2024-05-27T07:33:35.437815+0000","last_became_active":"2024-05-27T07:28:24.469559+0000","last_became_peered":"2024-05-27T07:28:24.469559+0000","last_unstale":"2024-05-27T07:33:35.437815+0000","last_undegraded":"2024-05-27T07:33:35.437815+0000","last_fullsized":"2024-05-27T07:33:35.437815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:57.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237901+0000","last_change":"2024-05-27T07:28:37.904549+0000","last_active":"2024-05-27T07:33:35.237901+0000","last_peered":"2024-05-27T07:33:35.237901+0000","last_clean":"2024-05-27T07:33:35.237901+0000","last_became_active":"2024-05-27T07:28:37.903882+0000","last_became_peered":"2024-05-27T07:28:37.903882+0000","last_unstale":"2024-05-27T07:33:35.237901+0000","last_undegraded":"2024-05-27T07:33:35.237901+0000","last_fullsized":"2024-05-27T07:33:35.237901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:41.607641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'44","reported_seq":563,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:40.741316+0000","last_change":"2024-05-27T07:28:20.410025+0000","last_active":"2024-05-27T07:33:40.741316+0000","last_peered":"2024-05-27T07:33:40.741316+0000","last_clean":"2024-05-27T07:33:40.741316+0000","last_became_active":"2024-05-27T07:28:20.409115+0000","last_became_peered":"2024-05-27T07:28:20.409115+0000","last_unstale":"2024-05-27T07:33:40.741316+0000","last_undegraded":"2024-05-27T07:33:40.741316+0000","last_fullsized":"2024-05-27T07:33:40.741316+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:56.834059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77315,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":471,"num_read_kb":309,"num_write":45,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436027+0000","last_change":"2024-05-27T07:28:24.469609+0000","last_active":"2024-05-27T07:33:35.436027+0000","last_peered":"2024-05-27T07:33:35.436027+0000","last_clean":"2024-05-27T07:33:35.436027+0000","last_became_active":"2024-05-27T07:28:24.469192+0000","last_became_peered":"2024-05-27T07:28:24.469192+0000","last_unstale":"2024-05-27T07:33:35.436027+0000","last_undegraded":"2024-05-27T07:33:35.436027+0000","last_fullsized":"2024-05-27T07:33:35.436027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:32:59.006298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.246044+0000","last_change":"2024-05-27T07:28:37.900745+0000","last_active":"2024-05-27T07:33:35.246044+0000","last_peered":"2024-05-27T07:33:35.246044+0000","last_clean":"2024-05-27T07:33:35.246044+0000","last_became_active":"2024-05-27T07:28:37.900205+0000","last_became_peered":"2024-05-27T07:28:37.900205+0000","last_unstale":"2024-05-27T07:33:35.246044+0000","last_undegraded":"2024-05-27T07:33:35.246044+0000","last_fullsized":"2024-05-27T07:33:35.246044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:52:32.235201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435338+0000","last_change":"2024-05-27T07:28:20.413380+0000","last_active":"2024-05-27T07:33:35.435338+0000","last_peered":"2024-05-27T07:33:35.435338+0000","last_clean":"2024-05-27T07:33:35.435338+0000","last_became_active":"2024-05-27T07:28:20.412839+0000","last_became_peered":"2024-05-27T07:28:20.412839+0000","last_unstale":"2024-05-27T07:33:35.435338+0000","last_undegraded":"2024-05-27T07:33:35.435338+0000","last_fullsized":"2024-05-27T07:33:35.435338+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:57.481203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369637+0000","last_change":"2024-05-27T07:28:24.474228+0000","last_active":"2024-05-27T07:33:35.369637+0000","last_peered":"2024-05-27T07:33:35.369637+0000","last_clean":"2024-05-27T07:33:35.369637+0000","last_became_active":"2024-05-27T07:28:24.473833+0000","last_became_peered":"2024-05-27T07:28:24.473833+0000","last_unstale":"2024-05-27T07:33:35.369637+0000","last_undegraded":"2024-05-27T07:33:35.369637+0000","last_fullsized":"2024-05-27T07:33:35.369637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:52.456730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384646+0000","last_change":"2024-05-27T07:28:37.917630+0000","last_active":"2024-05-27T07:33:35.384646+0000","last_peered":"2024-05-27T07:33:35.384646+0000","last_clean":"2024-05-27T07:33:35.384646+0000","last_became_active":"2024-05-27T07:28:37.917253+0000","last_became_peered":"2024-05-27T07:28:37.917253+0000","last_unstale":"2024-05-27T07:33:35.384646+0000","last_undegraded":"2024-05-27T07:33:35.384646+0000","last_fullsized":"2024-05-27T07:33:35.384646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:05.069946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243812+0000","last_change":"2024-05-27T07:28:20.408988+0000","last_active":"2024-05-27T07:33:35.243812+0000","last_peered":"2024-05-27T07:33:35.243812+0000","last_clean":"2024-05-27T07:33:35.243812+0000","last_became_active":"2024-05-27T07:28:20.408313+0000","last_became_peered":"2024-05-27T07:28:20.408313+0000","last_unstale":"2024-05-27T07:33:35.243812+0000","last_undegraded":"2024-05-27T07:33:35.243812+0000","last_fullsized":"2024-05-27T07:33:35.243812+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:17.480412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435459+0000","last_change":"2024-05-27T07:28:24.480096+0000","last_active":"2024-05-27T07:33:35.435459+0000","last_peered":"2024-05-27T07:33:35.435459+0000","last_clean":"2024-05-27T07:33:35.435459+0000","last_became_active":"2024-05-27T07:28:24.477988+0000","last_became_peered":"2024-05-27T07:28:24.477988+0000","last_unstale":"2024-05-27T07:33:35.435459+0000","last_undegraded":"2024-05-27T07:33:35.435459+0000","last_fullsized":"2024-05-27T07:33:35.435459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:13:59.860458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.238087+0000","last_change":"2024-05-27T07:28:37.917518+0000","last_active":"2024-05-27T07:33:35.238087+0000","last_peered":"2024-05-27T07:33:35.238087+0000","last_clean":"2024-05-27T07:33:35.238087+0000","last_became_active":"2024-05-27T07:28:37.916971+0000","last_became_peered":"2024-05-27T07:28:37.916971+0000","last_unstale":"2024-05-27T07:33:35.238087+0000","last_undegraded":"2024-05-27T07:33:35.238087+0000","last_fullsized":"2024-05-27T07:33:35.238087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:17:53.067747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.783309+0000","last_change":"2024-05-27T07:28:20.416766+0000","last_active":"2024-05-27T07:33:39.783309+0000","last_peered":"2024-05-27T07:33:39.783309+0000","last_clean":"2024-05-27T07:33:39.783309+0000","last_became_active":"2024-05-27T07:28:20.416297+0000","last_became_peered":"2024-05-27T07:28:20.416297+0000","last_unstale":"2024-05-27T07:33:39.783309+0000","last_undegraded":"2024-05-27T07:33:39.783309+0000","last_fullsized":"2024-05-27T07:33:39.783309+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:02.841494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2254,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":233,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435388+0000","last_change":"2024-05-27T07:28:24.486869+0000","last_active":"2024-05-27T07:33:35.435388+0000","last_peered":"2024-05-27T07:33:35.435388+0000","last_clean":"2024-05-27T07:33:35.435388+0000","last_became_active":"2024-05-27T07:28:24.486451+0000","last_became_peered":"2024-05-27T07:28:24.486451+0000","last_unstale":"2024-05-27T07:33:35.435388+0000","last_undegraded":"2024-05-27T07:33:35.435388+0000","last_fullsized":"2024-05-27T07:33:35.435388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:10.722659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370829+0000","last_change":"2024-05-27T07:29:04.901632+0000","last_active":"2024-05-27T07:33:35.370829+0000","last_peered":"2024-05-27T07:33:35.370829+0000","last_clean":"2024-05-27T07:33:35.370829+0000","last_became_active":"2024-05-27T07:29:04.900203+0000","last_became_peered":"2024-05-27T07:29:04.900203+0000","last_unstale":"2024-05-27T07:33:35.370829+0000","last_undegraded":"2024-05-27T07:33:35.370829+0000","last_fullsized":"2024-05-27T07:33:35.370829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:57.834889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":63,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434045+0000","last_change":"2024-05-27T07:28:20.468162+0000","last_active":"2024-05-27T07:33:35.434045+0000","last_peered":"2024-05-27T07:33:35.434045+0000","last_clean":"2024-05-27T07:33:35.434045+0000","last_became_active":"2024-05-27T07:28:20.465226+0000","last_became_peered":"2024-05-27T07:28:20.465226+0000","last_unstale":"2024-05-27T07:33:35.434045+0000","last_undegraded":"2024-05-27T07:33:35.434045+0000","last_fullsized":"2024-05-27T07:33:35.434045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:21:34.110356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243300+0000","last_change":"2024-05-27T07:28:24.485053+0000","last_active":"2024-05-27T07:33:35.243300+0000","last_peered":"2024-05-27T07:33:35.243300+0000","last_clean":"2024-05-27T07:33:35.243300+0000","last_became_active":"2024-05-27T07:28:24.484391+0000","last_became_peered":"2024-05-27T07:28:24.484391+0000","last_unstale":"2024-05-27T07:33:35.243300+0000","last_undegraded":"2024-05-27T07:33:35.243300+0000","last_fullsized":"2024-05-27T07:33:35.243300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:05.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243032+0000","last_change":"2024-05-27T07:28:37.904731+0000","last_active":"2024-05-27T07:33:35.243032+0000","last_peered":"2024-05-27T07:33:35.243032+0000","last_clean":"2024-05-27T07:33:35.243032+0000","last_became_active":"2024-05-27T07:28:37.904284+0000","last_became_peered":"2024-05-27T07:28:37.904284+0000","last_unstale":"2024-05-27T07:33:35.243032+0000","last_undegraded":"2024-05-27T07:33:35.243032+0000","last_fullsized":"2024-05-27T07:33:35.243032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:23.198465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435888+0000","last_change":"2024-05-27T07:28:20.406829+0000","last_active":"2024-05-27T07:33:35.435888+0000","last_peered":"2024-05-27T07:33:35.435888+0000","last_clean":"2024-05-27T07:33:35.435888+0000","last_became_active":"2024-05-27T07:28:20.405497+0000","last_became_peered":"2024-05-27T07:28:20.405497+0000","last_unstale":"2024-05-27T07:33:35.435888+0000","last_undegraded":"2024-05-27T07:33:35.435888+0000","last_fullsized":"2024-05-27T07:33:35.435888+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:14.225630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385238+0000","last_change":"2024-05-27T07:28:24.488023+0000","last_active":"2024-05-27T07:33:35.385238+0000","last_peered":"2024-05-27T07:33:35.385238+0000","last_clean":"2024-05-27T07:33:35.385238+0000","last_became_active":"2024-05-27T07:28:24.487652+0000","last_became_peered":"2024-05-27T07:28:24.487652+0000","last_unstale":"2024-05-27T07:33:35.385238+0000","last_undegraded":"2024-05-27T07:33:35.385238+0000","last_fullsized":"2024-05-27T07:33:35.385238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:25.951157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434437+0000","last_change":"2024-05-27T07:28:37.968034+0000","last_active":"2024-05-27T07:33:35.434437+0000","last_peered":"2024-05-27T07:33:35.434437+0000","last_clean":"2024-05-27T07:33:35.434437+0000","last_became_active":"2024-05-27T07:28:37.967127+0000","last_became_peered":"2024-05-27T07:28:37.967127+0000","last_unstale":"2024-05-27T07:33:35.434437+0000","last_undegraded":"2024-05-27T07:33:35.434437+0000","last_fullsized":"2024-05-27T07:33:35.434437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:49.353423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.238627+0000","last_change":"2024-05-27T07:28:20.465782+0000","last_active":"2024-05-27T07:33:35.238627+0000","last_peered":"2024-05-27T07:33:35.238627+0000","last_clean":"2024-05-27T07:33:35.238627+0000","last_became_active":"2024-05-27T07:28:20.465134+0000","last_became_peered":"2024-05-27T07:28:20.465134+0000","last_unstale":"2024-05-27T07:33:35.238627+0000","last_undegraded":"2024-05-27T07:33:35.238627+0000","last_fullsized":"2024-05-27T07:33:35.238627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:17.992092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401705+0000","last_change":"2024-05-27T07:28:24.484691+0000","last_active":"2024-05-27T07:33:35.401705+0000","last_peered":"2024-05-27T07:33:35.401705+0000","last_clean":"2024-05-27T07:33:35.401705+0000","last_became_active":"2024-05-27T07:28:24.484249+0000","last_became_peered":"2024-05-27T07:28:24.484249+0000","last_unstale":"2024-05-27T07:33:35.401705+0000","last_undegraded":"2024-05-27T07:33:35.401705+0000","last_fullsized":"2024-05-27T07:33:35.401705+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:25.753276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369751+0000","last_change":"2024-05-27T07:28:37.907849+0000","last_active":"2024-05-27T07:33:35.369751+0000","last_peered":"2024-05-27T07:33:35.369751+0000","last_clean":"2024-05-27T07:33:35.369751+0000","last_became_active":"2024-05-27T07:28:37.907105+0000","last_became_peered":"2024-05-27T07:28:37.907105+0000","last_unstale":"2024-05-27T07:33:35.369751+0000","last_undegraded":"2024-05-27T07:33:35.369751+0000","last_fullsized":"2024-05-27T07:33:35.369751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:54.075240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.433916+0000","last_change":"2024-05-27T07:28:20.411302+0000","last_active":"2024-05-27T07:33:35.433916+0000","last_peered":"2024-05-27T07:33:35.433916+0000","last_clean":"2024-05-27T07:33:35.433916+0000","last_became_active":"2024-05-27T07:28:20.409144+0000","last_became_peered":"2024-05-27T07:28:20.409144+0000","last_unstale":"2024-05-27T07:33:35.433916+0000","last_undegraded":"2024-05-27T07:33:35.433916+0000","last_fullsized":"2024-05-27T07:33:35.433916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:49.985962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401624+0000","last_change":"2024-05-27T07:28:24.487742+0000","last_active":"2024-05-27T07:33:35.401624+0000","last_peered":"2024-05-27T07:33:35.401624+0000","last_clean":"2024-05-27T07:33:35.401624+0000","last_became_active":"2024-05-27T07:28:24.485279+0000","last_became_peered":"2024-05-27T07:28:24.485279+0000","last_unstale":"2024-05-27T07:33:35.401624+0000","last_undegraded":"2024-05-27T07:33:35.401624+0000","last_fullsized":"2024-05-27T07:33:35.401624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:15.049572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.371172+0000","last_change":"2024-05-27T07:28:37.968780+0000","last_active":"2024-05-27T07:33:35.371172+0000","last_peered":"2024-05-27T07:33:35.371172+0000","last_clean":"2024-05-27T07:33:35.371172+0000","last_became_active":"2024-05-27T07:28:37.968052+0000","last_became_peered":"2024-05-27T07:28:37.968052+0000","last_unstale":"2024-05-27T07:33:35.371172+0000","last_undegraded":"2024-05-27T07:33:35.371172+0000","last_fullsized":"2024-05-27T07:33:35.371172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:51:26.787969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244844+0000","last_change":"2024-05-27T07:28:24.485280+0000","last_active":"2024-05-27T07:33:35.244844+0000","last_peered":"2024-05-27T07:33:35.244844+0000","last_clean":"2024-05-27T07:33:35.244844+0000","last_became_active":"2024-05-27T07:28:24.484882+0000","last_became_peered":"2024-05-27T07:28:24.484882+0000","last_unstale":"2024-05-27T07:33:35.244844+0000","last_undegraded":"2024-05-27T07:33:35.244844+0000","last_fullsized":"2024-05-27T07:33:35.244844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:28.643857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":280,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.781145+0000","last_change":"2024-05-27T07:28:20.462593+0000","last_active":"2024-05-27T07:33:39.781145+0000","last_peered":"2024-05-27T07:33:39.781145+0000","last_clean":"2024-05-27T07:33:39.781145+0000","last_became_active":"2024-05-27T07:28:20.461128+0000","last_became_peered":"2024-05-27T07:28:20.461128+0000","last_unstale":"2024-05-27T07:33:39.781145+0000","last_undegraded":"2024-05-27T07:33:39.781145+0000","last_fullsized":"2024-05-27T07:33:39.781145+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:24:42.013315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436013+0000","last_change":"2024-05-27T07:28:37.911546+0000","last_active":"2024-05-27T07:33:35.436013+0000","last_peered":"2024-05-27T07:33:35.436013+0000","last_clean":"2024-05-27T07:33:35.436013+0000","last_became_active":"2024-05-27T07:28:37.910777+0000","last_became_peered":"2024-05-27T07:28:37.910777+0000","last_unstale":"2024-05-27T07:33:35.436013+0000","last_undegraded":"2024-05-27T07:33:35.436013+0000","last_fullsized":"2024-05-27T07:33:35.436013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:13.344317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336963+0000","last_change":"2024-05-27T07:28:24.483946+0000","last_active":"2024-05-27T07:33:35.336963+0000","last_peered":"2024-05-27T07:33:35.336963+0000","last_clean":"2024-05-27T07:33:35.336963+0000","last_became_active":"2024-05-27T07:28:24.481424+0000","last_became_peered":"2024-05-27T07:28:24.481424+0000","last_unstale":"2024-05-27T07:33:35.336963+0000","last_undegraded":"2024-05-27T07:33:35.336963+0000","last_fullsized":"2024-05-27T07:33:35.336963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:08.254046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243763+0000","last_change":"2024-05-27T07:28:20.414250+0000","last_active":"2024-05-27T07:33:35.243763+0000","last_peered":"2024-05-27T07:33:35.243763+0000","last_clean":"2024-05-27T07:33:35.243763+0000","last_became_active":"2024-05-27T07:28:20.413676+0000","last_became_peered":"2024-05-27T07:28:20.413676+0000","last_unstale":"2024-05-27T07:33:35.243763+0000","last_undegraded":"2024-05-27T07:33:35.243763+0000","last_fullsized":"2024-05-27T07:33:35.243763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:10.226771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'99","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:37.400097+0000","last_change":"2024-05-27T07:26:26.055626+0000","last_active":"2024-05-27T07:33:37.400097+0000","last_peered":"2024-05-27T07:33:37.400097+0000","last_clean":"2024-05-27T07:33:37.400097+0000","last_became_active":"2024-05-27T07:26:26.037376+0000","last_became_peered":"2024-05-27T07:26:26.037376+0000","last_unstale":"2024-05-27T07:33:37.400097+0000","last_undegraded":"2024-05-27T07:33:37.400097+0000","last_fullsized":"2024-05-27T07:33:37.400097+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_clean_scrub_stamp":"2024-05-27T07:21:26.641949+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:16.741856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244470+0000","last_change":"2024-05-27T07:28:37.973078+0000","last_active":"2024-05-27T07:33:35.244470+0000","last_peered":"2024-05-27T07:33:35.244470+0000","last_clean":"2024-05-27T07:33:35.244470+0000","last_became_active":"2024-05-27T07:28:37.972381+0000","last_became_peered":"2024-05-27T07:28:37.972381+0000","last_unstale":"2024-05-27T07:33:35.244470+0000","last_undegraded":"2024-05-27T07:33:35.244470+0000","last_fullsized":"2024-05-27T07:33:35.244470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:18:16.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369481+0000","last_change":"2024-05-27T07:28:24.483512+0000","last_active":"2024-05-27T07:33:35.369481+0000","last_peered":"2024-05-27T07:33:35.369481+0000","last_clean":"2024-05-27T07:33:35.369481+0000","last_became_active":"2024-05-27T07:28:24.482655+0000","last_became_peered":"2024-05-27T07:28:24.482655+0000","last_unstale":"2024-05-27T07:33:35.369481+0000","last_undegraded":"2024-05-27T07:33:35.369481+0000","last_fullsized":"2024-05-27T07:33:35.369481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:01:07.898504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434986+0000","last_change":"2024-05-27T07:28:20.408953+0000","last_active":"2024-05-27T07:33:35.434986+0000","last_peered":"2024-05-27T07:33:35.434986+0000","last_clean":"2024-05-27T07:33:35.434986+0000","last_became_active":"2024-05-27T07:28:20.407631+0000","last_became_peered":"2024-05-27T07:28:20.407631+0000","last_unstale":"2024-05-27T07:33:35.434986+0000","last_undegraded":"2024-05-27T07:33:35.434986+0000","last_fullsized":"2024-05-27T07:33:35.434986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:43.912338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384425+0000","last_change":"2024-05-27T07:28:37.907441+0000","last_active":"2024-05-27T07:33:35.384425+0000","last_peered":"2024-05-27T07:33:35.384425+0000","last_clean":"2024-05-27T07:33:35.384425+0000","last_became_active":"2024-05-27T07:28:37.906928+0000","last_became_peered":"2024-05-27T07:28:37.906928+0000","last_unstale":"2024-05-27T07:33:35.384425+0000","last_undegraded":"2024-05-27T07:33:35.384425+0000","last_fullsized":"2024-05-27T07:33:35.384425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:28:32.345826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435906+0000","last_change":"2024-05-27T07:28:20.467408+0000","last_active":"2024-05-27T07:33:35.435906+0000","last_peered":"2024-05-27T07:33:35.435906+0000","last_clean":"2024-05-27T07:33:35.435906+0000","last_became_active":"2024-05-27T07:28:20.466064+0000","last_became_peered":"2024-05-27T07:28:20.466064+0000","last_unstale":"2024-05-27T07:33:35.435906+0000","last_undegraded":"2024-05-27T07:33:35.435906+0000","last_fullsized":"2024-05-27T07:33:35.435906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:15:51.449514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.371394+0000","last_change":"2024-05-27T07:28:24.478475+0000","last_active":"2024-05-27T07:33:35.371394+0000","last_peered":"2024-05-27T07:33:35.371394+0000","last_clean":"2024-05-27T07:33:35.371394+0000","last_became_active":"2024-05-27T07:28:24.477936+0000","last_became_peered":"2024-05-27T07:28:24.477936+0000","last_unstale":"2024-05-27T07:33:35.371394+0000","last_undegraded":"2024-05-27T07:33:35.371394+0000","last_fullsized":"2024-05-27T07:33:35.371394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:53:17.500040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336847+0000","last_change":"2024-05-27T07:28:37.968192+0000","last_active":"2024-05-27T07:33:35.336847+0000","last_peered":"2024-05-27T07:33:35.336847+0000","last_clean":"2024-05-27T07:33:35.336847+0000","last_became_active":"2024-05-27T07:28:37.967603+0000","last_became_peered":"2024-05-27T07:28:37.967603+0000","last_unstale":"2024-05-27T07:33:35.336847+0000","last_undegraded":"2024-05-27T07:33:35.336847+0000","last_fullsized":"2024-05-27T07:33:35.336847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:53:36.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'49","reported_seq":129,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.622094+0000","last_change":"2024-05-27T07:29:04.901062+0000","last_active":"2024-05-27T07:33:35.622094+0000","last_peered":"2024-05-27T07:33:35.622094+0000","last_clean":"2024-05-27T07:33:35.622094+0000","last_became_active":"2024-05-27T07:29:04.892060+0000","last_became_peered":"2024-05-27T07:29:04.892060+0000","last_unstale":"2024-05-27T07:33:35.622094+0000","last_undegraded":"2024-05-27T07:33:35.622094+0000","last_fullsized":"2024-05-27T07:33:35.622094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:27:37.894263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337180+0000","last_change":"2024-05-27T07:28:24.477096+0000","last_active":"2024-05-27T07:33:35.337180+0000","last_peered":"2024-05-27T07:33:35.337180+0000","last_clean":"2024-05-27T07:33:35.337180+0000","last_became_active":"2024-05-27T07:28:24.476712+0000","last_became_peered":"2024-05-27T07:28:24.476712+0000","last_unstale":"2024-05-27T07:33:35.337180+0000","last_undegraded":"2024-05-27T07:33:35.337180+0000","last_fullsized":"2024-05-27T07:33:35.337180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:31:58.010761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434405+0000","last_change":"2024-05-27T07:28:37.894013+0000","last_active":"2024-05-27T07:33:35.434405+0000","last_peered":"2024-05-27T07:33:35.434405+0000","last_clean":"2024-05-27T07:33:35.434405+0000","last_became_active":"2024-05-27T07:28:37.893345+0000","last_became_peered":"2024-05-27T07:28:37.893345+0000","last_unstale":"2024-05-27T07:33:35.434405+0000","last_undegraded":"2024-05-27T07:33:35.434405+0000","last_fullsized":"2024-05-27T07:33:35.434405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:26.917731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244487+0000","last_change":"2024-05-27T07:28:20.744087+0000","last_active":"2024-05-27T07:33:35.244487+0000","last_peered":"2024-05-27T07:33:35.244487+0000","last_clean":"2024-05-27T07:33:35.244487+0000","last_became_active":"2024-05-27T07:28:20.742883+0000","last_became_peered":"2024-05-27T07:28:20.742883+0000","last_unstale":"2024-05-27T07:33:35.244487+0000","last_undegraded":"2024-05-27T07:33:35.244487+0000","last_fullsized":"2024-05-27T07:33:35.244487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:06.083348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431834+0000","last_change":"2024-05-27T07:28:24.480330+0000","last_active":"2024-05-27T07:33:35.431834+0000","last_peered":"2024-05-27T07:33:35.431834+0000","last_clean":"2024-05-27T07:33:35.431834+0000","last_became_active":"2024-05-27T07:28:24.479300+0000","last_became_peered":"2024-05-27T07:28:24.479300+0000","last_unstale":"2024-05-27T07:33:35.431834+0000","last_undegraded":"2024-05-27T07:33:35.431834+0000","last_fullsized":"2024-05-27T07:33:35.431834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:43.802273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243811+0000","last_change":"2024-05-27T07:28:37.973592+0000","last_active":"2024-05-27T07:33:35.243811+0000","last_peered":"2024-05-27T07:33:35.243811+0000","last_clean":"2024-05-27T07:33:35.243811+0000","last_became_active":"2024-05-27T07:28:37.972475+0000","last_became_peered":"2024-05-27T07:28:37.972475+0000","last_unstale":"2024-05-27T07:33:35.243811+0000","last_undegraded":"2024-05-27T07:33:35.243811+0000","last_fullsized":"2024-05-27T07:33:35.243811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:31.662384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434870+0000","last_change":"2024-05-27T07:29:04.890751+0000","last_active":"2024-05-27T07:33:35.434870+0000","last_peered":"2024-05-27T07:33:35.434870+0000","last_clean":"2024-05-27T07:33:35.434870+0000","last_became_active":"2024-05-27T07:29:04.889447+0000","last_became_peered":"2024-05-27T07:29:04.889447+0000","last_unstale":"2024-05-27T07:33:35.434870+0000","last_undegraded":"2024-05-27T07:33:35.434870+0000","last_fullsized":"2024-05-27T07:33:35.434870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:13:34.743691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434302+0000","last_change":"2024-05-27T07:28:24.474298+0000","last_active":"2024-05-27T07:33:35.434302+0000","last_peered":"2024-05-27T07:33:35.434302+0000","last_clean":"2024-05-27T07:33:35.434302+0000","last_became_active":"2024-05-27T07:28:24.473877+0000","last_became_peered":"2024-05-27T07:28:24.473877+0000","last_unstale":"2024-05-27T07:33:35.434302+0000","last_undegraded":"2024-05-27T07:33:35.434302+0000","last_fullsized":"2024-05-27T07:33:35.434302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:31.545883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.239267+0000","last_change":"2024-05-27T07:28:37.905417+0000","last_active":"2024-05-27T07:33:35.239267+0000","last_peered":"2024-05-27T07:33:35.239267+0000","last_clean":"2024-05-27T07:33:35.239267+0000","last_became_active":"2024-05-27T07:28:37.904968+0000","last_became_peered":"2024-05-27T07:28:37.904968+0000","last_unstale":"2024-05-27T07:33:35.239267+0000","last_undegraded":"2024-05-27T07:33:35.239267+0000","last_fullsized":"2024-05-27T07:33:35.239267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:02.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337111+0000","last_change":"2024-05-27T07:28:20.392647+0000","last_active":"2024-05-27T07:33:35.337111+0000","last_peered":"2024-05-27T07:33:35.337111+0000","last_clean":"2024-05-27T07:33:35.337111+0000","last_became_active":"2024-05-27T07:28:20.392073+0000","last_became_peered":"2024-05-27T07:28:20.392073+0000","last_unstale":"2024-05-27T07:33:35.337111+0000","last_undegraded":"2024-05-27T07:33:35.337111+0000","last_fullsized":"2024-05-27T07:33:35.337111+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:18.777628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384408+0000","last_change":"2024-05-27T07:28:24.490522+0000","last_active":"2024-05-27T07:33:35.384408+0000","last_peered":"2024-05-27T07:33:35.384408+0000","last_clean":"2024-05-27T07:33:35.384408+0000","last_became_active":"2024-05-27T07:28:24.485228+0000","last_became_peered":"2024-05-27T07:28:24.485228+0000","last_unstale":"2024-05-27T07:33:35.384408+0000","last_undegraded":"2024-05-27T07:33:35.384408+0000","last_fullsized":"2024-05-27T07:33:35.384408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:14.221016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243254+0000","last_change":"2024-05-27T07:28:37.900449+0000","last_active":"2024-05-27T07:33:35.243254+0000","last_peered":"2024-05-27T07:33:35.243254+0000","last_clean":"2024-05-27T07:33:35.243254+0000","last_became_active":"2024-05-27T07:28:37.899526+0000","last_became_peered":"2024-05-27T07:28:37.899526+0000","last_unstale":"2024-05-27T07:33:35.243254+0000","last_undegraded":"2024-05-27T07:33:35.243254+0000","last_fullsized":"2024-05-27T07:33:35.243254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:29:49.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437781+0000","last_change":"2024-05-27T07:28:20.405236+0000","last_active":"2024-05-27T07:33:35.437781+0000","last_peered":"2024-05-27T07:33:35.437781+0000","last_clean":"2024-05-27T07:33:35.437781+0000","last_became_active":"2024-05-27T07:28:20.404619+0000","last_became_peered":"2024-05-27T07:28:20.404619+0000","last_unstale":"2024-05-27T07:33:35.437781+0000","last_undegraded":"2024-05-27T07:33:35.437781+0000","last_fullsized":"2024-05-27T07:33:35.437781+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:17.671250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337323+0000","last_change":"2024-05-27T07:28:24.490661+0000","last_active":"2024-05-27T07:33:35.337323+0000","last_peered":"2024-05-27T07:33:35.337323+0000","last_clean":"2024-05-27T07:33:35.337323+0000","last_became_active":"2024-05-27T07:28:24.487299+0000","last_became_peered":"2024-05-27T07:28:24.487299+0000","last_unstale":"2024-05-27T07:33:35.337323+0000","last_undegraded":"2024-05-27T07:33:35.337323+0000","last_fullsized":"2024-05-27T07:33:35.337323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:50:27.291721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243449+0000","last_change":"2024-05-27T07:28:37.970463+0000","last_active":"2024-05-27T07:33:35.243449+0000","last_peered":"2024-05-27T07:33:35.243449+0000","last_clean":"2024-05-27T07:33:35.243449+0000","last_became_active":"2024-05-27T07:28:37.968068+0000","last_became_peered":"2024-05-27T07:28:37.968068+0000","last_unstale":"2024-05-27T07:33:35.243449+0000","last_undegraded":"2024-05-27T07:33:35.243449+0000","last_fullsized":"2024-05-27T07:33:35.243449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:22:00.825999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434404+0000","last_change":"2024-05-27T07:28:20.406203+0000","last_active":"2024-05-27T07:33:35.434404+0000","last_peered":"2024-05-27T07:33:35.434404+0000","last_clean":"2024-05-27T07:33:35.434404+0000","last_became_active":"2024-05-27T07:28:20.405404+0000","last_became_peered":"2024-05-27T07:28:20.405404+0000","last_unstale":"2024-05-27T07:33:35.434404+0000","last_undegraded":"2024-05-27T07:33:35.434404+0000","last_fullsized":"2024-05-27T07:33:35.434404+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:15.274419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434004+0000","last_change":"2024-05-27T07:28:24.473747+0000","last_active":"2024-05-27T07:33:35.434004+0000","last_peered":"2024-05-27T07:33:35.434004+0000","last_clean":"2024-05-27T07:33:35.434004+0000","last_became_active":"2024-05-27T07:28:24.473313+0000","last_became_peered":"2024-05-27T07:28:24.473313+0000","last_unstale":"2024-05-27T07:33:35.434004+0000","last_undegraded":"2024-05-27T07:33:35.434004+0000","last_fullsized":"2024-05-27T07:33:35.434004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:19:40.107840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434078+0000","last_change":"2024-05-27T07:28:37.907410+0000","last_active":"2024-05-27T07:33:35.434078+0000","last_peered":"2024-05-27T07:33:35.434078+0000","last_clean":"2024-05-27T07:33:35.434078+0000","last_became_active":"2024-05-27T07:28:37.906753+0000","last_became_peered":"2024-05-27T07:28:37.906753+0000","last_unstale":"2024-05-27T07:33:35.434078+0000","last_undegraded":"2024-05-27T07:33:35.434078+0000","last_fullsized":"2024-05-27T07:33:35.434078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:53.612015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437366+0000","last_change":"2024-05-27T07:28:20.744068+0000","last_active":"2024-05-27T07:33:35.437366+0000","last_peered":"2024-05-27T07:33:35.437366+0000","last_clean":"2024-05-27T07:33:35.437366+0000","last_became_active":"2024-05-27T07:28:20.742686+0000","last_became_peered":"2024-05-27T07:28:20.742686+0000","last_unstale":"2024-05-27T07:33:35.437366+0000","last_undegraded":"2024-05-27T07:33:35.437366+0000","last_fullsized":"2024-05-27T07:33:35.437366+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:54.361007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436607+0000","last_change":"2024-05-27T07:28:24.475743+0000","last_active":"2024-05-27T07:33:35.436607+0000","last_peered":"2024-05-27T07:33:35.436607+0000","last_clean":"2024-05-27T07:33:35.436607+0000","last_became_active":"2024-05-27T07:28:24.474742+0000","last_became_peered":"2024-05-27T07:28:24.474742+0000","last_unstale":"2024-05-27T07:33:35.436607+0000","last_undegraded":"2024-05-27T07:33:35.436607+0000","last_fullsized":"2024-05-27T07:33:35.436607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:58:47.061573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431413+0000","last_change":"2024-05-27T07:28:37.914935+0000","last_active":"2024-05-27T07:33:35.431413+0000","last_peered":"2024-05-27T07:33:35.431413+0000","last_clean":"2024-05-27T07:33:35.431413+0000","last_became_active":"2024-05-27T07:28:37.911829+0000","last_became_peered":"2024-05-27T07:28:37.911829+0000","last_unstale":"2024-05-27T07:33:35.431413+0000","last_undegraded":"2024-05-27T07:33:35.431413+0000","last_fullsized":"2024-05-27T07:33:35.431413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:29.260539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"92'39","reported_seq":95,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.608079+0000","last_change":"2024-05-27T07:28:20.412554+0000","last_active":"2024-05-27T07:33:35.608079+0000","last_peered":"2024-05-27T07:33:35.608079+0000","last_clean":"2024-05-27T07:33:35.608079+0000","last_became_active":"2024-05-27T07:28:20.412024+0000","last_became_peered":"2024-05-27T07:28:20.412024+0000","last_unstale":"2024-05-27T07:33:35.608079+0000","last_undegraded":"2024-05-27T07:33:35.608079+0000","last_fullsized":"2024-05-27T07:33:35.608079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:29.244093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435695+0000","last_change":"2024-05-27T07:28:24.481966+0000","last_active":"2024-05-27T07:33:35.435695+0000","last_peered":"2024-05-27T07:33:35.435695+0000","last_clean":"2024-05-27T07:33:35.435695+0000","last_became_active":"2024-05-27T07:28:24.480775+0000","last_became_peered":"2024-05-27T07:28:24.480775+0000","last_unstale":"2024-05-27T07:33:35.435695+0000","last_undegraded":"2024-05-27T07:33:35.435695+0000","last_fullsized":"2024-05-27T07:33:35.435695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:03.018174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242513+0000","last_change":"2024-05-27T07:28:37.902893+0000","last_active":"2024-05-27T07:33:35.242513+0000","last_peered":"2024-05-27T07:33:35.242513+0000","last_clean":"2024-05-27T07:33:35.242513+0000","last_became_active":"2024-05-27T07:28:37.898447+0000","last_became_peered":"2024-05-27T07:28:37.898447+0000","last_unstale":"2024-05-27T07:33:35.242513+0000","last_undegraded":"2024-05-27T07:33:35.242513+0000","last_fullsized":"2024-05-27T07:33:35.242513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:55:49.754803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370906+0000","last_change":"2024-05-27T07:28:20.408072+0000","last_active":"2024-05-27T07:33:35.370906+0000","last_peered":"2024-05-27T07:33:35.370906+0000","last_clean":"2024-05-27T07:33:35.370906+0000","last_became_active":"2024-05-27T07:28:20.407174+0000","last_became_peered":"2024-05-27T07:28:20.407174+0000","last_unstale":"2024-05-27T07:33:35.370906+0000","last_undegraded":"2024-05-27T07:33:35.370906+0000","last_fullsized":"2024-05-27T07:33:35.370906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:15:52.984967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437053+0000","last_change":"2024-05-27T07:28:24.475891+0000","last_active":"2024-05-27T07:33:35.437053+0000","last_peered":"2024-05-27T07:33:35.437053+0000","last_clean":"2024-05-27T07:33:35.437053+0000","last_became_active":"2024-05-27T07:28:24.475498+0000","last_became_peered":"2024-05-27T07:28:24.475498+0000","last_unstale":"2024-05-27T07:33:35.437053+0000","last_undegraded":"2024-05-27T07:33:35.437053+0000","last_fullsized":"2024-05-27T07:33:35.437053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:50.669118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384465+0000","last_change":"2024-05-27T07:29:04.891583+0000","last_active":"2024-05-27T07:33:35.384465+0000","last_peered":"2024-05-27T07:33:35.384465+0000","last_clean":"2024-05-27T07:33:35.384465+0000","last_became_active":"2024-05-27T07:29:04.890187+0000","last_became_peered":"2024-05-27T07:29:04.890187+0000","last_unstale":"2024-05-27T07:33:35.384465+0000","last_undegraded":"2024-05-27T07:33:35.384465+0000","last_fullsized":"2024-05-27T07:33:35.384465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:03.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436357+0000","last_change":"2024-05-27T07:28:24.478672+0000","last_active":"2024-05-27T07:33:35.436357+0000","last_peered":"2024-05-27T07:33:35.436357+0000","last_clean":"2024-05-27T07:33:35.436357+0000","last_became_active":"2024-05-27T07:28:24.478331+0000","last_became_peered":"2024-05-27T07:28:24.478331+0000","last_unstale":"2024-05-27T07:33:35.436357+0000","last_undegraded":"2024-05-27T07:33:35.436357+0000","last_fullsized":"2024-05-27T07:33:35.436357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:56:35.112855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437255+0000","last_change":"2024-05-27T07:28:24.487962+0000","last_active":"2024-05-27T07:33:35.437255+0000","last_peered":"2024-05-27T07:33:35.437255+0000","last_clean":"2024-05-27T07:33:35.437255+0000","last_became_active":"2024-05-27T07:28:24.487583+0000","last_became_peered":"2024-05-27T07:28:24.487583+0000","last_unstale":"2024-05-27T07:33:35.437255+0000","last_undegraded":"2024-05-27T07:33:35.437255+0000","last_fullsized":"2024-05-27T07:33:35.437255+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:25.758293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431727+0000","last_change":"2024-05-27T07:28:24.466058+0000","last_active":"2024-05-27T07:33:35.431727+0000","last_peered":"2024-05-27T07:33:35.431727+0000","last_clean":"2024-05-27T07:33:35.431727+0000","last_became_active":"2024-05-27T07:28:24.465709+0000","last_became_peered":"2024-05-27T07:28:24.465709+0000","last_unstale":"2024-05-27T07:33:35.431727+0000","last_undegraded":"2024-05-27T07:33:35.431727+0000","last_fullsized":"2024-05-27T07:33:35.431727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:38.064077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369986+0000","last_change":"2024-05-27T07:28:24.471585+0000","last_active":"2024-05-27T07:33:35.369986+0000","last_peered":"2024-05-27T07:33:35.369986+0000","last_clean":"2024-05-27T07:33:35.369986+0000","last_became_active":"2024-05-27T07:28:24.470948+0000","last_became_peered":"2024-05-27T07:28:24.470948+0000","last_unstale":"2024-05-27T07:33:35.369986+0000","last_undegraded":"2024-05-27T07:33:35.369986+0000","last_fullsized":"2024-05-27T07:33:35.369986+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:48.213323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385238+0000","last_change":"2024-05-27T07:28:24.477954+0000","last_active":"2024-05-27T07:33:35.385238+0000","last_peered":"2024-05-27T07:33:35.385238+0000","last_clean":"2024-05-27T07:33:35.385238+0000","last_became_active":"2024-05-27T07:28:24.477495+0000","last_became_peered":"2024-05-27T07:28:24.477495+0000","last_unstale":"2024-05-27T07:33:35.385238+0000","last_undegraded":"2024-05-27T07:33:35.385238+0000","last_fullsized":"2024-05-27T07:33:35.385238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:50.371439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437604+0000","last_change":"2024-05-27T07:28:24.476422+0000","last_active":"2024-05-27T07:33:35.437604+0000","last_peered":"2024-05-27T07:33:35.437604+0000","last_clean":"2024-05-27T07:33:35.437604+0000","last_became_active":"2024-05-27T07:28:24.476095+0000","last_became_peered":"2024-05-27T07:28:24.476095+0000","last_unstale":"2024-05-27T07:33:35.437604+0000","last_undegraded":"2024-05-27T07:33:35.437604+0000","last_fullsized":"2024-05-27T07:33:35.437604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:51.667169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337797+0000","last_change":"2024-05-27T07:28:24.486431+0000","last_active":"2024-05-27T07:33:35.337797+0000","last_peered":"2024-05-27T07:33:35.337797+0000","last_clean":"2024-05-27T07:33:35.337797+0000","last_became_active":"2024-05-27T07:28:24.482427+0000","last_became_peered":"2024-05-27T07:28:24.482427+0000","last_unstale":"2024-05-27T07:33:35.337797+0000","last_undegraded":"2024-05-27T07:33:35.337797+0000","last_fullsized":"2024-05-27T07:33:35.337797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:44:37.857793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243086+0000","last_change":"2024-05-27T07:28:24.484393+0000","last_active":"2024-05-27T07:33:35.243086+0000","last_peered":"2024-05-27T07:33:35.243086+0000","last_clean":"2024-05-27T07:33:35.243086+0000","last_became_active":"2024-05-27T07:28:24.483130+0000","last_became_peered":"2024-05-27T07:28:24.483130+0000","last_unstale":"2024-05-27T07:33:35.243086+0000","last_undegraded":"2024-05-27T07:33:35.243086+0000","last_fullsized":"2024-05-27T07:33:35.243086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:34.924573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337254+0000","last_change":"2024-05-27T07:28:24.486274+0000","last_active":"2024-05-27T07:33:35.337254+0000","last_peered":"2024-05-27T07:33:35.337254+0000","last_clean":"2024-05-27T07:33:35.337254+0000","last_became_active":"2024-05-27T07:28:24.485880+0000","last_became_peered":"2024-05-27T07:28:24.485880+0000","last_unstale":"2024-05-27T07:33:35.337254+0000","last_undegraded":"2024-05-27T07:33:35.337254+0000","last_fullsized":"2024-05-27T07:33:35.337254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:33.287615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436357+0000","last_change":"2024-05-27T07:28:24.477909+0000","last_active":"2024-05-27T07:33:35.436357+0000","last_peered":"2024-05-27T07:33:35.436357+0000","last_clean":"2024-05-27T07:33:35.436357+0000","last_became_active":"2024-05-27T07:28:24.477345+0000","last_became_peered":"2024-05-27T07:28:24.477345+0000","last_unstale":"2024-05-27T07:33:35.436357+0000","last_undegraded":"2024-05-27T07:33:35.436357+0000","last_fullsized":"2024-05-27T07:33:35.436357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:26:03.958489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370232+0000","last_change":"2024-05-27T07:28:24.477151+0000","last_active":"2024-05-27T07:33:35.370232+0000","last_peered":"2024-05-27T07:33:35.370232+0000","last_clean":"2024-05-27T07:33:35.370232+0000","last_became_active":"2024-05-27T07:28:24.474036+0000","last_became_peered":"2024-05-27T07:28:24.474036+0000","last_unstale":"2024-05-27T07:33:35.370232+0000","last_undegraded":"2024-05-27T07:33:35.370232+0000","last_fullsized":"2024-05-27T07:33:35.370232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:16.787252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385340+0000","last_change":"2024-05-27T07:28:24.483601+0000","last_active":"2024-05-27T07:33:35.385340+0000","last_peered":"2024-05-27T07:33:35.385340+0000","last_clean":"2024-05-27T07:33:35.385340+0000","last_became_active":"2024-05-27T07:28:24.481080+0000","last_became_peered":"2024-05-27T07:28:24.481080+0000","last_unstale":"2024-05-27T07:33:35.385340+0000","last_undegraded":"2024-05-27T07:33:35.385340+0000","last_fullsized":"2024-05-27T07:33:35.385340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:12.208506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":607900,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1496,"num_read_kb":866,"num_write":304,"num_write_kb":1455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2052096,"data_stored":1823700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645393,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712552,"statfs":{"total":95995035648,"available":95961653248,"internally_reserved":0,"allocated":1454080,"data_stored":925589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841624,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28872,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716280,"statfs":{"total":95995035648,"available":95965470720,"internally_reserved":0,"allocated":1900544,"data_stored":1327702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005152,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1474560,"data_stored":929921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201385,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":532529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397616,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32808,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712344,"statfs":{"total":95995035648,"available":95961440256,"internally_reserved":0,"allocated":1671168,"data_stored":1130206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561142,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":527785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1138688,"data_stored":602490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738530,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":529173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":604333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149837,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1597440,"data_stored":1052795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953606,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":534101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757375,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37108,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708044,"statfs":{"total":95995035648,"available":95957037056,"internally_reserved":0,"allocated":1814528,"data_stored":1255728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":77315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":212159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:33:44.007 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T07:33:44.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:43 smithi086 ceph-mon[34687]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:33:44.202 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:43 smithi086 ceph-mon[34687]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:33:44.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:43 smithi086 ceph-mon[34687]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:33:44.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:43 smithi086 ceph-mon[34687]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 880 B/s wr, 16 op/s 2024-05-27T07:33:44.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:43 smithi149 ceph-mon[35408]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:33:44.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:43 smithi149 ceph-mon[35408]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:33:44.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:43 smithi149 ceph-mon[35408]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:33:44.206 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:43 smithi149 ceph-mon[35408]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 880 B/s wr, 16 op/s 2024-05-27T07:33:44.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:43 smithi063 ceph-mon[31613]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:33:44.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:43 smithi063 ceph-mon[31613]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:33:44.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:43 smithi063 ceph-mon[31613]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:33:44.210 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:43 smithi063 ceph-mon[31613]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 880 B/s wr, 16 op/s 2024-05-27T07:33:44.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.332+0000 7f8cfd600640 1 -- 172.21.15.63:0/3838213097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cf810c140 msgr2=0x7f8cf8105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:44.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.332+0000 7f8cfd600640 1 --2- 172.21.15.63:0/3838213097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cf810c140 0x7f8cf8105d80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009900 tx=0x7f8ce002a3f0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.332+0000 7f8cfd600640 1 -- 172.21.15.63:0/3838213097 shutdown_connections 2024-05-27T07:33:44.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.332+0000 7f8cfd600640 1 --2- 172.21.15.63:0/3838213097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf81063f0 0x7f8cf81127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.332+0000 7f8cfd600640 1 --2- 172.21.15.63:0/3838213097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cf810c140 0x7f8cf8105d80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.332+0000 7f8cfd600640 1 --2- 172.21.15.63:0/3838213097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf810bca0 0x7f8cf8105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.332+0000 7f8cfd600640 1 -- 172.21.15.63:0/3838213097 >> 172.21.15.63:0/3838213097 conn(0x7f8cf8101360 msgr2=0x7f8cf81037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:44.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.332+0000 7f8cfd600640 1 -- 172.21.15.63:0/3838213097 shutdown_connections 2024-05-27T07:33:44.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.332+0000 7f8cfd600640 1 -- 172.21.15.63:0/3838213097 wait complete. 2024-05-27T07:33:44.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.333+0000 7f8cfd600640 1 Processor -- start 2024-05-27T07:33:44.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.333+0000 7f8cfd600640 1 -- start start 2024-05-27T07:33:44.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cfd600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cf81063f0 0x7f8cf819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:44.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cfd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf810bca0 0x7f8cf819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:44.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cfd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf810c140 0x7f8cf81a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:44.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cfd600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8115d70 con 0x7f8cf81063f0 2024-05-27T07:33:44.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cfd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8115bf0 con 0x7f8cf810c140 2024-05-27T07:33:44.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cfd600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8115ef0 con 0x7f8cf810bca0 2024-05-27T07:33:44.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cf81063f0 0x7f8cf819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:44.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cf7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cf81063f0 0x7f8cf819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42378/0 (socket says 172.21.15.63:42378) 2024-05-27T07:33:44.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cf7e00640 1 -- 172.21.15.63:0/3842836168 learned_addr learned my addr 172.21.15.63:0/3842836168 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:44.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cfcc00640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf810c140 0x7f8cf81a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:44.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.334+0000 7f8cefe00640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf810bca0 0x7f8cf819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:44.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.335+0000 7f8cf7e00640 1 -- 172.21.15.63:0/3842836168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf810bca0 msgr2=0x7f8cf819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:44.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.335+0000 7f8cf7e00640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf810bca0 0x7f8cf819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.335+0000 7f8cf7e00640 1 -- 172.21.15.63:0/3842836168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf810c140 msgr2=0x7f8cf81a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:44.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.335+0000 7f8cf7e00640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf810c140 0x7f8cf81a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.335+0000 7f8cf7e00640 1 -- 172.21.15.63:0/3842836168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf81a1650 con 0x7f8cf81063f0 2024-05-27T07:33:44.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.335+0000 7f8cf7e00640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cf81063f0 0x7f8cf819c7f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800e9c0 tx=0x7f8ce800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:44.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.335+0000 7f8cf5600640 1 -- 172.21.15.63:0/3842836168 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce800cd80 con 0x7f8cf81063f0 2024-05-27T07:33:44.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.336+0000 7f8cf5600640 1 -- 172.21.15.63:0/3842836168 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ce80054b0 con 0x7f8cf81063f0 2024-05-27T07:33:44.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.336+0000 7f8cf5600640 1 -- 172.21.15.63:0/3842836168 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8010430 con 0x7f8cf81063f0 2024-05-27T07:33:44.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.336+0000 7f8cfd600640 1 -- 172.21.15.63:0/3842836168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf81a1940 con 0x7f8cf81063f0 2024-05-27T07:33:44.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.336+0000 7f8cfd600640 1 -- 172.21.15.63:0/3842836168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf81a1e10 con 0x7f8cf81063f0 2024-05-27T07:33:44.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.337+0000 7f8cfd600640 1 -- 172.21.15.63:0/3842836168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf8106f80 con 0x7f8cf81063f0 2024-05-27T07:33:44.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.340+0000 7f8cf5600640 1 -- 172.21.15.63:0/3842836168 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f8ce8004f20 con 0x7f8cf81063f0 2024-05-27T07:33:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.341+0000 7f8cf5600640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8cdc0b2a70 0x7f8cdc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.342+0000 7f8cf5600640 1 -- 172.21.15.63:0/3842836168 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f8ce80e34e0 con 0x7f8cf81063f0 2024-05-27T07:33:44.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.342+0000 7f8cefe00640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8cdc0b2a70 0x7f8cdc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:44.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.342+0000 7f8cefe00640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8cdc0b2a70 0x7f8cdc0b4f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8ce002a910 tx=0x7f8ce0005a40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:44.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.346+0000 7f8cf5600640 1 -- 172.21.15.63:0/3842836168 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ce80aa8a0 con 0x7f8cf81063f0 2024-05-27T07:33:44.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.569+0000 7f8cfd600640 1 -- 172.21.15.63:0/3842836168 --> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8cf8101e80 con 0x7f8cdc0b2a70 2024-05-27T07:33:44.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.581+0000 7f8cf5600640 1 -- 172.21.15.63:0/3842836168 <== mgr.15216 v2:172.21.15.63:6800/1343315489 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339430 (secure 0 0 0) 0x7f8cf8101e80 con 0x7f8cdc0b2a70 2024-05-27T07:33:44.579 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:33:44.581 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-27T07:33:44.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 -- 172.21.15.63:0/3842836168 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8cdc0b2a70 msgr2=0x7f8cdc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8cdc0b2a70 0x7f8cdc0b4f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8ce002a910 tx=0x7f8ce0005a40 comp rx=0 tx=0).stop 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 -- 172.21.15.63:0/3842836168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cf81063f0 msgr2=0x7f8cf819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cf81063f0 0x7f8cf819c7f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800e9c0 tx=0x7f8ce800ee80 comp rx=0 tx=0).stop 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 -- 172.21.15.63:0/3842836168 shutdown_connections 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8cdc0b2a70 0x7f8cdc0b4f80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cf810c140 0x7f8cf81a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cf810bca0 0x7f8cf819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 --2- 172.21.15.63:0/3842836168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cf81063f0 0x7f8cf819c7f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 -- 172.21.15.63:0/3842836168 >> 172.21.15.63:0/3842836168 conn(0x7f8cf8101360 msgr2=0x7f8cf8101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 -- 172.21.15.63:0/3842836168 shutdown_connections 2024-05-27T07:33:44.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.586+0000 7f8cfd600640 1 -- 172.21.15.63:0/3842836168 wait complete. 2024-05-27T07:33:44.604 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T07:33:43.267988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1198502,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1640,"num_read_kb":1114,"num_write":441,"num_write_kb":3607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377776,"kb_used_data":16112,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124564048,"statfs":{"total":1151940427776,"available":1151553585152,"internally_reserved":0,"allocated":16498688,"data_stored":9952352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3302,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":201,"num_write":17,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.976226"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385739+0000","last_change":"2024-05-27T07:28:24.485151+0000","last_active":"2024-05-27T07:33:35.385739+0000","last_peered":"2024-05-27T07:33:35.385739+0000","last_clean":"2024-05-27T07:33:35.385739+0000","last_became_active":"2024-05-27T07:28:24.483965+0000","last_became_peered":"2024-05-27T07:28:24.483965+0000","last_unstale":"2024-05-27T07:33:35.385739+0000","last_undegraded":"2024-05-27T07:33:35.385739+0000","last_fullsized":"2024-05-27T07:33:35.385739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:18.376755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431746+0000","last_change":"2024-05-27T07:28:24.483608+0000","last_active":"2024-05-27T07:33:35.431746+0000","last_peered":"2024-05-27T07:33:35.431746+0000","last_clean":"2024-05-27T07:33:35.431746+0000","last_became_active":"2024-05-27T07:28:24.479875+0000","last_became_peered":"2024-05-27T07:28:24.479875+0000","last_unstale":"2024-05-27T07:33:35.431746+0000","last_undegraded":"2024-05-27T07:33:35.431746+0000","last_fullsized":"2024-05-27T07:33:35.431746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:30.129708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431562+0000","last_change":"2024-05-27T07:28:24.470104+0000","last_active":"2024-05-27T07:33:35.431562+0000","last_peered":"2024-05-27T07:33:35.431562+0000","last_clean":"2024-05-27T07:33:35.431562+0000","last_became_active":"2024-05-27T07:28:24.469756+0000","last_became_peered":"2024-05-27T07:28:24.469756+0000","last_unstale":"2024-05-27T07:33:35.431562+0000","last_undegraded":"2024-05-27T07:33:35.431562+0000","last_fullsized":"2024-05-27T07:33:35.431562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:36.907691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243760+0000","last_change":"2024-05-27T07:28:24.472381+0000","last_active":"2024-05-27T07:33:35.243760+0000","last_peered":"2024-05-27T07:33:35.243760+0000","last_clean":"2024-05-27T07:33:35.243760+0000","last_became_active":"2024-05-27T07:28:24.469959+0000","last_became_peered":"2024-05-27T07:28:24.469959+0000","last_unstale":"2024-05-27T07:33:35.243760+0000","last_undegraded":"2024-05-27T07:33:35.243760+0000","last_fullsized":"2024-05-27T07:33:35.243760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:15.267554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434585+0000","last_change":"2024-05-27T07:28:24.474210+0000","last_active":"2024-05-27T07:33:35.434585+0000","last_peered":"2024-05-27T07:33:35.434585+0000","last_clean":"2024-05-27T07:33:35.434585+0000","last_became_active":"2024-05-27T07:28:24.473688+0000","last_became_peered":"2024-05-27T07:28:24.473688+0000","last_unstale":"2024-05-27T07:33:35.434585+0000","last_undegraded":"2024-05-27T07:33:35.434585+0000","last_fullsized":"2024-05-27T07:33:35.434585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:38.733536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401818+0000","last_change":"2024-05-27T07:28:24.484679+0000","last_active":"2024-05-27T07:33:35.401818+0000","last_peered":"2024-05-27T07:33:35.401818+0000","last_clean":"2024-05-27T07:33:35.401818+0000","last_became_active":"2024-05-27T07:28:24.484251+0000","last_became_peered":"2024-05-27T07:28:24.484251+0000","last_unstale":"2024-05-27T07:33:35.401818+0000","last_undegraded":"2024-05-27T07:33:35.401818+0000","last_fullsized":"2024-05-27T07:33:35.401818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:28:09.507871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434697+0000","last_change":"2024-05-27T07:28:24.472694+0000","last_active":"2024-05-27T07:33:35.434697+0000","last_peered":"2024-05-27T07:33:35.434697+0000","last_clean":"2024-05-27T07:33:35.434697+0000","last_became_active":"2024-05-27T07:28:24.472100+0000","last_became_peered":"2024-05-27T07:28:24.472100+0000","last_unstale":"2024-05-27T07:33:35.434697+0000","last_undegraded":"2024-05-27T07:33:35.434697+0000","last_fullsized":"2024-05-27T07:33:35.434697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336904+0000","last_change":"2024-05-27T07:28:24.488577+0000","last_active":"2024-05-27T07:33:35.336904+0000","last_peered":"2024-05-27T07:33:35.336904+0000","last_clean":"2024-05-27T07:33:35.336904+0000","last_became_active":"2024-05-27T07:28:24.488180+0000","last_became_peered":"2024-05-27T07:28:24.488180+0000","last_unstale":"2024-05-27T07:33:35.336904+0000","last_undegraded":"2024-05-27T07:33:35.336904+0000","last_fullsized":"2024-05-27T07:33:35.336904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:17.130058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437122+0000","last_change":"2024-05-27T07:28:24.480109+0000","last_active":"2024-05-27T07:33:35.437122+0000","last_peered":"2024-05-27T07:33:35.437122+0000","last_clean":"2024-05-27T07:33:35.437122+0000","last_became_active":"2024-05-27T07:28:24.479722+0000","last_became_peered":"2024-05-27T07:28:24.479722+0000","last_unstale":"2024-05-27T07:33:35.437122+0000","last_undegraded":"2024-05-27T07:33:35.437122+0000","last_fullsized":"2024-05-27T07:33:35.437122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:27:36.074612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.402840+0000","last_change":"2024-05-27T07:28:24.490389+0000","last_active":"2024-05-27T07:33:35.402840+0000","last_peered":"2024-05-27T07:33:35.402840+0000","last_clean":"2024-05-27T07:33:35.402840+0000","last_became_active":"2024-05-27T07:28:24.488081+0000","last_became_peered":"2024-05-27T07:28:24.488081+0000","last_unstale":"2024-05-27T07:33:35.402840+0000","last_undegraded":"2024-05-27T07:33:35.402840+0000","last_fullsized":"2024-05-27T07:33:35.402840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:06.318235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436339+0000","last_change":"2024-05-27T07:28:24.477062+0000","last_active":"2024-05-27T07:33:35.436339+0000","last_peered":"2024-05-27T07:33:35.436339+0000","last_clean":"2024-05-27T07:33:35.436339+0000","last_became_active":"2024-05-27T07:28:24.475322+0000","last_became_peered":"2024-05-27T07:28:24.475322+0000","last_unstale":"2024-05-27T07:33:35.436339+0000","last_undegraded":"2024-05-27T07:33:35.436339+0000","last_fullsized":"2024-05-27T07:33:35.436339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:30.365295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.238628+0000","last_change":"2024-05-27T07:29:04.901715+0000","last_active":"2024-05-27T07:33:35.238628+0000","last_peered":"2024-05-27T07:33:35.238628+0000","last_clean":"2024-05-27T07:33:35.238628+0000","last_became_active":"2024-05-27T07:29:04.900968+0000","last_became_peered":"2024-05-27T07:29:04.900968+0000","last_unstale":"2024-05-27T07:33:35.238628+0000","last_undegraded":"2024-05-27T07:33:35.238628+0000","last_fullsized":"2024-05-27T07:33:35.238628+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:16:34.033701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370124+0000","last_change":"2024-05-27T07:28:24.477504+0000","last_active":"2024-05-27T07:33:35.370124+0000","last_peered":"2024-05-27T07:33:35.370124+0000","last_clean":"2024-05-27T07:33:35.370124+0000","last_became_active":"2024-05-27T07:28:24.476709+0000","last_became_peered":"2024-05-27T07:28:24.476709+0000","last_unstale":"2024-05-27T07:33:35.370124+0000","last_undegraded":"2024-05-27T07:33:35.370124+0000","last_fullsized":"2024-05-27T07:33:35.370124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:05:48.456347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.432633+0000","last_change":"2024-05-27T07:28:24.485567+0000","last_active":"2024-05-27T07:33:35.432633+0000","last_peered":"2024-05-27T07:33:35.432633+0000","last_clean":"2024-05-27T07:33:35.432633+0000","last_became_active":"2024-05-27T07:28:24.485175+0000","last_became_peered":"2024-05-27T07:28:24.485175+0000","last_unstale":"2024-05-27T07:33:35.432633+0000","last_undegraded":"2024-05-27T07:33:35.432633+0000","last_fullsized":"2024-05-27T07:33:35.432633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:25:30.062813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431453+0000","last_change":"2024-05-27T07:29:04.900657+0000","last_active":"2024-05-27T07:33:35.431453+0000","last_peered":"2024-05-27T07:33:35.431453+0000","last_clean":"2024-05-27T07:33:35.431453+0000","last_became_active":"2024-05-27T07:29:04.900093+0000","last_became_peered":"2024-05-27T07:29:04.900093+0000","last_unstale":"2024-05-27T07:33:35.431453+0000","last_undegraded":"2024-05-27T07:33:35.431453+0000","last_fullsized":"2024-05-27T07:33:35.431453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:28:04.157825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434350+0000","last_change":"2024-05-27T07:28:24.470503+0000","last_active":"2024-05-27T07:33:35.434350+0000","last_peered":"2024-05-27T07:33:35.434350+0000","last_clean":"2024-05-27T07:33:35.434350+0000","last_became_active":"2024-05-27T07:28:24.470128+0000","last_became_peered":"2024-05-27T07:28:24.470128+0000","last_unstale":"2024-05-27T07:33:35.434350+0000","last_undegraded":"2024-05-27T07:33:35.434350+0000","last_fullsized":"2024-05-27T07:33:35.434350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:46:43.149635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385138+0000","last_change":"2024-05-27T07:28:24.491382+0000","last_active":"2024-05-27T07:33:35.385138+0000","last_peered":"2024-05-27T07:33:35.385138+0000","last_clean":"2024-05-27T07:33:35.385138+0000","last_became_active":"2024-05-27T07:28:24.486155+0000","last_became_peered":"2024-05-27T07:28:24.486155+0000","last_unstale":"2024-05-27T07:33:35.385138+0000","last_undegraded":"2024-05-27T07:33:35.385138+0000","last_fullsized":"2024-05-27T07:33:35.385138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:48.408573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.239172+0000","last_change":"2024-05-27T07:28:24.489140+0000","last_active":"2024-05-27T07:33:35.239172+0000","last_peered":"2024-05-27T07:33:35.239172+0000","last_clean":"2024-05-27T07:33:35.239172+0000","last_became_active":"2024-05-27T07:28:24.488733+0000","last_became_peered":"2024-05-27T07:28:24.488733+0000","last_unstale":"2024-05-27T07:33:35.239172+0000","last_undegraded":"2024-05-27T07:33:35.239172+0000","last_fullsized":"2024-05-27T07:33:35.239172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:02:57.711197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437255+0000","last_change":"2024-05-27T07:28:24.486022+0000","last_active":"2024-05-27T07:33:35.437255+0000","last_peered":"2024-05-27T07:33:35.437255+0000","last_clean":"2024-05-27T07:33:35.437255+0000","last_became_active":"2024-05-27T07:28:24.485639+0000","last_became_peered":"2024-05-27T07:28:24.485639+0000","last_unstale":"2024-05-27T07:33:35.437255+0000","last_undegraded":"2024-05-27T07:33:35.437255+0000","last_fullsized":"2024-05-27T07:33:35.437255+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:07.629402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435201+0000","last_change":"2024-05-27T07:28:37.898984+0000","last_active":"2024-05-27T07:33:35.435201+0000","last_peered":"2024-05-27T07:33:35.435201+0000","last_clean":"2024-05-27T07:33:35.435201+0000","last_became_active":"2024-05-27T07:28:37.898132+0000","last_became_peered":"2024-05-27T07:28:37.898132+0000","last_unstale":"2024-05-27T07:33:35.435201+0000","last_undegraded":"2024-05-27T07:33:35.435201+0000","last_fullsized":"2024-05-27T07:33:35.435201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:51.987386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.239432+0000","last_change":"2024-05-27T07:28:20.402788+0000","last_active":"2024-05-27T07:33:35.239432+0000","last_peered":"2024-05-27T07:33:35.239432+0000","last_clean":"2024-05-27T07:33:35.239432+0000","last_became_active":"2024-05-27T07:28:20.402041+0000","last_became_peered":"2024-05-27T07:28:20.402041+0000","last_unstale":"2024-05-27T07:33:35.239432+0000","last_undegraded":"2024-05-27T07:33:35.239432+0000","last_fullsized":"2024-05-27T07:33:35.239432+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:44.421979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337479+0000","last_change":"2024-05-27T07:28:24.485193+0000","last_active":"2024-05-27T07:33:35.337479+0000","last_peered":"2024-05-27T07:33:35.337479+0000","last_clean":"2024-05-27T07:33:35.337479+0000","last_became_active":"2024-05-27T07:28:24.484086+0000","last_became_peered":"2024-05-27T07:28:24.484086+0000","last_unstale":"2024-05-27T07:33:35.337479+0000","last_undegraded":"2024-05-27T07:33:35.337479+0000","last_fullsized":"2024-05-27T07:33:35.337479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:10.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242950+0000","last_change":"2024-05-27T07:28:37.902602+0000","last_active":"2024-05-27T07:33:35.242950+0000","last_peered":"2024-05-27T07:33:35.242950+0000","last_clean":"2024-05-27T07:33:35.242950+0000","last_became_active":"2024-05-27T07:28:37.902120+0000","last_became_peered":"2024-05-27T07:28:37.902120+0000","last_unstale":"2024-05-27T07:33:35.242950+0000","last_undegraded":"2024-05-27T07:33:35.242950+0000","last_fullsized":"2024-05-27T07:33:35.242950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:21.209851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337842+0000","last_change":"2024-05-27T07:28:20.465500+0000","last_active":"2024-05-27T07:33:35.337842+0000","last_peered":"2024-05-27T07:33:35.337842+0000","last_clean":"2024-05-27T07:33:35.337842+0000","last_became_active":"2024-05-27T07:28:20.464951+0000","last_became_peered":"2024-05-27T07:28:20.464951+0000","last_unstale":"2024-05-27T07:33:35.337842+0000","last_undegraded":"2024-05-27T07:33:35.337842+0000","last_fullsized":"2024-05-27T07:33:35.337842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:23:25.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.432047+0000","last_change":"2024-05-27T07:28:24.467655+0000","last_active":"2024-05-27T07:33:35.432047+0000","last_peered":"2024-05-27T07:33:35.432047+0000","last_clean":"2024-05-27T07:33:35.432047+0000","last_became_active":"2024-05-27T07:28:24.467194+0000","last_became_peered":"2024-05-27T07:28:24.467194+0000","last_unstale":"2024-05-27T07:33:35.432047+0000","last_undegraded":"2024-05-27T07:33:35.432047+0000","last_fullsized":"2024-05-27T07:33:35.432047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:38.024514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243715+0000","last_change":"2024-05-27T07:29:04.975679+0000","last_active":"2024-05-27T07:33:35.243715+0000","last_peered":"2024-05-27T07:33:35.243715+0000","last_clean":"2024-05-27T07:33:35.243715+0000","last_became_active":"2024-05-27T07:29:04.974323+0000","last_became_peered":"2024-05-27T07:29:04.974323+0000","last_unstale":"2024-05-27T07:33:35.243715+0000","last_undegraded":"2024-05-27T07:33:35.243715+0000","last_fullsized":"2024-05-27T07:33:35.243715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:50.489737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435362+0000","last_change":"2024-05-27T07:28:20.408117+0000","last_active":"2024-05-27T07:33:35.435362+0000","last_peered":"2024-05-27T07:33:35.435362+0000","last_clean":"2024-05-27T07:33:35.435362+0000","last_became_active":"2024-05-27T07:28:20.407723+0000","last_became_peered":"2024-05-27T07:28:20.407723+0000","last_unstale":"2024-05-27T07:33:35.435362+0000","last_undegraded":"2024-05-27T07:33:35.435362+0000","last_fullsized":"2024-05-27T07:33:35.435362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:20:31.376656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437235+0000","last_change":"2024-05-27T07:28:24.490269+0000","last_active":"2024-05-27T07:33:35.437235+0000","last_peered":"2024-05-27T07:33:35.437235+0000","last_clean":"2024-05-27T07:33:35.437235+0000","last_became_active":"2024-05-27T07:28:24.488849+0000","last_became_peered":"2024-05-27T07:28:24.488849+0000","last_unstale":"2024-05-27T07:33:35.437235+0000","last_undegraded":"2024-05-27T07:33:35.437235+0000","last_fullsized":"2024-05-27T07:33:35.437235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:00.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242985+0000","last_change":"2024-05-27T07:28:37.908658+0000","last_active":"2024-05-27T07:33:35.242985+0000","last_peered":"2024-05-27T07:33:35.242985+0000","last_clean":"2024-05-27T07:33:35.242985+0000","last_became_active":"2024-05-27T07:28:37.908056+0000","last_became_peered":"2024-05-27T07:28:37.908056+0000","last_unstale":"2024-05-27T07:33:35.242985+0000","last_undegraded":"2024-05-27T07:33:35.242985+0000","last_fullsized":"2024-05-27T07:33:35.242985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:31.853606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436498+0000","last_change":"2024-05-27T07:28:20.410944+0000","last_active":"2024-05-27T07:33:35.436498+0000","last_peered":"2024-05-27T07:33:35.436498+0000","last_clean":"2024-05-27T07:33:35.436498+0000","last_became_active":"2024-05-27T07:28:20.409789+0000","last_became_peered":"2024-05-27T07:28:20.409789+0000","last_unstale":"2024-05-27T07:33:35.436498+0000","last_undegraded":"2024-05-27T07:33:35.436498+0000","last_fullsized":"2024-05-27T07:33:35.436498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:53:34.962397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244941+0000","last_change":"2024-05-27T07:28:24.482828+0000","last_active":"2024-05-27T07:33:35.244941+0000","last_peered":"2024-05-27T07:33:35.244941+0000","last_clean":"2024-05-27T07:33:35.244941+0000","last_became_active":"2024-05-27T07:28:24.482475+0000","last_became_peered":"2024-05-27T07:28:24.482475+0000","last_unstale":"2024-05-27T07:33:35.244941+0000","last_undegraded":"2024-05-27T07:33:35.244941+0000","last_fullsized":"2024-05-27T07:33:35.244941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:53:32.064441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436828+0000","last_change":"2024-05-27T07:29:04.905734+0000","last_active":"2024-05-27T07:33:35.436828+0000","last_peered":"2024-05-27T07:33:35.436828+0000","last_clean":"2024-05-27T07:33:35.436828+0000","last_became_active":"2024-05-27T07:29:04.902387+0000","last_became_peered":"2024-05-27T07:29:04.902387+0000","last_unstale":"2024-05-27T07:33:35.436828+0000","last_undegraded":"2024-05-27T07:33:35.436828+0000","last_fullsized":"2024-05-27T07:33:35.436828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:35.993193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":285,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.741281+0000","last_change":"2024-05-27T07:28:20.412258+0000","last_active":"2024-05-27T07:33:39.741281+0000","last_peered":"2024-05-27T07:33:39.741281+0000","last_clean":"2024-05-27T07:33:39.741281+0000","last_became_active":"2024-05-27T07:28:20.411764+0000","last_became_peered":"2024-05-27T07:28:20.411764+0000","last_unstale":"2024-05-27T07:33:39.741281+0000","last_undegraded":"2024-05-27T07:33:39.741281+0000","last_fullsized":"2024-05-27T07:33:39.741281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:38.462887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":231,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434849+0000","last_change":"2024-05-27T07:28:24.482179+0000","last_active":"2024-05-27T07:33:35.434849+0000","last_peered":"2024-05-27T07:33:35.434849+0000","last_clean":"2024-05-27T07:33:35.434849+0000","last_became_active":"2024-05-27T07:28:24.480682+0000","last_became_peered":"2024-05-27T07:28:24.480682+0000","last_unstale":"2024-05-27T07:33:35.434849+0000","last_undegraded":"2024-05-27T07:33:35.434849+0000","last_fullsized":"2024-05-27T07:33:35.434849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:51.403862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436757+0000","last_change":"2024-05-27T07:28:37.903716+0000","last_active":"2024-05-27T07:33:35.436757+0000","last_peered":"2024-05-27T07:33:35.436757+0000","last_clean":"2024-05-27T07:33:35.436757+0000","last_became_active":"2024-05-27T07:28:37.901083+0000","last_became_peered":"2024-05-27T07:28:37.901083+0000","last_unstale":"2024-05-27T07:33:35.436757+0000","last_undegraded":"2024-05-27T07:33:35.436757+0000","last_fullsized":"2024-05-27T07:33:35.436757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:44:11.606711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401548+0000","last_change":"2024-05-27T07:28:20.392441+0000","last_active":"2024-05-27T07:33:35.401548+0000","last_peered":"2024-05-27T07:33:35.401548+0000","last_clean":"2024-05-27T07:33:35.401548+0000","last_became_active":"2024-05-27T07:28:20.390983+0000","last_became_peered":"2024-05-27T07:28:20.390983+0000","last_unstale":"2024-05-27T07:33:35.401548+0000","last_undegraded":"2024-05-27T07:33:35.401548+0000","last_fullsized":"2024-05-27T07:33:35.401548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:33:30.338043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237689+0000","last_change":"2024-05-27T07:30:05.518915+0000","last_active":"2024-05-27T07:33:35.237689+0000","last_peered":"2024-05-27T07:33:35.237689+0000","last_clean":"2024-05-27T07:33:35.237689+0000","last_became_active":"2024-05-27T07:30:05.517894+0000","last_became_peered":"2024-05-27T07:30:05.517894+0000","last_unstale":"2024-05-27T07:33:35.237689+0000","last_undegraded":"2024-05-27T07:33:35.237689+0000","last_fullsized":"2024-05-27T07:33:35.237689+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:51.353713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,7],"acting":[9,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384762+0000","last_change":"2024-05-27T07:28:37.913352+0000","last_active":"2024-05-27T07:33:35.384762+0000","last_peered":"2024-05-27T07:33:35.384762+0000","last_clean":"2024-05-27T07:33:35.384762+0000","last_became_active":"2024-05-27T07:28:37.913027+0000","last_became_peered":"2024-05-27T07:28:37.913027+0000","last_unstale":"2024-05-27T07:33:35.384762+0000","last_undegraded":"2024-05-27T07:33:35.384762+0000","last_fullsized":"2024-05-27T07:33:35.384762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:03.485594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369823+0000","last_change":"2024-05-27T07:28:20.406094+0000","last_active":"2024-05-27T07:33:35.369823+0000","last_peered":"2024-05-27T07:33:35.369823+0000","last_clean":"2024-05-27T07:33:35.369823+0000","last_became_active":"2024-05-27T07:28:20.405641+0000","last_became_peered":"2024-05-27T07:28:20.405641+0000","last_unstale":"2024-05-27T07:33:35.369823+0000","last_undegraded":"2024-05-27T07:33:35.369823+0000","last_fullsized":"2024-05-27T07:33:35.369823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:16.598743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337048+0000","last_change":"2024-05-27T07:28:24.474696+0000","last_active":"2024-05-27T07:33:35.337048+0000","last_peered":"2024-05-27T07:33:35.337048+0000","last_clean":"2024-05-27T07:33:35.337048+0000","last_became_active":"2024-05-27T07:28:24.474319+0000","last_became_peered":"2024-05-27T07:28:24.474319+0000","last_unstale":"2024-05-27T07:33:35.337048+0000","last_undegraded":"2024-05-27T07:33:35.337048+0000","last_fullsized":"2024-05-27T07:33:35.337048+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:53.010898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431719+0000","last_change":"2024-05-27T07:28:37.910386+0000","last_active":"2024-05-27T07:33:35.431719+0000","last_peered":"2024-05-27T07:33:35.431719+0000","last_clean":"2024-05-27T07:33:35.431719+0000","last_became_active":"2024-05-27T07:28:37.909941+0000","last_became_peered":"2024-05-27T07:28:37.909941+0000","last_unstale":"2024-05-27T07:33:35.431719+0000","last_undegraded":"2024-05-27T07:33:35.431719+0000","last_fullsized":"2024-05-27T07:33:35.431719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:15.820318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242969+0000","last_change":"2024-05-27T07:28:20.411855+0000","last_active":"2024-05-27T07:33:35.242969+0000","last_peered":"2024-05-27T07:33:35.242969+0000","last_clean":"2024-05-27T07:33:35.242969+0000","last_became_active":"2024-05-27T07:28:20.411169+0000","last_became_peered":"2024-05-27T07:28:20.411169+0000","last_unstale":"2024-05-27T07:33:35.242969+0000","last_undegraded":"2024-05-27T07:33:35.242969+0000","last_fullsized":"2024-05-27T07:33:35.242969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:37.278507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237559+0000","last_change":"2024-05-27T07:28:24.485799+0000","last_active":"2024-05-27T07:33:35.237559+0000","last_peered":"2024-05-27T07:33:35.237559+0000","last_clean":"2024-05-27T07:33:35.237559+0000","last_became_active":"2024-05-27T07:28:24.485395+0000","last_became_peered":"2024-05-27T07:28:24.485395+0000","last_unstale":"2024-05-27T07:33:35.237559+0000","last_undegraded":"2024-05-27T07:33:35.237559+0000","last_fullsized":"2024-05-27T07:33:35.237559+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:11.178129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336979+0000","last_change":"2024-05-27T07:28:37.898929+0000","last_active":"2024-05-27T07:33:35.336979+0000","last_peered":"2024-05-27T07:33:35.336979+0000","last_clean":"2024-05-27T07:33:35.336979+0000","last_became_active":"2024-05-27T07:28:37.898371+0000","last_became_peered":"2024-05-27T07:28:37.898371+0000","last_unstale":"2024-05-27T07:33:35.336979+0000","last_undegraded":"2024-05-27T07:33:35.336979+0000","last_fullsized":"2024-05-27T07:33:35.336979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:48.867729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":280,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.740411+0000","last_change":"2024-05-27T07:28:20.407117+0000","last_active":"2024-05-27T07:33:39.740411+0000","last_peered":"2024-05-27T07:33:39.740411+0000","last_clean":"2024-05-27T07:33:39.740411+0000","last_became_active":"2024-05-27T07:28:20.403562+0000","last_became_peered":"2024-05-27T07:28:20.403562+0000","last_unstale":"2024-05-27T07:33:39.740411+0000","last_undegraded":"2024-05-27T07:33:39.740411+0000","last_fullsized":"2024-05-27T07:33:39.740411+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:25.903886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.438353+0000","last_change":"2024-05-27T07:28:24.489771+0000","last_active":"2024-05-27T07:33:35.438353+0000","last_peered":"2024-05-27T07:33:35.438353+0000","last_clean":"2024-05-27T07:33:35.438353+0000","last_became_active":"2024-05-27T07:28:24.489163+0000","last_became_peered":"2024-05-27T07:28:24.489163+0000","last_unstale":"2024-05-27T07:33:35.438353+0000","last_undegraded":"2024-05-27T07:33:35.438353+0000","last_fullsized":"2024-05-27T07:33:35.438353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:08.053634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237277+0000","last_change":"2024-05-27T07:28:37.914598+0000","last_active":"2024-05-27T07:33:35.237277+0000","last_peered":"2024-05-27T07:33:35.237277+0000","last_clean":"2024-05-27T07:33:35.237277+0000","last_became_active":"2024-05-27T07:28:37.913916+0000","last_became_peered":"2024-05-27T07:28:37.913916+0000","last_unstale":"2024-05-27T07:33:35.237277+0000","last_undegraded":"2024-05-27T07:33:35.237277+0000","last_fullsized":"2024-05-27T07:33:35.237277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:01.475789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436369+0000","last_change":"2024-05-27T07:29:04.901366+0000","last_active":"2024-05-27T07:33:35.436369+0000","last_peered":"2024-05-27T07:33:35.436369+0000","last_clean":"2024-05-27T07:33:35.436369+0000","last_became_active":"2024-05-27T07:29:04.900355+0000","last_became_peered":"2024-05-27T07:29:04.900355+0000","last_unstale":"2024-05-27T07:33:35.436369+0000","last_undegraded":"2024-05-27T07:33:35.436369+0000","last_fullsized":"2024-05-27T07:33:35.436369+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:10:03.349345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.432117+0000","last_change":"2024-05-27T07:28:24.477771+0000","last_active":"2024-05-27T07:33:35.432117+0000","last_peered":"2024-05-27T07:33:35.432117+0000","last_clean":"2024-05-27T07:33:35.432117+0000","last_became_active":"2024-05-27T07:28:24.474179+0000","last_became_peered":"2024-05-27T07:28:24.474179+0000","last_unstale":"2024-05-27T07:33:35.432117+0000","last_undegraded":"2024-05-27T07:33:35.432117+0000","last_fullsized":"2024-05-27T07:33:35.432117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:08:04.218264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243888+0000","last_change":"2024-05-27T07:29:04.896583+0000","last_active":"2024-05-27T07:33:35.243888+0000","last_peered":"2024-05-27T07:33:35.243888+0000","last_clean":"2024-05-27T07:33:35.243888+0000","last_became_active":"2024-05-27T07:29:04.894118+0000","last_became_peered":"2024-05-27T07:29:04.894118+0000","last_unstale":"2024-05-27T07:33:35.243888+0000","last_undegraded":"2024-05-27T07:33:35.243888+0000","last_fullsized":"2024-05-27T07:33:35.243888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:17.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244297+0000","last_change":"2024-05-27T07:28:20.466851+0000","last_active":"2024-05-27T07:33:35.244297+0000","last_peered":"2024-05-27T07:33:35.244297+0000","last_clean":"2024-05-27T07:33:35.244297+0000","last_became_active":"2024-05-27T07:28:20.465133+0000","last_became_peered":"2024-05-27T07:28:20.465133+0000","last_unstale":"2024-05-27T07:33:35.244297+0000","last_undegraded":"2024-05-27T07:33:35.244297+0000","last_fullsized":"2024-05-27T07:33:35.244297+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:11.025588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437815+0000","last_change":"2024-05-27T07:28:24.469924+0000","last_active":"2024-05-27T07:33:35.437815+0000","last_peered":"2024-05-27T07:33:35.437815+0000","last_clean":"2024-05-27T07:33:35.437815+0000","last_became_active":"2024-05-27T07:28:24.469559+0000","last_became_peered":"2024-05-27T07:28:24.469559+0000","last_unstale":"2024-05-27T07:33:35.437815+0000","last_undegraded":"2024-05-27T07:33:35.437815+0000","last_fullsized":"2024-05-27T07:33:35.437815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:57.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237901+0000","last_change":"2024-05-27T07:28:37.904549+0000","last_active":"2024-05-27T07:33:35.237901+0000","last_peered":"2024-05-27T07:33:35.237901+0000","last_clean":"2024-05-27T07:33:35.237901+0000","last_became_active":"2024-05-27T07:28:37.903882+0000","last_became_peered":"2024-05-27T07:28:37.903882+0000","last_unstale":"2024-05-27T07:33:35.237901+0000","last_undegraded":"2024-05-27T07:33:35.237901+0000","last_fullsized":"2024-05-27T07:33:35.237901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:41.607641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'44","reported_seq":563,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:40.741316+0000","last_change":"2024-05-27T07:28:20.410025+0000","last_active":"2024-05-27T07:33:40.741316+0000","last_peered":"2024-05-27T07:33:40.741316+0000","last_clean":"2024-05-27T07:33:40.741316+0000","last_became_active":"2024-05-27T07:28:20.409115+0000","last_became_peered":"2024-05-27T07:28:20.409115+0000","last_unstale":"2024-05-27T07:33:40.741316+0000","last_undegraded":"2024-05-27T07:33:40.741316+0000","last_fullsized":"2024-05-27T07:33:40.741316+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:56.834059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77315,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":471,"num_read_kb":309,"num_write":45,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436027+0000","last_change":"2024-05-27T07:28:24.469609+0000","last_active":"2024-05-27T07:33:35.436027+0000","last_peered":"2024-05-27T07:33:35.436027+0000","last_clean":"2024-05-27T07:33:35.436027+0000","last_became_active":"2024-05-27T07:28:24.469192+0000","last_became_peered":"2024-05-27T07:28:24.469192+0000","last_unstale":"2024-05-27T07:33:35.436027+0000","last_undegraded":"2024-05-27T07:33:35.436027+0000","last_fullsized":"2024-05-27T07:33:35.436027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:32:59.006298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.246044+0000","last_change":"2024-05-27T07:28:37.900745+0000","last_active":"2024-05-27T07:33:35.246044+0000","last_peered":"2024-05-27T07:33:35.246044+0000","last_clean":"2024-05-27T07:33:35.246044+0000","last_became_active":"2024-05-27T07:28:37.900205+0000","last_became_peered":"2024-05-27T07:28:37.900205+0000","last_unstale":"2024-05-27T07:33:35.246044+0000","last_undegraded":"2024-05-27T07:33:35.246044+0000","last_fullsized":"2024-05-27T07:33:35.246044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:52:32.235201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435338+0000","last_change":"2024-05-27T07:28:20.413380+0000","last_active":"2024-05-27T07:33:35.435338+0000","last_peered":"2024-05-27T07:33:35.435338+0000","last_clean":"2024-05-27T07:33:35.435338+0000","last_became_active":"2024-05-27T07:28:20.412839+0000","last_became_peered":"2024-05-27T07:28:20.412839+0000","last_unstale":"2024-05-27T07:33:35.435338+0000","last_undegraded":"2024-05-27T07:33:35.435338+0000","last_fullsized":"2024-05-27T07:33:35.435338+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:57.481203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369637+0000","last_change":"2024-05-27T07:28:24.474228+0000","last_active":"2024-05-27T07:33:35.369637+0000","last_peered":"2024-05-27T07:33:35.369637+0000","last_clean":"2024-05-27T07:33:35.369637+0000","last_became_active":"2024-05-27T07:28:24.473833+0000","last_became_peered":"2024-05-27T07:28:24.473833+0000","last_unstale":"2024-05-27T07:33:35.369637+0000","last_undegraded":"2024-05-27T07:33:35.369637+0000","last_fullsized":"2024-05-27T07:33:35.369637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:52.456730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384646+0000","last_change":"2024-05-27T07:28:37.917630+0000","last_active":"2024-05-27T07:33:35.384646+0000","last_peered":"2024-05-27T07:33:35.384646+0000","last_clean":"2024-05-27T07:33:35.384646+0000","last_became_active":"2024-05-27T07:28:37.917253+0000","last_became_peered":"2024-05-27T07:28:37.917253+0000","last_unstale":"2024-05-27T07:33:35.384646+0000","last_undegraded":"2024-05-27T07:33:35.384646+0000","last_fullsized":"2024-05-27T07:33:35.384646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:05.069946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243812+0000","last_change":"2024-05-27T07:28:20.408988+0000","last_active":"2024-05-27T07:33:35.243812+0000","last_peered":"2024-05-27T07:33:35.243812+0000","last_clean":"2024-05-27T07:33:35.243812+0000","last_became_active":"2024-05-27T07:28:20.408313+0000","last_became_peered":"2024-05-27T07:28:20.408313+0000","last_unstale":"2024-05-27T07:33:35.243812+0000","last_undegraded":"2024-05-27T07:33:35.243812+0000","last_fullsized":"2024-05-27T07:33:35.243812+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:17.480412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435459+0000","last_change":"2024-05-27T07:28:24.480096+0000","last_active":"2024-05-27T07:33:35.435459+0000","last_peered":"2024-05-27T07:33:35.435459+0000","last_clean":"2024-05-27T07:33:35.435459+0000","last_became_active":"2024-05-27T07:28:24.477988+0000","last_became_peered":"2024-05-27T07:28:24.477988+0000","last_unstale":"2024-05-27T07:33:35.435459+0000","last_undegraded":"2024-05-27T07:33:35.435459+0000","last_fullsized":"2024-05-27T07:33:35.435459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:13:59.860458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.238087+0000","last_change":"2024-05-27T07:28:37.917518+0000","last_active":"2024-05-27T07:33:35.238087+0000","last_peered":"2024-05-27T07:33:35.238087+0000","last_clean":"2024-05-27T07:33:35.238087+0000","last_became_active":"2024-05-27T07:28:37.916971+0000","last_became_peered":"2024-05-27T07:28:37.916971+0000","last_unstale":"2024-05-27T07:33:35.238087+0000","last_undegraded":"2024-05-27T07:33:35.238087+0000","last_fullsized":"2024-05-27T07:33:35.238087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:17:53.067747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.783309+0000","last_change":"2024-05-27T07:28:20.416766+0000","last_active":"2024-05-27T07:33:39.783309+0000","last_peered":"2024-05-27T07:33:39.783309+0000","last_clean":"2024-05-27T07:33:39.783309+0000","last_became_active":"2024-05-27T07:28:20.416297+0000","last_became_peered":"2024-05-27T07:28:20.416297+0000","last_unstale":"2024-05-27T07:33:39.783309+0000","last_undegraded":"2024-05-27T07:33:39.783309+0000","last_fullsized":"2024-05-27T07:33:39.783309+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:02.841494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2254,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":233,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435388+0000","last_change":"2024-05-27T07:28:24.486869+0000","last_active":"2024-05-27T07:33:35.435388+0000","last_peered":"2024-05-27T07:33:35.435388+0000","last_clean":"2024-05-27T07:33:35.435388+0000","last_became_active":"2024-05-27T07:28:24.486451+0000","last_became_peered":"2024-05-27T07:28:24.486451+0000","last_unstale":"2024-05-27T07:33:35.435388+0000","last_undegraded":"2024-05-27T07:33:35.435388+0000","last_fullsized":"2024-05-27T07:33:35.435388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:10.722659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370829+0000","last_change":"2024-05-27T07:29:04.901632+0000","last_active":"2024-05-27T07:33:35.370829+0000","last_peered":"2024-05-27T07:33:35.370829+0000","last_clean":"2024-05-27T07:33:35.370829+0000","last_became_active":"2024-05-27T07:29:04.900203+0000","last_became_peered":"2024-05-27T07:29:04.900203+0000","last_unstale":"2024-05-27T07:33:35.370829+0000","last_undegraded":"2024-05-27T07:33:35.370829+0000","last_fullsized":"2024-05-27T07:33:35.370829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:57.834889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":63,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434045+0000","last_change":"2024-05-27T07:28:20.468162+0000","last_active":"2024-05-27T07:33:35.434045+0000","last_peered":"2024-05-27T07:33:35.434045+0000","last_clean":"2024-05-27T07:33:35.434045+0000","last_became_active":"2024-05-27T07:28:20.465226+0000","last_became_peered":"2024-05-27T07:28:20.465226+0000","last_unstale":"2024-05-27T07:33:35.434045+0000","last_undegraded":"2024-05-27T07:33:35.434045+0000","last_fullsized":"2024-05-27T07:33:35.434045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:21:34.110356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243300+0000","last_change":"2024-05-27T07:28:24.485053+0000","last_active":"2024-05-27T07:33:35.243300+0000","last_peered":"2024-05-27T07:33:35.243300+0000","last_clean":"2024-05-27T07:33:35.243300+0000","last_became_active":"2024-05-27T07:28:24.484391+0000","last_became_peered":"2024-05-27T07:28:24.484391+0000","last_unstale":"2024-05-27T07:33:35.243300+0000","last_undegraded":"2024-05-27T07:33:35.243300+0000","last_fullsized":"2024-05-27T07:33:35.243300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:05.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243032+0000","last_change":"2024-05-27T07:28:37.904731+0000","last_active":"2024-05-27T07:33:35.243032+0000","last_peered":"2024-05-27T07:33:35.243032+0000","last_clean":"2024-05-27T07:33:35.243032+0000","last_became_active":"2024-05-27T07:28:37.904284+0000","last_became_peered":"2024-05-27T07:28:37.904284+0000","last_unstale":"2024-05-27T07:33:35.243032+0000","last_undegraded":"2024-05-27T07:33:35.243032+0000","last_fullsized":"2024-05-27T07:33:35.243032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:23.198465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435888+0000","last_change":"2024-05-27T07:28:20.406829+0000","last_active":"2024-05-27T07:33:35.435888+0000","last_peered":"2024-05-27T07:33:35.435888+0000","last_clean":"2024-05-27T07:33:35.435888+0000","last_became_active":"2024-05-27T07:28:20.405497+0000","last_became_peered":"2024-05-27T07:28:20.405497+0000","last_unstale":"2024-05-27T07:33:35.435888+0000","last_undegraded":"2024-05-27T07:33:35.435888+0000","last_fullsized":"2024-05-27T07:33:35.435888+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:14.225630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385238+0000","last_change":"2024-05-27T07:28:24.488023+0000","last_active":"2024-05-27T07:33:35.385238+0000","last_peered":"2024-05-27T07:33:35.385238+0000","last_clean":"2024-05-27T07:33:35.385238+0000","last_became_active":"2024-05-27T07:28:24.487652+0000","last_became_peered":"2024-05-27T07:28:24.487652+0000","last_unstale":"2024-05-27T07:33:35.385238+0000","last_undegraded":"2024-05-27T07:33:35.385238+0000","last_fullsized":"2024-05-27T07:33:35.385238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:25.951157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434437+0000","last_change":"2024-05-27T07:28:37.968034+0000","last_active":"2024-05-27T07:33:35.434437+0000","last_peered":"2024-05-27T07:33:35.434437+0000","last_clean":"2024-05-27T07:33:35.434437+0000","last_became_active":"2024-05-27T07:28:37.967127+0000","last_became_peered":"2024-05-27T07:28:37.967127+0000","last_unstale":"2024-05-27T07:33:35.434437+0000","last_undegraded":"2024-05-27T07:33:35.434437+0000","last_fullsized":"2024-05-27T07:33:35.434437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:49.353423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.238627+0000","last_change":"2024-05-27T07:28:20.465782+0000","last_active":"2024-05-27T07:33:35.238627+0000","last_peered":"2024-05-27T07:33:35.238627+0000","last_clean":"2024-05-27T07:33:35.238627+0000","last_became_active":"2024-05-27T07:28:20.465134+0000","last_became_peered":"2024-05-27T07:28:20.465134+0000","last_unstale":"2024-05-27T07:33:35.238627+0000","last_undegraded":"2024-05-27T07:33:35.238627+0000","last_fullsized":"2024-05-27T07:33:35.238627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:17.992092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401705+0000","last_change":"2024-05-27T07:28:24.484691+0000","last_active":"2024-05-27T07:33:35.401705+0000","last_peered":"2024-05-27T07:33:35.401705+0000","last_clean":"2024-05-27T07:33:35.401705+0000","last_became_active":"2024-05-27T07:28:24.484249+0000","last_became_peered":"2024-05-27T07:28:24.484249+0000","last_unstale":"2024-05-27T07:33:35.401705+0000","last_undegraded":"2024-05-27T07:33:35.401705+0000","last_fullsized":"2024-05-27T07:33:35.401705+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:25.753276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369751+0000","last_change":"2024-05-27T07:28:37.907849+0000","last_active":"2024-05-27T07:33:35.369751+0000","last_peered":"2024-05-27T07:33:35.369751+0000","last_clean":"2024-05-27T07:33:35.369751+0000","last_became_active":"2024-05-27T07:28:37.907105+0000","last_became_peered":"2024-05-27T07:28:37.907105+0000","last_unstale":"2024-05-27T07:33:35.369751+0000","last_undegraded":"2024-05-27T07:33:35.369751+0000","last_fullsized":"2024-05-27T07:33:35.369751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:54.075240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.433916+0000","last_change":"2024-05-27T07:28:20.411302+0000","last_active":"2024-05-27T07:33:35.433916+0000","last_peered":"2024-05-27T07:33:35.433916+0000","last_clean":"2024-05-27T07:33:35.433916+0000","last_became_active":"2024-05-27T07:28:20.409144+0000","last_became_peered":"2024-05-27T07:28:20.409144+0000","last_unstale":"2024-05-27T07:33:35.433916+0000","last_undegraded":"2024-05-27T07:33:35.433916+0000","last_fullsized":"2024-05-27T07:33:35.433916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:49.985962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401624+0000","last_change":"2024-05-27T07:28:24.487742+0000","last_active":"2024-05-27T07:33:35.401624+0000","last_peered":"2024-05-27T07:33:35.401624+0000","last_clean":"2024-05-27T07:33:35.401624+0000","last_became_active":"2024-05-27T07:28:24.485279+0000","last_became_peered":"2024-05-27T07:28:24.485279+0000","last_unstale":"2024-05-27T07:33:35.401624+0000","last_undegraded":"2024-05-27T07:33:35.401624+0000","last_fullsized":"2024-05-27T07:33:35.401624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:15.049572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.371172+0000","last_change":"2024-05-27T07:28:37.968780+0000","last_active":"2024-05-27T07:33:35.371172+0000","last_peered":"2024-05-27T07:33:35.371172+0000","last_clean":"2024-05-27T07:33:35.371172+0000","last_became_active":"2024-05-27T07:28:37.968052+0000","last_became_peered":"2024-05-27T07:28:37.968052+0000","last_unstale":"2024-05-27T07:33:35.371172+0000","last_undegraded":"2024-05-27T07:33:35.371172+0000","last_fullsized":"2024-05-27T07:33:35.371172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:51:26.787969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244844+0000","last_change":"2024-05-27T07:28:24.485280+0000","last_active":"2024-05-27T07:33:35.244844+0000","last_peered":"2024-05-27T07:33:35.244844+0000","last_clean":"2024-05-27T07:33:35.244844+0000","last_became_active":"2024-05-27T07:28:24.484882+0000","last_became_peered":"2024-05-27T07:28:24.484882+0000","last_unstale":"2024-05-27T07:33:35.244844+0000","last_undegraded":"2024-05-27T07:33:35.244844+0000","last_fullsized":"2024-05-27T07:33:35.244844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:28.643857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":280,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.781145+0000","last_change":"2024-05-27T07:28:20.462593+0000","last_active":"2024-05-27T07:33:39.781145+0000","last_peered":"2024-05-27T07:33:39.781145+0000","last_clean":"2024-05-27T07:33:39.781145+0000","last_became_active":"2024-05-27T07:28:20.461128+0000","last_became_peered":"2024-05-27T07:28:20.461128+0000","last_unstale":"2024-05-27T07:33:39.781145+0000","last_undegraded":"2024-05-27T07:33:39.781145+0000","last_fullsized":"2024-05-27T07:33:39.781145+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:24:42.013315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436013+0000","last_change":"2024-05-27T07:28:37.911546+0000","last_active":"2024-05-27T07:33:35.436013+0000","last_peered":"2024-05-27T07:33:35.436013+0000","last_clean":"2024-05-27T07:33:35.436013+0000","last_became_active":"2024-05-27T07:28:37.910777+0000","last_became_peered":"2024-05-27T07:28:37.910777+0000","last_unstale":"2024-05-27T07:33:35.436013+0000","last_undegraded":"2024-05-27T07:33:35.436013+0000","last_fullsized":"2024-05-27T07:33:35.436013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:13.344317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336963+0000","last_change":"2024-05-27T07:28:24.483946+0000","last_active":"2024-05-27T07:33:35.336963+0000","last_peered":"2024-05-27T07:33:35.336963+0000","last_clean":"2024-05-27T07:33:35.336963+0000","last_became_active":"2024-05-27T07:28:24.481424+0000","last_became_peered":"2024-05-27T07:28:24.481424+0000","last_unstale":"2024-05-27T07:33:35.336963+0000","last_undegraded":"2024-05-27T07:33:35.336963+0000","last_fullsized":"2024-05-27T07:33:35.336963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:08.254046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243763+0000","last_change":"2024-05-27T07:28:20.414250+0000","last_active":"2024-05-27T07:33:35.243763+0000","last_peered":"2024-05-27T07:33:35.243763+0000","last_clean":"2024-05-27T07:33:35.243763+0000","last_became_active":"2024-05-27T07:28:20.413676+0000","last_became_peered":"2024-05-27T07:28:20.413676+0000","last_unstale":"2024-05-27T07:33:35.243763+0000","last_undegraded":"2024-05-27T07:33:35.243763+0000","last_fullsized":"2024-05-27T07:33:35.243763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:10.226771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'99","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:37.400097+0000","last_change":"2024-05-27T07:26:26.055626+0000","last_active":"2024-05-27T07:33:37.400097+0000","last_peered":"2024-05-27T07:33:37.400097+0000","last_clean":"2024-05-27T07:33:37.400097+0000","last_became_active":"2024-05-27T07:26:26.037376+0000","last_became_peered":"2024-05-27T07:26:26.037376+0000","last_unstale":"2024-05-27T07:33:37.400097+0000","last_undegraded":"2024-05-27T07:33:37.400097+0000","last_fullsized":"2024-05-27T07:33:37.400097+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_clean_scrub_stamp":"2024-05-27T07:21:26.641949+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:16.741856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244470+0000","last_change":"2024-05-27T07:28:37.973078+0000","last_active":"2024-05-27T07:33:35.244470+0000","last_peered":"2024-05-27T07:33:35.244470+0000","last_clean":"2024-05-27T07:33:35.244470+0000","last_became_active":"2024-05-27T07:28:37.972381+0000","last_became_peered":"2024-05-27T07:28:37.972381+0000","last_unstale":"2024-05-27T07:33:35.244470+0000","last_undegraded":"2024-05-27T07:33:35.244470+0000","last_fullsized":"2024-05-27T07:33:35.244470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:18:16.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369481+0000","last_change":"2024-05-27T07:28:24.483512+0000","last_active":"2024-05-27T07:33:35.369481+0000","last_peered":"2024-05-27T07:33:35.369481+0000","last_clean":"2024-05-27T07:33:35.369481+0000","last_became_active":"2024-05-27T07:28:24.482655+0000","last_became_peered":"2024-05-27T07:28:24.482655+0000","last_unstale":"2024-05-27T07:33:35.369481+0000","last_undegraded":"2024-05-27T07:33:35.369481+0000","last_fullsized":"2024-05-27T07:33:35.369481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:01:07.898504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434986+0000","last_change":"2024-05-27T07:28:20.408953+0000","last_active":"2024-05-27T07:33:35.434986+0000","last_peered":"2024-05-27T07:33:35.434986+0000","last_clean":"2024-05-27T07:33:35.434986+0000","last_became_active":"2024-05-27T07:28:20.407631+0000","last_became_peered":"2024-05-27T07:28:20.407631+0000","last_unstale":"2024-05-27T07:33:35.434986+0000","last_undegraded":"2024-05-27T07:33:35.434986+0000","last_fullsized":"2024-05-27T07:33:35.434986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:43.912338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384425+0000","last_change":"2024-05-27T07:28:37.907441+0000","last_active":"2024-05-27T07:33:35.384425+0000","last_peered":"2024-05-27T07:33:35.384425+0000","last_clean":"2024-05-27T07:33:35.384425+0000","last_became_active":"2024-05-27T07:28:37.906928+0000","last_became_peered":"2024-05-27T07:28:37.906928+0000","last_unstale":"2024-05-27T07:33:35.384425+0000","last_undegraded":"2024-05-27T07:33:35.384425+0000","last_fullsized":"2024-05-27T07:33:35.384425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:28:32.345826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435906+0000","last_change":"2024-05-27T07:28:20.467408+0000","last_active":"2024-05-27T07:33:35.435906+0000","last_peered":"2024-05-27T07:33:35.435906+0000","last_clean":"2024-05-27T07:33:35.435906+0000","last_became_active":"2024-05-27T07:28:20.466064+0000","last_became_peered":"2024-05-27T07:28:20.466064+0000","last_unstale":"2024-05-27T07:33:35.435906+0000","last_undegraded":"2024-05-27T07:33:35.435906+0000","last_fullsized":"2024-05-27T07:33:35.435906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:15:51.449514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.371394+0000","last_change":"2024-05-27T07:28:24.478475+0000","last_active":"2024-05-27T07:33:35.371394+0000","last_peered":"2024-05-27T07:33:35.371394+0000","last_clean":"2024-05-27T07:33:35.371394+0000","last_became_active":"2024-05-27T07:28:24.477936+0000","last_became_peered":"2024-05-27T07:28:24.477936+0000","last_unstale":"2024-05-27T07:33:35.371394+0000","last_undegraded":"2024-05-27T07:33:35.371394+0000","last_fullsized":"2024-05-27T07:33:35.371394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:53:17.500040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336847+0000","last_change":"2024-05-27T07:28:37.968192+0000","last_active":"2024-05-27T07:33:35.336847+0000","last_peered":"2024-05-27T07:33:35.336847+0000","last_clean":"2024-05-27T07:33:35.336847+0000","last_became_active":"2024-05-27T07:28:37.967603+0000","last_became_peered":"2024-05-27T07:28:37.967603+0000","last_unstale":"2024-05-27T07:33:35.336847+0000","last_undegraded":"2024-05-27T07:33:35.336847+0000","last_fullsized":"2024-05-27T07:33:35.336847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:53:36.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'49","reported_seq":129,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.622094+0000","last_change":"2024-05-27T07:29:04.901062+0000","last_active":"2024-05-27T07:33:35.622094+0000","last_peered":"2024-05-27T07:33:35.622094+0000","last_clean":"2024-05-27T07:33:35.622094+0000","last_became_active":"2024-05-27T07:29:04.892060+0000","last_became_peered":"2024-05-27T07:29:04.892060+0000","last_unstale":"2024-05-27T07:33:35.622094+0000","last_undegraded":"2024-05-27T07:33:35.622094+0000","last_fullsized":"2024-05-27T07:33:35.622094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:27:37.894263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337180+0000","last_change":"2024-05-27T07:28:24.477096+0000","last_active":"2024-05-27T07:33:35.337180+0000","last_peered":"2024-05-27T07:33:35.337180+0000","last_clean":"2024-05-27T07:33:35.337180+0000","last_became_active":"2024-05-27T07:28:24.476712+0000","last_became_peered":"2024-05-27T07:28:24.476712+0000","last_unstale":"2024-05-27T07:33:35.337180+0000","last_undegraded":"2024-05-27T07:33:35.337180+0000","last_fullsized":"2024-05-27T07:33:35.337180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:31:58.010761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434405+0000","last_change":"2024-05-27T07:28:37.894013+0000","last_active":"2024-05-27T07:33:35.434405+0000","last_peered":"2024-05-27T07:33:35.434405+0000","last_clean":"2024-05-27T07:33:35.434405+0000","last_became_active":"2024-05-27T07:28:37.893345+0000","last_became_peered":"2024-05-27T07:28:37.893345+0000","last_unstale":"2024-05-27T07:33:35.434405+0000","last_undegraded":"2024-05-27T07:33:35.434405+0000","last_fullsized":"2024-05-27T07:33:35.434405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:26.917731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244487+0000","last_change":"2024-05-27T07:28:20.744087+0000","last_active":"2024-05-27T07:33:35.244487+0000","last_peered":"2024-05-27T07:33:35.244487+0000","last_clean":"2024-05-27T07:33:35.244487+0000","last_became_active":"2024-05-27T07:28:20.742883+0000","last_became_peered":"2024-05-27T07:28:20.742883+0000","last_unstale":"2024-05-27T07:33:35.244487+0000","last_undegraded":"2024-05-27T07:33:35.244487+0000","last_fullsized":"2024-05-27T07:33:35.244487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:06.083348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431834+0000","last_change":"2024-05-27T07:28:24.480330+0000","last_active":"2024-05-27T07:33:35.431834+0000","last_peered":"2024-05-27T07:33:35.431834+0000","last_clean":"2024-05-27T07:33:35.431834+0000","last_became_active":"2024-05-27T07:28:24.479300+0000","last_became_peered":"2024-05-27T07:28:24.479300+0000","last_unstale":"2024-05-27T07:33:35.431834+0000","last_undegraded":"2024-05-27T07:33:35.431834+0000","last_fullsized":"2024-05-27T07:33:35.431834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:43.802273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243811+0000","last_change":"2024-05-27T07:28:37.973592+0000","last_active":"2024-05-27T07:33:35.243811+0000","last_peered":"2024-05-27T07:33:35.243811+0000","last_clean":"2024-05-27T07:33:35.243811+0000","last_became_active":"2024-05-27T07:28:37.972475+0000","last_became_peered":"2024-05-27T07:28:37.972475+0000","last_unstale":"2024-05-27T07:33:35.243811+0000","last_undegraded":"2024-05-27T07:33:35.243811+0000","last_fullsized":"2024-05-27T07:33:35.243811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:31.662384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434870+0000","last_change":"2024-05-27T07:29:04.890751+0000","last_active":"2024-05-27T07:33:35.434870+0000","last_peered":"2024-05-27T07:33:35.434870+0000","last_clean":"2024-05-27T07:33:35.434870+0000","last_became_active":"2024-05-27T07:29:04.889447+0000","last_became_peered":"2024-05-27T07:29:04.889447+0000","last_unstale":"2024-05-27T07:33:35.434870+0000","last_undegraded":"2024-05-27T07:33:35.434870+0000","last_fullsized":"2024-05-27T07:33:35.434870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:13:34.743691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434302+0000","last_change":"2024-05-27T07:28:24.474298+0000","last_active":"2024-05-27T07:33:35.434302+0000","last_peered":"2024-05-27T07:33:35.434302+0000","last_clean":"2024-05-27T07:33:35.434302+0000","last_became_active":"2024-05-27T07:28:24.473877+0000","last_became_peered":"2024-05-27T07:28:24.473877+0000","last_unstale":"2024-05-27T07:33:35.434302+0000","last_undegraded":"2024-05-27T07:33:35.434302+0000","last_fullsized":"2024-05-27T07:33:35.434302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:31.545883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.239267+0000","last_change":"2024-05-27T07:28:37.905417+0000","last_active":"2024-05-27T07:33:35.239267+0000","last_peered":"2024-05-27T07:33:35.239267+0000","last_clean":"2024-05-27T07:33:35.239267+0000","last_became_active":"2024-05-27T07:28:37.904968+0000","last_became_peered":"2024-05-27T07:28:37.904968+0000","last_unstale":"2024-05-27T07:33:35.239267+0000","last_undegraded":"2024-05-27T07:33:35.239267+0000","last_fullsized":"2024-05-27T07:33:35.239267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:02.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337111+0000","last_change":"2024-05-27T07:28:20.392647+0000","last_active":"2024-05-27T07:33:35.337111+0000","last_peered":"2024-05-27T07:33:35.337111+0000","last_clean":"2024-05-27T07:33:35.337111+0000","last_became_active":"2024-05-27T07:28:20.392073+0000","last_became_peered":"2024-05-27T07:28:20.392073+0000","last_unstale":"2024-05-27T07:33:35.337111+0000","last_undegraded":"2024-05-27T07:33:35.337111+0000","last_fullsized":"2024-05-27T07:33:35.337111+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:18.777628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384408+0000","last_change":"2024-05-27T07:28:24.490522+0000","last_active":"2024-05-27T07:33:35.384408+0000","last_peered":"2024-05-27T07:33:35.384408+0000","last_clean":"2024-05-27T07:33:35.384408+0000","last_became_active":"2024-05-27T07:28:24.485228+0000","last_became_peered":"2024-05-27T07:28:24.485228+0000","last_unstale":"2024-05-27T07:33:35.384408+0000","last_undegraded":"2024-05-27T07:33:35.384408+0000","last_fullsized":"2024-05-27T07:33:35.384408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:14.221016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243254+0000","last_change":"2024-05-27T07:28:37.900449+0000","last_active":"2024-05-27T07:33:35.243254+0000","last_peered":"2024-05-27T07:33:35.243254+0000","last_clean":"2024-05-27T07:33:35.243254+0000","last_became_active":"2024-05-27T07:28:37.899526+0000","last_became_peered":"2024-05-27T07:28:37.899526+0000","last_unstale":"2024-05-27T07:33:35.243254+0000","last_undegraded":"2024-05-27T07:33:35.243254+0000","last_fullsized":"2024-05-27T07:33:35.243254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:29:49.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437781+0000","last_change":"2024-05-27T07:28:20.405236+0000","last_active":"2024-05-27T07:33:35.437781+0000","last_peered":"2024-05-27T07:33:35.437781+0000","last_clean":"2024-05-27T07:33:35.437781+0000","last_became_active":"2024-05-27T07:28:20.404619+0000","last_became_peered":"2024-05-27T07:28:20.404619+0000","last_unstale":"2024-05-27T07:33:35.437781+0000","last_undegraded":"2024-05-27T07:33:35.437781+0000","last_fullsized":"2024-05-27T07:33:35.437781+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:17.671250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337323+0000","last_change":"2024-05-27T07:28:24.490661+0000","last_active":"2024-05-27T07:33:35.337323+0000","last_peered":"2024-05-27T07:33:35.337323+0000","last_clean":"2024-05-27T07:33:35.337323+0000","last_became_active":"2024-05-27T07:28:24.487299+0000","last_became_peered":"2024-05-27T07:28:24.487299+0000","last_unstale":"2024-05-27T07:33:35.337323+0000","last_undegraded":"2024-05-27T07:33:35.337323+0000","last_fullsized":"2024-05-27T07:33:35.337323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:50:27.291721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243449+0000","last_change":"2024-05-27T07:28:37.970463+0000","last_active":"2024-05-27T07:33:35.243449+0000","last_peered":"2024-05-27T07:33:35.243449+0000","last_clean":"2024-05-27T07:33:35.243449+0000","last_became_active":"2024-05-27T07:28:37.968068+0000","last_became_peered":"2024-05-27T07:28:37.968068+0000","last_unstale":"2024-05-27T07:33:35.243449+0000","last_undegraded":"2024-05-27T07:33:35.243449+0000","last_fullsized":"2024-05-27T07:33:35.243449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:22:00.825999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434404+0000","last_change":"2024-05-27T07:28:20.406203+0000","last_active":"2024-05-27T07:33:35.434404+0000","last_peered":"2024-05-27T07:33:35.434404+0000","last_clean":"2024-05-27T07:33:35.434404+0000","last_became_active":"2024-05-27T07:28:20.405404+0000","last_became_peered":"2024-05-27T07:28:20.405404+0000","last_unstale":"2024-05-27T07:33:35.434404+0000","last_undegraded":"2024-05-27T07:33:35.434404+0000","last_fullsized":"2024-05-27T07:33:35.434404+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:15.274419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434004+0000","last_change":"2024-05-27T07:28:24.473747+0000","last_active":"2024-05-27T07:33:35.434004+0000","last_peered":"2024-05-27T07:33:35.434004+0000","last_clean":"2024-05-27T07:33:35.434004+0000","last_became_active":"2024-05-27T07:28:24.473313+0000","last_became_peered":"2024-05-27T07:28:24.473313+0000","last_unstale":"2024-05-27T07:33:35.434004+0000","last_undegraded":"2024-05-27T07:33:35.434004+0000","last_fullsized":"2024-05-27T07:33:35.434004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:19:40.107840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434078+0000","last_change":"2024-05-27T07:28:37.907410+0000","last_active":"2024-05-27T07:33:35.434078+0000","last_peered":"2024-05-27T07:33:35.434078+0000","last_clean":"2024-05-27T07:33:35.434078+0000","last_became_active":"2024-05-27T07:28:37.906753+0000","last_became_peered":"2024-05-27T07:28:37.906753+0000","last_unstale":"2024-05-27T07:33:35.434078+0000","last_undegraded":"2024-05-27T07:33:35.434078+0000","last_fullsized":"2024-05-27T07:33:35.434078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:53.612015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437366+0000","last_change":"2024-05-27T07:28:20.744068+0000","last_active":"2024-05-27T07:33:35.437366+0000","last_peered":"2024-05-27T07:33:35.437366+0000","last_clean":"2024-05-27T07:33:35.437366+0000","last_became_active":"2024-05-27T07:28:20.742686+0000","last_became_peered":"2024-05-27T07:28:20.742686+0000","last_unstale":"2024-05-27T07:33:35.437366+0000","last_undegraded":"2024-05-27T07:33:35.437366+0000","last_fullsized":"2024-05-27T07:33:35.437366+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:54.361007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436607+0000","last_change":"2024-05-27T07:28:24.475743+0000","last_active":"2024-05-27T07:33:35.436607+0000","last_peered":"2024-05-27T07:33:35.436607+0000","last_clean":"2024-05-27T07:33:35.436607+0000","last_became_active":"2024-05-27T07:28:24.474742+0000","last_became_peered":"2024-05-27T07:28:24.474742+0000","last_unstale":"2024-05-27T07:33:35.436607+0000","last_undegraded":"2024-05-27T07:33:35.436607+0000","last_fullsized":"2024-05-27T07:33:35.436607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:58:47.061573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431413+0000","last_change":"2024-05-27T07:28:37.914935+0000","last_active":"2024-05-27T07:33:35.431413+0000","last_peered":"2024-05-27T07:33:35.431413+0000","last_clean":"2024-05-27T07:33:35.431413+0000","last_became_active":"2024-05-27T07:28:37.911829+0000","last_became_peered":"2024-05-27T07:28:37.911829+0000","last_unstale":"2024-05-27T07:33:35.431413+0000","last_undegraded":"2024-05-27T07:33:35.431413+0000","last_fullsized":"2024-05-27T07:33:35.431413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:29.260539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"92'39","reported_seq":95,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.608079+0000","last_change":"2024-05-27T07:28:20.412554+0000","last_active":"2024-05-27T07:33:35.608079+0000","last_peered":"2024-05-27T07:33:35.608079+0000","last_clean":"2024-05-27T07:33:35.608079+0000","last_became_active":"2024-05-27T07:28:20.412024+0000","last_became_peered":"2024-05-27T07:28:20.412024+0000","last_unstale":"2024-05-27T07:33:35.608079+0000","last_undegraded":"2024-05-27T07:33:35.608079+0000","last_fullsized":"2024-05-27T07:33:35.608079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:29.244093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435695+0000","last_change":"2024-05-27T07:28:24.481966+0000","last_active":"2024-05-27T07:33:35.435695+0000","last_peered":"2024-05-27T07:33:35.435695+0000","last_clean":"2024-05-27T07:33:35.435695+0000","last_became_active":"2024-05-27T07:28:24.480775+0000","last_became_peered":"2024-05-27T07:28:24.480775+0000","last_unstale":"2024-05-27T07:33:35.435695+0000","last_undegraded":"2024-05-27T07:33:35.435695+0000","last_fullsized":"2024-05-27T07:33:35.435695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:03.018174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242513+0000","last_change":"2024-05-27T07:28:37.902893+0000","last_active":"2024-05-27T07:33:35.242513+0000","last_peered":"2024-05-27T07:33:35.242513+0000","last_clean":"2024-05-27T07:33:35.242513+0000","last_became_active":"2024-05-27T07:28:37.898447+0000","last_became_peered":"2024-05-27T07:28:37.898447+0000","last_unstale":"2024-05-27T07:33:35.242513+0000","last_undegraded":"2024-05-27T07:33:35.242513+0000","last_fullsized":"2024-05-27T07:33:35.242513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:55:49.754803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370906+0000","last_change":"2024-05-27T07:28:20.408072+0000","last_active":"2024-05-27T07:33:35.370906+0000","last_peered":"2024-05-27T07:33:35.370906+0000","last_clean":"2024-05-27T07:33:35.370906+0000","last_became_active":"2024-05-27T07:28:20.407174+0000","last_became_peered":"2024-05-27T07:28:20.407174+0000","last_unstale":"2024-05-27T07:33:35.370906+0000","last_undegraded":"2024-05-27T07:33:35.370906+0000","last_fullsized":"2024-05-27T07:33:35.370906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:15:52.984967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437053+0000","last_change":"2024-05-27T07:28:24.475891+0000","last_active":"2024-05-27T07:33:35.437053+0000","last_peered":"2024-05-27T07:33:35.437053+0000","last_clean":"2024-05-27T07:33:35.437053+0000","last_became_active":"2024-05-27T07:28:24.475498+0000","last_became_peered":"2024-05-27T07:28:24.475498+0000","last_unstale":"2024-05-27T07:33:35.437053+0000","last_undegraded":"2024-05-27T07:33:35.437053+0000","last_fullsized":"2024-05-27T07:33:35.437053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:50.669118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384465+0000","last_change":"2024-05-27T07:29:04.891583+0000","last_active":"2024-05-27T07:33:35.384465+0000","last_peered":"2024-05-27T07:33:35.384465+0000","last_clean":"2024-05-27T07:33:35.384465+0000","last_became_active":"2024-05-27T07:29:04.890187+0000","last_became_peered":"2024-05-27T07:29:04.890187+0000","last_unstale":"2024-05-27T07:33:35.384465+0000","last_undegraded":"2024-05-27T07:33:35.384465+0000","last_fullsized":"2024-05-27T07:33:35.384465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:03.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436357+0000","last_change":"2024-05-27T07:28:24.478672+0000","last_active":"2024-05-27T07:33:35.436357+0000","last_peered":"2024-05-27T07:33:35.436357+0000","last_clean":"2024-05-27T07:33:35.436357+0000","last_became_active":"2024-05-27T07:28:24.478331+0000","last_became_peered":"2024-05-27T07:28:24.478331+0000","last_unstale":"2024-05-27T07:33:35.436357+0000","last_undegraded":"2024-05-27T07:33:35.436357+0000","last_fullsized":"2024-05-27T07:33:35.436357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:56:35.112855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437255+0000","last_change":"2024-05-27T07:28:24.487962+0000","last_active":"2024-05-27T07:33:35.437255+0000","last_peered":"2024-05-27T07:33:35.437255+0000","last_clean":"2024-05-27T07:33:35.437255+0000","last_became_active":"2024-05-27T07:28:24.487583+0000","last_became_peered":"2024-05-27T07:28:24.487583+0000","last_unstale":"2024-05-27T07:33:35.437255+0000","last_undegraded":"2024-05-27T07:33:35.437255+0000","last_fullsized":"2024-05-27T07:33:35.437255+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:25.758293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431727+0000","last_change":"2024-05-27T07:28:24.466058+0000","last_active":"2024-05-27T07:33:35.431727+0000","last_peered":"2024-05-27T07:33:35.431727+0000","last_clean":"2024-05-27T07:33:35.431727+0000","last_became_active":"2024-05-27T07:28:24.465709+0000","last_became_peered":"2024-05-27T07:28:24.465709+0000","last_unstale":"2024-05-27T07:33:35.431727+0000","last_undegraded":"2024-05-27T07:33:35.431727+0000","last_fullsized":"2024-05-27T07:33:35.431727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:38.064077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369986+0000","last_change":"2024-05-27T07:28:24.471585+0000","last_active":"2024-05-27T07:33:35.369986+0000","last_peered":"2024-05-27T07:33:35.369986+0000","last_clean":"2024-05-27T07:33:35.369986+0000","last_became_active":"2024-05-27T07:28:24.470948+0000","last_became_peered":"2024-05-27T07:28:24.470948+0000","last_unstale":"2024-05-27T07:33:35.369986+0000","last_undegraded":"2024-05-27T07:33:35.369986+0000","last_fullsized":"2024-05-27T07:33:35.369986+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:48.213323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385238+0000","last_change":"2024-05-27T07:28:24.477954+0000","last_active":"2024-05-27T07:33:35.385238+0000","last_peered":"2024-05-27T07:33:35.385238+0000","last_clean":"2024-05-27T07:33:35.385238+0000","last_became_active":"2024-05-27T07:28:24.477495+0000","last_became_peered":"2024-05-27T07:28:24.477495+0000","last_unstale":"2024-05-27T07:33:35.385238+0000","last_undegraded":"2024-05-27T07:33:35.385238+0000","last_fullsized":"2024-05-27T07:33:35.385238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:50.371439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437604+0000","last_change":"2024-05-27T07:28:24.476422+0000","last_active":"2024-05-27T07:33:35.437604+0000","last_peered":"2024-05-27T07:33:35.437604+0000","last_clean":"2024-05-27T07:33:35.437604+0000","last_became_active":"2024-05-27T07:28:24.476095+0000","last_became_peered":"2024-05-27T07:28:24.476095+0000","last_unstale":"2024-05-27T07:33:35.437604+0000","last_undegraded":"2024-05-27T07:33:35.437604+0000","last_fullsized":"2024-05-27T07:33:35.437604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:51.667169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337797+0000","last_change":"2024-05-27T07:28:24.486431+0000","last_active":"2024-05-27T07:33:35.337797+0000","last_peered":"2024-05-27T07:33:35.337797+0000","last_clean":"2024-05-27T07:33:35.337797+0000","last_became_active":"2024-05-27T07:28:24.482427+0000","last_became_peered":"2024-05-27T07:28:24.482427+0000","last_unstale":"2024-05-27T07:33:35.337797+0000","last_undegraded":"2024-05-27T07:33:35.337797+0000","last_fullsized":"2024-05-27T07:33:35.337797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:44:37.857793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243086+0000","last_change":"2024-05-27T07:28:24.484393+0000","last_active":"2024-05-27T07:33:35.243086+0000","last_peered":"2024-05-27T07:33:35.243086+0000","last_clean":"2024-05-27T07:33:35.243086+0000","last_became_active":"2024-05-27T07:28:24.483130+0000","last_became_peered":"2024-05-27T07:28:24.483130+0000","last_unstale":"2024-05-27T07:33:35.243086+0000","last_undegraded":"2024-05-27T07:33:35.243086+0000","last_fullsized":"2024-05-27T07:33:35.243086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:34.924573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337254+0000","last_change":"2024-05-27T07:28:24.486274+0000","last_active":"2024-05-27T07:33:35.337254+0000","last_peered":"2024-05-27T07:33:35.337254+0000","last_clean":"2024-05-27T07:33:35.337254+0000","last_became_active":"2024-05-27T07:28:24.485880+0000","last_became_peered":"2024-05-27T07:28:24.485880+0000","last_unstale":"2024-05-27T07:33:35.337254+0000","last_undegraded":"2024-05-27T07:33:35.337254+0000","last_fullsized":"2024-05-27T07:33:35.337254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:33.287615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436357+0000","last_change":"2024-05-27T07:28:24.477909+0000","last_active":"2024-05-27T07:33:35.436357+0000","last_peered":"2024-05-27T07:33:35.436357+0000","last_clean":"2024-05-27T07:33:35.436357+0000","last_became_active":"2024-05-27T07:28:24.477345+0000","last_became_peered":"2024-05-27T07:28:24.477345+0000","last_unstale":"2024-05-27T07:33:35.436357+0000","last_undegraded":"2024-05-27T07:33:35.436357+0000","last_fullsized":"2024-05-27T07:33:35.436357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:26:03.958489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370232+0000","last_change":"2024-05-27T07:28:24.477151+0000","last_active":"2024-05-27T07:33:35.370232+0000","last_peered":"2024-05-27T07:33:35.370232+0000","last_clean":"2024-05-27T07:33:35.370232+0000","last_became_active":"2024-05-27T07:28:24.474036+0000","last_became_peered":"2024-05-27T07:28:24.474036+0000","last_unstale":"2024-05-27T07:33:35.370232+0000","last_undegraded":"2024-05-27T07:33:35.370232+0000","last_fullsized":"2024-05-27T07:33:35.370232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:16.787252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385340+0000","last_change":"2024-05-27T07:28:24.483601+0000","last_active":"2024-05-27T07:33:35.385340+0000","last_peered":"2024-05-27T07:33:35.385340+0000","last_clean":"2024-05-27T07:33:35.385340+0000","last_became_active":"2024-05-27T07:28:24.481080+0000","last_became_peered":"2024-05-27T07:28:24.481080+0000","last_unstale":"2024-05-27T07:33:35.385340+0000","last_undegraded":"2024-05-27T07:33:35.385340+0000","last_fullsized":"2024-05-27T07:33:35.385340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:12.208506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":607900,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1496,"num_read_kb":866,"num_write":304,"num_write_kb":1455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2052096,"data_stored":1823700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645393,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712552,"statfs":{"total":95995035648,"available":95961653248,"internally_reserved":0,"allocated":1454080,"data_stored":925589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841624,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28872,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716280,"statfs":{"total":95995035648,"available":95965470720,"internally_reserved":0,"allocated":1900544,"data_stored":1327702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005152,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1474560,"data_stored":929921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201385,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":532529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397616,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32808,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712344,"statfs":{"total":95995035648,"available":95961440256,"internally_reserved":0,"allocated":1671168,"data_stored":1130206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561142,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":527785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1138688,"data_stored":602490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738530,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":529173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":604333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149837,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1597440,"data_stored":1052795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953606,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":534101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757375,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37108,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708044,"statfs":{"total":95995035648,"available":95957037056,"internally_reserved":0,"allocated":1814528,"data_stored":1255728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":77315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":212159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:33:44.610 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-27T07:33:44.610 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T07:33:44.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.944+0000 7ff432400640 1 -- 172.21.15.63:0/4255329412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c10d8f0 msgr2=0x7ff42c10fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:44.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.944+0000 7ff432400640 1 --2- 172.21.15.63:0/4255329412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c10d8f0 0x7ff42c10fd10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff418009900 tx=0x7ff41802a480 comp rx=0 tx=0).stop 2024-05-27T07:33:44.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.944+0000 7ff432400640 1 -- 172.21.15.63:0/4255329412 shutdown_connections 2024-05-27T07:33:44.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.944+0000 7ff432400640 1 --2- 172.21.15.63:0/4255329412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c110250 0x7ff42c1126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.944+0000 7ff432400640 1 --2- 172.21.15.63:0/4255329412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c10d8f0 0x7ff42c10fd10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.944+0000 7ff432400640 1 --2- 172.21.15.63:0/4255329412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff42c075660 0x7ff42c075a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.944+0000 7ff432400640 1 -- 172.21.15.63:0/4255329412 >> 172.21.15.63:0/4255329412 conn(0x7ff42c079100 msgr2=0x7ff42c079560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:44.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.944+0000 7ff432400640 1 -- 172.21.15.63:0/4255329412 shutdown_connections 2024-05-27T07:33:44.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.944+0000 7ff432400640 1 -- 172.21.15.63:0/4255329412 wait complete. 2024-05-27T07:33:44.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.945+0000 7ff432400640 1 Processor -- start 2024-05-27T07:33:44.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.946+0000 7ff432400640 1 -- start start 2024-05-27T07:33:44.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.946+0000 7ff432400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c075660 0x7ff42c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:44.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.946+0000 7ff432400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c10d8f0 0x7ff42c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:44.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.946+0000 7ff432400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff42c110250 0x7ff42c1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:44.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.946+0000 7ff432400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c115cd0 con 0x7ff42c10d8f0 2024-05-27T07:33:44.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.946+0000 7ff432400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c115b50 con 0x7ff42c075660 2024-05-27T07:33:44.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.946+0000 7ff432400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c115e50 con 0x7ff42c110250 2024-05-27T07:33:44.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.947+0000 7ff42be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c10d8f0 0x7ff42c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:44.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.947+0000 7ff42be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c10d8f0 0x7ff42c19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42392/0 (socket says 172.21.15.63:42392) 2024-05-27T07:33:44.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.947+0000 7ff42be00640 1 -- 172.21.15.63:0/2910739540 learned_addr learned my addr 172.21.15.63:0/2910739540 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:44.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.947+0000 7ff431000640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c075660 0x7ff42c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:44.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.947+0000 7ff431a00640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff42c110250 0x7ff42c1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:44.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.947+0000 7ff42be00640 1 -- 172.21.15.63:0/2910739540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff42c110250 msgr2=0x7ff42c1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:44.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.947+0000 7ff42be00640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff42c110250 0x7ff42c1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.947+0000 7ff42be00640 1 -- 172.21.15.63:0/2910739540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c075660 msgr2=0x7ff42c19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:44.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.947+0000 7ff42be00640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c075660 0x7ff42c19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:44.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.947+0000 7ff42be00640 1 -- 172.21.15.63:0/2910739540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c1a15f0 con 0x7ff42c10d8f0 2024-05-27T07:33:44.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.948+0000 7ff42be00640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c10d8f0 0x7ff42c19ccf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff418009230 tx=0x7ff418004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:44.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.948+0000 7ff429600640 1 -- 172.21.15.63:0/2910739540 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41803f030 con 0x7ff42c10d8f0 2024-05-27T07:33:44.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.948+0000 7ff432400640 1 -- 172.21.15.63:0/2910739540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c1a1880 con 0x7ff42c10d8f0 2024-05-27T07:33:44.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.948+0000 7ff429600640 1 -- 172.21.15.63:0/2910739540 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff418003a80 con 0x7ff42c10d8f0 2024-05-27T07:33:44.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.948+0000 7ff429600640 1 -- 172.21.15.63:0/2910739540 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418035070 con 0x7ff42c10d8f0 2024-05-27T07:33:44.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.948+0000 7ff432400640 1 -- 172.21.15.63:0/2910739540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c1a1dc0 con 0x7ff42c10d8f0 2024-05-27T07:33:44.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.950+0000 7ff432400640 1 -- 172.21.15.63:0/2910739540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c069d00 con 0x7ff42c10d8f0 2024-05-27T07:33:44.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.953+0000 7ff429600640 1 -- 172.21.15.63:0/2910739540 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7ff418042030 con 0x7ff42c10d8f0 2024-05-27T07:33:44.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.954+0000 7ff429600640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4140b2a70 0x7ff4140b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:44.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.955+0000 7ff429600640 1 -- 172.21.15.63:0/2910739540 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7ff418103750 con 0x7ff42c10d8f0 2024-05-27T07:33:44.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.955+0000 7ff431000640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4140b2a70 0x7ff4140b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:44.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.956+0000 7ff431000640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4140b2a70 0x7ff4140b4f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff420005fd0 tx=0x7ff42000b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:44.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:44.957+0000 7ff429600640 1 -- 172.21.15.63:0/2910739540 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff4180cabe0 con 0x7ff42c10d8f0 2024-05-27T07:33:45.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.169+0000 7ff432400640 1 -- 172.21.15.63:0/2910739540 --> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff42c102890 con 0x7ff4140b2a70 2024-05-27T07:33:45.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.198+0000 7ff429600640 1 -- 172.21.15.63:0/2910739540 <== mgr.15216 v2:172.21.15.63:6800/1343315489 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339430 (secure 0 0 0) 0x7ff42c102890 con 0x7ff4140b2a70 2024-05-27T07:33:45.193 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:33:45.198 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-27T07:33:45.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 -- 172.21.15.63:0/2910739540 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4140b2a70 msgr2=0x7ff4140b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4140b2a70 0x7ff4140b4f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff420005fd0 tx=0x7ff42000b390 comp rx=0 tx=0).stop 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 -- 172.21.15.63:0/2910739540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c10d8f0 msgr2=0x7ff42c19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c10d8f0 0x7ff42c19ccf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff418009230 tx=0x7ff418004270 comp rx=0 tx=0).stop 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 -- 172.21.15.63:0/2910739540 shutdown_connections 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4140b2a70 0x7ff4140b4f80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff42c110250 0x7ff42c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff42c10d8f0 0x7ff42c19ccf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 --2- 172.21.15.63:0/2910739540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c075660 0x7ff42c19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 -- 172.21.15.63:0/2910739540 >> 172.21.15.63:0/2910739540 conn(0x7ff42c079100 msgr2=0x7ff42c102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 -- 172.21.15.63:0/2910739540 shutdown_connections 2024-05-27T07:33:45.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:45.207+0000 7ff432400640 1 -- 172.21.15.63:0/2910739540 wait complete. 2024-05-27T07:33:45.233 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T07:33:43.267988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1198502,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1640,"num_read_kb":1114,"num_write":441,"num_write_kb":3607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377776,"kb_used_data":16112,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124564048,"statfs":{"total":1151940427776,"available":1151553585152,"internally_reserved":0,"allocated":16498688,"data_stored":9952352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3302,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":201,"num_write":17,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.976226"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385739+0000","last_change":"2024-05-27T07:28:24.485151+0000","last_active":"2024-05-27T07:33:35.385739+0000","last_peered":"2024-05-27T07:33:35.385739+0000","last_clean":"2024-05-27T07:33:35.385739+0000","last_became_active":"2024-05-27T07:28:24.483965+0000","last_became_peered":"2024-05-27T07:28:24.483965+0000","last_unstale":"2024-05-27T07:33:35.385739+0000","last_undegraded":"2024-05-27T07:33:35.385739+0000","last_fullsized":"2024-05-27T07:33:35.385739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:18.376755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431746+0000","last_change":"2024-05-27T07:28:24.483608+0000","last_active":"2024-05-27T07:33:35.431746+0000","last_peered":"2024-05-27T07:33:35.431746+0000","last_clean":"2024-05-27T07:33:35.431746+0000","last_became_active":"2024-05-27T07:28:24.479875+0000","last_became_peered":"2024-05-27T07:28:24.479875+0000","last_unstale":"2024-05-27T07:33:35.431746+0000","last_undegraded":"2024-05-27T07:33:35.431746+0000","last_fullsized":"2024-05-27T07:33:35.431746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:30.129708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431562+0000","last_change":"2024-05-27T07:28:24.470104+0000","last_active":"2024-05-27T07:33:35.431562+0000","last_peered":"2024-05-27T07:33:35.431562+0000","last_clean":"2024-05-27T07:33:35.431562+0000","last_became_active":"2024-05-27T07:28:24.469756+0000","last_became_peered":"2024-05-27T07:28:24.469756+0000","last_unstale":"2024-05-27T07:33:35.431562+0000","last_undegraded":"2024-05-27T07:33:35.431562+0000","last_fullsized":"2024-05-27T07:33:35.431562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:36.907691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243760+0000","last_change":"2024-05-27T07:28:24.472381+0000","last_active":"2024-05-27T07:33:35.243760+0000","last_peered":"2024-05-27T07:33:35.243760+0000","last_clean":"2024-05-27T07:33:35.243760+0000","last_became_active":"2024-05-27T07:28:24.469959+0000","last_became_peered":"2024-05-27T07:28:24.469959+0000","last_unstale":"2024-05-27T07:33:35.243760+0000","last_undegraded":"2024-05-27T07:33:35.243760+0000","last_fullsized":"2024-05-27T07:33:35.243760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:15.267554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434585+0000","last_change":"2024-05-27T07:28:24.474210+0000","last_active":"2024-05-27T07:33:35.434585+0000","last_peered":"2024-05-27T07:33:35.434585+0000","last_clean":"2024-05-27T07:33:35.434585+0000","last_became_active":"2024-05-27T07:28:24.473688+0000","last_became_peered":"2024-05-27T07:28:24.473688+0000","last_unstale":"2024-05-27T07:33:35.434585+0000","last_undegraded":"2024-05-27T07:33:35.434585+0000","last_fullsized":"2024-05-27T07:33:35.434585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:38.733536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401818+0000","last_change":"2024-05-27T07:28:24.484679+0000","last_active":"2024-05-27T07:33:35.401818+0000","last_peered":"2024-05-27T07:33:35.401818+0000","last_clean":"2024-05-27T07:33:35.401818+0000","last_became_active":"2024-05-27T07:28:24.484251+0000","last_became_peered":"2024-05-27T07:28:24.484251+0000","last_unstale":"2024-05-27T07:33:35.401818+0000","last_undegraded":"2024-05-27T07:33:35.401818+0000","last_fullsized":"2024-05-27T07:33:35.401818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:28:09.507871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434697+0000","last_change":"2024-05-27T07:28:24.472694+0000","last_active":"2024-05-27T07:33:35.434697+0000","last_peered":"2024-05-27T07:33:35.434697+0000","last_clean":"2024-05-27T07:33:35.434697+0000","last_became_active":"2024-05-27T07:28:24.472100+0000","last_became_peered":"2024-05-27T07:28:24.472100+0000","last_unstale":"2024-05-27T07:33:35.434697+0000","last_undegraded":"2024-05-27T07:33:35.434697+0000","last_fullsized":"2024-05-27T07:33:35.434697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:25:42.756421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336904+0000","last_change":"2024-05-27T07:28:24.488577+0000","last_active":"2024-05-27T07:33:35.336904+0000","last_peered":"2024-05-27T07:33:35.336904+0000","last_clean":"2024-05-27T07:33:35.336904+0000","last_became_active":"2024-05-27T07:28:24.488180+0000","last_became_peered":"2024-05-27T07:28:24.488180+0000","last_unstale":"2024-05-27T07:33:35.336904+0000","last_undegraded":"2024-05-27T07:33:35.336904+0000","last_fullsized":"2024-05-27T07:33:35.336904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:17.130058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437122+0000","last_change":"2024-05-27T07:28:24.480109+0000","last_active":"2024-05-27T07:33:35.437122+0000","last_peered":"2024-05-27T07:33:35.437122+0000","last_clean":"2024-05-27T07:33:35.437122+0000","last_became_active":"2024-05-27T07:28:24.479722+0000","last_became_peered":"2024-05-27T07:28:24.479722+0000","last_unstale":"2024-05-27T07:33:35.437122+0000","last_undegraded":"2024-05-27T07:33:35.437122+0000","last_fullsized":"2024-05-27T07:33:35.437122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:27:36.074612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.402840+0000","last_change":"2024-05-27T07:28:24.490389+0000","last_active":"2024-05-27T07:33:35.402840+0000","last_peered":"2024-05-27T07:33:35.402840+0000","last_clean":"2024-05-27T07:33:35.402840+0000","last_became_active":"2024-05-27T07:28:24.488081+0000","last_became_peered":"2024-05-27T07:28:24.488081+0000","last_unstale":"2024-05-27T07:33:35.402840+0000","last_undegraded":"2024-05-27T07:33:35.402840+0000","last_fullsized":"2024-05-27T07:33:35.402840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:06.318235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436339+0000","last_change":"2024-05-27T07:28:24.477062+0000","last_active":"2024-05-27T07:33:35.436339+0000","last_peered":"2024-05-27T07:33:35.436339+0000","last_clean":"2024-05-27T07:33:35.436339+0000","last_became_active":"2024-05-27T07:28:24.475322+0000","last_became_peered":"2024-05-27T07:28:24.475322+0000","last_unstale":"2024-05-27T07:33:35.436339+0000","last_undegraded":"2024-05-27T07:33:35.436339+0000","last_fullsized":"2024-05-27T07:33:35.436339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:30.365295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.238628+0000","last_change":"2024-05-27T07:29:04.901715+0000","last_active":"2024-05-27T07:33:35.238628+0000","last_peered":"2024-05-27T07:33:35.238628+0000","last_clean":"2024-05-27T07:33:35.238628+0000","last_became_active":"2024-05-27T07:29:04.900968+0000","last_became_peered":"2024-05-27T07:29:04.900968+0000","last_unstale":"2024-05-27T07:33:35.238628+0000","last_undegraded":"2024-05-27T07:33:35.238628+0000","last_fullsized":"2024-05-27T07:33:35.238628+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:16:34.033701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370124+0000","last_change":"2024-05-27T07:28:24.477504+0000","last_active":"2024-05-27T07:33:35.370124+0000","last_peered":"2024-05-27T07:33:35.370124+0000","last_clean":"2024-05-27T07:33:35.370124+0000","last_became_active":"2024-05-27T07:28:24.476709+0000","last_became_peered":"2024-05-27T07:28:24.476709+0000","last_unstale":"2024-05-27T07:33:35.370124+0000","last_undegraded":"2024-05-27T07:33:35.370124+0000","last_fullsized":"2024-05-27T07:33:35.370124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:05:48.456347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.432633+0000","last_change":"2024-05-27T07:28:24.485567+0000","last_active":"2024-05-27T07:33:35.432633+0000","last_peered":"2024-05-27T07:33:35.432633+0000","last_clean":"2024-05-27T07:33:35.432633+0000","last_became_active":"2024-05-27T07:28:24.485175+0000","last_became_peered":"2024-05-27T07:28:24.485175+0000","last_unstale":"2024-05-27T07:33:35.432633+0000","last_undegraded":"2024-05-27T07:33:35.432633+0000","last_fullsized":"2024-05-27T07:33:35.432633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:25:30.062813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431453+0000","last_change":"2024-05-27T07:29:04.900657+0000","last_active":"2024-05-27T07:33:35.431453+0000","last_peered":"2024-05-27T07:33:35.431453+0000","last_clean":"2024-05-27T07:33:35.431453+0000","last_became_active":"2024-05-27T07:29:04.900093+0000","last_became_peered":"2024-05-27T07:29:04.900093+0000","last_unstale":"2024-05-27T07:33:35.431453+0000","last_undegraded":"2024-05-27T07:33:35.431453+0000","last_fullsized":"2024-05-27T07:33:35.431453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:28:04.157825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434350+0000","last_change":"2024-05-27T07:28:24.470503+0000","last_active":"2024-05-27T07:33:35.434350+0000","last_peered":"2024-05-27T07:33:35.434350+0000","last_clean":"2024-05-27T07:33:35.434350+0000","last_became_active":"2024-05-27T07:28:24.470128+0000","last_became_peered":"2024-05-27T07:28:24.470128+0000","last_unstale":"2024-05-27T07:33:35.434350+0000","last_undegraded":"2024-05-27T07:33:35.434350+0000","last_fullsized":"2024-05-27T07:33:35.434350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:46:43.149635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385138+0000","last_change":"2024-05-27T07:28:24.491382+0000","last_active":"2024-05-27T07:33:35.385138+0000","last_peered":"2024-05-27T07:33:35.385138+0000","last_clean":"2024-05-27T07:33:35.385138+0000","last_became_active":"2024-05-27T07:28:24.486155+0000","last_became_peered":"2024-05-27T07:28:24.486155+0000","last_unstale":"2024-05-27T07:33:35.385138+0000","last_undegraded":"2024-05-27T07:33:35.385138+0000","last_fullsized":"2024-05-27T07:33:35.385138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:48.408573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.239172+0000","last_change":"2024-05-27T07:28:24.489140+0000","last_active":"2024-05-27T07:33:35.239172+0000","last_peered":"2024-05-27T07:33:35.239172+0000","last_clean":"2024-05-27T07:33:35.239172+0000","last_became_active":"2024-05-27T07:28:24.488733+0000","last_became_peered":"2024-05-27T07:28:24.488733+0000","last_unstale":"2024-05-27T07:33:35.239172+0000","last_undegraded":"2024-05-27T07:33:35.239172+0000","last_fullsized":"2024-05-27T07:33:35.239172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:02:57.711197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437255+0000","last_change":"2024-05-27T07:28:24.486022+0000","last_active":"2024-05-27T07:33:35.437255+0000","last_peered":"2024-05-27T07:33:35.437255+0000","last_clean":"2024-05-27T07:33:35.437255+0000","last_became_active":"2024-05-27T07:28:24.485639+0000","last_became_peered":"2024-05-27T07:28:24.485639+0000","last_unstale":"2024-05-27T07:33:35.437255+0000","last_undegraded":"2024-05-27T07:33:35.437255+0000","last_fullsized":"2024-05-27T07:33:35.437255+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:07.629402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435201+0000","last_change":"2024-05-27T07:28:37.898984+0000","last_active":"2024-05-27T07:33:35.435201+0000","last_peered":"2024-05-27T07:33:35.435201+0000","last_clean":"2024-05-27T07:33:35.435201+0000","last_became_active":"2024-05-27T07:28:37.898132+0000","last_became_peered":"2024-05-27T07:28:37.898132+0000","last_unstale":"2024-05-27T07:33:35.435201+0000","last_undegraded":"2024-05-27T07:33:35.435201+0000","last_fullsized":"2024-05-27T07:33:35.435201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:51.987386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.239432+0000","last_change":"2024-05-27T07:28:20.402788+0000","last_active":"2024-05-27T07:33:35.239432+0000","last_peered":"2024-05-27T07:33:35.239432+0000","last_clean":"2024-05-27T07:33:35.239432+0000","last_became_active":"2024-05-27T07:28:20.402041+0000","last_became_peered":"2024-05-27T07:28:20.402041+0000","last_unstale":"2024-05-27T07:33:35.239432+0000","last_undegraded":"2024-05-27T07:33:35.239432+0000","last_fullsized":"2024-05-27T07:33:35.239432+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:44.421979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337479+0000","last_change":"2024-05-27T07:28:24.485193+0000","last_active":"2024-05-27T07:33:35.337479+0000","last_peered":"2024-05-27T07:33:35.337479+0000","last_clean":"2024-05-27T07:33:35.337479+0000","last_became_active":"2024-05-27T07:28:24.484086+0000","last_became_peered":"2024-05-27T07:28:24.484086+0000","last_unstale":"2024-05-27T07:33:35.337479+0000","last_undegraded":"2024-05-27T07:33:35.337479+0000","last_fullsized":"2024-05-27T07:33:35.337479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:10.407022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242950+0000","last_change":"2024-05-27T07:28:37.902602+0000","last_active":"2024-05-27T07:33:35.242950+0000","last_peered":"2024-05-27T07:33:35.242950+0000","last_clean":"2024-05-27T07:33:35.242950+0000","last_became_active":"2024-05-27T07:28:37.902120+0000","last_became_peered":"2024-05-27T07:28:37.902120+0000","last_unstale":"2024-05-27T07:33:35.242950+0000","last_undegraded":"2024-05-27T07:33:35.242950+0000","last_fullsized":"2024-05-27T07:33:35.242950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:21.209851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337842+0000","last_change":"2024-05-27T07:28:20.465500+0000","last_active":"2024-05-27T07:33:35.337842+0000","last_peered":"2024-05-27T07:33:35.337842+0000","last_clean":"2024-05-27T07:33:35.337842+0000","last_became_active":"2024-05-27T07:28:20.464951+0000","last_became_peered":"2024-05-27T07:28:20.464951+0000","last_unstale":"2024-05-27T07:33:35.337842+0000","last_undegraded":"2024-05-27T07:33:35.337842+0000","last_fullsized":"2024-05-27T07:33:35.337842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:23:25.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.432047+0000","last_change":"2024-05-27T07:28:24.467655+0000","last_active":"2024-05-27T07:33:35.432047+0000","last_peered":"2024-05-27T07:33:35.432047+0000","last_clean":"2024-05-27T07:33:35.432047+0000","last_became_active":"2024-05-27T07:28:24.467194+0000","last_became_peered":"2024-05-27T07:28:24.467194+0000","last_unstale":"2024-05-27T07:33:35.432047+0000","last_undegraded":"2024-05-27T07:33:35.432047+0000","last_fullsized":"2024-05-27T07:33:35.432047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:38.024514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243715+0000","last_change":"2024-05-27T07:29:04.975679+0000","last_active":"2024-05-27T07:33:35.243715+0000","last_peered":"2024-05-27T07:33:35.243715+0000","last_clean":"2024-05-27T07:33:35.243715+0000","last_became_active":"2024-05-27T07:29:04.974323+0000","last_became_peered":"2024-05-27T07:29:04.974323+0000","last_unstale":"2024-05-27T07:33:35.243715+0000","last_undegraded":"2024-05-27T07:33:35.243715+0000","last_fullsized":"2024-05-27T07:33:35.243715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:50.489737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435362+0000","last_change":"2024-05-27T07:28:20.408117+0000","last_active":"2024-05-27T07:33:35.435362+0000","last_peered":"2024-05-27T07:33:35.435362+0000","last_clean":"2024-05-27T07:33:35.435362+0000","last_became_active":"2024-05-27T07:28:20.407723+0000","last_became_peered":"2024-05-27T07:28:20.407723+0000","last_unstale":"2024-05-27T07:33:35.435362+0000","last_undegraded":"2024-05-27T07:33:35.435362+0000","last_fullsized":"2024-05-27T07:33:35.435362+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:20:31.376656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437235+0000","last_change":"2024-05-27T07:28:24.490269+0000","last_active":"2024-05-27T07:33:35.437235+0000","last_peered":"2024-05-27T07:33:35.437235+0000","last_clean":"2024-05-27T07:33:35.437235+0000","last_became_active":"2024-05-27T07:28:24.488849+0000","last_became_peered":"2024-05-27T07:28:24.488849+0000","last_unstale":"2024-05-27T07:33:35.437235+0000","last_undegraded":"2024-05-27T07:33:35.437235+0000","last_fullsized":"2024-05-27T07:33:35.437235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:00.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242985+0000","last_change":"2024-05-27T07:28:37.908658+0000","last_active":"2024-05-27T07:33:35.242985+0000","last_peered":"2024-05-27T07:33:35.242985+0000","last_clean":"2024-05-27T07:33:35.242985+0000","last_became_active":"2024-05-27T07:28:37.908056+0000","last_became_peered":"2024-05-27T07:28:37.908056+0000","last_unstale":"2024-05-27T07:33:35.242985+0000","last_undegraded":"2024-05-27T07:33:35.242985+0000","last_fullsized":"2024-05-27T07:33:35.242985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:31.853606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436498+0000","last_change":"2024-05-27T07:28:20.410944+0000","last_active":"2024-05-27T07:33:35.436498+0000","last_peered":"2024-05-27T07:33:35.436498+0000","last_clean":"2024-05-27T07:33:35.436498+0000","last_became_active":"2024-05-27T07:28:20.409789+0000","last_became_peered":"2024-05-27T07:28:20.409789+0000","last_unstale":"2024-05-27T07:33:35.436498+0000","last_undegraded":"2024-05-27T07:33:35.436498+0000","last_fullsized":"2024-05-27T07:33:35.436498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:53:34.962397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244941+0000","last_change":"2024-05-27T07:28:24.482828+0000","last_active":"2024-05-27T07:33:35.244941+0000","last_peered":"2024-05-27T07:33:35.244941+0000","last_clean":"2024-05-27T07:33:35.244941+0000","last_became_active":"2024-05-27T07:28:24.482475+0000","last_became_peered":"2024-05-27T07:28:24.482475+0000","last_unstale":"2024-05-27T07:33:35.244941+0000","last_undegraded":"2024-05-27T07:33:35.244941+0000","last_fullsized":"2024-05-27T07:33:35.244941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:53:32.064441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436828+0000","last_change":"2024-05-27T07:29:04.905734+0000","last_active":"2024-05-27T07:33:35.436828+0000","last_peered":"2024-05-27T07:33:35.436828+0000","last_clean":"2024-05-27T07:33:35.436828+0000","last_became_active":"2024-05-27T07:29:04.902387+0000","last_became_peered":"2024-05-27T07:29:04.902387+0000","last_unstale":"2024-05-27T07:33:35.436828+0000","last_undegraded":"2024-05-27T07:33:35.436828+0000","last_fullsized":"2024-05-27T07:33:35.436828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:35.993193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":285,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.741281+0000","last_change":"2024-05-27T07:28:20.412258+0000","last_active":"2024-05-27T07:33:39.741281+0000","last_peered":"2024-05-27T07:33:39.741281+0000","last_clean":"2024-05-27T07:33:39.741281+0000","last_became_active":"2024-05-27T07:28:20.411764+0000","last_became_peered":"2024-05-27T07:28:20.411764+0000","last_unstale":"2024-05-27T07:33:39.741281+0000","last_undegraded":"2024-05-27T07:33:39.741281+0000","last_fullsized":"2024-05-27T07:33:39.741281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:38.462887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":231,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434849+0000","last_change":"2024-05-27T07:28:24.482179+0000","last_active":"2024-05-27T07:33:35.434849+0000","last_peered":"2024-05-27T07:33:35.434849+0000","last_clean":"2024-05-27T07:33:35.434849+0000","last_became_active":"2024-05-27T07:28:24.480682+0000","last_became_peered":"2024-05-27T07:28:24.480682+0000","last_unstale":"2024-05-27T07:33:35.434849+0000","last_undegraded":"2024-05-27T07:33:35.434849+0000","last_fullsized":"2024-05-27T07:33:35.434849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:12:51.403862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436757+0000","last_change":"2024-05-27T07:28:37.903716+0000","last_active":"2024-05-27T07:33:35.436757+0000","last_peered":"2024-05-27T07:33:35.436757+0000","last_clean":"2024-05-27T07:33:35.436757+0000","last_became_active":"2024-05-27T07:28:37.901083+0000","last_became_peered":"2024-05-27T07:28:37.901083+0000","last_unstale":"2024-05-27T07:33:35.436757+0000","last_undegraded":"2024-05-27T07:33:35.436757+0000","last_fullsized":"2024-05-27T07:33:35.436757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:44:11.606711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401548+0000","last_change":"2024-05-27T07:28:20.392441+0000","last_active":"2024-05-27T07:33:35.401548+0000","last_peered":"2024-05-27T07:33:35.401548+0000","last_clean":"2024-05-27T07:33:35.401548+0000","last_became_active":"2024-05-27T07:28:20.390983+0000","last_became_peered":"2024-05-27T07:28:20.390983+0000","last_unstale":"2024-05-27T07:33:35.401548+0000","last_undegraded":"2024-05-27T07:33:35.401548+0000","last_fullsized":"2024-05-27T07:33:35.401548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:33:30.338043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237689+0000","last_change":"2024-05-27T07:30:05.518915+0000","last_active":"2024-05-27T07:33:35.237689+0000","last_peered":"2024-05-27T07:33:35.237689+0000","last_clean":"2024-05-27T07:33:35.237689+0000","last_became_active":"2024-05-27T07:30:05.517894+0000","last_became_peered":"2024-05-27T07:30:05.517894+0000","last_unstale":"2024-05-27T07:33:35.237689+0000","last_undegraded":"2024-05-27T07:33:35.237689+0000","last_fullsized":"2024-05-27T07:33:35.237689+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:06:51.353713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,7],"acting":[9,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384762+0000","last_change":"2024-05-27T07:28:37.913352+0000","last_active":"2024-05-27T07:33:35.384762+0000","last_peered":"2024-05-27T07:33:35.384762+0000","last_clean":"2024-05-27T07:33:35.384762+0000","last_became_active":"2024-05-27T07:28:37.913027+0000","last_became_peered":"2024-05-27T07:28:37.913027+0000","last_unstale":"2024-05-27T07:33:35.384762+0000","last_undegraded":"2024-05-27T07:33:35.384762+0000","last_fullsized":"2024-05-27T07:33:35.384762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:03.485594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369823+0000","last_change":"2024-05-27T07:28:20.406094+0000","last_active":"2024-05-27T07:33:35.369823+0000","last_peered":"2024-05-27T07:33:35.369823+0000","last_clean":"2024-05-27T07:33:35.369823+0000","last_became_active":"2024-05-27T07:28:20.405641+0000","last_became_peered":"2024-05-27T07:28:20.405641+0000","last_unstale":"2024-05-27T07:33:35.369823+0000","last_undegraded":"2024-05-27T07:33:35.369823+0000","last_fullsized":"2024-05-27T07:33:35.369823+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:55:16.598743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337048+0000","last_change":"2024-05-27T07:28:24.474696+0000","last_active":"2024-05-27T07:33:35.337048+0000","last_peered":"2024-05-27T07:33:35.337048+0000","last_clean":"2024-05-27T07:33:35.337048+0000","last_became_active":"2024-05-27T07:28:24.474319+0000","last_became_peered":"2024-05-27T07:28:24.474319+0000","last_unstale":"2024-05-27T07:33:35.337048+0000","last_undegraded":"2024-05-27T07:33:35.337048+0000","last_fullsized":"2024-05-27T07:33:35.337048+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:35:53.010898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431719+0000","last_change":"2024-05-27T07:28:37.910386+0000","last_active":"2024-05-27T07:33:35.431719+0000","last_peered":"2024-05-27T07:33:35.431719+0000","last_clean":"2024-05-27T07:33:35.431719+0000","last_became_active":"2024-05-27T07:28:37.909941+0000","last_became_peered":"2024-05-27T07:28:37.909941+0000","last_unstale":"2024-05-27T07:33:35.431719+0000","last_undegraded":"2024-05-27T07:33:35.431719+0000","last_fullsized":"2024-05-27T07:33:35.431719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:15.820318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242969+0000","last_change":"2024-05-27T07:28:20.411855+0000","last_active":"2024-05-27T07:33:35.242969+0000","last_peered":"2024-05-27T07:33:35.242969+0000","last_clean":"2024-05-27T07:33:35.242969+0000","last_became_active":"2024-05-27T07:28:20.411169+0000","last_became_peered":"2024-05-27T07:28:20.411169+0000","last_unstale":"2024-05-27T07:33:35.242969+0000","last_undegraded":"2024-05-27T07:33:35.242969+0000","last_fullsized":"2024-05-27T07:33:35.242969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:37.278507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237559+0000","last_change":"2024-05-27T07:28:24.485799+0000","last_active":"2024-05-27T07:33:35.237559+0000","last_peered":"2024-05-27T07:33:35.237559+0000","last_clean":"2024-05-27T07:33:35.237559+0000","last_became_active":"2024-05-27T07:28:24.485395+0000","last_became_peered":"2024-05-27T07:28:24.485395+0000","last_unstale":"2024-05-27T07:33:35.237559+0000","last_undegraded":"2024-05-27T07:33:35.237559+0000","last_fullsized":"2024-05-27T07:33:35.237559+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:11.178129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336979+0000","last_change":"2024-05-27T07:28:37.898929+0000","last_active":"2024-05-27T07:33:35.336979+0000","last_peered":"2024-05-27T07:33:35.336979+0000","last_clean":"2024-05-27T07:33:35.336979+0000","last_became_active":"2024-05-27T07:28:37.898371+0000","last_became_peered":"2024-05-27T07:28:37.898371+0000","last_unstale":"2024-05-27T07:33:35.336979+0000","last_undegraded":"2024-05-27T07:33:35.336979+0000","last_fullsized":"2024-05-27T07:33:35.336979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:03:48.867729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":280,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.740411+0000","last_change":"2024-05-27T07:28:20.407117+0000","last_active":"2024-05-27T07:33:39.740411+0000","last_peered":"2024-05-27T07:33:39.740411+0000","last_clean":"2024-05-27T07:33:39.740411+0000","last_became_active":"2024-05-27T07:28:20.403562+0000","last_became_peered":"2024-05-27T07:28:20.403562+0000","last_unstale":"2024-05-27T07:33:39.740411+0000","last_undegraded":"2024-05-27T07:33:39.740411+0000","last_fullsized":"2024-05-27T07:33:39.740411+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:38:25.903886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.438353+0000","last_change":"2024-05-27T07:28:24.489771+0000","last_active":"2024-05-27T07:33:35.438353+0000","last_peered":"2024-05-27T07:33:35.438353+0000","last_clean":"2024-05-27T07:33:35.438353+0000","last_became_active":"2024-05-27T07:28:24.489163+0000","last_became_peered":"2024-05-27T07:28:24.489163+0000","last_unstale":"2024-05-27T07:33:35.438353+0000","last_undegraded":"2024-05-27T07:33:35.438353+0000","last_fullsized":"2024-05-27T07:33:35.438353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:08.053634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237277+0000","last_change":"2024-05-27T07:28:37.914598+0000","last_active":"2024-05-27T07:33:35.237277+0000","last_peered":"2024-05-27T07:33:35.237277+0000","last_clean":"2024-05-27T07:33:35.237277+0000","last_became_active":"2024-05-27T07:28:37.913916+0000","last_became_peered":"2024-05-27T07:28:37.913916+0000","last_unstale":"2024-05-27T07:33:35.237277+0000","last_undegraded":"2024-05-27T07:33:35.237277+0000","last_fullsized":"2024-05-27T07:33:35.237277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:13:01.475789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436369+0000","last_change":"2024-05-27T07:29:04.901366+0000","last_active":"2024-05-27T07:33:35.436369+0000","last_peered":"2024-05-27T07:33:35.436369+0000","last_clean":"2024-05-27T07:33:35.436369+0000","last_became_active":"2024-05-27T07:29:04.900355+0000","last_became_peered":"2024-05-27T07:29:04.900355+0000","last_unstale":"2024-05-27T07:33:35.436369+0000","last_undegraded":"2024-05-27T07:33:35.436369+0000","last_fullsized":"2024-05-27T07:33:35.436369+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:10:03.349345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.432117+0000","last_change":"2024-05-27T07:28:24.477771+0000","last_active":"2024-05-27T07:33:35.432117+0000","last_peered":"2024-05-27T07:33:35.432117+0000","last_clean":"2024-05-27T07:33:35.432117+0000","last_became_active":"2024-05-27T07:28:24.474179+0000","last_became_peered":"2024-05-27T07:28:24.474179+0000","last_unstale":"2024-05-27T07:33:35.432117+0000","last_undegraded":"2024-05-27T07:33:35.432117+0000","last_fullsized":"2024-05-27T07:33:35.432117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:08:04.218264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243888+0000","last_change":"2024-05-27T07:29:04.896583+0000","last_active":"2024-05-27T07:33:35.243888+0000","last_peered":"2024-05-27T07:33:35.243888+0000","last_clean":"2024-05-27T07:33:35.243888+0000","last_became_active":"2024-05-27T07:29:04.894118+0000","last_became_peered":"2024-05-27T07:29:04.894118+0000","last_unstale":"2024-05-27T07:33:35.243888+0000","last_undegraded":"2024-05-27T07:33:35.243888+0000","last_fullsized":"2024-05-27T07:33:35.243888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:17.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244297+0000","last_change":"2024-05-27T07:28:20.466851+0000","last_active":"2024-05-27T07:33:35.244297+0000","last_peered":"2024-05-27T07:33:35.244297+0000","last_clean":"2024-05-27T07:33:35.244297+0000","last_became_active":"2024-05-27T07:28:20.465133+0000","last_became_peered":"2024-05-27T07:28:20.465133+0000","last_unstale":"2024-05-27T07:33:35.244297+0000","last_undegraded":"2024-05-27T07:33:35.244297+0000","last_fullsized":"2024-05-27T07:33:35.244297+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:11.025588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437815+0000","last_change":"2024-05-27T07:28:24.469924+0000","last_active":"2024-05-27T07:33:35.437815+0000","last_peered":"2024-05-27T07:33:35.437815+0000","last_clean":"2024-05-27T07:33:35.437815+0000","last_became_active":"2024-05-27T07:28:24.469559+0000","last_became_peered":"2024-05-27T07:28:24.469559+0000","last_unstale":"2024-05-27T07:33:35.437815+0000","last_undegraded":"2024-05-27T07:33:35.437815+0000","last_fullsized":"2024-05-27T07:33:35.437815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:57.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.237901+0000","last_change":"2024-05-27T07:28:37.904549+0000","last_active":"2024-05-27T07:33:35.237901+0000","last_peered":"2024-05-27T07:33:35.237901+0000","last_clean":"2024-05-27T07:33:35.237901+0000","last_became_active":"2024-05-27T07:28:37.903882+0000","last_became_peered":"2024-05-27T07:28:37.903882+0000","last_unstale":"2024-05-27T07:33:35.237901+0000","last_undegraded":"2024-05-27T07:33:35.237901+0000","last_fullsized":"2024-05-27T07:33:35.237901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:41.607641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'44","reported_seq":563,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:40.741316+0000","last_change":"2024-05-27T07:28:20.410025+0000","last_active":"2024-05-27T07:33:40.741316+0000","last_peered":"2024-05-27T07:33:40.741316+0000","last_clean":"2024-05-27T07:33:40.741316+0000","last_became_active":"2024-05-27T07:28:20.409115+0000","last_became_peered":"2024-05-27T07:28:20.409115+0000","last_unstale":"2024-05-27T07:33:40.741316+0000","last_undegraded":"2024-05-27T07:33:40.741316+0000","last_fullsized":"2024-05-27T07:33:40.741316+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:56.834059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77315,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":471,"num_read_kb":309,"num_write":45,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436027+0000","last_change":"2024-05-27T07:28:24.469609+0000","last_active":"2024-05-27T07:33:35.436027+0000","last_peered":"2024-05-27T07:33:35.436027+0000","last_clean":"2024-05-27T07:33:35.436027+0000","last_became_active":"2024-05-27T07:28:24.469192+0000","last_became_peered":"2024-05-27T07:28:24.469192+0000","last_unstale":"2024-05-27T07:33:35.436027+0000","last_undegraded":"2024-05-27T07:33:35.436027+0000","last_fullsized":"2024-05-27T07:33:35.436027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:32:59.006298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.246044+0000","last_change":"2024-05-27T07:28:37.900745+0000","last_active":"2024-05-27T07:33:35.246044+0000","last_peered":"2024-05-27T07:33:35.246044+0000","last_clean":"2024-05-27T07:33:35.246044+0000","last_became_active":"2024-05-27T07:28:37.900205+0000","last_became_peered":"2024-05-27T07:28:37.900205+0000","last_unstale":"2024-05-27T07:33:35.246044+0000","last_undegraded":"2024-05-27T07:33:35.246044+0000","last_fullsized":"2024-05-27T07:33:35.246044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:52:32.235201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435338+0000","last_change":"2024-05-27T07:28:20.413380+0000","last_active":"2024-05-27T07:33:35.435338+0000","last_peered":"2024-05-27T07:33:35.435338+0000","last_clean":"2024-05-27T07:33:35.435338+0000","last_became_active":"2024-05-27T07:28:20.412839+0000","last_became_peered":"2024-05-27T07:28:20.412839+0000","last_unstale":"2024-05-27T07:33:35.435338+0000","last_undegraded":"2024-05-27T07:33:35.435338+0000","last_fullsized":"2024-05-27T07:33:35.435338+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:36:57.481203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369637+0000","last_change":"2024-05-27T07:28:24.474228+0000","last_active":"2024-05-27T07:33:35.369637+0000","last_peered":"2024-05-27T07:33:35.369637+0000","last_clean":"2024-05-27T07:33:35.369637+0000","last_became_active":"2024-05-27T07:28:24.473833+0000","last_became_peered":"2024-05-27T07:28:24.473833+0000","last_unstale":"2024-05-27T07:33:35.369637+0000","last_undegraded":"2024-05-27T07:33:35.369637+0000","last_fullsized":"2024-05-27T07:33:35.369637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:09:52.456730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384646+0000","last_change":"2024-05-27T07:28:37.917630+0000","last_active":"2024-05-27T07:33:35.384646+0000","last_peered":"2024-05-27T07:33:35.384646+0000","last_clean":"2024-05-27T07:33:35.384646+0000","last_became_active":"2024-05-27T07:28:37.917253+0000","last_became_peered":"2024-05-27T07:28:37.917253+0000","last_unstale":"2024-05-27T07:33:35.384646+0000","last_undegraded":"2024-05-27T07:33:35.384646+0000","last_fullsized":"2024-05-27T07:33:35.384646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:05:05.069946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243812+0000","last_change":"2024-05-27T07:28:20.408988+0000","last_active":"2024-05-27T07:33:35.243812+0000","last_peered":"2024-05-27T07:33:35.243812+0000","last_clean":"2024-05-27T07:33:35.243812+0000","last_became_active":"2024-05-27T07:28:20.408313+0000","last_became_peered":"2024-05-27T07:28:20.408313+0000","last_unstale":"2024-05-27T07:33:35.243812+0000","last_undegraded":"2024-05-27T07:33:35.243812+0000","last_fullsized":"2024-05-27T07:33:35.243812+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:17.480412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435459+0000","last_change":"2024-05-27T07:28:24.480096+0000","last_active":"2024-05-27T07:33:35.435459+0000","last_peered":"2024-05-27T07:33:35.435459+0000","last_clean":"2024-05-27T07:33:35.435459+0000","last_became_active":"2024-05-27T07:28:24.477988+0000","last_became_peered":"2024-05-27T07:28:24.477988+0000","last_unstale":"2024-05-27T07:33:35.435459+0000","last_undegraded":"2024-05-27T07:33:35.435459+0000","last_fullsized":"2024-05-27T07:33:35.435459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:13:59.860458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.238087+0000","last_change":"2024-05-27T07:28:37.917518+0000","last_active":"2024-05-27T07:33:35.238087+0000","last_peered":"2024-05-27T07:33:35.238087+0000","last_clean":"2024-05-27T07:33:35.238087+0000","last_became_active":"2024-05-27T07:28:37.916971+0000","last_became_peered":"2024-05-27T07:28:37.916971+0000","last_unstale":"2024-05-27T07:33:35.238087+0000","last_undegraded":"2024-05-27T07:33:35.238087+0000","last_fullsized":"2024-05-27T07:33:35.238087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:17:53.067747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.783309+0000","last_change":"2024-05-27T07:28:20.416766+0000","last_active":"2024-05-27T07:33:39.783309+0000","last_peered":"2024-05-27T07:33:39.783309+0000","last_clean":"2024-05-27T07:33:39.783309+0000","last_became_active":"2024-05-27T07:28:20.416297+0000","last_became_peered":"2024-05-27T07:28:20.416297+0000","last_unstale":"2024-05-27T07:33:39.783309+0000","last_undegraded":"2024-05-27T07:33:39.783309+0000","last_fullsized":"2024-05-27T07:33:39.783309+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:20:02.841494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2254,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":233,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435388+0000","last_change":"2024-05-27T07:28:24.486869+0000","last_active":"2024-05-27T07:33:35.435388+0000","last_peered":"2024-05-27T07:33:35.435388+0000","last_clean":"2024-05-27T07:33:35.435388+0000","last_became_active":"2024-05-27T07:28:24.486451+0000","last_became_peered":"2024-05-27T07:28:24.486451+0000","last_unstale":"2024-05-27T07:33:35.435388+0000","last_undegraded":"2024-05-27T07:33:35.435388+0000","last_fullsized":"2024-05-27T07:33:35.435388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:10.722659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370829+0000","last_change":"2024-05-27T07:29:04.901632+0000","last_active":"2024-05-27T07:33:35.370829+0000","last_peered":"2024-05-27T07:33:35.370829+0000","last_clean":"2024-05-27T07:33:35.370829+0000","last_became_active":"2024-05-27T07:29:04.900203+0000","last_became_peered":"2024-05-27T07:29:04.900203+0000","last_unstale":"2024-05-27T07:33:35.370829+0000","last_undegraded":"2024-05-27T07:33:35.370829+0000","last_fullsized":"2024-05-27T07:33:35.370829+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:57.834889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":63,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434045+0000","last_change":"2024-05-27T07:28:20.468162+0000","last_active":"2024-05-27T07:33:35.434045+0000","last_peered":"2024-05-27T07:33:35.434045+0000","last_clean":"2024-05-27T07:33:35.434045+0000","last_became_active":"2024-05-27T07:28:20.465226+0000","last_became_peered":"2024-05-27T07:28:20.465226+0000","last_unstale":"2024-05-27T07:33:35.434045+0000","last_undegraded":"2024-05-27T07:33:35.434045+0000","last_fullsized":"2024-05-27T07:33:35.434045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:21:34.110356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243300+0000","last_change":"2024-05-27T07:28:24.485053+0000","last_active":"2024-05-27T07:33:35.243300+0000","last_peered":"2024-05-27T07:33:35.243300+0000","last_clean":"2024-05-27T07:33:35.243300+0000","last_became_active":"2024-05-27T07:28:24.484391+0000","last_became_peered":"2024-05-27T07:28:24.484391+0000","last_unstale":"2024-05-27T07:33:35.243300+0000","last_undegraded":"2024-05-27T07:33:35.243300+0000","last_fullsized":"2024-05-27T07:33:35.243300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:25:05.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243032+0000","last_change":"2024-05-27T07:28:37.904731+0000","last_active":"2024-05-27T07:33:35.243032+0000","last_peered":"2024-05-27T07:33:35.243032+0000","last_clean":"2024-05-27T07:33:35.243032+0000","last_became_active":"2024-05-27T07:28:37.904284+0000","last_became_peered":"2024-05-27T07:28:37.904284+0000","last_unstale":"2024-05-27T07:33:35.243032+0000","last_undegraded":"2024-05-27T07:33:35.243032+0000","last_fullsized":"2024-05-27T07:33:35.243032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:23.198465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435888+0000","last_change":"2024-05-27T07:28:20.406829+0000","last_active":"2024-05-27T07:33:35.435888+0000","last_peered":"2024-05-27T07:33:35.435888+0000","last_clean":"2024-05-27T07:33:35.435888+0000","last_became_active":"2024-05-27T07:28:20.405497+0000","last_became_peered":"2024-05-27T07:28:20.405497+0000","last_unstale":"2024-05-27T07:33:35.435888+0000","last_undegraded":"2024-05-27T07:33:35.435888+0000","last_fullsized":"2024-05-27T07:33:35.435888+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:14.225630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385238+0000","last_change":"2024-05-27T07:28:24.488023+0000","last_active":"2024-05-27T07:33:35.385238+0000","last_peered":"2024-05-27T07:33:35.385238+0000","last_clean":"2024-05-27T07:33:35.385238+0000","last_became_active":"2024-05-27T07:28:24.487652+0000","last_became_peered":"2024-05-27T07:28:24.487652+0000","last_unstale":"2024-05-27T07:33:35.385238+0000","last_undegraded":"2024-05-27T07:33:35.385238+0000","last_fullsized":"2024-05-27T07:33:35.385238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:25.951157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434437+0000","last_change":"2024-05-27T07:28:37.968034+0000","last_active":"2024-05-27T07:33:35.434437+0000","last_peered":"2024-05-27T07:33:35.434437+0000","last_clean":"2024-05-27T07:33:35.434437+0000","last_became_active":"2024-05-27T07:28:37.967127+0000","last_became_peered":"2024-05-27T07:28:37.967127+0000","last_unstale":"2024-05-27T07:33:35.434437+0000","last_undegraded":"2024-05-27T07:33:35.434437+0000","last_fullsized":"2024-05-27T07:33:35.434437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:49.353423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.238627+0000","last_change":"2024-05-27T07:28:20.465782+0000","last_active":"2024-05-27T07:33:35.238627+0000","last_peered":"2024-05-27T07:33:35.238627+0000","last_clean":"2024-05-27T07:33:35.238627+0000","last_became_active":"2024-05-27T07:28:20.465134+0000","last_became_peered":"2024-05-27T07:28:20.465134+0000","last_unstale":"2024-05-27T07:33:35.238627+0000","last_undegraded":"2024-05-27T07:33:35.238627+0000","last_fullsized":"2024-05-27T07:33:35.238627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:06:17.992092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401705+0000","last_change":"2024-05-27T07:28:24.484691+0000","last_active":"2024-05-27T07:33:35.401705+0000","last_peered":"2024-05-27T07:33:35.401705+0000","last_clean":"2024-05-27T07:33:35.401705+0000","last_became_active":"2024-05-27T07:28:24.484249+0000","last_became_peered":"2024-05-27T07:28:24.484249+0000","last_unstale":"2024-05-27T07:33:35.401705+0000","last_undegraded":"2024-05-27T07:33:35.401705+0000","last_fullsized":"2024-05-27T07:33:35.401705+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:25.753276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369751+0000","last_change":"2024-05-27T07:28:37.907849+0000","last_active":"2024-05-27T07:33:35.369751+0000","last_peered":"2024-05-27T07:33:35.369751+0000","last_clean":"2024-05-27T07:33:35.369751+0000","last_became_active":"2024-05-27T07:28:37.907105+0000","last_became_peered":"2024-05-27T07:28:37.907105+0000","last_unstale":"2024-05-27T07:33:35.369751+0000","last_undegraded":"2024-05-27T07:33:35.369751+0000","last_fullsized":"2024-05-27T07:33:35.369751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:54.075240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.433916+0000","last_change":"2024-05-27T07:28:20.411302+0000","last_active":"2024-05-27T07:33:35.433916+0000","last_peered":"2024-05-27T07:33:35.433916+0000","last_clean":"2024-05-27T07:33:35.433916+0000","last_became_active":"2024-05-27T07:28:20.409144+0000","last_became_peered":"2024-05-27T07:28:20.409144+0000","last_unstale":"2024-05-27T07:33:35.433916+0000","last_undegraded":"2024-05-27T07:33:35.433916+0000","last_fullsized":"2024-05-27T07:33:35.433916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:49.985962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.401624+0000","last_change":"2024-05-27T07:28:24.487742+0000","last_active":"2024-05-27T07:33:35.401624+0000","last_peered":"2024-05-27T07:33:35.401624+0000","last_clean":"2024-05-27T07:33:35.401624+0000","last_became_active":"2024-05-27T07:28:24.485279+0000","last_became_peered":"2024-05-27T07:28:24.485279+0000","last_unstale":"2024-05-27T07:33:35.401624+0000","last_undegraded":"2024-05-27T07:33:35.401624+0000","last_fullsized":"2024-05-27T07:33:35.401624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:15.049572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.371172+0000","last_change":"2024-05-27T07:28:37.968780+0000","last_active":"2024-05-27T07:33:35.371172+0000","last_peered":"2024-05-27T07:33:35.371172+0000","last_clean":"2024-05-27T07:33:35.371172+0000","last_became_active":"2024-05-27T07:28:37.968052+0000","last_became_peered":"2024-05-27T07:28:37.968052+0000","last_unstale":"2024-05-27T07:33:35.371172+0000","last_undegraded":"2024-05-27T07:33:35.371172+0000","last_fullsized":"2024-05-27T07:33:35.371172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:51:26.787969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244844+0000","last_change":"2024-05-27T07:28:24.485280+0000","last_active":"2024-05-27T07:33:35.244844+0000","last_peered":"2024-05-27T07:33:35.244844+0000","last_clean":"2024-05-27T07:33:35.244844+0000","last_became_active":"2024-05-27T07:28:24.484882+0000","last_became_peered":"2024-05-27T07:28:24.484882+0000","last_unstale":"2024-05-27T07:33:35.244844+0000","last_undegraded":"2024-05-27T07:33:35.244844+0000","last_fullsized":"2024-05-27T07:33:35.244844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:28.643857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":280,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:39.781145+0000","last_change":"2024-05-27T07:28:20.462593+0000","last_active":"2024-05-27T07:33:39.781145+0000","last_peered":"2024-05-27T07:33:39.781145+0000","last_clean":"2024-05-27T07:33:39.781145+0000","last_became_active":"2024-05-27T07:28:20.461128+0000","last_became_peered":"2024-05-27T07:28:20.461128+0000","last_unstale":"2024-05-27T07:33:39.781145+0000","last_undegraded":"2024-05-27T07:33:39.781145+0000","last_fullsized":"2024-05-27T07:33:39.781145+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:24:42.013315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436013+0000","last_change":"2024-05-27T07:28:37.911546+0000","last_active":"2024-05-27T07:33:35.436013+0000","last_peered":"2024-05-27T07:33:35.436013+0000","last_clean":"2024-05-27T07:33:35.436013+0000","last_became_active":"2024-05-27T07:28:37.910777+0000","last_became_peered":"2024-05-27T07:28:37.910777+0000","last_unstale":"2024-05-27T07:33:35.436013+0000","last_undegraded":"2024-05-27T07:33:35.436013+0000","last_fullsized":"2024-05-27T07:33:35.436013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:13.344317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336963+0000","last_change":"2024-05-27T07:28:24.483946+0000","last_active":"2024-05-27T07:33:35.336963+0000","last_peered":"2024-05-27T07:33:35.336963+0000","last_clean":"2024-05-27T07:33:35.336963+0000","last_became_active":"2024-05-27T07:28:24.481424+0000","last_became_peered":"2024-05-27T07:28:24.481424+0000","last_unstale":"2024-05-27T07:33:35.336963+0000","last_undegraded":"2024-05-27T07:33:35.336963+0000","last_fullsized":"2024-05-27T07:33:35.336963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:04:08.254046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243763+0000","last_change":"2024-05-27T07:28:20.414250+0000","last_active":"2024-05-27T07:33:35.243763+0000","last_peered":"2024-05-27T07:33:35.243763+0000","last_clean":"2024-05-27T07:33:35.243763+0000","last_became_active":"2024-05-27T07:28:20.413676+0000","last_became_peered":"2024-05-27T07:28:20.413676+0000","last_unstale":"2024-05-27T07:33:35.243763+0000","last_undegraded":"2024-05-27T07:33:35.243763+0000","last_fullsized":"2024-05-27T07:33:35.243763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:10.226771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'99","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:37.400097+0000","last_change":"2024-05-27T07:26:26.055626+0000","last_active":"2024-05-27T07:33:37.400097+0000","last_peered":"2024-05-27T07:33:37.400097+0000","last_clean":"2024-05-27T07:33:37.400097+0000","last_became_active":"2024-05-27T07:26:26.037376+0000","last_became_peered":"2024-05-27T07:26:26.037376+0000","last_unstale":"2024-05-27T07:33:37.400097+0000","last_undegraded":"2024-05-27T07:33:37.400097+0000","last_fullsized":"2024-05-27T07:33:37.400097+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:21:26.641949+0000","last_clean_scrub_stamp":"2024-05-27T07:21:26.641949+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:16.741856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244470+0000","last_change":"2024-05-27T07:28:37.973078+0000","last_active":"2024-05-27T07:33:35.244470+0000","last_peered":"2024-05-27T07:33:35.244470+0000","last_clean":"2024-05-27T07:33:35.244470+0000","last_became_active":"2024-05-27T07:28:37.972381+0000","last_became_peered":"2024-05-27T07:28:37.972381+0000","last_unstale":"2024-05-27T07:33:35.244470+0000","last_undegraded":"2024-05-27T07:33:35.244470+0000","last_fullsized":"2024-05-27T07:33:35.244470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:18:16.912193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369481+0000","last_change":"2024-05-27T07:28:24.483512+0000","last_active":"2024-05-27T07:33:35.369481+0000","last_peered":"2024-05-27T07:33:35.369481+0000","last_clean":"2024-05-27T07:33:35.369481+0000","last_became_active":"2024-05-27T07:28:24.482655+0000","last_became_peered":"2024-05-27T07:28:24.482655+0000","last_unstale":"2024-05-27T07:33:35.369481+0000","last_undegraded":"2024-05-27T07:33:35.369481+0000","last_fullsized":"2024-05-27T07:33:35.369481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:01:07.898504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434986+0000","last_change":"2024-05-27T07:28:20.408953+0000","last_active":"2024-05-27T07:33:35.434986+0000","last_peered":"2024-05-27T07:33:35.434986+0000","last_clean":"2024-05-27T07:33:35.434986+0000","last_became_active":"2024-05-27T07:28:20.407631+0000","last_became_peered":"2024-05-27T07:28:20.407631+0000","last_unstale":"2024-05-27T07:33:35.434986+0000","last_undegraded":"2024-05-27T07:33:35.434986+0000","last_fullsized":"2024-05-27T07:33:35.434986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:43.912338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384425+0000","last_change":"2024-05-27T07:28:37.907441+0000","last_active":"2024-05-27T07:33:35.384425+0000","last_peered":"2024-05-27T07:33:35.384425+0000","last_clean":"2024-05-27T07:33:35.384425+0000","last_became_active":"2024-05-27T07:28:37.906928+0000","last_became_peered":"2024-05-27T07:28:37.906928+0000","last_unstale":"2024-05-27T07:33:35.384425+0000","last_undegraded":"2024-05-27T07:33:35.384425+0000","last_fullsized":"2024-05-27T07:33:35.384425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:28:32.345826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435906+0000","last_change":"2024-05-27T07:28:20.467408+0000","last_active":"2024-05-27T07:33:35.435906+0000","last_peered":"2024-05-27T07:33:35.435906+0000","last_clean":"2024-05-27T07:33:35.435906+0000","last_became_active":"2024-05-27T07:28:20.466064+0000","last_became_peered":"2024-05-27T07:28:20.466064+0000","last_unstale":"2024-05-27T07:33:35.435906+0000","last_undegraded":"2024-05-27T07:33:35.435906+0000","last_fullsized":"2024-05-27T07:33:35.435906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:15:51.449514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.371394+0000","last_change":"2024-05-27T07:28:24.478475+0000","last_active":"2024-05-27T07:33:35.371394+0000","last_peered":"2024-05-27T07:33:35.371394+0000","last_clean":"2024-05-27T07:33:35.371394+0000","last_became_active":"2024-05-27T07:28:24.477936+0000","last_became_peered":"2024-05-27T07:28:24.477936+0000","last_unstale":"2024-05-27T07:33:35.371394+0000","last_undegraded":"2024-05-27T07:33:35.371394+0000","last_fullsized":"2024-05-27T07:33:35.371394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:53:17.500040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.336847+0000","last_change":"2024-05-27T07:28:37.968192+0000","last_active":"2024-05-27T07:33:35.336847+0000","last_peered":"2024-05-27T07:33:35.336847+0000","last_clean":"2024-05-27T07:33:35.336847+0000","last_became_active":"2024-05-27T07:28:37.967603+0000","last_became_peered":"2024-05-27T07:28:37.967603+0000","last_unstale":"2024-05-27T07:33:35.336847+0000","last_undegraded":"2024-05-27T07:33:35.336847+0000","last_fullsized":"2024-05-27T07:33:35.336847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:53:36.651238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'49","reported_seq":129,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.622094+0000","last_change":"2024-05-27T07:29:04.901062+0000","last_active":"2024-05-27T07:33:35.622094+0000","last_peered":"2024-05-27T07:33:35.622094+0000","last_clean":"2024-05-27T07:33:35.622094+0000","last_became_active":"2024-05-27T07:29:04.892060+0000","last_became_peered":"2024-05-27T07:29:04.892060+0000","last_unstale":"2024-05-27T07:33:35.622094+0000","last_undegraded":"2024-05-27T07:33:35.622094+0000","last_fullsized":"2024-05-27T07:33:35.622094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:27:37.894263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337180+0000","last_change":"2024-05-27T07:28:24.477096+0000","last_active":"2024-05-27T07:33:35.337180+0000","last_peered":"2024-05-27T07:33:35.337180+0000","last_clean":"2024-05-27T07:33:35.337180+0000","last_became_active":"2024-05-27T07:28:24.476712+0000","last_became_peered":"2024-05-27T07:28:24.476712+0000","last_unstale":"2024-05-27T07:33:35.337180+0000","last_undegraded":"2024-05-27T07:33:35.337180+0000","last_fullsized":"2024-05-27T07:33:35.337180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:31:58.010761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434405+0000","last_change":"2024-05-27T07:28:37.894013+0000","last_active":"2024-05-27T07:33:35.434405+0000","last_peered":"2024-05-27T07:33:35.434405+0000","last_clean":"2024-05-27T07:33:35.434405+0000","last_became_active":"2024-05-27T07:28:37.893345+0000","last_became_peered":"2024-05-27T07:28:37.893345+0000","last_unstale":"2024-05-27T07:33:35.434405+0000","last_undegraded":"2024-05-27T07:33:35.434405+0000","last_fullsized":"2024-05-27T07:33:35.434405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:26.917731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.244487+0000","last_change":"2024-05-27T07:28:20.744087+0000","last_active":"2024-05-27T07:33:35.244487+0000","last_peered":"2024-05-27T07:33:35.244487+0000","last_clean":"2024-05-27T07:33:35.244487+0000","last_became_active":"2024-05-27T07:28:20.742883+0000","last_became_peered":"2024-05-27T07:28:20.742883+0000","last_unstale":"2024-05-27T07:33:35.244487+0000","last_undegraded":"2024-05-27T07:33:35.244487+0000","last_fullsized":"2024-05-27T07:33:35.244487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:06.083348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431834+0000","last_change":"2024-05-27T07:28:24.480330+0000","last_active":"2024-05-27T07:33:35.431834+0000","last_peered":"2024-05-27T07:33:35.431834+0000","last_clean":"2024-05-27T07:33:35.431834+0000","last_became_active":"2024-05-27T07:28:24.479300+0000","last_became_peered":"2024-05-27T07:28:24.479300+0000","last_unstale":"2024-05-27T07:33:35.431834+0000","last_undegraded":"2024-05-27T07:33:35.431834+0000","last_fullsized":"2024-05-27T07:33:35.431834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:59:43.802273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243811+0000","last_change":"2024-05-27T07:28:37.973592+0000","last_active":"2024-05-27T07:33:35.243811+0000","last_peered":"2024-05-27T07:33:35.243811+0000","last_clean":"2024-05-27T07:33:35.243811+0000","last_became_active":"2024-05-27T07:28:37.972475+0000","last_became_peered":"2024-05-27T07:28:37.972475+0000","last_unstale":"2024-05-27T07:33:35.243811+0000","last_undegraded":"2024-05-27T07:33:35.243811+0000","last_fullsized":"2024-05-27T07:33:35.243811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:31.662384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434870+0000","last_change":"2024-05-27T07:29:04.890751+0000","last_active":"2024-05-27T07:33:35.434870+0000","last_peered":"2024-05-27T07:33:35.434870+0000","last_clean":"2024-05-27T07:33:35.434870+0000","last_became_active":"2024-05-27T07:29:04.889447+0000","last_became_peered":"2024-05-27T07:29:04.889447+0000","last_unstale":"2024-05-27T07:33:35.434870+0000","last_undegraded":"2024-05-27T07:33:35.434870+0000","last_fullsized":"2024-05-27T07:33:35.434870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:13:34.743691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434302+0000","last_change":"2024-05-27T07:28:24.474298+0000","last_active":"2024-05-27T07:33:35.434302+0000","last_peered":"2024-05-27T07:33:35.434302+0000","last_clean":"2024-05-27T07:33:35.434302+0000","last_became_active":"2024-05-27T07:28:24.473877+0000","last_became_peered":"2024-05-27T07:28:24.473877+0000","last_unstale":"2024-05-27T07:33:35.434302+0000","last_undegraded":"2024-05-27T07:33:35.434302+0000","last_fullsized":"2024-05-27T07:33:35.434302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:31.545883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.239267+0000","last_change":"2024-05-27T07:28:37.905417+0000","last_active":"2024-05-27T07:33:35.239267+0000","last_peered":"2024-05-27T07:33:35.239267+0000","last_clean":"2024-05-27T07:33:35.239267+0000","last_became_active":"2024-05-27T07:28:37.904968+0000","last_became_peered":"2024-05-27T07:28:37.904968+0000","last_unstale":"2024-05-27T07:33:35.239267+0000","last_undegraded":"2024-05-27T07:33:35.239267+0000","last_fullsized":"2024-05-27T07:33:35.239267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:02.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337111+0000","last_change":"2024-05-27T07:28:20.392647+0000","last_active":"2024-05-27T07:33:35.337111+0000","last_peered":"2024-05-27T07:33:35.337111+0000","last_clean":"2024-05-27T07:33:35.337111+0000","last_became_active":"2024-05-27T07:28:20.392073+0000","last_became_peered":"2024-05-27T07:28:20.392073+0000","last_unstale":"2024-05-27T07:33:35.337111+0000","last_undegraded":"2024-05-27T07:33:35.337111+0000","last_fullsized":"2024-05-27T07:33:35.337111+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:18.777628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384408+0000","last_change":"2024-05-27T07:28:24.490522+0000","last_active":"2024-05-27T07:33:35.384408+0000","last_peered":"2024-05-27T07:33:35.384408+0000","last_clean":"2024-05-27T07:33:35.384408+0000","last_became_active":"2024-05-27T07:28:24.485228+0000","last_became_peered":"2024-05-27T07:28:24.485228+0000","last_unstale":"2024-05-27T07:33:35.384408+0000","last_undegraded":"2024-05-27T07:33:35.384408+0000","last_fullsized":"2024-05-27T07:33:35.384408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:32:14.221016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243254+0000","last_change":"2024-05-27T07:28:37.900449+0000","last_active":"2024-05-27T07:33:35.243254+0000","last_peered":"2024-05-27T07:33:35.243254+0000","last_clean":"2024-05-27T07:33:35.243254+0000","last_became_active":"2024-05-27T07:28:37.899526+0000","last_became_peered":"2024-05-27T07:28:37.899526+0000","last_unstale":"2024-05-27T07:33:35.243254+0000","last_undegraded":"2024-05-27T07:33:35.243254+0000","last_fullsized":"2024-05-27T07:33:35.243254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:29:49.533462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437781+0000","last_change":"2024-05-27T07:28:20.405236+0000","last_active":"2024-05-27T07:33:35.437781+0000","last_peered":"2024-05-27T07:33:35.437781+0000","last_clean":"2024-05-27T07:33:35.437781+0000","last_became_active":"2024-05-27T07:28:20.404619+0000","last_became_peered":"2024-05-27T07:28:20.404619+0000","last_unstale":"2024-05-27T07:33:35.437781+0000","last_undegraded":"2024-05-27T07:33:35.437781+0000","last_fullsized":"2024-05-27T07:33:35.437781+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:17.671250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337323+0000","last_change":"2024-05-27T07:28:24.490661+0000","last_active":"2024-05-27T07:33:35.337323+0000","last_peered":"2024-05-27T07:33:35.337323+0000","last_clean":"2024-05-27T07:33:35.337323+0000","last_became_active":"2024-05-27T07:28:24.487299+0000","last_became_peered":"2024-05-27T07:28:24.487299+0000","last_unstale":"2024-05-27T07:33:35.337323+0000","last_undegraded":"2024-05-27T07:33:35.337323+0000","last_fullsized":"2024-05-27T07:33:35.337323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:50:27.291721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243449+0000","last_change":"2024-05-27T07:28:37.970463+0000","last_active":"2024-05-27T07:33:35.243449+0000","last_peered":"2024-05-27T07:33:35.243449+0000","last_clean":"2024-05-27T07:33:35.243449+0000","last_became_active":"2024-05-27T07:28:37.968068+0000","last_became_peered":"2024-05-27T07:28:37.968068+0000","last_unstale":"2024-05-27T07:33:35.243449+0000","last_undegraded":"2024-05-27T07:33:35.243449+0000","last_fullsized":"2024-05-27T07:33:35.243449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:22:00.825999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434404+0000","last_change":"2024-05-27T07:28:20.406203+0000","last_active":"2024-05-27T07:33:35.434404+0000","last_peered":"2024-05-27T07:33:35.434404+0000","last_clean":"2024-05-27T07:33:35.434404+0000","last_became_active":"2024-05-27T07:28:20.405404+0000","last_became_peered":"2024-05-27T07:28:20.405404+0000","last_unstale":"2024-05-27T07:33:35.434404+0000","last_undegraded":"2024-05-27T07:33:35.434404+0000","last_fullsized":"2024-05-27T07:33:35.434404+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:15:15.274419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434004+0000","last_change":"2024-05-27T07:28:24.473747+0000","last_active":"2024-05-27T07:33:35.434004+0000","last_peered":"2024-05-27T07:33:35.434004+0000","last_clean":"2024-05-27T07:33:35.434004+0000","last_became_active":"2024-05-27T07:28:24.473313+0000","last_became_peered":"2024-05-27T07:28:24.473313+0000","last_unstale":"2024-05-27T07:33:35.434004+0000","last_undegraded":"2024-05-27T07:33:35.434004+0000","last_fullsized":"2024-05-27T07:33:35.434004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:19:40.107840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.434078+0000","last_change":"2024-05-27T07:28:37.907410+0000","last_active":"2024-05-27T07:33:35.434078+0000","last_peered":"2024-05-27T07:33:35.434078+0000","last_clean":"2024-05-27T07:33:35.434078+0000","last_became_active":"2024-05-27T07:28:37.906753+0000","last_became_peered":"2024-05-27T07:28:37.906753+0000","last_unstale":"2024-05-27T07:33:35.434078+0000","last_undegraded":"2024-05-27T07:33:35.434078+0000","last_fullsized":"2024-05-27T07:33:35.434078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:53.612015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437366+0000","last_change":"2024-05-27T07:28:20.744068+0000","last_active":"2024-05-27T07:33:35.437366+0000","last_peered":"2024-05-27T07:33:35.437366+0000","last_clean":"2024-05-27T07:33:35.437366+0000","last_became_active":"2024-05-27T07:28:20.742686+0000","last_became_peered":"2024-05-27T07:28:20.742686+0000","last_unstale":"2024-05-27T07:33:35.437366+0000","last_undegraded":"2024-05-27T07:33:35.437366+0000","last_fullsized":"2024-05-27T07:33:35.437366+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:54.361007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436607+0000","last_change":"2024-05-27T07:28:24.475743+0000","last_active":"2024-05-27T07:33:35.436607+0000","last_peered":"2024-05-27T07:33:35.436607+0000","last_clean":"2024-05-27T07:33:35.436607+0000","last_became_active":"2024-05-27T07:28:24.474742+0000","last_became_peered":"2024-05-27T07:28:24.474742+0000","last_unstale":"2024-05-27T07:33:35.436607+0000","last_undegraded":"2024-05-27T07:33:35.436607+0000","last_fullsized":"2024-05-27T07:33:35.436607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:58:47.061573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431413+0000","last_change":"2024-05-27T07:28:37.914935+0000","last_active":"2024-05-27T07:33:35.431413+0000","last_peered":"2024-05-27T07:33:35.431413+0000","last_clean":"2024-05-27T07:33:35.431413+0000","last_became_active":"2024-05-27T07:28:37.911829+0000","last_became_peered":"2024-05-27T07:28:37.911829+0000","last_unstale":"2024-05-27T07:33:35.431413+0000","last_undegraded":"2024-05-27T07:33:35.431413+0000","last_fullsized":"2024-05-27T07:33:35.431413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:29.260539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"92'39","reported_seq":95,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.608079+0000","last_change":"2024-05-27T07:28:20.412554+0000","last_active":"2024-05-27T07:33:35.608079+0000","last_peered":"2024-05-27T07:33:35.608079+0000","last_clean":"2024-05-27T07:33:35.608079+0000","last_became_active":"2024-05-27T07:28:20.412024+0000","last_became_peered":"2024-05-27T07:28:20.412024+0000","last_unstale":"2024-05-27T07:33:35.608079+0000","last_undegraded":"2024-05-27T07:33:35.608079+0000","last_fullsized":"2024-05-27T07:33:35.608079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:28:29.244093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.435695+0000","last_change":"2024-05-27T07:28:24.481966+0000","last_active":"2024-05-27T07:33:35.435695+0000","last_peered":"2024-05-27T07:33:35.435695+0000","last_clean":"2024-05-27T07:33:35.435695+0000","last_became_active":"2024-05-27T07:28:24.480775+0000","last_became_peered":"2024-05-27T07:28:24.480775+0000","last_unstale":"2024-05-27T07:33:35.435695+0000","last_undegraded":"2024-05-27T07:33:35.435695+0000","last_fullsized":"2024-05-27T07:33:35.435695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:13:03.018174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.242513+0000","last_change":"2024-05-27T07:28:37.902893+0000","last_active":"2024-05-27T07:33:35.242513+0000","last_peered":"2024-05-27T07:33:35.242513+0000","last_clean":"2024-05-27T07:33:35.242513+0000","last_became_active":"2024-05-27T07:28:37.898447+0000","last_became_peered":"2024-05-27T07:28:37.898447+0000","last_unstale":"2024-05-27T07:33:35.242513+0000","last_undegraded":"2024-05-27T07:33:35.242513+0000","last_fullsized":"2024-05-27T07:33:35.242513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:36.837886+0000","last_clean_scrub_stamp":"2024-05-27T07:28:36.837886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:55:49.754803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370906+0000","last_change":"2024-05-27T07:28:20.408072+0000","last_active":"2024-05-27T07:33:35.370906+0000","last_peered":"2024-05-27T07:33:35.370906+0000","last_clean":"2024-05-27T07:33:35.370906+0000","last_became_active":"2024-05-27T07:28:20.407174+0000","last_became_peered":"2024-05-27T07:28:20.407174+0000","last_unstale":"2024-05-27T07:33:35.370906+0000","last_undegraded":"2024-05-27T07:33:35.370906+0000","last_fullsized":"2024-05-27T07:33:35.370906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:19.347131+0000","last_clean_scrub_stamp":"2024-05-27T07:28:19.347131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:15:52.984967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437053+0000","last_change":"2024-05-27T07:28:24.475891+0000","last_active":"2024-05-27T07:33:35.437053+0000","last_peered":"2024-05-27T07:33:35.437053+0000","last_clean":"2024-05-27T07:33:35.437053+0000","last_became_active":"2024-05-27T07:28:24.475498+0000","last_became_peered":"2024-05-27T07:28:24.475498+0000","last_unstale":"2024-05-27T07:33:35.437053+0000","last_undegraded":"2024-05-27T07:33:35.437053+0000","last_fullsized":"2024-05-27T07:33:35.437053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:50.669118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.384465+0000","last_change":"2024-05-27T07:29:04.891583+0000","last_active":"2024-05-27T07:33:35.384465+0000","last_peered":"2024-05-27T07:33:35.384465+0000","last_clean":"2024-05-27T07:33:35.384465+0000","last_became_active":"2024-05-27T07:29:04.890187+0000","last_became_peered":"2024-05-27T07:29:04.890187+0000","last_unstale":"2024-05-27T07:33:35.384465+0000","last_undegraded":"2024-05-27T07:33:35.384465+0000","last_fullsized":"2024-05-27T07:33:35.384465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:03.920565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436357+0000","last_change":"2024-05-27T07:28:24.478672+0000","last_active":"2024-05-27T07:33:35.436357+0000","last_peered":"2024-05-27T07:33:35.436357+0000","last_clean":"2024-05-27T07:33:35.436357+0000","last_became_active":"2024-05-27T07:28:24.478331+0000","last_became_peered":"2024-05-27T07:28:24.478331+0000","last_unstale":"2024-05-27T07:33:35.436357+0000","last_undegraded":"2024-05-27T07:33:35.436357+0000","last_fullsized":"2024-05-27T07:33:35.436357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:56:35.112855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437255+0000","last_change":"2024-05-27T07:28:24.487962+0000","last_active":"2024-05-27T07:33:35.437255+0000","last_peered":"2024-05-27T07:33:35.437255+0000","last_clean":"2024-05-27T07:33:35.437255+0000","last_became_active":"2024-05-27T07:28:24.487583+0000","last_became_peered":"2024-05-27T07:28:24.487583+0000","last_unstale":"2024-05-27T07:33:35.437255+0000","last_undegraded":"2024-05-27T07:33:35.437255+0000","last_fullsized":"2024-05-27T07:33:35.437255+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:44:25.758293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.431727+0000","last_change":"2024-05-27T07:28:24.466058+0000","last_active":"2024-05-27T07:33:35.431727+0000","last_peered":"2024-05-27T07:33:35.431727+0000","last_clean":"2024-05-27T07:33:35.431727+0000","last_became_active":"2024-05-27T07:28:24.465709+0000","last_became_peered":"2024-05-27T07:28:24.465709+0000","last_unstale":"2024-05-27T07:33:35.431727+0000","last_undegraded":"2024-05-27T07:33:35.431727+0000","last_fullsized":"2024-05-27T07:33:35.431727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:38.064077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.369986+0000","last_change":"2024-05-27T07:28:24.471585+0000","last_active":"2024-05-27T07:33:35.369986+0000","last_peered":"2024-05-27T07:33:35.369986+0000","last_clean":"2024-05-27T07:33:35.369986+0000","last_became_active":"2024-05-27T07:28:24.470948+0000","last_became_peered":"2024-05-27T07:28:24.470948+0000","last_unstale":"2024-05-27T07:33:35.369986+0000","last_undegraded":"2024-05-27T07:33:35.369986+0000","last_fullsized":"2024-05-27T07:33:35.369986+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:48.213323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385238+0000","last_change":"2024-05-27T07:28:24.477954+0000","last_active":"2024-05-27T07:33:35.385238+0000","last_peered":"2024-05-27T07:33:35.385238+0000","last_clean":"2024-05-27T07:33:35.385238+0000","last_became_active":"2024-05-27T07:28:24.477495+0000","last_became_peered":"2024-05-27T07:28:24.477495+0000","last_unstale":"2024-05-27T07:33:35.385238+0000","last_undegraded":"2024-05-27T07:33:35.385238+0000","last_fullsized":"2024-05-27T07:33:35.385238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:50.371439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.437604+0000","last_change":"2024-05-27T07:28:24.476422+0000","last_active":"2024-05-27T07:33:35.437604+0000","last_peered":"2024-05-27T07:33:35.437604+0000","last_clean":"2024-05-27T07:33:35.437604+0000","last_became_active":"2024-05-27T07:28:24.476095+0000","last_became_peered":"2024-05-27T07:28:24.476095+0000","last_unstale":"2024-05-27T07:33:35.437604+0000","last_undegraded":"2024-05-27T07:33:35.437604+0000","last_fullsized":"2024-05-27T07:33:35.437604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:51.667169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337797+0000","last_change":"2024-05-27T07:28:24.486431+0000","last_active":"2024-05-27T07:33:35.337797+0000","last_peered":"2024-05-27T07:33:35.337797+0000","last_clean":"2024-05-27T07:33:35.337797+0000","last_became_active":"2024-05-27T07:28:24.482427+0000","last_became_peered":"2024-05-27T07:28:24.482427+0000","last_unstale":"2024-05-27T07:33:35.337797+0000","last_undegraded":"2024-05-27T07:33:35.337797+0000","last_fullsized":"2024-05-27T07:33:35.337797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:44:37.857793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.243086+0000","last_change":"2024-05-27T07:28:24.484393+0000","last_active":"2024-05-27T07:33:35.243086+0000","last_peered":"2024-05-27T07:33:35.243086+0000","last_clean":"2024-05-27T07:33:35.243086+0000","last_became_active":"2024-05-27T07:28:24.483130+0000","last_became_peered":"2024-05-27T07:28:24.483130+0000","last_unstale":"2024-05-27T07:33:35.243086+0000","last_undegraded":"2024-05-27T07:33:35.243086+0000","last_fullsized":"2024-05-27T07:33:35.243086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:17:34.924573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.337254+0000","last_change":"2024-05-27T07:28:24.486274+0000","last_active":"2024-05-27T07:33:35.337254+0000","last_peered":"2024-05-27T07:33:35.337254+0000","last_clean":"2024-05-27T07:33:35.337254+0000","last_became_active":"2024-05-27T07:28:24.485880+0000","last_became_peered":"2024-05-27T07:28:24.485880+0000","last_unstale":"2024-05-27T07:33:35.337254+0000","last_undegraded":"2024-05-27T07:33:35.337254+0000","last_fullsized":"2024-05-27T07:33:35.337254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:33.287615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.436357+0000","last_change":"2024-05-27T07:28:24.477909+0000","last_active":"2024-05-27T07:33:35.436357+0000","last_peered":"2024-05-27T07:33:35.436357+0000","last_clean":"2024-05-27T07:33:35.436357+0000","last_became_active":"2024-05-27T07:28:24.477345+0000","last_became_peered":"2024-05-27T07:28:24.477345+0000","last_unstale":"2024-05-27T07:33:35.436357+0000","last_undegraded":"2024-05-27T07:33:35.436357+0000","last_fullsized":"2024-05-27T07:33:35.436357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:26:03.958489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.370232+0000","last_change":"2024-05-27T07:28:24.477151+0000","last_active":"2024-05-27T07:33:35.370232+0000","last_peered":"2024-05-27T07:33:35.370232+0000","last_clean":"2024-05-27T07:33:35.370232+0000","last_became_active":"2024-05-27T07:28:24.474036+0000","last_became_peered":"2024-05-27T07:28:24.474036+0000","last_unstale":"2024-05-27T07:33:35.370232+0000","last_undegraded":"2024-05-27T07:33:35.370232+0000","last_fullsized":"2024-05-27T07:33:35.370232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:16.787252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:33:35.385340+0000","last_change":"2024-05-27T07:28:24.483601+0000","last_active":"2024-05-27T07:33:35.385340+0000","last_peered":"2024-05-27T07:33:35.385340+0000","last_clean":"2024-05-27T07:33:35.385340+0000","last_became_active":"2024-05-27T07:28:24.481080+0000","last_became_peered":"2024-05-27T07:28:24.481080+0000","last_unstale":"2024-05-27T07:33:35.385340+0000","last_undegraded":"2024-05-27T07:33:35.385340+0000","last_fullsized":"2024-05-27T07:33:35.385340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:28:23.404589+0000","last_clean_scrub_stamp":"2024-05-27T07:28:23.404589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:12.208506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":607900,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1496,"num_read_kb":866,"num_write":304,"num_write_kb":1455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2052096,"data_stored":1823700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645393,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712552,"statfs":{"total":95995035648,"available":95961653248,"internally_reserved":0,"allocated":1454080,"data_stored":925589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841624,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28872,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716280,"statfs":{"total":95995035648,"available":95965470720,"internally_reserved":0,"allocated":1900544,"data_stored":1327702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005152,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1474560,"data_stored":929921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201385,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":532529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397616,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32808,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712344,"statfs":{"total":95995035648,"available":95961440256,"internally_reserved":0,"allocated":1671168,"data_stored":1130206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561142,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":527785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1138688,"data_stored":602490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738530,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":529173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1163264,"data_stored":604333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149837,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1597440,"data_stored":1052795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953606,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":534101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757375,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37108,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708044,"statfs":{"total":95995035648,"available":95957037056,"internally_reserved":0,"allocated":1814528,"data_stored":1255728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":77315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":212159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:33:45.239 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:33:45.350 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:33:45.350 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:33:45.350 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:33:45.350 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:33:45.350 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.350 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.350 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.350 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.350 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.350 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.351 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.351 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:45.351 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:45.351 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:45.351 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:33:45.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:33:45.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:33:45.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:33:45.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:45.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:45.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:45.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: Updating smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:33:45.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: Updating smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:33:45.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: Updating smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:45.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:45.730 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:46.165 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:46 smithi063 ceph-mon[31613]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:33:46.166 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:46 smithi063 ceph-mon[31613]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:33:46.166 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:46 smithi063 ceph-mon[31613]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 684 B/s wr, 12 op/s 2024-05-27T07:33:46.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:46 smithi086 ceph-mon[34687]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:33:46.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:46 smithi086 ceph-mon[34687]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:33:46.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:46 smithi086 ceph-mon[34687]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 684 B/s wr, 12 op/s 2024-05-27T07:33:46.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:46 smithi149 ceph-mon[35408]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:33:46.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:46 smithi149 ceph-mon[35408]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:33:46.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:46 smithi149 ceph-mon[35408]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 684 B/s wr, 12 op/s 2024-05-27T07:33:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.266+0000 7fdda1a00700 1 -- 172.21.15.63:0/2997711616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd9c110980 msgr2=0x7fdd9c110d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.266+0000 7fdda1a00700 1 --2- 172.21.15.63:0/2997711616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd9c110980 0x7fdd9c110d90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009990 tx=0x7fdd8c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.266+0000 7fdda1a00700 1 -- 172.21.15.63:0/2997711616 shutdown_connections 2024-05-27T07:33:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.266+0000 7fdda1a00700 1 --2- 172.21.15.63:0/2997711616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c10b2a0 0x7fdd9c117820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.267+0000 7fdda1a00700 1 --2- 172.21.15.63:0/2997711616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd9c110980 0x7fdd9c110d90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.267+0000 7fdda1a00700 1 --2- 172.21.15.63:0/2997711616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd9c10a950 0x7fdd9c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.267+0000 7fdda1a00700 1 -- 172.21.15.63:0/2997711616 >> 172.21.15.63:0/2997711616 conn(0x7fdd9c076d20 msgr2=0x7fdd9c077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.267+0000 7fdda1a00700 1 -- 172.21.15.63:0/2997711616 shutdown_connections 2024-05-27T07:33:48.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.267+0000 7fdda1a00700 1 -- 172.21.15.63:0/2997711616 wait complete. 2024-05-27T07:33:48.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.269+0000 7fdda1a00700 1 Processor -- start 2024-05-27T07:33:48.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.269+0000 7fdda1a00700 1 -- start start 2024-05-27T07:33:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdda1a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd9c10a950 0x7fdd9c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdda1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c10b2a0 0x7fdd9c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdda1a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd9c110980 0x7fdd9c1a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdda1a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c11a930 con 0x7fdd9c10a950 2024-05-27T07:33:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdda1a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c11a7b0 con 0x7fdd9c10b2a0 2024-05-27T07:33:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdda1a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c11aab0 con 0x7fdd9c110980 2024-05-27T07:33:48.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdd9aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd9c10a950 0x7fdd9c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:48.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdd9aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd9c10a950 0x7fdd9c1a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42412/0 (socket says 172.21.15.63:42412) 2024-05-27T07:33:48.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdd9aa00700 1 -- 172.21.15.63:0/1323682603 learned_addr learned my addr 172.21.15.63:0/1323682603 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:48.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdd9b400700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd9c110980 0x7fdd9c1a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:48.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.270+0000 7fdd9a000700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c10b2a0 0x7fdd9c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:48.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.271+0000 7fdd9aa00700 1 -- 172.21.15.63:0/1323682603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd9c110980 msgr2=0x7fdd9c1a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:48.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.271+0000 7fdd9aa00700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd9c110980 0x7fdd9c1a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.271+0000 7fdd9aa00700 1 -- 172.21.15.63:0/1323682603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c10b2a0 msgr2=0x7fdd9c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:48.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.271+0000 7fdd9aa00700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c10b2a0 0x7fdd9c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.271+0000 7fdd9aa00700 1 -- 172.21.15.63:0/1323682603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c1a61e0 con 0x7fdd9c10a950 2024-05-27T07:33:48.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.271+0000 7fdd9aa00700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd9c10a950 0x7fdd9c1a13c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c000c00 tx=0x7fdd8c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:48.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.271+0000 7fdd93400700 1 -- 172.21.15.63:0/1323682603 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c027070 con 0x7fdd9c10a950 2024-05-27T07:33:48.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.272+0000 7fdd93400700 1 -- 172.21.15.63:0/1323682603 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd8c00fb80 con 0x7fdd9c10a950 2024-05-27T07:33:48.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.272+0000 7fdda1a00700 1 -- 172.21.15.63:0/1323682603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9c1a6470 con 0x7fdd9c10a950 2024-05-27T07:33:48.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.272+0000 7fdd93400700 1 -- 172.21.15.63:0/1323682603 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c022d00 con 0x7fdd9c10a950 2024-05-27T07:33:48.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.272+0000 7fdda1a00700 1 -- 172.21.15.63:0/1323682603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9c1a6950 con 0x7fdd9c10a950 2024-05-27T07:33:48.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.273+0000 7fdda1a00700 1 -- 172.21.15.63:0/1323682603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9c0512b0 con 0x7fdd9c10a950 2024-05-27T07:33:48.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.275+0000 7fdd93400700 1 -- 172.21.15.63:0/1323682603 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fdd8c02a070 con 0x7fdd9c10a950 2024-05-27T07:33:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.277+0000 7fdd93400700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdd800b2b10 0x7fdd800b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.277+0000 7fdd93400700 1 -- 172.21.15.63:0/1323682603 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fdd8c0e3b10 con 0x7fdd9c10a950 2024-05-27T07:33:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.277+0000 7fdd9a000700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdd800b2b10 0x7fdd800b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.278+0000 7fdd9a000700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdd800b2b10 0x7fdd800b5020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdd88005950 tx=0x7fdd880058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.283+0000 7fdd93400700 1 -- 172.21.15.63:0/1323682603 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd8c013070 con 0x7fdd9c10a950 2024-05-27T07:33:48.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.570+0000 7fdda1a00700 1 -- 172.21.15.63:0/1323682603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd9c02e5e0 con 0x7fdd9c10a950 2024-05-27T07:33:48.565 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:48 smithi063 ceph-mon[31613]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 559 B/s wr, 13 op/s 2024-05-27T07:33:48.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.573+0000 7fdd93400700 1 -- 172.21.15.63:0/1323682603 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11861 (secure 0 0 0) 0x7fdd8c0aaf60 con 0x7fdd9c10a950 2024-05-27T07:33:48.567 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:33:48.567 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:33:33.871810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:33:48.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.576+0000 7fdda1a00700 1 -- 172.21.15.63:0/1323682603 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdd800b2b10 msgr2=0x7fdd800b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:48.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.576+0000 7fdda1a00700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdd800b2b10 0x7fdd800b5020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdd88005950 tx=0x7fdd880058e0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.576+0000 7fdda1a00700 1 -- 172.21.15.63:0/1323682603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd9c10a950 msgr2=0x7fdd9c1a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:48.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.576+0000 7fdda1a00700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd9c10a950 0x7fdd9c1a13c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c000c00 tx=0x7fdd8c0047b0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.576+0000 7fdda1a00700 1 -- 172.21.15.63:0/1323682603 shutdown_connections 2024-05-27T07:33:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.576+0000 7fdda1a00700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdd800b2b10 0x7fdd800b5020 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.576+0000 7fdda1a00700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd9c110980 0x7fdd9c1a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.576+0000 7fdda1a00700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd9c10a950 0x7fdd9c1a13c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.576+0000 7fdda1a00700 1 --2- 172.21.15.63:0/1323682603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9c10b2a0 0x7fdd9c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.576+0000 7fdda1a00700 1 -- 172.21.15.63:0/1323682603 >> 172.21.15.63:0/1323682603 conn(0x7fdd9c076d20 msgr2=0x7fdd9c109380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.577+0000 7fdda1a00700 1 -- 172.21.15.63:0/1323682603 shutdown_connections 2024-05-27T07:33:48.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:48.577+0000 7fdda1a00700 1 -- 172.21.15.63:0/1323682603 wait complete. 2024-05-27T07:33:48.572 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 27 2024-05-27T07:33:48.702 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:48 smithi086 ceph-mon[34687]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 559 B/s wr, 13 op/s 2024-05-27T07:33:48.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:48 smithi149 ceph-mon[35408]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 559 B/s wr, 13 op/s 2024-05-27T07:33:49.490 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:33:49.609 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:49 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1323682603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:49.702 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:49 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1323682603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:49.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:49 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1323682603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:33:49.976 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:50.399 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:50 smithi063 ceph-mon[31613]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T07:33:50.648 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:50 smithi149 ceph-mon[35408]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T07:33:50.702 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:50 smithi086 ceph-mon[34687]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T07:33:52.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.457+0000 7ff9ef200700 1 -- 172.21.15.63:0/984813519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0105bd0 msgr2=0x7ff9f010e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:52.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.457+0000 7ff9ef200700 1 --2- 172.21.15.63:0/984813519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0105bd0 0x7ff9f010e120 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009990 tx=0x7ff9dc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.457+0000 7ff9ef200700 1 -- 172.21.15.63:0/984813519 shutdown_connections 2024-05-27T07:33:52.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.457+0000 7ff9ef200700 1 --2- 172.21.15.63:0/984813519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f010e6f0 0x7ff9f0110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.457+0000 7ff9ef200700 1 --2- 172.21.15.63:0/984813519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f0105280 0x7ff9f0105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.457+0000 7ff9ef200700 1 --2- 172.21.15.63:0/984813519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0105bd0 0x7ff9f010e120 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.457+0000 7ff9ef200700 1 -- 172.21.15.63:0/984813519 >> 172.21.15.63:0/984813519 conn(0x7ff9f0078b40 msgr2=0x7ff9f0078f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:52.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.457+0000 7ff9ef200700 1 -- 172.21.15.63:0/984813519 shutdown_connections 2024-05-27T07:33:52.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.458+0000 7ff9ef200700 1 -- 172.21.15.63:0/984813519 wait complete. 2024-05-27T07:33:52.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.459+0000 7ff9ef200700 1 Processor -- start 2024-05-27T07:33:52.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.459+0000 7ff9ef200700 1 -- start start 2024-05-27T07:33:52.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.459+0000 7ff9ef200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f0105280 0x7ff9f019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:52.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.459+0000 7ff9ef200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0105bd0 0x7ff9f019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:52.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.459+0000 7ff9ef200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f010e6f0 0x7ff9f01a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:52.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.459+0000 7ff9ef200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01141c0 con 0x7ff9f0105bd0 2024-05-27T07:33:52.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.459+0000 7ff9ef200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0114040 con 0x7ff9f0105280 2024-05-27T07:33:52.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.459+0000 7ff9ef200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0114340 con 0x7ff9f010e6f0 2024-05-27T07:33:52.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.460+0000 7ff9ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0105bd0 0x7ff9f019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:52.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.460+0000 7ff9ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0105bd0 0x7ff9f019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36944/0 (socket says 172.21.15.63:36944) 2024-05-27T07:33:52.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.460+0000 7ff9ed400700 1 -- 172.21.15.63:0/1749788657 learned_addr learned my addr 172.21.15.63:0/1749788657 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:52.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.460+0000 7ff9ede00700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f0105280 0x7ff9f019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:52.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.460+0000 7ff9ee800700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f010e6f0 0x7ff9f01a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:52.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.460+0000 7ff9ed400700 1 -- 172.21.15.63:0/1749788657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f010e6f0 msgr2=0x7ff9f01a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:52.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.460+0000 7ff9ed400700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f010e6f0 0x7ff9f01a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.460+0000 7ff9ed400700 1 -- 172.21.15.63:0/1749788657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f0105280 msgr2=0x7ff9f019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:52.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.460+0000 7ff9ed400700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f0105280 0x7ff9f019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.460+0000 7ff9ed400700 1 -- 172.21.15.63:0/1749788657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f01a1bd0 con 0x7ff9f0105bd0 2024-05-27T07:33:52.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.461+0000 7ff9ed400700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0105bd0 0x7ff9f019d350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc000c00 tx=0x7ff9dc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:52.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.461+0000 7ff9e6a00700 1 -- 172.21.15.63:0/1749788657 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc027070 con 0x7ff9f0105bd0 2024-05-27T07:33:52.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.461+0000 7ff9e6a00700 1 -- 172.21.15.63:0/1749788657 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9dc00fc80 con 0x7ff9f0105bd0 2024-05-27T07:33:52.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.461+0000 7ff9e6a00700 1 -- 172.21.15.63:0/1749788657 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc022d60 con 0x7ff9f0105bd0 2024-05-27T07:33:52.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.461+0000 7ff9ef200700 1 -- 172.21.15.63:0/1749788657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f01a1e60 con 0x7ff9f0105bd0 2024-05-27T07:33:52.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.461+0000 7ff9ef200700 1 -- 172.21.15.63:0/1749788657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f01a22f0 con 0x7ff9f0105bd0 2024-05-27T07:33:52.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.463+0000 7ff9ef200700 1 -- 172.21.15.63:0/1749788657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f00512b0 con 0x7ff9f0105bd0 2024-05-27T07:33:52.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.465+0000 7ff9e6a00700 1 -- 172.21.15.63:0/1749788657 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7ff9dc02a070 con 0x7ff9f0105bd0 2024-05-27T07:33:52.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.467+0000 7ff9e6a00700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2b90 0x7ff9d40b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:52.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.467+0000 7ff9e6a00700 1 -- 172.21.15.63:0/1749788657 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7ff9dc0e3ba0 con 0x7ff9f0105bd0 2024-05-27T07:33:52.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.467+0000 7ff9ede00700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2b90 0x7ff9d40b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:52.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.468+0000 7ff9ede00700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2b90 0x7ff9d40b50a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0006fd0 tx=0x7ff9e0008040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:52.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.472+0000 7ff9e6a00700 1 -- 172.21.15.63:0/1749788657 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9dc05fdf0 con 0x7ff9f0105bd0 2024-05-27T07:33:52.702 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:52 smithi086 ceph-mon[34687]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T07:33:52.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:52 smithi149 ceph-mon[35408]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T07:33:52.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:52 smithi063 ceph-mon[31613]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-27T07:33:52.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.718+0000 7ff9ef200700 1 -- 172.21.15.63:0/1749788657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9f01a9890 con 0x7ff9f0105bd0 2024-05-27T07:33:52.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.722+0000 7ff9e6a00700 1 -- 172.21.15.63:0/1749788657 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+25011 (secure 0 0 0) 0x7ff9dc013070 con 0x7ff9f0105bd0 2024-05-27T07:33:52.717 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:33:52.717 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":92,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T07:33:35.220889+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6},{"from":3,"to":7},{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/1961493183":"2024-05-28T07:33:35.220779+0000","172.21.15.63:6801/675305989":"2024-05-28T07:33:35.220779+0000","172.21.15.63:0/1202958531":"2024-05-28T07:33:35.220779+0000","172.21.15.63:6800/675305989":"2024-05-28T07:33:35.220779+0000","172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3428263920":"2024-05-28T07:33:35.220779+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1558308201":"2024-05-28T07:33:35.220779+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:33:52.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.726+0000 7ff9ef200700 1 -- 172.21.15.63:0/1749788657 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2b90 msgr2=0x7ff9d40b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:52.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.726+0000 7ff9ef200700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2b90 0x7ff9d40b50a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0006fd0 tx=0x7ff9e0008040 comp rx=0 tx=0).stop 2024-05-27T07:33:52.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.726+0000 7ff9ef200700 1 -- 172.21.15.63:0/1749788657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0105bd0 msgr2=0x7ff9f019d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:52.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.726+0000 7ff9ef200700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0105bd0 0x7ff9f019d350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc000c00 tx=0x7ff9dc0036f0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.726+0000 7ff9ef200700 1 -- 172.21.15.63:0/1749788657 shutdown_connections 2024-05-27T07:33:52.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.726+0000 7ff9ef200700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2b90 0x7ff9d40b50a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.726+0000 7ff9ef200700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f010e6f0 0x7ff9f01a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.726+0000 7ff9ef200700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f0105280 0x7ff9f019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.726+0000 7ff9ef200700 1 --2- 172.21.15.63:0/1749788657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0105bd0 0x7ff9f019d350 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:52.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.726+0000 7ff9ef200700 1 -- 172.21.15.63:0/1749788657 >> 172.21.15.63:0/1749788657 conn(0x7ff9f0078b40 msgr2=0x7ff9f0102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:52.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.727+0000 7ff9ef200700 1 -- 172.21.15.63:0/1749788657 shutdown_connections 2024-05-27T07:33:52.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:52.727+0000 7ff9ef200700 1 -- 172.21.15.63:0/1749788657 wait complete. 2024-05-27T07:33:53.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:53 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1749788657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:33:53.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:53 smithi063 ceph-mon[31613]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 511 B/s wr, 14 op/s 2024-05-27T07:33:53.610 DEBUG:tasks.fwd_scrub:Yielding 2024-05-27T07:33:53.610 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T07:33:53.625 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:33:53.626 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T07:33:53.626 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-27T07:33:53.627 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-27T07:33:53.627 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:33:53.627 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:33:53.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:53 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1749788657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:33:53.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:53 smithi086 ceph-mon[34687]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 511 B/s wr, 14 op/s 2024-05-27T07:33:53.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:53 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1749788657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:33:53.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:53 smithi149 ceph-mon[35408]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 511 B/s wr, 14 op/s 2024-05-27T07:33:53.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.910+0000 7f2647400640 1 -- 172.21.15.63:0/2467482705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2640104da0 msgr2=0x7f26401051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:53.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.910+0000 7f2647400640 1 --2- 172.21.15.63:0/2467482705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2640104da0 0x7f26401051b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2634009900 tx=0x7f263402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:33:53.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.910+0000 7f2647400640 1 -- 172.21.15.63:0/2467482705 shutdown_connections 2024-05-27T07:33:53.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.910+0000 7f2647400640 1 --2- 172.21.15.63:0/2467482705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f264010e1f0 0x7f26401106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:53.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.910+0000 7f2647400640 1 --2- 172.21.15.63:0/2467482705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264010b800 0x7f264010dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:53.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.910+0000 7f2647400640 1 --2- 172.21.15.63:0/2467482705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2640104da0 0x7f26401051b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:53.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.910+0000 7f2647400640 1 -- 172.21.15.63:0/2467482705 >> 172.21.15.63:0/2467482705 conn(0x7f2640078270 msgr2=0x7f26400786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:53.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.910+0000 7f2647400640 1 -- 172.21.15.63:0/2467482705 shutdown_connections 2024-05-27T07:33:53.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.910+0000 7f2647400640 1 -- 172.21.15.63:0/2467482705 wait complete. 2024-05-27T07:33:53.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.912+0000 7f2647400640 1 Processor -- start 2024-05-27T07:33:53.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.912+0000 7f2647400640 1 -- start start 2024-05-27T07:33:53.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.913+0000 7f2647400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640104da0 0x7f2640102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:53.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.913+0000 7f2647400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264010b800 0x7f2640102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:53.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.913+0000 7f2647400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264010e1f0 0x7f26401030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:53.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.913+0000 7f2647400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2640113d50 con 0x7f264010e1f0 2024-05-27T07:33:53.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.913+0000 7f2647400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2640113bd0 con 0x7f264010b800 2024-05-27T07:33:53.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.913+0000 7f2647400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2640113ed0 con 0x7f2640104da0 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.913+0000 7f2645400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264010e1f0 0x7f26401030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.913+0000 7f2645400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264010e1f0 0x7f26401030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36952/0 (socket says 172.21.15.63:36952) 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.913+0000 7f2645400640 1 -- 172.21.15.63:0/643763716 learned_addr learned my addr 172.21.15.63:0/643763716 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.913+0000 7f2644a00640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640104da0 0x7f2640102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.914+0000 7f263fe00640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264010b800 0x7f2640102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.914+0000 7f2645400640 1 -- 172.21.15.63:0/643763716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640104da0 msgr2=0x7f2640102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.914+0000 7f2645400640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640104da0 0x7f2640102660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.914+0000 7f2645400640 1 -- 172.21.15.63:0/643763716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264010b800 msgr2=0x7f2640102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.914+0000 7f2645400640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264010b800 0x7f2640102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.914+0000 7f2645400640 1 -- 172.21.15.63:0/643763716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26401a8b30 con 0x7f264010e1f0 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.914+0000 7f2644a00640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640104da0 0x7f2640102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:53.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.914+0000 7f2645400640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264010e1f0 0x7f26401030e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2638009fd0 tx=0x7f263800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:53.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.914+0000 7f263d600640 1 -- 172.21.15.63:0/643763716 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638017070 con 0x7f264010e1f0 2024-05-27T07:33:53.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.915+0000 7f2647400640 1 -- 172.21.15.63:0/643763716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26401a8e20 con 0x7f264010e1f0 2024-05-27T07:33:53.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.915+0000 7f263d600640 1 -- 172.21.15.63:0/643763716 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2638003e00 con 0x7f264010e1f0 2024-05-27T07:33:53.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.915+0000 7f263d600640 1 -- 172.21.15.63:0/643763716 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638010460 con 0x7f264010e1f0 2024-05-27T07:33:53.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.915+0000 7f2647400640 1 -- 172.21.15.63:0/643763716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26401a9280 con 0x7f264010e1f0 2024-05-27T07:33:53.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.916+0000 7f2647400640 1 -- 172.21.15.63:0/643763716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2640050860 con 0x7f264010e1f0 2024-05-27T07:33:53.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.920+0000 7f263d600640 1 -- 172.21.15.63:0/643763716 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f2638003970 con 0x7f264010e1f0 2024-05-27T07:33:53.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.923+0000 7f263d600640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26280b2a70 0x7f26280b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:53.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.923+0000 7f263d600640 1 -- 172.21.15.63:0/643763716 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f26380e2e70 con 0x7f264010e1f0 2024-05-27T07:33:53.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.923+0000 7f2644a00640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26280b2a70 0x7f26280b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:53.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.924+0000 7f263d600640 1 -- 172.21.15.63:0/643763716 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26380aa230 con 0x7f264010e1f0 2024-05-27T07:33:53.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:53.925+0000 7f2644a00640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26280b2a70 0x7f26280b4f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2634006010 tx=0x7f2634005a20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:54.093 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:54.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.166+0000 7f2647400640 1 -- 172.21.15.63:0/643763716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f26401a9650 con 0x7f264010e1f0 2024-05-27T07:33:54.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.172+0000 7f263d600640 1 -- 172.21.15.63:0/643763716 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v31)=0 v31) ==== 129+0+0 (secure 0 0 0) 0x7f26380a9b30 con 0x7f264010e1f0 2024-05-27T07:33:54.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 -- 172.21.15.63:0/643763716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26280b2a70 msgr2=0x7f26280b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:54.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26280b2a70 0x7f26280b4f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2634006010 tx=0x7f2634005a20 comp rx=0 tx=0).stop 2024-05-27T07:33:54.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 -- 172.21.15.63:0/643763716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264010e1f0 msgr2=0x7f26401030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:54.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264010e1f0 0x7f26401030e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2638009fd0 tx=0x7f263800b260 comp rx=0 tx=0).stop 2024-05-27T07:33:54.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 -- 172.21.15.63:0/643763716 shutdown_connections 2024-05-27T07:33:54.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26280b2a70 0x7f26280b4f80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264010e1f0 0x7f26401030e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264010b800 0x7f2640102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 --2- 172.21.15.63:0/643763716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2640104da0 0x7f2640102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 -- 172.21.15.63:0/643763716 >> 172.21.15.63:0/643763716 conn(0x7f2640078270 msgr2=0x7f2640101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:54.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 -- 172.21.15.63:0/643763716 shutdown_connections 2024-05-27T07:33:54.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.176+0000 7f2647400640 1 -- 172.21.15.63:0/643763716 wait complete. 2024-05-27T07:33:54.198 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-27T07:33:54.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.477+0000 7f014fc00640 1 -- 172.21.15.63:0/3662643136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0148104d90 msgr2=0x7f01481051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:54.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.477+0000 7f014fc00640 1 --2- 172.21.15.63:0/3662643136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0148104d90 0x7f01481051a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f013c009900 tx=0x7f013c02a410 comp rx=0 tx=0).stop 2024-05-27T07:33:54.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.477+0000 7f014fc00640 1 -- 172.21.15.63:0/3662643136 shutdown_connections 2024-05-27T07:33:54.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.477+0000 7f014fc00640 1 --2- 172.21.15.63:0/3662643136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f014810e1e0 0x7f01481106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.477+0000 7f014fc00640 1 --2- 172.21.15.63:0/3662643136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f014810b7f0 0x7f014810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.477+0000 7f014fc00640 1 --2- 172.21.15.63:0/3662643136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0148104d90 0x7f01481051a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.477+0000 7f014fc00640 1 -- 172.21.15.63:0/3662643136 >> 172.21.15.63:0/3662643136 conn(0x7f01480782d0 msgr2=0x7f0148078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:54.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.477+0000 7f014fc00640 1 -- 172.21.15.63:0/3662643136 shutdown_connections 2024-05-27T07:33:54.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.478+0000 7f014fc00640 1 -- 172.21.15.63:0/3662643136 wait complete. 2024-05-27T07:33:54.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.480+0000 7f014fc00640 1 Processor -- start 2024-05-27T07:33:54.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.480+0000 7f014fc00640 1 -- start start 2024-05-27T07:33:54.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.480+0000 7f014fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0148104d90 0x7f014819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:54.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.480+0000 7f014fc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f014810b7f0 0x7f014819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:54.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.480+0000 7f014fc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f014810e1e0 0x7f014819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:54.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.480+0000 7f014fc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0148113cf0 con 0x7f014810b7f0 2024-05-27T07:33:54.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.480+0000 7f014fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0148113b70 con 0x7f0148104d90 2024-05-27T07:33:54.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.480+0000 7f014fc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0148113e70 con 0x7f014810e1e0 2024-05-27T07:33:54.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.481+0000 7f0147e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f014810b7f0 0x7f014819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:54.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.481+0000 7f0147e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f014810b7f0 0x7f014819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36970/0 (socket says 172.21.15.63:36970) 2024-05-27T07:33:54.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.481+0000 7f0147e00640 1 -- 172.21.15.63:0/3584339630 learned_addr learned my addr 172.21.15.63:0/3584339630 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:54.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.481+0000 7f014d200640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0148104d90 0x7f014819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:54.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.481+0000 7f014dc00640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f014810e1e0 0x7f014819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:54.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.481+0000 7f0147e00640 1 -- 172.21.15.63:0/3584339630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f014810e1e0 msgr2=0x7f014819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:54.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.481+0000 7f0147e00640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f014810e1e0 0x7f014819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.481+0000 7f0147e00640 1 -- 172.21.15.63:0/3584339630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0148104d90 msgr2=0x7f014819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:54.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.481+0000 7f0147e00640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0148104d90 0x7f014819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.481+0000 7f0147e00640 1 -- 172.21.15.63:0/3584339630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014819f5b0 con 0x7f014810b7f0 2024-05-27T07:33:54.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.482+0000 7f0147e00640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f014810b7f0 0x7f014819acb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f013400e9f0 tx=0x7f013400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:54.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.482+0000 7f0145600640 1 -- 172.21.15.63:0/3584339630 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013400cd80 con 0x7f014810b7f0 2024-05-27T07:33:54.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.482+0000 7f0145600640 1 -- 172.21.15.63:0/3584339630 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01340108c0 con 0x7f014810b7f0 2024-05-27T07:33:54.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.482+0000 7f0145600640 1 -- 172.21.15.63:0/3584339630 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01340187c0 con 0x7f014810b7f0 2024-05-27T07:33:54.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.482+0000 7f014fc00640 1 -- 172.21.15.63:0/3584339630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014819f8a0 con 0x7f014810b7f0 2024-05-27T07:33:54.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.482+0000 7f014fc00640 1 -- 172.21.15.63:0/3584339630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014819fda0 con 0x7f014810b7f0 2024-05-27T07:33:54.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.484+0000 7f014fc00640 1 -- 172.21.15.63:0/3584339630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014806a230 con 0x7f014810b7f0 2024-05-27T07:33:54.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.488+0000 7f0145600640 1 -- 172.21.15.63:0/3584339630 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f0134010430 con 0x7f014810b7f0 2024-05-27T07:33:54.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.489+0000 7f0145600640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01300b2af0 0x7f01300b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:54.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.490+0000 7f0145600640 1 -- 172.21.15.63:0/3584339630 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f01340e3450 con 0x7f014810b7f0 2024-05-27T07:33:54.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.490+0000 7f014d200640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01300b2af0 0x7f01300b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:54.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.491+0000 7f014d200640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01300b2af0 0x7f01300b5000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f013c0057c0 tx=0x7f013c0058d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:54.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.492+0000 7f0145600640 1 -- 172.21.15.63:0/3584339630 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f014819fda0 con 0x7f014810b7f0 2024-05-27T07:33:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.710+0000 7f014fc00640 1 -- 172.21.15.63:0/3584339630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f01481a7370 con 0x7f014810b7f0 2024-05-27T07:33:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.721+0000 7f0145600640 1 -- 172.21.15.63:0/3584339630 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f01340aa870 con 0x7f014810b7f0 2024-05-27T07:33:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.723+0000 7f014fc00640 1 -- 172.21.15.63:0/3584339630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01300b2af0 msgr2=0x7f01300b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.723+0000 7f014fc00640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01300b2af0 0x7f01300b5000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f013c0057c0 tx=0x7f013c0058d0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.723+0000 7f014fc00640 1 -- 172.21.15.63:0/3584339630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f014810b7f0 msgr2=0x7f014819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:54.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.723+0000 7f014fc00640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f014810b7f0 0x7f014819acb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f013400e9f0 tx=0x7f013400eeb0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.724+0000 7f014fc00640 1 -- 172.21.15.63:0/3584339630 shutdown_connections 2024-05-27T07:33:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.724+0000 7f014fc00640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01300b2af0 0x7f01300b5000 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.724+0000 7f014fc00640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f014810e1e0 0x7f014819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.724+0000 7f014fc00640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f014810b7f0 0x7f014819acb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.724+0000 7f014fc00640 1 --2- 172.21.15.63:0/3584339630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0148104d90 0x7f014819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.724+0000 7f014fc00640 1 -- 172.21.15.63:0/3584339630 >> 172.21.15.63:0/3584339630 conn(0x7f01480782d0 msgr2=0x7f01481012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.724+0000 7f014fc00640 1 -- 172.21.15.63:0/3584339630 shutdown_connections 2024-05-27T07:33:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:54.724+0000 7f014fc00640 1 -- 172.21.15.63:0/3584339630 wait complete. 2024-05-27T07:33:54.750 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-27T07:33:55.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.025+0000 7fd975c00640 1 -- 172.21.15.63:0/2479330653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd970114910 msgr2=0x7fd970116d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.025+0000 7fd975c00640 1 --2- 172.21.15.63:0/2479330653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd970114910 0x7fd970116d30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd968009ca0 tx=0x7fd96802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.025+0000 7fd975c00640 1 -- 172.21.15.63:0/2479330653 shutdown_connections 2024-05-27T07:33:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.025+0000 7fd975c00640 1 --2- 172.21.15.63:0/2479330653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd970114910 0x7fd970116d30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.025+0000 7fd975c00640 1 --2- 172.21.15.63:0/2479330653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd970111fb0 0x7fd9701143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.025+0000 7fd975c00640 1 --2- 172.21.15.63:0/2479330653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd970078720 0x7fd970078b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.025+0000 7fd975c00640 1 -- 172.21.15.63:0/2479330653 >> 172.21.15.63:0/2479330653 conn(0x7fd970101460 msgr2=0x7fd970103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.026+0000 7fd975c00640 1 -- 172.21.15.63:0/2479330653 shutdown_connections 2024-05-27T07:33:55.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.026+0000 7fd975c00640 1 -- 172.21.15.63:0/2479330653 wait complete. 2024-05-27T07:33:55.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.027+0000 7fd975c00640 1 Processor -- start 2024-05-27T07:33:55.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.027+0000 7fd975c00640 1 -- start start 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd975c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd970078720 0x7fd9701a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd975c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd970111fb0 0x7fd9701a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd975c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd970114910 0x7fd9701a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd975c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd97011a1a0 con 0x7fd970114910 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd975c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd97011a020 con 0x7fd970111fb0 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd975c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd97011a320 con 0x7fd970078720 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd975200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd970114910 0x7fd9701a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd975200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd970114910 0x7fd9701a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36992/0 (socket says 172.21.15.63:36992) 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd975200640 1 -- 172.21.15.63:0/3958337534 learned_addr learned my addr 172.21.15.63:0/3958337534 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd96fe00640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd970078720 0x7fd9701a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.028+0000 7fd96f400640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd970111fb0 0x7fd9701a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.029+0000 7fd975200640 1 -- 172.21.15.63:0/3958337534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd970078720 msgr2=0x7fd9701a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.029+0000 7fd975200640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd970078720 0x7fd9701a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.029+0000 7fd975200640 1 -- 172.21.15.63:0/3958337534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd970111fb0 msgr2=0x7fd9701a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.029+0000 7fd975200640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd970111fb0 0x7fd9701a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.029+0000 7fd975200640 1 -- 172.21.15.63:0/3958337534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9701a1060 con 0x7fd970114910 2024-05-27T07:33:55.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.029+0000 7fd96f400640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd970111fb0 0x7fd9701a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.029+0000 7fd975200640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd970114910 0x7fd9701a09c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd968005cc0 tx=0x7fd968032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.030+0000 7fd96cc00640 1 -- 172.21.15.63:0/3958337534 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968032e00 con 0x7fd970114910 2024-05-27T07:33:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.030+0000 7fd96cc00640 1 -- 172.21.15.63:0/3958337534 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9680044d0 con 0x7fd970114910 2024-05-27T07:33:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.030+0000 7fd975c00640 1 -- 172.21.15.63:0/3958337534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9701a12f0 con 0x7fd970114910 2024-05-27T07:33:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.030+0000 7fd96cc00640 1 -- 172.21.15.63:0/3958337534 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968030b70 con 0x7fd970114910 2024-05-27T07:33:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.030+0000 7fd975c00640 1 -- 172.21.15.63:0/3958337534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9701ad630 con 0x7fd970114910 2024-05-27T07:33:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.031+0000 7fd975c00640 1 -- 172.21.15.63:0/3958337534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd970050860 con 0x7fd970114910 2024-05-27T07:33:55.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.034+0000 7fd96cc00640 1 -- 172.21.15.63:0/3958337534 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fd968030d10 con 0x7fd970114910 2024-05-27T07:33:55.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.035+0000 7fd96cc00640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9540b2a70 0x7fd9540b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:55.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.036+0000 7fd96cc00640 1 -- 172.21.15.63:0/3958337534 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fd968104280 con 0x7fd970114910 2024-05-27T07:33:55.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.036+0000 7fd96fe00640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9540b2a70 0x7fd9540b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:55.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.037+0000 7fd96fe00640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9540b2a70 0x7fd9540b4f80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd9701a1b00 tx=0x7fd96000b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:55.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.037+0000 7fd96cc00640 1 -- 172.21.15.63:0/3958337534 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9680cb7e0 con 0x7fd970114910 2024-05-27T07:33:55.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.255+0000 7fd975c00640 1 -- 172.21.15.63:0/3958337534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fd9701a1ef0 con 0x7fd970114910 2024-05-27T07:33:55.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.263+0000 7fd96cc00640 1 -- 172.21.15.63:0/3958337534 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7fd9680cb0e0 con 0x7fd970114910 2024-05-27T07:33:55.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 -- 172.21.15.63:0/3958337534 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9540b2a70 msgr2=0x7fd9540b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:55.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9540b2a70 0x7fd9540b4f80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd9701a1b00 tx=0x7fd96000b390 comp rx=0 tx=0).stop 2024-05-27T07:33:55.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 -- 172.21.15.63:0/3958337534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd970114910 msgr2=0x7fd9701a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:55.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd970114910 0x7fd9701a09c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd968005cc0 tx=0x7fd968032a10 comp rx=0 tx=0).stop 2024-05-27T07:33:55.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 -- 172.21.15.63:0/3958337534 shutdown_connections 2024-05-27T07:33:55.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9540b2a70 0x7fd9540b4f80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd970114910 0x7fd9701a09c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd970111fb0 0x7fd9701a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 --2- 172.21.15.63:0/3958337534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd970078720 0x7fd9701a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 -- 172.21.15.63:0/3958337534 >> 172.21.15.63:0/3958337534 conn(0x7fd970101460 msgr2=0x7fd97007b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:55.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 -- 172.21.15.63:0/3958337534 shutdown_connections 2024-05-27T07:33:55.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.266+0000 7fd975c00640 1 -- 172.21.15.63:0/3958337534 wait complete. 2024-05-27T07:33:55.291 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-27T07:33:55.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:55 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/643763716' entity='client.admin' 2024-05-27T07:33:55.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:55 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:55.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:55 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:55.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:55 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:55.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:55 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:55.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:55 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3584339630' entity='client.admin' 2024-05-27T07:33:55.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:55 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:55.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:55 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:55.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:55 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:55.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:55 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:55.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:55 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/643763716' entity='client.admin' 2024-05-27T07:33:55.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:55 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:55.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:55 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:55.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:55 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:55.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:55 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:55.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:55 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3584339630' entity='client.admin' 2024-05-27T07:33:55.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:55 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:55.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:55 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:55.457 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:55 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:55.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:55 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:55.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:55 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/643763716' entity='client.admin' 2024-05-27T07:33:55.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:55 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:55.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:55 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:55.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:55 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:55.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:55 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:55.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:55 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3584339630' entity='client.admin' 2024-05-27T07:33:55.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:55 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:55.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:55 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:55.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:55 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:55.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:55 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.562+0000 7f7329000640 1 -- 172.21.15.63:0/3851850318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f732410b690 msgr2=0x7f732410dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.562+0000 7f7329000640 1 --2- 172.21.15.63:0/3851850318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f732410b690 0x7f732410dab0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7310009900 tx=0x7f731002a410 comp rx=0 tx=0).stop 2024-05-27T07:33:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.562+0000 7f7329000640 1 -- 172.21.15.63:0/3851850318 shutdown_connections 2024-05-27T07:33:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.562+0000 7f7329000640 1 --2- 172.21.15.63:0/3851850318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732410e080 0x7f7324110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.562+0000 7f7329000640 1 --2- 172.21.15.63:0/3851850318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f732410b690 0x7f732410dab0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.562+0000 7f7329000640 1 --2- 172.21.15.63:0/3851850318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7324104c30 0x7f7324105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.562+0000 7f7329000640 1 -- 172.21.15.63:0/3851850318 >> 172.21.15.63:0/3851850318 conn(0x7f7324078160 msgr2=0x7f73240785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.562+0000 7f7329000640 1 -- 172.21.15.63:0/3851850318 shutdown_connections 2024-05-27T07:33:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.562+0000 7f7329000640 1 -- 172.21.15.63:0/3851850318 wait complete. 2024-05-27T07:33:55.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.563+0000 7f7329000640 1 Processor -- start 2024-05-27T07:33:55.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.563+0000 7f7329000640 1 -- start start 2024-05-27T07:33:55.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7329000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7324104c30 0x7f73241a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:55.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7329000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f732410b690 0x7f73241a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:55.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7329000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732410e080 0x7f732419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7329000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7324113b50 con 0x7f7324104c30 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7329000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73241139d0 con 0x7f732410e080 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7329000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7324113cd0 con 0x7f732410b690 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7323400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7324104c30 0x7f73241a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7323400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7324104c30 0x7f73241a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37012/0 (socket says 172.21.15.63:37012) 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7323400640 1 -- 172.21.15.63:0/1676134689 learned_addr learned my addr 172.21.15.63:0/1676134689 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7322a00640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f732410b690 0x7f73241a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7323400640 1 -- 172.21.15.63:0/1676134689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f732410b690 msgr2=0x7f73241a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7323400640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f732410b690 0x7f73241a2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7323400640 1 -- 172.21.15.63:0/1676134689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732410e080 msgr2=0x7f732419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7323e00640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732410e080 0x7f732419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7323400640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732410e080 0x7f732419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7323400640 1 -- 172.21.15.63:0/1676134689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732419ce00 con 0x7f7324104c30 2024-05-27T07:33:55.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.564+0000 7f7322a00640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f732410b690 0x7f73241a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:33:55.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.565+0000 7f7323400640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7324104c30 0x7f73241a2550 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f731400d770 tx=0x7f731400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:55.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.565+0000 7f731be00640 1 -- 172.21.15.63:0/1676134689 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314017070 con 0x7f7324104c30 2024-05-27T07:33:55.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.565+0000 7f731be00640 1 -- 172.21.15.63:0/1676134689 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73140054b0 con 0x7f7324104c30 2024-05-27T07:33:55.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.565+0000 7f7329000640 1 -- 172.21.15.63:0/1676134689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732419d090 con 0x7f7324104c30 2024-05-27T07:33:55.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.565+0000 7f731be00640 1 -- 172.21.15.63:0/1676134689 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314004c60 con 0x7f7324104c30 2024-05-27T07:33:55.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.565+0000 7f7329000640 1 -- 172.21.15.63:0/1676134689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241a92f0 con 0x7f7324104c30 2024-05-27T07:33:55.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.566+0000 7f7329000640 1 -- 172.21.15.63:0/1676134689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73240757d0 con 0x7f7324104c30 2024-05-27T07:33:55.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.569+0000 7f731be00640 1 -- 172.21.15.63:0/1676134689 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f7314004e00 con 0x7f7324104c30 2024-05-27T07:33:55.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.570+0000 7f731be00640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73080b2a70 0x7f73080b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:55.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.570+0000 7f731be00640 1 -- 172.21.15.63:0/1676134689 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f73140e27b0 con 0x7f7324104c30 2024-05-27T07:33:55.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.570+0000 7f7322a00640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73080b2a70 0x7f73080b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:55.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.571+0000 7f7322a00640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73080b2a70 0x7f73080b4f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7310006010 tx=0x7f7310000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:55.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.575+0000 7f731be00640 1 -- 172.21.15.63:0/1676134689 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73140a9b70 con 0x7f7324104c30 2024-05-27T07:33:55.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:55.818+0000 7f7329000640 1 -- 172.21.15.63:0/1676134689 --> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f732410f510 con 0x7f73080b2a70 2024-05-27T07:33:55.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.001+0000 7f731be00640 1 -- 172.21.15.63:0/1676134689 <== mgr.15216 v2:172.21.15.63:6800/1343315489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f732410f510 con 0x7f73080b2a70 2024-05-27T07:33:55.995 INFO:teuthology.orchestra.run.smithi063.stdout:Schedule set for path / 2024-05-27T07:33:55.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.004+0000 7f7329000640 1 -- 172.21.15.63:0/1676134689 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73080b2a70 msgr2=0x7f73080b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:55.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.004+0000 7f7329000640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73080b2a70 0x7f73080b4f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7310006010 tx=0x7f7310000bc0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.004+0000 7f7329000640 1 -- 172.21.15.63:0/1676134689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7324104c30 msgr2=0x7f73241a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:55.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.004+0000 7f7329000640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7324104c30 0x7f73241a2550 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f731400d770 tx=0x7f731400dc30 comp rx=0 tx=0).stop 2024-05-27T07:33:55.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.004+0000 7f7329000640 1 -- 172.21.15.63:0/1676134689 shutdown_connections 2024-05-27T07:33:55.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.004+0000 7f7329000640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73080b2a70 0x7f73080b4f80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.004+0000 7f7329000640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732410e080 0x7f732419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.004+0000 7f7329000640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f732410b690 0x7f73241a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.004+0000 7f7329000640 1 --2- 172.21.15.63:0/1676134689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7324104c30 0x7f73241a2550 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:55.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.004+0000 7f7329000640 1 -- 172.21.15.63:0/1676134689 >> 172.21.15.63:0/1676134689 conn(0x7f7324078160 msgr2=0x7f732410a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:55.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.005+0000 7f7329000640 1 -- 172.21.15.63:0/1676134689 shutdown_connections 2024-05-27T07:33:55.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.005+0000 7f7329000640 1 -- 172.21.15.63:0/1676134689 wait complete. 2024-05-27T07:33:56.030 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-27T07:33:56.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.315+0000 7fb76fe00640 1 -- 172.21.15.63:0/1583630646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb770102840 msgr2=0x7fb770102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.315+0000 7fb76fe00640 1 --2- 172.21.15.63:0/1583630646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb770102840 0x7fb770102c50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb760009900 tx=0x7fb76002a410 comp rx=0 tx=0).stop 2024-05-27T07:33:56.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.315+0000 7fb76fe00640 1 -- 172.21.15.63:0/1583630646 shutdown_connections 2024-05-27T07:33:56.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.315+0000 7fb76fe00640 1 --2- 172.21.15.63:0/1583630646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7701103e0 0x7fb770112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.315+0000 7fb76fe00640 1 --2- 172.21.15.63:0/1583630646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb77010da80 0x7fb77010fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.315+0000 7fb76fe00640 1 --2- 172.21.15.63:0/1583630646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb770102840 0x7fb770102c50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.315+0000 7fb76fe00640 1 -- 172.21.15.63:0/1583630646 >> 172.21.15.63:0/1583630646 conn(0x7fb7700fe210 msgr2=0x7fb770100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:56.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.315+0000 7fb76fe00640 1 -- 172.21.15.63:0/1583630646 shutdown_connections 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.315+0000 7fb76fe00640 1 -- 172.21.15.63:0/1583630646 wait complete. 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.316+0000 7fb76fe00640 1 Processor -- start 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.316+0000 7fb76fe00640 1 -- start start 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.316+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb770102840 0x7fb77019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77010da80 0x7fb77019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7701103e0 0x7fb7701a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb770115d30 con 0x7fb7701103e0 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb770115bb0 con 0x7fb77010da80 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb770115eb0 con 0x7fb770102840 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7701103e0 0x7fb7701a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7701103e0 0x7fb7701a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37030/0 (socket says 172.21.15.63:37030) 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76f400640 1 -- 172.21.15.63:0/2570752812 learned_addr learned my addr 172.21.15.63:0/2570752812 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76ea00640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb770102840 0x7fb77019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76e000640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77010da80 0x7fb77019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76f400640 1 -- 172.21.15.63:0/2570752812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb770102840 msgr2=0x7fb77019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76f400640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb770102840 0x7fb77019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76f400640 1 -- 172.21.15.63:0/2570752812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77010da80 msgr2=0x7fb77019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76f400640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77010da80 0x7fb77019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.317+0000 7fb76f400640 1 -- 172.21.15.63:0/2570752812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7701a1650 con 0x7fb7701103e0 2024-05-27T07:33:56.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.318+0000 7fb76f400640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7701103e0 0x7fb7701a1010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb76800eaa0 tx=0x7fb76800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:56.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.318+0000 7fb767400640 1 -- 172.21.15.63:0/2570752812 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76800cdb0 con 0x7fb7701103e0 2024-05-27T07:33:56.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.318+0000 7fb76fe00640 1 -- 172.21.15.63:0/2570752812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7701a1940 con 0x7fb7701103e0 2024-05-27T07:33:56.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.318+0000 7fb767400640 1 -- 172.21.15.63:0/2570752812 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb768005020 con 0x7fb7701103e0 2024-05-27T07:33:56.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.318+0000 7fb767400640 1 -- 172.21.15.63:0/2570752812 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680116d0 con 0x7fb7701103e0 2024-05-27T07:33:56.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.318+0000 7fb76fe00640 1 -- 172.21.15.63:0/2570752812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7701a1e40 con 0x7fb7701103e0 2024-05-27T07:33:56.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.320+0000 7fb76fe00640 1 -- 172.21.15.63:0/2570752812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77006a230 con 0x7fb7701103e0 2024-05-27T07:33:56.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.324+0000 7fb767400640 1 -- 172.21.15.63:0/2570752812 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fb7680051d0 con 0x7fb7701103e0 2024-05-27T07:33:56.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.325+0000 7fb767400640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7540b2a70 0x7fb7540b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.326+0000 7fb767400640 1 -- 172.21.15.63:0/2570752812 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb7680e3250 con 0x7fb7701103e0 2024-05-27T07:33:56.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.326+0000 7fb76ea00640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7540b2a70 0x7fb7540b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.327+0000 7fb76ea00640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7540b2a70 0x7fb7540b4f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb760000c00 tx=0x7fb760009680 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:56.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.328+0000 7fb767400640 1 -- 172.21.15.63:0/2570752812 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7701a1e40 con 0x7fb7701103e0 2024-05-27T07:33:56.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.520+0000 7f39b1600700 1 -- 172.21.15.63:0/949727481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39ac10c5a0 msgr2=0x7f39ac10c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.520+0000 7f39b1600700 1 --2- 172.21.15.63:0/949727481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39ac10c5a0 0x7f39ac10c9b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f399c009990 tx=0x7f399c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.520+0000 7f39b1600700 1 -- 172.21.15.63:0/949727481 shutdown_connections 2024-05-27T07:33:56.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.520+0000 7f39b1600700 1 --2- 172.21.15.63:0/949727481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac106eb0 0x7f39ac113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.520+0000 7f39b1600700 1 --2- 172.21.15.63:0/949727481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39ac10c5a0 0x7f39ac10c9b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.520+0000 7f39b1600700 1 --2- 172.21.15.63:0/949727481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39ac106560 0x7f39ac106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.520+0000 7f39b1600700 1 -- 172.21.15.63:0/949727481 >> 172.21.15.63:0/949727481 conn(0x7f39ac102040 msgr2=0x7f39ac104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:56.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.520+0000 7f39b1600700 1 -- 172.21.15.63:0/949727481 shutdown_connections 2024-05-27T07:33:56.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.521+0000 7f39b1600700 1 -- 172.21.15.63:0/949727481 wait complete. 2024-05-27T07:33:56.516 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:56 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3958337534' entity='client.admin' 2024-05-27T07:33:56.516 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:56 smithi063 ceph-mon[31613]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-27T07:33:56.516 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:56 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:56.516 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:56 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:56.516 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:56 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:56.516 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:56 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:56.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.522+0000 7f39b1600700 1 Processor -- start 2024-05-27T07:33:56.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.522+0000 7f39b1600700 1 -- start start 2024-05-27T07:33:56.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.523+0000 7f39b1600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39ac106560 0x7f39ac19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.523+0000 7f39b1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac106eb0 0x7f39ac19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.523+0000 7f39b1600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39ac10c5a0 0x7f39ac1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.523+0000 7f39b1600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac116560 con 0x7f39ac10c5a0 2024-05-27T07:33:56.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.523+0000 7f39b1600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1163e0 con 0x7f39ac106eb0 2024-05-27T07:33:56.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.523+0000 7f39b1600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1166e0 con 0x7f39ac106560 2024-05-27T07:33:56.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39ab400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39ac10c5a0 0x7f39ac1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39ab400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39ac10c5a0 0x7f39ac1a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37056/0 (socket says 172.21.15.63:37056) 2024-05-27T07:33:56.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39ab400700 1 -- 172.21.15.63:0/1647703339 learned_addr learned my addr 172.21.15.63:0/1647703339 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:56.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39aaa00700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39ac106560 0x7f39ac19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39aa000700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac106eb0 0x7f39ac19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39ab400700 1 -- 172.21.15.63:0/1647703339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39ac106560 msgr2=0x7f39ac19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39ab400700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39ac106560 0x7f39ac19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39ab400700 1 -- 172.21.15.63:0/1647703339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac106eb0 msgr2=0x7f39ac19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39ab400700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac106eb0 0x7f39ac19d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39ab400700 1 -- 172.21.15.63:0/1647703339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ac1a1e00 con 0x7f39ac10c5a0 2024-05-27T07:33:56.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.524+0000 7f39aa000700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac106eb0 0x7f39ac19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:33:56.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.525+0000 7f39ab400700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39ac10c5a0 0x7f39ac1a1760 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f39a4007ee0 tx=0x7f39a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:56.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.525+0000 7f39a3400700 1 -- 172.21.15.63:0/1647703339 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a400dbd0 con 0x7f39ac10c5a0 2024-05-27T07:33:56.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.525+0000 7f39b1600700 1 -- 172.21.15.63:0/1647703339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ac1a20f0 con 0x7f39ac10c5a0 2024-05-27T07:33:56.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.525+0000 7f39a3400700 1 -- 172.21.15.63:0/1647703339 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39a400dd70 con 0x7f39ac10c5a0 2024-05-27T07:33:56.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.525+0000 7f39a3400700 1 -- 172.21.15.63:0/1647703339 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40058d0 con 0x7f39ac10c5a0 2024-05-27T07:33:56.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.525+0000 7f39b1600700 1 -- 172.21.15.63:0/1647703339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ac1a9a10 con 0x7f39ac10c5a0 2024-05-27T07:33:56.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.529+0000 7f39a3400700 1 -- 172.21.15.63:0/1647703339 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f39a401a020 con 0x7f39ac10c5a0 2024-05-27T07:33:56.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.530+0000 7f39c00cab80 1 Processor -- start 2024-05-27T07:33:56.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.530+0000 7f39c00cab80 1 -- start start 2024-05-27T07:33:56.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.531+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbe73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.531+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbe78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.531+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.531+0000 7f39c00cab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a54cb15ec0 con 0x55a54cb54a70 2024-05-27T07:33:56.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.531+0000 7f39c00cab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a54cb05a60 con 0x55a54ca68ec0 2024-05-27T07:33:56.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.531+0000 7f39c00cab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a54cb09aa0 con 0x55a54cbe7ef0 2024-05-27T07:33:56.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.531+0000 7f39aa000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbe73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.531+0000 7f39aa000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbe73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37062/0 (socket says 172.21.15.63:37062) 2024-05-27T07:33:56.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.531+0000 7f39aa000700 1 -- 172.21.15.63:0/3717483496 learned_addr learned my addr 172.21.15.63:0/3717483496 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:56.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.532+0000 7f39aaa00700 1 --2- 172.21.15.63:0/3717483496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbe78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.532+0000 7f39aa000700 1 --2- 172.21.15.63:0/3717483496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.532+0000 7f39aa000700 1 -- 172.21.15.63:0/3717483496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 msgr2=0x55a54cbf4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:56.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.532+0000 7f39aa000700 1 --2- 172.21.15.63:0/3717483496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.532+0000 7f39a3400700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f39900b2b10 0x7f39900b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.532+0000 7f39aa000700 1 -- 172.21.15.63:0/3717483496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 msgr2=0x55a54cbe78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:56.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.532+0000 7f39aa000700 1 --2- 172.21.15.63:0/3717483496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbe78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.532+0000 7f39aa000700 1 -- 172.21.15.63:0/3717483496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54cace4a0 con 0x55a54cb54a70 2024-05-27T07:33:56.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.532+0000 7f39a3400700 1 -- 172.21.15.63:0/1647703339 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f39a40e2480 con 0x7f39ac10c5a0 2024-05-27T07:33:56.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.532+0000 7f39aa000700 1 --2- 172.21.15.63:0/3717483496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbe73b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3998009fa0 tx=0x7f3998009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294780d9059af75e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:56.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.533+0000 7f39ab400700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f39900b2b10 0x7f39900b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.533+0000 7f3997e00700 1 -- 172.21.15.63:0/3717483496 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399801a030 con 0x55a54cb54a70 2024-05-27T07:33:56.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.533+0000 7f3997e00700 1 -- 172.21.15.63:0/3717483496 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3998014de0 con 0x55a54cb54a70 2024-05-27T07:33:56.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.533+0000 7f3997e00700 1 -- 172.21.15.63:0/3717483496 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998003d10 con 0x55a54cb54a70 2024-05-27T07:33:56.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.534+0000 7f39c00cab80 1 -- 172.21.15.63:0/3717483496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 msgr2=0x55a54cbe73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.534+0000 7f39ab400700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f39900b2b10 0x7f39900b5020 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39a40df6e0 tx=0x7f39a400df20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:56.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.534+0000 7f39c00cab80 1 --2- 172.21.15.63:0/3717483496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbe73b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3998009fa0 tx=0x7f3998009730 comp rx=0 tx=0).stop 2024-05-27T07:33:56.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.534+0000 7f39c00cab80 1 -- 172.21.15.63:0/3717483496 shutdown_connections 2024-05-27T07:33:56.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.534+0000 7f39c00cab80 1 --2- 172.21.15.63:0/3717483496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.534+0000 7f39c00cab80 1 --2- 172.21.15.63:0/3717483496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbe73b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.534+0000 7f39c00cab80 1 --2- 172.21.15.63:0/3717483496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbe78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.534+0000 7f39c00cab80 1 -- 172.21.15.63:0/3717483496 >> 172.21.15.63:0/3717483496 conn(0x55a54cb02410 msgr2=0x55a54cbe3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:56.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.534+0000 7f39c00cab80 1 -- 172.21.15.63:0/3717483496 shutdown_connections 2024-05-27T07:33:56.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.534+0000 7f39c00cab80 1 -- 172.21.15.63:0/3717483496 wait complete. 2024-05-27T07:33:56.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.535+0000 7f39c00cab80 1 -- start start 2024-05-27T07:33:56.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.535+0000 7f39c00cab80 1 Processor -- start 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.536+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54ca68ec0 0x55a54cbf1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.536+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54cb54a70 0x55a54cbf1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.536+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.536+0000 7f39c00cab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a54ca0c020 con 0x55a54ca68ec0 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.536+0000 7f39c00cab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a54caa67e0 con 0x55a54cb54a70 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.536+0000 7f39c00cab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a54cbf4c00 con 0x55a54cbe7ef0 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39c00cab80 10 client.0 _ll_register_callbacks cb 0x55a54caafcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aa000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54ca68ec0 0x55a54cbf1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aa000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54ca68ec0 0x55a54cbf1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37072/0 (socket says 172.21.15.63:37072) 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aa000700 1 -- 172.21.15.63:0/920869369 learned_addr learned my addr 172.21.15.63:0/920869369 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aaa00700 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54cb54a70 0x55a54cbf1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aa000700 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aa000700 1 -- 172.21.15.63:0/920869369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 msgr2=0x55a54cbf2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:56.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aa000700 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aa000700 1 -- 172.21.15.63:0/920869369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54cb54a70 msgr2=0x55a54cbf1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aa000700 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54cb54a70 0x55a54cbf1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aa000700 1 -- 172.21.15.63:0/920869369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54ca55950 con 0x55a54ca68ec0 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.537+0000 7f39aaa00700 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54cb54a70 0x55a54cbf1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.538+0000 7f39aa000700 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54ca68ec0 0x55a54cbf1680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3998009950 tx=0x7f39980038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938169ff5ca8c534 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.538+0000 7f3994c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.538+0000 7f3994c00700 1 -- 172.21.15.63:0/920869369 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399801f470 con 0x55a54ca68ec0 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.538+0000 7f39c00cab80 1 -- 172.21.15.63:0/920869369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3998014de0 con 0x55a54ca68ec0 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.538+0000 7f3994c00700 1 -- 172.21.15.63:0/920869369 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f399801f610 con 0x55a54ca68ec0 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f3994c00700 1 -- 172.21.15.63:0/920869369 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f399801f810 con 0x55a54ca68ec0 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f39c00cab80 10 client.15393 fetch_fsmap learned FSMap version 27 2024-05-27T07:33:56.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f39c00cab80 1 -- 172.21.15.63:0/920869369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a54cad60e0 con 0x55a54ca68ec0 2024-05-27T07:33:56.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f3994c00700 1 -- 172.21.15.63:0/920869369 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3998014de0 con 0x55a54ca68ec0 2024-05-27T07:33:56.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f3994c00700 1 -- 172.21.15.63:0/920869369 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f399800f040 con 0x55a54ca68ec0 2024-05-27T07:33:56.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f39c00cab80 10 client.15393 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:33:56.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f39c00cab80 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:33:56.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f39c00cab80 20 client.15393 populate_metadata read hostname 'smithi063' 2024-05-27T07:33:56.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f39c00cab80 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a54cc01a00 0x55a54cc03e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f39c00cab80 4 client.15393 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:33:56.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.539+0000 7f39c00cab80 1 -- 172.21.15.63:0/920869369 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a54cb17e10 con 0x55a54cc01a00 2024-05-27T07:33:56.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.540+0000 7f39aaa00700 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a54cc01a00 0x55a54cc03e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.541+0000 7f39aaa00700 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a54cc01a00 0x55a54cc03e20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552b63365250ef37 server_cookie=16c952a29b7b637f in_seq=0 out_seq=0 2024-05-27T07:33:56.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.541+0000 7f3994c00700 10 client.15393 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:33:56.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.543+0000 7f3994c00700 1 -- 172.21.15.63:0/920869369 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a54cb17e10 con 0x55a54cc01a00 2024-05-27T07:33:56.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.543+0000 7f3994c00700 10 client.15393 handle_command_reply: tid=0 2024-05-27T07:33:56.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.543+0000 7f39c00cab80 1 client.15393 shutdown 2024-05-27T07:33:56.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 1 -- 172.21.15.63:0/920869369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54ca68ec0 msgr2=0x55a54cbf1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54ca68ec0 0x55a54cbf1680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3998009950 tx=0x7f39980038b0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 1 -- 172.21.15.63:0/920869369 shutdown_connections 2024-05-27T07:33:56.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a54cc01a00 0x55a54cc03e20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54ca68ec0 0x55a54cbf1680 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 1 --2- 172.21.15.63:0/920869369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54cb54a70 0x55a54cbf1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 1 -- 172.21.15.63:0/920869369 >> 172.21.15.63:0/920869369 conn(0x55a54cb02410 msgr2=0x55a54cbe73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:56.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f3994c00700 0 client.15393 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:33:56.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 1 -- 172.21.15.63:0/920869369 shutdown_connections 2024-05-27T07:33:56.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 1 -- 172.21.15.63:0/920869369 wait complete. 2024-05-27T07:33:56.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.544+0000 7f39c00cab80 20 client.15393 trim_cache size 0 max 16384 2024-05-27T07:33:56.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.567+0000 7f39c00cab80 1 Processor -- start 2024-05-27T07:33:56.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.567+0000 7f39c00cab80 1 -- start start 2024-05-27T07:33:56.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.567+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.567+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cb54a70 0x55a54cbfdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.567+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cbe7ef0 0x55a54cbeb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.567+0000 7f39c00cab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b4f00 con 0x55a54cbe7ef0 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.567+0000 7f39c00cab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b2790 con 0x55a54ca68ec0 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.567+0000 7f39c00cab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b05e0 con 0x55a54cb54a70 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.568+0000 7f39aa000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cbe7ef0 0x55a54cbeb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.568+0000 7f39aa000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cbe7ef0 0x55a54cbeb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37084/0 (socket says 172.21.15.63:37084) 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.568+0000 7f39aa000700 1 -- 172.21.15.63:0/3236637148 learned_addr learned my addr 172.21.15.63:0/3236637148 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.568+0000 7f39aa000700 1 --2- 172.21.15.63:0/3236637148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.568+0000 7f39aaa00700 1 --2- 172.21.15.63:0/3236637148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cb54a70 0x55a54cbfdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.568+0000 7f39aa000700 1 -- 172.21.15.63:0/3236637148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cb54a70 msgr2=0x55a54cbfdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.568+0000 7f39aa000700 1 --2- 172.21.15.63:0/3236637148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cb54a70 0x55a54cbfdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.568+0000 7f39aa000700 1 -- 172.21.15.63:0/3236637148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 msgr2=0x55a54cbece70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.568+0000 7f39aa000700 1 --2- 172.21.15.63:0/3236637148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbece70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.568+0000 7f39aa000700 1 -- 172.21.15.63:0/3236637148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54cbdf2c0 con 0x55a54cbe7ef0 2024-05-27T07:33:56.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.569+0000 7f39aa000700 1 --2- 172.21.15.63:0/3236637148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cbe7ef0 0x55a54cbeb5d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3998014fd0 tx=0x7f39980094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bf95b5cafed379 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:56.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.569+0000 7f3997400700 1 -- 172.21.15.63:0/3236637148 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399801a070 con 0x55a54cbe7ef0 2024-05-27T07:33:56.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.569+0000 7f3997400700 1 -- 172.21.15.63:0/3236637148 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39980095c0 con 0x55a54cbe7ef0 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.569+0000 7f3997400700 1 -- 172.21.15.63:0/3236637148 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399801fc10 con 0x55a54cbe7ef0 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.570+0000 7f39c00cab80 1 -- 172.21.15.63:0/3236637148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cbe7ef0 msgr2=0x55a54cbeb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.570+0000 7f39c00cab80 1 --2- 172.21.15.63:0/3236637148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cbe7ef0 0x55a54cbeb5d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3998014fd0 tx=0x7f39980094b0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.570+0000 7f39c00cab80 1 -- 172.21.15.63:0/3236637148 shutdown_connections 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.570+0000 7f39c00cab80 1 --2- 172.21.15.63:0/3236637148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cbe7ef0 0x55a54cbeb5d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.570+0000 7f39c00cab80 1 --2- 172.21.15.63:0/3236637148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbece70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.570+0000 7f39c00cab80 1 --2- 172.21.15.63:0/3236637148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cb54a70 0x55a54cbfdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.570+0000 7f39c00cab80 1 -- 172.21.15.63:0/3236637148 >> 172.21.15.63:0/3236637148 conn(0x55a54cb02410 msgr2=0x55a54cbf1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.570+0000 7f39c00cab80 1 -- 172.21.15.63:0/3236637148 shutdown_connections 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.570+0000 7f39c00cab80 1 -- 172.21.15.63:0/3236637148 wait complete. 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.571+0000 7f39c00cab80 1 -- start start 2024-05-27T07:33:56.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.571+0000 7f39c00cab80 1 Processor -- start 2024-05-27T07:33:56.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.571+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbfa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbf5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39c00cab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39c00cab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b0c50 con 0x55a54cb54a70 2024-05-27T07:33:56.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39c00cab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b20c0 con 0x55a54ca68ec0 2024-05-27T07:33:56.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39c00cab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac1b1320 con 0x55a54cbe7ef0 2024-05-27T07:33:56.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39c00cab80 10 client.0 _ll_register_callbacks cb 0x55a54caafcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbf5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbf5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37100/0 (socket says 172.21.15.63:37100) 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aaa00700 1 -- 172.21.15.63:0/1141908977 learned_addr learned my addr 172.21.15.63:0/1141908977 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aa000700 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbfa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aa000700 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aaa00700 1 -- 172.21.15.63:0/1141908977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 msgr2=0x55a54cbf6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aaa00700 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aaa00700 1 -- 172.21.15.63:0/1141908977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 msgr2=0x55a54cbfa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aaa00700 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbfa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aaa00700 1 -- 172.21.15.63:0/1141908977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54cbf67d0 con 0x55a54cb54a70 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.572+0000 7f39aa000700 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.573+0000 7f39aaa00700 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbf5a00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f39ac1079d0 tx=0x7f399c0053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05cae6da80fda5e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:56.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.573+0000 7f3994c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:33:56.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.573+0000 7f3994c00700 1 -- 172.21.15.63:0/1141908977 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c0039d0 con 0x55a54cb54a70 2024-05-27T07:33:56.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.573+0000 7f3994c00700 1 -- 172.21.15.63:0/1141908977 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f399c003b70 con 0x55a54cb54a70 2024-05-27T07:33:56.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.573+0000 7f39c00cab80 1 -- 172.21.15.63:0/1141908977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39980095c0 con 0x55a54cb54a70 2024-05-27T07:33:56.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.573+0000 7f3994c00700 1 -- 172.21.15.63:0/1141908977 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f399c037050 con 0x55a54cb54a70 2024-05-27T07:33:56.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.573+0000 7f39c00cab80 10 client.15399 fetch_fsmap learned FSMap version 27 2024-05-27T07:33:56.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.573+0000 7f39c00cab80 1 -- 172.21.15.63:0/1141908977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a54cc6c350 con 0x55a54cb54a70 2024-05-27T07:33:56.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.574+0000 7f3994c00700 1 -- 172.21.15.63:0/1141908977 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f39980095c0 con 0x55a54cb54a70 2024-05-27T07:33:56.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.574+0000 7f3994c00700 1 -- 172.21.15.63:0/1141908977 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f399c00f4f0 con 0x55a54cb54a70 2024-05-27T07:33:56.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.574+0000 7f39c00cab80 10 client.15399 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:33:56.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.574+0000 7f39c00cab80 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:33:56.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.574+0000 7f39c00cab80 20 client.15399 populate_metadata read hostname 'smithi063' 2024-05-27T07:33:56.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.574+0000 7f39c00cab80 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a54cc6c9e0 0x55a54cc6ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:56.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.574+0000 7f39c00cab80 4 client.15399 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:33:56.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.574+0000 7f39c00cab80 1 -- 172.21.15.63:0/1141908977 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a54cbe21a0 con 0x55a54cc6c9e0 2024-05-27T07:33:56.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.575+0000 7f39aa000700 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a54cc6c9e0 0x55a54cc6ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:56.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.576+0000 7f39aa000700 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a54cc6c9e0 0x55a54cc6ee00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4617d24e73a8fda server_cookie=57e940bbc58e4b1e in_seq=0 out_seq=0 2024-05-27T07:33:56.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.576+0000 7f3994c00700 10 client.15399 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f3994c00700 1 -- 172.21.15.63:0/1141908977 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a54cbe21a0 con 0x55a54cc6c9e0 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f3994c00700 10 client.15399 handle_command_reply: tid=0 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 client.15399 shutdown 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 -- 172.21.15.63:0/1141908977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 msgr2=0x55a54cbf5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbf5a00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f39ac1079d0 tx=0x7f399c0053d0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 -- 172.21.15.63:0/1141908977 shutdown_connections 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a54cc6c9e0 0x55a54cc6ee00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a54cbe7ef0 0x55a54cbf6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a54ca68ec0 0x55a54cbfa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 --2- 172.21.15.63:0/1141908977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a54cb54a70 0x55a54cbf5a00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 -- 172.21.15.63:0/1141908977 >> 172.21.15.63:0/1141908977 conn(0x55a54cb02410 msgr2=0x55a54cbf0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:56.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f3994c00700 0 client.15399 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:33:56.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 -- 172.21.15.63:0/1141908977 shutdown_connections 2024-05-27T07:33:56.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.580+0000 7f39c00cab80 1 -- 172.21.15.63:0/1141908977 wait complete. 2024-05-27T07:33:56.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.581+0000 7f39c00cab80 20 client.15399 trim_cache size 0 max 16384 2024-05-27T07:33:56.576 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:33:56.576 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:33:56.576 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "8cf45254-c799-4756-95a5-11dce3b1c1df", 2024-05-27T07:33:56.576 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:33:56.576 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:33:56.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 -- 172.21.15.63:0/1647703339 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f39900b2b10 msgr2=0x7f39900b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f39900b2b10 0x7f39900b5020 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39a40df6e0 tx=0x7f39a400df20 comp rx=0 tx=0).stop 2024-05-27T07:33:56.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 -- 172.21.15.63:0/1647703339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39ac10c5a0 msgr2=0x7f39ac1a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39ac10c5a0 0x7f39ac1a1760 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f39a4007ee0 tx=0x7f39a400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 -- 172.21.15.63:0/1647703339 shutdown_connections 2024-05-27T07:33:56.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f39900b2b10 0x7f39900b5020 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39ac10c5a0 0x7f39ac1a1760 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39ac106560 0x7f39ac19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 --2- 172.21.15.63:0/1647703339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39ac106eb0 0x7f39ac19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 -- 172.21.15.63:0/1647703339 >> 172.21.15.63:0/1647703339 conn(0x7f39ac102040 msgr2=0x7f39ac103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:56.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 -- 172.21.15.63:0/1647703339 shutdown_connections 2024-05-27T07:33:56.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.582+0000 7f39b1600700 1 -- 172.21.15.63:0/1647703339 wait complete. 2024-05-27T07:33:56.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.597+0000 7fb76fe00640 1 -- 172.21.15.63:0/2570752812 --> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fb770100d40 con 0x7fb7540b2a70 2024-05-27T07:33:56.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:56 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3958337534' entity='client.admin' 2024-05-27T07:33:56.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:56 smithi086 ceph-mon[34687]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-27T07:33:56.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:56 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:56.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:56 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:56.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:56 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:56.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:56 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:56.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:56 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3958337534' entity='client.admin' 2024-05-27T07:33:56.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:56 smithi149 ceph-mon[35408]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-27T07:33:56.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:56 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:56.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:56 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:56.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:56 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:56.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:56 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:33:56.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.741+0000 7fb767400640 1 -- 172.21.15.63:0/2570752812 <== mgr.15216 v2:172.21.15.63:6800/1343315489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fb770100d40 con 0x7fb7540b2a70 2024-05-27T07:33:56.735 INFO:teuthology.orchestra.run.smithi063.stdout:Retention added to path / 2024-05-27T07:33:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 -- 172.21.15.63:0/2570752812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7540b2a70 msgr2=0x7fb7540b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7540b2a70 0x7fb7540b4f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb760000c00 tx=0x7fb760009680 comp rx=0 tx=0).stop 2024-05-27T07:33:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 -- 172.21.15.63:0/2570752812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7701103e0 msgr2=0x7fb7701a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7701103e0 0x7fb7701a1010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb76800eaa0 tx=0x7fb76800ef60 comp rx=0 tx=0).stop 2024-05-27T07:33:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 -- 172.21.15.63:0/2570752812 shutdown_connections 2024-05-27T07:33:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7540b2a70 0x7fb7540b4f80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7701103e0 0x7fb7701a1010 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77010da80 0x7fb77019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 --2- 172.21.15.63:0/2570752812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb770102840 0x7fb77019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:56.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 -- 172.21.15.63:0/2570752812 >> 172.21.15.63:0/2570752812 conn(0x7fb7700fe210 msgr2=0x7fb770100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:56.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 -- 172.21.15.63:0/2570752812 shutdown_connections 2024-05-27T07:33:56.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:56.744+0000 7fb76fe00640 1 -- 172.21.15.63:0/2570752812 wait complete. 2024-05-27T07:33:56.762 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-27T07:33:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.021+0000 7f2943200640 1 -- 172.21.15.63:0/702153961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294410c170 msgr2=0x7f2944105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.021+0000 7f2943200640 1 --2- 172.21.15.63:0/702153961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294410c170 0x7f2944105db0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2930009900 tx=0x7f293002a410 comp rx=0 tx=0).stop 2024-05-27T07:33:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.022+0000 7f2943200640 1 -- 172.21.15.63:0/702153961 shutdown_connections 2024-05-27T07:33:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.022+0000 7f2943200640 1 --2- 172.21.15.63:0/702153961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2944106420 0x7f29441127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.022+0000 7f2943200640 1 --2- 172.21.15.63:0/702153961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294410c170 0x7f2944105db0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.022+0000 7f2943200640 1 --2- 172.21.15.63:0/702153961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294410bcd0 0x7f2944105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.022+0000 7f2943200640 1 -- 172.21.15.63:0/702153961 >> 172.21.15.63:0/702153961 conn(0x7f29441013d0 msgr2=0x7f2944103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.022+0000 7f2943200640 1 -- 172.21.15.63:0/702153961 shutdown_connections 2024-05-27T07:33:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.022+0000 7f2943200640 1 -- 172.21.15.63:0/702153961 wait complete. 2024-05-27T07:33:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.023+0000 7f2943200640 1 Processor -- start 2024-05-27T07:33:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.023+0000 7f2943200640 1 -- start start 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.023+0000 7f2943200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2944106420 0x7f29441a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.023+0000 7f2943200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294410bcd0 0x7f29441a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.023+0000 7f2943200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294410c170 0x7f294419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.023+0000 7f2943200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2944115e50 con 0x7f294410bcd0 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.023+0000 7f2943200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2944115cd0 con 0x7f294410c170 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.023+0000 7f2943200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2944115fd0 con 0x7f2944106420 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.024+0000 7f2941400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294410bcd0 0x7f29441a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.024+0000 7f2941400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294410bcd0 0x7f29441a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37118/0 (socket says 172.21.15.63:37118) 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.024+0000 7f2941400640 1 -- 172.21.15.63:0/3002808917 learned_addr learned my addr 172.21.15.63:0/3002808917 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.024+0000 7f2942800640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294410c170 0x7f294419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:57.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.024+0000 7f2941e00640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2944106420 0x7f29441a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:57.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.025+0000 7f2941400640 1 -- 172.21.15.63:0/3002808917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2944106420 msgr2=0x7f29441a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.025+0000 7f2941400640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2944106420 0x7f29441a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.025+0000 7f2941400640 1 -- 172.21.15.63:0/3002808917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294410c170 msgr2=0x7f294419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.025+0000 7f2941400640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294410c170 0x7f294419c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.025+0000 7f2941400640 1 -- 172.21.15.63:0/3002808917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294419ce90 con 0x7f294410bcd0 2024-05-27T07:33:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.025+0000 7f2941400640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294410bcd0 0x7f29441a2ac0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2930005940 tx=0x7f29300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.025+0000 7f293aa00640 1 -- 172.21.15.63:0/3002808917 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293003f030 con 0x7f294410bcd0 2024-05-27T07:33:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.026+0000 7f2943200640 1 -- 172.21.15.63:0/3002808917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294419d120 con 0x7f294410bcd0 2024-05-27T07:33:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.026+0000 7f293aa00640 1 -- 172.21.15.63:0/3002808917 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2930003c90 con 0x7f294410bcd0 2024-05-27T07:33:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.026+0000 7f293aa00640 1 -- 172.21.15.63:0/3002808917 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930035070 con 0x7f294410bcd0 2024-05-27T07:33:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.026+0000 7f2943200640 1 -- 172.21.15.63:0/3002808917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29441a93d0 con 0x7f294410bcd0 2024-05-27T07:33:57.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.027+0000 7f2943200640 1 -- 172.21.15.63:0/3002808917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2944106fb0 con 0x7f294410bcd0 2024-05-27T07:33:57.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.030+0000 7f293aa00640 1 -- 172.21.15.63:0/3002808917 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f2930042030 con 0x7f294410bcd0 2024-05-27T07:33:57.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.031+0000 7f293aa00640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29280b2af0 0x7f29280b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:57.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.032+0000 7f293aa00640 1 -- 172.21.15.63:0/3002808917 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f29301033f0 con 0x7f294410bcd0 2024-05-27T07:33:57.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.032+0000 7f2941e00640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29280b2af0 0x7f29280b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:57.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.033+0000 7f2941e00640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29280b2af0 0x7f29280b5000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f294419dae0 tx=0x7f29340090e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:57.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.036+0000 7f293aa00640 1 -- 172.21.15.63:0/3002808917 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29300ca230 con 0x7f294410bcd0 2024-05-27T07:33:57.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.268+0000 7f2943200640 1 -- 172.21.15.63:0/3002808917 --> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2944112030 con 0x7f29280b2af0 2024-05-27T07:33:57.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.286+0000 7f293aa00640 1 -- 172.21.15.63:0/3002808917 <== mgr.15216 v2:172.21.15.63:6800/1343315489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2944112030 con 0x7f29280b2af0 2024-05-27T07:33:57.280 INFO:teuthology.orchestra.run.smithi063.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-27T00:00:00", "created": "2024-05-27T07:33:55", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-27T07:33:57.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.288+0000 7f2943200640 1 -- 172.21.15.63:0/3002808917 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29280b2af0 msgr2=0x7f29280b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:57.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.288+0000 7f2943200640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29280b2af0 0x7f29280b5000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f294419dae0 tx=0x7f29340090e0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.289+0000 7f2943200640 1 -- 172.21.15.63:0/3002808917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294410bcd0 msgr2=0x7f29441a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:57.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.289+0000 7f2943200640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294410bcd0 0x7f29441a2ac0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2930005940 tx=0x7f29300042f0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.289+0000 7f2943200640 1 -- 172.21.15.63:0/3002808917 shutdown_connections 2024-05-27T07:33:57.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.289+0000 7f2943200640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29280b2af0 0x7f29280b5000 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.289+0000 7f2943200640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294410c170 0x7f294419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.289+0000 7f2943200640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294410bcd0 0x7f29441a2ac0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.289+0000 7f2943200640 1 --2- 172.21.15.63:0/3002808917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2944106420 0x7f29441a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.289+0000 7f2943200640 1 -- 172.21.15.63:0/3002808917 >> 172.21.15.63:0/3002808917 conn(0x7f29441013d0 msgr2=0x7f2944111920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:57.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.289+0000 7f2943200640 1 -- 172.21.15.63:0/3002808917 shutdown_connections 2024-05-27T07:33:57.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.289+0000 7f2943200640 1 -- 172.21.15.63:0/3002808917 wait complete. 2024-05-27T07:33:57.308 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-27T07:33:57.558 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:57 smithi063 ceph-mon[31613]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:57.561 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:33:57.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.601+0000 7f9dbc800640 1 -- 172.21.15.63:0/1839537183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db410c340 msgr2=0x7f9db4105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:57.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.601+0000 7f9dbc800640 1 --2- 172.21.15.63:0/1839537183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db410c340 0x7f9db4105f80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009900 tx=0x7f9da402a410 comp rx=0 tx=0).stop 2024-05-27T07:33:57.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.601+0000 7f9dbc800640 1 -- 172.21.15.63:0/1839537183 shutdown_connections 2024-05-27T07:33:57.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.601+0000 7f9dbc800640 1 --2- 172.21.15.63:0/1839537183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9db41065f0 0x7f9db41129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.601+0000 7f9dbc800640 1 --2- 172.21.15.63:0/1839537183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db410c340 0x7f9db4105f80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.601+0000 7f9dbc800640 1 --2- 172.21.15.63:0/1839537183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db410bea0 0x7f9db4105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.601+0000 7f9dbc800640 1 -- 172.21.15.63:0/1839537183 >> 172.21.15.63:0/1839537183 conn(0x7f9db40771f0 msgr2=0x7f9db4077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:57.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.601+0000 7f9dbc800640 1 -- 172.21.15.63:0/1839537183 shutdown_connections 2024-05-27T07:33:57.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.601+0000 7f9dbc800640 1 -- 172.21.15.63:0/1839537183 wait complete. 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.602+0000 7f9dbc800640 1 Processor -- start 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.602+0000 7f9dbc800640 1 -- start start 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.603+0000 7f9dbc800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db41065f0 0x7f9db41a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.603+0000 7f9dbc800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db410bea0 0x7f9db41a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.603+0000 7f9dbc800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9db410c340 0x7f9db419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.603+0000 7f9dbc800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4115f90 con 0x7f9db410bea0 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.603+0000 7f9dbc800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4115e10 con 0x7f9db41065f0 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.603+0000 7f9dbc800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4116110 con 0x7f9db410c340 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.603+0000 7f9db9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db410bea0 0x7f9db41a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.603+0000 7f9db9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db410bea0 0x7f9db41a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37132/0 (socket says 172.21.15.63:37132) 2024-05-27T07:33:57.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.603+0000 7f9db9400640 1 -- 172.21.15.63:0/2779714825 learned_addr learned my addr 172.21.15.63:0/2779714825 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.603+0000 7f9db9e00640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db41065f0 0x7f9db41a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.604+0000 7f9dba800640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9db410c340 0x7f9db419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.604+0000 7f9db9400640 1 -- 172.21.15.63:0/2779714825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9db410c340 msgr2=0x7f9db419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.604+0000 7f9db9400640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9db410c340 0x7f9db419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.604+0000 7f9db9400640 1 -- 172.21.15.63:0/2779714825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db41065f0 msgr2=0x7f9db41a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.604+0000 7f9db9400640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db41065f0 0x7f9db41a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.604+0000 7f9db9400640 1 -- 172.21.15.63:0/2779714825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db419cf30 con 0x7f9db410bea0 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.604+0000 7f9db9400640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db410bea0 0x7f9db41a2b60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9da402a930 tx=0x7f9da4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.605+0000 7f9daea00640 1 -- 172.21.15.63:0/2779714825 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da403f070 con 0x7f9db410bea0 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.605+0000 7f9daea00640 1 -- 172.21.15.63:0/2779714825 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9da4003e10 con 0x7f9db410bea0 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.605+0000 7f9daea00640 1 -- 172.21.15.63:0/2779714825 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4035030 con 0x7f9db410bea0 2024-05-27T07:33:57.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.605+0000 7f9dbc800640 1 -- 172.21.15.63:0/2779714825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db419d1c0 con 0x7f9db410bea0 2024-05-27T07:33:57.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.605+0000 7f9dbc800640 1 -- 172.21.15.63:0/2779714825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db419d6b0 con 0x7f9db410bea0 2024-05-27T07:33:57.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.606+0000 7f9dbc800640 1 -- 172.21.15.63:0/2779714825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db4107180 con 0x7f9db410bea0 2024-05-27T07:33:57.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.610+0000 7f9daea00640 1 -- 172.21.15.63:0/2779714825 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f9da4042070 con 0x7f9db410bea0 2024-05-27T07:33:57.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.612+0000 7f9daea00640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d9c0b2a70 0x7f9d9c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:57.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.612+0000 7f9daea00640 1 -- 172.21.15.63:0/2779714825 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f9da4103840 con 0x7f9db410bea0 2024-05-27T07:33:57.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.612+0000 7f9db9e00640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d9c0b2a70 0x7f9d9c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:57.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.613+0000 7f9db9e00640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d9c0b2a70 0x7f9d9c0b4f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9da8005fd0 tx=0x7f9da800bf30 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:57.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.614+0000 7f9daea00640 1 -- 172.21.15.63:0/2779714825 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9db419d6b0 con 0x7f9db410bea0 2024-05-27T07:33:57.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:57 smithi086 ceph-mon[34687]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:57.706 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:57 smithi149 ceph-mon[35408]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:57.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.865+0000 7f9dbc800640 1 -- 172.21.15.63:0/2779714825 --> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9db4063100 con 0x7f9d9c0b2a70 2024-05-27T07:33:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.878+0000 7f9daea00640 1 -- 172.21.15.63:0/2779714825 <== mgr.15216 v2:172.21.15.63:6800/1343315489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9db4063100 con 0x7f9d9c0b2a70 2024-05-27T07:33:57.872 INFO:teuthology.orchestra.run.smithi063.stdout:/ 1m 6m3h 2024-05-27T07:33:57.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.880+0000 7f9dbc800640 1 -- 172.21.15.63:0/2779714825 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d9c0b2a70 msgr2=0x7f9d9c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:57.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.880+0000 7f9dbc800640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d9c0b2a70 0x7f9d9c0b4f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9da8005fd0 tx=0x7f9da800bf30 comp rx=0 tx=0).stop 2024-05-27T07:33:57.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.880+0000 7f9dbc800640 1 -- 172.21.15.63:0/2779714825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db410bea0 msgr2=0x7f9db41a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:57.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.880+0000 7f9dbc800640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db410bea0 0x7f9db41a2b60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9da402a930 tx=0x7f9da4003c20 comp rx=0 tx=0).stop 2024-05-27T07:33:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.881+0000 7f9dbc800640 1 -- 172.21.15.63:0/2779714825 shutdown_connections 2024-05-27T07:33:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.881+0000 7f9dbc800640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d9c0b2a70 0x7f9d9c0b4f80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.881+0000 7f9dbc800640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9db410c340 0x7f9db419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.881+0000 7f9dbc800640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9db410bea0 0x7f9db41a2b60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.881+0000 7f9dbc800640 1 --2- 172.21.15.63:0/2779714825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9db41065f0 0x7f9db41a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.881+0000 7f9dbc800640 1 -- 172.21.15.63:0/2779714825 >> 172.21.15.63:0/2779714825 conn(0x7f9db40771f0 msgr2=0x7f9db4107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.881+0000 7f9dbc800640 1 -- 172.21.15.63:0/2779714825 shutdown_connections 2024-05-27T07:33:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:33:57.881+0000 7f9dbc800640 1 -- 172.21.15.63:0/2779714825 wait complete. 2024-05-27T07:33:57.901 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-27T07:33:58.007 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:33:58.011 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-27T07:33:58.021 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-27T07:33:58.030 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T07:33:58.031 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-27T07:33:58.035 INFO:teuthology.run_tasks:Running task workunit... 2024-05-27T07:33:58.062 INFO:tasks.workunit:Pulling workunits from ref 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:33:58.063 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-27T07:33:58.063 INFO:tasks.workunit:timeout=3h 2024-05-27T07:33:58.063 INFO:tasks.workunit:cleanup=True 2024-05-27T07:33:58.063 DEBUG:teuthology.orchestra.run.smithi063:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:58.103 INFO:teuthology.orchestra.run.smithi063.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:58.104 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-27T07:33:58.104 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-05-27T07:33:58.104 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:33:58.104 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-27T07:33:58.104 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-27 07:30:04.919991355 +0000 2024-05-27T07:33:58.104 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-27 07:30:04.919991355 +0000 2024-05-27T07:33:58.104 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-27 07:33:33.323967841 +0000 2024-05-27T07:33:58.104 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-05-27T07:33:58.104 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-27T07:33:58.105 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-27T07:33:58.238 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:33:58.299 INFO:tasks.workunit.client.0.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-27T07:33:58.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:58 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:33:58.426 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:33:58.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:33:58.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:33:58.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:58 smithi063 ceph-mon[31613]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:58.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:58 smithi063 ceph-mon[31613]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:58.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:58 smithi063 ceph-mon[31613]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T07:33:58.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:58 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:33:58.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:33:58.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:33:58.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:33:58.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:58 smithi086 ceph-mon[34687]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:58.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:58 smithi086 ceph-mon[34687]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:58.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:58 smithi086 ceph-mon[34687]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T07:33:58.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:58 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:33:58.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:33:58.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:33:58.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:33:58.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:58 smithi149 ceph-mon[35408]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:58.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:58 smithi149 ceph-mon[35408]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:58.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:58 smithi149 ceph-mon[35408]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T07:33:59.459 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:33:59 smithi063 ceph-mon[31613]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:59.604 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:33:59 smithi086 ceph-mon[34687]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:59.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:33:59 smithi149 ceph-mon[35408]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:34:00.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:00 smithi063 ceph-mon[31613]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-05-27T07:34:00.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.563+0000 7fc20d400700 1 -- 172.21.15.63:0/212425154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2081109a0 msgr2=0x7fc208112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:00.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.563+0000 7fc20d400700 1 --2- 172.21.15.63:0/212425154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2081109a0 0x7fc208112e00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc20000b0a0 tx=0x7fc20000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.563+0000 7fc20d400700 1 -- 172.21.15.63:0/212425154 shutdown_connections 2024-05-27T07:34:00.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.563+0000 7fc20d400700 1 --2- 172.21.15.63:0/212425154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2081109a0 0x7fc208112e00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.563+0000 7fc20d400700 1 --2- 172.21.15.63:0/212425154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc208075d20 0x7fc208076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.563+0000 7fc20d400700 1 --2- 172.21.15.63:0/212425154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc208076670 0x7fc208110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.563+0000 7fc20d400700 1 -- 172.21.15.63:0/212425154 >> 172.21.15.63:0/212425154 conn(0x7fc208079d00 msgr2=0x7fc20807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:00.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.563+0000 7fc20d400700 1 -- 172.21.15.63:0/212425154 shutdown_connections 2024-05-27T07:34:00.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.563+0000 7fc20d400700 1 -- 172.21.15.63:0/212425154 wait complete. 2024-05-27T07:34:00.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.564+0000 7fc20d400700 1 Processor -- start 2024-05-27T07:34:00.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.564+0000 7fc20d400700 1 -- start start 2024-05-27T07:34:00.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.565+0000 7fc20d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc208075d20 0x7fc20819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.565+0000 7fc20d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc208076670 0x7fc20819d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.565+0000 7fc20d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2081109a0 0x7fc2081a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.565+0000 7fc20d400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc208116330 con 0x7fc2081109a0 2024-05-27T07:34:00.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.565+0000 7fc20d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081161b0 con 0x7fc208076670 2024-05-27T07:34:00.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.565+0000 7fc20d400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081164b0 con 0x7fc208075d20 2024-05-27T07:34:00.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.565+0000 7fc20ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2081109a0 0x7fc2081a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.565+0000 7fc20ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2081109a0 0x7fc2081a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58038/0 (socket says 172.21.15.63:58038) 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.565+0000 7fc20ca00700 1 -- 172.21.15.63:0/3859432920 learned_addr learned my addr 172.21.15.63:0/3859432920 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.566+0000 7fc207400700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc208076670 0x7fc20819d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.566+0000 7fc207e00700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc208075d20 0x7fc20819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.566+0000 7fc20ca00700 1 -- 172.21.15.63:0/3859432920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc208075d20 msgr2=0x7fc20819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.566+0000 7fc20ca00700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc208075d20 0x7fc20819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.566+0000 7fc20ca00700 1 -- 172.21.15.63:0/3859432920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc208076670 msgr2=0x7fc20819d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.566+0000 7fc20ca00700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc208076670 0x7fc20819d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.566+0000 7fc20ca00700 1 -- 172.21.15.63:0/3859432920 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2081a1bf0 con 0x7fc2081109a0 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.566+0000 7fc20ca00700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2081109a0 0x7fc2081a1550 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc20000b830 tx=0x7fc200004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.567+0000 7fc204c00700 1 -- 172.21.15.63:0/3859432920 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200021070 con 0x7fc2081109a0 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.567+0000 7fc204c00700 1 -- 172.21.15.63:0/3859432920 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc200007c80 con 0x7fc2081109a0 2024-05-27T07:34:00.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.567+0000 7fc20d400700 1 -- 172.21.15.63:0/3859432920 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2081a1e80 con 0x7fc2081109a0 2024-05-27T07:34:00.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.567+0000 7fc204c00700 1 -- 172.21.15.63:0/3859432920 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2000052a0 con 0x7fc2081109a0 2024-05-27T07:34:00.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.567+0000 7fc20d400700 1 -- 172.21.15.63:0/3859432920 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2081a2360 con 0x7fc2081109a0 2024-05-27T07:34:00.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.570+0000 7fc204c00700 1 -- 172.21.15.63:0/3859432920 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fc200024070 con 0x7fc2081109a0 2024-05-27T07:34:00.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.572+0000 7fc21d458b80 1 Processor -- start 2024-05-27T07:34:00.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.572+0000 7fc21d458b80 1 -- start start 2024-05-27T07:34:00.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.572+0000 7fc204c00700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1ec0b2b10 0x7fc1ec0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.572+0000 7fc204c00700 1 -- 172.21.15.63:0/3859432920 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fc20000f020 con 0x7fc2081109a0 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55675596fe40 0x556755aedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc207400700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1ec0b2b10 0x7fc1ec0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 0x556755aee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b03700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc21d458b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556755a1ce40 con 0x556755b012e0 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc21d458b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556755a0c9e0 con 0x556755aee100 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc21d458b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556755a10a20 con 0x55675596fe40 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc20ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b03700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc20ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b03700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58040/0 (socket says 172.21.15.63:58040) 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc20ca00700 1 -- 172.21.15.63:0/22591717 learned_addr learned my addr 172.21.15.63:0/22591717 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc207e00700 1 --2- 172.21.15.63:0/22591717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55675596fe40 0x556755aedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.573+0000 7fc207400700 1 --2- 172.21.15.63:0/22591717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 0x556755aee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.574+0000 7fc20ca00700 1 -- 172.21.15.63:0/22591717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55675596fe40 msgr2=0x556755aedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.574+0000 7fc20ca00700 1 --2- 172.21.15.63:0/22591717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55675596fe40 0x556755aedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.574+0000 7fc20ca00700 1 -- 172.21.15.63:0/22591717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 msgr2=0x556755aee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.574+0000 7fc20ca00700 1 --2- 172.21.15.63:0/22591717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 0x556755aee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.574+0000 7fc20ca00700 1 -- 172.21.15.63:0/22591717 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567559d5420 con 0x556755b012e0 2024-05-27T07:34:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.574+0000 7fc207400700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1ec0b2b10 0x7fc1ec0b5020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005950 tx=0x7fc1f400bf80 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.574+0000 7fc20ca00700 1 --2- 172.21.15.63:0/22591717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b03700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc2000a7110 tx=0x7fc200004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7172ea9a9d5010 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.574+0000 7fc1fd600700 1 -- 172.21.15.63:0/22591717 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2000ab090 con 0x556755b012e0 2024-05-27T07:34:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.574+0000 7fc1fd600700 1 -- 172.21.15.63:0/22591717 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2000af440 con 0x556755b012e0 2024-05-27T07:34:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.574+0000 7fc1fd600700 1 -- 172.21.15.63:0/22591717 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2000b3c30 con 0x556755b012e0 2024-05-27T07:34:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.575+0000 7fc21d458b80 1 -- 172.21.15.63:0/22591717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 msgr2=0x556755b03700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.575+0000 7fc21d458b80 1 --2- 172.21.15.63:0/22591717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b03700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc2000a7110 tx=0x7fc200004f90 comp rx=0 tx=0).stop 2024-05-27T07:34:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.576+0000 7fc21d458b80 1 -- 172.21.15.63:0/22591717 shutdown_connections 2024-05-27T07:34:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.576+0000 7fc21d458b80 1 --2- 172.21.15.63:0/22591717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b03700 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.576+0000 7fc21d458b80 1 --2- 172.21.15.63:0/22591717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55675596fe40 0x556755aedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.576+0000 7fc21d458b80 1 --2- 172.21.15.63:0/22591717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 0x556755aee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.576+0000 7fc21d458b80 1 -- 172.21.15.63:0/22591717 >> 172.21.15.63:0/22591717 conn(0x5567559cffc0 msgr2=0x556755ae9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:00.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.576+0000 7fc21d458b80 1 -- 172.21.15.63:0/22591717 shutdown_connections 2024-05-27T07:34:00.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.576+0000 7fc21d458b80 1 -- 172.21.15.63:0/22591717 wait complete. 2024-05-27T07:34:00.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.576+0000 7fc21d458b80 1 -- start start 2024-05-27T07:34:00.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.577+0000 7fc21d458b80 1 Processor -- start 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.577+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55675596fe40 0x556755afe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.577+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 0x556755afe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.577+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755b012e0 0x556755afed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.577+0000 7fc21d458b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556755912fa0 con 0x55675596fe40 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.577+0000 7fc21d458b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5567559ad760 con 0x556755aee100 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.577+0000 7fc21d458b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556755b03f10 con 0x556755b012e0 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.577+0000 7fc21d458b80 10 client.0 _ll_register_callbacks cb 0x5567559b6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc207e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55675596fe40 0x556755afe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc207e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55675596fe40 0x556755afe310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58052/0 (socket says 172.21.15.63:58052) 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc207e00700 1 -- 172.21.15.63:0/148374106 learned_addr learned my addr 172.21.15.63:0/148374106 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc207400700 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 0x556755afe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc20ca00700 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755b012e0 0x556755afed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc207e00700 1 -- 172.21.15.63:0/148374106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755b012e0 msgr2=0x556755afed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:00.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc207e00700 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755b012e0 0x556755afed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc207e00700 1 -- 172.21.15.63:0/148374106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 msgr2=0x556755afe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:00.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc207e00700 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 0x556755afe850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc207e00700 1 -- 172.21.15.63:0/148374106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55675595c8d0 con 0x55675596fe40 2024-05-27T07:34:00.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.578+0000 7fc207400700 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 0x556755afe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:34:00.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.579+0000 7fc207e00700 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55675596fe40 0x556755afe310 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8010860 tx=0x7fc1f8010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9306ff8b14cc6189 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:00.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.579+0000 7fc1f2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:00.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.579+0000 7fc1f2000700 1 -- 172.21.15.63:0/148374106 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800ec50 con 0x55675596fe40 2024-05-27T07:34:00.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.579+0000 7fc1f2000700 1 -- 172.21.15.63:0/148374106 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fc1f800edf0 con 0x55675596fe40 2024-05-27T07:34:00.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.579+0000 7fc21d458b80 1 -- 172.21.15.63:0/148374106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2000af440 con 0x55675596fe40 2024-05-27T07:34:00.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.579+0000 7fc1f2000700 1 -- 172.21.15.63:0/148374106 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc1f8024050 con 0x55675596fe40 2024-05-27T07:34:00.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.579+0000 7fc21d458b80 10 client.15423 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:00.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.579+0000 7fc21d458b80 1 -- 172.21.15.63:0/148374106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567559dd060 con 0x55675596fe40 2024-05-27T07:34:00.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.580+0000 7fc1f2000700 1 -- 172.21.15.63:0/148374106 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2000af440 con 0x55675596fe40 2024-05-27T07:34:00.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.580+0000 7fc1f2000700 1 -- 172.21.15.63:0/148374106 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc1f801b430 con 0x55675596fe40 2024-05-27T07:34:00.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.580+0000 7fc21d458b80 10 client.15423 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:00.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.580+0000 7fc21d458b80 10 client.15423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:00.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.580+0000 7fc21d458b80 20 client.15423 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:00.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.580+0000 7fc21d458b80 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556755b10ee0 0x556755b13360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.580+0000 7fc21d458b80 4 client.15423 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:34:00.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.580+0000 7fc21d458b80 1 -- 172.21.15.63:0/148374106 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556755a1ed90 con 0x556755b10ee0 2024-05-27T07:34:00.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.581+0000 7fc207400700 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556755b10ee0 0x556755b13360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.582+0000 7fc207400700 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556755b10ee0 0x556755b13360 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53a48a0360310be8 server_cookie=86769fc99210a237 in_seq=0 out_seq=0 2024-05-27T07:34:00.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.582+0000 7fc1f2000700 10 client.15423 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:00.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc1f2000700 1 -- 172.21.15.63:0/148374106 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556755a1ed90 con 0x556755b10ee0 2024-05-27T07:34:00.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc1f2000700 10 client.15423 handle_command_reply: tid=0 2024-05-27T07:34:00.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 client.15423 shutdown 2024-05-27T07:34:00.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 -- 172.21.15.63:0/148374106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55675596fe40 msgr2=0x556755afe310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55675596fe40 0x556755afe310 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8010860 tx=0x7fc1f8010c20 comp rx=0 tx=0).stop 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 -- 172.21.15.63:0/148374106 shutdown_connections 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556755b10ee0 0x556755b13360 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755b012e0 0x556755afed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55675596fe40 0x556755afe310 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 --2- 172.21.15.63:0/148374106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556755aee100 0x556755afe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 -- 172.21.15.63:0/148374106 >> 172.21.15.63:0/148374106 conn(0x5567559cffc0 msgr2=0x556755ae8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc1f2000700 0 client.15423 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 -- 172.21.15.63:0/148374106 shutdown_connections 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.584+0000 7fc21d458b80 1 -- 172.21.15.63:0/148374106 wait complete. 2024-05-27T07:34:00.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.585+0000 7fc21d458b80 20 client.15423 trim_cache size 0 max 16384 2024-05-27T07:34:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.607+0000 7fc21d458b80 1 Processor -- start 2024-05-27T07:34:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.607+0000 7fc21d458b80 1 -- start start 2024-05-27T07:34:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.608+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 0x556755af98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.608+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755aee100 0x556755b0cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.608+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b18b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.608+0000 7fc21d458b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081b0890 con 0x556755b012e0 2024-05-27T07:34:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.608+0000 7fc21d458b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081ae120 con 0x55675596fe40 2024-05-27T07:34:00.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.608+0000 7fc21d458b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081abf70 con 0x556755aee100 2024-05-27T07:34:00.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.608+0000 7fc20ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b18b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.608+0000 7fc20ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b18b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58066/0 (socket says 172.21.15.63:58066) 2024-05-27T07:34:00.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.608+0000 7fc20ca00700 1 -- 172.21.15.63:0/1413730081 learned_addr learned my addr 172.21.15.63:0/1413730081 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:00.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc207400700 1 --2- 172.21.15.63:0/1413730081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755aee100 0x556755b0cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc207e00700 1 --2- 172.21.15.63:0/1413730081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 0x556755af98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc20ca00700 1 -- 172.21.15.63:0/1413730081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755aee100 msgr2=0x556755b0cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:00.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc20ca00700 1 --2- 172.21.15.63:0/1413730081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755aee100 0x556755b0cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc20ca00700 1 -- 172.21.15.63:0/1413730081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 msgr2=0x556755af98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:00.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc20ca00700 1 --2- 172.21.15.63:0/1413730081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 0x556755af98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc20ca00700 1 -- 172.21.15.63:0/1413730081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556755b0e8b0 con 0x556755b012e0 2024-05-27T07:34:00.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc207e00700 1 --2- 172.21.15.63:0/1413730081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 0x556755af98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:00.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc20ca00700 1 --2- 172.21.15.63:0/1413730081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b18b20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x556755b03d20 tx=0x7fc2000aaa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a78832d11f6b148 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:00.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc1fcc00700 1 -- 172.21.15.63:0/1413730081 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2000c4070 con 0x556755b012e0 2024-05-27T07:34:00.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc1fcc00700 1 -- 172.21.15.63:0/1413730081 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2000af440 con 0x556755b012e0 2024-05-27T07:34:00.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.609+0000 7fc1fcc00700 1 -- 172.21.15.63:0/1413730081 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2000ab550 con 0x556755b012e0 2024-05-27T07:34:00.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.610+0000 7fc21d458b80 1 -- 172.21.15.63:0/1413730081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 msgr2=0x556755b18b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:00.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.610+0000 7fc21d458b80 1 --2- 172.21.15.63:0/1413730081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b18b20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x556755b03d20 tx=0x7fc2000aaa90 comp rx=0 tx=0).stop 2024-05-27T07:34:00.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.610+0000 7fc21d458b80 1 -- 172.21.15.63:0/1413730081 shutdown_connections 2024-05-27T07:34:00.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.610+0000 7fc21d458b80 1 --2- 172.21.15.63:0/1413730081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755b012e0 0x556755b18b20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.610+0000 7fc21d458b80 1 --2- 172.21.15.63:0/1413730081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 0x556755af98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.610+0000 7fc21d458b80 1 --2- 172.21.15.63:0/1413730081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755aee100 0x556755b0cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.610+0000 7fc21d458b80 1 -- 172.21.15.63:0/1413730081 >> 172.21.15.63:0/1413730081 conn(0x5567559cffc0 msgr2=0x556755af0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:00.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.610+0000 7fc21d458b80 1 -- 172.21.15.63:0/1413730081 shutdown_connections 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.611+0000 7fc21d458b80 1 -- 172.21.15.63:0/1413730081 wait complete. 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.611+0000 7fc21d458b80 1 -- start start 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.611+0000 7fc21d458b80 1 Processor -- start 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.612+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 0x556755b09370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.612+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755aee100 0x556755b098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.612+0000 7fc21d458b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755b012e0 0x556755b04120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.612+0000 7fc21d458b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081ac5e0 con 0x556755aee100 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.612+0000 7fc21d458b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081ada50 con 0x55675596fe40 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.612+0000 7fc21d458b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081accb0 con 0x556755b012e0 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.612+0000 7fc21d458b80 10 client.0 _ll_register_callbacks cb 0x5567559b6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.612+0000 7fc207400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755aee100 0x556755b098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.612+0000 7fc207400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755aee100 0x556755b098b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58076/0 (socket says 172.21.15.63:58076) 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.612+0000 7fc207400700 1 -- 172.21.15.63:0/2367709202 learned_addr learned my addr 172.21.15.63:0/2367709202 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:00.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc207e00700 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 0x556755b09370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc20ca00700 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755b012e0 0x556755b04120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc207400700 1 -- 172.21.15.63:0/2367709202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755b012e0 msgr2=0x556755b04120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:00.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc207400700 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755b012e0 0x556755b04120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc207400700 1 -- 172.21.15.63:0/2367709202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 msgr2=0x556755b09370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:00.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc207400700 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 0x556755b09370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc207400700 1 -- 172.21.15.63:0/2367709202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556755aff100 con 0x556755aee100 2024-05-27T07:34:00.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc207e00700 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 0x556755b09370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:00.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc207400700 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755aee100 0x556755b098b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4019000 tx=0x7fc1f40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e85fc25e2c4c322 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:00.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc1f2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc1f2000700 1 -- 172.21.15.63:0/2367709202 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f402c070 con 0x556755aee100 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc1f2000700 1 -- 172.21.15.63:0/2367709202 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fc1f4027450 con 0x556755aee100 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.613+0000 7fc21d458b80 1 -- 172.21.15.63:0/2367709202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2000af440 con 0x556755aee100 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc1f2000700 1 -- 172.21.15.63:0/2367709202 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc1f4034050 con 0x556755aee100 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc21d458b80 10 client.15429 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc21d458b80 1 -- 172.21.15.63:0/2367709202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556755b7c1e0 con 0x556755aee100 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc1f2000700 1 -- 172.21.15.63:0/2367709202 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2000af440 con 0x556755aee100 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc1f2000700 1 -- 172.21.15.63:0/2367709202 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc1f402f030 con 0x556755aee100 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc21d458b80 10 client.15429 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc21d458b80 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc21d458b80 20 client.15429 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc21d458b80 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556755b7c950 0x556755b7ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:00.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc21d458b80 4 client.15429 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:34:00.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.614+0000 7fc21d458b80 1 -- 172.21.15.63:0/2367709202 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556755a18570 con 0x556755b7c950 2024-05-27T07:34:00.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.615+0000 7fc207e00700 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556755b7c950 0x556755b7ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.616+0000 7fc207e00700 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556755b7c950 0x556755b7ed70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29817fd54401a1a server_cookie=ef101bc3d8e11a03 in_seq=0 out_seq=0 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.616+0000 7fc1f2000700 10 client.15429 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc1f2000700 1 -- 172.21.15.63:0/2367709202 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556755a18570 con 0x556755b7c950 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc1f2000700 10 client.15429 handle_command_reply: tid=0 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 client.15429 shutdown 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 -- 172.21.15.63:0/2367709202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755aee100 msgr2=0x556755b098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755aee100 0x556755b098b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4019000 tx=0x7fc1f40047c0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 -- 172.21.15.63:0/2367709202 shutdown_connections 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556755b7c950 0x556755b7ed70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556755b012e0 0x556755b04120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55675596fe40 0x556755b09370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 --2- 172.21.15.63:0/2367709202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556755aee100 0x556755b098b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc1f2000700 0 client.15429 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:00.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 -- 172.21.15.63:0/2367709202 >> 172.21.15.63:0/2367709202 conn(0x5567559cffc0 msgr2=0x556755aef8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:00.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 -- 172.21.15.63:0/2367709202 shutdown_connections 2024-05-27T07:34:00.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 1 -- 172.21.15.63:0/2367709202 wait complete. 2024-05-27T07:34:00.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.617+0000 7fc21d458b80 20 client.15429 trim_cache size 0 max 16384 2024-05-27T07:34:00.613 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:34:00.613 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:34:00.613 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:34:00.613 INFO:teuthology.orchestra.run.smithi063.stdout: "8cf45254-c799-4756-95a5-11dce3b1c1df": { 2024-05-27T07:34:00.613 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:34:00.614 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "8cf45254-c799-4756-95a5-11dce3b1c1df", 2024-05-27T07:34:00.614 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:34:00.614 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:34:00.614 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:34:00.614 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:34:00.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.618+0000 7fc20d400700 1 -- 172.21.15.63:0/3859432920 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1ec0b2b10 msgr2=0x7fc1ec0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:00.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.618+0000 7fc20d400700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1ec0b2b10 0x7fc1ec0b5020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005950 tx=0x7fc1f400bf80 comp rx=0 tx=0).stop 2024-05-27T07:34:00.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.618+0000 7fc20d400700 1 -- 172.21.15.63:0/3859432920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2081109a0 msgr2=0x7fc2081a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:00.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.618+0000 7fc20d400700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2081109a0 0x7fc2081a1550 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc20000b830 tx=0x7fc200004db0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.619+0000 7fc20d400700 1 -- 172.21.15.63:0/3859432920 shutdown_connections 2024-05-27T07:34:00.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.619+0000 7fc20d400700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1ec0b2b10 0x7fc1ec0b5020 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.619+0000 7fc20d400700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2081109a0 0x7fc2081a1550 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.619+0000 7fc20d400700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc208075d20 0x7fc20819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.619+0000 7fc20d400700 1 --2- 172.21.15.63:0/3859432920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc208076670 0x7fc20819d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:00.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.619+0000 7fc20d400700 1 -- 172.21.15.63:0/3859432920 >> 172.21.15.63:0/3859432920 conn(0x7fc208079d00 msgr2=0x7fc208100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:00.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.619+0000 7fc20d400700 1 -- 172.21.15.63:0/3859432920 shutdown_connections 2024-05-27T07:34:00.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:00.619+0000 7fc20d400700 1 -- 172.21.15.63:0/3859432920 wait complete. 2024-05-27T07:34:00.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:00 smithi086 ceph-mon[34687]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-05-27T07:34:00.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:00 smithi149 ceph-mon[35408]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-05-27T07:34:00.960 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:34:00 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:34:00.772+0000 7f85b8800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:34:00.960 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:34:00 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:34:00.772+0000 7f85b8800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:34:00.960 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:34:00 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:34:00.772+0000 7f85b8800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:34:00.960 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:34:00 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:34:00.772+0000 7f85b8800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:34:00.960 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 07:34:00 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T07:34:00.772+0000 7f85b8800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:34:01.599 INFO:tasks.cephfs.filesystem:scrub status for tag:8cf45254-c799-4756-95a5-11dce3b1c1df - {'path': '/', 'tag': '8cf45254-c799-4756-95a5-11dce3b1c1df', 'options': 'recursive,force'} 2024-05-27T07:34:02.633 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:02 smithi063 ceph-mon[31613]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 76 KiB/s wr, 48 op/s 2024-05-27T07:34:02.633 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:02 smithi063 ceph-mon[31613]: mgrmap e25: x(active, since 26s), standbys: z, y 2024-05-27T07:34:02.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:02 smithi086 ceph-mon[34687]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 76 KiB/s wr, 48 op/s 2024-05-27T07:34:02.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:02 smithi086 ceph-mon[34687]: mgrmap e25: x(active, since 26s), standbys: z, y 2024-05-27T07:34:02.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:02 smithi149 ceph-mon[35408]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 76 KiB/s wr, 48 op/s 2024-05-27T07:34:02.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:02 smithi149 ceph-mon[35408]: mgrmap e25: x(active, since 26s), standbys: z, y 2024-05-27T07:34:04.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:04 smithi086 ceph-mon[34687]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 100 KiB/s wr, 54 op/s 2024-05-27T07:34:04.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:04 smithi149 ceph-mon[35408]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 100 KiB/s wr, 54 op/s 2024-05-27T07:34:04.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:04 smithi063 ceph-mon[31613]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 100 KiB/s wr, 54 op/s 2024-05-27T07:34:06.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:06 smithi086 ceph-mon[34687]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 100 KiB/s wr, 52 op/s 2024-05-27T07:34:06.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:06 smithi149 ceph-mon[35408]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 100 KiB/s wr, 52 op/s 2024-05-27T07:34:06.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:06 smithi063 ceph-mon[31613]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 100 KiB/s wr, 52 op/s 2024-05-27T07:34:08.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:08 smithi086 ceph-mon[34687]: pgmap v19: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-05-27T07:34:08.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:08 smithi149 ceph-mon[35408]: pgmap v19: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-05-27T07:34:08.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:08 smithi063 ceph-mon[31613]: pgmap v19: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-05-27T07:34:10.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:10 smithi086 ceph-mon[34687]: pgmap v20: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 38 op/s 2024-05-27T07:34:10.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:10 smithi149 ceph-mon[35408]: pgmap v20: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 38 op/s 2024-05-27T07:34:10.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:10 smithi063 ceph-mon[31613]: pgmap v20: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 38 op/s 2024-05-27T07:34:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.354+0000 7f03d2c00640 1 -- 172.21.15.63:0/1004182193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03cc075660 msgr2=0x7f03cc075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.354+0000 7f03d2c00640 1 --2- 172.21.15.63:0/1004182193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03cc075660 0x7f03cc075a70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009ca0 tx=0x7f03c002a950 comp rx=0 tx=0).stop 2024-05-27T07:34:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.354+0000 7f03d2c00640 1 -- 172.21.15.63:0/1004182193 shutdown_connections 2024-05-27T07:34:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.354+0000 7f03d2c00640 1 --2- 172.21.15.63:0/1004182193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc1102c0 0x7f03cc112720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.354+0000 7f03d2c00640 1 --2- 172.21.15.63:0/1004182193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03cc10d960 0x7f03cc10fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.354+0000 7f03d2c00640 1 --2- 172.21.15.63:0/1004182193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03cc075660 0x7f03cc075a70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.354+0000 7f03d2c00640 1 -- 172.21.15.63:0/1004182193 >> 172.21.15.63:0/1004182193 conn(0x7f03cc079100 msgr2=0x7f03cc079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.354+0000 7f03d2c00640 1 -- 172.21.15.63:0/1004182193 shutdown_connections 2024-05-27T07:34:11.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.354+0000 7f03d2c00640 1 -- 172.21.15.63:0/1004182193 wait complete. 2024-05-27T07:34:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.356+0000 7f03d2c00640 1 Processor -- start 2024-05-27T07:34:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.356+0000 7f03d2c00640 1 -- start start 2024-05-27T07:34:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.356+0000 7f03d2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03cc075660 0x7f03cc19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.356+0000 7f03d2c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc10d960 0x7f03cc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.356+0000 7f03d2c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03cc1102c0 0x7f03cc1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.356+0000 7f03d2c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc115d90 con 0x7f03cc1102c0 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.356+0000 7f03d2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc115c10 con 0x7f03cc075660 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.356+0000 7f03d2c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc115f10 con 0x7f03cc10d960 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.357+0000 7f03d2200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03cc1102c0 0x7f03cc1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.357+0000 7f03d2200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03cc1102c0 0x7f03cc1a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48568/0 (socket says 172.21.15.63:48568) 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.357+0000 7f03d2200640 1 -- 172.21.15.63:0/4092686223 learned_addr learned my addr 172.21.15.63:0/4092686223 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.357+0000 7f03d1800640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03cc075660 0x7f03cc19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.357+0000 7f03d0e00640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc10d960 0x7f03cc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.357+0000 7f03d2200640 1 -- 172.21.15.63:0/4092686223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc10d960 msgr2=0x7f03cc19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.357+0000 7f03d2200640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc10d960 0x7f03cc19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.357+0000 7f03d2200640 1 -- 172.21.15.63:0/4092686223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03cc075660 msgr2=0x7f03cc19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.357+0000 7f03d2200640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03cc075660 0x7f03cc19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.357+0000 7f03d2200640 1 -- 172.21.15.63:0/4092686223 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc1a15f0 con 0x7f03cc1102c0 2024-05-27T07:34:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.358+0000 7f03d2200640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03cc1102c0 0x7f03cc1a0fb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f03c400e970 tx=0x7f03c400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.364+0000 7f03ca000640 1 -- 172.21.15.63:0/4092686223 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c400ccc0 con 0x7f03cc1102c0 2024-05-27T07:34:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.364+0000 7f03ca000640 1 -- 172.21.15.63:0/4092686223 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03c400ce60 con 0x7f03cc1102c0 2024-05-27T07:34:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.364+0000 7f03ca000640 1 -- 172.21.15.63:0/4092686223 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4010430 con 0x7f03cc1102c0 2024-05-27T07:34:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.364+0000 7f03d2c00640 1 -- 172.21.15.63:0/4092686223 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc1a18e0 con 0x7f03cc1102c0 2024-05-27T07:34:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.364+0000 7f03d2c00640 1 -- 172.21.15.63:0/4092686223 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc1a1de0 con 0x7f03cc1102c0 2024-05-27T07:34:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.368+0000 7f03db22e740 1 Processor -- start 2024-05-27T07:34:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.368+0000 7f03ca000640 1 -- 172.21.15.63:0/4092686223 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f03c4004ef0 con 0x7f03cc1102c0 2024-05-27T07:34:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.369+0000 7f03db22e740 1 -- start start 2024-05-27T07:34:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.369+0000 7f03db22e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323acda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.369+0000 7f03db22e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613237022f0 0x561323acdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.369+0000 7f03db22e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132370a4d0 0x561323ace4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.369+0000 7f03db22e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561323a3eb90 con 0x56132372b1b0 2024-05-27T07:34:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.369+0000 7f03db22e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561323a2e730 con 0x56132370a4d0 2024-05-27T07:34:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.369+0000 7f03db22e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561323a32770 con 0x5613237022f0 2024-05-27T07:34:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.369+0000 7f03d0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323acda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.370+0000 7f03d0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323acda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48570/0 (socket says 172.21.15.63:48570) 2024-05-27T07:34:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.370+0000 7f03d0e00640 1 -- 172.21.15.63:0/54655268 learned_addr learned my addr 172.21.15.63:0/54655268 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.370+0000 7f03d1800640 1 --2- 172.21.15.63:0/54655268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613237022f0 0x561323acdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.370+0000 7f03d0e00640 1 --2- 172.21.15.63:0/54655268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132370a4d0 0x561323ace4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.370+0000 7f03d0e00640 1 -- 172.21.15.63:0/54655268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613237022f0 msgr2=0x561323acdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.370+0000 7f03d0e00640 1 --2- 172.21.15.63:0/54655268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613237022f0 0x561323acdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.370+0000 7f03d0e00640 1 -- 172.21.15.63:0/54655268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132370a4d0 msgr2=0x561323ace4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.370+0000 7f03d0e00640 1 --2- 172.21.15.63:0/54655268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132370a4d0 0x561323ace4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.370+0000 7f03d0e00640 1 -- 172.21.15.63:0/54655268 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613239c78b0 con 0x56132372b1b0 2024-05-27T07:34:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.371+0000 7f03d0e00640 1 --2- 172.21.15.63:0/54655268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323acda60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f03b800bfd0 tx=0x7f03b800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39094c183ea49be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.371+0000 7f03ca000640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03b40b2b80 0x7f03b40b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.371+0000 7f03bea00640 1 -- 172.21.15.63:0/54655268 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8019030 con 0x56132372b1b0 2024-05-27T07:34:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.371+0000 7f03bea00640 1 -- 172.21.15.63:0/54655268 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03b8011950 con 0x56132372b1b0 2024-05-27T07:34:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.371+0000 7f03ca000640 1 -- 172.21.15.63:0/4092686223 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f03c40e2f90 con 0x7f03cc1102c0 2024-05-27T07:34:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.371+0000 7f03bea00640 1 -- 172.21.15.63:0/54655268 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b800e6f0 con 0x56132372b1b0 2024-05-27T07:34:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.371+0000 7f03d1800640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03b40b2b80 0x7f03b40b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.372+0000 7f03db22e740 1 -- 172.21.15.63:0/54655268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 msgr2=0x561323acda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.372+0000 7f03d1800640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03b40b2b80 0x7f03b40b5090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f03c002af40 tx=0x7f03c0005d80 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.372+0000 7f03db22e740 1 --2- 172.21.15.63:0/54655268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323acda60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f03b800bfd0 tx=0x7f03b800b6a0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.372+0000 7f03db22e740 1 -- 172.21.15.63:0/54655268 shutdown_connections 2024-05-27T07:34:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.372+0000 7f03db22e740 1 --2- 172.21.15.63:0/54655268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132370a4d0 0x561323ace4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.372+0000 7f03db22e740 1 --2- 172.21.15.63:0/54655268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613237022f0 0x561323acdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.372+0000 7f03db22e740 1 --2- 172.21.15.63:0/54655268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323acda60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.372+0000 7f03db22e740 1 -- 172.21.15.63:0/54655268 >> 172.21.15.63:0/54655268 conn(0x56132371f300 msgr2=0x561323aca210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.372+0000 7f03db22e740 1 -- 172.21.15.63:0/54655268 shutdown_connections 2024-05-27T07:34:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.372+0000 7f03db22e740 1 -- 172.21.15.63:0/54655268 wait complete. 2024-05-27T07:34:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.373+0000 7f03db22e740 1 -- start start 2024-05-27T07:34:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.373+0000 7f03db22e740 1 Processor -- start 2024-05-27T07:34:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03db22e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323ad7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03db22e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132370a4d0 0x561323ad8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03db22e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323ae3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03db22e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561323714790 con 0x56132372b1b0 2024-05-27T07:34:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03db22e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5613239d6bc0 con 0x5613237022f0 2024-05-27T07:34:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03db22e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561323adab90 con 0x56132370a4d0 2024-05-27T07:34:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03db22e740 10 client.0 _ll_register_callbacks cb 0x5613239b4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03d2200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323ae3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03d2200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323ae3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48576/0 (socket says 172.21.15.63:48576) 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03d2200640 1 -- 172.21.15.63:0/4027788646 learned_addr learned my addr 172.21.15.63:0/4027788646 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03d0e00640 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132370a4d0 0x561323ad8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.374+0000 7f03d1800640 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323ad7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.375+0000 7f03d2200640 1 -- 172.21.15.63:0/4027788646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132370a4d0 msgr2=0x561323ad8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.375+0000 7f03d2200640 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132370a4d0 0x561323ad8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.375+0000 7f03d2200640 1 -- 172.21.15.63:0/4027788646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 msgr2=0x561323ad7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.375+0000 7f03d2200640 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323ad7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.375+0000 7f03d2200640 1 -- 172.21.15.63:0/4027788646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561323743210 con 0x56132372b1b0 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.375+0000 7f03d2200640 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323ae3ec0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f03c40a9c50 tx=0x7f03c4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c0a66a40dc656d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.375+0000 7f03b3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.375+0000 7f03b3400640 1 -- 172.21.15.63:0/4027788646 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c40aa3c0 con 0x56132372b1b0 2024-05-27T07:34:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.375+0000 7f03b3400640 1 -- 172.21.15.63:0/4027788646 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f03c40aa130 con 0x56132372b1b0 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.375+0000 7f03db22e740 1 -- 172.21.15.63:0/4027788646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b8011950 con 0x56132372b1b0 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03b3400640 1 -- 172.21.15.63:0/4027788646 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03c40a9cc0 con 0x56132372b1b0 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03db22e740 10 client.15447 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03db22e740 1 -- 172.21.15.63:0/4027788646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56132380f620 con 0x56132372b1b0 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03b3400640 1 -- 172.21.15.63:0/4027788646 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03b8011950 con 0x56132372b1b0 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03b3400640 1 -- 172.21.15.63:0/4027788646 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f03c40e0440 con 0x56132372b1b0 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03db22e740 10 client.15447 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03db22e740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03db22e740 20 client.15447 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03db22e740 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613236e9d80 0x561323ae9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03db22e740 4 client.15447 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:34:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.376+0000 7f03db22e740 1 -- 172.21.15.63:0/4027788646 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561323a40ae0 con 0x5613236e9d80 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.377+0000 7f03d1800640 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613236e9d80 0x561323ae9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.378+0000 7f03d1800640 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613236e9d80 0x561323ae9fa0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36caa015867e0237 server_cookie=f561b7666f1c44f7 in_seq=0 out_seq=0 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.378+0000 7f03b3400640 10 client.15447 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03b3400640 1 -- 172.21.15.63:0/4027788646 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561323a40ae0 con 0x5613236e9d80 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03b3400640 10 client.15447 handle_command_reply: tid=0 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 client.15447 shutdown 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 -- 172.21.15.63:0/4027788646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 msgr2=0x561323ae3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323ae3ec0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f03c40a9c50 tx=0x7f03c4005710 comp rx=0 tx=0).stop 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 -- 172.21.15.63:0/4027788646 shutdown_connections 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613236e9d80 0x561323ae9fa0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132372b1b0 0x561323ae3ec0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03b3400640 0 client.15447 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132370a4d0 0x561323ad8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 --2- 172.21.15.63:0/4027788646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323ad7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 -- 172.21.15.63:0/4027788646 >> 172.21.15.63:0/4027788646 conn(0x56132371f300 msgr2=0x561323acd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 -- 172.21.15.63:0/4027788646 shutdown_connections 2024-05-27T07:34:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.380+0000 7f03db22e740 1 -- 172.21.15.63:0/4027788646 wait complete. 2024-05-27T07:34:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.381+0000 7f03db22e740 20 client.15447 trim_cache size 0 max 16384 2024-05-27T07:34:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.398+0000 7f03db22e740 1 Processor -- start 2024-05-27T07:34:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.398+0000 7f03db22e740 1 -- start start 2024-05-27T07:34:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.398+0000 7f03db22e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323ac8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.398+0000 7f03db22e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323aef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.398+0000 7f03db22e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132372b1b0 0x561323aefca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.398+0000 7f03db22e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b5a30 con 0x56132370a4d0 2024-05-27T07:34:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.398+0000 7f03db22e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b3300 con 0x5613237022f0 2024-05-27T07:34:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.398+0000 7f03db22e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b1150 con 0x56132372b1b0 2024-05-27T07:34:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.398+0000 7f03d0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323aef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.399+0000 7f03d0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323aef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48578/0 (socket says 172.21.15.63:48578) 2024-05-27T07:34:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.399+0000 7f03d0e00640 1 -- 172.21.15.63:0/1228157382 learned_addr learned my addr 172.21.15.63:0/1228157382 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.399+0000 7f03d2200640 1 --2- 172.21.15.63:0/1228157382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132372b1b0 0x561323aefca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.399+0000 7f03d1800640 1 --2- 172.21.15.63:0/1228157382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323ac8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.399+0000 7f03d0e00640 1 -- 172.21.15.63:0/1228157382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132372b1b0 msgr2=0x561323aefca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.399+0000 7f03d0e00640 1 --2- 172.21.15.63:0/1228157382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132372b1b0 0x561323aefca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.399+0000 7f03d0e00640 1 -- 172.21.15.63:0/1228157382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 msgr2=0x561323ac8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.399+0000 7f03d0e00640 1 --2- 172.21.15.63:0/1228157382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323ac8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.399+0000 7f03d0e00640 1 -- 172.21.15.63:0/1228157382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561323ace880 con 0x56132370a4d0 2024-05-27T07:34:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.400+0000 7f03d0e00640 1 --2- 172.21.15.63:0/1228157382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323aef760 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f03b8013ec0 tx=0x7f03b800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf53cf25d1728bc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.400+0000 7f03be000640 1 -- 172.21.15.63:0/1228157382 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8019040 con 0x56132370a4d0 2024-05-27T07:34:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.400+0000 7f03be000640 1 -- 172.21.15.63:0/1228157382 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03b8005590 con 0x56132370a4d0 2024-05-27T07:34:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.400+0000 7f03be000640 1 -- 172.21.15.63:0/1228157382 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8004b10 con 0x56132370a4d0 2024-05-27T07:34:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.401+0000 7f03db22e740 1 -- 172.21.15.63:0/1228157382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 msgr2=0x561323aef760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.401+0000 7f03db22e740 1 --2- 172.21.15.63:0/1228157382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323aef760 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f03b8013ec0 tx=0x7f03b800bde0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.401+0000 7f03db22e740 1 -- 172.21.15.63:0/1228157382 shutdown_connections 2024-05-27T07:34:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.401+0000 7f03db22e740 1 --2- 172.21.15.63:0/1228157382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132372b1b0 0x561323aefca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.401+0000 7f03db22e740 1 --2- 172.21.15.63:0/1228157382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323aef760 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.401+0000 7f03db22e740 1 --2- 172.21.15.63:0/1228157382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323ac8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.401+0000 7f03db22e740 1 -- 172.21.15.63:0/1228157382 >> 172.21.15.63:0/1228157382 conn(0x56132371f300 msgr2=0x561323ac8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.401+0000 7f03db22e740 1 -- 172.21.15.63:0/1228157382 shutdown_connections 2024-05-27T07:34:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.401+0000 7f03db22e740 1 -- 172.21.15.63:0/1228157382 wait complete. 2024-05-27T07:34:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.402+0000 7f03db22e740 1 -- start start 2024-05-27T07:34:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.402+0000 7f03db22e740 1 Processor -- start 2024-05-27T07:34:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.402+0000 7f03db22e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323adfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.402+0000 7f03db22e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323ae02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.402+0000 7f03db22e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132372b1b0 0x561323add270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.402+0000 7f03db22e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b17c0 con 0x56132370a4d0 2024-05-27T07:34:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.402+0000 7f03db22e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b2c30 con 0x5613237022f0 2024-05-27T07:34:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.402+0000 7f03db22e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b1e90 con 0x56132372b1b0 2024-05-27T07:34:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.402+0000 7f03db22e740 10 client.0 _ll_register_callbacks cb 0x7f03c40a5c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.403+0000 7f03d0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323ae02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.403+0000 7f03d0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323ae02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48582/0 (socket says 172.21.15.63:48582) 2024-05-27T07:34:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.403+0000 7f03d0e00640 1 -- 172.21.15.63:0/1618867653 learned_addr learned my addr 172.21.15.63:0/1618867653 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.403+0000 7f03d1800640 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323adfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.403+0000 7f03d2200640 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132372b1b0 0x561323add270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.403+0000 7f03d0e00640 1 -- 172.21.15.63:0/1618867653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132372b1b0 msgr2=0x561323add270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.403+0000 7f03d0e00640 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132372b1b0 0x561323add270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.403+0000 7f03d0e00640 1 -- 172.21.15.63:0/1618867653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 msgr2=0x561323adfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.403+0000 7f03d0e00640 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323adfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.403+0000 7f03d0e00640 1 -- 172.21.15.63:0/1618867653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561323add8b0 con 0x56132370a4d0 2024-05-27T07:34:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.404+0000 7f03d0e00640 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323ae02d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f03b80058c0 tx=0x7f03b800b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b0ded7672b264b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.404+0000 7f03b3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.404+0000 7f03b3400640 1 -- 172.21.15.63:0/1618867653 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8033020 con 0x56132370a4d0 2024-05-27T07:34:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.404+0000 7f03b3400640 1 -- 172.21.15.63:0/1618867653 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f03b8004790 con 0x56132370a4d0 2024-05-27T07:34:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.404+0000 7f03db22e740 1 -- 172.21.15.63:0/1618867653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b8005590 con 0x56132370a4d0 2024-05-27T07:34:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.404+0000 7f03b3400640 1 -- 172.21.15.63:0/1618867653 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03b803e050 con 0x56132370a4d0 2024-05-27T07:34:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.404+0000 7f03db22e740 10 client.15453 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.404+0000 7f03db22e740 1 -- 172.21.15.63:0/1618867653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561323b5a720 con 0x56132370a4d0 2024-05-27T07:34:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.405+0000 7f03b3400640 1 -- 172.21.15.63:0/1618867653 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03b8005590 con 0x56132370a4d0 2024-05-27T07:34:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.405+0000 7f03b3400640 1 -- 172.21.15.63:0/1618867653 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f03b8011970 con 0x56132370a4d0 2024-05-27T07:34:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.405+0000 7f03db22e740 10 client.15453 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.405+0000 7f03db22e740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.405+0000 7f03db22e740 20 client.15453 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.405+0000 7f03db22e740 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613236bf1c0 0x561323b5cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.405+0000 7f03db22e740 4 client.15453 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:34:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.405+0000 7f03db22e740 1 -- 172.21.15.63:0/1618867653 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561323a3a2c0 con 0x5613236bf1c0 2024-05-27T07:34:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.406+0000 7f03d1800640 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613236bf1c0 0x561323b5cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.407+0000 7f03d1800640 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613236bf1c0 0x561323b5cc00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d4ce129ef9c12a server_cookie=7eae712a62573ff9 in_seq=0 out_seq=0 2024-05-27T07:34:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.407+0000 7f03b3400640 10 client.15453 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03b3400640 1 -- 172.21.15.63:0/1618867653 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561323a3a2c0 con 0x5613236bf1c0 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03b3400640 10 client.15453 handle_command_reply: tid=0 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03db22e740 1 client.15453 shutdown 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03db22e740 1 -- 172.21.15.63:0/1618867653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 msgr2=0x561323ae02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03db22e740 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323ae02d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f03b80058c0 tx=0x7f03b800b6b0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03db22e740 1 -- 172.21.15.63:0/1618867653 shutdown_connections 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03db22e740 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613236bf1c0 0x561323b5cc00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03db22e740 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132372b1b0 0x561323add270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03db22e740 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132370a4d0 0x561323ae02d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03b3400640 0 client.15453 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03db22e740 1 --2- 172.21.15.63:0/1618867653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613237022f0 0x561323adfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.440+0000 7f03db22e740 1 -- 172.21.15.63:0/1618867653 >> 172.21.15.63:0/1618867653 conn(0x56132371f300 msgr2=0x561323aca1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.441+0000 7f03db22e740 1 -- 172.21.15.63:0/1618867653 shutdown_connections 2024-05-27T07:34:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.441+0000 7f03db22e740 1 -- 172.21.15.63:0/1618867653 wait complete. 2024-05-27T07:34:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.441+0000 7f03db22e740 20 client.15453 trim_cache size 0 max 16384 2024-05-27T07:34:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:34:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:34:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:34:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.442+0000 7f03d2c00640 1 -- 172.21.15.63:0/4092686223 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03b40b2b80 msgr2=0x7f03b40b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.442+0000 7f03d2c00640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03b40b2b80 0x7f03b40b5090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f03c002af40 tx=0x7f03c0005d80 comp rx=0 tx=0).stop 2024-05-27T07:34:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.442+0000 7f03d2c00640 1 -- 172.21.15.63:0/4092686223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03cc1102c0 msgr2=0x7f03cc1a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.442+0000 7f03d2c00640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03cc1102c0 0x7f03cc1a0fb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f03c400e970 tx=0x7f03c400ed70 comp rx=0 tx=0).stop 2024-05-27T07:34:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.442+0000 7f03d2c00640 1 -- 172.21.15.63:0/4092686223 shutdown_connections 2024-05-27T07:34:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.442+0000 7f03d2c00640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03b40b2b80 0x7f03b40b5090 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.442+0000 7f03d2c00640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03cc1102c0 0x7f03cc1a0fb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.442+0000 7f03d2c00640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03cc10d960 0x7f03cc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.442+0000 7f03d2c00640 1 --2- 172.21.15.63:0/4092686223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03cc075660 0x7f03cc19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.442+0000 7f03d2c00640 1 -- 172.21.15.63:0/4092686223 >> 172.21.15.63:0/4092686223 conn(0x7f03cc079100 msgr2=0x7f03cc1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.443+0000 7f03d2c00640 1 -- 172.21.15.63:0/4092686223 shutdown_connections 2024-05-27T07:34:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:11.443+0000 7f03d2c00640 1 -- 172.21.15.63:0/4092686223 wait complete. 2024-05-27T07:34:11.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:11 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:34:11.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:11 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:34:11.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:11 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:34:12.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:12 smithi086 ceph-mon[34687]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 38 op/s 2024-05-27T07:34:12.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:12 smithi149 ceph-mon[35408]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 38 op/s 2024-05-27T07:34:12.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:12 smithi063 ceph-mon[31613]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 38 op/s 2024-05-27T07:34:13.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:13 smithi086 ceph-mon[34687]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-05-27T07:34:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:13 smithi149 ceph-mon[35408]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-05-27T07:34:13.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:13 smithi063 ceph-mon[31613]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-05-27T07:34:16.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:16 smithi086 ceph-mon[34687]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-27T07:34:16.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:16 smithi149 ceph-mon[35408]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-27T07:34:16.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:16 smithi063 ceph-mon[31613]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-27T07:34:18.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:18 smithi086 ceph-mon[34687]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-05-27T07:34:18.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:18 smithi149 ceph-mon[35408]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-05-27T07:34:18.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:18 smithi063 ceph-mon[31613]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-05-27T07:34:20.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:20 smithi086 ceph-mon[34687]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-27T07:34:20.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:20 smithi149 ceph-mon[35408]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-27T07:34:20.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:20 smithi063 ceph-mon[31613]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-27T07:34:21.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:21 smithi086 ceph-mon[34687]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-27T07:34:21.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:21 smithi149 ceph-mon[35408]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-27T07:34:21.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:21 smithi063 ceph-mon[31613]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-27T07:34:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.681+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1864148794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b810c170 msgr2=0x7fa4b8105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.681+0000 7fa4b7e00640 1 --2- 172.21.15.63:0/1864148794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b810c170 0x7fa4b8105db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009900 tx=0x7fa4a002a410 comp rx=0 tx=0).stop 2024-05-27T07:34:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.681+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1864148794 shutdown_connections 2024-05-27T07:34:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.681+0000 7fa4b7e00640 1 --2- 172.21.15.63:0/1864148794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b8106420 0x7fa4b81127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.681+0000 7fa4b7e00640 1 --2- 172.21.15.63:0/1864148794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b810c170 0x7fa4b8105db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.681+0000 7fa4b7e00640 1 --2- 172.21.15.63:0/1864148794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4b810bcd0 0x7fa4b8105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.681+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1864148794 >> 172.21.15.63:0/1864148794 conn(0x7fa4b81013d0 msgr2=0x7fa4b81037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.681+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1864148794 shutdown_connections 2024-05-27T07:34:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.681+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1864148794 wait complete. 2024-05-27T07:34:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.683+0000 7fa4b7e00640 1 Processor -- start 2024-05-27T07:34:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.683+0000 7fa4b7e00640 1 -- start start 2024-05-27T07:34:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4b7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4b8106420 0x7fa4b81a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4b7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b810bcd0 0x7fa4b81a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4b7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b810c170 0x7fa4b819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4b7e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8115da0 con 0x7fa4b810c170 2024-05-27T07:34:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4b7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8115c20 con 0x7fa4b8106420 2024-05-27T07:34:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4b7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8115f20 con 0x7fa4b810bcd0 2024-05-27T07:34:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4b7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b810c170 0x7fa4b819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4b7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b810c170 0x7fa4b819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35522/0 (socket says 172.21.15.63:35522) 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4b7400640 1 -- 172.21.15.63:0/1189275164 learned_addr learned my addr 172.21.15.63:0/1189275164 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4b8106420 0x7fa4b81a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.684+0000 7fa4ae000640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b810bcd0 0x7fa4b81a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4b7400640 1 -- 172.21.15.63:0/1189275164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b810bcd0 msgr2=0x7fa4b81a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4b7400640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b810bcd0 0x7fa4b81a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4b7400640 1 -- 172.21.15.63:0/1189275164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4b8106420 msgr2=0x7fa4b81a2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4b7400640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4b8106420 0x7fa4b81a2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4b7400640 1 -- 172.21.15.63:0/1189275164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b819ccc0 con 0x7fa4b810c170 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4ae000640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b810bcd0 0x7fa4b81a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4b7400640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b810c170 0x7fa4b819c680 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c970 tx=0x7fa4b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4afe00640 1 -- 172.21.15.63:0/1189275164 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0007bb0 con 0x7fa4b810c170 2024-05-27T07:34:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4afe00640 1 -- 172.21.15.63:0/1189275164 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4b0007d50 con 0x7fa4b810c170 2024-05-27T07:34:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1189275164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b819cfb0 con 0x7fa4b810c170 2024-05-27T07:34:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4afe00640 1 -- 172.21.15.63:0/1189275164 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0018660 con 0x7fa4b810c170 2024-05-27T07:34:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.685+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1189275164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81a92a0 con 0x7fa4b810c170 2024-05-27T07:34:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.689+0000 7fa4c45a0740 1 Processor -- start 2024-05-27T07:34:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.689+0000 7fa4afe00640 1 -- 172.21.15.63:0/1189275164 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7fa4b000f970 con 0x7fa4b810c170 2024-05-27T07:34:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.689+0000 7fa4c45a0740 1 -- start start 2024-05-27T07:34:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.690+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb9c1b0 0x55900ff3ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.690+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff3efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.690+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 0x55900ff3f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.690+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55900feafb90 con 0x55900fb732f0 2024-05-27T07:34:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.690+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55900fe9f730 con 0x55900fb7b4d0 2024-05-27T07:34:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.690+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55900fea3770 con 0x55900fb9c1b0 2024-05-27T07:34:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.690+0000 7fa4b6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff3efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.690+0000 7fa4b6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff3efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35538/0 (socket says 172.21.15.63:35538) 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.690+0000 7fa4b6a00640 1 -- 172.21.15.63:0/2735196193 learned_addr learned my addr 172.21.15.63:0/2735196193 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.690+0000 7fa4ae000640 1 --2- 172.21.15.63:0/2735196193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb9c1b0 0x55900ff3ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.691+0000 7fa4ae000640 1 --2- 172.21.15.63:0/2735196193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 0x55900ff3f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.691+0000 7fa4b6a00640 1 -- 172.21.15.63:0/2735196193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb9c1b0 msgr2=0x55900ff3ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.691+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/2735196193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb9c1b0 0x55900ff3ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.691+0000 7fa4b6a00640 1 -- 172.21.15.63:0/2735196193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 msgr2=0x55900ff3f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.691+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/2735196193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 0x55900ff3f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.691+0000 7fa4b6a00640 1 -- 172.21.15.63:0/2735196193 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900fe388b0 con 0x55900fb732f0 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.691+0000 7fa4afe00640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa49c0b2b80 0x7fa49c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.691+0000 7fa4afe00640 1 -- 172.21.15.63:0/1189275164 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fa4b00e2cb0 con 0x7fa4b810c170 2024-05-27T07:34:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.691+0000 7fa4ae000640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa49c0b2b80 0x7fa49c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.691+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/2735196193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff3efa0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009ed0 tx=0x7fa4a8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112c0e50020f6664 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.692+0000 7fa4a7400640 1 -- 172.21.15.63:0/2735196193 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8016070 con 0x55900fb732f0 2024-05-27T07:34:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.692+0000 7fa4a7400640 1 -- 172.21.15.63:0/2735196193 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a8004030 con 0x55900fb732f0 2024-05-27T07:34:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.692+0000 7fa4a7400640 1 -- 172.21.15.63:0/2735196193 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80126b0 con 0x55900fb732f0 2024-05-27T07:34:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.693+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2735196193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 msgr2=0x55900ff3efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.693+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/2735196193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff3efa0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009ed0 tx=0x7fa4a8010740 comp rx=0 tx=0).stop 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.693+0000 7fa4ae000640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa49c0b2b80 0x7fa49c0b5090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009230 tx=0x7fa4a0005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.693+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2735196193 shutdown_connections 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.693+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/2735196193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 0x55900ff3f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.693+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/2735196193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff3efa0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.693+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/2735196193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb9c1b0 0x55900ff3ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.693+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2735196193 >> 172.21.15.63:0/2735196193 conn(0x55900fb90300 msgr2=0x55900ff3b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.693+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2735196193 shutdown_connections 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.693+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2735196193 wait complete. 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.694+0000 7fa4c45a0740 1 -- start start 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.694+0000 7fa4c45a0740 1 Processor -- start 2024-05-27T07:34:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.694+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff48c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.694+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 0x55900ff49150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.694+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb9c1b0 0x55900ff54ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.694+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55900fb85790 con 0x55900fb732f0 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.694+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55900fe47bc0 con 0x55900fb7b4d0 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.694+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55900ff4bb90 con 0x55900fb9c1b0 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4c45a0740 10 client.0 _ll_register_callbacks cb 0x55900fe25400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4b6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff48c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4b6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff48c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35550/0 (socket says 172.21.15.63:35550) 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4b6a00640 1 -- 172.21.15.63:0/787556868 learned_addr learned my addr 172.21.15.63:0/787556868 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4b7400640 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb9c1b0 0x55900ff54ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4ae000640 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 0x55900ff49150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4b6a00640 1 -- 172.21.15.63:0/787556868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb9c1b0 msgr2=0x55900ff54ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb9c1b0 0x55900ff54ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4b6a00640 1 -- 172.21.15.63:0/787556868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 msgr2=0x55900ff49150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 0x55900ff49150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4b6a00640 1 -- 172.21.15.63:0/787556868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900fbb4210 con 0x55900fb732f0 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.695+0000 7fa4ae000640 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 0x55900ff49150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.696+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff48c10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8019000 tx=0x7fa4a8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96414a6b57f6eb3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.696+0000 7fa49be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.696+0000 7fa49be00640 1 -- 172.21.15.63:0/787556868 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8028060 con 0x55900fb732f0 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.696+0000 7fa49be00640 1 -- 172.21.15.63:0/787556868 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fa4a8004c50 con 0x55900fb732f0 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.696+0000 7fa4c45a0740 1 -- 172.21.15.63:0/787556868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a8004030 con 0x55900fb732f0 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.696+0000 7fa49be00640 1 -- 172.21.15.63:0/787556868 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4a802f050 con 0x55900fb732f0 2024-05-27T07:34:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.696+0000 7fa4c45a0740 10 client.15465 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.696+0000 7fa4c45a0740 1 -- 172.21.15.63:0/787556868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55900fc80620 con 0x55900fb732f0 2024-05-27T07:34:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.697+0000 7fa49be00640 1 -- 172.21.15.63:0/787556868 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a8004030 con 0x55900fb732f0 2024-05-27T07:34:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.697+0000 7fa49be00640 1 -- 172.21.15.63:0/787556868 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa4a800f040 con 0x55900fb732f0 2024-05-27T07:34:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.697+0000 7fa4c45a0740 10 client.15465 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.697+0000 7fa4c45a0740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.697+0000 7fa4c45a0740 20 client.15465 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.697+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55900fb5ad80 0x55900ff5ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.697+0000 7fa4c45a0740 4 client.15465 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:34:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.697+0000 7fa4c45a0740 1 -- 172.21.15.63:0/787556868 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55900feb1ae0 con 0x55900fb5ad80 2024-05-27T07:34:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.698+0000 7fa4ae000640 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55900fb5ad80 0x55900ff5ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.699+0000 7fa4ae000640 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55900fb5ad80 0x55900ff5ad10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=199ac3aa18dce90e server_cookie=3ccbc87cda80accb in_seq=0 out_seq=0 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.699+0000 7fa49be00640 10 client.15465 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.700+0000 7fa49be00640 1 -- 172.21.15.63:0/787556868 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55900feb1ae0 con 0x55900fb5ad80 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.700+0000 7fa49be00640 10 client.15465 handle_command_reply: tid=0 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 client.15465 shutdown 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 -- 172.21.15.63:0/787556868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 msgr2=0x55900ff48c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff48c10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8019000 tx=0x7fa4a8004410 comp rx=0 tx=0).stop 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 -- 172.21.15.63:0/787556868 shutdown_connections 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55900fb5ad80 0x55900ff5ad10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb9c1b0 0x55900ff54ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb7b4d0 0x55900ff49150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/787556868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff48c10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa49be00640 0 client.15465 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 -- 172.21.15.63:0/787556868 >> 172.21.15.63:0/787556868 conn(0x55900fb90300 msgr2=0x55900ff3e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 -- 172.21.15.63:0/787556868 shutdown_connections 2024-05-27T07:34:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 1 -- 172.21.15.63:0/787556868 wait complete. 2024-05-27T07:34:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.701+0000 7fa4c45a0740 20 client.15465 trim_cache size 0 max 16384 2024-05-27T07:34:24.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:24 smithi086 ceph-mon[34687]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-27T07:34:24.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:24 smithi149 ceph-mon[35408]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-27T07:34:24.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:24 smithi063 ceph-mon[31613]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-27T07:34:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4c45a0740 1 Processor -- start 2024-05-27T07:34:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4c45a0740 1 -- start start 2024-05-27T07:34:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff54e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb7b4d0 0x55900ff45e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 0x55900ff46390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81b5ae0 con 0x55900fb732f0 2024-05-27T07:34:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81b33b0 con 0x55900fb9c1b0 2024-05-27T07:34:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81b1200 con 0x55900fb7b4d0 2024-05-27T07:34:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4b6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff54e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4b6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff54e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35560/0 (socket says 172.21.15.63:35560) 2024-05-27T07:34:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.719+0000 7fa4b6a00640 1 -- 172.21.15.63:0/3592175340 learned_addr learned my addr 172.21.15.63:0/3592175340 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.720+0000 7fa4b7400640 1 --2- 172.21.15.63:0/3592175340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 0x55900ff46390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.720+0000 7fa4b6a00640 1 -- 172.21.15.63:0/3592175340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb7b4d0 msgr2=0x55900ff45e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.720+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/3592175340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb7b4d0 0x55900ff45e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.720+0000 7fa4b6a00640 1 -- 172.21.15.63:0/3592175340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 msgr2=0x55900ff46390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.720+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/3592175340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 0x55900ff46390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.720+0000 7fa4b6a00640 1 -- 172.21.15.63:0/3592175340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900ff3f810 con 0x55900fb732f0 2024-05-27T07:34:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.720+0000 7fa4b7400640 1 --2- 172.21.15.63:0/3592175340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 0x55900ff46390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.720+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/3592175340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff54e30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8010780 tx=0x7fa4a8005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b17bac37987446 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.721+0000 7fa4a6a00640 1 -- 172.21.15.63:0/3592175340 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8028070 con 0x55900fb732f0 2024-05-27T07:34:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.721+0000 7fa4a6a00640 1 -- 172.21.15.63:0/3592175340 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a8004860 con 0x55900fb732f0 2024-05-27T07:34:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.721+0000 7fa4a6a00640 1 -- 172.21.15.63:0/3592175340 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8016070 con 0x55900fb732f0 2024-05-27T07:34:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 -- 172.21.15.63:0/3592175340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 msgr2=0x55900ff54e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/3592175340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff54e30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8010780 tx=0x7fa4a8005cf0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 -- 172.21.15.63:0/3592175340 shutdown_connections 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/3592175340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 0x55900ff46390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/3592175340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb7b4d0 0x55900ff45e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/3592175340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb732f0 0x55900ff54e30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 -- 172.21.15.63:0/3592175340 >> 172.21.15.63:0/3592175340 conn(0x55900fb90300 msgr2=0x55900ff48ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 -- 172.21.15.63:0/3592175340 shutdown_connections 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 -- 172.21.15.63:0/3592175340 wait complete. 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 -- start start 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.722+0000 7fa4c45a0740 1 Processor -- start 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.723+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb732f0 0x55900ff5cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.723+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb7b4d0 0x55900ff5d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.723+0000 7fa4c45a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 0x55900ff5d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.723+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81b1870 con 0x55900fb7b4d0 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.723+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81b2ce0 con 0x55900fb9c1b0 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.723+0000 7fa4c45a0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81b1f40 con 0x55900fb732f0 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.723+0000 7fa4c45a0740 10 client.0 _ll_register_callbacks cb 0x7fa4a8009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.723+0000 7fa4ae000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb7b4d0 0x55900ff5d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.723+0000 7fa4ae000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb7b4d0 0x55900ff5d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35562/0 (socket says 172.21.15.63:35562) 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.723+0000 7fa4ae000640 1 -- 172.21.15.63:0/2446696296 learned_addr learned my addr 172.21.15.63:0/2446696296 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb732f0 0x55900ff5cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa4b7400640 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 0x55900ff5d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa4ae000640 1 -- 172.21.15.63:0/2446696296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb732f0 msgr2=0x55900ff5cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa4ae000640 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb732f0 0x55900ff5cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa4ae000640 1 -- 172.21.15.63:0/2446696296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 msgr2=0x55900ff5d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa4ae000640 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 0x55900ff5d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa4ae000640 1 -- 172.21.15.63:0/2446696296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900ffc1310 con 0x55900fb7b4d0 2024-05-27T07:34:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa4b7400640 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 0x55900ff5d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa4ae000640 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb7b4d0 0x55900ff5d270 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0004650 tx=0x7fa4a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2247ff56c3f143 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa49be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa49be00640 1 -- 172.21.15.63:0/2446696296 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0035040 con 0x55900fb7b4d0 2024-05-27T07:34:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2446696296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a8004860 con 0x55900fb7b4d0 2024-05-27T07:34:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.724+0000 7fa49be00640 1 -- 172.21.15.63:0/2446696296 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fa4a00035f0 con 0x55900fb7b4d0 2024-05-27T07:34:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa49be00640 1 -- 172.21.15.63:0/2446696296 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4a0062050 con 0x55900fb7b4d0 2024-05-27T07:34:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa4c45a0740 10 client.15471 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2446696296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55900ffc25a0 con 0x55900fb7b4d0 2024-05-27T07:34:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa49be00640 1 -- 172.21.15.63:0/2446696296 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4a8004860 con 0x55900fb7b4d0 2024-05-27T07:34:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa49be00640 1 -- 172.21.15.63:0/2446696296 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa4a003a050 con 0x55900fb7b4d0 2024-05-27T07:34:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa4c45a0740 10 client.15471 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa4c45a0740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa4c45a0740 20 client.15471 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55900fb301c0 0x55900ffc4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa4c45a0740 4 client.15471 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:34:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.725+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2446696296 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55900feab2c0 con 0x55900fb301c0 2024-05-27T07:34:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.726+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55900fb301c0 0x55900ffc4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.727+0000 7fa4b6a00640 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55900fb301c0 0x55900ffc4d10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=191f9ebef15ffba2 server_cookie=f373ae9711cb8881 in_seq=0 out_seq=0 2024-05-27T07:34:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.727+0000 7fa49be00640 10 client.15471 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa49be00640 1 -- 172.21.15.63:0/2446696296 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55900feab2c0 con 0x55900fb301c0 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa49be00640 10 client.15471 handle_command_reply: tid=0 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa4c45a0740 1 client.15471 shutdown 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2446696296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb7b4d0 msgr2=0x55900ff5d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb7b4d0 0x55900ff5d270 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0004650 tx=0x7fa4a0009260 comp rx=0 tx=0).stop 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2446696296 shutdown_connections 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55900fb301c0 0x55900ffc4d10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55900fb9c1b0 0x55900ff5d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55900fb7b4d0 0x55900ff5d270 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa4c45a0740 1 --2- 172.21.15.63:0/2446696296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55900fb732f0 0x55900ff5cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa49be00640 0 client.15471 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.737+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2446696296 >> 172.21.15.63:0/2446696296 conn(0x55900fb90300 msgr2=0x55900ff433f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.738+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2446696296 shutdown_connections 2024-05-27T07:34:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.738+0000 7fa4c45a0740 1 -- 172.21.15.63:0/2446696296 wait complete. 2024-05-27T07:34:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.738+0000 7fa4c45a0740 20 client.15471 trim_cache size 0 max 16384 2024-05-27T07:34:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:34:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:34:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:34:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1189275164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa49c0b2b80 msgr2=0x7fa49c0b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa49c0b2b80 0x7fa49c0b5090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009230 tx=0x7fa4a0005fd0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1189275164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b810c170 msgr2=0x7fa4b819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b810c170 0x7fa4b819c680 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c970 tx=0x7fa4b000ce30 comp rx=0 tx=0).stop 2024-05-27T07:34:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1189275164 shutdown_connections 2024-05-27T07:34:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa49c0b2b80 0x7fa49c0b5090 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4b810c170 0x7fa4b819c680 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b810bcd0 0x7fa4b81a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 --2- 172.21.15.63:0/1189275164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4b8106420 0x7fa4b81a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1189275164 >> 172.21.15.63:0/1189275164 conn(0x7fa4b81013d0 msgr2=0x7fa4b81117c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1189275164 shutdown_connections 2024-05-27T07:34:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:24.739+0000 7fa4b7e00640 1 -- 172.21.15.63:0/1189275164 wait complete. 2024-05-27T07:34:26.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:26 smithi086 ceph-mon[34687]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-27T07:34:26.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:26 smithi149 ceph-mon[35408]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-27T07:34:26.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:26 smithi063 ceph-mon[31613]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-27T07:34:27.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:27 smithi086 ceph-mon[34687]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-27T07:34:27.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:27 smithi149 ceph-mon[35408]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-27T07:34:27.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:27 smithi063 ceph-mon[31613]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-27T07:34:30.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:30 smithi086 ceph-mon[34687]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:30.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:30 smithi149 ceph-mon[35408]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:30.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:30 smithi063 ceph-mon[31613]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:31.601 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:34:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:34:32.538 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:32 smithi063 ceph-mon[31613]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:32.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:32 smithi086 ceph-mon[34687]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:32.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:32 smithi149 ceph-mon[35408]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:33.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:33 smithi063 ceph-mon[31613]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:33.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:33 smithi086 ceph-mon[34687]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:33.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:33 smithi149 ceph-mon[35408]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:34.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.539+0000 7f209a800700 1 -- 172.21.15.63:0/2779202135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20941054f0 msgr2=0x7f2094105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:34.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.539+0000 7f209a800700 1 --2- 172.21.15.63:0/2779202135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20941054f0 0x7f2094105900 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2084009990 tx=0x7f2084009ca0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.539+0000 7f209a800700 1 -- 172.21.15.63:0/2779202135 shutdown_connections 2024-05-27T07:34:34.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.539+0000 7f209a800700 1 --2- 172.21.15.63:0/2779202135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209410e8d0 0x7f2094110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.539+0000 7f209a800700 1 --2- 172.21.15.63:0/2779202135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20941054f0 0x7f2094105900 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.539+0000 7f209a800700 1 --2- 172.21.15.63:0/2779202135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2094105e40 0x7f209410e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.539+0000 7f209a800700 1 -- 172.21.15.63:0/2779202135 >> 172.21.15.63:0/2779202135 conn(0x7f20940fedb0 msgr2=0x7f20941011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:34.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.539+0000 7f209a800700 1 -- 172.21.15.63:0/2779202135 shutdown_connections 2024-05-27T07:34:34.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.539+0000 7f209a800700 1 -- 172.21.15.63:0/2779202135 wait complete. 2024-05-27T07:34:34.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.541+0000 7f209a800700 1 Processor -- start 2024-05-27T07:34:34.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.541+0000 7f209a800700 1 -- start start 2024-05-27T07:34:34.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.541+0000 7f209a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20941054f0 0x7f20941a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.541+0000 7f209a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2094105e40 0x7f20941a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.541+0000 7f209a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209410e8d0 0x7f20941ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.541+0000 7f209a800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f20941142d0 con 0x7f20941054f0 2024-05-27T07:34:34.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.541+0000 7f209a800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2094114150 con 0x7f209410e8d0 2024-05-27T07:34:34.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.541+0000 7f209a800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2094114450 con 0x7f2094105e40 2024-05-27T07:34:34.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.542+0000 7f2093400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20941054f0 0x7f20941a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.542+0000 7f2093400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20941054f0 0x7f20941a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35404/0 (socket says 172.21.15.63:35404) 2024-05-27T07:34:34.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.542+0000 7f2093400700 1 -- 172.21.15.63:0/2995703759 learned_addr learned my addr 172.21.15.63:0/2995703759 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:34.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.542+0000 7f2092a00700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2094105e40 0x7f20941a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.542+0000 7f2093e00700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209410e8d0 0x7f20941ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.542+0000 7f2093400700 1 -- 172.21.15.63:0/2995703759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2094105e40 msgr2=0x7f20941a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:34.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.542+0000 7f2093400700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2094105e40 0x7f20941a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.543+0000 7f2093400700 1 -- 172.21.15.63:0/2995703759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209410e8d0 msgr2=0x7f20941ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:34.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.543+0000 7f2093400700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209410e8d0 0x7f20941ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.543+0000 7f2093400700 1 -- 172.21.15.63:0/2995703759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20941ad570 con 0x7f20941054f0 2024-05-27T07:34:34.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.543+0000 7f2093e00700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209410e8d0 0x7f20941ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:34:34.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.543+0000 7f2093400700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20941054f0 0x7f20941a1fb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2084000c00 tx=0x7f20840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:34.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.543+0000 7f208be00700 1 -- 172.21.15.63:0/2995703759 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084027070 con 0x7f20941054f0 2024-05-27T07:34:34.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.543+0000 7f208be00700 1 -- 172.21.15.63:0/2995703759 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f208400fb80 con 0x7f20941054f0 2024-05-27T07:34:34.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.543+0000 7f209a800700 1 -- 172.21.15.63:0/2995703759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20941ad740 con 0x7f20941054f0 2024-05-27T07:34:34.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.543+0000 7f208be00700 1 -- 172.21.15.63:0/2995703759 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084022d00 con 0x7f20941054f0 2024-05-27T07:34:34.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.544+0000 7f209a800700 1 -- 172.21.15.63:0/2995703759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20941adbf0 con 0x7f20941054f0 2024-05-27T07:34:34.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.548+0000 7f208be00700 1 -- 172.21.15.63:0/2995703759 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f208402a070 con 0x7f20941054f0 2024-05-27T07:34:34.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.549+0000 7f20a93fcb80 1 Processor -- start 2024-05-27T07:34:34.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.549+0000 7f20a93fcb80 1 -- start start 2024-05-27T07:34:34.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.549+0000 7f208be00700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20780b2ba0 0x7f20780b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f208be00700 1 -- 172.21.15.63:0/2995703759 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f20840e3ac0 con 0x7f20941054f0 2024-05-27T07:34:34.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df7dee40 0x5576df9637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df95d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df95e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5576df88be40 con 0x5576df963d30 2024-05-27T07:34:34.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576df87b9e0 con 0x5576df95ddb0 2024-05-27T07:34:34.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5576df87fa20 con 0x5576df7dee40 2024-05-27T07:34:34.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f2093e00700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20780b2ba0 0x7f20780b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f2092a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df95d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f2092a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df95d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35408/0 (socket says 172.21.15.63:35408) 2024-05-27T07:34:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f2092a00700 1 -- 172.21.15.63:0/4087594170 learned_addr learned my addr 172.21.15.63:0/4087594170 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.550+0000 7f2093400700 1 --2- 172.21.15.63:0/4087594170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df7dee40 0x5576df9637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.551+0000 7f2093e00700 1 --2- 172.21.15.63:0/4087594170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df95e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.551+0000 7f2092a00700 1 -- 172.21.15.63:0/4087594170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df7dee40 msgr2=0x5576df9637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.551+0000 7f2092a00700 1 --2- 172.21.15.63:0/4087594170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df7dee40 0x5576df9637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.551+0000 7f2092a00700 1 -- 172.21.15.63:0/4087594170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 msgr2=0x5576df95e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.551+0000 7f2092a00700 1 --2- 172.21.15.63:0/4087594170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df95e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.551+0000 7f2092a00700 1 -- 172.21.15.63:0/4087594170 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576df844420 con 0x5576df963d30 2024-05-27T07:34:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.551+0000 7f2093e00700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20780b2ba0 0x7f20780b50b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f208c0076f0 tx=0x7f208c009f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:34.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.551+0000 7f2092a00700 1 --2- 172.21.15.63:0/4087594170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df95d870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f208000bd30 tx=0x7f20800106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3436f4d5376a091 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:34.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.552+0000 7f2088c00700 1 -- 172.21.15.63:0/4087594170 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080010e10 con 0x5576df963d30 2024-05-27T07:34:34.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.552+0000 7f2088c00700 1 -- 172.21.15.63:0/4087594170 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20800044c0 con 0x5576df963d30 2024-05-27T07:34:34.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.552+0000 7f2088c00700 1 -- 172.21.15.63:0/4087594170 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080003e60 con 0x5576df963d30 2024-05-27T07:34:34.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.553+0000 7f20a93fcb80 1 -- 172.21.15.63:0/4087594170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 msgr2=0x5576df95d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.553+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/4087594170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df95d870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f208000bd30 tx=0x7f20800106b0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.553+0000 7f20a93fcb80 1 -- 172.21.15.63:0/4087594170 shutdown_connections 2024-05-27T07:34:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.553+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/4087594170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df95e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.553+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/4087594170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df7dee40 0x5576df9637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.553+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/4087594170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df95d870 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.553+0000 7f20a93fcb80 1 -- 172.21.15.63:0/4087594170 >> 172.21.15.63:0/4087594170 conn(0x5576df83efc0 msgr2=0x5576df9588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.553+0000 7f20a93fcb80 1 -- 172.21.15.63:0/4087594170 shutdown_connections 2024-05-27T07:34:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.553+0000 7f20a93fcb80 1 -- 172.21.15.63:0/4087594170 wait complete. 2024-05-27T07:34:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.554+0000 7f20a93fcb80 1 -- start start 2024-05-27T07:34:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.554+0000 7f20a93fcb80 1 Processor -- start 2024-05-27T07:34:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df7dee40 0x5576df967420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df967960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df963d30 0x5576df967ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5576df781fa0 con 0x5576df7dee40 2024-05-27T07:34:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576df81c760 con 0x5576df95ddb0 2024-05-27T07:34:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5576df96a800 con 0x5576df963d30 2024-05-27T07:34:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f20a93fcb80 10 client.0 _ll_register_callbacks cb 0x5576df825c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f2093400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df7dee40 0x5576df967420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f2093400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df7dee40 0x5576df967420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35420/0 (socket says 172.21.15.63:35420) 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f2093400700 1 -- 172.21.15.63:0/3823695239 learned_addr learned my addr 172.21.15.63:0/3823695239 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f2092a00700 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df967960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.555+0000 7f2093e00700 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df963d30 0x5576df967ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.556+0000 7f2093400700 1 -- 172.21.15.63:0/3823695239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df963d30 msgr2=0x5576df967ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.556+0000 7f2093400700 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df963d30 0x5576df967ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.556+0000 7f2093400700 1 -- 172.21.15.63:0/3823695239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 msgr2=0x5576df967960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.556+0000 7f2093400700 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df967960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.556+0000 7f2093400700 1 -- 172.21.15.63:0/3823695239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576df7cb8d0 con 0x5576df7dee40 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.556+0000 7f2093400700 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df7dee40 0x5576df967420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f20840e0d50 tx=0x7f20840aadc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d78924ef7d56de server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.556+0000 7f207d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:34.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.556+0000 7f207d600700 1 -- 172.21.15.63:0/3823695239 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20840aa880 con 0x5576df7dee40 2024-05-27T07:34:34.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.556+0000 7f207d600700 1 -- 172.21.15.63:0/3823695239 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f20840b10d0 con 0x5576df7dee40 2024-05-27T07:34:34.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.556+0000 7f20a93fcb80 1 -- 172.21.15.63:0/3823695239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20800044c0 con 0x5576df7dee40 2024-05-27T07:34:34.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f207d600700 1 -- 172.21.15.63:0/3823695239 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20840aec30 con 0x5576df7dee40 2024-05-27T07:34:34.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f20a93fcb80 10 client.15483 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:34.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f20a93fcb80 1 -- 172.21.15.63:0/3823695239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576df84c060 con 0x5576df7dee40 2024-05-27T07:34:34.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f207d600700 1 -- 172.21.15.63:0/3823695239 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20800044c0 con 0x5576df7dee40 2024-05-27T07:34:34.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f207d600700 1 -- 172.21.15.63:0/3823695239 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2084092110 con 0x5576df7dee40 2024-05-27T07:34:34.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f20a93fcb80 10 client.15483 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:34.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f20a93fcb80 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:34.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f20a93fcb80 20 client.15483 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:34.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5576df977540 0x5576df9799a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f20a93fcb80 4 client.15483 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:34:34.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.557+0000 7f20a93fcb80 1 -- 172.21.15.63:0/3823695239 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576df88dd90 con 0x5576df977540 2024-05-27T07:34:34.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.558+0000 7f2092a00700 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5576df977540 0x5576df9799a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.559+0000 7f207d600700 10 client.15483 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:34.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.559+0000 7f2092a00700 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5576df977540 0x5576df9799a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef19c7bb4eac7618 server_cookie=1033c3c2ac236ffe in_seq=0 out_seq=0 2024-05-27T07:34:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.561+0000 7f207d600700 1 -- 172.21.15.63:0/3823695239 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5576df88dd90 con 0x5576df977540 2024-05-27T07:34:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.561+0000 7f207d600700 10 client.15483 handle_command_reply: tid=0 2024-05-27T07:34:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.561+0000 7f20a93fcb80 1 client.15483 shutdown 2024-05-27T07:34:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 1 -- 172.21.15.63:0/3823695239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df7dee40 msgr2=0x5576df967420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df7dee40 0x5576df967420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f20840e0d50 tx=0x7f20840aadc0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 1 -- 172.21.15.63:0/3823695239 shutdown_connections 2024-05-27T07:34:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5576df977540 0x5576df9799a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df963d30 0x5576df967ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df7dee40 0x5576df967420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/3823695239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df967960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 1 -- 172.21.15.63:0/3823695239 >> 172.21.15.63:0/3823695239 conn(0x5576df83efc0 msgr2=0x5576df958580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f207d600700 0 client.15483 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 1 -- 172.21.15.63:0/3823695239 shutdown_connections 2024-05-27T07:34:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 1 -- 172.21.15.63:0/3823695239 wait complete. 2024-05-27T07:34:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.562+0000 7f20a93fcb80 20 client.15483 trim_cache size 0 max 16384 2024-05-27T07:34:34.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.585+0000 7f20a93fcb80 1 Processor -- start 2024-05-27T07:34:34.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.585+0000 7f20a93fcb80 1 -- start start 2024-05-27T07:34:34.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.586+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df7dee40 0x5576df95d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.586+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df973860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.586+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df964b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.586+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b9190 con 0x5576df963d30 2024-05-27T07:34:34.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.586+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b6a20 con 0x5576df95ddb0 2024-05-27T07:34:34.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.586+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b4870 con 0x5576df7dee40 2024-05-27T07:34:34.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.586+0000 7f2093e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df964b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.586+0000 7f2093e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df964b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35426/0 (socket says 172.21.15.63:35426) 2024-05-27T07:34:34.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.586+0000 7f2093e00700 1 -- 172.21.15.63:0/1612700797 learned_addr learned my addr 172.21.15.63:0/1612700797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:34.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.586+0000 7f2093400700 1 --2- 172.21.15.63:0/1612700797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df7dee40 0x5576df95d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.587+0000 7f2092a00700 1 --2- 172.21.15.63:0/1612700797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df973860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.587+0000 7f2093e00700 1 -- 172.21.15.63:0/1612700797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df7dee40 msgr2=0x5576df95d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:34.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.587+0000 7f2093e00700 1 --2- 172.21.15.63:0/1612700797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df7dee40 0x5576df95d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.587+0000 7f2093e00700 1 -- 172.21.15.63:0/1612700797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 msgr2=0x5576df973860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:34.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.587+0000 7f2093e00700 1 --2- 172.21.15.63:0/1612700797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df973860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.587+0000 7f2093e00700 1 -- 172.21.15.63:0/1612700797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576df968210 con 0x5576df963d30 2024-05-27T07:34:34.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.587+0000 7f2093e00700 1 --2- 172.21.15.63:0/1612700797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df964b40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f208c0197d0 tx=0x7f208c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8caafdd7a95502a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:34.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.588+0000 7f207fe00700 1 -- 172.21.15.63:0/1612700797 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c021050 con 0x5576df963d30 2024-05-27T07:34:34.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.588+0000 7f207fe00700 1 -- 172.21.15.63:0/1612700797 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f208c02b070 con 0x5576df963d30 2024-05-27T07:34:34.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.588+0000 7f207fe00700 1 -- 172.21.15.63:0/1612700797 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c012040 con 0x5576df963d30 2024-05-27T07:34:34.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.589+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1612700797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 msgr2=0x5576df964b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:34.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.589+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/1612700797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df964b40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f208c0197d0 tx=0x7f208c01c000 comp rx=0 tx=0).stop 2024-05-27T07:34:34.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.589+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1612700797 shutdown_connections 2024-05-27T07:34:34.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.589+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/1612700797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df963d30 0x5576df964b40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.589+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/1612700797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df7dee40 0x5576df95d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.589+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/1612700797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df95ddb0 0x5576df973860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.589+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1612700797 >> 172.21.15.63:0/1612700797 conn(0x5576df83efc0 msgr2=0x5576df964430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:34.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.589+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1612700797 shutdown_connections 2024-05-27T07:34:34.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.589+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1612700797 wait complete. 2024-05-27T07:34:34.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.589+0000 7f20a93fcb80 1 -- start start 2024-05-27T07:34:34.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.590+0000 7f20a93fcb80 1 Processor -- start 2024-05-27T07:34:34.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.590+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df7dee40 0x5576df96ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.590+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df95ddb0 0x5576df97e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.590+0000 7f20a93fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df963d30 0x5576df97ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.590+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b4ee0 con 0x5576df95ddb0 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.590+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b6350 con 0x5576df7dee40 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.590+0000 7f20a93fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b55b0 con 0x5576df963d30 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.590+0000 7f20a93fcb80 10 client.0 _ll_register_callbacks cb 0x5576df825c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2092a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df95ddb0 0x5576df97e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2092a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df95ddb0 0x5576df97e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35434/0 (socket says 172.21.15.63:35434) 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2092a00700 1 -- 172.21.15.63:0/1643509302 learned_addr learned my addr 172.21.15.63:0/1643509302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2093400700 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df7dee40 0x5576df96ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2093e00700 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df963d30 0x5576df97ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2092a00700 1 -- 172.21.15.63:0/1643509302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df963d30 msgr2=0x5576df97ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2092a00700 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df963d30 0x5576df97ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2092a00700 1 -- 172.21.15.63:0/1643509302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df7dee40 msgr2=0x5576df96ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2092a00700 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df7dee40 0x5576df96ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2092a00700 1 -- 172.21.15.63:0/1643509302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576df97f1b0 con 0x5576df95ddb0 2024-05-27T07:34:34.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.591+0000 7f2093400700 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df7dee40 0x5576df96ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:34:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.592+0000 7f2092a00700 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df95ddb0 0x5576df97e730 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f208000bd30 tx=0x7f208000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240a4bf2afd7b3dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.592+0000 7f207d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.592+0000 7f207d600700 1 -- 172.21.15.63:0/1643509302 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208002d070 con 0x5576df95ddb0 2024-05-27T07:34:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.592+0000 7f207d600700 1 -- 172.21.15.63:0/1643509302 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f2080003a60 con 0x5576df95ddb0 2024-05-27T07:34:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.592+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1643509302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f208c02b070 con 0x5576df95ddb0 2024-05-27T07:34:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.592+0000 7f207d600700 1 -- 172.21.15.63:0/1643509302 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2080036050 con 0x5576df95ddb0 2024-05-27T07:34:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.592+0000 7f20a93fcb80 10 client.15489 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.592+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1643509302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576df9e1c80 con 0x5576df95ddb0 2024-05-27T07:34:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.593+0000 7f207d600700 1 -- 172.21.15.63:0/1643509302 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f208c02b070 con 0x5576df95ddb0 2024-05-27T07:34:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.593+0000 7f207d600700 1 -- 172.21.15.63:0/1643509302 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2080030020 con 0x5576df95ddb0 2024-05-27T07:34:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.593+0000 7f20a93fcb80 10 client.15489 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.593+0000 7f20a93fcb80 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.593+0000 7f20a93fcb80 20 client.15489 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.593+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5576df9e2220 0x5576df9e4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.593+0000 7f20a93fcb80 4 client.15489 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:34:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.593+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1643509302 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5576df887570 con 0x5576df9e2220 2024-05-27T07:34:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.594+0000 7f2093400700 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5576df9e2220 0x5576df9e4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:34.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.595+0000 7f2093400700 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5576df9e2220 0x5576df9e4640 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce595ac6df4b76ad server_cookie=600b06fd67c526e4 in_seq=0 out_seq=0 2024-05-27T07:34:34.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.595+0000 7f207d600700 10 client.15489 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:34.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.595+0000 7f207d600700 1 -- 172.21.15.63:0/1643509302 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5576df887570 con 0x5576df9e2220 2024-05-27T07:34:34.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.595+0000 7f207d600700 10 client.15489 handle_command_reply: tid=0 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.595+0000 7f20a93fcb80 1 client.15489 shutdown 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1643509302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df95ddb0 msgr2=0x5576df97e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df95ddb0 0x5576df97e730 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f208000bd30 tx=0x7f208000bf40 comp rx=0 tx=0).stop 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1643509302 shutdown_connections 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5576df9e2220 0x5576df9e4640 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576df963d30 0x5576df97ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576df7dee40 0x5576df96ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 1 --2- 172.21.15.63:0/1643509302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5576df95ddb0 0x5576df97e730 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1643509302 >> 172.21.15.63:0/1643509302 conn(0x5576df83efc0 msgr2=0x5576df9619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f207d600700 0 client.15489 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1643509302 shutdown_connections 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 1 -- 172.21.15.63:0/1643509302 wait complete. 2024-05-27T07:34:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.596+0000 7f20a93fcb80 20 client.15489 trim_cache size 0 max 16384 2024-05-27T07:34:34.593 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:34:34.593 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:34:34.593 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:34:34.593 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:34:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.597+0000 7f209a800700 1 -- 172.21.15.63:0/2995703759 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20780b2ba0 msgr2=0x7f20780b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.597+0000 7f209a800700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20780b2ba0 0x7f20780b50b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f208c0076f0 tx=0x7f208c009f90 comp rx=0 tx=0).stop 2024-05-27T07:34:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.597+0000 7f209a800700 1 -- 172.21.15.63:0/2995703759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20941054f0 msgr2=0x7f20941a1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.597+0000 7f209a800700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20941054f0 0x7f20941a1fb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2084000c00 tx=0x7f20840047b0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.597+0000 7f209a800700 1 -- 172.21.15.63:0/2995703759 shutdown_connections 2024-05-27T07:34:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.597+0000 7f209a800700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20780b2ba0 0x7f20780b50b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.597+0000 7f209a800700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209410e8d0 0x7f20941ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.597+0000 7f209a800700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20941054f0 0x7f20941a1fb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.597+0000 7f209a800700 1 --2- 172.21.15.63:0/2995703759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2094105e40 0x7f20941a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.597+0000 7f209a800700 1 -- 172.21.15.63:0/2995703759 >> 172.21.15.63:0/2995703759 conn(0x7f20940fedb0 msgr2=0x7f2094103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.598+0000 7f209a800700 1 -- 172.21.15.63:0/2995703759 shutdown_connections 2024-05-27T07:34:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:34.598+0000 7f209a800700 1 -- 172.21.15.63:0/2995703759 wait complete. 2024-05-27T07:34:35.566 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:34:35.567 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:34:36.035 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:34:36.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:36 smithi063 ceph-mon[31613]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:36.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:36 smithi086 ceph-mon[34687]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:36.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:36 smithi149 ceph-mon[35408]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:37.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:37 smithi086 ceph-mon[34687]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:37.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:37 smithi149 ceph-mon[35408]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:37.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:37 smithi063 ceph-mon[31613]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.969+0000 7fdf4dc00640 1 -- 172.21.15.63:0/3008045805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf4810a890 msgr2=0x7fdf48116b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.969+0000 7fdf4dc00640 1 --2- 172.21.15.63:0/3008045805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf4810a890 0x7fdf48116b50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009ca0 tx=0x7fdf4002a950 comp rx=0 tx=0).stop 2024-05-27T07:34:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.969+0000 7fdf4dc00640 1 -- 172.21.15.63:0/3008045805 shutdown_connections 2024-05-27T07:34:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.969+0000 7fdf4dc00640 1 --2- 172.21.15.63:0/3008045805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf4810a890 0x7fdf48116b50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.969+0000 7fdf4dc00640 1 --2- 172.21.15.63:0/3008045805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf48110520 0x7fdf4810a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.969+0000 7fdf4dc00640 1 --2- 172.21.15.63:0/3008045805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf48110110 0x7fdf48109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.969+0000 7fdf4dc00640 1 -- 172.21.15.63:0/3008045805 >> 172.21.15.63:0/3008045805 conn(0x7fdf48076070 msgr2=0x7fdf48078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.969+0000 7fdf4dc00640 1 -- 172.21.15.63:0/3008045805 shutdown_connections 2024-05-27T07:34:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.969+0000 7fdf4dc00640 1 -- 172.21.15.63:0/3008045805 wait complete. 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.970+0000 7fdf4dc00640 1 Processor -- start 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.970+0000 7fdf4dc00640 1 -- start start 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.971+0000 7fdf4dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf48110110 0x7fdf481a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.971+0000 7fdf4dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf48110520 0x7fdf481a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.971+0000 7fdf4dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481a0660 0x7fdf481a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.971+0000 7fdf4dc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4811a1b0 con 0x7fdf481a0660 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.971+0000 7fdf4dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4811a030 con 0x7fdf48110520 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.971+0000 7fdf4dc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4811a330 con 0x7fdf48110110 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.971+0000 7fdf4d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481a0660 0x7fdf481a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.971+0000 7fdf4d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481a0660 0x7fdf481a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35452/0 (socket says 172.21.15.63:35452) 2024-05-27T07:34:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.971+0000 7fdf4d200640 1 -- 172.21.15.63:0/853897761 learned_addr learned my addr 172.21.15.63:0/853897761 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.971+0000 7fdf47e00640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf48110110 0x7fdf481a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.972+0000 7fdf47400640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf48110520 0x7fdf481a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.972+0000 7fdf4d200640 1 -- 172.21.15.63:0/853897761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf48110110 msgr2=0x7fdf481a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.972+0000 7fdf4d200640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf48110110 0x7fdf481a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.972+0000 7fdf4d200640 1 -- 172.21.15.63:0/853897761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf48110520 msgr2=0x7fdf481a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.972+0000 7fdf4d200640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf48110520 0x7fdf481a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.972+0000 7fdf4d200640 1 -- 172.21.15.63:0/853897761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf481a1150 con 0x7fdf481a0660 2024-05-27T07:34:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.972+0000 7fdf4d200640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481a0660 0x7fdf481a0a70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009dc0 tx=0x7fdf40003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.972+0000 7fdf44c00640 1 -- 172.21.15.63:0/853897761 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40034040 con 0x7fdf481a0660 2024-05-27T07:34:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.973+0000 7fdf44c00640 1 -- 172.21.15.63:0/853897761 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf40030850 con 0x7fdf481a0660 2024-05-27T07:34:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.973+0000 7fdf44c00640 1 -- 172.21.15.63:0/853897761 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf400046e0 con 0x7fdf481a0660 2024-05-27T07:34:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.973+0000 7fdf4dc00640 1 -- 172.21.15.63:0/853897761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf481a13e0 con 0x7fdf481a0660 2024-05-27T07:34:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.973+0000 7fdf4dc00640 1 -- 172.21.15.63:0/853897761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf481ad460 con 0x7fdf481a0660 2024-05-27T07:34:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.977+0000 7fdf56194740 1 Processor -- start 2024-05-27T07:34:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.977+0000 7fdf44c00640 1 -- 172.21.15.63:0/853897761 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7fdf40037020 con 0x7fdf481a0660 2024-05-27T07:34:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.977+0000 7fdf56194740 1 -- start start 2024-05-27T07:34:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf56194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dced31b0 0x5649dd275a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf56194740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceaa2f0 0x5649dd275fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf56194740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceb24d0 0x5649dd2764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf56194740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5649dd1e6b90 con 0x5649dceb24d0 2024-05-27T07:34:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf56194740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5649dd1d6730 con 0x5649dceaa2f0 2024-05-27T07:34:37.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf56194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5649dd1da770 con 0x5649dced31b0 2024-05-27T07:34:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf47400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceb24d0 0x5649dd2764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf47400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceb24d0 0x5649dd2764e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35460/0 (socket says 172.21.15.63:35460) 2024-05-27T07:34:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf47400640 1 -- 172.21.15.63:0/1991067734 learned_addr learned my addr 172.21.15.63:0/1991067734 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf47e00640 1 --2- 172.21.15.63:0/1991067734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceaa2f0 0x5649dd275fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.978+0000 7fdf47400640 1 --2- 172.21.15.63:0/1991067734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dced31b0 0x5649dd275a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.979+0000 7fdf47400640 1 -- 172.21.15.63:0/1991067734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dced31b0 msgr2=0x5649dd275a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.979+0000 7fdf47400640 1 --2- 172.21.15.63:0/1991067734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dced31b0 0x5649dd275a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.979+0000 7fdf47400640 1 -- 172.21.15.63:0/1991067734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceaa2f0 msgr2=0x5649dd275fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.979+0000 7fdf47400640 1 --2- 172.21.15.63:0/1991067734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceaa2f0 0x5649dd275fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.979+0000 7fdf47400640 1 -- 172.21.15.63:0/1991067734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649dd16f8b0 con 0x5649dceb24d0 2024-05-27T07:34:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.979+0000 7fdf44c00640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdf2c0b2b80 0x7fdf2c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.979+0000 7fdf44c00640 1 -- 172.21.15.63:0/853897761 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fdf40103ba0 con 0x7fdf481a0660 2024-05-27T07:34:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.979+0000 7fdf47400640 1 --2- 172.21.15.63:0/1991067734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceb24d0 0x5649dd2764e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400cfd0 tx=0x7fdf3400c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e198088e04b244 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.979+0000 7fdf47e00640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdf2c0b2b80 0x7fdf2c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.980+0000 7fdf3d600640 1 -- 172.21.15.63:0/1991067734 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34019020 con 0x5649dceb24d0 2024-05-27T07:34:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.980+0000 7fdf3d600640 1 -- 172.21.15.63:0/1991067734 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf34011b90 con 0x5649dceb24d0 2024-05-27T07:34:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.980+0000 7fdf3d600640 1 -- 172.21.15.63:0/1991067734 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400fa60 con 0x5649dceb24d0 2024-05-27T07:34:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.981+0000 7fdf56194740 1 -- 172.21.15.63:0/1991067734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceb24d0 msgr2=0x5649dd2764e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.981+0000 7fdf47e00640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdf2c0b2b80 0x7fdf2c0b5090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5649dd276e10 tx=0x7fdf3800b4d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.981+0000 7fdf56194740 1 --2- 172.21.15.63:0/1991067734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceb24d0 0x5649dd2764e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400cfd0 tx=0x7fdf3400c3f0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.981+0000 7fdf56194740 1 -- 172.21.15.63:0/1991067734 shutdown_connections 2024-05-27T07:34:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.981+0000 7fdf56194740 1 --2- 172.21.15.63:0/1991067734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceb24d0 0x5649dd2764e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.981+0000 7fdf56194740 1 --2- 172.21.15.63:0/1991067734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceaa2f0 0x5649dd275fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.981+0000 7fdf56194740 1 --2- 172.21.15.63:0/1991067734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dced31b0 0x5649dd275a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.981+0000 7fdf56194740 1 -- 172.21.15.63:0/1991067734 >> 172.21.15.63:0/1991067734 conn(0x5649dcec7300 msgr2=0x5649dd272210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.981+0000 7fdf56194740 1 -- 172.21.15.63:0/1991067734 shutdown_connections 2024-05-27T07:34:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.981+0000 7fdf56194740 1 -- 172.21.15.63:0/1991067734 wait complete. 2024-05-27T07:34:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.982+0000 7fdf56194740 1 -- start start 2024-05-27T07:34:37.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.982+0000 7fdf56194740 1 Processor -- start 2024-05-27T07:34:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.982+0000 7fdf56194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dceaa2f0 0x5649dd27fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.982+0000 7fdf56194740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceb24d0 0x5649dd2801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.982+0000 7fdf56194740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.982+0000 7fdf56194740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5649dcebc790 con 0x5649dced31b0 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.982+0000 7fdf56194740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5649dd17ebc0 con 0x5649dceb24d0 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.982+0000 7fdf56194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5649dd282b90 con 0x5649dceaa2f0 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf56194740 10 client.0 _ll_register_callbacks cb 0x5649dd15c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf4d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf4d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35464/0 (socket says 172.21.15.63:35464) 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf4d200640 1 -- 172.21.15.63:0/225811111 learned_addr learned my addr 172.21.15.63:0/225811111 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf47e00640 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dceaa2f0 0x5649dd27fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf47400640 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceb24d0 0x5649dd2801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf4d200640 1 -- 172.21.15.63:0/225811111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dceaa2f0 msgr2=0x5649dd27fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf4d200640 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dceaa2f0 0x5649dd27fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf4d200640 1 -- 172.21.15.63:0/225811111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceb24d0 msgr2=0x5649dd2801c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf4d200640 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceb24d0 0x5649dd2801c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.983+0000 7fdf4d200640 1 -- 172.21.15.63:0/225811111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649dceeb210 con 0x5649dced31b0 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.984+0000 7fdf4d200640 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdf40100f90 tx=0x7fdf400ca3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fc86bc35f4d0e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.984+0000 7fdf32000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.984+0000 7fdf32000640 1 -- 172.21.15.63:0/225811111 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4002c040 con 0x5649dced31b0 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.984+0000 7fdf32000640 1 -- 172.21.15.63:0/225811111 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fdf400ed2e0 con 0x5649dced31b0 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.984+0000 7fdf56194740 1 -- 172.21.15.63:0/225811111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf34011b90 con 0x5649dced31b0 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.984+0000 7fdf32000640 1 -- 172.21.15.63:0/225811111 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf400ca4f0 con 0x5649dced31b0 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.984+0000 7fdf56194740 10 client.15501 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.984+0000 7fdf56194740 1 -- 172.21.15.63:0/225811111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649dcfb7620 con 0x5649dced31b0 2024-05-27T07:34:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.985+0000 7fdf32000640 1 -- 172.21.15.63:0/225811111 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf34011b90 con 0x5649dced31b0 2024-05-27T07:34:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.985+0000 7fdf32000640 1 -- 172.21.15.63:0/225811111 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdf400d33b0 con 0x5649dced31b0 2024-05-27T07:34:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.985+0000 7fdf56194740 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.985+0000 7fdf56194740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.985+0000 7fdf56194740 20 client.15501 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.985+0000 7fdf56194740 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5649dce91d80 0x5649dd291f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.985+0000 7fdf56194740 4 client.15501 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:34:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.985+0000 7fdf56194740 1 -- 172.21.15.63:0/225811111 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649dd1e8ae0 con 0x5649dce91d80 2024-05-27T07:34:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.986+0000 7fdf47e00640 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5649dce91d80 0x5649dd291f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.987+0000 7fdf47e00640 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5649dce91d80 0x5649dd291f80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18de587077877d2e server_cookie=2b432f47e29e327d in_seq=0 out_seq=0 2024-05-27T07:34:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.987+0000 7fdf32000640 10 client.15501 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.988+0000 7fdf32000640 1 -- 172.21.15.63:0/225811111 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5649dd1e8ae0 con 0x5649dce91d80 2024-05-27T07:34:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf32000640 10 client.15501 handle_command_reply: tid=0 2024-05-27T07:34:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 client.15501 shutdown 2024-05-27T07:34:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 -- 172.21.15.63:0/225811111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 msgr2=0x5649dd28bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdf40100f90 tx=0x7fdf400ca3f0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 -- 172.21.15.63:0/225811111 shutdown_connections 2024-05-27T07:34:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5649dce91d80 0x5649dd291f80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bec0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceb24d0 0x5649dd2801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 --2- 172.21.15.63:0/225811111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dceaa2f0 0x5649dd27fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 -- 172.21.15.63:0/225811111 >> 172.21.15.63:0/225811111 conn(0x5649dcec7300 msgr2=0x5649dd277780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf32000640 0 client.15501 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 -- 172.21.15.63:0/225811111 shutdown_connections 2024-05-27T07:34:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 1 -- 172.21.15.63:0/225811111 wait complete. 2024-05-27T07:34:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:37.989+0000 7fdf56194740 20 client.15501 trim_cache size 0 max 16384 2024-05-27T07:34:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.007+0000 7fdf56194740 1 Processor -- start 2024-05-27T07:34:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.007+0000 7fdf56194740 1 -- start start 2024-05-27T07:34:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.007+0000 7fdf56194740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceaa2f0 0x5649dd27b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf56194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dceb24d0 0x5649dd2759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf56194740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf56194740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf481b9da0 con 0x5649dced31b0 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf56194740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf481b7670 con 0x5649dceaa2f0 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf56194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdf481b54c0 con 0x5649dceb24d0 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf4d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf4d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35480/0 (socket says 172.21.15.63:35480) 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf4d200640 1 -- 172.21.15.63:0/3803543086 learned_addr learned my addr 172.21.15.63:0/3803543086 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf47e00640 1 --2- 172.21.15.63:0/3803543086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceaa2f0 0x5649dd27b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf47400640 1 --2- 172.21.15.63:0/3803543086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dceb24d0 0x5649dd2759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf4d200640 1 -- 172.21.15.63:0/3803543086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dceb24d0 msgr2=0x5649dd2759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf4d200640 1 --2- 172.21.15.63:0/3803543086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dceb24d0 0x5649dd2759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf4d200640 1 -- 172.21.15.63:0/3803543086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceaa2f0 msgr2=0x5649dd27b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:38.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.008+0000 7fdf4d200640 1 --2- 172.21.15.63:0/3803543086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceaa2f0 0x5649dd27b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.009+0000 7fdf4d200640 1 -- 172.21.15.63:0/3803543086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649dd276e40 con 0x5649dced31b0 2024-05-27T07:34:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.009+0000 7fdf4d200640 1 --2- 172.21.15.63:0/3803543086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bc30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdf400ca690 tx=0x7fdf400ca460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c5981b95f4af5d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.009+0000 7fdf3cc00640 1 -- 172.21.15.63:0/3803543086 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4002c040 con 0x5649dced31b0 2024-05-27T07:34:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.009+0000 7fdf3cc00640 1 -- 172.21.15.63:0/3803543086 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf400cb0e0 con 0x5649dced31b0 2024-05-27T07:34:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.009+0000 7fdf3cc00640 1 -- 172.21.15.63:0/3803543086 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40101190 con 0x5649dced31b0 2024-05-27T07:34:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.010+0000 7fdf56194740 1 -- 172.21.15.63:0/3803543086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 msgr2=0x5649dd28bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.010+0000 7fdf56194740 1 --2- 172.21.15.63:0/3803543086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bc30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdf400ca690 tx=0x7fdf400ca460 comp rx=0 tx=0).stop 2024-05-27T07:34:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.010+0000 7fdf56194740 1 -- 172.21.15.63:0/3803543086 shutdown_connections 2024-05-27T07:34:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.010+0000 7fdf56194740 1 --2- 172.21.15.63:0/3803543086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dced31b0 0x5649dd28bc30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.010+0000 7fdf56194740 1 --2- 172.21.15.63:0/3803543086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dceb24d0 0x5649dd2759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.010+0000 7fdf56194740 1 --2- 172.21.15.63:0/3803543086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceaa2f0 0x5649dd27b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.010+0000 7fdf56194740 1 -- 172.21.15.63:0/3803543086 >> 172.21.15.63:0/3803543086 conn(0x5649dcec7300 msgr2=0x5649dd281dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.010+0000 7fdf56194740 1 -- 172.21.15.63:0/3803543086 shutdown_connections 2024-05-27T07:34:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.010+0000 7fdf56194740 1 -- 172.21.15.63:0/3803543086 wait complete. 2024-05-27T07:34:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.011+0000 7fdf56194740 1 -- start start 2024-05-27T07:34:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.011+0000 7fdf56194740 1 Processor -- start 2024-05-27T07:34:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf56194740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceaa2f0 0x5649dd282de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf56194740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceb24d0 0x5649dd283320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf56194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dced31b0 0x5649dd283860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf56194740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf481b5b30 con 0x5649dceaa2f0 2024-05-27T07:34:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf56194740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf481b6fa0 con 0x5649dceb24d0 2024-05-27T07:34:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf56194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdf481b6200 con 0x5649dced31b0 2024-05-27T07:34:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf56194740 10 client.0 _ll_register_callbacks cb 0x7fdf4002fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf47e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceaa2f0 0x5649dd282de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf47e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceaa2f0 0x5649dd282de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35484/0 (socket says 172.21.15.63:35484) 2024-05-27T07:34:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf47e00640 1 -- 172.21.15.63:0/3630286154 learned_addr learned my addr 172.21.15.63:0/3630286154 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf47e00640 1 -- 172.21.15.63:0/3630286154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dced31b0 msgr2=0x5649dd283860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf4d200640 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dced31b0 0x5649dd283860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf47400640 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceb24d0 0x5649dd283320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf47e00640 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dced31b0 0x5649dd283860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf47e00640 1 -- 172.21.15.63:0/3630286154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceb24d0 msgr2=0x5649dd283320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf47e00640 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceb24d0 0x5649dd283320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.012+0000 7fdf47e00640 1 -- 172.21.15.63:0/3630286154 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649dd2f87b0 con 0x5649dceaa2f0 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.013+0000 7fdf47e00640 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceaa2f0 0x5649dd282de0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdf38012fd0 tx=0x7fdf380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791d4a8aff9e71a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.013+0000 7fdf32000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.013+0000 7fdf32000640 1 -- 172.21.15.63:0/3630286154 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38004220 con 0x5649dceaa2f0 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.013+0000 7fdf32000640 1 -- 172.21.15.63:0/3630286154 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fdf380043c0 con 0x5649dceaa2f0 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.013+0000 7fdf56194740 1 -- 172.21.15.63:0/3630286154 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf400cb0e0 con 0x5649dceaa2f0 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.013+0000 7fdf32000640 1 -- 172.21.15.63:0/3630286154 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf3803a050 con 0x5649dceaa2f0 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.013+0000 7fdf56194740 10 client.15507 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.013+0000 7fdf56194740 1 -- 172.21.15.63:0/3630286154 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5649dd2f9aa0 con 0x5649dceaa2f0 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.014+0000 7fdf32000640 1 -- 172.21.15.63:0/3630286154 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf400cb0e0 con 0x5649dceaa2f0 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.014+0000 7fdf32000640 1 -- 172.21.15.63:0/3630286154 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdf38003a40 con 0x5649dceaa2f0 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.014+0000 7fdf56194740 10 client.15507 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.014+0000 7fdf56194740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.014+0000 7fdf56194740 20 client.15507 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.014+0000 7fdf56194740 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5649dce671c0 0x5649dd2fc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.014+0000 7fdf56194740 4 client.15507 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:34:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.014+0000 7fdf56194740 1 -- 172.21.15.63:0/3630286154 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649dd1e22c0 con 0x5649dce671c0 2024-05-27T07:34:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.015+0000 7fdf47400640 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5649dce671c0 0x5649dd2fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.016+0000 7fdf32000640 10 client.15507 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.016+0000 7fdf47400640 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5649dce671c0 0x5649dd2fc240 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f7a6793bc7ae81 server_cookie=5d9798ce21053dcb in_seq=0 out_seq=0 2024-05-27T07:34:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf32000640 1 -- 172.21.15.63:0/3630286154 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5649dd1e22c0 con 0x5649dce671c0 2024-05-27T07:34:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf32000640 10 client.15507 handle_command_reply: tid=0 2024-05-27T07:34:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 client.15507 shutdown 2024-05-27T07:34:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 -- 172.21.15.63:0/3630286154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceaa2f0 msgr2=0x5649dd282de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceaa2f0 0x5649dd282de0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdf38012fd0 tx=0x7fdf380046c0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 -- 172.21.15.63:0/3630286154 shutdown_connections 2024-05-27T07:34:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5649dce671c0 0x5649dd2fc240 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649dced31b0 0x5649dd283860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649dceb24d0 0x5649dd283320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 --2- 172.21.15.63:0/3630286154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649dceaa2f0 0x5649dd282de0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf32000640 0 client.15507 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 -- 172.21.15.63:0/3630286154 >> 172.21.15.63:0/3630286154 conn(0x5649dcec7300 msgr2=0x5649dd27f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 -- 172.21.15.63:0/3630286154 shutdown_connections 2024-05-27T07:34:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 1 -- 172.21.15.63:0/3630286154 wait complete. 2024-05-27T07:34:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.026+0000 7fdf56194740 20 client.15507 trim_cache size 0 max 16384 2024-05-27T07:34:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:34:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:34:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:34:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.027+0000 7fdf4dc00640 1 -- 172.21.15.63:0/853897761 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdf2c0b2b80 msgr2=0x7fdf2c0b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.027+0000 7fdf4dc00640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdf2c0b2b80 0x7fdf2c0b5090 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5649dd276e10 tx=0x7fdf3800b4d0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.027+0000 7fdf4dc00640 1 -- 172.21.15.63:0/853897761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481a0660 msgr2=0x7fdf481a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.027+0000 7fdf4dc00640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481a0660 0x7fdf481a0a70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009dc0 tx=0x7fdf40003f10 comp rx=0 tx=0).stop 2024-05-27T07:34:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.027+0000 7fdf4dc00640 1 -- 172.21.15.63:0/853897761 shutdown_connections 2024-05-27T07:34:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.027+0000 7fdf4dc00640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdf2c0b2b80 0x7fdf2c0b5090 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.027+0000 7fdf4dc00640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481a0660 0x7fdf481a0a70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.027+0000 7fdf4dc00640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf48110520 0x7fdf481a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.027+0000 7fdf4dc00640 1 --2- 172.21.15.63:0/853897761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf48110110 0x7fdf481a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.027+0000 7fdf4dc00640 1 -- 172.21.15.63:0/853897761 >> 172.21.15.63:0/853897761 conn(0x7fdf48076070 msgr2=0x7fdf4810d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.028+0000 7fdf4dc00640 1 -- 172.21.15.63:0/853897761 shutdown_connections 2024-05-27T07:34:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:38.028+0000 7fdf4dc00640 1 -- 172.21.15.63:0/853897761 wait complete. 2024-05-27T07:34:38.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.431+0000 7feb67a00700 1 -- 172.21.15.63:0/3077464976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb60110960 msgr2=0x7feb60110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:38.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.431+0000 7feb67a00700 1 --2- 172.21.15.63:0/3077464976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb60110960 0x7feb60110d70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7feb54009990 tx=0x7feb54009ca0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.432+0000 7feb67a00700 1 -- 172.21.15.63:0/3077464976 shutdown_connections 2024-05-27T07:34:38.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.432+0000 7feb67a00700 1 --2- 172.21.15.63:0/3077464976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6010b2d0 0x7feb60117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.432+0000 7feb67a00700 1 --2- 172.21.15.63:0/3077464976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb60110960 0x7feb60110d70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.432+0000 7feb67a00700 1 --2- 172.21.15.63:0/3077464976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6010a980 0x7feb6010ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.432+0000 7feb67a00700 1 -- 172.21.15.63:0/3077464976 >> 172.21.15.63:0/3077464976 conn(0x7feb60077c70 msgr2=0x7feb600780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:38.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.432+0000 7feb67a00700 1 -- 172.21.15.63:0/3077464976 shutdown_connections 2024-05-27T07:34:38.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.432+0000 7feb67a00700 1 -- 172.21.15.63:0/3077464976 wait complete. 2024-05-27T07:34:38.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.433+0000 7feb67a00700 1 Processor -- start 2024-05-27T07:34:38.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.433+0000 7feb67a00700 1 -- start start 2024-05-27T07:34:38.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb67a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6010a980 0x7feb601a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb67a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6010b2d0 0x7feb601a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb67a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb60110960 0x7feb601a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb67a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb6011a910 con 0x7feb60110960 2024-05-27T07:34:38.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb67a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb6011a790 con 0x7feb6010a980 2024-05-27T07:34:38.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb67a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feb6011aa90 con 0x7feb6010b2d0 2024-05-27T07:34:38.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb65a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb60110960 0x7feb601a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb65a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb60110960 0x7feb601a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35498/0 (socket says 172.21.15.63:35498) 2024-05-27T07:34:38.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb65a00700 1 -- 172.21.15.63:0/1219062752 learned_addr learned my addr 172.21.15.63:0/1219062752 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:38.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb65000700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6010a980 0x7feb601a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.434+0000 7feb5fe00700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6010b2d0 0x7feb601a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.435+0000 7feb65a00700 1 -- 172.21.15.63:0/1219062752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6010b2d0 msgr2=0x7feb601a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:38.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.435+0000 7feb65a00700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6010b2d0 0x7feb601a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.435+0000 7feb65a00700 1 -- 172.21.15.63:0/1219062752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6010a980 msgr2=0x7feb601a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:38.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.435+0000 7feb65a00700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6010a980 0x7feb601a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.435+0000 7feb65a00700 1 -- 172.21.15.63:0/1219062752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb601a6250 con 0x7feb60110960 2024-05-27T07:34:38.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.435+0000 7feb65a00700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb60110960 0x7feb601a5bb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7feb58007ee0 tx=0x7feb5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:38.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.435+0000 7feb5d600700 1 -- 172.21.15.63:0/1219062752 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5800dd20 con 0x7feb60110960 2024-05-27T07:34:38.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.436+0000 7feb5d600700 1 -- 172.21.15.63:0/1219062752 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb58004020 con 0x7feb60110960 2024-05-27T07:34:38.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.436+0000 7feb67a00700 1 -- 172.21.15.63:0/1219062752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb601a6540 con 0x7feb60110960 2024-05-27T07:34:38.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.436+0000 7feb5d600700 1 -- 172.21.15.63:0/1219062752 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58015660 con 0x7feb60110960 2024-05-27T07:34:38.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.436+0000 7feb67a00700 1 -- 172.21.15.63:0/1219062752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb601ade60 con 0x7feb60110960 2024-05-27T07:34:38.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.437+0000 7feb67a00700 1 -- 172.21.15.63:0/1219062752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6010c070 con 0x7feb60110960 2024-05-27T07:34:38.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.440+0000 7feb5d600700 1 -- 172.21.15.63:0/1219062752 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7feb580041d0 con 0x7feb60110960 2024-05-27T07:34:38.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.441+0000 7feb5d600700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7feb480b2ba0 0x7feb480b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:38.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.442+0000 7feb5d600700 1 -- 172.21.15.63:0/1219062752 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7feb580e2dc0 con 0x7feb60110960 2024-05-27T07:34:38.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.442+0000 7feb65000700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7feb480b2ba0 0x7feb480b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:38.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.443+0000 7feb65000700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7feb480b2ba0 0x7feb480b50b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb5400b5c0 tx=0x7feb54005960 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:38.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.444+0000 7feb5d600700 1 -- 172.21.15.63:0/1219062752 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb58010070 con 0x7feb60110960 2024-05-27T07:34:38.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.736+0000 7feb67a00700 1 -- 172.21.15.63:0/1219062752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb6002e5e0 con 0x7feb60110960 2024-05-27T07:34:38.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.737+0000 7feb5d600700 1 -- 172.21.15.63:0/1219062752 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11861 (secure 0 0 0) 0x7feb580aa1f0 con 0x7feb60110960 2024-05-27T07:34:38.733 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:34:38.733 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:33:33.871810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:34:38.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.741+0000 7feb67a00700 1 -- 172.21.15.63:0/1219062752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7feb480b2ba0 msgr2=0x7feb480b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:38.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.741+0000 7feb67a00700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7feb480b2ba0 0x7feb480b50b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb5400b5c0 tx=0x7feb54005960 comp rx=0 tx=0).stop 2024-05-27T07:34:38.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.741+0000 7feb67a00700 1 -- 172.21.15.63:0/1219062752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb60110960 msgr2=0x7feb601a5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:38.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.741+0000 7feb67a00700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb60110960 0x7feb601a5bb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7feb58007ee0 tx=0x7feb5800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.741+0000 7feb67a00700 1 -- 172.21.15.63:0/1219062752 shutdown_connections 2024-05-27T07:34:38.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.741+0000 7feb67a00700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7feb480b2ba0 0x7feb480b50b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.742+0000 7feb67a00700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb60110960 0x7feb601a5bb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.742+0000 7feb67a00700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6010a980 0x7feb601a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.742+0000 7feb67a00700 1 --2- 172.21.15.63:0/1219062752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6010b2d0 0x7feb601a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:38.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.742+0000 7feb67a00700 1 -- 172.21.15.63:0/1219062752 >> 172.21.15.63:0/1219062752 conn(0x7feb60077c70 msgr2=0x7feb6007bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:38.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.742+0000 7feb67a00700 1 -- 172.21.15.63:0/1219062752 shutdown_connections 2024-05-27T07:34:38.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:38.742+0000 7feb67a00700 1 -- 172.21.15.63:0/1219062752 wait complete. 2024-05-27T07:34:38.739 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 27 2024-05-27T07:34:39.142 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:38 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1219062752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:34:39.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:38 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1219062752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:34:39.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:38 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1219062752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:34:39.700 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:34:39.960 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:39 smithi063 ceph-mon[31613]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:34:40.176 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:34:40.203 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:39 smithi086 ceph-mon[34687]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:34:40.207 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:39 smithi149 ceph-mon[35408]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:34:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.633+0000 7f1e7de00700 1 -- 172.21.15.63:0/2234499982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e78108550 msgr2=0x7f1e7810feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.633+0000 7f1e7de00700 1 --2- 172.21.15.63:0/2234499982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e78108550 0x7f1e7810feb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000b0d0 tx=0x7f1e7000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.634+0000 7f1e7de00700 1 -- 172.21.15.63:0/2234499982 shutdown_connections 2024-05-27T07:34:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.634+0000 7f1e7de00700 1 --2- 172.21.15.63:0/2234499982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e78108550 0x7f1e7810feb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.634+0000 7f1e7de00700 1 --2- 172.21.15.63:0/2234499982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e781032d0 0x7f1e781036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.634+0000 7f1e7de00700 1 --2- 172.21.15.63:0/2234499982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e78107c00 0x7f1e78108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.634+0000 7f1e7de00700 1 -- 172.21.15.63:0/2234499982 >> 172.21.15.63:0/2234499982 conn(0x7f1e78078ab0 msgr2=0x7f1e78078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:42.630 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:42 smithi063 ceph-mon[31613]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.634+0000 7f1e7de00700 1 -- 172.21.15.63:0/2234499982 shutdown_connections 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.634+0000 7f1e7de00700 1 -- 172.21.15.63:0/2234499982 wait complete. 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.635+0000 7f1e7de00700 1 Processor -- start 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.635+0000 7f1e7de00700 1 -- start start 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.636+0000 7f1e7de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e781032d0 0x7f1e78104ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.636+0000 7f1e7de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e78107c00 0x7f1e781051e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.636+0000 7f1e7de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e78108550 0x7f1e78105720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.636+0000 7f1e7de00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78113030 con 0x7f1e78107c00 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.636+0000 7f1e7de00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78112eb0 con 0x7f1e78108550 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.636+0000 7f1e7de00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781131b0 con 0x7f1e781032d0 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.636+0000 7f1e77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e78107c00 0x7f1e781051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.636+0000 7f1e77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e78107c00 0x7f1e781051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36190/0 (socket says 172.21.15.63:36190) 2024-05-27T07:34:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.636+0000 7f1e77e00700 1 -- 172.21.15.63:0/1239291301 learned_addr learned my addr 172.21.15.63:0/1239291301 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.636+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e781032d0 0x7f1e78104ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.637+0000 7f1e7d400700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e78108550 0x7f1e78105720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.637+0000 7f1e77e00700 1 -- 172.21.15.63:0/1239291301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e781032d0 msgr2=0x7f1e78104ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.637+0000 7f1e77e00700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e781032d0 0x7f1e78104ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.637+0000 7f1e77e00700 1 -- 172.21.15.63:0/1239291301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e78108550 msgr2=0x7f1e78105720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.637+0000 7f1e77e00700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e78108550 0x7f1e78105720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.637+0000 7f1e77e00700 1 -- 172.21.15.63:0/1239291301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e781a8ef0 con 0x7f1e78107c00 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.637+0000 7f1e7d400700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e78108550 0x7f1e78105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.637+0000 7f1e77e00700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e78107c00 0x7f1e781051e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400d980 tx=0x7f1e6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.638+0000 7f1e75600700 1 -- 172.21.15.63:0/1239291301 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64017070 con 0x7f1e78107c00 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.638+0000 7f1e7de00700 1 -- 172.21.15.63:0/1239291301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e781a91e0 con 0x7f1e78107c00 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.638+0000 7f1e75600700 1 -- 172.21.15.63:0/1239291301 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e64009dc0 con 0x7f1e78107c00 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.638+0000 7f1e75600700 1 -- 172.21.15.63:0/1239291301 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6400f3c0 con 0x7f1e78107c00 2024-05-27T07:34:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.638+0000 7f1e7de00700 1 -- 172.21.15.63:0/1239291301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e781a9770 con 0x7f1e78107c00 2024-05-27T07:34:42.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.642+0000 7f1e75600700 1 -- 172.21.15.63:0/1239291301 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f1e64009930 con 0x7f1e78107c00 2024-05-27T07:34:42.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.643+0000 7f1e8de52b80 1 Processor -- start 2024-05-27T07:34:42.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.643+0000 7f1e8de52b80 1 -- start start 2024-05-27T07:34:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.643+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a613ee40 0x5591a62bced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.643+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd410 0x5591a62c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.643+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a62bd820 0x5591a62ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5591a61ebe40 con 0x5591a62bd820 2024-05-27T07:34:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5591a61db9e0 con 0x5591a613ee40 2024-05-27T07:34:42.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5591a61dfa20 con 0x5591a62bd410 2024-05-27T07:34:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e75600700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1e600b2c20 0x7f1e600b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a62bd820 0x5591a62ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a62bd820 0x5591a62ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36206/0 (socket says 172.21.15.63:36206) 2024-05-27T07:34:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e77e00700 1 -- 172.21.15.63:0/529528314 learned_addr learned my addr 172.21.15.63:0/529528314 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/529528314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd410 0x5591a62c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e7d400700 1 --2- 172.21.15.63:0/529528314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a613ee40 0x5591a62bced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e75600700 1 -- 172.21.15.63:0/1239291301 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f1e640e2670 con 0x7f1e78107c00 2024-05-27T07:34:42.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.644+0000 7f1e7d400700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1e600b2c20 0x7f1e600b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.645+0000 7f1e77e00700 1 -- 172.21.15.63:0/529528314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd410 msgr2=0x5591a62c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:42.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.645+0000 7f1e77e00700 1 --2- 172.21.15.63:0/529528314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd410 0x5591a62c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.645+0000 7f1e77e00700 1 -- 172.21.15.63:0/529528314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a613ee40 msgr2=0x5591a62bced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:42.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.645+0000 7f1e77e00700 1 --2- 172.21.15.63:0/529528314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a613ee40 0x5591a62bced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.645+0000 7f1e77e00700 1 -- 172.21.15.63:0/529528314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a61a4420 con 0x5591a62bd820 2024-05-27T07:34:42.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.645+0000 7f1e77e00700 1 --2- 172.21.15.63:0/529528314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a62bd820 0x5591a62ca020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1e640a90a0 tx=0x7f1e64076f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e838c2222be979d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:42.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.645+0000 7f1e7d400700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1e600b2c20 0x7f1e600b5130 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1e70005bd0 tx=0x7f1e70005b00 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:42.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.646+0000 7f1e6a000700 1 -- 172.21.15.63:0/529528314 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64020070 con 0x5591a62bd820 2024-05-27T07:34:42.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.646+0000 7f1e6a000700 1 -- 172.21.15.63:0/529528314 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e640a8eb0 con 0x5591a62bd820 2024-05-27T07:34:42.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.646+0000 7f1e6a000700 1 -- 172.21.15.63:0/529528314 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e640ad750 con 0x5591a62bd820 2024-05-27T07:34:42.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.647+0000 7f1e8de52b80 1 -- 172.21.15.63:0/529528314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a62bd820 msgr2=0x5591a62ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.647+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/529528314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a62bd820 0x5591a62ca020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1e640a90a0 tx=0x7f1e64076f70 comp rx=0 tx=0).stop 2024-05-27T07:34:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.647+0000 7f1e8de52b80 1 -- 172.21.15.63:0/529528314 shutdown_connections 2024-05-27T07:34:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.647+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/529528314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a62bd820 0x5591a62ca020 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.647+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/529528314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a613ee40 0x5591a62bced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.647+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/529528314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd410 0x5591a62c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.647+0000 7f1e8de52b80 1 -- 172.21.15.63:0/529528314 >> 172.21.15.63:0/529528314 conn(0x5591a619efc0 msgr2=0x5591a62b8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.647+0000 7f1e8de52b80 1 -- 172.21.15.63:0/529528314 shutdown_connections 2024-05-27T07:34:42.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.647+0000 7f1e8de52b80 1 -- 172.21.15.63:0/529528314 wait complete. 2024-05-27T07:34:42.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.648+0000 7f1e8de52b80 1 -- start start 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.648+0000 7f1e8de52b80 1 Processor -- start 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.648+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.648+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 0x5591a62c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.648+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd820 0x5591a62c8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.648+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5591a60e1fa0 con 0x5591a613ee40 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.648+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5591a617c760 con 0x5591a62bd410 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.648+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5591a62ca930 con 0x5591a62bd820 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.648+0000 7f1e8de52b80 10 client.0 _ll_register_callbacks cb 0x5591a6185c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e7ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e7ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36222/0 (socket says 172.21.15.63:36222) 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e7ca00700 1 -- 172.21.15.63:0/3196515541 learned_addr learned my addr 172.21.15.63:0/3196515541 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e77e00700 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 0x5591a62c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e7d400700 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd820 0x5591a62c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e7ca00700 1 -- 172.21.15.63:0/3196515541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd820 msgr2=0x5591a62c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:42.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd820 0x5591a62c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e7ca00700 1 -- 172.21.15.63:0/3196515541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 msgr2=0x5591a62c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:42.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 0x5591a62c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e7ca00700 1 -- 172.21.15.63:0/3196515541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a612b8d0 con 0x5591a613ee40 2024-05-27T07:34:42.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.649+0000 7f1e77e00700 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 0x5591a62c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:34:42.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.650+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62c7590 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00bba0 tx=0x7f1e6c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8e1d40e6102876 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:42.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.650+0000 7f1e5ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:42.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.650+0000 7f1e5ea00700 1 -- 172.21.15.63:0/3196515541 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c003d90 con 0x5591a613ee40 2024-05-27T07:34:42.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.650+0000 7f1e5ea00700 1 -- 172.21.15.63:0/3196515541 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f1e6c004830 con 0x5591a613ee40 2024-05-27T07:34:42.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.650+0000 7f1e8de52b80 1 -- 172.21.15.63:0/3196515541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e640a8eb0 con 0x5591a613ee40 2024-05-27T07:34:42.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.650+0000 7f1e5ea00700 1 -- 172.21.15.63:0/3196515541 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e6c024050 con 0x5591a613ee40 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.650+0000 7f1e8de52b80 10 client.15525 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.650+0000 7f1e8de52b80 1 -- 172.21.15.63:0/3196515541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5591a61ac060 con 0x5591a613ee40 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.651+0000 7f1e5ea00700 1 -- 172.21.15.63:0/3196515541 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e640a8eb0 con 0x5591a613ee40 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.651+0000 7f1e5ea00700 1 -- 172.21.15.63:0/3196515541 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1e6c011440 con 0x5591a613ee40 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.651+0000 7f1e8de52b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.651+0000 7f1e8de52b80 10 client.15525 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.651+0000 7f1e8de52b80 20 client.15525 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.651+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5591a62d7950 0x5591a62d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.651+0000 7f1e8de52b80 4 client.15525 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.651+0000 7f1e8de52b80 1 -- 172.21.15.63:0/3196515541 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591a61edd90 con 0x5591a62d7950 2024-05-27T07:34:42.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.652+0000 7f1e77e00700 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5591a62d7950 0x5591a62d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.653+0000 7f1e5ea00700 10 client.15525 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:34:42.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.653+0000 7f1e77e00700 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5591a62d7950 0x5591a62d9d90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a69b736e35035fe2 server_cookie=addd63c50c0f657a in_seq=0 out_seq=0 2024-05-27T07:34:42.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e5ea00700 1 -- 172.21.15.63:0/3196515541 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5591a61edd90 con 0x5591a62d7950 2024-05-27T07:34:42.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e5ea00700 10 client.15525 handle_command_reply: tid=0 2024-05-27T07:34:42.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e8de52b80 1 client.15525 shutdown 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e8de52b80 1 -- 172.21.15.63:0/3196515541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 msgr2=0x5591a62c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62c7590 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00bba0 tx=0x7f1e6c00bf60 comp rx=0 tx=0).stop 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e8de52b80 1 -- 172.21.15.63:0/3196515541 shutdown_connections 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd820 0x5591a62c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5591a62d7950 0x5591a62d9d90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e5ea00700 0 client.15525 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62c7590 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/3196515541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 0x5591a62c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.655+0000 7f1e8de52b80 1 -- 172.21.15.63:0/3196515541 >> 172.21.15.63:0/3196515541 conn(0x5591a619efc0 msgr2=0x5591a62b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.656+0000 7f1e8de52b80 1 -- 172.21.15.63:0/3196515541 shutdown_connections 2024-05-27T07:34:42.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.656+0000 7f1e8de52b80 1 -- 172.21.15.63:0/3196515541 wait complete. 2024-05-27T07:34:42.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.656+0000 7f1e8de52b80 20 client.15525 trim_cache size 0 max 16384 2024-05-27T07:34:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.672+0000 7f1e8de52b80 1 Processor -- start 2024-05-27T07:34:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.673+0000 7f1e8de52b80 1 -- start start 2024-05-27T07:34:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.673+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a619efc0 0x5591a62c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.673+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a613ee40 0x5591a62c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.673+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 0x5591a62c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.673+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781b4d30 con 0x5591a619efc0 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.673+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781b25c0 con 0x5591a62bd410 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.673+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781b0410 con 0x5591a613ee40 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.673+0000 7f1e7ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a619efc0 0x5591a62c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e7ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a619efc0 0x5591a62c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36232/0 (socket says 172.21.15.63:36232) 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e7ca00700 1 -- 172.21.15.63:0/309916787 learned_addr learned my addr 172.21.15.63:0/309916787 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e77e00700 1 --2- 172.21.15.63:0/309916787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a613ee40 0x5591a62c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e7d400700 1 --2- 172.21.15.63:0/309916787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 0x5591a62c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e7ca00700 1 -- 172.21.15.63:0/309916787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a613ee40 msgr2=0x5591a62c2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:42.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/309916787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a613ee40 0x5591a62c2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e7ca00700 1 -- 172.21.15.63:0/309916787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 msgr2=0x5591a62c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/309916787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 0x5591a62c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e7ca00700 1 -- 172.21.15.63:0/309916787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a62ca390 con 0x5591a619efc0 2024-05-27T07:34:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e7d400700 1 --2- 172.21.15.63:0/309916787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 0x5591a62c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.674+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/309916787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a619efc0 0x5591a62c68b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c011890 tx=0x7f1e6c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5412ad95df90bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.675+0000 7f1e69600700 1 -- 172.21.15.63:0/309916787 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c021040 con 0x5591a619efc0 2024-05-27T07:34:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.675+0000 7f1e69600700 1 -- 172.21.15.63:0/309916787 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e6c003e50 con 0x5591a619efc0 2024-05-27T07:34:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.675+0000 7f1e69600700 1 -- 172.21.15.63:0/309916787 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c0054d0 con 0x5591a619efc0 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.676+0000 7f1e8de52b80 1 -- 172.21.15.63:0/309916787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a619efc0 msgr2=0x5591a62c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.676+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/309916787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a619efc0 0x5591a62c68b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c011890 tx=0x7f1e6c00b480 comp rx=0 tx=0).stop 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.676+0000 7f1e8de52b80 1 -- 172.21.15.63:0/309916787 shutdown_connections 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.676+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/309916787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a62bd410 0x5591a62c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.676+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/309916787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a619efc0 0x5591a62c68b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.676+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/309916787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a613ee40 0x5591a62c2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.676+0000 7f1e8de52b80 1 -- 172.21.15.63:0/309916787 >> 172.21.15.63:0/309916787 conn(0x5591a62b1f60 msgr2=0x5591a62c7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.676+0000 7f1e8de52b80 1 -- 172.21.15.63:0/309916787 shutdown_connections 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.676+0000 7f1e8de52b80 1 -- 172.21.15.63:0/309916787 wait complete. 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.677+0000 7f1e8de52b80 1 -- start start 2024-05-27T07:34:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.677+0000 7f1e8de52b80 1 Processor -- start 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.677+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.677+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a619efc0 0x5591a62cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.677+0000 7f1e8de52b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd410 0x5591a62cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.677+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781b0a80 con 0x5591a613ee40 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.677+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781b1ef0 con 0x5591a619efc0 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.677+0000 7f1e8de52b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781b1150 con 0x5591a62bd410 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e8de52b80 10 client.0 _ll_register_callbacks cb 0x5591a6185c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e7ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e7ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36242/0 (socket says 172.21.15.63:36242) 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e7ca00700 1 -- 172.21.15.63:0/1968722545 learned_addr learned my addr 172.21.15.63:0/1968722545 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e77e00700 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a619efc0 0x5591a62cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e7d400700 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd410 0x5591a62cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e7ca00700 1 -- 172.21.15.63:0/1968722545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd410 msgr2=0x5591a62cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd410 0x5591a62cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e7ca00700 1 -- 172.21.15.63:0/1968722545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a619efc0 msgr2=0x5591a62cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a619efc0 0x5591a62cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e7ca00700 1 -- 172.21.15.63:0/1968722545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a62ba350 con 0x5591a613ee40 2024-05-27T07:34:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.678+0000 7f1e77e00700 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a619efc0 0x5591a62cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:34:42.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.679+0000 7f1e7ca00700 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62d01e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c011790 tx=0x7f1e6c012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c86f1b9ccb46cae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:42.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.679+0000 7f1e5ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:42.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.679+0000 7f1e5ea00700 1 -- 172.21.15.63:0/1968722545 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c036070 con 0x5591a613ee40 2024-05-27T07:34:42.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.679+0000 7f1e5ea00700 1 -- 172.21.15.63:0/1968722545 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f1e6c005420 con 0x5591a613ee40 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.679+0000 7f1e8de52b80 1 -- 172.21.15.63:0/1968722545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e6c003e50 con 0x5591a613ee40 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e5ea00700 1 -- 172.21.15.63:0/1968722545 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e6c042050 con 0x5591a613ee40 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e8de52b80 10 client.15531 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e8de52b80 1 -- 172.21.15.63:0/1968722545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5591a6341fd0 con 0x5591a613ee40 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e5ea00700 1 -- 172.21.15.63:0/1968722545 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e6c003e50 con 0x5591a613ee40 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e5ea00700 1 -- 172.21.15.63:0/1968722545 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1e6c01b470 con 0x5591a613ee40 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e8de52b80 10 client.15531 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e8de52b80 10 client.15531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e8de52b80 20 client.15531 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5591a62bd820 0x5591a6344630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e8de52b80 4 client.15531 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.680+0000 7f1e8de52b80 1 -- 172.21.15.63:0/1968722545 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591a61e7570 con 0x5591a62bd820 2024-05-27T07:34:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.681+0000 7f1e77e00700 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5591a62bd820 0x5591a6344630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:42.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.682+0000 7f1e77e00700 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5591a62bd820 0x5591a6344630 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2fb71d717f004b8d server_cookie=d7c3297ebca96985 in_seq=0 out_seq=0 2024-05-27T07:34:42.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.682+0000 7f1e5ea00700 10 client.15531 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:34:42.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.682+0000 7f1e5ea00700 1 -- 172.21.15.63:0/1968722545 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591a61e7570 con 0x5591a62bd820 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.682+0000 7f1e5ea00700 10 client.15531 handle_command_reply: tid=0 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.682+0000 7f1e8de52b80 1 client.15531 shutdown 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 1 -- 172.21.15.63:0/1968722545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 msgr2=0x5591a62d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62d01e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c011790 tx=0x7f1e6c012b40 comp rx=0 tx=0).stop 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 1 -- 172.21.15.63:0/1968722545 shutdown_connections 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5591a62bd410 0x5591a62cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5591a62bd820 0x5591a6344630 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5591a613ee40 0x5591a62d01e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 1 --2- 172.21.15.63:0/1968722545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591a619efc0 0x5591a62cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e5ea00700 0 client.15531 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 1 -- 172.21.15.63:0/1968722545 >> 172.21.15.63:0/1968722545 conn(0x5591a62b1f60 msgr2=0x5591a62c7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 1 -- 172.21.15.63:0/1968722545 shutdown_connections 2024-05-27T07:34:42.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 1 -- 172.21.15.63:0/1968722545 wait complete. 2024-05-27T07:34:42.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.683+0000 7f1e8de52b80 20 client.15531 trim_cache size 0 max 16384 2024-05-27T07:34:42.681 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 -- 172.21.15.63:0/1239291301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1e600b2c20 msgr2=0x7f1e600b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1e600b2c20 0x7f1e600b5130 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1e70005bd0 tx=0x7f1e70005b00 comp rx=0 tx=0).stop 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 -- 172.21.15.63:0/1239291301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e78107c00 msgr2=0x7f1e781051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e78107c00 0x7f1e781051e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400d980 tx=0x7f1e6400dc90 comp rx=0 tx=0).stop 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 -- 172.21.15.63:0/1239291301 shutdown_connections 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1e600b2c20 0x7f1e600b5130 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e78108550 0x7f1e78105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e781032d0 0x7f1e78104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 --2- 172.21.15.63:0/1239291301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e78107c00 0x7f1e781051e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 -- 172.21.15.63:0/1239291301 >> 172.21.15.63:0/1239291301 conn(0x7f1e78078ab0 msgr2=0x7f1e78106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 -- 172.21.15.63:0/1239291301 shutdown_connections 2024-05-27T07:34:42.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:42.684+0000 7f1e7de00700 1 -- 172.21.15.63:0/1239291301 wait complete. 2024-05-27T07:34:42.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:42 smithi086 ceph-mon[34687]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:42.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:42 smithi149 ceph-mon[35408]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:43.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:43 smithi063 ceph-mon[31613]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:43.667 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:34:43.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:43 smithi086 ceph-mon[34687]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:43.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:43 smithi149 ceph-mon[35408]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:44.128 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:34:46.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:46 smithi086 ceph-mon[34687]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:46.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:46 smithi149 ceph-mon[35408]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:46.709 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:46 smithi063 ceph-mon[31613]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.715+0000 7fb415e00700 1 -- 172.21.15.63:0/1675354296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb41010b890 msgr2=0x7fb4101131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.715+0000 7fb415e00700 1 --2- 172.21.15.63:0/1675354296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb41010b890 0x7fb4101131f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b0d0 tx=0x7fb40800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.715+0000 7fb415e00700 1 -- 172.21.15.63:0/1675354296 shutdown_connections 2024-05-27T07:34:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.715+0000 7fb415e00700 1 --2- 172.21.15.63:0/1675354296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb41010b890 0x7fb4101131f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.715+0000 7fb415e00700 1 --2- 172.21.15.63:0/1675354296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb410106610 0x7fb410106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.715+0000 7fb415e00700 1 --2- 172.21.15.63:0/1675354296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41010af40 0x7fb41010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.715+0000 7fb415e00700 1 -- 172.21.15.63:0/1675354296 >> 172.21.15.63:0/1675354296 conn(0x7fb410102020 msgr2=0x7fb410104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.715+0000 7fb415e00700 1 -- 172.21.15.63:0/1675354296 shutdown_connections 2024-05-27T07:34:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.715+0000 7fb415e00700 1 -- 172.21.15.63:0/1675354296 wait complete. 2024-05-27T07:34:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.717+0000 7fb415e00700 1 Processor -- start 2024-05-27T07:34:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.717+0000 7fb415e00700 1 -- start start 2024-05-27T07:34:46.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb415e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb410106610 0x7fb41019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb415e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb41010af40 0x7fb41019d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb415e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41010b890 0x7fb4101a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb415e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101163c0 con 0x7fb41010af40 2024-05-27T07:34:46.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb415e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb410116240 con 0x7fb410106610 2024-05-27T07:34:46.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb415e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb410116540 con 0x7fb41010b890 2024-05-27T07:34:46.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb40fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb41010af40 0x7fb41019d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb40fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb41010af40 0x7fb41019d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36274/0 (socket says 172.21.15.63:36274) 2024-05-27T07:34:46.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb40fe00700 1 -- 172.21.15.63:0/758270250 learned_addr learned my addr 172.21.15.63:0/758270250 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:46.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb414a00700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb410106610 0x7fb41019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.718+0000 7fb415400700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41010b890 0x7fb4101a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.719+0000 7fb40fe00700 1 -- 172.21.15.63:0/758270250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41010b890 msgr2=0x7fb4101a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:46.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.719+0000 7fb40fe00700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41010b890 0x7fb4101a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.719+0000 7fb40fe00700 1 -- 172.21.15.63:0/758270250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb410106610 msgr2=0x7fb41019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:46.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.719+0000 7fb40fe00700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb410106610 0x7fb41019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.719+0000 7fb40fe00700 1 -- 172.21.15.63:0/758270250 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4101a1c00 con 0x7fb41010af40 2024-05-27T07:34:46.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.719+0000 7fb40fe00700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb41010af40 0x7fb41019d2d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00d980 tx=0x7fb3fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:46.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.719+0000 7fb40d600700 1 -- 172.21.15.63:0/758270250 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc017070 con 0x7fb41010af40 2024-05-27T07:34:46.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.720+0000 7fb40d600700 1 -- 172.21.15.63:0/758270250 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3fc00fd20 con 0x7fb41010af40 2024-05-27T07:34:46.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.720+0000 7fb415e00700 1 -- 172.21.15.63:0/758270250 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4101a1ef0 con 0x7fb41010af40 2024-05-27T07:34:46.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.720+0000 7fb40d600700 1 -- 172.21.15.63:0/758270250 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc00e3b0 con 0x7fb41010af40 2024-05-27T07:34:46.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.720+0000 7fb415e00700 1 -- 172.21.15.63:0/758270250 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101a9810 con 0x7fb41010af40 2024-05-27T07:34:46.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.724+0000 7fb40d600700 1 -- 172.21.15.63:0/758270250 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7fb3fc00f890 con 0x7fb41010af40 2024-05-27T07:34:46.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.725+0000 7fb425e9fb80 1 Processor -- start 2024-05-27T07:34:46.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.725+0000 7fb425e9fb80 1 -- start start 2024-05-27T07:34:46.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.725+0000 7fb40d600700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3f80b2c20 0x7fb3f80b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.725+0000 7fb40d600700 1 -- 172.21.15.63:0/758270250 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb3fc0e25c0 con 0x7fb41010af40 2024-05-27T07:34:46.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.725+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d328c2e40 0x556d32a40ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.725+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41410 0x556d32a4bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.726+0000 7fb415400700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3f80b2c20 0x7fb3f80b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.726+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41820 0x556d32a4e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.726+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556d3296fe40 con 0x556d32a41820 2024-05-27T07:34:46.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.726+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d3295f9e0 con 0x556d328c2e40 2024-05-27T07:34:46.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.726+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556d32963a20 con 0x556d32a41410 2024-05-27T07:34:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.726+0000 7fb415400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41820 0x556d32a4e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.726+0000 7fb415400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41820 0x556d32a4e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36282/0 (socket says 172.21.15.63:36282) 2024-05-27T07:34:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.726+0000 7fb414a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d328c2e40 0x556d32a40ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.726+0000 7fb415400700 1 -- 172.21.15.63:0/3546969966 learned_addr learned my addr 172.21.15.63:0/3546969966 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.726+0000 7fb40fe00700 1 --2- 172.21.15.63:0/3546969966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41410 0x556d32a4bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.727+0000 7fb415400700 1 -- 172.21.15.63:0/3546969966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41410 msgr2=0x556d32a4bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.727+0000 7fb415400700 1 --2- 172.21.15.63:0/3546969966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41410 0x556d32a4bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.727+0000 7fb415400700 1 -- 172.21.15.63:0/3546969966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d328c2e40 msgr2=0x556d32a40ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.727+0000 7fb415400700 1 --2- 172.21.15.63:0/3546969966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d328c2e40 0x556d32a40ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.727+0000 7fb415400700 1 -- 172.21.15.63:0/3546969966 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d32928420 con 0x556d32a41820 2024-05-27T07:34:46.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.727+0000 7fb415400700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3f80b2c20 0x7fb3f80b5130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b0d0 tx=0x7fb408009ba0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:46.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.727+0000 7fb415400700 1 --2- 172.21.15.63:0/3546969966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41820 0x556d32a4e020 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b730 tx=0x7fb408027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5f76c8c3618ed5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:46.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.727+0000 7fb402000700 1 -- 172.21.15.63:0/3546969966 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408003cc0 con 0x556d32a41820 2024-05-27T07:34:46.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.728+0000 7fb402000700 1 -- 172.21.15.63:0/3546969966 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb408007d60 con 0x556d32a41820 2024-05-27T07:34:46.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.728+0000 7fb402000700 1 -- 172.21.15.63:0/3546969966 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40801ec50 con 0x556d32a41820 2024-05-27T07:34:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.729+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3546969966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41820 msgr2=0x556d32a4e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.729+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/3546969966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41820 0x556d32a4e020 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b730 tx=0x7fb408027000 comp rx=0 tx=0).stop 2024-05-27T07:34:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.729+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3546969966 shutdown_connections 2024-05-27T07:34:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.729+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/3546969966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41820 0x556d32a4e020 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.729+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/3546969966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d328c2e40 0x556d32a40ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.729+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/3546969966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41410 0x556d32a4bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.729+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3546969966 >> 172.21.15.63:0/3546969966 conn(0x556d32922fc0 msgr2=0x556d32a3cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.729+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3546969966 shutdown_connections 2024-05-27T07:34:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.729+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3546969966 wait complete. 2024-05-27T07:34:46.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.729+0000 7fb425e9fb80 1 -- start start 2024-05-27T07:34:46.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.730+0000 7fb425e9fb80 1 Processor -- start 2024-05-27T07:34:46.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.730+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d328c2e40 0x556d32a4b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.730+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a4bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.730+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41820 0x556d32a4c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.730+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556d32865fa0 con 0x556d32a41410 2024-05-27T07:34:46.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.730+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d32900760 con 0x556d32a41820 2024-05-27T07:34:46.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.730+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556d32a4e8d0 con 0x556d328c2e40 2024-05-27T07:34:46.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb425e9fb80 10 client.0 _ll_register_callbacks cb 0x556d32909c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:46.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb40fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a4bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb40fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a4bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36296/0 (socket says 172.21.15.63:36296) 2024-05-27T07:34:46.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb40fe00700 1 -- 172.21.15.63:0/3573149753 learned_addr learned my addr 172.21.15.63:0/3573149753 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:46.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb415400700 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41820 0x556d32a4c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb414a00700 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d328c2e40 0x556d32a4b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb40fe00700 1 -- 172.21.15.63:0/3573149753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d328c2e40 msgr2=0x556d32a4b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:46.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb40fe00700 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d328c2e40 0x556d32a4b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb40fe00700 1 -- 172.21.15.63:0/3573149753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41820 msgr2=0x556d32a4c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:46.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb40fe00700 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41820 0x556d32a4c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.731+0000 7fb40fe00700 1 -- 172.21.15.63:0/3573149753 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d328af8d0 con 0x556d32a41410 2024-05-27T07:34:46.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.732+0000 7fb40fe00700 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a4bb30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0a6560 tx=0x7fb3fc0a9830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3864972ec6717dbb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:46.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.732+0000 7fb3f6a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:46.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.732+0000 7fb3f6a00700 1 -- 172.21.15.63:0/3573149753 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc020070 con 0x556d32a41410 2024-05-27T07:34:46.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.732+0000 7fb3f6a00700 1 -- 172.21.15.63:0/3573149753 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb3fc0a8550 con 0x556d32a41410 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.732+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3573149753 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb408007d60 con 0x556d32a41410 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.732+0000 7fb3f6a00700 1 -- 172.21.15.63:0/3573149753 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb3fc0e23e0 con 0x556d32a41410 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.732+0000 7fb425e9fb80 10 client.15543 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.732+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3573149753 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d32930060 con 0x556d32a41410 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.733+0000 7fb3f6a00700 1 -- 172.21.15.63:0/3573149753 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb408007d60 con 0x556d32a41410 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.733+0000 7fb3f6a00700 1 -- 172.21.15.63:0/3573149753 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb3fc0ad120 con 0x556d32a41410 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.733+0000 7fb425e9fb80 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.733+0000 7fb425e9fb80 10 client.15543 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.733+0000 7fb425e9fb80 20 client.15543 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.733+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556d32a5b750 0x556d32a5dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.733+0000 7fb425e9fb80 4 client.15543 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.733+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3573149753 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d32971d90 con 0x556d32a5b750 2024-05-27T07:34:46.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.734+0000 7fb414a00700 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556d32a5b750 0x556d32a5dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.735+0000 7fb414a00700 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556d32a5b750 0x556d32a5dba0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75a5848934ac6d2b server_cookie=c653f407ca27bc97 in_seq=0 out_seq=0 2024-05-27T07:34:46.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.735+0000 7fb3f6a00700 10 client.15543 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:34:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb3f6a00700 1 -- 172.21.15.63:0/3573149753 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556d32971d90 con 0x556d32a5b750 2024-05-27T07:34:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb3f6a00700 10 client.15543 handle_command_reply: tid=0 2024-05-27T07:34:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 client.15543 shutdown 2024-05-27T07:34:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3573149753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 msgr2=0x556d32a4bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a4bb30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0a6560 tx=0x7fb3fc0a9830 comp rx=0 tx=0).stop 2024-05-27T07:34:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3573149753 shutdown_connections 2024-05-27T07:34:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41820 0x556d32a4c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d328c2e40 0x556d32a4b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556d32a5b750 0x556d32a5dba0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/3573149753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a4bb30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3573149753 >> 172.21.15.63:0/3573149753 conn(0x556d32922fc0 msgr2=0x556d32a3bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb3f6a00700 0 client.15543 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:34:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3573149753 shutdown_connections 2024-05-27T07:34:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.737+0000 7fb425e9fb80 1 -- 172.21.15.63:0/3573149753 wait complete. 2024-05-27T07:34:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.738+0000 7fb425e9fb80 20 client.15543 trim_cache size 0 max 16384 2024-05-27T07:34:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.756+0000 7fb425e9fb80 1 Processor -- start 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.756+0000 7fb425e9fb80 1 -- start start 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.756+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d328c2e40 0x556d32a3ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.756+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41410 0x556d32a4d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.756+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41820 0x556d32a57970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.757+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b4c70 con 0x556d328c2e40 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.757+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b2500 con 0x556d32a41410 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.757+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b0350 con 0x556d32a41820 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.757+0000 7fb414a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d328c2e40 0x556d32a3ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.757+0000 7fb414a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d328c2e40 0x556d32a3ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36312/0 (socket says 172.21.15.63:36312) 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.757+0000 7fb414a00700 1 -- 172.21.15.63:0/594816981 learned_addr learned my addr 172.21.15.63:0/594816981 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.757+0000 7fb40fe00700 1 --2- 172.21.15.63:0/594816981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41410 0x556d32a4d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.757+0000 7fb415400700 1 --2- 172.21.15.63:0/594816981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41820 0x556d32a57970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.757+0000 7fb414a00700 1 -- 172.21.15.63:0/594816981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41820 msgr2=0x556d32a57970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:46.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.757+0000 7fb414a00700 1 --2- 172.21.15.63:0/594816981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41820 0x556d32a57970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.758+0000 7fb414a00700 1 -- 172.21.15.63:0/594816981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41410 msgr2=0x556d32a4d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:46.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.758+0000 7fb414a00700 1 --2- 172.21.15.63:0/594816981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41410 0x556d32a4d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.758+0000 7fb414a00700 1 -- 172.21.15.63:0/594816981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d32a4c6b0 con 0x556d328c2e40 2024-05-27T07:34:46.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.758+0000 7fb40fe00700 1 --2- 172.21.15.63:0/594816981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41410 0x556d32a4d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:34:46.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.758+0000 7fb414a00700 1 --2- 172.21.15.63:0/594816981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d328c2e40 0x556d32a3ce30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x556d32a5e2d0 tx=0x7fb404009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1394731e9ff9b5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:46.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.758+0000 7fb401600700 1 -- 172.21.15.63:0/594816981 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404011e20 con 0x556d328c2e40 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.758+0000 7fb401600700 1 -- 172.21.15.63:0/594816981 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb404004cc0 con 0x556d328c2e40 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.758+0000 7fb401600700 1 -- 172.21.15.63:0/594816981 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40401e680 con 0x556d328c2e40 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.759+0000 7fb425e9fb80 1 -- 172.21.15.63:0/594816981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d328c2e40 msgr2=0x556d32a3ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.759+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/594816981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d328c2e40 0x556d32a3ce30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x556d32a5e2d0 tx=0x7fb404009260 comp rx=0 tx=0).stop 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.759+0000 7fb425e9fb80 1 -- 172.21.15.63:0/594816981 shutdown_connections 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.759+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/594816981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41820 0x556d32a57970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.759+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/594816981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d328c2e40 0x556d32a3ce30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.759+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/594816981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d32a41410 0x556d32a4d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.759+0000 7fb425e9fb80 1 -- 172.21.15.63:0/594816981 >> 172.21.15.63:0/594816981 conn(0x556d32922fc0 msgr2=0x556d32a4b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.760+0000 7fb425e9fb80 1 -- 172.21.15.63:0/594816981 shutdown_connections 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.760+0000 7fb425e9fb80 1 -- 172.21.15.63:0/594816981 wait complete. 2024-05-27T07:34:46.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.760+0000 7fb425e9fb80 1 -- start start 2024-05-27T07:34:46.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.760+0000 7fb425e9fb80 1 Processor -- start 2024-05-27T07:34:46.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.761+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d328c2e40 0x556d32a540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.761+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.761+0000 7fb425e9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41820 0x556d32a48660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.761+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b09c0 con 0x556d32a41410 2024-05-27T07:34:46.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.761+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b1e30 con 0x556d328c2e40 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.761+0000 7fb425e9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b1090 con 0x556d32a41820 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.761+0000 7fb425e9fb80 10 client.0 _ll_register_callbacks cb 0x556d32909c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.761+0000 7fb40fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.761+0000 7fb40fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a48120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36314/0 (socket says 172.21.15.63:36314) 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.761+0000 7fb40fe00700 1 -- 172.21.15.63:0/90086997 learned_addr learned my addr 172.21.15.63:0/90086997 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.762+0000 7fb414a00700 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d328c2e40 0x556d32a540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.762+0000 7fb415400700 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41820 0x556d32a48660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.762+0000 7fb40fe00700 1 -- 172.21.15.63:0/90086997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41820 msgr2=0x556d32a48660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.762+0000 7fb40fe00700 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41820 0x556d32a48660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.762+0000 7fb40fe00700 1 -- 172.21.15.63:0/90086997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d328c2e40 msgr2=0x556d32a540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.762+0000 7fb40fe00700 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d328c2e40 0x556d32a540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.762+0000 7fb40fe00700 1 -- 172.21.15.63:0/90086997 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d32a48c20 con 0x556d32a41410 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.762+0000 7fb40fe00700 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a48120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0a8ff0 tx=0x7fb3fc0ca450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ece03aab36ea637 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.762+0000 7fb3f6a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.763+0000 7fb3f6a00700 1 -- 172.21.15.63:0/90086997 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc020030 con 0x556d32a41410 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.763+0000 7fb3f6a00700 1 -- 172.21.15.63:0/90086997 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb3fc0c3df0 con 0x556d32a41410 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.763+0000 7fb425e9fb80 1 -- 172.21.15.63:0/90086997 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb404004cc0 con 0x556d32a41410 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.763+0000 7fb3f6a00700 1 -- 172.21.15.63:0/90086997 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb3fc0b2150 con 0x556d32a41410 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.763+0000 7fb425e9fb80 10 client.15549 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.763+0000 7fb425e9fb80 1 -- 172.21.15.63:0/90086997 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d32ac5fd0 con 0x556d32a41410 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.763+0000 7fb3f6a00700 1 -- 172.21.15.63:0/90086997 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb404004cc0 con 0x556d32a41410 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.763+0000 7fb3f6a00700 1 -- 172.21.15.63:0/90086997 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb3fc0a4c60 con 0x556d32a41410 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.763+0000 7fb425e9fb80 10 client.15549 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.764+0000 7fb425e9fb80 10 client.15549 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:34:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.764+0000 7fb425e9fb80 20 client.15549 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.764+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556d32ac6490 0x556d32ac88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.764+0000 7fb425e9fb80 4 client.15549 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.764+0000 7fb425e9fb80 1 -- 172.21.15.63:0/90086997 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d3296b570 con 0x556d32ac6490 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.764+0000 7fb414a00700 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556d32ac6490 0x556d32ac88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.765+0000 7fb414a00700 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556d32ac6490 0x556d32ac88b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=515d32f43604f096 server_cookie=bfed53756c521286 in_seq=0 out_seq=0 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.765+0000 7fb3f6a00700 10 client.15549 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb3f6a00700 1 -- 172.21.15.63:0/90086997 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d3296b570 con 0x556d32ac6490 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb3f6a00700 10 client.15549 handle_command_reply: tid=0 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 client.15549 shutdown 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 -- 172.21.15.63:0/90086997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 msgr2=0x556d32a48120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a48120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0a8ff0 tx=0x7fb3fc0ca450 comp rx=0 tx=0).stop 2024-05-27T07:34:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 -- 172.21.15.63:0/90086997 shutdown_connections 2024-05-27T07:34:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d32a41820 0x556d32a48660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d328c2e40 0x556d32a540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556d32ac6490 0x556d32ac88b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 --2- 172.21.15.63:0/90086997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d32a41410 0x556d32a48120 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 -- 172.21.15.63:0/90086997 >> 172.21.15.63:0/90086997 conn(0x556d32922fc0 msgr2=0x556d32a4b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb3f6a00700 0 client.15549 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:34:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 -- 172.21.15.63:0/90086997 shutdown_connections 2024-05-27T07:34:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 1 -- 172.21.15.63:0/90086997 wait complete. 2024-05-27T07:34:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.766+0000 7fb425e9fb80 20 client.15549 trim_cache size 0 max 16384 2024-05-27T07:34:46.766 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:34:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.767+0000 7fb415e00700 1 -- 172.21.15.63:0/758270250 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3f80b2c20 msgr2=0x7fb3f80b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.767+0000 7fb415e00700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3f80b2c20 0x7fb3f80b5130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b0d0 tx=0x7fb408009ba0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.767+0000 7fb415e00700 1 -- 172.21.15.63:0/758270250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb41010af40 msgr2=0x7fb41019d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.767+0000 7fb415e00700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb41010af40 0x7fb41019d2d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00d980 tx=0x7fb3fc00dc90 comp rx=0 tx=0).stop 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.767+0000 7fb415e00700 1 -- 172.21.15.63:0/758270250 shutdown_connections 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.767+0000 7fb415e00700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3f80b2c20 0x7fb3f80b5130 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.767+0000 7fb415e00700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb41010b890 0x7fb4101a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.767+0000 7fb415e00700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb410106610 0x7fb41019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.767+0000 7fb415e00700 1 --2- 172.21.15.63:0/758270250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb41010af40 0x7fb41019d2d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.767+0000 7fb415e00700 1 -- 172.21.15.63:0/758270250 >> 172.21.15.63:0/758270250 conn(0x7fb410102020 msgr2=0x7fb41010a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.768+0000 7fb415e00700 1 -- 172.21.15.63:0/758270250 shutdown_connections 2024-05-27T07:34:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:46.768+0000 7fb415e00700 1 -- 172.21.15.63:0/758270250 wait complete. 2024-05-27T07:34:47.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:47 smithi063 ceph-mon[31613]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:47.699 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:34:47.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:47 smithi086 ceph-mon[34687]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:47.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:47 smithi149 ceph-mon[35408]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:48.179 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:34:50.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.607+0000 7f855a600700 1 -- 172.21.15.63:0/1917718366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8554107de0 msgr2=0x7f85541081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.607+0000 7f855a600700 1 --2- 172.21.15.63:0/1917718366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8554107de0 0x7f85541081f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f85400099c0 tx=0x7f8540009cd0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.607+0000 7f855a600700 1 -- 172.21.15.63:0/1917718366 shutdown_connections 2024-05-27T07:34:50.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.607+0000 7f855a600700 1 --2- 172.21.15.63:0/1917718366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8554108730 0x7f8554110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.607+0000 7f855a600700 1 --2- 172.21.15.63:0/1917718366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85541034b0 0x7f85541038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.607+0000 7f855a600700 1 --2- 172.21.15.63:0/1917718366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8554107de0 0x7f85541081f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.607+0000 7f855a600700 1 -- 172.21.15.63:0/1917718366 >> 172.21.15.63:0/1917718366 conn(0x7f85540fede0 msgr2=0x7f8554101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:50.604 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:50 smithi063 ceph-mon[31613]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:50.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.608+0000 7f855a600700 1 -- 172.21.15.63:0/1917718366 shutdown_connections 2024-05-27T07:34:50.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.609+0000 7f855a600700 1 -- 172.21.15.63:0/1917718366 wait complete. 2024-05-27T07:34:50.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.610+0000 7f855a600700 1 Processor -- start 2024-05-27T07:34:50.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.610+0000 7f855a600700 1 -- start start 2024-05-27T07:34:50.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.611+0000 7f855a600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85541034b0 0x7f855419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.611+0000 7f855a600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8554107de0 0x7f855419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.611+0000 7f855a600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8554108730 0x7f855419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.611+0000 7f855a600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85541131f0 con 0x7f85541034b0 2024-05-27T07:34:50.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.611+0000 7f855a600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8554113070 con 0x7f8554107de0 2024-05-27T07:34:50.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.611+0000 7f855a600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8554113370 con 0x7f8554108730 2024-05-27T07:34:50.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.611+0000 7f8553400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85541034b0 0x7f855419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.611+0000 7f8553400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85541034b0 0x7f855419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37838/0 (socket says 172.21.15.63:37838) 2024-05-27T07:34:50.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.611+0000 7f8553400700 1 -- 172.21.15.63:0/4129320616 learned_addr learned my addr 172.21.15.63:0/4129320616 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:50.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.612+0000 7f8552a00700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8554107de0 0x7f855419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.612+0000 7f8553400700 1 -- 172.21.15.63:0/4129320616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8554108730 msgr2=0x7f855419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.612+0000 7f8553400700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8554108730 0x7f855419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.612+0000 7f8553400700 1 -- 172.21.15.63:0/4129320616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8554107de0 msgr2=0x7f855419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.612+0000 7f8553400700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8554107de0 0x7f855419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.612+0000 7f8553400700 1 -- 172.21.15.63:0/4129320616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855419fbb0 con 0x7f85541034b0 2024-05-27T07:34:50.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.612+0000 7f8553400700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85541034b0 0x7f855419ad90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f854400d950 tx=0x7f854400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:50.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.613+0000 7f854be00700 1 -- 172.21.15.63:0/4129320616 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544016070 con 0x7f85541034b0 2024-05-27T07:34:50.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.613+0000 7f854be00700 1 -- 172.21.15.63:0/4129320616 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8544010460 con 0x7f85541034b0 2024-05-27T07:34:50.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.613+0000 7f855a600700 1 -- 172.21.15.63:0/4129320616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855419fea0 con 0x7f85541034b0 2024-05-27T07:34:50.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.613+0000 7f854be00700 1 -- 172.21.15.63:0/4129320616 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854400f620 con 0x7f85541034b0 2024-05-27T07:34:50.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.613+0000 7f855a600700 1 -- 172.21.15.63:0/4129320616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85541a03b0 con 0x7f85541034b0 2024-05-27T07:34:50.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.618+0000 7f854be00700 1 -- 172.21.15.63:0/4129320616 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f8544019070 con 0x7f85541034b0 2024-05-27T07:34:50.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.618+0000 7f85690f9b80 1 Processor -- start 2024-05-27T07:34:50.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.619+0000 7f85690f9b80 1 -- start start 2024-05-27T07:34:50.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.619+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 0x56269d43b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.619+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d43bd30 0x56269d435870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.619+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d435db0 0x56269d4361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.619+0000 7f85690f9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56269d363e40 con 0x56269d43bd30 2024-05-27T07:34:50.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.619+0000 7f85690f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56269d3539e0 con 0x56269d435db0 2024-05-27T07:34:50.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.619+0000 7f85690f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56269d357a20 con 0x56269d2b6e40 2024-05-27T07:34:50.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f8553400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d43bd30 0x56269d435870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f8553400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d43bd30 0x56269d435870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37848/0 (socket says 172.21.15.63:37848) 2024-05-27T07:34:50.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f8553400700 1 -- 172.21.15.63:0/3946658106 learned_addr learned my addr 172.21.15.63:0/3946658106 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:50.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f8552a00700 1 --2- 172.21.15.63:0/3946658106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d435db0 0x56269d4361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f8553e00700 1 --2- 172.21.15.63:0/3946658106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 0x56269d43b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f854be00700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85380b2ba0 0x7f85380b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f854be00700 1 -- 172.21.15.63:0/4129320616 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f85440e2840 con 0x7f85541034b0 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f8553400700 1 -- 172.21.15.63:0/3946658106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 msgr2=0x56269d43b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f8553400700 1 --2- 172.21.15.63:0/3946658106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 0x56269d43b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f8553400700 1 -- 172.21.15.63:0/3946658106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d435db0 msgr2=0x56269d4361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f8553400700 1 --2- 172.21.15.63:0/3946658106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d435db0 0x56269d4361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.620+0000 7f8553400700 1 -- 172.21.15.63:0/3946658106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56269d31c420 con 0x56269d43bd30 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.621+0000 7f8553e00700 1 --2- 172.21.15.63:0/3946658106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 0x56269d43b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.621+0000 7f8553e00700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85380b2ba0 0x7f85380b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.621+0000 7f8553400700 1 --2- 172.21.15.63:0/3946658106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d43bd30 0x56269d435870 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f85440df950 tx=0x7f8544000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64f469ca4103d1d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.621+0000 7f8548c00700 1 -- 172.21.15.63:0/3946658106 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85440a8ec0 con 0x56269d43bd30 2024-05-27T07:34:50.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.621+0000 7f8548c00700 1 -- 172.21.15.63:0/3946658106 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85440a9080 con 0x56269d43bd30 2024-05-27T07:34:50.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.622+0000 7f8548c00700 1 -- 172.21.15.63:0/3946658106 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85440a8ce0 con 0x56269d43bd30 2024-05-27T07:34:50.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.622+0000 7f85690f9b80 1 -- 172.21.15.63:0/3946658106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d43bd30 msgr2=0x56269d435870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.622+0000 7f8553e00700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85380b2ba0 0x7f85380b50b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x56269d436b30 tx=0x7f854c004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:50.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.622+0000 7f85690f9b80 1 --2- 172.21.15.63:0/3946658106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d43bd30 0x56269d435870 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f85440df950 tx=0x7f8544000bc0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.623+0000 7f85690f9b80 1 -- 172.21.15.63:0/3946658106 shutdown_connections 2024-05-27T07:34:50.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.623+0000 7f85690f9b80 1 --2- 172.21.15.63:0/3946658106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d435db0 0x56269d4361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.623+0000 7f85690f9b80 1 --2- 172.21.15.63:0/3946658106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 0x56269d43b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.623+0000 7f85690f9b80 1 --2- 172.21.15.63:0/3946658106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d43bd30 0x56269d435870 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.623+0000 7f85690f9b80 1 -- 172.21.15.63:0/3946658106 >> 172.21.15.63:0/3946658106 conn(0x56269d316fc0 msgr2=0x56269d4308d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:50.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.623+0000 7f85690f9b80 1 -- 172.21.15.63:0/3946658106 shutdown_connections 2024-05-27T07:34:50.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.623+0000 7f85690f9b80 1 -- 172.21.15.63:0/3946658106 wait complete. 2024-05-27T07:34:50.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.623+0000 7f85690f9b80 1 -- start start 2024-05-27T07:34:50.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.624+0000 7f85690f9b80 1 Processor -- start 2024-05-27T07:34:50.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.624+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 0x56269d43f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.624+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d43f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.624+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d43bd30 0x56269d43ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.624+0000 7f85690f9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56269d259fa0 con 0x56269d435db0 2024-05-27T07:34:50.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.624+0000 7f85690f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56269d2f4760 con 0x56269d43bd30 2024-05-27T07:34:50.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.624+0000 7f85690f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56269d4427a0 con 0x56269d2b6e40 2024-05-27T07:34:50.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.625+0000 7f85690f9b80 10 client.0 _ll_register_callbacks cb 0x56269d2fdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:50.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.625+0000 7f8552a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d43f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.625+0000 7f8552a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d43f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37858/0 (socket says 172.21.15.63:37858) 2024-05-27T07:34:50.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.625+0000 7f8552a00700 1 -- 172.21.15.63:0/1049115225 learned_addr learned my addr 172.21.15.63:0/1049115225 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:50.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.625+0000 7f8553400700 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 0x56269d43f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.625+0000 7f8553e00700 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d43bd30 0x56269d43ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.626+0000 7f8552a00700 1 -- 172.21.15.63:0/1049115225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 msgr2=0x56269d43f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:50.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.626+0000 7f8552a00700 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 0x56269d43f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.626+0000 7f8552a00700 1 -- 172.21.15.63:0/1049115225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d43bd30 msgr2=0x56269d43ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.626+0000 7f8552a00700 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d43bd30 0x56269d43ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.626+0000 7f8552a00700 1 -- 172.21.15.63:0/1049115225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56269d2a38d0 con 0x56269d435db0 2024-05-27T07:34:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.626+0000 7f8552a00700 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d43f9e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8554076b30 tx=0x7f85400057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1116a11b85f019 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.626+0000 7f853d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.626+0000 7f853d600700 1 -- 172.21.15.63:0/1049115225 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540027060 con 0x56269d435db0 2024-05-27T07:34:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.627+0000 7f853d600700 1 -- 172.21.15.63:0/1049115225 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f854000bba0 con 0x56269d435db0 2024-05-27T07:34:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.627+0000 7f85690f9b80 1 -- 172.21.15.63:0/1049115225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85440a9080 con 0x56269d435db0 2024-05-27T07:34:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.627+0000 7f853d600700 1 -- 172.21.15.63:0/1049115225 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f854002e050 con 0x56269d435db0 2024-05-27T07:34:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.627+0000 7f85690f9b80 10 client.15561 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:50.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.627+0000 7f85690f9b80 1 -- 172.21.15.63:0/1049115225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56269d324060 con 0x56269d435db0 2024-05-27T07:34:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.628+0000 7f853d600700 1 -- 172.21.15.63:0/1049115225 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85440a9080 con 0x56269d435db0 2024-05-27T07:34:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.628+0000 7f853d600700 1 -- 172.21.15.63:0/1049115225 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f8540021070 con 0x56269d435db0 2024-05-27T07:34:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.628+0000 7f85690f9b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.628+0000 7f85690f9b80 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:50.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.628+0000 7f85690f9b80 20 client.15561 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.628+0000 7f85690f9b80 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56269d457e80 0x56269d45a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.628+0000 7f85690f9b80 4 client.15561 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:34:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.628+0000 7f85690f9b80 1 -- 172.21.15.63:0/1049115225 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56269d365d90 con 0x56269d457e80 2024-05-27T07:34:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.629+0000 7f8553400700 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56269d457e80 0x56269d45a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.629+0000 7f8553400700 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56269d457e80 0x56269d45a2a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c24ecf5b7ee71ac0 server_cookie=23f01e7743005b8f in_seq=0 out_seq=0 2024-05-27T07:34:50.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.629+0000 7f853d600700 10 client.15561 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:50.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.631+0000 7f853d600700 1 -- 172.21.15.63:0/1049115225 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56269d365d90 con 0x56269d457e80 2024-05-27T07:34:50.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.631+0000 7f853d600700 10 client.15561 handle_command_reply: tid=0 2024-05-27T07:34:50.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.631+0000 7f85690f9b80 1 client.15561 shutdown 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 1 -- 172.21.15.63:0/1049115225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 msgr2=0x56269d43f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d43f9e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8554076b30 tx=0x7f85400057c0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 1 -- 172.21.15.63:0/1049115225 shutdown_connections 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56269d457e80 0x56269d45a2a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d43bd30 0x56269d43ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d2b6e40 0x56269d43f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 1 --2- 172.21.15.63:0/1049115225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d43f9e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f853d600700 0 client.15561 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 1 -- 172.21.15.63:0/1049115225 >> 172.21.15.63:0/1049115225 conn(0x56269d316fc0 msgr2=0x56269d42fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 1 -- 172.21.15.63:0/1049115225 shutdown_connections 2024-05-27T07:34:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 1 -- 172.21.15.63:0/1049115225 wait complete. 2024-05-27T07:34:50.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.632+0000 7f85690f9b80 20 client.15561 trim_cache size 0 max 16384 2024-05-27T07:34:50.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.652+0000 7f85690f9b80 1 Processor -- start 2024-05-27T07:34:50.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.652+0000 7f85690f9b80 1 -- start start 2024-05-27T07:34:50.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.653+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d2b6e40 0x56269d435870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.653+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d435db0 0x56269d43e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.653+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d43bd30 0x56269d441940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.653+0000 7f85690f9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85541b09d0 con 0x56269d2b6e40 2024-05-27T07:34:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.653+0000 7f85690f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85541ae260 con 0x56269d43bd30 2024-05-27T07:34:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.653+0000 7f85690f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85541ac0d0 con 0x56269d435db0 2024-05-27T07:34:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.653+0000 7f8553400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d2b6e40 0x56269d435870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.653+0000 7f8553400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d2b6e40 0x56269d435870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37860/0 (socket says 172.21.15.63:37860) 2024-05-27T07:34:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.653+0000 7f8553400700 1 -- 172.21.15.63:0/2302548824 learned_addr learned my addr 172.21.15.63:0/2302548824 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.653+0000 7f8553e00700 1 --2- 172.21.15.63:0/2302548824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d43bd30 0x56269d441940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.654+0000 7f8552a00700 1 --2- 172.21.15.63:0/2302548824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d435db0 0x56269d43e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.654+0000 7f8553400700 1 -- 172.21.15.63:0/2302548824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d435db0 msgr2=0x56269d43e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:50.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.654+0000 7f8553400700 1 --2- 172.21.15.63:0/2302548824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d435db0 0x56269d43e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.654+0000 7f8553400700 1 -- 172.21.15.63:0/2302548824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d43bd30 msgr2=0x56269d441940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:50.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.654+0000 7f8553400700 1 --2- 172.21.15.63:0/2302548824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d43bd30 0x56269d441940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.654+0000 7f8553400700 1 -- 172.21.15.63:0/2302548824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56269d440560 con 0x56269d2b6e40 2024-05-27T07:34:50.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.654+0000 7f8553400700 1 --2- 172.21.15.63:0/2302548824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d2b6e40 0x56269d435870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f85440a67e0 tx=0x7f8544005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb749bd35b80def server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:50.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.654+0000 7f853fe00700 1 -- 172.21.15.63:0/2302548824 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85440c3070 con 0x56269d2b6e40 2024-05-27T07:34:50.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.654+0000 7f853fe00700 1 -- 172.21.15.63:0/2302548824 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85440ad920 con 0x56269d2b6e40 2024-05-27T07:34:50.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.654+0000 7f853fe00700 1 -- 172.21.15.63:0/2302548824 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85440a97d0 con 0x56269d2b6e40 2024-05-27T07:34:50.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.655+0000 7f85690f9b80 1 -- 172.21.15.63:0/2302548824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d2b6e40 msgr2=0x56269d435870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.655+0000 7f85690f9b80 1 --2- 172.21.15.63:0/2302548824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d2b6e40 0x56269d435870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f85440a67e0 tx=0x7f8544005910 comp rx=0 tx=0).stop 2024-05-27T07:34:50.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.655+0000 7f85690f9b80 1 -- 172.21.15.63:0/2302548824 shutdown_connections 2024-05-27T07:34:50.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.655+0000 7f85690f9b80 1 --2- 172.21.15.63:0/2302548824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d43bd30 0x56269d441940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.655+0000 7f85690f9b80 1 --2- 172.21.15.63:0/2302548824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d2b6e40 0x56269d435870 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.655+0000 7f85690f9b80 1 --2- 172.21.15.63:0/2302548824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d435db0 0x56269d43e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.655+0000 7f85690f9b80 1 -- 172.21.15.63:0/2302548824 >> 172.21.15.63:0/2302548824 conn(0x56269d316fc0 msgr2=0x56269d439210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:50.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.656+0000 7f85690f9b80 1 -- 172.21.15.63:0/2302548824 shutdown_connections 2024-05-27T07:34:50.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.656+0000 7f85690f9b80 1 -- 172.21.15.63:0/2302548824 wait complete. 2024-05-27T07:34:50.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.656+0000 7f85690f9b80 1 -- start start 2024-05-27T07:34:50.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.657+0000 7f85690f9b80 1 Processor -- start 2024-05-27T07:34:50.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.657+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d2b6e40 0x56269d450950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.657+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d45f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.657+0000 7f85690f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d43bd30 0x56269d45f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f85690f9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85541ac720 con 0x56269d435db0 2024-05-27T07:34:50.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f85690f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85541adb90 con 0x56269d2b6e40 2024-05-27T07:34:50.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f85690f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85541acdf0 con 0x56269d43bd30 2024-05-27T07:34:50.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f85690f9b80 10 client.0 _ll_register_callbacks cb 0x56269d2fdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:50.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8552a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d45f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8552a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d45f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37876/0 (socket says 172.21.15.63:37876) 2024-05-27T07:34:50.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8552a00700 1 -- 172.21.15.63:0/3686491360 learned_addr learned my addr 172.21.15.63:0/3686491360 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:50.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8553400700 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d2b6e40 0x56269d450950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8553e00700 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d43bd30 0x56269d45f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8552a00700 1 -- 172.21.15.63:0/3686491360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d43bd30 msgr2=0x56269d45f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8552a00700 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d43bd30 0x56269d45f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8552a00700 1 -- 172.21.15.63:0/3686491360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d2b6e40 msgr2=0x56269d450950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8552a00700 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d2b6e40 0x56269d450950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8552a00700 1 -- 172.21.15.63:0/3686491360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56269d45fd30 con 0x56269d435db0 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.658+0000 7f8553400700 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d2b6e40 0x56269d450950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.659+0000 7f8552a00700 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d45f1b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8540005a60 tx=0x7f8540009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5596045d019a3845 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.659+0000 7f853d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.659+0000 7f853d600700 1 -- 172.21.15.63:0/3686491360 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540027050 con 0x56269d435db0 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.659+0000 7f85690f9b80 1 -- 172.21.15.63:0/3686491360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85440ad920 con 0x56269d435db0 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.659+0000 7f853d600700 1 -- 172.21.15.63:0/3686491360 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f854000bba0 con 0x56269d435db0 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.659+0000 7f853d600700 1 -- 172.21.15.63:0/3686491360 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f854000bda0 con 0x56269d435db0 2024-05-27T07:34:50.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.659+0000 7f85690f9b80 10 client.15567 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.659+0000 7f85690f9b80 1 -- 172.21.15.63:0/3686491360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56269d4c26d0 con 0x56269d435db0 2024-05-27T07:34:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.660+0000 7f853d600700 1 -- 172.21.15.63:0/3686491360 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85440ad920 con 0x56269d435db0 2024-05-27T07:34:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.660+0000 7f853d600700 1 -- 172.21.15.63:0/3686491360 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f854000fce0 con 0x56269d435db0 2024-05-27T07:34:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.660+0000 7f85690f9b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.660+0000 7f85690f9b80 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.660+0000 7f85690f9b80 20 client.15567 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.660+0000 7f85690f9b80 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56269d4c2d50 0x56269d4c5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.660+0000 7f85690f9b80 4 client.15567 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:34:50.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.660+0000 7f85690f9b80 1 -- 172.21.15.63:0/3686491360 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56269d35f570 con 0x56269d4c2d50 2024-05-27T07:34:50.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.661+0000 7f8553400700 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56269d4c2d50 0x56269d4c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:50.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.662+0000 7f8553400700 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56269d4c2d50 0x56269d4c5170 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f3be5dbedf1d059 server_cookie=edd4c68a211dc7fb in_seq=0 out_seq=0 2024-05-27T07:34:50.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.662+0000 7f853d600700 10 client.15567 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:50.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f853d600700 1 -- 172.21.15.63:0/3686491360 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56269d35f570 con 0x56269d4c2d50 2024-05-27T07:34:50.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f853d600700 10 client.15567 handle_command_reply: tid=0 2024-05-27T07:34:50.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f85690f9b80 1 client.15567 shutdown 2024-05-27T07:34:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f85690f9b80 1 -- 172.21.15.63:0/3686491360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 msgr2=0x56269d45f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f85690f9b80 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d45f1b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8540005a60 tx=0x7f8540009310 comp rx=0 tx=0).stop 2024-05-27T07:34:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f85690f9b80 1 -- 172.21.15.63:0/3686491360 shutdown_connections 2024-05-27T07:34:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f85690f9b80 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56269d4c2d50 0x56269d4c5170 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f85690f9b80 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56269d43bd30 0x56269d45f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f85690f9b80 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56269d2b6e40 0x56269d450950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f85690f9b80 1 --2- 172.21.15.63:0/3686491360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56269d435db0 0x56269d45f1b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f85690f9b80 1 -- 172.21.15.63:0/3686491360 >> 172.21.15.63:0/3686491360 conn(0x56269d316fc0 msgr2=0x56269d42fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f853d600700 0 client.15567 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.663+0000 7f85690f9b80 1 -- 172.21.15.63:0/3686491360 shutdown_connections 2024-05-27T07:34:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.664+0000 7f85690f9b80 1 -- 172.21.15.63:0/3686491360 wait complete. 2024-05-27T07:34:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.664+0000 7f85690f9b80 20 client.15567 trim_cache size 0 max 16384 2024-05-27T07:34:50.660 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:34:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.664+0000 7f855a600700 1 -- 172.21.15.63:0/4129320616 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85380b2ba0 msgr2=0x7f85380b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.664+0000 7f855a600700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85380b2ba0 0x7f85380b50b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x56269d436b30 tx=0x7f854c004ad0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.664+0000 7f855a600700 1 -- 172.21.15.63:0/4129320616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85541034b0 msgr2=0x7f855419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.664+0000 7f855a600700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85541034b0 0x7f855419ad90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f854400d950 tx=0x7f854400dd10 comp rx=0 tx=0).stop 2024-05-27T07:34:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.665+0000 7f855a600700 1 -- 172.21.15.63:0/4129320616 shutdown_connections 2024-05-27T07:34:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.665+0000 7f855a600700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85380b2ba0 0x7f85380b50b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.665+0000 7f855a600700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8554108730 0x7f855419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.665+0000 7f855a600700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85541034b0 0x7f855419ad90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.665+0000 7f855a600700 1 --2- 172.21.15.63:0/4129320616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8554107de0 0x7f855419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.665+0000 7f855a600700 1 -- 172.21.15.63:0/4129320616 >> 172.21.15.63:0/4129320616 conn(0x7f85540fede0 msgr2=0x7f8554075e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.665+0000 7f855a600700 1 -- 172.21.15.63:0/4129320616 shutdown_connections 2024-05-27T07:34:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:50.665+0000 7f855a600700 1 -- 172.21.15.63:0/4129320616 wait complete. 2024-05-27T07:34:50.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:50 smithi086 ceph-mon[34687]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:50.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:50 smithi149 ceph-mon[35408]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:34:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.291+0000 7f34a9800640 1 -- 172.21.15.63:0/2222159316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a4078720 msgr2=0x7f34a4078b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.291+0000 7f34a9800640 1 --2- 172.21.15.63:0/2222159316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a4078720 0x7f34a4078b30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3494009900 tx=0x7f349402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.292+0000 7f34a9800640 1 -- 172.21.15.63:0/2222159316 shutdown_connections 2024-05-27T07:34:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.292+0000 7f34a9800640 1 --2- 172.21.15.63:0/2222159316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a4114900 0x7f34a4116d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.292+0000 7f34a9800640 1 --2- 172.21.15.63:0/2222159316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34a4111fa0 0x7f34a41143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.292+0000 7f34a9800640 1 --2- 172.21.15.63:0/2222159316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a4078720 0x7f34a4078b30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.292+0000 7f34a9800640 1 -- 172.21.15.63:0/2222159316 >> 172.21.15.63:0/2222159316 conn(0x7f34a41013b0 msgr2=0x7f34a41037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.292+0000 7f34a9800640 1 -- 172.21.15.63:0/2222159316 shutdown_connections 2024-05-27T07:34:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.292+0000 7f34a9800640 1 -- 172.21.15.63:0/2222159316 wait complete. 2024-05-27T07:34:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.293+0000 7f34a9800640 1 Processor -- start 2024-05-27T07:34:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.293+0000 7f34a9800640 1 -- start start 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.294+0000 7f34a9800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34a4078720 0x7f34a41a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.294+0000 7f34a9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a4111fa0 0x7f34a41a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.294+0000 7f34a9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a4114900 0x7f34a41a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.294+0000 7f34a9800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34a411a140 con 0x7f34a4111fa0 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.294+0000 7f34a9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4119fc0 con 0x7f34a4114900 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.294+0000 7f34a9800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34a411a2c0 con 0x7f34a4078720 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.294+0000 7f349b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a4111fa0 0x7f34a41a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.294+0000 7f349b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a4111fa0 0x7f34a41a6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37892/0 (socket says 172.21.15.63:37892) 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.294+0000 7f349b400640 1 -- 172.21.15.63:0/1386495763 learned_addr learned my addr 172.21.15.63:0/1386495763 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.294+0000 7f34a3e00640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34a4078720 0x7f34a41a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.295+0000 7f34a8e00640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a4114900 0x7f34a41a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.295+0000 7f349b400640 1 -- 172.21.15.63:0/1386495763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34a4078720 msgr2=0x7f34a41a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.295+0000 7f349b400640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34a4078720 0x7f34a41a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.295+0000 7f349b400640 1 -- 172.21.15.63:0/1386495763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a4114900 msgr2=0x7f34a41a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.295+0000 7f349b400640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a4114900 0x7f34a41a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.295+0000 7f349b400640 1 -- 172.21.15.63:0/1386495763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a41a1010 con 0x7f34a4111fa0 2024-05-27T07:34:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.295+0000 7f349b400640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a4111fa0 0x7f34a41a6d50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f348c00c970 tx=0x7f348c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.296+0000 7f34a1600640 1 -- 172.21.15.63:0/1386495763 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c007bb0 con 0x7f34a4111fa0 2024-05-27T07:34:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.296+0000 7f34a1600640 1 -- 172.21.15.63:0/1386495763 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f348c007d50 con 0x7f34a4111fa0 2024-05-27T07:34:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.296+0000 7f34a9800640 1 -- 172.21.15.63:0/1386495763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a41a1300 con 0x7f34a4111fa0 2024-05-27T07:34:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.296+0000 7f34a9800640 1 -- 172.21.15.63:0/1386495763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a41a1800 con 0x7f34a4111fa0 2024-05-27T07:34:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.296+0000 7f34a1600640 1 -- 172.21.15.63:0/1386495763 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c018660 con 0x7f34a4111fa0 2024-05-27T07:34:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.300+0000 7f34b1d9f740 1 Processor -- start 2024-05-27T07:34:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.300+0000 7f34b1d9f740 1 -- start start 2024-05-27T07:34:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34a1600640 1 -- 172.21.15.63:0/1386495763 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f348c018900 con 0x7f34a4111fa0 2024-05-27T07:34:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e89a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381abe2f0 0x55a381e89fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381ac64d0 0x55a381e8a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a381dfab90 con 0x55a381abe2f0 2024-05-27T07:34:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a381dea730 con 0x55a381ae71b0 2024-05-27T07:34:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a381dee770 con 0x55a381ac64d0 2024-05-27T07:34:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381abe2f0 0x55a381e89fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381abe2f0 0x55a381e89fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37906/0 (socket says 172.21.15.63:37906) 2024-05-27T07:34:51.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34a3e00640 1 -- 172.21.15.63:0/2995336367 learned_addr learned my addr 172.21.15.63:0/2995336367 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f34a8e00640 1 --2- 172.21.15.63:0/2995336367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e89a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.301+0000 7f349b400640 1 --2- 172.21.15.63:0/2995336367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381ac64d0 0x55a381e8a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.302+0000 7f34a3e00640 1 -- 172.21.15.63:0/2995336367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381ac64d0 msgr2=0x55a381e8a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.302+0000 7f34a3e00640 1 --2- 172.21.15.63:0/2995336367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381ac64d0 0x55a381e8a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.302+0000 7f34a3e00640 1 -- 172.21.15.63:0/2995336367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 msgr2=0x55a381e89a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.302+0000 7f34a3e00640 1 --2- 172.21.15.63:0/2995336367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e89a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.302+0000 7f34a3e00640 1 -- 172.21.15.63:0/2995336367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a381d838b0 con 0x55a381abe2f0 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.302+0000 7f34a8e00640 1 --2- 172.21.15.63:0/2995336367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e89a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.302+0000 7f34a3e00640 1 --2- 172.21.15.63:0/2995336367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381abe2f0 0x55a381e89fa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f349402a910 tx=0x7f3494031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6ed5474530bf27 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.303+0000 7f3499600640 1 -- 172.21.15.63:0/2995336367 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494035030 con 0x55a381abe2f0 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.303+0000 7f3499600640 1 -- 172.21.15.63:0/2995336367 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34940044c0 con 0x55a381abe2f0 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.303+0000 7f3499600640 1 -- 172.21.15.63:0/2995336367 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494003c20 con 0x55a381abe2f0 2024-05-27T07:34:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.303+0000 7f34a1600640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34880b2b00 0x7f34880b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.303+0000 7f34a1600640 1 -- 172.21.15.63:0/1386495763 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f348c0e2bd0 con 0x7f34a4111fa0 2024-05-27T07:34:51.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.303+0000 7f34a8e00640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34880b2b00 0x7f34880b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.304+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2995336367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381abe2f0 msgr2=0x55a381e89fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.304+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/2995336367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381abe2f0 0x55a381e89fa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f349402a910 tx=0x7f3494031dc0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.304+0000 7f34a8e00640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34880b2b00 0x7f34880b5010 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55a381e86bd0 tx=0x7f349c007870 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.304+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2995336367 shutdown_connections 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.304+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/2995336367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381ac64d0 0x55a381e8a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.304+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/2995336367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381abe2f0 0x55a381e89fa0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.304+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/2995336367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e89a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.304+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2995336367 >> 172.21.15.63:0/2995336367 conn(0x55a381adb300 msgr2=0x55a381e86210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.304+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2995336367 shutdown_connections 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.304+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2995336367 wait complete. 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.305+0000 7f34b1d9f740 1 -- start start 2024-05-27T07:34:51.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.305+0000 7f34b1d9f740 1 Processor -- start 2024-05-27T07:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e93b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e940d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e9fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a381ad0790 con 0x55a381ac64d0 2024-05-27T07:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a381d92bc0 con 0x55a381ae71b0 2024-05-27T07:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a381e96b30 con 0x55a381abe2f0 2024-05-27T07:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f34b1d9f740 10 client.0 _ll_register_callbacks cb 0x55a381d70400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f349b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f349b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e940d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37908/0 (socket says 172.21.15.63:37908) 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f349b400640 1 -- 172.21.15.63:0/2867868771 learned_addr learned my addr 172.21.15.63:0/2867868771 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.306+0000 7f34a3e00640 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e93b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.307+0000 7f34a8e00640 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e9fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.307+0000 7f349b400640 1 -- 172.21.15.63:0/2867868771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 msgr2=0x55a381e93b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.307+0000 7f349b400640 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e93b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.307+0000 7f349b400640 1 -- 172.21.15.63:0/2867868771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 msgr2=0x55a381e9fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.307+0000 7f349b400640 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e9fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.307+0000 7f349b400640 1 -- 172.21.15.63:0/2867868771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a381aff210 con 0x55a381ac64d0 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.307+0000 7f349b400640 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e940d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f348c020000 tx=0x7f348c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c101913997e4349 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.307+0000 7f3492000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.307+0000 7f3492000640 1 -- 172.21.15.63:0/2867868771 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c0a9250 con 0x55a381ac64d0 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.308+0000 7f3492000640 1 -- 172.21.15.63:0/2867868771 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f348c0a5ef0 con 0x55a381ac64d0 2024-05-27T07:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.308+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2867868771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a381dfcae0 con 0x55a381ac64d0 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.308+0000 7f3492000640 1 -- 172.21.15.63:0/2867868771 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f348c0a5090 con 0x55a381ac64d0 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.308+0000 7f34b1d9f740 10 client.15579 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.308+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2867868771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a381bcb620 con 0x55a381ac64d0 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.308+0000 7f3492000640 1 -- 172.21.15.63:0/2867868771 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a381dfcae0 con 0x55a381ac64d0 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.308+0000 7f3492000640 1 -- 172.21.15.63:0/2867868771 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f348c020030 con 0x55a381ac64d0 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.308+0000 7f34b1d9f740 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.308+0000 7f34b1d9f740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.308+0000 7f34b1d9f740 20 client.15579 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.309+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a381aa5d80 0x55a381ea5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.309+0000 7f34b1d9f740 4 client.15579 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.309+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2867868771 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a381df62c0 con 0x55a381aa5d80 2024-05-27T07:34:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.309+0000 7f34a3e00640 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a381aa5d80 0x55a381ea5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.310+0000 7f3492000640 10 client.15579 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.310+0000 7f34a3e00640 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a381aa5d80 0x55a381ea5e10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9db9905d0c8600dc server_cookie=c30e15e7174ae257 in_seq=0 out_seq=0 2024-05-27T07:34:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.312+0000 7f3492000640 1 -- 172.21.15.63:0/2867868771 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a381df62c0 con 0x55a381aa5d80 2024-05-27T07:34:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.312+0000 7f3492000640 10 client.15579 handle_command_reply: tid=0 2024-05-27T07:34:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.312+0000 7f34b1d9f740 1 client.15579 shutdown 2024-05-27T07:34:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.312+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2867868771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 msgr2=0x55a381e940d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.312+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e940d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f348c020000 tx=0x7f348c007f90 comp rx=0 tx=0).stop 2024-05-27T07:34:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.313+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2867868771 shutdown_connections 2024-05-27T07:34:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.313+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a381aa5d80 0x55a381ea5e10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.313+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e9fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.313+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e940d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.313+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/2867868771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e93b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.313+0000 7f3492000640 0 client.15579 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.313+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2867868771 >> 172.21.15.63:0/2867868771 conn(0x55a381adb300 msgr2=0x55a381e8b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.313+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2867868771 shutdown_connections 2024-05-27T07:34:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.313+0000 7f34b1d9f740 1 -- 172.21.15.63:0/2867868771 wait complete. 2024-05-27T07:34:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.313+0000 7f34b1d9f740 20 client.15579 trim_cache size 0 max 16384 2024-05-27T07:34:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.331+0000 7f34b1d9f740 1 Processor -- start 2024-05-27T07:34:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.331+0000 7f34b1d9f740 1 -- start start 2024-05-27T07:34:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.331+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.331+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e89a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.331+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e8d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.331+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3480012670 con 0x55a381ac64d0 2024-05-27T07:34:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.331+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f348000ff40 con 0x55a381ae71b0 2024-05-27T07:34:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.331+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f348000dd90 con 0x55a381abe2f0 2024-05-27T07:34:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.331+0000 7f349b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e89a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f349b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e89a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37922/0 (socket says 172.21.15.63:37922) 2024-05-27T07:34:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f349b400640 1 -- 172.21.15.63:0/3034116127 learned_addr learned my addr 172.21.15.63:0/3034116127 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f34a3e00640 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f34a8e00640 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e8d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f349b400640 1 -- 172.21.15.63:0/3034116127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 msgr2=0x55a381e962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f349b400640 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f349b400640 1 -- 172.21.15.63:0/3034116127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 msgr2=0x55a381e8d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f349b400640 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e8d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f349b400640 1 -- 172.21.15.63:0/3034116127 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a381e920e0 con 0x55a381ac64d0 2024-05-27T07:34:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f34a8e00640 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e8d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f34a3e00640 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.332+0000 7f349b400640 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e89a20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f348c0dfeb0 tx=0x7f348c0a9790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1ecdf2504e0fe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.333+0000 7f3498c00640 1 -- 172.21.15.63:0/3034116127 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c0a9070 con 0x55a381ac64d0 2024-05-27T07:34:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.333+0000 7f3498c00640 1 -- 172.21.15.63:0/3034116127 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f348c0a9410 con 0x55a381ac64d0 2024-05-27T07:34:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.333+0000 7f3498c00640 1 -- 172.21.15.63:0/3034116127 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c0a54d0 con 0x55a381ac64d0 2024-05-27T07:34:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.334+0000 7f34b1d9f740 1 -- 172.21.15.63:0/3034116127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 msgr2=0x55a381e89a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.334+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e89a20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f348c0dfeb0 tx=0x7f348c0a9790 comp rx=0 tx=0).stop 2024-05-27T07:34:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.334+0000 7f34b1d9f740 1 -- 172.21.15.63:0/3034116127 shutdown_connections 2024-05-27T07:34:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.334+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ae71b0 0x55a381e8d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.334+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ac64d0 0x55a381e89a20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.334+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/3034116127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.334+0000 7f34b1d9f740 1 -- 172.21.15.63:0/3034116127 >> 172.21.15.63:0/3034116127 conn(0x55a381adb300 msgr2=0x55a381e93b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.334+0000 7f34b1d9f740 1 -- 172.21.15.63:0/3034116127 shutdown_connections 2024-05-27T07:34:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.334+0000 7f34b1d9f740 1 -- 172.21.15.63:0/3034116127 wait complete. 2024-05-27T07:34:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.334+0000 7f34b1d9f740 1 -- start start 2024-05-27T07:34:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.335+0000 7f34b1d9f740 1 Processor -- start 2024-05-27T07:34:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.335+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e97470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.335+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ac64d0 0x55a381e979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.335+0000 7f34b1d9f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ae71b0 0x55a381f0c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.335+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f348000e400 con 0x55a381ae71b0 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.335+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f348000f870 con 0x55a381ac64d0 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.335+0000 7f34b1d9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f348000ead0 con 0x55a381abe2f0 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.335+0000 7f34b1d9f740 10 client.0 _ll_register_callbacks cb 0x7f348c00a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f34a8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ae71b0 0x55a381f0c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f34a8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ae71b0 0x55a381f0c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37938/0 (socket says 172.21.15.63:37938) 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f34a8e00640 1 -- 172.21.15.63:0/398034855 learned_addr learned my addr 172.21.15.63:0/398034855 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f349b400640 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ac64d0 0x55a381e979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f34a3e00640 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e97470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f34a8e00640 1 -- 172.21.15.63:0/398034855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 msgr2=0x55a381e97470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:51.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f34a8e00640 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e97470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f34a8e00640 1 -- 172.21.15.63:0/398034855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ac64d0 msgr2=0x55a381e979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f34a8e00640 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ac64d0 0x55a381e979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f34a8e00640 1 -- 172.21.15.63:0/398034855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a381f0c980 con 0x55a381ae71b0 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.336+0000 7f34a3e00640 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e97470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.337+0000 7f34a8e00640 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ae71b0 0x55a381f0c440 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f349c011ec0 tx=0x7f349c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55da78a89ce678de server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.337+0000 7f3492000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.337+0000 7f3492000640 1 -- 172.21.15.63:0/398034855 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c01fea0 con 0x55a381ae71b0 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.337+0000 7f3492000640 1 -- 172.21.15.63:0/398034855 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f349c020850 con 0x55a381ae71b0 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.337+0000 7f34b1d9f740 1 -- 172.21.15.63:0/398034855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f348c0a9410 con 0x55a381ae71b0 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.337+0000 7f3492000640 1 -- 172.21.15.63:0/398034855 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f349c033050 con 0x55a381ae71b0 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.337+0000 7f34b1d9f740 10 client.15585 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.337+0000 7f34b1d9f740 1 -- 172.21.15.63:0/398034855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a381f0db20 con 0x55a381ae71b0 2024-05-27T07:34:51.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.338+0000 7f3492000640 1 -- 172.21.15.63:0/398034855 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f348c0a9410 con 0x55a381ae71b0 2024-05-27T07:34:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.338+0000 7f3492000640 1 -- 172.21.15.63:0/398034855 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f349c028a20 con 0x55a381ae71b0 2024-05-27T07:34:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.338+0000 7f34b1d9f740 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.338+0000 7f34b1d9f740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.338+0000 7f34b1d9f740 20 client.15585 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.338+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a381a7b1c0 0x55a381f10310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.338+0000 7f34b1d9f740 4 client.15585 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:34:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.338+0000 7f34b1d9f740 1 -- 172.21.15.63:0/398034855 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a381d704b0 con 0x55a381a7b1c0 2024-05-27T07:34:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.339+0000 7f34a3e00640 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a381a7b1c0 0x55a381f10310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.340+0000 7f34a3e00640 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a381a7b1c0 0x55a381f10310 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd10ec6a500964f server_cookie=a2d4885bd338f129 in_seq=0 out_seq=0 2024-05-27T07:34:51.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.340+0000 7f3492000640 10 client.15585 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f3492000640 1 -- 172.21.15.63:0/398034855 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a381d704b0 con 0x55a381a7b1c0 2024-05-27T07:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f3492000640 10 client.15585 handle_command_reply: tid=0 2024-05-27T07:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 client.15585 shutdown 2024-05-27T07:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 -- 172.21.15.63:0/398034855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ae71b0 msgr2=0x55a381f0c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ae71b0 0x55a381f0c440 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f349c011ec0 tx=0x7f349c004a40 comp rx=0 tx=0).stop 2024-05-27T07:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 -- 172.21.15.63:0/398034855 shutdown_connections 2024-05-27T07:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a381a7b1c0 0x55a381f10310 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a381ae71b0 0x55a381f0c440 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a381ac64d0 0x55a381e979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 --2- 172.21.15.63:0/398034855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a381abe2f0 0x55a381e97470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f3492000640 0 client.15585 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 -- 172.21.15.63:0/398034855 >> 172.21.15.63:0/398034855 conn(0x55a381adb300 msgr2=0x55a381e95240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 -- 172.21.15.63:0/398034855 shutdown_connections 2024-05-27T07:34:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 1 -- 172.21.15.63:0/398034855 wait complete. 2024-05-27T07:34:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.350+0000 7f34b1d9f740 20 client.15585 trim_cache size 0 max 16384 2024-05-27T07:34:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:34:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:34:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 -- 172.21.15.63:0/1386495763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34880b2b00 msgr2=0x7f34880b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34880b2b00 0x7f34880b5010 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55a381e86bd0 tx=0x7f349c007870 comp rx=0 tx=0).stop 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 -- 172.21.15.63:0/1386495763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a4111fa0 msgr2=0x7f34a41a6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a4111fa0 0x7f34a41a6d50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f348c00c970 tx=0x7f348c00ce30 comp rx=0 tx=0).stop 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 -- 172.21.15.63:0/1386495763 shutdown_connections 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34880b2b00 0x7f34880b5010 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a4114900 0x7f34a41a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34a4111fa0 0x7f34a41a6d50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 --2- 172.21.15.63:0/1386495763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34a4078720 0x7f34a41a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 -- 172.21.15.63:0/1386495763 >> 172.21.15.63:0/1386495763 conn(0x7f34a41013b0 msgr2=0x7f34a407b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 -- 172.21.15.63:0/1386495763 shutdown_connections 2024-05-27T07:34:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:34:51.351+0000 7f34a9800640 1 -- 172.21.15.63:0/1386495763 wait complete. 2024-05-27T07:34:51.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:51 smithi063 ceph-mon[31613]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:51.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:51 smithi086 ceph-mon[34687]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:51.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:51 smithi149 ceph-mon[35408]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:53.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:53 smithi086 ceph-mon[34687]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:34:53.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:53 smithi149 ceph-mon[35408]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:34:53.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:53 smithi063 ceph-mon[31613]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-27T07:34:55.350 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:34:55.350 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:34:55.859 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:34:56.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:56 smithi086 ceph-mon[34687]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:34:56.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:56 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:56.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:56 smithi149 ceph-mon[35408]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:34:56.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:56 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:56.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:56 smithi063 ceph-mon[31613]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:34:56.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:56 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:57.633 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:57 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:57.633 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:57 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:57.633 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:57 smithi063 ceph-mon[31613]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:57.633 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:57 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:34:57.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:57 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:57.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:57 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:57.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:57 smithi086 ceph-mon[34687]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:57.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:57 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:34:57.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:57 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:57.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:57 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:57.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:57 smithi149 ceph-mon[35408]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-27T07:34:57.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:57 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:34:58.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.328+0000 7f80eca00700 1 -- 172.21.15.63:0/4179151601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e810b890 msgr2=0x7f80e81131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.328+0000 7f80eca00700 1 --2- 172.21.15.63:0/4179151601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e810b890 0x7f80e81131f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b0d0 tx=0x7f80e000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.328+0000 7f80eca00700 1 -- 172.21.15.63:0/4179151601 shutdown_connections 2024-05-27T07:34:58.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.328+0000 7f80eca00700 1 --2- 172.21.15.63:0/4179151601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e810b890 0x7f80e81131f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.328+0000 7f80eca00700 1 --2- 172.21.15.63:0/4179151601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8106610 0x7f80e8106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.328+0000 7f80eca00700 1 --2- 172.21.15.63:0/4179151601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e810af40 0x7f80e810b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.328+0000 7f80eca00700 1 -- 172.21.15.63:0/4179151601 >> 172.21.15.63:0/4179151601 conn(0x7f80e8102020 msgr2=0x7f80e8104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:58.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.329+0000 7f80eca00700 1 -- 172.21.15.63:0/4179151601 shutdown_connections 2024-05-27T07:34:58.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.329+0000 7f80eca00700 1 -- 172.21.15.63:0/4179151601 wait complete. 2024-05-27T07:34:58.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.330+0000 7f80eca00700 1 Processor -- start 2024-05-27T07:34:58.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.330+0000 7f80eca00700 1 -- start start 2024-05-27T07:34:58.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.331+0000 7f80eca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8106610 0x7f80e819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.331+0000 7f80eca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e810af40 0x7f80e819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.331+0000 7f80eca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e810b890 0x7f80e81a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.331+0000 7f80eca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80e81163c0 con 0x7f80e810af40 2024-05-27T07:34:58.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.331+0000 7f80eca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80e8116240 con 0x7f80e8106610 2024-05-27T07:34:58.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.331+0000 7f80eca00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f80e8116540 con 0x7f80e810b890 2024-05-27T07:34:58.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.332+0000 7f80e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e810af40 0x7f80e819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.332+0000 7f80e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e810af40 0x7f80e819d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37960/0 (socket says 172.21.15.63:37960) 2024-05-27T07:34:58.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.332+0000 7f80e6a00700 1 -- 172.21.15.63:0/1902259903 learned_addr learned my addr 172.21.15.63:0/1902259903 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:58.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.332+0000 7f80e7e00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e810b890 0x7f80e81a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.332+0000 7f80e6a00700 1 -- 172.21.15.63:0/1902259903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e810b890 msgr2=0x7f80e81a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.332+0000 7f80e6a00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e810b890 0x7f80e81a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.332+0000 7f80e6a00700 1 -- 172.21.15.63:0/1902259903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8106610 msgr2=0x7f80e819cdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:34:58.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.332+0000 7f80e6a00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8106610 0x7f80e819cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.332+0000 7f80e6a00700 1 -- 172.21.15.63:0/1902259903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e81a1bf0 con 0x7f80e810af40 2024-05-27T07:34:58.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.333+0000 7f80e7e00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e810b890 0x7f80e81a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:34:58.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.333+0000 7f80e6a00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e810af40 0x7f80e819d370 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c9e0 tx=0x7f80d400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:58.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.334+0000 7f80dfe00700 1 -- 172.21.15.63:0/1902259903 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4007aa0 con 0x7f80e810af40 2024-05-27T07:34:58.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.334+0000 7f80dfe00700 1 -- 172.21.15.63:0/1902259903 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80d4007c40 con 0x7f80e810af40 2024-05-27T07:34:58.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.334+0000 7f80dfe00700 1 -- 172.21.15.63:0/1902259903 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4018660 con 0x7f80e810af40 2024-05-27T07:34:58.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.334+0000 7f80eca00700 1 -- 172.21.15.63:0/1902259903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e81a1ee0 con 0x7f80e810af40 2024-05-27T07:34:58.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.334+0000 7f80eca00700 1 -- 172.21.15.63:0/1902259903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e81a23d0 con 0x7f80e810af40 2024-05-27T07:34:58.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.339+0000 7f80fcbbeb80 1 Processor -- start 2024-05-27T07:34:58.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.339+0000 7f80fcbbeb80 1 -- start start 2024-05-27T07:34:58.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.340+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f453ec0 0x562c5f5d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.340+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5dcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.340+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f5d28a0 0x562c5f5df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.340+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562c5f500ec0 con 0x562c5f5d2490 2024-05-27T07:34:58.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.340+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562c5f4f0a60 con 0x562c5f5d28a0 2024-05-27T07:34:58.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.340+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562c5f4f4aa0 con 0x562c5f453ec0 2024-05-27T07:34:58.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.341+0000 7f80e7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f453ec0 0x562c5f5d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.341+0000 7f80e7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f453ec0 0x562c5f5d1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:36006/0 (socket says 172.21.15.63:36006) 2024-05-27T07:34:58.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.341+0000 7f80e7e00700 1 -- 172.21.15.63:0/1875116449 learned_addr learned my addr 172.21.15.63:0/1875116449 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:58.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.341+0000 7f80e7e00700 1 -- 172.21.15.63:0/1875116449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f5d28a0 msgr2=0x562c5f5df0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:34:58.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.341+0000 7f80dfe00700 1 -- 172.21.15.63:0/1902259903 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f80d400f450 con 0x7f80e810af40 2024-05-27T07:34:58.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.341+0000 7f80e7e00700 1 --2- 172.21.15.63:0/1875116449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f5d28a0 0x562c5f5df0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.341+0000 7f80e7e00700 1 -- 172.21.15.63:0/1875116449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 msgr2=0x562c5f5dcb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:34:58.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.341+0000 7f80e7e00700 1 --2- 172.21.15.63:0/1875116449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5dcb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.341+0000 7f80e7e00700 1 -- 172.21.15.63:0/1875116449 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c5f4b94a0 con 0x562c5f453ec0 2024-05-27T07:34:58.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.342+0000 7f80e7e00700 1 --2- 172.21.15.63:0/1875116449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f453ec0 0x562c5f5d1f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009d90 tx=0x7f80e0009dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97099cff1420452e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:58.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.342+0000 7f80dcc00700 1 -- 172.21.15.63:0/1875116449 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0022070 con 0x562c5f453ec0 2024-05-27T07:34:58.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.343+0000 7f80dcc00700 1 -- 172.21.15.63:0/1875116449 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80e0007a50 con 0x562c5f453ec0 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.343+0000 7f80dcc00700 1 -- 172.21.15.63:0/1875116449 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e001ca00 con 0x562c5f453ec0 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.343+0000 7f80dfe00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f80cc0b2c20 0x7f80cc0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.344+0000 7f80dfe00700 1 -- 172.21.15.63:0/1902259903 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f80d40e2b20 con 0x7f80e810af40 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.344+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/1875116449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f453ec0 msgr2=0x562c5f5d1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.344+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/1875116449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f453ec0 0x562c5f5d1f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009d90 tx=0x7f80e0009dc0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.344+0000 7f80e7400700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f80cc0b2c20 0x7f80cc0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.344+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/1875116449 shutdown_connections 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.344+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/1875116449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f5d28a0 0x562c5f5df0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.344+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/1875116449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f453ec0 0x562c5f5d1f50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.344+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/1875116449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5dcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.344+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/1875116449 >> 172.21.15.63:0/1875116449 conn(0x562c5f4b4040 msgr2=0x562c5f5cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:58.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.345+0000 7f80e7400700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f80cc0b2c20 0x7f80cc0b5130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x562c5f5d34b0 tx=0x7f80d800b3f0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:58.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.345+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/1875116449 shutdown_connections 2024-05-27T07:34:58.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.345+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/1875116449 wait complete. 2024-05-27T07:34:58.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.346+0000 7f80fcbbeb80 1 -- start start 2024-05-27T07:34:58.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.346+0000 7f80fcbbeb80 1 Processor -- start 2024-05-27T07:34:58.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.347+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f453ec0 0x562c5f5eb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.347+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f5d2490 0x562c5f5eb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.347+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f5d28a0 0x562c5f5e5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.347+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562c5f3f7020 con 0x562c5f453ec0 2024-05-27T07:34:58.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.347+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562c5f4917e0 con 0x562c5f5d2490 2024-05-27T07:34:58.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.347+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562c5f5df9b0 con 0x562c5f5d28a0 2024-05-27T07:34:58.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.347+0000 7f80fcbbeb80 10 client.0 _ll_register_callbacks cb 0x562c5f49acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:58.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.347+0000 7f80e7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f453ec0 0x562c5f5eb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.348+0000 7f80e7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f453ec0 0x562c5f5eb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37976/0 (socket says 172.21.15.63:37976) 2024-05-27T07:34:58.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.348+0000 7f80e6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f5d2490 0x562c5f5eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.348+0000 7f80e7400700 1 -- 172.21.15.63:0/2337838640 learned_addr learned my addr 172.21.15.63:0/2337838640 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:58.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.348+0000 7f80e7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f5d28a0 0x562c5f5e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.348+0000 7f80e7400700 1 -- 172.21.15.63:0/2337838640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f5d28a0 msgr2=0x562c5f5e5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:58.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.348+0000 7f80e7400700 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f5d28a0 0x562c5f5e5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.348+0000 7f80e7400700 1 -- 172.21.15.63:0/2337838640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f5d2490 msgr2=0x562c5f5eb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:58.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.348+0000 7f80e7400700 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f5d2490 0x562c5f5eb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.348+0000 7f80e7400700 1 -- 172.21.15.63:0/2337838640 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c5f440950 con 0x562c5f453ec0 2024-05-27T07:34:58.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.349+0000 7f80e7400700 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f453ec0 0x562c5f5eb200 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f80d800dc80 tx=0x7f80d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e0e9b3b29e2a6c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:58.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.349+0000 7f80d1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:34:58.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.349+0000 7f80d1600700 1 -- 172.21.15.63:0/2337838640 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8021020 con 0x562c5f453ec0 2024-05-27T07:34:58.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.349+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2337838640 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e0007a50 con 0x562c5f453ec0 2024-05-27T07:34:58.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.349+0000 7f80d1600700 1 -- 172.21.15.63:0/2337838640 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f80d8010a70 con 0x562c5f453ec0 2024-05-27T07:34:58.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.349+0000 7f80d1600700 1 -- 172.21.15.63:0/2337838640 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80d802c050 con 0x562c5f453ec0 2024-05-27T07:34:58.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.349+0000 7f80fcbbeb80 10 client.15594 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:58.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.349+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2337838640 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c5f4c10e0 con 0x562c5f453ec0 2024-05-27T07:34:58.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.350+0000 7f80d1600700 1 -- 172.21.15.63:0/2337838640 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80e0007a50 con 0x562c5f453ec0 2024-05-27T07:34:58.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.350+0000 7f80d1600700 1 -- 172.21.15.63:0/2337838640 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f80d801dcc0 con 0x562c5f453ec0 2024-05-27T07:34:58.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.350+0000 7f80fcbbeb80 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:58.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.350+0000 7f80fcbbeb80 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:58.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.350+0000 7f80fcbbeb80 20 client.15594 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.350+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562c5f5f52e0 0x562c5f5f7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.350+0000 7f80fcbbeb80 4 client.15594 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:34:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.350+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2337838640 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c5f502e10 con 0x562c5f5f52e0 2024-05-27T07:34:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.351+0000 7f80e6a00700 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562c5f5f52e0 0x562c5f5f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.352+0000 7f80d1600700 10 client.15594 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:58.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.352+0000 7f80e6a00700 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562c5f5f52e0 0x562c5f5f7770 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e10dbc051546e3b server_cookie=2c712c3ec25b7673 in_seq=0 out_seq=0 2024-05-27T07:34:58.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80d1600700 1 -- 172.21.15.63:0/2337838640 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562c5f502e10 con 0x562c5f5f52e0 2024-05-27T07:34:58.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80d1600700 10 client.15594 handle_command_reply: tid=0 2024-05-27T07:34:58.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80fcbbeb80 1 client.15594 shutdown 2024-05-27T07:34:58.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2337838640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f453ec0 msgr2=0x562c5f5eb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f453ec0 0x562c5f5eb200 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f80d800dc80 tx=0x7f80d80047c0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2337838640 shutdown_connections 2024-05-27T07:34:58.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562c5f5f52e0 0x562c5f5f7770 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f5d28a0 0x562c5f5e5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f453ec0 0x562c5f5eb200 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2337838640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f5d2490 0x562c5f5eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2337838640 >> 172.21.15.63:0/2337838640 conn(0x562c5f4b4040 msgr2=0x562c5f5cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:58.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.354+0000 7f80d1600700 0 client.15594 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:58.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.355+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2337838640 shutdown_connections 2024-05-27T07:34:58.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.355+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2337838640 wait complete. 2024-05-27T07:34:58.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.355+0000 7f80fcbbeb80 20 client.15594 trim_cache size 0 max 16384 2024-05-27T07:34:58.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.401+0000 7f80fcbbeb80 1 Processor -- start 2024-05-27T07:34:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.401+0000 7f80fcbbeb80 1 -- start start 2024-05-27T07:34:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.401+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 0x562c5f5fcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.401+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 0x562c5f5fd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.401+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5e27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.401+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80e81b0700 con 0x562c5f5d2490 2024-05-27T07:34:58.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.401+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80e81adf90 con 0x562c5f453ec0 2024-05-27T07:34:58.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.401+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f80e81abde0 con 0x562c5f4b4040 2024-05-27T07:34:58.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.402+0000 7f80e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5e27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.402+0000 7f80e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5e27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37984/0 (socket says 172.21.15.63:37984) 2024-05-27T07:34:58.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.402+0000 7f80e7e00700 1 -- 172.21.15.63:0/1085638357 learned_addr learned my addr 172.21.15.63:0/1085638357 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:58.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.402+0000 7f80e7400700 1 --2- 172.21.15.63:0/1085638357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 0x562c5f5fcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.402+0000 7f80e6a00700 1 --2- 172.21.15.63:0/1085638357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 0x562c5f5fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.403+0000 7f80e7e00700 1 -- 172.21.15.63:0/1085638357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 msgr2=0x562c5f5fcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:58.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.403+0000 7f80e7e00700 1 --2- 172.21.15.63:0/1085638357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 0x562c5f5fcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.403+0000 7f80e7e00700 1 -- 172.21.15.63:0/1085638357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 msgr2=0x562c5f5fd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:58.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.403+0000 7f80e7e00700 1 --2- 172.21.15.63:0/1085638357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 0x562c5f5fd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.403+0000 7f80e7e00700 1 -- 172.21.15.63:0/1085638357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c5f5f2d60 con 0x562c5f5d2490 2024-05-27T07:34:58.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.403+0000 7f80e7e00700 1 --2- 172.21.15.63:0/1085638357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5e27e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f80e0000f80 tx=0x7f80e0006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925076a5f7fb9f9c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:58.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.404+0000 7f80d3e00700 1 -- 172.21.15.63:0/1085638357 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e002d070 con 0x562c5f5d2490 2024-05-27T07:34:58.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.404+0000 7f80d3e00700 1 -- 172.21.15.63:0/1085638357 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80e0007c50 con 0x562c5f5d2490 2024-05-27T07:34:58.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.404+0000 7f80d3e00700 1 -- 172.21.15.63:0/1085638357 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0022040 con 0x562c5f5d2490 2024-05-27T07:34:58.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.405+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/1085638357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 msgr2=0x562c5f5e27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.405+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/1085638357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5e27e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f80e0000f80 tx=0x7f80e0006040 comp rx=0 tx=0).stop 2024-05-27T07:34:58.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.405+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/1085638357 shutdown_connections 2024-05-27T07:34:58.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.405+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/1085638357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5e27e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.405+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/1085638357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 0x562c5f5fcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.405+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/1085638357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 0x562c5f5fd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.405+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/1085638357 >> 172.21.15.63:0/1085638357 conn(0x562c5f5c6fe0 msgr2=0x562c5f5d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:58.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.406+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/1085638357 shutdown_connections 2024-05-27T07:34:58.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.406+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/1085638357 wait complete. 2024-05-27T07:34:58.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.406+0000 7f80fcbbeb80 1 -- start start 2024-05-27T07:34:58.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.407+0000 7f80fcbbeb80 1 Processor -- start 2024-05-27T07:34:58.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.407+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 0x562c5f5f0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.407+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 0x562c5f5f0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.407+0000 7f80fcbbeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.407+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80e81ac450 con 0x562c5f5d2490 2024-05-27T07:34:58.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.407+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80e81ad8c0 con 0x562c5f453ec0 2024-05-27T07:34:58.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.407+0000 7f80fcbbeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f80e81acb20 con 0x562c5f4b4040 2024-05-27T07:34:58.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.407+0000 7f80fcbbeb80 10 client.0 _ll_register_callbacks cb 0x562c5f49acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:34:58.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.408+0000 7f80e7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 0x562c5f5f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.408+0000 7f80e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.408+0000 7f80e6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 0x562c5f5f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.408+0000 7f80e6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 0x562c5f5f0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:36034/0 (socket says 172.21.15.63:36034) 2024-05-27T07:34:58.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.408+0000 7f80e6a00700 1 -- 172.21.15.63:0/2087822869 learned_addr learned my addr 172.21.15.63:0/2087822869 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:34:58.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.409+0000 7f80e7400700 1 -- 172.21.15.63:0/2087822869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 msgr2=0x562c5f5f0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:58.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.409+0000 7f80e7400700 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 0x562c5f5f0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.409+0000 7f80e7400700 1 -- 172.21.15.63:0/2087822869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 msgr2=0x562c5f5f0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:34:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.409+0000 7f80e7400700 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5f0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.409+0000 7f80e7400700 1 -- 172.21.15.63:0/2087822869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c5f5cf3d0 con 0x562c5f453ec0 2024-05-27T07:34:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.409+0000 7f80e7e00700 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:34:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.410+0000 7f80e6a00700 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 0x562c5f5f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:34:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.410+0000 7f80e7400700 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 0x562c5f5f0300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f80d800f490 tx=0x7f80d800d650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=204096ca5fd598c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:58.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.410+0000 7f80d1600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:34:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.411+0000 7f80d1600700 1 -- 172.21.15.63:0/2087822869 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8021040 con 0x562c5f453ec0 2024-05-27T07:34:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.411+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2087822869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e0007c50 con 0x562c5f453ec0 2024-05-27T07:34:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.411+0000 7f80d1600700 1 -- 172.21.15.63:0/2087822869 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f80d800f700 con 0x562c5f453ec0 2024-05-27T07:34:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.411+0000 7f80d1600700 1 -- 172.21.15.63:0/2087822869 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80d8045050 con 0x562c5f453ec0 2024-05-27T07:34:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.411+0000 7f80fcbbeb80 10 client.35363 fetch_fsmap learned FSMap version 27 2024-05-27T07:34:58.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.411+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2087822869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c5f660ef0 con 0x562c5f453ec0 2024-05-27T07:34:58.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.412+0000 7f80d1600700 1 -- 172.21.15.63:0/2087822869 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80e0007c50 con 0x562c5f453ec0 2024-05-27T07:34:58.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.412+0000 7f80d1600700 1 -- 172.21.15.63:0/2087822869 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f80d801d6b0 con 0x562c5f453ec0 2024-05-27T07:34:58.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.412+0000 7f80fcbbeb80 10 client.35363 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:34:58.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.412+0000 7f80fcbbeb80 10 client.35363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:34:58.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.412+0000 7f80fcbbeb80 20 client.35363 populate_metadata read hostname 'smithi063' 2024-05-27T07:34:58.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.412+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562c5f5d28a0 0x562c5f663610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.412+0000 7f80fcbbeb80 4 client.35363 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:34:58.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.412+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2087822869 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c5f4fc5f0 con 0x562c5f5d28a0 2024-05-27T07:34:58.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.413+0000 7f80e6a00700 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562c5f5d28a0 0x562c5f663610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.414+0000 7f80e6a00700 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562c5f5d28a0 0x562c5f663610 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc806a37738fc8e server_cookie=65eebf47a2e691f8 in_seq=0 out_seq=0 2024-05-27T07:34:58.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.414+0000 7f80d1600700 10 client.35363 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:58.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.418+0000 7f80d1600700 1 -- 172.21.15.63:0/2087822869 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c5f4fc5f0 con 0x562c5f5d28a0 2024-05-27T07:34:58.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.418+0000 7f80d1600700 10 client.35363 handle_command_reply: tid=0 2024-05-27T07:34:58.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.418+0000 7f80fcbbeb80 1 client.35363 shutdown 2024-05-27T07:34:58.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.418+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2087822869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 msgr2=0x562c5f5f0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.418+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 0x562c5f5f0300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f80d800f490 tx=0x7f80d800d650 comp rx=0 tx=0).stop 2024-05-27T07:34:58.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.418+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2087822869 shutdown_connections 2024-05-27T07:34:58.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.418+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562c5f5d28a0 0x562c5f663610 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.419+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c5f5d2490 0x562c5f5f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.419+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c5f453ec0 0x562c5f5f0300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.419+0000 7f80fcbbeb80 1 --2- 172.21.15.63:0/2087822869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562c5f4b4040 0x562c5f5f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.419+0000 7f80d1600700 0 client.35363 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:34:58.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.419+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2087822869 >> 172.21.15.63:0/2087822869 conn(0x562c5f5c6fe0 msgr2=0x562c5f5cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:58.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.419+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2087822869 shutdown_connections 2024-05-27T07:34:58.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.419+0000 7f80fcbbeb80 1 -- 172.21.15.63:0/2087822869 wait complete. 2024-05-27T07:34:58.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.419+0000 7f80fcbbeb80 20 client.35363 trim_cache size 0 max 16384 2024-05-27T07:34:58.418 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:34:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:34:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "716a7780-6390-41c3-b429-d012d6c0d454", 2024-05-27T07:34:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:34:58.418 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:34:58.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 -- 172.21.15.63:0/1902259903 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f80cc0b2c20 msgr2=0x7f80cc0b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f80cc0b2c20 0x7f80cc0b5130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x562c5f5d34b0 tx=0x7f80d800b3f0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 -- 172.21.15.63:0/1902259903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e810af40 msgr2=0x7f80e819d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e810af40 0x7f80e819d370 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c9e0 tx=0x7f80d400cda0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 -- 172.21.15.63:0/1902259903 shutdown_connections 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f80cc0b2c20 0x7f80cc0b5130 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e810b890 0x7f80e81a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80e8106610 0x7f80e819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 --2- 172.21.15.63:0/1902259903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80e810af40 0x7f80e819d370 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 -- 172.21.15.63:0/1902259903 >> 172.21.15.63:0/1902259903 conn(0x7f80e8102020 msgr2=0x7f80e810a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.420+0000 7f80eca00700 1 -- 172.21.15.63:0/1902259903 shutdown_connections 2024-05-27T07:34:58.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:34:58.421+0000 7f80eca00700 1 -- 172.21.15.63:0/1902259903 wait complete. 2024-05-27T07:34:59.326 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:34:59.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:59 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:34:59.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:34:59.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:34:59.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:34:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:34:59.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:59 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:34:59.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:34:59.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:34:59.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:34:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:34:59.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:59 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:34:59.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:34:59.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:34:59.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:34:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:34:59.870 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:00.605 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:00 smithi086 ceph-mon[34687]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:00.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:00 smithi149 ceph-mon[35408]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:00.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:00 smithi063 ceph-mon[31613]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:01.643 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:01 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:35:01.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:01 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:35:01.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:01 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:35:02.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.289+0000 7f1b8ac00700 1 -- 172.21.15.63:0/2154529104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b8410e6f0 msgr2=0x7f1b84110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:02.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.289+0000 7f1b8ac00700 1 --2- 172.21.15.63:0/2154529104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b8410e6f0 0x7f1b84110bd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00b0a0 tx=0x7f1b7c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.289+0000 7f1b8ac00700 1 -- 172.21.15.63:0/2154529104 shutdown_connections 2024-05-27T07:35:02.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.289+0000 7f1b8ac00700 1 --2- 172.21.15.63:0/2154529104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b8410e6f0 0x7f1b84110bd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.289+0000 7f1b8ac00700 1 --2- 172.21.15.63:0/2154529104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b84105280 0x7f1b84105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.289+0000 7f1b8ac00700 1 --2- 172.21.15.63:0/2154529104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b84105bd0 0x7f1b8410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.289+0000 7f1b8ac00700 1 -- 172.21.15.63:0/2154529104 >> 172.21.15.63:0/2154529104 conn(0x7f1b84078ab0 msgr2=0x7f1b84078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:02.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.289+0000 7f1b8ac00700 1 -- 172.21.15.63:0/2154529104 shutdown_connections 2024-05-27T07:35:02.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.291+0000 7f1b8ac00700 1 -- 172.21.15.63:0/2154529104 wait complete. 2024-05-27T07:35:02.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.293+0000 7f1b8ac00700 1 Processor -- start 2024-05-27T07:35:02.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.293+0000 7f1b8ac00700 1 -- start start 2024-05-27T07:35:02.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b8ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b84105280 0x7f1b841a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b8ac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b84105bd0 0x7f1b841a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b8ac00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b8410e6f0 0x7f1b8419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b8ac00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b841141c0 con 0x7f1b8410e6f0 2024-05-27T07:35:02.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b8ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b84114040 con 0x7f1b84105280 2024-05-27T07:35:02.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b8ac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b84114340 con 0x7f1b84105bd0 2024-05-27T07:35:02.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b8a200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b8410e6f0 0x7f1b8419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b89800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b84105280 0x7f1b841a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b8a200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b8410e6f0 0x7f1b8419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34336/0 (socket says 172.21.15.63:34336) 2024-05-27T07:35:02.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b8a200700 1 -- 172.21.15.63:0/1062655891 learned_addr learned my addr 172.21.15.63:0/1062655891 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:02.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.294+0000 7f1b88e00700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b84105bd0 0x7f1b841a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.295+0000 7f1b8a200700 1 -- 172.21.15.63:0/1062655891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b84105bd0 msgr2=0x7f1b841a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:02.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.295+0000 7f1b8a200700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b84105bd0 0x7f1b841a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.295+0000 7f1b8a200700 1 -- 172.21.15.63:0/1062655891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b84105280 msgr2=0x7f1b841a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:02.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.295+0000 7f1b8a200700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b84105280 0x7f1b841a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.295+0000 7f1b8a200700 1 -- 172.21.15.63:0/1062655891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8419b070 con 0x7f1b8410e6f0 2024-05-27T07:35:02.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.295+0000 7f1b88e00700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b84105bd0 0x7f1b841a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:02.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.295+0000 7f1b89800700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b84105280 0x7f1b841a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:02.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.295+0000 7f1b8a200700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b8410e6f0 0x7f1b8419a9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00b070 tx=0x7f1b7c004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.296+0000 7f1b82000700 1 -- 172.21.15.63:0/1062655891 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c021070 con 0x7f1b8410e6f0 2024-05-27T07:35:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.296+0000 7f1b8ac00700 1 -- 172.21.15.63:0/1062655891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8419b300 con 0x7f1b8410e6f0 2024-05-27T07:35:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.296+0000 7f1b8ac00700 1 -- 172.21.15.63:0/1062655891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b841a7630 con 0x7f1b8410e6f0 2024-05-27T07:35:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.296+0000 7f1b82000700 1 -- 172.21.15.63:0/1062655891 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b7c0075c0 con 0x7f1b8410e6f0 2024-05-27T07:35:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.296+0000 7f1b82000700 1 -- 172.21.15.63:0/1062655891 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c004480 con 0x7f1b8410e6f0 2024-05-27T07:35:02.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.302+0000 7f1b9aca7b80 1 Processor -- start 2024-05-27T07:35:02.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.302+0000 7f1b82000700 1 -- 172.21.15.63:0/1062655891 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f1b7c004620 con 0x7f1b8410e6f0 2024-05-27T07:35:02.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.303+0000 7f1b9aca7b80 1 -- start start 2024-05-27T07:35:02.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.303+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc09d49f0 0x55efc0a68750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.303+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a68c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.303+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc0a722c0 0x55efc0a746e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.303+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55efc0995e40 con 0x55efc0a722c0 2024-05-27T07:35:02.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.303+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55efc09859e0 con 0x55efc08e8e40 2024-05-27T07:35:02.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.303+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55efc0989a20 con 0x55efc09d49f0 2024-05-27T07:35:02.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.304+0000 7f1b89800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a68c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.304+0000 7f1b89800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a68c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44490/0 (socket says 172.21.15.63:44490) 2024-05-27T07:35:02.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.304+0000 7f1b89800700 1 -- 172.21.15.63:0/3984639068 learned_addr learned my addr 172.21.15.63:0/3984639068 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:02.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.304+0000 7f1b88e00700 1 --2- 172.21.15.63:0/3984639068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc0a722c0 0x55efc0a746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.304+0000 7f1b88e00700 1 --2- 172.21.15.63:0/3984639068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc09d49f0 0x55efc0a68750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.304+0000 7f1b89800700 1 -- 172.21.15.63:0/3984639068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc09d49f0 msgr2=0x55efc0a68750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:02.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.305+0000 7f1b89800700 1 --2- 172.21.15.63:0/3984639068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc09d49f0 0x55efc0a68750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.305+0000 7f1b89800700 1 -- 172.21.15.63:0/3984639068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc0a722c0 msgr2=0x55efc0a746e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.305+0000 7f1b89800700 1 --2- 172.21.15.63:0/3984639068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc0a722c0 0x55efc0a746e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.305+0000 7f1b89800700 1 -- 172.21.15.63:0/3984639068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc094e420 con 0x55efc08e8e40 2024-05-27T07:35:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.305+0000 7f1b82000700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b6c0b2c20 0x7f1b6c0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.305+0000 7f1b88e00700 1 --2- 172.21.15.63:0/3984639068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc0a722c0 0x55efc0a746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.305+0000 7f1b82000700 1 -- 172.21.15.63:0/1062655891 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f1b7c00f020 con 0x7f1b8410e6f0 2024-05-27T07:35:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.305+0000 7f1b89800700 1 --2- 172.21.15.63:0/3984639068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a68c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b780078c0 tx=0x7f1b78007bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c66898b7606525e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.306+0000 7f1b88e00700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b6c0b2c20 0x7f1b6c0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.306+0000 7f1b76a00700 1 -- 172.21.15.63:0/3984639068 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800d670 con 0x55efc08e8e40 2024-05-27T07:35:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.306+0000 7f1b76a00700 1 -- 172.21.15.63:0/3984639068 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b78018070 con 0x55efc08e8e40 2024-05-27T07:35:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.306+0000 7f1b76a00700 1 -- 172.21.15.63:0/3984639068 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78013c00 con 0x55efc08e8e40 2024-05-27T07:35:02.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.307+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3984639068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 msgr2=0x55efc0a68c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:02.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.307+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/3984639068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a68c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b780078c0 tx=0x7f1b78007bd0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.307+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3984639068 shutdown_connections 2024-05-27T07:35:02.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.307+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/3984639068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc0a722c0 0x55efc0a746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.307+0000 7f1b88e00700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b6c0b2c20 0x7f1b6c0b5130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b70005950 tx=0x7f1b7000b440 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:02.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.308+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/3984639068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc09d49f0 0x55efc0a68750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.308+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/3984639068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a68c90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.308+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3984639068 >> 172.21.15.63:0/3984639068 conn(0x55efc0982390 msgr2=0x55efc0a63710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:02.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.308+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3984639068 shutdown_connections 2024-05-27T07:35:02.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.308+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3984639068 wait complete. 2024-05-27T07:35:02.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.309+0000 7f1b9aca7b80 1 -- start start 2024-05-27T07:35:02.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.309+0000 7f1b9aca7b80 1 Processor -- start 2024-05-27T07:35:02.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.310+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 0x55efc0a71a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.310+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a6adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.310+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 0x55efc0a6b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.310+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55efc088bfa0 con 0x55efc09d49f0 2024-05-27T07:35:02.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.310+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55efc0926760 con 0x55efc0a722c0 2024-05-27T07:35:02.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.310+0000 7f1b89800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 0x55efc0a71a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.311+0000 7f1b89800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 0x55efc0a71a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41232/0 (socket says 172.21.15.63:41232) 2024-05-27T07:35:02.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.311+0000 7f1b89800700 1 -- 172.21.15.63:0/3465158216 learned_addr learned my addr 172.21.15.63:0/3465158216 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:02.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.311+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3465158216 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55efc0a79210 con 0x55efc08e8e40 2024-05-27T07:35:02.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.311+0000 7f1b88e00700 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a6adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.311+0000 7f1b8a200700 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 0x55efc0a6b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.311+0000 7f1b9aca7b80 10 client.0 _ll_register_callbacks cb 0x55efc092fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:02.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.312+0000 7f1b8a200700 1 -- 172.21.15.63:0/3465158216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 msgr2=0x55efc0a71a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:02.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.312+0000 7f1b8a200700 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 0x55efc0a71a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.312+0000 7f1b8a200700 1 -- 172.21.15.63:0/3465158216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 msgr2=0x55efc0a6adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:02.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.312+0000 7f1b8a200700 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a6adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.312+0000 7f1b8a200700 1 -- 172.21.15.63:0/3465158216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc08d58d0 con 0x55efc0a722c0 2024-05-27T07:35:02.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.312+0000 7f1b8a200700 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 0x55efc0a6b2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0e1310 tx=0x7f1b7c0ac3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c746eb0f64cc3de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:02.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.312+0000 7f1b6b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:35:02.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.313+0000 7f1b6b400700 1 -- 172.21.15.63:0/3465158216 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c007c00 con 0x55efc0a722c0 2024-05-27T07:35:02.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.313+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3465158216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b78018070 con 0x55efc0a722c0 2024-05-27T07:35:02.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.313+0000 7f1b6b400700 1 -- 172.21.15.63:0/3465158216 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f1b7c0ab2f0 con 0x55efc0a722c0 2024-05-27T07:35:02.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.313+0000 7f1b6b400700 1 -- 172.21.15.63:0/3465158216 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b7c0ab990 con 0x55efc0a722c0 2024-05-27T07:35:02.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.313+0000 7f1b9aca7b80 10 client.35375 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:02.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.313+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3465158216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55efc0956060 con 0x55efc0a722c0 2024-05-27T07:35:02.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.314+0000 7f1b6b400700 1 -- 172.21.15.63:0/3465158216 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b78018070 con 0x55efc0a722c0 2024-05-27T07:35:02.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.314+0000 7f1b6b400700 1 -- 172.21.15.63:0/3465158216 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1b7c011070 con 0x55efc0a722c0 2024-05-27T07:35:02.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.314+0000 7f1b9aca7b80 10 client.35375 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:02.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.314+0000 7f1b9aca7b80 10 client.35375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:02.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.314+0000 7f1b9aca7b80 20 client.35375 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:02.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.314+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc0a8a5a0 0x55efc0a8c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.314+0000 7f1b9aca7b80 4 client.35375 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:02.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.314+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3465158216 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc0997d90 con 0x55efc0a8a5a0 2024-05-27T07:35:02.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.315+0000 7f1b89800700 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc0a8a5a0 0x55efc0a8c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.316+0000 7f1b89800700 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc0a8a5a0 0x55efc0a8c9c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a0ec4ba571a55b server_cookie=9ad132284d25a38e in_seq=0 out_seq=0 2024-05-27T07:35:02.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.316+0000 7f1b6b400700 10 client.35375 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:02.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.318+0000 7f1b6b400700 1 -- 172.21.15.63:0/3465158216 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55efc0997d90 con 0x55efc0a8a5a0 2024-05-27T07:35:02.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.318+0000 7f1b6b400700 10 client.35375 handle_command_reply: tid=0 2024-05-27T07:35:02.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.318+0000 7f1b9aca7b80 1 client.35375 shutdown 2024-05-27T07:35:02.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.318+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3465158216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 msgr2=0x55efc0a6b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:02.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.318+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 0x55efc0a6b2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0e1310 tx=0x7f1b7c0ac3c0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.318+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3465158216 shutdown_connections 2024-05-27T07:35:02.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.318+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc0a8a5a0 0x55efc0a8c9c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.319+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 0x55efc0a6b2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.319+0000 7f1b6b400700 0 client.35375 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:02.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.319+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 0x55efc0a71a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.319+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/3465158216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a6adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.319+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3465158216 >> 172.21.15.63:0/3465158216 conn(0x55efc0982390 msgr2=0x55efc0a63260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:02.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.319+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3465158216 shutdown_connections 2024-05-27T07:35:02.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.319+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/3465158216 wait complete. 2024-05-27T07:35:02.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.319+0000 7f1b9aca7b80 20 client.35375 trim_cache size 0 max 16384 2024-05-27T07:35:02.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.342+0000 7f1b9aca7b80 1 Processor -- start 2024-05-27T07:35:02.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.342+0000 7f1b9aca7b80 1 -- start start 2024-05-27T07:35:02.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.342+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 0x55efc0a75410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.343+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a6b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.343+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 0x55efc0a736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.343+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b64011400 con 0x55efc09d49f0 2024-05-27T07:35:02.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.343+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6400ec90 con 0x55efc0a722c0 2024-05-27T07:35:02.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.343+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6400cae0 con 0x55efc08e8e40 2024-05-27T07:35:02.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.343+0000 7f1b88e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a6b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.343+0000 7f1b88e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a6b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34372/0 (socket says 172.21.15.63:34372) 2024-05-27T07:35:02.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.343+0000 7f1b88e00700 1 -- 172.21.15.63:0/2079213619 learned_addr learned my addr 172.21.15.63:0/2079213619 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:02.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.343+0000 7f1b89800700 1 --2- 172.21.15.63:0/2079213619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 0x55efc0a75410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.344+0000 7f1b8a200700 1 --2- 172.21.15.63:0/2079213619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 0x55efc0a736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.344+0000 7f1b88e00700 1 -- 172.21.15.63:0/2079213619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 msgr2=0x55efc0a75410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:02.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.344+0000 7f1b88e00700 1 --2- 172.21.15.63:0/2079213619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 0x55efc0a75410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.344+0000 7f1b88e00700 1 -- 172.21.15.63:0/2079213619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 msgr2=0x55efc0a736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:02.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.344+0000 7f1b88e00700 1 --2- 172.21.15.63:0/2079213619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 0x55efc0a736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.344+0000 7f1b88e00700 1 -- 172.21.15.63:0/2079213619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc0a68e30 con 0x55efc09d49f0 2024-05-27T07:35:02.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.345+0000 7f1b88e00700 1 --2- 172.21.15.63:0/2079213619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a6b2a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b84102650 tx=0x7f1b70015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44ebca1694bf153 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:02.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.346+0000 7f1b76000700 1 -- 172.21.15.63:0/2079213619 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7002a640 con 0x55efc09d49f0 2024-05-27T07:35:02.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.346+0000 7f1b76000700 1 -- 172.21.15.63:0/2079213619 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b70029870 con 0x55efc09d49f0 2024-05-27T07:35:02.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.346+0000 7f1b76000700 1 -- 172.21.15.63:0/2079213619 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70010780 con 0x55efc09d49f0 2024-05-27T07:35:02.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.347+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/2079213619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 msgr2=0x55efc0a6b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:02.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.347+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/2079213619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a6b2a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b84102650 tx=0x7f1b70015070 comp rx=0 tx=0).stop 2024-05-27T07:35:02.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.347+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/2079213619 shutdown_connections 2024-05-27T07:35:02.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.347+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/2079213619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc0a722c0 0x55efc0a736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.347+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/2079213619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc08e8e40 0x55efc0a75410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.347+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/2079213619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a6b2a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.347+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/2079213619 >> 172.21.15.63:0/2079213619 conn(0x55efc0982390 msgr2=0x55efc0a6ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:02.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.348+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/2079213619 shutdown_connections 2024-05-27T07:35:02.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.348+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/2079213619 wait complete. 2024-05-27T07:35:02.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.348+0000 7f1b9aca7b80 1 -- start start 2024-05-27T07:35:02.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.349+0000 7f1b9aca7b80 1 Processor -- start 2024-05-27T07:35:02.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.349+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a87190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.349+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a87730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.349+0000 7f1b9aca7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc0a722c0 0x55efc0a81dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.349+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6400d150 con 0x55efc09d49f0 2024-05-27T07:35:02.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.349+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6400e5c0 con 0x55efc08e8e40 2024-05-27T07:35:02.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.349+0000 7f1b9aca7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6400d820 con 0x55efc0a722c0 2024-05-27T07:35:02.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.350+0000 7f1b9aca7b80 10 client.0 _ll_register_callbacks cb 0x55efc092fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:02.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.350+0000 7f1b8a200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc0a722c0 0x55efc0a81dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.350+0000 7f1b8a200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc0a722c0 0x55efc0a81dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41250/0 (socket says 172.21.15.63:41250) 2024-05-27T07:35:02.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.350+0000 7f1b8a200700 1 -- 172.21.15.63:0/4167713519 learned_addr learned my addr 172.21.15.63:0/4167713519 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:02.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.350+0000 7f1b89800700 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a87190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.351+0000 7f1b88e00700 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a87730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.351+0000 7f1b89800700 1 -- 172.21.15.63:0/4167713519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc0a722c0 msgr2=0x55efc0a81dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:02.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.351+0000 7f1b89800700 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc0a722c0 0x55efc0a81dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.351+0000 7f1b89800700 1 -- 172.21.15.63:0/4167713519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 msgr2=0x55efc0a87730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:02.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.351+0000 7f1b89800700 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a87730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.351+0000 7f1b89800700 1 -- 172.21.15.63:0/4167713519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc0a82560 con 0x55efc08e8e40 2024-05-27T07:35:02.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.351+0000 7f1b8a200700 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc0a722c0 0x55efc0a81dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:35:02.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.352+0000 7f1b89800700 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a87190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55efc0a6a630 tx=0x7f1b780071f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ea54802afa22405 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:02.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.352+0000 7f1b6b400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:35:02.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.352+0000 7f1b6b400700 1 -- 172.21.15.63:0/4167713519 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7802d070 con 0x55efc08e8e40 2024-05-27T07:35:02.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.352+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/4167713519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b70029870 con 0x55efc08e8e40 2024-05-27T07:35:02.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.352+0000 7f1b6b400700 1 -- 172.21.15.63:0/4167713519 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f1b78013cf0 con 0x55efc08e8e40 2024-05-27T07:35:02.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.353+0000 7f1b6b400700 1 -- 172.21.15.63:0/4167713519 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b7800d670 con 0x55efc08e8e40 2024-05-27T07:35:02.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.353+0000 7f1b9aca7b80 10 client.35381 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:02.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.353+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/4167713519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55efc0af5a60 con 0x55efc08e8e40 2024-05-27T07:35:02.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.354+0000 7f1b6b400700 1 -- 172.21.15.63:0/4167713519 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b70029870 con 0x55efc08e8e40 2024-05-27T07:35:02.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.354+0000 7f1b6b400700 1 -- 172.21.15.63:0/4167713519 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1b78027070 con 0x55efc08e8e40 2024-05-27T07:35:02.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.354+0000 7f1b9aca7b80 10 client.35381 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:02.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.354+0000 7f1b9aca7b80 10 client.35381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:02.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.354+0000 7f1b9aca7b80 20 client.35381 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:02.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.355+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc0a8a5a0 0x55efc0af7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:02.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.355+0000 7f1b9aca7b80 4 client.35381 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:35:02.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.355+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/4167713519 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55efc0991570 con 0x55efc0a8a5a0 2024-05-27T07:35:02.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.355+0000 7f1b88e00700 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc0a8a5a0 0x55efc0af7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:02.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.356+0000 7f1b88e00700 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc0a8a5a0 0x55efc0af7df0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27af921a50843585 server_cookie=db78250c6bb38cb9 in_seq=0 out_seq=0 2024-05-27T07:35:02.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.356+0000 7f1b6b400700 10 client.35381 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:02.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b6b400700 1 -- 172.21.15.63:0/4167713519 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55efc0991570 con 0x55efc0a8a5a0 2024-05-27T07:35:02.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b6b400700 10 client.35381 handle_command_reply: tid=0 2024-05-27T07:35:02.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b9aca7b80 1 client.35381 shutdown 2024-05-27T07:35:02.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/4167713519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 msgr2=0x55efc0a87190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:02.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a87190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55efc0a6a630 tx=0x7f1b780071f0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/4167713519 shutdown_connections 2024-05-27T07:35:02.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc0a8a5a0 0x55efc0af7df0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc0a722c0 0x55efc0a81dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc08e8e40 0x55efc0a87190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b9aca7b80 1 --2- 172.21.15.63:0/4167713519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc09d49f0 0x55efc0a87730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/4167713519 >> 172.21.15.63:0/4167713519 conn(0x55efc0982390 msgr2=0x55efc0a67120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:02.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.357+0000 7f1b6b400700 0 client.35381 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:02.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.358+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/4167713519 shutdown_connections 2024-05-27T07:35:02.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.358+0000 7f1b9aca7b80 1 -- 172.21.15.63:0/4167713519 wait complete. 2024-05-27T07:35:02.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.358+0000 7f1b9aca7b80 20 client.35381 trim_cache size 0 max 16384 2024-05-27T07:35:02.354 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:35:02.354 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:35:02.354 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:35:02.354 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:35:02.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.359+0000 7f1b8ac00700 1 -- 172.21.15.63:0/1062655891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b6c0b2c20 msgr2=0x7f1b6c0b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:02.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.359+0000 7f1b8ac00700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b6c0b2c20 0x7f1b6c0b5130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b70005950 tx=0x7f1b7000b440 comp rx=0 tx=0).stop 2024-05-27T07:35:02.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.359+0000 7f1b8ac00700 1 -- 172.21.15.63:0/1062655891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b8410e6f0 msgr2=0x7f1b8419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:02.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.359+0000 7f1b8ac00700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b8410e6f0 0x7f1b8419a9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00b070 tx=0x7f1b7c004e00 comp rx=0 tx=0).stop 2024-05-27T07:35:02.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.359+0000 7f1b8ac00700 1 -- 172.21.15.63:0/1062655891 shutdown_connections 2024-05-27T07:35:02.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.359+0000 7f1b8ac00700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b6c0b2c20 0x7f1b6c0b5130 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.359+0000 7f1b8ac00700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b8410e6f0 0x7f1b8419a9c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.359+0000 7f1b8ac00700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b84105280 0x7f1b841a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.359+0000 7f1b8ac00700 1 --2- 172.21.15.63:0/1062655891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b84105bd0 0x7f1b841a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:02.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.359+0000 7f1b8ac00700 1 -- 172.21.15.63:0/1062655891 >> 172.21.15.63:0/1062655891 conn(0x7f1b84078ab0 msgr2=0x7f1b840fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:02.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.360+0000 7f1b8ac00700 1 -- 172.21.15.63:0/1062655891 shutdown_connections 2024-05-27T07:35:02.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:02.360+0000 7f1b8ac00700 1 -- 172.21.15.63:0/1062655891 wait complete. 2024-05-27T07:35:02.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:02 smithi086 ceph-mon[34687]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T07:35:02.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:02 smithi149 ceph-mon[35408]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T07:35:02.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:02 smithi063 ceph-mon[31613]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T07:35:03.216 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:35:03.216 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:35:03.732 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:04.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:04 smithi063 ceph-mon[31613]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-27T07:35:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.678+0000 7fabaf200640 1 -- 172.21.15.63:0/1942067138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb0104250 msgr2=0x7fabb01105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.678+0000 7fabaf200640 1 --2- 172.21.15.63:0/1942067138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb0104250 0x7fabb01105d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faba8009ca0 tx=0x7faba802e990 comp rx=0 tx=0).stop 2024-05-27T07:35:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.678+0000 7fabaf200640 1 -- 172.21.15.63:0/1942067138 shutdown_connections 2024-05-27T07:35:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.678+0000 7fabaf200640 1 --2- 172.21.15.63:0/1942067138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb0104250 0x7fabb01105d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.678+0000 7fabaf200640 1 --2- 172.21.15.63:0/1942067138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0109f50 0x7fabb0103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.678+0000 7fabaf200640 1 --2- 172.21.15.63:0/1942067138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabb0109ab0 0x7fabb01036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.678+0000 7fabaf200640 1 -- 172.21.15.63:0/1942067138 >> 172.21.15.63:0/1942067138 conn(0x7fabb0079240 msgr2=0x7fabb0079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.679+0000 7fabaf200640 1 -- 172.21.15.63:0/1942067138 shutdown_connections 2024-05-27T07:35:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.679+0000 7fabaf200640 1 -- 172.21.15.63:0/1942067138 wait complete. 2024-05-27T07:35:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.680+0000 7fabaf200640 1 Processor -- start 2024-05-27T07:35:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.680+0000 7fabaf200640 1 -- start start 2024-05-27T07:35:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.681+0000 7fabaf200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb0104250 0x7fabb01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.681+0000 7fabaf200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabb0109ab0 0x7fabb0198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.681+0000 7fabaf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0109f50 0x7fabb019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.681+0000 7fabaf200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0111960 con 0x7fabb0104250 2024-05-27T07:35:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.681+0000 7fabaf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01117e0 con 0x7fabb0109f50 2024-05-27T07:35:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.681+0000 7fabaf200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fabb0111ae0 con 0x7fabb0109ab0 2024-05-27T07:35:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.681+0000 7fabade00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb0104250 0x7fabb01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.682+0000 7fabade00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb0104250 0x7fabb01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34414/0 (socket says 172.21.15.63:34414) 2024-05-27T07:35:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.682+0000 7fabade00640 1 -- 172.21.15.63:0/2145860427 learned_addr learned my addr 172.21.15.63:0/2145860427 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.682+0000 7fabad400640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabb0109ab0 0x7fabb0198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.682+0000 7fabae800640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0109f50 0x7fabb019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.682+0000 7fabade00640 1 -- 172.21.15.63:0/2145860427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabb0109ab0 msgr2=0x7fabb0198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.682+0000 7fabade00640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabb0109ab0 0x7fabb0198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.682+0000 7fabade00640 1 -- 172.21.15.63:0/2145860427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0109f50 msgr2=0x7fabb019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.682+0000 7fabade00640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0109f50 0x7fabb019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.682+0000 7fabade00640 1 -- 172.21.15.63:0/2145860427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb019d220 con 0x7fabb0104250 2024-05-27T07:35:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.682+0000 7fabad400640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabb0109ab0 0x7fabb0198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.683+0000 7fabade00640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb0104250 0x7fabb01983c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7faba000e9f0 tx=0x7faba000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.683+0000 7faba6a00640 1 -- 172.21.15.63:0/2145860427 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba000ccc0 con 0x7fabb0104250 2024-05-27T07:35:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.683+0000 7faba6a00640 1 -- 172.21.15.63:0/2145860427 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faba00054b0 con 0x7fabb0104250 2024-05-27T07:35:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.683+0000 7faba6a00640 1 -- 172.21.15.63:0/2145860427 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0010430 con 0x7fabb0104250 2024-05-27T07:35:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.683+0000 7fabaf200640 1 -- 172.21.15.63:0/2145860427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb019d510 con 0x7fabb0104250 2024-05-27T07:35:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.684+0000 7fabaf200640 1 -- 172.21.15.63:0/2145860427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb019da10 con 0x7fabb0104250 2024-05-27T07:35:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.688+0000 7fabbb9d5740 1 Processor -- start 2024-05-27T07:35:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.689+0000 7fabbb9d5740 1 -- start start 2024-05-27T07:35:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.689+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff5efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.689+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff2242f0 0x559bff5effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.689+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff22c4d0 0x559bff5f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.689+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559bff560b90 con 0x559bff2242f0 2024-05-27T07:35:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.689+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559bff550730 con 0x559bff24d1b0 2024-05-27T07:35:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.689+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559bff554770 con 0x559bff22c4d0 2024-05-27T07:35:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.690+0000 7fabae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff5efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.690+0000 7fabae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff5efa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44554/0 (socket says 172.21.15.63:44554) 2024-05-27T07:35:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.690+0000 7fabae800640 1 -- 172.21.15.63:0/3018509087 learned_addr learned my addr 172.21.15.63:0/3018509087 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.690+0000 7fabad400640 1 --2- 172.21.15.63:0/3018509087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff22c4d0 0x559bff5f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.690+0000 7faba6a00640 1 -- 172.21.15.63:0/2145860427 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7faba00046c0 con 0x7fabb0104250 2024-05-27T07:35:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.691+0000 7fabae800640 1 -- 172.21.15.63:0/3018509087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff22c4d0 msgr2=0x559bff5f04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.691+0000 7fabae800640 1 --2- 172.21.15.63:0/3018509087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff22c4d0 0x559bff5f04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.691+0000 7fabae800640 1 -- 172.21.15.63:0/3018509087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff2242f0 msgr2=0x559bff5effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.691+0000 7fabae800640 1 --2- 172.21.15.63:0/3018509087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff2242f0 0x559bff5effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.691+0000 7fabae800640 1 -- 172.21.15.63:0/3018509087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bff4e98b0 con 0x559bff24d1b0 2024-05-27T07:35:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.691+0000 7fabad400640 1 --2- 172.21.15.63:0/3018509087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff22c4d0 0x559bff5f04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:35:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.692+0000 7fabae800640 1 --2- 172.21.15.63:0/3018509087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff5efa60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faba8009a20 tx=0x7faba80093a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80c252bea8c134a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.692+0000 7fab9b000640 1 -- 172.21.15.63:0/3018509087 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba804d070 con 0x559bff24d1b0 2024-05-27T07:35:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.692+0000 7fab9b000640 1 -- 172.21.15.63:0/3018509087 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faba8003d30 con 0x559bff24d1b0 2024-05-27T07:35:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.692+0000 7fab9b000640 1 -- 172.21.15.63:0/3018509087 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8038050 con 0x559bff24d1b0 2024-05-27T07:35:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.693+0000 7faba6a00640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fab940b2b80 0x7fab940b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.693+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3018509087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 msgr2=0x559bff5efa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.693+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/3018509087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff5efa60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faba8009a20 tx=0x7faba80093a0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.694+0000 7faba6a00640 1 -- 172.21.15.63:0/2145860427 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7faba00e2fa0 con 0x7fabb0104250 2024-05-27T07:35:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.694+0000 7fabade00640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fab940b2b80 0x7fab940b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.694+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3018509087 shutdown_connections 2024-05-27T07:35:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.694+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/3018509087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff22c4d0 0x559bff5f04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.694+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/3018509087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff2242f0 0x559bff5effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.694+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/3018509087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff5efa60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.694+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3018509087 >> 172.21.15.63:0/3018509087 conn(0x559bff241300 msgr2=0x559bff5ec210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.695+0000 7fabade00640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fab940b2b80 0x7fab940b5090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faba00e01a0 tx=0x7faba00e3220 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.695+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3018509087 shutdown_connections 2024-05-27T07:35:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.695+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3018509087 wait complete. 2024-05-27T07:35:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.696+0000 7fabbb9d5740 1 -- start start 2024-05-27T07:35:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.696+0000 7fabbb9d5740 1 Processor -- start 2024-05-27T07:35:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.697+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 0x559bff5f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.697+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff22c4d0 0x559bff5fa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.697+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff605e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.697+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559bff236790 con 0x559bff22c4d0 2024-05-27T07:35:04.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.697+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559bff4f8bc0 con 0x559bff24d1b0 2024-05-27T07:35:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.697+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559bff5fcb30 con 0x559bff2242f0 2024-05-27T07:35:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.697+0000 7fabbb9d5740 10 client.0 _ll_register_callbacks cb 0x559bff4d6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabad400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff22c4d0 0x559bff5fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabad400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff22c4d0 0x559bff5fa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34434/0 (socket says 172.21.15.63:34434) 2024-05-27T07:35:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabad400640 1 -- 172.21.15.63:0/4135576412 learned_addr learned my addr 172.21.15.63:0/4135576412 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabae800640 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 0x559bff5f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabae800640 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff605e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabad400640 1 -- 172.21.15.63:0/4135576412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 msgr2=0x559bff5f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabad400640 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 0x559bff5f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabad400640 1 -- 172.21.15.63:0/4135576412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 msgr2=0x559bff605e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabad400640 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff605e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabad400640 1 -- 172.21.15.63:0/4135576412 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bff265210 con 0x559bff22c4d0 2024-05-27T07:35:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.698+0000 7fabae800640 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff605e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.699+0000 7fabad400640 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff22c4d0 0x559bff5fa170 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab9c012700 tx=0x7fab9c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24242798f2ebb5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.699+0000 7fab8f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.699+0000 7fab8f400640 1 -- 172.21.15.63:0/4135576412 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c004260 con 0x559bff22c4d0 2024-05-27T07:35:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.699+0000 7fab8f400640 1 -- 172.21.15.63:0/4135576412 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fab9c004400 con 0x559bff22c4d0 2024-05-27T07:35:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.699+0000 7fabbb9d5740 1 -- 172.21.15.63:0/4135576412 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba8003d30 con 0x559bff22c4d0 2024-05-27T07:35:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.700+0000 7fab8f400640 1 -- 172.21.15.63:0/4135576412 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab9c025050 con 0x559bff22c4d0 2024-05-27T07:35:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.700+0000 7fabbb9d5740 10 client.15627 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.700+0000 7fabbb9d5740 1 -- 172.21.15.63:0/4135576412 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559bff331620 con 0x559bff22c4d0 2024-05-27T07:35:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.700+0000 7fab8f400640 1 -- 172.21.15.63:0/4135576412 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faba8003d30 con 0x559bff22c4d0 2024-05-27T07:35:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.700+0000 7fab8f400640 1 -- 172.21.15.63:0/4135576412 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fab9c003cc0 con 0x559bff22c4d0 2024-05-27T07:35:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.701+0000 7fabbb9d5740 10 client.15627 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.701+0000 7fabbb9d5740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.701+0000 7fabbb9d5740 20 client.15627 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.701+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559bff20bd80 0x559bff6146b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.701+0000 7fabbb9d5740 4 client.15627 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.701+0000 7fabbb9d5740 1 -- 172.21.15.63:0/4135576412 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bff562ae0 con 0x559bff20bd80 2024-05-27T07:35:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.702+0000 7fabae800640 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559bff20bd80 0x559bff6146b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.703+0000 7fabae800640 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559bff20bd80 0x559bff6146b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0a22652687ec241 server_cookie=3716b7c50c9ba9c0 in_seq=0 out_seq=0 2024-05-27T07:35:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.703+0000 7fab8f400640 10 client.15627 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.705+0000 7fab8f400640 1 -- 172.21.15.63:0/4135576412 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559bff562ae0 con 0x559bff20bd80 2024-05-27T07:35:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.705+0000 7fab8f400640 10 client.15627 handle_command_reply: tid=0 2024-05-27T07:35:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.705+0000 7fabbb9d5740 1 client.15627 shutdown 2024-05-27T07:35:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.705+0000 7fabbb9d5740 1 -- 172.21.15.63:0/4135576412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff22c4d0 msgr2=0x559bff5fa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.705+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff22c4d0 0x559bff5fa170 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab9c012700 tx=0x7fab9c012bc0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.706+0000 7fabbb9d5740 1 -- 172.21.15.63:0/4135576412 shutdown_connections 2024-05-27T07:35:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.706+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559bff20bd80 0x559bff6146b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.706+0000 7fab8f400640 0 client.15627 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.706+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff605e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.706+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff22c4d0 0x559bff5fa170 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.706+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/4135576412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 0x559bff5f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.706+0000 7fabbb9d5740 1 -- 172.21.15.63:0/4135576412 >> 172.21.15.63:0/4135576412 conn(0x559bff241300 msgr2=0x559bff5ef890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.706+0000 7fabbb9d5740 1 -- 172.21.15.63:0/4135576412 shutdown_connections 2024-05-27T07:35:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.706+0000 7fabbb9d5740 1 -- 172.21.15.63:0/4135576412 wait complete. 2024-05-27T07:35:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.706+0000 7fabbb9d5740 20 client.15627 trim_cache size 0 max 16384 2024-05-27T07:35:04.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:04 smithi086 ceph-mon[34687]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-27T07:35:04.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:04 smithi149 ceph-mon[35408]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-27T07:35:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.739+0000 7fabbb9d5740 1 Processor -- start 2024-05-27T07:35:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.739+0000 7fabbb9d5740 1 -- start start 2024-05-27T07:35:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.739+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 0x559bff5f5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.739+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff605bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.739+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff5f9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.739+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01b16d0 con 0x559bff1e11c0 2024-05-27T07:35:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.739+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01aefa0 con 0x559bff24d1b0 2024-05-27T07:35:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.739+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01acdf0 con 0x559bff2242f0 2024-05-27T07:35:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.740+0000 7fabae800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 0x559bff5f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.740+0000 7fabae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff5f9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.740+0000 7fabae800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 0x559bff5f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41298/0 (socket says 172.21.15.63:41298) 2024-05-27T07:35:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.740+0000 7fabae800640 1 -- 172.21.15.63:0/2591147119 learned_addr learned my addr 172.21.15.63:0/2591147119 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.740+0000 7fabad400640 1 --2- 172.21.15.63:0/2591147119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff605bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.741+0000 7fabae800640 1 -- 172.21.15.63:0/2591147119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 msgr2=0x559bff5f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.741+0000 7fabae800640 1 --2- 172.21.15.63:0/2591147119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 0x559bff5f5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.741+0000 7fabae800640 1 -- 172.21.15.63:0/2591147119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 msgr2=0x559bff605bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.741+0000 7fabae800640 1 --2- 172.21.15.63:0/2591147119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff605bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.741+0000 7fabae800640 1 -- 172.21.15.63:0/2591147119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bff5ef3f0 con 0x559bff1e11c0 2024-05-27T07:35:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.742+0000 7fabad400640 1 --2- 172.21.15.63:0/2591147119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff605bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:35:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.742+0000 7fabae800640 1 --2- 172.21.15.63:0/2591147119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff5f9bd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faba8009620 tx=0x7faba802e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c90f74ab0739e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.743+0000 7fab9a600640 1 -- 172.21.15.63:0/2591147119 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba804d040 con 0x559bff1e11c0 2024-05-27T07:35:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.743+0000 7fab9a600640 1 -- 172.21.15.63:0/2591147119 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faba804e040 con 0x559bff1e11c0 2024-05-27T07:35:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.743+0000 7fab9a600640 1 -- 172.21.15.63:0/2591147119 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8004560 con 0x559bff1e11c0 2024-05-27T07:35:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.744+0000 7fabbb9d5740 1 -- 172.21.15.63:0/2591147119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 msgr2=0x559bff5f9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.744+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/2591147119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff5f9bd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faba8009620 tx=0x7faba802e990 comp rx=0 tx=0).stop 2024-05-27T07:35:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.745+0000 7fabbb9d5740 1 -- 172.21.15.63:0/2591147119 shutdown_connections 2024-05-27T07:35:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.745+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/2591147119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff5f9bd0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.745+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/2591147119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff24d1b0 0x559bff605bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.745+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/2591147119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff2242f0 0x559bff5f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.745+0000 7fabbb9d5740 1 -- 172.21.15.63:0/2591147119 >> 172.21.15.63:0/2591147119 conn(0x559bff241300 msgr2=0x559bff5f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.745+0000 7fabbb9d5740 1 -- 172.21.15.63:0/2591147119 shutdown_connections 2024-05-27T07:35:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.745+0000 7fabbb9d5740 1 -- 172.21.15.63:0/2591147119 wait complete. 2024-05-27T07:35:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.746+0000 7fabbb9d5740 1 -- start start 2024-05-27T07:35:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.746+0000 7fabbb9d5740 1 Processor -- start 2024-05-27T07:35:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.747+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff602260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.747+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff2242f0 0x559bff5fd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.748+0000 7fabbb9d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff24d1b0 0x559bff5fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.748+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01ad460 con 0x559bff1e11c0 2024-05-27T07:35:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.748+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01ae8d0 con 0x559bff2242f0 2024-05-27T07:35:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.748+0000 7fabbb9d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fabb01adb30 con 0x559bff24d1b0 2024-05-27T07:35:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.748+0000 7fabbb9d5740 10 client.0 _ll_register_callbacks cb 0x7fab9c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.748+0000 7fabae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff602260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.748+0000 7fabae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff602260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34444/0 (socket says 172.21.15.63:34444) 2024-05-27T07:35:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.748+0000 7fabae800640 1 -- 172.21.15.63:0/3044036279 learned_addr learned my addr 172.21.15.63:0/3044036279 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.748+0000 7fabad400640 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff2242f0 0x559bff5fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.749+0000 7fabae800640 1 -- 172.21.15.63:0/3044036279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff24d1b0 msgr2=0x559bff5fdb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:35:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.749+0000 7fabae800640 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff24d1b0 0x559bff5fdb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.749+0000 7fabae800640 1 -- 172.21.15.63:0/3044036279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff2242f0 msgr2=0x559bff5fd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.749+0000 7fabae800640 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff2242f0 0x559bff5fd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.749+0000 7fabae800640 1 -- 172.21.15.63:0/3044036279 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bff5fe0d0 con 0x559bff1e11c0 2024-05-27T07:35:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.749+0000 7fabae800640 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff602260 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faba8005e30 tx=0x7faba8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce2dfc89218facc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.749+0000 7fab8f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.750+0000 7fab8f400640 1 -- 172.21.15.63:0/3044036279 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8064020 con 0x559bff1e11c0 2024-05-27T07:35:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.750+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3044036279 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba804e040 con 0x559bff1e11c0 2024-05-27T07:35:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.750+0000 7fab8f400640 1 -- 172.21.15.63:0/3044036279 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7faba8034690 con 0x559bff1e11c0 2024-05-27T07:35:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.750+0000 7fab8f400640 1 -- 172.21.15.63:0/3044036279 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faba806f050 con 0x559bff1e11c0 2024-05-27T07:35:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.750+0000 7fabbb9d5740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.750+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3044036279 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559bff67c950 con 0x559bff1e11c0 2024-05-27T07:35:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.751+0000 7fab8f400640 1 -- 172.21.15.63:0/3044036279 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faba804e040 con 0x559bff1e11c0 2024-05-27T07:35:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.751+0000 7fab8f400640 1 -- 172.21.15.63:0/3044036279 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faba800b040 con 0x559bff1e11c0 2024-05-27T07:35:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.751+0000 7fabbb9d5740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.751+0000 7fabbb9d5740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.752+0000 7fabbb9d5740 20 client.15633 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.752+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559bff67cbd0 0x559bff67eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.752+0000 7fabbb9d5740 4 client.15633 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:35:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.752+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3044036279 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559bff55c2c0 con 0x559bff67cbd0 2024-05-27T07:35:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.752+0000 7fabad400640 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559bff67cbd0 0x559bff67eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.754+0000 7fabad400640 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559bff67cbd0 0x559bff67eff0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec27003238a02f8 server_cookie=3de4e588059fb419 in_seq=0 out_seq=0 2024-05-27T07:35:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.754+0000 7fab8f400640 10 client.15633 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.770+0000 7fab8f400640 1 -- 172.21.15.63:0/3044036279 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559bff55c2c0 con 0x559bff67cbd0 2024-05-27T07:35:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.770+0000 7fab8f400640 10 client.15633 handle_command_reply: tid=0 2024-05-27T07:35:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.770+0000 7fabbb9d5740 1 client.15633 shutdown 2024-05-27T07:35:04.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3044036279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 msgr2=0x559bff602260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff602260 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faba8005e30 tx=0x7faba8009bf0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3044036279 shutdown_connections 2024-05-27T07:35:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559bff67cbd0 0x559bff67eff0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bff24d1b0 0x559bff5fdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559bff2242f0 0x559bff5fd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fabbb9d5740 1 --2- 172.21.15.63:0/3044036279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559bff1e11c0 0x559bff602260 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3044036279 >> 172.21.15.63:0/3044036279 conn(0x559bff241300 msgr2=0x559bff5f9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fab8f400640 0 client.15633 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3044036279 shutdown_connections 2024-05-27T07:35:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.771+0000 7fabbb9d5740 1 -- 172.21.15.63:0/3044036279 wait complete. 2024-05-27T07:35:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.772+0000 7fabbb9d5740 20 client.15633 trim_cache size 0 max 16384 2024-05-27T07:35:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:35:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:35:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:35:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 -- 172.21.15.63:0/2145860427 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fab940b2b80 msgr2=0x7fab940b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fab940b2b80 0x7fab940b5090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faba00e01a0 tx=0x7faba00e3220 comp rx=0 tx=0).stop 2024-05-27T07:35:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 -- 172.21.15.63:0/2145860427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb0104250 msgr2=0x7fabb01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb0104250 0x7fabb01983c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7faba000e9f0 tx=0x7faba000eeb0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 -- 172.21.15.63:0/2145860427 shutdown_connections 2024-05-27T07:35:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fab940b2b80 0x7fab940b5090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabb0109f50 0x7fabb019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabb0109ab0 0x7fabb0198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 --2- 172.21.15.63:0/2145860427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabb0104250 0x7fabb01983c0 secure :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7faba000e9f0 tx=0x7faba000eeb0 comp rx=0 tx=0).stop 2024-05-27T07:35:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 -- 172.21.15.63:0/2145860427 >> 172.21.15.63:0/2145860427 conn(0x7fabb0079240 msgr2=0x7fabb0100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.773+0000 7fabaf200640 1 -- 172.21.15.63:0/2145860427 shutdown_connections 2024-05-27T07:35:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:04.774+0000 7fabaf200640 1 -- 172.21.15.63:0/2145860427 wait complete. 2024-05-27T07:35:06.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.061+0000 7f20a5200700 1 -- 172.21.15.63:0/545496575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20a01032d0 msgr2=0x7f20a01036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.061+0000 7f20a5200700 1 --2- 172.21.15.63:0/545496575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20a01032d0 0x7f20a01036e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f20900099c0 tx=0x7f2090009cd0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.061+0000 7f20a5200700 1 -- 172.21.15.63:0/545496575 shutdown_connections 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.061+0000 7f20a5200700 1 --2- 172.21.15.63:0/545496575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20a0108550 0x7f20a010feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.061+0000 7f20a5200700 1 --2- 172.21.15.63:0/545496575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20a01032d0 0x7f20a01036e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.061+0000 7f20a5200700 1 --2- 172.21.15.63:0/545496575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0107c00 0x7f20a0108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.061+0000 7f20a5200700 1 -- 172.21.15.63:0/545496575 >> 172.21.15.63:0/545496575 conn(0x7f20a0078ab0 msgr2=0x7f20a0078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.062+0000 7f20a5200700 1 -- 172.21.15.63:0/545496575 shutdown_connections 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.062+0000 7f20a5200700 1 -- 172.21.15.63:0/545496575 wait complete. 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.063+0000 7f20a5200700 1 Processor -- start 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.063+0000 7f20a5200700 1 -- start start 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.064+0000 7f20a5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20a01032d0 0x7f20a0110890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:06.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.064+0000 7f20a5200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0107c00 0x7f20a0110dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:06.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.064+0000 7f20a5200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20a0108550 0x7f20a0111310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.064+0000 7f20a5200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f20a01141d0 con 0x7f20a01032d0 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.064+0000 7f20a5200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20a0114050 con 0x7f20a0107c00 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.065+0000 7f209f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20a01032d0 0x7f20a0110890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.065+0000 7f209f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20a01032d0 0x7f20a0110890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34468/0 (socket says 172.21.15.63:34468) 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.065+0000 7f209f400700 1 -- 172.21.15.63:0/1665438377 learned_addr learned my addr 172.21.15.63:0/1665438377 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.065+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f20a0114350 con 0x7f20a0108550 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.065+0000 7f209fe00700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20a0108550 0x7f20a0111310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.065+0000 7f209ea00700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0107c00 0x7f20a0110dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.066+0000 7f209f400700 1 -- 172.21.15.63:0/1665438377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20a0108550 msgr2=0x7f20a0111310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.066+0000 7f209f400700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20a0108550 0x7f20a0111310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.066+0000 7f209f400700 1 -- 172.21.15.63:0/1665438377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0107c00 msgr2=0x7f20a0110dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:06.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.066+0000 7f209f400700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0107c00 0x7f20a0110dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.066+0000 7f209f400700 1 -- 172.21.15.63:0/1665438377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a01a8f40 con 0x7f20a01032d0 2024-05-27T07:35:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.066+0000 7f209fe00700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20a0108550 0x7f20a0111310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.066+0000 7f209ea00700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0107c00 0x7f20a0110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.066+0000 7f209f400700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20a01032d0 0x7f20a0110890 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2090000c00 tx=0x7f209000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:06.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.067+0000 7f2097e00700 1 -- 172.21.15.63:0/1665438377 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090027070 con 0x7f20a01032d0 2024-05-27T07:35:06.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.067+0000 7f2097e00700 1 -- 172.21.15.63:0/1665438377 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f209000fe50 con 0x7f20a01032d0 2024-05-27T07:35:06.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.067+0000 7f2097e00700 1 -- 172.21.15.63:0/1665438377 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20900173f0 con 0x7f20a01032d0 2024-05-27T07:35:06.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.067+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a01a91d0 con 0x7f20a01032d0 2024-05-27T07:35:06.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.067+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a01a95d0 con 0x7f20a01032d0 2024-05-27T07:35:06.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.069+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2080005330 con 0x7f20a01032d0 2024-05-27T07:35:06.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.072+0000 7f2097e00700 1 -- 172.21.15.63:0/1665438377 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f209002a070 con 0x7f20a01032d0 2024-05-27T07:35:06.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.075+0000 7f2097e00700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20840b2c20 0x7f20840b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.075+0000 7f2097e00700 1 -- 172.21.15.63:0/1665438377 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f20900e3a80 con 0x7f20a01032d0 2024-05-27T07:35:06.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.076+0000 7f209ea00700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20840b2c20 0x7f20840b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:06.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.079+0000 7f209ea00700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20840b2c20 0x7f20840b5130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f208c005950 tx=0x7f208c0058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:06.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.084+0000 7f2097e00700 1 -- 172.21.15.63:0/1665438377 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f20900e5e20 con 0x7f20a01032d0 2024-05-27T07:35:06.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.365+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20800058f0 con 0x7f20a01032d0 2024-05-27T07:35:06.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.366+0000 7f2097e00700 1 -- 172.21.15.63:0/1665438377 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11861 (secure 0 0 0) 0x7f2090013070 con 0x7f20a01032d0 2024-05-27T07:35:06.361 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:35:06.361 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:33:33.871810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:35:06.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20840b2c20 msgr2=0x7f20840b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:06.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20840b2c20 0x7f20840b5130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f208c005950 tx=0x7f208c0058e0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20a01032d0 msgr2=0x7f20a0110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:06.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20a01032d0 0x7f20a0110890 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2090000c00 tx=0x7f209000f510 comp rx=0 tx=0).stop 2024-05-27T07:35:06.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 shutdown_connections 2024-05-27T07:35:06.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f20840b2c20 0x7f20840b5130 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20a0108550 0x7f20a0111310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20a01032d0 0x7f20a0110890 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 --2- 172.21.15.63:0/1665438377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20a0107c00 0x7f20a0110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:06.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 >> 172.21.15.63:0/1665438377 conn(0x7f20a0078ab0 msgr2=0x7f20a01008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:06.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.370+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 shutdown_connections 2024-05-27T07:35:06.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:06.371+0000 7f20a5200700 1 -- 172.21.15.63:0/1665438377 wait complete. 2024-05-27T07:35:06.366 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 27 2024-05-27T07:35:06.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:06 smithi086 ceph-mon[34687]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-27T07:35:06.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:06 smithi149 ceph-mon[35408]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-27T07:35:06.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:06 smithi063 ceph-mon[31613]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-27T07:35:07.282 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:35:07.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:07 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1665438377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:35:07.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:07 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1665438377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:35:07.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:07 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1665438377' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:35:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:08.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:08 smithi086 ceph-mon[34687]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-27T07:35:08.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:08 smithi086 ceph-mon[34687]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T07:35:08.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:08 smithi149 ceph-mon[35408]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-27T07:35:08.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:08 smithi149 ceph-mon[35408]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T07:35:08.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:08 smithi063 ceph-mon[31613]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-27T07:35:08.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:08 smithi063 ceph-mon[31613]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T07:35:10.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.206+0000 7f6fd7e00700 1 -- 172.21.15.63:0/4101600245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd8075b50 msgr2=0x7f6fd8075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:10.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.206+0000 7f6fd7e00700 1 --2- 172.21.15.63:0/4101600245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd8075b50 0x7f6fd8075f60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009990 tx=0x7f6fc8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.206+0000 7f6fd7e00700 1 -- 172.21.15.63:0/4101600245 shutdown_connections 2024-05-27T07:35:10.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.206+0000 7f6fd7e00700 1 --2- 172.21.15.63:0/4101600245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd8115170 0x7f6fd8117590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.206+0000 7f6fd7e00700 1 --2- 172.21.15.63:0/4101600245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd8075b50 0x7f6fd8075f60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.206+0000 7f6fd7e00700 1 --2- 172.21.15.63:0/4101600245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd80764a0 0x7f6fd8114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.206+0000 7f6fd7e00700 1 -- 172.21.15.63:0/4101600245 >> 172.21.15.63:0/4101600245 conn(0x7f6fd8079d00 msgr2=0x7f6fd807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.207+0000 7f6fd7e00700 1 -- 172.21.15.63:0/4101600245 shutdown_connections 2024-05-27T07:35:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.207+0000 7f6fd7e00700 1 -- 172.21.15.63:0/4101600245 wait complete. 2024-05-27T07:35:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.209+0000 7f6fd7e00700 1 Processor -- start 2024-05-27T07:35:10.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.209+0000 7f6fd7e00700 1 -- start start 2024-05-27T07:35:10.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.210+0000 7f6fd7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd80764a0 0x7f6fd81a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.210+0000 7f6fd7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd8115170 0x7f6fd81a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.210+0000 7f6fd7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd81a5b00 0x7f6fd81a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.210+0000 7f6fd7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd811a780 con 0x7f6fd8115170 2024-05-27T07:35:10.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.210+0000 7f6fd7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd811a600 con 0x7f6fd80764a0 2024-05-27T07:35:10.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.210+0000 7f6fd7e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd811a900 con 0x7f6fd81a5b00 2024-05-27T07:35:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.210+0000 7f6fd6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd8115170 0x7f6fd81a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.210+0000 7f6fd6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd8115170 0x7f6fd81a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59128/0 (socket says 172.21.15.63:59128) 2024-05-27T07:35:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.210+0000 7f6fd6000700 1 -- 172.21.15.63:0/1322916361 learned_addr learned my addr 172.21.15.63:0/1322916361 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.210+0000 7f6fd7400700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd81a5b00 0x7f6fd81a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.211+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd80764a0 0x7f6fd81a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.211+0000 7f6fd6000700 1 -- 172.21.15.63:0/1322916361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd81a5b00 msgr2=0x7f6fd81a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.211+0000 7f6fd6000700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd81a5b00 0x7f6fd81a5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.211+0000 7f6fd6000700 1 -- 172.21.15.63:0/1322916361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd80764a0 msgr2=0x7f6fd81a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.211+0000 7f6fd6000700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd80764a0 0x7f6fd81a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.211+0000 7f6fd6000700 1 -- 172.21.15.63:0/1322916361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd81a65c0 con 0x7f6fd8115170 2024-05-27T07:35:10.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.212+0000 7f6fd6000700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd8115170 0x7f6fd81a1930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400d950 tx=0x7f6fc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:10.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.212+0000 7f6fcf400700 1 -- 172.21.15.63:0/1322916361 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4016070 con 0x7f6fd8115170 2024-05-27T07:35:10.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.212+0000 7f6fcf400700 1 -- 172.21.15.63:0/1322916361 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fc400fd20 con 0x7f6fd8115170 2024-05-27T07:35:10.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.212+0000 7f6fcf400700 1 -- 172.21.15.63:0/1322916361 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc400e3b0 con 0x7f6fd8115170 2024-05-27T07:35:10.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.213+0000 7f6fd7e00700 1 -- 172.21.15.63:0/1322916361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd81a68b0 con 0x7f6fd8115170 2024-05-27T07:35:10.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.213+0000 7f6fd7e00700 1 -- 172.21.15.63:0/1322916361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd81ae040 con 0x7f6fd8115170 2024-05-27T07:35:10.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.220+0000 7f6fec637b80 1 Processor -- start 2024-05-27T07:35:10.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.220+0000 7f6fec637b80 1 -- start start 2024-05-27T07:35:10.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.220+0000 7f6fcf400700 1 -- 172.21.15.63:0/1322916361 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6fc4019070 con 0x7f6fd8115170 2024-05-27T07:35:10.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.220+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634e00c79f0 0x5634e015b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.220+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634dffdbe40 0x5634e015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.221+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e01652c0 0x5634e01676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.221+0000 7f6fec637b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5634e0088e40 con 0x5634e00c79f0 2024-05-27T07:35:10.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.221+0000 7f6fec637b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634e00789e0 con 0x5634dffdbe40 2024-05-27T07:35:10.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.221+0000 7f6fec637b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5634e007ca20 con 0x5634e01652c0 2024-05-27T07:35:10.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.221+0000 7f6fd7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634e00c79f0 0x5634e015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.221+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634dffdbe40 0x5634e015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.221+0000 7f6fd7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634e00c79f0 0x5634e015b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59140/0 (socket says 172.21.15.63:59140) 2024-05-27T07:35:10.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.221+0000 7f6fd7400700 1 -- 172.21.15.63:0/1875234143 learned_addr learned my addr 172.21.15.63:0/1875234143 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:10.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.222+0000 7f6fd7400700 1 -- 172.21.15.63:0/1875234143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e01652c0 msgr2=0x5634e01676e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:10.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.222+0000 7f6fd7400700 1 --2- 172.21.15.63:0/1875234143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e01652c0 0x5634e01676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.222+0000 7f6fd7400700 1 -- 172.21.15.63:0/1875234143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634dffdbe40 msgr2=0x5634e015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:10.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.222+0000 7f6fd7400700 1 --2- 172.21.15.63:0/1875234143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634dffdbe40 0x5634e015bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.222+0000 7f6fd7400700 1 -- 172.21.15.63:0/1875234143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e0041420 con 0x5634e00c79f0 2024-05-27T07:35:10.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.223+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/1875234143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634dffdbe40 0x5634e015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:10.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.223+0000 7f6fcf400700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6fbc0b2a70 0x7f6fbc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.223+0000 7f6fcf400700 1 -- 172.21.15.63:0/1322916361 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f6fc40e2400 con 0x7f6fd8115170 2024-05-27T07:35:10.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.223+0000 7f6fd7400700 1 --2- 172.21.15.63:0/1875234143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634e00c79f0 0x5634e015b750 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000f410 tx=0x7f6fd000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18157bd7f4c188b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:10.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.224+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6fbc0b2a70 0x7f6fbc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.224+0000 7f6fc3e00700 1 -- 172.21.15.63:0/1875234143 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0018030 con 0x5634e00c79f0 2024-05-27T07:35:10.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.224+0000 7f6fc3e00700 1 -- 172.21.15.63:0/1875234143 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fd000fe50 con 0x5634e00c79f0 2024-05-27T07:35:10.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.224+0000 7f6fc3e00700 1 -- 172.21.15.63:0/1875234143 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0005020 con 0x5634e00c79f0 2024-05-27T07:35:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.226+0000 7f6fec637b80 1 -- 172.21.15.63:0/1875234143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634e00c79f0 msgr2=0x5634e015b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.226+0000 7f6fec637b80 1 --2- 172.21.15.63:0/1875234143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634e00c79f0 0x5634e015b750 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000f410 tx=0x7f6fd000f720 comp rx=0 tx=0).stop 2024-05-27T07:35:10.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.226+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6fbc0b2a70 0x7f6fbc0b4f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80098e0 tx=0x7f6fc800b540 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:10.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.226+0000 7f6fec637b80 1 -- 172.21.15.63:0/1875234143 shutdown_connections 2024-05-27T07:35:10.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.226+0000 7f6fec637b80 1 --2- 172.21.15.63:0/1875234143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e01652c0 0x5634e01676e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.226+0000 7f6fec637b80 1 --2- 172.21.15.63:0/1875234143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634e00c79f0 0x5634e015b750 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.226+0000 7f6fec637b80 1 --2- 172.21.15.63:0/1875234143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634dffdbe40 0x5634e015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.226+0000 7f6fec637b80 1 -- 172.21.15.63:0/1875234143 >> 172.21.15.63:0/1875234143 conn(0x5634e0075390 msgr2=0x5634e0156710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:10.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.226+0000 7f6fec637b80 1 -- 172.21.15.63:0/1875234143 shutdown_connections 2024-05-27T07:35:10.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.226+0000 7f6fec637b80 1 -- 172.21.15.63:0/1875234143 wait complete. 2024-05-27T07:35:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.228+0000 7f6fec637b80 1 -- start start 2024-05-27T07:35:10.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.228+0000 7f6fec637b80 1 Processor -- start 2024-05-27T07:35:10.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.229+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.229+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 0x5634e01627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.229+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e01652c0 0x5634e0162d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.229+0000 7f6fec637b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5634dff7efa0 con 0x5634dffdbe40 2024-05-27T07:35:10.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.229+0000 7f6fec637b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634e0019760 con 0x5634e01652c0 2024-05-27T07:35:10.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.229+0000 7f6fec637b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5634e0167e00 con 0x5634e00c79f0 2024-05-27T07:35:10.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.229+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.229+0000 7f6fec637b80 10 client.0 _ll_register_callbacks cb 0x5634e0022c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:10.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.229+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59148/0 (socket says 172.21.15.63:59148) 2024-05-27T07:35:10.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.229+0000 7f6fd6a00700 1 -- 172.21.15.63:0/3319636165 learned_addr learned my addr 172.21.15.63:0/3319636165 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:10.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.230+0000 7f6fd6000700 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 0x5634e01627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.230+0000 7f6fd7400700 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e01652c0 0x5634e0162d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.230+0000 7f6fd6a00700 1 -- 172.21.15.63:0/3319636165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 msgr2=0x5634e01627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:10.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.230+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 0x5634e01627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.230+0000 7f6fd6a00700 1 -- 172.21.15.63:0/3319636165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e01652c0 msgr2=0x5634e0162d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:10.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.230+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e01652c0 0x5634e0162d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.230+0000 7f6fd6a00700 1 -- 172.21.15.63:0/3319636165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634dffc88d0 con 0x5634dffdbe40 2024-05-27T07:35:10.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.231+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01622a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8015000 tx=0x7f6fc8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d681c42ce8e92887 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:10.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.231+0000 7f6fc0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:10.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.232+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3319636165 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8034060 con 0x5634dffdbe40 2024-05-27T07:35:10.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.232+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3319636165 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f6fc800f670 con 0x5634dffdbe40 2024-05-27T07:35:10.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.232+0000 7f6fec637b80 1 -- 172.21.15.63:0/3319636165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fd000fe50 con 0x5634dffdbe40 2024-05-27T07:35:10.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.232+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3319636165 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fc803b050 con 0x5634dffdbe40 2024-05-27T07:35:10.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.232+0000 7f6fec637b80 10 client.15651 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:10.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.232+0000 7f6fec637b80 1 -- 172.21.15.63:0/3319636165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5634e0049060 con 0x5634dffdbe40 2024-05-27T07:35:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.233+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3319636165 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fd000fe50 con 0x5634dffdbe40 2024-05-27T07:35:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.233+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3319636165 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6fc8024980 con 0x5634dffdbe40 2024-05-27T07:35:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.233+0000 7f6fec637b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.233+0000 7f6fec637b80 10 client.15651 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:35:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.233+0000 7f6fec637b80 20 client.15651 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.233+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5634e0174cb0 0x5634e01770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.233+0000 7f6fec637b80 4 client.15651 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:10.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.233+0000 7f6fec637b80 1 -- 172.21.15.63:0/3319636165 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634e008ad90 con 0x5634e0174cb0 2024-05-27T07:35:10.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.234+0000 7f6fd6000700 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5634e0174cb0 0x5634e01770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.235+0000 7f6fd6000700 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5634e0174cb0 0x5634e01770f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f80628dfd8b6414c server_cookie=90c15ba449cc0b14 in_seq=0 out_seq=0 2024-05-27T07:35:10.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.235+0000 7f6fc0c00700 10 client.15651 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:35:10.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.237+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3319636165 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5634e008ad90 con 0x5634e0174cb0 2024-05-27T07:35:10.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.237+0000 7f6fc0c00700 10 client.15651 handle_command_reply: tid=0 2024-05-27T07:35:10.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.237+0000 7f6fec637b80 1 client.15651 shutdown 2024-05-27T07:35:10.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.238+0000 7f6fec637b80 1 -- 172.21.15.63:0/3319636165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 msgr2=0x5634e01622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:10.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.238+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01622a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8015000 tx=0x7f6fc8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.238+0000 7f6fec637b80 1 -- 172.21.15.63:0/3319636165 shutdown_connections 2024-05-27T07:35:10.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.238+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e01652c0 0x5634e0162d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.238+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5634e0174cb0 0x5634e01770f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.238+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01622a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.238+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3319636165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 0x5634e01627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.238+0000 7f6fec637b80 1 -- 172.21.15.63:0/3319636165 >> 172.21.15.63:0/3319636165 conn(0x5634e0075390 msgr2=0x5634e0156300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:10.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.238+0000 7f6fc0c00700 0 client.15651 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:35:10.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.238+0000 7f6fec637b80 1 -- 172.21.15.63:0/3319636165 shutdown_connections 2024-05-27T07:35:10.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.239+0000 7f6fec637b80 1 -- 172.21.15.63:0/3319636165 wait complete. 2024-05-27T07:35:10.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.239+0000 7f6fec637b80 20 client.15651 trim_cache size 0 max 16384 2024-05-27T07:35:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.255+0000 7f6fec637b80 1 Processor -- start 2024-05-27T07:35:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.255+0000 7f6fec637b80 1 -- start start 2024-05-27T07:35:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.256+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e0170f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.256+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 0x5634e015b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.256+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e01652c0 0x5634e015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.256+0000 7f6fec637b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd81b9790 con 0x5634dffdbe40 2024-05-27T07:35:10.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.256+0000 7f6fec637b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd81b7020 con 0x5634e01652c0 2024-05-27T07:35:10.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.256+0000 7f6fec637b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd81b4e70 con 0x5634e00c79f0 2024-05-27T07:35:10.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.256+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e0170f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.256+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e0170f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59154/0 (socket says 172.21.15.63:59154) 2024-05-27T07:35:10.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.256+0000 7f6fd6a00700 1 -- 172.21.15.63:0/1007290659 learned_addr learned my addr 172.21.15.63:0/1007290659 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:10.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.257+0000 7f6fd7400700 1 --2- 172.21.15.63:0/1007290659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e01652c0 0x5634e015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.257+0000 7f6fd6000700 1 --2- 172.21.15.63:0/1007290659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 0x5634e015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.257+0000 7f6fd6a00700 1 -- 172.21.15.63:0/1007290659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 msgr2=0x5634e015b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:10.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.257+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/1007290659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 0x5634e015b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.257+0000 7f6fd6a00700 1 -- 172.21.15.63:0/1007290659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e01652c0 msgr2=0x5634e015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:10.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.257+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/1007290659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e01652c0 0x5634e015bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.257+0000 7f6fd6a00700 1 -- 172.21.15.63:0/1007290659 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e015c1d0 con 0x5634dffdbe40 2024-05-27T07:35:10.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.257+0000 7f6fd6000700 1 --2- 172.21.15.63:0/1007290659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 0x5634e015b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:10.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.257+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/1007290659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e0170f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000c00 tx=0x7f6fc80093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff70561452bf96 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:10.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.258+0000 7f6fc3400700 1 -- 172.21.15.63:0/1007290659 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8034040 con 0x5634dffdbe40 2024-05-27T07:35:10.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.258+0000 7f6fc3400700 1 -- 172.21.15.63:0/1007290659 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fc8017ca0 con 0x5634dffdbe40 2024-05-27T07:35:10.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.258+0000 7f6fc3400700 1 -- 172.21.15.63:0/1007290659 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800f670 con 0x5634dffdbe40 2024-05-27T07:35:10.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.259+0000 7f6fec637b80 1 -- 172.21.15.63:0/1007290659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 msgr2=0x5634e0170f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:10.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.259+0000 7f6fec637b80 1 --2- 172.21.15.63:0/1007290659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e0170f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000c00 tx=0x7f6fc80093b0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.259+0000 7f6fec637b80 1 -- 172.21.15.63:0/1007290659 shutdown_connections 2024-05-27T07:35:10.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.259+0000 7f6fec637b80 1 --2- 172.21.15.63:0/1007290659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e01652c0 0x5634e015bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.259+0000 7f6fec637b80 1 --2- 172.21.15.63:0/1007290659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e0170f00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.259+0000 7f6fec637b80 1 --2- 172.21.15.63:0/1007290659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e00c79f0 0x5634e015b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.259+0000 7f6fec637b80 1 -- 172.21.15.63:0/1007290659 >> 172.21.15.63:0/1007290659 conn(0x5634e0075390 msgr2=0x5634e0160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:10.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.259+0000 7f6fec637b80 1 -- 172.21.15.63:0/1007290659 shutdown_connections 2024-05-27T07:35:10.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.259+0000 7f6fec637b80 1 -- 172.21.15.63:0/1007290659 wait complete. 2024-05-27T07:35:10.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.260+0000 7f6fec637b80 1 -- start start 2024-05-27T07:35:10.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.260+0000 7f6fec637b80 1 Processor -- start 2024-05-27T07:35:10.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.261+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.261+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e00c79f0 0x5634e01685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.261+0000 7f6fec637b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e01652c0 0x5634e0168b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.261+0000 7f6fec637b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd81b54e0 con 0x5634dffdbe40 2024-05-27T07:35:10.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.261+0000 7f6fec637b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd81b6950 con 0x5634e00c79f0 2024-05-27T07:35:10.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.261+0000 7f6fec637b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd81b5bb0 con 0x5634e01652c0 2024-05-27T07:35:10.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.261+0000 7f6fec637b80 10 client.0 _ll_register_callbacks cb 0x5634e0022c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:10.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.262+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.262+0000 7f6fd7400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e01652c0 0x5634e0168b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.262+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59166/0 (socket says 172.21.15.63:59166) 2024-05-27T07:35:10.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.262+0000 7f6fd6a00700 1 -- 172.21.15.63:0/3015304153 learned_addr learned my addr 172.21.15.63:0/3015304153 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:10.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.262+0000 7f6fd6000700 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e00c79f0 0x5634e01685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.262+0000 7f6fd6a00700 1 -- 172.21.15.63:0/3015304153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e01652c0 msgr2=0x5634e0168b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:10.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.262+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e01652c0 0x5634e0168b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.262+0000 7f6fd6a00700 1 -- 172.21.15.63:0/3015304153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e00c79f0 msgr2=0x5634e01685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:10.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.262+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e00c79f0 0x5634e01685f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.262+0000 7f6fd6a00700 1 -- 172.21.15.63:0/3015304153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e01dd970 con 0x5634dffdbe40 2024-05-27T07:35:10.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.263+0000 7f6fd6a00700 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01680b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009990 tx=0x7f6fc8017860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e8763a89b4e431 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:10.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.263+0000 7f6fc0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:10.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.263+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3015304153 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8046070 con 0x5634dffdbe40 2024-05-27T07:35:10.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.263+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3015304153 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f6fc8024750 con 0x5634dffdbe40 2024-05-27T07:35:10.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.263+0000 7f6fec637b80 1 -- 172.21.15.63:0/3015304153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fc8017ca0 con 0x5634dffdbe40 2024-05-27T07:35:10.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.264+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3015304153 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fc8034040 con 0x5634dffdbe40 2024-05-27T07:35:10.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.264+0000 7f6fec637b80 10 client.15657 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:10.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.264+0000 7f6fec637b80 1 -- 172.21.15.63:0/3015304153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5634e01deb20 con 0x5634dffdbe40 2024-05-27T07:35:10.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.264+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3015304153 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fc8017ca0 con 0x5634dffdbe40 2024-05-27T07:35:10.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.265+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3015304153 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6fc802f470 con 0x5634dffdbe40 2024-05-27T07:35:10.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.265+0000 7f6fec637b80 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:10.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.265+0000 7f6fec637b80 10 client.15657 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:35:10.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.265+0000 7f6fec637b80 20 client.15657 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:10.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.265+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5634e01df000 0x5634e01e1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:10.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.266+0000 7f6fec637b80 4 client.15657 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:35:10.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.266+0000 7f6fec637b80 1 -- 172.21.15.63:0/3015304153 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634e0084570 con 0x5634e01df000 2024-05-27T07:35:10.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.266+0000 7f6fd6000700 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5634e01df000 0x5634e01e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:10.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.267+0000 7f6fd6000700 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5634e01df000 0x5634e01e1420 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2829e2f10b3d9929 server_cookie=ff44e10a0915449 in_seq=0 out_seq=0 2024-05-27T07:35:10.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.267+0000 7f6fc0c00700 10 client.15657 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:35:10.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fc0c00700 1 -- 172.21.15.63:0/3015304153 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634e0084570 con 0x5634e01df000 2024-05-27T07:35:10.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fc0c00700 10 client.15657 handle_command_reply: tid=0 2024-05-27T07:35:10.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fec637b80 1 client.15657 shutdown 2024-05-27T07:35:10.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fec637b80 1 -- 172.21.15.63:0/3015304153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 msgr2=0x5634e01680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:10.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01680b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009990 tx=0x7f6fc8017860 comp rx=0 tx=0).stop 2024-05-27T07:35:10.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fec637b80 1 -- 172.21.15.63:0/3015304153 shutdown_connections 2024-05-27T07:35:10.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634e01652c0 0x5634e0168b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5634e01df000 0x5634e01e1420 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634dffdbe40 0x5634e01680b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fec637b80 1 --2- 172.21.15.63:0/3015304153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634e00c79f0 0x5634e01685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.268+0000 7f6fec637b80 1 -- 172.21.15.63:0/3015304153 >> 172.21.15.63:0/3015304153 conn(0x5634e0075390 msgr2=0x5634e015d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:10.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.269+0000 7f6fc0c00700 0 client.15657 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:35:10.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.269+0000 7f6fec637b80 1 -- 172.21.15.63:0/3015304153 shutdown_connections 2024-05-27T07:35:10.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.269+0000 7f6fec637b80 1 -- 172.21.15.63:0/3015304153 wait complete. 2024-05-27T07:35:10.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.269+0000 7f6fec637b80 20 client.15657 trim_cache size 0 max 16384 2024-05-27T07:35:10.265 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:35:10.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.270+0000 7f6fd7e00700 1 -- 172.21.15.63:0/1322916361 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6fbc0b2a70 msgr2=0x7f6fbc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.270+0000 7f6fd7e00700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6fbc0b2a70 0x7f6fbc0b4f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6fc80098e0 tx=0x7f6fc800b540 comp rx=0 tx=0).stop 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.270+0000 7f6fd7e00700 1 -- 172.21.15.63:0/1322916361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd8115170 msgr2=0x7f6fd81a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.270+0000 7f6fd7e00700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd8115170 0x7f6fd81a1930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400d950 tx=0x7f6fc400dc60 comp rx=0 tx=0).stop 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.271+0000 7f6fd7e00700 1 -- 172.21.15.63:0/1322916361 shutdown_connections 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.271+0000 7f6fd7e00700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6fbc0b2a70 0x7f6fbc0b4f80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.271+0000 7f6fd7e00700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd81a5b00 0x7f6fd81a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.271+0000 7f6fd7e00700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fd80764a0 0x7f6fd81a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.271+0000 7f6fd7e00700 1 --2- 172.21.15.63:0/1322916361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fd8115170 0x7f6fd81a1930 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.271+0000 7f6fd7e00700 1 -- 172.21.15.63:0/1322916361 >> 172.21.15.63:0/1322916361 conn(0x7f6fd8079d00 msgr2=0x7f6fd8101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.271+0000 7f6fd7e00700 1 -- 172.21.15.63:0/1322916361 shutdown_connections 2024-05-27T07:35:10.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:10.271+0000 7f6fd7e00700 1 -- 172.21.15.63:0/1322916361 wait complete. 2024-05-27T07:35:10.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:10 smithi063 ceph-mon[31613]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-27T07:35:10.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:10 smithi086 ceph-mon[34687]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-27T07:35:10.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:10 smithi149 ceph-mon[35408]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-27T07:35:11.202 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:35:11.674 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:12.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:12 smithi063 ceph-mon[31613]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-27T07:35:12.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:12 smithi086 ceph-mon[34687]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-27T07:35:12.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:12 smithi149 ceph-mon[35408]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-27T07:35:13.596 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:13 smithi063 ceph-mon[31613]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-27T07:35:13.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:13 smithi086 ceph-mon[34687]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-27T07:35:13.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:13 smithi149 ceph-mon[35408]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-27T07:35:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.391+0000 7f742cc00700 1 -- 172.21.15.63:0/4020177188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428103a20 msgr2=0x7f7428110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.391+0000 7f742cc00700 1 --2- 172.21.15.63:0/4020177188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428103a20 0x7f7428110740 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7414009990 tx=0x7f7414009ca0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.392+0000 7f742cc00700 1 -- 172.21.15.63:0/4020177188 shutdown_connections 2024-05-27T07:35:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.392+0000 7f742cc00700 1 --2- 172.21.15.63:0/4020177188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428110c80 0x7f74281130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.392+0000 7f742cc00700 1 --2- 172.21.15.63:0/4020177188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74281030d0 0x7f74281034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.392+0000 7f742cc00700 1 --2- 172.21.15.63:0/4020177188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428103a20 0x7f7428110740 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.392+0000 7f742cc00700 1 -- 172.21.15.63:0/4020177188 >> 172.21.15.63:0/4020177188 conn(0x7f74280fedb0 msgr2=0x7f74281011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:14.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.393+0000 7f742cc00700 1 -- 172.21.15.63:0/4020177188 shutdown_connections 2024-05-27T07:35:14.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.394+0000 7f742cc00700 1 -- 172.21.15.63:0/4020177188 wait complete. 2024-05-27T07:35:14.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.395+0000 7f742cc00700 1 Processor -- start 2024-05-27T07:35:14.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.396+0000 7f742cc00700 1 -- start start 2024-05-27T07:35:14.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.396+0000 7f742cc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281030d0 0x7f742819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.396+0000 7f742cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428103a20 0x7f742819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.397+0000 7f742cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428110c80 0x7f74281a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.397+0000 7f742cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7428116540 con 0x7f7428110c80 2024-05-27T07:35:14.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.397+0000 7f742cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74281163c0 con 0x7f7428103a20 2024-05-27T07:35:14.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.397+0000 7f742cc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74281166c0 con 0x7f74281030d0 2024-05-27T07:35:14.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.397+0000 7f7425400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428103a20 0x7f742819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.397+0000 7f7425400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428103a20 0x7f742819d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:43244/0 (socket says 172.21.15.63:43244) 2024-05-27T07:35:14.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.397+0000 7f7425400700 1 -- 172.21.15.63:0/280515949 learned_addr learned my addr 172.21.15.63:0/280515949 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:14.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.398+0000 7f7425e00700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281030d0 0x7f742819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.398+0000 7f7426800700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428110c80 0x7f74281a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.398+0000 7f7425400700 1 -- 172.21.15.63:0/280515949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281030d0 msgr2=0x7f742819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:14.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.398+0000 7f7425400700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281030d0 0x7f742819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.399+0000 7f7425400700 1 -- 172.21.15.63:0/280515949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428110c80 msgr2=0x7f74281a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:14.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.399+0000 7f7425400700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428110c80 0x7f74281a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.399+0000 7f7425400700 1 -- 172.21.15.63:0/280515949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74281a1e40 con 0x7f7428103a20 2024-05-27T07:35:14.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.400+0000 7f7425400700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428103a20 0x7f742819d510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74140047b0 tx=0x7f741400bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:14.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.401+0000 7f741ea00700 1 -- 172.21.15.63:0/280515949 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414026070 con 0x7f7428103a20 2024-05-27T07:35:14.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.401+0000 7f742cc00700 1 -- 172.21.15.63:0/280515949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74281a20d0 con 0x7f7428103a20 2024-05-27T07:35:14.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.401+0000 7f742cc00700 1 -- 172.21.15.63:0/280515949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74281a9a50 con 0x7f7428103a20 2024-05-27T07:35:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.405+0000 7f741ea00700 1 -- 172.21.15.63:0/280515949 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7414021410 con 0x7f7428103a20 2024-05-27T07:35:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.405+0000 7f741ea00700 1 -- 172.21.15.63:0/280515949 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414017650 con 0x7f7428103a20 2024-05-27T07:35:14.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.408+0000 7f743b83ab80 1 Processor -- start 2024-05-27T07:35:14.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.409+0000 7f741ea00700 1 -- 172.21.15.63:0/280515949 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7414009d10 con 0x7f7428103a20 2024-05-27T07:35:14.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.409+0000 7f743b83ab80 1 -- start start 2024-05-27T07:35:14.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.411+0000 7f741ea00700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b2a70 0x7f740c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.412+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f187d9f0 0x5614f1910330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.412+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f1910870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.412+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1910e70 0x5614f191d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.412+0000 7f743b83ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5614f183ee40 con 0x5614f1791e40 2024-05-27T07:35:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.412+0000 7f743b83ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5614f182e9e0 con 0x5614f1910e70 2024-05-27T07:35:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.412+0000 7f743b83ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5614f1832a20 con 0x5614f187d9f0 2024-05-27T07:35:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.412+0000 7f741ea00700 1 -- 172.21.15.63:0/280515949 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f74140e3430 con 0x7f7428103a20 2024-05-27T07:35:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.412+0000 7f7425e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f187d9f0 0x5614f1910330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.413+0000 7f7425400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f1910870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.413+0000 7f7425400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f1910870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59190/0 (socket says 172.21.15.63:59190) 2024-05-27T07:35:14.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.413+0000 7f7425400700 1 -- 172.21.15.63:0/2968917208 learned_addr learned my addr 172.21.15.63:0/2968917208 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:14.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.413+0000 7f7426800700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b2a70 0x7f740c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.414+0000 7f7425e00700 1 -- 172.21.15.63:0/2968917208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1910e70 msgr2=0x5614f191d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:14.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.414+0000 7f7425e00700 1 --2- 172.21.15.63:0/2968917208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1910e70 0x5614f191d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.414+0000 7f7425e00700 1 -- 172.21.15.63:0/2968917208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 msgr2=0x5614f1910870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:14.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.414+0000 7f7425e00700 1 --2- 172.21.15.63:0/2968917208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f1910870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.414+0000 7f7425e00700 1 -- 172.21.15.63:0/2968917208 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f17f7420 con 0x5614f187d9f0 2024-05-27T07:35:14.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.414+0000 7f7425400700 1 --2- 172.21.15.63:0/2968917208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f1910870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:35:14.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.415+0000 7f7425e00700 1 --2- 172.21.15.63:0/2968917208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f187d9f0 0x5614f1910330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f741800bc80 tx=0x7f741800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e88c2f80f58cc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:14.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.415+0000 7f7426800700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b2a70 0x7f740c0b4f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7420009420 tx=0x7f7420009920 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:14.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.416+0000 7f7413400700 1 -- 172.21.15.63:0/2968917208 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418010ca0 con 0x5614f187d9f0 2024-05-27T07:35:14.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.416+0000 7f7413400700 1 -- 172.21.15.63:0/2968917208 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7418010e40 con 0x5614f187d9f0 2024-05-27T07:35:14.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.416+0000 7f7413400700 1 -- 172.21.15.63:0/2968917208 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418005280 con 0x5614f187d9f0 2024-05-27T07:35:14.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.417+0000 7f743b83ab80 1 -- 172.21.15.63:0/2968917208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f187d9f0 msgr2=0x5614f1910330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:14.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.417+0000 7f743b83ab80 1 --2- 172.21.15.63:0/2968917208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f187d9f0 0x5614f1910330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f741800bc80 tx=0x7f741800bf90 comp rx=0 tx=0).stop 2024-05-27T07:35:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.418+0000 7f743b83ab80 1 -- 172.21.15.63:0/2968917208 shutdown_connections 2024-05-27T07:35:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.418+0000 7f743b83ab80 1 --2- 172.21.15.63:0/2968917208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1910e70 0x5614f191d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.418+0000 7f743b83ab80 1 --2- 172.21.15.63:0/2968917208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f187d9f0 0x5614f1910330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.418+0000 7f743b83ab80 1 --2- 172.21.15.63:0/2968917208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f1910870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.418+0000 7f743b83ab80 1 -- 172.21.15.63:0/2968917208 >> 172.21.15.63:0/2968917208 conn(0x5614f182b390 msgr2=0x5614f190c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.418+0000 7f743b83ab80 1 -- 172.21.15.63:0/2968917208 shutdown_connections 2024-05-27T07:35:14.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.418+0000 7f743b83ab80 1 -- 172.21.15.63:0/2968917208 wait complete. 2024-05-27T07:35:14.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.419+0000 7f743b83ab80 1 -- start start 2024-05-27T07:35:14.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.420+0000 7f743b83ab80 1 Processor -- start 2024-05-27T07:35:14.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.420+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1791e40 0x5614f191a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.421+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 0x5614f191ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.421+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f191b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.421+0000 7f743b83ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5614f1734fa0 con 0x5614f187d9f0 2024-05-27T07:35:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.421+0000 7f743b83ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5614f17cf760 con 0x5614f1791e40 2024-05-27T07:35:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.421+0000 7f743b83ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5614f191db80 con 0x5614f1910e70 2024-05-27T07:35:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.421+0000 7f743b83ab80 10 client.0 _ll_register_callbacks cb 0x5614f17d8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:14.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.421+0000 7f7426800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f191b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.421+0000 7f7426800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f191b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:44104/0 (socket says 172.21.15.63:44104) 2024-05-27T07:35:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.421+0000 7f7426800700 1 -- 172.21.15.63:0/1071158814 learned_addr learned my addr 172.21.15.63:0/1071158814 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.422+0000 7f7425e00700 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1791e40 0x5614f191a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.422+0000 7f7425400700 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 0x5614f191ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.423+0000 7f7426800700 1 -- 172.21.15.63:0/1071158814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1791e40 msgr2=0x5614f191a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.423+0000 7f7426800700 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1791e40 0x5614f191a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.423+0000 7f7426800700 1 -- 172.21.15.63:0/1071158814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 msgr2=0x5614f191ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.423+0000 7f7426800700 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 0x5614f191ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.423+0000 7f7426800700 1 -- 172.21.15.63:0/1071158814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f177e8d0 con 0x5614f1910e70 2024-05-27T07:35:14.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.423+0000 7f7426800700 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f191b0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7420017c20 tx=0x7f7420004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3c99e9d6489177 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.423+0000 7f7407e00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:35:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.424+0000 7f7407e00700 1 -- 172.21.15.63:0/1071158814 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420004db0 con 0x5614f1910e70 2024-05-27T07:35:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.424+0000 7f743b83ab80 1 -- 172.21.15.63:0/1071158814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7418010e40 con 0x5614f1910e70 2024-05-27T07:35:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.424+0000 7f7407e00700 1 -- 172.21.15.63:0/1071158814 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f7420018820 con 0x5614f1910e70 2024-05-27T07:35:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.425+0000 7f7407e00700 1 -- 172.21.15.63:0/1071158814 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f742002e050 con 0x5614f1910e70 2024-05-27T07:35:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.425+0000 7f743b83ab80 10 client.25324 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:14.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.425+0000 7f743b83ab80 1 -- 172.21.15.63:0/1071158814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614f17ff060 con 0x5614f1910e70 2024-05-27T07:35:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.425+0000 7f7407e00700 1 -- 172.21.15.63:0/1071158814 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7418010e40 con 0x5614f1910e70 2024-05-27T07:35:14.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.426+0000 7f7407e00700 1 -- 172.21.15.63:0/1071158814 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7420023a50 con 0x5614f1910e70 2024-05-27T07:35:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.426+0000 7f743b83ab80 10 client.25324 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.426+0000 7f743b83ab80 10 client.25324 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:35:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.426+0000 7f743b83ab80 20 client.25324 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.426+0000 7f743b83ab80 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5614f1921ae0 0x5614f1923f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.426+0000 7f743b83ab80 4 client.25324 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:14.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.426+0000 7f743b83ab80 1 -- 172.21.15.63:0/1071158814 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614f1840d90 con 0x5614f1921ae0 2024-05-27T07:35:14.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.427+0000 7f7425e00700 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5614f1921ae0 0x5614f1923f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.428+0000 7f7425e00700 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5614f1921ae0 0x5614f1923f20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f78f7926fa7bca58 server_cookie=f1214b0bbd0a9ef2 in_seq=0 out_seq=0 2024-05-27T07:35:14.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.428+0000 7f7407e00700 10 client.25324 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:35:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.430+0000 7f7407e00700 1 -- 172.21.15.63:0/1071158814 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5614f1840d90 con 0x5614f1921ae0 2024-05-27T07:35:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f7407e00700 10 client.25324 handle_command_reply: tid=0 2024-05-27T07:35:14.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f743b83ab80 1 client.25324 shutdown 2024-05-27T07:35:14.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f743b83ab80 1 -- 172.21.15.63:0/1071158814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 msgr2=0x5614f191b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:14.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f743b83ab80 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f191b0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7420017c20 tx=0x7f7420004ad0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f743b83ab80 1 -- 172.21.15.63:0/1071158814 shutdown_connections 2024-05-27T07:35:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f743b83ab80 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f191b0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f743b83ab80 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1791e40 0x5614f191a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f743b83ab80 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5614f1921ae0 0x5614f1923f20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f743b83ab80 1 --2- 172.21.15.63:0/1071158814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 0x5614f191ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f7407e00700 0 client.25324 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:35:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.431+0000 7f743b83ab80 1 -- 172.21.15.63:0/1071158814 >> 172.21.15.63:0/1071158814 conn(0x5614f182b390 msgr2=0x5614f1910310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.432+0000 7f743b83ab80 1 -- 172.21.15.63:0/1071158814 shutdown_connections 2024-05-27T07:35:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.432+0000 7f743b83ab80 1 -- 172.21.15.63:0/1071158814 wait complete. 2024-05-27T07:35:14.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.432+0000 7f743b83ab80 20 client.25324 trim_cache size 0 max 16384 2024-05-27T07:35:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.464+0000 7f743b83ab80 1 Processor -- start 2024-05-27T07:35:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.465+0000 7f743b83ab80 1 -- start start 2024-05-27T07:35:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.465+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f190c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.465+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f187d9f0 0x5614f1915e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.465+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f1914030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.465+0000 7f743b83ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74080114f0 con 0x5614f1791e40 2024-05-27T07:35:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.465+0000 7f743b83ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f740800ed80 con 0x5614f187d9f0 2024-05-27T07:35:14.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.465+0000 7f743b83ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f740800cbd0 con 0x5614f1910e70 2024-05-27T07:35:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.466+0000 7f7425e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f190c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.466+0000 7f7425e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f190c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59198/0 (socket says 172.21.15.63:59198) 2024-05-27T07:35:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.466+0000 7f7425e00700 1 -- 172.21.15.63:0/2712935481 learned_addr learned my addr 172.21.15.63:0/2712935481 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.466+0000 7f7426800700 1 --2- 172.21.15.63:0/2712935481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f1914030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.467+0000 7f7425e00700 1 -- 172.21.15.63:0/2712935481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 msgr2=0x5614f1914030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.467+0000 7f7425e00700 1 --2- 172.21.15.63:0/2712935481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f1914030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.467+0000 7f7425e00700 1 -- 172.21.15.63:0/2712935481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f187d9f0 msgr2=0x5614f1915e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:14.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.467+0000 7f7425e00700 1 --2- 172.21.15.63:0/2712935481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f187d9f0 0x5614f1915e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.467+0000 7f7425e00700 1 -- 172.21.15.63:0/2712935481 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f191f560 con 0x5614f1791e40 2024-05-27T07:35:14.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.467+0000 7f7425e00700 1 --2- 172.21.15.63:0/2712935481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f190c660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x5614f1912ff0 tx=0x7f7418005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40863420d378fbad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:14.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.468+0000 7f7412a00700 1 -- 172.21.15.63:0/2712935481 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418004240 con 0x5614f1791e40 2024-05-27T07:35:14.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.468+0000 7f7412a00700 1 -- 172.21.15.63:0/2712935481 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74180043e0 con 0x5614f1791e40 2024-05-27T07:35:14.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.468+0000 7f7412a00700 1 -- 172.21.15.63:0/2712935481 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74180116a0 con 0x5614f1791e40 2024-05-27T07:35:14.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.469+0000 7f743b83ab80 1 -- 172.21.15.63:0/2712935481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 msgr2=0x5614f190c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:14.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.469+0000 7f743b83ab80 1 --2- 172.21.15.63:0/2712935481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f190c660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x5614f1912ff0 tx=0x7f7418005970 comp rx=0 tx=0).stop 2024-05-27T07:35:14.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.469+0000 7f743b83ab80 1 -- 172.21.15.63:0/2712935481 shutdown_connections 2024-05-27T07:35:14.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.469+0000 7f743b83ab80 1 --2- 172.21.15.63:0/2712935481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f1914030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.469+0000 7f743b83ab80 1 --2- 172.21.15.63:0/2712935481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f1791e40 0x5614f190c660 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.469+0000 7f743b83ab80 1 --2- 172.21.15.63:0/2712935481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f187d9f0 0x5614f1915e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.469+0000 7f743b83ab80 1 -- 172.21.15.63:0/2712935481 >> 172.21.15.63:0/2712935481 conn(0x5614f182b390 msgr2=0x5614f190bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:14.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.469+0000 7f743b83ab80 1 -- 172.21.15.63:0/2712935481 shutdown_connections 2024-05-27T07:35:14.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.469+0000 7f743b83ab80 1 -- 172.21.15.63:0/2712935481 wait complete. 2024-05-27T07:35:14.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.470+0000 7f743b83ab80 1 -- start start 2024-05-27T07:35:14.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.470+0000 7f743b83ab80 1 Processor -- start 2024-05-27T07:35:14.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.471+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 0x5614f1915370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.471+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f19158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.471+0000 7f743b83ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1915df0 0x5614f1927b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.471+0000 7f743b83ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f740800d240 con 0x5614f187d9f0 2024-05-27T07:35:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.471+0000 7f743b83ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f740800e6b0 con 0x5614f1915df0 2024-05-27T07:35:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.471+0000 7f743b83ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f740800d910 con 0x5614f1910e70 2024-05-27T07:35:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.471+0000 7f743b83ab80 10 client.0 _ll_register_callbacks cb 0x5614f17d8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.472+0000 7f7425e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 0x5614f1915370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.472+0000 7f7425e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 0x5614f1915370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59214/0 (socket says 172.21.15.63:59214) 2024-05-27T07:35:14.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.472+0000 7f7425e00700 1 -- 172.21.15.63:0/4088130047 learned_addr learned my addr 172.21.15.63:0/4088130047 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.472+0000 7f7425400700 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f19158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.472+0000 7f7426800700 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1915df0 0x5614f1927b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.473+0000 7f7425e00700 1 -- 172.21.15.63:0/4088130047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 msgr2=0x5614f19158b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.473+0000 7f7425e00700 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f19158b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.473+0000 7f7425e00700 1 -- 172.21.15.63:0/4088130047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1915df0 msgr2=0x5614f1927b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.473+0000 7f7425e00700 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1915df0 0x5614f1927b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.473+0000 7f7425e00700 1 -- 172.21.15.63:0/4088130047 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f191b430 con 0x5614f187d9f0 2024-05-27T07:35:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.473+0000 7f7425e00700 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 0x5614f1915370 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f741800bc80 tx=0x7f7418010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d17aaf5a553bd04 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:14.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.473+0000 7f7407e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.474+0000 7f7407e00700 1 -- 172.21.15.63:0/4088130047 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418039020 con 0x5614f187d9f0 2024-05-27T07:35:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.474+0000 7f7407e00700 1 -- 172.21.15.63:0/4088130047 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f7418003680 con 0x5614f187d9f0 2024-05-27T07:35:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.474+0000 7f743b83ab80 1 -- 172.21.15.63:0/4088130047 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74180043e0 con 0x5614f187d9f0 2024-05-27T07:35:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.474+0000 7f7407e00700 1 -- 172.21.15.63:0/4088130047 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7418004240 con 0x5614f187d9f0 2024-05-27T07:35:14.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.474+0000 7f743b83ab80 10 client.15675 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.474+0000 7f743b83ab80 1 -- 172.21.15.63:0/4088130047 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614f19a6950 con 0x5614f187d9f0 2024-05-27T07:35:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.475+0000 7f7407e00700 1 -- 172.21.15.63:0/4088130047 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74180043e0 con 0x5614f187d9f0 2024-05-27T07:35:14.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.475+0000 7f7407e00700 1 -- 172.21.15.63:0/4088130047 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f74180050f0 con 0x5614f187d9f0 2024-05-27T07:35:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.475+0000 7f743b83ab80 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.475+0000 7f743b83ab80 10 client.15675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:35:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.475+0000 7f743b83ab80 20 client.15675 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.475+0000 7f743b83ab80 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5614f19a6de0 0x5614f19a9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.476+0000 7f743b83ab80 4 client.15675 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:35:14.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.476+0000 7f743b83ab80 1 -- 172.21.15.63:0/4088130047 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614f183a570 con 0x5614f19a6de0 2024-05-27T07:35:14.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.477+0000 7f7425400700 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5614f19a6de0 0x5614f19a9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:14.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.478+0000 7f7425400700 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5614f19a6de0 0x5614f19a9200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f7b950216fb253c server_cookie=335ddf59626e145c in_seq=0 out_seq=0 2024-05-27T07:35:14.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.478+0000 7f7407e00700 10 client.15675 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:35:14.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.479+0000 7f7407e00700 1 -- 172.21.15.63:0/4088130047 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614f183a570 con 0x5614f19a6de0 2024-05-27T07:35:14.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.479+0000 7f7407e00700 10 client.15675 handle_command_reply: tid=0 2024-05-27T07:35:14.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.479+0000 7f743b83ab80 1 client.15675 shutdown 2024-05-27T07:35:14.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.479+0000 7f743b83ab80 1 -- 172.21.15.63:0/4088130047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 msgr2=0x5614f1915370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:14.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.479+0000 7f743b83ab80 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 0x5614f1915370 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f741800bc80 tx=0x7f7418010c20 comp rx=0 tx=0).stop 2024-05-27T07:35:14.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.480+0000 7f743b83ab80 1 -- 172.21.15.63:0/4088130047 shutdown_connections 2024-05-27T07:35:14.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.480+0000 7f743b83ab80 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f1915df0 0x5614f1927b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.480+0000 7f743b83ab80 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5614f19a6de0 0x5614f19a9200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.480+0000 7f743b83ab80 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614f187d9f0 0x5614f1915370 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.480+0000 7f743b83ab80 1 --2- 172.21.15.63:0/4088130047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614f1910e70 0x5614f19158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.480+0000 7f7407e00700 0 client.15675 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:35:14.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.480+0000 7f743b83ab80 1 -- 172.21.15.63:0/4088130047 >> 172.21.15.63:0/4088130047 conn(0x5614f182b390 msgr2=0x5614f1911ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:14.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.480+0000 7f743b83ab80 1 -- 172.21.15.63:0/4088130047 shutdown_connections 2024-05-27T07:35:14.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.480+0000 7f743b83ab80 1 -- 172.21.15.63:0/4088130047 wait complete. 2024-05-27T07:35:14.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.481+0000 7f743b83ab80 20 client.15675 trim_cache size 0 max 16384 2024-05-27T07:35:14.476 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:35:14.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.482+0000 7f742cc00700 1 -- 172.21.15.63:0/280515949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b2a70 msgr2=0x7f740c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:14.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.482+0000 7f742cc00700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b2a70 0x7f740c0b4f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7420009420 tx=0x7f7420009920 comp rx=0 tx=0).stop 2024-05-27T07:35:14.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.482+0000 7f742cc00700 1 -- 172.21.15.63:0/280515949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428103a20 msgr2=0x7f742819d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:14.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.482+0000 7f742cc00700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428103a20 0x7f742819d510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74140047b0 tx=0x7f741400bfa0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.482+0000 7f742cc00700 1 -- 172.21.15.63:0/280515949 shutdown_connections 2024-05-27T07:35:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.482+0000 7f742cc00700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b2a70 0x7f740c0b4f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.482+0000 7f742cc00700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428110c80 0x7f74281a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.482+0000 7f742cc00700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281030d0 0x7f742819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.482+0000 7f742cc00700 1 --2- 172.21.15.63:0/280515949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428103a20 0x7f742819d510 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.482+0000 7f742cc00700 1 -- 172.21.15.63:0/280515949 >> 172.21.15.63:0/280515949 conn(0x7f74280fedb0 msgr2=0x7f7428101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.483+0000 7f742cc00700 1 -- 172.21.15.63:0/280515949 shutdown_connections 2024-05-27T07:35:14.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:14.483+0000 7f742cc00700 1 -- 172.21.15.63:0/280515949 wait complete. 2024-05-27T07:35:15.490 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:35:15.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:15 smithi086 ceph-mon[34687]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-27T07:35:15.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:15 smithi149 ceph-mon[35408]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-27T07:35:15.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:15 smithi063 ceph-mon[31613]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-27T07:35:16.118 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:17.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:17 smithi086 ceph-mon[34687]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-27T07:35:17.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:17 smithi149 ceph-mon[35408]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-27T07:35:17.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:17 smithi063 ceph-mon[31613]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-27T07:35:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.108+0000 7fb907400640 1 -- 172.21.15.63:0/3767703518 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4036780 con 0x7fb90810a130 2024-05-27T07:35:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.109+0000 7fb90f400640 1 -- 172.21.15.63:0/3767703518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90810a130 msgr2=0x7fb908103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.109+0000 7fb90f400640 1 --2- 172.21.15.63:0/3767703518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90810a130 0x7fb908103d70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009980 tx=0x7fb8f402e820 comp rx=0 tx=0).stop 2024-05-27T07:35:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.110+0000 7fb90f400640 1 -- 172.21.15.63:0/3767703518 shutdown_connections 2024-05-27T07:35:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.110+0000 7fb90f400640 1 --2- 172.21.15.63:0/3767703518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9081043e0 0x7fb9081107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.110+0000 7fb90f400640 1 --2- 172.21.15.63:0/3767703518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90810a130 0x7fb908103d70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.110+0000 7fb90f400640 1 --2- 172.21.15.63:0/3767703518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb908109c90 0x7fb908103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.110+0000 7fb90f400640 1 -- 172.21.15.63:0/3767703518 >> 172.21.15.63:0/3767703518 conn(0x7fb908079390 msgr2=0x7fb9080797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.110+0000 7fb90f400640 1 -- 172.21.15.63:0/3767703518 shutdown_connections 2024-05-27T07:35:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.110+0000 7fb90f400640 1 -- 172.21.15.63:0/3767703518 wait complete. 2024-05-27T07:35:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.111+0000 7fb90f400640 1 Processor -- start 2024-05-27T07:35:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.111+0000 7fb90f400640 1 -- start start 2024-05-27T07:35:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.112+0000 7fb90f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9081043e0 0x7fb908198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.112+0000 7fb90f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb908109c90 0x7fb908198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.112+0000 7fb90f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90810a130 0x7fb90819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.112+0000 7fb90f400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb908111b40 con 0x7fb90810a130 2024-05-27T07:35:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.112+0000 7fb90f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081119c0 con 0x7fb908109c90 2024-05-27T07:35:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.112+0000 7fb90f400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb908111cc0 con 0x7fb9081043e0 2024-05-27T07:35:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.113+0000 7fb90d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90810a130 0x7fb90819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.113+0000 7fb90d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90810a130 0x7fb90819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59236/0 (socket says 172.21.15.63:59236) 2024-05-27T07:35:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.113+0000 7fb90d400640 1 -- 172.21.15.63:0/3358804050 learned_addr learned my addr 172.21.15.63:0/3358804050 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.113+0000 7fb90ca00640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9081043e0 0x7fb908198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.113+0000 7fb907e00640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb908109c90 0x7fb908198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.114+0000 7fb90d400640 1 -- 172.21.15.63:0/3358804050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9081043e0 msgr2=0x7fb908198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.114+0000 7fb90d400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9081043e0 0x7fb908198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.114+0000 7fb90d400640 1 -- 172.21.15.63:0/3358804050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb908109c90 msgr2=0x7fb908198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.114+0000 7fb90d400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb908109c90 0x7fb908198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.114+0000 7fb90d400640 1 -- 172.21.15.63:0/3358804050 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90819d480 con 0x7fb90810a130 2024-05-27T07:35:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.115+0000 7fb90d400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90810a130 0x7fb90819ce40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb90000eaa0 tx=0x7fb90000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.115+0000 7fb905600640 1 -- 172.21.15.63:0/3358804050 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90000cdb0 con 0x7fb90810a130 2024-05-27T07:35:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.115+0000 7fb905600640 1 -- 172.21.15.63:0/3358804050 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb900005020 con 0x7fb90810a130 2024-05-27T07:35:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.115+0000 7fb90f400640 1 -- 172.21.15.63:0/3358804050 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90819d770 con 0x7fb90810a130 2024-05-27T07:35:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.116+0000 7fb90f400640 1 -- 172.21.15.63:0/3358804050 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90819dc40 con 0x7fb90810a130 2024-05-27T07:35:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.116+0000 7fb905600640 1 -- 172.21.15.63:0/3358804050 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90000cdb0 con 0x7fb90810a130 2024-05-27T07:35:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.121+0000 7fb916477740 1 Processor -- start 2024-05-27T07:35:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.121+0000 7fb905600640 1 -- 172.21.15.63:0/3358804050 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb900004a40 con 0x7fb90810a130 2024-05-27T07:35:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.121+0000 7fb916477740 1 -- start start 2024-05-27T07:35:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.122+0000 7fb916477740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac06c01b0 0x55bac0a6b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.122+0000 7fb916477740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a6b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.123+0000 7fb916477740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac069f4d0 0x55bac0a6baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.123+0000 7fb916477740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bac09d3b90 con 0x55bac06972f0 2024-05-27T07:35:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.123+0000 7fb916477740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bac09c3730 con 0x55bac06c01b0 2024-05-27T07:35:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.123+0000 7fb916477740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bac09c7770 con 0x55bac069f4d0 2024-05-27T07:35:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.123+0000 7fb90ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a6b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.123+0000 7fb90ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a6b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59244/0 (socket says 172.21.15.63:59244) 2024-05-27T07:35:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.123+0000 7fb90ca00640 1 -- 172.21.15.63:0/3188649345 learned_addr learned my addr 172.21.15.63:0/3188649345 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.123+0000 7fb905600640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8f00b2b50 0x7fb8f00b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.124+0000 7fb907e00640 1 --2- 172.21.15.63:0/3188649345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac06c01b0 0x55bac0a6b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.125+0000 7fb907e00640 1 --2- 172.21.15.63:0/3188649345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac069f4d0 0x55bac0a6baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.124+0000 7fb905600640 1 -- 172.21.15.63:0/3358804050 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb9000e26c0 con 0x7fb90810a130 2024-05-27T07:35:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.126+0000 7fb907e00640 1 -- 172.21.15.63:0/3188649345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac069f4d0 msgr2=0x55bac0a6baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.126+0000 7fb907e00640 1 --2- 172.21.15.63:0/3188649345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac069f4d0 0x55bac0a6baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.126+0000 7fb907e00640 1 -- 172.21.15.63:0/3188649345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 msgr2=0x55bac0a6b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.126+0000 7fb907e00640 1 --2- 172.21.15.63:0/3188649345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a6b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.126+0000 7fb907e00640 1 -- 172.21.15.63:0/3188649345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac095c8b0 con 0x55bac06c01b0 2024-05-27T07:35:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.126+0000 7fb90d400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8f00b2b50 0x7fb8f00b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.126+0000 7fb907e00640 1 --2- 172.21.15.63:0/3188649345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac06c01b0 0x55bac0a6b070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009230 tx=0x7fb8f4009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b8cd45807642a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.127+0000 7fb8f9c00640 1 -- 172.21.15.63:0/3188649345 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4048070 con 0x55bac06c01b0 2024-05-27T07:35:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.127+0000 7fb8f9c00640 1 -- 172.21.15.63:0/3188649345 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f4004020 con 0x55bac06c01b0 2024-05-27T07:35:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.127+0000 7fb8f9c00640 1 -- 172.21.15.63:0/3188649345 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40035f0 con 0x55bac06c01b0 2024-05-27T07:35:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.129+0000 7fb916477740 1 -- 172.21.15.63:0/3188649345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac06c01b0 msgr2=0x55bac0a6b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.129+0000 7fb916477740 1 --2- 172.21.15.63:0/3188649345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac06c01b0 0x55bac0a6b070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009230 tx=0x7fb8f4009560 comp rx=0 tx=0).stop 2024-05-27T07:35:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.129+0000 7fb90d400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8f00b2b50 0x7fb8f00b5060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb90000cfd0 tx=0x7fb900005b20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.129+0000 7fb916477740 1 -- 172.21.15.63:0/3188649345 shutdown_connections 2024-05-27T07:35:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.129+0000 7fb916477740 1 --2- 172.21.15.63:0/3188649345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac069f4d0 0x55bac0a6baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.129+0000 7fb916477740 1 --2- 172.21.15.63:0/3188649345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a6b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.129+0000 7fb916477740 1 --2- 172.21.15.63:0/3188649345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac06c01b0 0x55bac0a6b070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.129+0000 7fb916477740 1 -- 172.21.15.63:0/3188649345 >> 172.21.15.63:0/3188649345 conn(0x55bac06b4300 msgr2=0x55bac0a67820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.129+0000 7fb916477740 1 -- 172.21.15.63:0/3188649345 shutdown_connections 2024-05-27T07:35:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.129+0000 7fb916477740 1 -- 172.21.15.63:0/3188649345 wait complete. 2024-05-27T07:35:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.130+0000 7fb916477740 1 -- start start 2024-05-27T07:35:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.131+0000 7fb916477740 1 Processor -- start 2024-05-27T07:35:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.132+0000 7fb916477740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.132+0000 7fb916477740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 0x55bac0a75a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.132+0000 7fb907e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.133+0000 7fb907e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a754e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59250/0 (socket says 172.21.15.63:59250) 2024-05-27T07:35:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.133+0000 7fb907e00640 1 -- 172.21.15.63:0/348052839 learned_addr learned my addr 172.21.15.63:0/348052839 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.133+0000 7fb916477740 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06c01b0 0x55bac0a81470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.133+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bac06a9790 con 0x55bac06972f0 2024-05-27T07:35:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.133+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bac096bbc0 con 0x55bac069f4d0 2024-05-27T07:35:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.133+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bac0a78140 con 0x55bac06c01b0 2024-05-27T07:35:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.134+0000 7fb90ca00640 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 0x55bac0a75a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.134+0000 7fb907e00640 1 -- 172.21.15.63:0/348052839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06c01b0 msgr2=0x55bac0a81470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.134+0000 7fb907e00640 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06c01b0 0x55bac0a81470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.134+0000 7fb916477740 10 client.0 _ll_register_callbacks cb 0x55bac0949400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.134+0000 7fb907e00640 1 -- 172.21.15.63:0/348052839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 msgr2=0x55bac0a75a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.134+0000 7fb907e00640 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 0x55bac0a75a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.134+0000 7fb907e00640 1 -- 172.21.15.63:0/348052839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac06d8210 con 0x55bac06972f0 2024-05-27T07:35:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.134+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f4004020 con 0x55bac06972f0 2024-05-27T07:35:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.134+0000 7fb90ca00640 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 0x55bac0a75a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.135+0000 7fb907e00640 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a754e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb8f402ed40 tx=0x7fb8f40092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b473c83268ec7c63 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.135+0000 7fb8ee000640 10 client.15687 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.135+0000 7fb8ee000640 1 -- 172.21.15.63:0/348052839 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f405b050 con 0x55bac06972f0 2024-05-27T07:35:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.136+0000 7fb8ee000640 1 -- 172.21.15.63:0/348052839 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb8f4036620 con 0x55bac06972f0 2024-05-27T07:35:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.136+0000 7fb8ee000640 1 -- 172.21.15.63:0/348052839 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8f4048040 con 0x55bac06972f0 2024-05-27T07:35:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.138+0000 7fb916477740 10 client.15687 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.138+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55bac07a4620 con 0x55bac06972f0 2024-05-27T07:35:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.138+0000 7fb8ee000640 1 -- 172.21.15.63:0/348052839 <== mon.0 v2:172.21.15.63:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8f405e020 con 0x55bac06972f0 2024-05-27T07:35:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.138+0000 7fb916477740 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.138+0000 7fb916477740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.138+0000 7fb916477740 20 client.15687 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.138+0000 7fb916477740 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bac067ed80 0x55bac0a870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.138+0000 7fb916477740 4 client.15687 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.138+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bac09d5ae0 con 0x55bac067ed80 2024-05-27T07:35:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.139+0000 7fb90ca00640 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bac067ed80 0x55bac0a870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.140+0000 7fb8ee000640 10 client.15687 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.140+0000 7fb90ca00640 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bac067ed80 0x55bac0a870a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52902863a64cb server_cookie=64f21a19915847a9 in_seq=0 out_seq=0 2024-05-27T07:35:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.142+0000 7fb8ee000640 1 -- 172.21.15.63:0/348052839 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bac09d5ae0 con 0x55bac067ed80 2024-05-27T07:35:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.142+0000 7fb8ee000640 10 client.15687 handle_command_reply: tid=0 2024-05-27T07:35:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb916477740 1 client.15687 shutdown 2024-05-27T07:35:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 msgr2=0x55bac0a754e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb916477740 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a754e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb8f402ed40 tx=0x7fb8f40092a0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 shutdown_connections 2024-05-27T07:35:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb916477740 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bac067ed80 0x55bac0a870a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb916477740 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06c01b0 0x55bac0a81470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb916477740 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 0x55bac0a75a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb916477740 1 --2- 172.21.15.63:0/348052839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06972f0 0x55bac0a754e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 >> 172.21.15.63:0/348052839 conn(0x55bac06b4300 msgr2=0x55bac0a6ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb8ee000640 0 client.15687 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.143+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 shutdown_connections 2024-05-27T07:35:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.144+0000 7fb916477740 1 -- 172.21.15.63:0/348052839 wait complete. 2024-05-27T07:35:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.144+0000 7fb916477740 20 client.15687 trim_cache size 0 max 16384 2024-05-27T07:35:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.176+0000 7fb916477740 1 Processor -- start 2024-05-27T07:35:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.177+0000 7fb916477740 1 -- start start 2024-05-27T07:35:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.178+0000 7fb916477740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 0x55bac0a5e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.179+0000 7fb916477740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06c01b0 0x55bac0a813e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.179+0000 7fb916477740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 0x55bac0a77100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.179+0000 7fb916477740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081b1810 con 0x55bac06c01b0 2024-05-27T07:35:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.179+0000 7fb916477740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081af0e0 con 0x55bac069f4d0 2024-05-27T07:35:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.179+0000 7fb916477740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081acf30 con 0x55bac06541c0 2024-05-27T07:35:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.179+0000 7fb907e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 0x55bac0a5e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.179+0000 7fb90ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06c01b0 0x55bac0a813e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.179+0000 7fb90ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06c01b0 0x55bac0a813e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59266/0 (socket says 172.21.15.63:59266) 2024-05-27T07:35:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.179+0000 7fb90ca00640 1 -- 172.21.15.63:0/1617147810 learned_addr learned my addr 172.21.15.63:0/1617147810 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.180+0000 7fb907e00640 1 --2- 172.21.15.63:0/1617147810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 0x55bac0a77100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.180+0000 7fb90ca00640 1 -- 172.21.15.63:0/1617147810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 msgr2=0x55bac0a77100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.180+0000 7fb90ca00640 1 --2- 172.21.15.63:0/1617147810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 0x55bac0a77100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.180+0000 7fb90ca00640 1 -- 172.21.15.63:0/1617147810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 msgr2=0x55bac0a5e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.180+0000 7fb90ca00640 1 --2- 172.21.15.63:0/1617147810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 0x55bac0a5e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.180+0000 7fb90ca00640 1 -- 172.21.15.63:0/1617147810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac0a6f830 con 0x55bac06c01b0 2024-05-27T07:35:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.180+0000 7fb907e00640 1 --2- 172.21.15.63:0/1617147810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 0x55bac0a77100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.181+0000 7fb90ca00640 1 --2- 172.21.15.63:0/1617147810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06c01b0 0x55bac0a813e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55bac0a687b0 tx=0x7fb8fc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59e5989679aba44 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.181+0000 7fb8f9200640 1 -- 172.21.15.63:0/1617147810 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc00bdc0 con 0x55bac06c01b0 2024-05-27T07:35:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.182+0000 7fb8f9200640 1 -- 172.21.15.63:0/1617147810 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8fc0054b0 con 0x55bac06c01b0 2024-05-27T07:35:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.182+0000 7fb8f9200640 1 -- 172.21.15.63:0/1617147810 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc011440 con 0x55bac06c01b0 2024-05-27T07:35:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.182+0000 7fb916477740 1 -- 172.21.15.63:0/1617147810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06c01b0 msgr2=0x55bac0a813e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.182+0000 7fb916477740 1 --2- 172.21.15.63:0/1617147810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06c01b0 0x55bac0a813e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55bac0a687b0 tx=0x7fb8fc00b390 comp rx=0 tx=0).stop 2024-05-27T07:35:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.183+0000 7fb916477740 1 -- 172.21.15.63:0/1617147810 shutdown_connections 2024-05-27T07:35:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.183+0000 7fb916477740 1 --2- 172.21.15.63:0/1617147810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 0x55bac0a77100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.183+0000 7fb916477740 1 --2- 172.21.15.63:0/1617147810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac06c01b0 0x55bac0a813e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.183+0000 7fb916477740 1 --2- 172.21.15.63:0/1617147810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac069f4d0 0x55bac0a5e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.183+0000 7fb916477740 1 -- 172.21.15.63:0/1617147810 >> 172.21.15.63:0/1617147810 conn(0x55bac06b4300 msgr2=0x55bac0a769f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.183+0000 7fb916477740 1 -- 172.21.15.63:0/1617147810 shutdown_connections 2024-05-27T07:35:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.183+0000 7fb916477740 1 -- 172.21.15.63:0/1617147810 wait complete. 2024-05-27T07:35:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.183+0000 7fb916477740 1 -- start start 2024-05-27T07:35:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.184+0000 7fb916477740 1 Processor -- start 2024-05-27T07:35:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.184+0000 7fb916477740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 0x55bac0ae6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.184+0000 7fb916477740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac069f4d0 0x55bac0ae6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.184+0000 7fb916477740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac0a87630 0x55bac0a87a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.184+0000 7fb916477740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081ad5a0 con 0x55bac069f4d0 2024-05-27T07:35:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.184+0000 7fb916477740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081aea10 con 0x55bac0a87630 2024-05-27T07:35:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.184+0000 7fb916477740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9081adc70 con 0x55bac06541c0 2024-05-27T07:35:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.184+0000 7fb916477740 10 client.0 _ll_register_callbacks cb 0x7fb8fc004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.185+0000 7fb90ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac069f4d0 0x55bac0ae6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.185+0000 7fb90ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac069f4d0 0x55bac0ae6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59280/0 (socket says 172.21.15.63:59280) 2024-05-27T07:35:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.185+0000 7fb90ca00640 1 -- 172.21.15.63:0/3126712903 learned_addr learned my addr 172.21.15.63:0/3126712903 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.185+0000 7fb907e00640 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac0a87630 0x55bac0a87a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.185+0000 7fb907e00640 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 0x55bac0ae6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.185+0000 7fb90ca00640 1 -- 172.21.15.63:0/3126712903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 msgr2=0x55bac0ae6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.185+0000 7fb90ca00640 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 0x55bac0ae6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.185+0000 7fb90ca00640 1 -- 172.21.15.63:0/3126712903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac0a87630 msgr2=0x55bac0a87a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.185+0000 7fb90ca00640 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac0a87630 0x55bac0a87a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.185+0000 7fb90ca00640 1 -- 172.21.15.63:0/3126712903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac0a88130 con 0x55bac069f4d0 2024-05-27T07:35:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.186+0000 7fb90ca00640 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac069f4d0 0x55bac0ae6dd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55bac0a82200 tx=0x7fb8fc00bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d973fcf7360007c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.186+0000 7fb8ee000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.187+0000 7fb8ee000640 1 -- 172.21.15.63:0/3126712903 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc004f90 con 0x55bac069f4d0 2024-05-27T07:35:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.187+0000 7fb916477740 1 -- 172.21.15.63:0/3126712903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8fc0054b0 con 0x55bac069f4d0 2024-05-27T07:35:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.187+0000 7fb8ee000640 1 -- 172.21.15.63:0/3126712903 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb8fc005130 con 0x55bac069f4d0 2024-05-27T07:35:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.187+0000 7fb8ee000640 1 -- 172.21.15.63:0/3126712903 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8fc004730 con 0x55bac069f4d0 2024-05-27T07:35:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.187+0000 7fb916477740 10 client.15693 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.187+0000 7fb916477740 1 -- 172.21.15.63:0/3126712903 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bac0aef2c0 con 0x55bac069f4d0 2024-05-27T07:35:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.188+0000 7fb8ee000640 1 -- 172.21.15.63:0/3126712903 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8fc0054b0 con 0x55bac069f4d0 2024-05-27T07:35:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.188+0000 7fb8ee000640 1 -- 172.21.15.63:0/3126712903 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8fc030060 con 0x55bac069f4d0 2024-05-27T07:35:18.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.188+0000 7fb916477740 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.188+0000 7fb916477740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.188+0000 7fb916477740 20 client.15693 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.188+0000 7fb916477740 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bac0aef920 0x55bac0af1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.188+0000 7fb916477740 4 client.15693 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:35:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.188+0000 7fb916477740 1 -- 172.21.15.63:0/3126712903 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bac09cf2c0 con 0x55bac0aef920 2024-05-27T07:35:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.189+0000 7fb907e00640 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bac0aef920 0x55bac0af1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.190+0000 7fb907e00640 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bac0aef920 0x55bac0af1d40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22daa6c5f68c2e49 server_cookie=310d5fc205dea9b1 in_seq=0 out_seq=0 2024-05-27T07:35:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.190+0000 7fb8ee000640 10 client.15693 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb8ee000640 1 -- 172.21.15.63:0/3126712903 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bac09cf2c0 con 0x55bac0aef920 2024-05-27T07:35:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb8ee000640 10 client.15693 handle_command_reply: tid=0 2024-05-27T07:35:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb916477740 1 client.15693 shutdown 2024-05-27T07:35:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb916477740 1 -- 172.21.15.63:0/3126712903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac069f4d0 msgr2=0x55bac0ae6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb916477740 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac069f4d0 0x55bac0ae6dd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55bac0a82200 tx=0x7fb8fc00bdc0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb916477740 1 -- 172.21.15.63:0/3126712903 shutdown_connections 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb916477740 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bac0aef920 0x55bac0af1d40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb916477740 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bac0a87630 0x55bac0a87a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb916477740 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bac069f4d0 0x55bac0ae6dd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb916477740 1 --2- 172.21.15.63:0/3126712903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bac06541c0 0x55bac0ae6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.205+0000 7fb916477740 1 -- 172.21.15.63:0/3126712903 >> 172.21.15.63:0/3126712903 conn(0x55bac06b4300 msgr2=0x55bac0a77340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.206+0000 7fb8ee000640 0 client.15693 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.206+0000 7fb916477740 1 -- 172.21.15.63:0/3126712903 shutdown_connections 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.206+0000 7fb916477740 1 -- 172.21.15.63:0/3126712903 wait complete. 2024-05-27T07:35:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.206+0000 7fb916477740 20 client.15693 trim_cache size 0 max 16384 2024-05-27T07:35:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:35:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:35:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:35:18.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 -- 172.21.15.63:0/3358804050 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8f00b2b50 msgr2=0x7fb8f00b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8f00b2b50 0x7fb8f00b5060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb90000cfd0 tx=0x7fb900005b20 comp rx=0 tx=0).stop 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 -- 172.21.15.63:0/3358804050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90810a130 msgr2=0x7fb90819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90810a130 0x7fb90819ce40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb90000eaa0 tx=0x7fb90000ef60 comp rx=0 tx=0).stop 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 -- 172.21.15.63:0/3358804050 shutdown_connections 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8f00b2b50 0x7fb8f00b5060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90810a130 0x7fb90819ce40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb908109c90 0x7fb908198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 --2- 172.21.15.63:0/3358804050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9081043e0 0x7fb908198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 -- 172.21.15.63:0/3358804050 >> 172.21.15.63:0/3358804050 conn(0x7fb908079390 msgr2=0x7fb9081023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 -- 172.21.15.63:0/3358804050 shutdown_connections 2024-05-27T07:35:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:18.207+0000 7fb90f400640 1 -- 172.21.15.63:0/3358804050 wait complete. 2024-05-27T07:35:18.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.533+0000 7faa65600700 1 -- 172.21.15.63:0/801995748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa601034b0 msgr2=0x7faa601038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.533+0000 7faa65600700 1 --2- 172.21.15.63:0/801995748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa601034b0 0x7faa601038c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7faa500099c0 tx=0x7faa50009cd0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.533+0000 7faa65600700 1 -- 172.21.15.63:0/801995748 shutdown_connections 2024-05-27T07:35:18.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.533+0000 7faa65600700 1 --2- 172.21.15.63:0/801995748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa60108730 0x7faa60110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.533+0000 7faa65600700 1 --2- 172.21.15.63:0/801995748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa601034b0 0x7faa601038c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.533+0000 7faa65600700 1 --2- 172.21.15.63:0/801995748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa60107de0 0x7faa601081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.533+0000 7faa65600700 1 -- 172.21.15.63:0/801995748 >> 172.21.15.63:0/801995748 conn(0x7faa600fedb0 msgr2=0x7faa601011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.533+0000 7faa65600700 1 -- 172.21.15.63:0/801995748 shutdown_connections 2024-05-27T07:35:18.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.533+0000 7faa65600700 1 -- 172.21.15.63:0/801995748 wait complete. 2024-05-27T07:35:18.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.535+0000 7faa65600700 1 Processor -- start 2024-05-27T07:35:18.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.535+0000 7faa65600700 1 -- start start 2024-05-27T07:35:18.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.535+0000 7faa65600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa601034b0 0x7faa6019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.535+0000 7faa65600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa60107de0 0x7faa6019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.536+0000 7faa65600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa60108730 0x7faa6019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.536+0000 7faa65600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa601143b0 con 0x7faa60108730 2024-05-27T07:35:18.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.536+0000 7faa65600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa60114230 con 0x7faa601034b0 2024-05-27T07:35:18.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.536+0000 7faa65600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa60114530 con 0x7faa60107de0 2024-05-27T07:35:18.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.536+0000 7faa5f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa60108730 0x7faa6019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.536+0000 7faa5ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa601034b0 0x7faa6019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.536+0000 7faa5f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa60108730 0x7faa6019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59290/0 (socket says 172.21.15.63:59290) 2024-05-27T07:35:18.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.536+0000 7faa5f400700 1 -- 172.21.15.63:0/3889052852 learned_addr learned my addr 172.21.15.63:0/3889052852 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:18.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.537+0000 7faa5f400700 1 -- 172.21.15.63:0/3889052852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa60107de0 msgr2=0x7faa6019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.538+0000 7faa5e000700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa60107de0 0x7faa6019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.539+0000 7faa5f400700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa60107de0 0x7faa6019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.539+0000 7faa5f400700 1 -- 172.21.15.63:0/3889052852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa601034b0 msgr2=0x7faa6019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.539+0000 7faa5f400700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa601034b0 0x7faa6019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.539+0000 7faa5f400700 1 -- 172.21.15.63:0/3889052852 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6019fc90 con 0x7faa60108730 2024-05-27T07:35:18.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.539+0000 7faa5ea00700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa601034b0 0x7faa6019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:18.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.539+0000 7faa5e000700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa60107de0 0x7faa6019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:18.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.539+0000 7faa5f400700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa60108730 0x7faa6019f5f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faa58007ee0 tx=0x7faa5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.540+0000 7faa57400700 1 -- 172.21.15.63:0/3889052852 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5800dbd0 con 0x7faa60108730 2024-05-27T07:35:18.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.540+0000 7faa65600700 1 -- 172.21.15.63:0/3889052852 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6019ff80 con 0x7faa60108730 2024-05-27T07:35:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.540+0000 7faa57400700 1 -- 172.21.15.63:0/3889052852 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa5800dd70 con 0x7faa60108730 2024-05-27T07:35:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.540+0000 7faa65600700 1 -- 172.21.15.63:0/3889052852 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa601a78a0 con 0x7faa60108730 2024-05-27T07:35:18.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.540+0000 7faa57400700 1 -- 172.21.15.63:0/3889052852 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580058d0 con 0x7faa60108730 2024-05-27T07:35:18.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.545+0000 7faa74158b80 1 Processor -- start 2024-05-27T07:35:18.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.547+0000 7faa57400700 1 -- 172.21.15.63:0/3889052852 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faa5801a020 con 0x7faa60108730 2024-05-27T07:35:18.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.547+0000 7faa74158b80 1 -- start start 2024-05-27T07:35:18.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.548+0000 7faa74158b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574592a89f0 0x55745933b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.548+0000 7faa74158b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 0x55745933b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.548+0000 7faa74158b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x557459348110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.548+0000 7faa74158b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557459269e40 con 0x55745933be70 2024-05-27T07:35:18.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.548+0000 7faa74158b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574592599e0 con 0x5574592a89f0 2024-05-27T07:35:18.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.548+0000 7faa74158b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55745925da20 con 0x5574591bce40 2024-05-27T07:35:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.548+0000 7faa5e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x557459348110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.548+0000 7faa5e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x557459348110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59292/0 (socket says 172.21.15.63:59292) 2024-05-27T07:35:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.548+0000 7faa5e000700 1 -- 172.21.15.63:0/1572161571 learned_addr learned my addr 172.21.15.63:0/1572161571 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.548+0000 7faa5ea00700 1 --2- 172.21.15.63:0/1572161571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 0x55745933b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.549+0000 7faa5e000700 1 --2- 172.21.15.63:0/1572161571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574592a89f0 0x55745933b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.549+0000 7faa5e000700 1 -- 172.21.15.63:0/1572161571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 msgr2=0x55745933b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.549+0000 7faa5e000700 1 --2- 172.21.15.63:0/1572161571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 0x55745933b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.549+0000 7faa5e000700 1 -- 172.21.15.63:0/1572161571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574592a89f0 msgr2=0x55745933b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.549+0000 7faa5e000700 1 --2- 172.21.15.63:0/1572161571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574592a89f0 0x55745933b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.549+0000 7faa5e000700 1 -- 172.21.15.63:0/1572161571 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557459222420 con 0x55745933be70 2024-05-27T07:35:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.550+0000 7faa57400700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa440b2a70 0x7faa440b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.550+0000 7faa5e000700 1 --2- 172.21.15.63:0/1572161571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x557459348110 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00bea0 tx=0x7faa4c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf57dbab64d1a73f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.550+0000 7faa57400700 1 -- 172.21.15.63:0/3889052852 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7faa580e20f0 con 0x7faa60108730 2024-05-27T07:35:18.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.551+0000 7faa5ea00700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa440b2a70 0x7faa440b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.550+0000 7faa4be00700 1 -- 172.21.15.63:0/1572161571 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c01b0f0 con 0x55745933be70 2024-05-27T07:35:18.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.551+0000 7faa4be00700 1 -- 172.21.15.63:0/1572161571 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa4c01b720 con 0x55745933be70 2024-05-27T07:35:18.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.551+0000 7faa4be00700 1 -- 172.21.15.63:0/1572161571 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00ea10 con 0x55745933be70 2024-05-27T07:35:18.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.552+0000 7faa74158b80 1 -- 172.21.15.63:0/1572161571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 msgr2=0x557459348110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.552+0000 7faa74158b80 1 --2- 172.21.15.63:0/1572161571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x557459348110 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00bea0 tx=0x7faa4c00b480 comp rx=0 tx=0).stop 2024-05-27T07:35:18.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.552+0000 7faa74158b80 1 -- 172.21.15.63:0/1572161571 shutdown_connections 2024-05-27T07:35:18.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.552+0000 7faa74158b80 1 --2- 172.21.15.63:0/1572161571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x557459348110 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.552+0000 7faa74158b80 1 --2- 172.21.15.63:0/1572161571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574592a89f0 0x55745933b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.552+0000 7faa74158b80 1 --2- 172.21.15.63:0/1572161571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 0x55745933b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.552+0000 7faa74158b80 1 -- 172.21.15.63:0/1572161571 >> 172.21.15.63:0/1572161571 conn(0x557459256390 msgr2=0x5574593374c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.552+0000 7faa74158b80 1 -- 172.21.15.63:0/1572161571 shutdown_connections 2024-05-27T07:35:18.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.552+0000 7faa74158b80 1 -- 172.21.15.63:0/1572161571 wait complete. 2024-05-27T07:35:18.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.552+0000 7faa5ea00700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa440b2a70 0x7faa440b4f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa50006010 tx=0x7faa500058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.553+0000 7faa74158b80 1 -- start start 2024-05-27T07:35:18.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.553+0000 7faa74158b80 1 Processor -- start 2024-05-27T07:35:18.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa74158b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 0x557459345650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa74158b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459345b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa74158b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745933be70 0x5574593460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa74158b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55745915ffa0 con 0x5574592a89f0 2024-05-27T07:35:18.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa74158b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574591fa760 con 0x55745933be70 2024-05-27T07:35:18.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa74158b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557459348b80 con 0x5574591bce40 2024-05-27T07:35:18.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa74158b80 10 client.0 _ll_register_callbacks cb 0x557459203c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:18.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa5ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 0x557459345650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa5ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 0x557459345650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:44222/0 (socket says 172.21.15.63:44222) 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa5ea00700 1 -- 172.21.15.63:0/4040944595 learned_addr learned my addr 172.21.15.63:0/4040944595 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.554+0000 7faa5e000700 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459345b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.555+0000 7faa5e000700 1 -- 172.21.15.63:0/4040944595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 msgr2=0x557459345650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.555+0000 7faa5e000700 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 0x557459345650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.555+0000 7faa5e000700 1 -- 172.21.15.63:0/4040944595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745933be70 msgr2=0x5574593460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.555+0000 7faa5f400700 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745933be70 0x5574593460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.555+0000 7faa5e000700 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745933be70 0x5574593460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.555+0000 7faa5e000700 1 -- 172.21.15.63:0/4040944595 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574591a98d0 con 0x5574592a89f0 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.555+0000 7faa5f400700 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745933be70 0x5574593460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.555+0000 7faa5e000700 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459345b90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faa4c023000 tx=0x7faa4c010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac89b893d945d4f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.556+0000 7faa48c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:18.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.556+0000 7faa48c00700 1 -- 172.21.15.63:0/4040944595 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c004220 con 0x5574592a89f0 2024-05-27T07:35:18.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.556+0000 7faa48c00700 1 -- 172.21.15.63:0/4040944595 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7faa4c0043c0 con 0x5574592a89f0 2024-05-27T07:35:18.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.556+0000 7faa74158b80 1 -- 172.21.15.63:0/4040944595 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa4c01b720 con 0x5574592a89f0 2024-05-27T07:35:18.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.556+0000 7faa48c00700 1 -- 172.21.15.63:0/4040944595 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa4c037050 con 0x5574592a89f0 2024-05-27T07:35:18.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.556+0000 7faa74158b80 10 client.15705 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:18.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.556+0000 7faa74158b80 1 -- 172.21.15.63:0/4040944595 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55745922a060 con 0x5574592a89f0 2024-05-27T07:35:18.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.557+0000 7faa48c00700 1 -- 172.21.15.63:0/4040944595 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa4c01b720 con 0x5574592a89f0 2024-05-27T07:35:18.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.557+0000 7faa48c00700 1 -- 172.21.15.63:0/4040944595 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faa4c00f040 con 0x5574592a89f0 2024-05-27T07:35:18.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.557+0000 7faa74158b80 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:18.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.557+0000 7faa74158b80 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:18.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.557+0000 7faa74158b80 20 client.15705 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:18.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.557+0000 7faa74158b80 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557459355990 0x557459357db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.557+0000 7faa74158b80 4 client.15705 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:18.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.557+0000 7faa74158b80 1 -- 172.21.15.63:0/4040944595 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55745926bd90 con 0x557459355990 2024-05-27T07:35:18.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.558+0000 7faa5ea00700 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557459355990 0x557459357db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.559+0000 7faa5ea00700 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557459355990 0x557459357db0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ffad2b81856bbe2 server_cookie=b2b42e51dbcc1e33 in_seq=0 out_seq=0 2024-05-27T07:35:18.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.559+0000 7faa48c00700 10 client.15705 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:18.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.561+0000 7faa48c00700 1 -- 172.21.15.63:0/4040944595 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55745926bd90 con 0x557459355990 2024-05-27T07:35:18.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.561+0000 7faa48c00700 10 client.15705 handle_command_reply: tid=0 2024-05-27T07:35:18.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.561+0000 7faa74158b80 1 client.15705 shutdown 2024-05-27T07:35:18.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 1 -- 172.21.15.63:0/4040944595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 msgr2=0x557459345b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459345b90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faa4c023000 tx=0x7faa4c010fc0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 1 -- 172.21.15.63:0/4040944595 shutdown_connections 2024-05-27T07:35:18.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557459355990 0x557459357db0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55745933be70 0x5574593460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574591bce40 0x557459345650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 1 --2- 172.21.15.63:0/4040944595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459345b90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 1 -- 172.21.15.63:0/4040944595 >> 172.21.15.63:0/4040944595 conn(0x557459256390 msgr2=0x55745933b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa48c00700 0 client.15705 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:18.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 1 -- 172.21.15.63:0/4040944595 shutdown_connections 2024-05-27T07:35:18.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 1 -- 172.21.15.63:0/4040944595 wait complete. 2024-05-27T07:35:18.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.562+0000 7faa74158b80 20 client.15705 trim_cache size 0 max 16384 2024-05-27T07:35:18.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.578+0000 7faa74158b80 1 Processor -- start 2024-05-27T07:35:18.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.579+0000 7faa74158b80 1 -- start start 2024-05-27T07:35:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.579+0000 7faa74158b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 0x557459351e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.579+0000 7faa74158b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574592a89f0 0x5574593374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.579+0000 7faa74158b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x55745933b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.579+0000 7faa74158b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa400114f0 con 0x55745933be70 2024-05-27T07:35:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.579+0000 7faa74158b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa4000ed80 con 0x5574591bce40 2024-05-27T07:35:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.579+0000 7faa74158b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa4000cbd0 con 0x5574592a89f0 2024-05-27T07:35:18.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.579+0000 7faa5f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x55745933b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.579+0000 7faa5f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x55745933b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59310/0 (socket says 172.21.15.63:59310) 2024-05-27T07:35:18.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.579+0000 7faa5f400700 1 -- 172.21.15.63:0/3898724973 learned_addr learned my addr 172.21.15.63:0/3898724973 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:18.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.580+0000 7faa5ea00700 1 --2- 172.21.15.63:0/3898724973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 0x557459351e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.580+0000 7faa5f400700 1 -- 172.21.15.63:0/3898724973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574592a89f0 msgr2=0x5574593374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.580+0000 7faa5f400700 1 --2- 172.21.15.63:0/3898724973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574592a89f0 0x5574593374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.580+0000 7faa5f400700 1 -- 172.21.15.63:0/3898724973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 msgr2=0x557459351e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.580+0000 7faa5f400700 1 --2- 172.21.15.63:0/3898724973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 0x557459351e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.580+0000 7faa5f400700 1 -- 172.21.15.63:0/3898724973 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557459348710 con 0x55745933be70 2024-05-27T07:35:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.581+0000 7faa5f400700 1 --2- 172.21.15.63:0/3898724973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x55745933b790 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7faa580a6270 tx=0x7faa580a8a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52705bfd53cb71a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.582+0000 7faa4b400700 1 -- 172.21.15.63:0/3898724973 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580df900 con 0x55745933be70 2024-05-27T07:35:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.582+0000 7faa4b400700 1 -- 172.21.15.63:0/3898724973 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa580a9700 con 0x55745933be70 2024-05-27T07:35:18.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.582+0000 7faa4b400700 1 -- 172.21.15.63:0/3898724973 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa580b19d0 con 0x55745933be70 2024-05-27T07:35:18.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.583+0000 7faa74158b80 1 -- 172.21.15.63:0/3898724973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 msgr2=0x55745933b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.583+0000 7faa74158b80 1 --2- 172.21.15.63:0/3898724973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x55745933b790 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7faa580a6270 tx=0x7faa580a8a20 comp rx=0 tx=0).stop 2024-05-27T07:35:18.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.583+0000 7faa74158b80 1 -- 172.21.15.63:0/3898724973 shutdown_connections 2024-05-27T07:35:18.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.583+0000 7faa74158b80 1 --2- 172.21.15.63:0/3898724973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55745933be70 0x55745933b790 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.583+0000 7faa74158b80 1 --2- 172.21.15.63:0/3898724973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 0x557459351e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.583+0000 7faa74158b80 1 --2- 172.21.15.63:0/3898724973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574592a89f0 0x5574593374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.583+0000 7faa74158b80 1 -- 172.21.15.63:0/3898724973 >> 172.21.15.63:0/3898724973 conn(0x557459256390 msgr2=0x557459345430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.584+0000 7faa74158b80 1 -- 172.21.15.63:0/3898724973 shutdown_connections 2024-05-27T07:35:18.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.584+0000 7faa74158b80 1 -- 172.21.15.63:0/3898724973 wait complete. 2024-05-27T07:35:18.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.584+0000 7faa74158b80 1 -- start start 2024-05-27T07:35:18.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.585+0000 7faa74158b80 1 Processor -- start 2024-05-27T07:35:18.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.585+0000 7faa74158b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 0x557459357700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.585+0000 7faa74158b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459357c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.585+0000 7faa74158b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55745933be70 0x557459358180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.585+0000 7faa74158b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa4000d240 con 0x5574592a89f0 2024-05-27T07:35:18.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.586+0000 7faa74158b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa4000e6b0 con 0x5574591bce40 2024-05-27T07:35:18.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.586+0000 7faa74158b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa4000d910 con 0x55745933be70 2024-05-27T07:35:18.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.586+0000 7faa74158b80 10 client.0 _ll_register_callbacks cb 0x557459203c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:18.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.586+0000 7faa5ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 0x557459357700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.586+0000 7faa5e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459357c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.586+0000 7faa5e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459357c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59314/0 (socket says 172.21.15.63:59314) 2024-05-27T07:35:18.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.586+0000 7faa5e000700 1 -- 172.21.15.63:0/2218149040 learned_addr learned my addr 172.21.15.63:0/2218149040 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:18.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.586+0000 7faa5f400700 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55745933be70 0x557459358180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.587+0000 7faa5e000700 1 -- 172.21.15.63:0/2218149040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55745933be70 msgr2=0x557459358180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.587+0000 7faa5e000700 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55745933be70 0x557459358180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.587+0000 7faa5e000700 1 -- 172.21.15.63:0/2218149040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 msgr2=0x557459357700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:18.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.587+0000 7faa5e000700 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 0x557459357700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.587+0000 7faa5e000700 1 -- 172.21.15.63:0/2218149040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557459333240 con 0x5574592a89f0 2024-05-27T07:35:18.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.587+0000 7faa5ea00700 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 0x557459357700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:35:18.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.587+0000 7faa5e000700 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459357c40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00b830 tx=0x7faa4c005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac21047926d7f3ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:18.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.587+0000 7faa48c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:18.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.588+0000 7faa48c00700 1 -- 172.21.15.63:0/2218149040 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00bb80 con 0x5574592a89f0 2024-05-27T07:35:18.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.588+0000 7faa48c00700 1 -- 172.21.15.63:0/2218149040 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7faa4c004710 con 0x5574592a89f0 2024-05-27T07:35:18.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.588+0000 7faa74158b80 1 -- 172.21.15.63:0/2218149040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa580a9700 con 0x5574592a89f0 2024-05-27T07:35:18.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.588+0000 7faa48c00700 1 -- 172.21.15.63:0/2218149040 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa4c011240 con 0x5574592a89f0 2024-05-27T07:35:18.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.588+0000 7faa74158b80 10 client.15711 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:18.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.588+0000 7faa74158b80 1 -- 172.21.15.63:0/2218149040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5574593bf1e0 con 0x5574592a89f0 2024-05-27T07:35:18.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.589+0000 7faa48c00700 1 -- 172.21.15.63:0/2218149040 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa580a9700 con 0x5574592a89f0 2024-05-27T07:35:18.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.589+0000 7faa48c00700 1 -- 172.21.15.63:0/2218149040 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faa4c004d40 con 0x5574592a89f0 2024-05-27T07:35:18.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.589+0000 7faa74158b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:18.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.589+0000 7faa74158b80 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:18.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.589+0000 7faa74158b80 20 client.15711 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:18.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.589+0000 7faa74158b80 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5574593bf860 0x5574593c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:18.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.589+0000 7faa74158b80 4 client.15711 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:35:18.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.589+0000 7faa74158b80 1 -- 172.21.15.63:0/2218149040 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557459265570 con 0x5574593bf860 2024-05-27T07:35:18.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.590+0000 7faa5ea00700 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5574593bf860 0x5574593c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:18.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.591+0000 7faa5ea00700 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5574593bf860 0x5574593c1c80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d61d0ec8c3111e5 server_cookie=305ea09167859379 in_seq=0 out_seq=0 2024-05-27T07:35:18.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.591+0000 7faa48c00700 10 client.15711 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:18.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa48c00700 1 -- 172.21.15.63:0/2218149040 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557459265570 con 0x5574593bf860 2024-05-27T07:35:18.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa48c00700 10 client.15711 handle_command_reply: tid=0 2024-05-27T07:35:18.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa74158b80 1 client.15711 shutdown 2024-05-27T07:35:18.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa74158b80 1 -- 172.21.15.63:0/2218149040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 msgr2=0x557459357c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa74158b80 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459357c40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00b830 tx=0x7faa4c005950 comp rx=0 tx=0).stop 2024-05-27T07:35:18.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa74158b80 1 -- 172.21.15.63:0/2218149040 shutdown_connections 2024-05-27T07:35:18.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa74158b80 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5574593bf860 0x5574593c1c80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa74158b80 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55745933be70 0x557459358180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa74158b80 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574591bce40 0x557459357700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa74158b80 1 --2- 172.21.15.63:0/2218149040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5574592a89f0 0x557459357c40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa74158b80 1 -- 172.21.15.63:0/2218149040 >> 172.21.15.63:0/2218149040 conn(0x557459256390 msgr2=0x5574593364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa48c00700 0 client.15711 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:18.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.592+0000 7faa74158b80 1 -- 172.21.15.63:0/2218149040 shutdown_connections 2024-05-27T07:35:18.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.593+0000 7faa74158b80 1 -- 172.21.15.63:0/2218149040 wait complete. 2024-05-27T07:35:18.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.593+0000 7faa74158b80 20 client.15711 trim_cache size 0 max 16384 2024-05-27T07:35:18.589 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:35:18.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.594+0000 7faa65600700 1 -- 172.21.15.63:0/3889052852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa440b2a70 msgr2=0x7faa440b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.594+0000 7faa65600700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa440b2a70 0x7faa440b4f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa50006010 tx=0x7faa500058e0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.594+0000 7faa65600700 1 -- 172.21.15.63:0/3889052852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa60108730 msgr2=0x7faa6019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:18.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.594+0000 7faa65600700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa60108730 0x7faa6019f5f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7faa58007ee0 tx=0x7faa5800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.595+0000 7faa65600700 1 -- 172.21.15.63:0/3889052852 shutdown_connections 2024-05-27T07:35:18.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.595+0000 7faa65600700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa440b2a70 0x7faa440b4f80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.595+0000 7faa65600700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa60108730 0x7faa6019f5f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.595+0000 7faa65600700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa601034b0 0x7faa6019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.595+0000 7faa65600700 1 --2- 172.21.15.63:0/3889052852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa60107de0 0x7faa6019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:18.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.595+0000 7faa65600700 1 -- 172.21.15.63:0/3889052852 >> 172.21.15.63:0/3889052852 conn(0x7faa600fedb0 msgr2=0x7faa601011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:18.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.595+0000 7faa65600700 1 -- 172.21.15.63:0/3889052852 shutdown_connections 2024-05-27T07:35:18.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:18.595+0000 7faa65600700 1 -- 172.21.15.63:0/3889052852 wait complete. 2024-05-27T07:35:19.413 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:19 smithi063 ceph-mon[31613]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:35:19.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:19 smithi086 ceph-mon[34687]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:35:19.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:19 smithi149 ceph-mon[35408]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-27T07:35:20.478 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:35:20.478 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:35:20.905 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:21.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:21 smithi086 ceph-mon[34687]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T07:35:21.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:21 smithi149 ceph-mon[35408]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T07:35:21.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:21 smithi063 ceph-mon[31613]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T07:35:23.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.236+0000 7f68e5a00700 1 -- 172.21.15.63:0/880759566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68e010b890 msgr2=0x7f68e01131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:23.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.236+0000 7f68e5a00700 1 --2- 172.21.15.63:0/880759566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68e010b890 0x7f68e01131f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f68d800b0d0 tx=0x7f68d800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.236+0000 7f68e5a00700 1 -- 172.21.15.63:0/880759566 shutdown_connections 2024-05-27T07:35:23.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.236+0000 7f68e5a00700 1 --2- 172.21.15.63:0/880759566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68e010b890 0x7f68e01131f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.236+0000 7f68e5a00700 1 --2- 172.21.15.63:0/880759566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e0106610 0x7f68e0106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.236+0000 7f68e5a00700 1 --2- 172.21.15.63:0/880759566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68e010af40 0x7f68e010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.236+0000 7f68e5a00700 1 -- 172.21.15.63:0/880759566 >> 172.21.15.63:0/880759566 conn(0x7f68e0102020 msgr2=0x7f68e0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:23.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.237+0000 7f68e5a00700 1 -- 172.21.15.63:0/880759566 shutdown_connections 2024-05-27T07:35:23.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.237+0000 7f68e5a00700 1 -- 172.21.15.63:0/880759566 wait complete. 2024-05-27T07:35:23.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.239+0000 7f68e5a00700 1 Processor -- start 2024-05-27T07:35:23.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.239+0000 7f68e5a00700 1 -- start start 2024-05-27T07:35:23.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.239+0000 7f68e5a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68e0106610 0x7f68e019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.239+0000 7f68e5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68e010af40 0x7f68e019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.240+0000 7f68e5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e010b890 0x7f68e01a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.240+0000 7f68e5a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68e01163c0 con 0x7f68e010af40 2024-05-27T07:35:23.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.240+0000 7f68e5a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0116240 con 0x7f68e010b890 2024-05-27T07:35:23.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.240+0000 7f68e5a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0116540 con 0x7f68e0106610 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.240+0000 7f68df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68e010af40 0x7f68e019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.240+0000 7f68df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68e010af40 0x7f68e019d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34698/0 (socket says 172.21.15.63:34698) 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.240+0000 7f68df400700 1 -- 172.21.15.63:0/236557565 learned_addr learned my addr 172.21.15.63:0/236557565 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.240+0000 7f68dfe00700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68e0106610 0x7f68e019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.241+0000 7f68e5000700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e010b890 0x7f68e01a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.241+0000 7f68df400700 1 -- 172.21.15.63:0/236557565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68e0106610 msgr2=0x7f68e019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.241+0000 7f68df400700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68e0106610 0x7f68e019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.241+0000 7f68df400700 1 -- 172.21.15.63:0/236557565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e010b890 msgr2=0x7f68e01a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.241+0000 7f68df400700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e010b890 0x7f68e01a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.241+0000 7f68df400700 1 -- 172.21.15.63:0/236557565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e01a1be0 con 0x7f68e010af40 2024-05-27T07:35:23.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.241+0000 7f68dfe00700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68e0106610 0x7f68e019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:23.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.241+0000 7f68df400700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68e010af40 0x7f68e019d2b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00d980 tx=0x7f68cc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:23.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.242+0000 7f68dcc00700 1 -- 172.21.15.63:0/236557565 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc017070 con 0x7f68e010af40 2024-05-27T07:35:23.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.242+0000 7f68e5a00700 1 -- 172.21.15.63:0/236557565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e01a1ed0 con 0x7f68e010af40 2024-05-27T07:35:23.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.242+0000 7f68dcc00700 1 -- 172.21.15.63:0/236557565 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68cc010460 con 0x7f68e010af40 2024-05-27T07:35:23.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.242+0000 7f68dcc00700 1 -- 172.21.15.63:0/236557565 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc00de60 con 0x7f68e010af40 2024-05-27T07:35:23.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.242+0000 7f68e5a00700 1 -- 172.21.15.63:0/236557565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e01a97f0 con 0x7f68e010af40 2024-05-27T07:35:23.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.248+0000 7f68f5b7eb80 1 Processor -- start 2024-05-27T07:35:23.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.248+0000 7f68f5b7eb80 1 -- start start 2024-05-27T07:35:23.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.249+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0553a70 0x55c5c05e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.249+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.249+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c05f1340 0x55c5c05f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.249+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c5c0514ec0 con 0x55c5c0467ec0 2024-05-27T07:35:23.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.249+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5c0504a60 con 0x55c5c05f1340 2024-05-27T07:35:23.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.249+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c5c0508aa0 con 0x55c5c0553a70 2024-05-27T07:35:23.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.250+0000 7f68dcc00700 1 -- 172.21.15.63:0/236557565 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f68cc010610 con 0x7f68e010af40 2024-05-27T07:35:23.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.250+0000 7f68dfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.250+0000 7f68dfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05e7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34702/0 (socket says 172.21.15.63:34702) 2024-05-27T07:35:23.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.250+0000 7f68dfe00700 1 -- 172.21.15.63:0/3977135082 learned_addr learned my addr 172.21.15.63:0/3977135082 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:23.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.250+0000 7f68dfe00700 1 -- 172.21.15.63:0/3977135082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0553a70 msgr2=0x55c5c05e77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:23.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.250+0000 7f68dfe00700 1 --2- 172.21.15.63:0/3977135082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0553a70 0x55c5c05e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.250+0000 7f68dfe00700 1 -- 172.21.15.63:0/3977135082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c05f1340 msgr2=0x55c5c05f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:23.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.250+0000 7f68dfe00700 1 --2- 172.21.15.63:0/3977135082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c05f1340 0x55c5c05f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.250+0000 7f68dfe00700 1 -- 172.21.15.63:0/3977135082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c04cd4a0 con 0x55c5c0467ec0 2024-05-27T07:35:23.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.251+0000 7f68dfe00700 1 --2- 172.21.15.63:0/3977135082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05e7d10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f68d000bd90 tx=0x7f68d000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82dca790bdb2bad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:23.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.252+0000 7f68d5600700 1 -- 172.21.15.63:0/3977135082 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d000fe50 con 0x55c5c0467ec0 2024-05-27T07:35:23.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.252+0000 7f68d5600700 1 -- 172.21.15.63:0/3977135082 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68d00118d0 con 0x55c5c0467ec0 2024-05-27T07:35:23.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.252+0000 7f68dcc00700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68c40b2a70 0x7f68c40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.253+0000 7f68dcc00700 1 -- 172.21.15.63:0/236557565 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f68cc0e1e20 con 0x7f68e010af40 2024-05-27T07:35:23.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.253+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/3977135082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 msgr2=0x55c5c05e7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:23.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.253+0000 7f68d5600700 1 -- 172.21.15.63:0/3977135082 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0010750 con 0x55c5c0467ec0 2024-05-27T07:35:23.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.253+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/3977135082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05e7d10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f68d000bd90 tx=0x7f68d000f640 comp rx=0 tx=0).stop 2024-05-27T07:35:23.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.253+0000 7f68e5000700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68c40b2a70 0x7f68c40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.254+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/3977135082 shutdown_connections 2024-05-27T07:35:23.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.254+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/3977135082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c05f1340 0x55c5c05f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.254+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/3977135082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0553a70 0x55c5c05e77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.254+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/3977135082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05e7d10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.254+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/3977135082 >> 172.21.15.63:0/3977135082 conn(0x55c5c0501410 msgr2=0x55c5c05e2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:23.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.254+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/3977135082 shutdown_connections 2024-05-27T07:35:23.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.254+0000 7f68e5000700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68c40b2a70 0x7f68c40b4f80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68d80062a0 tx=0x7f68d8006230 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:23.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.255+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/3977135082 wait complete. 2024-05-27T07:35:23.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.255+0000 7f68f5b7eb80 1 -- start start 2024-05-27T07:35:23.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.256+0000 7f68f5b7eb80 1 Processor -- start 2024-05-27T07:35:23.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.257+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05ee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.257+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0553a70 0x55c5c05ee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.257+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c05f1340 0x55c5c05eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.257+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c5c040b020 con 0x55c5c0467ec0 2024-05-27T07:35:23.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.257+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5c04a57e0 con 0x55c5c05f1340 2024-05-27T07:35:23.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.257+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c5c0604ef0 con 0x55c5c0553a70 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.257+0000 7f68f5b7eb80 10 client.0 _ll_register_callbacks cb 0x55c5c04aecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.257+0000 7f68dfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.257+0000 7f68dfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05ee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34710/0 (socket says 172.21.15.63:34710) 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.257+0000 7f68dfe00700 1 -- 172.21.15.63:0/1580808564 learned_addr learned my addr 172.21.15.63:0/1580808564 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.258+0000 7f68df400700 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0553a70 0x55c5c05ee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.258+0000 7f68e5000700 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c05f1340 0x55c5c05eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.258+0000 7f68dfe00700 1 -- 172.21.15.63:0/1580808564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0553a70 msgr2=0x55c5c05ee7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.258+0000 7f68dfe00700 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0553a70 0x55c5c05ee7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.258+0000 7f68dfe00700 1 -- 172.21.15.63:0/1580808564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c05f1340 msgr2=0x55c5c05eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.258+0000 7f68dfe00700 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c05f1340 0x55c5c05eece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.258+0000 7f68dfe00700 1 -- 172.21.15.63:0/1580808564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c0454950 con 0x55c5c0467ec0 2024-05-27T07:35:23.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.258+0000 7f68dfe00700 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05ee260 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f68d0005950 tx=0x7f68d0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f5a986a6c69eb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:23.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.259+0000 7f68ca000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:23.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.259+0000 7f68ca000700 1 -- 172.21.15.63:0/1580808564 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d002b060 con 0x55c5c0467ec0 2024-05-27T07:35:23.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.259+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/1580808564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d00118d0 con 0x55c5c0467ec0 2024-05-27T07:35:23.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.259+0000 7f68ca000700 1 -- 172.21.15.63:0/1580808564 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f68d0011bc0 con 0x55c5c0467ec0 2024-05-27T07:35:23.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.259+0000 7f68ca000700 1 -- 172.21.15.63:0/1580808564 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68d0032050 con 0x55c5c0467ec0 2024-05-27T07:35:23.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.259+0000 7f68f5b7eb80 10 client.15723 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:23.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.259+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/1580808564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5c04d50e0 con 0x55c5c0467ec0 2024-05-27T07:35:23.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.260+0000 7f68ca000700 1 -- 172.21.15.63:0/1580808564 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68d00118d0 con 0x55c5c0467ec0 2024-05-27T07:35:23.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.260+0000 7f68ca000700 1 -- 172.21.15.63:0/1580808564 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f68d0004af0 con 0x55c5c0467ec0 2024-05-27T07:35:23.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.261+0000 7f68f5b7eb80 10 client.15723 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:23.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.261+0000 7f68f5b7eb80 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:23.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.261+0000 7f68f5b7eb80 20 client.15723 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:23.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.261+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c5c0611b70 0x55c5c0613f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.261+0000 7f68f5b7eb80 4 client.15723 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:23.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.261+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/1580808564 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5c0516e10 con 0x55c5c0611b70 2024-05-27T07:35:23.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.262+0000 7f68df400700 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c5c0611b70 0x55c5c0613f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.263+0000 7f68df400700 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c5c0611b70 0x55c5c0613f90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4249d96af39ba2e0 server_cookie=b15e6b3216285594 in_seq=0 out_seq=0 2024-05-27T07:35:23.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.263+0000 7f68ca000700 10 client.15723 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:23.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.265+0000 7f68ca000700 1 -- 172.21.15.63:0/1580808564 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c5c0516e10 con 0x55c5c0611b70 2024-05-27T07:35:23.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.265+0000 7f68ca000700 10 client.15723 handle_command_reply: tid=0 2024-05-27T07:35:23.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.265+0000 7f68f5b7eb80 1 client.15723 shutdown 2024-05-27T07:35:23.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.266+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/1580808564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 msgr2=0x55c5c05ee260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:23.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.266+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05ee260 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f68d0005950 tx=0x7f68d0011720 comp rx=0 tx=0).stop 2024-05-27T07:35:23.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.266+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/1580808564 shutdown_connections 2024-05-27T07:35:23.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.266+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c5c0611b70 0x55c5c0613f90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.266+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c05f1340 0x55c5c05eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.266+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0467ec0 0x55c5c05ee260 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.266+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/1580808564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0553a70 0x55c5c05ee7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.266+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/1580808564 >> 172.21.15.63:0/1580808564 conn(0x55c5c0501410 msgr2=0x55c5c05e7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:23.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.267+0000 7f68ca000700 0 client.15723 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:23.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.267+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/1580808564 shutdown_connections 2024-05-27T07:35:23.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.267+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/1580808564 wait complete. 2024-05-27T07:35:23.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.268+0000 7f68f5b7eb80 20 client.15723 trim_cache size 0 max 16384 2024-05-27T07:35:23.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.304+0000 7f68f5b7eb80 1 Processor -- start 2024-05-27T07:35:23.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.304+0000 7f68f5b7eb80 1 -- start start 2024-05-27T07:35:23.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.305+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0467ec0 0x55c5c060e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.305+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0553a70 0x55c5c0616ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.305+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c05f1340 0x55c5c06170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.305+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68e01ac4d0 con 0x55c5c05f1340 2024-05-27T07:35:23.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.305+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68e01a9d60 con 0x55c5c0553a70 2024-05-27T07:35:23.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.305+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0073bd0 con 0x55c5c0467ec0 2024-05-27T07:35:23.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.306+0000 7f68e5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c05f1340 0x55c5c06170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.306+0000 7f68e5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c05f1340 0x55c5c06170e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34726/0 (socket says 172.21.15.63:34726) 2024-05-27T07:35:23.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.306+0000 7f68e5000700 1 -- 172.21.15.63:0/135445744 learned_addr learned my addr 172.21.15.63:0/135445744 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:23.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.306+0000 7f68dfe00700 1 --2- 172.21.15.63:0/135445744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0467ec0 0x55c5c060e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.306+0000 7f68df400700 1 --2- 172.21.15.63:0/135445744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0553a70 0x55c5c0616ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.306+0000 7f68e5000700 1 -- 172.21.15.63:0/135445744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0467ec0 msgr2=0x55c5c060e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:23.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.306+0000 7f68e5000700 1 --2- 172.21.15.63:0/135445744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0467ec0 0x55c5c060e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.306+0000 7f68e5000700 1 -- 172.21.15.63:0/135445744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0553a70 msgr2=0x55c5c0616ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:23.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.306+0000 7f68e5000700 1 --2- 172.21.15.63:0/135445744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0553a70 0x55c5c0616ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.306+0000 7f68e5000700 1 -- 172.21.15.63:0/135445744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c05e8580 con 0x55c5c05f1340 2024-05-27T07:35:23.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.307+0000 7f68e5000700 1 --2- 172.21.15.63:0/135445744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c05f1340 0x55c5c06170e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009e60 tx=0x7f68d801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6da96579621faba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:23.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.307+0000 7f68d4c00700 1 -- 172.21.15.63:0/135445744 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d802e070 con 0x55c5c05f1340 2024-05-27T07:35:23.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.307+0000 7f68d4c00700 1 -- 172.21.15.63:0/135445744 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68d8007d00 con 0x55c5c05f1340 2024-05-27T07:35:23.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.307+0000 7f68d4c00700 1 -- 172.21.15.63:0/135445744 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d802a660 con 0x55c5c05f1340 2024-05-27T07:35:23.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.308+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/135445744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c05f1340 msgr2=0x55c5c06170e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:23.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.308+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/135445744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c05f1340 0x55c5c06170e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009e60 tx=0x7f68d801c000 comp rx=0 tx=0).stop 2024-05-27T07:35:23.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.309+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/135445744 shutdown_connections 2024-05-27T07:35:23.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.309+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/135445744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c05f1340 0x55c5c06170e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.309+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/135445744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c0467ec0 0x55c5c060e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.309+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/135445744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0553a70 0x55c5c0616ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.309+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/135445744 >> 172.21.15.63:0/135445744 conn(0x55c5c0501410 msgr2=0x55c5c05e6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:23.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.309+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/135445744 shutdown_connections 2024-05-27T07:35:23.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.309+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/135445744 wait complete. 2024-05-27T07:35:23.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.309+0000 7f68f5b7eb80 1 -- start start 2024-05-27T07:35:23.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.310+0000 7f68f5b7eb80 1 Processor -- start 2024-05-27T07:35:23.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.310+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0467ec0 0x55c5c06151f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.310+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0553a70 0x55c5c0615730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.310+0000 7f68f5b7eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c05f1340 0x55c5c067dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.310+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0074240 con 0x55c5c0553a70 2024-05-27T07:35:23.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.310+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68e00756b0 con 0x55c5c0467ec0 2024-05-27T07:35:23.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.310+0000 7f68f5b7eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68e0074910 con 0x55c5c05f1340 2024-05-27T07:35:23.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.310+0000 7f68f5b7eb80 10 client.0 _ll_register_callbacks cb 0x55c5c04aecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:23.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.311+0000 7f68df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0553a70 0x55c5c0615730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.311+0000 7f68df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0553a70 0x55c5c0615730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34732/0 (socket says 172.21.15.63:34732) 2024-05-27T07:35:23.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.311+0000 7f68df400700 1 -- 172.21.15.63:0/551987873 learned_addr learned my addr 172.21.15.63:0/551987873 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:23.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.311+0000 7f68e5000700 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c05f1340 0x55c5c067dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.311+0000 7f68df400700 1 -- 172.21.15.63:0/551987873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c05f1340 msgr2=0x55c5c067dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:23.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.311+0000 7f68df400700 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c05f1340 0x55c5c067dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.312+0000 7f68df400700 1 -- 172.21.15.63:0/551987873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0467ec0 msgr2=0x55c5c06151f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:23.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.312+0000 7f68dfe00700 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0467ec0 0x55c5c06151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.312+0000 7f68df400700 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0467ec0 0x55c5c06151f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.312+0000 7f68df400700 1 -- 172.21.15.63:0/551987873 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c067e1a0 con 0x55c5c0553a70 2024-05-27T07:35:23.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.312+0000 7f68dfe00700 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0467ec0 0x55c5c06151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:23.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.312+0000 7f68df400700 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0553a70 0x55c5c0615730 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0a4db0 tx=0x7f68cc010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf9458064a2ad3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:23.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.312+0000 7f68ca000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:23.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.312+0000 7f68ca000700 1 -- 172.21.15.63:0/551987873 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc020040 con 0x55c5c0553a70 2024-05-27T07:35:23.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.312+0000 7f68ca000700 1 -- 172.21.15.63:0/551987873 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f68cc0a8f90 con 0x55c5c0553a70 2024-05-27T07:35:23.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.312+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/551987873 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d8007d00 con 0x55c5c0553a70 2024-05-27T07:35:23.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.313+0000 7f68ca000700 1 -- 172.21.15.63:0/551987873 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68cc0ad0e0 con 0x55c5c0553a70 2024-05-27T07:35:23.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.313+0000 7f68f5b7eb80 10 client.15729 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:23.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.313+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/551987873 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5c06854c0 con 0x55c5c0553a70 2024-05-27T07:35:23.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.313+0000 7f68ca000700 1 -- 172.21.15.63:0/551987873 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68d8007d00 con 0x55c5c0553a70 2024-05-27T07:35:23.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.313+0000 7f68ca000700 1 -- 172.21.15.63:0/551987873 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f68cc0a7f90 con 0x55c5c0553a70 2024-05-27T07:35:23.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.314+0000 7f68f5b7eb80 10 client.15729 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:23.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.314+0000 7f68f5b7eb80 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:23.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.314+0000 7f68f5b7eb80 20 client.15729 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:23.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.314+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c5c0685bf0 0x55c5c0688010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:23.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.314+0000 7f68f5b7eb80 4 client.15729 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:35:23.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.314+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/551987873 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c5c05105f0 con 0x55c5c0685bf0 2024-05-27T07:35:23.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.315+0000 7f68dfe00700 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c5c0685bf0 0x55c5c0688010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:23.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.316+0000 7f68dfe00700 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c5c0685bf0 0x55c5c0688010 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8905f85ba9ff93e server_cookie=8c6b18cae4a76742 in_seq=0 out_seq=0 2024-05-27T07:35:23.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.316+0000 7f68ca000700 10 client.15729 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:23.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.318+0000 7f68ca000700 1 -- 172.21.15.63:0/551987873 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c5c05105f0 con 0x55c5c0685bf0 2024-05-27T07:35:23.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.318+0000 7f68ca000700 10 client.15729 handle_command_reply: tid=0 2024-05-27T07:35:23.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.318+0000 7f68f5b7eb80 1 client.15729 shutdown 2024-05-27T07:35:23.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.319+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/551987873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0553a70 msgr2=0x55c5c0615730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:23.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.319+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0553a70 0x55c5c0615730 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0a4db0 tx=0x7f68cc010f30 comp rx=0 tx=0).stop 2024-05-27T07:35:23.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.319+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/551987873 shutdown_connections 2024-05-27T07:35:23.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.319+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c5c0685bf0 0x55c5c0688010 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.320+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5c05f1340 0x55c5c067dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.320+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5c0467ec0 0x55c5c06151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.320+0000 7f68f5b7eb80 1 --2- 172.21.15.63:0/551987873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c5c0553a70 0x55c5c0615730 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.320+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/551987873 >> 172.21.15.63:0/551987873 conn(0x55c5c0501410 msgr2=0x55c5c05e6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:23.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.320+0000 7f68ca000700 0 client.15729 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:23.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.321+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/551987873 shutdown_connections 2024-05-27T07:35:23.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.321+0000 7f68f5b7eb80 1 -- 172.21.15.63:0/551987873 wait complete. 2024-05-27T07:35:23.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.321+0000 7f68f5b7eb80 20 client.15729 trim_cache size 0 max 16384 2024-05-27T07:35:23.316 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:35:23.317 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:35:23.317 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "e4baeb80-c7ef-4eb3-921f-2aed77335a77", 2024-05-27T07:35:23.317 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:35:23.317 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:35:23.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 -- 172.21.15.63:0/236557565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68c40b2a70 msgr2=0x7f68c40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:23.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68c40b2a70 0x7f68c40b4f80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68d80062a0 tx=0x7f68d8006230 comp rx=0 tx=0).stop 2024-05-27T07:35:23.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 -- 172.21.15.63:0/236557565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68e010af40 msgr2=0x7f68e019d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:23.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68e010af40 0x7f68e019d2b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00d980 tx=0x7f68cc00dd40 comp rx=0 tx=0).stop 2024-05-27T07:35:23.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 -- 172.21.15.63:0/236557565 shutdown_connections 2024-05-27T07:35:23.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68c40b2a70 0x7f68c40b4f80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e010b890 0x7f68e01a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68e0106610 0x7f68e019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 --2- 172.21.15.63:0/236557565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68e010af40 0x7f68e019d2b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:23.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 -- 172.21.15.63:0/236557565 >> 172.21.15.63:0/236557565 conn(0x7f68e0102020 msgr2=0x7f68e010a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:23.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 -- 172.21.15.63:0/236557565 shutdown_connections 2024-05-27T07:35:23.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:23.322+0000 7f68e5a00700 1 -- 172.21.15.63:0/236557565 wait complete. 2024-05-27T07:35:23.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:23 smithi086 ceph-mon[34687]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:23.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:23 smithi149 ceph-mon[35408]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:23.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:23 smithi063 ceph-mon[31613]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:24.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:35:24.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:24 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:35:24.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:24 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:24.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:24 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:35:24.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:24 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:24.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:24 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:35:24.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:24 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:24.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:24 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:35:24.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:24 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:24.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:24 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:35:24.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:24 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:24.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:24 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:35:24.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:24 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:24.733 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:25.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:25 smithi086 ceph-mon[34687]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:25.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:25 smithi149 ceph-mon[35408]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:25.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:25 smithi063 ceph-mon[31613]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:26.567 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:26 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:35:26.703 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:26 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:35:26.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:26 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:35:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.268+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3266717874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc810b890 msgr2=0x7fbbc81131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.268+0000 7fbbc7e00700 1 --2- 172.21.15.63:0/3266717874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc810b890 0x7fbbc81131f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000b0d0 tx=0x7fbbc000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.269+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3266717874 shutdown_connections 2024-05-27T07:35:27.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.269+0000 7fbbc7e00700 1 --2- 172.21.15.63:0/3266717874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc810b890 0x7fbbc81131f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.269+0000 7fbbc7e00700 1 --2- 172.21.15.63:0/3266717874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc8106610 0x7fbbc8106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.269+0000 7fbbc7e00700 1 --2- 172.21.15.63:0/3266717874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc810af40 0x7fbbc810b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.269+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3266717874 >> 172.21.15.63:0/3266717874 conn(0x7fbbc8102020 msgr2=0x7fbbc8104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.269+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3266717874 shutdown_connections 2024-05-27T07:35:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.269+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3266717874 wait complete. 2024-05-27T07:35:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.270+0000 7fbbc7e00700 1 Processor -- start 2024-05-27T07:35:27.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.271+0000 7fbbc7e00700 1 -- start start 2024-05-27T07:35:27.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.271+0000 7fbbc7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc8106610 0x7fbbc81105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.271+0000 7fbbc7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc810af40 0x7fbbc8110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.271+0000 7fbbc7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc810b890 0x7fbbc8111050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.271+0000 7fbbc7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81163c0 con 0x7fbbc8106610 2024-05-27T07:35:27.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.271+0000 7fbbc7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8116240 con 0x7fbbc810af40 2024-05-27T07:35:27.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.271+0000 7fbbc7e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8116540 con 0x7fbbc810b890 2024-05-27T07:35:27.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.272+0000 7fbbc6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc8106610 0x7fbbc81105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.272+0000 7fbbc6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc8106610 0x7fbbc81105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34742/0 (socket says 172.21.15.63:34742) 2024-05-27T07:35:27.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.272+0000 7fbbc6a00700 1 -- 172.21.15.63:0/3038274262 learned_addr learned my addr 172.21.15.63:0/3038274262 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:27.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.272+0000 7fbbc6000700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc810af40 0x7fbbc8110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.272+0000 7fbbc7400700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc810b890 0x7fbbc8111050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.272+0000 7fbbc6a00700 1 -- 172.21.15.63:0/3038274262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc810b890 msgr2=0x7fbbc8111050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.273+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc810b890 0x7fbbc8111050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.273+0000 7fbbc6a00700 1 -- 172.21.15.63:0/3038274262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc810af40 msgr2=0x7fbbc8110b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.273+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc810af40 0x7fbbc8110b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.273+0000 7fbbc6a00700 1 -- 172.21.15.63:0/3038274262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc81ad1d0 con 0x7fbbc8106610 2024-05-27T07:35:27.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.273+0000 7fbbc7400700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc810b890 0x7fbbc8111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:27.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.273+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc8106610 0x7fbbc81105d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c9e0 tx=0x7fbbb800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:27.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.274+0000 7fbbcca00700 1 -- 172.21.15.63:0/3038274262 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8007ac0 con 0x7fbbc8106610 2024-05-27T07:35:27.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.274+0000 7fbbcca00700 1 -- 172.21.15.63:0/3038274262 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbb8007c60 con 0x7fbbc8106610 2024-05-27T07:35:27.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.274+0000 7fbbcca00700 1 -- 172.21.15.63:0/3038274262 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8018660 con 0x7fbbc8106610 2024-05-27T07:35:27.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.274+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3038274262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc81ad4c0 con 0x7fbbc8106610 2024-05-27T07:35:27.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.274+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3038274262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc81ad9b0 con 0x7fbbc8106610 2024-05-27T07:35:27.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.281+0000 7fbbdc851b80 1 Processor -- start 2024-05-27T07:35:27.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.281+0000 7fbbcca00700 1 -- 172.21.15.63:0/3038274262 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbbb800f450 con 0x7fbbc8106610 2024-05-27T07:35:27.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.281+0000 7fbbdc851b80 1 -- start start 2024-05-27T07:35:27.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.282+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 0x55adb38f2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.282+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb3772e40 0x55adb38f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.282+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb38fc2c0 0x55adb38fe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.282+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55adb381fe40 con 0x55adb38fc2c0 2024-05-27T07:35:27.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.282+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55adb380f9e0 con 0x55adb385e9f0 2024-05-27T07:35:27.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.282+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55adb3813a20 con 0x55adb3772e40 2024-05-27T07:35:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbc7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 0x55adb38f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbc7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 0x55adb38f2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:50374/0 (socket says 172.21.15.63:50374) 2024-05-27T07:35:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbc7400700 1 -- 172.21.15.63:0/1233123158 learned_addr learned my addr 172.21.15.63:0/1233123158 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbc6000700 1 --2- 172.21.15.63:0/1233123158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb38fc2c0 0x55adb38fe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/1233123158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb3772e40 0x55adb38f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbc7400700 1 -- 172.21.15.63:0/1233123158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb3772e40 msgr2=0x55adb38f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:27.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbc7400700 1 --2- 172.21.15.63:0/1233123158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb3772e40 0x55adb38f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbcca00700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbbac0b2bc0 0x7fbbac0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbc7400700 1 -- 172.21.15.63:0/1233123158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb38fc2c0 msgr2=0x55adb38fe6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:27.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbc7400700 1 --2- 172.21.15.63:0/1233123158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb38fc2c0 0x55adb38fe6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.283+0000 7fbbc7400700 1 -- 172.21.15.63:0/1233123158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adb37d8420 con 0x55adb385e9f0 2024-05-27T07:35:27.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.284+0000 7fbbcca00700 1 -- 172.21.15.63:0/3038274262 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fbbb80e28e0 con 0x7fbbc8106610 2024-05-27T07:35:27.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.284+0000 7fbbc7400700 1 --2- 172.21.15.63:0/1233123158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 0x55adb38f2750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009d40 tx=0x7fbbc0009d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a20ef97c85c996 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:27.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.285+0000 7fbbc7400700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbbac0b2bc0 0x7fbbac0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.285+0000 7fbbbcc00700 1 -- 172.21.15.63:0/1233123158 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc001f030 con 0x55adb385e9f0 2024-05-27T07:35:27.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.285+0000 7fbbbcc00700 1 -- 172.21.15.63:0/1233123158 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbc000bd00 con 0x55adb385e9f0 2024-05-27T07:35:27.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.285+0000 7fbbbcc00700 1 -- 172.21.15.63:0/1233123158 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc00050b0 con 0x55adb385e9f0 2024-05-27T07:35:27.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.286+0000 7fbbdc851b80 1 -- 172.21.15.63:0/1233123158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 msgr2=0x55adb38f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.286+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/1233123158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 0x55adb38f2750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009d40 tx=0x7fbbc0009d70 comp rx=0 tx=0).stop 2024-05-27T07:35:27.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.286+0000 7fbbc7400700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbbac0b2bc0 0x7fbbac0b50d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009900 tx=0x7fbbc00053c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:27.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.286+0000 7fbbdc851b80 1 -- 172.21.15.63:0/1233123158 shutdown_connections 2024-05-27T07:35:27.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.286+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/1233123158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb38fc2c0 0x55adb38fe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.286+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/1233123158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 0x55adb38f2750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.286+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/1233123158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb3772e40 0x55adb38f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.286+0000 7fbbdc851b80 1 -- 172.21.15.63:0/1233123158 >> 172.21.15.63:0/1233123158 conn(0x55adb380c390 msgr2=0x55adb38ed710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:27.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.287+0000 7fbbdc851b80 1 -- 172.21.15.63:0/1233123158 shutdown_connections 2024-05-27T07:35:27.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.287+0000 7fbbdc851b80 1 -- 172.21.15.63:0/1233123158 wait complete. 2024-05-27T07:35:27.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.288+0000 7fbbdc851b80 1 -- start start 2024-05-27T07:35:27.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.288+0000 7fbbdc851b80 1 Processor -- start 2024-05-27T07:35:27.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb3772e40 0x55adb38f9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 0x55adb38f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb38fc2c0 0x55adb38f9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55adb3715fa0 con 0x55adb38fc2c0 2024-05-27T07:35:27.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55adb37b0760 con 0x55adb3772e40 2024-05-27T07:35:27.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55adb38feda0 con 0x55adb385e9f0 2024-05-27T07:35:27.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbdc851b80 10 client.0 _ll_register_callbacks cb 0x55adb37b9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:27.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbc6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb3772e40 0x55adb38f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbc6000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 0x55adb38f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbc6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb3772e40 0x55adb38f9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:50390/0 (socket says 172.21.15.63:50390) 2024-05-27T07:35:27.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.289+0000 7fbbc6a00700 1 -- 172.21.15.63:0/3903072172 learned_addr learned my addr 172.21.15.63:0/3903072172 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:27.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.290+0000 7fbbc7400700 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb38fc2c0 0x55adb38f9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.290+0000 7fbbc6000700 1 -- 172.21.15.63:0/3903072172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb3772e40 msgr2=0x55adb38f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:27.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.291+0000 7fbbc6000700 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb3772e40 0x55adb38f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.291+0000 7fbbc6000700 1 -- 172.21.15.63:0/3903072172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb38fc2c0 msgr2=0x55adb38f9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:27.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.291+0000 7fbbc6000700 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb38fc2c0 0x55adb38f9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.291+0000 7fbbc6000700 1 -- 172.21.15.63:0/3903072172 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adb375f8d0 con 0x55adb385e9f0 2024-05-27T07:35:27.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.291+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb3772e40 0x55adb38f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:35:27.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.292+0000 7fbbc6000700 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 0x55adb38f97d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8105b00 tx=0x7fbbb40129b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=490484e36b5025c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:27.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.292+0000 7fbbb1600700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:35:27.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.292+0000 7fbbb1600700 1 -- 172.21.15.63:0/3903072172 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4010850 con 0x55adb385e9f0 2024-05-27T07:35:27.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.292+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3903072172 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc000bd00 con 0x55adb385e9f0 2024-05-27T07:35:27.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.292+0000 7fbbb1600700 1 -- 172.21.15.63:0/3903072172 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fbbb40109f0 con 0x55adb385e9f0 2024-05-27T07:35:27.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.293+0000 7fbbb1600700 1 -- 172.21.15.63:0/3903072172 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbb4027050 con 0x55adb385e9f0 2024-05-27T07:35:27.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.293+0000 7fbbdc851b80 10 client.25381 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:27.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.293+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3903072172 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55adb37e0060 con 0x55adb385e9f0 2024-05-27T07:35:27.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.293+0000 7fbbb1600700 1 -- 172.21.15.63:0/3903072172 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbc000bd00 con 0x55adb385e9f0 2024-05-27T07:35:27.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.294+0000 7fbbb1600700 1 -- 172.21.15.63:0/3903072172 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbbb401cbf0 con 0x55adb385e9f0 2024-05-27T07:35:27.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.294+0000 7fbbdc851b80 10 client.25381 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:27.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.294+0000 7fbbdc851b80 10 client.25381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:27.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.294+0000 7fbbdc851b80 20 client.25381 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:27.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.294+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55adb390bcc0 0x55adb390e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.294+0000 7fbbdc851b80 4 client.25381 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:27.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.294+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3903072172 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adb3821d90 con 0x55adb390bcc0 2024-05-27T07:35:27.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.295+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55adb390bcc0 0x55adb390e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.296+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55adb390bcc0 0x55adb390e0e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fcb662ac614f24a server_cookie=d12a165aff9e6e58 in_seq=0 out_seq=0 2024-05-27T07:35:27.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.296+0000 7fbbb1600700 10 client.25381 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:27.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbb1600700 1 -- 172.21.15.63:0/3903072172 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55adb3821d90 con 0x55adb390bcc0 2024-05-27T07:35:27.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbb1600700 10 client.25381 handle_command_reply: tid=0 2024-05-27T07:35:27.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbdc851b80 1 client.25381 shutdown 2024-05-27T07:35:27.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3903072172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 msgr2=0x55adb38f97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 0x55adb38f97d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8105b00 tx=0x7fbbb40129b0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3903072172 shutdown_connections 2024-05-27T07:35:27.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55adb390bcc0 0x55adb390e0e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb38fc2c0 0x55adb38f9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb3772e40 0x55adb38f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/3903072172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 0x55adb38f97d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3903072172 >> 172.21.15.63:0/3903072172 conn(0x55adb380c390 msgr2=0x55adb38f2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:27.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.298+0000 7fbbb1600700 0 client.25381 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:27.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.299+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3903072172 shutdown_connections 2024-05-27T07:35:27.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.299+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3903072172 wait complete. 2024-05-27T07:35:27.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.299+0000 7fbbdc851b80 20 client.25381 trim_cache size 0 max 16384 2024-05-27T07:35:27.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.337+0000 7fbbdc851b80 1 Processor -- start 2024-05-27T07:35:27.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.337+0000 7fbbdc851b80 1 -- start start 2024-05-27T07:35:27.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.338+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb3772e40 0x55adb3907e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.338+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 0x55adb39138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.338+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb38fc2c0 0x55adb3913e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.338+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b9050 con 0x55adb3772e40 2024-05-27T07:35:27.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.338+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b68e0 con 0x55adb38fc2c0 2024-05-27T07:35:27.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.338+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b4730 con 0x55adb385e9f0 2024-05-27T07:35:27.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.338+0000 7fbbc6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb3772e40 0x55adb3907e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.339+0000 7fbbc6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb3772e40 0x55adb3907e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34784/0 (socket says 172.21.15.63:34784) 2024-05-27T07:35:27.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.339+0000 7fbbc6a00700 1 -- 172.21.15.63:0/3795428922 learned_addr learned my addr 172.21.15.63:0/3795428922 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:27.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.339+0000 7fbbc6000700 1 --2- 172.21.15.63:0/3795428922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 0x55adb39138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.339+0000 7fbbc6a00700 1 -- 172.21.15.63:0/3795428922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 msgr2=0x55adb39138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:27.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.339+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/3795428922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 0x55adb39138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.339+0000 7fbbc6a00700 1 -- 172.21.15.63:0/3795428922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb38fc2c0 msgr2=0x55adb3913e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:35:27.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.339+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/3795428922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb38fc2c0 0x55adb3913e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.339+0000 7fbbc6a00700 1 -- 172.21.15.63:0/3795428922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adb3909790 con 0x55adb3772e40 2024-05-27T07:35:27.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.339+0000 7fbbc6000700 1 --2- 172.21.15.63:0/3795428922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 0x55adb39138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:27.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.340+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/3795428922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb3772e40 0x55adb3907e40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80a6a60 tx=0x7fbbb80e4cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f1c4c5a794a81e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:27.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.340+0000 7fbbb3e00700 1 -- 172.21.15.63:0/3795428922 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb80a9ef0 con 0x55adb3772e40 2024-05-27T07:35:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.340+0000 7fbbb3e00700 1 -- 172.21.15.63:0/3795428922 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbb80a97f0 con 0x55adb3772e40 2024-05-27T07:35:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.340+0000 7fbbb3e00700 1 -- 172.21.15.63:0/3795428922 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb80a9cb0 con 0x55adb3772e40 2024-05-27T07:35:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.341+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3795428922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb3772e40 msgr2=0x55adb3907e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.341+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/3795428922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb3772e40 0x55adb3907e40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80a6a60 tx=0x7fbbb80e4cb0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.341+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3795428922 shutdown_connections 2024-05-27T07:35:27.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.341+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/3795428922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb38fc2c0 0x55adb3913e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.341+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/3795428922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb3772e40 0x55adb3907e40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.341+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/3795428922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb385e9f0 0x55adb39138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.341+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3795428922 >> 172.21.15.63:0/3795428922 conn(0x55adb380c390 msgr2=0x55adb38ecf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:27.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.341+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3795428922 shutdown_connections 2024-05-27T07:35:27.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.341+0000 7fbbdc851b80 1 -- 172.21.15.63:0/3795428922 wait complete. 2024-05-27T07:35:27.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.342+0000 7fbbdc851b80 1 -- start start 2024-05-27T07:35:27.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.342+0000 7fbbdc851b80 1 Processor -- start 2024-05-27T07:35:27.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.343+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 0x55adb38ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.343+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb38fc2c0 0x55adb3900280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.343+0000 7fbbdc851b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb39007c0 0x55adb3975790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.343+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b4da0 con 0x55adb39007c0 2024-05-27T07:35:27.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.343+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b6210 con 0x55adb385e9f0 2024-05-27T07:35:27.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.343+0000 7fbbdc851b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b5470 con 0x55adb38fc2c0 2024-05-27T07:35:27.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.343+0000 7fbbdc851b80 10 client.0 _ll_register_callbacks cb 0x55adb37b9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:27.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.344+0000 7fbbc7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb39007c0 0x55adb3975790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.344+0000 7fbbc6000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb38fc2c0 0x55adb3900280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.344+0000 7fbbc7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb39007c0 0x55adb3975790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34786/0 (socket says 172.21.15.63:34786) 2024-05-27T07:35:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.344+0000 7fbbc7400700 1 -- 172.21.15.63:0/449557816 learned_addr learned my addr 172.21.15.63:0/449557816 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.344+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 0x55adb38ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.344+0000 7fbbc7400700 1 -- 172.21.15.63:0/449557816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb38fc2c0 msgr2=0x55adb3900280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.344+0000 7fbbc7400700 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb38fc2c0 0x55adb3900280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.344+0000 7fbbc7400700 1 -- 172.21.15.63:0/449557816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 msgr2=0x55adb38ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.344+0000 7fbbc7400700 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 0x55adb38ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.344+0000 7fbbc7400700 1 -- 172.21.15.63:0/449557816 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adb3904c40 con 0x55adb39007c0 2024-05-27T07:35:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.345+0000 7fbbc7400700 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb39007c0 0x55adb3975790 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0000f80 tx=0x7fbbc0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4adcc4d49cbcf7d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:27.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.345+0000 7fbbb1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:27.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.345+0000 7fbbb1600700 1 -- 172.21.15.63:0/449557816 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc001f040 con 0x55adb39007c0 2024-05-27T07:35:27.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.346+0000 7fbbb1600700 1 -- 172.21.15.63:0/449557816 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fbbc0004c60 con 0x55adb39007c0 2024-05-27T07:35:27.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.346+0000 7fbbdc851b80 1 -- 172.21.15.63:0/449557816 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb80a97f0 con 0x55adb39007c0 2024-05-27T07:35:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.346+0000 7fbbb1600700 1 -- 172.21.15.63:0/449557816 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbc0040050 con 0x55adb39007c0 2024-05-27T07:35:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.346+0000 7fbbdc851b80 10 client.15747 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.346+0000 7fbbdc851b80 1 -- 172.21.15.63:0/449557816 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55adb3976d70 con 0x55adb39007c0 2024-05-27T07:35:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.346+0000 7fbbb1600700 1 -- 172.21.15.63:0/449557816 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbb80a97f0 con 0x55adb39007c0 2024-05-27T07:35:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.347+0000 7fbbb1600700 1 -- 172.21.15.63:0/449557816 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbbc0011070 con 0x55adb39007c0 2024-05-27T07:35:27.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.347+0000 7fbbdc851b80 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.347+0000 7fbbdc851b80 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.347+0000 7fbbdc851b80 20 client.15747 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.347+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55adb3977050 0x55adb3979470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.347+0000 7fbbdc851b80 4 client.15747 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:35:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.347+0000 7fbbdc851b80 1 -- 172.21.15.63:0/449557816 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55adb381b570 con 0x55adb3977050 2024-05-27T07:35:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.348+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55adb3977050 0x55adb3979470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:27.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.348+0000 7fbbc6a00700 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55adb3977050 0x55adb3979470 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e22e3471080a5 server_cookie=30612cc736017b72 in_seq=0 out_seq=0 2024-05-27T07:35:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.348+0000 7fbbb1600700 10 client.15747 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbb1600700 1 -- 172.21.15.63:0/449557816 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55adb381b570 con 0x55adb3977050 2024-05-27T07:35:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbb1600700 10 client.15747 handle_command_reply: tid=0 2024-05-27T07:35:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbdc851b80 1 client.15747 shutdown 2024-05-27T07:35:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbdc851b80 1 -- 172.21.15.63:0/449557816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb39007c0 msgr2=0x55adb3975790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb39007c0 0x55adb3975790 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0000f80 tx=0x7fbbc0023000 comp rx=0 tx=0).stop 2024-05-27T07:35:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbdc851b80 1 -- 172.21.15.63:0/449557816 shutdown_connections 2024-05-27T07:35:27.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55adb3977050 0x55adb3979470 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55adb39007c0 0x55adb3975790 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adb385e9f0 0x55adb38ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbdc851b80 1 --2- 172.21.15.63:0/449557816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adb38fc2c0 0x55adb3900280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbdc851b80 1 -- 172.21.15.63:0/449557816 >> 172.21.15.63:0/449557816 conn(0x55adb380c390 msgr2=0x55adb38f48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.349+0000 7fbbb1600700 0 client.15747 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.350+0000 7fbbdc851b80 1 -- 172.21.15.63:0/449557816 shutdown_connections 2024-05-27T07:35:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.350+0000 7fbbdc851b80 1 -- 172.21.15.63:0/449557816 wait complete. 2024-05-27T07:35:27.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.350+0000 7fbbdc851b80 20 client.15747 trim_cache size 0 max 16384 2024-05-27T07:35:27.346 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:35:27.346 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:35:27.346 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:35:27.346 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:35:27.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3038274262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbbac0b2bc0 msgr2=0x7fbbac0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbbac0b2bc0 0x7fbbac0b50d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009900 tx=0x7fbbc00053c0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3038274262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc8106610 msgr2=0x7fbbc81105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc8106610 0x7fbbc81105d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c9e0 tx=0x7fbbb800cda0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3038274262 shutdown_connections 2024-05-27T07:35:27.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbbac0b2bc0 0x7fbbac0b50d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc810b890 0x7fbbc8111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc8106610 0x7fbbc81105d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 --2- 172.21.15.63:0/3038274262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc810af40 0x7fbbc8110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3038274262 >> 172.21.15.63:0/3038274262 conn(0x7fbbc8102020 msgr2=0x7fbbc810a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:27.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3038274262 shutdown_connections 2024-05-27T07:35:27.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:27.351+0000 7fbbc7e00700 1 -- 172.21.15.63:0/3038274262 wait complete. 2024-05-27T07:35:27.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:27 smithi086 ceph-mon[34687]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:35:27.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:27 smithi149 ceph-mon[35408]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:35:27.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:27 smithi063 ceph-mon[31613]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:35:28.322 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:35:28.322 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:35:28.766 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:29.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:29 smithi086 ceph-mon[34687]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:35:29.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:29 smithi149 ceph-mon[35408]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:35:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:29 smithi063 ceph-mon[31613]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-27T07:35:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.447+0000 7f4fada00640 1 -- 172.21.15.63:0/2776809922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fa81027c0 msgr2=0x7f4fa8102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.447+0000 7f4fada00640 1 --2- 172.21.15.63:0/2776809922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fa81027c0 0x7f4fa8102bd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009900 tx=0x7f4f9802a410 comp rx=0 tx=0).stop 2024-05-27T07:35:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.447+0000 7f4fada00640 1 -- 172.21.15.63:0/2776809922 shutdown_connections 2024-05-27T07:35:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.447+0000 7f4fada00640 1 --2- 172.21.15.63:0/2776809922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa8110360 0x7f4fa8112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.447+0000 7f4fada00640 1 --2- 172.21.15.63:0/2776809922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa810da00 0x7f4fa810fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.447+0000 7f4fada00640 1 --2- 172.21.15.63:0/2776809922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fa81027c0 0x7f4fa8102bd0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.447+0000 7f4fada00640 1 -- 172.21.15.63:0/2776809922 >> 172.21.15.63:0/2776809922 conn(0x7f4fa80fe210 msgr2=0x7f4fa8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.447+0000 7f4fada00640 1 -- 172.21.15.63:0/2776809922 shutdown_connections 2024-05-27T07:35:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.447+0000 7f4fada00640 1 -- 172.21.15.63:0/2776809922 wait complete. 2024-05-27T07:35:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.449+0000 7f4fada00640 1 Processor -- start 2024-05-27T07:35:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.449+0000 7f4fada00640 1 -- start start 2024-05-27T07:35:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.450+0000 7f4fada00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fa81027c0 0x7f4fa819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.450+0000 7f4fada00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa810da00 0x7f4fa819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.450+0000 7f4fada00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa8110360 0x7f4fa81a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.450+0000 7f4fada00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8115d20 con 0x7f4fa81027c0 2024-05-27T07:35:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.450+0000 7f4fada00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8115ba0 con 0x7f4fa810da00 2024-05-27T07:35:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.450+0000 7f4fada00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8115ea0 con 0x7f4fa8110360 2024-05-27T07:35:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.450+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fa81027c0 0x7f4fa819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.450+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fa81027c0 0x7f4fa819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44130/0 (socket says 172.21.15.63:44130) 2024-05-27T07:35:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.450+0000 7f4fa7e00640 1 -- 172.21.15.63:0/1069313352 learned_addr learned my addr 172.21.15.63:0/1069313352 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.451+0000 7f4fad000640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa8110360 0x7f4fa81a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.451+0000 7f4fa7400640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa810da00 0x7f4fa819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.451+0000 7f4fa7e00640 1 -- 172.21.15.63:0/1069313352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa8110360 msgr2=0x7f4fa81a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.451+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa8110360 0x7f4fa81a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.451+0000 7f4fa7e00640 1 -- 172.21.15.63:0/1069313352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa810da00 msgr2=0x7f4fa819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.451+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa810da00 0x7f4fa819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.451+0000 7f4fa7e00640 1 -- 172.21.15.63:0/1069313352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa81a1620 con 0x7f4fa81027c0 2024-05-27T07:35:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.452+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fa81027c0 0x7f4fa819c7c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009230 tx=0x7f4f980319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.452+0000 7f4fa4c00640 1 -- 172.21.15.63:0/1069313352 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9803f030 con 0x7f4fa81027c0 2024-05-27T07:35:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.452+0000 7f4fa4c00640 1 -- 172.21.15.63:0/1069313352 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f980043d0 con 0x7f4fa81027c0 2024-05-27T07:35:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.452+0000 7f4fada00640 1 -- 172.21.15.63:0/1069313352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa81a18b0 con 0x7f4fa81027c0 2024-05-27T07:35:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.452+0000 7f4fa4c00640 1 -- 172.21.15.63:0/1069313352 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98035070 con 0x7f4fa81027c0 2024-05-27T07:35:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.452+0000 7f4fada00640 1 -- 172.21.15.63:0/1069313352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa81a1da0 con 0x7f4fa81027c0 2024-05-27T07:35:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.457+0000 7f4fa4c00640 1 -- 172.21.15.63:0/1069313352 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4f98042030 con 0x7f4fa81027c0 2024-05-27T07:35:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.457+0000 7f4fb601e740 1 Processor -- start 2024-05-27T07:35:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.458+0000 7f4fb601e740 1 -- start start 2024-05-27T07:35:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.458+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aeb1b0 0x559013e8da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.458+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e8dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.458+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aca4d0 0x559013e8e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.458+0000 7f4fb601e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559013dfeb90 con 0x559013ac22f0 2024-05-27T07:35:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.458+0000 7f4fb601e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559013dee730 con 0x559013aeb1b0 2024-05-27T07:35:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.458+0000 7f4fb601e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559013df2770 con 0x559013aca4d0 2024-05-27T07:35:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.459+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e8dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.459+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e8dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44132/0 (socket says 172.21.15.63:44132) 2024-05-27T07:35:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.459+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3257625319 learned_addr learned my addr 172.21.15.63:0/3257625319 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.459+0000 7f4fad000640 1 --2- 172.21.15.63:0/3257625319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aeb1b0 0x559013e8da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.459+0000 7f4fa7400640 1 --2- 172.21.15.63:0/3257625319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aca4d0 0x559013e8e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.459+0000 7f4fa4c00640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f8c0b2950 0x7f4f8c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.460+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3257625319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aca4d0 msgr2=0x559013e8e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.460+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/3257625319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aca4d0 0x559013e8e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.460+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3257625319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aeb1b0 msgr2=0x559013e8da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.460+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/3257625319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aeb1b0 0x559013e8da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.460+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3257625319 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559013d878b0 con 0x559013ac22f0 2024-05-27T07:35:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.460+0000 7f4fa4c00640 1 -- 172.21.15.63:0/1069313352 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f4f98102cf0 con 0x7f4fa81027c0 2024-05-27T07:35:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.460+0000 7f4fad000640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f8c0b2950 0x7f4f8c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.460+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/3257625319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e8dfa0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4f98100040 tx=0x7f4f9802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838c7044640d9222 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.461+0000 7f4f9d600640 1 -- 172.21.15.63:0/3257625319 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98104030 con 0x559013ac22f0 2024-05-27T07:35:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.461+0000 7f4f9d600640 1 -- 172.21.15.63:0/3257625319 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f980c9820 con 0x559013ac22f0 2024-05-27T07:35:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.461+0000 7f4f9d600640 1 -- 172.21.15.63:0/3257625319 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980c9480 con 0x559013ac22f0 2024-05-27T07:35:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.462+0000 7f4fb601e740 1 -- 172.21.15.63:0/3257625319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 msgr2=0x559013e8dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.462+0000 7f4fad000640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f8c0b2950 0x7f4f8c0b4e60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0005c40 tx=0x7f4fa0004a40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.462+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3257625319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e8dfa0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4f98100040 tx=0x7f4f9802a410 comp rx=0 tx=0).stop 2024-05-27T07:35:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.462+0000 7f4fb601e740 1 -- 172.21.15.63:0/3257625319 shutdown_connections 2024-05-27T07:35:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.462+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3257625319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aca4d0 0x559013e8e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.462+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3257625319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e8dfa0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.462+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3257625319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aeb1b0 0x559013e8da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.462+0000 7f4fb601e740 1 -- 172.21.15.63:0/3257625319 >> 172.21.15.63:0/3257625319 conn(0x559013adf300 msgr2=0x559013e8a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.462+0000 7f4fb601e740 1 -- 172.21.15.63:0/3257625319 shutdown_connections 2024-05-27T07:35:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.463+0000 7f4fb601e740 1 -- 172.21.15.63:0/3257625319 wait complete. 2024-05-27T07:35:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.463+0000 7f4fb601e740 1 -- start start 2024-05-27T07:35:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.464+0000 7f4fb601e740 1 Processor -- start 2024-05-27T07:35:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e97ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aca4d0 0x559013e980e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aeb1b0 0x559013ea3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fb601e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559013ad4790 con 0x559013ac22f0 2024-05-27T07:35:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fb601e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559013d96bc0 con 0x559013aeb1b0 2024-05-27T07:35:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fb601e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559013dfeb90 con 0x559013aca4d0 2024-05-27T07:35:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fb601e740 10 client.0 _ll_register_callbacks cb 0x559013d74400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e97ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e97ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44148/0 (socket says 172.21.15.63:44148) 2024-05-27T07:35:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3096745194 learned_addr learned my addr 172.21.15.63:0/3096745194 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fa7400640 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aca4d0 0x559013e980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.465+0000 7f4fad000640 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aeb1b0 0x559013ea3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.466+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3096745194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aca4d0 msgr2=0x559013e980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.466+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aca4d0 0x559013e980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.466+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3096745194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aeb1b0 msgr2=0x559013ea3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.466+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aeb1b0 0x559013ea3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.466+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3096745194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559013b03210 con 0x559013ac22f0 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.466+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e97ba0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4f98100180 tx=0x7f4f980c6250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d296742fdf211 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.466+0000 7f4f92000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.467+0000 7f4f92000640 1 -- 172.21.15.63:0/3096745194 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980c9480 con 0x559013ac22f0 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.467+0000 7f4fb601e740 1 -- 172.21.15.63:0/3096745194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f980c9820 con 0x559013ac22f0 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.467+0000 7f4f92000640 1 -- 172.21.15.63:0/3096745194 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f4f980ca170 con 0x559013ac22f0 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.467+0000 7f4f92000640 1 -- 172.21.15.63:0/3096745194 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f980e9050 con 0x559013ac22f0 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.467+0000 7f4fb601e740 10 client.15759 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.467+0000 7f4fb601e740 1 -- 172.21.15.63:0/3096745194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559013bcf620 con 0x559013ac22f0 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.467+0000 7f4f92000640 1 -- 172.21.15.63:0/3096745194 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f980c9820 con 0x559013ac22f0 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.468+0000 7f4f92000640 1 -- 172.21.15.63:0/3096745194 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4f980d2630 con 0x559013ac22f0 2024-05-27T07:35:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.468+0000 7f4fb601e740 10 client.15759 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.468+0000 7f4fb601e740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.468+0000 7f4fb601e740 20 client.15759 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.468+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559013aa9d80 0x559013ea9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.468+0000 7f4fb601e740 4 client.15759 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.468+0000 7f4fb601e740 1 -- 172.21.15.63:0/3096745194 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559013e00ae0 con 0x559013aa9d80 2024-05-27T07:35:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.469+0000 7f4fa7400640 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559013aa9d80 0x559013ea9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.470+0000 7f4f92000640 10 client.15759 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.470+0000 7f4fa7400640 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559013aa9d80 0x559013ea9900 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7e89d8071e22d0 server_cookie=33fba35f4b57d0ba in_seq=0 out_seq=0 2024-05-27T07:35:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4f92000640 1 -- 172.21.15.63:0/3096745194 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559013e00ae0 con 0x559013aa9d80 2024-05-27T07:35:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4f92000640 10 client.15759 handle_command_reply: tid=0 2024-05-27T07:35:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4fb601e740 1 client.15759 shutdown 2024-05-27T07:35:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4fb601e740 1 -- 172.21.15.63:0/3096745194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 msgr2=0x559013e97ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e97ba0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4f98100180 tx=0x7f4f980c6250 comp rx=0 tx=0).stop 2024-05-27T07:35:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4fb601e740 1 -- 172.21.15.63:0/3096745194 shutdown_connections 2024-05-27T07:35:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559013aa9d80 0x559013ea9900 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aeb1b0 0x559013ea3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aca4d0 0x559013e980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3096745194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e97ba0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4f92000640 0 client.15759 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.472+0000 7f4fb601e740 1 -- 172.21.15.63:0/3096745194 >> 172.21.15.63:0/3096745194 conn(0x559013adf300 msgr2=0x559013e8da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.473+0000 7f4fb601e740 1 -- 172.21.15.63:0/3096745194 shutdown_connections 2024-05-27T07:35:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.473+0000 7f4fb601e740 1 -- 172.21.15.63:0/3096745194 wait complete. 2024-05-27T07:35:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.473+0000 7f4fb601e740 20 client.15759 trim_cache size 0 max 16384 2024-05-27T07:35:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.490+0000 7f4fb601e740 1 Processor -- start 2024-05-27T07:35:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.490+0000 7f4fb601e740 1 -- start start 2024-05-27T07:35:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.491+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e9a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.491+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aca4d0 0x559013e97b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.491+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aeb1b0 0x559013e98080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.491+0000 7f4fb601e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81b59a0 con 0x559013ac22f0 2024-05-27T07:35:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.491+0000 7f4fb601e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81b3270 con 0x559013aca4d0 2024-05-27T07:35:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.491+0000 7f4fb601e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81b10c0 con 0x559013aeb1b0 2024-05-27T07:35:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.491+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e9a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.491+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e9a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44150/0 (socket says 172.21.15.63:44150) 2024-05-27T07:35:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.492+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3868637496 learned_addr learned my addr 172.21.15.63:0/3868637496 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.492+0000 7f4fa7400640 1 --2- 172.21.15.63:0/3868637496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aca4d0 0x559013e97b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.492+0000 7f4fad000640 1 --2- 172.21.15.63:0/3868637496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aeb1b0 0x559013e98080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.492+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3868637496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aeb1b0 msgr2=0x559013e98080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.492+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/3868637496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aeb1b0 0x559013e98080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.492+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3868637496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aca4d0 msgr2=0x559013e97b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.492+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/3868637496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aca4d0 0x559013e97b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.492+0000 7f4fa7e00640 1 -- 172.21.15.63:0/3868637496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559013e8b360 con 0x559013ac22f0 2024-05-27T07:35:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.493+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/3868637496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e9a2d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4f98005f10 tx=0x7f4f98100180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86be8d95f58b051e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.493+0000 7f4f9cc00640 1 -- 172.21.15.63:0/3868637496 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98049070 con 0x559013ac22f0 2024-05-27T07:35:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.493+0000 7f4f9cc00640 1 -- 172.21.15.63:0/3868637496 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f980f0020 con 0x559013ac22f0 2024-05-27T07:35:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.493+0000 7f4f9cc00640 1 -- 172.21.15.63:0/3868637496 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98100560 con 0x559013ac22f0 2024-05-27T07:35:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.495+0000 7f4fb601e740 1 -- 172.21.15.63:0/3868637496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 msgr2=0x559013e9a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.495+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3868637496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e9a2d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4f98005f10 tx=0x7f4f98100180 comp rx=0 tx=0).stop 2024-05-27T07:35:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.495+0000 7f4fb601e740 1 -- 172.21.15.63:0/3868637496 shutdown_connections 2024-05-27T07:35:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.495+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3868637496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013aeb1b0 0x559013e98080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.495+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3868637496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aca4d0 0x559013e97b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.495+0000 7f4fb601e740 1 --2- 172.21.15.63:0/3868637496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013ac22f0 0x559013e9a2d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.495+0000 7f4fb601e740 1 -- 172.21.15.63:0/3868637496 >> 172.21.15.63:0/3868637496 conn(0x559013adf300 msgr2=0x559013e97430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.495+0000 7f4fb601e740 1 -- 172.21.15.63:0/3868637496 shutdown_connections 2024-05-27T07:35:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.495+0000 7f4fb601e740 1 -- 172.21.15.63:0/3868637496 wait complete. 2024-05-27T07:35:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.496+0000 7f4fb601e740 1 -- start start 2024-05-27T07:35:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.496+0000 7f4fb601e740 1 Processor -- start 2024-05-27T07:35:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.497+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aca4d0 0x559013e9ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.497+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013aeb1b0 0x559013e9b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.497+0000 7f4fb601e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013a7f1c0 0x559013f0ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.497+0000 7f4fb601e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81b1730 con 0x559013aeb1b0 2024-05-27T07:35:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.497+0000 7f4fb601e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81b2ba0 con 0x559013aca4d0 2024-05-27T07:35:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.497+0000 7f4fb601e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81b1e00 con 0x559013a7f1c0 2024-05-27T07:35:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.497+0000 7f4fb601e740 10 client.0 _ll_register_callbacks cb 0x7f4fa0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.497+0000 7f4fa7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013aeb1b0 0x559013e9b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.497+0000 7f4fa7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013aeb1b0 0x559013e9b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44152/0 (socket says 172.21.15.63:44152) 2024-05-27T07:35:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.497+0000 7f4fa7400640 1 -- 172.21.15.63:0/615869825 learned_addr learned my addr 172.21.15.63:0/615869825 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.498+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aca4d0 0x559013e9ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.498+0000 7f4fad000640 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013a7f1c0 0x559013f0ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.498+0000 7f4fa7400640 1 -- 172.21.15.63:0/615869825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013a7f1c0 msgr2=0x559013f0ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.498+0000 7f4fa7400640 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013a7f1c0 0x559013f0ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.498+0000 7f4fa7400640 1 -- 172.21.15.63:0/615869825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aca4d0 msgr2=0x559013e9ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.498+0000 7f4fa7400640 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aca4d0 0x559013e9ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.498+0000 7f4fa7400640 1 -- 172.21.15.63:0/615869825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559013f10470 con 0x559013aeb1b0 2024-05-27T07:35:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.499+0000 7f4fa7400640 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013aeb1b0 0x559013e9b3c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x559013ea9fe0 tx=0x7f4f940044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2dc453d0b383e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.499+0000 7f4f92000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.499+0000 7f4f92000640 1 -- 172.21.15.63:0/615869825 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94010430 con 0x559013aeb1b0 2024-05-27T07:35:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.499+0000 7f4f92000640 1 -- 172.21.15.63:0/615869825 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f4f940105d0 con 0x559013aeb1b0 2024-05-27T07:35:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.499+0000 7f4fb601e740 1 -- 172.21.15.63:0/615869825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f980f0020 con 0x559013aeb1b0 2024-05-27T07:35:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.500+0000 7f4f92000640 1 -- 172.21.15.63:0/615869825 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f94031050 con 0x559013aeb1b0 2024-05-27T07:35:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.500+0000 7f4fb601e740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.500+0000 7f4fb601e740 1 -- 172.21.15.63:0/615869825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559013f11600 con 0x559013aeb1b0 2024-05-27T07:35:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.500+0000 7f4f92000640 1 -- 172.21.15.63:0/615869825 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f980f0020 con 0x559013aeb1b0 2024-05-27T07:35:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.500+0000 7f4f92000640 1 -- 172.21.15.63:0/615869825 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4f94028750 con 0x559013aeb1b0 2024-05-27T07:35:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.500+0000 7f4fb601e740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.500+0000 7f4fb601e740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.500+0000 7f4fb601e740 20 client.15765 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.500+0000 7f4fb601e740 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559013f11d60 0x559013f14180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.501+0000 7f4fb601e740 4 client.15765 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:35:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.501+0000 7f4fb601e740 1 -- 172.21.15.63:0/615869825 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559013dfa2c0 con 0x559013f11d60 2024-05-27T07:35:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.501+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559013f11d60 0x559013f14180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.502+0000 7f4fa7e00640 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559013f11d60 0x559013f14180 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28d3d23031cd963 server_cookie=4355254888b22c1f in_seq=0 out_seq=0 2024-05-27T07:35:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.502+0000 7f4f92000640 10 client.15765 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.512+0000 7f4f92000640 1 -- 172.21.15.63:0/615869825 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559013dfa2c0 con 0x559013f11d60 2024-05-27T07:35:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.512+0000 7f4f92000640 10 client.15765 handle_command_reply: tid=0 2024-05-27T07:35:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.512+0000 7f4fb601e740 1 client.15765 shutdown 2024-05-27T07:35:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.512+0000 7f4fb601e740 1 -- 172.21.15.63:0/615869825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013aeb1b0 msgr2=0x559013e9b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.512+0000 7f4fb601e740 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013aeb1b0 0x559013e9b3c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x559013ea9fe0 tx=0x7f4f940044d0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.513+0000 7f4fb601e740 1 -- 172.21.15.63:0/615869825 shutdown_connections 2024-05-27T07:35:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.513+0000 7f4fb601e740 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559013f11d60 0x559013f14180 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.513+0000 7f4fb601e740 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559013a7f1c0 0x559013f0ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.513+0000 7f4fb601e740 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559013aeb1b0 0x559013e9b3c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.513+0000 7f4fb601e740 1 --2- 172.21.15.63:0/615869825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559013aca4d0 0x559013e9ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.513+0000 7f4f92000640 0 client.15765 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.513+0000 7f4fb601e740 1 -- 172.21.15.63:0/615869825 >> 172.21.15.63:0/615869825 conn(0x559013adf300 msgr2=0x559013e97e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.513+0000 7f4fb601e740 1 -- 172.21.15.63:0/615869825 shutdown_connections 2024-05-27T07:35:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.513+0000 7f4fb601e740 1 -- 172.21.15.63:0/615869825 wait complete. 2024-05-27T07:35:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.513+0000 7f4fb601e740 20 client.15765 trim_cache size 0 max 16384 2024-05-27T07:35:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:35:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:35:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:35:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 -- 172.21.15.63:0/1069313352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f8c0b2950 msgr2=0x7f4f8c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f8c0b2950 0x7f4f8c0b4e60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0005c40 tx=0x7f4fa0004a40 comp rx=0 tx=0).stop 2024-05-27T07:35:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 -- 172.21.15.63:0/1069313352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fa81027c0 msgr2=0x7f4fa819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fa81027c0 0x7f4fa819c7c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009230 tx=0x7f4f980319a0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 -- 172.21.15.63:0/1069313352 shutdown_connections 2024-05-27T07:35:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f8c0b2950 0x7f4f8c0b4e60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa8110360 0x7f4fa81a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fa810da00 0x7f4fa819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 --2- 172.21.15.63:0/1069313352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fa81027c0 0x7f4fa819c7c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 -- 172.21.15.63:0/1069313352 >> 172.21.15.63:0/1069313352 conn(0x7f4fa80fe210 msgr2=0x7f4fa8103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 -- 172.21.15.63:0/1069313352 shutdown_connections 2024-05-27T07:35:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:31.514+0000 7f4fada00640 1 -- 172.21.15.63:0/1069313352 wait complete. 2024-05-27T07:35:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.555+0000 7fe9fd000700 1 -- 172.21.15.63:0/1155721967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f80764a0 msgr2=0x7fe9f8114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.555+0000 7fe9fd000700 1 --2- 172.21.15.63:0/1155721967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f80764a0 0x7fe9f8114e20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009990 tx=0x7fe9e4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.555+0000 7fe9fd000700 1 -- 172.21.15.63:0/1155721967 shutdown_connections 2024-05-27T07:35:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.555+0000 7fe9fd000700 1 --2- 172.21.15.63:0/1155721967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f8115360 0x7fe9f8117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.555+0000 7fe9fd000700 1 --2- 172.21.15.63:0/1155721967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f8075b50 0x7fe9f8075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.555+0000 7fe9fd000700 1 --2- 172.21.15.63:0/1155721967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f80764a0 0x7fe9f8114e20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.555+0000 7fe9fd000700 1 -- 172.21.15.63:0/1155721967 >> 172.21.15.63:0/1155721967 conn(0x7fe9f8100030 msgr2=0x7fe9f8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.556+0000 7fe9fd000700 1 -- 172.21.15.63:0/1155721967 shutdown_connections 2024-05-27T07:35:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.556+0000 7fe9fd000700 1 -- 172.21.15.63:0/1155721967 wait complete. 2024-05-27T07:35:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.557+0000 7fe9fd000700 1 Processor -- start 2024-05-27T07:35:31.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.557+0000 7fe9fd000700 1 -- start start 2024-05-27T07:35:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.558+0000 7fe9fd000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f8075b50 0x7fe9f81a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.558+0000 7fe9fd000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f80764a0 0x7fe9f81a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.558+0000 7fe9fd000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f8115360 0x7fe9f81a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.558+0000 7fe9fd000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f811a960 con 0x7fe9f8075b50 2024-05-27T07:35:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.558+0000 7fe9fd000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f811a7e0 con 0x7fe9f8115360 2024-05-27T07:35:31.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.558+0000 7fe9fd000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f811aae0 con 0x7fe9f80764a0 2024-05-27T07:35:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.559+0000 7fe9f5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f8075b50 0x7fe9f81a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.559+0000 7fe9f5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f8075b50 0x7fe9f81a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44180/0 (socket says 172.21.15.63:44180) 2024-05-27T07:35:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.559+0000 7fe9f5e00700 1 -- 172.21.15.63:0/3622502848 learned_addr learned my addr 172.21.15.63:0/3622502848 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.559+0000 7fe9f5400700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f80764a0 0x7fe9f81a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.559+0000 7fe9f6800700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f8115360 0x7fe9f81a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.559+0000 7fe9f5e00700 1 -- 172.21.15.63:0/3622502848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f80764a0 msgr2=0x7fe9f81a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.559+0000 7fe9f5e00700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f80764a0 0x7fe9f81a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.559+0000 7fe9f5e00700 1 -- 172.21.15.63:0/3622502848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f8115360 msgr2=0x7fe9f81a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.559+0000 7fe9f5e00700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f8115360 0x7fe9f81a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.559+0000 7fe9f5e00700 1 -- 172.21.15.63:0/3622502848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f81a6240 con 0x7fe9f8075b50 2024-05-27T07:35:31.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.560+0000 7fe9f5e00700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f8075b50 0x7fe9f81a1370 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800d950 tx=0x7fe9e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:31.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.560+0000 7fe9eea00700 1 -- 172.21.15.63:0/3622502848 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8016070 con 0x7fe9f8075b50 2024-05-27T07:35:31.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.560+0000 7fe9eea00700 1 -- 172.21.15.63:0/3622502848 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9e8009dd0 con 0x7fe9f8075b50 2024-05-27T07:35:31.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.560+0000 7fe9eea00700 1 -- 172.21.15.63:0/3622502848 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e800f3c0 con 0x7fe9f8075b50 2024-05-27T07:35:31.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.560+0000 7fe9fd000700 1 -- 172.21.15.63:0/3622502848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f81a6530 con 0x7fe9f8075b50 2024-05-27T07:35:31.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.561+0000 7fe9fd000700 1 -- 172.21.15.63:0/3622502848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f81ade50 con 0x7fe9f8075b50 2024-05-27T07:35:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.562+0000 7fe9fd000700 1 -- 172.21.15.63:0/3622502848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f8077290 con 0x7fe9f8075b50 2024-05-27T07:35:31.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.565+0000 7fe9eea00700 1 -- 172.21.15.63:0/3622502848 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe9e8019070 con 0x7fe9f8075b50 2024-05-27T07:35:31.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.567+0000 7fe9eea00700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9dc0b2a70 0x7fe9dc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:31.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.568+0000 7fe9eea00700 1 -- 172.21.15.63:0/3622502848 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fe9e80e2520 con 0x7fe9f8075b50 2024-05-27T07:35:31.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.568+0000 7fe9f5400700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9dc0b2a70 0x7fe9dc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:31.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.569+0000 7fe9f5400700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9dc0b2a70 0x7fe9dc0b4f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8103610 tx=0x7fe9e4005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:31.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.570+0000 7fe9eea00700 1 -- 172.21.15.63:0/3622502848 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9e80a9b90 con 0x7fe9f8075b50 2024-05-27T07:35:31.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:31 smithi086 ceph-mon[34687]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:35:31.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:31 smithi149 ceph-mon[35408]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:35:31.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:31 smithi063 ceph-mon[31613]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:35:31.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.856+0000 7fe9fd000700 1 -- 172.21.15.63:0/3622502848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9f81a26d0 con 0x7fe9f8075b50 2024-05-27T07:35:31.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.858+0000 7fe9eea00700 1 -- 172.21.15.63:0/3622502848 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11861 (secure 0 0 0) 0x7fe9e80a9490 con 0x7fe9f8075b50 2024-05-27T07:35:31.853 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:35:31.853 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:33:33.871810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/1769339501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1769339501},{"type":"v1","addr":"172.21.15.86:6833","nonce":1769339501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:35:31.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.860+0000 7fe9fd000700 1 -- 172.21.15.63:0/3622502848 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9dc0b2a70 msgr2=0x7fe9dc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.860+0000 7fe9fd000700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9dc0b2a70 0x7fe9dc0b4f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8103610 tx=0x7fe9e4005fd0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.860+0000 7fe9fd000700 1 -- 172.21.15.63:0/3622502848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f8075b50 msgr2=0x7fe9f81a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:31.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.860+0000 7fe9fd000700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f8075b50 0x7fe9f81a1370 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800d950 tx=0x7fe9e800dc60 comp rx=0 tx=0).stop 2024-05-27T07:35:31.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.860+0000 7fe9fd000700 1 -- 172.21.15.63:0/3622502848 shutdown_connections 2024-05-27T07:35:31.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.860+0000 7fe9fd000700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9dc0b2a70 0x7fe9dc0b4f80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.860+0000 7fe9fd000700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f8115360 0x7fe9f81a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.860+0000 7fe9fd000700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f8075b50 0x7fe9f81a1370 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.860+0000 7fe9fd000700 1 --2- 172.21.15.63:0/3622502848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9f80764a0 0x7fe9f81a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:31.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.860+0000 7fe9fd000700 1 -- 172.21.15.63:0/3622502848 >> 172.21.15.63:0/3622502848 conn(0x7fe9f8100030 msgr2=0x7fe9f8100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:31.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.861+0000 7fe9fd000700 1 -- 172.21.15.63:0/3622502848 shutdown_connections 2024-05-27T07:35:31.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:31.861+0000 7fe9fd000700 1 -- 172.21.15.63:0/3622502848 wait complete. 2024-05-27T07:35:31.857 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 27 2024-05-27T07:35:32.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:32 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3622502848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:35:32.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:32 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3622502848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:35:32.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:32 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3622502848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:35:32.876 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:35:33.329 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:33.442 INFO:tasks.workunit.client.0.smithi063.stderr:Updating files: 55% (6760/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 90% (11114/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-27T07:35:33.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:33 smithi063 ceph-mon[31613]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:35:33.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:33 smithi086 ceph-mon[34687]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:35:33.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:33 smithi149 ceph-mon[35408]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:35:34.398 INFO:tasks.workunit.client.0.smithi063.stderr:Note: switching to '38f9d900c83c63799fbdbe61acc9a11b0d3554a6'. 2024-05-27T07:35:34.398 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-27T07:35:34.398 INFO:tasks.workunit.client.0.smithi063.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-27T07:35:34.398 INFO:tasks.workunit.client.0.smithi063.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-27T07:35:34.398 INFO:tasks.workunit.client.0.smithi063.stderr:state without impacting any branches by switching back to a branch. 2024-05-27T07:35:34.398 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-27T07:35:34.398 INFO:tasks.workunit.client.0.smithi063.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-27T07:35:34.398 INFO:tasks.workunit.client.0.smithi063.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-27T07:35:34.398 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-27T07:35:34.399 INFO:tasks.workunit.client.0.smithi063.stderr: git switch -c 2024-05-27T07:35:34.399 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-27T07:35:34.399 INFO:tasks.workunit.client.0.smithi063.stderr:Or undo this operation with: 2024-05-27T07:35:34.399 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-27T07:35:34.399 INFO:tasks.workunit.client.0.smithi063.stderr: git switch - 2024-05-27T07:35:34.399 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-27T07:35:34.399 INFO:tasks.workunit.client.0.smithi063.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-27T07:35:34.399 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-27T07:35:34.399 INFO:tasks.workunit.client.0.smithi063.stderr:HEAD is now at 38f9d900c83 Merge PR #55758 into wip-vshankar-testing-20240521.063558-debug 2024-05-27T07:35:34.413 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-27T07:35:34.493 INFO:tasks.workunit.client.0.smithi063.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-27T07:35:34.501 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T07:35:34.501 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-27T07:35:35.505 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:35 smithi063 ceph-mon[31613]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:35:35.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:35 smithi086 ceph-mon[34687]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:35:35.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:35 smithi149 ceph-mon[35408]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-27T07:35:35.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.941+0000 7fb903400700 1 -- 172.21.15.63:0/366440201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8fc109860 msgr2=0x7fb8fc1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:35.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.941+0000 7fb903400700 1 --2- 172.21.15.63:0/366440201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8fc109860 0x7fb8fc1111c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b0d0 tx=0x7fb8f400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.942+0000 7fb903400700 1 -- 172.21.15.63:0/366440201 shutdown_connections 2024-05-27T07:35:35.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.942+0000 7fb903400700 1 --2- 172.21.15.63:0/366440201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8fc109860 0x7fb8fc1111c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.942+0000 7fb903400700 1 --2- 172.21.15.63:0/366440201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8fc1045e0 0x7fb8fc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.942+0000 7fb903400700 1 --2- 172.21.15.63:0/366440201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8fc108f10 0x7fb8fc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.942+0000 7fb903400700 1 -- 172.21.15.63:0/366440201 >> 172.21.15.63:0/366440201 conn(0x7fb8fc100030 msgr2=0x7fb8fc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:35.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.942+0000 7fb903400700 1 -- 172.21.15.63:0/366440201 shutdown_connections 2024-05-27T07:35:35.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.942+0000 7fb903400700 1 -- 172.21.15.63:0/366440201 wait complete. 2024-05-27T07:35:35.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.943+0000 7fb903400700 1 Processor -- start 2024-05-27T07:35:35.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.943+0000 7fb903400700 1 -- start start 2024-05-27T07:35:35.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb903400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8fc1045e0 0x7fb8fc19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb903400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8fc108f10 0x7fb8fc19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb903400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8fc109860 0x7fb8fc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb903400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc114340 con 0x7fb8fc109860 2024-05-27T07:35:35.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb903400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1141c0 con 0x7fb8fc1045e0 2024-05-27T07:35:35.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb903400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1144c0 con 0x7fb8fc108f10 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb901400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8fc109860 0x7fb8fc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb901400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8fc109860 0x7fb8fc19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44208/0 (socket says 172.21.15.63:44208) 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb901400700 1 -- 172.21.15.63:0/2169010340 learned_addr learned my addr 172.21.15.63:0/2169010340 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb900a00700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8fc1045e0 0x7fb8fc19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.944+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8fc108f10 0x7fb8fc19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.945+0000 7fb901400700 1 -- 172.21.15.63:0/2169010340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8fc108f10 msgr2=0x7fb8fc19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.945+0000 7fb901400700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8fc108f10 0x7fb8fc19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.945+0000 7fb901400700 1 -- 172.21.15.63:0/2169010340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8fc1045e0 msgr2=0x7fb8fc19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.945+0000 7fb901400700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8fc1045e0 0x7fb8fc19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.945+0000 7fb901400700 1 -- 172.21.15.63:0/2169010340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc19fd20 con 0x7fb8fc109860 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.945+0000 7fb900a00700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8fc1045e0 0x7fb8fc19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:35.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.945+0000 7fb901400700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8fc109860 0x7fb8fc19f680 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009850 tx=0x7fb8f4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:35.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.945+0000 7fb8f9600700 1 -- 172.21.15.63:0/2169010340 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f401f070 con 0x7fb8fc109860 2024-05-27T07:35:35.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.946+0000 7fb8f9600700 1 -- 172.21.15.63:0/2169010340 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f4005500 con 0x7fb8fc109860 2024-05-27T07:35:35.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.946+0000 7fb903400700 1 -- 172.21.15.63:0/2169010340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8fc19ffb0 con 0x7fb8fc109860 2024-05-27T07:35:35.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.946+0000 7fb8f9600700 1 -- 172.21.15.63:0/2169010340 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f401b9a0 con 0x7fb8fc109860 2024-05-27T07:35:35.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.946+0000 7fb903400700 1 -- 172.21.15.63:0/2169010340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8fc1a0490 con 0x7fb8fc109860 2024-05-27T07:35:35.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.950+0000 7fb8f9600700 1 -- 172.21.15.63:0/2169010340 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb8f4022070 con 0x7fb8fc109860 2024-05-27T07:35:35.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.951+0000 7fb911ef8b80 1 Processor -- start 2024-05-27T07:35:35.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.951+0000 7fb911ef8b80 1 -- start start 2024-05-27T07:35:35.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.951+0000 7fb8f9600700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e40b29f0 0x7fb8e40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeeccb9f0 0x55bbeed5e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 0x55bbeed5e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb8f9600700 1 -- 172.21.15.63:0/2169010340 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb8f400f020 con 0x7fb8fc109860 2024-05-27T07:35:35.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeed5ee70 0x55bbeed6b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bbeec8ce40 con 0x55bbeed5ee70 2024-05-27T07:35:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bbeec7c9e0 con 0x55bbeebdfe40 2024-05-27T07:35:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bbeec80a20 con 0x55bbeeccb9f0 2024-05-27T07:35:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e40b29f0 0x7fb8e40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb901400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeed5ee70 0x55bbeed6b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb901400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeed5ee70 0x55bbeed6b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44224/0 (socket says 172.21.15.63:44224) 2024-05-27T07:35:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb901400700 1 -- 172.21.15.63:0/1583704668 learned_addr learned my addr 172.21.15.63:0/1583704668 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb900a00700 1 --2- 172.21.15.63:0/1583704668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeeccb9f0 0x55bbeed5e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.952+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/1583704668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 0x55bbeed5e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.953+0000 7fb901400700 1 -- 172.21.15.63:0/1583704668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeeccb9f0 msgr2=0x55bbeed5e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.953+0000 7fb901400700 1 --2- 172.21.15.63:0/1583704668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeeccb9f0 0x55bbeed5e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.953+0000 7fb901400700 1 -- 172.21.15.63:0/1583704668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 msgr2=0x55bbeed5e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.953+0000 7fb901400700 1 --2- 172.21.15.63:0/1583704668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 0x55bbeed5e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.953+0000 7fb901400700 1 -- 172.21.15.63:0/1583704668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbeec45420 con 0x55bbeed5ee70 2024-05-27T07:35:35.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.953+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/1583704668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 0x55bbeed5e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:35.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.953+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e40b29f0 0x7fb8e40b4f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800b6c0 tx=0x7fb8e8012890 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:35.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.953+0000 7fb901400700 1 --2- 172.21.15.63:0/1583704668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeed5ee70 0x55bbeed6b110 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4006090 tx=0x7fb8f40aa560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb340f23a1062ef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:35.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.954+0000 7fb8ee000700 1 -- 172.21.15.63:0/1583704668 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40aadf0 con 0x55bbeed5ee70 2024-05-27T07:35:35.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.954+0000 7fb8ee000700 1 -- 172.21.15.63:0/1583704668 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f40af1a0 con 0x55bbeed5ee70 2024-05-27T07:35:35.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.954+0000 7fb8ee000700 1 -- 172.21.15.63:0/1583704668 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40235b0 con 0x55bbeed5ee70 2024-05-27T07:35:35.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.955+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1583704668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeed5ee70 msgr2=0x55bbeed6b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:35.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.955+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1583704668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeed5ee70 0x55bbeed6b110 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4006090 tx=0x7fb8f40aa560 comp rx=0 tx=0).stop 2024-05-27T07:35:35.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.955+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1583704668 shutdown_connections 2024-05-27T07:35:35.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.955+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1583704668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeed5ee70 0x55bbeed6b110 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.955+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1583704668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeeccb9f0 0x55bbeed5e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.955+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1583704668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 0x55bbeed5e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.955+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1583704668 >> 172.21.15.63:0/1583704668 conn(0x55bbeec79390 msgr2=0x55bbeed5a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:35.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.955+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1583704668 shutdown_connections 2024-05-27T07:35:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.955+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1583704668 wait complete. 2024-05-27T07:35:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.956+0000 7fb911ef8b80 1 -- start start 2024-05-27T07:35:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.956+0000 7fb911ef8b80 1 Processor -- start 2024-05-27T07:35:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.956+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeebdfe40 0x55bbeed68660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.956+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed68ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.956+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 0x55bbeed690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.956+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bbeeb82fa0 con 0x55bbeeccb9f0 2024-05-27T07:35:35.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.956+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bbeec1d760 con 0x55bbeed5ee70 2024-05-27T07:35:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.956+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bbeed6b940 con 0x55bbeebdfe40 2024-05-27T07:35:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb911ef8b80 10 client.0 _ll_register_callbacks cb 0x55bbeec26c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb8fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed68ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb8fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed68ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44240/0 (socket says 172.21.15.63:44240) 2024-05-27T07:35:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb8fbe00700 1 -- 172.21.15.63:0/1415411120 learned_addr learned my addr 172.21.15.63:0/1415411120 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb901400700 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 0x55bbeed690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb900a00700 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeebdfe40 0x55bbeed68660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb8fbe00700 1 -- 172.21.15.63:0/1415411120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeebdfe40 msgr2=0x55bbeed68660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeebdfe40 0x55bbeed68660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb8fbe00700 1 -- 172.21.15.63:0/1415411120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 msgr2=0x55bbeed690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 0x55bbeed690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.957+0000 7fb8fbe00700 1 -- 172.21.15.63:0/1415411120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbeebcc8d0 con 0x55bbeeccb9f0 2024-05-27T07:35:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.958+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed68ba0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800f710 tx=0x7fb8e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1ce77fded2c1b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.958+0000 7fb8e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.958+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1415411120 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8020810 con 0x55bbeeccb9f0 2024-05-27T07:35:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.958+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1415411120 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb8e80209b0 con 0x55bbeeccb9f0 2024-05-27T07:35:35.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.958+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1415411120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f40af1a0 con 0x55bbeeccb9f0 2024-05-27T07:35:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.958+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1415411120 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8e8020bb0 con 0x55bbeeccb9f0 2024-05-27T07:35:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.958+0000 7fb911ef8b80 10 client.15783 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.959+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1415411120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bbeec4d060 con 0x55bbeeccb9f0 2024-05-27T07:35:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.959+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1415411120 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f40af1a0 con 0x55bbeeccb9f0 2024-05-27T07:35:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.959+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1415411120 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8e8028c30 con 0x55bbeeccb9f0 2024-05-27T07:35:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.959+0000 7fb911ef8b80 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.959+0000 7fb911ef8b80 10 client.15783 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:35:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.959+0000 7fb911ef8b80 20 client.15783 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.959+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bbeed78840 0x55bbeed7ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.959+0000 7fb911ef8b80 4 client.15783 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:35.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.959+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1415411120 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbeec8ed90 con 0x55bbeed78840 2024-05-27T07:35:35.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.960+0000 7fb900a00700 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bbeed78840 0x55bbeed7ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.961+0000 7fb900a00700 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bbeed78840 0x55bbeed7ac80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c399e4774a684886 server_cookie=e4beb7c1b72cdf34 in_seq=0 out_seq=0 2024-05-27T07:35:35.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.961+0000 7fb8e2a00700 10 client.15783 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:35:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.963+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1415411120 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bbeec8ed90 con 0x55bbeed78840 2024-05-27T07:35:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.963+0000 7fb8e2a00700 10 client.15783 handle_command_reply: tid=0 2024-05-27T07:35:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.963+0000 7fb911ef8b80 1 client.15783 shutdown 2024-05-27T07:35:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.963+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1415411120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 msgr2=0x55bbeed68ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.963+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed68ba0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800f710 tx=0x7fb8e80047c0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.964+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1415411120 shutdown_connections 2024-05-27T07:35:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.964+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 0x55bbeed690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.964+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeebdfe40 0x55bbeed68660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.964+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bbeed78840 0x55bbeed7ac80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.964+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1415411120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed68ba0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.964+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1415411120 >> 172.21.15.63:0/1415411120 conn(0x55bbeec79390 msgr2=0x55bbeed5e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:35.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.964+0000 7fb8e2a00700 0 client.15783 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:35:35.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.964+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1415411120 shutdown_connections 2024-05-27T07:35:35.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.964+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1415411120 wait complete. 2024-05-27T07:35:35.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.964+0000 7fb911ef8b80 20 client.15783 trim_cache size 0 max 16384 2024-05-27T07:35:35.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.981+0000 7fb911ef8b80 1 Processor -- start 2024-05-27T07:35:35.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.981+0000 7fb911ef8b80 1 -- start start 2024-05-27T07:35:35.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeebdfe40 0x55bbeed63e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed675c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 0x55bbeed5e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1b2c80 con 0x55bbeeccb9f0 2024-05-27T07:35:35.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1b0510 con 0x55bbeed5ee70 2024-05-27T07:35:35.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1ae360 con 0x55bbeebdfe40 2024-05-27T07:35:35.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb8fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb8fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed675c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44248/0 (socket says 172.21.15.63:44248) 2024-05-27T07:35:35.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb8fbe00700 1 -- 172.21.15.63:0/3935050995 learned_addr learned my addr 172.21.15.63:0/3935050995 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:35.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb900a00700 1 --2- 172.21.15.63:0/3935050995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeebdfe40 0x55bbeed63e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.982+0000 7fb901400700 1 --2- 172.21.15.63:0/3935050995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 0x55bbeed5e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.983+0000 7fb8fbe00700 1 -- 172.21.15.63:0/3935050995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeebdfe40 msgr2=0x55bbeed63e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:35.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.983+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/3935050995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeebdfe40 0x55bbeed63e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.983+0000 7fb8fbe00700 1 -- 172.21.15.63:0/3935050995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 msgr2=0x55bbeed5e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:35.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.983+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/3935050995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 0x55bbeed5e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.983+0000 7fb8fbe00700 1 -- 172.21.15.63:0/3935050995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbeed69450 con 0x55bbeeccb9f0 2024-05-27T07:35:35.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.983+0000 7fb901400700 1 --2- 172.21.15.63:0/3935050995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 0x55bbeed5e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:35.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.983+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/3935050995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed675c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800bb70 tx=0x7fb8e800fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e0130fa488156a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:35.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.983+0000 7fb8ed600700 1 -- 172.21.15.63:0/3935050995 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8039020 con 0x55bbeeccb9f0 2024-05-27T07:35:35.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.983+0000 7fb8ed600700 1 -- 172.21.15.63:0/3935050995 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8e8020810 con 0x55bbeeccb9f0 2024-05-27T07:35:35.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.983+0000 7fb8ed600700 1 -- 172.21.15.63:0/3935050995 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8035870 con 0x55bbeeccb9f0 2024-05-27T07:35:35.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.984+0000 7fb911ef8b80 1 -- 172.21.15.63:0/3935050995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 msgr2=0x55bbeed675c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:35.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.984+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/3935050995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed675c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800bb70 tx=0x7fb8e800fe60 comp rx=0 tx=0).stop 2024-05-27T07:35:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.984+0000 7fb911ef8b80 1 -- 172.21.15.63:0/3935050995 shutdown_connections 2024-05-27T07:35:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.984+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/3935050995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeed5ee70 0x55bbeed5e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.984+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/3935050995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeebdfe40 0x55bbeed63e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.984+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/3935050995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed675c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.984+0000 7fb911ef8b80 1 -- 172.21.15.63:0/3935050995 >> 172.21.15.63:0/3935050995 conn(0x55bbeec79390 msgr2=0x55bbeed5e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.985+0000 7fb911ef8b80 1 -- 172.21.15.63:0/3935050995 shutdown_connections 2024-05-27T07:35:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.985+0000 7fb911ef8b80 1 -- 172.21.15.63:0/3935050995 wait complete. 2024-05-27T07:35:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.985+0000 7fb911ef8b80 1 -- start start 2024-05-27T07:35:35.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.985+0000 7fb911ef8b80 1 Processor -- start 2024-05-27T07:35:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.986+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 0x55bbeed62560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.986+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed62aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.986+0000 7fb911ef8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeed5ee70 0x55bbeed62fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.986+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1ae9d0 con 0x55bbeeccb9f0 2024-05-27T07:35:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.986+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1afe40 con 0x55bbeebdfe40 2024-05-27T07:35:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.986+0000 7fb911ef8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc1af0a0 con 0x55bbeed5ee70 2024-05-27T07:35:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.986+0000 7fb911ef8b80 10 client.0 _ll_register_callbacks cb 0x55bbeec26c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.986+0000 7fb8fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed62aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.986+0000 7fb8fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed62aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44264/0 (socket says 172.21.15.63:44264) 2024-05-27T07:35:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.986+0000 7fb8fbe00700 1 -- 172.21.15.63:0/1806411567 learned_addr learned my addr 172.21.15.63:0/1806411567 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb900a00700 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 0x55bbeed62560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb901400700 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeed5ee70 0x55bbeed62fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb8fbe00700 1 -- 172.21.15.63:0/1806411567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeed5ee70 msgr2=0x55bbeed62fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeed5ee70 0x55bbeed62fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb8fbe00700 1 -- 172.21.15.63:0/1806411567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 msgr2=0x55bbeed62560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 0x55bbeed62560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb8fbe00700 1 -- 172.21.15.63:0/1806411567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbeed6fcc0 con 0x55bbeeccb9f0 2024-05-27T07:35:35.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb901400700 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeed5ee70 0x55bbeed62fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb900a00700 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 0x55bbeed62560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb8fbe00700 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed62aa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8035490 tx=0x7fb8e8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323b804177d54e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb8e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.987+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1806411567 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8042070 con 0x55bbeeccb9f0 2024-05-27T07:35:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.988+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1806411567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e8020810 con 0x55bbeeccb9f0 2024-05-27T07:35:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.988+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1806411567 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb8e8005660 con 0x55bbeeccb9f0 2024-05-27T07:35:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.988+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1806411567 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8e804e050 con 0x55bbeeccb9f0 2024-05-27T07:35:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.988+0000 7fb911ef8b80 10 client.15789 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:35.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.988+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1806411567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bbeed70e90 con 0x55bbeeccb9f0 2024-05-27T07:35:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.988+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1806411567 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8e8020810 con 0x55bbeeccb9f0 2024-05-27T07:35:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.988+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1806411567 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8e80354f0 con 0x55bbeeccb9f0 2024-05-27T07:35:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.989+0000 7fb911ef8b80 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.989+0000 7fb911ef8b80 10 client.15789 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:35:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.989+0000 7fb911ef8b80 20 client.15789 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.989+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bbeed71550 0x55bbeede9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.989+0000 7fb911ef8b80 4 client.15789 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:35:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.989+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1806411567 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbeec88570 con 0x55bbeed71550 2024-05-27T07:35:35.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.989+0000 7fb900a00700 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bbeed71550 0x55bbeede9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.990+0000 7fb900a00700 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bbeed71550 0x55bbeede9770 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b66169c291709af server_cookie=c12dbe9dbfc708ab in_seq=0 out_seq=0 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.990+0000 7fb8e2a00700 10 client.15789 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.991+0000 7fb8e2a00700 1 -- 172.21.15.63:0/1806411567 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbeec88570 con 0x55bbeed71550 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.991+0000 7fb8e2a00700 10 client.15789 handle_command_reply: tid=0 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.991+0000 7fb911ef8b80 1 client.15789 shutdown 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.991+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1806411567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 msgr2=0x55bbeed62aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.991+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed62aa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8035490 tx=0x7fb8e8004a50 comp rx=0 tx=0).stop 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.992+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1806411567 shutdown_connections 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.992+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bbeed5ee70 0x55bbeed62fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.992+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbeebdfe40 0x55bbeed62560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.992+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bbeed71550 0x55bbeede9770 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.992+0000 7fb911ef8b80 1 --2- 172.21.15.63:0/1806411567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbeeccb9f0 0x55bbeed62aa0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.992+0000 7fb8e2a00700 0 client.15789 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:35:35.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.992+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1806411567 >> 172.21.15.63:0/1806411567 conn(0x55bbeec79390 msgr2=0x55bbeed5fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:35.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.992+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1806411567 shutdown_connections 2024-05-27T07:35:35.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.992+0000 7fb911ef8b80 1 -- 172.21.15.63:0/1806411567 wait complete. 2024-05-27T07:35:35.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.992+0000 7fb911ef8b80 20 client.15789 trim_cache size 0 max 16384 2024-05-27T07:35:35.989 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:35:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 -- 172.21.15.63:0/2169010340 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e40b29f0 msgr2=0x7fb8e40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e40b29f0 0x7fb8e40b4f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800b6c0 tx=0x7fb8e8012890 comp rx=0 tx=0).stop 2024-05-27T07:35:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 -- 172.21.15.63:0/2169010340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8fc109860 msgr2=0x7fb8fc19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8fc109860 0x7fb8fc19f680 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009850 tx=0x7fb8f4007d90 comp rx=0 tx=0).stop 2024-05-27T07:35:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 -- 172.21.15.63:0/2169010340 shutdown_connections 2024-05-27T07:35:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e40b29f0 0x7fb8e40b4f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8fc109860 0x7fb8fc19f680 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8fc1045e0 0x7fb8fc19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 --2- 172.21.15.63:0/2169010340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8fc108f10 0x7fb8fc19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:35.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 -- 172.21.15.63:0/2169010340 >> 172.21.15.63:0/2169010340 conn(0x7fb8fc100030 msgr2=0x7fb8fc108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:35.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 -- 172.21.15.63:0/2169010340 shutdown_connections 2024-05-27T07:35:35.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:35.993+0000 7fb903400700 1 -- 172.21.15.63:0/2169010340 wait complete. 2024-05-27T07:35:36.184 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-27T07:35:36.390 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-27T07:35:36.483 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T07:35:36.491 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T07:35:36.491 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-27T07:35:36.578 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T07:35:36.589 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T07:35:36.589 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-27T07:35:36.633 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-05-27T07:35:36.634 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-05-27T07:35:36.634 DEBUG:teuthology.orchestra.run.smithi063:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-27T07:35:36.779 INFO:tasks.workunit.client.0.smithi063.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-27T07:35:36.782 INFO:tasks.workunit.client.0.smithi063.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-05-27T07:35:36.783 INFO:tasks.workunit.client.0.smithi063.stderr:+ trap -- retry EXIT 2024-05-27T07:35:36.783 INFO:tasks.workunit.client.0.smithi063.stderr:+ rm -rf ffsb 2024-05-27T07:35:36.788 INFO:tasks.workunit.client.0.smithi063.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-05-27T07:35:36.805 INFO:tasks.workunit.client.0.smithi063.stderr:Cloning into 'ffsb'... 2024-05-27T07:35:37.080 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:35:37.563 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:37.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:37 smithi086 ceph-mon[34687]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T07:35:37.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:37 smithi149 ceph-mon[35408]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T07:35:37.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:37 smithi063 ceph-mon[31613]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T07:35:39.640 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:39 smithi063 ceph-mon[31613]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:39.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:39 smithi086 ceph-mon[34687]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:39.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:39 smithi149 ceph-mon[35408]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-27T07:35:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.040+0000 7fb0df200700 1 -- 172.21.15.63:0/667233717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0e01043f0 msgr2=0x7fb0e0104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.040+0000 7fb0df200700 1 --2- 172.21.15.63:0/667233717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0e01043f0 0x7fb0e0104800 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00099c0 tx=0x7fb0d0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.040+0000 7fb0df200700 1 -- 172.21.15.63:0/667233717 shutdown_connections 2024-05-27T07:35:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.040+0000 7fb0df200700 1 --2- 172.21.15.63:0/667233717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0e0109670 0x7fb0e0110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.040+0000 7fb0df200700 1 --2- 172.21.15.63:0/667233717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0e01043f0 0x7fb0e0104800 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.040+0000 7fb0df200700 1 --2- 172.21.15.63:0/667233717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0e0108d20 0x7fb0e0109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.040+0000 7fb0df200700 1 -- 172.21.15.63:0/667233717 >> 172.21.15.63:0/667233717 conn(0x7fb0e0079d00 msgr2=0x7fb0e007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:40.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.040+0000 7fb0df200700 1 -- 172.21.15.63:0/667233717 shutdown_connections 2024-05-27T07:35:40.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.040+0000 7fb0df200700 1 -- 172.21.15.63:0/667233717 wait complete. 2024-05-27T07:35:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.041+0000 7fb0df200700 1 Processor -- start 2024-05-27T07:35:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.041+0000 7fb0df200700 1 -- start start 2024-05-27T07:35:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.041+0000 7fb0df200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0e01043f0 0x7fb0e01a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0df200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0e0108d20 0x7fb0e01a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0df200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0e0109670 0x7fb0e019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0df200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e0114150 con 0x7fb0e01043f0 2024-05-27T07:35:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0df200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e0113fd0 con 0x7fb0e0109670 2024-05-27T07:35:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0df200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e01142d0 con 0x7fb0e0108d20 2024-05-27T07:35:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0e01043f0 0x7fb0e01a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0e01043f0 0x7fb0e01a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35852/0 (socket says 172.21.15.63:35852) 2024-05-27T07:35:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0dde00700 1 -- 172.21.15.63:0/2533048635 learned_addr learned my addr 172.21.15.63:0/2533048635 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0dd400700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0e0108d20 0x7fb0e01a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0dde00700 1 -- 172.21.15.63:0/2533048635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0e0108d20 msgr2=0x7fb0e01a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0dde00700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0e0108d20 0x7fb0e01a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0dde00700 1 -- 172.21.15.63:0/2533048635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0e0109670 msgr2=0x7fb0e019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0de800700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0e0109670 0x7fb0e019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0dde00700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0e0109670 0x7fb0e019a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.042+0000 7fb0dde00700 1 -- 172.21.15.63:0/2533048635 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e019afe0 con 0x7fb0e01043f0 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.043+0000 7fb0dde00700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0e01043f0 0x7fb0e01a0710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0000c00 tx=0x7fb0d000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.043+0000 7fb0d6a00700 1 -- 172.21.15.63:0/2533048635 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000f940 con 0x7fb0e01043f0 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.043+0000 7fb0df200700 1 -- 172.21.15.63:0/2533048635 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e019b270 con 0x7fb0e01043f0 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.043+0000 7fb0d6a00700 1 -- 172.21.15.63:0/2533048635 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0d0004830 con 0x7fb0e01043f0 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.043+0000 7fb0d6a00700 1 -- 172.21.15.63:0/2533048635 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00054b0 con 0x7fb0e01043f0 2024-05-27T07:35:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.043+0000 7fb0df200700 1 -- 172.21.15.63:0/2533048635 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e019b740 con 0x7fb0e01043f0 2024-05-27T07:35:40.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.049+0000 7fb0d6a00700 1 -- 172.21.15.63:0/2533048635 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb0d0029060 con 0x7fb0e01043f0 2024-05-27T07:35:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.050+0000 7fb0f3c73b80 1 Processor -- start 2024-05-27T07:35:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.051+0000 7fb0f3c73b80 1 -- start start 2024-05-27T07:35:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.051+0000 7fb0d6a00700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0c40b2a70 0x7fb0c40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.051+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd5d99f0 0x55e3bd66d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.051+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd4ede40 0x55e3bd66dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.051+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd6772c0 0x55e3bd6796e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.051+0000 7fb0d6a00700 1 -- 172.21.15.63:0/2533048635 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb0d00e3750 con 0x7fb0e01043f0 2024-05-27T07:35:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.051+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e3bd59ae40 con 0x55e3bd6772c0 2024-05-27T07:35:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.051+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3bd58a9e0 con 0x55e3bd5d99f0 2024-05-27T07:35:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.051+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3bd58ea20 con 0x55e3bd4ede40 2024-05-27T07:35:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.052+0000 7fb0de800700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0c40b2a70 0x7fb0c40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.052+0000 7fb0de800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd6772c0 0x55e3bd6796e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.052+0000 7fb0dde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd5d99f0 0x55e3bd66d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.052+0000 7fb0dd400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd4ede40 0x55e3bd66dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.052+0000 7fb0de800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd6772c0 0x55e3bd6796e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35862/0 (socket says 172.21.15.63:35862) 2024-05-27T07:35:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.052+0000 7fb0de800700 1 -- 172.21.15.63:0/1890161275 learned_addr learned my addr 172.21.15.63:0/1890161275 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0de800700 1 -- 172.21.15.63:0/1890161275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd4ede40 msgr2=0x55e3bd66dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0de800700 1 --2- 172.21.15.63:0/1890161275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd4ede40 0x55e3bd66dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0de800700 1 -- 172.21.15.63:0/1890161275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd5d99f0 msgr2=0x55e3bd66d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0de800700 1 --2- 172.21.15.63:0/1890161275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd5d99f0 0x55e3bd66d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0de800700 1 -- 172.21.15.63:0/1890161275 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3bd553420 con 0x55e3bd6772c0 2024-05-27T07:35:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0dde00700 1 --2- 172.21.15.63:0/1890161275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd5d99f0 0x55e3bd66d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0de800700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0c40b2a70 0x7fb0c40b4f80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80094f0 tx=0x7fb0d8009b70 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0de800700 1 --2- 172.21.15.63:0/1890161275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd6772c0 0x55e3bd6796e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80175d0 tx=0x7fb0d801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4038931803a11569 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0cb400700 1 -- 172.21.15.63:0/1890161275 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8017d90 con 0x55e3bd6772c0 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0cb400700 1 -- 172.21.15.63:0/1890161275 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0d8004bb0 con 0x55e3bd6772c0 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.053+0000 7fb0cb400700 1 -- 172.21.15.63:0/1890161275 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8004410 con 0x55e3bd6772c0 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.054+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1890161275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd6772c0 msgr2=0x55e3bd6796e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.054+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/1890161275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd6772c0 0x55e3bd6796e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80175d0 tx=0x7fb0d801a000 comp rx=0 tx=0).stop 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.055+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1890161275 shutdown_connections 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.055+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/1890161275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd6772c0 0x55e3bd6796e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.055+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/1890161275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd5d99f0 0x55e3bd66d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.055+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/1890161275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd4ede40 0x55e3bd66dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.055+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1890161275 >> 172.21.15.63:0/1890161275 conn(0x55e3bd587390 msgr2=0x55e3bd668710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.055+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1890161275 shutdown_connections 2024-05-27T07:35:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.055+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1890161275 wait complete. 2024-05-27T07:35:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.055+0000 7fb0f3c73b80 1 -- start start 2024-05-27T07:35:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.056+0000 7fb0f3c73b80 1 Processor -- start 2024-05-27T07:35:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.056+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd6702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.056+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd5d99f0 0x55e3bd670800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.056+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 0x55e3bd670d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.056+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e3bd490fa0 con 0x55e3bd4ede40 2024-05-27T07:35:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.056+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3bd52b760 con 0x55e3bd5d99f0 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.056+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3bd679e00 con 0x55e3bd6772c0 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.056+0000 7fb0f3c73b80 10 client.0 _ll_register_callbacks cb 0x55e3bd534c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.057+0000 7fb0dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd6702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.057+0000 7fb0dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd6702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35864/0 (socket says 172.21.15.63:35864) 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.057+0000 7fb0dde00700 1 -- 172.21.15.63:0/1472613636 learned_addr learned my addr 172.21.15.63:0/1472613636 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.057+0000 7fb0dde00700 1 -- 172.21.15.63:0/1472613636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 msgr2=0x55e3bd670d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.057+0000 7fb0de800700 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 0x55e3bd670d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.057+0000 7fb0dde00700 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 0x55e3bd670d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.057+0000 7fb0dde00700 1 -- 172.21.15.63:0/1472613636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd5d99f0 msgr2=0x55e3bd670800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.057+0000 7fb0dde00700 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd5d99f0 0x55e3bd670800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.057+0000 7fb0dde00700 1 -- 172.21.15.63:0/1472613636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3bd4da8d0 con 0x55e3bd4ede40 2024-05-27T07:35:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.057+0000 7fb0de800700 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 0x55e3bd670d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.058+0000 7fb0dde00700 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd6702c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0021000 tx=0x7fb0d0004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5c9504df05f552 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.058+0000 7fb0bfe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.058+0000 7fb0bfe00700 1 -- 172.21.15.63:0/1472613636 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0053be0 con 0x55e3bd4ede40 2024-05-27T07:35:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.058+0000 7fb0bfe00700 1 -- 172.21.15.63:0/1472613636 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb0d0058620 con 0x55e3bd4ede40 2024-05-27T07:35:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.058+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1472613636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d8004bb0 con 0x55e3bd4ede40 2024-05-27T07:35:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.058+0000 7fb0bfe00700 1 -- 172.21.15.63:0/1472613636 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0d00c4050 con 0x55e3bd4ede40 2024-05-27T07:35:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.058+0000 7fb0f3c73b80 10 client.15801 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.058+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1472613636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3bd55b060 con 0x55e3bd4ede40 2024-05-27T07:35:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.059+0000 7fb0bfe00700 1 -- 172.21.15.63:0/1472613636 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0d8004bb0 con 0x55e3bd4ede40 2024-05-27T07:35:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.059+0000 7fb0bfe00700 1 -- 172.21.15.63:0/1472613636 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb0d00b3090 con 0x55e3bd4ede40 2024-05-27T07:35:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.059+0000 7fb0f3c73b80 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.059+0000 7fb0f3c73b80 10 client.15801 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:35:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.059+0000 7fb0f3c73b80 20 client.15801 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.059+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3bd68f450 0x55e3bd691940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.059+0000 7fb0f3c73b80 4 client.15801 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.059+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1472613636 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3bd59cd90 con 0x55e3bd68f450 2024-05-27T07:35:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.060+0000 7fb0dd400700 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3bd68f450 0x55e3bd691940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.061+0000 7fb0dd400700 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3bd68f450 0x55e3bd691940 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a0fc63fbb027b4c server_cookie=1d24becc3a422c29 in_seq=0 out_seq=0 2024-05-27T07:35:40.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.061+0000 7fb0bfe00700 10 client.15801 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:35:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.063+0000 7fb0bfe00700 1 -- 172.21.15.63:0/1472613636 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e3bd59cd90 con 0x55e3bd68f450 2024-05-27T07:35:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.063+0000 7fb0bfe00700 10 client.15801 handle_command_reply: tid=0 2024-05-27T07:35:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 client.15801 shutdown 2024-05-27T07:35:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1472613636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 msgr2=0x55e3bd6702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd6702c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0021000 tx=0x7fb0d0004f10 comp rx=0 tx=0).stop 2024-05-27T07:35:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1472613636 shutdown_connections 2024-05-27T07:35:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 0x55e3bd670d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3bd68f450 0x55e3bd691940 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd6702c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/1472613636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd5d99f0 0x55e3bd670800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1472613636 >> 172.21.15.63:0/1472613636 conn(0x55e3bd587390 msgr2=0x55e3bd66d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0bfe00700 0 client.15801 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:35:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1472613636 shutdown_connections 2024-05-27T07:35:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/1472613636 wait complete. 2024-05-27T07:35:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.064+0000 7fb0f3c73b80 20 client.15801 trim_cache size 0 max 16384 2024-05-27T07:35:40.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.083+0000 7fb0f3c73b80 1 Processor -- start 2024-05-27T07:35:40.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.083+0000 7fb0f3c73b80 1 -- start start 2024-05-27T07:35:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.083+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd4ede40 0x55e3bd67d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.083+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd5d99f0 0x55e3bd668740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.083+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 0x55e3bd67a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.083+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e01b07d0 con 0x55e3bd5d99f0 2024-05-27T07:35:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.083+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e01ae060 con 0x55e3bd4ede40 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.083+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e01abed0 con 0x55e3bd6772c0 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.084+0000 7fb0dd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd5d99f0 0x55e3bd668740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.084+0000 7fb0dd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd5d99f0 0x55e3bd668740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35868/0 (socket says 172.21.15.63:35868) 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.084+0000 7fb0dd400700 1 -- 172.21.15.63:0/924720045 learned_addr learned my addr 172.21.15.63:0/924720045 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.084+0000 7fb0dde00700 1 --2- 172.21.15.63:0/924720045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd4ede40 0x55e3bd67d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.084+0000 7fb0de800700 1 --2- 172.21.15.63:0/924720045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 0x55e3bd67a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.084+0000 7fb0dd400700 1 -- 172.21.15.63:0/924720045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 msgr2=0x55e3bd67a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.084+0000 7fb0dd400700 1 --2- 172.21.15.63:0/924720045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 0x55e3bd67a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.084+0000 7fb0dd400700 1 -- 172.21.15.63:0/924720045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd4ede40 msgr2=0x55e3bd67d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.084+0000 7fb0dd400700 1 --2- 172.21.15.63:0/924720045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd4ede40 0x55e3bd67d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.084+0000 7fb0dd400700 1 -- 172.21.15.63:0/924720045 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3bd67aca0 con 0x55e3bd5d99f0 2024-05-27T07:35:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.085+0000 7fb0dd400700 1 --2- 172.21.15.63:0/924720045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd5d99f0 0x55e3bd668740 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0100d40 tx=0x7fb0cc005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fa3351ea9824c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.085+0000 7fb0caa00700 1 -- 172.21.15.63:0/924720045 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc013d30 con 0x55e3bd5d99f0 2024-05-27T07:35:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.085+0000 7fb0caa00700 1 -- 172.21.15.63:0/924720045 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0cc01b840 con 0x55e3bd5d99f0 2024-05-27T07:35:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.085+0000 7fb0caa00700 1 -- 172.21.15.63:0/924720045 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc0119a0 con 0x55e3bd5d99f0 2024-05-27T07:35:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.086+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/924720045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd5d99f0 msgr2=0x55e3bd668740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.086+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/924720045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd5d99f0 0x55e3bd668740 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0100d40 tx=0x7fb0cc005ce0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.086+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/924720045 shutdown_connections 2024-05-27T07:35:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.086+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/924720045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd6772c0 0x55e3bd67a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.086+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/924720045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd4ede40 0x55e3bd67d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.086+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/924720045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd5d99f0 0x55e3bd668740 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.086+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/924720045 >> 172.21.15.63:0/924720045 conn(0x55e3bd587390 msgr2=0x55e3bd67a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.087+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/924720045 shutdown_connections 2024-05-27T07:35:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.087+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/924720045 wait complete. 2024-05-27T07:35:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.087+0000 7fb0f3c73b80 1 -- start start 2024-05-27T07:35:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.087+0000 7fb0f3c73b80 1 Processor -- start 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.088+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd68b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.088+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd5d99f0 0x55e3bd68bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.088+0000 7fb0f3c73b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd6772c0 0x55e3bd67b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.088+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e01ac520 con 0x55e3bd4ede40 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.088+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e01ad990 con 0x55e3bd6772c0 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.088+0000 7fb0f3c73b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e01acbf0 con 0x55e3bd5d99f0 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.088+0000 7fb0f3c73b80 10 client.0 _ll_register_callbacks cb 0x55e3bd534c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.088+0000 7fb0dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd68b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.088+0000 7fb0dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd68b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35882/0 (socket says 172.21.15.63:35882) 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.088+0000 7fb0dde00700 1 -- 172.21.15.63:0/2118229352 learned_addr learned my addr 172.21.15.63:0/2118229352 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.089+0000 7fb0de800700 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd6772c0 0x55e3bd67b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.089+0000 7fb0dd400700 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd5d99f0 0x55e3bd68bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.089+0000 7fb0dde00700 1 -- 172.21.15.63:0/2118229352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd5d99f0 msgr2=0x55e3bd68bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.089+0000 7fb0dde00700 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd5d99f0 0x55e3bd68bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.089+0000 7fb0dde00700 1 -- 172.21.15.63:0/2118229352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd6772c0 msgr2=0x55e3bd67b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.089+0000 7fb0dde00700 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd6772c0 0x55e3bd67b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.089+0000 7fb0dde00700 1 -- 172.21.15.63:0/2118229352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3bd68c5c0 con 0x55e3bd4ede40 2024-05-27T07:35:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.089+0000 7fb0dde00700 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd68b880 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00aa680 tx=0x7fb0d0053e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d75f7cdc220c5b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.089+0000 7fb0bfe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.090+0000 7fb0bfe00700 1 -- 172.21.15.63:0/2118229352 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00c8030 con 0x55e3bd4ede40 2024-05-27T07:35:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.090+0000 7fb0bfe00700 1 -- 172.21.15.63:0/2118229352 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb0d0058070 con 0x55e3bd4ede40 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.090+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/2118229352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0cc01b840 con 0x55e3bd4ede40 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.090+0000 7fb0bfe00700 1 -- 172.21.15.63:0/2118229352 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0d008f050 con 0x55e3bd4ede40 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.090+0000 7fb0f3c73b80 10 client.15807 fetch_fsmap learned FSMap version 27 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.090+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/2118229352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3bd6f94b0 con 0x55e3bd4ede40 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.090+0000 7fb0bfe00700 1 -- 172.21.15.63:0/2118229352 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0cc01b840 con 0x55e3bd4ede40 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.090+0000 7fb0bfe00700 1 -- 172.21.15.63:0/2118229352 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb0d00bfb10 con 0x55e3bd4ede40 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.091+0000 7fb0f3c73b80 10 client.15807 fetch_fsmap finished waiting for FSMap version 27 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.091+0000 7fb0f3c73b80 10 client.15807 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.091+0000 7fb0f3c73b80 20 client.15807 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.091+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3bd6f9a60 0x55e3bd6fbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.091+0000 7fb0f3c73b80 4 client.15807 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.091+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/2118229352 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3bd596570 con 0x55e3bd6f9a60 2024-05-27T07:35:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.091+0000 7fb0dd400700 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3bd6f9a60 0x55e3bd6fbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.093+0000 7fb0dd400700 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3bd6f9a60 0x55e3bd6fbe80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9f319fa2694846f server_cookie=7ee7d8bed2f02ea1 in_seq=0 out_seq=0 2024-05-27T07:35:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.093+0000 7fb0bfe00700 10 client.15807 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:35:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0bfe00700 1 -- 172.21.15.63:0/2118229352 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3bd596570 con 0x55e3bd6f9a60 2024-05-27T07:35:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0bfe00700 10 client.15807 handle_command_reply: tid=0 2024-05-27T07:35:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 client.15807 shutdown 2024-05-27T07:35:40.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/2118229352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 msgr2=0x55e3bd68b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:40.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd68b880 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00aa680 tx=0x7fb0d0053e10 comp rx=0 tx=0).stop 2024-05-27T07:35:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/2118229352 shutdown_connections 2024-05-27T07:35:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3bd6772c0 0x55e3bd67b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3bd6f9a60 0x55e3bd6fbe80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3bd4ede40 0x55e3bd68b880 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 --2- 172.21.15.63:0/2118229352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3bd5d99f0 0x55e3bd68bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/2118229352 >> 172.21.15.63:0/2118229352 conn(0x55e3bd587390 msgr2=0x55e3bd66c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0bfe00700 0 client.15807 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:35:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/2118229352 shutdown_connections 2024-05-27T07:35:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 1 -- 172.21.15.63:0/2118229352 wait complete. 2024-05-27T07:35:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.094+0000 7fb0f3c73b80 20 client.15807 trim_cache size 0 max 16384 2024-05-27T07:35:40.091 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:35:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.095+0000 7fb0df200700 1 -- 172.21.15.63:0/2533048635 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0c40b2a70 msgr2=0x7fb0c40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.095+0000 7fb0df200700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0c40b2a70 0x7fb0c40b4f80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80094f0 tx=0x7fb0d8009b70 comp rx=0 tx=0).stop 2024-05-27T07:35:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.095+0000 7fb0df200700 1 -- 172.21.15.63:0/2533048635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0e01043f0 msgr2=0x7fb0e01a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.095+0000 7fb0df200700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0e01043f0 0x7fb0e01a0710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0000c00 tx=0x7fb0d000f510 comp rx=0 tx=0).stop 2024-05-27T07:35:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.095+0000 7fb0df200700 1 -- 172.21.15.63:0/2533048635 shutdown_connections 2024-05-27T07:35:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.095+0000 7fb0df200700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0c40b2a70 0x7fb0c40b4f80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.095+0000 7fb0df200700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0e0109670 0x7fb0e019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.095+0000 7fb0df200700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0e01043f0 0x7fb0e01a0710 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.095+0000 7fb0df200700 1 --2- 172.21.15.63:0/2533048635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0e0108d20 0x7fb0e01a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.095+0000 7fb0df200700 1 -- 172.21.15.63:0/2533048635 >> 172.21.15.63:0/2533048635 conn(0x7fb0e0079d00 msgr2=0x7fb0e01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.096+0000 7fb0df200700 1 -- 172.21.15.63:0/2533048635 shutdown_connections 2024-05-27T07:35:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:40.096+0000 7fb0df200700 1 -- 172.21.15.63:0/2533048635 wait complete. 2024-05-27T07:35:41.063 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:35:41.540 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:42.633 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:42 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:35:42.633 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:42 smithi063 ceph-mon[31613]: pgmap v66: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 133 KiB/s wr, 26 op/s 2024-05-27T07:35:42.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:42 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:35:42.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:42 smithi086 ceph-mon[34687]: pgmap v66: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 133 KiB/s wr, 26 op/s 2024-05-27T07:35:42.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:42 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:35:42.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:42 smithi149 ceph-mon[35408]: pgmap v66: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 133 KiB/s wr, 26 op/s 2024-05-27T07:35:43.493 INFO:tasks.workunit.client.0.smithi063.stderr:Updating files: 91% (54/59) Updating files: 93% (55/59) Updating files: 94% (56/59) Updating files: 96% (57/59) Updating files: 98% (58/59) Updating files: 100% (59/59) Updating files: 100% (59/59), done. 2024-05-27T07:35:43.694 INFO:tasks.workunit.client.0.smithi063.stderr:+ trap - EXIT 2024-05-27T07:35:43.695 INFO:tasks.workunit.client.0.smithi063.stderr:+ cd ffsb 2024-05-27T07:35:43.698 INFO:tasks.workunit.client.0.smithi063.stderr:+ ./configure 2024-05-27T07:35:43.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:43 smithi086 ceph-mon[34687]: pgmap v67: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 133 KiB/s wr, 23 op/s 2024-05-27T07:35:43.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:43 smithi149 ceph-mon[35408]: pgmap v67: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 133 KiB/s wr, 23 op/s 2024-05-27T07:35:43.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:43 smithi063 ceph-mon[31613]: pgmap v67: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 133 KiB/s wr, 23 op/s 2024-05-27T07:35:44.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.017+0000 7ffb28a00700 1 -- 172.21.15.63:0/2330684444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb24103870 msgr2=0x7ffb24110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.017+0000 7ffb28a00700 1 --2- 172.21.15.63:0/2330684444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb24103870 0x7ffb24110590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009990 tx=0x7ffb10009ca0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.017+0000 7ffb28a00700 1 -- 172.21.15.63:0/2330684444 shutdown_connections 2024-05-27T07:35:44.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.017+0000 7ffb28a00700 1 --2- 172.21.15.63:0/2330684444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24110ad0 0x7ffb24112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.017+0000 7ffb28a00700 1 --2- 172.21.15.63:0/2330684444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb24102f20 0x7ffb24103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.017+0000 7ffb28a00700 1 --2- 172.21.15.63:0/2330684444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb24103870 0x7ffb24110590 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.017+0000 7ffb28a00700 1 -- 172.21.15.63:0/2330684444 >> 172.21.15.63:0/2330684444 conn(0x7ffb24078af0 msgr2=0x7ffb24078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.017+0000 7ffb28a00700 1 -- 172.21.15.63:0/2330684444 shutdown_connections 2024-05-27T07:35:44.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.017+0000 7ffb28a00700 1 -- 172.21.15.63:0/2330684444 wait complete. 2024-05-27T07:35:44.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.019+0000 7ffb28a00700 1 Processor -- start 2024-05-27T07:35:44.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.019+0000 7ffb28a00700 1 -- start start 2024-05-27T07:35:44.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.019+0000 7ffb28a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb24102f20 0x7ffb2419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.019+0000 7ffb28a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb24103870 0x7ffb2419d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.019+0000 7ffb28a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24110ad0 0x7ffb241a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.019+0000 7ffb28a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb24116330 con 0x7ffb24103870 2024-05-27T07:35:44.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.019+0000 7ffb28a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241161b0 con 0x7ffb24110ad0 2024-05-27T07:35:44.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.019+0000 7ffb28a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241164b0 con 0x7ffb24102f20 2024-05-27T07:35:44.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.020+0000 7ffb21e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb24103870 0x7ffb2419d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.020+0000 7ffb21e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb24103870 0x7ffb2419d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35904/0 (socket says 172.21.15.63:35904) 2024-05-27T07:35:44.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.020+0000 7ffb21e00700 1 -- 172.21.15.63:0/3541110359 learned_addr learned my addr 172.21.15.63:0/3541110359 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:44.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.020+0000 7ffb22800700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb24102f20 0x7ffb2419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.020+0000 7ffb23200700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24110ad0 0x7ffb241a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.020+0000 7ffb21e00700 1 -- 172.21.15.63:0/3541110359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb24102f20 msgr2=0x7ffb2419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.020+0000 7ffb21e00700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb24102f20 0x7ffb2419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.020+0000 7ffb21e00700 1 -- 172.21.15.63:0/3541110359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24110ad0 msgr2=0x7ffb241a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.020+0000 7ffb21e00700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24110ad0 0x7ffb241a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.020+0000 7ffb21e00700 1 -- 172.21.15.63:0/3541110359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb241a1c10 con 0x7ffb24103870 2024-05-27T07:35:44.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.021+0000 7ffb21e00700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb24103870 0x7ffb2419d2e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ffb10000c00 tx=0x7ffb10005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.021+0000 7ffb1b400700 1 -- 172.21.15.63:0/3541110359 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10027070 con 0x7ffb24103870 2024-05-27T07:35:44.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.021+0000 7ffb1b400700 1 -- 172.21.15.63:0/3541110359 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb1000fc40 con 0x7ffb24103870 2024-05-27T07:35:44.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.021+0000 7ffb28a00700 1 -- 172.21.15.63:0/3541110359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb241a1ea0 con 0x7ffb24103870 2024-05-27T07:35:44.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.021+0000 7ffb1b400700 1 -- 172.21.15.63:0/3541110359 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10022bc0 con 0x7ffb24103870 2024-05-27T07:35:44.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.022+0000 7ffb28a00700 1 -- 172.21.15.63:0/3541110359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb241a2380 con 0x7ffb24103870 2024-05-27T07:35:44.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.026+0000 7ffb1b400700 1 -- 172.21.15.63:0/3541110359 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffb1002a070 con 0x7ffb24103870 2024-05-27T07:35:44.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.026+0000 7ffb37e9ab80 1 Processor -- start 2024-05-27T07:35:44.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.026+0000 7ffb37e9ab80 1 -- start start 2024-05-27T07:35:44.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.027+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ccf58e40 0x5627cd0d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.027+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627cd0d7410 0x5627cd0e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.027+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7820 0x5627cd0e4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.027+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5627cd005e40 con 0x5627cd0d7820 2024-05-27T07:35:44.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.027+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627ccff59e0 con 0x5627ccf58e40 2024-05-27T07:35:44.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.027+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5627ccff9a20 con 0x5627cd0d7410 2024-05-27T07:35:44.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.027+0000 7ffb21e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7820 0x5627cd0e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.027+0000 7ffb21e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7820 0x5627cd0e4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35918/0 (socket says 172.21.15.63:35918) 2024-05-27T07:35:44.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.027+0000 7ffb21e00700 1 -- 172.21.15.63:0/1173294403 learned_addr learned my addr 172.21.15.63:0/1173294403 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:44.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.028+0000 7ffb22800700 1 --2- 172.21.15.63:0/1173294403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627cd0d7410 0x5627cd0e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.028+0000 7ffb23200700 1 --2- 172.21.15.63:0/1173294403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ccf58e40 0x5627cd0d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.028+0000 7ffb21e00700 1 -- 172.21.15.63:0/1173294403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627cd0d7410 msgr2=0x5627cd0e1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.028+0000 7ffb21e00700 1 --2- 172.21.15.63:0/1173294403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627cd0d7410 0x5627cd0e1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.028+0000 7ffb21e00700 1 -- 172.21.15.63:0/1173294403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ccf58e40 msgr2=0x5627cd0d6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.028+0000 7ffb21e00700 1 --2- 172.21.15.63:0/1173294403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ccf58e40 0x5627cd0d6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.028+0000 7ffb21e00700 1 -- 172.21.15.63:0/1173294403 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ccfbe420 con 0x5627cd0d7820 2024-05-27T07:35:44.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.028+0000 7ffb23200700 1 --2- 172.21.15.63:0/1173294403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ccf58e40 0x5627cd0d6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:44.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.028+0000 7ffb1b400700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb080b29f0 0x7ffb080b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.029+0000 7ffb23200700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb080b29f0 0x7ffb080b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.029+0000 7ffb1b400700 1 -- 172.21.15.63:0/3541110359 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7ffb100e3560 con 0x7ffb24103870 2024-05-27T07:35:44.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.029+0000 7ffb21e00700 1 --2- 172.21.15.63:0/1173294403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7820 0x5627cd0e4020 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffb100e0820 tx=0x7ffb100e9980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8391c6c3a0dbd183 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.029+0000 7ffb0fe00700 1 -- 172.21.15.63:0/1173294403 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb100a63c0 con 0x5627cd0d7820 2024-05-27T07:35:44.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.029+0000 7ffb0fe00700 1 -- 172.21.15.63:0/1173294403 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb100aa010 con 0x5627cd0d7820 2024-05-27T07:35:44.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.029+0000 7ffb0fe00700 1 -- 172.21.15.63:0/1173294403 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb100920d0 con 0x5627cd0d7820 2024-05-27T07:35:44.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.030+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/1173294403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7820 msgr2=0x5627cd0e4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.030+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/1173294403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7820 0x5627cd0e4020 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffb100e0820 tx=0x7ffb100e9980 comp rx=0 tx=0).stop 2024-05-27T07:35:44.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.030+0000 7ffb23200700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb080b29f0 0x7ffb080b4f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5627cd0d3690 tx=0x7ffb1c009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.031+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/1173294403 shutdown_connections 2024-05-27T07:35:44.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.031+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/1173294403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7820 0x5627cd0e4020 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.031+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/1173294403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627ccf58e40 0x5627cd0d6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.031+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/1173294403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627cd0d7410 0x5627cd0e1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.031+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/1173294403 >> 172.21.15.63:0/1173294403 conn(0x5627ccfb8fc0 msgr2=0x5627cd0d2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.031+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/1173294403 shutdown_connections 2024-05-27T07:35:44.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.031+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/1173294403 wait complete. 2024-05-27T07:35:44.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.032+0000 7ffb37e9ab80 1 -- start start 2024-05-27T07:35:44.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.032+0000 7ffb37e9ab80 1 Processor -- start 2024-05-27T07:35:44.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627ccf58e40 0x5627cd0e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd0e2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5627ccefbfa0 con 0x5627cd0d7410 2024-05-27T07:35:44.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627ccf96760 con 0x5627cd0d7820 2024-05-27T07:35:44.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5627cd0e4750 con 0x5627ccf58e40 2024-05-27T07:35:44.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb37e9ab80 10 client.0 _ll_register_callbacks cb 0x5627ccf9fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:44.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb21e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb22800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627ccf58e40 0x5627cd0e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb21e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35926/0 (socket says 172.21.15.63:35926) 2024-05-27T07:35:44.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.033+0000 7ffb21e00700 1 -- 172.21.15.63:0/207845534 learned_addr learned my addr 172.21.15.63:0/207845534 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:44.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.034+0000 7ffb23200700 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd0e2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.034+0000 7ffb21e00700 1 -- 172.21.15.63:0/207845534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627ccf58e40 msgr2=0x5627cd0e15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.034+0000 7ffb21e00700 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627ccf58e40 0x5627cd0e15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.034+0000 7ffb21e00700 1 -- 172.21.15.63:0/207845534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 msgr2=0x5627cd0e2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.034+0000 7ffb21e00700 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd0e2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.034+0000 7ffb21e00700 1 -- 172.21.15.63:0/207845534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ccf458d0 con 0x5627cd0d7410 2024-05-27T07:35:44.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.034+0000 7ffb21e00700 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e1ae0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffb100a7770 tx=0x7ffb1000b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf4970bf20293b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.034+0000 7ffb0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:44.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.035+0000 7ffb0cc00700 1 -- 172.21.15.63:0/207845534 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb100a9c70 con 0x5627cd0d7410 2024-05-27T07:35:44.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.035+0000 7ffb0cc00700 1 -- 172.21.15.63:0/207845534 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7ffb100e3120 con 0x5627cd0d7410 2024-05-27T07:35:44.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.035+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/207845534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb100aa010 con 0x5627cd0d7410 2024-05-27T07:35:44.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.035+0000 7ffb0cc00700 1 -- 172.21.15.63:0/207845534 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb100a9e50 con 0x5627cd0d7410 2024-05-27T07:35:44.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.035+0000 7ffb37e9ab80 10 client.15819 fetch_fsmap learned FSMap version 28 2024-05-27T07:35:44.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.035+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/207845534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5627ccfc6060 con 0x5627cd0d7410 2024-05-27T07:35:44.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.036+0000 7ffb0cc00700 1 -- 172.21.15.63:0/207845534 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb100aa010 con 0x5627cd0d7410 2024-05-27T07:35:44.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.036+0000 7ffb0cc00700 1 -- 172.21.15.63:0/207845534 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffb100b2e40 con 0x5627cd0d7410 2024-05-27T07:35:44.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.036+0000 7ffb37e9ab80 10 client.15819 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:35:44.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.036+0000 7ffb37e9ab80 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:44.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.036+0000 7ffb37e9ab80 20 client.15819 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:44.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.036+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627cd0f14f0 0x5627cd0f3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.036+0000 7ffb37e9ab80 4 client.15819 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.036+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/207845534 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627cd007d90 con 0x5627cd0f14f0 2024-05-27T07:35:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.037+0000 7ffb22800700 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627cd0f14f0 0x5627cd0f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.038+0000 7ffb22800700 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627cd0f14f0 0x5627cd0f3910 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc693949eebecf8e server_cookie=135ca4c6011948bf in_seq=0 out_seq=0 2024-05-27T07:35:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.038+0000 7ffb0cc00700 10 client.15819 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:44.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.040+0000 7ffb0cc00700 1 -- 172.21.15.63:0/207845534 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5627cd007d90 con 0x5627cd0f14f0 2024-05-27T07:35:44.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.040+0000 7ffb0cc00700 10 client.15819 handle_command_reply: tid=0 2024-05-27T07:35:44.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.040+0000 7ffb37e9ab80 1 client.15819 shutdown 2024-05-27T07:35:44.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.040+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/207845534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 msgr2=0x5627cd0e1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.040+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e1ae0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffb100a7770 tx=0x7ffb1000b540 comp rx=0 tx=0).stop 2024-05-27T07:35:44.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.040+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/207845534 shutdown_connections 2024-05-27T07:35:44.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.041+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627cd0f14f0 0x5627cd0f3910 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.041+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd0e2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.041+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627ccf58e40 0x5627cd0e15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.041+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/207845534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e1ae0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.041+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/207845534 >> 172.21.15.63:0/207845534 conn(0x5627ccfb8fc0 msgr2=0x5627cd0d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.041+0000 7ffb0cc00700 0 client.15819 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.041+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/207845534 shutdown_connections 2024-05-27T07:35:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.041+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/207845534 wait complete. 2024-05-27T07:35:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.041+0000 7ffb37e9ab80 20 client.15819 trim_cache size 0 max 16384 2024-05-27T07:35:44.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.060+0000 7ffb37e9ab80 1 Processor -- start 2024-05-27T07:35:44.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.060+0000 7ffb37e9ab80 1 -- start start 2024-05-27T07:35:44.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627ccf58e40 0x5627cd0f3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627cd0d7410 0x5627cd0dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd0e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241b4c50 con 0x5627ccf58e40 2024-05-27T07:35:44.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241b24e0 con 0x5627cd0d7820 2024-05-27T07:35:44.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241b0330 con 0x5627cd0d7410 2024-05-27T07:35:44.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb22800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627ccf58e40 0x5627cd0f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb22800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627ccf58e40 0x5627cd0f3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35936/0 (socket says 172.21.15.63:35936) 2024-05-27T07:35:44.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb22800700 1 -- 172.21.15.63:0/2997095791 learned_addr learned my addr 172.21.15.63:0/2997095791 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:44.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb21e00700 1 --2- 172.21.15.63:0/2997095791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627cd0d7410 0x5627cd0dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.061+0000 7ffb23200700 1 --2- 172.21.15.63:0/2997095791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd0e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.062+0000 7ffb22800700 1 -- 172.21.15.63:0/2997095791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627cd0d7410 msgr2=0x5627cd0dcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.062+0000 7ffb22800700 1 --2- 172.21.15.63:0/2997095791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627cd0d7410 0x5627cd0dcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.062+0000 7ffb22800700 1 -- 172.21.15.63:0/2997095791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 msgr2=0x5627cd0e3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.062+0000 7ffb22800700 1 --2- 172.21.15.63:0/2997095791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd0e3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.062+0000 7ffb22800700 1 -- 172.21.15.63:0/2997095791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627cd0dfb10 con 0x5627ccf58e40 2024-05-27T07:35:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.062+0000 7ffb22800700 1 --2- 172.21.15.63:0/2997095791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627ccf58e40 0x5627cd0f3910 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffb14004140 tx=0x7ffb1400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bda1a16a866b477 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.062+0000 7ffb0f400700 1 -- 172.21.15.63:0/2997095791 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14003c80 con 0x5627ccf58e40 2024-05-27T07:35:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.062+0000 7ffb0f400700 1 -- 172.21.15.63:0/2997095791 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb14003e20 con 0x5627ccf58e40 2024-05-27T07:35:44.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.062+0000 7ffb0f400700 1 -- 172.21.15.63:0/2997095791 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb14011440 con 0x5627ccf58e40 2024-05-27T07:35:44.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.063+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/2997095791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627ccf58e40 msgr2=0x5627cd0f3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.063+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/2997095791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627ccf58e40 0x5627cd0f3910 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffb14004140 tx=0x7ffb1400b410 comp rx=0 tx=0).stop 2024-05-27T07:35:44.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.063+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/2997095791 shutdown_connections 2024-05-27T07:35:44.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.063+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/2997095791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd0e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.063+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/2997095791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627ccf58e40 0x5627cd0f3910 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.063+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/2997095791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627cd0d7410 0x5627cd0dcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.063+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/2997095791 >> 172.21.15.63:0/2997095791 conn(0x5627ccfb8fc0 msgr2=0x5627cd0d27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.064+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/2997095791 shutdown_connections 2024-05-27T07:35:44.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.064+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/2997095791 wait complete. 2024-05-27T07:35:44.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.064+0000 7ffb37e9ab80 1 -- start start 2024-05-27T07:35:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.064+0000 7ffb37e9ab80 1 Processor -- start 2024-05-27T07:35:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.065+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627ccf58e40 0x5627cd0e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.065+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.065+0000 7ffb37e9ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.065+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241b09a0 con 0x5627cd0d7410 2024-05-27T07:35:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.065+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241b1e10 con 0x5627cd0d7820 2024-05-27T07:35:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.065+0000 7ffb37e9ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb241b1070 con 0x5627ccf58e40 2024-05-27T07:35:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.065+0000 7ffb37e9ab80 10 client.0 _ll_register_callbacks cb 0x5627ccf9fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:44.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.065+0000 7ffb21e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.065+0000 7ffb21e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35942/0 (socket says 172.21.15.63:35942) 2024-05-27T07:35:44.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.065+0000 7ffb21e00700 1 -- 172.21.15.63:0/3165075512 learned_addr learned my addr 172.21.15.63:0/3165075512 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:44.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb22800700 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627ccf58e40 0x5627cd0e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb23200700 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb21e00700 1 -- 172.21.15.63:0/3165075512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627ccf58e40 msgr2=0x5627cd0e4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb21e00700 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627ccf58e40 0x5627cd0e4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb21e00700 1 -- 172.21.15.63:0/3165075512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 msgr2=0x5627cd159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb21e00700 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb21e00700 1 -- 172.21.15.63:0/3165075512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627cd0e5920 con 0x5627cd0d7410 2024-05-27T07:35:44.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb21e00700 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e5020 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ffb100aad00 tx=0x7ffb10009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972f2082d144d57d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb0cc00700 1 -- 172.21.15.63:0/3165075512 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb100aa2d0 con 0x5627cd0d7410 2024-05-27T07:35:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.066+0000 7ffb0cc00700 1 -- 172.21.15.63:0/3165075512 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7ffb100afa40 con 0x5627cd0d7410 2024-05-27T07:35:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/3165075512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb14003e20 con 0x5627cd0d7410 2024-05-27T07:35:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb0cc00700 1 -- 172.21.15.63:0/3165075512 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb100a9c70 con 0x5627cd0d7410 2024-05-27T07:35:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb37e9ab80 10 client.15825 fetch_fsmap learned FSMap version 28 2024-05-27T07:35:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/3165075512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5627cd0e5920 con 0x5627cd0d7410 2024-05-27T07:35:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb0cc00700 1 -- 172.21.15.63:0/3165075512 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb14003e20 con 0x5627cd0d7410 2024-05-27T07:35:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb0cc00700 1 -- 172.21.15.63:0/3165075512 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffb100a9760 con 0x5627cd0d7410 2024-05-27T07:35:44.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb37e9ab80 10 client.15825 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:35:44.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb37e9ab80 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:44.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb37e9ab80 20 client.15825 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:44.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627cd15b630 0x5627cd15da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb37e9ab80 4 client.15825 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:35:44.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.067+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/3165075512 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627cd001570 con 0x5627cd15b630 2024-05-27T07:35:44.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.068+0000 7ffb22800700 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627cd15b630 0x5627cd15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.069+0000 7ffb22800700 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627cd15b630 0x5627cd15da50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72af4c9adae133b0 server_cookie=b0c3ce6acb3d2e70 in_seq=0 out_seq=0 2024-05-27T07:35:44.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.069+0000 7ffb0cc00700 10 client.15825 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.075+0000 7ffb0cc00700 1 -- 172.21.15.63:0/3165075512 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627cd001570 con 0x5627cd15b630 2024-05-27T07:35:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.075+0000 7ffb0cc00700 10 client.15825 handle_command_reply: tid=0 2024-05-27T07:35:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.075+0000 7ffb37e9ab80 1 client.15825 shutdown 2024-05-27T07:35:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/3165075512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 msgr2=0x5627cd0e5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e5020 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ffb100aad00 tx=0x7ffb10009ca0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/3165075512 shutdown_connections 2024-05-27T07:35:44.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627cd15b630 0x5627cd15da50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627cd0d7820 0x5627cd159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627ccf58e40 0x5627cd0e4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 1 --2- 172.21.15.63:0/3165075512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627cd0d7410 0x5627cd0e5020 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb0cc00700 0 client.15825 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:44.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/3165075512 >> 172.21.15.63:0/3165075512 conn(0x5627ccfb8fc0 msgr2=0x5627cd0e1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/3165075512 shutdown_connections 2024-05-27T07:35:44.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 1 -- 172.21.15.63:0/3165075512 wait complete. 2024-05-27T07:35:44.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.076+0000 7ffb37e9ab80 20 client.15825 trim_cache size 0 max 16384 2024-05-27T07:35:44.073 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:35:44.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 -- 172.21.15.63:0/3541110359 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb080b29f0 msgr2=0x7ffb080b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb080b29f0 0x7ffb080b4f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5627cd0d3690 tx=0x7ffb1c009040 comp rx=0 tx=0).stop 2024-05-27T07:35:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 -- 172.21.15.63:0/3541110359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb24103870 msgr2=0x7ffb2419d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb24103870 0x7ffb2419d2e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ffb10000c00 tx=0x7ffb10005ef0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 -- 172.21.15.63:0/3541110359 shutdown_connections 2024-05-27T07:35:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb080b29f0 0x7ffb080b4f00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb24110ad0 0x7ffb241a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb24102f20 0x7ffb2419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 --2- 172.21.15.63:0/3541110359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb24103870 0x7ffb2419d2e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 -- 172.21.15.63:0/3541110359 >> 172.21.15.63:0/3541110359 conn(0x7ffb24078af0 msgr2=0x7ffb24111fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 -- 172.21.15.63:0/3541110359 shutdown_connections 2024-05-27T07:35:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:44.077+0000 7ffb28a00700 1 -- 172.21.15.63:0/3541110359 wait complete. 2024-05-27T07:35:44.154 INFO:tasks.workunit.client.0.smithi063.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-27T07:35:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.742+0000 7f9b5d400640 1 -- 172.21.15.63:0/2039208708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b58112160 msgr2=0x7f9b58114580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.742+0000 7f9b5d400640 1 --2- 172.21.15.63:0/2039208708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b58112160 0x7f9b58114580 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009900 tx=0x7f9b4002a480 comp rx=0 tx=0).stop 2024-05-27T07:35:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.742+0000 7f9b5d400640 1 -- 172.21.15.63:0/2039208708 shutdown_connections 2024-05-27T07:35:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.742+0000 7f9b5d400640 1 --2- 172.21.15.63:0/2039208708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b58114ac0 0x7f9b58116ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.742+0000 7f9b5d400640 1 --2- 172.21.15.63:0/2039208708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b58112160 0x7f9b58114580 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.742+0000 7f9b5d400640 1 --2- 172.21.15.63:0/2039208708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b58075480 0x7f9b58075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.742+0000 7f9b5d400640 1 -- 172.21.15.63:0/2039208708 >> 172.21.15.63:0/2039208708 conn(0x7f9b58079240 msgr2=0x7f9b58079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.742+0000 7f9b5d400640 1 -- 172.21.15.63:0/2039208708 shutdown_connections 2024-05-27T07:35:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.742+0000 7f9b5d400640 1 -- 172.21.15.63:0/2039208708 wait complete. 2024-05-27T07:35:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.744+0000 7f9b5d400640 1 Processor -- start 2024-05-27T07:35:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.744+0000 7f9b5d400640 1 -- start start 2024-05-27T07:35:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.744+0000 7f9b5d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b58075480 0x7f9b581a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.744+0000 7f9b5d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b58112160 0x7f9b581a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.744+0000 7f9b5d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b58114ac0 0x7f9b581a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.744+0000 7f9b5d400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5811a120 con 0x7f9b58075480 2024-05-27T07:35:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.744+0000 7f9b5d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b58119fa0 con 0x7f9b58114ac0 2024-05-27T07:35:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.744+0000 7f9b5d400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5811a2a0 con 0x7f9b58112160 2024-05-27T07:35:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b57e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b58075480 0x7f9b581a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b57e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b58075480 0x7f9b581a6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35964/0 (socket says 172.21.15.63:35964) 2024-05-27T07:35:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b57e00640 1 -- 172.21.15.63:0/1212128828 learned_addr learned my addr 172.21.15.63:0/1212128828 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b58114ac0 0x7f9b581a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b57400640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b58112160 0x7f9b581a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b57e00640 1 -- 172.21.15.63:0/1212128828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b58112160 msgr2=0x7f9b581a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b57e00640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b58112160 0x7f9b581a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b57e00640 1 -- 172.21.15.63:0/1212128828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b58114ac0 msgr2=0x7f9b581a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b57e00640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b58114ac0 0x7f9b581a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b57e00640 1 -- 172.21.15.63:0/1212128828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b581a1100 con 0x7f9b58075480 2024-05-27T07:35:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.745+0000 7f9b57e00640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b58075480 0x7f9b581a6850 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800d770 tx=0x7f9b4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.746+0000 7f9b54c00640 1 -- 172.21.15.63:0/1212128828 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48017070 con 0x7f9b58075480 2024-05-27T07:35:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.746+0000 7f9b54c00640 1 -- 172.21.15.63:0/1212128828 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b480054b0 con 0x7f9b58075480 2024-05-27T07:35:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.746+0000 7f9b5d400640 1 -- 172.21.15.63:0/1212128828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b581a13f0 con 0x7f9b58075480 2024-05-27T07:35:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.746+0000 7f9b54c00640 1 -- 172.21.15.63:0/1212128828 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48004c60 con 0x7f9b58075480 2024-05-27T07:35:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.746+0000 7f9b5d400640 1 -- 172.21.15.63:0/1212128828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b581ad6d0 con 0x7f9b58075480 2024-05-27T07:35:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.749+0000 7f9b659a4740 1 Processor -- start 2024-05-27T07:35:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.749+0000 7f9b659a4740 1 -- start start 2024-05-27T07:35:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.750+0000 7f9b54c00640 1 -- 172.21.15.63:0/1212128828 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9b48004e00 con 0x7f9b58075480 2024-05-27T07:35:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.750+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790df1b0 0x563079481a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.750+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790b62f0 0x563079481fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.750+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790be4d0 0x5630794824e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.750+0000 7f9b659a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5630793f2b90 con 0x5630790df1b0 2024-05-27T07:35:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.750+0000 7f9b659a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630793e2730 con 0x5630790b62f0 2024-05-27T07:35:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.750+0000 7f9b659a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5630793e6770 con 0x5630790be4d0 2024-05-27T07:35:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.750+0000 7f9b5ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790df1b0 0x563079481a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.750+0000 7f9b5ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790df1b0 0x563079481a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35976/0 (socket says 172.21.15.63:35976) 2024-05-27T07:35:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.750+0000 7f9b5ca00640 1 -- 172.21.15.63:0/2629374353 learned_addr learned my addr 172.21.15.63:0/2629374353 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.751+0000 7f9b57e00640 1 --2- 172.21.15.63:0/2629374353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790b62f0 0x563079481fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.751+0000 7f9b57400640 1 --2- 172.21.15.63:0/2629374353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790be4d0 0x5630794824e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.751+0000 7f9b5ca00640 1 -- 172.21.15.63:0/2629374353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790be4d0 msgr2=0x5630794824e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.751+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/2629374353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790be4d0 0x5630794824e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.751+0000 7f9b5ca00640 1 -- 172.21.15.63:0/2629374353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790b62f0 msgr2=0x563079481fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.751+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/2629374353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790b62f0 0x563079481fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.751+0000 7f9b5ca00640 1 -- 172.21.15.63:0/2629374353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307937b8b0 con 0x5630790df1b0 2024-05-27T07:35:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.751+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/2629374353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790df1b0 0x563079481a60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009e60 tx=0x7f9b50010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9449c4cb9d0968f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.752+0000 7f9b4d200640 1 -- 172.21.15.63:0/2629374353 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50016070 con 0x5630790df1b0 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.752+0000 7f9b4d200640 1 -- 172.21.15.63:0/2629374353 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b50004d20 con 0x5630790df1b0 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.752+0000 7f9b4d200640 1 -- 172.21.15.63:0/2629374353 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b500126b0 con 0x5630790df1b0 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.752+0000 7f9b54c00640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b3c0b29d0 0x7f9b3c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.752+0000 7f9b54c00640 1 -- 172.21.15.63:0/1212128828 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f9b480e24b0 con 0x7f9b58075480 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.752+0000 7f9b57e00640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b3c0b29d0 0x7f9b3c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.753+0000 7f9b659a4740 1 -- 172.21.15.63:0/2629374353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790df1b0 msgr2=0x563079481a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.753+0000 7f9b659a4740 1 --2- 172.21.15.63:0/2629374353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790df1b0 0x563079481a60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009e60 tx=0x7f9b50010740 comp rx=0 tx=0).stop 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.753+0000 7f9b57e00640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b3c0b29d0 0x7f9b3c0b4ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x56307947f550 tx=0x7f9b480e5c90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.753+0000 7f9b659a4740 1 -- 172.21.15.63:0/2629374353 shutdown_connections 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.753+0000 7f9b659a4740 1 --2- 172.21.15.63:0/2629374353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790be4d0 0x5630794824e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.753+0000 7f9b659a4740 1 --2- 172.21.15.63:0/2629374353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790b62f0 0x563079481fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.753+0000 7f9b659a4740 1 --2- 172.21.15.63:0/2629374353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790df1b0 0x563079481a60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.753+0000 7f9b659a4740 1 -- 172.21.15.63:0/2629374353 >> 172.21.15.63:0/2629374353 conn(0x5630790d3300 msgr2=0x56307947e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.753+0000 7f9b659a4740 1 -- 172.21.15.63:0/2629374353 shutdown_connections 2024-05-27T07:35:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.753+0000 7f9b659a4740 1 -- 172.21.15.63:0/2629374353 wait complete. 2024-05-27T07:35:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.754+0000 7f9b659a4740 1 -- start start 2024-05-27T07:35:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.754+0000 7f9b659a4740 1 Processor -- start 2024-05-27T07:35:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.754+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307948bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.754+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790be4d0 0x56307948c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790df1b0 0x563079497ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b659a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5630790c8790 con 0x5630790b62f0 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b659a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56307938abc0 con 0x5630790be4d0 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b659a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56307948eb90 con 0x5630790df1b0 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b659a4740 10 client.0 _ll_register_callbacks cb 0x563079368400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b5ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307948bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b5ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307948bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35992/0 (socket says 172.21.15.63:35992) 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b5ca00640 1 -- 172.21.15.63:0/3562055582 learned_addr learned my addr 172.21.15.63:0/3562055582 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b57400640 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790be4d0 0x56307948c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b5ca00640 1 -- 172.21.15.63:0/3562055582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790df1b0 msgr2=0x563079497ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790df1b0 0x563079497ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b5ca00640 1 -- 172.21.15.63:0/3562055582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790be4d0 msgr2=0x56307948c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790be4d0 0x56307948c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.755+0000 7f9b5ca00640 1 -- 172.21.15.63:0/3562055582 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630790f7210 con 0x5630790b62f0 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.756+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307948bb90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9b50005cc0 tx=0x7f9b50009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f27429df87cbc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.756+0000 7f9b45600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.756+0000 7f9b45600640 1 -- 172.21.15.63:0/3562055582 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50016050 con 0x5630790b62f0 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.756+0000 7f9b659a4740 1 -- 172.21.15.63:0/3562055582 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b50004d20 con 0x5630790b62f0 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.756+0000 7f9b45600640 1 -- 172.21.15.63:0/3562055582 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f9b500053e0 con 0x5630790b62f0 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b45600640 1 -- 172.21.15.63:0/3562055582 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9b50030050 con 0x5630790b62f0 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b659a4740 10 client.15837 fetch_fsmap learned FSMap version 28 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b659a4740 1 -- 172.21.15.63:0/3562055582 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630791c3620 con 0x5630790b62f0 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b45600640 1 -- 172.21.15.63:0/3562055582 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b50004d20 con 0x5630790b62f0 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b45600640 1 -- 172.21.15.63:0/3562055582 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9b5000f040 con 0x5630790b62f0 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b659a4740 10 client.15837 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b659a4740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b659a4740 20 client.15837 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b659a4740 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56307909dd80 0x56307949dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b659a4740 4 client.15837 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.757+0000 7f9b659a4740 1 -- 172.21.15.63:0/3562055582 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630793f4ae0 con 0x56307909dd80 2024-05-27T07:35:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.758+0000 7f9b57400640 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56307909dd80 0x56307949dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.759+0000 7f9b57400640 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56307909dd80 0x56307949dcf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6ff4c880a00f6a server_cookie=9b96118a445d5cac in_seq=0 out_seq=0 2024-05-27T07:35:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.759+0000 7f9b45600640 10 client.15837 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:44.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b45600640 1 -- 172.21.15.63:0/3562055582 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5630793f4ae0 con 0x56307909dd80 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b45600640 10 client.15837 handle_command_reply: tid=0 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 client.15837 shutdown 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 -- 172.21.15.63:0/3562055582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 msgr2=0x56307948bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307948bb90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9b50005cc0 tx=0x7f9b50009ba0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 -- 172.21.15.63:0/3562055582 shutdown_connections 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56307909dd80 0x56307949dcf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790df1b0 0x563079497ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790be4d0 0x56307948c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b45600640 0 client.15837 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 --2- 172.21.15.63:0/3562055582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307948bb90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 -- 172.21.15.63:0/3562055582 >> 172.21.15.63:0/3562055582 conn(0x5630790d3300 msgr2=0x563079481890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 -- 172.21.15.63:0/3562055582 shutdown_connections 2024-05-27T07:35:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.761+0000 7f9b659a4740 1 -- 172.21.15.63:0/3562055582 wait complete. 2024-05-27T07:35:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.762+0000 7f9b659a4740 20 client.15837 trim_cache size 0 max 16384 2024-05-27T07:35:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.778+0000 7f9b659a4740 1 Processor -- start 2024-05-27T07:35:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.779+0000 7f9b659a4740 1 -- start start 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.779+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790b62f0 0x56307947d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.779+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790be4d0 0x56307948e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.779+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790df1b0 0x563079497e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.779+0000 7f9b659a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b9ea0 con 0x5630790be4d0 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.779+0000 7f9b659a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b7770 con 0x5630790b62f0 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.779+0000 7f9b659a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b55c0 con 0x5630790df1b0 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.779+0000 7f9b57400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790be4d0 0x56307948e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.779+0000 7f9b57400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790be4d0 0x56307948e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36004/0 (socket says 172.21.15.63:36004) 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.779+0000 7f9b57400640 1 -- 172.21.15.63:0/959733286 learned_addr learned my addr 172.21.15.63:0/959733286 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.780+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/959733286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790b62f0 0x56307947d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.780+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/959733286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790df1b0 0x563079497e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.780+0000 7f9b57400640 1 -- 172.21.15.63:0/959733286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790df1b0 msgr2=0x563079497e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.780+0000 7f9b57400640 1 --2- 172.21.15.63:0/959733286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790df1b0 0x563079497e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.780+0000 7f9b57400640 1 -- 172.21.15.63:0/959733286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790b62f0 msgr2=0x56307947d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.780+0000 7f9b57400640 1 --2- 172.21.15.63:0/959733286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790b62f0 0x56307947d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.780+0000 7f9b57400640 1 -- 172.21.15.63:0/959733286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307948a0e0 con 0x5630790be4d0 2024-05-27T07:35:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.780+0000 7f9b57400640 1 --2- 172.21.15.63:0/959733286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790be4d0 0x56307948e2c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9b4002a9a0 tx=0x7f9b40009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7706dc48eab6ca70 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.780+0000 7f9b47e00640 1 -- 172.21.15.63:0/959733286 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40048070 con 0x5630790be4d0 2024-05-27T07:35:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.781+0000 7f9b47e00640 1 -- 172.21.15.63:0/959733286 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b400054d0 con 0x5630790be4d0 2024-05-27T07:35:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.781+0000 7f9b47e00640 1 -- 172.21.15.63:0/959733286 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40035070 con 0x5630790be4d0 2024-05-27T07:35:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.781+0000 7f9b659a4740 1 -- 172.21.15.63:0/959733286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790be4d0 msgr2=0x56307948e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.781+0000 7f9b659a4740 1 --2- 172.21.15.63:0/959733286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790be4d0 0x56307948e2c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9b4002a9a0 tx=0x7f9b40009640 comp rx=0 tx=0).stop 2024-05-27T07:35:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.781+0000 7f9b659a4740 1 -- 172.21.15.63:0/959733286 shutdown_connections 2024-05-27T07:35:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.781+0000 7f9b659a4740 1 --2- 172.21.15.63:0/959733286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790df1b0 0x563079497e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.781+0000 7f9b659a4740 1 --2- 172.21.15.63:0/959733286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790be4d0 0x56307948e2c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.781+0000 7f9b659a4740 1 --2- 172.21.15.63:0/959733286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790b62f0 0x56307947d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.781+0000 7f9b659a4740 1 -- 172.21.15.63:0/959733286 >> 172.21.15.63:0/959733286 conn(0x5630790d3300 msgr2=0x56307948b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.782+0000 7f9b659a4740 1 -- 172.21.15.63:0/959733286 shutdown_connections 2024-05-27T07:35:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.782+0000 7f9b659a4740 1 -- 172.21.15.63:0/959733286 wait complete. 2024-05-27T07:35:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.782+0000 7f9b659a4740 1 -- start start 2024-05-27T07:35:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.782+0000 7f9b659a4740 1 Processor -- start 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307949dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790be4d0 0x56307949e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b659a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790df1b0 0x56307949e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b659a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b5c30 con 0x5630790b62f0 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b659a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b70a0 con 0x5630790df1b0 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b659a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9b581b6300 con 0x5630790be4d0 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b659a4740 10 client.0 _ll_register_callbacks cb 0x7f9b50009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b5ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307949dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b5ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307949dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36006/0 (socket says 172.21.15.63:36006) 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b5ca00640 1 -- 172.21.15.63:0/2648437366 learned_addr learned my addr 172.21.15.63:0/2648437366 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.783+0000 7f9b57400640 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790be4d0 0x56307949e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.784+0000 7f9b5ca00640 1 -- 172.21.15.63:0/2648437366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790be4d0 msgr2=0x56307949e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.784+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790be4d0 0x56307949e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.784+0000 7f9b5ca00640 1 -- 172.21.15.63:0/2648437366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790df1b0 msgr2=0x56307949e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.784+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790df1b0 0x56307949e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.784+0000 7f9b5ca00640 1 -- 172.21.15.63:0/2648437366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630795042b0 con 0x5630790b62f0 2024-05-27T07:35:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.784+0000 7f9b5ca00640 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307949dc90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000a010 tx=0x7f9b50025980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0d6996b6ebede0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.784+0000 7f9b45600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.784+0000 7f9b45600640 1 -- 172.21.15.63:0/2648437366 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50016040 con 0x5630790b62f0 2024-05-27T07:35:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.784+0000 7f9b45600640 1 -- 172.21.15.63:0/2648437366 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f9b50004c50 con 0x5630790b62f0 2024-05-27T07:35:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.784+0000 7f9b659a4740 1 -- 172.21.15.63:0/2648437366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b400054d0 con 0x5630790b62f0 2024-05-27T07:35:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b45600640 1 -- 172.21.15.63:0/2648437366 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9b50043050 con 0x5630790b62f0 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b659a4740 10 client.15843 fetch_fsmap learned FSMap version 28 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b659a4740 1 -- 172.21.15.63:0/2648437366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563079505520 con 0x5630790b62f0 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b45600640 1 -- 172.21.15.63:0/2648437366 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b400054d0 con 0x5630790b62f0 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b45600640 1 -- 172.21.15.63:0/2648437366 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9b5003e030 con 0x5630790b62f0 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b659a4740 10 client.15843 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b659a4740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b659a4740 20 client.15843 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b659a4740 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630790731c0 0x563079507cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b659a4740 4 client.15843 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.785+0000 7f9b659a4740 1 -- 172.21.15.63:0/2648437366 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630793ee2c0 con 0x5630790731c0 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.786+0000 7f9b57400640 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630790731c0 0x563079507cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.787+0000 7f9b57400640 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630790731c0 0x563079507cb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24bbe7c1c5e9ab69 server_cookie=7812192c781ecfc5 in_seq=0 out_seq=0 2024-05-27T07:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.787+0000 7f9b45600640 10 client.15843 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.971+0000 7f9b45600640 1 -- 172.21.15.63:0/2648437366 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5630793ee2c0 con 0x5630790731c0 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.971+0000 7f9b45600640 10 client.15843 handle_command_reply: tid=0 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.971+0000 7f9b659a4740 1 client.15843 shutdown 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 1 -- 172.21.15.63:0/2648437366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 msgr2=0x56307949dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307949dc90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000a010 tx=0x7f9b50025980 comp rx=0 tx=0).stop 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 1 -- 172.21.15.63:0/2648437366 shutdown_connections 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630790731c0 0x563079507cb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630790df1b0 0x56307949e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630790be4d0 0x56307949e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b45600640 0 client.15843 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 1 --2- 172.21.15.63:0/2648437366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630790b62f0 0x56307949dc90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 1 -- 172.21.15.63:0/2648437366 >> 172.21.15.63:0/2648437366 conn(0x5630790d3300 msgr2=0x56307948be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 1 -- 172.21.15.63:0/2648437366 shutdown_connections 2024-05-27T07:35:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 1 -- 172.21.15.63:0/2648437366 wait complete. 2024-05-27T07:35:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.972+0000 7f9b659a4740 20 client.15843 trim_cache size 0 max 16384 2024-05-27T07:35:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:35:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:35:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:35:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 -- 172.21.15.63:0/1212128828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b3c0b29d0 msgr2=0x7f9b3c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b3c0b29d0 0x7f9b3c0b4ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x56307947f550 tx=0x7f9b480e5c90 comp rx=0 tx=0).stop 2024-05-27T07:35:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 -- 172.21.15.63:0/1212128828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b58075480 msgr2=0x7f9b581a6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b58075480 0x7f9b581a6850 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800d770 tx=0x7f9b4800dc30 comp rx=0 tx=0).stop 2024-05-27T07:35:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 -- 172.21.15.63:0/1212128828 shutdown_connections 2024-05-27T07:35:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b3c0b29d0 0x7f9b3c0b4ee0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b58114ac0 0x7f9b581a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b58112160 0x7f9b581a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 --2- 172.21.15.63:0/1212128828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b58075480 0x7f9b581a6850 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 -- 172.21.15.63:0/1212128828 >> 172.21.15.63:0/1212128828 conn(0x7f9b58079240 msgr2=0x7f9b581125d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 -- 172.21.15.63:0/1212128828 shutdown_connections 2024-05-27T07:35:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:44.973+0000 7f9b5d400640 1 -- 172.21.15.63:0/1212128828 wait complete. 2024-05-27T07:35:45.190 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether build environment is sane... yes 2024-05-27T07:35:45.248 INFO:tasks.workunit.client.0.smithi063.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-27T07:35:45.250 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gawk... gawk 2024-05-27T07:35:45.329 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether make sets $(MAKE)... yes 2024-05-27T07:35:45.488 INFO:tasks.workunit.client.0.smithi063.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-27T07:35:45.488 INFO:tasks.workunit.client.0.smithi063.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-27T07:35:45.549 INFO:tasks.workunit.client.0.smithi063.stdout:checking for style of include used by make... GNU 2024-05-27T07:35:45.575 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gcc... gcc 2024-05-27T07:35:45.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:45 smithi086 ceph-mon[34687]: pgmap v68: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 133 KiB/s wr, 23 op/s 2024-05-27T07:35:45.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:45 smithi149 ceph-mon[35408]: pgmap v68: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 133 KiB/s wr, 23 op/s 2024-05-27T07:35:45.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:45 smithi063 ceph-mon[31613]: pgmap v68: 129 pgs: 129 active+clean; 2.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 133 KiB/s wr, 23 op/s 2024-05-27T07:35:45.901 INFO:tasks.workunit.client.0.smithi063.stdout:checking for C compiler default output file name... a.out 2024-05-27T07:35:45.918 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether the C compiler works... yes 2024-05-27T07:35:45.934 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether we are cross compiling... no 2024-05-27T07:35:45.991 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:35:46.009 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:35:46.090 INFO:tasks.workunit.client.0.smithi063.stdout:checking for suffix of executables... 2024-05-27T07:35:46.230 INFO:tasks.workunit.client.0.smithi063.stdout:checking for suffix of object files... o 2024-05-27T07:35:46.363 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether we are using the GNU C compiler... yes 2024-05-27T07:35:46.477 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether gcc accepts -g... yes 2024-05-27T07:35:46.483 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:46.747 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-27T07:35:47.346 INFO:tasks.workunit.client.0.smithi063.stdout:checking dependency style of gcc... gcc3 2024-05-27T07:35:47.617 INFO:tasks.workunit.client.0.smithi063.stdout:checking how to run the C preprocessor... gcc -E 2024-05-27T07:35:47.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:47 smithi063 ceph-mon[31613]: pgmap v69: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 286 KiB/s wr, 118 op/s 2024-05-27T07:35:47.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:47 smithi086 ceph-mon[34687]: pgmap v69: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 286 KiB/s wr, 118 op/s 2024-05-27T07:35:47.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:47 smithi149 ceph-mon[35408]: pgmap v69: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 286 KiB/s wr, 118 op/s 2024-05-27T07:35:47.870 INFO:tasks.workunit.client.0.smithi063.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-27T07:35:47.877 INFO:tasks.workunit.client.0.smithi063.stdout:checking for egrep... /usr/bin/grep -E 2024-05-27T07:35:47.925 INFO:tasks.workunit.client.0.smithi063.stdout:checking for AIX... no 2024-05-27T07:35:48.016 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gcc... (cached) gcc 2024-05-27T07:35:48.065 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-27T07:35:48.069 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-27T07:35:48.070 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-27T07:35:48.071 INFO:tasks.workunit.client.0.smithi063.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-27T07:35:48.327 INFO:tasks.workunit.client.0.smithi063.stdout:checking for main in -lm... yes 2024-05-27T07:35:48.624 INFO:tasks.workunit.client.0.smithi063.stdout:checking for main in -lpthread... yes 2024-05-27T07:35:48.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.978+0000 7f601a200700 1 -- 172.21.15.63:0/2083431348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6014105320 msgr2=0x7f6014105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:48.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.978+0000 7f601a200700 1 --2- 172.21.15.63:0/2083431348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6014105320 0x7f6014105730 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6008009990 tx=0x7f6008009ca0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.978+0000 7f601a200700 1 -- 172.21.15.63:0/2083431348 shutdown_connections 2024-05-27T07:35:48.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.978+0000 7f601a200700 1 --2- 172.21.15.63:0/2083431348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f601410e700 0x7f6014110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.978+0000 7f601a200700 1 --2- 172.21.15.63:0/2083431348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6014105320 0x7f6014105730 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.978+0000 7f601a200700 1 --2- 172.21.15.63:0/2083431348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6014105c70 0x7f601410e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.978+0000 7f601a200700 1 -- 172.21.15.63:0/2083431348 >> 172.21.15.63:0/2083431348 conn(0x7f6014078ab0 msgr2=0x7f6014078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:48.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.978+0000 7f601a200700 1 -- 172.21.15.63:0/2083431348 shutdown_connections 2024-05-27T07:35:48.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.978+0000 7f601a200700 1 -- 172.21.15.63:0/2083431348 wait complete. 2024-05-27T07:35:48.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.979+0000 7f601a200700 1 Processor -- start 2024-05-27T07:35:48.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.980+0000 7f601a200700 1 -- start start 2024-05-27T07:35:48.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.980+0000 7f601a200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6014105320 0x7f601419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.980+0000 7f601a200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6014105c70 0x7f601419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.980+0000 7f601a200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601410e700 0x7f60141a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.980+0000 7f601a200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6014114140 con 0x7f601410e700 2024-05-27T07:35:48.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.980+0000 7f601a200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6014113fc0 con 0x7f6014105320 2024-05-27T07:35:48.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.980+0000 7f601a200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60141142c0 con 0x7f6014105c70 2024-05-27T07:35:48.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6019800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601410e700 0x7f60141a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:48.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6019800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601410e700 0x7f60141a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57818/0 (socket says 172.21.15.63:57818) 2024-05-27T07:35:48.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6019800700 1 -- 172.21.15.63:0/1332219818 learned_addr learned my addr 172.21.15.63:0/1332219818 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:48.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6018e00700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6014105320 0x7f601419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:48.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6013e00700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6014105c70 0x7f601419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6019800700 1 -- 172.21.15.63:0/1332219818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6014105c70 msgr2=0x7f601419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6019800700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6014105c70 0x7f601419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6019800700 1 -- 172.21.15.63:0/1332219818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6014105320 msgr2=0x7f601419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6019800700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6014105320 0x7f601419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6019800700 1 -- 172.21.15.63:0/1332219818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60141a1d80 con 0x7f601410e700 2024-05-27T07:35:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.981+0000 7f6013e00700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6014105c70 0x7f601419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:48.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.982+0000 7f6019800700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601410e700 0x7f60141a16e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f600c007ee0 tx=0x7f600c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.982+0000 7f6011600700 1 -- 172.21.15.63:0/1332219818 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c00ddd0 con 0x7f601410e700 2024-05-27T07:35:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.982+0000 7f6011600700 1 -- 172.21.15.63:0/1332219818 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f600c004020 con 0x7f601410e700 2024-05-27T07:35:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.982+0000 7f601a200700 1 -- 172.21.15.63:0/1332219818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60141a2070 con 0x7f601410e700 2024-05-27T07:35:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.982+0000 7f6011600700 1 -- 172.21.15.63:0/1332219818 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c015660 con 0x7f601410e700 2024-05-27T07:35:48.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.982+0000 7f601a200700 1 -- 172.21.15.63:0/1332219818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60141a9990 con 0x7f601410e700 2024-05-27T07:35:48.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.988+0000 7f6011600700 1 -- 172.21.15.63:0/1332219818 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f600c015800 con 0x7f601410e700 2024-05-27T07:35:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.989+0000 7f602a412b80 1 Processor -- start 2024-05-27T07:35:48.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.989+0000 7f602a412b80 1 -- start start 2024-05-27T07:35:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.990+0000 7f602a412b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f6efa70 0x55706f783710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.990+0000 7f602a412b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f783c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.990+0000 7f602a412b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f797f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.990+0000 7f602a412b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55706f6b0ec0 con 0x55706f603ec0 2024-05-27T07:35:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.990+0000 7f602a412b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55706f6a0a60 con 0x55706f795b50 2024-05-27T07:35:48.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.990+0000 7f602a412b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55706f6a4aa0 con 0x55706f6efa70 2024-05-27T07:35:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.990+0000 7f6018e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f783c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.990+0000 7f6018e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f783c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57826/0 (socket says 172.21.15.63:57826) 2024-05-27T07:35:48.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.990+0000 7f6018e00700 1 -- 172.21.15.63:0/1012236124 learned_addr learned my addr 172.21.15.63:0/1012236124 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.990+0000 7f6011600700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ffc0b2a70 0x7f5ffc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.991+0000 7f6011600700 1 -- 172.21.15.63:0/1332219818 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f600c0e2390 con 0x7f601410e700 2024-05-27T07:35:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.991+0000 7f6019800700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ffc0b2a70 0x7f5ffc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.991+0000 7f6018e00700 1 -- 172.21.15.63:0/1012236124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f6efa70 msgr2=0x55706f783710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:48.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.991+0000 7f6018e00700 1 --2- 172.21.15.63:0/1012236124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f6efa70 0x55706f783710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.991+0000 7f6018e00700 1 -- 172.21.15.63:0/1012236124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 msgr2=0x55706f797f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:35:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.991+0000 7f6018e00700 1 --2- 172.21.15.63:0/1012236124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f797f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.991+0000 7f6018e00700 1 -- 172.21.15.63:0/1012236124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706f6694a0 con 0x55706f603ec0 2024-05-27T07:35:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.992+0000 7f6019800700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ffc0b2a70 0x7f5ffc0b4f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f600c0a9290 tx=0x7f600c004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.992+0000 7f6018e00700 1 --2- 172.21.15.63:0/1012236124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f783c50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6008009750 tx=0x7f6008003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749f231624f6263a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.992+0000 7f6006000700 1 -- 172.21.15.63:0/1012236124 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008027070 con 0x55706f603ec0 2024-05-27T07:35:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.992+0000 7f6006000700 1 -- 172.21.15.63:0/1012236124 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60080037c0 con 0x55706f603ec0 2024-05-27T07:35:48.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.992+0000 7f6006000700 1 -- 172.21.15.63:0/1012236124 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600800baa0 con 0x55706f603ec0 2024-05-27T07:35:48.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.994+0000 7f602a412b80 1 -- 172.21.15.63:0/1012236124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 msgr2=0x55706f783c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:48.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.994+0000 7f602a412b80 1 --2- 172.21.15.63:0/1012236124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f783c50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6008009750 tx=0x7f6008003680 comp rx=0 tx=0).stop 2024-05-27T07:35:48.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.994+0000 7f602a412b80 1 -- 172.21.15.63:0/1012236124 shutdown_connections 2024-05-27T07:35:48.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.994+0000 7f602a412b80 1 --2- 172.21.15.63:0/1012236124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f797f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.994+0000 7f602a412b80 1 --2- 172.21.15.63:0/1012236124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f6efa70 0x55706f783710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.994+0000 7f602a412b80 1 --2- 172.21.15.63:0/1012236124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f783c50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.994+0000 7f602a412b80 1 -- 172.21.15.63:0/1012236124 >> 172.21.15.63:0/1012236124 conn(0x55706f69d410 msgr2=0x55706f77e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:48.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.994+0000 7f602a412b80 1 -- 172.21.15.63:0/1012236124 shutdown_connections 2024-05-27T07:35:48.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.994+0000 7f602a412b80 1 -- 172.21.15.63:0/1012236124 wait complete. 2024-05-27T07:35:48.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.995+0000 7f602a412b80 1 -- start start 2024-05-27T07:35:48.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.995+0000 7f602a412b80 1 Processor -- start 2024-05-27T07:35:48.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.996+0000 7f602a412b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 0x55706f792b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.996+0000 7f602a412b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f793070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.996+0000 7f602a412b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f7935b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.996+0000 7f602a412b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55706f5a7020 con 0x55706f6efa70 2024-05-27T07:35:48.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.996+0000 7f602a412b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55706f6417e0 con 0x55706f795b50 2024-05-27T07:35:48.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.996+0000 7f602a412b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55706f7985a0 con 0x55706f603ec0 2024-05-27T07:35:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.996+0000 7f602a412b80 10 client.0 _ll_register_callbacks cb 0x55706f64acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.997+0000 7f6013e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 0x55706f792b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.997+0000 7f6013e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 0x55706f792b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53702/0 (socket says 172.21.15.63:53702) 2024-05-27T07:35:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.997+0000 7f6013e00700 1 -- 172.21.15.63:0/2316790557 learned_addr learned my addr 172.21.15.63:0/2316790557 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.997+0000 7f6013e00700 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f7935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:48.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.997+0000 7f6018e00700 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f793070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.998+0000 7f6018e00700 1 -- 172.21.15.63:0/2316790557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 msgr2=0x55706f792b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.998+0000 7f6018e00700 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 0x55706f792b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.998+0000 7f6018e00700 1 -- 172.21.15.63:0/2316790557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 msgr2=0x55706f7935b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.998+0000 7f6018e00700 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f7935b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.998+0000 7f6018e00700 1 -- 172.21.15.63:0/2316790557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706f5f0950 con 0x55706f6efa70 2024-05-27T07:35:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.998+0000 7f6018e00700 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f793070 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55706f77fc20 tx=0x7f6008003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa525694fe1f18f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:48.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.998+0000 7f5ffaa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.999+0000 7f5ffaa00700 1 -- 172.21.15.63:0/2316790557 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008031060 con 0x55706f6efa70 2024-05-27T07:35:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.999+0000 7f602a412b80 1 -- 172.21.15.63:0/2316790557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60080037c0 con 0x55706f6efa70 2024-05-27T07:35:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.999+0000 7f5ffaa00700 1 -- 172.21.15.63:0/2316790557 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f6008005410 con 0x55706f6efa70 2024-05-27T07:35:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.999+0000 7f5ffaa00700 1 -- 172.21.15.63:0/2316790557 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6008038050 con 0x55706f6efa70 2024-05-27T07:35:48.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.999+0000 7f602a412b80 10 client.15855 fetch_fsmap learned FSMap version 28 2024-05-27T07:35:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:48.999+0000 7f602a412b80 1 -- 172.21.15.63:0/2316790557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55706f6710e0 con 0x55706f6efa70 2024-05-27T07:35:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.000+0000 7f5ffaa00700 1 -- 172.21.15.63:0/2316790557 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60080037c0 con 0x55706f6efa70 2024-05-27T07:35:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.000+0000 7f5ffaa00700 1 -- 172.21.15.63:0/2316790557 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6008022510 con 0x55706f6efa70 2024-05-27T07:35:48.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.000+0000 7f602a412b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:35:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.000+0000 7f602a412b80 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.000+0000 7f602a412b80 20 client.15855 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.000+0000 7f602a412b80 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55706f7a5450 0x55706f7a7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.000+0000 7f602a412b80 4 client.15855 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.000+0000 7f602a412b80 1 -- 172.21.15.63:0/2316790557 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55706f6b2e10 con 0x55706f7a5450 2024-05-27T07:35:48.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.001+0000 7f6013e00700 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55706f7a5450 0x55706f7a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:48.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.002+0000 7f6013e00700 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55706f7a5450 0x55706f7a7870 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f2c5f0537a10d8 server_cookie=eb95cbb7c08f8d54 in_seq=0 out_seq=0 2024-05-27T07:35:48.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.002+0000 7f5ffaa00700 10 client.15855 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:49.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.005+0000 7f5ffaa00700 1 -- 172.21.15.63:0/2316790557 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55706f6b2e10 con 0x55706f7a5450 2024-05-27T07:35:49.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.005+0000 7f5ffaa00700 10 client.15855 handle_command_reply: tid=0 2024-05-27T07:35:49.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.005+0000 7f602a412b80 1 client.15855 shutdown 2024-05-27T07:35:49.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f602a412b80 1 -- 172.21.15.63:0/2316790557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 msgr2=0x55706f793070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:49.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f602a412b80 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f793070 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55706f77fc20 tx=0x7f6008003d30 comp rx=0 tx=0).stop 2024-05-27T07:35:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f602a412b80 1 -- 172.21.15.63:0/2316790557 shutdown_connections 2024-05-27T07:35:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f602a412b80 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55706f7a5450 0x55706f7a7870 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f602a412b80 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f7935b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f602a412b80 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 0x55706f792b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f602a412b80 1 --2- 172.21.15.63:0/2316790557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f793070 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f602a412b80 1 -- 172.21.15.63:0/2316790557 >> 172.21.15.63:0/2316790557 conn(0x55706f69d410 msgr2=0x55706f783710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f5ffaa00700 0 client.15855 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f602a412b80 1 -- 172.21.15.63:0/2316790557 shutdown_connections 2024-05-27T07:35:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.006+0000 7f602a412b80 1 -- 172.21.15.63:0/2316790557 wait complete. 2024-05-27T07:35:49.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.007+0000 7f602a412b80 20 client.15855 trim_cache size 0 max 16384 2024-05-27T07:35:49.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.032+0000 7f602a412b80 1 Processor -- start 2024-05-27T07:35:49.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.033+0000 7f602a412b80 1 -- start start 2024-05-27T07:35:49.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.033+0000 7f602a412b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f77e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:49.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.033+0000 7f602a412b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f6efa70 0x55706f7948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:49.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.033+0000 7f602a412b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f795b50 0x55706f78fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:49.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.033+0000 7f602a412b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60141b07d0 con 0x55706f603ec0 2024-05-27T07:35:49.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.033+0000 7f602a412b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60141ae060 con 0x55706f6efa70 2024-05-27T07:35:49.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.033+0000 7f602a412b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60141abf90 con 0x55706f795b50 2024-05-27T07:35:49.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.033+0000 7f6013e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f77e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:49.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.033+0000 7f6013e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f77e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57838/0 (socket says 172.21.15.63:57838) 2024-05-27T07:35:49.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.033+0000 7f6013e00700 1 -- 172.21.15.63:0/1651811899 learned_addr learned my addr 172.21.15.63:0/1651811899 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:49.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.034+0000 7f6018e00700 1 --2- 172.21.15.63:0/1651811899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f6efa70 0x55706f7948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:49.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.034+0000 7f6013e00700 1 -- 172.21.15.63:0/1651811899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f795b50 msgr2=0x55706f78fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:49.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.034+0000 7f6013e00700 1 --2- 172.21.15.63:0/1651811899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f795b50 0x55706f78fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.034+0000 7f6013e00700 1 -- 172.21.15.63:0/1651811899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f6efa70 msgr2=0x55706f7948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:49.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.034+0000 7f6013e00700 1 --2- 172.21.15.63:0/1651811899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f6efa70 0x55706f7948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.034+0000 7f6013e00700 1 -- 172.21.15.63:0/1651811899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706f7985d0 con 0x55706f603ec0 2024-05-27T07:35:49.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.034+0000 7f6013e00700 1 --2- 172.21.15.63:0/1651811899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f77e7c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f60000140a0 tx=0x7f60000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebceb5a970ba0383 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:49.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.035+0000 7f6005600700 1 -- 172.21.15.63:0/1651811899 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000023020 con 0x55706f603ec0 2024-05-27T07:35:49.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.035+0000 7f6005600700 1 -- 172.21.15.63:0/1651811899 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6000014d10 con 0x55706f603ec0 2024-05-27T07:35:49.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.035+0000 7f6005600700 1 -- 172.21.15.63:0/1651811899 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000012740 con 0x55706f603ec0 2024-05-27T07:35:49.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.036+0000 7f602a412b80 1 -- 172.21.15.63:0/1651811899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 msgr2=0x55706f77e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:49.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.036+0000 7f602a412b80 1 --2- 172.21.15.63:0/1651811899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f77e7c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f60000140a0 tx=0x7f60000047c0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.036+0000 7f602a412b80 1 -- 172.21.15.63:0/1651811899 shutdown_connections 2024-05-27T07:35:49.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.036+0000 7f602a412b80 1 --2- 172.21.15.63:0/1651811899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f795b50 0x55706f78fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.036+0000 7f602a412b80 1 --2- 172.21.15.63:0/1651811899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f603ec0 0x55706f77e7c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.036+0000 7f602a412b80 1 --2- 172.21.15.63:0/1651811899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f6efa70 0x55706f7948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.036+0000 7f602a412b80 1 -- 172.21.15.63:0/1651811899 >> 172.21.15.63:0/1651811899 conn(0x55706f69d410 msgr2=0x55706f7941e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:49.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.036+0000 7f602a412b80 1 -- 172.21.15.63:0/1651811899 shutdown_connections 2024-05-27T07:35:49.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.036+0000 7f602a412b80 1 -- 172.21.15.63:0/1651811899 wait complete. 2024-05-27T07:35:49.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.037+0000 7f602a412b80 1 -- start start 2024-05-27T07:35:49.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.037+0000 7f602a412b80 1 Processor -- start 2024-05-27T07:35:49.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.037+0000 7f602a412b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 0x55706f7aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:49.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.037+0000 7f602a412b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f7a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.037+0000 7f602a412b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f7a9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.037+0000 7f602a412b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60141ac540 con 0x55706f6efa70 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.037+0000 7f602a412b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60141ad990 con 0x55706f795b50 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.037+0000 7f602a412b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60141acbf0 con 0x55706f603ec0 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.037+0000 7f602a412b80 10 client.0 _ll_register_callbacks cb 0x55706f64acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.038+0000 7f6018e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f7a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.038+0000 7f6018e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f7a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57846/0 (socket says 172.21.15.63:57846) 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.038+0000 7f6018e00700 1 -- 172.21.15.63:0/1751068585 learned_addr learned my addr 172.21.15.63:0/1751068585 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.038+0000 7f6013e00700 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 0x55706f7aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.038+0000 7f6013e00700 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f7a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.038+0000 7f6018e00700 1 -- 172.21.15.63:0/1751068585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 msgr2=0x55706f7aaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.038+0000 7f6018e00700 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 0x55706f7aaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.038+0000 7f6018e00700 1 -- 172.21.15.63:0/1751068585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 msgr2=0x55706f7a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:49.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.038+0000 7f6018e00700 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f7a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.038+0000 7f6018e00700 1 -- 172.21.15.63:0/1751068585 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706f7a99c0 con 0x55706f6efa70 2024-05-27T07:35:49.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.039+0000 7f6018e00700 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f7a8e40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55706f77acd0 tx=0x7f600800b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca91c4dfa5e4a761 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:49.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.039+0000 7f5ffaa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:49.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.039+0000 7f5ffaa00700 1 -- 172.21.15.63:0/1751068585 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008031040 con 0x55706f6efa70 2024-05-27T07:35:49.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.039+0000 7f602a412b80 1 -- 172.21.15.63:0/1751068585 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6000014d10 con 0x55706f6efa70 2024-05-27T07:35:49.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.039+0000 7f5ffaa00700 1 -- 172.21.15.63:0/1751068585 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f6008005210 con 0x55706f6efa70 2024-05-27T07:35:49.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.039+0000 7f5ffaa00700 1 -- 172.21.15.63:0/1751068585 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6008027040 con 0x55706f6efa70 2024-05-27T07:35:49.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.039+0000 7f602a412b80 10 client.15861 fetch_fsmap learned FSMap version 28 2024-05-27T07:35:49.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.039+0000 7f602a412b80 1 -- 172.21.15.63:0/1751068585 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55706f810ac0 con 0x55706f6efa70 2024-05-27T07:35:49.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.040+0000 7f5ffaa00700 1 -- 172.21.15.63:0/1751068585 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6000014d10 con 0x55706f6efa70 2024-05-27T07:35:49.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.040+0000 7f5ffaa00700 1 -- 172.21.15.63:0/1751068585 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f600800ba60 con 0x55706f6efa70 2024-05-27T07:35:49.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.040+0000 7f602a412b80 10 client.15861 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:35:49.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.040+0000 7f602a412b80 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:49.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.040+0000 7f602a412b80 20 client.15861 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:49.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.040+0000 7f602a412b80 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55706f811000 0x55706f813420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:49.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.040+0000 7f602a412b80 4 client.15861 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:35:49.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.040+0000 7f602a412b80 1 -- 172.21.15.63:0/1751068585 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55706f6ac5f0 con 0x55706f811000 2024-05-27T07:35:49.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.041+0000 7f6013e00700 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55706f811000 0x55706f813420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:49.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.042+0000 7f6013e00700 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55706f811000 0x55706f813420 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368ffd8f7460753f server_cookie=112cfdc5fcde0081 in_seq=0 out_seq=0 2024-05-27T07:35:49.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.042+0000 7f5ffaa00700 10 client.15861 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:49.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f5ffaa00700 1 -- 172.21.15.63:0/1751068585 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55706f6ac5f0 con 0x55706f811000 2024-05-27T07:35:49.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f5ffaa00700 10 client.15861 handle_command_reply: tid=0 2024-05-27T07:35:49.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 client.15861 shutdown 2024-05-27T07:35:49.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 -- 172.21.15.63:0/1751068585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 msgr2=0x55706f7a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:49.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f7a8e40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55706f77acd0 tx=0x7f600800b6f0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 -- 172.21.15.63:0/1751068585 shutdown_connections 2024-05-27T07:35:49.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55706f811000 0x55706f813420 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55706f795b50 0x55706f7a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55706f603ec0 0x55706f7aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 --2- 172.21.15.63:0/1751068585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55706f6efa70 0x55706f7a8e40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 -- 172.21.15.63:0/1751068585 >> 172.21.15.63:0/1751068585 conn(0x55706f69d410 msgr2=0x55706f793340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:49.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f5ffaa00700 0 client.15861 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:49.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 -- 172.21.15.63:0/1751068585 shutdown_connections 2024-05-27T07:35:49.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.052+0000 7f602a412b80 1 -- 172.21.15.63:0/1751068585 wait complete. 2024-05-27T07:35:49.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.053+0000 7f602a412b80 20 client.15861 trim_cache size 0 max 16384 2024-05-27T07:35:49.050 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:35:49.050 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:35:49.050 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "93c714b9-a451-4891-8c78-d02f6d291298", 2024-05-27T07:35:49.050 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:35:49.050 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:35:49.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.053+0000 7f601a200700 1 -- 172.21.15.63:0/1332219818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ffc0b2a70 msgr2=0x7f5ffc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:49.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.053+0000 7f601a200700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ffc0b2a70 0x7f5ffc0b4f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f600c0a9290 tx=0x7f600c004ad0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.053+0000 7f601a200700 1 -- 172.21.15.63:0/1332219818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601410e700 msgr2=0x7f60141a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:49.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.053+0000 7f601a200700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601410e700 0x7f60141a16e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f600c007ee0 tx=0x7f600c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.053+0000 7f601a200700 1 -- 172.21.15.63:0/1332219818 shutdown_connections 2024-05-27T07:35:49.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.053+0000 7f601a200700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ffc0b2a70 0x7f5ffc0b4f80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.053+0000 7f601a200700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f601410e700 0x7f60141a16e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.053+0000 7f601a200700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6014105320 0x7f601419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.053+0000 7f601a200700 1 --2- 172.21.15.63:0/1332219818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6014105c70 0x7f601419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:49.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.054+0000 7f601a200700 1 -- 172.21.15.63:0/1332219818 >> 172.21.15.63:0/1332219818 conn(0x7f6014078ab0 msgr2=0x7f6014103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:49.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.054+0000 7f601a200700 1 -- 172.21.15.63:0/1332219818 shutdown_connections 2024-05-27T07:35:49.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:49.054+0000 7f601a200700 1 -- 172.21.15.63:0/1332219818 wait complete. 2024-05-27T07:35:49.327 INFO:tasks.workunit.client.0.smithi063.stdout:checking for ANSI C header files... yes 2024-05-27T07:35:49.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:49 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:35:49.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:49 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:49.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:49 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:35:49.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:49 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:49.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:49 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:35:49.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:49 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:49.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:49 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:35:49.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:49 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:49.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:49 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:35:49.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:49 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:49.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:49 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:35:49.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:49 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:49.712 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-27T07:35:49.979 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/types.h... yes 2024-05-27T07:35:50.234 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:35:50.260 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/stat.h... yes 2024-05-27T07:35:50.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:35:50.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:35:50.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:35:50.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:35:50.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:35:50.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:35:50.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:50 smithi063 ceph-mon[31613]: pgmap v70: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 286 KiB/s wr, 115 op/s 2024-05-27T07:35:50.484 INFO:tasks.workunit.client.0.smithi063.stdout:checking for stdlib.h... yes 2024-05-27T07:35:50.690 INFO:tasks.workunit.client.0.smithi063.stdout:checking for string.h... yes 2024-05-27T07:35:50.703 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:35:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:35:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:35:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:35:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:35:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:35:50.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:35:50.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:50 smithi086 ceph-mon[34687]: pgmap v70: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 286 KiB/s wr, 115 op/s 2024-05-27T07:35:50.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:35:50.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:35:50.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:35:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:35:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:35:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:35:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:35:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:50 smithi149 ceph-mon[35408]: pgmap v70: 129 pgs: 129 active+clean; 4.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 286 KiB/s wr, 115 op/s 2024-05-27T07:35:50.902 INFO:tasks.workunit.client.0.smithi063.stdout:checking for memory.h... yes 2024-05-27T07:35:51.153 INFO:tasks.workunit.client.0.smithi063.stdout:checking for strings.h... yes 2024-05-27T07:35:51.386 INFO:tasks.workunit.client.0.smithi063.stdout:checking for inttypes.h... yes 2024-05-27T07:35:51.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:51 smithi063 ceph-mon[31613]: pgmap v71: 129 pgs: 129 active+clean; 5.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 399 KiB/s wr, 163 op/s 2024-05-27T07:35:51.647 INFO:tasks.workunit.client.0.smithi063.stdout:checking for stdint.h... yes 2024-05-27T07:35:51.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:51 smithi086 ceph-mon[34687]: pgmap v71: 129 pgs: 129 active+clean; 5.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 399 KiB/s wr, 163 op/s 2024-05-27T07:35:51.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:51 smithi149 ceph-mon[35408]: pgmap v71: 129 pgs: 129 active+clean; 5.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 399 KiB/s wr, 163 op/s 2024-05-27T07:35:51.897 INFO:tasks.workunit.client.0.smithi063.stdout:checking for unistd.h... yes 2024-05-27T07:35:52.193 INFO:tasks.workunit.client.0.smithi063.stdout:checking pthread.h usability... yes 2024-05-27T07:35:52.296 INFO:tasks.workunit.client.0.smithi063.stdout:checking pthread.h presence... yes 2024-05-27T07:35:52.300 INFO:tasks.workunit.client.0.smithi063.stdout:checking for pthread.h... yes 2024-05-27T07:35:52.553 INFO:tasks.workunit.client.0.smithi063.stdout:checking fcntl.h usability... yes 2024-05-27T07:35:52.661 INFO:tasks.workunit.client.0.smithi063.stdout:checking fcntl.h presence... yes 2024-05-27T07:35:52.665 INFO:tasks.workunit.client.0.smithi063.stdout:checking for fcntl.h... yes 2024-05-27T07:35:52.921 INFO:tasks.workunit.client.0.smithi063.stdout:checking limits.h usability... yes 2024-05-27T07:35:53.022 INFO:tasks.workunit.client.0.smithi063.stdout:checking limits.h presence... yes 2024-05-27T07:35:53.026 INFO:tasks.workunit.client.0.smithi063.stdout:checking for limits.h... yes 2024-05-27T07:35:53.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.037+0000 7f6d3a000700 1 -- 172.21.15.63:0/1902875810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d34105bd0 msgr2=0x7f6d3410e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:53.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.037+0000 7f6d3a000700 1 --2- 172.21.15.63:0/1902875810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d34105bd0 0x7f6d3410e120 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009990 tx=0x7f6d20009ca0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.037+0000 7f6d3a000700 1 -- 172.21.15.63:0/1902875810 shutdown_connections 2024-05-27T07:35:53.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.037+0000 7f6d3a000700 1 --2- 172.21.15.63:0/1902875810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d3410e6f0 0x7f6d34110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.037+0000 7f6d3a000700 1 --2- 172.21.15.63:0/1902875810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d34105280 0x7f6d34105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.037+0000 7f6d3a000700 1 --2- 172.21.15.63:0/1902875810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d34105bd0 0x7f6d3410e120 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.037+0000 7f6d3a000700 1 -- 172.21.15.63:0/1902875810 >> 172.21.15.63:0/1902875810 conn(0x7f6d34078ab0 msgr2=0x7f6d34078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:53.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.037+0000 7f6d3a000700 1 -- 172.21.15.63:0/1902875810 shutdown_connections 2024-05-27T07:35:53.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.037+0000 7f6d3a000700 1 -- 172.21.15.63:0/1902875810 wait complete. 2024-05-27T07:35:53.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.038+0000 7f6d3a000700 1 Processor -- start 2024-05-27T07:35:53.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.039+0000 7f6d3a000700 1 -- start start 2024-05-27T07:35:53.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.039+0000 7f6d3a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d34105280 0x7f6d3419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.039+0000 7f6d3a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d34105bd0 0x7f6d3419d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.039+0000 7f6d3a000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d3410e6f0 0x7f6d341a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.039+0000 7f6d3a000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d34114100 con 0x7f6d34105bd0 2024-05-27T07:35:53.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.039+0000 7f6d3a000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d34113f80 con 0x7f6d34105280 2024-05-27T07:35:53.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.039+0000 7f6d3a000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6d34114280 con 0x7f6d3410e6f0 2024-05-27T07:35:53.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d33e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d34105bd0 0x7f6d3419d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d33e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d34105bd0 0x7f6d3419d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57868/0 (socket says 172.21.15.63:57868) 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d33e00700 1 -- 172.21.15.63:0/4248909998 learned_addr learned my addr 172.21.15.63:0/4248909998 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d38c00700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d34105280 0x7f6d3419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d39600700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d3410e6f0 0x7f6d341a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d33e00700 1 -- 172.21.15.63:0/4248909998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d3410e6f0 msgr2=0x7f6d341a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d33e00700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d3410e6f0 0x7f6d341a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d33e00700 1 -- 172.21.15.63:0/4248909998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d34105280 msgr2=0x7f6d3419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d33e00700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d34105280 0x7f6d3419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d33e00700 1 -- 172.21.15.63:0/4248909998 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d341a1be0 con 0x7f6d34105bd0 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.040+0000 7f6d39600700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d3410e6f0 0x7f6d341a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:53.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.041+0000 7f6d33e00700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d34105bd0 0x7f6d3419d2b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6d20000c00 tx=0x7f6d20009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:53.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.041+0000 7f6d31600700 1 -- 172.21.15.63:0/4248909998 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20027070 con 0x7f6d34105bd0 2024-05-27T07:35:53.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.041+0000 7f6d3a000700 1 -- 172.21.15.63:0/4248909998 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d341a1e70 con 0x7f6d34105bd0 2024-05-27T07:35:53.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.041+0000 7f6d31600700 1 -- 172.21.15.63:0/4248909998 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d2000fc70 con 0x7f6d34105bd0 2024-05-27T07:35:53.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.041+0000 7f6d31600700 1 -- 172.21.15.63:0/4248909998 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20022d40 con 0x7f6d34105bd0 2024-05-27T07:35:53.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.041+0000 7f6d3a000700 1 -- 172.21.15.63:0/4248909998 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d341a97f0 con 0x7f6d34105bd0 2024-05-27T07:35:53.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.045+0000 7f6d31600700 1 -- 172.21.15.63:0/4248909998 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6d2002a070 con 0x7f6d34105bd0 2024-05-27T07:35:53.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d4a1f9b80 1 Processor -- start 2024-05-27T07:35:53.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d4a1f9b80 1 -- start start 2024-05-27T07:35:53.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d31600700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6d1c0b29f0 0x7f6d1c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d31600700 1 -- 172.21.15.63:0/4248909998 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f6d200e3790 con 0x7f6d34105bd0 2024-05-27T07:35:53.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d39600700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6d1c0b29f0 0x7f6d1c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae093c9f0 0x55fae09d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae0850e40 0x55fae09d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae09da2c0 0x55fae09dc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fae08fde40 con 0x55fae09da2c0 2024-05-27T07:35:53.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fae08ed9e0 con 0x55fae0850e40 2024-05-27T07:35:53.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.047+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fae08f1a20 con 0x55fae093c9f0 2024-05-27T07:35:53.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.048+0000 7f6d33e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae0850e40 0x55fae09d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.048+0000 7f6d33e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae0850e40 0x55fae09d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:50776/0 (socket says 172.21.15.63:50776) 2024-05-27T07:35:53.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.048+0000 7f6d33e00700 1 -- 172.21.15.63:0/472591183 learned_addr learned my addr 172.21.15.63:0/472591183 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:53.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.048+0000 7f6d39600700 1 --2- 172.21.15.63:0/472591183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae09da2c0 0x55fae09dc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.048+0000 7f6d38c00700 1 --2- 172.21.15.63:0/472591183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae093c9f0 0x55fae09d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.048+0000 7f6d39600700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6d1c0b29f0 0x7f6d1c0b4f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009d40 tx=0x7f6d2c0077e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:53.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.049+0000 7f6d39600700 1 -- 172.21.15.63:0/472591183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae093c9f0 msgr2=0x55fae09d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:53.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.049+0000 7f6d39600700 1 --2- 172.21.15.63:0/472591183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae093c9f0 0x55fae09d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.049+0000 7f6d39600700 1 -- 172.21.15.63:0/472591183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae0850e40 msgr2=0x55fae09d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:53.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.049+0000 7f6d39600700 1 --2- 172.21.15.63:0/472591183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae0850e40 0x55fae09d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.049+0000 7f6d39600700 1 -- 172.21.15.63:0/472591183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae08b6420 con 0x55fae09da2c0 2024-05-27T07:35:53.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.049+0000 7f6d39600700 1 --2- 172.21.15.63:0/472591183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae09da2c0 0x55fae09dc6e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c007be0 tx=0x7f6d2c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e138eea93cc719 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:53.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.049+0000 7f6d26000700 1 -- 172.21.15.63:0/472591183 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c019e60 con 0x55fae09da2c0 2024-05-27T07:35:53.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.049+0000 7f6d26000700 1 -- 172.21.15.63:0/472591183 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d2c01d040 con 0x55fae09da2c0 2024-05-27T07:35:53.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.050+0000 7f6d26000700 1 -- 172.21.15.63:0/472591183 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c021e50 con 0x55fae09da2c0 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.051+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/472591183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae09da2c0 msgr2=0x55fae09dc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.051+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/472591183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae09da2c0 0x55fae09dc6e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c007be0 tx=0x7f6d2c01a000 comp rx=0 tx=0).stop 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.051+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/472591183 shutdown_connections 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.051+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/472591183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae09da2c0 0x55fae09dc6e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.051+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/472591183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae093c9f0 0x55fae09d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.051+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/472591183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae0850e40 0x55fae09d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.051+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/472591183 >> 172.21.15.63:0/472591183 conn(0x55fae08ea390 msgr2=0x55fae09cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.051+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/472591183 shutdown_connections 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.051+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/472591183 wait complete. 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.052+0000 7f6d4a1f9b80 1 -- start start 2024-05-27T07:35:53.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.052+0000 7f6d4a1f9b80 1 Processor -- start 2024-05-27T07:35:53.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.052+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae0850e40 0x55fae09d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.052+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.052+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 0x55fae09d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.052+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fae07f3fa0 con 0x55fae093c9f0 2024-05-27T07:35:53.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fae088e760 con 0x55fae09da2c0 2024-05-27T07:35:53.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fae09dce00 con 0x55fae0850e40 2024-05-27T07:35:53.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d4a1f9b80 10 client.0 _ll_register_callbacks cb 0x55fae0897c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:53.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d33e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d33e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09d7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57886/0 (socket says 172.21.15.63:57886) 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d33e00700 1 -- 172.21.15.63:0/685520373 learned_addr learned my addr 172.21.15.63:0/685520373 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d39600700 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 0x55fae09d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d38c00700 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae0850e40 0x55fae09d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d33e00700 1 -- 172.21.15.63:0/685520373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae0850e40 msgr2=0x55fae09d7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d33e00700 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae0850e40 0x55fae09d7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d33e00700 1 -- 172.21.15.63:0/685520373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 msgr2=0x55fae09d7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d33e00700 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 0x55fae09d7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d33e00700 1 -- 172.21.15.63:0/685520373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae083d8d0 con 0x55fae093c9f0 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.053+0000 7f6d39600700 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 0x55fae09d7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.054+0000 7f6d33e00700 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09d7760 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6d200aa670 tx=0x7f6d200aac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97429b08ced3088 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.054+0000 7f6d1aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.054+0000 7f6d1aa00700 1 -- 172.21.15.63:0/685520373 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2000fe20 con 0x55fae093c9f0 2024-05-27T07:35:53.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.054+0000 7f6d1aa00700 1 -- 172.21.15.63:0/685520373 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f6d200e3360 con 0x55fae093c9f0 2024-05-27T07:35:53.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.054+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/685520373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d2c01d040 con 0x55fae093c9f0 2024-05-27T07:35:53.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.054+0000 7f6d1aa00700 1 -- 172.21.15.63:0/685520373 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d200e3560 con 0x55fae093c9f0 2024-05-27T07:35:53.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.054+0000 7f6d4a1f9b80 10 client.15873 fetch_fsmap learned FSMap version 28 2024-05-27T07:35:53.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.054+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/685520373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fae08be060 con 0x55fae093c9f0 2024-05-27T07:35:53.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.055+0000 7f6d1aa00700 1 -- 172.21.15.63:0/685520373 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d2c01d040 con 0x55fae093c9f0 2024-05-27T07:35:53.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.055+0000 7f6d1aa00700 1 -- 172.21.15.63:0/685520373 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6d200a9990 con 0x55fae093c9f0 2024-05-27T07:35:53.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.055+0000 7f6d4a1f9b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:35:53.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.055+0000 7f6d4a1f9b80 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:53.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.055+0000 7f6d4a1f9b80 20 client.15873 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:53.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.055+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fae09e9c90 0x55fae09ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.055+0000 7f6d4a1f9b80 4 client.15873 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:53.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.055+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/685520373 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fae08ffd90 con 0x55fae09e9c90 2024-05-27T07:35:53.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.056+0000 7f6d38c00700 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fae09e9c90 0x55fae09ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.057+0000 7f6d38c00700 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fae09e9c90 0x55fae09ec0d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6960481bbabc9ec8 server_cookie=ef845f1478eec13f in_seq=0 out_seq=0 2024-05-27T07:35:53.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.057+0000 7f6d1aa00700 10 client.15873 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:53.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.060+0000 7f6d1aa00700 1 -- 172.21.15.63:0/685520373 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fae08ffd90 con 0x55fae09e9c90 2024-05-27T07:35:53.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.060+0000 7f6d1aa00700 10 client.15873 handle_command_reply: tid=0 2024-05-27T07:35:53.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.060+0000 7f6d4a1f9b80 1 client.15873 shutdown 2024-05-27T07:35:53.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.060+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/685520373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 msgr2=0x55fae09d7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:53.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.060+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09d7760 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6d200aa670 tx=0x7f6d200aac70 comp rx=0 tx=0).stop 2024-05-27T07:35:53.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.061+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/685520373 shutdown_connections 2024-05-27T07:35:53.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.061+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fae09e9c90 0x55fae09ec0d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.061+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 0x55fae09d7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.061+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae0850e40 0x55fae09d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.061+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/685520373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09d7760 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.061+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/685520373 >> 172.21.15.63:0/685520373 conn(0x55fae08ea390 msgr2=0x55fae09d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:53.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.061+0000 7f6d1aa00700 0 client.15873 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:53.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.061+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/685520373 shutdown_connections 2024-05-27T07:35:53.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.061+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/685520373 wait complete. 2024-05-27T07:35:53.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.061+0000 7f6d4a1f9b80 20 client.15873 trim_cache size 0 max 16384 2024-05-27T07:35:53.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.086+0000 7f6d4a1f9b80 1 Processor -- start 2024-05-27T07:35:53.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.086+0000 7f6d4a1f9b80 1 -- start start 2024-05-27T07:35:53.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.086+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae0850e40 0x55fae09e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.086+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae093c9f0 0x55fae09f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.086+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae09da2c0 0x55fae09f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.086+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341b4c10 con 0x55fae0850e40 2024-05-27T07:35:53.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.086+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341b24a0 con 0x55fae093c9f0 2024-05-27T07:35:53.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.086+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341b02f0 con 0x55fae09da2c0 2024-05-27T07:35:53.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.087+0000 7f6d38c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae0850e40 0x55fae09e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.087+0000 7f6d38c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae0850e40 0x55fae09e5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57902/0 (socket says 172.21.15.63:57902) 2024-05-27T07:35:53.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.087+0000 7f6d38c00700 1 -- 172.21.15.63:0/1719519605 learned_addr learned my addr 172.21.15.63:0/1719519605 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:53.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.087+0000 7f6d39600700 1 --2- 172.21.15.63:0/1719519605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae09da2c0 0x55fae09f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.087+0000 7f6d33e00700 1 --2- 172.21.15.63:0/1719519605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae093c9f0 0x55fae09f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.087+0000 7f6d38c00700 1 -- 172.21.15.63:0/1719519605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae09da2c0 msgr2=0x55fae09f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.087+0000 7f6d38c00700 1 --2- 172.21.15.63:0/1719519605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae09da2c0 0x55fae09f1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.087+0000 7f6d38c00700 1 -- 172.21.15.63:0/1719519605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae093c9f0 msgr2=0x55fae09f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.087+0000 7f6d38c00700 1 --2- 172.21.15.63:0/1719519605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae093c9f0 0x55fae09f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.088+0000 7f6d38c00700 1 -- 172.21.15.63:0/1719519605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae09d8010 con 0x55fae0850e40 2024-05-27T07:35:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.088+0000 7f6d38c00700 1 --2- 172.21.15.63:0/1719519605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae0850e40 0x55fae09e5ea0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6d28000c00 tx=0x7f6d280058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6fdaace93f2140 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:53.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.088+0000 7f6d25600700 1 -- 172.21.15.63:0/1719519605 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2801f070 con 0x55fae0850e40 2024-05-27T07:35:53.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.088+0000 7f6d25600700 1 -- 172.21.15.63:0/1719519605 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d280054b0 con 0x55fae0850e40 2024-05-27T07:35:53.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.089+0000 7f6d25600700 1 -- 172.21.15.63:0/1719519605 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2801b2c0 con 0x55fae0850e40 2024-05-27T07:35:53.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.089+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/1719519605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae0850e40 msgr2=0x55fae09e5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:53.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.089+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/1719519605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae0850e40 0x55fae09e5ea0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6d28000c00 tx=0x7f6d280058e0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.090+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/1719519605 shutdown_connections 2024-05-27T07:35:53.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.090+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/1719519605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae09da2c0 0x55fae09f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.090+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/1719519605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae0850e40 0x55fae09e5ea0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.090+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/1719519605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae093c9f0 0x55fae09f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.090+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/1719519605 >> 172.21.15.63:0/1719519605 conn(0x55fae08ea390 msgr2=0x55fae09ebca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:53.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.090+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/1719519605 shutdown_connections 2024-05-27T07:35:53.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.090+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/1719519605 wait complete. 2024-05-27T07:35:53.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.090+0000 7f6d4a1f9b80 1 -- start start 2024-05-27T07:35:53.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.091+0000 7f6d4a1f9b80 1 Processor -- start 2024-05-27T07:35:53.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.091+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae0850e40 0x55fae09e24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.091+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09dd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.091+0000 7f6d4a1f9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 0x55fae09ddd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.091+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341b0960 con 0x55fae093c9f0 2024-05-27T07:35:53.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.091+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341b1dd0 con 0x55fae09da2c0 2024-05-27T07:35:53.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.091+0000 7f6d4a1f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6d341b1030 con 0x55fae0850e40 2024-05-27T07:35:53.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.091+0000 7f6d4a1f9b80 10 client.0 _ll_register_callbacks cb 0x55fae0897c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:53.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.092+0000 7f6d33e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09dd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.092+0000 7f6d33e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09dd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57904/0 (socket says 172.21.15.63:57904) 2024-05-27T07:35:53.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.092+0000 7f6d33e00700 1 -- 172.21.15.63:0/3533664213 learned_addr learned my addr 172.21.15.63:0/3533664213 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:53.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.092+0000 7f6d38c00700 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae0850e40 0x55fae09e24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.092+0000 7f6d39600700 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 0x55fae09ddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.092+0000 7f6d33e00700 1 -- 172.21.15.63:0/3533664213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae0850e40 msgr2=0x55fae09e24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.093+0000 7f6d33e00700 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae0850e40 0x55fae09e24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.093+0000 7f6d33e00700 1 -- 172.21.15.63:0/3533664213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 msgr2=0x55fae09ddd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.093+0000 7f6d33e00700 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 0x55fae09ddd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.093+0000 7f6d33e00700 1 -- 172.21.15.63:0/3533664213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae09de2b0 con 0x55fae093c9f0 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.093+0000 7f6d33e00700 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09dd830 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6d200cd170 tx=0x7f6d2000fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6728571e763f7c60 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.093+0000 7f6d1aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.093+0000 7f6d1aa00700 1 -- 172.21.15.63:0/3533664213 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d200b3550 con 0x55fae093c9f0 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.093+0000 7f6d1aa00700 1 -- 172.21.15.63:0/3533664213 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f6d200a6d20 con 0x55fae093c9f0 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.093+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/3533664213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d280054b0 con 0x55fae093c9f0 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d1aa00700 1 -- 172.21.15.63:0/3533664213 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d2000fe20 con 0x55fae093c9f0 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d4a1f9b80 10 client.15879 fetch_fsmap learned FSMap version 28 2024-05-27T07:35:53.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/3533664213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fae0a543f0 con 0x55fae093c9f0 2024-05-27T07:35:53.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d1aa00700 1 -- 172.21.15.63:0/3533664213 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d280054b0 con 0x55fae093c9f0 2024-05-27T07:35:53.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d1aa00700 1 -- 172.21.15.63:0/3533664213 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6d200a9400 con 0x55fae093c9f0 2024-05-27T07:35:53.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d4a1f9b80 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:35:53.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d4a1f9b80 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:53.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d4a1f9b80 20 client.15879 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:53.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fae0a549f0 0x55fae0a56e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:53.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d4a1f9b80 4 client.15879 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:35:53.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.094+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/3533664213 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fae08f9570 con 0x55fae0a549f0 2024-05-27T07:35:53.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.095+0000 7f6d38c00700 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fae0a549f0 0x55fae0a56e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:53.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.097+0000 7f6d38c00700 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fae0a549f0 0x55fae0a56e10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd75f802a243536 server_cookie=5cc93bc6cbc3824c in_seq=0 out_seq=0 2024-05-27T07:35:53.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.097+0000 7f6d1aa00700 10 client.15879 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:53.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d1aa00700 1 -- 172.21.15.63:0/3533664213 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fae08f9570 con 0x55fae0a549f0 2024-05-27T07:35:53.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d1aa00700 10 client.15879 handle_command_reply: tid=0 2024-05-27T07:35:53.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d4a1f9b80 1 client.15879 shutdown 2024-05-27T07:35:53.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/3533664213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 msgr2=0x55fae09dd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:53.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09dd830 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6d200cd170 tx=0x7f6d2000fa10 comp rx=0 tx=0).stop 2024-05-27T07:35:53.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/3533664213 shutdown_connections 2024-05-27T07:35:53.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fae0a549f0 0x55fae0a56e10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fae09da2c0 0x55fae09ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fae0850e40 0x55fae09e24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d4a1f9b80 1 --2- 172.21.15.63:0/3533664213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fae093c9f0 0x55fae09dd830 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/3533664213 >> 172.21.15.63:0/3533664213 conn(0x55fae08ea390 msgr2=0x55fae09ebca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:53.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d1aa00700 0 client.15879 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:53.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.098+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/3533664213 shutdown_connections 2024-05-27T07:35:53.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.099+0000 7f6d4a1f9b80 1 -- 172.21.15.63:0/3533664213 wait complete. 2024-05-27T07:35:53.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.099+0000 7f6d4a1f9b80 20 client.15879 trim_cache size 0 max 16384 2024-05-27T07:35:53.096 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:35:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:35:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:35:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: "93c714b9-a451-4891-8c78-d02f6d291298": { 2024-05-27T07:35:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:35:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "93c714b9-a451-4891-8c78-d02f6d291298", 2024-05-27T07:35:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:35:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:35:53.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:35:53.097 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:35:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 -- 172.21.15.63:0/4248909998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6d1c0b29f0 msgr2=0x7f6d1c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:53.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6d1c0b29f0 0x7f6d1c0b4f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009d40 tx=0x7f6d2c0077e0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 -- 172.21.15.63:0/4248909998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d34105bd0 msgr2=0x7f6d3419d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:53.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d34105bd0 0x7f6d3419d2b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6d20000c00 tx=0x7f6d20009f90 comp rx=0 tx=0).stop 2024-05-27T07:35:53.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 -- 172.21.15.63:0/4248909998 shutdown_connections 2024-05-27T07:35:53.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6d1c0b29f0 0x7f6d1c0b4f00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d3410e6f0 0x7f6d341a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d34105280 0x7f6d3419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 --2- 172.21.15.63:0/4248909998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d34105bd0 0x7f6d3419d2b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:53.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 -- 172.21.15.63:0/4248909998 >> 172.21.15.63:0/4248909998 conn(0x7f6d34078ab0 msgr2=0x7f6d34102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:53.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 -- 172.21.15.63:0/4248909998 shutdown_connections 2024-05-27T07:35:53.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:35:53.100+0000 7f6d3a000700 1 -- 172.21.15.63:0/4248909998 wait complete. 2024-05-27T07:35:53.100 INFO:tasks.workunit.client.0.smithi063.stdout:checking for stdint.h... (cached) yes 2024-05-27T07:35:53.333 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/time.h usability... yes 2024-05-27T07:35:53.441 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/time.h presence... yes 2024-05-27T07:35:53.444 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/time.h... yes 2024-05-27T07:35:53.551 INFO:tasks.workunit.client.0.smithi063.stdout:checking for unistd.h... (cached) yes 2024-05-27T07:35:53.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:53 smithi086 ceph-mon[34687]: pgmap v72: 129 pgs: 129 active+clean; 5.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 267 KiB/s wr, 142 op/s 2024-05-27T07:35:53.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:53 smithi149 ceph-mon[35408]: pgmap v72: 129 pgs: 129 active+clean; 5.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 267 KiB/s wr, 142 op/s 2024-05-27T07:35:53.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:53 smithi063 ceph-mon[31613]: pgmap v72: 129 pgs: 129 active+clean; 5.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 267 KiB/s wr, 142 op/s 2024-05-27T07:35:53.816 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/vfs.h usability... yes 2024-05-27T07:35:53.916 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/vfs.h presence... yes 2024-05-27T07:35:53.919 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/vfs.h... yes 2024-05-27T07:35:54.060 INFO:tasks.cephfs.filesystem:scrub status for tag:93c714b9-a451-4891-8c78-d02f6d291298 - {'path': '/', 'tag': '93c714b9-a451-4891-8c78-d02f6d291298', 'options': 'recursive,force'} 2024-05-27T07:35:54.159 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/limits.h usability... no 2024-05-27T07:35:54.284 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/limits.h presence... no 2024-05-27T07:35:54.287 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/limits.h... no 2024-05-27T07:35:54.454 INFO:tasks.workunit.client.0.smithi063.stdout:checking for an ANSI C-conforming const... yes 2024-05-27T07:35:54.641 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-27T07:35:54.652 INFO:tasks.workunit.client.0.smithi063.stdout:checking for function prototypes... yes 2024-05-27T07:35:54.912 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether setvbuf arguments are reversed... no 2024-05-27T07:35:55.286 INFO:tasks.workunit.client.0.smithi063.stdout:checking for system... yes 2024-05-27T07:35:55.650 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gettimeofday... yes 2024-05-27T07:35:55.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:55 smithi086 ceph-mon[34687]: pgmap v73: 129 pgs: 129 active+clean; 5.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 267 KiB/s wr, 142 op/s 2024-05-27T07:35:55.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:55 smithi149 ceph-mon[35408]: pgmap v73: 129 pgs: 129 active+clean; 5.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 267 KiB/s wr, 142 op/s 2024-05-27T07:35:55.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:55 smithi063 ceph-mon[31613]: pgmap v73: 129 pgs: 129 active+clean; 5.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 267 KiB/s wr, 142 op/s 2024-05-27T07:35:56.004 INFO:tasks.workunit.client.0.smithi063.stdout:checking for mkdir... yes 2024-05-27T07:35:56.504 INFO:tasks.workunit.client.0.smithi063.stdout:checking for strerror... yes 2024-05-27T07:35:56.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:56 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:35:56.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:56 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:35:56.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:56 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:35:57.067 INFO:tasks.workunit.client.0.smithi063.stdout:checking for open64... yes 2024-05-27T07:35:57.434 INFO:tasks.workunit.client.0.smithi063.stdout:checking for stat64... yes 2024-05-27T07:35:57.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:57 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:35:57.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:57 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:35:57.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:57 smithi086 ceph-mon[34687]: pgmap v74: 129 pgs: 129 active+clean; 6.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 364 KiB/s wr, 168 op/s 2024-05-27T07:35:57.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:57 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:35:57.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:57 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:35:57.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:57 smithi149 ceph-mon[35408]: pgmap v74: 129 pgs: 129 active+clean; 6.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 364 KiB/s wr, 168 op/s 2024-05-27T07:35:57.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:57 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:35:57.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:57 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:35:57.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:57 smithi063 ceph-mon[31613]: pgmap v74: 129 pgs: 129 active+clean; 6.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 364 KiB/s wr, 168 op/s 2024-05-27T07:35:57.774 INFO:tasks.workunit.client.0.smithi063.stdout:checking for fseeko64... yes 2024-05-27T07:35:58.037 INFO:tasks.workunit.client.0.smithi063.stdout:checking for lrand48_r... yes 2024-05-27T07:35:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.202+0000 7fe33da00640 1 -- 172.21.15.63:0/2048794807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe338112310 msgr2=0x7fe338114730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.202+0000 7fe33da00640 1 --2- 172.21.15.63:0/2048794807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe338112310 0x7fe338114730 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe324009900 tx=0x7fe32402a480 comp rx=0 tx=0).stop 2024-05-27T07:35:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.202+0000 7fe33da00640 1 -- 172.21.15.63:0/2048794807 shutdown_connections 2024-05-27T07:35:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.202+0000 7fe33da00640 1 --2- 172.21.15.63:0/2048794807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338114c70 0x7fe338117090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.202+0000 7fe33da00640 1 --2- 172.21.15.63:0/2048794807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe338112310 0x7fe338114730 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.202+0000 7fe33da00640 1 --2- 172.21.15.63:0/2048794807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338075480 0x7fe338075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.202+0000 7fe33da00640 1 -- 172.21.15.63:0/2048794807 >> 172.21.15.63:0/2048794807 conn(0x7fe3380ff480 msgr2=0x7fe3381018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.202+0000 7fe33da00640 1 -- 172.21.15.63:0/2048794807 shutdown_connections 2024-05-27T07:35:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.202+0000 7fe33da00640 1 -- 172.21.15.63:0/2048794807 wait complete. 2024-05-27T07:35:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.203+0000 7fe33da00640 1 Processor -- start 2024-05-27T07:35:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.204+0000 7fe33da00640 1 -- start start 2024-05-27T07:35:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.204+0000 7fe33da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338075480 0x7fe3381a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.204+0000 7fe33da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe338112310 0x7fe3381a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.204+0000 7fe33da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338114c70 0x7fe3381a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.204+0000 7fe33da00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe33811a320 con 0x7fe338112310 2024-05-27T07:35:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.204+0000 7fe33da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe33811a1a0 con 0x7fe338114c70 2024-05-27T07:35:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.204+0000 7fe33da00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe33811a4a0 con 0x7fe338075480 2024-05-27T07:35:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.204+0000 7fe336000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe338112310 0x7fe3381a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.205+0000 7fe336000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe338112310 0x7fe3381a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57918/0 (socket says 172.21.15.63:57918) 2024-05-27T07:35:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.205+0000 7fe336000640 1 -- 172.21.15.63:0/2121563424 learned_addr learned my addr 172.21.15.63:0/2121563424 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.205+0000 7fe337400640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338114c70 0x7fe3381a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.205+0000 7fe336a00640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338075480 0x7fe3381a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.205+0000 7fe336000640 1 -- 172.21.15.63:0/2121563424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338075480 msgr2=0x7fe3381a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.205+0000 7fe336000640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338075480 0x7fe3381a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.205+0000 7fe336000640 1 -- 172.21.15.63:0/2121563424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338114c70 msgr2=0x7fe3381a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.205+0000 7fe336000640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338114c70 0x7fe3381a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.205+0000 7fe336000640 1 -- 172.21.15.63:0/2121563424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3381a5c40 con 0x7fe338112310 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.206+0000 7fe336000640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe338112310 0x7fe3381a1340 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe324009230 tx=0x7fe3240319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.206+0000 7fe32f400640 1 -- 172.21.15.63:0/2121563424 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32403f030 con 0x7fe338112310 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.206+0000 7fe32f400640 1 -- 172.21.15.63:0/2121563424 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3240043d0 con 0x7fe338112310 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.206+0000 7fe33da00640 1 -- 172.21.15.63:0/2121563424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3381a5ed0 con 0x7fe338112310 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.206+0000 7fe32f400640 1 -- 172.21.15.63:0/2121563424 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324035070 con 0x7fe338112310 2024-05-27T07:35:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.207+0000 7fe33da00640 1 -- 172.21.15.63:0/2121563424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3381a63c0 con 0x7fe338112310 2024-05-27T07:35:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.210+0000 7fe3449d5740 1 Processor -- start 2024-05-27T07:35:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.211+0000 7fe3449d5740 1 -- start start 2024-05-27T07:35:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.211+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3631b0 0x557d3e705860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.211+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 0x557d3e705da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.211+0000 7fe32f400640 1 -- 172.21.15.63:0/2121563424 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe324042030 con 0x7fe338112310 2024-05-27T07:35:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.211+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e3424d0 0x557d3e7062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.211+0000 7fe3449d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557d3e676b90 con 0x557d3e3631b0 2024-05-27T07:35:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.211+0000 7fe3449d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d3e666730 con 0x557d3e33a2f0 2024-05-27T07:35:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.211+0000 7fe3449d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557d3e66a770 con 0x557d3e3424d0 2024-05-27T07:35:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.211+0000 7fe337400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3631b0 0x557d3e705860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.212+0000 7fe337400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3631b0 0x557d3e705860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57926/0 (socket says 172.21.15.63:57926) 2024-05-27T07:35:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.212+0000 7fe337400640 1 -- 172.21.15.63:0/2036139085 learned_addr learned my addr 172.21.15.63:0/2036139085 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.212+0000 7fe336a00640 1 --2- 172.21.15.63:0/2036139085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 0x557d3e705da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.212+0000 7fe337400640 1 -- 172.21.15.63:0/2036139085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e3424d0 msgr2=0x557d3e7062e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.212+0000 7fe337400640 1 --2- 172.21.15.63:0/2036139085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e3424d0 0x557d3e7062e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.212+0000 7fe337400640 1 -- 172.21.15.63:0/2036139085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 msgr2=0x557d3e705da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.212+0000 7fe337400640 1 --2- 172.21.15.63:0/2036139085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 0x557d3e705da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.212+0000 7fe337400640 1 -- 172.21.15.63:0/2036139085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3e5ff8b0 con 0x557d3e3631b0 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.212+0000 7fe337400640 1 --2- 172.21.15.63:0/2036139085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3631b0 0x557d3e705860 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe330009f50 tx=0x7fe3300118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce39f2e0c40c7a6b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.213+0000 7fe323e00640 1 -- 172.21.15.63:0/2036139085 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330004b20 con 0x557d3e3631b0 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.213+0000 7fe323e00640 1 -- 172.21.15.63:0/2036139085 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe330004cc0 con 0x557d3e3631b0 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.213+0000 7fe323e00640 1 -- 172.21.15.63:0/2036139085 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330005280 con 0x557d3e3631b0 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.213+0000 7fe32f400640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe31c0b29d0 0x7fe31c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe32f400640 1 -- 172.21.15.63:0/2121563424 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fe324102d70 con 0x7fe338112310 2024-05-27T07:35:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe336a00640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe31c0b29d0 0x7fe31c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe3449d5740 1 -- 172.21.15.63:0/2036139085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3631b0 msgr2=0x557d3e705860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe3449d5740 1 --2- 172.21.15.63:0/2036139085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3631b0 0x557d3e705860 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe330009f50 tx=0x7fe3300118d0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe3449d5740 1 -- 172.21.15.63:0/2036139085 shutdown_connections 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe3449d5740 1 --2- 172.21.15.63:0/2036139085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e3424d0 0x557d3e7062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe3449d5740 1 --2- 172.21.15.63:0/2036139085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 0x557d3e705da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe3449d5740 1 --2- 172.21.15.63:0/2036139085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3631b0 0x557d3e705860 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe3449d5740 1 -- 172.21.15.63:0/2036139085 >> 172.21.15.63:0/2036139085 conn(0x557d3e357300 msgr2=0x557d3e702010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe3449d5740 1 -- 172.21.15.63:0/2036139085 shutdown_connections 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.214+0000 7fe3449d5740 1 -- 172.21.15.63:0/2036139085 wait complete. 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.215+0000 7fe336a00640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe31c0b29d0 0x7fe31c0b4ee0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x557d3e706c10 tx=0x7fe328005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.215+0000 7fe3449d5740 1 -- start start 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.215+0000 7fe3449d5740 1 Processor -- start 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 0x557d3e70f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e70ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e3631b0 0x557d3e71bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe3449d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557d3e34c790 con 0x557d3e3424d0 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe3449d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d3e60ebc0 con 0x557d3e33a2f0 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe3449d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557d3e712990 con 0x557d3e3631b0 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe3449d5740 10 client.0 _ll_register_callbacks cb 0x557d3e5ec400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe336000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e70ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe336000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e70ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57928/0 (socket says 172.21.15.63:57928) 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe336000640 1 -- 172.21.15.63:0/190072031 learned_addr learned my addr 172.21.15.63:0/190072031 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.216+0000 7fe336a00640 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 0x557d3e70f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.217+0000 7fe337400640 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e3631b0 0x557d3e71bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.217+0000 7fe336000640 1 -- 172.21.15.63:0/190072031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e3631b0 msgr2=0x557d3e71bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.217+0000 7fe336000640 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e3631b0 0x557d3e71bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.217+0000 7fe336000640 1 -- 172.21.15.63:0/190072031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 msgr2=0x557d3e70f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.217+0000 7fe336000640 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 0x557d3e70f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.217+0000 7fe336000640 1 -- 172.21.15.63:0/190072031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3e37b210 con 0x557d3e3424d0 2024-05-27T07:35:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.217+0000 7fe337400640 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e3631b0 0x557d3e71bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.217+0000 7fe336a00640 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 0x557d3e70f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:35:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.217+0000 7fe336000640 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e70ff30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe3240c5db0 tx=0x7fe3240c96f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e285c957f0cfcf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.217+0000 7fe320c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.218+0000 7fe320c00640 1 -- 172.21.15.63:0/190072031 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324049070 con 0x557d3e3424d0 2024-05-27T07:35:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.218+0000 7fe320c00640 1 -- 172.21.15.63:0/190072031 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fe3240d2650 con 0x557d3e3424d0 2024-05-27T07:35:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.218+0000 7fe3449d5740 1 -- 172.21.15.63:0/190072031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe330004cc0 con 0x557d3e3424d0 2024-05-27T07:35:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.218+0000 7fe320c00640 1 -- 172.21.15.63:0/190072031 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe324100580 con 0x557d3e3424d0 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.218+0000 7fe3449d5740 10 client.15891 fetch_fsmap learned FSMap version 28 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.218+0000 7fe3449d5740 1 -- 172.21.15.63:0/190072031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d3e447620 con 0x557d3e3424d0 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.218+0000 7fe320c00640 1 -- 172.21.15.63:0/190072031 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe330004cc0 con 0x557d3e3424d0 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.219+0000 7fe320c00640 1 -- 172.21.15.63:0/190072031 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe3240c56a0 con 0x557d3e3424d0 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.219+0000 7fe3449d5740 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.219+0000 7fe3449d5740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.219+0000 7fe3449d5740 20 client.15891 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.219+0000 7fe3449d5740 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557d3e321d80 0x557d3e721c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.219+0000 7fe3449d5740 4 client.15891 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.219+0000 7fe3449d5740 1 -- 172.21.15.63:0/190072031 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d3e678ae0 con 0x557d3e321d80 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.220+0000 7fe336a00640 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557d3e321d80 0x557d3e721c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.221+0000 7fe336a00640 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557d3e321d80 0x557d3e721c40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4351dfbe5ecd96 server_cookie=3d89cb35ccd22208 in_seq=0 out_seq=0 2024-05-27T07:35:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.221+0000 7fe320c00640 10 client.15891 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.222+0000 7fe320c00640 1 -- 172.21.15.63:0/190072031 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557d3e678ae0 con 0x557d3e321d80 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.222+0000 7fe320c00640 10 client.15891 handle_command_reply: tid=0 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 client.15891 shutdown 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 -- 172.21.15.63:0/190072031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 msgr2=0x557d3e70ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e70ff30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe3240c5db0 tx=0x7fe3240c96f0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 -- 172.21.15.63:0/190072031 shutdown_connections 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557d3e321d80 0x557d3e721c40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e3631b0 0x557d3e71bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e70ff30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 --2- 172.21.15.63:0/190072031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e33a2f0 0x557d3e70f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 -- 172.21.15.63:0/190072031 >> 172.21.15.63:0/190072031 conn(0x557d3e357300 msgr2=0x557d3e705690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe320c00640 0 client.15891 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 -- 172.21.15.63:0/190072031 shutdown_connections 2024-05-27T07:35:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 1 -- 172.21.15.63:0/190072031 wait complete. 2024-05-27T07:35:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.223+0000 7fe3449d5740 20 client.15891 trim_cache size 0 max 16384 2024-05-27T07:35:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.241+0000 7fe3449d5740 1 Processor -- start 2024-05-27T07:35:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.241+0000 7fe3449d5740 1 -- start start 2024-05-27T07:35:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.241+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e357300 0x557d3e70ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.241+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e33a2f0 0x557d3e711b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.241+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e727400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.242+0000 7fe3449d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381ba000 con 0x557d3e3424d0 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.242+0000 7fe3449d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381b78d0 con 0x557d3e357300 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.242+0000 7fe3449d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381b5720 con 0x557d3e33a2f0 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.242+0000 7fe337400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e727400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.242+0000 7fe337400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e727400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57934/0 (socket says 172.21.15.63:57934) 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.242+0000 7fe337400640 1 -- 172.21.15.63:0/3533780635 learned_addr learned my addr 172.21.15.63:0/3533780635 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.242+0000 7fe336a00640 1 --2- 172.21.15.63:0/3533780635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e357300 0x557d3e70ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.242+0000 7fe336000640 1 --2- 172.21.15.63:0/3533780635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e33a2f0 0x557d3e711b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.243+0000 7fe337400640 1 -- 172.21.15.63:0/3533780635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e33a2f0 msgr2=0x557d3e711b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.243+0000 7fe337400640 1 --2- 172.21.15.63:0/3533780635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e33a2f0 0x557d3e711b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.243+0000 7fe337400640 1 -- 172.21.15.63:0/3533780635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e357300 msgr2=0x557d3e70ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.243+0000 7fe337400640 1 --2- 172.21.15.63:0/3533780635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e357300 0x557d3e70ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.243+0000 7fe337400640 1 -- 172.21.15.63:0/3533780635 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3e70df40 con 0x557d3e3424d0 2024-05-27T07:35:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.243+0000 7fe337400640 1 --2- 172.21.15.63:0/3533780635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e727400 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe330012fd0 tx=0x7fe330003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea5eaa163072a55 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.244+0000 7fe323400640 1 -- 172.21.15.63:0/3533780635 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330005070 con 0x557d3e3424d0 2024-05-27T07:35:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.244+0000 7fe323400640 1 -- 172.21.15.63:0/3533780635 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe330027070 con 0x557d3e3424d0 2024-05-27T07:35:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.244+0000 7fe323400640 1 -- 172.21.15.63:0/3533780635 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33000fc30 con 0x557d3e3424d0 2024-05-27T07:35:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.245+0000 7fe3449d5740 1 -- 172.21.15.63:0/3533780635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 msgr2=0x557d3e727400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.245+0000 7fe3449d5740 1 --2- 172.21.15.63:0/3533780635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e727400 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe330012fd0 tx=0x7fe330003a20 comp rx=0 tx=0).stop 2024-05-27T07:35:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.245+0000 7fe3449d5740 1 -- 172.21.15.63:0/3533780635 shutdown_connections 2024-05-27T07:35:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.245+0000 7fe3449d5740 1 --2- 172.21.15.63:0/3533780635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e727400 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.245+0000 7fe3449d5740 1 --2- 172.21.15.63:0/3533780635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e33a2f0 0x557d3e711b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.245+0000 7fe3449d5740 1 --2- 172.21.15.63:0/3533780635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e357300 0x557d3e70ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.245+0000 7fe3449d5740 1 -- 172.21.15.63:0/3533780635 >> 172.21.15.63:0/3533780635 conn(0x557d3e709dc0 msgr2=0x557d3e70a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.245+0000 7fe3449d5740 1 -- 172.21.15.63:0/3533780635 shutdown_connections 2024-05-27T07:35:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.246+0000 7fe3449d5740 1 -- 172.21.15.63:0/3533780635 wait complete. 2024-05-27T07:35:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.246+0000 7fe3449d5740 1 -- start start 2024-05-27T07:35:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.247+0000 7fe3449d5740 1 Processor -- start 2024-05-27T07:35:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.248+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e33a2f0 0x557d3e715110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.248+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e715650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.248+0000 7fe3449d5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e357300 0x557d3e7882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.248+0000 7fe3449d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381b5d90 con 0x557d3e3424d0 2024-05-27T07:35:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.248+0000 7fe3449d5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381b7200 con 0x557d3e357300 2024-05-27T07:35:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.248+0000 7fe3449d5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381b6460 con 0x557d3e33a2f0 2024-05-27T07:35:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.248+0000 7fe3449d5740 10 client.0 _ll_register_callbacks cb 0x7fe324009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:35:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.249+0000 7fe336000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e715650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.249+0000 7fe336000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e715650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57936/0 (socket says 172.21.15.63:57936) 2024-05-27T07:35:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.249+0000 7fe336000640 1 -- 172.21.15.63:0/2547042229 learned_addr learned my addr 172.21.15.63:0/2547042229 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:35:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.249+0000 7fe336a00640 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e33a2f0 0x557d3e715110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.250+0000 7fe336000640 1 -- 172.21.15.63:0/2547042229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e33a2f0 msgr2=0x557d3e715110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.250+0000 7fe336000640 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e33a2f0 0x557d3e715110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.250+0000 7fe336000640 1 -- 172.21.15.63:0/2547042229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e357300 msgr2=0x557d3e7882d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:35:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.250+0000 7fe336000640 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e357300 0x557d3e7882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.250+0000 7fe336000640 1 -- 172.21.15.63:0/2547042229 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d3e788810 con 0x557d3e3424d0 2024-05-27T07:35:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.251+0000 7fe336000640 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e715650 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x557d3e7126c0 tx=0x7fe3240d2900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcac8b455527a150 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.251+0000 7fe320c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:35:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.251+0000 7fe320c00640 1 -- 172.21.15.63:0/2547042229 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324049070 con 0x557d3e3424d0 2024-05-27T07:35:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.251+0000 7fe320c00640 1 -- 172.21.15.63:0/2547042229 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fe3240c91a0 con 0x557d3e3424d0 2024-05-27T07:35:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.251+0000 7fe3449d5740 1 -- 172.21.15.63:0/2547042229 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe330027070 con 0x557d3e3424d0 2024-05-27T07:35:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.252+0000 7fe320c00640 1 -- 172.21.15.63:0/2547042229 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3240af050 con 0x557d3e3424d0 2024-05-27T07:35:58.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.253+0000 7fe3449d5740 10 client.15897 fetch_fsmap learned FSMap version 28 2024-05-27T07:35:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.253+0000 7fe3449d5740 1 -- 172.21.15.63:0/2547042229 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d3e788810 con 0x557d3e3424d0 2024-05-27T07:35:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.253+0000 7fe320c00640 1 -- 172.21.15.63:0/2547042229 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe330027070 con 0x557d3e3424d0 2024-05-27T07:35:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.253+0000 7fe320c00640 1 -- 172.21.15.63:0/2547042229 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe3240cf1c0 con 0x557d3e3424d0 2024-05-27T07:35:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.253+0000 7fe3449d5740 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:35:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.253+0000 7fe3449d5740 10 client.15897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:35:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.253+0000 7fe3449d5740 20 client.15897 populate_metadata read hostname 'smithi063' 2024-05-27T07:35:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.254+0000 7fe3449d5740 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557d3e3631b0 0x557d3e78bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.254+0000 7fe3449d5740 4 client.15897 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:35:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.254+0000 7fe3449d5740 1 -- 172.21.15.63:0/2547042229 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d3e6722c0 con 0x557d3e3631b0 2024-05-27T07:35:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.255+0000 7fe336a00640 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557d3e3631b0 0x557d3e78bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.256+0000 7fe336a00640 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557d3e3631b0 0x557d3e78bec0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8493fc560ec475 server_cookie=628d9dd9f4b79ab4 in_seq=0 out_seq=0 2024-05-27T07:35:58.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.256+0000 7fe320c00640 10 client.15897 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.327+0000 7fe320c00640 1 -- 172.21.15.63:0/2547042229 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557d3e6722c0 con 0x557d3e3631b0 2024-05-27T07:35:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.327+0000 7fe320c00640 10 client.15897 handle_command_reply: tid=0 2024-05-27T07:35:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.327+0000 7fe3449d5740 1 client.15897 shutdown 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 1 -- 172.21.15.63:0/2547042229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 msgr2=0x557d3e715650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e715650 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x557d3e7126c0 tx=0x7fe3240d2900 comp rx=0 tx=0).stop 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 1 -- 172.21.15.63:0/2547042229 shutdown_connections 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557d3e3631b0 0x557d3e78bec0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d3e357300 0x557d3e7882d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d3e3424d0 0x557d3e715650 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 1 --2- 172.21.15.63:0/2547042229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d3e33a2f0 0x557d3e715110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 1 -- 172.21.15.63:0/2547042229 >> 172.21.15.63:0/2547042229 conn(0x557d3e709dc0 msgr2=0x557d3e7117d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe320c00640 0 client.15897 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 1 -- 172.21.15.63:0/2547042229 shutdown_connections 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 1 -- 172.21.15.63:0/2547042229 wait complete. 2024-05-27T07:35:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.328+0000 7fe3449d5740 20 client.15897 trim_cache size 0 max 16384 2024-05-27T07:35:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:35:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:35:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:35:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 -- 172.21.15.63:0/2121563424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe31c0b29d0 msgr2=0x7fe31c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe31c0b29d0 0x7fe31c0b4ee0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x557d3e706c10 tx=0x7fe328005f90 comp rx=0 tx=0).stop 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 -- 172.21.15.63:0/2121563424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe338112310 msgr2=0x7fe3381a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe338112310 0x7fe3381a1340 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe324009230 tx=0x7fe3240319a0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 -- 172.21.15.63:0/2121563424 shutdown_connections 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe31c0b29d0 0x7fe31c0b4ee0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338114c70 0x7fe3381a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe338112310 0x7fe3381a1340 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 --2- 172.21.15.63:0/2121563424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338075480 0x7fe3381a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 -- 172.21.15.63:0/2121563424 >> 172.21.15.63:0/2121563424 conn(0x7fe3380ff480 msgr2=0x7fe3380ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 -- 172.21.15.63:0/2121563424 shutdown_connections 2024-05-27T07:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:35:58.329+0000 7fe33da00640 1 -- 172.21.15.63:0/2121563424 wait complete. 2024-05-27T07:35:58.361 INFO:tasks.workunit.client.0.smithi063.stdout:checking for srand48_r... yes 2024-05-27T07:35:58.424 INFO:tasks.workunit.client.0.smithi063.stdout:configure: creating ./config.status 2024-05-27T07:35:58.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:58 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:58.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:58 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:58.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:58 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:58.832 INFO:tasks.workunit.client.0.smithi063.stdout:config.status: creating Makefile 2024-05-27T07:35:58.904 INFO:tasks.workunit.client.0.smithi063.stdout:config.status: creating config.h 2024-05-27T07:35:58.992 INFO:tasks.workunit.client.0.smithi063.stdout:config.status: executing depfiles commands 2024-05-27T07:35:59.220 INFO:tasks.workunit.client.0.smithi063.stdout:config.status: executing default commands 2024-05-27T07:35:59.444 INFO:tasks.workunit.client.0.smithi063.stderr:+ make 2024-05-27T07:35:59.603 INFO:tasks.workunit.client.0.smithi063.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-05-27T07:35:59.645 INFO:tasks.workunit.client.0.smithi063.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-05-27T07:35:59.651 INFO:tasks.workunit.client.0.smithi063.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-05-27T07:35:59.651 INFO:tasks.workunit.client.0.smithi063.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-05-27T07:35:59.651 INFO:tasks.workunit.client.0.smithi063.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-05-27T07:35:59.651 INFO:tasks.workunit.client.0.smithi063.stderr: any GNU archive site. 2024-05-27T07:35:59.677 INFO:tasks.workunit.client.0.smithi063.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-05-27T07:35:59.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:59 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:59.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:59 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:35:59.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:59 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:35:59.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:35:59 smithi086 ceph-mon[34687]: pgmap v75: 129 pgs: 129 active+clean; 6.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 210 KiB/s wr, 74 op/s 2024-05-27T07:35:59.704 INFO:tasks.workunit.client.0.smithi063.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-05-27T07:35:59.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:59 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:59.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:59 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:35:59.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:59 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:35:59.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:35:59 smithi149 ceph-mon[35408]: pgmap v75: 129 pgs: 129 active+clean; 6.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 210 KiB/s wr, 74 op/s 2024-05-27T07:35:59.708 INFO:tasks.workunit.client.0.smithi063.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-05-27T07:35:59.709 INFO:tasks.workunit.client.0.smithi063.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-05-27T07:35:59.709 INFO:tasks.workunit.client.0.smithi063.stderr: You might want to install the `Automake' and `Perl' packages. 2024-05-27T07:35:59.709 INFO:tasks.workunit.client.0.smithi063.stderr: Grab them from any GNU archive site. 2024-05-27T07:35:59.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:59 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:59.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:59 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:35:59.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:59 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:35:59.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:35:59 smithi063 ceph-mon[31613]: pgmap v75: 129 pgs: 129 active+clean; 6.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 210 KiB/s wr, 74 op/s 2024-05-27T07:36:00.670 INFO:tasks.workunit.client.0.smithi063.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-05-27T07:36:01.359 INFO:tasks.workunit.client.0.smithi063.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-05-27T07:36:01.361 INFO:tasks.workunit.client.0.smithi063.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-05-27T07:36:01.361 INFO:tasks.workunit.client.0.smithi063.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-05-27T07:36:01.361 INFO:tasks.workunit.client.0.smithi063.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-05-27T07:36:01.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:01 smithi086 ceph-mon[34687]: pgmap v76: 129 pgs: 129 active+clean; 3.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 319 KiB/s wr, 129 op/s 2024-05-27T07:36:01.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:01 smithi149 ceph-mon[35408]: pgmap v76: 129 pgs: 129 active+clean; 3.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 319 KiB/s wr, 129 op/s 2024-05-27T07:36:01.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:01 smithi063 ceph-mon[31613]: pgmap v76: 129 pgs: 129 active+clean; 3.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 319 KiB/s wr, 129 op/s 2024-05-27T07:36:01.740 INFO:tasks.workunit.client.0.smithi063.stdout:/bin/sh ./config.status --recheck 2024-05-27T07:36:02.025 INFO:tasks.workunit.client.0.smithi063.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-05-27T07:36:02.667 INFO:tasks.workunit.client.0.smithi063.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-27T07:36:03.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:03 smithi086 ceph-mon[34687]: pgmap v77: 129 pgs: 129 active+clean; 3.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 206 KiB/s wr, 81 op/s 2024-05-27T07:36:03.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:03 smithi149 ceph-mon[35408]: pgmap v77: 129 pgs: 129 active+clean; 3.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 206 KiB/s wr, 81 op/s 2024-05-27T07:36:03.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:03 smithi063 ceph-mon[31613]: pgmap v77: 129 pgs: 129 active+clean; 3.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 206 KiB/s wr, 81 op/s 2024-05-27T07:36:03.711 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether build environment is sane... yes 2024-05-27T07:36:03.744 INFO:tasks.workunit.client.0.smithi063.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-27T07:36:03.746 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gawk... gawk 2024-05-27T07:36:03.817 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether make sets $(MAKE)... yes 2024-05-27T07:36:03.958 INFO:tasks.workunit.client.0.smithi063.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-27T07:36:03.959 INFO:tasks.workunit.client.0.smithi063.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-27T07:36:04.018 INFO:tasks.workunit.client.0.smithi063.stdout:checking for style of include used by make... GNU 2024-05-27T07:36:04.044 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gcc... gcc 2024-05-27T07:36:04.493 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether the C compiler works... yes 2024-05-27T07:36:04.497 INFO:tasks.workunit.client.0.smithi063.stdout:checking for C compiler default output file name... a.out 2024-05-27T07:36:04.729 INFO:tasks.workunit.client.0.smithi063.stdout:checking for suffix of executables... 2024-05-27T07:36:04.958 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether we are cross compiling... no 2024-05-27T07:36:05.141 INFO:tasks.workunit.client.0.smithi063.stdout:checking for suffix of object files... o 2024-05-27T07:36:05.293 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether we are using the GNU C compiler... yes 2024-05-27T07:36:05.439 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether gcc accepts -g... yes 2024-05-27T07:36:05.617 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-27T07:36:05.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:05 smithi086 ceph-mon[34687]: pgmap v78: 129 pgs: 129 active+clean; 3.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 206 KiB/s wr, 81 op/s 2024-05-27T07:36:05.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:05 smithi149 ceph-mon[35408]: pgmap v78: 129 pgs: 129 active+clean; 3.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 206 KiB/s wr, 81 op/s 2024-05-27T07:36:05.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:05 smithi063 ceph-mon[31613]: pgmap v78: 129 pgs: 129 active+clean; 3.0 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 206 KiB/s wr, 81 op/s 2024-05-27T07:36:06.244 INFO:tasks.workunit.client.0.smithi063.stdout:checking dependency style of gcc... gcc3 2024-05-27T07:36:06.499 INFO:tasks.workunit.client.0.smithi063.stdout:checking how to run the C preprocessor... gcc -E 2024-05-27T07:36:06.761 INFO:tasks.workunit.client.0.smithi063.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-27T07:36:06.769 INFO:tasks.workunit.client.0.smithi063.stdout:checking for egrep... /usr/bin/grep -E 2024-05-27T07:36:07.423 INFO:tasks.workunit.client.0.smithi063.stdout:checking for ANSI C header files... yes 2024-05-27T07:36:07.639 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/types.h... yes 2024-05-27T07:36:07.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:07 smithi086 ceph-mon[34687]: pgmap v79: 129 pgs: 129 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 318 KiB/s wr, 100 op/s 2024-05-27T07:36:07.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:07 smithi149 ceph-mon[35408]: pgmap v79: 129 pgs: 129 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 318 KiB/s wr, 100 op/s 2024-05-27T07:36:07.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:07 smithi063 ceph-mon[31613]: pgmap v79: 129 pgs: 129 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 318 KiB/s wr, 100 op/s 2024-05-27T07:36:07.862 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/stat.h... yes 2024-05-27T07:36:08.085 INFO:tasks.workunit.client.0.smithi063.stdout:checking for stdlib.h... yes 2024-05-27T07:36:08.320 INFO:tasks.workunit.client.0.smithi063.stdout:checking for string.h... yes 2024-05-27T07:36:08.479 INFO:tasks.workunit.client.0.smithi063.stdout:checking for memory.h... yes 2024-05-27T07:36:08.692 INFO:tasks.workunit.client.0.smithi063.stdout:checking for strings.h... yes 2024-05-27T07:36:08.928 INFO:tasks.workunit.client.0.smithi063.stdout:checking for inttypes.h... yes 2024-05-27T07:36:09.149 INFO:tasks.workunit.client.0.smithi063.stdout:checking for stdint.h... yes 2024-05-27T07:36:09.376 INFO:tasks.workunit.client.0.smithi063.stdout:checking for unistd.h... yes 2024-05-27T07:36:09.624 INFO:tasks.workunit.client.0.smithi063.stdout:checking minix/config.h usability... no 2024-05-27T07:36:09.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:09 smithi086 ceph-mon[34687]: pgmap v80: 129 pgs: 129 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 221 KiB/s wr, 74 op/s 2024-05-27T07:36:09.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:09 smithi149 ceph-mon[35408]: pgmap v80: 129 pgs: 129 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 221 KiB/s wr, 74 op/s 2024-05-27T07:36:09.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:09 smithi063 ceph-mon[31613]: pgmap v80: 129 pgs: 129 active+clean; 4.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 221 KiB/s wr, 74 op/s 2024-05-27T07:36:09.789 INFO:tasks.workunit.client.0.smithi063.stdout:checking minix/config.h presence... no 2024-05-27T07:36:09.791 INFO:tasks.workunit.client.0.smithi063.stdout:checking for minix/config.h... no 2024-05-27T07:36:09.954 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-05-27T07:36:09.972 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gcc... (cached) gcc 2024-05-27T07:36:10.210 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-27T07:36:10.212 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-27T07:36:10.215 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-27T07:36:10.217 INFO:tasks.workunit.client.0.smithi063.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-27T07:36:10.465 INFO:tasks.workunit.client.0.smithi063.stdout:checking for main in -lm... yes 2024-05-27T07:36:10.737 INFO:tasks.workunit.client.0.smithi063.stdout:checking for main in -lpthread... yes 2024-05-27T07:36:10.754 INFO:tasks.workunit.client.0.smithi063.stdout:checking for ANSI C header files... (cached) yes 2024-05-27T07:36:10.921 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-27T07:36:11.149 INFO:tasks.workunit.client.0.smithi063.stdout:checking pthread.h usability... yes 2024-05-27T07:36:11.251 INFO:tasks.workunit.client.0.smithi063.stdout:checking pthread.h presence... yes 2024-05-27T07:36:11.254 INFO:tasks.workunit.client.0.smithi063.stdout:checking for pthread.h... yes 2024-05-27T07:36:11.445 INFO:tasks.workunit.client.0.smithi063.stdout:checking fcntl.h usability... yes 2024-05-27T07:36:11.523 INFO:tasks.workunit.client.0.smithi063.stdout:checking fcntl.h presence... yes 2024-05-27T07:36:11.524 INFO:tasks.workunit.client.0.smithi063.stdout:checking for fcntl.h... yes 2024-05-27T07:36:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.557+0000 7f32b6600640 1 -- 172.21.15.63:0/3608963192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b01032e0 msgr2=0x7f32b010f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.557+0000 7f32b6600640 1 --2- 172.21.15.63:0/3608963192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b01032e0 0x7f32b010f6b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009900 tx=0x7f32a802a3f0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.557+0000 7f32b6600640 1 -- 172.21.15.63:0/3608963192 shutdown_connections 2024-05-27T07:36:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.557+0000 7f32b6600640 1 --2- 172.21.15.63:0/3608963192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b01032e0 0x7f32b010f6b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.557+0000 7f32b6600640 1 --2- 172.21.15.63:0/3608963192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b0109030 0x7f32b0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.557+0000 7f32b6600640 1 --2- 172.21.15.63:0/3608963192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b0108b90 0x7f32b0102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.557+0000 7f32b6600640 1 -- 172.21.15.63:0/3608963192 >> 172.21.15.63:0/3608963192 conn(0x7f32b00782d0 msgr2=0x7f32b0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.558+0000 7f32b6600640 1 -- 172.21.15.63:0/3608963192 shutdown_connections 2024-05-27T07:36:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.558+0000 7f32b6600640 1 -- 172.21.15.63:0/3608963192 wait complete. 2024-05-27T07:36:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.559+0000 7f32b6600640 1 Processor -- start 2024-05-27T07:36:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.559+0000 7f32b6600640 1 -- start start 2024-05-27T07:36:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.559+0000 7f32b6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b01032e0 0x7f32b0111290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.559+0000 7f32b6600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b0108b90 0x7f32b01117d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.559+0000 7f32b6600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0109030 0x7f32b0111d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.559+0000 7f32b6600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0113d50 con 0x7f32b0109030 2024-05-27T07:36:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.559+0000 7f32b6600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0113bd0 con 0x7f32b01032e0 2024-05-27T07:36:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.559+0000 7f32b6600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0113ed0 con 0x7f32b0108b90 2024-05-27T07:36:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.560+0000 7f32afe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0109030 0x7f32b0111d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.560+0000 7f32afe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0109030 0x7f32b0111d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50002/0 (socket says 172.21.15.63:50002) 2024-05-27T07:36:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.560+0000 7f32afe00640 1 -- 172.21.15.63:0/2463369334 learned_addr learned my addr 172.21.15.63:0/2463369334 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.560+0000 7f32af400640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b01032e0 0x7f32b0111290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.560+0000 7f32a6a00640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b0108b90 0x7f32b01117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.560+0000 7f32afe00640 1 -- 172.21.15.63:0/2463369334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b0108b90 msgr2=0x7f32b01117d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.560+0000 7f32afe00640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b0108b90 0x7f32b01117d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.560+0000 7f32afe00640 1 -- 172.21.15.63:0/2463369334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b01032e0 msgr2=0x7f32b0111290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.560+0000 7f32afe00640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b01032e0 0x7f32b0111290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.560+0000 7f32afe00640 1 -- 172.21.15.63:0/2463369334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b01a8b10 con 0x7f32b0109030 2024-05-27T07:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.561+0000 7f32afe00640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0109030 0x7f32b0111d10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f32a8005ab0 tx=0x7f32a80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.561+0000 7f32acc00640 1 -- 172.21.15.63:0/2463369334 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a803f030 con 0x7f32b0109030 2024-05-27T07:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.561+0000 7f32acc00640 1 -- 172.21.15.63:0/2463369334 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32a8003a80 con 0x7f32b0109030 2024-05-27T07:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.561+0000 7f32acc00640 1 -- 172.21.15.63:0/2463369334 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8035070 con 0x7f32b0109030 2024-05-27T07:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.561+0000 7f32b6600640 1 -- 172.21.15.63:0/2463369334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b01a8da0 con 0x7f32b0109030 2024-05-27T07:36:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.561+0000 7f32b6600640 1 -- 172.21.15.63:0/2463369334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b01a9180 con 0x7f32b0109030 2024-05-27T07:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.564+0000 7f32bd5bc740 1 Processor -- start 2024-05-27T07:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.565+0000 7f32acc00640 1 -- 172.21.15.63:0/2463369334 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f32a8042030 con 0x7f32b0109030 2024-05-27T07:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.565+0000 7f32bd5bc740 1 -- start start 2024-05-27T07:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.566+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4311b0 0x55eb3d7d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.566+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.566+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4104d0 0x55eb3d7d42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.566+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d744b90 con 0x55eb3d4311b0 2024-05-27T07:36:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.566+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d734730 con 0x55eb3d4104d0 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.566+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d738770 con 0x55eb3d4082f0 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.566+0000 7f32a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4311b0 0x55eb3d7d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.566+0000 7f32a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4311b0 0x55eb3d7d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50016/0 (socket says 172.21.15.63:50016) 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.566+0000 7f32a6a00640 1 -- 172.21.15.63:0/2966997086 learned_addr learned my addr 172.21.15.63:0/2966997086 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32af400640 1 --2- 172.21.15.63:0/2966997086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32a6a00640 1 -- 172.21.15.63:0/2966997086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 msgr2=0x55eb3d7d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32a6a00640 1 --2- 172.21.15.63:0/2966997086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32a6a00640 1 -- 172.21.15.63:0/2966997086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4104d0 msgr2=0x55eb3d7d42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32a6a00640 1 --2- 172.21.15.63:0/2966997086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4104d0 0x55eb3d7d42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32a6a00640 1 -- 172.21.15.63:0/2966997086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3d6cd8b0 con 0x55eb3d4311b0 2024-05-27T07:36:11.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32a6a00640 1 --2- 172.21.15.63:0/2966997086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4311b0 0x55eb3d7d3860 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f329800bc50 tx=0x7f3298011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68f387d6f524b69 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32acc00640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32940b2950 0x7f32940b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32a4c00640 1 -- 172.21.15.63:0/2966997086 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329801a400 con 0x55eb3d4311b0 2024-05-27T07:36:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32acc00640 1 -- 172.21.15.63:0/2463369334 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f32a8102fb0 con 0x7f32b0109030 2024-05-27T07:36:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.567+0000 7f32a4c00640 1 -- 172.21.15.63:0/2966997086 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f329801aa30 con 0x55eb3d4311b0 2024-05-27T07:36:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.568+0000 7f32a4c00640 1 -- 172.21.15.63:0/2966997086 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298012880 con 0x55eb3d4311b0 2024-05-27T07:36:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.568+0000 7f32af400640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32940b2950 0x7f32940b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.568+0000 7f32bd5bc740 1 -- 172.21.15.63:0/2966997086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4311b0 msgr2=0x55eb3d7d3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.568+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/2966997086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4311b0 0x55eb3d7d3860 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f329800bc50 tx=0x7f3298011750 comp rx=0 tx=0).stop 2024-05-27T07:36:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.569+0000 7f32af400640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32940b2950 0x7f32940b4e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55eb3d7d4c10 tx=0x7f32a0009580 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.569+0000 7f32bd5bc740 1 -- 172.21.15.63:0/2966997086 shutdown_connections 2024-05-27T07:36:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.569+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/2966997086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4104d0 0x55eb3d7d42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.569+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/2966997086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.569+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/2966997086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4311b0 0x55eb3d7d3860 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.569+0000 7f32bd5bc740 1 -- 172.21.15.63:0/2966997086 >> 172.21.15.63:0/2966997086 conn(0x55eb3d425300 msgr2=0x55eb3d7d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.569+0000 7f32bd5bc740 1 -- 172.21.15.63:0/2966997086 shutdown_connections 2024-05-27T07:36:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.569+0000 7f32bd5bc740 1 -- 172.21.15.63:0/2966997086 wait complete. 2024-05-27T07:36:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.569+0000 7f32bd5bc740 1 -- start start 2024-05-27T07:36:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.570+0000 7f32bd5bc740 1 Processor -- start 2024-05-27T07:36:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.570+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7dda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.570+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7ddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.570+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.570+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d41a790 con 0x55eb3d4104d0 2024-05-27T07:36:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.570+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d6dcbc0 con 0x55eb3d4311b0 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.570+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d7e0990 con 0x55eb3d4082f0 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.570+0000 7f32bd5bc740 10 client.0 _ll_register_callbacks cb 0x55eb3d6ba400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7ddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7ddfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50026/0 (socket says 172.21.15.63:50026) 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32a6a00640 1 -- 172.21.15.63:0/3832560114 learned_addr learned my addr 172.21.15.63:0/3832560114 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32af400640 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32afe00640 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32a6a00640 1 -- 172.21.15.63:0/3832560114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 msgr2=0x55eb3d7dda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32a6a00640 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7dda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32a6a00640 1 -- 172.21.15.63:0/3832560114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 msgr2=0x55eb3d7e9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32a6a00640 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32a6a00640 1 -- 172.21.15.63:0/3832560114 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3d449210 con 0x55eb3d4104d0 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.571+0000 7f32afe00640 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.572+0000 7f32a6a00640 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7ddfa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f329800bfd0 tx=0x7f329801abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f91617ca4ea9e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.572+0000 7f329d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.572+0000 7f329d600640 1 -- 172.21.15.63:0/3832560114 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329801a610 con 0x55eb3d4104d0 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.572+0000 7f329d600640 1 -- 172.21.15.63:0/3832560114 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f329801a7b0 con 0x55eb3d4104d0 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.572+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3832560114 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329801aa30 con 0x55eb3d4104d0 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f329d600640 1 -- 172.21.15.63:0/3832560114 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3298030050 con 0x55eb3d4104d0 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f32bd5bc740 10 client.15909 fetch_fsmap learned FSMap version 28 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3832560114 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb3d515620 con 0x55eb3d4104d0 2024-05-27T07:36:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f329d600640 1 -- 172.21.15.63:0/3832560114 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f329801aa30 con 0x55eb3d4104d0 2024-05-27T07:36:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f329d600640 1 -- 172.21.15.63:0/3832560114 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3298004b50 con 0x55eb3d4104d0 2024-05-27T07:36:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f32bd5bc740 10 client.15909 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:36:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f32bd5bc740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f32bd5bc740 20 client.15909 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3efd80 0x55eb3d7efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f32bd5bc740 4 client.15909 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:36:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.573+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3832560114 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb3d746ae0 con 0x55eb3d3efd80 2024-05-27T07:36:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.574+0000 7f32af400640 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3efd80 0x55eb3d7efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.575+0000 7f329d600640 10 client.15909 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.575+0000 7f32af400640 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3efd80 0x55eb3d7efc10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a66f8f7d32d499c server_cookie=92d6416a156a88cd in_seq=0 out_seq=0 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.577+0000 7f329d600640 1 -- 172.21.15.63:0/3832560114 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eb3d746ae0 con 0x55eb3d3efd80 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.577+0000 7f329d600640 10 client.15909 handle_command_reply: tid=0 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.577+0000 7f32bd5bc740 1 client.15909 shutdown 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.577+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3832560114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 msgr2=0x55eb3d7ddfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.577+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7ddfa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f329800bfd0 tx=0x7f329801abe0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.578+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3832560114 shutdown_connections 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.578+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3efd80 0x55eb3d7efc10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.578+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.578+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7ddfa0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.578+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3832560114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7dda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.578+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3832560114 >> 172.21.15.63:0/3832560114 conn(0x55eb3d425300 msgr2=0x55eb3d7d3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.578+0000 7f329d600640 0 client.15909 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.578+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3832560114 shutdown_connections 2024-05-27T07:36:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.578+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3832560114 wait complete. 2024-05-27T07:36:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.578+0000 7f32bd5bc740 20 client.15909 trim_cache size 0 max 16384 2024-05-27T07:36:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.597+0000 7f32bd5bc740 1 Processor -- start 2024-05-27T07:36:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.597+0000 7f32bd5bc740 1 -- start start 2024-05-27T07:36:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.597+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.597+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.597+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.597+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b5b20 con 0x55eb3d4104d0 2024-05-27T07:36:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.597+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b33f0 con 0x55eb3d4311b0 2024-05-27T07:36:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.597+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b1240 con 0x55eb3d4082f0 2024-05-27T07:36:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.597+0000 7f32a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50042/0 (socket says 172.21.15.63:50042) 2024-05-27T07:36:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32a6a00640 1 -- 172.21.15.63:0/1082304518 learned_addr learned my addr 172.21.15.63:0/1082304518 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32af400640 1 --2- 172.21.15.63:0/1082304518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32afe00640 1 --2- 172.21.15.63:0/1082304518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32a6a00640 1 -- 172.21.15.63:0/1082304518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 msgr2=0x55eb3d7d9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32a6a00640 1 --2- 172.21.15.63:0/1082304518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7d9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32a6a00640 1 -- 172.21.15.63:0/1082304518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 msgr2=0x55eb3d7e9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32a6a00640 1 --2- 172.21.15.63:0/1082304518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32a6a00640 1 -- 172.21.15.63:0/1082304518 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3d7d4c40 con 0x55eb3d4104d0 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32af400640 1 --2- 172.21.15.63:0/1082304518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.598+0000 7f32a6a00640 1 --2- 172.21.15.63:0/1082304518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e9a30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3298011790 tx=0x7f3298005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb04834a5e80e6a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.599+0000 7f329fe00640 1 -- 172.21.15.63:0/1082304518 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298012d70 con 0x55eb3d4104d0 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.599+0000 7f329fe00640 1 -- 172.21.15.63:0/1082304518 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3298039070 con 0x55eb3d4104d0 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.599+0000 7f329fe00640 1 -- 172.21.15.63:0/1082304518 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298005290 con 0x55eb3d4104d0 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.600+0000 7f32bd5bc740 1 -- 172.21.15.63:0/1082304518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 msgr2=0x55eb3d7e9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.600+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/1082304518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e9a30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3298011790 tx=0x7f3298005f30 comp rx=0 tx=0).stop 2024-05-27T07:36:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.600+0000 7f32bd5bc740 1 -- 172.21.15.63:0/1082304518 shutdown_connections 2024-05-27T07:36:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.600+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/1082304518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.600+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/1082304518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e9a30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.600+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/1082304518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.600+0000 7f32bd5bc740 1 -- 172.21.15.63:0/1082304518 >> 172.21.15.63:0/1082304518 conn(0x55eb3d425300 msgr2=0x55eb3d7dfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.600+0000 7f32bd5bc740 1 -- 172.21.15.63:0/1082304518 shutdown_connections 2024-05-27T07:36:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.600+0000 7f32bd5bc740 1 -- 172.21.15.63:0/1082304518 wait complete. 2024-05-27T07:36:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.600+0000 7f32bd5bc740 1 -- start start 2024-05-27T07:36:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.601+0000 7f32bd5bc740 1 Processor -- start 2024-05-27T07:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.601+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7e2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.601+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.601+0000 7f32bd5bc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.601+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b18b0 con 0x55eb3d4104d0 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.601+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b2d20 con 0x55eb3d4311b0 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.601+0000 7f32bd5bc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01b1f80 con 0x55eb3d4082f0 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.601+0000 7f32bd5bc740 10 client.0 _ll_register_callbacks cb 0x7f329800bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.601+0000 7f32a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f32a6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50046/0 (socket says 172.21.15.63:50046) 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f32a6a00640 1 -- 172.21.15.63:0/3131150629 learned_addr learned my addr 172.21.15.63:0/3131150629 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f32afe00640 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f32af400640 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7e2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f32a6a00640 1 -- 172.21.15.63:0/3131150629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 msgr2=0x55eb3d7e2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f32a6a00640 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7e2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f32a6a00640 1 -- 172.21.15.63:0/3131150629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 msgr2=0x55eb3d7e33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f32a6a00640 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f32a6a00640 1 -- 172.21.15.63:0/3131150629 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3d856180 con 0x55eb3d4104d0 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f32a6a00640 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e2ea0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3298011db0 tx=0x7f32980126e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38104b450b3acdc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.602+0000 7f329d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.603+0000 7f329d600640 1 -- 172.21.15.63:0/3131150629 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298012b70 con 0x55eb3d4104d0 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.603+0000 7f329d600640 1 -- 172.21.15.63:0/3131150629 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f3298012d10 con 0x55eb3d4104d0 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.603+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3131150629 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3298039070 con 0x55eb3d4104d0 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.603+0000 7f329d600640 1 -- 172.21.15.63:0/3131150629 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f329804d050 con 0x55eb3d4104d0 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.603+0000 7f32bd5bc740 10 client.15915 fetch_fsmap learned FSMap version 28 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.603+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3131150629 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb3d8572b0 con 0x55eb3d4104d0 2024-05-27T07:36:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.603+0000 7f329d600640 1 -- 172.21.15.63:0/3131150629 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3298039070 con 0x55eb3d4104d0 2024-05-27T07:36:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.604+0000 7f329d600640 1 -- 172.21.15.63:0/3131150629 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3298004a60 con 0x55eb3d4104d0 2024-05-27T07:36:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.604+0000 7f32bd5bc740 10 client.15915 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:36:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.604+0000 7f32bd5bc740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.604+0000 7f32bd5bc740 20 client.15915 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.604+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3c51c0 0x55eb3d859ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.604+0000 7f32bd5bc740 4 client.15915 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:36:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.604+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3131150629 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb3d7402c0 con 0x55eb3d3c51c0 2024-05-27T07:36:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.604+0000 7f32af400640 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3c51c0 0x55eb3d859ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.605+0000 7f32af400640 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3c51c0 0x55eb3d859ab0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910551a0a4511a31 server_cookie=afcced79aec49323 in_seq=0 out_seq=0 2024-05-27T07:36:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.605+0000 7f329d600640 10 client.15915 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:11.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:11 smithi086 ceph-mon[34687]: pgmap v81: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 334 KiB/s wr, 99 op/s 2024-05-27T07:36:11.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:11 smithi149 ceph-mon[35408]: pgmap v81: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 334 KiB/s wr, 99 op/s 2024-05-27T07:36:11.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:11 smithi063 ceph-mon[31613]: pgmap v81: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 334 KiB/s wr, 99 op/s 2024-05-27T07:36:11.721 INFO:tasks.workunit.client.0.smithi063.stdout:checking limits.h usability... yes 2024-05-27T07:36:11.812 INFO:tasks.workunit.client.0.smithi063.stdout:checking limits.h presence... yes 2024-05-27T07:36:11.813 INFO:tasks.workunit.client.0.smithi063.stdout:checking for limits.h... yes 2024-05-27T07:36:11.908 INFO:tasks.workunit.client.0.smithi063.stdout:checking for stdint.h... (cached) yes 2024-05-27T07:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f329d600640 1 -- 172.21.15.63:0/3131150629 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55eb3d7402c0 con 0x55eb3d3c51c0 2024-05-27T07:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f329d600640 10 client.15915 handle_command_reply: tid=0 2024-05-27T07:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 client.15915 shutdown 2024-05-27T07:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3131150629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 msgr2=0x55eb3d7e2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e2ea0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3298011db0 tx=0x7f32980126e0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3131150629 shutdown_connections 2024-05-27T07:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3c51c0 0x55eb3d859ab0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d4311b0 0x55eb3d7e33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d4104d0 0x55eb3d7e2ea0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 --2- 172.21.15.63:0/3131150629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d4082f0 0x55eb3d7e2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f329d600640 0 client.15915 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3131150629 >> 172.21.15.63:0/3131150629 conn(0x55eb3d425300 msgr2=0x55eb3d7dcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3131150629 shutdown_connections 2024-05-27T07:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.933+0000 7f32bd5bc740 1 -- 172.21.15.63:0/3131150629 wait complete. 2024-05-27T07:36:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.934+0000 7f32bd5bc740 20 client.15915 trim_cache size 0 max 16384 2024-05-27T07:36:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:36:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:36:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:36:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.934+0000 7f32b6600640 1 -- 172.21.15.63:0/2463369334 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32940b2950 msgr2=0x7f32940b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.934+0000 7f32b6600640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32940b2950 0x7f32940b4e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55eb3d7d4c10 tx=0x7f32a0009580 comp rx=0 tx=0).stop 2024-05-27T07:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.934+0000 7f32b6600640 1 -- 172.21.15.63:0/2463369334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0109030 msgr2=0x7f32b0111d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.934+0000 7f32b6600640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0109030 0x7f32b0111d10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f32a8005ab0 tx=0x7f32a80040e0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.934+0000 7f32b6600640 1 -- 172.21.15.63:0/2463369334 shutdown_connections 2024-05-27T07:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.934+0000 7f32b6600640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32940b2950 0x7f32940b4e60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.935+0000 7f32b6600640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32b0109030 0x7f32b0111d10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.935+0000 7f32b6600640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b0108b90 0x7f32b01117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.935+0000 7f32b6600640 1 --2- 172.21.15.63:0/2463369334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b01032e0 0x7f32b0111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.935+0000 7f32b6600640 1 -- 172.21.15.63:0/2463369334 >> 172.21.15.63:0/2463369334 conn(0x7f32b00782d0 msgr2=0x7f32b00fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.935+0000 7f32b6600640 1 -- 172.21.15.63:0/2463369334 shutdown_connections 2024-05-27T07:36:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:11.935+0000 7f32b6600640 1 -- 172.21.15.63:0/2463369334 wait complete. 2024-05-27T07:36:12.057 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/time.h usability... yes 2024-05-27T07:36:12.138 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/time.h presence... yes 2024-05-27T07:36:12.147 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/time.h... yes 2024-05-27T07:36:12.220 INFO:tasks.workunit.client.0.smithi063.stdout:checking for unistd.h... (cached) yes 2024-05-27T07:36:12.424 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/vfs.h usability... yes 2024-05-27T07:36:12.510 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/vfs.h presence... yes 2024-05-27T07:36:12.512 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/vfs.h... yes 2024-05-27T07:36:12.775 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/limits.h usability... no 2024-05-27T07:36:12.936 INFO:tasks.workunit.client.0.smithi063.stdout:checking sys/limits.h presence... no 2024-05-27T07:36:12.938 INFO:tasks.workunit.client.0.smithi063.stdout:checking for sys/limits.h... no 2024-05-27T07:36:13.069 INFO:tasks.workunit.client.0.smithi063.stdout:checking for an ANSI C-conforming const... yes 2024-05-27T07:36:13.199 INFO:tasks.workunit.client.0.smithi063.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-27T07:36:13.541 INFO:tasks.workunit.client.0.smithi063.stdout:checking for system... yes 2024-05-27T07:36:13.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:13 smithi086 ceph-mon[34687]: pgmap v82: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 225 KiB/s wr, 43 op/s 2024-05-27T07:36:13.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:13 smithi149 ceph-mon[35408]: pgmap v82: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 225 KiB/s wr, 43 op/s 2024-05-27T07:36:13.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:13 smithi063 ceph-mon[31613]: pgmap v82: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 225 KiB/s wr, 43 op/s 2024-05-27T07:36:13.870 INFO:tasks.workunit.client.0.smithi063.stdout:checking for gettimeofday... yes 2024-05-27T07:36:14.233 INFO:tasks.workunit.client.0.smithi063.stdout:checking for mkdir... yes 2024-05-27T07:36:14.542 INFO:tasks.workunit.client.0.smithi063.stdout:checking for strerror... yes 2024-05-27T07:36:14.883 INFO:tasks.workunit.client.0.smithi063.stdout:checking for open64... yes 2024-05-27T07:36:15.209 INFO:tasks.workunit.client.0.smithi063.stdout:checking for stat64... yes 2024-05-27T07:36:15.538 INFO:tasks.workunit.client.0.smithi063.stdout:checking for fseeko64... yes 2024-05-27T07:36:15.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:15 smithi086 ceph-mon[34687]: pgmap v83: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 225 KiB/s wr, 43 op/s 2024-05-27T07:36:15.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:15 smithi149 ceph-mon[35408]: pgmap v83: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 225 KiB/s wr, 43 op/s 2024-05-27T07:36:15.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:15 smithi063 ceph-mon[31613]: pgmap v83: 129 pgs: 129 active+clean; 5.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 225 KiB/s wr, 43 op/s 2024-05-27T07:36:15.884 INFO:tasks.workunit.client.0.smithi063.stdout:checking for lrand48_r... yes 2024-05-27T07:36:16.406 INFO:tasks.workunit.client.0.smithi063.stdout:checking for srand48_r... yes 2024-05-27T07:36:16.494 INFO:tasks.workunit.client.0.smithi063.stdout:configure: creating ./config.status 2024-05-27T07:36:16.904 INFO:tasks.workunit.client.0.smithi063.stdout: /bin/sh ./config.status 2024-05-27T07:36:17.209 INFO:tasks.workunit.client.0.smithi063.stdout:config.status: creating Makefile 2024-05-27T07:36:17.282 INFO:tasks.workunit.client.0.smithi063.stdout:config.status: creating config.h 2024-05-27T07:36:17.399 INFO:tasks.workunit.client.0.smithi063.stdout:config.status: executing depfiles commands 2024-05-27T07:36:17.467 INFO:tasks.workunit.client.0.smithi063.stdout:config.status: executing default commands 2024-05-27T07:36:17.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:17 smithi086 ceph-mon[34687]: pgmap v84: 129 pgs: 129 active+clean; 6.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 328 KiB/s wr, 97 op/s 2024-05-27T07:36:17.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:17 smithi149 ceph-mon[35408]: pgmap v84: 129 pgs: 129 active+clean; 6.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 328 KiB/s wr, 97 op/s 2024-05-27T07:36:17.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:17 smithi063 ceph-mon[31613]: pgmap v84: 129 pgs: 129 active+clean; 6.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 328 KiB/s wr, 97 op/s 2024-05-27T07:36:17.735 INFO:tasks.workunit.client.0.smithi063.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-05-27T07:36:18.208 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f9beb200700 time 2024-05-27T07:36:17.841313+0000 2024-05-27T07:36:18.208 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:36:18.208 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:18.208 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f9c00d78ba6] 2024-05-27T07:36:18.208 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f9c00d78dd1] 2024-05-27T07:36:18.208 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x564c9f15b23d] 2024-05-27T07:36:18.208 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 4: (SnapRealm::build_snap_set() const+0x323) [0x564c9f146971] 2024-05-27T07:36:18.208 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 5: (SnapRealm::check_cache() const+0x13e) [0x564c9f144fa0] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x564c9f145651] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x564c9f0da347] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x564c9edd687e] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x564c9eddbe94] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x564c9ede6969] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 11: (MDLog::_replay_thread()+0x1f38) [0x564c9f17f48c] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 12: (MDLog::ReplayThread::entry()+0x11) [0x564c9f1889cb] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 13: (Thread::entry_wrapper()+0x43) [0x7f9c00d4772d] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 14: (Thread::_entry_func(void*)+0xd) [0x7f9c00d47749] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9bff8eb1ca] 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 16: clone() 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: *** Caught signal (Aborted) ** 2024-05-27T07:36:18.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: in thread 7f9beb200700 thread_name:md_log_replay 2024-05-27T07:36:18.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:17.843+0000 7f9beb200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f9beb200700 time 2024-05-27T07:36:17.841313+0000 2024-05-27T07:36:18.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:36:18.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:18.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f9c00d78ba6] 2024-05-27T07:36:18.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f9c00d78dd1] 2024-05-27T07:36:18.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x564c9f15b23d] 2024-05-27T07:36:18.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 4: (SnapRealm::build_snap_set() const+0x323) [0x564c9f146971] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 5: (SnapRealm::check_cache() const+0x13e) [0x564c9f144fa0] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x564c9f145651] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x564c9f0da347] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x564c9edd687e] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x564c9eddbe94] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x564c9ede6969] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 11: (MDLog::_replay_thread()+0x1f38) [0x564c9f17f48c] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 12: (MDLog::ReplayThread::entry()+0x11) [0x564c9f1889cb] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 13: (Thread::entry_wrapper()+0x43) [0x7f9c00d4772d] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 14: (Thread::_entry_func(void*)+0xd) [0x7f9c00d47749] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9bff8eb1ca] 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 16: clone() 2024-05-27T07:36:18.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.212 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:18.212 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9bff8f5d20] 2024-05-27T07:36:18.212 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2: gsignal() 2024-05-27T07:36:18.212 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 3: abort() 2024-05-27T07:36:18.212 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f9c00d78ced] 2024-05-27T07:36:18.212 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f9c00d78dd1] 2024-05-27T07:36:18.212 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x564c9f15b23d] 2024-05-27T07:36:18.212 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 7: (SnapRealm::build_snap_set() const+0x323) [0x564c9f146971] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 8: (SnapRealm::check_cache() const+0x13e) [0x564c9f144fa0] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x564c9f145651] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x564c9f0da347] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x564c9edd687e] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x564c9eddbe94] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x564c9ede6969] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 14: (MDLog::_replay_thread()+0x1f38) [0x564c9f17f48c] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 15: (MDLog::ReplayThread::entry()+0x11) [0x564c9f1889cb] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 16: (Thread::entry_wrapper()+0x43) [0x7f9c00d4772d] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 17: (Thread::_entry_func(void*)+0xd) [0x7f9c00d47749] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f9bff8eb1ca] 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 19: clone() 2024-05-27T07:36:18.213 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:17.847+0000 7f9beb200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: in thread 7f9beb200700 thread_name:md_log_replay 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9bff8f5d20] 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2: gsignal() 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 3: abort() 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f9c00d78ced] 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f9c00d78dd1] 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x564c9f15b23d] 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 7: (SnapRealm::build_snap_set() const+0x323) [0x564c9f146971] 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 8: (SnapRealm::check_cache() const+0x13e) [0x564c9f144fa0] 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x564c9f145651] 2024-05-27T07:36:18.214 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x564c9f0da347] 2024-05-27T07:36:18.215 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x564c9edd687e] 2024-05-27T07:36:18.215 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x564c9eddbe94] 2024-05-27T07:36:18.215 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x564c9ede6969] 2024-05-27T07:36:18.215 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 14: (MDLog::_replay_thread()+0x1f38) [0x564c9f17f48c] 2024-05-27T07:36:18.215 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 15: (MDLog::ReplayThread::entry()+0x11) [0x564c9f1889cb] 2024-05-27T07:36:18.215 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 16: (Thread::entry_wrapper()+0x43) [0x7f9c00d4772d] 2024-05-27T07:36:18.215 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 17: (Thread::_entry_func(void*)+0xd) [0x7f9c00d47749] 2024-05-27T07:36:18.215 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f9bff8eb1ca] 2024-05-27T07:36:18.215 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 19: clone() 2024-05-27T07:36:18.215 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: -1> 2024-05-27T07:36:17.843+0000 7f9beb200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f9beb200700 time 2024-05-27T07:36:17.841313+0000 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f9c00d78ba6] 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f9c00d78dd1] 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x564c9f15b23d] 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 4: (SnapRealm::build_snap_set() const+0x323) [0x564c9f146971] 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 5: (SnapRealm::check_cache() const+0x13e) [0x564c9f144fa0] 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x564c9f145651] 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x564c9f0da347] 2024-05-27T07:36:18.216 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x564c9edd687e] 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x564c9eddbe94] 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x564c9ede6969] 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 11: (MDLog::_replay_thread()+0x1f38) [0x564c9f17f48c] 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 12: (MDLog::ReplayThread::entry()+0x11) [0x564c9f1889cb] 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 13: (Thread::entry_wrapper()+0x43) [0x7f9c00d4772d] 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 14: (Thread::_entry_func(void*)+0xd) [0x7f9c00d47749] 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9bff8eb1ca] 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 16: clone() 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 0> 2024-05-27T07:36:17.847+0000 7f9beb200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:36:18.217 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: in thread 7f9beb200700 thread_name:md_log_replay 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9bff8f5d20] 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2: gsignal() 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 3: abort() 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f9c00d78ced] 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f9c00d78dd1] 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x564c9f15b23d] 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 7: (SnapRealm::build_snap_set() const+0x323) [0x564c9f146971] 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 8: (SnapRealm::check_cache() const+0x13e) [0x564c9f144fa0] 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x564c9f145651] 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x564c9f0da347] 2024-05-27T07:36:18.218 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x564c9edd687e] 2024-05-27T07:36:18.219 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x564c9eddbe94] 2024-05-27T07:36:18.219 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x564c9ede6969] 2024-05-27T07:36:18.219 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 14: (MDLog::_replay_thread()+0x1f38) [0x564c9f17f48c] 2024-05-27T07:36:18.219 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 15: (MDLog::ReplayThread::entry()+0x11) [0x564c9f1889cb] 2024-05-27T07:36:18.219 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 16: (Thread::entry_wrapper()+0x43) [0x7f9c00d4772d] 2024-05-27T07:36:18.219 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 17: (Thread::_entry_func(void*)+0xd) [0x7f9c00d47749] 2024-05-27T07:36:18.219 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f9bff8eb1ca] 2024-05-27T07:36:18.219 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 19: clone() 2024-05-27T07:36:18.219 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:36:18.219 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: -1> 2024-05-27T07:36:17.843+0000 7f9beb200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f9beb200700 time 2024-05-27T07:36:17.841313+0000 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f9c00d78ba6] 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f9c00d78dd1] 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x564c9f15b23d] 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 4: (SnapRealm::build_snap_set() const+0x323) [0x564c9f146971] 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 5: (SnapRealm::check_cache() const+0x13e) [0x564c9f144fa0] 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x564c9f145651] 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x564c9f0da347] 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x564c9edd687e] 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x564c9eddbe94] 2024-05-27T07:36:18.220 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x564c9ede6969] 2024-05-27T07:36:18.221 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 11: (MDLog::_replay_thread()+0x1f38) [0x564c9f17f48c] 2024-05-27T07:36:18.221 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 12: (MDLog::ReplayThread::entry()+0x11) [0x564c9f1889cb] 2024-05-27T07:36:18.221 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 13: (Thread::entry_wrapper()+0x43) [0x7f9c00d4772d] 2024-05-27T07:36:18.221 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 14: (Thread::_entry_func(void*)+0xd) [0x7f9c00d47749] 2024-05-27T07:36:18.221 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9bff8eb1ca] 2024-05-27T07:36:18.221 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 16: clone() 2024-05-27T07:36:18.221 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.221 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 0> 2024-05-27T07:36:17.847+0000 7f9beb200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: in thread 7f9beb200700 thread_name:md_log_replay 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9bff8f5d20] 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2: gsignal() 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 3: abort() 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f9c00d78ced] 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f9c00d78dd1] 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x564c9f15b23d] 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 7: (SnapRealm::build_snap_set() const+0x323) [0x564c9f146971] 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 8: (SnapRealm::check_cache() const+0x13e) [0x564c9f144fa0] 2024-05-27T07:36:18.222 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x564c9f145651] 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x564c9f0da347] 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x564c9edd687e] 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x564c9eddbe94] 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x564c9ede6969] 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 14: (MDLog::_replay_thread()+0x1f38) [0x564c9f17f48c] 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 15: (MDLog::ReplayThread::entry()+0x11) [0x564c9f1889cb] 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 16: (Thread::entry_wrapper()+0x43) [0x7f9c00d4772d] 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 17: (Thread::_entry_func(void*)+0xd) [0x7f9c00d47749] 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f9bff8eb1ca] 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 19: clone() 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[64172]: 2024-05-27T07:36:18.223 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:18 smithi086 podman[69317]: 2024-05-27 07:36:18.006592661 +0000 UTC m=+0.038533528 container died b6a4ca6736a2da27ceec249db559e39592a377bac3a2b6858a2b8db0e1ddf142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-27T07:36:18.226 INFO:tasks.workunit.client.0.smithi063.stdout:rm -f stamp-h1 2024-05-27T07:36:18.242 INFO:tasks.workunit.client.0.smithi063.stdout:touch config.h.in 2024-05-27T07:36:18.253 INFO:tasks.workunit.client.0.smithi063.stdout:cd . && /bin/sh ./config.status config.h 2024-05-27T07:36:18.517 INFO:tasks.workunit.client.0.smithi063.stdout:config.status: creating config.h 2024-05-27T07:36:18.663 INFO:tasks.workunit.client.0.smithi063.stdout:make all-am 2024-05-27T07:36:18.818 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-27T07:36:18.819 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-05-27T07:36:19.204 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:18 smithi086 podman[69317]: 2024-05-27 07:36:18.884905138 +0000 UTC m=+0.916846063 container remove b6a4ca6736a2da27ceec249db559e39592a377bac3a2b6858a2b8db0e1ddf142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521) 2024-05-27T07:36:19.204 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:18 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:36:19.292 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-05-27T07:36:19.326 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-05-27T07:36:19.704 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:19 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-05-27T07:36:19.705 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:19 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Consumed 4.405s CPU time. 2024-05-27T07:36:19.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:19 smithi086 ceph-mon[34687]: pgmap v85: 129 pgs: 129 active+clean; 6.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 216 KiB/s wr, 78 op/s 2024-05-27T07:36:19.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:19 smithi149 ceph-mon[35408]: pgmap v85: 129 pgs: 129 active+clean; 6.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 216 KiB/s wr, 78 op/s 2024-05-27T07:36:19.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:19 smithi063 ceph-mon[31613]: pgmap v85: 129 pgs: 129 active+clean; 6.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 216 KiB/s wr, 78 op/s 2024-05-27T07:36:19.954 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-05-27T07:36:19.984 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-05-27T07:36:20.687 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-05-27T07:36:20.722 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-05-27T07:36:21.349 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-05-27T07:36:21.375 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-05-27T07:36:21.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:21 smithi086 ceph-mon[34687]: pgmap v86: 129 pgs: 129 active+clean; 6.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 291 KiB/s wr, 97 op/s 2024-05-27T07:36:21.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:21 smithi149 ceph-mon[35408]: pgmap v86: 129 pgs: 129 active+clean; 6.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 291 KiB/s wr, 97 op/s 2024-05-27T07:36:21.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:21 smithi063 ceph-mon[31613]: pgmap v86: 129 pgs: 129 active+clean; 6.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 291 KiB/s wr, 97 op/s 2024-05-27T07:36:22.033 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-05-27T07:36:22.061 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-05-27T07:36:22.585 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-05-27T07:36:22.612 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-05-27T07:36:22.960 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-05-27T07:36:22.985 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-05-27T07:36:23.393 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-05-27T07:36:23.421 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-05-27T07:36:23.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:23 smithi086 ceph-mon[34687]: pgmap v87: 129 pgs: 129 active+clean; 6.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 178 KiB/s wr, 71 op/s 2024-05-27T07:36:23.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:23 smithi149 ceph-mon[35408]: pgmap v87: 129 pgs: 129 active+clean; 6.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 178 KiB/s wr, 71 op/s 2024-05-27T07:36:23.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:23 smithi063 ceph-mon[31613]: pgmap v87: 129 pgs: 129 active+clean; 6.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 178 KiB/s wr, 71 op/s 2024-05-27T07:36:24.000 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-05-27T07:36:24.025 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-05-27T07:36:24.061 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:36:24.537 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-05-27T07:36:24.557 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-05-27T07:36:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:36:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.162+0000 7fb4a6600640 1 -- 172.21.15.63:0/96717711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4a01029a0 msgr2=0x7fb4a0102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.162+0000 7fb4a6600640 1 --2- 172.21.15.63:0/96717711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4a01029a0 0x7fb4a0102db0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb490009900 tx=0x7fb49002a410 comp rx=0 tx=0).stop 2024-05-27T07:36:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.162+0000 7fb4a6600640 1 -- 172.21.15.63:0/96717711 shutdown_connections 2024-05-27T07:36:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.162+0000 7fb4a6600640 1 --2- 172.21.15.63:0/96717711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0110540 0x7fb4a0112960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.162+0000 7fb4a6600640 1 --2- 172.21.15.63:0/96717711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4a010dbe0 0x7fb4a0110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.162+0000 7fb4a6600640 1 --2- 172.21.15.63:0/96717711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4a01029a0 0x7fb4a0102db0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.162+0000 7fb4a6600640 1 -- 172.21.15.63:0/96717711 >> 172.21.15.63:0/96717711 conn(0x7fb4a00782d0 msgr2=0x7fb4a0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.162+0000 7fb4a6600640 1 -- 172.21.15.63:0/96717711 shutdown_connections 2024-05-27T07:36:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.162+0000 7fb4a6600640 1 -- 172.21.15.63:0/96717711 wait complete. 2024-05-27T07:36:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.164+0000 7fb4a6600640 1 Processor -- start 2024-05-27T07:36:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.164+0000 7fb4a6600640 1 -- start start 2024-05-27T07:36:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb4a6600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4a01029a0 0x7fb4a01a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb4a6600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4a010dbe0 0x7fb4a01a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb4a6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0110540 0x7fb4a019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb4a6600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0115ef0 con 0x7fb4a010dbe0 2024-05-27T07:36:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb4a6600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0115d70 con 0x7fb4a0110540 2024-05-27T07:36:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb4a6600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0116070 con 0x7fb4a01029a0 2024-05-27T07:36:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb49ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4a010dbe0 0x7fb4a01a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb49ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4a010dbe0 0x7fb4a01a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45960/0 (socket says 172.21.15.63:45960) 2024-05-27T07:36:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb49ea00640 1 -- 172.21.15.63:0/3588252907 learned_addr learned my addr 172.21.15.63:0/3588252907 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb49f400640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4a01029a0 0x7fb4a01a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.165+0000 7fb49fe00640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0110540 0x7fb4a019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.166+0000 7fb49ea00640 1 -- 172.21.15.63:0/3588252907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4a01029a0 msgr2=0x7fb4a01a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.166+0000 7fb49ea00640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4a01029a0 0x7fb4a01a25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.166+0000 7fb49ea00640 1 -- 172.21.15.63:0/3588252907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0110540 msgr2=0x7fb4a019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.166+0000 7fb49ea00640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0110540 0x7fb4a019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.166+0000 7fb49ea00640 1 -- 172.21.15.63:0/3588252907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a019ce60 con 0x7fb4a010dbe0 2024-05-27T07:36:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.166+0000 7fb49ea00640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4a010dbe0 0x7fb4a01a2b00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00d770 tx=0x7fb48c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.167+0000 7fb497e00640 1 -- 172.21.15.63:0/3588252907 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c017070 con 0x7fb4a010dbe0 2024-05-27T07:36:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.167+0000 7fb497e00640 1 -- 172.21.15.63:0/3588252907 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb48c0054b0 con 0x7fb4a010dbe0 2024-05-27T07:36:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.167+0000 7fb4a6600640 1 -- 172.21.15.63:0/3588252907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a019d150 con 0x7fb4a010dbe0 2024-05-27T07:36:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.167+0000 7fb497e00640 1 -- 172.21.15.63:0/3588252907 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c004c60 con 0x7fb4a010dbe0 2024-05-27T07:36:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.167+0000 7fb4a6600640 1 -- 172.21.15.63:0/3588252907 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a019d650 con 0x7fb4a010dbe0 2024-05-27T07:36:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.171+0000 7fb4ad5e6740 1 Processor -- start 2024-05-27T07:36:25.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.171+0000 7fb4ad5e6740 1 -- start start 2024-05-27T07:36:25.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.171+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9e61b0 0x562afad88860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.171+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad88da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.171+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.171+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562afacf9b90 con 0x562afa9c54d0 2024-05-27T07:36:25.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.171+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562aface9730 con 0x562afa9e61b0 2024-05-27T07:36:25.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.171+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562afaced770 con 0x562afa9bd2f0 2024-05-27T07:36:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.172+0000 7fb497e00640 1 -- 172.21.15.63:0/3588252907 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb48c010460 con 0x7fb4a010dbe0 2024-05-27T07:36:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.172+0000 7fb49f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad88da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.172+0000 7fb49f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad88da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:40510/0 (socket says 172.21.15.63:40510) 2024-05-27T07:36:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.172+0000 7fb49f400640 1 -- 172.21.15.63:0/1338472440 learned_addr learned my addr 172.21.15.63:0/1338472440 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.173+0000 7fb49fe00640 1 --2- 172.21.15.63:0/1338472440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9e61b0 0x562afad88860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.173+0000 7fb49ea00640 1 --2- 172.21.15.63:0/1338472440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.173+0000 7fb49f400640 1 -- 172.21.15.63:0/1338472440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9e61b0 msgr2=0x562afad88860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.173+0000 7fb49f400640 1 --2- 172.21.15.63:0/1338472440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9e61b0 0x562afad88860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.173+0000 7fb49f400640 1 -- 172.21.15.63:0/1338472440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 msgr2=0x562afad892e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.173+0000 7fb49f400640 1 --2- 172.21.15.63:0/1338472440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad892e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.173+0000 7fb49f400640 1 -- 172.21.15.63:0/1338472440 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562afac828b0 con 0x562afa9bd2f0 2024-05-27T07:36:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.173+0000 7fb49ea00640 1 --2- 172.21.15.63:0/1338472440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.174+0000 7fb49f400640 1 --2- 172.21.15.63:0/1338472440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad88da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb490000c00 tx=0x7fb490031dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d76e3c3a2eb2c5e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.174+0000 7fb497e00640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4840b2950 0x7fb4840b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.174+0000 7fb494c00640 1 -- 172.21.15.63:0/1338472440 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490035030 con 0x562afa9bd2f0 2024-05-27T07:36:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.175+0000 7fb497e00640 1 -- 172.21.15.63:0/3588252907 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb48c0e2270 con 0x7fb4a010dbe0 2024-05-27T07:36:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.175+0000 7fb494c00640 1 -- 172.21.15.63:0/1338472440 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb490040070 con 0x562afa9bd2f0 2024-05-27T07:36:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.175+0000 7fb494c00640 1 -- 172.21.15.63:0/1338472440 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490003a10 con 0x562afa9bd2f0 2024-05-27T07:36:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.175+0000 7fb49fe00640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4840b2950 0x7fb4840b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.176+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/1338472440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 msgr2=0x562afad88da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.176+0000 7fb49fe00640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4840b2950 0x7fb4840b4e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x562afad859d0 tx=0x7fb498007520 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.176+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/1338472440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad88da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb490000c00 tx=0x7fb490031dc0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.176+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/1338472440 shutdown_connections 2024-05-27T07:36:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.176+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/1338472440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.176+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/1338472440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad88da0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.176+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/1338472440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9e61b0 0x562afad88860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.176+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/1338472440 >> 172.21.15.63:0/1338472440 conn(0x562afa9da300 msgr2=0x562afad85010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.177+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/1338472440 shutdown_connections 2024-05-27T07:36:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.177+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/1338472440 wait complete. 2024-05-27T07:36:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.177+0000 7fb4ad5e6740 1 -- start start 2024-05-27T07:36:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.178+0000 7fb4ad5e6740 1 Processor -- start 2024-05-27T07:36:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.178+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad92990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.178+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9c54d0 0x562afad92ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.178+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.178+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562afa9cf790 con 0x562afa9e61b0 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.178+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562afac91bc0 con 0x562afa9c54d0 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.178+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562afad95930 con 0x562afa9bd2f0 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.178+0000 7fb4ad5e6740 10 client.0 _ll_register_callbacks cb 0x562afac6f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45986/0 (socket says 172.21.15.63:45986) 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49fe00640 1 -- 172.21.15.63:0/471623620 learned_addr learned my addr 172.21.15.63:0/471623620 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49ea00640 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9c54d0 0x562afad92ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49f400640 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad92990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49fe00640 1 -- 172.21.15.63:0/471623620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 msgr2=0x562afad92990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49fe00640 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad92990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49fe00640 1 -- 172.21.15.63:0/471623620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9c54d0 msgr2=0x562afad92ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49fe00640 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9c54d0 0x562afad92ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49fe00640 1 -- 172.21.15.63:0/471623620 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562afa9fe210 con 0x562afa9e61b0 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.179+0000 7fb49f400640 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad92990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.180+0000 7fb49fe00640 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9ecf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb49801ebf0 tx=0x7fb498004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e3372846ea1bc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.180+0000 7fb489600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.180+0000 7fb489600640 1 -- 172.21.15.63:0/471623620 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49802b060 con 0x562afa9e61b0 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.180+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/471623620 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb490040070 con 0x562afa9e61b0 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.180+0000 7fb489600640 1 -- 172.21.15.63:0/471623620 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb498016030 con 0x562afa9e61b0 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.181+0000 7fb489600640 1 -- 172.21.15.63:0/471623620 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb498032050 con 0x562afa9e61b0 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.181+0000 7fb4ad5e6740 10 client.15927 fetch_fsmap learned FSMap version 28 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.181+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/471623620 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562afaaca620 con 0x562afa9e61b0 2024-05-27T07:36:25.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.182+0000 7fb489600640 1 -- 172.21.15.63:0/471623620 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb490040070 con 0x562afa9e61b0 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.182+0000 7fb489600640 1 -- 172.21.15.63:0/471623620 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb498026cc0 con 0x562afa9e61b0 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.182+0000 7fb4ad5e6740 10 client.15927 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.182+0000 7fb4ad5e6740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.182+0000 7fb4ad5e6740 20 client.15927 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.182+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562afa9a4d80 0x562afada4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.182+0000 7fb4ad5e6740 4 client.15927 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.182+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/471623620 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562afacfbae0 con 0x562afa9a4d80 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.183+0000 7fb49f400640 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562afa9a4d80 0x562afada4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.184+0000 7fb49f400640 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562afa9a4d80 0x562afada4dd0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bf9b9cac044af28 server_cookie=8dd217dd00f359e6 in_seq=0 out_seq=0 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.184+0000 7fb489600640 10 client.15927 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb489600640 1 -- 172.21.15.63:0/471623620 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562afacfbae0 con 0x562afa9a4d80 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb489600640 10 client.15927 handle_command_reply: tid=0 2024-05-27T07:36:25.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb4ad5e6740 1 client.15927 shutdown 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/471623620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 msgr2=0x562afad9ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9ecf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb49801ebf0 tx=0x7fb498004ab0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/471623620 shutdown_connections 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562afa9a4d80 0x562afada4dd0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9ecf0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9c54d0 0x562afad92ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/471623620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9bd2f0 0x562afad92990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/471623620 >> 172.21.15.63:0/471623620 conn(0x562afa9da300 msgr2=0x562afad88690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.186+0000 7fb489600640 0 client.15927 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.187+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/471623620 shutdown_connections 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.187+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/471623620 wait complete. 2024-05-27T07:36:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.187+0000 7fb4ad5e6740 20 client.15927 trim_cache size 0 max 16384 2024-05-27T07:36:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.206+0000 7fb4ad5e6740 1 Processor -- start 2024-05-27T07:36:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.206+0000 7fb4ad5e6740 1 -- start start 2024-05-27T07:36:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.206+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 0x562afad8cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.207+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9c54d0 0x562afad9ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.207+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.207+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4800126c0 con 0x562afa9e61b0 2024-05-27T07:36:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.207+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb48000ff90 con 0x562afa9bd2f0 2024-05-27T07:36:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.207+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb48000dde0 con 0x562afa9c54d0 2024-05-27T07:36:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.207+0000 7fb49fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.207+0000 7fb49fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46002/0 (socket says 172.21.15.63:46002) 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.207+0000 7fb49fe00640 1 -- 172.21.15.63:0/1790308315 learned_addr learned my addr 172.21.15.63:0/1790308315 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.207+0000 7fb49f400640 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 0x562afad8cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.207+0000 7fb49ea00640 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9c54d0 0x562afad9ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb49fe00640 1 -- 172.21.15.63:0/1790308315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9c54d0 msgr2=0x562afad9ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb49fe00640 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9c54d0 0x562afad9ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb49fe00640 1 -- 172.21.15.63:0/1790308315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 msgr2=0x562afad8cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb49fe00640 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 0x562afad8cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb49fe00640 1 -- 172.21.15.63:0/1790308315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562afad89c40 con 0x562afa9e61b0 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb49f400640 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 0x562afad8cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb49ea00640 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9c54d0 0x562afad9ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb49fe00640 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9efa0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb498000f80 tx=0x7fb498010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf5d5c315765997 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb48be00640 1 -- 172.21.15.63:0/1790308315 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498003b70 con 0x562afa9e61b0 2024-05-27T07:36:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb48be00640 1 -- 172.21.15.63:0/1790308315 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb498003d10 con 0x562afa9e61b0 2024-05-27T07:36:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.208+0000 7fb48be00640 1 -- 172.21.15.63:0/1790308315 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980044c0 con 0x562afa9e61b0 2024-05-27T07:36:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.209+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/1790308315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 msgr2=0x562afad9efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.209+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9efa0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb498000f80 tx=0x7fb498010d90 comp rx=0 tx=0).stop 2024-05-27T07:36:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.209+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/1790308315 shutdown_connections 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.209+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9e61b0 0x562afad9efa0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.209+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9c54d0 0x562afad9ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.209+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/1790308315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 0x562afad8cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.209+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/1790308315 >> 172.21.15.63:0/1790308315 conn(0x562afa9da300 msgr2=0x562afad8d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.209+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/1790308315 shutdown_connections 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.210+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/1790308315 wait complete. 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.210+0000 7fb4ad5e6740 1 -- start start 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.210+0000 7fb4ad5e6740 1 Processor -- start 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 0x562afad9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad99830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb4ad5e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9e61b0 0x562afad99e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb48000e450 con 0x562afa9c54d0 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb48000f8c0 con 0x562afa9bd2f0 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb4ad5e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb48000eb20 con 0x562afa9e61b0 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb4ad5e6740 10 client.0 _ll_register_callbacks cb 0x7fb498007b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb49ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad99830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb49ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad99830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46004/0 (socket says 172.21.15.63:46004) 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb49ea00640 1 -- 172.21.15.63:0/3544185777 learned_addr learned my addr 172.21.15.63:0/3544185777 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb49f400640 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 0x562afad9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb49fe00640 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9e61b0 0x562afad99e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb49ea00640 1 -- 172.21.15.63:0/3544185777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9e61b0 msgr2=0x562afad99e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb49ea00640 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9e61b0 0x562afad99e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb49ea00640 1 -- 172.21.15.63:0/3544185777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 msgr2=0x562afad9b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb49ea00640 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 0x562afad9b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.211+0000 7fb49ea00640 1 -- 172.21.15.63:0/3544185777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562afad9a5f0 con 0x562afa9c54d0 2024-05-27T07:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.212+0000 7fb49ea00640 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad99830 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00d740 tx=0x7fb48c0a90d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2809f44eaae5768f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.212+0000 7fb489600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.212+0000 7fb489600640 1 -- 172.21.15.63:0/3544185777 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c020040 con 0x562afa9c54d0 2024-05-27T07:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.212+0000 7fb489600640 1 -- 172.21.15.63:0/3544185777 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fb48c0b1dc0 con 0x562afa9c54d0 2024-05-27T07:36:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.212+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/3544185777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb498003d10 con 0x562afa9c54d0 2024-05-27T07:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.212+0000 7fb489600640 1 -- 172.21.15.63:0/3544185777 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb48c0a8da0 con 0x562afa9c54d0 2024-05-27T07:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.212+0000 7fb4ad5e6740 10 client.15933 fetch_fsmap learned FSMap version 28 2024-05-27T07:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.212+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/3544185777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562afae0c4c0 con 0x562afa9c54d0 2024-05-27T07:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.213+0000 7fb489600640 1 -- 172.21.15.63:0/3544185777 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb498003d10 con 0x562afa9c54d0 2024-05-27T07:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.213+0000 7fb489600640 1 -- 172.21.15.63:0/3544185777 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb48c0a4c00 con 0x562afa9c54d0 2024-05-27T07:36:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.213+0000 7fb4ad5e6740 10 client.15933 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.213+0000 7fb4ad5e6740 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.213+0000 7fb4ad5e6740 20 client.15933 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.213+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562afa97a1c0 0x562afae0ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.213+0000 7fb4ad5e6740 4 client.15933 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.213+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/3544185777 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562afacf52c0 con 0x562afa97a1c0 2024-05-27T07:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.214+0000 7fb49f400640 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562afa97a1c0 0x562afae0ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.215+0000 7fb49f400640 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562afa97a1c0 0x562afae0ec90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab8b0f00bee254cb server_cookie=cbc5a7100ba2dc22 in_seq=0 out_seq=0 2024-05-27T07:36:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.215+0000 7fb489600640 10 client.15933 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:25.229 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-05-27T07:36:25.264 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-05-27T07:36:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb489600640 1 -- 172.21.15.63:0/3544185777 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562afacf52c0 con 0x562afa97a1c0 2024-05-27T07:36:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb489600640 10 client.15933 handle_command_reply: tid=0 2024-05-27T07:36:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 client.15933 shutdown 2024-05-27T07:36:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/3544185777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 msgr2=0x562afad99830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad99830 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00d740 tx=0x7fb48c0a90d0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/3544185777 shutdown_connections 2024-05-27T07:36:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562afa97a1c0 0x562afae0ec90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562afa9e61b0 0x562afad99e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562afa9c54d0 0x562afad99830 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb489600640 0 client.15933 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 --2- 172.21.15.63:0/3544185777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562afa9bd2f0 0x562afad9b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/3544185777 >> 172.21.15.63:0/3544185777 conn(0x562afa9da300 msgr2=0x562afad8dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/3544185777 shutdown_connections 2024-05-27T07:36:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 1 -- 172.21.15.63:0/3544185777 wait complete. 2024-05-27T07:36:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.285+0000 7fb4ad5e6740 20 client.15933 trim_cache size 0 max 16384 2024-05-27T07:36:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:36:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:36:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:36:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.286+0000 7fb4a6600640 1 -- 172.21.15.63:0/3588252907 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4840b2950 msgr2=0x7fb4840b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.286+0000 7fb4a6600640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4840b2950 0x7fb4840b4e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x562afad859d0 tx=0x7fb498007520 comp rx=0 tx=0).stop 2024-05-27T07:36:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.286+0000 7fb4a6600640 1 -- 172.21.15.63:0/3588252907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4a010dbe0 msgr2=0x7fb4a01a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.286+0000 7fb4a6600640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4a010dbe0 0x7fb4a01a2b00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00d770 tx=0x7fb48c00dc30 comp rx=0 tx=0).stop 2024-05-27T07:36:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.286+0000 7fb4a6600640 1 -- 172.21.15.63:0/3588252907 shutdown_connections 2024-05-27T07:36:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.286+0000 7fb4a6600640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4840b2950 0x7fb4840b4e60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.286+0000 7fb4a6600640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a0110540 0x7fb4a019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.286+0000 7fb4a6600640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4a010dbe0 0x7fb4a01a2b00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.286+0000 7fb4a6600640 1 --2- 172.21.15.63:0/3588252907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4a01029a0 0x7fb4a01a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.286+0000 7fb4a6600640 1 -- 172.21.15.63:0/3588252907 >> 172.21.15.63:0/3588252907 conn(0x7fb4a00782d0 msgr2=0x7fb4a01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.287+0000 7fb4a6600640 1 -- 172.21.15.63:0/3588252907 shutdown_connections 2024-05-27T07:36:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:25.287+0000 7fb4a6600640 1 -- 172.21.15.63:0/3588252907 wait complete. 2024-05-27T07:36:25.647 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-05-27T07:36:25.674 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-05-27T07:36:25.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:25 smithi086 ceph-mon[34687]: pgmap v88: 129 pgs: 129 active+clean; 6.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 178 KiB/s wr, 71 op/s 2024-05-27T07:36:25.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:25 smithi149 ceph-mon[35408]: pgmap v88: 129 pgs: 129 active+clean; 6.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 178 KiB/s wr, 71 op/s 2024-05-27T07:36:25.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:25 smithi063 ceph-mon[31613]: pgmap v88: 129 pgs: 129 active+clean; 6.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 178 KiB/s wr, 71 op/s 2024-05-27T07:36:26.205 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-05-27T07:36:26.230 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-05-27T07:36:26.859 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-05-27T07:36:26.880 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-05-27T07:36:27.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.030+0000 7f85e1000700 1 -- 172.21.15.63:0/177125416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85dc10a980 msgr2=0x7f85dc10ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:27.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.030+0000 7f85e1000700 1 --2- 172.21.15.63:0/177125416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85dc10a980 0x7f85dc10ad90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009990 tx=0x7f85c8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.030+0000 7f85e1000700 1 -- 172.21.15.63:0/177125416 shutdown_connections 2024-05-27T07:36:27.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.030+0000 7f85e1000700 1 --2- 172.21.15.63:0/177125416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc10b2d0 0x7f85dc117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.030+0000 7f85e1000700 1 --2- 172.21.15.63:0/177125416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85dc110960 0x7f85dc110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.030+0000 7f85e1000700 1 --2- 172.21.15.63:0/177125416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85dc10a980 0x7f85dc10ad90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.030+0000 7f85e1000700 1 -- 172.21.15.63:0/177125416 >> 172.21.15.63:0/177125416 conn(0x7f85dc077c70 msgr2=0x7f85dc0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:27.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.030+0000 7f85e1000700 1 -- 172.21.15.63:0/177125416 shutdown_connections 2024-05-27T07:36:27.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.030+0000 7f85e1000700 1 -- 172.21.15.63:0/177125416 wait complete. 2024-05-27T07:36:27.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.032+0000 7f85e1000700 1 Processor -- start 2024-05-27T07:36:27.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.032+0000 7f85e1000700 1 -- start start 2024-05-27T07:36:27.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.032+0000 7f85e1000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85dc10a980 0x7f85dc1a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.032+0000 7f85e1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc10b2d0 0x7f85dc1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.032+0000 7f85e1000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85dc110960 0x7f85dc1a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.032+0000 7f85e1000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc11a910 con 0x7f85dc110960 2024-05-27T07:36:27.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.032+0000 7f85e1000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc11a790 con 0x7f85dc10b2d0 2024-05-27T07:36:27.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.032+0000 7f85e1000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc11aa90 con 0x7f85dc10a980 2024-05-27T07:36:27.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.033+0000 7f85da800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85dc110960 0x7f85dc1a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.033+0000 7f85da800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85dc110960 0x7f85dc1a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46024/0 (socket says 172.21.15.63:46024) 2024-05-27T07:36:27.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.033+0000 7f85da800700 1 -- 172.21.15.63:0/2727312775 learned_addr learned my addr 172.21.15.63:0/2727312775 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:27.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.033+0000 7f85d9e00700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85dc10a980 0x7f85dc1a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.033+0000 7f85d9400700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc10b2d0 0x7f85dc1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.033+0000 7f85da800700 1 -- 172.21.15.63:0/2727312775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85dc10a980 msgr2=0x7f85dc1a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:27.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.033+0000 7f85da800700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85dc10a980 0x7f85dc1a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.033+0000 7f85da800700 1 -- 172.21.15.63:0/2727312775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc10b2d0 msgr2=0x7f85dc1a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:27.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.033+0000 7f85da800700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc10b2d0 0x7f85dc1a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.033+0000 7f85da800700 1 -- 172.21.15.63:0/2727312775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc1a6100 con 0x7f85dc110960 2024-05-27T07:36:27.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.034+0000 7f85da800700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85dc110960 0x7f85dc1a5b10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f85d4007d70 tx=0x7f85d400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:27.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.034+0000 7f85d2a00700 1 -- 172.21.15.63:0/2727312775 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4016020 con 0x7f85dc110960 2024-05-27T07:36:27.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.034+0000 7f85d2a00700 1 -- 172.21.15.63:0/2727312775 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d400ad40 con 0x7f85dc110960 2024-05-27T07:36:27.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.034+0000 7f85d2a00700 1 -- 172.21.15.63:0/2727312775 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d40115e0 con 0x7f85dc110960 2024-05-27T07:36:27.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.034+0000 7f85e1000700 1 -- 172.21.15.63:0/2727312775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85dc1a63f0 con 0x7f85dc110960 2024-05-27T07:36:27.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.034+0000 7f85e1000700 1 -- 172.21.15.63:0/2727312775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85dc1a68e0 con 0x7f85dc110960 2024-05-27T07:36:27.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.039+0000 7f85d2a00700 1 -- 172.21.15.63:0/2727312775 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f85d4019020 con 0x7f85dc110960 2024-05-27T07:36:27.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.040+0000 7f85efc27b80 1 Processor -- start 2024-05-27T07:36:27.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.040+0000 7f85efc27b80 1 -- start start 2024-05-27T07:36:27.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.040+0000 7f85d2a00700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85c00b29f0 0x7f85c00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.040+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acad869f0 0x555acae19330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.040+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae19870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.040+0000 7f85d2a00700 1 -- 172.21.15.63:0/2727312775 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f85d40e2970 con 0x7f85dc110960 2024-05-27T07:36:27.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.040+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acae19e70 0x555acae26110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.041+0000 7f85efc27b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555acad47e40 con 0x555acac9ae40 2024-05-27T07:36:27.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.041+0000 7f85efc27b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555acad379e0 con 0x555acad869f0 2024-05-27T07:36:27.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.041+0000 7f85efc27b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555acad3ba20 con 0x555acae19e70 2024-05-27T07:36:27.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.041+0000 7f85d9400700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85c00b29f0 0x7f85c00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.041+0000 7f85d9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae19870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.041+0000 7f85d9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acad869f0 0x555acae19330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.041+0000 7f85d9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae19870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46026/0 (socket says 172.21.15.63:46026) 2024-05-27T07:36:27.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.041+0000 7f85d9400700 1 -- 172.21.15.63:0/4195636797 learned_addr learned my addr 172.21.15.63:0/4195636797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:27.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.041+0000 7f85da800700 1 --2- 172.21.15.63:0/4195636797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acae19e70 0x555acae26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.042+0000 7f85d9400700 1 -- 172.21.15.63:0/4195636797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acae19e70 msgr2=0x555acae26110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.042+0000 7f85d9400700 1 --2- 172.21.15.63:0/4195636797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acae19e70 0x555acae26110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.042+0000 7f85d9400700 1 -- 172.21.15.63:0/4195636797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acad869f0 msgr2=0x555acae19330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.042+0000 7f85d9400700 1 --2- 172.21.15.63:0/4195636797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acad869f0 0x555acae19330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.042+0000 7f85d9400700 1 -- 172.21.15.63:0/4195636797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555acad00420 con 0x555acac9ae40 2024-05-27T07:36:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.042+0000 7f85d9400700 1 --2- 172.21.15.63:0/4195636797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae19870 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f85dc06d110 tx=0x7f85c8005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629c1be663686fe0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.042+0000 7f85d9400700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85c00b29f0 0x7f85c00b4f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009990 tx=0x7f85c801c000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.042+0000 7f85c7400700 1 -- 172.21.15.63:0/4195636797 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c802b070 con 0x555acac9ae40 2024-05-27T07:36:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.043+0000 7f85c7400700 1 -- 172.21.15.63:0/4195636797 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85c800be60 con 0x555acac9ae40 2024-05-27T07:36:27.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.043+0000 7f85c7400700 1 -- 172.21.15.63:0/4195636797 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8027bb0 con 0x555acac9ae40 2024-05-27T07:36:27.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.044+0000 7f85efc27b80 1 -- 172.21.15.63:0/4195636797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 msgr2=0x555acae19870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:27.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.044+0000 7f85efc27b80 1 --2- 172.21.15.63:0/4195636797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae19870 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f85dc06d110 tx=0x7f85c8005620 comp rx=0 tx=0).stop 2024-05-27T07:36:27.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.044+0000 7f85efc27b80 1 -- 172.21.15.63:0/4195636797 shutdown_connections 2024-05-27T07:36:27.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.044+0000 7f85efc27b80 1 --2- 172.21.15.63:0/4195636797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acae19e70 0x555acae26110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.044+0000 7f85efc27b80 1 --2- 172.21.15.63:0/4195636797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acad869f0 0x555acae19330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.044+0000 7f85efc27b80 1 --2- 172.21.15.63:0/4195636797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae19870 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.044+0000 7f85efc27b80 1 -- 172.21.15.63:0/4195636797 >> 172.21.15.63:0/4195636797 conn(0x555acad34390 msgr2=0x555acae154c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:27.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.044+0000 7f85efc27b80 1 -- 172.21.15.63:0/4195636797 shutdown_connections 2024-05-27T07:36:27.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.044+0000 7f85efc27b80 1 -- 172.21.15.63:0/4195636797 wait complete. 2024-05-27T07:36:27.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.045+0000 7f85efc27b80 1 -- start start 2024-05-27T07:36:27.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.045+0000 7f85efc27b80 1 Processor -- start 2024-05-27T07:36:27.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 0x555acae236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acad869f0 0x555acae23c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acae19e70 0x555acae24160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85efc27b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555acac3dfa0 con 0x555acae19e70 2024-05-27T07:36:27.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85efc27b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555acacd8760 con 0x555acad869f0 2024-05-27T07:36:27.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85efc27b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555acae26b20 con 0x555acac9ae40 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85efc27b80 10 client.0 _ll_register_callbacks cb 0x555acace1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85da800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acae19e70 0x555acae24160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85d9e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 0x555acae236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85da800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acae19e70 0x555acae24160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46032/0 (socket says 172.21.15.63:46032) 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85da800700 1 -- 172.21.15.63:0/3315083944 learned_addr learned my addr 172.21.15.63:0/3315083944 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.046+0000 7f85d9400700 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acad869f0 0x555acae23c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.047+0000 7f85da800700 1 -- 172.21.15.63:0/3315083944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 msgr2=0x555acae236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.047+0000 7f85da800700 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 0x555acae236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.047+0000 7f85da800700 1 -- 172.21.15.63:0/3315083944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acad869f0 msgr2=0x555acae23c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.047+0000 7f85da800700 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acad869f0 0x555acae23c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.047+0000 7f85da800700 1 -- 172.21.15.63:0/3315083944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555acac878d0 con 0x555acae19e70 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.047+0000 7f85da800700 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acae19e70 0x555acae24160 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f85d40a9690 tx=0x7f85d40e0220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16670a5ab415e7f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.047+0000 7f85bbe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.048+0000 7f85bbe00700 1 -- 172.21.15.63:0/3315083944 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d40cc2d0 con 0x555acae19e70 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.048+0000 7f85bbe00700 1 -- 172.21.15.63:0/3315083944 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f85d40a9bf0 con 0x555acae19e70 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.048+0000 7f85efc27b80 1 -- 172.21.15.63:0/3315083944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c800be60 con 0x555acae19e70 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.048+0000 7f85bbe00700 1 -- 172.21.15.63:0/3315083944 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85d40c4080 con 0x555acae19e70 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.048+0000 7f85efc27b80 10 client.15945 fetch_fsmap learned FSMap version 28 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.048+0000 7f85efc27b80 1 -- 172.21.15.63:0/3315083944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555acad08060 con 0x555acae19e70 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.048+0000 7f85bbe00700 1 -- 172.21.15.63:0/3315083944 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85c800be60 con 0x555acae19e70 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.048+0000 7f85bbe00700 1 -- 172.21.15.63:0/3315083944 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f85d40b2360 con 0x555acae19e70 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.049+0000 7f85efc27b80 10 client.15945 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.049+0000 7f85efc27b80 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.049+0000 7f85efc27b80 20 client.15945 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:27.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.049+0000 7f85efc27b80 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555acae33b40 0x555acae35f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.049+0000 7f85efc27b80 4 client.15945 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:36:27.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.049+0000 7f85efc27b80 1 -- 172.21.15.63:0/3315083944 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555acad49d90 con 0x555acae33b40 2024-05-27T07:36:27.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.049+0000 7f85d9e00700 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555acae33b40 0x555acae35f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.050+0000 7f85d9e00700 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555acae33b40 0x555acae35f90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c32661140cc8a472 server_cookie=9c71bca7ec6cec94 in_seq=0 out_seq=0 2024-05-27T07:36:27.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.050+0000 7f85bbe00700 10 client.15945 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:27.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.053+0000 7f85bbe00700 1 -- 172.21.15.63:0/3315083944 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555acad49d90 con 0x555acae33b40 2024-05-27T07:36:27.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.053+0000 7f85bbe00700 10 client.15945 handle_command_reply: tid=0 2024-05-27T07:36:27.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 client.15945 shutdown 2024-05-27T07:36:27.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 -- 172.21.15.63:0/3315083944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acae19e70 msgr2=0x555acae24160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:27.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acae19e70 0x555acae24160 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f85d40a9690 tx=0x7f85d40e0220 comp rx=0 tx=0).stop 2024-05-27T07:36:27.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 -- 172.21.15.63:0/3315083944 shutdown_connections 2024-05-27T07:36:27.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555acae33b40 0x555acae35f90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acae19e70 0x555acae24160 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 0x555acae236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 --2- 172.21.15.63:0/3315083944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acad869f0 0x555acae23c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85bbe00700 0 client.15945 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:27.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 -- 172.21.15.63:0/3315083944 >> 172.21.15.63:0/3315083944 conn(0x555acad34390 msgr2=0x555acae19350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:27.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 -- 172.21.15.63:0/3315083944 shutdown_connections 2024-05-27T07:36:27.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 1 -- 172.21.15.63:0/3315083944 wait complete. 2024-05-27T07:36:27.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.054+0000 7f85efc27b80 20 client.15945 trim_cache size 0 max 16384 2024-05-27T07:36:27.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.078+0000 7f85efc27b80 1 Processor -- start 2024-05-27T07:36:27.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.078+0000 7f85efc27b80 1 -- start start 2024-05-27T07:36:27.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.079+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae1ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.079+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acad869f0 0x555acae257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.079+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acae19e70 0x555acae1b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.079+0000 7f85efc27b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1b9050 con 0x555acac9ae40 2024-05-27T07:36:27.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.079+0000 7f85efc27b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1b68e0 con 0x555acae19e70 2024-05-27T07:36:27.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.079+0000 7f85efc27b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1b4730 con 0x555acad869f0 2024-05-27T07:36:27.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.079+0000 7f85d9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae1ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.079+0000 7f85d9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae1ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46044/0 (socket says 172.21.15.63:46044) 2024-05-27T07:36:27.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.079+0000 7f85d9e00700 1 -- 172.21.15.63:0/3312359584 learned_addr learned my addr 172.21.15.63:0/3312359584 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:27.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.080+0000 7f85da800700 1 --2- 172.21.15.63:0/3312359584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acae19e70 0x555acae1b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.080+0000 7f85d9400700 1 --2- 172.21.15.63:0/3312359584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acad869f0 0x555acae257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.080+0000 7f85d9e00700 1 -- 172.21.15.63:0/3312359584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acad869f0 msgr2=0x555acae257c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:27.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.080+0000 7f85d9e00700 1 --2- 172.21.15.63:0/3312359584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acad869f0 0x555acae257c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.080+0000 7f85d9e00700 1 -- 172.21.15.63:0/3312359584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acae19e70 msgr2=0x555acae1b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:27.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.080+0000 7f85d9e00700 1 --2- 172.21.15.63:0/3312359584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acae19e70 0x555acae1b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.080+0000 7f85d9e00700 1 -- 172.21.15.63:0/3312359584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555acae244d0 con 0x555acac9ae40 2024-05-27T07:36:27.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.080+0000 7f85d9e00700 1 --2- 172.21.15.63:0/3312359584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae1ee90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x555acae36740 tx=0x7f85cc012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f94c853439e264 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:27.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.081+0000 7f85c6a00700 1 -- 172.21.15.63:0/3312359584 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc012d00 con 0x555acac9ae40 2024-05-27T07:36:27.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.081+0000 7f85c6a00700 1 -- 172.21.15.63:0/3312359584 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85cc0044e0 con 0x555acac9ae40 2024-05-27T07:36:27.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.081+0000 7f85c6a00700 1 -- 172.21.15.63:0/3312359584 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc01c430 con 0x555acac9ae40 2024-05-27T07:36:27.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.082+0000 7f85efc27b80 1 -- 172.21.15.63:0/3312359584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 msgr2=0x555acae1ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:27.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.082+0000 7f85efc27b80 1 --2- 172.21.15.63:0/3312359584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae1ee90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x555acae36740 tx=0x7f85cc012730 comp rx=0 tx=0).stop 2024-05-27T07:36:27.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.082+0000 7f85efc27b80 1 -- 172.21.15.63:0/3312359584 shutdown_connections 2024-05-27T07:36:27.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.082+0000 7f85efc27b80 1 --2- 172.21.15.63:0/3312359584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acae19e70 0x555acae1b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.082+0000 7f85efc27b80 1 --2- 172.21.15.63:0/3312359584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acac9ae40 0x555acae1ee90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.082+0000 7f85efc27b80 1 --2- 172.21.15.63:0/3312359584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acad869f0 0x555acae257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.082+0000 7f85efc27b80 1 -- 172.21.15.63:0/3312359584 >> 172.21.15.63:0/3312359584 conn(0x555acad34390 msgr2=0x555acae1aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:27.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.082+0000 7f85efc27b80 1 -- 172.21.15.63:0/3312359584 shutdown_connections 2024-05-27T07:36:27.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.082+0000 7f85efc27b80 1 -- 172.21.15.63:0/3312359584 wait complete. 2024-05-27T07:36:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.083+0000 7f85efc27b80 1 -- start start 2024-05-27T07:36:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.083+0000 7f85efc27b80 1 Processor -- start 2024-05-27T07:36:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.083+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 0x555acae1e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.083+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acad869f0 0x555acae1e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.083+0000 7f85efc27b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acae19e70 0x555acae1ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.083+0000 7f85efc27b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1b4da0 con 0x555acad869f0 2024-05-27T07:36:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.083+0000 7f85efc27b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1b6210 con 0x555acae19e70 2024-05-27T07:36:27.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.083+0000 7f85efc27b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1b5470 con 0x555acac9ae40 2024-05-27T07:36:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85efc27b80 10 client.0 _ll_register_callbacks cb 0x555acace1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85d9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acad869f0 0x555acae1e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85d9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acad869f0 0x555acae1e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46050/0 (socket says 172.21.15.63:46050) 2024-05-27T07:36:27.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85d9400700 1 -- 172.21.15.63:0/773193746 learned_addr learned my addr 172.21.15.63:0/773193746 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85d9e00700 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 0x555acae1e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85da800700 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acae19e70 0x555acae1ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85d9400700 1 -- 172.21.15.63:0/773193746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 msgr2=0x555acae1e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85d9400700 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 0x555acae1e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85d9400700 1 -- 172.21.15.63:0/773193746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acae19e70 msgr2=0x555acae1ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85d9400700 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acae19e70 0x555acae1ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85d9400700 1 -- 172.21.15.63:0/773193746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555acae2a3d0 con 0x555acad869f0 2024-05-27T07:36:27.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.084+0000 7f85d9e00700 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 0x555acae1e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.085+0000 7f85d9400700 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acad869f0 0x555acae1e7f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f85c8005850 tx=0x7f85c80096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852f46b8b0dd2efd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.085+0000 7f85bbe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.085+0000 7f85bbe00700 1 -- 172.21.15.63:0/773193746 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8039060 con 0x555acad869f0 2024-05-27T07:36:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.085+0000 7f85bbe00700 1 -- 172.21.15.63:0/773193746 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f85c8027890 con 0x555acad869f0 2024-05-27T07:36:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.085+0000 7f85efc27b80 1 -- 172.21.15.63:0/773193746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85cc0044e0 con 0x555acad869f0 2024-05-27T07:36:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.085+0000 7f85bbe00700 1 -- 172.21.15.63:0/773193746 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85c802b040 con 0x555acad869f0 2024-05-27T07:36:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.086+0000 7f85efc27b80 10 client.15951 fetch_fsmap learned FSMap version 28 2024-05-27T07:36:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.086+0000 7f85efc27b80 1 -- 172.21.15.63:0/773193746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555acae9e900 con 0x555acad869f0 2024-05-27T07:36:27.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.086+0000 7f85bbe00700 1 -- 172.21.15.63:0/773193746 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85cc0044e0 con 0x555acad869f0 2024-05-27T07:36:27.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.086+0000 7f85bbe00700 1 -- 172.21.15.63:0/773193746 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f85c800f7b0 con 0x555acad869f0 2024-05-27T07:36:27.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.086+0000 7f85efc27b80 10 client.15951 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T07:36:27.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.086+0000 7f85efc27b80 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:27.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.086+0000 7f85efc27b80 20 client.15951 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:27.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.086+0000 7f85efc27b80 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555acae9ec70 0x555acaea1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:27.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.086+0000 7f85efc27b80 4 client.15951 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:36:27.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.086+0000 7f85efc27b80 1 -- 172.21.15.63:0/773193746 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555acad43570 con 0x555acae9ec70 2024-05-27T07:36:27.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.087+0000 7f85d9e00700 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555acae9ec70 0x555acaea1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:27.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.088+0000 7f85d9e00700 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555acae9ec70 0x555acaea1090 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62b2d35189d5778b server_cookie=6bfa72bb5f1cd3b4 in_seq=0 out_seq=0 2024-05-27T07:36:27.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.088+0000 7f85bbe00700 10 client.15951 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:27.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85bbe00700 1 -- 172.21.15.63:0/773193746 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555acad43570 con 0x555acae9ec70 2024-05-27T07:36:27.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85bbe00700 10 client.15951 handle_command_reply: tid=0 2024-05-27T07:36:27.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85efc27b80 1 client.15951 shutdown 2024-05-27T07:36:27.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85efc27b80 1 -- 172.21.15.63:0/773193746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acad869f0 msgr2=0x555acae1e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:27.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85efc27b80 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acad869f0 0x555acae1e7f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f85c8005850 tx=0x7f85c80096e0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85efc27b80 1 -- 172.21.15.63:0/773193746 shutdown_connections 2024-05-27T07:36:27.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85efc27b80 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555acae9ec70 0x555acaea1090 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85efc27b80 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555acae19e70 0x555acae1ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85efc27b80 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555acac9ae40 0x555acae1e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85efc27b80 1 --2- 172.21.15.63:0/773193746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555acad869f0 0x555acae1e7f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85efc27b80 1 -- 172.21.15.63:0/773193746 >> 172.21.15.63:0/773193746 conn(0x555acad34390 msgr2=0x555acae23c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:27.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85bbe00700 0 client.15951 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:27.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.089+0000 7f85efc27b80 1 -- 172.21.15.63:0/773193746 shutdown_connections 2024-05-27T07:36:27.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.090+0000 7f85efc27b80 1 -- 172.21.15.63:0/773193746 wait complete. 2024-05-27T07:36:27.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.090+0000 7f85efc27b80 20 client.15951 trim_cache size 0 max 16384 2024-05-27T07:36:27.087 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:36:27.087 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:36:27.088 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:36:27.088 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:36:27.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.090+0000 7f85e1000700 1 -- 172.21.15.63:0/2727312775 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85c00b29f0 msgr2=0x7f85c00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:27.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.090+0000 7f85e1000700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85c00b29f0 0x7f85c00b4f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009990 tx=0x7f85c801c000 comp rx=0 tx=0).stop 2024-05-27T07:36:27.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.090+0000 7f85e1000700 1 -- 172.21.15.63:0/2727312775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85dc110960 msgr2=0x7f85dc1a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:27.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.090+0000 7f85e1000700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85dc110960 0x7f85dc1a5b10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f85d4007d70 tx=0x7f85d400a340 comp rx=0 tx=0).stop 2024-05-27T07:36:27.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.091+0000 7f85e1000700 1 -- 172.21.15.63:0/2727312775 shutdown_connections 2024-05-27T07:36:27.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.091+0000 7f85e1000700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85c00b29f0 0x7f85c00b4f00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.091+0000 7f85e1000700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85dc110960 0x7f85dc1a5b10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.091+0000 7f85e1000700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85dc10a980 0x7f85dc1a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.091+0000 7f85e1000700 1 --2- 172.21.15.63:0/2727312775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc10b2d0 0x7f85dc1a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:27.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.091+0000 7f85e1000700 1 -- 172.21.15.63:0/2727312775 >> 172.21.15.63:0/2727312775 conn(0x7f85dc077c70 msgr2=0x7f85dc07b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:27.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.091+0000 7f85e1000700 1 -- 172.21.15.63:0/2727312775 shutdown_connections 2024-05-27T07:36:27.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:27.091+0000 7f85e1000700 1 -- 172.21.15.63:0/2727312775 wait complete. 2024-05-27T07:36:27.130 INFO:tasks.workunit.client.0.smithi063.stderr:In file included from parser.h:22, 2024-05-27T07:36:27.131 INFO:tasks.workunit.client.0.smithi063.stderr: from parser.c:25: 2024-05-27T07:36:27.131 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c: In function ‘init_filesys’: 2024-05-27T07:36:27.134 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-27T07:36:27.134 INFO:tasks.workunit.client.0.smithi063.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-27T07:36:27.134 INFO:tasks.workunit.client.0.smithi063.stderr: | ^ 2024-05-27T07:36:27.134 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-27T07:36:27.134 INFO:tasks.workunit.client.0.smithi063.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-27T07:36:27.134 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~ 2024-05-27T07:36:27.134 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-27T07:36:27.134 INFO:tasks.workunit.client.0.smithi063.stderr: 36 | container_of(ptr, type, member) 2024-05-27T07:36:27.134 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~ 2024-05-27T07:36:27.134 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~ 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr: | ^ 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~ 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-27T07:36:27.135 INFO:tasks.workunit.client.0.smithi063.stderr: 36 | container_of(ptr, type, member) 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~ 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~ 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr: | ^ 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-27T07:36:27.136 INFO:tasks.workunit.client.0.smithi063.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~ 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr: 36 | container_of(ptr, type, member) 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~ 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~ 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-27T07:36:27.137 INFO:tasks.workunit.client.0.smithi063.stderr: | ^ 2024-05-27T07:36:27.138 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-27T07:36:27.138 INFO:tasks.workunit.client.0.smithi063.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-27T07:36:27.138 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~ 2024-05-27T07:36:27.138 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-27T07:36:27.138 INFO:tasks.workunit.client.0.smithi063.stderr: 36 | container_of(ptr, type, member) 2024-05-27T07:36:27.138 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~ 2024-05-27T07:36:27.138 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-27T07:36:27.138 INFO:tasks.workunit.client.0.smithi063.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-27T07:36:27.138 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~ 2024-05-27T07:36:27.138 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c: In function ‘init_tg_stats’: 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr: | ^ 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~ 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr: 36 | container_of(ptr, type, member) 2024-05-27T07:36:27.139 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~ 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~ 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr: 971 | list_for_each_entry(tmp_list, 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr: | ^ 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-27T07:36:27.140 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~ 2024-05-27T07:36:27.141 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-27T07:36:27.141 INFO:tasks.workunit.client.0.smithi063.stderr: 36 | container_of(ptr, type, member) 2024-05-27T07:36:27.141 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~ 2024-05-27T07:36:27.141 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-27T07:36:27.141 INFO:tasks.workunit.client.0.smithi063.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-27T07:36:27.141 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~ 2024-05-27T07:36:27.141 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-27T07:36:27.141 INFO:tasks.workunit.client.0.smithi063.stderr: 971 | list_for_each_entry(tmp_list, 2024-05-27T07:36:27.141 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-27T07:36:27.141 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr: | ^ 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~ 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr: 36 | container_of(ptr, type, member) 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~ 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~ 2024-05-27T07:36:27.142 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-27T07:36:27.143 INFO:tasks.workunit.client.0.smithi063.stderr: 980 | list_for_each_entry(tmp_list, 2024-05-27T07:36:27.143 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-27T07:36:27.143 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-27T07:36:27.143 INFO:tasks.workunit.client.0.smithi063.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-27T07:36:27.143 INFO:tasks.workunit.client.0.smithi063.stderr: | ^ 2024-05-27T07:36:27.143 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-05-27T07:36:27.143 INFO:tasks.workunit.client.0.smithi063.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-27T07:36:27.143 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~ 2024-05-27T07:36:27.143 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-05-27T07:36:27.144 INFO:tasks.workunit.client.0.smithi063.stderr: 36 | container_of(ptr, type, member) 2024-05-27T07:36:27.144 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~ 2024-05-27T07:36:27.144 INFO:tasks.workunit.client.0.smithi063.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-05-27T07:36:27.144 INFO:tasks.workunit.client.0.smithi063.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-27T07:36:27.144 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~ 2024-05-27T07:36:27.144 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-05-27T07:36:27.144 INFO:tasks.workunit.client.0.smithi063.stderr: 980 | list_for_each_entry(tmp_list, 2024-05-27T07:36:27.144 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-27T07:36:27.199 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c: In function ‘get_optstr’: 2024-05-27T07:36:27.199 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-05-27T07:36:27.199 INFO:tasks.workunit.client.0.smithi063.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-05-27T07:36:27.199 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-05-27T07:36:27.199 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:200:14: note: source object declared here 2024-05-27T07:36:27.199 INFO:tasks.workunit.client.0.smithi063.stderr: 200 | char temp[BUFSIZE]; 2024-05-27T07:36:27.199 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~ 2024-05-27T07:36:27.217 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c: In function ‘size64_convert’: 2024-05-27T07:36:27.217 INFO:tasks.workunit.client.0.smithi063.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-05-27T07:36:27.217 INFO:tasks.workunit.client.0.smithi063.stderr: 119 | memcpy(unit, "\0", 3); 2024-05-27T07:36:27.217 INFO:tasks.workunit.client.0.smithi063.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-05-27T07:36:27.683 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-05-27T07:36:27.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:27 smithi086 ceph-mon[34687]: pgmap v89: 129 pgs: 129 active+clean; 3.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 247 KiB/s wr, 93 op/s 2024-05-27T07:36:27.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:27 smithi149 ceph-mon[35408]: pgmap v89: 129 pgs: 129 active+clean; 3.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 247 KiB/s wr, 93 op/s 2024-05-27T07:36:27.709 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-05-27T07:36:27.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:27 smithi063 ceph-mon[31613]: pgmap v89: 129 pgs: 129 active+clean; 3.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 247 KiB/s wr, 93 op/s 2024-05-27T07:36:28.236 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-05-27T07:36:28.261 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-05-27T07:36:28.744 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-05-27T07:36:28.763 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:36:28.763 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:36:28.772 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-05-27T07:36:28.954 INFO:tasks.workunit.client.0.smithi063.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-05-27T07:36:28.977 INFO:tasks.workunit.client.0.smithi063.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-05-27T07:36:29.274 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:36:29.314 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-27T07:36:29.317 INFO:tasks.workunit.client.0.smithi063.stderr:+ cd .. 2024-05-27T07:36:29.321 INFO:tasks.workunit.client.0.smithi063.stderr:+ mkdir tmp 2024-05-27T07:36:29.329 INFO:tasks.workunit.client.0.smithi063.stderr:+ cd tmp 2024-05-27T07:36:29.333 INFO:tasks.workunit.client.0.smithi063.stderr:+ for f in $mydir/*.ffsb 2024-05-27T07:36:29.333 INFO:tasks.workunit.client.0.smithi063.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-05-27T07:36:29.336 INFO:tasks.workunit.client.0.smithi063.stdout:FFSB version 6.0-RC2 started 2024-05-27T07:36:29.336 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T07:36:29.342 INFO:tasks.workunit.client.0.smithi063.stdout:benchmark time = 300 2024-05-27T07:36:29.342 INFO:tasks.workunit.client.0.smithi063.stdout:ThreadGroup 0 2024-05-27T07:36:29.342 INFO:tasks.workunit.client.0.smithi063.stdout:================ 2024-05-27T07:36:29.342 INFO:tasks.workunit.client.0.smithi063.stdout: num_threads = 32 2024-05-27T07:36:29.342 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: read_random = off 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: read_size = 0 (0B) 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: read_blocksize = 0 (0B) 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: read_skip = off 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: read_skipsize = 0 (0B) 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: write_random = on 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: write_size = 5242880 (5MB) 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: fsync_file = 0 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: write_blocksize = 4096 (4KB) 2024-05-27T07:36:29.343 INFO:tasks.workunit.client.0.smithi063.stdout: wait time = 0 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: op weights 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: read = 0 (0.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: readall = 0 (0.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: write = 1 (100.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: create = 0 (0.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: append = 0 (0.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: delete = 0 (0.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: metaop = 0 (0.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: createdir = 0 (0.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: stat = 0 (0.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: writeall = 0 (0.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: writeall_fsync = 0 (0.00%) 2024-05-27T07:36:29.344 INFO:tasks.workunit.client.0.smithi063.stdout: open_close = 0 (0.00%) 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout: write_fsync = 0 (0.00%) 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout: create_fsync = 0 (0.00%) 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout: append_fsync = 0 (0.00%) 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout:FileSystem . 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout:========== 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout: num_dirs = 0 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout: starting files = 128 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout: min file size = 104857600 (100MB) 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout: max file size = 104857600 (100MB) 2024-05-27T07:36:29.345 INFO:tasks.workunit.client.0.smithi063.stdout: directio = off 2024-05-27T07:36:29.346 INFO:tasks.workunit.client.0.smithi063.stdout: alignedio = on 2024-05-27T07:36:29.346 INFO:tasks.workunit.client.0.smithi063.stdout: bufferedio = off 2024-05-27T07:36:29.346 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T07:36:29.346 INFO:tasks.workunit.client.0.smithi063.stdout: aging is off 2024-05-27T07:36:29.346 INFO:tasks.workunit.client.0.smithi063.stdout: current utilization = 0.04% 2024-05-27T07:36:29.346 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T07:36:29.346 INFO:tasks.workunit.client.0.smithi063.stdout:checking existing fs: . 2024-05-27T07:36:29.347 INFO:tasks.workunit.client.0.smithi063.stdout:recreating new fileset 2024-05-27T07:36:29.347 INFO:tasks.workunit.client.0.smithi063.stderr:opendir: No such file or directory 2024-05-27T07:36:29.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:29 smithi086 ceph-mon[34687]: pgmap v90: 129 pgs: 129 active+clean; 3.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 144 KiB/s wr, 40 op/s 2024-05-27T07:36:29.704 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:29 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 1. 2024-05-27T07:36:29.704 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:29 smithi086 systemd[1]: Stopped Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:36:29.704 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:29 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Consumed 4.405s CPU time. 2024-05-27T07:36:29.704 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:29 smithi086 systemd[1]: Starting Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:36:29.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:29 smithi149 ceph-mon[35408]: pgmap v90: 129 pgs: 129 active+clean; 3.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 144 KiB/s wr, 40 op/s 2024-05-27T07:36:29.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:29 smithi063 ceph-mon[31613]: pgmap v90: 129 pgs: 129 active+clean; 3.5 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 144 KiB/s wr, 40 op/s 2024-05-27T07:36:30.186 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:29 smithi086 podman[69407]: 2024-05-27 07:36:29.771553197 +0000 UTC m=+0.033418556 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:36:30.186 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:29 smithi086 podman[69407]: 2024-05-27 07:36:29.885932708 +0000 UTC m=+0.147798104 container create 864f5397b10eefc55b6aa14400c32e308f0cf81101b58e384f8abf68dd06b034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:36:30.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:30 smithi086 ceph-mon[34687]: MDS daemon mds.k is removed because it is dead or otherwise unavailable. 2024-05-27T07:36:30.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:30 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T07:36:30.455 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:30 smithi086 podman[69407]: 2024-05-27 07:36:30.190617682 +0000 UTC m=+0.452483088 container init 864f5397b10eefc55b6aa14400c32e308f0cf81101b58e384f8abf68dd06b034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:36:30.455 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:30 smithi086 podman[69407]: 2024-05-27 07:36:30.201956014 +0000 UTC m=+0.463821380 container start 864f5397b10eefc55b6aa14400c32e308f0cf81101b58e384f8abf68dd06b034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T07:36:30.455 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:30 smithi086 bash[69407]: 864f5397b10eefc55b6aa14400c32e308f0cf81101b58e384f8abf68dd06b034 2024-05-27T07:36:30.455 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:30 smithi086 ceph-mds[69426]: -- 172.21.15.86:0/353540656 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e3b81da40 con 0x563e3b865a80 2024-05-27T07:36:30.455 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:30 smithi086 systemd[1]: Started Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:36:30.455 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:36:30 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[69422]: starting mds.k at 2024-05-27T07:36:30.681 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:30 smithi149 ceph-mon[35408]: MDS daemon mds.k is removed because it is dead or otherwise unavailable. 2024-05-27T07:36:30.681 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:30 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T07:36:30.682 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f017cc00700 time 2024-05-27T07:36:30.216657+0000 2024-05-27T07:36:30.682 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f019261dba6] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f019261ddd1] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a36a64823d] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55a36a633971] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 5: (SnapRealm::check_cache() const+0x13e) [0x55a36a631fa0] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a36a632651] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a36a5c7347] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a36a2c387e] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a36a2c8f2f] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55a36a2d3969] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 11: (MDLog::_replay_thread()+0x1f38) [0x55a36a66c48c] 2024-05-27T07:36:30.683 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55a36a6759cb] 2024-05-27T07:36:30.684 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 13: (Thread::entry_wrapper()+0x43) [0x7f01925ec72d] 2024-05-27T07:36:30.684 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 14: (Thread::_entry_func(void*)+0xd) [0x7f01925ec749] 2024-05-27T07:36:30.684 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f01911901ca] 2024-05-27T07:36:30.684 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 16: clone() 2024-05-27T07:36:30.684 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.220+0000 7f017cc00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f017cc00700 time 2024-05-27T07:36:30.216657+0000 2024-05-27T07:36:30.684 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:36:30.684 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.684 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f019261dba6] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f019261ddd1] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a36a64823d] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55a36a633971] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 5: (SnapRealm::check_cache() const+0x13e) [0x55a36a631fa0] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a36a632651] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a36a5c7347] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a36a2c387e] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a36a2c8f2f] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55a36a2d3969] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 11: (MDLog::_replay_thread()+0x1f38) [0x55a36a66c48c] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55a36a6759cb] 2024-05-27T07:36:30.685 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 13: (Thread::entry_wrapper()+0x43) [0x7f01925ec72d] 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 14: (Thread::_entry_func(void*)+0xd) [0x7f01925ec749] 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f01911901ca] 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 16: clone() 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: *** Caught signal (Aborted) ** 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: in thread 7f017cc00700 thread_name:md_log_replay 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f019119ad20] 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2: gsignal() 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 3: abort() 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f019261dced] 2024-05-27T07:36:30.686 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f019261ddd1] 2024-05-27T07:36:30.687 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a36a64823d] 2024-05-27T07:36:30.687 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55a36a633971] 2024-05-27T07:36:30.687 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 8: (SnapRealm::check_cache() const+0x13e) [0x55a36a631fa0] 2024-05-27T07:36:30.687 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a36a632651] 2024-05-27T07:36:30.687 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a36a5c7347] 2024-05-27T07:36:30.687 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a36a2c387e] 2024-05-27T07:36:30.687 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a36a2c8f2f] 2024-05-27T07:36:30.687 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55a36a2d3969] 2024-05-27T07:36:30.687 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 14: (MDLog::_replay_thread()+0x1f38) [0x55a36a66c48c] 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55a36a6759cb] 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 16: (Thread::entry_wrapper()+0x43) [0x7f01925ec72d] 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 17: (Thread::_entry_func(void*)+0xd) [0x7f01925ec749] 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f01911901ca] 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 19: clone() 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.225+0000 7f017cc00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: in thread 7f017cc00700 thread_name:md_log_replay 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f019119ad20] 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2: gsignal() 2024-05-27T07:36:30.688 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 3: abort() 2024-05-27T07:36:30.689 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f019261dced] 2024-05-27T07:36:30.689 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f019261ddd1] 2024-05-27T07:36:30.689 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a36a64823d] 2024-05-27T07:36:30.689 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55a36a633971] 2024-05-27T07:36:30.689 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 8: (SnapRealm::check_cache() const+0x13e) [0x55a36a631fa0] 2024-05-27T07:36:30.689 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a36a632651] 2024-05-27T07:36:30.689 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a36a5c7347] 2024-05-27T07:36:30.689 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a36a2c387e] 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a36a2c8f2f] 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55a36a2d3969] 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 14: (MDLog::_replay_thread()+0x1f38) [0x55a36a66c48c] 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55a36a6759cb] 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 16: (Thread::entry_wrapper()+0x43) [0x7f01925ec72d] 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 17: (Thread::_entry_func(void*)+0xd) [0x7f01925ec749] 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f01911901ca] 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 19: clone() 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: -1> 2024-05-27T07:36:30.220+0000 7f017cc00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f017cc00700 time 2024-05-27T07:36:30.216657+0000 2024-05-27T07:36:30.690 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f019261dba6] 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f019261ddd1] 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a36a64823d] 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55a36a633971] 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 5: (SnapRealm::check_cache() const+0x13e) [0x55a36a631fa0] 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a36a632651] 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a36a5c7347] 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a36a2c387e] 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a36a2c8f2f] 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55a36a2d3969] 2024-05-27T07:36:30.691 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 11: (MDLog::_replay_thread()+0x1f38) [0x55a36a66c48c] 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55a36a6759cb] 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 13: (Thread::entry_wrapper()+0x43) [0x7f01925ec72d] 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 14: (Thread::_entry_func(void*)+0xd) [0x7f01925ec749] 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f01911901ca] 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 16: clone() 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 0> 2024-05-27T07:36:30.225+0000 7f017cc00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: in thread 7f017cc00700 thread_name:md_log_replay 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f019119ad20] 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2: gsignal() 2024-05-27T07:36:30.692 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 3: abort() 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f019261dced] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f019261ddd1] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a36a64823d] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55a36a633971] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 8: (SnapRealm::check_cache() const+0x13e) [0x55a36a631fa0] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a36a632651] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a36a5c7347] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a36a2c387e] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a36a2c8f2f] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55a36a2d3969] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 14: (MDLog::_replay_thread()+0x1f38) [0x55a36a66c48c] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55a36a6759cb] 2024-05-27T07:36:30.693 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 16: (Thread::entry_wrapper()+0x43) [0x7f01925ec72d] 2024-05-27T07:36:30.694 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 17: (Thread::_entry_func(void*)+0xd) [0x7f01925ec749] 2024-05-27T07:36:30.694 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f01911901ca] 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 19: clone() 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: -1> 2024-05-27T07:36:30.220+0000 7f017cc00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f017cc00700 time 2024-05-27T07:36:30.216657+0000 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f019261dba6] 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f019261ddd1] 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a36a64823d] 2024-05-27T07:36:30.695 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55a36a633971] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 5: (SnapRealm::check_cache() const+0x13e) [0x55a36a631fa0] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a36a632651] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a36a5c7347] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a36a2c387e] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a36a2c8f2f] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55a36a2d3969] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 11: (MDLog::_replay_thread()+0x1f38) [0x55a36a66c48c] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55a36a6759cb] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 13: (Thread::entry_wrapper()+0x43) [0x7f01925ec72d] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 14: (Thread::_entry_func(void*)+0xd) [0x7f01925ec749] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f01911901ca] 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 16: clone() 2024-05-27T07:36:30.696 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 0> 2024-05-27T07:36:30.225+0000 7f017cc00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: in thread 7f017cc00700 thread_name:md_log_replay 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f019119ad20] 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2: gsignal() 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 3: abort() 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f019261dced] 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f019261ddd1] 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a36a64823d] 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55a36a633971] 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 8: (SnapRealm::check_cache() const+0x13e) [0x55a36a631fa0] 2024-05-27T07:36:30.697 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a36a632651] 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a36a5c7347] 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a36a2c387e] 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a36a2c8f2f] 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55a36a2d3969] 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 14: (MDLog::_replay_thread()+0x1f38) [0x55a36a66c48c] 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55a36a6759cb] 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 16: (Thread::entry_wrapper()+0x43) [0x7f01925ec72d] 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 17: (Thread::_entry_func(void*)+0xd) [0x7f01925ec749] 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f01911901ca] 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 19: clone() 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[65639]: 2024-05-27T07:36:30.698 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:30 smithi149 podman[70428]: 2024-05-27 07:36:30.358958189 +0000 UTC m=+0.036995786 container died 1e9496991c7b9f07e89de00845bafc6a09ed2a5a7a97dc088e6db5e65ec1e06e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-27T07:36:30.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:30 smithi063 ceph-mon[31613]: MDS daemon mds.k is removed because it is dead or otherwise unavailable. 2024-05-27T07:36:30.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:30 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T07:36:31.709 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:31 smithi149 podman[70428]: 2024-05-27 07:36:31.214085748 +0000 UTC m=+0.892123368 container remove 1e9496991c7b9f07e89de00845bafc6a09ed2a5a7a97dc088e6db5e65ec1e06e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-27T07:36:31.709 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:31 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:36:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.806+0000 7f7e17600700 1 -- 172.21.15.63:0/1058733226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e1010e930 msgr2=0x7f7e10110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.806+0000 7f7e17600700 1 --2- 172.21.15.63:0/1058733226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e1010e930 0x7f7e10110e00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800b0a0 tx=0x7f7e0800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:36:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.806+0000 7f7e17600700 1 -- 172.21.15.63:0/1058733226 shutdown_connections 2024-05-27T07:36:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.806+0000 7f7e17600700 1 --2- 172.21.15.63:0/1058733226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e1010e930 0x7f7e10110e00 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.806+0000 7f7e17600700 1 --2- 172.21.15.63:0/1058733226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10105470 0x7f7e10105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.806+0000 7f7e17600700 1 --2- 172.21.15.63:0/1058733226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e10105e50 0x7f7e1010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.806+0000 7f7e17600700 1 -- 172.21.15.63:0/1058733226 >> 172.21.15.63:0/1058733226 conn(0x7f7e100fef40 msgr2=0x7f7e10101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.807+0000 7f7e17600700 1 -- 172.21.15.63:0/1058733226 shutdown_connections 2024-05-27T07:36:31.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.807+0000 7f7e17600700 1 -- 172.21.15.63:0/1058733226 wait complete. 2024-05-27T07:36:31.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.808+0000 7f7e17600700 1 Processor -- start 2024-05-27T07:36:31.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.808+0000 7f7e17600700 1 -- start start 2024-05-27T07:36:31.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.809+0000 7f7e17600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10105470 0x7f7e10103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:31.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.809+0000 7f7e17600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e10105e50 0x7f7e101036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:31.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.809+0000 7f7e17600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e1010e930 0x7f7e10103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:31.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.809+0000 7f7e17600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e10114350 con 0x7f7e10105e50 2024-05-27T07:36:31.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.809+0000 7f7e17600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e101141d0 con 0x7f7e10105470 2024-05-27T07:36:31.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.809+0000 7f7e17600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e101144d0 con 0x7f7e1010e930 2024-05-27T07:36:31.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.809+0000 7f7e0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e10105e50 0x7f7e101036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:31.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.809+0000 7f7e0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e10105e50 0x7f7e101036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44910/0 (socket says 172.21.15.63:44910) 2024-05-27T07:36:31.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.810+0000 7f7e0fe00700 1 -- 172.21.15.63:0/3805002623 learned_addr learned my addr 172.21.15.63:0/3805002623 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:31.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.810+0000 7f7e14c00700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10105470 0x7f7e10103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:31.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.810+0000 7f7e15600700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e1010e930 0x7f7e10103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:31.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.810+0000 7f7e14c00700 1 -- 172.21.15.63:0/3805002623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e1010e930 msgr2=0x7f7e10103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:31.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.810+0000 7f7e14c00700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e1010e930 0x7f7e10103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:31.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.810+0000 7f7e14c00700 1 -- 172.21.15.63:0/3805002623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e10105e50 msgr2=0x7f7e101036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:31.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.810+0000 7f7e14c00700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e10105e50 0x7f7e101036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:31.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.810+0000 7f7e14c00700 1 -- 172.21.15.63:0/3805002623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e101a9040 con 0x7f7e10105470 2024-05-27T07:36:31.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.810+0000 7f7e0fe00700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e10105e50 0x7f7e101036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:36:31.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.811+0000 7f7e14c00700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10105470 0x7f7e10103160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400d750 tx=0x7f7e0400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:31.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.811+0000 7f7e0d600700 1 -- 172.21.15.63:0/3805002623 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04014030 con 0x7f7e10105470 2024-05-27T07:36:31.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.811+0000 7f7e17600700 1 -- 172.21.15.63:0/3805002623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e101a92d0 con 0x7f7e10105470 2024-05-27T07:36:31.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.812+0000 7f7e17600700 1 -- 172.21.15.63:0/3805002623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e101a9860 con 0x7f7e10105470 2024-05-27T07:36:31.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.812+0000 7f7e0d600700 1 -- 172.21.15.63:0/3805002623 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e040044b0 con 0x7f7e10105470 2024-05-27T07:36:31.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.812+0000 7f7e0d600700 1 -- 172.21.15.63:0/3805002623 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0401d060 con 0x7f7e10105470 2024-05-27T07:36:31.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.813+0000 7f7e17600700 1 -- 172.21.15.63:0/3805002623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e100512b0 con 0x7f7e10105470 2024-05-27T07:36:31.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.817+0000 7f7e0d600700 1 -- 172.21.15.63:0/3805002623 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7e04004950 con 0x7f7e10105470 2024-05-27T07:36:31.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.818+0000 7f7e0d600700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df80b29f0 0x7f7df80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:31.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.819+0000 7f7e0d600700 1 -- 172.21.15.63:0/3805002623 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f7e040e2590 con 0x7f7e10105470 2024-05-27T07:36:31.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.819+0000 7f7e0fe00700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df80b29f0 0x7f7df80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:31.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.820+0000 7f7e0fe00700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df80b29f0 0x7f7df80b4f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7e101024a0 tx=0x7f7dfc00b480 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:31.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:31.833+0000 7f7e0d600700 1 -- 172.21.15.63:0/3805002623 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e04004750 con 0x7f7e10105470 2024-05-27T07:36:32.193 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:31 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-05-27T07:36:32.193 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:31 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 1.888s CPU time. 2024-05-27T07:36:32.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:32 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6832/3714867941,v1:172.21.15.86:6833/3714867941] up:boot 2024-05-27T07:36:32.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:32 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:36:32.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:36:32.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:32 smithi063 ceph-mon[31613]: pgmap v91: 129 pgs: 129 active+clean; 8.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 562 KiB/s wr, 52 op/s 2024-05-27T07:36:32.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.196+0000 7f7e17600700 1 -- 172.21.15.63:0/3805002623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e1002e5e0 con 0x7f7e10105470 2024-05-27T07:36:32.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.213+0000 7f7e0d600700 1 -- 172.21.15.63:0/3805002623 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11865 (secure 0 0 0) 0x7f7e040a9c00 con 0x7f7e10105470 2024-05-27T07:36:32.210 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:36:32.211 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":35654,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:36:30.184071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/2496628912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":2496628912},{"type":"v1","addr":"172.21.15.149:6835","nonce":2496628912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:36:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 -- 172.21.15.63:0/3805002623 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df80b29f0 msgr2=0x7f7df80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df80b29f0 0x7f7df80b4f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7e101024a0 tx=0x7f7dfc00b480 comp rx=0 tx=0).stop 2024-05-27T07:36:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 -- 172.21.15.63:0/3805002623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10105470 msgr2=0x7f7e10103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:32.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10105470 0x7f7e10103160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400d750 tx=0x7f7e0400da60 comp rx=0 tx=0).stop 2024-05-27T07:36:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 -- 172.21.15.63:0/3805002623 shutdown_connections 2024-05-27T07:36:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df80b29f0 0x7f7df80b4f00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e1010e930 0x7f7e10103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e10105470 0x7f7e10103160 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:32.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 --2- 172.21.15.63:0/3805002623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e10105e50 0x7f7e101036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 -- 172.21.15.63:0/3805002623 >> 172.21.15.63:0/3805002623 conn(0x7f7e100fef40 msgr2=0x7f7e100ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 -- 172.21.15.63:0/3805002623 shutdown_connections 2024-05-27T07:36:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:32.217+0000 7f7e17600700 1 -- 172.21.15.63:0/3805002623 wait complete. 2024-05-27T07:36:32.215 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 30 2024-05-27T07:36:32.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:32 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6832/3714867941,v1:172.21.15.86:6833/3714867941] up:boot 2024-05-27T07:36:32.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:32 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:36:32.454 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:36:32.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:32 smithi086 ceph-mon[34687]: pgmap v91: 129 pgs: 129 active+clean; 8.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 562 KiB/s wr, 52 op/s 2024-05-27T07:36:32.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:32 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6832/3714867941,v1:172.21.15.86:6833/3714867941] up:boot 2024-05-27T07:36:32.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:32 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:36:32.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:36:32.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:32 smithi149 ceph-mon[35408]: pgmap v91: 129 pgs: 129 active+clean; 8.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 562 KiB/s wr, 52 op/s 2024-05-27T07:36:33.233 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:36:33.458 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:33 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3805002623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:36:33.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:33 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3805002623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:36:33.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:33 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3805002623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:36:33.737 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:36:34.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:34 smithi086 ceph-mon[34687]: pgmap v92: 129 pgs: 129 active+clean; 8.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 487 KiB/s wr, 33 op/s 2024-05-27T07:36:34.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:34 smithi149 ceph-mon[35408]: pgmap v92: 129 pgs: 129 active+clean; 8.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 487 KiB/s wr, 33 op/s 2024-05-27T07:36:34.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:34 smithi063 ceph-mon[31613]: pgmap v92: 129 pgs: 129 active+clean; 8.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 487 KiB/s wr, 33 op/s 2024-05-27T07:36:35.650 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:35 smithi063 ceph-mon[31613]: pgmap v93: 129 pgs: 129 active+clean; 8.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 487 KiB/s wr, 33 op/s 2024-05-27T07:36:35.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:35 smithi086 ceph-mon[34687]: pgmap v93: 129 pgs: 129 active+clean; 8.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 487 KiB/s wr, 33 op/s 2024-05-27T07:36:35.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:35 smithi149 ceph-mon[35408]: pgmap v93: 129 pgs: 129 active+clean; 8.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 487 KiB/s wr, 33 op/s 2024-05-27T07:36:36.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.335+0000 7fbad3200700 1 -- 172.21.15.63:0/3412310097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbad4103300 msgr2=0x7fbad4103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:36.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.335+0000 7fbad3200700 1 --2- 172.21.15.63:0/3412310097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbad4103300 0x7fbad4103710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbac40099c0 tx=0x7fbac4009cd0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.335+0000 7fbad3200700 1 -- 172.21.15.63:0/3412310097 shutdown_connections 2024-05-27T07:36:36.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.335+0000 7fbad3200700 1 --2- 172.21.15.63:0/3412310097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad4108580 0x7fbad410fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.335+0000 7fbad3200700 1 --2- 172.21.15.63:0/3412310097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbad4103300 0x7fbad4103710 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.335+0000 7fbad3200700 1 --2- 172.21.15.63:0/3412310097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad4107c30 0x7fbad4108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.335+0000 7fbad3200700 1 -- 172.21.15.63:0/3412310097 >> 172.21.15.63:0/3412310097 conn(0x7fbad4078c30 msgr2=0x7fbad4079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:36.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.335+0000 7fbad3200700 1 -- 172.21.15.63:0/3412310097 shutdown_connections 2024-05-27T07:36:36.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.335+0000 7fbad3200700 1 -- 172.21.15.63:0/3412310097 wait complete. 2024-05-27T07:36:36.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.336+0000 7fbad3200700 1 Processor -- start 2024-05-27T07:36:36.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.337+0000 7fbad3200700 1 -- start start 2024-05-27T07:36:36.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.337+0000 7fbad3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad4103300 0x7fbad4100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.337+0000 7fbad3200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbad4107c30 0x7fbad4100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.337+0000 7fbad3200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad4108580 0x7fbad4100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.337+0000 7fbad3200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41130b0 con 0x7fbad4107c30 2024-05-27T07:36:36.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.337+0000 7fbad3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4112f30 con 0x7fbad4103300 2024-05-27T07:36:36.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.337+0000 7fbad3200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4113230 con 0x7fbad4108580 2024-05-27T07:36:36.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.338+0000 7fbad1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbad4107c30 0x7fbad4100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.338+0000 7fbad1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbad4107c30 0x7fbad4100950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44926/0 (socket says 172.21.15.63:44926) 2024-05-27T07:36:36.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.338+0000 7fbad1400700 1 -- 172.21.15.63:0/2112721968 learned_addr learned my addr 172.21.15.63:0/2112721968 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:36.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.338+0000 7fbad1e00700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad4103300 0x7fbad4100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.338+0000 7fbad2800700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad4108580 0x7fbad4100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.338+0000 7fbad1400700 1 -- 172.21.15.63:0/2112721968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad4108580 msgr2=0x7fbad4100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:36.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.338+0000 7fbad1400700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad4108580 0x7fbad4100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.338+0000 7fbad1400700 1 -- 172.21.15.63:0/2112721968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad4103300 msgr2=0x7fbad4100410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:36.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.338+0000 7fbad1400700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad4103300 0x7fbad4100410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.338+0000 7fbad1400700 1 -- 172.21.15.63:0/2112721968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad41a8f70 con 0x7fbad4107c30 2024-05-27T07:36:36.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.339+0000 7fbad1400700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbad4107c30 0x7fbad4100950 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbac000d980 tx=0x7fbac000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:36.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.339+0000 7fbacaa00700 1 -- 172.21.15.63:0/2112721968 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0017070 con 0x7fbad4107c30 2024-05-27T07:36:36.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.339+0000 7fbad3200700 1 -- 172.21.15.63:0/2112721968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad41a9260 con 0x7fbad4107c30 2024-05-27T07:36:36.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.339+0000 7fbacaa00700 1 -- 172.21.15.63:0/2112721968 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbac000fd20 con 0x7fbad4107c30 2024-05-27T07:36:36.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.339+0000 7fbacaa00700 1 -- 172.21.15.63:0/2112721968 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac000e3b0 con 0x7fbad4107c30 2024-05-27T07:36:36.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.340+0000 7fbad3200700 1 -- 172.21.15.63:0/2112721968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41a97a0 con 0x7fbad4107c30 2024-05-27T07:36:36.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.344+0000 7fbacaa00700 1 -- 172.21.15.63:0/2112721968 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbac000f890 con 0x7fbad4107c30 2024-05-27T07:36:36.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.345+0000 7fbae7798b80 1 Processor -- start 2024-05-27T07:36:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.345+0000 7fbae7798b80 1 -- start start 2024-05-27T07:36:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.346+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766199e40 0x564766317ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.346+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766318410 0x564766322ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.346+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318820 0x564766325020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.346+0000 7fbae7798b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564766246e40 con 0x564766318410 2024-05-27T07:36:36.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.346+0000 7fbae7798b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5647662369e0 con 0x564766318820 2024-05-27T07:36:36.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.346+0000 7fbae7798b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56476623aa20 con 0x564766199e40 2024-05-27T07:36:36.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.346+0000 7fbad1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766318410 0x564766322ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbad1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766318410 0x564766322ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44942/0 (socket says 172.21.15.63:44942) 2024-05-27T07:36:36.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbad1e00700 1 -- 172.21.15.63:0/3739764477 learned_addr learned my addr 172.21.15.63:0/3739764477 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:36.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbad2800700 1 --2- 172.21.15.63:0/3739764477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766199e40 0x564766317ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbad1400700 1 --2- 172.21.15.63:0/3739764477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318820 0x564766325020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbacaa00700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbab80b29f0 0x7fbab80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbacaa00700 1 -- 172.21.15.63:0/2112721968 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fbac00e2800 con 0x7fbad4107c30 2024-05-27T07:36:36.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbad2800700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbab80b29f0 0x7fbab80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbad1e00700 1 -- 172.21.15.63:0/3739764477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766199e40 msgr2=0x564766317ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:36.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbad1e00700 1 --2- 172.21.15.63:0/3739764477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766199e40 0x564766317ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbad1e00700 1 -- 172.21.15.63:0/3739764477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318820 msgr2=0x564766325020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:36.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbad1e00700 1 --2- 172.21.15.63:0/3739764477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318820 0x564766325020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.347+0000 7fbad1e00700 1 -- 172.21.15.63:0/3739764477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647661ff420 con 0x564766318410 2024-05-27T07:36:36.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.348+0000 7fbad1e00700 1 --2- 172.21.15.63:0/3739764477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766318410 0x564766322ad0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbac400b580 tx=0x7fbac40096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67fabec95e586d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:36.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.348+0000 7fbad2800700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbab80b29f0 0x7fbab80b4f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x564766314690 tx=0x7fbacc004b40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:36.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.348+0000 7fbabf400700 1 -- 172.21.15.63:0/3739764477 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4027070 con 0x564766318410 2024-05-27T07:36:36.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.348+0000 7fbabf400700 1 -- 172.21.15.63:0/3739764477 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbac400b790 con 0x564766318410 2024-05-27T07:36:36.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.349+0000 7fbabf400700 1 -- 172.21.15.63:0/3739764477 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac40039a0 con 0x564766318410 2024-05-27T07:36:36.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.349+0000 7fbae7798b80 1 -- 172.21.15.63:0/3739764477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766318410 msgr2=0x564766322ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:36.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.349+0000 7fbae7798b80 1 --2- 172.21.15.63:0/3739764477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766318410 0x564766322ad0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbac400b580 tx=0x7fbac40096e0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.350+0000 7fbae7798b80 1 -- 172.21.15.63:0/3739764477 shutdown_connections 2024-05-27T07:36:36.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.350+0000 7fbae7798b80 1 --2- 172.21.15.63:0/3739764477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318820 0x564766325020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.350+0000 7fbae7798b80 1 --2- 172.21.15.63:0/3739764477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766199e40 0x564766317ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.350+0000 7fbae7798b80 1 --2- 172.21.15.63:0/3739764477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766318410 0x564766322ad0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.350+0000 7fbae7798b80 1 -- 172.21.15.63:0/3739764477 >> 172.21.15.63:0/3739764477 conn(0x5647661f9fc0 msgr2=0x564766313d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:36.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.350+0000 7fbae7798b80 1 -- 172.21.15.63:0/3739764477 shutdown_connections 2024-05-27T07:36:36.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.350+0000 7fbae7798b80 1 -- 172.21.15.63:0/3739764477 wait complete. 2024-05-27T07:36:36.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.351+0000 7fbae7798b80 1 -- start start 2024-05-27T07:36:36.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.351+0000 7fbae7798b80 1 Processor -- start 2024-05-27T07:36:36.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x564766322500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766318410 0x564766322a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318820 0x564766322f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbae7798b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56476613cfa0 con 0x564766199e40 2024-05-27T07:36:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbae7798b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5647661d7760 con 0x564766318820 2024-05-27T07:36:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbae7798b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564766325980 con 0x564766318410 2024-05-27T07:36:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbae7798b80 10 client.0 _ll_register_callbacks cb 0x5647661e0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbad1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x564766322500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbad1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x564766322500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44954/0 (socket says 172.21.15.63:44954) 2024-05-27T07:36:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbad1e00700 1 -- 172.21.15.63:0/2717568369 learned_addr learned my addr 172.21.15.63:0/2717568369 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.352+0000 7fbad2800700 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318820 0x564766322f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.353+0000 7fbad1400700 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766318410 0x564766322a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.353+0000 7fbad1e00700 1 -- 172.21.15.63:0/2717568369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766318410 msgr2=0x564766322a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.353+0000 7fbad1e00700 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766318410 0x564766322a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.353+0000 7fbad1e00700 1 -- 172.21.15.63:0/2717568369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318820 msgr2=0x564766322f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.353+0000 7fbad1e00700 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318820 0x564766322f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.353+0000 7fbad1e00700 1 -- 172.21.15.63:0/2717568369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647661868d0 con 0x564766199e40 2024-05-27T07:36:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.353+0000 7fbad1e00700 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x564766322500 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbac4005f50 tx=0x7fbac40053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9288f746f2233160 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.354+0000 7fbab3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.354+0000 7fbab3e00700 1 -- 172.21.15.63:0/2717568369 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4030020 con 0x564766199e40 2024-05-27T07:36:36.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.354+0000 7fbae7798b80 1 -- 172.21.15.63:0/2717568369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac400b790 con 0x564766199e40 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.354+0000 7fbab3e00700 1 -- 172.21.15.63:0/2717568369 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fbac400bb40 con 0x564766199e40 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.354+0000 7fbab3e00700 1 -- 172.21.15.63:0/2717568369 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbac4027050 con 0x564766199e40 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.355+0000 7fbae7798b80 10 client.15975 fetch_fsmap learned FSMap version 30 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.355+0000 7fbae7798b80 1 -- 172.21.15.63:0/2717568369 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564766207060 con 0x564766199e40 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.355+0000 7fbab3e00700 1 -- 172.21.15.63:0/2717568369 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbac400b790 con 0x564766199e40 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.355+0000 7fbab3e00700 1 -- 172.21.15.63:0/2717568369 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fbac400f4f0 con 0x564766199e40 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.355+0000 7fbae7798b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.355+0000 7fbae7798b80 10 client.15975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.355+0000 7fbae7798b80 20 client.15975 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.355+0000 7fbae7798b80 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x564766332700 0x564766334b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.355+0000 7fbae7798b80 4 client.15975 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:36:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.355+0000 7fbae7798b80 1 -- 172.21.15.63:0/2717568369 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564766248d90 con 0x564766332700 2024-05-27T07:36:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.356+0000 7fbad1400700 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x564766332700 0x564766334b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.357+0000 7fbad1400700 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x564766332700 0x564766334b20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1824ecd5e0167b5 server_cookie=203f0b2894ef9795 in_seq=0 out_seq=0 2024-05-27T07:36:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.357+0000 7fbab3e00700 10 client.15975 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:36:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbab3e00700 1 -- 172.21.15.63:0/2717568369 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564766248d90 con 0x564766332700 2024-05-27T07:36:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbab3e00700 10 client.15975 handle_command_reply: tid=0 2024-05-27T07:36:36.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbae7798b80 1 client.15975 shutdown 2024-05-27T07:36:36.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbae7798b80 1 -- 172.21.15.63:0/2717568369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 msgr2=0x564766322500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:36.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbae7798b80 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x564766322500 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbac4005f50 tx=0x7fbac40053e0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbae7798b80 1 -- 172.21.15.63:0/2717568369 shutdown_connections 2024-05-27T07:36:36.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbae7798b80 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318820 0x564766322f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbae7798b80 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x564766332700 0x564766334b20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbae7798b80 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x564766322500 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbae7798b80 1 --2- 172.21.15.63:0/2717568369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766318410 0x564766322a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbab3e00700 0 client.15975 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:36:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.359+0000 7fbae7798b80 1 -- 172.21.15.63:0/2717568369 >> 172.21.15.63:0/2717568369 conn(0x5647661f9fc0 msgr2=0x564766312ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.360+0000 7fbae7798b80 1 -- 172.21.15.63:0/2717568369 shutdown_connections 2024-05-27T07:36:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.360+0000 7fbae7798b80 1 -- 172.21.15.63:0/2717568369 wait complete. 2024-05-27T07:36:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.360+0000 7fbae7798b80 20 client.15975 trim_cache size 0 max 16384 2024-05-27T07:36:36.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.377+0000 7fbae7798b80 1 Processor -- start 2024-05-27T07:36:36.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.378+0000 7fbae7798b80 1 -- start start 2024-05-27T07:36:36.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.378+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647661f9fc0 0x56476631dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.378+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x56476632ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.378+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 0x564766324860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.378+0000 7fbae7798b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41b4de0 con 0x564766199e40 2024-05-27T07:36:36.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.378+0000 7fbae7798b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41b2670 con 0x564766318410 2024-05-27T07:36:36.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.378+0000 7fbae7798b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41b04c0 con 0x5647661f9fc0 2024-05-27T07:36:36.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.378+0000 7fbad1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x56476632ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.379+0000 7fbad1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x56476632ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44970/0 (socket says 172.21.15.63:44970) 2024-05-27T07:36:36.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.379+0000 7fbad1400700 1 -- 172.21.15.63:0/2931937995 learned_addr learned my addr 172.21.15.63:0/2931937995 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:36.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.379+0000 7fbad1e00700 1 --2- 172.21.15.63:0/2931937995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647661f9fc0 0x56476631dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.379+0000 7fbad2800700 1 --2- 172.21.15.63:0/2931937995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 0x564766324860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.379+0000 7fbad1400700 1 -- 172.21.15.63:0/2931937995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647661f9fc0 msgr2=0x56476631dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.379+0000 7fbad1400700 1 --2- 172.21.15.63:0/2931937995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647661f9fc0 0x56476631dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.379+0000 7fbad1400700 1 -- 172.21.15.63:0/2931937995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 msgr2=0x564766324860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.379+0000 7fbad1400700 1 --2- 172.21.15.63:0/2931937995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 0x564766324860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.379+0000 7fbad1400700 1 -- 172.21.15.63:0/2931937995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564766317a40 con 0x564766199e40 2024-05-27T07:36:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.379+0000 7fbad2800700 1 --2- 172.21.15.63:0/2931937995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 0x564766324860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.380+0000 7fbad1400700 1 --2- 172.21.15.63:0/2931937995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x56476632ec20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbac00a6980 tx=0x7fbac0091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66732d0a0d58d439 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.380+0000 7fbabea00700 1 -- 172.21.15.63:0/2931937995 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0020040 con 0x564766199e40 2024-05-27T07:36:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.380+0000 7fbabea00700 1 -- 172.21.15.63:0/2931937995 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbac00a9220 con 0x564766199e40 2024-05-27T07:36:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.380+0000 7fbabea00700 1 -- 172.21.15.63:0/2931937995 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac00b2570 con 0x564766199e40 2024-05-27T07:36:36.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.381+0000 7fbae7798b80 1 -- 172.21.15.63:0/2931937995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 msgr2=0x56476632ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:36.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.381+0000 7fbae7798b80 1 --2- 172.21.15.63:0/2931937995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x56476632ec20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbac00a6980 tx=0x7fbac0091040 comp rx=0 tx=0).stop 2024-05-27T07:36:36.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.381+0000 7fbae7798b80 1 -- 172.21.15.63:0/2931937995 shutdown_connections 2024-05-27T07:36:36.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.381+0000 7fbae7798b80 1 --2- 172.21.15.63:0/2931937995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 0x564766324860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.381+0000 7fbae7798b80 1 --2- 172.21.15.63:0/2931937995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647661f9fc0 0x56476631dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.381+0000 7fbae7798b80 1 --2- 172.21.15.63:0/2931937995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564766199e40 0x56476632ec20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.381+0000 7fbae7798b80 1 -- 172.21.15.63:0/2931937995 >> 172.21.15.63:0/2931937995 conn(0x56476630cf60 msgr2=0x564766322d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:36.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.381+0000 7fbae7798b80 1 -- 172.21.15.63:0/2931937995 shutdown_connections 2024-05-27T07:36:36.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.381+0000 7fbae7798b80 1 -- 172.21.15.63:0/2931937995 wait complete. 2024-05-27T07:36:36.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.382+0000 7fbae7798b80 1 -- start start 2024-05-27T07:36:36.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.382+0000 7fbae7798b80 1 Processor -- start 2024-05-27T07:36:36.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.383+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766199e40 0x564766338090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.383+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647661f9fc0 0x564766335410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.383+0000 7fbae7798b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 0x564766335950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.383+0000 7fbae7798b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41b0b30 con 0x5647661f9fc0 2024-05-27T07:36:36.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.383+0000 7fbae7798b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41b1fa0 con 0x564766318410 2024-05-27T07:36:36.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.383+0000 7fbae7798b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41b1200 con 0x564766199e40 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.383+0000 7fbae7798b80 10 client.0 _ll_register_callbacks cb 0x5647661e0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.383+0000 7fbad1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647661f9fc0 0x564766335410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.383+0000 7fbad1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647661f9fc0 0x564766335410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44974/0 (socket says 172.21.15.63:44974) 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.383+0000 7fbad1400700 1 -- 172.21.15.63:0/72488071 learned_addr learned my addr 172.21.15.63:0/72488071 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.384+0000 7fbad1e00700 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766199e40 0x564766338090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.384+0000 7fbad2800700 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 0x564766335950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.384+0000 7fbad1400700 1 -- 172.21.15.63:0/72488071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766199e40 msgr2=0x564766338090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.384+0000 7fbad1400700 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766199e40 0x564766338090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.384+0000 7fbad1400700 1 -- 172.21.15.63:0/72488071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 msgr2=0x564766335950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.384+0000 7fbad1400700 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 0x564766335950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.384+0000 7fbad1400700 1 -- 172.21.15.63:0/72488071 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564766335f10 con 0x5647661f9fc0 2024-05-27T07:36:36.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.384+0000 7fbad1400700 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647661f9fc0 0x564766335410 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbac000bfd0 tx=0x7fbac00a9610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2111852006308078 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:36.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.384+0000 7fbab3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:36.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.385+0000 7fbab3e00700 1 -- 172.21.15.63:0/72488071 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0020070 con 0x5647661f9fc0 2024-05-27T07:36:36.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.385+0000 7fbab3e00700 1 -- 172.21.15.63:0/72488071 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fbac00910b0 con 0x5647661f9fc0 2024-05-27T07:36:36.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.385+0000 7fbae7798b80 1 -- 172.21.15.63:0/72488071 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac00a9220 con 0x5647661f9fc0 2024-05-27T07:36:36.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.385+0000 7fbab3e00700 1 -- 172.21.15.63:0/72488071 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbac00e0010 con 0x5647661f9fc0 2024-05-27T07:36:36.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.385+0000 7fbae7798b80 10 client.15981 fetch_fsmap learned FSMap version 30 2024-05-27T07:36:36.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.385+0000 7fbae7798b80 1 -- 172.21.15.63:0/72488071 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647663a54d0 con 0x5647661f9fc0 2024-05-27T07:36:36.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.385+0000 7fbab3e00700 1 -- 172.21.15.63:0/72488071 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbac00a9220 con 0x5647661f9fc0 2024-05-27T07:36:36.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.385+0000 7fbab3e00700 1 -- 172.21.15.63:0/72488071 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fbac00e23b0 con 0x5647661f9fc0 2024-05-27T07:36:36.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.386+0000 7fbae7798b80 10 client.15981 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:36:36.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.386+0000 7fbae7798b80 10 client.15981 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:36:36.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.386+0000 7fbae7798b80 20 client.15981 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:36.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.386+0000 7fbae7798b80 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x564766318820 0x5647663a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:36.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.386+0000 7fbae7798b80 4 client.15981 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:36:36.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.386+0000 7fbae7798b80 1 -- 172.21.15.63:0/72488071 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564766242570 con 0x564766318820 2024-05-27T07:36:36.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.387+0000 7fbad1e00700 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x564766318820 0x5647663a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:36.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbad1e00700 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x564766318820 0x5647663a7bf0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d40a5b5697e2366 server_cookie=53f86637621d4f46 in_seq=0 out_seq=0 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbab3e00700 10 client.15981 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbab3e00700 1 -- 172.21.15.63:0/72488071 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564766242570 con 0x564766318820 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbab3e00700 10 client.15981 handle_command_reply: tid=0 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbae7798b80 1 client.15981 shutdown 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbae7798b80 1 -- 172.21.15.63:0/72488071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647661f9fc0 msgr2=0x564766335410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbae7798b80 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647661f9fc0 0x564766335410 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbac000bfd0 tx=0x7fbac00a9610 comp rx=0 tx=0).stop 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbae7798b80 1 -- 172.21.15.63:0/72488071 shutdown_connections 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbae7798b80 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564766318410 0x564766335950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbae7798b80 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564766199e40 0x564766338090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.388+0000 7fbae7798b80 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x564766318820 0x5647663a7bf0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.389+0000 7fbae7798b80 1 --2- 172.21.15.63:0/72488071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647661f9fc0 0x564766335410 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.389+0000 7fbae7798b80 1 -- 172.21.15.63:0/72488071 >> 172.21.15.63:0/72488071 conn(0x56476630cf60 msgr2=0x5647663222e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:36.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.389+0000 7fbab3e00700 0 client.15981 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:36:36.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.389+0000 7fbae7798b80 1 -- 172.21.15.63:0/72488071 shutdown_connections 2024-05-27T07:36:36.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.389+0000 7fbae7798b80 1 -- 172.21.15.63:0/72488071 wait complete. 2024-05-27T07:36:36.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.389+0000 7fbae7798b80 20 client.15981 trim_cache size 0 max 16384 2024-05-27T07:36:36.387 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.390+0000 7fbad3200700 1 -- 172.21.15.63:0/2112721968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbab80b29f0 msgr2=0x7fbab80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.390+0000 7fbad3200700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbab80b29f0 0x7fbab80b4f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x564766314690 tx=0x7fbacc004b40 comp rx=0 tx=0).stop 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.390+0000 7fbad3200700 1 -- 172.21.15.63:0/2112721968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbad4107c30 msgr2=0x7fbad4100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.390+0000 7fbad3200700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbad4107c30 0x7fbad4100950 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbac000d980 tx=0x7fbac000dc90 comp rx=0 tx=0).stop 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.390+0000 7fbad3200700 1 -- 172.21.15.63:0/2112721968 shutdown_connections 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.390+0000 7fbad3200700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbab80b29f0 0x7fbab80b4f00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.390+0000 7fbad3200700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbad4108580 0x7fbad4100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.390+0000 7fbad3200700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbad4103300 0x7fbad4100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.390+0000 7fbad3200700 1 --2- 172.21.15.63:0/2112721968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbad4107c30 0x7fbad4100950 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.390+0000 7fbad3200700 1 -- 172.21.15.63:0/2112721968 >> 172.21.15.63:0/2112721968 conn(0x7fbad4078c30 msgr2=0x7fbad406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.391+0000 7fbad3200700 1 -- 172.21.15.63:0/2112721968 shutdown_connections 2024-05-27T07:36:36.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:36.391+0000 7fbad3200700 1 -- 172.21.15.63:0/2112721968 wait complete. 2024-05-27T07:36:37.412 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:36:37.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:37 smithi086 ceph-mon[34687]: pgmap v94: 129 pgs: 129 active+clean; 40 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.1 MiB/s wr, 37 op/s 2024-05-27T07:36:37.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:37 smithi149 ceph-mon[35408]: pgmap v94: 129 pgs: 129 active+clean; 40 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.1 MiB/s wr, 37 op/s 2024-05-27T07:36:37.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:37 smithi063 ceph-mon[31613]: pgmap v94: 129 pgs: 129 active+clean; 40 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.1 MiB/s wr, 37 op/s 2024-05-27T07:36:37.923 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:36:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.545+0000 7fc7ee400640 1 -- 172.21.15.63:0/653309027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e8110320 msgr2=0x7fc7e8112740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.545+0000 7fc7ee400640 1 --2- 172.21.15.63:0/653309027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e8110320 0x7fc7e8112740 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009ca0 tx=0x7fc7e002a950 comp rx=0 tx=0).stop 2024-05-27T07:36:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.545+0000 7fc7ee400640 1 -- 172.21.15.63:0/653309027 shutdown_connections 2024-05-27T07:36:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.545+0000 7fc7ee400640 1 --2- 172.21.15.63:0/653309027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e8110320 0x7fc7e8112740 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.545+0000 7fc7ee400640 1 --2- 172.21.15.63:0/653309027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7e810d9c0 0x7fc7e810fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.545+0000 7fc7ee400640 1 --2- 172.21.15.63:0/653309027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e8075420 0x7fc7e8075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.545+0000 7fc7ee400640 1 -- 172.21.15.63:0/653309027 >> 172.21.15.63:0/653309027 conn(0x7fc7e8079100 msgr2=0x7fc7e8079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.545+0000 7fc7ee400640 1 -- 172.21.15.63:0/653309027 shutdown_connections 2024-05-27T07:36:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.545+0000 7fc7ee400640 1 -- 172.21.15.63:0/653309027 wait complete. 2024-05-27T07:36:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.546+0000 7fc7ee400640 1 Processor -- start 2024-05-27T07:36:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.547+0000 7fc7ee400640 1 -- start start 2024-05-27T07:36:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.547+0000 7fc7ee400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7e8075420 0x7fc7e819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.547+0000 7fc7ee400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e810d9c0 0x7fc7e819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.547+0000 7fc7ee400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e8110320 0x7fc7e81a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.547+0000 7fc7ee400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8115d40 con 0x7fc7e810d9c0 2024-05-27T07:36:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.547+0000 7fc7ee400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8115bc0 con 0x7fc7e8110320 2024-05-27T07:36:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.547+0000 7fc7ee400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8115ec0 con 0x7fc7e8075420 2024-05-27T07:36:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.547+0000 7fc7e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e810d9c0 0x7fc7e819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.547+0000 7fc7e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e810d9c0 0x7fc7e819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44988/0 (socket says 172.21.15.63:44988) 2024-05-27T07:36:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.548+0000 7fc7e7e00640 1 -- 172.21.15.63:0/560029707 learned_addr learned my addr 172.21.15.63:0/560029707 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.548+0000 7fc7ed000640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7e8075420 0x7fc7e819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.548+0000 7fc7eda00640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e8110320 0x7fc7e81a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.548+0000 7fc7e7e00640 1 -- 172.21.15.63:0/560029707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7e8075420 msgr2=0x7fc7e819c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.548+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7e8075420 0x7fc7e819c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.548+0000 7fc7e7e00640 1 -- 172.21.15.63:0/560029707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e8110320 msgr2=0x7fc7e81a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.548+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e8110320 0x7fc7e81a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.548+0000 7fc7e7e00640 1 -- 172.21.15.63:0/560029707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e81a15e0 con 0x7fc7e810d9c0 2024-05-27T07:36:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.549+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e810d9c0 0x7fc7e819cce0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400d770 tx=0x7fc7d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.549+0000 7fc7e5600640 1 -- 172.21.15.63:0/560029707 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4017070 con 0x7fc7e810d9c0 2024-05-27T07:36:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.549+0000 7fc7e5600640 1 -- 172.21.15.63:0/560029707 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7d4009c30 con 0x7fc7e810d9c0 2024-05-27T07:36:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.549+0000 7fc7ee400640 1 -- 172.21.15.63:0/560029707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e81a18d0 con 0x7fc7e810d9c0 2024-05-27T07:36:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.549+0000 7fc7ee400640 1 -- 172.21.15.63:0/560029707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e81a9250 con 0x7fc7e810d9c0 2024-05-27T07:36:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.549+0000 7fc7e5600640 1 -- 172.21.15.63:0/560029707 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d400f620 con 0x7fc7e810d9c0 2024-05-27T07:36:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.553+0000 7fc7f69fb740 1 Processor -- start 2024-05-27T07:36:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.553+0000 7fc7f69fb740 1 -- start start 2024-05-27T07:36:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.554+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x562646142a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.554+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 0x562646142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.554+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 0x5626461434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.554+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5626460b3b90 con 0x562645da01b0 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.554+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5626460a3730 con 0x562645d772f0 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.554+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5626460a7770 con 0x562645d7f4d0 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.554+0000 7fc7e5600640 1 -- 172.21.15.63:0/560029707 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc7d401a070 con 0x7fc7e810d9c0 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.554+0000 7fc7eda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x562646142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7ed000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 0x562646142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7ed000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 0x562646142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:49864/0 (socket says 172.21.15.63:49864) 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7ed000640 1 -- 172.21.15.63:0/3543831928 learned_addr learned my addr 172.21.15.63:0/3543831928 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7eda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x562646142a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44990/0 (socket says 172.21.15.63:44990) 2024-05-27T07:36:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/3543831928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 0x5626461434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7ed000640 1 -- 172.21.15.63:0/3543831928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 msgr2=0x5626461434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7ed000640 1 --2- 172.21.15.63:0/3543831928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 0x5626461434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7ed000640 1 -- 172.21.15.63:0/3543831928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 msgr2=0x562646142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7ed000640 1 --2- 172.21.15.63:0/3543831928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x562646142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7ed000640 1 -- 172.21.15.63:0/3543831928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56264603c8b0 con 0x562645d772f0 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7eda00640 1 --2- 172.21.15.63:0/3543831928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x562646142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.555+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/3543831928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 0x5626461434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.556+0000 7fc7ed000640 1 --2- 172.21.15.63:0/3543831928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 0x562646142fa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009cf0 tx=0x7fc7dc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9ac22b22185d8af server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.556+0000 7fc7da000640 1 -- 172.21.15.63:0/3543831928 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc004230 con 0x562645d772f0 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.556+0000 7fc7da000640 1 -- 172.21.15.63:0/3543831928 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7dc0043d0 con 0x562645d772f0 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.556+0000 7fc7da000640 1 -- 172.21.15.63:0/3543831928 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc019750 con 0x562645d772f0 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.556+0000 7fc7e5600640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc7d00b29d0 0x7fc7d00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.557+0000 7fc7e5600640 1 -- 172.21.15.63:0/560029707 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fc7d40e21d0 con 0x7fc7e810d9c0 2024-05-27T07:36:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.557+0000 7fc7eda00640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc7d00b29d0 0x7fc7d00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.557+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3543831928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 msgr2=0x562646142fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.557+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/3543831928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 0x562646142fa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009cf0 tx=0x7fc7dc011810 comp rx=0 tx=0).stop 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.558+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3543831928 shutdown_connections 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.558+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/3543831928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 0x5626461434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.558+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/3543831928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 0x562646142fa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.558+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/3543831928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x562646142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.558+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3543831928 >> 172.21.15.63:0/3543831928 conn(0x562645d94300 msgr2=0x56264613f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.558+0000 7fc7eda00640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc7d00b29d0 0x7fc7d00b4ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x56264613fbd0 tx=0x7fc7e0005c50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.558+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3543831928 shutdown_connections 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.558+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3543831928 wait complete. 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.558+0000 7fc7f69fb740 1 -- start start 2024-05-27T07:36:38.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.559+0000 7fc7f69fb740 1 Processor -- start 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.559+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d772f0 0x56264614cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.559+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d7f4d0 0x56264614d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.559+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645da01b0 0x562646158f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.559+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562645d89790 con 0x562645d7f4d0 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.559+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56264604bbc0 con 0x562645da01b0 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.559+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56264614fb90 con 0x562645d772f0 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.559+0000 7fc7f69fb740 10 client.0 _ll_register_callbacks cb 0x562646029400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.560+0000 7fc7e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d7f4d0 0x56264614d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.560+0000 7fc7e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d7f4d0 0x56264614d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44996/0 (socket says 172.21.15.63:44996) 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.560+0000 7fc7e7e00640 1 -- 172.21.15.63:0/3161629676 learned_addr learned my addr 172.21.15.63:0/3161629676 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.560+0000 7fc7ed000640 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d772f0 0x56264614cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.560+0000 7fc7eda00640 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645da01b0 0x562646158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7e7e00640 1 -- 172.21.15.63:0/3161629676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d772f0 msgr2=0x56264614cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d772f0 0x56264614cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7e7e00640 1 -- 172.21.15.63:0/3161629676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645da01b0 msgr2=0x562646158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645da01b0 0x562646158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7e7e00640 1 -- 172.21.15.63:0/3161629676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562645db8210 con 0x562645d7f4d0 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7eda00640 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645da01b0 0x562646158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d7f4d0 0x56264614d0f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400d740 tx=0x7fc7d40a8b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141df44da4ac9d30 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7cea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7cea00640 1 -- 172.21.15.63:0/3161629676 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4021050 con 0x562645d7f4d0 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7cea00640 1 -- 172.21.15.63:0/3161629676 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fc7d40a8660 con 0x562645d7f4d0 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.561+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3161629676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7dc0043d0 con 0x562645d7f4d0 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7cea00640 1 -- 172.21.15.63:0/3161629676 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc7d4010e20 con 0x562645d7f4d0 2024-05-27T07:36:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7f69fb740 10 client.15993 fetch_fsmap learned FSMap version 30 2024-05-27T07:36:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3161629676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562645e84620 con 0x562645d7f4d0 2024-05-27T07:36:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7cea00640 1 -- 172.21.15.63:0/3161629676 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7dc0043d0 con 0x562645d7f4d0 2024-05-27T07:36:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7cea00640 1 -- 172.21.15.63:0/3161629676 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc7d40ad150 con 0x562645d7f4d0 2024-05-27T07:36:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7f69fb740 10 client.15993 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:36:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7f69fb740 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7f69fb740 20 client.15993 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562645d5ed80 0x56264615ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7f69fb740 4 client.15993 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:36:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.562+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3161629676 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626460b5ae0 con 0x562645d5ed80 2024-05-27T07:36:38.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.563+0000 7fc7ed000640 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562645d5ed80 0x56264615ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.564+0000 7fc7ed000640 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562645d5ed80 0x56264615ef10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e80bb42d10a1d5e server_cookie=4519141cce8bb336 in_seq=0 out_seq=0 2024-05-27T07:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.564+0000 7fc7cea00640 10 client.15993 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.566+0000 7fc7cea00640 1 -- 172.21.15.63:0/3161629676 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5626460b5ae0 con 0x562645d5ed80 2024-05-27T07:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.566+0000 7fc7cea00640 10 client.15993 handle_command_reply: tid=0 2024-05-27T07:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.566+0000 7fc7f69fb740 1 client.15993 shutdown 2024-05-27T07:36:38.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.566+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3161629676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d7f4d0 msgr2=0x56264614d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.566+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d7f4d0 0x56264614d0f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400d740 tx=0x7fc7d40a8b40 comp rx=0 tx=0).stop 2024-05-27T07:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.566+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3161629676 shutdown_connections 2024-05-27T07:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.566+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562645d5ed80 0x56264615ef10 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.567+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645da01b0 0x562646158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.567+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d7f4d0 0x56264614d0f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.567+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/3161629676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d772f0 0x56264614cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.567+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3161629676 >> 172.21.15.63:0/3161629676 conn(0x562645d94300 msgr2=0x562646144780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.567+0000 7fc7cea00640 0 client.15993 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.567+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3161629676 shutdown_connections 2024-05-27T07:36:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.567+0000 7fc7f69fb740 1 -- 172.21.15.63:0/3161629676 wait complete. 2024-05-27T07:36:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.567+0000 7fc7f69fb740 20 client.15993 trim_cache size 0 max 16384 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.584+0000 7fc7f69fb740 1 Processor -- start 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.584+0000 7fc7f69fb740 1 -- start start 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.584+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d772f0 0x562646158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.584+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d7f4d0 0x562646159200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.584+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645da01b0 0x5626461646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.584+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c8012740 con 0x562645d772f0 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.584+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c8010010 con 0x562645d7f4d0 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.584+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c800de60 con 0x562645da01b0 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.584+0000 7fc7ed000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d772f0 0x562646158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.584+0000 7fc7ed000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d772f0 0x562646158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45010/0 (socket says 172.21.15.63:45010) 2024-05-27T07:36:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.585+0000 7fc7ed000640 1 -- 172.21.15.63:0/4092402375 learned_addr learned my addr 172.21.15.63:0/4092402375 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.585+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/4092402375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d7f4d0 0x562646159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.585+0000 7fc7eda00640 1 --2- 172.21.15.63:0/4092402375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645da01b0 0x5626461646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.585+0000 7fc7ed000640 1 -- 172.21.15.63:0/4092402375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645da01b0 msgr2=0x5626461646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.585+0000 7fc7ed000640 1 --2- 172.21.15.63:0/4092402375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645da01b0 0x5626461646d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.585+0000 7fc7ed000640 1 -- 172.21.15.63:0/4092402375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d7f4d0 msgr2=0x562646159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.585+0000 7fc7ed000640 1 --2- 172.21.15.63:0/4092402375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d7f4d0 0x562646159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.585+0000 7fc7ed000640 1 -- 172.21.15.63:0/4092402375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56264614b100 con 0x562645d772f0 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.585+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/4092402375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d7f4d0 0x562646159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.585+0000 7fc7ed000640 1 --2- 172.21.15.63:0/4092402375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d772f0 0x562646158cc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009250 tx=0x7fc7dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d0db9706c01fc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.586+0000 7fc7d9600640 1 -- 172.21.15.63:0/4092402375 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc004010 con 0x562645d772f0 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.586+0000 7fc7d9600640 1 -- 172.21.15.63:0/4092402375 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7dc003a80 con 0x562645d772f0 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.586+0000 7fc7d9600640 1 -- 172.21.15.63:0/4092402375 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc0052a0 con 0x562645d772f0 2024-05-27T07:36:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.587+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4092402375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d772f0 msgr2=0x562646158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.587+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/4092402375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d772f0 0x562646158cc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009250 tx=0x7fc7dc011810 comp rx=0 tx=0).stop 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.587+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4092402375 shutdown_connections 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.587+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/4092402375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645da01b0 0x5626461646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.587+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/4092402375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d7f4d0 0x562646159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.587+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/4092402375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645d772f0 0x562646158cc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.587+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4092402375 >> 172.21.15.63:0/4092402375 conn(0x562645d94300 msgr2=0x562646147390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.587+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4092402375 shutdown_connections 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.587+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4092402375 wait complete. 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.588+0000 7fc7f69fb740 1 -- start start 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.588+0000 7fc7f69fb740 1 Processor -- start 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.588+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 0x562646152010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.588+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 0x562646152550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.588+0000 7fc7f69fb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x5626461c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.588+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c800e4d0 con 0x562645da01b0 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.588+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c800f940 con 0x562645d772f0 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.588+0000 7fc7f69fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c800eba0 con 0x562645d7f4d0 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.588+0000 7fc7f69fb740 10 client.0 _ll_register_callbacks cb 0x7fc7d4009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.589+0000 7fc7eda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x5626461c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.589+0000 7fc7eda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x5626461c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45016/0 (socket says 172.21.15.63:45016) 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.589+0000 7fc7eda00640 1 -- 172.21.15.63:0/4248537658 learned_addr learned my addr 172.21.15.63:0/4248537658 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.589+0000 7fc7ed000640 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 0x562646152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.589+0000 7fc7e7e00640 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 0x562646152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.589+0000 7fc7eda00640 1 -- 172.21.15.63:0/4248537658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 msgr2=0x562646152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.589+0000 7fc7eda00640 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 0x562646152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.589+0000 7fc7eda00640 1 -- 172.21.15.63:0/4248537658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 msgr2=0x562646152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.589+0000 7fc7eda00640 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 0x562646152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.589+0000 7fc7eda00640 1 -- 172.21.15.63:0/4248537658 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562646152e30 con 0x562645da01b0 2024-05-27T07:36:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.590+0000 7fc7eda00640 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x5626461c55a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc7e002fae0 tx=0x7fc7e00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c0321e7937d518 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.590+0000 7fc7cea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.590+0000 7fc7cea00640 1 -- 172.21.15.63:0/4248537658 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0057060 con 0x562645da01b0 2024-05-27T07:36:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.590+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4248537658 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7dc003a80 con 0x562645da01b0 2024-05-27T07:36:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.590+0000 7fc7cea00640 1 -- 172.21.15.63:0/4248537658 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7fc7e0004450 con 0x562645da01b0 2024-05-27T07:36:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.590+0000 7fc7cea00640 1 -- 172.21.15.63:0/4248537658 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc7e005e050 con 0x562645da01b0 2024-05-27T07:36:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.590+0000 7fc7f69fb740 10 client.15999 fetch_fsmap learned FSMap version 30 2024-05-27T07:36:38.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.590+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4248537658 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626461c6af0 con 0x562645da01b0 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.591+0000 7fc7cea00640 1 -- 172.21.15.63:0/4248537658 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7dc003a80 con 0x562645da01b0 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.591+0000 7fc7cea00640 1 -- 172.21.15.63:0/4248537658 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc7e0051070 con 0x562645da01b0 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.591+0000 7fc7f69fb740 10 client.15999 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.591+0000 7fc7f69fb740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.591+0000 7fc7f69fb740 20 client.15999 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.591+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562645d341c0 0x5626461c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.591+0000 7fc7f69fb740 4 client.15999 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.591+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4248537658 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626460af2c0 con 0x562645d341c0 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.592+0000 7fc7ed000640 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562645d341c0 0x5626461c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.593+0000 7fc7cea00640 10 client.15999 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:38.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.593+0000 7fc7ed000640 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562645d341c0 0x5626461c9360 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=402410607ecf2423 server_cookie=1cc4ae56761107aa in_seq=0 out_seq=0 2024-05-27T07:36:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.644+0000 7fc7cea00640 1 -- 172.21.15.63:0/4248537658 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5626460af2c0 con 0x562645d341c0 2024-05-27T07:36:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.644+0000 7fc7cea00640 10 client.15999 handle_command_reply: tid=0 2024-05-27T07:36:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.644+0000 7fc7f69fb740 1 client.15999 shutdown 2024-05-27T07:36:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4248537658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 msgr2=0x5626461c55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x5626461c55a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc7e002fae0 tx=0x7fc7e00095c0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4248537658 shutdown_connections 2024-05-27T07:36:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562645d341c0 0x5626461c9360 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562645da01b0 0x5626461c55a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562645d7f4d0 0x562646152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 1 --2- 172.21.15.63:0/4248537658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562645d772f0 0x562646152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7cea00640 0 client.15999 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4248537658 >> 172.21.15.63:0/4248537658 conn(0x562645d94300 msgr2=0x5626461586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:38.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4248537658 shutdown_connections 2024-05-27T07:36:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 1 -- 172.21.15.63:0/4248537658 wait complete. 2024-05-27T07:36:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.645+0000 7fc7f69fb740 20 client.15999 trim_cache size 0 max 16384 2024-05-27T07:36:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:36:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:36:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:36:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 -- 172.21.15.63:0/560029707 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc7d00b29d0 msgr2=0x7fc7d00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc7d00b29d0 0x7fc7d00b4ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x56264613fbd0 tx=0x7fc7e0005c50 comp rx=0 tx=0).stop 2024-05-27T07:36:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 -- 172.21.15.63:0/560029707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e810d9c0 msgr2=0x7fc7e819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e810d9c0 0x7fc7e819cce0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400d770 tx=0x7fc7d400dc30 comp rx=0 tx=0).stop 2024-05-27T07:36:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 -- 172.21.15.63:0/560029707 shutdown_connections 2024-05-27T07:36:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc7d00b29d0 0x7fc7d00b4ee0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7e8110320 0x7fc7e81a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7e810d9c0 0x7fc7e819cce0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 --2- 172.21.15.63:0/560029707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7e8075420 0x7fc7e819c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 -- 172.21.15.63:0/560029707 >> 172.21.15.63:0/560029707 conn(0x7fc7e8079100 msgr2=0x7fc7e81022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 -- 172.21.15.63:0/560029707 shutdown_connections 2024-05-27T07:36:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:38.646+0000 7fc7ee400640 1 -- 172.21.15.63:0/560029707 wait complete. 2024-05-27T07:36:39.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:39 smithi086 ceph-mon[34687]: pgmap v95: 129 pgs: 129 active+clean; 40 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.1 MiB/s wr, 15 op/s 2024-05-27T07:36:39.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:39 smithi149 ceph-mon[35408]: pgmap v95: 129 pgs: 129 active+clean; 40 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.1 MiB/s wr, 15 op/s 2024-05-27T07:36:39.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:39 smithi063 ceph-mon[31613]: pgmap v95: 129 pgs: 129 active+clean; 40 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.1 MiB/s wr, 15 op/s 2024-05-27T07:36:40.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.458+0000 7f742e200700 1 -- 172.21.15.63:0/1070308097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428105e40 msgr2=0x7f742810e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.458+0000 7f742e200700 1 --2- 172.21.15.63:0/1070308097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428105e40 0x7f742810e390 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7414009990 tx=0x7f7414009ca0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.458+0000 7f742e200700 1 -- 172.21.15.63:0/1070308097 shutdown_connections 2024-05-27T07:36:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.458+0000 7f742e200700 1 --2- 172.21.15.63:0/1070308097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f742810e920 0x7f7428110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.458+0000 7f742e200700 1 --2- 172.21.15.63:0/1070308097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281054f0 0x7f7428105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.458+0000 7f742e200700 1 --2- 172.21.15.63:0/1070308097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7428105e40 0x7f742810e390 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.458+0000 7f742e200700 1 -- 172.21.15.63:0/1070308097 >> 172.21.15.63:0/1070308097 conn(0x7f74280fef20 msgr2=0x7f7428101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.458+0000 7f742e200700 1 -- 172.21.15.63:0/1070308097 shutdown_connections 2024-05-27T07:36:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.458+0000 7f742e200700 1 -- 172.21.15.63:0/1070308097 wait complete. 2024-05-27T07:36:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.460+0000 7f742e200700 1 Processor -- start 2024-05-27T07:36:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.460+0000 7f742e200700 1 -- start start 2024-05-27T07:36:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f742e200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281054f0 0x7f742819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f742e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428105e40 0x7f742819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f742e200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f742810e920 0x7f74281a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f742e200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7428114300 con 0x7f742810e920 2024-05-27T07:36:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f742e200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7428114180 con 0x7f7428105e40 2024-05-27T07:36:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f742e200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7428114480 con 0x7f74281054f0 2024-05-27T07:36:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f7427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f742810e920 0x7f74281a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f7427400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281054f0 0x7f742819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f7427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f742810e920 0x7f74281a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57254/0 (socket says 172.21.15.63:57254) 2024-05-27T07:36:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f7427e00700 1 -- 172.21.15.63:0/1462844350 learned_addr learned my addr 172.21.15.63:0/1462844350 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.461+0000 7f7426a00700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428105e40 0x7f742819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.462+0000 7f7427e00700 1 -- 172.21.15.63:0/1462844350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281054f0 msgr2=0x7f742819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.462+0000 7f7427e00700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281054f0 0x7f742819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.462+0000 7f7427e00700 1 -- 172.21.15.63:0/1462844350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428105e40 msgr2=0x7f742819d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.462+0000 7f7427e00700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428105e40 0x7f742819d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.462+0000 7f7427e00700 1 -- 172.21.15.63:0/1462844350 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74281a1e20 con 0x7f742810e920 2024-05-27T07:36:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.462+0000 7f7427e00700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f742810e920 0x7f74281a1780 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7420007ee0 tx=0x7f742000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.463+0000 7f741fe00700 1 -- 172.21.15.63:0/1462844350 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420004bb0 con 0x7f742810e920 2024-05-27T07:36:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.463+0000 7f742e200700 1 -- 172.21.15.63:0/1462844350 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74281a2110 con 0x7f742810e920 2024-05-27T07:36:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.463+0000 7f741fe00700 1 -- 172.21.15.63:0/1462844350 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7420004d50 con 0x7f742810e920 2024-05-27T07:36:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.463+0000 7f741fe00700 1 -- 172.21.15.63:0/1462844350 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200058b0 con 0x7f742810e920 2024-05-27T07:36:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.463+0000 7f742e200700 1 -- 172.21.15.63:0/1462844350 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74281a9a30 con 0x7f742810e920 2024-05-27T07:36:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.468+0000 7f743cd47b80 1 Processor -- start 2024-05-27T07:36:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.468+0000 7f741fe00700 1 -- 172.21.15.63:0/1462844350 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f742001a020 con 0x7f742810e920 2024-05-27T07:36:40.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.468+0000 7f743cd47b80 1 -- start start 2024-05-27T07:36:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.469+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e647fbbe40 0x55e6481407f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.469+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e648140d30 0x55e64813a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.469+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e64813b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.469+0000 7f743cd47b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e648068e40 con 0x55e647fbbe40 2024-05-27T07:36:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.469+0000 7f743cd47b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6480589e0 con 0x55e64813adb0 2024-05-27T07:36:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.469+0000 7f743cd47b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e64805ca20 con 0x55e648140d30 2024-05-27T07:36:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.470+0000 7f7426a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e647fbbe40 0x55e6481407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.470+0000 7f7426a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e647fbbe40 0x55e6481407f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57262/0 (socket says 172.21.15.63:57262) 2024-05-27T07:36:40.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.470+0000 7f7426a00700 1 -- 172.21.15.63:0/772617525 learned_addr learned my addr 172.21.15.63:0/772617525 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:40.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.470+0000 7f7427400700 1 --2- 172.21.15.63:0/772617525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e648140d30 0x55e64813a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.470+0000 7f7426a00700 1 --2- 172.21.15.63:0/772617525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e64813b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.470+0000 7f7426a00700 1 -- 172.21.15.63:0/772617525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e648140d30 msgr2=0x55e64813a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.470+0000 7f7426a00700 1 --2- 172.21.15.63:0/772617525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e648140d30 0x55e64813a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.470+0000 7f7426a00700 1 -- 172.21.15.63:0/772617525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 msgr2=0x55e64813b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.470+0000 7f7426a00700 1 --2- 172.21.15.63:0/772617525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e64813b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.470+0000 7f7426a00700 1 -- 172.21.15.63:0/772617525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e648021420 con 0x55e647fbbe40 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.471+0000 7f7426a00700 1 --2- 172.21.15.63:0/772617525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e647fbbe40 0x55e6481407f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7414009990 tx=0x7f741401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6919a5550105fd9f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.471+0000 7f741fe00700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b29f0 0x7f740c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.471+0000 7f741fe00700 1 -- 172.21.15.63:0/1462844350 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f74200e1ff0 con 0x7f742810e920 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.471+0000 7f741cc00700 1 -- 172.21.15.63:0/772617525 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741400bbb0 con 0x55e647fbbe40 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.471+0000 7f741cc00700 1 -- 172.21.15.63:0/772617525 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74140055a0 con 0x55e647fbbe40 2024-05-27T07:36:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.471+0000 7f741cc00700 1 -- 172.21.15.63:0/772617525 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414003dc0 con 0x55e647fbbe40 2024-05-27T07:36:40.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.471+0000 7f7427400700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b29f0 0x7f740c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.472+0000 7f743cd47b80 1 -- 172.21.15.63:0/772617525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e647fbbe40 msgr2=0x55e6481407f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:40.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.472+0000 7f743cd47b80 1 --2- 172.21.15.63:0/772617525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e647fbbe40 0x55e6481407f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7414009990 tx=0x7f741401a040 comp rx=0 tx=0).stop 2024-05-27T07:36:40.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.472+0000 7f7427400700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b29f0 0x7f740c0b4f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7418005950 tx=0x7f741800b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:40.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.472+0000 7f743cd47b80 1 -- 172.21.15.63:0/772617525 shutdown_connections 2024-05-27T07:36:40.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.472+0000 7f743cd47b80 1 --2- 172.21.15.63:0/772617525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e64813b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.472+0000 7f743cd47b80 1 --2- 172.21.15.63:0/772617525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e647fbbe40 0x55e6481407f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.472+0000 7f743cd47b80 1 --2- 172.21.15.63:0/772617525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e648140d30 0x55e64813a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.472+0000 7f743cd47b80 1 -- 172.21.15.63:0/772617525 >> 172.21.15.63:0/772617525 conn(0x55e64801bfc0 msgr2=0x55e6481358d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:40.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.473+0000 7f743cd47b80 1 -- 172.21.15.63:0/772617525 shutdown_connections 2024-05-27T07:36:40.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.473+0000 7f743cd47b80 1 -- 172.21.15.63:0/772617525 wait complete. 2024-05-27T07:36:40.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.473+0000 7f743cd47b80 1 -- start start 2024-05-27T07:36:40.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.474+0000 7f743cd47b80 1 Processor -- start 2024-05-27T07:36:40.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.474+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e647fbbe40 0x55e6481445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.474+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e648144ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.474+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e648140d30 0x55e648145020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.474+0000 7f743cd47b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e647f5efa0 con 0x55e648140d30 2024-05-27T07:36:40.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.474+0000 7f743cd47b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e647ff9760 con 0x55e64813adb0 2024-05-27T07:36:40.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.474+0000 7f743cd47b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e6481477a0 con 0x55e647fbbe40 2024-05-27T07:36:40.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.474+0000 7f743cd47b80 10 client.0 _ll_register_callbacks cb 0x55e648002c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:40.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.475+0000 7f7426a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e648144ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.475+0000 7f7426a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e648144ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:36496/0 (socket says 172.21.15.63:36496) 2024-05-27T07:36:40.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.475+0000 7f7427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e648140d30 0x55e648145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.475+0000 7f7426a00700 1 -- 172.21.15.63:0/762573422 learned_addr learned my addr 172.21.15.63:0/762573422 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:40.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.475+0000 7f7427400700 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e647fbbe40 0x55e6481445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.476+0000 7f7427e00700 1 -- 172.21.15.63:0/762573422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e647fbbe40 msgr2=0x55e6481445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.476+0000 7f7427e00700 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e647fbbe40 0x55e6481445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.476+0000 7f7427e00700 1 -- 172.21.15.63:0/762573422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 msgr2=0x55e648144ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.476+0000 7f7427e00700 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e648144ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.476+0000 7f7427e00700 1 -- 172.21.15.63:0/762573422 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e647fa88d0 con 0x55e648140d30 2024-05-27T07:36:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.476+0000 7f7427e00700 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e648140d30 0x55e648145020 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f74200df3d0 tx=0x7f7420005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614ddacef77e2e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.476+0000 7f7411600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.476+0000 7f7411600700 1 -- 172.21.15.63:0/762573422 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200a9660 con 0x55e648140d30 2024-05-27T07:36:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.476+0000 7f7411600700 1 -- 172.21.15.63:0/762573422 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f74200a91c0 con 0x55e648140d30 2024-05-27T07:36:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.476+0000 7f743cd47b80 1 -- 172.21.15.63:0/762573422 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74140055a0 con 0x55e648140d30 2024-05-27T07:36:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.477+0000 7f7411600700 1 -- 172.21.15.63:0/762573422 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74200a8f60 con 0x55e648140d30 2024-05-27T07:36:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.477+0000 7f743cd47b80 10 client.16011 fetch_fsmap learned FSMap version 30 2024-05-27T07:36:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.477+0000 7f743cd47b80 1 -- 172.21.15.63:0/762573422 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e648029060 con 0x55e648140d30 2024-05-27T07:36:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.477+0000 7f7411600700 1 -- 172.21.15.63:0/762573422 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74140055a0 con 0x55e648140d30 2024-05-27T07:36:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.477+0000 7f7411600700 1 -- 172.21.15.63:0/762573422 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f74200ac070 con 0x55e648140d30 2024-05-27T07:36:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.477+0000 7f743cd47b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:36:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.477+0000 7f743cd47b80 10 client.16011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:36:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.477+0000 7f743cd47b80 20 client.16011 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.478+0000 7f743cd47b80 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e6481546e0 0x55e648156b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.478+0000 7f743cd47b80 4 client.16011 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:36:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.478+0000 7f743cd47b80 1 -- 172.21.15.63:0/762573422 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e64806ad90 con 0x55e6481546e0 2024-05-27T07:36:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.478+0000 7f7427400700 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e6481546e0 0x55e648156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.479+0000 7f7411600700 10 client.16011 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:36:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.479+0000 7f7427400700 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e6481546e0 0x55e648156b20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab2afe2efa82435a server_cookie=21dc77671ad4dece in_seq=0 out_seq=0 2024-05-27T07:36:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.481+0000 7f7411600700 1 -- 172.21.15.63:0/762573422 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e64806ad90 con 0x55e6481546e0 2024-05-27T07:36:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.481+0000 7f7411600700 10 client.16011 handle_command_reply: tid=0 2024-05-27T07:36:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 client.16011 shutdown 2024-05-27T07:36:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 -- 172.21.15.63:0/762573422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e648140d30 msgr2=0x55e648145020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e648140d30 0x55e648145020 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f74200df3d0 tx=0x7f7420005250 comp rx=0 tx=0).stop 2024-05-27T07:36:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 -- 172.21.15.63:0/762573422 shutdown_connections 2024-05-27T07:36:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e6481546e0 0x55e648156b20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e648140d30 0x55e648145020 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e647fbbe40 0x55e6481445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 --2- 172.21.15.63:0/762573422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e648144ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 -- 172.21.15.63:0/762573422 >> 172.21.15.63:0/762573422 conn(0x55e64801bfc0 msgr2=0x55e648135580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f7411600700 0 client.16011 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:36:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 -- 172.21.15.63:0/762573422 shutdown_connections 2024-05-27T07:36:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.482+0000 7f743cd47b80 1 -- 172.21.15.63:0/762573422 wait complete. 2024-05-27T07:36:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.483+0000 7f743cd47b80 20 client.16011 trim_cache size 0 max 16384 2024-05-27T07:36:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.503+0000 7f743cd47b80 1 Processor -- start 2024-05-27T07:36:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.503+0000 7f743cd47b80 1 -- start start 2024-05-27T07:36:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.504+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64813a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.504+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e647fbbe40 0x55e64813fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.504+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e64813eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.504+0000 7f743cd47b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b4e70 con 0x55e64801bfc0 2024-05-27T07:36:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.504+0000 7f743cd47b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b2700 con 0x55e64813adb0 2024-05-27T07:36:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.504+0000 7f743cd47b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b0550 con 0x55e647fbbe40 2024-05-27T07:36:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.504+0000 7f7427400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64813a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.504+0000 7f7427400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64813a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57292/0 (socket says 172.21.15.63:57292) 2024-05-27T07:36:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.504+0000 7f7427400700 1 -- 172.21.15.63:0/917471897 learned_addr learned my addr 172.21.15.63:0/917471897 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:40.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.505+0000 7f7427e00700 1 --2- 172.21.15.63:0/917471897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e64813eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.505+0000 7f7426a00700 1 --2- 172.21.15.63:0/917471897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e647fbbe40 0x55e64813fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.505+0000 7f7427400700 1 -- 172.21.15.63:0/917471897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e647fbbe40 msgr2=0x55e64813fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:40.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.505+0000 7f7427400700 1 --2- 172.21.15.63:0/917471897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e647fbbe40 0x55e64813fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.505+0000 7f7427400700 1 -- 172.21.15.63:0/917471897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 msgr2=0x55e64813eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.505+0000 7f7427400700 1 --2- 172.21.15.63:0/917471897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e64813eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.505+0000 7f7427400700 1 -- 172.21.15.63:0/917471897 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e648143ef0 con 0x55e64801bfc0 2024-05-27T07:36:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.505+0000 7f7427e00700 1 --2- 172.21.15.63:0/917471897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e64813eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.505+0000 7f7427400700 1 --2- 172.21.15.63:0/917471897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64813a870 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7418003d20 tx=0x7f7418019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55908e7eb4b58c14 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.506+0000 7f7413e00700 1 -- 172.21.15.63:0/917471897 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741801e9c0 con 0x55e64801bfc0 2024-05-27T07:36:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.506+0000 7f7413e00700 1 -- 172.21.15.63:0/917471897 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f741802f070 con 0x55e64801bfc0 2024-05-27T07:36:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.506+0000 7f7413e00700 1 -- 172.21.15.63:0/917471897 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741801d670 con 0x55e64801bfc0 2024-05-27T07:36:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.507+0000 7f743cd47b80 1 -- 172.21.15.63:0/917471897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 msgr2=0x55e64813a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.507+0000 7f743cd47b80 1 --2- 172.21.15.63:0/917471897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64813a870 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7418003d20 tx=0x7f7418019000 comp rx=0 tx=0).stop 2024-05-27T07:36:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.507+0000 7f743cd47b80 1 -- 172.21.15.63:0/917471897 shutdown_connections 2024-05-27T07:36:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.507+0000 7f743cd47b80 1 --2- 172.21.15.63:0/917471897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e64813adb0 0x55e64813eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.507+0000 7f743cd47b80 1 --2- 172.21.15.63:0/917471897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64813a870 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.507+0000 7f743cd47b80 1 --2- 172.21.15.63:0/917471897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e647fbbe40 0x55e64813fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.507+0000 7f743cd47b80 1 -- 172.21.15.63:0/917471897 >> 172.21.15.63:0/917471897 conn(0x55e648156e10 msgr2=0x55e648144380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.507+0000 7f743cd47b80 1 -- 172.21.15.63:0/917471897 shutdown_connections 2024-05-27T07:36:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.507+0000 7f743cd47b80 1 -- 172.21.15.63:0/917471897 wait complete. 2024-05-27T07:36:40.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.509+0000 7f743cd47b80 1 -- start start 2024-05-27T07:36:40.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.509+0000 7f743cd47b80 1 Processor -- start 2024-05-27T07:36:40.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.509+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e647fbbe40 0x55e64814cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.509+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64814d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.509+0000 7f743cd47b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e64813adb0 0x55e64815b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.509+0000 7f743cd47b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b0bc0 con 0x55e64801bfc0 2024-05-27T07:36:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.509+0000 7f743cd47b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b2030 con 0x55e647fbbe40 2024-05-27T07:36:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.509+0000 7f743cd47b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74281b1290 con 0x55e64813adb0 2024-05-27T07:36:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.509+0000 7f743cd47b80 10 client.0 _ll_register_callbacks cb 0x55e648002c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7426a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64814d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7426a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64814d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57298/0 (socket says 172.21.15.63:57298) 2024-05-27T07:36:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7426a00700 1 -- 172.21.15.63:0/55651600 learned_addr learned my addr 172.21.15.63:0/55651600 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7427400700 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e647fbbe40 0x55e64814cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7427e00700 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e64813adb0 0x55e64815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7426a00700 1 -- 172.21.15.63:0/55651600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e64813adb0 msgr2=0x55e64815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7426a00700 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e64813adb0 0x55e64815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7426a00700 1 -- 172.21.15.63:0/55651600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e647fbbe40 msgr2=0x55e64814cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7426a00700 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e647fbbe40 0x55e64814cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7426a00700 1 -- 172.21.15.63:0/55651600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e64815bee0 con 0x55e64801bfc0 2024-05-27T07:36:40.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7427400700 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e647fbbe40 0x55e64814cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:40.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.510+0000 7f7427e00700 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e64813adb0 0x55e64815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:36:40.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.511+0000 7f7426a00700 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64814d130 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7414005d90 tx=0x7f74140053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac5bba28c2d13a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:40.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.511+0000 7f7411600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:40.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.511+0000 7f7411600700 1 -- 172.21.15.63:0/55651600 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741400b540 con 0x55e64801bfc0 2024-05-27T07:36:40.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.511+0000 7f743cd47b80 1 -- 172.21.15.63:0/55651600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f741802f070 con 0x55e64801bfc0 2024-05-27T07:36:40.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.512+0000 7f7411600700 1 -- 172.21.15.63:0/55651600 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9395+0+0 (secure 0 0 0) 0x7f7414003830 con 0x55e64801bfc0 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.512+0000 7f7411600700 1 -- 172.21.15.63:0/55651600 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f741403d050 con 0x55e64801bfc0 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.512+0000 7f743cd47b80 10 client.16017 fetch_fsmap learned FSMap version 30 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.512+0000 7f743cd47b80 1 -- 172.21.15.63:0/55651600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6481bec20 con 0x55e64801bfc0 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.513+0000 7f7411600700 1 -- 172.21.15.63:0/55651600 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f741802f070 con 0x55e64801bfc0 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.513+0000 7f7411600700 1 -- 172.21.15.63:0/55651600 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f741400fa10 con 0x55e64801bfc0 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.513+0000 7f743cd47b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.513+0000 7f743cd47b80 10 client.16017 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.513+0000 7f743cd47b80 20 client.16017 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.513+0000 7f743cd47b80 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e648140d30 0x55e6481c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.513+0000 7f743cd47b80 4 client.16017 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:36:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.513+0000 7f743cd47b80 1 -- 172.21.15.63:0/55651600 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e648064570 con 0x55e648140d30 2024-05-27T07:36:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.514+0000 7f7427400700 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e648140d30 0x55e6481c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.515+0000 7f7427400700 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e648140d30 0x55e6481c0e60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2625aa704d1b3ecd server_cookie=c909079c3b8465cb in_seq=0 out_seq=0 2024-05-27T07:36:40.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.515+0000 7f7411600700 10 client.16017 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:36:40.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.515+0000 7f7411600700 1 -- 172.21.15.63:0/55651600 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e648064570 con 0x55e648140d30 2024-05-27T07:36:40.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f7411600700 10 client.16017 handle_command_reply: tid=0 2024-05-27T07:36:40.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 client.16017 shutdown 2024-05-27T07:36:40.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 -- 172.21.15.63:0/55651600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 msgr2=0x55e64814d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:40.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64814d130 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7414005d90 tx=0x7f74140053f0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 -- 172.21.15.63:0/55651600 shutdown_connections 2024-05-27T07:36:40.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e64813adb0 0x55e64815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e647fbbe40 0x55e64814cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e648140d30 0x55e6481c0e60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 --2- 172.21.15.63:0/55651600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e64801bfc0 0x55e64814d130 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 -- 172.21.15.63:0/55651600 >> 172.21.15.63:0/55651600 conn(0x55e648156e10 msgr2=0x55e64813e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:40.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f7411600700 0 client.16017 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:36:40.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 -- 172.21.15.63:0/55651600 shutdown_connections 2024-05-27T07:36:40.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.516+0000 7f743cd47b80 1 -- 172.21.15.63:0/55651600 wait complete. 2024-05-27T07:36:40.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.517+0000 7f743cd47b80 20 client.16017 trim_cache size 0 max 16384 2024-05-27T07:36:40.515 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:36:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.519+0000 7f742e200700 1 -- 172.21.15.63:0/1462844350 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b29f0 msgr2=0x7f740c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.519+0000 7f742e200700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b29f0 0x7f740c0b4f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7418005950 tx=0x7f741800b410 comp rx=0 tx=0).stop 2024-05-27T07:36:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.519+0000 7f742e200700 1 -- 172.21.15.63:0/1462844350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f742810e920 msgr2=0x7f74281a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.519+0000 7f742e200700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f742810e920 0x7f74281a1780 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7420007ee0 tx=0x7f742000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.520+0000 7f742e200700 1 -- 172.21.15.63:0/1462844350 shutdown_connections 2024-05-27T07:36:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.520+0000 7f742e200700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f740c0b29f0 0x7f740c0b4f00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.520+0000 7f742e200700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f742810e920 0x7f74281a1780 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.520+0000 7f742e200700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74281054f0 0x7f742819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.520+0000 7f742e200700 1 --2- 172.21.15.63:0/1462844350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7428105e40 0x7f742819d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:40.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.520+0000 7f742e200700 1 -- 172.21.15.63:0/1462844350 >> 172.21.15.63:0/1462844350 conn(0x7f74280fef20 msgr2=0x7f7428103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:40.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.520+0000 7f742e200700 1 -- 172.21.15.63:0/1462844350 shutdown_connections 2024-05-27T07:36:40.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:40.520+0000 7f742e200700 1 -- 172.21.15.63:0/1462844350 wait complete. 2024-05-27T07:36:41.387 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:36:41.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:41 smithi086 ceph-mon[34687]: pgmap v96: 129 pgs: 129 active+clean; 73 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.8 MiB/s wr, 27 op/s 2024-05-27T07:36:41.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:41 smithi149 ceph-mon[35408]: pgmap v96: 129 pgs: 129 active+clean; 73 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.8 MiB/s wr, 27 op/s 2024-05-27T07:36:41.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:41 smithi063 ceph-mon[31613]: pgmap v96: 129 pgs: 129 active+clean; 73 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.8 MiB/s wr, 27 op/s 2024-05-27T07:36:41.903 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:36:42.368 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:41 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-05-27T07:36:42.368 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:41 smithi149 systemd[1]: Stopped Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:36:42.368 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:41 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 1.888s CPU time. 2024-05-27T07:36:42.368 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:41 smithi149 systemd[1]: Starting Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:36:42.674 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:42 smithi149 podman[70520]: 2024-05-27 07:36:42.27934535 +0000 UTC m=+0.039013889 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:36:42.674 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:42 smithi149 podman[70520]: 2024-05-27 07:36:42.395543232 +0000 UTC m=+0.155211787 container create 426d5dd01d27061066b5c1742dafd99354fc93119c58a2c5f821900c7e725d17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-27T07:36:42.674 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:42 smithi149 podman[70520]: 2024-05-27 07:36:42.677948266 +0000 UTC m=+0.437616804 container init 426d5dd01d27061066b5c1742dafd99354fc93119c58a2c5f821900c7e725d17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-27T07:36:42.958 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:42 smithi149 ceph-mon[35408]: daemon mds.f restarted 2024-05-27T07:36:42.959 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:42 smithi149 podman[70520]: 2024-05-27 07:36:42.688269759 +0000 UTC m=+0.447938285 container start 426d5dd01d27061066b5c1742dafd99354fc93119c58a2c5f821900c7e725d17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:36:42.959 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:42 smithi149 bash[70520]: 426d5dd01d27061066b5c1742dafd99354fc93119c58a2c5f821900c7e725d17 2024-05-27T07:36:42.959 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:42 smithi149 systemd[1]: Started Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:36:42.959 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:42 smithi149 ceph-mds[70538]: -- 172.21.15.149:0/352877170 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559260dcba40 con 0x559260e13a80 2024-05-27T07:36:42.959 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 07:36:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: starting mds.f at 2024-05-27T07:36:43.204 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:42 smithi086 ceph-mon[34687]: daemon mds.f restarted 2024-05-27T07:36:43.211 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:42 smithi063 ceph-mon[31613]: daemon mds.f restarted 2024-05-27T07:36:44.031 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:43 smithi063 ceph-mon[31613]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T07:36:44.032 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:43 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/1954751988,v1:172.21.15.149:6835/1954751988] up:boot 2024-05-27T07:36:44.032 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:43 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T07:36:44.032 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:43 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:36:44.032 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:43 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:36:44.032 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:43 smithi063 ceph-mon[31613]: pgmap v98: 129 pgs: 129 active+clean; 73 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:44.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:43 smithi086 ceph-mon[34687]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T07:36:44.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:43 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/1954751988,v1:172.21.15.149:6835/1954751988] up:boot 2024-05-27T07:36:44.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:43 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T07:36:44.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:43 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:36:44.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:43 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:36:44.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:43 smithi086 ceph-mon[34687]: pgmap v98: 129 pgs: 129 active+clean; 73 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:44.208 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:43 smithi149 ceph-mon[35408]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T07:36:44.208 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:43 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/1954751988,v1:172.21.15.149:6835/1954751988] up:boot 2024-05-27T07:36:44.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:43 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T07:36:44.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:43 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:36:44.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:43 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:36:44.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:43 smithi149 ceph-mon[35408]: pgmap v98: 129 pgs: 129 active+clean; 73 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:44.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.421+0000 7f13c7200700 1 -- 172.21.15.63:0/2383655964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0105c10 msgr2=0x7f13c010e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:44.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.421+0000 7f13c7200700 1 --2- 172.21.15.63:0/2383655964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0105c10 0x7f13c010e160 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009990 tx=0x7f13b0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.421+0000 7f13c7200700 1 -- 172.21.15.63:0/2383655964 shutdown_connections 2024-05-27T07:36:44.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.421+0000 7f13c7200700 1 --2- 172.21.15.63:0/2383655964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c010e730 0x7f13c0110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.421+0000 7f13c7200700 1 --2- 172.21.15.63:0/2383655964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c01052c0 0x7f13c01056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.421+0000 7f13c7200700 1 --2- 172.21.15.63:0/2383655964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0105c10 0x7f13c010e160 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.421+0000 7f13c7200700 1 -- 172.21.15.63:0/2383655964 >> 172.21.15.63:0/2383655964 conn(0x7f13c0078af0 msgr2=0x7f13c0078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:44.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.421+0000 7f13c7200700 1 -- 172.21.15.63:0/2383655964 shutdown_connections 2024-05-27T07:36:44.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.421+0000 7f13c7200700 1 -- 172.21.15.63:0/2383655964 wait complete. 2024-05-27T07:36:44.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.423+0000 7f13c7200700 1 Processor -- start 2024-05-27T07:36:44.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.423+0000 7f13c7200700 1 -- start start 2024-05-27T07:36:44.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c7200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c01052c0 0x7f13c019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c7200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0105c10 0x7f13c019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c010e730 0x7f13c01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c7200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0114170 con 0x7f13c0105c10 2024-05-27T07:36:44.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0113ff0 con 0x7f13c010e730 2024-05-27T07:36:44.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c7200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01142f0 con 0x7f13c01052c0 2024-05-27T07:36:44.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0105c10 0x7f13c019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c01052c0 0x7f13c019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0105c10 0x7f13c019d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57316/0 (socket says 172.21.15.63:57316) 2024-05-27T07:36:44.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c01052c0 0x7f13c019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:52356/0 (socket says 172.21.15.63:52356) 2024-05-27T07:36:44.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c5400700 1 -- 172.21.15.63:0/888020049 learned_addr learned my addr 172.21.15.63:0/888020049 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:44.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.424+0000 7f13c6800700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c010e730 0x7f13c01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.425+0000 7f13c5400700 1 -- 172.21.15.63:0/888020049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c01052c0 msgr2=0x7f13c019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:44.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.425+0000 7f13c5400700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c01052c0 0x7f13c019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.425+0000 7f13c5400700 1 -- 172.21.15.63:0/888020049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c010e730 msgr2=0x7f13c01a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:44.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.425+0000 7f13c5400700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c010e730 0x7f13c01a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.425+0000 7f13c5400700 1 -- 172.21.15.63:0/888020049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c01a1bc0 con 0x7f13c0105c10 2024-05-27T07:36:44.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.426+0000 7f13c5400700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0105c10 0x7f13c019d340 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f13b00047b0 tx=0x7f13b000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:44.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.426+0000 7f13baa00700 1 -- 172.21.15.63:0/888020049 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0026070 con 0x7f13c0105c10 2024-05-27T07:36:44.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.426+0000 7f13c7200700 1 -- 172.21.15.63:0/888020049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c01a1e50 con 0x7f13c0105c10 2024-05-27T07:36:44.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.426+0000 7f13baa00700 1 -- 172.21.15.63:0/888020049 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13b0021410 con 0x7f13c0105c10 2024-05-27T07:36:44.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.426+0000 7f13baa00700 1 -- 172.21.15.63:0/888020049 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0017650 con 0x7f13c0105c10 2024-05-27T07:36:44.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.427+0000 7f13c7200700 1 -- 172.21.15.63:0/888020049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c01a2380 con 0x7f13c0105c10 2024-05-27T07:36:44.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.433+0000 7f13baa00700 1 -- 172.21.15.63:0/888020049 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f13b0009d10 con 0x7f13c0105c10 2024-05-27T07:36:44.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.434+0000 7f13d72bab80 1 Processor -- start 2024-05-27T07:36:44.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.434+0000 7f13d72bab80 1 -- start start 2024-05-27T07:36:44.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.434+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4905e40 0x556bd4a83ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.434+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4a84410 0x556bd4a8ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.434+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84820 0x556bd4a91020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.434+0000 7f13d72bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556bd49b2e40 con 0x556bd4a84820 2024-05-27T07:36:44.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.434+0000 7f13d72bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556bd49a29e0 con 0x556bd4a84410 2024-05-27T07:36:44.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.434+0000 7f13d72bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556bd49a6a20 con 0x556bd4905e40 2024-05-27T07:36:44.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.435+0000 7f13c5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84820 0x556bd4a91020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.435+0000 7f13c5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84820 0x556bd4a91020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57318/0 (socket says 172.21.15.63:57318) 2024-05-27T07:36:44.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.435+0000 7f13c5400700 1 -- 172.21.15.63:0/558824946 learned_addr learned my addr 172.21.15.63:0/558824946 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:44.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.435+0000 7f13c5e00700 1 --2- 172.21.15.63:0/558824946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4a84410 0x556bd4a8ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.435+0000 7f13c6800700 1 --2- 172.21.15.63:0/558824946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4905e40 0x556bd4a83ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.435+0000 7f13baa00700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13a80b29f0 0x7f13a80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.436+0000 7f13c5400700 1 -- 172.21.15.63:0/558824946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4905e40 msgr2=0x556bd4a83ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:44.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.436+0000 7f13c5400700 1 --2- 172.21.15.63:0/558824946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4905e40 0x556bd4a83ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.436+0000 7f13c5400700 1 -- 172.21.15.63:0/558824946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4a84410 msgr2=0x556bd4a8ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:44.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.436+0000 7f13c5400700 1 --2- 172.21.15.63:0/558824946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4a84410 0x556bd4a8ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.436+0000 7f13c5400700 1 -- 172.21.15.63:0/558824946 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bd496b420 con 0x556bd4a84820 2024-05-27T07:36:44.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.436+0000 7f13baa00700 1 -- 172.21.15.63:0/888020049 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f13b00e3780 con 0x7f13c0105c10 2024-05-27T07:36:44.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.436+0000 7f13c6800700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13a80b29f0 0x7f13a80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.437+0000 7f13c5400700 1 --2- 172.21.15.63:0/558824946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84820 0x556bd4a91020 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f13b00e0b50 tx=0x7f13b006fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d079f5caac0b541 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:44.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.437+0000 7f13af400700 1 -- 172.21.15.63:0/558824946 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b00aa180 con 0x556bd4a84820 2024-05-27T07:36:44.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.437+0000 7f13c6800700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13a80b29f0 0x7f13a80b4f00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x556bd4a80690 tx=0x7f13bc009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:44.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.437+0000 7f13af400700 1 -- 172.21.15.63:0/558824946 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13b00095d0 con 0x556bd4a84820 2024-05-27T07:36:44.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.437+0000 7f13af400700 1 -- 172.21.15.63:0/558824946 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b00a9fa0 con 0x556bd4a84820 2024-05-27T07:36:44.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.438+0000 7f13d72bab80 1 -- 172.21.15.63:0/558824946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84820 msgr2=0x556bd4a91020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:44.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.438+0000 7f13d72bab80 1 --2- 172.21.15.63:0/558824946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84820 0x556bd4a91020 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f13b00e0b50 tx=0x7f13b006fd20 comp rx=0 tx=0).stop 2024-05-27T07:36:44.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.438+0000 7f13d72bab80 1 -- 172.21.15.63:0/558824946 shutdown_connections 2024-05-27T07:36:44.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.438+0000 7f13d72bab80 1 --2- 172.21.15.63:0/558824946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84820 0x556bd4a91020 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.438+0000 7f13d72bab80 1 --2- 172.21.15.63:0/558824946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4905e40 0x556bd4a83ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.438+0000 7f13d72bab80 1 --2- 172.21.15.63:0/558824946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4a84410 0x556bd4a8ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.438+0000 7f13d72bab80 1 -- 172.21.15.63:0/558824946 >> 172.21.15.63:0/558824946 conn(0x556bd4965fc0 msgr2=0x556bd4a7fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:44.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.438+0000 7f13d72bab80 1 -- 172.21.15.63:0/558824946 shutdown_connections 2024-05-27T07:36:44.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.438+0000 7f13d72bab80 1 -- 172.21.15.63:0/558824946 wait complete. 2024-05-27T07:36:44.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.439+0000 7f13d72bab80 1 -- start start 2024-05-27T07:36:44.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.439+0000 7f13d72bab80 1 Processor -- start 2024-05-27T07:36:44.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.440+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4905e40 0x556bd4a8e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.440+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4a84410 0x556bd4a8ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.440+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4a84820 0x556bd4a8ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.440+0000 7f13d72bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556bd48a8fa0 con 0x556bd4905e40 2024-05-27T07:36:44.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.440+0000 7f13d72bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556bd4943760 con 0x556bd4a84410 2024-05-27T07:36:44.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.440+0000 7f13d72bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556bd49b2e40 con 0x556bd4a84820 2024-05-27T07:36:44.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.440+0000 7f13d72bab80 10 client.0 _ll_register_callbacks cb 0x556bd494cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:44.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.440+0000 7f13c5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4905e40 0x556bd4a8e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.440+0000 7f13c5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4905e40 0x556bd4a8e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57330/0 (socket says 172.21.15.63:57330) 2024-05-27T07:36:44.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.440+0000 7f13c5e00700 1 -- 172.21.15.63:0/1472993574 learned_addr learned my addr 172.21.15.63:0/1472993574 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:44.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.441+0000 7f13c5400700 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4a84410 0x556bd4a8ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.441+0000 7f13c5e00700 1 -- 172.21.15.63:0/1472993574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4a84820 msgr2=0x556bd4a8ef70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:36:44.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.441+0000 7f13c5e00700 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4a84820 0x556bd4a8ef70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.441+0000 7f13c5e00700 1 -- 172.21.15.63:0/1472993574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4a84410 msgr2=0x556bd4a8ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:44.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.441+0000 7f13c5e00700 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4a84410 0x556bd4a8ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.441+0000 7f13c5e00700 1 -- 172.21.15.63:0/1472993574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bd48f28d0 con 0x556bd4905e40 2024-05-27T07:36:44.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.441+0000 7f13c5e00700 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4905e40 0x556bd4a8e4f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f13b4010a50 tx=0x7f13b4010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34a5ea041eef587 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:44.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.441+0000 7f13a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:44.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.442+0000 7f13a3e00700 1 -- 172.21.15.63:0/1472993574 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4004260 con 0x556bd4905e40 2024-05-27T07:36:44.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.442+0000 7f13a3e00700 1 -- 172.21.15.63:0/1472993574 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f13b4004400 con 0x556bd4905e40 2024-05-27T07:36:44.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.442+0000 7f13d72bab80 1 -- 172.21.15.63:0/1472993574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b00095d0 con 0x556bd4905e40 2024-05-27T07:36:44.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.442+0000 7f13a3e00700 1 -- 172.21.15.63:0/1472993574 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f13b400f890 con 0x556bd4905e40 2024-05-27T07:36:44.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.442+0000 7f13d72bab80 10 client.16029 fetch_fsmap learned FSMap version 32 2024-05-27T07:36:44.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.442+0000 7f13d72bab80 1 -- 172.21.15.63:0/1472993574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556bd4973060 con 0x556bd4905e40 2024-05-27T07:36:44.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.443+0000 7f13a3e00700 1 -- 172.21.15.63:0/1472993574 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13b00095d0 con 0x556bd4905e40 2024-05-27T07:36:44.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.443+0000 7f13a3e00700 1 -- 172.21.15.63:0/1472993574 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f13b4003c40 con 0x556bd4905e40 2024-05-27T07:36:44.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.443+0000 7f13d72bab80 10 client.16029 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:36:44.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.443+0000 7f13d72bab80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:44.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.443+0000 7f13d72bab80 20 client.16029 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:44.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.443+0000 7f13d72bab80 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556bd4a9e550 0x556bd4aa0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.443+0000 7f13d72bab80 4 client.16029 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:36:44.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.443+0000 7f13d72bab80 1 -- 172.21.15.63:0/1472993574 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bd49b4d90 con 0x556bd4a9e550 2024-05-27T07:36:44.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.444+0000 7f13c5400700 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556bd4a9e550 0x556bd4aa0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.445+0000 7f13c5400700 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556bd4a9e550 0x556bd4aa0a60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fdbfbd70776d799 server_cookie=5dc80687f72ae21b in_seq=0 out_seq=0 2024-05-27T07:36:44.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.445+0000 7f13a3e00700 10 client.16029 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:44.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.447+0000 7f13a3e00700 1 -- 172.21.15.63:0/1472993574 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556bd49b4d90 con 0x556bd4a9e550 2024-05-27T07:36:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.447+0000 7f13a3e00700 10 client.16029 handle_command_reply: tid=0 2024-05-27T07:36:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.447+0000 7f13d72bab80 1 client.16029 shutdown 2024-05-27T07:36:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13d72bab80 1 -- 172.21.15.63:0/1472993574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4905e40 msgr2=0x556bd4a8e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:44.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13d72bab80 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4905e40 0x556bd4a8e4f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f13b4010a50 tx=0x7f13b4010e10 comp rx=0 tx=0).stop 2024-05-27T07:36:44.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13d72bab80 1 -- 172.21.15.63:0/1472993574 shutdown_connections 2024-05-27T07:36:44.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13d72bab80 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556bd4a9e550 0x556bd4aa0a60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13d72bab80 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4a84820 0x556bd4a8ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13d72bab80 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4905e40 0x556bd4a8e4f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13d72bab80 1 --2- 172.21.15.63:0/1472993574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4a84410 0x556bd4a8ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13d72bab80 1 -- 172.21.15.63:0/1472993574 >> 172.21.15.63:0/1472993574 conn(0x556bd4965fc0 msgr2=0x556bd4a7eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:44.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13a3e00700 0 client.16029 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:44.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13d72bab80 1 -- 172.21.15.63:0/1472993574 shutdown_connections 2024-05-27T07:36:44.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.448+0000 7f13d72bab80 1 -- 172.21.15.63:0/1472993574 wait complete. 2024-05-27T07:36:44.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.449+0000 7f13d72bab80 20 client.16029 trim_cache size 0 max 16384 2024-05-27T07:36:44.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.469+0000 7f13d72bab80 1 Processor -- start 2024-05-27T07:36:44.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.469+0000 7f13d72bab80 1 -- start start 2024-05-27T07:36:44.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4965fc0 0x556bd4a9a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4905e40 0x556bd4a8e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a8ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13d72bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b4cd0 con 0x556bd4a84410 2024-05-27T07:36:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13d72bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b2500 con 0x556bd4905e40 2024-05-27T07:36:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13d72bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b0350 con 0x556bd4965fc0 2024-05-27T07:36:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13c6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a8ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13c6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a8ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57342/0 (socket says 172.21.15.63:57342) 2024-05-27T07:36:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13c6800700 1 -- 172.21.15.63:0/1207471804 learned_addr learned my addr 172.21.15.63:0/1207471804 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13c5e00700 1 --2- 172.21.15.63:0/1207471804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4965fc0 0x556bd4a9a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.470+0000 7f13c5400700 1 --2- 172.21.15.63:0/1207471804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4905e40 0x556bd4a8e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.471+0000 7f13c6800700 1 -- 172.21.15.63:0/1207471804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4965fc0 msgr2=0x556bd4a9a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.471+0000 7f13c6800700 1 --2- 172.21.15.63:0/1207471804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4965fc0 0x556bd4a9a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.471+0000 7f13c6800700 1 -- 172.21.15.63:0/1207471804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4905e40 msgr2=0x556bd4a8e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.471+0000 7f13c6800700 1 --2- 172.21.15.63:0/1207471804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4905e40 0x556bd4a8e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.471+0000 7f13c6800700 1 -- 172.21.15.63:0/1207471804 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bd4a8ca60 con 0x556bd4a84410 2024-05-27T07:36:44.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.471+0000 7f13c6800700 1 --2- 172.21.15.63:0/1207471804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a8ef20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0178e0 tx=0x7f13bc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf19dae97882dd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:44.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.472+0000 7f13aea00700 1 -- 172.21.15.63:0/1207471804 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc022640 con 0x556bd4a84410 2024-05-27T07:36:44.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.472+0000 7f13aea00700 1 -- 172.21.15.63:0/1207471804 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13bc027070 con 0x556bd4a84410 2024-05-27T07:36:44.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.472+0000 7f13aea00700 1 -- 172.21.15.63:0/1207471804 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc016b00 con 0x556bd4a84410 2024-05-27T07:36:44.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.473+0000 7f13d72bab80 1 -- 172.21.15.63:0/1207471804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 msgr2=0x556bd4a8ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:44.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.473+0000 7f13d72bab80 1 --2- 172.21.15.63:0/1207471804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a8ef20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0178e0 tx=0x7f13bc004b40 comp rx=0 tx=0).stop 2024-05-27T07:36:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.473+0000 7f13d72bab80 1 -- 172.21.15.63:0/1207471804 shutdown_connections 2024-05-27T07:36:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.473+0000 7f13d72bab80 1 --2- 172.21.15.63:0/1207471804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a8ef20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.473+0000 7f13d72bab80 1 --2- 172.21.15.63:0/1207471804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4965fc0 0x556bd4a9a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.473+0000 7f13d72bab80 1 --2- 172.21.15.63:0/1207471804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4905e40 0x556bd4a8e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.473+0000 7f13d72bab80 1 -- 172.21.15.63:0/1207471804 >> 172.21.15.63:0/1207471804 conn(0x556bd4a78f60 msgr2=0x556bd4a8e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.473+0000 7f13d72bab80 1 -- 172.21.15.63:0/1207471804 shutdown_connections 2024-05-27T07:36:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.473+0000 7f13d72bab80 1 -- 172.21.15.63:0/1207471804 wait complete. 2024-05-27T07:36:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.474+0000 7f13d72bab80 1 -- start start 2024-05-27T07:36:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.474+0000 7f13d72bab80 1 Processor -- start 2024-05-27T07:36:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.475+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4905e40 0x556bd4a970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.475+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4965fc0 0x556bd4a92400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.475+0000 7f13d72bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a92a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.475+0000 7f13d72bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b09c0 con 0x556bd4a84410 2024-05-27T07:36:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.475+0000 7f13d72bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b1e30 con 0x556bd4965fc0 2024-05-27T07:36:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.475+0000 7f13d72bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13c01b1090 con 0x556bd4905e40 2024-05-27T07:36:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.475+0000 7f13d72bab80 10 client.0 _ll_register_callbacks cb 0x556bd4a7f680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.475+0000 7f13c6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a92a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.475+0000 7f13c6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a92a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57350/0 (socket says 172.21.15.63:57350) 2024-05-27T07:36:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.475+0000 7f13c6800700 1 -- 172.21.15.63:0/4276987208 learned_addr learned my addr 172.21.15.63:0/4276987208 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.476+0000 7f13c5e00700 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4905e40 0x556bd4a970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.476+0000 7f13c5400700 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4965fc0 0x556bd4a92400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.476+0000 7f13c6800700 1 -- 172.21.15.63:0/4276987208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4905e40 msgr2=0x556bd4a970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.476+0000 7f13c6800700 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4905e40 0x556bd4a970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.476+0000 7f13c6800700 1 -- 172.21.15.63:0/4276987208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4965fc0 msgr2=0x556bd4a92400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.476+0000 7f13c6800700 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4965fc0 0x556bd4a92400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.476+0000 7f13c6800700 1 -- 172.21.15.63:0/4276987208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bd4a931d0 con 0x556bd4a84410 2024-05-27T07:36:44.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.477+0000 7f13c6800700 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a92a50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009010 tx=0x7f13bc016ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ec4fc517219a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:44.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.477+0000 7f13a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.477+0000 7f13a3e00700 1 -- 172.21.15.63:0/4276987208 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc022850 con 0x556bd4a84410 2024-05-27T07:36:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.477+0000 7f13d72bab80 1 -- 172.21.15.63:0/4276987208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13bc027070 con 0x556bd4a84410 2024-05-27T07:36:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.477+0000 7f13a3e00700 1 -- 172.21.15.63:0/4276987208 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f13bc0229f0 con 0x556bd4a84410 2024-05-27T07:36:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.477+0000 7f13a3e00700 1 -- 172.21.15.63:0/4276987208 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f13bc022bf0 con 0x556bd4a84410 2024-05-27T07:36:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.477+0000 7f13d72bab80 10 client.16035 fetch_fsmap learned FSMap version 32 2024-05-27T07:36:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.477+0000 7f13d72bab80 1 -- 172.21.15.63:0/4276987208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556bd4b09160 con 0x556bd4a84410 2024-05-27T07:36:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.478+0000 7f13a3e00700 1 -- 172.21.15.63:0/4276987208 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13bc027070 con 0x556bd4a84410 2024-05-27T07:36:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.478+0000 7f13a3e00700 1 -- 172.21.15.63:0/4276987208 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f13bc035cb0 con 0x556bd4a84410 2024-05-27T07:36:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.478+0000 7f13d72bab80 10 client.16035 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:36:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.478+0000 7f13d72bab80 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.478+0000 7f13d72bab80 20 client.16035 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.478+0000 7f13d72bab80 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556bd4a84820 0x556bd4b0b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.478+0000 7f13d72bab80 4 client.16035 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:36:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.478+0000 7f13d72bab80 1 -- 172.21.15.63:0/4276987208 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bd49ae570 con 0x556bd4a84820 2024-05-27T07:36:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.479+0000 7f13c5e00700 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556bd4a84820 0x556bd4b0b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.480+0000 7f13c5e00700 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556bd4a84820 0x556bd4b0b8e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa5850f56ef7c363 server_cookie=76a0a89000b42b8d in_seq=0 out_seq=0 2024-05-27T07:36:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.480+0000 7f13a3e00700 10 client.16035 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13a3e00700 1 -- 172.21.15.63:0/4276987208 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bd49ae570 con 0x556bd4a84820 2024-05-27T07:36:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13a3e00700 10 client.16035 handle_command_reply: tid=0 2024-05-27T07:36:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 client.16035 shutdown 2024-05-27T07:36:44.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 -- 172.21.15.63:0/4276987208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 msgr2=0x556bd4a92a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:44.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a92a50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009010 tx=0x7f13bc016ee0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 -- 172.21.15.63:0/4276987208 shutdown_connections 2024-05-27T07:36:44.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556bd4a84820 0x556bd4b0b8e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556bd4a84410 0x556bd4a92a50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556bd4905e40 0x556bd4a970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 --2- 172.21.15.63:0/4276987208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd4965fc0 0x556bd4a92400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 -- 172.21.15.63:0/4276987208 >> 172.21.15.63:0/4276987208 conn(0x556bd4a78f60 msgr2=0x556bd4a8ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:44.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13a3e00700 0 client.16035 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:44.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 -- 172.21.15.63:0/4276987208 shutdown_connections 2024-05-27T07:36:44.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.481+0000 7f13d72bab80 1 -- 172.21.15.63:0/4276987208 wait complete. 2024-05-27T07:36:44.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13d72bab80 20 client.16035 trim_cache size 0 max 16384 2024-05-27T07:36:44.478 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13c7200700 1 -- 172.21.15.63:0/888020049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13a80b29f0 msgr2=0x7f13a80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13c7200700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13a80b29f0 0x7f13a80b4f00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x556bd4a80690 tx=0x7f13bc009040 comp rx=0 tx=0).stop 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13c7200700 1 -- 172.21.15.63:0/888020049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0105c10 msgr2=0x7f13c019d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13c7200700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0105c10 0x7f13c019d340 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f13b00047b0 tx=0x7f13b000bfa0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13c7200700 1 -- 172.21.15.63:0/888020049 shutdown_connections 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13c7200700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13a80b29f0 0x7f13a80b4f00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13c7200700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c010e730 0x7f13c01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13c7200700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c01052c0 0x7f13c019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13c7200700 1 --2- 172.21.15.63:0/888020049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13c0105c10 0x7f13c019d340 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.482+0000 7f13c7200700 1 -- 172.21.15.63:0/888020049 >> 172.21.15.63:0/888020049 conn(0x7f13c0078af0 msgr2=0x7f13c0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.483+0000 7f13c7200700 1 -- 172.21.15.63:0/888020049 shutdown_connections 2024-05-27T07:36:44.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:44.483+0000 7f13c7200700 1 -- 172.21.15.63:0/888020049 wait complete. 2024-05-27T07:36:45.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:45 smithi086 ceph-mon[34687]: pgmap v99: 129 pgs: 129 active+clean; 73 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:45.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:45 smithi149 ceph-mon[35408]: pgmap v99: 129 pgs: 129 active+clean; 73 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:45.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:45 smithi063 ceph-mon[31613]: pgmap v99: 129 pgs: 129 active+clean; 73 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:46.450 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:36:46.450 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:36:46.948 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:36:47.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:47 smithi086 ceph-mon[34687]: pgmap v100: 129 pgs: 129 active+clean; 105 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:47.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:47 smithi149 ceph-mon[35408]: pgmap v100: 129 pgs: 129 active+clean; 105 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:47.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:47 smithi063 ceph-mon[31613]: pgmap v100: 129 pgs: 129 active+clean; 105 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:49.461 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:49 smithi063 ceph-mon[31613]: pgmap v101: 129 pgs: 129 active+clean; 105 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:49.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.493+0000 7f48ae200700 1 -- 172.21.15.63:0/750105481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a810e730 msgr2=0x7f48a8110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:49.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.493+0000 7f48ae200700 1 --2- 172.21.15.63:0/750105481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a810e730 0x7f48a8110bd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f48a000b0a0 tx=0x7f48a000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.493+0000 7f48ae200700 1 -- 172.21.15.63:0/750105481 shutdown_connections 2024-05-27T07:36:49.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.493+0000 7f48ae200700 1 --2- 172.21.15.63:0/750105481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a810e730 0x7f48a8110bd0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.493+0000 7f48ae200700 1 --2- 172.21.15.63:0/750105481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48a8105300 0x7f48a8105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.493+0000 7f48ae200700 1 --2- 172.21.15.63:0/750105481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a8105c50 0x7f48a810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.493+0000 7f48ae200700 1 -- 172.21.15.63:0/750105481 >> 172.21.15.63:0/750105481 conn(0x7f48a8078ab0 msgr2=0x7f48a8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:49.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.493+0000 7f48ae200700 1 -- 172.21.15.63:0/750105481 shutdown_connections 2024-05-27T07:36:49.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.494+0000 7f48ae200700 1 -- 172.21.15.63:0/750105481 wait complete. 2024-05-27T07:36:49.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.495+0000 7f48ae200700 1 Processor -- start 2024-05-27T07:36:49.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.495+0000 7f48ae200700 1 -- start start 2024-05-27T07:36:49.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.496+0000 7f48ae200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a8105300 0x7f48a81a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.496+0000 7f48ae200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a8105c50 0x7f48a81a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.496+0000 7f48ae200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48a810e730 0x7f48a819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.496+0000 7f48ae200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8114100 con 0x7f48a8105300 2024-05-27T07:36:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.496+0000 7f48ae200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8113f80 con 0x7f48a8105c50 2024-05-27T07:36:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.496+0000 7f48ae200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8114280 con 0x7f48a810e730 2024-05-27T07:36:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.496+0000 7f48ace00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a8105300 0x7f48a81a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.496+0000 7f48ace00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a8105300 0x7f48a81a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59010/0 (socket says 172.21.15.63:59010) 2024-05-27T07:36:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.496+0000 7f48ace00700 1 -- 172.21.15.63:0/2876783770 learned_addr learned my addr 172.21.15.63:0/2876783770 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:49.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.497+0000 7f48a7e00700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a8105c50 0x7f48a81a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.497+0000 7f48ad800700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48a810e730 0x7f48a819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.497+0000 7f48ace00700 1 -- 172.21.15.63:0/2876783770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48a810e730 msgr2=0x7f48a819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:49.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.497+0000 7f48ace00700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48a810e730 0x7f48a819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.497+0000 7f48ace00700 1 -- 172.21.15.63:0/2876783770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a8105c50 msgr2=0x7f48a81a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:49.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.497+0000 7f48ace00700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a8105c50 0x7f48a81a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.497+0000 7f48ace00700 1 -- 172.21.15.63:0/2876783770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a819d170 con 0x7f48a8105300 2024-05-27T07:36:49.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.497+0000 7f48ad800700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48a810e730 0x7f48a819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:49.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.498+0000 7f48ace00700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a8105300 0x7f48a81a28a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f489c00d950 tx=0x7f489c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:49.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.498+0000 7f48a5600700 1 -- 172.21.15.63:0/2876783770 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c016070 con 0x7f48a8105300 2024-05-27T07:36:49.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.498+0000 7f48a5600700 1 -- 172.21.15.63:0/2876783770 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f489c010460 con 0x7f48a8105300 2024-05-27T07:36:49.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.498+0000 7f48a5600700 1 -- 172.21.15.63:0/2876783770 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c00f620 con 0x7f48a8105300 2024-05-27T07:36:49.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.498+0000 7f48ae200700 1 -- 172.21.15.63:0/2876783770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a819d460 con 0x7f48a8105300 2024-05-27T07:36:49.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.498+0000 7f48ae200700 1 -- 172.21.15.63:0/2876783770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a81a9730 con 0x7f48a8105300 2024-05-27T07:36:49.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.504+0000 7f48a5600700 1 -- 172.21.15.63:0/2876783770 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f489c019070 con 0x7f48a8105300 2024-05-27T07:36:49.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.504+0000 7f48be326b80 1 Processor -- start 2024-05-27T07:36:49.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.504+0000 7f48be326b80 1 -- start start 2024-05-27T07:36:49.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.505+0000 7f48be326b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdf4fec0 0x5562ce0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.505+0000 7f48be326b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0d8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.505+0000 7f48be326b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce8a0 0x5562ce0db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.505+0000 7f48be326b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5562cdffcec0 con 0x5562cdf4fec0 2024-05-27T07:36:49.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.505+0000 7f48be326b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5562cdfeca60 con 0x5562ce0ce8a0 2024-05-27T07:36:49.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.505+0000 7f48be326b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5562cdff0aa0 con 0x5562ce0ce490 2024-05-27T07:36:49.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.505+0000 7f48ad800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdf4fec0 0x5562ce0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.505+0000 7f48ad800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdf4fec0 0x5562ce0cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59016/0 (socket says 172.21.15.63:59016) 2024-05-27T07:36:49.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.505+0000 7f48ad800700 1 -- 172.21.15.63:0/2115576540 learned_addr learned my addr 172.21.15.63:0/2115576540 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:49.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.506+0000 7f48a7e00700 1 --2- 172.21.15.63:0/2115576540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce8a0 0x5562ce0db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.506+0000 7f48ace00700 1 --2- 172.21.15.63:0/2115576540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0d8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.506+0000 7f48a5600700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48900b2a70 0x7f48900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.506+0000 7f48ad800700 1 -- 172.21.15.63:0/2115576540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 msgr2=0x5562ce0d8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:49.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.506+0000 7f48ad800700 1 --2- 172.21.15.63:0/2115576540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0d8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.506+0000 7f48ad800700 1 -- 172.21.15.63:0/2115576540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce8a0 msgr2=0x5562ce0db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:49.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.506+0000 7f48ad800700 1 --2- 172.21.15.63:0/2115576540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce8a0 0x5562ce0db0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.506+0000 7f48ad800700 1 -- 172.21.15.63:0/2115576540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562cdfb54a0 con 0x5562cdf4fec0 2024-05-27T07:36:49.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.506+0000 7f48a5600700 1 -- 172.21.15.63:0/2876783770 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f489c0e2550 con 0x7f48a8105300 2024-05-27T07:36:49.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.507+0000 7f48ad800700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48900b2a70 0x7f48900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.507+0000 7f48ad800700 1 --2- 172.21.15.63:0/2115576540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdf4fec0 0x5562ce0cdf50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f48a0006130 tx=0x7f48a0009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6dfcf9344251ae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:49.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.507+0000 7f489a000700 1 -- 172.21.15.63:0/2115576540 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0004d90 con 0x5562cdf4fec0 2024-05-27T07:36:49.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.507+0000 7f489a000700 1 -- 172.21.15.63:0/2115576540 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48a0022040 con 0x5562cdf4fec0 2024-05-27T07:36:49.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.508+0000 7f489a000700 1 -- 172.21.15.63:0/2115576540 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a001c640 con 0x5562cdf4fec0 2024-05-27T07:36:49.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.508+0000 7f48be326b80 1 -- 172.21.15.63:0/2115576540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdf4fec0 msgr2=0x5562ce0cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:49.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.508+0000 7f48be326b80 1 --2- 172.21.15.63:0/2115576540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdf4fec0 0x5562ce0cdf50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f48a0006130 tx=0x7f48a0009dd0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.508+0000 7f48ad800700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48900b2a70 0x7f48900b4f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009e60 tx=0x7f48a00299c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:49.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.509+0000 7f48be326b80 1 -- 172.21.15.63:0/2115576540 shutdown_connections 2024-05-27T07:36:49.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.509+0000 7f48be326b80 1 --2- 172.21.15.63:0/2115576540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce8a0 0x5562ce0db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.509+0000 7f48be326b80 1 --2- 172.21.15.63:0/2115576540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdf4fec0 0x5562ce0cdf50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.509+0000 7f48be326b80 1 --2- 172.21.15.63:0/2115576540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0d8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.509+0000 7f48be326b80 1 -- 172.21.15.63:0/2115576540 >> 172.21.15.63:0/2115576540 conn(0x5562cdfb0040 msgr2=0x5562ce0c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:49.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.509+0000 7f48be326b80 1 -- 172.21.15.63:0/2115576540 shutdown_connections 2024-05-27T07:36:49.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.509+0000 7f48be326b80 1 -- 172.21.15.63:0/2115576540 wait complete. 2024-05-27T07:36:49.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.510+0000 7f48be326b80 1 -- start start 2024-05-27T07:36:49.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.510+0000 7f48be326b80 1 Processor -- start 2024-05-27T07:36:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48be326b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0d86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48be326b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0d8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48be326b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562ce0ce8a0 0x5562ce0d9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48be326b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5562cdef3020 con 0x5562ce0ce8a0 2024-05-27T07:36:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48be326b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5562cdf8d7e0 con 0x5562cdf4fec0 2024-05-27T07:36:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48be326b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5562ce0dba60 con 0x5562ce0ce490 2024-05-27T07:36:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48be326b80 10 client.0 _ll_register_callbacks cb 0x5562cdf96cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48ad800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562ce0ce8a0 0x5562ce0d9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48ad800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562ce0ce8a0 0x5562ce0d9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59032/0 (socket says 172.21.15.63:59032) 2024-05-27T07:36:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48ad800700 1 -- 172.21.15.63:0/1243179105 learned_addr learned my addr 172.21.15.63:0/1243179105 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48a7e00700 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0d8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.511+0000 7f48ace00700 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0d86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.512+0000 7f48ad800700 1 -- 172.21.15.63:0/1243179105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 msgr2=0x5562ce0d8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.512+0000 7f48ad800700 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0d8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.512+0000 7f48ad800700 1 -- 172.21.15.63:0/1243179105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 msgr2=0x5562ce0d86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.512+0000 7f48ad800700 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0d86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.512+0000 7f48ad800700 1 -- 172.21.15.63:0/1243179105 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562cdf3c950 con 0x5562ce0ce8a0 2024-05-27T07:36:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.512+0000 7f48ace00700 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0d86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.512+0000 7f48ad800700 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562ce0ce8a0 0x5562ce0d9120 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f48a0004e80 tx=0x7f48a0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8b005dbf60a45e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.512+0000 7f488ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.513+0000 7f488ea00700 1 -- 172.21.15.63:0/1243179105 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0029710 con 0x5562ce0ce8a0 2024-05-27T07:36:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.513+0000 7f488ea00700 1 -- 172.21.15.63:0/1243179105 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f48a00075c0 con 0x5562ce0ce8a0 2024-05-27T07:36:49.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.513+0000 7f48be326b80 1 -- 172.21.15.63:0/1243179105 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48a0022040 con 0x5562ce0ce8a0 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.514+0000 7f488ea00700 1 -- 172.21.15.63:0/1243179105 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f48a0044050 con 0x5562ce0ce8a0 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.514+0000 7f48be326b80 10 client.16047 fetch_fsmap learned FSMap version 32 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.514+0000 7f48be326b80 1 -- 172.21.15.63:0/1243179105 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562cdfbd0e0 con 0x5562ce0ce8a0 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.514+0000 7f488ea00700 1 -- 172.21.15.63:0/1243179105 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48a0022040 con 0x5562ce0ce8a0 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.515+0000 7f488ea00700 1 -- 172.21.15.63:0/1243179105 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f48a0011070 con 0x5562ce0ce8a0 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.515+0000 7f48be326b80 10 client.16047 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.515+0000 7f48be326b80 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.515+0000 7f48be326b80 20 client.16047 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.515+0000 7f48be326b80 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562ce0e8860 0x5562ce0eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.515+0000 7f48be326b80 4 client.16047 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:36:49.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.515+0000 7f48be326b80 1 -- 172.21.15.63:0/1243179105 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562cdffee10 con 0x5562ce0e8860 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.516+0000 7f48ace00700 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562ce0e8860 0x5562ce0eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.517+0000 7f488ea00700 10 client.16047 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.517+0000 7f48ace00700 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562ce0e8860 0x5562ce0eac80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66faa12084e7e6c5 server_cookie=4354410393a0e12c in_seq=0 out_seq=0 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f488ea00700 1 -- 172.21.15.63:0/1243179105 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5562cdffee10 con 0x5562ce0e8860 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f488ea00700 10 client.16047 handle_command_reply: tid=0 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f48be326b80 1 client.16047 shutdown 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f48be326b80 1 -- 172.21.15.63:0/1243179105 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562ce0ce8a0 msgr2=0x5562ce0d9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f48be326b80 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562ce0ce8a0 0x5562ce0d9120 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f48a0004e80 tx=0x7f48a0020000 comp rx=0 tx=0).stop 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f48be326b80 1 -- 172.21.15.63:0/1243179105 shutdown_connections 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f48be326b80 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562ce0e8860 0x5562ce0eac80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f48be326b80 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562ce0ce8a0 0x5562ce0d9120 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f48be326b80 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0d86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f48be326b80 1 --2- 172.21.15.63:0/1243179105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0d8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f48be326b80 1 -- 172.21.15.63:0/1243179105 >> 172.21.15.63:0/1243179105 conn(0x5562cdfb0040 msgr2=0x5562ce0c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:49.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.519+0000 7f488ea00700 0 client.16047 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:49.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.520+0000 7f48be326b80 1 -- 172.21.15.63:0/1243179105 shutdown_connections 2024-05-27T07:36:49.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.520+0000 7f48be326b80 1 -- 172.21.15.63:0/1243179105 wait complete. 2024-05-27T07:36:49.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.520+0000 7f48be326b80 20 client.16047 trim_cache size 0 max 16384 2024-05-27T07:36:49.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.552+0000 7f48be326b80 1 Processor -- start 2024-05-27T07:36:49.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.552+0000 7f48be326b80 1 -- start start 2024-05-27T07:36:49.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.552+0000 7f48be326b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 0x5562ce0d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.552+0000 7f48be326b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562cdf4fec0 0x5562ce0dab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.552+0000 7f48be326b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce490 0x5562ce0e4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.552+0000 7f48be326b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48a81b4d10 con 0x5562cdfb0040 2024-05-27T07:36:49.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.552+0000 7f48be326b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48a81b2540 con 0x5562ce0ce490 2024-05-27T07:36:49.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.552+0000 7f48be326b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48a81b0390 con 0x5562cdf4fec0 2024-05-27T07:36:49.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48ace00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 0x5562ce0d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48ace00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 0x5562ce0d9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59038/0 (socket says 172.21.15.63:59038) 2024-05-27T07:36:49.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48ace00700 1 -- 172.21.15.63:0/1339181162 learned_addr learned my addr 172.21.15.63:0/1339181162 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:49.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48ad800700 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce490 0x5562ce0e4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48a7e00700 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562cdf4fec0 0x5562ce0dab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48ace00700 1 -- 172.21.15.63:0/1339181162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562cdf4fec0 msgr2=0x5562ce0dab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:49.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48ace00700 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562cdf4fec0 0x5562ce0dab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48ace00700 1 -- 172.21.15.63:0/1339181162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce490 msgr2=0x5562ce0e4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:49.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48ace00700 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce490 0x5562ce0e4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48ace00700 1 -- 172.21.15.63:0/1339181162 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ce0d6c10 con 0x5562cdfb0040 2024-05-27T07:36:49.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48ad800700 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce490 0x5562ce0e4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:36:49.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.553+0000 7f48a7e00700 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562cdf4fec0 0x5562ce0dab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:36:49.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.554+0000 7f48ace00700 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 0x5562ce0d9090 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f489c0a6730 tx=0x7f489c0e2850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5264085881310e31 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:49.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.554+0000 7f4899600700 1 -- 172.21.15.63:0/1339181162 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c020040 con 0x5562cdfb0040 2024-05-27T07:36:49.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.554+0000 7f4899600700 1 -- 172.21.15.63:0/1339181162 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f489c0a94c0 con 0x5562cdfb0040 2024-05-27T07:36:49.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.554+0000 7f4899600700 1 -- 172.21.15.63:0/1339181162 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c0e2160 con 0x5562cdfb0040 2024-05-27T07:36:49.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.555+0000 7f48be326b80 1 -- 172.21.15.63:0/1339181162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 msgr2=0x5562ce0d9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:49.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.555+0000 7f48be326b80 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 0x5562ce0d9090 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f489c0a6730 tx=0x7f489c0e2850 comp rx=0 tx=0).stop 2024-05-27T07:36:49.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.556+0000 7f48be326b80 1 -- 172.21.15.63:0/1339181162 shutdown_connections 2024-05-27T07:36:49.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.556+0000 7f48be326b80 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562ce0ce490 0x5562ce0e4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.556+0000 7f48be326b80 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 0x5562ce0d9090 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.556+0000 7f48be326b80 1 --2- 172.21.15.63:0/1339181162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562cdf4fec0 0x5562ce0dab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.556+0000 7f48be326b80 1 -- 172.21.15.63:0/1339181162 >> 172.21.15.63:0/1339181162 conn(0x5562ce0c2fe0 msgr2=0x5562ce0d2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:49.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.556+0000 7f48be326b80 1 -- 172.21.15.63:0/1339181162 shutdown_connections 2024-05-27T07:36:49.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.556+0000 7f48be326b80 1 -- 172.21.15.63:0/1339181162 wait complete. 2024-05-27T07:36:49.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.556+0000 7f48be326b80 1 -- start start 2024-05-27T07:36:49.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.557+0000 7f48be326b80 1 Processor -- start 2024-05-27T07:36:49.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.557+0000 7f48be326b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.557+0000 7f48be326b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 0x5562ce0ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.557+0000 7f48be326b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.557+0000 7f48be326b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48a81b0a00 con 0x5562cdfb0040 2024-05-27T07:36:49.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.557+0000 7f48be326b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48a81b1e70 con 0x5562cdf4fec0 2024-05-27T07:36:49.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.557+0000 7f48be326b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48a81b10d0 con 0x5562ce0ce490 2024-05-27T07:36:49.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.557+0000 7f48be326b80 10 client.0 _ll_register_callbacks cb 0x5562cdf96cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:49.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.558+0000 7f48ace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.558+0000 7f48ace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0edb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:42318/0 (socket says 172.21.15.63:42318) 2024-05-27T07:36:49.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.558+0000 7f48ace00700 1 -- 172.21.15.63:0/217522348 learned_addr learned my addr 172.21.15.63:0/217522348 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:49.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.558+0000 7f48ad800700 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.559+0000 7f48ace00700 1 -- 172.21.15.63:0/217522348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 msgr2=0x5562ce0ec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:49.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.559+0000 7f48ace00700 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0ec070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.559+0000 7f48ace00700 1 -- 172.21.15.63:0/217522348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 msgr2=0x5562ce0ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:49.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.559+0000 7f48ace00700 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 0x5562ce0ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.559+0000 7f48ace00700 1 -- 172.21.15.63:0/217522348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ce0ec810 con 0x5562cdf4fec0 2024-05-27T07:36:49.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.559+0000 7f48ace00700 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0edb80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f489c0ac040 tx=0x7f489c0a5980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f76ce303bda87b17 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:49.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.559+0000 7f488ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:36:49.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.559+0000 7f488ea00700 1 -- 172.21.15.63:0/217522348 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c020030 con 0x5562cdf4fec0 2024-05-27T07:36:49.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.559+0000 7f48be326b80 1 -- 172.21.15.63:0/217522348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f489c0a94c0 con 0x5562cdf4fec0 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.559+0000 7f488ea00700 1 -- 172.21.15.63:0/217522348 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f489c0a5ab0 con 0x5562cdf4fec0 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.560+0000 7f488ea00700 1 -- 172.21.15.63:0/217522348 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f489c0b2320 con 0x5562cdf4fec0 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.560+0000 7f48be326b80 10 client.35744 fetch_fsmap learned FSMap version 32 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.560+0000 7f48be326b80 1 -- 172.21.15.63:0/217522348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562ce15c370 con 0x5562cdf4fec0 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.560+0000 7f488ea00700 1 -- 172.21.15.63:0/217522348 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f489c0a94c0 con 0x5562cdf4fec0 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.561+0000 7f488ea00700 1 -- 172.21.15.63:0/217522348 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f489c0a3be0 con 0x5562cdf4fec0 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.561+0000 7f48be326b80 10 client.35744 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.561+0000 7f48be326b80 10 client.35744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.561+0000 7f48be326b80 20 client.35744 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.561+0000 7f48be326b80 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562ce0ce8a0 0x5562ce15ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:49.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.561+0000 7f48be326b80 4 client.35744 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:36:49.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.561+0000 7f48be326b80 1 -- 172.21.15.63:0/217522348 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5562cdff85f0 con 0x5562ce0ce8a0 2024-05-27T07:36:49.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.562+0000 7f48a7e00700 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562ce0ce8a0 0x5562ce15ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:49.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.563+0000 7f488ea00700 10 client.35744 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:49.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.563+0000 7f48a7e00700 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562ce0ce8a0 0x5562ce15ea10 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc839ae85665060 server_cookie=77305c9a335f96aa in_seq=0 out_seq=0 2024-05-27T07:36:49.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.566+0000 7f488ea00700 1 -- 172.21.15.63:0/217522348 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5562cdff85f0 con 0x5562ce0ce8a0 2024-05-27T07:36:49.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.566+0000 7f488ea00700 10 client.35744 handle_command_reply: tid=0 2024-05-27T07:36:49.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.566+0000 7f48be326b80 1 client.35744 shutdown 2024-05-27T07:36:49.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.567+0000 7f48be326b80 1 -- 172.21.15.63:0/217522348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 msgr2=0x5562ce0edb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:49.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.567+0000 7f48be326b80 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0edb80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f489c0ac040 tx=0x7f489c0a5980 comp rx=0 tx=0).stop 2024-05-27T07:36:49.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.567+0000 7f48be326b80 1 -- 172.21.15.63:0/217522348 shutdown_connections 2024-05-27T07:36:49.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.567+0000 7f48be326b80 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562ce0ce8a0 0x5562ce15ea10 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.567+0000 7f48be326b80 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562ce0ce490 0x5562ce0ec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.567+0000 7f48be326b80 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562cdf4fec0 0x5562ce0edb80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.567+0000 7f48be326b80 1 --2- 172.21.15.63:0/217522348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562cdfb0040 0x5562ce0ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.567+0000 7f48be326b80 1 -- 172.21.15.63:0/217522348 >> 172.21.15.63:0/217522348 conn(0x5562ce0c2fe0 msgr2=0x5562ce0e3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:49.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.567+0000 7f488ea00700 0 client.35744 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:49.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.568+0000 7f48be326b80 1 -- 172.21.15.63:0/217522348 shutdown_connections 2024-05-27T07:36:49.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.568+0000 7f48be326b80 1 -- 172.21.15.63:0/217522348 wait complete. 2024-05-27T07:36:49.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.568+0000 7f48be326b80 20 client.35744 trim_cache size 0 max 16384 2024-05-27T07:36:49.565 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:36:49.565 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:36:49.565 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "4fd28b72-bd36-41d3-a048-928e0a5da755", 2024-05-27T07:36:49.565 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:36:49.565 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:36:49.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.569+0000 7f48ae200700 1 -- 172.21.15.63:0/2876783770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48900b2a70 msgr2=0x7f48900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:49.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.569+0000 7f48ae200700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48900b2a70 0x7f48900b4f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009e60 tx=0x7f48a00299c0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.569+0000 7f48ae200700 1 -- 172.21.15.63:0/2876783770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a8105300 msgr2=0x7f48a81a28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:49.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.569+0000 7f48ae200700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a8105300 0x7f48a81a28a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f489c00d950 tx=0x7f489c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:36:49.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.569+0000 7f48ae200700 1 -- 172.21.15.63:0/2876783770 shutdown_connections 2024-05-27T07:36:49.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.569+0000 7f48ae200700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48900b2a70 0x7f48900b4f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.569+0000 7f48ae200700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48a810e730 0x7f48a819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.569+0000 7f48ae200700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48a8105300 0x7f48a81a28a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.569+0000 7f48ae200700 1 --2- 172.21.15.63:0/2876783770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a8105c50 0x7f48a81a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:49.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.569+0000 7f48ae200700 1 -- 172.21.15.63:0/2876783770 >> 172.21.15.63:0/2876783770 conn(0x7f48a8078ab0 msgr2=0x7f48a8104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:49.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.570+0000 7f48ae200700 1 -- 172.21.15.63:0/2876783770 shutdown_connections 2024-05-27T07:36:49.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:49.570+0000 7f48ae200700 1 -- 172.21.15.63:0/2876783770 wait complete. 2024-05-27T07:36:49.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:49 smithi086 ceph-mon[34687]: pgmap v101: 129 pgs: 129 active+clean; 105 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:49.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:49 smithi149 ceph-mon[35408]: pgmap v101: 129 pgs: 129 active+clean; 105 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.4 MiB/s wr, 19 op/s 2024-05-27T07:36:50.466 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:36:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:36:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:36:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:36:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:36:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:36:50.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:36:50.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:36:50.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:36:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:36:50.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:36:50.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:36:50.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:36:50.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:36:50.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:36:50.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:36:50.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:36:50.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:50.964 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:36:51.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:51 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:36:51.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:51.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:51 smithi086 ceph-mon[34687]: pgmap v102: 129 pgs: 129 active+clean; 133 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 39 op/s 2024-05-27T07:36:51.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:51 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:36:51.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:51.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:51 smithi149 ceph-mon[35408]: pgmap v102: 129 pgs: 129 active+clean; 133 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 39 op/s 2024-05-27T07:36:51.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:51 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:36:51.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:36:51.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:51 smithi063 ceph-mon[31613]: pgmap v102: 129 pgs: 129 active+clean; 133 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 39 op/s 2024-05-27T07:36:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.904+0000 7fbf11e00640 1 -- 172.21.15.63:0/3251136643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf0c075420 msgr2=0x7fbf0c075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.904+0000 7fbf11e00640 1 --2- 172.21.15.63:0/3251136643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf0c075420 0x7fbf0c075830 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009900 tx=0x7fbf0002a480 comp rx=0 tx=0).stop 2024-05-27T07:36:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.905+0000 7fbf11e00640 1 -- 172.21.15.63:0/3251136643 shutdown_connections 2024-05-27T07:36:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.905+0000 7fbf11e00640 1 --2- 172.21.15.63:0/3251136643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c110300 0x7fbf0c112720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.905+0000 7fbf11e00640 1 --2- 172.21.15.63:0/3251136643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c10d9a0 0x7fbf0c10fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.905+0000 7fbf11e00640 1 --2- 172.21.15.63:0/3251136643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf0c075420 0x7fbf0c075830 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.905+0000 7fbf11e00640 1 -- 172.21.15.63:0/3251136643 >> 172.21.15.63:0/3251136643 conn(0x7fbf0c079140 msgr2=0x7fbf0c079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.905+0000 7fbf11e00640 1 -- 172.21.15.63:0/3251136643 shutdown_connections 2024-05-27T07:36:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.905+0000 7fbf11e00640 1 -- 172.21.15.63:0/3251136643 wait complete. 2024-05-27T07:36:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.906+0000 7fbf11e00640 1 Processor -- start 2024-05-27T07:36:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.906+0000 7fbf11e00640 1 -- start start 2024-05-27T07:36:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf11e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c075420 0x7fbf0c1a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf11e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c10d9a0 0x7fbf0c1a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf11e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf0c110300 0x7fbf0c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf11e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c115d20 con 0x7fbf0c110300 2024-05-27T07:36:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf11e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c115ba0 con 0x7fbf0c10d9a0 2024-05-27T07:36:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf11e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c115ea0 con 0x7fbf0c075420 2024-05-27T07:36:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf11400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf0c110300 0x7fbf0c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf11400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf0c110300 0x7fbf0c19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59050/0 (socket says 172.21.15.63:59050) 2024-05-27T07:36:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf11400640 1 -- 172.21.15.63:0/644274925 learned_addr learned my addr 172.21.15.63:0/644274925 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf10a00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c075420 0x7fbf0c1a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.907+0000 7fbf0be00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c10d9a0 0x7fbf0c1a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.908+0000 7fbf11400640 1 -- 172.21.15.63:0/644274925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c075420 msgr2=0x7fbf0c1a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.908+0000 7fbf11400640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c075420 0x7fbf0c1a24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.908+0000 7fbf11400640 1 -- 172.21.15.63:0/644274925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c10d9a0 msgr2=0x7fbf0c1a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.908+0000 7fbf11400640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c10d9a0 0x7fbf0c1a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.908+0000 7fbf11400640 1 -- 172.21.15.63:0/644274925 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0c19cdd0 con 0x7fbf0c110300 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.908+0000 7fbf0be00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c10d9a0 0x7fbf0c1a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.908+0000 7fbf11400640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf0c110300 0x7fbf0c19c6a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400eaa0 tx=0x7fbf0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.909+0000 7fbf09600640 1 -- 172.21.15.63:0/644274925 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0400cdb0 con 0x7fbf0c110300 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.909+0000 7fbf11e00640 1 -- 172.21.15.63:0/644274925 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0c19d060 con 0x7fbf0c110300 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.909+0000 7fbf09600640 1 -- 172.21.15.63:0/644274925 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf04005020 con 0x7fbf0c110300 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.909+0000 7fbf09600640 1 -- 172.21.15.63:0/644274925 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040116d0 con 0x7fbf0c110300 2024-05-27T07:36:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.909+0000 7fbf11e00640 1 -- 172.21.15.63:0/644274925 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0c1a9270 con 0x7fbf0c110300 2024-05-27T07:36:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.913+0000 7fbf1a3e6740 1 Processor -- start 2024-05-27T07:36:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.915+0000 7fbf09600640 1 -- 172.21.15.63:0/644274925 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbf04004b20 con 0x7fbf0c110300 2024-05-27T07:36:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.915+0000 7fbf1a3e6740 1 -- start start 2024-05-27T07:36:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.915+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df18ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.915+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df18bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.916+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df18c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.916+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c8df0fcb90 con 0x55c8dede91b0 2024-05-27T07:36:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.916+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c8df0ec730 con 0x55c8dedc84d0 2024-05-27T07:36:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.916+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c8df0f0770 con 0x55c8dedc02f0 2024-05-27T07:36:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.916+0000 7fbf0be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df18ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.916+0000 7fbf0be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df18ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59054/0 (socket says 172.21.15.63:59054) 2024-05-27T07:36:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.916+0000 7fbf0be00640 1 -- 172.21.15.63:0/3124573436 learned_addr learned my addr 172.21.15.63:0/3124573436 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.916+0000 7fbf10a00640 1 --2- 172.21.15.63:0/3124573436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df18bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.917+0000 7fbf0be00640 1 --2- 172.21.15.63:0/3124573436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df18c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.917+0000 7fbf0be00640 1 -- 172.21.15.63:0/3124573436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 msgr2=0x55c8df18bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.917+0000 7fbf0be00640 1 --2- 172.21.15.63:0/3124573436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df18bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.917+0000 7fbf0be00640 1 -- 172.21.15.63:0/3124573436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 msgr2=0x55c8df18c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.917+0000 7fbf0be00640 1 --2- 172.21.15.63:0/3124573436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df18c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.917+0000 7fbf0be00640 1 -- 172.21.15.63:0/3124573436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8df0858b0 con 0x55c8dede91b0 2024-05-27T07:36:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.917+0000 7fbf0be00640 1 --2- 172.21.15.63:0/3124573436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df18ba60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbef80135e0 tx=0x7fbef800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c692625e4ea9e558 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.918+0000 7fbefdc00640 1 -- 172.21.15.63:0/3124573436 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8019030 con 0x55c8dede91b0 2024-05-27T07:36:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.918+0000 7fbefdc00640 1 -- 172.21.15.63:0/3124573436 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbef8011950 con 0x55c8dede91b0 2024-05-27T07:36:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.918+0000 7fbefdc00640 1 -- 172.21.15.63:0/3124573436 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef801d820 con 0x55c8dede91b0 2024-05-27T07:36:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.918+0000 7fbf09600640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbef40b2950 0x7fbef40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.918+0000 7fbf09600640 1 -- 172.21.15.63:0/644274925 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fbf040e2b90 con 0x7fbf0c110300 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.918+0000 7fbf10a00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbef40b2950 0x7fbef40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.919+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3124573436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 msgr2=0x55c8df18ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.919+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/3124573436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df18ba60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbef80135e0 tx=0x7fbef800b6d0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.919+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3124573436 shutdown_connections 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.919+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/3124573436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df18c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.919+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/3124573436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df18bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.919+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/3124573436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df18ba60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.919+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3124573436 >> 172.21.15.63:0/3124573436 conn(0x55c8deddd300 msgr2=0x55c8df188210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.919+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3124573436 shutdown_connections 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.919+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3124573436 wait complete. 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.919+0000 7fbf10a00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbef40b2950 0x7fbef40b4e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbf000057c0 tx=0x7fbf000058d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.920+0000 7fbf1a3e6740 1 -- start start 2024-05-27T07:36:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.920+0000 7fbf1a3e6740 1 Processor -- start 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.921+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df195c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.921+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df1961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.921+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.921+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c8dedd2790 con 0x55c8dede91b0 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.921+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c8df094bc0 con 0x55c8dedc84d0 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.921+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c8df198b90 con 0x55c8dedc02f0 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.921+0000 7fbf1a3e6740 10 client.0 _ll_register_callbacks cb 0x55c8df072400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.921+0000 7fbf11400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.921+0000 7fbf11400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59064/0 (socket says 172.21.15.63:59064) 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.921+0000 7fbf11400640 1 -- 172.21.15.63:0/4171314684 learned_addr learned my addr 172.21.15.63:0/4171314684 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.922+0000 7fbf10a00640 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.922+0000 7fbf0be00640 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df1961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.922+0000 7fbf11400640 1 -- 172.21.15.63:0/4171314684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 msgr2=0x55c8df195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.922+0000 7fbf11400640 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.922+0000 7fbf11400640 1 -- 172.21.15.63:0/4171314684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 msgr2=0x55c8df1961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.922+0000 7fbf11400640 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df1961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.922+0000 7fbf11400640 1 -- 172.21.15.63:0/4171314684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8dee01210 con 0x55c8dede91b0 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.922+0000 7fbf11400640 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a1ec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fbf040a5b20 tx=0x7fbf04005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6b1bf146467d5d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.922+0000 7fbef2000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.923+0000 7fbef2000640 1 -- 172.21.15.63:0/4171314684 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0401f040 con 0x55c8dede91b0 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.923+0000 7fbef2000640 1 -- 172.21.15.63:0/4171314684 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fbf040a5d30 con 0x55c8dede91b0 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.923+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/4171314684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef8011950 con 0x55c8dede91b0 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.923+0000 7fbef2000640 1 -- 172.21.15.63:0/4171314684 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbf040c4150 con 0x55c8dede91b0 2024-05-27T07:36:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.923+0000 7fbf1a3e6740 10 client.16062 fetch_fsmap learned FSMap version 32 2024-05-27T07:36:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.923+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/4171314684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c8deecd620 con 0x55c8dede91b0 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.924+0000 7fbef2000640 1 -- 172.21.15.63:0/4171314684 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbef8011950 con 0x55c8dede91b0 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.924+0000 7fbef2000640 1 -- 172.21.15.63:0/4171314684 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbf0400f040 con 0x55c8dede91b0 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.924+0000 7fbf1a3e6740 10 client.16062 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.924+0000 7fbf1a3e6740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.924+0000 7fbf1a3e6740 20 client.16062 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.924+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c8deda7d80 0x55c8df1a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.924+0000 7fbf1a3e6740 4 client.16062 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.924+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/4171314684 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8df0feae0 con 0x55c8deda7d80 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.925+0000 7fbf10a00640 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c8deda7d80 0x55c8df1a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.926+0000 7fbf10a00640 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c8deda7d80 0x55c8df1a7f20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55184c65081f2214 server_cookie=46503f4f6399088e in_seq=0 out_seq=0 2024-05-27T07:36:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.926+0000 7fbef2000640 10 client.16062 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbef2000640 1 -- 172.21.15.63:0/4171314684 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c8df0feae0 con 0x55c8deda7d80 2024-05-27T07:36:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbef2000640 10 client.16062 handle_command_reply: tid=0 2024-05-27T07:36:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbf1a3e6740 1 client.16062 shutdown 2024-05-27T07:36:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/4171314684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 msgr2=0x55c8df1a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a1ec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fbf040a5b20 tx=0x7fbf04005720 comp rx=0 tx=0).stop 2024-05-27T07:36:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/4171314684 shutdown_connections 2024-05-27T07:36:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c8deda7d80 0x55c8df1a7f20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a1ec0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df1961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/4171314684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbef2000640 0 client.16062 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.928+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/4171314684 >> 172.21.15.63:0/4171314684 conn(0x55c8deddd300 msgr2=0x55c8df18b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.929+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/4171314684 shutdown_connections 2024-05-27T07:36:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.929+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/4171314684 wait complete. 2024-05-27T07:36:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.929+0000 7fbf1a3e6740 20 client.16062 trim_cache size 0 max 16384 2024-05-27T07:36:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.953+0000 7fbf1a3e6740 1 Processor -- start 2024-05-27T07:36:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.953+0000 7fbf1a3e6740 1 -- start start 2024-05-27T07:36:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.953+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df191480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.953+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df1a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.953+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.953+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b5a90 con 0x55c8dede91b0 2024-05-27T07:36:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.953+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b3300 con 0x55c8dedc84d0 2024-05-27T07:36:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.953+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b1150 con 0x55c8dedc02f0 2024-05-27T07:36:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.954+0000 7fbf11400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.954+0000 7fbf11400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59068/0 (socket says 172.21.15.63:59068) 2024-05-27T07:36:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.954+0000 7fbf11400640 1 -- 172.21.15.63:0/2811824330 learned_addr learned my addr 172.21.15.63:0/2811824330 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.954+0000 7fbf10a00640 1 --2- 172.21.15.63:0/2811824330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df191480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.954+0000 7fbf0be00640 1 --2- 172.21.15.63:0/2811824330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df1a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.954+0000 7fbf11400640 1 -- 172.21.15.63:0/2811824330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 msgr2=0x55c8df191480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.954+0000 7fbf11400640 1 --2- 172.21.15.63:0/2811824330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df191480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.954+0000 7fbf11400640 1 -- 172.21.15.63:0/2811824330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 msgr2=0x55c8df1a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.954+0000 7fbf11400640 1 --2- 172.21.15.63:0/2811824330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df1a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.954+0000 7fbf11400640 1 -- 172.21.15.63:0/2811824330 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8df1a26b0 con 0x55c8dede91b0 2024-05-27T07:36:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.955+0000 7fbf11400640 1 --2- 172.21.15.63:0/2811824330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a2170 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbf040a8ec0 tx=0x7fbf040a99a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f901030d7ee2331 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.955+0000 7fbefd200640 1 -- 172.21.15.63:0/2811824330 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0401f070 con 0x55c8dede91b0 2024-05-27T07:36:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.955+0000 7fbefd200640 1 -- 172.21.15.63:0/2811824330 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf040ade50 con 0x55c8dede91b0 2024-05-27T07:36:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.955+0000 7fbefd200640 1 -- 172.21.15.63:0/2811824330 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040a9210 con 0x55c8dede91b0 2024-05-27T07:36:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.956+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/2811824330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 msgr2=0x55c8df1a2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.956+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/2811824330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a2170 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbf040a8ec0 tx=0x7fbf040a99a0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.957+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/2811824330 shutdown_connections 2024-05-27T07:36:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.957+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/2811824330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dede91b0 0x55c8df1a2170 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.957+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/2811824330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df1a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.957+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/2811824330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dedc02f0 0x55c8df191480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.957+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/2811824330 >> 172.21.15.63:0/2811824330 conn(0x55c8deddd300 msgr2=0x55c8df197dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.957+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/2811824330 shutdown_connections 2024-05-27T07:36:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.957+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/2811824330 wait complete. 2024-05-27T07:36:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.957+0000 7fbf1a3e6740 1 -- start start 2024-05-27T07:36:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf1a3e6740 1 Processor -- start 2024-05-27T07:36:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dedc02f0 0x55c8df19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df19b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf1a3e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dede91b0 0x55c8df19be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b17c0 con 0x55c8dedc02f0 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b2c30 con 0x55c8dedc84d0 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf1a3e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b1e90 con 0x55c8dede91b0 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf1a3e6740 10 client.0 _ll_register_callbacks cb 0x7fbf04004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf10a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dedc02f0 0x55c8df19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf10a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dedc02f0 0x55c8df19e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59080/0 (socket says 172.21.15.63:59080) 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.958+0000 7fbf10a00640 1 -- 172.21.15.63:0/3202764858 learned_addr learned my addr 172.21.15.63:0/3202764858 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.959+0000 7fbf0be00640 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.959+0000 7fbf11400640 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dede91b0 0x55c8df19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.959+0000 7fbf10a00640 1 -- 172.21.15.63:0/3202764858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dede91b0 msgr2=0x55c8df19be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.959+0000 7fbf10a00640 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dede91b0 0x55c8df19be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.959+0000 7fbf10a00640 1 -- 172.21.15.63:0/3202764858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 msgr2=0x55c8df19b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.959+0000 7fbf10a00640 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df19b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.959+0000 7fbf10a00640 1 -- 172.21.15.63:0/3202764858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8df19c600 con 0x55c8dedc02f0 2024-05-27T07:36:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.959+0000 7fbf11400640 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dede91b0 0x55c8df19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:36:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.959+0000 7fbf10a00640 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dedc02f0 0x55c8df19e5a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbf000316d0 tx=0x7fbf00009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c46d801ed05e26c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.959+0000 7fbef2000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.960+0000 7fbef2000640 1 -- 172.21.15.63:0/3202764858 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00035040 con 0x55c8dedc02f0 2024-05-27T07:36:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.960+0000 7fbef2000640 1 -- 172.21.15.63:0/3202764858 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fbf00003c40 con 0x55c8dedc02f0 2024-05-27T07:36:51.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.960+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3202764858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf040ade50 con 0x55c8dedc02f0 2024-05-27T07:36:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.960+0000 7fbef2000640 1 -- 172.21.15.63:0/3202764858 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbf00003e40 con 0x55c8dedc02f0 2024-05-27T07:36:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.960+0000 7fbf1a3e6740 10 client.16068 fetch_fsmap learned FSMap version 32 2024-05-27T07:36:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.960+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3202764858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c8df218de0 con 0x55c8dedc02f0 2024-05-27T07:36:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.961+0000 7fbef2000640 1 -- 172.21.15.63:0/3202764858 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf040ade50 con 0x55c8dedc02f0 2024-05-27T07:36:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.961+0000 7fbef2000640 1 -- 172.21.15.63:0/3202764858 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbf0005a070 con 0x55c8dedc02f0 2024-05-27T07:36:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.961+0000 7fbf1a3e6740 10 client.16068 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:36:51.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.961+0000 7fbf1a3e6740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.961+0000 7fbf1a3e6740 20 client.16068 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.961+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c8ded7d1c0 0x55c8df21b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.961+0000 7fbf1a3e6740 4 client.16068 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:36:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.961+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3202764858 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8df0f82c0 con 0x55c8ded7d1c0 2024-05-27T07:36:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.962+0000 7fbf0be00640 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c8ded7d1c0 0x55c8df21b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.963+0000 7fbf0be00640 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c8ded7d1c0 0x55c8df21b5b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe4a54918b00b3 server_cookie=dc982ccca2ed4d06 in_seq=0 out_seq=0 2024-05-27T07:36:51.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:51.963+0000 7fbef2000640 10 client.16068 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.007+0000 7fbef2000640 1 -- 172.21.15.63:0/3202764858 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c8df0f82c0 con 0x55c8ded7d1c0 2024-05-27T07:36:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.007+0000 7fbef2000640 10 client.16068 handle_command_reply: tid=0 2024-05-27T07:36:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.007+0000 7fbf1a3e6740 1 client.16068 shutdown 2024-05-27T07:36:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3202764858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dedc02f0 msgr2=0x55c8df19e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dedc02f0 0x55c8df19e5a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbf000316d0 tx=0x7fbf00009260 comp rx=0 tx=0).stop 2024-05-27T07:36:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3202764858 shutdown_connections 2024-05-27T07:36:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c8ded7d1c0 0x55c8df21b5b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c8dede91b0 0x55c8df19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8dedc84d0 0x55c8df19b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 1 --2- 172.21.15.63:0/3202764858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c8dedc02f0 0x55c8df19e5a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbef2000640 0 client.16068 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3202764858 >> 172.21.15.63:0/3202764858 conn(0x55c8deddd300 msgr2=0x55c8df197ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3202764858 shutdown_connections 2024-05-27T07:36:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 1 -- 172.21.15.63:0/3202764858 wait complete. 2024-05-27T07:36:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.008+0000 7fbf1a3e6740 20 client.16068 trim_cache size 0 max 16384 2024-05-27T07:36:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:36:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:36:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:36:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.009+0000 7fbf11e00640 1 -- 172.21.15.63:0/644274925 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbef40b2950 msgr2=0x7fbef40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.009+0000 7fbf11e00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbef40b2950 0x7fbef40b4e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbf000057c0 tx=0x7fbf000058d0 comp rx=0 tx=0).stop 2024-05-27T07:36:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.009+0000 7fbf11e00640 1 -- 172.21.15.63:0/644274925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf0c110300 msgr2=0x7fbf0c19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.009+0000 7fbf11e00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf0c110300 0x7fbf0c19c6a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400eaa0 tx=0x7fbf0400ef60 comp rx=0 tx=0).stop 2024-05-27T07:36:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.010+0000 7fbf11e00640 1 -- 172.21.15.63:0/644274925 shutdown_connections 2024-05-27T07:36:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.010+0000 7fbf11e00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbef40b2950 0x7fbef40b4e60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.010+0000 7fbf11e00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf0c110300 0x7fbf0c19c6a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.010+0000 7fbf11e00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf0c10d9a0 0x7fbf0c1a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.010+0000 7fbf11e00640 1 --2- 172.21.15.63:0/644274925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf0c075420 0x7fbf0c1a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.010+0000 7fbf11e00640 1 -- 172.21.15.63:0/644274925 >> 172.21.15.63:0/644274925 conn(0x7fbf0c079140 msgr2=0x7fbf0c102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.010+0000 7fbf11e00640 1 -- 172.21.15.63:0/644274925 shutdown_connections 2024-05-27T07:36:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:36:52.010+0000 7fbf11e00640 1 -- 172.21.15.63:0/644274925 wait complete. 2024-05-27T07:36:53.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.529+0000 7f9a7b000700 1 -- 172.21.15.63:0/3017086126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a741043f0 msgr2=0x7f9a74104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.529+0000 7f9a7b000700 1 --2- 172.21.15.63:0/3017086126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a741043f0 0x7f9a74104800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9a680099c0 tx=0x7f9a68009cd0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.526 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:53 smithi063 ceph-mon[31613]: pgmap v103: 129 pgs: 129 active+clean; 133 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.7 MiB/s wr, 37 op/s 2024-05-27T07:36:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.529+0000 7f9a7b000700 1 -- 172.21.15.63:0/3017086126 shutdown_connections 2024-05-27T07:36:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.529+0000 7f9a7b000700 1 --2- 172.21.15.63:0/3017086126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a74109670 0x7f9a74110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.529+0000 7f9a7b000700 1 --2- 172.21.15.63:0/3017086126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a741043f0 0x7f9a74104800 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.529+0000 7f9a7b000700 1 --2- 172.21.15.63:0/3017086126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a74108d20 0x7f9a74109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.529+0000 7f9a7b000700 1 -- 172.21.15.63:0/3017086126 >> 172.21.15.63:0/3017086126 conn(0x7f9a74079d00 msgr2=0x7f9a7407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.530+0000 7f9a7b000700 1 -- 172.21.15.63:0/3017086126 shutdown_connections 2024-05-27T07:36:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.530+0000 7f9a7b000700 1 -- 172.21.15.63:0/3017086126 wait complete. 2024-05-27T07:36:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.531+0000 7f9a7b000700 1 Processor -- start 2024-05-27T07:36:53.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.532+0000 7f9a7b000700 1 -- start start 2024-05-27T07:36:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.532+0000 7f9a7b000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a741043f0 0x7f9a7419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.532+0000 7f9a7b000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a74108d20 0x7f9a7419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.532+0000 7f9a7b000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a74109670 0x7f9a741a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.532+0000 7f9a7b000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a74114150 con 0x7f9a74108d20 2024-05-27T07:36:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.532+0000 7f9a7b000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a74113fd0 con 0x7f9a741043f0 2024-05-27T07:36:53.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.532+0000 7f9a7b000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9a741142d0 con 0x7f9a74109670 2024-05-27T07:36:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.533+0000 7f9a79200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a74108d20 0x7f9a7419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.533+0000 7f9a79200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a74108d20 0x7f9a7419d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59110/0 (socket says 172.21.15.63:59110) 2024-05-27T07:36:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.533+0000 7f9a79200700 1 -- 172.21.15.63:0/842018327 learned_addr learned my addr 172.21.15.63:0/842018327 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.533+0000 7f9a79c00700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a741043f0 0x7f9a7419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.533+0000 7f9a7a600700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a74109670 0x7f9a741a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.533+0000 7f9a79200700 1 -- 172.21.15.63:0/842018327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a74109670 msgr2=0x7f9a741a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.533+0000 7f9a79200700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a74109670 0x7f9a741a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.533+0000 7f9a79200700 1 -- 172.21.15.63:0/842018327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a741043f0 msgr2=0x7f9a7419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.533+0000 7f9a79200700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a741043f0 0x7f9a7419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.534+0000 7f9a79200700 1 -- 172.21.15.63:0/842018327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a741a1b80 con 0x7f9a74108d20 2024-05-27T07:36:53.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.534+0000 7f9a79200700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a74108d20 0x7f9a7419d300 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000d980 tx=0x7f9a6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:53.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.534+0000 7f9a72000700 1 -- 172.21.15.63:0/842018327 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60017070 con 0x7f9a74108d20 2024-05-27T07:36:53.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.534+0000 7f9a72000700 1 -- 172.21.15.63:0/842018327 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a60009dc0 con 0x7f9a74108d20 2024-05-27T07:36:53.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.534+0000 7f9a7b000700 1 -- 172.21.15.63:0/842018327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a741a1e70 con 0x7f9a74108d20 2024-05-27T07:36:53.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.535+0000 7f9a72000700 1 -- 172.21.15.63:0/842018327 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6000f3c0 con 0x7f9a74108d20 2024-05-27T07:36:53.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.536+0000 7f9a7b000700 1 -- 172.21.15.63:0/842018327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a741a23b0 con 0x7f9a74108d20 2024-05-27T07:36:53.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.540+0000 7f9a72000700 1 -- 172.21.15.63:0/842018327 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9a60009930 con 0x7f9a74108d20 2024-05-27T07:36:53.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.542+0000 7f9a8b067b80 1 Processor -- start 2024-05-27T07:36:53.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.542+0000 7f9a8b067b80 1 -- start start 2024-05-27T07:36:53.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a72000700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a5c0b29f0 0x7f9a5c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677a8f9f0 0x55d677b23750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6779a3e40 0x55d677b23c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a72000700 1 -- 172.21.15.63:0/842018327 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f9a600e1e10 con 0x7f9a74108d20 2024-05-27T07:36:53.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677b2d2c0 0x55d677b2f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d677a50e40 con 0x55d6779a3e40 2024-05-27T07:36:53.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d677a409e0 con 0x55d677a8f9f0 2024-05-27T07:36:53.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d677a44a20 con 0x55d677b2d2c0 2024-05-27T07:36:53.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a7a600700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a5c0b29f0 0x7f9a5c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a79200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6779a3e40 0x55d677b23c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.543+0000 7f9a79200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6779a3e40 0x55d677b23c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59122/0 (socket says 172.21.15.63:59122) 2024-05-27T07:36:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.544+0000 7f9a79200700 1 -- 172.21.15.63:0/240590794 learned_addr learned my addr 172.21.15.63:0/240590794 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.544+0000 7f9a7a600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677b2d2c0 0x55d677b2f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.544+0000 7f9a79c00700 1 --2- 172.21.15.63:0/240590794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677a8f9f0 0x55d677b23750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.544+0000 7f9a79200700 1 -- 172.21.15.63:0/240590794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677b2d2c0 msgr2=0x55d677b2f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.544+0000 7f9a79200700 1 --2- 172.21.15.63:0/240590794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677b2d2c0 0x55d677b2f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.544+0000 7f9a79200700 1 -- 172.21.15.63:0/240590794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677a8f9f0 msgr2=0x55d677b23750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.544+0000 7f9a79200700 1 --2- 172.21.15.63:0/240590794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677a8f9f0 0x55d677b23750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.544+0000 7f9a79200700 1 -- 172.21.15.63:0/240590794 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d677a09420 con 0x55d6779a3e40 2024-05-27T07:36:53.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.545+0000 7f9a79200700 1 --2- 172.21.15.63:0/240590794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6779a3e40 0x55d677b23c90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9a600a8d10 tx=0x7f9a60009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ea6457131ad269 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:53.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.545+0000 7f9a7a600700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a5c0b29f0 0x7f9a5c0b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c007fd0 tx=0x7f9a6c0132f0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:53.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.545+0000 7f9a66a00700 1 -- 172.21.15.63:0/240590794 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60020070 con 0x55d6779a3e40 2024-05-27T07:36:53.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.545+0000 7f9a66a00700 1 -- 172.21.15.63:0/240590794 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a600a8d80 con 0x55d6779a3e40 2024-05-27T07:36:53.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.545+0000 7f9a66a00700 1 -- 172.21.15.63:0/240590794 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a600a8fe0 con 0x55d6779a3e40 2024-05-27T07:36:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.546+0000 7f9a8b067b80 1 -- 172.21.15.63:0/240590794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6779a3e40 msgr2=0x55d677b23c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.546+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/240590794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6779a3e40 0x55d677b23c90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9a600a8d10 tx=0x7f9a60009d50 comp rx=0 tx=0).stop 2024-05-27T07:36:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.547+0000 7f9a8b067b80 1 -- 172.21.15.63:0/240590794 shutdown_connections 2024-05-27T07:36:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.547+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/240590794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677b2d2c0 0x55d677b2f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.547+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/240590794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677a8f9f0 0x55d677b23750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.547+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/240590794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6779a3e40 0x55d677b23c90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.547+0000 7f9a8b067b80 1 -- 172.21.15.63:0/240590794 >> 172.21.15.63:0/240590794 conn(0x55d677a3d390 msgr2=0x55d677b1e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.547+0000 7f9a8b067b80 1 -- 172.21.15.63:0/240590794 shutdown_connections 2024-05-27T07:36:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.547+0000 7f9a8b067b80 1 -- 172.21.15.63:0/240590794 wait complete. 2024-05-27T07:36:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.548+0000 7f9a8b067b80 1 -- start start 2024-05-27T07:36:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.548+0000 7f9a8b067b80 1 Processor -- start 2024-05-27T07:36:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.548+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 0x55d677b2a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.548+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677a8f9f0 0x55d677b2a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.548+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677b2d2c0 0x55d677b2ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.548+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d677946fa0 con 0x55d677b2d2c0 2024-05-27T07:36:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.548+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d6779e1760 con 0x55d6779a3e40 2024-05-27T07:36:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.548+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d677b2fda0 con 0x55d677a8f9f0 2024-05-27T07:36:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a8b067b80 10 client.0 _ll_register_callbacks cb 0x55d6779eac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:53.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a7a600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677b2d2c0 0x55d677b2ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a7a600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677b2d2c0 0x55d677b2ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59130/0 (socket says 172.21.15.63:59130) 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a7a600700 1 -- 172.21.15.63:0/2011783376 learned_addr learned my addr 172.21.15.63:0/2011783376 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a79200700 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677a8f9f0 0x55d677b2a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a79c00700 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 0x55d677b2a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a7a600700 1 -- 172.21.15.63:0/2011783376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677a8f9f0 msgr2=0x55d677b2a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a7a600700 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677a8f9f0 0x55d677b2a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a7a600700 1 -- 172.21.15.63:0/2011783376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 msgr2=0x55d677b2a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a7a600700 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 0x55d677b2a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.549+0000 7f9a7a600700 1 -- 172.21.15.63:0/2011783376 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6779908d0 con 0x55d677b2d2c0 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.550+0000 7f9a79c00700 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 0x55d677b2a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.550+0000 7f9a7a600700 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677b2d2c0 0x55d677b2ad20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c007ea0 tx=0x7f9a6c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53f6ecdda0d3f3f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.550+0000 7f9a5b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:53.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.550+0000 7f9a5b400700 1 -- 172.21.15.63:0/2011783376 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c01f040 con 0x55d677b2d2c0 2024-05-27T07:36:53.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.550+0000 7f9a8b067b80 1 -- 172.21.15.63:0/2011783376 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a600a8d80 con 0x55d677b2d2c0 2024-05-27T07:36:53.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.550+0000 7f9a5b400700 1 -- 172.21.15.63:0/2011783376 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f9a6c007a00 con 0x55d677b2d2c0 2024-05-27T07:36:53.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.551+0000 7f9a5b400700 1 -- 172.21.15.63:0/2011783376 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9a6c031050 con 0x55d677b2d2c0 2024-05-27T07:36:53.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.551+0000 7f9a8b067b80 10 client.16080 fetch_fsmap learned FSMap version 32 2024-05-27T07:36:53.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.551+0000 7f9a8b067b80 1 -- 172.21.15.63:0/2011783376 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d677a11060 con 0x55d677b2d2c0 2024-05-27T07:36:53.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.551+0000 7f9a5b400700 1 -- 172.21.15.63:0/2011783376 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a600a8d80 con 0x55d677b2d2c0 2024-05-27T07:36:53.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.551+0000 7f9a5b400700 1 -- 172.21.15.63:0/2011783376 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9a6c016040 con 0x55d677b2d2c0 2024-05-27T07:36:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.552+0000 7f9a8b067b80 10 client.16080 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:36:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.552+0000 7f9a8b067b80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.552+0000 7f9a8b067b80 20 client.16080 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.552+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d677b3cde0 0x55d677b3f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.552+0000 7f9a8b067b80 4 client.16080 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:36:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.552+0000 7f9a8b067b80 1 -- 172.21.15.63:0/2011783376 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d677a52d90 con 0x55d677b3cde0 2024-05-27T07:36:53.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.552+0000 7f9a79c00700 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d677b3cde0 0x55d677b3f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.553+0000 7f9a79c00700 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d677b3cde0 0x55d677b3f290 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cddde91210892e3 server_cookie=abe7c30e49d14991 in_seq=0 out_seq=0 2024-05-27T07:36:53.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.553+0000 7f9a5b400700 10 client.16080 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:53.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.555+0000 7f9a5b400700 1 -- 172.21.15.63:0/2011783376 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d677a52d90 con 0x55d677b3cde0 2024-05-27T07:36:53.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.555+0000 7f9a5b400700 10 client.16080 handle_command_reply: tid=0 2024-05-27T07:36:53.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.555+0000 7f9a8b067b80 1 client.16080 shutdown 2024-05-27T07:36:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a8b067b80 1 -- 172.21.15.63:0/2011783376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677b2d2c0 msgr2=0x55d677b2ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677b2d2c0 0x55d677b2ad20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c007ea0 tx=0x7f9a6c01a000 comp rx=0 tx=0).stop 2024-05-27T07:36:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a8b067b80 1 -- 172.21.15.63:0/2011783376 shutdown_connections 2024-05-27T07:36:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d677b3cde0 0x55d677b3f290 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677b2d2c0 0x55d677b2ad20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 0x55d677b2a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a5b400700 0 client.16080 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:53.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/2011783376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677a8f9f0 0x55d677b2a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a8b067b80 1 -- 172.21.15.63:0/2011783376 >> 172.21.15.63:0/2011783376 conn(0x55d677a3d390 msgr2=0x55d677b1e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:53.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a8b067b80 1 -- 172.21.15.63:0/2011783376 shutdown_connections 2024-05-27T07:36:53.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.556+0000 7f9a8b067b80 1 -- 172.21.15.63:0/2011783376 wait complete. 2024-05-27T07:36:53.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.557+0000 7f9a8b067b80 20 client.16080 trim_cache size 0 max 16384 2024-05-27T07:36:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.585+0000 7f9a8b067b80 1 Processor -- start 2024-05-27T07:36:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.585+0000 7f9a8b067b80 1 -- start start 2024-05-27T07:36:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.585+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6779a3e40 0x55d677b1e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.585+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.585+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677b2d2c0 0x55d677b38e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.585+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a741b4d00 con 0x55d677a8f9f0 2024-05-27T07:36:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.585+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a741b2530 con 0x55d677b2d2c0 2024-05-27T07:36:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.585+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9a741b0380 con 0x55d6779a3e40 2024-05-27T07:36:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a79200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a79200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b295c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59140/0 (socket says 172.21.15.63:59140) 2024-05-27T07:36:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a79200700 1 -- 172.21.15.63:0/1069191388 learned_addr learned my addr 172.21.15.63:0/1069191388 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a79c00700 1 --2- 172.21.15.63:0/1069191388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6779a3e40 0x55d677b1e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a7a600700 1 --2- 172.21.15.63:0/1069191388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677b2d2c0 0x55d677b38e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a79200700 1 -- 172.21.15.63:0/1069191388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6779a3e40 msgr2=0x55d677b1e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a79200700 1 --2- 172.21.15.63:0/1069191388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6779a3e40 0x55d677b1e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a79200700 1 -- 172.21.15.63:0/1069191388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677b2d2c0 msgr2=0x55d677b38e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a79200700 1 --2- 172.21.15.63:0/1069191388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677b2d2c0 0x55d677b38e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a79200700 1 -- 172.21.15.63:0/1069191388 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d677b2cb40 con 0x55d677a8f9f0 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a7a600700 1 --2- 172.21.15.63:0/1069191388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677b2d2c0 0x55d677b38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.586+0000 7f9a79200700 1 --2- 172.21.15.63:0/1069191388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b295c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9a600a8a80 tx=0x7f9a600a8b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6019fe3514d1eae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.587+0000 7f9a66000700 1 -- 172.21.15.63:0/1069191388 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60020040 con 0x55d677a8f9f0 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.587+0000 7f9a66000700 1 -- 172.21.15.63:0/1069191388 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a600ad130 con 0x55d677a8f9f0 2024-05-27T07:36:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.587+0000 7f9a66000700 1 -- 172.21.15.63:0/1069191388 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a600b1750 con 0x55d677a8f9f0 2024-05-27T07:36:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.588+0000 7f9a8b067b80 1 -- 172.21.15.63:0/1069191388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 msgr2=0x55d677b295c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.588+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/1069191388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b295c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9a600a8a80 tx=0x7f9a600a8b50 comp rx=0 tx=0).stop 2024-05-27T07:36:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.588+0000 7f9a8b067b80 1 -- 172.21.15.63:0/1069191388 shutdown_connections 2024-05-27T07:36:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.588+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/1069191388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d677b2d2c0 0x55d677b38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.588+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/1069191388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6779a3e40 0x55d677b1e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.588+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/1069191388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b295c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.588+0000 7f9a8b067b80 1 -- 172.21.15.63:0/1069191388 >> 172.21.15.63:0/1069191388 conn(0x55d677a3d390 msgr2=0x55d677b22120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.588+0000 7f9a8b067b80 1 -- 172.21.15.63:0/1069191388 shutdown_connections 2024-05-27T07:36:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.588+0000 7f9a8b067b80 1 -- 172.21.15.63:0/1069191388 wait complete. 2024-05-27T07:36:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.589+0000 7f9a8b067b80 1 -- start start 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.589+0000 7f9a8b067b80 1 Processor -- start 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 0x55d677b35880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b30c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a8b067b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677b2d2c0 0x55d677b312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a741b09f0 con 0x55d677a8f9f0 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a741b1e60 con 0x55d6779a3e40 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a8b067b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9a741b10c0 con 0x55d677b2d2c0 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a8b067b80 10 client.0 _ll_register_callbacks cb 0x55d6779eac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a79200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b30c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a79200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b30c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59152/0 (socket says 172.21.15.63:59152) 2024-05-27T07:36:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a79200700 1 -- 172.21.15.63:0/3520628236 learned_addr learned my addr 172.21.15.63:0/3520628236 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:36:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.590+0000 7f9a79c00700 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 0x55d677b35880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.591+0000 7f9a7a600700 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677b2d2c0 0x55d677b312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.591+0000 7f9a79200700 1 -- 172.21.15.63:0/3520628236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677b2d2c0 msgr2=0x55d677b312b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.591+0000 7f9a79200700 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677b2d2c0 0x55d677b312b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.591+0000 7f9a79200700 1 -- 172.21.15.63:0/3520628236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 msgr2=0x55d677b35880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:36:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.591+0000 7f9a79200700 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 0x55d677b35880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.591+0000 7f9a79200700 1 -- 172.21.15.63:0/3520628236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d677b31a50 con 0x55d677a8f9f0 2024-05-27T07:36:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.591+0000 7f9a79c00700 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 0x55d677b35880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.591+0000 7f9a79200700 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b30c80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9a600a8d10 tx=0x7f9a600a9340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1aaa9e8f071ba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.591+0000 7f9a5b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:36:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.591+0000 7f9a5b400700 1 -- 172.21.15.63:0/3520628236 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60087060 con 0x55d677a8f9f0 2024-05-27T07:36:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.592+0000 7f9a5b400700 1 -- 172.21.15.63:0/3520628236 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f9a60010460 con 0x55d677a8f9f0 2024-05-27T07:36:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.592+0000 7f9a8b067b80 1 -- 172.21.15.63:0/3520628236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a600ad130 con 0x55d677a8f9f0 2024-05-27T07:36:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.592+0000 7f9a5b400700 1 -- 172.21.15.63:0/3520628236 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9a6008e050 con 0x55d677a8f9f0 2024-05-27T07:36:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.592+0000 7f9a8b067b80 10 client.16086 fetch_fsmap learned FSMap version 32 2024-05-27T07:36:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.592+0000 7f9a8b067b80 1 -- 172.21.15.63:0/3520628236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d677ba7950 con 0x55d677a8f9f0 2024-05-27T07:36:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.592+0000 7f9a5b400700 1 -- 172.21.15.63:0/3520628236 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a600ad130 con 0x55d677a8f9f0 2024-05-27T07:36:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.592+0000 7f9a5b400700 1 -- 172.21.15.63:0/3520628236 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9a600a7b70 con 0x55d677a8f9f0 2024-05-27T07:36:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.593+0000 7f9a8b067b80 10 client.16086 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:36:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.593+0000 7f9a8b067b80 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:36:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.593+0000 7f9a8b067b80 20 client.16086 populate_metadata read hostname 'smithi063' 2024-05-27T07:36:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.593+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d677ba7b20 0x55d677ba9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.593+0000 7f9a8b067b80 4 client.16086 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:36:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.593+0000 7f9a8b067b80 1 -- 172.21.15.63:0/3520628236 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d677b24160 con 0x55d677ba7b20 2024-05-27T07:36:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.593+0000 7f9a79c00700 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d677ba7b20 0x55d677ba9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.594+0000 7f9a79c00700 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d677ba7b20 0x55d677ba9f40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c2b02e767274b4 server_cookie=a246f207365e3eab in_seq=0 out_seq=0 2024-05-27T07:36:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.594+0000 7f9a5b400700 10 client.16086 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:53.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a5b400700 1 -- 172.21.15.63:0/3520628236 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d677b24160 con 0x55d677ba7b20 2024-05-27T07:36:53.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a5b400700 10 client.16086 handle_command_reply: tid=0 2024-05-27T07:36:53.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a8b067b80 1 client.16086 shutdown 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a8b067b80 1 -- 172.21.15.63:0/3520628236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 msgr2=0x55d677b30c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b30c80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9a600a8d10 tx=0x7f9a600a9340 comp rx=0 tx=0).stop 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a8b067b80 1 -- 172.21.15.63:0/3520628236 shutdown_connections 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d677ba7b20 0x55d677ba9f40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d677b2d2c0 0x55d677b312b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6779a3e40 0x55d677b35880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a8b067b80 1 --2- 172.21.15.63:0/3520628236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d677a8f9f0 0x55d677b30c80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a5b400700 0 client.16086 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.595+0000 7f9a8b067b80 1 -- 172.21.15.63:0/3520628236 >> 172.21.15.63:0/3520628236 conn(0x55d677a3d390 msgr2=0x55d677b29320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.596+0000 7f9a8b067b80 1 -- 172.21.15.63:0/3520628236 shutdown_connections 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.596+0000 7f9a8b067b80 1 -- 172.21.15.63:0/3520628236 wait complete. 2024-05-27T07:36:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.596+0000 7f9a8b067b80 20 client.16086 trim_cache size 0 max 16384 2024-05-27T07:36:53.593 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:36:53.593 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:36:53.593 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:36:53.593 INFO:teuthology.orchestra.run.smithi063.stdout: "4fd28b72-bd36-41d3-a048-928e0a5da755": { 2024-05-27T07:36:53.593 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:36:53.593 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "4fd28b72-bd36-41d3-a048-928e0a5da755", 2024-05-27T07:36:53.593 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:36:53.593 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:36:53.593 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:36:53.593 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:36:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 -- 172.21.15.63:0/842018327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a5c0b29f0 msgr2=0x7f9a5c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a5c0b29f0 0x7f9a5c0b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c007fd0 tx=0x7f9a6c0132f0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 -- 172.21.15.63:0/842018327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a74108d20 msgr2=0x7f9a7419d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a74108d20 0x7f9a7419d300 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000d980 tx=0x7f9a6000dc90 comp rx=0 tx=0).stop 2024-05-27T07:36:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 -- 172.21.15.63:0/842018327 shutdown_connections 2024-05-27T07:36:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a5c0b29f0 0x7f9a5c0b4f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a74109670 0x7f9a741a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a741043f0 0x7f9a7419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 --2- 172.21.15.63:0/842018327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a74108d20 0x7f9a7419d300 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:53.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 -- 172.21.15.63:0/842018327 >> 172.21.15.63:0/842018327 conn(0x7f9a74079d00 msgr2=0x7f9a74102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:53.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 -- 172.21.15.63:0/842018327 shutdown_connections 2024-05-27T07:36:53.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:36:53.597+0000 7f9a7b000700 1 -- 172.21.15.63:0/842018327 wait complete. 2024-05-27T07:36:53.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:53 smithi086 ceph-mon[34687]: pgmap v103: 129 pgs: 129 active+clean; 133 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.7 MiB/s wr, 37 op/s 2024-05-27T07:36:53.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:53 smithi149 ceph-mon[35408]: pgmap v103: 129 pgs: 129 active+clean; 133 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.7 MiB/s wr, 37 op/s 2024-05-27T07:36:54.609 INFO:tasks.cephfs.filesystem:scrub status for tag:4fd28b72-bd36-41d3-a048-928e0a5da755 - {'path': '/', 'tag': '4fd28b72-bd36-41d3-a048-928e0a5da755', 'options': 'recursive,force'} 2024-05-27T07:36:56.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:56 smithi086 ceph-mon[34687]: pgmap v104: 129 pgs: 129 active+clean; 133 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.0 MiB/s wr, 32 op/s 2024-05-27T07:36:56.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:56 smithi149 ceph-mon[35408]: pgmap v104: 129 pgs: 129 active+clean; 133 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.0 MiB/s wr, 32 op/s 2024-05-27T07:36:56.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:56 smithi063 ceph-mon[31613]: pgmap v104: 129 pgs: 129 active+clean; 133 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.0 MiB/s wr, 32 op/s 2024-05-27T07:36:57.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:57 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:36:57.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:57 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:36:57.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:57 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:36:57.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:57 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:36:57.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:57 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:36:57.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:57 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:36:57.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:57 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:36:57.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:57 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:36:57.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:57 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:36:58.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:58 smithi086 ceph-mon[34687]: pgmap v105: 129 pgs: 129 active+clean; 165 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 39 op/s 2024-05-27T07:36:58.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:58 smithi149 ceph-mon[35408]: pgmap v105: 129 pgs: 129 active+clean; 165 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 39 op/s 2024-05-27T07:36:58.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:58 smithi063 ceph-mon[31613]: pgmap v105: 129 pgs: 129 active+clean; 165 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 39 op/s 2024-05-27T07:36:59.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:36:59 smithi086 ceph-mon[34687]: pgmap v106: 129 pgs: 129 active+clean; 165 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.0 MiB/s wr, 35 op/s 2024-05-27T07:36:59.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:36:59 smithi149 ceph-mon[35408]: pgmap v106: 129 pgs: 129 active+clean; 165 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.0 MiB/s wr, 35 op/s 2024-05-27T07:36:59.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:36:59 smithi063 ceph-mon[31613]: pgmap v106: 129 pgs: 129 active+clean; 165 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.0 MiB/s wr, 35 op/s 2024-05-27T07:37:00.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:00 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:00.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:00 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:00.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:00 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:01.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:01 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:01.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:01 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:01.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:01 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:37:01.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:01 smithi086 ceph-mon[34687]: pgmap v107: 129 pgs: 129 active+clean; 188 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.0 MiB/s wr, 53 op/s 2024-05-27T07:37:01.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:01 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:01.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:01 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:01.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:01 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:37:01.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:01 smithi149 ceph-mon[35408]: pgmap v107: 129 pgs: 129 active+clean; 188 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.0 MiB/s wr, 53 op/s 2024-05-27T07:37:01.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:01 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:01.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:01 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:01.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:01 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:37:01.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:01 smithi063 ceph-mon[31613]: pgmap v107: 129 pgs: 129 active+clean; 188 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.0 MiB/s wr, 53 op/s 2024-05-27T07:37:03.704 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:03 smithi086 ceph-mon[34687]: pgmap v108: 129 pgs: 129 active+clean; 188 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-05-27T07:37:03.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:03 smithi149 ceph-mon[35408]: pgmap v108: 129 pgs: 129 active+clean; 188 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-05-27T07:37:03.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:03 smithi063 ceph-mon[31613]: pgmap v108: 129 pgs: 129 active+clean; 188 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-05-27T07:37:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.261+0000 7f3456a00640 1 -- 172.21.15.63:0/2119525739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f345010a890 msgr2=0x7f3450116b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.261+0000 7f3456a00640 1 --2- 172.21.15.63:0/2119525739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f345010a890 0x7f3450116b50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3448009ca0 tx=0x7f344802a950 comp rx=0 tx=0).stop 2024-05-27T07:37:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.262+0000 7f3456a00640 1 -- 172.21.15.63:0/2119525739 shutdown_connections 2024-05-27T07:37:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.262+0000 7f3456a00640 1 --2- 172.21.15.63:0/2119525739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f345010a890 0x7f3450116b50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.262+0000 7f3456a00640 1 --2- 172.21.15.63:0/2119525739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450110520 0x7f345010a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.262+0000 7f3456a00640 1 --2- 172.21.15.63:0/2119525739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450110110 0x7f3450109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.262+0000 7f3456a00640 1 -- 172.21.15.63:0/2119525739 >> 172.21.15.63:0/2119525739 conn(0x7f3450076070 msgr2=0x7f3450078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.262+0000 7f3456a00640 1 -- 172.21.15.63:0/2119525739 shutdown_connections 2024-05-27T07:37:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.262+0000 7f3456a00640 1 -- 172.21.15.63:0/2119525739 wait complete. 2024-05-27T07:37:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.263+0000 7f3456a00640 1 Processor -- start 2024-05-27T07:37:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.264+0000 7f3456a00640 1 -- start start 2024-05-27T07:37:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.264+0000 7f3456a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450110110 0x7f34501a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.264+0000 7f3456a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450110520 0x7f34501a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.264+0000 7f3456a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34501a4f40 0x7f34501a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.264+0000 7f3456a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f345011a180 con 0x7f34501a4f40 2024-05-27T07:37:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.264+0000 7f3456a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f345011a000 con 0x7f3450110110 2024-05-27T07:37:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.264+0000 7f3456a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f345011a300 con 0x7f3450110520 2024-05-27T07:37:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.265+0000 7f3456000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34501a4f40 0x7f34501a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.265+0000 7f3456000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34501a4f40 0x7f34501a5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45124/0 (socket says 172.21.15.63:45124) 2024-05-27T07:37:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.265+0000 7f3456000640 1 -- 172.21.15.63:0/728611984 learned_addr learned my addr 172.21.15.63:0/728611984 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.265+0000 7f3455600640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450110110 0x7f34501a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.265+0000 7f3454c00640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450110520 0x7f34501a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.266+0000 7f3456000640 1 -- 172.21.15.63:0/728611984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450110520 msgr2=0x7f34501a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.266+0000 7f3456000640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450110520 0x7f34501a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.266+0000 7f3456000640 1 -- 172.21.15.63:0/728611984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450110110 msgr2=0x7f34501a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.266+0000 7f3456000640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450110110 0x7f34501a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.266+0000 7f3456000640 1 -- 172.21.15.63:0/728611984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34501a5990 con 0x7f34501a4f40 2024-05-27T07:37:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.266+0000 7f3456000640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34501a4f40 0x7f34501a5350 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f344802ae70 tx=0x7f3448003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.267+0000 7f344e000640 1 -- 172.21.15.63:0/728611984 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448034040 con 0x7f34501a4f40 2024-05-27T07:37:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.267+0000 7f344e000640 1 -- 172.21.15.63:0/728611984 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3448030850 con 0x7f34501a4f40 2024-05-27T07:37:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.267+0000 7f3456a00640 1 -- 172.21.15.63:0/728611984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34501a5c20 con 0x7f34501a4f40 2024-05-27T07:37:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.267+0000 7f344e000640 1 -- 172.21.15.63:0/728611984 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448004600 con 0x7f34501a4f40 2024-05-27T07:37:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.267+0000 7f3456a00640 1 -- 172.21.15.63:0/728611984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34501ad420 con 0x7f34501a4f40 2024-05-27T07:37:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.271+0000 7f345f16d740 1 Processor -- start 2024-05-27T07:37:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.271+0000 7f344e000640 1 -- 172.21.15.63:0/728611984 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3448037020 con 0x7f34501a4f40 2024-05-27T07:37:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.271+0000 7f345f16d740 1 -- start start 2024-05-27T07:37:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.272+0000 7f345f16d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.272+0000 7f345f16d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.272+0000 7f345f16d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.272+0000 7f345f16d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b96c351b90 con 0x55b96c03e1b0 2024-05-27T07:37:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.272+0000 7f345f16d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b96c341730 con 0x55b96c01d4d0 2024-05-27T07:37:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.272+0000 7f345f16d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b96c345770 con 0x55b96c0152f0 2024-05-27T07:37:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.272+0000 7f3454c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.273+0000 7f3454c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3e0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45138/0 (socket says 172.21.15.63:45138) 2024-05-27T07:37:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.273+0000 7f3454c00640 1 -- 172.21.15.63:0/366756395 learned_addr learned my addr 172.21.15.63:0/366756395 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.273+0000 7f3455600640 1 --2- 172.21.15.63:0/366756395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.273+0000 7f3454c00640 1 --2- 172.21.15.63:0/366756395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.273+0000 7f3454c00640 1 -- 172.21.15.63:0/366756395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 msgr2=0x55b96c3e0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.273+0000 7f3454c00640 1 --2- 172.21.15.63:0/366756395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3e0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.273+0000 7f3454c00640 1 -- 172.21.15.63:0/366756395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 msgr2=0x55b96c3e14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.273+0000 7f3454c00640 1 --2- 172.21.15.63:0/366756395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3e14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.273+0000 7f3454c00640 1 -- 172.21.15.63:0/366756395 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b96c2da8b0 con 0x55b96c03e1b0 2024-05-27T07:37:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.273+0000 7f3454c00640 1 --2- 172.21.15.63:0/366756395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3e0a60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f343c013550 tx=0x7f343c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfc5db2aee6c3b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.274+0000 7f344e000640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34380b29d0 0x7f34380b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.274+0000 7f344e000640 1 -- 172.21.15.63:0/728611984 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f3448103600 con 0x7f34501a4f40 2024-05-27T07:37:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.274+0000 7f3442600640 1 -- 172.21.15.63:0/366756395 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c004250 con 0x55b96c03e1b0 2024-05-27T07:37:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.274+0000 7f3455600640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34380b29d0 0x7f34380b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.274+0000 7f3442600640 1 -- 172.21.15.63:0/366756395 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f343c0043f0 con 0x55b96c03e1b0 2024-05-27T07:37:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.274+0000 7f3442600640 1 -- 172.21.15.63:0/366756395 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c011720 con 0x55b96c03e1b0 2024-05-27T07:37:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.275+0000 7f345f16d740 1 -- 172.21.15.63:0/366756395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 msgr2=0x55b96c3e0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.275+0000 7f3455600640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34380b29d0 0x7f34380b4ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55b96c3e1e10 tx=0x7f344400b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.275+0000 7f345f16d740 1 --2- 172.21.15.63:0/366756395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3e0a60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f343c013550 tx=0x7f343c005850 comp rx=0 tx=0).stop 2024-05-27T07:37:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.275+0000 7f345f16d740 1 -- 172.21.15.63:0/366756395 shutdown_connections 2024-05-27T07:37:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.275+0000 7f345f16d740 1 --2- 172.21.15.63:0/366756395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.275+0000 7f345f16d740 1 --2- 172.21.15.63:0/366756395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.275+0000 7f345f16d740 1 --2- 172.21.15.63:0/366756395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3e0a60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.275+0000 7f345f16d740 1 -- 172.21.15.63:0/366756395 >> 172.21.15.63:0/366756395 conn(0x55b96c032300 msgr2=0x55b96c3dd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.275+0000 7f345f16d740 1 -- 172.21.15.63:0/366756395 shutdown_connections 2024-05-27T07:37:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.275+0000 7f345f16d740 1 -- 172.21.15.63:0/366756395 wait complete. 2024-05-27T07:37:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.276+0000 7f345f16d740 1 -- start start 2024-05-27T07:37:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.276+0000 7f345f16d740 1 Processor -- start 2024-05-27T07:37:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f345f16d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f345f16d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3eb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f345f16d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f345f16d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b96c027790 con 0x55b96c03e1b0 2024-05-27T07:37:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f345f16d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b96c2e9bc0 con 0x55b96c01d4d0 2024-05-27T07:37:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f345f16d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b96c3edb90 con 0x55b96c0152f0 2024-05-27T07:37:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f345f16d740 10 client.0 _ll_register_callbacks cb 0x55b96c2c7400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f3456000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f3456000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45142/0 (socket says 172.21.15.63:45142) 2024-05-27T07:37:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f3456000640 1 -- 172.21.15.63:0/2795727022 learned_addr learned my addr 172.21.15.63:0/2795727022 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.277+0000 7f3455600640 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.278+0000 7f3454c00640 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.278+0000 7f3456000640 1 -- 172.21.15.63:0/2795727022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 msgr2=0x55b96c3eaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.278+0000 7f3456000640 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3eaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.278+0000 7f3456000640 1 -- 172.21.15.63:0/2795727022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 msgr2=0x55b96c3eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.278+0000 7f3456000640 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3eb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.278+0000 7f3456000640 1 -- 172.21.15.63:0/2795727022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b96c056210 con 0x55b96c03e1b0 2024-05-27T07:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.278+0000 7f3455600640 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3eaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.278+0000 7f3454c00640 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.278+0000 7f3456000640 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6ec0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f34480c6700 tx=0x7f34480ca250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf48d7d9b186167b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.278+0000 7f3436a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.279+0000 7f3436a00640 1 -- 172.21.15.63:0/2795727022 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344802c040 con 0x55b96c03e1b0 2024-05-27T07:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.279+0000 7f3436a00640 1 -- 172.21.15.63:0/2795727022 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f34480c6c70 con 0x55b96c03e1b0 2024-05-27T07:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.279+0000 7f345f16d740 1 -- 172.21.15.63:0/2795727022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f343c0043f0 con 0x55b96c03e1b0 2024-05-27T07:37:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.279+0000 7f3436a00640 1 -- 172.21.15.63:0/2795727022 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f34480cea30 con 0x55b96c03e1b0 2024-05-27T07:37:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.279+0000 7f345f16d740 10 client.16098 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.279+0000 7f345f16d740 1 -- 172.21.15.63:0/2795727022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b96c122620 con 0x55b96c03e1b0 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.279+0000 7f3436a00640 1 -- 172.21.15.63:0/2795727022 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f343c0043f0 con 0x55b96c03e1b0 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.280+0000 7f3436a00640 1 -- 172.21.15.63:0/2795727022 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f344804d030 con 0x55b96c03e1b0 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.280+0000 7f345f16d740 10 client.16098 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.280+0000 7f345f16d740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.280+0000 7f345f16d740 20 client.16098 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.280+0000 7f345f16d740 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b96bffcd80 0x55b96c3fcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.280+0000 7f345f16d740 4 client.16098 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.280+0000 7f345f16d740 1 -- 172.21.15.63:0/2795727022 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b96c353ae0 con 0x55b96bffcd80 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.281+0000 7f3455600640 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b96bffcd80 0x55b96c3fcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.282+0000 7f3455600640 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b96bffcd80 0x55b96c3fcfc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3102e210a52aee server_cookie=d93da165d56b48cf in_seq=0 out_seq=0 2024-05-27T07:37:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.282+0000 7f3436a00640 10 client.16098 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f3436a00640 1 -- 172.21.15.63:0/2795727022 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b96c353ae0 con 0x55b96bffcd80 2024-05-27T07:37:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f3436a00640 10 client.16098 handle_command_reply: tid=0 2024-05-27T07:37:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 client.16098 shutdown 2024-05-27T07:37:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 -- 172.21.15.63:0/2795727022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 msgr2=0x55b96c3f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6ec0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f34480c6700 tx=0x7f34480ca250 comp rx=0 tx=0).stop 2024-05-27T07:37:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 -- 172.21.15.63:0/2795727022 shutdown_connections 2024-05-27T07:37:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b96bffcd80 0x55b96c3fcfc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6ec0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 --2- 172.21.15.63:0/2795727022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3eaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 -- 172.21.15.63:0/2795727022 >> 172.21.15.63:0/2795727022 conn(0x55b96c032300 msgr2=0x55b96c3e0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f3436a00640 0 client.16098 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 -- 172.21.15.63:0/2795727022 shutdown_connections 2024-05-27T07:37:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 1 -- 172.21.15.63:0/2795727022 wait complete. 2024-05-27T07:37:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.284+0000 7f345f16d740 20 client.16098 trim_cache size 0 max 16384 2024-05-27T07:37:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.306+0000 7f345f16d740 1 Processor -- start 2024-05-27T07:37:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.306+0000 7f345f16d740 1 -- start start 2024-05-27T07:37:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.307+0000 7f345f16d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.307+0000 7f345f16d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3dbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.307+0000 7f345f16d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.307+0000 7f345f16d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34501b9c40 con 0x55b96c03e1b0 2024-05-27T07:37:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.307+0000 7f345f16d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34501b74b0 con 0x55b96c01d4d0 2024-05-27T07:37:05.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.307+0000 7f345f16d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34501b5300 con 0x55b96c0152f0 2024-05-27T07:37:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.307+0000 7f3456000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.307+0000 7f3456000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45154/0 (socket says 172.21.15.63:45154) 2024-05-27T07:37:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.307+0000 7f3456000640 1 -- 172.21.15.63:0/4066455538 learned_addr learned my addr 172.21.15.63:0/4066455538 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.308+0000 7f3454c00640 1 --2- 172.21.15.63:0/4066455538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3dbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.308+0000 7f3455600640 1 --2- 172.21.15.63:0/4066455538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.308+0000 7f3456000640 1 -- 172.21.15.63:0/4066455538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 msgr2=0x55b96c3e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.308+0000 7f3456000640 1 --2- 172.21.15.63:0/4066455538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.308+0000 7f3456000640 1 -- 172.21.15.63:0/4066455538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 msgr2=0x55b96c3dbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.308+0000 7f3456000640 1 --2- 172.21.15.63:0/4066455538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3dbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.308+0000 7f3456000640 1 -- 172.21.15.63:0/4066455538 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b96c3e1e40 con 0x55b96c03e1b0 2024-05-27T07:37:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.308+0000 7f3455600640 1 --2- 172.21.15.63:0/4066455538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.308+0000 7f3456000640 1 --2- 172.21.15.63:0/4066455538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6c30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3448032000 tx=0x7f344802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5627bc5d40530d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.308+0000 7f3441c00640 1 -- 172.21.15.63:0/4066455538 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344802c040 con 0x55b96c03e1b0 2024-05-27T07:37:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.309+0000 7f3441c00640 1 -- 172.21.15.63:0/4066455538 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34480c6c70 con 0x55b96c03e1b0 2024-05-27T07:37:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.309+0000 7f3441c00640 1 -- 172.21.15.63:0/4066455538 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34480c6910 con 0x55b96c03e1b0 2024-05-27T07:37:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.310+0000 7f345f16d740 1 -- 172.21.15.63:0/4066455538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 msgr2=0x55b96c3f6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.310+0000 7f345f16d740 1 --2- 172.21.15.63:0/4066455538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6c30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3448032000 tx=0x7f344802a950 comp rx=0 tx=0).stop 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.310+0000 7f345f16d740 1 -- 172.21.15.63:0/4066455538 shutdown_connections 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.310+0000 7f345f16d740 1 --2- 172.21.15.63:0/4066455538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c3f6c30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.310+0000 7f345f16d740 1 --2- 172.21.15.63:0/4066455538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3dbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.310+0000 7f345f16d740 1 --2- 172.21.15.63:0/4066455538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.310+0000 7f345f16d740 1 -- 172.21.15.63:0/4066455538 >> 172.21.15.63:0/4066455538 conn(0x55b96c032300 msgr2=0x55b96c3db810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.310+0000 7f345f16d740 1 -- 172.21.15.63:0/4066455538 shutdown_connections 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.310+0000 7f345f16d740 1 -- 172.21.15.63:0/4066455538 wait complete. 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.311+0000 7f345f16d740 1 -- start start 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.311+0000 7f345f16d740 1 Processor -- start 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.311+0000 7f345f16d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3ee410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.311+0000 7f345f16d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3ee950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.311+0000 7f345f16d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c463680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.311+0000 7f345f16d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34501b5970 con 0x55b96c03e1b0 2024-05-27T07:37:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.311+0000 7f345f16d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34501b6de0 con 0x55b96c01d4d0 2024-05-27T07:37:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.311+0000 7f345f16d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34501b6040 con 0x55b96c0152f0 2024-05-27T07:37:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f345f16d740 10 client.0 _ll_register_callbacks cb 0x7f3448009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f3456000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c463680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f3456000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c463680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45164/0 (socket says 172.21.15.63:45164) 2024-05-27T07:37:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f3456000640 1 -- 172.21.15.63:0/3539455383 learned_addr learned my addr 172.21.15.63:0/3539455383 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f3454c00640 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f3455600640 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f3456000640 1 -- 172.21.15.63:0/3539455383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 msgr2=0x55b96c3ee410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f3456000640 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3ee410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f3456000640 1 -- 172.21.15.63:0/3539455383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 msgr2=0x55b96c3ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f3456000640 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.312+0000 7f3456000640 1 -- 172.21.15.63:0/3539455383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b96c3e1b40 con 0x55b96c03e1b0 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.313+0000 7f3456000640 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c463680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f34480c6700 tx=0x7f34480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3892736f4c3302 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.313+0000 7f3436a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.313+0000 7f3436a00640 1 -- 172.21.15.63:0/3539455383 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344802c040 con 0x55b96c03e1b0 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.313+0000 7f3436a00640 1 -- 172.21.15.63:0/3539455383 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f34480d3260 con 0x55b96c03e1b0 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.313+0000 7f345f16d740 1 -- 172.21.15.63:0/3539455383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34480c6c70 con 0x55b96c03e1b0 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.313+0000 7f3436a00640 1 -- 172.21.15.63:0/3539455383 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f34480d34e0 con 0x55b96c03e1b0 2024-05-27T07:37:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.313+0000 7f345f16d740 10 client.16104 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.313+0000 7f345f16d740 1 -- 172.21.15.63:0/3539455383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b96c3e1b40 con 0x55b96c03e1b0 2024-05-27T07:37:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.314+0000 7f3436a00640 1 -- 172.21.15.63:0/3539455383 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34480c6c70 con 0x55b96c03e1b0 2024-05-27T07:37:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.314+0000 7f3436a00640 1 -- 172.21.15.63:0/3539455383 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f344804d070 con 0x55b96c03e1b0 2024-05-27T07:37:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.314+0000 7f345f16d740 10 client.16104 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.314+0000 7f345f16d740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.314+0000 7f345f16d740 20 client.16104 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.314+0000 7f345f16d740 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b96bfd21c0 0x55b96c466e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.314+0000 7f345f16d740 4 client.16104 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:37:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.314+0000 7f345f16d740 1 -- 172.21.15.63:0/3539455383 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b96c34d2c0 con 0x55b96bfd21c0 2024-05-27T07:37:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.315+0000 7f3455600640 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b96bfd21c0 0x55b96c466e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.316+0000 7f3455600640 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b96bfd21c0 0x55b96c466e10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b63b2f5ef68080b server_cookie=24e794212bbe5db1 in_seq=0 out_seq=0 2024-05-27T07:37:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.316+0000 7f3436a00640 10 client.16104 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.376+0000 7f3436a00640 1 -- 172.21.15.63:0/3539455383 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b96c34d2c0 con 0x55b96bfd21c0 2024-05-27T07:37:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.376+0000 7f3436a00640 10 client.16104 handle_command_reply: tid=0 2024-05-27T07:37:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 client.16104 shutdown 2024-05-27T07:37:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 -- 172.21.15.63:0/3539455383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 msgr2=0x55b96c463680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c463680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f34480c6700 tx=0x7f34480095c0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 -- 172.21.15.63:0/3539455383 shutdown_connections 2024-05-27T07:37:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b96bfd21c0 0x55b96c466e10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b96c03e1b0 0x55b96c463680 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b96c01d4d0 0x55b96c3ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 --2- 172.21.15.63:0/3539455383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b96c0152f0 0x55b96c3ee410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 -- 172.21.15.63:0/3539455383 >> 172.21.15.63:0/3539455383 conn(0x55b96c032300 msgr2=0x55b96c3dd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f3436a00640 0 client.16104 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 -- 172.21.15.63:0/3539455383 shutdown_connections 2024-05-27T07:37:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 1 -- 172.21.15.63:0/3539455383 wait complete. 2024-05-27T07:37:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.377+0000 7f345f16d740 20 client.16104 trim_cache size 0 max 16384 2024-05-27T07:37:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:37:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:37:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:37:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.378+0000 7f3456a00640 1 -- 172.21.15.63:0/728611984 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34380b29d0 msgr2=0x7f34380b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.378+0000 7f3456a00640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34380b29d0 0x7f34380b4ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55b96c3e1e10 tx=0x7f344400b410 comp rx=0 tx=0).stop 2024-05-27T07:37:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.378+0000 7f3456a00640 1 -- 172.21.15.63:0/728611984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34501a4f40 msgr2=0x7f34501a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.378+0000 7f3456a00640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34501a4f40 0x7f34501a5350 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f344802ae70 tx=0x7f3448003ce0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.378+0000 7f3456a00640 1 -- 172.21.15.63:0/728611984 shutdown_connections 2024-05-27T07:37:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.378+0000 7f3456a00640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34380b29d0 0x7f34380b4ee0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.378+0000 7f3456a00640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34501a4f40 0x7f34501a5350 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.378+0000 7f3456a00640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450110520 0x7f34501a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.378+0000 7f3456a00640 1 --2- 172.21.15.63:0/728611984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3450110110 0x7f34501a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.378+0000 7f3456a00640 1 -- 172.21.15.63:0/728611984 >> 172.21.15.63:0/728611984 conn(0x7f3450076070 msgr2=0x7f3450077120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.379+0000 7f3456a00640 1 -- 172.21.15.63:0/728611984 shutdown_connections 2024-05-27T07:37:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:05.379+0000 7f3456a00640 1 -- 172.21.15.63:0/728611984 wait complete. 2024-05-27T07:37:05.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:05 smithi086 ceph-mon[34687]: pgmap v109: 129 pgs: 129 active+clean; 188 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-05-27T07:37:05.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:05 smithi149 ceph-mon[35408]: pgmap v109: 129 pgs: 129 active+clean; 188 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-05-27T07:37:05.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:05 smithi063 ceph-mon[31613]: pgmap v109: 129 pgs: 129 active+clean; 188 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 25 op/s 2024-05-27T07:37:07.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:07 smithi086 ceph-mon[34687]: pgmap v110: 129 pgs: 129 active+clean; 213 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 33 op/s 2024-05-27T07:37:07.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:07 smithi149 ceph-mon[35408]: pgmap v110: 129 pgs: 129 active+clean; 213 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 33 op/s 2024-05-27T07:37:07.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:07 smithi063 ceph-mon[31613]: pgmap v110: 129 pgs: 129 active+clean; 213 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 33 op/s 2024-05-27T07:37:09.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:09 smithi086 ceph-mon[34687]: pgmap v111: 129 pgs: 129 active+clean; 213 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 25 op/s 2024-05-27T07:37:09.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:09 smithi149 ceph-mon[35408]: pgmap v111: 129 pgs: 129 active+clean; 213 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 25 op/s 2024-05-27T07:37:09.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:09 smithi063 ceph-mon[31613]: pgmap v111: 129 pgs: 129 active+clean; 213 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 25 op/s 2024-05-27T07:37:11.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:11 smithi086 ceph-mon[34687]: pgmap v112: 129 pgs: 129 active+clean; 233 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 29 op/s 2024-05-27T07:37:11.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:11 smithi149 ceph-mon[35408]: pgmap v112: 129 pgs: 129 active+clean; 233 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 29 op/s 2024-05-27T07:37:11.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:11 smithi063 ceph-mon[31613]: pgmap v112: 129 pgs: 129 active+clean; 233 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 29 op/s 2024-05-27T07:37:13.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:13 smithi086 ceph-mon[34687]: pgmap v113: 129 pgs: 129 active+clean; 233 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:37:13.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:13 smithi149 ceph-mon[35408]: pgmap v113: 129 pgs: 129 active+clean; 233 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:37:13.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:13 smithi063 ceph-mon[31613]: pgmap v113: 129 pgs: 129 active+clean; 233 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:37:15.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:15 smithi086 ceph-mon[34687]: pgmap v114: 129 pgs: 129 active+clean; 233 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:37:15.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:15 smithi149 ceph-mon[35408]: pgmap v114: 129 pgs: 129 active+clean; 233 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:37:15.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:15 smithi063 ceph-mon[31613]: pgmap v114: 129 pgs: 129 active+clean; 233 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:37:17.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:17 smithi086 ceph-mon[34687]: pgmap v115: 129 pgs: 129 active+clean; 257 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T07:37:17.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:17 smithi149 ceph-mon[35408]: pgmap v115: 129 pgs: 129 active+clean; 257 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T07:37:17.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:17 smithi063 ceph-mon[31613]: pgmap v115: 129 pgs: 129 active+clean; 257 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T07:37:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.618+0000 7faa87200640 1 -- 172.21.15.63:0/2560468613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa88104250 msgr2=0x7faa881105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.619+0000 7faa87200640 1 --2- 172.21.15.63:0/2560468613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa88104250 0x7faa881105d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7faa80009ca0 tx=0x7faa8002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.619+0000 7faa87200640 1 -- 172.21.15.63:0/2560468613 shutdown_connections 2024-05-27T07:37:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.619+0000 7faa87200640 1 --2- 172.21.15.63:0/2560468613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa88104250 0x7faa881105d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.619+0000 7faa87200640 1 --2- 172.21.15.63:0/2560468613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa88109f50 0x7faa88103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.619+0000 7faa87200640 1 --2- 172.21.15.63:0/2560468613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa88109ab0 0x7faa881036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.619+0000 7faa87200640 1 -- 172.21.15.63:0/2560468613 >> 172.21.15.63:0/2560468613 conn(0x7faa88079240 msgr2=0x7faa88079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.619+0000 7faa87200640 1 -- 172.21.15.63:0/2560468613 shutdown_connections 2024-05-27T07:37:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.619+0000 7faa87200640 1 -- 172.21.15.63:0/2560468613 wait complete. 2024-05-27T07:37:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.620+0000 7faa87200640 1 Processor -- start 2024-05-27T07:37:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.620+0000 7faa87200640 1 -- start start 2024-05-27T07:37:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa87200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa88104250 0x7faa8819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa87200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa88109ab0 0x7faa8819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa87200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa88109f50 0x7faa8819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa87200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa88113ba0 con 0x7faa88104250 2024-05-27T07:37:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa87200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa88113a20 con 0x7faa88109f50 2024-05-27T07:37:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa87200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa88113d20 con 0x7faa88109ab0 2024-05-27T07:37:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa85e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa88104250 0x7faa8819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa85e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa88104250 0x7faa8819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45336/0 (socket says 172.21.15.63:45336) 2024-05-27T07:37:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa85e00640 1 -- 172.21.15.63:0/1589313181 learned_addr learned my addr 172.21.15.63:0/1589313181 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa7fe00640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa88109ab0 0x7faa8819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.621+0000 7faa86800640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa88109f50 0x7faa8819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.622+0000 7faa85e00640 1 -- 172.21.15.63:0/1589313181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa88109ab0 msgr2=0x7faa8819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.622+0000 7faa85e00640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa88109ab0 0x7faa8819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.622+0000 7faa85e00640 1 -- 172.21.15.63:0/1589313181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa88109f50 msgr2=0x7faa8819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.622+0000 7faa85e00640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa88109f50 0x7faa8819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.622+0000 7faa85e00640 1 -- 172.21.15.63:0/1589313181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8819f420 con 0x7faa88104250 2024-05-27T07:37:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.622+0000 7faa85e00640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa88104250 0x7faa8819a5c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7faa7800e9c0 tx=0x7faa7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.622+0000 7faa7ea00640 1 -- 172.21.15.63:0/1589313181 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7800cd80 con 0x7faa88104250 2024-05-27T07:37:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.623+0000 7faa7ea00640 1 -- 172.21.15.63:0/1589313181 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa780054b0 con 0x7faa88104250 2024-05-27T07:37:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.623+0000 7faa87200640 1 -- 172.21.15.63:0/1589313181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8819f710 con 0x7faa88104250 2024-05-27T07:37:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.623+0000 7faa7ea00640 1 -- 172.21.15.63:0/1589313181 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78010430 con 0x7faa88104250 2024-05-27T07:37:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.623+0000 7faa87200640 1 -- 172.21.15.63:0/1589313181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa881a7090 con 0x7faa88104250 2024-05-27T07:37:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.626+0000 7faa939ce740 1 Processor -- start 2024-05-27T07:37:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.627+0000 7faa939ce740 1 -- start start 2024-05-27T07:37:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.627+0000 7faa939ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259bc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.627+0000 7faa939ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f12f0 0x5613259bce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.627+0000 7faa939ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f94d0 0x5613259bd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.627+0000 7faa939ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56132592db90 con 0x5613255f12f0 2024-05-27T07:37:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.627+0000 7faa939ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56132591d730 con 0x56132561a1b0 2024-05-27T07:37:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.627+0000 7faa939ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561325921770 con 0x5613255f94d0 2024-05-27T07:37:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.628+0000 7faa86800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259bc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.628+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f94d0 0x5613259bd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.628+0000 7faa86800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259bc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:37136/0 (socket says 172.21.15.63:37136) 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.628+0000 7faa86800640 1 -- 172.21.15.63:0/4245399448 learned_addr learned my addr 172.21.15.63:0/4245399448 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.628+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f94d0 0x5613259bd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:52228/0 (socket says 172.21.15.63:52228) 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.628+0000 7faa7ea00640 1 -- 172.21.15.63:0/1589313181 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faa78004f20 con 0x7faa88104250 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.628+0000 7faa85e00640 1 --2- 172.21.15.63:0/4245399448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f12f0 0x5613259bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.628+0000 7faa86800640 1 -- 172.21.15.63:0/4245399448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f94d0 msgr2=0x5613259bd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.628+0000 7faa86800640 1 --2- 172.21.15.63:0/4245399448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f94d0 0x5613259bd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.628+0000 7faa86800640 1 -- 172.21.15.63:0/4245399448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f12f0 msgr2=0x5613259bce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.629+0000 7faa86800640 1 --2- 172.21.15.63:0/4245399448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f12f0 0x5613259bce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.629+0000 7faa86800640 1 -- 172.21.15.63:0/4245399448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613258b68b0 con 0x56132561a1b0 2024-05-27T07:37:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.629+0000 7faa7fe00640 1 --2- 172.21.15.63:0/4245399448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f94d0 0x5613259bd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:37:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.629+0000 7faa86800640 1 --2- 172.21.15.63:0/4245399448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259bc950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa80005cc0 tx=0x7faa80009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f10b9cf99eb2d602 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.630+0000 7faa7ea00640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa6c0b2950 0x7faa6c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.630+0000 7faa77000640 1 -- 172.21.15.63:0/4245399448 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80031750 con 0x56132561a1b0 2024-05-27T07:37:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.630+0000 7faa77000640 1 -- 172.21.15.63:0/4245399448 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa80036040 con 0x56132561a1b0 2024-05-27T07:37:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.630+0000 7faa77000640 1 -- 172.21.15.63:0/4245399448 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa800042a0 con 0x56132561a1b0 2024-05-27T07:37:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.630+0000 7faa7ea00640 1 -- 172.21.15.63:0/1589313181 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7faa780e30e0 con 0x7faa88104250 2024-05-27T07:37:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.630+0000 7faa85e00640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa6c0b2950 0x7faa6c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.631+0000 7faa939ce740 1 -- 172.21.15.63:0/4245399448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 msgr2=0x5613259bc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.631+0000 7faa939ce740 1 --2- 172.21.15.63:0/4245399448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259bc950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa80005cc0 tx=0x7faa80009a20 comp rx=0 tx=0).stop 2024-05-27T07:37:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.631+0000 7faa85e00640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa6c0b2950 0x7faa6c0b4e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x5613259ba2c0 tx=0x7faa7800cf20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.631+0000 7faa939ce740 1 -- 172.21.15.63:0/4245399448 shutdown_connections 2024-05-27T07:37:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.631+0000 7faa939ce740 1 --2- 172.21.15.63:0/4245399448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f94d0 0x5613259bd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.631+0000 7faa939ce740 1 --2- 172.21.15.63:0/4245399448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f12f0 0x5613259bce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.631+0000 7faa939ce740 1 --2- 172.21.15.63:0/4245399448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259bc950 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.631+0000 7faa939ce740 1 -- 172.21.15.63:0/4245399448 >> 172.21.15.63:0/4245399448 conn(0x56132560e300 msgr2=0x5613259b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.631+0000 7faa939ce740 1 -- 172.21.15.63:0/4245399448 shutdown_connections 2024-05-27T07:37:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.632+0000 7faa939ce740 1 -- 172.21.15.63:0/4245399448 wait complete. 2024-05-27T07:37:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.632+0000 7faa939ce740 1 -- start start 2024-05-27T07:37:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.632+0000 7faa939ce740 1 Processor -- start 2024-05-27T07:37:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.633+0000 7faa939ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613255f12f0 0x5613259cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.633+0000 7faa939ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.633+0000 7faa939ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132561a1b0 0x5613259d7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.633+0000 7faa939ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561325603790 con 0x5613255f94d0 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.633+0000 7faa939ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5613258c5bc0 con 0x5613255f12f0 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.633+0000 7faa939ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613259cdf40 con 0x56132561a1b0 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.633+0000 7faa939ce740 10 client.0 _ll_register_callbacks cb 0x5613258a3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.633+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.633+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45354/0 (socket says 172.21.15.63:45354) 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.633+0000 7faa7fe00640 1 -- 172.21.15.63:0/1568167569 learned_addr learned my addr 172.21.15.63:0/1568167569 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.634+0000 7faa86800640 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613255f12f0 0x5613259cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.634+0000 7faa86800640 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132561a1b0 0x5613259d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.634+0000 7faa7fe00640 1 -- 172.21.15.63:0/1568167569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132561a1b0 msgr2=0x5613259d7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.634+0000 7faa7fe00640 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132561a1b0 0x5613259d7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.634+0000 7faa7fe00640 1 -- 172.21.15.63:0/1568167569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613255f12f0 msgr2=0x5613259cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.634+0000 7faa7fe00640 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613255f12f0 0x5613259cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.634+0000 7faa7fe00640 1 -- 172.21.15.63:0/1568167569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561325632210 con 0x5613255f94d0 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.634+0000 7faa86800640 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132561a1b0 0x5613259d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.634+0000 7faa7fe00640 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259cb560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faa7000de90 tx=0x7faa70012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e4daaf181440d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.635+0000 7faa6b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.635+0000 7faa6b400640 1 -- 172.21.15.63:0/1568167569 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700107e0 con 0x5613255f94d0 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.635+0000 7faa939ce740 1 -- 172.21.15.63:0/1568167569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa80036040 con 0x5613255f94d0 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.635+0000 7faa6b400640 1 -- 172.21.15.63:0/1568167569 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7faa70010980 con 0x5613255f94d0 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.635+0000 7faa6b400640 1 -- 172.21.15.63:0/1568167569 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faa70027050 con 0x5613255f94d0 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.635+0000 7faa939ce740 10 client.16116 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.635+0000 7faa939ce740 1 -- 172.21.15.63:0/1568167569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5613256fe620 con 0x5613255f94d0 2024-05-27T07:37:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.636+0000 7faa6b400640 1 -- 172.21.15.63:0/1568167569 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa80036040 con 0x5613255f94d0 2024-05-27T07:37:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.636+0000 7faa6b400640 1 -- 172.21.15.63:0/1568167569 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7faa7001cbd0 con 0x5613255f94d0 2024-05-27T07:37:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.636+0000 7faa939ce740 10 client.16116 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.636+0000 7faa939ce740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.636+0000 7faa939ce740 20 client.16116 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.636+0000 7faa939ce740 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613255d8d80 0x5613259dd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.636+0000 7faa939ce740 4 client.16116 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.636+0000 7faa939ce740 1 -- 172.21.15.63:0/1568167569 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56132592fae0 con 0x5613255d8d80 2024-05-27T07:37:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.637+0000 7faa86800640 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613255d8d80 0x5613259dd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.638+0000 7faa86800640 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613255d8d80 0x5613259dd330 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0bf01fcd53dcc9f server_cookie=e0040cd053bddc0b in_seq=0 out_seq=0 2024-05-27T07:37:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.638+0000 7faa6b400640 10 client.16116 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa6b400640 1 -- 172.21.15.63:0/1568167569 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56132592fae0 con 0x5613255d8d80 2024-05-27T07:37:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa6b400640 10 client.16116 handle_command_reply: tid=0 2024-05-27T07:37:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa939ce740 1 client.16116 shutdown 2024-05-27T07:37:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa939ce740 1 -- 172.21.15.63:0/1568167569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 msgr2=0x5613259cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa939ce740 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259cb560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faa7000de90 tx=0x7faa70012820 comp rx=0 tx=0).stop 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa939ce740 1 -- 172.21.15.63:0/1568167569 shutdown_connections 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa939ce740 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613255d8d80 0x5613259dd330 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa939ce740 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56132561a1b0 0x5613259d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa939ce740 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259cb560 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa939ce740 1 --2- 172.21.15.63:0/1568167569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613255f12f0 0x5613259cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa939ce740 1 -- 172.21.15.63:0/1568167569 >> 172.21.15.63:0/1568167569 conn(0x56132560e300 msgr2=0x5613259bc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.640+0000 7faa6b400640 0 client.16116 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.641+0000 7faa939ce740 1 -- 172.21.15.63:0/1568167569 shutdown_connections 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.641+0000 7faa939ce740 1 -- 172.21.15.63:0/1568167569 wait complete. 2024-05-27T07:37:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.641+0000 7faa939ce740 20 client.16116 trim_cache size 0 max 16384 2024-05-27T07:37:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.658+0000 7faa939ce740 1 Processor -- start 2024-05-27T07:37:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.658+0000 7faa939ce740 1 -- start start 2024-05-27T07:37:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.658+0000 7faa939ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f12f0 0x5613259bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.658+0000 7faa939ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613255f94d0 0x5613259bd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.659+0000 7faa939ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132561a1b0 0x5613259d6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.659+0000 7faa939ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa881b1750 con 0x56132561a1b0 2024-05-27T07:37:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.659+0000 7faa939ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa881aefc0 con 0x5613255f94d0 2024-05-27T07:37:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.659+0000 7faa939ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa881ace10 con 0x5613255f12f0 2024-05-27T07:37:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.659+0000 7faa86800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132561a1b0 0x5613259d6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.659+0000 7faa86800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132561a1b0 0x5613259d6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45360/0 (socket says 172.21.15.63:45360) 2024-05-27T07:37:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.659+0000 7faa86800640 1 -- 172.21.15.63:0/1924074842 learned_addr learned my addr 172.21.15.63:0/1924074842 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.659+0000 7faa86800640 1 --2- 172.21.15.63:0/1924074842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f12f0 0x5613259bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.659+0000 7faa7fe00640 1 --2- 172.21.15.63:0/1924074842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613255f94d0 0x5613259bd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.660+0000 7faa86800640 1 -- 172.21.15.63:0/1924074842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f12f0 msgr2=0x5613259bf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.660+0000 7faa86800640 1 --2- 172.21.15.63:0/1924074842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f12f0 0x5613259bf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.660+0000 7faa86800640 1 -- 172.21.15.63:0/1924074842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613255f94d0 msgr2=0x5613259bd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.660+0000 7faa86800640 1 --2- 172.21.15.63:0/1924074842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613255f94d0 0x5613259bd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.660+0000 7faa86800640 1 -- 172.21.15.63:0/1924074842 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613259b3210 con 0x56132561a1b0 2024-05-27T07:37:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.660+0000 7faa86800640 1 --2- 172.21.15.63:0/1924074842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132561a1b0 0x5613259d6fe0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faa800096c0 tx=0x7faa80005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b781ae4267b73da server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.661+0000 7faa76600640 1 -- 172.21.15.63:0/1924074842 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa800315d0 con 0x56132561a1b0 2024-05-27T07:37:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.661+0000 7faa76600640 1 -- 172.21.15.63:0/1924074842 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa80036070 con 0x56132561a1b0 2024-05-27T07:37:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.661+0000 7faa76600640 1 -- 172.21.15.63:0/1924074842 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80004720 con 0x56132561a1b0 2024-05-27T07:37:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.662+0000 7faa939ce740 1 -- 172.21.15.63:0/1924074842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132561a1b0 msgr2=0x5613259d6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.662+0000 7faa939ce740 1 --2- 172.21.15.63:0/1924074842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132561a1b0 0x5613259d6fe0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faa800096c0 tx=0x7faa80005c50 comp rx=0 tx=0).stop 2024-05-27T07:37:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.662+0000 7faa939ce740 1 -- 172.21.15.63:0/1924074842 shutdown_connections 2024-05-27T07:37:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.662+0000 7faa939ce740 1 --2- 172.21.15.63:0/1924074842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56132561a1b0 0x5613259d6fe0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.662+0000 7faa939ce740 1 --2- 172.21.15.63:0/1924074842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613255f94d0 0x5613259bd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.662+0000 7faa939ce740 1 --2- 172.21.15.63:0/1924074842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f12f0 0x5613259bf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.662+0000 7faa939ce740 1 -- 172.21.15.63:0/1924074842 >> 172.21.15.63:0/1924074842 conn(0x56132560e300 msgr2=0x5613259ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.662+0000 7faa939ce740 1 -- 172.21.15.63:0/1924074842 shutdown_connections 2024-05-27T07:37:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.662+0000 7faa939ce740 1 -- 172.21.15.63:0/1924074842 wait complete. 2024-05-27T07:37:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.663+0000 7faa939ce740 1 -- start start 2024-05-27T07:37:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.663+0000 7faa939ce740 1 Processor -- start 2024-05-27T07:37:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.663+0000 7faa939ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f12f0 0x5613259c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.663+0000 7faa939ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259c5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.663+0000 7faa939ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.663+0000 7faa939ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa881ad480 con 0x5613255f94d0 2024-05-27T07:37:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.663+0000 7faa939ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa881ae8f0 con 0x56132561a1b0 2024-05-27T07:37:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.663+0000 7faa939ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa881adb50 con 0x5613255f12f0 2024-05-27T07:37:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.663+0000 7faa939ce740 10 client.0 _ll_register_callbacks cb 0x7faa7000d6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.664+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.664+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45370/0 (socket says 172.21.15.63:45370) 2024-05-27T07:37:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.664+0000 7faa7fe00640 1 -- 172.21.15.63:0/1395760614 learned_addr learned my addr 172.21.15.63:0/1395760614 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.664+0000 7faa7fe00640 1 -- 172.21.15.63:0/1395760614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f12f0 msgr2=0x5613259c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.664+0000 7faa86800640 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.664+0000 7faa7fe00640 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f12f0 0x5613259c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.664+0000 7faa7fe00640 1 -- 172.21.15.63:0/1395760614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 msgr2=0x5613259c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.664+0000 7faa7fe00640 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.664+0000 7faa7fe00640 1 -- 172.21.15.63:0/1395760614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613259d23d0 con 0x5613255f94d0 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.664+0000 7faa86800640 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.665+0000 7faa7fe00640 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259c5510 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x5613259cdcd0 tx=0x7faa70005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5991f2fe9389db8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.665+0000 7faa6b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.665+0000 7faa6b400640 1 -- 172.21.15.63:0/1395760614 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70004640 con 0x5613255f94d0 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.665+0000 7faa939ce740 1 -- 172.21.15.63:0/1395760614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa80036070 con 0x5613255f94d0 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.665+0000 7faa6b400640 1 -- 172.21.15.63:0/1395760614 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7faa700047e0 con 0x5613255f94d0 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.665+0000 7faa6b400640 1 -- 172.21.15.63:0/1395760614 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faa7003d050 con 0x5613255f94d0 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.665+0000 7faa939ce740 10 client.16122 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.665+0000 7faa939ce740 1 -- 172.21.15.63:0/1395760614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5613259d3620 con 0x5613255f94d0 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.666+0000 7faa6b400640 1 -- 172.21.15.63:0/1395760614 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa80036070 con 0x5613255f94d0 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.666+0000 7faa6b400640 1 -- 172.21.15.63:0/1395760614 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7faa70034070 con 0x5613255f94d0 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.666+0000 7faa939ce740 10 client.16122 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:18.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.666+0000 7faa939ce740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.666+0000 7faa939ce740 20 client.16122 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.666+0000 7faa939ce740 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613255ae1c0 0x561325a47a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.666+0000 7faa939ce740 4 client.16122 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:37:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.666+0000 7faa939ce740 1 -- 172.21.15.63:0/1395760614 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613259292c0 con 0x5613255ae1c0 2024-05-27T07:37:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.667+0000 7faa86800640 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613255ae1c0 0x561325a47a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.668+0000 7faa86800640 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613255ae1c0 0x561325a47a10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51dee8b7bb86c7c4 server_cookie=971cc72f6843d95 in_seq=0 out_seq=0 2024-05-27T07:37:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.668+0000 7faa6b400640 10 client.16122 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa6b400640 1 -- 172.21.15.63:0/1395760614 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5613259292c0 con 0x5613255ae1c0 2024-05-27T07:37:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa6b400640 10 client.16122 handle_command_reply: tid=0 2024-05-27T07:37:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa939ce740 1 client.16122 shutdown 2024-05-27T07:37:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa939ce740 1 -- 172.21.15.63:0/1395760614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 msgr2=0x5613259c5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa939ce740 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259c5510 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x5613259cdcd0 tx=0x7faa70005270 comp rx=0 tx=0).stop 2024-05-27T07:37:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa939ce740 1 -- 172.21.15.63:0/1395760614 shutdown_connections 2024-05-27T07:37:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa939ce740 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613255ae1c0 0x561325a47a10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa939ce740 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56132561a1b0 0x5613259c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa939ce740 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613255f94d0 0x5613259c5510 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa939ce740 1 --2- 172.21.15.63:0/1395760614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613255f12f0 0x5613259c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa6b400640 0 client.16122 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.707+0000 7faa939ce740 1 -- 172.21.15.63:0/1395760614 >> 172.21.15.63:0/1395760614 conn(0x56132560e300 msgr2=0x5613259beea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.708+0000 7faa939ce740 1 -- 172.21.15.63:0/1395760614 shutdown_connections 2024-05-27T07:37:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.708+0000 7faa939ce740 1 -- 172.21.15.63:0/1395760614 wait complete. 2024-05-27T07:37:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.708+0000 7faa939ce740 20 client.16122 trim_cache size 0 max 16384 2024-05-27T07:37:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:37:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:37:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:37:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 -- 172.21.15.63:0/1589313181 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa6c0b2950 msgr2=0x7faa6c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa6c0b2950 0x7faa6c0b4e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x5613259ba2c0 tx=0x7faa7800cf20 comp rx=0 tx=0).stop 2024-05-27T07:37:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 -- 172.21.15.63:0/1589313181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa88104250 msgr2=0x7faa8819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa88104250 0x7faa8819a5c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7faa7800e9c0 tx=0x7faa7800ee80 comp rx=0 tx=0).stop 2024-05-27T07:37:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 -- 172.21.15.63:0/1589313181 shutdown_connections 2024-05-27T07:37:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faa6c0b2950 0x7faa6c0b4e60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa88109f50 0x7faa8819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa88109ab0 0x7faa8819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 --2- 172.21.15.63:0/1589313181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa88104250 0x7faa8819a5c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 -- 172.21.15.63:0/1589313181 >> 172.21.15.63:0/1589313181 conn(0x7faa88079240 msgr2=0x7faa88100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.709+0000 7faa87200640 1 -- 172.21.15.63:0/1589313181 shutdown_connections 2024-05-27T07:37:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:18.710+0000 7faa87200640 1 -- 172.21.15.63:0/1589313181 wait complete. 2024-05-27T07:37:19.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:19 smithi086 ceph-mon[34687]: pgmap v116: 129 pgs: 129 active+clean; 257 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:37:19.793 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:19 smithi063 ceph-mon[31613]: pgmap v116: 129 pgs: 129 active+clean; 257 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:37:19.794 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:19 smithi149 ceph-mon[35408]: pgmap v116: 129 pgs: 129 active+clean; 257 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:37:21.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:21 smithi086 ceph-mon[34687]: pgmap v117: 129 pgs: 129 active+clean; 281 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:37:21.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:21 smithi149 ceph-mon[35408]: pgmap v117: 129 pgs: 129 active+clean; 281 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:37:21.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:21 smithi063 ceph-mon[31613]: pgmap v117: 129 pgs: 129 active+clean; 281 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:37:23.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:23 smithi086 ceph-mon[34687]: pgmap v118: 129 pgs: 129 active+clean; 281 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:37:23.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:23 smithi149 ceph-mon[35408]: pgmap v118: 129 pgs: 129 active+clean; 281 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:37:23.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:23 smithi063 ceph-mon[31613]: pgmap v118: 129 pgs: 129 active+clean; 281 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:37:24.611 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:37:25.113 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:37:25.614 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:25 smithi063 ceph-mon[31613]: pgmap v119: 129 pgs: 129 active+clean; 281 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:37:25.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:25 smithi086 ceph-mon[34687]: pgmap v119: 129 pgs: 129 active+clean; 281 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:37:25.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:25 smithi149 ceph-mon[35408]: pgmap v119: 129 pgs: 129 active+clean; 281 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:37:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.511+0000 7f1900a00700 1 -- 172.21.15.63:0/2554691850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18fc105300 msgr2=0x7f18fc105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.511+0000 7f1900a00700 1 --2- 172.21.15.63:0/2554691850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18fc105300 0x7f18fc105710 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009990 tx=0x7f18ec009ca0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.511+0000 7f1900a00700 1 -- 172.21.15.63:0/2554691850 shutdown_connections 2024-05-27T07:37:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.511+0000 7f1900a00700 1 --2- 172.21.15.63:0/2554691850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18fc10e730 0x7f18fc110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.511+0000 7f1900a00700 1 --2- 172.21.15.63:0/2554691850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18fc105300 0x7f18fc105710 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.511+0000 7f1900a00700 1 --2- 172.21.15.63:0/2554691850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18fc105c50 0x7f18fc10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.511+0000 7f1900a00700 1 -- 172.21.15.63:0/2554691850 >> 172.21.15.63:0/2554691850 conn(0x7f18fc078c30 msgr2=0x7f18fc079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.511+0000 7f1900a00700 1 -- 172.21.15.63:0/2554691850 shutdown_connections 2024-05-27T07:37:27.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.511+0000 7f1900a00700 1 -- 172.21.15.63:0/2554691850 wait complete. 2024-05-27T07:37:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.513+0000 7f1900a00700 1 Processor -- start 2024-05-27T07:37:27.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.513+0000 7f1900a00700 1 -- start start 2024-05-27T07:37:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.513+0000 7f1900a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18fc105300 0x7f18fc19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.513+0000 7f1900a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18fc105c50 0x7f18fc19d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.513+0000 7f1900a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18fc10e730 0x7f18fc1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.513+0000 7f1900a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc114130 con 0x7f18fc105c50 2024-05-27T07:37:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.513+0000 7f1900a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc113fb0 con 0x7f18fc10e730 2024-05-27T07:37:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.513+0000 7f1900a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1142b0 con 0x7f18fc105300 2024-05-27T07:37:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.514+0000 7f18faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18fc105c50 0x7f18fc19d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.514+0000 7f18faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18fc105c50 0x7f18fc19d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48734/0 (socket says 172.21.15.63:48734) 2024-05-27T07:37:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.514+0000 7f18faa00700 1 -- 172.21.15.63:0/1873083079 learned_addr learned my addr 172.21.15.63:0/1873083079 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:27.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.514+0000 7f18fb400700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18fc105300 0x7f18fc19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.514+0000 7f18fbe00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18fc10e730 0x7f18fc1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.512 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:27 smithi063 ceph-mon[31613]: pgmap v120: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.515+0000 7f18faa00700 1 -- 172.21.15.63:0/1873083079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18fc105300 msgr2=0x7f18fc19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.515+0000 7f18faa00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18fc105300 0x7f18fc19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.515+0000 7f18faa00700 1 -- 172.21.15.63:0/1873083079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18fc10e730 msgr2=0x7f18fc1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.515+0000 7f18faa00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18fc10e730 0x7f18fc1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.515+0000 7f18faa00700 1 -- 172.21.15.63:0/1873083079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc1a1c00 con 0x7f18fc105c50 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.515+0000 7f18fbe00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18fc10e730 0x7f18fc1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.515+0000 7f18faa00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18fc105c50 0x7f18fc19d320 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f18e800d980 tx=0x7f18e800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.516+0000 7f18f3e00700 1 -- 172.21.15.63:0/1873083079 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8016070 con 0x7f18fc105c50 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.516+0000 7f18f3e00700 1 -- 172.21.15.63:0/1873083079 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18e8004020 con 0x7f18fc105c50 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.516+0000 7f18f3e00700 1 -- 172.21.15.63:0/1873083079 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8004bb0 con 0x7f18fc105c50 2024-05-27T07:37:27.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.516+0000 7f1900a00700 1 -- 172.21.15.63:0/1873083079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18fc1a1ef0 con 0x7f18fc105c50 2024-05-27T07:37:27.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.516+0000 7f1900a00700 1 -- 172.21.15.63:0/1873083079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18fc1a9810 con 0x7f18fc105c50 2024-05-27T07:37:27.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.522+0000 7f1910b7bb80 1 Processor -- start 2024-05-27T07:37:27.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.522+0000 7f1910b7bb80 1 -- start start 2024-05-27T07:37:27.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f18f3e00700 1 -- 172.21.15.63:0/1873083079 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f18e800bb60 con 0x7f18fc105c50 2024-05-27T07:37:27.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e1526e40 0x5625e16a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 0x5625e16afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625e15d3e40 con 0x5625e1526e40 2024-05-27T07:37:27.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5625e15c39e0 con 0x5625e16a5410 2024-05-27T07:37:27.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5625e15c7a20 con 0x5625e16a5820 2024-05-27T07:37:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f18fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e1526e40 0x5625e16a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f18fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e1526e40 0x5625e16a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48738/0 (socket says 172.21.15.63:48738) 2024-05-27T07:37:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f18fbe00700 1 -- 172.21.15.63:0/2918840499 learned_addr learned my addr 172.21.15.63:0/2918840499 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.523+0000 7f18faa00700 1 --2- 172.21.15.63:0/2918840499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.524+0000 7f18fb400700 1 --2- 172.21.15.63:0/2918840499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 0x5625e16afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.524+0000 7f18fbe00700 1 -- 172.21.15.63:0/2918840499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 msgr2=0x5625e16b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.524+0000 7f18fbe00700 1 --2- 172.21.15.63:0/2918840499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.524+0000 7f18fbe00700 1 -- 172.21.15.63:0/2918840499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 msgr2=0x5625e16afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.524+0000 7f18fbe00700 1 --2- 172.21.15.63:0/2918840499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 0x5625e16afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.524+0000 7f18fbe00700 1 -- 172.21.15.63:0/2918840499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625e158c420 con 0x5625e1526e40 2024-05-27T07:37:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.524+0000 7f18fb400700 1 --2- 172.21.15.63:0/2918840499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 0x5625e16afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:27.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.525+0000 7f18fbe00700 1 --2- 172.21.15.63:0/2918840499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e1526e40 0x5625e16a4ed0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009dd0 tx=0x7f18f40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b2bb1dfd6d24c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.525+0000 7f18f3e00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18e00b2b40 0x7f18e00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.525+0000 7f18f0c00700 1 -- 172.21.15.63:0/2918840499 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4004bb0 con 0x5625e1526e40 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.525+0000 7f18f0c00700 1 -- 172.21.15.63:0/2918840499 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18f4004d50 con 0x5625e1526e40 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.525+0000 7f18f3e00700 1 -- 172.21.15.63:0/1873083079 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f18e80e2510 con 0x7f18fc105c50 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.525+0000 7f18f0c00700 1 -- 172.21.15.63:0/2918840499 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4014410 con 0x5625e1526e40 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.527+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2918840499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e1526e40 msgr2=0x5625e16a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.527+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/2918840499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e1526e40 0x5625e16a4ed0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009dd0 tx=0x7f18f40077e0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.527+0000 7f18fb400700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18e00b2b40 0x7f18e00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.527+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2918840499 shutdown_connections 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.527+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/2918840499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.527+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/2918840499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e1526e40 0x5625e16a4ed0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.527+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/2918840499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 0x5625e16afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.527+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2918840499 >> 172.21.15.63:0/2918840499 conn(0x5625e1586fc0 msgr2=0x5625e16a0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:27.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.527+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2918840499 shutdown_connections 2024-05-27T07:37:27.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.528+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2918840499 wait complete. 2024-05-27T07:37:27.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.528+0000 7f18fb400700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18e00b2b40 0x7f18e00b5050 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5625e16a6430 tx=0x7f18ec01a040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:27.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.528+0000 7f1910b7bb80 1 -- start start 2024-05-27T07:37:27.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.528+0000 7f1910b7bb80 1 Processor -- start 2024-05-27T07:37:27.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.529+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 0x5625e16af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.529+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.529+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.529+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625e14c9fa0 con 0x5625e16a5410 2024-05-27T07:37:27.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.529+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5625e1564760 con 0x5625e1526e40 2024-05-27T07:37:27.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.529+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5625e16b2990 con 0x5625e16a5820 2024-05-27T07:37:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.529+0000 7f1910b7bb80 10 client.0 _ll_register_callbacks cb 0x5625e156dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48746/0 (socket says 172.21.15.63:48746) 2024-05-27T07:37:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18faa00700 1 -- 172.21.15.63:0/230809214 learned_addr learned my addr 172.21.15.63:0/230809214 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:27.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18fb400700 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 0x5625e16af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18fbe00700 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18faa00700 1 -- 172.21.15.63:0/230809214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 msgr2=0x5625e16b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18faa00700 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18faa00700 1 -- 172.21.15.63:0/230809214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 msgr2=0x5625e16af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18faa00700 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 0x5625e16af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18faa00700 1 -- 172.21.15.63:0/230809214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625e15138d0 con 0x5625e16a5410 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.530+0000 7f18fbe00700 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.531+0000 7f18faa00700 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16afad0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f18e8007fd0 tx=0x7f18e8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8d780f43587b40 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.531+0000 7f18e5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.531+0000 7f18e5600700 1 -- 172.21.15.63:0/230809214 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e80a8d70 con 0x5625e16a5410 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.531+0000 7f18e5600700 1 -- 172.21.15.63:0/230809214 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f18e80b2000 con 0x5625e16a5410 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.531+0000 7f1910b7bb80 1 -- 172.21.15.63:0/230809214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f4004d50 con 0x5625e16a5410 2024-05-27T07:37:27.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.532+0000 7f18e5600700 1 -- 172.21.15.63:0/230809214 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f18e80a8b90 con 0x5625e16a5410 2024-05-27T07:37:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.532+0000 7f1910b7bb80 10 client.16134 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.532+0000 7f1910b7bb80 1 -- 172.21.15.63:0/230809214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5625e1594060 con 0x5625e16a5410 2024-05-27T07:37:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.532+0000 7f18e5600700 1 -- 172.21.15.63:0/230809214 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18f4004d50 con 0x5625e16a5410 2024-05-27T07:37:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.532+0000 7f18e5600700 1 -- 172.21.15.63:0/230809214 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f18e80e2170 con 0x5625e16a5410 2024-05-27T07:37:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.533+0000 7f1910b7bb80 10 client.16134 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.533+0000 7f1910b7bb80 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.533+0000 7f1910b7bb80 20 client.16134 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.533+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625e16bf8f0 0x5625e16c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.533+0000 7f1910b7bb80 4 client.16134 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:27.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.533+0000 7f1910b7bb80 1 -- 172.21.15.63:0/230809214 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625e15d5d90 con 0x5625e16bf8f0 2024-05-27T07:37:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.533+0000 7f18fb400700 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625e16bf8f0 0x5625e16c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.534+0000 7f18fb400700 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625e16bf8f0 0x5625e16c1d60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ac66389e49b9 server_cookie=74874eadec618812 in_seq=0 out_seq=0 2024-05-27T07:37:27.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.534+0000 7f18e5600700 10 client.16134 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.536+0000 7f18e5600700 1 -- 172.21.15.63:0/230809214 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5625e15d5d90 con 0x5625e16bf8f0 2024-05-27T07:37:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.536+0000 7f18e5600700 10 client.16134 handle_command_reply: tid=0 2024-05-27T07:37:27.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.536+0000 7f1910b7bb80 1 client.16134 shutdown 2024-05-27T07:37:27.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 1 -- 172.21.15.63:0/230809214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 msgr2=0x5625e16afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:27.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16afad0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f18e8007fd0 tx=0x7f18e8005e50 comp rx=0 tx=0).stop 2024-05-27T07:37:27.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 1 -- 172.21.15.63:0/230809214 shutdown_connections 2024-05-27T07:37:27.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625e16bf8f0 0x5625e16c1d60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 0x5625e16af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f18e5600700 0 client.16134 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:27.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/230809214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16afad0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 1 -- 172.21.15.63:0/230809214 >> 172.21.15.63:0/230809214 conn(0x5625e1586fc0 msgr2=0x5625e16a0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:27.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 1 -- 172.21.15.63:0/230809214 shutdown_connections 2024-05-27T07:37:27.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 1 -- 172.21.15.63:0/230809214 wait complete. 2024-05-27T07:37:27.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.537+0000 7f1910b7bb80 20 client.16134 trim_cache size 0 max 16384 2024-05-27T07:37:27.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.568+0000 7f1910b7bb80 1 Processor -- start 2024-05-27T07:37:27.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.568+0000 7f1910b7bb80 1 -- start start 2024-05-27T07:37:27.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.569+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 0x5625e16aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.569+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16bba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.569+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16a0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.569+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b4d40 con 0x5625e16a5410 2024-05-27T07:37:27.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.569+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b2570 con 0x5625e1526e40 2024-05-27T07:37:27.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.569+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b03c0 con 0x5625e16a5820 2024-05-27T07:37:27.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.569+0000 7f18faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16bba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.569+0000 7f18faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16bba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48748/0 (socket says 172.21.15.63:48748) 2024-05-27T07:37:27.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.569+0000 7f18faa00700 1 -- 172.21.15.63:0/3820989603 learned_addr learned my addr 172.21.15.63:0/3820989603 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:27.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.569+0000 7f18fbe00700 1 --2- 172.21.15.63:0/3820989603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.570+0000 7f18fb400700 1 --2- 172.21.15.63:0/3820989603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 0x5625e16aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.570+0000 7f18faa00700 1 -- 172.21.15.63:0/3820989603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 msgr2=0x5625e16a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:27.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.570+0000 7f18faa00700 1 --2- 172.21.15.63:0/3820989603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.570+0000 7f18faa00700 1 -- 172.21.15.63:0/3820989603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 msgr2=0x5625e16aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:27.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.570+0000 7f18faa00700 1 --2- 172.21.15.63:0/3820989603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 0x5625e16aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.570+0000 7f18faa00700 1 -- 172.21.15.63:0/3820989603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625e169b1d0 con 0x5625e16a5410 2024-05-27T07:37:27.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.570+0000 7f18fb400700 1 --2- 172.21.15.63:0/3820989603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 0x5625e16aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:27.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.570+0000 7f18faa00700 1 --2- 172.21.15.63:0/3820989603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16bba90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f18e80a54a0 tx=0x7f18e80a99e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72be6dcb1d3c5b9f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:27.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.571+0000 7f18e7e00700 1 -- 172.21.15.63:0/3820989603 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e80e9070 con 0x5625e16a5410 2024-05-27T07:37:27.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.571+0000 7f18e7e00700 1 -- 172.21.15.63:0/3820989603 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18e80044d0 con 0x5625e16a5410 2024-05-27T07:37:27.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.571+0000 7f18e7e00700 1 -- 172.21.15.63:0/3820989603 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e80a8680 con 0x5625e16a5410 2024-05-27T07:37:27.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.572+0000 7f1910b7bb80 1 -- 172.21.15.63:0/3820989603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 msgr2=0x5625e16bba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:27.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.572+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/3820989603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16bba90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f18e80a54a0 tx=0x7f18e80a99e0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.572+0000 7f1910b7bb80 1 -- 172.21.15.63:0/3820989603 shutdown_connections 2024-05-27T07:37:27.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.572+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/3820989603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e16a5820 0x5625e16a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.572+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/3820989603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e1526e40 0x5625e16aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.572+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/3820989603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5410 0x5625e16bba90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.572+0000 7f1910b7bb80 1 -- 172.21.15.63:0/3820989603 >> 172.21.15.63:0/3820989603 conn(0x5625e1586fc0 msgr2=0x5625e169fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:27.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.572+0000 7f1910b7bb80 1 -- 172.21.15.63:0/3820989603 shutdown_connections 2024-05-27T07:37:27.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.573+0000 7f1910b7bb80 1 -- 172.21.15.63:0/3820989603 wait complete. 2024-05-27T07:37:27.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.573+0000 7f1910b7bb80 1 -- start start 2024-05-27T07:37:27.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.574+0000 7f1910b7bb80 1 Processor -- start 2024-05-27T07:37:27.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.574+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e1526e40 0x5625e16b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.575+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 0x5625e16b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.575+0000 7f1910b7bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5820 0x5625e17293f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.575+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b0a30 con 0x5625e16a5820 2024-05-27T07:37:27.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.575+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b1ea0 con 0x5625e16a5410 2024-05-27T07:37:27.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.575+0000 7f1910b7bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b1100 con 0x5625e1526e40 2024-05-27T07:37:27.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.575+0000 7f1910b7bb80 10 client.0 _ll_register_callbacks cb 0x5625e156dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:27.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.575+0000 7f18fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5820 0x5625e17293f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.575+0000 7f18fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5820 0x5625e17293f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48754/0 (socket says 172.21.15.63:48754) 2024-05-27T07:37:27.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.575+0000 7f18fbe00700 1 -- 172.21.15.63:0/2220164215 learned_addr learned my addr 172.21.15.63:0/2220164215 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:27.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.575+0000 7f18fb400700 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e1526e40 0x5625e16b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.576+0000 7f18faa00700 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 0x5625e16b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.576+0000 7f18fbe00700 1 -- 172.21.15.63:0/2220164215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e1526e40 msgr2=0x5625e16b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:27.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.576+0000 7f18fbe00700 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e1526e40 0x5625e16b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.576+0000 7f18fbe00700 1 -- 172.21.15.63:0/2220164215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 msgr2=0x5625e16b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:27.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.576+0000 7f18fbe00700 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 0x5625e16b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.576+0000 7f18fbe00700 1 -- 172.21.15.63:0/2220164215 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625e16a2350 con 0x5625e16a5820 2024-05-27T07:37:27.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.576+0000 7f18fbe00700 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5820 0x5625e17293f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f18f4000f80 tx=0x7f18f4014910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c7080922a9738 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:27.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.576+0000 7f18e5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:27.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.577+0000 7f18e5600700 1 -- 172.21.15.63:0/2220164215 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4023070 con 0x5625e16a5820 2024-05-27T07:37:27.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.577+0000 7f18e5600700 1 -- 172.21.15.63:0/2220164215 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f18f4014d40 con 0x5625e16a5820 2024-05-27T07:37:27.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.577+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2220164215 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5625e15cf570 con 0x5625e16a5820 2024-05-27T07:37:27.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.578+0000 7f18e5600700 1 -- 172.21.15.63:0/2220164215 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f18f4004a90 con 0x5625e16a5820 2024-05-27T07:37:27.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.578+0000 7f1910b7bb80 10 client.16140 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:27.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.578+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2220164215 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5625e172a9a0 con 0x5625e16a5820 2024-05-27T07:37:27.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.578+0000 7f18e5600700 1 -- 172.21.15.63:0/2220164215 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5625e15cf570 con 0x5625e16a5820 2024-05-27T07:37:27.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.578+0000 7f18e5600700 1 -- 172.21.15.63:0/2220164215 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f18f4005050 con 0x5625e16a5820 2024-05-27T07:37:27.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.578+0000 7f1910b7bb80 10 client.16140 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:27.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.578+0000 7f1910b7bb80 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:27.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.578+0000 7f1910b7bb80 20 client.16140 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:27.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.579+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625e172ad20 0x5625e172d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:27.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.579+0000 7f1910b7bb80 4 client.16140 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:37:27.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.579+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2220164215 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5625e1446250 con 0x5625e172ad20 2024-05-27T07:37:27.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.579+0000 7f18fb400700 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625e172ad20 0x5625e172d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:27.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.580+0000 7f18fb400700 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625e172ad20 0x5625e172d140 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ed16a0511f1554 server_cookie=15d3c5ca2d6e2414 in_seq=0 out_seq=0 2024-05-27T07:37:27.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.580+0000 7f18e5600700 10 client.16140 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:27.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f18e5600700 1 -- 172.21.15.63:0/2220164215 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5625e1446250 con 0x5625e172ad20 2024-05-27T07:37:27.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f18e5600700 10 client.16140 handle_command_reply: tid=0 2024-05-27T07:37:27.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f1910b7bb80 1 client.16140 shutdown 2024-05-27T07:37:27.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2220164215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5820 msgr2=0x5625e17293f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:27.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5820 0x5625e17293f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f18f4000f80 tx=0x7f18f4014910 comp rx=0 tx=0).stop 2024-05-27T07:37:27.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2220164215 shutdown_connections 2024-05-27T07:37:27.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625e172ad20 0x5625e172d140 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625e16a5820 0x5625e17293f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625e1526e40 0x5625e16b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f1910b7bb80 1 --2- 172.21.15.63:0/2220164215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625e16a5410 0x5625e16b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f18e5600700 0 client.16140 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:27.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.581+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2220164215 >> 172.21.15.63:0/2220164215 conn(0x5625e1586fc0 msgr2=0x5625e16b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:27.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.582+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2220164215 shutdown_connections 2024-05-27T07:37:27.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.582+0000 7f1910b7bb80 1 -- 172.21.15.63:0/2220164215 wait complete. 2024-05-27T07:37:27.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.582+0000 7f1910b7bb80 20 client.16140 trim_cache size 0 max 16384 2024-05-27T07:37:27.581 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:37:27.581 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:37:27.581 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:37:27.581 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:37:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.583+0000 7f1900a00700 1 -- 172.21.15.63:0/1873083079 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18e00b2b40 msgr2=0x7f18e00b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.583+0000 7f1900a00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18e00b2b40 0x7f18e00b5050 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5625e16a6430 tx=0x7f18ec01a040 comp rx=0 tx=0).stop 2024-05-27T07:37:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.583+0000 7f1900a00700 1 -- 172.21.15.63:0/1873083079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18fc105c50 msgr2=0x7f18fc19d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.583+0000 7f1900a00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18fc105c50 0x7f18fc19d320 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f18e800d980 tx=0x7f18e800dd40 comp rx=0 tx=0).stop 2024-05-27T07:37:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.583+0000 7f1900a00700 1 -- 172.21.15.63:0/1873083079 shutdown_connections 2024-05-27T07:37:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.583+0000 7f1900a00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18e00b2b40 0x7f18e00b5050 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.583+0000 7f1900a00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18fc10e730 0x7f18fc1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.583+0000 7f1900a00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18fc105300 0x7f18fc19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.583+0000 7f1900a00700 1 --2- 172.21.15.63:0/1873083079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18fc105c50 0x7f18fc19d320 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:27.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.583+0000 7f1900a00700 1 -- 172.21.15.63:0/1873083079 >> 172.21.15.63:0/1873083079 conn(0x7f18fc078c30 msgr2=0x7f18fc0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:27.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.584+0000 7f1900a00700 1 -- 172.21.15.63:0/1873083079 shutdown_connections 2024-05-27T07:37:27.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:27.584+0000 7f1900a00700 1 -- 172.21.15.63:0/1873083079 wait complete. 2024-05-27T07:37:27.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:27 smithi086 ceph-mon[34687]: pgmap v120: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:37:27.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:27 smithi149 ceph-mon[35408]: pgmap v120: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:37:28.594 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:37:28.594 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:37:29.076 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:37:29.602 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:29 smithi063 ceph-mon[31613]: pgmap v121: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:37:29.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:29 smithi086 ceph-mon[34687]: pgmap v121: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:37:29.708 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:29 smithi149 ceph-mon[35408]: pgmap v121: 129 pgs: 129 active+clean; 305 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:37:31.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:31 smithi063 ceph-mon[31613]: pgmap v122: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 14 op/s 2024-05-27T07:37:31.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:31 smithi086 ceph-mon[34687]: pgmap v122: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 14 op/s 2024-05-27T07:37:31.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.710+0000 7f2a13200700 1 -- 172.21.15.63:0/3142764618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a14105c50 msgr2=0x7f2a1410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.710+0000 7f2a13200700 1 --2- 172.21.15.63:0/3142764618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a14105c50 0x7f2a1410e1a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009990 tx=0x7f2a00009ca0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.710+0000 7f2a13200700 1 -- 172.21.15.63:0/3142764618 shutdown_connections 2024-05-27T07:37:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.710+0000 7f2a13200700 1 --2- 172.21.15.63:0/3142764618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a1410e730 0x7f2a14110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.710+0000 7f2a13200700 1 --2- 172.21.15.63:0/3142764618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a14105300 0x7f2a14105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.710+0000 7f2a13200700 1 --2- 172.21.15.63:0/3142764618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a14105c50 0x7f2a1410e1a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.710+0000 7f2a13200700 1 -- 172.21.15.63:0/3142764618 >> 172.21.15.63:0/3142764618 conn(0x7f2a14078ab0 msgr2=0x7f2a14078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.710+0000 7f2a13200700 1 -- 172.21.15.63:0/3142764618 shutdown_connections 2024-05-27T07:37:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.710+0000 7f2a13200700 1 -- 172.21.15.63:0/3142764618 wait complete. 2024-05-27T07:37:31.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:31 smithi149 ceph-mon[35408]: pgmap v122: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 14 op/s 2024-05-27T07:37:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.712+0000 7f2a13200700 1 Processor -- start 2024-05-27T07:37:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.712+0000 7f2a13200700 1 -- start start 2024-05-27T07:37:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.712+0000 7f2a13200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14105300 0x7f2a1419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.712+0000 7f2a13200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a14105c50 0x7f2a1419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.712+0000 7f2a13200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a1410e730 0x7f2a141a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.712+0000 7f2a13200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14114100 con 0x7f2a1410e730 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.712+0000 7f2a13200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14113f80 con 0x7f2a14105300 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.712+0000 7f2a13200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2a14114280 con 0x7f2a14105c50 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.713+0000 7f2a12800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a1410e730 0x7f2a141a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.713+0000 7f2a12800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a1410e730 0x7f2a141a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49524/0 (socket says 172.21.15.63:49524) 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.713+0000 7f2a12800700 1 -- 172.21.15.63:0/3173558355 learned_addr learned my addr 172.21.15.63:0/3173558355 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.713+0000 7f2a11e00700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14105300 0x7f2a1419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.713+0000 7f2a11400700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a14105c50 0x7f2a1419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.713+0000 7f2a12800700 1 -- 172.21.15.63:0/3173558355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a14105c50 msgr2=0x7f2a1419d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.713+0000 7f2a12800700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a14105c50 0x7f2a1419d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.713+0000 7f2a12800700 1 -- 172.21.15.63:0/3173558355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14105300 msgr2=0x7f2a1419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.713+0000 7f2a12800700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14105300 0x7f2a1419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.713+0000 7f2a12800700 1 -- 172.21.15.63:0/3173558355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a141a1bd0 con 0x7f2a1410e730 2024-05-27T07:37:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.714+0000 7f2a12800700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a1410e730 0x7f2a141a15e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c007d70 tx=0x7f2a0c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.714+0000 7f2a0aa00700 1 -- 172.21.15.63:0/3173558355 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c016020 con 0x7f2a1410e730 2024-05-27T07:37:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.714+0000 7f2a0aa00700 1 -- 172.21.15.63:0/3173558355 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a0c00ac90 con 0x7f2a1410e730 2024-05-27T07:37:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.714+0000 7f2a13200700 1 -- 172.21.15.63:0/3173558355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a141a1ec0 con 0x7f2a1410e730 2024-05-27T07:37:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.715+0000 7f2a0aa00700 1 -- 172.21.15.63:0/3173558355 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0115e0 con 0x7f2a1410e730 2024-05-27T07:37:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.715+0000 7f2a13200700 1 -- 172.21.15.63:0/3173558355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a141a2380 con 0x7f2a1410e730 2024-05-27T07:37:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.716+0000 7f2a13200700 1 -- 172.21.15.63:0/3173558355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f4005330 con 0x7f2a1410e730 2024-05-27T07:37:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.719+0000 7f2a0aa00700 1 -- 172.21.15.63:0/3173558355 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2a0c019020 con 0x7f2a1410e730 2024-05-27T07:37:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.721+0000 7f2a0aa00700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29f80b29f0 0x7f29f80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.721+0000 7f2a0aa00700 1 -- 172.21.15.63:0/3173558355 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2a0c0e2c80 con 0x7f2a1410e730 2024-05-27T07:37:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.721+0000 7f2a11e00700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29f80b29f0 0x7f29f80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.722+0000 7f2a11e00700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29f80b29f0 0x7f29f80b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a04005950 tx=0x7f2a0400b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:31.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:31.724+0000 7f2a0aa00700 1 -- 172.21.15.63:0/3173558355 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a0c020020 con 0x7f2a1410e730 2024-05-27T07:37:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.979+0000 7f5746c00640 1 -- 172.21.15.63:0/2242317882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5740110520 msgr2=0x7f574010a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.979+0000 7f5746c00640 1 --2- 172.21.15.63:0/2242317882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5740110520 0x7f574010a200 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f572c009900 tx=0x7f572c02a480 comp rx=0 tx=0).stop 2024-05-27T07:37:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.979+0000 7f5746c00640 1 -- 172.21.15.63:0/2242317882 shutdown_connections 2024-05-27T07:37:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.979+0000 7f5746c00640 1 --2- 172.21.15.63:0/2242317882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574010a890 0x7f5740116b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.979+0000 7f5746c00640 1 --2- 172.21.15.63:0/2242317882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5740110520 0x7f574010a200 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.979+0000 7f5746c00640 1 --2- 172.21.15.63:0/2242317882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740110110 0x7f5740109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.979+0000 7f5746c00640 1 -- 172.21.15.63:0/2242317882 >> 172.21.15.63:0/2242317882 conn(0x7f5740076070 msgr2=0x7f5740078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.979+0000 7f5746c00640 1 -- 172.21.15.63:0/2242317882 shutdown_connections 2024-05-27T07:37:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.979+0000 7f5746c00640 1 -- 172.21.15.63:0/2242317882 wait complete. 2024-05-27T07:37:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.980+0000 7f5746c00640 1 Processor -- start 2024-05-27T07:37:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.980+0000 7f5746c00640 1 -- start start 2024-05-27T07:37:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.981+0000 7f5746c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740110110 0x7f57401a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.981+0000 7f5746c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5740110520 0x7f57401a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.981+0000 7f5746c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57401a4f40 0x7f57401a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.981+0000 7f5746c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f574011a140 con 0x7f57401a4f40 2024-05-27T07:37:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.981+0000 7f5746c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5740119fc0 con 0x7f5740110520 2024-05-27T07:37:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.981+0000 7f5746c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f574011a2c0 con 0x7f5740110110 2024-05-27T07:37:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.981+0000 7f5746200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57401a4f40 0x7f57401a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.981+0000 7f5746200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57401a4f40 0x7f57401a5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49540/0 (socket says 172.21.15.63:49540) 2024-05-27T07:37:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.981+0000 7f5746200640 1 -- 172.21.15.63:0/865558840 learned_addr learned my addr 172.21.15.63:0/865558840 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.982+0000 7f5745800640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740110110 0x7f57401a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.982+0000 7f5744e00640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5740110520 0x7f57401a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.982+0000 7f5746200640 1 -- 172.21.15.63:0/865558840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740110110 msgr2=0x7f57401a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.982+0000 7f5746200640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740110110 0x7f57401a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.982+0000 7f5746200640 1 -- 172.21.15.63:0/865558840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5740110520 msgr2=0x7f57401a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.982+0000 7f5746200640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5740110520 0x7f57401a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.982+0000 7f5746200640 1 -- 172.21.15.63:0/865558840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57401a5910 con 0x7f57401a4f40 2024-05-27T07:37:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.982+0000 7f5746200640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57401a4f40 0x7f57401a5350 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f573800cd10 tx=0x7f573800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.983+0000 7f573e000640 1 -- 172.21.15.63:0/865558840 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573800f450 con 0x7f57401a4f40 2024-05-27T07:37:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.983+0000 7f573e000640 1 -- 172.21.15.63:0/865558840 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f573800fa80 con 0x7f57401a4f40 2024-05-27T07:37:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.983+0000 7f5746c00640 1 -- 172.21.15.63:0/865558840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57401a5c00 con 0x7f57401a4f40 2024-05-27T07:37:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.983+0000 7f573e000640 1 -- 172.21.15.63:0/865558840 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738018ad0 con 0x7f57401a4f40 2024-05-27T07:37:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.983+0000 7f5746c00640 1 -- 172.21.15.63:0/865558840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57401ad3d0 con 0x7f57401a4f40 2024-05-27T07:37:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.988+0000 7f574f24a740 1 Processor -- start 2024-05-27T07:37:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.988+0000 7f573e000640 1 -- 172.21.15.63:0/865558840 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f573800fc30 con 0x7f57401a4f40 2024-05-27T07:37:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.989+0000 7f574f24a740 1 -- start start 2024-05-27T07:37:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.989+0000 7f574f24a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e43031b0 0x5590e46a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.989+0000 7f574f24a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42da2f0 0x5590e46a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.989+0000 7f574f24a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.989+0000 7f574f24a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590e4616b90 con 0x5590e42e24d0 2024-05-27T07:37:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.989+0000 7f574f24a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5590e4606730 con 0x5590e43031b0 2024-05-27T07:37:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.989+0000 7f574f24a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5590e460a770 con 0x5590e42da2f0 2024-05-27T07:37:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.990+0000 7f5744e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.990+0000 7f5744e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49550/0 (socket says 172.21.15.63:49550) 2024-05-27T07:37:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.990+0000 7f5744e00640 1 -- 172.21.15.63:0/1064812136 learned_addr learned my addr 172.21.15.63:0/1064812136 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.990+0000 7f5745800640 1 --2- 172.21.15.63:0/1064812136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42da2f0 0x5590e46a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.990+0000 7f5744e00640 1 --2- 172.21.15.63:0/1064812136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e43031b0 0x5590e46a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.991+0000 7f5744e00640 1 -- 172.21.15.63:0/1064812136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42da2f0 msgr2=0x5590e46a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.991+0000 7f5744e00640 1 --2- 172.21.15.63:0/1064812136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42da2f0 0x5590e46a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.991+0000 7f5744e00640 1 -- 172.21.15.63:0/1064812136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e43031b0 msgr2=0x5590e46a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.991+0000 7f5744e00640 1 --2- 172.21.15.63:0/1064812136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e43031b0 0x5590e46a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.991+0000 7f5744e00640 1 -- 172.21.15.63:0/1064812136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e459f8b0 con 0x5590e42e24d0 2024-05-27T07:37:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.991+0000 7f573e000640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57280b2950 0x7f57280b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.991+0000 7f573e000640 1 -- 172.21.15.63:0/865558840 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f57380e30c0 con 0x7f57401a4f40 2024-05-27T07:37:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.991+0000 7f5744e00640 1 --2- 172.21.15.63:0/1064812136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a64e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f572c0096c0 tx=0x7f572c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1416b76b6e976c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.992+0000 7f5745800640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57280b2950 0x7f57280b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.992+0000 7f5732a00640 1 -- 172.21.15.63:0/1064812136 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c035040 con 0x5590e42e24d0 2024-05-27T07:37:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.992+0000 7f5732a00640 1 -- 172.21.15.63:0/1064812136 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f572c0044c0 con 0x5590e42e24d0 2024-05-27T07:37:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.992+0000 7f5732a00640 1 -- 172.21.15.63:0/1064812136 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c003bc0 con 0x5590e42e24d0 2024-05-27T07:37:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.993+0000 7f574f24a740 1 -- 172.21.15.63:0/1064812136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 msgr2=0x5590e46a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.993+0000 7f574f24a740 1 --2- 172.21.15.63:0/1064812136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a64e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f572c0096c0 tx=0x7f572c005fd0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.993+0000 7f5745800640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57280b2950 0x7f57280b4e60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5590e46a6e10 tx=0x7f5734005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.993+0000 7f574f24a740 1 -- 172.21.15.63:0/1064812136 shutdown_connections 2024-05-27T07:37:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.993+0000 7f574f24a740 1 --2- 172.21.15.63:0/1064812136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a64e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.993+0000 7f574f24a740 1 --2- 172.21.15.63:0/1064812136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42da2f0 0x5590e46a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.993+0000 7f574f24a740 1 --2- 172.21.15.63:0/1064812136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e43031b0 0x5590e46a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.993+0000 7f574f24a740 1 -- 172.21.15.63:0/1064812136 >> 172.21.15.63:0/1064812136 conn(0x5590e42f7300 msgr2=0x5590e46a2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.993+0000 7f574f24a740 1 -- 172.21.15.63:0/1064812136 shutdown_connections 2024-05-27T07:37:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.994+0000 7f574f24a740 1 -- 172.21.15.63:0/1064812136 wait complete. 2024-05-27T07:37:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.994+0000 7f574f24a740 1 -- start start 2024-05-27T07:37:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.994+0000 7f574f24a740 1 Processor -- start 2024-05-27T07:37:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.995+0000 7f574f24a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42da2f0 0x5590e46afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.995+0000 7f574f24a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e42e24d0 0x5590e46b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.995+0000 7f574f24a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e43031b0 0x5590e46bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.995+0000 7f574f24a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590e42ec790 con 0x5590e42da2f0 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.995+0000 7f574f24a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5590e45aebc0 con 0x5590e42e24d0 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.995+0000 7f574f24a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5590e46b2b30 con 0x5590e43031b0 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.995+0000 7f574f24a740 10 client.0 _ll_register_callbacks cb 0x5590e458c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.995+0000 7f5745800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42da2f0 0x5590e46afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.995+0000 7f5745800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42da2f0 0x5590e46afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49560/0 (socket says 172.21.15.63:49560) 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.995+0000 7f5745800640 1 -- 172.21.15.63:0/3410742831 learned_addr learned my addr 172.21.15.63:0/3410742831 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.996+0000 7f5746200640 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e43031b0 0x5590e46bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.996+0000 7f5744e00640 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e42e24d0 0x5590e46b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.996+0000 7f5745800640 1 -- 172.21.15.63:0/3410742831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e43031b0 msgr2=0x5590e46bbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.996+0000 7f5745800640 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e43031b0 0x5590e46bbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.996+0000 7f5745800640 1 -- 172.21.15.63:0/3410742831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e42e24d0 msgr2=0x5590e46b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.996+0000 7f5745800640 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e42e24d0 0x5590e46b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.996+0000 7f5745800640 1 -- 172.21.15.63:0/3410742831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e431b210 con 0x5590e42da2f0 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.996+0000 7f5745800640 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42da2f0 0x5590e46afed0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f573401cbe0 tx=0x7f5734004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10a51b354cce91e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.996+0000 7f5727400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.997+0000 7f5727400640 1 -- 172.21.15.63:0/3410742831 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573402c060 con 0x5590e42da2f0 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.997+0000 7f574f24a740 1 -- 172.21.15.63:0/3410742831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590e4618ae0 con 0x5590e42da2f0 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.997+0000 7f5727400640 1 -- 172.21.15.63:0/3410742831 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f5734016030 con 0x5590e42da2f0 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.997+0000 7f5727400640 1 -- 172.21.15.63:0/3410742831 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5734033050 con 0x5590e42da2f0 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.997+0000 7f574f24a740 10 client.16158 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.997+0000 7f574f24a740 1 -- 172.21.15.63:0/3410742831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5590e43e7620 con 0x5590e42da2f0 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.998+0000 7f5727400640 1 -- 172.21.15.63:0/3410742831 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5590e4618ae0 con 0x5590e42da2f0 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.998+0000 7f5727400640 1 -- 172.21.15.63:0/3410742831 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f573401a530 con 0x5590e42da2f0 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.998+0000 7f574f24a740 10 client.16158 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.998+0000 7f574f24a740 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.998+0000 7f574f24a740 20 client.16158 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.998+0000 7f574f24a740 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590e42c1d80 0x5590e46c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.998+0000 7f574f24a740 4 client.16158 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.998+0000 7f574f24a740 1 -- 172.21.15.63:0/3410742831 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590e46122c0 con 0x5590e42c1d80 2024-05-27T07:37:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:31.999+0000 7f5744e00640 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590e42c1d80 0x5590e46c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.000+0000 7f5727400640 10 client.16158 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.000+0000 7f5744e00640 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590e42c1d80 0x5590e46c1f60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2b5455eeaf26f4 server_cookie=5e0233a6cd12ca4d in_seq=0 out_seq=0 2024-05-27T07:37:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f5727400640 1 -- 172.21.15.63:0/3410742831 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5590e46122c0 con 0x5590e42c1d80 2024-05-27T07:37:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f5727400640 10 client.16158 handle_command_reply: tid=0 2024-05-27T07:37:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f574f24a740 1 client.16158 shutdown 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f574f24a740 1 -- 172.21.15.63:0/3410742831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42da2f0 msgr2=0x5590e46afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f574f24a740 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42da2f0 0x5590e46afed0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f573401cbe0 tx=0x7f5734004730 comp rx=0 tx=0).stop 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f574f24a740 1 -- 172.21.15.63:0/3410742831 shutdown_connections 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f574f24a740 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590e42c1d80 0x5590e46c1f60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f574f24a740 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e43031b0 0x5590e46bbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f5727400640 0 client.16158 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f574f24a740 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e42e24d0 0x5590e46b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f574f24a740 1 --2- 172.21.15.63:0/3410742831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42da2f0 0x5590e46afed0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.002+0000 7f574f24a740 1 -- 172.21.15.63:0/3410742831 >> 172.21.15.63:0/3410742831 conn(0x5590e42f7300 msgr2=0x5590e46a7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.003+0000 7f574f24a740 1 -- 172.21.15.63:0/3410742831 shutdown_connections 2024-05-27T07:37:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.003+0000 7f574f24a740 1 -- 172.21.15.63:0/3410742831 wait complete. 2024-05-27T07:37:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.003+0000 7f574f24a740 20 client.16158 trim_cache size 0 max 16384 2024-05-27T07:37:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.020+0000 7f574f24a740 1 Processor -- start 2024-05-27T07:37:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.020+0000 7f574f24a740 1 -- start start 2024-05-27T07:37:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f574f24a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e46a80b0 0x5590e46bbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f574f24a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42f7300 0x5590e46bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f574f24a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f574f24a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57401b9d50 con 0x5590e42e24d0 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f574f24a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57401b75c0 con 0x5590e46a80b0 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f574f24a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57401b5410 con 0x5590e42f7300 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f5746200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f5746200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49566/0 (socket says 172.21.15.63:49566) 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f5746200640 1 -- 172.21.15.63:0/972632956 learned_addr learned my addr 172.21.15.63:0/972632956 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f5744e00640 1 --2- 172.21.15.63:0/972632956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42f7300 0x5590e46bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.021+0000 7f5745800640 1 --2- 172.21.15.63:0/972632956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e46a80b0 0x5590e46bbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.022+0000 7f5746200640 1 -- 172.21.15.63:0/972632956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42f7300 msgr2=0x5590e46bc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.022+0000 7f5746200640 1 --2- 172.21.15.63:0/972632956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42f7300 0x5590e46bc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.022+0000 7f5746200640 1 -- 172.21.15.63:0/972632956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e46a80b0 msgr2=0x5590e46bbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.022+0000 7f5746200640 1 --2- 172.21.15.63:0/972632956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e46a80b0 0x5590e46bbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.022+0000 7f5746200640 1 -- 172.21.15.63:0/972632956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e46ae210 con 0x5590e42e24d0 2024-05-27T07:37:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.022+0000 7f5746200640 1 --2- 172.21.15.63:0/972632956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a9710 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f57380e0550 tx=0x7f57380a9c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261d4a9501a27877 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.022+0000 7f5732000640 1 -- 172.21.15.63:0/972632956 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57380c3020 con 0x5590e42e24d0 2024-05-27T07:37:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.022+0000 7f5732000640 1 -- 172.21.15.63:0/972632956 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57380ae3e0 con 0x5590e42e24d0 2024-05-27T07:37:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.022+0000 7f5732000640 1 -- 172.21.15.63:0/972632956 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57380b2c60 con 0x5590e42e24d0 2024-05-27T07:37:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.024+0000 7f574f24a740 1 -- 172.21.15.63:0/972632956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 msgr2=0x5590e46a9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.024+0000 7f574f24a740 1 --2- 172.21.15.63:0/972632956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a9710 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f57380e0550 tx=0x7f57380a9c00 comp rx=0 tx=0).stop 2024-05-27T07:37:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.024+0000 7f574f24a740 1 -- 172.21.15.63:0/972632956 shutdown_connections 2024-05-27T07:37:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.024+0000 7f574f24a740 1 --2- 172.21.15.63:0/972632956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e42e24d0 0x5590e46a9710 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.024+0000 7f574f24a740 1 --2- 172.21.15.63:0/972632956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42f7300 0x5590e46bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.024+0000 7f574f24a740 1 --2- 172.21.15.63:0/972632956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e46a80b0 0x5590e46bbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.024+0000 7f574f24a740 1 -- 172.21.15.63:0/972632956 >> 172.21.15.63:0/972632956 conn(0x5590e46b2380 msgr2=0x5590e46affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.024+0000 7f574f24a740 1 -- 172.21.15.63:0/972632956 shutdown_connections 2024-05-27T07:37:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.024+0000 7f574f24a740 1 -- 172.21.15.63:0/972632956 wait complete. 2024-05-27T07:37:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.025+0000 7f574f24a740 1 -- start start 2024-05-27T07:37:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.025+0000 7f574f24a740 1 Processor -- start 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f574f24a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42e24d0 0x5590e46b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f574f24a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e42f7300 0x5590e46b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f574f24a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e46a80b0 0x5590e46b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f574f24a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57401b5a80 con 0x5590e46a80b0 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f574f24a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57401b6ef0 con 0x5590e42f7300 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f574f24a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57401b6150 con 0x5590e42e24d0 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f574f24a740 10 client.0 _ll_register_callbacks cb 0x7f573400a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f5746200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e46a80b0 0x5590e46b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f5746200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e46a80b0 0x5590e46b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49572/0 (socket says 172.21.15.63:49572) 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f5746200640 1 -- 172.21.15.63:0/3258619746 learned_addr learned my addr 172.21.15.63:0/3258619746 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f5744e00640 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e42f7300 0x5590e46b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.026+0000 7f5745800640 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42e24d0 0x5590e46b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.027+0000 7f5746200640 1 -- 172.21.15.63:0/3258619746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42e24d0 msgr2=0x5590e46b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.027+0000 7f5746200640 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42e24d0 0x5590e46b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.027+0000 7f5746200640 1 -- 172.21.15.63:0/3258619746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e42f7300 msgr2=0x5590e46b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.027+0000 7f5746200640 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e42f7300 0x5590e46b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.027+0000 7f5746200640 1 -- 172.21.15.63:0/3258619746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e46b3fa0 con 0x5590e46a80b0 2024-05-27T07:37:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.027+0000 7f5745800640 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42e24d0 0x5590e46b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.027+0000 7f5746200640 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e46a80b0 0x5590e46b3a60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f57380a9fc0 tx=0x7f57380b2e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebd284d8e4facdc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.027+0000 7f5727400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.027+0000 7f5727400640 1 -- 172.21.15.63:0/3258619746 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57380c3070 con 0x5590e46a80b0 2024-05-27T07:37:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.028+0000 7f5727400640 1 -- 172.21.15.63:0/3258619746 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f57380b2a00 con 0x5590e46a80b0 2024-05-27T07:37:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.028+0000 7f574f24a740 1 -- 172.21.15.63:0/3258619746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57380ae3e0 con 0x5590e46a80b0 2024-05-27T07:37:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.028+0000 7f5727400640 1 -- 172.21.15.63:0/3258619746 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f57380b2c00 con 0x5590e46a80b0 2024-05-27T07:37:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.028+0000 7f574f24a740 10 client.16164 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.028+0000 7f574f24a740 1 -- 172.21.15.63:0/3258619746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5590e472a270 con 0x5590e46a80b0 2024-05-27T07:37:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.028+0000 7f5727400640 1 -- 172.21.15.63:0/3258619746 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57380ae3e0 con 0x5590e46a80b0 2024-05-27T07:37:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.028+0000 7f5727400640 1 -- 172.21.15.63:0/3258619746 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f57380a59a0 con 0x5590e46a80b0 2024-05-27T07:37:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.029+0000 7f574f24a740 10 client.16164 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.029+0000 7f574f24a740 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.029+0000 7f574f24a740 20 client.16164 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.029+0000 7f574f24a740 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590e43031b0 0x5590e472c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.029+0000 7f574f24a740 4 client.16164 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:37:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.029+0000 7f574f24a740 1 -- 172.21.15.63:0/3258619746 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590e458c4b0 con 0x5590e43031b0 2024-05-27T07:37:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.029+0000 7f5745800640 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590e43031b0 0x5590e472c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.030+0000 7f5745800640 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590e43031b0 0x5590e472c950 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34260b9f8af2b365 server_cookie=3f6ae316871b2e00 in_seq=0 out_seq=0 2024-05-27T07:37:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.030+0000 7f5727400640 10 client.16164 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.067+0000 7f5727400640 1 -- 172.21.15.63:0/3258619746 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5590e458c4b0 con 0x5590e43031b0 2024-05-27T07:37:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f5727400640 10 client.16164 handle_command_reply: tid=0 2024-05-27T07:37:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 client.16164 shutdown 2024-05-27T07:37:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 -- 172.21.15.63:0/3258619746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e46a80b0 msgr2=0x5590e46b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e46a80b0 0x5590e46b3a60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f57380a9fc0 tx=0x7f57380b2e90 comp rx=0 tx=0).stop 2024-05-27T07:37:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 -- 172.21.15.63:0/3258619746 shutdown_connections 2024-05-27T07:37:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590e43031b0 0x5590e472c950 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590e46a80b0 0x5590e46b3a60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590e42f7300 0x5590e46b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 --2- 172.21.15.63:0/3258619746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590e42e24d0 0x5590e46b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f5727400640 0 client.16164 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 -- 172.21.15.63:0/3258619746 >> 172.21.15.63:0/3258619746 conn(0x5590e46b2380 msgr2=0x5590e46a5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 -- 172.21.15.63:0/3258619746 shutdown_connections 2024-05-27T07:37:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 1 -- 172.21.15.63:0/3258619746 wait complete. 2024-05-27T07:37:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.068+0000 7f574f24a740 20 client.16164 trim_cache size 0 max 16384 2024-05-27T07:37:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:37:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:37:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:37:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 -- 172.21.15.63:0/865558840 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57280b2950 msgr2=0x7f57280b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57280b2950 0x7f57280b4e60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5590e46a6e10 tx=0x7f5734005f90 comp rx=0 tx=0).stop 2024-05-27T07:37:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 -- 172.21.15.63:0/865558840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57401a4f40 msgr2=0x7f57401a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57401a4f40 0x7f57401a5350 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f573800cd10 tx=0x7f573800a620 comp rx=0 tx=0).stop 2024-05-27T07:37:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 -- 172.21.15.63:0/865558840 shutdown_connections 2024-05-27T07:37:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57280b2950 0x7f57280b4e60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57401a4f40 0x7f57401a5350 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5740110520 0x7f57401a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 --2- 172.21.15.63:0/865558840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740110110 0x7f57401a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 -- 172.21.15.63:0/865558840 >> 172.21.15.63:0/865558840 conn(0x7f5740076070 msgr2=0x7f5740076780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 -- 172.21.15.63:0/865558840 shutdown_connections 2024-05-27T07:37:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:32.070+0000 7f5746c00640 1 -- 172.21.15.63:0/865558840 wait complete. 2024-05-27T07:37:32.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.084+0000 7f2a13200700 1 -- 172.21.15.63:0/3173558355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29f4005620 con 0x7f2a1410e730 2024-05-27T07:37:32.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.085+0000 7f2a0aa00700 1 -- 172.21.15.63:0/3173558355 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11865 (secure 0 0 0) 0x7f2a0c0aa3a0 con 0x7f2a1410e730 2024-05-27T07:37:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:37:32.083 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:36:42.884621+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T07:37:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 -- 172.21.15.63:0/3173558355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29f80b29f0 msgr2=0x7f29f80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29f80b29f0 0x7f29f80b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a04005950 tx=0x7f2a0400b410 comp rx=0 tx=0).stop 2024-05-27T07:37:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 -- 172.21.15.63:0/3173558355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a1410e730 msgr2=0x7f2a141a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a1410e730 0x7f2a141a15e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c007d70 tx=0x7f2a0c00a340 comp rx=0 tx=0).stop 2024-05-27T07:37:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 -- 172.21.15.63:0/3173558355 shutdown_connections 2024-05-27T07:37:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29f80b29f0 0x7f29f80b4f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a1410e730 0x7f2a141a15e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a14105300 0x7f2a1419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 --2- 172.21.15.63:0/3173558355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a14105c50 0x7f2a1419d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 -- 172.21.15.63:0/3173558355 >> 172.21.15.63:0/3173558355 conn(0x7f2a14078ab0 msgr2=0x7f2a140ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 -- 172.21.15.63:0/3173558355 shutdown_connections 2024-05-27T07:37:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:32.089+0000 7f2a13200700 1 -- 172.21.15.63:0/3173558355 wait complete. 2024-05-27T07:37:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 32 2024-05-27T07:37:32.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:32 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3173558355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:37:32.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:32 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3173558355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:37:32.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:32 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3173558355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:37:33.082 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:37:33.562 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:37:33.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:33 smithi063 ceph-mon[31613]: pgmap v123: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:37:33.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:33 smithi086 ceph-mon[34687]: pgmap v123: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:37:33.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:33 smithi149 ceph-mon[35408]: pgmap v123: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:37:35.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:35 smithi086 ceph-mon[34687]: pgmap v124: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:37:35.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:35 smithi149 ceph-mon[35408]: pgmap v124: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:37:35.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:35 smithi063 ceph-mon[31613]: pgmap v124: 129 pgs: 129 active+clean; 345 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:37:36.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.068+0000 7f215be00700 1 -- 172.21.15.63:0/173419303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c105280 msgr2=0x7f215c105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:36.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.068+0000 7f215be00700 1 --2- 172.21.15.63:0/173419303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c105280 0x7f215c105690 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f214c009990 tx=0x7f214c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.068+0000 7f215be00700 1 -- 172.21.15.63:0/173419303 shutdown_connections 2024-05-27T07:37:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.068+0000 7f215be00700 1 --2- 172.21.15.63:0/173419303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f215c10e6f0 0x7f215c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.068+0000 7f215be00700 1 --2- 172.21.15.63:0/173419303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c105280 0x7f215c105690 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.068+0000 7f215be00700 1 --2- 172.21.15.63:0/173419303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f215c105bd0 0x7f215c10e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.068+0000 7f215be00700 1 -- 172.21.15.63:0/173419303 >> 172.21.15.63:0/173419303 conn(0x7f215c078ab0 msgr2=0x7f215c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.068+0000 7f215be00700 1 -- 172.21.15.63:0/173419303 shutdown_connections 2024-05-27T07:37:36.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.068+0000 7f215be00700 1 -- 172.21.15.63:0/173419303 wait complete. 2024-05-27T07:37:36.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.069+0000 7f215be00700 1 Processor -- start 2024-05-27T07:37:36.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.070+0000 7f215be00700 1 -- start start 2024-05-27T07:37:36.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.070+0000 7f215be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f215c105280 0x7f215c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.070+0000 7f215be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c105bd0 0x7f215c19d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.070+0000 7f215be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f215c10e6f0 0x7f215c1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.070+0000 7f215be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f215c114140 con 0x7f215c105bd0 2024-05-27T07:37:36.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.070+0000 7f215be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f215c113fc0 con 0x7f215c105280 2024-05-27T07:37:36.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.070+0000 7f215be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1142c0 con 0x7f215c10e6f0 2024-05-27T07:37:36.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.071+0000 7f215a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c105bd0 0x7f215c19d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.071+0000 7f215a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c105bd0 0x7f215c19d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49586/0 (socket says 172.21.15.63:49586) 2024-05-27T07:37:36.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.071+0000 7f215a000700 1 -- 172.21.15.63:0/2257729296 learned_addr learned my addr 172.21.15.63:0/2257729296 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:36.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.071+0000 7f215aa00700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f215c105280 0x7f215c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.071+0000 7f215a000700 1 -- 172.21.15.63:0/2257729296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f215c10e6f0 msgr2=0x7f215c1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:36.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.071+0000 7f215a000700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f215c10e6f0 0x7f215c1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.071+0000 7f215a000700 1 -- 172.21.15.63:0/2257729296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f215c105280 msgr2=0x7f215c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:36.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.071+0000 7f215a000700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f215c105280 0x7f215c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.071+0000 7f215a000700 1 -- 172.21.15.63:0/2257729296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c1a1c40 con 0x7f215c105bd0 2024-05-27T07:37:36.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.071+0000 7f215aa00700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f215c105280 0x7f215c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:36.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.072+0000 7f215a000700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c105bd0 0x7f215c19d310 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f214800e9a0 tx=0x7f214800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:36.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.072+0000 7f2153400700 1 -- 172.21.15.63:0/2257729296 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214800cbd0 con 0x7f215c105bd0 2024-05-27T07:37:36.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.072+0000 7f2153400700 1 -- 172.21.15.63:0/2257729296 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f214800cd70 con 0x7f215c105bd0 2024-05-27T07:37:36.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.072+0000 7f215be00700 1 -- 172.21.15.63:0/2257729296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c1a1f30 con 0x7f215c105bd0 2024-05-27T07:37:36.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.072+0000 7f2153400700 1 -- 172.21.15.63:0/2257729296 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148018680 con 0x7f215c105bd0 2024-05-27T07:37:36.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.072+0000 7f215be00700 1 -- 172.21.15.63:0/2257729296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c1a9850 con 0x7f215c105bd0 2024-05-27T07:37:36.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.078+0000 7f2170543b80 1 Processor -- start 2024-05-27T07:37:36.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.078+0000 7f2153400700 1 -- 172.21.15.63:0/2257729296 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2148018820 con 0x7f215c105bd0 2024-05-27T07:37:36.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.078+0000 7f2170543b80 1 -- start start 2024-05-27T07:37:36.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.079+0000 7f2170543b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77667ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.079+0000 7f2170543b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 0x55bc77672ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.079+0000 7f2170543b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668820 0x55bc77675020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.079+0000 7f2170543b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bc77596e40 con 0x55bc774e9e40 2024-05-27T07:37:36.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.079+0000 7f2170543b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc775869e0 con 0x55bc77668410 2024-05-27T07:37:36.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.079+0000 7f2170543b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bc7758aa20 con 0x55bc77668820 2024-05-27T07:37:36.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f215b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77667ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f215b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77667ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49592/0 (socket says 172.21.15.63:49592) 2024-05-27T07:37:36.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f215b400700 1 -- 172.21.15.63:0/2802607146 learned_addr learned my addr 172.21.15.63:0/2802607146 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:36.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f215aa00700 1 --2- 172.21.15.63:0/2802607146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 0x55bc77672ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f215a000700 1 --2- 172.21.15.63:0/2802607146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668820 0x55bc77675020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f215b400700 1 -- 172.21.15.63:0/2802607146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668820 msgr2=0x55bc77675020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:36.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f215b400700 1 --2- 172.21.15.63:0/2802607146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668820 0x55bc77675020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f215b400700 1 -- 172.21.15.63:0/2802607146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 msgr2=0x55bc77672ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:36.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f215b400700 1 --2- 172.21.15.63:0/2802607146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 0x55bc77672ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f215b400700 1 -- 172.21.15.63:0/2802607146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7754f420 con 0x55bc774e9e40 2024-05-27T07:37:36.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.080+0000 7f2153400700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21400b2a70 0x7f21400b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.081+0000 7f2153400700 1 -- 172.21.15.63:0/2257729296 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f21480e2ce0 con 0x7f215c105bd0 2024-05-27T07:37:36.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.081+0000 7f215aa00700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21400b2a70 0x7f21400b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.081+0000 7f215b400700 1 --2- 172.21.15.63:0/2802607146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77667ed0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2154009be0 tx=0x7f2154009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5c19ae7fc57db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:36.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.082+0000 7f2147e00700 1 -- 172.21.15.63:0/2802607146 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154004bb0 con 0x55bc774e9e40 2024-05-27T07:37:36.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.082+0000 7f2147e00700 1 -- 172.21.15.63:0/2802607146 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2154004d50 con 0x55bc774e9e40 2024-05-27T07:37:36.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.082+0000 7f2147e00700 1 -- 172.21.15.63:0/2802607146 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154013650 con 0x55bc774e9e40 2024-05-27T07:37:36.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.083+0000 7f2170543b80 1 -- 172.21.15.63:0/2802607146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 msgr2=0x55bc77667ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:36.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.083+0000 7f2170543b80 1 --2- 172.21.15.63:0/2802607146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77667ed0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2154009be0 tx=0x7f2154009ef0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.083+0000 7f215aa00700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21400b2a70 0x7f21400b4f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f214c006010 tx=0x7f214c01a040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:36.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.083+0000 7f2170543b80 1 -- 172.21.15.63:0/2802607146 shutdown_connections 2024-05-27T07:37:36.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.083+0000 7f2170543b80 1 --2- 172.21.15.63:0/2802607146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668820 0x55bc77675020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.083+0000 7f2170543b80 1 --2- 172.21.15.63:0/2802607146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77667ed0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.083+0000 7f2170543b80 1 --2- 172.21.15.63:0/2802607146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 0x55bc77672ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.083+0000 7f2170543b80 1 -- 172.21.15.63:0/2802607146 >> 172.21.15.63:0/2802607146 conn(0x55bc77549fc0 msgr2=0x55bc77663d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.083+0000 7f2170543b80 1 -- 172.21.15.63:0/2802607146 shutdown_connections 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.083+0000 7f2170543b80 1 -- 172.21.15.63:0/2802607146 wait complete. 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.084+0000 7f2170543b80 1 -- start start 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.084+0000 7f2170543b80 1 Processor -- start 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.085+0000 7f2170543b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc774e9e40 0x55bc77672600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.085+0000 7f2170543b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 0x55bc77672b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.085+0000 7f2170543b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668820 0x55bc77673080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.085+0000 7f2170543b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bc7748cfa0 con 0x55bc77668820 2024-05-27T07:37:36.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.085+0000 7f2170543b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc77527760 con 0x55bc77668410 2024-05-27T07:37:36.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.085+0000 7f2170543b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bc77675990 con 0x55bc774e9e40 2024-05-27T07:37:36.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.085+0000 7f2170543b80 10 client.0 _ll_register_callbacks cb 0x55bc77530c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:36.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.085+0000 7f215b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668820 0x55bc77673080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.086+0000 7f215a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 0x55bc77672b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.086+0000 7f215aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc774e9e40 0x55bc77672600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.086+0000 7f215a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 0x55bc77672b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:42554/0 (socket says 172.21.15.63:42554) 2024-05-27T07:37:36.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.086+0000 7f215a000700 1 -- 172.21.15.63:0/2111790985 learned_addr learned my addr 172.21.15.63:0/2111790985 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:36.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.086+0000 7f215b400700 1 -- 172.21.15.63:0/2111790985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc774e9e40 msgr2=0x55bc77672600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:36.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.086+0000 7f215b400700 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc774e9e40 0x55bc77672600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.086+0000 7f215b400700 1 -- 172.21.15.63:0/2111790985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 msgr2=0x55bc77672b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:36.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.086+0000 7f215b400700 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 0x55bc77672b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.086+0000 7f215b400700 1 -- 172.21.15.63:0/2111790985 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc774d68d0 con 0x55bc77668820 2024-05-27T07:37:36.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.087+0000 7f215b400700 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668820 0x55bc77673080 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55bc77664690 tx=0x7f2154003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c57f552542fc02 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:36.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.087+0000 7f2144c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:36.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.087+0000 7f2144c00700 1 -- 172.21.15.63:0/2111790985 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154021070 con 0x55bc77668820 2024-05-27T07:37:36.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.087+0000 7f2144c00700 1 -- 172.21.15.63:0/2111790985 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2154005430 con 0x55bc77668820 2024-05-27T07:37:36.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.088+0000 7f2170543b80 1 -- 172.21.15.63:0/2111790985 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2154004d50 con 0x55bc77668820 2024-05-27T07:37:36.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.088+0000 7f2144c00700 1 -- 172.21.15.63:0/2111790985 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f215402d050 con 0x55bc77668820 2024-05-27T07:37:36.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.088+0000 7f2170543b80 10 client.16176 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:36.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.088+0000 7f2170543b80 1 -- 172.21.15.63:0/2111790985 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc77557060 con 0x55bc77668820 2024-05-27T07:37:36.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.089+0000 7f2144c00700 1 -- 172.21.15.63:0/2111790985 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2154004d50 con 0x55bc77668820 2024-05-27T07:37:36.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.089+0000 7f2144c00700 1 -- 172.21.15.63:0/2111790985 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2154012070 con 0x55bc77668820 2024-05-27T07:37:36.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.089+0000 7f2170543b80 10 client.16176 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:36.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.089+0000 7f2170543b80 10 client.16176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:37:36.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.089+0000 7f2170543b80 20 client.16176 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:36.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.089+0000 7f2170543b80 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bc77682870 0x55bc77684c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.089+0000 7f2170543b80 4 client.16176 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:36.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.089+0000 7f2170543b80 1 -- 172.21.15.63:0/2111790985 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc77598d90 con 0x55bc77682870 2024-05-27T07:37:36.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.090+0000 7f215aa00700 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bc77682870 0x55bc77684c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.091+0000 7f215aa00700 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bc77682870 0x55bc77684c90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e16b3fcd0aadea6 server_cookie=f4ca93e7ec42464b in_seq=0 out_seq=0 2024-05-27T07:37:36.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.091+0000 7f2144c00700 10 client.16176 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.093+0000 7f2144c00700 1 -- 172.21.15.63:0/2111790985 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bc77598d90 con 0x55bc77682870 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.093+0000 7f2144c00700 10 client.16176 handle_command_reply: tid=0 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.093+0000 7f2170543b80 1 client.16176 shutdown 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.093+0000 7f2170543b80 1 -- 172.21.15.63:0/2111790985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668820 msgr2=0x55bc77673080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.093+0000 7f2170543b80 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668820 0x55bc77673080 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55bc77664690 tx=0x7f2154003cb0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.094+0000 7f2170543b80 1 -- 172.21.15.63:0/2111790985 shutdown_connections 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.094+0000 7f2170543b80 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bc77682870 0x55bc77684c90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.094+0000 7f2170543b80 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668820 0x55bc77673080 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.094+0000 7f2170543b80 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc774e9e40 0x55bc77672600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.094+0000 7f2170543b80 1 --2- 172.21.15.63:0/2111790985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668410 0x55bc77672b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.094+0000 7f2144c00700 0 client.16176 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:37:36.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.094+0000 7f2170543b80 1 -- 172.21.15.63:0/2111790985 >> 172.21.15.63:0/2111790985 conn(0x55bc77549fc0 msgr2=0x55bc77662ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:36.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.094+0000 7f2170543b80 1 -- 172.21.15.63:0/2111790985 shutdown_connections 2024-05-27T07:37:36.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.094+0000 7f2170543b80 1 -- 172.21.15.63:0/2111790985 wait complete. 2024-05-27T07:37:36.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.094+0000 7f2170543b80 20 client.16176 trim_cache size 0 max 16384 2024-05-27T07:37:36.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.115+0000 7f2170543b80 1 Processor -- start 2024-05-27T07:37:36.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.115+0000 7f2170543b80 1 -- start start 2024-05-27T07:37:36.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f2170543b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77674aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f2170543b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668410 0x55bc77663e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f2170543b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 0x55bc776717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f2170543b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b4f70 con 0x55bc774e9e40 2024-05-27T07:37:36.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f2170543b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b27a0 con 0x55bc77668820 2024-05-27T07:37:36.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f2170543b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b05f0 con 0x55bc77668410 2024-05-27T07:37:36.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f215aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77674aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f215aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77674aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49614/0 (socket says 172.21.15.63:49614) 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f215aa00700 1 -- 172.21.15.63:0/205857036 learned_addr learned my addr 172.21.15.63:0/205857036 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f215a000700 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668410 0x55bc77663e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.116+0000 7f215b400700 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 0x55bc776717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.117+0000 7f215aa00700 1 -- 172.21.15.63:0/205857036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668410 msgr2=0x55bc77663e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.117+0000 7f215aa00700 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668410 0x55bc77663e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.117+0000 7f215aa00700 1 -- 172.21.15.63:0/205857036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 msgr2=0x55bc776717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.117+0000 7f215aa00700 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 0x55bc776717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.117+0000 7f215aa00700 1 -- 172.21.15.63:0/205857036 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc77665350 con 0x55bc774e9e40 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.117+0000 7f215a000700 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668410 0x55bc77663e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.117+0000 7f215b400700 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 0x55bc776717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.117+0000 7f215aa00700 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77674aa0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f214c009990 tx=0x7f214c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8a10e524906ff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:36.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.117+0000 7f2147400700 1 -- 172.21.15.63:0/205857036 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c004a70 con 0x55bc774e9e40 2024-05-27T07:37:36.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.118+0000 7f2147400700 1 -- 172.21.15.63:0/205857036 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f214c004c10 con 0x55bc774e9e40 2024-05-27T07:37:36.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.118+0000 7f2147400700 1 -- 172.21.15.63:0/205857036 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c032680 con 0x55bc774e9e40 2024-05-27T07:37:36.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.119+0000 7f2170543b80 1 -- 172.21.15.63:0/205857036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 msgr2=0x55bc77674aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:36.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.119+0000 7f2170543b80 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77674aa0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f214c009990 tx=0x7f214c023000 comp rx=0 tx=0).stop 2024-05-27T07:37:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.119+0000 7f2170543b80 1 -- 172.21.15.63:0/205857036 shutdown_connections 2024-05-27T07:37:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.119+0000 7f2170543b80 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 0x55bc776717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.119+0000 7f2170543b80 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc774e9e40 0x55bc77674aa0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.119+0000 7f2170543b80 1 --2- 172.21.15.63:0/205857036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc77668410 0x55bc77663e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.119+0000 7f2170543b80 1 -- 172.21.15.63:0/205857036 >> 172.21.15.63:0/205857036 conn(0x55bc77549fc0 msgr2=0x55bc7767dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.119+0000 7f2170543b80 1 -- 172.21.15.63:0/205857036 shutdown_connections 2024-05-27T07:37:36.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.119+0000 7f2170543b80 1 -- 172.21.15.63:0/205857036 wait complete. 2024-05-27T07:37:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.120+0000 7f2170543b80 1 -- start start 2024-05-27T07:37:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.120+0000 7f2170543b80 1 Processor -- start 2024-05-27T07:37:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f2170543b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc774e9e40 0x55bc776e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f2170543b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668410 0x55bc776e59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f2170543b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 0x55bc77685250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f2170543b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b0c60 con 0x55bc77668410 2024-05-27T07:37:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f2170543b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b20d0 con 0x55bc77668820 2024-05-27T07:37:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f2170543b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1b1330 con 0x55bc774e9e40 2024-05-27T07:37:36.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f2170543b80 10 client.0 _ll_register_callbacks cb 0x55bc77530c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:36.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f215a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668410 0x55bc776e59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f215a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668410 0x55bc776e59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49622/0 (socket says 172.21.15.63:49622) 2024-05-27T07:37:36.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f215a000700 1 -- 172.21.15.63:0/981059296 learned_addr learned my addr 172.21.15.63:0/981059296 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f215aa00700 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc774e9e40 0x55bc776e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.121+0000 7f215b400700 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 0x55bc77685250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.122+0000 7f215a000700 1 -- 172.21.15.63:0/981059296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc774e9e40 msgr2=0x55bc776e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.122+0000 7f215a000700 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc774e9e40 0x55bc776e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.122+0000 7f215a000700 1 -- 172.21.15.63:0/981059296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 msgr2=0x55bc77685250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.122+0000 7f215a000700 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 0x55bc77685250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.122+0000 7f215a000700 1 -- 172.21.15.63:0/981059296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc776858c0 con 0x55bc77668410 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.122+0000 7f215a000700 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668410 0x55bc776e59c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f21480a98f0 tx=0x7f21480aa130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb7bfff685d9f84 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.122+0000 7f2144c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.122+0000 7f2144c00700 1 -- 172.21.15.63:0/981059296 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21480a9360 con 0x55bc77668410 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.122+0000 7f2144c00700 1 -- 172.21.15.63:0/981059296 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f21480a9060 con 0x55bc77668410 2024-05-27T07:37:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.122+0000 7f2170543b80 1 -- 172.21.15.63:0/981059296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214c004c10 con 0x55bc77668410 2024-05-27T07:37:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2144c00700 1 -- 172.21.15.63:0/981059296 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f21480a5380 con 0x55bc77668410 2024-05-27T07:37:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2170543b80 10 client.16182 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2170543b80 1 -- 172.21.15.63:0/981059296 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc776ecaf0 con 0x55bc77668410 2024-05-27T07:37:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2144c00700 1 -- 172.21.15.63:0/981059296 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f214c004c10 con 0x55bc77668410 2024-05-27T07:37:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2144c00700 1 -- 172.21.15.63:0/981059296 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f21480c40d0 con 0x55bc77668410 2024-05-27T07:37:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2170543b80 10 client.16182 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2170543b80 10 client.16182 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:37:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2170543b80 20 client.16182 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2170543b80 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bc776ed220 0x55bc776ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2170543b80 4 client.16182 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:37:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.123+0000 7f2170543b80 1 -- 172.21.15.63:0/981059296 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc77592570 con 0x55bc776ed220 2024-05-27T07:37:36.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.124+0000 7f215aa00700 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bc776ed220 0x55bc776ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:36.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.125+0000 7f2144c00700 10 client.16182 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:37:36.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.125+0000 7f215aa00700 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bc776ed220 0x55bc776ef640 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a20d90f7c91d4175 server_cookie=e9222d63611eca89 in_seq=0 out_seq=0 2024-05-27T07:37:36.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2144c00700 1 -- 172.21.15.63:0/981059296 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc77592570 con 0x55bc776ed220 2024-05-27T07:37:36.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2144c00700 10 client.16182 handle_command_reply: tid=0 2024-05-27T07:37:36.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 client.16182 shutdown 2024-05-27T07:37:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 -- 172.21.15.63:0/981059296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668410 msgr2=0x55bc776e59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668410 0x55bc776e59c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f21480a98f0 tx=0x7f21480aa130 comp rx=0 tx=0).stop 2024-05-27T07:37:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 -- 172.21.15.63:0/981059296 shutdown_connections 2024-05-27T07:37:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc77668820 0x55bc77685250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc774e9e40 0x55bc776e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55bc776ed220 0x55bc776ef640 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 --2- 172.21.15.63:0/981059296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc77668410 0x55bc776e59c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 -- 172.21.15.63:0/981059296 >> 172.21.15.63:0/981059296 conn(0x55bc77549fc0 msgr2=0x55bc77671550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:36.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2144c00700 0 client.16182 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:37:36.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 -- 172.21.15.63:0/981059296 shutdown_connections 2024-05-27T07:37:36.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.126+0000 7f2170543b80 1 -- 172.21.15.63:0/981059296 wait complete. 2024-05-27T07:37:36.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.127+0000 7f2170543b80 20 client.16182 trim_cache size 0 max 16384 2024-05-27T07:37:36.127 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:37:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.127+0000 7f215be00700 1 -- 172.21.15.63:0/2257729296 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21400b2a70 msgr2=0x7f21400b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.127+0000 7f215be00700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21400b2a70 0x7f21400b4f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f214c006010 tx=0x7f214c01a040 comp rx=0 tx=0).stop 2024-05-27T07:37:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.127+0000 7f215be00700 1 -- 172.21.15.63:0/2257729296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c105bd0 msgr2=0x7f215c19d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.127+0000 7f215be00700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c105bd0 0x7f215c19d310 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f214800e9a0 tx=0x7f214800ed60 comp rx=0 tx=0).stop 2024-05-27T07:37:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.127+0000 7f215be00700 1 -- 172.21.15.63:0/2257729296 shutdown_connections 2024-05-27T07:37:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.127+0000 7f215be00700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21400b2a70 0x7f21400b4f80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.128+0000 7f215be00700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f215c10e6f0 0x7f215c1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.128+0000 7f215be00700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f215c105280 0x7f215c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.128+0000 7f215be00700 1 --2- 172.21.15.63:0/2257729296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f215c105bd0 0x7f215c19d310 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.128+0000 7f215be00700 1 -- 172.21.15.63:0/2257729296 >> 172.21.15.63:0/2257729296 conn(0x7f215c078ab0 msgr2=0x7f215c102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.128+0000 7f215be00700 1 -- 172.21.15.63:0/2257729296 shutdown_connections 2024-05-27T07:37:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:36.128+0000 7f215be00700 1 -- 172.21.15.63:0/2257729296 wait complete. 2024-05-27T07:37:37.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:37 smithi086 ceph-mon[34687]: pgmap v125: 129 pgs: 129 active+clean; 377 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:37:37.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:37 smithi149 ceph-mon[35408]: pgmap v125: 129 pgs: 129 active+clean; 377 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:37:37.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:37 smithi063 ceph-mon[31613]: pgmap v125: 129 pgs: 129 active+clean; 377 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:37:38.334 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:37:38.840 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:37:39.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:39 smithi086 ceph-mon[34687]: pgmap v126: 129 pgs: 129 active+clean; 377 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 11 op/s 2024-05-27T07:37:39.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:39 smithi149 ceph-mon[35408]: pgmap v126: 129 pgs: 129 active+clean; 377 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 11 op/s 2024-05-27T07:37:39.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:39 smithi063 ceph-mon[31613]: pgmap v126: 129 pgs: 129 active+clean; 377 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 11 op/s 2024-05-27T07:37:41.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.416+0000 7fbedf600700 1 -- 172.21.15.63:0/2653802202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbed8103260 msgr2=0x7fbed8103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:41.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.416+0000 7fbedf600700 1 --2- 172.21.15.63:0/2653802202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbed8103260 0x7fbed8103670 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009990 tx=0x7fbec8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.417+0000 7fbedf600700 1 -- 172.21.15.63:0/2653802202 shutdown_connections 2024-05-27T07:37:41.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.417+0000 7fbedf600700 1 --2- 172.21.15.63:0/2653802202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed81086b0 0x7fbed8108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.417+0000 7fbedf600700 1 --2- 172.21.15.63:0/2653802202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbed8103260 0x7fbed8103670 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.417+0000 7fbedf600700 1 --2- 172.21.15.63:0/2653802202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbed8103bb0 0x7fbed81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.417+0000 7fbedf600700 1 -- 172.21.15.63:0/2653802202 >> 172.21.15.63:0/2653802202 conn(0x7fbed80fef20 msgr2=0x7fbed8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:41.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.417+0000 7fbedf600700 1 -- 172.21.15.63:0/2653802202 shutdown_connections 2024-05-27T07:37:41.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.417+0000 7fbedf600700 1 -- 172.21.15.63:0/2653802202 wait complete. 2024-05-27T07:37:41.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.418+0000 7fbedf600700 1 Processor -- start 2024-05-27T07:37:41.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.419+0000 7fbedf600700 1 -- start start 2024-05-27T07:37:41.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.419+0000 7fbedf600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbed8103260 0x7fbed81a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.419+0000 7fbedf600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbed8103bb0 0x7fbed81a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.419+0000 7fbedf600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed81086b0 0x7fbed81a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.419+0000 7fbedf600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbed811a960 con 0x7fbed8103bb0 2024-05-27T07:37:41.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.419+0000 7fbedf600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbed811a7e0 con 0x7fbed81086b0 2024-05-27T07:37:41.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.419+0000 7fbedf600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbed811aae0 con 0x7fbed8103260 2024-05-27T07:37:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.420+0000 7fbecfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbed8103bb0 0x7fbed81a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.420+0000 7fbecfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbed8103bb0 0x7fbed81a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38236/0 (socket says 172.21.15.63:38236) 2024-05-27T07:37:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.420+0000 7fbecfe00700 1 -- 172.21.15.63:0/800770911 learned_addr learned my addr 172.21.15.63:0/800770911 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:41.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.420+0000 7fbedcc00700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbed8103260 0x7fbed81a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.420+0000 7fbedd600700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed81086b0 0x7fbed81a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.421+0000 7fbecfe00700 1 -- 172.21.15.63:0/800770911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbed8103260 msgr2=0x7fbed81a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:41.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.421+0000 7fbecfe00700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbed8103260 0x7fbed81a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.421+0000 7fbecfe00700 1 -- 172.21.15.63:0/800770911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed81086b0 msgr2=0x7fbed81a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:41.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.421+0000 7fbecfe00700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed81086b0 0x7fbed81a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.421+0000 7fbecfe00700 1 -- 172.21.15.63:0/800770911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed81a6280 con 0x7fbed8103bb0 2024-05-27T07:37:41.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.421+0000 7fbedcc00700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbed8103260 0x7fbed81a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:41.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.421+0000 7fbecfe00700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbed8103bb0 0x7fbed81a1950 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbec400b530 tx=0x7fbec400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:41.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.422+0000 7fbed6000700 1 -- 172.21.15.63:0/800770911 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4015020 con 0x7fbed8103bb0 2024-05-27T07:37:41.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.422+0000 7fbed6000700 1 -- 172.21.15.63:0/800770911 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbec4011cc0 con 0x7fbed8103bb0 2024-05-27T07:37:41.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.422+0000 7fbedf600700 1 -- 172.21.15.63:0/800770911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed81a6570 con 0x7fbed8103bb0 2024-05-27T07:37:41.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.422+0000 7fbed6000700 1 -- 172.21.15.63:0/800770911 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4010dc0 con 0x7fbed8103bb0 2024-05-27T07:37:41.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.422+0000 7fbedf600700 1 -- 172.21.15.63:0/800770911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed81ade90 con 0x7fbed8103bb0 2024-05-27T07:37:41.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.428+0000 7fbeee240b80 1 Processor -- start 2024-05-27T07:37:41.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.428+0000 7fbed6000700 1 -- 172.21.15.63:0/800770911 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbec4018020 con 0x7fbed8103bb0 2024-05-27T07:37:41.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.429+0000 7fbeee240b80 1 -- start start 2024-05-27T07:37:41.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.429+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea2567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.429+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea256d30 0x55e3ea250870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.429+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 0x55e3ea2511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.429+0000 7fbeee240b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e3ea17ee40 con 0x55e3ea0d1e40 2024-05-27T07:37:41.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.429+0000 7fbeee240b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3ea16e9e0 con 0x55e3ea250db0 2024-05-27T07:37:41.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.429+0000 7fbeee240b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3ea172a20 con 0x55e3ea256d30 2024-05-27T07:37:41.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.430+0000 7fbedd600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea2567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.430+0000 7fbedd600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea2567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38238/0 (socket says 172.21.15.63:38238) 2024-05-27T07:37:41.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.430+0000 7fbedd600700 1 -- 172.21.15.63:0/1769297316 learned_addr learned my addr 172.21.15.63:0/1769297316 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:41.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.430+0000 7fbecfe00700 1 --2- 172.21.15.63:0/1769297316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 0x55e3ea2511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.430+0000 7fbedcc00700 1 --2- 172.21.15.63:0/1769297316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea256d30 0x55e3ea250870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.431+0000 7fbecfe00700 1 -- 172.21.15.63:0/1769297316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea256d30 msgr2=0x55e3ea250870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:41.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.431+0000 7fbecfe00700 1 --2- 172.21.15.63:0/1769297316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea256d30 0x55e3ea250870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.431+0000 7fbecfe00700 1 -- 172.21.15.63:0/1769297316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 msgr2=0x55e3ea2567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:41.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.431+0000 7fbecfe00700 1 --2- 172.21.15.63:0/1769297316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea2567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.431+0000 7fbecfe00700 1 -- 172.21.15.63:0/1769297316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ea137420 con 0x55e3ea250db0 2024-05-27T07:37:41.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.431+0000 7fbed6000700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbebc0b2a70 0x7fbebc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.431+0000 7fbecfe00700 1 --2- 172.21.15.63:0/1769297316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 0x55e3ea2511c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbec40df550 tx=0x7fbec40ea5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff40101410c2fb63 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:41.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.432+0000 7fbed6000700 1 -- 172.21.15.63:0/800770911 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fbec40e2060 con 0x7fbed8103bb0 2024-05-27T07:37:41.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.432+0000 7fbece000700 1 -- 172.21.15.63:0/1769297316 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec40a4fa0 con 0x55e3ea250db0 2024-05-27T07:37:41.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.432+0000 7fbece000700 1 -- 172.21.15.63:0/1769297316 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbec40a8bf0 con 0x55e3ea250db0 2024-05-27T07:37:41.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.432+0000 7fbece000700 1 -- 172.21.15.63:0/1769297316 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec40a8850 con 0x55e3ea250db0 2024-05-27T07:37:41.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.432+0000 7fbedd600700 1 --2- 172.21.15.63:0/1769297316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea2567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:37:41.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.433+0000 7fbedd600700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbebc0b2a70 0x7fbebc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.434+0000 7fbeee240b80 1 -- 172.21.15.63:0/1769297316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 msgr2=0x55e3ea2511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:41.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.434+0000 7fbeee240b80 1 --2- 172.21.15.63:0/1769297316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 0x55e3ea2511c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbec40df550 tx=0x7fbec40ea5d0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.434+0000 7fbeee240b80 1 -- 172.21.15.63:0/1769297316 shutdown_connections 2024-05-27T07:37:41.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.434+0000 7fbeee240b80 1 --2- 172.21.15.63:0/1769297316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 0x55e3ea2511c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.434+0000 7fbeee240b80 1 --2- 172.21.15.63:0/1769297316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea2567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.434+0000 7fbeee240b80 1 --2- 172.21.15.63:0/1769297316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea256d30 0x55e3ea250870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.434+0000 7fbeee240b80 1 -- 172.21.15.63:0/1769297316 >> 172.21.15.63:0/1769297316 conn(0x55e3ea131fc0 msgr2=0x55e3ea24b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:41.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.434+0000 7fbeee240b80 1 -- 172.21.15.63:0/1769297316 shutdown_connections 2024-05-27T07:37:41.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.434+0000 7fbedd600700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbebc0b2a70 0x7fbebc0b4f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55e3ea251b30 tx=0x7fbed0009100 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:41.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.435+0000 7fbeee240b80 1 -- 172.21.15.63:0/1769297316 wait complete. 2024-05-27T07:37:41.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.435+0000 7fbeee240b80 1 -- start start 2024-05-27T07:37:41.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.436+0000 7fbeee240b80 1 Processor -- start 2024-05-27T07:37:41.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.436+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea25a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.436+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 0x55e3ea25a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.436+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea256d30 0x55e3ea25af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.436+0000 7fbeee240b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e3ea074fa0 con 0x55e3ea0d1e40 2024-05-27T07:37:41.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.436+0000 7fbeee240b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3ea10f760 con 0x55e3ea250db0 2024-05-27T07:37:41.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.436+0000 7fbeee240b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3ea25d680 con 0x55e3ea256d30 2024-05-27T07:37:41.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.437+0000 7fbeee240b80 10 client.0 _ll_register_callbacks cb 0x55e3ea118c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:41.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.437+0000 7fbedcc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea25a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.437+0000 7fbedcc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea25a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38244/0 (socket says 172.21.15.63:38244) 2024-05-27T07:37:41.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.437+0000 7fbedcc00700 1 -- 172.21.15.63:0/2155594427 learned_addr learned my addr 172.21.15.63:0/2155594427 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:41.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.437+0000 7fbedd600700 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea256d30 0x55e3ea25af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.438+0000 7fbecfe00700 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 0x55e3ea25a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.438+0000 7fbedcc00700 1 -- 172.21.15.63:0/2155594427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea256d30 msgr2=0x55e3ea25af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:41.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.438+0000 7fbedcc00700 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea256d30 0x55e3ea25af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.438+0000 7fbedcc00700 1 -- 172.21.15.63:0/2155594427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 msgr2=0x55e3ea25a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:41.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.438+0000 7fbedcc00700 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 0x55e3ea25a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.438+0000 7fbedcc00700 1 -- 172.21.15.63:0/2155594427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ea0be8d0 con 0x55e3ea0d1e40 2024-05-27T07:37:41.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.438+0000 7fbecfe00700 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 0x55e3ea25a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:37:41.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.439+0000 7fbedcc00700 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea25a4b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbec8015000 tx=0x7fbec8003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfb58c949193457 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:41.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.439+0000 7fbec2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:41.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.440+0000 7fbec2a00700 1 -- 172.21.15.63:0/2155594427 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8027060 con 0x55e3ea0d1e40 2024-05-27T07:37:41.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.440+0000 7fbec2a00700 1 -- 172.21.15.63:0/2155594427 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fbec8005450 con 0x55e3ea0d1e40 2024-05-27T07:37:41.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.440+0000 7fbeee240b80 1 -- 172.21.15.63:0/2155594427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec40a8bf0 con 0x55e3ea0d1e40 2024-05-27T07:37:41.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.440+0000 7fbec2a00700 1 -- 172.21.15.63:0/2155594427 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbec802e050 con 0x55e3ea0d1e40 2024-05-27T07:37:41.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.440+0000 7fbeee240b80 10 client.16194 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:41.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.440+0000 7fbeee240b80 1 -- 172.21.15.63:0/2155594427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e3ea13f060 con 0x55e3ea0d1e40 2024-05-27T07:37:41.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.441+0000 7fbec2a00700 1 -- 172.21.15.63:0/2155594427 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbec40a8bf0 con 0x55e3ea0d1e40 2024-05-27T07:37:41.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.442+0000 7fbec2a00700 1 -- 172.21.15.63:0/2155594427 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbec8021070 con 0x55e3ea0d1e40 2024-05-27T07:37:41.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.442+0000 7fbeee240b80 10 client.16194 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:41.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.442+0000 7fbeee240b80 10 client.16194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:37:41.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.442+0000 7fbeee240b80 20 client.16194 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:41.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.442+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3ea26a520 0x55e3ea26c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.442+0000 7fbeee240b80 4 client.16194 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:41.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.442+0000 7fbeee240b80 1 -- 172.21.15.63:0/2155594427 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3ea180d90 con 0x55e3ea26a520 2024-05-27T07:37:41.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.443+0000 7fbecfe00700 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3ea26a520 0x55e3ea26c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.445+0000 7fbecfe00700 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3ea26a520 0x55e3ea26c940 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e84267179df693ab server_cookie=65c5e391aea8ed32 in_seq=0 out_seq=0 2024-05-27T07:37:41.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.445+0000 7fbec2a00700 10 client.16194 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:37:41.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.448+0000 7fbec2a00700 1 -- 172.21.15.63:0/2155594427 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e3ea180d90 con 0x55e3ea26a520 2024-05-27T07:37:41.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.448+0000 7fbec2a00700 10 client.16194 handle_command_reply: tid=0 2024-05-27T07:37:41.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.448+0000 7fbeee240b80 1 client.16194 shutdown 2024-05-27T07:37:41.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 1 -- 172.21.15.63:0/2155594427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 msgr2=0x55e3ea25a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:41.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea25a4b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbec8015000 tx=0x7fbec8003f50 comp rx=0 tx=0).stop 2024-05-27T07:37:41.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 1 -- 172.21.15.63:0/2155594427 shutdown_connections 2024-05-27T07:37:41.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea256d30 0x55e3ea25af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3ea26a520 0x55e3ea26c940 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea0d1e40 0x55e3ea25a4b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2155594427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea250db0 0x55e3ea25a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 1 -- 172.21.15.63:0/2155594427 >> 172.21.15.63:0/2155594427 conn(0x55e3ea131fc0 msgr2=0x55e3ea24b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:41.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbec2a00700 0 client.16194 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:37:41.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 1 -- 172.21.15.63:0/2155594427 shutdown_connections 2024-05-27T07:37:41.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 1 -- 172.21.15.63:0/2155594427 wait complete. 2024-05-27T07:37:41.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.449+0000 7fbeee240b80 20 client.16194 trim_cache size 0 max 16384 2024-05-27T07:37:41.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.472+0000 7fbeee240b80 1 Processor -- start 2024-05-27T07:37:41.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.472+0000 7fbeee240b80 1 -- start start 2024-05-27T07:37:41.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.472+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea0d1e40 0x55e3ea250870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.472+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea250db0 0x55e3ea25c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.472+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea256d30 0x55e3ea2666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.472+0000 7fbeee240b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81b93c0 con 0x55e3ea256d30 2024-05-27T07:37:41.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.472+0000 7fbeee240b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81b6bf0 con 0x55e3ea0d1e40 2024-05-27T07:37:41.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.472+0000 7fbeee240b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81b4a40 con 0x55e3ea250db0 2024-05-27T07:37:41.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.473+0000 7fbedd600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea256d30 0x55e3ea2666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.473+0000 7fbedd600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea256d30 0x55e3ea2666e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38250/0 (socket says 172.21.15.63:38250) 2024-05-27T07:37:41.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.473+0000 7fbedd600700 1 -- 172.21.15.63:0/2675583757 learned_addr learned my addr 172.21.15.63:0/2675583757 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:41.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.473+0000 7fbecfe00700 1 --2- 172.21.15.63:0/2675583757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea250db0 0x55e3ea25c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.473+0000 7fbedcc00700 1 --2- 172.21.15.63:0/2675583757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea0d1e40 0x55e3ea250870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.473+0000 7fbedd600700 1 -- 172.21.15.63:0/2675583757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea250db0 msgr2=0x55e3ea25c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.473+0000 7fbedd600700 1 --2- 172.21.15.63:0/2675583757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea250db0 0x55e3ea25c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.473+0000 7fbedd600700 1 -- 172.21.15.63:0/2675583757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea0d1e40 msgr2=0x55e3ea250870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.473+0000 7fbedd600700 1 --2- 172.21.15.63:0/2675583757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea0d1e40 0x55e3ea250870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.473+0000 7fbedd600700 1 -- 172.21.15.63:0/2675583757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ea25b2a0 con 0x55e3ea256d30 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.474+0000 7fbedd600700 1 --2- 172.21.15.63:0/2675583757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea256d30 0x55e3ea2666e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbed0004660 tx=0x7fbed0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80a7c30f097b2b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.474+0000 7fbecd600700 1 -- 172.21.15.63:0/2675583757 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0007b10 con 0x55e3ea256d30 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.474+0000 7fbecd600700 1 -- 172.21.15.63:0/2675583757 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbed001c030 con 0x55e3ea256d30 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.474+0000 7fbecd600700 1 -- 172.21.15.63:0/2675583757 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed002c400 con 0x55e3ea256d30 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.475+0000 7fbeee240b80 1 -- 172.21.15.63:0/2675583757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea256d30 msgr2=0x55e3ea2666e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.475+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2675583757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea256d30 0x55e3ea2666e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbed0004660 tx=0x7fbed0016000 comp rx=0 tx=0).stop 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.476+0000 7fbeee240b80 1 -- 172.21.15.63:0/2675583757 shutdown_connections 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.476+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2675583757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea256d30 0x55e3ea2666e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.476+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2675583757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea0d1e40 0x55e3ea250870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.476+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2675583757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea250db0 0x55e3ea25c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.476+0000 7fbeee240b80 1 -- 172.21.15.63:0/2675583757 >> 172.21.15.63:0/2675583757 conn(0x55e3ea131fc0 msgr2=0x55e3ea254210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:41.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.476+0000 7fbeee240b80 1 -- 172.21.15.63:0/2675583757 shutdown_connections 2024-05-27T07:37:41.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.476+0000 7fbeee240b80 1 -- 172.21.15.63:0/2675583757 wait complete. 2024-05-27T07:37:41.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.476+0000 7fbeee240b80 1 -- start start 2024-05-27T07:37:41.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.477+0000 7fbeee240b80 1 Processor -- start 2024-05-27T07:37:41.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.477+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea0d1e40 0x55e3ea2628d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.477+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea250db0 0x55e3ea262e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.477+0000 7fbeee240b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea256d30 0x55e3ea2710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.478+0000 7fbeee240b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81b50b0 con 0x55e3ea250db0 2024-05-27T07:37:41.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.478+0000 7fbeee240b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81b6520 con 0x55e3ea256d30 2024-05-27T07:37:41.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.478+0000 7fbeee240b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbed81b5780 con 0x55e3ea0d1e40 2024-05-27T07:37:41.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.478+0000 7fbeee240b80 10 client.0 _ll_register_callbacks cb 0x55e3ea118c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:41.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.478+0000 7fbecfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea250db0 0x55e3ea262e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.478+0000 7fbecfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea250db0 0x55e3ea262e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38260/0 (socket says 172.21.15.63:38260) 2024-05-27T07:37:41.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.478+0000 7fbecfe00700 1 -- 172.21.15.63:0/2698467489 learned_addr learned my addr 172.21.15.63:0/2698467489 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:41.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.478+0000 7fbedcc00700 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea0d1e40 0x55e3ea2628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.478+0000 7fbedd600700 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea256d30 0x55e3ea2710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.479+0000 7fbecfe00700 1 -- 172.21.15.63:0/2698467489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea0d1e40 msgr2=0x55e3ea2628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:41.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.479+0000 7fbecfe00700 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea0d1e40 0x55e3ea2628d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.479+0000 7fbecfe00700 1 -- 172.21.15.63:0/2698467489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea256d30 msgr2=0x55e3ea2710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:41.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.479+0000 7fbecfe00700 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea256d30 0x55e3ea2710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.479+0000 7fbecfe00700 1 -- 172.21.15.63:0/2698467489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ea272b50 con 0x55e3ea250db0 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.479+0000 7fbecfe00700 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea250db0 0x55e3ea262e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbec40df600 tx=0x7fbec4011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69c30abb610de39 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.479+0000 7fbec2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.479+0000 7fbec2a00700 1 -- 172.21.15.63:0/2698467489 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec40a8a30 con 0x55e3ea250db0 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.479+0000 7fbec2a00700 1 -- 172.21.15.63:0/2698467489 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fbec4010530 con 0x55e3ea250db0 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.480+0000 7fbeee240b80 1 -- 172.21.15.63:0/2698467489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed001c030 con 0x55e3ea250db0 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.480+0000 7fbec2a00700 1 -- 172.21.15.63:0/2698467489 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbec40a8eb0 con 0x55e3ea250db0 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.480+0000 7fbeee240b80 10 client.16200 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.480+0000 7fbeee240b80 1 -- 172.21.15.63:0/2698467489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e3ea2d4420 con 0x55e3ea250db0 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.480+0000 7fbec2a00700 1 -- 172.21.15.63:0/2698467489 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbed001c030 con 0x55e3ea250db0 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.480+0000 7fbec2a00700 1 -- 172.21.15.63:0/2698467489 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbec40eaca0 con 0x55e3ea250db0 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.481+0000 7fbeee240b80 10 client.16200 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.481+0000 7fbeee240b80 10 client.16200 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.481+0000 7fbeee240b80 20 client.16200 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:41.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.481+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3ea2d4790 0x55e3ea2d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:41.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.481+0000 7fbeee240b80 4 client.16200 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:37:41.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.481+0000 7fbeee240b80 1 -- 172.21.15.63:0/2698467489 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3ea17a570 con 0x55e3ea2d4790 2024-05-27T07:37:41.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.482+0000 7fbedcc00700 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3ea2d4790 0x55e3ea2d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:41.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.483+0000 7fbedcc00700 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3ea2d4790 0x55e3ea2d6bb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d961635d2c0300ff server_cookie=f557cd75d98893e8 in_seq=0 out_seq=0 2024-05-27T07:37:41.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.483+0000 7fbec2a00700 10 client.16200 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:37:41.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.483+0000 7fbec2a00700 1 -- 172.21.15.63:0/2698467489 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3ea17a570 con 0x55e3ea2d4790 2024-05-27T07:37:41.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.483+0000 7fbec2a00700 10 client.16200 handle_command_reply: tid=0 2024-05-27T07:37:41.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.483+0000 7fbeee240b80 1 client.16200 shutdown 2024-05-27T07:37:41.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbeee240b80 1 -- 172.21.15.63:0/2698467489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea250db0 msgr2=0x55e3ea262e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:41.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea250db0 0x55e3ea262e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbec40df600 tx=0x7fbec4011e70 comp rx=0 tx=0).stop 2024-05-27T07:37:41.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbeee240b80 1 -- 172.21.15.63:0/2698467489 shutdown_connections 2024-05-27T07:37:41.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3ea256d30 0x55e3ea2710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3ea0d1e40 0x55e3ea2628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e3ea2d4790 0x55e3ea2d6bb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbeee240b80 1 --2- 172.21.15.63:0/2698467489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3ea250db0 0x55e3ea262e10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbec2a00700 0 client.16200 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:37:41.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbeee240b80 1 -- 172.21.15.63:0/2698467489 >> 172.21.15.63:0/2698467489 conn(0x55e3ea131fc0 msgr2=0x55e3ea255da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:41.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbeee240b80 1 -- 172.21.15.63:0/2698467489 shutdown_connections 2024-05-27T07:37:41.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.484+0000 7fbeee240b80 1 -- 172.21.15.63:0/2698467489 wait complete. 2024-05-27T07:37:41.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.485+0000 7fbeee240b80 20 client.16200 trim_cache size 0 max 16384 2024-05-27T07:37:41.483 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:37:41.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.485+0000 7fbedf600700 1 -- 172.21.15.63:0/800770911 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbebc0b2a70 msgr2=0x7fbebc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:41.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.485+0000 7fbedf600700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbebc0b2a70 0x7fbebc0b4f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55e3ea251b30 tx=0x7fbed0009100 comp rx=0 tx=0).stop 2024-05-27T07:37:41.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.485+0000 7fbedf600700 1 -- 172.21.15.63:0/800770911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbed8103bb0 msgr2=0x7fbed81a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:41.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.485+0000 7fbedf600700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbed8103bb0 0x7fbed81a1950 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbec400b530 tx=0x7fbec400b840 comp rx=0 tx=0).stop 2024-05-27T07:37:41.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.486+0000 7fbedf600700 1 -- 172.21.15.63:0/800770911 shutdown_connections 2024-05-27T07:37:41.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.486+0000 7fbedf600700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbebc0b2a70 0x7fbebc0b4f80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.486+0000 7fbedf600700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbed81086b0 0x7fbed81a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.486+0000 7fbedf600700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbed8103260 0x7fbed81a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.486+0000 7fbedf600700 1 --2- 172.21.15.63:0/800770911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbed8103bb0 0x7fbed81a1950 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:41.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.486+0000 7fbedf600700 1 -- 172.21.15.63:0/800770911 >> 172.21.15.63:0/800770911 conn(0x7fbed80fef20 msgr2=0x7fbed81060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:41.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.486+0000 7fbedf600700 1 -- 172.21.15.63:0/800770911 shutdown_connections 2024-05-27T07:37:41.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:41.486+0000 7fbedf600700 1 -- 172.21.15.63:0/800770911 wait complete. 2024-05-27T07:37:41.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:41 smithi086 ceph-mon[34687]: pgmap v127: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 14 op/s 2024-05-27T07:37:41.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:41 smithi149 ceph-mon[35408]: pgmap v127: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 14 op/s 2024-05-27T07:37:41.711 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:41 smithi063 ceph-mon[31613]: pgmap v127: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 14 op/s 2024-05-27T07:37:42.506 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:37:43.018 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:37:43.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:43 smithi086 ceph-mon[34687]: pgmap v128: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:37:43.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:43 smithi149 ceph-mon[35408]: pgmap v128: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:37:43.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:43 smithi063 ceph-mon[31613]: pgmap v128: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:37:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.306+0000 7fb908c00640 1 -- 172.21.15.63:0/3469964411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90410e1e0 msgr2=0x7fb9041106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.306+0000 7fb908c00640 1 --2- 172.21.15.63:0/3469964411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90410e1e0 0x7fb9041106d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009ca0 tx=0x7fb8fc02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.306+0000 7fb908c00640 1 -- 172.21.15.63:0/3469964411 shutdown_connections 2024-05-27T07:37:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.306+0000 7fb908c00640 1 --2- 172.21.15.63:0/3469964411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90410e1e0 0x7fb9041106d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.306+0000 7fb908c00640 1 --2- 172.21.15.63:0/3469964411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb90410b7f0 0x7fb90410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.306+0000 7fb908c00640 1 --2- 172.21.15.63:0/3469964411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb904104d90 0x7fb9041051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.306+0000 7fb908c00640 1 -- 172.21.15.63:0/3469964411 >> 172.21.15.63:0/3469964411 conn(0x7fb9040782d0 msgr2=0x7fb904078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.306+0000 7fb908c00640 1 -- 172.21.15.63:0/3469964411 shutdown_connections 2024-05-27T07:37:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.306+0000 7fb908c00640 1 -- 172.21.15.63:0/3469964411 wait complete. 2024-05-27T07:37:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.307+0000 7fb908c00640 1 Processor -- start 2024-05-27T07:37:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.308+0000 7fb908c00640 1 -- start start 2024-05-27T07:37:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.308+0000 7fb908c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb904104d90 0x7fb9041a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.308+0000 7fb908c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90410b7f0 0x7fb9041a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.308+0000 7fb908c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90410e1e0 0x7fb90419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.308+0000 7fb908c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb904113d40 con 0x7fb90410b7f0 2024-05-27T07:37:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.308+0000 7fb908c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb904113bc0 con 0x7fb90410e1e0 2024-05-27T07:37:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.308+0000 7fb908c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb904113ec0 con 0x7fb904104d90 2024-05-27T07:37:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.309+0000 7fb901400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90410b7f0 0x7fb9041a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.309+0000 7fb901400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90410b7f0 0x7fb9041a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38278/0 (socket says 172.21.15.63:38278) 2024-05-27T07:37:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.309+0000 7fb901400640 1 -- 172.21.15.63:0/3102843016 learned_addr learned my addr 172.21.15.63:0/3102843016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.309+0000 7fb901e00640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb904104d90 0x7fb9041a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.309+0000 7fb902800640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90410e1e0 0x7fb90419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.309+0000 7fb901400640 1 -- 172.21.15.63:0/3102843016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb904104d90 msgr2=0x7fb9041a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.309+0000 7fb901400640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb904104d90 0x7fb9041a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.309+0000 7fb901400640 1 -- 172.21.15.63:0/3102843016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90410e1e0 msgr2=0x7fb90419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.309+0000 7fb901400640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90410e1e0 0x7fb90419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.309+0000 7fb901400640 1 -- 172.21.15.63:0/3102843016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90419ce80 con 0x7fb90410b7f0 2024-05-27T07:37:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.310+0000 7fb901400640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90410b7f0 0x7fb9041a2ab0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000d770 tx=0x7fb8f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.310+0000 7fb8faa00640 1 -- 172.21.15.63:0/3102843016 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0017070 con 0x7fb90410b7f0 2024-05-27T07:37:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.310+0000 7fb8faa00640 1 -- 172.21.15.63:0/3102843016 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f00054b0 con 0x7fb90410b7f0 2024-05-27T07:37:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.310+0000 7fb908c00640 1 -- 172.21.15.63:0/3102843016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90419d170 con 0x7fb90410b7f0 2024-05-27T07:37:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.310+0000 7fb8faa00640 1 -- 172.21.15.63:0/3102843016 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0004c60 con 0x7fb90410b7f0 2024-05-27T07:37:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.310+0000 7fb908c00640 1 -- 172.21.15.63:0/3102843016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9041a93c0 con 0x7fb90410b7f0 2024-05-27T07:37:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.314+0000 7fb90fd86740 1 Processor -- start 2024-05-27T07:37:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.315+0000 7fb90fd86740 1 -- start start 2024-05-27T07:37:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.315+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 0x55d180622860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.315+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802572f0 0x55d180622da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.315+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d18025f4d0 0x55d1806232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.315+0000 7fb90fd86740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d180593b90 con 0x55d18025f4d0 2024-05-27T07:37:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.315+0000 7fb90fd86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d180583730 con 0x55d1802572f0 2024-05-27T07:37:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.315+0000 7fb90fd86740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d180587770 con 0x55d1802801b0 2024-05-27T07:37:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.316+0000 7fb901e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802572f0 0x55d180622da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.316+0000 7fb902800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 0x55d180622860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.316+0000 7fb901e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802572f0 0x55d180622da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:45228/0 (socket says 172.21.15.63:45228) 2024-05-27T07:37:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.316+0000 7fb901e00640 1 -- 172.21.15.63:0/180066861 learned_addr learned my addr 172.21.15.63:0/180066861 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.317+0000 7fb901e00640 1 -- 172.21.15.63:0/180066861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 msgr2=0x55d180622860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.317+0000 7fb8faa00640 1 -- 172.21.15.63:0/3102843016 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb8f0004e00 con 0x7fb90410b7f0 2024-05-27T07:37:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.317+0000 7fb901e00640 1 --2- 172.21.15.63:0/180066861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 0x55d180622860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.317+0000 7fb901e00640 1 -- 172.21.15.63:0/180066861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d18025f4d0 msgr2=0x55d1806232e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:37:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.317+0000 7fb901e00640 1 --2- 172.21.15.63:0/180066861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d18025f4d0 0x55d1806232e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.317+0000 7fb901e00640 1 -- 172.21.15.63:0/180066861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18051c8b0 con 0x55d1802572f0 2024-05-27T07:37:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.317+0000 7fb902800640 1 --2- 172.21.15.63:0/180066861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 0x55d180622860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:37:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.317+0000 7fb901e00640 1 --2- 172.21.15.63:0/180066861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802572f0 0x55d180622da0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400bcb0 tx=0x7fb8f40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9e2efe7bc5f8ade server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.318+0000 7fb8ef000640 1 -- 172.21.15.63:0/180066861 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f401c480 con 0x55d1802572f0 2024-05-27T07:37:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.318+0000 7fb8ef000640 1 -- 172.21.15.63:0/180066861 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f401cab0 con 0x55d1802572f0 2024-05-27T07:37:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.318+0000 7fb8ef000640 1 -- 172.21.15.63:0/180066861 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4011b60 con 0x55d1802572f0 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.319+0000 7fb90fd86740 1 -- 172.21.15.63:0/180066861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802572f0 msgr2=0x55d180622da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.319+0000 7fb90fd86740 1 --2- 172.21.15.63:0/180066861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802572f0 0x55d180622da0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400bcb0 tx=0x7fb8f40107a0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.319+0000 7fb8faa00640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e80b29d0 0x7fb8e80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.319+0000 7fb90fd86740 1 -- 172.21.15.63:0/180066861 shutdown_connections 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.319+0000 7fb90fd86740 1 --2- 172.21.15.63:0/180066861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d18025f4d0 0x55d1806232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.319+0000 7fb90fd86740 1 --2- 172.21.15.63:0/180066861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802572f0 0x55d180622da0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.319+0000 7fb90fd86740 1 --2- 172.21.15.63:0/180066861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 0x55d180622860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.319+0000 7fb90fd86740 1 -- 172.21.15.63:0/180066861 >> 172.21.15.63:0/180066861 conn(0x55d180274300 msgr2=0x55d18061f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.319+0000 7fb8faa00640 1 -- 172.21.15.63:0/3102843016 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fb8f00e2270 con 0x7fb90410b7f0 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.320+0000 7fb902800640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e80b29d0 0x7fb8e80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.320+0000 7fb90fd86740 1 -- 172.21.15.63:0/180066861 shutdown_connections 2024-05-27T07:37:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.320+0000 7fb90fd86740 1 -- 172.21.15.63:0/180066861 wait complete. 2024-05-27T07:37:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.321+0000 7fb902800640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e80b29d0 0x7fb8e80b4ee0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006010 tx=0x7fb8fc004a80 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.321+0000 7fb90fd86740 1 -- start start 2024-05-27T07:37:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.321+0000 7fb90fd86740 1 Processor -- start 2024-05-27T07:37:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d18062cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d18025f4d0 0x55d18062d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 0x55d180638cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb90fd86740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d180269790 con 0x55d1802572f0 2024-05-27T07:37:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb90fd86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d18052bbc0 con 0x55d18025f4d0 2024-05-27T07:37:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb90fd86740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d18062f990 con 0x55d1802801b0 2024-05-27T07:37:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb90fd86740 10 client.0 _ll_register_callbacks cb 0x55d180509400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb901e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d18062cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb901e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d18062cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38288/0 (socket says 172.21.15.63:38288) 2024-05-27T07:37:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb901e00640 1 -- 172.21.15.63:0/404743614 learned_addr learned my addr 172.21.15.63:0/404743614 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb901400640 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d18025f4d0 0x55d18062d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.322+0000 7fb902800640 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 0x55d180638cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.323+0000 7fb901e00640 1 -- 172.21.15.63:0/404743614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 msgr2=0x55d180638cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.323+0000 7fb901e00640 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 0x55d180638cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.323+0000 7fb901e00640 1 -- 172.21.15.63:0/404743614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d18025f4d0 msgr2=0x55d18062d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.323+0000 7fb901e00640 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d18025f4d0 0x55d18062d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.323+0000 7fb901e00640 1 -- 172.21.15.63:0/404743614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d180298210 con 0x55d1802572f0 2024-05-27T07:37:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.323+0000 7fb901e00640 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d18062cd10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4011010 tx=0x7fb8f400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2a8354e3823d08 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.323+0000 7fb8e3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.323+0000 7fb8e3400640 1 -- 172.21.15.63:0/404743614 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f402a060 con 0x55d1802572f0 2024-05-27T07:37:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.323+0000 7fb8e3400640 1 -- 172.21.15.63:0/404743614 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fb8f4004030 con 0x55d1802572f0 2024-05-27T07:37:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.323+0000 7fb90fd86740 1 -- 172.21.15.63:0/404743614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f401cab0 con 0x55d1802572f0 2024-05-27T07:37:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.324+0000 7fb8e3400640 1 -- 172.21.15.63:0/404743614 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb8f4031050 con 0x55d1802572f0 2024-05-27T07:37:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.324+0000 7fb90fd86740 10 client.16209 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.324+0000 7fb90fd86740 1 -- 172.21.15.63:0/404743614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d180364620 con 0x55d1802572f0 2024-05-27T07:37:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.324+0000 7fb8e3400640 1 -- 172.21.15.63:0/404743614 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f401cab0 con 0x55d1802572f0 2024-05-27T07:37:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.324+0000 7fb8e3400640 1 -- 172.21.15.63:0/404743614 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb8f40118f0 con 0x55d1802572f0 2024-05-27T07:37:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.325+0000 7fb90fd86740 10 client.16209 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.325+0000 7fb90fd86740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.325+0000 7fb90fd86740 20 client.16209 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.325+0000 7fb90fd86740 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d18023ed80 0x55d18063ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.325+0000 7fb90fd86740 4 client.16209 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.325+0000 7fb90fd86740 1 -- 172.21.15.63:0/404743614 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d180595ae0 con 0x55d18023ed80 2024-05-27T07:37:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.326+0000 7fb901400640 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d18023ed80 0x55d18063ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.327+0000 7fb901400640 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d18023ed80 0x55d18063ec60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae58a04bbfbd848 server_cookie=1841ca1d2d71df90 in_seq=0 out_seq=0 2024-05-27T07:37:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.327+0000 7fb8e3400640 10 client.16209 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb8e3400640 1 -- 172.21.15.63:0/404743614 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d180595ae0 con 0x55d18023ed80 2024-05-27T07:37:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb8e3400640 10 client.16209 handle_command_reply: tid=0 2024-05-27T07:37:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb90fd86740 1 client.16209 shutdown 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb90fd86740 1 -- 172.21.15.63:0/404743614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 msgr2=0x55d18062cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb90fd86740 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d18062cd10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4011010 tx=0x7fb8f400bda0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb90fd86740 1 -- 172.21.15.63:0/404743614 shutdown_connections 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb90fd86740 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d18023ed80 0x55d18063ec60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb90fd86740 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802801b0 0x55d180638cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb90fd86740 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d18025f4d0 0x55d18062d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb8e3400640 0 client.16209 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb90fd86740 1 --2- 172.21.15.63:0/404743614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d18062cd10 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.329+0000 7fb90fd86740 1 -- 172.21.15.63:0/404743614 >> 172.21.15.63:0/404743614 conn(0x55d180274300 msgr2=0x55d180624580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.330+0000 7fb90fd86740 1 -- 172.21.15.63:0/404743614 shutdown_connections 2024-05-27T07:37:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.330+0000 7fb90fd86740 1 -- 172.21.15.63:0/404743614 wait complete. 2024-05-27T07:37:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.330+0000 7fb90fd86740 20 client.16209 trim_cache size 0 max 16384 2024-05-27T07:37:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.348+0000 7fb90fd86740 1 Processor -- start 2024-05-27T07:37:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.348+0000 7fb90fd86740 1 -- start start 2024-05-27T07:37:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.349+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d1806282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.349+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d18025f4d0 0x55d180638c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.349+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802801b0 0x55d180626480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.349+0000 7fb90fd86740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041b5c60 con 0x55d1802572f0 2024-05-27T07:37:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.349+0000 7fb90fd86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041b34d0 con 0x55d1802801b0 2024-05-27T07:37:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.349+0000 7fb90fd86740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041b1320 con 0x55d18025f4d0 2024-05-27T07:37:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.349+0000 7fb901e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d1806282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.349+0000 7fb901400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d18025f4d0 0x55d180638c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.349+0000 7fb901e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d1806282e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38294/0 (socket says 172.21.15.63:38294) 2024-05-27T07:37:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.349+0000 7fb901e00640 1 -- 172.21.15.63:0/182161495 learned_addr learned my addr 172.21.15.63:0/182161495 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.350+0000 7fb902800640 1 --2- 172.21.15.63:0/182161495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802801b0 0x55d180626480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.350+0000 7fb901e00640 1 -- 172.21.15.63:0/182161495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d18025f4d0 msgr2=0x55d180638c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.350+0000 7fb901e00640 1 --2- 172.21.15.63:0/182161495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d18025f4d0 0x55d180638c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.350+0000 7fb901e00640 1 -- 172.21.15.63:0/182161495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802801b0 msgr2=0x55d180626480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.350+0000 7fb901e00640 1 --2- 172.21.15.63:0/182161495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802801b0 0x55d180626480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.350+0000 7fb901e00640 1 -- 172.21.15.63:0/182161495 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18062b050 con 0x55d1802572f0 2024-05-27T07:37:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.350+0000 7fb901400640 1 --2- 172.21.15.63:0/182161495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d18025f4d0 0x55d180638c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.350+0000 7fb902800640 1 --2- 172.21.15.63:0/182161495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802801b0 0x55d180626480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.350+0000 7fb901e00640 1 --2- 172.21.15.63:0/182161495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d1806282e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b3d0 tx=0x7fb8f400b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8ecb36c944a34a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.351+0000 7fb8ee600640 1 -- 172.21.15.63:0/182161495 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f402a070 con 0x55d1802572f0 2024-05-27T07:37:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.351+0000 7fb8ee600640 1 -- 172.21.15.63:0/182161495 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f400be10 con 0x55d1802572f0 2024-05-27T07:37:45.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.351+0000 7fb8ee600640 1 -- 172.21.15.63:0/182161495 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4004280 con 0x55d1802572f0 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.352+0000 7fb90fd86740 1 -- 172.21.15.63:0/182161495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 msgr2=0x55d1806282e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.352+0000 7fb90fd86740 1 --2- 172.21.15.63:0/182161495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d1806282e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b3d0 tx=0x7fb8f400b570 comp rx=0 tx=0).stop 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.352+0000 7fb90fd86740 1 -- 172.21.15.63:0/182161495 shutdown_connections 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.352+0000 7fb90fd86740 1 --2- 172.21.15.63:0/182161495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1802801b0 0x55d180626480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.352+0000 7fb90fd86740 1 --2- 172.21.15.63:0/182161495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d18025f4d0 0x55d180638c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.352+0000 7fb90fd86740 1 --2- 172.21.15.63:0/182161495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802572f0 0x55d1806282e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.352+0000 7fb90fd86740 1 -- 172.21.15.63:0/182161495 >> 172.21.15.63:0/182161495 conn(0x55d180274300 msgr2=0x55d18062cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.352+0000 7fb90fd86740 1 -- 172.21.15.63:0/182161495 shutdown_connections 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.352+0000 7fb90fd86740 1 -- 172.21.15.63:0/182161495 wait complete. 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.353+0000 7fb90fd86740 1 -- start start 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.353+0000 7fb90fd86740 1 Processor -- start 2024-05-27T07:37:45.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802572f0 0x55d18063ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d18025f4d0 0x55d18063f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb90fd86740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802801b0 0x55d18063f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb90fd86740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041b1990 con 0x55d1802801b0 2024-05-27T07:37:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb90fd86740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041b2e00 con 0x55d18025f4d0 2024-05-27T07:37:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb90fd86740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041b2060 con 0x55d1802572f0 2024-05-27T07:37:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb90fd86740 10 client.0 _ll_register_callbacks cb 0x7fb8fc009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb902800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802801b0 0x55d18063f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb901e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802572f0 0x55d18063ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb902800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802801b0 0x55d18063f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38300/0 (socket says 172.21.15.63:38300) 2024-05-27T07:37:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb902800640 1 -- 172.21.15.63:0/640239496 learned_addr learned my addr 172.21.15.63:0/640239496 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.354+0000 7fb901400640 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d18025f4d0 0x55d18063f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.355+0000 7fb902800640 1 -- 172.21.15.63:0/640239496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802572f0 msgr2=0x55d18063ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.355+0000 7fb902800640 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802572f0 0x55d18063ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.355+0000 7fb902800640 1 -- 172.21.15.63:0/640239496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d18025f4d0 msgr2=0x55d18063f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.355+0000 7fb902800640 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d18025f4d0 0x55d18063f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.355+0000 7fb902800640 1 -- 172.21.15.63:0/640239496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1806a5220 con 0x55d1802801b0 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.355+0000 7fb902800640 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802801b0 0x55d18063f680 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009ca0 tx=0x7fb8fc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2ca475f1135015 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.355+0000 7fb8e3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.356+0000 7fb8e3400640 1 -- 172.21.15.63:0/640239496 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc004250 con 0x55d1802801b0 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.356+0000 7fb90fd86740 1 -- 172.21.15.63:0/640239496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f400be10 con 0x55d1802801b0 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.356+0000 7fb8e3400640 1 -- 172.21.15.63:0/640239496 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fb8fc042030 con 0x55d1802801b0 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.356+0000 7fb8e3400640 1 -- 172.21.15.63:0/640239496 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb8fc0043f0 con 0x55d1802801b0 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.356+0000 7fb90fd86740 10 client.16215 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.356+0000 7fb90fd86740 1 -- 172.21.15.63:0/640239496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1806a6490 con 0x55d1802801b0 2024-05-27T07:37:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.356+0000 7fb8e3400640 1 -- 172.21.15.63:0/640239496 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8f400be10 con 0x55d1802801b0 2024-05-27T07:37:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.357+0000 7fb8e3400640 1 -- 172.21.15.63:0/640239496 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb8fc030c80 con 0x55d1802801b0 2024-05-27T07:37:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.357+0000 7fb90fd86740 10 client.16215 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.357+0000 7fb90fd86740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.357+0000 7fb90fd86740 20 client.16215 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.357+0000 7fb90fd86740 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d1802141c0 0x55d1806a8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.357+0000 7fb90fd86740 4 client.16215 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:37:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.357+0000 7fb90fd86740 1 -- 172.21.15.63:0/640239496 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d180625080 con 0x55d1802141c0 2024-05-27T07:37:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.357+0000 7fb901e00640 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d1802141c0 0x55d1806a8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.359+0000 7fb901e00640 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d1802141c0 0x55d1806a8ce0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e591c36df32adae server_cookie=5156a1d257dad861 in_seq=0 out_seq=0 2024-05-27T07:37:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.359+0000 7fb8e3400640 10 client.16215 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.396+0000 7fb8e3400640 1 -- 172.21.15.63:0/640239496 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d180625080 con 0x55d1802141c0 2024-05-27T07:37:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.396+0000 7fb8e3400640 10 client.16215 handle_command_reply: tid=0 2024-05-27T07:37:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.396+0000 7fb90fd86740 1 client.16215 shutdown 2024-05-27T07:37:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.396+0000 7fb90fd86740 1 -- 172.21.15.63:0/640239496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802801b0 msgr2=0x55d18063f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.396+0000 7fb90fd86740 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802801b0 0x55d18063f680 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009ca0 tx=0x7fb8fc02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.397+0000 7fb90fd86740 1 -- 172.21.15.63:0/640239496 shutdown_connections 2024-05-27T07:37:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.397+0000 7fb90fd86740 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d1802141c0 0x55d1806a8ce0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.397+0000 7fb90fd86740 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1802801b0 0x55d18063f680 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.397+0000 7fb90fd86740 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d18025f4d0 0x55d18063f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.397+0000 7fb90fd86740 1 --2- 172.21.15.63:0/640239496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1802572f0 0x55d18063ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.397+0000 7fb90fd86740 1 -- 172.21.15.63:0/640239496 >> 172.21.15.63:0/640239496 conn(0x55d180274300 msgr2=0x55d1806225f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.397+0000 7fb8e3400640 0 client.16215 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.397+0000 7fb90fd86740 1 -- 172.21.15.63:0/640239496 shutdown_connections 2024-05-27T07:37:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.397+0000 7fb90fd86740 1 -- 172.21.15.63:0/640239496 wait complete. 2024-05-27T07:37:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.398+0000 7fb90fd86740 20 client.16215 trim_cache size 0 max 16384 2024-05-27T07:37:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:37:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:37:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:37:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.399+0000 7fb908c00640 1 -- 172.21.15.63:0/3102843016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e80b29d0 msgr2=0x7fb8e80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.399+0000 7fb908c00640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e80b29d0 0x7fb8e80b4ee0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006010 tx=0x7fb8fc004a80 comp rx=0 tx=0).stop 2024-05-27T07:37:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.399+0000 7fb908c00640 1 -- 172.21.15.63:0/3102843016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90410b7f0 msgr2=0x7fb9041a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.399+0000 7fb908c00640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90410b7f0 0x7fb9041a2ab0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000d770 tx=0x7fb8f000dc30 comp rx=0 tx=0).stop 2024-05-27T07:37:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.399+0000 7fb908c00640 1 -- 172.21.15.63:0/3102843016 shutdown_connections 2024-05-27T07:37:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.399+0000 7fb908c00640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8e80b29d0 0x7fb8e80b4ee0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.399+0000 7fb908c00640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb90410e1e0 0x7fb90419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.399+0000 7fb908c00640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb90410b7f0 0x7fb9041a2ab0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.399+0000 7fb908c00640 1 --2- 172.21.15.63:0/3102843016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb904104d90 0x7fb9041a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.399+0000 7fb908c00640 1 -- 172.21.15.63:0/3102843016 >> 172.21.15.63:0/3102843016 conn(0x7fb9040782d0 msgr2=0x7fb90410a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.400+0000 7fb908c00640 1 -- 172.21.15.63:0/3102843016 shutdown_connections 2024-05-27T07:37:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:45.400+0000 7fb908c00640 1 -- 172.21.15.63:0/3102843016 wait complete. 2024-05-27T07:37:45.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.491+0000 7fef20c00700 1 -- 172.21.15.63:0/2780120552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef1c106eb0 msgr2=0x7fef1c113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.491+0000 7fef20c00700 1 --2- 172.21.15.63:0/2780120552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef1c106eb0 0x7fef1c113340 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fef1400b0a0 tx=0x7fef1400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.491+0000 7fef20c00700 1 -- 172.21.15.63:0/2780120552 shutdown_connections 2024-05-27T07:37:45.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.491+0000 7fef20c00700 1 --2- 172.21.15.63:0/2780120552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef1c106eb0 0x7fef1c113340 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.491+0000 7fef20c00700 1 --2- 172.21.15.63:0/2780120552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef1c10c5a0 0x7fef1c10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.491+0000 7fef20c00700 1 --2- 172.21.15.63:0/2780120552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c106560 0x7fef1c106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.491+0000 7fef20c00700 1 -- 172.21.15.63:0/2780120552 >> 172.21.15.63:0/2780120552 conn(0x7fef1c102040 msgr2=0x7fef1c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.491+0000 7fef20c00700 1 -- 172.21.15.63:0/2780120552 shutdown_connections 2024-05-27T07:37:45.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.492+0000 7fef20c00700 1 -- 172.21.15.63:0/2780120552 wait complete. 2024-05-27T07:37:45.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.493+0000 7fef20c00700 1 Processor -- start 2024-05-27T07:37:45.490 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:45 smithi063 ceph-mon[31613]: pgmap v129: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:37:45.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.493+0000 7fef20c00700 1 -- start start 2024-05-27T07:37:45.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.493+0000 7fef20c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c106560 0x7fef1c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.494+0000 7fef20c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef1c106eb0 0x7fef1c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.494+0000 7fef20c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef1c10c5a0 0x7fef1c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.494+0000 7fef20c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1165d0 con 0x7fef1c10c5a0 2024-05-27T07:37:45.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.494+0000 7fef20c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c116450 con 0x7fef1c106560 2024-05-27T07:37:45.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.494+0000 7fef20c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c116750 con 0x7fef1c106eb0 2024-05-27T07:37:45.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.494+0000 7fef1a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef1c10c5a0 0x7fef1c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.494+0000 7fef1a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef1c10c5a0 0x7fef1c1a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38324/0 (socket says 172.21.15.63:38324) 2024-05-27T07:37:45.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.494+0000 7fef1a800700 1 -- 172.21.15.63:0/3986844454 learned_addr learned my addr 172.21.15.63:0/3986844454 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:45.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.494+0000 7fef19e00700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c106560 0x7fef1c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.494+0000 7fef19400700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef1c106eb0 0x7fef1c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.495+0000 7fef1a800700 1 -- 172.21.15.63:0/3986844454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef1c106eb0 msgr2=0x7fef1c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.495+0000 7fef1a800700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef1c106eb0 0x7fef1c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.495+0000 7fef1a800700 1 -- 172.21.15.63:0/3986844454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c106560 msgr2=0x7fef1c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.495+0000 7fef1a800700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c106560 0x7fef1c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.495+0000 7fef1a800700 1 -- 172.21.15.63:0/3986844454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c1a1dd0 con 0x7fef1c10c5a0 2024-05-27T07:37:45.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.495+0000 7fef1a800700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef1c10c5a0 0x7fef1c1a1730 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fef1400b070 tx=0x7fef14004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.495+0000 7fef12a00700 1 -- 172.21.15.63:0/3986844454 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14021070 con 0x7fef1c10c5a0 2024-05-27T07:37:45.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.495+0000 7fef12a00700 1 -- 172.21.15.63:0/3986844454 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef14007b80 con 0x7fef1c10c5a0 2024-05-27T07:37:45.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.496+0000 7fef20c00700 1 -- 172.21.15.63:0/3986844454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1c1a2060 con 0x7fef1c10c5a0 2024-05-27T07:37:45.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.496+0000 7fef12a00700 1 -- 172.21.15.63:0/3986844454 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14004650 con 0x7fef1c10c5a0 2024-05-27T07:37:45.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.496+0000 7fef20c00700 1 -- 172.21.15.63:0/3986844454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1c1a99e0 con 0x7fef1c10c5a0 2024-05-27T07:37:45.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.501+0000 7fef12a00700 1 -- 172.21.15.63:0/3986844454 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fef14003a00 con 0x7fef1c10c5a0 2024-05-27T07:37:45.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.501+0000 7fef2f786b80 1 Processor -- start 2024-05-27T07:37:45.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.501+0000 7fef2f786b80 1 -- start start 2024-05-27T07:37:45.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.502+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83aacc9f0 0x55f83ab5f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.502+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83a9e0e40 0x55f83ab5f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.502+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab6c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.502+0000 7fef2f786b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f83aa8de40 con 0x55f83a9e0e40 2024-05-27T07:37:45.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.502+0000 7fef2f786b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f83aa7d9e0 con 0x55f83aacc9f0 2024-05-27T07:37:45.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.502+0000 7fef2f786b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f83aa81a20 con 0x55f83ab5fe70 2024-05-27T07:37:45.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.502+0000 7fef19e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83a9e0e40 0x55f83ab5f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.502+0000 7fef19e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83a9e0e40 0x55f83ab5f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38336/0 (socket says 172.21.15.63:38336) 2024-05-27T07:37:45.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.502+0000 7fef19e00700 1 -- 172.21.15.63:0/2484557491 learned_addr learned my addr 172.21.15.63:0/2484557491 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:45.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.503+0000 7fef19400700 1 --2- 172.21.15.63:0/2484557491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83aacc9f0 0x55f83ab5f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.503+0000 7fef19e00700 1 -- 172.21.15.63:0/2484557491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 msgr2=0x55f83ab6c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.503+0000 7fef12a00700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef000b2a70 0x7fef000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.503+0000 7fef19400700 1 --2- 172.21.15.63:0/2484557491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab6c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.503+0000 7fef19e00700 1 --2- 172.21.15.63:0/2484557491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab6c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.503+0000 7fef19e00700 1 -- 172.21.15.63:0/2484557491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83aacc9f0 msgr2=0x55f83ab5f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.503+0000 7fef19e00700 1 --2- 172.21.15.63:0/2484557491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83aacc9f0 0x55f83ab5f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.503+0000 7fef19e00700 1 -- 172.21.15.63:0/2484557491 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83aa46420 con 0x55f83a9e0e40 2024-05-27T07:37:45.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.503+0000 7fef19400700 1 --2- 172.21.15.63:0/2484557491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab6c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:45.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.503+0000 7fef12a00700 1 -- 172.21.15.63:0/3986844454 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fef1400f020 con 0x7fef1c10c5a0 2024-05-27T07:37:45.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.504+0000 7fef1a800700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef000b2a70 0x7fef000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.504+0000 7fef19e00700 1 --2- 172.21.15.63:0/2484557491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83a9e0e40 0x55f83ab5f870 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fef0c007c90 tx=0x7fef0c0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbe4c2ff9fffca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.505+0000 7fef07400700 1 -- 172.21.15.63:0/2484557491 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c010cb0 con 0x55f83a9e0e40 2024-05-27T07:37:45.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.505+0000 7fef07400700 1 -- 172.21.15.63:0/2484557491 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef0c015070 con 0x55f83a9e0e40 2024-05-27T07:37:45.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.505+0000 7fef07400700 1 -- 172.21.15.63:0/2484557491 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c003d90 con 0x55f83a9e0e40 2024-05-27T07:37:45.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.506+0000 7fef2f786b80 1 -- 172.21.15.63:0/2484557491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83a9e0e40 msgr2=0x55f83ab5f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.506+0000 7fef2f786b80 1 --2- 172.21.15.63:0/2484557491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83a9e0e40 0x55f83ab5f870 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fef0c007c90 tx=0x7fef0c0103d0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.506+0000 7fef1a800700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef000b2a70 0x7fef000b4f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fef1400d010 tx=0x7fef14004f70 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.506+0000 7fef2f786b80 1 -- 172.21.15.63:0/2484557491 shutdown_connections 2024-05-27T07:37:45.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.506+0000 7fef2f786b80 1 --2- 172.21.15.63:0/2484557491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab6c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.506+0000 7fef2f786b80 1 --2- 172.21.15.63:0/2484557491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83aacc9f0 0x55f83ab5f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.506+0000 7fef2f786b80 1 --2- 172.21.15.63:0/2484557491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83a9e0e40 0x55f83ab5f870 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.506+0000 7fef2f786b80 1 -- 172.21.15.63:0/2484557491 >> 172.21.15.63:0/2484557491 conn(0x55f83aa7a390 msgr2=0x55f83ab5b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.506+0000 7fef2f786b80 1 -- 172.21.15.63:0/2484557491 shutdown_connections 2024-05-27T07:37:45.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.506+0000 7fef2f786b80 1 -- 172.21.15.63:0/2484557491 wait complete. 2024-05-27T07:37:45.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.507+0000 7fef2f786b80 1 -- start start 2024-05-27T07:37:45.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.507+0000 7fef2f786b80 1 Processor -- start 2024-05-27T07:37:45.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83aacc9f0 0x55f83ab69c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83ab5fe70 0x55f83ab6a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef2f786b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f83a983fa0 con 0x55f83ab5fe70 2024-05-27T07:37:45.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef2f786b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f83aa1e760 con 0x55f83a9e0e40 2024-05-27T07:37:45.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef2f786b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f83ab6cb80 con 0x55f83aacc9f0 2024-05-27T07:37:45.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef2f786b80 10 client.0 _ll_register_callbacks cb 0x55f83aa27c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:45.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef19400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83ab5fe70 0x55f83ab6a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef19400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83ab5fe70 0x55f83ab6a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38342/0 (socket says 172.21.15.63:38342) 2024-05-27T07:37:45.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef19400700 1 -- 172.21.15.63:0/2073409293 learned_addr learned my addr 172.21.15.63:0/2073409293 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:45.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef19e00700 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83aacc9f0 0x55f83ab69c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.508+0000 7fef19400700 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.509+0000 7fef19400700 1 -- 172.21.15.63:0/2073409293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83aacc9f0 msgr2=0x55f83ab69c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.509+0000 7fef19400700 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83aacc9f0 0x55f83ab69c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.509+0000 7fef19400700 1 -- 172.21.15.63:0/2073409293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 msgr2=0x55f83ab696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.509+0000 7fef19400700 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.509+0000 7fef19400700 1 -- 172.21.15.63:0/2073409293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83a9cd8d0 con 0x55f83ab5fe70 2024-05-27T07:37:45.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.509+0000 7fef19400700 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83ab5fe70 0x55f83ab6a140 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fef1c075b70 tx=0x7fef080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a015aadd5ccb3a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.509+0000 7feefbe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:45.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.510+0000 7feefbe00700 1 -- 172.21.15.63:0/2073409293 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef080122d0 con 0x55f83ab5fe70 2024-05-27T07:37:45.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.510+0000 7feefbe00700 1 -- 172.21.15.63:0/2073409293 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fef08004260 con 0x55f83ab5fe70 2024-05-27T07:37:45.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.510+0000 7fef2f786b80 1 -- 172.21.15.63:0/2073409293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef0c015070 con 0x55f83ab5fe70 2024-05-27T07:37:45.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.510+0000 7feefbe00700 1 -- 172.21.15.63:0/2073409293 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fef08028050 con 0x55f83ab5fe70 2024-05-27T07:37:45.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.510+0000 7fef2f786b80 10 client.16227 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:45.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.510+0000 7fef2f786b80 1 -- 172.21.15.63:0/2073409293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f83aa4e060 con 0x55f83ab5fe70 2024-05-27T07:37:45.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.511+0000 7feefbe00700 1 -- 172.21.15.63:0/2073409293 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef0c015070 con 0x55f83ab5fe70 2024-05-27T07:37:45.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.511+0000 7feefbe00700 1 -- 172.21.15.63:0/2073409293 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fef08003680 con 0x55f83ab5fe70 2024-05-27T07:37:45.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.511+0000 7fef2f786b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:45.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.511+0000 7fef2f786b80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:45.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.511+0000 7fef2f786b80 20 client.16227 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:45.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.511+0000 7fef2f786b80 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f83ab79a60 0x55f83ab7bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.511+0000 7fef2f786b80 4 client.16227 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:45.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.511+0000 7fef2f786b80 1 -- 172.21.15.63:0/2073409293 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83aa8fd90 con 0x55f83ab79a60 2024-05-27T07:37:45.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.512+0000 7fef19e00700 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f83ab79a60 0x55f83ab7bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.513+0000 7feefbe00700 10 client.16227 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:45.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.513+0000 7fef19e00700 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f83ab79a60 0x55f83ab7bf10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67daf4909072560b server_cookie=4387242fac961018 in_seq=0 out_seq=0 2024-05-27T07:37:45.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7feefbe00700 1 -- 172.21.15.63:0/2073409293 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f83aa8fd90 con 0x55f83ab79a60 2024-05-27T07:37:45.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7feefbe00700 10 client.16227 handle_command_reply: tid=0 2024-05-27T07:37:45.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7fef2f786b80 1 client.16227 shutdown 2024-05-27T07:37:45.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7fef2f786b80 1 -- 172.21.15.63:0/2073409293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83ab5fe70 msgr2=0x55f83ab6a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7fef2f786b80 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83ab5fe70 0x55f83ab6a140 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fef1c075b70 tx=0x7fef080047c0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7fef2f786b80 1 -- 172.21.15.63:0/2073409293 shutdown_connections 2024-05-27T07:37:45.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7fef2f786b80 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f83ab79a60 0x55f83ab7bf10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7fef2f786b80 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83ab5fe70 0x55f83ab6a140 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7feefbe00700 0 client.16227 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:45.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7fef2f786b80 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7fef2f786b80 1 --2- 172.21.15.63:0/2073409293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83aacc9f0 0x55f83ab69c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.515+0000 7fef2f786b80 1 -- 172.21.15.63:0/2073409293 >> 172.21.15.63:0/2073409293 conn(0x55f83aa7a390 msgr2=0x55f83ab5f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.516+0000 7fef2f786b80 1 -- 172.21.15.63:0/2073409293 shutdown_connections 2024-05-27T07:37:45.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.516+0000 7fef2f786b80 1 -- 172.21.15.63:0/2073409293 wait complete. 2024-05-27T07:37:45.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.516+0000 7fef2f786b80 20 client.16227 trim_cache size 0 max 16384 2024-05-27T07:37:45.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.535+0000 7fef2f786b80 1 Processor -- start 2024-05-27T07:37:45.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.535+0000 7fef2f786b80 1 -- start start 2024-05-27T07:37:45.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab64e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab5f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef2f786b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1b4df0 con 0x55f83aacc9f0 2024-05-27T07:37:45.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef2f786b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1b2620 con 0x55f83a9e0e40 2024-05-27T07:37:45.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef2f786b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1b0470 con 0x55f83ab5fe70 2024-05-27T07:37:45.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef19e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab5f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef19e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab5f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38348/0 (socket says 172.21.15.63:38348) 2024-05-27T07:37:45.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef19e00700 1 -- 172.21.15.63:0/646320735 learned_addr learned my addr 172.21.15.63:0/646320735 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:45.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef19400700 1 --2- 172.21.15.63:0/646320735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab64e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.536+0000 7fef19400700 1 --2- 172.21.15.63:0/646320735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.537+0000 7fef19e00700 1 -- 172.21.15.63:0/646320735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 msgr2=0x55f83ab816d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.537+0000 7fef19e00700 1 --2- 172.21.15.63:0/646320735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab816d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.537+0000 7fef19e00700 1 -- 172.21.15.63:0/646320735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 msgr2=0x55f83ab64e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.537+0000 7fef19e00700 1 --2- 172.21.15.63:0/646320735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab64e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.537+0000 7fef19e00700 1 -- 172.21.15.63:0/646320735 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83ab60a50 con 0x55f83aacc9f0 2024-05-27T07:37:45.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.537+0000 7fef19400700 1 --2- 172.21.15.63:0/646320735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab816d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:45.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.537+0000 7fef19e00700 1 --2- 172.21.15.63:0/646320735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab5f7c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fef0c007c90 tx=0x7fef0c0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d83b61b2e447f71 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.537+0000 7fef06a00700 1 -- 172.21.15.63:0/646320735 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c02b070 con 0x55f83aacc9f0 2024-05-27T07:37:45.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.538+0000 7fef06a00700 1 -- 172.21.15.63:0/646320735 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef0c015040 con 0x55f83aacc9f0 2024-05-27T07:37:45.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.538+0000 7fef06a00700 1 -- 172.21.15.63:0/646320735 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c01a470 con 0x55f83aacc9f0 2024-05-27T07:37:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.539+0000 7fef2f786b80 1 -- 172.21.15.63:0/646320735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 msgr2=0x55f83ab5f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.539+0000 7fef2f786b80 1 --2- 172.21.15.63:0/646320735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab5f7c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fef0c007c90 tx=0x7fef0c0075f0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.539+0000 7fef2f786b80 1 -- 172.21.15.63:0/646320735 shutdown_connections 2024-05-27T07:37:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.539+0000 7fef2f786b80 1 --2- 172.21.15.63:0/646320735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab816d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.539+0000 7fef2f786b80 1 --2- 172.21.15.63:0/646320735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab64e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.539+0000 7fef2f786b80 1 --2- 172.21.15.63:0/646320735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab5f7c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.539+0000 7fef2f786b80 1 -- 172.21.15.63:0/646320735 >> 172.21.15.63:0/646320735 conn(0x55f83aa7a390 msgr2=0x55f83ab5f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.539+0000 7fef2f786b80 1 -- 172.21.15.63:0/646320735 shutdown_connections 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.539+0000 7fef2f786b80 1 -- 172.21.15.63:0/646320735 wait complete. 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.539+0000 7fef2f786b80 1 -- start start 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.540+0000 7fef2f786b80 1 Processor -- start 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.540+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.540+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab72600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.540+0000 7fef2f786b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab6d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.540+0000 7fef2f786b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1b0ae0 con 0x55f83aacc9f0 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.540+0000 7fef2f786b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1b1f50 con 0x55f83a9e0e40 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.540+0000 7fef2f786b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1b11b0 con 0x55f83ab5fe70 2024-05-27T07:37:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef2f786b80 10 client.0 _ll_register_callbacks cb 0x55f83aa27c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab72600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab72600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38350/0 (socket says 172.21.15.63:38350) 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19e00700 1 -- 172.21.15.63:0/3553983371 learned_addr learned my addr 172.21.15.63:0/3553983371 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19400700 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19400700 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab6d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19e00700 1 -- 172.21.15.63:0/3553983371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 msgr2=0x55f83ab6d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19e00700 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab6d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19e00700 1 -- 172.21.15.63:0/3553983371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 msgr2=0x55f83ab720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19e00700 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19e00700 1 -- 172.21.15.63:0/3553983371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83ab6dc70 con 0x55f83aacc9f0 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.541+0000 7fef19400700 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab6d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.542+0000 7fef19e00700 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab72600 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fef0c005cb0 tx=0x7fef0c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ff874c961aee61 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.542+0000 7feefbe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.542+0000 7feefbe00700 1 -- 172.21.15.63:0/3553983371 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c034030 con 0x55f83aacc9f0 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.542+0000 7fef2f786b80 1 -- 172.21.15.63:0/3553983371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef0c015040 con 0x55f83aacc9f0 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.542+0000 7feefbe00700 1 -- 172.21.15.63:0/3553983371 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fef0c0038e0 con 0x55f83aacc9f0 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.543+0000 7feefbe00700 1 -- 172.21.15.63:0/3553983371 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fef0c03e050 con 0x55f83aacc9f0 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.543+0000 7fef2f786b80 10 client.16233 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.543+0000 7fef2f786b80 1 -- 172.21.15.63:0/3553983371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f83abe4000 con 0x55f83aacc9f0 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.543+0000 7feefbe00700 1 -- 172.21.15.63:0/3553983371 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef0c015040 con 0x55f83aacc9f0 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.543+0000 7feefbe00700 1 -- 172.21.15.63:0/3553983371 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fef0c030920 con 0x55f83aacc9f0 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.543+0000 7fef2f786b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.543+0000 7fef2f786b80 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.543+0000 7fef2f786b80 20 client.16233 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.543+0000 7fef2f786b80 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f83abe4490 0x55f83abe68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.544+0000 7fef2f786b80 4 client.16233 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.544+0000 7fef2f786b80 1 -- 172.21.15.63:0/3553983371 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f83aa89570 con 0x55f83abe4490 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.544+0000 7fef19400700 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f83abe4490 0x55f83abe68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.545+0000 7fef19400700 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f83abe4490 0x55f83abe68b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0adc2075c865d07 server_cookie=6c47b17d7c8e3e17 in_seq=0 out_seq=0 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.545+0000 7feefbe00700 10 client.16233 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7feefbe00700 1 -- 172.21.15.63:0/3553983371 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f83aa89570 con 0x55f83abe4490 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7feefbe00700 10 client.16233 handle_command_reply: tid=0 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7fef2f786b80 1 client.16233 shutdown 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7fef2f786b80 1 -- 172.21.15.63:0/3553983371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 msgr2=0x55f83ab72600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7fef2f786b80 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab72600 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fef0c005cb0 tx=0x7fef0c010d80 comp rx=0 tx=0).stop 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7fef2f786b80 1 -- 172.21.15.63:0/3553983371 shutdown_connections 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7fef2f786b80 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f83abe4490 0x55f83abe68b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7fef2f786b80 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f83ab5fe70 0x55f83ab6d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7fef2f786b80 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f83a9e0e40 0x55f83ab720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7fef2f786b80 1 --2- 172.21.15.63:0/3553983371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f83aacc9f0 0x55f83ab72600 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7fef2f786b80 1 -- 172.21.15.63:0/3553983371 >> 172.21.15.63:0/3553983371 conn(0x55f83aa7a390 msgr2=0x55f83ab694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.546+0000 7feefbe00700 0 client.16233 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.547+0000 7fef2f786b80 1 -- 172.21.15.63:0/3553983371 shutdown_connections 2024-05-27T07:37:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.547+0000 7fef2f786b80 1 -- 172.21.15.63:0/3553983371 wait complete. 2024-05-27T07:37:45.550 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:37:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.547+0000 7fef2f786b80 20 client.16233 trim_cache size 0 max 16384 2024-05-27T07:37:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.548+0000 7fef20c00700 1 -- 172.21.15.63:0/3986844454 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef000b2a70 msgr2=0x7fef000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.548+0000 7fef20c00700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef000b2a70 0x7fef000b4f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fef1400d010 tx=0x7fef14004f70 comp rx=0 tx=0).stop 2024-05-27T07:37:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.548+0000 7fef20c00700 1 -- 172.21.15.63:0/3986844454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef1c10c5a0 msgr2=0x7fef1c1a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.548+0000 7fef20c00700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef1c10c5a0 0x7fef1c1a1730 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fef1400b070 tx=0x7fef14004c80 comp rx=0 tx=0).stop 2024-05-27T07:37:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.548+0000 7fef20c00700 1 -- 172.21.15.63:0/3986844454 shutdown_connections 2024-05-27T07:37:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.548+0000 7fef20c00700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef000b2a70 0x7fef000b4f80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.548+0000 7fef20c00700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef1c10c5a0 0x7fef1c1a1730 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.548+0000 7fef20c00700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c106560 0x7fef1c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.548+0000 7fef20c00700 1 --2- 172.21.15.63:0/3986844454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef1c106eb0 0x7fef1c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.548+0000 7fef20c00700 1 -- 172.21.15.63:0/3986844454 >> 172.21.15.63:0/3986844454 conn(0x7fef1c102040 msgr2=0x7fef1c1035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.549+0000 7fef20c00700 1 -- 172.21.15.63:0/3986844454 shutdown_connections 2024-05-27T07:37:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:45.549+0000 7fef20c00700 1 -- 172.21.15.63:0/3986844454 wait complete. 2024-05-27T07:37:45.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:45 smithi086 ceph-mon[34687]: pgmap v129: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:37:45.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:45 smithi149 ceph-mon[35408]: pgmap v129: 129 pgs: 129 active+clean; 405 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:37:47.557 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:37:47.558 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:37:48.071 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:37:48.564 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:48 smithi063 ceph-mon[31613]: pgmap v130: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 18 op/s 2024-05-27T07:37:48.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:48 smithi086 ceph-mon[34687]: pgmap v130: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 18 op/s 2024-05-27T07:37:48.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:48 smithi149 ceph-mon[35408]: pgmap v130: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 18 op/s 2024-05-27T07:37:50.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.660+0000 7f17f6000700 1 -- 172.21.15.63:0/1999636091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f010c5a0 msgr2=0x7f17f010c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:50.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.660+0000 7f17f6000700 1 --2- 172.21.15.63:0/1999636091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f010c5a0 0x7f17f010c9b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009990 tx=0x7f17e0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.659 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:50 smithi063 ceph-mon[31613]: pgmap v131: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:37:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.660+0000 7f17f6000700 1 -- 172.21.15.63:0/1999636091 shutdown_connections 2024-05-27T07:37:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.660+0000 7f17f6000700 1 --2- 172.21.15.63:0/1999636091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17f0106eb0 0x7f17f0113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.660+0000 7f17f6000700 1 --2- 172.21.15.63:0/1999636091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f010c5a0 0x7f17f010c9b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.660+0000 7f17f6000700 1 --2- 172.21.15.63:0/1999636091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17f0106560 0x7f17f0106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.660+0000 7f17f6000700 1 -- 172.21.15.63:0/1999636091 >> 172.21.15.63:0/1999636091 conn(0x7f17f0102040 msgr2=0x7f17f0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.661+0000 7f17f6000700 1 -- 172.21.15.63:0/1999636091 shutdown_connections 2024-05-27T07:37:50.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.661+0000 7f17f6000700 1 -- 172.21.15.63:0/1999636091 wait complete. 2024-05-27T07:37:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.662+0000 7f17f6000700 1 Processor -- start 2024-05-27T07:37:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.663+0000 7f17f6000700 1 -- start start 2024-05-27T07:37:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.663+0000 7f17f6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17f0106560 0x7f17f01107b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.663+0000 7f17f6000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17f0106eb0 0x7f17f0110cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.663+0000 7f17f6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f010c5a0 0x7f17f0111230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.663+0000 7f17f6000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0116560 con 0x7f17f010c5a0 2024-05-27T07:37:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.663+0000 7f17f6000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01163e0 con 0x7f17f0106560 2024-05-27T07:37:50.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.663+0000 7f17f6000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01166e0 con 0x7f17f0106eb0 2024-05-27T07:37:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.663+0000 7f17efe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f010c5a0 0x7f17f0111230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.664+0000 7f17efe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f010c5a0 0x7f17f0111230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54112/0 (socket says 172.21.15.63:54112) 2024-05-27T07:37:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.664+0000 7f17ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17f0106560 0x7f17f01107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.664+0000 7f17efe00700 1 -- 172.21.15.63:0/3223830882 learned_addr learned my addr 172.21.15.63:0/3223830882 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.664+0000 7f17eea00700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17f0106eb0 0x7f17f0110cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.664+0000 7f17efe00700 1 -- 172.21.15.63:0/3223830882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17f0106eb0 msgr2=0x7f17f0110cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.664+0000 7f17efe00700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17f0106eb0 0x7f17f0110cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.664+0000 7f17efe00700 1 -- 172.21.15.63:0/3223830882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17f0106560 msgr2=0x7f17f01107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:50.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.664+0000 7f17efe00700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17f0106560 0x7f17f01107b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.664+0000 7f17efe00700 1 -- 172.21.15.63:0/3223830882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f01a2c80 con 0x7f17f010c5a0 2024-05-27T07:37:50.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.665+0000 7f17efe00700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f010c5a0 0x7f17f0111230 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f17e8007ee0 tx=0x7f17e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:50.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.665+0000 7f17e7e00700 1 -- 172.21.15.63:0/3223830882 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8004bb0 con 0x7f17f010c5a0 2024-05-27T07:37:50.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.665+0000 7f17e7e00700 1 -- 172.21.15.63:0/3223830882 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17e8004d50 con 0x7f17f010c5a0 2024-05-27T07:37:50.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.665+0000 7f17f6000700 1 -- 172.21.15.63:0/3223830882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f01a2f70 con 0x7f17f010c5a0 2024-05-27T07:37:50.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.665+0000 7f17e7e00700 1 -- 172.21.15.63:0/3223830882 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80058b0 con 0x7f17f010c5a0 2024-05-27T07:37:50.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.665+0000 7f17f6000700 1 -- 172.21.15.63:0/3223830882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f01a3460 con 0x7f17f010c5a0 2024-05-27T07:37:50.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.670+0000 7f17e7e00700 1 -- 172.21.15.63:0/3223830882 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f17e801a020 con 0x7f17f010c5a0 2024-05-27T07:37:50.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.671+0000 7f1804b1db80 1 Processor -- start 2024-05-27T07:37:50.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.671+0000 7f1804b1db80 1 -- start start 2024-05-27T07:37:50.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.672+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1dc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.672+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a05dec0 0x55785a1dc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.672+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a1dcef0 0x55785a1e9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.672+0000 7f1804b1db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55785a10aec0 con 0x55785a149a70 2024-05-27T07:37:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.672+0000 7f1804b1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55785a0faa60 con 0x55785a05dec0 2024-05-27T07:37:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.672+0000 7f1804b1db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55785a0feaa0 con 0x55785a1dcef0 2024-05-27T07:37:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.672+0000 7f17eea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1dc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.672+0000 7f17eea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1dc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54128/0 (socket says 172.21.15.63:54128) 2024-05-27T07:37:50.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17eea00700 1 -- 172.21.15.63:0/98842136 learned_addr learned my addr 172.21.15.63:0/98842136 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:50.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17e7e00700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f17d40b29f0 0x7f17d40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17ef400700 1 --2- 172.21.15.63:0/98842136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a05dec0 0x55785a1dc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17eea00700 1 --2- 172.21.15.63:0/98842136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a1dcef0 0x55785a1e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17efe00700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f17d40b29f0 0x7f17d40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17e7e00700 1 -- 172.21.15.63:0/3223830882 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f17e80e1ff0 con 0x7f17f010c5a0 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17eea00700 1 -- 172.21.15.63:0/98842136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a1dcef0 msgr2=0x55785a1e9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17eea00700 1 --2- 172.21.15.63:0/98842136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a1dcef0 0x55785a1e9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17eea00700 1 -- 172.21.15.63:0/98842136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a05dec0 msgr2=0x55785a1dc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17eea00700 1 --2- 172.21.15.63:0/98842136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a05dec0 0x55785a1dc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.673+0000 7f17eea00700 1 -- 172.21.15.63:0/98842136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55785a0c34a0 con 0x55785a149a70 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.674+0000 7f17eea00700 1 --2- 172.21.15.63:0/98842136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1dc3b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f17dc005d90 tx=0x7f17dc005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d02d99fd8312e41 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.674+0000 7f17efe00700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f17d40b29f0 0x7f17d40b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f17e80a8c20 tx=0x7f17e80a4ec0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.674+0000 7f17e4c00700 1 -- 172.21.15.63:0/98842136 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc004030 con 0x55785a149a70 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.675+0000 7f17e4c00700 1 -- 172.21.15.63:0/98842136 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17dc004830 con 0x55785a149a70 2024-05-27T07:37:50.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.675+0000 7f17e4c00700 1 -- 172.21.15.63:0/98842136 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc005280 con 0x55785a149a70 2024-05-27T07:37:50.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.675+0000 7f1804b1db80 1 -- 172.21.15.63:0/98842136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 msgr2=0x55785a1dc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:50.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.675+0000 7f1804b1db80 1 --2- 172.21.15.63:0/98842136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1dc3b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f17dc005d90 tx=0x7f17dc005d20 comp rx=0 tx=0).stop 2024-05-27T07:37:50.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.676+0000 7f1804b1db80 1 -- 172.21.15.63:0/98842136 shutdown_connections 2024-05-27T07:37:50.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.676+0000 7f1804b1db80 1 --2- 172.21.15.63:0/98842136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a1dcef0 0x55785a1e9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.676+0000 7f1804b1db80 1 --2- 172.21.15.63:0/98842136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1dc3b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.676+0000 7f1804b1db80 1 --2- 172.21.15.63:0/98842136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a05dec0 0x55785a1dc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.676+0000 7f1804b1db80 1 -- 172.21.15.63:0/98842136 >> 172.21.15.63:0/98842136 conn(0x55785a0f7410 msgr2=0x55785a1d8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:50.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.676+0000 7f1804b1db80 1 -- 172.21.15.63:0/98842136 shutdown_connections 2024-05-27T07:37:50.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.676+0000 7f1804b1db80 1 -- 172.21.15.63:0/98842136 wait complete. 2024-05-27T07:37:50.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.677+0000 7f1804b1db80 1 -- start start 2024-05-27T07:37:50.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.677+0000 7f1804b1db80 1 Processor -- start 2024-05-27T07:37:50.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.678+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a05dec0 0x55785a1e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.678+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a149a70 0x55785a1e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.678+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a1dcef0 0x55785a1e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.678+0000 7f1804b1db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55785a001020 con 0x55785a1dcef0 2024-05-27T07:37:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.678+0000 7f1804b1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55785a09b7e0 con 0x55785a05dec0 2024-05-27T07:37:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.678+0000 7f1804b1db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55785a1e9c00 con 0x55785a149a70 2024-05-27T07:37:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.678+0000 7f1804b1db80 10 client.0 _ll_register_callbacks cb 0x55785a0a4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.678+0000 7f17eea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a1dcef0 0x55785a1e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.678+0000 7f17eea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a1dcef0 0x55785a1e71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54138/0 (socket says 172.21.15.63:54138) 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.678+0000 7f17eea00700 1 -- 172.21.15.63:0/2585292608 learned_addr learned my addr 172.21.15.63:0/2585292608 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.679+0000 7f17ef400700 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a149a70 0x55785a1e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.679+0000 7f17eea00700 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a05dec0 0x55785a1e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.679+0000 7f17ef400700 1 -- 172.21.15.63:0/2585292608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a05dec0 msgr2=0x55785a1e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.679+0000 7f17ef400700 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a05dec0 0x55785a1e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.679+0000 7f17ef400700 1 -- 172.21.15.63:0/2585292608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a1dcef0 msgr2=0x55785a1e71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.679+0000 7f17ef400700 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a1dcef0 0x55785a1e71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.679+0000 7f17ef400700 1 -- 172.21.15.63:0/2585292608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785a04a950 con 0x55785a149a70 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.679+0000 7f17eea00700 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a1dcef0 0x55785a1e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:37:50.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.680+0000 7f17ef400700 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a149a70 0x55785a1e6ca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f17e00098e0 tx=0x7f17e0005af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed902fe1f4324325 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.680+0000 7f17d9600700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.680+0000 7f17d9600700 1 -- 172.21.15.63:0/2585292608 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0003bd0 con 0x55785a149a70 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.681+0000 7f17d9600700 1 -- 172.21.15.63:0/2585292608 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f17e0003d70 con 0x55785a149a70 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.681+0000 7f1804b1db80 1 -- 172.21.15.63:0/2585292608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17dc004830 con 0x55785a149a70 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.681+0000 7f17d9600700 1 -- 172.21.15.63:0/2585292608 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f17e000b770 con 0x55785a149a70 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.681+0000 7f1804b1db80 10 client.25810 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.681+0000 7f1804b1db80 1 -- 172.21.15.63:0/2585292608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55785a0cb0e0 con 0x55785a149a70 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.682+0000 7f17d9600700 1 -- 172.21.15.63:0/2585292608 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17dc004830 con 0x55785a149a70 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.682+0000 7f17d9600700 1 -- 172.21.15.63:0/2585292608 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f17e00237e0 con 0x55785a149a70 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.682+0000 7f1804b1db80 10 client.25810 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.682+0000 7f1804b1db80 10 client.25810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.683+0000 7f1804b1db80 20 client.25810 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:50.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.683+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55785a1f6d10 0x55785a1f9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.683+0000 7f1804b1db80 4 client.25810 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:50.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.683+0000 7f1804b1db80 1 -- 172.21.15.63:0/2585292608 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55785a10ce10 con 0x55785a1f6d10 2024-05-27T07:37:50.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.683+0000 7f17eea00700 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55785a1f6d10 0x55785a1f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.684+0000 7f17eea00700 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55785a1f6d10 0x55785a1f9160 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4480eadd457447bf server_cookie=a8589b53011b18dd in_seq=0 out_seq=0 2024-05-27T07:37:50.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.684+0000 7f17d9600700 10 client.25810 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:50.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.686+0000 7f17d9600700 1 -- 172.21.15.63:0/2585292608 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55785a10ce10 con 0x55785a1f6d10 2024-05-27T07:37:50.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.686+0000 7f17d9600700 10 client.25810 handle_command_reply: tid=0 2024-05-27T07:37:50.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 client.25810 shutdown 2024-05-27T07:37:50.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 -- 172.21.15.63:0/2585292608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a149a70 msgr2=0x55785a1e6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:50.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a149a70 0x55785a1e6ca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f17e00098e0 tx=0x7f17e0005af0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 -- 172.21.15.63:0/2585292608 shutdown_connections 2024-05-27T07:37:50.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55785a1f6d10 0x55785a1f9160 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a1dcef0 0x55785a1e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a05dec0 0x55785a1e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2585292608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a149a70 0x55785a1e6ca0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 -- 172.21.15.63:0/2585292608 >> 172.21.15.63:0/2585292608 conn(0x55785a0f7410 msgr2=0x55785a1dc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:50.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f17d9600700 0 client.25810 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:50.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 -- 172.21.15.63:0/2585292608 shutdown_connections 2024-05-27T07:37:50.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 1 -- 172.21.15.63:0/2585292608 wait complete. 2024-05-27T07:37:50.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.687+0000 7f1804b1db80 20 client.25810 trim_cache size 0 max 16384 2024-05-27T07:37:50.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:50 smithi086 ceph-mon[34687]: pgmap v131: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:37:50.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:50 smithi149 ceph-mon[35408]: pgmap v131: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:37:50.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.713+0000 7f1804b1db80 1 Processor -- start 2024-05-27T07:37:50.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.713+0000 7f1804b1db80 1 -- start start 2024-05-27T07:37:50.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.713+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a05dec0 0x55785a1e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.713+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.713+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 0x55785a1fe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.713+0000 7f1804b1db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0011550 con 0x55785a149a70 2024-05-27T07:37:50.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.713+0000 7f1804b1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17d000ed80 con 0x55785a1dcef0 2024-05-27T07:37:50.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.713+0000 7f1804b1db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f17d000cbd0 con 0x55785a05dec0 2024-05-27T07:37:50.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.714+0000 7f17ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.714+0000 7f17ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1e5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54150/0 (socket says 172.21.15.63:54150) 2024-05-27T07:37:50.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.714+0000 7f17ef400700 1 -- 172.21.15.63:0/2510732643 learned_addr learned my addr 172.21.15.63:0/2510732643 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:50.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.714+0000 7f17eea00700 1 --2- 172.21.15.63:0/2510732643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a05dec0 0x55785a1e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.714+0000 7f17eea00700 1 --2- 172.21.15.63:0/2510732643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 0x55785a1fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.714+0000 7f17ef400700 1 -- 172.21.15.63:0/2510732643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a05dec0 msgr2=0x55785a1e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:50.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.714+0000 7f17ef400700 1 --2- 172.21.15.63:0/2510732643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a05dec0 0x55785a1e1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.714+0000 7f17ef400700 1 -- 172.21.15.63:0/2510732643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 msgr2=0x55785a1fe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:50.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.714+0000 7f17ef400700 1 --2- 172.21.15.63:0/2510732643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 0x55785a1fe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.714+0000 7f17ef400700 1 -- 172.21.15.63:0/2510732643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55785a1d9080 con 0x55785a149a70 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.715+0000 7f17eea00700 1 --2- 172.21.15.63:0/2510732643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 0x55785a1fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.715+0000 7f17ef400700 1 --2- 172.21.15.63:0/2510732643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1e5760 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f17e0023e10 tx=0x7f17e0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e1d3218c538215 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.715+0000 7f17dbe00700 1 -- 172.21.15.63:0/2510732643 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0031030 con 0x55785a149a70 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.715+0000 7f17dbe00700 1 -- 172.21.15.63:0/2510732643 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17e0028070 con 0x55785a149a70 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.715+0000 7f17dbe00700 1 -- 172.21.15.63:0/2510732643 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0023470 con 0x55785a149a70 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.716+0000 7f1804b1db80 1 -- 172.21.15.63:0/2510732643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 msgr2=0x55785a1e5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.716+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2510732643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1e5760 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f17e0023e10 tx=0x7f17e0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.716+0000 7f1804b1db80 1 -- 172.21.15.63:0/2510732643 shutdown_connections 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.716+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2510732643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 0x55785a1fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.716+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2510732643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a05dec0 0x55785a1e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.717+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2510732643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1e5760 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.717+0000 7f1804b1db80 1 -- 172.21.15.63:0/2510732643 >> 172.21.15.63:0/2510732643 conn(0x55785a0f7410 msgr2=0x55785a1e5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:50.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.717+0000 7f1804b1db80 1 -- 172.21.15.63:0/2510732643 shutdown_connections 2024-05-27T07:37:50.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.717+0000 7f1804b1db80 1 -- 172.21.15.63:0/2510732643 wait complete. 2024-05-27T07:37:50.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.718+0000 7f1804b1db80 1 -- start start 2024-05-27T07:37:50.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.718+0000 7f1804b1db80 1 Processor -- start 2024-05-27T07:37:50.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a05dec0 0x55785a1eb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1eb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f1804b1db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 0x55785a2607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f1804b1db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17d000d240 con 0x55785a149a70 2024-05-27T07:37:50.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f1804b1db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17d000e6b0 con 0x55785a1dcef0 2024-05-27T07:37:50.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f1804b1db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f17d000d910 con 0x55785a05dec0 2024-05-27T07:37:50.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f1804b1db80 10 client.0 _ll_register_callbacks cb 0x55785a0a4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:50.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f17ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1eb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f17ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1eb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54166/0 (socket says 172.21.15.63:54166) 2024-05-27T07:37:50.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f17ef400700 1 -- 172.21.15.63:0/2466152385 learned_addr learned my addr 172.21.15.63:0/2466152385 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:50.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.719+0000 7f17eea00700 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a05dec0 0x55785a1eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.720+0000 7f17eea00700 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 0x55785a2607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.720+0000 7f17ef400700 1 -- 172.21.15.63:0/2466152385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a05dec0 msgr2=0x55785a1eb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:50.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.720+0000 7f17ef400700 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a05dec0 0x55785a1eb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.720+0000 7f17ef400700 1 -- 172.21.15.63:0/2466152385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 msgr2=0x55785a2607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:50.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.720+0000 7f17ef400700 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 0x55785a2607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.720+0000 7f17ef400700 1 -- 172.21.15.63:0/2466152385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785a260d30 con 0x55785a149a70 2024-05-27T07:37:50.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.721+0000 7f17ef400700 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1eb8a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f17e000bbc0 tx=0x7f17e000bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa1163300efe414 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:50.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.721+0000 7f17d9600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:50.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.721+0000 7f17d9600700 1 -- 172.21.15.63:0/2466152385 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0031040 con 0x55785a149a70 2024-05-27T07:37:50.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.721+0000 7f1804b1db80 1 -- 172.21.15.63:0/2466152385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e0028070 con 0x55785a149a70 2024-05-27T07:37:50.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.721+0000 7f17d9600700 1 -- 172.21.15.63:0/2466152385 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f17e0023670 con 0x55785a149a70 2024-05-27T07:37:50.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.722+0000 7f17d9600700 1 -- 172.21.15.63:0/2466152385 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f17e004e050 con 0x55785a149a70 2024-05-27T07:37:50.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.722+0000 7f1804b1db80 10 client.16251 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:50.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.722+0000 7f1804b1db80 1 -- 172.21.15.63:0/2466152385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55785a261ef0 con 0x55785a149a70 2024-05-27T07:37:50.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.722+0000 7f17d9600700 1 -- 172.21.15.63:0/2466152385 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17e0028070 con 0x55785a149a70 2024-05-27T07:37:50.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.722+0000 7f17d9600700 1 -- 172.21.15.63:0/2466152385 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f17e0036930 con 0x55785a149a70 2024-05-27T07:37:50.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.722+0000 7f1804b1db80 10 client.16251 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:50.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.722+0000 7f1804b1db80 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:50.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.722+0000 7f1804b1db80 20 client.16251 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:50.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.722+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55785a2625a0 0x55785a2649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:50.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.723+0000 7f1804b1db80 4 client.16251 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:37:50.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.723+0000 7f1804b1db80 1 -- 172.21.15.63:0/2466152385 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55785a1065f0 con 0x55785a2625a0 2024-05-27T07:37:50.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.723+0000 7f17eea00700 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55785a2625a0 0x55785a2649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:50.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.724+0000 7f17eea00700 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55785a2625a0 0x55785a2649c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f342ba4176ba64 server_cookie=e5df273ca3990aea in_seq=0 out_seq=0 2024-05-27T07:37:50.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.724+0000 7f17d9600700 10 client.16251 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:50.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.726+0000 7f17d9600700 1 -- 172.21.15.63:0/2466152385 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55785a1065f0 con 0x55785a2625a0 2024-05-27T07:37:50.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.727+0000 7f17d9600700 10 client.16251 handle_command_reply: tid=0 2024-05-27T07:37:50.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.727+0000 7f1804b1db80 1 client.16251 shutdown 2024-05-27T07:37:50.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.727+0000 7f1804b1db80 1 -- 172.21.15.63:0/2466152385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 msgr2=0x55785a1eb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:50.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.727+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1eb8a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f17e000bbc0 tx=0x7f17e000bbf0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.727+0000 7f1804b1db80 1 -- 172.21.15.63:0/2466152385 shutdown_connections 2024-05-27T07:37:50.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.727+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55785a2625a0 0x55785a2649c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.727+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55785a1dcef0 0x55785a2607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.727+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55785a05dec0 0x55785a1eb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.728+0000 7f17d9600700 0 client.16251 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:50.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.727+0000 7f1804b1db80 1 --2- 172.21.15.63:0/2466152385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55785a149a70 0x55785a1eb8a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.728+0000 7f1804b1db80 1 -- 172.21.15.63:0/2466152385 >> 172.21.15.63:0/2466152385 conn(0x55785a0f7410 msgr2=0x55785a1e57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:50.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.728+0000 7f1804b1db80 1 -- 172.21.15.63:0/2466152385 shutdown_connections 2024-05-27T07:37:50.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.728+0000 7f1804b1db80 1 -- 172.21.15.63:0/2466152385 wait complete. 2024-05-27T07:37:50.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.728+0000 7f1804b1db80 20 client.16251 trim_cache size 0 max 16384 2024-05-27T07:37:50.726 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:37:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:37:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "7cad8cad-e7b1-49e1-83d4-3107960b86f2", 2024-05-27T07:37:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:37:50.726 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:37:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.729+0000 7f17f6000700 1 -- 172.21.15.63:0/3223830882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f17d40b29f0 msgr2=0x7f17d40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.729+0000 7f17f6000700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f17d40b29f0 0x7f17d40b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f17e80a8c20 tx=0x7f17e80a4ec0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.729+0000 7f17f6000700 1 -- 172.21.15.63:0/3223830882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f010c5a0 msgr2=0x7f17f0111230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.729+0000 7f17f6000700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f010c5a0 0x7f17f0111230 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f17e8007ee0 tx=0x7f17e800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.729+0000 7f17f6000700 1 -- 172.21.15.63:0/3223830882 shutdown_connections 2024-05-27T07:37:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.729+0000 7f17f6000700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f17d40b29f0 0x7f17d40b4f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.729+0000 7f17f6000700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f010c5a0 0x7f17f0111230 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.729+0000 7f17f6000700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17f0106560 0x7f17f01107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.729+0000 7f17f6000700 1 --2- 172.21.15.63:0/3223830882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17f0106eb0 0x7f17f0110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:50.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.729+0000 7f17f6000700 1 -- 172.21.15.63:0/3223830882 >> 172.21.15.63:0/3223830882 conn(0x7f17f0102040 msgr2=0x7f17f01039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:50.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.730+0000 7f17f6000700 1 -- 172.21.15.63:0/3223830882 shutdown_connections 2024-05-27T07:37:50.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:50.730+0000 7f17f6000700 1 -- 172.21.15.63:0/3223830882 wait complete. 2024-05-27T07:37:51.720 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:37:51.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:51 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:37:51.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:51 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:37:51.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:51 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:37:51.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:51 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:37:51.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:51 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:37:51.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:51 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:37:51.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:51 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:37:51.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:51 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:37:51.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:51 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:37:51.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:51 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:37:51.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:51 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:37:51.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:51 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:37:51.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:51 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:37:51.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:51 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:37:51.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:51 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:37:51.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:51.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:52.223 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:37:52.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:52 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:37:52.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:52 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:52.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:52 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:37:52.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:52 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:52.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:52 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:37:52.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:52 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:52.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:52 smithi063 ceph-mon[31613]: pgmap v132: 129 pgs: 129 active+clean; 469 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.7 MiB/s wr, 39 op/s 2024-05-27T07:37:52.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:52 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:37:52.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:52 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:52.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:52 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:37:52.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:52 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:52.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:52 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:37:52.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:52 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:52.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:52 smithi086 ceph-mon[34687]: pgmap v132: 129 pgs: 129 active+clean; 469 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.7 MiB/s wr, 39 op/s 2024-05-27T07:37:52.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:52 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:37:52.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:52 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:52.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:52 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:37:52.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:52 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:52.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:52 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:37:52.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:52 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:37:52.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:52 smithi149 ceph-mon[35408]: pgmap v132: 129 pgs: 129 active+clean; 469 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.7 MiB/s wr, 39 op/s 2024-05-27T07:37:54.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.806+0000 7f8933c00700 1 -- 172.21.15.63:0/677547217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f892c1030f0 msgr2=0x7f892c103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.806+0000 7f8933c00700 1 --2- 172.21.15.63:0/677547217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f892c1030f0 0x7f892c103500 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8920009990 tx=0x7f8920009ca0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.806+0000 7f8933c00700 1 -- 172.21.15.63:0/677547217 shutdown_connections 2024-05-27T07:37:54.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.806+0000 7f8933c00700 1 --2- 172.21.15.63:0/677547217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c110ca0 0x7f892c1130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.806+0000 7f8933c00700 1 --2- 172.21.15.63:0/677547217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f892c1030f0 0x7f892c103500 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.806+0000 7f8933c00700 1 --2- 172.21.15.63:0/677547217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f892c103a40 0x7f892c110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.806+0000 7f8933c00700 1 -- 172.21.15.63:0/677547217 >> 172.21.15.63:0/677547217 conn(0x7f892c0fedb0 msgr2=0x7f892c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:54.804 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:54 smithi063 ceph-mon[31613]: pgmap v133: 129 pgs: 129 active+clean; 469 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.3 MiB/s wr, 35 op/s 2024-05-27T07:37:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.806+0000 7f8933c00700 1 -- 172.21.15.63:0/677547217 shutdown_connections 2024-05-27T07:37:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.807+0000 7f8933c00700 1 -- 172.21.15.63:0/677547217 wait complete. 2024-05-27T07:37:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.808+0000 7f8933c00700 1 Processor -- start 2024-05-27T07:37:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.808+0000 7f8933c00700 1 -- start start 2024-05-27T07:37:54.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f8933c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c1030f0 0x7f892c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f8933c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f892c103a40 0x7f892c19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f8933c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f892c110ca0 0x7f892c1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f8933c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f892c116550 con 0x7f892c103a40 2024-05-27T07:37:54.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f8933c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f892c1163d0 con 0x7f892c1030f0 2024-05-27T07:37:54.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f8933c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f892c1166d0 con 0x7f892c110ca0 2024-05-27T07:37:54.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f892be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f892c103a40 0x7f892c19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f892be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f892c103a40 0x7f892c19d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54192/0 (socket says 172.21.15.63:54192) 2024-05-27T07:37:54.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f892be00700 1 -- 172.21.15.63:0/3347180142 learned_addr learned my addr 172.21.15.63:0/3347180142 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:54.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f8931200700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c1030f0 0x7f892c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.809+0000 7f8931c00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f892c110ca0 0x7f892c1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.810+0000 7f892be00700 1 -- 172.21.15.63:0/3347180142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f892c110ca0 msgr2=0x7f892c1a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.810+0000 7f892be00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f892c110ca0 0x7f892c1a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.810+0000 7f892be00700 1 -- 172.21.15.63:0/3347180142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c1030f0 msgr2=0x7f892c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.810+0000 7f892be00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c1030f0 0x7f892c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.810+0000 7f892be00700 1 -- 172.21.15.63:0/3347180142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c1a1e50 con 0x7f892c103a40 2024-05-27T07:37:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.810+0000 7f892be00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f892c103a40 0x7f892c19d520 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f891800d980 tx=0x7f891800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.811+0000 7f8929600700 1 -- 172.21.15.63:0/3347180142 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918016070 con 0x7f892c103a40 2024-05-27T07:37:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.811+0000 7f8929600700 1 -- 172.21.15.63:0/3347180142 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8918010460 con 0x7f892c103a40 2024-05-27T07:37:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.811+0000 7f8933c00700 1 -- 172.21.15.63:0/3347180142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c1a2140 con 0x7f892c103a40 2024-05-27T07:37:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.811+0000 7f8933c00700 1 -- 172.21.15.63:0/3347180142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c1a9a60 con 0x7f892c103a40 2024-05-27T07:37:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.811+0000 7f8929600700 1 -- 172.21.15.63:0/3347180142 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891800de60 con 0x7f892c103a40 2024-05-27T07:37:54.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.816+0000 7f8929600700 1 -- 172.21.15.63:0/3347180142 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8918019070 con 0x7f892c103a40 2024-05-27T07:37:54.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.817+0000 7f8942690b80 1 Processor -- start 2024-05-27T07:37:54.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.817+0000 7f8942690b80 1 -- start start 2024-05-27T07:37:54.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.818+0000 7f8942690b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682ee67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.818+0000 7f8942690b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee6d30 0x55b682ee0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.818+0000 7f8942690b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee0db0 0x55b682ee11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.818+0000 7f8942690b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b682e0ee40 con 0x55b682ee6d30 2024-05-27T07:37:54.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.818+0000 7f8942690b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b682dfe9e0 con 0x55b682ee0db0 2024-05-27T07:37:54.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.818+0000 7f8942690b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b682e02a20 con 0x55b682d61e40 2024-05-27T07:37:54.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.818+0000 7f8931200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee6d30 0x55b682ee0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.818+0000 7f8931200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee6d30 0x55b682ee0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54194/0 (socket says 172.21.15.63:54194) 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.818+0000 7f8931200700 1 -- 172.21.15.63:0/3700996692 learned_addr learned my addr 172.21.15.63:0/3700996692 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.818+0000 7f8929600700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f89140b29f0 0x7f89140b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.819+0000 7f8931c00700 1 --2- 172.21.15.63:0/3700996692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682ee67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.819+0000 7f8931c00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f89140b29f0 0x7f89140b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.819+0000 7f8929600700 1 -- 172.21.15.63:0/3347180142 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f89180e2500 con 0x7f892c103a40 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.819+0000 7f8931200700 1 -- 172.21.15.63:0/3700996692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 msgr2=0x55b682ee67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.819+0000 7f8931200700 1 --2- 172.21.15.63:0/3700996692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682ee67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.819+0000 7f8931200700 1 -- 172.21.15.63:0/3700996692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee0db0 msgr2=0x55b682ee11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.819+0000 7f892be00700 1 --2- 172.21.15.63:0/3700996692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee0db0 0x55b682ee11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.819+0000 7f8931200700 1 --2- 172.21.15.63:0/3700996692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee0db0 0x55b682ee11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.819+0000 7f8931200700 1 -- 172.21.15.63:0/3700996692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b682dc7420 con 0x55b682ee6d30 2024-05-27T07:37:54.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.819+0000 7f892be00700 1 --2- 172.21.15.63:0/3700996692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee0db0 0x55b682ee11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:54.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.820+0000 7f8931200700 1 --2- 172.21.15.63:0/3700996692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee6d30 0x55b682ee0870 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f89200096a0 tx=0x7f89200096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b99a302f90780d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:54.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.820+0000 7f891e000700 1 -- 172.21.15.63:0/3700996692 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920027070 con 0x55b682ee6d30 2024-05-27T07:37:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.820+0000 7f891e000700 1 -- 172.21.15.63:0/3700996692 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8920004ce0 con 0x55b682ee6d30 2024-05-27T07:37:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.820+0000 7f891e000700 1 -- 172.21.15.63:0/3700996692 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892000b7f0 con 0x55b682ee6d30 2024-05-27T07:37:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.821+0000 7f8942690b80 1 -- 172.21.15.63:0/3700996692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee6d30 msgr2=0x55b682ee0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.821+0000 7f8942690b80 1 --2- 172.21.15.63:0/3700996692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee6d30 0x55b682ee0870 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f89200096a0 tx=0x7f89200096d0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.821+0000 7f8931c00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f89140b29f0 0x7f89140b4f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55b682ee1b30 tx=0x7f892400b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.822+0000 7f8942690b80 1 -- 172.21.15.63:0/3700996692 shutdown_connections 2024-05-27T07:37:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.822+0000 7f8942690b80 1 --2- 172.21.15.63:0/3700996692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee0db0 0x55b682ee11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.822+0000 7f8942690b80 1 --2- 172.21.15.63:0/3700996692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682ee67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.822+0000 7f8942690b80 1 --2- 172.21.15.63:0/3700996692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee6d30 0x55b682ee0870 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.822+0000 7f8942690b80 1 -- 172.21.15.63:0/3700996692 >> 172.21.15.63:0/3700996692 conn(0x55b682dc1fc0 msgr2=0x55b682edb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.822+0000 7f8942690b80 1 -- 172.21.15.63:0/3700996692 shutdown_connections 2024-05-27T07:37:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.822+0000 7f8942690b80 1 -- 172.21.15.63:0/3700996692 wait complete. 2024-05-27T07:37:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.822+0000 7f8942690b80 1 -- start start 2024-05-27T07:37:54.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.823+0000 7f8942690b80 1 Processor -- start 2024-05-27T07:37:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.823+0000 7f8942690b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682eea520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.823+0000 7f8942690b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682eeaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.823+0000 7f8942690b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee6d30 0x55b682eeafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.824+0000 7f8942690b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b682d04fa0 con 0x55b682ee0db0 2024-05-27T07:37:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.824+0000 7f8942690b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b682d9f760 con 0x55b682ee6d30 2024-05-27T07:37:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.824+0000 7f8942690b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b682eed7a0 con 0x55b682d61e40 2024-05-27T07:37:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.824+0000 7f8942690b80 10 client.0 _ll_register_callbacks cb 0x55b682da8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.824+0000 7f892be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682eeaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.824+0000 7f892be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682eeaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54196/0 (socket says 172.21.15.63:54196) 2024-05-27T07:37:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.824+0000 7f892be00700 1 -- 172.21.15.63:0/693503306 learned_addr learned my addr 172.21.15.63:0/693503306 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.824+0000 7f8931c00700 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee6d30 0x55b682eeafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.824+0000 7f8931200700 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682eea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.825+0000 7f892be00700 1 -- 172.21.15.63:0/693503306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 msgr2=0x55b682eea520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.825+0000 7f892be00700 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682eea520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.825+0000 7f892be00700 1 -- 172.21.15.63:0/693503306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee6d30 msgr2=0x55b682eeafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.825+0000 7f892be00700 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee6d30 0x55b682eeafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.825+0000 7f892be00700 1 -- 172.21.15.63:0/693503306 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b682d4e8d0 con 0x55b682ee0db0 2024-05-27T07:37:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.825+0000 7f892be00700 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682eeaa60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f89180a9170 tx=0x7f8918010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3afa9aa149c72c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.825+0000 7f8912a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.825+0000 7f8912a00700 1 -- 172.21.15.63:0/693503306 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89180ad820 con 0x55b682ee0db0 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.825+0000 7f8912a00700 1 -- 172.21.15.63:0/693503306 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f89180a5700 con 0x55b682ee0db0 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.825+0000 7f8942690b80 1 -- 172.21.15.63:0/693503306 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8920004ce0 con 0x55b682ee0db0 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8912a00700 1 -- 172.21.15.63:0/693503306 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f89180b22d0 con 0x55b682ee0db0 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8942690b80 10 client.16263 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8942690b80 1 -- 172.21.15.63:0/693503306 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b682dcf060 con 0x55b682ee0db0 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8912a00700 1 -- 172.21.15.63:0/693503306 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8920004ce0 con 0x55b682ee0db0 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8912a00700 1 -- 172.21.15.63:0/693503306 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f89180a96d0 con 0x55b682ee0db0 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8942690b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8942690b80 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8942690b80 20 client.16263 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8942690b80 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b682efa670 0x55b682efca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8942690b80 4 client.16263 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:54.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.826+0000 7f8942690b80 1 -- 172.21.15.63:0/693503306 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b682e10d90 con 0x55b682efa670 2024-05-27T07:37:54.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.827+0000 7f8931200700 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b682efa670 0x55b682efca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.828+0000 7f8912a00700 10 client.16263 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:54.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.828+0000 7f8931200700 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b682efa670 0x55b682efca90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31c2873eed710b8d server_cookie=667d7d7fd7834a9 in_seq=0 out_seq=0 2024-05-27T07:37:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.830+0000 7f8912a00700 1 -- 172.21.15.63:0/693503306 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b682e10d90 con 0x55b682efa670 2024-05-27T07:37:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.830+0000 7f8912a00700 10 client.16263 handle_command_reply: tid=0 2024-05-27T07:37:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.830+0000 7f8942690b80 1 client.16263 shutdown 2024-05-27T07:37:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 1 -- 172.21.15.63:0/693503306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 msgr2=0x55b682eeaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682eeaa60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f89180a9170 tx=0x7f8918010b40 comp rx=0 tx=0).stop 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 1 -- 172.21.15.63:0/693503306 shutdown_connections 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b682efa670 0x55b682efca90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee6d30 0x55b682eeafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682eea520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 1 --2- 172.21.15.63:0/693503306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682eeaa60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 1 -- 172.21.15.63:0/693503306 >> 172.21.15.63:0/693503306 conn(0x55b682dc1fc0 msgr2=0x55b682edb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8912a00700 0 client.16263 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 1 -- 172.21.15.63:0/693503306 shutdown_connections 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 1 -- 172.21.15.63:0/693503306 wait complete. 2024-05-27T07:37:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.831+0000 7f8942690b80 20 client.16263 trim_cache size 0 max 16384 2024-05-27T07:37:54.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.857+0000 7f8942690b80 1 Processor -- start 2024-05-27T07:37:54.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.857+0000 7f8942690b80 1 -- start start 2024-05-27T07:37:54.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.857+0000 7f8942690b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682ee0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.857+0000 7f8942690b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682ee9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.857+0000 7f8942690b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee6d30 0x55b682ee4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.857+0000 7f8942690b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f892c1b4ef0 con 0x55b682ee0db0 2024-05-27T07:37:54.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.857+0000 7f8942690b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f892c1b2720 con 0x55b682ee6d30 2024-05-27T07:37:54.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.857+0000 7f8942690b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f892c1b0570 con 0x55b682d61e40 2024-05-27T07:37:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.858+0000 7f892be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682ee9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.858+0000 7f892be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682ee9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54198/0 (socket says 172.21.15.63:54198) 2024-05-27T07:37:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.858+0000 7f892be00700 1 -- 172.21.15.63:0/4168392475 learned_addr learned my addr 172.21.15.63:0/4168392475 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.858+0000 7f8931c00700 1 --2- 172.21.15.63:0/4168392475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee6d30 0x55b682ee4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.858+0000 7f8931200700 1 --2- 172.21.15.63:0/4168392475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682ee0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.858+0000 7f892be00700 1 -- 172.21.15.63:0/4168392475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 msgr2=0x55b682ee0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.858+0000 7f892be00700 1 --2- 172.21.15.63:0/4168392475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682ee0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.858+0000 7f892be00700 1 -- 172.21.15.63:0/4168392475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee6d30 msgr2=0x55b682ee4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.858+0000 7f892be00700 1 --2- 172.21.15.63:0/4168392475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee6d30 0x55b682ee4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.859+0000 7f892be00700 1 -- 172.21.15.63:0/4168392475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b682ee8ab0 con 0x55b682ee0db0 2024-05-27T07:37:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.859+0000 7f8931200700 1 --2- 172.21.15.63:0/4168392475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682ee0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.859+0000 7f892be00700 1 --2- 172.21.15.63:0/4168392475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682ee9500 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f89180a9400 tx=0x7f891800d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d75558fceb5348f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.859+0000 7f891d600700 1 -- 172.21.15.63:0/4168392475 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89180a9240 con 0x55b682ee0db0 2024-05-27T07:37:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.859+0000 7f891d600700 1 -- 172.21.15.63:0/4168392475 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89180e2150 con 0x55b682ee0db0 2024-05-27T07:37:54.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.859+0000 7f891d600700 1 -- 172.21.15.63:0/4168392475 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89180a3be0 con 0x55b682ee0db0 2024-05-27T07:37:54.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.860+0000 7f8942690b80 1 -- 172.21.15.63:0/4168392475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 msgr2=0x55b682ee9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.860+0000 7f8942690b80 1 --2- 172.21.15.63:0/4168392475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682ee9500 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f89180a9400 tx=0x7f891800d910 comp rx=0 tx=0).stop 2024-05-27T07:37:54.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.860+0000 7f8942690b80 1 -- 172.21.15.63:0/4168392475 shutdown_connections 2024-05-27T07:37:54.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.860+0000 7f8942690b80 1 --2- 172.21.15.63:0/4168392475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682ee6d30 0x55b682ee4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.860+0000 7f8942690b80 1 --2- 172.21.15.63:0/4168392475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682d61e40 0x55b682ee0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.860+0000 7f8942690b80 1 --2- 172.21.15.63:0/4168392475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682ee9500 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.860+0000 7f8942690b80 1 -- 172.21.15.63:0/4168392475 >> 172.21.15.63:0/4168392475 conn(0x55b682dc1fc0 msgr2=0x55b682eea300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.861+0000 7f8942690b80 1 -- 172.21.15.63:0/4168392475 shutdown_connections 2024-05-27T07:37:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.861+0000 7f8942690b80 1 -- 172.21.15.63:0/4168392475 wait complete. 2024-05-27T07:37:54.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.861+0000 7f8942690b80 1 -- start start 2024-05-27T07:37:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.861+0000 7f8942690b80 1 Processor -- start 2024-05-27T07:37:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.862+0000 7f8942690b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682d61e40 0x55b682ef2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.862+0000 7f8942690b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682f01820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.862+0000 7f8942690b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682ee6d30 0x55b682f01d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.862+0000 7f8942690b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f892c1b0be0 con 0x55b682ee0db0 2024-05-27T07:37:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.862+0000 7f8942690b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f892c1b2050 con 0x55b682d61e40 2024-05-27T07:37:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.862+0000 7f8942690b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f892c1b12b0 con 0x55b682ee6d30 2024-05-27T07:37:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.862+0000 7f8942690b80 10 client.0 _ll_register_callbacks cb 0x55b682da8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f892be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682f01820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f892be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682f01820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54214/0 (socket says 172.21.15.63:54214) 2024-05-27T07:37:54.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f892be00700 1 -- 172.21.15.63:0/1709543963 learned_addr learned my addr 172.21.15.63:0/1709543963 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:54.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f8931200700 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682d61e40 0x55b682ef2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f8931c00700 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682ee6d30 0x55b682f01d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f892be00700 1 -- 172.21.15.63:0/1709543963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682ee6d30 msgr2=0x55b682f01d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f892be00700 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682ee6d30 0x55b682f01d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f892be00700 1 -- 172.21.15.63:0/1709543963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682d61e40 msgr2=0x55b682ef2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:54.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f892be00700 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682d61e40 0x55b682ef2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f892be00700 1 -- 172.21.15.63:0/1709543963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b682f022a0 con 0x55b682ee0db0 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.863+0000 7f892be00700 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682f01820 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f89180df990 tx=0x7f89180a5910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9170fe175cb1da06 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.864+0000 7f8912a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.864+0000 7f8912a00700 1 -- 172.21.15.63:0/1709543963 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89180a5330 con 0x55b682ee0db0 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.864+0000 7f8912a00700 1 -- 172.21.15.63:0/1709543963 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f89180a3df0 con 0x55b682ee0db0 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.864+0000 7f8942690b80 1 -- 172.21.15.63:0/1709543963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89180e2150 con 0x55b682ee0db0 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.864+0000 7f8912a00700 1 -- 172.21.15.63:0/1709543963 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f89180ad820 con 0x55b682ee0db0 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.864+0000 7f8942690b80 10 client.16269 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.864+0000 7f8942690b80 1 -- 172.21.15.63:0/1709543963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b682f022a0 con 0x55b682ee0db0 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.865+0000 7f8912a00700 1 -- 172.21.15.63:0/1709543963 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89180e2150 con 0x55b682ee0db0 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.865+0000 7f8912a00700 1 -- 172.21.15.63:0/1709543963 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f89180ca030 con 0x55b682ee0db0 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.865+0000 7f8942690b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.865+0000 7f8942690b80 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.865+0000 7f8942690b80 20 client.16269 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.865+0000 7f8942690b80 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b682f64f20 0x55b682f67340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.865+0000 7f8942690b80 4 client.16269 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:37:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.865+0000 7f8942690b80 1 -- 172.21.15.63:0/1709543963 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b682e0a570 con 0x55b682f64f20 2024-05-27T07:37:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.866+0000 7f8931200700 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b682f64f20 0x55b682f67340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.867+0000 7f8931200700 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b682f64f20 0x55b682f67340 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8b61bd9d5043f2 server_cookie=33a5ff825fbc5b80 in_seq=0 out_seq=0 2024-05-27T07:37:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.867+0000 7f8912a00700 10 client.16269 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.867+0000 7f8912a00700 1 -- 172.21.15.63:0/1709543963 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b682e0a570 con 0x55b682f64f20 2024-05-27T07:37:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.867+0000 7f8912a00700 10 client.16269 handle_command_reply: tid=0 2024-05-27T07:37:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.867+0000 7f8942690b80 1 client.16269 shutdown 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 1 -- 172.21.15.63:0/1709543963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 msgr2=0x55b682f01820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682f01820 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f89180df990 tx=0x7f89180a5910 comp rx=0 tx=0).stop 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 1 -- 172.21.15.63:0/1709543963 shutdown_connections 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b682f64f20 0x55b682f67340 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b682ee6d30 0x55b682f01d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b682d61e40 0x55b682ef2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 1 --2- 172.21.15.63:0/1709543963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b682ee0db0 0x55b682f01820 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 1 -- 172.21.15.63:0/1709543963 >> 172.21.15.63:0/1709543963 conn(0x55b682dc1fc0 msgr2=0x55b682eead30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8912a00700 0 client.16269 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 1 -- 172.21.15.63:0/1709543963 shutdown_connections 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 1 -- 172.21.15.63:0/1709543963 wait complete. 2024-05-27T07:37:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.868+0000 7f8942690b80 20 client.16269 trim_cache size 0 max 16384 2024-05-27T07:37:54.867 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:37:54.867 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:37:54.867 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:37:54.867 INFO:teuthology.orchestra.run.smithi063.stdout: "7cad8cad-e7b1-49e1-83d4-3107960b86f2": { 2024-05-27T07:37:54.867 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:37:54.867 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "7cad8cad-e7b1-49e1-83d4-3107960b86f2", 2024-05-27T07:37:54.867 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:37:54.868 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:37:54.868 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:37:54.868 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:37:54.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 -- 172.21.15.63:0/3347180142 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f89140b29f0 msgr2=0x7f89140b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f89140b29f0 0x7f89140b4f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55b682ee1b30 tx=0x7f892400b040 comp rx=0 tx=0).stop 2024-05-27T07:37:54.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 -- 172.21.15.63:0/3347180142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f892c103a40 msgr2=0x7f892c19d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f892c103a40 0x7f892c19d520 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f891800d980 tx=0x7f891800dd40 comp rx=0 tx=0).stop 2024-05-27T07:37:54.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 -- 172.21.15.63:0/3347180142 shutdown_connections 2024-05-27T07:37:54.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f89140b29f0 0x7f89140b4f00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f892c110ca0 0x7f892c1a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f892c1030f0 0x7f892c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 --2- 172.21.15.63:0/3347180142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f892c103a40 0x7f892c19d520 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 -- 172.21.15.63:0/3347180142 >> 172.21.15.63:0/3347180142 conn(0x7f892c0fedb0 msgr2=0x7f892c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:54.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.869+0000 7f8933c00700 1 -- 172.21.15.63:0/3347180142 shutdown_connections 2024-05-27T07:37:54.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:37:54.870+0000 7f8933c00700 1 -- 172.21.15.63:0/3347180142 wait complete. 2024-05-27T07:37:54.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:54 smithi086 ceph-mon[34687]: pgmap v133: 129 pgs: 129 active+clean; 469 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.3 MiB/s wr, 35 op/s 2024-05-27T07:37:54.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:54 smithi149 ceph-mon[35408]: pgmap v133: 129 pgs: 129 active+clean; 469 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.3 MiB/s wr, 35 op/s 2024-05-27T07:37:55.875 INFO:tasks.cephfs.filesystem:scrub status for tag:7cad8cad-e7b1-49e1-83d4-3107960b86f2 - {'path': '/', 'tag': '7cad8cad-e7b1-49e1-83d4-3107960b86f2', 'options': 'recursive,force'} 2024-05-27T07:37:56.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:56 smithi086 ceph-mon[34687]: pgmap v134: 129 pgs: 129 active+clean; 469 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.3 MiB/s wr, 35 op/s 2024-05-27T07:37:56.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:56 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:37:56.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:56 smithi149 ceph-mon[35408]: pgmap v134: 129 pgs: 129 active+clean; 469 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.3 MiB/s wr, 35 op/s 2024-05-27T07:37:56.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:56 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:37:56.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:56 smithi063 ceph-mon[31613]: pgmap v134: 129 pgs: 129 active+clean; 469 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.3 MiB/s wr, 35 op/s 2024-05-27T07:37:56.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:56 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:37:57.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:57 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:37:57.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:57 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:37:57.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:57 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:37:57.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:57 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:37:57.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:57 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:37:57.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:57 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:37:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.640+0000 7f5be9e00640 1 -- 172.21.15.63:0/1243067212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be410e000 msgr2=0x7f5be41104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.640+0000 7f5be9e00640 1 --2- 172.21.15.63:0/1243067212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be410e000 0x7f5be41104f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009ca0 tx=0x7f5bdc02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.640+0000 7f5be9e00640 1 -- 172.21.15.63:0/1243067212 shutdown_connections 2024-05-27T07:37:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.640+0000 7f5be9e00640 1 --2- 172.21.15.63:0/1243067212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be410e000 0x7f5be41104f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.640+0000 7f5be9e00640 1 --2- 172.21.15.63:0/1243067212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be410b610 0x7f5be410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.640+0000 7f5be9e00640 1 --2- 172.21.15.63:0/1243067212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5be4104bb0 0x7f5be4104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.640+0000 7f5be9e00640 1 -- 172.21.15.63:0/1243067212 >> 172.21.15.63:0/1243067212 conn(0x7f5be40fe210 msgr2=0x7f5be4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.640+0000 7f5be9e00640 1 -- 172.21.15.63:0/1243067212 shutdown_connections 2024-05-27T07:37:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.640+0000 7f5be9e00640 1 -- 172.21.15.63:0/1243067212 wait complete. 2024-05-27T07:37:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.641+0000 7f5be9e00640 1 Processor -- start 2024-05-27T07:37:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.642+0000 7f5be9e00640 1 -- start start 2024-05-27T07:37:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.642+0000 7f5be9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be4104bb0 0x7f5be4102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.642+0000 7f5be9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5be410b610 0x7f5be41029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.642+0000 7f5be9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be410e000 0x7f5be4102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.642+0000 7f5be9e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4113b60 con 0x7f5be4104bb0 2024-05-27T07:37:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.642+0000 7f5be9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41139e0 con 0x7f5be410b610 2024-05-27T07:37:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.642+0000 7f5be9e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4113ce0 con 0x7f5be410e000 2024-05-27T07:37:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.642+0000 7f5be8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be4104bb0 0x7f5be4102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.642+0000 7f5be8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be4104bb0 0x7f5be4102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54242/0 (socket says 172.21.15.63:54242) 2024-05-27T07:37:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.642+0000 7f5be8a00640 1 -- 172.21.15.63:0/3601949533 learned_addr learned my addr 172.21.15.63:0/3601949533 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.643+0000 7f5be3e00640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5be410b610 0x7f5be41029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.643+0000 7f5be9400640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be410e000 0x7f5be4102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.643+0000 7f5be8a00640 1 -- 172.21.15.63:0/3601949533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be410e000 msgr2=0x7f5be4102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.643+0000 7f5be8a00640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be410e000 0x7f5be4102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.643+0000 7f5be8a00640 1 -- 172.21.15.63:0/3601949533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5be410b610 msgr2=0x7f5be41029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.643+0000 7f5be8a00640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5be410b610 0x7f5be41029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.643+0000 7f5be8a00640 1 -- 172.21.15.63:0/3601949533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be41a88e0 con 0x7f5be4104bb0 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.643+0000 7f5be8a00640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be4104bb0 0x7f5be4102470 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800d950 tx=0x7f5bd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.644+0000 7f5be1600640 1 -- 172.21.15.63:0/3601949533 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8017030 con 0x7f5be4104bb0 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.644+0000 7f5be1600640 1 -- 172.21.15.63:0/3601949533 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bd8004020 con 0x7f5be4104bb0 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.644+0000 7f5be1600640 1 -- 172.21.15.63:0/3601949533 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80055d0 con 0x7f5be4104bb0 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.644+0000 7f5be9e00640 1 -- 172.21.15.63:0/3601949533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be41a8bd0 con 0x7f5be4104bb0 2024-05-27T07:37:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.644+0000 7f5be9e00640 1 -- 172.21.15.63:0/3601949533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be41a9170 con 0x7f5be4104bb0 2024-05-27T07:37:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.648+0000 7f5bf256d740 1 Processor -- start 2024-05-27T07:37:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.649+0000 7f5bf256d740 1 -- start start 2024-05-27T07:37:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.649+0000 7f5be1600640 1 -- 172.21.15.63:0/3601949533 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5bd8009a30 con 0x7f5be4104bb0 2024-05-27T07:37:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.649+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e49a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.649+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e49fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.649+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e4a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.649+0000 7f5bf256d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b404dbab90 con 0x55b404a7e2f0 2024-05-27T07:37:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.649+0000 7f5bf256d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b404daa730 con 0x55b404aa71b0 2024-05-27T07:37:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.649+0000 7f5bf256d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b404dae770 con 0x55b404a864d0 2024-05-27T07:37:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.650+0000 7f5be8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e49fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.650+0000 7f5be8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e49fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54256/0 (socket says 172.21.15.63:54256) 2024-05-27T07:37:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.650+0000 7f5be8a00640 1 -- 172.21.15.63:0/940780690 learned_addr learned my addr 172.21.15.63:0/940780690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.650+0000 7f5be9400640 1 --2- 172.21.15.63:0/940780690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e49a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.650+0000 7f5be3e00640 1 --2- 172.21.15.63:0/940780690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e4a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.651+0000 7f5be8a00640 1 -- 172.21.15.63:0/940780690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 msgr2=0x55b404e4a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.651+0000 7f5be8a00640 1 --2- 172.21.15.63:0/940780690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e4a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.651+0000 7f5be8a00640 1 -- 172.21.15.63:0/940780690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 msgr2=0x55b404e49a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.651+0000 7f5be8a00640 1 --2- 172.21.15.63:0/940780690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e49a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.651+0000 7f5be8a00640 1 -- 172.21.15.63:0/940780690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b404d438b0 con 0x55b404a7e2f0 2024-05-27T07:37:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.651+0000 7f5be9400640 1 --2- 172.21.15.63:0/940780690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e49a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:37:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.651+0000 7f5be1600640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bcc0b29d0 0x7f5bcc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.651+0000 7f5be8a00640 1 --2- 172.21.15.63:0/940780690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e49fa0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80df710 tx=0x7f5bd80e88a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff84533812d9d011 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.652+0000 7f5be1600640 1 -- 172.21.15.63:0/3601949533 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f5bd80e23c0 con 0x7f5be4104bb0 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.652+0000 7f5bd5c00640 1 -- 172.21.15.63:0/940780690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80a51f0 con 0x55b404a7e2f0 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.652+0000 7f5bd5c00640 1 -- 172.21.15.63:0/940780690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bd80a8e40 con 0x55b404a7e2f0 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.652+0000 7f5bd5c00640 1 -- 172.21.15.63:0/940780690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80a8aa0 con 0x55b404a7e2f0 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.652+0000 7f5be9400640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bcc0b29d0 0x7f5bcc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.653+0000 7f5bf256d740 1 -- 172.21.15.63:0/940780690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 msgr2=0x55b404e49fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.653+0000 7f5bf256d740 1 --2- 172.21.15.63:0/940780690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e49fa0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80df710 tx=0x7f5bd80e88a0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.653+0000 7f5be9400640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bcc0b29d0 0x7f5bcc0b4ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005e30 tx=0x7f5bdc005d80 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.653+0000 7f5bf256d740 1 -- 172.21.15.63:0/940780690 shutdown_connections 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.653+0000 7f5bf256d740 1 --2- 172.21.15.63:0/940780690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e4a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.653+0000 7f5bf256d740 1 --2- 172.21.15.63:0/940780690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e49fa0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.653+0000 7f5bf256d740 1 --2- 172.21.15.63:0/940780690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e49a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.653+0000 7f5bf256d740 1 -- 172.21.15.63:0/940780690 >> 172.21.15.63:0/940780690 conn(0x55b404a9b300 msgr2=0x55b404e46210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.653+0000 7f5bf256d740 1 -- 172.21.15.63:0/940780690 shutdown_connections 2024-05-27T07:37:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.653+0000 7f5bf256d740 1 -- 172.21.15.63:0/940780690 wait complete. 2024-05-27T07:37:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.654+0000 7f5bf256d740 1 -- start start 2024-05-27T07:37:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.654+0000 7f5bf256d740 1 Processor -- start 2024-05-27T07:37:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.655+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e53c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.655+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.655+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e5fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.655+0000 7f5bf256d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b404a90790 con 0x55b404a7e2f0 2024-05-27T07:37:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.655+0000 7f5bf256d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b404d52bc0 con 0x55b404aa71b0 2024-05-27T07:37:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.655+0000 7f5bf256d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b404e56b90 con 0x55b404a864d0 2024-05-27T07:37:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.655+0000 7f5bf256d740 10 client.0 _ll_register_callbacks cb 0x55b404d30400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.655+0000 7f5be8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e53c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.655+0000 7f5be8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e53c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54266/0 (socket says 172.21.15.63:54266) 2024-05-27T07:37:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.655+0000 7f5be8a00640 1 -- 172.21.15.63:0/1822534470 learned_addr learned my addr 172.21.15.63:0/1822534470 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.656+0000 7f5be9400640 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e5fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.656+0000 7f5be8a00640 1 -- 172.21.15.63:0/1822534470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 msgr2=0x55b404e541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.656+0000 7f5be8a00640 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.656+0000 7f5be8a00640 1 -- 172.21.15.63:0/1822534470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 msgr2=0x55b404e5fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.656+0000 7f5be8a00640 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e5fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.656+0000 7f5be8a00640 1 -- 172.21.15.63:0/1822534470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b404abf210 con 0x55b404a7e2f0 2024-05-27T07:37:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.656+0000 7f5be8a00640 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e53c60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80a53b0 tx=0x7f5bd80e2730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa7596357202b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.656+0000 7f5bca000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.657+0000 7f5bca000640 1 -- 172.21.15.63:0/1822534470 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80a8c80 con 0x55b404a7e2f0 2024-05-27T07:37:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.657+0000 7f5bca000640 1 -- 172.21.15.63:0/1822534470 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f5bd80e2120 con 0x55b404a7e2f0 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.657+0000 7f5bf256d740 1 -- 172.21.15.63:0/1822534470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bd80a8e40 con 0x55b404a7e2f0 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.657+0000 7f5bca000640 1 -- 172.21.15.63:0/1822534470 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5bd80a9100 con 0x55b404a7e2f0 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.657+0000 7f5bf256d740 10 client.16281 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.657+0000 7f5bf256d740 1 -- 172.21.15.63:0/1822534470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b404b8b620 con 0x55b404a7e2f0 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.658+0000 7f5bca000640 1 -- 172.21.15.63:0/1822534470 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bd80a8e40 con 0x55b404a7e2f0 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.658+0000 7f5bca000640 1 -- 172.21.15.63:0/1822534470 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5bd80dfa20 con 0x55b404a7e2f0 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.658+0000 7f5bf256d740 10 client.16281 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.658+0000 7f5bf256d740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.658+0000 7f5bf256d740 20 client.16281 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.658+0000 7f5bf256d740 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b404a65d80 0x55b404e65d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.658+0000 7f5bf256d740 4 client.16281 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:37:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.658+0000 7f5bf256d740 1 -- 172.21.15.63:0/1822534470 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b404dbcae0 con 0x55b404a65d80 2024-05-27T07:37:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.659+0000 7f5be3e00640 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b404a65d80 0x55b404e65d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.660+0000 7f5be3e00640 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b404a65d80 0x55b404e65d00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631a7c563349a0c2 server_cookie=6d690d28e21b8153 in_seq=0 out_seq=0 2024-05-27T07:37:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.660+0000 7f5bca000640 10 client.16281 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.662+0000 7f5bca000640 1 -- 172.21.15.63:0/1822534470 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b404dbcae0 con 0x55b404a65d80 2024-05-27T07:37:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.662+0000 7f5bca000640 10 client.16281 handle_command_reply: tid=0 2024-05-27T07:37:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.662+0000 7f5bf256d740 1 client.16281 shutdown 2024-05-27T07:37:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.662+0000 7f5bf256d740 1 -- 172.21.15.63:0/1822534470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 msgr2=0x55b404e53c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.662+0000 7f5bf256d740 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e53c60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80a53b0 tx=0x7f5bd80e2730 comp rx=0 tx=0).stop 2024-05-27T07:37:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.663+0000 7f5bf256d740 1 -- 172.21.15.63:0/1822534470 shutdown_connections 2024-05-27T07:37:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.663+0000 7f5bf256d740 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b404a65d80 0x55b404e65d00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.663+0000 7f5bf256d740 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e5fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.663+0000 7f5bf256d740 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.663+0000 7f5bf256d740 1 --2- 172.21.15.63:0/1822534470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e53c60 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.663+0000 7f5bf256d740 1 -- 172.21.15.63:0/1822534470 >> 172.21.15.63:0/1822534470 conn(0x55b404a9b300 msgr2=0x55b404e49830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.663+0000 7f5bca000640 0 client.16281 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.663+0000 7f5bf256d740 1 -- 172.21.15.63:0/1822534470 shutdown_connections 2024-05-27T07:37:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.664+0000 7f5bf256d740 1 -- 172.21.15.63:0/1822534470 wait complete. 2024-05-27T07:37:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.664+0000 7f5bf256d740 20 client.16281 trim_cache size 0 max 16384 2024-05-27T07:37:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.685+0000 7f5bf256d740 1 Processor -- start 2024-05-27T07:37:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.685+0000 7f5bf256d740 1 -- start start 2024-05-27T07:37:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.686+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e4f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.686+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e5fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.686+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e50ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.686+0000 7f5bf256d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b5ab0 con 0x55b404a7e2f0 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.686+0000 7f5bf256d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b3320 con 0x55b404aa71b0 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.686+0000 7f5bf256d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b1170 con 0x55b404a864d0 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.686+0000 7f5be8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e4f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.686+0000 7f5be8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e4f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54268/0 (socket says 172.21.15.63:54268) 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.686+0000 7f5be8a00640 1 -- 172.21.15.63:0/1457805203 learned_addr learned my addr 172.21.15.63:0/1457805203 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.686+0000 7f5be3e00640 1 --2- 172.21.15.63:0/1457805203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e5fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5be9400640 1 --2- 172.21.15.63:0/1457805203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e50ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5be8a00640 1 -- 172.21.15.63:0/1457805203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 msgr2=0x55b404e5fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5be8a00640 1 --2- 172.21.15.63:0/1457805203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e5fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5be8a00640 1 -- 172.21.15.63:0/1457805203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 msgr2=0x55b404e50ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5be8a00640 1 --2- 172.21.15.63:0/1457805203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e50ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5be8a00640 1 -- 172.21.15.63:0/1457805203 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b404e60370 con 0x55b404a7e2f0 2024-05-27T07:37:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5be9400640 1 --2- 172.21.15.63:0/1457805203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e50ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5be8a00640 1 --2- 172.21.15.63:0/1457805203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e4f440 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80a92c0 tx=0x7f5bd8009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed32671bdb58e789 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5bd5200640 1 -- 172.21.15.63:0/1457805203 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80a8c80 con 0x55b404a7e2f0 2024-05-27T07:37:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5bd5200640 1 -- 172.21.15.63:0/1457805203 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bd80a8e40 con 0x55b404a7e2f0 2024-05-27T07:37:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.687+0000 7f5bd5200640 1 -- 172.21.15.63:0/1457805203 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80a8aa0 con 0x55b404a7e2f0 2024-05-27T07:37:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.689+0000 7f5bf256d740 1 -- 172.21.15.63:0/1457805203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 msgr2=0x55b404e4f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.689+0000 7f5bf256d740 1 --2- 172.21.15.63:0/1457805203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e4f440 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80a92c0 tx=0x7f5bd8009f30 comp rx=0 tx=0).stop 2024-05-27T07:37:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.689+0000 7f5bf256d740 1 -- 172.21.15.63:0/1457805203 shutdown_connections 2024-05-27T07:37:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.689+0000 7f5bf256d740 1 --2- 172.21.15.63:0/1457805203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404e50ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.689+0000 7f5bf256d740 1 --2- 172.21.15.63:0/1457805203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a864d0 0x55b404e5fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.689+0000 7f5bf256d740 1 --2- 172.21.15.63:0/1457805203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a7e2f0 0x55b404e4f440 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.689+0000 7f5bf256d740 1 -- 172.21.15.63:0/1457805203 >> 172.21.15.63:0/1457805203 conn(0x55b404a9b300 msgr2=0x55b404e53f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.689+0000 7f5bf256d740 1 -- 172.21.15.63:0/1457805203 shutdown_connections 2024-05-27T07:37:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.689+0000 7f5bf256d740 1 -- 172.21.15.63:0/1457805203 wait complete. 2024-05-27T07:37:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.689+0000 7f5bf256d740 1 -- start start 2024-05-27T07:37:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.690+0000 7f5bf256d740 1 Processor -- start 2024-05-27T07:37:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.690+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a7e2f0 0x55b404e682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.690+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a864d0 0x55b404e68800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.690+0000 7f5bf256d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404ecc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.690+0000 7f5bf256d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b17e0 con 0x55b404a864d0 2024-05-27T07:37:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.690+0000 7f5bf256d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b2c50 con 0x55b404aa71b0 2024-05-27T07:37:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.690+0000 7f5bf256d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41b1eb0 con 0x55b404a7e2f0 2024-05-27T07:37:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.690+0000 7f5bf256d740 10 client.0 _ll_register_callbacks cb 0x7f5bdc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:37:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.690+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a864d0 0x55b404e68800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a864d0 0x55b404e68800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54282/0 (socket says 172.21.15.63:54282) 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be3e00640 1 -- 172.21.15.63:0/3814641760 learned_addr learned my addr 172.21.15.63:0/3814641760 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be9400640 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404ecc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be8a00640 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a7e2f0 0x55b404e682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be3e00640 1 -- 172.21.15.63:0/3814641760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a7e2f0 msgr2=0x55b404e682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be3e00640 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a7e2f0 0x55b404e682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be3e00640 1 -- 172.21.15.63:0/3814641760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 msgr2=0x55b404ecc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be3e00640 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404ecc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be3e00640 1 -- 172.21.15.63:0/3814641760 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b404ecc690 con 0x55b404a864d0 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be8a00640 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a7e2f0 0x55b404e682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5be3e00640 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a864d0 0x55b404e68800 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x55b404dbabf0 tx=0x7f5bd0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285a9bf80801aa61 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.691+0000 7f5bca000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.692+0000 7f5bca000640 1 -- 172.21.15.63:0/3814641760 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0023670 con 0x55b404a864d0 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.692+0000 7f5bca000640 1 -- 172.21.15.63:0/3814641760 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f5bd0023810 con 0x55b404a864d0 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.692+0000 7f5bf256d740 1 -- 172.21.15.63:0/3814641760 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bd80a8e40 con 0x55b404a864d0 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.692+0000 7f5bca000640 1 -- 172.21.15.63:0/3814641760 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5bd002c050 con 0x55b404a864d0 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.692+0000 7f5bf256d740 10 client.16287 fetch_fsmap learned FSMap version 32 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.692+0000 7f5bf256d740 1 -- 172.21.15.63:0/3814641760 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b404ecd8c0 con 0x55b404a864d0 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.692+0000 7f5bca000640 1 -- 172.21.15.63:0/3814641760 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bd80a8e40 con 0x55b404a864d0 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.693+0000 7f5bca000640 1 -- 172.21.15.63:0/3814641760 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5bd001a7c0 con 0x55b404a864d0 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.693+0000 7f5bf256d740 10 client.16287 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.693+0000 7f5bf256d740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.693+0000 7f5bf256d740 20 client.16287 populate_metadata read hostname 'smithi063' 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.693+0000 7f5bf256d740 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b404a3b1c0 0x55b404ed0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.693+0000 7f5bf256d740 4 client.16287 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:37:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.693+0000 7f5bf256d740 1 -- 172.21.15.63:0/3814641760 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b404db62c0 con 0x55b404a3b1c0 2024-05-27T07:37:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.694+0000 7f5be8a00640 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b404a3b1c0 0x55b404ed0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.694+0000 7f5be8a00640 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b404a3b1c0 0x55b404ed0110 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9309918466643ac server_cookie=5fa9adad7aed6560 in_seq=0 out_seq=0 2024-05-27T07:37:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.694+0000 7f5bca000640 10 client.16287 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bca000640 1 -- 172.21.15.63:0/3814641760 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b404db62c0 con 0x55b404a3b1c0 2024-05-27T07:37:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bca000640 10 client.16287 handle_command_reply: tid=0 2024-05-27T07:37:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 client.16287 shutdown 2024-05-27T07:37:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 -- 172.21.15.63:0/3814641760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a864d0 msgr2=0x55b404e68800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a864d0 0x55b404e68800 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x55b404dbabf0 tx=0x7f5bd0011810 comp rx=0 tx=0).stop 2024-05-27T07:37:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 -- 172.21.15.63:0/3814641760 shutdown_connections 2024-05-27T07:37:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b404a3b1c0 0x55b404ed0110 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b404aa71b0 0x55b404ecc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b404a864d0 0x55b404e68800 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 --2- 172.21.15.63:0/3814641760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b404a7e2f0 0x55b404e682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 -- 172.21.15.63:0/3814641760 >> 172.21.15.63:0/3814641760 conn(0x55b404a9b300 msgr2=0x55b404e53c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bca000640 0 client.16287 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:37:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 -- 172.21.15.63:0/3814641760 shutdown_connections 2024-05-27T07:37:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.735+0000 7f5bf256d740 1 -- 172.21.15.63:0/3814641760 wait complete. 2024-05-27T07:37:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.736+0000 7f5bf256d740 20 client.16287 trim_cache size 0 max 16384 2024-05-27T07:37:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:37:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:37:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:37:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 -- 172.21.15.63:0/3601949533 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bcc0b29d0 msgr2=0x7f5bcc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bcc0b29d0 0x7f5bcc0b4ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005e30 tx=0x7f5bdc005d80 comp rx=0 tx=0).stop 2024-05-27T07:37:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 -- 172.21.15.63:0/3601949533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be4104bb0 msgr2=0x7f5be4102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be4104bb0 0x7f5be4102470 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800d950 tx=0x7f5bd800de10 comp rx=0 tx=0).stop 2024-05-27T07:37:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 -- 172.21.15.63:0/3601949533 shutdown_connections 2024-05-27T07:37:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bcc0b29d0 0x7f5bcc0b4ee0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be410e000 0x7f5be4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5be410b610 0x7f5be41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 --2- 172.21.15.63:0/3601949533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be4104bb0 0x7f5be4102470 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 -- 172.21.15.63:0/3601949533 >> 172.21.15.63:0/3601949533 conn(0x7f5be40fe210 msgr2=0x7f5be410e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 -- 172.21.15.63:0/3601949533 shutdown_connections 2024-05-27T07:37:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:37:58.737+0000 7f5be9e00640 1 -- 172.21.15.63:0/3601949533 wait complete. 2024-05-27T07:37:58.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:37:58 smithi086 ceph-mon[34687]: pgmap v135: 129 pgs: 129 active+clean; 497 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.7 MiB/s wr, 38 op/s 2024-05-27T07:37:58.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:37:58 smithi149 ceph-mon[35408]: pgmap v135: 129 pgs: 129 active+clean; 497 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.7 MiB/s wr, 38 op/s 2024-05-27T07:37:58.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:37:58 smithi063 ceph-mon[31613]: pgmap v135: 129 pgs: 129 active+clean; 497 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.7 MiB/s wr, 38 op/s 2024-05-27T07:38:00.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:00 smithi086 ceph-mon[34687]: pgmap v136: 129 pgs: 129 active+clean; 497 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.0 MiB/s wr, 31 op/s 2024-05-27T07:38:00.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:00 smithi149 ceph-mon[35408]: pgmap v136: 129 pgs: 129 active+clean; 497 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.0 MiB/s wr, 31 op/s 2024-05-27T07:38:00.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:00 smithi063 ceph-mon[31613]: pgmap v136: 129 pgs: 129 active+clean; 497 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.0 MiB/s wr, 31 op/s 2024-05-27T07:38:01.864 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:01 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:38:01.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:01 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:38:01.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:01 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:38:02.784 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:02 smithi063 ceph-mon[31613]: pgmap v137: 129 pgs: 129 active+clean; 523 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.1 MiB/s wr, 53 op/s 2024-05-27T07:38:02.784 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:02 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:38:02.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:02 smithi086 ceph-mon[34687]: pgmap v137: 129 pgs: 129 active+clean; 523 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.1 MiB/s wr, 53 op/s 2024-05-27T07:38:02.955 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:02 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:38:02.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:02 smithi149 ceph-mon[35408]: pgmap v137: 129 pgs: 129 active+clean; 523 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.1 MiB/s wr, 53 op/s 2024-05-27T07:38:02.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:02 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:38:03.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:03 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:03.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:03 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:03.857 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:03 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:38:03.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:03 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:03.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:03 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:03.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:03 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:38:03.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:03 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:03.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:03 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:03.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:03 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:38:04.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:04 smithi063 ceph-mon[31613]: pgmap v138: 129 pgs: 129 active+clean; 523 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.5 MiB/s wr, 25 op/s 2024-05-27T07:38:05.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:04 smithi086 ceph-mon[34687]: pgmap v138: 129 pgs: 129 active+clean; 523 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.5 MiB/s wr, 25 op/s 2024-05-27T07:38:05.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:04 smithi149 ceph-mon[35408]: pgmap v138: 129 pgs: 129 active+clean; 523 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.5 MiB/s wr, 25 op/s 2024-05-27T07:38:07.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:06 smithi086 ceph-mon[34687]: pgmap v139: 129 pgs: 129 active+clean; 523 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.5 MiB/s wr, 25 op/s 2024-05-27T07:38:07.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:06 smithi149 ceph-mon[35408]: pgmap v139: 129 pgs: 129 active+clean; 523 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.5 MiB/s wr, 25 op/s 2024-05-27T07:38:07.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:06 smithi063 ceph-mon[31613]: pgmap v139: 129 pgs: 129 active+clean; 523 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.5 MiB/s wr, 25 op/s 2024-05-27T07:38:09.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:08 smithi086 ceph-mon[34687]: pgmap v140: 129 pgs: 129 active+clean; 550 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 29 op/s 2024-05-27T07:38:09.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:08 smithi149 ceph-mon[35408]: pgmap v140: 129 pgs: 129 active+clean; 550 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 29 op/s 2024-05-27T07:38:09.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:08 smithi063 ceph-mon[31613]: pgmap v140: 129 pgs: 129 active+clean; 550 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 29 op/s 2024-05-27T07:38:11.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:10 smithi086 ceph-mon[34687]: pgmap v141: 129 pgs: 129 active+clean; 550 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 26 op/s 2024-05-27T07:38:11.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:10 smithi149 ceph-mon[35408]: pgmap v141: 129 pgs: 129 active+clean; 550 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 26 op/s 2024-05-27T07:38:11.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:10 smithi063 ceph-mon[31613]: pgmap v141: 129 pgs: 129 active+clean; 550 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 26 op/s 2024-05-27T07:38:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.976+0000 7f6aa3e00640 1 -- 172.21.15.63:0/2309840915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa410c190 msgr2=0x7f6aa4105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.976+0000 7f6aa3e00640 1 --2- 172.21.15.63:0/2309840915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa410c190 0x7f6aa4105dd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6a90009900 tx=0x7f6a9002a410 comp rx=0 tx=0).stop 2024-05-27T07:38:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.976+0000 7f6aa3e00640 1 -- 172.21.15.63:0/2309840915 shutdown_connections 2024-05-27T07:38:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.976+0000 7f6aa3e00640 1 --2- 172.21.15.63:0/2309840915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa4106440 0x7f6aa4112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.976+0000 7f6aa3e00640 1 --2- 172.21.15.63:0/2309840915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa410c190 0x7f6aa4105dd0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.976+0000 7f6aa3e00640 1 --2- 172.21.15.63:0/2309840915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6aa410bcf0 0x7f6aa4105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.976+0000 7f6aa3e00640 1 -- 172.21.15.63:0/2309840915 >> 172.21.15.63:0/2309840915 conn(0x7f6aa407d510 msgr2=0x7f6aa407d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.976+0000 7f6aa3e00640 1 -- 172.21.15.63:0/2309840915 shutdown_connections 2024-05-27T07:38:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.976+0000 7f6aa3e00640 1 -- 172.21.15.63:0/2309840915 wait complete. 2024-05-27T07:38:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.978+0000 7f6aa3e00640 1 Processor -- start 2024-05-27T07:38:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.978+0000 7f6aa3e00640 1 -- start start 2024-05-27T07:38:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.978+0000 7f6aa3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4106440 0x7f6aa41a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.978+0000 7f6aa3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa410bcf0 0x7f6aa41a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.978+0000 7f6aa3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6aa410c190 0x7f6aa419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.978+0000 7f6aa3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa4115de0 con 0x7f6aa4106440 2024-05-27T07:38:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.978+0000 7f6aa3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa4115c60 con 0x7f6aa410bcf0 2024-05-27T07:38:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.978+0000 7f6aa3e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa4115f60 con 0x7f6aa410c190 2024-05-27T07:38:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.979+0000 7f6aa2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4106440 0x7f6aa41a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.979+0000 7f6aa2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4106440 0x7f6aa41a24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47912/0 (socket says 172.21.15.63:47912) 2024-05-27T07:38:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.979+0000 7f6aa2a00640 1 -- 172.21.15.63:0/471095316 learned_addr learned my addr 172.21.15.63:0/471095316 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.979+0000 7f6aa2000640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa410bcf0 0x7f6aa41a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.979+0000 7f6aa3400640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6aa410c190 0x7f6aa419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.979+0000 7f6aa2a00640 1 -- 172.21.15.63:0/471095316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6aa410c190 msgr2=0x7f6aa419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.979+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6aa410c190 0x7f6aa419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.979+0000 7f6aa2a00640 1 -- 172.21.15.63:0/471095316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa410bcf0 msgr2=0x7f6aa41a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.979+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa410bcf0 0x7f6aa41a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.979+0000 7f6aa2a00640 1 -- 172.21.15.63:0/471095316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa419cd90 con 0x7f6aa4106440 2024-05-27T07:38:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.980+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4106440 0x7f6aa41a24e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400ea50 tx=0x7f6a9400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.980+0000 7f6a9b400640 1 -- 172.21.15.63:0/471095316 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400cdc0 con 0x7f6aa4106440 2024-05-27T07:38:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.980+0000 7f6a9b400640 1 -- 172.21.15.63:0/471095316 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a94005020 con 0x7f6aa4106440 2024-05-27T07:38:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.980+0000 7f6aa3e00640 1 -- 172.21.15.63:0/471095316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa419d020 con 0x7f6aa4106440 2024-05-27T07:38:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.980+0000 7f6a9b400640 1 -- 172.21.15.63:0/471095316 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a940116d0 con 0x7f6aa4106440 2024-05-27T07:38:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.980+0000 7f6aa3e00640 1 -- 172.21.15.63:0/471095316 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa41a9280 con 0x7f6aa4106440 2024-05-27T07:38:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.984+0000 7f6ab0a5b740 1 Processor -- start 2024-05-27T07:38:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.984+0000 7f6a9b400640 1 -- 172.21.15.63:0/471095316 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6a94004a40 con 0x7f6aa4106440 2024-05-27T07:38:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.985+0000 7f6ab0a5b740 1 -- start start 2024-05-27T07:38:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.985+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18de1b0 0x55b5a1c80a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.985+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18b52f0 0x55b5a1c80fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.985+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.985+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b5a1bf1b90 con 0x55b5a18bd4d0 2024-05-27T07:38:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.985+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5a1be1730 con 0x55b5a18b52f0 2024-05-27T07:38:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.985+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b5a1be5770 con 0x55b5a18de1b0 2024-05-27T07:38:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.985+0000 7f6aa2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.985+0000 7f6aa2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c814e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47920/0 (socket says 172.21.15.63:47920) 2024-05-27T07:38:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.985+0000 7f6aa2000640 1 -- 172.21.15.63:0/3782155386 learned_addr learned my addr 172.21.15.63:0/3782155386 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.986+0000 7f6aa3400640 1 --2- 172.21.15.63:0/3782155386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18de1b0 0x55b5a1c80a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.986+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/3782155386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18b52f0 0x55b5a1c80fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.986+0000 7f6aa2000640 1 -- 172.21.15.63:0/3782155386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18de1b0 msgr2=0x55b5a1c80a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.986+0000 7f6aa2000640 1 --2- 172.21.15.63:0/3782155386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18de1b0 0x55b5a1c80a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.986+0000 7f6aa2000640 1 -- 172.21.15.63:0/3782155386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18b52f0 msgr2=0x55b5a1c80fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.986+0000 7f6aa2000640 1 --2- 172.21.15.63:0/3782155386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18b52f0 0x55b5a1c80fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.986+0000 7f6aa2000640 1 -- 172.21.15.63:0/3782155386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a1b7a8b0 con 0x55b5a18bd4d0 2024-05-27T07:38:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.986+0000 7f6a9b400640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6a880b2950 0x7f6a880b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.986+0000 7f6aa2000640 1 --2- 172.21.15.63:0/3782155386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c814e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6a900057c0 tx=0x7f6a90004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bbbf2b731a41fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.987+0000 7f6a9b400640 1 -- 172.21.15.63:0/471095316 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f6a940e2fb0 con 0x7f6aa4106440 2024-05-27T07:38:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.987+0000 7f6aa3400640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6a880b2950 0x7f6a880b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.987+0000 7f6a8fe00640 1 -- 172.21.15.63:0/3782155386 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90035030 con 0x55b5a18bd4d0 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.987+0000 7f6a8fe00640 1 -- 172.21.15.63:0/3782155386 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a900054d0 con 0x55b5a18bd4d0 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.987+0000 7f6a8fe00640 1 -- 172.21.15.63:0/3782155386 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a900035f0 con 0x55b5a18bd4d0 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.988+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/3782155386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18bd4d0 msgr2=0x55b5a1c814e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.988+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/3782155386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c814e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6a900057c0 tx=0x7f6a90004200 comp rx=0 tx=0).stop 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.988+0000 7f6aa3400640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6a880b2950 0x7f6a880b4e60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c005fd0 tx=0x7f6a9c00b5d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.988+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/3782155386 shutdown_connections 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.988+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/3782155386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c814e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.988+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/3782155386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18b52f0 0x55b5a1c80fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.988+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/3782155386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18de1b0 0x55b5a1c80a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.988+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/3782155386 >> 172.21.15.63:0/3782155386 conn(0x55b5a18d2300 msgr2=0x55b5a1c7d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.988+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/3782155386 shutdown_connections 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.988+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/3782155386 wait complete. 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.989+0000 7f6ab0a5b740 1 -- start start 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.989+0000 7f6ab0a5b740 1 Processor -- start 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.990+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18b52f0 0x55b5a1c8ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.990+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c8b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.990+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c96e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.990+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b5a18c7790 con 0x55b5a18de1b0 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.990+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5a1b89bc0 con 0x55b5a18bd4d0 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.990+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b5a1c8db30 con 0x55b5a18b52f0 2024-05-27T07:38:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.990+0000 7f6ab0a5b740 10 client.0 _ll_register_callbacks cb 0x55b5a1b67400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.990+0000 7f6aa3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c96e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.990+0000 7f6aa3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c96e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47932/0 (socket says 172.21.15.63:47932) 2024-05-27T07:38:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.990+0000 7f6aa3400640 1 -- 172.21.15.63:0/2296883384 learned_addr learned my addr 172.21.15.63:0/2296883384 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.991+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18b52f0 0x55b5a1c8ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.991+0000 7f6aa2000640 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c8b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.991+0000 7f6aa3400640 1 -- 172.21.15.63:0/2296883384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18b52f0 msgr2=0x55b5a1c8ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.991+0000 7f6aa3400640 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18b52f0 0x55b5a1c8ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.991+0000 7f6aa3400640 1 -- 172.21.15.63:0/2296883384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18bd4d0 msgr2=0x55b5a1c8b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.991+0000 7f6aa3400640 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c8b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.991+0000 7f6aa3400640 1 -- 172.21.15.63:0/2296883384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a18f6210 con 0x55b5a18de1b0 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.991+0000 7f6aa3400640 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c96e60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c012700 tx=0x7f6a9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e611d77ba5d8a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.991+0000 7f6a8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.991+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2296883384 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c005220 con 0x55b5a18de1b0 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.992+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2296883384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a900054d0 con 0x55b5a18de1b0 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.992+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2296883384 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f6a9c0053c0 con 0x55b5a18de1b0 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.992+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2296883384 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6a9c02f050 con 0x55b5a18de1b0 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.992+0000 7f6ab0a5b740 10 client.16299 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.992+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2296883384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b5a19c2620 con 0x55b5a18de1b0 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.992+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2296883384 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a900054d0 con 0x55b5a18de1b0 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.993+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2296883384 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6a9c01f980 con 0x55b5a18de1b0 2024-05-27T07:38:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.993+0000 7f6ab0a5b740 10 client.16299 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.993+0000 7f6ab0a5b740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.993+0000 7f6ab0a5b740 20 client.16299 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.993+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b5a189cd80 0x55b5a1c9cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.993+0000 7f6ab0a5b740 4 client.16299 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:38:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.993+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2296883384 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5a1bf3ae0 con 0x55b5a189cd80 2024-05-27T07:38:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.993+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b5a189cd80 0x55b5a1c9cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.994+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b5a189cd80 0x55b5a1c9cf40 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa91acfbd9a3b0ff server_cookie=aee173f96b86f332 in_seq=0 out_seq=0 2024-05-27T07:38:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.994+0000 7f6a8cc00640 10 client.16299 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.996+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2296883384 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b5a1bf3ae0 con 0x55b5a189cd80 2024-05-27T07:38:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.996+0000 7f6a8cc00640 10 client.16299 handle_command_reply: tid=0 2024-05-27T07:38:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.996+0000 7f6ab0a5b740 1 client.16299 shutdown 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2296883384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 msgr2=0x55b5a1c96e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c96e60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c012700 tx=0x7f6a9c0046c0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2296883384 shutdown_connections 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b5a189cd80 0x55b5a1c9cf40 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c96e60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c8b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2296883384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18b52f0 0x55b5a1c8ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2296883384 >> 172.21.15.63:0/2296883384 conn(0x55b5a18d2300 msgr2=0x55b5a1c82780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6a8cc00640 0 client.16299 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2296883384 shutdown_connections 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2296883384 wait complete. 2024-05-27T07:38:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:11.997+0000 7f6ab0a5b740 20 client.16299 trim_cache size 0 max 16384 2024-05-27T07:38:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.018+0000 7f6ab0a5b740 1 Processor -- start 2024-05-27T07:38:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.018+0000 7f6ab0a5b740 1 -- start start 2024-05-27T07:38:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.018+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18b52f0 0x55b5a1c807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.018+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c96bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.018+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18de1b0 0x55b5a1c97110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.018+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41b5920 con 0x55b5a18b52f0 2024-05-27T07:38:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.018+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41b31f0 con 0x55b5a18de1b0 2024-05-27T07:38:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.018+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41b1040 con 0x55b5a18bd4d0 2024-05-27T07:38:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18b52f0 0x55b5a1c807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18b52f0 0x55b5a1c807f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47942/0 (socket says 172.21.15.63:47942) 2024-05-27T07:38:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa2a00640 1 -- 172.21.15.63:0/263068839 learned_addr learned my addr 172.21.15.63:0/263068839 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa3400640 1 --2- 172.21.15.63:0/263068839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18de1b0 0x55b5a1c97110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa2000640 1 --2- 172.21.15.63:0/263068839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c96bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa2a00640 1 -- 172.21.15.63:0/263068839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18bd4d0 msgr2=0x55b5a1c96bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/263068839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c96bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa2a00640 1 -- 172.21.15.63:0/263068839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18de1b0 msgr2=0x55b5a1c97110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/263068839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18de1b0 0x55b5a1c97110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa2a00640 1 -- 172.21.15.63:0/263068839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a1c7e130 con 0x55b5a18b52f0 2024-05-27T07:38:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.019+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/263068839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18b52f0 0x55b5a1c807f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6a940a9e50 tx=0x7f6a9400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9069bc2859db7ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.020+0000 7f6a8f400640 1 -- 172.21.15.63:0/263068839 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9401f040 con 0x55b5a18b52f0 2024-05-27T07:38:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.020+0000 7f6a8f400640 1 -- 172.21.15.63:0/263068839 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a940a9ec0 con 0x55b5a18b52f0 2024-05-27T07:38:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.020+0000 7f6a8f400640 1 -- 172.21.15.63:0/263068839 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a940cc040 con 0x55b5a18b52f0 2024-05-27T07:38:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.021+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/263068839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18b52f0 msgr2=0x55b5a1c807f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.021+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/263068839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18b52f0 0x55b5a1c807f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6a940a9e50 tx=0x7f6a9400c830 comp rx=0 tx=0).stop 2024-05-27T07:38:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.021+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/263068839 shutdown_connections 2024-05-27T07:38:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.021+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/263068839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18de1b0 0x55b5a1c97110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.021+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/263068839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c96bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.021+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/263068839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18b52f0 0x55b5a1c807f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.021+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/263068839 >> 172.21.15.63:0/263068839 conn(0x55b5a18d2300 msgr2=0x55b5a1c8aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.021+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/263068839 shutdown_connections 2024-05-27T07:38:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.021+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/263068839 wait complete. 2024-05-27T07:38:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.022+0000 7f6ab0a5b740 1 -- start start 2024-05-27T07:38:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.022+0000 7f6ab0a5b740 1 Processor -- start 2024-05-27T07:38:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.022+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18b52f0 0x55b5a1c93400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.022+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c8e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.022+0000 7f6ab0a5b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c8ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.022+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41b16b0 con 0x55b5a18de1b0 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.022+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41b2b20 con 0x55b5a18bd4d0 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.022+0000 7f6ab0a5b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa41b1d80 con 0x55b5a18b52f0 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6ab0a5b740 10 client.0 _ll_register_callbacks cb 0x7f6a9c00b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6aa3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c8ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6aa3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c8ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47952/0 (socket says 172.21.15.63:47952) 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6aa3400640 1 -- 172.21.15.63:0/2831320593 learned_addr learned my addr 172.21.15.63:0/2831320593 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6aa2000640 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c8e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18b52f0 0x55b5a1c93400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6aa3400640 1 -- 172.21.15.63:0/2831320593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18b52f0 msgr2=0x55b5a1c93400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:12.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6aa3400640 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18b52f0 0x55b5a1c93400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6aa3400640 1 -- 172.21.15.63:0/2831320593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18bd4d0 msgr2=0x55b5a1c8e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6aa3400640 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c8e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.023+0000 7f6aa3400640 1 -- 172.21.15.63:0/2831320593 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a1c7e320 con 0x55b5a18de1b0 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.024+0000 7f6aa3400640 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c8ec90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c012fd0 tx=0x7f6a9c011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c3bd446a71485 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.024+0000 7f6a8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.024+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2831320593 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c005040 con 0x55b5a18de1b0 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.024+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2831320593 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f6a9c0051e0 con 0x55b5a18de1b0 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.024+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2831320593 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a940a9ec0 con 0x55b5a18de1b0 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.024+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2831320593 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6a9c0053e0 con 0x55b5a18de1b0 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.024+0000 7f6ab0a5b740 10 client.16305 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.024+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2831320593 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b5a1d05240 con 0x55b5a18de1b0 2024-05-27T07:38:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.025+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2831320593 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a940a9ec0 con 0x55b5a18de1b0 2024-05-27T07:38:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.025+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2831320593 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6a9c03b020 con 0x55b5a18de1b0 2024-05-27T07:38:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.025+0000 7f6ab0a5b740 10 client.16305 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.025+0000 7f6ab0a5b740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.025+0000 7f6ab0a5b740 20 client.16305 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.025+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b5a18721c0 0x55b5a1d07a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.025+0000 7f6ab0a5b740 4 client.16305 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:38:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.025+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2831320593 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5a1bed2c0 con 0x55b5a18721c0 2024-05-27T07:38:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.026+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b5a18721c0 0x55b5a1d07a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.027+0000 7f6aa2a00640 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b5a18721c0 0x55b5a1d07a30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a0989ac483d507 server_cookie=b5642d56322b25c2 in_seq=0 out_seq=0 2024-05-27T07:38:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.027+0000 7f6a8cc00640 10 client.16305 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.062+0000 7f6a8cc00640 1 -- 172.21.15.63:0/2831320593 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b5a1bed2c0 con 0x55b5a18721c0 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.062+0000 7f6a8cc00640 10 client.16305 handle_command_reply: tid=0 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.062+0000 7f6ab0a5b740 1 client.16305 shutdown 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2831320593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 msgr2=0x55b5a1c8ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c8ec90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c012fd0 tx=0x7f6a9c011f50 comp rx=0 tx=0).stop 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2831320593 shutdown_connections 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b5a18721c0 0x55b5a1d07a30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b5a18de1b0 0x55b5a1c8ec90 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5a18bd4d0 0x55b5a1c8e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 1 --2- 172.21.15.63:0/2831320593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b5a18b52f0 0x55b5a1c93400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6a8cc00640 0 client.16305 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2831320593 >> 172.21.15.63:0/2831320593 conn(0x55b5a18d2300 msgr2=0x55b5a1c8ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2831320593 shutdown_connections 2024-05-27T07:38:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 1 -- 172.21.15.63:0/2831320593 wait complete. 2024-05-27T07:38:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.063+0000 7f6ab0a5b740 20 client.16305 trim_cache size 0 max 16384 2024-05-27T07:38:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:38:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:38:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:38:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.064+0000 7f6aa3e00640 1 -- 172.21.15.63:0/471095316 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6a880b2950 msgr2=0x7f6a880b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.064+0000 7f6aa3e00640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6a880b2950 0x7f6a880b4e60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c005fd0 tx=0x7f6a9c00b5d0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.064+0000 7f6aa3e00640 1 -- 172.21.15.63:0/471095316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4106440 msgr2=0x7f6aa41a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.064+0000 7f6aa3e00640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4106440 0x7f6aa41a24e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400ea50 tx=0x7f6a9400ef10 comp rx=0 tx=0).stop 2024-05-27T07:38:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.065+0000 7f6aa3e00640 1 -- 172.21.15.63:0/471095316 shutdown_connections 2024-05-27T07:38:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.065+0000 7f6aa3e00640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6a880b2950 0x7f6a880b4e60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.065+0000 7f6aa3e00640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6aa410c190 0x7f6aa419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.065+0000 7f6aa3e00640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6aa410bcf0 0x7f6aa41a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.065+0000 7f6aa3e00640 1 --2- 172.21.15.63:0/471095316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4106440 0x7f6aa41a24e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.065+0000 7f6aa3e00640 1 -- 172.21.15.63:0/471095316 >> 172.21.15.63:0/471095316 conn(0x7f6aa407d510 msgr2=0x7f6aa4104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.065+0000 7f6aa3e00640 1 -- 172.21.15.63:0/471095316 shutdown_connections 2024-05-27T07:38:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:12.065+0000 7f6aa3e00640 1 -- 172.21.15.63:0/471095316 wait complete. 2024-05-27T07:38:13.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:12 smithi086 ceph-mon[34687]: pgmap v142: 129 pgs: 129 active+clean; 570 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 29 op/s 2024-05-27T07:38:13.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:12 smithi149 ceph-mon[35408]: pgmap v142: 129 pgs: 129 active+clean; 570 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 29 op/s 2024-05-27T07:38:13.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:12 smithi063 ceph-mon[31613]: pgmap v142: 129 pgs: 129 active+clean; 570 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 29 op/s 2024-05-27T07:38:15.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:14 smithi086 ceph-mon[34687]: pgmap v143: 129 pgs: 129 active+clean; 570 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 6 op/s 2024-05-27T07:38:15.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:14 smithi149 ceph-mon[35408]: pgmap v143: 129 pgs: 129 active+clean; 570 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 6 op/s 2024-05-27T07:38:15.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:14 smithi063 ceph-mon[31613]: pgmap v143: 129 pgs: 129 active+clean; 570 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 6 op/s 2024-05-27T07:38:17.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:16 smithi086 ceph-mon[34687]: pgmap v144: 129 pgs: 129 active+clean; 570 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 6 op/s 2024-05-27T07:38:17.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:16 smithi149 ceph-mon[35408]: pgmap v144: 129 pgs: 129 active+clean; 570 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 6 op/s 2024-05-27T07:38:17.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:16 smithi063 ceph-mon[31613]: pgmap v144: 129 pgs: 129 active+clean; 570 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 MiB/s wr, 6 op/s 2024-05-27T07:38:19.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:18 smithi086 ceph-mon[34687]: pgmap v145: 129 pgs: 129 active+clean; 594 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 13 op/s 2024-05-27T07:38:19.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:18 smithi149 ceph-mon[35408]: pgmap v145: 129 pgs: 129 active+clean; 594 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 13 op/s 2024-05-27T07:38:19.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:18 smithi063 ceph-mon[31613]: pgmap v145: 129 pgs: 129 active+clean; 594 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 13 op/s 2024-05-27T07:38:21.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:20 smithi086 ceph-mon[34687]: pgmap v146: 129 pgs: 129 active+clean; 594 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:38:21.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:20 smithi149 ceph-mon[35408]: pgmap v146: 129 pgs: 129 active+clean; 594 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:38:21.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:20 smithi063 ceph-mon[31613]: pgmap v146: 129 pgs: 129 active+clean; 594 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:38:23.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:22 smithi086 ceph-mon[34687]: pgmap v147: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 12 op/s 2024-05-27T07:38:23.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:22 smithi149 ceph-mon[35408]: pgmap v147: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 12 op/s 2024-05-27T07:38:23.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:22 smithi063 ceph-mon[31613]: pgmap v147: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 12 op/s 2024-05-27T07:38:25.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:24 smithi086 ceph-mon[34687]: pgmap v148: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:38:25.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:24 smithi149 ceph-mon[35408]: pgmap v148: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:38:25.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:24 smithi063 ceph-mon[31613]: pgmap v148: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:38:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.292+0000 7f32eac00640 1 -- 172.21.15.63:0/541548856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e410b6b0 msgr2=0x7f32e410dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.292+0000 7f32eac00640 1 --2- 172.21.15.63:0/541548856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e410b6b0 0x7f32e410dad0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009900 tx=0x7f32d002a480 comp rx=0 tx=0).stop 2024-05-27T07:38:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.293+0000 7f32eac00640 1 -- 172.21.15.63:0/541548856 shutdown_connections 2024-05-27T07:38:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.293+0000 7f32eac00640 1 --2- 172.21.15.63:0/541548856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e410e010 0x7f32e4110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.293+0000 7f32eac00640 1 --2- 172.21.15.63:0/541548856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e410b6b0 0x7f32e410dad0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.293+0000 7f32eac00640 1 --2- 172.21.15.63:0/541548856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e4104c50 0x7f32e4105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.293+0000 7f32eac00640 1 -- 172.21.15.63:0/541548856 >> 172.21.15.63:0/541548856 conn(0x7f32e40fe210 msgr2=0x7f32e4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.293+0000 7f32eac00640 1 -- 172.21.15.63:0/541548856 shutdown_connections 2024-05-27T07:38:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.293+0000 7f32eac00640 1 -- 172.21.15.63:0/541548856 wait complete. 2024-05-27T07:38:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.294+0000 7f32eac00640 1 Processor -- start 2024-05-27T07:38:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.294+0000 7f32eac00640 1 -- start start 2024-05-27T07:38:25.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.295+0000 7f32eac00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e4104c50 0x7f32e419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.295+0000 7f32eac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e410b6b0 0x7f32e419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.295+0000 7f32eac00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e410e010 0x7f32e41a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.295+0000 7f32eac00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4113b70 con 0x7f32e4104c50 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.295+0000 7f32eac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41139f0 con 0x7f32e410b6b0 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.295+0000 7f32eac00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4113cf0 con 0x7f32e410e010 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.295+0000 7f32e9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e4104c50 0x7f32e419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.295+0000 7f32e9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e4104c50 0x7f32e419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54244/0 (socket says 172.21.15.63:54244) 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.295+0000 7f32e9800640 1 -- 172.21.15.63:0/863446074 learned_addr learned my addr 172.21.15.63:0/863446074 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.296+0000 7f32e8e00640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e410b6b0 0x7f32e419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.296+0000 7f32ea200640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e410e010 0x7f32e41a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.296+0000 7f32e9800640 1 -- 172.21.15.63:0/863446074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e410e010 msgr2=0x7f32e41a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:25.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.296+0000 7f32e9800640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e410e010 0x7f32e41a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.296+0000 7f32e9800640 1 -- 172.21.15.63:0/863446074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e410b6b0 msgr2=0x7f32e419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.296+0000 7f32e9800640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e410b6b0 0x7f32e419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.296+0000 7f32e9800640 1 -- 172.21.15.63:0/863446074 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e41a15b0 con 0x7f32e4104c50 2024-05-27T07:38:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.296+0000 7f32e8e00640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e410b6b0 0x7f32e419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:38:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.296+0000 7f32e9800640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e4104c50 0x7f32e419c7d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f32d800d980 tx=0x7f32d800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.297+0000 7f32e2000640 1 -- 172.21.15.63:0/863446074 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8017070 con 0x7f32e4104c50 2024-05-27T07:38:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.297+0000 7f32e2000640 1 -- 172.21.15.63:0/863446074 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32d8009c30 con 0x7f32e4104c50 2024-05-27T07:38:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.297+0000 7f32eac00640 1 -- 172.21.15.63:0/863446074 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e41a18a0 con 0x7f32e4104c50 2024-05-27T07:38:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.297+0000 7f32e2000640 1 -- 172.21.15.63:0/863446074 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d800f3c0 con 0x7f32e4104c50 2024-05-27T07:38:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.297+0000 7f32eac00640 1 -- 172.21.15.63:0/863446074 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e41a1df0 con 0x7f32e4104c50 2024-05-27T07:38:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.300+0000 7f32f3268740 1 Processor -- start 2024-05-27T07:38:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32f3268740 1 -- start start 2024-05-27T07:38:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32f3268740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e687a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32f3268740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e687fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32f3268740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2c44d0 0x559f2e6884e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32f3268740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559f2e5f8b90 con 0x559f2e2e51b0 2024-05-27T07:38:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32f3268740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f2e5e8730 con 0x559f2e2c44d0 2024-05-27T07:38:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32f3268740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559f2e5ec770 con 0x559f2e2bc2f0 2024-05-27T07:38:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32e2000640 1 -- 172.21.15.63:0/863446074 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f32d8010460 con 0x7f32e4104c50 2024-05-27T07:38:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32ea200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e687a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32ea200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e687a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54254/0 (socket says 172.21.15.63:54254) 2024-05-27T07:38:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.301+0000 7f32ea200640 1 -- 172.21.15.63:0/1621310705 learned_addr learned my addr 172.21.15.63:0/1621310705 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.302+0000 7f32e8e00640 1 --2- 172.21.15.63:0/1621310705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2c44d0 0x559f2e6884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.302+0000 7f32e9800640 1 --2- 172.21.15.63:0/1621310705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e687fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.302+0000 7f32ea200640 1 -- 172.21.15.63:0/1621310705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 msgr2=0x559f2e687fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.302+0000 7f32ea200640 1 --2- 172.21.15.63:0/1621310705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e687fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.302+0000 7f32ea200640 1 -- 172.21.15.63:0/1621310705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2c44d0 msgr2=0x559f2e6884e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.302+0000 7f32ea200640 1 --2- 172.21.15.63:0/1621310705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2c44d0 0x559f2e6884e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.302+0000 7f32ea200640 1 -- 172.21.15.63:0/1621310705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f2e5818b0 con 0x559f2e2e51b0 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.302+0000 7f32e9800640 1 --2- 172.21.15.63:0/1621310705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e687fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.302+0000 7f32ea200640 1 --2- 172.21.15.63:0/1621310705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e687a60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f32dc007dd0 tx=0x7f32dc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a76a1019ab05671 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.303+0000 7f32d6a00640 1 -- 172.21.15.63:0/1621310705 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc014070 con 0x559f2e2e51b0 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.303+0000 7f32d6a00640 1 -- 172.21.15.63:0/1621310705 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32dc004d20 con 0x559f2e2e51b0 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.303+0000 7f32d6a00640 1 -- 172.21.15.63:0/1621310705 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0106c0 con 0x559f2e2e51b0 2024-05-27T07:38:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.303+0000 7f32e2000640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32cc0b2950 0x7f32cc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.303+0000 7f32e2000640 1 -- 172.21.15.63:0/863446074 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f32d80e1ff0 con 0x7f32e4104c50 2024-05-27T07:38:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.303+0000 7f32e9800640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32cc0b2950 0x7f32cc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.304+0000 7f32f3268740 1 -- 172.21.15.63:0/1621310705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 msgr2=0x559f2e687a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.304+0000 7f32f3268740 1 --2- 172.21.15.63:0/1621310705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e687a60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f32dc007dd0 tx=0x7f32dc00e720 comp rx=0 tx=0).stop 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.304+0000 7f32f3268740 1 -- 172.21.15.63:0/1621310705 shutdown_connections 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.304+0000 7f32f3268740 1 --2- 172.21.15.63:0/1621310705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2c44d0 0x559f2e6884e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.304+0000 7f32f3268740 1 --2- 172.21.15.63:0/1621310705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e687fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.304+0000 7f32f3268740 1 --2- 172.21.15.63:0/1621310705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e687a60 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.304+0000 7f32f3268740 1 -- 172.21.15.63:0/1621310705 >> 172.21.15.63:0/1621310705 conn(0x559f2e2d9300 msgr2=0x559f2e684210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.304+0000 7f32e9800640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32cc0b2950 0x7f32cc0b4e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x559f2e685550 tx=0x7f32d8000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.304+0000 7f32f3268740 1 -- 172.21.15.63:0/1621310705 shutdown_connections 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.304+0000 7f32f3268740 1 -- 172.21.15.63:0/1621310705 wait complete. 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.305+0000 7f32f3268740 1 -- start start 2024-05-27T07:38:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.305+0000 7f32f3268740 1 Processor -- start 2024-05-27T07:38:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32f3268740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2bc2f0 0x559f2e691c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32f3268740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e692140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32f3268740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2e51b0 0x559f2e69dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32f3268740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559f2e2ce790 con 0x559f2e2c44d0 2024-05-27T07:38:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32f3268740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f2e590bc0 con 0x559f2e2bc2f0 2024-05-27T07:38:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32f3268740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559f2e694b90 con 0x559f2e2e51b0 2024-05-27T07:38:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32f3268740 10 client.0 _ll_register_callbacks cb 0x559f2e56e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32e8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e692140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32e8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e692140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54270/0 (socket says 172.21.15.63:54270) 2024-05-27T07:38:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32e8e00640 1 -- 172.21.15.63:0/2257664515 learned_addr learned my addr 172.21.15.63:0/2257664515 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32ea200640 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2bc2f0 0x559f2e691c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.306+0000 7f32ea200640 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2e51b0 0x559f2e69dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.307+0000 7f32e8e00640 1 -- 172.21.15.63:0/2257664515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2e51b0 msgr2=0x559f2e69dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.307+0000 7f32e8e00640 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2e51b0 0x559f2e69dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.307+0000 7f32e8e00640 1 -- 172.21.15.63:0/2257664515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2bc2f0 msgr2=0x559f2e691c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.307+0000 7f32e8e00640 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2bc2f0 0x559f2e691c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.307+0000 7f32e8e00640 1 -- 172.21.15.63:0/2257664515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2e2fd210 con 0x559f2e2c44d0 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.307+0000 7f32e8e00640 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e692140 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009900 tx=0x7f32d0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6daa12a9bba5d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.307+0000 7f32cb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.307+0000 7f32cb400640 1 -- 172.21.15.63:0/2257664515 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0035030 con 0x559f2e2c44d0 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.307+0000 7f32cb400640 1 -- 172.21.15.63:0/2257664515 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f32d00048c0 con 0x559f2e2c44d0 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.308+0000 7f32f3268740 1 -- 172.21.15.63:0/2257664515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32dc004d20 con 0x559f2e2c44d0 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.308+0000 7f32cb400640 1 -- 172.21.15.63:0/2257664515 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f32d004c050 con 0x559f2e2c44d0 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.308+0000 7f32f3268740 10 client.16317 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.308+0000 7f32f3268740 1 -- 172.21.15.63:0/2257664515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559f2e3c9620 con 0x559f2e2c44d0 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.308+0000 7f32cb400640 1 -- 172.21.15.63:0/2257664515 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32dc004d20 con 0x559f2e2c44d0 2024-05-27T07:38:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.308+0000 7f32cb400640 1 -- 172.21.15.63:0/2257664515 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f32d0003c30 con 0x559f2e2c44d0 2024-05-27T07:38:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.309+0000 7f32f3268740 10 client.16317 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.309+0000 7f32f3268740 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.309+0000 7f32f3268740 20 client.16317 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.309+0000 7f32f3268740 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559f2e2a3d80 0x559f2e6a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.309+0000 7f32f3268740 4 client.16317 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:38:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.309+0000 7f32f3268740 1 -- 172.21.15.63:0/2257664515 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f2e5faae0 con 0x559f2e2a3d80 2024-05-27T07:38:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.309+0000 7f32ea200640 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559f2e2a3d80 0x559f2e6a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.310+0000 7f32ea200640 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559f2e2a3d80 0x559f2e6a3dd0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a97db688d2c15800 server_cookie=b5242be0deeb9909 in_seq=0 out_seq=0 2024-05-27T07:38:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.310+0000 7f32cb400640 10 client.16317 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.312+0000 7f32cb400640 1 -- 172.21.15.63:0/2257664515 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559f2e5faae0 con 0x559f2e2a3d80 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.312+0000 7f32cb400640 10 client.16317 handle_command_reply: tid=0 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.312+0000 7f32f3268740 1 client.16317 shutdown 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 1 -- 172.21.15.63:0/2257664515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 msgr2=0x559f2e692140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e692140 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009900 tx=0x7f32d0005250 comp rx=0 tx=0).stop 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 1 -- 172.21.15.63:0/2257664515 shutdown_connections 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559f2e2a3d80 0x559f2e6a3dd0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2e51b0 0x559f2e69dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e692140 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 1 --2- 172.21.15.63:0/2257664515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2bc2f0 0x559f2e691c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32cb400640 0 client.16317 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 1 -- 172.21.15.63:0/2257664515 >> 172.21.15.63:0/2257664515 conn(0x559f2e2d9300 msgr2=0x559f2e687890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 1 -- 172.21.15.63:0/2257664515 shutdown_connections 2024-05-27T07:38:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 1 -- 172.21.15.63:0/2257664515 wait complete. 2024-05-27T07:38:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.313+0000 7f32f3268740 20 client.16317 trim_cache size 0 max 16384 2024-05-27T07:38:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.332+0000 7f32f3268740 1 Processor -- start 2024-05-27T07:38:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.333+0000 7f32f3268740 1 -- start start 2024-05-27T07:38:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.333+0000 7f32f3268740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e694330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.333+0000 7f32f3268740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2c44d0 0x559f2e69dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.333+0000 7f32f3268740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e69e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.333+0000 7f32f3268740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41b5a20 con 0x559f2e2e51b0 2024-05-27T07:38:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.333+0000 7f32f3268740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41b32f0 con 0x559f2e2c44d0 2024-05-27T07:38:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.333+0000 7f32f3268740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41b1140 con 0x559f2e2bc2f0 2024-05-27T07:38:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.333+0000 7f32ea200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e69e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.333+0000 7f32ea200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e69e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54284/0 (socket says 172.21.15.63:54284) 2024-05-27T07:38:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.333+0000 7f32ea200640 1 -- 172.21.15.63:0/1474019715 learned_addr learned my addr 172.21.15.63:0/1474019715 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.334+0000 7f32ea200640 1 --2- 172.21.15.63:0/1474019715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e694330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.334+0000 7f32e8e00640 1 --2- 172.21.15.63:0/1474019715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2c44d0 0x559f2e69dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.334+0000 7f32ea200640 1 -- 172.21.15.63:0/1474019715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 msgr2=0x559f2e694330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.334+0000 7f32ea200640 1 --2- 172.21.15.63:0/1474019715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e694330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.334+0000 7f32ea200640 1 -- 172.21.15.63:0/1474019715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2c44d0 msgr2=0x559f2e69dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.334+0000 7f32ea200640 1 --2- 172.21.15.63:0/1474019715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2c44d0 0x559f2e69dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.334+0000 7f32ea200640 1 -- 172.21.15.63:0/1474019715 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f2e6852d0 con 0x559f2e2e51b0 2024-05-27T07:38:25.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.334+0000 7f32ea200640 1 --2- 172.21.15.63:0/1474019715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e69e170 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f32dc007b60 tx=0x7f32dc0076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dea8c292b8620d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.335+0000 7f32d6000640 1 -- 172.21.15.63:0/1474019715 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc014030 con 0x559f2e2e51b0 2024-05-27T07:38:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.335+0000 7f32d6000640 1 -- 172.21.15.63:0/1474019715 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32dc02e070 con 0x559f2e2e51b0 2024-05-27T07:38:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.335+0000 7f32d6000640 1 -- 172.21.15.63:0/1474019715 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc021040 con 0x559f2e2e51b0 2024-05-27T07:38:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.336+0000 7f32f3268740 1 -- 172.21.15.63:0/1474019715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 msgr2=0x559f2e69e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.336+0000 7f32f3268740 1 --2- 172.21.15.63:0/1474019715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e69e170 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f32dc007b60 tx=0x7f32dc0076c0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.336+0000 7f32f3268740 1 -- 172.21.15.63:0/1474019715 shutdown_connections 2024-05-27T07:38:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.336+0000 7f32f3268740 1 --2- 172.21.15.63:0/1474019715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2e51b0 0x559f2e69e170 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.336+0000 7f32f3268740 1 --2- 172.21.15.63:0/1474019715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2c44d0 0x559f2e69dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.336+0000 7f32f3268740 1 --2- 172.21.15.63:0/1474019715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e694330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.336+0000 7f32f3268740 1 -- 172.21.15.63:0/1474019715 >> 172.21.15.63:0/1474019715 conn(0x559f2e2d9300 msgr2=0x559f2e691ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.336+0000 7f32f3268740 1 -- 172.21.15.63:0/1474019715 shutdown_connections 2024-05-27T07:38:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.336+0000 7f32f3268740 1 -- 172.21.15.63:0/1474019715 wait complete. 2024-05-27T07:38:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.336+0000 7f32f3268740 1 -- start start 2024-05-27T07:38:25.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.337+0000 7f32f3268740 1 Processor -- start 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.337+0000 7f32f3268740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e697320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.337+0000 7f32f3268740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e697860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.337+0000 7f32f3268740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2e51b0 0x559f2e70a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.337+0000 7f32f3268740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41b17b0 con 0x559f2e2c44d0 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.337+0000 7f32f3268740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41b2c20 con 0x559f2e2e51b0 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.337+0000 7f32f3268740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41b1e80 con 0x559f2e2bc2f0 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.337+0000 7f32f3268740 10 client.0 _ll_register_callbacks cb 0x7f32d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32e8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e697860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32e8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e697860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54300/0 (socket says 172.21.15.63:54300) 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32e8e00640 1 -- 172.21.15.63:0/302310132 learned_addr learned my addr 172.21.15.63:0/302310132 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:25.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32ea200640 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e697320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32e8e00640 1 -- 172.21.15.63:0/302310132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 msgr2=0x559f2e697320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32e8e00640 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e697320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32e8e00640 1 -- 172.21.15.63:0/302310132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2e51b0 msgr2=0x559f2e70a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32e8e00640 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2e51b0 0x559f2e70a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32e8e00640 1 -- 172.21.15.63:0/302310132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2e70a9f0 con 0x559f2e2c44d0 2024-05-27T07:38:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32ea200640 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e697320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:38:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32e8e00640 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e697860 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f32d003b430 tx=0x7f32d0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5178bd4056f283 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.338+0000 7f32cb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.339+0000 7f32cb400640 1 -- 172.21.15.63:0/302310132 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0035040 con 0x559f2e2c44d0 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.339+0000 7f32cb400640 1 -- 172.21.15.63:0/302310132 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f32d003ba80 con 0x559f2e2c44d0 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.339+0000 7f32f3268740 1 -- 172.21.15.63:0/302310132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32dc02e070 con 0x559f2e2c44d0 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.339+0000 7f32cb400640 1 -- 172.21.15.63:0/302310132 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f32d005e050 con 0x559f2e2c44d0 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.339+0000 7f32f3268740 10 client.16323 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.339+0000 7f32f3268740 1 -- 172.21.15.63:0/302310132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559f2e70bc40 con 0x559f2e2c44d0 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.340+0000 7f32cb400640 1 -- 172.21.15.63:0/302310132 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32dc02e070 con 0x559f2e2c44d0 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.340+0000 7f32cb400640 1 -- 172.21.15.63:0/302310132 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f32d00035f0 con 0x559f2e2c44d0 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.340+0000 7f32f3268740 10 client.16323 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.340+0000 7f32f3268740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.340+0000 7f32f3268740 20 client.16323 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.340+0000 7f32f3268740 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559f2e2791c0 0x559f2e70e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.340+0000 7f32f3268740 4 client.16323 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:38:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.340+0000 7f32f3268740 1 -- 172.21.15.63:0/302310132 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f2e5f42c0 con 0x559f2e2791c0 2024-05-27T07:38:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.341+0000 7f32ea200640 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559f2e2791c0 0x559f2e70e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.341+0000 7f32ea200640 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559f2e2791c0 0x559f2e70e4b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9dde6e3a77a40e server_cookie=62b920594512923c in_seq=0 out_seq=0 2024-05-27T07:38:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.341+0000 7f32cb400640 10 client.16323 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.378+0000 7f32cb400640 1 -- 172.21.15.63:0/302310132 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559f2e5f42c0 con 0x559f2e2791c0 2024-05-27T07:38:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.378+0000 7f32cb400640 10 client.16323 handle_command_reply: tid=0 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.378+0000 7f32f3268740 1 client.16323 shutdown 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32f3268740 1 -- 172.21.15.63:0/302310132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 msgr2=0x559f2e697860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32f3268740 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e697860 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f32d003b430 tx=0x7f32d0005db0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32f3268740 1 -- 172.21.15.63:0/302310132 shutdown_connections 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32f3268740 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559f2e2791c0 0x559f2e70e4b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32f3268740 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f2e2e51b0 0x559f2e70a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32f3268740 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f2e2c44d0 0x559f2e697860 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32f3268740 1 --2- 172.21.15.63:0/302310132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f2e2bc2f0 0x559f2e697320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32f3268740 1 -- 172.21.15.63:0/302310132 >> 172.21.15.63:0/302310132 conn(0x559f2e2d9300 msgr2=0x559f2e6932b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32cb400640 0 client.16323 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32f3268740 1 -- 172.21.15.63:0/302310132 shutdown_connections 2024-05-27T07:38:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.379+0000 7f32f3268740 1 -- 172.21.15.63:0/302310132 wait complete. 2024-05-27T07:38:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.380+0000 7f32f3268740 20 client.16323 trim_cache size 0 max 16384 2024-05-27T07:38:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:38:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:38:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:38:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.380+0000 7f32eac00640 1 -- 172.21.15.63:0/863446074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32cc0b2950 msgr2=0x7f32cc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.380+0000 7f32eac00640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32cc0b2950 0x7f32cc0b4e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x559f2e685550 tx=0x7f32d8000bc0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.380+0000 7f32eac00640 1 -- 172.21.15.63:0/863446074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e4104c50 msgr2=0x7f32e419c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.381+0000 7f32eac00640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e4104c50 0x7f32e419c7d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f32d800d980 tx=0x7f32d800de40 comp rx=0 tx=0).stop 2024-05-27T07:38:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.381+0000 7f32eac00640 1 -- 172.21.15.63:0/863446074 shutdown_connections 2024-05-27T07:38:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.381+0000 7f32eac00640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32cc0b2950 0x7f32cc0b4e60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.381+0000 7f32eac00640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e410e010 0x7f32e41a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.381+0000 7f32eac00640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e410b6b0 0x7f32e419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.381+0000 7f32eac00640 1 --2- 172.21.15.63:0/863446074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32e4104c50 0x7f32e419c7d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.381+0000 7f32eac00640 1 -- 172.21.15.63:0/863446074 >> 172.21.15.63:0/863446074 conn(0x7f32e40fe210 msgr2=0x7f32e410e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.381+0000 7f32eac00640 1 -- 172.21.15.63:0/863446074 shutdown_connections 2024-05-27T07:38:25.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:25.381+0000 7f32eac00640 1 -- 172.21.15.63:0/863446074 wait complete. 2024-05-27T07:38:25.877 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:38:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:38:27.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:26 smithi086 ceph-mon[34687]: pgmap v149: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:38:27.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:26 smithi149 ceph-mon[35408]: pgmap v149: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:38:27.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:26 smithi063 ceph-mon[31613]: pgmap v149: 129 pgs: 129 active+clean; 614 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:38:29.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.042+0000 7f8bafe00700 1 -- 172.21.15.63:0/1419565191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bb0105bd0 msgr2=0x7f8bb010e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:29.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.042+0000 7f8bafe00700 1 --2- 172.21.15.63:0/1419565191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bb0105bd0 0x7f8bb010e120 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009990 tx=0x7f8b9c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.042+0000 7f8bafe00700 1 -- 172.21.15.63:0/1419565191 shutdown_connections 2024-05-27T07:38:29.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.042+0000 7f8bafe00700 1 --2- 172.21.15.63:0/1419565191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb010e6f0 0x7f8bb0110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.042+0000 7f8bafe00700 1 --2- 172.21.15.63:0/1419565191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb0105280 0x7f8bb0105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.042+0000 7f8bafe00700 1 --2- 172.21.15.63:0/1419565191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bb0105bd0 0x7f8bb010e120 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.043+0000 7f8bafe00700 1 -- 172.21.15.63:0/1419565191 >> 172.21.15.63:0/1419565191 conn(0x7f8bb0078ab0 msgr2=0x7f8bb0078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:29.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.043+0000 7f8bafe00700 1 -- 172.21.15.63:0/1419565191 shutdown_connections 2024-05-27T07:38:29.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.043+0000 7f8bafe00700 1 -- 172.21.15.63:0/1419565191 wait complete. 2024-05-27T07:38:29.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.044+0000 7f8bafe00700 1 Processor -- start 2024-05-27T07:38:29.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.044+0000 7f8bafe00700 1 -- start start 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8bafe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bb0105280 0x7f8bb019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8bafe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105bd0 0x7f8bb019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8bafe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb010e6f0 0x7f8bb01a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8bafe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0114120 con 0x7f8bb0105280 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8bafe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0113fa0 con 0x7f8bb010e6f0 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8bafe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01142a0 con 0x7f8bb0105bd0 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8baea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bb0105280 0x7f8bb019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8baea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bb0105280 0x7f8bb019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48170/0 (socket says 172.21.15.63:48170) 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8baea00700 1 -- 172.21.15.63:0/3697883066 learned_addr learned my addr 172.21.15.63:0/3697883066 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8bae000700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105bd0 0x7f8bb019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.045+0000 7f8baf400700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb010e6f0 0x7f8bb01a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8baea00700 1 -- 172.21.15.63:0/3697883066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105bd0 msgr2=0x7f8bb019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:29.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8baea00700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105bd0 0x7f8bb019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8baea00700 1 -- 172.21.15.63:0/3697883066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb010e6f0 msgr2=0x7f8bb01a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:29.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8baea00700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb010e6f0 0x7f8bb01a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8baea00700 1 -- 172.21.15.63:0/3697883066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb01a1d60 con 0x7f8bb0105280 2024-05-27T07:38:29.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8bae000700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105bd0 0x7f8bb019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:29.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8baea00700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bb0105280 0x7f8bb019ce90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000d980 tx=0x7f8ba000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:29.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8ba7400700 1 -- 172.21.15.63:0/3697883066 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0016070 con 0x7f8bb0105280 2024-05-27T07:38:29.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8ba7400700 1 -- 172.21.15.63:0/3697883066 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ba0004020 con 0x7f8bb0105280 2024-05-27T07:38:29.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8bafe00700 1 -- 172.21.15.63:0/3697883066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb01a2050 con 0x7f8bb0105280 2024-05-27T07:38:29.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.046+0000 7f8ba7400700 1 -- 172.21.15.63:0/3697883066 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0004bb0 con 0x7f8bb0105280 2024-05-27T07:38:29.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.047+0000 7f8bafe00700 1 -- 172.21.15.63:0/3697883066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb01a9970 con 0x7f8bb0105280 2024-05-27T07:38:29.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.052+0000 7f8bc408bb80 1 Processor -- start 2024-05-27T07:38:29.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.052+0000 7f8bc408bb80 1 -- start start 2024-05-27T07:38:29.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.052+0000 7f8ba7400700 1 -- 172.21.15.63:0/3697883066 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8ba000bb60 con 0x7f8bb0105280 2024-05-27T07:38:29.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e3683750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e3683c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f9e35b0e40 con 0x55f9e368d2c0 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9e35a09e0 con 0x55f9e35ef9f0 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f9e35a4a20 con 0x55f9e3503e40 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8bae000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8bae000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48180/0 (socket says 172.21.15.63:48180) 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8bae000700 1 -- 172.21.15.63:0/2808916314 learned_addr learned my addr 172.21.15.63:0/2808916314 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8baf400700 1 --2- 172.21.15.63:0/2808916314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e3683750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.053+0000 7f8baea00700 1 --2- 172.21.15.63:0/2808916314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e3683c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.054+0000 7f8bae000700 1 -- 172.21.15.63:0/2808916314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 msgr2=0x55f9e3683c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.054+0000 7f8bae000700 1 --2- 172.21.15.63:0/2808916314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e3683c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.054+0000 7f8bae000700 1 -- 172.21.15.63:0/2808916314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 msgr2=0x55f9e3683750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:29.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.054+0000 7f8bae000700 1 --2- 172.21.15.63:0/2808916314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e3683750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.054+0000 7f8bae000700 1 -- 172.21.15.63:0/2808916314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e3569420 con 0x55f9e368d2c0 2024-05-27T07:38:29.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.054+0000 7f8bae000700 1 --2- 172.21.15.63:0/2808916314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368f6e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c015000 tx=0x7f8b9c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414f732b88e7d6aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.055+0000 7f8b9be00700 1 -- 172.21.15.63:0/2808916314 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c005160 con 0x55f9e368d2c0 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.055+0000 7f8ba7400700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b940b29f0 0x7f8b940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.055+0000 7f8b9be00700 1 -- 172.21.15.63:0/2808916314 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b9c0037c0 con 0x55f9e368d2c0 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.055+0000 7f8ba7400700 1 -- 172.21.15.63:0/3697883066 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f8ba00e2840 con 0x7f8bb0105280 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.055+0000 7f8b9be00700 1 -- 172.21.15.63:0/2808916314 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c00b9d0 con 0x55f9e368d2c0 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.055+0000 7f8baf400700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b940b29f0 0x7f8b940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.056+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2808916314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 msgr2=0x55f9e368f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.056+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2808916314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368f6e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c015000 tx=0x7f8b9c0096a0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.056+0000 7f8baf400700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b940b29f0 0x7f8b940b4f00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55f9e367fa90 tx=0x7f8ba8009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.056+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2808916314 shutdown_connections 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.056+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2808916314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368f6e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.056+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2808916314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e3683750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.056+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2808916314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e3683c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.056+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2808916314 >> 172.21.15.63:0/2808916314 conn(0x55f9e359d390 msgr2=0x55f9e367e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:29.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.056+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2808916314 shutdown_connections 2024-05-27T07:38:29.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.056+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2808916314 wait complete. 2024-05-27T07:38:29.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.057+0000 7f8bc408bb80 1 -- start start 2024-05-27T07:38:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.057+0000 7f8bc408bb80 1 Processor -- start 2024-05-27T07:38:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.058+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e368a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.058+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e368a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.058+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.058+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f9e34a6fa0 con 0x55f9e368d2c0 2024-05-27T07:38:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.058+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9e3541760 con 0x55f9e35ef9f0 2024-05-27T07:38:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.058+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f9e368fda0 con 0x55f9e3503e40 2024-05-27T07:38:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.058+0000 7f8bc408bb80 10 client.0 _ll_register_callbacks cb 0x55f9e354ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:29.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.059+0000 7f8baf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.059+0000 7f8baf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48188/0 (socket says 172.21.15.63:48188) 2024-05-27T07:38:29.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.059+0000 7f8baf400700 1 -- 172.21.15.63:0/2940270127 learned_addr learned my addr 172.21.15.63:0/2940270127 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:29.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.059+0000 7f8bae000700 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e368a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.059+0000 7f8baea00700 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e368a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.059+0000 7f8baf400700 1 -- 172.21.15.63:0/2940270127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 msgr2=0x55f9e368a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:29.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.059+0000 7f8baf400700 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e368a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.059+0000 7f8baf400700 1 -- 172.21.15.63:0/2940270127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 msgr2=0x55f9e368a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:29.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.059+0000 7f8baf400700 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e368a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.059+0000 7f8baf400700 1 -- 172.21.15.63:0/2940270127 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e34f08d0 con 0x55f9e368d2c0 2024-05-27T07:38:29.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.060+0000 7f8baf400700 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368abe0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009fd0 tx=0x7f8ba8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab50c29dd92f65 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:29.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.060+0000 7f8b98c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:29.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.060+0000 7f8b98c00700 1 -- 172.21.15.63:0/2940270127 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba801e050 con 0x55f9e368d2c0 2024-05-27T07:38:29.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.060+0000 7f8b98c00700 1 -- 172.21.15.63:0/2940270127 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f8ba8004bb0 con 0x55f9e368d2c0 2024-05-27T07:38:29.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.060+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2940270127 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b9c0037c0 con 0x55f9e368d2c0 2024-05-27T07:38:29.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.061+0000 7f8b98c00700 1 -- 172.21.15.63:0/2940270127 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8ba8030050 con 0x55f9e368d2c0 2024-05-27T07:38:29.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.061+0000 7f8bc408bb80 10 client.16335 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:29.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.061+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2940270127 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f9e3571060 con 0x55f9e368d2c0 2024-05-27T07:38:29.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.061+0000 7f8b98c00700 1 -- 172.21.15.63:0/2940270127 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b9c0037c0 con 0x55f9e368d2c0 2024-05-27T07:38:29.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.062+0000 7f8b98c00700 1 -- 172.21.15.63:0/2940270127 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8ba80235d0 con 0x55f9e368d2c0 2024-05-27T07:38:29.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.062+0000 7f8bc408bb80 10 client.16335 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:29.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.062+0000 7f8bc408bb80 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:29.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.062+0000 7f8bc408bb80 20 client.16335 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:29.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.062+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f9e369cc30 0x55f9e369f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.062+0000 7f8bc408bb80 4 client.16335 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:38:29.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.062+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2940270127 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9e35b2d90 con 0x55f9e369cc30 2024-05-27T07:38:29.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.063+0000 7f8baea00700 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f9e369cc30 0x55f9e369f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.064+0000 7f8b98c00700 10 client.16335 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:29.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.064+0000 7f8baea00700 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f9e369cc30 0x55f9e369f0e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c50cfdf69e71a0 server_cookie=ccaae9ca9f0d9676 in_seq=0 out_seq=0 2024-05-27T07:38:29.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8b98c00700 1 -- 172.21.15.63:0/2940270127 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f9e35b2d90 con 0x55f9e369cc30 2024-05-27T07:38:29.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8b98c00700 10 client.16335 handle_command_reply: tid=0 2024-05-27T07:38:29.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8bc408bb80 1 client.16335 shutdown 2024-05-27T07:38:29.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2940270127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 msgr2=0x55f9e368abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:29.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368abe0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009fd0 tx=0x7f8ba8004ad0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2940270127 shutdown_connections 2024-05-27T07:38:29.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f9e369cc30 0x55f9e369f0e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e368abe0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e368a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2940270127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e368a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2940270127 >> 172.21.15.63:0/2940270127 conn(0x55f9e359d390 msgr2=0x55f9e367e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:29.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8b98c00700 0 client.16335 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:29.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.066+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2940270127 shutdown_connections 2024-05-27T07:38:29.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.067+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2940270127 wait complete. 2024-05-27T07:38:29.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.067+0000 7f8bc408bb80 20 client.16335 trim_cache size 0 max 16384 2024-05-27T07:38:29.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.093+0000 7f8bc408bb80 1 Processor -- start 2024-05-27T07:38:29.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.093+0000 7f8bc408bb80 1 -- start start 2024-05-27T07:38:29.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.093+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e367e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.093+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e3698e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.093+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e36a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.093+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01b4c70 con 0x55f9e368d2c0 2024-05-27T07:38:29.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.093+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01b2500 con 0x55f9e35ef9f0 2024-05-27T07:38:29.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.093+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01b0350 con 0x55f9e3503e40 2024-05-27T07:38:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.094+0000 7f8baf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e36a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.094+0000 7f8baf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e36a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48194/0 (socket says 172.21.15.63:48194) 2024-05-27T07:38:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.094+0000 7f8baf400700 1 -- 172.21.15.63:0/2305950236 learned_addr learned my addr 172.21.15.63:0/2305950236 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.094+0000 7f8bae000700 1 --2- 172.21.15.63:0/2305950236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e3698e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.094+0000 7f8baea00700 1 --2- 172.21.15.63:0/2305950236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e367e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.094+0000 7f8baf400700 1 -- 172.21.15.63:0/2305950236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 msgr2=0x55f9e367e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.094+0000 7f8baf400700 1 --2- 172.21.15.63:0/2305950236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e367e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.094+0000 7f8baf400700 1 -- 172.21.15.63:0/2305950236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 msgr2=0x55f9e3698e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.095+0000 7f8baf400700 1 --2- 172.21.15.63:0/2305950236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e3698e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.095+0000 7f8baf400700 1 -- 172.21.15.63:0/2305950236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e368af50 con 0x55f9e368d2c0 2024-05-27T07:38:29.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.095+0000 7f8baf400700 1 --2- 172.21.15.63:0/2305950236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e36a48f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8007fd0 tx=0x7f8ba8019fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2d5b613ae26ef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:29.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.095+0000 7f8b9b400700 1 -- 172.21.15.63:0/2305950236 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba801e040 con 0x55f9e368d2c0 2024-05-27T07:38:29.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.095+0000 7f8b9b400700 1 -- 172.21.15.63:0/2305950236 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ba801f040 con 0x55f9e368d2c0 2024-05-27T07:38:29.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.096+0000 7f8b9b400700 1 -- 172.21.15.63:0/2305950236 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8016980 con 0x55f9e368d2c0 2024-05-27T07:38:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.097+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2305950236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 msgr2=0x55f9e36a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.097+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2305950236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e36a48f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8007fd0 tx=0x7f8ba8019fa0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.097+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2305950236 shutdown_connections 2024-05-27T07:38:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.097+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2305950236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e36a48f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.097+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2305950236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e367e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.097+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/2305950236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e3698e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.097+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2305950236 >> 172.21.15.63:0/2305950236 conn(0x55f9e359d390 msgr2=0x55f9e3683ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:29.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.097+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2305950236 shutdown_connections 2024-05-27T07:38:29.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.097+0000 7f8bc408bb80 1 -- 172.21.15.63:0/2305950236 wait complete. 2024-05-27T07:38:29.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.097+0000 7f8bc408bb80 1 -- start start 2024-05-27T07:38:29.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.098+0000 7f8bc408bb80 1 Processor -- start 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.098+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e3695380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.098+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e36958c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.098+0000 7f8bc408bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e3690840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.098+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01b09c0 con 0x55f9e368d2c0 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.098+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01b1e30 con 0x55f9e35ef9f0 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.098+0000 7f8bc408bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01b1090 con 0x55f9e3503e40 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.098+0000 7f8bc408bb80 10 client.0 _ll_register_callbacks cb 0x55f9e354ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8baf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e3690840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8baf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e3690840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48198/0 (socket says 172.21.15.63:48198) 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8baf400700 1 -- 172.21.15.63:0/1132330053 learned_addr learned my addr 172.21.15.63:0/1132330053 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8baea00700 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e3695380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8bae000700 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e36958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8baf400700 1 -- 172.21.15.63:0/1132330053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 msgr2=0x55f9e3695380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:29.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8baf400700 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e3695380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8baf400700 1 -- 172.21.15.63:0/1132330053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 msgr2=0x55f9e36958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:29.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8baf400700 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e36958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8baf400700 1 -- 172.21.15.63:0/1132330053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e3690fc0 con 0x55f9e368d2c0 2024-05-27T07:38:29.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8baea00700 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e3695380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:29.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.099+0000 7f8bae000700 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e36958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:29.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.100+0000 7f8baf400700 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e3690840 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8005e90 tx=0x7f8ba8005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5d1ec659ace4d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:29.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.100+0000 7f8b98c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:29.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.100+0000 7f8b98c00700 1 -- 172.21.15.63:0/1132330053 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8040020 con 0x55f9e368d2c0 2024-05-27T07:38:29.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.100+0000 7f8bc408bb80 1 -- 172.21.15.63:0/1132330053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ba801f040 con 0x55f9e368d2c0 2024-05-27T07:38:29.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.100+0000 7f8b98c00700 1 -- 172.21.15.63:0/1132330053 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f8ba8023440 con 0x55f9e368d2c0 2024-05-27T07:38:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.101+0000 7f8b98c00700 1 -- 172.21.15.63:0/1132330053 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8ba804b050 con 0x55f9e368d2c0 2024-05-27T07:38:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.101+0000 7f8bc408bb80 10 client.16341 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.101+0000 7f8bc408bb80 1 -- 172.21.15.63:0/1132330053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f9e3707310 con 0x55f9e368d2c0 2024-05-27T07:38:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.101+0000 7f8b98c00700 1 -- 172.21.15.63:0/1132330053 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ba801f040 con 0x55f9e368d2c0 2024-05-27T07:38:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.101+0000 7f8b98c00700 1 -- 172.21.15.63:0/1132330053 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8ba803c440 con 0x55f9e368d2c0 2024-05-27T07:38:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.101+0000 7f8bc408bb80 10 client.16341 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.101+0000 7f8bc408bb80 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.101+0000 7f8bc408bb80 20 client.16341 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.101+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f9e3707900 0x55f9e3709d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.102+0000 7f8bc408bb80 4 client.16341 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.102+0000 7f8bc408bb80 1 -- 172.21.15.63:0/1132330053 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f9e35ac570 con 0x55f9e3707900 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.102+0000 7f8baea00700 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f9e3707900 0x55f9e3709d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.103+0000 7f8baea00700 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f9e3707900 0x55f9e3709d20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b125cb410bf1c6b0 server_cookie=dff1faf40f611e61 in_seq=0 out_seq=0 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.103+0000 7f8b98c00700 10 client.16341 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8b98c00700 1 -- 172.21.15.63:0/1132330053 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f9e35ac570 con 0x55f9e3707900 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8b98c00700 10 client.16341 handle_command_reply: tid=0 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 client.16341 shutdown 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 -- 172.21.15.63:0/1132330053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 msgr2=0x55f9e3690840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e3690840 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8005e90 tx=0x7f8ba8005f70 comp rx=0 tx=0).stop 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 -- 172.21.15.63:0/1132330053 shutdown_connections 2024-05-27T07:38:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f9e3707900 0x55f9e3709d20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f9e368d2c0 0x55f9e3690840 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9e3503e40 0x55f9e3695380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 --2- 172.21.15.63:0/1132330053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9e35ef9f0 0x55f9e36958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8b98c00700 0 client.16341 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:29.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 -- 172.21.15.63:0/1132330053 >> 172.21.15.63:0/1132330053 conn(0x55f9e359d390 msgr2=0x55f9e3689f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:29.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 -- 172.21.15.63:0/1132330053 shutdown_connections 2024-05-27T07:38:29.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.104+0000 7f8bc408bb80 1 -- 172.21.15.63:0/1132330053 wait complete. 2024-05-27T07:38:29.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.105+0000 7f8bc408bb80 20 client.16341 trim_cache size 0 max 16384 2024-05-27T07:38:29.104 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:38:29.104 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:38:29.104 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:38:29.104 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:38:29.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 -- 172.21.15.63:0/3697883066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b940b29f0 msgr2=0x7f8b940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:29.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b940b29f0 0x7f8b940b4f00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55f9e367fa90 tx=0x7f8ba8009040 comp rx=0 tx=0).stop 2024-05-27T07:38:29.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 -- 172.21.15.63:0/3697883066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bb0105280 msgr2=0x7f8bb019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:29.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bb0105280 0x7f8bb019ce90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000d980 tx=0x7f8ba000dd40 comp rx=0 tx=0).stop 2024-05-27T07:38:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 -- 172.21.15.63:0/3697883066 shutdown_connections 2024-05-27T07:38:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b940b29f0 0x7f8b940b4f00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bb010e6f0 0x7f8bb01a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bb0105280 0x7f8bb019ce90 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 --2- 172.21.15.63:0/3697883066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8bb0105bd0 0x7f8bb019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 -- 172.21.15.63:0/3697883066 >> 172.21.15.63:0/3697883066 conn(0x7f8bb0078ab0 msgr2=0x7f8bb0103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 -- 172.21.15.63:0/3697883066 shutdown_connections 2024-05-27T07:38:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:29.106+0000 7f8bafe00700 1 -- 172.21.15.63:0/3697883066 wait complete. 2024-05-27T07:38:29.160 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:28 smithi063 ceph-mon[31613]: pgmap v150: 129 pgs: 129 active+clean; 638 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:38:29.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:28 smithi086 ceph-mon[34687]: pgmap v150: 129 pgs: 129 active+clean; 638 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:38:29.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:28 smithi149 ceph-mon[35408]: pgmap v150: 129 pgs: 129 active+clean; 638 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:38:30.045 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:38:30.045 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:38:30.560 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:38:31.049 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:30 smithi063 ceph-mon[31613]: pgmap v151: 129 pgs: 129 active+clean; 638 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:38:31.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:30 smithi086 ceph-mon[34687]: pgmap v151: 129 pgs: 129 active+clean; 638 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:38:31.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:30 smithi149 ceph-mon[35408]: pgmap v151: 129 pgs: 129 active+clean; 638 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:38:32.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.872+0000 7f258cc00700 1 -- 172.21.15.63:0/3955369036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2588109860 msgr2=0x7f25881111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:32.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.872+0000 7f258cc00700 1 --2- 172.21.15.63:0/3955369036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2588109860 0x7f25881111c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f258000b0d0 tx=0x7f258000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:38:32.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.872+0000 7f258cc00700 1 -- 172.21.15.63:0/3955369036 shutdown_connections 2024-05-27T07:38:32.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.872+0000 7f258cc00700 1 --2- 172.21.15.63:0/3955369036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2588109860 0x7f25881111c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:32.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.872+0000 7f258cc00700 1 --2- 172.21.15.63:0/3955369036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25881045e0 0x7f25881049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:32.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.872+0000 7f258cc00700 1 --2- 172.21.15.63:0/3955369036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2588108f10 0x7f2588109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:32.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.872+0000 7f258cc00700 1 -- 172.21.15.63:0/3955369036 >> 172.21.15.63:0/3955369036 conn(0x7f2588100030 msgr2=0x7f2588102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:32.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.873+0000 7f258cc00700 1 -- 172.21.15.63:0/3955369036 shutdown_connections 2024-05-27T07:38:32.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.873+0000 7f258cc00700 1 -- 172.21.15.63:0/3955369036 wait complete. 2024-05-27T07:38:32.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.874+0000 7f258cc00700 1 Processor -- start 2024-05-27T07:38:32.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.874+0000 7f258cc00700 1 -- start start 2024-05-27T07:38:32.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.874+0000 7f258cc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25881045e0 0x7f258819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:32.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.875+0000 7f258cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2588108f10 0x7f258819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:32.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.875+0000 7f258cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2588109860 0x7f25881a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:32.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.875+0000 7f258cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2588114340 con 0x7f2588108f10 2024-05-27T07:38:32.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.875+0000 7f258cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25881141c0 con 0x7f2588109860 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.875+0000 7f258cc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25881144c0 con 0x7f25881045e0 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.875+0000 7f2585400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2588108f10 0x7f258819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.875+0000 7f2585400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2588108f10 0x7f258819d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48216/0 (socket says 172.21.15.63:48216) 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.875+0000 7f2585400700 1 -- 172.21.15.63:0/3918334521 learned_addr learned my addr 172.21.15.63:0/3918334521 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.875+0000 7f2585e00700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25881045e0 0x7f258819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.875+0000 7f2586800700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2588109860 0x7f25881a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.876+0000 7f2585400700 1 -- 172.21.15.63:0/3918334521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25881045e0 msgr2=0x7f258819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.876+0000 7f2585400700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25881045e0 0x7f258819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.876+0000 7f2585400700 1 -- 172.21.15.63:0/3918334521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2588109860 msgr2=0x7f25881a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.876+0000 7f2585400700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2588109860 0x7f25881a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:32.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.876+0000 7f2585400700 1 -- 172.21.15.63:0/3918334521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25881a1d50 con 0x7f2588108f10 2024-05-27T07:38:32.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.876+0000 7f2585400700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2588108f10 0x7f258819d4d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f257400c8f0 tx=0x7f257400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:32.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.876+0000 7f257ea00700 1 -- 172.21.15.63:0/3918334521 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740078d0 con 0x7f2588108f10 2024-05-27T07:38:32.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.877+0000 7f257ea00700 1 -- 172.21.15.63:0/3918334521 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f257400f450 con 0x7f2588108f10 2024-05-27T07:38:32.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.877+0000 7f258cc00700 1 -- 172.21.15.63:0/3918334521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25881a2040 con 0x7f2588108f10 2024-05-27T07:38:32.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.877+0000 7f257ea00700 1 -- 172.21.15.63:0/3918334521 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574018660 con 0x7f2588108f10 2024-05-27T07:38:32.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.877+0000 7f258cc00700 1 -- 172.21.15.63:0/3918334521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25881a2580 con 0x7f2588108f10 2024-05-27T07:38:32.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.878+0000 7f258cc00700 1 -- 172.21.15.63:0/3918334521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25880512b0 con 0x7f2588108f10 2024-05-27T07:38:32.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.881+0000 7f257ea00700 1 -- 172.21.15.63:0/3918334521 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f257400f900 con 0x7f2588108f10 2024-05-27T07:38:32.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.883+0000 7f257ea00700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f256c0b29f0 0x7f256c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:32.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.883+0000 7f257ea00700 1 -- 172.21.15.63:0/3918334521 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f25740e2780 con 0x7f2588108f10 2024-05-27T07:38:32.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.884+0000 7f2585e00700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f256c0b29f0 0x7f256c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:32.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.885+0000 7f2585e00700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f256c0b29f0 0x7f256c0b4f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f257800ac50 tx=0x7f257800a380 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:32.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:32.889+0000 7f257ea00700 1 -- 172.21.15.63:0/3918334521 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2574014030 con 0x7f2588108f10 2024-05-27T07:38:33.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:32 smithi086 ceph-mon[34687]: pgmap v152: 129 pgs: 129 active+clean; 674 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T07:38:33.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:32 smithi149 ceph-mon[35408]: pgmap v152: 129 pgs: 129 active+clean; 674 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T07:38:33.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:32 smithi063 ceph-mon[31613]: pgmap v152: 129 pgs: 129 active+clean; 674 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T07:38:33.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.238+0000 7f258cc00700 1 -- 172.21.15.63:0/3918334521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25881a9b00 con 0x7f2588108f10 2024-05-27T07:38:33.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.240+0000 7f257ea00700 1 -- 172.21.15.63:0/3918334521 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f25740a9d90 con 0x7f2588108f10 2024-05-27T07:38:33.237 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:38:33.237 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:38:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.244+0000 7f258cc00700 1 -- 172.21.15.63:0/3918334521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f256c0b29f0 msgr2=0x7f256c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.244+0000 7f258cc00700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f256c0b29f0 0x7f256c0b4f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f257800ac50 tx=0x7f257800a380 comp rx=0 tx=0).stop 2024-05-27T07:38:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.244+0000 7f258cc00700 1 -- 172.21.15.63:0/3918334521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2588108f10 msgr2=0x7f258819d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.244+0000 7f258cc00700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2588108f10 0x7f258819d4d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f257400c8f0 tx=0x7f257400cc00 comp rx=0 tx=0).stop 2024-05-27T07:38:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.244+0000 7f258cc00700 1 -- 172.21.15.63:0/3918334521 shutdown_connections 2024-05-27T07:38:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.244+0000 7f258cc00700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f256c0b29f0 0x7f256c0b4f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.244+0000 7f258cc00700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2588109860 0x7f25881a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.244+0000 7f258cc00700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25881045e0 0x7f258819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.244+0000 7f258cc00700 1 --2- 172.21.15.63:0/3918334521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2588108f10 0x7f258819d4d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.244+0000 7f258cc00700 1 -- 172.21.15.63:0/3918334521 >> 172.21.15.63:0/3918334521 conn(0x7f2588100030 msgr2=0x7f2588110ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:33.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.245+0000 7f258cc00700 1 -- 172.21.15.63:0/3918334521 shutdown_connections 2024-05-27T07:38:33.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:33.245+0000 7f258cc00700 1 -- 172.21.15.63:0/3918334521 wait complete. 2024-05-27T07:38:33.242 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:38:34.073 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:38:34.161 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:33 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3918334521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:38:34.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:33 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3918334521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:38:34.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:33 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3918334521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:38:34.603 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:38:35.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:34 smithi063 ceph-mon[31613]: pgmap v153: 129 pgs: 129 active+clean; 674 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:35.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:34 smithi086 ceph-mon[34687]: pgmap v153: 129 pgs: 129 active+clean; 674 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:35.210 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:34 smithi149 ceph-mon[35408]: pgmap v153: 129 pgs: 129 active+clean; 674 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:37.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.044+0000 7fedab400700 1 -- 172.21.15.63:0/2045293940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda4103a20 msgr2=0x7feda4110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:37.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.044+0000 7fedab400700 1 --2- 172.21.15.63:0/2045293940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda4103a20 0x7feda4110740 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fed90009990 tx=0x7fed90009ca0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.044+0000 7fedab400700 1 -- 172.21.15.63:0/2045293940 shutdown_connections 2024-05-27T07:38:37.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.044+0000 7fedab400700 1 --2- 172.21.15.63:0/2045293940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda4110c80 0x7feda41130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.044+0000 7fedab400700 1 --2- 172.21.15.63:0/2045293940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda41030d0 0x7feda41034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.044+0000 7fedab400700 1 --2- 172.21.15.63:0/2045293940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda4103a20 0x7feda4110740 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.044+0000 7fedab400700 1 -- 172.21.15.63:0/2045293940 >> 172.21.15.63:0/2045293940 conn(0x7feda40fedb0 msgr2=0x7feda41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:37.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.044+0000 7fedab400700 1 -- 172.21.15.63:0/2045293940 shutdown_connections 2024-05-27T07:38:37.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.044+0000 7fedab400700 1 -- 172.21.15.63:0/2045293940 wait complete. 2024-05-27T07:38:37.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.045+0000 7fedab400700 1 Processor -- start 2024-05-27T07:38:37.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.046+0000 7fedab400700 1 -- start start 2024-05-27T07:38:37.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.046+0000 7fedab400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda41030d0 0x7feda419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.046+0000 7fedab400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda4103a20 0x7feda419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.046+0000 7fedab400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda4110c80 0x7feda41a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.046+0000 7fedab400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feda41164f0 con 0x7feda41030d0 2024-05-27T07:38:37.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.046+0000 7fedab400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feda4116370 con 0x7feda4110c80 2024-05-27T07:38:37.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.046+0000 7fedab400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feda4116670 con 0x7feda4103a20 2024-05-27T07:38:37.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.046+0000 7feda8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda41030d0 0x7feda419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.047+0000 7feda8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda41030d0 0x7feda419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48236/0 (socket says 172.21.15.63:48236) 2024-05-27T07:38:37.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.047+0000 7feda8a00700 1 -- 172.21.15.63:0/94739495 learned_addr learned my addr 172.21.15.63:0/94739495 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:37.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.047+0000 7feda3e00700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda4103a20 0x7feda419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.047+0000 7feda9400700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda4110c80 0x7feda41a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.047+0000 7feda8a00700 1 -- 172.21.15.63:0/94739495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda4103a20 msgr2=0x7feda419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:37.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.047+0000 7feda8a00700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda4103a20 0x7feda419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.047+0000 7feda8a00700 1 -- 172.21.15.63:0/94739495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda4110c80 msgr2=0x7feda41a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:37.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.047+0000 7feda8a00700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda4110c80 0x7feda41a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.047+0000 7feda8a00700 1 -- 172.21.15.63:0/94739495 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda41a1df0 con 0x7feda41030d0 2024-05-27T07:38:37.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.047+0000 7feda9400700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda4110c80 0x7feda41a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:37.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.048+0000 7feda8a00700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda41030d0 0x7feda419cf20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fed9800d950 tx=0x7fed9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:37.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.048+0000 7feda1600700 1 -- 172.21.15.63:0/94739495 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98016070 con 0x7feda41030d0 2024-05-27T07:38:37.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.048+0000 7fedab400700 1 -- 172.21.15.63:0/94739495 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda41a20e0 con 0x7feda41030d0 2024-05-27T07:38:37.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.048+0000 7fedab400700 1 -- 172.21.15.63:0/94739495 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda41a9a00 con 0x7feda41030d0 2024-05-27T07:38:37.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.048+0000 7feda1600700 1 -- 172.21.15.63:0/94739495 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed98009dd0 con 0x7feda41030d0 2024-05-27T07:38:37.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.048+0000 7feda1600700 1 -- 172.21.15.63:0/94739495 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9800f3c0 con 0x7feda41030d0 2024-05-27T07:38:37.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.053+0000 7feda1600700 1 -- 172.21.15.63:0/94739495 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fed98019070 con 0x7feda41030d0 2024-05-27T07:38:37.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.054+0000 7fedb9eefb80 1 Processor -- start 2024-05-27T07:38:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.054+0000 7fedb9eefb80 1 -- start start 2024-05-27T07:38:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.055+0000 7feda1600700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed8c0b2a70 0x7fed8c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.055+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e30c8e40 0x5641e324d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.055+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e324dd30 0x5641e3247870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.055+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e32481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.055+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5641e3175e40 con 0x5641e3247db0 2024-05-27T07:38:37.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.055+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5641e31659e0 con 0x5641e324dd30 2024-05-27T07:38:37.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.055+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641e3169a20 con 0x5641e30c8e40 2024-05-27T07:38:37.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.055+0000 7feda1600700 1 -- 172.21.15.63:0/94739495 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fed980e29b0 con 0x7feda41030d0 2024-05-27T07:38:37.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.055+0000 7feda9400700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed8c0b2a70 0x7fed8c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.056+0000 7feda9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e32481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.056+0000 7feda3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e324dd30 0x5641e3247870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.056+0000 7feda9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e32481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48248/0 (socket says 172.21.15.63:48248) 2024-05-27T07:38:37.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.056+0000 7feda9400700 1 -- 172.21.15.63:0/3232143783 learned_addr learned my addr 172.21.15.63:0/3232143783 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:37.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.056+0000 7feda8a00700 1 --2- 172.21.15.63:0/3232143783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e30c8e40 0x5641e324d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.056+0000 7feda9400700 1 -- 172.21.15.63:0/3232143783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e30c8e40 msgr2=0x5641e324d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:37.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.056+0000 7feda9400700 1 --2- 172.21.15.63:0/3232143783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e30c8e40 0x5641e324d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.056+0000 7feda9400700 1 -- 172.21.15.63:0/3232143783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e324dd30 msgr2=0x5641e3247870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.056+0000 7feda9400700 1 --2- 172.21.15.63:0/3232143783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e324dd30 0x5641e3247870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.056+0000 7feda9400700 1 -- 172.21.15.63:0/3232143783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e312e420 con 0x5641e3247db0 2024-05-27T07:38:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.057+0000 7feda9400700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed8c0b2a70 0x7fed8c0b4f80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009fd0 tx=0x7fed9c009d20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.057+0000 7feda9400700 1 --2- 172.21.15.63:0/3232143783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e32481c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0189d0 tx=0x7fed9c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22486dd8f950809 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.057+0000 7fed96000700 1 -- 172.21.15.63:0/3232143783 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c007500 con 0x5641e3247db0 2024-05-27T07:38:37.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.058+0000 7fed96000700 1 -- 172.21.15.63:0/3232143783 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed9c027070 con 0x5641e3247db0 2024-05-27T07:38:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.058+0000 7fed96000700 1 -- 172.21.15.63:0/3232143783 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c022a40 con 0x5641e3247db0 2024-05-27T07:38:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.059+0000 7fedb9eefb80 1 -- 172.21.15.63:0/3232143783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 msgr2=0x5641e32481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:37.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.059+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/3232143783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e32481c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0189d0 tx=0x7fed9c009680 comp rx=0 tx=0).stop 2024-05-27T07:38:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.059+0000 7fedb9eefb80 1 -- 172.21.15.63:0/3232143783 shutdown_connections 2024-05-27T07:38:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.059+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/3232143783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e32481c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.059+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/3232143783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e30c8e40 0x5641e324d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.059+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/3232143783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e324dd30 0x5641e3247870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.059+0000 7fedb9eefb80 1 -- 172.21.15.63:0/3232143783 >> 172.21.15.63:0/3232143783 conn(0x5641e3128fc0 msgr2=0x5641e32428d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:37.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.059+0000 7fedb9eefb80 1 -- 172.21.15.63:0/3232143783 shutdown_connections 2024-05-27T07:38:37.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.060+0000 7fedb9eefb80 1 -- 172.21.15.63:0/3232143783 wait complete. 2024-05-27T07:38:37.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.060+0000 7fedb9eefb80 1 -- start start 2024-05-27T07:38:37.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.060+0000 7fedb9eefb80 1 Processor -- start 2024-05-27T07:38:37.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.061+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 0x5641e32514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.061+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e3247db0 0x5641e32519e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.061+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e3251f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.061+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5641e306bfa0 con 0x5641e324dd30 2024-05-27T07:38:37.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.061+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5641e3106760 con 0x5641e30c8e40 2024-05-27T07:38:37.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.061+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641e32547a0 con 0x5641e3247db0 2024-05-27T07:38:37.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.061+0000 7fedb9eefb80 10 client.0 _ll_register_callbacks cb 0x5641e310fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:37.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.062+0000 7feda9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e3251f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.062+0000 7feda9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e3251f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48260/0 (socket says 172.21.15.63:48260) 2024-05-27T07:38:37.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.062+0000 7feda9400700 1 -- 172.21.15.63:0/2673942069 learned_addr learned my addr 172.21.15.63:0/2673942069 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:37.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.062+0000 7feda3e00700 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e3247db0 0x5641e32519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.062+0000 7feda8a00700 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 0x5641e32514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.063+0000 7feda9400700 1 -- 172.21.15.63:0/2673942069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e3247db0 msgr2=0x5641e32519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:37.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.063+0000 7feda9400700 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e3247db0 0x5641e32519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.063+0000 7feda9400700 1 -- 172.21.15.63:0/2673942069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 msgr2=0x5641e32514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:37.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.063+0000 7feda9400700 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 0x5641e32514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.063+0000 7feda9400700 1 -- 172.21.15.63:0/2673942069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e30b58d0 con 0x5641e324dd30 2024-05-27T07:38:37.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.063+0000 7feda9400700 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e3251f20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fed9c019010 tx=0x7fed9c0076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2f13859099f9d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:37.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.063+0000 7fed8aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:37.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.064+0000 7fed8aa00700 1 -- 172.21.15.63:0/2673942069 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c02f070 con 0x5641e324dd30 2024-05-27T07:38:37.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.064+0000 7fed8aa00700 1 -- 172.21.15.63:0/2673942069 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fed9c01ab60 con 0x5641e324dd30 2024-05-27T07:38:37.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.064+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2673942069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed9c027070 con 0x5641e324dd30 2024-05-27T07:38:37.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.064+0000 7fed8aa00700 1 -- 172.21.15.63:0/2673942069 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fed9c03d050 con 0x5641e324dd30 2024-05-27T07:38:37.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.064+0000 7fedb9eefb80 10 client.16359 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:37.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.064+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2673942069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641e3136060 con 0x5641e324dd30 2024-05-27T07:38:37.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.064+0000 7fed8aa00700 1 -- 172.21.15.63:0/2673942069 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed9c027070 con 0x5641e324dd30 2024-05-27T07:38:37.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.065+0000 7fed8aa00700 1 -- 172.21.15.63:0/2673942069 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fed9c034460 con 0x5641e324dd30 2024-05-27T07:38:37.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.065+0000 7fedb9eefb80 10 client.16359 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:37.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.065+0000 7fedb9eefb80 10 client.16359 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:38:37.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.065+0000 7fedb9eefb80 20 client.16359 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:37.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.065+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5641e32615e0 0x5641e3263a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.065+0000 7fedb9eefb80 4 client.16359 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:38:37.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.065+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2673942069 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641e3177d90 con 0x5641e32615e0 2024-05-27T07:38:37.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.066+0000 7feda8a00700 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5641e32615e0 0x5641e3263a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.067+0000 7feda8a00700 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5641e32615e0 0x5641e3263a00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de902616192bfaca server_cookie=9ab235ba690461a2 in_seq=0 out_seq=0 2024-05-27T07:38:37.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.067+0000 7fed8aa00700 10 client.16359 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:38:37.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fed8aa00700 1 -- 172.21.15.63:0/2673942069 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5641e3177d90 con 0x5641e32615e0 2024-05-27T07:38:37.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fed8aa00700 10 client.16359 handle_command_reply: tid=0 2024-05-27T07:38:37.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fedb9eefb80 1 client.16359 shutdown 2024-05-27T07:38:37.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2673942069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 msgr2=0x5641e3251f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:37.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e3251f20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fed9c019010 tx=0x7fed9c0076a0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2673942069 shutdown_connections 2024-05-27T07:38:37.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5641e32615e0 0x5641e3263a00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e3251f20 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 0x5641e32514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/2673942069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e3247db0 0x5641e32519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.069+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2673942069 >> 172.21.15.63:0/2673942069 conn(0x5641e3128fc0 msgr2=0x5641e3242580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:37.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.070+0000 7fed8aa00700 0 client.16359 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:38:37.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.070+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2673942069 shutdown_connections 2024-05-27T07:38:37.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.070+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2673942069 wait complete. 2024-05-27T07:38:37.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.070+0000 7fedb9eefb80 20 client.16359 trim_cache size 0 max 16384 2024-05-27T07:38:37.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.089+0000 7fedb9eefb80 1 Processor -- start 2024-05-27T07:38:37.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.089+0000 7fedb9eefb80 1 -- start start 2024-05-27T07:38:37.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.090+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 0x5641e3247870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.090+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e3247db0 0x5641e325da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.090+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e324bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.090+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feda41b4e00 con 0x5641e324dd30 2024-05-27T07:38:37.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.090+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feda41b2690 con 0x5641e30c8e40 2024-05-27T07:38:37.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.090+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feda41b04e0 con 0x5641e3247db0 2024-05-27T07:38:37.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.090+0000 7feda9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e324bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.090+0000 7feda9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e324bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48262/0 (socket says 172.21.15.63:48262) 2024-05-27T07:38:37.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.090+0000 7feda9400700 1 -- 172.21.15.63:0/2766735293 learned_addr learned my addr 172.21.15.63:0/2766735293 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:37.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.091+0000 7feda3e00700 1 --2- 172.21.15.63:0/2766735293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e3247db0 0x5641e325da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.091+0000 7feda8a00700 1 --2- 172.21.15.63:0/2766735293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 0x5641e3247870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.091+0000 7feda9400700 1 -- 172.21.15.63:0/2766735293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e3247db0 msgr2=0x5641e325da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:37.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.091+0000 7feda9400700 1 --2- 172.21.15.63:0/2766735293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e3247db0 0x5641e325da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.091+0000 7feda9400700 1 -- 172.21.15.63:0/2766735293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 msgr2=0x5641e3247870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:37.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.091+0000 7feda9400700 1 --2- 172.21.15.63:0/2766735293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 0x5641e3247870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.091+0000 7feda9400700 1 -- 172.21.15.63:0/2766735293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e324fa10 con 0x5641e324dd30 2024-05-27T07:38:37.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.091+0000 7feda8a00700 1 --2- 172.21.15.63:0/2766735293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 0x5641e3247870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:37.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.091+0000 7feda9400700 1 --2- 172.21.15.63:0/2766735293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e324bb70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fed9c01a720 tx=0x7fed9c034e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965f052017194109 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:37.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.092+0000 7fed95600700 1 -- 172.21.15.63:0/2766735293 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c02f070 con 0x5641e324dd30 2024-05-27T07:38:37.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.092+0000 7fed95600700 1 -- 172.21.15.63:0/2766735293 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed9c027040 con 0x5641e324dd30 2024-05-27T07:38:37.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.092+0000 7fed95600700 1 -- 172.21.15.63:0/2766735293 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c022780 con 0x5641e324dd30 2024-05-27T07:38:37.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.093+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2766735293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 msgr2=0x5641e324bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:37.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.093+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/2766735293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e324bb70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fed9c01a720 tx=0x7fed9c034e90 comp rx=0 tx=0).stop 2024-05-27T07:38:37.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.093+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2766735293 shutdown_connections 2024-05-27T07:38:37.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.093+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/2766735293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e324dd30 0x5641e324bb70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.093+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/2766735293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e30c8e40 0x5641e3247870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.093+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/2766735293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e3247db0 0x5641e325da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.093+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2766735293 >> 172.21.15.63:0/2766735293 conn(0x5641e3128fc0 msgr2=0x5641e3251460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:37.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.093+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2766735293 shutdown_connections 2024-05-27T07:38:37.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.093+0000 7fedb9eefb80 1 -- 172.21.15.63:0/2766735293 wait complete. 2024-05-27T07:38:37.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.094+0000 7fedb9eefb80 1 -- start start 2024-05-27T07:38:37.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.094+0000 7fedb9eefb80 1 Processor -- start 2024-05-27T07:38:37.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e30c8e40 0x5641e3266dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e3264150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7fedb9eefb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e324dd30 0x5641e3264690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feda41b0b50 con 0x5641e3247db0 2024-05-27T07:38:37.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feda41b1fc0 con 0x5641e324dd30 2024-05-27T07:38:37.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7fedb9eefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feda41b1220 con 0x5641e30c8e40 2024-05-27T07:38:37.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7fedb9eefb80 10 client.0 _ll_register_callbacks cb 0x5641e310fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:37.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7feda3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e3264150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7feda3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e3264150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48278/0 (socket says 172.21.15.63:48278) 2024-05-27T07:38:37.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7feda3e00700 1 -- 172.21.15.63:0/1808007202 learned_addr learned my addr 172.21.15.63:0/1808007202 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:37.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.095+0000 7feda8a00700 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e30c8e40 0x5641e3266dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.096+0000 7feda3e00700 1 -- 172.21.15.63:0/1808007202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e30c8e40 msgr2=0x5641e3266dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:37.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.096+0000 7feda3e00700 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e30c8e40 0x5641e3266dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.096+0000 7feda9400700 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e324dd30 0x5641e3264690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.096+0000 7feda3e00700 1 -- 172.21.15.63:0/1808007202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e324dd30 msgr2=0x5641e3264690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:37.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.096+0000 7feda3e00700 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e324dd30 0x5641e3264690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.096+0000 7feda3e00700 1 -- 172.21.15.63:0/1808007202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e3264bd0 con 0x5641e3247db0 2024-05-27T07:38:37.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.096+0000 7feda3e00700 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e3264150 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fed900098e0 tx=0x7fed90005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29d0f9f8068ed9a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:37.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.096+0000 7fed8aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:37.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.097+0000 7fed8aa00700 1 -- 172.21.15.63:0/1808007202 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90003d10 con 0x5641e3247db0 2024-05-27T07:38:37.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.097+0000 7fed8aa00700 1 -- 172.21.15.63:0/1808007202 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fed9000b7d0 con 0x5641e3247db0 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.097+0000 7fedb9eefb80 1 -- 172.21.15.63:0/1808007202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed9c027040 con 0x5641e3247db0 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.097+0000 7fed8aa00700 1 -- 172.21.15.63:0/1808007202 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fed90034050 con 0x5641e3247db0 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.097+0000 7fedb9eefb80 10 client.16365 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.097+0000 7fedb9eefb80 1 -- 172.21.15.63:0/1808007202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641e32cb9f0 con 0x5641e3247db0 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.097+0000 7fed8aa00700 1 -- 172.21.15.63:0/1808007202 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed9c027040 con 0x5641e3247db0 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.098+0000 7fed8aa00700 1 -- 172.21.15.63:0/1808007202 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fed9000f880 con 0x5641e3247db0 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.098+0000 7fedb9eefb80 10 client.16365 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.098+0000 7fedb9eefb80 10 client.16365 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.098+0000 7fedb9eefb80 20 client.16365 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.098+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5641e32cc180 0x5641e32ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.098+0000 7fedb9eefb80 4 client.16365 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:38:37.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.098+0000 7fedb9eefb80 1 -- 172.21.15.63:0/1808007202 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641e3171570 con 0x5641e32cc180 2024-05-27T07:38:37.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.099+0000 7feda8a00700 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5641e32cc180 0x5641e32ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:37.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.099+0000 7feda8a00700 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5641e32cc180 0x5641e32ce5a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b871a3f7f7c41812 server_cookie=bce0da0628932850 in_seq=0 out_seq=0 2024-05-27T07:38:37.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.099+0000 7fed8aa00700 10 client.16365 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:38:37.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fed8aa00700 1 -- 172.21.15.63:0/1808007202 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641e3171570 con 0x5641e32cc180 2024-05-27T07:38:37.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fed8aa00700 10 client.16365 handle_command_reply: tid=0 2024-05-27T07:38:37.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fedb9eefb80 1 client.16365 shutdown 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fedb9eefb80 1 -- 172.21.15.63:0/1808007202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 msgr2=0x5641e3264150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e3264150 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fed900098e0 tx=0x7fed90005770 comp rx=0 tx=0).stop 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fedb9eefb80 1 -- 172.21.15.63:0/1808007202 shutdown_connections 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641e324dd30 0x5641e3264690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641e30c8e40 0x5641e3266dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5641e32cc180 0x5641e32ce5a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fedb9eefb80 1 --2- 172.21.15.63:0/1808007202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641e3247db0 0x5641e3264150 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.100+0000 7fedb9eefb80 1 -- 172.21.15.63:0/1808007202 >> 172.21.15.63:0/1808007202 conn(0x5641e3128fc0 msgr2=0x5641e324b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.101+0000 7fed8aa00700 0 client.16365 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.101+0000 7fedb9eefb80 1 -- 172.21.15.63:0/1808007202 shutdown_connections 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.101+0000 7fedb9eefb80 1 -- 172.21.15.63:0/1808007202 wait complete. 2024-05-27T07:38:37.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.101+0000 7fedb9eefb80 20 client.16365 trim_cache size 0 max 16384 2024-05-27T07:38:37.099 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:38:37.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 -- 172.21.15.63:0/94739495 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed8c0b2a70 msgr2=0x7fed8c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:37.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed8c0b2a70 0x7fed8c0b4f80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009fd0 tx=0x7fed9c009d20 comp rx=0 tx=0).stop 2024-05-27T07:38:37.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 -- 172.21.15.63:0/94739495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda41030d0 msgr2=0x7feda419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:37.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda41030d0 0x7feda419cf20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fed9800d950 tx=0x7fed9800dc60 comp rx=0 tx=0).stop 2024-05-27T07:38:37.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 -- 172.21.15.63:0/94739495 shutdown_connections 2024-05-27T07:38:37.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed8c0b2a70 0x7fed8c0b4f80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda4110c80 0x7feda41a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feda41030d0 0x7feda419cf20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 --2- 172.21.15.63:0/94739495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda4103a20 0x7feda419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:37.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 -- 172.21.15.63:0/94739495 >> 172.21.15.63:0/94739495 conn(0x7feda40fedb0 msgr2=0x7feda4101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:37.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 -- 172.21.15.63:0/94739495 shutdown_connections 2024-05-27T07:38:37.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:37.102+0000 7fedab400700 1 -- 172.21.15.63:0/94739495 wait complete. 2024-05-27T07:38:37.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:36 smithi086 ceph-mon[34687]: pgmap v154: 129 pgs: 129 active+clean; 674 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:37.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:36 smithi149 ceph-mon[35408]: pgmap v154: 129 pgs: 129 active+clean; 674 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:37.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:36 smithi063 ceph-mon[31613]: pgmap v154: 129 pgs: 129 active+clean; 674 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:37.995 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:38:38.512 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:38:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.638+0000 7fcd79c00640 1 -- 172.21.15.63:0/2415263840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd74109c70 msgr2=0x7fcd74103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.638+0000 7fcd79c00640 1 --2- 172.21.15.63:0/2415263840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd74109c70 0x7fcd74103810 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009900 tx=0x7fcd6402a410 comp rx=0 tx=0).stop 2024-05-27T07:38:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.638+0000 7fcd79c00640 1 -- 172.21.15.63:0/2415263840 shutdown_connections 2024-05-27T07:38:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.638+0000 7fcd79c00640 1 --2- 172.21.15.63:0/2415263840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd741043c0 0x7fcd74110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.638+0000 7fcd79c00640 1 --2- 172.21.15.63:0/2415263840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd7410a110 0x7fcd74103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.638+0000 7fcd79c00640 1 --2- 172.21.15.63:0/2415263840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd74109c70 0x7fcd74103810 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.638+0000 7fcd79c00640 1 -- 172.21.15.63:0/2415263840 >> 172.21.15.63:0/2415263840 conn(0x7fcd740ff350 msgr2=0x7fcd741017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.639+0000 7fcd79c00640 1 -- 172.21.15.63:0/2415263840 shutdown_connections 2024-05-27T07:38:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.639+0000 7fcd79c00640 1 -- 172.21.15.63:0/2415263840 wait complete. 2024-05-27T07:38:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.640+0000 7fcd79c00640 1 Processor -- start 2024-05-27T07:38:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.640+0000 7fcd79c00640 1 -- start start 2024-05-27T07:38:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.641+0000 7fcd79c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd741043c0 0x7fcd7419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.641+0000 7fcd79c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd74109c70 0x7fcd7419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.641+0000 7fcd79c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd7410a110 0x7fcd741a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.641+0000 7fcd79c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74113d60 con 0x7fcd74109c70 2024-05-27T07:38:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.641+0000 7fcd79c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74113be0 con 0x7fcd741043c0 2024-05-27T07:38:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.641+0000 7fcd79c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74113ee0 con 0x7fcd7410a110 2024-05-27T07:38:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.641+0000 7fcd72000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd74109c70 0x7fcd7419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.641+0000 7fcd72000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd74109c70 0x7fcd7419dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48302/0 (socket says 172.21.15.63:48302) 2024-05-27T07:38:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.641+0000 7fcd72000640 1 -- 172.21.15.63:0/3072487901 learned_addr learned my addr 172.21.15.63:0/3072487901 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.641+0000 7fcd73400640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd7410a110 0x7fcd741a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.642+0000 7fcd72a00640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd741043c0 0x7fcd7419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.642+0000 7fcd72000640 1 -- 172.21.15.63:0/3072487901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd7410a110 msgr2=0x7fcd741a86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.642+0000 7fcd72000640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd7410a110 0x7fcd741a86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.642+0000 7fcd72000640 1 -- 172.21.15.63:0/3072487901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd741043c0 msgr2=0x7fcd7419d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.642+0000 7fcd72000640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd741043c0 0x7fcd7419d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.642+0000 7fcd72000640 1 -- 172.21.15.63:0/3072487901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd741a8be0 con 0x7fcd74109c70 2024-05-27T07:38:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.642+0000 7fcd72000640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd74109c70 0x7fcd7419dd40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000d740 tx=0x7fcd6000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.643+0000 7fcd6b400640 1 -- 172.21.15.63:0/3072487901 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60017070 con 0x7fcd74109c70 2024-05-27T07:38:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.643+0000 7fcd6b400640 1 -- 172.21.15.63:0/3072487901 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd6000de50 con 0x7fcd74109c70 2024-05-27T07:38:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.643+0000 7fcd79c00640 1 -- 172.21.15.63:0/3072487901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd741a8e70 con 0x7fcd74109c70 2024-05-27T07:38:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.643+0000 7fcd6b400640 1 -- 172.21.15.63:0/3072487901 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60020060 con 0x7fcd74109c70 2024-05-27T07:38:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.643+0000 7fcd79c00640 1 -- 172.21.15.63:0/3072487901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd741a92d0 con 0x7fcd74109c70 2024-05-27T07:38:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.647+0000 7fcd80b93740 1 Processor -- start 2024-05-27T07:38:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd80b93740 1 -- start start 2024-05-27T07:38:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c901b0 0x564958032860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x564958032da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c6f4d0 0x5649580332e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd80b93740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564957fa3b90 con 0x564957c672f0 2024-05-27T07:38:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd80b93740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564957f93730 con 0x564957c901b0 2024-05-27T07:38:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd80b93740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564957f97770 con 0x564957c6f4d0 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd72a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x564958032da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd72a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x564958032da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48304/0 (socket says 172.21.15.63:48304) 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd6b400640 1 -- 172.21.15.63:0/3072487901 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcd60003b50 con 0x7fcd74109c70 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.648+0000 7fcd72a00640 1 -- 172.21.15.63:0/1875198198 learned_addr learned my addr 172.21.15.63:0/1875198198 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.649+0000 7fcd73400640 1 --2- 172.21.15.63:0/1875198198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c901b0 0x564958032860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.649+0000 7fcd72a00640 1 -- 172.21.15.63:0/1875198198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c6f4d0 msgr2=0x5649580332e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.649+0000 7fcd72a00640 1 --2- 172.21.15.63:0/1875198198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c6f4d0 0x5649580332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.649+0000 7fcd72a00640 1 -- 172.21.15.63:0/1875198198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c901b0 msgr2=0x564958032860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.649+0000 7fcd72a00640 1 --2- 172.21.15.63:0/1875198198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c901b0 0x564958032860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.649+0000 7fcd72a00640 1 -- 172.21.15.63:0/1875198198 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564957f2c8b0 con 0x564957c672f0 2024-05-27T07:38:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.649+0000 7fcd72a00640 1 --2- 172.21.15.63:0/1875198198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x564958032da0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcd64006010 tx=0x7fcd64004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbcb540e94d6a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.650+0000 7fcd5fe00640 1 -- 172.21.15.63:0/1875198198 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64035030 con 0x564957c672f0 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.650+0000 7fcd5fe00640 1 -- 172.21.15.63:0/1875198198 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd640054d0 con 0x564957c672f0 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.650+0000 7fcd5fe00640 1 -- 172.21.15.63:0/1875198198 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64003610 con 0x564957c672f0 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd6b400640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcd580b2950 0x7fcd580b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd6b400640 1 -- 172.21.15.63:0/3072487901 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fcd600e22a0 con 0x7fcd74109c70 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd73400640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcd580b2950 0x7fcd580b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd80b93740 1 -- 172.21.15.63:0/1875198198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 msgr2=0x564958032da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd80b93740 1 --2- 172.21.15.63:0/1875198198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x564958032da0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcd64006010 tx=0x7fcd64004200 comp rx=0 tx=0).stop 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd80b93740 1 -- 172.21.15.63:0/1875198198 shutdown_connections 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd80b93740 1 --2- 172.21.15.63:0/1875198198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c6f4d0 0x5649580332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd80b93740 1 --2- 172.21.15.63:0/1875198198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x564958032da0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd80b93740 1 --2- 172.21.15.63:0/1875198198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c901b0 0x564958032860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd80b93740 1 -- 172.21.15.63:0/1875198198 >> 172.21.15.63:0/1875198198 conn(0x564957c84300 msgr2=0x56495802f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd80b93740 1 -- 172.21.15.63:0/1875198198 shutdown_connections 2024-05-27T07:38:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.651+0000 7fcd80b93740 1 -- 172.21.15.63:0/1875198198 wait complete. 2024-05-27T07:38:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.652+0000 7fcd73400640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcd580b2950 0x7fcd580b4e60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x56495802f9d0 tx=0x7fcd6c009580 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.652+0000 7fcd80b93740 1 -- start start 2024-05-27T07:38:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.652+0000 7fcd80b93740 1 Processor -- start 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.653+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x56495803ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.653+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c6f4d0 0x56495803cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.653+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c901b0 0x564958048c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.653+0000 7fcd80b93740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564957c79790 con 0x564957c672f0 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.653+0000 7fcd80b93740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564957f3bbc0 con 0x564957c901b0 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.653+0000 7fcd80b93740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56495803f930 con 0x564957c6f4d0 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.653+0000 7fcd80b93740 10 client.0 _ll_register_callbacks cb 0x564957f19400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.654+0000 7fcd72a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x56495803ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.654+0000 7fcd72a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x56495803ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48318/0 (socket says 172.21.15.63:48318) 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.654+0000 7fcd72a00640 1 -- 172.21.15.63:0/98148120 learned_addr learned my addr 172.21.15.63:0/98148120 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.654+0000 7fcd72000640 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c6f4d0 0x56495803cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.654+0000 7fcd73400640 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c901b0 0x564958048c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.654+0000 7fcd72a00640 1 -- 172.21.15.63:0/98148120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c6f4d0 msgr2=0x56495803cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.654+0000 7fcd72a00640 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c6f4d0 0x56495803cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.654+0000 7fcd72a00640 1 -- 172.21.15.63:0/98148120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c901b0 msgr2=0x564958048c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.654+0000 7fcd72a00640 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c901b0 0x564958048c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.654+0000 7fcd72a00640 1 -- 172.21.15.63:0/98148120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564957ca8210 con 0x564957c672f0 2024-05-27T07:38:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.655+0000 7fcd72a00640 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x56495803ca70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcd640057c0 tx=0x7fcd64005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d8658f6e888e9f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.655+0000 7fcd5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.655+0000 7fcd5cc00640 1 -- 172.21.15.63:0/98148120 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64051060 con 0x564957c672f0 2024-05-27T07:38:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.655+0000 7fcd5cc00640 1 -- 172.21.15.63:0/98148120 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fcd640035f0 con 0x564957c672f0 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.655+0000 7fcd80b93740 1 -- 172.21.15.63:0/98148120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd640054d0 con 0x564957c672f0 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.655+0000 7fcd5cc00640 1 -- 172.21.15.63:0/98148120 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcd64058050 con 0x564957c672f0 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.655+0000 7fcd80b93740 10 client.16377 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.655+0000 7fcd80b93740 1 -- 172.21.15.63:0/98148120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564957d74620 con 0x564957c672f0 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.656+0000 7fcd5cc00640 1 -- 172.21.15.63:0/98148120 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd640054d0 con 0x564957c672f0 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.656+0000 7fcd5cc00640 1 -- 172.21.15.63:0/98148120 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcd6404b030 con 0x564957c672f0 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.656+0000 7fcd80b93740 10 client.16377 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.656+0000 7fcd80b93740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.656+0000 7fcd80b93740 20 client.16377 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.656+0000 7fcd80b93740 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564957c4ed80 0x56495804eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.656+0000 7fcd80b93740 4 client.16377 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.656+0000 7fcd80b93740 1 -- 172.21.15.63:0/98148120 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564957fa5ae0 con 0x564957c4ed80 2024-05-27T07:38:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.657+0000 7fcd72000640 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564957c4ed80 0x56495804eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.658+0000 7fcd72000640 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564957c4ed80 0x56495804eae0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5118f423b626168b server_cookie=ca4eb61141eeb3eb in_seq=0 out_seq=0 2024-05-27T07:38:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.658+0000 7fcd5cc00640 10 client.16377 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd5cc00640 1 -- 172.21.15.63:0/98148120 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564957fa5ae0 con 0x564957c4ed80 2024-05-27T07:38:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd5cc00640 10 client.16377 handle_command_reply: tid=0 2024-05-27T07:38:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd80b93740 1 client.16377 shutdown 2024-05-27T07:38:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd80b93740 1 -- 172.21.15.63:0/98148120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 msgr2=0x56495803ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd80b93740 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x56495803ca70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcd640057c0 tx=0x7fcd64005710 comp rx=0 tx=0).stop 2024-05-27T07:38:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd80b93740 1 -- 172.21.15.63:0/98148120 shutdown_connections 2024-05-27T07:38:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd80b93740 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564957c4ed80 0x56495804eae0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd80b93740 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c901b0 0x564958048c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd80b93740 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c6f4d0 0x56495803cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd80b93740 1 --2- 172.21.15.63:0/98148120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c672f0 0x56495803ca70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd5cc00640 0 client.16377 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd80b93740 1 -- 172.21.15.63:0/98148120 >> 172.21.15.63:0/98148120 conn(0x564957c84300 msgr2=0x564958034580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.660+0000 7fcd80b93740 1 -- 172.21.15.63:0/98148120 shutdown_connections 2024-05-27T07:38:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.661+0000 7fcd80b93740 1 -- 172.21.15.63:0/98148120 wait complete. 2024-05-27T07:38:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.661+0000 7fcd80b93740 20 client.16377 trim_cache size 0 max 16384 2024-05-27T07:38:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.682+0000 7fcd80b93740 1 Processor -- start 2024-05-27T07:38:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.682+0000 7fcd80b93740 1 -- start start 2024-05-27T07:38:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.683+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c672f0 0x56495802e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.683+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c6f4d0 0x564958033020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.683+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x564958048c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.683+0000 7fcd80b93740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b5b60 con 0x564957c901b0 2024-05-27T07:38:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.683+0000 7fcd80b93740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b3430 con 0x564957c6f4d0 2024-05-27T07:38:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.683+0000 7fcd80b93740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b1280 con 0x564957c672f0 2024-05-27T07:38:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.683+0000 7fcd73400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x564958048c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.683+0000 7fcd73400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x564958048c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48328/0 (socket says 172.21.15.63:48328) 2024-05-27T07:38:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.683+0000 7fcd73400640 1 -- 172.21.15.63:0/696270340 learned_addr learned my addr 172.21.15.63:0/696270340 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.683+0000 7fcd72a00640 1 --2- 172.21.15.63:0/696270340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c672f0 0x56495802e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.684+0000 7fcd72000640 1 --2- 172.21.15.63:0/696270340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c6f4d0 0x564958033020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.684+0000 7fcd73400640 1 -- 172.21.15.63:0/696270340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c672f0 msgr2=0x56495802e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.684+0000 7fcd73400640 1 --2- 172.21.15.63:0/696270340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c672f0 0x56495802e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.684+0000 7fcd73400640 1 -- 172.21.15.63:0/696270340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c6f4d0 msgr2=0x564958033020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.684+0000 7fcd73400640 1 --2- 172.21.15.63:0/696270340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c6f4d0 0x564958033020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.684+0000 7fcd73400640 1 -- 172.21.15.63:0/696270340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56495802fc00 con 0x564957c901b0 2024-05-27T07:38:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.684+0000 7fcd73400640 1 --2- 172.21.15.63:0/696270340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x564958048c00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0127d0 tx=0x7fcd6c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a8a6368fd0544f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.685+0000 7fcd5f400640 1 -- 172.21.15.63:0/696270340 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c01fb40 con 0x564957c901b0 2024-05-27T07:38:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.685+0000 7fcd5f400640 1 -- 172.21.15.63:0/696270340 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd6c012e10 con 0x564957c901b0 2024-05-27T07:38:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.685+0000 7fcd5f400640 1 -- 172.21.15.63:0/696270340 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c028630 con 0x564957c901b0 2024-05-27T07:38:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.686+0000 7fcd80b93740 1 -- 172.21.15.63:0/696270340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 msgr2=0x564958048c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.686+0000 7fcd80b93740 1 --2- 172.21.15.63:0/696270340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x564958048c00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0127d0 tx=0x7fcd6c014000 comp rx=0 tx=0).stop 2024-05-27T07:38:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.686+0000 7fcd80b93740 1 -- 172.21.15.63:0/696270340 shutdown_connections 2024-05-27T07:38:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.686+0000 7fcd80b93740 1 --2- 172.21.15.63:0/696270340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x564958048c00 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.686+0000 7fcd80b93740 1 --2- 172.21.15.63:0/696270340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c6f4d0 0x564958033020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.686+0000 7fcd80b93740 1 --2- 172.21.15.63:0/696270340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c672f0 0x56495802e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.686+0000 7fcd80b93740 1 -- 172.21.15.63:0/696270340 >> 172.21.15.63:0/696270340 conn(0x564957c84300 msgr2=0x56495803cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.686+0000 7fcd80b93740 1 -- 172.21.15.63:0/696270340 shutdown_connections 2024-05-27T07:38:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.686+0000 7fcd80b93740 1 -- 172.21.15.63:0/696270340 wait complete. 2024-05-27T07:38:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.686+0000 7fcd80b93740 1 -- start start 2024-05-27T07:38:38.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.687+0000 7fcd80b93740 1 Processor -- start 2024-05-27T07:38:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.687+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c672f0 0x564958052120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.687+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c6f4d0 0x56495804d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.687+0000 7fcd80b93740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x56495804d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.687+0000 7fcd80b93740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b18f0 con 0x564957c901b0 2024-05-27T07:38:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.687+0000 7fcd80b93740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b2d60 con 0x564957c6f4d0 2024-05-27T07:38:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.687+0000 7fcd80b93740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741b1fc0 con 0x564957c672f0 2024-05-27T07:38:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.687+0000 7fcd80b93740 10 client.0 _ll_register_callbacks cb 0x7fcd640040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.688+0000 7fcd73400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x56495804d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.688+0000 7fcd73400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x56495804d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48344/0 (socket says 172.21.15.63:48344) 2024-05-27T07:38:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.688+0000 7fcd73400640 1 -- 172.21.15.63:0/2269704241 learned_addr learned my addr 172.21.15.63:0/2269704241 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.688+0000 7fcd72a00640 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c672f0 0x564958052120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.688+0000 7fcd72000640 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c6f4d0 0x56495804d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.688+0000 7fcd73400640 1 -- 172.21.15.63:0/2269704241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c672f0 msgr2=0x564958052120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.688+0000 7fcd73400640 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c672f0 0x564958052120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.688+0000 7fcd73400640 1 -- 172.21.15.63:0/2269704241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c6f4d0 msgr2=0x56495804d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.688+0000 7fcd73400640 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c6f4d0 0x56495804d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.688+0000 7fcd73400640 1 -- 172.21.15.63:0/2269704241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56495804dff0 con 0x564957c901b0 2024-05-27T07:38:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.689+0000 7fcd73400640 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x56495804d9b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009fa0 tx=0x7fcd6c0125b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6770295ecb990329 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:38.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.689+0000 7fcd5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.689+0000 7fcd5cc00640 1 -- 172.21.15.63:0/2269704241 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c036020 con 0x564957c901b0 2024-05-27T07:38:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.689+0000 7fcd5cc00640 1 -- 172.21.15.63:0/2269704241 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fcd6c004a40 con 0x564957c901b0 2024-05-27T07:38:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.689+0000 7fcd80b93740 1 -- 172.21.15.63:0/2269704241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6c012e10 con 0x564957c901b0 2024-05-27T07:38:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.689+0000 7fcd5cc00640 1 -- 172.21.15.63:0/2269704241 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcd6c041050 con 0x564957c901b0 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.689+0000 7fcd80b93740 10 client.16383 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.689+0000 7fcd80b93740 1 -- 172.21.15.63:0/2269704241 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5649580b6d20 con 0x564957c901b0 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.690+0000 7fcd5cc00640 1 -- 172.21.15.63:0/2269704241 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd6c012e10 con 0x564957c901b0 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.690+0000 7fcd5cc00640 1 -- 172.21.15.63:0/2269704241 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcd6c00f040 con 0x564957c901b0 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.690+0000 7fcd80b93740 10 client.16383 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.690+0000 7fcd80b93740 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.690+0000 7fcd80b93740 20 client.16383 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.690+0000 7fcd80b93740 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564957c241c0 0x5649580b9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.690+0000 7fcd80b93740 4 client.16383 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.690+0000 7fcd80b93740 1 -- 172.21.15.63:0/2269704241 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564957f9f2c0 con 0x564957c241c0 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.691+0000 7fcd72a00640 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564957c241c0 0x5649580b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.692+0000 7fcd72a00640 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564957c241c0 0x5649580b9470 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a2ce04827be5b5f server_cookie=f75c4fe93d9765dd in_seq=0 out_seq=0 2024-05-27T07:38:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.692+0000 7fcd5cc00640 10 client.16383 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.727+0000 7fcd5cc00640 1 -- 172.21.15.63:0/2269704241 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564957f9f2c0 con 0x564957c241c0 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.727+0000 7fcd5cc00640 10 client.16383 handle_command_reply: tid=0 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.727+0000 7fcd80b93740 1 client.16383 shutdown 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 1 -- 172.21.15.63:0/2269704241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 msgr2=0x56495804d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x56495804d9b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009fa0 tx=0x7fcd6c0125b0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 1 -- 172.21.15.63:0/2269704241 shutdown_connections 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564957c241c0 0x5649580b9470 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564957c901b0 0x56495804d9b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564957c6f4d0 0x56495804d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 1 --2- 172.21.15.63:0/2269704241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564957c672f0 0x564958052120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 1 -- 172.21.15.63:0/2269704241 >> 172.21.15.63:0/2269704241 conn(0x564957c84300 msgr2=0x56495803cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd5cc00640 0 client.16383 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 1 -- 172.21.15.63:0/2269704241 shutdown_connections 2024-05-27T07:38:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 1 -- 172.21.15.63:0/2269704241 wait complete. 2024-05-27T07:38:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.728+0000 7fcd80b93740 20 client.16383 trim_cache size 0 max 16384 2024-05-27T07:38:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:38:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:38:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:38:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 -- 172.21.15.63:0/3072487901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcd580b2950 msgr2=0x7fcd580b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcd580b2950 0x7fcd580b4e60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x56495802f9d0 tx=0x7fcd6c009580 comp rx=0 tx=0).stop 2024-05-27T07:38:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 -- 172.21.15.63:0/3072487901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd74109c70 msgr2=0x7fcd7419dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd74109c70 0x7fcd7419dd40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000d740 tx=0x7fcd6000dc00 comp rx=0 tx=0).stop 2024-05-27T07:38:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 -- 172.21.15.63:0/3072487901 shutdown_connections 2024-05-27T07:38:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcd580b2950 0x7fcd580b4e60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd7410a110 0x7fcd741a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd74109c70 0x7fcd7419dd40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 --2- 172.21.15.63:0/3072487901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd741043c0 0x7fcd7419d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 -- 172.21.15.63:0/3072487901 >> 172.21.15.63:0/3072487901 conn(0x7fcd740ff350 msgr2=0x7fcd740ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 -- 172.21.15.63:0/3072487901 shutdown_connections 2024-05-27T07:38:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:38.729+0000 7fcd79c00640 1 -- 172.21.15.63:0/3072487901 wait complete. 2024-05-27T07:38:39.177 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:38 smithi063 ceph-mon[31613]: pgmap v155: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:38:39.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:38 smithi086 ceph-mon[34687]: pgmap v155: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:38:39.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:38 smithi149 ceph-mon[35408]: pgmap v155: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:38:41.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:40 smithi086 ceph-mon[34687]: pgmap v156: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:38:41.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:40 smithi149 ceph-mon[35408]: pgmap v156: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:38:41.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:40 smithi063 ceph-mon[31613]: pgmap v156: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:38:42.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.294+0000 7f2bdf200700 1 -- 172.21.15.63:0/3034843361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be010af10 msgr2=0x7f2be010b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:42.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.294+0000 7f2bdf200700 1 --2- 172.21.15.63:0/3034843361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be010af10 0x7f2be010b320 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80099c0 tx=0x7f2bc8009cd0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.294+0000 7f2bdf200700 1 -- 172.21.15.63:0/3034843361 shutdown_connections 2024-05-27T07:38:42.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.294+0000 7f2bdf200700 1 --2- 172.21.15.63:0/3034843361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be010b860 0x7f2be01131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.294+0000 7f2bdf200700 1 --2- 172.21.15.63:0/3034843361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be0106630 0x7f2be0106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.294+0000 7f2bdf200700 1 --2- 172.21.15.63:0/3034843361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be010af10 0x7f2be010b320 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.294+0000 7f2bdf200700 1 -- 172.21.15.63:0/3034843361 >> 172.21.15.63:0/3034843361 conn(0x7f2be0102020 msgr2=0x7f2be0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:42.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.294+0000 7f2bdf200700 1 -- 172.21.15.63:0/3034843361 shutdown_connections 2024-05-27T07:38:42.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.295+0000 7f2bdf200700 1 -- 172.21.15.63:0/3034843361 wait complete. 2024-05-27T07:38:42.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.296+0000 7f2bdf200700 1 Processor -- start 2024-05-27T07:38:42.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.296+0000 7f2bdf200700 1 -- start start 2024-05-27T07:38:42.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.297+0000 7f2bdf200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0106630 0x7f2be019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.297+0000 7f2bdf200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be010af10 0x7f2be019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.297+0000 7f2bdf200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be010b860 0x7f2be01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.297+0000 7f2bdf200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0116370 con 0x7f2be010af10 2024-05-27T07:38:42.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.297+0000 7f2bdf200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01161f0 con 0x7f2be010b860 2024-05-27T07:38:42.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.297+0000 7f2bdf200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01164f0 con 0x7f2be0106630 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.297+0000 7f2bd7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be010af10 0x7f2be019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.297+0000 7f2bd7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be010af10 0x7f2be019d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56938/0 (socket says 172.21.15.63:56938) 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.297+0000 7f2bd7e00700 1 -- 172.21.15.63:0/618913364 learned_addr learned my addr 172.21.15.63:0/618913364 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.298+0000 7f2bdde00700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0106630 0x7f2be019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.298+0000 7f2bde800700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be010b860 0x7f2be01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.298+0000 7f2bd7e00700 1 -- 172.21.15.63:0/618913364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0106630 msgr2=0x7f2be019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.298+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0106630 0x7f2be019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.298+0000 7f2bd7e00700 1 -- 172.21.15.63:0/618913364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be010b860 msgr2=0x7f2be01a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.298+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be010b860 0x7f2be01a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.298+0000 7f2bd7e00700 1 -- 172.21.15.63:0/618913364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be01a1c70 con 0x7f2be010af10 2024-05-27T07:38:42.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.298+0000 7f2bde800700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be010b860 0x7f2be01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:42.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.299+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be010af10 0x7f2be019d340 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80098e0 tx=0x7f2bc80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:42.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.299+0000 7f2bd6a00700 1 -- 172.21.15.63:0/618913364 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8027070 con 0x7f2be010af10 2024-05-27T07:38:42.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.299+0000 7f2bdf200700 1 -- 172.21.15.63:0/618913364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be01a1f00 con 0x7f2be010af10 2024-05-27T07:38:42.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.299+0000 7f2bd6a00700 1 -- 172.21.15.63:0/618913364 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bc800fd30 con 0x7f2be010af10 2024-05-27T07:38:42.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.299+0000 7f2bd6a00700 1 -- 172.21.15.63:0/618913364 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8022dc0 con 0x7f2be010af10 2024-05-27T07:38:42.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.299+0000 7f2bdf200700 1 -- 172.21.15.63:0/618913364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be01a9880 con 0x7f2be010af10 2024-05-27T07:38:42.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.303+0000 7f2bd6a00700 1 -- 172.21.15.63:0/618913364 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2bc802a070 con 0x7f2be010af10 2024-05-27T07:38:42.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.305+0000 7f2bf37f5b80 1 Processor -- start 2024-05-27T07:38:42.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.305+0000 7f2bd6a00700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2bc40b2a70 0x7f2bc40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.305+0000 7f2bf37f5b80 1 -- start start 2024-05-27T07:38:42.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.305+0000 7f2bd6a00700 1 -- 172.21.15.63:0/618913364 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2bc80e3830 con 0x7f2be010af10 2024-05-27T07:38:42.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.305+0000 7f2bde800700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2bc40b2a70 0x7f2bc40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.306+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820ef3c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.306+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820edbce40 0x55820ef3cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.306+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820ef462c0 0x55820ef486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.306+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55820ee69e40 con 0x55820eea89f0 2024-05-27T07:38:42.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.306+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55820ee599e0 con 0x55820ef462c0 2024-05-27T07:38:42.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.306+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55820ee5da20 con 0x55820edbce40 2024-05-27T07:38:42.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.306+0000 7f2bdde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820ef3c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.306+0000 7f2bdde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820ef3c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56946/0 (socket says 172.21.15.63:56946) 2024-05-27T07:38:42.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.306+0000 7f2bdde00700 1 -- 172.21.15.63:0/728174841 learned_addr learned my addr 172.21.15.63:0/728174841 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:42.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.306+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/728174841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820edbce40 0x55820ef3cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.307+0000 7f2bde800700 1 --2- 172.21.15.63:0/728174841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820ef462c0 0x55820ef486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.307+0000 7f2bde800700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2bc40b2a70 0x7f2bc40b4f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009d20 tx=0x7f2bd8009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:42.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.307+0000 7f2bdde00700 1 -- 172.21.15.63:0/728174841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820edbce40 msgr2=0x55820ef3cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:42.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.307+0000 7f2bdde00700 1 --2- 172.21.15.63:0/728174841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820edbce40 0x55820ef3cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.307+0000 7f2bdde00700 1 -- 172.21.15.63:0/728174841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820ef462c0 msgr2=0x55820ef486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:42.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.307+0000 7f2bdde00700 1 --2- 172.21.15.63:0/728174841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820ef462c0 0x55820ef486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.307+0000 7f2bdde00700 1 -- 172.21.15.63:0/728174841 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820ee22420 con 0x55820eea89f0 2024-05-27T07:38:42.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.307+0000 7f2bdde00700 1 --2- 172.21.15.63:0/728174841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820ef3c750 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000bd70 tx=0x7f2bd00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc21ce65ae8b971 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:42.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.308+0000 7f2bcf400700 1 -- 172.21.15.63:0/728174841 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0010e10 con 0x55820eea89f0 2024-05-27T07:38:42.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.308+0000 7f2bcf400700 1 -- 172.21.15.63:0/728174841 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bd001c910 con 0x55820eea89f0 2024-05-27T07:38:42.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.308+0000 7f2bcf400700 1 -- 172.21.15.63:0/728174841 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0011870 con 0x55820eea89f0 2024-05-27T07:38:42.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.309+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/728174841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 msgr2=0x55820ef3c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:42.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.309+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/728174841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820ef3c750 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000bd70 tx=0x7f2bd00106b0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.309+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/728174841 shutdown_connections 2024-05-27T07:38:42.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.309+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/728174841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820ef462c0 0x55820ef486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.309+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/728174841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820ef3c750 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.309+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/728174841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820edbce40 0x55820ef3cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.309+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/728174841 >> 172.21.15.63:0/728174841 conn(0x55820ee56390 msgr2=0x55820ef37710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:42.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.309+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/728174841 shutdown_connections 2024-05-27T07:38:42.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.309+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/728174841 wait complete. 2024-05-27T07:38:42.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.310+0000 7f2bf37f5b80 1 -- start start 2024-05-27T07:38:42.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.310+0000 7f2bf37f5b80 1 Processor -- start 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.311+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef43200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.311+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 0x55820ef43740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.311+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef43c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.311+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55820ed5ffa0 con 0x55820edbce40 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.311+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55820edfa760 con 0x55820eea89f0 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.311+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55820ef51610 con 0x55820ef462c0 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.311+0000 7f2bf37f5b80 10 client.0 _ll_register_callbacks cb 0x55820ee03c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.311+0000 7f2bdde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef43200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.311+0000 7f2bdde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef43200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56954/0 (socket says 172.21.15.63:56954) 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.311+0000 7f2bdde00700 1 -- 172.21.15.63:0/3800289256 learned_addr learned my addr 172.21.15.63:0/3800289256 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:42.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 0x55820ef43740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bde800700 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef43c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bdde00700 1 -- 172.21.15.63:0/3800289256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 msgr2=0x55820ef43c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:42.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bdde00700 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef43c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bdde00700 1 -- 172.21.15.63:0/3800289256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 msgr2=0x55820ef43740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:42.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bdde00700 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 0x55820ef43740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bdde00700 1 -- 172.21.15.63:0/3800289256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55820eda98d0 con 0x55820edbce40 2024-05-27T07:38:42.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 0x55820ef43740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:42.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bde800700 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef43c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bdde00700 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef43200 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0010ec0 tx=0x7f2bd0010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ac0b043b619147 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.312+0000 7f2bc3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.313+0000 7f2bc3e00700 1 -- 172.21.15.63:0/3800289256 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd001ccc0 con 0x55820edbce40 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.313+0000 7f2bc3e00700 1 -- 172.21.15.63:0/3800289256 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2bd0004030 con 0x55820edbce40 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.313+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3800289256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd001c910 con 0x55820edbce40 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.313+0000 7f2bc3e00700 1 -- 172.21.15.63:0/3800289256 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2bd002f050 con 0x55820edbce40 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.313+0000 7f2bf37f5b80 10 client.16395 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.313+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3800289256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55820ee2a060 con 0x55820edbce40 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.314+0000 7f2bc3e00700 1 -- 172.21.15.63:0/3800289256 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bd001c910 con 0x55820edbce40 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.314+0000 7f2bc3e00700 1 -- 172.21.15.63:0/3800289256 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2bd0026480 con 0x55820edbce40 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.314+0000 7f2bf37f5b80 10 client.16395 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:42.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.314+0000 7f2bf37f5b80 10 client.16395 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:38:42.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.314+0000 7f2bf37f5b80 20 client.16395 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:42.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.314+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55820ef5e440 0x55820ef60860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.314+0000 7f2bf37f5b80 4 client.16395 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:38:42.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.314+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3800289256 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55820ee6bd90 con 0x55820ef5e440 2024-05-27T07:38:42.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.315+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55820ef5e440 0x55820ef60860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.316+0000 7f2bc3e00700 10 client.16395 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:38:42.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.316+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55820ef5e440 0x55820ef60860 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb4ae4e4f6ca2680 server_cookie=7fd311b77515d0e7 in_seq=0 out_seq=0 2024-05-27T07:38:42.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bc3e00700 1 -- 172.21.15.63:0/3800289256 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55820ee6bd90 con 0x55820ef5e440 2024-05-27T07:38:42.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bc3e00700 10 client.16395 handle_command_reply: tid=0 2024-05-27T07:38:42.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bf37f5b80 1 client.16395 shutdown 2024-05-27T07:38:42.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3800289256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 msgr2=0x55820ef43200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:42.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef43200 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0010ec0 tx=0x7f2bd0010fa0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3800289256 shutdown_connections 2024-05-27T07:38:42.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef43c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55820ef5e440 0x55820ef60860 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef43200 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/3800289256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 0x55820ef43740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3800289256 >> 172.21.15.63:0/3800289256 conn(0x55820ee56390 msgr2=0x55820ef37260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:42.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.318+0000 7f2bc3e00700 0 client.16395 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:38:42.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.319+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3800289256 shutdown_connections 2024-05-27T07:38:42.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.319+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3800289256 wait complete. 2024-05-27T07:38:42.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.319+0000 7f2bf37f5b80 20 client.16395 trim_cache size 0 max 16384 2024-05-27T07:38:42.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.340+0000 7f2bf37f5b80 1 Processor -- start 2024-05-27T07:38:42.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.340+0000 7f2bf37f5b80 1 -- start start 2024-05-27T07:38:42.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.341+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef42160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.341+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 0x55820ef5a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.341+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef3b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.341+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc0114f0 con 0x55820edbce40 2024-05-27T07:38:42.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.341+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc00ed80 con 0x55820eea89f0 2024-05-27T07:38:42.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.341+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc00cbd0 con 0x55820ef462c0 2024-05-27T07:38:42.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.341+0000 7f2bdde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef42160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.341+0000 7f2bdde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef42160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56966/0 (socket says 172.21.15.63:56966) 2024-05-27T07:38:42.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.341+0000 7f2bdde00700 1 -- 172.21.15.63:0/3783849310 learned_addr learned my addr 172.21.15.63:0/3783849310 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:42.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 0x55820ef5a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bde800700 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef3b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bdde00700 1 -- 172.21.15.63:0/3783849310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 msgr2=0x55820ef3b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:42.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bdde00700 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef3b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bdde00700 1 -- 172.21.15.63:0/3783849310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 msgr2=0x55820ef5a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:42.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bdde00700 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 0x55820ef5a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bdde00700 1 -- 172.21.15.63:0/3783849310 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820ef37c50 con 0x55820edbce40 2024-05-27T07:38:42.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 0x55820ef5a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:38:42.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bde800700 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef3b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:38:42.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bdde00700 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef42160 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00106f0 tx=0x7f2bd000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c64adf21a25e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:42.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bcea00700 1 -- 172.21.15.63:0/3783849310 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0026480 con 0x55820edbce40 2024-05-27T07:38:42.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.342+0000 7f2bcea00700 1 -- 172.21.15.63:0/3783849310 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bd00055a0 con 0x55820edbce40 2024-05-27T07:38:42.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.343+0000 7f2bcea00700 1 -- 172.21.15.63:0/3783849310 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd001cc60 con 0x55820edbce40 2024-05-27T07:38:42.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.344+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3783849310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 msgr2=0x55820ef42160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:42.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.344+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef42160 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00106f0 tx=0x7f2bd000b850 comp rx=0 tx=0).stop 2024-05-27T07:38:42.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.344+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3783849310 shutdown_connections 2024-05-27T07:38:42.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.344+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef3b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.344+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820edbce40 0x55820ef42160 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.344+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/3783849310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820eea89f0 0x55820ef5a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.344+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3783849310 >> 172.21.15.63:0/3783849310 conn(0x55820ee56390 msgr2=0x55820ef3b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:42.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.344+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3783849310 shutdown_connections 2024-05-27T07:38:42.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.344+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/3783849310 wait complete. 2024-05-27T07:38:42.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.345+0000 7f2bf37f5b80 1 -- start start 2024-05-27T07:38:42.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.345+0000 7f2bf37f5b80 1 Processor -- start 2024-05-27T07:38:42.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.345+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820edbce40 0x55820efc10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.345+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820efc15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.345+0000 7f2bf37f5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef51e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.345+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc00d240 con 0x55820eea89f0 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.345+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc00e6b0 con 0x55820edbce40 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.345+0000 7f2bf37f5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc00d910 con 0x55820ef462c0 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.346+0000 7f2bf37f5b80 10 client.0 _ll_register_callbacks cb 0x55820ee03c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.346+0000 7f2bd7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820efc15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.346+0000 7f2bd7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820efc15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56972/0 (socket says 172.21.15.63:56972) 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.346+0000 7f2bd7e00700 1 -- 172.21.15.63:0/1997844868 learned_addr learned my addr 172.21.15.63:0/1997844868 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.346+0000 7f2bdde00700 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820edbce40 0x55820efc10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.346+0000 7f2bde800700 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef51e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.346+0000 7f2bd7e00700 1 -- 172.21.15.63:0/1997844868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 msgr2=0x55820ef51e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.347+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef51e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.347+0000 7f2bd7e00700 1 -- 172.21.15.63:0/1997844868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820edbce40 msgr2=0x55820efc10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.347+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820edbce40 0x55820efc10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.347+0000 7f2bd7e00700 1 -- 172.21.15.63:0/1997844868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55820ef43ff0 con 0x55820eea89f0 2024-05-27T07:38:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.347+0000 7f2bd7e00700 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820efc15f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800ff90 tx=0x7f2bc80cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fedc49626549be1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.347+0000 7f2bc3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.347+0000 7f2bc3e00700 1 -- 172.21.15.63:0/1997844868 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80e3620 con 0x55820eea89f0 2024-05-27T07:38:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.347+0000 7f2bc3e00700 1 -- 172.21.15.63:0/1997844868 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2bc80b3110 con 0x55820eea89f0 2024-05-27T07:38:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.347+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/1997844868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55820ee65570 con 0x55820eea89f0 2024-05-27T07:38:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bc3e00700 1 -- 172.21.15.63:0/1997844868 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2bc80b10b0 con 0x55820eea89f0 2024-05-27T07:38:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bf37f5b80 10 client.16401 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:42.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/1997844868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55820efc8490 con 0x55820eea89f0 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bc3e00700 1 -- 172.21.15.63:0/1997844868 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55820ee65570 con 0x55820eea89f0 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bc3e00700 1 -- 172.21.15.63:0/1997844868 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2bc80bfb90 con 0x55820eea89f0 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bf37f5b80 10 client.16401 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bf37f5b80 10 client.16401 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bf37f5b80 20 client.16401 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55820efc8b20 0x55820efcaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bf37f5b80 4 client.16401 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.348+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/1997844868 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55820ecdc250 con 0x55820efc8b20 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.349+0000 7f2bdde00700 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55820efc8b20 0x55820efcaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.350+0000 7f2bdde00700 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55820efc8b20 0x55820efcaf40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=933c3cf6e29e66f8 server_cookie=f65eb84e2fb5410b in_seq=0 out_seq=0 2024-05-27T07:38:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.350+0000 7f2bc3e00700 10 client.16401 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:38:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.350+0000 7f2bc3e00700 1 -- 172.21.15.63:0/1997844868 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55820ecdc250 con 0x55820efc8b20 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.350+0000 7f2bc3e00700 10 client.16401 handle_command_reply: tid=0 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.350+0000 7f2bf37f5b80 1 client.16401 shutdown 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/1997844868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 msgr2=0x55820efc15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820efc15f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800ff90 tx=0x7f2bc80cd040 comp rx=0 tx=0).stop 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/1997844868 shutdown_connections 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820ef462c0 0x55820ef51e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55820edbce40 0x55820efc10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55820efc8b20 0x55820efcaf40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 1 --2- 172.21.15.63:0/1997844868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55820eea89f0 0x55820efc15f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/1997844868 >> 172.21.15.63:0/1997844868 conn(0x55820ee56390 msgr2=0x55820ef42fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:42.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bc3e00700 0 client.16401 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:38:42.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/1997844868 shutdown_connections 2024-05-27T07:38:42.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 1 -- 172.21.15.63:0/1997844868 wait complete. 2024-05-27T07:38:42.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.351+0000 7f2bf37f5b80 20 client.16401 trim_cache size 0 max 16384 2024-05-27T07:38:42.350 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:38:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.352+0000 7f2bdf200700 1 -- 172.21.15.63:0/618913364 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2bc40b2a70 msgr2=0x7f2bc40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.352+0000 7f2bdf200700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2bc40b2a70 0x7f2bc40b4f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009d20 tx=0x7f2bd8009040 comp rx=0 tx=0).stop 2024-05-27T07:38:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.352+0000 7f2bdf200700 1 -- 172.21.15.63:0/618913364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be010af10 msgr2=0x7f2be019d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.352+0000 7f2bdf200700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be010af10 0x7f2be019d340 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80098e0 tx=0x7f2bc80047b0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.352+0000 7f2bdf200700 1 -- 172.21.15.63:0/618913364 shutdown_connections 2024-05-27T07:38:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.352+0000 7f2bdf200700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2bc40b2a70 0x7f2bc40b4f80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.352+0000 7f2bdf200700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2be010b860 0x7f2be01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.352+0000 7f2bdf200700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2be0106630 0x7f2be019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.352+0000 7f2bdf200700 1 --2- 172.21.15.63:0/618913364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be010af10 0x7f2be019d340 secure :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80098e0 tx=0x7f2bc80047b0 comp rx=0 tx=0).stop 2024-05-27T07:38:42.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.352+0000 7f2bdf200700 1 -- 172.21.15.63:0/618913364 >> 172.21.15.63:0/618913364 conn(0x7f2be0102020 msgr2=0x7f2be010c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:42.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.353+0000 7f2bdf200700 1 -- 172.21.15.63:0/618913364 shutdown_connections 2024-05-27T07:38:42.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:42.353+0000 7f2bdf200700 1 -- 172.21.15.63:0/618913364 wait complete. 2024-05-27T07:38:43.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:42 smithi086 ceph-mon[34687]: pgmap v157: 129 pgs: 129 active+clean; 734 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:38:43.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:42 smithi149 ceph-mon[35408]: pgmap v157: 129 pgs: 129 active+clean; 734 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:38:43.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:42 smithi063 ceph-mon[31613]: pgmap v157: 129 pgs: 129 active+clean; 734 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:38:43.376 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:38:43.898 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:38:45.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:44 smithi086 ceph-mon[34687]: pgmap v158: 129 pgs: 129 active+clean; 734 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:45.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:44 smithi149 ceph-mon[35408]: pgmap v158: 129 pgs: 129 active+clean; 734 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:45.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:44 smithi063 ceph-mon[31613]: pgmap v158: 129 pgs: 129 active+clean; 734 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:46.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.376+0000 7f802d600700 1 -- 172.21.15.63:0/3948894087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f802810af60 msgr2=0x7f802810b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:46.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.376+0000 7f802d600700 1 --2- 172.21.15.63:0/3948894087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f802810af60 0x7f802810b370 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f80140099c0 tx=0x7f8014009cd0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.376+0000 7f802d600700 1 -- 172.21.15.63:0/3948894087 shutdown_connections 2024-05-27T07:38:46.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.376+0000 7f802d600700 1 --2- 172.21.15.63:0/3948894087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f802810b8b0 0x7f8028113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.376+0000 7f802d600700 1 --2- 172.21.15.63:0/3948894087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8028106630 0x7f8028106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.376+0000 7f802d600700 1 --2- 172.21.15.63:0/3948894087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f802810af60 0x7f802810b370 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.376+0000 7f802d600700 1 -- 172.21.15.63:0/3948894087 >> 172.21.15.63:0/3948894087 conn(0x7f8028101ec0 msgr2=0x7f8028104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:46.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.376+0000 7f802d600700 1 -- 172.21.15.63:0/3948894087 shutdown_connections 2024-05-27T07:38:46.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.377+0000 7f802d600700 1 -- 172.21.15.63:0/3948894087 wait complete. 2024-05-27T07:38:46.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.378+0000 7f802d600700 1 Processor -- start 2024-05-27T07:38:46.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.378+0000 7f802d600700 1 -- start start 2024-05-27T07:38:46.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f802d600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8028106630 0x7f802819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f802d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f802810af60 0x7f802819d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f802d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f802810b8b0 0x7f80281a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f802d600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8028116390 con 0x7f802810b8b0 2024-05-27T07:38:46.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f802d600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8028116210 con 0x7f802810af60 2024-05-27T07:38:46.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f802d600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8028116510 con 0x7f8028106630 2024-05-27T07:38:46.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f802cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f802810b8b0 0x7f80281a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f802cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f802810b8b0 0x7f80281a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56986/0 (socket says 172.21.15.63:56986) 2024-05-27T07:38:46.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f802cc00700 1 -- 172.21.15.63:0/85822937 learned_addr learned my addr 172.21.15.63:0/85822937 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:46.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f8027400700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f802810af60 0x7f802819d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.379+0000 7f8027e00700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8028106630 0x7f802819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.380+0000 7f802cc00700 1 -- 172.21.15.63:0/85822937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8028106630 msgr2=0x7f802819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:46.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.380+0000 7f802cc00700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8028106630 0x7f802819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.380+0000 7f802cc00700 1 -- 172.21.15.63:0/85822937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f802810af60 msgr2=0x7f802819d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:46.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.380+0000 7f802cc00700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f802810af60 0x7f802819d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.380+0000 7f802cc00700 1 -- 172.21.15.63:0/85822937 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80281a1b80 con 0x7f802810b8b0 2024-05-27T07:38:46.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.380+0000 7f802cc00700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f802810b8b0 0x7f80281a1590 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f802000ada0 tx=0x7f802000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:46.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.380+0000 7f8024c00700 1 -- 172.21.15.63:0/85822937 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020017020 con 0x7f802810b8b0 2024-05-27T07:38:46.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.381+0000 7f8024c00700 1 -- 172.21.15.63:0/85822937 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f802000ce10 con 0x7f802810b8b0 2024-05-27T07:38:46.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.381+0000 7f802d600700 1 -- 172.21.15.63:0/85822937 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80281a1e70 con 0x7f802810b8b0 2024-05-27T07:38:46.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.381+0000 7f8024c00700 1 -- 172.21.15.63:0/85822937 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020005440 con 0x7f802810b8b0 2024-05-27T07:38:46.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.381+0000 7f802d600700 1 -- 172.21.15.63:0/85822937 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281a2380 con 0x7f802810b8b0 2024-05-27T07:38:46.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.385+0000 7f8024c00700 1 -- 172.21.15.63:0/85822937 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8020004ad0 con 0x7f802810b8b0 2024-05-27T07:38:46.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.387+0000 7f8024c00700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f800c0b29f0 0x7f800c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.387+0000 7f803d6c1b80 1 Processor -- start 2024-05-27T07:38:46.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.387+0000 7f8024c00700 1 -- 172.21.15.63:0/85822937 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f80200e2130 con 0x7f802810b8b0 2024-05-27T07:38:46.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.387+0000 7f8027e00700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f800c0b29f0 0x7f800c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.387+0000 7f803d6c1b80 1 -- start start 2024-05-27T07:38:46.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f752690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 0x55968f752bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f766ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55968f67fe40 con 0x55968f764ad0 2024-05-27T07:38:46.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55968f66f9e0 con 0x55968f5d2e40 2024-05-27T07:38:46.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55968f673a20 con 0x55968f6be9f0 2024-05-27T07:38:46.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f8027e00700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f800c0b29f0 0x7f800c0b4f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8018005ea0 tx=0x7f8018005e30 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:46.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f802cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f766ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f802cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f766ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56992/0 (socket says 172.21.15.63:56992) 2024-05-27T07:38:46.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f802cc00700 1 -- 172.21.15.63:0/811344056 learned_addr learned my addr 172.21.15.63:0/811344056 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:46.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f8027e00700 1 --2- 172.21.15.63:0/811344056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f752690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.388+0000 7f8027400700 1 --2- 172.21.15.63:0/811344056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 0x55968f752bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.389+0000 7f802cc00700 1 -- 172.21.15.63:0/811344056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 msgr2=0x55968f752690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:46.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.389+0000 7f802cc00700 1 --2- 172.21.15.63:0/811344056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f752690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.389+0000 7f802cc00700 1 -- 172.21.15.63:0/811344056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 msgr2=0x55968f752bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:46.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.389+0000 7f802cc00700 1 --2- 172.21.15.63:0/811344056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 0x55968f752bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.389+0000 7f802cc00700 1 -- 172.21.15.63:0/811344056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55968f638420 con 0x55968f764ad0 2024-05-27T07:38:46.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.389+0000 7f802cc00700 1 --2- 172.21.15.63:0/811344056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f766ef0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8020006130 tx=0x7f80200a8910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71706b4fd16ab1bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:46.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.390+0000 7f801d600700 1 -- 172.21.15.63:0/811344056 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200ad480 con 0x55968f764ad0 2024-05-27T07:38:46.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.390+0000 7f801d600700 1 -- 172.21.15.63:0/811344056 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80200b1ea0 con 0x55968f764ad0 2024-05-27T07:38:46.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.390+0000 7f801d600700 1 -- 172.21.15.63:0/811344056 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200a95c0 con 0x55968f764ad0 2024-05-27T07:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.391+0000 7f803d6c1b80 1 -- 172.21.15.63:0/811344056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 msgr2=0x55968f766ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.391+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/811344056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f766ef0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8020006130 tx=0x7f80200a8910 comp rx=0 tx=0).stop 2024-05-27T07:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.391+0000 7f803d6c1b80 1 -- 172.21.15.63:0/811344056 shutdown_connections 2024-05-27T07:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.391+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/811344056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f766ef0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.391+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/811344056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f752690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.391+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/811344056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 0x55968f752bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.391+0000 7f803d6c1b80 1 -- 172.21.15.63:0/811344056 >> 172.21.15.63:0/811344056 conn(0x55968f66c390 msgr2=0x55968f74d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:46.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.391+0000 7f803d6c1b80 1 -- 172.21.15.63:0/811344056 shutdown_connections 2024-05-27T07:38:46.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.391+0000 7f803d6c1b80 1 -- 172.21.15.63:0/811344056 wait complete. 2024-05-27T07:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.392+0000 7f803d6c1b80 1 -- start start 2024-05-27T07:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.392+0000 7f803d6c1b80 1 Processor -- start 2024-05-27T07:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 0x55968f761a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f761f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f7624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55968f575fa0 con 0x55968f764ad0 2024-05-27T07:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55968f610760 con 0x55968f5d2e40 2024-05-27T07:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55968f767580 con 0x55968f6be9f0 2024-05-27T07:38:46.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f803d6c1b80 10 client.0 _ll_register_callbacks cb 0x55968f619c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f802cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f7624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f802cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f7624a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57004/0 (socket says 172.21.15.63:57004) 2024-05-27T07:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f802cc00700 1 -- 172.21.15.63:0/2242954733 learned_addr learned my addr 172.21.15.63:0/2242954733 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:46.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f8027400700 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f761f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.393+0000 7f8027e00700 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 0x55968f761a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.394+0000 7f802cc00700 1 -- 172.21.15.63:0/2242954733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 msgr2=0x55968f761f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.394+0000 7f802cc00700 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f761f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.394+0000 7f802cc00700 1 -- 172.21.15.63:0/2242954733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 msgr2=0x55968f761a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.394+0000 7f802cc00700 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 0x55968f761a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.394+0000 7f802cc00700 1 -- 172.21.15.63:0/2242954733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968f5bf8d0 con 0x55968f764ad0 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.394+0000 7f8027e00700 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 0x55968f761a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.394+0000 7f802cc00700 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f7624a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8020004ff0 tx=0x7f80200ad350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfdc41046b2f4f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.394+0000 7f8012000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.394+0000 7f8012000700 1 -- 172.21.15.63:0/2242954733 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200ad480 con 0x55968f764ad0 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.395+0000 7f8012000700 1 -- 172.21.15.63:0/2242954733 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f80200b1c20 con 0x55968f764ad0 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.395+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2242954733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80200b1ea0 con 0x55968f764ad0 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.395+0000 7f8012000700 1 -- 172.21.15.63:0/2242954733 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f80200ca050 con 0x55968f764ad0 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.395+0000 7f803d6c1b80 10 client.16413 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.395+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2242954733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55968f640060 con 0x55968f764ad0 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.395+0000 7f8012000700 1 -- 172.21.15.63:0/2242954733 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80200b1ea0 con 0x55968f764ad0 2024-05-27T07:38:46.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.396+0000 7f8012000700 1 -- 172.21.15.63:0/2242954733 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f80200cb030 con 0x55968f764ad0 2024-05-27T07:38:46.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.396+0000 7f803d6c1b80 10 client.16413 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:46.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.396+0000 7f803d6c1b80 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:46.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.396+0000 7f803d6c1b80 20 client.16413 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:46.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.396+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55968f7742b0 0x55968f7766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.396+0000 7f803d6c1b80 4 client.16413 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:38:46.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.396+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2242954733 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55968f681d90 con 0x55968f7742b0 2024-05-27T07:38:46.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.397+0000 7f8027e00700 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55968f7742b0 0x55968f7766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.398+0000 7f8027e00700 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55968f7742b0 0x55968f7766d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d2b99a95e3ddfb9 server_cookie=f1b58585c1cff4ce in_seq=0 out_seq=0 2024-05-27T07:38:46.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.398+0000 7f8012000700 10 client.16413 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:46.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f8012000700 1 -- 172.21.15.63:0/2242954733 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55968f681d90 con 0x55968f7742b0 2024-05-27T07:38:46.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f8012000700 10 client.16413 handle_command_reply: tid=0 2024-05-27T07:38:46.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 client.16413 shutdown 2024-05-27T07:38:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2242954733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 msgr2=0x55968f7624a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f7624a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8020004ff0 tx=0x7f80200ad350 comp rx=0 tx=0).stop 2024-05-27T07:38:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2242954733 shutdown_connections 2024-05-27T07:38:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55968f7742b0 0x55968f7766d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f764ad0 0x55968f7624a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f8012000700 0 client.16413 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:46.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f5d2e40 0x55968f761a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2242954733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f761f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2242954733 >> 172.21.15.63:0/2242954733 conn(0x55968f66c390 msgr2=0x55968f752690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:46.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2242954733 shutdown_connections 2024-05-27T07:38:46.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.400+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2242954733 wait complete. 2024-05-27T07:38:46.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.401+0000 7f803d6c1b80 20 client.16413 trim_cache size 0 max 16384 2024-05-27T07:38:46.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.421+0000 7f803d6c1b80 1 Processor -- start 2024-05-27T07:38:46.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.421+0000 7f803d6c1b80 1 -- start start 2024-05-27T07:38:46.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.422+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f5d2e40 0x55968f75d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.422+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f6be9f0 0x55968f74d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.422+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f764ad0 0x55968f75ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.422+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80281b0790 con 0x55968f6be9f0 2024-05-27T07:38:46.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.422+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80281ae020 con 0x55968f764ad0 2024-05-27T07:38:46.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.422+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f80281abe90 con 0x55968f5d2e40 2024-05-27T07:38:46.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.422+0000 7f8027400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f6be9f0 0x55968f74d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.422+0000 7f8027400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f6be9f0 0x55968f74d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57018/0 (socket says 172.21.15.63:57018) 2024-05-27T07:38:46.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.422+0000 7f8027400700 1 -- 172.21.15.63:0/780094685 learned_addr learned my addr 172.21.15.63:0/780094685 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.423+0000 7f8027e00700 1 --2- 172.21.15.63:0/780094685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f5d2e40 0x55968f75d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.423+0000 7f802cc00700 1 --2- 172.21.15.63:0/780094685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f764ad0 0x55968f75ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.423+0000 7f8027400700 1 -- 172.21.15.63:0/780094685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f5d2e40 msgr2=0x55968f75d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.423+0000 7f8027400700 1 --2- 172.21.15.63:0/780094685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f5d2e40 0x55968f75d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.423+0000 7f8027400700 1 -- 172.21.15.63:0/780094685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f764ad0 msgr2=0x55968f75ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.423+0000 7f8027400700 1 --2- 172.21.15.63:0/780094685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f764ad0 0x55968f75ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.423+0000 7f8027400700 1 -- 172.21.15.63:0/780094685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55968f762810 con 0x55968f6be9f0 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.424+0000 7f8027400700 1 --2- 172.21.15.63:0/780094685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f6be9f0 0x55968f74d740 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f80140058e0 tx=0x7f8014005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e5a6ac42aeea1a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.424+0000 7f801cc00700 1 -- 172.21.15.63:0/780094685 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014029070 con 0x55968f6be9f0 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.424+0000 7f801cc00700 1 -- 172.21.15.63:0/780094685 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f801400bc20 con 0x55968f6be9f0 2024-05-27T07:38:46.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.424+0000 7f801cc00700 1 -- 172.21.15.63:0/780094685 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014003d60 con 0x55968f6be9f0 2024-05-27T07:38:46.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.425+0000 7f803d6c1b80 1 -- 172.21.15.63:0/780094685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f6be9f0 msgr2=0x55968f74d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:46.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.425+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/780094685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f6be9f0 0x55968f74d740 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f80140058e0 tx=0x7f8014005040 comp rx=0 tx=0).stop 2024-05-27T07:38:46.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.425+0000 7f803d6c1b80 1 -- 172.21.15.63:0/780094685 shutdown_connections 2024-05-27T07:38:46.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.425+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/780094685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f764ad0 0x55968f75ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.425+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/780094685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f5d2e40 0x55968f75d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.425+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/780094685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f6be9f0 0x55968f74d740 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.425+0000 7f803d6c1b80 1 -- 172.21.15.63:0/780094685 >> 172.21.15.63:0/780094685 conn(0x55968f66c390 msgr2=0x55968f75ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:46.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.425+0000 7f803d6c1b80 1 -- 172.21.15.63:0/780094685 shutdown_connections 2024-05-27T07:38:46.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.425+0000 7f803d6c1b80 1 -- 172.21.15.63:0/780094685 wait complete. 2024-05-27T07:38:46.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.426+0000 7f803d6c1b80 1 -- start start 2024-05-27T07:38:46.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.426+0000 7f803d6c1b80 1 Processor -- start 2024-05-27T07:38:46.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.427+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f5d2e40 0x55968f7797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.427+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f779d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.427+0000 7f803d6c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f764ad0 0x55968f776c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.427+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80281ac4e0 con 0x55968f5d2e40 2024-05-27T07:38:46.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.427+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80281ad950 con 0x55968f764ad0 2024-05-27T07:38:46.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.427+0000 7f803d6c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f80281acbb0 con 0x55968f6be9f0 2024-05-27T07:38:46.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.427+0000 7f803d6c1b80 10 client.0 _ll_register_callbacks cb 0x55968f619c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:46.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.427+0000 7f8027e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f5d2e40 0x55968f7797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.427+0000 7f8027e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f5d2e40 0x55968f7797f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57020/0 (socket says 172.21.15.63:57020) 2024-05-27T07:38:46.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.427+0000 7f8027e00700 1 -- 172.21.15.63:0/2199090578 learned_addr learned my addr 172.21.15.63:0/2199090578 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:46.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.428+0000 7f8027400700 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f779d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.428+0000 7f802cc00700 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f764ad0 0x55968f776c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.428+0000 7f8027e00700 1 -- 172.21.15.63:0/2199090578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 msgr2=0x55968f779d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:46.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.428+0000 7f8027e00700 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f779d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.428+0000 7f8027e00700 1 -- 172.21.15.63:0/2199090578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f764ad0 msgr2=0x55968f776c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:46.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.428+0000 7f8027e00700 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f764ad0 0x55968f776c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.428+0000 7f8027e00700 1 -- 172.21.15.63:0/2199090578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968f777310 con 0x55968f5d2e40 2024-05-27T07:38:46.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.428+0000 7f8027e00700 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f5d2e40 0x55968f7797f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x55968f753510 tx=0x7f80180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840c8054914aa8eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:46.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.428+0000 7f8012000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:46.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.429+0000 7f8012000700 1 -- 172.21.15.63:0/2199090578 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018027640 con 0x55968f5d2e40 2024-05-27T07:38:46.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.429+0000 7f8012000700 1 -- 172.21.15.63:0/2199090578 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f80180277e0 con 0x55968f5d2e40 2024-05-27T07:38:46.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.429+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2199090578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801400bc20 con 0x55968f5d2e40 2024-05-27T07:38:46.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.429+0000 7f8012000700 1 -- 172.21.15.63:0/2199090578 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8018039050 con 0x55968f5d2e40 2024-05-27T07:38:46.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.429+0000 7f803d6c1b80 10 client.16419 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:46.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.429+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2199090578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55968f7de500 con 0x55968f5d2e40 2024-05-27T07:38:46.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.430+0000 7f8012000700 1 -- 172.21.15.63:0/2199090578 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f801400bc20 con 0x55968f5d2e40 2024-05-27T07:38:46.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.430+0000 7f8012000700 1 -- 172.21.15.63:0/2199090578 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f801801d420 con 0x55968f5d2e40 2024-05-27T07:38:46.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.430+0000 7f803d6c1b80 10 client.16419 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:46.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.430+0000 7f803d6c1b80 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:46.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.430+0000 7f803d6c1b80 20 client.16419 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:46.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.430+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55968f7742b0 0x55968f7e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:46.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.430+0000 7f803d6c1b80 4 client.16419 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:38:46.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.430+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2199090578 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55968f67b570 con 0x55968f7742b0 2024-05-27T07:38:46.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.431+0000 7f8027400700 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55968f7742b0 0x55968f7e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:46.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.432+0000 7f8012000700 10 client.16419 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:46.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.432+0000 7f8027400700 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55968f7742b0 0x55968f7e0c30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9445e5aea25a68 server_cookie=d5f3694d674284da in_seq=0 out_seq=0 2024-05-27T07:38:46.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.432+0000 7f8012000700 1 -- 172.21.15.63:0/2199090578 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55968f67b570 con 0x55968f7742b0 2024-05-27T07:38:46.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.432+0000 7f8012000700 10 client.16419 handle_command_reply: tid=0 2024-05-27T07:38:46.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.432+0000 7f803d6c1b80 1 client.16419 shutdown 2024-05-27T07:38:46.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2199090578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f5d2e40 msgr2=0x55968f7797f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:46.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f5d2e40 0x55968f7797f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x55968f753510 tx=0x7f80180047c0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2199090578 shutdown_connections 2024-05-27T07:38:46.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55968f7742b0 0x55968f7e0c30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55968f764ad0 0x55968f776c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55968f5d2e40 0x55968f7797f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 1 --2- 172.21.15.63:0/2199090578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55968f6be9f0 0x55968f779d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2199090578 >> 172.21.15.63:0/2199090578 conn(0x55968f66c390 msgr2=0x55968f751120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:46.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f8012000700 0 client.16419 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:46.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2199090578 shutdown_connections 2024-05-27T07:38:46.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 1 -- 172.21.15.63:0/2199090578 wait complete. 2024-05-27T07:38:46.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.433+0000 7f803d6c1b80 20 client.16419 trim_cache size 0 max 16384 2024-05-27T07:38:46.431 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:38:46.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.434+0000 7f802d600700 1 -- 172.21.15.63:0/85822937 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f800c0b29f0 msgr2=0x7f800c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:46.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.434+0000 7f802d600700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f800c0b29f0 0x7f800c0b4f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8018005ea0 tx=0x7f8018005e30 comp rx=0 tx=0).stop 2024-05-27T07:38:46.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.434+0000 7f802d600700 1 -- 172.21.15.63:0/85822937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f802810b8b0 msgr2=0x7f80281a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:46.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.434+0000 7f802d600700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f802810b8b0 0x7f80281a1590 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f802000ada0 tx=0x7f802000c330 comp rx=0 tx=0).stop 2024-05-27T07:38:46.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.434+0000 7f802d600700 1 -- 172.21.15.63:0/85822937 shutdown_connections 2024-05-27T07:38:46.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.434+0000 7f802d600700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f800c0b29f0 0x7f800c0b4f00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.434+0000 7f802d600700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f802810b8b0 0x7f80281a1590 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.434+0000 7f802d600700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8028106630 0x7f802819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.434+0000 7f802d600700 1 --2- 172.21.15.63:0/85822937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f802810af60 0x7f802819d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:46.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.434+0000 7f802d600700 1 -- 172.21.15.63:0/85822937 >> 172.21.15.63:0/85822937 conn(0x7f8028101ec0 msgr2=0x7f802810c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:46.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.435+0000 7f802d600700 1 -- 172.21.15.63:0/85822937 shutdown_connections 2024-05-27T07:38:46.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:46.435+0000 7f802d600700 1 -- 172.21.15.63:0/85822937 wait complete. 2024-05-27T07:38:47.205 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:46 smithi086 ceph-mon[34687]: pgmap v159: 129 pgs: 129 active+clean; 734 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:47.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:46 smithi149 ceph-mon[35408]: pgmap v159: 129 pgs: 129 active+clean; 734 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:47.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:46 smithi063 ceph-mon[31613]: pgmap v159: 129 pgs: 129 active+clean; 734 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:48.450 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:38:48.450 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:38:48.967 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:38:49.209 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:48 smithi149 ceph-mon[35408]: pgmap v160: 129 pgs: 129 active+clean; 766 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:38:49.212 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:48 smithi063 ceph-mon[31613]: pgmap v160: 129 pgs: 129 active+clean; 766 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:38:49.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:48 smithi086 ceph-mon[34687]: pgmap v160: 129 pgs: 129 active+clean; 766 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:38:51.096 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:50 smithi063 ceph-mon[31613]: pgmap v161: 129 pgs: 129 active+clean; 766 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:51.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:50 smithi086 ceph-mon[34687]: pgmap v161: 129 pgs: 129 active+clean; 766 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:51.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:50 smithi149 ceph-mon[35408]: pgmap v161: 129 pgs: 129 active+clean; 766 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:38:51.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.480+0000 7f6afe600700 1 -- 172.21.15.63:0/2071215373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6af8109860 msgr2=0x7f6af81111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.480+0000 7f6afe600700 1 --2- 172.21.15.63:0/2071215373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6af8109860 0x7f6af81111c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6af000b0d0 tx=0x7f6af000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.480+0000 7f6afe600700 1 -- 172.21.15.63:0/2071215373 shutdown_connections 2024-05-27T07:38:51.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.480+0000 7f6afe600700 1 --2- 172.21.15.63:0/2071215373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6af8109860 0x7f6af81111c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.480+0000 7f6afe600700 1 --2- 172.21.15.63:0/2071215373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af81045e0 0x7f6af81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.480+0000 7f6afe600700 1 --2- 172.21.15.63:0/2071215373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af8108f10 0x7f6af8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.480+0000 7f6afe600700 1 -- 172.21.15.63:0/2071215373 >> 172.21.15.63:0/2071215373 conn(0x7f6af8100030 msgr2=0x7f6af8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:51.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.480+0000 7f6afe600700 1 -- 172.21.15.63:0/2071215373 shutdown_connections 2024-05-27T07:38:51.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.480+0000 7f6afe600700 1 -- 172.21.15.63:0/2071215373 wait complete. 2024-05-27T07:38:51.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.482+0000 7f6afe600700 1 Processor -- start 2024-05-27T07:38:51.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.482+0000 7f6afe600700 1 -- start start 2024-05-27T07:38:51.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.482+0000 7f6afe600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af81045e0 0x7f6af819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.482+0000 7f6afe600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6af8108f10 0x7f6af819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.483+0000 7f6afe600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af8109860 0x7f6af81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.483+0000 7f6afe600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8114340 con 0x7f6af8108f10 2024-05-27T07:38:51.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.483+0000 7f6afe600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81141c0 con 0x7f6af8109860 2024-05-27T07:38:51.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.483+0000 7f6afe600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81144c0 con 0x7f6af81045e0 2024-05-27T07:38:51.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.483+0000 7f6af6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6af8108f10 0x7f6af819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.483+0000 7f6af6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6af8108f10 0x7f6af819d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50044/0 (socket says 172.21.15.63:50044) 2024-05-27T07:38:51.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.483+0000 7f6af6a00700 1 -- 172.21.15.63:0/1969008636 learned_addr learned my addr 172.21.15.63:0/1969008636 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:51.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.483+0000 7f6af7400700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af81045e0 0x7f6af819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.483+0000 7f6af7e00700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af8109860 0x7f6af81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6af6a00700 1 -- 172.21.15.63:0/1969008636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af81045e0 msgr2=0x7f6af819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6af6a00700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af81045e0 0x7f6af819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6af6a00700 1 -- 172.21.15.63:0/1969008636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af8109860 msgr2=0x7f6af81a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6af6a00700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af8109860 0x7f6af81a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6af6a00700 1 -- 172.21.15.63:0/1969008636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af81a1e00 con 0x7f6af8108f10 2024-05-27T07:38:51.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6af7e00700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af8109860 0x7f6af81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:51.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6af6a00700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6af8108f10 0x7f6af819d4d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400d980 tx=0x7f6ae400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:51.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6aefe00700 1 -- 172.21.15.63:0/1969008636 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4017070 con 0x7f6af8108f10 2024-05-27T07:38:51.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6aefe00700 1 -- 172.21.15.63:0/1969008636 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ae4009dc0 con 0x7f6af8108f10 2024-05-27T07:38:51.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6aefe00700 1 -- 172.21.15.63:0/1969008636 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae400f3c0 con 0x7f6af8108f10 2024-05-27T07:38:51.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.484+0000 7f6afe600700 1 -- 172.21.15.63:0/1969008636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af81a20f0 con 0x7f6af8108f10 2024-05-27T07:38:51.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.485+0000 7f6afe600700 1 -- 172.21.15.63:0/1969008636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af81a9a10 con 0x7f6af8108f10 2024-05-27T07:38:51.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.490+0000 7f6b0d069b80 1 Processor -- start 2024-05-27T07:38:51.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.490+0000 7f6b0d069b80 1 -- start start 2024-05-27T07:38:51.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.491+0000 7f6aefe00700 1 -- 172.21.15.63:0/1969008636 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6ae4009930 con 0x7f6af8108f10 2024-05-27T07:38:51.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.491+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac65923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.491+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 0x55dac65928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.491+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6592ef0 0x55dac659f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.491+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dac64c0ec0 con 0x55dac6413ec0 2024-05-27T07:38:51.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.491+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dac64b0a60 con 0x55dac64ffa70 2024-05-27T07:38:51.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.491+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dac64b4aa0 con 0x55dac6592ef0 2024-05-27T07:38:51.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.491+0000 7f6af7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac65923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.492+0000 7f6af7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 0x55dac65928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.492+0000 7f6af7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac65923b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44662/0 (socket says 172.21.15.63:44662) 2024-05-27T07:38:51.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.492+0000 7f6af7e00700 1 -- 172.21.15.63:0/2011424002 learned_addr learned my addr 172.21.15.63:0/2011424002 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:51.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.492+0000 7f6af6a00700 1 --2- 172.21.15.63:0/2011424002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6592ef0 0x55dac659f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.492+0000 7f6af7e00700 1 -- 172.21.15.63:0/2011424002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6592ef0 msgr2=0x55dac659f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.492+0000 7f6af7e00700 1 --2- 172.21.15.63:0/2011424002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6592ef0 0x55dac659f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.492+0000 7f6af7e00700 1 -- 172.21.15.63:0/2011424002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 msgr2=0x55dac65928f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.492+0000 7f6af7e00700 1 --2- 172.21.15.63:0/2011424002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 0x55dac65928f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.492+0000 7f6af7e00700 1 -- 172.21.15.63:0/2011424002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dac64794a0 con 0x55dac64ffa70 2024-05-27T07:38:51.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.493+0000 7f6aefe00700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6adc0b29f0 0x7f6adc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.493+0000 7f6aefe00700 1 -- 172.21.15.63:0/1969008636 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f6ae40e2440 con 0x7f6af8108f10 2024-05-27T07:38:51.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.493+0000 7f6af7e00700 1 --2- 172.21.15.63:0/2011424002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac65923b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009dc0 tx=0x7f6af0009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa44ef5efe6d8a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:51.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.493+0000 7f6af7400700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6adc0b29f0 0x7f6adc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.494+0000 7f6aecc00700 1 -- 172.21.15.63:0/2011424002 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0022070 con 0x55dac64ffa70 2024-05-27T07:38:51.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.494+0000 7f6aecc00700 1 -- 172.21.15.63:0/2011424002 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6af0009350 con 0x55dac64ffa70 2024-05-27T07:38:51.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.494+0000 7f6aecc00700 1 -- 172.21.15.63:0/2011424002 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af001c8c0 con 0x55dac64ffa70 2024-05-27T07:38:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.495+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2011424002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 msgr2=0x55dac65923b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.495+0000 7f6af7400700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6adc0b29f0 0x7f6adc0b4f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55dac658f810 tx=0x7f6ae800b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.495+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/2011424002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac65923b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009dc0 tx=0x7f6af0009df0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.495+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2011424002 shutdown_connections 2024-05-27T07:38:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.495+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/2011424002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6592ef0 0x55dac659f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.495+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/2011424002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac65923b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.495+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/2011424002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 0x55dac65928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.495+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2011424002 >> 172.21.15.63:0/2011424002 conn(0x55dac64ad410 msgr2=0x55dac658e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.495+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2011424002 shutdown_connections 2024-05-27T07:38:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.495+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2011424002 wait complete. 2024-05-27T07:38:51.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.496+0000 7f6b0d069b80 1 -- start start 2024-05-27T07:38:51.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.496+0000 7f6b0d069b80 1 Processor -- start 2024-05-27T07:38:51.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.497+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 0x55dac659c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.497+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac659cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.497+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6592ef0 0x55dac659d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.497+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dac63b7020 con 0x55dac6413ec0 2024-05-27T07:38:51.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.497+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dac64517e0 con 0x55dac64ffa70 2024-05-27T07:38:51.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.497+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dac659fba0 con 0x55dac6592ef0 2024-05-27T07:38:51.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.497+0000 7f6b0d069b80 10 client.0 _ll_register_callbacks cb 0x55dac645acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:51.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.497+0000 7f6af7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 0x55dac659c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.497+0000 7f6af7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 0x55dac659c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50070/0 (socket says 172.21.15.63:50070) 2024-05-27T07:38:51.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.497+0000 7f6af7400700 1 -- 172.21.15.63:0/3941098911 learned_addr learned my addr 172.21.15.63:0/3941098911 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:51.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.498+0000 7f6af6a00700 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac659cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.498+0000 7f6af7e00700 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6592ef0 0x55dac659d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.498+0000 7f6af7400700 1 -- 172.21.15.63:0/3941098911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6592ef0 msgr2=0x55dac659d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.498+0000 7f6af7400700 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6592ef0 0x55dac659d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.498+0000 7f6af7400700 1 -- 172.21.15.63:0/3941098911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 msgr2=0x55dac659cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.498+0000 7f6af7400700 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac659cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.498+0000 7f6af7400700 1 -- 172.21.15.63:0/3941098911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dac6400950 con 0x55dac6413ec0 2024-05-27T07:38:51.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.498+0000 7f6af6a00700 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac659cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:51.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.499+0000 7f6af7400700 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 0x55dac659c6e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8003b00 tx=0x7f6ae80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712daf3697df4597 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:51.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.499+0000 7f6ae1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:51.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.499+0000 7f6ae1600700 1 -- 172.21.15.63:0/3941098911 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae802c060 con 0x55dac6413ec0 2024-05-27T07:38:51.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.499+0000 7f6ae1600700 1 -- 172.21.15.63:0/3941098911 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f6ae8017030 con 0x55dac6413ec0 2024-05-27T07:38:51.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.499+0000 7f6b0d069b80 1 -- 172.21.15.63:0/3941098911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6af0009350 con 0x55dac6413ec0 2024-05-27T07:38:51.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.500+0000 7f6ae1600700 1 -- 172.21.15.63:0/3941098911 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ae8033050 con 0x55dac6413ec0 2024-05-27T07:38:51.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.500+0000 7f6b0d069b80 10 client.16431 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:51.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.500+0000 7f6b0d069b80 1 -- 172.21.15.63:0/3941098911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dac64810e0 con 0x55dac6413ec0 2024-05-27T07:38:51.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.500+0000 7f6ae1600700 1 -- 172.21.15.63:0/3941098911 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6af0009350 con 0x55dac6413ec0 2024-05-27T07:38:51.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.500+0000 7f6ae1600700 1 -- 172.21.15.63:0/3941098911 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6ae8011af0 con 0x55dac6413ec0 2024-05-27T07:38:51.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.500+0000 7f6b0d069b80 10 client.16431 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:51.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.501+0000 7f6b0d069b80 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:51.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.501+0000 7f6b0d069b80 20 client.16431 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:51.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.501+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dac65acbd0 0x55dac65af070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.501+0000 7f6b0d069b80 4 client.16431 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:38:51.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.501+0000 7f6b0d069b80 1 -- 172.21.15.63:0/3941098911 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dac64c2e10 con 0x55dac65acbd0 2024-05-27T07:38:51.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.501+0000 7f6af6a00700 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dac65acbd0 0x55dac65af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.503+0000 7f6af6a00700 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dac65acbd0 0x55dac65af070 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4826cf7db7668362 server_cookie=81bac6d393be0440 in_seq=0 out_seq=0 2024-05-27T07:38:51.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.503+0000 7f6ae1600700 10 client.16431 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.504+0000 7f6ae1600700 1 -- 172.21.15.63:0/3941098911 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dac64c2e10 con 0x55dac65acbd0 2024-05-27T07:38:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.504+0000 7f6ae1600700 10 client.16431 handle_command_reply: tid=0 2024-05-27T07:38:51.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 client.16431 shutdown 2024-05-27T07:38:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 -- 172.21.15.63:0/3941098911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 msgr2=0x55dac659c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 0x55dac659c6e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8003b00 tx=0x7f6ae80047c0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 -- 172.21.15.63:0/3941098911 shutdown_connections 2024-05-27T07:38:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dac65acbd0 0x55dac65af070 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6592ef0 0x55dac659d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac6413ec0 0x55dac659c6e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/3941098911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac64ffa70 0x55dac659cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 -- 172.21.15.63:0/3941098911 >> 172.21.15.63:0/3941098911 conn(0x55dac64ad410 msgr2=0x55dac6592390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:51.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6ae1600700 0 client.16431 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:51.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 -- 172.21.15.63:0/3941098911 shutdown_connections 2024-05-27T07:38:51.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.505+0000 7f6b0d069b80 1 -- 172.21.15.63:0/3941098911 wait complete. 2024-05-27T07:38:51.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.506+0000 7f6b0d069b80 20 client.16431 trim_cache size 0 max 16384 2024-05-27T07:38:51.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.537+0000 7f6b0d069b80 1 Processor -- start 2024-05-27T07:38:51.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.537+0000 7f6b0d069b80 1 -- start start 2024-05-27T07:38:51.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.538+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6413ec0 0x55dac6597f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.538+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac659e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.538+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 0x55dac65a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.538+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81b4e00 con 0x55dac64ffa70 2024-05-27T07:38:51.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.538+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81b2690 con 0x55dac6592ef0 2024-05-27T07:38:51.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.538+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81b04e0 con 0x55dac6413ec0 2024-05-27T07:38:51.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.538+0000 7f6af6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac659e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.538+0000 7f6af6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac659e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50086/0 (socket says 172.21.15.63:50086) 2024-05-27T07:38:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.538+0000 7f6af6a00700 1 -- 172.21.15.63:0/390484728 learned_addr learned my addr 172.21.15.63:0/390484728 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.538+0000 7f6af7400700 1 --2- 172.21.15.63:0/390484728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6413ec0 0x55dac6597f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.539+0000 7f6af7e00700 1 --2- 172.21.15.63:0/390484728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 0x55dac65a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.539+0000 7f6af6a00700 1 -- 172.21.15.63:0/390484728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6413ec0 msgr2=0x55dac6597f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.539+0000 7f6af6a00700 1 --2- 172.21.15.63:0/390484728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6413ec0 0x55dac6597f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.539+0000 7f6af6a00700 1 -- 172.21.15.63:0/390484728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 msgr2=0x55dac65a8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.539+0000 7f6af6a00700 1 --2- 172.21.15.63:0/390484728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 0x55dac65a8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.539+0000 7f6af6a00700 1 -- 172.21.15.63:0/390484728 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dac659ac50 con 0x55dac64ffa70 2024-05-27T07:38:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.539+0000 7f6af6a00700 1 --2- 172.21.15.63:0/390484728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac659e7c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4000c00 tx=0x7f6ae40e4950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ce0920443c5c25 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.540+0000 7f6ae3e00700 1 -- 172.21.15.63:0/390484728 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4020040 con 0x55dac64ffa70 2024-05-27T07:38:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.540+0000 7f6ae3e00700 1 -- 172.21.15.63:0/390484728 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ae40a8ec0 con 0x55dac64ffa70 2024-05-27T07:38:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.540+0000 7f6ae3e00700 1 -- 172.21.15.63:0/390484728 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40b2210 con 0x55dac64ffa70 2024-05-27T07:38:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.541+0000 7f6b0d069b80 1 -- 172.21.15.63:0/390484728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 msgr2=0x55dac659e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.541+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/390484728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac659e7c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4000c00 tx=0x7f6ae40e4950 comp rx=0 tx=0).stop 2024-05-27T07:38:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.541+0000 7f6b0d069b80 1 -- 172.21.15.63:0/390484728 shutdown_connections 2024-05-27T07:38:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.541+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/390484728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 0x55dac65a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.541+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/390484728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6413ec0 0x55dac6597f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.541+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/390484728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac659e7c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.541+0000 7f6b0d069b80 1 -- 172.21.15.63:0/390484728 >> 172.21.15.63:0/390484728 conn(0x55dac64ad410 msgr2=0x55dac659b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:51.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.541+0000 7f6b0d069b80 1 -- 172.21.15.63:0/390484728 shutdown_connections 2024-05-27T07:38:51.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.541+0000 7f6b0d069b80 1 -- 172.21.15.63:0/390484728 wait complete. 2024-05-27T07:38:51.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.542+0000 7f6b0d069b80 1 -- start start 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.542+0000 7f6b0d069b80 1 Processor -- start 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6413ec0 0x55dac65a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac65a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6b0d069b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 0x55dac659fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81b0b50 con 0x55dac64ffa70 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81b1fc0 con 0x55dac6592ef0 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6b0d069b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81b1220 con 0x55dac6413ec0 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6b0d069b80 10 client.0 _ll_register_callbacks cb 0x55dac645acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6af6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac65a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6af6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac65a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50094/0 (socket says 172.21.15.63:50094) 2024-05-27T07:38:51.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6af6a00700 1 -- 172.21.15.63:0/2854995759 learned_addr learned my addr 172.21.15.63:0/2854995759 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:51.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6af7400700 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6413ec0 0x55dac65a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.543+0000 7f6af7e00700 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 0x55dac659fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6af6a00700 1 -- 172.21.15.63:0/2854995759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6413ec0 msgr2=0x55dac65a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6af6a00700 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6413ec0 0x55dac65a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6af6a00700 1 -- 172.21.15.63:0/2854995759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 msgr2=0x55dac659fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6af6a00700 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 0x55dac659fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6af6a00700 1 -- 172.21.15.63:0/2854995759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dac65a0390 con 0x55dac64ffa70 2024-05-27T07:38:51.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6af7e00700 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 0x55dac659fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:38:51.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6af6a00700 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac65a53e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40a90b0 tx=0x7f6ae40a8b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700dd4387799fd04 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6ae1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6ae1600700 1 -- 172.21.15.63:0/2854995759 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4020070 con 0x55dac64ffa70 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6ae1600700 1 -- 172.21.15.63:0/2854995759 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f6ae4010460 con 0x55dac64ffa70 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.544+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2854995759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae40a8ec0 con 0x55dac64ffa70 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6ae1600700 1 -- 172.21.15.63:0/2854995759 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ae40c0080 con 0x55dac64ffa70 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6b0d069b80 10 client.16437 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2854995759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dac6617ea0 con 0x55dac64ffa70 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6ae1600700 1 -- 172.21.15.63:0/2854995759 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ae40a8ec0 con 0x55dac64ffa70 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6ae1600700 1 -- 172.21.15.63:0/2854995759 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6ae40a9610 con 0x55dac64ffa70 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6b0d069b80 10 client.16437 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6b0d069b80 10 client.16437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6b0d069b80 20 client.16437 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:51.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dac6618560 0x55dac661a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6b0d069b80 4 client.16437 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:38:51.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.545+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2854995759 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dac64bc5f0 con 0x55dac6618560 2024-05-27T07:38:51.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.546+0000 7f6af7400700 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dac6618560 0x55dac661a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.547+0000 7f6af7400700 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dac6618560 0x55dac661a980 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b0ef5dbba246c11 server_cookie=daa23d91386ab150 in_seq=0 out_seq=0 2024-05-27T07:38:51.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.547+0000 7f6ae1600700 10 client.16437 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:51.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.549+0000 7f6ae1600700 1 -- 172.21.15.63:0/2854995759 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dac64bc5f0 con 0x55dac6618560 2024-05-27T07:38:51.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.549+0000 7f6ae1600700 10 client.16437 handle_command_reply: tid=0 2024-05-27T07:38:51.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.549+0000 7f6b0d069b80 1 client.16437 shutdown 2024-05-27T07:38:51.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2854995759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 msgr2=0x55dac65a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac65a53e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40a90b0 tx=0x7f6ae40a8b20 comp rx=0 tx=0).stop 2024-05-27T07:38:51.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2854995759 shutdown_connections 2024-05-27T07:38:51.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dac6618560 0x55dac661a980 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dac6592ef0 0x55dac659fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dac6413ec0 0x55dac65a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6b0d069b80 1 --2- 172.21.15.63:0/2854995759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dac64ffa70 0x55dac65a53e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6ae1600700 0 client.16437 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:51.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2854995759 >> 172.21.15.63:0/2854995759 conn(0x55dac64ad410 msgr2=0x55dac659dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:51.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2854995759 shutdown_connections 2024-05-27T07:38:51.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.550+0000 7f6b0d069b80 1 -- 172.21.15.63:0/2854995759 wait complete. 2024-05-27T07:38:51.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.551+0000 7f6b0d069b80 20 client.16437 trim_cache size 0 max 16384 2024-05-27T07:38:51.549 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:38:51.549 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:38:51.549 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "02a74816-6245-422a-9bfb-3f9b57de53dc", 2024-05-27T07:38:51.549 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:38:51.549 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:38:51.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.551+0000 7f6afe600700 1 -- 172.21.15.63:0/1969008636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6adc0b29f0 msgr2=0x7f6adc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.551+0000 7f6afe600700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6adc0b29f0 0x7f6adc0b4f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55dac658f810 tx=0x7f6ae800b410 comp rx=0 tx=0).stop 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.551+0000 7f6afe600700 1 -- 172.21.15.63:0/1969008636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6af8108f10 msgr2=0x7f6af819d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.551+0000 7f6afe600700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6af8108f10 0x7f6af819d4d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400d980 tx=0x7f6ae400dc90 comp rx=0 tx=0).stop 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.552+0000 7f6afe600700 1 -- 172.21.15.63:0/1969008636 shutdown_connections 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.552+0000 7f6afe600700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6adc0b29f0 0x7f6adc0b4f00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.552+0000 7f6afe600700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af8109860 0x7f6af81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.552+0000 7f6afe600700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6af81045e0 0x7f6af819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.552+0000 7f6afe600700 1 --2- 172.21.15.63:0/1969008636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6af8108f10 0x7f6af819d4d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.552+0000 7f6afe600700 1 -- 172.21.15.63:0/1969008636 >> 172.21.15.63:0/1969008636 conn(0x7f6af8100030 msgr2=0x7f6af8110ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.552+0000 7f6afe600700 1 -- 172.21.15.63:0/1969008636 shutdown_connections 2024-05-27T07:38:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:51.552+0000 7f6afe600700 1 -- 172.21.15.63:0/1969008636 wait complete. 2024-05-27T07:38:51.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.963+0000 7f60f5000640 1 -- 172.21.15.63:0/2931509358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f010da00 msgr2=0x7f60f010fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.963+0000 7f60f5000640 1 --2- 172.21.15.63:0/2931509358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f010da00 0x7f60f010fe20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009900 tx=0x7f60dc02a410 comp rx=0 tx=0).stop 2024-05-27T07:38:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.963+0000 7f60f5000640 1 -- 172.21.15.63:0/2931509358 shutdown_connections 2024-05-27T07:38:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.963+0000 7f60f5000640 1 --2- 172.21.15.63:0/2931509358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0110360 0x7f60f0112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.963+0000 7f60f5000640 1 --2- 172.21.15.63:0/2931509358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f010da00 0x7f60f010fe20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.963+0000 7f60f5000640 1 --2- 172.21.15.63:0/2931509358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60f01027c0 0x7f60f0102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.963+0000 7f60f5000640 1 -- 172.21.15.63:0/2931509358 >> 172.21.15.63:0/2931509358 conn(0x7f60f00fe210 msgr2=0x7f60f0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.963+0000 7f60f5000640 1 -- 172.21.15.63:0/2931509358 shutdown_connections 2024-05-27T07:38:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.963+0000 7f60f5000640 1 -- 172.21.15.63:0/2931509358 wait complete. 2024-05-27T07:38:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.969+0000 7f60f5000640 1 Processor -- start 2024-05-27T07:38:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.969+0000 7f60f5000640 1 -- start start 2024-05-27T07:38:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60f5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f01027c0 0x7f60f019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60f5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f010da00 0x7f60f019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60f5000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60f0110360 0x7f60f01a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60f5000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0115d20 con 0x7f60f01027c0 2024-05-27T07:38:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60f5000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0115ba0 con 0x7f60f010da00 2024-05-27T07:38:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60f5000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0115ea0 con 0x7f60f0110360 2024-05-27T07:38:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60ef400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f01027c0 0x7f60f019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60ef400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f01027c0 0x7f60f019c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50102/0 (socket says 172.21.15.63:50102) 2024-05-27T07:38:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60ef400640 1 -- 172.21.15.63:0/839388009 learned_addr learned my addr 172.21.15.63:0/839388009 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60efe00640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60f0110360 0x7f60f01a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60ef400640 1 -- 172.21.15.63:0/839388009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60f0110360 msgr2=0x7f60f01a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60ef400640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60f0110360 0x7f60f01a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60ef400640 1 -- 172.21.15.63:0/839388009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f010da00 msgr2=0x7f60f019cce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:38:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60ef400640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f010da00 0x7f60f019cce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60ef400640 1 -- 172.21.15.63:0/839388009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f01a15e0 con 0x7f60f01027c0 2024-05-27T07:38:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60efe00640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60f0110360 0x7f60f01a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:38:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.970+0000 7f60ef400640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f01027c0 0x7f60f019c780 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f60e000d950 tx=0x7f60e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.977+0000 7f60e7e00640 1 -- 172.21.15.63:0/839388009 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0017070 con 0x7f60f01027c0 2024-05-27T07:38:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.977+0000 7f60e7e00640 1 -- 172.21.15.63:0/839388009 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60e0009c30 con 0x7f60f01027c0 2024-05-27T07:38:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.977+0000 7f60f5000640 1 -- 172.21.15.63:0/839388009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f01a18d0 con 0x7f60f01027c0 2024-05-27T07:38:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.977+0000 7f60f5000640 1 -- 172.21.15.63:0/839388009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f01a9250 con 0x7f60f01027c0 2024-05-27T07:38:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.977+0000 7f60e7e00640 1 -- 172.21.15.63:0/839388009 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e000f620 con 0x7f60f01027c0 2024-05-27T07:38:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.977+0000 7f60fd5af740 1 Processor -- start 2024-05-27T07:38:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.983+0000 7f60fd5af740 1 -- start start 2024-05-27T07:38:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.983+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b48105ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.983+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 0x55b48105efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.983+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 0x55b48105f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.983+0000 7f60fd5af740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b480fcfb90 con 0x55b480c932f0 2024-05-27T07:38:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.983+0000 7f60fd5af740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b480fbf730 con 0x55b480c9b4d0 2024-05-27T07:38:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.983+0000 7f60fd5af740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b480fc3770 con 0x55b480cbc1b0 2024-05-27T07:38:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.983+0000 7f60efe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b48105ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.983+0000 7f60efe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b48105ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:55880/0 (socket says 172.21.15.63:55880) 2024-05-27T07:38:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.983+0000 7f60efe00640 1 -- 172.21.15.63:0/218957092 learned_addr learned my addr 172.21.15.63:0/218957092 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.984+0000 7f60eea00640 1 --2- 172.21.15.63:0/218957092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 0x55b48105f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.985+0000 7f60efe00640 1 -- 172.21.15.63:0/218957092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 msgr2=0x55b48105f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.985+0000 7f60efe00640 1 --2- 172.21.15.63:0/218957092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 0x55b48105f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.985+0000 7f60efe00640 1 -- 172.21.15.63:0/218957092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 msgr2=0x55b48105efa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:38:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.985+0000 7f60efe00640 1 --2- 172.21.15.63:0/218957092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 0x55b48105efa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.985+0000 7f60efe00640 1 -- 172.21.15.63:0/218957092 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b480f588b0 con 0x55b480cbc1b0 2024-05-27T07:38:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.985+0000 7f60e7e00640 1 -- 172.21.15.63:0/839388009 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f60e000f7c0 con 0x7f60f01027c0 2024-05-27T07:38:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.985+0000 7f60efe00640 1 --2- 172.21.15.63:0/218957092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b48105ea60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009e40 tx=0x7f60e800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bc4c7fcb939aadd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.987+0000 7f60e7e00640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60d40b29d0 0x7f60d40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.987+0000 7f60e7e00640 1 -- 172.21.15.63:0/839388009 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f60e00e2570 con 0x7f60f01027c0 2024-05-27T07:38:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.987+0000 7f60ef400640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60d40b29d0 0x7f60d40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.988+0000 7f60ef400640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60d40b29d0 0x7f60d40b4ee0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b48105c550 tx=0x7f60e000bf60 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.991+0000 7f60e4c00640 1 -- 172.21.15.63:0/218957092 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8004d20 con 0x55b480cbc1b0 2024-05-27T07:38:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.991+0000 7f60e4c00640 1 -- 172.21.15.63:0/218957092 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60e80054b0 con 0x55b480cbc1b0 2024-05-27T07:38:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.991+0000 7f60e4c00640 1 -- 172.21.15.63:0/218957092 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8011700 con 0x55b480cbc1b0 2024-05-27T07:38:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.992+0000 7f60fd5af740 1 -- 172.21.15.63:0/218957092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 msgr2=0x55b48105ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.992+0000 7f60fd5af740 1 --2- 172.21.15.63:0/218957092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b48105ea60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009e40 tx=0x7f60e800f730 comp rx=0 tx=0).stop 2024-05-27T07:38:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.993+0000 7f60fd5af740 1 -- 172.21.15.63:0/218957092 shutdown_connections 2024-05-27T07:38:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.993+0000 7f60fd5af740 1 --2- 172.21.15.63:0/218957092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 0x55b48105f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.993+0000 7f60fd5af740 1 --2- 172.21.15.63:0/218957092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 0x55b48105efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.993+0000 7f60fd5af740 1 --2- 172.21.15.63:0/218957092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b48105ea60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.993+0000 7f60fd5af740 1 -- 172.21.15.63:0/218957092 >> 172.21.15.63:0/218957092 conn(0x55b480cb0300 msgr2=0x55b48105b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.993+0000 7f60fd5af740 1 -- 172.21.15.63:0/218957092 shutdown_connections 2024-05-27T07:38:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.993+0000 7f60fd5af740 1 -- 172.21.15.63:0/218957092 wait complete. 2024-05-27T07:38:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.993+0000 7f60fd5af740 1 -- start start 2024-05-27T07:38:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.994+0000 7f60fd5af740 1 Processor -- start 2024-05-27T07:38:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.994+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 0x55b481068b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b4810690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b481074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60fd5af740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b480ca5790 con 0x55b480c9b4d0 2024-05-27T07:38:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60fd5af740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b480f67bc0 con 0x55b480c932f0 2024-05-27T07:38:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60fd5af740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b48106bb90 con 0x55b480cbc1b0 2024-05-27T07:38:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60fd5af740 10 client.0 _ll_register_callbacks cb 0x55b480f45400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 0x55b481068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60eea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b4810690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 0x55b481068b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44744/0 (socket says 172.21.15.63:44744) 2024-05-27T07:38:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60eea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b4810690d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50106/0 (socket says 172.21.15.63:50106) 2024-05-27T07:38:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60efe00640 1 -- 172.21.15.63:0/2726275424 learned_addr learned my addr 172.21.15.63:0/2726275424 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.995+0000 7f60efe00640 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b481074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.996+0000 7f60eea00640 1 -- 172.21.15.63:0/2726275424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 msgr2=0x55b481074ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.996+0000 7f60eea00640 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b481074ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.996+0000 7f60eea00640 1 -- 172.21.15.63:0/2726275424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 msgr2=0x55b481068b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.996+0000 7f60eea00640 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 0x55b481068b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.996+0000 7f60eea00640 1 -- 172.21.15.63:0/2726275424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b480cd4210 con 0x55b480c9b4d0 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.996+0000 7f60eea00640 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b4810690d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009230 tx=0x7f60dc031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed97e311671d5d68 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.996+0000 7f60d9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.997+0000 7f60d9600640 1 -- 172.21.15.63:0/2726275424 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc040070 con 0x55b480c9b4d0 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.997+0000 7f60d9600640 1 -- 172.21.15.63:0/2726275424 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f60dc031810 con 0x55b480c9b4d0 2024-05-27T07:38:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.997+0000 7f60fd5af740 1 -- 172.21.15.63:0/2726275424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60e80054b0 con 0x55b480c9b4d0 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.997+0000 7f60d9600640 1 -- 172.21.15.63:0/2726275424 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f60dc035040 con 0x55b480c9b4d0 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.997+0000 7f60fd5af740 10 client.16446 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.997+0000 7f60fd5af740 1 -- 172.21.15.63:0/2726275424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b480da0620 con 0x55b480c9b4d0 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.998+0000 7f60d9600640 1 -- 172.21.15.63:0/2726275424 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60e80054b0 con 0x55b480c9b4d0 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.998+0000 7f60d9600640 1 -- 172.21.15.63:0/2726275424 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f60dc0046c0 con 0x55b480c9b4d0 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.998+0000 7f60fd5af740 10 client.16446 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.998+0000 7f60fd5af740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.998+0000 7f60fd5af740 20 client.16446 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.998+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b480c7ad80 0x55b48107ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.998+0000 7f60fd5af740 4 client.16446 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.998+0000 7f60fd5af740 1 -- 172.21.15.63:0/2726275424 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b480fd1ae0 con 0x55b480c7ad80 2024-05-27T07:38:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:51.999+0000 7f60efe00640 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b480c7ad80 0x55b48107ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.000+0000 7f60efe00640 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b480c7ad80 0x55b48107ae60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f91fd54b4d7d6bc server_cookie=70b03bf1c19fe5ca in_seq=0 out_seq=0 2024-05-27T07:38:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.000+0000 7f60d9600640 10 client.16446 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60d9600640 1 -- 172.21.15.63:0/2726275424 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b480fd1ae0 con 0x55b480c7ad80 2024-05-27T07:38:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60d9600640 10 client.16446 handle_command_reply: tid=0 2024-05-27T07:38:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60fd5af740 1 client.16446 shutdown 2024-05-27T07:38:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60fd5af740 1 -- 172.21.15.63:0/2726275424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 msgr2=0x55b4810690d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b4810690d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009230 tx=0x7f60dc031ca0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60fd5af740 1 -- 172.21.15.63:0/2726275424 shutdown_connections 2024-05-27T07:38:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b480c7ad80 0x55b48107ae60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b481074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b4810690d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2726275424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 0x55b481068b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60fd5af740 1 -- 172.21.15.63:0/2726275424 >> 172.21.15.63:0/2726275424 conn(0x55b480cb0300 msgr2=0x55b48105e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60d9600640 0 client.16446 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.002+0000 7f60fd5af740 1 -- 172.21.15.63:0/2726275424 shutdown_connections 2024-05-27T07:38:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.003+0000 7f60fd5af740 1 -- 172.21.15.63:0/2726275424 wait complete. 2024-05-27T07:38:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.003+0000 7f60fd5af740 20 client.16446 trim_cache size 0 max 16384 2024-05-27T07:38:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.027+0000 7f60fd5af740 1 Processor -- start 2024-05-27T07:38:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.027+0000 7f60fd5af740 1 -- start start 2024-05-27T07:38:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.027+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 0x55b4810643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.027+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b48106a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.027+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b481074c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.027+0000 7f60fd5af740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60d00126b0 con 0x55b480c9b4d0 2024-05-27T07:38:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.027+0000 7f60fd5af740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60d000ff80 con 0x55b480c932f0 2024-05-27T07:38:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.027+0000 7f60fd5af740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60d000ddd0 con 0x55b480cbc1b0 2024-05-27T07:38:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.028+0000 7f60eea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b48106a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.028+0000 7f60eea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b48106a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50120/0 (socket says 172.21.15.63:50120) 2024-05-27T07:38:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.028+0000 7f60eea00640 1 -- 172.21.15.63:0/1310585424 learned_addr learned my addr 172.21.15.63:0/1310585424 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.028+0000 7f60efe00640 1 --2- 172.21.15.63:0/1310585424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 0x55b4810643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.028+0000 7f60efe00640 1 --2- 172.21.15.63:0/1310585424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b481074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.029+0000 7f60eea00640 1 -- 172.21.15.63:0/1310585424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 msgr2=0x55b481074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.029+0000 7f60eea00640 1 --2- 172.21.15.63:0/1310585424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b481074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.029+0000 7f60eea00640 1 -- 172.21.15.63:0/1310585424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 msgr2=0x55b4810643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.029+0000 7f60eea00640 1 --2- 172.21.15.63:0/1310585424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 0x55b4810643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.029+0000 7f60eea00640 1 -- 172.21.15.63:0/1310585424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b481059610 con 0x55b480c9b4d0 2024-05-27T07:38:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.029+0000 7f60efe00640 1 --2- 172.21.15.63:0/1310585424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b481074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.029+0000 7f60eea00640 1 --2- 172.21.15.63:0/1310585424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b48106a9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f60dc004b10 tx=0x7f60dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2534b6dbbacfd769 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.029+0000 7f60dbe00640 1 -- 172.21.15.63:0/1310585424 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc040040 con 0x55b480c9b4d0 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.029+0000 7f60dbe00640 1 -- 172.21.15.63:0/1310585424 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60dc005d50 con 0x55b480c9b4d0 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.029+0000 7f60dbe00640 1 -- 172.21.15.63:0/1310585424 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc031610 con 0x55b480c9b4d0 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.030+0000 7f60fd5af740 1 -- 172.21.15.63:0/1310585424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 msgr2=0x55b48106a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.030+0000 7f60fd5af740 1 --2- 172.21.15.63:0/1310585424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b48106a9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f60dc004b10 tx=0x7f60dc000bc0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.030+0000 7f60fd5af740 1 -- 172.21.15.63:0/1310585424 shutdown_connections 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.030+0000 7f60fd5af740 1 --2- 172.21.15.63:0/1310585424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b481074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.030+0000 7f60fd5af740 1 --2- 172.21.15.63:0/1310585424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c9b4d0 0x55b48106a9c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.030+0000 7f60fd5af740 1 --2- 172.21.15.63:0/1310585424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c932f0 0x55b4810643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.030+0000 7f60fd5af740 1 -- 172.21.15.63:0/1310585424 >> 172.21.15.63:0/1310585424 conn(0x55b480cb0300 msgr2=0x55b48106a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.031+0000 7f60fd5af740 1 -- 172.21.15.63:0/1310585424 shutdown_connections 2024-05-27T07:38:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.031+0000 7f60fd5af740 1 -- 172.21.15.63:0/1310585424 wait complete. 2024-05-27T07:38:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.031+0000 7f60fd5af740 1 -- start start 2024-05-27T07:38:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.031+0000 7f60fd5af740 1 Processor -- start 2024-05-27T07:38:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.032+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 0x55b48106c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.032+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 0x55b48106cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.032+0000 7f60fd5af740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b4810db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.032+0000 7f60fd5af740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60d000e440 con 0x55b480c932f0 2024-05-27T07:38:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.032+0000 7f60fd5af740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60d000f8b0 con 0x55b480c9b4d0 2024-05-27T07:38:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.032+0000 7f60fd5af740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60d000eb10 con 0x55b480cbc1b0 2024-05-27T07:38:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.032+0000 7f60fd5af740 10 client.0 _ll_register_callbacks cb 0x7f60dc009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.032+0000 7f60efe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 0x55b48106c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.032+0000 7f60efe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 0x55b48106c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50126/0 (socket says 172.21.15.63:50126) 2024-05-27T07:38:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.032+0000 7f60efe00640 1 -- 172.21.15.63:0/2644742948 learned_addr learned my addr 172.21.15.63:0/2644742948 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60eea00640 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 0x55b48106cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60efe00640 1 -- 172.21.15.63:0/2644742948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 msgr2=0x55b4810db4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60efe00640 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b4810db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60efe00640 1 -- 172.21.15.63:0/2644742948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 msgr2=0x55b48106cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60efe00640 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 0x55b48106cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60efe00640 1 -- 172.21.15.63:0/2644742948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4810dbab0 con 0x55b480c932f0 2024-05-27T07:38:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60eea00640 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 0x55b48106cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:38:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60efe00640 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 0x55b48106c640 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f60e8000f80 tx=0x7f60e8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc750652fc34824 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60d9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60d9600640 1 -- 172.21.15.63:0/2644742948 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e802f070 con 0x55b480c932f0 2024-05-27T07:38:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60d9600640 1 -- 172.21.15.63:0/2644742948 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f60e8004c80 con 0x55b480c932f0 2024-05-27T07:38:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.033+0000 7f60fd5af740 1 -- 172.21.15.63:0/2644742948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60dc005d50 con 0x55b480c932f0 2024-05-27T07:38:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60d9600640 1 -- 172.21.15.63:0/2644742948 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f60e8022040 con 0x55b480c932f0 2024-05-27T07:38:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60fd5af740 10 client.16452 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60fd5af740 1 -- 172.21.15.63:0/2644742948 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b4810e2d80 con 0x55b480c932f0 2024-05-27T07:38:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60d9600640 1 -- 172.21.15.63:0/2644742948 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60dc005d50 con 0x55b480c932f0 2024-05-27T07:38:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60d9600640 1 -- 172.21.15.63:0/2644742948 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f60e8028040 con 0x55b480c932f0 2024-05-27T07:38:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60fd5af740 10 client.16452 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60fd5af740 10 client.16452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60fd5af740 20 client.16452 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b480c501c0 0x55b4810e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60fd5af740 4 client.16452 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:38:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.034+0000 7f60fd5af740 1 -- 172.21.15.63:0/2644742948 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b480fcb2c0 con 0x55b480c501c0 2024-05-27T07:38:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.035+0000 7f60eea00640 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b480c501c0 0x55b4810e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.036+0000 7f60eea00640 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b480c501c0 0x55b4810e53b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84793a3e540639f server_cookie=8207dcbb1c3bcffa in_seq=0 out_seq=0 2024-05-27T07:38:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.036+0000 7f60d9600640 10 client.16452 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60d9600640 1 -- 172.21.15.63:0/2644742948 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b480fcb2c0 con 0x55b480c501c0 2024-05-27T07:38:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60d9600640 10 client.16452 handle_command_reply: tid=0 2024-05-27T07:38:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60fd5af740 1 client.16452 shutdown 2024-05-27T07:38:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60fd5af740 1 -- 172.21.15.63:0/2644742948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 msgr2=0x55b48106c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 0x55b48106c640 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f60e8000f80 tx=0x7f60e8009920 comp rx=0 tx=0).stop 2024-05-27T07:38:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60fd5af740 1 -- 172.21.15.63:0/2644742948 shutdown_connections 2024-05-27T07:38:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b480c501c0 0x55b4810e53b0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b480cbc1b0 0x55b4810db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b480c9b4d0 0x55b48106cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60fd5af740 1 --2- 172.21.15.63:0/2644742948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b480c932f0 0x55b48106c640 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60fd5af740 1 -- 172.21.15.63:0/2644742948 >> 172.21.15.63:0/2644742948 conn(0x55b480cb0300 msgr2=0x55b48106a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.084+0000 7f60d9600640 0 client.16452 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.085+0000 7f60fd5af740 1 -- 172.21.15.63:0/2644742948 shutdown_connections 2024-05-27T07:38:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.085+0000 7f60fd5af740 1 -- 172.21.15.63:0/2644742948 wait complete. 2024-05-27T07:38:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.085+0000 7f60fd5af740 20 client.16452 trim_cache size 0 max 16384 2024-05-27T07:38:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:38:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:38:52.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:38:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.086+0000 7f60f5000640 1 -- 172.21.15.63:0/839388009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60d40b29d0 msgr2=0x7f60d40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.086+0000 7f60f5000640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60d40b29d0 0x7f60d40b4ee0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b48105c550 tx=0x7f60e000bf60 comp rx=0 tx=0).stop 2024-05-27T07:38:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.086+0000 7f60f5000640 1 -- 172.21.15.63:0/839388009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f01027c0 msgr2=0x7f60f019c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.086+0000 7f60f5000640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f01027c0 0x7f60f019c780 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f60e000d950 tx=0x7f60e000de10 comp rx=0 tx=0).stop 2024-05-27T07:38:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.087+0000 7f60f5000640 1 -- 172.21.15.63:0/839388009 shutdown_connections 2024-05-27T07:38:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.087+0000 7f60f5000640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60d40b29d0 0x7f60d40b4ee0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.087+0000 7f60f5000640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60f0110360 0x7f60f01a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.087+0000 7f60f5000640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f010da00 0x7f60f019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.087+0000 7f60f5000640 1 --2- 172.21.15.63:0/839388009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60f01027c0 0x7f60f019c780 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.087+0000 7f60f5000640 1 -- 172.21.15.63:0/839388009 >> 172.21.15.63:0/839388009 conn(0x7f60f00fe210 msgr2=0x7f60f0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.087+0000 7f60f5000640 1 -- 172.21.15.63:0/839388009 shutdown_connections 2024-05-27T07:38:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:38:52.087+0000 7f60f5000640 1 -- 172.21.15.63:0/839388009 wait complete. 2024-05-27T07:38:52.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:51 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:38:52.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:51 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:38:52.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:51 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:38:52.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:51 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:38:52.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:51 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:38:52.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:51 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.455 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:38:52.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:51 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:38:52.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:51 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:38:52.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:51 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:38:52.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:51 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:38:52.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:51 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:38:52.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:51 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:51 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:38:52.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:51 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:38:52.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:51 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:38:52.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:51 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:38:52.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:51 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:38:52.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:51 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:52.949 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:38:53.271 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:52 smithi063 ceph-mon[31613]: pgmap v162: 129 pgs: 129 active+clean; 794 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 14 op/s 2024-05-27T07:38:53.271 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:52 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:38:53.272 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:52 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:53.272 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:52 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:38:53.272 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:52 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:38:53.272 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:52 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:53.272 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:52 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:53.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:52 smithi086 ceph-mon[34687]: pgmap v162: 129 pgs: 129 active+clean; 794 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 14 op/s 2024-05-27T07:38:53.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:52 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:38:53.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:52 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:53.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:52 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:38:53.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:52 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:38:53.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:52 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:53.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:52 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:53.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:52 smithi149 ceph-mon[35408]: pgmap v162: 129 pgs: 129 active+clean; 794 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 14 op/s 2024-05-27T07:38:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:52 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:38:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:52 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:52 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:38:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:52 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:38:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:52 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:53.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:52 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:38:55.125 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:54 smithi063 ceph-mon[31613]: pgmap v163: 129 pgs: 129 active+clean; 794 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:38:55.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.394+0000 7fb4cc400700 1 -- 172.21.15.63:0/1782534908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c41054f0 msgr2=0x7fb4c4105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:55.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.394+0000 7fb4cc400700 1 --2- 172.21.15.63:0/1782534908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c41054f0 0x7fb4c4105900 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009990 tx=0x7fb4b8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.395+0000 7fb4cc400700 1 -- 172.21.15.63:0/1782534908 shutdown_connections 2024-05-27T07:38:55.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.395+0000 7fb4cc400700 1 --2- 172.21.15.63:0/1782534908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c410e8d0 0x7fb4c4110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.395+0000 7fb4cc400700 1 --2- 172.21.15.63:0/1782534908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c41054f0 0x7fb4c4105900 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.395+0000 7fb4cc400700 1 --2- 172.21.15.63:0/1782534908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4105e40 0x7fb4c410e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.395+0000 7fb4cc400700 1 -- 172.21.15.63:0/1782534908 >> 172.21.15.63:0/1782534908 conn(0x7fb4c40fedb0 msgr2=0x7fb4c41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:55.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.395+0000 7fb4cc400700 1 -- 172.21.15.63:0/1782534908 shutdown_connections 2024-05-27T07:38:55.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.395+0000 7fb4cc400700 1 -- 172.21.15.63:0/1782534908 wait complete. 2024-05-27T07:38:55.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.397+0000 7fb4cc400700 1 Processor -- start 2024-05-27T07:38:55.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.397+0000 7fb4cc400700 1 -- start start 2024-05-27T07:38:55.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.397+0000 7fb4cc400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41054f0 0x7fb4c41031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.397+0000 7fb4cc400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4105e40 0x7fb4c4103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.397+0000 7fb4cc400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c410e8d0 0x7fb4c4103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.397+0000 7fb4cc400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41142d0 con 0x7fb4c410e8d0 2024-05-27T07:38:55.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.397+0000 7fb4cc400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4114150 con 0x7fb4c41054f0 2024-05-27T07:38:55.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.397+0000 7fb4cc400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4114450 con 0x7fb4c4105e40 2024-05-27T07:38:55.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.398+0000 7fb4ca400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c410e8d0 0x7fb4c4103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.398+0000 7fb4ca400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c410e8d0 0x7fb4c4103c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50154/0 (socket says 172.21.15.63:50154) 2024-05-27T07:38:55.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.398+0000 7fb4ca400700 1 -- 172.21.15.63:0/1147629573 learned_addr learned my addr 172.21.15.63:0/1147629573 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:55.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.398+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41054f0 0x7fb4c41031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.398+0000 7fb4c9000700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4105e40 0x7fb4c4103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.398+0000 7fb4ca400700 1 -- 172.21.15.63:0/1147629573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4105e40 msgr2=0x7fb4c4103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:55.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.399+0000 7fb4ca400700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4105e40 0x7fb4c4103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.399+0000 7fb4ca400700 1 -- 172.21.15.63:0/1147629573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41054f0 msgr2=0x7fb4c41031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:55.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.399+0000 7fb4ca400700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41054f0 0x7fb4c41031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.399+0000 7fb4ca400700 1 -- 172.21.15.63:0/1147629573 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c41a90a0 con 0x7fb4c410e8d0 2024-05-27T07:38:55.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.399+0000 7fb4ca400700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c410e8d0 0x7fb4c4103c60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc007ee0 tx=0x7fb4bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:55.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.399+0000 7fb4c2000700 1 -- 172.21.15.63:0/1147629573 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc004bb0 con 0x7fb4c410e8d0 2024-05-27T07:38:55.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.399+0000 7fb4c2000700 1 -- 172.21.15.63:0/1147629573 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4bc004d50 con 0x7fb4c410e8d0 2024-05-27T07:38:55.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.399+0000 7fb4cc400700 1 -- 172.21.15.63:0/1147629573 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c41a9390 con 0x7fb4c410e8d0 2024-05-27T07:38:55.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.399+0000 7fb4c2000700 1 -- 172.21.15.63:0/1147629573 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0058b0 con 0x7fb4c410e8d0 2024-05-27T07:38:55.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.400+0000 7fb4cc400700 1 -- 172.21.15.63:0/1147629573 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c41a9880 con 0x7fb4c410e8d0 2024-05-27T07:38:55.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.404+0000 7fb4c2000700 1 -- 172.21.15.63:0/1147629573 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb4bc01a020 con 0x7fb4c410e8d0 2024-05-27T07:38:55.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.407+0000 7fb4c2000700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4ac0b29f0 0x7fb4ac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.407+0000 7fb4daec2b80 1 Processor -- start 2024-05-27T07:38:55.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.407+0000 7fb4daec2b80 1 -- start start 2024-05-27T07:38:55.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.407+0000 7fb4c2000700 1 -- 172.21.15.63:0/1147629573 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fb4bc0e1fe0 con 0x7fb4c410e8d0 2024-05-27T07:38:55.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.407+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4ac0b29f0 0x7fb4ac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d7c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d76870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 0x5579a1d771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5579a1ca4e40 con 0x5579a1d76db0 2024-05-27T07:38:55.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579a1c949e0 con 0x5579a1bf7e40 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5579a1c98a20 con 0x5579a1d7cd30 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4ac0b29f0 0x7fb4ac0b4f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80098e0 tx=0x7fb4b8005960 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4c9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d7c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4c9000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d76870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4c9a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d7c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44778/0 (socket says 172.21.15.63:44778) 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.408+0000 7fb4c9a00700 1 -- 172.21.15.63:0/2144943997 learned_addr learned my addr 172.21.15.63:0/2144943997 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.409+0000 7fb4ca400700 1 --2- 172.21.15.63:0/2144943997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 0x5579a1d771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.409+0000 7fb4c9a00700 1 -- 172.21.15.63:0/2144943997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 msgr2=0x5579a1d76870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.409+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/2144943997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d76870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.409+0000 7fb4c9a00700 1 -- 172.21.15.63:0/2144943997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 msgr2=0x5579a1d771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:55.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.409+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/2144943997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 0x5579a1d771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.409+0000 7fb4c9a00700 1 -- 172.21.15.63:0/2144943997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a1c5d420 con 0x5579a1bf7e40 2024-05-27T07:38:55.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.410+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/2144943997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d7c7f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009990 tx=0x7fb4b801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17bd4b4a3a3f3d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:55.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.410+0000 7fb4b2a00700 1 -- 172.21.15.63:0/2144943997 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8003ba0 con 0x5579a1bf7e40 2024-05-27T07:38:55.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.410+0000 7fb4b2a00700 1 -- 172.21.15.63:0/2144943997 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4b8003d40 con 0x5579a1bf7e40 2024-05-27T07:38:55.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.410+0000 7fb4b2a00700 1 -- 172.21.15.63:0/2144943997 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800f920 con 0x5579a1bf7e40 2024-05-27T07:38:55.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.411+0000 7fb4daec2b80 1 -- 172.21.15.63:0/2144943997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 msgr2=0x5579a1d7c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:55.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.411+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/2144943997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d7c7f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009990 tx=0x7fb4b801e000 comp rx=0 tx=0).stop 2024-05-27T07:38:55.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.411+0000 7fb4daec2b80 1 -- 172.21.15.63:0/2144943997 shutdown_connections 2024-05-27T07:38:55.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.411+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/2144943997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 0x5579a1d771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.411+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/2144943997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d7c7f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.411+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/2144943997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d76870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.411+0000 7fb4daec2b80 1 -- 172.21.15.63:0/2144943997 >> 172.21.15.63:0/2144943997 conn(0x5579a1c57fc0 msgr2=0x5579a1d718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:55.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.412+0000 7fb4daec2b80 1 -- 172.21.15.63:0/2144943997 shutdown_connections 2024-05-27T07:38:55.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.412+0000 7fb4daec2b80 1 -- 172.21.15.63:0/2144943997 wait complete. 2024-05-27T07:38:55.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.412+0000 7fb4daec2b80 1 -- start start 2024-05-27T07:38:55.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.413+0000 7fb4daec2b80 1 Processor -- start 2024-05-27T07:38:55.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.413+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1bf7e40 0x5579a1d804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.413+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1d76db0 0x5579a1d80a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.413+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d80f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.413+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5579a1b9afa0 con 0x5579a1bf7e40 2024-05-27T07:38:55.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.413+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579a1c35760 con 0x5579a1d76db0 2024-05-27T07:38:55.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.413+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5579a1d837a0 con 0x5579a1d7cd30 2024-05-27T07:38:55.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.413+0000 7fb4daec2b80 10 client.0 _ll_register_callbacks cb 0x5579a1c3ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:55.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4c9a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1bf7e40 0x5579a1d804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4c9a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1bf7e40 0x5579a1d804d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50166/0 (socket says 172.21.15.63:50166) 2024-05-27T07:38:55.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4c9a00700 1 -- 172.21.15.63:0/3552475110 learned_addr learned my addr 172.21.15.63:0/3552475110 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:55.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4c9000700 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1d76db0 0x5579a1d80a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4ca400700 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d80f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4c9a00700 1 -- 172.21.15.63:0/3552475110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 msgr2=0x5579a1d80f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:55.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d80f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4c9a00700 1 -- 172.21.15.63:0/3552475110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1d76db0 msgr2=0x5579a1d80a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:55.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1d76db0 0x5579a1d80a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4c9a00700 1 -- 172.21.15.63:0/3552475110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a1be48d0 con 0x5579a1bf7e40 2024-05-27T07:38:55.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.414+0000 7fb4ca400700 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d80f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:38:55.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.415+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1bf7e40 0x5579a1d804d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80059d0 tx=0x7fb4b8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af55e819a4a975b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:55.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.415+0000 7fb4a7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.415+0000 7fb4a7400700 1 -- 172.21.15.63:0/3552475110 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8036070 con 0x5579a1bf7e40 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.415+0000 7fb4a7400700 1 -- 172.21.15.63:0/3552475110 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fb4b800f7b0 con 0x5579a1bf7e40 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.415+0000 7fb4daec2b80 1 -- 172.21.15.63:0/3552475110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b8003d40 con 0x5579a1bf7e40 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.416+0000 7fb4a7400700 1 -- 172.21.15.63:0/3552475110 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb4b8040050 con 0x5579a1bf7e40 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.416+0000 7fb4daec2b80 10 client.16464 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.416+0000 7fb4daec2b80 1 -- 172.21.15.63:0/3552475110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5579a1c65060 con 0x5579a1bf7e40 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.416+0000 7fb4a7400700 1 -- 172.21.15.63:0/3552475110 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4b8003d40 con 0x5579a1bf7e40 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.416+0000 7fb4a7400700 1 -- 172.21.15.63:0/3552475110 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb4b8022070 con 0x5579a1bf7e40 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.416+0000 7fb4daec2b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.416+0000 7fb4daec2b80 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.416+0000 7fb4daec2b80 20 client.16464 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.416+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579a1d904f0 0x5579a1d92910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.417+0000 7fb4daec2b80 4 client.16464 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:38:55.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.417+0000 7fb4daec2b80 1 -- 172.21.15.63:0/3552475110 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579a1ca6d90 con 0x5579a1d904f0 2024-05-27T07:38:55.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.417+0000 7fb4c9000700 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579a1d904f0 0x5579a1d92910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.418+0000 7fb4c9000700 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579a1d904f0 0x5579a1d92910 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8684821cb115c3d server_cookie=df6817d7d84de216 in_seq=0 out_seq=0 2024-05-27T07:38:55.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.418+0000 7fb4a7400700 10 client.16464 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:55.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.420+0000 7fb4a7400700 1 -- 172.21.15.63:0/3552475110 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5579a1ca6d90 con 0x5579a1d904f0 2024-05-27T07:38:55.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.420+0000 7fb4a7400700 10 client.16464 handle_command_reply: tid=0 2024-05-27T07:38:55.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.420+0000 7fb4daec2b80 1 client.16464 shutdown 2024-05-27T07:38:55.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.420+0000 7fb4daec2b80 1 -- 172.21.15.63:0/3552475110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1bf7e40 msgr2=0x5579a1d804d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:55.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.420+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1bf7e40 0x5579a1d804d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80059d0 tx=0x7fb4b8005760 comp rx=0 tx=0).stop 2024-05-27T07:38:55.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.421+0000 7fb4daec2b80 1 -- 172.21.15.63:0/3552475110 shutdown_connections 2024-05-27T07:38:55.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.421+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579a1d904f0 0x5579a1d92910 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.421+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d80f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.421+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1bf7e40 0x5579a1d804d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.421+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/3552475110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1d76db0 0x5579a1d80a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.421+0000 7fb4a7400700 0 client.16464 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:55.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.421+0000 7fb4daec2b80 1 -- 172.21.15.63:0/3552475110 >> 172.21.15.63:0/3552475110 conn(0x5579a1c57fc0 msgr2=0x5579a1d71580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:55.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.421+0000 7fb4daec2b80 1 -- 172.21.15.63:0/3552475110 shutdown_connections 2024-05-27T07:38:55.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.421+0000 7fb4daec2b80 1 -- 172.21.15.63:0/3552475110 wait complete. 2024-05-27T07:38:55.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.421+0000 7fb4daec2b80 20 client.16464 trim_cache size 0 max 16384 2024-05-27T07:38:55.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.450+0000 7fb4daec2b80 1 Processor -- start 2024-05-27T07:38:55.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.450+0000 7fb4daec2b80 1 -- start start 2024-05-27T07:38:55.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.450+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d76870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.450+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 0x5579a1d7ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.450+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d8ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.450+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b4da0 con 0x5579a1d76db0 2024-05-27T07:38:55.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.450+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b2630 con 0x5579a1bf7e40 2024-05-27T07:38:55.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.450+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b0480 con 0x5579a1d7cd30 2024-05-27T07:38:55.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.451+0000 7fb4c9000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 0x5579a1d7ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.451+0000 7fb4c9000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 0x5579a1d7ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50178/0 (socket says 172.21.15.63:50178) 2024-05-27T07:38:55.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.451+0000 7fb4c9000700 1 -- 172.21.15.63:0/1652977352 learned_addr learned my addr 172.21.15.63:0/1652977352 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:55.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.451+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/1652977352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d76870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.451+0000 7fb4ca400700 1 --2- 172.21.15.63:0/1652977352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d8ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.451+0000 7fb4c9000700 1 -- 172.21.15.63:0/1652977352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 msgr2=0x5579a1d8ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:55.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.451+0000 7fb4c9000700 1 --2- 172.21.15.63:0/1652977352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d8ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.451+0000 7fb4c9000700 1 -- 172.21.15.63:0/1652977352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 msgr2=0x5579a1d76870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:55.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.451+0000 7fb4c9000700 1 --2- 172.21.15.63:0/1652977352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d76870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.451+0000 7fb4c9000700 1 -- 172.21.15.63:0/1652977352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a1d7c370 con 0x5579a1d76db0 2024-05-27T07:38:55.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.452+0000 7fb4c9000700 1 --2- 172.21.15.63:0/1652977352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 0x5579a1d7ac40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4102470 tx=0x7fb4b400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b1b2494b99dc05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:55.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.452+0000 7fb4b2000700 1 -- 172.21.15.63:0/1652977352 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4003a60 con 0x5579a1d76db0 2024-05-27T07:38:55.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.452+0000 7fb4b2000700 1 -- 172.21.15.63:0/1652977352 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4b4004830 con 0x5579a1d76db0 2024-05-27T07:38:55.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.452+0000 7fb4b2000700 1 -- 172.21.15.63:0/1652977352 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40054b0 con 0x5579a1d76db0 2024-05-27T07:38:55.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:54 smithi149 ceph-mon[35408]: pgmap v163: 129 pgs: 129 active+clean; 794 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:38:55.463 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:54 smithi086 ceph-mon[34687]: pgmap v163: 129 pgs: 129 active+clean; 794 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:38:55.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.453+0000 7fb4daec2b80 1 -- 172.21.15.63:0/1652977352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 msgr2=0x5579a1d7ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:55.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.453+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/1652977352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 0x5579a1d7ac40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4102470 tx=0x7fb4b400b410 comp rx=0 tx=0).stop 2024-05-27T07:38:55.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.453+0000 7fb4daec2b80 1 -- 172.21.15.63:0/1652977352 shutdown_connections 2024-05-27T07:38:55.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.453+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/1652977352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d7cd30 0x5579a1d8ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.453+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/1652977352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d76870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.453+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/1652977352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d76db0 0x5579a1d7ac40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.453+0000 7fb4daec2b80 1 -- 172.21.15.63:0/1652977352 >> 172.21.15.63:0/1652977352 conn(0x5579a1c57fc0 msgr2=0x5579a1d802b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:55.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.454+0000 7fb4daec2b80 1 -- 172.21.15.63:0/1652977352 shutdown_connections 2024-05-27T07:38:55.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.454+0000 7fb4daec2b80 1 -- 172.21.15.63:0/1652977352 wait complete. 2024-05-27T07:38:55.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.454+0000 7fb4daec2b80 1 -- start start 2024-05-27T07:38:55.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.454+0000 7fb4daec2b80 1 Processor -- start 2024-05-27T07:38:55.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.455+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d94110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.455+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d76db0 0x5579a1d94650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.455+0000 7fb4daec2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d7cd30 0x5579a1df3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.455+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b0af0 con 0x5579a1d7cd30 2024-05-27T07:38:55.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.455+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b1f60 con 0x5579a1bf7e40 2024-05-27T07:38:55.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.455+0000 7fb4daec2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b11c0 con 0x5579a1d76db0 2024-05-27T07:38:55.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.455+0000 7fb4daec2b80 10 client.0 _ll_register_callbacks cb 0x5579a1c3ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:38:55.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.455+0000 7fb4ca400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d7cd30 0x5579a1df3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.455+0000 7fb4ca400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d7cd30 0x5579a1df3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50194/0 (socket says 172.21.15.63:50194) 2024-05-27T07:38:55.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.456+0000 7fb4ca400700 1 -- 172.21.15.63:0/274539051 learned_addr learned my addr 172.21.15.63:0/274539051 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:38:55.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.456+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d94110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.456+0000 7fb4c9000700 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d76db0 0x5579a1d94650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.456+0000 7fb4ca400700 1 -- 172.21.15.63:0/274539051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d76db0 msgr2=0x5579a1d94650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:55.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.456+0000 7fb4ca400700 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d76db0 0x5579a1d94650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.456+0000 7fb4ca400700 1 -- 172.21.15.63:0/274539051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 msgr2=0x5579a1d94110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:38:55.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.456+0000 7fb4ca400700 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d94110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.456+0000 7fb4ca400700 1 -- 172.21.15.63:0/274539051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a1df42e0 con 0x5579a1d7cd30 2024-05-27T07:38:55.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.457+0000 7fb4ca400700 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d7cd30 0x5579a1df3da0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0a8ec0 tx=0x7fb4bc0df850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada4d93c8cd3804b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:55.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.457+0000 7fb4a7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:38:55.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.457+0000 7fb4a7400700 1 -- 172.21.15.63:0/274539051 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0aec80 con 0x5579a1d7cd30 2024-05-27T07:38:55.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.457+0000 7fb4a7400700 1 -- 172.21.15.63:0/274539051 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fb4bc0a9190 con 0x5579a1d7cd30 2024-05-27T07:38:55.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.457+0000 7fb4daec2b80 1 -- 172.21.15.63:0/274539051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b4004830 con 0x5579a1d7cd30 2024-05-27T07:38:55.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.457+0000 7fb4a7400700 1 -- 172.21.15.63:0/274539051 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb4bc0a9630 con 0x5579a1d7cd30 2024-05-27T07:38:55.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.457+0000 7fb4daec2b80 10 client.16470 fetch_fsmap learned FSMap version 33 2024-05-27T07:38:55.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.457+0000 7fb4daec2b80 1 -- 172.21.15.63:0/274539051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5579a1dfb600 con 0x5579a1d7cd30 2024-05-27T07:38:55.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.458+0000 7fb4a7400700 1 -- 172.21.15.63:0/274539051 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4b4004830 con 0x5579a1d7cd30 2024-05-27T07:38:55.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.458+0000 7fb4a7400700 1 -- 172.21.15.63:0/274539051 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb4bc0b1900 con 0x5579a1d7cd30 2024-05-27T07:38:55.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.458+0000 7fb4daec2b80 10 client.16470 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:38:55.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.458+0000 7fb4daec2b80 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:38:55.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.458+0000 7fb4daec2b80 20 client.16470 populate_metadata read hostname 'smithi063' 2024-05-27T07:38:55.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.458+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579a1dfbcb0 0x5579a1dfe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:55.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.458+0000 7fb4daec2b80 4 client.16470 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:38:55.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.458+0000 7fb4daec2b80 1 -- 172.21.15.63:0/274539051 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5579a1ca0570 con 0x5579a1dfbcb0 2024-05-27T07:38:55.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.459+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579a1dfbcb0 0x5579a1dfe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:55.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.460+0000 7fb4a7400700 10 client.16470 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:55.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.460+0000 7fb4c9a00700 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579a1dfbcb0 0x5579a1dfe0d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5af7a85d06bfa5ca server_cookie=270e5cbba39b6c0f in_seq=0 out_seq=0 2024-05-27T07:38:55.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.460+0000 7fb4a7400700 1 -- 172.21.15.63:0/274539051 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5579a1ca0570 con 0x5579a1dfbcb0 2024-05-27T07:38:55.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.460+0000 7fb4a7400700 10 client.16470 handle_command_reply: tid=0 2024-05-27T07:38:55.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.460+0000 7fb4daec2b80 1 client.16470 shutdown 2024-05-27T07:38:55.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.460+0000 7fb4daec2b80 1 -- 172.21.15.63:0/274539051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d7cd30 msgr2=0x5579a1df3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:55.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.460+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d7cd30 0x5579a1df3da0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0a8ec0 tx=0x7fb4bc0df850 comp rx=0 tx=0).stop 2024-05-27T07:38:55.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.460+0000 7fb4daec2b80 1 -- 172.21.15.63:0/274539051 shutdown_connections 2024-05-27T07:38:55.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.460+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579a1dfbcb0 0x5579a1dfe0d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.461+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579a1d7cd30 0x5579a1df3da0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.461+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a1bf7e40 0x5579a1d94110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.461+0000 7fb4daec2b80 1 --2- 172.21.15.63:0/274539051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579a1d76db0 0x5579a1d94650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.461+0000 7fb4daec2b80 1 -- 172.21.15.63:0/274539051 >> 172.21.15.63:0/274539051 conn(0x5579a1c57fc0 msgr2=0x5579a1d80ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:55.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.461+0000 7fb4a7400700 0 client.16470 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:38:55.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.461+0000 7fb4daec2b80 1 -- 172.21.15.63:0/274539051 shutdown_connections 2024-05-27T07:38:55.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.461+0000 7fb4daec2b80 1 -- 172.21.15.63:0/274539051 wait complete. 2024-05-27T07:38:55.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.461+0000 7fb4daec2b80 20 client.16470 trim_cache size 0 max 16384 2024-05-27T07:38:55.474 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:38:55.474 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:38:55.474 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:38:55.474 INFO:teuthology.orchestra.run.smithi063.stdout: "02a74816-6245-422a-9bfb-3f9b57de53dc": { 2024-05-27T07:38:55.475 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:38:55.475 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "02a74816-6245-422a-9bfb-3f9b57de53dc", 2024-05-27T07:38:55.475 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:38:55.475 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:38:55.475 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:38:55.475 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:38:55.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 -- 172.21.15.63:0/1147629573 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4ac0b29f0 msgr2=0x7fb4ac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:55.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4ac0b29f0 0x7fb4ac0b4f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80098e0 tx=0x7fb4b8005960 comp rx=0 tx=0).stop 2024-05-27T07:38:55.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 -- 172.21.15.63:0/1147629573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c410e8d0 msgr2=0x7fb4c4103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:55.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c410e8d0 0x7fb4c4103c60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc007ee0 tx=0x7fb4bc00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 -- 172.21.15.63:0/1147629573 shutdown_connections 2024-05-27T07:38:55.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4ac0b29f0 0x7fb4ac0b4f00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4c410e8d0 0x7fb4c4103c60 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41054f0 0x7fb4c41031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 --2- 172.21.15.63:0/1147629573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4c4105e40 0x7fb4c4103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:55.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 -- 172.21.15.63:0/1147629573 >> 172.21.15.63:0/1147629573 conn(0x7fb4c40fedb0 msgr2=0x7fb4c40ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:55.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 -- 172.21.15.63:0/1147629573 shutdown_connections 2024-05-27T07:38:55.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:38:55.462+0000 7fb4cc400700 1 -- 172.21.15.63:0/1147629573 wait complete. 2024-05-27T07:38:56.476 INFO:tasks.cephfs.filesystem:scrub status for tag:02a74816-6245-422a-9bfb-3f9b57de53dc - {'path': '/', 'tag': '02a74816-6245-422a-9bfb-3f9b57de53dc', 'options': 'recursive,force'} 2024-05-27T07:38:57.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:57 smithi086 ceph-mon[34687]: pgmap v164: 129 pgs: 129 active+clean; 794 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:38:57.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:57 smithi149 ceph-mon[35408]: pgmap v164: 129 pgs: 129 active+clean; 794 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:38:57.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:57 smithi063 ceph-mon[31613]: pgmap v164: 129 pgs: 129 active+clean; 794 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:38:59.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:38:59 smithi086 ceph-mon[34687]: pgmap v165: 129 pgs: 129 active+clean; 826 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 38 op/s 2024-05-27T07:38:59.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:38:59 smithi149 ceph-mon[35408]: pgmap v165: 129 pgs: 129 active+clean; 826 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 38 op/s 2024-05-27T07:38:59.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:38:59 smithi063 ceph-mon[31613]: pgmap v165: 129 pgs: 129 active+clean; 826 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 38 op/s 2024-05-27T07:39:01.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:01 smithi086 ceph-mon[34687]: pgmap v166: 129 pgs: 129 active+clean; 826 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.0 MiB/s wr, 35 op/s 2024-05-27T07:39:01.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:01 smithi149 ceph-mon[35408]: pgmap v166: 129 pgs: 129 active+clean; 826 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.0 MiB/s wr, 35 op/s 2024-05-27T07:39:01.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:01 smithi063 ceph-mon[31613]: pgmap v166: 129 pgs: 129 active+clean; 826 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.0 MiB/s wr, 35 op/s 2024-05-27T07:39:02.549 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:39:02.549 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:39:02.550 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:39:02.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:39:02.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:39:02.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:39:02.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:39:02.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:39:02.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:39:03.608 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:03 smithi086 ceph-mon[34687]: pgmap v167: 129 pgs: 129 active+clean; 852 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.1 MiB/s wr, 53 op/s 2024-05-27T07:39:03.608 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:03 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:03.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:03 smithi149 ceph-mon[35408]: pgmap v167: 129 pgs: 129 active+clean; 852 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.1 MiB/s wr, 53 op/s 2024-05-27T07:39:03.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:03 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:03.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:03 smithi063 ceph-mon[31613]: pgmap v167: 129 pgs: 129 active+clean; 852 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.1 MiB/s wr, 53 op/s 2024-05-27T07:39:03.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:03 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.346+0000 7f2952c00640 1 -- 172.21.15.63:0/2156175406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294c10d9e0 msgr2=0x7f294c10fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.346+0000 7f2952c00640 1 --2- 172.21.15.63:0/2156175406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294c10d9e0 0x7f294c10fe00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2938009900 tx=0x7f293802a480 comp rx=0 tx=0).stop 2024-05-27T07:39:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.346+0000 7f2952c00640 1 -- 172.21.15.63:0/2156175406 shutdown_connections 2024-05-27T07:39:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.346+0000 7f2952c00640 1 --2- 172.21.15.63:0/2156175406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294c110340 0x7f294c112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.346+0000 7f2952c00640 1 --2- 172.21.15.63:0/2156175406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294c10d9e0 0x7f294c10fe00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.346+0000 7f2952c00640 1 --2- 172.21.15.63:0/2156175406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294c075400 0x7f294c075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.346+0000 7f2952c00640 1 -- 172.21.15.63:0/2156175406 >> 172.21.15.63:0/2156175406 conn(0x7f294c079100 msgr2=0x7f294c079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.347+0000 7f2952c00640 1 -- 172.21.15.63:0/2156175406 shutdown_connections 2024-05-27T07:39:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.347+0000 7f2952c00640 1 -- 172.21.15.63:0/2156175406 wait complete. 2024-05-27T07:39:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.348+0000 7f2952c00640 1 Processor -- start 2024-05-27T07:39:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.348+0000 7f2952c00640 1 -- start start 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2952c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294c075400 0x7f294c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2952c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294c10d9e0 0x7f294c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2952c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294c110340 0x7f294c1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2952c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f294c115db0 con 0x7f294c075400 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2952c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f294c115c30 con 0x7f294c110340 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2952c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f294c115f30 con 0x7f294c10d9e0 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2951800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294c075400 0x7f294c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2951800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294c075400 0x7f294c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57980/0 (socket says 172.21.15.63:57980) 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2951800640 1 -- 172.21.15.63:0/1224586981 learned_addr learned my addr 172.21.15.63:0/1224586981 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2950e00640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294c10d9e0 0x7f294c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.349+0000 7f2952200640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294c110340 0x7f294c1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.350+0000 7f2951800640 1 -- 172.21.15.63:0/1224586981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294c10d9e0 msgr2=0x7f294c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.350+0000 7f2951800640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294c10d9e0 0x7f294c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.350+0000 7f2951800640 1 -- 172.21.15.63:0/1224586981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294c110340 msgr2=0x7f294c1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.350+0000 7f2951800640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294c110340 0x7f294c1a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.350+0000 7f2951800640 1 -- 172.21.15.63:0/1224586981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c1a15d0 con 0x7f294c075400 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.350+0000 7f2951800640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294c075400 0x7f294c19c7f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f294000c950 tx=0x7f294000ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.351+0000 7f294a000640 1 -- 172.21.15.63:0/1224586981 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940007c70 con 0x7f294c075400 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.351+0000 7f2952c00640 1 -- 172.21.15.63:0/1224586981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c1a18c0 con 0x7f294c075400 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.351+0000 7f294a000640 1 -- 172.21.15.63:0/1224586981 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2940007e10 con 0x7f294c075400 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.351+0000 7f294a000640 1 -- 172.21.15.63:0/1224586981 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940018740 con 0x7f294c075400 2024-05-27T07:39:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.351+0000 7f2952c00640 1 -- 172.21.15.63:0/1224586981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c1a1e30 con 0x7f294c075400 2024-05-27T07:39:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.355+0000 7f295b263740 1 Processor -- start 2024-05-27T07:39:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.355+0000 7f295b263740 1 -- start start 2024-05-27T07:39:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.355+0000 7f295b263740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fd21b0 0x55d758374a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.355+0000 7f295b263740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fa92f0 0x55d758374fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.355+0000 7f295b263740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fb14d0 0x55d7583754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.355+0000 7f295b263740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d7582e5b90 con 0x55d757fd21b0 2024-05-27T07:39:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.355+0000 7f295b263740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7582d5730 con 0x55d757fa92f0 2024-05-27T07:39:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.355+0000 7f295b263740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d7582d9770 con 0x55d757fb14d0 2024-05-27T07:39:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.356+0000 7f2952200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fd21b0 0x55d758374a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.356+0000 7f2950e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fb14d0 0x55d7583754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.356+0000 7f2952200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fd21b0 0x55d758374a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57990/0 (socket says 172.21.15.63:57990) 2024-05-27T07:39:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.356+0000 7f2952200640 1 -- 172.21.15.63:0/15503398 learned_addr learned my addr 172.21.15.63:0/15503398 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.356+0000 7f2950e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fb14d0 0x55d7583754e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:47770/0 (socket says 172.21.15.63:47770) 2024-05-27T07:39:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.356+0000 7f294a000640 1 -- 172.21.15.63:0/1224586981 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f29400188e0 con 0x7f294c075400 2024-05-27T07:39:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.357+0000 7f2952200640 1 -- 172.21.15.63:0/15503398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fb14d0 msgr2=0x55d7583754e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.357+0000 7f2952200640 1 --2- 172.21.15.63:0/15503398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fb14d0 0x55d7583754e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.357+0000 7f2952200640 1 -- 172.21.15.63:0/15503398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fa92f0 msgr2=0x55d758374fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.357+0000 7f2952200640 1 --2- 172.21.15.63:0/15503398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fa92f0 0x55d758374fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.357+0000 7f2952200640 1 -- 172.21.15.63:0/15503398 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75826e8b0 con 0x55d757fd21b0 2024-05-27T07:39:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.357+0000 7f2952200640 1 --2- 172.21.15.63:0/15503398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fd21b0 0x55d758374a60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2944009e60 tx=0x7f2944010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eae0613cb6c5bae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.358+0000 7f293ea00640 1 -- 172.21.15.63:0/15503398 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944016070 con 0x55d757fd21b0 2024-05-27T07:39:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.358+0000 7f293ea00640 1 -- 172.21.15.63:0/15503398 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2944004d20 con 0x55d757fd21b0 2024-05-27T07:39:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.358+0000 7f293ea00640 1 -- 172.21.15.63:0/15503398 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29440126b0 con 0x55d757fd21b0 2024-05-27T07:39:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.359+0000 7f295b263740 1 -- 172.21.15.63:0/15503398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fd21b0 msgr2=0x55d758374a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.359+0000 7f295b263740 1 --2- 172.21.15.63:0/15503398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fd21b0 0x55d758374a60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2944009e60 tx=0x7f2944010740 comp rx=0 tx=0).stop 2024-05-27T07:39:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.359+0000 7f295b263740 1 -- 172.21.15.63:0/15503398 shutdown_connections 2024-05-27T07:39:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.359+0000 7f295b263740 1 --2- 172.21.15.63:0/15503398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fb14d0 0x55d7583754e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.359+0000 7f295b263740 1 --2- 172.21.15.63:0/15503398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fa92f0 0x55d758374fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.359+0000 7f295b263740 1 --2- 172.21.15.63:0/15503398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fd21b0 0x55d758374a60 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.359+0000 7f295b263740 1 -- 172.21.15.63:0/15503398 >> 172.21.15.63:0/15503398 conn(0x55d757fc6300 msgr2=0x55d758371210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.359+0000 7f295b263740 1 -- 172.21.15.63:0/15503398 shutdown_connections 2024-05-27T07:39:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.359+0000 7f295b263740 1 -- 172.21.15.63:0/15503398 wait complete. 2024-05-27T07:39:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.359+0000 7f294a000640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29340b2860 0x7f29340b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.360+0000 7f294a000640 1 -- 172.21.15.63:0/1224586981 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f29400e2bf0 con 0x7f294c075400 2024-05-27T07:39:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.360+0000 7f2950e00640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29340b2860 0x7f29340b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.360+0000 7f295b263740 1 -- start start 2024-05-27T07:39:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.360+0000 7f295b263740 1 Processor -- start 2024-05-27T07:39:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.361+0000 7f2950e00640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29340b2860 0x7f29340b4d70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f29380057c0 tx=0x7f2938009680 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.361+0000 7f295b263740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d75837ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.361+0000 7f295b263740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fb14d0 0x55d75837f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.361+0000 7f295b263740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d75838aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.361+0000 7f295b263740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d757fbb790 con 0x55d757fa92f0 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.361+0000 7f295b263740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d75827dbc0 con 0x55d757fb14d0 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.361+0000 7f295b263740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d758381b90 con 0x55d757fd21b0 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.361+0000 7f295b263740 10 client.0 _ll_register_callbacks cb 0x55d75825b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.361+0000 7f2951800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d75837ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.361+0000 7f2951800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d75837ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57998/0 (socket says 172.21.15.63:57998) 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.362+0000 7f2951800640 1 -- 172.21.15.63:0/564533141 learned_addr learned my addr 172.21.15.63:0/564533141 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.362+0000 7f2950e00640 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fb14d0 0x55d75837f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.362+0000 7f2951800640 1 -- 172.21.15.63:0/564533141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 msgr2=0x55d75838aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.362+0000 7f2952200640 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d75838aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.362+0000 7f2951800640 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d75838aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.362+0000 7f2951800640 1 -- 172.21.15.63:0/564533141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fb14d0 msgr2=0x55d75837f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.362+0000 7f2951800640 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fb14d0 0x55d75837f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.362+0000 7f2951800640 1 -- 172.21.15.63:0/564533141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d757fea210 con 0x55d757fa92f0 2024-05-27T07:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.362+0000 7f2951800640 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d75837ec10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f29400dffd0 tx=0x7f294000fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2839e2794377c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.362+0000 7f2933400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.363+0000 7f2933400640 1 -- 172.21.15.63:0/564533141 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400e0400 con 0x55d757fa92f0 2024-05-27T07:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.363+0000 7f2933400640 1 -- 172.21.15.63:0/564533141 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f29400a9d60 con 0x55d757fa92f0 2024-05-27T07:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.363+0000 7f295b263740 1 -- 172.21.15.63:0/564533141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2944004d20 con 0x55d757fa92f0 2024-05-27T07:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.363+0000 7f2933400640 1 -- 172.21.15.63:0/564533141 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f29400adeb0 con 0x55d757fa92f0 2024-05-27T07:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.363+0000 7f295b263740 10 client.16482 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.363+0000 7f295b263740 1 -- 172.21.15.63:0/564533141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d7580b6620 con 0x55d757fa92f0 2024-05-27T07:39:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.363+0000 7f2933400640 1 -- 172.21.15.63:0/564533141 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2944004d20 con 0x55d757fa92f0 2024-05-27T07:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.364+0000 7f2933400640 1 -- 172.21.15.63:0/564533141 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f29400b24d0 con 0x55d757fa92f0 2024-05-27T07:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.364+0000 7f295b263740 10 client.16482 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.364+0000 7f295b263740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.364+0000 7f295b263740 20 client.16482 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.364+0000 7f295b263740 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d757f90d80 0x55d758390e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.364+0000 7f295b263740 4 client.16482 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.364+0000 7f295b263740 1 -- 172.21.15.63:0/564533141 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7582e7ae0 con 0x55d757f90d80 2024-05-27T07:39:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.365+0000 7f2950e00640 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d757f90d80 0x55d758390e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.366+0000 7f2950e00640 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d757f90d80 0x55d758390e70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad92e31256e8241 server_cookie=59bd929972d24f85 in_seq=0 out_seq=0 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.366+0000 7f2933400640 10 client.16482 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f2933400640 1 -- 172.21.15.63:0/564533141 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d7582e7ae0 con 0x55d757f90d80 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f2933400640 10 client.16482 handle_command_reply: tid=0 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 client.16482 shutdown 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 -- 172.21.15.63:0/564533141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 msgr2=0x55d75837ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d75837ec10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f29400dffd0 tx=0x7f294000fd30 comp rx=0 tx=0).stop 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 -- 172.21.15.63:0/564533141 shutdown_connections 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d757f90d80 0x55d758390e70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d75838aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fb14d0 0x55d75837f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 --2- 172.21.15.63:0/564533141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d75837ec10 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f2933400640 0 client.16482 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 -- 172.21.15.63:0/564533141 >> 172.21.15.63:0/564533141 conn(0x55d757fc6300 msgr2=0x55d758374830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 -- 172.21.15.63:0/564533141 shutdown_connections 2024-05-27T07:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.368+0000 7f295b263740 1 -- 172.21.15.63:0/564533141 wait complete. 2024-05-27T07:39:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.369+0000 7f295b263740 20 client.16482 trim_cache size 0 max 16384 2024-05-27T07:39:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.388+0000 7f295b263740 1 Processor -- start 2024-05-27T07:39:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.388+0000 7f295b263740 1 -- start start 2024-05-27T07:39:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f295b263740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fa92f0 0x55d75837a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f295b263740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fb14d0 0x55d758380d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f295b263740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d7583812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f295b263740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b59b0 con 0x55d757fb14d0 2024-05-27T07:39:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f295b263740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b3280 con 0x55d757fa92f0 2024-05-27T07:39:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f295b263740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b10d0 con 0x55d757fd21b0 2024-05-27T07:39:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f2950e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fb14d0 0x55d758380d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f2950e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fb14d0 0x55d758380d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58006/0 (socket says 172.21.15.63:58006) 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f2950e00640 1 -- 172.21.15.63:0/679903232 learned_addr learned my addr 172.21.15.63:0/679903232 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f2952200640 1 --2- 172.21.15.63:0/679903232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d7583812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.389+0000 7f2951800640 1 --2- 172.21.15.63:0/679903232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fa92f0 0x55d75837a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.390+0000 7f2950e00640 1 -- 172.21.15.63:0/679903232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 msgr2=0x55d7583812a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.390+0000 7f2950e00640 1 --2- 172.21.15.63:0/679903232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d7583812a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.390+0000 7f2950e00640 1 -- 172.21.15.63:0/679903232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fa92f0 msgr2=0x55d75837a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.390+0000 7f2950e00640 1 --2- 172.21.15.63:0/679903232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fa92f0 0x55d75837a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.390+0000 7f2950e00640 1 -- 172.21.15.63:0/679903232 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75837d160 con 0x55d757fb14d0 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.390+0000 7f2952200640 1 --2- 172.21.15.63:0/679903232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d7583812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.390+0000 7f2950e00640 1 --2- 172.21.15.63:0/679903232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fb14d0 0x55d758380d60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2938000c00 tx=0x7f293803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3de1feca8b9df7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.390+0000 7f293e000640 1 -- 172.21.15.63:0/679903232 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938057070 con 0x55d757fb14d0 2024-05-27T07:39:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.391+0000 7f293e000640 1 -- 172.21.15.63:0/679903232 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29380055b0 con 0x55d757fb14d0 2024-05-27T07:39:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.391+0000 7f293e000640 1 -- 172.21.15.63:0/679903232 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938035040 con 0x55d757fb14d0 2024-05-27T07:39:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.392+0000 7f295b263740 1 -- 172.21.15.63:0/679903232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fb14d0 msgr2=0x55d758380d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.392+0000 7f295b263740 1 --2- 172.21.15.63:0/679903232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fb14d0 0x55d758380d60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2938000c00 tx=0x7f293803e000 comp rx=0 tx=0).stop 2024-05-27T07:39:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.392+0000 7f295b263740 1 -- 172.21.15.63:0/679903232 shutdown_connections 2024-05-27T07:39:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.392+0000 7f295b263740 1 --2- 172.21.15.63:0/679903232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d7583812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.392+0000 7f295b263740 1 --2- 172.21.15.63:0/679903232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fb14d0 0x55d758380d60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.392+0000 7f295b263740 1 --2- 172.21.15.63:0/679903232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fa92f0 0x55d75837a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.392+0000 7f295b263740 1 -- 172.21.15.63:0/679903232 >> 172.21.15.63:0/679903232 conn(0x55d757fc6300 msgr2=0x55d75837ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.392+0000 7f295b263740 1 -- 172.21.15.63:0/679903232 shutdown_connections 2024-05-27T07:39:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.392+0000 7f295b263740 1 -- 172.21.15.63:0/679903232 wait complete. 2024-05-27T07:39:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.393+0000 7f295b263740 1 -- start start 2024-05-27T07:39:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.393+0000 7f295b263740 1 Processor -- start 2024-05-27T07:39:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f295b263740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d7583874b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f295b263740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fb14d0 0x55d758384820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f295b263740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d758384d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f295b263740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b1740 con 0x55d757fa92f0 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f295b263740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b2bb0 con 0x55d757fb14d0 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f295b263740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1b1e10 con 0x55d757fd21b0 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f295b263740 10 client.0 _ll_register_callbacks cb 0x7f29400a59c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f2951800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d7583874b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f2951800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d7583874b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58022/0 (socket says 172.21.15.63:58022) 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f2951800640 1 -- 172.21.15.63:0/2454944248 learned_addr learned my addr 172.21.15.63:0/2454944248 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f2952200640 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d758384d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.394+0000 7f2950e00640 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fb14d0 0x55d758384820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.395+0000 7f2951800640 1 -- 172.21.15.63:0/2454944248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 msgr2=0x55d758384d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.395+0000 7f2951800640 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d758384d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.395+0000 7f2951800640 1 -- 172.21.15.63:0/2454944248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fb14d0 msgr2=0x55d758384820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.395+0000 7f2951800640 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fb14d0 0x55d758384820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.395+0000 7f2951800640 1 -- 172.21.15.63:0/2454944248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d758385450 con 0x55d757fa92f0 2024-05-27T07:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.396+0000 7f2951800640 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d7583874b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f29400a5b80 tx=0x7f29400a9bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33631acb115c57b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.396+0000 7f2933400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.396+0000 7f2933400640 1 -- 172.21.15.63:0/2454944248 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400c8020 con 0x55d757fa92f0 2024-05-27T07:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.396+0000 7f2933400640 1 -- 172.21.15.63:0/2454944248 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f294006f2f0 con 0x55d757fa92f0 2024-05-27T07:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.396+0000 7f295b263740 1 -- 172.21.15.63:0/2454944248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29380055b0 con 0x55d757fa92f0 2024-05-27T07:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.396+0000 7f2933400640 1 -- 172.21.15.63:0/2454944248 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f29400e0400 con 0x55d757fa92f0 2024-05-27T07:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.396+0000 7f295b263740 10 client.16488 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.396+0000 7f295b263740 1 -- 172.21.15.63:0/2454944248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d7583f9370 con 0x55d757fa92f0 2024-05-27T07:39:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.397+0000 7f2933400640 1 -- 172.21.15.63:0/2454944248 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29380055b0 con 0x55d757fa92f0 2024-05-27T07:39:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.397+0000 7f2933400640 1 -- 172.21.15.63:0/2454944248 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f29400cc110 con 0x55d757fa92f0 2024-05-27T07:39:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.397+0000 7f295b263740 10 client.16488 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.397+0000 7f295b263740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.397+0000 7f295b263740 20 client.16488 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.397+0000 7f295b263740 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d757f661c0 0x55d7583fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.397+0000 7f295b263740 4 client.16488 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:39:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.397+0000 7f295b263740 1 -- 172.21.15.63:0/2454944248 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7582e12c0 con 0x55d757f661c0 2024-05-27T07:39:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.398+0000 7f2950e00640 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d757f661c0 0x55d7583fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.399+0000 7f2950e00640 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d757f661c0 0x55d7583fb8d0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e0b0b91962061b server_cookie=112933fa8b82892a in_seq=0 out_seq=0 2024-05-27T07:39:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.399+0000 7f2933400640 10 client.16488 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.440+0000 7f2933400640 1 -- 172.21.15.63:0/2454944248 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d7582e12c0 con 0x55d757f661c0 2024-05-27T07:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.440+0000 7f2933400640 10 client.16488 handle_command_reply: tid=0 2024-05-27T07:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.440+0000 7f295b263740 1 client.16488 shutdown 2024-05-27T07:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.440+0000 7f295b263740 1 -- 172.21.15.63:0/2454944248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 msgr2=0x55d7583874b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.440+0000 7f295b263740 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d7583874b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f29400a5b80 tx=0x7f29400a9bc0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.441+0000 7f295b263740 1 -- 172.21.15.63:0/2454944248 shutdown_connections 2024-05-27T07:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.441+0000 7f295b263740 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d757f661c0 0x55d7583fb8d0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.441+0000 7f295b263740 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d757fd21b0 0x55d758384d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.441+0000 7f295b263740 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d757fb14d0 0x55d758384820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.441+0000 7f295b263740 1 --2- 172.21.15.63:0/2454944248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d757fa92f0 0x55d7583874b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.441+0000 7f2933400640 0 client.16488 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.441+0000 7f295b263740 1 -- 172.21.15.63:0/2454944248 >> 172.21.15.63:0/2454944248 conn(0x55d757fc6300 msgr2=0x55d758380cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.441+0000 7f295b263740 1 -- 172.21.15.63:0/2454944248 shutdown_connections 2024-05-27T07:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.441+0000 7f295b263740 1 -- 172.21.15.63:0/2454944248 wait complete. 2024-05-27T07:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.441+0000 7f295b263740 20 client.16488 trim_cache size 0 max 16384 2024-05-27T07:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:39:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:39:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 -- 172.21.15.63:0/1224586981 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29340b2860 msgr2=0x7f29340b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29340b2860 0x7f29340b4d70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f29380057c0 tx=0x7f2938009680 comp rx=0 tx=0).stop 2024-05-27T07:39:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 -- 172.21.15.63:0/1224586981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294c075400 msgr2=0x7f294c19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294c075400 0x7f294c19c7f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f294000c950 tx=0x7f294000ce10 comp rx=0 tx=0).stop 2024-05-27T07:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 -- 172.21.15.63:0/1224586981 shutdown_connections 2024-05-27T07:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29340b2860 0x7f29340b4d70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f294c110340 0x7f294c1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f294c10d9e0 0x7f294c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 --2- 172.21.15.63:0/1224586981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294c075400 0x7f294c19c7f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 -- 172.21.15.63:0/1224586981 >> 172.21.15.63:0/1224586981 conn(0x7f294c079100 msgr2=0x7f294c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 -- 172.21.15.63:0/1224586981 shutdown_connections 2024-05-27T07:39:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:05.442+0000 7f2952c00640 1 -- 172.21.15.63:0/1224586981 wait complete. 2024-05-27T07:39:05.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:05 smithi086 ceph-mon[34687]: pgmap v168: 129 pgs: 129 active+clean; 852 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.8 MiB/s wr, 50 op/s 2024-05-27T07:39:05.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:05 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:39:05.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:05 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:39:05.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:05 smithi149 ceph-mon[35408]: pgmap v168: 129 pgs: 129 active+clean; 852 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.8 MiB/s wr, 50 op/s 2024-05-27T07:39:05.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:05 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:39:05.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:05 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:39:05.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:05 smithi063 ceph-mon[31613]: pgmap v168: 129 pgs: 129 active+clean; 852 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.8 MiB/s wr, 50 op/s 2024-05-27T07:39:05.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:05 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:39:05.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:05 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:39:06.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:06 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:06.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:06 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:06.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:06 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:39:06.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:06 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:06.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:06 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:06.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:06 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:39:06.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:06 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:06.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:06 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:06.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:06 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:39:07.455 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:07 smithi086 ceph-mon[34687]: pgmap v169: 129 pgs: 129 active+clean; 852 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.8 MiB/s wr, 50 op/s 2024-05-27T07:39:07.459 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:07 smithi149 ceph-mon[35408]: pgmap v169: 129 pgs: 129 active+clean; 852 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.8 MiB/s wr, 50 op/s 2024-05-27T07:39:07.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:07 smithi063 ceph-mon[31613]: pgmap v169: 129 pgs: 129 active+clean; 852 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.8 MiB/s wr, 50 op/s 2024-05-27T07:39:09.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:09 smithi063 ceph-mon[31613]: pgmap v170: 129 pgs: 129 active+clean; 879 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 57 op/s 2024-05-27T07:39:09.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:09 smithi086 ceph-mon[34687]: pgmap v170: 129 pgs: 129 active+clean; 879 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 57 op/s 2024-05-27T07:39:09.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:09 smithi149 ceph-mon[35408]: pgmap v170: 129 pgs: 129 active+clean; 879 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 57 op/s 2024-05-27T07:39:11.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:11 smithi086 ceph-mon[34687]: pgmap v171: 129 pgs: 129 active+clean; 879 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 25 op/s 2024-05-27T07:39:11.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:11 smithi149 ceph-mon[35408]: pgmap v171: 129 pgs: 129 active+clean; 879 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 25 op/s 2024-05-27T07:39:11.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:11 smithi063 ceph-mon[31613]: pgmap v171: 129 pgs: 129 active+clean; 879 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 25 op/s 2024-05-27T07:39:13.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:13 smithi086 ceph-mon[34687]: pgmap v172: 129 pgs: 129 active+clean; 907 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.7 MiB/s wr, 28 op/s 2024-05-27T07:39:13.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:13 smithi149 ceph-mon[35408]: pgmap v172: 129 pgs: 129 active+clean; 907 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.7 MiB/s wr, 28 op/s 2024-05-27T07:39:13.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:13 smithi063 ceph-mon[31613]: pgmap v172: 129 pgs: 129 active+clean; 907 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.7 MiB/s wr, 28 op/s 2024-05-27T07:39:15.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:15 smithi086 ceph-mon[34687]: pgmap v173: 129 pgs: 129 active+clean; 907 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 MiB/s wr, 10 op/s 2024-05-27T07:39:15.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:15 smithi149 ceph-mon[35408]: pgmap v173: 129 pgs: 129 active+clean; 907 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 MiB/s wr, 10 op/s 2024-05-27T07:39:15.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:15 smithi063 ceph-mon[31613]: pgmap v173: 129 pgs: 129 active+clean; 907 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 MiB/s wr, 10 op/s 2024-05-27T07:39:17.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:17 smithi086 ceph-mon[34687]: pgmap v174: 129 pgs: 129 active+clean; 907 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 MiB/s wr, 10 op/s 2024-05-27T07:39:17.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:17 smithi149 ceph-mon[35408]: pgmap v174: 129 pgs: 129 active+clean; 907 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 MiB/s wr, 10 op/s 2024-05-27T07:39:17.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:17 smithi063 ceph-mon[31613]: pgmap v174: 129 pgs: 129 active+clean; 907 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 MiB/s wr, 10 op/s 2024-05-27T07:39:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.696+0000 7fe443200640 1 -- 172.21.15.63:0/1672758541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe444109f70 msgr2=0x7fe444103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.696+0000 7fe443200640 1 --2- 172.21.15.63:0/1672758541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe444109f70 0x7fe444103bb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe430009900 tx=0x7fe43002a3f0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.696+0000 7fe443200640 1 -- 172.21.15.63:0/1672758541 shutdown_connections 2024-05-27T07:39:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.696+0000 7fe443200640 1 --2- 172.21.15.63:0/1672758541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe444104220 0x7fe4441105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.696+0000 7fe443200640 1 --2- 172.21.15.63:0/1672758541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe444109f70 0x7fe444103bb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.696+0000 7fe443200640 1 --2- 172.21.15.63:0/1672758541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe444109ad0 0x7fe444103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.696+0000 7fe443200640 1 -- 172.21.15.63:0/1672758541 >> 172.21.15.63:0/1672758541 conn(0x7fe444079100 msgr2=0x7fe444079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.696+0000 7fe443200640 1 -- 172.21.15.63:0/1672758541 shutdown_connections 2024-05-27T07:39:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.696+0000 7fe443200640 1 -- 172.21.15.63:0/1672758541 wait complete. 2024-05-27T07:39:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.697+0000 7fe443200640 1 Processor -- start 2024-05-27T07:39:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.698+0000 7fe443200640 1 -- start start 2024-05-27T07:39:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.698+0000 7fe443200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe444104220 0x7fe44419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.698+0000 7fe443200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe444109ad0 0x7fe44419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.698+0000 7fe443200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe444109f70 0x7fe44419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.698+0000 7fe443200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe444113b70 con 0x7fe444109f70 2024-05-27T07:39:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.698+0000 7fe443200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441139f0 con 0x7fe444109ad0 2024-05-27T07:39:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.698+0000 7fe443200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe444113cf0 con 0x7fe444104220 2024-05-27T07:39:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.699+0000 7fe442800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe444109f70 0x7fe44419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.699+0000 7fe442800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe444109f70 0x7fe44419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56640/0 (socket says 172.21.15.63:56640) 2024-05-27T07:39:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.699+0000 7fe442800640 1 -- 172.21.15.63:0/4279331184 learned_addr learned my addr 172.21.15.63:0/4279331184 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.699+0000 7fe441400640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe444109ad0 0x7fe44419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.699+0000 7fe441e00640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe444104220 0x7fe44419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.699+0000 7fe442800640 1 -- 172.21.15.63:0/4279331184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe444104220 msgr2=0x7fe44419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.699+0000 7fe442800640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe444104220 0x7fe44419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.699+0000 7fe442800640 1 -- 172.21.15.63:0/4279331184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe444109ad0 msgr2=0x7fe44419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.699+0000 7fe442800640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe444109ad0 0x7fe44419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.699+0000 7fe442800640 1 -- 172.21.15.63:0/4279331184 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44419f3c0 con 0x7fe444109f70 2024-05-27T07:39:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.700+0000 7fe442800640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe444109f70 0x7fe44419ee00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00cd10 tx=0x7fe43c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.700+0000 7fe43aa00640 1 -- 172.21.15.63:0/4279331184 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c00f450 con 0x7fe444109f70 2024-05-27T07:39:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.700+0000 7fe43aa00640 1 -- 172.21.15.63:0/4279331184 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe43c00fa80 con 0x7fe444109f70 2024-05-27T07:39:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.700+0000 7fe443200640 1 -- 172.21.15.63:0/4279331184 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44419f6b0 con 0x7fe444109f70 2024-05-27T07:39:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.700+0000 7fe43aa00640 1 -- 172.21.15.63:0/4279331184 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c018ad0 con 0x7fe444109f70 2024-05-27T07:39:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.700+0000 7fe443200640 1 -- 172.21.15.63:0/4279331184 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44419fb80 con 0x7fe444109f70 2024-05-27T07:39:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.704+0000 7fe43aa00640 1 -- 172.21.15.63:0/4279331184 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe43c00fc30 con 0x7fe444109f70 2024-05-27T07:39:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.704+0000 7fe44fbcf740 1 Processor -- start 2024-05-27T07:39:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.704+0000 7fe44fbcf740 1 -- start start 2024-05-27T07:39:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.705+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeda1b0 0x563ad027c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.705+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeb12f0 0x563ad027ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.705+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad027d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.705+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563ad01edb90 con 0x563acfeda1b0 2024-05-27T07:39:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.705+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563ad01dd730 con 0x563acfeb12f0 2024-05-27T07:39:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.705+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ad01e1770 con 0x563acfeb94d0 2024-05-27T07:39:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.705+0000 7fe441400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeda1b0 0x563ad027c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.705+0000 7fe441400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeda1b0 0x563ad027c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56652/0 (socket says 172.21.15.63:56652) 2024-05-27T07:39:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.705+0000 7fe441400640 1 -- 172.21.15.63:0/1258680706 learned_addr learned my addr 172.21.15.63:0/1258680706 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.705+0000 7fe441e00640 1 --2- 172.21.15.63:0/1258680706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeb12f0 0x563ad027ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.706+0000 7fe441400640 1 --2- 172.21.15.63:0/1258680706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad027d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.706+0000 7fe441400640 1 -- 172.21.15.63:0/1258680706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 msgr2=0x563ad027d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.706+0000 7fe441400640 1 --2- 172.21.15.63:0/1258680706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad027d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.706+0000 7fe441400640 1 -- 172.21.15.63:0/1258680706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeb12f0 msgr2=0x563ad027ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.706+0000 7fe441400640 1 --2- 172.21.15.63:0/1258680706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeb12f0 0x563ad027ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.706+0000 7fe441400640 1 -- 172.21.15.63:0/1258680706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ad01768b0 con 0x563acfeda1b0 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.706+0000 7fe43aa00640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe4280b2950 0x7fe4280b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.706+0000 7fe441400640 1 --2- 172.21.15.63:0/1258680706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeda1b0 0x563ad027c8d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe4300057c0 tx=0x7fe430005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55af90519b9d197 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.707+0000 7fe43aa00640 1 -- 172.21.15.63:0/4279331184 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fe43c0e2d90 con 0x7fe444109f70 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.707+0000 7fe441e00640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe4280b2950 0x7fe4280b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.707+0000 7fe42f000640 1 -- 172.21.15.63:0/1258680706 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430044020 con 0x563acfeda1b0 2024-05-27T07:39:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.707+0000 7fe42f000640 1 -- 172.21.15.63:0/1258680706 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4300044c0 con 0x563acfeda1b0 2024-05-27T07:39:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.707+0000 7fe42f000640 1 -- 172.21.15.63:0/1258680706 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430035040 con 0x563acfeda1b0 2024-05-27T07:39:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.708+0000 7fe44fbcf740 1 -- 172.21.15.63:0/1258680706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeda1b0 msgr2=0x563ad027c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.708+0000 7fe441e00640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe4280b2950 0x7fe4280b4e60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x563ad027a550 tx=0x7fe43400b4d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.708+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/1258680706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeda1b0 0x563ad027c8d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe4300057c0 tx=0x7fe430005fd0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.708+0000 7fe44fbcf740 1 -- 172.21.15.63:0/1258680706 shutdown_connections 2024-05-27T07:39:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.708+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/1258680706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad027d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.708+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/1258680706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeb12f0 0x563ad027ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.708+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/1258680706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeda1b0 0x563ad027c8d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.708+0000 7fe44fbcf740 1 -- 172.21.15.63:0/1258680706 >> 172.21.15.63:0/1258680706 conn(0x563acfece300 msgr2=0x563ad0279210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.708+0000 7fe44fbcf740 1 -- 172.21.15.63:0/1258680706 shutdown_connections 2024-05-27T07:39:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.708+0000 7fe44fbcf740 1 -- 172.21.15.63:0/1258680706 wait complete. 2024-05-27T07:39:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.709+0000 7fe44fbcf740 1 -- start start 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.709+0000 7fe44fbcf740 1 Processor -- start 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.710+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.710+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeb94d0 0x563ad028b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.710+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeda1b0 0x563ad02972d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.710+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563acfec3790 con 0x563acfeb12f0 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.710+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563ad0185bc0 con 0x563acfeb94d0 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.710+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ad028dfa0 con 0x563acfeda1b0 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.710+0000 7fe44fbcf740 10 client.0 _ll_register_callbacks cb 0x563ad0163400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.710+0000 7fe441e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.711+0000 7fe441e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56656/0 (socket says 172.21.15.63:56656) 2024-05-27T07:39:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.711+0000 7fe441e00640 1 -- 172.21.15.63:0/196975404 learned_addr learned my addr 172.21.15.63:0/196975404 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.711+0000 7fe441400640 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeb94d0 0x563ad028b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.711+0000 7fe442800640 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeda1b0 0x563ad02972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.711+0000 7fe441e00640 1 -- 172.21.15.63:0/196975404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeda1b0 msgr2=0x563ad02972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.711+0000 7fe441e00640 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeda1b0 0x563ad02972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.711+0000 7fe441e00640 1 -- 172.21.15.63:0/196975404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeb94d0 msgr2=0x563ad028b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.711+0000 7fe441e00640 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeb94d0 0x563ad028b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.711+0000 7fe441e00640 1 -- 172.21.15.63:0/196975404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acfef2210 con 0x563acfeb12f0 2024-05-27T07:39:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.712+0000 7fe441e00640 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028b040 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe4340106a0 tx=0x7fe4340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97debf3c6241a450 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.712+0000 7fe423400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.712+0000 7fe423400640 1 -- 172.21.15.63:0/196975404 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43400f910 con 0x563acfeb12f0 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.712+0000 7fe423400640 1 -- 172.21.15.63:0/196975404 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fe43400fab0 con 0x563acfeb12f0 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.712+0000 7fe44fbcf740 1 -- 172.21.15.63:0/196975404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563ad01efae0 con 0x563acfeb12f0 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.712+0000 7fe423400640 1 -- 172.21.15.63:0/196975404 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe43400fcb0 con 0x563acfeb12f0 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.712+0000 7fe44fbcf740 10 client.16500 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.712+0000 7fe44fbcf740 1 -- 172.21.15.63:0/196975404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563acffbe620 con 0x563acfeb12f0 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.713+0000 7fe423400640 1 -- 172.21.15.63:0/196975404 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x563ad01efae0 con 0x563acfeb12f0 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.713+0000 7fe423400640 1 -- 172.21.15.63:0/196975404 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe43401f3f0 con 0x563acfeb12f0 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.713+0000 7fe44fbcf740 10 client.16500 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.713+0000 7fe44fbcf740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.714+0000 7fe44fbcf740 20 client.16500 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.714+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563acfe98d80 0x563ad029d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.714+0000 7fe44fbcf740 4 client.16500 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.714+0000 7fe44fbcf740 1 -- 172.21.15.63:0/196975404 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ad01e92c0 con 0x563acfe98d80 2024-05-27T07:39:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.714+0000 7fe441400640 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563acfe98d80 0x563ad029d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.715+0000 7fe441400640 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563acfe98d80 0x563ad029d310 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06731b2ec460af1 server_cookie=903e3acf8a178dfb in_seq=0 out_seq=0 2024-05-27T07:39:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.715+0000 7fe423400640 10 client.16500 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.717+0000 7fe423400640 1 -- 172.21.15.63:0/196975404 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563ad01e92c0 con 0x563acfe98d80 2024-05-27T07:39:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.717+0000 7fe423400640 10 client.16500 handle_command_reply: tid=0 2024-05-27T07:39:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.717+0000 7fe44fbcf740 1 client.16500 shutdown 2024-05-27T07:39:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 1 -- 172.21.15.63:0/196975404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 msgr2=0x563ad028b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028b040 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe4340106a0 tx=0x7fe4340046c0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 1 -- 172.21.15.63:0/196975404 shutdown_connections 2024-05-27T07:39:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563acfe98d80 0x563ad029d310 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeda1b0 0x563ad02972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeb94d0 0x563ad028b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/196975404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028b040 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 1 -- 172.21.15.63:0/196975404 >> 172.21.15.63:0/196975404 conn(0x563acfece300 msgr2=0x563ad027e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe423400640 0 client.16500 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 1 -- 172.21.15.63:0/196975404 shutdown_connections 2024-05-27T07:39:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 1 -- 172.21.15.63:0/196975404 wait complete. 2024-05-27T07:39:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.718+0000 7fe44fbcf740 20 client.16500 trim_cache size 0 max 16384 2024-05-27T07:39:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.738+0000 7fe44fbcf740 1 Processor -- start 2024-05-27T07:39:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.738+0000 7fe44fbcf740 1 -- start start 2024-05-27T07:39:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.738+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.738+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad027d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.738+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeda1b0 0x563ad0284ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.738+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441b1470 con 0x563acfeb12f0 2024-05-27T07:39:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.738+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441aed40 con 0x563acfeda1b0 2024-05-27T07:39:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.738+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441acb90 con 0x563acfeb94d0 2024-05-27T07:39:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.739+0000 7fe441e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.739+0000 7fe441e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56668/0 (socket says 172.21.15.63:56668) 2024-05-27T07:39:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.739+0000 7fe441e00640 1 -- 172.21.15.63:0/150935690 learned_addr learned my addr 172.21.15.63:0/150935690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.739+0000 7fe442800640 1 --2- 172.21.15.63:0/150935690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeda1b0 0x563ad0284ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.739+0000 7fe441400640 1 --2- 172.21.15.63:0/150935690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad027d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.739+0000 7fe441e00640 1 -- 172.21.15.63:0/150935690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 msgr2=0x563ad027d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.739+0000 7fe441e00640 1 --2- 172.21.15.63:0/150935690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad027d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.739+0000 7fe441e00640 1 -- 172.21.15.63:0/150935690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeda1b0 msgr2=0x563ad0284ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.739+0000 7fe441e00640 1 --2- 172.21.15.63:0/150935690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeda1b0 0x563ad0284ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.739+0000 7fe441e00640 1 -- 172.21.15.63:0/150935690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ad027de10 con 0x563acfeb12f0 2024-05-27T07:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.740+0000 7fe441e00640 1 --2- 172.21.15.63:0/150935690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028d770 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe43401f890 tx=0x7fe434005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a2c5daa44cb6df server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.740+0000 7fe42e600640 1 -- 172.21.15.63:0/150935690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434039070 con 0x563acfeb12f0 2024-05-27T07:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.740+0000 7fe42e600640 1 -- 172.21.15.63:0/150935690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe43401fe50 con 0x563acfeb12f0 2024-05-27T07:39:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.740+0000 7fe42e600640 1 -- 172.21.15.63:0/150935690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434035690 con 0x563acfeb12f0 2024-05-27T07:39:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.741+0000 7fe44fbcf740 1 -- 172.21.15.63:0/150935690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 msgr2=0x563ad028d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.741+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/150935690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028d770 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe43401f890 tx=0x7fe434005740 comp rx=0 tx=0).stop 2024-05-27T07:39:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.741+0000 7fe44fbcf740 1 -- 172.21.15.63:0/150935690 shutdown_connections 2024-05-27T07:39:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.741+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/150935690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeda1b0 0x563ad0284ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.741+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/150935690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad027d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.741+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/150935690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad028d770 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.741+0000 7fe44fbcf740 1 -- 172.21.15.63:0/150935690 >> 172.21.15.63:0/150935690 conn(0x563acfece300 msgr2=0x563ad027f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.741+0000 7fe44fbcf740 1 -- 172.21.15.63:0/150935690 shutdown_connections 2024-05-27T07:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.741+0000 7fe44fbcf740 1 -- 172.21.15.63:0/150935690 wait complete. 2024-05-27T07:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.742+0000 7fe44fbcf740 1 -- start start 2024-05-27T07:39:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.742+0000 7fe44fbcf740 1 Processor -- start 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.742+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad0284f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.742+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad0285440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.742+0000 7fe44fbcf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeda1b0 0x563ad0285980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441ad200 con 0x563acfeb12f0 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441ae670 con 0x563acfeda1b0 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe44fbcf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe4441ad8d0 con 0x563acfeb94d0 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe44fbcf740 10 client.0 _ll_register_callbacks cb 0x7fe43400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe441e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad0284f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe441e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad0284f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56684/0 (socket says 172.21.15.63:56684) 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe441e00640 1 -- 172.21.15.63:0/4021660823 learned_addr learned my addr 172.21.15.63:0/4021660823 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe441400640 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad0285440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe442800640 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeda1b0 0x563ad0285980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe441e00640 1 -- 172.21.15.63:0/4021660823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 msgr2=0x563ad0285440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe441e00640 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad0285440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe441e00640 1 -- 172.21.15.63:0/4021660823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeda1b0 msgr2=0x563ad0285980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe441e00640 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeda1b0 0x563ad0285980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.743+0000 7fe441e00640 1 -- 172.21.15.63:0/4021660823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ad0292390 con 0x563acfeb12f0 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.744+0000 7fe441e00640 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad0284f00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe434012ea0 tx=0x7fe43401f620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708fdf2d28cc45b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.744+0000 7fe423400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.744+0000 7fe423400640 1 -- 172.21.15.63:0/4021660823 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434039040 con 0x563acfeb12f0 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.744+0000 7fe423400640 1 -- 172.21.15.63:0/4021660823 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fe434005240 con 0x563acfeb12f0 2024-05-27T07:39:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.744+0000 7fe44fbcf740 1 -- 172.21.15.63:0/4021660823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe43401fe50 con 0x563acfeb12f0 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.744+0000 7fe423400640 1 -- 172.21.15.63:0/4021660823 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe43404f050 con 0x563acfeb12f0 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.744+0000 7fe44fbcf740 10 client.16506 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.745+0000 7fe44fbcf740 1 -- 172.21.15.63:0/4021660823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ad0293620 con 0x563acfeb12f0 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.745+0000 7fe423400640 1 -- 172.21.15.63:0/4021660823 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe43401fe50 con 0x563acfeb12f0 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.745+0000 7fe423400640 1 -- 172.21.15.63:0/4021660823 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe43403e450 con 0x563acfeb12f0 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.745+0000 7fe44fbcf740 10 client.16506 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.745+0000 7fe44fbcf740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.745+0000 7fe44fbcf740 20 client.16506 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.745+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563acfe6e1c0 0x563ad03079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.745+0000 7fe44fbcf740 4 client.16506 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.745+0000 7fe44fbcf740 1 -- 172.21.15.63:0/4021660823 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ad01634b0 con 0x563acfe6e1c0 2024-05-27T07:39:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.746+0000 7fe441400640 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563acfe6e1c0 0x563ad03079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.747+0000 7fe441400640 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563acfe6e1c0 0x563ad03079b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986e6b2b90c4a4f9 server_cookie=6e1848b7403c6312 in_seq=0 out_seq=0 2024-05-27T07:39:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.747+0000 7fe423400640 10 client.16506 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.788+0000 7fe423400640 1 -- 172.21.15.63:0/4021660823 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563ad01634b0 con 0x563acfe6e1c0 2024-05-27T07:39:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.788+0000 7fe423400640 10 client.16506 handle_command_reply: tid=0 2024-05-27T07:39:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.788+0000 7fe44fbcf740 1 client.16506 shutdown 2024-05-27T07:39:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.788+0000 7fe44fbcf740 1 -- 172.21.15.63:0/4021660823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 msgr2=0x563ad0284f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.788+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad0284f00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe434012ea0 tx=0x7fe43401f620 comp rx=0 tx=0).stop 2024-05-27T07:39:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.789+0000 7fe44fbcf740 1 -- 172.21.15.63:0/4021660823 shutdown_connections 2024-05-27T07:39:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.789+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563acfe6e1c0 0x563ad03079b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.789+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563acfeda1b0 0x563ad0285980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.789+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563acfeb94d0 0x563ad0285440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.789+0000 7fe44fbcf740 1 --2- 172.21.15.63:0/4021660823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563acfeb12f0 0x563ad0284f00 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.789+0000 7fe44fbcf740 1 -- 172.21.15.63:0/4021660823 >> 172.21.15.63:0/4021660823 conn(0x563acfece300 msgr2=0x563ad028a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.789+0000 7fe423400640 0 client.16506 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.789+0000 7fe44fbcf740 1 -- 172.21.15.63:0/4021660823 shutdown_connections 2024-05-27T07:39:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.789+0000 7fe44fbcf740 1 -- 172.21.15.63:0/4021660823 wait complete. 2024-05-27T07:39:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.789+0000 7fe44fbcf740 20 client.16506 trim_cache size 0 max 16384 2024-05-27T07:39:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:39:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:39:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:39:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.790+0000 7fe443200640 1 -- 172.21.15.63:0/4279331184 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe4280b2950 msgr2=0x7fe4280b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.790+0000 7fe443200640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe4280b2950 0x7fe4280b4e60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x563ad027a550 tx=0x7fe43400b4d0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.790+0000 7fe443200640 1 -- 172.21.15.63:0/4279331184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe444109f70 msgr2=0x7fe44419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.790+0000 7fe443200640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe444109f70 0x7fe44419ee00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00cd10 tx=0x7fe43c00a620 comp rx=0 tx=0).stop 2024-05-27T07:39:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.790+0000 7fe443200640 1 -- 172.21.15.63:0/4279331184 shutdown_connections 2024-05-27T07:39:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.790+0000 7fe443200640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe4280b2950 0x7fe4280b4e60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.790+0000 7fe443200640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe444109f70 0x7fe44419ee00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.790+0000 7fe443200640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe444109ad0 0x7fe44419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.790+0000 7fe443200640 1 --2- 172.21.15.63:0/4279331184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe444104220 0x7fe44419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.790+0000 7fe443200640 1 -- 172.21.15.63:0/4279331184 >> 172.21.15.63:0/4279331184 conn(0x7fe444079100 msgr2=0x7fe4441022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.791+0000 7fe443200640 1 -- 172.21.15.63:0/4279331184 shutdown_connections 2024-05-27T07:39:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:18.791+0000 7fe443200640 1 -- 172.21.15.63:0/4279331184 wait complete. 2024-05-27T07:39:19.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:19 smithi086 ceph-mon[34687]: pgmap v175: 129 pgs: 129 active+clean; 923 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 14 op/s 2024-05-27T07:39:19.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:19 smithi149 ceph-mon[35408]: pgmap v175: 129 pgs: 129 active+clean; 923 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 14 op/s 2024-05-27T07:39:19.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:19 smithi063 ceph-mon[31613]: pgmap v175: 129 pgs: 129 active+clean; 923 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 14 op/s 2024-05-27T07:39:21.513 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:21 smithi149 ceph-mon[35408]: pgmap v176: 129 pgs: 129 active+clean; 923 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:39:21.705 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:21 smithi086 ceph-mon[34687]: pgmap v176: 129 pgs: 129 active+clean; 923 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:39:21.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:21 smithi063 ceph-mon[31613]: pgmap v176: 129 pgs: 129 active+clean; 923 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:39:23.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:23 smithi086 ceph-mon[34687]: pgmap v177: 129 pgs: 129 active+clean; 947 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:39:23.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:23 smithi149 ceph-mon[35408]: pgmap v177: 129 pgs: 129 active+clean; 947 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:39:23.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:23 smithi063 ceph-mon[31613]: pgmap v177: 129 pgs: 129 active+clean; 947 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:39:25.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:25 smithi086 ceph-mon[34687]: pgmap v178: 129 pgs: 129 active+clean; 947 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T07:39:25.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:25 smithi149 ceph-mon[35408]: pgmap v178: 129 pgs: 129 active+clean; 947 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T07:39:25.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:25 smithi063 ceph-mon[31613]: pgmap v178: 129 pgs: 129 active+clean; 947 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T07:39:26.478 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:39:27.009 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:39:27.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:27 smithi063 ceph-mon[31613]: pgmap v179: 129 pgs: 129 active+clean; 947 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T07:39:27.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:27 smithi086 ceph-mon[34687]: pgmap v179: 129 pgs: 129 active+clean; 947 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T07:39:27.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:27 smithi149 ceph-mon[35408]: pgmap v179: 129 pgs: 129 active+clean; 947 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T07:39:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.439+0000 7f26ae000700 1 -- 172.21.15.63:0/1054360297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8108f10 msgr2=0x7f26a8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.439+0000 7f26ae000700 1 --2- 172.21.15.63:0/1054360297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8108f10 0x7f26a8109320 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f26940099c0 tx=0x7f2694009cd0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.440+0000 7f26ae000700 1 -- 172.21.15.63:0/1054360297 shutdown_connections 2024-05-27T07:39:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.440+0000 7f26ae000700 1 --2- 172.21.15.63:0/1054360297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a8109860 0x7f26a81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.440+0000 7f26ae000700 1 --2- 172.21.15.63:0/1054360297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a81045e0 0x7f26a81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.440+0000 7f26ae000700 1 --2- 172.21.15.63:0/1054360297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8108f10 0x7f26a8109320 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.440+0000 7f26ae000700 1 -- 172.21.15.63:0/1054360297 >> 172.21.15.63:0/1054360297 conn(0x7f26a8100030 msgr2=0x7f26a8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.440+0000 7f26ae000700 1 -- 172.21.15.63:0/1054360297 shutdown_connections 2024-05-27T07:39:29.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.440+0000 7f26ae000700 1 -- 172.21.15.63:0/1054360297 wait complete. 2024-05-27T07:39:29.439 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:29 smithi063 ceph-mon[31613]: pgmap v180: 129 pgs: 129 active+clean; 971 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:39:29.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.441+0000 7f26ae000700 1 Processor -- start 2024-05-27T07:39:29.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.442+0000 7f26ae000700 1 -- start start 2024-05-27T07:39:29.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.442+0000 7f26ae000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a81045e0 0x7f26a8075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.442+0000 7f26ae000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8108f10 0x7f26a8075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.442+0000 7f26ae000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a8109860 0x7f26a8076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.442+0000 7f26ae000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8114370 con 0x7f26a8108f10 2024-05-27T07:39:29.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.442+0000 7f26ae000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81141f0 con 0x7f26a81045e0 2024-05-27T07:39:29.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.442+0000 7f26ae000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81144f0 con 0x7f26a8109860 2024-05-27T07:39:29.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.443+0000 7f26a6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8108f10 0x7f26a8075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.443+0000 7f26a6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8108f10 0x7f26a8075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34718/0 (socket says 172.21.15.63:34718) 2024-05-27T07:39:29.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.443+0000 7f26a6a00700 1 -- 172.21.15.63:0/453168286 learned_addr learned my addr 172.21.15.63:0/453168286 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:29.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.443+0000 7f26a7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a81045e0 0x7f26a8075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.443+0000 7f26a7e00700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a8109860 0x7f26a8076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.444+0000 7f26a6a00700 1 -- 172.21.15.63:0/453168286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a8109860 msgr2=0x7f26a8076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:29.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.444+0000 7f26a6a00700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a8109860 0x7f26a8076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.444+0000 7f26a6a00700 1 -- 172.21.15.63:0/453168286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a81045e0 msgr2=0x7f26a8075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:29.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.444+0000 7f26a6a00700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a81045e0 0x7f26a8075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.444+0000 7f26a6a00700 1 -- 172.21.15.63:0/453168286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a81a9160 con 0x7f26a8108f10 2024-05-27T07:39:29.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.444+0000 7f26a6a00700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8108f10 0x7f26a8075ff0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f26940098e0 tx=0x7f26940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:29.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.445+0000 7f269fe00700 1 -- 172.21.15.63:0/453168286 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694027070 con 0x7f26a8108f10 2024-05-27T07:39:29.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.445+0000 7f269fe00700 1 -- 172.21.15.63:0/453168286 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f269400fc00 con 0x7f26a8108f10 2024-05-27T07:39:29.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.445+0000 7f26ae000700 1 -- 172.21.15.63:0/453168286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a81a93f0 con 0x7f26a8108f10 2024-05-27T07:39:29.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.445+0000 7f269fe00700 1 -- 172.21.15.63:0/453168286 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694022ca0 con 0x7f26a8108f10 2024-05-27T07:39:29.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.445+0000 7f26ae000700 1 -- 172.21.15.63:0/453168286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a81a98a0 con 0x7f26a8108f10 2024-05-27T07:39:29.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.452+0000 7f269fe00700 1 -- 172.21.15.63:0/453168286 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2694022470 con 0x7f26a8108f10 2024-05-27T07:39:29.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.452+0000 7f26bcb4bb80 1 Processor -- start 2024-05-27T07:39:29.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.452+0000 7f26bcb4bb80 1 -- start start 2024-05-27T07:39:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.453+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x556959921250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.453+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x5569599217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.453+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959936940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.453+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55695984fe40 con 0x55695988e9f0 2024-05-27T07:39:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.453+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55695983f9e0 con 0x5569597a2e40 2024-05-27T07:39:29.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.453+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556959843a20 con 0x556959921db0 2024-05-27T07:39:29.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.454+0000 7f26a7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x5569599217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.454+0000 7f26a7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x5569599217b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:38584/0 (socket says 172.21.15.63:38584) 2024-05-27T07:39:29.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.454+0000 7f26a7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x556959921250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.454+0000 7f26a7400700 1 -- 172.21.15.63:0/2574288901 learned_addr learned my addr 172.21.15.63:0/2574288901 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:29.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.454+0000 7f26a6a00700 1 --2- 172.21.15.63:0/2574288901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959936940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.454+0000 7f269fe00700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f268c0b2a70 0x7f268c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.455+0000 7f26a7400700 1 -- 172.21.15.63:0/2574288901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 msgr2=0x556959936940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:29.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.455+0000 7f26a7400700 1 --2- 172.21.15.63:0/2574288901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959936940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.455+0000 7f26a7400700 1 -- 172.21.15.63:0/2574288901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 msgr2=0x556959921250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:29.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.455+0000 7f26a7400700 1 --2- 172.21.15.63:0/2574288901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x556959921250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.455+0000 7f26a7400700 1 -- 172.21.15.63:0/2574288901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556959808420 con 0x5569597a2e40 2024-05-27T07:39:29.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.455+0000 7f26a7e00700 1 --2- 172.21.15.63:0/2574288901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x556959921250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:39:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.455+0000 7f269fe00700 1 -- 172.21.15.63:0/453168286 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f26940e3840 con 0x7f26a8108f10 2024-05-27T07:39:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.456+0000 7f26a7e00700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f268c0b2a70 0x7f268c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.456+0000 7f26a7400700 1 --2- 172.21.15.63:0/2574288901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x5569599217b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f269800bd70 tx=0x7f26980106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2e27e287c09e57a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:29.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.457+0000 7f26a7e00700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f268c0b2a70 0x7f268c0b4f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009710 tx=0x7f26a0004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:29.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.457+0000 7f269cc00700 1 -- 172.21.15.63:0/2574288901 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698010e10 con 0x5569597a2e40 2024-05-27T07:39:29.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.457+0000 7f269cc00700 1 -- 172.21.15.63:0/2574288901 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f269801c910 con 0x5569597a2e40 2024-05-27T07:39:29.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.457+0000 7f269cc00700 1 -- 172.21.15.63:0/2574288901 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698011870 con 0x5569597a2e40 2024-05-27T07:39:29.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.458+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2574288901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 msgr2=0x5569599217b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:29.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.458+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2574288901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x5569599217b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f269800bd70 tx=0x7f26980106b0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.459+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2574288901 shutdown_connections 2024-05-27T07:39:29.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.459+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2574288901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959936940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.459+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2574288901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x556959921250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.459+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2574288901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x5569599217b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.459+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2574288901 >> 172.21.15.63:0/2574288901 conn(0x55695983c390 msgr2=0x55695991d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:29.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.459+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2574288901 shutdown_connections 2024-05-27T07:39:29.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.460+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2574288901 wait complete. 2024-05-27T07:39:29.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.460+0000 7f26bcb4bb80 1 -- start start 2024-05-27T07:39:29.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.461+0000 7f26bcb4bb80 1 Processor -- start 2024-05-27T07:39:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.461+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x556959933eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.461+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x5569599343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.461+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959934930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.461+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556959745fa0 con 0x55695988e9f0 2024-05-27T07:39:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.461+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5569597e0760 con 0x5569597a2e40 2024-05-27T07:39:29.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.461+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5569599373e0 con 0x556959921db0 2024-05-27T07:39:29.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.462+0000 7f26bcb4bb80 10 client.0 _ll_register_callbacks cb 0x5569597e9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:29.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.462+0000 7f26a6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x5569599343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.462+0000 7f26a6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x5569599343f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34730/0 (socket says 172.21.15.63:34730) 2024-05-27T07:39:29.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.462+0000 7f26a6a00700 1 -- 172.21.15.63:0/1948089307 learned_addr learned my addr 172.21.15.63:0/1948089307 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:29.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.462+0000 7f26a7e00700 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959934930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.462+0000 7f26a7400700 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x556959933eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.463+0000 7f26a6a00700 1 -- 172.21.15.63:0/1948089307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 msgr2=0x556959934930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.463+0000 7f26a6a00700 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959934930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.463+0000 7f26a6a00700 1 -- 172.21.15.63:0/1948089307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 msgr2=0x556959933eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.463+0000 7f26a6a00700 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x556959933eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.463+0000 7f26a6a00700 1 -- 172.21.15.63:0/1948089307 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55695978f8d0 con 0x55695988e9f0 2024-05-27T07:39:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.463+0000 7f26a7400700 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x556959933eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:39:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.463+0000 7f26a6a00700 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x5569599343f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f26940e1220 tx=0x7f26940b3460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afa3898d74c0b25 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.463+0000 7f2691600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.464+0000 7f2691600700 1 -- 172.21.15.63:0/1948089307 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26940a6700 con 0x55695988e9f0 2024-05-27T07:39:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.464+0000 7f2691600700 1 -- 172.21.15.63:0/1948089307 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f269400fdb0 con 0x55695988e9f0 2024-05-27T07:39:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.464+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/1948089307 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f269801c910 con 0x55695988e9f0 2024-05-27T07:39:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.464+0000 7f2691600700 1 -- 172.21.15.63:0/1948089307 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f26940cd110 con 0x55695988e9f0 2024-05-27T07:39:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.464+0000 7f26bcb4bb80 10 client.16518 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.464+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/1948089307 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556959810060 con 0x55695988e9f0 2024-05-27T07:39:29.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.464+0000 7f2691600700 1 -- 172.21.15.63:0/1948089307 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f269801c910 con 0x55695988e9f0 2024-05-27T07:39:29.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.465+0000 7f2691600700 1 -- 172.21.15.63:0/1948089307 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f26940a61b0 con 0x55695988e9f0 2024-05-27T07:39:29.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.465+0000 7f26bcb4bb80 10 client.16518 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:29.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.465+0000 7f26bcb4bb80 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:29.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.465+0000 7f26bcb4bb80 20 client.16518 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:29.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.465+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556959944340 0x5569599467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.465+0000 7f26bcb4bb80 4 client.16518 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:29.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.465+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/1948089307 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556959851d90 con 0x556959944340 2024-05-27T07:39:29.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.466+0000 7f26a7400700 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556959944340 0x5569599467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.467+0000 7f26a7400700 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556959944340 0x5569599467c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee81f953742b936c server_cookie=433cf92a2321cfb5 in_seq=0 out_seq=0 2024-05-27T07:39:29.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.467+0000 7f2691600700 10 client.16518 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:29.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.469+0000 7f2691600700 1 -- 172.21.15.63:0/1948089307 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556959851d90 con 0x556959944340 2024-05-27T07:39:29.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.469+0000 7f2691600700 10 client.16518 handle_command_reply: tid=0 2024-05-27T07:39:29.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.469+0000 7f26bcb4bb80 1 client.16518 shutdown 2024-05-27T07:39:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/1948089307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 msgr2=0x5569599343f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x5569599343f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f26940e1220 tx=0x7f26940b3460 comp rx=0 tx=0).stop 2024-05-27T07:39:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/1948089307 shutdown_connections 2024-05-27T07:39:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556959944340 0x5569599467c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959934930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x556959933eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/1948089307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x5569599343f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f2691600700 0 client.16518 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/1948089307 >> 172.21.15.63:0/1948089307 conn(0x55695983c390 msgr2=0x55695991d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/1948089307 shutdown_connections 2024-05-27T07:39:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.470+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/1948089307 wait complete. 2024-05-27T07:39:29.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.471+0000 7f26bcb4bb80 20 client.16518 trim_cache size 0 max 16384 2024-05-27T07:39:29.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.499+0000 7f26bcb4bb80 1 Processor -- start 2024-05-27T07:39:29.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.499+0000 7f26bcb4bb80 1 -- start start 2024-05-27T07:39:29.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569597a2e40 0x55695992f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55695988e9f0 0x556959921780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959932e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81b0a70 con 0x5569597a2e40 2024-05-27T07:39:29.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81ae300 con 0x55695988e9f0 2024-05-27T07:39:29.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81ac180 con 0x556959921db0 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569597a2e40 0x55695992f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569597a2e40 0x55695992f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34736/0 (socket says 172.21.15.63:34736) 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26a7400700 1 -- 172.21.15.63:0/2033241963 learned_addr learned my addr 172.21.15.63:0/2033241963 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26a6a00700 1 --2- 172.21.15.63:0/2033241963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55695988e9f0 0x556959921780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.500+0000 7f26a7e00700 1 --2- 172.21.15.63:0/2033241963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959932e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.501+0000 7f26a7400700 1 -- 172.21.15.63:0/2033241963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 msgr2=0x556959932e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.501+0000 7f26a7400700 1 --2- 172.21.15.63:0/2033241963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959932e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.501+0000 7f26a7400700 1 -- 172.21.15.63:0/2033241963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55695988e9f0 msgr2=0x556959921780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.501+0000 7f26a7400700 1 --2- 172.21.15.63:0/2033241963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55695988e9f0 0x556959921780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.501+0000 7f26a7400700 1 -- 172.21.15.63:0/2033241963 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569599221c0 con 0x5569597a2e40 2024-05-27T07:39:29.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.501+0000 7f26a6a00700 1 --2- 172.21.15.63:0/2033241963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55695988e9f0 0x556959921780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:29.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.501+0000 7f26a7400700 1 --2- 172.21.15.63:0/2033241963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569597a2e40 0x55695992f6f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f269800bd40 tx=0x7f269800b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2fcbd0fb1f38c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:29.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.502+0000 7f2693e00700 1 -- 172.21.15.63:0/2033241963 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980052c0 con 0x5569597a2e40 2024-05-27T07:39:29.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.502+0000 7f2693e00700 1 -- 172.21.15.63:0/2033241963 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2698005460 con 0x5569597a2e40 2024-05-27T07:39:29.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.502+0000 7f2693e00700 1 -- 172.21.15.63:0/2033241963 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980286b0 con 0x5569597a2e40 2024-05-27T07:39:29.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.503+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2033241963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569597a2e40 msgr2=0x55695992f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:29.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.503+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2033241963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569597a2e40 0x55695992f6f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f269800bd40 tx=0x7f269800b670 comp rx=0 tx=0).stop 2024-05-27T07:39:29.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.503+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2033241963 shutdown_connections 2024-05-27T07:39:29.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.503+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2033241963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x556959932e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.503+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2033241963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569597a2e40 0x55695992f6f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.503+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2033241963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55695988e9f0 0x556959921780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.503+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2033241963 >> 172.21.15.63:0/2033241963 conn(0x55695983c390 msgr2=0x556959923980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:29.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.504+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2033241963 shutdown_connections 2024-05-27T07:39:29.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.504+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2033241963 wait complete. 2024-05-27T07:39:29.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.504+0000 7f26bcb4bb80 1 -- start start 2024-05-27T07:39:29.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.505+0000 7f26bcb4bb80 1 Processor -- start 2024-05-27T07:39:29.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.505+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x55695992eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.505+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x55695992f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.505+0000 7f26bcb4bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x55695992f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.505+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81ac7c0 con 0x55695988e9f0 2024-05-27T07:39:29.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.505+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81adc30 con 0x5569597a2e40 2024-05-27T07:39:29.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.505+0000 7f26bcb4bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81ace90 con 0x556959921db0 2024-05-27T07:39:29.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.505+0000 7f26bcb4bb80 10 client.0 _ll_register_callbacks cb 0x5569597e9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x55695992f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x55695992f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34750/0 (socket says 172.21.15.63:34750) 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a6a00700 1 -- 172.21.15.63:0/2665925473 learned_addr learned my addr 172.21.15.63:0/2665925473 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a7400700 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x55695992eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a7e00700 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x55695992f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a6a00700 1 -- 172.21.15.63:0/2665925473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 msgr2=0x55695992f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a6a00700 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x55695992f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a6a00700 1 -- 172.21.15.63:0/2665925473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 msgr2=0x55695992eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a6a00700 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x55695992eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a6a00700 1 -- 172.21.15.63:0/2665925473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556959934ca0 con 0x55695988e9f0 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.506+0000 7f26a7400700 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x55695992eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:29.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.507+0000 7f26a6a00700 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x55695992f0c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f26940e92f0 tx=0x7f26940a9ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dd2b492b18aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:29.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.507+0000 7f2691600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:29.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.507+0000 7f2691600700 1 -- 172.21.15.63:0/2665925473 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26940a9fb0 con 0x55695988e9f0 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.507+0000 7f2691600700 1 -- 172.21.15.63:0/2665925473 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f26940a6e30 con 0x55695988e9f0 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.507+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2665925473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2698005460 con 0x55695988e9f0 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.508+0000 7f2691600700 1 -- 172.21.15.63:0/2665925473 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f26940aa190 con 0x55695988e9f0 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.508+0000 7f26bcb4bb80 10 client.16524 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.508+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2665925473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556959934ca0 con 0x55695988e9f0 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.508+0000 7f2691600700 1 -- 172.21.15.63:0/2665925473 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2698005460 con 0x55695988e9f0 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.508+0000 7f2691600700 1 -- 172.21.15.63:0/2665925473 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f26940e3470 con 0x55695988e9f0 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.508+0000 7f26bcb4bb80 10 client.16524 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.508+0000 7f26bcb4bb80 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.508+0000 7f26bcb4bb80 20 client.16524 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.509+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569599af390 0x5569599b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.509+0000 7f26bcb4bb80 4 client.16524 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:39:29.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.509+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2665925473 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55695984b570 con 0x5569599af390 2024-05-27T07:39:29.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.509+0000 7f26a7400700 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569599af390 0x5569599b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:29.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.510+0000 7f26a7400700 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569599af390 0x5569599b17b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0fa3feb371b6 server_cookie=c75afd1f2fd764f0 in_seq=0 out_seq=0 2024-05-27T07:39:29.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.510+0000 7f2691600700 10 client.16524 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:29.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f2691600700 1 -- 172.21.15.63:0/2665925473 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55695984b570 con 0x5569599af390 2024-05-27T07:39:29.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f2691600700 10 client.16524 handle_command_reply: tid=0 2024-05-27T07:39:29.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f26bcb4bb80 1 client.16524 shutdown 2024-05-27T07:39:29.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2665925473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 msgr2=0x55695992f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:29.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x55695992f0c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f26940e92f0 tx=0x7f26940a9ee0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2665925473 shutdown_connections 2024-05-27T07:39:29.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569599af390 0x5569599b17b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556959921db0 0x55695992f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569597a2e40 0x55695992eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f26bcb4bb80 1 --2- 172.21.15.63:0/2665925473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55695988e9f0 0x55695992f0c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2665925473 >> 172.21.15.63:0/2665925473 conn(0x55695983c390 msgr2=0x556959922f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:29.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.511+0000 7f2691600700 0 client.16524 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:29.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.512+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2665925473 shutdown_connections 2024-05-27T07:39:29.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.512+0000 7f26bcb4bb80 1 -- 172.21.15.63:0/2665925473 wait complete. 2024-05-27T07:39:29.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.512+0000 7f26bcb4bb80 20 client.16524 trim_cache size 0 max 16384 2024-05-27T07:39:29.511 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:39:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:39:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:39:29.511 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:39:29.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.513+0000 7f26ae000700 1 -- 172.21.15.63:0/453168286 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f268c0b2a70 msgr2=0x7f268c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:29.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.513+0000 7f26ae000700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f268c0b2a70 0x7f268c0b4f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009710 tx=0x7f26a0004ad0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.513+0000 7f26ae000700 1 -- 172.21.15.63:0/453168286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8108f10 msgr2=0x7f26a8075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:29.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.513+0000 7f26ae000700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8108f10 0x7f26a8075ff0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f26940098e0 tx=0x7f26940047b0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.513+0000 7f26ae000700 1 -- 172.21.15.63:0/453168286 shutdown_connections 2024-05-27T07:39:29.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.513+0000 7f26ae000700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f268c0b2a70 0x7f268c0b4f80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.514+0000 7f26ae000700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a8109860 0x7f26a8076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.514+0000 7f26ae000700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a81045e0 0x7f26a8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.514+0000 7f26ae000700 1 --2- 172.21.15.63:0/453168286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a8108f10 0x7f26a8075ff0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:29.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.514+0000 7f26ae000700 1 -- 172.21.15.63:0/453168286 >> 172.21.15.63:0/453168286 conn(0x7f26a8100030 msgr2=0x7f26a810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:29.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.514+0000 7f26ae000700 1 -- 172.21.15.63:0/453168286 shutdown_connections 2024-05-27T07:39:29.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:29.514+0000 7f26ae000700 1 -- 172.21.15.63:0/453168286 wait complete. 2024-05-27T07:39:29.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:29 smithi086 ceph-mon[34687]: pgmap v180: 129 pgs: 129 active+clean; 971 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:39:29.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:29 smithi149 ceph-mon[35408]: pgmap v180: 129 pgs: 129 active+clean; 971 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:39:30.371 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:39:30.372 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:39:30.886 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:39:31.541 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:31 smithi063 ceph-mon[31613]: pgmap v181: 129 pgs: 129 active+clean; 971 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:39:31.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:31 smithi086 ceph-mon[34687]: pgmap v181: 129 pgs: 129 active+clean; 971 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:39:31.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:31 smithi149 ceph-mon[35408]: pgmap v181: 129 pgs: 129 active+clean; 971 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:39:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.045+0000 7f2f53200640 1 -- 172.21.15.63:0/2016804216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5410e000 msgr2=0x7f2f541104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.045+0000 7f2f53200640 1 --2- 172.21.15.63:0/2016804216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5410e000 0x7f2f541104f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009ca0 tx=0x7f2f4c02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.045+0000 7f2f53200640 1 -- 172.21.15.63:0/2016804216 shutdown_connections 2024-05-27T07:39:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.045+0000 7f2f53200640 1 --2- 172.21.15.63:0/2016804216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5410e000 0x7f2f541104f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.045+0000 7f2f53200640 1 --2- 172.21.15.63:0/2016804216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f5410b610 0x7f2f5410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.045+0000 7f2f53200640 1 --2- 172.21.15.63:0/2016804216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f54104bb0 0x7f2f54104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.045+0000 7f2f53200640 1 -- 172.21.15.63:0/2016804216 >> 172.21.15.63:0/2016804216 conn(0x7f2f540fe210 msgr2=0x7f2f54100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.045+0000 7f2f53200640 1 -- 172.21.15.63:0/2016804216 shutdown_connections 2024-05-27T07:39:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.045+0000 7f2f53200640 1 -- 172.21.15.63:0/2016804216 wait complete. 2024-05-27T07:39:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.046+0000 7f2f53200640 1 Processor -- start 2024-05-27T07:39:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.046+0000 7f2f53200640 1 -- start start 2024-05-27T07:39:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f53200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f54104bb0 0x7f2f5419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f53200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5410b610 0x7f2f5419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f53200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f5410e000 0x7f2f5419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f53200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54113b60 con 0x7f2f5410b610 2024-05-27T07:39:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f53200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541139e0 con 0x7f2f54104bb0 2024-05-27T07:39:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f53200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54113ce0 con 0x7f2f5410e000 2024-05-27T07:39:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f51400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5410b610 0x7f2f5419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f51400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5410b610 0x7f2f5419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34776/0 (socket says 172.21.15.63:34776) 2024-05-27T07:39:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f51400640 1 -- 172.21.15.63:0/1410799644 learned_addr learned my addr 172.21.15.63:0/1410799644 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f51e00640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f54104bb0 0x7f2f5419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.047+0000 7f2f52800640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f5410e000 0x7f2f5419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.048+0000 7f2f51400640 1 -- 172.21.15.63:0/1410799644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f5410e000 msgr2=0x7f2f5419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.048+0000 7f2f51400640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f5410e000 0x7f2f5419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.048+0000 7f2f51400640 1 -- 172.21.15.63:0/1410799644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f54104bb0 msgr2=0x7f2f5419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.048+0000 7f2f51400640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f54104bb0 0x7f2f5419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.048+0000 7f2f51400640 1 -- 172.21.15.63:0/1410799644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5419f380 con 0x7f2f5410b610 2024-05-27T07:39:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.048+0000 7f2f52800640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f5410e000 0x7f2f5419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.048+0000 7f2f51e00640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f54104bb0 0x7f2f5419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.048+0000 7f2f51400640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5410b610 0x7f2f5419aa80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000e9f0 tx=0x7f2f4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.048+0000 7f2f4aa00640 1 -- 172.21.15.63:0/1410799644 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000cd80 con 0x7f2f5410b610 2024-05-27T07:39:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.049+0000 7f2f4aa00640 1 -- 172.21.15.63:0/1410799644 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f400108c0 con 0x7f2f5410b610 2024-05-27T07:39:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.049+0000 7f2f53200640 1 -- 172.21.15.63:0/1410799644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5419f670 con 0x7f2f5410b610 2024-05-27T07:39:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.049+0000 7f2f4aa00640 1 -- 172.21.15.63:0/1410799644 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f400187c0 con 0x7f2f5410b610 2024-05-27T07:39:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.049+0000 7f2f53200640 1 -- 172.21.15.63:0/1410799644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f541a6ff0 con 0x7f2f5410b610 2024-05-27T07:39:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.053+0000 7f2f5fc64740 1 Processor -- start 2024-05-27T07:39:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.053+0000 7f2f5fc64740 1 -- start start 2024-05-27T07:39:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.053+0000 7f2f4aa00640 1 -- 172.21.15.63:0/1410799644 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2f40010430 con 0x7f2f5410b610 2024-05-27T07:39:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.053+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b491b0 0x559e31eeb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.053+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 0x559e31eebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.053+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31eec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.053+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559e31e5cb90 con 0x559e31b491b0 2024-05-27T07:39:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.053+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e31e4c730 con 0x559e31b202f0 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.053+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559e31e50770 con 0x559e31b284d0 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f52800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b491b0 0x559e31eeb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f52800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b491b0 0x559e31eeb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34792/0 (socket says 172.21.15.63:34792) 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f52800640 1 -- 172.21.15.63:0/1247760355 learned_addr learned my addr 172.21.15.63:0/1247760355 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f51e00640 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 0x559e31eebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f51400640 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31eec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f52800640 1 -- 172.21.15.63:0/1247760355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 msgr2=0x559e31eec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f52800640 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31eec3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f52800640 1 -- 172.21.15.63:0/1247760355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 msgr2=0x559e31eebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f52800640 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 0x559e31eebe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f52800640 1 -- 172.21.15.63:0/1247760355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e31de58b0 con 0x559e31b491b0 2024-05-27T07:39:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f51e00640 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 0x559e31eebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.054+0000 7f2f51400640 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31eec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:39:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.055+0000 7f2f52800640 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b491b0 0x559e31eeb930 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0094e0 tx=0x7f2f4c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72af124ebd3735d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.055+0000 7f2f3f000640 1 -- 172.21.15.63:0/1247760355 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c031980 con 0x559e31b491b0 2024-05-27T07:39:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.055+0000 7f2f3f000640 1 -- 172.21.15.63:0/1247760355 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f4c040070 con 0x559e31b491b0 2024-05-27T07:39:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.055+0000 7f2f3f000640 1 -- 172.21.15.63:0/1247760355 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c004400 con 0x559e31b491b0 2024-05-27T07:39:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.056+0000 7f2f4aa00640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f380b2950 0x7f2f380b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.056+0000 7f2f4aa00640 1 -- 172.21.15.63:0/1410799644 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2f400e2e50 con 0x7f2f5410b610 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.056+0000 7f2f5fc64740 1 -- 172.21.15.63:0/1247760355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b491b0 msgr2=0x559e31eeb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.056+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b491b0 0x559e31eeb930 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0094e0 tx=0x7f2f4c009a20 comp rx=0 tx=0).stop 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.056+0000 7f2f51e00640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f380b2950 0x7f2f380b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.057+0000 7f2f5fc64740 1 -- 172.21.15.63:0/1247760355 shutdown_connections 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.057+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31eec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.057+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 0x559e31eebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.057+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/1247760355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b491b0 0x559e31eeb930 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.057+0000 7f2f5fc64740 1 -- 172.21.15.63:0/1247760355 >> 172.21.15.63:0/1247760355 conn(0x559e31b3d300 msgr2=0x559e31ee8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.057+0000 7f2f5fc64740 1 -- 172.21.15.63:0/1247760355 shutdown_connections 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.057+0000 7f2f5fc64740 1 -- 172.21.15.63:0/1247760355 wait complete. 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.057+0000 7f2f51e00640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f380b2950 0x7f2f380b4e60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2f44005d00 tx=0x7f2f44005c50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.058+0000 7f2f5fc64740 1 -- start start 2024-05-27T07:39:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.058+0000 7f2f5fc64740 1 Processor -- start 2024-05-27T07:39:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.058+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 0x559e31efa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.058+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b284d0 0x559e31efa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.058+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b491b0 0x559e31f06320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.058+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559e31b32790 con 0x559e31b284d0 2024-05-27T07:39:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.058+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e31df4bc0 con 0x559e31b202f0 2024-05-27T07:39:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.058+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559e31efcff0 con 0x559e31b491b0 2024-05-27T07:39:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.058+0000 7f2f5fc64740 10 client.0 _ll_register_callbacks cb 0x559e31dd2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.059+0000 7f2f51400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b284d0 0x559e31efa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.059+0000 7f2f51400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b284d0 0x559e31efa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34808/0 (socket says 172.21.15.63:34808) 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.059+0000 7f2f51400640 1 -- 172.21.15.63:0/3710603079 learned_addr learned my addr 172.21.15.63:0/3710603079 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.059+0000 7f2f51e00640 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 0x559e31efa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.059+0000 7f2f52800640 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b491b0 0x559e31f06320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.059+0000 7f2f51400640 1 -- 172.21.15.63:0/3710603079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b491b0 msgr2=0x559e31f06320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.059+0000 7f2f51400640 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b491b0 0x559e31f06320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.059+0000 7f2f51400640 1 -- 172.21.15.63:0/3710603079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 msgr2=0x559e31efa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.059+0000 7f2f51400640 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 0x559e31efa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.059+0000 7f2f51400640 1 -- 172.21.15.63:0/3710603079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e31b61210 con 0x559e31b284d0 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.060+0000 7f2f51400640 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b284d0 0x559e31efa5a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2f400a9a60 tx=0x7f2f400a9930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add3a31c0ba73996 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.060+0000 7f2f33400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.060+0000 7f2f33400640 1 -- 172.21.15.63:0/3710603079 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f400ae110 con 0x559e31b284d0 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.060+0000 7f2f33400640 1 -- 172.21.15.63:0/3710603079 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2f400e0450 con 0x559e31b284d0 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.060+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3710603079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f4c040070 con 0x559e31b284d0 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.060+0000 7f2f33400640 1 -- 172.21.15.63:0/3710603079 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2f400e0650 con 0x559e31b284d0 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.060+0000 7f2f5fc64740 10 client.16536 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.060+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3710603079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e31c2d620 con 0x559e31b284d0 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.061+0000 7f2f33400640 1 -- 172.21.15.63:0/3710603079 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f4c040070 con 0x559e31b284d0 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.061+0000 7f2f33400640 1 -- 172.21.15.63:0/3710603079 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2f400a8a30 con 0x559e31b284d0 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.061+0000 7f2f5fc64740 10 client.16536 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.061+0000 7f2f5fc64740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.061+0000 7f2f5fc64740 20 client.16536 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.061+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e31b07d80 0x559e31f0c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.061+0000 7f2f5fc64740 4 client.16536 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.061+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3710603079 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e31e5eae0 con 0x559e31b07d80 2024-05-27T07:39:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.062+0000 7f2f51e00640 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e31b07d80 0x559e31f0c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.063+0000 7f2f33400640 10 client.16536 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.063+0000 7f2f51e00640 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e31b07d80 0x559e31f0c260 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=585d123d2b3e1f84 server_cookie=299312a947108dcb in_seq=0 out_seq=0 2024-05-27T07:39:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f33400640 1 -- 172.21.15.63:0/3710603079 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559e31e5eae0 con 0x559e31b07d80 2024-05-27T07:39:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f33400640 10 client.16536 handle_command_reply: tid=0 2024-05-27T07:39:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 client.16536 shutdown 2024-05-27T07:39:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3710603079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b284d0 msgr2=0x559e31efa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b284d0 0x559e31efa5a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2f400a9a60 tx=0x7f2f400a9930 comp rx=0 tx=0).stop 2024-05-27T07:39:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3710603079 shutdown_connections 2024-05-27T07:39:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e31b07d80 0x559e31f0c260 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b491b0 0x559e31f06320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b284d0 0x559e31efa5a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/3710603079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b202f0 0x559e31efa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f33400640 0 client.16536 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3710603079 >> 172.21.15.63:0/3710603079 conn(0x559e31b3d300 msgr2=0x559e31eee140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3710603079 shutdown_connections 2024-05-27T07:39:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.065+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3710603079 wait complete. 2024-05-27T07:39:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.066+0000 7f2f5fc64740 20 client.16536 trim_cache size 0 max 16384 2024-05-27T07:39:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.085+0000 7f2f5fc64740 1 Processor -- start 2024-05-27T07:39:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.085+0000 7f2f5fc64740 1 -- start start 2024-05-27T07:39:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31efa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b491b0 0x559e31ef3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541b1680 con 0x559e31b202f0 2024-05-27T07:39:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541aef50 con 0x559e31b491b0 2024-05-27T07:39:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541acda0 con 0x559e31b284d0 2024-05-27T07:39:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f51e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f51e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34810/0 (socket says 172.21.15.63:34810) 2024-05-27T07:39:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f51e00640 1 -- 172.21.15.63:0/3820075694 learned_addr learned my addr 172.21.15.63:0/3820075694 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f51400640 1 --2- 172.21.15.63:0/3820075694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31efa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.086+0000 7f2f52800640 1 --2- 172.21.15.63:0/3820075694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b491b0 0x559e31ef3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.087+0000 7f2f51e00640 1 -- 172.21.15.63:0/3820075694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 msgr2=0x559e31efa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.087+0000 7f2f51e00640 1 --2- 172.21.15.63:0/3820075694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31efa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.087+0000 7f2f51e00640 1 -- 172.21.15.63:0/3820075694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b491b0 msgr2=0x559e31ef3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.087+0000 7f2f51e00640 1 --2- 172.21.15.63:0/3820075694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b491b0 0x559e31ef3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.087+0000 7f2f51e00640 1 -- 172.21.15.63:0/3820075694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e31eee6b0 con 0x559e31b202f0 2024-05-27T07:39:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.088+0000 7f2f51e00640 1 --2- 172.21.15.63:0/3820075694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef58a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x559e31efccd0 tx=0x7f2f44014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd166bf7a1bef36e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.088+0000 7f2f3e600640 1 -- 172.21.15.63:0/3820075694 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44004020 con 0x559e31b202f0 2024-05-27T07:39:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.088+0000 7f2f3e600640 1 -- 172.21.15.63:0/3820075694 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f440041c0 con 0x559e31b202f0 2024-05-27T07:39:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.088+0000 7f2f3e600640 1 -- 172.21.15.63:0/3820075694 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44003850 con 0x559e31b202f0 2024-05-27T07:39:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.089+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3820075694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 msgr2=0x559e31ef58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.089+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/3820075694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef58a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x559e31efccd0 tx=0x7f2f44014000 comp rx=0 tx=0).stop 2024-05-27T07:39:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.089+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3820075694 shutdown_connections 2024-05-27T07:39:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.089+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/3820075694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b491b0 0x559e31ef3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.089+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/3820075694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31efa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.089+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/3820075694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef58a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.089+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3820075694 >> 172.21.15.63:0/3820075694 conn(0x559e31b3d300 msgr2=0x559e31ee8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.089+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3820075694 shutdown_connections 2024-05-27T07:39:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.089+0000 7f2f5fc64740 1 -- 172.21.15.63:0/3820075694 wait complete. 2024-05-27T07:39:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.090+0000 7f2f5fc64740 1 -- start start 2024-05-27T07:39:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.090+0000 7f2f5fc64740 1 Processor -- start 2024-05-27T07:39:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.091+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.091+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31ef42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.091+0000 7f2f5fc64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b491b0 0x559e31ef4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.091+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541ad410 con 0x559e31b202f0 2024-05-27T07:39:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.091+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541ae880 con 0x559e31b491b0 2024-05-27T07:39:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.091+0000 7f2f5fc64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541adae0 con 0x559e31b284d0 2024-05-27T07:39:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.091+0000 7f2f5fc64740 10 client.0 _ll_register_callbacks cb 0x7f2f400a5c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.092+0000 7f2f51e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.092+0000 7f2f51e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34816/0 (socket says 172.21.15.63:34816) 2024-05-27T07:39:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.092+0000 7f2f51e00640 1 -- 172.21.15.63:0/660285072 learned_addr learned my addr 172.21.15.63:0/660285072 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.092+0000 7f2f52800640 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b491b0 0x559e31ef4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.092+0000 7f2f51400640 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31ef42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.093+0000 7f2f51e00640 1 -- 172.21.15.63:0/660285072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 msgr2=0x559e31ef42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.093+0000 7f2f51e00640 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31ef42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.093+0000 7f2f51e00640 1 -- 172.21.15.63:0/660285072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b491b0 msgr2=0x559e31ef4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.093+0000 7f2f51e00640 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b491b0 0x559e31ef4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.093+0000 7f2f51e00640 1 -- 172.21.15.63:0/660285072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e31f01280 con 0x559e31b202f0 2024-05-27T07:39:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.093+0000 7f2f51e00640 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef3d90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2f4402c040 tx=0x7f2f44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad5b24bfc03c154 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.093+0000 7f2f33400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.094+0000 7f2f33400640 1 -- 172.21.15.63:0/660285072 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44038070 con 0x559e31b202f0 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.094+0000 7f2f33400640 1 -- 172.21.15.63:0/660285072 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2f44029670 con 0x559e31b202f0 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.094+0000 7f2f5fc64740 1 -- 172.21.15.63:0/660285072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f440041c0 con 0x559e31b202f0 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.094+0000 7f2f33400640 1 -- 172.21.15.63:0/660285072 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2f44044050 con 0x559e31b202f0 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.094+0000 7f2f5fc64740 10 client.16542 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.094+0000 7f2f5fc64740 1 -- 172.21.15.63:0/660285072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e31f024d0 con 0x559e31b202f0 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.094+0000 7f2f33400640 1 -- 172.21.15.63:0/660285072 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f440041c0 con 0x559e31b202f0 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.095+0000 7f2f33400640 1 -- 172.21.15.63:0/660285072 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2f44003cf0 con 0x559e31b202f0 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.095+0000 7f2f5fc64740 10 client.16542 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.095+0000 7f2f5fc64740 10 client.16542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.095+0000 7f2f5fc64740 20 client.16542 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.095+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e31add1c0 0x559e31f76920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.095+0000 7f2f5fc64740 4 client.16542 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:39:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.095+0000 7f2f5fc64740 1 -- 172.21.15.63:0/660285072 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e31e582c0 con 0x559e31add1c0 2024-05-27T07:39:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.096+0000 7f2f51400640 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e31add1c0 0x559e31f76920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.097+0000 7f2f33400640 10 client.16542 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.097+0000 7f2f51400640 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e31add1c0 0x559e31f76920 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26cb02ef993012d2 server_cookie=fb7482554e65d1d in_seq=0 out_seq=0 2024-05-27T07:39:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.139+0000 7f2f33400640 1 -- 172.21.15.63:0/660285072 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559e31e582c0 con 0x559e31add1c0 2024-05-27T07:39:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.139+0000 7f2f33400640 10 client.16542 handle_command_reply: tid=0 2024-05-27T07:39:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.139+0000 7f2f5fc64740 1 client.16542 shutdown 2024-05-27T07:39:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 1 -- 172.21.15.63:0/660285072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 msgr2=0x559e31ef3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef3d90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2f4402c040 tx=0x7f2f44004730 comp rx=0 tx=0).stop 2024-05-27T07:39:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 1 -- 172.21.15.63:0/660285072 shutdown_connections 2024-05-27T07:39:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e31add1c0 0x559e31f76920 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e31b491b0 0x559e31ef4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e31b284d0 0x559e31ef42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 1 --2- 172.21.15.63:0/660285072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e31b202f0 0x559e31ef3d90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f33400640 0 client.16542 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 1 -- 172.21.15.63:0/660285072 >> 172.21.15.63:0/660285072 conn(0x559e31b3d300 msgr2=0x559e31ee7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 1 -- 172.21.15.63:0/660285072 shutdown_connections 2024-05-27T07:39:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 1 -- 172.21.15.63:0/660285072 wait complete. 2024-05-27T07:39:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.140+0000 7f2f5fc64740 20 client.16542 trim_cache size 0 max 16384 2024-05-27T07:39:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:39:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:39:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:39:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 -- 172.21.15.63:0/1410799644 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f380b2950 msgr2=0x7f2f380b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f380b2950 0x7f2f380b4e60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2f44005d00 tx=0x7f2f44005c50 comp rx=0 tx=0).stop 2024-05-27T07:39:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 -- 172.21.15.63:0/1410799644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5410b610 msgr2=0x7f2f5419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5410b610 0x7f2f5419aa80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000e9f0 tx=0x7f2f4000eeb0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 -- 172.21.15.63:0/1410799644 shutdown_connections 2024-05-27T07:39:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f380b2950 0x7f2f380b4e60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f5410e000 0x7f2f5419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5410b610 0x7f2f5419aa80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 --2- 172.21.15.63:0/1410799644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f54104bb0 0x7f2f5419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 -- 172.21.15.63:0/1410799644 >> 172.21.15.63:0/1410799644 conn(0x7f2f540fe210 msgr2=0x7f2f5410e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 -- 172.21.15.63:0/1410799644 shutdown_connections 2024-05-27T07:39:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:32.141+0000 7f2f53200640 1 -- 172.21.15.63:0/1410799644 wait complete. 2024-05-27T07:39:33.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.423+0000 7f1d95a00700 1 -- 172.21.15.63:0/3226293629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d901052c0 msgr2=0x7f1d901056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:33.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.423+0000 7f1d95a00700 1 --2- 172.21.15.63:0/3226293629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d901052c0 0x7f1d901056d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009990 tx=0x7f1d80009ca0 comp rx=0 tx=0).stop 2024-05-27T07:39:33.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.424+0000 7f1d95a00700 1 -- 172.21.15.63:0/3226293629 shutdown_connections 2024-05-27T07:39:33.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.424+0000 7f1d95a00700 1 --2- 172.21.15.63:0/3226293629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d9010e730 0x7f1d90110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:33.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.424+0000 7f1d95a00700 1 --2- 172.21.15.63:0/3226293629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d901052c0 0x7f1d901056d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:33.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.424+0000 7f1d95a00700 1 --2- 172.21.15.63:0/3226293629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d90105c10 0x7f1d9010e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:33.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.424+0000 7f1d95a00700 1 -- 172.21.15.63:0/3226293629 >> 172.21.15.63:0/3226293629 conn(0x7f1d90078af0 msgr2=0x7f1d90078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:33.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.424+0000 7f1d95a00700 1 -- 172.21.15.63:0/3226293629 shutdown_connections 2024-05-27T07:39:33.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.424+0000 7f1d95a00700 1 -- 172.21.15.63:0/3226293629 wait complete. 2024-05-27T07:39:33.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.426+0000 7f1d95a00700 1 Processor -- start 2024-05-27T07:39:33.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.426+0000 7f1d95a00700 1 -- start start 2024-05-27T07:39:33.425 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:33 smithi063 ceph-mon[31613]: pgmap v182: 129 pgs: 129 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:39:33.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.427+0000 7f1d95a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d901052c0 0x7f1d901a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:33.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.427+0000 7f1d95a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d90105c10 0x7f1d901a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:33.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.427+0000 7f1d95a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d9010e730 0x7f1d9019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:33.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.427+0000 7f1d95a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d90114180 con 0x7f1d90105c10 2024-05-27T07:39:33.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.427+0000 7f1d95a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d90114000 con 0x7f1d901052c0 2024-05-27T07:39:33.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.427+0000 7f1d95a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1d90114300 con 0x7f1d9010e730 2024-05-27T07:39:33.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.427+0000 7f1d8f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d90105c10 0x7f1d901a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:33.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.427+0000 7f1d8f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d90105c10 0x7f1d901a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34826/0 (socket says 172.21.15.63:34826) 2024-05-27T07:39:33.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.427+0000 7f1d8f400700 1 -- 172.21.15.63:0/3751848803 learned_addr learned my addr 172.21.15.63:0/3751848803 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:33.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.427+0000 7f1d8fe00700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d901052c0 0x7f1d901a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.428+0000 7f1d95000700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d9010e730 0x7f1d9019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.428+0000 7f1d8f400700 1 -- 172.21.15.63:0/3751848803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d9010e730 msgr2=0x7f1d9019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.428+0000 7f1d8f400700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d9010e730 0x7f1d9019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.428+0000 7f1d8f400700 1 -- 172.21.15.63:0/3751848803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d901052c0 msgr2=0x7f1d901a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.428+0000 7f1d8f400700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d901052c0 0x7f1d901a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.428+0000 7f1d8f400700 1 -- 172.21.15.63:0/3751848803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9019b070 con 0x7f1d90105c10 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.429+0000 7f1d8f400700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d90105c10 0x7f1d901a0ce0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800d980 tx=0x7f1d7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.429+0000 7f1d8cc00700 1 -- 172.21.15.63:0/3751848803 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78017070 con 0x7f1d90105c10 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.429+0000 7f1d95a00700 1 -- 172.21.15.63:0/3751848803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9019b360 con 0x7f1d90105c10 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.429+0000 7f1d8cc00700 1 -- 172.21.15.63:0/3751848803 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d7800fd20 con 0x7f1d90105c10 2024-05-27T07:39:33.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.429+0000 7f1d8cc00700 1 -- 172.21.15.63:0/3751848803 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7800e3b0 con 0x7f1d90105c10 2024-05-27T07:39:33.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.430+0000 7f1d95a00700 1 -- 172.21.15.63:0/3751848803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d90102ee0 con 0x7f1d90105c10 2024-05-27T07:39:33.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.431+0000 7f1d95a00700 1 -- 172.21.15.63:0/3751848803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9006c8d0 con 0x7f1d90105c10 2024-05-27T07:39:33.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.434+0000 7f1d8cc00700 1 -- 172.21.15.63:0/3751848803 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1d7800f890 con 0x7f1d90105c10 2024-05-27T07:39:33.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.436+0000 7f1d8cc00700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d740b29f0 0x7f1d740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:33.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.436+0000 7f1d8cc00700 1 -- 172.21.15.63:0/3751848803 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f1d780e2470 con 0x7f1d90105c10 2024-05-27T07:39:33.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.436+0000 7f1d8fe00700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d740b29f0 0x7f1d740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:33.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.437+0000 7f1d8fe00700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d740b29f0 0x7f1d740b4f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1d9019bb00 tx=0x7f1d8000b540 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:33.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.443+0000 7f1d8cc00700 1 -- 172.21.15.63:0/3751848803 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d780a9a80 con 0x7f1d90105c10 2024-05-27T07:39:33.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:33 smithi086 ceph-mon[34687]: pgmap v182: 129 pgs: 129 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:39:33.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:33 smithi149 ceph-mon[35408]: pgmap v182: 129 pgs: 129 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:39:33.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.755+0000 7f1d95a00700 1 -- 172.21.15.63:0/3751848803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d9019bef0 con 0x7f1d90105c10 2024-05-27T07:39:33.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.757+0000 7f1d8cc00700 1 -- 172.21.15.63:0/3751848803 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f1d780a9380 con 0x7f1d90105c10 2024-05-27T07:39:33.755 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:39:33.755 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:39:33.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.760+0000 7f1d95a00700 1 -- 172.21.15.63:0/3751848803 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d740b29f0 msgr2=0x7f1d740b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:33.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.760+0000 7f1d95a00700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d740b29f0 0x7f1d740b4f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1d9019bb00 tx=0x7f1d8000b540 comp rx=0 tx=0).stop 2024-05-27T07:39:33.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.760+0000 7f1d95a00700 1 -- 172.21.15.63:0/3751848803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d90105c10 msgr2=0x7f1d901a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:33.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.760+0000 7f1d95a00700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d90105c10 0x7f1d901a0ce0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800d980 tx=0x7f1d7800dc90 comp rx=0 tx=0).stop 2024-05-27T07:39:33.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.760+0000 7f1d95a00700 1 -- 172.21.15.63:0/3751848803 shutdown_connections 2024-05-27T07:39:33.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.760+0000 7f1d95a00700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d740b29f0 0x7f1d740b4f00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:33.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.761+0000 7f1d95a00700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d9010e730 0x7f1d9019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:33.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.761+0000 7f1d95a00700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d901052c0 0x7f1d901a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:33.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.761+0000 7f1d95a00700 1 --2- 172.21.15.63:0/3751848803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d90105c10 0x7f1d901a0ce0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:33.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.761+0000 7f1d95a00700 1 -- 172.21.15.63:0/3751848803 >> 172.21.15.63:0/3751848803 conn(0x7f1d90078af0 msgr2=0x7f1d900fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:33.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.761+0000 7f1d95a00700 1 -- 172.21.15.63:0/3751848803 shutdown_connections 2024-05-27T07:39:33.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:33.761+0000 7f1d95a00700 1 -- 172.21.15.63:0/3751848803 wait complete. 2024-05-27T07:39:33.760 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:39:34.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:34 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3751848803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:39:34.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:34 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3751848803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:39:34.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:34 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3751848803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:39:34.733 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:39:35.238 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:39:35.573 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:35 smithi063 ceph-mon[31613]: pgmap v183: 129 pgs: 129 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.0 MiB/s wr, 6 op/s 2024-05-27T07:39:35.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:35 smithi086 ceph-mon[34687]: pgmap v183: 129 pgs: 129 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.0 MiB/s wr, 6 op/s 2024-05-27T07:39:35.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:35 smithi149 ceph-mon[35408]: pgmap v183: 129 pgs: 129 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.0 MiB/s wr, 6 op/s 2024-05-27T07:39:37.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:37 smithi063 ceph-mon[31613]: pgmap v184: 129 pgs: 129 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.0 MiB/s wr, 6 op/s 2024-05-27T07:39:37.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:37 smithi086 ceph-mon[34687]: pgmap v184: 129 pgs: 129 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.0 MiB/s wr, 6 op/s 2024-05-27T07:39:37.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:37 smithi149 ceph-mon[35408]: pgmap v184: 129 pgs: 129 active+clean; 1007 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.0 MiB/s wr, 6 op/s 2024-05-27T07:39:37.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.738+0000 7f6404400700 1 -- 172.21.15.63:0/3750825942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc110b70 msgr2=0x7f63fc112fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:37.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.738+0000 7f6404400700 1 --2- 172.21.15.63:0/3750825942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc110b70 0x7f63fc112fd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f63f400b0a0 tx=0x7f63f400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.738+0000 7f6404400700 1 -- 172.21.15.63:0/3750825942 shutdown_connections 2024-05-27T07:39:37.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.738+0000 7f6404400700 1 --2- 172.21.15.63:0/3750825942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc110b70 0x7f63fc112fd0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.738+0000 7f6404400700 1 --2- 172.21.15.63:0/3750825942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63fc076e30 0x7f63fc077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.738+0000 7f6404400700 1 --2- 172.21.15.63:0/3750825942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63fc077780 0x7f63fc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.738+0000 7f6404400700 1 -- 172.21.15.63:0/3750825942 >> 172.21.15.63:0/3750825942 conn(0x7f63fc0ffe90 msgr2=0x7f63fc1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:37.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.738+0000 7f6404400700 1 -- 172.21.15.63:0/3750825942 shutdown_connections 2024-05-27T07:39:37.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.738+0000 7f6404400700 1 -- 172.21.15.63:0/3750825942 wait complete. 2024-05-27T07:39:37.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.739+0000 7f6404400700 1 Processor -- start 2024-05-27T07:39:37.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.739+0000 7f6404400700 1 -- start start 2024-05-27T07:39:37.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.740+0000 7f6404400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63fc076e30 0x7f63fc19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.740+0000 7f6404400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc077780 0x7f63fc19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.740+0000 7f6404400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63fc110b70 0x7f63fc1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.740+0000 7f6404400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc116500 con 0x7f63fc077780 2024-05-27T07:39:37.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.740+0000 7f6404400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc116380 con 0x7f63fc110b70 2024-05-27T07:39:37.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.740+0000 7f6404400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc116680 con 0x7f63fc076e30 2024-05-27T07:39:37.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.740+0000 7f6401000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc077780 0x7f63fc19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.740+0000 7f6401000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc077780 0x7f63fc19d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34832/0 (socket says 172.21.15.63:34832) 2024-05-27T07:39:37.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.740+0000 7f6401000700 1 -- 172.21.15.63:0/291162374 learned_addr learned my addr 172.21.15.63:0/291162374 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:37.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.741+0000 7f6401a00700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63fc076e30 0x7f63fc19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.741+0000 7f6402400700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63fc110b70 0x7f63fc1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.741+0000 7f6401000700 1 -- 172.21.15.63:0/291162374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63fc076e30 msgr2=0x7f63fc19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:37.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.741+0000 7f6401000700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63fc076e30 0x7f63fc19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.741+0000 7f6401000700 1 -- 172.21.15.63:0/291162374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63fc110b70 msgr2=0x7f63fc1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:37.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.741+0000 7f6401000700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63fc110b70 0x7f63fc1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.741+0000 7f6401000700 1 -- 172.21.15.63:0/291162374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc1a1da0 con 0x7f63fc077780 2024-05-27T07:39:37.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.741+0000 7f6402400700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63fc110b70 0x7f63fc1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:39:37.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.742+0000 7f6401000700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc077780 0x7f63fc19d470 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f63e800d950 tx=0x7f63e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:37.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.742+0000 7f63fa000700 1 -- 172.21.15.63:0/291162374 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8016070 con 0x7f63fc077780 2024-05-27T07:39:37.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.742+0000 7f6404400700 1 -- 172.21.15.63:0/291162374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63fc1a2030 con 0x7f63fc077780 2024-05-27T07:39:37.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.742+0000 7f63fa000700 1 -- 172.21.15.63:0/291162374 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63e8009dd0 con 0x7f63fc077780 2024-05-27T07:39:37.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.742+0000 7f63fa000700 1 -- 172.21.15.63:0/291162374 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e800f3c0 con 0x7f63fc077780 2024-05-27T07:39:37.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.742+0000 7f6404400700 1 -- 172.21.15.63:0/291162374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63fc1a99b0 con 0x7f63fc077780 2024-05-27T07:39:37.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.747+0000 7f63fa000700 1 -- 172.21.15.63:0/291162374 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f63e8019070 con 0x7f63fc077780 2024-05-27T07:39:37.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.748+0000 7f6412ed6b80 1 Processor -- start 2024-05-27T07:39:37.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.749+0000 7f6412ed6b80 1 -- start start 2024-05-27T07:39:37.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.749+0000 7f63fa000700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f63e40b29f0 0x7f63e40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.749+0000 7f63fa000700 1 -- 172.21.15.63:0/291162374 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f63e80e25e0 con 0x7f63fc077780 2024-05-27T07:39:37.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.749+0000 7f6402400700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f63e40b29f0 0x7f63e40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.749+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805559f0 0x559b805e8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.749+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b80469e40 0x559b805e8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.749+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 0x559b805f5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.749+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559b80516e40 con 0x559b805559f0 2024-05-27T07:39:37.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.749+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559b805069e0 con 0x559b805e8e70 2024-05-27T07:39:37.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.749+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559b8050aa20 con 0x559b80469e40 2024-05-27T07:39:37.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.750+0000 7f6401a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805559f0 0x559b805e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.750+0000 7f6401a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805559f0 0x559b805e8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34840/0 (socket says 172.21.15.63:34840) 2024-05-27T07:39:37.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.750+0000 7f6401a00700 1 -- 172.21.15.63:0/3199345639 learned_addr learned my addr 172.21.15.63:0/3199345639 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:37.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.750+0000 7f6401000700 1 --2- 172.21.15.63:0/3199345639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b80469e40 0x559b805e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.750+0000 7f6402400700 1 --2- 172.21.15.63:0/3199345639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 0x559b805f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.750+0000 7f6402400700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f63e40b29f0 0x7f63e40b4f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f63f400b070 tx=0x7f63f4009200 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:37.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.750+0000 7f6401a00700 1 -- 172.21.15.63:0/3199345639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b80469e40 msgr2=0x559b805e8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:37.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.750+0000 7f6401a00700 1 --2- 172.21.15.63:0/3199345639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b80469e40 0x559b805e8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.750+0000 7f6401a00700 1 -- 172.21.15.63:0/3199345639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 msgr2=0x559b805f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:37.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.750+0000 7f6401a00700 1 --2- 172.21.15.63:0/3199345639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 0x559b805f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.751+0000 7f6401a00700 1 -- 172.21.15.63:0/3199345639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b804cf420 con 0x559b805559f0 2024-05-27T07:39:37.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.751+0000 7f6402400700 1 --2- 172.21.15.63:0/3199345639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 0x559b805f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:37.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.751+0000 7f6401a00700 1 --2- 172.21.15.63:0/3199345639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805559f0 0x559b805e8330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f63f000bd70 tx=0x7f63f00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ee236b34f9565e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:37.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.751+0000 7f63eea00700 1 -- 172.21.15.63:0/3199345639 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0010e10 con 0x559b805559f0 2024-05-27T07:39:37.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.751+0000 7f63eea00700 1 -- 172.21.15.63:0/3199345639 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63f001c910 con 0x559b805559f0 2024-05-27T07:39:37.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.751+0000 7f63eea00700 1 -- 172.21.15.63:0/3199345639 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0011870 con 0x559b805559f0 2024-05-27T07:39:37.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.753+0000 7f6412ed6b80 1 -- 172.21.15.63:0/3199345639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805559f0 msgr2=0x559b805e8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:37.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.753+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/3199345639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805559f0 0x559b805e8330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f63f000bd70 tx=0x7f63f00106b0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.753+0000 7f6412ed6b80 1 -- 172.21.15.63:0/3199345639 shutdown_connections 2024-05-27T07:39:37.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.753+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/3199345639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 0x559b805f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.753+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/3199345639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805559f0 0x559b805e8330 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.753+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/3199345639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b80469e40 0x559b805e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.753+0000 7f6412ed6b80 1 -- 172.21.15.63:0/3199345639 >> 172.21.15.63:0/3199345639 conn(0x559b80503390 msgr2=0x559b805e44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:37.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.753+0000 7f6412ed6b80 1 -- 172.21.15.63:0/3199345639 shutdown_connections 2024-05-27T07:39:37.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.753+0000 7f6412ed6b80 1 -- 172.21.15.63:0/3199345639 wait complete. 2024-05-27T07:39:37.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.754+0000 7f6412ed6b80 1 -- start start 2024-05-27T07:39:37.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.754+0000 7f6412ed6b80 1 Processor -- start 2024-05-27T07:39:37.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b80469e40 0x559b805f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 0x559b805f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805f3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559b8040cfa0 con 0x559b805e8e70 2024-05-27T07:39:37.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559b804a7760 con 0x559b80469e40 2024-05-27T07:39:37.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559b805f5b80 con 0x559b805559f0 2024-05-27T07:39:37.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6412ed6b80 10 client.0 _ll_register_callbacks cb 0x559b804b0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:37.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6402400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6402400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34852/0 (socket says 172.21.15.63:34852) 2024-05-27T07:39:37.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6402400700 1 -- 172.21.15.63:0/2893622922 learned_addr learned my addr 172.21.15.63:0/2893622922 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:37.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.755+0000 7f6401000700 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 0x559b805f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.756+0000 7f6401a00700 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b80469e40 0x559b805f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.756+0000 7f6402400700 1 -- 172.21.15.63:0/2893622922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 msgr2=0x559b805f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:37.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.756+0000 7f6402400700 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 0x559b805f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.756+0000 7f6402400700 1 -- 172.21.15.63:0/2893622922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b80469e40 msgr2=0x559b805f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:37.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.756+0000 7f6402400700 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b80469e40 0x559b805f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.756+0000 7f6402400700 1 -- 172.21.15.63:0/2893622922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b804568d0 con 0x559b805e8e70 2024-05-27T07:39:37.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.756+0000 7f6401000700 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 0x559b805f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:37.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.756+0000 7f6401a00700 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b80469e40 0x559b805f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.756+0000 7f6402400700 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805f3100 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f63f4004e80 tx=0x7f63f400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d377375ca1cdbfa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.756+0000 7f63e3400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.757+0000 7f63e3400700 1 -- 172.21.15.63:0/2893622922 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4024070 con 0x559b805e8e70 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.757+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2893622922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f001c910 con 0x559b805e8e70 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.757+0000 7f63e3400700 1 -- 172.21.15.63:0/2893622922 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f63f401d860 con 0x559b805e8e70 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.757+0000 7f63e3400700 1 -- 172.21.15.63:0/2893622922 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f63f401da60 con 0x559b805e8e70 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.757+0000 7f6412ed6b80 10 client.16560 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.757+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2893622922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559b804d7060 con 0x559b805e8e70 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.758+0000 7f63e3400700 1 -- 172.21.15.63:0/2893622922 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63f001c910 con 0x559b805e8e70 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.758+0000 7f63e3400700 1 -- 172.21.15.63:0/2893622922 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f63f4011070 con 0x559b805e8e70 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.758+0000 7f6412ed6b80 10 client.16560 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.758+0000 7f6412ed6b80 10 client.16560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:39:37.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.758+0000 7f6412ed6b80 20 client.16560 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:37.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.758+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x559b80602ed0 0x559b80605380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.758+0000 7f6412ed6b80 4 client.16560 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:37.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.758+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2893622922 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b80518d90 con 0x559b80602ed0 2024-05-27T07:39:37.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.759+0000 7f6401a00700 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x559b80602ed0 0x559b80605380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.760+0000 7f6401a00700 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x559b80602ed0 0x559b80605380 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4059c2f488eee9d server_cookie=9510cf52b0f6907 in_seq=0 out_seq=0 2024-05-27T07:39:37.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.760+0000 7f63e3400700 10 client.16560 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:39:37.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.762+0000 7f63e3400700 1 -- 172.21.15.63:0/2893622922 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559b80518d90 con 0x559b80602ed0 2024-05-27T07:39:37.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.762+0000 7f63e3400700 10 client.16560 handle_command_reply: tid=0 2024-05-27T07:39:37.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.762+0000 7f6412ed6b80 1 client.16560 shutdown 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.762+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2893622922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 msgr2=0x559b805f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.762+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805f3100 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f63f4004e80 tx=0x7f63f400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.762+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2893622922 shutdown_connections 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.762+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x559b80602ed0 0x559b80605380 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.762+0000 7f63e3400700 0 client.16560 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.762+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805f3100 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.763+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b80469e40 0x559b805f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.763+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/2893622922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 0x559b805f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.763+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2893622922 >> 172.21.15.63:0/2893622922 conn(0x559b80503390 msgr2=0x559b805e8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.763+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2893622922 shutdown_connections 2024-05-27T07:39:37.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.763+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2893622922 wait complete. 2024-05-27T07:39:37.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.763+0000 7f6412ed6b80 20 client.16560 trim_cache size 0 max 16384 2024-05-27T07:39:37.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.783+0000 7f6412ed6b80 1 Processor -- start 2024-05-27T07:39:37.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.784+0000 7f6412ed6b80 1 -- start start 2024-05-27T07:39:37.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.784+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b80469e40 0x559b805edef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.784+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 0x559b805e8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.784+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 0x559b805fec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.784+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1b4f30 con 0x559b80469e40 2024-05-27T07:39:37.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.784+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1b27c0 con 0x559b805e8e70 2024-05-27T07:39:37.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.784+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1b0610 con 0x559b805559f0 2024-05-27T07:39:37.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.784+0000 7f6401a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b80469e40 0x559b805edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.784+0000 7f6401a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b80469e40 0x559b805edef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34860/0 (socket says 172.21.15.63:34860) 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.784+0000 7f6401a00700 1 -- 172.21.15.63:0/2194572671 learned_addr learned my addr 172.21.15.63:0/2194572671 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.785+0000 7f6401000700 1 --2- 172.21.15.63:0/2194572671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 0x559b805e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.785+0000 7f6402400700 1 --2- 172.21.15.63:0/2194572671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 0x559b805fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.785+0000 7f6401a00700 1 -- 172.21.15.63:0/2194572671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 msgr2=0x559b805e8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.785+0000 7f6401a00700 1 --2- 172.21.15.63:0/2194572671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 0x559b805e8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.785+0000 7f6401a00700 1 -- 172.21.15.63:0/2194572671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 msgr2=0x559b805fec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.785+0000 7f6401a00700 1 --2- 172.21.15.63:0/2194572671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 0x559b805fec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.785+0000 7f6401a00700 1 -- 172.21.15.63:0/2194572671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b805e0240 con 0x559b80469e40 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.785+0000 7f6401a00700 1 --2- 172.21.15.63:0/2194572671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b80469e40 0x559b805edef0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f63f0011010 tx=0x7f63f0005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848f2e2c960671e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.786+0000 7f63ee000700 1 -- 172.21.15.63:0/2194572671 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0005240 con 0x559b80469e40 2024-05-27T07:39:37.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.786+0000 7f63ee000700 1 -- 172.21.15.63:0/2194572671 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63f0003680 con 0x559b80469e40 2024-05-27T07:39:37.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.786+0000 7f63ee000700 1 -- 172.21.15.63:0/2194572671 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f001c760 con 0x559b80469e40 2024-05-27T07:39:37.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.787+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2194572671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b80469e40 msgr2=0x559b805edef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:37.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.787+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/2194572671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b80469e40 0x559b805edef0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f63f0011010 tx=0x7f63f0005d20 comp rx=0 tx=0).stop 2024-05-27T07:39:37.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.787+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2194572671 shutdown_connections 2024-05-27T07:39:37.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.787+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/2194572671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805e8e70 0x559b805fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.787+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/2194572671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b80469e40 0x559b805edef0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.787+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/2194572671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b805559f0 0x559b805e8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.787+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2194572671 >> 172.21.15.63:0/2194572671 conn(0x559b80503390 msgr2=0x559b805e8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:37.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.787+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2194572671 shutdown_connections 2024-05-27T07:39:37.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.787+0000 7f6412ed6b80 1 -- 172.21.15.63:0/2194572671 wait complete. 2024-05-27T07:39:37.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.788+0000 7f6412ed6b80 1 -- start start 2024-05-27T07:39:37.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.788+0000 7f6412ed6b80 1 Processor -- start 2024-05-27T07:39:37.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.788+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b80469e40 0x559b805ec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.788+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805559f0 0x559b805ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.789+0000 7f6412ed6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.789+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1b0c80 con 0x559b805e8e70 2024-05-27T07:39:37.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.789+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1b20f0 con 0x559b805559f0 2024-05-27T07:39:37.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.789+0000 7f6412ed6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1b1350 con 0x559b80469e40 2024-05-27T07:39:37.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.789+0000 7f6412ed6b80 10 client.0 _ll_register_callbacks cb 0x559b804b0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:37.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.789+0000 7f6402400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.789+0000 7f6401000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805559f0 0x559b805ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.789+0000 7f6402400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805ecfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34868/0 (socket says 172.21.15.63:34868) 2024-05-27T07:39:37.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.789+0000 7f6402400700 1 -- 172.21.15.63:0/1899564590 learned_addr learned my addr 172.21.15.63:0/1899564590 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:37.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.789+0000 7f6401a00700 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b80469e40 0x559b805ec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.790+0000 7f6402400700 1 -- 172.21.15.63:0/1899564590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b80469e40 msgr2=0x559b805ec560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.790+0000 7f6402400700 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b80469e40 0x559b805ec560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.790+0000 7f6402400700 1 -- 172.21.15.63:0/1899564590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805559f0 msgr2=0x559b805ecaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.790+0000 7f6402400700 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805559f0 0x559b805ecaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.790+0000 7f6402400700 1 -- 172.21.15.63:0/1899564590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b805f9f40 con 0x559b805e8e70 2024-05-27T07:39:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.790+0000 7f6402400700 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805ecfe0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000f80 tx=0x7f63f4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee2b250eec75fbf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.790+0000 7f63e3400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.790+0000 7f63e3400700 1 -- 172.21.15.63:0/1899564590 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4040060 con 0x559b805e8e70 2024-05-27T07:39:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.790+0000 7f63e3400700 1 -- 172.21.15.63:0/1899564590 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f63f4032440 con 0x559b805e8e70 2024-05-27T07:39:37.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.791+0000 7f6412ed6b80 1 -- 172.21.15.63:0/1899564590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f0003680 con 0x559b805e8e70 2024-05-27T07:39:37.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.791+0000 7f63e3400700 1 -- 172.21.15.63:0/1899564590 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f63f4047050 con 0x559b805e8e70 2024-05-27T07:39:37.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.791+0000 7f6412ed6b80 10 client.16566 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:37.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.791+0000 7f6412ed6b80 1 -- 172.21.15.63:0/1899564590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559b805fb180 con 0x559b805e8e70 2024-05-27T07:39:37.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.791+0000 7f63e3400700 1 -- 172.21.15.63:0/1899564590 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63f0003680 con 0x559b805e8e70 2024-05-27T07:39:37.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.791+0000 7f63e3400700 1 -- 172.21.15.63:0/1899564590 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f63f4011040 con 0x559b805e8e70 2024-05-27T07:39:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.792+0000 7f6412ed6b80 10 client.16566 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.792+0000 7f6412ed6b80 10 client.16566 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:39:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.792+0000 7f6412ed6b80 20 client.16566 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.792+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x559b8066da50 0x559b8066fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.792+0000 7f6412ed6b80 4 client.16566 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:39:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.792+0000 7f6412ed6b80 1 -- 172.21.15.63:0/1899564590 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b805e3120 con 0x559b8066da50 2024-05-27T07:39:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.792+0000 7f6401a00700 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x559b8066da50 0x559b8066fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.793+0000 7f6401a00700 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x559b8066da50 0x559b8066fe70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f5760e265d69cd5 server_cookie=74f196436dbbba36 in_seq=0 out_seq=0 2024-05-27T07:39:37.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.793+0000 7f63e3400700 10 client.16566 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.794+0000 7f63e3400700 1 -- 172.21.15.63:0/1899564590 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b805e3120 con 0x559b8066da50 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.794+0000 7f63e3400700 10 client.16566 handle_command_reply: tid=0 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.794+0000 7f6412ed6b80 1 client.16566 shutdown 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.794+0000 7f6412ed6b80 1 -- 172.21.15.63:0/1899564590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 msgr2=0x559b805ecfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.794+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805ecfe0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000f80 tx=0x7f63f4005f10 comp rx=0 tx=0).stop 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.794+0000 7f6412ed6b80 1 -- 172.21.15.63:0/1899564590 shutdown_connections 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.794+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x559b8066da50 0x559b8066fe70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.795+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b805e8e70 0x559b805ecfe0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.795+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b80469e40 0x559b805ec560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.795+0000 7f6412ed6b80 1 --2- 172.21.15.63:0/1899564590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559b805559f0 0x559b805ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.795+0000 7f63e3400700 0 client.16566 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.795+0000 7f6412ed6b80 1 -- 172.21.15.63:0/1899564590 >> 172.21.15.63:0/1899564590 conn(0x559b80503390 msgr2=0x559b805e9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:37.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.795+0000 7f6412ed6b80 1 -- 172.21.15.63:0/1899564590 shutdown_connections 2024-05-27T07:39:37.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.795+0000 7f6412ed6b80 1 -- 172.21.15.63:0/1899564590 wait complete. 2024-05-27T07:39:37.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.795+0000 7f6412ed6b80 20 client.16566 trim_cache size 0 max 16384 2024-05-27T07:39:37.797 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.796+0000 7f6404400700 1 -- 172.21.15.63:0/291162374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f63e40b29f0 msgr2=0x7f63e40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.796+0000 7f6404400700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f63e40b29f0 0x7f63e40b4f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f63f400b070 tx=0x7f63f4009200 comp rx=0 tx=0).stop 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.796+0000 7f6404400700 1 -- 172.21.15.63:0/291162374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc077780 msgr2=0x7f63fc19d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.796+0000 7f6404400700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc077780 0x7f63fc19d470 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f63e800d950 tx=0x7f63e800dc60 comp rx=0 tx=0).stop 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.796+0000 7f6404400700 1 -- 172.21.15.63:0/291162374 shutdown_connections 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.796+0000 7f6404400700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f63e40b29f0 0x7f63e40b4f00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.796+0000 7f6404400700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63fc110b70 0x7f63fc1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.796+0000 7f6404400700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f63fc076e30 0x7f63fc19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.796+0000 7f6404400700 1 --2- 172.21.15.63:0/291162374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc077780 0x7f63fc19d470 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.796+0000 7f6404400700 1 -- 172.21.15.63:0/291162374 >> 172.21.15.63:0/291162374 conn(0x7f63fc0ffe90 msgr2=0x7f63fc100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.797+0000 7f6404400700 1 -- 172.21.15.63:0/291162374 shutdown_connections 2024-05-27T07:39:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:37.797+0000 7f6404400700 1 -- 172.21.15.63:0/291162374 wait complete. 2024-05-27T07:39:38.795 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:39:39.284 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:39:39.649 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:39 smithi063 ceph-mon[31613]: pgmap v185: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:39:39.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:39 smithi086 ceph-mon[34687]: pgmap v185: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:39:39.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:39 smithi149 ceph-mon[35408]: pgmap v185: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:39:41.429 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:41 smithi063 ceph-mon[31613]: pgmap v186: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:39:41.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:41 smithi086 ceph-mon[34687]: pgmap v186: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:39:41.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:41 smithi149 ceph-mon[35408]: pgmap v186: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:39:41.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.750+0000 7f2b39c00700 1 -- 172.21.15.63:0/210557602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34105bd0 msgr2=0x7f2b3410e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:41.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.750+0000 7f2b39c00700 1 --2- 172.21.15.63:0/210557602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34105bd0 0x7f2b3410e120 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009990 tx=0x7f2b20009ca0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.750+0000 7f2b39c00700 1 -- 172.21.15.63:0/210557602 shutdown_connections 2024-05-27T07:39:41.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.750+0000 7f2b39c00700 1 --2- 172.21.15.63:0/210557602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b3410e6f0 0x7f2b34110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.750+0000 7f2b39c00700 1 --2- 172.21.15.63:0/210557602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b34105280 0x7f2b34105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.750+0000 7f2b39c00700 1 --2- 172.21.15.63:0/210557602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34105bd0 0x7f2b3410e120 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.750+0000 7f2b39c00700 1 -- 172.21.15.63:0/210557602 >> 172.21.15.63:0/210557602 conn(0x7f2b34078ab0 msgr2=0x7f2b34078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:41.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.750+0000 7f2b39c00700 1 -- 172.21.15.63:0/210557602 shutdown_connections 2024-05-27T07:39:41.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.751+0000 7f2b39c00700 1 -- 172.21.15.63:0/210557602 wait complete. 2024-05-27T07:39:41.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.752+0000 7f2b39c00700 1 Processor -- start 2024-05-27T07:39:41.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.752+0000 7f2b39c00700 1 -- start start 2024-05-27T07:39:41.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b39c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34105280 0x7f2b3419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b39c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b34105bd0 0x7f2b3419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b39c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b3410e6f0 0x7f2b341a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b39c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b34114170 con 0x7f2b34105280 2024-05-27T07:39:41.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b39c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b34113ff0 con 0x7f2b34105bd0 2024-05-27T07:39:41.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b39c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341142f0 con 0x7f2b3410e6f0 2024-05-27T07:39:41.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b33e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34105280 0x7f2b3419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b33e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34105280 0x7f2b3419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39254/0 (socket says 172.21.15.63:39254) 2024-05-27T07:39:41.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b33e00700 1 -- 172.21.15.63:0/260595133 learned_addr learned my addr 172.21.15.63:0/260595133 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:41.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b33400700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b34105bd0 0x7f2b3419d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.753+0000 7f2b39200700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b3410e6f0 0x7f2b341a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.754+0000 7f2b33e00700 1 -- 172.21.15.63:0/260595133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b3410e6f0 msgr2=0x7f2b341a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:41.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.754+0000 7f2b33e00700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b3410e6f0 0x7f2b341a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.754+0000 7f2b33e00700 1 -- 172.21.15.63:0/260595133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b34105bd0 msgr2=0x7f2b3419d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:41.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.754+0000 7f2b33e00700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b34105bd0 0x7f2b3419d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.754+0000 7f2b33e00700 1 -- 172.21.15.63:0/260595133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b341a1c50 con 0x7f2b34105280 2024-05-27T07:39:41.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.754+0000 7f2b39200700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b3410e6f0 0x7f2b341a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:41.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.754+0000 7f2b33e00700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34105280 0x7f2b3419cd80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400d980 tx=0x7f2b2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:41.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.755+0000 7f2b30c00700 1 -- 172.21.15.63:0/260595133 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24016070 con 0x7f2b34105280 2024-05-27T07:39:41.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.755+0000 7f2b30c00700 1 -- 172.21.15.63:0/260595133 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b24004830 con 0x7f2b34105280 2024-05-27T07:39:41.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.755+0000 7f2b30c00700 1 -- 172.21.15.63:0/260595133 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24005440 con 0x7f2b34105280 2024-05-27T07:39:41.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.755+0000 7f2b39c00700 1 -- 172.21.15.63:0/260595133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b341a1f40 con 0x7f2b34105280 2024-05-27T07:39:41.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.755+0000 7f2b39c00700 1 -- 172.21.15.63:0/260595133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b341a9860 con 0x7f2b34105280 2024-05-27T07:39:41.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.760+0000 7f2b30c00700 1 -- 172.21.15.63:0/260595133 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2b240055e0 con 0x7f2b34105280 2024-05-27T07:39:41.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.761+0000 7f2b49c4cb80 1 Processor -- start 2024-05-27T07:39:41.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.761+0000 7f2b49c4cb80 1 -- start start 2024-05-27T07:39:41.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.761+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.761+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7410 0x5582eb9c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.762+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7820 0x5582eb9c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.762+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5582eb8e5e40 con 0x5582eb838e40 2024-05-27T07:39:41.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.762+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5582eb8d59e0 con 0x5582eb9b7410 2024-05-27T07:39:41.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.762+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5582eb8d9a20 con 0x5582eb9b7820 2024-05-27T07:39:41.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.762+0000 7f2b39200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.762+0000 7f2b39200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39256/0 (socket says 172.21.15.63:39256) 2024-05-27T07:39:41.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.762+0000 7f2b39200700 1 -- 172.21.15.63:0/3870753161 learned_addr learned my addr 172.21.15.63:0/3870753161 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:41.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.762+0000 7f2b30c00700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b180b2a70 0x7f2b180b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.762+0000 7f2b33e00700 1 --2- 172.21.15.63:0/3870753161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7410 0x5582eb9c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.762+0000 7f2b33400700 1 --2- 172.21.15.63:0/3870753161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7820 0x5582eb9c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.763+0000 7f2b30c00700 1 -- 172.21.15.63:0/260595133 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2b240e2710 con 0x7f2b34105280 2024-05-27T07:39:41.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.763+0000 7f2b39200700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b180b2a70 0x7f2b180b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.763+0000 7f2b39200700 1 -- 172.21.15.63:0/3870753161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7820 msgr2=0x5582eb9c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:41.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.763+0000 7f2b39200700 1 --2- 172.21.15.63:0/3870753161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7820 0x5582eb9c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.763+0000 7f2b39200700 1 -- 172.21.15.63:0/3870753161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7410 msgr2=0x5582eb9c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:41.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.763+0000 7f2b39200700 1 --2- 172.21.15.63:0/3870753161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7410 0x5582eb9c1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.763+0000 7f2b39200700 1 -- 172.21.15.63:0/3870753161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582eb89e420 con 0x5582eb838e40 2024-05-27T07:39:41.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.764+0000 7f2b39200700 1 --2- 172.21.15.63:0/3870753161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9b6ed0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009550 tx=0x7f2b2c0185e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f8c268636603c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:41.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.764+0000 7f2b39200700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b180b2a70 0x7f2b180b4f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c018930 tx=0x7f2b2c00d000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:41.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.764+0000 7f2b29600700 1 -- 172.21.15.63:0/3870753161 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c01f070 con 0x5582eb838e40 2024-05-27T07:39:41.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.764+0000 7f2b29600700 1 -- 172.21.15.63:0/3870753161 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b2c007500 con 0x5582eb838e40 2024-05-27T07:39:41.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.764+0000 7f2b29600700 1 -- 172.21.15.63:0/3870753161 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c012040 con 0x5582eb838e40 2024-05-27T07:39:41.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.765+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3870753161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 msgr2=0x5582eb9b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:41.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.765+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/3870753161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9b6ed0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009550 tx=0x7f2b2c0185e0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.766+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3870753161 shutdown_connections 2024-05-27T07:39:41.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.766+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/3870753161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7820 0x5582eb9c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.766+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/3870753161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9b6ed0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.766+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/3870753161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7410 0x5582eb9c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.766+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3870753161 >> 172.21.15.63:0/3870753161 conn(0x5582eb898fc0 msgr2=0x5582eb9b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:41.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.766+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3870753161 shutdown_connections 2024-05-27T07:39:41.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.766+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3870753161 wait complete. 2024-05-27T07:39:41.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.766+0000 7f2b49c4cb80 1 -- start start 2024-05-27T07:39:41.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.767+0000 7f2b49c4cb80 1 Processor -- start 2024-05-27T07:39:41.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.767+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb838e40 0x5582eb9c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.767+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7410 0x5582eb9c1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.767+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 0x5582eb9c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.767+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5582eb7dbfa0 con 0x5582eb9b7410 2024-05-27T07:39:41.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.767+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5582eb876760 con 0x5582eb9b7820 2024-05-27T07:39:41.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.767+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5582eb9c4930 con 0x5582eb838e40 2024-05-27T07:39:41.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.768+0000 7f2b49c4cb80 10 client.0 _ll_register_callbacks cb 0x5582eb87fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:41.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.768+0000 7f2b33400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7410 0x5582eb9c1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.768+0000 7f2b33400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7410 0x5582eb9c1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39266/0 (socket says 172.21.15.63:39266) 2024-05-27T07:39:41.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.768+0000 7f2b33400700 1 -- 172.21.15.63:0/2240232996 learned_addr learned my addr 172.21.15.63:0/2240232996 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:41.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.768+0000 7f2b33e00700 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb838e40 0x5582eb9c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.768+0000 7f2b39200700 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 0x5582eb9c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.768+0000 7f2b33400700 1 -- 172.21.15.63:0/2240232996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb838e40 msgr2=0x5582eb9c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:41.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.768+0000 7f2b33400700 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb838e40 0x5582eb9c16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.768+0000 7f2b33400700 1 -- 172.21.15.63:0/2240232996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 msgr2=0x5582eb9c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:41.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.769+0000 7f2b33400700 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 0x5582eb9c2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.769+0000 7f2b33400700 1 -- 172.21.15.63:0/2240232996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582eb8258d0 con 0x5582eb9b7410 2024-05-27T07:39:41.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.769+0000 7f2b39200700 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 0x5582eb9c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:39:41.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.769+0000 7f2b33400700 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7410 0x5582eb9c1c20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009990 tx=0x7f2b200057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9ef9bf89c1e1f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:41.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.769+0000 7f2b1e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.770+0000 7f2b1e000700 1 -- 172.21.15.63:0/2240232996 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20003bd0 con 0x5582eb9b7410 2024-05-27T07:39:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.770+0000 7f2b1e000700 1 -- 172.21.15.63:0/2240232996 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2b20003d70 con 0x5582eb9b7410 2024-05-27T07:39:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.770+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/2240232996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b2c007500 con 0x5582eb9b7410 2024-05-27T07:39:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.770+0000 7f2b1e000700 1 -- 172.21.15.63:0/2240232996 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2b20031050 con 0x5582eb9b7410 2024-05-27T07:39:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.770+0000 7f2b49c4cb80 10 client.16578 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.770+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/2240232996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5582eb8a6060 con 0x5582eb9b7410 2024-05-27T07:39:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.770+0000 7f2b1e000700 1 -- 172.21.15.63:0/2240232996 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b2c007500 con 0x5582eb9b7410 2024-05-27T07:39:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.771+0000 7f2b1e000700 1 -- 172.21.15.63:0/2240232996 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2b20023700 con 0x5582eb9b7410 2024-05-27T07:39:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.771+0000 7f2b49c4cb80 10 client.16578 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.771+0000 7f2b49c4cb80 10 client.16578 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:39:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.771+0000 7f2b49c4cb80 20 client.16578 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.771+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5582eb9d1a30 0x5582eb9d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.771+0000 7f2b49c4cb80 4 client.16578 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.771+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/2240232996 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582eb8e7d90 con 0x5582eb9d1a30 2024-05-27T07:39:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.772+0000 7f2b33e00700 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5582eb9d1a30 0x5582eb9d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.773+0000 7f2b1e000700 10 client.16578 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:39:41.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.773+0000 7f2b33e00700 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5582eb9d1a30 0x5582eb9d3e80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e8276f574986130 server_cookie=b857418b6fc61f1a in_seq=0 out_seq=0 2024-05-27T07:39:41.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b1e000700 1 -- 172.21.15.63:0/2240232996 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5582eb8e7d90 con 0x5582eb9d1a30 2024-05-27T07:39:41.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b1e000700 10 client.16578 handle_command_reply: tid=0 2024-05-27T07:39:41.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b49c4cb80 1 client.16578 shutdown 2024-05-27T07:39:41.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/2240232996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7410 msgr2=0x5582eb9c1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:41.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7410 0x5582eb9c1c20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009990 tx=0x7f2b200057e0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/2240232996 shutdown_connections 2024-05-27T07:39:41.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 0x5582eb9c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb838e40 0x5582eb9c16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5582eb9d1a30 0x5582eb9d3e80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/2240232996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7410 0x5582eb9c1c20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/2240232996 >> 172.21.15.63:0/2240232996 conn(0x5582eb898fc0 msgr2=0x5582eb9b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:41.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b1e000700 0 client.16578 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:39:41.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.775+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/2240232996 shutdown_connections 2024-05-27T07:39:41.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.776+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/2240232996 wait complete. 2024-05-27T07:39:41.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.776+0000 7f2b49c4cb80 20 client.16578 trim_cache size 0 max 16384 2024-05-27T07:39:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.796+0000 7f2b49c4cb80 1 Processor -- start 2024-05-27T07:39:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.796+0000 7f2b49c4cb80 1 -- start start 2024-05-27T07:39:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.796+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb838e40 0x5582eb9b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.796+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 0x5582eb9c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.796+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7820 0x5582eb9bb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.796+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b4d60 con 0x5582eb9b7820 2024-05-27T07:39:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.796+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b25f0 con 0x5582eb838e40 2024-05-27T07:39:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.796+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b0440 con 0x5582eb9b7410 2024-05-27T07:39:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b39200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7820 0x5582eb9bb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b39200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7820 0x5582eb9bb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39280/0 (socket says 172.21.15.63:39280) 2024-05-27T07:39:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b39200700 1 -- 172.21.15.63:0/431413685 learned_addr learned my addr 172.21.15.63:0/431413685 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:41.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b33e00700 1 --2- 172.21.15.63:0/431413685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb838e40 0x5582eb9b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b33400700 1 --2- 172.21.15.63:0/431413685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 0x5582eb9c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b39200700 1 -- 172.21.15.63:0/431413685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 msgr2=0x5582eb9c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:41.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b39200700 1 --2- 172.21.15.63:0/431413685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 0x5582eb9c3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b39200700 1 -- 172.21.15.63:0/431413685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb838e40 msgr2=0x5582eb9b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:41.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b39200700 1 --2- 172.21.15.63:0/431413685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb838e40 0x5582eb9b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b39200700 1 -- 172.21.15.63:0/431413685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582eb9b4350 con 0x5582eb9b7820 2024-05-27T07:39:41.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.797+0000 7f2b33400700 1 --2- 172.21.15.63:0/431413685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 0x5582eb9c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:41.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.798+0000 7f2b39200700 1 --2- 172.21.15.63:0/431413685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7820 0x5582eb9bb880 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009360 tx=0x7f2b2c005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546d9cabfedcaaeb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:41.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.798+0000 7f2b28c00700 1 -- 172.21.15.63:0/431413685 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c030070 con 0x5582eb9b7820 2024-05-27T07:39:41.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.798+0000 7f2b28c00700 1 -- 172.21.15.63:0/431413685 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b2c01ab50 con 0x5582eb9b7820 2024-05-27T07:39:41.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.798+0000 7f2b28c00700 1 -- 172.21.15.63:0/431413685 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c01f070 con 0x5582eb9b7820 2024-05-27T07:39:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.799+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/431413685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7820 msgr2=0x5582eb9bb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.799+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/431413685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7820 0x5582eb9bb880 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009360 tx=0x7f2b2c005aa0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.799+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/431413685 shutdown_connections 2024-05-27T07:39:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.799+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/431413685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb9b7820 0x5582eb9bb880 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.799+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/431413685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb838e40 0x5582eb9b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.799+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/431413685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 0x5582eb9c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.799+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/431413685 >> 172.21.15.63:0/431413685 conn(0x5582eb898fc0 msgr2=0x5582eb9c14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.800+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/431413685 shutdown_connections 2024-05-27T07:39:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.800+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/431413685 wait complete. 2024-05-27T07:39:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.800+0000 7f2b49c4cb80 1 -- start start 2024-05-27T07:39:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.801+0000 7f2b49c4cb80 1 Processor -- start 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.801+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.801+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 0x5582eb9ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.801+0000 7f2b49c4cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 0x5582eb9c5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.801+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b0ab0 con 0x5582eb838e40 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.801+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b1f20 con 0x5582eb9b7820 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.801+0000 7f2b49c4cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b1180 con 0x5582eb9b7410 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.801+0000 7f2b49c4cb80 10 client.0 _ll_register_callbacks cb 0x5582eb87fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b33e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b33e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39286/0 (socket says 172.21.15.63:39286) 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b33e00700 1 -- 172.21.15.63:0/3598634906 learned_addr learned my addr 172.21.15.63:0/3598634906 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b33400700 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 0x5582eb9ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b39200700 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 0x5582eb9c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b33e00700 1 -- 172.21.15.63:0/3598634906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 msgr2=0x5582eb9ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b33e00700 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 0x5582eb9ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b33e00700 1 -- 172.21.15.63:0/3598634906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 msgr2=0x5582eb9c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b33e00700 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 0x5582eb9c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b33e00700 1 -- 172.21.15.63:0/3598634906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582eb9c5a20 con 0x5582eb838e40 2024-05-27T07:39:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.802+0000 7f2b33400700 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 0x5582eb9ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.803+0000 7f2b33e00700 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9c9e50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2b240dfb50 tx=0x7f2b240e4c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5447283e462afa9d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.803+0000 7f2b1e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.803+0000 7f2b1e000700 1 -- 172.21.15.63:0/3598634906 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b240a9680 con 0x5582eb838e40 2024-05-27T07:39:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.803+0000 7f2b1e000700 1 -- 172.21.15.63:0/3598634906 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f2b2400fe70 con 0x5582eb838e40 2024-05-27T07:39:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.803+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3598634906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b2c01ab50 con 0x5582eb838e40 2024-05-27T07:39:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.803+0000 7f2b1e000700 1 -- 172.21.15.63:0/3598634906 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2b240ada30 con 0x5582eb838e40 2024-05-27T07:39:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.803+0000 7f2b49c4cb80 10 client.16584 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.804+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3598634906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5582eba3bff0 con 0x5582eb838e40 2024-05-27T07:39:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.804+0000 7f2b1e000700 1 -- 172.21.15.63:0/3598634906 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b2c01ab50 con 0x5582eb838e40 2024-05-27T07:39:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.804+0000 7f2b1e000700 1 -- 172.21.15.63:0/3598634906 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2b240e2320 con 0x5582eb838e40 2024-05-27T07:39:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.804+0000 7f2b49c4cb80 10 client.16584 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.804+0000 7f2b49c4cb80 10 client.16584 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:39:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.804+0000 7f2b49c4cb80 20 client.16584 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.804+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5582eba3c600 0x5582eba3ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.804+0000 7f2b49c4cb80 4 client.16584 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:39:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.804+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3598634906 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582eb8e1570 con 0x5582eba3c600 2024-05-27T07:39:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.805+0000 7f2b33400700 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5582eba3c600 0x5582eba3ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.806+0000 7f2b33400700 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5582eba3c600 0x5582eba3ea20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16a33c6093152e44 server_cookie=dae17376a344b668 in_seq=0 out_seq=0 2024-05-27T07:39:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.806+0000 7f2b1e000700 10 client.16584 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:39:41.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.806+0000 7f2b1e000700 1 -- 172.21.15.63:0/3598634906 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582eb8e1570 con 0x5582eba3c600 2024-05-27T07:39:41.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.806+0000 7f2b1e000700 10 client.16584 handle_command_reply: tid=0 2024-05-27T07:39:41.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 client.16584 shutdown 2024-05-27T07:39:41.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3598634906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 msgr2=0x5582eb9c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:41.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9c9e50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2b240dfb50 tx=0x7f2b240e4c20 comp rx=0 tx=0).stop 2024-05-27T07:39:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3598634906 shutdown_connections 2024-05-27T07:39:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582eb9b7820 0x5582eb9c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5582eba3c600 0x5582eba3ea20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582eb838e40 0x5582eb9c9e50 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 --2- 172.21.15.63:0/3598634906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582eb9b7410 0x5582eb9ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3598634906 >> 172.21.15.63:0/3598634906 conn(0x5582eb898fc0 msgr2=0x5582eb9c1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:41.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b1e000700 0 client.16584 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:39:41.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3598634906 shutdown_connections 2024-05-27T07:39:41.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.807+0000 7f2b49c4cb80 1 -- 172.21.15.63:0/3598634906 wait complete. 2024-05-27T07:39:41.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.808+0000 7f2b49c4cb80 20 client.16584 trim_cache size 0 max 16384 2024-05-27T07:39:41.808 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:39:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.808+0000 7f2b39c00700 1 -- 172.21.15.63:0/260595133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b180b2a70 msgr2=0x7f2b180b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.808+0000 7f2b39c00700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b180b2a70 0x7f2b180b4f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c018930 tx=0x7f2b2c00d000 comp rx=0 tx=0).stop 2024-05-27T07:39:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.808+0000 7f2b39c00700 1 -- 172.21.15.63:0/260595133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34105280 msgr2=0x7f2b3419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.808+0000 7f2b39c00700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34105280 0x7f2b3419cd80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400d980 tx=0x7f2b2400dc90 comp rx=0 tx=0).stop 2024-05-27T07:39:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.809+0000 7f2b39c00700 1 -- 172.21.15.63:0/260595133 shutdown_connections 2024-05-27T07:39:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.809+0000 7f2b39c00700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b180b2a70 0x7f2b180b4f80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.809+0000 7f2b39c00700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b3410e6f0 0x7f2b341a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.809+0000 7f2b39c00700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b34105280 0x7f2b3419cd80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.809+0000 7f2b39c00700 1 --2- 172.21.15.63:0/260595133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b34105bd0 0x7f2b3419d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.809+0000 7f2b39c00700 1 -- 172.21.15.63:0/260595133 >> 172.21.15.63:0/260595133 conn(0x7f2b34078ab0 msgr2=0x7f2b340ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.809+0000 7f2b39c00700 1 -- 172.21.15.63:0/260595133 shutdown_connections 2024-05-27T07:39:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:41.809+0000 7f2b39c00700 1 -- 172.21.15.63:0/260595133 wait complete. 2024-05-27T07:39:42.812 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:39:43.308 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:39:43.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:43 smithi086 ceph-mon[34687]: pgmap v187: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:39:43.709 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:43 smithi149 ceph-mon[35408]: pgmap v187: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:39:43.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:43 smithi063 ceph-mon[31613]: pgmap v187: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.396+0000 7f167ec00640 1 -- 172.21.15.63:0/2343930117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1678106610 msgr2=0x7f16781129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.396+0000 7f167ec00640 1 --2- 172.21.15.63:0/2343930117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1678106610 0x7f16781129e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1670009ca0 tx=0x7f167002a8b0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.396+0000 7f167ec00640 1 -- 172.21.15.63:0/2343930117 shutdown_connections 2024-05-27T07:39:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.396+0000 7f167ec00640 1 --2- 172.21.15.63:0/2343930117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1678106610 0x7f16781129e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.396+0000 7f167ec00640 1 --2- 172.21.15.63:0/2343930117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f167810c360 0x7f1678105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.396+0000 7f167ec00640 1 --2- 172.21.15.63:0/2343930117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167810bec0 0x7f1678105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.396+0000 7f167ec00640 1 -- 172.21.15.63:0/2343930117 >> 172.21.15.63:0/2343930117 conn(0x7f16781016b0 msgr2=0x7f1678103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.396+0000 7f167ec00640 1 -- 172.21.15.63:0/2343930117 shutdown_connections 2024-05-27T07:39:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.396+0000 7f167ec00640 1 -- 172.21.15.63:0/2343930117 wait complete. 2024-05-27T07:39:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.397+0000 7f167ec00640 1 Processor -- start 2024-05-27T07:39:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.398+0000 7f167ec00640 1 -- start start 2024-05-27T07:39:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.398+0000 7f167ec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1678106610 0x7f16781a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.398+0000 7f167ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167810bec0 0x7f16781a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.398+0000 7f167ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167810c360 0x7f167819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.398+0000 7f167ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1678115fb0 con 0x7f167810bec0 2024-05-27T07:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.398+0000 7f167ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1678115e30 con 0x7f167810c360 2024-05-27T07:39:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.398+0000 7f167ec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1678116130 con 0x7f1678106610 2024-05-27T07:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.398+0000 7f1677400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167810bec0 0x7f16781a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.399+0000 7f1677400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167810bec0 0x7f16781a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39300/0 (socket says 172.21.15.63:39300) 2024-05-27T07:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.399+0000 7f1677400640 1 -- 172.21.15.63:0/4276951980 learned_addr learned my addr 172.21.15.63:0/4276951980 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.399+0000 7f1677e00640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1678106610 0x7f16781a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.399+0000 7f167cc00640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167810c360 0x7f167819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.399+0000 7f1677400640 1 -- 172.21.15.63:0/4276951980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1678106610 msgr2=0x7f16781a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.399+0000 7f1677400640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1678106610 0x7f16781a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.399+0000 7f1677400640 1 -- 172.21.15.63:0/4276951980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167810c360 msgr2=0x7f167819c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.399+0000 7f1677400640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167810c360 0x7f167819c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.399+0000 7f1677400640 1 -- 172.21.15.63:0/4276951980 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167819cd70 con 0x7f167810bec0 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.399+0000 7f1677400640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167810bec0 0x7f16781a2af0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f166400c970 tx=0x7f166400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.400+0000 7f1674c00640 1 -- 172.21.15.63:0/4276951980 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664007c70 con 0x7f167810bec0 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.400+0000 7f1674c00640 1 -- 172.21.15.63:0/4276951980 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1664007e10 con 0x7f167810bec0 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.400+0000 7f1674c00640 1 -- 172.21.15.63:0/4276951980 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16640186f0 con 0x7f167810bec0 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.400+0000 7f167ec00640 1 -- 172.21.15.63:0/4276951980 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167819d060 con 0x7f167810bec0 2024-05-27T07:39:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.400+0000 7f167ec00640 1 -- 172.21.15.63:0/4276951980 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16781a9350 con 0x7f167810bec0 2024-05-27T07:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.404+0000 7f1674c00640 1 -- 172.21.15.63:0/4276951980 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f166400fa90 con 0x7f167810bec0 2024-05-27T07:39:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.404+0000 7f1685c3e740 1 Processor -- start 2024-05-27T07:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.404+0000 7f1685c3e740 1 -- start start 2024-05-27T07:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.405+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.405+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f072f0 0x5567a82d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.405+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f0f4d0 0x5567a82d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.405+0000 7f1685c3e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5567a8243b90 con 0x5567a7f301b0 2024-05-27T07:39:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.405+0000 7f1685c3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5567a8233730 con 0x5567a7f072f0 2024-05-27T07:39:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.405+0000 7f1685c3e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5567a8237770 con 0x5567a7f0f4d0 2024-05-27T07:39:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.405+0000 7f167cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.405+0000 7f167cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39302/0 (socket says 172.21.15.63:39302) 2024-05-27T07:39:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.405+0000 7f167cc00640 1 -- 172.21.15.63:0/3685734047 learned_addr learned my addr 172.21.15.63:0/3685734047 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.406+0000 7f1677e00640 1 --2- 172.21.15.63:0/3685734047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f072f0 0x5567a82d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.406+0000 7f1677400640 1 --2- 172.21.15.63:0/3685734047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f0f4d0 0x5567a82d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.406+0000 7f167cc00640 1 -- 172.21.15.63:0/3685734047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f0f4d0 msgr2=0x5567a82d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.406+0000 7f167cc00640 1 --2- 172.21.15.63:0/3685734047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f0f4d0 0x5567a82d32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.406+0000 7f167cc00640 1 -- 172.21.15.63:0/3685734047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f072f0 msgr2=0x5567a82d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.406+0000 7f167cc00640 1 --2- 172.21.15.63:0/3685734047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f072f0 0x5567a82d2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.406+0000 7f167cc00640 1 -- 172.21.15.63:0/3685734047 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a81cc8b0 con 0x5567a7f301b0 2024-05-27T07:39:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.406+0000 7f1674c00640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f165c0b29d0 0x7f165c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.406+0000 7f1674c00640 1 -- 172.21.15.63:0/4276951980 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f16640e2a00 con 0x7f167810bec0 2024-05-27T07:39:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.406+0000 7f167cc00640 1 --2- 172.21.15.63:0/3685734047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82d2860 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1670035000 tx=0x7f1670009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9838c8a70f5519c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.407+0000 7f1677400640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f165c0b29d0 0x7f165c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.407+0000 7f166d200640 1 -- 172.21.15.63:0/3685734047 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16700307b0 con 0x5567a7f301b0 2024-05-27T07:39:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.407+0000 7f166d200640 1 -- 172.21.15.63:0/3685734047 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1670036040 con 0x5567a7f301b0 2024-05-27T07:39:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.407+0000 7f166d200640 1 -- 172.21.15.63:0/3685734047 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670004430 con 0x5567a7f301b0 2024-05-27T07:39:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.408+0000 7f1685c3e740 1 -- 172.21.15.63:0/3685734047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 msgr2=0x5567a82d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.408+0000 7f1685c3e740 1 --2- 172.21.15.63:0/3685734047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82d2860 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1670035000 tx=0x7f1670009a20 comp rx=0 tx=0).stop 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.408+0000 7f1677400640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f165c0b29d0 0x7f165c0b4ee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f16640a6b80 tx=0x7f16640b2040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.408+0000 7f1685c3e740 1 -- 172.21.15.63:0/3685734047 shutdown_connections 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.408+0000 7f1685c3e740 1 --2- 172.21.15.63:0/3685734047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f0f4d0 0x5567a82d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.408+0000 7f1685c3e740 1 --2- 172.21.15.63:0/3685734047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f072f0 0x5567a82d2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.408+0000 7f1685c3e740 1 --2- 172.21.15.63:0/3685734047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82d2860 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.408+0000 7f1685c3e740 1 -- 172.21.15.63:0/3685734047 >> 172.21.15.63:0/3685734047 conn(0x5567a7f24300 msgr2=0x5567a82cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.408+0000 7f1685c3e740 1 -- 172.21.15.63:0/3685734047 shutdown_connections 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.408+0000 7f1685c3e740 1 -- 172.21.15.63:0/3685734047 wait complete. 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.409+0000 7f1685c3e740 1 -- start start 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.409+0000 7f1685c3e740 1 Processor -- start 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f072f0 0x5567a82dca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f0f4d0 0x5567a82dcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f1685c3e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5567a7f19790 con 0x5567a7f301b0 2024-05-27T07:39:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f1685c3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5567a81dbbc0 con 0x5567a7f072f0 2024-05-27T07:39:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f1685c3e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5567a82df930 con 0x5567a7f0f4d0 2024-05-27T07:39:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f1685c3e740 10 client.0 _ll_register_callbacks cb 0x5567a81b9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f167cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f167cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82e8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39306/0 (socket says 172.21.15.63:39306) 2024-05-27T07:39:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f167cc00640 1 -- 172.21.15.63:0/817047641 learned_addr learned my addr 172.21.15.63:0/817047641 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.410+0000 7f1677e00640 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f0f4d0 0x5567a82dcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.411+0000 7f167cc00640 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f072f0 0x5567a82dca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.411+0000 7f167cc00640 1 -- 172.21.15.63:0/817047641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f0f4d0 msgr2=0x5567a82dcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.411+0000 7f167cc00640 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f0f4d0 0x5567a82dcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.411+0000 7f167cc00640 1 -- 172.21.15.63:0/817047641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f072f0 msgr2=0x5567a82dca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.411+0000 7f167cc00640 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f072f0 0x5567a82dca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.411+0000 7f167cc00640 1 -- 172.21.15.63:0/817047641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a7f48210 con 0x5567a7f301b0 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.411+0000 7f167cc00640 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82e8c60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1670004980 tx=0x7f16700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a15d60336d652d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.412+0000 7f1661600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.412+0000 7f1661600640 1 -- 172.21.15.63:0/817047641 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670058060 con 0x5567a7f301b0 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.412+0000 7f1685c3e740 1 -- 172.21.15.63:0/817047641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1670036040 con 0x5567a7f301b0 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.412+0000 7f1661600640 1 -- 172.21.15.63:0/817047641 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f1670003970 con 0x5567a7f301b0 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.412+0000 7f1661600640 1 -- 172.21.15.63:0/817047641 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f167004a040 con 0x5567a7f301b0 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.412+0000 7f1685c3e740 10 client.16596 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.412+0000 7f1685c3e740 1 -- 172.21.15.63:0/817047641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5567a8014620 con 0x5567a7f301b0 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.413+0000 7f1661600640 1 -- 172.21.15.63:0/817047641 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1670036040 con 0x5567a7f301b0 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.413+0000 7f1661600640 1 -- 172.21.15.63:0/817047641 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1670050070 con 0x5567a7f301b0 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.413+0000 7f1685c3e740 10 client.16596 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.413+0000 7f1685c3e740 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.413+0000 7f1685c3e740 20 client.16596 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.413+0000 7f1685c3e740 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5567a7eeed80 0x5567a82eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.413+0000 7f1685c3e740 4 client.16596 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.413+0000 7f1685c3e740 1 -- 172.21.15.63:0/817047641 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567a8245ae0 con 0x5567a7eeed80 2024-05-27T07:39:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.414+0000 7f1677e00640 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5567a7eeed80 0x5567a82eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.415+0000 7f1677e00640 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5567a7eeed80 0x5567a82eea80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef869b5b1e72271 server_cookie=15dfb393cb8efcdf in_seq=0 out_seq=0 2024-05-27T07:39:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.415+0000 7f1661600640 10 client.16596 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.416+0000 7f1661600640 1 -- 172.21.15.63:0/817047641 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5567a8245ae0 con 0x5567a7eeed80 2024-05-27T07:39:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.416+0000 7f1661600640 10 client.16596 handle_command_reply: tid=0 2024-05-27T07:39:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 client.16596 shutdown 2024-05-27T07:39:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 -- 172.21.15.63:0/817047641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 msgr2=0x5567a82e8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82e8c60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1670004980 tx=0x7f16700095c0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 -- 172.21.15.63:0/817047641 shutdown_connections 2024-05-27T07:39:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5567a7eeed80 0x5567a82eea80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82e8c60 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f0f4d0 0x5567a82dcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 --2- 172.21.15.63:0/817047641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f072f0 0x5567a82dca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1661600640 0 client.16596 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 -- 172.21.15.63:0/817047641 >> 172.21.15.63:0/817047641 conn(0x5567a7f24300 msgr2=0x5567a82d2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 -- 172.21.15.63:0/817047641 shutdown_connections 2024-05-27T07:39:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 1 -- 172.21.15.63:0/817047641 wait complete. 2024-05-27T07:39:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.417+0000 7f1685c3e740 20 client.16596 trim_cache size 0 max 16384 2024-05-27T07:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.438+0000 7f1685c3e740 1 Processor -- start 2024-05-27T07:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.438+0000 7f1685c3e740 1 -- start start 2024-05-27T07:39:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.438+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f072f0 0x5567a82dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.438+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f0f4d0 0x5567a82f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.438+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f301b0 0x5567a82f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.438+0000 7f1685c3e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16781b5ba0 con 0x5567a7f072f0 2024-05-27T07:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.438+0000 7f1685c3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16781b3470 con 0x5567a7f0f4d0 2024-05-27T07:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.438+0000 7f1685c3e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16781b12c0 con 0x5567a7f301b0 2024-05-27T07:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.439+0000 7f167cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f072f0 0x5567a82dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.439+0000 7f167cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f072f0 0x5567a82dcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39314/0 (socket says 172.21.15.63:39314) 2024-05-27T07:39:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.439+0000 7f167cc00640 1 -- 172.21.15.63:0/1106568688 learned_addr learned my addr 172.21.15.63:0/1106568688 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.439+0000 7f1677e00640 1 --2- 172.21.15.63:0/1106568688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f0f4d0 0x5567a82f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.439+0000 7f167cc00640 1 -- 172.21.15.63:0/1106568688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f301b0 msgr2=0x5567a82f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.439+0000 7f167cc00640 1 --2- 172.21.15.63:0/1106568688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f301b0 0x5567a82f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.439+0000 7f167cc00640 1 -- 172.21.15.63:0/1106568688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f0f4d0 msgr2=0x5567a82f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.439+0000 7f167cc00640 1 --2- 172.21.15.63:0/1106568688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f0f4d0 0x5567a82f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.439+0000 7f167cc00640 1 -- 172.21.15.63:0/1106568688 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567a82d3c40 con 0x5567a7f072f0 2024-05-27T07:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.440+0000 7f167cc00640 1 --2- 172.21.15.63:0/1106568688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f072f0 0x5567a82dcf90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f16700094e0 tx=0x7f1670005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78880819f3796965 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.440+0000 7f1663e00640 1 -- 172.21.15.63:0/1106568688 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670058070 con 0x5567a7f072f0 2024-05-27T07:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.440+0000 7f1663e00640 1 -- 172.21.15.63:0/1106568688 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1670036070 con 0x5567a7f072f0 2024-05-27T07:39:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.440+0000 7f1663e00640 1 -- 172.21.15.63:0/1106568688 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670003970 con 0x5567a7f072f0 2024-05-27T07:39:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.441+0000 7f1685c3e740 1 -- 172.21.15.63:0/1106568688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f072f0 msgr2=0x5567a82dcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.441+0000 7f1685c3e740 1 --2- 172.21.15.63:0/1106568688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f072f0 0x5567a82dcf90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f16700094e0 tx=0x7f1670005bf0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.441+0000 7f1685c3e740 1 -- 172.21.15.63:0/1106568688 shutdown_connections 2024-05-27T07:39:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.441+0000 7f1685c3e740 1 --2- 172.21.15.63:0/1106568688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f301b0 0x5567a82f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.441+0000 7f1685c3e740 1 --2- 172.21.15.63:0/1106568688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f0f4d0 0x5567a82f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.441+0000 7f1685c3e740 1 --2- 172.21.15.63:0/1106568688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f072f0 0x5567a82dcf90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.441+0000 7f1685c3e740 1 -- 172.21.15.63:0/1106568688 >> 172.21.15.63:0/1106568688 conn(0x5567a7f24300 msgr2=0x5567a82d7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.441+0000 7f1685c3e740 1 -- 172.21.15.63:0/1106568688 shutdown_connections 2024-05-27T07:39:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.441+0000 7f1685c3e740 1 -- 172.21.15.63:0/1106568688 wait complete. 2024-05-27T07:39:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.442+0000 7f1685c3e740 1 -- start start 2024-05-27T07:39:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.442+0000 7f1685c3e740 1 Processor -- start 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f072f0 0x5567a834e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f0f4d0 0x5567a834e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f1685c3e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82f1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f1685c3e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16781b1930 con 0x5567a7f301b0 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f1685c3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16781b2da0 con 0x5567a7f0f4d0 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f1685c3e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16781b2000 con 0x5567a7f072f0 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f1685c3e740 10 client.0 _ll_register_callbacks cb 0x7f1670030f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f167cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f167cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82f1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39328/0 (socket says 172.21.15.63:39328) 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f167cc00640 1 -- 172.21.15.63:0/252680386 learned_addr learned my addr 172.21.15.63:0/252680386 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f167cc00640 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f072f0 0x5567a834e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.443+0000 7f1677e00640 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f0f4d0 0x5567a834e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.444+0000 7f167cc00640 1 -- 172.21.15.63:0/252680386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f072f0 msgr2=0x5567a834e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.444+0000 7f167cc00640 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f072f0 0x5567a834e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.444+0000 7f167cc00640 1 -- 172.21.15.63:0/252680386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f0f4d0 msgr2=0x5567a834e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.444+0000 7f167cc00640 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f0f4d0 0x5567a834e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.444+0000 7f167cc00640 1 -- 172.21.15.63:0/252680386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567a82f1760 con 0x5567a7f301b0 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.444+0000 7f167cc00640 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82f1170 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1670009960 tx=0x7f1670065040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3dd16b580a40cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.444+0000 7f1661600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.444+0000 7f1661600640 1 -- 172.21.15.63:0/252680386 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1670058040 con 0x5567a7f301b0 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.444+0000 7f1661600640 1 -- 172.21.15.63:0/252680386 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f1670004b40 con 0x5567a7f301b0 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.444+0000 7f1685c3e740 1 -- 172.21.15.63:0/252680386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1670036070 con 0x5567a7f301b0 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1661600640 1 -- 172.21.15.63:0/252680386 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1670004450 con 0x5567a7f301b0 2024-05-27T07:39:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1685c3e740 10 client.16602 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1685c3e740 1 -- 172.21.15.63:0/252680386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5567a83569b0 con 0x5567a7f301b0 2024-05-27T07:39:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1661600640 1 -- 172.21.15.63:0/252680386 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1670036070 con 0x5567a7f301b0 2024-05-27T07:39:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1661600640 1 -- 172.21.15.63:0/252680386 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1670050040 con 0x5567a7f301b0 2024-05-27T07:39:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1685c3e740 10 client.16602 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1685c3e740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1685c3e740 20 client.16602 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1685c3e740 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5567a7ec41c0 0x5567a8358e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1685c3e740 4 client.16602 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:39:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.445+0000 7f1685c3e740 1 -- 172.21.15.63:0/252680386 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567a823f2c0 con 0x5567a7ec41c0 2024-05-27T07:39:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.446+0000 7f1677e00640 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5567a7ec41c0 0x5567a8358e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.447+0000 7f1677e00640 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5567a7ec41c0 0x5567a8358e30 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c8b6f8f1a650cea server_cookie=f6f5a8e51873d0a6 in_seq=0 out_seq=0 2024-05-27T07:39:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.447+0000 7f1661600640 10 client.16602 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.485+0000 7f1661600640 1 -- 172.21.15.63:0/252680386 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5567a823f2c0 con 0x5567a7ec41c0 2024-05-27T07:39:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.485+0000 7f1661600640 10 client.16602 handle_command_reply: tid=0 2024-05-27T07:39:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.485+0000 7f1685c3e740 1 client.16602 shutdown 2024-05-27T07:39:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.485+0000 7f1685c3e740 1 -- 172.21.15.63:0/252680386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 msgr2=0x5567a82f1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.485+0000 7f1685c3e740 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82f1170 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1670009960 tx=0x7f1670065040 comp rx=0 tx=0).stop 2024-05-27T07:39:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.486+0000 7f1685c3e740 1 -- 172.21.15.63:0/252680386 shutdown_connections 2024-05-27T07:39:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.486+0000 7f1685c3e740 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5567a7ec41c0 0x5567a8358e30 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.486+0000 7f1685c3e740 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5567a7f301b0 0x5567a82f1170 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.486+0000 7f1685c3e740 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567a7f0f4d0 0x5567a834e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.486+0000 7f1685c3e740 1 --2- 172.21.15.63:0/252680386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5567a7f072f0 0x5567a834e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.486+0000 7f1685c3e740 1 -- 172.21.15.63:0/252680386 >> 172.21.15.63:0/252680386 conn(0x5567a7f24300 msgr2=0x5567a82dc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.486+0000 7f1661600640 0 client.16602 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.486+0000 7f1685c3e740 1 -- 172.21.15.63:0/252680386 shutdown_connections 2024-05-27T07:39:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.486+0000 7f1685c3e740 1 -- 172.21.15.63:0/252680386 wait complete. 2024-05-27T07:39:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.486+0000 7f1685c3e740 20 client.16602 trim_cache size 0 max 16384 2024-05-27T07:39:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:39:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:39:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:39:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.487+0000 7f167ec00640 1 -- 172.21.15.63:0/4276951980 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f165c0b29d0 msgr2=0x7f165c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.487+0000 7f167ec00640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f165c0b29d0 0x7f165c0b4ee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f16640a6b80 tx=0x7f16640b2040 comp rx=0 tx=0).stop 2024-05-27T07:39:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.487+0000 7f167ec00640 1 -- 172.21.15.63:0/4276951980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167810bec0 msgr2=0x7f16781a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.487+0000 7f167ec00640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167810bec0 0x7f16781a2af0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f166400c970 tx=0x7f166400ce30 comp rx=0 tx=0).stop 2024-05-27T07:39:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.487+0000 7f167ec00640 1 -- 172.21.15.63:0/4276951980 shutdown_connections 2024-05-27T07:39:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.487+0000 7f167ec00640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f165c0b29d0 0x7f165c0b4ee0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.487+0000 7f167ec00640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167810c360 0x7f167819c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.487+0000 7f167ec00640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167810bec0 0x7f16781a2af0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.488+0000 7f167ec00640 1 --2- 172.21.15.63:0/4276951980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1678106610 0x7f16781a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.488+0000 7f167ec00640 1 -- 172.21.15.63:0/4276951980 >> 172.21.15.63:0/4276951980 conn(0x7f16781016b0 msgr2=0x7f16781086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.488+0000 7f167ec00640 1 -- 172.21.15.63:0/4276951980 shutdown_connections 2024-05-27T07:39:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:45.488+0000 7f167ec00640 1 -- 172.21.15.63:0/4276951980 wait complete. 2024-05-27T07:39:45.649 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:45 smithi063 ceph-mon[31613]: pgmap v188: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:39:45.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:45 smithi086 ceph-mon[34687]: pgmap v188: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:39:45.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:45 smithi149 ceph-mon[35408]: pgmap v188: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:39:46.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.939+0000 7fcde8400700 1 -- 172.21.15.63:0/525297938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcde0107de0 msgr2=0x7fcde01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.939+0000 7fcde8400700 1 --2- 172.21.15.63:0/525297938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcde0107de0 0x7fcde01081f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0099c0 tx=0x7fcdcc009cd0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.939+0000 7fcde8400700 1 -- 172.21.15.63:0/525297938 shutdown_connections 2024-05-27T07:39:46.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.939+0000 7fcde8400700 1 --2- 172.21.15.63:0/525297938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde0108730 0x7fcde0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.939+0000 7fcde8400700 1 --2- 172.21.15.63:0/525297938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde01034b0 0x7fcde01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.939+0000 7fcde8400700 1 --2- 172.21.15.63:0/525297938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcde0107de0 0x7fcde01081f0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.939+0000 7fcde8400700 1 -- 172.21.15.63:0/525297938 >> 172.21.15.63:0/525297938 conn(0x7fcde00fedb0 msgr2=0x7fcde01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:46.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.939+0000 7fcde8400700 1 -- 172.21.15.63:0/525297938 shutdown_connections 2024-05-27T07:39:46.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.939+0000 7fcde8400700 1 -- 172.21.15.63:0/525297938 wait complete. 2024-05-27T07:39:46.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.941+0000 7fcde8400700 1 Processor -- start 2024-05-27T07:39:46.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.941+0000 7fcde8400700 1 -- start start 2024-05-27T07:39:46.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde8400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde01034b0 0x7fcde019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde8400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0107de0 0x7fcde019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde8400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcde0108730 0x7fcde01a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde8400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01131f0 con 0x7fcde0108730 2024-05-27T07:39:46.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde8400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0113070 con 0x7fcde01034b0 2024-05-27T07:39:46.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde8400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0113370 con 0x7fcde0107de0 2024-05-27T07:39:46.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde6400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcde0108730 0x7fcde01a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde01034b0 0x7fcde019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde6400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcde0108730 0x7fcde01a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39334/0 (socket says 172.21.15.63:39334) 2024-05-27T07:39:46.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde6400700 1 -- 172.21.15.63:0/3430577501 learned_addr learned my addr 172.21.15.63:0/3430577501 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:46.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.942+0000 7fcde5000700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0107de0 0x7fcde019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.943+0000 7fcde6400700 1 -- 172.21.15.63:0/3430577501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0107de0 msgr2=0x7fcde019d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.943+0000 7fcde6400700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0107de0 0x7fcde019d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.943+0000 7fcde6400700 1 -- 172.21.15.63:0/3430577501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde01034b0 msgr2=0x7fcde019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.943+0000 7fcde6400700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde01034b0 0x7fcde019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.943+0000 7fcde6400700 1 -- 172.21.15.63:0/3430577501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde01a1e10 con 0x7fcde0108730 2024-05-27T07:39:46.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.943+0000 7fcde6400700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcde0108730 0x7fcde01a1820 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8007d70 tx=0x7fcdd800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:46.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.944+0000 7fcdde000700 1 -- 172.21.15.63:0/3430577501 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8016020 con 0x7fcde0108730 2024-05-27T07:39:46.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.944+0000 7fcdde000700 1 -- 172.21.15.63:0/3430577501 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd800ac90 con 0x7fcde0108730 2024-05-27T07:39:46.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.944+0000 7fcdde000700 1 -- 172.21.15.63:0/3430577501 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80115e0 con 0x7fcde0108730 2024-05-27T07:39:46.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.944+0000 7fcde8400700 1 -- 172.21.15.63:0/3430577501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde01a2100 con 0x7fcde0108730 2024-05-27T07:39:46.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.944+0000 7fcde8400700 1 -- 172.21.15.63:0/3430577501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde01a25c0 con 0x7fcde0108730 2024-05-27T07:39:46.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.949+0000 7fcdde000700 1 -- 172.21.15.63:0/3430577501 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcdd8019020 con 0x7fcde0108730 2024-05-27T07:39:46.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.949+0000 7fcdf7015b80 1 Processor -- start 2024-05-27T07:39:46.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.949+0000 7fcdf7015b80 1 -- start start 2024-05-27T07:39:46.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.950+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d1f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.950+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485ba0e40 0x55f485d1f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.950+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 0x55f485d2c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.950+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f485c4de40 con 0x55f485c8c9f0 2024-05-27T07:39:46.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.950+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f485c3d9e0 con 0x55f485ba0e40 2024-05-27T07:39:46.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.950+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f485c41a20 con 0x55f485d1fe70 2024-05-27T07:39:46.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcde5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d1f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcde5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d1f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39338/0 (socket says 172.21.15.63:39338) 2024-05-27T07:39:46.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcde5000700 1 -- 172.21.15.63:0/787365516 learned_addr learned my addr 172.21.15.63:0/787365516 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:46.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcde5a00700 1 --2- 172.21.15.63:0/787365516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485ba0e40 0x55f485d1f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcde5000700 1 --2- 172.21.15.63:0/787365516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 0x55f485d2c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcde5000700 1 -- 172.21.15.63:0/787365516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 msgr2=0x55f485d2c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:46.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcde5000700 1 --2- 172.21.15.63:0/787365516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 0x55f485d2c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcde5000700 1 -- 172.21.15.63:0/787365516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485ba0e40 msgr2=0x55f485d1f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:46.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcde5000700 1 --2- 172.21.15.63:0/787365516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485ba0e40 0x55f485d1f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcde5000700 1 -- 172.21.15.63:0/787365516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f485c06420 con 0x55f485c8c9f0 2024-05-27T07:39:46.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.951+0000 7fcdde000700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcdc80b2a70 0x7fcdc80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.952+0000 7fcdde000700 1 -- 172.21.15.63:0/3430577501 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fcdd80e2740 con 0x7fcde0108730 2024-05-27T07:39:46.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.952+0000 7fcde5a00700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcdc80b2a70 0x7fcdc80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.952+0000 7fcde5000700 1 --2- 172.21.15.63:0/787365516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d1f330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0099c0 tx=0x7fcdcc009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3916f67240c09275 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:46.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.952+0000 7fcdd2a00700 1 -- 172.21.15.63:0/787365516 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc005220 con 0x55f485c8c9f0 2024-05-27T07:39:46.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.953+0000 7fcdd2a00700 1 -- 172.21.15.63:0/787365516 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdcc00b790 con 0x55f485c8c9f0 2024-05-27T07:39:46.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.953+0000 7fcdd2a00700 1 -- 172.21.15.63:0/787365516 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0038f0 con 0x55f485c8c9f0 2024-05-27T07:39:46.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.954+0000 7fcdf7015b80 1 -- 172.21.15.63:0/787365516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 msgr2=0x55f485d1f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.954+0000 7fcde5a00700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcdc80b2a70 0x7fcdc80b4f80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55f485d1c790 tx=0x7fcdd400b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:46.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.954+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/787365516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d1f330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0099c0 tx=0x7fcdcc009510 comp rx=0 tx=0).stop 2024-05-27T07:39:46.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.954+0000 7fcdf7015b80 1 -- 172.21.15.63:0/787365516 shutdown_connections 2024-05-27T07:39:46.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.954+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/787365516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 0x55f485d2c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.954+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/787365516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d1f330 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.954+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/787365516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485ba0e40 0x55f485d1f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.954+0000 7fcdf7015b80 1 -- 172.21.15.63:0/787365516 >> 172.21.15.63:0/787365516 conn(0x55f485c3a390 msgr2=0x55f485d1b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:46.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.954+0000 7fcdf7015b80 1 -- 172.21.15.63:0/787365516 shutdown_connections 2024-05-27T07:39:46.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.954+0000 7fcdf7015b80 1 -- 172.21.15.63:0/787365516 wait complete. 2024-05-27T07:39:46.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.955+0000 7fcdf7015b80 1 -- start start 2024-05-27T07:39:46.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.955+0000 7fcdf7015b80 1 Processor -- start 2024-05-27T07:39:46.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485ba0e40 0x55f485d38310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d38850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 0x55f485d32710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f485b43fa0 con 0x55f485c8c9f0 2024-05-27T07:39:46.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f485bde760 con 0x55f485ba0e40 2024-05-27T07:39:46.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f485d2cb20 con 0x55f485d1fe70 2024-05-27T07:39:46.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcdf7015b80 10 client.0 _ll_register_callbacks cb 0x55f485be7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:46.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcde5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d38850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcde5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d38850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39350/0 (socket says 172.21.15.63:39350) 2024-05-27T07:39:46.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcde5000700 1 -- 172.21.15.63:0/3965082386 learned_addr learned my addr 172.21.15.63:0/3965082386 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.956+0000 7fcde5a00700 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485ba0e40 0x55f485d38310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.957+0000 7fcde6400700 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 0x55f485d32710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.957+0000 7fcde5000700 1 -- 172.21.15.63:0/3965082386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 msgr2=0x55f485d32710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.957+0000 7fcde5000700 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 0x55f485d32710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.957+0000 7fcde5000700 1 -- 172.21.15.63:0/3965082386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485ba0e40 msgr2=0x55f485d38310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.957+0000 7fcde5000700 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485ba0e40 0x55f485d38310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.957+0000 7fcde5000700 1 -- 172.21.15.63:0/3965082386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f485b8d8d0 con 0x55f485c8c9f0 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.957+0000 7fcde6400700 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 0x55f485d32710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.957+0000 7fcde5000700 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d38850 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0058e0 tx=0x7fcdcc005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c6f33901fd10d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.957+0000 7fcdc7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.957+0000 7fcdc7400700 1 -- 172.21.15.63:0/3965082386 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0038f0 con 0x55f485c8c9f0 2024-05-27T07:39:46.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.958+0000 7fcdc7400700 1 -- 172.21.15.63:0/3965082386 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fcdcc0036f0 con 0x55f485c8c9f0 2024-05-27T07:39:46.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.958+0000 7fcdf7015b80 1 -- 172.21.15.63:0/3965082386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdcc00b790 con 0x55f485c8c9f0 2024-05-27T07:39:46.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.958+0000 7fcdc7400700 1 -- 172.21.15.63:0/3965082386 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc036050 con 0x55f485c8c9f0 2024-05-27T07:39:46.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.958+0000 7fcdf7015b80 10 client.16614 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:46.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.958+0000 7fcdf7015b80 1 -- 172.21.15.63:0/3965082386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f485c0e060 con 0x55f485c8c9f0 2024-05-27T07:39:46.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.958+0000 7fcdc7400700 1 -- 172.21.15.63:0/3965082386 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdcc00b790 con 0x55f485c8c9f0 2024-05-27T07:39:46.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.959+0000 7fcdc7400700 1 -- 172.21.15.63:0/3965082386 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcdcc00fc90 con 0x55f485c8c9f0 2024-05-27T07:39:46.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.959+0000 7fcdf7015b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:46.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.959+0000 7fcdf7015b80 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:46.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.959+0000 7fcdf7015b80 20 client.16614 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:46.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.959+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f485d42230 0x55f485d44650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.959+0000 7fcdf7015b80 4 client.16614 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:46.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.959+0000 7fcdf7015b80 1 -- 172.21.15.63:0/3965082386 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f485c4fd90 con 0x55f485d42230 2024-05-27T07:39:46.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.960+0000 7fcde5a00700 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f485d42230 0x55f485d44650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.961+0000 7fcde5a00700 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f485d42230 0x55f485d44650 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a139674afe5d89 server_cookie=8df5dc2758bc3714 in_seq=0 out_seq=0 2024-05-27T07:39:46.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.961+0000 7fcdc7400700 10 client.16614 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:46.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdc7400700 1 -- 172.21.15.63:0/3965082386 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f485c4fd90 con 0x55f485d42230 2024-05-27T07:39:46.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdc7400700 10 client.16614 handle_command_reply: tid=0 2024-05-27T07:39:46.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdf7015b80 1 client.16614 shutdown 2024-05-27T07:39:46.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdf7015b80 1 -- 172.21.15.63:0/3965082386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 msgr2=0x55f485d38850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d38850 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0058e0 tx=0x7fcdcc005500 comp rx=0 tx=0).stop 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdf7015b80 1 -- 172.21.15.63:0/3965082386 shutdown_connections 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f485d42230 0x55f485d44650 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485d1fe70 0x55f485d32710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485ba0e40 0x55f485d38310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/3965082386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485c8c9f0 0x55f485d38850 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdc7400700 0 client.16614 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdf7015b80 1 -- 172.21.15.63:0/3965082386 >> 172.21.15.63:0/3965082386 conn(0x55f485c3a390 msgr2=0x55f485d1f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.963+0000 7fcdf7015b80 1 -- 172.21.15.63:0/3965082386 shutdown_connections 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.964+0000 7fcdf7015b80 1 -- 172.21.15.63:0/3965082386 wait complete. 2024-05-27T07:39:46.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.964+0000 7fcdf7015b80 20 client.16614 trim_cache size 0 max 16384 2024-05-27T07:39:46.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.985+0000 7fcdf7015b80 1 Processor -- start 2024-05-27T07:39:46.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.985+0000 7fcdf7015b80 1 -- start start 2024-05-27T07:39:46.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d2ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485c8c9f0 0x55f485d1f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 0x55f485d32c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01b0960 con 0x55f485ba0e40 2024-05-27T07:39:46.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01ae1f0 con 0x55f485d1fe70 2024-05-27T07:39:46.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01ac040 con 0x55f485c8c9f0 2024-05-27T07:39:46.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcde5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d2ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcde5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d2ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39360/0 (socket says 172.21.15.63:39360) 2024-05-27T07:39:46.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcde5a00700 1 -- 172.21.15.63:0/1152264876 learned_addr learned my addr 172.21.15.63:0/1152264876 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:46.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcde6400700 1 --2- 172.21.15.63:0/1152264876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 0x55f485d32c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.986+0000 7fcde5000700 1 --2- 172.21.15.63:0/1152264876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485c8c9f0 0x55f485d1f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.987+0000 7fcde5a00700 1 -- 172.21.15.63:0/1152264876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485c8c9f0 msgr2=0x55f485d1f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:46.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.987+0000 7fcde5a00700 1 --2- 172.21.15.63:0/1152264876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485c8c9f0 0x55f485d1f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.987+0000 7fcde5a00700 1 -- 172.21.15.63:0/1152264876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 msgr2=0x55f485d32c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:46.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.987+0000 7fcde5a00700 1 --2- 172.21.15.63:0/1152264876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 0x55f485d32c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.987+0000 7fcde5a00700 1 -- 172.21.15.63:0/1152264876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f485d33140 con 0x55f485ba0e40 2024-05-27T07:39:46.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.987+0000 7fcde5a00700 1 --2- 172.21.15.63:0/1152264876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d2ded0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcdd402afd0 tx=0x7fcdd4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36a28cbaf93b924 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:46.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.988+0000 7fcdd2000700 1 -- 172.21.15.63:0/1152264876 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4003df0 con 0x55f485ba0e40 2024-05-27T07:39:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.988+0000 7fcdd2000700 1 -- 172.21.15.63:0/1152264876 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd4030070 con 0x55f485ba0e40 2024-05-27T07:39:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.988+0000 7fcdd2000700 1 -- 172.21.15.63:0/1152264876 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd402aae0 con 0x55f485ba0e40 2024-05-27T07:39:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.989+0000 7fcdf7015b80 1 -- 172.21.15.63:0/1152264876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 msgr2=0x55f485d2ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.989+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/1152264876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d2ded0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcdd402afd0 tx=0x7fcdd4016000 comp rx=0 tx=0).stop 2024-05-27T07:39:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.989+0000 7fcdf7015b80 1 -- 172.21.15.63:0/1152264876 shutdown_connections 2024-05-27T07:39:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.989+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/1152264876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 0x55f485d32c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.989+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/1152264876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d2ded0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.989+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/1152264876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485c8c9f0 0x55f485d1f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.989+0000 7fcdf7015b80 1 -- 172.21.15.63:0/1152264876 >> 172.21.15.63:0/1152264876 conn(0x55f485c3a390 msgr2=0x55f485d324f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.989+0000 7fcdf7015b80 1 -- 172.21.15.63:0/1152264876 shutdown_connections 2024-05-27T07:39:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.989+0000 7fcdf7015b80 1 -- 172.21.15.63:0/1152264876 wait complete. 2024-05-27T07:39:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.990+0000 7fcdf7015b80 1 -- start start 2024-05-27T07:39:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.990+0000 7fcdf7015b80 1 Processor -- start 2024-05-27T07:39:46.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d3d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485c8c9f0 0x55f485d3d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcdf7015b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 0x55f485daaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01ac6b0 con 0x55f485ba0e40 2024-05-27T07:39:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01adb20 con 0x55f485d1fe70 2024-05-27T07:39:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcdf7015b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01acd80 con 0x55f485c8c9f0 2024-05-27T07:39:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcdf7015b80 10 client.0 _ll_register_callbacks cb 0x55f485be7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:46.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcde5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d3d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcde5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d3d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39366/0 (socket says 172.21.15.63:39366) 2024-05-27T07:39:46.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcde5a00700 1 -- 172.21.15.63:0/2780678665 learned_addr learned my addr 172.21.15.63:0/2780678665 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:46.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcde5000700 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485c8c9f0 0x55f485d3d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.991+0000 7fcde6400700 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 0x55f485daaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.992+0000 7fcde6400700 1 -- 172.21.15.63:0/2780678665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485c8c9f0 msgr2=0x55f485d3d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:46.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.992+0000 7fcde6400700 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485c8c9f0 0x55f485d3d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.992+0000 7fcde6400700 1 -- 172.21.15.63:0/2780678665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 msgr2=0x55f485d3d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:46.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.992+0000 7fcde6400700 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d3d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.992+0000 7fcde6400700 1 -- 172.21.15.63:0/2780678665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f485daafe0 con 0x55f485d1fe70 2024-05-27T07:39:46.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.992+0000 7fcde5a00700 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d3d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:39:46.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.992+0000 7fcde6400700 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 0x55f485daaaa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80dfbd0 tx=0x7fcdd80a9ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac27af609239d12c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:46.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.993+0000 7fcdc7400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:39:46.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.993+0000 7fcdc7400700 1 -- 172.21.15.63:0/2780678665 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800ae40 con 0x55f485d1fe70 2024-05-27T07:39:46.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.993+0000 7fcdf7015b80 1 -- 172.21.15.63:0/2780678665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd4030070 con 0x55f485d1fe70 2024-05-27T07:39:46.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.993+0000 7fcdc7400700 1 -- 172.21.15.63:0/2780678665 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fcdd80910e0 con 0x55f485d1fe70 2024-05-27T07:39:46.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.993+0000 7fcdc7400700 1 -- 172.21.15.63:0/2780678665 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcdd80e2520 con 0x55f485d1fe70 2024-05-27T07:39:46.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.994+0000 7fcdf7015b80 10 client.36245 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:46.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.994+0000 7fcdf7015b80 1 -- 172.21.15.63:0/2780678665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f485dac190 con 0x55f485d1fe70 2024-05-27T07:39:46.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.994+0000 7fcdc7400700 1 -- 172.21.15.63:0/2780678665 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdd4030070 con 0x55f485d1fe70 2024-05-27T07:39:46.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.994+0000 7fcdc7400700 1 -- 172.21.15.63:0/2780678665 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcdd80bebb0 con 0x55f485d1fe70 2024-05-27T07:39:46.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.995+0000 7fcdf7015b80 10 client.36245 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:46.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.995+0000 7fcdf7015b80 10 client.36245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:46.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.995+0000 7fcdf7015b80 20 client.36245 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:46.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.995+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f485dac8a0 0x55f485daecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:46.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.995+0000 7fcdf7015b80 4 client.36245 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:39:46.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.995+0000 7fcdf7015b80 1 -- 172.21.15.63:0/2780678665 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f485c49570 con 0x55f485dac8a0 2024-05-27T07:39:46.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.995+0000 7fcde5a00700 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f485dac8a0 0x55f485daecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:46.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.996+0000 7fcde5a00700 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f485dac8a0 0x55f485daecc0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e41f3a23ed572a server_cookie=f3559ec0a70030e1 in_seq=0 out_seq=0 2024-05-27T07:39:46.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.996+0000 7fcdc7400700 10 client.36245 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:46.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdc7400700 1 -- 172.21.15.63:0/2780678665 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f485c49570 con 0x55f485dac8a0 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdc7400700 10 client.36245 handle_command_reply: tid=0 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdf7015b80 1 client.36245 shutdown 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdf7015b80 1 -- 172.21.15.63:0/2780678665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 msgr2=0x55f485daaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 0x55f485daaaa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80dfbd0 tx=0x7fcdd80a9ca0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdf7015b80 1 -- 172.21.15.63:0/2780678665 shutdown_connections 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f485dac8a0 0x55f485daecc0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f485d1fe70 0x55f485daaaa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f485ba0e40 0x55f485d3d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdf7015b80 1 --2- 172.21.15.63:0/2780678665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f485c8c9f0 0x55f485d3d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdf7015b80 1 -- 172.21.15.63:0/2780678665 >> 172.21.15.63:0/2780678665 conn(0x55f485c3a390 msgr2=0x55f485d30ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.997+0000 7fcdc7400700 0 client.36245 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:46.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.998+0000 7fcdf7015b80 1 -- 172.21.15.63:0/2780678665 shutdown_connections 2024-05-27T07:39:46.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.998+0000 7fcdf7015b80 1 -- 172.21.15.63:0/2780678665 wait complete. 2024-05-27T07:39:46.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.998+0000 7fcdf7015b80 20 client.36245 trim_cache size 0 max 16384 2024-05-27T07:39:46.997 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 -- 172.21.15.63:0/3430577501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcdc80b2a70 msgr2=0x7fcdc80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcdc80b2a70 0x7fcdc80b4f80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55f485d1c790 tx=0x7fcdd400b410 comp rx=0 tx=0).stop 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 -- 172.21.15.63:0/3430577501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcde0108730 msgr2=0x7fcde01a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcde0108730 0x7fcde01a1820 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8007d70 tx=0x7fcdd800a340 comp rx=0 tx=0).stop 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 -- 172.21.15.63:0/3430577501 shutdown_connections 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcdc80b2a70 0x7fcdc80b4f80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcde0108730 0x7fcde01a1820 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde01034b0 0x7fcde019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 --2- 172.21.15.63:0/3430577501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0107de0 0x7fcde019d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 -- 172.21.15.63:0/3430577501 >> 172.21.15.63:0/3430577501 conn(0x7fcde00fedb0 msgr2=0x7fcde0109830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 -- 172.21.15.63:0/3430577501 shutdown_connections 2024-05-27T07:39:46.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:46.999+0000 7fcde8400700 1 -- 172.21.15.63:0/3430577501 wait complete. 2024-05-27T07:39:47.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:47 smithi086 ceph-mon[34687]: pgmap v189: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:39:47.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:47 smithi149 ceph-mon[35408]: pgmap v189: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:39:47.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:47 smithi063 ceph-mon[31613]: pgmap v189: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:39:48.957 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:39:48.958 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:39:49.490 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:39:49.657 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:49 smithi063 ceph-mon[31613]: pgmap v190: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 17 op/s 2024-05-27T07:39:49.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:49 smithi086 ceph-mon[34687]: pgmap v190: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 17 op/s 2024-05-27T07:39:49.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:49 smithi149 ceph-mon[35408]: pgmap v190: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 17 op/s 2024-05-27T07:39:51.637 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:51 smithi063 ceph-mon[31613]: pgmap v191: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:39:51.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:51 smithi086 ceph-mon[34687]: pgmap v191: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:39:51.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:51 smithi149 ceph-mon[35408]: pgmap v191: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:39:51.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.902+0000 7f3bee200700 1 -- 172.21.15.63:0/549158582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3be8105320 msgr2=0x7f3be8105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:51.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.902+0000 7f3bee200700 1 --2- 172.21.15.63:0/549158582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3be8105320 0x7f3be8105730 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009990 tx=0x7f3bdc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.902+0000 7f3bee200700 1 -- 172.21.15.63:0/549158582 shutdown_connections 2024-05-27T07:39:51.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.902+0000 7f3bee200700 1 --2- 172.21.15.63:0/549158582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be810e700 0x7f3be8110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.902+0000 7f3bee200700 1 --2- 172.21.15.63:0/549158582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3be8105320 0x7f3be8105730 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.902+0000 7f3bee200700 1 --2- 172.21.15.63:0/549158582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be8105c70 0x7f3be810e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.902+0000 7f3bee200700 1 -- 172.21.15.63:0/549158582 >> 172.21.15.63:0/549158582 conn(0x7f3be8078ab0 msgr2=0x7f3be8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:51.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.902+0000 7f3bee200700 1 -- 172.21.15.63:0/549158582 shutdown_connections 2024-05-27T07:39:51.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.902+0000 7f3bee200700 1 -- 172.21.15.63:0/549158582 wait complete. 2024-05-27T07:39:51.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.904+0000 7f3bee200700 1 Processor -- start 2024-05-27T07:39:51.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.904+0000 7f3bee200700 1 -- start start 2024-05-27T07:39:51.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.905+0000 7f3bee200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be8105320 0x7f3be8103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.905+0000 7f3bee200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3be8105c70 0x7f3be8103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.905+0000 7f3bee200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be810e700 0x7f3be8103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.905+0000 7f3bee200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3be8114140 con 0x7f3be8105c70 2024-05-27T07:39:51.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.905+0000 7f3bee200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3be8113fc0 con 0x7f3be810e700 2024-05-27T07:39:51.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.905+0000 7f3bee200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81142c0 con 0x7f3be8105320 2024-05-27T07:39:51.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.905+0000 7f3be7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3be8105c70 0x7f3be8103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.905+0000 7f3be7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3be8105c70 0x7f3be8103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45016/0 (socket says 172.21.15.63:45016) 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.905+0000 7f3be7e00700 1 -- 172.21.15.63:0/3768072832 learned_addr learned my addr 172.21.15.63:0/3768072832 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.905+0000 7f3bece00700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be8105320 0x7f3be8103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.906+0000 7f3bed800700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be810e700 0x7f3be8103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.906+0000 7f3be7e00700 1 -- 172.21.15.63:0/3768072832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be8105320 msgr2=0x7f3be8103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.906+0000 7f3be7e00700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be8105320 0x7f3be8103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.906+0000 7f3be7e00700 1 -- 172.21.15.63:0/3768072832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be810e700 msgr2=0x7f3be8103a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.906+0000 7f3be7e00700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be810e700 0x7f3be8103a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.906+0000 7f3be7e00700 1 -- 172.21.15.63:0/3768072832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be81a8e40 con 0x7f3be8105c70 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.906+0000 7f3bed800700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be810e700 0x7f3be8103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.906+0000 7f3be7e00700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3be8105c70 0x7f3be8103550 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400c9b0 tx=0x7f3bd400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.907+0000 7f3be5600700 1 -- 172.21.15.63:0/3768072832 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4007b00 con 0x7f3be8105c70 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.907+0000 7f3bee200700 1 -- 172.21.15.63:0/3768072832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be81a9130 con 0x7f3be8105c70 2024-05-27T07:39:51.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.907+0000 7f3be5600700 1 -- 172.21.15.63:0/3768072832 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bd4007ca0 con 0x7f3be8105c70 2024-05-27T07:39:51.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.907+0000 7f3bee200700 1 -- 172.21.15.63:0/3768072832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be81a96c0 con 0x7f3be8105c70 2024-05-27T07:39:51.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.907+0000 7f3be5600700 1 -- 172.21.15.63:0/3768072832 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4004a30 con 0x7f3be8105c70 2024-05-27T07:39:51.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.913+0000 7f3be5600700 1 -- 172.21.15.63:0/3768072832 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3bd4004c90 con 0x7f3be8105c70 2024-05-27T07:39:51.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.913+0000 7f3bfe3dab80 1 Processor -- start 2024-05-27T07:39:51.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.914+0000 7f3bfe3dab80 1 -- start start 2024-05-27T07:39:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.914+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea32b7a70 0x561ea334b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.914+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea31cbec0 0x561ea334bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.914+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 0x561ea3357760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.914+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561ea3278ec0 con 0x561ea31cbec0 2024-05-27T07:39:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.914+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ea3268a60 con 0x561ea32b7a70 2024-05-27T07:39:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.914+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561ea326caa0 con 0x561ea3355340 2024-05-27T07:39:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3bece00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea31cbec0 0x561ea334bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3bece00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea31cbec0 0x561ea334bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45028/0 (socket says 172.21.15.63:45028) 2024-05-27T07:39:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3bece00700 1 -- 172.21.15.63:0/1093134160 learned_addr learned my addr 172.21.15.63:0/1093134160 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3bed800700 1 --2- 172.21.15.63:0/1093134160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea32b7a70 0x561ea334b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3be7e00700 1 --2- 172.21.15.63:0/1093134160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 0x561ea3357760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3bece00700 1 -- 172.21.15.63:0/1093134160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 msgr2=0x561ea3357760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3bece00700 1 --2- 172.21.15.63:0/1093134160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 0x561ea3357760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3be5600700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd00b29f0 0x7f3bd00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3bece00700 1 -- 172.21.15.63:0/1093134160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea32b7a70 msgr2=0x561ea334b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3bece00700 1 --2- 172.21.15.63:0/1093134160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea32b7a70 0x561ea334b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.915+0000 7f3bece00700 1 -- 172.21.15.63:0/1093134160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea32314a0 con 0x561ea31cbec0 2024-05-27T07:39:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.916+0000 7f3be5600700 1 -- 172.21.15.63:0/3768072832 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f3bd40e2a20 con 0x7f3be8105c70 2024-05-27T07:39:51.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.916+0000 7f3bece00700 1 --2- 172.21.15.63:0/1093134160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea31cbec0 0x561ea334bd10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc005f50 tx=0x7f3bdc009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3aa6d94056eea0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:51.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.916+0000 7f3bed800700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd00b29f0 0x7f3bd00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.916+0000 7f3bda000700 1 -- 172.21.15.63:0/1093134160 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc027070 con 0x561ea31cbec0 2024-05-27T07:39:51.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.916+0000 7f3bda000700 1 -- 172.21.15.63:0/1093134160 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bdc00bc60 con 0x561ea31cbec0 2024-05-27T07:39:51.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.916+0000 7f3bda000700 1 -- 172.21.15.63:0/1093134160 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc022d80 con 0x561ea31cbec0 2024-05-27T07:39:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.917+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1093134160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea31cbec0 msgr2=0x561ea334bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.917+0000 7f3bed800700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd00b29f0 0x7f3bd00b4f00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x561ea3347b10 tx=0x7f3be0004b40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.917+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/1093134160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea31cbec0 0x561ea334bd10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc005f50 tx=0x7f3bdc009f00 comp rx=0 tx=0).stop 2024-05-27T07:39:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.918+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1093134160 shutdown_connections 2024-05-27T07:39:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.918+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/1093134160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 0x561ea3357760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.918+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/1093134160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea32b7a70 0x561ea334b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.918+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/1093134160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea31cbec0 0x561ea334bd10 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.918+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1093134160 >> 172.21.15.63:0/1093134160 conn(0x561ea3265410 msgr2=0x561ea3346790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.918+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1093134160 shutdown_connections 2024-05-27T07:39:51.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.918+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1093134160 wait complete. 2024-05-27T07:39:51.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.919+0000 7f3bfe3dab80 1 -- start start 2024-05-27T07:39:51.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.919+0000 7f3bfe3dab80 1 Processor -- start 2024-05-27T07:39:51.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea31cbec0 0x561ea33522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea32b7a70 0x561ea3352820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea3355340 0x561ea3352d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561ea316f020 con 0x561ea3355340 2024-05-27T07:39:51.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ea32097e0 con 0x561ea32b7a70 2024-05-27T07:39:51.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561ea3357e20 con 0x561ea31cbec0 2024-05-27T07:39:51.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bfe3dab80 10 client.0 _ll_register_callbacks cb 0x561ea3212cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:51.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bed800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea3355340 0x561ea3352d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bece00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea31cbec0 0x561ea33522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bed800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea3355340 0x561ea3352d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45032/0 (socket says 172.21.15.63:45032) 2024-05-27T07:39:51.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3bed800700 1 -- 172.21.15.63:0/1937848521 learned_addr learned my addr 172.21.15.63:0/1937848521 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:51.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.920+0000 7f3be7e00700 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea32b7a70 0x561ea3352820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.921+0000 7f3bed800700 1 -- 172.21.15.63:0/1937848521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea31cbec0 msgr2=0x561ea33522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:51.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.921+0000 7f3bed800700 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea31cbec0 0x561ea33522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.921+0000 7f3bed800700 1 -- 172.21.15.63:0/1937848521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea32b7a70 msgr2=0x561ea3352820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:51.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.921+0000 7f3bed800700 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea32b7a70 0x561ea3352820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.921+0000 7f3bed800700 1 -- 172.21.15.63:0/1937848521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea31b8950 con 0x561ea3355340 2024-05-27T07:39:51.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.921+0000 7f3bed800700 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea3355340 0x561ea3352d60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3be0015f90 tx=0x7f3be0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa1d6d315b7ccbe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:51.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.922+0000 7f3bcea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:51.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.922+0000 7f3bcea00700 1 -- 172.21.15.63:0/1937848521 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0018b10 con 0x561ea3355340 2024-05-27T07:39:51.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.922+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1937848521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bdc00bc60 con 0x561ea3355340 2024-05-27T07:39:51.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.922+0000 7f3bcea00700 1 -- 172.21.15.63:0/1937848521 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f3be0018cb0 con 0x561ea3355340 2024-05-27T07:39:51.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.922+0000 7f3bcea00700 1 -- 172.21.15.63:0/1937848521 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3be0030050 con 0x561ea3355340 2024-05-27T07:39:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.922+0000 7f3bfe3dab80 10 client.16632 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.922+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1937848521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561ea32390e0 con 0x561ea3355340 2024-05-27T07:39:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.923+0000 7f3bcea00700 1 -- 172.21.15.63:0/1937848521 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bdc00bc60 con 0x561ea3355340 2024-05-27T07:39:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.923+0000 7f3bcea00700 1 -- 172.21.15.63:0/1937848521 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3be002b3b0 con 0x561ea3355340 2024-05-27T07:39:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.923+0000 7f3bfe3dab80 10 client.16632 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.923+0000 7f3bfe3dab80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.923+0000 7f3bfe3dab80 20 client.16632 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.923+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ea3364d10 0x561ea3367190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.923+0000 7f3bfe3dab80 4 client.16632 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.923+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1937848521 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ea327ae10 con 0x561ea3364d10 2024-05-27T07:39:51.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.924+0000 7f3bece00700 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ea3364d10 0x561ea3367190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.925+0000 7f3bece00700 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ea3364d10 0x561ea3367190 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4770e373eb076b69 server_cookie=c1a801e6738c959d in_seq=0 out_seq=0 2024-05-27T07:39:51.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.925+0000 7f3bcea00700 10 client.16632 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:51.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bcea00700 1 -- 172.21.15.63:0/1937848521 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561ea327ae10 con 0x561ea3364d10 2024-05-27T07:39:51.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bcea00700 10 client.16632 handle_command_reply: tid=0 2024-05-27T07:39:51.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bfe3dab80 1 client.16632 shutdown 2024-05-27T07:39:51.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1937848521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea3355340 msgr2=0x561ea3352d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:51.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea3355340 0x561ea3352d60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3be0015f90 tx=0x7f3be0004ad0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1937848521 shutdown_connections 2024-05-27T07:39:51.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ea3364d10 0x561ea3367190 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea3355340 0x561ea3352d60 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea31cbec0 0x561ea33522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/1937848521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea32b7a70 0x561ea3352820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.927+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1937848521 >> 172.21.15.63:0/1937848521 conn(0x561ea3265410 msgr2=0x561ea3346380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:51.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.928+0000 7f3bcea00700 0 client.16632 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:51.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.928+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1937848521 shutdown_connections 2024-05-27T07:39:51.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.928+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/1937848521 wait complete. 2024-05-27T07:39:51.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.928+0000 7f3bfe3dab80 20 client.16632 trim_cache size 0 max 16384 2024-05-27T07:39:51.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.951+0000 7f3bfe3dab80 1 Processor -- start 2024-05-27T07:39:51.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.951+0000 7f3bfe3dab80 1 -- start start 2024-05-27T07:39:51.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.952+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea31cbec0 0x561ea334db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.952+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3351490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.952+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 0x561ea3360ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.952+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b4b20 con 0x561ea32b7a70 2024-05-27T07:39:51.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.952+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b23b0 con 0x561ea31cbec0 2024-05-27T07:39:51.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.952+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b0200 con 0x561ea3355340 2024-05-27T07:39:51.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.952+0000 7f3be7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3351490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.952+0000 7f3be7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3351490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45038/0 (socket says 172.21.15.63:45038) 2024-05-27T07:39:51.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.952+0000 7f3be7e00700 1 -- 172.21.15.63:0/3359014403 learned_addr learned my addr 172.21.15.63:0/3359014403 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:51.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.952+0000 7f3bece00700 1 --2- 172.21.15.63:0/3359014403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea31cbec0 0x561ea334db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.953+0000 7f3bed800700 1 --2- 172.21.15.63:0/3359014403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 0x561ea3360ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.953+0000 7f3be7e00700 1 -- 172.21.15.63:0/3359014403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 msgr2=0x561ea3360ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:51.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.953+0000 7f3be7e00700 1 --2- 172.21.15.63:0/3359014403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 0x561ea3360ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.953+0000 7f3be7e00700 1 -- 172.21.15.63:0/3359014403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea31cbec0 msgr2=0x561ea334db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:51.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.953+0000 7f3be7e00700 1 --2- 172.21.15.63:0/3359014403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea31cbec0 0x561ea334db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.953+0000 7f3be7e00700 1 -- 172.21.15.63:0/3359014403 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea3346cd0 con 0x561ea32b7a70 2024-05-27T07:39:51.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.953+0000 7f3bed800700 1 --2- 172.21.15.63:0/3359014403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 0x561ea3360ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:39:51.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.953+0000 7f3be7e00700 1 --2- 172.21.15.63:0/3359014403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3351490 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400cfd0 tx=0x7f3bd40e2d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95da0a3ae4458b4f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:51.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.954+0000 7f3bd9600700 1 -- 172.21.15.63:0/3359014403 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40a5120 con 0x561ea32b7a70 2024-05-27T07:39:51.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.954+0000 7f3bd9600700 1 -- 172.21.15.63:0/3359014403 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bd4007e50 con 0x561ea32b7a70 2024-05-27T07:39:51.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.954+0000 7f3bd9600700 1 -- 172.21.15.63:0/3359014403 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40b27f0 con 0x561ea32b7a70 2024-05-27T07:39:51.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.955+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/3359014403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 msgr2=0x561ea3351490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:51.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.955+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/3359014403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3351490 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400cfd0 tx=0x7f3bd40e2d20 comp rx=0 tx=0).stop 2024-05-27T07:39:51.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.955+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/3359014403 shutdown_connections 2024-05-27T07:39:51.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.955+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/3359014403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea3355340 0x561ea3360ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.955+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/3359014403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea31cbec0 0x561ea334db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.955+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/3359014403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3351490 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.955+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/3359014403 >> 172.21.15.63:0/3359014403 conn(0x561ea3265410 msgr2=0x561ea334a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:51.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.955+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/3359014403 shutdown_connections 2024-05-27T07:39:51.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.955+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/3359014403 wait complete. 2024-05-27T07:39:51.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.956+0000 7f3bfe3dab80 1 -- start start 2024-05-27T07:39:51.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.956+0000 7f3bfe3dab80 1 Processor -- start 2024-05-27T07:39:51.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.957+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea31cbec0 0x561ea335d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.957+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3358b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.957+0000 7f3bfe3dab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea3355340 0x561ea3359130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.957+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b0870 con 0x561ea32b7a70 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.957+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b1ce0 con 0x561ea3355340 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.957+0000 7f3bfe3dab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81b0f40 con 0x561ea31cbec0 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.957+0000 7f3bfe3dab80 10 client.0 _ll_register_callbacks cb 0x561ea3212cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.957+0000 7f3be7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3358b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.957+0000 7f3be7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3358b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45048/0 (socket says 172.21.15.63:45048) 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.957+0000 7f3be7e00700 1 -- 172.21.15.63:0/4181869928 learned_addr learned my addr 172.21.15.63:0/4181869928 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.958+0000 7f3bed800700 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea3355340 0x561ea3359130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.958+0000 7f3bece00700 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea31cbec0 0x561ea335d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.958+0000 7f3be7e00700 1 -- 172.21.15.63:0/4181869928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea31cbec0 msgr2=0x561ea335d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:51.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.958+0000 7f3be7e00700 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea31cbec0 0x561ea335d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.958+0000 7f3be7e00700 1 -- 172.21.15.63:0/4181869928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea3355340 msgr2=0x561ea3359130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:51.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.958+0000 7f3be7e00700 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea3355340 0x561ea3359130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.958+0000 7f3be7e00700 1 -- 172.21.15.63:0/4181869928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea33598d0 con 0x561ea32b7a70 2024-05-27T07:39:51.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.958+0000 7f3be7e00700 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3358b90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40e0400 tx=0x7f3bd40165e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7357f26858794a44 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:51.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.958+0000 7f3bcea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:51.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.958+0000 7f3bcea00700 1 -- 172.21.15.63:0/4181869928 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40a94a0 con 0x561ea32b7a70 2024-05-27T07:39:51.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.959+0000 7f3bcea00700 1 -- 172.21.15.63:0/4181869928 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f3bd40a9bf0 con 0x561ea32b7a70 2024-05-27T07:39:51.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.959+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/4181869928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd4007e50 con 0x561ea32b7a70 2024-05-27T07:39:51.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.959+0000 7f3bcea00700 1 -- 172.21.15.63:0/4181869928 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3bd40aa090 con 0x561ea32b7a70 2024-05-27T07:39:51.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.959+0000 7f3bfe3dab80 10 client.16638 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:51.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.959+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/4181869928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561ea33cf760 con 0x561ea32b7a70 2024-05-27T07:39:51.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.959+0000 7f3bcea00700 1 -- 172.21.15.63:0/4181869928 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bd4007e50 con 0x561ea32b7a70 2024-05-27T07:39:51.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.959+0000 7f3bcea00700 1 -- 172.21.15.63:0/4181869928 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3bd406f200 con 0x561ea32b7a70 2024-05-27T07:39:51.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.960+0000 7f3bfe3dab80 10 client.16638 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:51.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.960+0000 7f3bfe3dab80 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:51.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.960+0000 7f3bfe3dab80 20 client.16638 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:51.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.960+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ea33cfd00 0x561ea33d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:51.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.960+0000 7f3bfe3dab80 4 client.16638 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:39:51.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.960+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/4181869928 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ea32745f0 con 0x561ea33cfd00 2024-05-27T07:39:51.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.960+0000 7f3bece00700 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ea33cfd00 0x561ea33d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:51.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.961+0000 7f3bece00700 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ea33cfd00 0x561ea33d2120 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d94577895d03ca server_cookie=1a046958472f196c in_seq=0 out_seq=0 2024-05-27T07:39:51.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.961+0000 7f3bcea00700 10 client.16638 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:51.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.963+0000 7f3bcea00700 1 -- 172.21.15.63:0/4181869928 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ea32745f0 con 0x561ea33cfd00 2024-05-27T07:39:51.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.963+0000 7f3bcea00700 10 client.16638 handle_command_reply: tid=0 2024-05-27T07:39:51.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.963+0000 7f3bfe3dab80 1 client.16638 shutdown 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.964+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/4181869928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 msgr2=0x561ea3358b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.964+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3358b90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40e0400 tx=0x7f3bd40165e0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.964+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/4181869928 shutdown_connections 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.964+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ea33cfd00 0x561ea33d2120 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.964+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ea3355340 0x561ea3359130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.964+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ea31cbec0 0x561ea335d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.964+0000 7f3bfe3dab80 1 --2- 172.21.15.63:0/4181869928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ea32b7a70 0x561ea3358b90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.964+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/4181869928 >> 172.21.15.63:0/4181869928 conn(0x561ea3265410 msgr2=0x561ea334d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.964+0000 7f3bcea00700 0 client.16638 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.965+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/4181869928 shutdown_connections 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.965+0000 7f3bfe3dab80 1 -- 172.21.15.63:0/4181869928 wait complete. 2024-05-27T07:39:51.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.965+0000 7f3bfe3dab80 20 client.16638 trim_cache size 0 max 16384 2024-05-27T07:39:51.964 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:39:51.964 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:39:51.964 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "57ed263c-0f04-4b6f-9d3f-fbdedb61bd6b", 2024-05-27T07:39:51.964 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:39:51.964 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:39:51.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 -- 172.21.15.63:0/3768072832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd00b29f0 msgr2=0x7f3bd00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:51.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd00b29f0 0x7f3bd00b4f00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x561ea3347b10 tx=0x7f3be0004b40 comp rx=0 tx=0).stop 2024-05-27T07:39:51.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 -- 172.21.15.63:0/3768072832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3be8105c70 msgr2=0x7f3be8103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:51.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3be8105c70 0x7f3be8103550 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400c9b0 tx=0x7f3bd400cd70 comp rx=0 tx=0).stop 2024-05-27T07:39:51.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 -- 172.21.15.63:0/3768072832 shutdown_connections 2024-05-27T07:39:51.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd00b29f0 0x7f3bd00b4f00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be810e700 0x7f3be8103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be8105320 0x7f3be8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 --2- 172.21.15.63:0/3768072832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3be8105c70 0x7f3be8103550 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:51.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 -- 172.21.15.63:0/3768072832 >> 172.21.15.63:0/3768072832 conn(0x7f3be8078ab0 msgr2=0x7f3be80ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:51.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 -- 172.21.15.63:0/3768072832 shutdown_connections 2024-05-27T07:39:51.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:51.966+0000 7f3bee200700 1 -- 172.21.15.63:0/3768072832 wait complete. 2024-05-27T07:39:52.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:52 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:39:52.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:52 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:52.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:52 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:39:52.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:52 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:52.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:52 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:39:52.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:52 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:52.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:52 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:39:52.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:52 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:52.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:52 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:39:52.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:52 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:52.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:52 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:39:52.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:52 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:52.874 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:39:53.370 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:39:53.697 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: pgmap v192: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:39:53.697 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:39:53.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:53 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: pgmap v192: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:39:53.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:39:53.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:39:53.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:39:53.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:39:53.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:39:53.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:39:53.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:53 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: pgmap v192: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:39:53.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:39:53.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:39:53.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:39:53.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:39:53.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:39:53.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:53.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:39:53.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:53 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:39:55.665 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:55 smithi063 ceph-mon[31613]: pgmap v193: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:39:55.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:55 smithi086 ceph-mon[34687]: pgmap v193: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:39:55.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:55 smithi149 ceph-mon[35408]: pgmap v193: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:39:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.823+0000 7f29c4e00700 1 -- 172.21.15.63:0/3549650177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c00764a0 msgr2=0x7f29c0114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.823+0000 7f29c4e00700 1 --2- 172.21.15.63:0/3549650177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c00764a0 0x7f29c0114e20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009990 tx=0x7f29ac009ca0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.823+0000 7f29c4e00700 1 -- 172.21.15.63:0/3549650177 shutdown_connections 2024-05-27T07:39:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.823+0000 7f29c4e00700 1 --2- 172.21.15.63:0/3549650177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c0115360 0x7f29c0117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.823+0000 7f29c4e00700 1 --2- 172.21.15.63:0/3549650177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c0075b50 0x7f29c0075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.823+0000 7f29c4e00700 1 --2- 172.21.15.63:0/3549650177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c00764a0 0x7f29c0114e20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.823+0000 7f29c4e00700 1 -- 172.21.15.63:0/3549650177 >> 172.21.15.63:0/3549650177 conn(0x7f29c0100030 msgr2=0x7f29c0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.823+0000 7f29c4e00700 1 -- 172.21.15.63:0/3549650177 shutdown_connections 2024-05-27T07:39:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.823+0000 7f29c4e00700 1 -- 172.21.15.63:0/3549650177 wait complete. 2024-05-27T07:39:55.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.825+0000 7f29c4e00700 1 Processor -- start 2024-05-27T07:39:55.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.825+0000 7f29c4e00700 1 -- start start 2024-05-27T07:39:55.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.826+0000 7f29c4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c0075b50 0x7f29c01a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.826+0000 7f29c4e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c00764a0 0x7f29c01a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.826+0000 7f29c4e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c0115360 0x7f29c01a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.826+0000 7f29c4e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29c011a960 con 0x7f29c00764a0 2024-05-27T07:39:55.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.826+0000 7f29c4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29c011a7e0 con 0x7f29c0075b50 2024-05-27T07:39:55.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.826+0000 7f29c4e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29c011aae0 con 0x7f29c0115360 2024-05-27T07:39:55.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.826+0000 7f29bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c00764a0 0x7f29c01a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.826+0000 7f29bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c00764a0 0x7f29c01a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45062/0 (socket says 172.21.15.63:45062) 2024-05-27T07:39:55.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.826+0000 7f29bd400700 1 -- 172.21.15.63:0/773272742 learned_addr learned my addr 172.21.15.63:0/773272742 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:55.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.827+0000 7f29bde00700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c0075b50 0x7f29c01a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.827+0000 7f29be800700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c0115360 0x7f29c01a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.827+0000 7f29bd400700 1 -- 172.21.15.63:0/773272742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c0115360 msgr2=0x7f29c01a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:55.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.827+0000 7f29bd400700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c0115360 0x7f29c01a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.827+0000 7f29bd400700 1 -- 172.21.15.63:0/773272742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c0075b50 msgr2=0x7f29c01a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:55.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.827+0000 7f29bd400700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c0075b50 0x7f29c01a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.827+0000 7f29bd400700 1 -- 172.21.15.63:0/773272742 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c01a6260 con 0x7f29c00764a0 2024-05-27T07:39:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.828+0000 7f29bd400700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c00764a0 0x7f29c01a1930 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f29ac005910 tx=0x7f29ac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.828+0000 7f29b6a00700 1 -- 172.21.15.63:0/773272742 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac004830 con 0x7f29c00764a0 2024-05-27T07:39:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.828+0000 7f29b6a00700 1 -- 172.21.15.63:0/773272742 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29ac005020 con 0x7f29c00764a0 2024-05-27T07:39:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.828+0000 7f29c4e00700 1 -- 172.21.15.63:0/773272742 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c01a64f0 con 0x7f29c00764a0 2024-05-27T07:39:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.828+0000 7f29b6a00700 1 -- 172.21.15.63:0/773272742 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00f960 con 0x7f29c00764a0 2024-05-27T07:39:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.828+0000 7f29c4e00700 1 -- 172.21.15.63:0/773272742 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c01a6970 con 0x7f29c00764a0 2024-05-27T07:39:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.832+0000 7f29b6a00700 1 -- 172.21.15.63:0/773272742 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f29ac0054d0 con 0x7f29c00764a0 2024-05-27T07:39:55.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.834+0000 7f29b6a00700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a40b29f0 0x7f29a40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.834+0000 7f29b6a00700 1 -- 172.21.15.63:0/773272742 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f29ac0e3880 con 0x7f29c00764a0 2024-05-27T07:39:55.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.834+0000 7f29bde00700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a40b29f0 0x7f29a40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.835+0000 7f29d3949b80 1 Processor -- start 2024-05-27T07:39:55.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.835+0000 7f29d3949b80 1 -- start start 2024-05-27T07:39:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.835+0000 7f29bde00700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a40b29f0 0x7f29a40b4f00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005950 tx=0x7f29b000b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.835+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807b29e40 0x559807cae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.835+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807ca8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.835+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 0x559807ca91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.835+0000 7f29d3949b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559807bd6e40 con 0x559807caed30 2024-05-27T07:39:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.835+0000 7f29d3949b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559807bc69e0 con 0x559807ca8db0 2024-05-27T07:39:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.835+0000 7f29d3949b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559807bcaa20 con 0x559807b29e40 2024-05-27T07:39:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.836+0000 7f29bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807ca8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.836+0000 7f29bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807ca8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45068/0 (socket says 172.21.15.63:45068) 2024-05-27T07:39:55.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.836+0000 7f29bd400700 1 -- 172.21.15.63:0/3502387276 learned_addr learned my addr 172.21.15.63:0/3502387276 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:55.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.836+0000 7f29bde00700 1 --2- 172.21.15.63:0/3502387276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807b29e40 0x559807cae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.836+0000 7f29be800700 1 --2- 172.21.15.63:0/3502387276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 0x559807ca91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.836+0000 7f29bd400700 1 -- 172.21.15.63:0/3502387276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807b29e40 msgr2=0x559807cae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:55.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.836+0000 7f29bd400700 1 --2- 172.21.15.63:0/3502387276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807b29e40 0x559807cae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.836+0000 7f29bd400700 1 -- 172.21.15.63:0/3502387276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 msgr2=0x559807ca91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:55.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.837+0000 7f29bd400700 1 --2- 172.21.15.63:0/3502387276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 0x559807ca91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.837+0000 7f29bd400700 1 -- 172.21.15.63:0/3502387276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559807b8f420 con 0x559807caed30 2024-05-27T07:39:55.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.837+0000 7f29be800700 1 --2- 172.21.15.63:0/3502387276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 0x559807ca91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:55.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.837+0000 7f29bd400700 1 --2- 172.21.15.63:0/3502387276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807ca8870 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0aa7b0 tx=0x7f29ac0aa700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24eb119ff5f8a16b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:55.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.837+0000 7f29ab400700 1 -- 172.21.15.63:0/3502387276 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0cd0b0 con 0x559807caed30 2024-05-27T07:39:55.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.837+0000 7f29ab400700 1 -- 172.21.15.63:0/3502387276 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29ac0aa820 con 0x559807caed30 2024-05-27T07:39:55.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.837+0000 7f29ab400700 1 -- 172.21.15.63:0/3502387276 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0e3650 con 0x559807caed30 2024-05-27T07:39:55.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.838+0000 7f29d3949b80 1 -- 172.21.15.63:0/3502387276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 msgr2=0x559807ca8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:55.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.838+0000 7f29d3949b80 1 --2- 172.21.15.63:0/3502387276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807ca8870 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0aa7b0 tx=0x7f29ac0aa700 comp rx=0 tx=0).stop 2024-05-27T07:39:55.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.839+0000 7f29d3949b80 1 -- 172.21.15.63:0/3502387276 shutdown_connections 2024-05-27T07:39:55.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.839+0000 7f29d3949b80 1 --2- 172.21.15.63:0/3502387276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 0x559807ca91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.839+0000 7f29d3949b80 1 --2- 172.21.15.63:0/3502387276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807b29e40 0x559807cae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.839+0000 7f29d3949b80 1 --2- 172.21.15.63:0/3502387276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807ca8870 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.839+0000 7f29d3949b80 1 -- 172.21.15.63:0/3502387276 >> 172.21.15.63:0/3502387276 conn(0x559807b89fc0 msgr2=0x559807ca38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:55.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.839+0000 7f29d3949b80 1 -- 172.21.15.63:0/3502387276 shutdown_connections 2024-05-27T07:39:55.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.839+0000 7f29d3949b80 1 -- 172.21.15.63:0/3502387276 wait complete. 2024-05-27T07:39:55.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.840+0000 7f29d3949b80 1 -- start start 2024-05-27T07:39:55.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.840+0000 7f29d3949b80 1 Processor -- start 2024-05-27T07:39:55.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807b29e40 0x559807cb25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807ca8db0 0x559807cb2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cb3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29d3949b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559807accfa0 con 0x559807caed30 2024-05-27T07:39:55.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29d3949b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559807b67760 con 0x559807b29e40 2024-05-27T07:39:55.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29d3949b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559807cb55c0 con 0x559807ca8db0 2024-05-27T07:39:55.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29d3949b80 10 client.0 _ll_register_callbacks cb 0x559807b70c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:55.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29be800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cb3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29bde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807b29e40 0x559807cb25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29be800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cb3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45074/0 (socket says 172.21.15.63:45074) 2024-05-27T07:39:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29be800700 1 -- 172.21.15.63:0/2242143896 learned_addr learned my addr 172.21.15.63:0/2242143896 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.841+0000 7f29bd400700 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807ca8db0 0x559807cb2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.842+0000 7f29bde00700 1 -- 172.21.15.63:0/2242143896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807ca8db0 msgr2=0x559807cb2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.842+0000 7f29bde00700 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807ca8db0 0x559807cb2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.842+0000 7f29bde00700 1 -- 172.21.15.63:0/2242143896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 msgr2=0x559807cb3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.842+0000 7f29bde00700 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cb3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.842+0000 7f29bde00700 1 -- 172.21.15.63:0/2242143896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559807b168d0 con 0x559807b29e40 2024-05-27T07:39:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.843+0000 7f29bde00700 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807b29e40 0x559807cb25d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f29b000ba80 tx=0x7f29b00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95103e3e668e5e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.843+0000 7f299fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:39:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.843+0000 7f299fe00700 1 -- 172.21.15.63:0/2242143896 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0028470 con 0x559807b29e40 2024-05-27T07:39:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.843+0000 7f299fe00700 1 -- 172.21.15.63:0/2242143896 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f29b0028610 con 0x559807b29e40 2024-05-27T07:39:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.843+0000 7f29d3949b80 1 -- 172.21.15.63:0/2242143896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29ac0aa820 con 0x559807b29e40 2024-05-27T07:39:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.844+0000 7f299fe00700 1 -- 172.21.15.63:0/2242143896 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f29b0028810 con 0x559807b29e40 2024-05-27T07:39:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.844+0000 7f29d3949b80 10 client.36272 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.844+0000 7f29d3949b80 1 -- 172.21.15.63:0/2242143896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559807b97060 con 0x559807b29e40 2024-05-27T07:39:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.844+0000 7f299fe00700 1 -- 172.21.15.63:0/2242143896 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29ac0aa820 con 0x559807b29e40 2024-05-27T07:39:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.845+0000 7f299fe00700 1 -- 172.21.15.63:0/2242143896 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f29b001f9e0 con 0x559807b29e40 2024-05-27T07:39:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.845+0000 7f29d3949b80 10 client.36272 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.845+0000 7f29d3949b80 10 client.36272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.845+0000 7f29d3949b80 20 client.36272 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.845+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559807cc2550 0x559807cc4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.845+0000 7f29d3949b80 4 client.36272 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.845+0000 7f29d3949b80 1 -- 172.21.15.63:0/2242143896 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559807bd8d90 con 0x559807cc2550 2024-05-27T07:39:55.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.846+0000 7f29bd400700 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559807cc2550 0x559807cc4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.847+0000 7f29bd400700 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559807cc2550 0x559807cc4970 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128445c189ee8314 server_cookie=86ccbce03bd78e90 in_seq=0 out_seq=0 2024-05-27T07:39:55.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.847+0000 7f299fe00700 10 client.36272 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:55.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f299fe00700 1 -- 172.21.15.63:0/2242143896 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559807bd8d90 con 0x559807cc2550 2024-05-27T07:39:55.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f299fe00700 10 client.36272 handle_command_reply: tid=0 2024-05-27T07:39:55.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f29d3949b80 1 client.36272 shutdown 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f29d3949b80 1 -- 172.21.15.63:0/2242143896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807b29e40 msgr2=0x559807cb25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807b29e40 0x559807cb25d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f29b000ba80 tx=0x7f29b00047c0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f29d3949b80 1 -- 172.21.15.63:0/2242143896 shutdown_connections 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559807cc2550 0x559807cc4970 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cb3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807b29e40 0x559807cb25d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2242143896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807ca8db0 0x559807cb2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f29d3949b80 1 -- 172.21.15.63:0/2242143896 >> 172.21.15.63:0/2242143896 conn(0x559807b89fc0 msgr2=0x559807ca3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f299fe00700 0 client.36272 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.849+0000 7f29d3949b80 1 -- 172.21.15.63:0/2242143896 shutdown_connections 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.850+0000 7f29d3949b80 1 -- 172.21.15.63:0/2242143896 wait complete. 2024-05-27T07:39:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.850+0000 7f29d3949b80 20 client.36272 trim_cache size 0 max 16384 2024-05-27T07:39:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.882+0000 7f29d3949b80 1 Processor -- start 2024-05-27T07:39:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.882+0000 7f29d3949b80 1 -- start start 2024-05-27T07:39:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807b29e40 0x559807ca8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 0x559807cacb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cbe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29d3949b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01b9280 con 0x559807caed30 2024-05-27T07:39:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29d3949b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01b6b10 con 0x559807ca8db0 2024-05-27T07:39:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29d3949b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01b4960 con 0x559807b29e40 2024-05-27T07:39:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29be800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cbe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29bde00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807b29e40 0x559807ca8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29be800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cbe550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45080/0 (socket says 172.21.15.63:45080) 2024-05-27T07:39:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29be800700 1 -- 172.21.15.63:0/1407610723 learned_addr learned my addr 172.21.15.63:0/1407610723 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.883+0000 7f29bd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 0x559807cacb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.884+0000 7f29be800700 1 -- 172.21.15.63:0/1407610723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807b29e40 msgr2=0x559807ca8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.884+0000 7f29be800700 1 --2- 172.21.15.63:0/1407610723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807b29e40 0x559807ca8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.884+0000 7f29be800700 1 -- 172.21.15.63:0/1407610723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 msgr2=0x559807cacb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.884+0000 7f29be800700 1 --2- 172.21.15.63:0/1407610723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 0x559807cacb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.884+0000 7f29be800700 1 -- 172.21.15.63:0/1407610723 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559807ca44f0 con 0x559807caed30 2024-05-27T07:39:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.884+0000 7f29be800700 1 --2- 172.21.15.63:0/1407610723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cbe550 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009ed0 tx=0x7f29b80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8142f45e20690d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.885+0000 7f29aaa00700 1 -- 172.21.15.63:0/1407610723 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b801b070 con 0x559807caed30 2024-05-27T07:39:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.885+0000 7f29aaa00700 1 -- 172.21.15.63:0/1407610723 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29b8007e00 con 0x559807caed30 2024-05-27T07:39:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.885+0000 7f29aaa00700 1 -- 172.21.15.63:0/1407610723 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8005280 con 0x559807caed30 2024-05-27T07:39:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.886+0000 7f29d3949b80 1 -- 172.21.15.63:0/1407610723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 msgr2=0x559807cbe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.886+0000 7f29d3949b80 1 --2- 172.21.15.63:0/1407610723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cbe550 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009ed0 tx=0x7f29b80077e0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.886+0000 7f29d3949b80 1 -- 172.21.15.63:0/1407610723 shutdown_connections 2024-05-27T07:39:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.886+0000 7f29d3949b80 1 --2- 172.21.15.63:0/1407610723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807caed30 0x559807cbe550 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.886+0000 7f29d3949b80 1 --2- 172.21.15.63:0/1407610723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807b29e40 0x559807ca8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.886+0000 7f29d3949b80 1 --2- 172.21.15.63:0/1407610723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807ca8db0 0x559807cacb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.886+0000 7f29d3949b80 1 -- 172.21.15.63:0/1407610723 >> 172.21.15.63:0/1407610723 conn(0x559807b89fc0 msgr2=0x559807cb23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.886+0000 7f29d3949b80 1 -- 172.21.15.63:0/1407610723 shutdown_connections 2024-05-27T07:39:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.886+0000 7f29d3949b80 1 -- 172.21.15.63:0/1407610723 wait complete. 2024-05-27T07:39:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.887+0000 7f29d3949b80 1 -- start start 2024-05-27T07:39:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.887+0000 7f29d3949b80 1 Processor -- start 2024-05-27T07:39:55.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.887+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807b29e40 0x559807cba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.887+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807ca8db0 0x559807cbabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.887+0000 7f29d3949b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807caed30 0x559807cb7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.888+0000 7f29d3949b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01b4fd0 con 0x559807b29e40 2024-05-27T07:39:55.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.888+0000 7f29d3949b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01b6440 con 0x559807caed30 2024-05-27T07:39:55.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.888+0000 7f29d3949b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01b56a0 con 0x559807ca8db0 2024-05-27T07:39:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.888+0000 7f29d3949b80 10 client.0 _ll_register_callbacks cb 0x559807b70c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.888+0000 7f29bde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807b29e40 0x559807cba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.888+0000 7f29bde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807b29e40 0x559807cba6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45096/0 (socket says 172.21.15.63:45096) 2024-05-27T07:39:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.888+0000 7f29bde00700 1 -- 172.21.15.63:0/2972152924 learned_addr learned my addr 172.21.15.63:0/2972152924 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.888+0000 7f29bd400700 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807ca8db0 0x559807cbabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.888+0000 7f29be800700 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807caed30 0x559807cb7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.889+0000 7f29bde00700 1 -- 172.21.15.63:0/2972152924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807ca8db0 msgr2=0x559807cbabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:55.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.889+0000 7f29bde00700 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807ca8db0 0x559807cbabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.889+0000 7f29bde00700 1 -- 172.21.15.63:0/2972152924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807caed30 msgr2=0x559807cb7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:55.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.889+0000 7f29bde00700 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807caed30 0x559807cb7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.889+0000 7f29bde00700 1 -- 172.21.15.63:0/2972152924 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559807cbb250 con 0x559807b29e40 2024-05-27T07:39:55.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.889+0000 7f29be800700 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807caed30 0x559807cb7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:39:55.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.889+0000 7f29bde00700 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807b29e40 0x559807cba6a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f29b000ba80 tx=0x7f29b0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db72af7d9ef7a603 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:55.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.889+0000 7f299fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:55.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.889+0000 7f299fe00700 1 -- 172.21.15.63:0/2972152924 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b001f840 con 0x559807b29e40 2024-05-27T07:39:55.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.890+0000 7f299fe00700 1 -- 172.21.15.63:0/2972152924 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7f29b001f9e0 con 0x559807b29e40 2024-05-27T07:39:55.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.890+0000 7f29d3949b80 1 -- 172.21.15.63:0/2972152924 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b8007e00 con 0x559807b29e40 2024-05-27T07:39:55.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.890+0000 7f299fe00700 1 -- 172.21.15.63:0/2972152924 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f29b001e460 con 0x559807b29e40 2024-05-27T07:39:55.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.890+0000 7f29d3949b80 10 client.16656 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:55.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.890+0000 7f29d3949b80 1 -- 172.21.15.63:0/2972152924 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559807d2d4b0 con 0x559807b29e40 2024-05-27T07:39:55.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.890+0000 7f299fe00700 1 -- 172.21.15.63:0/2972152924 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29b8007e00 con 0x559807b29e40 2024-05-27T07:39:55.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.891+0000 7f299fe00700 1 -- 172.21.15.63:0/2972152924 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f29b003d410 con 0x559807b29e40 2024-05-27T07:39:55.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.891+0000 7f29d3949b80 10 client.16656 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:55.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.891+0000 7f29d3949b80 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:55.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.891+0000 7f29d3949b80 20 client.16656 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:55.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.891+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559807d2dc40 0x559807d30060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:55.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.891+0000 7f29d3949b80 4 client.16656 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:39:55.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.891+0000 7f29d3949b80 1 -- 172.21.15.63:0/2972152924 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559807bd2570 con 0x559807d2dc40 2024-05-27T07:39:55.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.891+0000 7f29bd400700 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559807d2dc40 0x559807d30060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:55.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.893+0000 7f299fe00700 10 client.16656 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:55.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.893+0000 7f29bd400700 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559807d2dc40 0x559807d30060 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9126ec926051ba48 server_cookie=2c5b2c00f9dfec16 in_seq=0 out_seq=0 2024-05-27T07:39:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.893+0000 7f299fe00700 1 -- 172.21.15.63:0/2972152924 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559807bd2570 con 0x559807d2dc40 2024-05-27T07:39:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.893+0000 7f299fe00700 10 client.16656 handle_command_reply: tid=0 2024-05-27T07:39:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.893+0000 7f29d3949b80 1 client.16656 shutdown 2024-05-27T07:39:55.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.893+0000 7f29d3949b80 1 -- 172.21.15.63:0/2972152924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807b29e40 msgr2=0x559807cba6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:55.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.893+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807b29e40 0x559807cba6a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f29b000ba80 tx=0x7f29b0013000 comp rx=0 tx=0).stop 2024-05-27T07:39:55.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.894+0000 7f29d3949b80 1 -- 172.21.15.63:0/2972152924 shutdown_connections 2024-05-27T07:39:55.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.894+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559807d2dc40 0x559807d30060 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.894+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559807caed30 0x559807cb7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.894+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559807b29e40 0x559807cba6a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.894+0000 7f29d3949b80 1 --2- 172.21.15.63:0/2972152924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559807ca8db0 0x559807cbabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.894+0000 7f299fe00700 0 client.16656 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:55.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.894+0000 7f29d3949b80 1 -- 172.21.15.63:0/2972152924 >> 172.21.15.63:0/2972152924 conn(0x559807b89fc0 msgr2=0x559807cb2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.894+0000 7f29d3949b80 1 -- 172.21.15.63:0/2972152924 shutdown_connections 2024-05-27T07:39:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.894+0000 7f29d3949b80 1 -- 172.21.15.63:0/2972152924 wait complete. 2024-05-27T07:39:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.894+0000 7f29d3949b80 20 client.16656 trim_cache size 0 max 16384 2024-05-27T07:39:55.893 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:39:55.893 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:39:55.893 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:39:55.893 INFO:teuthology.orchestra.run.smithi063.stdout: "57ed263c-0f04-4b6f-9d3f-fbdedb61bd6b": { 2024-05-27T07:39:55.893 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:39:55.893 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "57ed263c-0f04-4b6f-9d3f-fbdedb61bd6b", 2024-05-27T07:39:55.894 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:39:55.894 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:39:55.894 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:39:55.894 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:39:55.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.895+0000 7f29c4e00700 1 -- 172.21.15.63:0/773272742 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a40b29f0 msgr2=0x7f29a40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:55.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.895+0000 7f29c4e00700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a40b29f0 0x7f29a40b4f00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005950 tx=0x7f29b000b410 comp rx=0 tx=0).stop 2024-05-27T07:39:55.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.895+0000 7f29c4e00700 1 -- 172.21.15.63:0/773272742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c00764a0 msgr2=0x7f29c01a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:55.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.895+0000 7f29c4e00700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c00764a0 0x7f29c01a1930 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f29ac005910 tx=0x7f29ac0047b0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.895+0000 7f29c4e00700 1 -- 172.21.15.63:0/773272742 shutdown_connections 2024-05-27T07:39:55.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.895+0000 7f29c4e00700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a40b29f0 0x7f29a40b4f00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.895+0000 7f29c4e00700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c0115360 0x7f29c01a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.895+0000 7f29c4e00700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c0075b50 0x7f29c01a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.895+0000 7f29c4e00700 1 --2- 172.21.15.63:0/773272742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c00764a0 0x7f29c01a1930 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:55.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.895+0000 7f29c4e00700 1 -- 172.21.15.63:0/773272742 >> 172.21.15.63:0/773272742 conn(0x7f29c0100030 msgr2=0x7f29c0101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:55.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.896+0000 7f29c4e00700 1 -- 172.21.15.63:0/773272742 shutdown_connections 2024-05-27T07:39:55.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:39:55.896+0000 7f29c4e00700 1 -- 172.21.15.63:0/773272742 wait complete. 2024-05-27T07:39:56.876 INFO:tasks.cephfs.filesystem:scrub status for tag:57ed263c-0f04-4b6f-9d3f-fbdedb61bd6b - {'path': '/', 'tag': '57ed263c-0f04-4b6f-9d3f-fbdedb61bd6b', 'options': 'recursive,force'} 2024-05-27T07:39:57.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:57 smithi086 ceph-mon[34687]: pgmap v194: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:39:57.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:57 smithi149 ceph-mon[35408]: pgmap v194: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:39:57.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:57 smithi063 ceph-mon[31613]: pgmap v194: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:39:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.725+0000 7fed66600640 1 -- 172.21.15.63:0/962235090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed60109f70 msgr2=0x7fed60103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.725+0000 7fed66600640 1 --2- 172.21.15.63:0/962235090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed60109f70 0x7fed60103bb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009900 tx=0x7fed4c02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.725+0000 7fed66600640 1 -- 172.21.15.63:0/962235090 shutdown_connections 2024-05-27T07:39:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.725+0000 7fed66600640 1 --2- 172.21.15.63:0/962235090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60104220 0x7fed601105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.725+0000 7fed66600640 1 --2- 172.21.15.63:0/962235090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed60109f70 0x7fed60103bb0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.725+0000 7fed66600640 1 --2- 172.21.15.63:0/962235090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed60109ad0 0x7fed60103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.725+0000 7fed66600640 1 -- 172.21.15.63:0/962235090 >> 172.21.15.63:0/962235090 conn(0x7fed60079100 msgr2=0x7fed60079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.725+0000 7fed66600640 1 -- 172.21.15.63:0/962235090 shutdown_connections 2024-05-27T07:39:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.725+0000 7fed66600640 1 -- 172.21.15.63:0/962235090 wait complete. 2024-05-27T07:39:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.727+0000 7fed66600640 1 Processor -- start 2024-05-27T07:39:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.727+0000 7fed66600640 1 -- start start 2024-05-27T07:39:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.727+0000 7fed66600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60104220 0x7fed60074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.727+0000 7fed66600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed60109ad0 0x7fed60075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.727+0000 7fed66600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed60109f70 0x7fed60075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.728+0000 7fed66600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed60113b70 con 0x7fed60109f70 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.728+0000 7fed66600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed601139f0 con 0x7fed60104220 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.728+0000 7fed66600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed60113cf0 con 0x7fed60109ad0 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.728+0000 7fed65c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed60109f70 0x7fed60075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.728+0000 7fed65c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed60109f70 0x7fed60075a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45108/0 (socket says 172.21.15.63:45108) 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.728+0000 7fed65c00640 1 -- 172.21.15.63:0/3068303558 learned_addr learned my addr 172.21.15.63:0/3068303558 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.728+0000 7fed65200640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60104220 0x7fed60074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.728+0000 7fed5fe00640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed60109ad0 0x7fed60075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed65c00640 1 -- 172.21.15.63:0/3068303558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed60109ad0 msgr2=0x7fed60075500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed65c00640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed60109ad0 0x7fed60075500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed65c00640 1 -- 172.21.15.63:0/3068303558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60104220 msgr2=0x7fed60074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed65c00640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60104220 0x7fed60074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed65c00640 1 -- 172.21.15.63:0/3068303558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed601a8970 con 0x7fed60109f70 2024-05-27T07:39:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed65c00640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed60109f70 0x7fed60075a40 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fed5800eaa0 tx=0x7fed5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed5d600640 1 -- 172.21.15.63:0/3068303558 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5800cdb0 con 0x7fed60109f70 2024-05-27T07:39:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed5d600640 1 -- 172.21.15.63:0/3068303558 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed58005020 con 0x7fed60109f70 2024-05-27T07:39:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed66600640 1 -- 172.21.15.63:0/3068303558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed601a8c60 con 0x7fed60109f70 2024-05-27T07:39:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed5d600640 1 -- 172.21.15.63:0/3068303558 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed580116d0 con 0x7fed60109f70 2024-05-27T07:39:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.729+0000 7fed66600640 1 -- 172.21.15.63:0/3068303558 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed601a9130 con 0x7fed60109f70 2024-05-27T07:39:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.733+0000 7fed6ec10740 1 Processor -- start 2024-05-27T07:39:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.734+0000 7fed5d600640 1 -- 172.21.15.63:0/3068303558 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fed580051d0 con 0x7fed60109f70 2024-05-27T07:39:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.734+0000 7fed6ec10740 1 -- start start 2024-05-27T07:39:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017be1b0 0x556201b60a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b60fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed6ec10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556201ad1b90 con 0x55620179d4d0 2024-05-27T07:39:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed6ec10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556201ac1730 con 0x5562017952f0 2024-05-27T07:39:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed6ec10740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556201ac5770 con 0x5562017be1b0 2024-05-27T07:39:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed5fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed5fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45114/0 (socket says 172.21.15.63:45114) 2024-05-27T07:39:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed5fe00640 1 -- 172.21.15.63:0/2597578271 learned_addr learned my addr 172.21.15.63:0/2597578271 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed65200640 1 --2- 172.21.15.63:0/2597578271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b60fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.735+0000 7fed5fe00640 1 --2- 172.21.15.63:0/2597578271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017be1b0 0x556201b60a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.736+0000 7fed5d600640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed480b2950 0x7fed480b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.736+0000 7fed5fe00640 1 -- 172.21.15.63:0/2597578271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017be1b0 msgr2=0x556201b60a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.736+0000 7fed5fe00640 1 --2- 172.21.15.63:0/2597578271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017be1b0 0x556201b60a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.736+0000 7fed5fe00640 1 -- 172.21.15.63:0/2597578271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 msgr2=0x556201b60fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.736+0000 7fed5fe00640 1 --2- 172.21.15.63:0/2597578271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b60fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.736+0000 7fed5fe00640 1 -- 172.21.15.63:0/2597578271 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556201a5a8b0 con 0x55620179d4d0 2024-05-27T07:39:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.736+0000 7fed5d600640 1 -- 172.21.15.63:0/3068303558 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fed580e2b40 con 0x7fed60109f70 2024-05-27T07:39:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.736+0000 7fed65c00640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed480b2950 0x7fed480b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.736+0000 7fed5fe00640 1 --2- 172.21.15.63:0/2597578271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b614e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0096c0 tx=0x7fed4c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81b65c7dd03dfea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.737+0000 7fed52000640 1 -- 172.21.15.63:0/2597578271 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c035070 con 0x55620179d4d0 2024-05-27T07:39:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.737+0000 7fed52000640 1 -- 172.21.15.63:0/2597578271 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed4c0044c0 con 0x55620179d4d0 2024-05-27T07:39:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.737+0000 7fed52000640 1 -- 172.21.15.63:0/2597578271 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c003d90 con 0x55620179d4d0 2024-05-27T07:39:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.738+0000 7fed6ec10740 1 -- 172.21.15.63:0/2597578271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 msgr2=0x556201b614e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.738+0000 7fed65c00640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed480b2950 0x7fed480b4e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fed580a9a90 tx=0x7fed5800cf50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.738+0000 7fed6ec10740 1 --2- 172.21.15.63:0/2597578271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b614e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0096c0 tx=0x7fed4c005fd0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.738+0000 7fed6ec10740 1 -- 172.21.15.63:0/2597578271 shutdown_connections 2024-05-27T07:39:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.738+0000 7fed6ec10740 1 --2- 172.21.15.63:0/2597578271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b614e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.738+0000 7fed6ec10740 1 --2- 172.21.15.63:0/2597578271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b60fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.738+0000 7fed6ec10740 1 --2- 172.21.15.63:0/2597578271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017be1b0 0x556201b60a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.738+0000 7fed6ec10740 1 -- 172.21.15.63:0/2597578271 >> 172.21.15.63:0/2597578271 conn(0x5562017b2300 msgr2=0x556201b5d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.738+0000 7fed6ec10740 1 -- 172.21.15.63:0/2597578271 shutdown_connections 2024-05-27T07:39:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.738+0000 7fed6ec10740 1 -- 172.21.15.63:0/2597578271 wait complete. 2024-05-27T07:39:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.739+0000 7fed6ec10740 1 -- start start 2024-05-27T07:39:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.739+0000 7fed6ec10740 1 Processor -- start 2024-05-27T07:39:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b6ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b6b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017be1b0 0x556201b76e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed6ec10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5562017a7790 con 0x55620179d4d0 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed6ec10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556201a69bc0 con 0x5562017952f0 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed6ec10740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556201b6db30 con 0x5562017be1b0 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed6ec10740 10 client.0 _ll_register_callbacks cb 0x556201a47400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed65200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b6b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed65200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b6b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45118/0 (socket says 172.21.15.63:45118) 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed65200640 1 -- 172.21.15.63:0/3030186352 learned_addr learned my addr 172.21.15.63:0/3030186352 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed5fe00640 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b6ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.740+0000 7fed5fe00640 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017be1b0 0x556201b76e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.741+0000 7fed65200640 1 -- 172.21.15.63:0/3030186352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017be1b0 msgr2=0x556201b76e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.741+0000 7fed65200640 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017be1b0 0x556201b76e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.741+0000 7fed65200640 1 -- 172.21.15.63:0/3030186352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 msgr2=0x556201b6ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.741+0000 7fed65200640 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b6ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.741+0000 7fed65200640 1 -- 172.21.15.63:0/3030186352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562017d6210 con 0x55620179d4d0 2024-05-27T07:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.741+0000 7fed65200640 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b6b1b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fed5400b8f0 tx=0x7fed5400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a665a6629d8937d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.741+0000 7fed46a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.741+0000 7fed46a00640 1 -- 172.21.15.63:0/3030186352 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54003e20 con 0x55620179d4d0 2024-05-27T07:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.741+0000 7fed46a00640 1 -- 172.21.15.63:0/3030186352 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fed54005020 con 0x55620179d4d0 2024-05-27T07:39:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.741+0000 7fed6ec10740 1 -- 172.21.15.63:0/3030186352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556201ad3ae0 con 0x55620179d4d0 2024-05-27T07:39:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed46a00640 1 -- 172.21.15.63:0/3030186352 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fed54024050 con 0x55620179d4d0 2024-05-27T07:39:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed6ec10740 10 client.16668 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed6ec10740 1 -- 172.21.15.63:0/3030186352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5562018a2620 con 0x55620179d4d0 2024-05-27T07:39:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed46a00640 1 -- 172.21.15.63:0/3030186352 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556201ad3ae0 con 0x55620179d4d0 2024-05-27T07:39:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed46a00640 1 -- 172.21.15.63:0/3030186352 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fed5401b470 con 0x55620179d4d0 2024-05-27T07:39:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed6ec10740 10 client.16668 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed6ec10740 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed6ec10740 20 client.16668 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed6ec10740 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55620177cd80 0x556201b7cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed6ec10740 4 client.16668 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:39:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.742+0000 7fed6ec10740 1 -- 172.21.15.63:0/3030186352 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556201acd2c0 con 0x55620177cd80 2024-05-27T07:39:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.743+0000 7fed5fe00640 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55620177cd80 0x556201b7cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.744+0000 7fed5fe00640 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55620177cd80 0x556201b7cf50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5401e213bf3cf28d server_cookie=71cbacef1e60070e in_seq=0 out_seq=0 2024-05-27T07:39:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.744+0000 7fed46a00640 10 client.16668 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.746+0000 7fed46a00640 1 -- 172.21.15.63:0/3030186352 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556201acd2c0 con 0x55620177cd80 2024-05-27T07:39:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.746+0000 7fed46a00640 10 client.16668 handle_command_reply: tid=0 2024-05-27T07:39:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.746+0000 7fed6ec10740 1 client.16668 shutdown 2024-05-27T07:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 1 -- 172.21.15.63:0/3030186352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 msgr2=0x556201b6b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b6b1b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fed5400b8f0 tx=0x7fed5400bdb0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 1 -- 172.21.15.63:0/3030186352 shutdown_connections 2024-05-27T07:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55620177cd80 0x556201b7cf50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017be1b0 0x556201b76e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b6b1b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 1 --2- 172.21.15.63:0/3030186352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b6ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed46a00640 0 client.16668 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 1 -- 172.21.15.63:0/3030186352 >> 172.21.15.63:0/3030186352 conn(0x5562017b2300 msgr2=0x556201b62780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 1 -- 172.21.15.63:0/3030186352 shutdown_connections 2024-05-27T07:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 1 -- 172.21.15.63:0/3030186352 wait complete. 2024-05-27T07:39:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.747+0000 7fed6ec10740 20 client.16668 trim_cache size 0 max 16384 2024-05-27T07:39:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.765+0000 7fed6ec10740 1 Processor -- start 2024-05-27T07:39:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.766+0000 7fed6ec10740 1 -- start start 2024-05-27T07:39:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.766+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017952f0 0x556201b609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.766+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b76bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.766+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017be1b0 0x556201b77110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.766+0000 7fed6ec10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b5990 con 0x55620179d4d0 2024-05-27T07:39:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.766+0000 7fed6ec10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b3260 con 0x5562017be1b0 2024-05-27T07:39:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.766+0000 7fed6ec10740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b10b0 con 0x5562017952f0 2024-05-27T07:39:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.766+0000 7fed65200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b76bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.767+0000 7fed65200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b76bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45122/0 (socket says 172.21.15.63:45122) 2024-05-27T07:39:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.767+0000 7fed65200640 1 -- 172.21.15.63:0/650132905 learned_addr learned my addr 172.21.15.63:0/650132905 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.767+0000 7fed5fe00640 1 --2- 172.21.15.63:0/650132905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017952f0 0x556201b609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.767+0000 7fed5fe00640 1 --2- 172.21.15.63:0/650132905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017be1b0 0x556201b77110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.767+0000 7fed65200640 1 -- 172.21.15.63:0/650132905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017952f0 msgr2=0x556201b609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.767+0000 7fed65200640 1 --2- 172.21.15.63:0/650132905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017952f0 0x556201b609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.767+0000 7fed65200640 1 -- 172.21.15.63:0/650132905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017be1b0 msgr2=0x556201b77110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.767+0000 7fed65200640 1 --2- 172.21.15.63:0/650132905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017be1b0 0x556201b77110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.767+0000 7fed65200640 1 -- 172.21.15.63:0/650132905 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556201b77650 con 0x55620179d4d0 2024-05-27T07:39:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.767+0000 7fed65200640 1 --2- 172.21.15.63:0/650132905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b76bd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fed54005710 tx=0x7fed54005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff36ea52dfdc34d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.768+0000 7fed51600640 1 -- 172.21.15.63:0/650132905 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5402c020 con 0x55620179d4d0 2024-05-27T07:39:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.768+0000 7fed51600640 1 -- 172.21.15.63:0/650132905 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed5400b400 con 0x55620179d4d0 2024-05-27T07:39:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.768+0000 7fed51600640 1 -- 172.21.15.63:0/650132905 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed540118d0 con 0x55620179d4d0 2024-05-27T07:39:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.769+0000 7fed6ec10740 1 -- 172.21.15.63:0/650132905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 msgr2=0x556201b76bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.769+0000 7fed6ec10740 1 --2- 172.21.15.63:0/650132905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b76bd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fed54005710 tx=0x7fed54005740 comp rx=0 tx=0).stop 2024-05-27T07:39:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.769+0000 7fed6ec10740 1 -- 172.21.15.63:0/650132905 shutdown_connections 2024-05-27T07:39:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.769+0000 7fed6ec10740 1 --2- 172.21.15.63:0/650132905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017be1b0 0x556201b77110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.769+0000 7fed6ec10740 1 --2- 172.21.15.63:0/650132905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55620179d4d0 0x556201b76bd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.769+0000 7fed6ec10740 1 --2- 172.21.15.63:0/650132905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562017952f0 0x556201b609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.769+0000 7fed6ec10740 1 -- 172.21.15.63:0/650132905 >> 172.21.15.63:0/650132905 conn(0x5562017b2300 msgr2=0x556201b6af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.769+0000 7fed6ec10740 1 -- 172.21.15.63:0/650132905 shutdown_connections 2024-05-27T07:39:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.769+0000 7fed6ec10740 1 -- 172.21.15.63:0/650132905 wait complete. 2024-05-27T07:39:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.770+0000 7fed6ec10740 1 -- start start 2024-05-27T07:39:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.770+0000 7fed6ec10740 1 Processor -- start 2024-05-27T07:39:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b73110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55620179d4d0 0x556201b73650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed6ec10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562017be1b0 0x556201b6e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed6ec10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b1720 con 0x5562017be1b0 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed6ec10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b2b90 con 0x5562017952f0 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed6ec10740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b1df0 con 0x55620179d4d0 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed6ec10740 10 client.0 _ll_register_callbacks cb 0x7fed54004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed5fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562017be1b0 0x556201b6e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed5fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562017be1b0 0x556201b6e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45124/0 (socket says 172.21.15.63:45124) 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed5fe00640 1 -- 172.21.15.63:0/1183252400 learned_addr learned my addr 172.21.15.63:0/1183252400 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed65200640 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55620179d4d0 0x556201b73650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.771+0000 7fed5fe00640 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.772+0000 7fed5fe00640 1 -- 172.21.15.63:0/1183252400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55620179d4d0 msgr2=0x556201b73650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.772+0000 7fed5fe00640 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55620179d4d0 0x556201b73650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.772+0000 7fed5fe00640 1 -- 172.21.15.63:0/1183252400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 msgr2=0x556201b73110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:39:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.772+0000 7fed5fe00640 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b73110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.772+0000 7fed5fe00640 1 -- 172.21.15.63:0/1183252400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556201b6ed00 con 0x5562017be1b0 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.772+0000 7fed5fe00640 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562017be1b0 0x556201b6e570 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fed4c056010 tx=0x7fed540044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e84651b3204cb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.772+0000 7fed46a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.772+0000 7fed46a00640 1 -- 172.21.15.63:0/1183252400 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c035040 con 0x5562017be1b0 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.772+0000 7fed6ec10740 1 -- 172.21.15.63:0/1183252400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed5400b400 con 0x5562017be1b0 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.772+0000 7fed46a00640 1 -- 172.21.15.63:0/1183252400 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9395+0+0 (secure 0 0 0) 0x7fed4c031a80 con 0x5562017be1b0 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed46a00640 1 -- 172.21.15.63:0/1183252400 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fed4c067050 con 0x5562017be1b0 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed6ec10740 10 client.16674 fetch_fsmap learned FSMap version 33 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed6ec10740 1 -- 172.21.15.63:0/1183252400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556201be5110 con 0x5562017be1b0 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed46a00640 1 -- 172.21.15.63:0/1183252400 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed5400b400 con 0x5562017be1b0 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed46a00640 1 -- 172.21.15.63:0/1183252400 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fed4c0542a0 con 0x5562017be1b0 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed6ec10740 10 client.16674 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed6ec10740 10 client.16674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:39:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed6ec10740 20 client.16674 populate_metadata read hostname 'smithi063' 2024-05-27T07:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed6ec10740 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562017521c0 0x556201be7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed6ec10740 4 client.16674 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.773+0000 7fed6ec10740 1 -- 172.21.15.63:0/1183252400 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556201a474b0 con 0x5562017521c0 2024-05-27T07:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.774+0000 7fed65200640 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562017521c0 0x556201be7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.775+0000 7fed65200640 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562017521c0 0x556201be7400 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897b4c8755cf8c96 server_cookie=dee85c19d64f62a7 in_seq=0 out_seq=0 2024-05-27T07:39:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.775+0000 7fed46a00640 10 client.16674 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.828+0000 7fed46a00640 1 -- 172.21.15.63:0/1183252400 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556201a474b0 con 0x5562017521c0 2024-05-27T07:39:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.828+0000 7fed46a00640 10 client.16674 handle_command_reply: tid=0 2024-05-27T07:39:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.828+0000 7fed6ec10740 1 client.16674 shutdown 2024-05-27T07:39:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed6ec10740 1 -- 172.21.15.63:0/1183252400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562017be1b0 msgr2=0x556201b6e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed6ec10740 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562017be1b0 0x556201b6e570 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fed4c056010 tx=0x7fed540044a0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed6ec10740 1 -- 172.21.15.63:0/1183252400 shutdown_connections 2024-05-27T07:39:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed6ec10740 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562017521c0 0x556201be7400 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed6ec10740 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562017be1b0 0x556201b6e570 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed6ec10740 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55620179d4d0 0x556201b73650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed6ec10740 1 --2- 172.21.15.63:0/1183252400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562017952f0 0x556201b73110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed6ec10740 1 -- 172.21.15.63:0/1183252400 >> 172.21.15.63:0/1183252400 conn(0x5562017b2300 msgr2=0x556201b6c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed46a00640 0 client.16674 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:39:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed6ec10740 1 -- 172.21.15.63:0/1183252400 shutdown_connections 2024-05-27T07:39:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.829+0000 7fed6ec10740 1 -- 172.21.15.63:0/1183252400 wait complete. 2024-05-27T07:39:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.830+0000 7fed6ec10740 20 client.16674 trim_cache size 0 max 16384 2024-05-27T07:39:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:39:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:39:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 -- 172.21.15.63:0/3068303558 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed480b2950 msgr2=0x7fed480b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed480b2950 0x7fed480b4e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fed580a9a90 tx=0x7fed5800cf50 comp rx=0 tx=0).stop 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 -- 172.21.15.63:0/3068303558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed60109f70 msgr2=0x7fed60075a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed60109f70 0x7fed60075a40 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fed5800eaa0 tx=0x7fed5800ef60 comp rx=0 tx=0).stop 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 -- 172.21.15.63:0/3068303558 shutdown_connections 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed480b2950 0x7fed480b4e60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed60109f70 0x7fed60075a40 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed60109ad0 0x7fed60075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 --2- 172.21.15.63:0/3068303558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed60104220 0x7fed60074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 -- 172.21.15.63:0/3068303558 >> 172.21.15.63:0/3068303558 conn(0x7fed60079100 msgr2=0x7fed601022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 -- 172.21.15.63:0/3068303558 shutdown_connections 2024-05-27T07:39:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:39:58.831+0000 7fed66600640 1 -- 172.21.15.63:0/3068303558 wait complete. 2024-05-27T07:39:59.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:39:59 smithi086 ceph-mon[34687]: pgmap v195: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.7 MiB/s wr, 39 op/s 2024-05-27T07:39:59.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:39:59 smithi149 ceph-mon[35408]: pgmap v195: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.7 MiB/s wr, 39 op/s 2024-05-27T07:39:59.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:39:59 smithi063 ceph-mon[31613]: pgmap v195: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.7 MiB/s wr, 39 op/s 2024-05-27T07:40:00.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:00 smithi086 ceph-mon[34687]: overall HEALTH_OK 2024-05-27T07:40:00.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:00 smithi149 ceph-mon[35408]: overall HEALTH_OK 2024-05-27T07:40:00.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:00 smithi063 ceph-mon[31613]: overall HEALTH_OK 2024-05-27T07:40:01.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:01 smithi086 ceph-mon[34687]: pgmap v196: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.3 MiB/s wr, 32 op/s 2024-05-27T07:40:01.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:01 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:40:01.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:01 smithi149 ceph-mon[35408]: pgmap v196: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.3 MiB/s wr, 32 op/s 2024-05-27T07:40:01.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:01 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:40:01.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:01 smithi063 ceph-mon[31613]: pgmap v196: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.3 MiB/s wr, 32 op/s 2024-05-27T07:40:01.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:01 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:40:02.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:40:02.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:40:02.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:02 smithi086 ceph-mon[34687]: pgmap v197: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.6 MiB/s wr, 56 op/s 2024-05-27T07:40:02.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:40:02.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:40:02.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:02 smithi149 ceph-mon[35408]: pgmap v197: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.6 MiB/s wr, 56 op/s 2024-05-27T07:40:02.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:40:02.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:40:02.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:02 smithi063 ceph-mon[31613]: pgmap v197: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.6 MiB/s wr, 56 op/s 2024-05-27T07:40:04.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:04 smithi063 ceph-mon[31613]: pgmap v198: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.2 MiB/s wr, 52 op/s 2024-05-27T07:40:04.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:04 smithi086 ceph-mon[34687]: pgmap v198: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.2 MiB/s wr, 52 op/s 2024-05-27T07:40:04.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:04 smithi149 ceph-mon[35408]: pgmap v198: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.2 MiB/s wr, 52 op/s 2024-05-27T07:40:06.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:06 smithi086 ceph-mon[34687]: pgmap v199: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.2 MiB/s wr, 52 op/s 2024-05-27T07:40:06.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:06 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:40:06.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:06 smithi149 ceph-mon[35408]: pgmap v199: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.2 MiB/s wr, 52 op/s 2024-05-27T07:40:06.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:06 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:40:06.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:06 smithi063 ceph-mon[31613]: pgmap v199: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.2 MiB/s wr, 52 op/s 2024-05-27T07:40:06.962 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:06 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:40:07.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:07 smithi086 ceph-mon[34687]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T07:40:07.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:07 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:07 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:07 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:07 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:07 smithi149 ceph-mon[35408]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T07:40:07.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:07 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:07 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:07 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:07 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:07 smithi063 ceph-mon[31613]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T07:40:07.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:07 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:07 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:07 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:07.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:07 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:09.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:09 smithi086 ceph-mon[34687]: pgmap v201: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.9 MiB/s wr, 32 op/s 2024-05-27T07:40:09.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:09 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:09.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:09 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:40:09.457 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:09 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:09.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:09 smithi149 ceph-mon[35408]: pgmap v201: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.9 MiB/s wr, 32 op/s 2024-05-27T07:40:09.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:09 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:09.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:09 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:40:09.461 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:09 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:09.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:09 smithi063 ceph-mon[31613]: pgmap v201: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.9 MiB/s wr, 32 op/s 2024-05-27T07:40:09.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:09 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:09.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:09 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:40:09.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:09 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:40:11.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:11 smithi086 ceph-mon[34687]: pgmap v202: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.9 MiB/s wr, 32 op/s 2024-05-27T07:40:11.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:11 smithi149 ceph-mon[35408]: pgmap v202: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.9 MiB/s wr, 32 op/s 2024-05-27T07:40:11.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:11 smithi063 ceph-mon[31613]: pgmap v202: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.9 MiB/s wr, 32 op/s 2024-05-27T07:40:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.087+0000 7f9c51800640 1 -- 172.21.15.63:0/3018481733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c10e000 msgr2=0x7f9c4c1104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.087+0000 7f9c51800640 1 --2- 172.21.15.63:0/3018481733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c10e000 0x7f9c4c1104f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009be0 tx=0x7f9c4402a6d0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.087+0000 7f9c51800640 1 -- 172.21.15.63:0/3018481733 shutdown_connections 2024-05-27T07:40:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.087+0000 7f9c51800640 1 --2- 172.21.15.63:0/3018481733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c10e000 0x7f9c4c1104f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.087+0000 7f9c51800640 1 --2- 172.21.15.63:0/3018481733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c4c10b610 0x7f9c4c10da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.087+0000 7f9c51800640 1 --2- 172.21.15.63:0/3018481733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c104bb0 0x7f9c4c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.087+0000 7f9c51800640 1 -- 172.21.15.63:0/3018481733 >> 172.21.15.63:0/3018481733 conn(0x7f9c4c0fe210 msgr2=0x7f9c4c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.087+0000 7f9c51800640 1 -- 172.21.15.63:0/3018481733 shutdown_connections 2024-05-27T07:40:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.087+0000 7f9c51800640 1 -- 172.21.15.63:0/3018481733 wait complete. 2024-05-27T07:40:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.088+0000 7f9c51800640 1 Processor -- start 2024-05-27T07:40:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.088+0000 7f9c51800640 1 -- start start 2024-05-27T07:40:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.089+0000 7f9c51800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c4c104bb0 0x7f9c4c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.089+0000 7f9c51800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c10b610 0x7f9c4c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.089+0000 7f9c51800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c10e000 0x7f9c4c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.089+0000 7f9c51800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c113b10 con 0x7f9c4c10e000 2024-05-27T07:40:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.089+0000 7f9c51800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c113990 con 0x7f9c4c10b610 2024-05-27T07:40:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.089+0000 7f9c51800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c113c90 con 0x7f9c4c104bb0 2024-05-27T07:40:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.089+0000 7f9c50e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c10e000 0x7f9c4c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.089+0000 7f9c50e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c10e000 0x7f9c4c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59576/0 (socket says 172.21.15.63:59576) 2024-05-27T07:40:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.089+0000 7f9c50e00640 1 -- 172.21.15.63:0/2877920849 learned_addr learned my addr 172.21.15.63:0/2877920849 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.090+0000 7f9c4be00640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c4c104bb0 0x7f9c4c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.090+0000 7f9c4b400640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c10b610 0x7f9c4c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.090+0000 7f9c50e00640 1 -- 172.21.15.63:0/2877920849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c4c104bb0 msgr2=0x7f9c4c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.090+0000 7f9c50e00640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c4c104bb0 0x7f9c4c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.090+0000 7f9c50e00640 1 -- 172.21.15.63:0/2877920849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c10b610 msgr2=0x7f9c4c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.090+0000 7f9c50e00640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c10b610 0x7f9c4c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.090+0000 7f9c50e00640 1 -- 172.21.15.63:0/2877920849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c19f3b0 con 0x7f9c4c10e000 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.090+0000 7f9c4b400640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c10b610 0x7f9c4c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.090+0000 7f9c4be00640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c4c104bb0 0x7f9c4c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.090+0000 7f9c50e00640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c10e000 0x7f9c4c19ed70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009b30 tx=0x7f9c44030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.091+0000 7f9c48c00640 1 -- 172.21.15.63:0/2877920849 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44041020 con 0x7f9c4c10e000 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.091+0000 7f9c51800640 1 -- 172.21.15.63:0/2877920849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c19f640 con 0x7f9c4c10e000 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.091+0000 7f9c48c00640 1 -- 172.21.15.63:0/2877920849 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c44037040 con 0x7f9c4c10e000 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.091+0000 7f9c48c00640 1 -- 172.21.15.63:0/2877920849 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44036070 con 0x7f9c4c10e000 2024-05-27T07:40:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.091+0000 7f9c51800640 1 -- 172.21.15.63:0/2877920849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c19fb80 con 0x7f9c4c10e000 2024-05-27T07:40:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.095+0000 7f9c59e19740 1 Processor -- start 2024-05-27T07:40:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.095+0000 7f9c48c00640 1 -- 172.21.15.63:0/2877920849 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9c44044020 con 0x7f9c4c10e000 2024-05-27T07:40:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.096+0000 7f9c59e19740 1 -- start start 2024-05-27T07:40:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.096+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d540e1b0 0x5558d57b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.096+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.096+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53ed4d0 0x5558d57b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.096+0000 7f9c59e19740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5558d5721b90 con 0x5558d53e52f0 2024-05-27T07:40:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.096+0000 7f9c59e19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5558d5711730 con 0x5558d53ed4d0 2024-05-27T07:40:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.096+0000 7f9c59e19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5558d5715770 con 0x5558d540e1b0 2024-05-27T07:40:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.096+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.096+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59588/0 (socket says 172.21.15.63:59588) 2024-05-27T07:40:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.096+0000 7f9c4be00640 1 -- 172.21.15.63:0/1890245899 learned_addr learned my addr 172.21.15.63:0/1890245899 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.097+0000 7f9c4b400640 1 --2- 172.21.15.63:0/1890245899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d540e1b0 0x5558d57b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.097+0000 7f9c4b400640 1 --2- 172.21.15.63:0/1890245899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53ed4d0 0x5558d57b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.097+0000 7f9c4be00640 1 -- 172.21.15.63:0/1890245899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d540e1b0 msgr2=0x5558d57b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.097+0000 7f9c4be00640 1 --2- 172.21.15.63:0/1890245899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d540e1b0 0x5558d57b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.097+0000 7f9c4be00640 1 -- 172.21.15.63:0/1890245899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53ed4d0 msgr2=0x5558d57b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.097+0000 7f9c4be00640 1 --2- 172.21.15.63:0/1890245899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53ed4d0 0x5558d57b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.097+0000 7f9c4be00640 1 -- 172.21.15.63:0/1890245899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558d56aa8b0 con 0x5558d53e52f0 2024-05-27T07:40:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.097+0000 7f9c4be00640 1 --2- 172.21.15.63:0/1890245899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57b53b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009d20 tx=0x7f9c3c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d49062b5545715c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.098+0000 7f9c48c00640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c300b29d0 0x7f9c300b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.098+0000 7f9c41600640 1 -- 172.21.15.63:0/1890245899 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c012650 con 0x5558d53e52f0 2024-05-27T07:40:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.098+0000 7f9c48c00640 1 -- 172.21.15.63:0/2877920849 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9571+0+0 (secure 0 0 0) 0x7f9c44103350 con 0x7f9c4c10e000 2024-05-27T07:40:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.098+0000 7f9c41600640 1 -- 172.21.15.63:0/1890245899 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c3c009e10 con 0x5558d53e52f0 2024-05-27T07:40:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.098+0000 7f9c41600640 1 -- 172.21.15.63:0/1890245899 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00fae0 con 0x5558d53e52f0 2024-05-27T07:40:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.098+0000 7f9c4b400640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c300b29d0 0x7f9c300b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.099+0000 7f9c59e19740 1 -- 172.21.15.63:0/1890245899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 msgr2=0x5558d57b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.099+0000 7f9c4b400640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c300b29d0 0x7f9c300b4ee0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5558d57b1fe0 tx=0x7f9c38005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.099+0000 7f9c59e19740 1 --2- 172.21.15.63:0/1890245899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57b53b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009d20 tx=0x7f9c3c0117b0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.099+0000 7f9c59e19740 1 -- 172.21.15.63:0/1890245899 shutdown_connections 2024-05-27T07:40:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.099+0000 7f9c59e19740 1 --2- 172.21.15.63:0/1890245899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53ed4d0 0x5558d57b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.099+0000 7f9c59e19740 1 --2- 172.21.15.63:0/1890245899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57b53b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.099+0000 7f9c59e19740 1 --2- 172.21.15.63:0/1890245899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d540e1b0 0x5558d57b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.099+0000 7f9c59e19740 1 -- 172.21.15.63:0/1890245899 >> 172.21.15.63:0/1890245899 conn(0x5558d5402300 msgr2=0x5558d57b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.099+0000 7f9c59e19740 1 -- 172.21.15.63:0/1890245899 shutdown_connections 2024-05-27T07:40:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.099+0000 7f9c59e19740 1 -- 172.21.15.63:0/1890245899 wait complete. 2024-05-27T07:40:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.100+0000 7f9c59e19740 1 -- start start 2024-05-27T07:40:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.100+0000 7f9c59e19740 1 Processor -- start 2024-05-27T07:40:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.101+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57bf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.101+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53ed4d0 0x5558d57bf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.101+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d540e1b0 0x5558d57cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.101+0000 7f9c59e19740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5558d53f7790 con 0x5558d53e52f0 2024-05-27T07:40:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.101+0000 7f9c59e19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5558d56b9bc0 con 0x5558d53ed4d0 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.101+0000 7f9c59e19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5558d57c1fa0 con 0x5558d540e1b0 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.101+0000 7f9c59e19740 10 client.0 _ll_register_callbacks cb 0x5558d5697400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.101+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57bf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.101+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57bf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59590/0 (socket says 172.21.15.63:59590) 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.101+0000 7f9c4be00640 1 -- 172.21.15.63:0/2068586658 learned_addr learned my addr 172.21.15.63:0/2068586658 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.102+0000 7f9c4b400640 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53ed4d0 0x5558d57bf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.102+0000 7f9c50e00640 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d540e1b0 0x5558d57cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.102+0000 7f9c4be00640 1 -- 172.21.15.63:0/2068586658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d540e1b0 msgr2=0x5558d57cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.102+0000 7f9c4be00640 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d540e1b0 0x5558d57cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.102+0000 7f9c4be00640 1 -- 172.21.15.63:0/2068586658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53ed4d0 msgr2=0x5558d57bf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.102+0000 7f9c4be00640 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53ed4d0 0x5558d57bf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.102+0000 7f9c4be00640 1 -- 172.21.15.63:0/2068586658 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558d5426210 con 0x5558d53e52f0 2024-05-27T07:40:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.102+0000 7f9c4be00640 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57bf300 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c000c00 tx=0x7f9c3c004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd8ac4476cae35 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.102+0000 7f9c36000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.103+0000 7f9c36000640 1 -- 172.21.15.63:0/2068586658 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c028070 con 0x5558d53e52f0 2024-05-27T07:40:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.103+0000 7f9c36000640 1 -- 172.21.15.63:0/2068586658 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9571+0+0 (secure 0 0 0) 0x7f9c3c00fa60 con 0x5558d53e52f0 2024-05-27T07:40:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.103+0000 7f9c59e19740 1 -- 172.21.15.63:0/2068586658 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c3c009e10 con 0x5558d53e52f0 2024-05-27T07:40:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.103+0000 7f9c36000640 1 -- 172.21.15.63:0/2068586658 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9c3c034050 con 0x5558d53e52f0 2024-05-27T07:40:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.103+0000 7f9c59e19740 10 client.16686 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.103+0000 7f9c59e19740 1 -- 172.21.15.63:0/2068586658 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5558d54f2620 con 0x5558d53e52f0 2024-05-27T07:40:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.104+0000 7f9c36000640 1 -- 172.21.15.63:0/2068586658 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c3c009e10 con 0x5558d53e52f0 2024-05-27T07:40:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.104+0000 7f9c36000640 1 -- 172.21.15.63:0/2068586658 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9c3c012450 con 0x5558d53e52f0 2024-05-27T07:40:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.104+0000 7f9c59e19740 10 client.16686 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.104+0000 7f9c59e19740 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.104+0000 7f9c59e19740 20 client.16686 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.104+0000 7f9c59e19740 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558d53ccd80 0x5558d57d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.104+0000 7f9c59e19740 4 client.16686 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:40:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.104+0000 7f9c59e19740 1 -- 172.21.15.63:0/2068586658 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558d5723ae0 con 0x5558d53ccd80 2024-05-27T07:40:12.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.105+0000 7f9c4b400640 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558d53ccd80 0x5558d57d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.106+0000 7f9c4b400640 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558d53ccd80 0x5558d57d1250 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e35a214ed7362c6 server_cookie=73720c91bb6816de in_seq=0 out_seq=0 2024-05-27T07:40:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.106+0000 7f9c36000640 10 client.16686 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.108+0000 7f9c36000640 1 -- 172.21.15.63:0/2068586658 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5558d5723ae0 con 0x5558d53ccd80 2024-05-27T07:40:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.108+0000 7f9c36000640 10 client.16686 handle_command_reply: tid=0 2024-05-27T07:40:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.108+0000 7f9c59e19740 1 client.16686 shutdown 2024-05-27T07:40:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.108+0000 7f9c59e19740 1 -- 172.21.15.63:0/2068586658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 msgr2=0x5558d57bf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.108+0000 7f9c59e19740 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57bf300 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c000c00 tx=0x7f9c3c004590 comp rx=0 tx=0).stop 2024-05-27T07:40:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.109+0000 7f9c59e19740 1 -- 172.21.15.63:0/2068586658 shutdown_connections 2024-05-27T07:40:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.109+0000 7f9c59e19740 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558d53ccd80 0x5558d57d1250 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.109+0000 7f9c59e19740 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d540e1b0 0x5558d57cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.109+0000 7f9c59e19740 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53ed4d0 0x5558d57bf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.109+0000 7f9c36000640 0 client.16686 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.109+0000 7f9c59e19740 1 --2- 172.21.15.63:0/2068586658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53e52f0 0x5558d57bf300 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.109+0000 7f9c59e19740 1 -- 172.21.15.63:0/2068586658 >> 172.21.15.63:0/2068586658 conn(0x5558d5402300 msgr2=0x5558d57b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.109+0000 7f9c59e19740 1 -- 172.21.15.63:0/2068586658 shutdown_connections 2024-05-27T07:40:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.109+0000 7f9c59e19740 1 -- 172.21.15.63:0/2068586658 wait complete. 2024-05-27T07:40:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.109+0000 7f9c59e19740 20 client.16686 trim_cache size 0 max 16384 2024-05-27T07:40:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.136+0000 7f9c59e19740 1 Processor -- start 2024-05-27T07:40:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.136+0000 7f9c59e19740 1 -- start start 2024-05-27T07:40:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.136+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53e52f0 0x5558d57be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.136+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d53ed4d0 0x5558d57ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.136+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d540e1b0 0x5558d57cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.136+0000 7f9c59e19740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1b1860 con 0x5558d540e1b0 2024-05-27T07:40:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.136+0000 7f9c59e19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1af130 con 0x5558d53e52f0 2024-05-27T07:40:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.136+0000 7f9c59e19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1acfa0 con 0x5558d53ed4d0 2024-05-27T07:40:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.137+0000 7f9c50e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d540e1b0 0x5558d57cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.137+0000 7f9c50e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d540e1b0 0x5558d57cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59606/0 (socket says 172.21.15.63:59606) 2024-05-27T07:40:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.137+0000 7f9c50e00640 1 -- 172.21.15.63:0/1142525693 learned_addr learned my addr 172.21.15.63:0/1142525693 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.137+0000 7f9c4b400640 1 --2- 172.21.15.63:0/1142525693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d53ed4d0 0x5558d57ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.137+0000 7f9c4be00640 1 --2- 172.21.15.63:0/1142525693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53e52f0 0x5558d57be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.137+0000 7f9c50e00640 1 -- 172.21.15.63:0/1142525693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d53ed4d0 msgr2=0x5558d57ba8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.137+0000 7f9c50e00640 1 --2- 172.21.15.63:0/1142525693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d53ed4d0 0x5558d57ba8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.137+0000 7f9c50e00640 1 -- 172.21.15.63:0/1142525693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53e52f0 msgr2=0x5558d57be3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.137+0000 7f9c50e00640 1 --2- 172.21.15.63:0/1142525693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53e52f0 0x5558d57be3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.137+0000 7f9c50e00640 1 -- 172.21.15.63:0/1142525693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558d57bd640 con 0x5558d540e1b0 2024-05-27T07:40:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.138+0000 7f9c50e00640 1 --2- 172.21.15.63:0/1142525693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d540e1b0 0x5558d57cb240 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9c440c9fc0 tx=0x7f9c440c9af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a4b08633f3a66f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.138+0000 7f9c40c00640 1 -- 172.21.15.63:0/1142525693 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c440e4070 con 0x5558d540e1b0 2024-05-27T07:40:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.138+0000 7f9c40c00640 1 -- 172.21.15.63:0/1142525693 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c440ca2c0 con 0x5558d540e1b0 2024-05-27T07:40:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.138+0000 7f9c40c00640 1 -- 172.21.15.63:0/1142525693 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c441008a0 con 0x5558d540e1b0 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.139+0000 7f9c59e19740 1 -- 172.21.15.63:0/1142525693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d540e1b0 msgr2=0x5558d57cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.139+0000 7f9c59e19740 1 --2- 172.21.15.63:0/1142525693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d540e1b0 0x5558d57cb240 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9c440c9fc0 tx=0x7f9c440c9af0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.139+0000 7f9c59e19740 1 -- 172.21.15.63:0/1142525693 shutdown_connections 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.139+0000 7f9c59e19740 1 --2- 172.21.15.63:0/1142525693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d540e1b0 0x5558d57cb240 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.139+0000 7f9c59e19740 1 --2- 172.21.15.63:0/1142525693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d53ed4d0 0x5558d57ba8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.139+0000 7f9c59e19740 1 --2- 172.21.15.63:0/1142525693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d53e52f0 0x5558d57be3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.139+0000 7f9c59e19740 1 -- 172.21.15.63:0/1142525693 >> 172.21.15.63:0/1142525693 conn(0x5558d5402300 msgr2=0x5558d57ab810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.139+0000 7f9c59e19740 1 -- 172.21.15.63:0/1142525693 shutdown_connections 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.139+0000 7f9c59e19740 1 -- 172.21.15.63:0/1142525693 wait complete. 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.140+0000 7f9c59e19740 1 -- start start 2024-05-27T07:40:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.140+0000 7f9c59e19740 1 Processor -- start 2024-05-27T07:40:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d53e52f0 0x5558d57d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53ed4d0 0x5558d57d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c59e19740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d540e1b0 0x5558d5831660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c59e19740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1ad5f0 con 0x5558d53ed4d0 2024-05-27T07:40:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c59e19740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1aea60 con 0x5558d540e1b0 2024-05-27T07:40:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c59e19740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1adcc0 con 0x5558d53e52f0 2024-05-27T07:40:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c59e19740 10 client.0 _ll_register_callbacks cb 0x7f9c380099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53ed4d0 0x5558d57d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c4b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53ed4d0 0x5558d57d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59622/0 (socket says 172.21.15.63:59622) 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c4b400640 1 -- 172.21.15.63:0/4003248772 learned_addr learned my addr 172.21.15.63:0/4003248772 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c4be00640 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d53e52f0 0x5558d57d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c50e00640 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d540e1b0 0x5558d5831660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c4b400640 1 -- 172.21.15.63:0/4003248772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d53e52f0 msgr2=0x5558d57d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.141+0000 7f9c4b400640 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d53e52f0 0x5558d57d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.142+0000 7f9c4b400640 1 -- 172.21.15.63:0/4003248772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d540e1b0 msgr2=0x5558d5831660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.142+0000 7f9c4b400640 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d540e1b0 0x5558d5831660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.142+0000 7f9c4b400640 1 -- 172.21.15.63:0/4003248772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558d5831ba0 con 0x5558d53ed4d0 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.142+0000 7f9c50e00640 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d540e1b0 0x5558d5831660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.142+0000 7f9c4b400640 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53ed4d0 0x5558d57d1d40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x5558d53f7880 tx=0x7f9c380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f379fd47cdf5931 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.142+0000 7f9c36000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.142+0000 7f9c36000640 1 -- 172.21.15.63:0/4003248772 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3802cbd0 con 0x5558d53ed4d0 2024-05-27T07:40:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.142+0000 7f9c36000640 1 -- 172.21.15.63:0/4003248772 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9571+0+0 (secure 0 0 0) 0x7f9c38005020 con 0x5558d53ed4d0 2024-05-27T07:40:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.143+0000 7f9c59e19740 1 -- 172.21.15.63:0/4003248772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c440ca2c0 con 0x5558d53ed4d0 2024-05-27T07:40:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.143+0000 7f9c36000640 1 -- 172.21.15.63:0/4003248772 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9c3803f050 con 0x5558d53ed4d0 2024-05-27T07:40:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.143+0000 7f9c59e19740 10 client.16692 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.143+0000 7f9c59e19740 1 -- 172.21.15.63:0/4003248772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5558d5838e80 con 0x5558d53ed4d0 2024-05-27T07:40:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.143+0000 7f9c36000640 1 -- 172.21.15.63:0/4003248772 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c440ca2c0 con 0x5558d53ed4d0 2024-05-27T07:40:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.144+0000 7f9c36000640 1 -- 172.21.15.63:0/4003248772 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9c3801c040 con 0x5558d53ed4d0 2024-05-27T07:40:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.144+0000 7f9c59e19740 10 client.16692 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.144+0000 7f9c59e19740 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.144+0000 7f9c59e19740 20 client.16692 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.144+0000 7f9c59e19740 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558d53a21c0 0x5558d583b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.144+0000 7f9c59e19740 4 client.16692 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:40:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.144+0000 7f9c59e19740 1 -- 172.21.15.63:0/4003248772 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558d571d2c0 con 0x5558d53a21c0 2024-05-27T07:40:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.144+0000 7f9c4be00640 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558d53a21c0 0x5558d583b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.145+0000 7f9c4be00640 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558d53a21c0 0x5558d583b6f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7cdc39b6c96edc5 server_cookie=8183223f0a70739a in_seq=0 out_seq=0 2024-05-27T07:40:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.145+0000 7f9c36000640 10 client.16692 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.189+0000 7f9c36000640 1 -- 172.21.15.63:0/4003248772 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5558d571d2c0 con 0x5558d53a21c0 2024-05-27T07:40:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.189+0000 7f9c36000640 10 client.16692 handle_command_reply: tid=0 2024-05-27T07:40:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.189+0000 7f9c59e19740 1 client.16692 shutdown 2024-05-27T07:40:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.189+0000 7f9c59e19740 1 -- 172.21.15.63:0/4003248772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53ed4d0 msgr2=0x5558d57d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.189+0000 7f9c59e19740 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53ed4d0 0x5558d57d1d40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x5558d53f7880 tx=0x7f9c380046c0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.189+0000 7f9c59e19740 1 -- 172.21.15.63:0/4003248772 shutdown_connections 2024-05-27T07:40:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.190+0000 7f9c59e19740 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558d53a21c0 0x5558d583b6f0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.190+0000 7f9c59e19740 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558d540e1b0 0x5558d5831660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.190+0000 7f9c59e19740 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558d53ed4d0 0x5558d57d1d40 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.190+0000 7f9c59e19740 1 --2- 172.21.15.63:0/4003248772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558d53e52f0 0x5558d57d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.190+0000 7f9c36000640 0 client.16692 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.190+0000 7f9c59e19740 1 -- 172.21.15.63:0/4003248772 >> 172.21.15.63:0/4003248772 conn(0x5558d5402300 msgr2=0x5558d57be150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.190+0000 7f9c59e19740 1 -- 172.21.15.63:0/4003248772 shutdown_connections 2024-05-27T07:40:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.190+0000 7f9c59e19740 1 -- 172.21.15.63:0/4003248772 wait complete. 2024-05-27T07:40:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.190+0000 7f9c59e19740 20 client.16692 trim_cache size 0 max 16384 2024-05-27T07:40:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:40:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:40:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:40:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 -- 172.21.15.63:0/2877920849 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c300b29d0 msgr2=0x7f9c300b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c300b29d0 0x7f9c300b4ee0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5558d57b1fe0 tx=0x7f9c38005f90 comp rx=0 tx=0).stop 2024-05-27T07:40:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 -- 172.21.15.63:0/2877920849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c10e000 msgr2=0x7f9c4c19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c10e000 0x7f9c4c19ed70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009b30 tx=0x7f9c44030d50 comp rx=0 tx=0).stop 2024-05-27T07:40:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 -- 172.21.15.63:0/2877920849 shutdown_connections 2024-05-27T07:40:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c300b29d0 0x7f9c300b4ee0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c4c10e000 0x7f9c4c19ed70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4c10b610 0x7f9c4c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 --2- 172.21.15.63:0/2877920849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c4c104bb0 0x7f9c4c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 -- 172.21.15.63:0/2877920849 >> 172.21.15.63:0/2877920849 conn(0x7f9c4c0fe210 msgr2=0x7f9c4c10e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 -- 172.21.15.63:0/2877920849 shutdown_connections 2024-05-27T07:40:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:12.191+0000 7f9c51800640 1 -- 172.21.15.63:0/2877920849 wait complete. 2024-05-27T07:40:13.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:13 smithi086 ceph-mon[34687]: pgmap v203: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.6 MiB/s wr, 8 op/s 2024-05-27T07:40:13.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:13 smithi149 ceph-mon[35408]: pgmap v203: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.6 MiB/s wr, 8 op/s 2024-05-27T07:40:13.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:13 smithi063 ceph-mon[31613]: pgmap v203: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.6 MiB/s wr, 8 op/s 2024-05-27T07:40:15.456 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:15 smithi086 ceph-mon[34687]: pgmap v204: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.6 MiB/s wr, 8 op/s 2024-05-27T07:40:15.460 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:15 smithi149 ceph-mon[35408]: pgmap v204: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.6 MiB/s wr, 8 op/s 2024-05-27T07:40:15.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:15 smithi063 ceph-mon[31613]: pgmap v204: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.6 MiB/s wr, 8 op/s 2024-05-27T07:40:16.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:16 smithi086 ceph-mon[34687]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T07:40:16.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:16 smithi149 ceph-mon[35408]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T07:40:16.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:16 smithi063 ceph-mon[31613]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T07:40:17.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:17 smithi086 ceph-mon[34687]: pgmap v206: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-05-27T07:40:17.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:17 smithi149 ceph-mon[35408]: pgmap v206: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-05-27T07:40:17.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:17 smithi063 ceph-mon[31613]: pgmap v206: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-05-27T07:40:19.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:19 smithi086 ceph-mon[34687]: pgmap v207: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:19.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:19 smithi149 ceph-mon[35408]: pgmap v207: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:19.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:19 smithi063 ceph-mon[31613]: pgmap v207: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:21.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:21 smithi086 ceph-mon[34687]: pgmap v208: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:21.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:21 smithi149 ceph-mon[35408]: pgmap v208: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:21.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:21 smithi063 ceph-mon[31613]: pgmap v208: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:23.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:23 smithi086 ceph-mon[34687]: pgmap v209: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:23.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:23 smithi149 ceph-mon[35408]: pgmap v209: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:23.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:23 smithi063 ceph-mon[31613]: pgmap v209: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.442+0000 7f8b3d400640 1 -- 172.21.15.63:0/2513855618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b38109f50 msgr2=0x7f8b38103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.442+0000 7f8b3d400640 1 --2- 172.21.15.63:0/2513855618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b38109f50 0x7f8b38103b90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009900 tx=0x7f8b2402a410 comp rx=0 tx=0).stop 2024-05-27T07:40:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.442+0000 7f8b3d400640 1 -- 172.21.15.63:0/2513855618 shutdown_connections 2024-05-27T07:40:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.442+0000 7f8b3d400640 1 --2- 172.21.15.63:0/2513855618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b38104200 0x7f8b381105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.442+0000 7f8b3d400640 1 --2- 172.21.15.63:0/2513855618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b38109f50 0x7f8b38103b90 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.442+0000 7f8b3d400640 1 --2- 172.21.15.63:0/2513855618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38109ab0 0x7f8b38103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.442+0000 7f8b3d400640 1 -- 172.21.15.63:0/2513855618 >> 172.21.15.63:0/2513855618 conn(0x7f8b38079140 msgr2=0x7f8b38079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.442+0000 7f8b3d400640 1 -- 172.21.15.63:0/2513855618 shutdown_connections 2024-05-27T07:40:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.442+0000 7f8b3d400640 1 -- 172.21.15.63:0/2513855618 wait complete. 2024-05-27T07:40:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.443+0000 7f8b3d400640 1 Processor -- start 2024-05-27T07:40:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.444+0000 7f8b3d400640 1 -- start start 2024-05-27T07:40:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.444+0000 7f8b3d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38104200 0x7f8b3819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.444+0000 7f8b3d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b38109ab0 0x7f8b3819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.444+0000 7f8b3d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b38109f50 0x7f8b381a8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.444+0000 7f8b3d400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38113ba0 con 0x7f8b38109f50 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.444+0000 7f8b3d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38113a20 con 0x7f8b38104200 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.444+0000 7f8b3d400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38113d20 con 0x7f8b38109ab0 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.445+0000 7f8b3ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b38109f50 0x7f8b381a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.445+0000 7f8b3ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b38109f50 0x7f8b381a8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47854/0 (socket says 172.21.15.63:47854) 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.445+0000 7f8b3ca00640 1 -- 172.21.15.63:0/637789083 learned_addr learned my addr 172.21.15.63:0/637789083 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.445+0000 7f8b37e00640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38104200 0x7f8b3819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.445+0000 7f8b37400640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b38109ab0 0x7f8b3819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.445+0000 7f8b3ca00640 1 -- 172.21.15.63:0/637789083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b38109ab0 msgr2=0x7f8b3819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.445+0000 7f8b3ca00640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b38109ab0 0x7f8b3819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.445+0000 7f8b3ca00640 1 -- 172.21.15.63:0/637789083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38104200 msgr2=0x7f8b3819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.445+0000 7f8b3ca00640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38104200 0x7f8b3819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.445+0000 7f8b3ca00640 1 -- 172.21.15.63:0/637789083 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b381a8a40 con 0x7f8b38109f50 2024-05-27T07:40:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.446+0000 7f8b3ca00640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b38109f50 0x7f8b381a8500 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000eaa0 tx=0x7f8b3000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.446+0000 7f8b34c00640 1 -- 172.21.15.63:0/637789083 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000cdb0 con 0x7f8b38109f50 2024-05-27T07:40:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.446+0000 7f8b34c00640 1 -- 172.21.15.63:0/637789083 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b30005020 con 0x7f8b38109f50 2024-05-27T07:40:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.446+0000 7f8b3d400640 1 -- 172.21.15.63:0/637789083 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b381a8cd0 con 0x7f8b38109f50 2024-05-27T07:40:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.446+0000 7f8b34c00640 1 -- 172.21.15.63:0/637789083 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b300116d0 con 0x7f8b38109f50 2024-05-27T07:40:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.446+0000 7f8b3d400640 1 -- 172.21.15.63:0/637789083 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b381a91a0 con 0x7f8b38109f50 2024-05-27T07:40:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.450+0000 7f8b459a2740 1 Processor -- start 2024-05-27T07:40:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.451+0000 7f8b34c00640 1 -- 172.21.15.63:0/637789083 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8b300051d0 con 0x7f8b38109f50 2024-05-27T07:40:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.451+0000 7f8b459a2740 1 -- start start 2024-05-27T07:40:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.451+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cbba1b0 0x56381cf5ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.451+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf5cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.451+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb994d0 0x56381cf5d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.451+0000 7f8b459a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56381cecdb90 con 0x56381cb994d0 2024-05-27T07:40:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.451+0000 7f8b459a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56381cebd730 con 0x56381cb912f0 2024-05-27T07:40:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.451+0000 7f8b459a2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56381cec1770 con 0x56381cbba1b0 2024-05-27T07:40:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.452+0000 7f8b37400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb994d0 0x56381cf5d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.452+0000 7f8b37400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb994d0 0x56381cf5d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47862/0 (socket says 172.21.15.63:47862) 2024-05-27T07:40:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.452+0000 7f8b37400640 1 -- 172.21.15.63:0/3070945364 learned_addr learned my addr 172.21.15.63:0/3070945364 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.452+0000 7f8b37e00640 1 --2- 172.21.15.63:0/3070945364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf5cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.452+0000 7f8b37400640 1 --2- 172.21.15.63:0/3070945364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cbba1b0 0x56381cf5ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.453+0000 7f8b37400640 1 -- 172.21.15.63:0/3070945364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cbba1b0 msgr2=0x56381cf5ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.453+0000 7f8b37400640 1 --2- 172.21.15.63:0/3070945364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cbba1b0 0x56381cf5ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.453+0000 7f8b37400640 1 -- 172.21.15.63:0/3070945364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 msgr2=0x56381cf5cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.453+0000 7f8b37400640 1 --2- 172.21.15.63:0/3070945364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf5cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.453+0000 7f8b37400640 1 -- 172.21.15.63:0/3070945364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56381ce568b0 con 0x56381cb994d0 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.453+0000 7f8b37e00640 1 --2- 172.21.15.63:0/3070945364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf5cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.453+0000 7f8b34c00640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b1c0b2950 0x7f8b1c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.453+0000 7f8b34c00640 1 -- 172.21.15.63:0/637789083 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f8b300e2cb0 con 0x7f8b38109f50 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.453+0000 7f8b37400640 1 --2- 172.21.15.63:0/3070945364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb994d0 0x56381cf5d4e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8b24005760 tx=0x7f8b240056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ba56870a24d689 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.453+0000 7f8b37e00640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b1c0b2950 0x7f8b1c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.454+0000 7f8b2d200640 1 -- 172.21.15.63:0/3070945364 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24035040 con 0x56381cb994d0 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.454+0000 7f8b2d200640 1 -- 172.21.15.63:0/3070945364 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b2403d440 con 0x56381cb994d0 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.454+0000 7f8b2d200640 1 -- 172.21.15.63:0/3070945364 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24031620 con 0x56381cb994d0 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.455+0000 7f8b459a2740 1 -- 172.21.15.63:0/3070945364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb994d0 msgr2=0x56381cf5d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.455+0000 7f8b459a2740 1 --2- 172.21.15.63:0/3070945364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb994d0 0x56381cf5d4e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8b24005760 tx=0x7f8b240056f0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.455+0000 7f8b37e00640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b1c0b2950 0x7f8b1c0b4e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x56381cf5de10 tx=0x7f8b2800b4d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.455+0000 7f8b459a2740 1 -- 172.21.15.63:0/3070945364 shutdown_connections 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.455+0000 7f8b459a2740 1 --2- 172.21.15.63:0/3070945364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb994d0 0x56381cf5d4e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.455+0000 7f8b459a2740 1 --2- 172.21.15.63:0/3070945364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf5cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.455+0000 7f8b459a2740 1 --2- 172.21.15.63:0/3070945364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cbba1b0 0x56381cf5ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.455+0000 7f8b459a2740 1 -- 172.21.15.63:0/3070945364 >> 172.21.15.63:0/3070945364 conn(0x56381cbae300 msgr2=0x56381cf59210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.455+0000 7f8b459a2740 1 -- 172.21.15.63:0/3070945364 shutdown_connections 2024-05-27T07:40:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.455+0000 7f8b459a2740 1 -- 172.21.15.63:0/3070945364 wait complete. 2024-05-27T07:40:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.456+0000 7f8b459a2740 1 -- start start 2024-05-27T07:40:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.456+0000 7f8b459a2740 1 Processor -- start 2024-05-27T07:40:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb912f0 0x56381cf66c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf671a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cbba1b0 0x56381cf72e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b459a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56381cba3790 con 0x56381cb912f0 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b459a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56381ce65bc0 con 0x56381cbba1b0 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b459a2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56381cf69b30 con 0x56381cb994d0 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b459a2740 10 client.0 _ll_register_callbacks cb 0x56381ce43400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb912f0 0x56381cf66c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb912f0 0x56381cf66c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47864/0 (socket says 172.21.15.63:47864) 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b37e00640 1 -- 172.21.15.63:0/4073572583 learned_addr learned my addr 172.21.15.63:0/4073572583 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.457+0000 7f8b37400640 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf671a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.458+0000 7f8b3ca00640 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cbba1b0 0x56381cf72e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.458+0000 7f8b37e00640 1 -- 172.21.15.63:0/4073572583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 msgr2=0x56381cf671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.458+0000 7f8b37e00640 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf671a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.458+0000 7f8b37e00640 1 -- 172.21.15.63:0/4073572583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cbba1b0 msgr2=0x56381cf72e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.458+0000 7f8b37e00640 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cbba1b0 0x56381cf72e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.458+0000 7f8b37e00640 1 -- 172.21.15.63:0/4073572583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381cbd2210 con 0x56381cb912f0 2024-05-27T07:40:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.458+0000 7f8b37e00640 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb912f0 0x56381cf66c60 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8b28005d20 tx=0x7f8b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52b3bcdde1d2f1b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.458+0000 7f8b21600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.458+0000 7f8b21600640 1 -- 172.21.15.63:0/4073572583 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800f990 con 0x56381cb912f0 2024-05-27T07:40:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.459+0000 7f8b21600640 1 -- 172.21.15.63:0/4073572583 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f8b2800fb30 con 0x56381cb912f0 2024-05-27T07:40:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.458+0000 7f8b459a2740 1 -- 172.21.15.63:0/4073572583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b2403d440 con 0x56381cb912f0 2024-05-27T07:40:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.459+0000 7f8b21600640 1 -- 172.21.15.63:0/4073572583 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8b28030050 con 0x56381cb912f0 2024-05-27T07:40:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.459+0000 7f8b459a2740 10 client.16704 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.459+0000 7f8b459a2740 1 -- 172.21.15.63:0/4073572583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56381cc9e620 con 0x56381cb912f0 2024-05-27T07:40:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.459+0000 7f8b21600640 1 -- 172.21.15.63:0/4073572583 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b2403d440 con 0x56381cb912f0 2024-05-27T07:40:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.459+0000 7f8b21600640 1 -- 172.21.15.63:0/4073572583 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8b2801f3f0 con 0x56381cb912f0 2024-05-27T07:40:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.460+0000 7f8b459a2740 10 client.16704 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.460+0000 7f8b459a2740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.460+0000 7f8b459a2740 20 client.16704 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.460+0000 7f8b459a2740 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56381cb78d80 0x56381cf78f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.460+0000 7f8b459a2740 4 client.16704 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:40:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.460+0000 7f8b459a2740 1 -- 172.21.15.63:0/4073572583 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56381cecfae0 con 0x56381cb78d80 2024-05-27T07:40:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.460+0000 7f8b37400640 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56381cb78d80 0x56381cf78f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.461+0000 7f8b37400640 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56381cb78d80 0x56381cf78f30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b174491fcd24eb5 server_cookie=ffcf675113d66a98 in_seq=0 out_seq=0 2024-05-27T07:40:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.461+0000 7f8b21600640 10 client.16704 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.463+0000 7f8b21600640 1 -- 172.21.15.63:0/4073572583 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56381cecfae0 con 0x56381cb78d80 2024-05-27T07:40:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.463+0000 7f8b21600640 10 client.16704 handle_command_reply: tid=0 2024-05-27T07:40:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.463+0000 7f8b459a2740 1 client.16704 shutdown 2024-05-27T07:40:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 1 -- 172.21.15.63:0/4073572583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb912f0 msgr2=0x56381cf66c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb912f0 0x56381cf66c60 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8b28005d20 tx=0x7f8b280046c0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 1 -- 172.21.15.63:0/4073572583 shutdown_connections 2024-05-27T07:40:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56381cb78d80 0x56381cf78f30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cbba1b0 0x56381cf72e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf671a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 1 --2- 172.21.15.63:0/4073572583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cb912f0 0x56381cf66c60 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b21600640 0 client.16704 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 1 -- 172.21.15.63:0/4073572583 >> 172.21.15.63:0/4073572583 conn(0x56381cbae300 msgr2=0x56381cf5e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 1 -- 172.21.15.63:0/4073572583 shutdown_connections 2024-05-27T07:40:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 1 -- 172.21.15.63:0/4073572583 wait complete. 2024-05-27T07:40:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.464+0000 7f8b459a2740 20 client.16704 trim_cache size 0 max 16384 2024-05-27T07:40:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.482+0000 7f8b459a2740 1 Processor -- start 2024-05-27T07:40:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.482+0000 7f8b459a2740 1 -- start start 2024-05-27T07:40:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.483+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf69390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.483+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf61e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.483+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.483+0000 7f8b459a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381b5b30 con 0x56381cbae300 2024-05-27T07:40:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.483+0000 7f8b459a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381b3400 con 0x56381cb912f0 2024-05-27T07:40:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.483+0000 7f8b459a2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381b1270 con 0x56381cb994d0 2024-05-27T07:40:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.483+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf69390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.483+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf69390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47874/0 (socket says 172.21.15.63:47874) 2024-05-27T07:40:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.483+0000 7f8b37e00640 1 -- 172.21.15.63:0/2730734095 learned_addr learned my addr 172.21.15.63:0/2730734095 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.484+0000 7f8b37e00640 1 -- 172.21.15.63:0/2730734095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 msgr2=0x56381cf623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.484+0000 7f8b3ca00640 1 --2- 172.21.15.63:0/2730734095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.484+0000 7f8b37e00640 1 --2- 172.21.15.63:0/2730734095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.484+0000 7f8b37e00640 1 -- 172.21.15.63:0/2730734095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 msgr2=0x56381cf61e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.484+0000 7f8b37e00640 1 --2- 172.21.15.63:0/2730734095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf61e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.484+0000 7f8b37e00640 1 -- 172.21.15.63:0/2730734095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56381cf57570 con 0x56381cbae300 2024-05-27T07:40:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.484+0000 7f8b3ca00640 1 --2- 172.21.15.63:0/2730734095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.484+0000 7f8b37e00640 1 --2- 172.21.15.63:0/2730734095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf69390 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8b28005780 tx=0x7f8b28012d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d825a8dfc065e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.485+0000 7f8b23e00640 1 -- 172.21.15.63:0/2730734095 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2802d040 con 0x56381cbae300 2024-05-27T07:40:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.485+0000 7f8b23e00640 1 -- 172.21.15.63:0/2730734095 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b280054b0 con 0x56381cbae300 2024-05-27T07:40:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.485+0000 7f8b23e00640 1 -- 172.21.15.63:0/2730734095 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800f9f0 con 0x56381cbae300 2024-05-27T07:40:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.486+0000 7f8b459a2740 1 -- 172.21.15.63:0/2730734095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 msgr2=0x56381cf69390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.486+0000 7f8b459a2740 1 --2- 172.21.15.63:0/2730734095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf69390 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8b28005780 tx=0x7f8b28012d60 comp rx=0 tx=0).stop 2024-05-27T07:40:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.486+0000 7f8b459a2740 1 -- 172.21.15.63:0/2730734095 shutdown_connections 2024-05-27T07:40:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.486+0000 7f8b459a2740 1 --2- 172.21.15.63:0/2730734095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.486+0000 7f8b459a2740 1 --2- 172.21.15.63:0/2730734095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf61e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.486+0000 7f8b459a2740 1 --2- 172.21.15.63:0/2730734095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf69390 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.486+0000 7f8b459a2740 1 -- 172.21.15.63:0/2730734095 >> 172.21.15.63:0/2730734095 conn(0x56381cf5f0b0 msgr2=0x56381cf66fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.486+0000 7f8b459a2740 1 -- 172.21.15.63:0/2730734095 shutdown_connections 2024-05-27T07:40:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.486+0000 7f8b459a2740 1 -- 172.21.15.63:0/2730734095 wait complete. 2024-05-27T07:40:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.487+0000 7f8b459a2740 1 -- start start 2024-05-27T07:40:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.487+0000 7f8b459a2740 1 Processor -- start 2024-05-27T07:40:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.488+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf6f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.488+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf6a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.488+0000 7f8b459a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf6ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.488+0000 7f8b459a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381b18c0 con 0x56381cbae300 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.488+0000 7f8b459a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381b2d30 con 0x56381cb912f0 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.488+0000 7f8b459a2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8b381b1f90 con 0x56381cb994d0 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.488+0000 7f8b459a2740 10 client.0 _ll_register_callbacks cb 0x7f8b2800bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b3ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf6ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b3ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf6ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47880/0 (socket says 172.21.15.63:47880) 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b3ca00640 1 -- 172.21.15.63:0/2464893634 learned_addr learned my addr 172.21.15.63:0/2464893634 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b37e00640 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf6f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b37400640 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf6a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b3ca00640 1 -- 172.21.15.63:0/2464893634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 msgr2=0x56381cf6a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b3ca00640 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf6a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b3ca00640 1 -- 172.21.15.63:0/2464893634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 msgr2=0x56381cf6f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b3ca00640 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf6f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b3ca00640 1 -- 172.21.15.63:0/2464893634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381cf6b3d0 con 0x56381cbae300 2024-05-27T07:40:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.489+0000 7f8b3ca00640 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf6ace0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8b300a6de0 tx=0x7f8b300a5300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f1fee01963b6a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.490+0000 7f8b21600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.490+0000 7f8b21600640 1 -- 172.21.15.63:0/2464893634 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3001f070 con 0x56381cbae300 2024-05-27T07:40:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.490+0000 7f8b21600640 1 -- 172.21.15.63:0/2464893634 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f8b300a8fe0 con 0x56381cbae300 2024-05-27T07:40:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.490+0000 7f8b459a2740 1 -- 172.21.15.63:0/2464893634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b280054b0 con 0x56381cbae300 2024-05-27T07:40:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.490+0000 7f8b21600640 1 -- 172.21.15.63:0/2464893634 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8b300aa270 con 0x56381cbae300 2024-05-27T07:40:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.490+0000 7f8b459a2740 10 client.16710 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.490+0000 7f8b459a2740 1 -- 172.21.15.63:0/2464893634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56381cfe13e0 con 0x56381cbae300 2024-05-27T07:40:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.491+0000 7f8b21600640 1 -- 172.21.15.63:0/2464893634 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b280054b0 con 0x56381cbae300 2024-05-27T07:40:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.491+0000 7f8b21600640 1 -- 172.21.15.63:0/2464893634 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8b3000f040 con 0x56381cbae300 2024-05-27T07:40:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.491+0000 7f8b459a2740 10 client.16710 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.491+0000 7f8b459a2740 10 client.16710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.491+0000 7f8b459a2740 20 client.16710 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.491+0000 7f8b459a2740 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56381cbba1b0 0x56381cfe3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.491+0000 7f8b459a2740 4 client.16710 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:40:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.491+0000 7f8b459a2740 1 -- 172.21.15.63:0/2464893634 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56381cec92c0 con 0x56381cbba1b0 2024-05-27T07:40:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.492+0000 7f8b37e00640 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56381cbba1b0 0x56381cfe3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.493+0000 7f8b37e00640 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56381cbba1b0 0x56381cfe3bc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=909d09caa969d3ea server_cookie=30b8ae369688f487 in_seq=0 out_seq=0 2024-05-27T07:40:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.493+0000 7f8b21600640 10 client.16710 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.528+0000 7f8b21600640 1 -- 172.21.15.63:0/2464893634 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56381cec92c0 con 0x56381cbba1b0 2024-05-27T07:40:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.528+0000 7f8b21600640 10 client.16710 handle_command_reply: tid=0 2024-05-27T07:40:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.528+0000 7f8b459a2740 1 client.16710 shutdown 2024-05-27T07:40:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.528+0000 7f8b459a2740 1 -- 172.21.15.63:0/2464893634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 msgr2=0x56381cf6ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.528+0000 7f8b459a2740 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf6ace0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8b300a6de0 tx=0x7f8b300a5300 comp rx=0 tx=0).stop 2024-05-27T07:40:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.529+0000 7f8b459a2740 1 -- 172.21.15.63:0/2464893634 shutdown_connections 2024-05-27T07:40:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.529+0000 7f8b459a2740 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56381cbba1b0 0x56381cfe3bc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.529+0000 7f8b459a2740 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56381cbae300 0x56381cf6ace0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.529+0000 7f8b459a2740 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56381cb994d0 0x56381cf6a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.529+0000 7f8b459a2740 1 --2- 172.21.15.63:0/2464893634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56381cb912f0 0x56381cf6f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.529+0000 7f8b21600640 0 client.16710 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.529+0000 7f8b459a2740 1 -- 172.21.15.63:0/2464893634 >> 172.21.15.63:0/2464893634 conn(0x56381cf5f0b0 msgr2=0x56381cf66fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.529+0000 7f8b459a2740 1 -- 172.21.15.63:0/2464893634 shutdown_connections 2024-05-27T07:40:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.529+0000 7f8b459a2740 1 -- 172.21.15.63:0/2464893634 wait complete. 2024-05-27T07:40:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.529+0000 7f8b459a2740 20 client.16710 trim_cache size 0 max 16384 2024-05-27T07:40:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:40:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:40:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:40:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.530+0000 7f8b3d400640 1 -- 172.21.15.63:0/637789083 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b1c0b2950 msgr2=0x7f8b1c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.530+0000 7f8b3d400640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b1c0b2950 0x7f8b1c0b4e60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x56381cf5de10 tx=0x7f8b2800b4d0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.530+0000 7f8b3d400640 1 -- 172.21.15.63:0/637789083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b38109f50 msgr2=0x7f8b381a8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.530+0000 7f8b3d400640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b38109f50 0x7f8b381a8500 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000eaa0 tx=0x7f8b3000ef60 comp rx=0 tx=0).stop 2024-05-27T07:40:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.530+0000 7f8b3d400640 1 -- 172.21.15.63:0/637789083 shutdown_connections 2024-05-27T07:40:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.530+0000 7f8b3d400640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b1c0b2950 0x7f8b1c0b4e60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.530+0000 7f8b3d400640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b38109f50 0x7f8b381a8500 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.530+0000 7f8b3d400640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b38109ab0 0x7f8b3819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.530+0000 7f8b3d400640 1 --2- 172.21.15.63:0/637789083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b38104200 0x7f8b3819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.530+0000 7f8b3d400640 1 -- 172.21.15.63:0/637789083 >> 172.21.15.63:0/637789083 conn(0x7f8b38079140 msgr2=0x7f8b38100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.531+0000 7f8b3d400640 1 -- 172.21.15.63:0/637789083 shutdown_connections 2024-05-27T07:40:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:25.531+0000 7f8b3d400640 1 -- 172.21.15.63:0/637789083 wait complete. 2024-05-27T07:40:25.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:25 smithi086 ceph-mon[34687]: pgmap v210: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:25.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:25 smithi149 ceph-mon[35408]: pgmap v210: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:25.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:25 smithi063 ceph-mon[31613]: pgmap v210: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:40:26.877 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:40:27.396 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:27.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:27 smithi086 ceph-mon[34687]: pgmap v211: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:40:27.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:27 smithi149 ceph-mon[35408]: pgmap v211: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:40:27.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:27 smithi063 ceph-mon[31613]: pgmap v211: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:40:29.584 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:29 smithi063 ceph-mon[31613]: pgmap v212: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:40:29.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:29 smithi086 ceph-mon[34687]: pgmap v212: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:40:29.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:29 smithi149 ceph-mon[35408]: pgmap v212: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:40:29.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.973+0000 7f207e800700 1 -- 172.21.15.63:0/2576359547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f207810e900 msgr2=0x7f2078110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:29.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.973+0000 7f207e800700 1 --2- 172.21.15.63:0/2576359547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f207810e900 0x7f2078110de0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f207000b0a0 tx=0x7f207000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.974+0000 7f207e800700 1 -- 172.21.15.63:0/2576359547 shutdown_connections 2024-05-27T07:40:29.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.974+0000 7f207e800700 1 --2- 172.21.15.63:0/2576359547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f207810e900 0x7f2078110de0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.974+0000 7f207e800700 1 --2- 172.21.15.63:0/2576359547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20781054d0 0x7f20781058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.974+0000 7f207e800700 1 --2- 172.21.15.63:0/2576359547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2078105e20 0x7f207810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.974+0000 7f207e800700 1 -- 172.21.15.63:0/2576359547 >> 172.21.15.63:0/2576359547 conn(0x7f20780fef40 msgr2=0x7f2078101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:29.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.974+0000 7f207e800700 1 -- 172.21.15.63:0/2576359547 shutdown_connections 2024-05-27T07:40:29.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.974+0000 7f207e800700 1 -- 172.21.15.63:0/2576359547 wait complete. 2024-05-27T07:40:29.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.975+0000 7f207e800700 1 Processor -- start 2024-05-27T07:40:29.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.975+0000 7f207e800700 1 -- start start 2024-05-27T07:40:29.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.976+0000 7f207e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20781054d0 0x7f207819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.976+0000 7f207e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2078105e20 0x7f207819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.976+0000 7f207e800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f207810e900 0x7f207819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.976+0000 7f207e800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2078114380 con 0x7f2078105e20 2024-05-27T07:40:29.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.976+0000 7f207e800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2078114200 con 0x7f20781054d0 2024-05-27T07:40:29.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.976+0000 7f207e800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2078114500 con 0x7f207810e900 2024-05-27T07:40:29.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.976+0000 7f206fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2078105e20 0x7f207819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.976+0000 7f206fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2078105e20 0x7f207819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44446/0 (socket says 172.21.15.63:44446) 2024-05-27T07:40:29.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.976+0000 7f206fe00700 1 -- 172.21.15.63:0/2721087320 learned_addr learned my addr 172.21.15.63:0/2721087320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:29.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.977+0000 7f2077400700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20781054d0 0x7f207819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.977+0000 7f2077e00700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f207810e900 0x7f207819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.977+0000 7f206fe00700 1 -- 172.21.15.63:0/2721087320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f207810e900 msgr2=0x7f207819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.977+0000 7f206fe00700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f207810e900 0x7f207819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.977+0000 7f206fe00700 1 -- 172.21.15.63:0/2721087320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20781054d0 msgr2=0x7f207819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.977+0000 7f206fe00700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20781054d0 0x7f207819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.977+0000 7f206fe00700 1 -- 172.21.15.63:0/2721087320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207819fc10 con 0x7f2078105e20 2024-05-27T07:40:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.978+0000 7f206fe00700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2078105e20 0x7f207819b330 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f206000e970 tx=0x7f206000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:29.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.978+0000 7f2074c00700 1 -- 172.21.15.63:0/2721087320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060004a30 con 0x7f2078105e20 2024-05-27T07:40:29.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.978+0000 7f2074c00700 1 -- 172.21.15.63:0/2721087320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f206000cc70 con 0x7f2078105e20 2024-05-27T07:40:29.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.978+0000 7f207e800700 1 -- 172.21.15.63:0/2721087320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207819fea0 con 0x7f2078105e20 2024-05-27T07:40:29.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.978+0000 7f2074c00700 1 -- 172.21.15.63:0/2721087320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2060005520 con 0x7f2078105e20 2024-05-27T07:40:29.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.978+0000 7f207e800700 1 -- 172.21.15.63:0/2721087320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20781030f0 con 0x7f2078105e20 2024-05-27T07:40:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.983+0000 7f2074c00700 1 -- 172.21.15.63:0/2721087320 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2060021440 con 0x7f2078105e20 2024-05-27T07:40:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.984+0000 7f208d269b80 1 Processor -- start 2024-05-27T07:40:29.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.984+0000 7f208d269b80 1 -- start start 2024-05-27T07:40:29.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f208d269b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 0x556eaa8be250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f208d269b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa73fe40 0x556eaa8be7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f208d269b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f208d269b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556eaa7ece40 con 0x556eaa8bedb0 2024-05-27T07:40:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f208d269b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556eaa7dc9e0 con 0x556eaa82b9f0 2024-05-27T07:40:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f208d269b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556eaa7e0a20 con 0x556eaa73fe40 2024-05-27T07:40:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f206fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f2077e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 0x556eaa8be250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f206fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44460/0 (socket says 172.21.15.63:44460) 2024-05-27T07:40:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f206fe00700 1 -- 172.21.15.63:0/2679926517 learned_addr learned my addr 172.21.15.63:0/2679926517 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.985+0000 7f2077400700 1 --2- 172.21.15.63:0/2679926517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa73fe40 0x556eaa8be7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.986+0000 7f2074c00700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f205c0b2b20 0x7f205c0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.986+0000 7f2074c00700 1 -- 172.21.15.63:0/2721087320 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f20600e2a30 con 0x7f2078105e20 2024-05-27T07:40:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.986+0000 7f2077e00700 1 -- 172.21.15.63:0/2679926517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa73fe40 msgr2=0x556eaa8be7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.986+0000 7f2077e00700 1 --2- 172.21.15.63:0/2679926517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa73fe40 0x556eaa8be7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.986+0000 7f2077e00700 1 -- 172.21.15.63:0/2679926517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 msgr2=0x556eaa8d3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.986+0000 7f2077e00700 1 --2- 172.21.15.63:0/2679926517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.986+0000 7f2077e00700 1 -- 172.21.15.63:0/2679926517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaa7a5420 con 0x556eaa82b9f0 2024-05-27T07:40:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.986+0000 7f2077400700 1 --2- 172.21.15.63:0/2679926517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa73fe40 0x556eaa8be7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:40:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.986+0000 7f2077e00700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f205c0b2b20 0x7f205c0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.987+0000 7f206fe00700 1 --2- 172.21.15.63:0/2679926517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:40:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.987+0000 7f2077e00700 1 --2- 172.21.15.63:0/2679926517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 0x556eaa8be250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2070009fd0 tx=0x7f20700093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c18b670c48443a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.987+0000 7f206cc00700 1 -- 172.21.15.63:0/2679926517 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207002e070 con 0x556eaa82b9f0 2024-05-27T07:40:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.988+0000 7f2077e00700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f205c0b2b20 0x7f205c0b5030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2070009900 tx=0x7f207002c000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.988+0000 7f206cc00700 1 -- 172.21.15.63:0/2679926517 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f207000bd90 con 0x556eaa82b9f0 2024-05-27T07:40:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.988+0000 7f206cc00700 1 -- 172.21.15.63:0/2679926517 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20700053c0 con 0x556eaa82b9f0 2024-05-27T07:40:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.989+0000 7f208d269b80 1 -- 172.21.15.63:0/2679926517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 msgr2=0x556eaa8be250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.989+0000 7f208d269b80 1 --2- 172.21.15.63:0/2679926517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 0x556eaa8be250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2070009fd0 tx=0x7f20700093b0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.989+0000 7f208d269b80 1 -- 172.21.15.63:0/2679926517 shutdown_connections 2024-05-27T07:40:29.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.989+0000 7f208d269b80 1 --2- 172.21.15.63:0/2679926517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.989+0000 7f208d269b80 1 --2- 172.21.15.63:0/2679926517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 0x556eaa8be250 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.989+0000 7f208d269b80 1 --2- 172.21.15.63:0/2679926517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa73fe40 0x556eaa8be7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.989+0000 7f208d269b80 1 -- 172.21.15.63:0/2679926517 >> 172.21.15.63:0/2679926517 conn(0x556eaa7d9390 msgr2=0x556eaa8ba4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:29.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.989+0000 7f208d269b80 1 -- 172.21.15.63:0/2679926517 shutdown_connections 2024-05-27T07:40:29.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.989+0000 7f208d269b80 1 -- 172.21.15.63:0/2679926517 wait complete. 2024-05-27T07:40:29.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.990+0000 7f208d269b80 1 -- start start 2024-05-27T07:40:29.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.990+0000 7f208d269b80 1 Processor -- start 2024-05-27T07:40:29.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.991+0000 7f208d269b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa73fe40 0x556eaa8ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.991+0000 7f208d269b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa82b9f0 0x556eaa8cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.991+0000 7f208d269b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8bedb0 0x556eaa8cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.991+0000 7f208d269b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556eaa6e2fa0 con 0x556eaa73fe40 2024-05-27T07:40:29.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.991+0000 7f208d269b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556eaa77d760 con 0x556eaa8bedb0 2024-05-27T07:40:29.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.991+0000 7f208d269b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556eaa8d43e0 con 0x556eaa82b9f0 2024-05-27T07:40:29.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.991+0000 7f208d269b80 10 client.0 _ll_register_callbacks cb 0x556eaa786c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:29.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.992+0000 7f2077400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa73fe40 0x556eaa8ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.992+0000 7f2077400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa73fe40 0x556eaa8ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44472/0 (socket says 172.21.15.63:44472) 2024-05-27T07:40:29.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.992+0000 7f2077400700 1 -- 172.21.15.63:0/3872270056 learned_addr learned my addr 172.21.15.63:0/3872270056 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:29.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.992+0000 7f206fe00700 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa82b9f0 0x556eaa8cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.992+0000 7f2077e00700 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8bedb0 0x556eaa8cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.992+0000 7f2077400700 1 -- 172.21.15.63:0/3872270056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa82b9f0 msgr2=0x556eaa8cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.992+0000 7f2077400700 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa82b9f0 0x556eaa8cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.992+0000 7f2077400700 1 -- 172.21.15.63:0/3872270056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8bedb0 msgr2=0x556eaa8cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.992+0000 7f2077400700 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8bedb0 0x556eaa8cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.992+0000 7f2077400700 1 -- 172.21.15.63:0/3872270056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaa72c8d0 con 0x556eaa73fe40 2024-05-27T07:40:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.993+0000 7f2077e00700 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8bedb0 0x556eaa8cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.993+0000 7f2077400700 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa73fe40 0x556eaa8ce890 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f206800bb70 tx=0x7f206800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230cb2d8a1312355 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.993+0000 7f2065600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.993+0000 7f2065600700 1 -- 172.21.15.63:0/3872270056 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068003ca0 con 0x556eaa73fe40 2024-05-27T07:40:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.993+0000 7f2065600700 1 -- 172.21.15.63:0/3872270056 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f2068004830 con 0x556eaa73fe40 2024-05-27T07:40:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.994+0000 7f208d269b80 1 -- 172.21.15.63:0/3872270056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207000bd90 con 0x556eaa73fe40 2024-05-27T07:40:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.994+0000 7f2065600700 1 -- 172.21.15.63:0/3872270056 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2068024050 con 0x556eaa73fe40 2024-05-27T07:40:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.994+0000 7f208d269b80 10 client.16722 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.994+0000 7f208d269b80 1 -- 172.21.15.63:0/3872270056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556eaa7ad060 con 0x556eaa73fe40 2024-05-27T07:40:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.994+0000 7f2065600700 1 -- 172.21.15.63:0/3872270056 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f207000bd90 con 0x556eaa73fe40 2024-05-27T07:40:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.995+0000 7f2065600700 1 -- 172.21.15.63:0/3872270056 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f206801b470 con 0x556eaa73fe40 2024-05-27T07:40:29.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.995+0000 7f208d269b80 10 client.16722 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.995+0000 7f208d269b80 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.995+0000 7f208d269b80 20 client.16722 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.995+0000 7f208d269b80 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556eaa8e13a0 0x556eaa8e3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.995+0000 7f208d269b80 4 client.16722 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:40:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.995+0000 7f208d269b80 1 -- 172.21.15.63:0/3872270056 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eaa7eed90 con 0x556eaa8e13a0 2024-05-27T07:40:29.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.996+0000 7f206fe00700 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556eaa8e13a0 0x556eaa8e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.997+0000 7f206fe00700 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556eaa8e13a0 0x556eaa8e3850 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=422b7a706039fed8 server_cookie=6015e38679c30189 in_seq=0 out_seq=0 2024-05-27T07:40:29.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.997+0000 7f2065600700 10 client.16722 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:29.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f2065600700 1 -- 172.21.15.63:0/3872270056 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556eaa7eed90 con 0x556eaa8e13a0 2024-05-27T07:40:29.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f2065600700 10 client.16722 handle_command_reply: tid=0 2024-05-27T07:40:29.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f208d269b80 1 client.16722 shutdown 2024-05-27T07:40:29.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f208d269b80 1 -- 172.21.15.63:0/3872270056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa73fe40 msgr2=0x556eaa8ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:29.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f208d269b80 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa73fe40 0x556eaa8ce890 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f206800bb70 tx=0x7f206800bf30 comp rx=0 tx=0).stop 2024-05-27T07:40:29.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f208d269b80 1 -- 172.21.15.63:0/3872270056 shutdown_connections 2024-05-27T07:40:29.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f208d269b80 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556eaa8e13a0 0x556eaa8e3850 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f208d269b80 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8bedb0 0x556eaa8cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f208d269b80 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa73fe40 0x556eaa8ce890 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f208d269b80 1 --2- 172.21.15.63:0/3872270056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa82b9f0 0x556eaa8cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:29.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f2065600700 0 client.16722 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:29.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:29.999+0000 7f208d269b80 1 -- 172.21.15.63:0/3872270056 >> 172.21.15.63:0/3872270056 conn(0x556eaa7d9390 msgr2=0x556eaa8ba010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:29.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.000+0000 7f208d269b80 1 -- 172.21.15.63:0/3872270056 shutdown_connections 2024-05-27T07:40:29.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.000+0000 7f208d269b80 1 -- 172.21.15.63:0/3872270056 wait complete. 2024-05-27T07:40:29.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.000+0000 7f208d269b80 20 client.16722 trim_cache size 0 max 16384 2024-05-27T07:40:30.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.040+0000 7f208d269b80 1 Processor -- start 2024-05-27T07:40:30.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.040+0000 7f208d269b80 1 -- start start 2024-05-27T07:40:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.040+0000 7f208d269b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 0x556eaa8be7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.040+0000 7f208d269b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8dd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.040+0000 7f208d269b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa8b9940 0x556eaa8c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.040+0000 7f208d269b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2054011610 con 0x556eaa8bedb0 2024-05-27T07:40:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.040+0000 7f208d269b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f205400eea0 con 0x556eaa82b9f0 2024-05-27T07:40:30.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.040+0000 7f208d269b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f205400cd10 con 0x556eaa8b9940 2024-05-27T07:40:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.041+0000 7f206fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8dd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.041+0000 7f206fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8dd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44486/0 (socket says 172.21.15.63:44486) 2024-05-27T07:40:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.041+0000 7f206fe00700 1 -- 172.21.15.63:0/574748389 learned_addr learned my addr 172.21.15.63:0/574748389 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:30.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.041+0000 7f2077e00700 1 --2- 172.21.15.63:0/574748389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa8b9940 0x556eaa8c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.041+0000 7f2077400700 1 --2- 172.21.15.63:0/574748389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 0x556eaa8be7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.042+0000 7f206fe00700 1 -- 172.21.15.63:0/574748389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa8b9940 msgr2=0x556eaa8c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.042+0000 7f206fe00700 1 --2- 172.21.15.63:0/574748389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa8b9940 0x556eaa8c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.042+0000 7f206fe00700 1 -- 172.21.15.63:0/574748389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 msgr2=0x556eaa8be7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.042+0000 7f206fe00700 1 --2- 172.21.15.63:0/574748389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 0x556eaa8be7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.042+0000 7f206fe00700 1 -- 172.21.15.63:0/574748389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaa8cce00 con 0x556eaa8bedb0 2024-05-27T07:40:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.042+0000 7f206fe00700 1 --2- 172.21.15.63:0/574748389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8dd510 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2060009230 tx=0x7f20600cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d65cafb164bbf02 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.043+0000 7f2067e00700 1 -- 172.21.15.63:0/574748389 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20600a9240 con 0x556eaa8bedb0 2024-05-27T07:40:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.043+0000 7f2067e00700 1 -- 172.21.15.63:0/574748389 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20600b2090 con 0x556eaa8bedb0 2024-05-27T07:40:30.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.043+0000 7f2067e00700 1 -- 172.21.15.63:0/574748389 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20600a5080 con 0x556eaa8bedb0 2024-05-27T07:40:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.044+0000 7f208d269b80 1 -- 172.21.15.63:0/574748389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 msgr2=0x556eaa8dd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.044+0000 7f208d269b80 1 --2- 172.21.15.63:0/574748389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8dd510 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2060009230 tx=0x7f20600cc040 comp rx=0 tx=0).stop 2024-05-27T07:40:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.044+0000 7f208d269b80 1 -- 172.21.15.63:0/574748389 shutdown_connections 2024-05-27T07:40:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.044+0000 7f208d269b80 1 --2- 172.21.15.63:0/574748389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa8b9940 0x556eaa8c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.044+0000 7f208d269b80 1 --2- 172.21.15.63:0/574748389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa82b9f0 0x556eaa8be7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.044+0000 7f208d269b80 1 --2- 172.21.15.63:0/574748389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8dd510 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.044+0000 7f208d269b80 1 -- 172.21.15.63:0/574748389 >> 172.21.15.63:0/574748389 conn(0x556eaa7d9390 msgr2=0x556eaa8be0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.045+0000 7f208d269b80 1 -- 172.21.15.63:0/574748389 shutdown_connections 2024-05-27T07:40:30.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.045+0000 7f208d269b80 1 -- 172.21.15.63:0/574748389 wait complete. 2024-05-27T07:40:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.045+0000 7f208d269b80 1 -- start start 2024-05-27T07:40:30.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.046+0000 7f208d269b80 1 Processor -- start 2024-05-27T07:40:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.047+0000 7f208d269b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa82b9f0 0x556eaa8d5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.047+0000 7f208d269b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8b9940 0x556eaa8d5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.047+0000 7f208d269b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.047+0000 7f208d269b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f205400d360 con 0x556eaa8bedb0 2024-05-27T07:40:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.047+0000 7f208d269b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f205400e7d0 con 0x556eaa8b9940 2024-05-27T07:40:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.047+0000 7f208d269b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f205400da30 con 0x556eaa82b9f0 2024-05-27T07:40:30.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.047+0000 7f208d269b80 10 client.0 _ll_register_callbacks cb 0x556eaa786c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.047+0000 7f206fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8b9940 0x556eaa8d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.047+0000 7f2077e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.048+0000 7f2077e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44498/0 (socket says 172.21.15.63:44498) 2024-05-27T07:40:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.048+0000 7f2077e00700 1 -- 172.21.15.63:0/3798510246 learned_addr learned my addr 172.21.15.63:0/3798510246 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.048+0000 7f2077400700 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa82b9f0 0x556eaa8d5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.048+0000 7f206fe00700 1 -- 172.21.15.63:0/3798510246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa82b9f0 msgr2=0x556eaa8d5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.048+0000 7f206fe00700 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa82b9f0 0x556eaa8d5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.048+0000 7f206fe00700 1 -- 172.21.15.63:0/3798510246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 msgr2=0x556eaa8d5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.048+0000 7f206fe00700 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.048+0000 7f206fe00700 1 -- 172.21.15.63:0/3798510246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaa94b020 con 0x556eaa8b9940 2024-05-27T07:40:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.049+0000 7f206fe00700 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8b9940 0x556eaa8d5aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2060009230 tx=0x7f20600a94f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eb9044a4eae20ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.049+0000 7f2065600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:40:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.049+0000 7f2065600700 1 -- 172.21.15.63:0/3798510246 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20600a9060 con 0x556eaa8b9940 2024-05-27T07:40:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.049+0000 7f208d269b80 1 -- 172.21.15.63:0/3798510246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556eaa7e8570 con 0x556eaa8b9940 2024-05-27T07:40:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.050+0000 7f2065600700 1 -- 172.21.15.63:0/3798510246 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f206000ca70 con 0x556eaa8b9940 2024-05-27T07:40:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.050+0000 7f2065600700 1 -- 172.21.15.63:0/3798510246 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20600a5080 con 0x556eaa8b9940 2024-05-27T07:40:30.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.050+0000 7f208d269b80 10 client.36341 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.050+0000 7f208d269b80 1 -- 172.21.15.63:0/3798510246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556eaa94c290 con 0x556eaa8b9940 2024-05-27T07:40:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.051+0000 7f2065600700 1 -- 172.21.15.63:0/3798510246 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556eaa7e8570 con 0x556eaa8b9940 2024-05-27T07:40:30.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.051+0000 7f2065600700 1 -- 172.21.15.63:0/3798510246 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f20600cc0b0 con 0x556eaa8b9940 2024-05-27T07:40:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.051+0000 7f208d269b80 10 client.36341 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.051+0000 7f208d269b80 10 client.36341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.051+0000 7f208d269b80 20 client.36341 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.051+0000 7f208d269b80 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556eaa94c7e0 0x556eaa94ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.051+0000 7f208d269b80 4 client.36341 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:40:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.051+0000 7f208d269b80 1 -- 172.21.15.63:0/3798510246 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556eaa65f250 con 0x556eaa94c7e0 2024-05-27T07:40:30.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.052+0000 7f2077400700 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556eaa94c7e0 0x556eaa94ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.053+0000 7f2065600700 10 client.36341 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.053+0000 7f2077400700 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556eaa94c7e0 0x556eaa94ec00 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc35e91e1de31ea1 server_cookie=c69b60606b3e999b in_seq=0 out_seq=0 2024-05-27T07:40:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.053+0000 7f2065600700 1 -- 172.21.15.63:0/3798510246 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556eaa65f250 con 0x556eaa94c7e0 2024-05-27T07:40:30.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.053+0000 7f2065600700 10 client.36341 handle_command_reply: tid=0 2024-05-27T07:40:30.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.053+0000 7f208d269b80 1 client.36341 shutdown 2024-05-27T07:40:30.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f208d269b80 1 -- 172.21.15.63:0/3798510246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8b9940 msgr2=0x556eaa8d5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f208d269b80 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8b9940 0x556eaa8d5aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2060009230 tx=0x7f20600a94f0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f208d269b80 1 -- 172.21.15.63:0/3798510246 shutdown_connections 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f208d269b80 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556eaa94c7e0 0x556eaa94ec00 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f208d269b80 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556eaa8bedb0 0x556eaa8d5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f208d269b80 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556eaa82b9f0 0x556eaa8d5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f208d269b80 1 --2- 172.21.15.63:0/3798510246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556eaa8b9940 0x556eaa8d5aa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f208d269b80 1 -- 172.21.15.63:0/3798510246 >> 172.21.15.63:0/3798510246 conn(0x556eaa7d9390 msgr2=0x556eaa8c98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f2065600700 0 client.36341 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f208d269b80 1 -- 172.21.15.63:0/3798510246 shutdown_connections 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.054+0000 7f208d269b80 1 -- 172.21.15.63:0/3798510246 wait complete. 2024-05-27T07:40:30.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.055+0000 7f208d269b80 20 client.36341 trim_cache size 0 max 16384 2024-05-27T07:40:30.053 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:40:30.053 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:40:30.053 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:40:30.053 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:40:30.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.055+0000 7f207e800700 1 -- 172.21.15.63:0/2721087320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f205c0b2b20 msgr2=0x7f205c0b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:30.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.055+0000 7f207e800700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f205c0b2b20 0x7f205c0b5030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2070009900 tx=0x7f207002c000 comp rx=0 tx=0).stop 2024-05-27T07:40:30.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.055+0000 7f207e800700 1 -- 172.21.15.63:0/2721087320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2078105e20 msgr2=0x7f207819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:30.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.055+0000 7f207e800700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2078105e20 0x7f207819b330 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f206000e970 tx=0x7f206000ec80 comp rx=0 tx=0).stop 2024-05-27T07:40:30.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.056+0000 7f207e800700 1 -- 172.21.15.63:0/2721087320 shutdown_connections 2024-05-27T07:40:30.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.056+0000 7f207e800700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f205c0b2b20 0x7f205c0b5030 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.056+0000 7f207e800700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f207810e900 0x7f207819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.056+0000 7f207e800700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20781054d0 0x7f207819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.056+0000 7f207e800700 1 --2- 172.21.15.63:0/2721087320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2078105e20 0x7f207819b330 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.056+0000 7f207e800700 1 -- 172.21.15.63:0/2721087320 >> 172.21.15.63:0/2721087320 conn(0x7f20780fef40 msgr2=0x7f20780ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:30.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.056+0000 7f207e800700 1 -- 172.21.15.63:0/2721087320 shutdown_connections 2024-05-27T07:40:30.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:30.056+0000 7f207e800700 1 -- 172.21.15.63:0/2721087320 wait complete. 2024-05-27T07:40:31.047 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:40:31.047 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:40:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:31.703 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:31 smithi063 ceph-mon[31613]: pgmap v213: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:40:31.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:31 smithi086 ceph-mon[34687]: pgmap v213: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:40:31.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:31 smithi149 ceph-mon[35408]: pgmap v213: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:40:33.655 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:33 smithi063 ceph-mon[31613]: pgmap v214: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:40:33.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:33 smithi086 ceph-mon[34687]: pgmap v214: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:40:33.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:33 smithi149 ceph-mon[35408]: pgmap v214: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:40:34.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.043+0000 7f710be00700 1 -- 172.21.15.63:0/3641477659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710c109670 msgr2=0x7f710c110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:34.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.043+0000 7f710be00700 1 --2- 172.21.15.63:0/3641477659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710c109670 0x7f710c110fd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f710400b0d0 tx=0x7f710400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:40:34.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.043+0000 7f710be00700 1 -- 172.21.15.63:0/3641477659 shutdown_connections 2024-05-27T07:40:34.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.043+0000 7f710be00700 1 --2- 172.21.15.63:0/3641477659 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710c109670 0x7f710c110fd0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:34.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.043+0000 7f710be00700 1 --2- 172.21.15.63:0/3641477659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f710c1043f0 0x7f710c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:34.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.043+0000 7f710be00700 1 --2- 172.21.15.63:0/3641477659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710c108d20 0x7f710c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:34.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.043+0000 7f710be00700 1 -- 172.21.15.63:0/3641477659 >> 172.21.15.63:0/3641477659 conn(0x7f710c079d00 msgr2=0x7f710c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:34.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.044+0000 7f710be00700 1 -- 172.21.15.63:0/3641477659 shutdown_connections 2024-05-27T07:40:34.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.044+0000 7f710be00700 1 -- 172.21.15.63:0/3641477659 wait complete. 2024-05-27T07:40:34.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.046+0000 7f710be00700 1 Processor -- start 2024-05-27T07:40:34.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.046+0000 7f710be00700 1 -- start start 2024-05-27T07:40:34.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.046+0000 7f710be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f710c1043f0 0x7f710c1a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:34.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.046+0000 7f710be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710c108d20 0x7f710c1a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:34.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.047+0000 7f710be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710c109670 0x7f710c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:34.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.047+0000 7f710be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f710c114150 con 0x7f710c109670 2024-05-27T07:40:34.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.047+0000 7f710be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f710c113fd0 con 0x7f710c108d20 2024-05-27T07:40:34.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.047+0000 7f710be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1142d0 con 0x7f710c1043f0 2024-05-27T07:40:34.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.047+0000 7f710b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710c109670 0x7f710c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:34.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.047+0000 7f710b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710c109670 0x7f710c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44510/0 (socket says 172.21.15.63:44510) 2024-05-27T07:40:34.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.047+0000 7f710b400700 1 -- 172.21.15.63:0/1271241755 learned_addr learned my addr 172.21.15.63:0/1271241755 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:34.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.047+0000 7f710aa00700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f710c1043f0 0x7f710c1a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:34.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.047+0000 7f710a000700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710c108d20 0x7f710c1a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:34.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.048+0000 7f710b400700 1 -- 172.21.15.63:0/1271241755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f710c1043f0 msgr2=0x7f710c1a29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:34.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.048+0000 7f710b400700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f710c1043f0 0x7f710c1a29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:34.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.048+0000 7f710b400700 1 -- 172.21.15.63:0/1271241755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710c108d20 msgr2=0x7f710c1a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:34.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.048+0000 7f710b400700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710c108d20 0x7f710c1a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:34.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.048+0000 7f710b400700 1 -- 172.21.15.63:0/1271241755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c19d1f0 con 0x7f710c109670 2024-05-27T07:40:34.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.048+0000 7f710b400700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710c109670 0x7f710c19cb50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7104009850 tx=0x7f7104007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:34.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.049+0000 7f7103400700 1 -- 172.21.15.63:0/1271241755 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710401f070 con 0x7f710c109670 2024-05-27T07:40:34.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.049+0000 7f7103400700 1 -- 172.21.15.63:0/1271241755 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7104005500 con 0x7f710c109670 2024-05-27T07:40:34.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.049+0000 7f7103400700 1 -- 172.21.15.63:0/1271241755 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710401b8f0 con 0x7f710c109670 2024-05-27T07:40:34.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.049+0000 7f710be00700 1 -- 172.21.15.63:0/1271241755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710c19d480 con 0x7f710c109670 2024-05-27T07:40:34.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.049+0000 7f710be00700 1 -- 172.21.15.63:0/1271241755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710c19d9b0 con 0x7f710c109670 2024-05-27T07:40:34.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.050+0000 7f710be00700 1 -- 172.21.15.63:0/1271241755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710c1058a0 con 0x7f710c109670 2024-05-27T07:40:34.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.055+0000 7f7103400700 1 -- 172.21.15.63:0/1271241755 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7104022070 con 0x7f710c109670 2024-05-27T07:40:34.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.056+0000 7f7103400700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70f00b2a70 0x7f70f00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:34.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.057+0000 7f7103400700 1 -- 172.21.15.63:0/1271241755 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f710400f020 con 0x7f710c109670 2024-05-27T07:40:34.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.057+0000 7f710aa00700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70f00b2a70 0x7f70f00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:34.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.058+0000 7f710aa00700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70f00b2a70 0x7f70f00b4f80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f70fc005950 tx=0x7f70fc00b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:34.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.060+0000 7f7103400700 1 -- 172.21.15.63:0/1271241755 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f710c19d9b0 con 0x7f710c109670 2024-05-27T07:40:34.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.364+0000 7f710be00700 1 -- 172.21.15.63:0/1271241755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f710c1a9830 con 0x7f710c109670 2024-05-27T07:40:34.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.366+0000 7f7103400700 1 -- 172.21.15.63:0/1271241755 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f7104029020 con 0x7f710c109670 2024-05-27T07:40:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:40:34.364 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:40:34.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.369+0000 7f710be00700 1 -- 172.21.15.63:0/1271241755 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70f00b2a70 msgr2=0x7f70f00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:34.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.369+0000 7f710be00700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70f00b2a70 0x7f70f00b4f80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f70fc005950 tx=0x7f70fc00b410 comp rx=0 tx=0).stop 2024-05-27T07:40:34.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.369+0000 7f710be00700 1 -- 172.21.15.63:0/1271241755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710c109670 msgr2=0x7f710c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:34.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.369+0000 7f710be00700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710c109670 0x7f710c19cb50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7104009850 tx=0x7f7104007d90 comp rx=0 tx=0).stop 2024-05-27T07:40:34.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.369+0000 7f710be00700 1 -- 172.21.15.63:0/1271241755 shutdown_connections 2024-05-27T07:40:34.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.369+0000 7f710be00700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70f00b2a70 0x7f70f00b4f80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:34.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.369+0000 7f710be00700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710c109670 0x7f710c19cb50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:34.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.369+0000 7f710be00700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f710c1043f0 0x7f710c1a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:34.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.369+0000 7f710be00700 1 --2- 172.21.15.63:0/1271241755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710c108d20 0x7f710c1a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:34.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.369+0000 7f710be00700 1 -- 172.21.15.63:0/1271241755 >> 172.21.15.63:0/1271241755 conn(0x7f710c079d00 msgr2=0x7f710c10f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:34.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.370+0000 7f710be00700 1 -- 172.21.15.63:0/1271241755 shutdown_connections 2024-05-27T07:40:34.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:34.370+0000 7f710be00700 1 -- 172.21.15.63:0/1271241755 wait complete. 2024-05-27T07:40:34.369 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:40:34.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:34 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1271241755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:34.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:34 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1271241755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:34.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:34 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1271241755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:35.358 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:40:35.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:35 smithi086 ceph-mon[34687]: pgmap v215: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:40:35.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:35 smithi149 ceph-mon[35408]: pgmap v215: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:40:35.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:35 smithi063 ceph-mon[31613]: pgmap v215: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:40:35.864 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:37.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:37 smithi086 ceph-mon[34687]: pgmap v216: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:40:37.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:37 smithi149 ceph-mon[35408]: pgmap v216: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:40:37.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:37 smithi063 ceph-mon[31613]: pgmap v216: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:40:38.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.457+0000 7f60d7e00700 1 -- 172.21.15.63:0/2925603756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d01034b0 msgr2=0x7f60d01038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.457+0000 7f60d7e00700 1 --2- 172.21.15.63:0/2925603756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d01034b0 0x7f60d01038c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f60c400b0d0 tx=0x7f60c400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.457+0000 7f60d7e00700 1 -- 172.21.15.63:0/2925603756 shutdown_connections 2024-05-27T07:40:38.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.457+0000 7f60d7e00700 1 --2- 172.21.15.63:0/2925603756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d0108730 0x7f60d0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.457+0000 7f60d7e00700 1 --2- 172.21.15.63:0/2925603756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d01034b0 0x7f60d01038c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.457+0000 7f60d7e00700 1 --2- 172.21.15.63:0/2925603756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d0107de0 0x7f60d01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.457+0000 7f60d7e00700 1 -- 172.21.15.63:0/2925603756 >> 172.21.15.63:0/2925603756 conn(0x7f60d00fedb0 msgr2=0x7f60d01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.457+0000 7f60d7e00700 1 -- 172.21.15.63:0/2925603756 shutdown_connections 2024-05-27T07:40:38.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.457+0000 7f60d7e00700 1 -- 172.21.15.63:0/2925603756 wait complete. 2024-05-27T07:40:38.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.459+0000 7f60d7e00700 1 Processor -- start 2024-05-27T07:40:38.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.459+0000 7f60d7e00700 1 -- start start 2024-05-27T07:40:38.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.459+0000 7f60d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d01034b0 0x7f60d019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.459+0000 7f60d7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d0107de0 0x7f60d019d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.459+0000 7f60d7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d0108730 0x7f60d01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.459+0000 7f60d7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113210 con 0x7f60d0108730 2024-05-27T07:40:38.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.459+0000 7f60d7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113090 con 0x7f60d01034b0 2024-05-27T07:40:38.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.459+0000 7f60d7e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113390 con 0x7f60d0107de0 2024-05-27T07:40:38.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.460+0000 7f60d5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d01034b0 0x7f60d019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.460+0000 7f60d5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d01034b0 0x7f60d019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:42570/0 (socket says 172.21.15.63:42570) 2024-05-27T07:40:38.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.460+0000 7f60d5400700 1 -- 172.21.15.63:0/239071104 learned_addr learned my addr 172.21.15.63:0/239071104 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:38.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.460+0000 7f60d4a00700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d0107de0 0x7f60d019d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.461+0000 7f60d5e00700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d0108730 0x7f60d01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.461+0000 7f60d5400700 1 -- 172.21.15.63:0/239071104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d0107de0 msgr2=0x7f60d019d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.461+0000 7f60d5400700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d0107de0 0x7f60d019d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.461+0000 7f60d5400700 1 -- 172.21.15.63:0/239071104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d0108730 msgr2=0x7f60d01a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.461+0000 7f60d5400700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d0108730 0x7f60d01a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.461+0000 7f60d5400700 1 -- 172.21.15.63:0/239071104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d01a1e40 con 0x7f60d01034b0 2024-05-27T07:40:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.462+0000 7f60d5400700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d01034b0 0x7f60d019cf70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60c4007d90 tx=0x7f60c4007dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.462+0000 7f60ce000700 1 -- 172.21.15.63:0/239071104 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c401f070 con 0x7f60d01034b0 2024-05-27T07:40:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.462+0000 7f60d7e00700 1 -- 172.21.15.63:0/239071104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d01a20d0 con 0x7f60d01034b0 2024-05-27T07:40:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.462+0000 7f60d7e00700 1 -- 172.21.15.63:0/239071104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d01a9a50 con 0x7f60d01034b0 2024-05-27T07:40:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.462+0000 7f60ce000700 1 -- 172.21.15.63:0/239071104 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60c4005500 con 0x7f60d01034b0 2024-05-27T07:40:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.463+0000 7f60ce000700 1 -- 172.21.15.63:0/239071104 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c401b740 con 0x7f60d01034b0 2024-05-27T07:40:38.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.468+0000 7f60ce000700 1 -- 172.21.15.63:0/239071104 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f60c4022070 con 0x7f60d01034b0 2024-05-27T07:40:38.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.468+0000 7f60e68bab80 1 Processor -- start 2024-05-27T07:40:38.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.468+0000 7f60e68bab80 1 -- start start 2024-05-27T07:40:38.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.469+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da6fd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.469+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da6fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.469+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da6fde70 0x5648da70a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.469+0000 7f60e68bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5648da62be40 con 0x5648da57ee40 2024-05-27T07:40:38.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.469+0000 7f60e68bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648da61b9e0 con 0x5648da6fde70 2024-05-27T07:40:38.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.469+0000 7f60e68bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648da61fa20 con 0x5648da66a9f0 2024-05-27T07:40:38.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.469+0000 7f60d5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da6fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60d5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da6fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44532/0 (socket says 172.21.15.63:44532) 2024-05-27T07:40:38.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60d5400700 1 -- 172.21.15.63:0/35683670 learned_addr learned my addr 172.21.15.63:0/35683670 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:38.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60d4a00700 1 --2- 172.21.15.63:0/35683670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da6fde70 0x5648da70a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60d5e00700 1 --2- 172.21.15.63:0/35683670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da6fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60d4a00700 1 -- 172.21.15.63:0/35683670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 msgr2=0x5648da6fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60d4a00700 1 --2- 172.21.15.63:0/35683670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da6fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60d4a00700 1 -- 172.21.15.63:0/35683670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 msgr2=0x5648da6fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60d4a00700 1 --2- 172.21.15.63:0/35683670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da6fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60ce000700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60b80b2a70 0x7f60b80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60d4a00700 1 -- 172.21.15.63:0/35683670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648da5e4420 con 0x5648da6fde70 2024-05-27T07:40:38.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.470+0000 7f60d5400700 1 --2- 172.21.15.63:0/35683670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da6fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:40:38.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.471+0000 7f60ce000700 1 -- 172.21.15.63:0/239071104 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f60c40e34e0 con 0x7f60d01034b0 2024-05-27T07:40:38.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.471+0000 7f60d5e00700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60b80b2a70 0x7f60b80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.471+0000 7f60d4a00700 1 --2- 172.21.15.63:0/35683670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da6fde70 0x5648da70a110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f60bc005950 tx=0x7f60bc00bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f71e1e25db7688 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.472+0000 7f60c2a00700 1 -- 172.21.15.63:0/35683670 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc010c70 con 0x5648da6fde70 2024-05-27T07:40:38.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.472+0000 7f60d5e00700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60b80b2a70 0x7f60b80b4f80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5648da6ffff0 tx=0x7f60c800b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.472+0000 7f60c2a00700 1 -- 172.21.15.63:0/35683670 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60bc010e10 con 0x5648da6fde70 2024-05-27T07:40:38.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.472+0000 7f60c2a00700 1 -- 172.21.15.63:0/35683670 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc005020 con 0x5648da6fde70 2024-05-27T07:40:38.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.473+0000 7f60e68bab80 1 -- 172.21.15.63:0/35683670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da6fde70 msgr2=0x5648da70a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.473+0000 7f60e68bab80 1 --2- 172.21.15.63:0/35683670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da6fde70 0x5648da70a110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f60bc005950 tx=0x7f60bc00bf60 comp rx=0 tx=0).stop 2024-05-27T07:40:38.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.473+0000 7f60e68bab80 1 -- 172.21.15.63:0/35683670 shutdown_connections 2024-05-27T07:40:38.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.473+0000 7f60e68bab80 1 --2- 172.21.15.63:0/35683670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da6fde70 0x5648da70a110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.473+0000 7f60e68bab80 1 --2- 172.21.15.63:0/35683670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da6fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.473+0000 7f60e68bab80 1 --2- 172.21.15.63:0/35683670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da6fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.473+0000 7f60e68bab80 1 -- 172.21.15.63:0/35683670 >> 172.21.15.63:0/35683670 conn(0x5648da618390 msgr2=0x5648da6f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.473+0000 7f60e68bab80 1 -- 172.21.15.63:0/35683670 shutdown_connections 2024-05-27T07:40:38.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.473+0000 7f60e68bab80 1 -- 172.21.15.63:0/35683670 wait complete. 2024-05-27T07:40:38.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.474+0000 7f60e68bab80 1 -- start start 2024-05-27T07:40:38.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.474+0000 7f60e68bab80 1 Processor -- start 2024-05-27T07:40:38.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.475+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da707680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.475+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da66a9f0 0x5648da707bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.475+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da6fde70 0x5648da708100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.475+0000 7f60e68bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5648da521fa0 con 0x5648da57ee40 2024-05-27T07:40:38.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.475+0000 7f60e68bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648da5bc760 con 0x5648da66a9f0 2024-05-27T07:40:38.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.475+0000 7f60e68bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648da70ab80 con 0x5648da6fde70 2024-05-27T07:40:38.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.475+0000 7f60e68bab80 10 client.0 _ll_register_callbacks cb 0x5648da5c5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:38.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da707680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da707680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44548/0 (socket says 172.21.15.63:44548) 2024-05-27T07:40:38.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d5400700 1 -- 172.21.15.63:0/3079103532 learned_addr learned my addr 172.21.15.63:0/3079103532 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:38.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d4a00700 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da66a9f0 0x5648da707bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d5e00700 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da6fde70 0x5648da708100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d5400700 1 -- 172.21.15.63:0/3079103532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da6fde70 msgr2=0x5648da708100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d5400700 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da6fde70 0x5648da708100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d5400700 1 -- 172.21.15.63:0/3079103532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da66a9f0 msgr2=0x5648da707bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d5400700 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da66a9f0 0x5648da707bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d5400700 1 -- 172.21.15.63:0/3079103532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648da56b8d0 con 0x5648da57ee40 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d5e00700 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da6fde70 0x5648da708100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.476+0000 7f60d4a00700 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da66a9f0 0x5648da707bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.477+0000 7f60d5400700 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da707680 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f60c4007e30 tx=0x7f60c40e37e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d425818a8829e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.477+0000 7f60b7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.477+0000 7f60b7400700 1 -- 172.21.15.63:0/3079103532 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40a6350 con 0x5648da57ee40 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.477+0000 7f60b7400700 1 -- 172.21.15.63:0/3079103532 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f60c40aa6f0 con 0x5648da57ee40 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.477+0000 7f60e68bab80 1 -- 172.21.15.63:0/3079103532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60bc010e10 con 0x5648da57ee40 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.477+0000 7f60b7400700 1 -- 172.21.15.63:0/3079103532 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f60c40a9c00 con 0x5648da57ee40 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.477+0000 7f60e68bab80 10 client.16746 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.477+0000 7f60e68bab80 1 -- 172.21.15.63:0/3079103532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5648da5ec060 con 0x5648da57ee40 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.478+0000 7f60b7400700 1 -- 172.21.15.63:0/3079103532 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60bc010e10 con 0x5648da57ee40 2024-05-27T07:40:38.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.478+0000 7f60b7400700 1 -- 172.21.15.63:0/3079103532 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f60c4011070 con 0x5648da57ee40 2024-05-27T07:40:38.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.478+0000 7f60e68bab80 10 client.16746 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:38.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.478+0000 7f60e68bab80 10 client.16746 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:40:38.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.478+0000 7f60e68bab80 20 client.16746 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:38.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.478+0000 7f60e68bab80 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648da717c50 0x5648da71a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.478+0000 7f60e68bab80 4 client.16746 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:40:38.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.478+0000 7f60e68bab80 1 -- 172.21.15.63:0/3079103532 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648da62dd90 con 0x5648da717c50 2024-05-27T07:40:38.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.479+0000 7f60d4a00700 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648da717c50 0x5648da71a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.480+0000 7f60d4a00700 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648da717c50 0x5648da71a0c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c63b11718725132 server_cookie=b62669554f03fb3f in_seq=0 out_seq=0 2024-05-27T07:40:38.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.480+0000 7f60b7400700 10 client.16746 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:40:38.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60b7400700 1 -- 172.21.15.63:0/3079103532 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5648da62dd90 con 0x5648da717c50 2024-05-27T07:40:38.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60b7400700 10 client.16746 handle_command_reply: tid=0 2024-05-27T07:40:38.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60e68bab80 1 client.16746 shutdown 2024-05-27T07:40:38.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60e68bab80 1 -- 172.21.15.63:0/3079103532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 msgr2=0x5648da707680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60e68bab80 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da707680 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f60c4007e30 tx=0x7f60c40e37e0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60e68bab80 1 -- 172.21.15.63:0/3079103532 shutdown_connections 2024-05-27T07:40:38.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60e68bab80 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da6fde70 0x5648da708100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60e68bab80 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648da717c50 0x5648da71a0c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60e68bab80 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da57ee40 0x5648da707680 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60e68bab80 1 --2- 172.21.15.63:0/3079103532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da66a9f0 0x5648da707bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60e68bab80 1 -- 172.21.15.63:0/3079103532 >> 172.21.15.63:0/3079103532 conn(0x5648da618390 msgr2=0x5648da6fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.482+0000 7f60b7400700 0 client.16746 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:40:38.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.483+0000 7f60e68bab80 1 -- 172.21.15.63:0/3079103532 shutdown_connections 2024-05-27T07:40:38.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.483+0000 7f60e68bab80 1 -- 172.21.15.63:0/3079103532 wait complete. 2024-05-27T07:40:38.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.483+0000 7f60e68bab80 20 client.16746 trim_cache size 0 max 16384 2024-05-27T07:40:38.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.504+0000 7f60e68bab80 1 Processor -- start 2024-05-27T07:40:38.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.504+0000 7f60e68bab80 1 -- start start 2024-05-27T07:40:38.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.504+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da57ee40 0x5648da702e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.504+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da7065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.504+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da713cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.504+0000 7f60e68bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0011610 con 0x5648da6fde70 2024-05-27T07:40:38.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.504+0000 7f60e68bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60b000eea0 con 0x5648da57ee40 2024-05-27T07:40:38.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.504+0000 7f60e68bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60b000cd10 con 0x5648da66a9f0 2024-05-27T07:40:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.505+0000 7f60d5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da713cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.505+0000 7f60d5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da713cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44564/0 (socket says 172.21.15.63:44564) 2024-05-27T07:40:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.505+0000 7f60d5e00700 1 -- 172.21.15.63:0/2221883167 learned_addr learned my addr 172.21.15.63:0/2221883167 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.505+0000 7f60d4a00700 1 --2- 172.21.15.63:0/2221883167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da7065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.505+0000 7f60d5400700 1 --2- 172.21.15.63:0/2221883167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da57ee40 0x5648da702e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.505+0000 7f60d5e00700 1 -- 172.21.15.63:0/2221883167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 msgr2=0x5648da7065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.505+0000 7f60d5e00700 1 --2- 172.21.15.63:0/2221883167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da7065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.505+0000 7f60d5e00700 1 -- 172.21.15.63:0/2221883167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da57ee40 msgr2=0x5648da702e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.505+0000 7f60d5e00700 1 --2- 172.21.15.63:0/2221883167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da57ee40 0x5648da702e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.505+0000 7f60d5e00700 1 -- 172.21.15.63:0/2221883167 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648da708470 con 0x5648da6fde70 2024-05-27T07:40:38.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.506+0000 7f60d5e00700 1 --2- 172.21.15.63:0/2221883167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da713cb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f60c8004280 tx=0x7f60c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6608a10d68ea274f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.506+0000 7f60c2000700 1 -- 172.21.15.63:0/2221883167 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c801e0d0 con 0x5648da6fde70 2024-05-27T07:40:38.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.506+0000 7f60c2000700 1 -- 172.21.15.63:0/2221883167 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60c801e270 con 0x5648da6fde70 2024-05-27T07:40:38.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.506+0000 7f60c2000700 1 -- 172.21.15.63:0/2221883167 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c801ed90 con 0x5648da6fde70 2024-05-27T07:40:38.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.507+0000 7f60e68bab80 1 -- 172.21.15.63:0/2221883167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 msgr2=0x5648da713cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.507+0000 7f60e68bab80 1 --2- 172.21.15.63:0/2221883167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da713cb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f60c8004280 tx=0x7f60c8016000 comp rx=0 tx=0).stop 2024-05-27T07:40:38.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.507+0000 7f60e68bab80 1 -- 172.21.15.63:0/2221883167 shutdown_connections 2024-05-27T07:40:38.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.507+0000 7f60e68bab80 1 --2- 172.21.15.63:0/2221883167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da713cb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.507+0000 7f60e68bab80 1 --2- 172.21.15.63:0/2221883167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da57ee40 0x5648da702e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.507+0000 7f60e68bab80 1 --2- 172.21.15.63:0/2221883167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da7065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.507+0000 7f60e68bab80 1 -- 172.21.15.63:0/2221883167 >> 172.21.15.63:0/2221883167 conn(0x5648da618390 msgr2=0x5648da6feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.507+0000 7f60e68bab80 1 -- 172.21.15.63:0/2221883167 shutdown_connections 2024-05-27T07:40:38.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.507+0000 7f60e68bab80 1 -- 172.21.15.63:0/2221883167 wait complete. 2024-05-27T07:40:38.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.508+0000 7f60e68bab80 1 -- start start 2024-05-27T07:40:38.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.508+0000 7f60e68bab80 1 Processor -- start 2024-05-27T07:40:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.509+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da57ee40 0x5648da7103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.509+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da701c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.509+0000 7f60e68bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da7021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.509+0000 7f60e68bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60b000d360 con 0x5648da6fde70 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.509+0000 7f60e68bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60b000e7d0 con 0x5648da57ee40 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.509+0000 7f60e68bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60b000da30 con 0x5648da66a9f0 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.509+0000 7f60e68bab80 10 client.0 _ll_register_callbacks cb 0x5648da5c5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.509+0000 7f60d5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da7021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.509+0000 7f60d5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da7021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44578/0 (socket says 172.21.15.63:44578) 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.510+0000 7f60d5e00700 1 -- 172.21.15.63:0/593672094 learned_addr learned my addr 172.21.15.63:0/593672094 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.510+0000 7f60d5400700 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da57ee40 0x5648da7103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.510+0000 7f60d4a00700 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da701c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.510+0000 7f60d5e00700 1 -- 172.21.15.63:0/593672094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 msgr2=0x5648da701c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.510+0000 7f60d5e00700 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da701c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.510+0000 7f60d5e00700 1 -- 172.21.15.63:0/593672094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da57ee40 msgr2=0x5648da7103f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.510+0000 7f60d5e00700 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da57ee40 0x5648da7103f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.510+0000 7f60d5e00700 1 -- 172.21.15.63:0/593672094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648da7026f0 con 0x5648da6fde70 2024-05-27T07:40:38.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.511+0000 7f60d5e00700 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da7021b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f60c8029040 tx=0x7f60c801e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cdbd78177084ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.511+0000 7f60b7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.511+0000 7f60b7400700 1 -- 172.21.15.63:0/593672094 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8034060 con 0x5648da6fde70 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.511+0000 7f60b7400700 1 -- 172.21.15.63:0/593672094 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f60c8005260 con 0x5648da6fde70 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.511+0000 7f60e68bab80 1 -- 172.21.15.63:0/593672094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60c801e270 con 0x5648da6fde70 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.511+0000 7f60b7400700 1 -- 172.21.15.63:0/593672094 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f60c8005460 con 0x5648da6fde70 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.511+0000 7f60e68bab80 10 client.16752 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.511+0000 7f60e68bab80 1 -- 172.21.15.63:0/593672094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5648da70eb60 con 0x5648da6fde70 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.512+0000 7f60b7400700 1 -- 172.21.15.63:0/593672094 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60c801e270 con 0x5648da6fde70 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.512+0000 7f60b7400700 1 -- 172.21.15.63:0/593672094 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f60c8027530 con 0x5648da6fde70 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.512+0000 7f60e68bab80 10 client.16752 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:38.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.512+0000 7f60e68bab80 10 client.16752 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:40:38.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.512+0000 7f60e68bab80 20 client.16752 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:38.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.512+0000 7f60e68bab80 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648da782aa0 0x5648da784ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.512+0000 7f60e68bab80 4 client.16752 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:40:38.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.512+0000 7f60e68bab80 1 -- 172.21.15.63:0/593672094 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648da627570 con 0x5648da782aa0 2024-05-27T07:40:38.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.513+0000 7f60d5400700 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648da782aa0 0x5648da784ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.514+0000 7f60d5400700 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648da782aa0 0x5648da784ec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69e8b558b21269f6 server_cookie=ddf41ebd944f9115 in_seq=0 out_seq=0 2024-05-27T07:40:38.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.514+0000 7f60b7400700 10 client.16752 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:40:38.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.514+0000 7f60b7400700 1 -- 172.21.15.63:0/593672094 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648da627570 con 0x5648da782aa0 2024-05-27T07:40:38.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.514+0000 7f60b7400700 10 client.16752 handle_command_reply: tid=0 2024-05-27T07:40:38.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.514+0000 7f60e68bab80 1 client.16752 shutdown 2024-05-27T07:40:38.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 1 -- 172.21.15.63:0/593672094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 msgr2=0x5648da7021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da7021b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f60c8029040 tx=0x7f60c801e8a0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 1 -- 172.21.15.63:0/593672094 shutdown_connections 2024-05-27T07:40:38.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648da782aa0 0x5648da784ec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648da6fde70 0x5648da7021b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648da57ee40 0x5648da7103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 1 --2- 172.21.15.63:0/593672094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648da66a9f0 0x5648da701c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 1 -- 172.21.15.63:0/593672094 >> 172.21.15.63:0/593672094 conn(0x5648da618390 msgr2=0x5648da6feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60b7400700 0 client.16752 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:40:38.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 1 -- 172.21.15.63:0/593672094 shutdown_connections 2024-05-27T07:40:38.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 1 -- 172.21.15.63:0/593672094 wait complete. 2024-05-27T07:40:38.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.515+0000 7f60e68bab80 20 client.16752 trim_cache size 0 max 16384 2024-05-27T07:40:38.514 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:40:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.516+0000 7f60d7e00700 1 -- 172.21.15.63:0/239071104 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60b80b2a70 msgr2=0x7f60b80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.516+0000 7f60d7e00700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60b80b2a70 0x7f60b80b4f80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5648da6ffff0 tx=0x7f60c800b390 comp rx=0 tx=0).stop 2024-05-27T07:40:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.516+0000 7f60d7e00700 1 -- 172.21.15.63:0/239071104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d01034b0 msgr2=0x7f60d019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.516+0000 7f60d7e00700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d01034b0 0x7f60d019cf70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60c4007d90 tx=0x7f60c4007dc0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.517+0000 7f60d7e00700 1 -- 172.21.15.63:0/239071104 shutdown_connections 2024-05-27T07:40:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.517+0000 7f60d7e00700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60b80b2a70 0x7f60b80b4f80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.517+0000 7f60d7e00700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60d0108730 0x7f60d01a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.517+0000 7f60d7e00700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d01034b0 0x7f60d019cf70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.517+0000 7f60d7e00700 1 --2- 172.21.15.63:0/239071104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60d0107de0 0x7f60d019d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.517+0000 7f60d7e00700 1 -- 172.21.15.63:0/239071104 >> 172.21.15.63:0/239071104 conn(0x7f60d00fedb0 msgr2=0x7f60d01097d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.517+0000 7f60d7e00700 1 -- 172.21.15.63:0/239071104 shutdown_connections 2024-05-27T07:40:38.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:38.517+0000 7f60d7e00700 1 -- 172.21.15.63:0/239071104 wait complete. 2024-05-27T07:40:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.780+0000 7ff691800640 1 -- 172.21.15.63:0/957515946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff68c108ec0 msgr2=0x7ff68c102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.780+0000 7ff691800640 1 --2- 172.21.15.63:0/957515946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff68c108ec0 0x7ff68c102b10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff678009900 tx=0x7ff67802a410 comp rx=0 tx=0).stop 2024-05-27T07:40:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.780+0000 7ff691800640 1 -- 172.21.15.63:0/957515946 shutdown_connections 2024-05-27T07:40:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.780+0000 7ff691800640 1 --2- 172.21.15.63:0/957515946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c103180 0x7ff68c10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.780+0000 7ff691800640 1 --2- 172.21.15.63:0/957515946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff68c108ec0 0x7ff68c102b10 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.780+0000 7ff691800640 1 --2- 172.21.15.63:0/957515946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff68c108a20 0x7ff68c1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.780+0000 7ff691800640 1 -- 172.21.15.63:0/957515946 >> 172.21.15.63:0/957515946 conn(0x7ff68c078160 msgr2=0x7ff68c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.780+0000 7ff691800640 1 -- 172.21.15.63:0/957515946 shutdown_connections 2024-05-27T07:40:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.781+0000 7ff691800640 1 -- 172.21.15.63:0/957515946 wait complete. 2024-05-27T07:40:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.782+0000 7ff691800640 1 Processor -- start 2024-05-27T07:40:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.782+0000 7ff691800640 1 -- start start 2024-05-27T07:40:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.783+0000 7ff691800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff68c103180 0x7ff68c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.783+0000 7ff691800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c108a20 0x7ff68c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.783+0000 7ff691800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff68c108ec0 0x7ff68c1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.783+0000 7ff691800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c112a30 con 0x7ff68c108ec0 2024-05-27T07:40:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.783+0000 7ff691800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1128b0 con 0x7ff68c108a20 2024-05-27T07:40:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.783+0000 7ff691800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c112bb0 con 0x7ff68c103180 2024-05-27T07:40:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.783+0000 7ff690e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff68c108ec0 0x7ff68c1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.783+0000 7ff68be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff68c103180 0x7ff68c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.784+0000 7ff68b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c108a20 0x7ff68c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.784+0000 7ff68be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff68c103180 0x7ff68c19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:38096/0 (socket says 172.21.15.63:38096) 2024-05-27T07:40:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.784+0000 7ff68be00640 1 -- 172.21.15.63:0/2900348694 learned_addr learned my addr 172.21.15.63:0/2900348694 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.784+0000 7ff690e00640 1 -- 172.21.15.63:0/2900348694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff68c103180 msgr2=0x7ff68c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.784+0000 7ff690e00640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff68c103180 0x7ff68c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.784+0000 7ff690e00640 1 -- 172.21.15.63:0/2900348694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c108a20 msgr2=0x7ff68c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.784+0000 7ff690e00640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c108a20 0x7ff68c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.784+0000 7ff690e00640 1 -- 172.21.15.63:0/2900348694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c1a1600 con 0x7ff68c108ec0 2024-05-27T07:40:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.785+0000 7ff690e00640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff68c108ec0 0x7ff68c1a0fc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff68400da10 tx=0x7ff68400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.785+0000 7ff688c00640 1 -- 172.21.15.63:0/2900348694 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68400bd70 con 0x7ff68c108ec0 2024-05-27T07:40:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.785+0000 7ff688c00640 1 -- 172.21.15.63:0/2900348694 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff684003e00 con 0x7ff68c108ec0 2024-05-27T07:40:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.785+0000 7ff691800640 1 -- 172.21.15.63:0/2900348694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c1a18f0 con 0x7ff68c108ec0 2024-05-27T07:40:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.785+0000 7ff688c00640 1 -- 172.21.15.63:0/2900348694 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6840106c0 con 0x7ff68c108ec0 2024-05-27T07:40:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.785+0000 7ff691800640 1 -- 172.21.15.63:0/2900348694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c1a1dc0 con 0x7ff68c108ec0 2024-05-27T07:40:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.790+0000 7ff688c00640 1 -- 172.21.15.63:0/2900348694 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff684003970 con 0x7ff68c108ec0 2024-05-27T07:40:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.790+0000 7ff699dc7740 1 Processor -- start 2024-05-27T07:40:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.791+0000 7ff699dc7740 1 -- start start 2024-05-27T07:40:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.791+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1ff1b0 0x561bff5a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.791+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.791+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1de4d0 0x561bff5a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.791+0000 7ff699dc7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561bff512b90 con 0x561bff1d62f0 2024-05-27T07:40:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.791+0000 7ff699dc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bff502730 con 0x561bff1de4d0 2024-05-27T07:40:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.791+0000 7ff699dc7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561bff506770 con 0x561bff1ff1b0 2024-05-27T07:40:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.792+0000 7ff68be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.792+0000 7ff68be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44602/0 (socket says 172.21.15.63:44602) 2024-05-27T07:40:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.792+0000 7ff68be00640 1 -- 172.21.15.63:0/3292184243 learned_addr learned my addr 172.21.15.63:0/3292184243 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.792+0000 7ff68b400640 1 --2- 172.21.15.63:0/3292184243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1ff1b0 0x561bff5a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.792+0000 7ff68b400640 1 --2- 172.21.15.63:0/3292184243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1de4d0 0x561bff5a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.792+0000 7ff688c00640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff6700b2950 0x7ff6700b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.793+0000 7ff68be00640 1 -- 172.21.15.63:0/3292184243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1ff1b0 msgr2=0x561bff5a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.793+0000 7ff68be00640 1 --2- 172.21.15.63:0/3292184243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1ff1b0 0x561bff5a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.793+0000 7ff68be00640 1 -- 172.21.15.63:0/3292184243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1de4d0 msgr2=0x561bff5a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.793+0000 7ff688c00640 1 -- 172.21.15.63:0/2900348694 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7ff6840e2e50 con 0x7ff68c108ec0 2024-05-27T07:40:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.793+0000 7ff68be00640 1 --2- 172.21.15.63:0/3292184243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1de4d0 0x561bff5a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.793+0000 7ff68be00640 1 -- 172.21.15.63:0/3292184243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bff49b8b0 con 0x561bff1d62f0 2024-05-27T07:40:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.793+0000 7ff690e00640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff6700b2950 0x7ff6700b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.793+0000 7ff68be00640 1 --2- 172.21.15.63:0/3292184243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a1fa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00bcc0 tx=0x7ff67c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf8a070effbf7d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.794+0000 7ff681600640 1 -- 172.21.15.63:0/3292184243 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c004230 con 0x561bff1d62f0 2024-05-27T07:40:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.794+0000 7ff681600640 1 -- 172.21.15.63:0/3292184243 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff67c0043d0 con 0x561bff1d62f0 2024-05-27T07:40:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.794+0000 7ff681600640 1 -- 172.21.15.63:0/3292184243 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c0116a0 con 0x561bff1d62f0 2024-05-27T07:40:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.795+0000 7ff699dc7740 1 -- 172.21.15.63:0/3292184243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 msgr2=0x561bff5a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.795+0000 7ff699dc7740 1 --2- 172.21.15.63:0/3292184243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a1fa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00bcc0 tx=0x7ff67c0107a0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.795+0000 7ff690e00640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff6700b2950 0x7ff6700b4e60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff68400b870 tx=0x7ff684004540 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.795+0000 7ff699dc7740 1 -- 172.21.15.63:0/3292184243 shutdown_connections 2024-05-27T07:40:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.795+0000 7ff699dc7740 1 --2- 172.21.15.63:0/3292184243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1de4d0 0x561bff5a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.795+0000 7ff699dc7740 1 --2- 172.21.15.63:0/3292184243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a1fa0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.795+0000 7ff699dc7740 1 --2- 172.21.15.63:0/3292184243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1ff1b0 0x561bff5a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.795+0000 7ff699dc7740 1 -- 172.21.15.63:0/3292184243 >> 172.21.15.63:0/3292184243 conn(0x561bff1f3300 msgr2=0x561bff59e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.796+0000 7ff699dc7740 1 -- 172.21.15.63:0/3292184243 shutdown_connections 2024-05-27T07:40:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.796+0000 7ff699dc7740 1 -- 172.21.15.63:0/3292184243 wait complete. 2024-05-27T07:40:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.796+0000 7ff699dc7740 1 -- start start 2024-05-27T07:40:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.796+0000 7ff699dc7740 1 Processor -- start 2024-05-27T07:40:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.797+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1d62f0 0x561bff5abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.797+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1de4d0 0x561bff5ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.797+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.797+0000 7ff699dc7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561bff1e8790 con 0x561bff1ff1b0 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.797+0000 7ff699dc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bff4aabc0 con 0x561bff1de4d0 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.797+0000 7ff699dc7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561bff5aeb90 con 0x561bff1d62f0 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.797+0000 7ff699dc7740 10 client.0 _ll_register_callbacks cb 0x561bff488400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44604/0 (socket says 172.21.15.63:44604) 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68b400640 1 -- 172.21.15.63:0/742531173 learned_addr learned my addr 172.21.15.63:0/742531173 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68be00640 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1de4d0 0x561bff5ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68b400640 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1d62f0 0x561bff5abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68b400640 1 -- 172.21.15.63:0/742531173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1d62f0 msgr2=0x561bff5abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68b400640 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1d62f0 0x561bff5abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68b400640 1 -- 172.21.15.63:0/742531173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1de4d0 msgr2=0x561bff5ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68b400640 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1de4d0 0x561bff5ac420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68b400640 1 -- 172.21.15.63:0/742531173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bff217210 con 0x561bff1ff1b0 2024-05-27T07:40:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.798+0000 7ff68b400640 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b7ec0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff678009900 tx=0x7ff678009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5628f1755b664b71 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.799+0000 7ff676000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.799+0000 7ff676000640 1 -- 172.21.15.63:0/742531173 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678035040 con 0x561bff1ff1b0 2024-05-27T07:40:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.799+0000 7ff699dc7740 1 -- 172.21.15.63:0/742531173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff67c0043d0 con 0x561bff1ff1b0 2024-05-27T07:40:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.799+0000 7ff676000640 1 -- 172.21.15.63:0/742531173 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7ff6780035f0 con 0x561bff1ff1b0 2024-05-27T07:40:38.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.799+0000 7ff676000640 1 -- 172.21.15.63:0/742531173 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff678053050 con 0x561bff1ff1b0 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.799+0000 7ff699dc7740 10 client.16764 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.799+0000 7ff699dc7740 1 -- 172.21.15.63:0/742531173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561bff2e3620 con 0x561bff1ff1b0 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.800+0000 7ff676000640 1 -- 172.21.15.63:0/742531173 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff67c0043d0 con 0x561bff1ff1b0 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.800+0000 7ff676000640 1 -- 172.21.15.63:0/742531173 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff67803d610 con 0x561bff1ff1b0 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.800+0000 7ff699dc7740 10 client.16764 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.800+0000 7ff699dc7740 10 client.16764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.800+0000 7ff699dc7740 20 client.16764 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.800+0000 7ff699dc7740 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bff1bdd80 0x561bff5bdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.800+0000 7ff699dc7740 4 client.16764 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.800+0000 7ff699dc7740 1 -- 172.21.15.63:0/742531173 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bff514ae0 con 0x561bff1bdd80 2024-05-27T07:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.801+0000 7ff68be00640 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bff1bdd80 0x561bff5bdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.802+0000 7ff68be00640 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bff1bdd80 0x561bff5bdef0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89236e30c5cc0b8 server_cookie=cd18eea58b570f35 in_seq=0 out_seq=0 2024-05-27T07:40:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.802+0000 7ff676000640 10 client.16764 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.803+0000 7ff676000640 1 -- 172.21.15.63:0/742531173 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561bff514ae0 con 0x561bff1bdd80 2024-05-27T07:40:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.803+0000 7ff676000640 10 client.16764 handle_command_reply: tid=0 2024-05-27T07:40:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 client.16764 shutdown 2024-05-27T07:40:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 -- 172.21.15.63:0/742531173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 msgr2=0x561bff5b7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b7ec0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff678009900 tx=0x7ff678009260 comp rx=0 tx=0).stop 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 -- 172.21.15.63:0/742531173 shutdown_connections 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bff1bdd80 0x561bff5bdef0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b7ec0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1de4d0 0x561bff5ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 --2- 172.21.15.63:0/742531173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1d62f0 0x561bff5abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff676000640 0 client.16764 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 -- 172.21.15.63:0/742531173 >> 172.21.15.63:0/742531173 conn(0x561bff1f3300 msgr2=0x561bff5a1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 -- 172.21.15.63:0/742531173 shutdown_connections 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 1 -- 172.21.15.63:0/742531173 wait complete. 2024-05-27T07:40:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.804+0000 7ff699dc7740 20 client.16764 trim_cache size 0 max 16384 2024-05-27T07:40:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.822+0000 7ff699dc7740 1 Processor -- start 2024-05-27T07:40:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.822+0000 7ff699dc7740 1 -- start start 2024-05-27T07:40:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.823+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.823+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1de4d0 0x561bff5adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.823+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1ff1b0 0x561bff5ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.823+0000 7ff699dc7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1b5b60 con 0x561bff1d62f0 2024-05-27T07:40:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.823+0000 7ff699dc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1b3430 con 0x561bff1ff1b0 2024-05-27T07:40:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.823+0000 7ff699dc7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1b12a0 con 0x561bff1de4d0 2024-05-27T07:40:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.823+0000 7ff68b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.823+0000 7ff68b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44618/0 (socket says 172.21.15.63:44618) 2024-05-27T07:40:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.823+0000 7ff68b400640 1 -- 172.21.15.63:0/1462198059 learned_addr learned my addr 172.21.15.63:0/1462198059 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.824+0000 7ff68b400640 1 -- 172.21.15.63:0/1462198059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1de4d0 msgr2=0x561bff5adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.824+0000 7ff68b400640 1 --2- 172.21.15.63:0/1462198059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1de4d0 0x561bff5adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.824+0000 7ff68b400640 1 -- 172.21.15.63:0/1462198059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1ff1b0 msgr2=0x561bff5ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.824+0000 7ff68b400640 1 --2- 172.21.15.63:0/1462198059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1ff1b0 0x561bff5ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.824+0000 7ff68b400640 1 -- 172.21.15.63:0/1462198059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bff5a61f0 con 0x561bff1d62f0 2024-05-27T07:40:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.824+0000 7ff68b400640 1 --2- 172.21.15.63:0/1462198059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a7440 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff6780319f0 tx=0x7ff678005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb46fa5674a27463 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.825+0000 7ff680c00640 1 -- 172.21.15.63:0/1462198059 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678035040 con 0x561bff1d62f0 2024-05-27T07:40:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.825+0000 7ff680c00640 1 -- 172.21.15.63:0/1462198059 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff678005850 con 0x561bff1d62f0 2024-05-27T07:40:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.825+0000 7ff680c00640 1 -- 172.21.15.63:0/1462198059 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678031c80 con 0x561bff1d62f0 2024-05-27T07:40:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.826+0000 7ff699dc7740 1 -- 172.21.15.63:0/1462198059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 msgr2=0x561bff5a7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.826+0000 7ff699dc7740 1 --2- 172.21.15.63:0/1462198059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a7440 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff6780319f0 tx=0x7ff678005fd0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.826+0000 7ff699dc7740 1 -- 172.21.15.63:0/1462198059 shutdown_connections 2024-05-27T07:40:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.826+0000 7ff699dc7740 1 --2- 172.21.15.63:0/1462198059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1ff1b0 0x561bff5ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.826+0000 7ff699dc7740 1 --2- 172.21.15.63:0/1462198059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1de4d0 0x561bff5adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.826+0000 7ff699dc7740 1 --2- 172.21.15.63:0/1462198059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1d62f0 0x561bff5a7440 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.826+0000 7ff699dc7740 1 -- 172.21.15.63:0/1462198059 >> 172.21.15.63:0/1462198059 conn(0x561bff1f3300 msgr2=0x561bff5ad3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.826+0000 7ff699dc7740 1 -- 172.21.15.63:0/1462198059 shutdown_connections 2024-05-27T07:40:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.826+0000 7ff699dc7740 1 -- 172.21.15.63:0/1462198059 wait complete. 2024-05-27T07:40:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.826+0000 7ff699dc7740 1 -- start start 2024-05-27T07:40:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.827+0000 7ff699dc7740 1 Processor -- start 2024-05-27T07:40:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.827+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1d62f0 0x561bff5b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.827+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1de4d0 0x561bff5b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.827+0000 7ff699dc7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.827+0000 7ff699dc7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1b18f0 con 0x561bff1ff1b0 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.827+0000 7ff699dc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1b2d60 con 0x561bff1d62f0 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.827+0000 7ff699dc7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1b1fc0 con 0x561bff1de4d0 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.827+0000 7ff699dc7740 10 client.0 _ll_register_callbacks cb 0x7ff678032c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44630/0 (socket says 172.21.15.63:44630) 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68b400640 1 -- 172.21.15.63:0/321193489 learned_addr learned my addr 172.21.15.63:0/321193489 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68b400640 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1d62f0 0x561bff5b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68be00640 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1de4d0 0x561bff5b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68b400640 1 -- 172.21.15.63:0/321193489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1de4d0 msgr2=0x561bff5b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68b400640 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1de4d0 0x561bff5b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68b400640 1 -- 172.21.15.63:0/321193489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1d62f0 msgr2=0x561bff5b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68b400640 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1d62f0 0x561bff5b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68b400640 1 -- 172.21.15.63:0/321193489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bff5b2090 con 0x561bff1ff1b0 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.828+0000 7ff68be00640 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1de4d0 0x561bff5b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.829+0000 7ff68b400640 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b1b50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff678009900 tx=0x7ff678000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210b5d125dec52f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.829+0000 7ff676000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.829+0000 7ff676000640 1 -- 172.21.15.63:0/321193489 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67805d040 con 0x561bff1ff1b0 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.829+0000 7ff699dc7740 1 -- 172.21.15.63:0/321193489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff678005850 con 0x561bff1ff1b0 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.829+0000 7ff676000640 1 -- 172.21.15.63:0/321193489 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7ff67803d650 con 0x561bff1ff1b0 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.829+0000 7ff676000640 1 -- 172.21.15.63:0/321193489 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff678035040 con 0x561bff1ff1b0 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.829+0000 7ff699dc7740 10 client.16770 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.830+0000 7ff699dc7740 1 -- 172.21.15.63:0/321193489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561bff6261f0 con 0x561bff1ff1b0 2024-05-27T07:40:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.830+0000 7ff676000640 1 -- 172.21.15.63:0/321193489 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff678005850 con 0x561bff1ff1b0 2024-05-27T07:40:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.830+0000 7ff676000640 1 -- 172.21.15.63:0/321193489 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff678063030 con 0x561bff1ff1b0 2024-05-27T07:40:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.830+0000 7ff699dc7740 10 client.16770 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.830+0000 7ff699dc7740 10 client.16770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.830+0000 7ff699dc7740 20 client.16770 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.830+0000 7ff699dc7740 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bff1931c0 0x561bff628820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.830+0000 7ff699dc7740 4 client.16770 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:40:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.830+0000 7ff699dc7740 1 -- 172.21.15.63:0/321193489 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bff50e2c0 con 0x561bff1931c0 2024-05-27T07:40:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.831+0000 7ff68be00640 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bff1931c0 0x561bff628820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.832+0000 7ff68be00640 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bff1931c0 0x561bff628820 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da43a148b3186a7a server_cookie=1575e21986c9d36 in_seq=0 out_seq=0 2024-05-27T07:40:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.832+0000 7ff676000640 10 client.16770 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff676000640 1 -- 172.21.15.63:0/321193489 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561bff50e2c0 con 0x561bff1931c0 2024-05-27T07:40:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff676000640 10 client.16770 handle_command_reply: tid=0 2024-05-27T07:40:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff699dc7740 1 client.16770 shutdown 2024-05-27T07:40:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff699dc7740 1 -- 172.21.15.63:0/321193489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 msgr2=0x561bff5b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff699dc7740 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b1b50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff678009900 tx=0x7ff678000bc0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff699dc7740 1 -- 172.21.15.63:0/321193489 shutdown_connections 2024-05-27T07:40:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff699dc7740 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bff1931c0 0x561bff628820 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff699dc7740 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bff1ff1b0 0x561bff5b1b50 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff699dc7740 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bff1de4d0 0x561bff5b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff699dc7740 1 --2- 172.21.15.63:0/321193489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bff1d62f0 0x561bff5b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff676000640 0 client.16770 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.866+0000 7ff699dc7740 1 -- 172.21.15.63:0/321193489 >> 172.21.15.63:0/321193489 conn(0x561bff1f3300 msgr2=0x561bff5addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.867+0000 7ff699dc7740 1 -- 172.21.15.63:0/321193489 shutdown_connections 2024-05-27T07:40:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.867+0000 7ff699dc7740 1 -- 172.21.15.63:0/321193489 wait complete. 2024-05-27T07:40:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.867+0000 7ff699dc7740 20 client.16770 trim_cache size 0 max 16384 2024-05-27T07:40:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:40:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:40:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:40:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 -- 172.21.15.63:0/2900348694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff6700b2950 msgr2=0x7ff6700b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff6700b2950 0x7ff6700b4e60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff68400b870 tx=0x7ff684004540 comp rx=0 tx=0).stop 2024-05-27T07:40:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 -- 172.21.15.63:0/2900348694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff68c108ec0 msgr2=0x7ff68c1a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff68c108ec0 0x7ff68c1a0fc0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff68400da10 tx=0x7ff68400ded0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 -- 172.21.15.63:0/2900348694 shutdown_connections 2024-05-27T07:40:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff6700b2950 0x7ff6700b4e60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff68c108ec0 0x7ff68c1a0fc0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c108a20 0x7ff68c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 --2- 172.21.15.63:0/2900348694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff68c103180 0x7ff68c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 -- 172.21.15.63:0/2900348694 >> 172.21.15.63:0/2900348694 conn(0x7ff68c078160 msgr2=0x7ff68c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 -- 172.21.15.63:0/2900348694 shutdown_connections 2024-05-27T07:40:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:38.868+0000 7ff691800640 1 -- 172.21.15.63:0/2900348694 wait complete. 2024-05-27T07:40:39.474 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:40:39.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:39 smithi086 ceph-mon[34687]: pgmap v217: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:40:39.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:39 smithi149 ceph-mon[35408]: pgmap v217: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:40:39.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:39 smithi063 ceph-mon[31613]: pgmap v217: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:40:39.978 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:41.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:41 smithi086 ceph-mon[34687]: pgmap v218: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 8 op/s 2024-05-27T07:40:41.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:41 smithi149 ceph-mon[35408]: pgmap v218: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 8 op/s 2024-05-27T07:40:41.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:41 smithi063 ceph-mon[31613]: pgmap v218: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 8 op/s 2024-05-27T07:40:42.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.574+0000 7f7322800700 1 -- 172.21.15.63:0/1747400995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731c1032d0 msgr2=0x7f731c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.574+0000 7f7322800700 1 --2- 172.21.15.63:0/1747400995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731c1032d0 0x7f731c1036e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f73100099c0 tx=0x7f7310009cd0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.574+0000 7f7322800700 1 -- 172.21.15.63:0/1747400995 shutdown_connections 2024-05-27T07:40:42.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.574+0000 7f7322800700 1 --2- 172.21.15.63:0/1747400995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f731c108550 0x7f731c10feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.574+0000 7f7322800700 1 --2- 172.21.15.63:0/1747400995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731c1032d0 0x7f731c1036e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.574+0000 7f7322800700 1 --2- 172.21.15.63:0/1747400995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731c107c00 0x7f731c108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.574+0000 7f7322800700 1 -- 172.21.15.63:0/1747400995 >> 172.21.15.63:0/1747400995 conn(0x7f731c078ab0 msgr2=0x7f731c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:42.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.575+0000 7f7322800700 1 -- 172.21.15.63:0/1747400995 shutdown_connections 2024-05-27T07:40:42.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.575+0000 7f7322800700 1 -- 172.21.15.63:0/1747400995 wait complete. 2024-05-27T07:40:42.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.576+0000 7f7322800700 1 Processor -- start 2024-05-27T07:40:42.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.576+0000 7f7322800700 1 -- start start 2024-05-27T07:40:42.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.576+0000 7f7322800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731c1032d0 0x7f731c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.576+0000 7f7322800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731c107c00 0x7f731c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7322800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f731c108550 0x7f731c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7322800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1130a0 con 0x7f731c1032d0 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7322800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f731c112f20 con 0x7f731c107c00 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7322800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f731c113220 con 0x7f731c108550 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7321400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731c1032d0 0x7f731c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7321400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731c1032d0 0x7f731c1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56836/0 (socket says 172.21.15.63:56836) 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7321400700 1 -- 172.21.15.63:0/686520474 learned_addr learned my addr 172.21.15.63:0/686520474 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7320a00700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731c107c00 0x7f731c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7321e00700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f731c108550 0x7f731c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7321400700 1 -- 172.21.15.63:0/686520474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f731c108550 msgr2=0x7f731c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7321400700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f731c108550 0x7f731c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7321400700 1 -- 172.21.15.63:0/686520474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731c107c00 msgr2=0x7f731c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7321400700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731c107c00 0x7f731c1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7321400700 1 -- 172.21.15.63:0/686520474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c19b0c0 con 0x7f731c1032d0 2024-05-27T07:40:42.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.577+0000 7f7321e00700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f731c108550 0x7f731c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:42.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.578+0000 7f7321400700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731c1032d0 0x7f731c1a07f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7310005910 tx=0x7f7310004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:42.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.578+0000 7f731a000700 1 -- 172.21.15.63:0/686520474 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310026070 con 0x7f731c1032d0 2024-05-27T07:40:42.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.578+0000 7f731a000700 1 -- 172.21.15.63:0/686520474 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7310003c50 con 0x7f731c1032d0 2024-05-27T07:40:42.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.578+0000 7f7322800700 1 -- 172.21.15.63:0/686520474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731c19b350 con 0x7f731c1032d0 2024-05-27T07:40:42.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.578+0000 7f731a000700 1 -- 172.21.15.63:0/686520474 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731000fb10 con 0x7f731c1032d0 2024-05-27T07:40:42.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.579+0000 7f7322800700 1 -- 172.21.15.63:0/686520474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731c1a7680 con 0x7f731c1032d0 2024-05-27T07:40:42.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.583+0000 7f731a000700 1 -- 172.21.15.63:0/686520474 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7310029070 con 0x7f731c1032d0 2024-05-27T07:40:42.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.584+0000 7f733287cb80 1 Processor -- start 2024-05-27T07:40:42.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.584+0000 7f733287cb80 1 -- start start 2024-05-27T07:40:42.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f733287cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c261a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f733287cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2624ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f733287cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2627430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f733287cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5644c2544e40 con 0x5644c261ac30 2024-05-27T07:40:42.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f733287cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5644c25349e0 con 0x5644c2497e40 2024-05-27T07:40:42.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f733287cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5644c2538a20 con 0x5644c261a820 2024-05-27T07:40:42.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f731a000700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73040b2a70 0x7f73040b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f7320a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2627430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f7320a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2627430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56852/0 (socket says 172.21.15.63:56852) 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f7320a00700 1 -- 172.21.15.63:0/3386694011 learned_addr learned my addr 172.21.15.63:0/3386694011 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f7321400700 1 --2- 172.21.15.63:0/3386694011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2624ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.585+0000 7f7321e00700 1 --2- 172.21.15.63:0/3386694011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c261a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.586+0000 7f7321e00700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73040b2a70 0x7f73040b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.586+0000 7f731a000700 1 -- 172.21.15.63:0/686520474 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f73100e3420 con 0x7f731c1032d0 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.586+0000 7f7320a00700 1 -- 172.21.15.63:0/3386694011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 msgr2=0x5644c2624ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.586+0000 7f7320a00700 1 --2- 172.21.15.63:0/3386694011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2624ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.586+0000 7f7320a00700 1 -- 172.21.15.63:0/3386694011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 msgr2=0x5644c261a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:42.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.586+0000 7f7320a00700 1 --2- 172.21.15.63:0/3386694011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c261a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.586+0000 7f7320a00700 1 -- 172.21.15.63:0/3386694011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c24fd420 con 0x5644c261ac30 2024-05-27T07:40:42.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.587+0000 7f7320a00700 1 --2- 172.21.15.63:0/3386694011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2627430 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7308009c40 tx=0x7f7308009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4704e25d874096f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:42.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.587+0000 7f7321e00700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73040b2a70 0x7f73040b4f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f73140060b0 tx=0x7f731400b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:42.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.587+0000 7f730ea00700 1 -- 172.21.15.63:0/3386694011 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308011bf0 con 0x5644c261ac30 2024-05-27T07:40:42.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.587+0000 7f730ea00700 1 -- 172.21.15.63:0/3386694011 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7308011d90 con 0x5644c261ac30 2024-05-27T07:40:42.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.587+0000 7f730ea00700 1 -- 172.21.15.63:0/3386694011 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730800f6a0 con 0x5644c261ac30 2024-05-27T07:40:42.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.588+0000 7f733287cb80 1 -- 172.21.15.63:0/3386694011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 msgr2=0x5644c2627430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.588+0000 7f733287cb80 1 --2- 172.21.15.63:0/3386694011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2627430 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7308009c40 tx=0x7f7308009f50 comp rx=0 tx=0).stop 2024-05-27T07:40:42.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.588+0000 7f733287cb80 1 -- 172.21.15.63:0/3386694011 shutdown_connections 2024-05-27T07:40:42.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.588+0000 7f733287cb80 1 --2- 172.21.15.63:0/3386694011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2627430 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.588+0000 7f733287cb80 1 --2- 172.21.15.63:0/3386694011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c261a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.588+0000 7f733287cb80 1 --2- 172.21.15.63:0/3386694011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2624ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.588+0000 7f733287cb80 1 -- 172.21.15.63:0/3386694011 >> 172.21.15.63:0/3386694011 conn(0x5644c24f7fc0 msgr2=0x5644c2611d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:42.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.589+0000 7f733287cb80 1 -- 172.21.15.63:0/3386694011 shutdown_connections 2024-05-27T07:40:42.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.589+0000 7f733287cb80 1 -- 172.21.15.63:0/3386694011 wait complete. 2024-05-27T07:40:42.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.589+0000 7f733287cb80 1 -- start start 2024-05-27T07:40:42.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.590+0000 7f733287cb80 1 Processor -- start 2024-05-27T07:40:42.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.590+0000 7f733287cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c26249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.590+0000 7f733287cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2624ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.590+0000 7f733287cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2625430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f733287cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5644c243afa0 con 0x5644c261ac30 2024-05-27T07:40:42.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f733287cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5644c24d5760 con 0x5644c2497e40 2024-05-27T07:40:42.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f733287cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5644c2627da0 con 0x5644c261a820 2024-05-27T07:40:42.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f733287cb80 10 client.0 _ll_register_callbacks cb 0x5644c24dec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:42.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f7321e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2625430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f7321e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2625430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56862/0 (socket says 172.21.15.63:56862) 2024-05-27T07:40:42.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f7321e00700 1 -- 172.21.15.63:0/719886165 learned_addr learned my addr 172.21.15.63:0/719886165 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:42.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f7321400700 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c26249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f7320a00700 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2624ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f7321e00700 1 -- 172.21.15.63:0/719886165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 msgr2=0x5644c2624ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:42.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f7321e00700 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2624ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f7321e00700 1 -- 172.21.15.63:0/719886165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 msgr2=0x5644c26249b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:42.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f7321e00700 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c26249b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.591+0000 7f7321e00700 1 -- 172.21.15.63:0/719886165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c24848d0 con 0x5644c261ac30 2024-05-27T07:40:42.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.592+0000 7f7321e00700 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2625430 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f73140093c0 tx=0x7f7314004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607e914da8f25420 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:42.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.592+0000 7f7303400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.592+0000 7f7303400700 1 -- 172.21.15.63:0/719886165 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314007500 con 0x5644c261ac30 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.592+0000 7f733287cb80 1 -- 172.21.15.63:0/719886165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7308011d90 con 0x5644c261ac30 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.592+0000 7f7303400700 1 -- 172.21.15.63:0/719886165 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f73140076a0 con 0x5644c261ac30 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.593+0000 7f7303400700 1 -- 172.21.15.63:0/719886165 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f73140078a0 con 0x5644c261ac30 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.593+0000 7f733287cb80 10 client.16782 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.593+0000 7f733287cb80 1 -- 172.21.15.63:0/719886165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5644c2505060 con 0x5644c261ac30 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.593+0000 7f7303400700 1 -- 172.21.15.63:0/719886165 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7308011d90 con 0x5644c261ac30 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.593+0000 7f7303400700 1 -- 172.21.15.63:0/719886165 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f731401c9c0 con 0x5644c261ac30 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.594+0000 7f733287cb80 10 client.16782 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.594+0000 7f733287cb80 10 client.16782 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:40:42.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.594+0000 7f733287cb80 20 client.16782 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:42.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.594+0000 7f733287cb80 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5644c2634d00 0x5644c26371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.594+0000 7f733287cb80 4 client.16782 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:40:42.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.594+0000 7f733287cb80 1 -- 172.21.15.63:0/719886165 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644c2546d90 con 0x5644c2634d00 2024-05-27T07:40:42.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.594+0000 7f7321400700 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5644c2634d00 0x5644c26371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.595+0000 7f7303400700 10 client.16782 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:40:42.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.595+0000 7f7321400700 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5644c2634d00 0x5644c26371a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f36557af37460e0 server_cookie=35b9609de4078214 in_seq=0 out_seq=0 2024-05-27T07:40:42.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.597+0000 7f7303400700 1 -- 172.21.15.63:0/719886165 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5644c2546d90 con 0x5644c2634d00 2024-05-27T07:40:42.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.597+0000 7f7303400700 10 client.16782 handle_command_reply: tid=0 2024-05-27T07:40:42.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.597+0000 7f733287cb80 1 client.16782 shutdown 2024-05-27T07:40:42.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 1 -- 172.21.15.63:0/719886165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 msgr2=0x5644c2625430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2625430 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f73140093c0 tx=0x7f7314004ad0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 1 -- 172.21.15.63:0/719886165 shutdown_connections 2024-05-27T07:40:42.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5644c2634d00 0x5644c26371a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c2625430 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c26249b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 1 --2- 172.21.15.63:0/719886165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2624ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 1 -- 172.21.15.63:0/719886165 >> 172.21.15.63:0/719886165 conn(0x5644c24f7fc0 msgr2=0x5644c2610ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:42.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f7303400700 0 client.16782 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:40:42.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 1 -- 172.21.15.63:0/719886165 shutdown_connections 2024-05-27T07:40:42.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 1 -- 172.21.15.63:0/719886165 wait complete. 2024-05-27T07:40:42.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.598+0000 7f733287cb80 20 client.16782 trim_cache size 0 max 16384 2024-05-27T07:40:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.620+0000 7f733287cb80 1 Processor -- start 2024-05-27T07:40:42.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.621+0000 7f733287cb80 1 -- start start 2024-05-27T07:40:42.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.621+0000 7f733287cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c2620240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.621+0000 7f733287cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2626e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.621+0000 7f733287cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c26239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.621+0000 7f733287cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1b0980 con 0x5644c261ac30 2024-05-27T07:40:42.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.621+0000 7f733287cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1ae210 con 0x5644c2497e40 2024-05-27T07:40:42.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.621+0000 7f733287cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1ac0a0 con 0x5644c261a820 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7321e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c26239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7321e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c26239b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56866/0 (socket says 172.21.15.63:56866) 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7321e00700 1 -- 172.21.15.63:0/246831605 learned_addr learned my addr 172.21.15.63:0/246831605 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7321400700 1 --2- 172.21.15.63:0/246831605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c2620240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7320a00700 1 --2- 172.21.15.63:0/246831605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2626e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7321e00700 1 -- 172.21.15.63:0/246831605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 msgr2=0x5644c2626e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7321e00700 1 --2- 172.21.15.63:0/246831605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2626e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7321e00700 1 -- 172.21.15.63:0/246831605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 msgr2=0x5644c2620240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7321e00700 1 --2- 172.21.15.63:0/246831605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c2620240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7321e00700 1 -- 172.21.15.63:0/246831605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644c260c1d0 con 0x5644c261ac30 2024-05-27T07:40:42.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.622+0000 7f7321e00700 1 --2- 172.21.15.63:0/246831605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c26239b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7314007690 tx=0x7f73140076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec1b0f5e18d6716 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:42.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.623+0000 7f730e000700 1 -- 172.21.15.63:0/246831605 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314039070 con 0x5644c261ac30 2024-05-27T07:40:42.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.623+0000 7f730e000700 1 -- 172.21.15.63:0/246831605 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7314011e20 con 0x5644c261ac30 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.623+0000 7f730e000700 1 -- 172.21.15.63:0/246831605 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731401c6b0 con 0x5644c261ac30 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.624+0000 7f733287cb80 1 -- 172.21.15.63:0/246831605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 msgr2=0x5644c26239b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.624+0000 7f733287cb80 1 --2- 172.21.15.63:0/246831605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c26239b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7314007690 tx=0x7f73140076c0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.624+0000 7f733287cb80 1 -- 172.21.15.63:0/246831605 shutdown_connections 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.624+0000 7f733287cb80 1 --2- 172.21.15.63:0/246831605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c26239b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.624+0000 7f733287cb80 1 --2- 172.21.15.63:0/246831605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c2620240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.624+0000 7f733287cb80 1 --2- 172.21.15.63:0/246831605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c2626e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.624+0000 7f733287cb80 1 -- 172.21.15.63:0/246831605 >> 172.21.15.63:0/246831605 conn(0x5644c24f7fc0 msgr2=0x5644c26232a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.624+0000 7f733287cb80 1 -- 172.21.15.63:0/246831605 shutdown_connections 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.625+0000 7f733287cb80 1 -- 172.21.15.63:0/246831605 wait complete. 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.625+0000 7f733287cb80 1 -- start start 2024-05-27T07:40:42.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.625+0000 7f733287cb80 1 Processor -- start 2024-05-27T07:40:42.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.626+0000 7f733287cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c26280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.626+0000 7f733287cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c26285f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.626+0000 7f733287cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c269d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.626+0000 7f733287cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1ac6f0 con 0x5644c261ac30 2024-05-27T07:40:42.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.626+0000 7f733287cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1adb40 con 0x5644c2497e40 2024-05-27T07:40:42.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.626+0000 7f733287cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f731c1acda0 con 0x5644c261a820 2024-05-27T07:40:42.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.626+0000 7f733287cb80 10 client.0 _ll_register_callbacks cb 0x5644c24dec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:42.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.626+0000 7f7321e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c269d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.627+0000 7f7321e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c269d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56876/0 (socket says 172.21.15.63:56876) 2024-05-27T07:40:42.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.627+0000 7f7321e00700 1 -- 172.21.15.63:0/3486427417 learned_addr learned my addr 172.21.15.63:0/3486427417 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:42.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.627+0000 7f7320a00700 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c26285f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.627+0000 7f7321400700 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c26280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.627+0000 7f7321e00700 1 -- 172.21.15.63:0/3486427417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 msgr2=0x5644c26285f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.627+0000 7f7321e00700 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c26285f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.627+0000 7f7321e00700 1 -- 172.21.15.63:0/3486427417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 msgr2=0x5644c26280b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.627+0000 7f7321e00700 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c26280b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.627+0000 7f7321e00700 1 -- 172.21.15.63:0/3486427417 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644c26257a0 con 0x5644c261ac30 2024-05-27T07:40:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.628+0000 7f7321e00700 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c269d820 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7314007d70 tx=0x7f731401ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8721bdb3d3d6068a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:42.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.628+0000 7f7303400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.628+0000 7f7303400700 1 -- 172.21.15.63:0/3486427417 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314039030 con 0x5644c261ac30 2024-05-27T07:40:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.628+0000 7f7303400700 1 -- 172.21.15.63:0/3486427417 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f7314014040 con 0x5644c261ac30 2024-05-27T07:40:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.628+0000 7f733287cb80 1 -- 172.21.15.63:0/3486427417 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7314011e20 con 0x5644c261ac30 2024-05-27T07:40:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.629+0000 7f7303400700 1 -- 172.21.15.63:0/3486427417 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7314004c80 con 0x5644c261ac30 2024-05-27T07:40:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.629+0000 7f733287cb80 10 client.16788 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:42.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.629+0000 7f733287cb80 1 -- 172.21.15.63:0/3486427417 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5644c2613350 con 0x5644c261ac30 2024-05-27T07:40:42.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.629+0000 7f7303400700 1 -- 172.21.15.63:0/3486427417 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7314011e20 con 0x5644c261ac30 2024-05-27T07:40:42.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.630+0000 7f7303400700 1 -- 172.21.15.63:0/3486427417 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f73140345a0 con 0x5644c261ac30 2024-05-27T07:40:42.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.630+0000 7f733287cb80 10 client.16788 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:42.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.630+0000 7f733287cb80 10 client.16788 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:40:42.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.630+0000 7f733287cb80 20 client.16788 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:42.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.630+0000 7f733287cb80 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5644c269eed0 0x5644c26a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.630+0000 7f733287cb80 4 client.16788 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:40:42.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.630+0000 7f733287cb80 1 -- 172.21.15.63:0/3486427417 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644c2540570 con 0x5644c269eed0 2024-05-27T07:40:42.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.630+0000 7f7321400700 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5644c269eed0 0x5644c26a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.631+0000 7f7321400700 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5644c269eed0 0x5644c26a12f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53f4b526fd70aefd server_cookie=714edacf9ac52af3 in_seq=0 out_seq=0 2024-05-27T07:40:42.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.631+0000 7f7303400700 10 client.16788 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:40:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f7303400700 1 -- 172.21.15.63:0/3486427417 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644c2540570 con 0x5644c269eed0 2024-05-27T07:40:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f7303400700 10 client.16788 handle_command_reply: tid=0 2024-05-27T07:40:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f733287cb80 1 client.16788 shutdown 2024-05-27T07:40:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f733287cb80 1 -- 172.21.15.63:0/3486427417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 msgr2=0x5644c269d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f733287cb80 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c269d820 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7314007d70 tx=0x7f731401ce20 comp rx=0 tx=0).stop 2024-05-27T07:40:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f733287cb80 1 -- 172.21.15.63:0/3486427417 shutdown_connections 2024-05-27T07:40:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f733287cb80 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5644c269eed0 0x5644c26a12f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f733287cb80 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5644c261ac30 0x5644c269d820 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f733287cb80 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644c2497e40 0x5644c26280b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f733287cb80 1 --2- 172.21.15.63:0/3486427417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5644c261a820 0x5644c26285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f733287cb80 1 -- 172.21.15.63:0/3486427417 >> 172.21.15.63:0/3486427417 conn(0x5644c24f7fc0 msgr2=0x5644c2624790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:42.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f7303400700 0 client.16788 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:40:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.632+0000 7f733287cb80 1 -- 172.21.15.63:0/3486427417 shutdown_connections 2024-05-27T07:40:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.633+0000 7f733287cb80 1 -- 172.21.15.63:0/3486427417 wait complete. 2024-05-27T07:40:42.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.633+0000 7f733287cb80 20 client.16788 trim_cache size 0 max 16384 2024-05-27T07:40:42.632 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 -- 172.21.15.63:0/686520474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73040b2a70 msgr2=0x7f73040b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73040b2a70 0x7f73040b4f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f73140060b0 tx=0x7f731400b040 comp rx=0 tx=0).stop 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 -- 172.21.15.63:0/686520474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731c1032d0 msgr2=0x7f731c1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731c1032d0 0x7f731c1a07f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7310005910 tx=0x7f7310004320 comp rx=0 tx=0).stop 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 -- 172.21.15.63:0/686520474 shutdown_connections 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73040b2a70 0x7f73040b4f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f731c108550 0x7f731c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731c1032d0 0x7f731c1a07f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 --2- 172.21.15.63:0/686520474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731c107c00 0x7f731c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 -- 172.21.15.63:0/686520474 >> 172.21.15.63:0/686520474 conn(0x7f731c078ab0 msgr2=0x7f731c06ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:42.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 -- 172.21.15.63:0/686520474 shutdown_connections 2024-05-27T07:40:42.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:42.634+0000 7f7322800700 1 -- 172.21.15.63:0/686520474 wait complete. 2024-05-27T07:40:43.602 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:40:43.684 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:43 smithi063 ceph-mon[31613]: pgmap v219: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-05-27T07:40:43.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:43 smithi086 ceph-mon[34687]: pgmap v219: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-05-27T07:40:43.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:43 smithi149 ceph-mon[35408]: pgmap v219: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-05-27T07:40:44.107 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:45.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:45 smithi063 ceph-mon[31613]: pgmap v220: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:40:45.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:45 smithi086 ceph-mon[34687]: pgmap v220: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:40:45.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:45 smithi149 ceph-mon[35408]: pgmap v220: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:40:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.607+0000 7f46f0800700 1 -- 172.21.15.63:0/4193255689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e81034b0 msgr2=0x7f46e81038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.607+0000 7f46f0800700 1 --2- 172.21.15.63:0/4193255689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e81034b0 0x7f46e81038c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f46e000b0d0 tx=0x7f46e000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.607+0000 7f46f0800700 1 -- 172.21.15.63:0/4193255689 shutdown_connections 2024-05-27T07:40:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.607+0000 7f46f0800700 1 --2- 172.21.15.63:0/4193255689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8108730 0x7f46e8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.607+0000 7f46f0800700 1 --2- 172.21.15.63:0/4193255689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e81034b0 0x7f46e81038c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.607+0000 7f46f0800700 1 --2- 172.21.15.63:0/4193255689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e8107de0 0x7f46e81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.607+0000 7f46f0800700 1 -- 172.21.15.63:0/4193255689 >> 172.21.15.63:0/4193255689 conn(0x7f46e80fedb0 msgr2=0x7f46e81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.607+0000 7f46f0800700 1 -- 172.21.15.63:0/4193255689 shutdown_connections 2024-05-27T07:40:46.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.607+0000 7f46f0800700 1 -- 172.21.15.63:0/4193255689 wait complete. 2024-05-27T07:40:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.609+0000 7f46f0800700 1 Processor -- start 2024-05-27T07:40:46.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.609+0000 7f46f0800700 1 -- start start 2024-05-27T07:40:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.609+0000 7f46f0800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e81034b0 0x7f46e819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.609+0000 7f46f0800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8107de0 0x7f46e819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.609+0000 7f46f0800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e8108730 0x7f46e81a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.609+0000 7f46f0800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8113200 con 0x7f46e8108730 2024-05-27T07:40:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.609+0000 7f46f0800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8113080 con 0x7f46e81034b0 2024-05-27T07:40:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.609+0000 7f46f0800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8113380 con 0x7f46e8107de0 2024-05-27T07:40:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.610+0000 7f46ee800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e8108730 0x7f46e81a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.610+0000 7f46ee800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e8108730 0x7f46e81a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56894/0 (socket says 172.21.15.63:56894) 2024-05-27T07:40:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.610+0000 7f46ee800700 1 -- 172.21.15.63:0/1859156445 learned_addr learned my addr 172.21.15.63:0/1859156445 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:46.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.610+0000 7f46ede00700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e81034b0 0x7f46e819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.610+0000 7f46ed400700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8107de0 0x7f46e819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.610+0000 7f46ee800700 1 -- 172.21.15.63:0/1859156445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8107de0 msgr2=0x7f46e819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.610+0000 7f46ee800700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8107de0 0x7f46e819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.610+0000 7f46ee800700 1 -- 172.21.15.63:0/1859156445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e81034b0 msgr2=0x7f46e819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.610+0000 7f46ee800700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e81034b0 0x7f46e819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.611+0000 7f46ee800700 1 -- 172.21.15.63:0/1859156445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e81a1ec0 con 0x7f46e8108730 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.611+0000 7f46ed400700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8107de0 0x7f46e819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.611+0000 7f46ee800700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e8108730 0x7f46e81a1820 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f46e400d980 tx=0x7f46e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.611+0000 7f46dea00700 1 -- 172.21.15.63:0/1859156445 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4017070 con 0x7f46e8108730 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.611+0000 7f46f0800700 1 -- 172.21.15.63:0/1859156445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e81a21b0 con 0x7f46e8108730 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.611+0000 7f46dea00700 1 -- 172.21.15.63:0/1859156445 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e4005020 con 0x7f46e8108730 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.611+0000 7f46dea00700 1 -- 172.21.15.63:0/1859156445 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4009990 con 0x7f46e8108730 2024-05-27T07:40:46.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.612+0000 7f46f0800700 1 -- 172.21.15.63:0/1859156445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e81a9ad0 con 0x7f46e8108730 2024-05-27T07:40:46.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.616+0000 7f46dea00700 1 -- 172.21.15.63:0/1859156445 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f46e4009b30 con 0x7f46e8108730 2024-05-27T07:40:46.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.617+0000 7f46ff379b80 1 Processor -- start 2024-05-27T07:40:46.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.617+0000 7f46ff379b80 1 -- start start 2024-05-27T07:40:46.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.618+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb3751d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.618+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37517870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.618+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb375181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.618+0000 7f46ff379b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eb37445e40 con 0x55eb3751dd30 2024-05-27T07:40:46.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.618+0000 7f46ff379b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eb374359e0 con 0x55eb37398e40 2024-05-27T07:40:46.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.618+0000 7f46ff379b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55eb37439a20 con 0x55eb37517db0 2024-05-27T07:40:46.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.618+0000 7f46ede00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37517870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.618+0000 7f46ede00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37517870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56906/0 (socket says 172.21.15.63:56906) 2024-05-27T07:40:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.618+0000 7f46ede00700 1 -- 172.21.15.63:0/828388487 learned_addr learned my addr 172.21.15.63:0/828388487 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46ed400700 1 --2- 172.21.15.63:0/828388487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb3751d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46ede00700 1 -- 172.21.15.63:0/828388487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 msgr2=0x55eb375181c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46ede00700 1 --2- 172.21.15.63:0/828388487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb375181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46ede00700 1 -- 172.21.15.63:0/828388487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 msgr2=0x55eb3751d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46ede00700 1 --2- 172.21.15.63:0/828388487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb3751d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46ede00700 1 -- 172.21.15.63:0/828388487 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb373fe420 con 0x55eb3751dd30 2024-05-27T07:40:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46ed400700 1 --2- 172.21.15.63:0/828388487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb3751d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:46.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46dea00700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46d40b2a70 0x7f46d40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46dea00700 1 -- 172.21.15.63:0/1859156445 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f46e40e2680 con 0x7f46e8108730 2024-05-27T07:40:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46ede00700 1 --2- 172.21.15.63:0/828388487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37517870 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009cb0 tx=0x7f46e0009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c2a28ea175471c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.619+0000 7f46ed400700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46d40b2a70 0x7f46d40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.620+0000 7f46d3400700 1 -- 172.21.15.63:0/828388487 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e00091d0 con 0x55eb3751dd30 2024-05-27T07:40:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.620+0000 7f46d3400700 1 -- 172.21.15.63:0/828388487 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e0020070 con 0x55eb3751dd30 2024-05-27T07:40:46.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.620+0000 7f46d3400700 1 -- 172.21.15.63:0/828388487 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e00075c0 con 0x55eb3751dd30 2024-05-27T07:40:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.621+0000 7f46ff379b80 1 -- 172.21.15.63:0/828388487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 msgr2=0x55eb37517870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.621+0000 7f46ed400700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46d40b2a70 0x7f46d40b4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55eb37518b30 tx=0x7f46d800b400 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.621+0000 7f46ff379b80 1 --2- 172.21.15.63:0/828388487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37517870 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009cb0 tx=0x7f46e0009ce0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.621+0000 7f46ff379b80 1 -- 172.21.15.63:0/828388487 shutdown_connections 2024-05-27T07:40:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.621+0000 7f46ff379b80 1 --2- 172.21.15.63:0/828388487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb375181c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.621+0000 7f46ff379b80 1 --2- 172.21.15.63:0/828388487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb3751d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.621+0000 7f46ff379b80 1 --2- 172.21.15.63:0/828388487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37517870 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.621+0000 7f46ff379b80 1 -- 172.21.15.63:0/828388487 >> 172.21.15.63:0/828388487 conn(0x55eb373f8fc0 msgr2=0x55eb375128d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:46.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.622+0000 7f46ff379b80 1 -- 172.21.15.63:0/828388487 shutdown_connections 2024-05-27T07:40:46.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.622+0000 7f46ff379b80 1 -- 172.21.15.63:0/828388487 wait complete. 2024-05-27T07:40:46.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.623+0000 7f46ff379b80 1 -- start start 2024-05-27T07:40:46.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.623+0000 7f46ff379b80 1 Processor -- start 2024-05-27T07:40:46.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.623+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb37521510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.623+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37517db0 0x55eb37521a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.623+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3751dd30 0x55eb37521f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.623+0000 7f46ff379b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eb3733bfa0 con 0x55eb37517db0 2024-05-27T07:40:46.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.624+0000 7f46ff379b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eb373d6760 con 0x55eb37398e40 2024-05-27T07:40:46.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.624+0000 7f46ff379b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55eb375247a0 con 0x55eb3751dd30 2024-05-27T07:40:46.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.624+0000 7f46ff379b80 10 client.0 _ll_register_callbacks cb 0x55eb373dfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:46.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.624+0000 7f46ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37517db0 0x55eb37521a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.624+0000 7f46ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37517db0 0x55eb37521a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56912/0 (socket says 172.21.15.63:56912) 2024-05-27T07:40:46.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.624+0000 7f46ed400700 1 -- 172.21.15.63:0/2289728404 learned_addr learned my addr 172.21.15.63:0/2289728404 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:46.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.624+0000 7f46ede00700 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb37521510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.625+0000 7f46ee800700 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3751dd30 0x55eb37521f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.625+0000 7f46ede00700 1 -- 172.21.15.63:0/2289728404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3751dd30 msgr2=0x55eb37521f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.625+0000 7f46ede00700 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3751dd30 0x55eb37521f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.625+0000 7f46ede00700 1 -- 172.21.15.63:0/2289728404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37517db0 msgr2=0x55eb37521a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.625+0000 7f46ede00700 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37517db0 0x55eb37521a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.625+0000 7f46ede00700 1 -- 172.21.15.63:0/2289728404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb373858d0 con 0x55eb37398e40 2024-05-27T07:40:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.625+0000 7f46ed400700 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37517db0 0x55eb37521a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:40:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.626+0000 7f46ede00700 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb37521510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46e0000f80 tx=0x7f46e00099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33eaf20fbbc8e12e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.626+0000 7f46c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:40:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.626+0000 7f46c7e00700 1 -- 172.21.15.63:0/2289728404 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0003a00 con 0x55eb37398e40 2024-05-27T07:40:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.626+0000 7f46ff379b80 1 -- 172.21.15.63:0/2289728404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e0020070 con 0x55eb37398e40 2024-05-27T07:40:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.626+0000 7f46c7e00700 1 -- 172.21.15.63:0/2289728404 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f46e000f040 con 0x55eb37398e40 2024-05-27T07:40:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.627+0000 7f46c7e00700 1 -- 172.21.15.63:0/2289728404 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f46e0032050 con 0x55eb37398e40 2024-05-27T07:40:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.627+0000 7f46ff379b80 10 client.36404 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.627+0000 7f46ff379b80 1 -- 172.21.15.63:0/2289728404 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eb37406060 con 0x55eb37398e40 2024-05-27T07:40:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.628+0000 7f46c7e00700 1 -- 172.21.15.63:0/2289728404 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e0020070 con 0x55eb37398e40 2024-05-27T07:40:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.628+0000 7f46c7e00700 1 -- 172.21.15.63:0/2289728404 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f46e0011070 con 0x55eb37398e40 2024-05-27T07:40:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.628+0000 7f46ff379b80 10 client.36404 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.628+0000 7f46ff379b80 10 client.36404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.628+0000 7f46ff379b80 20 client.36404 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.628+0000 7f46ff379b80 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb375314b0 0x55eb375338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.628+0000 7f46ff379b80 4 client.36404 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:40:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.628+0000 7f46ff379b80 1 -- 172.21.15.63:0/2289728404 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb37447d90 con 0x55eb375314b0 2024-05-27T07:40:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.629+0000 7f46ed400700 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb375314b0 0x55eb375338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.630+0000 7f46ed400700 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb375314b0 0x55eb375338d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bdb2de9d5dbba85 server_cookie=a19f161ea050b34a in_seq=0 out_seq=0 2024-05-27T07:40:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.630+0000 7f46c7e00700 10 client.36404 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:46.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46c7e00700 1 -- 172.21.15.63:0/2289728404 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eb37447d90 con 0x55eb375314b0 2024-05-27T07:40:46.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46c7e00700 10 client.36404 handle_command_reply: tid=0 2024-05-27T07:40:46.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46ff379b80 1 client.36404 shutdown 2024-05-27T07:40:46.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46ff379b80 1 -- 172.21.15.63:0/2289728404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 msgr2=0x55eb37521510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:46.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46ff379b80 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb37521510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46e0000f80 tx=0x7f46e00099e0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46ff379b80 1 -- 172.21.15.63:0/2289728404 shutdown_connections 2024-05-27T07:40:46.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46ff379b80 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb375314b0 0x55eb375338d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46ff379b80 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3751dd30 0x55eb37521f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46ff379b80 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb37521510 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46ff379b80 1 --2- 172.21.15.63:0/2289728404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37517db0 0x55eb37521a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46ff379b80 1 -- 172.21.15.63:0/2289728404 >> 172.21.15.63:0/2289728404 conn(0x55eb373f8fc0 msgr2=0x55eb37512580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.632+0000 7f46c7e00700 0 client.36404 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.633+0000 7f46ff379b80 1 -- 172.21.15.63:0/2289728404 shutdown_connections 2024-05-27T07:40:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.633+0000 7f46ff379b80 1 -- 172.21.15.63:0/2289728404 wait complete. 2024-05-27T07:40:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.633+0000 7f46ff379b80 20 client.36404 trim_cache size 0 max 16384 2024-05-27T07:40:46.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.656+0000 7f46ff379b80 1 Processor -- start 2024-05-27T07:40:46.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.656+0000 7f46ff379b80 1 -- start start 2024-05-27T07:40:46.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.656+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37398e40 0x55eb37517870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.656+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb3751bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.656+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3751dd30 0x55eb375239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.656+0000 7f46ff379b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81b4f60 con 0x55eb37398e40 2024-05-27T07:40:46.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.656+0000 7f46ff379b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81b27f0 con 0x55eb3751dd30 2024-05-27T07:40:46.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.656+0000 7f46ff379b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81b0660 con 0x55eb37517db0 2024-05-27T07:40:46.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ede00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37398e40 0x55eb37517870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ede00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37398e40 0x55eb37517870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56928/0 (socket says 172.21.15.63:56928) 2024-05-27T07:40:46.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ede00700 1 -- 172.21.15.63:0/1905861657 learned_addr learned my addr 172.21.15.63:0/1905861657 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:46.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ed400700 1 --2- 172.21.15.63:0/1905861657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb3751bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ee800700 1 --2- 172.21.15.63:0/1905861657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3751dd30 0x55eb375239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ede00700 1 -- 172.21.15.63:0/1905861657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 msgr2=0x55eb3751bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:46.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ede00700 1 --2- 172.21.15.63:0/1905861657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb3751bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ede00700 1 -- 172.21.15.63:0/1905861657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3751dd30 msgr2=0x55eb375239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:46.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ede00700 1 --2- 172.21.15.63:0/1905861657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3751dd30 0x55eb375239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ede00700 1 -- 172.21.15.63:0/1905861657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3750c130 con 0x55eb37398e40 2024-05-27T07:40:46.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.657+0000 7f46ee800700 1 --2- 172.21.15.63:0/1905861657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3751dd30 0x55eb375239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:46.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.658+0000 7f46ede00700 1 --2- 172.21.15.63:0/1905861657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37398e40 0x55eb37517870 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f46e0005bd0 tx=0x7f46e0009010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3573c3fab0c0d1af server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:46.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.658+0000 7f46d2a00700 1 -- 172.21.15.63:0/1905861657 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0005460 con 0x55eb37398e40 2024-05-27T07:40:46.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.658+0000 7f46d2a00700 1 -- 172.21.15.63:0/1905861657 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e0020040 con 0x55eb37398e40 2024-05-27T07:40:46.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.658+0000 7f46d2a00700 1 -- 172.21.15.63:0/1905861657 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e002d810 con 0x55eb37398e40 2024-05-27T07:40:46.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.660+0000 7f46ff379b80 1 -- 172.21.15.63:0/1905861657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37398e40 msgr2=0x55eb37517870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:46.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.660+0000 7f46ff379b80 1 --2- 172.21.15.63:0/1905861657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37398e40 0x55eb37517870 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f46e0005bd0 tx=0x7f46e0009010 comp rx=0 tx=0).stop 2024-05-27T07:40:46.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.660+0000 7f46ff379b80 1 -- 172.21.15.63:0/1905861657 shutdown_connections 2024-05-27T07:40:46.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.660+0000 7f46ff379b80 1 --2- 172.21.15.63:0/1905861657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3751dd30 0x55eb375239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.660+0000 7f46ff379b80 1 --2- 172.21.15.63:0/1905861657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb37398e40 0x55eb37517870 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.660+0000 7f46ff379b80 1 --2- 172.21.15.63:0/1905861657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb3751bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.660+0000 7f46ff379b80 1 -- 172.21.15.63:0/1905861657 >> 172.21.15.63:0/1905861657 conn(0x55eb373f8fc0 msgr2=0x55eb3751fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:46.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.660+0000 7f46ff379b80 1 -- 172.21.15.63:0/1905861657 shutdown_connections 2024-05-27T07:40:46.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.660+0000 7f46ff379b80 1 -- 172.21.15.63:0/1905861657 wait complete. 2024-05-27T07:40:46.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.661+0000 7f46ff379b80 1 -- start start 2024-05-27T07:40:46.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.661+0000 7f46ff379b80 1 Processor -- start 2024-05-27T07:40:46.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb375368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb375334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ff379b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37533a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ff379b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81b0cb0 con 0x55eb3751dd30 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ff379b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81b2120 con 0x55eb37398e40 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ff379b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46e81b1380 con 0x55eb37517db0 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ff379b80 10 client.0 _ll_register_callbacks cb 0x55eb373dfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ee800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37533a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ee800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37533a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56934/0 (socket says 172.21.15.63:56934) 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ee800700 1 -- 172.21.15.63:0/134357471 learned_addr learned my addr 172.21.15.63:0/134357471 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.662+0000 7f46ede00700 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb375368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.663+0000 7f46ed400700 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb375334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.663+0000 7f46ee800700 1 -- 172.21.15.63:0/134357471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 msgr2=0x55eb375334c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:46.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.663+0000 7f46ee800700 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb375334c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.663+0000 7f46ee800700 1 -- 172.21.15.63:0/134357471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 msgr2=0x55eb375368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.663+0000 7f46ee800700 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb375368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.663+0000 7f46ee800700 1 -- 172.21.15.63:0/134357471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb37522300 con 0x55eb3751dd30 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.663+0000 7f46ee800700 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37533a00 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f46e40dfac0 tx=0x7f46e40a8e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6882706ef5977a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.663+0000 7f46c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.664+0000 7f46c7e00700 1 -- 172.21.15.63:0/134357471 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e40c3060 con 0x55eb3751dd30 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.664+0000 7f46c7e00700 1 -- 172.21.15.63:0/134357471 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f46e40a5b90 con 0x55eb3751dd30 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.664+0000 7f46ff379b80 1 -- 172.21.15.63:0/134357471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e0020040 con 0x55eb3751dd30 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.664+0000 7f46c7e00700 1 -- 172.21.15.63:0/134357471 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f46e40ad950 con 0x55eb3751dd30 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.664+0000 7f46ff379b80 10 client.16806 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:46.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.664+0000 7f46ff379b80 1 -- 172.21.15.63:0/134357471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eb3759a780 con 0x55eb3751dd30 2024-05-27T07:40:46.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.665+0000 7f46c7e00700 1 -- 172.21.15.63:0/134357471 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e0020040 con 0x55eb3751dd30 2024-05-27T07:40:46.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.665+0000 7f46c7e00700 1 -- 172.21.15.63:0/134357471 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f46e40b2070 con 0x55eb3751dd30 2024-05-27T07:40:46.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.665+0000 7f46ff379b80 10 client.16806 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:46.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.665+0000 7f46ff379b80 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:46.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.665+0000 7f46ff379b80 20 client.16806 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:46.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.665+0000 7f46ff379b80 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3759ade0 0x55eb3759d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:46.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.665+0000 7f46ff379b80 4 client.16806 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:40:46.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.665+0000 7f46ff379b80 1 -- 172.21.15.63:0/134357471 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb37441570 con 0x55eb3759ade0 2024-05-27T07:40:46.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.666+0000 7f46ede00700 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3759ade0 0x55eb3759d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46ede00700 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3759ade0 0x55eb3759d200 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7775bcfac7bed456 server_cookie=df81ad83f74a29e8 in_seq=0 out_seq=0 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46c7e00700 10 client.16806 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46c7e00700 1 -- 172.21.15.63:0/134357471 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb37441570 con 0x55eb3759ade0 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46c7e00700 10 client.16806 handle_command_reply: tid=0 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46ff379b80 1 client.16806 shutdown 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46ff379b80 1 -- 172.21.15.63:0/134357471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 msgr2=0x55eb37533a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46ff379b80 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37533a00 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f46e40dfac0 tx=0x7f46e40a8e20 comp rx=0 tx=0).stop 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46ff379b80 1 -- 172.21.15.63:0/134357471 shutdown_connections 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46ff379b80 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3759ade0 0x55eb3759d200 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46ff379b80 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3751dd30 0x55eb37533a00 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46ff379b80 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb37398e40 0x55eb375368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.667+0000 7f46ff379b80 1 --2- 172.21.15.63:0/134357471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb37517db0 0x55eb375334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.668+0000 7f46ff379b80 1 -- 172.21.15.63:0/134357471 >> 172.21.15.63:0/134357471 conn(0x55eb373f8fc0 msgr2=0x55eb3751b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:46.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.668+0000 7f46c7e00700 0 client.16806 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:46.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.668+0000 7f46ff379b80 1 -- 172.21.15.63:0/134357471 shutdown_connections 2024-05-27T07:40:46.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.668+0000 7f46ff379b80 1 -- 172.21.15.63:0/134357471 wait complete. 2024-05-27T07:40:46.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.668+0000 7f46ff379b80 20 client.16806 trim_cache size 0 max 16384 2024-05-27T07:40:46.668 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:40:46.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 -- 172.21.15.63:0/1859156445 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46d40b2a70 msgr2=0x7f46d40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:46.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46d40b2a70 0x7f46d40b4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55eb37518b30 tx=0x7f46d800b400 comp rx=0 tx=0).stop 2024-05-27T07:40:46.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 -- 172.21.15.63:0/1859156445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e8108730 msgr2=0x7f46e81a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e8108730 0x7f46e81a1820 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f46e400d980 tx=0x7f46e400dc90 comp rx=0 tx=0).stop 2024-05-27T07:40:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 -- 172.21.15.63:0/1859156445 shutdown_connections 2024-05-27T07:40:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46d40b2a70 0x7f46d40b4f80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e8108730 0x7f46e81a1820 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46e81034b0 0x7f46e819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 --2- 172.21.15.63:0/1859156445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46e8107de0 0x7f46e819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 -- 172.21.15.63:0/1859156445 >> 172.21.15.63:0/1859156445 conn(0x7f46e80fedb0 msgr2=0x7f46e806b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.669+0000 7f46f0800700 1 -- 172.21.15.63:0/1859156445 shutdown_connections 2024-05-27T07:40:46.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:46.670+0000 7f46f0800700 1 -- 172.21.15.63:0/1859156445 wait complete. 2024-05-27T07:40:47.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:47 smithi086 ceph-mon[34687]: pgmap v221: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:40:47.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:47 smithi149 ceph-mon[35408]: pgmap v221: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:40:47.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:47 smithi063 ceph-mon[31613]: pgmap v221: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:40:48.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:48 smithi086 ceph-mon[34687]: pgmap v222: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-05-27T07:40:48.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:48 smithi149 ceph-mon[35408]: pgmap v222: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-05-27T07:40:48.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:48 smithi063 ceph-mon[31613]: pgmap v222: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-05-27T07:40:50.142 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:40:50.143 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:40:50.674 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:50.832 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:50 smithi063 ceph-mon[31613]: pgmap v223: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:40:50.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:50 smithi086 ceph-mon[34687]: pgmap v223: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:40:50.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:50 smithi149 ceph-mon[35408]: pgmap v223: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:40:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.107+0000 7ff24a000640 1 -- 172.21.15.63:0/1986520907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff24410d9c0 msgr2=0x7ff24410fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.107+0000 7ff24a000640 1 --2- 172.21.15.63:0/1986520907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff24410d9c0 0x7ff24410fde0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff230009900 tx=0x7ff23002a480 comp rx=0 tx=0).stop 2024-05-27T07:40:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.108+0000 7ff24a000640 1 -- 172.21.15.63:0/1986520907 shutdown_connections 2024-05-27T07:40:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.108+0000 7ff24a000640 1 --2- 172.21.15.63:0/1986520907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff244110320 0x7ff244112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.108+0000 7ff24a000640 1 --2- 172.21.15.63:0/1986520907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff24410d9c0 0x7ff24410fde0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.108+0000 7ff24a000640 1 --2- 172.21.15.63:0/1986520907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff244075420 0x7ff244075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.108+0000 7ff24a000640 1 -- 172.21.15.63:0/1986520907 >> 172.21.15.63:0/1986520907 conn(0x7ff244079100 msgr2=0x7ff244079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.108+0000 7ff24a000640 1 -- 172.21.15.63:0/1986520907 shutdown_connections 2024-05-27T07:40:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.108+0000 7ff24a000640 1 -- 172.21.15.63:0/1986520907 wait complete. 2024-05-27T07:40:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.109+0000 7ff24a000640 1 Processor -- start 2024-05-27T07:40:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.109+0000 7ff24a000640 1 -- start start 2024-05-27T07:40:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.110+0000 7ff24a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff244075420 0x7ff2441a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.110+0000 7ff24a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24410d9c0 0x7ff2441a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.110+0000 7ff24a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff244110320 0x7ff24419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.110+0000 7ff24a000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff244115d40 con 0x7ff244075420 2024-05-27T07:40:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.110+0000 7ff24a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff244115bc0 con 0x7ff244110320 2024-05-27T07:40:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.110+0000 7ff24a000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff244115ec0 con 0x7ff24410d9c0 2024-05-27T07:40:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.110+0000 7ff248c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff244075420 0x7ff2441a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.110+0000 7ff248c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff244075420 0x7ff2441a24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33952/0 (socket says 172.21.15.63:33952) 2024-05-27T07:40:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.110+0000 7ff248c00640 1 -- 172.21.15.63:0/3584678695 learned_addr learned my addr 172.21.15.63:0/3584678695 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.111+0000 7ff249600640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff244110320 0x7ff24419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.111+0000 7ff243e00640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24410d9c0 0x7ff2441a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.111+0000 7ff248c00640 1 -- 172.21.15.63:0/3584678695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24410d9c0 msgr2=0x7ff2441a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.111+0000 7ff248c00640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24410d9c0 0x7ff2441a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.111+0000 7ff248c00640 1 -- 172.21.15.63:0/3584678695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff244110320 msgr2=0x7ff24419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.111+0000 7ff248c00640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff244110320 0x7ff24419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.111+0000 7ff248c00640 1 -- 172.21.15.63:0/3584678695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24419cd90 con 0x7ff244075420 2024-05-27T07:40:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.111+0000 7ff248c00640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff244075420 0x7ff2441a24e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff23800e9f0 tx=0x7ff23800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.112+0000 7ff241600640 1 -- 172.21.15.63:0/3584678695 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23800cd80 con 0x7ff244075420 2024-05-27T07:40:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.112+0000 7ff241600640 1 -- 172.21.15.63:0/3584678695 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2380108c0 con 0x7ff244075420 2024-05-27T07:40:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.112+0000 7ff24a000640 1 -- 172.21.15.63:0/3584678695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24419d020 con 0x7ff244075420 2024-05-27T07:40:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.112+0000 7ff24a000640 1 -- 172.21.15.63:0/3584678695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2441a9280 con 0x7ff244075420 2024-05-27T07:40:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.112+0000 7ff241600640 1 -- 172.21.15.63:0/3584678695 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380187c0 con 0x7ff244075420 2024-05-27T07:40:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.115+0000 7ff25265e740 1 Processor -- start 2024-05-27T07:40:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.116+0000 7ff25265e740 1 -- start start 2024-05-27T07:40:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.116+0000 7ff25265e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be64816a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.116+0000 7ff25265e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64816fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.116+0000 7ff25265e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644534d0 0x55be648174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.116+0000 7ff25265e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55be64787b90 con 0x55be6444b2f0 2024-05-27T07:40:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.116+0000 7ff25265e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55be64777730 con 0x55be644534d0 2024-05-27T07:40:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.116+0000 7ff25265e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55be6477b770 con 0x55be644741b0 2024-05-27T07:40:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.116+0000 7ff241600640 1 -- 172.21.15.63:0/3584678695 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff238010430 con 0x7ff244075420 2024-05-27T07:40:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.117+0000 7ff248c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64816fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.117+0000 7ff243e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644534d0 0x55be648174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.117+0000 7ff248c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64816fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33956/0 (socket says 172.21.15.63:33956) 2024-05-27T07:40:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.117+0000 7ff248c00640 1 -- 172.21.15.63:0/3774997340 learned_addr learned my addr 172.21.15.63:0/3774997340 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.117+0000 7ff249600640 1 --2- 172.21.15.63:0/3774997340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be64816a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.117+0000 7ff248c00640 1 -- 172.21.15.63:0/3774997340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 msgr2=0x55be64816a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.117+0000 7ff248c00640 1 --2- 172.21.15.63:0/3774997340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be64816a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.117+0000 7ff248c00640 1 -- 172.21.15.63:0/3774997340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644534d0 msgr2=0x55be648174e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.117+0000 7ff248c00640 1 --2- 172.21.15.63:0/3774997340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644534d0 0x55be648174e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.117+0000 7ff248c00640 1 -- 172.21.15.63:0/3774997340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be647108b0 con 0x55be6444b2f0 2024-05-27T07:40:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.118+0000 7ff248c00640 1 --2- 172.21.15.63:0/3774997340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64816fa0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff2380e0110 tx=0x7ff2380eb570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb47c931beb8c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.118+0000 7ff235c00640 1 -- 172.21.15.63:0/3774997340 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380f6070 con 0x55be6444b2f0 2024-05-27T07:40:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.118+0000 7ff235c00640 1 -- 172.21.15.63:0/3774997340 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2380f1a90 con 0x55be6444b2f0 2024-05-27T07:40:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.118+0000 7ff235c00640 1 -- 172.21.15.63:0/3774997340 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380f09b0 con 0x55be6444b2f0 2024-05-27T07:40:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.118+0000 7ff241600640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff22c0b2950 0x7ff22c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.119+0000 7ff241600640 1 -- 172.21.15.63:0/3584678695 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7ff2380e2ea0 con 0x7ff244075420 2024-05-27T07:40:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.119+0000 7ff249600640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff22c0b2950 0x7ff22c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.119+0000 7ff25265e740 1 -- 172.21.15.63:0/3774997340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 msgr2=0x55be64816fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:52.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.119+0000 7ff25265e740 1 --2- 172.21.15.63:0/3774997340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64816fa0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff2380e0110 tx=0x7ff2380eb570 comp rx=0 tx=0).stop 2024-05-27T07:40:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.120+0000 7ff25265e740 1 -- 172.21.15.63:0/3774997340 shutdown_connections 2024-05-27T07:40:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.120+0000 7ff25265e740 1 --2- 172.21.15.63:0/3774997340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644534d0 0x55be648174e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.120+0000 7ff25265e740 1 --2- 172.21.15.63:0/3774997340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64816fa0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.120+0000 7ff25265e740 1 --2- 172.21.15.63:0/3774997340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be64816a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.120+0000 7ff25265e740 1 -- 172.21.15.63:0/3774997340 >> 172.21.15.63:0/3774997340 conn(0x55be64468300 msgr2=0x55be64813210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.120+0000 7ff249600640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff22c0b2950 0x7ff22c0b4e60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55be64813bd0 tx=0x7ff23c009760 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.120+0000 7ff25265e740 1 -- 172.21.15.63:0/3774997340 shutdown_connections 2024-05-27T07:40:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.120+0000 7ff25265e740 1 -- 172.21.15.63:0/3774997340 wait complete. 2024-05-27T07:40:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.120+0000 7ff25265e740 1 -- start start 2024-05-27T07:40:52.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.121+0000 7ff25265e740 1 Processor -- start 2024-05-27T07:40:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.121+0000 7ff25265e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6444b2f0 0x55be64820c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.121+0000 7ff25265e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64821150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.121+0000 7ff25265e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be6482cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.121+0000 7ff25265e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55be6445d790 con 0x55be644534d0 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.121+0000 7ff25265e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55be6471fbc0 con 0x55be6444b2f0 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.121+0000 7ff25265e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55be64823b90 con 0x55be644741b0 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.121+0000 7ff25265e740 10 client.0 _ll_register_callbacks cb 0x55be646fd400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.122+0000 7ff243e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64821150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.122+0000 7ff243e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64821150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33962/0 (socket says 172.21.15.63:33962) 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.122+0000 7ff243e00640 1 -- 172.21.15.63:0/2858839368 learned_addr learned my addr 172.21.15.63:0/2858839368 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.122+0000 7ff248c00640 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6444b2f0 0x55be64820c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.122+0000 7ff249600640 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be6482cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.122+0000 7ff243e00640 1 -- 172.21.15.63:0/2858839368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 msgr2=0x55be6482cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.122+0000 7ff243e00640 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be6482cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.122+0000 7ff243e00640 1 -- 172.21.15.63:0/2858839368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6444b2f0 msgr2=0x55be64820c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.122+0000 7ff243e00640 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6444b2f0 0x55be64820c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.122+0000 7ff243e00640 1 -- 172.21.15.63:0/2858839368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be6448c210 con 0x55be644534d0 2024-05-27T07:40:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.123+0000 7ff243e00640 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64821150 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff24406c550 tx=0x7ff230031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b5fdcdfb9609 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.123+0000 7ff22a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.123+0000 7ff22a000640 1 -- 172.21.15.63:0/2858839368 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230035040 con 0x55be644534d0 2024-05-27T07:40:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.123+0000 7ff25265e740 1 -- 172.21.15.63:0/2858839368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2380f1a90 con 0x55be644534d0 2024-05-27T07:40:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.123+0000 7ff22a000640 1 -- 172.21.15.63:0/2858839368 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7ff230004040 con 0x55be644534d0 2024-05-27T07:40:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.123+0000 7ff22a000640 1 -- 172.21.15.63:0/2858839368 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff23004e050 con 0x55be644534d0 2024-05-27T07:40:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.123+0000 7ff25265e740 10 client.16818 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.123+0000 7ff25265e740 1 -- 172.21.15.63:0/2858839368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55be64558620 con 0x55be644534d0 2024-05-27T07:40:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.124+0000 7ff22a000640 1 -- 172.21.15.63:0/2858839368 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2380f1a90 con 0x55be644534d0 2024-05-27T07:40:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.124+0000 7ff22a000640 1 -- 172.21.15.63:0/2858839368 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff230004750 con 0x55be644534d0 2024-05-27T07:40:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.124+0000 7ff25265e740 10 client.16818 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.124+0000 7ff25265e740 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.124+0000 7ff25265e740 20 client.16818 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.124+0000 7ff25265e740 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be64432d80 0x55be64832f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.124+0000 7ff25265e740 4 client.16818 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:40:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.124+0000 7ff25265e740 1 -- 172.21.15.63:0/2858839368 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be64789ae0 con 0x55be64432d80 2024-05-27T07:40:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.125+0000 7ff248c00640 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be64432d80 0x55be64832f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.126+0000 7ff248c00640 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be64432d80 0x55be64832f20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc43312d4da4133 server_cookie=2b55207f163d1175 in_seq=0 out_seq=0 2024-05-27T07:40:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.126+0000 7ff22a000640 10 client.16818 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff22a000640 1 -- 172.21.15.63:0/2858839368 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55be64789ae0 con 0x55be64432d80 2024-05-27T07:40:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff22a000640 10 client.16818 handle_command_reply: tid=0 2024-05-27T07:40:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 client.16818 shutdown 2024-05-27T07:40:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 -- 172.21.15.63:0/2858839368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 msgr2=0x55be64821150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64821150 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff24406c550 tx=0x7ff230031630 comp rx=0 tx=0).stop 2024-05-27T07:40:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 -- 172.21.15.63:0/2858839368 shutdown_connections 2024-05-27T07:40:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be64432d80 0x55be64832f20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be6482cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64821150 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 --2- 172.21.15.63:0/2858839368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6444b2f0 0x55be64820c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 -- 172.21.15.63:0/2858839368 >> 172.21.15.63:0/2858839368 conn(0x55be64468300 msgr2=0x55be64816890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff22a000640 0 client.16818 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 -- 172.21.15.63:0/2858839368 shutdown_connections 2024-05-27T07:40:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 1 -- 172.21.15.63:0/2858839368 wait complete. 2024-05-27T07:40:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.128+0000 7ff25265e740 20 client.16818 trim_cache size 0 max 16384 2024-05-27T07:40:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.147+0000 7ff25265e740 1 Processor -- start 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.148+0000 7ff25265e740 1 -- start start 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.148+0000 7ff25265e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6444b2f0 0x55be6481c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.148+0000 7ff25265e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64812240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.148+0000 7ff25265e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be64820bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.148+0000 7ff25265e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff2240127e0 con 0x55be644534d0 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.148+0000 7ff25265e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2240100b0 con 0x55be6444b2f0 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.148+0000 7ff25265e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff22400df20 con 0x55be644741b0 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.148+0000 7ff243e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64812240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.148+0000 7ff243e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64812240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33966/0 (socket says 172.21.15.63:33966) 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.148+0000 7ff243e00640 1 -- 172.21.15.63:0/4223755532 learned_addr learned my addr 172.21.15.63:0/4223755532 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.149+0000 7ff248c00640 1 --2- 172.21.15.63:0/4223755532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6444b2f0 0x55be6481c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.149+0000 7ff249600640 1 --2- 172.21.15.63:0/4223755532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be64820bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.149+0000 7ff243e00640 1 -- 172.21.15.63:0/4223755532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 msgr2=0x55be64820bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.149+0000 7ff243e00640 1 --2- 172.21.15.63:0/4223755532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be64820bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.149+0000 7ff243e00640 1 -- 172.21.15.63:0/4223755532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6444b2f0 msgr2=0x55be6481c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.149+0000 7ff243e00640 1 --2- 172.21.15.63:0/4223755532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6444b2f0 0x55be6481c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.149+0000 7ff243e00640 1 -- 172.21.15.63:0/4223755532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be64813c00 con 0x55be644534d0 2024-05-27T07:40:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.149+0000 7ff249600640 1 --2- 172.21.15.63:0/4223755532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be64820bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.149+0000 7ff243e00640 1 --2- 172.21.15.63:0/4223755532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64812240 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff230009850 tx=0x7ff2300318b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7395b25c038c6166 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.150+0000 7ff235200640 1 -- 172.21.15.63:0/4223755532 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230035070 con 0x55be644534d0 2024-05-27T07:40:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.150+0000 7ff235200640 1 -- 172.21.15.63:0/4223755532 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2300046c0 con 0x55be644534d0 2024-05-27T07:40:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.150+0000 7ff235200640 1 -- 172.21.15.63:0/4223755532 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2300055b0 con 0x55be644534d0 2024-05-27T07:40:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.151+0000 7ff25265e740 1 -- 172.21.15.63:0/4223755532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 msgr2=0x55be64812240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.151+0000 7ff25265e740 1 --2- 172.21.15.63:0/4223755532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64812240 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff230009850 tx=0x7ff2300318b0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.151+0000 7ff25265e740 1 -- 172.21.15.63:0/4223755532 shutdown_connections 2024-05-27T07:40:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.151+0000 7ff25265e740 1 --2- 172.21.15.63:0/4223755532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644741b0 0x55be64820bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.151+0000 7ff25265e740 1 --2- 172.21.15.63:0/4223755532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be644534d0 0x55be64812240 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.151+0000 7ff25265e740 1 --2- 172.21.15.63:0/4223755532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be6444b2f0 0x55be6481c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.151+0000 7ff25265e740 1 -- 172.21.15.63:0/4223755532 >> 172.21.15.63:0/4223755532 conn(0x55be64468300 msgr2=0x55be648204a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.151+0000 7ff25265e740 1 -- 172.21.15.63:0/4223755532 shutdown_connections 2024-05-27T07:40:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.151+0000 7ff25265e740 1 -- 172.21.15.63:0/4223755532 wait complete. 2024-05-27T07:40:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.152+0000 7ff25265e740 1 -- start start 2024-05-27T07:40:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.152+0000 7ff25265e740 1 Processor -- start 2024-05-27T07:40:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff25265e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64823e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff25265e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644534d0 0x55be648243b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff25265e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644741b0 0x55be648248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff25265e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff22400e570 con 0x55be6444b2f0 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff25265e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff22400f9e0 con 0x55be644741b0 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff25265e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff22400ec40 con 0x55be644534d0 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff25265e740 10 client.0 _ll_register_callbacks cb 0x7ff230009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff248c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64823e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff248c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64823e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33972/0 (socket says 172.21.15.63:33972) 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff248c00640 1 -- 172.21.15.63:0/3347146290 learned_addr learned my addr 172.21.15.63:0/3347146290 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff243e00640 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644534d0 0x55be648243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.153+0000 7ff249600640 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644741b0 0x55be648248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.154+0000 7ff248c00640 1 -- 172.21.15.63:0/3347146290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644534d0 msgr2=0x55be648243b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.154+0000 7ff248c00640 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644534d0 0x55be648243b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.154+0000 7ff248c00640 1 -- 172.21.15.63:0/3347146290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644741b0 msgr2=0x55be648248f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.154+0000 7ff248c00640 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644741b0 0x55be648248f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.154+0000 7ff248c00640 1 -- 172.21.15.63:0/3347146290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be64899630 con 0x55be6444b2f0 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.154+0000 7ff249600640 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644741b0 0x55be648248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.154+0000 7ff243e00640 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644534d0 0x55be648243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.154+0000 7ff248c00640 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64823e70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff2380a5de0 tx=0x7ff2380e6ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6242effaa86649e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.154+0000 7ff22a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.155+0000 7ff22a000640 1 -- 172.21.15.63:0/3347146290 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380a5c20 con 0x55be6444b2f0 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.155+0000 7ff22a000640 1 -- 172.21.15.63:0/3347146290 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7ff2380e6b30 con 0x55be6444b2f0 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.155+0000 7ff25265e740 1 -- 172.21.15.63:0/3347146290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2300046c0 con 0x55be6444b2f0 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.155+0000 7ff22a000640 1 -- 172.21.15.63:0/3347146290 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff238010720 con 0x55be6444b2f0 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.155+0000 7ff25265e740 10 client.16824 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:52.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.155+0000 7ff25265e740 1 -- 172.21.15.63:0/3347146290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55be6489a870 con 0x55be6444b2f0 2024-05-27T07:40:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.156+0000 7ff22a000640 1 -- 172.21.15.63:0/3347146290 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2300046c0 con 0x55be6444b2f0 2024-05-27T07:40:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.156+0000 7ff22a000640 1 -- 172.21.15.63:0/3347146290 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff2380c2070 con 0x55be6444b2f0 2024-05-27T07:40:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.156+0000 7ff25265e740 10 client.16824 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.156+0000 7ff25265e740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.156+0000 7ff25265e740 20 client.16824 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.156+0000 7ff25265e740 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be644081c0 0x55be6489cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.156+0000 7ff25265e740 4 client.16824 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:40:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.156+0000 7ff25265e740 1 -- 172.21.15.63:0/3347146290 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be647832c0 con 0x55be644081c0 2024-05-27T07:40:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.157+0000 7ff243e00640 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be644081c0 0x55be6489cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.158+0000 7ff243e00640 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be644081c0 0x55be6489cb70 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=169f51fb9101be0e server_cookie=93c93182ac81810 in_seq=0 out_seq=0 2024-05-27T07:40:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.158+0000 7ff22a000640 10 client.16824 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff22a000640 1 -- 172.21.15.63:0/3347146290 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55be647832c0 con 0x55be644081c0 2024-05-27T07:40:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff22a000640 10 client.16824 handle_command_reply: tid=0 2024-05-27T07:40:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 client.16824 shutdown 2024-05-27T07:40:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 -- 172.21.15.63:0/3347146290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 msgr2=0x55be64823e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64823e70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff2380a5de0 tx=0x7ff2380e6ac0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 -- 172.21.15.63:0/3347146290 shutdown_connections 2024-05-27T07:40:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be644081c0 0x55be6489cb70 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be644741b0 0x55be648248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be644534d0 0x55be648243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 --2- 172.21.15.63:0/3347146290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be6444b2f0 0x55be64823e70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 -- 172.21.15.63:0/3347146290 >> 172.21.15.63:0/3347146290 conn(0x55be64468300 msgr2=0x55be64811fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff22a000640 0 client.16824 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 -- 172.21.15.63:0/3347146290 shutdown_connections 2024-05-27T07:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.191+0000 7ff25265e740 1 -- 172.21.15.63:0/3347146290 wait complete. 2024-05-27T07:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.192+0000 7ff25265e740 20 client.16824 trim_cache size 0 max 16384 2024-05-27T07:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.192+0000 7ff24a000640 1 -- 172.21.15.63:0/3584678695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff22c0b2950 msgr2=0x7ff22c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.192+0000 7ff24a000640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff22c0b2950 0x7ff22c0b4e60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55be64813bd0 tx=0x7ff23c009760 comp rx=0 tx=0).stop 2024-05-27T07:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.192+0000 7ff24a000640 1 -- 172.21.15.63:0/3584678695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff244075420 msgr2=0x7ff2441a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.192+0000 7ff24a000640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff244075420 0x7ff2441a24e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff23800e9f0 tx=0x7ff23800eeb0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.193+0000 7ff24a000640 1 -- 172.21.15.63:0/3584678695 shutdown_connections 2024-05-27T07:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.193+0000 7ff24a000640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff22c0b2950 0x7ff22c0b4e60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.193+0000 7ff24a000640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff244110320 0x7ff24419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.193+0000 7ff24a000640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff24410d9c0 0x7ff2441a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.193+0000 7ff24a000640 1 --2- 172.21.15.63:0/3584678695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff244075420 0x7ff2441a24e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.193+0000 7ff24a000640 1 -- 172.21.15.63:0/3584678695 >> 172.21.15.63:0/3584678695 conn(0x7ff244079100 msgr2=0x7ff2441022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.193+0000 7ff24a000640 1 -- 172.21.15.63:0/3584678695 shutdown_connections 2024-05-27T07:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:40:52.193+0000 7ff24a000640 1 -- 172.21.15.63:0/3584678695 wait complete. 2024-05-27T07:40:52.724 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:52 smithi063 ceph-mon[31613]: pgmap v224: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:40:52.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:52 smithi086 ceph-mon[34687]: pgmap v224: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:40:52.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:52 smithi149 ceph-mon[35408]: pgmap v224: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:40:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.112+0000 7f6263200700 1 -- 172.21.15.63:0/918410242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6264109680 msgr2=0x7f6264110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.112+0000 7f6263200700 1 --2- 172.21.15.63:0/918410242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6264109680 0x7f6264110fe0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f625c00b0d0 tx=0x7f625c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.112+0000 7f6263200700 1 -- 172.21.15.63:0/918410242 shutdown_connections 2024-05-27T07:40:53.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.112+0000 7f6263200700 1 --2- 172.21.15.63:0/918410242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6264109680 0x7f6264110fe0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.112+0000 7f6263200700 1 --2- 172.21.15.63:0/918410242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6264104400 0x7f6264104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.112+0000 7f6263200700 1 --2- 172.21.15.63:0/918410242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6264108d30 0x7f6264109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.112+0000 7f6263200700 1 -- 172.21.15.63:0/918410242 >> 172.21.15.63:0/918410242 conn(0x7f6264079d40 msgr2=0x7f626407a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:53.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.113+0000 7f6263200700 1 -- 172.21.15.63:0/918410242 shutdown_connections 2024-05-27T07:40:53.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.113+0000 7f6263200700 1 -- 172.21.15.63:0/918410242 wait complete. 2024-05-27T07:40:53.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.114+0000 7f6263200700 1 Processor -- start 2024-05-27T07:40:53.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.115+0000 7f6263200700 1 -- start start 2024-05-27T07:40:53.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.115+0000 7f6263200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6264104400 0x7f6264075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.115+0000 7f6263200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6264108d30 0x7f6264075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.115+0000 7f6263200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6264109680 0x7f6264076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.115+0000 7f6263200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62641141b0 con 0x7f6264104400 2024-05-27T07:40:53.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.115+0000 7f6263200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6264114030 con 0x7f6264109680 2024-05-27T07:40:53.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.115+0000 7f6263200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6264114330 con 0x7f6264108d30 2024-05-27T07:40:53.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6261e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6264104400 0x7f6264075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6261e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6264104400 0x7f6264075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33990/0 (socket says 172.21.15.63:33990) 2024-05-27T07:40:53.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6261e00700 1 -- 172.21.15.63:0/244189119 learned_addr learned my addr 172.21.15.63:0/244189119 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:53.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6261400700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6264108d30 0x7f6264075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6262800700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6264109680 0x7f6264076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6261e00700 1 -- 172.21.15.63:0/244189119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6264108d30 msgr2=0x7f6264075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6261e00700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6264108d30 0x7f6264075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6261e00700 1 -- 172.21.15.63:0/244189119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6264109680 msgr2=0x7f6264076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6261e00700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6264109680 0x7f6264076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6261e00700 1 -- 172.21.15.63:0/244189119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62641a8ec0 con 0x7f6264104400 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6261400700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6264108d30 0x7f6264075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.116+0000 7f6262800700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6264109680 0x7f6264076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.117+0000 7f6261e00700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6264104400 0x7f6264075ab0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f625400e9a0 tx=0x7f625400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.117+0000 7f625aa00700 1 -- 172.21.15.63:0/244189119 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625400cbd0 con 0x7f6264104400 2024-05-27T07:40:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.117+0000 7f625aa00700 1 -- 172.21.15.63:0/244189119 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f625400cd70 con 0x7f6264104400 2024-05-27T07:40:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.117+0000 7f625aa00700 1 -- 172.21.15.63:0/244189119 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254018680 con 0x7f6264104400 2024-05-27T07:40:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.117+0000 7f6263200700 1 -- 172.21.15.63:0/244189119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62641a91b0 con 0x7f6264104400 2024-05-27T07:40:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.117+0000 7f6263200700 1 -- 172.21.15.63:0/244189119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62641a96a0 con 0x7f6264104400 2024-05-27T07:40:53.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.122+0000 7f625aa00700 1 -- 172.21.15.63:0/244189119 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6254010950 con 0x7f6264104400 2024-05-27T07:40:53.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.124+0000 7f625aa00700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62480b2a70 0x7f62480b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.124+0000 7f625aa00700 1 -- 172.21.15.63:0/244189119 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f62540e2cf0 con 0x7f6264104400 2024-05-27T07:40:53.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.124+0000 7f6277911b80 1 Processor -- start 2024-05-27T07:40:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.124+0000 7f6261400700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62480b2a70 0x7f62480b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.124+0000 7f6277911b80 1 -- start start 2024-05-27T07:40:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.125+0000 7f6277911b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547d43a70 0x55c547dd77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.125+0000 7f6277911b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547c57ec0 0x55c547dd7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.125+0000 7f6277911b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547de1340 0x55c547de3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.125+0000 7f6277911b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c547d04ec0 con 0x55c547d43a70 2024-05-27T07:40:53.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.125+0000 7f6277911b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c547cf4a60 con 0x55c547c57ec0 2024-05-27T07:40:53.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.125+0000 7f6277911b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c547cf8aa0 con 0x55c547de1340 2024-05-27T07:40:53.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.125+0000 7f6261e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547d43a70 0x55c547dd77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.125+0000 7f6261400700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62480b2a70 0x7f62480b4f80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6264100e20 tx=0x7f6250009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.125+0000 7f6261e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547d43a70 0x55c547dd77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33996/0 (socket says 172.21.15.63:33996) 2024-05-27T07:40:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.125+0000 7f6261e00700 1 -- 172.21.15.63:0/1519882712 learned_addr learned my addr 172.21.15.63:0/1519882712 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.126+0000 7f6261400700 1 --2- 172.21.15.63:0/1519882712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547c57ec0 0x55c547dd7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.126+0000 7f6262800700 1 --2- 172.21.15.63:0/1519882712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547de1340 0x55c547de3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.126+0000 7f6261e00700 1 -- 172.21.15.63:0/1519882712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547de1340 msgr2=0x55c547de3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.126+0000 7f6261e00700 1 --2- 172.21.15.63:0/1519882712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547de1340 0x55c547de3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.126+0000 7f6261e00700 1 -- 172.21.15.63:0/1519882712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547c57ec0 msgr2=0x55c547dd7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.126+0000 7f6261e00700 1 --2- 172.21.15.63:0/1519882712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547c57ec0 0x55c547dd7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.126+0000 7f6261e00700 1 -- 172.21.15.63:0/1519882712 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c547cbd4a0 con 0x55c547d43a70 2024-05-27T07:40:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.126+0000 7f6261e00700 1 --2- 172.21.15.63:0/1519882712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547d43a70 0x55c547dd77d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f62540a9b40 tx=0x7f62540a9ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50364073f2ab5668 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.127+0000 7f624f400700 1 -- 172.21.15.63:0/1519882712 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540cae60 con 0x55c547d43a70 2024-05-27T07:40:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.127+0000 7f624f400700 1 -- 172.21.15.63:0/1519882712 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62540aa2b0 con 0x55c547d43a70 2024-05-27T07:40:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.127+0000 7f624f400700 1 -- 172.21.15.63:0/1519882712 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540b2780 con 0x55c547d43a70 2024-05-27T07:40:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.128+0000 7f6277911b80 1 -- 172.21.15.63:0/1519882712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547d43a70 msgr2=0x55c547dd77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.128+0000 7f6277911b80 1 --2- 172.21.15.63:0/1519882712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547d43a70 0x55c547dd77d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f62540a9b40 tx=0x7f62540a9ab0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.128+0000 7f6277911b80 1 -- 172.21.15.63:0/1519882712 shutdown_connections 2024-05-27T07:40:53.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.128+0000 7f6277911b80 1 --2- 172.21.15.63:0/1519882712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547de1340 0x55c547de3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.128+0000 7f6277911b80 1 --2- 172.21.15.63:0/1519882712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547d43a70 0x55c547dd77d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.128+0000 7f6277911b80 1 --2- 172.21.15.63:0/1519882712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547c57ec0 0x55c547dd7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.128+0000 7f6277911b80 1 -- 172.21.15.63:0/1519882712 >> 172.21.15.63:0/1519882712 conn(0x55c547cf1410 msgr2=0x55c547dd2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.128+0000 7f6277911b80 1 -- 172.21.15.63:0/1519882712 shutdown_connections 2024-05-27T07:40:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.128+0000 7f6277911b80 1 -- 172.21.15.63:0/1519882712 wait complete. 2024-05-27T07:40:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.129+0000 7f6277911b80 1 -- start start 2024-05-27T07:40:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.129+0000 7f6277911b80 1 Processor -- start 2024-05-27T07:40:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.130+0000 7f6277911b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547c57ec0 0x55c547dde340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.130+0000 7f6277911b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547d43a70 0x55c547dde880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.130+0000 7f6277911b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547de1340 0x55c547ddedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.130+0000 7f6277911b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c547bfb020 con 0x55c547de1340 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.130+0000 7f6277911b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c547c957e0 con 0x55c547d43a70 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.130+0000 7f6277911b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c547de3e20 con 0x55c547c57ec0 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.130+0000 7f6277911b80 10 client.0 _ll_register_callbacks cb 0x55c547c9ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.131+0000 7f6262800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547de1340 0x55c547ddedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.131+0000 7f6262800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547de1340 0x55c547ddedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34012/0 (socket says 172.21.15.63:34012) 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.131+0000 7f6262800700 1 -- 172.21.15.63:0/1644519281 learned_addr learned my addr 172.21.15.63:0/1644519281 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.131+0000 7f6261e00700 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547c57ec0 0x55c547dde340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.131+0000 7f6261400700 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547d43a70 0x55c547dde880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.131+0000 7f6262800700 1 -- 172.21.15.63:0/1644519281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547c57ec0 msgr2=0x55c547dde340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.131+0000 7f6262800700 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547c57ec0 0x55c547dde340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.131+0000 7f6262800700 1 -- 172.21.15.63:0/1644519281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547d43a70 msgr2=0x55c547dde880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.131+0000 7f6262800700 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547d43a70 0x55c547dde880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.131+0000 7f6262800700 1 -- 172.21.15.63:0/1644519281 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c547c44950 con 0x55c547de1340 2024-05-27T07:40:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.132+0000 7f6262800700 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547de1340 0x55c547ddedc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f625c00bcd0 tx=0x7f625c00bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d2282f2d51af62 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.132+0000 7f6243e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.132+0000 7f6243e00700 1 -- 172.21.15.63:0/1644519281 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c021070 con 0x55c547de1340 2024-05-27T07:40:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.132+0000 7f6243e00700 1 -- 172.21.15.63:0/1644519281 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f625c0075c0 con 0x55c547de1340 2024-05-27T07:40:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.132+0000 7f6277911b80 1 -- 172.21.15.63:0/1644519281 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62540aa2b0 con 0x55c547de1340 2024-05-27T07:40:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.133+0000 7f6243e00700 1 -- 172.21.15.63:0/1644519281 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f625c02e050 con 0x55c547de1340 2024-05-27T07:40:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.133+0000 7f6277911b80 10 client.16836 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.133+0000 7f6277911b80 1 -- 172.21.15.63:0/1644519281 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c547cc50e0 con 0x55c547de1340 2024-05-27T07:40:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.133+0000 7f6243e00700 1 -- 172.21.15.63:0/1644519281 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62540aa2b0 con 0x55c547de1340 2024-05-27T07:40:53.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.133+0000 7f6243e00700 1 -- 172.21.15.63:0/1644519281 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f625c011070 con 0x55c547de1340 2024-05-27T07:40:53.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.134+0000 7f6277911b80 10 client.16836 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:53.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.134+0000 7f6277911b80 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:53.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.134+0000 7f6277911b80 20 client.16836 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:53.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.134+0000 7f6277911b80 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c547df0d70 0x55c547df3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.134+0000 7f6277911b80 4 client.16836 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:40:53.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.134+0000 7f6277911b80 1 -- 172.21.15.63:0/1644519281 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c547d06e10 con 0x55c547df0d70 2024-05-27T07:40:53.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.134+0000 7f6261e00700 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c547df0d70 0x55c547df3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.135+0000 7f6261e00700 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c547df0d70 0x55c547df3190 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999b01e9b8cbb4bf server_cookie=e4ab5523461b41cf in_seq=0 out_seq=0 2024-05-27T07:40:53.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.135+0000 7f6243e00700 10 client.16836 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:53.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.137+0000 7f6243e00700 1 -- 172.21.15.63:0/1644519281 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c547d06e10 con 0x55c547df0d70 2024-05-27T07:40:53.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.137+0000 7f6243e00700 10 client.16836 handle_command_reply: tid=0 2024-05-27T07:40:53.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 client.16836 shutdown 2024-05-27T07:40:53.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 -- 172.21.15.63:0/1644519281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547de1340 msgr2=0x55c547ddedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547de1340 0x55c547ddedc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f625c00bcd0 tx=0x7f625c00bd00 comp rx=0 tx=0).stop 2024-05-27T07:40:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 -- 172.21.15.63:0/1644519281 shutdown_connections 2024-05-27T07:40:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c547df0d70 0x55c547df3190 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547de1340 0x55c547ddedc0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547c57ec0 0x55c547dde340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 --2- 172.21.15.63:0/1644519281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547d43a70 0x55c547dde880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6243e00700 0 client.16836 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:53.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 -- 172.21.15.63:0/1644519281 >> 172.21.15.63:0/1644519281 conn(0x55c547cf1410 msgr2=0x55c547dd2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:53.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 -- 172.21.15.63:0/1644519281 shutdown_connections 2024-05-27T07:40:53.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.138+0000 7f6277911b80 1 -- 172.21.15.63:0/1644519281 wait complete. 2024-05-27T07:40:53.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.139+0000 7f6277911b80 20 client.16836 trim_cache size 0 max 16384 2024-05-27T07:40:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.168+0000 7f6277911b80 1 Processor -- start 2024-05-27T07:40:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.168+0000 7f6277911b80 1 -- start start 2024-05-27T07:40:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.168+0000 7f6277911b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547ddd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.168+0000 7f6277911b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547d43a70 0x55c547dd6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.168+0000 7f6277911b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547de1340 0x55c547decec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.168+0000 7f6277911b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62641b4e20 con 0x55c547c57ec0 2024-05-27T07:40:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.168+0000 7f6277911b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62641b26b0 con 0x55c547de1340 2024-05-27T07:40:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.168+0000 7f6277911b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62641b0520 con 0x55c547d43a70 2024-05-27T07:40:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.169+0000 7f6261e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547ddd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.169+0000 7f6261e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547ddd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34028/0 (socket says 172.21.15.63:34028) 2024-05-27T07:40:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.169+0000 7f6261e00700 1 -- 172.21.15.63:0/1331216887 learned_addr learned my addr 172.21.15.63:0/1331216887 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.169+0000 7f6262800700 1 --2- 172.21.15.63:0/1331216887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547de1340 0x55c547decec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.169+0000 7f6261400700 1 --2- 172.21.15.63:0/1331216887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547d43a70 0x55c547dd6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.169+0000 7f6261e00700 1 -- 172.21.15.63:0/1331216887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547d43a70 msgr2=0x55c547dd6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.169+0000 7f6261e00700 1 --2- 172.21.15.63:0/1331216887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547d43a70 0x55c547dd6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.169+0000 7f6261e00700 1 -- 172.21.15.63:0/1331216887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547de1340 msgr2=0x55c547decec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.169+0000 7f6261e00700 1 --2- 172.21.15.63:0/1331216887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547de1340 0x55c547decec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.169+0000 7f6261e00700 1 -- 172.21.15.63:0/1331216887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c547dd3c70 con 0x55c547c57ec0 2024-05-27T07:40:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.170+0000 7f6261e00700 1 --2- 172.21.15.63:0/1331216887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547ddd2a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f625400e970 tx=0x7f62540e2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f0e2c54fae0ece server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.170+0000 7f624ea00700 1 -- 172.21.15.63:0/1331216887 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540cae60 con 0x55c547c57ec0 2024-05-27T07:40:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.170+0000 7f624ea00700 1 -- 172.21.15.63:0/1331216887 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62540b2a10 con 0x55c547c57ec0 2024-05-27T07:40:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.170+0000 7f624ea00700 1 -- 172.21.15.63:0/1331216887 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540b27c0 con 0x55c547c57ec0 2024-05-27T07:40:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.171+0000 7f6277911b80 1 -- 172.21.15.63:0/1331216887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 msgr2=0x55c547ddd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.171+0000 7f6277911b80 1 --2- 172.21.15.63:0/1331216887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547ddd2a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f625400e970 tx=0x7f62540e2040 comp rx=0 tx=0).stop 2024-05-27T07:40:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.171+0000 7f6277911b80 1 -- 172.21.15.63:0/1331216887 shutdown_connections 2024-05-27T07:40:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.171+0000 7f6277911b80 1 --2- 172.21.15.63:0/1331216887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547de1340 0x55c547decec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.171+0000 7f6277911b80 1 --2- 172.21.15.63:0/1331216887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547ddd2a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.171+0000 7f6277911b80 1 --2- 172.21.15.63:0/1331216887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547d43a70 0x55c547dd6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.171+0000 7f6277911b80 1 -- 172.21.15.63:0/1331216887 >> 172.21.15.63:0/1331216887 conn(0x55c547cf1410 msgr2=0x55c547dd90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.171+0000 7f6277911b80 1 -- 172.21.15.63:0/1331216887 shutdown_connections 2024-05-27T07:40:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.171+0000 7f6277911b80 1 -- 172.21.15.63:0/1331216887 wait complete. 2024-05-27T07:40:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.172+0000 7f6277911b80 1 -- start start 2024-05-27T07:40:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.172+0000 7f6277911b80 1 Processor -- start 2024-05-27T07:40:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.173+0000 7f6277911b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547e5c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.173+0000 7f6277911b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547d43a70 0x55c547e5c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.173+0000 7f6277911b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547de1340 0x55c547de5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.173+0000 7f6277911b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62641b0b70 con 0x55c547c57ec0 2024-05-27T07:40:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.173+0000 7f6277911b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62641b1fe0 con 0x55c547d43a70 2024-05-27T07:40:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.173+0000 7f6277911b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62641b1240 con 0x55c547de1340 2024-05-27T07:40:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.173+0000 7f6277911b80 10 client.0 _ll_register_callbacks cb 0x55c547c9ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.173+0000 7f6261e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547e5c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.173+0000 7f6261e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547e5c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34034/0 (socket says 172.21.15.63:34034) 2024-05-27T07:40:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.173+0000 7f6261e00700 1 -- 172.21.15.63:0/3176180516 learned_addr learned my addr 172.21.15.63:0/3176180516 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.174+0000 7f6261400700 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547d43a70 0x55c547e5c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.174+0000 7f6261e00700 1 -- 172.21.15.63:0/3176180516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547de1340 msgr2=0x55c547de5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.174+0000 7f6261e00700 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547de1340 0x55c547de5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.174+0000 7f6261e00700 1 -- 172.21.15.63:0/3176180516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547d43a70 msgr2=0x55c547e5c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.174+0000 7f6261e00700 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547d43a70 0x55c547e5c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.174+0000 7f6261e00700 1 -- 172.21.15.63:0/3176180516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c547de5cd0 con 0x55c547c57ec0 2024-05-27T07:40:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.174+0000 7f6261e00700 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547e5c2e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f62540a9d70 tx=0x7f62540ae0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebe5168a6ad5913 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.174+0000 7f6243e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.175+0000 7f6243e00700 1 -- 172.21.15.63:0/3176180516 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540cae60 con 0x55c547c57ec0 2024-05-27T07:40:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.175+0000 7f6243e00700 1 -- 172.21.15.63:0/3176180516 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f62540b2780 con 0x55c547c57ec0 2024-05-27T07:40:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.175+0000 7f6277911b80 1 -- 172.21.15.63:0/3176180516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62540b2a10 con 0x55c547c57ec0 2024-05-27T07:40:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.175+0000 7f6243e00700 1 -- 172.21.15.63:0/3176180516 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f62540a8e10 con 0x55c547c57ec0 2024-05-27T07:40:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.175+0000 7f6277911b80 10 client.16842 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.175+0000 7f6277911b80 1 -- 172.21.15.63:0/3176180516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c547e64ac0 con 0x55c547c57ec0 2024-05-27T07:40:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.175+0000 7f6243e00700 1 -- 172.21.15.63:0/3176180516 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62540b2a10 con 0x55c547c57ec0 2024-05-27T07:40:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.175+0000 7f6243e00700 1 -- 172.21.15.63:0/3176180516 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f62540a8880 con 0x55c547c57ec0 2024-05-27T07:40:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.176+0000 7f6277911b80 10 client.16842 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.176+0000 7f6277911b80 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.176+0000 7f6277911b80 20 client.16842 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.176+0000 7f6277911b80 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c547e64c90 0x55c547e670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.176+0000 7f6277911b80 4 client.16842 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:40:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.176+0000 7f6277911b80 1 -- 172.21.15.63:0/3176180516 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c547d005f0 con 0x55c547e64c90 2024-05-27T07:40:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.176+0000 7f6261400700 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c547e64c90 0x55c547e670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.177+0000 7f6261400700 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c547e64c90 0x55c547e670b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed9bfb3d00aa9c7f server_cookie=ef2e86ea7280c6c3 in_seq=0 out_seq=0 2024-05-27T07:40:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.177+0000 7f6243e00700 10 client.16842 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:53.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.180+0000 7f6243e00700 1 -- 172.21.15.63:0/3176180516 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c547d005f0 con 0x55c547e64c90 2024-05-27T07:40:53.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.180+0000 7f6243e00700 10 client.16842 handle_command_reply: tid=0 2024-05-27T07:40:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.180+0000 7f6277911b80 1 client.16842 shutdown 2024-05-27T07:40:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.180+0000 7f6277911b80 1 -- 172.21.15.63:0/3176180516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 msgr2=0x55c547e5c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.180+0000 7f6277911b80 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547e5c2e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f62540a9d70 tx=0x7f62540ae0d0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.181+0000 7f6277911b80 1 -- 172.21.15.63:0/3176180516 shutdown_connections 2024-05-27T07:40:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.181+0000 7f6277911b80 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c547e64c90 0x55c547e670b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.181+0000 7f6277911b80 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c547de1340 0x55c547de5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.181+0000 7f6277911b80 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c547c57ec0 0x55c547e5c2e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.181+0000 7f6277911b80 1 --2- 172.21.15.63:0/3176180516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c547d43a70 0x55c547e5c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.181+0000 7f6277911b80 1 -- 172.21.15.63:0/3176180516 >> 172.21.15.63:0/3176180516 conn(0x55c547cf1410 msgr2=0x55c547dd6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.181+0000 7f6243e00700 0 client.16842 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.181+0000 7f6277911b80 1 -- 172.21.15.63:0/3176180516 shutdown_connections 2024-05-27T07:40:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.181+0000 7f6277911b80 1 -- 172.21.15.63:0/3176180516 wait complete. 2024-05-27T07:40:53.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.181+0000 7f6277911b80 20 client.16842 trim_cache size 0 max 16384 2024-05-27T07:40:53.181 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:40:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:40:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "5354509e-c894-4d87-9340-fa3c68884592", 2024-05-27T07:40:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:40:53.182 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.182+0000 7f6263200700 1 -- 172.21.15.63:0/244189119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62480b2a70 msgr2=0x7f62480b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.182+0000 7f6263200700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62480b2a70 0x7f62480b4f80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6264100e20 tx=0x7f6250009260 comp rx=0 tx=0).stop 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.182+0000 7f6263200700 1 -- 172.21.15.63:0/244189119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6264104400 msgr2=0x7f6264075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.182+0000 7f6263200700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6264104400 0x7f6264075ab0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f625400e9a0 tx=0x7f625400ed60 comp rx=0 tx=0).stop 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.183+0000 7f6263200700 1 -- 172.21.15.63:0/244189119 shutdown_connections 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.183+0000 7f6263200700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62480b2a70 0x7f62480b4f80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.183+0000 7f6263200700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6264109680 0x7f6264076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.183+0000 7f6263200700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6264104400 0x7f6264075ab0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.183+0000 7f6263200700 1 --2- 172.21.15.63:0/244189119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6264108d30 0x7f6264075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.183+0000 7f6263200700 1 -- 172.21.15.63:0/244189119 >> 172.21.15.63:0/244189119 conn(0x7f6264079d40 msgr2=0x7f6264101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.183+0000 7f6263200700 1 -- 172.21.15.63:0/244189119 shutdown_connections 2024-05-27T07:40:53.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:53.183+0000 7f6263200700 1 -- 172.21.15.63:0/244189119 wait complete. 2024-05-27T07:40:53.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:53 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:40:53.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:53 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:53.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:53 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:40:53.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:53 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:53.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:53 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:40:53.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:53 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:53.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:53 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:40:53.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:53 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:53.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:53 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:40:53.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:53 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:53.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:53 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:40:53.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:53 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.185 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:40:54.675 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:54.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:40:54.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:40:54.829 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:40:54.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:40:54.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:40:54.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:40:54.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.830 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:54 smithi063 ceph-mon[31613]: pgmap v225: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:40:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:54 smithi086 ceph-mon[34687]: pgmap v225: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:40:54.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:40:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:54 smithi149 ceph-mon[35408]: pgmap v225: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:40:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.140+0000 7f86a3e00700 1 -- 172.21.15.63:0/3285237395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86a410b8b0 msgr2=0x7f86a4113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.140+0000 7f86a3e00700 1 --2- 172.21.15.63:0/3285237395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86a410b8b0 0x7f86a4113210 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f869c00b0d0 tx=0x7f869c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.140+0000 7f86a3e00700 1 -- 172.21.15.63:0/3285237395 shutdown_connections 2024-05-27T07:40:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.140+0000 7f86a3e00700 1 --2- 172.21.15.63:0/3285237395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86a410b8b0 0x7f86a4113210 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.140+0000 7f86a3e00700 1 --2- 172.21.15.63:0/3285237395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a4106630 0x7f86a4106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.140+0000 7f86a3e00700 1 --2- 172.21.15.63:0/3285237395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86a410af60 0x7f86a410b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.140+0000 7f86a3e00700 1 -- 172.21.15.63:0/3285237395 >> 172.21.15.63:0/3285237395 conn(0x7f86a4101ec0 msgr2=0x7f86a4104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.140+0000 7f86a3e00700 1 -- 172.21.15.63:0/3285237395 shutdown_connections 2024-05-27T07:40:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.140+0000 7f86a3e00700 1 -- 172.21.15.63:0/3285237395 wait complete. 2024-05-27T07:40:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.142+0000 7f86a3e00700 1 Processor -- start 2024-05-27T07:40:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.142+0000 7f86a3e00700 1 -- start start 2024-05-27T07:40:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.142+0000 7f86a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a4106630 0x7f86a41a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.142+0000 7f86a3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86a410af60 0x7f86a41a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.142+0000 7f86a3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86a410b8b0 0x7f86a419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.143+0000 7f86a3e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4116390 con 0x7f86a410af60 2024-05-27T07:40:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.143+0000 7f86a3e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4116210 con 0x7f86a4106630 2024-05-27T07:40:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.143+0000 7f86a3e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4116510 con 0x7f86a410b8b0 2024-05-27T07:40:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.143+0000 7f86a2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86a410af60 0x7f86a41a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.143+0000 7f86a2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86a410af60 0x7f86a41a2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34046/0 (socket says 172.21.15.63:34046) 2024-05-27T07:40:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.143+0000 7f86a2000700 1 -- 172.21.15.63:0/4438446 learned_addr learned my addr 172.21.15.63:0/4438446 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.143+0000 7f86a2a00700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a4106630 0x7f86a41a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.143+0000 7f86a3400700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86a410b8b0 0x7f86a419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.144+0000 7f86a2000700 1 -- 172.21.15.63:0/4438446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86a410b8b0 msgr2=0x7f86a419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:57.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.144+0000 7f86a2000700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86a410b8b0 0x7f86a419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.144+0000 7f86a2000700 1 -- 172.21.15.63:0/4438446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a4106630 msgr2=0x7f86a41a2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:57.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.144+0000 7f86a2000700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a4106630 0x7f86a41a2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.144+0000 7f86a2000700 1 -- 172.21.15.63:0/4438446 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a419d140 con 0x7f86a410af60 2024-05-27T07:40:57.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.144+0000 7f86a2000700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86a410af60 0x7f86a41a2df0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f869000d950 tx=0x7f869000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.145+0000 7f869b400700 1 -- 172.21.15.63:0/4438446 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690016070 con 0x7f86a410af60 2024-05-27T07:40:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.145+0000 7f869b400700 1 -- 172.21.15.63:0/4438446 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8690004830 con 0x7f86a410af60 2024-05-27T07:40:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.145+0000 7f86a3e00700 1 -- 172.21.15.63:0/4438446 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a419d430 con 0x7f86a410af60 2024-05-27T07:40:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.145+0000 7f869b400700 1 -- 172.21.15.63:0/4438446 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690005410 con 0x7f86a410af60 2024-05-27T07:40:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.145+0000 7f86a3e00700 1 -- 172.21.15.63:0/4438446 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a41a9740 con 0x7f86a410af60 2024-05-27T07:40:57.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.151+0000 7f869b400700 1 -- 172.21.15.63:0/4438446 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8690004220 con 0x7f86a410af60 2024-05-27T07:40:57.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.151+0000 7f86b837fb80 1 Processor -- start 2024-05-27T07:40:57.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.151+0000 7f86b837fb80 1 -- start start 2024-05-27T07:40:57.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.152+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 0x5561534b9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.152+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.152+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 0x5561534c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.152+0000 7f86b837fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5561533e6e40 con 0x556153339e40 2024-05-27T07:40:57.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.152+0000 7f86b837fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561533d69e0 con 0x5561534c32c0 2024-05-27T07:40:57.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.152+0000 7f86b837fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5561533daa20 con 0x5561534259f0 2024-05-27T07:40:57.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.152+0000 7f86a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f86a3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 0x5561534b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f86a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34062/0 (socket says 172.21.15.63:34062) 2024-05-27T07:40:57.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f86a2a00700 1 -- 172.21.15.63:0/2767376546 learned_addr learned my addr 172.21.15.63:0/2767376546 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:57.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f86a2000700 1 --2- 172.21.15.63:0/2767376546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 0x5561534c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f869b400700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86880b2a70 0x7f86880b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f86a2a00700 1 -- 172.21.15.63:0/2767376546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 msgr2=0x5561534b9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:57.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f86a2a00700 1 --2- 172.21.15.63:0/2767376546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 0x5561534b9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f86a2a00700 1 -- 172.21.15.63:0/2767376546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 msgr2=0x5561534c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:57.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f86a2a00700 1 --2- 172.21.15.63:0/2767376546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 0x5561534c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f86a2a00700 1 -- 172.21.15.63:0/2767376546 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55615339f420 con 0x556153339e40 2024-05-27T07:40:57.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.153+0000 7f86a2000700 1 --2- 172.21.15.63:0/2767376546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 0x5561534c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:57.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.154+0000 7f869b400700 1 -- 172.21.15.63:0/4438446 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f86900e25e0 con 0x7f86a410af60 2024-05-27T07:40:57.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.154+0000 7f86a3400700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86880b2a70 0x7f86880b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.154+0000 7f86a2a00700 1 --2- 172.21.15.63:0/2767376546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9c90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f869400acf0 tx=0x7f869400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14395fa73488b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:57.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.154+0000 7f868fe00700 1 -- 172.21.15.63:0/2767376546 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869400fe50 con 0x556153339e40 2024-05-27T07:40:57.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.154+0000 7f868fe00700 1 -- 172.21.15.63:0/2767376546 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86940044b0 con 0x556153339e40 2024-05-27T07:40:57.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.154+0000 7f868fe00700 1 -- 172.21.15.63:0/2767376546 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694018440 con 0x556153339e40 2024-05-27T07:40:57.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.156+0000 7f86b837fb80 1 -- 172.21.15.63:0/2767376546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 msgr2=0x5561534b9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:57.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.156+0000 7f86b837fb80 1 --2- 172.21.15.63:0/2767376546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9c90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f869400acf0 tx=0x7f869400f3c0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.156+0000 7f86a3400700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86880b2a70 0x7f86880b4f80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f869c009e60 tx=0x7f869c0099e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:57.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.156+0000 7f86b837fb80 1 -- 172.21.15.63:0/2767376546 shutdown_connections 2024-05-27T07:40:57.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.156+0000 7f86b837fb80 1 --2- 172.21.15.63:0/2767376546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 0x5561534c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.156+0000 7f86b837fb80 1 --2- 172.21.15.63:0/2767376546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 0x5561534b9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.156+0000 7f86b837fb80 1 --2- 172.21.15.63:0/2767376546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9c90 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.156+0000 7f86b837fb80 1 -- 172.21.15.63:0/2767376546 >> 172.21.15.63:0/2767376546 conn(0x5561533d3390 msgr2=0x5561534b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.156+0000 7f86b837fb80 1 -- 172.21.15.63:0/2767376546 shutdown_connections 2024-05-27T07:40:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.156+0000 7f86b837fb80 1 -- 172.21.15.63:0/2767376546 wait complete. 2024-05-27T07:40:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.157+0000 7f86b837fb80 1 -- start start 2024-05-27T07:40:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.157+0000 7f86b837fb80 1 Processor -- start 2024-05-27T07:40:57.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534c0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534259f0 0x5561534c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534c32c0 0x5561534c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86b837fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5561532dcfa0 con 0x556153339e40 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86b837fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556153377760 con 0x5561534259f0 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86b837fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5561534c5e60 con 0x5561534c32c0 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86b837fb80 10 client.0 _ll_register_callbacks cb 0x556153380c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534c0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34078/0 (socket says 172.21.15.63:34078) 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86a2a00700 1 -- 172.21.15.63:0/304247457 learned_addr learned my addr 172.21.15.63:0/304247457 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86a3400700 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534c32c0 0x5561534c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.158+0000 7f86a2000700 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534259f0 0x5561534c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.159+0000 7f86a2a00700 1 -- 172.21.15.63:0/304247457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534c32c0 msgr2=0x5561534c0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.159+0000 7f86a2a00700 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534c32c0 0x5561534c0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.159+0000 7f86a2a00700 1 -- 172.21.15.63:0/304247457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534259f0 msgr2=0x5561534c06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.159+0000 7f86a2a00700 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534259f0 0x5561534c06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.159+0000 7f86a2a00700 1 -- 172.21.15.63:0/304247457 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561533268d0 con 0x556153339e40 2024-05-27T07:40:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.159+0000 7f86a2a00700 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534c0190 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f86940040a0 tx=0x7f8694004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b244c8cd3aa5f7f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.159+0000 7f868cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:57.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.159+0000 7f868cc00700 1 -- 172.21.15.63:0/304247457 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694026020 con 0x556153339e40 2024-05-27T07:40:57.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.160+0000 7f86b837fb80 1 -- 172.21.15.63:0/304247457 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86940044b0 con 0x556153339e40 2024-05-27T07:40:57.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.160+0000 7f868cc00700 1 -- 172.21.15.63:0/304247457 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f8694004750 con 0x556153339e40 2024-05-27T07:40:57.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.160+0000 7f868cc00700 1 -- 172.21.15.63:0/304247457 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8694031050 con 0x556153339e40 2024-05-27T07:40:57.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.160+0000 7f86b837fb80 10 client.16854 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.160+0000 7f86b837fb80 1 -- 172.21.15.63:0/304247457 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5561533a7060 con 0x556153339e40 2024-05-27T07:40:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.160+0000 7f868cc00700 1 -- 172.21.15.63:0/304247457 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86940044b0 con 0x556153339e40 2024-05-27T07:40:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.161+0000 7f868cc00700 1 -- 172.21.15.63:0/304247457 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8694005520 con 0x556153339e40 2024-05-27T07:40:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.161+0000 7f86b837fb80 10 client.16854 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.161+0000 7f86b837fb80 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.161+0000 7f86b837fb80 20 client.16854 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.161+0000 7f86b837fb80 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561534d2b70 0x5561534d4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.161+0000 7f86b837fb80 4 client.16854 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:40:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.161+0000 7f86b837fb80 1 -- 172.21.15.63:0/304247457 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561533e8d90 con 0x5561534d2b70 2024-05-27T07:40:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.162+0000 7f86a2000700 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561534d2b70 0x5561534d4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.163+0000 7f86a2000700 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561534d2b70 0x5561534d4f90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9382c71e0fb44b86 server_cookie=ab63be2115229588 in_seq=0 out_seq=0 2024-05-27T07:40:57.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.163+0000 7f868cc00700 10 client.16854 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:57.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.164+0000 7f868cc00700 1 -- 172.21.15.63:0/304247457 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5561533e8d90 con 0x5561534d2b70 2024-05-27T07:40:57.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.164+0000 7f868cc00700 10 client.16854 handle_command_reply: tid=0 2024-05-27T07:40:57.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 client.16854 shutdown 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 -- 172.21.15.63:0/304247457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 msgr2=0x5561534c0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534c0190 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f86940040a0 tx=0x7f8694004180 comp rx=0 tx=0).stop 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 -- 172.21.15.63:0/304247457 shutdown_connections 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561534d2b70 0x5561534d4f90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534c32c0 0x5561534c0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534c0190 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 --2- 172.21.15.63:0/304247457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534259f0 0x5561534c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f868cc00700 0 client.16854 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 -- 172.21.15.63:0/304247457 >> 172.21.15.63:0/304247457 conn(0x5561533d3390 msgr2=0x5561534b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 -- 172.21.15.63:0/304247457 shutdown_connections 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 1 -- 172.21.15.63:0/304247457 wait complete. 2024-05-27T07:40:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.165+0000 7f86b837fb80 20 client.16854 trim_cache size 0 max 16384 2024-05-27T07:40:57.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.195+0000 7f86b837fb80 1 Processor -- start 2024-05-27T07:40:57.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.195+0000 7f86b837fb80 1 -- start start 2024-05-27T07:40:57.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.195+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.195+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 0x5561534b8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.195+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 0x5561534da530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.195+0000 7f86b837fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b4d70 con 0x556153339e40 2024-05-27T07:40:57.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.195+0000 7f86b837fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b2600 con 0x5561534c32c0 2024-05-27T07:40:57.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.195+0000 7f86b837fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b0470 con 0x5561534259f0 2024-05-27T07:40:57.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34094/0 (socket says 172.21.15.63:34094) 2024-05-27T07:40:57.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a2a00700 1 -- 172.21.15.63:0/991806677 learned_addr learned my addr 172.21.15.63:0/991806677 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:57.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a3400700 1 --2- 172.21.15.63:0/991806677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 0x5561534da530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a2000700 1 --2- 172.21.15.63:0/991806677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 0x5561534b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a2a00700 1 -- 172.21.15.63:0/991806677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 msgr2=0x5561534b8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:57.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a2a00700 1 --2- 172.21.15.63:0/991806677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 0x5561534b8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a2a00700 1 -- 172.21.15.63:0/991806677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 msgr2=0x5561534da530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:57.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a2a00700 1 --2- 172.21.15.63:0/991806677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 0x5561534da530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a2a00700 1 -- 172.21.15.63:0/991806677 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561534b4c50 con 0x556153339e40 2024-05-27T07:40:57.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.196+0000 7f86a2000700 1 --2- 172.21.15.63:0/991806677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 0x5561534b8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:57.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.197+0000 7f86a2a00700 1 --2- 172.21.15.63:0/991806677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9dc0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f869400f400 tx=0x7f869400ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212a0dd51b3ca3e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:57.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.197+0000 7f868f400700 1 -- 172.21.15.63:0/991806677 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694026070 con 0x556153339e40 2024-05-27T07:40:57.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.197+0000 7f868f400700 1 -- 172.21.15.63:0/991806677 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86940226a0 con 0x556153339e40 2024-05-27T07:40:57.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.197+0000 7f868f400700 1 -- 172.21.15.63:0/991806677 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86940186a0 con 0x556153339e40 2024-05-27T07:40:57.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.198+0000 7f86b837fb80 1 -- 172.21.15.63:0/991806677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 msgr2=0x5561534b9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:57.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.198+0000 7f86b837fb80 1 --2- 172.21.15.63:0/991806677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9dc0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f869400f400 tx=0x7f869400ac20 comp rx=0 tx=0).stop 2024-05-27T07:40:57.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.199+0000 7f86b837fb80 1 -- 172.21.15.63:0/991806677 shutdown_connections 2024-05-27T07:40:57.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.199+0000 7f86b837fb80 1 --2- 172.21.15.63:0/991806677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534c32c0 0x5561534da530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.199+0000 7f86b837fb80 1 --2- 172.21.15.63:0/991806677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534b9dc0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.199+0000 7f86b837fb80 1 --2- 172.21.15.63:0/991806677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534259f0 0x5561534b8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.199+0000 7f86b837fb80 1 -- 172.21.15.63:0/991806677 >> 172.21.15.63:0/991806677 conn(0x5561533d3390 msgr2=0x5561534b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:57.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.199+0000 7f86b837fb80 1 -- 172.21.15.63:0/991806677 shutdown_connections 2024-05-27T07:40:57.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.199+0000 7f86b837fb80 1 -- 172.21.15.63:0/991806677 wait complete. 2024-05-27T07:40:57.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.199+0000 7f86b837fb80 1 -- start start 2024-05-27T07:40:57.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.200+0000 7f86b837fb80 1 Processor -- start 2024-05-27T07:40:57.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.200+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.200+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534259f0 0x5561534c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.200+0000 7f86b837fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534c32c0 0x5561534c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.200+0000 7f86b837fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b0ac0 con 0x556153339e40 2024-05-27T07:40:57.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.200+0000 7f86b837fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b1f30 con 0x5561534259f0 2024-05-27T07:40:57.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.200+0000 7f86b837fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b1190 con 0x5561534c32c0 2024-05-27T07:40:57.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.200+0000 7f86b837fb80 10 client.0 _ll_register_callbacks cb 0x556153380c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:40:57.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.201+0000 7f86a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.201+0000 7f86a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34110/0 (socket says 172.21.15.63:34110) 2024-05-27T07:40:57.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.201+0000 7f86a2a00700 1 -- 172.21.15.63:0/2763165418 learned_addr learned my addr 172.21.15.63:0/2763165418 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:40:57.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.201+0000 7f86a2000700 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534259f0 0x5561534c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.201+0000 7f86a3400700 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534c32c0 0x5561534c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.201+0000 7f86a2a00700 1 -- 172.21.15.63:0/2763165418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534c32c0 msgr2=0x5561534c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.201+0000 7f86a2a00700 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534c32c0 0x5561534c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.201+0000 7f86a2a00700 1 -- 172.21.15.63:0/2763165418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534259f0 msgr2=0x5561534c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:40:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.201+0000 7f86a2a00700 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534259f0 0x5561534c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.201+0000 7f86a2a00700 1 -- 172.21.15.63:0/2763165418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561534c0f80 con 0x556153339e40 2024-05-27T07:40:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.202+0000 7f86a2a00700 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534cb900 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f869400a820 tx=0x7f8694005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7063cdcb8f2a5ada server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.202+0000 7f868cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:40:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.202+0000 7f868cc00700 1 -- 172.21.15.63:0/2763165418 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694026040 con 0x556153339e40 2024-05-27T07:40:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.202+0000 7f868cc00700 1 -- 172.21.15.63:0/2763165418 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9483+0+0 (secure 0 0 0) 0x7f8694022af0 con 0x556153339e40 2024-05-27T07:40:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.202+0000 7f86b837fb80 1 -- 172.21.15.63:0/2763165418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86940226a0 con 0x556153339e40 2024-05-27T07:40:57.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f868cc00700 1 -- 172.21.15.63:0/2763165418 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8694022cf0 con 0x556153339e40 2024-05-27T07:40:57.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f86b837fb80 10 client.16860 fetch_fsmap learned FSMap version 33 2024-05-27T07:40:57.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f86b837fb80 1 -- 172.21.15.63:0/2763165418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55615353d1d0 con 0x556153339e40 2024-05-27T07:40:57.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f868cc00700 1 -- 172.21.15.63:0/2763165418 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86940226a0 con 0x556153339e40 2024-05-27T07:40:57.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f868cc00700 1 -- 172.21.15.63:0/2763165418 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8694036440 con 0x556153339e40 2024-05-27T07:40:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f86b837fb80 10 client.16860 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:40:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f86b837fb80 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:40:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f86b837fb80 20 client.16860 populate_metadata read hostname 'smithi063' 2024-05-27T07:40:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f86b837fb80 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55615353d7c0 0x55615353fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f86b837fb80 4 client.16860 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:40:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.203+0000 7f86b837fb80 1 -- 172.21.15.63:0/2763165418 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561533e2570 con 0x55615353d7c0 2024-05-27T07:40:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.204+0000 7f86a2000700 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55615353d7c0 0x55615353fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:57.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.205+0000 7f86a2000700 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55615353d7c0 0x55615353fbe0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=174904b918866203 server_cookie=1940c8037313f711 in_seq=0 out_seq=0 2024-05-27T07:40:57.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.205+0000 7f868cc00700 10 client.16860 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:57.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.205+0000 7f868cc00700 1 -- 172.21.15.63:0/2763165418 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5561533e2570 con 0x55615353d7c0 2024-05-27T07:40:57.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.205+0000 7f868cc00700 10 client.16860 handle_command_reply: tid=0 2024-05-27T07:40:57.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.205+0000 7f86b837fb80 1 client.16860 shutdown 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 1 -- 172.21.15.63:0/2763165418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 msgr2=0x5561534cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534cb900 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f869400a820 tx=0x7f8694005c20 comp rx=0 tx=0).stop 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 1 -- 172.21.15.63:0/2763165418 shutdown_connections 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55615353d7c0 0x55615353fbe0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561534c32c0 0x5561534c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556153339e40 0x5561534cb900 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 1 --2- 172.21.15.63:0/2763165418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561534259f0 0x5561534c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 1 -- 172.21.15.63:0/2763165418 >> 172.21.15.63:0/2763165418 conn(0x5561533d3390 msgr2=0x5561534bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f868cc00700 0 client.16860 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 1 -- 172.21.15.63:0/2763165418 shutdown_connections 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 1 -- 172.21.15.63:0/2763165418 wait complete. 2024-05-27T07:40:57.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.206+0000 7f86b837fb80 20 client.16860 trim_cache size 0 max 16384 2024-05-27T07:40:57.206 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:40:57.206 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:40:57.206 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:40:57.206 INFO:teuthology.orchestra.run.smithi063.stdout: "5354509e-c894-4d87-9340-fa3c68884592": { 2024-05-27T07:40:57.206 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:40:57.206 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "5354509e-c894-4d87-9340-fa3c68884592", 2024-05-27T07:40:57.206 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:40:57.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:40:57.207 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:40:57.207 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:40:57.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.207+0000 7f86a3e00700 1 -- 172.21.15.63:0/4438446 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86880b2a70 msgr2=0x7f86880b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:57.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.207+0000 7f86a3e00700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86880b2a70 0x7f86880b4f80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f869c009e60 tx=0x7f869c0099e0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.207+0000 7f86a3e00700 1 -- 172.21.15.63:0/4438446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86a410af60 msgr2=0x7f86a41a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:57.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.207+0000 7f86a3e00700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86a410af60 0x7f86a41a2df0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f869000d950 tx=0x7f869000dc60 comp rx=0 tx=0).stop 2024-05-27T07:40:57.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.207+0000 7f86a3e00700 1 -- 172.21.15.63:0/4438446 shutdown_connections 2024-05-27T07:40:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.207+0000 7f86a3e00700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86880b2a70 0x7f86880b4f80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.207+0000 7f86a3e00700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86a410b8b0 0x7f86a419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.207+0000 7f86a3e00700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a4106630 0x7f86a41a2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.207+0000 7f86a3e00700 1 --2- 172.21.15.63:0/4438446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86a410af60 0x7f86a41a2df0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.207+0000 7f86a3e00700 1 -- 172.21.15.63:0/4438446 >> 172.21.15.63:0/4438446 conn(0x7f86a4101ec0 msgr2=0x7f86a410a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.208+0000 7f86a3e00700 1 -- 172.21.15.63:0/4438446 shutdown_connections 2024-05-27T07:40:57.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:40:57.208+0000 7f86a3e00700 1 -- 172.21.15.63:0/4438446 wait complete. 2024-05-27T07:40:57.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:57 smithi063 ceph-mon[31613]: pgmap v226: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:40:57.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:57 smithi086 ceph-mon[34687]: pgmap v226: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:40:57.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:57 smithi149 ceph-mon[35408]: pgmap v226: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:40:58.103 INFO:tasks.cephfs.filesystem:scrub status for tag:5354509e-c894-4d87-9340-fa3c68884592 - {'path': '/', 'tag': '5354509e-c894-4d87-9340-fa3c68884592', 'options': 'recursive,force'} 2024-05-27T07:40:59.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:40:59 smithi086 ceph-mon[34687]: pgmap v227: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.4 MiB/s wr, 39 op/s 2024-05-27T07:40:59.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:40:59 smithi149 ceph-mon[35408]: pgmap v227: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.4 MiB/s wr, 39 op/s 2024-05-27T07:40:59.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:40:59 smithi063 ceph-mon[31613]: pgmap v227: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.4 MiB/s wr, 39 op/s 2024-05-27T07:41:01.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:01 smithi086 ceph-mon[34687]: pgmap v228: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.0 MiB/s wr, 35 op/s 2024-05-27T07:41:01.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:01 smithi149 ceph-mon[35408]: pgmap v228: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.0 MiB/s wr, 35 op/s 2024-05-27T07:41:01.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:01 smithi063 ceph-mon[31613]: pgmap v228: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.0 MiB/s wr, 35 op/s 2024-05-27T07:41:02.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:41:02.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:41:02.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:41:02.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:02 smithi086 ceph-mon[34687]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T07:41:02.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:41:02.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:41:02.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:41:02.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:02 smithi149 ceph-mon[35408]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T07:41:02.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:41:02.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:41:02.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:41:02.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:02 smithi063 ceph-mon[31613]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T07:41:03.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:03 smithi086 ceph-mon[34687]: pgmap v230: 129 pgs: 14 active+clean+snaptrim_wait, 114 active+clean, 1 active+clean+snaptrim; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.9 MiB/s wr, 63 op/s 2024-05-27T07:41:03.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:03 smithi149 ceph-mon[35408]: pgmap v230: 129 pgs: 14 active+clean+snaptrim_wait, 114 active+clean, 1 active+clean+snaptrim; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.9 MiB/s wr, 63 op/s 2024-05-27T07:41:03.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:03 smithi063 ceph-mon[31613]: pgmap v230: 129 pgs: 14 active+clean+snaptrim_wait, 114 active+clean, 1 active+clean+snaptrim; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.9 MiB/s wr, 63 op/s 2024-05-27T07:41:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.436+0000 7f4b07000640 1 -- 172.21.15.63:0/3217495118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b00109ad0 msgr2=0x7f4b00103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.436+0000 7f4b07000640 1 --2- 172.21.15.63:0/3217495118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b00109ad0 0x7f4b00103670 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009900 tx=0x7f4af402a410 comp rx=0 tx=0).stop 2024-05-27T07:41:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.436+0000 7f4b07000640 1 -- 172.21.15.63:0/3217495118 shutdown_connections 2024-05-27T07:41:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.436+0000 7f4b07000640 1 --2- 172.21.15.63:0/3217495118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b00104220 0x7f4b001105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.436+0000 7f4b07000640 1 --2- 172.21.15.63:0/3217495118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b00109f70 0x7f4b00103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.436+0000 7f4b07000640 1 --2- 172.21.15.63:0/3217495118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b00109ad0 0x7f4b00103670 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.436+0000 7f4b07000640 1 -- 172.21.15.63:0/3217495118 >> 172.21.15.63:0/3217495118 conn(0x7f4b0007a1b0 msgr2=0x7f4b0007a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.436+0000 7f4b07000640 1 -- 172.21.15.63:0/3217495118 shutdown_connections 2024-05-27T07:41:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.436+0000 7f4b07000640 1 -- 172.21.15.63:0/3217495118 wait complete. 2024-05-27T07:41:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.438+0000 7f4b07000640 1 Processor -- start 2024-05-27T07:41:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.438+0000 7f4b07000640 1 -- start start 2024-05-27T07:41:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.438+0000 7f4b07000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b00104220 0x7f4b0019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.438+0000 7f4b07000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b00109ad0 0x7f4b0019d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.438+0000 7f4b07000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b00109f70 0x7f4b001a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.438+0000 7f4b07000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00113b70 con 0x7f4b00104220 2024-05-27T07:41:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.438+0000 7f4b07000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001139f0 con 0x7f4b00109ad0 2024-05-27T07:41:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.438+0000 7f4b07000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00113cf0 con 0x7f4b00109f70 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b05c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b00104220 0x7f4b0019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b05c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b00104220 0x7f4b0019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40510/0 (socket says 172.21.15.63:40510) 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b05c00640 1 -- 172.21.15.63:0/1640042968 learned_addr learned my addr 172.21.15.63:0/1640042968 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b05200640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b00109ad0 0x7f4b0019d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b06600640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b00109f70 0x7f4b001a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b05c00640 1 -- 172.21.15.63:0/1640042968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b00109f70 msgr2=0x7f4b001a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b05c00640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b00109f70 0x7f4b001a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b05c00640 1 -- 172.21.15.63:0/1640042968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b00109ad0 msgr2=0x7f4b0019d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b05c00640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b00109ad0 0x7f4b0019d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b05c00640 1 -- 172.21.15.63:0/1640042968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b001a8a10 con 0x7f4b00104220 2024-05-27T07:41:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.439+0000 7f4b06600640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b00109f70 0x7f4b001a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:41:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.440+0000 7f4b05c00640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b00104220 0x7f4b0019d450 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4af4005940 tx=0x7f4af40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.440+0000 7f4afe000640 1 -- 172.21.15.63:0/1640042968 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af403f030 con 0x7f4b00104220 2024-05-27T07:41:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.440+0000 7f4afe000640 1 -- 172.21.15.63:0/1640042968 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4af4003c90 con 0x7f4b00104220 2024-05-27T07:41:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.440+0000 7f4b07000640 1 -- 172.21.15.63:0/1640042968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b001a8be0 con 0x7f4b00104220 2024-05-27T07:41:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.440+0000 7f4afe000640 1 -- 172.21.15.63:0/1640042968 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4035070 con 0x7f4b00104220 2024-05-27T07:41:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.440+0000 7f4b07000640 1 -- 172.21.15.63:0/1640042968 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b001a9180 con 0x7f4b00104220 2024-05-27T07:41:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.445+0000 7f4b0f5ae740 1 Processor -- start 2024-05-27T07:41:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.445+0000 7f4b0f5ae740 1 -- start start 2024-05-27T07:41:05.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.446+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7acda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.446+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd77022f0 0x561bd7acdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.446+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7ace4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.446+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561bd7a3eb90 con 0x561bd772b1b0 2024-05-27T07:41:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.446+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bd7a2e730 con 0x561bd770a4d0 2024-05-27T07:41:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.446+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561bd7a32770 con 0x561bd77022f0 2024-05-27T07:41:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.446+0000 7f4afe000640 1 -- 172.21.15.63:0/1640042968 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4af4042030 con 0x7f4b00104220 2024-05-27T07:41:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.446+0000 7f4b06600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7acda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.446+0000 7f4b06600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7acda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40516/0 (socket says 172.21.15.63:40516) 2024-05-27T07:41:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.446+0000 7f4b06600640 1 -- 172.21.15.63:0/808247885 learned_addr learned my addr 172.21.15.63:0/808247885 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.447+0000 7f4b05200640 1 --2- 172.21.15.63:0/808247885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7ace4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.447+0000 7f4b05c00640 1 --2- 172.21.15.63:0/808247885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd77022f0 0x561bd7acdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.447+0000 7f4b06600640 1 -- 172.21.15.63:0/808247885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd77022f0 msgr2=0x561bd7acdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.447+0000 7f4b06600640 1 --2- 172.21.15.63:0/808247885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd77022f0 0x561bd7acdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.447+0000 7f4b06600640 1 -- 172.21.15.63:0/808247885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 msgr2=0x561bd7ace4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.447+0000 7f4b06600640 1 --2- 172.21.15.63:0/808247885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7ace4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.447+0000 7f4b06600640 1 -- 172.21.15.63:0/808247885 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd79c78b0 con 0x561bd772b1b0 2024-05-27T07:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.447+0000 7f4b05200640 1 --2- 172.21.15.63:0/808247885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7ace4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.448+0000 7f4b06600640 1 --2- 172.21.15.63:0/808247885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7acda60 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009ea0 tx=0x7f4af8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9470ea18b072a5a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.448+0000 7f4af2a00640 1 -- 172.21.15.63:0/808247885 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8004d20 con 0x561bd772b1b0 2024-05-27T07:41:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.448+0000 7f4af2a00640 1 -- 172.21.15.63:0/808247885 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4af80054b0 con 0x561bd772b1b0 2024-05-27T07:41:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.448+0000 7f4af2a00640 1 -- 172.21.15.63:0/808247885 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8011710 con 0x561bd772b1b0 2024-05-27T07:41:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.448+0000 7f4afe000640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ae80b29d0 0x7f4ae80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.448+0000 7f4afe000640 1 -- 172.21.15.63:0/1640042968 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9571+0+0 (secure 0 0 0) 0x7f4af4102fc0 con 0x7f4b00104220 2024-05-27T07:41:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.448+0000 7f4b05c00640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ae80b29d0 0x7f4ae80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.449+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/808247885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 msgr2=0x561bd7acda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.449+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/808247885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7acda60 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009ea0 tx=0x7f4af8009f90 comp rx=0 tx=0).stop 2024-05-27T07:41:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.449+0000 7f4b05c00640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ae80b29d0 0x7f4ae80b4ee0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x561bd7acb550 tx=0x7f4af402a410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.449+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/808247885 shutdown_connections 2024-05-27T07:41:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.449+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/808247885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7ace4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.449+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/808247885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd77022f0 0x561bd7acdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.449+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/808247885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7acda60 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.449+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/808247885 >> 172.21.15.63:0/808247885 conn(0x561bd771f300 msgr2=0x561bd7aca210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.449+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/808247885 shutdown_connections 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.449+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/808247885 wait complete. 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.450+0000 7f4b0f5ae740 1 -- start start 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.450+0000 7f4b0f5ae740 1 Processor -- start 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.451+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.451+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7ad8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.451+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd772b1b0 0x561bd7ae3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.451+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561bd7714790 con 0x561bd77022f0 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.451+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bd79d6bc0 con 0x561bd770a4d0 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.451+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561bd7adab90 con 0x561bd772b1b0 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.451+0000 7f4b0f5ae740 10 client.0 _ll_register_callbacks cb 0x561bd79b4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.451+0000 7f4b06600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.451+0000 7f4b06600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40518/0 (socket says 172.21.15.63:40518) 2024-05-27T07:41:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.451+0000 7f4b06600640 1 -- 172.21.15.63:0/3872501095 learned_addr learned my addr 172.21.15.63:0/3872501095 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.452+0000 7f4b05200640 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7ad8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.452+0000 7f4b06600640 1 -- 172.21.15.63:0/3872501095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd772b1b0 msgr2=0x561bd7ae3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.452+0000 7f4b06600640 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd772b1b0 0x561bd7ae3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.452+0000 7f4b06600640 1 -- 172.21.15.63:0/3872501095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 msgr2=0x561bd7ad8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.452+0000 7f4b06600640 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7ad8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.452+0000 7f4b06600640 1 -- 172.21.15.63:0/3872501095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd7743210 con 0x561bd77022f0 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.452+0000 7f4b06600640 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad7c30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4af80093e0 tx=0x7f4af80056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cd6ceed8222d3d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.452+0000 7f4ae7400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.452+0000 7f4ae7400640 1 -- 172.21.15.63:0/3872501095 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8004be0 con 0x561bd77022f0 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.452+0000 7f4ae7400640 1 -- 172.21.15.63:0/3872501095 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9571+0+0 (secure 0 0 0) 0x7f4af8004d80 con 0x561bd77022f0 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.453+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3872501095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4af80054b0 con 0x561bd77022f0 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.453+0000 7f4ae7400640 1 -- 172.21.15.63:0/3872501095 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4af8030050 con 0x561bd77022f0 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.453+0000 7f4b0f5ae740 10 client.16872 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.453+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3872501095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561bd780f620 con 0x561bd77022f0 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.453+0000 7f4ae7400640 1 -- 172.21.15.63:0/3872501095 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4af80054b0 con 0x561bd77022f0 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.454+0000 7f4ae7400640 1 -- 172.21.15.63:0/3872501095 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4af8027500 con 0x561bd77022f0 2024-05-27T07:41:05.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.454+0000 7f4b0f5ae740 10 client.16872 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.454+0000 7f4b0f5ae740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.454+0000 7f4b0f5ae740 20 client.16872 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.454+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bd76e9d80 0x561bd7ae9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.454+0000 7f4b0f5ae740 4 client.16872 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.454+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3872501095 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd7a40ae0 con 0x561bd76e9d80 2024-05-27T07:41:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.454+0000 7f4b05200640 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bd76e9d80 0x561bd7ae9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.455+0000 7f4b05200640 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bd76e9d80 0x561bd7ae9df0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa3f72ccce23e96 server_cookie=fc3d0e1727dfee2a in_seq=0 out_seq=0 2024-05-27T07:41:05.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.455+0000 7f4ae7400640 10 client.16872 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.457+0000 7f4ae7400640 1 -- 172.21.15.63:0/3872501095 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561bd7a40ae0 con 0x561bd76e9d80 2024-05-27T07:41:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.457+0000 7f4ae7400640 10 client.16872 handle_command_reply: tid=0 2024-05-27T07:41:05.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.457+0000 7f4b0f5ae740 1 client.16872 shutdown 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3872501095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 msgr2=0x561bd7ad7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad7c30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4af80093e0 tx=0x7f4af80056e0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3872501095 shutdown_connections 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bd76e9d80 0x561bd7ae9df0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd772b1b0 0x561bd7ae3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7ad8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3872501095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad7c30 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3872501095 >> 172.21.15.63:0/3872501095 conn(0x561bd771f300 msgr2=0x561bd7acd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4ae7400640 0 client.16872 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3872501095 shutdown_connections 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3872501095 wait complete. 2024-05-27T07:41:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.458+0000 7f4b0f5ae740 20 client.16872 trim_cache size 0 max 16384 2024-05-27T07:41:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.478+0000 7f4b0f5ae740 1 Processor -- start 2024-05-27T07:41:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.479+0000 7f4b0f5ae740 1 -- start start 2024-05-27T07:41:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.479+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.479+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd770a4d0 0x561bd7ae3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.479+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd772b1b0 0x561bd7ad4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.479+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001b5c40 con 0x561bd77022f0 2024-05-27T07:41:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.479+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001b3510 con 0x561bd772b1b0 2024-05-27T07:41:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.479+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001b1380 con 0x561bd770a4d0 2024-05-27T07:41:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.479+0000 7f4b06600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.479+0000 7f4b06600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40520/0 (socket says 172.21.15.63:40520) 2024-05-27T07:41:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.479+0000 7f4b06600640 1 -- 172.21.15.63:0/68807000 learned_addr learned my addr 172.21.15.63:0/68807000 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.480+0000 7f4b05200640 1 --2- 172.21.15.63:0/68807000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd770a4d0 0x561bd7ae3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.480+0000 7f4b06600640 1 --2- 172.21.15.63:0/68807000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd772b1b0 0x561bd7ad4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.480+0000 7f4b06600640 1 -- 172.21.15.63:0/68807000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd770a4d0 msgr2=0x561bd7ae3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.480+0000 7f4b06600640 1 --2- 172.21.15.63:0/68807000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd770a4d0 0x561bd7ae3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.480+0000 7f4b06600640 1 -- 172.21.15.63:0/68807000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd772b1b0 msgr2=0x561bd7ad4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.480+0000 7f4b06600640 1 --2- 172.21.15.63:0/68807000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd772b1b0 0x561bd7ad4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.480+0000 7f4b06600640 1 -- 172.21.15.63:0/68807000 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd76e9fb0 con 0x561bd77022f0 2024-05-27T07:41:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.480+0000 7f4b06600640 1 --2- 172.21.15.63:0/68807000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad33e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4af8005cc0 tx=0x7f4af80099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97dee5e8f6514de server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.481+0000 7f4af2000640 1 -- 172.21.15.63:0/68807000 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800fbc0 con 0x561bd77022f0 2024-05-27T07:41:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.481+0000 7f4af2000640 1 -- 172.21.15.63:0/68807000 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4af8022070 con 0x561bd77022f0 2024-05-27T07:41:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.481+0000 7f4af2000640 1 -- 172.21.15.63:0/68807000 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af80276b0 con 0x561bd77022f0 2024-05-27T07:41:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.482+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/68807000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 msgr2=0x561bd7ad33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.482+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/68807000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad33e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4af8005cc0 tx=0x7f4af80099b0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.482+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/68807000 shutdown_connections 2024-05-27T07:41:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.482+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/68807000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd772b1b0 0x561bd7ad4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.482+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/68807000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd770a4d0 0x561bd7ae3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.482+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/68807000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd77022f0 0x561bd7ad33e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.482+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/68807000 >> 172.21.15.63:0/68807000 conn(0x561bd771f300 msgr2=0x561bd7ad7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.482+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/68807000 shutdown_connections 2024-05-27T07:41:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.482+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/68807000 wait complete. 2024-05-27T07:41:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.483+0000 7f4b0f5ae740 1 -- start start 2024-05-27T07:41:05.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.483+0000 7f4b0f5ae740 1 Processor -- start 2024-05-27T07:41:05.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.483+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd77022f0 0x561bd7aebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.483+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7aec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.483+0000 7f4b0f5ae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7b50240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.483+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001b19d0 con 0x561bd772b1b0 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.483+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001b2e40 con 0x561bd770a4d0 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.484+0000 7f4b0f5ae740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b001b20a0 con 0x561bd77022f0 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.484+0000 7f4b0f5ae740 10 client.0 _ll_register_callbacks cb 0x7f4af8009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.484+0000 7f4b06600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7b50240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.484+0000 7f4b06600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7b50240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40526/0 (socket says 172.21.15.63:40526) 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.484+0000 7f4b06600640 1 -- 172.21.15.63:0/3523554549 learned_addr learned my addr 172.21.15.63:0/3523554549 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.484+0000 7f4b05200640 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7aec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.484+0000 7f4b06600640 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd77022f0 0x561bd7aebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.485+0000 7f4b06600640 1 -- 172.21.15.63:0/3523554549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd77022f0 msgr2=0x561bd7aebf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.485+0000 7f4b06600640 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd77022f0 0x561bd7aebf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.485+0000 7f4b06600640 1 -- 172.21.15.63:0/3523554549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 msgr2=0x561bd7aec4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.485+0000 7f4b06600640 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7aec4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.485+0000 7f4b06600640 1 -- 172.21.15.63:0/3523554549 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd7aecd50 con 0x561bd772b1b0 2024-05-27T07:41:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.485+0000 7f4b06600640 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7b50240 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4af8010010 tx=0x7f4af801b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8972bb09710aca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.485+0000 7f4ae7400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.485+0000 7f4ae7400640 1 -- 172.21.15.63:0/3523554549 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af80055a0 con 0x561bd772b1b0 2024-05-27T07:41:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.485+0000 7f4ae7400640 1 -- 172.21.15.63:0/3523554549 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9571+0+0 (secure 0 0 0) 0x7f4af8027470 con 0x561bd772b1b0 2024-05-27T07:41:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.485+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3523554549 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4af8022070 con 0x561bd772b1b0 2024-05-27T07:41:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.486+0000 7f4ae7400640 1 -- 172.21.15.63:0/3523554549 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4af8027670 con 0x561bd772b1b0 2024-05-27T07:41:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.486+0000 7f4b0f5ae740 10 client.16878 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.486+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3523554549 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561bd7b517e0 con 0x561bd772b1b0 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.486+0000 7f4ae7400640 1 -- 172.21.15.63:0/3523554549 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4af8022070 con 0x561bd772b1b0 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.486+0000 7f4ae7400640 1 -- 172.21.15.63:0/3523554549 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4af8040040 con 0x561bd772b1b0 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.486+0000 7f4b0f5ae740 10 client.16878 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.486+0000 7f4b0f5ae740 10 client.16878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.486+0000 7f4b0f5ae740 20 client.16878 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.487+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bd76bf1c0 0x561bd7b53e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.487+0000 7f4b0f5ae740 4 client.16878 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.487+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3523554549 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bd7a3a2c0 con 0x561bd76bf1c0 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.487+0000 7f4b05200640 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bd76bf1c0 0x561bd7b53e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.488+0000 7f4b05200640 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bd76bf1c0 0x561bd7b53e80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=823035dfcd834230 server_cookie=a8f3113d381769e in_seq=0 out_seq=0 2024-05-27T07:41:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.488+0000 7f4ae7400640 10 client.16878 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4ae7400640 1 -- 172.21.15.63:0/3523554549 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561bd7a3a2c0 con 0x561bd76bf1c0 2024-05-27T07:41:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4ae7400640 10 client.16878 handle_command_reply: tid=0 2024-05-27T07:41:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 client.16878 shutdown 2024-05-27T07:41:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3523554549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 msgr2=0x561bd7b50240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7b50240 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4af8010010 tx=0x7f4af801b040 comp rx=0 tx=0).stop 2024-05-27T07:41:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3523554549 shutdown_connections 2024-05-27T07:41:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bd76bf1c0 0x561bd7b53e80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bd772b1b0 0x561bd7b50240 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bd770a4d0 0x561bd7aec4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 --2- 172.21.15.63:0/3523554549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bd77022f0 0x561bd7aebf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3523554549 >> 172.21.15.63:0/3523554549 conn(0x561bd771f300 msgr2=0x561bd7ad7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4ae7400640 0 client.16878 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3523554549 shutdown_connections 2024-05-27T07:41:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.524+0000 7f4b0f5ae740 1 -- 172.21.15.63:0/3523554549 wait complete. 2024-05-27T07:41:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.525+0000 7f4b0f5ae740 20 client.16878 trim_cache size 0 max 16384 2024-05-27T07:41:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:41:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:41:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:41:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.525+0000 7f4b07000640 1 -- 172.21.15.63:0/1640042968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ae80b29d0 msgr2=0x7f4ae80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:05.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.525+0000 7f4b07000640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ae80b29d0 0x7f4ae80b4ee0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x561bd7acb550 tx=0x7f4af402a410 comp rx=0 tx=0).stop 2024-05-27T07:41:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.525+0000 7f4b07000640 1 -- 172.21.15.63:0/1640042968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b00104220 msgr2=0x7f4b0019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.525+0000 7f4b07000640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b00104220 0x7f4b0019d450 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4af4005940 tx=0x7f4af40042f0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.526+0000 7f4b07000640 1 -- 172.21.15.63:0/1640042968 shutdown_connections 2024-05-27T07:41:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.526+0000 7f4b07000640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ae80b29d0 0x7f4ae80b4ee0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.526+0000 7f4b07000640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b00109f70 0x7f4b001a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.526+0000 7f4b07000640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b00109ad0 0x7f4b0019d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.526+0000 7f4b07000640 1 --2- 172.21.15.63:0/1640042968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b00104220 0x7f4b0019d450 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.526+0000 7f4b07000640 1 -- 172.21.15.63:0/1640042968 >> 172.21.15.63:0/1640042968 conn(0x7f4b0007a1b0 msgr2=0x7f4b001021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.526+0000 7f4b07000640 1 -- 172.21.15.63:0/1640042968 shutdown_connections 2024-05-27T07:41:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:05.526+0000 7f4b07000640 1 -- 172.21.15.63:0/1640042968 wait complete. 2024-05-27T07:41:05.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:05 smithi086 ceph-mon[34687]: pgmap v231: 129 pgs: 14 active+clean+snaptrim_wait, 114 active+clean, 1 active+clean+snaptrim; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.9 MiB/s wr, 63 op/s 2024-05-27T07:41:05.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:05 smithi149 ceph-mon[35408]: pgmap v231: 129 pgs: 14 active+clean+snaptrim_wait, 114 active+clean, 1 active+clean+snaptrim; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.9 MiB/s wr, 63 op/s 2024-05-27T07:41:05.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:05 smithi063 ceph-mon[31613]: pgmap v231: 129 pgs: 14 active+clean+snaptrim_wait, 114 active+clean, 1 active+clean+snaptrim; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.9 MiB/s wr, 63 op/s 2024-05-27T07:41:07.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:07 smithi086 ceph-mon[34687]: pgmap v232: 129 pgs: 14 active+clean+snaptrim_wait, 114 active+clean, 1 active+clean+snaptrim; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.9 MiB/s wr, 63 op/s 2024-05-27T07:41:07.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:07 smithi149 ceph-mon[35408]: pgmap v232: 129 pgs: 14 active+clean+snaptrim_wait, 114 active+clean, 1 active+clean+snaptrim; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.9 MiB/s wr, 63 op/s 2024-05-27T07:41:07.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:07 smithi063 ceph-mon[31613]: pgmap v232: 129 pgs: 14 active+clean+snaptrim_wait, 114 active+clean, 1 active+clean+snaptrim; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.9 MiB/s wr, 63 op/s 2024-05-27T07:41:09.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:09 smithi086 ceph-mon[34687]: pgmap v233: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 33 op/s 2024-05-27T07:41:09.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:09 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:41:09.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:09 smithi149 ceph-mon[35408]: pgmap v233: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 33 op/s 2024-05-27T07:41:09.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:09 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:41:09.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:09 smithi063 ceph-mon[31613]: pgmap v233: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 33 op/s 2024-05-27T07:41:09.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:09 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:41:11.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:11 smithi086 ceph-mon[34687]: pgmap v234: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 33 op/s 2024-05-27T07:41:11.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:11 smithi086 ceph-mon[34687]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T07:41:11.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:11 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:41:11.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:11 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:41:11.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:11 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:41:11.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:11 smithi149 ceph-mon[35408]: pgmap v234: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 33 op/s 2024-05-27T07:41:11.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:11 smithi149 ceph-mon[35408]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T07:41:11.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:11 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:41:11.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:11 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:41:11.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:11 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:41:11.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:11 smithi063 ceph-mon[31613]: pgmap v234: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 33 op/s 2024-05-27T07:41:11.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:11 smithi063 ceph-mon[31613]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T07:41:11.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:11 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:41:11.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:11 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:41:11.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:11 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:41:13.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:13 smithi086 ceph-mon[34687]: pgmap v236: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:41:13.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:13 smithi149 ceph-mon[35408]: pgmap v236: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:41:13.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:13 smithi063 ceph-mon[31613]: pgmap v236: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:41:15.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:15 smithi086 ceph-mon[34687]: pgmap v237: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:41:15.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:15 smithi149 ceph-mon[35408]: pgmap v237: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:41:15.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:15 smithi063 ceph-mon[31613]: pgmap v237: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:41:17.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:17 smithi086 ceph-mon[34687]: pgmap v238: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:41:17.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:17 smithi149 ceph-mon[35408]: pgmap v238: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:41:17.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:17 smithi063 ceph-mon[31613]: pgmap v238: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:41:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.773+0000 7f4826e00640 1 -- 172.21.15.63:0/2498004526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482010c180 msgr2=0x7f4820105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.773+0000 7f4826e00640 1 --2- 172.21.15.63:0/2498004526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482010c180 0x7f4820105dc0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f480c009900 tx=0x7f480c02a410 comp rx=0 tx=0).stop 2024-05-27T07:41:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.773+0000 7f4826e00640 1 -- 172.21.15.63:0/2498004526 shutdown_connections 2024-05-27T07:41:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.773+0000 7f4826e00640 1 --2- 172.21.15.63:0/2498004526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4820106430 0x7f4820112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.773+0000 7f4826e00640 1 --2- 172.21.15.63:0/2498004526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f482010c180 0x7f4820105dc0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.773+0000 7f4826e00640 1 --2- 172.21.15.63:0/2498004526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f482010bce0 0x7f4820105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.773+0000 7f4826e00640 1 -- 172.21.15.63:0/2498004526 >> 172.21.15.63:0/2498004526 conn(0x7f4820101460 msgr2=0x7f4820103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.774+0000 7f4826e00640 1 -- 172.21.15.63:0/2498004526 shutdown_connections 2024-05-27T07:41:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.774+0000 7f4826e00640 1 -- 172.21.15.63:0/2498004526 wait complete. 2024-05-27T07:41:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.775+0000 7f4826e00640 1 Processor -- start 2024-05-27T07:41:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.775+0000 7f4826e00640 1 -- start start 2024-05-27T07:41:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.776+0000 7f4826e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4820106430 0x7f48201a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.776+0000 7f4826e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f482010bce0 0x7f48201a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.776+0000 7f4826e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f482010c180 0x7f482019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.776+0000 7f4826e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4820115dd0 con 0x7f4820106430 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.776+0000 7f4826e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4820115c50 con 0x7f482010bce0 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.776+0000 7f4826e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4820115f50 con 0x7f482010c180 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.776+0000 7f481fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4820106430 0x7f48201a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.776+0000 7f481fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4820106430 0x7f48201a2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34532/0 (socket says 172.21.15.63:34532) 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.776+0000 7f481fe00640 1 -- 172.21.15.63:0/2217154186 learned_addr learned my addr 172.21.15.63:0/2217154186 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.777+0000 7f481f400640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f482010bce0 0x7f48201a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.777+0000 7f4824e00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f482010c180 0x7f482019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.777+0000 7f481fe00640 1 -- 172.21.15.63:0/2217154186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f482010c180 msgr2=0x7f482019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.777+0000 7f481fe00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f482010c180 0x7f482019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.777+0000 7f481fe00640 1 -- 172.21.15.63:0/2217154186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f482010bce0 msgr2=0x7f48201a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.777+0000 7f481fe00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f482010bce0 0x7f48201a2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.777+0000 7f481fe00640 1 -- 172.21.15.63:0/2217154186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482019cde0 con 0x7f4820106430 2024-05-27T07:41:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.777+0000 7f4824e00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f482010c180 0x7f482019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.777+0000 7f481fe00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4820106430 0x7f48201a2530 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f481000d740 tx=0x7f481000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.778+0000 7f481cc00640 1 -- 172.21.15.63:0/2217154186 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810017070 con 0x7f4820106430 2024-05-27T07:41:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.778+0000 7f481cc00640 1 -- 172.21.15.63:0/2217154186 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f481000de50 con 0x7f4820106430 2024-05-27T07:41:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.778+0000 7f481cc00640 1 -- 172.21.15.63:0/2217154186 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810004c60 con 0x7f4820106430 2024-05-27T07:41:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.778+0000 7f4826e00640 1 -- 172.21.15.63:0/2217154186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482019d070 con 0x7f4820106430 2024-05-27T07:41:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.778+0000 7f4826e00640 1 -- 172.21.15.63:0/2217154186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48201a92d0 con 0x7f4820106430 2024-05-27T07:41:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.782+0000 7f482dde1740 1 Processor -- start 2024-05-27T07:41:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.782+0000 7f482dde1740 1 -- start start 2024-05-27T07:41:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.782+0000 7f482dde1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 0x55e348d5fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.782+0000 7f482dde1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489942f0 0x55e348d5ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.782+0000 7f482dde1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.782+0000 7f482dde1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e348cd0b90 con 0x55e34899c4d0 2024-05-27T07:41:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.782+0000 7f482dde1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e348cc0730 con 0x55e3489bd1b0 2024-05-27T07:41:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.782+0000 7f482dde1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e348cc4770 con 0x55e3489942f0 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f481f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f481f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34536/0 (socket says 172.21.15.63:34536) 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f481f400640 1 -- 172.21.15.63:0/352241372 learned_addr learned my addr 172.21.15.63:0/352241372 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f4824e00640 1 --2- 172.21.15.63:0/352241372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 0x55e348d5fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f481cc00640 1 -- 172.21.15.63:0/2217154186 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4810004e00 con 0x7f4820106430 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f481f400640 1 -- 172.21.15.63:0/352241372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489942f0 msgr2=0x55e348d5ffa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f481f400640 1 --2- 172.21.15.63:0/352241372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489942f0 0x55e348d5ffa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f481f400640 1 -- 172.21.15.63:0/352241372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 msgr2=0x55e348d5fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f481f400640 1 --2- 172.21.15.63:0/352241372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 0x55e348d5fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f481f400640 1 -- 172.21.15.63:0/352241372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e348c598b0 con 0x55e34899c4d0 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.783+0000 7f4824e00640 1 --2- 172.21.15.63:0/352241372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 0x55e348d5fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.784+0000 7f481f400640 1 --2- 172.21.15.63:0/352241372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d604e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f480c0057c0 tx=0x7f480c004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1928ad1a1656b2f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.784+0000 7f4815600640 1 -- 172.21.15.63:0/352241372 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c035030 con 0x55e34899c4d0 2024-05-27T07:41:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.784+0000 7f4815600640 1 -- 172.21.15.63:0/352241372 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f480c0054d0 con 0x55e34899c4d0 2024-05-27T07:41:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.784+0000 7f4815600640 1 -- 172.21.15.63:0/352241372 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c0035f0 con 0x55e34899c4d0 2024-05-27T07:41:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.785+0000 7f481cc00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48040b29d0 0x7f48040b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.785+0000 7f482dde1740 1 -- 172.21.15.63:0/352241372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 msgr2=0x55e348d604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.785+0000 7f482dde1740 1 --2- 172.21.15.63:0/352241372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d604e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f480c0057c0 tx=0x7f480c004150 comp rx=0 tx=0).stop 2024-05-27T07:41:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.785+0000 7f481cc00640 1 -- 172.21.15.63:0/2217154186 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f48100e2390 con 0x7f4820106430 2024-05-27T07:41:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.785+0000 7f4824e00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48040b29d0 0x7f48040b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.786+0000 7f482dde1740 1 -- 172.21.15.63:0/352241372 shutdown_connections 2024-05-27T07:41:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.786+0000 7f482dde1740 1 --2- 172.21.15.63:0/352241372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d604e0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.786+0000 7f482dde1740 1 --2- 172.21.15.63:0/352241372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489942f0 0x55e348d5ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.786+0000 7f482dde1740 1 --2- 172.21.15.63:0/352241372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 0x55e348d5fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.786+0000 7f482dde1740 1 -- 172.21.15.63:0/352241372 >> 172.21.15.63:0/352241372 conn(0x55e3489b1300 msgr2=0x55e348d5c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.786+0000 7f482dde1740 1 -- 172.21.15.63:0/352241372 shutdown_connections 2024-05-27T07:41:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.786+0000 7f482dde1740 1 -- 172.21.15.63:0/352241372 wait complete. 2024-05-27T07:41:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.786+0000 7f4824e00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48040b29d0 0x7f48040b4ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55e348d5cbd0 tx=0x7f4818009500 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:18.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.787+0000 7f482dde1740 1 -- start start 2024-05-27T07:41:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.787+0000 7f482dde1740 1 Processor -- start 2024-05-27T07:41:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.787+0000 7f482dde1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489942f0 0x55e348d69be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f482dde1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f482dde1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 0x55e348d75e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f482dde1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e3489a6790 con 0x55e34899c4d0 2024-05-27T07:41:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f482dde1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e348c68bc0 con 0x55e3489bd1b0 2024-05-27T07:41:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f482dde1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e348d6cb30 con 0x55e3489942f0 2024-05-27T07:41:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f482dde1740 10 client.0 _ll_register_callbacks cb 0x55e348c46400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f481f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f481f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34550/0 (socket says 172.21.15.63:34550) 2024-05-27T07:41:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f481f400640 1 -- 172.21.15.63:0/3869778516 learned_addr learned my addr 172.21.15.63:0/3869778516 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f481fe00640 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489942f0 0x55e348d69be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f4824e00640 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 0x55e348d75e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f481f400640 1 -- 172.21.15.63:0/3869778516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489942f0 msgr2=0x55e348d69be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f481f400640 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489942f0 0x55e348d69be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f481f400640 1 -- 172.21.15.63:0/3869778516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 msgr2=0x55e348d75e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f481f400640 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 0x55e348d75e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f481f400640 1 -- 172.21.15.63:0/3869778516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3489d5210 con 0x55e34899c4d0 2024-05-27T07:41:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.788+0000 7f4824e00640 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 0x55e348d75e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:41:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.789+0000 7f481f400640 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6a120 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f480c02a450 tx=0x7f480c004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45510c7802d05063 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.789+0000 7f480a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.789+0000 7f480a000640 1 -- 172.21.15.63:0/3869778516 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c051060 con 0x55e34899c4d0 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.789+0000 7f482dde1740 1 -- 172.21.15.63:0/3869778516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f480c0054d0 con 0x55e34899c4d0 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.789+0000 7f480a000640 1 -- 172.21.15.63:0/3869778516 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f480c0050e0 con 0x55e34899c4d0 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f480a000640 1 -- 172.21.15.63:0/3869778516 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f480c058050 con 0x55e34899c4d0 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f482dde1740 10 client.16890 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f482dde1740 1 -- 172.21.15.63:0/3869778516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e348aa1620 con 0x55e34899c4d0 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f480a000640 1 -- 172.21.15.63:0/3869778516 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f480c0054d0 con 0x55e34899c4d0 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f480a000640 1 -- 172.21.15.63:0/3869778516 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f480c03b430 con 0x55e34899c4d0 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f482dde1740 10 client.16890 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f482dde1740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f482dde1740 20 client.16890 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f482dde1740 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e34897bd80 0x55e348d7bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f482dde1740 4 client.16890 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.790+0000 7f482dde1740 1 -- 172.21.15.63:0/3869778516 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e348cd2ae0 con 0x55e34897bd80 2024-05-27T07:41:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.791+0000 7f481fe00640 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e34897bd80 0x55e348d7bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.792+0000 7f480a000640 10 client.16890 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.792+0000 7f481fe00640 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e34897bd80 0x55e348d7bc20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474cab41d7cd552a server_cookie=c5a095b74d0b471a in_seq=0 out_seq=0 2024-05-27T07:41:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.794+0000 7f480a000640 1 -- 172.21.15.63:0/3869778516 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e348cd2ae0 con 0x55e34897bd80 2024-05-27T07:41:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.794+0000 7f480a000640 10 client.16890 handle_command_reply: tid=0 2024-05-27T07:41:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.794+0000 7f482dde1740 1 client.16890 shutdown 2024-05-27T07:41:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.794+0000 7f482dde1740 1 -- 172.21.15.63:0/3869778516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 msgr2=0x55e348d6a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.794+0000 7f482dde1740 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6a120 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f480c02a450 tx=0x7f480c004960 comp rx=0 tx=0).stop 2024-05-27T07:41:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.794+0000 7f482dde1740 1 -- 172.21.15.63:0/3869778516 shutdown_connections 2024-05-27T07:41:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.794+0000 7f482dde1740 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e34897bd80 0x55e348d7bc20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.795+0000 7f482dde1740 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489bd1b0 0x55e348d75e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.795+0000 7f482dde1740 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6a120 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.795+0000 7f482dde1740 1 --2- 172.21.15.63:0/3869778516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489942f0 0x55e348d69be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.795+0000 7f480a000640 0 client.16890 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.795+0000 7f482dde1740 1 -- 172.21.15.63:0/3869778516 >> 172.21.15.63:0/3869778516 conn(0x55e3489b1300 msgr2=0x55e348d61780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.795+0000 7f482dde1740 1 -- 172.21.15.63:0/3869778516 shutdown_connections 2024-05-27T07:41:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.795+0000 7f482dde1740 1 -- 172.21.15.63:0/3869778516 wait complete. 2024-05-27T07:41:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.795+0000 7f482dde1740 20 client.16890 trim_cache size 0 max 16384 2024-05-27T07:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.817+0000 7f482dde1740 1 Processor -- start 2024-05-27T07:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.817+0000 7f482dde1740 1 -- start start 2024-05-27T07:41:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.817+0000 7f482dde1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489942f0 0x55e348d653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.817+0000 7f482dde1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.817+0000 7f482dde1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489bd1b0 0x55e348d75dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.817+0000 7f482dde1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48201b5ab0 con 0x55e34899c4d0 2024-05-27T07:41:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.817+0000 7f482dde1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48201b3380 con 0x55e3489942f0 2024-05-27T07:41:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.817+0000 7f482dde1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48201b11f0 con 0x55e3489bd1b0 2024-05-27T07:41:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.818+0000 7f481f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.818+0000 7f481f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34558/0 (socket says 172.21.15.63:34558) 2024-05-27T07:41:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.818+0000 7f481f400640 1 -- 172.21.15.63:0/2445514826 learned_addr learned my addr 172.21.15.63:0/2445514826 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.818+0000 7f481fe00640 1 --2- 172.21.15.63:0/2445514826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489942f0 0x55e348d653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.818+0000 7f4824e00640 1 --2- 172.21.15.63:0/2445514826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489bd1b0 0x55e348d75dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.818+0000 7f481f400640 1 -- 172.21.15.63:0/2445514826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489bd1b0 msgr2=0x55e348d75dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.818+0000 7f481f400640 1 --2- 172.21.15.63:0/2445514826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489bd1b0 0x55e348d75dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.818+0000 7f481f400640 1 -- 172.21.15.63:0/2445514826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489942f0 msgr2=0x55e348d653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.818+0000 7f481f400640 1 --2- 172.21.15.63:0/2445514826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489942f0 0x55e348d653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.818+0000 7f481f400640 1 -- 172.21.15.63:0/2445514826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e348d5a610 con 0x55e34899c4d0 2024-05-27T07:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.819+0000 7f481f400640 1 --2- 172.21.15.63:0/2445514826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6bd30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f480c004650 tx=0x7f480c004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740b27d4409d0034 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.819+0000 7f4814c00640 1 -- 172.21.15.63:0/2445514826 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c051070 con 0x55e34899c4d0 2024-05-27T07:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.819+0000 7f4814c00640 1 -- 172.21.15.63:0/2445514826 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f480c005370 con 0x55e34899c4d0 2024-05-27T07:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.819+0000 7f4814c00640 1 -- 172.21.15.63:0/2445514826 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c035070 con 0x55e34899c4d0 2024-05-27T07:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.820+0000 7f482dde1740 1 -- 172.21.15.63:0/2445514826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 msgr2=0x55e348d6bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.820+0000 7f482dde1740 1 --2- 172.21.15.63:0/2445514826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6bd30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f480c004650 tx=0x7f480c004150 comp rx=0 tx=0).stop 2024-05-27T07:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.820+0000 7f482dde1740 1 -- 172.21.15.63:0/2445514826 shutdown_connections 2024-05-27T07:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.820+0000 7f482dde1740 1 --2- 172.21.15.63:0/2445514826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3489bd1b0 0x55e348d75dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.820+0000 7f482dde1740 1 --2- 172.21.15.63:0/2445514826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e34899c4d0 0x55e348d6bd30 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.820+0000 7f482dde1740 1 --2- 172.21.15.63:0/2445514826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489942f0 0x55e348d653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.820+0000 7f482dde1740 1 -- 172.21.15.63:0/2445514826 >> 172.21.15.63:0/2445514826 conn(0x55e3489b1300 msgr2=0x55e348d69470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.820+0000 7f482dde1740 1 -- 172.21.15.63:0/2445514826 shutdown_connections 2024-05-27T07:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.820+0000 7f482dde1740 1 -- 172.21.15.63:0/2445514826 wait complete. 2024-05-27T07:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.821+0000 7f482dde1740 1 -- start start 2024-05-27T07:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.821+0000 7f482dde1740 1 Processor -- start 2024-05-27T07:41:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f482dde1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489942f0 0x55e348d7bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f482dde1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34899c4d0 0x55e348d7c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f482dde1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3489bd1b0 0x55e348de2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f482dde1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48201b1840 con 0x55e3489bd1b0 2024-05-27T07:41:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f482dde1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48201b2cb0 con 0x55e3489942f0 2024-05-27T07:41:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f482dde1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48201b1f10 con 0x55e34899c4d0 2024-05-27T07:41:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f482dde1740 10 client.0 _ll_register_callbacks cb 0x7f480c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f4824e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3489bd1b0 0x55e348de2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f4824e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3489bd1b0 0x55e348de2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34574/0 (socket says 172.21.15.63:34574) 2024-05-27T07:41:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f4824e00640 1 -- 172.21.15.63:0/1023077101 learned_addr learned my addr 172.21.15.63:0/1023077101 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f481f400640 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34899c4d0 0x55e348d7c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.822+0000 7f481fe00640 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489942f0 0x55e348d7bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.823+0000 7f4824e00640 1 -- 172.21.15.63:0/1023077101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34899c4d0 msgr2=0x55e348d7c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.823+0000 7f4824e00640 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34899c4d0 0x55e348d7c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.823+0000 7f4824e00640 1 -- 172.21.15.63:0/1023077101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489942f0 msgr2=0x55e348d7bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.823+0000 7f4824e00640 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489942f0 0x55e348d7bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.823+0000 7f4824e00640 1 -- 172.21.15.63:0/1023077101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e348d7cb00 con 0x55e3489bd1b0 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.823+0000 7f4824e00640 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3489bd1b0 0x55e348de2060 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4818010a40 tx=0x7f4818014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5d536ef4ae5705 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.823+0000 7f480a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.823+0000 7f480a000640 1 -- 172.21.15.63:0/1023077101 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481801cd90 con 0x55e3489bd1b0 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.824+0000 7f480a000640 1 -- 172.21.15.63:0/1023077101 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f4818004b20 con 0x55e3489bd1b0 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.824+0000 7f482dde1740 1 -- 172.21.15.63:0/1023077101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f480c005370 con 0x55e3489bd1b0 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.824+0000 7f480a000640 1 -- 172.21.15.63:0/1023077101 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4818032050 con 0x55e3489bd1b0 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.824+0000 7f482dde1740 10 client.16896 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.824+0000 7f482dde1740 1 -- 172.21.15.63:0/1023077101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e348de3620 con 0x55e3489bd1b0 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.824+0000 7f480a000640 1 -- 172.21.15.63:0/1023077101 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f480c005370 con 0x55e3489bd1b0 2024-05-27T07:41:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.824+0000 7f480a000640 1 -- 172.21.15.63:0/1023077101 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4818027400 con 0x55e3489bd1b0 2024-05-27T07:41:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.825+0000 7f482dde1740 10 client.16896 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.825+0000 7f482dde1740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.825+0000 7f482dde1740 20 client.16896 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.825+0000 7f482dde1740 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3489511c0 0x55e348de5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.825+0000 7f482dde1740 4 client.16896 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:41:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.825+0000 7f482dde1740 1 -- 172.21.15.63:0/1023077101 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e348ccc2c0 con 0x55e3489511c0 2024-05-27T07:41:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.825+0000 7f481fe00640 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3489511c0 0x55e348de5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.826+0000 7f480a000640 10 client.16896 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.826+0000 7f481fe00640 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3489511c0 0x55e348de5e60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2eedea374fcca7e server_cookie=28db7fd121db929f in_seq=0 out_seq=0 2024-05-27T07:41:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.856+0000 7f480a000640 1 -- 172.21.15.63:0/1023077101 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e348ccc2c0 con 0x55e3489511c0 2024-05-27T07:41:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.856+0000 7f480a000640 10 client.16896 handle_command_reply: tid=0 2024-05-27T07:41:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.856+0000 7f482dde1740 1 client.16896 shutdown 2024-05-27T07:41:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f482dde1740 1 -- 172.21.15.63:0/1023077101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3489bd1b0 msgr2=0x55e348de2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f482dde1740 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3489bd1b0 0x55e348de2060 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4818010a40 tx=0x7f4818014040 comp rx=0 tx=0).stop 2024-05-27T07:41:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f482dde1740 1 -- 172.21.15.63:0/1023077101 shutdown_connections 2024-05-27T07:41:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f482dde1740 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3489511c0 0x55e348de5e60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f482dde1740 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3489bd1b0 0x55e348de2060 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f482dde1740 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e34899c4d0 0x55e348d7c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f482dde1740 1 --2- 172.21.15.63:0/1023077101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3489942f0 0x55e348d7bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f482dde1740 1 -- 172.21.15.63:0/1023077101 >> 172.21.15.63:0/1023077101 conn(0x55e3489b1300 msgr2=0x55e348d69e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f480a000640 0 client.16896 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f482dde1740 1 -- 172.21.15.63:0/1023077101 shutdown_connections 2024-05-27T07:41:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.857+0000 7f482dde1740 1 -- 172.21.15.63:0/1023077101 wait complete. 2024-05-27T07:41:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.858+0000 7f482dde1740 20 client.16896 trim_cache size 0 max 16384 2024-05-27T07:41:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:41:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:41:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:41:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 -- 172.21.15.63:0/2217154186 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48040b29d0 msgr2=0x7f48040b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48040b29d0 0x7f48040b4ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55e348d5cbd0 tx=0x7f4818009500 comp rx=0 tx=0).stop 2024-05-27T07:41:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 -- 172.21.15.63:0/2217154186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4820106430 msgr2=0x7f48201a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4820106430 0x7f48201a2530 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f481000d740 tx=0x7f481000dc00 comp rx=0 tx=0).stop 2024-05-27T07:41:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 -- 172.21.15.63:0/2217154186 shutdown_connections 2024-05-27T07:41:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48040b29d0 0x7f48040b4ee0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f482010c180 0x7f482019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f482010bce0 0x7f48201a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 --2- 172.21.15.63:0/2217154186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4820106430 0x7f48201a2530 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 -- 172.21.15.63:0/2217154186 >> 172.21.15.63:0/2217154186 conn(0x7f4820101460 msgr2=0x7f4820101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 -- 172.21.15.63:0/2217154186 shutdown_connections 2024-05-27T07:41:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:18.859+0000 7f4826e00640 1 -- 172.21.15.63:0/2217154186 wait complete. 2024-05-27T07:41:19.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:19 smithi086 ceph-mon[34687]: pgmap v239: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 8 op/s 2024-05-27T07:41:19.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:19 smithi149 ceph-mon[35408]: pgmap v239: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 8 op/s 2024-05-27T07:41:19.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:19 smithi063 ceph-mon[31613]: pgmap v239: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 8 op/s 2024-05-27T07:41:21.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:21 smithi086 ceph-mon[34687]: pgmap v240: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 8 op/s 2024-05-27T07:41:21.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:21 smithi149 ceph-mon[35408]: pgmap v240: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 8 op/s 2024-05-27T07:41:21.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:21 smithi063 ceph-mon[31613]: pgmap v240: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 8 op/s 2024-05-27T07:41:23.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:23 smithi086 ceph-mon[34687]: pgmap v241: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 14 op/s 2024-05-27T07:41:23.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:23 smithi149 ceph-mon[35408]: pgmap v241: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 14 op/s 2024-05-27T07:41:23.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:23 smithi063 ceph-mon[31613]: pgmap v241: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 14 op/s 2024-05-27T07:41:25.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:25 smithi086 ceph-mon[34687]: pgmap v242: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:41:25.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:25 smithi149 ceph-mon[35408]: pgmap v242: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:41:25.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:25 smithi063 ceph-mon[31613]: pgmap v242: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:41:27.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:27 smithi086 ceph-mon[34687]: pgmap v243: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:41:27.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:27 smithi149 ceph-mon[35408]: pgmap v243: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:41:27.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:27 smithi063 ceph-mon[31613]: pgmap v243: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:41:28.104 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:41:28.617 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:29.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:29 smithi086 ceph-mon[34687]: pgmap v244: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:41:29.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:29 smithi149 ceph-mon[35408]: pgmap v244: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:41:29.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:29 smithi063 ceph-mon[31613]: pgmap v244: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:41:31.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.120+0000 7f72b9000700 1 -- 172.21.15.63:0/203193371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410e700 msgr2=0x7f72b4110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:31.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.120+0000 7f72b9000700 1 --2- 172.21.15.63:0/203193371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410e700 0x7f72b4110be0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00b0a0 tx=0x7f72ac00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.120+0000 7f72b9000700 1 -- 172.21.15.63:0/203193371 shutdown_connections 2024-05-27T07:41:31.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.120+0000 7f72b9000700 1 --2- 172.21.15.63:0/203193371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410e700 0x7f72b4110be0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.120+0000 7f72b9000700 1 --2- 172.21.15.63:0/203193371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4105320 0x7f72b4105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.120+0000 7f72b9000700 1 --2- 172.21.15.63:0/203193371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b4105c70 0x7f72b410e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.120+0000 7f72b9000700 1 -- 172.21.15.63:0/203193371 >> 172.21.15.63:0/203193371 conn(0x7f72b4078af0 msgr2=0x7f72b4078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:31.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.121+0000 7f72b9000700 1 -- 172.21.15.63:0/203193371 shutdown_connections 2024-05-27T07:41:31.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.121+0000 7f72b9000700 1 -- 172.21.15.63:0/203193371 wait complete. 2024-05-27T07:41:31.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.122+0000 7f72b9000700 1 Processor -- start 2024-05-27T07:41:31.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.123+0000 7f72b9000700 1 -- start start 2024-05-27T07:41:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.123+0000 7f72b9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4105320 0x7f72b41a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.123+0000 7f72b9000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b4105c70 0x7f72b41a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.123+0000 7f72b9000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e700 0x7f72b41acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.123+0000 7f72b9000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4114160 con 0x7f72b4105c70 2024-05-27T07:41:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.123+0000 7f72b9000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4113fe0 con 0x7f72b4105320 2024-05-27T07:41:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.123+0000 7f72b9000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41142e0 con 0x7f72b410e700 2024-05-27T07:41:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.124+0000 7f72b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4105320 0x7f72b41a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.124+0000 7f72b3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4105320 0x7f72b41a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:36082/0 (socket says 172.21.15.63:36082) 2024-05-27T07:41:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.124+0000 7f72b3400700 1 -- 172.21.15.63:0/2209155070 learned_addr learned my addr 172.21.15.63:0/2209155070 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.124+0000 7f72b2a00700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b4105c70 0x7f72b41a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.124+0000 7f72b3e00700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e700 0x7f72b41acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.125+0000 7f72b2a00700 1 -- 172.21.15.63:0/2209155070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e700 msgr2=0x7f72b41acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.125+0000 7f72b2a00700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e700 0x7f72b41acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.125+0000 7f72b2a00700 1 -- 172.21.15.63:0/2209155070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4105320 msgr2=0x7f72b41a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.125+0000 7f72b2a00700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4105320 0x7f72b41a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.125+0000 7f72b2a00700 1 -- 172.21.15.63:0/2209155070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b41a2c60 con 0x7f72b4105c70 2024-05-27T07:41:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.125+0000 7f72b3400700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4105320 0x7f72b41a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:41:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.125+0000 7f72b3e00700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e700 0x7f72b41acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:41:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.126+0000 7f72b2a00700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b4105c70 0x7f72b41a2380 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f72a000d950 tx=0x7f72a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.126+0000 7f72abe00700 1 -- 172.21.15.63:0/2209155070 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0016070 con 0x7f72b4105c70 2024-05-27T07:41:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.126+0000 7f72abe00700 1 -- 172.21.15.63:0/2209155070 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a000fd20 con 0x7f72b4105c70 2024-05-27T07:41:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.126+0000 7f72b9000700 1 -- 172.21.15.63:0/2209155070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b41ad4c0 con 0x7f72b4105c70 2024-05-27T07:41:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.126+0000 7f72abe00700 1 -- 172.21.15.63:0/2209155070 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a000e3b0 con 0x7f72b4105c70 2024-05-27T07:41:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.126+0000 7f72b9000700 1 -- 172.21.15.63:0/2209155070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b41ada00 con 0x7f72b4105c70 2024-05-27T07:41:31.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.132+0000 7f72abe00700 1 -- 172.21.15.63:0/2209155070 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f72a0019070 con 0x7f72b4105c70 2024-05-27T07:41:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.132+0000 7f72c9136b80 1 Processor -- start 2024-05-27T07:41:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.132+0000 7f72c9136b80 1 -- start start 2024-05-27T07:41:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.133+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2db8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.133+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.133+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2db9820 0x55f4b2dc6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.133+0000 7f72c9136b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f4b2ce7e40 con 0x55f4b2c3ae40 2024-05-27T07:41:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.133+0000 7f72c9136b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f4b2cd79e0 con 0x55f4b2db9410 2024-05-27T07:41:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.133+0000 7f72c9136b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f4b2cdba20 con 0x55f4b2db9820 2024-05-27T07:41:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.133+0000 7f72b3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2db8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.134+0000 7f72b3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2db8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46044/0 (socket says 172.21.15.63:46044) 2024-05-27T07:41:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.134+0000 7f72b3e00700 1 -- 172.21.15.63:0/1388211332 learned_addr learned my addr 172.21.15.63:0/1388211332 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.134+0000 7f72b3400700 1 --2- 172.21.15.63:0/1388211332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.134+0000 7f72b2a00700 1 --2- 172.21.15.63:0/1388211332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2db9820 0x55f4b2dc6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.134+0000 7f72abe00700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72980b29f0 0x7f72980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.134+0000 7f72b3e00700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72980b29f0 0x7f72980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.134+0000 7f72b3400700 1 -- 172.21.15.63:0/1388211332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2db9820 msgr2=0x55f4b2dc6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.134+0000 7f72b3400700 1 --2- 172.21.15.63:0/1388211332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2db9820 0x55f4b2dc6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.135+0000 7f72b3400700 1 -- 172.21.15.63:0/1388211332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 msgr2=0x55f4b2db8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.135+0000 7f72b3400700 1 --2- 172.21.15.63:0/1388211332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2db8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.135+0000 7f72b3400700 1 -- 172.21.15.63:0/1388211332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4b2ca0420 con 0x55f4b2db9410 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.135+0000 7f72abe00700 1 -- 172.21.15.63:0/2209155070 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f72a00e2750 con 0x7f72b4105c70 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.135+0000 7f72b3e00700 1 --2- 172.21.15.63:0/1388211332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2db8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.135+0000 7f72b3400700 1 --2- 172.21.15.63:0/1388211332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc3ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f72a400f4c0 tx=0x7f72a400f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9610f6140f02207 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.136+0000 7f72b3e00700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72980b29f0 0x7f72980b4f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009610 tx=0x7f72ac00b3b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.136+0000 7f72a8c00700 1 -- 172.21.15.63:0/1388211332 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4004a30 con 0x55f4b2db9410 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.136+0000 7f72a8c00700 1 -- 172.21.15.63:0/1388211332 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a400fe50 con 0x55f4b2db9410 2024-05-27T07:41:31.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.136+0000 7f72a8c00700 1 -- 172.21.15.63:0/1388211332 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4010840 con 0x55f4b2db9410 2024-05-27T07:41:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.137+0000 7f72c9136b80 1 -- 172.21.15.63:0/1388211332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 msgr2=0x55f4b2dc3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.137+0000 7f72c9136b80 1 --2- 172.21.15.63:0/1388211332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc3ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f72a400f4c0 tx=0x7f72a400f7d0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.137+0000 7f72c9136b80 1 -- 172.21.15.63:0/1388211332 shutdown_connections 2024-05-27T07:41:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.137+0000 7f72c9136b80 1 --2- 172.21.15.63:0/1388211332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2db9820 0x55f4b2dc6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.137+0000 7f72c9136b80 1 --2- 172.21.15.63:0/1388211332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2db8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.137+0000 7f72c9136b80 1 --2- 172.21.15.63:0/1388211332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc3ad0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.137+0000 7f72c9136b80 1 -- 172.21.15.63:0/1388211332 >> 172.21.15.63:0/1388211332 conn(0x55f4b2c9afc0 msgr2=0x55f4b2db4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.137+0000 7f72c9136b80 1 -- 172.21.15.63:0/1388211332 shutdown_connections 2024-05-27T07:41:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.137+0000 7f72c9136b80 1 -- 172.21.15.63:0/1388211332 wait complete. 2024-05-27T07:41:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.138+0000 7f72c9136b80 1 -- start start 2024-05-27T07:41:31.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.138+0000 7f72c9136b80 1 Processor -- start 2024-05-27T07:41:31.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.139+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.139+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.139+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.139+0000 7f72c9136b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f4b2bddfa0 con 0x55f4b2db9820 2024-05-27T07:41:31.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.139+0000 7f72c9136b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f4b2c78760 con 0x55f4b2db9410 2024-05-27T07:41:31.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.139+0000 7f72c9136b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f4b2dc6990 con 0x55f4b2c3ae40 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.139+0000 7f72c9136b80 10 client.0 _ll_register_callbacks cb 0x55f4b2c81c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.139+0000 7f72b3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.139+0000 7f72b3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46050/0 (socket says 172.21.15.63:46050) 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.139+0000 7f72b3e00700 1 -- 172.21.15.63:0/2651289803 learned_addr learned my addr 172.21.15.63:0/2651289803 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.140+0000 7f72b3400700 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.140+0000 7f72b2a00700 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.140+0000 7f72b3e00700 1 -- 172.21.15.63:0/2651289803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 msgr2=0x55f4b2dc3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.140+0000 7f72b3e00700 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.140+0000 7f72b3e00700 1 -- 172.21.15.63:0/2651289803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 msgr2=0x55f4b2dc3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.140+0000 7f72b3e00700 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.140+0000 7f72b3e00700 1 -- 172.21.15.63:0/2651289803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4b2c278d0 con 0x55f4b2db9820 2024-05-27T07:41:31.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.141+0000 7f72b3e00700 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc4000 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00b830 tx=0x7f72ac0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29597d76bf129577 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:31.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.141+0000 7f729d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:31.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.141+0000 7f729d600700 1 -- 172.21.15.63:0/2651289803 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac02f020 con 0x55f4b2db9820 2024-05-27T07:41:31.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.141+0000 7f72c9136b80 1 -- 172.21.15.63:0/2651289803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a400fe50 con 0x55f4b2db9820 2024-05-27T07:41:31.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.141+0000 7f729d600700 1 -- 172.21.15.63:0/2651289803 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f72ac01eab0 con 0x55f4b2db9820 2024-05-27T07:41:31.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.142+0000 7f729d600700 1 -- 172.21.15.63:0/2651289803 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f72ac01ecb0 con 0x55f4b2db9820 2024-05-27T07:41:31.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.142+0000 7f72c9136b80 10 client.16908 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:31.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.142+0000 7f72c9136b80 1 -- 172.21.15.63:0/2651289803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f4b2ca8060 con 0x55f4b2db9820 2024-05-27T07:41:31.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.143+0000 7f729d600700 1 -- 172.21.15.63:0/2651289803 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a400fe50 con 0x55f4b2db9820 2024-05-27T07:41:31.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.143+0000 7f729d600700 1 -- 172.21.15.63:0/2651289803 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f72ac011070 con 0x55f4b2db9820 2024-05-27T07:41:31.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.143+0000 7f72c9136b80 10 client.16908 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:31.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.143+0000 7f72c9136b80 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:31.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.143+0000 7f72c9136b80 20 client.16908 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:31.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.144+0000 7f72c9136b80 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f4b2dd3790 0x55f4b2dd5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.144+0000 7f72c9136b80 4 client.16908 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:31.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.144+0000 7f72c9136b80 1 -- 172.21.15.63:0/2651289803 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4b2ce9d90 con 0x55f4b2dd3790 2024-05-27T07:41:31.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.145+0000 7f72b3400700 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f4b2dd3790 0x55f4b2dd5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.146+0000 7f72b3400700 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f4b2dd3790 0x55f4b2dd5c40 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59e89f0f86a44bcd server_cookie=d9edb1d5a0b959e4 in_seq=0 out_seq=0 2024-05-27T07:41:31.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.146+0000 7f729d600700 10 client.16908 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:31.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f729d600700 1 -- 172.21.15.63:0/2651289803 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f4b2ce9d90 con 0x55f4b2dd3790 2024-05-27T07:41:31.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f729d600700 10 client.16908 handle_command_reply: tid=0 2024-05-27T07:41:31.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f72c9136b80 1 client.16908 shutdown 2024-05-27T07:41:31.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f72c9136b80 1 -- 172.21.15.63:0/2651289803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 msgr2=0x55f4b2dc4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:31.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f72c9136b80 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc4000 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00b830 tx=0x7f72ac0099e0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f72c9136b80 1 -- 172.21.15.63:0/2651289803 shutdown_connections 2024-05-27T07:41:31.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f72c9136b80 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f4b2dd3790 0x55f4b2dd5c40 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f72c9136b80 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc4000 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f72c9136b80 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f729d600700 0 client.16908 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:31.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f72c9136b80 1 --2- 172.21.15.63:0/2651289803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.148+0000 7f72c9136b80 1 -- 172.21.15.63:0/2651289803 >> 172.21.15.63:0/2651289803 conn(0x55f4b2c9afc0 msgr2=0x55f4b2db3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:31.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.149+0000 7f72c9136b80 1 -- 172.21.15.63:0/2651289803 shutdown_connections 2024-05-27T07:41:31.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.149+0000 7f72c9136b80 1 -- 172.21.15.63:0/2651289803 wait complete. 2024-05-27T07:41:31.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.149+0000 7f72c9136b80 20 client.16908 trim_cache size 0 max 16384 2024-05-27T07:41:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.178+0000 7f72c9136b80 1 Processor -- start 2024-05-27T07:41:31.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.178+0000 7f72c9136b80 1 -- start start 2024-05-27T07:41:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.178+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.178+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dbedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.178+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2db9820 0x55f4b2dc5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.178+0000 7f72c9136b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b91a0 con 0x55f4b2c3ae40 2024-05-27T07:41:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.178+0000 7f72c9136b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b6a30 con 0x55f4b2db9410 2024-05-27T07:41:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.178+0000 7f72c9136b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b48a0 con 0x55f4b2db9820 2024-05-27T07:41:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.179+0000 7f72b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.179+0000 7f72b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46058/0 (socket says 172.21.15.63:46058) 2024-05-27T07:41:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.179+0000 7f72b3400700 1 -- 172.21.15.63:0/3256793451 learned_addr learned my addr 172.21.15.63:0/3256793451 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.179+0000 7f72b3400700 1 -- 172.21.15.63:0/3256793451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2db9820 msgr2=0x55f4b2dc5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.179+0000 7f72b2a00700 1 --2- 172.21.15.63:0/3256793451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dbedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.179+0000 7f72b3400700 1 --2- 172.21.15.63:0/3256793451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2db9820 0x55f4b2dc5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.179+0000 7f72b3400700 1 -- 172.21.15.63:0/3256793451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 msgr2=0x55f4b2dbedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:31.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.179+0000 7f72b3400700 1 --2- 172.21.15.63:0/3256793451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dbedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.180+0000 7f72b3400700 1 -- 172.21.15.63:0/3256793451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4b2daf1d0 con 0x55f4b2c3ae40 2024-05-27T07:41:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.180+0000 7f72b3400700 1 --2- 172.21.15.63:0/3256793451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc21e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f72a401a040 tx=0x7f72a400ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b339735234b280e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.180+0000 7f729fe00700 1 -- 172.21.15.63:0/3256793451 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4032070 con 0x55f4b2c3ae40 2024-05-27T07:41:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.180+0000 7f729fe00700 1 -- 172.21.15.63:0/3256793451 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a400fe50 con 0x55f4b2c3ae40 2024-05-27T07:41:31.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.180+0000 7f729fe00700 1 -- 172.21.15.63:0/3256793451 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4004f60 con 0x55f4b2c3ae40 2024-05-27T07:41:31.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.182+0000 7f72c9136b80 1 -- 172.21.15.63:0/3256793451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 msgr2=0x55f4b2dc21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:31.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.182+0000 7f72c9136b80 1 --2- 172.21.15.63:0/3256793451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc21e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f72a401a040 tx=0x7f72a400ab20 comp rx=0 tx=0).stop 2024-05-27T07:41:31.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.182+0000 7f72c9136b80 1 -- 172.21.15.63:0/3256793451 shutdown_connections 2024-05-27T07:41:31.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.182+0000 7f72c9136b80 1 --2- 172.21.15.63:0/3256793451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2db9820 0x55f4b2dc5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.182+0000 7f72c9136b80 1 --2- 172.21.15.63:0/3256793451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc21e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.182+0000 7f72c9136b80 1 --2- 172.21.15.63:0/3256793451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dbedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.182+0000 7f72c9136b80 1 -- 172.21.15.63:0/3256793451 >> 172.21.15.63:0/3256793451 conn(0x55f4b2c9afc0 msgr2=0x55f4b2dc3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:31.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.182+0000 7f72c9136b80 1 -- 172.21.15.63:0/3256793451 shutdown_connections 2024-05-27T07:41:31.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.182+0000 7f72c9136b80 1 -- 172.21.15.63:0/3256793451 wait complete. 2024-05-27T07:41:31.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.183+0000 7f72c9136b80 1 -- start start 2024-05-27T07:41:31.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.183+0000 7f72c9136b80 1 Processor -- start 2024-05-27T07:41:31.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72c9136b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72c9136b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b4ef0 con 0x55f4b2db9820 2024-05-27T07:41:31.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72c9136b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b6360 con 0x55f4b2db9410 2024-05-27T07:41:31.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72c9136b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b55c0 con 0x55f4b2c3ae40 2024-05-27T07:41:31.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72c9136b80 10 client.0 _ll_register_callbacks cb 0x55f4b2c81c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:31.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72b3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72b3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46070/0 (socket says 172.21.15.63:46070) 2024-05-27T07:41:31.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72b3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.184+0000 7f72b3e00700 1 -- 172.21.15.63:0/1237577448 learned_addr learned my addr 172.21.15.63:0/1237577448 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:31.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.185+0000 7f72b2a00700 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.185+0000 7f72b3400700 1 -- 172.21.15.63:0/1237577448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 msgr2=0x55f4b2dc8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:31.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.185+0000 7f72b3400700 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.185+0000 7f72b3400700 1 -- 172.21.15.63:0/1237577448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 msgr2=0x55f4b2dc85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:31.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.185+0000 7f72b3400700 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.185+0000 7f72b3400700 1 -- 172.21.15.63:0/1237577448 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4b2db6350 con 0x55f4b2c3ae40 2024-05-27T07:41:31.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.186+0000 7f72b3e00700 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:41:31.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.186+0000 7f72b3400700 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc7b20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72a401a040 tx=0x7f72a4004b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed448ffeab86add server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:31.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.186+0000 7f729d600700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:41:31.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.186+0000 7f729d600700 1 -- 172.21.15.63:0/1237577448 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4032030 con 0x55f4b2c3ae40 2024-05-27T07:41:31.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.186+0000 7f72c9136b80 1 -- 172.21.15.63:0/1237577448 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a400fe50 con 0x55f4b2c3ae40 2024-05-27T07:41:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.187+0000 7f729d600700 1 -- 172.21.15.63:0/1237577448 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f72a402a040 con 0x55f4b2c3ae40 2024-05-27T07:41:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.187+0000 7f729d600700 1 -- 172.21.15.63:0/1237577448 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f72a4040050 con 0x55f4b2c3ae40 2024-05-27T07:41:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.187+0000 7f72c9136b80 10 client.26368 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.187+0000 7f72c9136b80 1 -- 172.21.15.63:0/1237577448 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f4b2e3e480 con 0x55f4b2c3ae40 2024-05-27T07:41:31.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.188+0000 7f729d600700 1 -- 172.21.15.63:0/1237577448 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a400fe50 con 0x55f4b2c3ae40 2024-05-27T07:41:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.188+0000 7f729d600700 1 -- 172.21.15.63:0/1237577448 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f72a40055c0 con 0x55f4b2c3ae40 2024-05-27T07:41:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.188+0000 7f72c9136b80 10 client.26368 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.188+0000 7f72c9136b80 10 client.26368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.189+0000 7f72c9136b80 20 client.26368 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.189+0000 7f72c9136b80 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f4b2e3e8e0 0x55f4b2e40d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:31.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.189+0000 7f72c9136b80 4 client.26368 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:41:31.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.189+0000 7f72c9136b80 1 -- 172.21.15.63:0/1237577448 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f4b2ce3570 con 0x55f4b2e3e8e0 2024-05-27T07:41:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.192+0000 7f72b2a00700 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f4b2e3e8e0 0x55f4b2e40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.193+0000 7f72b2a00700 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f4b2e3e8e0 0x55f4b2e40d00 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10a20587b7db7ef server_cookie=ffddca18f48b0ed1 in_seq=0 out_seq=0 2024-05-27T07:41:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.193+0000 7f729d600700 10 client.26368 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f729d600700 1 -- 172.21.15.63:0/1237577448 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f4b2ce3570 con 0x55f4b2e3e8e0 2024-05-27T07:41:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f729d600700 10 client.26368 handle_command_reply: tid=0 2024-05-27T07:41:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f72c9136b80 1 client.26368 shutdown 2024-05-27T07:41:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f72c9136b80 1 -- 172.21.15.63:0/1237577448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 msgr2=0x55f4b2dc7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f72c9136b80 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc7b20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72a401a040 tx=0x7f72a4004b80 comp rx=0 tx=0).stop 2024-05-27T07:41:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f72c9136b80 1 -- 172.21.15.63:0/1237577448 shutdown_connections 2024-05-27T07:41:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f72c9136b80 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f4b2e3e8e0 0x55f4b2e40d00 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f72c9136b80 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f4b2db9820 0x55f4b2dc85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f72c9136b80 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f4b2c3ae40 0x55f4b2dc7b20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f72c9136b80 1 --2- 172.21.15.63:0/1237577448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4b2db9410 0x55f4b2dc8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f72c9136b80 1 -- 172.21.15.63:0/1237577448 >> 172.21.15.63:0/1237577448 conn(0x55f4b2c9afc0 msgr2=0x55f4b2dc4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.194+0000 7f729d600700 0 client.26368 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.195+0000 7f72c9136b80 1 -- 172.21.15.63:0/1237577448 shutdown_connections 2024-05-27T07:41:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.195+0000 7f72c9136b80 1 -- 172.21.15.63:0/1237577448 wait complete. 2024-05-27T07:41:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.195+0000 7f72c9136b80 20 client.26368 trim_cache size 0 max 16384 2024-05-27T07:41:31.202 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:41:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:41:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:41:31.203 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:41:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.196+0000 7f72b9000700 1 -- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72980b29f0 msgr2=0x7f72980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:31.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.196+0000 7f72b9000700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72980b29f0 0x7f72980b4f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009610 tx=0x7f72ac00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.196+0000 7f72b9000700 1 -- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b4105c70 msgr2=0x7f72b41a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:31.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.196+0000 7f72b9000700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b4105c70 0x7f72b41a2380 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f72a000d950 tx=0x7f72a000dc60 comp rx=0 tx=0).stop 2024-05-27T07:41:31.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.196+0000 7f72b9000700 1 -- 172.21.15.63:0/2209155070 shutdown_connections 2024-05-27T07:41:31.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.196+0000 7f72b9000700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72980b29f0 0x7f72980b4f00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.196+0000 7f72b9000700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e700 0x7f72b41acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.196+0000 7f72b9000700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4105320 0x7f72b41a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.196+0000 7f72b9000700 1 --2- 172.21.15.63:0/2209155070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b4105c70 0x7f72b41a2380 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:31.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.196+0000 7f72b9000700 1 -- 172.21.15.63:0/2209155070 >> 172.21.15.63:0/2209155070 conn(0x7f72b4078af0 msgr2=0x7f72b40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:31.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.197+0000 7f72b9000700 1 -- 172.21.15.63:0/2209155070 shutdown_connections 2024-05-27T07:41:31.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:31.197+0000 7f72b9000700 1 -- 172.21.15.63:0/2209155070 wait complete. 2024-05-27T07:41:31.676 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:31 smithi063 ceph-mon[31613]: pgmap v245: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:41:31.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:31 smithi086 ceph-mon[34687]: pgmap v245: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:41:31.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:31 smithi149 ceph-mon[35408]: pgmap v245: 129 pgs: 129 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:41:32.099 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:41:32.100 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:41:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.125+0000 7fa796200640 1 -- 172.21.15.63:0/893355522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7901065e0 msgr2=0x7fa7901129b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.125+0000 7fa796200640 1 --2- 172.21.15.63:0/893355522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7901065e0 0x7fa7901129b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa788009ca0 tx=0x7fa78802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.125+0000 7fa796200640 1 -- 172.21.15.63:0/893355522 shutdown_connections 2024-05-27T07:41:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.125+0000 7fa796200640 1 --2- 172.21.15.63:0/893355522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7901065e0 0x7fa7901129b0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.125+0000 7fa796200640 1 --2- 172.21.15.63:0/893355522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa79010c330 0x7fa790105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.125+0000 7fa796200640 1 --2- 172.21.15.63:0/893355522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79010be90 0x7fa790105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.125+0000 7fa796200640 1 -- 172.21.15.63:0/893355522 >> 172.21.15.63:0/893355522 conn(0x7fa790101530 msgr2=0x7fa790103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.125+0000 7fa796200640 1 -- 172.21.15.63:0/893355522 shutdown_connections 2024-05-27T07:41:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.125+0000 7fa796200640 1 -- 172.21.15.63:0/893355522 wait complete. 2024-05-27T07:41:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.127+0000 7fa796200640 1 Processor -- start 2024-05-27T07:41:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.127+0000 7fa796200640 1 -- start start 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.127+0000 7fa796200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7901065e0 0x7fa7901a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.128+0000 7fa796200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa79010be90 0x7fa7901a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.128+0000 7fa796200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79010c330 0x7fa79019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.128+0000 7fa796200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa790115f80 con 0x7fa79010be90 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.128+0000 7fa796200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa790115e00 con 0x7fa79010c330 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.128+0000 7fa796200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa790116100 con 0x7fa7901065e0 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.128+0000 7fa78ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa79010be90 0x7fa7901a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.128+0000 7fa78ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa79010be90 0x7fa7901a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46094/0 (socket says 172.21.15.63:46094) 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.128+0000 7fa78ea00640 1 -- 172.21.15.63:0/1554485151 learned_addr learned my addr 172.21.15.63:0/1554485151 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.128+0000 7fa78f400640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7901065e0 0x7fa7901a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.128+0000 7fa78fe00640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79010c330 0x7fa79019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.129+0000 7fa78ea00640 1 -- 172.21.15.63:0/1554485151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7901065e0 msgr2=0x7fa7901a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.129+0000 7fa78ea00640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7901065e0 0x7fa7901a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.129+0000 7fa78ea00640 1 -- 172.21.15.63:0/1554485151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79010c330 msgr2=0x7fa79019c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.129+0000 7fa78ea00640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79010c330 0x7fa79019c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.129+0000 7fa78ea00640 1 -- 172.21.15.63:0/1554485151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79019cec0 con 0x7fa79010be90 2024-05-27T07:41:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.129+0000 7fa78fe00640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79010c330 0x7fa79019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:41:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.129+0000 7fa78ea00640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa79010be90 0x7fa7901a2af0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00d740 tx=0x7fa77c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.130+0000 7fa787e00640 1 -- 172.21.15.63:0/1554485151 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c017070 con 0x7fa79010be90 2024-05-27T07:41:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.130+0000 7fa787e00640 1 -- 172.21.15.63:0/1554485151 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa77c00de50 con 0x7fa79010be90 2024-05-27T07:41:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.130+0000 7fa787e00640 1 -- 172.21.15.63:0/1554485151 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c003bb0 con 0x7fa79010be90 2024-05-27T07:41:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.130+0000 7fa796200640 1 -- 172.21.15.63:0/1554485151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79019d1b0 con 0x7fa79010be90 2024-05-27T07:41:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.130+0000 7fa796200640 1 -- 172.21.15.63:0/1554485151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7901a9400 con 0x7fa79010be90 2024-05-27T07:41:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.134+0000 7fa79d376740 1 Processor -- start 2024-05-27T07:41:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.135+0000 7fa79d376740 1 -- start start 2024-05-27T07:41:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.135+0000 7fa79d376740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59c81b0 0x5570c5d6a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.135+0000 7fa79d376740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 0x5570c5d6ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.135+0000 7fa79d376740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59a74d0 0x5570c5d6b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.135+0000 7fa79d376740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5570c5cdbb90 con 0x5570c59a74d0 2024-05-27T07:41:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.135+0000 7fa79d376740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5570c5ccb730 con 0x5570c59c81b0 2024-05-27T07:41:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.135+0000 7fa79d376740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5570c5ccf770 con 0x5570c599f2f0 2024-05-27T07:41:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.136+0000 7fa787e00640 1 -- 172.21.15.63:0/1554485151 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa77c003d50 con 0x7fa79010be90 2024-05-27T07:41:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.136+0000 7fa78f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 0x5570c5d6ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.136+0000 7fa78f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 0x5570c5d6ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:35098/0 (socket says 172.21.15.63:35098) 2024-05-27T07:41:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.136+0000 7fa78f400640 1 -- 172.21.15.63:0/2510773474 learned_addr learned my addr 172.21.15.63:0/2510773474 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.136+0000 7fa78ea00640 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59a74d0 0x5570c5d6b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.136+0000 7fa78fe00640 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59c81b0 0x5570c5d6a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.137+0000 7fa78f400640 1 -- 172.21.15.63:0/2510773474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59c81b0 msgr2=0x5570c5d6a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.137+0000 7fa78f400640 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59c81b0 0x5570c5d6a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.137+0000 7fa78f400640 1 -- 172.21.15.63:0/2510773474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59a74d0 msgr2=0x5570c5d6b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.137+0000 7fa78f400640 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59a74d0 0x5570c5d6b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.137+0000 7fa78f400640 1 -- 172.21.15.63:0/2510773474 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c5c648b0 con 0x5570c599f2f0 2024-05-27T07:41:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.137+0000 7fa78fe00640 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59c81b0 0x5570c5d6a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.137+0000 7fa78ea00640 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59a74d0 0x5570c5d6b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.137+0000 7fa78f400640 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 0x5570c5d6ada0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa78000bc70 tx=0x7fa7800107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce93daf95fcf220 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.138+0000 7fa77be00640 1 -- 172.21.15.63:0/2510773474 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780004030 con 0x5570c599f2f0 2024-05-27T07:41:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.138+0000 7fa787e00640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7740b2aa0 0x7fa7740b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.138+0000 7fa77be00640 1 -- 172.21.15.63:0/2510773474 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7800035f0 con 0x5570c599f2f0 2024-05-27T07:41:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.138+0000 7fa77be00640 1 -- 172.21.15.63:0/2510773474 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780011680 con 0x5570c599f2f0 2024-05-27T07:41:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.138+0000 7fa787e00640 1 -- 172.21.15.63:0/1554485151 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fa77c0e1e80 con 0x7fa79010be90 2024-05-27T07:41:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.139+0000 7fa78fe00640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7740b2aa0 0x7fa7740b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.139+0000 7fa79d376740 1 -- 172.21.15.63:0/2510773474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 msgr2=0x5570c5d6ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.139+0000 7fa79d376740 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 0x5570c5d6ada0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa78000bc70 tx=0x7fa7800107a0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.139+0000 7fa79d376740 1 -- 172.21.15.63:0/2510773474 shutdown_connections 2024-05-27T07:41:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.139+0000 7fa79d376740 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59a74d0 0x5570c5d6b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.139+0000 7fa79d376740 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 0x5570c5d6ada0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.139+0000 7fa79d376740 1 --2- 172.21.15.63:0/2510773474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59c81b0 0x5570c5d6a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.139+0000 7fa79d376740 1 -- 172.21.15.63:0/2510773474 >> 172.21.15.63:0/2510773474 conn(0x5570c59bc300 msgr2=0x5570c5d67010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.140+0000 7fa79d376740 1 -- 172.21.15.63:0/2510773474 shutdown_connections 2024-05-27T07:41:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.140+0000 7fa78fe00640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7740b2aa0 0x7fa7740b4fb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa788005e30 tx=0x7fa7880315b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.140+0000 7fa79d376740 1 -- 172.21.15.63:0/2510773474 wait complete. 2024-05-27T07:41:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.140+0000 7fa79d376740 1 -- start start 2024-05-27T07:41:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.141+0000 7fa79d376740 1 Processor -- start 2024-05-27T07:41:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.141+0000 7fa79d376740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c599f2f0 0x5570c5d74a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.141+0000 7fa79d376740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c59a74d0 0x5570c5d74f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.141+0000 7fa79d376740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d80cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.141+0000 7fa79d376740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5570c59b1790 con 0x5570c59c81b0 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.141+0000 7fa79d376740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5570c5c73bc0 con 0x5570c599f2f0 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.141+0000 7fa79d376740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5570c5d77990 con 0x5570c59a74d0 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.142+0000 7fa79d376740 10 client.0 _ll_register_callbacks cb 0x5570c5c51400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.142+0000 7fa78fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d80cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.142+0000 7fa78f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c599f2f0 0x5570c5d74a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.142+0000 7fa78f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c599f2f0 0x5570c5d74a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:36160/0 (socket says 172.21.15.63:36160) 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.142+0000 7fa78fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d80cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46116/0 (socket says 172.21.15.63:46116) 2024-05-27T07:41:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.142+0000 7fa78f400640 1 -- 172.21.15.63:0/3154340676 learned_addr learned my addr 172.21.15.63:0/3154340676 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.142+0000 7fa78ea00640 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c59a74d0 0x5570c5d74f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.143+0000 7fa78f400640 1 -- 172.21.15.63:0/3154340676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c59a74d0 msgr2=0x5570c5d74f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.143+0000 7fa78f400640 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c59a74d0 0x5570c5d74f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.143+0000 7fa78f400640 1 -- 172.21.15.63:0/3154340676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 msgr2=0x5570c5d80cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.143+0000 7fa78f400640 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d80cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.143+0000 7fa78f400640 1 -- 172.21.15.63:0/3154340676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c59e0210 con 0x5570c599f2f0 2024-05-27T07:41:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.143+0000 7fa78fe00640 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d80cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:41:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.143+0000 7fa78f400640 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c599f2f0 0x5570c5d74a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa780003ca0 tx=0x7fa78000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9b3c7f98d4f35e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.143+0000 7fa778c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:41:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.144+0000 7fa778c00640 1 -- 172.21.15.63:0/3154340676 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78002a060 con 0x5570c599f2f0 2024-05-27T07:41:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.144+0000 7fa79d376740 1 -- 172.21.15.63:0/3154340676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7800035f0 con 0x5570c599f2f0 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.144+0000 7fa778c00640 1 -- 172.21.15.63:0/3154340676 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fa780004000 con 0x5570c599f2f0 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.144+0000 7fa778c00640 1 -- 172.21.15.63:0/3154340676 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa780031050 con 0x5570c599f2f0 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.144+0000 7fa79d376740 10 client.36512 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.144+0000 7fa79d376740 1 -- 172.21.15.63:0/3154340676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5570c5aac620 con 0x5570c599f2f0 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.145+0000 7fa778c00640 1 -- 172.21.15.63:0/3154340676 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7800035f0 con 0x5570c599f2f0 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.145+0000 7fa778c00640 1 -- 172.21.15.63:0/3154340676 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa7800046c0 con 0x5570c599f2f0 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.145+0000 7fa79d376740 10 client.36512 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.145+0000 7fa79d376740 10 client.36512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.145+0000 7fa79d376740 20 client.36512 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.145+0000 7fa79d376740 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c5986d80 0x5570c5d86b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.145+0000 7fa79d376740 4 client.36512 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.145+0000 7fa79d376740 1 -- 172.21.15.63:0/3154340676 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570c5cddae0 con 0x5570c5986d80 2024-05-27T07:41:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.146+0000 7fa78ea00640 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c5986d80 0x5570c5d86b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.147+0000 7fa78ea00640 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c5986d80 0x5570c5d86b70 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ae41ae17fe7612 server_cookie=93efd208fbcfb4e4 in_seq=0 out_seq=0 2024-05-27T07:41:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.147+0000 7fa778c00640 10 client.36512 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.149+0000 7fa778c00640 1 -- 172.21.15.63:0/3154340676 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5570c5cddae0 con 0x5570c5986d80 2024-05-27T07:41:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.149+0000 7fa778c00640 10 client.36512 handle_command_reply: tid=0 2024-05-27T07:41:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.149+0000 7fa79d376740 1 client.36512 shutdown 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.149+0000 7fa79d376740 1 -- 172.21.15.63:0/3154340676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c599f2f0 msgr2=0x5570c5d74a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.149+0000 7fa79d376740 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c599f2f0 0x5570c5d74a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa780003ca0 tx=0x7fa78000bf90 comp rx=0 tx=0).stop 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.150+0000 7fa79d376740 1 -- 172.21.15.63:0/3154340676 shutdown_connections 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.150+0000 7fa79d376740 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c5986d80 0x5570c5d86b70 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.150+0000 7fa79d376740 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d80cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.150+0000 7fa79d376740 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c59a74d0 0x5570c5d74f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.150+0000 7fa79d376740 1 --2- 172.21.15.63:0/3154340676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c599f2f0 0x5570c5d74a20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.150+0000 7fa778c00640 0 client.36512 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.150+0000 7fa79d376740 1 -- 172.21.15.63:0/3154340676 >> 172.21.15.63:0/3154340676 conn(0x5570c59bc300 msgr2=0x5570c5d6a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.150+0000 7fa79d376740 1 -- 172.21.15.63:0/3154340676 shutdown_connections 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.150+0000 7fa79d376740 1 -- 172.21.15.63:0/3154340676 wait complete. 2024-05-27T07:41:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.150+0000 7fa79d376740 20 client.36512 trim_cache size 0 max 16384 2024-05-27T07:41:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.171+0000 7fa79d376740 1 Processor -- start 2024-05-27T07:41:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.172+0000 7fa79d376740 1 -- start start 2024-05-27T07:41:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.172+0000 7fa79d376740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 0x5570c5d80c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.172+0000 7fa79d376740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59a74d0 0x5570c5d76850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.172+0000 7fa79d376740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d71c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.172+0000 7fa79d376740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b5d40 con 0x5570c59c81b0 2024-05-27T07:41:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.172+0000 7fa79d376740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b3610 con 0x5570c59a74d0 2024-05-27T07:41:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.172+0000 7fa79d376740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b1480 con 0x5570c599f2f0 2024-05-27T07:41:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.172+0000 7fa78fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d71c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.173+0000 7fa78fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d71c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46120/0 (socket says 172.21.15.63:46120) 2024-05-27T07:41:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.173+0000 7fa78fe00640 1 -- 172.21.15.63:0/1100889581 learned_addr learned my addr 172.21.15.63:0/1100889581 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.173+0000 7fa78f400640 1 --2- 172.21.15.63:0/1100889581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 0x5570c5d80c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.173+0000 7fa78ea00640 1 --2- 172.21.15.63:0/1100889581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59a74d0 0x5570c5d76850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.173+0000 7fa78fe00640 1 -- 172.21.15.63:0/1100889581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 msgr2=0x5570c5d80c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.173+0000 7fa78fe00640 1 --2- 172.21.15.63:0/1100889581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 0x5570c5d80c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.173+0000 7fa78fe00640 1 -- 172.21.15.63:0/1100889581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59a74d0 msgr2=0x5570c5d76850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.173+0000 7fa78fe00640 1 --2- 172.21.15.63:0/1100889581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59a74d0 0x5570c5d76850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.173+0000 7fa78fe00640 1 -- 172.21.15.63:0/1100889581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c5d61050 con 0x5570c59c81b0 2024-05-27T07:41:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.173+0000 7fa78ea00640 1 --2- 172.21.15.63:0/1100889581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59a74d0 0x5570c5d76850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.174+0000 7fa78fe00640 1 --2- 172.21.15.63:0/1100889581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d71c60 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa788009ca0 tx=0x7fa78803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e31e28f61a36d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.174+0000 7fa77b400640 1 -- 172.21.15.63:0/1100889581 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788055070 con 0x5570c59c81b0 2024-05-27T07:41:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.174+0000 7fa77b400640 1 -- 172.21.15.63:0/1100889581 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa788036040 con 0x5570c59c81b0 2024-05-27T07:41:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.174+0000 7fa77b400640 1 -- 172.21.15.63:0/1100889581 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788045460 con 0x5570c59c81b0 2024-05-27T07:41:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.176+0000 7fa79d376740 1 -- 172.21.15.63:0/1100889581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 msgr2=0x5570c5d71c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.176+0000 7fa79d376740 1 --2- 172.21.15.63:0/1100889581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d71c60 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa788009ca0 tx=0x7fa78803a000 comp rx=0 tx=0).stop 2024-05-27T07:41:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.176+0000 7fa79d376740 1 -- 172.21.15.63:0/1100889581 shutdown_connections 2024-05-27T07:41:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.176+0000 7fa79d376740 1 --2- 172.21.15.63:0/1100889581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c59c81b0 0x5570c5d71c60 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.176+0000 7fa79d376740 1 --2- 172.21.15.63:0/1100889581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59a74d0 0x5570c5d76850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.176+0000 7fa79d376740 1 --2- 172.21.15.63:0/1100889581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c599f2f0 0x5570c5d80c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.176+0000 7fa79d376740 1 -- 172.21.15.63:0/1100889581 >> 172.21.15.63:0/1100889581 conn(0x5570c59bc300 msgr2=0x5570c5d76140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.176+0000 7fa79d376740 1 -- 172.21.15.63:0/1100889581 shutdown_connections 2024-05-27T07:41:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.176+0000 7fa79d376740 1 -- 172.21.15.63:0/1100889581 wait complete. 2024-05-27T07:41:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.176+0000 7fa79d376740 1 -- start start 2024-05-27T07:41:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.177+0000 7fa79d376740 1 Processor -- start 2024-05-27T07:41:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.177+0000 7fa79d376740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c599f2f0 0x5570c5d888f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa79d376740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c59a74d0 0x5570c5d88e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa79d376740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59c81b0 0x5570c5d89370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa79d376740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b1ad0 con 0x5570c599f2f0 2024-05-27T07:41:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa79d376740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b2f40 con 0x5570c59c81b0 2024-05-27T07:41:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa79d376740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b21a0 con 0x5570c59a74d0 2024-05-27T07:41:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa79d376740 10 client.0 _ll_register_callbacks cb 0x7fa78000bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa78f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c599f2f0 0x5570c5d888f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa78f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c599f2f0 0x5570c5d888f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46136/0 (socket says 172.21.15.63:46136) 2024-05-27T07:41:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa78f400640 1 -- 172.21.15.63:0/2761329328 learned_addr learned my addr 172.21.15.63:0/2761329328 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa78ea00640 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c59a74d0 0x5570c5d88e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.178+0000 7fa78f400640 1 -- 172.21.15.63:0/2761329328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c59a74d0 msgr2=0x5570c5d88e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.179+0000 7fa78f400640 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c59a74d0 0x5570c5d88e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.179+0000 7fa78f400640 1 -- 172.21.15.63:0/2761329328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59c81b0 msgr2=0x5570c5d89370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.179+0000 7fa78f400640 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59c81b0 0x5570c5d89370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.179+0000 7fa78f400640 1 -- 172.21.15.63:0/2761329328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c5decfc0 con 0x5570c599f2f0 2024-05-27T07:41:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.179+0000 7fa78f400640 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c599f2f0 0x5570c5d888f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa780010ca0 tx=0x7fa780003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdb8740a3af3045 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.179+0000 7fa778c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.179+0000 7fa778c00640 1 -- 172.21.15.63:0/2761329328 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78001f070 con 0x5570c599f2f0 2024-05-27T07:41:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.179+0000 7fa778c00640 1 -- 172.21.15.63:0/2761329328 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fa780003a30 con 0x5570c599f2f0 2024-05-27T07:41:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.180+0000 7fa79d376740 1 -- 172.21.15.63:0/2761329328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa788036040 con 0x5570c599f2f0 2024-05-27T07:41:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.180+0000 7fa778c00640 1 -- 172.21.15.63:0/2761329328 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa78002a070 con 0x5570c599f2f0 2024-05-27T07:41:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.180+0000 7fa79d376740 10 client.16932 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.180+0000 7fa79d376740 1 -- 172.21.15.63:0/2761329328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5570c5dee150 con 0x5570c599f2f0 2024-05-27T07:41:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.180+0000 7fa778c00640 1 -- 172.21.15.63:0/2761329328 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa788036040 con 0x5570c599f2f0 2024-05-27T07:41:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.181+0000 7fa778c00640 1 -- 172.21.15.63:0/2761329328 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa780003f30 con 0x5570c599f2f0 2024-05-27T07:41:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.181+0000 7fa79d376740 10 client.16932 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.181+0000 7fa79d376740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.181+0000 7fa79d376740 20 client.16932 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.181+0000 7fa79d376740 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c595c1c0 0x5570c5df0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.181+0000 7fa79d376740 4 client.16932 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:41:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.181+0000 7fa79d376740 1 -- 172.21.15.63:0/2761329328 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570c5cd72c0 con 0x5570c595c1c0 2024-05-27T07:41:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.182+0000 7fa78ea00640 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c595c1c0 0x5570c5df0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.182+0000 7fa78ea00640 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c595c1c0 0x5570c5df0870 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=944a0312bcfa0025 server_cookie=e2118c5853e89bc in_seq=0 out_seq=0 2024-05-27T07:41:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.183+0000 7fa778c00640 10 client.16932 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa778c00640 1 -- 172.21.15.63:0/2761329328 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5570c5cd72c0 con 0x5570c595c1c0 2024-05-27T07:41:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa778c00640 10 client.16932 handle_command_reply: tid=0 2024-05-27T07:41:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 client.16932 shutdown 2024-05-27T07:41:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 -- 172.21.15.63:0/2761329328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c599f2f0 msgr2=0x5570c5d888f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c599f2f0 0x5570c5d888f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa780010ca0 tx=0x7fa780003d70 comp rx=0 tx=0).stop 2024-05-27T07:41:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 -- 172.21.15.63:0/2761329328 shutdown_connections 2024-05-27T07:41:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c595c1c0 0x5570c5df0870 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c59c81b0 0x5570c5d89370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c59a74d0 0x5570c5d88e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 --2- 172.21.15.63:0/2761329328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c599f2f0 0x5570c5d888f0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 -- 172.21.15.63:0/2761329328 >> 172.21.15.63:0/2761329328 conn(0x5570c59bc300 msgr2=0x5570c5d75ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa778c00640 0 client.16932 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 -- 172.21.15.63:0/2761329328 shutdown_connections 2024-05-27T07:41:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.215+0000 7fa79d376740 1 -- 172.21.15.63:0/2761329328 wait complete. 2024-05-27T07:41:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.216+0000 7fa79d376740 20 client.16932 trim_cache size 0 max 16384 2024-05-27T07:41:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:41:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:41:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:41:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.216+0000 7fa796200640 1 -- 172.21.15.63:0/1554485151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7740b2aa0 msgr2=0x7fa7740b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.216+0000 7fa796200640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7740b2aa0 0x7fa7740b4fb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa788005e30 tx=0x7fa7880315b0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.216+0000 7fa796200640 1 -- 172.21.15.63:0/1554485151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa79010be90 msgr2=0x7fa7901a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.216+0000 7fa796200640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa79010be90 0x7fa7901a2af0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00d740 tx=0x7fa77c00dc00 comp rx=0 tx=0).stop 2024-05-27T07:41:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.217+0000 7fa796200640 1 -- 172.21.15.63:0/1554485151 shutdown_connections 2024-05-27T07:41:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.217+0000 7fa796200640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7740b2aa0 0x7fa7740b4fb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.217+0000 7fa796200640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa79010c330 0x7fa79019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.217+0000 7fa796200640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa79010be90 0x7fa7901a2af0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.217+0000 7fa796200640 1 --2- 172.21.15.63:0/1554485151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7901065e0 0x7fa7901a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.217+0000 7fa796200640 1 -- 172.21.15.63:0/1554485151 >> 172.21.15.63:0/1554485151 conn(0x7fa790101530 msgr2=0x7fa7901089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.217+0000 7fa796200640 1 -- 172.21.15.63:0/1554485151 shutdown_connections 2024-05-27T07:41:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:32.217+0000 7fa796200640 1 -- 172.21.15.63:0/1554485151 wait complete. 2024-05-27T07:41:32.585 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:33.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:33 smithi086 ceph-mon[34687]: pgmap v246: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:41:33.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:33 smithi149 ceph-mon[35408]: pgmap v246: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:41:33.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:33 smithi063 ceph-mon[31613]: pgmap v246: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:41:34.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.963+0000 7f26f8a00700 1 -- 172.21.15.63:0/398667095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f4107de0 msgr2=0x7f26f41081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:34.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.963+0000 7f26f8a00700 1 --2- 172.21.15.63:0/398667095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f4107de0 0x7f26f41081f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f26e00099c0 tx=0x7f26e0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:41:34.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.963+0000 7f26f8a00700 1 -- 172.21.15.63:0/398667095 shutdown_connections 2024-05-27T07:41:34.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.963+0000 7f26f8a00700 1 --2- 172.21.15.63:0/398667095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26f4108730 0x7f26f4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:34.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.963+0000 7f26f8a00700 1 --2- 172.21.15.63:0/398667095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26f41034b0 0x7f26f41038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:34.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.963+0000 7f26f8a00700 1 --2- 172.21.15.63:0/398667095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f4107de0 0x7f26f41081f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:34.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.963+0000 7f26f8a00700 1 -- 172.21.15.63:0/398667095 >> 172.21.15.63:0/398667095 conn(0x7f26f40fedb0 msgr2=0x7f26f41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:34.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.963+0000 7f26f8a00700 1 -- 172.21.15.63:0/398667095 shutdown_connections 2024-05-27T07:41:34.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.963+0000 7f26f8a00700 1 -- 172.21.15.63:0/398667095 wait complete. 2024-05-27T07:41:34.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.965+0000 7f26f8a00700 1 Processor -- start 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.965+0000 7f26f8a00700 1 -- start start 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.965+0000 7f26f8a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26f41034b0 0x7f26f41a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.965+0000 7f26f8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26f4107de0 0x7f26f41a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.965+0000 7f26f8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f4108730 0x7f26f419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.965+0000 7f26f8a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41131f0 con 0x7f26f4108730 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.965+0000 7f26f8a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4113070 con 0x7f26f4107de0 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.965+0000 7f26f8a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4113370 con 0x7f26f41034b0 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.966+0000 7f26f2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f4108730 0x7f26f419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.966+0000 7f26f2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f4108730 0x7f26f419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46154/0 (socket says 172.21.15.63:46154) 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.966+0000 7f26f2800700 1 -- 172.21.15.63:0/22393106 learned_addr learned my addr 172.21.15.63:0/22393106 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:34.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.966+0000 7f26f1400700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26f4107de0 0x7f26f41a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:34.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.966+0000 7f26f1e00700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26f41034b0 0x7f26f41a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:34.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.967+0000 7f26f2800700 1 -- 172.21.15.63:0/22393106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26f41034b0 msgr2=0x7f26f41a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:34.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.967+0000 7f26f2800700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26f41034b0 0x7f26f41a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:34.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.967+0000 7f26f2800700 1 -- 172.21.15.63:0/22393106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26f4107de0 msgr2=0x7f26f41a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:34.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.967+0000 7f26f2800700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26f4107de0 0x7f26f41a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:34.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.967+0000 7f26f2800700 1 -- 172.21.15.63:0/22393106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f419b1e0 con 0x7f26f4108730 2024-05-27T07:41:34.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.967+0000 7f26f2800700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f4108730 0x7f26f419ab00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f26ec007ee0 tx=0x7f26ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:34.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.967+0000 7f26eaa00700 1 -- 172.21.15.63:0/22393106 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec004bb0 con 0x7f26f4108730 2024-05-27T07:41:34.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.967+0000 7f26eaa00700 1 -- 172.21.15.63:0/22393106 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26ec004d50 con 0x7f26f4108730 2024-05-27T07:41:34.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.968+0000 7f26f8a00700 1 -- 172.21.15.63:0/22393106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f419b4d0 con 0x7f26f4108730 2024-05-27T07:41:34.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.968+0000 7f26eaa00700 1 -- 172.21.15.63:0/22393106 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0058b0 con 0x7f26f4108730 2024-05-27T07:41:34.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.968+0000 7f26f8a00700 1 -- 172.21.15.63:0/22393106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f41a7770 con 0x7f26f4108730 2024-05-27T07:41:34.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.969+0000 7f26f8a00700 1 -- 172.21.15.63:0/22393106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f40512b0 con 0x7f26f4108730 2024-05-27T07:41:34.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.972+0000 7f26eaa00700 1 -- 172.21.15.63:0/22393106 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f26ec01a020 con 0x7f26f4108730 2024-05-27T07:41:34.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.973+0000 7f26eaa00700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26d80b29f0 0x7f26d80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:34.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.974+0000 7f26eaa00700 1 -- 172.21.15.63:0/22393106 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f26ec0e26f0 con 0x7f26f4108730 2024-05-27T07:41:34.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.974+0000 7f26f1e00700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26d80b29f0 0x7f26d80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:34.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.975+0000 7f26f1e00700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26d80b29f0 0x7f26d80b4f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26f419bc40 tx=0x7f26e400b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:34.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:34.977+0000 7f26eaa00700 1 -- 172.21.15.63:0/22393106 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26ec010070 con 0x7f26f4108730 2024-05-27T07:41:35.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.318+0000 7f26f8a00700 1 -- 172.21.15.63:0/22393106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26f402e5e0 con 0x7f26f4108730 2024-05-27T07:41:35.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.320+0000 7f26eaa00700 1 -- 172.21.15.63:0/22393106 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f26ec0a9cb0 con 0x7f26f4108730 2024-05-27T07:41:35.319 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:41:35.319 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:41:35.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.323+0000 7f26f8a00700 1 -- 172.21.15.63:0/22393106 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26d80b29f0 msgr2=0x7f26d80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:35.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.323+0000 7f26f8a00700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26d80b29f0 0x7f26d80b4f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26f419bc40 tx=0x7f26e400b410 comp rx=0 tx=0).stop 2024-05-27T07:41:35.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.323+0000 7f26f8a00700 1 -- 172.21.15.63:0/22393106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f4108730 msgr2=0x7f26f419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:35.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.323+0000 7f26f8a00700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f4108730 0x7f26f419ab00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f26ec007ee0 tx=0x7f26ec00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:35.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.324+0000 7f26f8a00700 1 -- 172.21.15.63:0/22393106 shutdown_connections 2024-05-27T07:41:35.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.324+0000 7f26f8a00700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26d80b29f0 0x7f26d80b4f00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:35.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.324+0000 7f26f8a00700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26f4108730 0x7f26f419ab00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:35.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.324+0000 7f26f8a00700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26f41034b0 0x7f26f41a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:35.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.324+0000 7f26f8a00700 1 --2- 172.21.15.63:0/22393106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26f4107de0 0x7f26f41a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:35.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.324+0000 7f26f8a00700 1 -- 172.21.15.63:0/22393106 >> 172.21.15.63:0/22393106 conn(0x7f26f40fedb0 msgr2=0x7f26f406b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:35.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.324+0000 7f26f8a00700 1 -- 172.21.15.63:0/22393106 shutdown_connections 2024-05-27T07:41:35.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:35.324+0000 7f26f8a00700 1 -- 172.21.15.63:0/22393106 wait complete. 2024-05-27T07:41:35.324 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:41:35.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:35 smithi086 ceph-mon[34687]: pgmap v247: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:41:35.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:35 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/22393106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:41:35.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:35 smithi149 ceph-mon[35408]: pgmap v247: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:41:35.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:35 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/22393106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:41:35.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:35 smithi063 ceph-mon[31613]: pgmap v247: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:41:35.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:35 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/22393106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:41:36.272 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:41:36.848 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:37.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:37 smithi086 ceph-mon[34687]: pgmap v248: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:41:37.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:37 smithi149 ceph-mon[35408]: pgmap v248: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:41:37.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:37 smithi063 ceph-mon[31613]: pgmap v248: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:41:39.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.277+0000 7fd7e4000700 1 -- 172.21.15.63:0/1870232528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7dc103ab0 msgr2=0x7fd7dc1107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:39.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.277+0000 7fd7e4000700 1 --2- 172.21.15.63:0/1870232528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7dc103ab0 0x7fd7dc1107d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009990 tx=0x7fd7c8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.277+0000 7fd7e4000700 1 -- 172.21.15.63:0/1870232528 shutdown_connections 2024-05-27T07:41:39.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.277+0000 7fd7e4000700 1 --2- 172.21.15.63:0/1870232528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7dc110d10 0x7fd7dc113130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.277+0000 7fd7e4000700 1 --2- 172.21.15.63:0/1870232528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7dc103160 0x7fd7dc103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.277+0000 7fd7e4000700 1 --2- 172.21.15.63:0/1870232528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7dc103ab0 0x7fd7dc1107d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.277+0000 7fd7e4000700 1 -- 172.21.15.63:0/1870232528 >> 172.21.15.63:0/1870232528 conn(0x7fd7dc078df0 msgr2=0x7fd7dc079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:39.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.277+0000 7fd7e4000700 1 -- 172.21.15.63:0/1870232528 shutdown_connections 2024-05-27T07:41:39.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.277+0000 7fd7e4000700 1 -- 172.21.15.63:0/1870232528 wait complete. 2024-05-27T07:41:39.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.278+0000 7fd7e4000700 1 Processor -- start 2024-05-27T07:41:39.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.279+0000 7fd7e4000700 1 -- start start 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.279+0000 7fd7e4000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7dc103160 0x7fd7dc19d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.279+0000 7fd7e4000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7dc103ab0 0x7fd7dc19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.279+0000 7fd7e4000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7dc110d10 0x7fd7dc1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.279+0000 7fd7e4000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc116590 con 0x7fd7dc110d10 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.279+0000 7fd7e4000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc116410 con 0x7fd7dc103160 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.279+0000 7fd7e4000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc116710 con 0x7fd7dc103ab0 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.280+0000 7fd7e2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7dc110d10 0x7fd7dc1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.280+0000 7fd7e2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7dc110d10 0x7fd7dc1a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39406/0 (socket says 172.21.15.63:39406) 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.280+0000 7fd7e2000700 1 -- 172.21.15.63:0/419624633 learned_addr learned my addr 172.21.15.63:0/419624633 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.280+0000 7fd7e1600700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7dc103160 0x7fd7dc19d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.280+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7dc103ab0 0x7fd7dc19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.280+0000 7fd7e2000700 1 -- 172.21.15.63:0/419624633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7dc103ab0 msgr2=0x7fd7dc19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.280+0000 7fd7e2000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7dc103ab0 0x7fd7dc19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.280+0000 7fd7e2000700 1 -- 172.21.15.63:0/419624633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7dc103160 msgr2=0x7fd7dc19d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.280+0000 7fd7e2000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7dc103160 0x7fd7dc19d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.280+0000 7fd7e2000700 1 -- 172.21.15.63:0/419624633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7dc1a1f30 con 0x7fd7dc110d10 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.281+0000 7fd7e2000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7dc110d10 0x7fd7dc1a1890 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4007ee0 tx=0x7fd7d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.281+0000 7fd7da000700 1 -- 172.21.15.63:0/419624633 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d400dd20 con 0x7fd7dc110d10 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.281+0000 7fd7da000700 1 -- 172.21.15.63:0/419624633 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7d4004020 con 0x7fd7dc110d10 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.281+0000 7fd7e4000700 1 -- 172.21.15.63:0/419624633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7dc1a2220 con 0x7fd7dc110d10 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.281+0000 7fd7da000700 1 -- 172.21.15.63:0/419624633 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4015660 con 0x7fd7dc110d10 2024-05-27T07:41:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.281+0000 7fd7e4000700 1 -- 172.21.15.63:0/419624633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7dc1a9b40 con 0x7fd7dc110d10 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.285+0000 7fd7da000700 1 -- 172.21.15.63:0/419624633 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd7d40041d0 con 0x7fd7dc110d10 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.287+0000 7fd7f2b41b80 1 Processor -- start 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.287+0000 7fd7f2b41b80 1 -- start start 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.287+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.287+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 0x55f39fdab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.287+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdabdb0 0x55f39fdac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.287+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f39fcd9e40 con 0x55f39fc2ce40 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.287+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f39fcc99e0 con 0x55f39fdb1d30 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.287+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f39fccda20 con 0x55f39fdabdb0 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.287+0000 7fd7da000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7c40b29f0 0x7fd7c40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.288+0000 7fd7e0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.288+0000 7fd7e0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39414/0 (socket says 172.21.15.63:39414) 2024-05-27T07:41:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.288+0000 7fd7e1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 0x55f39fdab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.288+0000 7fd7e0c00700 1 -- 172.21.15.63:0/3463126306 learned_addr learned my addr 172.21.15.63:0/3463126306 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.288+0000 7fd7da000700 1 -- 172.21.15.63:0/419624633 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fd7d40e2370 con 0x7fd7dc110d10 2024-05-27T07:41:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.288+0000 7fd7e2000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7c40b29f0 0x7fd7c40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.288+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/3463126306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdabdb0 0x55f39fdac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.289+0000 7fd7e1600700 1 -- 172.21.15.63:0/3463126306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdabdb0 msgr2=0x55f39fdac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.289+0000 7fd7e1600700 1 --2- 172.21.15.63:0/3463126306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdabdb0 0x55f39fdac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.289+0000 7fd7e1600700 1 -- 172.21.15.63:0/3463126306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 msgr2=0x55f39fdb17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.289+0000 7fd7e1600700 1 --2- 172.21.15.63:0/3463126306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.289+0000 7fd7e1600700 1 -- 172.21.15.63:0/3463126306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39fc92420 con 0x55f39fdb1d30 2024-05-27T07:41:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.289+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/3463126306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:41:39.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.289+0000 7fd7e2000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7c40b29f0 0x7fd7c40b4f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4004a20 tx=0x7fd7d40096c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:39.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.289+0000 7fd7e1600700 1 --2- 172.21.15.63:0/3463126306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 0x55f39fdab870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009cb0 tx=0x7fd7d0011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74621b3d584e17e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.290+0000 7fd7cea00700 1 -- 172.21.15.63:0/3463126306 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0011d20 con 0x55f39fdb1d30 2024-05-27T07:41:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.290+0000 7fd7cea00700 1 -- 172.21.15.63:0/3463126306 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7d00044e0 con 0x55f39fdb1d30 2024-05-27T07:41:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.290+0000 7fd7cea00700 1 -- 172.21.15.63:0/3463126306 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0005280 con 0x55f39fdb1d30 2024-05-27T07:41:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.291+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3463126306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 msgr2=0x55f39fdab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.291+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3463126306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 0x55f39fdab870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009cb0 tx=0x7fd7d0011440 comp rx=0 tx=0).stop 2024-05-27T07:41:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.291+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3463126306 shutdown_connections 2024-05-27T07:41:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.291+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3463126306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdabdb0 0x55f39fdac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.291+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3463126306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.291+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3463126306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 0x55f39fdab870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.291+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3463126306 >> 172.21.15.63:0/3463126306 conn(0x55f39fc8cfc0 msgr2=0x55f39fda68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.291+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3463126306 shutdown_connections 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.291+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3463126306 wait complete. 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.292+0000 7fd7f2b41b80 1 -- start start 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.292+0000 7fd7f2b41b80 1 Processor -- start 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.293+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.293+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdabdb0 0x55f39fdb5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.293+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 0x55f39fdb5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.293+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f39fbcffa0 con 0x55f39fc2ce40 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.293+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f39fc6a760 con 0x55f39fdabdb0 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.293+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f39fdb8800 con 0x55f39fdb1d30 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.293+0000 7fd7f2b41b80 10 client.0 _ll_register_callbacks cb 0x55f39fc73c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.294+0000 7fd7e0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.294+0000 7fd7e0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39430/0 (socket says 172.21.15.63:39430) 2024-05-27T07:41:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.294+0000 7fd7e0c00700 1 -- 172.21.15.63:0/4252896514 learned_addr learned my addr 172.21.15.63:0/4252896514 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.294+0000 7fd7e1600700 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdabdb0 0x55f39fdb5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.294+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 0x55f39fdb5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.294+0000 7fd7e0c00700 1 -- 172.21.15.63:0/4252896514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 msgr2=0x55f39fdb5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.294+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 0x55f39fdb5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.294+0000 7fd7e0c00700 1 -- 172.21.15.63:0/4252896514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdabdb0 msgr2=0x55f39fdb5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.294+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdabdb0 0x55f39fdb5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.294+0000 7fd7e0c00700 1 -- 172.21.15.63:0/4252896514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f39fc198d0 con 0x55f39fc2ce40 2024-05-27T07:41:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.295+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb5510 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8015000 tx=0x7fd7c8005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba6904319fa5728 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.295+0000 7fd7c3400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.295+0000 7fd7c3400700 1 -- 172.21.15.63:0/4252896514 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800bac0 con 0x55f39fc2ce40 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.295+0000 7fd7c3400700 1 -- 172.21.15.63:0/4252896514 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fd7c800bc60 con 0x55f39fc2ce40 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.295+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/4252896514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d00044e0 con 0x55f39fc2ce40 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.296+0000 7fd7c3400700 1 -- 172.21.15.63:0/4252896514 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd7c800be60 con 0x55f39fc2ce40 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.296+0000 7fd7f2b41b80 10 client.16950 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.296+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/4252896514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f39fc9a060 con 0x55f39fc2ce40 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.296+0000 7fd7c3400700 1 -- 172.21.15.63:0/4252896514 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7d00044e0 con 0x55f39fc2ce40 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.296+0000 7fd7c3400700 1 -- 172.21.15.63:0/4252896514 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd7c8027040 con 0x55f39fc2ce40 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.297+0000 7fd7f2b41b80 10 client.16950 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.297+0000 7fd7f2b41b80 10 client.16950 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:41:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.297+0000 7fd7f2b41b80 20 client.16950 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:39.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.297+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f39fdcdf70 0x55f39fdd03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.297+0000 7fd7f2b41b80 4 client.16950 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:39.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.297+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/4252896514 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f39fcdbd90 con 0x55f39fdcdf70 2024-05-27T07:41:39.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.298+0000 7fd7e1600700 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f39fdcdf70 0x55f39fdd03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.299+0000 7fd7e1600700 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f39fdcdf70 0x55f39fdd03f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a428af17c55e58de server_cookie=2db3ad19a784c80c in_seq=0 out_seq=0 2024-05-27T07:41:39.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.299+0000 7fd7c3400700 10 client.16950 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:41:39.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7c3400700 1 -- 172.21.15.63:0/4252896514 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f39fcdbd90 con 0x55f39fdcdf70 2024-05-27T07:41:39.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7c3400700 10 client.16950 handle_command_reply: tid=0 2024-05-27T07:41:39.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7f2b41b80 1 client.16950 shutdown 2024-05-27T07:41:39.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/4252896514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 msgr2=0x55f39fdb5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:39.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb5510 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8015000 tx=0x7fd7c8005fb0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/4252896514 shutdown_connections 2024-05-27T07:41:39.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 0x55f39fdb5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f39fdcdf70 0x55f39fdd03f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fc2ce40 0x55f39fdb5510 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7c3400700 0 client.16950 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:41:39.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/4252896514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdabdb0 0x55f39fdb5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.301+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/4252896514 >> 172.21.15.63:0/4252896514 conn(0x55f39fc8cfc0 msgr2=0x55f39fda6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:39.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.302+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/4252896514 shutdown_connections 2024-05-27T07:41:39.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.302+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/4252896514 wait complete. 2024-05-27T07:41:39.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.302+0000 7fd7f2b41b80 20 client.16950 trim_cache size 0 max 16384 2024-05-27T07:41:39.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.327+0000 7fd7f2b41b80 1 Processor -- start 2024-05-27T07:41:39.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.327+0000 7fd7f2b41b80 1 -- start start 2024-05-27T07:41:39.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.327+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fc2ce40 0x55f39fdab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.327+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdafc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.327+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 0x55f39fdca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.327+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1b0b10 con 0x55f39fdabdb0 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.327+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1ae3a0 con 0x55f39fc2ce40 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.327+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1ac240 con 0x55f39fdb1d30 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e1600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdafc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e1600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdafc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39436/0 (socket says 172.21.15.63:39436) 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e1600700 1 -- 172.21.15.63:0/3127353967 learned_addr learned my addr 172.21.15.63:0/3127353967 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/3127353967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fc2ce40 0x55f39fdab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/3127353967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 0x55f39fdca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e1600700 1 -- 172.21.15.63:0/3127353967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 msgr2=0x55f39fdca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:39.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e1600700 1 --2- 172.21.15.63:0/3127353967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 0x55f39fdca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e1600700 1 -- 172.21.15.63:0/3127353967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fc2ce40 msgr2=0x55f39fdab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:39.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e1600700 1 --2- 172.21.15.63:0/3127353967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fc2ce40 0x55f39fdab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e1600700 1 -- 172.21.15.63:0/3127353967 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39fdb6300 con 0x55f39fdabdb0 2024-05-27T07:41:39.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.328+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/3127353967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 0x55f39fdca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:41:39.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.329+0000 7fd7e1600700 1 --2- 172.21.15.63:0/3127353967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdafc40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00092a0 tx=0x7fd7d0004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57ce943417b6d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:39.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.329+0000 7fd7ce000700 1 -- 172.21.15.63:0/3127353967 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0004490 con 0x55f39fdabdb0 2024-05-27T07:41:39.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.329+0000 7fd7ce000700 1 -- 172.21.15.63:0/3127353967 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7d00047d0 con 0x55f39fdabdb0 2024-05-27T07:41:39.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.329+0000 7fd7ce000700 1 -- 172.21.15.63:0/3127353967 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0003a60 con 0x55f39fdabdb0 2024-05-27T07:41:39.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.330+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3127353967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 msgr2=0x55f39fdafc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.330+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3127353967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdafc40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00092a0 tx=0x7fd7d0004050 comp rx=0 tx=0).stop 2024-05-27T07:41:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.330+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3127353967 shutdown_connections 2024-05-27T07:41:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.330+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3127353967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fdb1d30 0x55f39fdca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.330+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3127353967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fc2ce40 0x55f39fdab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.330+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3127353967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdafc40 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.330+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3127353967 >> 172.21.15.63:0/3127353967 conn(0x55f39fc8cfc0 msgr2=0x55f39fdb3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.331+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3127353967 shutdown_connections 2024-05-27T07:41:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.331+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3127353967 wait complete. 2024-05-27T07:41:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.331+0000 7fd7f2b41b80 1 -- start start 2024-05-27T07:41:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.331+0000 7fd7f2b41b80 1 Processor -- start 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.332+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fc2ce40 0x55f39fdc6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.332+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdc6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.332+0000 7fd7f2b41b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 0x55f39fdd50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.332+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1ac860 con 0x55f39fdabdb0 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.332+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1adcd0 con 0x55f39fdb1d30 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.332+0000 7fd7f2b41b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc1acf30 con 0x55f39fc2ce40 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.332+0000 7fd7f2b41b80 10 client.0 _ll_register_callbacks cb 0x55f39fc73c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.333+0000 7fd7e0c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fc2ce40 0x55f39fdc6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.333+0000 7fd7e0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 0x55f39fdd50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.333+0000 7fd7e1600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdc6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.333+0000 7fd7e0c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fc2ce40 0x55f39fdc6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:58946/0 (socket says 172.21.15.63:58946) 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.333+0000 7fd7e0c00700 1 -- 172.21.15.63:0/3406197569 learned_addr learned my addr 172.21.15.63:0/3406197569 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.333+0000 7fd7e1600700 1 -- 172.21.15.63:0/3406197569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fc2ce40 msgr2=0x55f39fdc6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.333+0000 7fd7e1600700 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fc2ce40 0x55f39fdc6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.333+0000 7fd7e1600700 1 -- 172.21.15.63:0/3406197569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 msgr2=0x55f39fdd50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.333+0000 7fd7e1600700 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 0x55f39fdd50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.333+0000 7fd7e1600700 1 -- 172.21.15.63:0/3406197569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f39fdd5850 con 0x55f39fdabdb0 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.334+0000 7fd7e1600700 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdc6b60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00040e0 tx=0x7fd7d0003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c822e52fb560ec02 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.334+0000 7fd7c3400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.334+0000 7fd7c3400700 1 -- 172.21.15.63:0/3406197569 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0004590 con 0x55f39fdabdb0 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.334+0000 7fd7c3400700 1 -- 172.21.15.63:0/3406197569 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fd7d001e070 con 0x55f39fdabdb0 2024-05-27T07:41:39.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.334+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3406197569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d00047d0 con 0x55f39fdabdb0 2024-05-27T07:41:39.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7c3400700 1 -- 172.21.15.63:0/3406197569 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd7d003e050 con 0x55f39fdabdb0 2024-05-27T07:41:39.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7f2b41b80 10 client.16956 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:39.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3406197569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f39fe384f0 con 0x55f39fdabdb0 2024-05-27T07:41:39.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7c3400700 1 -- 172.21.15.63:0/3406197569 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7d00047d0 con 0x55f39fdabdb0 2024-05-27T07:41:39.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7c3400700 1 -- 172.21.15.63:0/3406197569 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd7d0018070 con 0x55f39fdabdb0 2024-05-27T07:41:39.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7f2b41b80 10 client.16956 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:39.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7f2b41b80 10 client.16956 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:41:39.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7f2b41b80 20 client.16956 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:39.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f39fdcdf70 0x55f39fe3a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:39.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7f2b41b80 4 client.16956 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:41:39.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.335+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3406197569 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f39fcd5570 con 0x55f39fdcdf70 2024-05-27T07:41:39.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.336+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f39fdcdf70 0x55f39fe3a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:39.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.337+0000 7fd7e0c00700 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f39fdcdf70 0x55f39fe3a900 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21dd6ea899a2dc3e server_cookie=86155fdeb23ee6e5 in_seq=0 out_seq=0 2024-05-27T07:41:39.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.337+0000 7fd7c3400700 10 client.16956 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:41:39.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7c3400700 1 -- 172.21.15.63:0/3406197569 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f39fcd5570 con 0x55f39fdcdf70 2024-05-27T07:41:39.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7c3400700 10 client.16956 handle_command_reply: tid=0 2024-05-27T07:41:39.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 client.16956 shutdown 2024-05-27T07:41:39.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3406197569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 msgr2=0x55f39fdc6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:39.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdc6b60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00040e0 tx=0x7fd7d0003800 comp rx=0 tx=0).stop 2024-05-27T07:41:39.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3406197569 shutdown_connections 2024-05-27T07:41:39.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39fdb1d30 0x55f39fdd50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f39fc2ce40 0x55f39fdc6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f39fdcdf70 0x55f39fe3a900 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 --2- 172.21.15.63:0/3406197569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f39fdabdb0 0x55f39fdc6b60 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7c3400700 0 client.16956 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:41:39.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3406197569 >> 172.21.15.63:0/3406197569 conn(0x55f39fc8cfc0 msgr2=0x55f39fdaf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:39.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3406197569 shutdown_connections 2024-05-27T07:41:39.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 1 -- 172.21.15.63:0/3406197569 wait complete. 2024-05-27T07:41:39.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.338+0000 7fd7f2b41b80 20 client.16956 trim_cache size 0 max 16384 2024-05-27T07:41:39.339 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:41:39.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.339+0000 7fd7e4000700 1 -- 172.21.15.63:0/419624633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7c40b29f0 msgr2=0x7fd7c40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:39.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.339+0000 7fd7e4000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7c40b29f0 0x7fd7c40b4f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4004a20 tx=0x7fd7d40096c0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.339+0000 7fd7e4000700 1 -- 172.21.15.63:0/419624633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7dc110d10 msgr2=0x7fd7dc1a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:39.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.339+0000 7fd7e4000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7dc110d10 0x7fd7dc1a1890 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4007ee0 tx=0x7fd7d400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.340+0000 7fd7e4000700 1 -- 172.21.15.63:0/419624633 shutdown_connections 2024-05-27T07:41:39.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.340+0000 7fd7e4000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7c40b29f0 0x7fd7c40b4f00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.340+0000 7fd7e4000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7dc110d10 0x7fd7dc1a1890 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.340+0000 7fd7e4000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7dc103160 0x7fd7dc19d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.340+0000 7fd7e4000700 1 --2- 172.21.15.63:0/419624633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7dc103ab0 0x7fd7dc19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.340+0000 7fd7e4000700 1 -- 172.21.15.63:0/419624633 >> 172.21.15.63:0/419624633 conn(0x7fd7dc078df0 msgr2=0x7fd7dc112390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.340+0000 7fd7e4000700 1 -- 172.21.15.63:0/419624633 shutdown_connections 2024-05-27T07:41:39.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:39.340+0000 7fd7e4000700 1 -- 172.21.15.63:0/419624633 wait complete. 2024-05-27T07:41:39.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:39 smithi086 ceph-mon[34687]: pgmap v249: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:41:39.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:39 smithi149 ceph-mon[35408]: pgmap v249: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:41:39.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:39 smithi063 ceph-mon[31613]: pgmap v249: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:41:40.342 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:41:40.872 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:41.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:41 smithi086 ceph-mon[34687]: pgmap v250: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:41:41.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:41 smithi149 ceph-mon[35408]: pgmap v250: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:41:41.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:41 smithi063 ceph-mon[31613]: pgmap v250: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:41:43.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.370+0000 7f07cf200700 1 -- 172.21.15.63:0/3147169828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07c81043f0 msgr2=0x7f07c8104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:43.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.370+0000 7f07cf200700 1 --2- 172.21.15.63:0/3147169828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07c81043f0 0x7f07c8104800 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0099c0 tx=0x7f07bc009cd0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.370+0000 7f07cf200700 1 -- 172.21.15.63:0/3147169828 shutdown_connections 2024-05-27T07:41:43.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.370+0000 7f07cf200700 1 --2- 172.21.15.63:0/3147169828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07c8109670 0x7f07c8110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.370+0000 7f07cf200700 1 --2- 172.21.15.63:0/3147169828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07c81043f0 0x7f07c8104800 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.370+0000 7f07cf200700 1 --2- 172.21.15.63:0/3147169828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8108d20 0x7f07c8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.370+0000 7f07cf200700 1 -- 172.21.15.63:0/3147169828 >> 172.21.15.63:0/3147169828 conn(0x7f07c8079d00 msgr2=0x7f07c807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:43.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.370+0000 7f07cf200700 1 -- 172.21.15.63:0/3147169828 shutdown_connections 2024-05-27T07:41:43.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.370+0000 7f07cf200700 1 -- 172.21.15.63:0/3147169828 wait complete. 2024-05-27T07:41:43.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.371+0000 7f07cf200700 1 Processor -- start 2024-05-27T07:41:43.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.372+0000 7f07cf200700 1 -- start start 2024-05-27T07:41:43.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.372+0000 7f07cf200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07c81043f0 0x7f07c819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.372+0000 7f07cf200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8108d20 0x7f07c819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.372+0000 7f07cf200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07c8109670 0x7f07c81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.372+0000 7f07cf200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81141a0 con 0x7f07c8109670 2024-05-27T07:41:43.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.372+0000 7f07cf200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8114020 con 0x7f07c8108d20 2024-05-27T07:41:43.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.372+0000 7f07cf200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8114320 con 0x7f07c81043f0 2024-05-27T07:41:43.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.373+0000 7f07ce800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07c8109670 0x7f07c81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.373+0000 7f07cd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8108d20 0x7f07c819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.373+0000 7f07ce800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07c8109670 0x7f07c81a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39466/0 (socket says 172.21.15.63:39466) 2024-05-27T07:41:43.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.373+0000 7f07ce800700 1 -- 172.21.15.63:0/3910918712 learned_addr learned my addr 172.21.15.63:0/3910918712 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:43.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.373+0000 7f07cde00700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07c81043f0 0x7f07c819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.373+0000 7f07ce800700 1 -- 172.21.15.63:0/3910918712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07c81043f0 msgr2=0x7f07c819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:43.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.373+0000 7f07ce800700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07c81043f0 0x7f07c819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.373+0000 7f07ce800700 1 -- 172.21.15.63:0/3910918712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8108d20 msgr2=0x7f07c819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:43.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.373+0000 7f07ce800700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8108d20 0x7f07c819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.374+0000 7f07ce800700 1 -- 172.21.15.63:0/3910918712 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c81a1c50 con 0x7f07c8109670 2024-05-27T07:41:43.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.374+0000 7f07ce800700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07c8109670 0x7f07c81a1660 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f07c4007d70 tx=0x7f07c400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:43.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.374+0000 7f07c2a00700 1 -- 172.21.15.63:0/3910918712 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4016020 con 0x7f07c8109670 2024-05-27T07:41:43.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.374+0000 7f07c2a00700 1 -- 172.21.15.63:0/3910918712 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07c400ac90 con 0x7f07c8109670 2024-05-27T07:41:43.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.374+0000 7f07cf200700 1 -- 172.21.15.63:0/3910918712 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c81a1f40 con 0x7f07c8109670 2024-05-27T07:41:43.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.375+0000 7f07c2a00700 1 -- 172.21.15.63:0/3910918712 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c40115e0 con 0x7f07c8109670 2024-05-27T07:41:43.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.375+0000 7f07cf200700 1 -- 172.21.15.63:0/3910918712 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c81a2430 con 0x7f07c8109670 2024-05-27T07:41:43.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.380+0000 7f07c2a00700 1 -- 172.21.15.63:0/3910918712 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f07c4019020 con 0x7f07c8109670 2024-05-27T07:41:43.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.380+0000 7f07df305b80 1 Processor -- start 2024-05-27T07:41:43.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.380+0000 7f07df305b80 1 -- start start 2024-05-27T07:41:43.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.381+0000 7f07df305b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a8929f0 0x557d6a926750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.381+0000 7f07df305b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a7a6e40 0x557d6a926c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.381+0000 7f07df305b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a9326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.381+0000 7f07df305b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557d6a853e40 con 0x557d6a7a6e40 2024-05-27T07:41:43.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.381+0000 7f07df305b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d6a8439e0 con 0x557d6a9302c0 2024-05-27T07:41:43.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.381+0000 7f07df305b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557d6a847a20 con 0x557d6a8929f0 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.381+0000 7f07cde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a7a6e40 0x557d6a926c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.381+0000 7f07cde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a7a6e40 0x557d6a926c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39468/0 (socket says 172.21.15.63:39468) 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.381+0000 7f07cde00700 1 -- 172.21.15.63:0/886544325 learned_addr learned my addr 172.21.15.63:0/886544325 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.381+0000 7f07cd400700 1 --2- 172.21.15.63:0/886544325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a8929f0 0x557d6a926750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.382+0000 7f07cd400700 1 --2- 172.21.15.63:0/886544325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a9326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.382+0000 7f07cde00700 1 -- 172.21.15.63:0/886544325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a8929f0 msgr2=0x557d6a926750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.382+0000 7f07cde00700 1 --2- 172.21.15.63:0/886544325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a8929f0 0x557d6a926750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.382+0000 7f07cde00700 1 -- 172.21.15.63:0/886544325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 msgr2=0x557d6a9326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.382+0000 7f07cde00700 1 --2- 172.21.15.63:0/886544325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a9326e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.382+0000 7f07cde00700 1 -- 172.21.15.63:0/886544325 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6a80c420 con 0x557d6a7a6e40 2024-05-27T07:41:43.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.382+0000 7f07c2a00700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07b00b29f0 0x7f07b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.382+0000 7f07c2a00700 1 -- 172.21.15.63:0/3910918712 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f07c40e27c0 con 0x7f07c8109670 2024-05-27T07:41:43.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.382+0000 7f07cde00700 1 --2- 172.21.15.63:0/886544325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a7a6e40 0x557d6a926c90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0099c0 tx=0x7f07bc005100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ee053c717c2e7a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:43.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.382+0000 7f07cd400700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07b00b29f0 0x7f07b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.383+0000 7f07b7400700 1 -- 172.21.15.63:0/886544325 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc027070 con 0x557d6a7a6e40 2024-05-27T07:41:43.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.383+0000 7f07b7400700 1 -- 172.21.15.63:0/886544325 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07bc0055a0 con 0x557d6a7a6e40 2024-05-27T07:41:43.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.383+0000 7f07b7400700 1 -- 172.21.15.63:0/886544325 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc003ae0 con 0x557d6a7a6e40 2024-05-27T07:41:43.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.384+0000 7f07cd400700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07b00b29f0 0x7f07b00b4f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x557d6a928210 tx=0x7f07b800b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:43.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.384+0000 7f07df305b80 1 -- 172.21.15.63:0/886544325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a7a6e40 msgr2=0x557d6a926c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:43.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.384+0000 7f07df305b80 1 --2- 172.21.15.63:0/886544325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a7a6e40 0x557d6a926c90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0099c0 tx=0x7f07bc005100 comp rx=0 tx=0).stop 2024-05-27T07:41:43.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.384+0000 7f07df305b80 1 -- 172.21.15.63:0/886544325 shutdown_connections 2024-05-27T07:41:43.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.384+0000 7f07df305b80 1 --2- 172.21.15.63:0/886544325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a9326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.384+0000 7f07df305b80 1 --2- 172.21.15.63:0/886544325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a8929f0 0x557d6a926750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.384+0000 7f07df305b80 1 --2- 172.21.15.63:0/886544325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a7a6e40 0x557d6a926c90 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.384+0000 7f07df305b80 1 -- 172.21.15.63:0/886544325 >> 172.21.15.63:0/886544325 conn(0x557d6a840390 msgr2=0x557d6a921710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:43.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.385+0000 7f07df305b80 1 -- 172.21.15.63:0/886544325 shutdown_connections 2024-05-27T07:41:43.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.385+0000 7f07df305b80 1 -- 172.21.15.63:0/886544325 wait complete. 2024-05-27T07:41:43.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.385+0000 7f07df305b80 1 -- start start 2024-05-27T07:41:43.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.386+0000 7f07df305b80 1 Processor -- start 2024-05-27T07:41:43.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.386+0000 7f07df305b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a7a6e40 0x557d6a92d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.386+0000 7f07df305b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a92d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.386+0000 7f07df305b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a92ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.386+0000 7f07df305b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557d6a749fa0 con 0x557d6a8929f0 2024-05-27T07:41:43.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.386+0000 7f07df305b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d6a7e4760 con 0x557d6a9302c0 2024-05-27T07:41:43.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.386+0000 7f07df305b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557d6a932da0 con 0x557d6a7a6e40 2024-05-27T07:41:43.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.386+0000 7f07df305b80 10 client.0 _ll_register_callbacks cb 0x557d6a7edc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:43.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07cd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a92d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07cd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a92d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39480/0 (socket says 172.21.15.63:39480) 2024-05-27T07:41:43.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07cd400700 1 -- 172.21.15.63:0/3909692861 learned_addr learned my addr 172.21.15.63:0/3909692861 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:43.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07cde00700 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a7a6e40 0x557d6a92d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07ce800700 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a92ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07cd400700 1 -- 172.21.15.63:0/3909692861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a7a6e40 msgr2=0x557d6a92d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:43.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07cd400700 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a7a6e40 0x557d6a92d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07cd400700 1 -- 172.21.15.63:0/3909692861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 msgr2=0x557d6a92ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:43.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07cd400700 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a92ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07cd400700 1 -- 172.21.15.63:0/3909692861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6a7938d0 con 0x557d6a8929f0 2024-05-27T07:41:43.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.387+0000 7f07ce800700 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a92ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:43.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.388+0000 7f07cd400700 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a92d8b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f07b80046e0 tx=0x7f07b8005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c936e6dec205f6b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:43.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.388+0000 7f07abe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:43.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.388+0000 7f07abe00700 1 -- 172.21.15.63:0/3909692861 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8012b40 con 0x557d6a8929f0 2024-05-27T07:41:43.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.388+0000 7f07df305b80 1 -- 172.21.15.63:0/3909692861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07bc0055a0 con 0x557d6a8929f0 2024-05-27T07:41:43.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.388+0000 7f07abe00700 1 -- 172.21.15.63:0/3909692861 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f07b8012ce0 con 0x557d6a8929f0 2024-05-27T07:41:43.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07abe00700 1 -- 172.21.15.63:0/3909692861 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f07b8028470 con 0x557d6a8929f0 2024-05-27T07:41:43.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07df305b80 10 client.16968 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:43.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07df305b80 1 -- 172.21.15.63:0/3909692861 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557d6a814060 con 0x557d6a8929f0 2024-05-27T07:41:43.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07abe00700 1 -- 172.21.15.63:0/3909692861 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07bc0055a0 con 0x557d6a8929f0 2024-05-27T07:41:43.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07abe00700 1 -- 172.21.15.63:0/3909692861 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f07b801c650 con 0x557d6a8929f0 2024-05-27T07:41:43.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07df305b80 10 client.16968 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:43.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07df305b80 10 client.16968 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:41:43.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07df305b80 20 client.16968 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:43.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07df305b80 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x557d6a93fe60 0x557d6a9422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07df305b80 4 client.16968 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:43.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.389+0000 7f07df305b80 1 -- 172.21.15.63:0/3909692861 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d6a855d90 con 0x557d6a93fe60 2024-05-27T07:41:43.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.390+0000 7f07cde00700 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x557d6a93fe60 0x557d6a9422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.391+0000 7f07cde00700 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x557d6a93fe60 0x557d6a9422e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a49355ac7770bf5e server_cookie=88984fda6ce63c51 in_seq=0 out_seq=0 2024-05-27T07:41:43.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.391+0000 7f07abe00700 10 client.16968 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:41:43.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.393+0000 7f07abe00700 1 -- 172.21.15.63:0/3909692861 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557d6a855d90 con 0x557d6a93fe60 2024-05-27T07:41:43.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.393+0000 7f07abe00700 10 client.16968 handle_command_reply: tid=0 2024-05-27T07:41:43.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.393+0000 7f07df305b80 1 client.16968 shutdown 2024-05-27T07:41:43.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 1 -- 172.21.15.63:0/3909692861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 msgr2=0x557d6a92d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:43.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a92d8b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f07b80046e0 tx=0x7f07b8005d20 comp rx=0 tx=0).stop 2024-05-27T07:41:43.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 1 -- 172.21.15.63:0/3909692861 shutdown_connections 2024-05-27T07:41:43.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a92ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a7a6e40 0x557d6a92d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x557d6a93fe60 0x557d6a9422e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 1 --2- 172.21.15.63:0/3909692861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a92d8b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07abe00700 0 client.16968 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:41:43.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 1 -- 172.21.15.63:0/3909692861 >> 172.21.15.63:0/3909692861 conn(0x557d6a840390 msgr2=0x557d6a9213a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:43.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 1 -- 172.21.15.63:0/3909692861 shutdown_connections 2024-05-27T07:41:43.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 1 -- 172.21.15.63:0/3909692861 wait complete. 2024-05-27T07:41:43.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.394+0000 7f07df305b80 20 client.16968 trim_cache size 0 max 16384 2024-05-27T07:41:43.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.412+0000 7f07df305b80 1 Processor -- start 2024-05-27T07:41:43.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.412+0000 7f07df305b80 1 -- start start 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.412+0000 7f07df305b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a7a6e40 0x557d6a93bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.412+0000 7f07df305b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a947aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.412+0000 7f07df305b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a947fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.412+0000 7f07df305b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b4e80 con 0x557d6a8929f0 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.412+0000 7f07df305b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b2710 con 0x557d6a9302c0 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.412+0000 7f07df305b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b0580 con 0x557d6a7a6e40 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.413+0000 7f07cd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a947aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.413+0000 7f07cd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a947aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39494/0 (socket says 172.21.15.63:39494) 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.413+0000 7f07cd400700 1 -- 172.21.15.63:0/1305385981 learned_addr learned my addr 172.21.15.63:0/1305385981 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.413+0000 7f07cde00700 1 --2- 172.21.15.63:0/1305385981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a7a6e40 0x557d6a93bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.413+0000 7f07ce800700 1 --2- 172.21.15.63:0/1305385981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a947fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.414+0000 7f07cd400700 1 -- 172.21.15.63:0/1305385981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a7a6e40 msgr2=0x557d6a93bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:43.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.414+0000 7f07cd400700 1 --2- 172.21.15.63:0/1305385981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a7a6e40 0x557d6a93bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.414+0000 7f07cd400700 1 -- 172.21.15.63:0/1305385981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 msgr2=0x557d6a947fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:43.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.414+0000 7f07cd400700 1 --2- 172.21.15.63:0/1305385981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a947fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.414+0000 7f07cd400700 1 -- 172.21.15.63:0/1305385981 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6a922bf0 con 0x557d6a8929f0 2024-05-27T07:41:43.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.414+0000 7f07cd400700 1 --2- 172.21.15.63:0/1305385981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a947aa0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f07b801caf0 tx=0x7f07b801cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b184447401aa4a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:43.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.414+0000 7f07b6a00700 1 -- 172.21.15.63:0/1305385981 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8012b40 con 0x557d6a8929f0 2024-05-27T07:41:43.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.415+0000 7f07b6a00700 1 -- 172.21.15.63:0/1305385981 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07b8039070 con 0x557d6a8929f0 2024-05-27T07:41:43.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.415+0000 7f07b6a00700 1 -- 172.21.15.63:0/1305385981 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8005590 con 0x557d6a8929f0 2024-05-27T07:41:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.416+0000 7f07df305b80 1 -- 172.21.15.63:0/1305385981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 msgr2=0x557d6a947aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.416+0000 7f07df305b80 1 --2- 172.21.15.63:0/1305385981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a947aa0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f07b801caf0 tx=0x7f07b801cd60 comp rx=0 tx=0).stop 2024-05-27T07:41:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.416+0000 7f07df305b80 1 -- 172.21.15.63:0/1305385981 shutdown_connections 2024-05-27T07:41:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.416+0000 7f07df305b80 1 --2- 172.21.15.63:0/1305385981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a9302c0 0x557d6a947fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.416+0000 7f07df305b80 1 --2- 172.21.15.63:0/1305385981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a7a6e40 0x557d6a93bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.416+0000 7f07df305b80 1 --2- 172.21.15.63:0/1305385981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a947aa0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.416+0000 7f07df305b80 1 -- 172.21.15.63:0/1305385981 >> 172.21.15.63:0/1305385981 conn(0x557d6a840390 msgr2=0x557d6a92bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.416+0000 7f07df305b80 1 -- 172.21.15.63:0/1305385981 shutdown_connections 2024-05-27T07:41:43.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.416+0000 7f07df305b80 1 -- 172.21.15.63:0/1305385981 wait complete. 2024-05-27T07:41:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.416+0000 7f07df305b80 1 -- start start 2024-05-27T07:41:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.417+0000 7f07df305b80 1 Processor -- start 2024-05-27T07:41:43.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.417+0000 7f07df305b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a7a6e40 0x557d6a933130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07df305b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a933670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07df305b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a9302c0 0x557d6a9a8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07df305b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b0bd0 con 0x557d6a8929f0 2024-05-27T07:41:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07df305b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b2040 con 0x557d6a7a6e40 2024-05-27T07:41:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07df305b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b12a0 con 0x557d6a9302c0 2024-05-27T07:41:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07df305b80 10 client.0 _ll_register_callbacks cb 0x557d6a7edc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07cd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a933670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07cd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a933670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39500/0 (socket says 172.21.15.63:39500) 2024-05-27T07:41:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07cd400700 1 -- 172.21.15.63:0/2638726106 learned_addr learned my addr 172.21.15.63:0/2638726106 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:43.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07cde00700 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a7a6e40 0x557d6a933130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.418+0000 7f07ce800700 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a9302c0 0x557d6a9a8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.419+0000 7f07cd400700 1 -- 172.21.15.63:0/2638726106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a9302c0 msgr2=0x557d6a9a8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.419+0000 7f07cd400700 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a9302c0 0x557d6a9a8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.419+0000 7f07cd400700 1 -- 172.21.15.63:0/2638726106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a7a6e40 msgr2=0x557d6a933130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.419+0000 7f07cd400700 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a7a6e40 0x557d6a933130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.419+0000 7f07cd400700 1 -- 172.21.15.63:0/2638726106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6a933f50 con 0x557d6a8929f0 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.419+0000 7f07cd400700 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a933670 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f07b800bbd0 tx=0x7f07b8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96eb4a8387273adc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.419+0000 7f07abe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.419+0000 7f07abe00700 1 -- 172.21.15.63:0/2638726106 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8041030 con 0x557d6a8929f0 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.419+0000 7f07df305b80 1 -- 172.21.15.63:0/2638726106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8039070 con 0x557d6a8929f0 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.420+0000 7f07abe00700 1 -- 172.21.15.63:0/2638726106 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f07b801c5f0 con 0x557d6a8929f0 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.420+0000 7f07abe00700 1 -- 172.21.15.63:0/2638726106 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f07b804e050 con 0x557d6a8929f0 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.420+0000 7f07df305b80 10 client.16974 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:43.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.420+0000 7f07df305b80 1 -- 172.21.15.63:0/2638726106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557d6a9a9f10 con 0x557d6a8929f0 2024-05-27T07:41:43.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.420+0000 7f07abe00700 1 -- 172.21.15.63:0/2638726106 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07b8039070 con 0x557d6a8929f0 2024-05-27T07:41:43.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.421+0000 7f07abe00700 1 -- 172.21.15.63:0/2638726106 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f07b801e3f0 con 0x557d6a8929f0 2024-05-27T07:41:43.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.421+0000 7f07df305b80 10 client.16974 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:43.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.421+0000 7f07df305b80 10 client.16974 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:41:43.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.421+0000 7f07df305b80 20 client.16974 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:43.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.421+0000 7f07df305b80 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x557d6a9aa450 0x557d6a9ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:43.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.421+0000 7f07df305b80 4 client.16974 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:41:43.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.421+0000 7f07df305b80 1 -- 172.21.15.63:0/2638726106 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d6a84f570 con 0x557d6a9aa450 2024-05-27T07:41:43.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.421+0000 7f07cde00700 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x557d6a9aa450 0x557d6a9ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:43.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.422+0000 7f07cde00700 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x557d6a9aa450 0x557d6a9ac870 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b09ffeb86d9acd32 server_cookie=5dad1948a7fb10ca in_seq=0 out_seq=0 2024-05-27T07:41:43.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.422+0000 7f07abe00700 10 client.16974 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:41:43.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07abe00700 1 -- 172.21.15.63:0/2638726106 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d6a84f570 con 0x557d6a9aa450 2024-05-27T07:41:43.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07abe00700 10 client.16974 handle_command_reply: tid=0 2024-05-27T07:41:43.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07df305b80 1 client.16974 shutdown 2024-05-27T07:41:43.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07df305b80 1 -- 172.21.15.63:0/2638726106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 msgr2=0x557d6a933670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:43.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07df305b80 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a933670 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f07b800bbd0 tx=0x7f07b8014040 comp rx=0 tx=0).stop 2024-05-27T07:41:43.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07df305b80 1 -- 172.21.15.63:0/2638726106 shutdown_connections 2024-05-27T07:41:43.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07df305b80 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557d6a9302c0 0x557d6a9a8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07df305b80 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6a7a6e40 0x557d6a933130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07df305b80 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x557d6a9aa450 0x557d6a9ac870 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07df305b80 1 --2- 172.21.15.63:0/2638726106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d6a8929f0 0x557d6a933670 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07df305b80 1 -- 172.21.15.63:0/2638726106 >> 172.21.15.63:0/2638726106 conn(0x557d6a840390 msgr2=0x557d6a920fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:43.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.423+0000 7f07abe00700 0 client.16974 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:41:43.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.424+0000 7f07df305b80 1 -- 172.21.15.63:0/2638726106 shutdown_connections 2024-05-27T07:41:43.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.424+0000 7f07df305b80 1 -- 172.21.15.63:0/2638726106 wait complete. 2024-05-27T07:41:43.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.424+0000 7f07df305b80 20 client.16974 trim_cache size 0 max 16384 2024-05-27T07:41:43.424 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:41:43.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 -- 172.21.15.63:0/3910918712 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07b00b29f0 msgr2=0x7f07b00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:43.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07b00b29f0 0x7f07b00b4f00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x557d6a928210 tx=0x7f07b800b410 comp rx=0 tx=0).stop 2024-05-27T07:41:43.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 -- 172.21.15.63:0/3910918712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07c8109670 msgr2=0x7f07c81a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:43.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07c8109670 0x7f07c81a1660 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f07c4007d70 tx=0x7f07c400a340 comp rx=0 tx=0).stop 2024-05-27T07:41:43.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 -- 172.21.15.63:0/3910918712 shutdown_connections 2024-05-27T07:41:43.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07b00b29f0 0x7f07b00b4f00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07c8109670 0x7f07c81a1660 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07c81043f0 0x7f07c819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 --2- 172.21.15.63:0/3910918712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8108d20 0x7f07c819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:43.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 -- 172.21.15.63:0/3910918712 >> 172.21.15.63:0/3910918712 conn(0x7f07c8079d00 msgr2=0x7f07c8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:43.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 -- 172.21.15.63:0/3910918712 shutdown_connections 2024-05-27T07:41:43.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:43.425+0000 7f07cf200700 1 -- 172.21.15.63:0/3910918712 wait complete. 2024-05-27T07:41:43.706 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:43 smithi086 ceph-mon[34687]: pgmap v251: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:41:43.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:43 smithi149 ceph-mon[35408]: pgmap v251: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:41:43.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:43 smithi063 ceph-mon[31613]: pgmap v251: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:41:44.259 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:41:44.735 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.474+0000 7fc360a00640 1 -- 172.21.15.63:0/1534574108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc35c10b7f0 msgr2=0x7fc35c10dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.474+0000 7fc360a00640 1 --2- 172.21.15.63:0/1534574108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc35c10b7f0 0x7fc35c10dc10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc348009900 tx=0x7fc34802a3f0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.474+0000 7fc360a00640 1 -- 172.21.15.63:0/1534574108 shutdown_connections 2024-05-27T07:41:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.474+0000 7fc360a00640 1 --2- 172.21.15.63:0/1534574108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc35c10e1e0 0x7fc35c1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.474+0000 7fc360a00640 1 --2- 172.21.15.63:0/1534574108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc35c10b7f0 0x7fc35c10dc10 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.474+0000 7fc360a00640 1 --2- 172.21.15.63:0/1534574108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc35c104d90 0x7fc35c1051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.474+0000 7fc360a00640 1 -- 172.21.15.63:0/1534574108 >> 172.21.15.63:0/1534574108 conn(0x7fc35c0782d0 msgr2=0x7fc35c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.474+0000 7fc360a00640 1 -- 172.21.15.63:0/1534574108 shutdown_connections 2024-05-27T07:41:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.475+0000 7fc360a00640 1 -- 172.21.15.63:0/1534574108 wait complete. 2024-05-27T07:41:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.476+0000 7fc360a00640 1 Processor -- start 2024-05-27T07:41:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.476+0000 7fc360a00640 1 -- start start 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.476+0000 7fc360a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc35c104d90 0x7fc35c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.476+0000 7fc360a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc35c10b7f0 0x7fc35c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.476+0000 7fc360a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc35c10e1e0 0x7fc35c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc360a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c113d40 con 0x7fc35c10e1e0 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc360a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c113bc0 con 0x7fc35c104d90 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc360a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c113ec0 con 0x7fc35c10b7f0 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc35a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc35c10e1e0 0x7fc35c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc35a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc35c10e1e0 0x7fc35c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39522/0 (socket says 172.21.15.63:39522) 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc35a800640 1 -- 172.21.15.63:0/1088876908 learned_addr learned my addr 172.21.15.63:0/1088876908 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc359400640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc35c10b7f0 0x7fc35c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc359e00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc35c104d90 0x7fc35c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc35a800640 1 -- 172.21.15.63:0/1088876908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc35c10b7f0 msgr2=0x7fc35c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc35a800640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc35c10b7f0 0x7fc35c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc35a800640 1 -- 172.21.15.63:0/1088876908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc35c104d90 msgr2=0x7fc35c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.477+0000 7fc35a800640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc35c104d90 0x7fc35c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.478+0000 7fc35a800640 1 -- 172.21.15.63:0/1088876908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c19f640 con 0x7fc35c10e1e0 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.478+0000 7fc359400640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc35c10b7f0 0x7fc35c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.478+0000 7fc359e00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc35c104d90 0x7fc35c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.478+0000 7fc35a800640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc35c10e1e0 0x7fc35c19f000 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc35400eaa0 tx=0x7fc35400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.478+0000 7fc352a00640 1 -- 172.21.15.63:0/1088876908 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35400cdb0 con 0x7fc35c10e1e0 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.478+0000 7fc352a00640 1 -- 172.21.15.63:0/1088876908 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc354005020 con 0x7fc35c10e1e0 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.478+0000 7fc360a00640 1 -- 172.21.15.63:0/1088876908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35c19f930 con 0x7fc35c10e1e0 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.478+0000 7fc352a00640 1 -- 172.21.15.63:0/1088876908 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540116d0 con 0x7fc35c10e1e0 2024-05-27T07:41:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.478+0000 7fc360a00640 1 -- 172.21.15.63:0/1088876908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35c1a72b0 con 0x7fc35c10e1e0 2024-05-27T07:41:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.482+0000 7fc367a34740 1 Processor -- start 2024-05-27T07:41:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.483+0000 7fc352a00640 1 -- 172.21.15.63:0/1088876908 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc3540051d0 con 0x7fc35c10e1e0 2024-05-27T07:41:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.483+0000 7fc367a34740 1 -- start start 2024-05-27T07:41:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.484+0000 7fc367a34740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480596c1b0 0x564805d12c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.484+0000 7fc367a34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648059432f0 0x564805d131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.484+0000 7fc367a34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 0x564805d136f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.484+0000 7fc367a34740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564805c7fb90 con 0x56480596c1b0 2024-05-27T07:41:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.484+0000 7fc367a34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564805c6f730 con 0x5648059432f0 2024-05-27T07:41:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.484+0000 7fc367a34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564805c73770 con 0x56480594b4d0 2024-05-27T07:41:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.484+0000 7fc359400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480596c1b0 0x564805d12c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.484+0000 7fc359400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480596c1b0 0x564805d12c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39524/0 (socket says 172.21.15.63:39524) 2024-05-27T07:41:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.484+0000 7fc359400640 1 -- 172.21.15.63:0/3297020881 learned_addr learned my addr 172.21.15.63:0/3297020881 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.485+0000 7fc359e00640 1 --2- 172.21.15.63:0/3297020881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648059432f0 0x564805d131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.485+0000 7fc359400640 1 --2- 172.21.15.63:0/3297020881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 0x564805d136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.485+0000 7fc359400640 1 -- 172.21.15.63:0/3297020881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 msgr2=0x564805d136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.485+0000 7fc359400640 1 --2- 172.21.15.63:0/3297020881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 0x564805d136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.485+0000 7fc359400640 1 -- 172.21.15.63:0/3297020881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648059432f0 msgr2=0x564805d131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.485+0000 7fc359400640 1 --2- 172.21.15.63:0/3297020881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648059432f0 0x564805d131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.485+0000 7fc359400640 1 -- 172.21.15.63:0/3297020881 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564805c088b0 con 0x56480596c1b0 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.485+0000 7fc359e00640 1 --2- 172.21.15.63:0/3297020881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648059432f0 0x564805d131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.485+0000 7fc352a00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3400b29d0 0x7fc3400b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.485+0000 7fc359400640 1 --2- 172.21.15.63:0/3297020881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480596c1b0 0x564805d12c70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc348000c00 tx=0x7fc3480095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21467eb4b15fe5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.486+0000 7fc352a00640 1 -- 172.21.15.63:0/1088876908 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fc3540e2b90 con 0x7fc35c10e1e0 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.486+0000 7fc359e00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3400b29d0 0x7fc3400b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.486+0000 7fc347000640 1 -- 172.21.15.63:0/3297020881 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348043020 con 0x56480596c1b0 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.486+0000 7fc347000640 1 -- 172.21.15.63:0/3297020881 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc348005340 con 0x56480596c1b0 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.486+0000 7fc347000640 1 -- 172.21.15.63:0/3297020881 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348035070 con 0x56480596c1b0 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.487+0000 7fc367a34740 1 -- 172.21.15.63:0/3297020881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480596c1b0 msgr2=0x564805d12c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.487+0000 7fc359e00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3400b29d0 0x7fc3400b4ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x564805d14020 tx=0x7fc34c005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.487+0000 7fc367a34740 1 --2- 172.21.15.63:0/3297020881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480596c1b0 0x564805d12c70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc348000c00 tx=0x7fc3480095f0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.487+0000 7fc367a34740 1 -- 172.21.15.63:0/3297020881 shutdown_connections 2024-05-27T07:41:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.487+0000 7fc367a34740 1 --2- 172.21.15.63:0/3297020881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 0x564805d136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.487+0000 7fc367a34740 1 --2- 172.21.15.63:0/3297020881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648059432f0 0x564805d131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.487+0000 7fc367a34740 1 --2- 172.21.15.63:0/3297020881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480596c1b0 0x564805d12c70 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.487+0000 7fc367a34740 1 -- 172.21.15.63:0/3297020881 >> 172.21.15.63:0/3297020881 conn(0x564805960300 msgr2=0x564805d0f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.487+0000 7fc367a34740 1 -- 172.21.15.63:0/3297020881 shutdown_connections 2024-05-27T07:41:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.487+0000 7fc367a34740 1 -- 172.21.15.63:0/3297020881 wait complete. 2024-05-27T07:41:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.488+0000 7fc367a34740 1 -- start start 2024-05-27T07:41:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.488+0000 7fc367a34740 1 Processor -- start 2024-05-27T07:41:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc367a34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648059432f0 0x564805d1d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc367a34740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d1d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc367a34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d29070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc367a34740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564805955790 con 0x56480594b4d0 2024-05-27T07:41:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc367a34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564805c17bc0 con 0x56480596c1b0 2024-05-27T07:41:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc367a34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564805d1fd40 con 0x5648059432f0 2024-05-27T07:41:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc367a34740 10 client.0 _ll_register_callbacks cb 0x564805bf5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc359400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d1d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc359400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d1d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39528/0 (socket says 172.21.15.63:39528) 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc359400640 1 -- 172.21.15.63:0/1680449016 learned_addr learned my addr 172.21.15.63:0/1680449016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.489+0000 7fc359e00640 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648059432f0 0x564805d1d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc35a800640 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d29070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc359400640 1 -- 172.21.15.63:0/1680449016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648059432f0 msgr2=0x564805d1d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc359400640 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648059432f0 0x564805d1d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc359400640 1 -- 172.21.15.63:0/1680449016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 msgr2=0x564805d29070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc359400640 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d29070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc359400640 1 -- 172.21.15.63:0/1680449016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564805984210 con 0x56480594b4d0 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc359400640 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d1d650 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc348004e20 tx=0x7fc348004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b789dd6c1a2b0eac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc33b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc33b400640 1 -- 172.21.15.63:0/1680449016 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348043030 con 0x56480594b4d0 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc367a34740 1 -- 172.21.15.63:0/1680449016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc348005340 con 0x56480594b4d0 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.490+0000 7fc33b400640 1 -- 172.21.15.63:0/1680449016 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fc3480035f0 con 0x56480594b4d0 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.491+0000 7fc33b400640 1 -- 172.21.15.63:0/1680449016 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc348058050 con 0x56480594b4d0 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.491+0000 7fc367a34740 10 client.16986 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.491+0000 7fc367a34740 1 -- 172.21.15.63:0/1680449016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564805a50620 con 0x56480594b4d0 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.491+0000 7fc33b400640 1 -- 172.21.15.63:0/1680449016 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc348005340 con 0x56480594b4d0 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.491+0000 7fc33b400640 1 -- 172.21.15.63:0/1680449016 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc34803d570 con 0x56480594b4d0 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.491+0000 7fc367a34740 10 client.16986 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.491+0000 7fc367a34740 10 client.16986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.492+0000 7fc367a34740 20 client.16986 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.492+0000 7fc367a34740 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56480592ad80 0x564805d2ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.492+0000 7fc367a34740 4 client.16986 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.492+0000 7fc367a34740 1 -- 172.21.15.63:0/1680449016 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564805c81ae0 con 0x56480592ad80 2024-05-27T07:41:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.492+0000 7fc359e00640 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56480592ad80 0x564805d2ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.493+0000 7fc359e00640 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56480592ad80 0x564805d2ef50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d3779cd359b38c server_cookie=b6ea20caae78247f in_seq=0 out_seq=0 2024-05-27T07:41:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.493+0000 7fc33b400640 10 client.16986 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.495+0000 7fc33b400640 1 -- 172.21.15.63:0/1680449016 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564805c81ae0 con 0x56480592ad80 2024-05-27T07:41:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.495+0000 7fc33b400640 10 client.16986 handle_command_reply: tid=0 2024-05-27T07:41:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.495+0000 7fc367a34740 1 client.16986 shutdown 2024-05-27T07:41:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 1 -- 172.21.15.63:0/1680449016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 msgr2=0x564805d1d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d1d650 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc348004e20 tx=0x7fc348004e50 comp rx=0 tx=0).stop 2024-05-27T07:41:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 1 -- 172.21.15.63:0/1680449016 shutdown_connections 2024-05-27T07:41:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56480592ad80 0x564805d2ef50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d29070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d1d650 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 1 --2- 172.21.15.63:0/1680449016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648059432f0 0x564805d1d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc33b400640 0 client.16986 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 1 -- 172.21.15.63:0/1680449016 >> 172.21.15.63:0/1680449016 conn(0x564805960300 msgr2=0x564805d12a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 1 -- 172.21.15.63:0/1680449016 shutdown_connections 2024-05-27T07:41:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 1 -- 172.21.15.63:0/1680449016 wait complete. 2024-05-27T07:41:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.496+0000 7fc367a34740 20 client.16986 trim_cache size 0 max 16384 2024-05-27T07:41:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.513+0000 7fc367a34740 1 Processor -- start 2024-05-27T07:41:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.514+0000 7fc367a34740 1 -- start start 2024-05-27T07:41:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.514+0000 7fc367a34740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648059432f0 0x564805d0a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.514+0000 7fc367a34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 0x564805d28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.514+0000 7fc367a34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d29520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.514+0000 7fc367a34740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b1820 con 0x5648059432f0 2024-05-27T07:41:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.514+0000 7fc367a34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1af0f0 con 0x56480596c1b0 2024-05-27T07:41:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.514+0000 7fc367a34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1acf60 con 0x56480594b4d0 2024-05-27T07:41:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.514+0000 7fc359e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648059432f0 0x564805d0a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.514+0000 7fc359e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648059432f0 0x564805d0a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39544/0 (socket says 172.21.15.63:39544) 2024-05-27T07:41:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc359e00640 1 -- 172.21.15.63:0/2441576148 learned_addr learned my addr 172.21.15.63:0/2441576148 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc359400640 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 0x564805d28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc35a800640 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d29520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc359e00640 1 -- 172.21.15.63:0/2441576148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 msgr2=0x564805d28fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc359e00640 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 0x564805d28fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc359e00640 1 -- 172.21.15.63:0/2441576148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 msgr2=0x564805d29520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc359e00640 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d29520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc359e00640 1 -- 172.21.15.63:0/2441576148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564805d14050 con 0x5648059432f0 2024-05-27T07:41:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc359400640 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 0x564805d28fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc35a800640 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d29520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.515+0000 7fc359e00640 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648059432f0 0x564805d0a330 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x564805d103b0 tx=0x7fc34c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fffa9d813e6d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.516+0000 7fc346600640 1 -- 172.21.15.63:0/2441576148 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c005220 con 0x5648059432f0 2024-05-27T07:41:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.516+0000 7fc346600640 1 -- 172.21.15.63:0/2441576148 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc34c004c30 con 0x5648059432f0 2024-05-27T07:41:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.516+0000 7fc346600640 1 -- 172.21.15.63:0/2441576148 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c02a720 con 0x5648059432f0 2024-05-27T07:41:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.517+0000 7fc367a34740 1 -- 172.21.15.63:0/2441576148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648059432f0 msgr2=0x564805d0a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.517+0000 7fc367a34740 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648059432f0 0x564805d0a330 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x564805d103b0 tx=0x7fc34c016000 comp rx=0 tx=0).stop 2024-05-27T07:41:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.517+0000 7fc367a34740 1 -- 172.21.15.63:0/2441576148 shutdown_connections 2024-05-27T07:41:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.517+0000 7fc367a34740 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d29520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.517+0000 7fc367a34740 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56480594b4d0 0x564805d28fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.517+0000 7fc367a34740 1 --2- 172.21.15.63:0/2441576148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648059432f0 0x564805d0a330 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.517+0000 7fc367a34740 1 -- 172.21.15.63:0/2441576148 >> 172.21.15.63:0/2441576148 conn(0x564805960300 msgr2=0x564805d1e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.517+0000 7fc367a34740 1 -- 172.21.15.63:0/2441576148 shutdown_connections 2024-05-27T07:41:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.517+0000 7fc367a34740 1 -- 172.21.15.63:0/2441576148 wait complete. 2024-05-27T07:41:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.518+0000 7fc367a34740 1 -- start start 2024-05-27T07:41:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.518+0000 7fc367a34740 1 Processor -- start 2024-05-27T07:41:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc367a34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648059432f0 0x564805d8f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc367a34740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d8fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc367a34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d2f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc367a34740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1ad5b0 con 0x56480594b4d0 2024-05-27T07:41:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc367a34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1aea20 con 0x56480596c1b0 2024-05-27T07:41:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc367a34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1adc80 con 0x5648059432f0 2024-05-27T07:41:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc367a34740 10 client.0 _ll_register_callbacks cb 0x7fc348009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc359400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d8fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc359400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d8fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39552/0 (socket says 172.21.15.63:39552) 2024-05-27T07:41:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc359400640 1 -- 172.21.15.63:0/1517051536 learned_addr learned my addr 172.21.15.63:0/1517051536 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc359e00640 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648059432f0 0x564805d8f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc35a800640 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d2f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc359400640 1 -- 172.21.15.63:0/1517051536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648059432f0 msgr2=0x564805d8f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc359400640 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648059432f0 0x564805d8f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc359400640 1 -- 172.21.15.63:0/1517051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 msgr2=0x564805d2f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.519+0000 7fc359400640 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d2f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.520+0000 7fc359400640 1 -- 172.21.15.63:0/1517051536 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564805d2fbe0 con 0x56480594b4d0 2024-05-27T07:41:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.520+0000 7fc359400640 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d8fdb0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc3480059d0 tx=0x7fc34803ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655bb0df3804edac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.520+0000 7fc33b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.520+0000 7fc33b400640 1 -- 172.21.15.63:0/1517051536 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348043070 con 0x56480594b4d0 2024-05-27T07:41:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.520+0000 7fc33b400640 1 -- 172.21.15.63:0/1517051536 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fc34803d650 con 0x56480594b4d0 2024-05-27T07:41:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.520+0000 7fc367a34740 1 -- 172.21.15.63:0/1517051536 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc34c004c30 con 0x56480594b4d0 2024-05-27T07:41:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc33b400640 1 -- 172.21.15.63:0/1517051536 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc348065050 con 0x56480594b4d0 2024-05-27T07:41:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc367a34740 10 client.16992 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc367a34740 1 -- 172.21.15.63:0/1517051536 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564805d96e80 con 0x56480594b4d0 2024-05-27T07:41:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc33b400640 1 -- 172.21.15.63:0/1517051536 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc34c004c30 con 0x56480594b4d0 2024-05-27T07:41:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc33b400640 1 -- 172.21.15.63:0/1517051536 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc348003880 con 0x56480594b4d0 2024-05-27T07:41:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc367a34740 10 client.16992 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc367a34740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc367a34740 20 client.16992 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc367a34740 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5648059001c0 0x564805d99580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc367a34740 4 client.16992 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:41:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.521+0000 7fc367a34740 1 -- 172.21.15.63:0/1517051536 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564805c7b2c0 con 0x5648059001c0 2024-05-27T07:41:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.522+0000 7fc359e00640 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5648059001c0 0x564805d99580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.523+0000 7fc33b400640 10 client.16992 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.523+0000 7fc359e00640 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5648059001c0 0x564805d99580 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a42643c587c05d28 server_cookie=f8aff5c76da5f365 in_seq=0 out_seq=0 2024-05-27T07:41:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc33b400640 1 -- 172.21.15.63:0/1517051536 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564805c7b2c0 con 0x5648059001c0 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc33b400640 10 client.16992 handle_command_reply: tid=0 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc367a34740 1 client.16992 shutdown 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc367a34740 1 -- 172.21.15.63:0/1517051536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 msgr2=0x564805d8fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc367a34740 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d8fdb0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc3480059d0 tx=0x7fc34803ddc0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc367a34740 1 -- 172.21.15.63:0/1517051536 shutdown_connections 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc367a34740 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5648059001c0 0x564805d99580 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc367a34740 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56480596c1b0 0x564805d2f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc367a34740 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56480594b4d0 0x564805d8fdb0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc367a34740 1 --2- 172.21.15.63:0/1517051536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648059432f0 0x564805d8f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc367a34740 1 -- 172.21.15.63:0/1517051536 >> 172.21.15.63:0/1517051536 conn(0x564805960300 msgr2=0x564805d09d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc33b400640 0 client.16992 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.559+0000 7fc367a34740 1 -- 172.21.15.63:0/1517051536 shutdown_connections 2024-05-27T07:41:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.560+0000 7fc367a34740 1 -- 172.21.15.63:0/1517051536 wait complete. 2024-05-27T07:41:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.560+0000 7fc367a34740 20 client.16992 trim_cache size 0 max 16384 2024-05-27T07:41:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:41:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:41:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:41:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.561+0000 7fc360a00640 1 -- 172.21.15.63:0/1088876908 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3400b29d0 msgr2=0x7fc3400b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.561+0000 7fc360a00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3400b29d0 0x7fc3400b4ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x564805d14020 tx=0x7fc34c005f90 comp rx=0 tx=0).stop 2024-05-27T07:41:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.561+0000 7fc360a00640 1 -- 172.21.15.63:0/1088876908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc35c10e1e0 msgr2=0x7fc35c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.561+0000 7fc360a00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc35c10e1e0 0x7fc35c19f000 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc35400eaa0 tx=0x7fc35400ef60 comp rx=0 tx=0).stop 2024-05-27T07:41:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.561+0000 7fc360a00640 1 -- 172.21.15.63:0/1088876908 shutdown_connections 2024-05-27T07:41:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.561+0000 7fc360a00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3400b29d0 0x7fc3400b4ee0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.561+0000 7fc360a00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc35c10e1e0 0x7fc35c19f000 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.561+0000 7fc360a00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc35c10b7f0 0x7fc35c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.561+0000 7fc360a00640 1 --2- 172.21.15.63:0/1088876908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc35c104d90 0x7fc35c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.561+0000 7fc360a00640 1 -- 172.21.15.63:0/1088876908 >> 172.21.15.63:0/1088876908 conn(0x7fc35c0782d0 msgr2=0x7fc35c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.562+0000 7fc360a00640 1 -- 172.21.15.63:0/1088876908 shutdown_connections 2024-05-27T07:41:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:45.562+0000 7fc360a00640 1 -- 172.21.15.63:0/1088876908 wait complete. 2024-05-27T07:41:45.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:45 smithi063 ceph-mon[31613]: pgmap v252: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:41:45.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:45 smithi086 ceph-mon[34687]: pgmap v252: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:41:45.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:45 smithi149 ceph-mon[35408]: pgmap v252: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:41:46.731 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:46 smithi063 ceph-mon[31613]: pgmap v253: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:41:46.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:46 smithi086 ceph-mon[34687]: pgmap v253: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:41:46.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:46 smithi149 ceph-mon[35408]: pgmap v253: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:41:47.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.138+0000 7f9280400700 1 -- 172.21.15.63:0/2454420028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9278109880 msgr2=0x7f92781111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.138+0000 7f9280400700 1 --2- 172.21.15.63:0/2454420028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9278109880 0x7f92781111e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f927000b0d0 tx=0x7f927000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.138+0000 7f9280400700 1 -- 172.21.15.63:0/2454420028 shutdown_connections 2024-05-27T07:41:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.138+0000 7f9280400700 1 --2- 172.21.15.63:0/2454420028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9278109880 0x7f92781111e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.138+0000 7f9280400700 1 --2- 172.21.15.63:0/2454420028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278104600 0x7f9278104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.138+0000 7f9280400700 1 --2- 172.21.15.63:0/2454420028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278108f30 0x7f9278109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.138+0000 7f9280400700 1 -- 172.21.15.63:0/2454420028 >> 172.21.15.63:0/2454420028 conn(0x7f927807af60 msgr2=0x7f927807b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.138+0000 7f9280400700 1 -- 172.21.15.63:0/2454420028 shutdown_connections 2024-05-27T07:41:47.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.139+0000 7f9280400700 1 -- 172.21.15.63:0/2454420028 wait complete. 2024-05-27T07:41:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.140+0000 7f9280400700 1 Processor -- start 2024-05-27T07:41:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.140+0000 7f9280400700 1 -- start start 2024-05-27T07:41:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f9280400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278104600 0x7f927819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f9280400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278108f30 0x7f927819d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f9280400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9278109880 0x7f92781a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f9280400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92781143b0 con 0x7f9278109880 2024-05-27T07:41:47.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f9280400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9278114230 con 0x7f9278104600 2024-05-27T07:41:47.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f9280400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9278114530 con 0x7f9278108f30 2024-05-27T07:41:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f927e400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9278109880 0x7f92781a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f927e400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9278109880 0x7f92781a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39576/0 (socket says 172.21.15.63:39576) 2024-05-27T07:41:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f927e400700 1 -- 172.21.15.63:0/3275450610 learned_addr learned my addr 172.21.15.63:0/3275450610 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f927d000700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278108f30 0x7f927819d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.141+0000 7f927da00700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278104600 0x7f927819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.142+0000 7f927e400700 1 -- 172.21.15.63:0/3275450610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278108f30 msgr2=0x7f927819d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.142+0000 7f927e400700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278108f30 0x7f927819d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.142+0000 7f927e400700 1 -- 172.21.15.63:0/3275450610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278104600 msgr2=0x7f927819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.142+0000 7f927e400700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278104600 0x7f927819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.142+0000 7f927e400700 1 -- 172.21.15.63:0/3275450610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92781a1e10 con 0x7f9278109880 2024-05-27T07:41:47.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.142+0000 7f927d000700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278108f30 0x7f927819d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:47.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.142+0000 7f927da00700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278104600 0x7f927819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:47.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.142+0000 7f927e400700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9278109880 0x7f92781a1770 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9270007d10 tx=0x7f9270007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:47.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.143+0000 7f9276000700 1 -- 172.21.15.63:0/3275450610 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927001f070 con 0x7f9278109880 2024-05-27T07:41:47.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.143+0000 7f9276000700 1 -- 172.21.15.63:0/3275450610 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9270003e10 con 0x7f9278109880 2024-05-27T07:41:47.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.143+0000 7f9276000700 1 -- 172.21.15.63:0/3275450610 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927001b820 con 0x7f9278109880 2024-05-27T07:41:47.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.143+0000 7f9280400700 1 -- 172.21.15.63:0/3275450610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92781a20a0 con 0x7f9278109880 2024-05-27T07:41:47.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.143+0000 7f9280400700 1 -- 172.21.15.63:0/3275450610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92781a9a20 con 0x7f9278109880 2024-05-27T07:41:47.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.147+0000 7f9276000700 1 -- 172.21.15.63:0/3275450610 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9270022070 con 0x7f9278109880 2024-05-27T07:41:47.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.149+0000 7f928ee52b80 1 Processor -- start 2024-05-27T07:41:47.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.149+0000 7f928ee52b80 1 -- start start 2024-05-27T07:41:47.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.149+0000 7f9276000700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92600b2a70 0x7f92600b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.149+0000 7f9276000700 1 -- 172.21.15.63:0/3275450610 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f927000f020 con 0x7f9278109880 2024-05-27T07:41:47.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.149+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 0x56098f81b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.149+0000 7f927d000700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92600b2a70 0x7f92600b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.149+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f81b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.149+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f81be70 0x56098f828110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.150+0000 7f928ee52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56098f749e40 con 0x56098f81be70 2024-05-27T07:41:47.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.150+0000 7f928ee52b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56098f7399e0 con 0x56098f7889f0 2024-05-27T07:41:47.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.150+0000 7f928ee52b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56098f73da20 con 0x56098f69ce40 2024-05-27T07:41:47.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.150+0000 7f927e400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f81be70 0x56098f828110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.150+0000 7f927e400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f81be70 0x56098f828110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39578/0 (socket says 172.21.15.63:39578) 2024-05-27T07:41:47.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.150+0000 7f927e400700 1 -- 172.21.15.63:0/3058386051 learned_addr learned my addr 172.21.15.63:0/3058386051 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:47.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.150+0000 7f927da00700 1 --2- 172.21.15.63:0/3058386051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 0x56098f81b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.150+0000 7f927d000700 1 --2- 172.21.15.63:0/3058386051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f81b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.151+0000 7f927e400700 1 -- 172.21.15.63:0/3058386051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 msgr2=0x56098f81b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:47.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.151+0000 7f927e400700 1 --2- 172.21.15.63:0/3058386051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f81b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.151+0000 7f927d000700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92600b2a70 0x7f92600b4f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9264005e10 tx=0x7f926400b6d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:47.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.151+0000 7f927e400700 1 -- 172.21.15.63:0/3058386051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 msgr2=0x56098f81b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:47.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.151+0000 7f927e400700 1 --2- 172.21.15.63:0/3058386051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 0x56098f81b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.151+0000 7f927e400700 1 -- 172.21.15.63:0/3058386051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56098f702420 con 0x56098f81be70 2024-05-27T07:41:47.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.151+0000 7f927e400700 1 --2- 172.21.15.63:0/3058386051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f81be70 0x56098f828110 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f92700e1230 tx=0x7f92700aa6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8be8e7eea02f867 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:47.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.151+0000 7f926aa00700 1 -- 172.21.15.63:0/3058386051 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700aaeb0 con 0x56098f81be70 2024-05-27T07:41:47.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.151+0000 7f926aa00700 1 -- 172.21.15.63:0/3058386051 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92700af260 con 0x56098f81be70 2024-05-27T07:41:47.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.151+0000 7f926aa00700 1 -- 172.21.15.63:0/3058386051 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700b37f0 con 0x56098f81be70 2024-05-27T07:41:47.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.152+0000 7f928ee52b80 1 -- 172.21.15.63:0/3058386051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f81be70 msgr2=0x56098f828110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:47.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.152+0000 7f928ee52b80 1 --2- 172.21.15.63:0/3058386051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f81be70 0x56098f828110 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f92700e1230 tx=0x7f92700aa6a0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.153+0000 7f928ee52b80 1 -- 172.21.15.63:0/3058386051 shutdown_connections 2024-05-27T07:41:47.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.153+0000 7f928ee52b80 1 --2- 172.21.15.63:0/3058386051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f81be70 0x56098f828110 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.153+0000 7f928ee52b80 1 --2- 172.21.15.63:0/3058386051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 0x56098f81b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.153+0000 7f928ee52b80 1 --2- 172.21.15.63:0/3058386051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f81b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.153+0000 7f928ee52b80 1 -- 172.21.15.63:0/3058386051 >> 172.21.15.63:0/3058386051 conn(0x56098f736390 msgr2=0x56098f8174c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:47.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.153+0000 7f928ee52b80 1 -- 172.21.15.63:0/3058386051 shutdown_connections 2024-05-27T07:41:47.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.153+0000 7f928ee52b80 1 -- 172.21.15.63:0/3058386051 wait complete. 2024-05-27T07:41:47.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.154+0000 7f928ee52b80 1 -- start start 2024-05-27T07:41:47.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.154+0000 7f928ee52b80 1 Processor -- start 2024-05-27T07:41:47.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f825690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f825bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 0x56098f826110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f928ee52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56098f63ffa0 con 0x56098f7889f0 2024-05-27T07:41:47.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f928ee52b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56098f6da760 con 0x56098f81be70 2024-05-27T07:41:47.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f928ee52b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56098f828940 con 0x56098f69ce40 2024-05-27T07:41:47.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f928ee52b80 10 client.0 _ll_register_callbacks cb 0x56098f6e3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:47.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f927d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f825bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f927d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f825bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39580/0 (socket says 172.21.15.63:39580) 2024-05-27T07:41:47.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f927d000700 1 -- 172.21.15.63:0/2568034738 learned_addr learned my addr 172.21.15.63:0/2568034738 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:47.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.155+0000 7f927da00700 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f825690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.156+0000 7f927e400700 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 0x56098f826110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.156+0000 7f927d000700 1 -- 172.21.15.63:0/2568034738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 msgr2=0x56098f825690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:47.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.156+0000 7f927d000700 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f825690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.156+0000 7f927d000700 1 -- 172.21.15.63:0/2568034738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 msgr2=0x56098f826110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:47.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.156+0000 7f927d000700 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 0x56098f826110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.156+0000 7f927d000700 1 -- 172.21.15.63:0/2568034738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56098f6898d0 con 0x56098f7889f0 2024-05-27T07:41:47.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.156+0000 7f927e400700 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 0x56098f826110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:41:47.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.156+0000 7f927d000700 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f825bd0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f926400bb50 tx=0x7f92640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a58807c076c9ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:47.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.156+0000 7f925f400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:47.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.157+0000 7f925f400700 1 -- 172.21.15.63:0/2568034738 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264004220 con 0x56098f7889f0 2024-05-27T07:41:47.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.157+0000 7f928ee52b80 1 -- 172.21.15.63:0/2568034738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92700af260 con 0x56098f7889f0 2024-05-27T07:41:47.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.157+0000 7f925f400700 1 -- 172.21.15.63:0/2568034738 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f92640043c0 con 0x56098f7889f0 2024-05-27T07:41:47.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.157+0000 7f925f400700 1 -- 172.21.15.63:0/2568034738 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9264003680 con 0x56098f7889f0 2024-05-27T07:41:47.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.157+0000 7f928ee52b80 10 client.17004 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:47.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.157+0000 7f928ee52b80 1 -- 172.21.15.63:0/2568034738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56098f70a060 con 0x56098f7889f0 2024-05-27T07:41:47.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.158+0000 7f925f400700 1 -- 172.21.15.63:0/2568034738 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92700af260 con 0x56098f7889f0 2024-05-27T07:41:47.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.158+0000 7f925f400700 1 -- 172.21.15.63:0/2568034738 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f92640129a0 con 0x56098f7889f0 2024-05-27T07:41:47.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.158+0000 7f928ee52b80 10 client.17004 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:47.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.158+0000 7f928ee52b80 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:47.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.158+0000 7f928ee52b80 20 client.17004 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:47.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.158+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56098f835800 0x56098f837c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.158+0000 7f928ee52b80 4 client.17004 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:47.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.158+0000 7f928ee52b80 1 -- 172.21.15.63:0/2568034738 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56098f74bd90 con 0x56098f835800 2024-05-27T07:41:47.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.159+0000 7f927da00700 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56098f835800 0x56098f837c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.160+0000 7f927da00700 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56098f835800 0x56098f837c40 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d227ee27321df4 server_cookie=97c0ea7f15c52cea in_seq=0 out_seq=0 2024-05-27T07:41:47.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.160+0000 7f925f400700 10 client.17004 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:47.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f925f400700 1 -- 172.21.15.63:0/2568034738 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56098f74bd90 con 0x56098f835800 2024-05-27T07:41:47.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f925f400700 10 client.17004 handle_command_reply: tid=0 2024-05-27T07:41:47.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f928ee52b80 1 client.17004 shutdown 2024-05-27T07:41:47.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f928ee52b80 1 -- 172.21.15.63:0/2568034738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 msgr2=0x56098f825bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:47.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f825bd0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f926400bb50 tx=0x7f92640047c0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f928ee52b80 1 -- 172.21.15.63:0/2568034738 shutdown_connections 2024-05-27T07:41:47.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56098f835800 0x56098f837c40 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 0x56098f826110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f825690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2568034738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f825bd0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f928ee52b80 1 -- 172.21.15.63:0/2568034738 >> 172.21.15.63:0/2568034738 conn(0x56098f736390 msgr2=0x56098f81b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:47.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.162+0000 7f925f400700 0 client.17004 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:47.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.163+0000 7f928ee52b80 1 -- 172.21.15.63:0/2568034738 shutdown_connections 2024-05-27T07:41:47.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.163+0000 7f928ee52b80 1 -- 172.21.15.63:0/2568034738 wait complete. 2024-05-27T07:41:47.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.163+0000 7f928ee52b80 20 client.17004 trim_cache size 0 max 16384 2024-05-27T07:41:47.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.184+0000 7f928ee52b80 1 Processor -- start 2024-05-27T07:41:47.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.184+0000 7f928ee52b80 1 -- start start 2024-05-27T07:41:47.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.184+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f820e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.184+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f81b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.184+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 0x56098f8319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.184+0000 7f928ee52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b4f30 con 0x56098f7889f0 2024-05-27T07:41:47.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.184+0000 7f928ee52b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b27c0 con 0x56098f81be70 2024-05-27T07:41:47.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.184+0000 7f928ee52b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b0630 con 0x56098f69ce40 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.185+0000 7f927d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f81b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.185+0000 7f927d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f81b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39592/0 (socket says 172.21.15.63:39592) 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.185+0000 7f927d000700 1 -- 172.21.15.63:0/1175240229 learned_addr learned my addr 172.21.15.63:0/1175240229 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.185+0000 7f927da00700 1 --2- 172.21.15.63:0/1175240229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f820e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.185+0000 7f927e400700 1 --2- 172.21.15.63:0/1175240229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 0x56098f8319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.185+0000 7f927d000700 1 -- 172.21.15.63:0/1175240229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 msgr2=0x56098f820e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.185+0000 7f927d000700 1 --2- 172.21.15.63:0/1175240229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f820e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.185+0000 7f927d000700 1 -- 172.21.15.63:0/1175240229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 msgr2=0x56098f8319a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.185+0000 7f927d000700 1 --2- 172.21.15.63:0/1175240229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 0x56098f8319a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.185+0000 7f927d000700 1 -- 172.21.15.63:0/1175240229 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56098f813240 con 0x56098f7889f0 2024-05-27T07:41:47.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.186+0000 7f927d000700 1 --2- 172.21.15.63:0/1175240229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f81b830 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9264005de0 tx=0x7f926400b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c996394c26339d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.186+0000 7f926a000700 1 -- 172.21.15.63:0/1175240229 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926403f070 con 0x56098f7889f0 2024-05-27T07:41:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.186+0000 7f926a000700 1 -- 172.21.15.63:0/1175240229 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f926400e350 con 0x56098f7889f0 2024-05-27T07:41:47.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.187+0000 7f926a000700 1 -- 172.21.15.63:0/1175240229 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926403a440 con 0x56098f7889f0 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.187+0000 7f928ee52b80 1 -- 172.21.15.63:0/1175240229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 msgr2=0x56098f81b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.187+0000 7f928ee52b80 1 --2- 172.21.15.63:0/1175240229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f81b830 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9264005de0 tx=0x7f926400b380 comp rx=0 tx=0).stop 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.188+0000 7f928ee52b80 1 -- 172.21.15.63:0/1175240229 shutdown_connections 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.188+0000 7f928ee52b80 1 --2- 172.21.15.63:0/1175240229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f81be70 0x56098f8319a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.188+0000 7f928ee52b80 1 --2- 172.21.15.63:0/1175240229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f69ce40 0x56098f820e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.188+0000 7f928ee52b80 1 --2- 172.21.15.63:0/1175240229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f7889f0 0x56098f81b830 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.188+0000 7f928ee52b80 1 -- 172.21.15.63:0/1175240229 >> 172.21.15.63:0/1175240229 conn(0x56098f736390 msgr2=0x56098f81b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.188+0000 7f928ee52b80 1 -- 172.21.15.63:0/1175240229 shutdown_connections 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.188+0000 7f928ee52b80 1 -- 172.21.15.63:0/1175240229 wait complete. 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.188+0000 7f928ee52b80 1 -- start start 2024-05-27T07:41:47.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.189+0000 7f928ee52b80 1 Processor -- start 2024-05-27T07:41:47.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.189+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f69ce40 0x56098f81f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.189+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 0x56098f81f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.189+0000 7f928ee52b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f81be70 0x56098f81fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.189+0000 7f928ee52b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b0c80 con 0x56098f69ce40 2024-05-27T07:41:47.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.189+0000 7f928ee52b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b20f0 con 0x56098f7889f0 2024-05-27T07:41:47.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.189+0000 7f928ee52b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92781b1350 con 0x56098f81be70 2024-05-27T07:41:47.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.189+0000 7f928ee52b80 10 client.0 _ll_register_callbacks cb 0x56098f6e3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:47.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.190+0000 7f927da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f69ce40 0x56098f81f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.190+0000 7f927da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f69ce40 0x56098f81f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39594/0 (socket says 172.21.15.63:39594) 2024-05-27T07:41:47.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.190+0000 7f927da00700 1 -- 172.21.15.63:0/2291881412 learned_addr learned my addr 172.21.15.63:0/2291881412 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.190+0000 7f927d000700 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 0x56098f81f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.190+0000 7f927e400700 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f81be70 0x56098f81fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.190+0000 7f927da00700 1 -- 172.21.15.63:0/2291881412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f81be70 msgr2=0x56098f81fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.190+0000 7f927da00700 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f81be70 0x56098f81fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.190+0000 7f927da00700 1 -- 172.21.15.63:0/2291881412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 msgr2=0x56098f81f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.190+0000 7f927da00700 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 0x56098f81f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.190+0000 7f927da00700 1 -- 172.21.15.63:0/2291881412 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56098f82cb90 con 0x56098f69ce40 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.191+0000 7f927e400700 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f81be70 0x56098f81fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.191+0000 7f927d000700 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 0x56098f81f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.191+0000 7f927da00700 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f69ce40 0x56098f81f340 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x56098f8383b0 tx=0x7f926c014750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390f63f4de9a01a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:47.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.191+0000 7f925f400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:47.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.191+0000 7f925f400700 1 -- 172.21.15.63:0/2291881412 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c0142d0 con 0x56098f69ce40 2024-05-27T07:41:47.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.191+0000 7f925f400700 1 -- 172.21.15.63:0/2291881412 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f926c01e3f0 con 0x56098f69ce40 2024-05-27T07:41:47.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.191+0000 7f928ee52b80 1 -- 172.21.15.63:0/2291881412 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926400e350 con 0x56098f69ce40 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.192+0000 7f925f400700 1 -- 172.21.15.63:0/2291881412 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f926c030050 con 0x56098f69ce40 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.192+0000 7f928ee52b80 10 client.17010 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.192+0000 7f928ee52b80 1 -- 172.21.15.63:0/2291881412 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56098f82de80 con 0x56098f69ce40 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.192+0000 7f925f400700 1 -- 172.21.15.63:0/2291881412 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f926400e350 con 0x56098f69ce40 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.192+0000 7f925f400700 1 -- 172.21.15.63:0/2291881412 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f926c027820 con 0x56098f69ce40 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.192+0000 7f928ee52b80 10 client.17010 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.193+0000 7f928ee52b80 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.193+0000 7f928ee52b80 20 client.17010 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.193+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56098f8a0310 0x56098f8a2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.193+0000 7f928ee52b80 4 client.17010 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.193+0000 7f928ee52b80 1 -- 172.21.15.63:0/2291881412 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56098f745570 con 0x56098f8a0310 2024-05-27T07:41:47.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.193+0000 7f927d000700 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56098f8a0310 0x56098f8a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:47.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.194+0000 7f927d000700 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56098f8a0310 0x56098f8a2730 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37255fc190ce6e4c server_cookie=460a693a92dca8da in_seq=0 out_seq=0 2024-05-27T07:41:47.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.194+0000 7f925f400700 10 client.17010 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:47.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f925f400700 1 -- 172.21.15.63:0/2291881412 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56098f745570 con 0x56098f8a0310 2024-05-27T07:41:47.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f925f400700 10 client.17010 handle_command_reply: tid=0 2024-05-27T07:41:47.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f928ee52b80 1 client.17010 shutdown 2024-05-27T07:41:47.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f928ee52b80 1 -- 172.21.15.63:0/2291881412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f69ce40 msgr2=0x56098f81f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:47.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f69ce40 0x56098f81f340 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x56098f8383b0 tx=0x7f926c014750 comp rx=0 tx=0).stop 2024-05-27T07:41:47.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f928ee52b80 1 -- 172.21.15.63:0/2291881412 shutdown_connections 2024-05-27T07:41:47.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56098f8a0310 0x56098f8a2730 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56098f81be70 0x56098f81fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56098f69ce40 0x56098f81f340 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f928ee52b80 1 --2- 172.21.15.63:0/2291881412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56098f7889f0 0x56098f81f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f928ee52b80 1 -- 172.21.15.63:0/2291881412 >> 172.21.15.63:0/2291881412 conn(0x56098f736390 msgr2=0x56098f825610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:47.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.195+0000 7f925f400700 0 client.17010 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:47.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.196+0000 7f928ee52b80 1 -- 172.21.15.63:0/2291881412 shutdown_connections 2024-05-27T07:41:47.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.196+0000 7f928ee52b80 1 -- 172.21.15.63:0/2291881412 wait complete. 2024-05-27T07:41:47.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.196+0000 7f928ee52b80 20 client.17010 trim_cache size 0 max 16384 2024-05-27T07:41:47.197 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:41:47.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 -- 172.21.15.63:0/3275450610 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92600b2a70 msgr2=0x7f92600b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:47.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92600b2a70 0x7f92600b4f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9264005e10 tx=0x7f926400b6d0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 -- 172.21.15.63:0/3275450610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9278109880 msgr2=0x7f92781a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:47.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9278109880 0x7f92781a1770 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9270007d10 tx=0x7f9270007df0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 -- 172.21.15.63:0/3275450610 shutdown_connections 2024-05-27T07:41:47.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92600b2a70 0x7f92600b4f80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9278109880 0x7f92781a1770 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9278104600 0x7f927819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 --2- 172.21.15.63:0/3275450610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278108f30 0x7f927819d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:47.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 -- 172.21.15.63:0/3275450610 >> 172.21.15.63:0/3275450610 conn(0x7f927807af60 msgr2=0x7f9278110eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:47.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 -- 172.21.15.63:0/3275450610 shutdown_connections 2024-05-27T07:41:47.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:47.197+0000 7f9280400700 1 -- 172.21.15.63:0/3275450610 wait complete. 2024-05-27T07:41:48.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:48 smithi086 ceph-mon[34687]: pgmap v254: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-27T07:41:48.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:48 smithi149 ceph-mon[35408]: pgmap v254: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-27T07:41:48.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:48 smithi063 ceph-mon[31613]: pgmap v254: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-05-27T07:41:49.280 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:41:49.280 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:41:49.784 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:50.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:50 smithi086 ceph-mon[34687]: pgmap v255: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:41:50.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:50 smithi149 ceph-mon[35408]: pgmap v255: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:41:50.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:50 smithi063 ceph-mon[31613]: pgmap v255: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:41:52.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:52 smithi086 ceph-mon[34687]: pgmap v256: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:41:52.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:52 smithi149 ceph-mon[35408]: pgmap v256: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:41:52.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:52 smithi063 ceph-mon[31613]: pgmap v256: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:41:53.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.792+0000 7fe7be000700 1 -- 172.21.15.63:0/1428285465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7b8109860 msgr2=0x7fe7b81111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:53.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.792+0000 7fe7be000700 1 --2- 172.21.15.63:0/1428285465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7b8109860 0x7fe7b81111c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000b0d0 tx=0x7fe7b000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.792+0000 7fe7be000700 1 -- 172.21.15.63:0/1428285465 shutdown_connections 2024-05-27T07:41:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.792+0000 7fe7be000700 1 --2- 172.21.15.63:0/1428285465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7b8109860 0x7fe7b81111c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.792+0000 7fe7be000700 1 --2- 172.21.15.63:0/1428285465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7b81045e0 0x7fe7b81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.792+0000 7fe7be000700 1 --2- 172.21.15.63:0/1428285465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b8108f10 0x7fe7b8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.792+0000 7fe7be000700 1 -- 172.21.15.63:0/1428285465 >> 172.21.15.63:0/1428285465 conn(0x7fe7b8100030 msgr2=0x7fe7b8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.792+0000 7fe7be000700 1 -- 172.21.15.63:0/1428285465 shutdown_connections 2024-05-27T07:41:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.792+0000 7fe7be000700 1 -- 172.21.15.63:0/1428285465 wait complete. 2024-05-27T07:41:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.793+0000 7fe7be000700 1 Processor -- start 2024-05-27T07:41:53.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.794+0000 7fe7be000700 1 -- start start 2024-05-27T07:41:53.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.794+0000 7fe7be000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7b81045e0 0x7fe7b8075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.794+0000 7fe7be000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b8108f10 0x7fe7b8075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.794+0000 7fe7be000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7b8109860 0x7fe7b8076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.794+0000 7fe7be000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8114340 con 0x7fe7b81045e0 2024-05-27T07:41:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.794+0000 7fe7be000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81141c0 con 0x7fe7b8109860 2024-05-27T07:41:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.794+0000 7fe7be000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81144c0 con 0x7fe7b8108f10 2024-05-27T07:41:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7b81045e0 0x7fe7b8075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7b81045e0 0x7fe7b8075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33296/0 (socket says 172.21.15.63:33296) 2024-05-27T07:41:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7b7400700 1 -- 172.21.15.63:0/1632451205 learned_addr learned my addr 172.21.15.63:0/1632451205 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7aea00700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b8108f10 0x7fe7b8075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7b8109860 0x7fe7b8076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7b7400700 1 -- 172.21.15.63:0/1632451205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b8108f10 msgr2=0x7fe7b8075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:53.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7b7400700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b8108f10 0x7fe7b8075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7b7400700 1 -- 172.21.15.63:0/1632451205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7b8109860 msgr2=0x7fe7b8076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:53.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7b7400700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7b8109860 0x7fe7b8076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7b7400700 1 -- 172.21.15.63:0/1632451205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b81a90b0 con 0x7fe7b81045e0 2024-05-27T07:41:53.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.795+0000 7fe7aea00700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b8108f10 0x7fe7b8075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:53.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.796+0000 7fe7b7400700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7b81045e0 0x7fe7b8075ab0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800d950 tx=0x7fe7a800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:53.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.796+0000 7fe7b4c00700 1 -- 172.21.15.63:0/1632451205 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8016070 con 0x7fe7b81045e0 2024-05-27T07:41:53.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.796+0000 7fe7b4c00700 1 -- 172.21.15.63:0/1632451205 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7a8010460 con 0x7fe7b81045e0 2024-05-27T07:41:53.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.796+0000 7fe7be000700 1 -- 172.21.15.63:0/1632451205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b81a93a0 con 0x7fe7b81045e0 2024-05-27T07:41:53.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.796+0000 7fe7b4c00700 1 -- 172.21.15.63:0/1632451205 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800f620 con 0x7fe7b81045e0 2024-05-27T07:41:53.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.796+0000 7fe7be000700 1 -- 172.21.15.63:0/1632451205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b81a98e0 con 0x7fe7b81045e0 2024-05-27T07:41:53.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.801+0000 7fe7b4c00700 1 -- 172.21.15.63:0/1632451205 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe7a8019070 con 0x7fe7b81045e0 2024-05-27T07:41:53.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.802+0000 7fe7ccb30b80 1 Processor -- start 2024-05-27T07:41:53.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.802+0000 7fe7ccb30b80 1 -- start start 2024-05-27T07:41:53.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d46003b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d4600358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d460036240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d45ff63ec0 con 0x55d45feb6ec0 2024-05-27T07:41:53.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d45ff53a60 con 0x55d46003bdb0 2024-05-27T07:41:53.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d45ff57aa0 con 0x55d460035e30 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7b7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d46003b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7b7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d46003b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33312/0 (socket says 172.21.15.63:33312) 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7b7e00700 1 -- 172.21.15.63:0/1845350037 learned_addr learned my addr 172.21.15.63:0/1845350037 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7b7400700 1 --2- 172.21.15.63:0/1845350037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d4600358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.803+0000 7fe7aea00700 1 --2- 172.21.15.63:0/1845350037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d460036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.804+0000 7fe7b7e00700 1 -- 172.21.15.63:0/1845350037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 msgr2=0x55d460036240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.804+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/1845350037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d460036240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.804+0000 7fe7b7e00700 1 -- 172.21.15.63:0/1845350037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 msgr2=0x55d4600358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.804+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/1845350037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d4600358f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.804+0000 7fe7b7e00700 1 -- 172.21.15.63:0/1845350037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d45ff1c4a0 con 0x55d45feb6ec0 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.804+0000 7fe7b7400700 1 --2- 172.21.15.63:0/1845350037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d4600358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:53.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.804+0000 7fe7b4c00700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe79c0b29f0 0x7fe79c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.804+0000 7fe7b4c00700 1 -- 172.21.15.63:0/1632451205 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fe7a80e2680 con 0x7fe7b81045e0 2024-05-27T07:41:53.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.804+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/1845350037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d46003b870 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009bb0 tx=0x7fe7b0009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bdec8a4ef1a6e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:53.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.804+0000 7fe7b7400700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe79c0b29f0 0x7fe79c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.805+0000 7fe7acc00700 1 -- 172.21.15.63:0/1845350037 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0020030 con 0x55d45feb6ec0 2024-05-27T07:41:53.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.805+0000 7fe7acc00700 1 -- 172.21.15.63:0/1845350037 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7b0004a50 con 0x55d45feb6ec0 2024-05-27T07:41:53.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.805+0000 7fe7acc00700 1 -- 172.21.15.63:0/1845350037 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0007d90 con 0x55d45feb6ec0 2024-05-27T07:41:53.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.806+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1845350037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 msgr2=0x55d46003b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:53.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.806+0000 7fe7b7400700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe79c0b29f0 0x7fe79c0b4f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80a94d0 tx=0x7fe7a8000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:53.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.806+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/1845350037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d46003b870 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009bb0 tx=0x7fe7b0009be0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.806+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1845350037 shutdown_connections 2024-05-27T07:41:53.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.806+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/1845350037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d460036240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.806+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/1845350037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d46003b870 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.806+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/1845350037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d4600358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.806+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1845350037 >> 172.21.15.63:0/1845350037 conn(0x55d45ff17040 msgr2=0x55d460030950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:53.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.806+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1845350037 shutdown_connections 2024-05-27T07:41:53.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.806+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1845350037 wait complete. 2024-05-27T07:41:53.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.807+0000 7fe7ccb30b80 1 -- start start 2024-05-27T07:41:53.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.807+0000 7fe7ccb30b80 1 Processor -- start 2024-05-27T07:41:53.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.808+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d45feb6ec0 0x55d46003f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.808+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d460035e30 0x55d46003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.808+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460040060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.808+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d45fe5a020 con 0x55d460035e30 2024-05-27T07:41:53.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.808+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d45fef47e0 con 0x55d46003bdb0 2024-05-27T07:41:53.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.808+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d460042820 con 0x55d45feb6ec0 2024-05-27T07:41:53.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.808+0000 7fe7ccb30b80 10 client.0 _ll_register_callbacks cb 0x55d45fefdcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:53.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.808+0000 7fe7aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d460035e30 0x55d46003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.809+0000 7fe7aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d460035e30 0x55d46003fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33322/0 (socket says 172.21.15.63:33322) 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.809+0000 7fe7aea00700 1 -- 172.21.15.63:0/4205509452 learned_addr learned my addr 172.21.15.63:0/4205509452 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.809+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d45feb6ec0 0x55d46003f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.809+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460040060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.809+0000 7fe7aea00700 1 -- 172.21.15.63:0/4205509452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d45feb6ec0 msgr2=0x55d46003f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.809+0000 7fe7aea00700 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d45feb6ec0 0x55d46003f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.809+0000 7fe7aea00700 1 -- 172.21.15.63:0/4205509452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 msgr2=0x55d460040060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.809+0000 7fe7aea00700 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460040060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.809+0000 7fe7aea00700 1 -- 172.21.15.63:0/4205509452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d45fea3950 con 0x55d460035e30 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.809+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460040060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.810+0000 7fe7aea00700 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d460035e30 0x55d46003fb20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0003c50 tx=0x7fe7a00116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a289f4b59431167 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.810+0000 7fe7a5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:53.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.810+0000 7fe7a5600700 1 -- 172.21.15.63:0/4205509452 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0003e50 con 0x55d460035e30 2024-05-27T07:41:53.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.810+0000 7fe7a5600700 1 -- 172.21.15.63:0/4205509452 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fe7a0012450 con 0x55d460035e30 2024-05-27T07:41:53.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.810+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/4205509452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b0004a50 con 0x55d460035e30 2024-05-27T07:41:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.810+0000 7fe7a5600700 1 -- 172.21.15.63:0/4205509452 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe7a0012690 con 0x55d460035e30 2024-05-27T07:41:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.810+0000 7fe7ccb30b80 10 client.17022 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.810+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/4205509452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d45ff240e0 con 0x55d460035e30 2024-05-27T07:41:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.811+0000 7fe7a5600700 1 -- 172.21.15.63:0/4205509452 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7b0004a50 con 0x55d460035e30 2024-05-27T07:41:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.811+0000 7fe7a5600700 1 -- 172.21.15.63:0/4205509452 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe7a001b7b0 con 0x55d460035e30 2024-05-27T07:41:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.811+0000 7fe7ccb30b80 10 client.17022 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.811+0000 7fe7ccb30b80 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.811+0000 7fe7ccb30b80 20 client.17022 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.811+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d46004f820 0x55d460051c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.811+0000 7fe7ccb30b80 4 client.17022 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.811+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/4205509452 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d45ff65e10 con 0x55d46004f820 2024-05-27T07:41:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.812+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d46004f820 0x55d460051c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.813+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d46004f820 0x55d460051c70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=530b105eaa947298 server_cookie=7cb3aaa9fc461b4c in_seq=0 out_seq=0 2024-05-27T07:41:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.813+0000 7fe7a5600700 10 client.17022 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:53.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.815+0000 7fe7a5600700 1 -- 172.21.15.63:0/4205509452 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d45ff65e10 con 0x55d46004f820 2024-05-27T07:41:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.815+0000 7fe7a5600700 10 client.17022 handle_command_reply: tid=0 2024-05-27T07:41:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.815+0000 7fe7ccb30b80 1 client.17022 shutdown 2024-05-27T07:41:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.815+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/4205509452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d460035e30 msgr2=0x55d46003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.815+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d460035e30 0x55d46003fb20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0003c50 tx=0x7fe7a00116c0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.815+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/4205509452 shutdown_connections 2024-05-27T07:41:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.816+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d46004f820 0x55d460051c70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.816+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460040060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.816+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d45feb6ec0 0x55d46003f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.816+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/4205509452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d460035e30 0x55d46003fb20 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.816+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/4205509452 >> 172.21.15.63:0/4205509452 conn(0x55d45ff17040 msgr2=0x55d460036240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.816+0000 7fe7a5600700 0 client.17022 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.816+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/4205509452 shutdown_connections 2024-05-27T07:41:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.816+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/4205509452 wait complete. 2024-05-27T07:41:53.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.816+0000 7fe7ccb30b80 20 client.17022 trim_cache size 0 max 16384 2024-05-27T07:41:53.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.852+0000 7fe7ccb30b80 1 Processor -- start 2024-05-27T07:41:53.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.852+0000 7fe7ccb30b80 1 -- start start 2024-05-27T07:41:53.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.852+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d4600358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.852+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d460039830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.852+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460057450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.852+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81b4ef0 con 0x55d45feb6ec0 2024-05-27T07:41:53.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.852+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81b2780 con 0x55d46003bdb0 2024-05-27T07:41:53.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.852+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81b05f0 con 0x55d460035e30 2024-05-27T07:41:53.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.853+0000 7fe7b7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d4600358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.853+0000 7fe7b7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d4600358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33336/0 (socket says 172.21.15.63:33336) 2024-05-27T07:41:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.853+0000 7fe7b7e00700 1 -- 172.21.15.63:0/3922210416 learned_addr learned my addr 172.21.15.63:0/3922210416 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.853+0000 7fe7aea00700 1 --2- 172.21.15.63:0/3922210416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d460039830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.853+0000 7fe7b7e00700 1 -- 172.21.15.63:0/3922210416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 msgr2=0x55d460039830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.853+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/3922210416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d460039830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.853+0000 7fe7b7e00700 1 -- 172.21.15.63:0/3922210416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 msgr2=0x55d460057450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.853+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/3922210416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460057450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.853+0000 7fe7b7e00700 1 -- 172.21.15.63:0/3922210416 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d46003db50 con 0x55d45feb6ec0 2024-05-27T07:41:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.854+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/3922210416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d4600358f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007900 tx=0x7fe7b000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08d66dde4896df server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.854+0000 7fe7a7e00700 1 -- 172.21.15.63:0/3922210416 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0020040 con 0x55d45feb6ec0 2024-05-27T07:41:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.854+0000 7fe7a7e00700 1 -- 172.21.15.63:0/3922210416 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7b0007b30 con 0x55d45feb6ec0 2024-05-27T07:41:53.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.854+0000 7fe7a7e00700 1 -- 172.21.15.63:0/3922210416 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0005450 con 0x55d45feb6ec0 2024-05-27T07:41:53.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.855+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/3922210416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 msgr2=0x55d4600358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:53.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.855+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/3922210416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d4600358f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007900 tx=0x7fe7b000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.856+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/3922210416 shutdown_connections 2024-05-27T07:41:53.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.856+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/3922210416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460057450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.856+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/3922210416 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d4600358f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.856+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/3922210416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d460039830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.856+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/3922210416 >> 172.21.15.63:0/3922210416 conn(0x55d45ff17040 msgr2=0x55d46003fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:53.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.856+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/3922210416 shutdown_connections 2024-05-27T07:41:53.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.856+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/3922210416 wait complete. 2024-05-27T07:41:53.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.857+0000 7fe7ccb30b80 1 -- start start 2024-05-27T07:41:53.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.857+0000 7fe7ccb30b80 1 Processor -- start 2024-05-27T07:41:53.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d460047c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d4600481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7ccb30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460044a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81b0c40 con 0x55d45feb6ec0 2024-05-27T07:41:53.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81b20b0 con 0x55d46003bdb0 2024-05-27T07:41:53.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7ccb30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81b1310 con 0x55d460035e30 2024-05-27T07:41:53.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7ccb30b80 10 client.0 _ll_register_callbacks cb 0x55d45fefdcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:53.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7b7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d460047c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7b7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d460047c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33350/0 (socket says 172.21.15.63:33350) 2024-05-27T07:41:53.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7b7e00700 1 -- 172.21.15.63:0/1330188100 learned_addr learned my addr 172.21.15.63:0/1330188100 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:53.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.858+0000 7fe7aea00700 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d4600481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.859+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460044a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.859+0000 7fe7b7e00700 1 -- 172.21.15.63:0/1330188100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 msgr2=0x55d4600481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:53.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.859+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d4600481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.859+0000 7fe7b7e00700 1 -- 172.21.15.63:0/1330188100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 msgr2=0x55d460044a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:53.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.859+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460044a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.859+0000 7fe7b7e00700 1 -- 172.21.15.63:0/1330188100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d460045080 con 0x55d45feb6ec0 2024-05-27T07:41:53.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.859+0000 7fe7b7e00700 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d460047c70 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009c20 tx=0x7fe7b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5635c53a6fa2179 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:53.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.859+0000 7fe7a5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:53.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.860+0000 7fe7a5600700 1 -- 172.21.15.63:0/1330188100 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0020040 con 0x55d45feb6ec0 2024-05-27T07:41:53.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.860+0000 7fe7a5600700 1 -- 172.21.15.63:0/1330188100 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fe7b00091d0 con 0x55d45feb6ec0 2024-05-27T07:41:53.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.860+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1330188100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b0007b30 con 0x55d45feb6ec0 2024-05-27T07:41:53.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.860+0000 7fe7a5600700 1 -- 172.21.15.63:0/1330188100 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe7b00093d0 con 0x55d45feb6ec0 2024-05-27T07:41:53.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.860+0000 7fe7ccb30b80 10 client.17028 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:53.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.860+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1330188100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d4600babf0 con 0x55d45feb6ec0 2024-05-27T07:41:53.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.861+0000 7fe7a5600700 1 -- 172.21.15.63:0/1330188100 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7b0007b30 con 0x55d45feb6ec0 2024-05-27T07:41:53.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.861+0000 7fe7a5600700 1 -- 172.21.15.63:0/1330188100 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe7b0011070 con 0x55d45feb6ec0 2024-05-27T07:41:53.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.861+0000 7fe7ccb30b80 10 client.17028 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:53.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.861+0000 7fe7ccb30b80 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:53.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.861+0000 7fe7ccb30b80 20 client.17028 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:53.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.861+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d4600bae20 0x55d4600bd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:53.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.861+0000 7fe7ccb30b80 4 client.17028 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:41:53.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.861+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1330188100 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d45ff5f5f0 con 0x55d4600bae20 2024-05-27T07:41:53.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.862+0000 7fe7aea00700 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d4600bae20 0x55d4600bd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:53.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.863+0000 7fe7a5600700 10 client.17028 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:53.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.863+0000 7fe7aea00700 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d4600bae20 0x55d4600bd240 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abfd71f9e8989de0 server_cookie=ba80c377cd41b52f in_seq=0 out_seq=0 2024-05-27T07:41:53.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.865+0000 7fe7a5600700 1 -- 172.21.15.63:0/1330188100 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d45ff5f5f0 con 0x55d4600bae20 2024-05-27T07:41:53.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.865+0000 7fe7a5600700 10 client.17028 handle_command_reply: tid=0 2024-05-27T07:41:53.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.865+0000 7fe7ccb30b80 1 client.17028 shutdown 2024-05-27T07:41:53.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1330188100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 msgr2=0x55d460047c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:53.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d460047c70 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009c20 tx=0x7fe7b000b040 comp rx=0 tx=0).stop 2024-05-27T07:41:53.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1330188100 shutdown_connections 2024-05-27T07:41:53.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d4600bae20 0x55d4600bd240 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d46003bdb0 0x55d460044a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d45feb6ec0 0x55d460047c70 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7ccb30b80 1 --2- 172.21.15.63:0/1330188100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d460035e30 0x55d4600481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7a5600700 0 client.17028 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:53.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1330188100 >> 172.21.15.63:0/1330188100 conn(0x55d45ff17040 msgr2=0x55d46003fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:53.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1330188100 shutdown_connections 2024-05-27T07:41:53.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.866+0000 7fe7ccb30b80 1 -- 172.21.15.63:0/1330188100 wait complete. 2024-05-27T07:41:53.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.867+0000 7fe7ccb30b80 20 client.17028 trim_cache size 0 max 16384 2024-05-27T07:41:53.865 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:41:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:41:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "6a3f674b-a399-4456-b037-384754535b13", 2024-05-27T07:41:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:41:53.865 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:41:53.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.867+0000 7fe7be000700 1 -- 172.21.15.63:0/1632451205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe79c0b29f0 msgr2=0x7fe79c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:53.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.867+0000 7fe7be000700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe79c0b29f0 0x7fe79c0b4f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80a94d0 tx=0x7fe7a8000bc0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.867+0000 7fe7be000700 1 -- 172.21.15.63:0/1632451205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7b81045e0 msgr2=0x7fe7b8075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:53.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.867+0000 7fe7be000700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7b81045e0 0x7fe7b8075ab0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800d950 tx=0x7fe7a800dd10 comp rx=0 tx=0).stop 2024-05-27T07:41:53.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.868+0000 7fe7be000700 1 -- 172.21.15.63:0/1632451205 shutdown_connections 2024-05-27T07:41:53.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.868+0000 7fe7be000700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe79c0b29f0 0x7fe79c0b4f00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.868+0000 7fe7be000700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7b8109860 0x7fe7b8076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.868+0000 7fe7be000700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7b81045e0 0x7fe7b8075ab0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.868+0000 7fe7be000700 1 --2- 172.21.15.63:0/1632451205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b8108f10 0x7fe7b8075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:53.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.868+0000 7fe7be000700 1 -- 172.21.15.63:0/1632451205 >> 172.21.15.63:0/1632451205 conn(0x7fe7b8100030 msgr2=0x7fe7b810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:53.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.868+0000 7fe7be000700 1 -- 172.21.15.63:0/1632451205 shutdown_connections 2024-05-27T07:41:53.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:53.868+0000 7fe7be000700 1 -- 172.21.15.63:0/1632451205 wait complete. 2024-05-27T07:41:54.882 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: pgmap v257: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:41:54.882 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:41:54.882 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.882 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:41:54.882 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:41:54.883 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.883 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:41:54.883 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.883 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:41:54.883 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.883 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.884 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:41:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: pgmap v257: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:41:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:41:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:41:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:41:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:41:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:41:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: pgmap v257: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:55.366 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:55.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:41:55.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:55.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:41:55.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:55.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:41:55.835 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:55.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:41:55.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:55.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:41:55.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:55.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:41:55.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:55.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:41:55.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:55.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:41:55.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:55.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:41:55.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:41:56.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:56 smithi086 ceph-mon[34687]: pgmap v258: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:41:56.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:56 smithi149 ceph-mon[35408]: pgmap v258: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:41:56.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:56 smithi063 ceph-mon[31613]: pgmap v258: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:41:57.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.652+0000 7f35d2200700 1 -- 172.21.15.63:0/2349489672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35cc1045e0 msgr2=0x7f35cc1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:57.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.652+0000 7f35d2200700 1 --2- 172.21.15.63:0/2349489672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35cc1045e0 0x7f35cc1049f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0099c0 tx=0x7f35bc009cd0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.652+0000 7f35d2200700 1 -- 172.21.15.63:0/2349489672 shutdown_connections 2024-05-27T07:41:57.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.652+0000 7f35d2200700 1 --2- 172.21.15.63:0/2349489672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35cc109860 0x7f35cc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.652+0000 7f35d2200700 1 --2- 172.21.15.63:0/2349489672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35cc1045e0 0x7f35cc1049f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.652+0000 7f35d2200700 1 --2- 172.21.15.63:0/2349489672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35cc108f10 0x7f35cc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.652+0000 7f35d2200700 1 -- 172.21.15.63:0/2349489672 >> 172.21.15.63:0/2349489672 conn(0x7f35cc100030 msgr2=0x7f35cc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:57.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.652+0000 7f35d2200700 1 -- 172.21.15.63:0/2349489672 shutdown_connections 2024-05-27T07:41:57.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.652+0000 7f35d2200700 1 -- 172.21.15.63:0/2349489672 wait complete. 2024-05-27T07:41:57.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.654+0000 7f35d2200700 1 Processor -- start 2024-05-27T07:41:57.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.654+0000 7f35d2200700 1 -- start start 2024-05-27T07:41:57.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.655+0000 7f35d2200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35cc1045e0 0x7f35cc19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.655+0000 7f35d2200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35cc108f10 0x7f35cc19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.655+0000 7f35d2200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35cc109860 0x7f35cc1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.655+0000 7f35d2200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc114340 con 0x7f35cc109860 2024-05-27T07:41:57.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.655+0000 7f35d2200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1141c0 con 0x7f35cc108f10 2024-05-27T07:41:57.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.655+0000 7f35d2200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1144c0 con 0x7f35cc1045e0 2024-05-27T07:41:57.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.655+0000 7f35cbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35cc109860 0x7f35cc1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.655+0000 7f35cb400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35cc1045e0 0x7f35cc19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.655+0000 7f35cbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35cc109860 0x7f35cc1a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33378/0 (socket says 172.21.15.63:33378) 2024-05-27T07:41:57.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.656+0000 7f35cbe00700 1 -- 172.21.15.63:0/2826573141 learned_addr learned my addr 172.21.15.63:0/2826573141 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:57.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.656+0000 7f35caa00700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35cc108f10 0x7f35cc19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.656+0000 7f35cbe00700 1 -- 172.21.15.63:0/2826573141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35cc1045e0 msgr2=0x7f35cc19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:57.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.656+0000 7f35cbe00700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35cc1045e0 0x7f35cc19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.656+0000 7f35cbe00700 1 -- 172.21.15.63:0/2826573141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35cc108f10 msgr2=0x7f35cc19d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:57.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.656+0000 7f35cbe00700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35cc108f10 0x7f35cc19d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.656+0000 7f35cbe00700 1 -- 172.21.15.63:0/2826573141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc1a1e90 con 0x7f35cc109860 2024-05-27T07:41:57.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.657+0000 7f35cbe00700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35cc109860 0x7f35cc1a17f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f35c4007ee0 tx=0x7f35c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:57.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.657+0000 7f35c3e00700 1 -- 172.21.15.63:0/2826573141 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4004bb0 con 0x7f35cc109860 2024-05-27T07:41:57.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.657+0000 7f35c3e00700 1 -- 172.21.15.63:0/2826573141 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35c4004d50 con 0x7f35cc109860 2024-05-27T07:41:57.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.657+0000 7f35d2200700 1 -- 172.21.15.63:0/2826573141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc1a2180 con 0x7f35cc109860 2024-05-27T07:41:57.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.657+0000 7f35c3e00700 1 -- 172.21.15.63:0/2826573141 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c40058b0 con 0x7f35cc109860 2024-05-27T07:41:57.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.657+0000 7f35d2200700 1 -- 172.21.15.63:0/2826573141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc1a9aa0 con 0x7f35cc109860 2024-05-27T07:41:57.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.663+0000 7f35c3e00700 1 -- 172.21.15.63:0/2826573141 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f35c401a020 con 0x7f35cc109860 2024-05-27T07:41:57.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.664+0000 7f35e0c61b80 1 Processor -- start 2024-05-27T07:41:57.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.664+0000 7f35e0c61b80 1 -- start start 2024-05-27T07:41:57.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.665+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a32ee40 0x55979a4b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.665+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4b3d30 0x55979a4ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.665+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4addb0 0x55979a4ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.665+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55979a3dbe40 con 0x55979a4addb0 2024-05-27T07:41:57.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.665+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55979a3cb9e0 con 0x55979a4b3d30 2024-05-27T07:41:57.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.665+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55979a3cfa20 con 0x55979a32ee40 2024-05-27T07:41:57.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.665+0000 7f35caa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4addb0 0x55979a4ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.665+0000 7f35c3e00700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f35b00b2a70 0x7f35b00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.666+0000 7f35caa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4addb0 0x55979a4ae1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33390/0 (socket says 172.21.15.63:33390) 2024-05-27T07:41:57.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.666+0000 7f35caa00700 1 -- 172.21.15.63:0/1967674782 learned_addr learned my addr 172.21.15.63:0/1967674782 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:57.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.666+0000 7f35caa00700 1 --2- 172.21.15.63:0/1967674782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a32ee40 0x55979a4b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.666+0000 7f35cb400700 1 --2- 172.21.15.63:0/1967674782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4b3d30 0x55979a4ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.666+0000 7f35c3e00700 1 -- 172.21.15.63:0/2826573141 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f35c40e1fb0 con 0x7f35cc109860 2024-05-27T07:41:57.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.666+0000 7f35cbe00700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f35b00b2a70 0x7f35b00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.667+0000 7f35caa00700 1 -- 172.21.15.63:0/1967674782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a32ee40 msgr2=0x55979a4b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:57.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.667+0000 7f35caa00700 1 --2- 172.21.15.63:0/1967674782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a32ee40 0x55979a4b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.667+0000 7f35caa00700 1 -- 172.21.15.63:0/1967674782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4b3d30 msgr2=0x55979a4ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:57.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.667+0000 7f35caa00700 1 --2- 172.21.15.63:0/1967674782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4b3d30 0x55979a4ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.667+0000 7f35caa00700 1 -- 172.21.15.63:0/1967674782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55979a394420 con 0x55979a4addb0 2024-05-27T07:41:57.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.667+0000 7f35cb400700 1 --2- 172.21.15.63:0/1967674782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4b3d30 0x55979a4ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:41:57.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.667+0000 7f35cbe00700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f35b00b2a70 0x7f35b00b4f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35c40a8bd0 tx=0x7f35c4004b40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:57.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.667+0000 7f35caa00700 1 --2- 172.21.15.63:0/1967674782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4addb0 0x55979a4ae1c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f35b8005d90 tx=0x7f35b8005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de7c9f7215b1ce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:57.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.668+0000 7f35c0c00700 1 -- 172.21.15.63:0/1967674782 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8004830 con 0x55979a4addb0 2024-05-27T07:41:57.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.668+0000 7f35c0c00700 1 -- 172.21.15.63:0/1967674782 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35b8005020 con 0x55979a4addb0 2024-05-27T07:41:57.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.668+0000 7f35c0c00700 1 -- 172.21.15.63:0/1967674782 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8010650 con 0x55979a4addb0 2024-05-27T07:41:57.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.669+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1967674782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4addb0 msgr2=0x55979a4ae1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:57.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.669+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/1967674782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4addb0 0x55979a4ae1c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f35b8005d90 tx=0x7f35b8005d20 comp rx=0 tx=0).stop 2024-05-27T07:41:57.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.669+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1967674782 shutdown_connections 2024-05-27T07:41:57.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.669+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/1967674782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4addb0 0x55979a4ae1c0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.669+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/1967674782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a32ee40 0x55979a4b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.669+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/1967674782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4b3d30 0x55979a4ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.669+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1967674782 >> 172.21.15.63:0/1967674782 conn(0x55979a38efc0 msgr2=0x55979a4a88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:57.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.669+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1967674782 shutdown_connections 2024-05-27T07:41:57.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.669+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1967674782 wait complete. 2024-05-27T07:41:57.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.670+0000 7f35e0c61b80 1 -- start start 2024-05-27T07:41:57.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.670+0000 7f35e0c61b80 1 Processor -- start 2024-05-27T07:41:57.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.671+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a32ee40 0x55979a4b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.671+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4addb0 0x55979a4b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.671+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b3d30 0x55979a4b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.671+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55979a2d1fa0 con 0x55979a4b3d30 2024-05-27T07:41:57.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.671+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55979a36c760 con 0x55979a32ee40 2024-05-27T07:41:57.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.671+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55979a4ba800 con 0x55979a4addb0 2024-05-27T07:41:57.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.671+0000 7f35e0c61b80 10 client.0 _ll_register_callbacks cb 0x55979a375c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:57.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.672+0000 7f35caa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b3d30 0x55979a4b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.672+0000 7f35caa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b3d30 0x55979a4b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33394/0 (socket says 172.21.15.63:33394) 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.672+0000 7f35caa00700 1 -- 172.21.15.63:0/2302314103 learned_addr learned my addr 172.21.15.63:0/2302314103 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.672+0000 7f35cb400700 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4addb0 0x55979a4b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.672+0000 7f35caa00700 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a32ee40 0x55979a4b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.672+0000 7f35caa00700 1 -- 172.21.15.63:0/2302314103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4addb0 msgr2=0x55979a4b7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.672+0000 7f35caa00700 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4addb0 0x55979a4b7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.672+0000 7f35caa00700 1 -- 172.21.15.63:0/2302314103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a32ee40 msgr2=0x55979a4b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.672+0000 7f35caa00700 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a32ee40 0x55979a4b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.672+0000 7f35caa00700 1 -- 172.21.15.63:0/2302314103 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55979a31b8d0 con 0x55979a4b3d30 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.673+0000 7f35caa00700 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b3d30 0x55979a4b7fe0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f35b800b510 tx=0x7f35b800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1278f1c53ad6496 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.673+0000 7f35b5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:57.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.673+0000 7f35b5600700 1 -- 172.21.15.63:0/2302314103 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8026040 con 0x55979a4b3d30 2024-05-27T07:41:57.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.673+0000 7f35e0c61b80 1 -- 172.21.15.63:0/2302314103 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b8005020 con 0x55979a4b3d30 2024-05-27T07:41:57.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.673+0000 7f35b5600700 1 -- 172.21.15.63:0/2302314103 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f35b8004b80 con 0x55979a4b3d30 2024-05-27T07:41:57.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.673+0000 7f35b5600700 1 -- 172.21.15.63:0/2302314103 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f35b8036050 con 0x55979a4b3d30 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.673+0000 7f35e0c61b80 10 client.17040 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.673+0000 7f35e0c61b80 1 -- 172.21.15.63:0/2302314103 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55979a39c060 con 0x55979a4b3d30 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.674+0000 7f35b5600700 1 -- 172.21.15.63:0/2302314103 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35b8005020 con 0x55979a4b3d30 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.674+0000 7f35b5600700 1 -- 172.21.15.63:0/2302314103 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f35b801c030 con 0x55979a4b3d30 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.674+0000 7f35e0c61b80 10 client.17040 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.674+0000 7f35e0c61b80 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.674+0000 7f35e0c61b80 20 client.17040 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.674+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55979a4c75e0 0x55979a4c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.674+0000 7f35e0c61b80 4 client.17040 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.674+0000 7f35e0c61b80 1 -- 172.21.15.63:0/2302314103 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55979a3ddd90 con 0x55979a4c75e0 2024-05-27T07:41:57.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.675+0000 7f35cb400700 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55979a4c75e0 0x55979a4c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.676+0000 7f35cb400700 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55979a4c75e0 0x55979a4c9a00 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17161573f95ce8dd server_cookie=250d36a978cb32cd in_seq=0 out_seq=0 2024-05-27T07:41:57.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.676+0000 7f35b5600700 10 client.17040 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:57.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.678+0000 7f35b5600700 1 -- 172.21.15.63:0/2302314103 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55979a3ddd90 con 0x55979a4c75e0 2024-05-27T07:41:57.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.678+0000 7f35b5600700 10 client.17040 handle_command_reply: tid=0 2024-05-27T07:41:57.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.678+0000 7f35e0c61b80 1 client.17040 shutdown 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.678+0000 7f35e0c61b80 1 -- 172.21.15.63:0/2302314103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b3d30 msgr2=0x55979a4b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.678+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b3d30 0x55979a4b7fe0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f35b800b510 tx=0x7f35b800bc50 comp rx=0 tx=0).stop 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.678+0000 7f35e0c61b80 1 -- 172.21.15.63:0/2302314103 shutdown_connections 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.678+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55979a4c75e0 0x55979a4c9a00 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.678+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b3d30 0x55979a4b7fe0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.679+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a32ee40 0x55979a4b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.679+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/2302314103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4addb0 0x55979a4b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.679+0000 7f35e0c61b80 1 -- 172.21.15.63:0/2302314103 >> 172.21.15.63:0/2302314103 conn(0x55979a38efc0 msgr2=0x55979a4a8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.679+0000 7f35b5600700 0 client.17040 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.679+0000 7f35e0c61b80 1 -- 172.21.15.63:0/2302314103 shutdown_connections 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.679+0000 7f35e0c61b80 1 -- 172.21.15.63:0/2302314103 wait complete. 2024-05-27T07:41:57.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.679+0000 7f35e0c61b80 20 client.17040 trim_cache size 0 max 16384 2024-05-27T07:41:57.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.712+0000 7f35e0c61b80 1 Processor -- start 2024-05-27T07:41:57.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.712+0000 7f35e0c61b80 1 -- start start 2024-05-27T07:41:57.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.713+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a32ee40 0x55979a4ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.713+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4addb0 0x55979a4b9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.713+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4b1d60 0x55979a4c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.713+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b4fb0 con 0x55979a32ee40 2024-05-27T07:41:57.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.713+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b2840 con 0x55979a4addb0 2024-05-27T07:41:57.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.713+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b06b0 con 0x55979a4b1d60 2024-05-27T07:41:57.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.713+0000 7f35caa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a32ee40 0x55979a4ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.713+0000 7f35caa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a32ee40 0x55979a4ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33408/0 (socket says 172.21.15.63:33408) 2024-05-27T07:41:57.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.713+0000 7f35caa00700 1 -- 172.21.15.63:0/76086844 learned_addr learned my addr 172.21.15.63:0/76086844 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:57.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.713+0000 7f35cb400700 1 --2- 172.21.15.63:0/76086844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4addb0 0x55979a4b9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.714+0000 7f35caa00700 1 -- 172.21.15.63:0/76086844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4b1d60 msgr2=0x55979a4c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.714+0000 7f35caa00700 1 --2- 172.21.15.63:0/76086844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4b1d60 0x55979a4c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.714+0000 7f35caa00700 1 -- 172.21.15.63:0/76086844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4addb0 msgr2=0x55979a4b9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.714+0000 7f35caa00700 1 --2- 172.21.15.63:0/76086844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4addb0 0x55979a4b9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.714+0000 7f35caa00700 1 -- 172.21.15.63:0/76086844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55979a4a95f0 con 0x55979a32ee40 2024-05-27T07:41:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.714+0000 7f35caa00700 1 --2- 172.21.15.63:0/76086844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a32ee40 0x55979a4ad870 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f35b8025040 tx=0x7f35b8004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d2599aa8a953c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.714+0000 7f35b7e00700 1 -- 172.21.15.63:0/76086844 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8021070 con 0x55979a32ee40 2024-05-27T07:41:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.715+0000 7f35b7e00700 1 -- 172.21.15.63:0/76086844 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35b8005500 con 0x55979a32ee40 2024-05-27T07:41:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.715+0000 7f35b7e00700 1 -- 172.21.15.63:0/76086844 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8004be0 con 0x55979a32ee40 2024-05-27T07:41:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.716+0000 7f35e0c61b80 1 -- 172.21.15.63:0/76086844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a32ee40 msgr2=0x55979a4ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:57.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.716+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/76086844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a32ee40 0x55979a4ad870 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f35b8025040 tx=0x7f35b8004690 comp rx=0 tx=0).stop 2024-05-27T07:41:57.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.716+0000 7f35e0c61b80 1 -- 172.21.15.63:0/76086844 shutdown_connections 2024-05-27T07:41:57.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.716+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/76086844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4b1d60 0x55979a4c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.716+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/76086844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a32ee40 0x55979a4ad870 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.716+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/76086844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a4addb0 0x55979a4b9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.716+0000 7f35e0c61b80 1 -- 172.21.15.63:0/76086844 >> 172.21.15.63:0/76086844 conn(0x55979a38efc0 msgr2=0x55979a4b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.716+0000 7f35e0c61b80 1 -- 172.21.15.63:0/76086844 shutdown_connections 2024-05-27T07:41:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.717+0000 7f35e0c61b80 1 -- 172.21.15.63:0/76086844 wait complete. 2024-05-27T07:41:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.717+0000 7f35e0c61b80 1 -- start start 2024-05-27T07:41:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.718+0000 7f35e0c61b80 1 Processor -- start 2024-05-27T07:41:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a32ee40 0x55979a4bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4addb0 0x55979a4c0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35e0c61b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b1d60 0x55979a4bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b0d00 con 0x55979a4b1d60 2024-05-27T07:41:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b2170 con 0x55979a32ee40 2024-05-27T07:41:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35e0c61b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b13d0 con 0x55979a4addb0 2024-05-27T07:41:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35e0c61b80 10 client.0 _ll_register_callbacks cb 0x55979a375c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35caa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b1d60 0x55979a4bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35caa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b1d60 0x55979a4bca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33422/0 (socket says 172.21.15.63:33422) 2024-05-27T07:41:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35caa00700 1 -- 172.21.15.63:0/1020001767 learned_addr learned my addr 172.21.15.63:0/1020001767 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35cb400700 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4addb0 0x55979a4c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.719+0000 7f35caa00700 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a32ee40 0x55979a4bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.720+0000 7f35caa00700 1 -- 172.21.15.63:0/1020001767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4addb0 msgr2=0x55979a4c0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.720+0000 7f35caa00700 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4addb0 0x55979a4c0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.720+0000 7f35caa00700 1 -- 172.21.15.63:0/1020001767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a32ee40 msgr2=0x55979a4bfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.720+0000 7f35caa00700 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a32ee40 0x55979a4bfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.720+0000 7f35caa00700 1 -- 172.21.15.63:0/1020001767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55979a4bd150 con 0x55979a4b1d60 2024-05-27T07:41:57.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.721+0000 7f35caa00700 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b1d60 0x55979a4bca20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f35b800bcb0 tx=0x7f35b800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eac61e6f3efdfaa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:57.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.721+0000 7f35b5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:57.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.721+0000 7f35b5600700 1 -- 172.21.15.63:0/1020001767 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8021040 con 0x55979a4b1d60 2024-05-27T07:41:57.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.721+0000 7f35b5600700 1 -- 172.21.15.63:0/1020001767 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7f35b8043070 con 0x55979a4b1d60 2024-05-27T07:41:57.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.721+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1020001767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b8005500 con 0x55979a4b1d60 2024-05-27T07:41:57.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.721+0000 7f35b5600700 1 -- 172.21.15.63:0/1020001767 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f35b804c050 con 0x55979a4b1d60 2024-05-27T07:41:57.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.722+0000 7f35e0c61b80 10 client.17046 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:57.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.722+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1020001767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55979a53b0b0 con 0x55979a4b1d60 2024-05-27T07:41:57.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.722+0000 7f35b5600700 1 -- 172.21.15.63:0/1020001767 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35b8005500 con 0x55979a4b1d60 2024-05-27T07:41:57.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.722+0000 7f35b5600700 1 -- 172.21.15.63:0/1020001767 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f35b801c050 con 0x55979a4b1d60 2024-05-27T07:41:57.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.722+0000 7f35e0c61b80 10 client.17046 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:57.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.722+0000 7f35e0c61b80 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:57.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.722+0000 7f35e0c61b80 20 client.17046 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:57.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.723+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55979a53b620 0x55979a53da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:57.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.723+0000 7f35e0c61b80 4 client.17046 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:41:57.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.723+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1020001767 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55979a3d7570 con 0x55979a53b620 2024-05-27T07:41:57.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.723+0000 7f35cb400700 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55979a53b620 0x55979a53da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:57.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.725+0000 7f35cb400700 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55979a53b620 0x55979a53da40 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=928d07cc87c4487b server_cookie=17298d4c75515706 in_seq=0 out_seq=0 2024-05-27T07:41:57.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.725+0000 7f35b5600700 10 client.17046 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:57.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.725+0000 7f35b5600700 1 -- 172.21.15.63:0/1020001767 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55979a3d7570 con 0x55979a53b620 2024-05-27T07:41:57.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.725+0000 7f35b5600700 10 client.17046 handle_command_reply: tid=0 2024-05-27T07:41:57.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.725+0000 7f35e0c61b80 1 client.17046 shutdown 2024-05-27T07:41:57.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.725+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1020001767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b1d60 msgr2=0x55979a4bca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:57.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.725+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b1d60 0x55979a4bca20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f35b800bcb0 tx=0x7f35b800bd60 comp rx=0 tx=0).stop 2024-05-27T07:41:57.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.726+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1020001767 shutdown_connections 2024-05-27T07:41:57.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.726+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55979a53b620 0x55979a53da40 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.726+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55979a4b1d60 0x55979a4bca20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.726+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55979a32ee40 0x55979a4bfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.726+0000 7f35e0c61b80 1 --2- 172.21.15.63:0/1020001767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55979a4addb0 0x55979a4c0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.726+0000 7f35b5600700 0 client.17046 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:57.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.726+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1020001767 >> 172.21.15.63:0/1020001767 conn(0x55979a38efc0 msgr2=0x55979a4b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:57.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.726+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1020001767 shutdown_connections 2024-05-27T07:41:57.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.726+0000 7f35e0c61b80 1 -- 172.21.15.63:0/1020001767 wait complete. 2024-05-27T07:41:57.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.726+0000 7f35e0c61b80 20 client.17046 trim_cache size 0 max 16384 2024-05-27T07:41:57.725 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:41:57.725 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:41:57.725 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:41:57.725 INFO:teuthology.orchestra.run.smithi063.stdout: "6a3f674b-a399-4456-b037-384754535b13": { 2024-05-27T07:41:57.725 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:41:57.725 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "6a3f674b-a399-4456-b037-384754535b13", 2024-05-27T07:41:57.726 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:41:57.726 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:41:57.726 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:41:57.726 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:41:57.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.727+0000 7f35d2200700 1 -- 172.21.15.63:0/2826573141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f35b00b2a70 msgr2=0x7f35b00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:57.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.727+0000 7f35d2200700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f35b00b2a70 0x7f35b00b4f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35c40a8bd0 tx=0x7f35c4004b40 comp rx=0 tx=0).stop 2024-05-27T07:41:57.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.727+0000 7f35d2200700 1 -- 172.21.15.63:0/2826573141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35cc109860 msgr2=0x7f35cc1a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:57.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.727+0000 7f35d2200700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35cc109860 0x7f35cc1a17f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f35c4007ee0 tx=0x7f35c400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.728+0000 7f35d2200700 1 -- 172.21.15.63:0/2826573141 shutdown_connections 2024-05-27T07:41:57.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.728+0000 7f35d2200700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f35b00b2a70 0x7f35b00b4f80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.728+0000 7f35d2200700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35cc109860 0x7f35cc1a17f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.728+0000 7f35d2200700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35cc1045e0 0x7f35cc19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.728+0000 7f35d2200700 1 --2- 172.21.15.63:0/2826573141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35cc108f10 0x7f35cc19d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:57.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.728+0000 7f35d2200700 1 -- 172.21.15.63:0/2826573141 >> 172.21.15.63:0/2826573141 conn(0x7f35cc100030 msgr2=0x7f35cc10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:57.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.728+0000 7f35d2200700 1 -- 172.21.15.63:0/2826573141 shutdown_connections 2024-05-27T07:41:57.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:41:57.728+0000 7f35d2200700 1 -- 172.21.15.63:0/2826573141 wait complete. 2024-05-27T07:41:58.653 INFO:tasks.cephfs.filesystem:scrub status for tag:6a3f674b-a399-4456-b037-384754535b13 - {'path': '/', 'tag': '6a3f674b-a399-4456-b037-384754535b13', 'options': 'recursive,force'} 2024-05-27T07:41:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.819+0000 7fc4e6a00640 1 -- 172.21.15.63:0/3531710684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e010ea80 msgr2=0x7fc4e010eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.819+0000 7fc4e6a00640 1 --2- 172.21.15.63:0/3531710684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e010ea80 0x7fc4e010eeb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009900 tx=0x7fc4c802a3f0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.820+0000 7fc4e6a00640 1 -- 172.21.15.63:0/3531710684 shutdown_connections 2024-05-27T07:41:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.820+0000 7fc4e6a00640 1 --2- 172.21.15.63:0/3531710684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e010f540 0x7fc4e0116ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.820+0000 7fc4e6a00640 1 --2- 172.21.15.63:0/3531710684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e010ea80 0x7fc4e010eeb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.820+0000 7fc4e6a00640 1 --2- 172.21.15.63:0/3531710684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4e010a500 0x7fc4e010a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.820+0000 7fc4e6a00640 1 -- 172.21.15.63:0/3531710684 >> 172.21.15.63:0/3531710684 conn(0x7fc4e0077190 msgr2=0x7fc4e00775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.820+0000 7fc4e6a00640 1 -- 172.21.15.63:0/3531710684 shutdown_connections 2024-05-27T07:41:58.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.820+0000 7fc4e6a00640 1 -- 172.21.15.63:0/3531710684 wait complete. 2024-05-27T07:41:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.821+0000 7fc4e6a00640 1 Processor -- start 2024-05-27T07:41:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.821+0000 7fc4e6a00640 1 -- start start 2024-05-27T07:41:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.822+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4e010a500 0x7fc4e01a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.822+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e010ea80 0x7fc4e01a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.822+0000 7fc4e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e010f540 0x7fc4e01a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.822+0000 7fc4e6a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e011a340 con 0x7fc4e010f540 2024-05-27T07:41:58.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.822+0000 7fc4e6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e011a1c0 con 0x7fc4e010ea80 2024-05-27T07:41:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.822+0000 7fc4e6a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e011a4c0 con 0x7fc4e010a500 2024-05-27T07:41:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.822+0000 7fc4e4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e010f540 0x7fc4e01a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.823+0000 7fc4e4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e010f540 0x7fc4e01a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33438/0 (socket says 172.21.15.63:33438) 2024-05-27T07:41:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.823+0000 7fc4e4a00640 1 -- 172.21.15.63:0/7969414 learned_addr learned my addr 172.21.15.63:0/7969414 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.823+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4e010a500 0x7fc4e01a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.823+0000 7fc4d7400640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e010ea80 0x7fc4e01a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.823+0000 7fc4e4a00640 1 -- 172.21.15.63:0/7969414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4e010a500 msgr2=0x7fc4e01a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.823+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4e010a500 0x7fc4e01a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.823+0000 7fc4e4a00640 1 -- 172.21.15.63:0/7969414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e010ea80 msgr2=0x7fc4e01a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.823+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e010ea80 0x7fc4e01a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.823+0000 7fc4e4a00640 1 -- 172.21.15.63:0/7969414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e01a5c50 con 0x7fc4e010f540 2024-05-27T07:41:58.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.823+0000 7fc4d7400640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e010ea80 0x7fc4e01a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.824+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e010f540 0x7fc4e01a5610 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800eaa0 tx=0x7fc4d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.824+0000 7fc4dd600640 1 -- 172.21.15.63:0/7969414 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800cdb0 con 0x7fc4e010f540 2024-05-27T07:41:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.824+0000 7fc4dd600640 1 -- 172.21.15.63:0/7969414 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4d8005020 con 0x7fc4e010f540 2024-05-27T07:41:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.824+0000 7fc4e6a00640 1 -- 172.21.15.63:0/7969414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e01a5f40 con 0x7fc4e010f540 2024-05-27T07:41:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.824+0000 7fc4dd600640 1 -- 172.21.15.63:0/7969414 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80116d0 con 0x7fc4e010f540 2024-05-27T07:41:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.824+0000 7fc4e6a00640 1 -- 172.21.15.63:0/7969414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e01a6410 con 0x7fc4e010f540 2024-05-27T07:41:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.828+0000 7fc4eda05740 1 Processor -- start 2024-05-27T07:41:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.829+0000 7fc4dd600640 1 -- 172.21.15.63:0/7969414 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc4d80051d0 con 0x7fc4e010f540 2024-05-27T07:41:58.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.829+0000 7fc4eda05740 1 -- start start 2024-05-27T07:41:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.829+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d58860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.829+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da098d2f0 0x556da0d58da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.829+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09954d0 0x556da0d592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.829+0000 7fc4eda05740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556da0cc9b90 con 0x556da09b61b0 2024-05-27T07:41:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.829+0000 7fc4eda05740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556da0cb9730 con 0x556da098d2f0 2024-05-27T07:41:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.829+0000 7fc4eda05740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556da0cbd770 con 0x556da09954d0 2024-05-27T07:41:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.830+0000 7fc4d7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d58860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.830+0000 7fc4d7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d58860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33444/0 (socket says 172.21.15.63:33444) 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.830+0000 7fc4d7400640 1 -- 172.21.15.63:0/2077920559 learned_addr learned my addr 172.21.15.63:0/2077920559 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.830+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/2077920559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da098d2f0 0x556da0d58da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.830+0000 7fc4d7400640 1 --2- 172.21.15.63:0/2077920559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09954d0 0x556da0d592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.831+0000 7fc4d7400640 1 -- 172.21.15.63:0/2077920559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09954d0 msgr2=0x556da0d592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.831+0000 7fc4d7400640 1 --2- 172.21.15.63:0/2077920559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09954d0 0x556da0d592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.831+0000 7fc4d7400640 1 -- 172.21.15.63:0/2077920559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da098d2f0 msgr2=0x556da0d58da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.831+0000 7fc4d7400640 1 --2- 172.21.15.63:0/2077920559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da098d2f0 0x556da0d58da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.831+0000 7fc4d7400640 1 -- 172.21.15.63:0/2077920559 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da0c528b0 con 0x556da09b61b0 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.831+0000 7fc4d7400640 1 --2- 172.21.15.63:0/2077920559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d58860 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8000c00 tx=0x7fc4c80095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5f23bde2ce0175 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.831+0000 7fc4dd600640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4c40b2950 0x7fc4c40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.831+0000 7fc4d5200640 1 -- 172.21.15.63:0/2077920559 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8043020 con 0x556da09b61b0 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.831+0000 7fc4dd600640 1 -- 172.21.15.63:0/7969414 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fc4d80e2c20 con 0x7fc4e010f540 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.832+0000 7fc4d5200640 1 -- 172.21.15.63:0/2077920559 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4c80053c0 con 0x556da09b61b0 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.832+0000 7fc4d5200640 1 -- 172.21.15.63:0/2077920559 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8035070 con 0x556da09b61b0 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.832+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4c40b2950 0x7fc4c40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.833+0000 7fc4eda05740 1 -- 172.21.15.63:0/2077920559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 msgr2=0x556da0d58860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.833+0000 7fc4eda05740 1 --2- 172.21.15.63:0/2077920559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d58860 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8000c00 tx=0x7fc4c80095f0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.833+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4c40b2950 0x7fc4c40b4e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x556da0d59c10 tx=0x7fc4d000b4d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.833+0000 7fc4eda05740 1 -- 172.21.15.63:0/2077920559 shutdown_connections 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.833+0000 7fc4eda05740 1 --2- 172.21.15.63:0/2077920559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09954d0 0x556da0d592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.833+0000 7fc4eda05740 1 --2- 172.21.15.63:0/2077920559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da098d2f0 0x556da0d58da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.833+0000 7fc4eda05740 1 --2- 172.21.15.63:0/2077920559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d58860 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.833+0000 7fc4eda05740 1 -- 172.21.15.63:0/2077920559 >> 172.21.15.63:0/2077920559 conn(0x556da09aa300 msgr2=0x556da0d55010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.833+0000 7fc4eda05740 1 -- 172.21.15.63:0/2077920559 shutdown_connections 2024-05-27T07:41:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.833+0000 7fc4eda05740 1 -- 172.21.15.63:0/2077920559 wait complete. 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.834+0000 7fc4eda05740 1 -- start start 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.834+0000 7fc4eda05740 1 Processor -- start 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da098d2f0 0x556da0d62ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09954d0 0x556da0d631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d6ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4eda05740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556da099f790 con 0x556da09b61b0 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4eda05740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556da0c61bc0 con 0x556da098d2f0 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4eda05740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556da0d65930 con 0x556da09954d0 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4eda05740 10 client.0 _ll_register_callbacks cb 0x556da0c3f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4e4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d6ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4e4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d6ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33446/0 (socket says 172.21.15.63:33446) 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4e4a00640 1 -- 172.21.15.63:0/2506585890 learned_addr learned my addr 172.21.15.63:0/2506585890 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4d7400640 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09954d0 0x556da0d631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.835+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da098d2f0 0x556da0d62ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4e4a00640 1 -- 172.21.15.63:0/2506585890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09954d0 msgr2=0x556da0d631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09954d0 0x556da0d631e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4e4a00640 1 -- 172.21.15.63:0/2506585890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da098d2f0 msgr2=0x556da0d62ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da098d2f0 0x556da0d62ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4e4a00640 1 -- 172.21.15.63:0/2506585890 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da09ce210 con 0x556da09b61b0 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d6ec60 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80a5bc0 tx=0x7fc4d8005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfadd78bfda55f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4cd600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4cd600640 1 -- 172.21.15.63:0/2506585890 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d801f030 con 0x556da09b61b0 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4eda05740 1 -- 172.21.15.63:0/2506585890 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c80053c0 con 0x556da09b61b0 2024-05-27T07:41:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4cd600640 1 -- 172.21.15.63:0/2506585890 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fc4d80a5dd0 con 0x556da09b61b0 2024-05-27T07:41:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.836+0000 7fc4cd600640 1 -- 172.21.15.63:0/2506585890 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc4d80a9b40 con 0x556da09b61b0 2024-05-27T07:41:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.837+0000 7fc4eda05740 10 client.17058 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.837+0000 7fc4eda05740 1 -- 172.21.15.63:0/2506585890 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556da0a9a620 con 0x556da09b61b0 2024-05-27T07:41:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.837+0000 7fc4cd600640 1 -- 172.21.15.63:0/2506585890 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4c80053c0 con 0x556da09b61b0 2024-05-27T07:41:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.837+0000 7fc4cd600640 1 -- 172.21.15.63:0/2506585890 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc4d800f040 con 0x556da09b61b0 2024-05-27T07:41:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.837+0000 7fc4eda05740 10 client.17058 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.837+0000 7fc4eda05740 10 client.17058 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.837+0000 7fc4eda05740 20 client.17058 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.837+0000 7fc4eda05740 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556da0974d80 0x556da0d74ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.837+0000 7fc4eda05740 4 client.17058 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.837+0000 7fc4eda05740 1 -- 172.21.15.63:0/2506585890 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556da0ccbae0 con 0x556da0974d80 2024-05-27T07:41:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.838+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556da0974d80 0x556da0d74ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.839+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556da0974d80 0x556da0d74ca0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8166c53300e2b0 server_cookie=c0b3e8e7c4331f92 in_seq=0 out_seq=0 2024-05-27T07:41:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.839+0000 7fc4cd600640 10 client.17058 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.841+0000 7fc4cd600640 1 -- 172.21.15.63:0/2506585890 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556da0ccbae0 con 0x556da0974d80 2024-05-27T07:41:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.841+0000 7fc4cd600640 10 client.17058 handle_command_reply: tid=0 2024-05-27T07:41:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.841+0000 7fc4eda05740 1 client.17058 shutdown 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.841+0000 7fc4eda05740 1 -- 172.21.15.63:0/2506585890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 msgr2=0x556da0d6ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.841+0000 7fc4eda05740 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d6ec60 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80a5bc0 tx=0x7fc4d8005720 comp rx=0 tx=0).stop 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.841+0000 7fc4eda05740 1 -- 172.21.15.63:0/2506585890 shutdown_connections 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.841+0000 7fc4eda05740 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556da0974d80 0x556da0d74ca0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.842+0000 7fc4eda05740 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d6ec60 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.842+0000 7fc4eda05740 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09954d0 0x556da0d631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.842+0000 7fc4eda05740 1 --2- 172.21.15.63:0/2506585890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da098d2f0 0x556da0d62ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.842+0000 7fc4eda05740 1 -- 172.21.15.63:0/2506585890 >> 172.21.15.63:0/2506585890 conn(0x556da09aa300 msgr2=0x556da0d58630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.842+0000 7fc4cd600640 0 client.17058 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.842+0000 7fc4eda05740 1 -- 172.21.15.63:0/2506585890 shutdown_connections 2024-05-27T07:41:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.842+0000 7fc4eda05740 1 -- 172.21.15.63:0/2506585890 wait complete. 2024-05-27T07:41:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.842+0000 7fc4eda05740 20 client.17058 trim_cache size 0 max 16384 2024-05-27T07:41:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.864+0000 7fc4eda05740 1 Processor -- start 2024-05-27T07:41:58.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.864+0000 7fc4eda05740 1 -- start start 2024-05-27T07:41:58.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.864+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da098d2f0 0x556da0d5e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.864+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da09954d0 0x556da0d648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09b61b0 0x556da0d64e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4eda05740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e01b9f40 con 0x556da098d2f0 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4eda05740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e01b7810 con 0x556da09954d0 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4eda05740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e01b5680 con 0x556da09b61b0 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da098d2f0 0x556da0d5e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da098d2f0 0x556da0d5e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33452/0 (socket says 172.21.15.63:33452) 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4dfe00640 1 -- 172.21.15.63:0/954555638 learned_addr learned my addr 172.21.15.63:0/954555638 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/954555638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09b61b0 0x556da0d64e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4d7400640 1 --2- 172.21.15.63:0/954555638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da09954d0 0x556da0d648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4dfe00640 1 -- 172.21.15.63:0/954555638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09b61b0 msgr2=0x556da0d64e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/954555638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09b61b0 0x556da0d64e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4dfe00640 1 -- 172.21.15.63:0/954555638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da09954d0 msgr2=0x556da0d648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/954555638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da09954d0 0x556da0d648c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4dfe00640 1 -- 172.21.15.63:0/954555638 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da0d59c40 con 0x556da098d2f0 2024-05-27T07:41:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.865+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/954555638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09b61b0 0x556da0d64e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:41:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.866+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/954555638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da098d2f0 0x556da0d5e290 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000b7f0 tx=0x7fc4d0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84be07751cc15a86 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.866+0000 7fc4cfe00640 1 -- 172.21.15.63:0/954555638 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00295a0 con 0x556da098d2f0 2024-05-27T07:41:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.866+0000 7fc4cfe00640 1 -- 172.21.15.63:0/954555638 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4d002f070 con 0x556da098d2f0 2024-05-27T07:41:58.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.866+0000 7fc4cfe00640 1 -- 172.21.15.63:0/954555638 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0020bd0 con 0x556da098d2f0 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.867+0000 7fc4eda05740 1 -- 172.21.15.63:0/954555638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da098d2f0 msgr2=0x556da0d5e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.867+0000 7fc4eda05740 1 --2- 172.21.15.63:0/954555638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da098d2f0 0x556da0d5e290 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000b7f0 tx=0x7fc4d0015000 comp rx=0 tx=0).stop 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.867+0000 7fc4eda05740 1 -- 172.21.15.63:0/954555638 shutdown_connections 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.867+0000 7fc4eda05740 1 --2- 172.21.15.63:0/954555638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da09b61b0 0x556da0d64e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.867+0000 7fc4eda05740 1 --2- 172.21.15.63:0/954555638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da09954d0 0x556da0d648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.867+0000 7fc4eda05740 1 --2- 172.21.15.63:0/954555638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da098d2f0 0x556da0d5e290 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.867+0000 7fc4eda05740 1 -- 172.21.15.63:0/954555638 >> 172.21.15.63:0/954555638 conn(0x556da09aa300 msgr2=0x556da0d641b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.867+0000 7fc4eda05740 1 -- 172.21.15.63:0/954555638 shutdown_connections 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.867+0000 7fc4eda05740 1 -- 172.21.15.63:0/954555638 wait complete. 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.868+0000 7fc4eda05740 1 -- start start 2024-05-27T07:41:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.868+0000 7fc4eda05740 1 Processor -- start 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da098d2f0 0x556da0d68aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da09954d0 0x556da0d68fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4eda05740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d69520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4eda05740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e01b5cd0 con 0x556da09b61b0 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4eda05740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e01b7140 con 0x556da09954d0 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4eda05740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e01b63a0 con 0x556da098d2f0 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4eda05740 10 client.0 _ll_register_callbacks cb 0x7fc4d80a5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4e4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d69520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4e4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d69520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33454/0 (socket says 172.21.15.63:33454) 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4e4a00640 1 -- 172.21.15.63:0/4016746986 learned_addr learned my addr 172.21.15.63:0/4016746986 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:41:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da098d2f0 0x556da0d68aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.869+0000 7fc4d7400640 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da09954d0 0x556da0d68fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.870+0000 7fc4e4a00640 1 -- 172.21.15.63:0/4016746986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da098d2f0 msgr2=0x556da0d68aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.870+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da098d2f0 0x556da0d68aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.870+0000 7fc4e4a00640 1 -- 172.21.15.63:0/4016746986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da09954d0 msgr2=0x556da0d68fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.870+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da09954d0 0x556da0d68fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.870+0000 7fc4e4a00640 1 -- 172.21.15.63:0/4016746986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da0d638d0 con 0x556da09b61b0 2024-05-27T07:41:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.870+0000 7fc4e4a00640 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d69520 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80a5440 tx=0x7fc4d800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d6ffe9571b7f82 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.870+0000 7fc4cd600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.870+0000 7fc4cd600640 1 -- 172.21.15.63:0/4016746986 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d801f070 con 0x556da09b61b0 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.870+0000 7fc4cd600640 1 -- 172.21.15.63:0/4016746986 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9483+0+0 (secure 0 0 0) 0x7fc4d80a8da0 con 0x556da09b61b0 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.870+0000 7fc4eda05740 1 -- 172.21.15.63:0/4016746986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d002f070 con 0x556da09b61b0 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4cd600640 1 -- 172.21.15.63:0/4016746986 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc4d80a92b0 con 0x556da09b61b0 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4eda05740 10 client.17064 fetch_fsmap learned FSMap version 33 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4eda05740 1 -- 172.21.15.63:0/4016746986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556da0ddd450 con 0x556da09b61b0 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4cd600640 1 -- 172.21.15.63:0/4016746986 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4d002f070 con 0x556da09b61b0 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4cd600640 1 -- 172.21.15.63:0/4016746986 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc4d80af080 con 0x556da09b61b0 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4eda05740 10 client.17064 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4eda05740 10 client.17064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4eda05740 20 client.17064 populate_metadata read hostname 'smithi063' 2024-05-27T07:41:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4eda05740 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556da094a1c0 0x556da0ddfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4eda05740 4 client.17064 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:41:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.871+0000 7fc4eda05740 1 -- 172.21.15.63:0/4016746986 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556da0cc52c0 con 0x556da094a1c0 2024-05-27T07:41:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.872+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556da094a1c0 0x556da0ddfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.873+0000 7fc4dfe00640 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556da094a1c0 0x556da0ddfb50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d89cba98d0f4a48 server_cookie=3baa4a98e3669bba in_seq=0 out_seq=0 2024-05-27T07:41:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.873+0000 7fc4cd600640 10 client.17064 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4cd600640 1 -- 172.21.15.63:0/4016746986 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556da0cc52c0 con 0x556da094a1c0 2024-05-27T07:41:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4cd600640 10 client.17064 handle_command_reply: tid=0 2024-05-27T07:41:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 client.17064 shutdown 2024-05-27T07:41:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 -- 172.21.15.63:0/4016746986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 msgr2=0x556da0d69520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d69520 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80a5440 tx=0x7fc4d800c800 comp rx=0 tx=0).stop 2024-05-27T07:41:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 -- 172.21.15.63:0/4016746986 shutdown_connections 2024-05-27T07:41:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556da094a1c0 0x556da0ddfb50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556da09b61b0 0x556da0d69520 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556da09954d0 0x556da0d68fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 --2- 172.21.15.63:0/4016746986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556da098d2f0 0x556da0d68aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4cd600640 0 client.17064 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:41:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 -- 172.21.15.63:0/4016746986 >> 172.21.15.63:0/4016746986 conn(0x556da09aa300 msgr2=0x556da0d6d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 -- 172.21.15.63:0/4016746986 shutdown_connections 2024-05-27T07:41:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.909+0000 7fc4eda05740 1 -- 172.21.15.63:0/4016746986 wait complete. 2024-05-27T07:41:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.910+0000 7fc4eda05740 20 client.17064 trim_cache size 0 max 16384 2024-05-27T07:41:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:41:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:41:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:41:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 -- 172.21.15.63:0/7969414 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4c40b2950 msgr2=0x7fc4c40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4c40b2950 0x7fc4c40b4e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x556da0d59c10 tx=0x7fc4d000b4d0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 -- 172.21.15.63:0/7969414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e010f540 msgr2=0x7fc4e01a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e010f540 0x7fc4e01a5610 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800eaa0 tx=0x7fc4d800ef60 comp rx=0 tx=0).stop 2024-05-27T07:41:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 -- 172.21.15.63:0/7969414 shutdown_connections 2024-05-27T07:41:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4c40b2950 0x7fc4c40b4e60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e010f540 0x7fc4e01a5610 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e010ea80 0x7fc4e01a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 --2- 172.21.15.63:0/7969414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc4e010a500 0x7fc4e01a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 -- 172.21.15.63:0/7969414 >> 172.21.15.63:0/7969414 conn(0x7fc4e0077190 msgr2=0x7fc4e007a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 -- 172.21.15.63:0/7969414 shutdown_connections 2024-05-27T07:41:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:41:58.911+0000 7fc4e6a00640 1 -- 172.21.15.63:0/7969414 wait complete. 2024-05-27T07:41:58.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:41:58 smithi086 ceph-mon[34687]: pgmap v259: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.0 MiB/s wr, 39 op/s 2024-05-27T07:41:58.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:41:58 smithi149 ceph-mon[35408]: pgmap v259: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.0 MiB/s wr, 39 op/s 2024-05-27T07:41:58.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:41:58 smithi063 ceph-mon[31613]: pgmap v259: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.0 MiB/s wr, 39 op/s 2024-05-27T07:42:00.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:00 smithi086 ceph-mon[34687]: pgmap v260: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.3 MiB/s wr, 32 op/s 2024-05-27T07:42:00.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:00 smithi149 ceph-mon[35408]: pgmap v260: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.3 MiB/s wr, 32 op/s 2024-05-27T07:42:00.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:00 smithi063 ceph-mon[31613]: pgmap v260: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.3 MiB/s wr, 32 op/s 2024-05-27T07:42:01.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:01 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:42:01.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:01 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:42:01.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:01 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:42:02.902 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:42:02.902 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:42:02.902 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:02 smithi063 ceph-mon[31613]: pgmap v261: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 56 op/s 2024-05-27T07:42:02.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:42:02.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:42:02.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:02 smithi086 ceph-mon[34687]: pgmap v261: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 56 op/s 2024-05-27T07:42:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:42:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:42:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:02 smithi149 ceph-mon[35408]: pgmap v261: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 56 op/s 2024-05-27T07:42:04.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:04 smithi086 ceph-mon[34687]: pgmap v262: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.7 MiB/s wr, 53 op/s 2024-05-27T07:42:04.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:04 smithi149 ceph-mon[35408]: pgmap v262: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.7 MiB/s wr, 53 op/s 2024-05-27T07:42:04.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:04 smithi063 ceph-mon[31613]: pgmap v262: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.7 MiB/s wr, 53 op/s 2024-05-27T07:42:06.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:06 smithi086 ceph-mon[34687]: pgmap v263: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.7 MiB/s wr, 53 op/s 2024-05-27T07:42:06.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:06 smithi149 ceph-mon[35408]: pgmap v263: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.7 MiB/s wr, 53 op/s 2024-05-27T07:42:06.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:06 smithi063 ceph-mon[31613]: pgmap v263: 129 pgs: 129 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.7 MiB/s wr, 53 op/s 2024-05-27T07:42:07.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:07 smithi086 ceph-mon[34687]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T07:42:07.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:07 smithi149 ceph-mon[35408]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T07:42:07.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:07 smithi063 ceph-mon[31613]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T07:42:08.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:08 smithi086 ceph-mon[34687]: pgmap v265: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:42:08.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:08 smithi149 ceph-mon[35408]: pgmap v265: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:42:08.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:08 smithi063 ceph-mon[31613]: pgmap v265: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:42:10.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:10 smithi086 ceph-mon[34687]: pgmap v266: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:42:10.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:10 smithi149 ceph-mon[35408]: pgmap v266: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:42:10.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:10 smithi063 ceph-mon[31613]: pgmap v266: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:42:11.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:11 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:42:11.960 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:11 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:42:11.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:11 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:42:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.160+0000 7f9d6fe00640 1 -- 172.21.15.63:0/959521393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d701103c0 msgr2=0x7f9d701127e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.160+0000 7f9d6fe00640 1 --2- 172.21.15.63:0/959521393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d701103c0 0x7f9d701127e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009ca0 tx=0x7f9d6802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.161+0000 7f9d6fe00640 1 -- 172.21.15.63:0/959521393 shutdown_connections 2024-05-27T07:42:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.161+0000 7f9d6fe00640 1 --2- 172.21.15.63:0/959521393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d701103c0 0x7f9d701127e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.161+0000 7f9d6fe00640 1 --2- 172.21.15.63:0/959521393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d7010da60 0x7f9d7010fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.161+0000 7f9d6fe00640 1 --2- 172.21.15.63:0/959521393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d70102820 0x7f9d70102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.161+0000 7f9d6fe00640 1 -- 172.21.15.63:0/959521393 >> 172.21.15.63:0/959521393 conn(0x7f9d700fe210 msgr2=0x7f9d70100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.161+0000 7f9d6fe00640 1 -- 172.21.15.63:0/959521393 shutdown_connections 2024-05-27T07:42:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.161+0000 7f9d6fe00640 1 -- 172.21.15.63:0/959521393 wait complete. 2024-05-27T07:42:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.162+0000 7f9d6fe00640 1 Processor -- start 2024-05-27T07:42:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.163+0000 7f9d6fe00640 1 -- start start 2024-05-27T07:42:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.163+0000 7f9d6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d70102820 0x7f9d7019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.163+0000 7f9d6fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d7010da60 0x7f9d7019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.163+0000 7f9d6fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d701103c0 0x7f9d701a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.163+0000 7f9d6fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70115d80 con 0x7f9d701103c0 2024-05-27T07:42:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.163+0000 7f9d6fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70115c00 con 0x7f9d70102820 2024-05-27T07:42:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.163+0000 7f9d6fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70115f00 con 0x7f9d7010da60 2024-05-27T07:42:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.164+0000 7f9d6f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d701103c0 0x7f9d701a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.164+0000 7f9d6f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d701103c0 0x7f9d701a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43570/0 (socket says 172.21.15.63:43570) 2024-05-27T07:42:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.164+0000 7f9d6f400640 1 -- 172.21.15.63:0/404123389 learned_addr learned my addr 172.21.15.63:0/404123389 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.164+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d70102820 0x7f9d7019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.164+0000 7f9d6e000640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d7010da60 0x7f9d7019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.164+0000 7f9d6f400640 1 -- 172.21.15.63:0/404123389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d7010da60 msgr2=0x7f9d7019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.164+0000 7f9d6f400640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d7010da60 0x7f9d7019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.165+0000 7f9d6f400640 1 -- 172.21.15.63:0/404123389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d70102820 msgr2=0x7f9d7019c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.165+0000 7f9d6f400640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d70102820 0x7f9d7019c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.165+0000 7f9d6f400640 1 -- 172.21.15.63:0/404123389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d701a15c0 con 0x7f9d701103c0 2024-05-27T07:42:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.165+0000 7f9d6f400640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d701103c0 0x7f9d701a0f80 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9d68031040 tx=0x7f9d68032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.165+0000 7f9d67400640 1 -- 172.21.15.63:0/404123389 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68004040 con 0x7f9d701103c0 2024-05-27T07:42:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.165+0000 7f9d6fe00640 1 -- 172.21.15.63:0/404123389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d701a1850 con 0x7f9d701103c0 2024-05-27T07:42:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.166+0000 7f9d6fe00640 1 -- 172.21.15.63:0/404123389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d701a9230 con 0x7f9d701103c0 2024-05-27T07:42:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.166+0000 7f9d67400640 1 -- 172.21.15.63:0/404123389 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d68004670 con 0x7f9d701103c0 2024-05-27T07:42:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.166+0000 7f9d67400640 1 -- 172.21.15.63:0/404123389 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68030bb0 con 0x7f9d701103c0 2024-05-27T07:42:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.169+0000 7f9d7c776740 1 Processor -- start 2024-05-27T07:42:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.170+0000 7f9d67400640 1 -- 172.21.15.63:0/404123389 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9d6804e430 con 0x7f9d701103c0 2024-05-27T07:42:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.171+0000 7f9d7c776740 1 -- start start 2024-05-27T07:42:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.171+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a911b0 0x560425e33a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.171+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e33fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.171+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a704d0 0x560425e344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.171+0000 7f9d7c776740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560425da4b90 con 0x560425a682f0 2024-05-27T07:42:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.171+0000 7f9d7c776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560425d94730 con 0x560425a704d0 2024-05-27T07:42:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.171+0000 7f9d7c776740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560425d98770 con 0x560425a911b0 2024-05-27T07:42:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.172+0000 7f9d6ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e33fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.172+0000 7f9d6e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a704d0 0x560425e344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.172+0000 7f9d6ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e33fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43576/0 (socket says 172.21.15.63:43576) 2024-05-27T07:42:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.172+0000 7f9d6ea00640 1 -- 172.21.15.63:0/2029947288 learned_addr learned my addr 172.21.15.63:0/2029947288 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.172+0000 7f9d6e000640 1 --2- 172.21.15.63:0/2029947288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a911b0 0x560425e33a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.172+0000 7f9d6ea00640 1 -- 172.21.15.63:0/2029947288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a911b0 msgr2=0x560425e33a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.173+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/2029947288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a911b0 0x560425e33a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.173+0000 7f9d6ea00640 1 -- 172.21.15.63:0/2029947288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a704d0 msgr2=0x560425e344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.173+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/2029947288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a704d0 0x560425e344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.173+0000 7f9d6ea00640 1 -- 172.21.15.63:0/2029947288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560425d2d8b0 con 0x560425a682f0 2024-05-27T07:42:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.173+0000 7f9d67400640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d540b29d0 0x7f9d540b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.173+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/2029947288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e33fa0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000bd10 tx=0x7f9d600107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c971609743da7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.173+0000 7f9d67400640 1 -- 172.21.15.63:0/404123389 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9571+0+0 (secure 0 0 0) 0x7f9d68103d90 con 0x7f9d701103c0 2024-05-27T07:42:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.173+0000 7f9d5be00640 1 -- 172.21.15.63:0/2029947288 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60004030 con 0x560425a682f0 2024-05-27T07:42:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.174+0000 7f9d5be00640 1 -- 172.21.15.63:0/2029947288 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d600035f0 con 0x560425a682f0 2024-05-27T07:42:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.174+0000 7f9d5be00640 1 -- 172.21.15.63:0/2029947288 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60011660 con 0x560425a682f0 2024-05-27T07:42:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.174+0000 7f9d6e000640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d540b29d0 0x7f9d540b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.175+0000 7f9d7c776740 1 -- 172.21.15.63:0/2029947288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 msgr2=0x560425e33fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.175+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2029947288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e33fa0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000bd10 tx=0x7f9d600107a0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.175+0000 7f9d7c776740 1 -- 172.21.15.63:0/2029947288 shutdown_connections 2024-05-27T07:42:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.175+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2029947288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a704d0 0x560425e344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.175+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2029947288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e33fa0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.175+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2029947288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a911b0 0x560425e33a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.175+0000 7f9d7c776740 1 -- 172.21.15.63:0/2029947288 >> 172.21.15.63:0/2029947288 conn(0x560425a85300 msgr2=0x560425e30210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.175+0000 7f9d7c776740 1 -- 172.21.15.63:0/2029947288 shutdown_connections 2024-05-27T07:42:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.175+0000 7f9d6e000640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d540b29d0 0x7f9d540b4ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00bfd0 tx=0x7f9d5c0046c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.175+0000 7f9d7c776740 1 -- 172.21.15.63:0/2029947288 wait complete. 2024-05-27T07:42:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.176+0000 7f9d7c776740 1 -- start start 2024-05-27T07:42:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.176+0000 7f9d7c776740 1 Processor -- start 2024-05-27T07:42:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e3dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a704d0 0x560425e3e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 0x560425e49f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d7c776740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560425a7a790 con 0x560425a682f0 2024-05-27T07:42:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d7c776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560425d3cbc0 con 0x560425a911b0 2024-05-27T07:42:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d7c776740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560425e40b90 con 0x560425a704d0 2024-05-27T07:42:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d7c776740 10 client.0 _ll_register_callbacks cb 0x560425d1a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d6ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e3dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d6ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e3dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43578/0 (socket says 172.21.15.63:43578) 2024-05-27T07:42:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d6ea00640 1 -- 172.21.15.63:0/2741295878 learned_addr learned my addr 172.21.15.63:0/2741295878 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.177+0000 7f9d6e000640 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a704d0 0x560425e3e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.178+0000 7f9d6f400640 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 0x560425e49f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.178+0000 7f9d6ea00640 1 -- 172.21.15.63:0/2741295878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a704d0 msgr2=0x560425e3e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.178+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a704d0 0x560425e3e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.178+0000 7f9d6ea00640 1 -- 172.21.15.63:0/2741295878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 msgr2=0x560425e49f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:12.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.178+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 0x560425e49f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.178+0000 7f9d6ea00640 1 -- 172.21.15.63:0/2741295878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560425aa9210 con 0x560425a682f0 2024-05-27T07:42:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.178+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e3dbc0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9d6001a000 tx=0x7f9d6000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfefa9ec5f3e9a0e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.178+0000 7f9d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.179+0000 7f9d58c00640 1 -- 172.21.15.63:0/2741295878 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60029060 con 0x560425a682f0 2024-05-27T07:42:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.179+0000 7f9d58c00640 1 -- 172.21.15.63:0/2741295878 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9571+0+0 (secure 0 0 0) 0x7f9d60003fb0 con 0x560425a682f0 2024-05-27T07:42:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.179+0000 7f9d7c776740 1 -- 172.21.15.63:0/2741295878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d600035f0 con 0x560425a682f0 2024-05-27T07:42:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.179+0000 7f9d58c00640 1 -- 172.21.15.63:0/2741295878 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9d60030050 con 0x560425a682f0 2024-05-27T07:42:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.179+0000 7f9d7c776740 10 client.17076 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.179+0000 7f9d7c776740 1 -- 172.21.15.63:0/2741295878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560425b75620 con 0x560425a682f0 2024-05-27T07:42:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.180+0000 7f9d58c00640 1 -- 172.21.15.63:0/2741295878 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d600035f0 con 0x560425a682f0 2024-05-27T07:42:12.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.180+0000 7f9d58c00640 1 -- 172.21.15.63:0/2741295878 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9d60023030 con 0x560425a682f0 2024-05-27T07:42:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.180+0000 7f9d7c776740 10 client.17076 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.180+0000 7f9d7c776740 10 client.17076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.180+0000 7f9d7c776740 20 client.17076 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.180+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560425a4fd80 0x560425e4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.180+0000 7f9d7c776740 4 client.17076 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:42:12.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.180+0000 7f9d7c776740 1 -- 172.21.15.63:0/2741295878 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560425da6ae0 con 0x560425a4fd80 2024-05-27T07:42:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.181+0000 7f9d6e000640 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560425a4fd80 0x560425e4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.182+0000 7f9d6e000640 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560425a4fd80 0x560425e4fd90 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b8f4e00ec6c433 server_cookie=3ea40a848ccddd00 in_seq=0 out_seq=0 2024-05-27T07:42:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.182+0000 7f9d58c00640 10 client.17076 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d58c00640 1 -- 172.21.15.63:0/2741295878 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560425da6ae0 con 0x560425a4fd80 2024-05-27T07:42:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d58c00640 10 client.17076 handle_command_reply: tid=0 2024-05-27T07:42:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d7c776740 1 client.17076 shutdown 2024-05-27T07:42:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d7c776740 1 -- 172.21.15.63:0/2741295878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 msgr2=0x560425e3dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e3dbc0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9d6001a000 tx=0x7f9d6000b8e0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d7c776740 1 -- 172.21.15.63:0/2741295878 shutdown_connections 2024-05-27T07:42:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560425a4fd80 0x560425e4fd90 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 0x560425e49f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a704d0 0x560425e3e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2741295878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e3dbc0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d7c776740 1 -- 172.21.15.63:0/2741295878 >> 172.21.15.63:0/2741295878 conn(0x560425a85300 msgr2=0x560425e33890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d58c00640 0 client.17076 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.184+0000 7f9d7c776740 1 -- 172.21.15.63:0/2741295878 shutdown_connections 2024-05-27T07:42:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.185+0000 7f9d7c776740 1 -- 172.21.15.63:0/2741295878 wait complete. 2024-05-27T07:42:12.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.185+0000 7f9d7c776740 20 client.17076 trim_cache size 0 max 16384 2024-05-27T07:42:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.205+0000 7f9d7c776740 1 Processor -- start 2024-05-27T07:42:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.205+0000 7f9d7c776740 1 -- start start 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.205+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a682f0 0x560425e34800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.205+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a704d0 0x560425e2f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.205+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 0x560425e402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.205+0000 7f9d7c776740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d50012900 con 0x560425a704d0 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.205+0000 7f9d7c776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d500101d0 con 0x560425a911b0 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.205+0000 7f9d7c776740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5000e040 con 0x560425a682f0 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.206+0000 7f9d6e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a704d0 0x560425e2f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.206+0000 7f9d6e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a704d0 0x560425e2f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43592/0 (socket says 172.21.15.63:43592) 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.206+0000 7f9d6e000640 1 -- 172.21.15.63:0/2278142213 learned_addr learned my addr 172.21.15.63:0/2278142213 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.206+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/2278142213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a682f0 0x560425e34800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.206+0000 7f9d6f400640 1 --2- 172.21.15.63:0/2278142213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 0x560425e402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.206+0000 7f9d6e000640 1 -- 172.21.15.63:0/2278142213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a682f0 msgr2=0x560425e34800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.206+0000 7f9d6e000640 1 --2- 172.21.15.63:0/2278142213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a682f0 0x560425e34800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.206+0000 7f9d6e000640 1 -- 172.21.15.63:0/2278142213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 msgr2=0x560425e402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.206+0000 7f9d6e000640 1 --2- 172.21.15.63:0/2278142213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 0x560425e402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.206+0000 7f9d6e000640 1 -- 172.21.15.63:0/2278142213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560425e3eb20 con 0x560425a704d0 2024-05-27T07:42:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.207+0000 7f9d6f400640 1 --2- 172.21.15.63:0/2278142213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 0x560425e402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.207+0000 7f9d6e000640 1 --2- 172.21.15.63:0/2278142213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a704d0 0x560425e2f240 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c005fd0 tx=0x7f9d5c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2870a83056a1daa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.207+0000 7f9d5b400640 1 -- 172.21.15.63:0/2278142213 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c033070 con 0x560425a704d0 2024-05-27T07:42:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.207+0000 7f9d5b400640 1 -- 172.21.15.63:0/2278142213 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d5c010670 con 0x560425a704d0 2024-05-27T07:42:12.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.207+0000 7f9d5b400640 1 -- 172.21.15.63:0/2278142213 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c022810 con 0x560425a704d0 2024-05-27T07:42:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.208+0000 7f9d7c776740 1 -- 172.21.15.63:0/2278142213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a704d0 msgr2=0x560425e2f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.208+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2278142213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a704d0 0x560425e2f240 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c005fd0 tx=0x7f9d5c014040 comp rx=0 tx=0).stop 2024-05-27T07:42:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.209+0000 7f9d7c776740 1 -- 172.21.15.63:0/2278142213 shutdown_connections 2024-05-27T07:42:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.209+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2278142213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a911b0 0x560425e402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.209+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2278142213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a704d0 0x560425e2f240 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.209+0000 7f9d7c776740 1 --2- 172.21.15.63:0/2278142213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a682f0 0x560425e34800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.209+0000 7f9d7c776740 1 -- 172.21.15.63:0/2278142213 >> 172.21.15.63:0/2278142213 conn(0x560425a85300 msgr2=0x560425e3d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.209+0000 7f9d7c776740 1 -- 172.21.15.63:0/2278142213 shutdown_connections 2024-05-27T07:42:12.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.209+0000 7f9d7c776740 1 -- 172.21.15.63:0/2278142213 wait complete. 2024-05-27T07:42:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.209+0000 7f9d7c776740 1 -- start start 2024-05-27T07:42:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.209+0000 7f9d7c776740 1 Processor -- start 2024-05-27T07:42:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.210+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e40e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.210+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a704d0 0x560425e413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.210+0000 7f9d7c776740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a911b0 0x560425e418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.210+0000 7f9d7c776740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5000e690 con 0x560425a682f0 2024-05-27T07:42:12.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.210+0000 7f9d7c776740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5000fb00 con 0x560425a704d0 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.210+0000 7f9d7c776740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5000ed60 con 0x560425a911b0 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.210+0000 7f9d7c776740 10 client.0 _ll_register_callbacks cb 0x7f9d5c00bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.210+0000 7f9d6ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e40e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.210+0000 7f9d6ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e40e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43594/0 (socket says 172.21.15.63:43594) 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.210+0000 7f9d6ea00640 1 -- 172.21.15.63:0/1336076164 learned_addr learned my addr 172.21.15.63:0/1336076164 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.211+0000 7f9d6e000640 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a704d0 0x560425e413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.211+0000 7f9d6f400640 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a911b0 0x560425e418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.211+0000 7f9d6ea00640 1 -- 172.21.15.63:0/1336076164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a911b0 msgr2=0x560425e418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.211+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a911b0 0x560425e418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.211+0000 7f9d6ea00640 1 -- 172.21.15.63:0/1336076164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a704d0 msgr2=0x560425e413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.211+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a704d0 0x560425e413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.211+0000 7f9d6ea00640 1 -- 172.21.15.63:0/1336076164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560425eb6250 con 0x560425a682f0 2024-05-27T07:42:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.211+0000 7f9d6ea00640 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e40e70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9d60004740 tx=0x7f9d60004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a547ca8a83682243 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.211+0000 7f9d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.211+0000 7f9d58c00640 1 -- 172.21.15.63:0/1336076164 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60029040 con 0x560425a682f0 2024-05-27T07:42:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.212+0000 7f9d7c776740 1 -- 172.21.15.63:0/1336076164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5c010670 con 0x560425a682f0 2024-05-27T07:42:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.212+0000 7f9d58c00640 1 -- 172.21.15.63:0/1336076164 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9571+0+0 (secure 0 0 0) 0x7f9d60004fd0 con 0x560425a682f0 2024-05-27T07:42:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.212+0000 7f9d58c00640 1 -- 172.21.15.63:0/1336076164 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9d60042050 con 0x560425a682f0 2024-05-27T07:42:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.212+0000 7f9d7c776740 10 client.17082 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.212+0000 7f9d7c776740 1 -- 172.21.15.63:0/1336076164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560425eb7480 con 0x560425a682f0 2024-05-27T07:42:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.212+0000 7f9d58c00640 1 -- 172.21.15.63:0/1336076164 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d5c010670 con 0x560425a682f0 2024-05-27T07:42:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.213+0000 7f9d58c00640 1 -- 172.21.15.63:0/1336076164 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9d60023040 con 0x560425a682f0 2024-05-27T07:42:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.213+0000 7f9d7c776740 10 client.17082 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.213+0000 7f9d7c776740 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.213+0000 7f9d7c776740 20 client.17082 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.213+0000 7f9d7c776740 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560425a251c0 0x560425eb9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.213+0000 7f9d7c776740 4 client.17082 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:42:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.213+0000 7f9d7c776740 1 -- 172.21.15.63:0/1336076164 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560425da02c0 con 0x560425a251c0 2024-05-27T07:42:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.213+0000 7f9d6e000640 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560425a251c0 0x560425eb9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.214+0000 7f9d6e000640 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560425a251c0 0x560425eb9bd0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32150358baca3966 server_cookie=453dde962cca66fb in_seq=0 out_seq=0 2024-05-27T07:42:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.214+0000 7f9d58c00640 10 client.17082 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.246+0000 7f9d58c00640 1 -- 172.21.15.63:0/1336076164 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560425da02c0 con 0x560425a251c0 2024-05-27T07:42:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.246+0000 7f9d58c00640 10 client.17082 handle_command_reply: tid=0 2024-05-27T07:42:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.246+0000 7f9d7c776740 1 client.17082 shutdown 2024-05-27T07:42:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.246+0000 7f9d7c776740 1 -- 172.21.15.63:0/1336076164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 msgr2=0x560425e40e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.246+0000 7f9d7c776740 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e40e70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9d60004740 tx=0x7f9d60004970 comp rx=0 tx=0).stop 2024-05-27T07:42:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.246+0000 7f9d7c776740 1 -- 172.21.15.63:0/1336076164 shutdown_connections 2024-05-27T07:42:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.246+0000 7f9d7c776740 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560425a251c0 0x560425eb9bd0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.246+0000 7f9d7c776740 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560425a911b0 0x560425e418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.246+0000 7f9d7c776740 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560425a704d0 0x560425e413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.247+0000 7f9d58c00640 0 client.17082 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.246+0000 7f9d7c776740 1 --2- 172.21.15.63:0/1336076164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560425a682f0 0x560425e40e70 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.247+0000 7f9d7c776740 1 -- 172.21.15.63:0/1336076164 >> 172.21.15.63:0/1336076164 conn(0x560425a85300 msgr2=0x560425e3de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.247+0000 7f9d7c776740 1 -- 172.21.15.63:0/1336076164 shutdown_connections 2024-05-27T07:42:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.247+0000 7f9d7c776740 1 -- 172.21.15.63:0/1336076164 wait complete. 2024-05-27T07:42:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.247+0000 7f9d7c776740 20 client.17082 trim_cache size 0 max 16384 2024-05-27T07:42:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:42:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:42:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:42:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 -- 172.21.15.63:0/404123389 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d540b29d0 msgr2=0x7f9d540b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d540b29d0 0x7f9d540b4ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00bfd0 tx=0x7f9d5c0046c0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 -- 172.21.15.63:0/404123389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d701103c0 msgr2=0x7f9d701a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d701103c0 0x7f9d701a0f80 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9d68031040 tx=0x7f9d68032bc0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 -- 172.21.15.63:0/404123389 shutdown_connections 2024-05-27T07:42:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d540b29d0 0x7f9d540b4ee0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d701103c0 0x7f9d701a0f80 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d7010da60 0x7f9d7019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 --2- 172.21.15.63:0/404123389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d70102820 0x7f9d7019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 -- 172.21.15.63:0/404123389 >> 172.21.15.63:0/404123389 conn(0x7f9d700fe210 msgr2=0x7f9d70100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 -- 172.21.15.63:0/404123389 shutdown_connections 2024-05-27T07:42:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:12.248+0000 7f9d6fe00640 1 -- 172.21.15.63:0/404123389 wait complete. 2024-05-27T07:42:13.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:12 smithi086 ceph-mon[34687]: pgmap v267: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 7 op/s 2024-05-27T07:42:13.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:12 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:42:13.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:12 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:42:13.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:12 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:42:13.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:12 smithi149 ceph-mon[35408]: pgmap v267: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 7 op/s 2024-05-27T07:42:13.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:12 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:42:13.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:12 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:42:13.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:12 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:42:13.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:12 smithi063 ceph-mon[31613]: pgmap v267: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 7 op/s 2024-05-27T07:42:13.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:12 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:42:13.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:12 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:42:13.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:12 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:42:15.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:14 smithi086 ceph-mon[34687]: pgmap v268: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 7 op/s 2024-05-27T07:42:15.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:14 smithi149 ceph-mon[35408]: pgmap v268: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 7 op/s 2024-05-27T07:42:15.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:14 smithi063 ceph-mon[31613]: pgmap v268: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 7 op/s 2024-05-27T07:42:16.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:16 smithi086 ceph-mon[34687]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T07:42:16.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:16 smithi149 ceph-mon[35408]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T07:42:16.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:16 smithi063 ceph-mon[31613]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T07:42:17.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:17 smithi086 ceph-mon[34687]: pgmap v270: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-05-27T07:42:17.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:17 smithi149 ceph-mon[35408]: pgmap v270: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-05-27T07:42:17.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:17 smithi063 ceph-mon[31613]: pgmap v270: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-05-27T07:42:19.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:19 smithi086 ceph-mon[34687]: pgmap v271: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:42:19.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:19 smithi149 ceph-mon[35408]: pgmap v271: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:42:19.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:19 smithi063 ceph-mon[31613]: pgmap v271: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:42:21.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:21 smithi086 ceph-mon[34687]: pgmap v272: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:42:21.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:21 smithi149 ceph-mon[35408]: pgmap v272: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:42:21.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:21 smithi063 ceph-mon[31613]: pgmap v272: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:42:23.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:23 smithi086 ceph-mon[34687]: pgmap v273: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:42:23.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:23 smithi149 ceph-mon[35408]: pgmap v273: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:42:23.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:23 smithi063 ceph-mon[31613]: pgmap v273: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:42:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.493+0000 7f93e1600640 1 -- 172.21.15.63:0/4234981818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93dc10e020 msgr2=0x7f93dc110510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.493+0000 7f93e1600640 1 --2- 172.21.15.63:0/4234981818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93dc10e020 0x7f93dc110510 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009ca0 tx=0x7f93d402a9e0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.493+0000 7f93e1600640 1 -- 172.21.15.63:0/4234981818 shutdown_connections 2024-05-27T07:42:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.493+0000 7f93e1600640 1 --2- 172.21.15.63:0/4234981818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93dc10e020 0x7f93dc110510 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.493+0000 7f93e1600640 1 --2- 172.21.15.63:0/4234981818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc10b670 0x7f93dc10da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.493+0000 7f93e1600640 1 --2- 172.21.15.63:0/4234981818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc104c10 0x7f93dc105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.493+0000 7f93e1600640 1 -- 172.21.15.63:0/4234981818 >> 172.21.15.63:0/4234981818 conn(0x7f93dc0fe210 msgr2=0x7f93dc100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.493+0000 7f93e1600640 1 -- 172.21.15.63:0/4234981818 shutdown_connections 2024-05-27T07:42:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.493+0000 7f93e1600640 1 -- 172.21.15.63:0/4234981818 wait complete. 2024-05-27T07:42:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.495+0000 7f93e1600640 1 Processor -- start 2024-05-27T07:42:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.495+0000 7f93e1600640 1 -- start start 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.495+0000 7f93e1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc104c10 0x7f93dc19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.495+0000 7f93e1600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc10b670 0x7f93dc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.495+0000 7f93e1600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93dc10e020 0x7f93dc1a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.495+0000 7f93e1600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc113b80 con 0x7f93dc10e020 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.495+0000 7f93e1600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc113a00 con 0x7f93dc104c10 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.495+0000 7f93e1600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc113d00 con 0x7f93dc10b670 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93e0c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93dc10e020 0x7f93dc1a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93e0c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93dc10e020 0x7f93dc1a0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51112/0 (socket says 172.21.15.63:51112) 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93e0c00640 1 -- 172.21.15.63:0/1449803720 learned_addr learned my addr 172.21.15.63:0/1449803720 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93dbe00640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc104c10 0x7f93dc19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93db400640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc10b670 0x7f93dc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93e0c00640 1 -- 172.21.15.63:0/1449803720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc10b670 msgr2=0x7f93dc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93e0c00640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc10b670 0x7f93dc19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93e0c00640 1 -- 172.21.15.63:0/1449803720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc104c10 msgr2=0x7f93dc19c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93e0c00640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc104c10 0x7f93dc19c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93e0c00640 1 -- 172.21.15.63:0/1449803720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93dc1a15a0 con 0x7f93dc10e020 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.496+0000 7f93dbe00640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc104c10 0x7f93dc19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.497+0000 7f93e0c00640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93dc10e020 0x7f93dc1a0f60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f93d4004af0 tx=0x7f93d4003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.497+0000 7f93d8c00640 1 -- 172.21.15.63:0/1449803720 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4047070 con 0x7f93dc10e020 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.497+0000 7f93d8c00640 1 -- 172.21.15.63:0/1449803720 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93d402ee40 con 0x7f93dc10e020 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.497+0000 7f93d8c00640 1 -- 172.21.15.63:0/1449803720 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4033050 con 0x7f93dc10e020 2024-05-27T07:42:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.497+0000 7f93e1600640 1 -- 172.21.15.63:0/1449803720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93dc1a1830 con 0x7f93dc10e020 2024-05-27T07:42:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.497+0000 7f93e1600640 1 -- 172.21.15.63:0/1449803720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93dc1a1cd0 con 0x7f93dc10e020 2024-05-27T07:42:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.501+0000 7f93e9bee740 1 Processor -- start 2024-05-27T07:42:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.503+0000 7f93d8c00640 1 -- 172.21.15.63:0/1449803720 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f93d404a070 con 0x7f93dc10e020 2024-05-27T07:42:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.503+0000 7f93e9bee740 1 -- start start 2024-05-27T07:42:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.503+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909821b0 0x55e490d24a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.503+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d24fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.503+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909614d0 0x55e490d254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.503+0000 7f93e9bee740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e490c95b90 con 0x55e4909614d0 2024-05-27T07:42:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.503+0000 7f93e9bee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e490c85730 con 0x55e4909821b0 2024-05-27T07:42:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.503+0000 7f93e9bee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e490c89770 con 0x55e4909592f0 2024-05-27T07:42:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.504+0000 7f93db400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909614d0 0x55e490d254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.504+0000 7f93dbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d24fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.504+0000 7f93db400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909614d0 0x55e490d254e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51124/0 (socket says 172.21.15.63:51124) 2024-05-27T07:42:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.504+0000 7f93db400640 1 -- 172.21.15.63:0/618951085 learned_addr learned my addr 172.21.15.63:0/618951085 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.504+0000 7f93db400640 1 --2- 172.21.15.63:0/618951085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909821b0 0x55e490d24a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.505+0000 7f93d8c00640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93c00b2950 0x7f93c00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.505+0000 7f93db400640 1 -- 172.21.15.63:0/618951085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 msgr2=0x55e490d24fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.505+0000 7f93db400640 1 --2- 172.21.15.63:0/618951085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d24fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.505+0000 7f93db400640 1 -- 172.21.15.63:0/618951085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909821b0 msgr2=0x55e490d24a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.505+0000 7f93db400640 1 --2- 172.21.15.63:0/618951085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909821b0 0x55e490d24a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.505+0000 7f93db400640 1 -- 172.21.15.63:0/618951085 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e490c1e8b0 con 0x55e4909614d0 2024-05-27T07:42:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.505+0000 7f93d8c00640 1 -- 172.21.15.63:0/1449803720 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f93d4103780 con 0x7f93dc10e020 2024-05-27T07:42:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.505+0000 7f93e0c00640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93c00b2950 0x7f93c00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.505+0000 7f93db400640 1 --2- 172.21.15.63:0/618951085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909614d0 0x55e490d254e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f93c8014920 tx=0x7f93c800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0ecc6864e0ed1d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.505+0000 7f93d1200640 1 -- 172.21.15.63:0/618951085 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c80142d0 con 0x55e4909614d0 2024-05-27T07:42:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.506+0000 7f93d1200640 1 -- 172.21.15.63:0/618951085 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93c800fbf0 con 0x55e4909614d0 2024-05-27T07:42:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.506+0000 7f93d1200640 1 -- 172.21.15.63:0/618951085 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c80127d0 con 0x55e4909614d0 2024-05-27T07:42:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.507+0000 7f93e9bee740 1 -- 172.21.15.63:0/618951085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909614d0 msgr2=0x55e490d254e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.507+0000 7f93e0c00640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93c00b2950 0x7f93c00b4e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f93d40c6720 tx=0x7f93d403a040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.507+0000 7f93e9bee740 1 --2- 172.21.15.63:0/618951085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909614d0 0x55e490d254e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f93c8014920 tx=0x7f93c800c450 comp rx=0 tx=0).stop 2024-05-27T07:42:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.507+0000 7f93e9bee740 1 -- 172.21.15.63:0/618951085 shutdown_connections 2024-05-27T07:42:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.507+0000 7f93e9bee740 1 --2- 172.21.15.63:0/618951085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909614d0 0x55e490d254e0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.507+0000 7f93e9bee740 1 --2- 172.21.15.63:0/618951085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d24fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.507+0000 7f93e9bee740 1 --2- 172.21.15.63:0/618951085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909821b0 0x55e490d24a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.507+0000 7f93e9bee740 1 -- 172.21.15.63:0/618951085 >> 172.21.15.63:0/618951085 conn(0x55e490976300 msgr2=0x55e490d21210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.507+0000 7f93e9bee740 1 -- 172.21.15.63:0/618951085 shutdown_connections 2024-05-27T07:42:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.507+0000 7f93e9bee740 1 -- 172.21.15.63:0/618951085 wait complete. 2024-05-27T07:42:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.508+0000 7f93e9bee740 1 -- start start 2024-05-27T07:42:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.508+0000 7f93e9bee740 1 Processor -- start 2024-05-27T07:42:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909592f0 0x55e490d2eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d2f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909821b0 0x55e490d3aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93e9bee740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e49096b790 con 0x55e4909592f0 2024-05-27T07:42:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93e9bee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e490c2dbc0 con 0x55e4909614d0 2024-05-27T07:42:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93e9bee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e490d31b90 con 0x55e4909821b0 2024-05-27T07:42:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93e9bee740 10 client.0 _ll_register_callbacks cb 0x55e490c0b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93db400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909592f0 0x55e490d2eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93db400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909592f0 0x55e490d2eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51138/0 (socket says 172.21.15.63:51138) 2024-05-27T07:42:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93db400640 1 -- 172.21.15.63:0/534261886 learned_addr learned my addr 172.21.15.63:0/534261886 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.509+0000 7f93dbe00640 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d2f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.510+0000 7f93db400640 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909821b0 0x55e490d3aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.510+0000 7f93db400640 1 -- 172.21.15.63:0/534261886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909821b0 msgr2=0x55e490d3aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.510+0000 7f93db400640 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909821b0 0x55e490d3aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.510+0000 7f93db400640 1 -- 172.21.15.63:0/534261886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 msgr2=0x55e490d2f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.510+0000 7f93db400640 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d2f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.510+0000 7f93db400640 1 -- 172.21.15.63:0/534261886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e49099a210 con 0x55e4909592f0 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.510+0000 7f93db400640 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909592f0 0x55e490d2eea0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f93c80058c0 tx=0x7f93c8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a03cbabd653fb2f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.510+0000 7f93c5600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.511+0000 7f93c5600640 1 -- 172.21.15.63:0/534261886 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c800f960 con 0x55e4909592f0 2024-05-27T07:42:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.511+0000 7f93e9bee740 1 -- 172.21.15.63:0/534261886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c800fbf0 con 0x55e4909592f0 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.511+0000 7f93c5600640 1 -- 172.21.15.63:0/534261886 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f93c8004030 con 0x55e4909592f0 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.511+0000 7f93c5600640 1 -- 172.21.15.63:0/534261886 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f93c8034050 con 0x55e4909592f0 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.511+0000 7f93e9bee740 10 client.17094 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.511+0000 7f93e9bee740 1 -- 172.21.15.63:0/534261886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e490a66620 con 0x55e4909592f0 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.511+0000 7f93c5600640 1 -- 172.21.15.63:0/534261886 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93c800fbf0 con 0x55e4909592f0 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.512+0000 7f93c5600640 1 -- 172.21.15.63:0/534261886 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f93c802d060 con 0x55e4909592f0 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.512+0000 7f93e9bee740 10 client.17094 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.512+0000 7f93e9bee740 10 client.17094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.512+0000 7f93e9bee740 20 client.17094 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.512+0000 7f93e9bee740 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e490940d80 0x55e490d40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.512+0000 7f93e9bee740 4 client.17094 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:42:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.512+0000 7f93e9bee740 1 -- 172.21.15.63:0/534261886 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e490c97ae0 con 0x55e490940d80 2024-05-27T07:42:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.513+0000 7f93dbe00640 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e490940d80 0x55e490d40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.514+0000 7f93c5600640 10 client.17094 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.514+0000 7f93dbe00640 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e490940d80 0x55e490d40da0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c48ad939dd6b083 server_cookie=55f991bc8fb8f4c6 in_seq=0 out_seq=0 2024-05-27T07:42:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.515+0000 7f93c5600640 1 -- 172.21.15.63:0/534261886 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e490c97ae0 con 0x55e490940d80 2024-05-27T07:42:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.515+0000 7f93c5600640 10 client.17094 handle_command_reply: tid=0 2024-05-27T07:42:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 client.17094 shutdown 2024-05-27T07:42:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 -- 172.21.15.63:0/534261886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909592f0 msgr2=0x55e490d2eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909592f0 0x55e490d2eea0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f93c80058c0 tx=0x7f93c8005850 comp rx=0 tx=0).stop 2024-05-27T07:42:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 -- 172.21.15.63:0/534261886 shutdown_connections 2024-05-27T07:42:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e490940d80 0x55e490d40da0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909821b0 0x55e490d3aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d2f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 --2- 172.21.15.63:0/534261886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909592f0 0x55e490d2eea0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93c5600640 0 client.17094 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 -- 172.21.15.63:0/534261886 >> 172.21.15.63:0/534261886 conn(0x55e490976300 msgr2=0x55e490d26780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 -- 172.21.15.63:0/534261886 shutdown_connections 2024-05-27T07:42:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.516+0000 7f93e9bee740 1 -- 172.21.15.63:0/534261886 wait complete. 2024-05-27T07:42:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.517+0000 7f93e9bee740 20 client.17094 trim_cache size 0 max 16384 2024-05-27T07:42:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.539+0000 7f93e9bee740 1 Processor -- start 2024-05-27T07:42:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.539+0000 7f93e9bee740 1 -- start start 2024-05-27T07:42:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.539+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d24900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.539+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d3ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.540+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d3b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.540+0000 7f93e9bee740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1b5a00 con 0x55e4909821b0 2024-05-27T07:42:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.540+0000 7f93e9bee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1b32d0 con 0x55e4909614d0 2024-05-27T07:42:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.540+0000 7f93e9bee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1b1140 con 0x55e4909592f0 2024-05-27T07:42:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.540+0000 7f93db400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d3b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.540+0000 7f93db400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d3b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51144/0 (socket says 172.21.15.63:51144) 2024-05-27T07:42:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.540+0000 7f93db400640 1 -- 172.21.15.63:0/1566652683 learned_addr learned my addr 172.21.15.63:0/1566652683 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.540+0000 7f93dbe00640 1 --2- 172.21.15.63:0/1566652683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d3ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.540+0000 7f93db400640 1 --2- 172.21.15.63:0/1566652683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d24900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.541+0000 7f93db400640 1 -- 172.21.15.63:0/1566652683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 msgr2=0x55e490d24900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.541+0000 7f93db400640 1 --2- 172.21.15.63:0/1566652683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d24900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.541+0000 7f93db400640 1 -- 172.21.15.63:0/1566652683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 msgr2=0x55e490d3ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.541+0000 7f93db400640 1 --2- 172.21.15.63:0/1566652683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d3ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.541+0000 7f93db400640 1 -- 172.21.15.63:0/1566652683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e490d30150 con 0x55e4909821b0 2024-05-27T07:42:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.541+0000 7f93db400640 1 --2- 172.21.15.63:0/1566652683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d3b170 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x55e490c85880 tx=0x7f93c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bc5f57773eb55c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.541+0000 7f93c7e00640 1 -- 172.21.15.63:0/1566652683 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8004a30 con 0x55e4909821b0 2024-05-27T07:42:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.541+0000 7f93c7e00640 1 -- 172.21.15.63:0/1566652683 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93c800c910 con 0x55e4909821b0 2024-05-27T07:42:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.541+0000 7f93c7e00640 1 -- 172.21.15.63:0/1566652683 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c80051e0 con 0x55e4909821b0 2024-05-27T07:42:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.542+0000 7f93e9bee740 1 -- 172.21.15.63:0/1566652683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 msgr2=0x55e490d3b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.542+0000 7f93e9bee740 1 --2- 172.21.15.63:0/1566652683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d3b170 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x55e490c85880 tx=0x7f93c800cb70 comp rx=0 tx=0).stop 2024-05-27T07:42:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.542+0000 7f93e9bee740 1 -- 172.21.15.63:0/1566652683 shutdown_connections 2024-05-27T07:42:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.542+0000 7f93e9bee740 1 --2- 172.21.15.63:0/1566652683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d3b170 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.542+0000 7f93e9bee740 1 --2- 172.21.15.63:0/1566652683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d3ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.542+0000 7f93e9bee740 1 --2- 172.21.15.63:0/1566652683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d24900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.542+0000 7f93e9bee740 1 -- 172.21.15.63:0/1566652683 >> 172.21.15.63:0/1566652683 conn(0x55e490976300 msgr2=0x55e490d30de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.542+0000 7f93e9bee740 1 -- 172.21.15.63:0/1566652683 shutdown_connections 2024-05-27T07:42:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.543+0000 7f93e9bee740 1 -- 172.21.15.63:0/1566652683 wait complete. 2024-05-27T07:42:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.543+0000 7f93e9bee740 1 -- start start 2024-05-27T07:42:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.543+0000 7f93e9bee740 1 Processor -- start 2024-05-27T07:42:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.544+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d31de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.544+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d32320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.544+0000 7f93e9bee740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d32860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.544+0000 7f93e9bee740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1b1790 con 0x55e4909821b0 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.544+0000 7f93e9bee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1b2c00 con 0x55e4909614d0 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.544+0000 7f93e9bee740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1b1e60 con 0x55e4909592f0 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.544+0000 7f93e9bee740 10 client.0 _ll_register_callbacks cb 0x7f93c800ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.544+0000 7f93db400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d32860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.545+0000 7f93db400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d32860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51156/0 (socket says 172.21.15.63:51156) 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.545+0000 7f93db400640 1 -- 172.21.15.63:0/3757828028 learned_addr learned my addr 172.21.15.63:0/3757828028 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.545+0000 7f93dbe00640 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d32320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.545+0000 7f93db400640 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d31de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.545+0000 7f93db400640 1 -- 172.21.15.63:0/3757828028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 msgr2=0x55e490d31de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.545+0000 7f93db400640 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d31de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.545+0000 7f93db400640 1 -- 172.21.15.63:0/3757828028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 msgr2=0x55e490d32320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.545+0000 7f93db400640 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d32320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.545+0000 7f93db400640 1 -- 172.21.15.63:0/3757828028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e490da72c0 con 0x55e4909821b0 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.545+0000 7f93db400640 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d32860 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f93c80046e0 tx=0x7f93c800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5738ed6358226e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.546+0000 7f93c5600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.546+0000 7f93c5600640 1 -- 172.21.15.63:0/3757828028 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8004a40 con 0x55e4909821b0 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.546+0000 7f93e9bee740 1 -- 172.21.15.63:0/3757828028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c800c910 con 0x55e4909821b0 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.546+0000 7f93c5600640 1 -- 172.21.15.63:0/3757828028 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f93c800f730 con 0x55e4909821b0 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.546+0000 7f93c5600640 1 -- 172.21.15.63:0/3757828028 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f93c800f930 con 0x55e4909821b0 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.546+0000 7f93e9bee740 10 client.17100 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.546+0000 7f93e9bee740 1 -- 172.21.15.63:0/3757828028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e490da8490 con 0x55e4909821b0 2024-05-27T07:42:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.547+0000 7f93c5600640 1 -- 172.21.15.63:0/3757828028 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93c800c910 con 0x55e4909821b0 2024-05-27T07:42:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.547+0000 7f93c5600640 1 -- 172.21.15.63:0/3757828028 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f93c8037040 con 0x55e4909821b0 2024-05-27T07:42:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.547+0000 7f93e9bee740 10 client.17100 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.547+0000 7f93e9bee740 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.547+0000 7f93e9bee740 20 client.17100 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.547+0000 7f93e9bee740 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4909161c0 0x55e490daab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.547+0000 7f93e9bee740 4 client.17100 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:42:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.547+0000 7f93e9bee740 1 -- 172.21.15.63:0/3757828028 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e490c912c0 con 0x55e4909161c0 2024-05-27T07:42:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.548+0000 7f93dbe00640 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4909161c0 0x55e490daab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.549+0000 7f93dbe00640 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4909161c0 0x55e490daab70 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c8462249cb45d server_cookie=2e9c94b01959bdfc in_seq=0 out_seq=0 2024-05-27T07:42:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.549+0000 7f93c5600640 10 client.17100 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.579+0000 7f93c5600640 1 -- 172.21.15.63:0/3757828028 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e490c912c0 con 0x55e4909161c0 2024-05-27T07:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.579+0000 7f93c5600640 10 client.17100 handle_command_reply: tid=0 2024-05-27T07:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.579+0000 7f93e9bee740 1 client.17100 shutdown 2024-05-27T07:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 1 -- 172.21.15.63:0/3757828028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 msgr2=0x55e490d32860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d32860 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f93c80046e0 tx=0x7f93c800cd20 comp rx=0 tx=0).stop 2024-05-27T07:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 1 -- 172.21.15.63:0/3757828028 shutdown_connections 2024-05-27T07:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4909161c0 0x55e490daab70 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4909821b0 0x55e490d32860 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4909614d0 0x55e490d32320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93c5600640 0 client.17100 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 1 --2- 172.21.15.63:0/3757828028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4909592f0 0x55e490d31de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 1 -- 172.21.15.63:0/3757828028 >> 172.21.15.63:0/3757828028 conn(0x55e490976300 msgr2=0x55e490d2e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 1 -- 172.21.15.63:0/3757828028 shutdown_connections 2024-05-27T07:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 1 -- 172.21.15.63:0/3757828028 wait complete. 2024-05-27T07:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.580+0000 7f93e9bee740 20 client.17100 trim_cache size 0 max 16384 2024-05-27T07:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 -- 172.21.15.63:0/1449803720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93c00b2950 msgr2=0x7f93c00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93c00b2950 0x7f93c00b4e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f93d40c6720 tx=0x7f93d403a040 comp rx=0 tx=0).stop 2024-05-27T07:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 -- 172.21.15.63:0/1449803720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93dc10e020 msgr2=0x7f93dc1a0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93dc10e020 0x7f93dc1a0f60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f93d4004af0 tx=0x7f93d4003f30 comp rx=0 tx=0).stop 2024-05-27T07:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 -- 172.21.15.63:0/1449803720 shutdown_connections 2024-05-27T07:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93c00b2950 0x7f93c00b4e60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93dc10e020 0x7f93dc1a0f60 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc10b670 0x7f93dc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 --2- 172.21.15.63:0/1449803720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93dc104c10 0x7f93dc19c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 -- 172.21.15.63:0/1449803720 >> 172.21.15.63:0/1449803720 conn(0x7f93dc0fe210 msgr2=0x7f93dc109cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.581+0000 7f93e1600640 1 -- 172.21.15.63:0/1449803720 shutdown_connections 2024-05-27T07:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:25.582+0000 7f93e1600640 1 -- 172.21.15.63:0/1449803720 wait complete. 2024-05-27T07:42:25.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:25 smithi086 ceph-mon[34687]: pgmap v274: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:42:25.710 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:25 smithi149 ceph-mon[35408]: pgmap v274: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:42:25.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:25 smithi063 ceph-mon[31613]: pgmap v274: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:42:27.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:27 smithi086 ceph-mon[34687]: pgmap v275: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:42:27.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:27 smithi149 ceph-mon[35408]: pgmap v275: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:42:27.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:27 smithi063 ceph-mon[31613]: pgmap v275: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:42:28.653 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:42:29.166 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:29.621 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:29 smithi063 ceph-mon[31613]: pgmap v276: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T07:42:29.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:29 smithi086 ceph-mon[34687]: pgmap v276: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T07:42:29.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:29 smithi149 ceph-mon[35408]: pgmap v276: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T07:42:31.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:31 smithi086 ceph-mon[34687]: pgmap v277: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:42:32.074 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:31 smithi063 ceph-mon[31613]: pgmap v277: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:42:32.075 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:31 smithi149 ceph-mon[35408]: pgmap v277: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:42:32.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.781+0000 7f5779600700 1 -- 172.21.15.63:0/1637649164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5774105bd0 msgr2=0x7f577410e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:32.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.781+0000 7f5779600700 1 --2- 172.21.15.63:0/1637649164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5774105bd0 0x7f577410e120 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5760009990 tx=0x7f5760009ca0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.781+0000 7f5779600700 1 -- 172.21.15.63:0/1637649164 shutdown_connections 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.781+0000 7f5779600700 1 --2- 172.21.15.63:0/1637649164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577410e6f0 0x7f5774110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.781+0000 7f5779600700 1 --2- 172.21.15.63:0/1637649164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5774105280 0x7f5774105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.781+0000 7f5779600700 1 --2- 172.21.15.63:0/1637649164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5774105bd0 0x7f577410e120 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.781+0000 7f5779600700 1 -- 172.21.15.63:0/1637649164 >> 172.21.15.63:0/1637649164 conn(0x7f5774078ab0 msgr2=0x7f5774078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.781+0000 7f5779600700 1 -- 172.21.15.63:0/1637649164 shutdown_connections 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.781+0000 7f5779600700 1 -- 172.21.15.63:0/1637649164 wait complete. 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.782+0000 7f5779600700 1 Processor -- start 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.783+0000 7f5779600700 1 -- start start 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.783+0000 7f5779600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5774105280 0x7f577419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.783+0000 7f5779600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5774105bd0 0x7f577419d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.783+0000 7f5779600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577410e6f0 0x7f57741a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.783+0000 7f5779600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5774114150 con 0x7f5774105280 2024-05-27T07:42:32.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.783+0000 7f5779600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5774113fd0 con 0x7f577410e6f0 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.783+0000 7f5779600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57741142d0 con 0x7f5774105bd0 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.784+0000 7f5773e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5774105280 0x7f577419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.784+0000 7f5773e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5774105280 0x7f577419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53618/0 (socket says 172.21.15.63:53618) 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.784+0000 7f5773e00700 1 -- 172.21.15.63:0/1493437807 learned_addr learned my addr 172.21.15.63:0/1493437807 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.784+0000 7f5778c00700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577410e6f0 0x7f57741a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.784+0000 7f5773400700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5774105bd0 0x7f577419d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.784+0000 7f5773e00700 1 -- 172.21.15.63:0/1493437807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5774105bd0 msgr2=0x7f577419d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.784+0000 7f5773e00700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5774105bd0 0x7f577419d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.784+0000 7f5773e00700 1 -- 172.21.15.63:0/1493437807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577410e6f0 msgr2=0x7f57741a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.784+0000 7f5773e00700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577410e6f0 0x7f57741a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.784+0000 7f5773e00700 1 -- 172.21.15.63:0/1493437807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57741a1c70 con 0x7f5774105280 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.785+0000 7f5773e00700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5774105280 0x7f577419cda0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f576400d950 tx=0x7f576400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.785+0000 7f5770c00700 1 -- 172.21.15.63:0/1493437807 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764016070 con 0x7f5774105280 2024-05-27T07:42:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.785+0000 7f5770c00700 1 -- 172.21.15.63:0/1493437807 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f576400fd20 con 0x7f5774105280 2024-05-27T07:42:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.785+0000 7f5779600700 1 -- 172.21.15.63:0/1493437807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57741a1f60 con 0x7f5774105280 2024-05-27T07:42:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.785+0000 7f5770c00700 1 -- 172.21.15.63:0/1493437807 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576400e3b0 con 0x7f5774105280 2024-05-27T07:42:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.785+0000 7f5779600700 1 -- 172.21.15.63:0/1493437807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57741a9880 con 0x7f5774105280 2024-05-27T07:42:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.790+0000 7f5770c00700 1 -- 172.21.15.63:0/1493437807 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5764019070 con 0x7f5774105280 2024-05-27T07:42:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.791+0000 7f57896c4b80 1 Processor -- start 2024-05-27T07:42:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.791+0000 7f57896c4b80 1 -- start start 2024-05-27T07:42:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.792+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 0x562612400ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.792+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.792+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x56261240e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.792+0000 7f57896c4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56261232fe40 con 0x562612401820 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.792+0000 7f57896c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56261231f9e0 con 0x562612282e40 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.792+0000 7f57896c4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562612323a20 con 0x562612401410 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.792+0000 7f5773400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x56261240e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.792+0000 7f5773400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x56261240e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53624/0 (socket says 172.21.15.63:53624) 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.792+0000 7f5773400700 1 -- 172.21.15.63:0/65961524 learned_addr learned my addr 172.21.15.63:0/65961524 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.793+0000 7f5773e00700 1 --2- 172.21.15.63:0/65961524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.793+0000 7f5778c00700 1 --2- 172.21.15.63:0/65961524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 0x562612400ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.793+0000 7f5770c00700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57580b29f0 0x7f57580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.793+0000 7f5773400700 1 -- 172.21.15.63:0/65961524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 msgr2=0x56261240bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:32.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.793+0000 7f5773400700 1 --2- 172.21.15.63:0/65961524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.793+0000 7f5773400700 1 -- 172.21.15.63:0/65961524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 msgr2=0x562612400ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.793+0000 7f5773400700 1 --2- 172.21.15.63:0/65961524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 0x562612400ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.793+0000 7f5773400700 1 -- 172.21.15.63:0/65961524 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626122e8420 con 0x562612401820 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.793+0000 7f5778c00700 1 --2- 172.21.15.63:0/65961524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 0x562612400ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.793+0000 7f5778c00700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57580b29f0 0x7f57580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.794+0000 7f5770c00700 1 -- 172.21.15.63:0/1493437807 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f57640e2500 con 0x7f5774105280 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.794+0000 7f5773400700 1 --2- 172.21.15.63:0/65961524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x56261240e020 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5760015000 tx=0x7f5760005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fb16916d2247ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.794+0000 7f5778c00700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57580b29f0 0x7f57580b4f00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5626123fd690 tx=0x7f576c004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.795+0000 7f5769600700 1 -- 172.21.15.63:0/65961524 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760005390 con 0x562612401820 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.795+0000 7f5769600700 1 -- 172.21.15.63:0/65961524 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5760005530 con 0x562612401820 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.795+0000 7f5769600700 1 -- 172.21.15.63:0/65961524 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576000bca0 con 0x562612401820 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.796+0000 7f57896c4b80 1 -- 172.21.15.63:0/65961524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 msgr2=0x56261240e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:32.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.796+0000 7f57896c4b80 1 --2- 172.21.15.63:0/65961524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x56261240e020 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5760015000 tx=0x7f5760005f00 comp rx=0 tx=0).stop 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.796+0000 7f57896c4b80 1 -- 172.21.15.63:0/65961524 shutdown_connections 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.796+0000 7f57896c4b80 1 --2- 172.21.15.63:0/65961524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x56261240e020 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.796+0000 7f57896c4b80 1 --2- 172.21.15.63:0/65961524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 0x562612400ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.796+0000 7f57896c4b80 1 --2- 172.21.15.63:0/65961524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.796+0000 7f57896c4b80 1 -- 172.21.15.63:0/65961524 >> 172.21.15.63:0/65961524 conn(0x5626122e2fc0 msgr2=0x5626123fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.796+0000 7f57896c4b80 1 -- 172.21.15.63:0/65961524 shutdown_connections 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.796+0000 7f57896c4b80 1 -- 172.21.15.63:0/65961524 wait complete. 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.797+0000 7f57896c4b80 1 -- start start 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.797+0000 7f57896c4b80 1 Processor -- start 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.798+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x56261240b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.798+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.798+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 0x56261240c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.798+0000 7f57896c4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562612225fa0 con 0x562612282e40 2024-05-27T07:42:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.798+0000 7f57896c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5626122c0760 con 0x562612401820 2024-05-27T07:42:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.798+0000 7f57896c4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56261240e8d0 con 0x562612401410 2024-05-27T07:42:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.798+0000 7f57896c4b80 10 client.0 _ll_register_callbacks cb 0x5626122c9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.799+0000 7f5773e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x56261240b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.799+0000 7f5773e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x56261240b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53636/0 (socket says 172.21.15.63:53636) 2024-05-27T07:42:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.799+0000 7f5773e00700 1 -- 172.21.15.63:0/3413004619 learned_addr learned my addr 172.21.15.63:0/3413004619 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.799+0000 7f5773400700 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.799+0000 7f5778c00700 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 0x56261240c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.799+0000 7f5773e00700 1 -- 172.21.15.63:0/3413004619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 msgr2=0x56261240bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.799+0000 7f5773e00700 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.799+0000 7f5773e00700 1 -- 172.21.15.63:0/3413004619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 msgr2=0x56261240c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.799+0000 7f5773e00700 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 0x56261240c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.799+0000 7f5773e00700 1 -- 172.21.15.63:0/3413004619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261226f8d0 con 0x562612282e40 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.800+0000 7f5773e00700 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x56261240b5a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f576400b2f0 tx=0x7f57640e5c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8d5aa354403c8e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.800+0000 7f575e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.800+0000 7f575e000700 1 -- 172.21.15.63:0/3413004619 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764020070 con 0x562612282e40 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.801+0000 7f575e000700 1 -- 172.21.15.63:0/3413004619 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f57640e2160 con 0x562612282e40 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.801+0000 7f57896c4b80 1 -- 172.21.15.63:0/3413004619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5760005530 con 0x562612282e40 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.801+0000 7f575e000700 1 -- 172.21.15.63:0/3413004619 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f57640a8b90 con 0x562612282e40 2024-05-27T07:42:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.801+0000 7f57896c4b80 10 client.17112 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.801+0000 7f57896c4b80 1 -- 172.21.15.63:0/3413004619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5626122f0060 con 0x562612282e40 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.802+0000 7f575e000700 1 -- 172.21.15.63:0/3413004619 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5760005530 con 0x562612282e40 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.802+0000 7f575e000700 1 -- 172.21.15.63:0/3413004619 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f57640a9680 con 0x562612282e40 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.802+0000 7f57896c4b80 10 client.17112 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.802+0000 7f57896c4b80 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.802+0000 7f57896c4b80 20 client.17112 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.802+0000 7f57896c4b80 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56261241b860 0x56261241dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.802+0000 7f57896c4b80 4 client.17112 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.802+0000 7f57896c4b80 1 -- 172.21.15.63:0/3413004619 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562612331d90 con 0x56261241b860 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.803+0000 7f5773400700 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56261241b860 0x56261241dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.804+0000 7f5773400700 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56261241b860 0x56261241dc80 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b13797d6af9e6c server_cookie=6c8452a11489fd18 in_seq=0 out_seq=0 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.804+0000 7f575e000700 10 client.17112 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:32.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.806+0000 7f575e000700 1 -- 172.21.15.63:0/3413004619 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562612331d90 con 0x56261241b860 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.806+0000 7f575e000700 10 client.17112 handle_command_reply: tid=0 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.806+0000 7f57896c4b80 1 client.17112 shutdown 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.806+0000 7f57896c4b80 1 -- 172.21.15.63:0/3413004619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 msgr2=0x56261240b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.806+0000 7f57896c4b80 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x56261240b5a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f576400b2f0 tx=0x7f57640e5c40 comp rx=0 tx=0).stop 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.807+0000 7f57896c4b80 1 -- 172.21.15.63:0/3413004619 shutdown_connections 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.807+0000 7f57896c4b80 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56261241b860 0x56261241dc80 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.807+0000 7f57896c4b80 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 0x56261240c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.807+0000 7f57896c4b80 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x56261240b5a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.807+0000 7f57896c4b80 1 --2- 172.21.15.63:0/3413004619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.807+0000 7f57896c4b80 1 -- 172.21.15.63:0/3413004619 >> 172.21.15.63:0/3413004619 conn(0x5626122e2fc0 msgr2=0x5626123fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.807+0000 7f575e000700 0 client.17112 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.807+0000 7f57896c4b80 1 -- 172.21.15.63:0/3413004619 shutdown_connections 2024-05-27T07:42:32.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.807+0000 7f57896c4b80 1 -- 172.21.15.63:0/3413004619 wait complete. 2024-05-27T07:42:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.807+0000 7f57896c4b80 20 client.17112 trim_cache size 0 max 16384 2024-05-27T07:42:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.834+0000 7f57896c4b80 1 Processor -- start 2024-05-27T07:42:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.834+0000 7f57896c4b80 1 -- start start 2024-05-27T07:42:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.834+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x562612406dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.834+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.834+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 0x562612417970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.834+0000 7f57896c4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b4d90 con 0x562612282e40 2024-05-27T07:42:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.834+0000 7f57896c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b2620 con 0x562612401820 2024-05-27T07:42:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.834+0000 7f57896c4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b0490 con 0x562612401410 2024-05-27T07:42:32.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.835+0000 7f5773e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x562612406dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.835+0000 7f5773e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x562612406dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53652/0 (socket says 172.21.15.63:53652) 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.835+0000 7f5773e00700 1 -- 172.21.15.63:0/3276664173 learned_addr learned my addr 172.21.15.63:0/3276664173 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.835+0000 7f5773400700 1 --2- 172.21.15.63:0/3276664173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.835+0000 7f5778c00700 1 --2- 172.21.15.63:0/3276664173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 0x562612417970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.836+0000 7f5773e00700 1 -- 172.21.15.63:0/3276664173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 msgr2=0x56261240bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.836+0000 7f5773e00700 1 --2- 172.21.15.63:0/3276664173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.836+0000 7f5773e00700 1 -- 172.21.15.63:0/3276664173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 msgr2=0x562612417970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.836+0000 7f5773e00700 1 --2- 172.21.15.63:0/3276664173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 0x562612417970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.836+0000 7f5773e00700 1 -- 172.21.15.63:0/3276664173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626123fe110 con 0x562612282e40 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.836+0000 7f5778c00700 1 --2- 172.21.15.63:0/3276664173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 0x562612417970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.836+0000 7f5773e00700 1 --2- 172.21.15.63:0/3276664173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x562612406dd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5764010a20 tx=0x7f57640b23c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d183c6f4fefecd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:32.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.836+0000 7f5768c00700 1 -- 172.21.15.63:0/3276664173 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764020070 con 0x562612282e40 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.837+0000 7f5768c00700 1 -- 172.21.15.63:0/3276664173 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57640a9b20 con 0x562612282e40 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.837+0000 7f5768c00700 1 -- 172.21.15.63:0/3276664173 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57640e2360 con 0x562612282e40 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.838+0000 7f57896c4b80 1 -- 172.21.15.63:0/3276664173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 msgr2=0x562612406dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.838+0000 7f57896c4b80 1 --2- 172.21.15.63:0/3276664173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x562612406dd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5764010a20 tx=0x7f57640b23c0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.838+0000 7f57896c4b80 1 -- 172.21.15.63:0/3276664173 shutdown_connections 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.838+0000 7f57896c4b80 1 --2- 172.21.15.63:0/3276664173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612401820 0x562612417970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.838+0000 7f57896c4b80 1 --2- 172.21.15.63:0/3276664173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612282e40 0x562612406dd0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.838+0000 7f57896c4b80 1 --2- 172.21.15.63:0/3276664173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.838+0000 7f57896c4b80 1 -- 172.21.15.63:0/3276664173 >> 172.21.15.63:0/3276664173 conn(0x5626122e2fc0 msgr2=0x562612404900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.838+0000 7f57896c4b80 1 -- 172.21.15.63:0/3276664173 shutdown_connections 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.838+0000 7f57896c4b80 1 -- 172.21.15.63:0/3276664173 wait complete. 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.839+0000 7f57896c4b80 1 -- start start 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.839+0000 7f57896c4b80 1 Processor -- start 2024-05-27T07:42:32.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 0x56261240f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f57896c4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x5626124103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f57896c4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b0ae0 con 0x562612401820 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f57896c4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b1f50 con 0x562612282e40 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f57896c4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b11b0 con 0x562612401410 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f57896c4b80 10 client.0 _ll_register_callbacks cb 0x5626122c9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f5778c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x5626124103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f5778c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x5626124103e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53662/0 (socket says 172.21.15.63:53662) 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f5778c00700 1 -- 172.21.15.63:0/668094293 learned_addr learned my addr 172.21.15.63:0/668094293 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f5773400700 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.840+0000 7f5773e00700 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 0x56261240f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.841+0000 7f5778c00700 1 -- 172.21.15.63:0/668094293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 msgr2=0x56261240fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:32.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.841+0000 7f5778c00700 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.841+0000 7f5778c00700 1 -- 172.21.15.63:0/668094293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 msgr2=0x56261240f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.841+0000 7f5778c00700 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 0x56261240f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.841+0000 7f5778c00700 1 -- 172.21.15.63:0/668094293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626123fe350 con 0x562612401820 2024-05-27T07:42:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.842+0000 7f5778c00700 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x5626124103e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f576c008b70 tx=0x7f576c009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc7a6345bc64b38 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.842+0000 7f575e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.842+0000 7f575e000700 1 -- 172.21.15.63:0/668094293 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c029020 con 0x562612401820 2024-05-27T07:42:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.842+0000 7f575e000700 1 -- 172.21.15.63:0/668094293 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f576c017cf0 con 0x562612401820 2024-05-27T07:42:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.842+0000 7f57896c4b80 1 -- 172.21.15.63:0/668094293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57640a9b20 con 0x562612401820 2024-05-27T07:42:32.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.843+0000 7f575e000700 1 -- 172.21.15.63:0/668094293 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f576c034050 con 0x562612401820 2024-05-27T07:42:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.843+0000 7f57896c4b80 10 client.17118 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.843+0000 7f57896c4b80 1 -- 172.21.15.63:0/668094293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562612419330 con 0x562612401820 2024-05-27T07:42:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.843+0000 7f575e000700 1 -- 172.21.15.63:0/668094293 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57640a9b20 con 0x562612401820 2024-05-27T07:42:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.843+0000 7f575e000700 1 -- 172.21.15.63:0/668094293 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f576c025cb0 con 0x562612401820 2024-05-27T07:42:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.843+0000 7f57896c4b80 10 client.17118 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.843+0000 7f57896c4b80 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.843+0000 7f57896c4b80 20 client.17118 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.843+0000 7f57896c4b80 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562612486940 0x562612488d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.844+0000 7f57896c4b80 4 client.17118 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.844+0000 7f57896c4b80 1 -- 172.21.15.63:0/668094293 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56261232b570 con 0x562612486940 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.844+0000 7f5773e00700 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562612486940 0x562612488d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.845+0000 7f5773e00700 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562612486940 0x562612488d60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77329261558a047 server_cookie=eb60c9a2112bf267 in_seq=0 out_seq=0 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.845+0000 7f575e000700 10 client.17118 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.846+0000 7f575e000700 1 -- 172.21.15.63:0/668094293 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56261232b570 con 0x562612486940 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.846+0000 7f575e000700 10 client.17118 handle_command_reply: tid=0 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.846+0000 7f57896c4b80 1 client.17118 shutdown 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.846+0000 7f57896c4b80 1 -- 172.21.15.63:0/668094293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 msgr2=0x5626124103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.846+0000 7f57896c4b80 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x5626124103e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f576c008b70 tx=0x7f576c009100 comp rx=0 tx=0).stop 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.847+0000 7f57896c4b80 1 -- 172.21.15.63:0/668094293 shutdown_connections 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.847+0000 7f57896c4b80 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562612486940 0x562612488d60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.847+0000 7f57896c4b80 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562612401820 0x5626124103e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.847+0000 7f57896c4b80 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562612282e40 0x56261240f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.847+0000 7f57896c4b80 1 --2- 172.21.15.63:0/668094293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562612401410 0x56261240fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.847+0000 7f57896c4b80 1 -- 172.21.15.63:0/668094293 >> 172.21.15.63:0/668094293 conn(0x5626122e2fc0 msgr2=0x56261240bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:32.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.847+0000 7f575e000700 0 client.17118 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:32.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.847+0000 7f57896c4b80 1 -- 172.21.15.63:0/668094293 shutdown_connections 2024-05-27T07:42:32.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.847+0000 7f57896c4b80 1 -- 172.21.15.63:0/668094293 wait complete. 2024-05-27T07:42:32.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.847+0000 7f57896c4b80 20 client.17118 trim_cache size 0 max 16384 2024-05-27T07:42:32.101 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:42:32.102 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:42:32.102 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:42:32.102 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:42:32.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 -- 172.21.15.63:0/1493437807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57580b29f0 msgr2=0x7f57580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:32.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57580b29f0 0x7f57580b4f00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5626123fd690 tx=0x7f576c004ad0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 -- 172.21.15.63:0/1493437807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5774105280 msgr2=0x7f577419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:32.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5774105280 0x7f577419cda0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f576400d950 tx=0x7f576400dc60 comp rx=0 tx=0).stop 2024-05-27T07:42:32.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 -- 172.21.15.63:0/1493437807 shutdown_connections 2024-05-27T07:42:32.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57580b29f0 0x7f57580b4f00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577410e6f0 0x7f57741a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5774105280 0x7f577419cda0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 --2- 172.21.15.63:0/1493437807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5774105bd0 0x7f577419d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:32.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 -- 172.21.15.63:0/1493437807 >> 172.21.15.63:0/1493437807 conn(0x7f5774078ab0 msgr2=0x7f57740ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:32.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 -- 172.21.15.63:0/1493437807 shutdown_connections 2024-05-27T07:42:32.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:31.849+0000 7f5779600700 1 -- 172.21.15.63:0/1493437807 wait complete. 2024-05-27T07:42:32.790 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:42:32.791 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:42:33.318 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:33.695 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:33 smithi063 ceph-mon[31613]: pgmap v278: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:42:33.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:33 smithi086 ceph-mon[34687]: pgmap v278: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:42:33.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:33 smithi149 ceph-mon[35408]: pgmap v278: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:42:35.570 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:35 smithi063 ceph-mon[31613]: pgmap v279: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:42:35.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:35 smithi086 ceph-mon[34687]: pgmap v279: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:42:35.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:35 smithi149 ceph-mon[35408]: pgmap v279: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:42:35.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.832+0000 7f4f35400700 1 -- 172.21.15.63:0/3202682966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f3010e720 msgr2=0x7f4f30110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:35.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.832+0000 7f4f35400700 1 --2- 172.21.15.63:0/3202682966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f3010e720 0x7f4f30110c00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800b0a0 tx=0x7f4f2800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:42:35.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.832+0000 7f4f35400700 1 -- 172.21.15.63:0/3202682966 shutdown_connections 2024-05-27T07:42:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.832+0000 7f4f35400700 1 --2- 172.21.15.63:0/3202682966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f3010e720 0x7f4f30110c00 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.832+0000 7f4f35400700 1 --2- 172.21.15.63:0/3202682966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f30105340 0x7f4f30105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.832+0000 7f4f35400700 1 --2- 172.21.15.63:0/3202682966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f30105c90 0x7f4f3010e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.833+0000 7f4f35400700 1 -- 172.21.15.63:0/3202682966 >> 172.21.15.63:0/3202682966 conn(0x7f4f30078ab0 msgr2=0x7f4f30078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.833+0000 7f4f35400700 1 -- 172.21.15.63:0/3202682966 shutdown_connections 2024-05-27T07:42:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.833+0000 7f4f35400700 1 -- 172.21.15.63:0/3202682966 wait complete. 2024-05-27T07:42:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.834+0000 7f4f35400700 1 Processor -- start 2024-05-27T07:42:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.834+0000 7f4f35400700 1 -- start start 2024-05-27T07:42:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.835+0000 7f4f35400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f30105340 0x7f4f301a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.835+0000 7f4f35400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f30105c90 0x7f4f301a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.835+0000 7f4f35400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f3010e720 0x7f4f3019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.835+0000 7f4f35400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30114100 con 0x7f4f30105c90 2024-05-27T07:42:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.835+0000 7f4f35400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30113f80 con 0x7f4f3010e720 2024-05-27T07:42:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.835+0000 7f4f35400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30114280 con 0x7f4f30105340 2024-05-27T07:42:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f2fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f30105340 0x7f4f301a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f2f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f30105c90 0x7f4f301a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f2f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f30105c90 0x7f4f301a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53668/0 (socket says 172.21.15.63:53668) 2024-05-27T07:42:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f2fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f30105340 0x7f4f301a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:56350/0 (socket says 172.21.15.63:56350) 2024-05-27T07:42:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f34a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f3010e720 0x7f4f3019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f2f400700 1 -- 172.21.15.63:0/4035857748 learned_addr learned my addr 172.21.15.63:0/4035857748 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f2f400700 1 -- 172.21.15.63:0/4035857748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f30105340 msgr2=0x7f4f301a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f2f400700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f30105340 0x7f4f301a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f2f400700 1 -- 172.21.15.63:0/4035857748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f3010e720 msgr2=0x7f4f3019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f2f400700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f3010e720 0x7f4f3019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.836+0000 7f4f2f400700 1 -- 172.21.15.63:0/4035857748 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3019af40 con 0x7f4f30105c90 2024-05-27T07:42:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.837+0000 7f4f2f400700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f30105c90 0x7f4f301a0c20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00c9b0 tx=0x7f4f1c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:35.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.837+0000 7f4f2cc00700 1 -- 172.21.15.63:0/4035857748 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c007b00 con 0x7f4f30105c90 2024-05-27T07:42:35.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.837+0000 7f4f2cc00700 1 -- 172.21.15.63:0/4035857748 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f1c00ce50 con 0x7f4f30105c90 2024-05-27T07:42:35.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.837+0000 7f4f2cc00700 1 -- 172.21.15.63:0/4035857748 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c018660 con 0x7f4f30105c90 2024-05-27T07:42:35.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.838+0000 7f4f35400700 1 -- 172.21.15.63:0/4035857748 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3019b1d0 con 0x7f4f30105c90 2024-05-27T07:42:35.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.838+0000 7f4f35400700 1 -- 172.21.15.63:0/4035857748 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3019b6c0 con 0x7f4f30105c90 2024-05-27T07:42:35.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.840+0000 7f4f35400700 1 -- 172.21.15.63:0/4035857748 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f300512b0 con 0x7f4f30105c90 2024-05-27T07:42:35.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.843+0000 7f4f2cc00700 1 -- 172.21.15.63:0/4035857748 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4f1c00f450 con 0x7f4f30105c90 2024-05-27T07:42:35.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.845+0000 7f4f2cc00700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f140b2a70 0x7f4f140b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:35.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.845+0000 7f4f2cc00700 1 -- 172.21.15.63:0/4035857748 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f4f1c0e2860 con 0x7f4f30105c90 2024-05-27T07:42:35.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.845+0000 7f4f2fe00700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f140b2a70 0x7f4f140b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:35.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.846+0000 7f4f2fe00700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f140b2a70 0x7f4f140b4f80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f3019ba40 tx=0x7f4f2000b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:35.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:35.850+0000 7f4f2cc00700 1 -- 172.21.15.63:0/4035857748 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f1c014030 con 0x7f4f30105c90 2024-05-27T07:42:36.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.193+0000 7f4f35400700 1 -- 172.21.15.63:0/4035857748 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f3002e5e0 con 0x7f4f30105c90 2024-05-27T07:42:36.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.195+0000 7f4f2cc00700 1 -- 172.21.15.63:0/4035857748 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f4f1c0a9e20 con 0x7f4f30105c90 2024-05-27T07:42:36.194 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:42:36.195 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:42:36.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.200+0000 7f4f35400700 1 -- 172.21.15.63:0/4035857748 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f140b2a70 msgr2=0x7f4f140b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:36.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.200+0000 7f4f35400700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f140b2a70 0x7f4f140b4f80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f3019ba40 tx=0x7f4f2000b410 comp rx=0 tx=0).stop 2024-05-27T07:42:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.200+0000 7f4f35400700 1 -- 172.21.15.63:0/4035857748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f30105c90 msgr2=0x7f4f301a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.200+0000 7f4f35400700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f30105c90 0x7f4f301a0c20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00c9b0 tx=0x7f4f1c00ccc0 comp rx=0 tx=0).stop 2024-05-27T07:42:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.200+0000 7f4f35400700 1 -- 172.21.15.63:0/4035857748 shutdown_connections 2024-05-27T07:42:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.200+0000 7f4f35400700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4f140b2a70 0x7f4f140b4f80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.200+0000 7f4f35400700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f3010e720 0x7f4f3019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.200+0000 7f4f35400700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f30105340 0x7f4f301a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.200+0000 7f4f35400700 1 --2- 172.21.15.63:0/4035857748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f30105c90 0x7f4f301a0c20 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.200+0000 7f4f35400700 1 -- 172.21.15.63:0/4035857748 >> 172.21.15.63:0/4035857748 conn(0x7f4f30078ab0 msgr2=0x7f4f300ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.201+0000 7f4f35400700 1 -- 172.21.15.63:0/4035857748 shutdown_connections 2024-05-27T07:42:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:36.201+0000 7f4f35400700 1 -- 172.21.15.63:0/4035857748 wait complete. 2024-05-27T07:42:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:42:36.464 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:36 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4035857748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:42:36.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:36 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4035857748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:42:36.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:36 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4035857748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:42:37.171 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:42:37.706 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:37.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:37 smithi086 ceph-mon[34687]: pgmap v280: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:42:37.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:37 smithi149 ceph-mon[35408]: pgmap v280: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:42:37.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:37 smithi063 ceph-mon[31613]: pgmap v280: 129 pgs: 129 active+clean; 1.9 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:42:38.776 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:38 smithi063 ceph-mon[31613]: pgmap v281: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:42:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.827+0000 7f4855800640 1 -- 172.21.15.63:0/547230925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48501102f0 msgr2=0x7f4850112710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.827+0000 7f4855800640 1 --2- 172.21.15.63:0/547230925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48501102f0 0x7f4850112710 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4848009900 tx=0x7f484802a480 comp rx=0 tx=0).stop 2024-05-27T07:42:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.827+0000 7f4855800640 1 -- 172.21.15.63:0/547230925 shutdown_connections 2024-05-27T07:42:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.827+0000 7f4855800640 1 --2- 172.21.15.63:0/547230925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48501102f0 0x7f4850112710 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.827+0000 7f4855800640 1 --2- 172.21.15.63:0/547230925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485010d990 0x7f485010fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.827+0000 7f4855800640 1 --2- 172.21.15.63:0/547230925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850075480 0x7f4850075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.827+0000 7f4855800640 1 -- 172.21.15.63:0/547230925 >> 172.21.15.63:0/547230925 conn(0x7f4850079240 msgr2=0x7f4850079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.827+0000 7f4855800640 1 -- 172.21.15.63:0/547230925 shutdown_connections 2024-05-27T07:42:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.827+0000 7f4855800640 1 -- 172.21.15.63:0/547230925 wait complete. 2024-05-27T07:42:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.829+0000 7f4855800640 1 Processor -- start 2024-05-27T07:42:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.829+0000 7f4855800640 1 -- start start 2024-05-27T07:42:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.829+0000 7f4855800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850075480 0x7f485019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.829+0000 7f4855800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485010d990 0x7f485019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.829+0000 7f4855800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48501102f0 0x7f48501a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.829+0000 7f4855800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4850115d30 con 0x7f48501102f0 2024-05-27T07:42:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.829+0000 7f4855800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4850115bb0 con 0x7f4850075480 2024-05-27T07:42:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.829+0000 7f4855800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4850115eb0 con 0x7f485010d990 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.830+0000 7f4854e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48501102f0 0x7f48501a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.830+0000 7f4854e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48501102f0 0x7f48501a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53692/0 (socket says 172.21.15.63:53692) 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.830+0000 7f4854e00640 1 -- 172.21.15.63:0/1019882468 learned_addr learned my addr 172.21.15.63:0/1019882468 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.830+0000 7f484fe00640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850075480 0x7f485019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.830+0000 7f484f400640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485010d990 0x7f485019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.830+0000 7f4854e00640 1 -- 172.21.15.63:0/1019882468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485010d990 msgr2=0x7f485019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.830+0000 7f4854e00640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485010d990 0x7f485019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.830+0000 7f4854e00640 1 -- 172.21.15.63:0/1019882468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850075480 msgr2=0x7f485019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.830+0000 7f4854e00640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850075480 0x7f485019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.830+0000 7f4854e00640 1 -- 172.21.15.63:0/1019882468 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48501a1650 con 0x7f48501102f0 2024-05-27T07:42:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.831+0000 7f4854e00640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48501102f0 0x7f48501a1010 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4848005ab0 tx=0x7f48480040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.831+0000 7f484cc00640 1 -- 172.21.15.63:0/1019882468 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484803f030 con 0x7f48501102f0 2024-05-27T07:42:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.831+0000 7f484cc00640 1 -- 172.21.15.63:0/1019882468 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4848003a80 con 0x7f48501102f0 2024-05-27T07:42:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.831+0000 7f4855800640 1 -- 172.21.15.63:0/1019882468 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48501a18e0 con 0x7f48501102f0 2024-05-27T07:42:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.831+0000 7f484cc00640 1 -- 172.21.15.63:0/1019882468 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848035070 con 0x7f48501102f0 2024-05-27T07:42:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.831+0000 7f4855800640 1 -- 172.21.15.63:0/1019882468 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48501a1dd0 con 0x7f48501102f0 2024-05-27T07:42:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.836+0000 7f485de79740 1 Processor -- start 2024-05-27T07:42:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.836+0000 7f484cc00640 1 -- 172.21.15.63:0/1019882468 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4848042030 con 0x7f48501102f0 2024-05-27T07:42:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.836+0000 7f485de79740 1 -- start start 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.837+0000 7f485de79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294e71b0 0x55f529889a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.837+0000 7f485de79740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f529889fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.837+0000 7f485de79740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294c64d0 0x55f52988a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.837+0000 7f485de79740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f5297fab90 con 0x55f5294be2f0 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.837+0000 7f485de79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5297ea730 con 0x55f5294e71b0 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.837+0000 7f485de79740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f5297ee770 con 0x55f5294c64d0 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.837+0000 7f484fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f529889fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.837+0000 7f484fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f529889fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53704/0 (socket says 172.21.15.63:53704) 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.837+0000 7f484fe00640 1 -- 172.21.15.63:0/2116367365 learned_addr learned my addr 172.21.15.63:0/2116367365 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.838+0000 7f484f400640 1 --2- 172.21.15.63:0/2116367365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294e71b0 0x55f529889a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.838+0000 7f484f400640 1 --2- 172.21.15.63:0/2116367365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294c64d0 0x55f52988a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.838+0000 7f484fe00640 1 -- 172.21.15.63:0/2116367365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294c64d0 msgr2=0x55f52988a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.838+0000 7f484fe00640 1 --2- 172.21.15.63:0/2116367365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294c64d0 0x55f52988a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.838+0000 7f484fe00640 1 -- 172.21.15.63:0/2116367365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294e71b0 msgr2=0x55f529889a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.838+0000 7f484fe00640 1 --2- 172.21.15.63:0/2116367365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294e71b0 0x55f529889a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.838+0000 7f484fe00640 1 -- 172.21.15.63:0/2116367365 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5297838b0 con 0x55f5294be2f0 2024-05-27T07:42:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.838+0000 7f484cc00640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48340b2950 0x7f48340b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.838+0000 7f484fe00640 1 --2- 172.21.15.63:0/2116367365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f529889fa0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4840010420 tx=0x7f48400108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b734e91d0898027f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.839+0000 7f484cc00640 1 -- 172.21.15.63:0/1019882468 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f4848102ea0 con 0x7f48501102f0 2024-05-27T07:42:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.839+0000 7f484f400640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48340b2950 0x7f48340b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.839+0000 7f4845600640 1 -- 172.21.15.63:0/2116367365 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840016070 con 0x55f5294be2f0 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.839+0000 7f4845600640 1 -- 172.21.15.63:0/2116367365 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4840012ba0 con 0x55f5294be2f0 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.839+0000 7f4845600640 1 -- 172.21.15.63:0/2116367365 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484001bc70 con 0x55f5294be2f0 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.840+0000 7f485de79740 1 -- 172.21.15.63:0/2116367365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 msgr2=0x55f529889fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.840+0000 7f485de79740 1 --2- 172.21.15.63:0/2116367365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f529889fa0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4840010420 tx=0x7f48400108e0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.840+0000 7f484f400640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48340b2950 0x7f48340b4e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55f529886bd0 tx=0x7f483800b5b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.840+0000 7f485de79740 1 -- 172.21.15.63:0/2116367365 shutdown_connections 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.840+0000 7f485de79740 1 --2- 172.21.15.63:0/2116367365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294c64d0 0x55f52988a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.840+0000 7f485de79740 1 --2- 172.21.15.63:0/2116367365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f529889fa0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.840+0000 7f485de79740 1 --2- 172.21.15.63:0/2116367365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294e71b0 0x55f529889a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.840+0000 7f485de79740 1 -- 172.21.15.63:0/2116367365 >> 172.21.15.63:0/2116367365 conn(0x55f5294db300 msgr2=0x55f529886210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.840+0000 7f485de79740 1 -- 172.21.15.63:0/2116367365 shutdown_connections 2024-05-27T07:42:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.840+0000 7f485de79740 1 -- 172.21.15.63:0/2116367365 wait complete. 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.841+0000 7f485de79740 1 -- start start 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.841+0000 7f485de79740 1 Processor -- start 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.842+0000 7f485de79740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294be2f0 0x55f529893eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.842+0000 7f485de79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 0x55f5298943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.842+0000 7f485de79740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294e71b0 0x55f52989fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.842+0000 7f485de79740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f5294d0790 con 0x55f5294e71b0 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.842+0000 7f485de79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f529792bc0 con 0x55f5294c64d0 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.842+0000 7f485de79740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f529896b90 con 0x55f5294be2f0 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.842+0000 7f485de79740 10 client.0 _ll_register_callbacks cb 0x55f529770400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.842+0000 7f4854e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294e71b0 0x55f52989fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.842+0000 7f4854e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294e71b0 0x55f52989fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53706/0 (socket says 172.21.15.63:53706) 2024-05-27T07:42:38.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.842+0000 7f4854e00640 1 -- 172.21.15.63:0/877920160 learned_addr learned my addr 172.21.15.63:0/877920160 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f484fe00640 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294be2f0 0x55f529893eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f484f400640 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 0x55f5298943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f4854e00640 1 -- 172.21.15.63:0/877920160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294be2f0 msgr2=0x55f529893eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f4854e00640 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294be2f0 0x55f529893eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f4854e00640 1 -- 172.21.15.63:0/877920160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 msgr2=0x55f5298943f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f4854e00640 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 0x55f5298943f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f4854e00640 1 -- 172.21.15.63:0/877920160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5294ff210 con 0x55f5294e71b0 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f484fe00640 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294be2f0 0x55f529893eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f484f400640 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 0x55f5298943f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f4854e00640 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294e71b0 0x55f52989fef0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4848005760 tx=0x7f48480c9760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387853cfe5ac6759 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.843+0000 7f483e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.844+0000 7f483e000640 1 -- 172.21.15.63:0/877920160 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848049040 con 0x55f5294e71b0 2024-05-27T07:42:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.844+0000 7f483e000640 1 -- 172.21.15.63:0/877920160 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f48480e43e0 con 0x55f5294e71b0 2024-05-27T07:42:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.844+0000 7f485de79740 1 -- 172.21.15.63:0/877920160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4840012ba0 con 0x55f5294e71b0 2024-05-27T07:42:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.844+0000 7f483e000640 1 -- 172.21.15.63:0/877920160 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f48480c9e10 con 0x55f5294e71b0 2024-05-27T07:42:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.844+0000 7f485de79740 10 client.17136 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.844+0000 7f485de79740 1 -- 172.21.15.63:0/877920160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f5295cb620 con 0x55f5294e71b0 2024-05-27T07:42:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.844+0000 7f483e000640 1 -- 172.21.15.63:0/877920160 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4840012ba0 con 0x55f5294e71b0 2024-05-27T07:42:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.844+0000 7f483e000640 1 -- 172.21.15.63:0/877920160 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f48480c9070 con 0x55f5294e71b0 2024-05-27T07:42:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.845+0000 7f485de79740 10 client.17136 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.845+0000 7f485de79740 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.845+0000 7f485de79740 20 client.17136 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.845+0000 7f485de79740 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5294a5d80 0x55f5298a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.845+0000 7f485de79740 4 client.17136 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:42:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.845+0000 7f485de79740 1 -- 172.21.15.63:0/877920160 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5297fcae0 con 0x55f5294a5d80 2024-05-27T07:42:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.845+0000 7f484fe00640 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5294a5d80 0x55f5298a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.846+0000 7f484fe00640 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5294a5d80 0x55f5298a5f40 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0713e7e53ba42c7 server_cookie=f51e526c5ef5266e in_seq=0 out_seq=0 2024-05-27T07:42:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.846+0000 7f483e000640 10 client.17136 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.848+0000 7f483e000640 1 -- 172.21.15.63:0/877920160 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f5297fcae0 con 0x55f5294a5d80 2024-05-27T07:42:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.848+0000 7f483e000640 10 client.17136 handle_command_reply: tid=0 2024-05-27T07:42:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.848+0000 7f485de79740 1 client.17136 shutdown 2024-05-27T07:42:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 1 -- 172.21.15.63:0/877920160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294e71b0 msgr2=0x55f52989fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294e71b0 0x55f52989fef0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4848005760 tx=0x7f48480c9760 comp rx=0 tx=0).stop 2024-05-27T07:42:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 1 -- 172.21.15.63:0/877920160 shutdown_connections 2024-05-27T07:42:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5294a5d80 0x55f5298a5f40 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294e71b0 0x55f52989fef0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 0x55f5298943f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 1 --2- 172.21.15.63:0/877920160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294be2f0 0x55f529893eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f483e000640 0 client.17136 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 1 -- 172.21.15.63:0/877920160 >> 172.21.15.63:0/877920160 conn(0x55f5294db300 msgr2=0x55f529889a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 1 -- 172.21.15.63:0/877920160 shutdown_connections 2024-05-27T07:42:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 1 -- 172.21.15.63:0/877920160 wait complete. 2024-05-27T07:42:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.849+0000 7f485de79740 20 client.17136 trim_cache size 0 max 16384 2024-05-27T07:42:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.875+0000 7f485de79740 1 Processor -- start 2024-05-27T07:42:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.875+0000 7f485de79740 1 -- start start 2024-05-27T07:42:38.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.875+0000 7f485de79740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f52988f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.875+0000 7f485de79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 0x55f529895ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.875+0000 7f485de79740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294e71b0 0x55f529896010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.875+0000 7f485de79740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48501b5af0 con 0x55f5294be2f0 2024-05-27T07:42:38.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.875+0000 7f485de79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48501b33c0 con 0x55f5294c64d0 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.875+0000 7f485de79740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48501b1230 con 0x55f5294e71b0 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.876+0000 7f484fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f52988f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.876+0000 7f484fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f52988f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53720/0 (socket says 172.21.15.63:53720) 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.876+0000 7f484fe00640 1 -- 172.21.15.63:0/3966626676 learned_addr learned my addr 172.21.15.63:0/3966626676 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.876+0000 7f484f400640 1 --2- 172.21.15.63:0/3966626676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 0x55f529895ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.876+0000 7f4854e00640 1 --2- 172.21.15.63:0/3966626676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294e71b0 0x55f529896010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.876+0000 7f484fe00640 1 -- 172.21.15.63:0/3966626676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294e71b0 msgr2=0x55f529896010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.876+0000 7f484fe00640 1 --2- 172.21.15.63:0/3966626676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294e71b0 0x55f529896010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.876+0000 7f484fe00640 1 -- 172.21.15.63:0/3966626676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 msgr2=0x55f529895ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.876+0000 7f484fe00640 1 --2- 172.21.15.63:0/3966626676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 0x55f529895ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.876+0000 7f484fe00640 1 -- 172.21.15.63:0/3966626676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52988e1f0 con 0x55f5294be2f0 2024-05-27T07:42:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.877+0000 7f484fe00640 1 --2- 172.21.15.63:0/3966626676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f52988f440 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4840009f90 tx=0x7f48400096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ee9c56c8fa0a32 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.877+0000 7f4844c00640 1 -- 172.21.15.63:0/3966626676 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840016040 con 0x55f5294be2f0 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.877+0000 7f4844c00640 1 -- 172.21.15.63:0/3966626676 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48400129a0 con 0x55f5294be2f0 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.877+0000 7f4844c00640 1 -- 172.21.15.63:0/3966626676 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484001bc10 con 0x55f5294be2f0 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.878+0000 7f485de79740 1 -- 172.21.15.63:0/3966626676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 msgr2=0x55f52988f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.878+0000 7f485de79740 1 --2- 172.21.15.63:0/3966626676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f52988f440 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4840009f90 tx=0x7f48400096b0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.878+0000 7f485de79740 1 -- 172.21.15.63:0/3966626676 shutdown_connections 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.878+0000 7f485de79740 1 --2- 172.21.15.63:0/3966626676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294e71b0 0x55f529896010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.878+0000 7f485de79740 1 --2- 172.21.15.63:0/3966626676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294c64d0 0x55f529895ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.878+0000 7f485de79740 1 --2- 172.21.15.63:0/3966626676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294be2f0 0x55f52988f440 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.878+0000 7f485de79740 1 -- 172.21.15.63:0/3966626676 >> 172.21.15.63:0/3966626676 conn(0x55f5294db300 msgr2=0x55f5298953c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.879+0000 7f485de79740 1 -- 172.21.15.63:0/3966626676 shutdown_connections 2024-05-27T07:42:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.879+0000 7f485de79740 1 -- 172.21.15.63:0/3966626676 wait complete. 2024-05-27T07:42:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.879+0000 7f485de79740 1 -- start start 2024-05-27T07:42:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.879+0000 7f485de79740 1 Processor -- start 2024-05-27T07:42:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.880+0000 7f485de79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294be2f0 0x55f5298992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.880+0000 7f485de79740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294c64d0 0x55f529899800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.880+0000 7f485de79740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294e71b0 0x55f52990c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.880+0000 7f485de79740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48501b1880 con 0x55f5294c64d0 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.880+0000 7f485de79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48501b2cf0 con 0x55f5294be2f0 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.880+0000 7f485de79740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48501b1f50 con 0x55f5294e71b0 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.880+0000 7f485de79740 10 client.0 _ll_register_callbacks cb 0x7f4848005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.880+0000 7f484f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294c64d0 0x55f529899800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.880+0000 7f484f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294c64d0 0x55f529899800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53726/0 (socket says 172.21.15.63:53726) 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.880+0000 7f484f400640 1 -- 172.21.15.63:0/1060126442 learned_addr learned my addr 172.21.15.63:0/1060126442 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.881+0000 7f484fe00640 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294be2f0 0x55f5298992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.881+0000 7f4854e00640 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294e71b0 0x55f52990c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.881+0000 7f484f400640 1 -- 172.21.15.63:0/1060126442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294e71b0 msgr2=0x55f52990c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.881+0000 7f484f400640 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294e71b0 0x55f52990c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.881+0000 7f484f400640 1 -- 172.21.15.63:0/1060126442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294be2f0 msgr2=0x55f5298992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.881+0000 7f484f400640 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294be2f0 0x55f5298992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.881+0000 7f484f400640 1 -- 172.21.15.63:0/1060126442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52990cb50 con 0x55f5294c64d0 2024-05-27T07:42:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.881+0000 7f484fe00640 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294be2f0 0x55f5298992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.881+0000 7f484f400640 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294c64d0 0x55f529899800 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4838003f10 tx=0x7f4838004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab74e3b26078a5b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.881+0000 7f483e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.882+0000 7f483e000640 1 -- 172.21.15.63:0/1060126442 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483801f6f0 con 0x55f5294c64d0 2024-05-27T07:42:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.882+0000 7f483e000640 1 -- 172.21.15.63:0/1060126442 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f483801f890 con 0x55f5294c64d0 2024-05-27T07:42:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.882+0000 7f485de79740 1 -- 172.21.15.63:0/1060126442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48400129a0 con 0x55f5294c64d0 2024-05-27T07:42:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.882+0000 7f483e000640 1 -- 172.21.15.63:0/1060126442 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f483801fa90 con 0x55f5294c64d0 2024-05-27T07:42:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.882+0000 7f485de79740 10 client.17142 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.882+0000 7f485de79740 1 -- 172.21.15.63:0/1060126442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f52990dd80 con 0x55f5294c64d0 2024-05-27T07:42:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.883+0000 7f483e000640 1 -- 172.21.15.63:0/1060126442 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48400129a0 con 0x55f5294c64d0 2024-05-27T07:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.883+0000 7f483e000640 1 -- 172.21.15.63:0/1060126442 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f483802cbe0 con 0x55f5294c64d0 2024-05-27T07:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.883+0000 7f485de79740 10 client.17142 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.883+0000 7f485de79740 10 client.17142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.883+0000 7f485de79740 20 client.17142 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.883+0000 7f485de79740 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f52947b1c0 0x55f5299105f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.883+0000 7f485de79740 4 client.17142 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.883+0000 7f485de79740 1 -- 172.21.15.63:0/1060126442 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5297f62c0 con 0x55f52947b1c0 2024-05-27T07:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.884+0000 7f484fe00640 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f52947b1c0 0x55f5299105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.885+0000 7f484fe00640 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f52947b1c0 0x55f5299105f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a46680c27987758c server_cookie=eccf4d2a886cd2e in_seq=0 out_seq=0 2024-05-27T07:42:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.885+0000 7f483e000640 10 client.17142 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f483e000640 1 -- 172.21.15.63:0/1060126442 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f5297f62c0 con 0x55f52947b1c0 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f483e000640 10 client.17142 handle_command_reply: tid=0 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f485de79740 1 client.17142 shutdown 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f485de79740 1 -- 172.21.15.63:0/1060126442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294c64d0 msgr2=0x55f529899800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f485de79740 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294c64d0 0x55f529899800 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4838003f10 tx=0x7f4838004730 comp rx=0 tx=0).stop 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f485de79740 1 -- 172.21.15.63:0/1060126442 shutdown_connections 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f485de79740 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f52947b1c0 0x55f5299105f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f485de79740 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5294e71b0 0x55f52990c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f485de79740 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5294c64d0 0x55f529899800 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f485de79740 1 --2- 172.21.15.63:0/1060126442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5294be2f0 0x55f5298992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f485de79740 1 -- 172.21.15.63:0/1060126442 >> 172.21.15.63:0/1060126442 conn(0x55f5294db300 msgr2=0x55f529889f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.919+0000 7f483e000640 0 client.17142 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.920+0000 7f485de79740 1 -- 172.21.15.63:0/1060126442 shutdown_connections 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.920+0000 7f485de79740 1 -- 172.21.15.63:0/1060126442 wait complete. 2024-05-27T07:42:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.920+0000 7f485de79740 20 client.17142 trim_cache size 0 max 16384 2024-05-27T07:42:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:42:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:42:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:42:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 -- 172.21.15.63:0/1019882468 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48340b2950 msgr2=0x7f48340b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48340b2950 0x7f48340b4e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55f529886bd0 tx=0x7f483800b5b0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 -- 172.21.15.63:0/1019882468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48501102f0 msgr2=0x7f48501a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48501102f0 0x7f48501a1010 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4848005ab0 tx=0x7f48480040e0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 -- 172.21.15.63:0/1019882468 shutdown_connections 2024-05-27T07:42:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48340b2950 0x7f48340b4e60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48501102f0 0x7f48501a1010 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f485010d990 0x7f485019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 --2- 172.21.15.63:0/1019882468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850075480 0x7f485019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 -- 172.21.15.63:0/1019882468 >> 172.21.15.63:0/1019882468 conn(0x7f4850079240 msgr2=0x7f4850102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 -- 172.21.15.63:0/1019882468 shutdown_connections 2024-05-27T07:42:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:38.921+0000 7f4855800640 1 -- 172.21.15.63:0/1019882468 wait complete. 2024-05-27T07:42:38.956 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:38 smithi086 ceph-mon[34687]: pgmap v281: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:42:38.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:38 smithi149 ceph-mon[35408]: pgmap v281: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:42:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.093+0000 7fd5aea00700 1 -- 172.21.15.63:0/2794764094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a810c5a0 msgr2=0x7fd5a810c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.093+0000 7fd5aea00700 1 --2- 172.21.15.63:0/2794764094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a810c5a0 0x7fd5a810c9b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd598009990 tx=0x7fd598009ca0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.093+0000 7fd5aea00700 1 -- 172.21.15.63:0/2794764094 shutdown_connections 2024-05-27T07:42:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.093+0000 7fd5aea00700 1 --2- 172.21.15.63:0/2794764094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a8106eb0 0x7fd5a8113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.093+0000 7fd5aea00700 1 --2- 172.21.15.63:0/2794764094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a810c5a0 0x7fd5a810c9b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.093+0000 7fd5aea00700 1 --2- 172.21.15.63:0/2794764094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a8106560 0x7fd5a8106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.093+0000 7fd5aea00700 1 -- 172.21.15.63:0/2794764094 >> 172.21.15.63:0/2794764094 conn(0x7fd5a8102040 msgr2=0x7fd5a8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.093+0000 7fd5aea00700 1 -- 172.21.15.63:0/2794764094 shutdown_connections 2024-05-27T07:42:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.093+0000 7fd5aea00700 1 -- 172.21.15.63:0/2794764094 wait complete. 2024-05-27T07:42:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.095+0000 7fd5aea00700 1 Processor -- start 2024-05-27T07:42:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.095+0000 7fd5aea00700 1 -- start start 2024-05-27T07:42:40.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.096+0000 7fd5aea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a8106560 0x7fd5a819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.096+0000 7fd5aea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a8106eb0 0x7fd5a819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.096+0000 7fd5aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a810c5a0 0x7fd5a81a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.096+0000 7fd5aea00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81165d0 con 0x7fd5a810c5a0 2024-05-27T07:42:40.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.096+0000 7fd5aea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a8116450 con 0x7fd5a8106560 2024-05-27T07:42:40.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.096+0000 7fd5aea00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a8116750 con 0x7fd5a8106eb0 2024-05-27T07:42:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.096+0000 7fd5aca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a810c5a0 0x7fd5a81a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.096+0000 7fd5aca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a810c5a0 0x7fd5a81a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47630/0 (socket says 172.21.15.63:47630) 2024-05-27T07:42:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.096+0000 7fd5aca00700 1 -- 172.21.15.63:0/798649812 learned_addr learned my addr 172.21.15.63:0/798649812 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.097+0000 7fd5a7400700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a8106eb0 0x7fd5a819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.097+0000 7fd5a7e00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a8106560 0x7fd5a819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.097+0000 7fd5aca00700 1 -- 172.21.15.63:0/798649812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a8106eb0 msgr2=0x7fd5a819d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.097+0000 7fd5aca00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a8106eb0 0x7fd5a819d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.097+0000 7fd5aca00700 1 -- 172.21.15.63:0/798649812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a8106560 msgr2=0x7fd5a819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.097+0000 7fd5aca00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a8106560 0x7fd5a819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.097+0000 7fd5aca00700 1 -- 172.21.15.63:0/798649812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a81a1e50 con 0x7fd5a810c5a0 2024-05-27T07:42:40.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.097+0000 7fd5aca00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a810c5a0 0x7fd5a81a17b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000ef30 tx=0x7fd5a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:40.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.098+0000 7fd5a4c00700 1 -- 172.21.15.63:0/798649812 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00092e0 con 0x7fd5a810c5a0 2024-05-27T07:42:40.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.098+0000 7fd5a4c00700 1 -- 172.21.15.63:0/798649812 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5a0004bb0 con 0x7fd5a810c5a0 2024-05-27T07:42:40.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.098+0000 7fd5a4c00700 1 -- 172.21.15.63:0/798649812 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0004020 con 0x7fd5a810c5a0 2024-05-27T07:42:40.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.098+0000 7fd5aea00700 1 -- 172.21.15.63:0/798649812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a81a20e0 con 0x7fd5a810c5a0 2024-05-27T07:42:40.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.098+0000 7fd5aea00700 1 -- 172.21.15.63:0/798649812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a81a9a60 con 0x7fd5a810c5a0 2024-05-27T07:42:40.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.104+0000 7fd5a4c00700 1 -- 172.21.15.63:0/798649812 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd5a001b030 con 0x7fd5a810c5a0 2024-05-27T07:42:40.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.105+0000 7fd5bd59cb80 1 Processor -- start 2024-05-27T07:42:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.105+0000 7fd5bd59cb80 1 -- start start 2024-05-27T07:42:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.105+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dda179f0 0x5642ddaaa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.105+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dd92be40 0x5642ddaaa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.105+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.105+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5642dd9d8e40 con 0x5642ddaaae70 2024-05-27T07:42:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.105+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642dd9c89e0 con 0x5642dd92be40 2024-05-27T07:42:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.105+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5642dd9cca20 con 0x5642dda179f0 2024-05-27T07:42:40.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.106+0000 7fd5a4c00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd58c0b2a70 0x7fd58c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.106+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.106+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47646/0 (socket says 172.21.15.63:47646) 2024-05-27T07:42:40.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.106+0000 7fd5a7400700 1 -- 172.21.15.63:0/2162138402 learned_addr learned my addr 172.21.15.63:0/2162138402 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.106+0000 7fd5a7400700 1 --2- 172.21.15.63:0/2162138402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dda179f0 0x5642ddaaa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.106+0000 7fd5a4c00700 1 -- 172.21.15.63:0/798649812 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fd5a00e24e0 con 0x7fd5a810c5a0 2024-05-27T07:42:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.106+0000 7fd5aca00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd58c0b2a70 0x7fd58c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.106+0000 7fd5a7e00700 1 --2- 172.21.15.63:0/2162138402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dd92be40 0x5642ddaaa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.107+0000 7fd5a7400700 1 -- 172.21.15.63:0/2162138402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dda179f0 msgr2=0x5642ddaaa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.107+0000 7fd5a7400700 1 --2- 172.21.15.63:0/2162138402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dda179f0 0x5642ddaaa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.107+0000 7fd5a7400700 1 -- 172.21.15.63:0/2162138402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dd92be40 msgr2=0x5642ddaaa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.107+0000 7fd5a7400700 1 --2- 172.21.15.63:0/2162138402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dd92be40 0x5642ddaaa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.107+0000 7fd5a7400700 1 -- 172.21.15.63:0/2162138402 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642dd991420 con 0x5642ddaaae70 2024-05-27T07:42:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.107+0000 7fd5a7400700 1 --2- 172.21.15.63:0/2162138402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab7110 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd594012700 tx=0x7fd594009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0c042061999728 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.107+0000 7fd5aca00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd58c0b2a70 0x7fd58c0b4f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00a5510 tx=0x7fd5a0009480 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.108+0000 7fd59d600700 1 -- 172.21.15.63:0/2162138402 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594012da0 con 0x5642ddaaae70 2024-05-27T07:42:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.108+0000 7fd59d600700 1 -- 172.21.15.63:0/2162138402 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5940044c0 con 0x5642ddaaae70 2024-05-27T07:42:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.108+0000 7fd59d600700 1 -- 172.21.15.63:0/2162138402 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594005080 con 0x5642ddaaae70 2024-05-27T07:42:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.109+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/2162138402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 msgr2=0x5642ddab7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.109+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/2162138402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab7110 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd594012700 tx=0x7fd594009710 comp rx=0 tx=0).stop 2024-05-27T07:42:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.109+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/2162138402 shutdown_connections 2024-05-27T07:42:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.109+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/2162138402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab7110 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.109+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/2162138402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dda179f0 0x5642ddaaa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.109+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/2162138402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dd92be40 0x5642ddaaa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.109+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/2162138402 >> 172.21.15.63:0/2162138402 conn(0x5642dd9c5390 msgr2=0x5642ddaa64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.109+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/2162138402 shutdown_connections 2024-05-27T07:42:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.109+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/2162138402 wait complete. 2024-05-27T07:42:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.110+0000 7fd5bd59cb80 1 -- start start 2024-05-27T07:42:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.110+0000 7fd5bd59cb80 1 Processor -- start 2024-05-27T07:42:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddab4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642dda179f0 0x5642ddab4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642ddaaae70 0x5642ddab50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5642dd8cefa0 con 0x5642dda179f0 2024-05-27T07:42:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642dd969760 con 0x5642ddaaae70 2024-05-27T07:42:40.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5642ddab7b80 con 0x5642dd92be40 2024-05-27T07:42:40.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5bd59cb80 10 client.0 _ll_register_callbacks cb 0x5642dd972c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:40.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5a7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642dda179f0 0x5642ddab4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5a7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642dda179f0 0x5642ddab4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47648/0 (socket says 172.21.15.63:47648) 2024-05-27T07:42:40.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5a7e00700 1 -- 172.21.15.63:0/3169390726 learned_addr learned my addr 172.21.15.63:0/3169390726 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:40.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.111+0000 7fd5a7400700 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddab4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.112+0000 7fd5a7400700 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642ddaaae70 0x5642ddab50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.112+0000 7fd5a7e00700 1 -- 172.21.15.63:0/3169390726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 msgr2=0x5642ddab4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.112+0000 7fd5a7e00700 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddab4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.112+0000 7fd5a7e00700 1 -- 172.21.15.63:0/3169390726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642ddaaae70 msgr2=0x5642ddab50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.112+0000 7fd5a7e00700 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642ddaaae70 0x5642ddab50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.112+0000 7fd5a7e00700 1 -- 172.21.15.63:0/3169390726 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642dd9188d0 con 0x5642dda179f0 2024-05-27T07:42:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.113+0000 7fd5a7e00700 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642dda179f0 0x5642ddab4b80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd598009990 tx=0x7fd598005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a0b1c7f36d1ff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.113+0000 7fd592000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.113+0000 7fd592000700 1 -- 172.21.15.63:0/3169390726 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598028060 con 0x5642dda179f0 2024-05-27T07:42:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.113+0000 7fd592000700 1 -- 172.21.15.63:0/3169390726 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fd598003c60 con 0x5642dda179f0 2024-05-27T07:42:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.113+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3169390726 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5940044c0 con 0x5642dda179f0 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.113+0000 7fd592000700 1 -- 172.21.15.63:0/3169390726 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd59802f050 con 0x5642dda179f0 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.113+0000 7fd5bd59cb80 10 client.17154 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.113+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3169390726 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5642dd999060 con 0x5642dda179f0 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.114+0000 7fd592000700 1 -- 172.21.15.63:0/3169390726 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5940044c0 con 0x5642dda179f0 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.114+0000 7fd592000700 1 -- 172.21.15.63:0/3169390726 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd59800f580 con 0x5642dda179f0 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.114+0000 7fd5bd59cb80 10 client.17154 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.114+0000 7fd5bd59cb80 10 client.17154 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.114+0000 7fd5bd59cb80 20 client.17154 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.114+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5642ddac4ac0 0x5642ddac6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.114+0000 7fd5bd59cb80 4 client.17154 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:42:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.114+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3169390726 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642dd9dad90 con 0x5642ddac4ac0 2024-05-27T07:42:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.115+0000 7fd5a7400700 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5642ddac4ac0 0x5642ddac6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.116+0000 7fd5a7400700 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5642ddac4ac0 0x5642ddac6f10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=597ede9b4a974dd9 server_cookie=f4688e1426f11a9e in_seq=0 out_seq=0 2024-05-27T07:42:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.116+0000 7fd592000700 10 client.17154 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.118+0000 7fd592000700 1 -- 172.21.15.63:0/3169390726 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5642dd9dad90 con 0x5642ddac4ac0 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.118+0000 7fd592000700 10 client.17154 handle_command_reply: tid=0 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.118+0000 7fd5bd59cb80 1 client.17154 shutdown 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.118+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3169390726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642dda179f0 msgr2=0x5642ddab4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.118+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642dda179f0 0x5642ddab4b80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd598009990 tx=0x7fd598005750 comp rx=0 tx=0).stop 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.118+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3169390726 shutdown_connections 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.118+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642ddaaae70 0x5642ddab50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.119+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddab4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.119+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5642ddac4ac0 0x5642ddac6f10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.119+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/3169390726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642dda179f0 0x5642ddab4b80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.119+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3169390726 >> 172.21.15.63:0/3169390726 conn(0x5642dd9c5390 msgr2=0x5642ddaaa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:40.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.119+0000 7fd592000700 0 client.17154 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:42:40.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.119+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3169390726 shutdown_connections 2024-05-27T07:42:40.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.119+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3169390726 wait complete. 2024-05-27T07:42:40.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.119+0000 7fd5bd59cb80 20 client.17154 trim_cache size 0 max 16384 2024-05-27T07:42:40.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.140+0000 7fd5bd59cb80 1 Processor -- start 2024-05-27T07:42:40.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.140+0000 7fd5bd59cb80 1 -- start start 2024-05-27T07:42:40.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.141+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddaafe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.141+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 0x5642ddab35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.141+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddaaa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.141+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81b4fd0 con 0x5642ddaaae70 2024-05-27T07:42:40.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.141+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81b2860 con 0x5642dda179f0 2024-05-27T07:42:40.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.141+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81b06d0 con 0x5642dd92be40 2024-05-27T07:42:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.141+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddaaa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.141+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddaaa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47652/0 (socket says 172.21.15.63:47652) 2024-05-27T07:42:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.141+0000 7fd5a7400700 1 -- 172.21.15.63:0/3514008457 learned_addr learned my addr 172.21.15.63:0/3514008457 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.142+0000 7fd5a7e00700 1 --2- 172.21.15.63:0/3514008457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 0x5642ddab35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.142+0000 7fd5a7400700 1 -- 172.21.15.63:0/3514008457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 msgr2=0x5642ddaafe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.142+0000 7fd5a7400700 1 --2- 172.21.15.63:0/3514008457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddaafe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.142+0000 7fd5a7400700 1 -- 172.21.15.63:0/3514008457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 msgr2=0x5642ddab35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.142+0000 7fd5a7400700 1 --2- 172.21.15.63:0/3514008457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 0x5642ddab35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.142+0000 7fd5a7400700 1 -- 172.21.15.63:0/3514008457 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642ddab2bb0 con 0x5642ddaaae70 2024-05-27T07:42:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.142+0000 7fd5a7400700 1 --2- 172.21.15.63:0/3514008457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddaaa830 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd594012700 tx=0x7fd5940058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca594b47af58b937 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.143+0000 7fd59cc00700 1 -- 172.21.15.63:0/3514008457 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594005580 con 0x5642ddaaae70 2024-05-27T07:42:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.143+0000 7fd59cc00700 1 -- 172.21.15.63:0/3514008457 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd59402d8e0 con 0x5642ddaaae70 2024-05-27T07:42:40.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.143+0000 7fd59cc00700 1 -- 172.21.15.63:0/3514008457 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59400f8e0 con 0x5642ddaaae70 2024-05-27T07:42:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.144+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3514008457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 msgr2=0x5642ddaaa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.144+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/3514008457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddaaa830 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd594012700 tx=0x7fd5940058e0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.144+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3514008457 shutdown_connections 2024-05-27T07:42:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.144+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/3514008457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddaaa830 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.144+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/3514008457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddaafe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.144+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/3514008457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 0x5642ddab35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.144+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3514008457 >> 172.21.15.63:0/3514008457 conn(0x5642dd9c5390 msgr2=0x5642ddaaa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.144+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3514008457 shutdown_connections 2024-05-27T07:42:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.144+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/3514008457 wait complete. 2024-05-27T07:42:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.145+0000 7fd5bd59cb80 1 -- start start 2024-05-27T07:42:40.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.145+0000 7fd5bd59cb80 1 Processor -- start 2024-05-27T07:42:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.146+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddabcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.146+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 0x5642ddabd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.146+0000 7fd5bd59cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.146+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81b0d20 con 0x5642ddaaae70 2024-05-27T07:42:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.146+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81b2190 con 0x5642dda179f0 2024-05-27T07:42:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.146+0000 7fd5bd59cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81b13f0 con 0x5642dd92be40 2024-05-27T07:42:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.146+0000 7fd5bd59cb80 10 client.0 _ll_register_callbacks cb 0x5642dd972c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:40.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.147+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.147+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47668/0 (socket says 172.21.15.63:47668) 2024-05-27T07:42:40.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.147+0000 7fd5a7400700 1 -- 172.21.15.63:0/1651591139 learned_addr learned my addr 172.21.15.63:0/1651591139 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:40.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.147+0000 7fd5a7e00700 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 0x5642ddabd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.147+0000 7fd5a7400700 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddabcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.147+0000 7fd5a7e00700 1 -- 172.21.15.63:0/1651591139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 msgr2=0x5642ddabcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:40.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.147+0000 7fd5a7e00700 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddabcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.147+0000 7fd5a7e00700 1 -- 172.21.15.63:0/1651591139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 msgr2=0x5642ddab83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:40.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.147+0000 7fd5a7e00700 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.147+0000 7fd5a7e00700 1 -- 172.21.15.63:0/1651591139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ddab8ab0 con 0x5642dda179f0 2024-05-27T07:42:40.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.148+0000 7fd5a7400700 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:42:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.148+0000 7fd5a7e00700 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 0x5642ddabd4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd59800f950 tx=0x7fd598009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13cb55d3c5b2a13d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.148+0000 7fd592000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:42:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.148+0000 7fd592000700 1 -- 172.21.15.63:0/1651591139 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598028040 con 0x5642dda179f0 2024-05-27T07:42:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.149+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/1651591139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd59402d8e0 con 0x5642dda179f0 2024-05-27T07:42:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.149+0000 7fd592000700 1 -- 172.21.15.63:0/1651591139 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fd59800f670 con 0x5642dda179f0 2024-05-27T07:42:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.149+0000 7fd592000700 1 -- 172.21.15.63:0/1651591139 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd598043050 con 0x5642dda179f0 2024-05-27T07:42:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.149+0000 7fd5bd59cb80 10 client.36722 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.149+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/1651591139 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5642ddb37810 con 0x5642dda179f0 2024-05-27T07:42:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.150+0000 7fd592000700 1 -- 172.21.15.63:0/1651591139 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd59402d8e0 con 0x5642dda179f0 2024-05-27T07:42:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.150+0000 7fd592000700 1 -- 172.21.15.63:0/1651591139 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd59800b9b0 con 0x5642dda179f0 2024-05-27T07:42:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.150+0000 7fd5bd59cb80 10 client.36722 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.150+0000 7fd5bd59cb80 10 client.36722 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:42:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.150+0000 7fd5bd59cb80 20 client.36722 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.150+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5642ddb37df0 0x5642ddb3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.150+0000 7fd5bd59cb80 4 client.36722 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:42:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.150+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/1651591139 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642dd9d4570 con 0x5642ddb37df0 2024-05-27T07:42:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.151+0000 7fd5a7400700 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5642ddb37df0 0x5642ddb3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.152+0000 7fd5a7400700 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5642ddb37df0 0x5642ddb3a210 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=146708d29f4c9aed server_cookie=90137b7b9dd4c6e in_seq=0 out_seq=0 2024-05-27T07:42:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.152+0000 7fd592000700 10 client.36722 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:42:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.152+0000 7fd592000700 1 -- 172.21.15.63:0/1651591139 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642dd9d4570 con 0x5642ddb37df0 2024-05-27T07:42:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.152+0000 7fd592000700 10 client.36722 handle_command_reply: tid=0 2024-05-27T07:42:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.152+0000 7fd5bd59cb80 1 client.36722 shutdown 2024-05-27T07:42:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/1651591139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 msgr2=0x5642ddabd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 0x5642ddabd4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd59800f950 tx=0x7fd598009310 comp rx=0 tx=0).stop 2024-05-27T07:42:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/1651591139 shutdown_connections 2024-05-27T07:42:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5642ddb37df0 0x5642ddb3a210 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642ddaaae70 0x5642ddab83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642dd92be40 0x5642ddabcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd592000700 0 client.36722 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:42:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 1 --2- 172.21.15.63:0/1651591139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642dda179f0 0x5642ddabd4c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/1651591139 >> 172.21.15.63:0/1651591139 conn(0x5642dd9c5390 msgr2=0x5642ddab5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/1651591139 shutdown_connections 2024-05-27T07:42:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 1 -- 172.21.15.63:0/1651591139 wait complete. 2024-05-27T07:42:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.153+0000 7fd5bd59cb80 20 client.36722 trim_cache size 0 max 16384 2024-05-27T07:42:40.153 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:42:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.154+0000 7fd5aea00700 1 -- 172.21.15.63:0/798649812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd58c0b2a70 msgr2=0x7fd58c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.154+0000 7fd5aea00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd58c0b2a70 0x7fd58c0b4f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00a5510 tx=0x7fd5a0009480 comp rx=0 tx=0).stop 2024-05-27T07:42:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.154+0000 7fd5aea00700 1 -- 172.21.15.63:0/798649812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a810c5a0 msgr2=0x7fd5a81a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.154+0000 7fd5aea00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a810c5a0 0x7fd5a81a17b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000ef30 tx=0x7fd5a000c3a0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.154+0000 7fd5aea00700 1 -- 172.21.15.63:0/798649812 shutdown_connections 2024-05-27T07:42:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.154+0000 7fd5aea00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd58c0b2a70 0x7fd58c0b4f80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.154+0000 7fd5aea00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5a810c5a0 0x7fd5a81a17b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.154+0000 7fd5aea00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a8106560 0x7fd5a819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.154+0000 7fd5aea00700 1 --2- 172.21.15.63:0/798649812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a8106eb0 0x7fd5a819d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.154+0000 7fd5aea00700 1 -- 172.21.15.63:0/798649812 >> 172.21.15.63:0/798649812 conn(0x7fd5a8102040 msgr2=0x7fd5a81038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.155+0000 7fd5aea00700 1 -- 172.21.15.63:0/798649812 shutdown_connections 2024-05-27T07:42:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:40.155+0000 7fd5aea00700 1 -- 172.21.15.63:0/798649812 wait complete. 2024-05-27T07:42:40.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:40 smithi086 ceph-mon[34687]: pgmap v282: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:42:40.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:40 smithi149 ceph-mon[35408]: pgmap v282: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:42:40.963 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:40 smithi063 ceph-mon[31613]: pgmap v282: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:42:41.109 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:42:41.658 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:42.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:42 smithi086 ceph-mon[34687]: pgmap v283: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:42:42.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:42 smithi149 ceph-mon[35408]: pgmap v283: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:42:42.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:42 smithi063 ceph-mon[31613]: pgmap v283: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:42:44.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.067+0000 7f9b2da00700 1 -- 172.21.15.63:0/1899907167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b2810e6f0 msgr2=0x7f9b28110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:44.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.067+0000 7f9b2da00700 1 --2- 172.21.15.63:0/1899907167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b2810e6f0 0x7f9b28110bd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000b0a0 tx=0x7f9b2000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.067+0000 7f9b2da00700 1 -- 172.21.15.63:0/1899907167 shutdown_connections 2024-05-27T07:42:44.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.067+0000 7f9b2da00700 1 --2- 172.21.15.63:0/1899907167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b2810e6f0 0x7f9b28110bd0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.067+0000 7f9b2da00700 1 --2- 172.21.15.63:0/1899907167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b28105280 0x7f9b28105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.067+0000 7f9b2da00700 1 --2- 172.21.15.63:0/1899907167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b28105bd0 0x7f9b2810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.067+0000 7f9b2da00700 1 -- 172.21.15.63:0/1899907167 >> 172.21.15.63:0/1899907167 conn(0x7f9b28078ab0 msgr2=0x7f9b28078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:44.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.067+0000 7f9b2da00700 1 -- 172.21.15.63:0/1899907167 shutdown_connections 2024-05-27T07:42:44.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.067+0000 7f9b2da00700 1 -- 172.21.15.63:0/1899907167 wait complete. 2024-05-27T07:42:44.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.069+0000 7f9b2da00700 1 Processor -- start 2024-05-27T07:42:44.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.069+0000 7f9b2da00700 1 -- start start 2024-05-27T07:42:44.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b2da00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b28105280 0x7f9b281a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b2da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28105bd0 0x7f9b281a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b2da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2810e6f0 0x7f9b2819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b2da00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b28114100 con 0x7f9b28105bd0 2024-05-27T07:42:44.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b2da00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b28113f80 con 0x7f9b2810e6f0 2024-05-27T07:42:44.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b2da00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9b28114280 con 0x7f9b28105280 2024-05-27T07:42:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b27400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28105bd0 0x7f9b281a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b27400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28105bd0 0x7f9b281a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47672/0 (socket says 172.21.15.63:47672) 2024-05-27T07:42:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b27400700 1 -- 172.21.15.63:0/1658824923 learned_addr learned my addr 172.21.15.63:0/1658824923 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b27e00700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b28105280 0x7f9b281a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.070+0000 7f9b2d000700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2810e6f0 0x7f9b2819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.071+0000 7f9b27400700 1 -- 172.21.15.63:0/1658824923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b28105280 msgr2=0x7f9b281a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.071+0000 7f9b27400700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b28105280 0x7f9b281a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.071+0000 7f9b27400700 1 -- 172.21.15.63:0/1658824923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2810e6f0 msgr2=0x7f9b2819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.071+0000 7f9b27400700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2810e6f0 0x7f9b2819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.071+0000 7f9b27400700 1 -- 172.21.15.63:0/1658824923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2819d1e0 con 0x7f9b28105bd0 2024-05-27T07:42:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.071+0000 7f9b2d000700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2810e6f0 0x7f9b2819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.071+0000 7f9b27400700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28105bd0 0x7f9b281a2e20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400e9a0 tx=0x7f9b1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.072+0000 7f9b24c00700 1 -- 172.21.15.63:0/1658824923 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1400cbd0 con 0x7f9b28105bd0 2024-05-27T07:42:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.072+0000 7f9b24c00700 1 -- 172.21.15.63:0/1658824923 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b1400cd70 con 0x7f9b28105bd0 2024-05-27T07:42:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.072+0000 7f9b2da00700 1 -- 172.21.15.63:0/1658824923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2819d4d0 con 0x7f9b28105bd0 2024-05-27T07:42:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.072+0000 7f9b24c00700 1 -- 172.21.15.63:0/1658824923 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14018680 con 0x7f9b28105bd0 2024-05-27T07:42:44.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.072+0000 7f9b2da00700 1 -- 172.21.15.63:0/1658824923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b281a9770 con 0x7f9b28105bd0 2024-05-27T07:42:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.078+0000 7f9b3dabab80 1 Processor -- start 2024-05-27T07:42:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.078+0000 7f9b24c00700 1 -- 172.21.15.63:0/1658824923 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9b14010430 con 0x7f9b28105bd0 2024-05-27T07:42:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.078+0000 7f9b3dabab80 1 -- start start 2024-05-27T07:42:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.079+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 0x55ee8c401ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.079+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.079+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402820 0x55ee8c40f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.079+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c330e40 con 0x55ee8c402410 2024-05-27T07:42:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.079+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c3209e0 con 0x55ee8c402820 2024-05-27T07:42:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.079+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c324a20 con 0x55ee8c283e40 2024-05-27T07:42:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.079+0000 7f9b27e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.079+0000 7f9b27e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47686/0 (socket says 172.21.15.63:47686) 2024-05-27T07:42:44.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.079+0000 7f9b2d000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 0x55ee8c401ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.079+0000 7f9b27e00700 1 -- 172.21.15.63:0/3472442498 learned_addr learned my addr 172.21.15.63:0/3472442498 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:44.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.080+0000 7f9b27400700 1 --2- 172.21.15.63:0/3472442498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402820 0x55ee8c40f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.080+0000 7f9b27e00700 1 -- 172.21.15.63:0/3472442498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 msgr2=0x55ee8c401ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:44.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.080+0000 7f9b27e00700 1 --2- 172.21.15.63:0/3472442498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 0x55ee8c401ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.080+0000 7f9b27e00700 1 -- 172.21.15.63:0/3472442498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402820 msgr2=0x55ee8c40f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:44.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.080+0000 7f9b27e00700 1 --2- 172.21.15.63:0/3472442498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402820 0x55ee8c40f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.080+0000 7f9b27e00700 1 -- 172.21.15.63:0/3472442498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8c2e9420 con 0x55ee8c402410 2024-05-27T07:42:44.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.080+0000 7f9b2d000700 1 --2- 172.21.15.63:0/3472442498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 0x55ee8c401ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:42:44.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.080+0000 7f9b27400700 1 --2- 172.21.15.63:0/3472442498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402820 0x55ee8c40f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:44.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.080+0000 7f9b27e00700 1 --2- 172.21.15.63:0/3472442498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cad0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009d20 tx=0x7f9b18011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bec7f7cf79b259c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:44.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.080+0000 7f9b24c00700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b0c0b29f0 0x7f9b0c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.081+0000 7f9b24c00700 1 -- 172.21.15.63:0/1658824923 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f9b140e2c50 con 0x7f9b28105bd0 2024-05-27T07:42:44.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.081+0000 7f9b1d600700 1 -- 172.21.15.63:0/3472442498 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18004a30 con 0x55ee8c402410 2024-05-27T07:42:44.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.081+0000 7f9b2d000700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b0c0b29f0 0x7f9b0c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.081+0000 7f9b1d600700 1 -- 172.21.15.63:0/3472442498 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b180054b0 con 0x55ee8c402410 2024-05-27T07:42:44.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.081+0000 7f9b1d600700 1 -- 172.21.15.63:0/3472442498 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18019830 con 0x55ee8c402410 2024-05-27T07:42:44.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.082+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3472442498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 msgr2=0x55ee8c40cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:44.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.082+0000 7f9b2d000700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b0c0b29f0 0x7f9b0c0b4f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009900 tx=0x7f9b200060a0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:44.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.082+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/3472442498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cad0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009d20 tx=0x7f9b18011720 comp rx=0 tx=0).stop 2024-05-27T07:42:44.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.082+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3472442498 shutdown_connections 2024-05-27T07:42:44.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.082+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/3472442498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402820 0x55ee8c40f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.082+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/3472442498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 0x55ee8c401ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.082+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/3472442498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cad0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.082+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3472442498 >> 172.21.15.63:0/3472442498 conn(0x55ee8c2e3fc0 msgr2=0x55ee8c3fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:44.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.083+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3472442498 shutdown_connections 2024-05-27T07:42:44.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.083+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3472442498 wait complete. 2024-05-27T07:42:44.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.083+0000 7f9b3dabab80 1 -- start start 2024-05-27T07:42:44.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.084+0000 7f9b3dabab80 1 Processor -- start 2024-05-27T07:42:44.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.084+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c283e40 0x55ee8c40c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c402820 0x55ee8c40d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c226fa0 con 0x55ee8c402410 2024-05-27T07:42:44.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c2c1760 con 0x55ee8c283e40 2024-05-27T07:42:44.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ee8c40f990 con 0x55ee8c402820 2024-05-27T07:42:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b3dabab80 10 client.0 _ll_register_callbacks cb 0x55ee8c2cac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b27400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b27400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47690/0 (socket says 172.21.15.63:47690) 2024-05-27T07:42:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b27400700 1 -- 172.21.15.63:0/511530730 learned_addr learned my addr 172.21.15.63:0/511530730 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b27e00700 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c283e40 0x55ee8c40c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.085+0000 7f9b2d000700 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c402820 0x55ee8c40d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.086+0000 7f9b27400700 1 -- 172.21.15.63:0/511530730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c402820 msgr2=0x55ee8c40d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.086+0000 7f9b27400700 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c402820 0x55ee8c40d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.086+0000 7f9b27400700 1 -- 172.21.15.63:0/511530730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c283e40 msgr2=0x55ee8c40c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.086+0000 7f9b27400700 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c283e40 0x55ee8c40c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.086+0000 7f9b27400700 1 -- 172.21.15.63:0/511530730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8c2708d0 con 0x55ee8c402410 2024-05-27T07:42:44.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.086+0000 7f9b27e00700 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c283e40 0x55ee8c40c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:44.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.086+0000 7f9b27400700 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cb00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400f010 tx=0x7f9b140b27c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877cbd72ff02f2c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:44.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.086+0000 7f9b12000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.087+0000 7f9b12000700 1 -- 172.21.15.63:0/511530730 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140a9280 con 0x55ee8c402410 2024-05-27T07:42:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.087+0000 7f9b12000700 1 -- 172.21.15.63:0/511530730 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f9b140a6100 con 0x55ee8c402410 2024-05-27T07:42:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.087+0000 7f9b3dabab80 1 -- 172.21.15.63:0/511530730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b180054b0 con 0x55ee8c402410 2024-05-27T07:42:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.087+0000 7f9b12000700 1 -- 172.21.15.63:0/511530730 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9b140a52a0 con 0x55ee8c402410 2024-05-27T07:42:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.087+0000 7f9b3dabab80 10 client.17172 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.087+0000 7f9b3dabab80 1 -- 172.21.15.63:0/511530730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ee8c2f1060 con 0x55ee8c402410 2024-05-27T07:42:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.088+0000 7f9b12000700 1 -- 172.21.15.63:0/511530730 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b180054b0 con 0x55ee8c402410 2024-05-27T07:42:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.088+0000 7f9b12000700 1 -- 172.21.15.63:0/511530730 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9b140a5480 con 0x55ee8c402410 2024-05-27T07:42:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.088+0000 7f9b3dabab80 10 client.17172 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.088+0000 7f9b3dabab80 10 client.17172 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:42:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.088+0000 7f9b3dabab80 20 client.17172 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.088+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55ee8c41c8f0 0x55ee8c41ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.088+0000 7f9b3dabab80 4 client.17172 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:42:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.088+0000 7f9b3dabab80 1 -- 172.21.15.63:0/511530730 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee8c332d90 con 0x55ee8c41c8f0 2024-05-27T07:42:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.089+0000 7f9b27e00700 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55ee8c41c8f0 0x55ee8c41ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.090+0000 7f9b27e00700 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55ee8c41c8f0 0x55ee8c41ed30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be48b8e5b7a5f4dc server_cookie=301e1383ccaee4d0 in_seq=0 out_seq=0 2024-05-27T07:42:44.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.090+0000 7f9b12000700 10 client.17172 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:42:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.092+0000 7f9b12000700 1 -- 172.21.15.63:0/511530730 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ee8c332d90 con 0x55ee8c41c8f0 2024-05-27T07:42:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.092+0000 7f9b12000700 10 client.17172 handle_command_reply: tid=0 2024-05-27T07:42:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.092+0000 7f9b3dabab80 1 client.17172 shutdown 2024-05-27T07:42:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.092+0000 7f9b3dabab80 1 -- 172.21.15.63:0/511530730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 msgr2=0x55ee8c40cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.092+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cb00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400f010 tx=0x7f9b140b27c0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.093+0000 7f9b3dabab80 1 -- 172.21.15.63:0/511530730 shutdown_connections 2024-05-27T07:42:44.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.093+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c402820 0x55ee8c40d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.093+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c283e40 0x55ee8c40c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.093+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55ee8c41c8f0 0x55ee8c41ed30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.093+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/511530730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402410 0x55ee8c40cb00 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.093+0000 7f9b3dabab80 1 -- 172.21.15.63:0/511530730 >> 172.21.15.63:0/511530730 conn(0x55ee8c2e3fc0 msgr2=0x55ee8c3fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:44.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.093+0000 7f9b12000700 0 client.17172 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:42:44.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.093+0000 7f9b3dabab80 1 -- 172.21.15.63:0/511530730 shutdown_connections 2024-05-27T07:42:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.093+0000 7f9b3dabab80 1 -- 172.21.15.63:0/511530730 wait complete. 2024-05-27T07:42:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.093+0000 7f9b3dabab80 20 client.17172 trim_cache size 0 max 16384 2024-05-27T07:42:44.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.116+0000 7f9b3dabab80 1 Processor -- start 2024-05-27T07:42:44.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.116+0000 7f9b3dabab80 1 -- start start 2024-05-27T07:42:44.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.117+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 0x55ee8c40b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.117+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 0x55ee8c40ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.117+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402820 0x55ee8c407d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.117+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b281b4ec0 con 0x55ee8c402820 2024-05-27T07:42:44.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.117+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b281b2750 con 0x55ee8c402410 2024-05-27T07:42:44.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.117+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9b281b05c0 con 0x55ee8c283e40 2024-05-27T07:42:44.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.118+0000 7f9b2d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402820 0x55ee8c407d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.118+0000 7f9b2d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402820 0x55ee8c407d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47692/0 (socket says 172.21.15.63:47692) 2024-05-27T07:42:44.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.118+0000 7f9b2d000700 1 -- 172.21.15.63:0/450314538 learned_addr learned my addr 172.21.15.63:0/450314538 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:44.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.118+0000 7f9b27e00700 1 --2- 172.21.15.63:0/450314538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 0x55ee8c40b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.118+0000 7f9b27400700 1 --2- 172.21.15.63:0/450314538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 0x55ee8c40ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.118+0000 7f9b2d000700 1 -- 172.21.15.63:0/450314538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 msgr2=0x55ee8c40b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:44.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.118+0000 7f9b2d000700 1 --2- 172.21.15.63:0/450314538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 0x55ee8c40b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.118+0000 7f9b2d000700 1 -- 172.21.15.63:0/450314538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 msgr2=0x55ee8c40ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:44.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.118+0000 7f9b2d000700 1 --2- 172.21.15.63:0/450314538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 0x55ee8c40ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.118+0000 7f9b2d000700 1 -- 172.21.15.63:0/450314538 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8c418a80 con 0x55ee8c402820 2024-05-27T07:42:44.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.119+0000 7f9b2d000700 1 --2- 172.21.15.63:0/450314538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402820 0x55ee8c407d90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000b0a0 tx=0x7f9b2001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26654e4f705bfc2a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:44.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.119+0000 7f9b1cc00700 1 -- 172.21.15.63:0/450314538 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20023070 con 0x55ee8c402820 2024-05-27T07:42:44.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.119+0000 7f9b1cc00700 1 -- 172.21.15.63:0/450314538 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b20004c50 con 0x55ee8c402820 2024-05-27T07:42:44.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.119+0000 7f9b1cc00700 1 -- 172.21.15.63:0/450314538 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20005280 con 0x55ee8c402820 2024-05-27T07:42:44.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.120+0000 7f9b3dabab80 1 -- 172.21.15.63:0/450314538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402820 msgr2=0x55ee8c407d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:44.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.120+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/450314538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402820 0x55ee8c407d90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000b0a0 tx=0x7f9b2001f000 comp rx=0 tx=0).stop 2024-05-27T07:42:44.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.120+0000 7f9b3dabab80 1 -- 172.21.15.63:0/450314538 shutdown_connections 2024-05-27T07:42:44.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.120+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/450314538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c402820 0x55ee8c407d90 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.120+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/450314538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c283e40 0x55ee8c40b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.120+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/450314538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 0x55ee8c40ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.120+0000 7f9b3dabab80 1 -- 172.21.15.63:0/450314538 >> 172.21.15.63:0/450314538 conn(0x55ee8c2e3fc0 msgr2=0x55ee8c3fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:44.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.121+0000 7f9b3dabab80 1 -- 172.21.15.63:0/450314538 shutdown_connections 2024-05-27T07:42:44.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.121+0000 7f9b3dabab80 1 -- 172.21.15.63:0/450314538 wait complete. 2024-05-27T07:42:44.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.121+0000 7f9b3dabab80 1 -- start start 2024-05-27T07:42:44.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.122+0000 7f9b3dabab80 1 Processor -- start 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.122+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c283e40 0x55ee8c47f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.122+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 0x55ee8c47fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.122+0000 7f9b3dabab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c402820 0x55ee8c410210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.122+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b281b0c10 con 0x55ee8c283e40 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.122+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b281b2080 con 0x55ee8c402410 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.122+0000 7f9b3dabab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9b281b12e0 con 0x55ee8c402820 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.122+0000 7f9b3dabab80 10 client.0 _ll_register_callbacks cb 0x55ee8c4192b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b27e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c283e40 0x55ee8c47f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b27e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c283e40 0x55ee8c47f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47706/0 (socket says 172.21.15.63:47706) 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b27e00700 1 -- 172.21.15.63:0/3858106684 learned_addr learned my addr 172.21.15.63:0/3858106684 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:44.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b27400700 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 0x55ee8c47fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b2d000700 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c402820 0x55ee8c410210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b27e00700 1 -- 172.21.15.63:0/3858106684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c402820 msgr2=0x55ee8c410210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:44.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b27e00700 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c402820 0x55ee8c410210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b27e00700 1 -- 172.21.15.63:0/3858106684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 msgr2=0x55ee8c47fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:44.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b27e00700 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 0x55ee8c47fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b27e00700 1 -- 172.21.15.63:0/3858106684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8c410880 con 0x55ee8c283e40 2024-05-27T07:42:44.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.123+0000 7f9b27400700 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 0x55ee8c47fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:44.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.124+0000 7f9b27e00700 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c283e40 0x55ee8c47f6d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000c00 tx=0x7f9b180092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5bbbc8ef9e7c91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:44.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.124+0000 7f9b12000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:44.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.124+0000 7f9b12000700 1 -- 172.21.15.63:0/3858106684 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1802e070 con 0x55ee8c283e40 2024-05-27T07:42:44.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.124+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3858106684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b20004c50 con 0x55ee8c283e40 2024-05-27T07:42:44.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.125+0000 7f9b12000700 1 -- 172.21.15.63:0/3858106684 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f9b18005430 con 0x55ee8c283e40 2024-05-27T07:42:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.125+0000 7f9b12000700 1 -- 172.21.15.63:0/3858106684 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9b1803a050 con 0x55ee8c283e40 2024-05-27T07:42:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.125+0000 7f9b3dabab80 10 client.17178 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.125+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3858106684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ee8c486d60 con 0x55ee8c283e40 2024-05-27T07:42:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.125+0000 7f9b12000700 1 -- 172.21.15.63:0/3858106684 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b20004c50 con 0x55ee8c283e40 2024-05-27T07:42:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.126+0000 7f9b12000700 1 -- 172.21.15.63:0/3858106684 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9b1802a4b0 con 0x55ee8c283e40 2024-05-27T07:42:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.126+0000 7f9b3dabab80 10 client.17178 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.126+0000 7f9b3dabab80 10 client.17178 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:42:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.126+0000 7f9b3dabab80 20 client.17178 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.126+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55ee8c4871f0 0x55ee8c489610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.126+0000 7f9b3dabab80 4 client.17178 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:42:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.126+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3858106684 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee8c32c570 con 0x55ee8c4871f0 2024-05-27T07:42:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.127+0000 7f9b27400700 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55ee8c4871f0 0x55ee8c489610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:44.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.128+0000 7f9b27400700 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55ee8c4871f0 0x55ee8c489610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3667e51c64468ad8 server_cookie=370a825949be3621 in_seq=0 out_seq=0 2024-05-27T07:42:44.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.128+0000 7f9b12000700 10 client.17178 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:42:44.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b12000700 1 -- 172.21.15.63:0/3858106684 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee8c32c570 con 0x55ee8c4871f0 2024-05-27T07:42:44.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b12000700 10 client.17178 handle_command_reply: tid=0 2024-05-27T07:42:44.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 client.17178 shutdown 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3858106684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c283e40 msgr2=0x55ee8c47f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c283e40 0x55ee8c47f6d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000c00 tx=0x7f9b180092e0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3858106684 shutdown_connections 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee8c402820 0x55ee8c410210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55ee8c4871f0 0x55ee8c489610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee8c283e40 0x55ee8c47f6d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 --2- 172.21.15.63:0/3858106684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee8c402410 0x55ee8c47fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b12000700 0 client.17178 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3858106684 >> 172.21.15.63:0/3858106684 conn(0x55ee8c2e3fc0 msgr2=0x55ee8c417a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3858106684 shutdown_connections 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.129+0000 7f9b3dabab80 1 -- 172.21.15.63:0/3858106684 wait complete. 2024-05-27T07:42:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.130+0000 7f9b3dabab80 20 client.17178 trim_cache size 0 max 16384 2024-05-27T07:42:44.130 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:42:44.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.130+0000 7f9b2da00700 1 -- 172.21.15.63:0/1658824923 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b0c0b29f0 msgr2=0x7f9b0c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:44.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.130+0000 7f9b2da00700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b0c0b29f0 0x7f9b0c0b4f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009900 tx=0x7f9b200060a0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.130+0000 7f9b2da00700 1 -- 172.21.15.63:0/1658824923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28105bd0 msgr2=0x7f9b281a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.130+0000 7f9b2da00700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28105bd0 0x7f9b281a2e20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400e9a0 tx=0x7f9b1400ecb0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.131+0000 7f9b2da00700 1 -- 172.21.15.63:0/1658824923 shutdown_connections 2024-05-27T07:42:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.131+0000 7f9b2da00700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b0c0b29f0 0x7f9b0c0b4f00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.131+0000 7f9b2da00700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2810e6f0 0x7f9b2819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.131+0000 7f9b2da00700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b28105280 0x7f9b281a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.131+0000 7f9b2da00700 1 --2- 172.21.15.63:0/1658824923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b28105bd0 0x7f9b281a2e20 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.131+0000 7f9b2da00700 1 -- 172.21.15.63:0/1658824923 >> 172.21.15.63:0/1658824923 conn(0x7f9b28078ab0 msgr2=0x7f9b280ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.131+0000 7f9b2da00700 1 -- 172.21.15.63:0/1658824923 shutdown_connections 2024-05-27T07:42:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:44.131+0000 7f9b2da00700 1 -- 172.21.15.63:0/1658824923 wait complete. 2024-05-27T07:42:44.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:44 smithi086 ceph-mon[34687]: pgmap v284: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:42:44.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:44 smithi149 ceph-mon[35408]: pgmap v284: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:42:44.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:44 smithi063 ceph-mon[31613]: pgmap v284: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:42:45.111 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:42:45.596 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:46.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:46 smithi086 ceph-mon[34687]: pgmap v285: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:42:46.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:46 smithi149 ceph-mon[35408]: pgmap v285: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:42:46.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:46 smithi063 ceph-mon[31613]: pgmap v285: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T07:42:48.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.086+0000 7fb862a00700 1 -- 172.21.15.63:0/189830158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c103a40 msgr2=0x7fb85c110760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:48.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.086+0000 7fb862a00700 1 --2- 172.21.15.63:0/189830158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c103a40 0x7fb85c110760 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb848009990 tx=0x7fb848009ca0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.086+0000 7fb862a00700 1 -- 172.21.15.63:0/189830158 shutdown_connections 2024-05-27T07:42:48.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.086+0000 7fb862a00700 1 --2- 172.21.15.63:0/189830158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85c110ca0 0x7fb85c1130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.086+0000 7fb862a00700 1 --2- 172.21.15.63:0/189830158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb85c1030f0 0x7fb85c103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.086+0000 7fb862a00700 1 --2- 172.21.15.63:0/189830158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c103a40 0x7fb85c110760 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.086+0000 7fb862a00700 1 -- 172.21.15.63:0/189830158 >> 172.21.15.63:0/189830158 conn(0x7fb85c0fedb0 msgr2=0x7fb85c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:48.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.086+0000 7fb862a00700 1 -- 172.21.15.63:0/189830158 shutdown_connections 2024-05-27T07:42:48.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.086+0000 7fb862a00700 1 -- 172.21.15.63:0/189830158 wait complete. 2024-05-27T07:42:48.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.088+0000 7fb862a00700 1 Processor -- start 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.088+0000 7fb862a00700 1 -- start start 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.088+0000 7fb862a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c1030f0 0x7fb85c19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.088+0000 7fb862a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb85c103a40 0x7fb85c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.089+0000 7fb862a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85c110ca0 0x7fb85c1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.089+0000 7fb862a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c116550 con 0x7fb85c1030f0 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.089+0000 7fb862a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1163d0 con 0x7fb85c103a40 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.089+0000 7fb862a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1166d0 con 0x7fb85c110ca0 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.089+0000 7fb85be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c1030f0 0x7fb85c19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.089+0000 7fb85be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c1030f0 0x7fb85c19cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47726/0 (socket says 172.21.15.63:47726) 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.089+0000 7fb85be00700 1 -- 172.21.15.63:0/2403999620 learned_addr learned my addr 172.21.15.63:0/2403999620 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.089+0000 7fb85b400700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb85c103a40 0x7fb85c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.089+0000 7fb860a00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85c110ca0 0x7fb85c1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.090+0000 7fb85be00700 1 -- 172.21.15.63:0/2403999620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85c110ca0 msgr2=0x7fb85c1a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.090+0000 7fb85be00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85c110ca0 0x7fb85c1a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.090+0000 7fb85be00700 1 -- 172.21.15.63:0/2403999620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb85c103a40 msgr2=0x7fb85c19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.090+0000 7fb85be00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb85c103a40 0x7fb85c19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.090+0000 7fb85be00700 1 -- 172.21.15.63:0/2403999620 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85c1a1e70 con 0x7fb85c1030f0 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.090+0000 7fb85b400700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb85c103a40 0x7fb85c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.090+0000 7fb85be00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c1030f0 0x7fb85c19cfa0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00d950 tx=0x7fb84c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.090+0000 7fb858c00700 1 -- 172.21.15.63:0/2403999620 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c016070 con 0x7fb85c1030f0 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.090+0000 7fb858c00700 1 -- 172.21.15.63:0/2403999620 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb84c010460 con 0x7fb85c1030f0 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.090+0000 7fb862a00700 1 -- 172.21.15.63:0/2403999620 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85c1a2160 con 0x7fb85c1030f0 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.091+0000 7fb858c00700 1 -- 172.21.15.63:0/2403999620 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c00f620 con 0x7fb85c1030f0 2024-05-27T07:42:48.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.091+0000 7fb862a00700 1 -- 172.21.15.63:0/2403999620 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85c1a9a80 con 0x7fb85c1030f0 2024-05-27T07:42:48.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.095+0000 7fb858c00700 1 -- 172.21.15.63:0/2403999620 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb84c019070 con 0x7fb85c1030f0 2024-05-27T07:42:48.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.097+0000 7fb871619b80 1 Processor -- start 2024-05-27T07:42:48.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.097+0000 7fb871619b80 1 -- start start 2024-05-27T07:42:48.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.097+0000 7fb858c00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8400b29f0 0x7fb8400b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.097+0000 7fb871619b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2539f0 0x564a9e2e6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.097+0000 7fb871619b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e167e40 0x564a9e2e6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.097+0000 7fb858c00700 1 -- 172.21.15.63:0/2403999620 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fb84c0e2680 con 0x7fb85c1030f0 2024-05-27T07:42:48.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.097+0000 7fb871619b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2e6e70 0x564a9e2f3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.098+0000 7fb871619b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564a9e214e40 con 0x564a9e2e6e70 2024-05-27T07:42:48.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.098+0000 7fb871619b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a9e2049e0 con 0x564a9e2539f0 2024-05-27T07:42:48.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.098+0000 7fb871619b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564a9e208a20 con 0x564a9e167e40 2024-05-27T07:42:48.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.098+0000 7fb860a00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8400b29f0 0x7fb8400b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.098+0000 7fb860a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2e6e70 0x564a9e2f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.098+0000 7fb860a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2e6e70 0x564a9e2f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47732/0 (socket says 172.21.15.63:47732) 2024-05-27T07:42:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.098+0000 7fb860a00700 1 -- 172.21.15.63:0/1739671166 learned_addr learned my addr 172.21.15.63:0/1739671166 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.098+0000 7fb85b400700 1 --2- 172.21.15.63:0/1739671166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e167e40 0x564a9e2e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.098+0000 7fb85be00700 1 --2- 172.21.15.63:0/1739671166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2539f0 0x564a9e2e6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.099+0000 7fb860a00700 1 -- 172.21.15.63:0/1739671166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e167e40 msgr2=0x564a9e2e6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:48.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.099+0000 7fb860a00700 1 --2- 172.21.15.63:0/1739671166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e167e40 0x564a9e2e6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.099+0000 7fb860a00700 1 -- 172.21.15.63:0/1739671166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2539f0 msgr2=0x564a9e2e6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.099+0000 7fb860a00700 1 --2- 172.21.15.63:0/1739671166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2539f0 0x564a9e2e6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.099+0000 7fb860a00700 1 -- 172.21.15.63:0/1739671166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a9e1cd420 con 0x564a9e2e6e70 2024-05-27T07:42:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.099+0000 7fb860a00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8400b29f0 0x7fb8400b4f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb854009710 tx=0x7fb854009c70 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.099+0000 7fb860a00700 1 --2- 172.21.15.63:0/1739671166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2e6e70 0x564a9e2f3110 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb854018ac0 tx=0x7fb85401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405d30654ab4f4d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:48.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.100+0000 7fb851600700 1 -- 172.21.15.63:0/1739671166 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854004bb0 con 0x564a9e2e6e70 2024-05-27T07:42:48.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.100+0000 7fb851600700 1 -- 172.21.15.63:0/1739671166 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb85401e070 con 0x564a9e2e6e70 2024-05-27T07:42:48.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.100+0000 7fb851600700 1 -- 172.21.15.63:0/1739671166 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854022460 con 0x564a9e2e6e70 2024-05-27T07:42:48.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.101+0000 7fb871619b80 1 -- 172.21.15.63:0/1739671166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2e6e70 msgr2=0x564a9e2f3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:48.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.101+0000 7fb871619b80 1 --2- 172.21.15.63:0/1739671166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2e6e70 0x564a9e2f3110 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb854018ac0 tx=0x7fb85401b000 comp rx=0 tx=0).stop 2024-05-27T07:42:48.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.101+0000 7fb871619b80 1 -- 172.21.15.63:0/1739671166 shutdown_connections 2024-05-27T07:42:48.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.101+0000 7fb871619b80 1 --2- 172.21.15.63:0/1739671166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2e6e70 0x564a9e2f3110 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.101+0000 7fb871619b80 1 --2- 172.21.15.63:0/1739671166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2539f0 0x564a9e2e6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.101+0000 7fb871619b80 1 --2- 172.21.15.63:0/1739671166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e167e40 0x564a9e2e6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.101+0000 7fb871619b80 1 -- 172.21.15.63:0/1739671166 >> 172.21.15.63:0/1739671166 conn(0x564a9e201390 msgr2=0x564a9e2e24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:48.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.101+0000 7fb871619b80 1 -- 172.21.15.63:0/1739671166 shutdown_connections 2024-05-27T07:42:48.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.101+0000 7fb871619b80 1 -- 172.21.15.63:0/1739671166 wait complete. 2024-05-27T07:42:48.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.102+0000 7fb871619b80 1 -- start start 2024-05-27T07:42:48.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.102+0000 7fb871619b80 1 Processor -- start 2024-05-27T07:42:48.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb871619b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb871619b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e2539f0 0x564a9e2f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb871619b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2e6e70 0x564a9e2f1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb871619b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564a9e10afa0 con 0x564a9e167e40 2024-05-27T07:42:48.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb871619b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a9e1a5760 con 0x564a9e2e6e70 2024-05-27T07:42:48.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb871619b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564a9e2f3b20 con 0x564a9e2539f0 2024-05-27T07:42:48.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb871619b80 10 client.0 _ll_register_callbacks cb 0x564a9e1aec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:48.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb85be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb85be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2f05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47738/0 (socket says 172.21.15.63:47738) 2024-05-27T07:42:48.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb85be00700 1 -- 172.21.15.63:0/2883610789 learned_addr learned my addr 172.21.15.63:0/2883610789 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:48.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb860a00700 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2e6e70 0x564a9e2f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.103+0000 7fb85b400700 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e2539f0 0x564a9e2f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.104+0000 7fb85be00700 1 -- 172.21.15.63:0/2883610789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e2539f0 msgr2=0x564a9e2f0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:48.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.104+0000 7fb85be00700 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e2539f0 0x564a9e2f0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.104+0000 7fb85be00700 1 -- 172.21.15.63:0/2883610789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2e6e70 msgr2=0x564a9e2f1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:48.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.104+0000 7fb85be00700 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2e6e70 0x564a9e2f1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.104+0000 7fb85be00700 1 -- 172.21.15.63:0/2883610789 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a9e1548d0 con 0x564a9e167e40 2024-05-27T07:42:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.104+0000 7fb85be00700 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2f05d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0a55c0 tx=0x7fb84c010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d043b5d9d9a48522 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.104+0000 7fb846000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.105+0000 7fb846000700 1 -- 172.21.15.63:0/2883610789 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0c3060 con 0x564a9e167e40 2024-05-27T07:42:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.105+0000 7fb871619b80 1 -- 172.21.15.63:0/2883610789 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85401e070 con 0x564a9e167e40 2024-05-27T07:42:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.105+0000 7fb846000700 1 -- 172.21.15.63:0/2883610789 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fb84c0a97a0 con 0x564a9e167e40 2024-05-27T07:42:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.105+0000 7fb846000700 1 -- 172.21.15.63:0/2883610789 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb84c020050 con 0x564a9e167e40 2024-05-27T07:42:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.105+0000 7fb871619b80 10 client.17190 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:48.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.105+0000 7fb871619b80 1 -- 172.21.15.63:0/2883610789 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a9e1d5060 con 0x564a9e167e40 2024-05-27T07:42:48.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.106+0000 7fb846000700 1 -- 172.21.15.63:0/2883610789 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb85401e070 con 0x564a9e167e40 2024-05-27T07:42:48.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.106+0000 7fb846000700 1 -- 172.21.15.63:0/2883610789 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb84c0e21e0 con 0x564a9e167e40 2024-05-27T07:42:48.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.106+0000 7fb871619b80 10 client.17190 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:48.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.106+0000 7fb871619b80 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:48.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.106+0000 7fb871619b80 20 client.17190 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:48.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.106+0000 7fb871619b80 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9e300990 0x564a9e302db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.106+0000 7fb871619b80 4 client.17190 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:42:48.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.106+0000 7fb871619b80 1 -- 172.21.15.63:0/2883610789 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a9e216d90 con 0x564a9e300990 2024-05-27T07:42:48.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.107+0000 7fb85b400700 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9e300990 0x564a9e302db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.108+0000 7fb846000700 10 client.17190 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:48.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.108+0000 7fb85b400700 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9e300990 0x564a9e302db0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc749544e218e791 server_cookie=e78cdc00834f994b in_seq=0 out_seq=0 2024-05-27T07:42:48.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.110+0000 7fb846000700 1 -- 172.21.15.63:0/2883610789 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564a9e216d90 con 0x564a9e300990 2024-05-27T07:42:48.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.110+0000 7fb846000700 10 client.17190 handle_command_reply: tid=0 2024-05-27T07:42:48.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.110+0000 7fb871619b80 1 client.17190 shutdown 2024-05-27T07:42:48.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 1 -- 172.21.15.63:0/2883610789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 msgr2=0x564a9e2f05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:48.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2f05d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0a55c0 tx=0x7fb84c010820 comp rx=0 tx=0).stop 2024-05-27T07:42:48.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 1 -- 172.21.15.63:0/2883610789 shutdown_connections 2024-05-27T07:42:48.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9e300990 0x564a9e302db0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb846000700 0 client.17190 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:48.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2e6e70 0x564a9e2f1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2f05d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 1 --2- 172.21.15.63:0/2883610789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e2539f0 0x564a9e2f0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 1 -- 172.21.15.63:0/2883610789 >> 172.21.15.63:0/2883610789 conn(0x564a9e201390 msgr2=0x564a9e2e63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:48.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 1 -- 172.21.15.63:0/2883610789 shutdown_connections 2024-05-27T07:42:48.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 1 -- 172.21.15.63:0/2883610789 wait complete. 2024-05-27T07:42:48.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.111+0000 7fb871619b80 20 client.17190 trim_cache size 0 max 16384 2024-05-27T07:42:48.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.134+0000 7fb871619b80 1 Processor -- start 2024-05-27T07:42:48.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.134+0000 7fb871619b80 1 -- start start 2024-05-27T07:42:48.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.134+0000 7fb871619b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2ebde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.134+0000 7fb871619b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2539f0 0x564a9e2e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.134+0000 7fb871619b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e2e6e70 0x564a9e2fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.134+0000 7fb871619b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1b50e0 con 0x564a9e167e40 2024-05-27T07:42:48.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.134+0000 7fb871619b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1b2970 con 0x564a9e2539f0 2024-05-27T07:42:48.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.134+0000 7fb871619b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1b07e0 con 0x564a9e2e6e70 2024-05-27T07:42:48.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.135+0000 7fb85be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2ebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.135+0000 7fb85be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2ebde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47754/0 (socket says 172.21.15.63:47754) 2024-05-27T07:42:48.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.135+0000 7fb85be00700 1 -- 172.21.15.63:0/1709844378 learned_addr learned my addr 172.21.15.63:0/1709844378 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:48.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.135+0000 7fb85b400700 1 --2- 172.21.15.63:0/1709844378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2539f0 0x564a9e2e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.135+0000 7fb85be00700 1 -- 172.21.15.63:0/1709844378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e2e6e70 msgr2=0x564a9e2fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:48.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.135+0000 7fb85be00700 1 --2- 172.21.15.63:0/1709844378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e2e6e70 0x564a9e2fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.135+0000 7fb85be00700 1 -- 172.21.15.63:0/1709844378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2539f0 msgr2=0x564a9e2e25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:48.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.135+0000 7fb85be00700 1 --2- 172.21.15.63:0/1709844378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2539f0 0x564a9e2e25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.135+0000 7fb85be00700 1 -- 172.21.15.63:0/1709844378 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a9e2e7a50 con 0x564a9e167e40 2024-05-27T07:42:48.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.136+0000 7fb85be00700 1 --2- 172.21.15.63:0/1709844378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2ebde0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb84c005950 tx=0x7fb84c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7071d6e2c3a4a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:48.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.136+0000 7fb850c00700 1 -- 172.21.15.63:0/1709844378 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c06d070 con 0x564a9e167e40 2024-05-27T07:42:48.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.136+0000 7fb850c00700 1 -- 172.21.15.63:0/1709844378 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb84c0b23a0 con 0x564a9e167e40 2024-05-27T07:42:48.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.136+0000 7fb850c00700 1 -- 172.21.15.63:0/1709844378 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0c3070 con 0x564a9e167e40 2024-05-27T07:42:48.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.137+0000 7fb871619b80 1 -- 172.21.15.63:0/1709844378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 msgr2=0x564a9e2ebde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:48.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.137+0000 7fb871619b80 1 --2- 172.21.15.63:0/1709844378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2ebde0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb84c005950 tx=0x7fb84c000bc0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.137+0000 7fb871619b80 1 -- 172.21.15.63:0/1709844378 shutdown_connections 2024-05-27T07:42:48.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.137+0000 7fb871619b80 1 --2- 172.21.15.63:0/1709844378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e2e6e70 0x564a9e2fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.137+0000 7fb871619b80 1 --2- 172.21.15.63:0/1709844378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e167e40 0x564a9e2ebde0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.137+0000 7fb871619b80 1 --2- 172.21.15.63:0/1709844378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2539f0 0x564a9e2e25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.137+0000 7fb871619b80 1 -- 172.21.15.63:0/1709844378 >> 172.21.15.63:0/1709844378 conn(0x564a9e201390 msgr2=0x564a9e2eeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:48.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.137+0000 7fb871619b80 1 -- 172.21.15.63:0/1709844378 shutdown_connections 2024-05-27T07:42:48.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.137+0000 7fb871619b80 1 -- 172.21.15.63:0/1709844378 wait complete. 2024-05-27T07:42:48.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.138+0000 7fb871619b80 1 -- start start 2024-05-27T07:42:48.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.138+0000 7fb871619b80 1 Processor -- start 2024-05-27T07:42:48.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.139+0000 7fb871619b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e167e40 0x564a9e2f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.139+0000 7fb871619b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2539f0 0x564a9e2f4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.139+0000 7fb871619b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2e6e70 0x564a9e2f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.139+0000 7fb871619b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1b0e30 con 0x564a9e2539f0 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.139+0000 7fb871619b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1b22a0 con 0x564a9e2e6e70 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.139+0000 7fb871619b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1b1500 con 0x564a9e167e40 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.139+0000 7fb871619b80 10 client.0 _ll_register_callbacks cb 0x564a9e1aec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.139+0000 7fb85b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2539f0 0x564a9e2f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.139+0000 7fb85b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2539f0 0x564a9e2f4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47756/0 (socket says 172.21.15.63:47756) 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.139+0000 7fb85b400700 1 -- 172.21.15.63:0/1306581138 learned_addr learned my addr 172.21.15.63:0/1306581138 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.140+0000 7fb85be00700 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e167e40 0x564a9e2f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.140+0000 7fb860a00700 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2e6e70 0x564a9e2f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.140+0000 7fb85b400700 1 -- 172.21.15.63:0/1306581138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e167e40 msgr2=0x564a9e2f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:48.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.140+0000 7fb85b400700 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e167e40 0x564a9e2f93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.140+0000 7fb85b400700 1 -- 172.21.15.63:0/1306581138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2e6e70 msgr2=0x564a9e2f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:48.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.140+0000 7fb85b400700 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2e6e70 0x564a9e2f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.140+0000 7fb85b400700 1 -- 172.21.15.63:0/1306581138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a9e2f5300 con 0x564a9e2539f0 2024-05-27T07:42:48.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.140+0000 7fb85b400700 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2539f0 0x564a9e2f4720 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb848009750 tx=0x7fb8480053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c6877ebabb88c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:48.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.140+0000 7fb846000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:48.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.141+0000 7fb846000700 1 -- 172.21.15.63:0/1306581138 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848005690 con 0x564a9e2539f0 2024-05-27T07:42:48.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.141+0000 7fb846000700 1 -- 172.21.15.63:0/1306581138 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fb8480039d0 con 0x564a9e2539f0 2024-05-27T07:42:48.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.141+0000 7fb871619b80 1 -- 172.21.15.63:0/1306581138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb84c0b23a0 con 0x564a9e2539f0 2024-05-27T07:42:48.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.141+0000 7fb846000700 1 -- 172.21.15.63:0/1306581138 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb848036050 con 0x564a9e2539f0 2024-05-27T07:42:48.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.141+0000 7fb871619b80 10 client.17196 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:48.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.141+0000 7fb871619b80 1 -- 172.21.15.63:0/1306581138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a9e36b260 con 0x564a9e2539f0 2024-05-27T07:42:48.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.142+0000 7fb846000700 1 -- 172.21.15.63:0/1306581138 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb84c0b23a0 con 0x564a9e2539f0 2024-05-27T07:42:48.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.142+0000 7fb846000700 1 -- 172.21.15.63:0/1306581138 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb84800f460 con 0x564a9e2539f0 2024-05-27T07:42:48.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.142+0000 7fb871619b80 10 client.17196 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:48.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.142+0000 7fb871619b80 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:48.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.142+0000 7fb871619b80 20 client.17196 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:48.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.142+0000 7fb871619b80 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9e36b9a0 0x564a9e36ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:48.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.142+0000 7fb871619b80 4 client.17196 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:42:48.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.142+0000 7fb871619b80 1 -- 172.21.15.63:0/1306581138 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a9e210570 con 0x564a9e36b9a0 2024-05-27T07:42:48.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.143+0000 7fb85be00700 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9e36b9a0 0x564a9e36ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:48.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.144+0000 7fb846000700 10 client.17196 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:48.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.144+0000 7fb85be00700 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9e36b9a0 0x564a9e36ddc0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc3913ed1db5196a server_cookie=617a722955d1a715 in_seq=0 out_seq=0 2024-05-27T07:42:48.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.144+0000 7fb846000700 1 -- 172.21.15.63:0/1306581138 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a9e210570 con 0x564a9e36b9a0 2024-05-27T07:42:48.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.144+0000 7fb846000700 10 client.17196 handle_command_reply: tid=0 2024-05-27T07:42:48.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.144+0000 7fb871619b80 1 client.17196 shutdown 2024-05-27T07:42:48.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.144+0000 7fb871619b80 1 -- 172.21.15.63:0/1306581138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2539f0 msgr2=0x564a9e2f4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:48.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.144+0000 7fb871619b80 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2539f0 0x564a9e2f4720 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb848009750 tx=0x7fb8480053b0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.144+0000 7fb871619b80 1 -- 172.21.15.63:0/1306581138 shutdown_connections 2024-05-27T07:42:48.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.145+0000 7fb871619b80 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9e36b9a0 0x564a9e36ddc0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.145+0000 7fb871619b80 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9e2e6e70 0x564a9e2f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.145+0000 7fb871619b80 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9e167e40 0x564a9e2f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.145+0000 7fb871619b80 1 --2- 172.21.15.63:0/1306581138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9e2539f0 0x564a9e2f4720 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.145+0000 7fb846000700 0 client.17196 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:48.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.145+0000 7fb871619b80 1 -- 172.21.15.63:0/1306581138 >> 172.21.15.63:0/1306581138 conn(0x564a9e201390 msgr2=0x564a9e2ef650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:48.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.145+0000 7fb871619b80 1 -- 172.21.15.63:0/1306581138 shutdown_connections 2024-05-27T07:42:48.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.145+0000 7fb871619b80 1 -- 172.21.15.63:0/1306581138 wait complete. 2024-05-27T07:42:48.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.145+0000 7fb871619b80 20 client.17196 trim_cache size 0 max 16384 2024-05-27T07:42:48.146 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:42:48.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 -- 172.21.15.63:0/2403999620 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8400b29f0 msgr2=0x7fb8400b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:48.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8400b29f0 0x7fb8400b4f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb854009710 tx=0x7fb854009c70 comp rx=0 tx=0).stop 2024-05-27T07:42:48.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 -- 172.21.15.63:0/2403999620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c1030f0 msgr2=0x7fb85c19cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:48.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c1030f0 0x7fb85c19cfa0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00d950 tx=0x7fb84c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:42:48.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 -- 172.21.15.63:0/2403999620 shutdown_connections 2024-05-27T07:42:48.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8400b29f0 0x7fb8400b4f00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb85c110ca0 0x7fb85c1a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c1030f0 0x7fb85c19cfa0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 --2- 172.21.15.63:0/2403999620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb85c103a40 0x7fb85c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:48.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 -- 172.21.15.63:0/2403999620 >> 172.21.15.63:0/2403999620 conn(0x7fb85c0fedb0 msgr2=0x7fb85c112920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:48.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.146+0000 7fb862a00700 1 -- 172.21.15.63:0/2403999620 shutdown_connections 2024-05-27T07:42:48.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:48.147+0000 7fb862a00700 1 -- 172.21.15.63:0/2403999620 wait complete. 2024-05-27T07:42:48.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:48 smithi086 ceph-mon[34687]: pgmap v286: 129 pgs: 129 active+clean; 2.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.7 MiB/s wr, 15 op/s 2024-05-27T07:42:48.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:48 smithi149 ceph-mon[35408]: pgmap v286: 129 pgs: 129 active+clean; 2.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.7 MiB/s wr, 15 op/s 2024-05-27T07:42:48.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:48 smithi063 ceph-mon[31613]: pgmap v286: 129 pgs: 129 active+clean; 2.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.7 MiB/s wr, 15 op/s 2024-05-27T07:42:50.175 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:42:50.175 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:42:50.662 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:50.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:50 smithi086 ceph-mon[34687]: pgmap v287: 129 pgs: 129 active+clean; 2.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:42:50.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:50 smithi149 ceph-mon[35408]: pgmap v287: 129 pgs: 129 active+clean; 2.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:42:50.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:50 smithi063 ceph-mon[31613]: pgmap v287: 129 pgs: 129 active+clean; 2.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:42:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.173+0000 7f249a400640 1 -- 172.21.15.63:0/3969106199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494109f50 msgr2=0x7f2494103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.173+0000 7f249a400640 1 --2- 172.21.15.63:0/3969106199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494109f50 0x7f2494103be0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2480009900 tx=0x7f248002a3f0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.174+0000 7f249a400640 1 -- 172.21.15.63:0/3969106199 shutdown_connections 2024-05-27T07:42:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.174+0000 7f249a400640 1 --2- 172.21.15.63:0/3969106199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2494104250 0x7f24941105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.174+0000 7f249a400640 1 --2- 172.21.15.63:0/3969106199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494109f50 0x7f2494103be0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.174+0000 7f249a400640 1 --2- 172.21.15.63:0/3969106199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2494109ab0 0x7f24941036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.174+0000 7f249a400640 1 -- 172.21.15.63:0/3969106199 >> 172.21.15.63:0/3969106199 conn(0x7f2494079240 msgr2=0x7f2494079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.174+0000 7f249a400640 1 -- 172.21.15.63:0/3969106199 shutdown_connections 2024-05-27T07:42:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.174+0000 7f249a400640 1 -- 172.21.15.63:0/3969106199 wait complete. 2024-05-27T07:42:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.175+0000 7f249a400640 1 Processor -- start 2024-05-27T07:42:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.175+0000 7f249a400640 1 -- start start 2024-05-27T07:42:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f249a400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494104250 0x7f249419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f249a400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2494109ab0 0x7f249419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f249a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2494109f50 0x7f24941a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f249a400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2494113ba0 con 0x7f2494104250 2024-05-27T07:42:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f249a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2494113a20 con 0x7f2494109f50 2024-05-27T07:42:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f249a400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2494113d20 con 0x7f2494109ab0 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f2499000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494104250 0x7f249419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f2499000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494104250 0x7f249419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39658/0 (socket says 172.21.15.63:39658) 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f2499000640 1 -- 172.21.15.63:0/2951947499 learned_addr learned my addr 172.21.15.63:0/2951947499 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f2493e00640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2494109ab0 0x7f249419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.176+0000 7f2499a00640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2494109f50 0x7f24941a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.177+0000 7f2499000640 1 -- 172.21.15.63:0/2951947499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2494109ab0 msgr2=0x7f249419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.177+0000 7f2499000640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2494109ab0 0x7f249419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.177+0000 7f2499000640 1 -- 172.21.15.63:0/2951947499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2494109f50 msgr2=0x7f24941a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.177+0000 7f2499000640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2494109f50 0x7f24941a84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.177+0000 7f2499000640 1 -- 172.21.15.63:0/2951947499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24941a8a20 con 0x7f2494104250 2024-05-27T07:42:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.177+0000 7f2493e00640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2494109ab0 0x7f249419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.177+0000 7f2499000640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494104250 0x7f249419d640 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f248800d770 tx=0x7f248800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.177+0000 7f2491600640 1 -- 172.21.15.63:0/2951947499 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488017070 con 0x7f2494104250 2024-05-27T07:42:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.177+0000 7f2491600640 1 -- 172.21.15.63:0/2951947499 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2488009c30 con 0x7f2494104250 2024-05-27T07:42:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.178+0000 7f249a400640 1 -- 172.21.15.63:0/2951947499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24941a8cb0 con 0x7f2494104250 2024-05-27T07:42:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.178+0000 7f2491600640 1 -- 172.21.15.63:0/2951947499 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248800f620 con 0x7f2494104250 2024-05-27T07:42:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.178+0000 7f249a400640 1 -- 172.21.15.63:0/2951947499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941a91d0 con 0x7f2494104250 2024-05-27T07:42:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.181+0000 7f24a29d4740 1 Processor -- start 2024-05-27T07:42:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.182+0000 7f24a29d4740 1 -- start start 2024-05-27T07:42:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.182+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647205031b0 0x5647208a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.182+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204da2f0 0x5647208a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.182+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.182+0000 7f24a29d4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564720816b90 con 0x5647204e24d0 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.182+0000 7f24a29d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564720806730 con 0x5647204da2f0 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.182+0000 7f24a29d4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56472080a770 con 0x5647205031b0 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.183+0000 7f2493e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.183+0000 7f2493e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208a64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39674/0 (socket says 172.21.15.63:39674) 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.183+0000 7f2493e00640 1 -- 172.21.15.63:0/1214508117 learned_addr learned my addr 172.21.15.63:0/1214508117 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.183+0000 7f2491600640 1 -- 172.21.15.63:0/2951947499 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f248801a070 con 0x7f2494104250 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.183+0000 7f2499a00640 1 --2- 172.21.15.63:0/1214508117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647205031b0 0x5647208a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.183+0000 7f2493e00640 1 -- 172.21.15.63:0/1214508117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647205031b0 msgr2=0x5647208a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.183+0000 7f2493e00640 1 --2- 172.21.15.63:0/1214508117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647205031b0 0x5647208a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.183+0000 7f2493e00640 1 -- 172.21.15.63:0/1214508117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204da2f0 msgr2=0x5647208a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.183+0000 7f2493e00640 1 --2- 172.21.15.63:0/1214508117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204da2f0 0x5647208a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.183+0000 7f2493e00640 1 -- 172.21.15.63:0/1214508117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472079f8b0 con 0x5647204e24d0 2024-05-27T07:42:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.184+0000 7f2493e00640 1 --2- 172.21.15.63:0/1214508117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208a64e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f24800057c0 tx=0x7f2480004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41801cfc7ddee046 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.184+0000 7f2486000640 1 -- 172.21.15.63:0/1214508117 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480035030 con 0x5647204e24d0 2024-05-27T07:42:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.184+0000 7f2486000640 1 -- 172.21.15.63:0/1214508117 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2480005de0 con 0x5647204e24d0 2024-05-27T07:42:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.184+0000 7f2486000640 1 -- 172.21.15.63:0/1214508117 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24800035f0 con 0x5647204e24d0 2024-05-27T07:42:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.185+0000 7f2491600640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f247c0b2950 0x7f247c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.185+0000 7f2491600640 1 -- 172.21.15.63:0/2951947499 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f24880e2540 con 0x7f2494104250 2024-05-27T07:42:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.185+0000 7f24a29d4740 1 -- 172.21.15.63:0/1214508117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 msgr2=0x5647208a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.185+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1214508117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208a64e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f24800057c0 tx=0x7f2480004390 comp rx=0 tx=0).stop 2024-05-27T07:42:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.186+0000 7f2499a00640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f247c0b2950 0x7f247c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.186+0000 7f24a29d4740 1 -- 172.21.15.63:0/1214508117 shutdown_connections 2024-05-27T07:42:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.186+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1214508117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208a64e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.186+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1214508117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204da2f0 0x5647208a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.186+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1214508117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647205031b0 0x5647208a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.186+0000 7f24a29d4740 1 -- 172.21.15.63:0/1214508117 >> 172.21.15.63:0/1214508117 conn(0x5647204f7300 msgr2=0x5647208a2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.186+0000 7f24a29d4740 1 -- 172.21.15.63:0/1214508117 shutdown_connections 2024-05-27T07:42:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.186+0000 7f24a29d4740 1 -- 172.21.15.63:0/1214508117 wait complete. 2024-05-27T07:42:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.186+0000 7f2499a00640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f247c0b2950 0x7f247c0b4e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5647208a2bd0 tx=0x7f248c009580 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.187+0000 7f24a29d4740 1 -- start start 2024-05-27T07:42:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.187+0000 7f24a29d4740 1 Processor -- start 2024-05-27T07:42:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204da2f0 0x5647208afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647205031b0 0x5647208bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f24a29d4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5647204ec790 con 0x5647204e24d0 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f24a29d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5647207aebc0 con 0x5647204da2f0 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f24a29d4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5647208b2b30 con 0x5647205031b0 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f24a29d4740 10 client.0 _ll_register_callbacks cb 0x56472078c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f2493e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f2493e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39686/0 (socket says 172.21.15.63:39686) 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f2493e00640 1 -- 172.21.15.63:0/1118057689 learned_addr learned my addr 172.21.15.63:0/1118057689 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f2499000640 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204da2f0 0x5647208afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.188+0000 7f2499a00640 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647205031b0 0x5647208bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.189+0000 7f2493e00640 1 -- 172.21.15.63:0/1118057689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647205031b0 msgr2=0x5647208bbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.189+0000 7f2493e00640 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647205031b0 0x5647208bbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.189+0000 7f2493e00640 1 -- 172.21.15.63:0/1118057689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204da2f0 msgr2=0x5647208afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.189+0000 7f2493e00640 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204da2f0 0x5647208afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.189+0000 7f2493e00640 1 -- 172.21.15.63:0/1118057689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472051b210 con 0x5647204e24d0 2024-05-27T07:42:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.189+0000 7f2493e00640 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208b0170 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f248002a910 tx=0x7f2480004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9715e81ce21e55e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.189+0000 7f247aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.189+0000 7f247aa00640 1 -- 172.21.15.63:0/1118057689 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480035070 con 0x5647204e24d0 2024-05-27T07:42:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.190+0000 7f24a29d4740 1 -- 172.21.15.63:0/1118057689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564720818ae0 con 0x5647204e24d0 2024-05-27T07:42:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.190+0000 7f247aa00640 1 -- 172.21.15.63:0/1118057689 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f24800052f0 con 0x5647204e24d0 2024-05-27T07:42:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.190+0000 7f247aa00640 1 -- 172.21.15.63:0/1118057689 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f24800054f0 con 0x5647204e24d0 2024-05-27T07:42:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.190+0000 7f24a29d4740 10 client.17208 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.190+0000 7f24a29d4740 1 -- 172.21.15.63:0/1118057689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5647205e7620 con 0x5647204e24d0 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.190+0000 7f247aa00640 1 -- 172.21.15.63:0/1118057689 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x564720818ae0 con 0x5647204e24d0 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.190+0000 7f247aa00640 1 -- 172.21.15.63:0/1118057689 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f248004d540 con 0x5647204e24d0 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.191+0000 7f24a29d4740 10 client.17208 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.191+0000 7f24a29d4740 10 client.17208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.191+0000 7f24a29d4740 20 client.17208 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.191+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5647204c1d80 0x5647208c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.191+0000 7f24a29d4740 4 client.17208 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.191+0000 7f24a29d4740 1 -- 172.21.15.63:0/1118057689 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647208122c0 con 0x5647204c1d80 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.191+0000 7f2499000640 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5647204c1d80 0x5647208c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.192+0000 7f2499000640 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5647204c1d80 0x5647208c1c80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a112a0411af191 server_cookie=ef13ee974ede45aa in_seq=0 out_seq=0 2024-05-27T07:42:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.192+0000 7f247aa00640 10 client.17208 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.194+0000 7f247aa00640 1 -- 172.21.15.63:0/1118057689 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5647208122c0 con 0x5647204c1d80 2024-05-27T07:42:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.194+0000 7f247aa00640 10 client.17208 handle_command_reply: tid=0 2024-05-27T07:42:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.194+0000 7f24a29d4740 1 client.17208 shutdown 2024-05-27T07:42:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 1 -- 172.21.15.63:0/1118057689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 msgr2=0x5647208b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208b0170 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f248002a910 tx=0x7f2480004f70 comp rx=0 tx=0).stop 2024-05-27T07:42:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 1 -- 172.21.15.63:0/1118057689 shutdown_connections 2024-05-27T07:42:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5647204c1d80 0x5647208c1c80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647205031b0 0x5647208bbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208b0170 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1118057689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204da2f0 0x5647208afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 1 -- 172.21.15.63:0/1118057689 >> 172.21.15.63:0/1118057689 conn(0x5647204f7300 msgr2=0x5647208a7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f247aa00640 0 client.17208 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 1 -- 172.21.15.63:0/1118057689 shutdown_connections 2024-05-27T07:42:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 1 -- 172.21.15.63:0/1118057689 wait complete. 2024-05-27T07:42:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.195+0000 7f24a29d4740 20 client.17208 trim_cache size 0 max 16384 2024-05-27T07:42:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.217+0000 7f24a29d4740 1 Processor -- start 2024-05-27T07:42:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.217+0000 7f24a29d4740 1 -- start start 2024-05-27T07:42:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.217+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647204da2f0 0x5647208ab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.217+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204e24d0 0x5647208bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.217+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647205031b0 0x5647208a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.217+0000 7f24a29d4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24941b5b70 con 0x5647205031b0 2024-05-27T07:42:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.217+0000 7f24a29d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24941b3440 con 0x5647204e24d0 2024-05-27T07:42:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.217+0000 7f24a29d4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24941b12b0 con 0x5647204da2f0 2024-05-27T07:42:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.218+0000 7f2499a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647205031b0 0x5647208a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.218+0000 7f2499a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647205031b0 0x5647208a9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39700/0 (socket says 172.21.15.63:39700) 2024-05-27T07:42:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.218+0000 7f2499a00640 1 -- 172.21.15.63:0/1660919774 learned_addr learned my addr 172.21.15.63:0/1660919774 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.218+0000 7f2493e00640 1 --2- 172.21.15.63:0/1660919774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204e24d0 0x5647208bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.218+0000 7f2499000640 1 --2- 172.21.15.63:0/1660919774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647204da2f0 0x5647208ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.218+0000 7f2499a00640 1 -- 172.21.15.63:0/1660919774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647204da2f0 msgr2=0x5647208ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.218+0000 7f2499a00640 1 --2- 172.21.15.63:0/1660919774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647204da2f0 0x5647208ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.218+0000 7f2499a00640 1 -- 172.21.15.63:0/1660919774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204e24d0 msgr2=0x5647208bbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.218+0000 7f2499a00640 1 --2- 172.21.15.63:0/1660919774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204e24d0 0x5647208bbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.218+0000 7f2499a00640 1 -- 172.21.15.63:0/1660919774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472089d230 con 0x5647205031b0 2024-05-27T07:42:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.219+0000 7f2499a00640 1 --2- 172.21.15.63:0/1660919774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647205031b0 0x5647208a9710 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f248c012770 tx=0x7f248c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcd47b1f5092a66 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.219+0000 7f2485600640 1 -- 172.21.15.63:0/1660919774 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c01f830 con 0x5647205031b0 2024-05-27T07:42:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.219+0000 7f2485600640 1 -- 172.21.15.63:0/1660919774 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f248c01fe60 con 0x5647205031b0 2024-05-27T07:42:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.219+0000 7f2485600640 1 -- 172.21.15.63:0/1660919774 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c0283d0 con 0x5647205031b0 2024-05-27T07:42:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.220+0000 7f24a29d4740 1 -- 172.21.15.63:0/1660919774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647205031b0 msgr2=0x5647208a9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.220+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1660919774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647205031b0 0x5647208a9710 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f248c012770 tx=0x7f248c014000 comp rx=0 tx=0).stop 2024-05-27T07:42:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.220+0000 7f24a29d4740 1 -- 172.21.15.63:0/1660919774 shutdown_connections 2024-05-27T07:42:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.220+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1660919774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647205031b0 0x5647208a9710 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.220+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1660919774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647204e24d0 0x5647208bbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.220+0000 7f24a29d4740 1 --2- 172.21.15.63:0/1660919774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647204da2f0 0x5647208ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.220+0000 7f24a29d4740 1 -- 172.21.15.63:0/1660919774 >> 172.21.15.63:0/1660919774 conn(0x5647204f7300 msgr2=0x5647208b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.221+0000 7f24a29d4740 1 -- 172.21.15.63:0/1660919774 shutdown_connections 2024-05-27T07:42:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.221+0000 7f24a29d4740 1 -- 172.21.15.63:0/1660919774 wait complete. 2024-05-27T07:42:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.221+0000 7f24a29d4740 1 -- start start 2024-05-27T07:42:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.221+0000 7f24a29d4740 1 Processor -- start 2024-05-27T07:42:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.222+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647204da2f0 0x5647208c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.222+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.222+0000 7f24a29d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647205031b0 0x5647209280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.222+0000 7f24a29d4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24941b1900 con 0x5647204e24d0 2024-05-27T07:42:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.222+0000 7f24a29d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24941b2d70 con 0x5647205031b0 2024-05-27T07:42:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.222+0000 7f24a29d4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24941b1fd0 con 0x5647204da2f0 2024-05-27T07:42:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.222+0000 7f24a29d4740 10 client.0 _ll_register_callbacks cb 0x7f2480004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.222+0000 7f2493e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.222+0000 7f2493e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208c2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39702/0 (socket says 172.21.15.63:39702) 2024-05-27T07:42:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.222+0000 7f2493e00640 1 -- 172.21.15.63:0/3568501989 learned_addr learned my addr 172.21.15.63:0/3568501989 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.223+0000 7f2499000640 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647204da2f0 0x5647208c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.223+0000 7f2499a00640 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647205031b0 0x5647209280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.223+0000 7f2493e00640 1 -- 172.21.15.63:0/3568501989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647204da2f0 msgr2=0x5647208c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.223+0000 7f2493e00640 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647204da2f0 0x5647208c2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.223+0000 7f2493e00640 1 -- 172.21.15.63:0/3568501989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647205031b0 msgr2=0x5647209280c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.223+0000 7f2493e00640 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647205031b0 0x5647209280c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.223+0000 7f2493e00640 1 -- 172.21.15.63:0/3568501989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564720928600 con 0x5647204e24d0 2024-05-27T07:42:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.223+0000 7f2499a00640 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647205031b0 0x5647209280c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:42:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.223+0000 7f2493e00640 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208c2590 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f248004d990 tx=0x7f2480005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a0e86f25744fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.223+0000 7f247aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.224+0000 7f247aa00640 1 -- 172.21.15.63:0/3568501989 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480035040 con 0x5647204e24d0 2024-05-27T07:42:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.224+0000 7f247aa00640 1 -- 172.21.15.63:0/3568501989 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f248004d420 con 0x5647204e24d0 2024-05-27T07:42:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.224+0000 7f24a29d4740 1 -- 172.21.15.63:0/3568501989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f248c01fe60 con 0x5647204e24d0 2024-05-27T07:42:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.224+0000 7f247aa00640 1 -- 172.21.15.63:0/3568501989 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f248004d620 con 0x5647204e24d0 2024-05-27T07:42:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.224+0000 7f24a29d4740 10 client.17214 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.224+0000 7f24a29d4740 1 -- 172.21.15.63:0/3568501989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5647209296e0 con 0x5647204e24d0 2024-05-27T07:42:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.225+0000 7f247aa00640 1 -- 172.21.15.63:0/3568501989 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f248c01fe60 con 0x5647204e24d0 2024-05-27T07:42:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.225+0000 7f247aa00640 1 -- 172.21.15.63:0/3568501989 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f24800035f0 con 0x5647204e24d0 2024-05-27T07:42:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.225+0000 7f24a29d4740 10 client.17214 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.225+0000 7f24a29d4740 10 client.17214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.225+0000 7f24a29d4740 20 client.17214 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.225+0000 7f24a29d4740 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5647204971c0 0x56472092bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.225+0000 7f24a29d4740 4 client.17214 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:42:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.225+0000 7f24a29d4740 1 -- 172.21.15.63:0/3568501989 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56472078c4b0 con 0x5647204971c0 2024-05-27T07:42:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.226+0000 7f2499000640 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5647204971c0 0x56472092bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.227+0000 7f247aa00640 10 client.17214 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.227+0000 7f2499000640 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5647204971c0 0x56472092bc20 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74cb9bd5f39b0c5 server_cookie=871dd9dcd51f2560 in_seq=0 out_seq=0 2024-05-27T07:42:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.262+0000 7f247aa00640 1 -- 172.21.15.63:0/3568501989 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56472078c4b0 con 0x5647204971c0 2024-05-27T07:42:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.262+0000 7f247aa00640 10 client.17214 handle_command_reply: tid=0 2024-05-27T07:42:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.262+0000 7f24a29d4740 1 client.17214 shutdown 2024-05-27T07:42:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 1 -- 172.21.15.63:0/3568501989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 msgr2=0x5647208c2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208c2590 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f248004d990 tx=0x7f2480005060 comp rx=0 tx=0).stop 2024-05-27T07:42:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 1 -- 172.21.15.63:0/3568501989 shutdown_connections 2024-05-27T07:42:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5647204971c0 0x56472092bc20 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647205031b0 0x5647209280c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647204e24d0 0x5647208c2590 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 1 --2- 172.21.15.63:0/3568501989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647204da2f0 0x5647208c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 1 -- 172.21.15.63:0/3568501989 >> 172.21.15.63:0/3568501989 conn(0x5647204f7300 msgr2=0x5647208af170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f247aa00640 0 client.17214 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 1 -- 172.21.15.63:0/3568501989 shutdown_connections 2024-05-27T07:42:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 1 -- 172.21.15.63:0/3568501989 wait complete. 2024-05-27T07:42:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.263+0000 7f24a29d4740 20 client.17214 trim_cache size 0 max 16384 2024-05-27T07:42:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:42:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:42:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:42:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.264+0000 7f249a400640 1 -- 172.21.15.63:0/2951947499 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f247c0b2950 msgr2=0x7f247c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.264+0000 7f249a400640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f247c0b2950 0x7f247c0b4e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5647208a2bd0 tx=0x7f248c009580 comp rx=0 tx=0).stop 2024-05-27T07:42:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.264+0000 7f249a400640 1 -- 172.21.15.63:0/2951947499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494104250 msgr2=0x7f249419d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.264+0000 7f249a400640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494104250 0x7f249419d640 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f248800d770 tx=0x7f248800dc30 comp rx=0 tx=0).stop 2024-05-27T07:42:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.264+0000 7f249a400640 1 -- 172.21.15.63:0/2951947499 shutdown_connections 2024-05-27T07:42:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.264+0000 7f249a400640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f247c0b2950 0x7f247c0b4e60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.264+0000 7f249a400640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2494109f50 0x7f24941a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.264+0000 7f249a400640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2494109ab0 0x7f249419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.264+0000 7f249a400640 1 --2- 172.21.15.63:0/2951947499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494104250 0x7f249419d640 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.264+0000 7f249a400640 1 -- 172.21.15.63:0/2951947499 >> 172.21.15.63:0/2951947499 conn(0x7f2494079240 msgr2=0x7f2494100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.265+0000 7f249a400640 1 -- 172.21.15.63:0/2951947499 shutdown_connections 2024-05-27T07:42:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:42:52.265+0000 7f249a400640 1 -- 172.21.15.63:0/2951947499 wait complete. 2024-05-27T07:42:52.680 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:52 smithi063 ceph-mon[31613]: pgmap v288: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:42:52.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:52 smithi086 ceph-mon[34687]: pgmap v288: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:42:52.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:52 smithi149 ceph-mon[35408]: pgmap v288: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:42:53.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.107+0000 7fdcde400700 1 -- 172.21.15.63:0/1291079958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd8108730 msgr2=0x7fdcd8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:53.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.107+0000 7fdcde400700 1 --2- 172.21.15.63:0/1291079958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd8108730 0x7fdcd8110090 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000b0d0 tx=0x7fdcd000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.107+0000 7fdcde400700 1 -- 172.21.15.63:0/1291079958 shutdown_connections 2024-05-27T07:42:53.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.107+0000 7fdcde400700 1 --2- 172.21.15.63:0/1291079958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd8108730 0x7fdcd8110090 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.107+0000 7fdcde400700 1 --2- 172.21.15.63:0/1291079958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd81034b0 0x7fdcd81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.107+0000 7fdcde400700 1 --2- 172.21.15.63:0/1291079958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd8107de0 0x7fdcd81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.107+0000 7fdcde400700 1 -- 172.21.15.63:0/1291079958 >> 172.21.15.63:0/1291079958 conn(0x7fdcd80fedb0 msgr2=0x7fdcd81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:53.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.108+0000 7fdcde400700 1 -- 172.21.15.63:0/1291079958 shutdown_connections 2024-05-27T07:42:53.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.108+0000 7fdcde400700 1 -- 172.21.15.63:0/1291079958 wait complete. 2024-05-27T07:42:53.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.109+0000 7fdcde400700 1 Processor -- start 2024-05-27T07:42:53.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.109+0000 7fdcde400700 1 -- start start 2024-05-27T07:42:53.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.110+0000 7fdcde400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd81034b0 0x7fdcd8114a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.110+0000 7fdcde400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd8107de0 0x7fdcd8114f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.110+0000 7fdcde400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd8108730 0x7fdcd8115480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.110+0000 7fdcde400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd806c020 con 0x7fdcd8107de0 2024-05-27T07:42:53.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.110+0000 7fdcde400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd806bea0 con 0x7fdcd81034b0 2024-05-27T07:42:53.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.110+0000 7fdcde400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd806c1a0 con 0x7fdcd8108730 2024-05-27T07:42:53.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.111+0000 7fdcd6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd8107de0 0x7fdcd8114f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.111+0000 7fdcd6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd8107de0 0x7fdcd8114f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39724/0 (socket says 172.21.15.63:39724) 2024-05-27T07:42:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.111+0000 7fdcd6a00700 1 -- 172.21.15.63:0/2861482068 learned_addr learned my addr 172.21.15.63:0/2861482068 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.111+0000 7fdcd7400700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd81034b0 0x7fdcd8114a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.111+0000 7fdcd7e00700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd8108730 0x7fdcd8115480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.111+0000 7fdcd6a00700 1 -- 172.21.15.63:0/2861482068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd8108730 msgr2=0x7fdcd8115480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.111+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd8108730 0x7fdcd8115480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.111+0000 7fdcd6a00700 1 -- 172.21.15.63:0/2861482068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd81034b0 msgr2=0x7fdcd8114a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.111+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd81034b0 0x7fdcd8114a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.111+0000 7fdcd6a00700 1 -- 172.21.15.63:0/2861482068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd81ad420 con 0x7fdcd8107de0 2024-05-27T07:42:53.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.112+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd8107de0 0x7fdcd8114f40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400d980 tx=0x7fdcc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:53.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.112+0000 7fdccfe00700 1 -- 172.21.15.63:0/2861482068 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4017070 con 0x7fdcd8107de0 2024-05-27T07:42:53.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.112+0000 7fdccfe00700 1 -- 172.21.15.63:0/2861482068 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcc400fd20 con 0x7fdcd8107de0 2024-05-27T07:42:53.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.112+0000 7fdcde400700 1 -- 172.21.15.63:0/2861482068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd81ad710 con 0x7fdcd8107de0 2024-05-27T07:42:53.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.112+0000 7fdccfe00700 1 -- 172.21.15.63:0/2861482068 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400e3b0 con 0x7fdcd8107de0 2024-05-27T07:42:53.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.113+0000 7fdcde400700 1 -- 172.21.15.63:0/2861482068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd81adc50 con 0x7fdcd8107de0 2024-05-27T07:42:53.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.118+0000 7fdccfe00700 1 -- 172.21.15.63:0/2861482068 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdcc400f890 con 0x7fdcd8107de0 2024-05-27T07:42:53.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.118+0000 7fdcecf8eb80 1 Processor -- start 2024-05-27T07:42:53.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.119+0000 7fdcecf8eb80 1 -- start start 2024-05-27T07:42:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.119+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.119+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bb2db0 0x55d917bac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.119+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bace30 0x55d917bad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.119+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d917adaec0 con 0x55d917a2dec0 2024-05-27T07:42:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.119+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d917acaa60 con 0x55d917bb2db0 2024-05-27T07:42:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.119+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d917aceaa0 con 0x55d917bace30 2024-05-27T07:42:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.120+0000 7fdccfe00700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcbc0b29f0 0x7fdcbc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.120+0000 7fdcd7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bb2db0 0x55d917bac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.120+0000 7fdcd7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.120+0000 7fdcd7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39734/0 (socket says 172.21.15.63:39734) 2024-05-27T07:42:53.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.120+0000 7fdccfe00700 1 -- 172.21.15.63:0/2861482068 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fdcc40e2430 con 0x7fdcd8107de0 2024-05-27T07:42:53.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.120+0000 7fdcd7e00700 1 -- 172.21.15.63:0/2135646594 learned_addr learned my addr 172.21.15.63:0/2135646594 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:53.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.120+0000 7fdcd7e00700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcbc0b29f0 0x7fdcbc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.120+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/2135646594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bace30 0x55d917bad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.121+0000 7fdcd7e00700 1 -- 172.21.15.63:0/2135646594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bace30 msgr2=0x55d917bad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:53.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.121+0000 7fdcd7e00700 1 --2- 172.21.15.63:0/2135646594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bace30 0x55d917bad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.121+0000 7fdcd7e00700 1 -- 172.21.15.63:0/2135646594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bb2db0 msgr2=0x55d917bac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:53.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.121+0000 7fdcd7e00700 1 --2- 172.21.15.63:0/2135646594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bb2db0 0x55d917bac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.121+0000 7fdcd7e00700 1 -- 172.21.15.63:0/2135646594 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d917a934a0 con 0x55d917a2dec0 2024-05-27T07:42:53.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.121+0000 7fdcd7e00700 1 --2- 172.21.15.63:0/2135646594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb2870 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0004cf0 tx=0x7fdcd0004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17ef55dc644ca6e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:53.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.121+0000 7fdcd7e00700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcbc0b29f0 0x7fdcbc0b4f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009e60 tx=0x7fdcd0028000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:53.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.122+0000 7fdcccc00700 1 -- 172.21.15.63:0/2135646594 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd002f070 con 0x55d917a2dec0 2024-05-27T07:42:53.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.122+0000 7fdcccc00700 1 -- 172.21.15.63:0/2135646594 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcd001f440 con 0x55d917a2dec0 2024-05-27T07:42:53.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.122+0000 7fdcccc00700 1 -- 172.21.15.63:0/2135646594 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd002a670 con 0x55d917a2dec0 2024-05-27T07:42:53.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.123+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/2135646594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 msgr2=0x55d917bb2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:53.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.123+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/2135646594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb2870 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0004cf0 tx=0x7fdcd0004d20 comp rx=0 tx=0).stop 2024-05-27T07:42:53.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.123+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/2135646594 shutdown_connections 2024-05-27T07:42:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.123+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/2135646594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bace30 0x55d917bad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.123+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/2135646594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb2870 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.123+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/2135646594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bb2db0 0x55d917bac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.123+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/2135646594 >> 172.21.15.63:0/2135646594 conn(0x55d917a8e040 msgr2=0x55d917ba7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.123+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/2135646594 shutdown_connections 2024-05-27T07:42:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.124+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/2135646594 wait complete. 2024-05-27T07:42:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.124+0000 7fdcecf8eb80 1 -- start start 2024-05-27T07:42:53.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.124+0000 7fdcecf8eb80 1 Processor -- start 2024-05-27T07:42:53.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.125+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.125+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bace30 0x55d917bb6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.125+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bb2db0 0x55d917bb6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.125+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d9179d1020 con 0x55d917a2dec0 2024-05-27T07:42:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.125+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d917a6b7e0 con 0x55d917bace30 2024-05-27T07:42:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.125+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d917bb9820 con 0x55d917bb2db0 2024-05-27T07:42:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.125+0000 7fdcecf8eb80 10 client.0 _ll_register_callbacks cb 0x55d917a74cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.126+0000 7fdcd7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.126+0000 7fdcd7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39748/0 (socket says 172.21.15.63:39748) 2024-05-27T07:42:53.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.126+0000 7fdcd7400700 1 -- 172.21.15.63:0/1046424439 learned_addr learned my addr 172.21.15.63:0/1046424439 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.126+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bace30 0x55d917bb6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.126+0000 7fdcd7e00700 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bb2db0 0x55d917bb6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.126+0000 7fdcd7400700 1 -- 172.21.15.63:0/1046424439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bb2db0 msgr2=0x55d917bb6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.126+0000 7fdcd7400700 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bb2db0 0x55d917bb6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.126+0000 7fdcd7400700 1 -- 172.21.15.63:0/1046424439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bace30 msgr2=0x55d917bb6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.126+0000 7fdcd7400700 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bace30 0x55d917bb6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.126+0000 7fdcd7400700 1 -- 172.21.15.63:0/1046424439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d917a1a950 con 0x55d917a2dec0 2024-05-27T07:42:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.127+0000 7fdcd7400700 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb6550 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800bb70 tx=0x7fdcc800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a02da213a72f0fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:53.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.127+0000 7fdcc1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.127+0000 7fdcc1600700 1 -- 172.21.15.63:0/1046424439 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8003c20 con 0x55d917a2dec0 2024-05-27T07:42:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.127+0000 7fdcc1600700 1 -- 172.21.15.63:0/1046424439 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fdcc8003dc0 con 0x55d917a2dec0 2024-05-27T07:42:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.128+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/1046424439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd001f440 con 0x55d917a2dec0 2024-05-27T07:42:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.128+0000 7fdcc1600700 1 -- 172.21.15.63:0/1046424439 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdcc8024050 con 0x55d917a2dec0 2024-05-27T07:42:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.128+0000 7fdcecf8eb80 10 client.17226 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:53.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.128+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/1046424439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d917a9b0e0 con 0x55d917a2dec0 2024-05-27T07:42:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.129+0000 7fdcc1600700 1 -- 172.21.15.63:0/1046424439 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcd001f440 con 0x55d917a2dec0 2024-05-27T07:42:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.129+0000 7fdcc1600700 1 -- 172.21.15.63:0/1046424439 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdcc80114d0 con 0x55d917a2dec0 2024-05-27T07:42:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.129+0000 7fdcecf8eb80 10 client.17226 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.129+0000 7fdcecf8eb80 10 client.17226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.129+0000 7fdcecf8eb80 20 client.17226 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:53.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.129+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d917bc6740 0x55d917bc8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.129+0000 7fdcecf8eb80 4 client.17226 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:42:53.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.129+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/1046424439 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d917adce10 con 0x55d917bc6740 2024-05-27T07:42:53.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.130+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d917bc6740 0x55d917bc8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.131+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d917bc6740 0x55d917bc8b60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90aeb9733b69d431 server_cookie=fbe87563c547ef70 in_seq=0 out_seq=0 2024-05-27T07:42:53.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.131+0000 7fdcc1600700 10 client.17226 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.133+0000 7fdcc1600700 1 -- 172.21.15.63:0/1046424439 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d917adce10 con 0x55d917bc6740 2024-05-27T07:42:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.133+0000 7fdcc1600700 10 client.17226 handle_command_reply: tid=0 2024-05-27T07:42:53.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.133+0000 7fdcecf8eb80 1 client.17226 shutdown 2024-05-27T07:42:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.133+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/1046424439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 msgr2=0x55d917bb6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.133+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb6550 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800bb70 tx=0x7fdcc800be80 comp rx=0 tx=0).stop 2024-05-27T07:42:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.134+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/1046424439 shutdown_connections 2024-05-27T07:42:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.134+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d917bc6740 0x55d917bc8b60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.134+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bb2db0 0x55d917bb6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.134+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bb6550 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.134+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/1046424439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bace30 0x55d917bb6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.134+0000 7fdcc1600700 0 client.17226 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.134+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/1046424439 >> 172.21.15.63:0/1046424439 conn(0x55d917a8e040 msgr2=0x55d917ba7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.134+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/1046424439 shutdown_connections 2024-05-27T07:42:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.134+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/1046424439 wait complete. 2024-05-27T07:42:53.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.135+0000 7fdcecf8eb80 20 client.17226 trim_cache size 0 max 16384 2024-05-27T07:42:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.164+0000 7fdcecf8eb80 1 Processor -- start 2024-05-27T07:42:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.164+0000 7fdcecf8eb80 1 -- start start 2024-05-27T07:42:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.165+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917a2dec0 0x55d917bac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.165+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917bace30 0x55d917bb1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.165+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bb2db0 0x55d917bb88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.165+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd81b93d0 con 0x55d917bace30 2024-05-27T07:42:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.165+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd81b6c60 con 0x55d917bb2db0 2024-05-27T07:42:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.165+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd81b4ad0 con 0x55d917a2dec0 2024-05-27T07:42:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.165+0000 7fdcd6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917bace30 0x55d917bb1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.165+0000 7fdcd6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917bace30 0x55d917bb1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39762/0 (socket says 172.21.15.63:39762) 2024-05-27T07:42:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.165+0000 7fdcd6a00700 1 -- 172.21.15.63:0/3972243374 learned_addr learned my addr 172.21.15.63:0/3972243374 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.166+0000 7fdcd7400700 1 --2- 172.21.15.63:0/3972243374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917a2dec0 0x55d917bac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.166+0000 7fdcd7e00700 1 --2- 172.21.15.63:0/3972243374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bb2db0 0x55d917bb88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.166+0000 7fdcd6a00700 1 -- 172.21.15.63:0/3972243374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917a2dec0 msgr2=0x55d917bac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.166+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/3972243374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917a2dec0 0x55d917bac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.166+0000 7fdcd6a00700 1 -- 172.21.15.63:0/3972243374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bb2db0 msgr2=0x55d917bb88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.166+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/3972243374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bb2db0 0x55d917bb88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.166+0000 7fdcd6a00700 1 -- 172.21.15.63:0/3972243374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d917bb8df0 con 0x55d917bace30 2024-05-27T07:42:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.166+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/3972243374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917bace30 0x55d917bb1d40 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40a8ff0 tx=0x7fdcc40104e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e4c59e5343496d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.167+0000 7fdcc3e00700 1 -- 172.21.15.63:0/3972243374 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4020040 con 0x55d917bace30 2024-05-27T07:42:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.167+0000 7fdcc3e00700 1 -- 172.21.15.63:0/3972243374 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcc40a8e00 con 0x55d917bace30 2024-05-27T07:42:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.167+0000 7fdcc3e00700 1 -- 172.21.15.63:0/3972243374 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40e2120 con 0x55d917bace30 2024-05-27T07:42:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.168+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3972243374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917bace30 msgr2=0x55d917bb1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.168+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/3972243374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917bace30 0x55d917bb1d40 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40a8ff0 tx=0x7fdcc40104e0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.168+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3972243374 shutdown_connections 2024-05-27T07:42:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.168+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/3972243374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bb2db0 0x55d917bb88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.168+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/3972243374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917a2dec0 0x55d917bac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.168+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/3972243374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917bace30 0x55d917bb1d40 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.168+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3972243374 >> 172.21.15.63:0/3972243374 conn(0x55d917a8e040 msgr2=0x55d917bb6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.168+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3972243374 shutdown_connections 2024-05-27T07:42:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.168+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3972243374 wait complete. 2024-05-27T07:42:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.169+0000 7fdcecf8eb80 1 -- start start 2024-05-27T07:42:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.169+0000 7fdcecf8eb80 1 Processor -- start 2024-05-27T07:42:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bbba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bace30 0x55d917bbbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcecf8eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bb2db0 0x55d917bbc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd81b5120 con 0x55d917a2dec0 2024-05-27T07:42:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd81b6590 con 0x55d917bace30 2024-05-27T07:42:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcecf8eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd81b57f0 con 0x55d917bb2db0 2024-05-27T07:42:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcecf8eb80 10 client.0 _ll_register_callbacks cb 0x55d917a74cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcd7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bbba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcd7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bbba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39770/0 (socket says 172.21.15.63:39770) 2024-05-27T07:42:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcd7400700 1 -- 172.21.15.63:0/3014021857 learned_addr learned my addr 172.21.15.63:0/3014021857 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcd7e00700 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bb2db0 0x55d917bbc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.170+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bace30 0x55d917bbbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.171+0000 7fdcd7400700 1 -- 172.21.15.63:0/3014021857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bb2db0 msgr2=0x55d917bbc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.171+0000 7fdcd7400700 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bb2db0 0x55d917bbc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.171+0000 7fdcd7400700 1 -- 172.21.15.63:0/3014021857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bace30 msgr2=0x55d917bbbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.171+0000 7fdcd7400700 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bace30 0x55d917bbbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.171+0000 7fdcd7400700 1 -- 172.21.15.63:0/3014021857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d917c311f0 con 0x55d917a2dec0 2024-05-27T07:42:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.171+0000 7fdcd7400700 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bbba80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800bb70 tx=0x7fdcc80117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7e86f3f9a2928c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.171+0000 7fdcc1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.171+0000 7fdcc1600700 1 -- 172.21.15.63:0/3014021857 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8021040 con 0x55d917a2dec0 2024-05-27T07:42:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.171+0000 7fdcc1600700 1 -- 172.21.15.63:0/3014021857 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fdcc8011ae0 con 0x55d917a2dec0 2024-05-27T07:42:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.172+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3014021857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc40a8e00 con 0x55d917a2dec0 2024-05-27T07:42:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.172+0000 7fdcc1600700 1 -- 172.21.15.63:0/3014021857 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdcc8039050 con 0x55d917a2dec0 2024-05-27T07:42:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.172+0000 7fdcecf8eb80 10 client.17232 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.172+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3014021857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d917c324c0 con 0x55d917a2dec0 2024-05-27T07:42:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.172+0000 7fdcc1600700 1 -- 172.21.15.63:0/3014021857 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcc802d020 con 0x55d917a2dec0 2024-05-27T07:42:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.172+0000 7fdcc1600700 1 -- 172.21.15.63:0/3014021857 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdcc80046f0 con 0x55d917a2dec0 2024-05-27T07:42:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.173+0000 7fdcecf8eb80 10 client.17232 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.173+0000 7fdcecf8eb80 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.173+0000 7fdcecf8eb80 20 client.17232 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.173+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d917c32c50 0x55d917c35070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.173+0000 7fdcecf8eb80 4 client.17232 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:42:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.173+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3014021857 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d917ad65f0 con 0x55d917c32c50 2024-05-27T07:42:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.173+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d917c32c50 0x55d917c35070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.174+0000 7fdcd6a00700 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d917c32c50 0x55d917c35070 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8b113e9e06aa9b server_cookie=f3b0baf9e0c7c65e in_seq=0 out_seq=0 2024-05-27T07:42:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.174+0000 7fdcc1600700 10 client.17232 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.177+0000 7fdcc1600700 1 -- 172.21.15.63:0/3014021857 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d917ad65f0 con 0x55d917c32c50 2024-05-27T07:42:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.177+0000 7fdcc1600700 10 client.17232 handle_command_reply: tid=0 2024-05-27T07:42:53.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.177+0000 7fdcecf8eb80 1 client.17232 shutdown 2024-05-27T07:42:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.177+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3014021857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 msgr2=0x55d917bbba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bbba80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800bb70 tx=0x7fdcc80117a0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3014021857 shutdown_connections 2024-05-27T07:42:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d917c32c50 0x55d917c35070 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d917bb2db0 0x55d917bbc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d917a2dec0 0x55d917bbba80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcecf8eb80 1 --2- 172.21.15.63:0/3014021857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d917bace30 0x55d917bbbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3014021857 >> 172.21.15.63:0/3014021857 conn(0x55d917a8e040 msgr2=0x55d917ba6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcc1600700 0 client.17232 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:53.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3014021857 shutdown_connections 2024-05-27T07:42:53.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcecf8eb80 1 -- 172.21.15.63:0/3014021857 wait complete. 2024-05-27T07:42:53.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.178+0000 7fdcecf8eb80 20 client.17232 trim_cache size 0 max 16384 2024-05-27T07:42:53.178 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:42:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:42:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "d01b63a0-cef4-493a-a2ba-670431910d28", 2024-05-27T07:42:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:42:53.178 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:42:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.179+0000 7fdcde400700 1 -- 172.21.15.63:0/2861482068 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcbc0b29f0 msgr2=0x7fdcbc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.179+0000 7fdcde400700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcbc0b29f0 0x7fdcbc0b4f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009e60 tx=0x7fdcd0028000 comp rx=0 tx=0).stop 2024-05-27T07:42:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.179+0000 7fdcde400700 1 -- 172.21.15.63:0/2861482068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd8107de0 msgr2=0x7fdcd8114f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.179+0000 7fdcde400700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd8107de0 0x7fdcd8114f40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400d980 tx=0x7fdcc400dc90 comp rx=0 tx=0).stop 2024-05-27T07:42:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.180+0000 7fdcde400700 1 -- 172.21.15.63:0/2861482068 shutdown_connections 2024-05-27T07:42:53.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.180+0000 7fdcde400700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcbc0b29f0 0x7fdcbc0b4f00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.180+0000 7fdcde400700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd8108730 0x7fdcd8115480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.180+0000 7fdcde400700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd81034b0 0x7fdcd8114a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.180+0000 7fdcde400700 1 --2- 172.21.15.63:0/2861482068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd8107de0 0x7fdcd8114f40 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.180+0000 7fdcde400700 1 -- 172.21.15.63:0/2861482068 >> 172.21.15.63:0/2861482068 conn(0x7fdcd80fedb0 msgr2=0x7fdcd8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.180+0000 7fdcde400700 1 -- 172.21.15.63:0/2861482068 shutdown_connections 2024-05-27T07:42:53.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:53.180+0000 7fdcde400700 1 -- 172.21.15.63:0/2861482068 wait complete. 2024-05-27T07:42:53.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:53 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:42:53.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:53 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:53.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:53 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:42:53.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:53 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:53.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:53 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:42:53.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:53 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:53.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:53 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:42:53.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:53 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:53.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:53 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:42:53.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:53 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:53.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:53 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:42:53.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:53 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.137 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:42:54.611 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:54.951 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:42:54.951 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:42:54.951 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.951 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.951 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:42:54.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:42:54.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:42:54.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:42:54.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:54 smithi063 ceph-mon[31613]: pgmap v289: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:42:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:42:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:42:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:42:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:42:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:42:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:42:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:54 smithi086 ceph-mon[34687]: pgmap v289: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:42:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:42:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:42:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:42:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:42:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:42:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:42:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:42:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:54 smithi149 ceph-mon[35408]: pgmap v289: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:42:56.723 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:56 smithi063 ceph-mon[31613]: pgmap v290: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:42:56.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:56 smithi086 ceph-mon[34687]: pgmap v290: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:42:56.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:56 smithi149 ceph-mon[35408]: pgmap v290: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:42:57.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.101+0000 7f472fe00700 1 -- 172.21.15.63:0/3315233134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4730105300 msgr2=0x7f4730105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:57.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.101+0000 7f472fe00700 1 --2- 172.21.15.63:0/3315233134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4730105300 0x7f4730105710 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4720009990 tx=0x7f4720009ca0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.102+0000 7f472fe00700 1 -- 172.21.15.63:0/3315233134 shutdown_connections 2024-05-27T07:42:57.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.102+0000 7f472fe00700 1 --2- 172.21.15.63:0/3315233134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f473010e730 0x7f4730110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.102+0000 7f472fe00700 1 --2- 172.21.15.63:0/3315233134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4730105300 0x7f4730105710 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.102+0000 7f472fe00700 1 --2- 172.21.15.63:0/3315233134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730105c50 0x7f473010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.102+0000 7f472fe00700 1 -- 172.21.15.63:0/3315233134 >> 172.21.15.63:0/3315233134 conn(0x7f4730078ab0 msgr2=0x7f4730078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:57.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.102+0000 7f472fe00700 1 -- 172.21.15.63:0/3315233134 shutdown_connections 2024-05-27T07:42:57.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.102+0000 7f472fe00700 1 -- 172.21.15.63:0/3315233134 wait complete. 2024-05-27T07:42:57.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.103+0000 7f472fe00700 1 Processor -- start 2024-05-27T07:42:57.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.103+0000 7f472fe00700 1 -- start start 2024-05-27T07:42:57.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.104+0000 7f472fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4730105300 0x7f4730102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.104+0000 7f472fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4730105c50 0x7f4730103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.104+0000 7f472fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473010e730 0x7f4730103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.104+0000 7f472fe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4730114110 con 0x7f4730105300 2024-05-27T07:42:57.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.104+0000 7f472fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4730113f90 con 0x7f473010e730 2024-05-27T07:42:57.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.104+0000 7f472fe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4730114290 con 0x7f4730105c50 2024-05-27T07:42:57.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.104+0000 7f472ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4730105300 0x7f4730102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.104+0000 7f472ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4730105300 0x7f4730102ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39774/0 (socket says 172.21.15.63:39774) 2024-05-27T07:42:57.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.105+0000 7f472ea00700 1 -- 172.21.15.63:0/72241206 learned_addr learned my addr 172.21.15.63:0/72241206 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:57.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.105+0000 7f472f400700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473010e730 0x7f4730103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.105+0000 7f4727e00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4730105c50 0x7f4730103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.105+0000 7f472ea00700 1 -- 172.21.15.63:0/72241206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4730105c50 msgr2=0x7f4730103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:57.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.105+0000 7f472ea00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4730105c50 0x7f4730103530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.105+0000 7f472ea00700 1 -- 172.21.15.63:0/72241206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473010e730 msgr2=0x7f4730103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:57.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.105+0000 7f472ea00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473010e730 0x7f4730103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.105+0000 7f472ea00700 1 -- 172.21.15.63:0/72241206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47301a8ea0 con 0x7f4730105300 2024-05-27T07:42:57.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.106+0000 7f472ea00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4730105300 0x7f4730102ff0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f47200057c0 tx=0x7f4720009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:57.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.106+0000 7f4734a00700 1 -- 172.21.15.63:0/72241206 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720027070 con 0x7f4730105300 2024-05-27T07:42:57.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.106+0000 7f4734a00700 1 -- 172.21.15.63:0/72241206 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f472000fda0 con 0x7f4730105300 2024-05-27T07:42:57.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.106+0000 7f4734a00700 1 -- 172.21.15.63:0/72241206 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720022c70 con 0x7f4730105300 2024-05-27T07:42:57.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.106+0000 7f472fe00700 1 -- 172.21.15.63:0/72241206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47301a9130 con 0x7f4730105300 2024-05-27T07:42:57.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.106+0000 7f472fe00700 1 -- 172.21.15.63:0/72241206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47301a9660 con 0x7f4730105300 2024-05-27T07:42:57.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.111+0000 7f4734a00700 1 -- 172.21.15.63:0/72241206 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4720022470 con 0x7f4730105300 2024-05-27T07:42:57.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.112+0000 7f4744a0cb80 1 Processor -- start 2024-05-27T07:42:57.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.112+0000 7f4744a0cb80 1 -- start start 2024-05-27T07:42:57.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca670a9f0 0x563ca679e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca679ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca67a82c0 0x563ca67aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563ca66cbe40 con 0x563ca661ee40 2024-05-27T07:42:57.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563ca66bb9e0 con 0x563ca67a82c0 2024-05-27T07:42:57.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ca66bfa20 con 0x563ca670a9f0 2024-05-27T07:42:57.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f4734a00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f47140b2a70 0x7f47140b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f4734a00700 1 -- 172.21.15.63:0/72241206 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f47200e3890 con 0x7f4730105300 2024-05-27T07:42:57.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f472ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca679ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f472f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca670a9f0 0x563ca679e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f4727e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca67a82c0 0x563ca67aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f472f400700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f47140b2a70 0x7f47140b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f472ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca679ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39790/0 (socket says 172.21.15.63:39790) 2024-05-27T07:42:57.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.113+0000 7f472ea00700 1 -- 172.21.15.63:0/1183780278 learned_addr learned my addr 172.21.15.63:0/1183780278 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:57.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.114+0000 7f472ea00700 1 -- 172.21.15.63:0/1183780278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca670a9f0 msgr2=0x563ca679e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:57.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.114+0000 7f472ea00700 1 --2- 172.21.15.63:0/1183780278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca670a9f0 0x563ca679e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.114+0000 7f472ea00700 1 -- 172.21.15.63:0/1183780278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca67a82c0 msgr2=0x563ca67aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:57.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.114+0000 7f472ea00700 1 --2- 172.21.15.63:0/1183780278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca67a82c0 0x563ca67aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.114+0000 7f472ea00700 1 -- 172.21.15.63:0/1183780278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca6684420 con 0x563ca661ee40 2024-05-27T07:42:57.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.115+0000 7f472ea00700 1 --2- 172.21.15.63:0/1183780278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca679ec90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f47200a6860 tx=0x7f47200036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ff386721a3bb68 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:57.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.115+0000 7f472f400700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f47140b2a70 0x7f47140b4f80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f47280060b0 tx=0x7f47280091e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:57.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.115+0000 7f4724c00700 1 -- 172.21.15.63:0/1183780278 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47200aaee0 con 0x563ca661ee40 2024-05-27T07:42:57.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.115+0000 7f4724c00700 1 -- 172.21.15.63:0/1183780278 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47200aa7e0 con 0x563ca661ee40 2024-05-27T07:42:57.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.115+0000 7f4724c00700 1 -- 172.21.15.63:0/1183780278 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47200b3120 con 0x563ca661ee40 2024-05-27T07:42:57.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.117+0000 7f4744a0cb80 1 -- 172.21.15.63:0/1183780278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 msgr2=0x563ca679ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:57.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.117+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/1183780278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca679ec90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f47200a6860 tx=0x7f47200036f0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.117+0000 7f4744a0cb80 1 -- 172.21.15.63:0/1183780278 shutdown_connections 2024-05-27T07:42:57.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.117+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/1183780278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca67a82c0 0x563ca67aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.117+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/1183780278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca670a9f0 0x563ca679e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.117+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/1183780278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca679ec90 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.117+0000 7f4744a0cb80 1 -- 172.21.15.63:0/1183780278 >> 172.21.15.63:0/1183780278 conn(0x563ca66b8390 msgr2=0x563ca6799710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:57.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.117+0000 7f4744a0cb80 1 -- 172.21.15.63:0/1183780278 shutdown_connections 2024-05-27T07:42:57.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.117+0000 7f4744a0cb80 1 -- 172.21.15.63:0/1183780278 wait complete. 2024-05-27T07:42:57.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.118+0000 7f4744a0cb80 1 -- start start 2024-05-27T07:42:57.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.118+0000 7f4744a0cb80 1 Processor -- start 2024-05-27T07:42:57.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca670a9f0 0x563ca67a5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca67a82c0 0x563ca67a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563ca65c1fa0 con 0x563ca661ee40 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563ca665c760 con 0x563ca67a82c0 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ca67aac20 con 0x563ca670a9f0 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f4744a0cb80 10 client.0 _ll_register_callbacks cb 0x563ca6665c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f472ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f472ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67a51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39792/0 (socket says 172.21.15.63:39792) 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f472ea00700 1 -- 172.21.15.63:0/3314526807 learned_addr learned my addr 172.21.15.63:0/3314526807 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.119+0000 7f4727e00700 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca670a9f0 0x563ca67a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.120+0000 7f472f400700 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca67a82c0 0x563ca67a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.120+0000 7f472ea00700 1 -- 172.21.15.63:0/3314526807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca670a9f0 msgr2=0x563ca67a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:57.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.120+0000 7f472ea00700 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca670a9f0 0x563ca67a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.120+0000 7f472ea00700 1 -- 172.21.15.63:0/3314526807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca67a82c0 msgr2=0x563ca67a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.120+0000 7f472ea00700 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca67a82c0 0x563ca67a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.120+0000 7f472ea00700 1 -- 172.21.15.63:0/3314526807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca660b8d0 con 0x563ca661ee40 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.120+0000 7f472ea00700 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67a51c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f472000c010 tx=0x7f4720009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdf7e764c524488 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.120+0000 7f471d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.121+0000 7f471d600700 1 -- 172.21.15.63:0/3314526807 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47200aaee0 con 0x563ca661ee40 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.121+0000 7f471d600700 1 -- 172.21.15.63:0/3314526807 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f4720092110 con 0x563ca661ee40 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.121+0000 7f4744a0cb80 1 -- 172.21.15.63:0/3314526807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47200aa7e0 con 0x563ca661ee40 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.121+0000 7f471d600700 1 -- 172.21.15.63:0/3314526807 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f47200aeb90 con 0x563ca661ee40 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.121+0000 7f4744a0cb80 10 client.17244 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:57.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.121+0000 7f4744a0cb80 1 -- 172.21.15.63:0/3314526807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ca668c060 con 0x563ca661ee40 2024-05-27T07:42:57.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.122+0000 7f471d600700 1 -- 172.21.15.63:0/3314526807 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47200aa7e0 con 0x563ca661ee40 2024-05-27T07:42:57.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.122+0000 7f471d600700 1 -- 172.21.15.63:0/3314526807 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f47200bfba0 con 0x563ca661ee40 2024-05-27T07:42:57.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.122+0000 7f4744a0cb80 10 client.17244 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:57.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.122+0000 7f4744a0cb80 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:57.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.122+0000 7f4744a0cb80 20 client.17244 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:57.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.122+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ca67b7840 0x563ca67b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.122+0000 7f4744a0cb80 4 client.17244 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:42:57.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.122+0000 7f4744a0cb80 1 -- 172.21.15.63:0/3314526807 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ca66cdd90 con 0x563ca67b7840 2024-05-27T07:42:57.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.123+0000 7f4727e00700 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ca67b7840 0x563ca67b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.124+0000 7f471d600700 10 client.17244 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:57.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.124+0000 7f4727e00700 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ca67b7840 0x563ca67b9c60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970e92199efbae5c server_cookie=8338e88e0d763531 in_seq=0 out_seq=0 2024-05-27T07:42:57.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f471d600700 1 -- 172.21.15.63:0/3314526807 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563ca66cdd90 con 0x563ca67b7840 2024-05-27T07:42:57.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f471d600700 10 client.17244 handle_command_reply: tid=0 2024-05-27T07:42:57.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f4744a0cb80 1 client.17244 shutdown 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f4744a0cb80 1 -- 172.21.15.63:0/3314526807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 msgr2=0x563ca67a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67a51c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f472000c010 tx=0x7f4720009ca0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f4744a0cb80 1 -- 172.21.15.63:0/3314526807 shutdown_connections 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ca67b7840 0x563ca67b9c60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca67a82c0 0x563ca67a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67a51c0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/3314526807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca670a9f0 0x563ca67a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f4744a0cb80 1 -- 172.21.15.63:0/3314526807 >> 172.21.15.63:0/3314526807 conn(0x563ca66b8390 msgr2=0x563ca6799330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.126+0000 7f471d600700 0 client.17244 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.127+0000 7f4744a0cb80 1 -- 172.21.15.63:0/3314526807 shutdown_connections 2024-05-27T07:42:57.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.127+0000 7f4744a0cb80 1 -- 172.21.15.63:0/3314526807 wait complete. 2024-05-27T07:42:57.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.127+0000 7f4744a0cb80 20 client.17244 trim_cache size 0 max 16384 2024-05-27T07:42:57.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.155+0000 7f4744a0cb80 1 Processor -- start 2024-05-27T07:42:57.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.156+0000 7f4744a0cb80 1 -- start start 2024-05-27T07:42:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.156+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.156+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca670a9f0 0x563ca6799740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.156+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca67a82c0 0x563ca67a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.156+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f47301b4d90 con 0x563ca661ee40 2024-05-27T07:42:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.156+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47301b2620 con 0x563ca670a9f0 2024-05-27T07:42:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.156+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f47301b0490 con 0x563ca67a82c0 2024-05-27T07:42:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.157+0000 7f472ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.157+0000 7f472ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39802/0 (socket says 172.21.15.63:39802) 2024-05-27T07:42:57.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.157+0000 7f472ea00700 1 -- 172.21.15.63:0/1322100825 learned_addr learned my addr 172.21.15.63:0/1322100825 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:57.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.157+0000 7f4727e00700 1 --2- 172.21.15.63:0/1322100825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca670a9f0 0x563ca6799740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.157+0000 7f472f400700 1 --2- 172.21.15.63:0/1322100825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca67a82c0 0x563ca67a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.157+0000 7f472ea00700 1 -- 172.21.15.63:0/1322100825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca67a82c0 msgr2=0x563ca67a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:57.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.157+0000 7f472ea00700 1 --2- 172.21.15.63:0/1322100825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca67a82c0 0x563ca67a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.157+0000 7f472ea00700 1 -- 172.21.15.63:0/1322100825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca670a9f0 msgr2=0x563ca6799740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.157+0000 7f472ea00700 1 --2- 172.21.15.63:0/1322100825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca670a9f0 0x563ca6799740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.158+0000 7f472ea00700 1 -- 172.21.15.63:0/1322100825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca67a5fb0 con 0x563ca661ee40 2024-05-27T07:42:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.158+0000 7f472ea00700 1 --2- 172.21.15.63:0/1322100825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67b3df0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f47200e0c20 tx=0x7f47200b3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb9f191925e4edf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.158+0000 7f471fe00700 1 -- 172.21.15.63:0/1322100825 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47200aaee0 con 0x563ca661ee40 2024-05-27T07:42:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.158+0000 7f471fe00700 1 -- 172.21.15.63:0/1322100825 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47200aa7e0 con 0x563ca661ee40 2024-05-27T07:42:57.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.158+0000 7f471fe00700 1 -- 172.21.15.63:0/1322100825 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47200b35b0 con 0x563ca661ee40 2024-05-27T07:42:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.159+0000 7f4744a0cb80 1 -- 172.21.15.63:0/1322100825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 msgr2=0x563ca67b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.159+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/1322100825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67b3df0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f47200e0c20 tx=0x7f47200b3040 comp rx=0 tx=0).stop 2024-05-27T07:42:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.160+0000 7f4744a0cb80 1 -- 172.21.15.63:0/1322100825 shutdown_connections 2024-05-27T07:42:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.160+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/1322100825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca67a82c0 0x563ca67a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.160+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/1322100825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca661ee40 0x563ca67b3df0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.160+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/1322100825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca670a9f0 0x563ca6799740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.160+0000 7f4744a0cb80 1 -- 172.21.15.63:0/1322100825 >> 172.21.15.63:0/1322100825 conn(0x563ca66b8390 msgr2=0x563ca67a3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.160+0000 7f4744a0cb80 1 -- 172.21.15.63:0/1322100825 shutdown_connections 2024-05-27T07:42:57.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.160+0000 7f4744a0cb80 1 -- 172.21.15.63:0/1322100825 wait complete. 2024-05-27T07:42:57.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.160+0000 7f4744a0cb80 1 -- start start 2024-05-27T07:42:57.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.161+0000 7f4744a0cb80 1 Processor -- start 2024-05-27T07:42:57.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.161+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca661ee40 0x563ca67bd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.161+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca670a9f0 0x563ca67ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.161+0000 7f4744a0cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca67a82c0 0x563ca67bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.161+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f47301b0ae0 con 0x563ca67a82c0 2024-05-27T07:42:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.161+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47301b1f50 con 0x563ca670a9f0 2024-05-27T07:42:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.161+0000 7f4744a0cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f47301b11b0 con 0x563ca661ee40 2024-05-27T07:42:57.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.161+0000 7f4744a0cb80 10 client.0 _ll_register_callbacks cb 0x563ca6665c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:42:57.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.162+0000 7f472f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca67a82c0 0x563ca67bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.162+0000 7f472f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca67a82c0 0x563ca67bab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39818/0 (socket says 172.21.15.63:39818) 2024-05-27T07:42:57.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.162+0000 7f472f400700 1 -- 172.21.15.63:0/2823537940 learned_addr learned my addr 172.21.15.63:0/2823537940 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:42:57.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.162+0000 7f4727e00700 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca670a9f0 0x563ca67ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.162+0000 7f472ea00700 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca661ee40 0x563ca67bd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.162+0000 7f472f400700 1 -- 172.21.15.63:0/2823537940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca661ee40 msgr2=0x563ca67bd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:57.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.162+0000 7f472f400700 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca661ee40 0x563ca67bd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.162+0000 7f472f400700 1 -- 172.21.15.63:0/2823537940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca670a9f0 msgr2=0x563ca67ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:42:57.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.162+0000 7f472f400700 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca670a9f0 0x563ca67ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.162+0000 7f472f400700 1 -- 172.21.15.63:0/2823537940 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca67bb210 con 0x563ca67a82c0 2024-05-27T07:42:57.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.163+0000 7f472f400700 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca67a82c0 0x563ca67bab70 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f4728018e00 tx=0x7f4728009050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccd5835678252f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:57.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.163+0000 7f471d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:42:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.163+0000 7f471d600700 1 -- 172.21.15.63:0/2823537940 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472802b030 con 0x563ca67a82c0 2024-05-27T07:42:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.163+0000 7f471d600700 1 -- 172.21.15.63:0/2823537940 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f4728020040 con 0x563ca67a82c0 2024-05-27T07:42:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.163+0000 7f4744a0cb80 1 -- 172.21.15.63:0/2823537940 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47200aa7e0 con 0x563ca67a82c0 2024-05-27T07:42:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.164+0000 7f471d600700 1 -- 172.21.15.63:0/2823537940 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4728035050 con 0x563ca67a82c0 2024-05-27T07:42:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.164+0000 7f4744a0cb80 10 client.17250 fetch_fsmap learned FSMap version 33 2024-05-27T07:42:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.164+0000 7f4744a0cb80 1 -- 172.21.15.63:0/2823537940 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ca6821ef0 con 0x563ca67a82c0 2024-05-27T07:42:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.164+0000 7f471d600700 1 -- 172.21.15.63:0/2823537940 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47200aa7e0 con 0x563ca67a82c0 2024-05-27T07:42:57.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.164+0000 7f471d600700 1 -- 172.21.15.63:0/2823537940 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4728027910 con 0x563ca67a82c0 2024-05-27T07:42:57.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.165+0000 7f4744a0cb80 10 client.17250 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:42:57.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.165+0000 7f4744a0cb80 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:42:57.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.165+0000 7f4744a0cb80 20 client.17250 populate_metadata read hostname 'smithi063' 2024-05-27T07:42:57.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.165+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ca6822650 0x563ca6824a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:57.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.165+0000 7f4744a0cb80 4 client.17250 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:42:57.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.165+0000 7f4744a0cb80 1 -- 172.21.15.63:0/2823537940 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ca66c7570 con 0x563ca6822650 2024-05-27T07:42:57.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.165+0000 7f472ea00700 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ca6822650 0x563ca6824a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:57.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.167+0000 7f472ea00700 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ca6822650 0x563ca6824a70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=332870ad7b5a126d server_cookie=2aa860517df63b1d in_seq=0 out_seq=0 2024-05-27T07:42:57.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.167+0000 7f471d600700 10 client.17250 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:57.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.167+0000 7f471d600700 1 -- 172.21.15.63:0/2823537940 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563ca66c7570 con 0x563ca6822650 2024-05-27T07:42:57.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.167+0000 7f471d600700 10 client.17250 handle_command_reply: tid=0 2024-05-27T07:42:57.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.167+0000 7f4744a0cb80 1 client.17250 shutdown 2024-05-27T07:42:57.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 1 -- 172.21.15.63:0/2823537940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca67a82c0 msgr2=0x563ca67bab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:57.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca67a82c0 0x563ca67bab70 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f4728018e00 tx=0x7f4728009050 comp rx=0 tx=0).stop 2024-05-27T07:42:57.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 1 -- 172.21.15.63:0/2823537940 shutdown_connections 2024-05-27T07:42:57.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ca6822650 0x563ca6824a70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ca67a82c0 0x563ca67bab70 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ca661ee40 0x563ca67bd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 1 --2- 172.21.15.63:0/2823537940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ca670a9f0 0x563ca67ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 1 -- 172.21.15.63:0/2823537940 >> 172.21.15.63:0/2823537940 conn(0x563ca66b8390 msgr2=0x563ca67a4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f471d600700 0 client.17250 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:42:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 1 -- 172.21.15.63:0/2823537940 shutdown_connections 2024-05-27T07:42:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 1 -- 172.21.15.63:0/2823537940 wait complete. 2024-05-27T07:42:57.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.168+0000 7f4744a0cb80 20 client.17250 trim_cache size 0 max 16384 2024-05-27T07:42:57.169 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:42:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:42:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:42:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: "d01b63a0-cef4-493a-a2ba-670431910d28": { 2024-05-27T07:42:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:42:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "d01b63a0-cef4-493a-a2ba-670431910d28", 2024-05-27T07:42:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:42:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:42:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:42:57.169 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:42:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.169+0000 7f472fe00700 1 -- 172.21.15.63:0/72241206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f47140b2a70 msgr2=0x7f47140b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.169+0000 7f472fe00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f47140b2a70 0x7f47140b4f80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f47280060b0 tx=0x7f47280091e0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.169+0000 7f472fe00700 1 -- 172.21.15.63:0/72241206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4730105300 msgr2=0x7f4730102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.169+0000 7f472fe00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4730105300 0x7f4730102ff0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f47200057c0 tx=0x7f4720009f90 comp rx=0 tx=0).stop 2024-05-27T07:42:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.170+0000 7f472fe00700 1 -- 172.21.15.63:0/72241206 shutdown_connections 2024-05-27T07:42:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.170+0000 7f472fe00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f47140b2a70 0x7f47140b4f80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.170+0000 7f472fe00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473010e730 0x7f4730103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.170+0000 7f472fe00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4730105300 0x7f4730102ff0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.170+0000 7f472fe00700 1 --2- 172.21.15.63:0/72241206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4730105c50 0x7f4730103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:57.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.170+0000 7f472fe00700 1 -- 172.21.15.63:0/72241206 >> 172.21.15.63:0/72241206 conn(0x7f4730078ab0 msgr2=0x7f47300fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:57.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.170+0000 7f472fe00700 1 -- 172.21.15.63:0/72241206 shutdown_connections 2024-05-27T07:42:57.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:42:57.170+0000 7f472fe00700 1 -- 172.21.15.63:0/72241206 wait complete. 2024-05-27T07:42:58.039 INFO:tasks.cephfs.filesystem:scrub status for tag:d01b63a0-cef4-493a-a2ba-670431910d28 - {'path': '/', 'tag': 'd01b63a0-cef4-493a-a2ba-670431910d28', 'options': 'recursive,force'} 2024-05-27T07:42:58.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:42:58 smithi086 ceph-mon[34687]: pgmap v291: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.0 MiB/s wr, 40 op/s 2024-05-27T07:42:58.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:42:58 smithi149 ceph-mon[35408]: pgmap v291: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.0 MiB/s wr, 40 op/s 2024-05-27T07:42:58.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:42:58 smithi063 ceph-mon[31613]: pgmap v291: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.0 MiB/s wr, 40 op/s 2024-05-27T07:43:00.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:00 smithi086 ceph-mon[34687]: pgmap v292: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.0 MiB/s wr, 36 op/s 2024-05-27T07:43:00.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:00 smithi149 ceph-mon[35408]: pgmap v292: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.0 MiB/s wr, 36 op/s 2024-05-27T07:43:00.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:00 smithi063 ceph-mon[31613]: pgmap v292: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.0 MiB/s wr, 36 op/s 2024-05-27T07:43:01.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:01 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:43:02.071 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:01 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:43:02.071 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:01 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:43:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:43:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:43:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:02 smithi149 ceph-mon[35408]: pgmap v293: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.6 MiB/s wr, 57 op/s 2024-05-27T07:43:02.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:43:02.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:43:02.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:02 smithi063 ceph-mon[31613]: pgmap v293: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.6 MiB/s wr, 57 op/s 2024-05-27T07:43:03.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:43:03.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:43:03.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:02 smithi086 ceph-mon[34687]: pgmap v293: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.6 MiB/s wr, 57 op/s 2024-05-27T07:43:04.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:04 smithi063 ceph-mon[31613]: pgmap v294: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.9 MiB/s wr, 53 op/s 2024-05-27T07:43:05.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:04 smithi086 ceph-mon[34687]: pgmap v294: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.9 MiB/s wr, 53 op/s 2024-05-27T07:43:05.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:04 smithi149 ceph-mon[35408]: pgmap v294: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.9 MiB/s wr, 53 op/s 2024-05-27T07:43:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.505+0000 7fe71ac00640 1 -- 172.21.15.63:0/1423839555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71410e4e0 msgr2=0x7fe714110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.505+0000 7fe71ac00640 1 --2- 172.21.15.63:0/1423839555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71410e4e0 0x7fe714110900 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009ca0 tx=0x7fe70c02a8b0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.505+0000 7fe71ac00640 1 -- 172.21.15.63:0/1423839555 shutdown_connections 2024-05-27T07:43:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.505+0000 7fe71ac00640 1 --2- 172.21.15.63:0/1423839555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71410e4e0 0x7fe714110900 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.505+0000 7fe71ac00640 1 --2- 172.21.15.63:0/1423839555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe714101fe0 0x7fe7141023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.505+0000 7fe71ac00640 1 --2- 172.21.15.63:0/1423839555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714109b30 0x7fe714101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.505+0000 7fe71ac00640 1 -- 172.21.15.63:0/1423839555 >> 172.21.15.63:0/1423839555 conn(0x7fe7140fd950 msgr2=0x7fe7140ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.506+0000 7fe71ac00640 1 -- 172.21.15.63:0/1423839555 shutdown_connections 2024-05-27T07:43:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.506+0000 7fe71ac00640 1 -- 172.21.15.63:0/1423839555 wait complete. 2024-05-27T07:43:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.507+0000 7fe71ac00640 1 Processor -- start 2024-05-27T07:43:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.507+0000 7fe71ac00640 1 -- start start 2024-05-27T07:43:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe71ac00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe714101fe0 0x7fe714102c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe71ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714109b30 0x7fe7141031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe71ac00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71410e4e0 0x7fe714103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe71ac00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe714113ba0 con 0x7fe71410e4e0 2024-05-27T07:43:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe71ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe714113a20 con 0x7fe714109b30 2024-05-27T07:43:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe71ac00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe714113d20 con 0x7fe714101fe0 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe71a200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71410e4e0 0x7fe714103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe71a200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71410e4e0 0x7fe714103700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47370/0 (socket says 172.21.15.63:47370) 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe71a200640 1 -- 172.21.15.63:0/282035949 learned_addr learned my addr 172.21.15.63:0/282035949 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe719800640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe714101fe0 0x7fe714102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.508+0000 7fe718e00640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714109b30 0x7fe7141031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.509+0000 7fe71a200640 1 -- 172.21.15.63:0/282035949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe714101fe0 msgr2=0x7fe714102c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.509+0000 7fe71a200640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe714101fe0 0x7fe714102c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.509+0000 7fe71a200640 1 -- 172.21.15.63:0/282035949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714109b30 msgr2=0x7fe7141031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.509+0000 7fe71a200640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714109b30 0x7fe7141031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.509+0000 7fe71a200640 1 -- 172.21.15.63:0/282035949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7141a8900 con 0x7fe71410e4e0 2024-05-27T07:43:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.509+0000 7fe71a200640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71410e4e0 0x7fe714103700 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe70c005e30 tx=0x7fe70c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.509+0000 7fe712000640 1 -- 172.21.15.63:0/282035949 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c004040 con 0x7fe71410e4e0 2024-05-27T07:43:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.510+0000 7fe71ac00640 1 -- 172.21.15.63:0/282035949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7141a8b90 con 0x7fe71410e4e0 2024-05-27T07:43:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.510+0000 7fe712000640 1 -- 172.21.15.63:0/282035949 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe70c040070 con 0x7fe71410e4e0 2024-05-27T07:43:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.510+0000 7fe712000640 1 -- 172.21.15.63:0/282035949 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c045ae0 con 0x7fe71410e4e0 2024-05-27T07:43:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.510+0000 7fe71ac00640 1 -- 172.21.15.63:0/282035949 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7141a9130 con 0x7fe71410e4e0 2024-05-27T07:43:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.514+0000 7fe7231a4740 1 Processor -- start 2024-05-27T07:43:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.514+0000 7fe712000640 1 -- 172.21.15.63:0/282035949 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe70c045c80 con 0x7fe71410e4e0 2024-05-27T07:43:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.514+0000 7fe7231a4740 1 -- start start 2024-05-27T07:43:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.515+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea90e1b0 0x5629eacb0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.515+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8e52f0 0x5629eacb0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.515+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacb14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.515+0000 7fe7231a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5629eac21b90 con 0x5629ea8e52f0 2024-05-27T07:43:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.515+0000 7fe7231a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629eac11730 con 0x5629ea8ed4d0 2024-05-27T07:43:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.515+0000 7fe7231a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5629eac15770 con 0x5629ea90e1b0 2024-05-27T07:43:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.515+0000 7fe719800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8e52f0 0x5629eacb0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.515+0000 7fe719800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8e52f0 0x5629eacb0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47384/0 (socket says 172.21.15.63:47384) 2024-05-27T07:43:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.515+0000 7fe719800640 1 -- 172.21.15.63:0/2309317161 learned_addr learned my addr 172.21.15.63:0/2309317161 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.515+0000 7fe718e00640 1 --2- 172.21.15.63:0/2309317161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea90e1b0 0x5629eacb0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.516+0000 7fe718e00640 1 --2- 172.21.15.63:0/2309317161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacb14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.516+0000 7fe719800640 1 -- 172.21.15.63:0/2309317161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea90e1b0 msgr2=0x5629eacb0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.516+0000 7fe719800640 1 --2- 172.21.15.63:0/2309317161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea90e1b0 0x5629eacb0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.516+0000 7fe719800640 1 -- 172.21.15.63:0/2309317161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 msgr2=0x5629eacb14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.516+0000 7fe719800640 1 --2- 172.21.15.63:0/2309317161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacb14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.516+0000 7fe719800640 1 -- 172.21.15.63:0/2309317161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629eabaa8b0 con 0x5629ea8e52f0 2024-05-27T07:43:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.516+0000 7fe718e00640 1 --2- 172.21.15.63:0/2309317161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacb14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.516+0000 7fe712000640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6fc0b29d0 0x7fe6fc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.516+0000 7fe719800640 1 --2- 172.21.15.63:0/2309317161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8e52f0 0x5629eacb0fa0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe70800dbf0 tx=0x7fe70800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1c94e71675ef4b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.517+0000 7fe712000640 1 -- 172.21.15.63:0/282035949 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fe70c103b10 con 0x7fe71410e4e0 2024-05-27T07:43:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.517+0000 7fe718e00640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6fc0b29d0 0x7fe6fc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.517+0000 7fe706a00640 1 -- 172.21.15.63:0/2309317161 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708005020 con 0x5629ea8e52f0 2024-05-27T07:43:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.517+0000 7fe706a00640 1 -- 172.21.15.63:0/2309317161 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7080047a0 con 0x5629ea8e52f0 2024-05-27T07:43:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.517+0000 7fe706a00640 1 -- 172.21.15.63:0/2309317161 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708005650 con 0x5629ea8e52f0 2024-05-27T07:43:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.518+0000 7fe7231a4740 1 -- 172.21.15.63:0/2309317161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8e52f0 msgr2=0x5629eacb0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.518+0000 7fe7231a4740 1 --2- 172.21.15.63:0/2309317161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8e52f0 0x5629eacb0fa0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe70800dbf0 tx=0x7fe70800f730 comp rx=0 tx=0).stop 2024-05-27T07:43:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.518+0000 7fe718e00640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6fc0b29d0 0x7fe6fc0b4ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe70000cfd0 tx=0x7fe7000046c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.518+0000 7fe7231a4740 1 -- 172.21.15.63:0/2309317161 shutdown_connections 2024-05-27T07:43:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.518+0000 7fe7231a4740 1 --2- 172.21.15.63:0/2309317161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacb14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.518+0000 7fe7231a4740 1 --2- 172.21.15.63:0/2309317161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8e52f0 0x5629eacb0fa0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.518+0000 7fe7231a4740 1 --2- 172.21.15.63:0/2309317161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea90e1b0 0x5629eacb0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.518+0000 7fe7231a4740 1 -- 172.21.15.63:0/2309317161 >> 172.21.15.63:0/2309317161 conn(0x5629ea902300 msgr2=0x5629eacad210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.518+0000 7fe7231a4740 1 -- 172.21.15.63:0/2309317161 shutdown_connections 2024-05-27T07:43:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.518+0000 7fe7231a4740 1 -- 172.21.15.63:0/2309317161 wait complete. 2024-05-27T07:43:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.519+0000 7fe7231a4740 1 -- start start 2024-05-27T07:43:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.519+0000 7fe7231a4740 1 Processor -- start 2024-05-27T07:43:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea8e52f0 0x5629eacbac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacbb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacc6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe7231a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5629ea8f7790 con 0x5629ea90e1b0 2024-05-27T07:43:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe7231a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629eabb9bc0 con 0x5629ea8ed4d0 2024-05-27T07:43:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe7231a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5629eacbdb90 con 0x5629ea8e52f0 2024-05-27T07:43:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe7231a4740 10 client.0 _ll_register_callbacks cb 0x5629eab97400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe71a200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacc6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe71a200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacc6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47400/0 (socket says 172.21.15.63:47400) 2024-05-27T07:43:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe71a200640 1 -- 172.21.15.63:0/3141086625 learned_addr learned my addr 172.21.15.63:0/3141086625 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe718e00640 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacbb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.520+0000 7fe719800640 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea8e52f0 0x5629eacbac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.521+0000 7fe71a200640 1 -- 172.21.15.63:0/3141086625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea8e52f0 msgr2=0x5629eacbac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.521+0000 7fe71a200640 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea8e52f0 0x5629eacbac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.521+0000 7fe71a200640 1 -- 172.21.15.63:0/3141086625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 msgr2=0x5629eacbb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.521+0000 7fe71a200640 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacbb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.521+0000 7fe71a200640 1 -- 172.21.15.63:0/3141086625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ea926210 con 0x5629ea90e1b0 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.521+0000 7fe71a200640 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacc6f20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0c6bc0 tx=0x7fe70c045470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f513742fcc193bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.521+0000 7fe6fb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.521+0000 7fe6fb400640 1 -- 172.21.15.63:0/3141086625 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0c6a00 con 0x5629ea90e1b0 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.521+0000 7fe6fb400640 1 -- 172.21.15.63:0/3141086625 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fe70c0cada0 con 0x5629ea90e1b0 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.521+0000 7fe7231a4740 1 -- 172.21.15.63:0/3141086625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7080047a0 con 0x5629ea90e1b0 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe6fb400640 1 -- 172.21.15.63:0/3141086625 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe70c0cab40 con 0x5629ea90e1b0 2024-05-27T07:43:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe7231a4740 10 client.17262 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe7231a4740 1 -- 172.21.15.63:0/3141086625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5629ea9f2620 con 0x5629ea90e1b0 2024-05-27T07:43:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe6fb400640 1 -- 172.21.15.63:0/3141086625 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7080047a0 con 0x5629ea90e1b0 2024-05-27T07:43:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe6fb400640 1 -- 172.21.15.63:0/3141086625 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe70c04d030 con 0x5629ea90e1b0 2024-05-27T07:43:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe7231a4740 10 client.17262 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe7231a4740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe7231a4740 20 client.17262 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe7231a4740 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ea8ccd80 0x5629eacccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe7231a4740 4 client.17262 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.522+0000 7fe7231a4740 1 -- 172.21.15.63:0/3141086625 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629eac23ae0 con 0x5629ea8ccd80 2024-05-27T07:43:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.523+0000 7fe719800640 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ea8ccd80 0x5629eacccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.524+0000 7fe719800640 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ea8ccd80 0x5629eacccec0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b16567e42ce275 server_cookie=34a6127f337950a9 in_seq=0 out_seq=0 2024-05-27T07:43:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.524+0000 7fe6fb400640 10 client.17262 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe6fb400640 1 -- 172.21.15.63:0/3141086625 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5629eac23ae0 con 0x5629ea8ccd80 2024-05-27T07:43:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe6fb400640 10 client.17262 handle_command_reply: tid=0 2024-05-27T07:43:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe7231a4740 1 client.17262 shutdown 2024-05-27T07:43:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe7231a4740 1 -- 172.21.15.63:0/3141086625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 msgr2=0x5629eacc6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe7231a4740 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacc6f20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0c6bc0 tx=0x7fe70c045470 comp rx=0 tx=0).stop 2024-05-27T07:43:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe7231a4740 1 -- 172.21.15.63:0/3141086625 shutdown_connections 2024-05-27T07:43:05.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe7231a4740 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ea8ccd80 0x5629eacccec0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe7231a4740 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacc6f20 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe7231a4740 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacbb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe7231a4740 1 --2- 172.21.15.63:0/3141086625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea8e52f0 0x5629eacbac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe7231a4740 1 -- 172.21.15.63:0/3141086625 >> 172.21.15.63:0/3141086625 conn(0x5629ea902300 msgr2=0x5629eacb0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.526+0000 7fe6fb400640 0 client.17262 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.527+0000 7fe7231a4740 1 -- 172.21.15.63:0/3141086625 shutdown_connections 2024-05-27T07:43:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.527+0000 7fe7231a4740 1 -- 172.21.15.63:0/3141086625 wait complete. 2024-05-27T07:43:05.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.527+0000 7fe7231a4740 20 client.17262 trim_cache size 0 max 16384 2024-05-27T07:43:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.547+0000 7fe7231a4740 1 Processor -- start 2024-05-27T07:43:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.547+0000 7fe7231a4740 1 -- start start 2024-05-27T07:43:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8e52f0 0x5629eacbd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8ed4d0 0x5629eacac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea90e1b0 0x5629eacc6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe7231a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141b5ab0 con 0x5629ea8ed4d0 2024-05-27T07:43:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe7231a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141b3380 con 0x5629ea8e52f0 2024-05-27T07:43:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe7231a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141b11f0 con 0x5629ea90e1b0 2024-05-27T07:43:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe718e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8ed4d0 0x5629eacac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe718e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8ed4d0 0x5629eacac2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47402/0 (socket says 172.21.15.63:47402) 2024-05-27T07:43:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe718e00640 1 -- 172.21.15.63:0/2129347471 learned_addr learned my addr 172.21.15.63:0/2129347471 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe719800640 1 --2- 172.21.15.63:0/2129347471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8e52f0 0x5629eacbd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.548+0000 7fe71a200640 1 --2- 172.21.15.63:0/2129347471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea90e1b0 0x5629eacc6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.549+0000 7fe718e00640 1 -- 172.21.15.63:0/2129347471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea90e1b0 msgr2=0x5629eacc6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.549+0000 7fe718e00640 1 --2- 172.21.15.63:0/2129347471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea90e1b0 0x5629eacc6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.549+0000 7fe718e00640 1 -- 172.21.15.63:0/2129347471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8e52f0 msgr2=0x5629eacbd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.549+0000 7fe718e00640 1 --2- 172.21.15.63:0/2129347471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8e52f0 0x5629eacbd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.549+0000 7fe718e00640 1 -- 172.21.15.63:0/2129347471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629eacadc00 con 0x5629ea8ed4d0 2024-05-27T07:43:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.549+0000 7fe718e00640 1 --2- 172.21.15.63:0/2129347471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8ed4d0 0x5629eacac2e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe700011790 tx=0x7fe700014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5031f640bfa997b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.550+0000 7fe706000640 1 -- 172.21.15.63:0/2129347471 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70001f8d0 con 0x5629ea8ed4d0 2024-05-27T07:43:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.550+0000 7fe706000640 1 -- 172.21.15.63:0/2129347471 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe70002b480 con 0x5629ea8ed4d0 2024-05-27T07:43:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.550+0000 7fe706000640 1 -- 172.21.15.63:0/2129347471 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7000206a0 con 0x5629ea8ed4d0 2024-05-27T07:43:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.551+0000 7fe7231a4740 1 -- 172.21.15.63:0/2129347471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8ed4d0 msgr2=0x5629eacac2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.551+0000 7fe7231a4740 1 --2- 172.21.15.63:0/2129347471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8ed4d0 0x5629eacac2e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe700011790 tx=0x7fe700014040 comp rx=0 tx=0).stop 2024-05-27T07:43:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.551+0000 7fe7231a4740 1 -- 172.21.15.63:0/2129347471 shutdown_connections 2024-05-27T07:43:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.551+0000 7fe7231a4740 1 --2- 172.21.15.63:0/2129347471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea90e1b0 0x5629eacc6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.551+0000 7fe7231a4740 1 --2- 172.21.15.63:0/2129347471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea8ed4d0 0x5629eacac2e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.551+0000 7fe7231a4740 1 --2- 172.21.15.63:0/2129347471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8e52f0 0x5629eacbd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.551+0000 7fe7231a4740 1 -- 172.21.15.63:0/2129347471 >> 172.21.15.63:0/2129347471 conn(0x5629ea902300 msgr2=0x5629eacb1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.551+0000 7fe7231a4740 1 -- 172.21.15.63:0/2129347471 shutdown_connections 2024-05-27T07:43:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.551+0000 7fe7231a4740 1 -- 172.21.15.63:0/2129347471 wait complete. 2024-05-27T07:43:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.552+0000 7fe7231a4740 1 -- start start 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.552+0000 7fe7231a4740 1 Processor -- start 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.552+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea8e52f0 0x5629eacc35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.552+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacbe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.552+0000 7fe7231a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacbee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.552+0000 7fe7231a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141b1840 con 0x5629ea90e1b0 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.552+0000 7fe7231a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141b2cb0 con 0x5629ea8ed4d0 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.552+0000 7fe7231a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141b1f10 con 0x5629ea8e52f0 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.552+0000 7fe7231a4740 10 client.0 _ll_register_callbacks cb 0x7fe70c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.553+0000 7fe71a200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacbee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.553+0000 7fe71a200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacbee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47410/0 (socket says 172.21.15.63:47410) 2024-05-27T07:43:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.553+0000 7fe71a200640 1 -- 172.21.15.63:0/669284030 learned_addr learned my addr 172.21.15.63:0/669284030 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.553+0000 7fe718e00640 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacbe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.553+0000 7fe719800640 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea8e52f0 0x5629eacc35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.553+0000 7fe71a200640 1 -- 172.21.15.63:0/669284030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea8e52f0 msgr2=0x5629eacc35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.553+0000 7fe71a200640 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea8e52f0 0x5629eacc35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.553+0000 7fe71a200640 1 -- 172.21.15.63:0/669284030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 msgr2=0x5629eacbe900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.553+0000 7fe71a200640 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacbe900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.553+0000 7fe71a200640 1 -- 172.21.15.63:0/669284030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629eacbf5b0 con 0x5629ea90e1b0 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.554+0000 7fe71a200640 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacbee40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0c7db0 tx=0x7fe70c02a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698cdce517c2bdb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.554+0000 7fe6fb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.554+0000 7fe6fb400640 1 -- 172.21.15.63:0/669284030 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0c6a00 con 0x5629ea90e1b0 2024-05-27T07:43:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.554+0000 7fe6fb400640 1 -- 172.21.15.63:0/669284030 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fe70c004d50 con 0x5629ea90e1b0 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.554+0000 7fe7231a4740 1 -- 172.21.15.63:0/669284030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70002b480 con 0x5629ea90e1b0 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.554+0000 7fe6fb400640 1 -- 172.21.15.63:0/669284030 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe70c0ca490 con 0x5629ea90e1b0 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.554+0000 7fe7231a4740 10 client.17268 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.554+0000 7fe7231a4740 1 -- 172.21.15.63:0/669284030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5629eacbf5b0 con 0x5629ea90e1b0 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.554+0000 7fe6fb400640 1 -- 172.21.15.63:0/669284030 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe70002b480 con 0x5629ea90e1b0 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.555+0000 7fe6fb400640 1 -- 172.21.15.63:0/669284030 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe70c04d040 con 0x5629ea90e1b0 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.555+0000 7fe7231a4740 10 client.17268 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.555+0000 7fe7231a4740 10 client.17268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.555+0000 7fe7231a4740 20 client.17268 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.555+0000 7fe7231a4740 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ea8a21c0 0x5629ead37900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.555+0000 7fe7231a4740 4 client.17268 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:43:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.555+0000 7fe7231a4740 1 -- 172.21.15.63:0/669284030 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629eac1d2c0 con 0x5629ea8a21c0 2024-05-27T07:43:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.555+0000 7fe719800640 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ea8a21c0 0x5629ead37900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.556+0000 7fe719800640 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ea8a21c0 0x5629ead37900 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d52acf06865c0cf server_cookie=6c5f7584acbc7d35 in_seq=0 out_seq=0 2024-05-27T07:43:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.556+0000 7fe6fb400640 10 client.17268 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.592+0000 7fe6fb400640 1 -- 172.21.15.63:0/669284030 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5629eac1d2c0 con 0x5629ea8a21c0 2024-05-27T07:43:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.592+0000 7fe6fb400640 10 client.17268 handle_command_reply: tid=0 2024-05-27T07:43:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.592+0000 7fe7231a4740 1 client.17268 shutdown 2024-05-27T07:43:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.592+0000 7fe7231a4740 1 -- 172.21.15.63:0/669284030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 msgr2=0x5629eacbee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.592+0000 7fe7231a4740 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacbee40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0c7db0 tx=0x7fe70c02a8b0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.593+0000 7fe7231a4740 1 -- 172.21.15.63:0/669284030 shutdown_connections 2024-05-27T07:43:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.593+0000 7fe7231a4740 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ea8a21c0 0x5629ead37900 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.593+0000 7fe7231a4740 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ea90e1b0 0x5629eacbee40 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.593+0000 7fe7231a4740 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ea8ed4d0 0x5629eacbe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.593+0000 7fe7231a4740 1 --2- 172.21.15.63:0/669284030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ea8e52f0 0x5629eacc35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.593+0000 7fe7231a4740 1 -- 172.21.15.63:0/669284030 >> 172.21.15.63:0/669284030 conn(0x5629ea902300 msgr2=0x5629eacbae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.593+0000 7fe6fb400640 0 client.17268 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.593+0000 7fe7231a4740 1 -- 172.21.15.63:0/669284030 shutdown_connections 2024-05-27T07:43:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.593+0000 7fe7231a4740 1 -- 172.21.15.63:0/669284030 wait complete. 2024-05-27T07:43:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.593+0000 7fe7231a4740 20 client.17268 trim_cache size 0 max 16384 2024-05-27T07:43:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:43:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:43:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:43:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 -- 172.21.15.63:0/282035949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6fc0b29d0 msgr2=0x7fe6fc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6fc0b29d0 0x7fe6fc0b4ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe70000cfd0 tx=0x7fe7000046c0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 -- 172.21.15.63:0/282035949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71410e4e0 msgr2=0x7fe714103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71410e4e0 0x7fe714103700 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe70c005e30 tx=0x7fe70c003970 comp rx=0 tx=0).stop 2024-05-27T07:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 -- 172.21.15.63:0/282035949 shutdown_connections 2024-05-27T07:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6fc0b29d0 0x7fe6fc0b4ee0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71410e4e0 0x7fe714103700 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe714109b30 0x7fe7141031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 --2- 172.21.15.63:0/282035949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe714101fe0 0x7fe714102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 -- 172.21.15.63:0/282035949 >> 172.21.15.63:0/282035949 conn(0x7fe7140fd950 msgr2=0x7fe71410a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 -- 172.21.15.63:0/282035949 shutdown_connections 2024-05-27T07:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:05.594+0000 7fe71ac00640 1 -- 172.21.15.63:0/282035949 wait complete. 2024-05-27T07:43:07.011 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:06 smithi086 ceph-mon[34687]: pgmap v295: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.9 MiB/s wr, 53 op/s 2024-05-27T07:43:07.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:06 smithi149 ceph-mon[35408]: pgmap v295: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.9 MiB/s wr, 53 op/s 2024-05-27T07:43:07.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:06 smithi063 ceph-mon[31613]: pgmap v295: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.9 MiB/s wr, 53 op/s 2024-05-27T07:43:08.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:07 smithi086 ceph-mon[34687]: osdmap e100: 12 total, 12 up, 12 in 2024-05-27T07:43:08.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:07 smithi149 ceph-mon[35408]: osdmap e100: 12 total, 12 up, 12 in 2024-05-27T07:43:08.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:07 smithi063 ceph-mon[31613]: osdmap e100: 12 total, 12 up, 12 in 2024-05-27T07:43:09.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:08 smithi086 ceph-mon[34687]: pgmap v297: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 32 op/s 2024-05-27T07:43:09.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:08 smithi149 ceph-mon[35408]: pgmap v297: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 32 op/s 2024-05-27T07:43:09.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:08 smithi063 ceph-mon[31613]: pgmap v297: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 32 op/s 2024-05-27T07:43:11.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:10 smithi086 ceph-mon[34687]: pgmap v298: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 32 op/s 2024-05-27T07:43:11.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:10 smithi149 ceph-mon[35408]: pgmap v298: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 32 op/s 2024-05-27T07:43:11.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:10 smithi063 ceph-mon[31613]: pgmap v298: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 32 op/s 2024-05-27T07:43:13.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:12 smithi086 ceph-mon[34687]: pgmap v299: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T07:43:13.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:12 smithi149 ceph-mon[35408]: pgmap v299: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T07:43:13.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:12 smithi063 ceph-mon[31613]: pgmap v299: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T07:43:14.108 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:13 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:43:14.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:13 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:43:14.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:13 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:43:15.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:14 smithi086 ceph-mon[34687]: pgmap v300: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T07:43:15.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:43:15.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:43:15.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:43:15.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:14 smithi149 ceph-mon[35408]: pgmap v300: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T07:43:15.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:43:15.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:43:15.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:43:15.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:14 smithi063 ceph-mon[31613]: pgmap v300: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T07:43:15.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:43:15.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:43:15.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:43:16.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:16 smithi086 ceph-mon[34687]: osdmap e101: 12 total, 12 up, 12 in 2024-05-27T07:43:16.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:16 smithi149 ceph-mon[35408]: osdmap e101: 12 total, 12 up, 12 in 2024-05-27T07:43:16.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:16 smithi063 ceph-mon[31613]: osdmap e101: 12 total, 12 up, 12 in 2024-05-27T07:43:17.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:17 smithi086 ceph-mon[34687]: pgmap v302: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-05-27T07:43:17.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:17 smithi149 ceph-mon[35408]: pgmap v302: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-05-27T07:43:17.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:17 smithi063 ceph-mon[31613]: pgmap v302: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-05-27T07:43:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.834+0000 7ffa55800640 1 -- 172.21.15.63:0/3217614502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5010c190 msgr2=0x7ffa50105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.834+0000 7ffa55800640 1 --2- 172.21.15.63:0/3217614502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5010c190 0x7ffa50105dd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009900 tx=0x7ffa3c02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.835+0000 7ffa55800640 1 -- 172.21.15.63:0/3217614502 shutdown_connections 2024-05-27T07:43:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.835+0000 7ffa55800640 1 --2- 172.21.15.63:0/3217614502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa50106440 0x7ffa50112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.835+0000 7ffa55800640 1 --2- 172.21.15.63:0/3217614502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa5010c190 0x7ffa50105dd0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.835+0000 7ffa55800640 1 --2- 172.21.15.63:0/3217614502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5010bcf0 0x7ffa50105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.835+0000 7ffa55800640 1 -- 172.21.15.63:0/3217614502 >> 172.21.15.63:0/3217614502 conn(0x7ffa5007d510 msgr2=0x7ffa5007d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.835+0000 7ffa55800640 1 -- 172.21.15.63:0/3217614502 shutdown_connections 2024-05-27T07:43:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.835+0000 7ffa55800640 1 -- 172.21.15.63:0/3217614502 wait complete. 2024-05-27T07:43:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.836+0000 7ffa55800640 1 Processor -- start 2024-05-27T07:43:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.836+0000 7ffa55800640 1 -- start start 2024-05-27T07:43:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.837+0000 7ffa55800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa50106440 0x7ffa501a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.837+0000 7ffa55800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5010bcf0 0x7ffa501a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.837+0000 7ffa55800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5010c190 0x7ffa5019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.837+0000 7ffa55800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50115d70 con 0x7ffa50106440 2024-05-27T07:43:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.837+0000 7ffa55800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50115bf0 con 0x7ffa5010c190 2024-05-27T07:43:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.837+0000 7ffa55800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50115ef0 con 0x7ffa5010bcf0 2024-05-27T07:43:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.837+0000 7ffa4fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa50106440 0x7ffa501a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.837+0000 7ffa4fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa50106440 0x7ffa501a2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46490/0 (socket says 172.21.15.63:46490) 2024-05-27T07:43:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.837+0000 7ffa4fe00640 1 -- 172.21.15.63:0/416363001 learned_addr learned my addr 172.21.15.63:0/416363001 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.838+0000 7ffa4f400640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5010bcf0 0x7ffa501a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.838+0000 7ffa54e00640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5010c190 0x7ffa5019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.838+0000 7ffa4fe00640 1 -- 172.21.15.63:0/416363001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5010bcf0 msgr2=0x7ffa501a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.838+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5010bcf0 0x7ffa501a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.838+0000 7ffa4fe00640 1 -- 172.21.15.63:0/416363001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5010c190 msgr2=0x7ffa5019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.838+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5010c190 0x7ffa5019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.838+0000 7ffa4fe00640 1 -- 172.21.15.63:0/416363001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5019cdc0 con 0x7ffa50106440 2024-05-27T07:43:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.838+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa50106440 0x7ffa501a2510 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000d770 tx=0x7ffa4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.839+0000 7ffa4cc00640 1 -- 172.21.15.63:0/416363001 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40017070 con 0x7ffa50106440 2024-05-27T07:43:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.839+0000 7ffa4cc00640 1 -- 172.21.15.63:0/416363001 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa40009c30 con 0x7ffa50106440 2024-05-27T07:43:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.839+0000 7ffa55800640 1 -- 172.21.15.63:0/416363001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5019d050 con 0x7ffa50106440 2024-05-27T07:43:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.839+0000 7ffa4cc00640 1 -- 172.21.15.63:0/416363001 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4000f620 con 0x7ffa50106440 2024-05-27T07:43:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.839+0000 7ffa55800640 1 -- 172.21.15.63:0/416363001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa501a92b0 con 0x7ffa50106440 2024-05-27T07:43:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.843+0000 7ffa5dd99740 1 Processor -- start 2024-05-27T07:43:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.844+0000 7ffa5dd99740 1 -- start start 2024-05-27T07:43:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.844+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x56250053ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.844+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001712f0 0x56250053cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.844+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001794d0 0x56250053d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.844+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625004adb90 con 0x5625001794d0 2024-05-27T07:43:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.844+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56250049d730 con 0x56250019a1b0 2024-05-27T07:43:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.844+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5625004a1770 con 0x5625001712f0 2024-05-27T07:43:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.845+0000 7ffa4cc00640 1 -- 172.21.15.63:0/416363001 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffa4001a070 con 0x7ffa50106440 2024-05-27T07:43:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.845+0000 7ffa4f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001794d0 0x56250053d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.845+0000 7ffa4f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001794d0 0x56250053d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46494/0 (socket says 172.21.15.63:46494) 2024-05-27T07:43:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.845+0000 7ffa4f400640 1 -- 172.21.15.63:0/1974951134 learned_addr learned my addr 172.21.15.63:0/1974951134 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.845+0000 7ffa54e00640 1 --2- 172.21.15.63:0/1974951134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x56250053ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.845+0000 7ffa4f400640 1 -- 172.21.15.63:0/1974951134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001712f0 msgr2=0x56250053cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.845+0000 7ffa4f400640 1 --2- 172.21.15.63:0/1974951134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001712f0 0x56250053cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.845+0000 7ffa4f400640 1 -- 172.21.15.63:0/1974951134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 msgr2=0x56250053ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.845+0000 7ffa4f400640 1 --2- 172.21.15.63:0/1974951134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x56250053ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.845+0000 7ffa4f400640 1 -- 172.21.15.63:0/1974951134 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625004368b0 con 0x5625001794d0 2024-05-27T07:43:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.846+0000 7ffa4f400640 1 --2- 172.21.15.63:0/1974951134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001794d0 0x56250053d4e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c000c00 tx=0x7ffa3c032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ad7748e63244d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.846+0000 7ffa45600640 1 -- 172.21.15.63:0/1974951134 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c03f070 con 0x5625001794d0 2024-05-27T07:43:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.846+0000 7ffa45600640 1 -- 172.21.15.63:0/1974951134 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa3c004030 con 0x5625001794d0 2024-05-27T07:43:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.846+0000 7ffa45600640 1 -- 172.21.15.63:0/1974951134 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c035070 con 0x5625001794d0 2024-05-27T07:43:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.847+0000 7ffa4cc00640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa340b29d0 0x7ffa340b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.847+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1974951134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001794d0 msgr2=0x56250053d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.847+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1974951134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001794d0 0x56250053d4e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c000c00 tx=0x7ffa3c032bc0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.847+0000 7ffa4cc00640 1 -- 172.21.15.63:0/416363001 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7ffa400e2510 con 0x7ffa50106440 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.847+0000 7ffa54e00640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa340b29d0 0x7ffa340b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.847+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1974951134 shutdown_connections 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.847+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1974951134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001794d0 0x56250053d4e0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.847+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1974951134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001712f0 0x56250053cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.847+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1974951134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x56250053ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.847+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1974951134 >> 172.21.15.63:0/1974951134 conn(0x56250018e300 msgr2=0x562500539210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.848+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1974951134 shutdown_connections 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.848+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1974951134 wait complete. 2024-05-27T07:43:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.848+0000 7ffa54e00640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa340b29d0 0x7ffa340b4ee0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x562500539bd0 tx=0x7ffa48009740 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.848+0000 7ffa5dd99740 1 -- start start 2024-05-27T07:43:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.849+0000 7ffa5dd99740 1 Processor -- start 2024-05-27T07:43:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.849+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x562500546be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.849+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001794d0 0x562500547120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.849+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x562500552e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.849+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562500183790 con 0x5625001712f0 2024-05-27T07:43:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.849+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562500445bc0 con 0x56250019a1b0 2024-05-27T07:43:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.849+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562500549b30 con 0x5625001794d0 2024-05-27T07:43:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.849+0000 7ffa5dd99740 10 client.0 _ll_register_callbacks cb 0x562500423400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.850+0000 7ffa4fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x562500546be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.850+0000 7ffa4fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x562500546be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46502/0 (socket says 172.21.15.63:46502) 2024-05-27T07:43:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.850+0000 7ffa4fe00640 1 -- 172.21.15.63:0/1366398719 learned_addr learned my addr 172.21.15.63:0/1366398719 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.850+0000 7ffa4f400640 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001794d0 0x562500547120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.850+0000 7ffa54e00640 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x562500552e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.850+0000 7ffa4fe00640 1 -- 172.21.15.63:0/1366398719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001794d0 msgr2=0x562500547120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.850+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001794d0 0x562500547120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.850+0000 7ffa4fe00640 1 -- 172.21.15.63:0/1366398719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 msgr2=0x562500552e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.850+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x562500552e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.850+0000 7ffa4fe00640 1 -- 172.21.15.63:0/1366398719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625001b2210 con 0x5625001712f0 2024-05-27T07:43:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.851+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x562500546be0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ffa400a5450 tx=0x7ffa40010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b29213de5e7787 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.851+0000 7ffa3a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.851+0000 7ffa3a000640 1 -- 172.21.15.63:0/1366398719 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40021050 con 0x5625001712f0 2024-05-27T07:43:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.851+0000 7ffa3a000640 1 -- 172.21.15.63:0/1366398719 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7ffa400e2130 con 0x5625001712f0 2024-05-27T07:43:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.851+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1366398719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa3c004030 con 0x5625001712f0 2024-05-27T07:43:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.851+0000 7ffa3a000640 1 -- 172.21.15.63:0/1366398719 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ffa400dfcd0 con 0x5625001712f0 2024-05-27T07:43:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.851+0000 7ffa5dd99740 10 client.17280 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.851+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1366398719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56250027e620 con 0x5625001712f0 2024-05-27T07:43:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.852+0000 7ffa3a000640 1 -- 172.21.15.63:0/1366398719 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa3c004030 con 0x5625001712f0 2024-05-27T07:43:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.852+0000 7ffa3a000640 1 -- 172.21.15.63:0/1366398719 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ffa400a3af0 con 0x5625001712f0 2024-05-27T07:43:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.852+0000 7ffa5dd99740 10 client.17280 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.852+0000 7ffa5dd99740 10 client.17280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.852+0000 7ffa5dd99740 20 client.17280 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.852+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562500158d80 0x562500558d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.852+0000 7ffa5dd99740 4 client.17280 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.852+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1366398719 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625004afae0 con 0x562500158d80 2024-05-27T07:43:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.853+0000 7ffa4f400640 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562500158d80 0x562500558d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.854+0000 7ffa3a000640 10 client.17280 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.854+0000 7ffa4f400640 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562500158d80 0x562500558d90 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23ad8f31a07260b4 server_cookie=b5456e616d9093d0 in_seq=0 out_seq=0 2024-05-27T07:43:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa3a000640 1 -- 172.21.15.63:0/1366398719 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5625004afae0 con 0x562500158d80 2024-05-27T07:43:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa3a000640 10 client.17280 handle_command_reply: tid=0 2024-05-27T07:43:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 client.17280 shutdown 2024-05-27T07:43:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1366398719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 msgr2=0x562500546be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x562500546be0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ffa400a5450 tx=0x7ffa40010e40 comp rx=0 tx=0).stop 2024-05-27T07:43:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1366398719 shutdown_connections 2024-05-27T07:43:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562500158d80 0x562500558d90 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x562500552e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001794d0 0x562500547120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1366398719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x562500546be0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa3a000640 0 client.17280 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1366398719 >> 172.21.15.63:0/1366398719 conn(0x56250018e300 msgr2=0x56250053e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1366398719 shutdown_connections 2024-05-27T07:43:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1366398719 wait complete. 2024-05-27T07:43:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.856+0000 7ffa5dd99740 20 client.17280 trim_cache size 0 max 16384 2024-05-27T07:43:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.880+0000 7ffa5dd99740 1 Processor -- start 2024-05-27T07:43:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.880+0000 7ffa5dd99740 1 -- start start 2024-05-27T07:43:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.880+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x5625005423a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.880+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001794d0 0x562500548d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.880+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x562500546b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.881+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b5b00 con 0x5625001712f0 2024-05-27T07:43:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.881+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b33d0 con 0x56250019a1b0 2024-05-27T07:43:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.881+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b1240 con 0x5625001794d0 2024-05-27T07:43:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.881+0000 7ffa4fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x5625005423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.881+0000 7ffa4fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x5625005423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46510/0 (socket says 172.21.15.63:46510) 2024-05-27T07:43:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.881+0000 7ffa4fe00640 1 -- 172.21.15.63:0/1730152473 learned_addr learned my addr 172.21.15.63:0/1730152473 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.881+0000 7ffa54e00640 1 --2- 172.21.15.63:0/1730152473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x562500546b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.881+0000 7ffa4f400640 1 --2- 172.21.15.63:0/1730152473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001794d0 0x562500548d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.882+0000 7ffa4fe00640 1 -- 172.21.15.63:0/1730152473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001794d0 msgr2=0x562500548d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.882+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/1730152473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001794d0 0x562500548d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.882+0000 7ffa4fe00640 1 -- 172.21.15.63:0/1730152473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 msgr2=0x562500546b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.882+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/1730152473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x562500546b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.882+0000 7ffa4fe00640 1 -- 172.21.15.63:0/1730152473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562500539e30 con 0x5625001712f0 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.882+0000 7ffa54e00640 1 --2- 172.21.15.63:0/1730152473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x562500546b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.882+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/1730152473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x5625005423a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ffa400a3f10 tx=0x7ffa40009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4ed9995c59c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.882+0000 7ffa44c00640 1 -- 172.21.15.63:0/1730152473 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40021070 con 0x5625001712f0 2024-05-27T07:43:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.882+0000 7ffa44c00640 1 -- 172.21.15.63:0/1730152473 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa400a9ad0 con 0x5625001712f0 2024-05-27T07:43:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.882+0000 7ffa44c00640 1 -- 172.21.15.63:0/1730152473 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa400e2130 con 0x5625001712f0 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.883+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1730152473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 msgr2=0x5625005423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.883+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1730152473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x5625005423a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ffa400a3f10 tx=0x7ffa40009ea0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.884+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1730152473 shutdown_connections 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.884+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1730152473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56250019a1b0 0x562500546b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.884+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1730152473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001794d0 0x562500548d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.884+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/1730152473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625001712f0 0x5625005423a0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.884+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1730152473 >> 172.21.15.63:0/1730152473 conn(0x56250018e300 msgr2=0x562500546470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.884+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1730152473 shutdown_connections 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.884+0000 7ffa5dd99740 1 -- 172.21.15.63:0/1730152473 wait complete. 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.884+0000 7ffa5dd99740 1 -- start start 2024-05-27T07:43:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.885+0000 7ffa5dd99740 1 Processor -- start 2024-05-27T07:43:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.885+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001712f0 0x56250054eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.885+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625001794d0 0x56250054f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.885+0000 7ffa5dd99740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250019a1b0 0x56250054a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.885+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b1890 con 0x56250019a1b0 2024-05-27T07:43:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.885+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b2d00 con 0x5625001794d0 2024-05-27T07:43:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.885+0000 7ffa5dd99740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b1f60 con 0x5625001712f0 2024-05-27T07:43:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.885+0000 7ffa5dd99740 10 client.0 _ll_register_callbacks cb 0x7ffa40010ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.886+0000 7ffa54e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250019a1b0 0x56250054a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.886+0000 7ffa54e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250019a1b0 0x56250054a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46526/0 (socket says 172.21.15.63:46526) 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.886+0000 7ffa54e00640 1 -- 172.21.15.63:0/2736037176 learned_addr learned my addr 172.21.15.63:0/2736037176 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.886+0000 7ffa4f400640 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625001794d0 0x56250054f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.886+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001712f0 0x56250054eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.886+0000 7ffa54e00640 1 -- 172.21.15.63:0/2736037176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001712f0 msgr2=0x56250054eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.886+0000 7ffa54e00640 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001712f0 0x56250054eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.886+0000 7ffa54e00640 1 -- 172.21.15.63:0/2736037176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625001794d0 msgr2=0x56250054f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.886+0000 7ffa54e00640 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625001794d0 0x56250054f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.886+0000 7ffa54e00640 1 -- 172.21.15.63:0/2736037176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56250054aa40 con 0x56250019a1b0 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.887+0000 7ffa54e00640 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250019a1b0 0x56250054a380 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ffa4801f9a0 tx=0x7ffa48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb00407b0f1557be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.887+0000 7ffa3a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.887+0000 7ffa3a000640 1 -- 172.21.15.63:0/2736037176 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4802e060 con 0x56250019a1b0 2024-05-27T07:43:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.887+0000 7ffa3a000640 1 -- 172.21.15.63:0/2736037176 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7ffa48021da0 con 0x56250019a1b0 2024-05-27T07:43:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.887+0000 7ffa5dd99740 1 -- 172.21.15.63:0/2736037176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa400a9ad0 con 0x56250019a1b0 2024-05-27T07:43:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.887+0000 7ffa3a000640 1 -- 172.21.15.63:0/2736037176 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ffa48035050 con 0x56250019a1b0 2024-05-27T07:43:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.887+0000 7ffa5dd99740 10 client.17286 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.887+0000 7ffa5dd99740 1 -- 172.21.15.63:0/2736037176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5625005c95e0 con 0x56250019a1b0 2024-05-27T07:43:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.888+0000 7ffa3a000640 1 -- 172.21.15.63:0/2736037176 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa400a9ad0 con 0x56250019a1b0 2024-05-27T07:43:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.888+0000 7ffa3a000640 1 -- 172.21.15.63:0/2736037176 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ffa48028030 con 0x56250019a1b0 2024-05-27T07:43:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.888+0000 7ffa5dd99740 10 client.17286 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.888+0000 7ffa5dd99740 10 client.17286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.888+0000 7ffa5dd99740 20 client.17286 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.888+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56250012e1c0 0x5625005cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.888+0000 7ffa5dd99740 4 client.17286 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:43:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.888+0000 7ffa5dd99740 1 -- 172.21.15.63:0/2736037176 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625004a92c0 con 0x56250012e1c0 2024-05-27T07:43:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.889+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56250012e1c0 0x5625005cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.890+0000 7ffa4fe00640 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56250012e1c0 0x5625005cbd50 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb11afe6c047556 server_cookie=918d76de76ba98a9 in_seq=0 out_seq=0 2024-05-27T07:43:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.890+0000 7ffa3a000640 10 client.17286 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.923+0000 7ffa3a000640 1 -- 172.21.15.63:0/2736037176 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5625004a92c0 con 0x56250012e1c0 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.923+0000 7ffa3a000640 10 client.17286 handle_command_reply: tid=0 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.923+0000 7ffa5dd99740 1 client.17286 shutdown 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.923+0000 7ffa5dd99740 1 -- 172.21.15.63:0/2736037176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250019a1b0 msgr2=0x56250054a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.923+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250019a1b0 0x56250054a380 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ffa4801f9a0 tx=0x7ffa48004a40 comp rx=0 tx=0).stop 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.924+0000 7ffa5dd99740 1 -- 172.21.15.63:0/2736037176 shutdown_connections 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.924+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56250012e1c0 0x5625005cbd50 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.924+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250019a1b0 0x56250054a380 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.924+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625001794d0 0x56250054f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.924+0000 7ffa5dd99740 1 --2- 172.21.15.63:0/2736037176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625001712f0 0x56250054eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.924+0000 7ffa3a000640 0 client.17286 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.924+0000 7ffa5dd99740 1 -- 172.21.15.63:0/2736037176 >> 172.21.15.63:0/2736037176 conn(0x56250018e300 msgr2=0x562500546e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.924+0000 7ffa5dd99740 1 -- 172.21.15.63:0/2736037176 shutdown_connections 2024-05-27T07:43:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.924+0000 7ffa5dd99740 1 -- 172.21.15.63:0/2736037176 wait complete. 2024-05-27T07:43:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.924+0000 7ffa5dd99740 20 client.17286 trim_cache size 0 max 16384 2024-05-27T07:43:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:43:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:43:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:43:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 -- 172.21.15.63:0/416363001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa340b29d0 msgr2=0x7ffa340b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa340b29d0 0x7ffa340b4ee0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x562500539bd0 tx=0x7ffa48009740 comp rx=0 tx=0).stop 2024-05-27T07:43:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 -- 172.21.15.63:0/416363001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa50106440 msgr2=0x7ffa501a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa50106440 0x7ffa501a2510 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000d770 tx=0x7ffa4000dc30 comp rx=0 tx=0).stop 2024-05-27T07:43:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 -- 172.21.15.63:0/416363001 shutdown_connections 2024-05-27T07:43:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa340b29d0 0x7ffa340b4ee0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5010c190 0x7ffa5019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa5010bcf0 0x7ffa501a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 --2- 172.21.15.63:0/416363001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa50106440 0x7ffa501a2510 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 -- 172.21.15.63:0/416363001 >> 172.21.15.63:0/416363001 conn(0x7ffa5007d510 msgr2=0x7ffa50104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.925+0000 7ffa55800640 1 -- 172.21.15.63:0/416363001 shutdown_connections 2024-05-27T07:43:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:18.926+0000 7ffa55800640 1 -- 172.21.15.63:0/416363001 wait complete. 2024-05-27T07:43:19.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:19 smithi086 ceph-mon[34687]: pgmap v303: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 9 op/s 2024-05-27T07:43:19.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:19 smithi149 ceph-mon[35408]: pgmap v303: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 9 op/s 2024-05-27T07:43:19.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:19 smithi063 ceph-mon[31613]: pgmap v303: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 9 op/s 2024-05-27T07:43:21.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:21 smithi086 ceph-mon[34687]: pgmap v304: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 9 op/s 2024-05-27T07:43:21.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:21 smithi149 ceph-mon[35408]: pgmap v304: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 9 op/s 2024-05-27T07:43:21.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:21 smithi063 ceph-mon[31613]: pgmap v304: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 9 op/s 2024-05-27T07:43:23.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:23 smithi086 ceph-mon[34687]: pgmap v305: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:43:23.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:23 smithi149 ceph-mon[35408]: pgmap v305: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:43:23.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:23 smithi063 ceph-mon[31613]: pgmap v305: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:43:25.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:25 smithi086 ceph-mon[34687]: pgmap v306: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:43:25.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:25 smithi149 ceph-mon[35408]: pgmap v306: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:43:25.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:25 smithi063 ceph-mon[31613]: pgmap v306: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:43:27.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:27 smithi086 ceph-mon[34687]: pgmap v307: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T07:43:27.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:27 smithi149 ceph-mon[35408]: pgmap v307: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T07:43:27.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:27 smithi063 ceph-mon[31613]: pgmap v307: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T07:43:28.041 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:43:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:29.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:29 smithi086 ceph-mon[34687]: pgmap v308: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:43:29.710 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:29 smithi063 ceph-mon[31613]: pgmap v308: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:43:29.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:29 smithi149 ceph-mon[35408]: pgmap v308: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:43:31.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.055+0000 7fe54f000700 1 -- 172.21.15.63:0/1512651468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54810e6f0 msgr2=0x7fe548110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:31.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.055+0000 7fe54f000700 1 --2- 172.21.15.63:0/1512651468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54810e6f0 0x7fe548110bd0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe54000b0a0 tx=0x7fe54000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.056+0000 7fe54f000700 1 -- 172.21.15.63:0/1512651468 shutdown_connections 2024-05-27T07:43:31.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.056+0000 7fe54f000700 1 --2- 172.21.15.63:0/1512651468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe54810e6f0 0x7fe548110bd0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.056+0000 7fe54f000700 1 --2- 172.21.15.63:0/1512651468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe548105280 0x7fe548105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.056+0000 7fe54f000700 1 --2- 172.21.15.63:0/1512651468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe548105bd0 0x7fe54810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.056+0000 7fe54f000700 1 -- 172.21.15.63:0/1512651468 >> 172.21.15.63:0/1512651468 conn(0x7fe548078af0 msgr2=0x7fe548078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:31.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.056+0000 7fe54f000700 1 -- 172.21.15.63:0/1512651468 shutdown_connections 2024-05-27T07:43:31.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.056+0000 7fe54f000700 1 -- 172.21.15.63:0/1512651468 wait complete. 2024-05-27T07:43:31.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.058+0000 7fe54f000700 1 Processor -- start 2024-05-27T07:43:31.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.058+0000 7fe54f000700 1 -- start start 2024-05-27T07:43:31.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.059+0000 7fe54f000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe548105280 0x7fe548102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.059+0000 7fe54f000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe548105bd0 0x7fe5481034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.059+0000 7fe54f000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe54810e6f0 0x7fe5481039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.059+0000 7fe54f000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe548114120 con 0x7fe548105280 2024-05-27T07:43:31.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.059+0000 7fe54f000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe548113fa0 con 0x7fe548105bd0 2024-05-27T07:43:31.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.059+0000 7fe54f000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481142a0 con 0x7fe54810e6f0 2024-05-27T07:43:31.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.059+0000 7fe54dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe548105280 0x7fe548102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.060+0000 7fe54dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe548105280 0x7fe548102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35496/0 (socket says 172.21.15.63:35496) 2024-05-27T07:43:31.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.060+0000 7fe54dc00700 1 -- 172.21.15.63:0/1101337343 learned_addr learned my addr 172.21.15.63:0/1101337343 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:31.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.060+0000 7fe54d200700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe548105bd0 0x7fe5481034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.060+0000 7fe54e600700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe54810e6f0 0x7fe5481039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.060+0000 7fe54dc00700 1 -- 172.21.15.63:0/1101337343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe54810e6f0 msgr2=0x7fe5481039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:31.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.060+0000 7fe54dc00700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe54810e6f0 0x7fe5481039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.060+0000 7fe54dc00700 1 -- 172.21.15.63:0/1101337343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe548105bd0 msgr2=0x7fe5481034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:31.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.060+0000 7fe54dc00700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe548105bd0 0x7fe5481034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.060+0000 7fe54dc00700 1 -- 172.21.15.63:0/1101337343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5481a8e50 con 0x7fe548105280 2024-05-27T07:43:31.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.061+0000 7fe54e600700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe54810e6f0 0x7fe5481039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:43:31.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.061+0000 7fe54dc00700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe548105280 0x7fe548102f70 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00d980 tx=0x7fe53c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:31.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.061+0000 7fe546000700 1 -- 172.21.15.63:0/1101337343 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c017070 con 0x7fe548105280 2024-05-27T07:43:31.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.061+0000 7fe546000700 1 -- 172.21.15.63:0/1101337343 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe53c005020 con 0x7fe548105280 2024-05-27T07:43:31.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.061+0000 7fe546000700 1 -- 172.21.15.63:0/1101337343 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c009ab0 con 0x7fe548105280 2024-05-27T07:43:31.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.061+0000 7fe54f000700 1 -- 172.21.15.63:0/1101337343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5481a90e0 con 0x7fe548105280 2024-05-27T07:43:31.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.062+0000 7fe54f000700 1 -- 172.21.15.63:0/1101337343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5481a95d0 con 0x7fe548105280 2024-05-27T07:43:31.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.066+0000 7fe546000700 1 -- 172.21.15.63:0/1101337343 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe53c0051d0 con 0x7fe548105280 2024-05-27T07:43:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.067+0000 7fe55f22cb80 1 Processor -- start 2024-05-27T07:43:31.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.067+0000 7fe55f22cb80 1 -- start start 2024-05-27T07:43:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.067+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c111be40 0x5564c1299ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c129a410 0x5564c12a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5564c11c8e40 con 0x5564c129a410 2024-05-27T07:43:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564c11b89e0 con 0x5564c129a820 2024-05-27T07:43:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5564c11bca20 con 0x5564c111be40 2024-05-27T07:43:31.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe546000700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe5300b29f0 0x7fe5300b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe54dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c129a410 0x5564c12a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe54e600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c111be40 0x5564c1299ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe546000700 1 -- 172.21.15.63:0/1101337343 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fe53c0e28e0 con 0x7fe548105280 2024-05-27T07:43:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe54e600700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe5300b29f0 0x7fe5300b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe54e600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c111be40 0x5564c1299ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:38082/0 (socket says 172.21.15.63:38082) 2024-05-27T07:43:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.068+0000 7fe54e600700 1 -- 172.21.15.63:0/3397514292 learned_addr learned my addr 172.21.15.63:0/3397514292 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:31.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.069+0000 7fe54d200700 1 --2- 172.21.15.63:0/3397514292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.069+0000 7fe54dc00700 1 -- 172.21.15.63:0/3397514292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c111be40 msgr2=0x5564c1299ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.069+0000 7fe54dc00700 1 --2- 172.21.15.63:0/3397514292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c111be40 0x5564c1299ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.069+0000 7fe54dc00700 1 -- 172.21.15.63:0/3397514292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 msgr2=0x5564c12a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:31.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.069+0000 7fe54dc00700 1 --2- 172.21.15.63:0/3397514292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.069+0000 7fe54dc00700 1 -- 172.21.15.63:0/3397514292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564c1181420 con 0x5564c129a410 2024-05-27T07:43:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.070+0000 7fe54e600700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe5300b29f0 0x7fe5300b4f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe540009e60 tx=0x7fe54000cd80 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.070+0000 7fe54dc00700 1 --2- 172.21.15.63:0/3397514292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c129a410 0x5564c12a4ad0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0dfc70 tx=0x7fe53c076130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d104c92fde72405 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.070+0000 7fe53aa00700 1 -- 172.21.15.63:0/3397514292 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c077210 con 0x5564c129a410 2024-05-27T07:43:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.070+0000 7fe53aa00700 1 -- 172.21.15.63:0/3397514292 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe53c0a9310 con 0x5564c129a410 2024-05-27T07:43:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.070+0000 7fe53aa00700 1 -- 172.21.15.63:0/3397514292 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0b2660 con 0x5564c129a410 2024-05-27T07:43:31.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.071+0000 7fe55f22cb80 1 -- 172.21.15.63:0/3397514292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c129a410 msgr2=0x5564c12a4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.071+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/3397514292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c129a410 0x5564c12a4ad0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0dfc70 tx=0x7fe53c076130 comp rx=0 tx=0).stop 2024-05-27T07:43:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.071+0000 7fe55f22cb80 1 -- 172.21.15.63:0/3397514292 shutdown_connections 2024-05-27T07:43:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.071+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/3397514292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.071+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/3397514292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c111be40 0x5564c1299ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.071+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/3397514292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c129a410 0x5564c12a4ad0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.071+0000 7fe55f22cb80 1 -- 172.21.15.63:0/3397514292 >> 172.21.15.63:0/3397514292 conn(0x5564c117bfc0 msgr2=0x5564c1295d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.071+0000 7fe55f22cb80 1 -- 172.21.15.63:0/3397514292 shutdown_connections 2024-05-27T07:43:31.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.072+0000 7fe55f22cb80 1 -- 172.21.15.63:0/3397514292 wait complete. 2024-05-27T07:43:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.072+0000 7fe55f22cb80 1 -- start start 2024-05-27T07:43:31.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.073+0000 7fe55f22cb80 1 Processor -- start 2024-05-27T07:43:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.073+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12a4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a410 0x5564c12a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a820 0x5564c12a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5564c10befa0 con 0x5564c111be40 2024-05-27T07:43:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564c1159760 con 0x5564c129a410 2024-05-27T07:43:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5564c12a7990 con 0x5564c129a820 2024-05-27T07:43:31.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe55f22cb80 10 client.0 _ll_register_callbacks cb 0x5564c1162c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe54dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe54dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12a4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35514/0 (socket says 172.21.15.63:35514) 2024-05-27T07:43:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe54dc00700 1 -- 172.21.15.63:0/4251272343 learned_addr learned my addr 172.21.15.63:0/4251272343 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:31.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe54d200700 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a410 0x5564c12a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.074+0000 7fe54e600700 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a820 0x5564c12a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.075+0000 7fe54dc00700 1 -- 172.21.15.63:0/4251272343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a820 msgr2=0x5564c12a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.075+0000 7fe54dc00700 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a820 0x5564c12a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.075+0000 7fe54dc00700 1 -- 172.21.15.63:0/4251272343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a410 msgr2=0x5564c12a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.075+0000 7fe54dc00700 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a410 0x5564c12a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.075+0000 7fe54dc00700 1 -- 172.21.15.63:0/4251272343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564c11088d0 con 0x5564c111be40 2024-05-27T07:43:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.075+0000 7fe54dc00700 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12a4500 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0a5880 tx=0x7fe53c009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d66712d0b8dd92 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.075+0000 7fe52f400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.076+0000 7fe52f400700 1 -- 172.21.15.63:0/4251272343 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0054f0 con 0x5564c111be40 2024-05-27T07:43:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.076+0000 7fe52f400700 1 -- 172.21.15.63:0/4251272343 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fe53c0a9c70 con 0x5564c111be40 2024-05-27T07:43:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.076+0000 7fe55f22cb80 1 -- 172.21.15.63:0/4251272343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe53c0a9310 con 0x5564c111be40 2024-05-27T07:43:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.076+0000 7fe52f400700 1 -- 172.21.15.63:0/4251272343 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe53c0c2020 con 0x5564c111be40 2024-05-27T07:43:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.076+0000 7fe55f22cb80 10 client.17298 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:31.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.076+0000 7fe55f22cb80 1 -- 172.21.15.63:0/4251272343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5564c1189060 con 0x5564c111be40 2024-05-27T07:43:31.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.077+0000 7fe52f400700 1 -- 172.21.15.63:0/4251272343 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe53c0a9310 con 0x5564c111be40 2024-05-27T07:43:31.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.077+0000 7fe52f400700 1 -- 172.21.15.63:0/4251272343 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe53c0b21d0 con 0x5564c111be40 2024-05-27T07:43:31.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.077+0000 7fe55f22cb80 10 client.17298 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:31.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.077+0000 7fe55f22cb80 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:31.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.077+0000 7fe55f22cb80 20 client.17298 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:31.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.077+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564c12b4770 0x5564c12b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.077+0000 7fe55f22cb80 4 client.17298 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:31.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.077+0000 7fe55f22cb80 1 -- 172.21.15.63:0/4251272343 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564c11cad90 con 0x5564c12b4770 2024-05-27T07:43:31.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.078+0000 7fe54d200700 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564c12b4770 0x5564c12b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.079+0000 7fe54d200700 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564c12b4770 0x5564c12b6b90 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a503f0ecb785afc server_cookie=d6529382355dd78e in_seq=0 out_seq=0 2024-05-27T07:43:31.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.079+0000 7fe52f400700 10 client.17298 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:31.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.081+0000 7fe52f400700 1 -- 172.21.15.63:0/4251272343 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5564c11cad90 con 0x5564c12b4770 2024-05-27T07:43:31.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.081+0000 7fe52f400700 10 client.17298 handle_command_reply: tid=0 2024-05-27T07:43:31.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.081+0000 7fe55f22cb80 1 client.17298 shutdown 2024-05-27T07:43:31.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.081+0000 7fe55f22cb80 1 -- 172.21.15.63:0/4251272343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 msgr2=0x5564c12a4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:31.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.081+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12a4500 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0a5880 tx=0x7fe53c009f00 comp rx=0 tx=0).stop 2024-05-27T07:43:31.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.081+0000 7fe55f22cb80 1 -- 172.21.15.63:0/4251272343 shutdown_connections 2024-05-27T07:43:31.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.081+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564c12b4770 0x5564c12b6b90 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.081+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a820 0x5564c12a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.081+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12a4500 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.081+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/4251272343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a410 0x5564c12a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.082+0000 7fe55f22cb80 1 -- 172.21.15.63:0/4251272343 >> 172.21.15.63:0/4251272343 conn(0x5564c117bfc0 msgr2=0x5564c1294ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:31.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.082+0000 7fe52f400700 0 client.17298 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:31.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.082+0000 7fe55f22cb80 1 -- 172.21.15.63:0/4251272343 shutdown_connections 2024-05-27T07:43:31.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.082+0000 7fe55f22cb80 1 -- 172.21.15.63:0/4251272343 wait complete. 2024-05-27T07:43:31.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.082+0000 7fe55f22cb80 20 client.17298 trim_cache size 0 max 16384 2024-05-27T07:43:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.115+0000 7fe55f22cb80 1 Processor -- start 2024-05-27T07:43:31.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.116+0000 7fe55f22cb80 1 -- start start 2024-05-27T07:43:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.116+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c129fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.116+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 0x5564c12b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.116+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.116+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481b4cf0 con 0x5564c111be40 2024-05-27T07:43:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.116+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481b2580 con 0x5564c129a820 2024-05-27T07:43:31.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.116+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481b03f0 con 0x5564c129a410 2024-05-27T07:43:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.117+0000 7fe54dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c129fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.117+0000 7fe54dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c129fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35522/0 (socket says 172.21.15.63:35522) 2024-05-27T07:43:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.117+0000 7fe54dc00700 1 -- 172.21.15.63:0/819616465 learned_addr learned my addr 172.21.15.63:0/819616465 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.117+0000 7fe54d200700 1 --2- 172.21.15.63:0/819616465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 0x5564c12b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.117+0000 7fe54e600700 1 --2- 172.21.15.63:0/819616465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.118+0000 7fe54dc00700 1 -- 172.21.15.63:0/819616465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 msgr2=0x5564c12b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.118+0000 7fe54dc00700 1 --2- 172.21.15.63:0/819616465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 0x5564c12b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.118+0000 7fe54dc00700 1 -- 172.21.15.63:0/819616465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 msgr2=0x5564c12a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:31.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.118+0000 7fe54dc00700 1 --2- 172.21.15.63:0/819616465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.118+0000 7fe54dc00700 1 -- 172.21.15.63:0/819616465 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564c12a2a70 con 0x5564c111be40 2024-05-27T07:43:31.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.118+0000 7fe54dc00700 1 --2- 172.21.15.63:0/819616465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c129fcf0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0e01f0 tx=0x7fe53c0cc5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34c5f7088ed80d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:31.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.119+0000 7fe53a000700 1 -- 172.21.15.63:0/819616465 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0a56c0 con 0x5564c111be40 2024-05-27T07:43:31.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.119+0000 7fe53a000700 1 -- 172.21.15.63:0/819616465 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe53c0910b0 con 0x5564c111be40 2024-05-27T07:43:31.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.119+0000 7fe53a000700 1 -- 172.21.15.63:0/819616465 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0adbb0 con 0x5564c111be40 2024-05-27T07:43:31.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.120+0000 7fe55f22cb80 1 -- 172.21.15.63:0/819616465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 msgr2=0x5564c129fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:31.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.120+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/819616465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c129fcf0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe53c0e01f0 tx=0x7fe53c0cc5a0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.120+0000 7fe55f22cb80 1 -- 172.21.15.63:0/819616465 shutdown_connections 2024-05-27T07:43:31.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.120+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/819616465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.120+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/819616465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c129fcf0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.120+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/819616465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 0x5564c12b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.120+0000 7fe55f22cb80 1 -- 172.21.15.63:0/819616465 >> 172.21.15.63:0/819616465 conn(0x5564c117bfc0 msgr2=0x5564c12a42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:31.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.121+0000 7fe55f22cb80 1 -- 172.21.15.63:0/819616465 shutdown_connections 2024-05-27T07:43:31.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.121+0000 7fe55f22cb80 1 -- 172.21.15.63:0/819616465 wait complete. 2024-05-27T07:43:31.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.121+0000 7fe55f22cb80 1 -- start start 2024-05-27T07:43:31.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.122+0000 7fe55f22cb80 1 Processor -- start 2024-05-27T07:43:31.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.122+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12ba280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.122+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 0x5564c12b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.122+0000 7fe55f22cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.122+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481b0a40 con 0x5564c111be40 2024-05-27T07:43:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.122+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481b1eb0 con 0x5564c129a820 2024-05-27T07:43:31.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.122+0000 7fe55f22cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481b1110 con 0x5564c129a410 2024-05-27T07:43:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.122+0000 7fe55f22cb80 10 client.0 _ll_register_callbacks cb 0x5564c1162c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.123+0000 7fe54dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12ba280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.123+0000 7fe54dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12ba280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35530/0 (socket says 172.21.15.63:35530) 2024-05-27T07:43:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.123+0000 7fe54dc00700 1 -- 172.21.15.63:0/1243494495 learned_addr learned my addr 172.21.15.63:0/1243494495 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.123+0000 7fe54d200700 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 0x5564c12b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.123+0000 7fe54e600700 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.124+0000 7fe54d200700 1 -- 172.21.15.63:0/1243494495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 msgr2=0x5564c12b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.124+0000 7fe54d200700 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12b7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.124+0000 7fe54d200700 1 -- 172.21.15.63:0/1243494495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 msgr2=0x5564c12ba280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.124+0000 7fe54d200700 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12ba280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.124+0000 7fe54d200700 1 -- 172.21.15.63:0/1243494495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564c12b8270 con 0x5564c129a410 2024-05-27T07:43:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.124+0000 7fe54dc00700 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12ba280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:43:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.124+0000 7fe54d200700 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 0x5564c12b7600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5564c11c8f40 tx=0x7fe5340044e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5306be1876df01 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.124+0000 7fe52f400700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:43:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.125+0000 7fe52f400700 1 -- 172.21.15.63:0/1243494495 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534011440 con 0x5564c129a410 2024-05-27T07:43:31.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.125+0000 7fe55f22cb80 1 -- 172.21.15.63:0/1243494495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe53c0910b0 con 0x5564c129a410 2024-05-27T07:43:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.125+0000 7fe52f400700 1 -- 172.21.15.63:0/1243494495 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fe5340115e0 con 0x5564c129a410 2024-05-27T07:43:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.125+0000 7fe52f400700 1 -- 172.21.15.63:0/1243494495 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe53402c050 con 0x5564c129a410 2024-05-27T07:43:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.125+0000 7fe55f22cb80 10 client.26731 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.125+0000 7fe55f22cb80 1 -- 172.21.15.63:0/1243494495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5564c131ef60 con 0x5564c129a410 2024-05-27T07:43:31.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.126+0000 7fe52f400700 1 -- 172.21.15.63:0/1243494495 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe53c0910b0 con 0x5564c129a410 2024-05-27T07:43:31.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.126+0000 7fe52f400700 1 -- 172.21.15.63:0/1243494495 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe53402f020 con 0x5564c129a410 2024-05-27T07:43:31.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.126+0000 7fe55f22cb80 10 client.26731 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:31.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.126+0000 7fe55f22cb80 10 client.26731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:31.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.126+0000 7fe55f22cb80 20 client.26731 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:31.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.126+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564c131f670 0x5564c1321a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:31.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.127+0000 7fe55f22cb80 4 client.26731 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:43:31.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.127+0000 7fe55f22cb80 1 -- 172.21.15.63:0/1243494495 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564c11c4570 con 0x5564c131f670 2024-05-27T07:43:31.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.127+0000 7fe54dc00700 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564c131f670 0x5564c1321a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:31.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.129+0000 7fe52f400700 10 client.26731 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:31.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.129+0000 7fe54dc00700 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564c131f670 0x5564c1321a90 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de78cd0a9fdb5adf server_cookie=8eec11f71d70f48b in_seq=0 out_seq=0 2024-05-27T07:43:31.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.129+0000 7fe52f400700 1 -- 172.21.15.63:0/1243494495 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5564c11c4570 con 0x5564c131f670 2024-05-27T07:43:31.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.129+0000 7fe52f400700 10 client.26731 handle_command_reply: tid=0 2024-05-27T07:43:31.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.129+0000 7fe55f22cb80 1 client.26731 shutdown 2024-05-27T07:43:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.129+0000 7fe55f22cb80 1 -- 172.21.15.63:0/1243494495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 msgr2=0x5564c12b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.129+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 0x5564c12b7600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5564c11c8f40 tx=0x7fe5340044e0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.130+0000 7fe55f22cb80 1 -- 172.21.15.63:0/1243494495 shutdown_connections 2024-05-27T07:43:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.130+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564c131f670 0x5564c1321a90 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.130+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564c129a820 0x5564c12b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.130+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564c111be40 0x5564c12ba280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.130+0000 7fe55f22cb80 1 --2- 172.21.15.63:0/1243494495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564c129a410 0x5564c12b7600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.130+0000 7fe52f400700 0 client.26731 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:31.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.130+0000 7fe55f22cb80 1 -- 172.21.15.63:0/1243494495 >> 172.21.15.63:0/1243494495 conn(0x5564c117bfc0 msgr2=0x5564c12a4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:31.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.130+0000 7fe55f22cb80 1 -- 172.21.15.63:0/1243494495 shutdown_connections 2024-05-27T07:43:31.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.130+0000 7fe55f22cb80 1 -- 172.21.15.63:0/1243494495 wait complete. 2024-05-27T07:43:31.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.131+0000 7fe55f22cb80 20 client.26731 trim_cache size 0 max 16384 2024-05-27T07:43:31.131 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:43:31.131 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:43:31.131 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:43:31.131 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:43:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.131+0000 7fe54f000700 1 -- 172.21.15.63:0/1101337343 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe5300b29f0 msgr2=0x7fe5300b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.131+0000 7fe54f000700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe5300b29f0 0x7fe5300b4f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe540009e60 tx=0x7fe54000cd80 comp rx=0 tx=0).stop 2024-05-27T07:43:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.131+0000 7fe54f000700 1 -- 172.21.15.63:0/1101337343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe548105280 msgr2=0x7fe548102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.131+0000 7fe54f000700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe548105280 0x7fe548102f70 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00d980 tx=0x7fe53c00dc90 comp rx=0 tx=0).stop 2024-05-27T07:43:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.132+0000 7fe54f000700 1 -- 172.21.15.63:0/1101337343 shutdown_connections 2024-05-27T07:43:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.132+0000 7fe54f000700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe5300b29f0 0x7fe5300b4f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.132+0000 7fe54f000700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe54810e6f0 0x7fe5481039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.132+0000 7fe54f000700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe548105280 0x7fe548102f70 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.132+0000 7fe54f000700 1 --2- 172.21.15.63:0/1101337343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe548105bd0 0x7fe5481034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:31.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.132+0000 7fe54f000700 1 -- 172.21.15.63:0/1101337343 >> 172.21.15.63:0/1101337343 conn(0x7fe548078af0 msgr2=0x7fe5480fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.132+0000 7fe54f000700 1 -- 172.21.15.63:0/1101337343 shutdown_connections 2024-05-27T07:43:31.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:31.132+0000 7fe54f000700 1 -- 172.21.15.63:0/1101337343 wait complete. 2024-05-27T07:43:31.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:31 smithi086 ceph-mon[34687]: pgmap v309: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:43:31.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:31 smithi149 ceph-mon[35408]: pgmap v309: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:43:31.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:31 smithi063 ceph-mon[31613]: pgmap v309: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:43:32.126 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:43:32.126 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.178+0000 7f3ec3000640 1 -- 172.21.15.63:0/1916390140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ebc109ab0 msgr2=0x7f3ebc1036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.178+0000 7f3ec3000640 1 --2- 172.21.15.63:0/1916390140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ebc109ab0 0x7f3ebc1036a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009900 tx=0x7f3eb002a3f0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.179+0000 7f3ec3000640 1 -- 172.21.15.63:0/1916390140 shutdown_connections 2024-05-27T07:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.179+0000 7f3ec3000640 1 --2- 172.21.15.63:0/1916390140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ebc104250 0x7f3ebc1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.179+0000 7f3ec3000640 1 --2- 172.21.15.63:0/1916390140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc109f50 0x7f3ebc103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.179+0000 7f3ec3000640 1 --2- 172.21.15.63:0/1916390140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ebc109ab0 0x7f3ebc1036a0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.179+0000 7f3ec3000640 1 -- 172.21.15.63:0/1916390140 >> 172.21.15.63:0/1916390140 conn(0x7f3ebc079240 msgr2=0x7f3ebc079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.179+0000 7f3ec3000640 1 -- 172.21.15.63:0/1916390140 shutdown_connections 2024-05-27T07:43:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.179+0000 7f3ec3000640 1 -- 172.21.15.63:0/1916390140 wait complete. 2024-05-27T07:43:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.180+0000 7f3ec3000640 1 Processor -- start 2024-05-27T07:43:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.181+0000 7f3ec3000640 1 -- start start 2024-05-27T07:43:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.181+0000 7f3ec3000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ebc104250 0x7f3ebc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.181+0000 7f3ec3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc109ab0 0x7f3ebc19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.181+0000 7f3ec3000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ebc109f50 0x7f3ebc19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.181+0000 7f3ec3000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc113ba0 con 0x7f3ebc104250 2024-05-27T07:43:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.181+0000 7f3ec3000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc113a20 con 0x7f3ebc109ab0 2024-05-27T07:43:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.181+0000 7f3ec3000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc113d20 con 0x7f3ebc109f50 2024-05-27T07:43:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.182+0000 7f3ec1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ebc104250 0x7f3ebc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.182+0000 7f3ec1c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ebc104250 0x7f3ebc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35544/0 (socket says 172.21.15.63:35544) 2024-05-27T07:43:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.182+0000 7f3ec1c00640 1 -- 172.21.15.63:0/3452316894 learned_addr learned my addr 172.21.15.63:0/3452316894 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.182+0000 7f3ec1200640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc109ab0 0x7f3ebc19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.182+0000 7f3ec2600640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ebc109f50 0x7f3ebc19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.183+0000 7f3ec1c00640 1 -- 172.21.15.63:0/3452316894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ebc109f50 msgr2=0x7f3ebc19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.183+0000 7f3ec1c00640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ebc109f50 0x7f3ebc19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.183+0000 7f3ec1c00640 1 -- 172.21.15.63:0/3452316894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc109ab0 msgr2=0x7f3ebc19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.183+0000 7f3ec1c00640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc109ab0 0x7f3ebc19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.183+0000 7f3ec1c00640 1 -- 172.21.15.63:0/3452316894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ebc19f480 con 0x7f3ebc104250 2024-05-27T07:43:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.183+0000 7f3ec1c00640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ebc104250 0x7f3ebc19a620 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f3eb002a910 tx=0x7f3eb0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.184+0000 7f3eba000640 1 -- 172.21.15.63:0/3452316894 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb003f070 con 0x7f3ebc104250 2024-05-27T07:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.184+0000 7f3eba000640 1 -- 172.21.15.63:0/3452316894 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3eb0003e10 con 0x7f3ebc104250 2024-05-27T07:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.184+0000 7f3eba000640 1 -- 172.21.15.63:0/3452316894 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0035030 con 0x7f3ebc104250 2024-05-27T07:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.184+0000 7f3ec3000640 1 -- 172.21.15.63:0/3452316894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ebc19f710 con 0x7f3ebc104250 2024-05-27T07:43:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.184+0000 7f3ec3000640 1 -- 172.21.15.63:0/3452316894 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ebc19fbb0 con 0x7f3ebc104250 2024-05-27T07:43:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ecb635740 1 Processor -- start 2024-05-27T07:43:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ecb635740 1 -- start start 2024-05-27T07:43:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db86041b0 0x559db89ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db85db2f0 0x559db89ac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85e34d0 0x559db89aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ecb635740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559db8917b90 con 0x559db86041b0 2024-05-27T07:43:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ecb635740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559db8907730 con 0x559db85e34d0 2024-05-27T07:43:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ecb635740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559db890b770 con 0x559db85db2f0 2024-05-27T07:43:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ec2600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db86041b0 0x559db89ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ec2600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db86041b0 0x559db89ac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35546/0 (socket says 172.21.15.63:35546) 2024-05-27T07:43:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ec2600640 1 -- 172.21.15.63:0/2994979217 learned_addr learned my addr 172.21.15.63:0/2994979217 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.189+0000 7f3ec1200640 1 --2- 172.21.15.63:0/2994979217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85e34d0 0x559db89aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.190+0000 7f3eba000640 1 -- 172.21.15.63:0/3452316894 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3eb0042070 con 0x7f3ebc104250 2024-05-27T07:43:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.190+0000 7f3ec2600640 1 -- 172.21.15.63:0/2994979217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db85db2f0 msgr2=0x559db89ac550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:43:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.190+0000 7f3ec2600640 1 --2- 172.21.15.63:0/2994979217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db85db2f0 0x559db89ac550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.190+0000 7f3ec2600640 1 -- 172.21.15.63:0/2994979217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85e34d0 msgr2=0x559db89aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.190+0000 7f3ec2600640 1 --2- 172.21.15.63:0/2994979217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85e34d0 0x559db89aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.190+0000 7f3ec2600640 1 -- 172.21.15.63:0/2994979217 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db88a08b0 con 0x559db86041b0 2024-05-27T07:43:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.191+0000 7f3ec2600640 1 --2- 172.21.15.63:0/2994979217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db86041b0 0x559db89ac010 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4010370 tx=0x7f3eb4010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e359db22b324dbd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.191+0000 7f3eaea00640 1 -- 172.21.15.63:0/2994979217 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4016070 con 0x559db86041b0 2024-05-27T07:43:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.191+0000 7f3eaea00640 1 -- 172.21.15.63:0/2994979217 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3eb4004d20 con 0x559db86041b0 2024-05-27T07:43:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.191+0000 7f3eaea00640 1 -- 172.21.15.63:0/2994979217 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb40052c0 con 0x559db86041b0 2024-05-27T07:43:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.192+0000 7f3eba000640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3ea40b29d0 0x7f3ea40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.192+0000 7f3eba000640 1 -- 172.21.15.63:0/3452316894 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f3eb01030c0 con 0x7f3ebc104250 2024-05-27T07:43:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.192+0000 7f3ecb635740 1 -- 172.21.15.63:0/2994979217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db86041b0 msgr2=0x559db89ac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.192+0000 7f3ecb635740 1 --2- 172.21.15.63:0/2994979217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db86041b0 0x559db89ac010 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4010370 tx=0x7f3eb4010830 comp rx=0 tx=0).stop 2024-05-27T07:43:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.192+0000 7f3ec1c00640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3ea40b29d0 0x7f3ea40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.193+0000 7f3ecb635740 1 -- 172.21.15.63:0/2994979217 shutdown_connections 2024-05-27T07:43:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.193+0000 7f3ecb635740 1 --2- 172.21.15.63:0/2994979217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85e34d0 0x559db89aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.193+0000 7f3ecb635740 1 --2- 172.21.15.63:0/2994979217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db85db2f0 0x559db89ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.193+0000 7f3ecb635740 1 --2- 172.21.15.63:0/2994979217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db86041b0 0x559db89ac010 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.193+0000 7f3ecb635740 1 -- 172.21.15.63:0/2994979217 >> 172.21.15.63:0/2994979217 conn(0x559db85f8300 msgr2=0x559db89a2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.193+0000 7f3ecb635740 1 -- 172.21.15.63:0/2994979217 shutdown_connections 2024-05-27T07:43:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.193+0000 7f3ecb635740 1 -- 172.21.15.63:0/2994979217 wait complete. 2024-05-27T07:43:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.193+0000 7f3ec1c00640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3ea40b29d0 0x7f3ea40b4ee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x559db89a44d0 tx=0x7f3eb0009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.194+0000 7f3ecb635740 1 -- start start 2024-05-27T07:43:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.194+0000 7f3ecb635740 1 Processor -- start 2024-05-27T07:43:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.195+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85db2f0 0x559db89c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.195+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85e34d0 0x559db89c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.195+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 0x559db89c2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.195+0000 7f3ecb635740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559db85ed790 con 0x559db85db2f0 2024-05-27T07:43:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.195+0000 7f3ecb635740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559db88afbc0 con 0x559db85e34d0 2024-05-27T07:43:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.195+0000 7f3ecb635740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559db89ab3c0 con 0x559db86041b0 2024-05-27T07:43:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.195+0000 7f3ecb635740 10 client.0 _ll_register_callbacks cb 0x559db888d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.195+0000 7f3ec2600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85db2f0 0x559db89c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.195+0000 7f3ec2600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85db2f0 0x559db89c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35560/0 (socket says 172.21.15.63:35560) 2024-05-27T07:43:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.195+0000 7f3ec2600640 1 -- 172.21.15.63:0/298560750 learned_addr learned my addr 172.21.15.63:0/298560750 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.196+0000 7f3ec1200640 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85e34d0 0x559db89c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.196+0000 7f3ec2600640 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 0x559db89c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.196+0000 7f3ec2600640 1 -- 172.21.15.63:0/298560750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 msgr2=0x559db89c2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.196+0000 7f3ec2600640 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 0x559db89c2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.196+0000 7f3ec2600640 1 -- 172.21.15.63:0/298560750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85e34d0 msgr2=0x559db89c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.196+0000 7f3ec2600640 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85e34d0 0x559db89c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.196+0000 7f3ec2600640 1 -- 172.21.15.63:0/298560750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db861c210 con 0x559db85db2f0 2024-05-27T07:43:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.197+0000 7f3ec2600640 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85db2f0 0x559db89c7f10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098b0 tx=0x7f3eb4009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235ed6d54159c026 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.197+0000 7f3ea3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.197+0000 7f3ea3400640 1 -- 172.21.15.63:0/298560750 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4016070 con 0x559db85db2f0 2024-05-27T07:43:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.197+0000 7f3ea3400640 1 -- 172.21.15.63:0/298560750 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f3eb4004250 con 0x559db85db2f0 2024-05-27T07:43:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.197+0000 7f3ecb635740 1 -- 172.21.15.63:0/298560750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eb4004d20 con 0x559db85db2f0 2024-05-27T07:43:32.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.198+0000 7f3ea3400640 1 -- 172.21.15.63:0/298560750 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3eb4004450 con 0x559db85db2f0 2024-05-27T07:43:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.198+0000 7f3ecb635740 10 client.17316 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.198+0000 7f3ecb635740 1 -- 172.21.15.63:0/298560750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559db86e8620 con 0x559db85db2f0 2024-05-27T07:43:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.199+0000 7f3ea3400640 1 -- 172.21.15.63:0/298560750 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3eb4004d20 con 0x559db85db2f0 2024-05-27T07:43:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.199+0000 7f3ea3400640 1 -- 172.21.15.63:0/298560750 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3eb400f040 con 0x559db85db2f0 2024-05-27T07:43:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.199+0000 7f3ecb635740 10 client.17316 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.199+0000 7f3ecb635740 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.199+0000 7f3ecb635740 20 client.17316 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.199+0000 7f3ecb635740 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559db85c2d80 0x559db89d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.199+0000 7f3ecb635740 4 client.17316 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.199+0000 7f3ecb635740 1 -- 172.21.15.63:0/298560750 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559db8919ae0 con 0x559db85c2d80 2024-05-27T07:43:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.200+0000 7f3ec1200640 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559db85c2d80 0x559db89d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.201+0000 7f3ec1200640 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559db85c2d80 0x559db89d3f00 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35c10dd3d3e235b8 server_cookie=383f2f53177be3fc in_seq=0 out_seq=0 2024-05-27T07:43:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.201+0000 7f3ea3400640 10 client.17316 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.203+0000 7f3ea3400640 1 -- 172.21.15.63:0/298560750 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559db8919ae0 con 0x559db85c2d80 2024-05-27T07:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.203+0000 7f3ea3400640 10 client.17316 handle_command_reply: tid=0 2024-05-27T07:43:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.203+0000 7f3ecb635740 1 client.17316 shutdown 2024-05-27T07:43:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 1 -- 172.21.15.63:0/298560750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85db2f0 msgr2=0x559db89c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85db2f0 0x559db89c7f10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098b0 tx=0x7f3eb4009cf0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 1 -- 172.21.15.63:0/298560750 shutdown_connections 2024-05-27T07:43:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559db85c2d80 0x559db89d3f00 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 0x559db89c2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85e34d0 0x559db89c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 1 --2- 172.21.15.63:0/298560750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85db2f0 0x559db89c7f10 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 1 -- 172.21.15.63:0/298560750 >> 172.21.15.63:0/298560750 conn(0x559db85f8300 msgr2=0x559db89a2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ea3400640 0 client.17316 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 1 -- 172.21.15.63:0/298560750 shutdown_connections 2024-05-27T07:43:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 1 -- 172.21.15.63:0/298560750 wait complete. 2024-05-27T07:43:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.204+0000 7f3ecb635740 20 client.17316 trim_cache size 0 max 16384 2024-05-27T07:43:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.222+0000 7f3ecb635740 1 Processor -- start 2024-05-27T07:43:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.222+0000 7f3ecb635740 1 -- start start 2024-05-27T07:43:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.222+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db85db2f0 0x559db89afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.222+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.222+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db86041b0 0x559db89c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.222+0000 7f3ecb635740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc1b17c0 con 0x559db85e34d0 2024-05-27T07:43:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.222+0000 7f3ecb635740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc1af090 con 0x559db86041b0 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.222+0000 7f3ecb635740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc1acf00 con 0x559db85db2f0 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.223+0000 7f3ec1200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.223+0000 7f3ec1200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89ac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35576/0 (socket says 172.21.15.63:35576) 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.223+0000 7f3ec1200640 1 -- 172.21.15.63:0/1976789106 learned_addr learned my addr 172.21.15.63:0/1976789106 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.223+0000 7f3ec2600640 1 --2- 172.21.15.63:0/1976789106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db85db2f0 0x559db89afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.223+0000 7f3ec2600640 1 --2- 172.21.15.63:0/1976789106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db86041b0 0x559db89c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.223+0000 7f3ec1200640 1 -- 172.21.15.63:0/1976789106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db85db2f0 msgr2=0x559db89afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.223+0000 7f3ec1200640 1 --2- 172.21.15.63:0/1976789106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db85db2f0 0x559db89afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.223+0000 7f3ec1200640 1 -- 172.21.15.63:0/1976789106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db86041b0 msgr2=0x559db89c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.223+0000 7f3ec1200640 1 --2- 172.21.15.63:0/1976789106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db86041b0 0x559db89c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.223+0000 7f3ec1200640 1 -- 172.21.15.63:0/1976789106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db89aa080 con 0x559db85e34d0 2024-05-27T07:43:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.224+0000 7f3ec1200640 1 --2- 172.21.15.63:0/1976789106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89ac650 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc101720 tx=0x7f3ea800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fad9132598c1be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.224+0000 7f3eae000640 1 -- 172.21.15.63:0/1976789106 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8011440 con 0x559db85e34d0 2024-05-27T07:43:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.224+0000 7f3eae000640 1 -- 172.21.15.63:0/1976789106 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea8011a70 con 0x559db85e34d0 2024-05-27T07:43:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.224+0000 7f3eae000640 1 -- 172.21.15.63:0/1976789106 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8027760 con 0x559db85e34d0 2024-05-27T07:43:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.226+0000 7f3ecb635740 1 -- 172.21.15.63:0/1976789106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 msgr2=0x559db89ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.226+0000 7f3ecb635740 1 --2- 172.21.15.63:0/1976789106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89ac650 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc101720 tx=0x7f3ea800b390 comp rx=0 tx=0).stop 2024-05-27T07:43:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.226+0000 7f3ecb635740 1 -- 172.21.15.63:0/1976789106 shutdown_connections 2024-05-27T07:43:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.226+0000 7f3ecb635740 1 --2- 172.21.15.63:0/1976789106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db86041b0 0x559db89c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.226+0000 7f3ecb635740 1 --2- 172.21.15.63:0/1976789106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89ac650 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.226+0000 7f3ecb635740 1 --2- 172.21.15.63:0/1976789106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db85db2f0 0x559db89afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.226+0000 7f3ecb635740 1 -- 172.21.15.63:0/1976789106 >> 172.21.15.63:0/1976789106 conn(0x559db85f8300 msgr2=0x559db89af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.226+0000 7f3ecb635740 1 -- 172.21.15.63:0/1976789106 shutdown_connections 2024-05-27T07:43:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.226+0000 7f3ecb635740 1 -- 172.21.15.63:0/1976789106 wait complete. 2024-05-27T07:43:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.227+0000 7f3ecb635740 1 -- start start 2024-05-27T07:43:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.227+0000 7f3ecb635740 1 Processor -- start 2024-05-27T07:43:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85db2f0 0x559db89cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ecb635740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 0x559db8a3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ecb635740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc1ad550 con 0x559db85e34d0 2024-05-27T07:43:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ecb635740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc1ae9c0 con 0x559db85db2f0 2024-05-27T07:43:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ecb635740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc1adc20 con 0x559db86041b0 2024-05-27T07:43:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ecb635740 10 client.0 _ll_register_callbacks cb 0x7f3ea800b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ec1200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ec1200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35586/0 (socket says 172.21.15.63:35586) 2024-05-27T07:43:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ec1200640 1 -- 172.21.15.63:0/1706623448 learned_addr learned my addr 172.21.15.63:0/1706623448 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.228+0000 7f3ec2600640 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85db2f0 0x559db89cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.229+0000 7f3ec2600640 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 0x559db8a3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.229+0000 7f3ec1200640 1 -- 172.21.15.63:0/1706623448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 msgr2=0x559db8a3a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.229+0000 7f3ec1200640 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 0x559db8a3a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.229+0000 7f3ec1200640 1 -- 172.21.15.63:0/1706623448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85db2f0 msgr2=0x559db89cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.229+0000 7f3ec1200640 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85db2f0 0x559db89cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.229+0000 7f3ec1200640 1 -- 172.21.15.63:0/1706623448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db89ce0c0 con 0x559db85e34d0 2024-05-27T07:43:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.229+0000 7f3ec2600640 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 0x559db8a3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.229+0000 7f3ec1200640 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89cd840 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8005890 tx=0x7f3ea80044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518dd4b526d97592 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.230+0000 7f3ea3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.230+0000 7f3ea3400640 1 -- 172.21.15.63:0/1706623448 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8011640 con 0x559db85e34d0 2024-05-27T07:43:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.230+0000 7f3ea3400640 1 -- 172.21.15.63:0/1706623448 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f3ea80117e0 con 0x559db85e34d0 2024-05-27T07:43:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.230+0000 7f3ecb635740 1 -- 172.21.15.63:0/1706623448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea8011a70 con 0x559db85e34d0 2024-05-27T07:43:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.230+0000 7f3ea3400640 1 -- 172.21.15.63:0/1706623448 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3ea8030440 con 0x559db85e34d0 2024-05-27T07:43:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.230+0000 7f3ecb635740 10 client.17322 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.230+0000 7f3ecb635740 1 -- 172.21.15.63:0/1706623448 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559db8a3b8e0 con 0x559db85e34d0 2024-05-27T07:43:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.231+0000 7f3ea3400640 1 -- 172.21.15.63:0/1706623448 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea8011a70 con 0x559db85e34d0 2024-05-27T07:43:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.231+0000 7f3ea3400640 1 -- 172.21.15.63:0/1706623448 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3ea802e070 con 0x559db85e34d0 2024-05-27T07:43:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.231+0000 7f3ecb635740 10 client.17322 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.231+0000 7f3ecb635740 10 client.17322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.231+0000 7f3ecb635740 20 client.17322 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.231+0000 7f3ecb635740 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559db85981c0 0x559db8a3df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.231+0000 7f3ecb635740 4 client.17322 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:43:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.231+0000 7f3ecb635740 1 -- 172.21.15.63:0/1706623448 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559db89132c0 con 0x559db85981c0 2024-05-27T07:43:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.232+0000 7f3ec2600640 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559db85981c0 0x559db8a3df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.233+0000 7f3ec2600640 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559db85981c0 0x559db8a3df60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6deb85ff4bab473 server_cookie=a0e7a7f80ded27ed in_seq=0 out_seq=0 2024-05-27T07:43:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.233+0000 7f3ea3400640 10 client.17322 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.264+0000 7f3ea3400640 1 -- 172.21.15.63:0/1706623448 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559db89132c0 con 0x559db85981c0 2024-05-27T07:43:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.264+0000 7f3ea3400640 10 client.17322 handle_command_reply: tid=0 2024-05-27T07:43:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.264+0000 7f3ecb635740 1 client.17322 shutdown 2024-05-27T07:43:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.265+0000 7f3ecb635740 1 -- 172.21.15.63:0/1706623448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 msgr2=0x559db89cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.265+0000 7f3ecb635740 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89cd840 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8005890 tx=0x7f3ea80044e0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.265+0000 7f3ecb635740 1 -- 172.21.15.63:0/1706623448 shutdown_connections 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.265+0000 7f3ecb635740 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559db85981c0 0x559db8a3df60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.265+0000 7f3ecb635740 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559db86041b0 0x559db8a3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.265+0000 7f3ecb635740 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559db85e34d0 0x559db89cd840 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.265+0000 7f3ea3400640 0 client.17322 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.265+0000 7f3ecb635740 1 --2- 172.21.15.63:0/1706623448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559db85db2f0 0x559db89cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.265+0000 7f3ecb635740 1 -- 172.21.15.63:0/1706623448 >> 172.21.15.63:0/1706623448 conn(0x559db85f8300 msgr2=0x559db89c7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.265+0000 7f3ecb635740 1 -- 172.21.15.63:0/1706623448 shutdown_connections 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.266+0000 7f3ecb635740 1 -- 172.21.15.63:0/1706623448 wait complete. 2024-05-27T07:43:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.266+0000 7f3ecb635740 20 client.17322 trim_cache size 0 max 16384 2024-05-27T07:43:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:43:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:43:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 -- 172.21.15.63:0/3452316894 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3ea40b29d0 msgr2=0x7f3ea40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3ea40b29d0 0x7f3ea40b4ee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x559db89a44d0 tx=0x7f3eb0009260 comp rx=0 tx=0).stop 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 -- 172.21.15.63:0/3452316894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ebc104250 msgr2=0x7f3ebc19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ebc104250 0x7f3ebc19a620 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f3eb002a910 tx=0x7f3eb0003c20 comp rx=0 tx=0).stop 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 -- 172.21.15.63:0/3452316894 shutdown_connections 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3ea40b29d0 0x7f3ea40b4ee0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ebc109f50 0x7f3ebc19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc109ab0 0x7f3ebc19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 --2- 172.21.15.63:0/3452316894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ebc104250 0x7f3ebc19a620 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 -- 172.21.15.63:0/3452316894 >> 172.21.15.63:0/3452316894 conn(0x7f3ebc079240 msgr2=0x7f3ebc100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.267+0000 7f3ec3000640 1 -- 172.21.15.63:0/3452316894 shutdown_connections 2024-05-27T07:43:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:32.268+0000 7f3ec3000640 1 -- 172.21.15.63:0/3452316894 wait complete. 2024-05-27T07:43:32.641 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:33.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:33 smithi086 ceph-mon[34687]: pgmap v310: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:43:33.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:33 smithi063 ceph-mon[31613]: pgmap v310: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:43:33.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:33 smithi149 ceph-mon[35408]: pgmap v310: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:43:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.065+0000 7f570ae00700 1 -- 172.21.15.63:0/1695013014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5704109670 msgr2=0x7f5704110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.065+0000 7f570ae00700 1 --2- 172.21.15.63:0/1695013014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5704109670 0x7f5704110fd0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0099c0 tx=0x7f56fc009cd0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.065+0000 7f570ae00700 1 -- 172.21.15.63:0/1695013014 shutdown_connections 2024-05-27T07:43:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.065+0000 7f570ae00700 1 --2- 172.21.15.63:0/1695013014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5704109670 0x7f5704110fd0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.065+0000 7f570ae00700 1 --2- 172.21.15.63:0/1695013014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57041043f0 0x7f5704104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.065+0000 7f570ae00700 1 --2- 172.21.15.63:0/1695013014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5704108d20 0x7f5704109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.065+0000 7f570ae00700 1 -- 172.21.15.63:0/1695013014 >> 172.21.15.63:0/1695013014 conn(0x7f5704079d00 msgr2=0x7f570407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.065+0000 7f570ae00700 1 -- 172.21.15.63:0/1695013014 shutdown_connections 2024-05-27T07:43:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.065+0000 7f570ae00700 1 -- 172.21.15.63:0/1695013014 wait complete. 2024-05-27T07:43:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.066+0000 7f570ae00700 1 Processor -- start 2024-05-27T07:43:35.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.067+0000 7f570ae00700 1 -- start start 2024-05-27T07:43:35.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.067+0000 7f570ae00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57041043f0 0x7f57041a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:35.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.067+0000 7f570ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704108d20 0x7f57041a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:35.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.067+0000 7f570ae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5704109670 0x7f570419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:35.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.067+0000 7f570ae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57041141a0 con 0x7f5704109670 2024-05-27T07:43:35.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.067+0000 7f570ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5704114020 con 0x7f5704108d20 2024-05-27T07:43:35.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.067+0000 7f570ae00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5704114320 con 0x7f57041043f0 2024-05-27T07:43:35.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f570a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5704109670 0x7f570419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:35.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f570a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5704109670 0x7f570419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35594/0 (socket says 172.21.15.63:35594) 2024-05-27T07:43:35.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f570a400700 1 -- 172.21.15.63:0/2125514590 learned_addr learned my addr 172.21.15.63:0/2125514590 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f5709000700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704108d20 0x7f57041a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f5709a00700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57041043f0 0x7f57041a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f570a400700 1 -- 172.21.15.63:0/2125514590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57041043f0 msgr2=0x7f57041a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f570a400700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57041043f0 0x7f57041a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f570a400700 1 -- 172.21.15.63:0/2125514590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704108d20 msgr2=0x7f57041a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f570a400700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704108d20 0x7f57041a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f570a400700 1 -- 172.21.15.63:0/2125514590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570419b090 con 0x7f5704109670 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.068+0000 7f5709000700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704108d20 0x7f57041a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.069+0000 7f570a400700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5704109670 0x7f570419a9e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00b580 tx=0x7f56fc00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.069+0000 7f5702000700 1 -- 172.21.15.63:0/2125514590 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc027070 con 0x7f5704109670 2024-05-27T07:43:35.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.069+0000 7f5702000700 1 -- 172.21.15.63:0/2125514590 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56fc00fd20 con 0x7f5704109670 2024-05-27T07:43:35.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.069+0000 7f570ae00700 1 -- 172.21.15.63:0/2125514590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570419b320 con 0x7f5704109670 2024-05-27T07:43:35.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.069+0000 7f5702000700 1 -- 172.21.15.63:0/2125514590 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc022df0 con 0x7f5704109670 2024-05-27T07:43:35.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.069+0000 7f570ae00700 1 -- 172.21.15.63:0/2125514590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570419b800 con 0x7f5704109670 2024-05-27T07:43:35.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.071+0000 7f570ae00700 1 -- 172.21.15.63:0/2125514590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57040512b0 con 0x7f5704109670 2024-05-27T07:43:35.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.074+0000 7f5702000700 1 -- 172.21.15.63:0/2125514590 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f56fc02a070 con 0x7f5704109670 2024-05-27T07:43:35.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.075+0000 7f5702000700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56ec0b29f0 0x7f56ec0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:35.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.076+0000 7f5702000700 1 -- 172.21.15.63:0/2125514590 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f56fc0e3940 con 0x7f5704109670 2024-05-27T07:43:35.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.076+0000 7f5709a00700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56ec0b29f0 0x7f56ec0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:35.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.077+0000 7f5709a00700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56ec0b29f0 0x7f56ec0b4f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009550 tx=0x7f56f80091b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:35.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.079+0000 7f5702000700 1 -- 172.21.15.63:0/2125514590 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56fc013070 con 0x7f5704109670 2024-05-27T07:43:35.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.386+0000 7f570ae00700 1 -- 172.21.15.63:0/2125514590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f570402e5e0 con 0x7f5704109670 2024-05-27T07:43:35.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.388+0000 7f5702000700 1 -- 172.21.15.63:0/2125514590 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f56fc0aaf90 con 0x7f5704109670 2024-05-27T07:43:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:43:35.386 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:43:35.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.391+0000 7f570ae00700 1 -- 172.21.15.63:0/2125514590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56ec0b29f0 msgr2=0x7f56ec0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:35.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.391+0000 7f570ae00700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56ec0b29f0 0x7f56ec0b4f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009550 tx=0x7f56f80091b0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.391+0000 7f570ae00700 1 -- 172.21.15.63:0/2125514590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5704109670 msgr2=0x7f570419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:35.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.391+0000 7f570ae00700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5704109670 0x7f570419a9e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00b580 tx=0x7f56fc00f460 comp rx=0 tx=0).stop 2024-05-27T07:43:35.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.391+0000 7f570ae00700 1 -- 172.21.15.63:0/2125514590 shutdown_connections 2024-05-27T07:43:35.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.391+0000 7f570ae00700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56ec0b29f0 0x7f56ec0b4f00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.391+0000 7f570ae00700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5704109670 0x7f570419a9e0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.391+0000 7f570ae00700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57041043f0 0x7f57041a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.391+0000 7f570ae00700 1 --2- 172.21.15.63:0/2125514590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5704108d20 0x7f57041a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:35.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.391+0000 7f570ae00700 1 -- 172.21.15.63:0/2125514590 >> 172.21.15.63:0/2125514590 conn(0x7f5704079d00 msgr2=0x7f5704101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:35.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.392+0000 7f570ae00700 1 -- 172.21.15.63:0/2125514590 shutdown_connections 2024-05-27T07:43:35.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:35.392+0000 7f570ae00700 1 -- 172.21.15.63:0/2125514590 wait complete. 2024-05-27T07:43:35.391 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:43:35.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:35 smithi063 ceph-mon[31613]: pgmap v311: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T07:43:35.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:35 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2125514590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:43:35.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:35 smithi086 ceph-mon[34687]: pgmap v311: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T07:43:35.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:35 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2125514590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:43:35.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:35 smithi149 ceph-mon[35408]: pgmap v311: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T07:43:35.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:35 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2125514590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:43:36.356 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:43:36.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:36 smithi063 ceph-mon[31613]: pgmap v312: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T07:43:36.869 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:36.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:36 smithi086 ceph-mon[34687]: pgmap v312: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T07:43:36.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:36 smithi149 ceph-mon[35408]: pgmap v312: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T07:43:38.817 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:38 smithi063 ceph-mon[31613]: pgmap v313: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:43:38.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:38 smithi086 ceph-mon[34687]: pgmap v313: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:43:38.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:38 smithi149 ceph-mon[35408]: pgmap v313: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:43:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.622+0000 7f2247000700 1 -- 172.21.15.63:0/1817703329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2240109860 msgr2=0x7f22401111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.622+0000 7f2247000700 1 --2- 172.21.15.63:0/1817703329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2240109860 0x7f22401111c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f223800b0d0 tx=0x7f223800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.623+0000 7f2247000700 1 -- 172.21.15.63:0/1817703329 shutdown_connections 2024-05-27T07:43:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.623+0000 7f2247000700 1 --2- 172.21.15.63:0/1817703329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2240109860 0x7f22401111c0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.623+0000 7f2247000700 1 --2- 172.21.15.63:0/1817703329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22401045e0 0x7f22401049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.623+0000 7f2247000700 1 --2- 172.21.15.63:0/1817703329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2240108f10 0x7f2240109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.623+0000 7f2247000700 1 -- 172.21.15.63:0/1817703329 >> 172.21.15.63:0/1817703329 conn(0x7f2240100030 msgr2=0x7f2240102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.623+0000 7f2247000700 1 -- 172.21.15.63:0/1817703329 shutdown_connections 2024-05-27T07:43:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.623+0000 7f2247000700 1 -- 172.21.15.63:0/1817703329 wait complete. 2024-05-27T07:43:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.624+0000 7f2247000700 1 Processor -- start 2024-05-27T07:43:40.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.625+0000 7f2247000700 1 -- start start 2024-05-27T07:43:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.625+0000 7f2247000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22401045e0 0x7f224019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.625+0000 7f2247000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2240108f10 0x7f224019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.625+0000 7f2247000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2240109860 0x7f22401a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.625+0000 7f2247000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2240114390 con 0x7f22401045e0 2024-05-27T07:43:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.625+0000 7f2247000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2240114210 con 0x7f2240109860 2024-05-27T07:43:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.625+0000 7f2247000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2240114510 con 0x7f2240108f10 2024-05-27T07:43:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.626+0000 7f223fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22401045e0 0x7f224019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.626+0000 7f223fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22401045e0 0x7f224019cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37524/0 (socket says 172.21.15.63:37524) 2024-05-27T07:43:40.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.626+0000 7f223fe00700 1 -- 172.21.15.63:0/2116463204 learned_addr learned my addr 172.21.15.63:0/2116463204 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:40.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.626+0000 7f223f400700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2240108f10 0x7f224019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.626+0000 7f2245000700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2240109860 0x7f22401a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.626+0000 7f223fe00700 1 -- 172.21.15.63:0/2116463204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2240108f10 msgr2=0x7f224019d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.626+0000 7f223fe00700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2240108f10 0x7f224019d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.626+0000 7f223fe00700 1 -- 172.21.15.63:0/2116463204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2240109860 msgr2=0x7f22401a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.626+0000 7f223fe00700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2240109860 0x7f22401a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.626+0000 7f223fe00700 1 -- 172.21.15.63:0/2116463204 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22401a1e50 con 0x7f22401045e0 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.627+0000 7f223fe00700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22401045e0 0x7f224019cf80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f223000d950 tx=0x7f223000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.627+0000 7f223cc00700 1 -- 172.21.15.63:0/2116463204 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230016070 con 0x7f22401045e0 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.627+0000 7f223cc00700 1 -- 172.21.15.63:0/2116463204 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2230010460 con 0x7f22401045e0 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.627+0000 7f223cc00700 1 -- 172.21.15.63:0/2116463204 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223000f620 con 0x7f22401045e0 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.627+0000 7f2247000700 1 -- 172.21.15.63:0/2116463204 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22401a2140 con 0x7f22401045e0 2024-05-27T07:43:40.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.627+0000 7f2247000700 1 -- 172.21.15.63:0/2116463204 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22401a9a60 con 0x7f22401045e0 2024-05-27T07:43:40.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.632+0000 7f223cc00700 1 -- 172.21.15.63:0/2116463204 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2230019070 con 0x7f22401045e0 2024-05-27T07:43:40.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.633+0000 7f2255b8eb80 1 Processor -- start 2024-05-27T07:43:40.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.633+0000 7f2255b8eb80 1 -- start start 2024-05-27T07:43:40.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.633+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fb429f0 0x55cd1fbd5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.633+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.633+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.633+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cd1fb03e40 con 0x55cd1fa56e40 2024-05-27T07:43:40.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.633+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd1faf39e0 con 0x55cd1fbd5e70 2024-05-27T07:43:40.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.633+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cd1faf7a20 con 0x55cd1fb429f0 2024-05-27T07:43:40.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.634+0000 7f223fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.634+0000 7f223fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37530/0 (socket says 172.21.15.63:37530) 2024-05-27T07:43:40.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.634+0000 7f223fe00700 1 -- 172.21.15.63:0/3465574221 learned_addr learned my addr 172.21.15.63:0/3465574221 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:40.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.634+0000 7f223f400700 1 --2- 172.21.15.63:0/3465574221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.634+0000 7f2245000700 1 --2- 172.21.15.63:0/3465574221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fb429f0 0x55cd1fbd5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.634+0000 7f223fe00700 1 -- 172.21.15.63:0/3465574221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fb429f0 msgr2=0x55cd1fbd5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:40.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.634+0000 7f223fe00700 1 --2- 172.21.15.63:0/3465574221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fb429f0 0x55cd1fbd5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.634+0000 7f223fe00700 1 -- 172.21.15.63:0/3465574221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fbd5e70 msgr2=0x55cd1fbe2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:40.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.634+0000 7f223fe00700 1 --2- 172.21.15.63:0/3465574221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.635+0000 7f223fe00700 1 -- 172.21.15.63:0/3465574221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1fabc420 con 0x55cd1fa56e40 2024-05-27T07:43:40.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.635+0000 7f223cc00700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22240b29f0 0x7f22240b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.635+0000 7f223cc00700 1 -- 172.21.15.63:0/2116463204 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f22300e2650 con 0x7f22401045e0 2024-05-27T07:43:40.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.635+0000 7f223fe00700 1 --2- 172.21.15.63:0/3465574221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd5870 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f22300df8c0 tx=0x7f22300e8ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59f857dc16f29fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:40.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.635+0000 7f2245000700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22240b29f0 0x7f22240b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.636+0000 7f2235600700 1 -- 172.21.15.63:0/3465574221 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230020020 con 0x55cd1fa56e40 2024-05-27T07:43:40.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.636+0000 7f2235600700 1 -- 172.21.15.63:0/3465574221 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22300a9020 con 0x55cd1fa56e40 2024-05-27T07:43:40.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.636+0000 7f2235600700 1 -- 172.21.15.63:0/3465574221 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22300a92e0 con 0x55cd1fa56e40 2024-05-27T07:43:40.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.637+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3465574221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 msgr2=0x55cd1fbd5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:40.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.637+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/3465574221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd5870 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f22300df8c0 tx=0x7f22300e8ed0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.637+0000 7f2245000700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22240b29f0 0x7f22240b4f00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55cd1fbd7ff0 tx=0x7f223800b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:40.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.637+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3465574221 shutdown_connections 2024-05-27T07:43:40.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.637+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/3465574221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.637+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/3465574221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fb429f0 0x55cd1fbd5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.637+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/3465574221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd5870 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.637+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3465574221 >> 172.21.15.63:0/3465574221 conn(0x55cd1faf0390 msgr2=0x55cd1fbd14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:40.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.637+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3465574221 shutdown_connections 2024-05-27T07:43:40.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.637+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3465574221 wait complete. 2024-05-27T07:43:40.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.638+0000 7f2255b8eb80 1 -- start start 2024-05-27T07:43:40.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.638+0000 7f2255b8eb80 1 Processor -- start 2024-05-27T07:43:40.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.639+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fa56e40 0x55cd1fbdf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.639+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fb429f0 0x55cd1fbdfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.639+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.639+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f9f9fa0 con 0x55cd1fbd5e70 2024-05-27T07:43:40.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.639+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd1fa94760 con 0x55cd1fa56e40 2024-05-27T07:43:40.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.639+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cd1fbe2bd0 con 0x55cd1fb429f0 2024-05-27T07:43:40.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.639+0000 7f2255b8eb80 10 client.0 _ll_register_callbacks cb 0x55cd1fa9dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:40.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.639+0000 7f2245000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.639+0000 7f2245000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37546/0 (socket says 172.21.15.63:37546) 2024-05-27T07:43:40.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.639+0000 7f2245000700 1 -- 172.21.15.63:0/3398801583 learned_addr learned my addr 172.21.15.63:0/3398801583 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:40.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.640+0000 7f223f400700 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fb429f0 0x55cd1fbdfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.640+0000 7f223fe00700 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fa56e40 0x55cd1fbdf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.640+0000 7f2245000700 1 -- 172.21.15.63:0/3398801583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fb429f0 msgr2=0x55cd1fbdfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:40.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.640+0000 7f2245000700 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fb429f0 0x55cd1fbdfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.640+0000 7f2245000700 1 -- 172.21.15.63:0/3398801583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fa56e40 msgr2=0x55cd1fbdf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:40.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.640+0000 7f2245000700 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fa56e40 0x55cd1fbdf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.640+0000 7f2245000700 1 -- 172.21.15.63:0/3398801583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1fa438d0 con 0x55cd1fbd5e70 2024-05-27T07:43:40.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.641+0000 7f2245000700 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe01f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2238007940 tx=0x7f22380099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48442e65c7d245 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:40.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.641+0000 7f222a000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.641+0000 7f222a000700 1 -- 172.21.15.63:0/3398801583 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223802d070 con 0x55cd1fbd5e70 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.641+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3398801583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22300a9020 con 0x55cd1fbd5e70 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.641+0000 7f222a000700 1 -- 172.21.15.63:0/3398801583 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f2238029830 con 0x55cd1fbd5e70 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.641+0000 7f222a000700 1 -- 172.21.15.63:0/3398801583 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f223803a050 con 0x55cd1fbd5e70 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.641+0000 7f2255b8eb80 10 client.17340 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.641+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3398801583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cd1fac4060 con 0x55cd1fbd5e70 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.642+0000 7f222a000700 1 -- 172.21.15.63:0/3398801583 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22300a9020 con 0x55cd1fbd5e70 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.642+0000 7f222a000700 1 -- 172.21.15.63:0/3398801583 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2238011070 con 0x55cd1fbd5e70 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.642+0000 7f2255b8eb80 10 client.17340 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.642+0000 7f2255b8eb80 10 client.17340 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.642+0000 7f2255b8eb80 20 client.17340 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.642+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55cd1fbefb10 0x55cd1fbf1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.642+0000 7f2255b8eb80 4 client.17340 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:40.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.642+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3398801583 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd1fb05d90 con 0x55cd1fbefb10 2024-05-27T07:43:40.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.643+0000 7f223fe00700 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55cd1fbefb10 0x55cd1fbf1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.644+0000 7f222a000700 10 client.17340 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:43:40.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.644+0000 7f223fe00700 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55cd1fbefb10 0x55cd1fbf1f90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea5e96b22678a9b server_cookie=4ecdb0a608db596a in_seq=0 out_seq=0 2024-05-27T07:43:40.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.646+0000 7f222a000700 1 -- 172.21.15.63:0/3398801583 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cd1fb05d90 con 0x55cd1fbefb10 2024-05-27T07:43:40.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.646+0000 7f222a000700 10 client.17340 handle_command_reply: tid=0 2024-05-27T07:43:40.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.646+0000 7f2255b8eb80 1 client.17340 shutdown 2024-05-27T07:43:40.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.646+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3398801583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 msgr2=0x55cd1fbe01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:40.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.646+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe01f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2238007940 tx=0x7f22380099e0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.647+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3398801583 shutdown_connections 2024-05-27T07:43:40.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.647+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55cd1fbefb10 0x55cd1fbf1f90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.647+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe01f0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.647+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fa56e40 0x55cd1fbdf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.647+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/3398801583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fb429f0 0x55cd1fbdfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.647+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3398801583 >> 172.21.15.63:0/3398801583 conn(0x55cd1faf0390 msgr2=0x55cd1fbd53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:40.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.647+0000 7f222a000700 0 client.17340 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:43:40.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.647+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3398801583 shutdown_connections 2024-05-27T07:43:40.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.647+0000 7f2255b8eb80 1 -- 172.21.15.63:0/3398801583 wait complete. 2024-05-27T07:43:40.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.647+0000 7f2255b8eb80 20 client.17340 trim_cache size 0 max 16384 2024-05-27T07:43:40.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.664+0000 7f2255b8eb80 1 Processor -- start 2024-05-27T07:43:40.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.664+0000 7f2255b8eb80 1 -- start start 2024-05-27T07:43:40.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fa56e40 0x55cd1fbdaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fb429f0 0x55cd1fbe0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbd5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22401b5020 con 0x55cd1fbd5e70 2024-05-27T07:43:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22401b28b0 con 0x55cd1fb429f0 2024-05-27T07:43:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22401b0720 con 0x55cd1fa56e40 2024-05-27T07:43:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f2245000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbd5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f2245000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbd5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37550/0 (socket says 172.21.15.63:37550) 2024-05-27T07:43:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f2245000700 1 -- 172.21.15.63:0/4150845381 learned_addr learned my addr 172.21.15.63:0/4150845381 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f223fe00700 1 --2- 172.21.15.63:0/4150845381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fa56e40 0x55cd1fbdaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.665+0000 7f223f400700 1 --2- 172.21.15.63:0/4150845381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fb429f0 0x55cd1fbe0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.666+0000 7f2245000700 1 -- 172.21.15.63:0/4150845381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fa56e40 msgr2=0x55cd1fbdaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.666+0000 7f2245000700 1 --2- 172.21.15.63:0/4150845381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fa56e40 0x55cd1fbdaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.666+0000 7f2245000700 1 -- 172.21.15.63:0/4150845381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fb429f0 msgr2=0x55cd1fbe0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.666+0000 7f2245000700 1 --2- 172.21.15.63:0/4150845381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fb429f0 0x55cd1fbe0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.666+0000 7f2245000700 1 -- 172.21.15.63:0/4150845381 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1fbcd240 con 0x55cd1fbd5e70 2024-05-27T07:43:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.666+0000 7f2245000700 1 --2- 172.21.15.63:0/4150845381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbd5820 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f22380062a0 tx=0x7f223801ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68433e688c9503f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:40.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.666+0000 7f2234c00700 1 -- 172.21.15.63:0/4150845381 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223802d070 con 0x55cd1fbd5e70 2024-05-27T07:43:40.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.667+0000 7f2234c00700 1 -- 172.21.15.63:0/4150845381 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2238029630 con 0x55cd1fbd5e70 2024-05-27T07:43:40.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.667+0000 7f2234c00700 1 -- 172.21.15.63:0/4150845381 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238023070 con 0x55cd1fbd5e70 2024-05-27T07:43:40.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.668+0000 7f2255b8eb80 1 -- 172.21.15.63:0/4150845381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 msgr2=0x55cd1fbd5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.668+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/4150845381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbd5820 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f22380062a0 tx=0x7f223801ef20 comp rx=0 tx=0).stop 2024-05-27T07:43:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.668+0000 7f2255b8eb80 1 -- 172.21.15.63:0/4150845381 shutdown_connections 2024-05-27T07:43:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.668+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/4150845381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbd5820 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.668+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/4150845381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fa56e40 0x55cd1fbdaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.668+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/4150845381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fb429f0 0x55cd1fbe0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.668+0000 7f2255b8eb80 1 -- 172.21.15.63:0/4150845381 >> 172.21.15.63:0/4150845381 conn(0x55cd1faf0390 msgr2=0x55cd1fbe0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.668+0000 7f2255b8eb80 1 -- 172.21.15.63:0/4150845381 shutdown_connections 2024-05-27T07:43:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.668+0000 7f2255b8eb80 1 -- 172.21.15.63:0/4150845381 wait complete. 2024-05-27T07:43:40.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.668+0000 7f2255b8eb80 1 -- start start 2024-05-27T07:43:40.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.669+0000 7f2255b8eb80 1 Processor -- start 2024-05-27T07:43:40.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.669+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.669+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fb429f0 0x55cd1fbda020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.669+0000 7f2255b8eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.669+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22401b0d70 con 0x55cd1fa56e40 2024-05-27T07:43:40.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.669+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22401b21e0 con 0x55cd1fb429f0 2024-05-27T07:43:40.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.669+0000 7f2255b8eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22401b1440 con 0x55cd1fbd5e70 2024-05-27T07:43:40.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.669+0000 7f2255b8eb80 10 client.0 _ll_register_callbacks cb 0x55cd1fa9dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:40.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.670+0000 7f223fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.670+0000 7f223fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37560/0 (socket says 172.21.15.63:37560) 2024-05-27T07:43:40.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.670+0000 7f223fe00700 1 -- 172.21.15.63:0/960844053 learned_addr learned my addr 172.21.15.63:0/960844053 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:40.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.670+0000 7f223f400700 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fb429f0 0x55cd1fbda020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.670+0000 7f2245000700 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.670+0000 7f223fe00700 1 -- 172.21.15.63:0/960844053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fbd5e70 msgr2=0x55cd1fbe6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:40.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.670+0000 7f223fe00700 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.670+0000 7f223fe00700 1 -- 172.21.15.63:0/960844053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fb429f0 msgr2=0x55cd1fbda020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:40.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.670+0000 7f223fe00700 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fb429f0 0x55cd1fbda020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.670+0000 7f223fe00700 1 -- 172.21.15.63:0/960844053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1fbe73d0 con 0x55cd1fa56e40 2024-05-27T07:43:40.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.671+0000 7f223fe00700 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd9ae0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f22300a6780 tx=0x7f2230005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485ec1bf688a50b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:40.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.671+0000 7f222a000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:40.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.671+0000 7f222a000700 1 -- 172.21.15.63:0/960844053 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22300c2070 con 0x55cd1fa56e40 2024-05-27T07:43:40.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.671+0000 7f222a000700 1 -- 172.21.15.63:0/960844053 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f22300a57a0 con 0x55cd1fa56e40 2024-05-27T07:43:40.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.671+0000 7f2255b8eb80 1 -- 172.21.15.63:0/960844053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2238029630 con 0x55cd1fa56e40 2024-05-27T07:43:40.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.671+0000 7f222a000700 1 -- 172.21.15.63:0/960844053 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2230020040 con 0x55cd1fa56e40 2024-05-27T07:43:40.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.671+0000 7f2255b8eb80 10 client.17346 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:40.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.671+0000 7f2255b8eb80 1 -- 172.21.15.63:0/960844053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cd1fbe8630 con 0x55cd1fa56e40 2024-05-27T07:43:40.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.672+0000 7f222a000700 1 -- 172.21.15.63:0/960844053 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2238029630 con 0x55cd1fa56e40 2024-05-27T07:43:40.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.672+0000 7f222a000700 1 -- 172.21.15.63:0/960844053 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2230010930 con 0x55cd1fa56e40 2024-05-27T07:43:40.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.672+0000 7f2255b8eb80 10 client.17346 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:40.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.672+0000 7f2255b8eb80 10 client.17346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:43:40.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.672+0000 7f2255b8eb80 20 client.17346 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:40.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.672+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55cd1fc5a620 0x55cd1fc5ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:40.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.672+0000 7f2255b8eb80 4 client.17346 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:43:40.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.672+0000 7f2255b8eb80 1 -- 172.21.15.63:0/960844053 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd1faff570 con 0x55cd1fc5a620 2024-05-27T07:43:40.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.673+0000 7f223f400700 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55cd1fc5a620 0x55cd1fc5ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:40.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.674+0000 7f222a000700 10 client.17346 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:43:40.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.674+0000 7f223f400700 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55cd1fc5a620 0x55cd1fc5ca40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78fc4dff14d9cd0e server_cookie=2aef762cf2b2e60 in_seq=0 out_seq=0 2024-05-27T07:43:40.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.674+0000 7f222a000700 1 -- 172.21.15.63:0/960844053 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd1faff570 con 0x55cd1fc5a620 2024-05-27T07:43:40.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.674+0000 7f222a000700 10 client.17346 handle_command_reply: tid=0 2024-05-27T07:43:40.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.674+0000 7f2255b8eb80 1 client.17346 shutdown 2024-05-27T07:43:40.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 1 -- 172.21.15.63:0/960844053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 msgr2=0x55cd1fbd9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:40.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd9ae0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f22300a6780 tx=0x7f2230005c80 comp rx=0 tx=0).stop 2024-05-27T07:43:40.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 1 -- 172.21.15.63:0/960844053 shutdown_connections 2024-05-27T07:43:40.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd1fbd5e70 0x55cd1fbe6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55cd1fc5a620 0x55cd1fc5ca40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd1fa56e40 0x55cd1fbd9ae0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 1 --2- 172.21.15.63:0/960844053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd1fb429f0 0x55cd1fbda020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 1 -- 172.21.15.63:0/960844053 >> 172.21.15.63:0/960844053 conn(0x55cd1faf0390 msgr2=0x55cd1fbde6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:40.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f222a000700 0 client.17346 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:43:40.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 1 -- 172.21.15.63:0/960844053 shutdown_connections 2024-05-27T07:43:40.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 1 -- 172.21.15.63:0/960844053 wait complete. 2024-05-27T07:43:40.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.675+0000 7f2255b8eb80 20 client.17346 trim_cache size 0 max 16384 2024-05-27T07:43:40.675 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:43:40.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.676+0000 7f2247000700 1 -- 172.21.15.63:0/2116463204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22240b29f0 msgr2=0x7f22240b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:40.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.676+0000 7f2247000700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22240b29f0 0x7f22240b4f00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55cd1fbd7ff0 tx=0x7f223800b040 comp rx=0 tx=0).stop 2024-05-27T07:43:40.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.676+0000 7f2247000700 1 -- 172.21.15.63:0/2116463204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22401045e0 msgr2=0x7f224019cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:40.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.676+0000 7f2247000700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22401045e0 0x7f224019cf80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f223000d950 tx=0x7f223000dd10 comp rx=0 tx=0).stop 2024-05-27T07:43:40.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.676+0000 7f2247000700 1 -- 172.21.15.63:0/2116463204 shutdown_connections 2024-05-27T07:43:40.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.676+0000 7f2247000700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22240b29f0 0x7f22240b4f00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.676+0000 7f2247000700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2240109860 0x7f22401a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.676+0000 7f2247000700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22401045e0 0x7f224019cf80 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.676+0000 7f2247000700 1 --2- 172.21.15.63:0/2116463204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2240108f10 0x7f224019d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:40.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.676+0000 7f2247000700 1 -- 172.21.15.63:0/2116463204 >> 172.21.15.63:0/2116463204 conn(0x7f2240100030 msgr2=0x7f224010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:40.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.677+0000 7f2247000700 1 -- 172.21.15.63:0/2116463204 shutdown_connections 2024-05-27T07:43:40.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:40.677+0000 7f2247000700 1 -- 172.21.15.63:0/2116463204 wait complete. 2024-05-27T07:43:41.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:41 smithi063 ceph-mon[31613]: pgmap v314: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:43:41.631 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:43:41.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:41 smithi086 ceph-mon[34687]: pgmap v314: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:43:41.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:41 smithi149 ceph-mon[35408]: pgmap v314: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:43:42.110 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:43.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:43 smithi086 ceph-mon[34687]: pgmap v315: 129 pgs: 129 active+clean; 2.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:43:43.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:43 smithi149 ceph-mon[35408]: pgmap v315: 129 pgs: 129 active+clean; 2.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:43:43.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:43 smithi063 ceph-mon[31613]: pgmap v315: 129 pgs: 129 active+clean; 2.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:43:44.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.588+0000 7f769ac00700 1 -- 172.21.15.63:0/405093405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7694110bf0 msgr2=0x7f7694113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:44.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.588+0000 7f769ac00700 1 --2- 172.21.15.63:0/405093405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7694110bf0 0x7f7694113010 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f768c00b0a0 tx=0x7f768c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.588+0000 7f769ac00700 1 -- 172.21.15.63:0/405093405 shutdown_connections 2024-05-27T07:43:44.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.588+0000 7f769ac00700 1 --2- 172.21.15.63:0/405093405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7694110bf0 0x7f7694113010 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.588+0000 7f769ac00700 1 --2- 172.21.15.63:0/405093405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7694075af0 0x7f7694075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.588+0000 7f769ac00700 1 --2- 172.21.15.63:0/405093405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7694076440 0x7f76941106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.588+0000 7f769ac00700 1 -- 172.21.15.63:0/405093405 >> 172.21.15.63:0/405093405 conn(0x7f7694100030 msgr2=0x7f7694102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:44.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.588+0000 7f769ac00700 1 -- 172.21.15.63:0/405093405 shutdown_connections 2024-05-27T07:43:44.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.588+0000 7f769ac00700 1 -- 172.21.15.63:0/405093405 wait complete. 2024-05-27T07:43:44.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.590+0000 7f769ac00700 1 Processor -- start 2024-05-27T07:43:44.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.590+0000 7f769ac00700 1 -- start start 2024-05-27T07:43:44.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.591+0000 7f769ac00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7694075af0 0x7f769419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.591+0000 7f769ac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7694076440 0x7f769419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.591+0000 7f769ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7694110bf0 0x7f76941a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.591+0000 7f769ac00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76941165b0 con 0x7f7694075af0 2024-05-27T07:43:44.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.591+0000 7f769ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7694116430 con 0x7f7694110bf0 2024-05-27T07:43:44.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.591+0000 7f769ac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7694116730 con 0x7f7694076440 2024-05-27T07:43:44.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.591+0000 7f7693e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7694075af0 0x7f769419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.591+0000 7f7693e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7694075af0 0x7f769419cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37584/0 (socket says 172.21.15.63:37584) 2024-05-27T07:43:44.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.591+0000 7f7693e00700 1 -- 172.21.15.63:0/4254678133 learned_addr learned my addr 172.21.15.63:0/4254678133 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:44.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.591+0000 7f7693400700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7694076440 0x7f769419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.592+0000 7f7698c00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7694110bf0 0x7f76941a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.592+0000 7f7693e00700 1 -- 172.21.15.63:0/4254678133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7694076440 msgr2=0x7f769419d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:44.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.592+0000 7f7693e00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7694076440 0x7f769419d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.592+0000 7f7693e00700 1 -- 172.21.15.63:0/4254678133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7694110bf0 msgr2=0x7f76941a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:44.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.592+0000 7f7693e00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7694110bf0 0x7f76941a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.592+0000 7f7693e00700 1 -- 172.21.15.63:0/4254678133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76941a1d80 con 0x7f7694075af0 2024-05-27T07:43:44.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.593+0000 7f7693e00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7694075af0 0x7f769419cf60 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f768400d950 tx=0x7f768400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:44.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.593+0000 7f7690c00700 1 -- 172.21.15.63:0/4254678133 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684016070 con 0x7f7694075af0 2024-05-27T07:43:44.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.593+0000 7f7690c00700 1 -- 172.21.15.63:0/4254678133 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f768400fd20 con 0x7f7694075af0 2024-05-27T07:43:44.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.593+0000 7f769ac00700 1 -- 172.21.15.63:0/4254678133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76941a2010 con 0x7f7694075af0 2024-05-27T07:43:44.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.593+0000 7f7690c00700 1 -- 172.21.15.63:0/4254678133 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768400e3b0 con 0x7f7694075af0 2024-05-27T07:43:44.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.593+0000 7f769ac00700 1 -- 172.21.15.63:0/4254678133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76941a25a0 con 0x7f7694075af0 2024-05-27T07:43:44.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.597+0000 7f7690c00700 1 -- 172.21.15.63:0/4254678133 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7684019070 con 0x7f7694075af0 2024-05-27T07:43:44.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.598+0000 7f76a97c1b80 1 Processor -- start 2024-05-27T07:43:44.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.599+0000 7f76a97c1b80 1 -- start start 2024-05-27T07:43:44.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.599+0000 7f7690c00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76780b2a70 0x7f76780b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.599+0000 7f7690c00700 1 -- 172.21.15.63:0/4254678133 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f76840e2460 con 0x7f7694075af0 2024-05-27T07:43:44.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.599+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a076b89f0 0x564a0774b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.599+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a0774b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.599+0000 7f7698c00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76780b2a70 0x7f76780b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.599+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a07760940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.599+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564a07679e40 con 0x564a075cce40 2024-05-27T07:43:44.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.599+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a076699e0 con 0x564a076b89f0 2024-05-27T07:43:44.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.599+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564a0766da20 con 0x564a0774bdb0 2024-05-27T07:43:44.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.600+0000 7f7693400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a0774b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.600+0000 7f7693400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a0774b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37592/0 (socket says 172.21.15.63:37592) 2024-05-27T07:43:44.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.600+0000 7f7693400700 1 -- 172.21.15.63:0/214874573 learned_addr learned my addr 172.21.15.63:0/214874573 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:44.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.600+0000 7f7693e00700 1 --2- 172.21.15.63:0/214874573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a076b89f0 0x564a0774b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.601+0000 7f7693400700 1 -- 172.21.15.63:0/214874573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 msgr2=0x564a07760940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:44.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.601+0000 7f7693400700 1 --2- 172.21.15.63:0/214874573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a07760940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.601+0000 7f7693400700 1 -- 172.21.15.63:0/214874573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a076b89f0 msgr2=0x564a0774b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:44.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.601+0000 7f7693400700 1 --2- 172.21.15.63:0/214874573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a076b89f0 0x564a0774b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.601+0000 7f7693400700 1 -- 172.21.15.63:0/214874573 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a07632420 con 0x564a075cce40 2024-05-27T07:43:44.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.601+0000 7f7698c00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76780b2a70 0x7f76780b4f80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f768c00b830 tx=0x7f768c009270 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:44.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.601+0000 7f7693400700 1 --2- 172.21.15.63:0/214874573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a0774b7b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f768000ac60 tx=0x7f768000af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b299690a215ad9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:44.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.601+0000 7f7689600700 1 -- 172.21.15.63:0/214874573 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768000fd60 con 0x564a075cce40 2024-05-27T07:43:44.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.602+0000 7f7689600700 1 -- 172.21.15.63:0/214874573 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76800044c0 con 0x564a075cce40 2024-05-27T07:43:44.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.602+0000 7f7689600700 1 -- 172.21.15.63:0/214874573 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680018430 con 0x564a075cce40 2024-05-27T07:43:44.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.603+0000 7f76a97c1b80 1 -- 172.21.15.63:0/214874573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 msgr2=0x564a0774b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:44.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.603+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/214874573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a0774b7b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f768000ac60 tx=0x7f768000af70 comp rx=0 tx=0).stop 2024-05-27T07:43:44.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.603+0000 7f76a97c1b80 1 -- 172.21.15.63:0/214874573 shutdown_connections 2024-05-27T07:43:44.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.603+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/214874573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a07760940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.603+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/214874573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a076b89f0 0x564a0774b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.603+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/214874573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a0774b7b0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.603+0000 7f76a97c1b80 1 -- 172.21.15.63:0/214874573 >> 172.21.15.63:0/214874573 conn(0x564a07666390 msgr2=0x564a077474c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:44.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.603+0000 7f76a97c1b80 1 -- 172.21.15.63:0/214874573 shutdown_connections 2024-05-27T07:43:44.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.603+0000 7f76a97c1b80 1 -- 172.21.15.63:0/214874573 wait complete. 2024-05-27T07:43:44.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.604+0000 7f76a97c1b80 1 -- start start 2024-05-27T07:43:44.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.604+0000 7f76a97c1b80 1 Processor -- start 2024-05-27T07:43:44.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 0x564a0775de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a0775e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a0775e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564a0756ffa0 con 0x564a076b89f0 2024-05-27T07:43:44.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a0760a760 con 0x564a075cce40 2024-05-27T07:43:44.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564a077613e0 con 0x564a0774bdb0 2024-05-27T07:43:44.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f76a97c1b80 10 client.0 _ll_register_callbacks cb 0x564a07613c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:44.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f7693400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a0775e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f7693400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a0775e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37608/0 (socket says 172.21.15.63:37608) 2024-05-27T07:43:44.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f7693400700 1 -- 172.21.15.63:0/2593258025 learned_addr learned my addr 172.21.15.63:0/2593258025 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:44.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f7693e00700 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 0x564a0775de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.605+0000 7f7698c00700 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a0775e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.606+0000 7f7693400700 1 -- 172.21.15.63:0/2593258025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 msgr2=0x564a0775e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:44.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.606+0000 7f7693400700 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a0775e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.606+0000 7f7693400700 1 -- 172.21.15.63:0/2593258025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 msgr2=0x564a0775de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:44.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.606+0000 7f7693400700 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 0x564a0775de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.606+0000 7f7693400700 1 -- 172.21.15.63:0/2593258025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a075b98d0 con 0x564a076b89f0 2024-05-27T07:43:44.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.606+0000 7f7693e00700 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 0x564a0775de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:44.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.606+0000 7f7693400700 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a0775e370 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f768001a040 tx=0x7f768000fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed69d353c7d39d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:44.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.606+0000 7f767e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:44.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.607+0000 7f767e000700 1 -- 172.21.15.63:0/2593258025 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680026020 con 0x564a076b89f0 2024-05-27T07:43:44.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.607+0000 7f767e000700 1 -- 172.21.15.63:0/2593258025 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f7680004c60 con 0x564a076b89f0 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.607+0000 7f76a97c1b80 1 -- 172.21.15.63:0/2593258025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76800044c0 con 0x564a076b89f0 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.607+0000 7f767e000700 1 -- 172.21.15.63:0/2593258025 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7680031050 con 0x564a076b89f0 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.607+0000 7f76a97c1b80 10 client.17358 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.607+0000 7f76a97c1b80 1 -- 172.21.15.63:0/2593258025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a0763a060 con 0x564a076b89f0 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.608+0000 7f767e000700 1 -- 172.21.15.63:0/2593258025 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76800044c0 con 0x564a076b89f0 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.608+0000 7f767e000700 1 -- 172.21.15.63:0/2593258025 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7680003e30 con 0x564a076b89f0 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.608+0000 7f76a97c1b80 10 client.17358 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.608+0000 7f76a97c1b80 10 client.17358 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.608+0000 7f76a97c1b80 20 client.17358 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.608+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564a0776e180 0x564a077705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.608+0000 7f76a97c1b80 4 client.17358 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:44.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.608+0000 7f76a97c1b80 1 -- 172.21.15.63:0/2593258025 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a0767bd90 con 0x564a0776e180 2024-05-27T07:43:44.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.609+0000 7f7693e00700 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564a0776e180 0x564a077705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.610+0000 7f7693e00700 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564a0776e180 0x564a077705a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=474d403a24338e80 server_cookie=7fc5a9d0950ad4b9 in_seq=0 out_seq=0 2024-05-27T07:43:44.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.610+0000 7f767e000700 10 client.17358 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:43:44.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f767e000700 1 -- 172.21.15.63:0/2593258025 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564a0767bd90 con 0x564a0776e180 2024-05-27T07:43:44.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f767e000700 10 client.17358 handle_command_reply: tid=0 2024-05-27T07:43:44.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f76a97c1b80 1 client.17358 shutdown 2024-05-27T07:43:44.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f76a97c1b80 1 -- 172.21.15.63:0/2593258025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 msgr2=0x564a0775e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:44.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a0775e370 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f768001a040 tx=0x7f768000fed0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f76a97c1b80 1 -- 172.21.15.63:0/2593258025 shutdown_connections 2024-05-27T07:43:44.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a0775e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 0x564a0775de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564a0776e180 0x564a077705a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f767e000700 0 client.17358 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:43:44.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/2593258025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a0775e370 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.612+0000 7f76a97c1b80 1 -- 172.21.15.63:0/2593258025 >> 172.21.15.63:0/2593258025 conn(0x564a07666390 msgr2=0x564a077470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:44.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.613+0000 7f76a97c1b80 1 -- 172.21.15.63:0/2593258025 shutdown_connections 2024-05-27T07:43:44.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.613+0000 7f76a97c1b80 1 -- 172.21.15.63:0/2593258025 wait complete. 2024-05-27T07:43:44.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.613+0000 7f76a97c1b80 20 client.17358 trim_cache size 0 max 16384 2024-05-27T07:43:44.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.637+0000 7f76a97c1b80 1 Processor -- start 2024-05-27T07:43:44.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.638+0000 7f76a97c1b80 1 -- start start 2024-05-27T07:43:44.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.638+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a077595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.638+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a076b89f0 0x564a077474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.638+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0774bdb0 0x564a0776a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.638+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76941b0c10 con 0x564a075cce40 2024-05-27T07:43:44.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.638+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76941ae4a0 con 0x564a0774bdb0 2024-05-27T07:43:44.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.638+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f76941ac310 con 0x564a076b89f0 2024-05-27T07:43:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.638+0000 7f7693e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a077595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.639+0000 7f7693e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a077595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37612/0 (socket says 172.21.15.63:37612) 2024-05-27T07:43:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.639+0000 7f7693e00700 1 -- 172.21.15.63:0/3184677914 learned_addr learned my addr 172.21.15.63:0/3184677914 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.639+0000 7f7693400700 1 --2- 172.21.15.63:0/3184677914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a076b89f0 0x564a077474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.639+0000 7f7698c00700 1 --2- 172.21.15.63:0/3184677914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0774bdb0 0x564a0776a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.639+0000 7f7693e00700 1 -- 172.21.15.63:0/3184677914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a076b89f0 msgr2=0x564a077474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.639+0000 7f7693e00700 1 --2- 172.21.15.63:0/3184677914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a076b89f0 0x564a077474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.639+0000 7f7693e00700 1 -- 172.21.15.63:0/3184677914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0774bdb0 msgr2=0x564a0776a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.639+0000 7f7693e00700 1 --2- 172.21.15.63:0/3184677914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0774bdb0 0x564a0776a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.639+0000 7f7693e00700 1 -- 172.21.15.63:0/3184677914 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0774c1c0 con 0x564a075cce40 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.639+0000 7f7693e00700 1 --2- 172.21.15.63:0/3184677914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a077595f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f76840a53a0 tx=0x7f7684005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e11fe6866646acc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.640+0000 7f7688c00700 1 -- 172.21.15.63:0/3184677914 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76840c2020 con 0x564a075cce40 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.640+0000 7f7688c00700 1 -- 172.21.15.63:0/3184677914 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76840b2180 con 0x564a075cce40 2024-05-27T07:43:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.640+0000 7f7688c00700 1 -- 172.21.15.63:0/3184677914 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684020040 con 0x564a075cce40 2024-05-27T07:43:44.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.641+0000 7f76a97c1b80 1 -- 172.21.15.63:0/3184677914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 msgr2=0x564a077595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:44.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.641+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/3184677914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a077595f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f76840a53a0 tx=0x7f7684005910 comp rx=0 tx=0).stop 2024-05-27T07:43:44.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.641+0000 7f76a97c1b80 1 -- 172.21.15.63:0/3184677914 shutdown_connections 2024-05-27T07:43:44.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.641+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/3184677914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a0774bdb0 0x564a0776a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.641+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/3184677914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a075cce40 0x564a077595f0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.641+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/3184677914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a076b89f0 0x564a077474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.641+0000 7f76a97c1b80 1 -- 172.21.15.63:0/3184677914 >> 172.21.15.63:0/3184677914 conn(0x564a07666390 msgr2=0x564a0774d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:44.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.641+0000 7f76a97c1b80 1 -- 172.21.15.63:0/3184677914 shutdown_connections 2024-05-27T07:43:44.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.641+0000 7f76a97c1b80 1 -- 172.21.15.63:0/3184677914 wait complete. 2024-05-27T07:43:44.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.642+0000 7f76a97c1b80 1 -- start start 2024-05-27T07:43:44.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.642+0000 7f76a97c1b80 1 Processor -- start 2024-05-27T07:43:44.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.643+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 0x564a07758200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.643+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a07758740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.643+0000 7f76a97c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a07765670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.643+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76941ac960 con 0x564a076b89f0 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.643+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76941addd0 con 0x564a075cce40 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.643+0000 7f76a97c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f76941ad030 con 0x564a0774bdb0 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.643+0000 7f76a97c1b80 10 client.0 _ll_register_callbacks cb 0x564a07613c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.643+0000 7f7693400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a07758740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.643+0000 7f7693400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a07758740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37628/0 (socket says 172.21.15.63:37628) 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.643+0000 7f7693400700 1 -- 172.21.15.63:0/1635833618 learned_addr learned my addr 172.21.15.63:0/1635833618 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.644+0000 7f7693e00700 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 0x564a07758200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.644+0000 7f7698c00700 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a07765670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.644+0000 7f7693400700 1 -- 172.21.15.63:0/1635833618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 msgr2=0x564a07765670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.644+0000 7f7693400700 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a07765670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.644+0000 7f7693400700 1 -- 172.21.15.63:0/1635833618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 msgr2=0x564a07758200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:44.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.644+0000 7f7693400700 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 0x564a07758200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.644+0000 7f7693400700 1 -- 172.21.15.63:0/1635833618 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0775ec20 con 0x564a076b89f0 2024-05-27T07:43:44.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.644+0000 7f7698c00700 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a07765670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:44.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.644+0000 7f7693400700 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a07758740 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f76800041d0 tx=0x7f7680022690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9d1ce42935c73b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:44.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.644+0000 7f767e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:44.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.645+0000 7f767e000700 1 -- 172.21.15.63:0/1635833618 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680026070 con 0x564a076b89f0 2024-05-27T07:43:44.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.645+0000 7f767e000700 1 -- 172.21.15.63:0/1635833618 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f7680004650 con 0x564a076b89f0 2024-05-27T07:43:44.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.645+0000 7f76a97c1b80 1 -- 172.21.15.63:0/1635833618 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76840b2180 con 0x564a076b89f0 2024-05-27T07:43:44.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.645+0000 7f767e000700 1 -- 172.21.15.63:0/1635833618 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7680043050 con 0x564a076b89f0 2024-05-27T07:43:44.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.645+0000 7f76a97c1b80 10 client.17364 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:44.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.645+0000 7f76a97c1b80 1 -- 172.21.15.63:0/1635833618 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a07766bd0 con 0x564a076b89f0 2024-05-27T07:43:44.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.646+0000 7f767e000700 1 -- 172.21.15.63:0/1635833618 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76840b2180 con 0x564a076b89f0 2024-05-27T07:43:44.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.646+0000 7f767e000700 1 -- 172.21.15.63:0/1635833618 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7680010430 con 0x564a076b89f0 2024-05-27T07:43:44.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.646+0000 7f76a97c1b80 10 client.17364 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:44.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.646+0000 7f76a97c1b80 10 client.17364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:43:44.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.646+0000 7f76a97c1b80 20 client.17364 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:44.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.646+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564a077d8ad0 0x564a077daef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:44.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.646+0000 7f76a97c1b80 4 client.17364 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:43:44.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.646+0000 7f76a97c1b80 1 -- 172.21.15.63:0/1635833618 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a0774bb40 con 0x564a077d8ad0 2024-05-27T07:43:44.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.647+0000 7f7693e00700 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564a077d8ad0 0x564a077daef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:44.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.648+0000 7f7693e00700 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564a077d8ad0 0x564a077daef0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41cd6d7aa4689f01 server_cookie=736326c69782877b in_seq=0 out_seq=0 2024-05-27T07:43:44.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.648+0000 7f767e000700 10 client.17364 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.648+0000 7f767e000700 1 -- 172.21.15.63:0/1635833618 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a0774bb40 con 0x564a077d8ad0 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.648+0000 7f767e000700 10 client.17364 handle_command_reply: tid=0 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.648+0000 7f76a97c1b80 1 client.17364 shutdown 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.648+0000 7f76a97c1b80 1 -- 172.21.15.63:0/1635833618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 msgr2=0x564a07758740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.648+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a07758740 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f76800041d0 tx=0x7f7680022690 comp rx=0 tx=0).stop 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.649+0000 7f76a97c1b80 1 -- 172.21.15.63:0/1635833618 shutdown_connections 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.649+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a0774bdb0 0x564a07765670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.649+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a075cce40 0x564a07758200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.649+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564a077d8ad0 0x564a077daef0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.649+0000 7f76a97c1b80 1 --2- 172.21.15.63:0/1635833618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a076b89f0 0x564a07758740 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.649+0000 7f76a97c1b80 1 -- 172.21.15.63:0/1635833618 >> 172.21.15.63:0/1635833618 conn(0x564a07666390 msgr2=0x564a0774d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.649+0000 7f767e000700 0 client.17364 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:43:44.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.649+0000 7f76a97c1b80 1 -- 172.21.15.63:0/1635833618 shutdown_connections 2024-05-27T07:43:44.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.649+0000 7f76a97c1b80 1 -- 172.21.15.63:0/1635833618 wait complete. 2024-05-27T07:43:44.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.649+0000 7f76a97c1b80 20 client.17364 trim_cache size 0 max 16384 2024-05-27T07:43:44.649 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:43:44.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 -- 172.21.15.63:0/4254678133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76780b2a70 msgr2=0x7f76780b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:44.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76780b2a70 0x7f76780b4f80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f768c00b830 tx=0x7f768c009270 comp rx=0 tx=0).stop 2024-05-27T07:43:44.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 -- 172.21.15.63:0/4254678133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7694075af0 msgr2=0x7f769419cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:44.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7694075af0 0x7f769419cf60 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f768400d950 tx=0x7f768400dc60 comp rx=0 tx=0).stop 2024-05-27T07:43:44.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 -- 172.21.15.63:0/4254678133 shutdown_connections 2024-05-27T07:43:44.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76780b2a70 0x7f76780b4f80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7694110bf0 0x7f76941a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7694075af0 0x7f769419cf60 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 --2- 172.21.15.63:0/4254678133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7694076440 0x7f769419d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:44.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 -- 172.21.15.63:0/4254678133 >> 172.21.15.63:0/4254678133 conn(0x7f7694100030 msgr2=0x7f76941006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:44.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 -- 172.21.15.63:0/4254678133 shutdown_connections 2024-05-27T07:43:44.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:44.650+0000 7f769ac00700 1 -- 172.21.15.63:0/4254678133 wait complete. 2024-05-27T07:43:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.537+0000 7f18a6800640 1 -- 172.21.15.63:0/2630542682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a010bce0 msgr2=0x7f18a0105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.537+0000 7f18a6800640 1 --2- 172.21.15.63:0/2630542682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a010bce0 0x7f18a0105880 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1894009900 tx=0x7f189402a410 comp rx=0 tx=0).stop 2024-05-27T07:43:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.538+0000 7f18a6800640 1 -- 172.21.15.63:0/2630542682 shutdown_connections 2024-05-27T07:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.538+0000 7f18a6800640 1 --2- 172.21.15.63:0/2630542682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18a0106430 0x7f18a0112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.538+0000 7f18a6800640 1 --2- 172.21.15.63:0/2630542682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a010c180 0x7f18a0105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.538+0000 7f18a6800640 1 --2- 172.21.15.63:0/2630542682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a010bce0 0x7f18a0105880 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.538+0000 7f18a6800640 1 -- 172.21.15.63:0/2630542682 >> 172.21.15.63:0/2630542682 conn(0x7f18a0101380 msgr2=0x7f18a01037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.538+0000 7f18a6800640 1 -- 172.21.15.63:0/2630542682 shutdown_connections 2024-05-27T07:43:45.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.538+0000 7f18a6800640 1 -- 172.21.15.63:0/2630542682 wait complete. 2024-05-27T07:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.539+0000 7f18a6800640 1 Processor -- start 2024-05-27T07:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.539+0000 7f18a6800640 1 -- start start 2024-05-27T07:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a0106430 0x7f18a01a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a010bce0 0x7f18a01a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18a010c180 0x7f18a019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a6800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0115de0 con 0x7f18a0106430 2024-05-27T07:43:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0115c60 con 0x7f18a010c180 2024-05-27T07:43:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a6800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0115f60 con 0x7f18a010bce0 2024-05-27T07:43:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a0106430 0x7f18a01a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a0106430 0x7f18a01a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37646/0 (socket says 172.21.15.63:37646) 2024-05-27T07:43:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a5400640 1 -- 172.21.15.63:0/2629257889 learned_addr learned my addr 172.21.15.63:0/2629257889 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a4a00640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a010bce0 0x7f18a01a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.540+0000 7f18a5e00640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18a010c180 0x7f18a019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.541+0000 7f18a5400640 1 -- 172.21.15.63:0/2629257889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a010bce0 msgr2=0x7f18a01a2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.541+0000 7f18a5400640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a010bce0 0x7f18a01a2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.541+0000 7f18a5400640 1 -- 172.21.15.63:0/2629257889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18a010c180 msgr2=0x7f18a019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.541+0000 7f18a5400640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18a010c180 0x7f18a019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.541+0000 7f18a5400640 1 -- 172.21.15.63:0/2629257889 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a019cdf0 con 0x7f18a0106430 2024-05-27T07:43:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.541+0000 7f18a5e00640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18a010c180 0x7f18a019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:43:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.541+0000 7f18a5400640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a0106430 0x7f18a01a2540 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f189402a930 tx=0x7f1894003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.541+0000 7f189e000640 1 -- 172.21.15.63:0/2629257889 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189403f070 con 0x7f18a0106430 2024-05-27T07:43:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.541+0000 7f189e000640 1 -- 172.21.15.63:0/2629257889 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1894003e10 con 0x7f18a0106430 2024-05-27T07:43:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.542+0000 7f18a6800640 1 -- 172.21.15.63:0/2629257889 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a019d020 con 0x7f18a0106430 2024-05-27T07:43:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.542+0000 7f189e000640 1 -- 172.21.15.63:0/2629257889 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894035030 con 0x7f18a0106430 2024-05-27T07:43:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.542+0000 7f18a6800640 1 -- 172.21.15.63:0/2629257889 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a019d510 con 0x7f18a0106430 2024-05-27T07:43:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.546+0000 7f18aee51740 1 Processor -- start 2024-05-27T07:43:45.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.546+0000 7f18aee51740 1 -- start start 2024-05-27T07:43:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.546+0000 7f18aee51740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x56243898fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.547+0000 7f18aee51740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243898ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.547+0000 7f18aee51740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.547+0000 7f18aee51740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562438900b90 con 0x5624385cc4d0 2024-05-27T07:43:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.547+0000 7f18aee51740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5624388f0730 con 0x5624385ed1b0 2024-05-27T07:43:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.547+0000 7f18aee51740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5624388f4770 con 0x5624385c42f0 2024-05-27T07:43:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.547+0000 7f189e000640 1 -- 172.21.15.63:0/2629257889 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1894042070 con 0x7f18a0106430 2024-05-27T07:43:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.547+0000 7f18a4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.547+0000 7f18a4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389904e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37660/0 (socket says 172.21.15.63:37660) 2024-05-27T07:43:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.547+0000 7f18a4a00640 1 -- 172.21.15.63:0/2711585209 learned_addr learned my addr 172.21.15.63:0/2711585209 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.547+0000 7f18a5e00640 1 --2- 172.21.15.63:0/2711585209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x56243898fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.548+0000 7f18a5400640 1 --2- 172.21.15.63:0/2711585209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243898ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.548+0000 7f18a4a00640 1 -- 172.21.15.63:0/2711585209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 msgr2=0x56243898ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.548+0000 7f18a4a00640 1 --2- 172.21.15.63:0/2711585209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243898ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.548+0000 7f18a4a00640 1 -- 172.21.15.63:0/2711585209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 msgr2=0x56243898fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.548+0000 7f18a4a00640 1 --2- 172.21.15.63:0/2711585209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x56243898fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.548+0000 7f18a4a00640 1 -- 172.21.15.63:0/2711585209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624388898b0 con 0x5624385cc4d0 2024-05-27T07:43:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.549+0000 7f18a4a00640 1 --2- 172.21.15.63:0/2711585209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389904e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f188c00bb60 tx=0x7f188c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc303ad3140e4b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.549+0000 7f1892600640 1 -- 172.21.15.63:0/2711585209 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c015070 con 0x5624385cc4d0 2024-05-27T07:43:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.549+0000 7f1892600640 1 -- 172.21.15.63:0/2711585209 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f188c00be50 con 0x5624385cc4d0 2024-05-27T07:43:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.549+0000 7f1892600640 1 -- 172.21.15.63:0/2711585209 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c011440 con 0x5624385cc4d0 2024-05-27T07:43:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.549+0000 7f189e000640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18880b2950 0x7f18880b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.550+0000 7f189e000640 1 -- 172.21.15.63:0/2629257889 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f18941032b0 con 0x7f18a0106430 2024-05-27T07:43:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.550+0000 7f18a5e00640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18880b2950 0x7f18880b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.551+0000 7f18aee51740 1 -- 172.21.15.63:0/2711585209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 msgr2=0x5624389904e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.551+0000 7f18aee51740 1 --2- 172.21.15.63:0/2711585209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389904e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f188c00bb60 tx=0x7f188c00f730 comp rx=0 tx=0).stop 2024-05-27T07:43:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.551+0000 7f18a5e00640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18880b2950 0x7f18880b4e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x56243898cbd0 tx=0x7f1898009700 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.551+0000 7f18aee51740 1 -- 172.21.15.63:0/2711585209 shutdown_connections 2024-05-27T07:43:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.551+0000 7f18aee51740 1 --2- 172.21.15.63:0/2711585209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389904e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.551+0000 7f18aee51740 1 --2- 172.21.15.63:0/2711585209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243898ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.551+0000 7f18aee51740 1 --2- 172.21.15.63:0/2711585209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x56243898fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.551+0000 7f18aee51740 1 -- 172.21.15.63:0/2711585209 >> 172.21.15.63:0/2711585209 conn(0x5624385e1300 msgr2=0x56243898c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.551+0000 7f18aee51740 1 -- 172.21.15.63:0/2711585209 shutdown_connections 2024-05-27T07:43:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.551+0000 7f18aee51740 1 -- 172.21.15.63:0/2711585209 wait complete. 2024-05-27T07:43:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.552+0000 7f18aee51740 1 -- start start 2024-05-27T07:43:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.553+0000 7f18aee51740 1 Processor -- start 2024-05-27T07:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.553+0000 7f18aee51740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385c42f0 0x562438999b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.553+0000 7f18aee51740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385cc4d0 0x56243899a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.553+0000 7f18aee51740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x5624389a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.553+0000 7f18aee51740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5624385d6790 con 0x5624385c42f0 2024-05-27T07:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.553+0000 7f18aee51740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562438898bc0 con 0x5624385ed1b0 2024-05-27T07:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.553+0000 7f18aee51740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56243899cb90 con 0x5624385cc4d0 2024-05-27T07:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.553+0000 7f18aee51740 10 client.0 _ll_register_callbacks cb 0x562438876400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.554+0000 7f18a5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385c42f0 0x562438999b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.554+0000 7f18a5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385c42f0 0x562438999b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37674/0 (socket says 172.21.15.63:37674) 2024-05-27T07:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.554+0000 7f18a5400640 1 -- 172.21.15.63:0/3040936401 learned_addr learned my addr 172.21.15.63:0/3040936401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.554+0000 7f18a4a00640 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385cc4d0 0x56243899a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.554+0000 7f18a5e00640 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x5624389a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.554+0000 7f18a5400640 1 -- 172.21.15.63:0/3040936401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385cc4d0 msgr2=0x56243899a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.554+0000 7f18a5400640 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385cc4d0 0x56243899a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.554+0000 7f18a5400640 1 -- 172.21.15.63:0/3040936401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 msgr2=0x5624389a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.554+0000 7f18a5400640 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x5624389a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.554+0000 7f18a5400640 1 -- 172.21.15.63:0/3040936401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562438605210 con 0x5624385c42f0 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.555+0000 7f18a5400640 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385c42f0 0x562438999b90 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f18940c7440 tx=0x7f18940c9b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b020844e8d1d2c0b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.555+0000 7f1886a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.555+0000 7f1886a00640 1 -- 172.21.15.63:0/3040936401 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894049050 con 0x5624385c42f0 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.555+0000 7f1886a00640 1 -- 172.21.15.63:0/3040936401 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f1894031400 con 0x5624385c42f0 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.555+0000 7f18aee51740 1 -- 172.21.15.63:0/3040936401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f188c00be50 con 0x5624385c42f0 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f1886a00640 1 -- 172.21.15.63:0/3040936401 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f18940ca8d0 con 0x5624385c42f0 2024-05-27T07:43:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f18aee51740 10 client.17376 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f18aee51740 1 -- 172.21.15.63:0/3040936401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5624386d1620 con 0x5624385c42f0 2024-05-27T07:43:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f1886a00640 1 -- 172.21.15.63:0/3040936401 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f188c00be50 con 0x5624385c42f0 2024-05-27T07:43:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f1886a00640 1 -- 172.21.15.63:0/3040936401 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f18940ce120 con 0x5624385c42f0 2024-05-27T07:43:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f18aee51740 10 client.17376 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f18aee51740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f18aee51740 20 client.17376 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f18aee51740 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624385abd80 0x5624389abe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f18aee51740 4 client.17376 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.556+0000 7f18aee51740 1 -- 172.21.15.63:0/3040936401 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562438902ae0 con 0x5624385abd80 2024-05-27T07:43:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.557+0000 7f18a4a00640 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624385abd80 0x5624389abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.558+0000 7f18a4a00640 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624385abd80 0x5624389abe00 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b0537d1db52c02 server_cookie=fbae028bcb9e34c4 in_seq=0 out_seq=0 2024-05-27T07:43:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.558+0000 7f1886a00640 10 client.17376 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.560+0000 7f1886a00640 1 -- 172.21.15.63:0/3040936401 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562438902ae0 con 0x5624385abd80 2024-05-27T07:43:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.560+0000 7f1886a00640 10 client.17376 handle_command_reply: tid=0 2024-05-27T07:43:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.560+0000 7f18aee51740 1 client.17376 shutdown 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 1 -- 172.21.15.63:0/3040936401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385c42f0 msgr2=0x562438999b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385c42f0 0x562438999b90 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f18940c7440 tx=0x7f18940c9b20 comp rx=0 tx=0).stop 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 1 -- 172.21.15.63:0/3040936401 shutdown_connections 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624385abd80 0x5624389abe00 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x5624389a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385cc4d0 0x56243899a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 1 --2- 172.21.15.63:0/3040936401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385c42f0 0x562438999b90 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 1 -- 172.21.15.63:0/3040936401 >> 172.21.15.63:0/3040936401 conn(0x5624385e1300 msgr2=0x562438991780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f1886a00640 0 client.17376 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 1 -- 172.21.15.63:0/3040936401 shutdown_connections 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 1 -- 172.21.15.63:0/3040936401 wait complete. 2024-05-27T07:43:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.561+0000 7f18aee51740 20 client.17376 trim_cache size 0 max 16384 2024-05-27T07:43:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.580+0000 7f18aee51740 1 Processor -- start 2024-05-27T07:43:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.580+0000 7f18aee51740 1 -- start start 2024-05-27T07:43:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.581+0000 7f18aee51740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243899bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.581+0000 7f18aee51740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.581+0000 7f18aee51740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x5624389a6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.581+0000 7f18aee51740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01b5ae0 con 0x5624385cc4d0 2024-05-27T07:43:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.581+0000 7f18aee51740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01b33b0 con 0x5624385ed1b0 2024-05-27T07:43:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.581+0000 7f18aee51740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01b1220 con 0x5624385c42f0 2024-05-27T07:43:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.581+0000 7f18a4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.581+0000 7f18a4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389a5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37686/0 (socket says 172.21.15.63:37686) 2024-05-27T07:43:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.581+0000 7f18a4a00640 1 -- 172.21.15.63:0/1113637412 learned_addr learned my addr 172.21.15.63:0/1113637412 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.582+0000 7f18a5e00640 1 --2- 172.21.15.63:0/1113637412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x5624389a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.582+0000 7f18a5400640 1 --2- 172.21.15.63:0/1113637412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243899bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.582+0000 7f18a4a00640 1 -- 172.21.15.63:0/1113637412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 msgr2=0x56243899bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.582+0000 7f18a4a00640 1 --2- 172.21.15.63:0/1113637412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243899bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.582+0000 7f18a4a00640 1 -- 172.21.15.63:0/1113637412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 msgr2=0x5624389a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.582+0000 7f18a4a00640 1 --2- 172.21.15.63:0/1113637412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x5624389a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.582+0000 7f18a4a00640 1 -- 172.21.15.63:0/1113637412 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624385abfb0 con 0x5624385cc4d0 2024-05-27T07:43:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.582+0000 7f18a4a00640 1 --2- 172.21.15.63:0/1113637412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389a5c30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f188c00bb60 tx=0x7f188c003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b064afec8323a1fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.583+0000 7f1891c00640 1 -- 172.21.15.63:0/1113637412 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c02b070 con 0x5624385cc4d0 2024-05-27T07:43:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.583+0000 7f1891c00640 1 -- 172.21.15.63:0/1113637412 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f188c00b670 con 0x5624385cc4d0 2024-05-27T07:43:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.583+0000 7f1891c00640 1 -- 172.21.15.63:0/1113637412 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c015040 con 0x5624385cc4d0 2024-05-27T07:43:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.584+0000 7f18aee51740 1 -- 172.21.15.63:0/1113637412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 msgr2=0x5624389a5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.584+0000 7f18aee51740 1 --2- 172.21.15.63:0/1113637412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389a5c30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f188c00bb60 tx=0x7f188c003960 comp rx=0 tx=0).stop 2024-05-27T07:43:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.584+0000 7f18aee51740 1 -- 172.21.15.63:0/1113637412 shutdown_connections 2024-05-27T07:43:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.584+0000 7f18aee51740 1 --2- 172.21.15.63:0/1113637412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385ed1b0 0x5624389a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.584+0000 7f18aee51740 1 --2- 172.21.15.63:0/1113637412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385cc4d0 0x5624389a5c30 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.584+0000 7f18aee51740 1 --2- 172.21.15.63:0/1113637412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243899bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.584+0000 7f18aee51740 1 -- 172.21.15.63:0/1113637412 >> 172.21.15.63:0/1113637412 conn(0x5624385e1300 msgr2=0x562438999e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.584+0000 7f18aee51740 1 -- 172.21.15.63:0/1113637412 shutdown_connections 2024-05-27T07:43:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.584+0000 7f18aee51740 1 -- 172.21.15.63:0/1113637412 wait complete. 2024-05-27T07:43:45.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.585+0000 7f18aee51740 1 -- start start 2024-05-27T07:43:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.585+0000 7f18aee51740 1 Processor -- start 2024-05-27T07:43:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.586+0000 7f18aee51740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243899ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.586+0000 7f18aee51740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385cc4d0 0x56243899f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.586+0000 7f18aee51740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385ed1b0 0x56243899f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.586+0000 7f18aee51740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01b1870 con 0x5624385ed1b0 2024-05-27T07:43:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.586+0000 7f18aee51740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01b2ce0 con 0x5624385cc4d0 2024-05-27T07:43:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.586+0000 7f18aee51740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01b1f40 con 0x5624385c42f0 2024-05-27T07:43:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.586+0000 7f18aee51740 10 client.0 _ll_register_callbacks cb 0x7f1894005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.586+0000 7f18a5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385ed1b0 0x56243899f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.586+0000 7f18a5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385ed1b0 0x56243899f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37694/0 (socket says 172.21.15.63:37694) 2024-05-27T07:43:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.586+0000 7f18a5e00640 1 -- 172.21.15.63:0/1731390488 learned_addr learned my addr 172.21.15.63:0/1731390488 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.587+0000 7f18a5400640 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243899ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.587+0000 7f18a4a00640 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385cc4d0 0x56243899f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.587+0000 7f18a5e00640 1 -- 172.21.15.63:0/1731390488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 msgr2=0x56243899ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.587+0000 7f18a5e00640 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243899ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.587+0000 7f18a5e00640 1 -- 172.21.15.63:0/1731390488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385cc4d0 msgr2=0x56243899f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.587+0000 7f18a5e00640 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385cc4d0 0x56243899f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.587+0000 7f18a5e00640 1 -- 172.21.15.63:0/1731390488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562438a125a0 con 0x5624385ed1b0 2024-05-27T07:43:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.587+0000 7f18a5400640 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243899ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.587+0000 7f18a5e00640 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385ed1b0 0x56243899f760 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1898012ea0 tx=0x7f1898004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405bd5ac98e451fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.587+0000 7f1886a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.588+0000 7f1886a00640 1 -- 172.21.15.63:0/1731390488 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898020d90 con 0x5624385ed1b0 2024-05-27T07:43:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.588+0000 7f1886a00640 1 -- 172.21.15.63:0/1731390488 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f18980283e0 con 0x5624385ed1b0 2024-05-27T07:43:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.588+0000 7f18aee51740 1 -- 172.21.15.63:0/1731390488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f188c00b670 con 0x5624385ed1b0 2024-05-27T07:43:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.588+0000 7f1886a00640 1 -- 172.21.15.63:0/1731390488 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1898010e70 con 0x5624385ed1b0 2024-05-27T07:43:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.588+0000 7f18aee51740 10 client.17382 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.588+0000 7f18aee51740 1 -- 172.21.15.63:0/1731390488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562438a13870 con 0x5624385ed1b0 2024-05-27T07:43:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.589+0000 7f1886a00640 1 -- 172.21.15.63:0/1731390488 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f188c00b670 con 0x5624385ed1b0 2024-05-27T07:43:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.589+0000 7f1886a00640 1 -- 172.21.15.63:0/1731390488 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1898027070 con 0x5624385ed1b0 2024-05-27T07:43:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.589+0000 7f18aee51740 10 client.17382 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.589+0000 7f18aee51740 10 client.17382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.589+0000 7f18aee51740 20 client.17382 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.589+0000 7f18aee51740 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624385811c0 0x562438a160e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.589+0000 7f18aee51740 4 client.17382 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:43:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.589+0000 7f18aee51740 1 -- 172.21.15.63:0/1731390488 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624388fc2c0 con 0x5624385811c0 2024-05-27T07:43:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.590+0000 7f18a5400640 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624385811c0 0x562438a160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.591+0000 7f18a5400640 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624385811c0 0x562438a160e0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3086377fb532fe6d server_cookie=dc23032a0cc0da6f in_seq=0 out_seq=0 2024-05-27T07:43:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.591+0000 7f1886a00640 10 client.17382 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:45.597 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:43:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f1886a00640 1 -- 172.21.15.63:0/1731390488 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5624388fc2c0 con 0x5624385811c0 2024-05-27T07:43:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f1886a00640 10 client.17382 handle_command_reply: tid=0 2024-05-27T07:43:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 client.17382 shutdown 2024-05-27T07:43:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 -- 172.21.15.63:0/1731390488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385ed1b0 msgr2=0x56243899f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385ed1b0 0x56243899f760 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1898012ea0 tx=0x7f1898004a40 comp rx=0 tx=0).stop 2024-05-27T07:43:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 -- 172.21.15.63:0/1731390488 shutdown_connections 2024-05-27T07:43:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624385811c0 0x562438a160e0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5624385ed1b0 0x56243899f760 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5624385cc4d0 0x56243899f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 --2- 172.21.15.63:0/1731390488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5624385c42f0 0x56243899ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f1886a00640 0 client.17382 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 -- 172.21.15.63:0/1731390488 >> 172.21.15.63:0/1731390488 conn(0x5624385e1300 msgr2=0x56243898a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 -- 172.21.15.63:0/1731390488 shutdown_connections 2024-05-27T07:43:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.632+0000 7f18aee51740 1 -- 172.21.15.63:0/1731390488 wait complete. 2024-05-27T07:43:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.633+0000 7f18aee51740 20 client.17382 trim_cache size 0 max 16384 2024-05-27T07:43:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:43:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:43:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:43:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.633+0000 7f18a6800640 1 -- 172.21.15.63:0/2629257889 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18880b2950 msgr2=0x7f18880b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.633+0000 7f18a6800640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18880b2950 0x7f18880b4e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x56243898cbd0 tx=0x7f1898009700 comp rx=0 tx=0).stop 2024-05-27T07:43:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.633+0000 7f18a6800640 1 -- 172.21.15.63:0/2629257889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a0106430 msgr2=0x7f18a01a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.633+0000 7f18a6800640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a0106430 0x7f18a01a2540 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f189402a930 tx=0x7f1894003c20 comp rx=0 tx=0).stop 2024-05-27T07:43:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.634+0000 7f18a6800640 1 -- 172.21.15.63:0/2629257889 shutdown_connections 2024-05-27T07:43:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.634+0000 7f18a6800640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18880b2950 0x7f18880b4e60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.634+0000 7f18a6800640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18a010c180 0x7f18a019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.634+0000 7f18a6800640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18a010bce0 0x7f18a01a2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.634+0000 7f18a6800640 1 --2- 172.21.15.63:0/2629257889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a0106430 0x7f18a01a2540 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.634+0000 7f18a6800640 1 -- 172.21.15.63:0/2629257889 >> 172.21.15.63:0/2629257889 conn(0x7f18a0101380 msgr2=0x7f18a0103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.634+0000 7f18a6800640 1 -- 172.21.15.63:0/2629257889 shutdown_connections 2024-05-27T07:43:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:45.634+0000 7f18a6800640 1 -- 172.21.15.63:0/2629257889 wait complete. 2024-05-27T07:43:45.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:45 smithi063 ceph-mon[31613]: pgmap v316: 129 pgs: 129 active+clean; 2.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:43:45.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:45 smithi086 ceph-mon[34687]: pgmap v316: 129 pgs: 129 active+clean; 2.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:43:45.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:45 smithi149 ceph-mon[35408]: pgmap v316: 129 pgs: 129 active+clean; 2.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:43:46.111 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:47.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:47 smithi086 ceph-mon[34687]: pgmap v317: 129 pgs: 129 active+clean; 2.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:43:47.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:47 smithi149 ceph-mon[35408]: pgmap v317: 129 pgs: 129 active+clean; 2.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:43:47.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:47 smithi063 ceph-mon[31613]: pgmap v317: 129 pgs: 129 active+clean; 2.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:43:48.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.454+0000 7fc282c00700 1 -- 172.21.15.63:0/4064174543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc27c105e20 msgr2=0x7fc27c10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.454+0000 7fc282c00700 1 --2- 172.21.15.63:0/4064174543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc27c105e20 0x7fc27c10e370 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc268009990 tx=0x7fc268009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.454+0000 7fc282c00700 1 -- 172.21.15.63:0/4064174543 shutdown_connections 2024-05-27T07:43:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.454+0000 7fc282c00700 1 --2- 172.21.15.63:0/4064174543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc27c10e900 0x7fc27c110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.454+0000 7fc282c00700 1 --2- 172.21.15.63:0/4064174543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c1054d0 0x7fc27c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.454+0000 7fc282c00700 1 --2- 172.21.15.63:0/4064174543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc27c105e20 0x7fc27c10e370 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.454+0000 7fc282c00700 1 -- 172.21.15.63:0/4064174543 >> 172.21.15.63:0/4064174543 conn(0x7fc27c0fedb0 msgr2=0x7fc27c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.454+0000 7fc282c00700 1 -- 172.21.15.63:0/4064174543 shutdown_connections 2024-05-27T07:43:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.455+0000 7fc282c00700 1 -- 172.21.15.63:0/4064174543 wait complete. 2024-05-27T07:43:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.456+0000 7fc282c00700 1 Processor -- start 2024-05-27T07:43:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.456+0000 7fc282c00700 1 -- start start 2024-05-27T07:43:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.457+0000 7fc282c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc27c1054d0 0x7fc27c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.457+0000 7fc282c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc27c105e20 0x7fc27c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.457+0000 7fc282c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c10e900 0x7fc27c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.457+0000 7fc282c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1142e0 con 0x7fc27c1054d0 2024-05-27T07:43:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.457+0000 7fc282c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c114160 con 0x7fc27c10e900 2024-05-27T07:43:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.457+0000 7fc282c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c114460 con 0x7fc27c105e20 2024-05-27T07:43:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.457+0000 7fc27be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc27c1054d0 0x7fc27c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.457+0000 7fc27be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc27c1054d0 0x7fc27c19cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37704/0 (socket says 172.21.15.63:37704) 2024-05-27T07:43:48.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.457+0000 7fc27be00700 1 -- 172.21.15.63:0/4194256891 learned_addr learned my addr 172.21.15.63:0/4194256891 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:48.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.458+0000 7fc280c00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c10e900 0x7fc27c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.458+0000 7fc27b400700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc27c105e20 0x7fc27c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.458+0000 7fc27be00700 1 -- 172.21.15.63:0/4194256891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc27c105e20 msgr2=0x7fc27c19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.458+0000 7fc27be00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc27c105e20 0x7fc27c19d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.458+0000 7fc27be00700 1 -- 172.21.15.63:0/4194256891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c10e900 msgr2=0x7fc27c1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.458+0000 7fc27be00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c10e900 0x7fc27c1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.458+0000 7fc27be00700 1 -- 172.21.15.63:0/4194256891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c1a1e00 con 0x7fc27c1054d0 2024-05-27T07:43:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.458+0000 7fc27be00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc27c1054d0 0x7fc27c19cf30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00d950 tx=0x7fc26c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.459+0000 7fc278c00700 1 -- 172.21.15.63:0/4194256891 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c016070 con 0x7fc27c1054d0 2024-05-27T07:43:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.459+0000 7fc278c00700 1 -- 172.21.15.63:0/4194256891 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc26c010460 con 0x7fc27c1054d0 2024-05-27T07:43:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.459+0000 7fc282c00700 1 -- 172.21.15.63:0/4194256891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27c1a20f0 con 0x7fc27c1054d0 2024-05-27T07:43:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.459+0000 7fc278c00700 1 -- 172.21.15.63:0/4194256891 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00f620 con 0x7fc27c1054d0 2024-05-27T07:43:48.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.459+0000 7fc282c00700 1 -- 172.21.15.63:0/4194256891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27c1a9a10 con 0x7fc27c1054d0 2024-05-27T07:43:48.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.464+0000 7fc278c00700 1 -- 172.21.15.63:0/4194256891 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc26c019070 con 0x7fc27c1054d0 2024-05-27T07:43:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.465+0000 7fc291715b80 1 Processor -- start 2024-05-27T07:43:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.465+0000 7fc291715b80 1 -- start start 2024-05-27T07:43:48.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.465+0000 7fc291715b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a139a9f0 0x55b1a142d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc291715b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a142d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc291715b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a142de70 0x55b1a143a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc291715b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b1a135be40 con 0x55b1a142de70 2024-05-27T07:43:48.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc291715b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b1a134b9e0 con 0x55b1a12aee40 2024-05-27T07:43:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc291715b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b1a134fa20 con 0x55b1a139a9f0 2024-05-27T07:43:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc278c00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2600b29f0 0x7fc2600b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc27b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a142de70 0x55b1a143a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc27b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a142de70 0x55b1a143a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37712/0 (socket says 172.21.15.63:37712) 2024-05-27T07:43:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc27b400700 1 -- 172.21.15.63:0/978753504 learned_addr learned my addr 172.21.15.63:0/978753504 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc278c00700 1 -- 172.21.15.63:0/4194256891 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fc26c0e2620 con 0x7fc27c1054d0 2024-05-27T07:43:48.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc280c00700 1 --2- 172.21.15.63:0/978753504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a139a9f0 0x55b1a142d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.466+0000 7fc27be00700 1 --2- 172.21.15.63:0/978753504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a142d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.467+0000 7fc280c00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2600b29f0 0x7fc2600b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.467+0000 7fc27b400700 1 -- 172.21.15.63:0/978753504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a139a9f0 msgr2=0x55b1a142d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:48.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.467+0000 7fc27b400700 1 --2- 172.21.15.63:0/978753504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a139a9f0 0x55b1a142d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.467+0000 7fc27b400700 1 -- 172.21.15.63:0/978753504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 msgr2=0x55b1a142d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.467+0000 7fc27b400700 1 --2- 172.21.15.63:0/978753504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a142d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.467+0000 7fc27b400700 1 -- 172.21.15.63:0/978753504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a1314420 con 0x55b1a142de70 2024-05-27T07:43:48.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.467+0000 7fc27b400700 1 --2- 172.21.15.63:0/978753504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a142de70 0x55b1a143a110 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc268006010 tx=0x7fc2680096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa71813e79ff65f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:48.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.468+0000 7fc280c00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2600b29f0 0x7fc2600b4f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55b1a142fff0 tx=0x7fc27400b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:48.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.468+0000 7fc271600700 1 -- 172.21.15.63:0/978753504 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268027070 con 0x55b1a142de70 2024-05-27T07:43:48.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.468+0000 7fc271600700 1 -- 172.21.15.63:0/978753504 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc268004ce0 con 0x55b1a142de70 2024-05-27T07:43:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.468+0000 7fc271600700 1 -- 172.21.15.63:0/978753504 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26800ba30 con 0x55b1a142de70 2024-05-27T07:43:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.469+0000 7fc291715b80 1 -- 172.21.15.63:0/978753504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a142de70 msgr2=0x55b1a143a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.469+0000 7fc291715b80 1 --2- 172.21.15.63:0/978753504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a142de70 0x55b1a143a110 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc268006010 tx=0x7fc2680096a0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.469+0000 7fc291715b80 1 -- 172.21.15.63:0/978753504 shutdown_connections 2024-05-27T07:43:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.469+0000 7fc291715b80 1 --2- 172.21.15.63:0/978753504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a142de70 0x55b1a143a110 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.469+0000 7fc291715b80 1 --2- 172.21.15.63:0/978753504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a139a9f0 0x55b1a142d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.469+0000 7fc291715b80 1 --2- 172.21.15.63:0/978753504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a142d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.469+0000 7fc291715b80 1 -- 172.21.15.63:0/978753504 >> 172.21.15.63:0/978753504 conn(0x55b1a1348390 msgr2=0x55b1a14294c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:48.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.469+0000 7fc291715b80 1 -- 172.21.15.63:0/978753504 shutdown_connections 2024-05-27T07:43:48.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.470+0000 7fc291715b80 1 -- 172.21.15.63:0/978753504 wait complete. 2024-05-27T07:43:48.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.470+0000 7fc291715b80 1 -- start start 2024-05-27T07:43:48.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.471+0000 7fc291715b80 1 Processor -- start 2024-05-27T07:43:48.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.471+0000 7fc291715b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a14376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.471+0000 7fc291715b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1437c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.471+0000 7fc291715b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 0x55b1a1438140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.471+0000 7fc291715b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b1a1251fa0 con 0x55b1a139a9f0 2024-05-27T07:43:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.471+0000 7fc291715b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b1a12ec760 con 0x55b1a12aee40 2024-05-27T07:43:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.471+0000 7fc291715b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b1a143ab20 con 0x55b1a142de70 2024-05-27T07:43:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.471+0000 7fc291715b80 10 client.0 _ll_register_callbacks cb 0x55b1a12f5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc27b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1437c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc27b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1437c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37714/0 (socket says 172.21.15.63:37714) 2024-05-27T07:43:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc27b400700 1 -- 172.21.15.63:0/2637597615 learned_addr learned my addr 172.21.15.63:0/2637597615 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc27be00700 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a14376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc280c00700 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 0x55b1a1438140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc27b400700 1 -- 172.21.15.63:0/2637597615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 msgr2=0x55b1a1438140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc27b400700 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 0x55b1a1438140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc27b400700 1 -- 172.21.15.63:0/2637597615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 msgr2=0x55b1a14376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:48.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc27b400700 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a14376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc27b400700 1 -- 172.21.15.63:0/2637597615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a129b8d0 con 0x55b1a139a9f0 2024-05-27T07:43:48.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.472+0000 7fc27be00700 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a14376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:48.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.473+0000 7fc27b400700 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1437c00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc2680098e0 tx=0x7fc268004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7aa84a7d947fa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:48.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.473+0000 7fc266000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:48.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.473+0000 7fc266000700 1 -- 172.21.15.63:0/2637597615 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268027040 con 0x55b1a139a9f0 2024-05-27T07:43:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.473+0000 7fc266000700 1 -- 172.21.15.63:0/2637597615 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fc2680050b0 con 0x55b1a139a9f0 2024-05-27T07:43:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.473+0000 7fc291715b80 1 -- 172.21.15.63:0/2637597615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc268004ce0 con 0x55b1a139a9f0 2024-05-27T07:43:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.474+0000 7fc266000700 1 -- 172.21.15.63:0/2637597615 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc268038050 con 0x55b1a139a9f0 2024-05-27T07:43:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.474+0000 7fc291715b80 10 client.17394 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.474+0000 7fc291715b80 1 -- 172.21.15.63:0/2637597615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b1a131c060 con 0x55b1a139a9f0 2024-05-27T07:43:48.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.474+0000 7fc266000700 1 -- 172.21.15.63:0/2637597615 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc268004ce0 con 0x55b1a139a9f0 2024-05-27T07:43:48.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.475+0000 7fc266000700 1 -- 172.21.15.63:0/2637597615 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc268033030 con 0x55b1a139a9f0 2024-05-27T07:43:48.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.475+0000 7fc291715b80 10 client.17394 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:48.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.475+0000 7fc291715b80 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:48.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.475+0000 7fc291715b80 20 client.17394 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:48.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.475+0000 7fc291715b80 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a1447970 0x55b1a1449d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.475+0000 7fc291715b80 4 client.17394 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:48.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.475+0000 7fc291715b80 1 -- 172.21.15.63:0/2637597615 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1a135dd90 con 0x55b1a1447970 2024-05-27T07:43:48.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.476+0000 7fc27be00700 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a1447970 0x55b1a1449d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.477+0000 7fc266000700 10 client.17394 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:48.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.477+0000 7fc27be00700 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a1447970 0x55b1a1449d90 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d65941d6a28137dc server_cookie=acc06c88e72c38fe in_seq=0 out_seq=0 2024-05-27T07:43:48.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.479+0000 7fc266000700 1 -- 172.21.15.63:0/2637597615 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b1a135dd90 con 0x55b1a1447970 2024-05-27T07:43:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.479+0000 7fc266000700 10 client.17394 handle_command_reply: tid=0 2024-05-27T07:43:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.479+0000 7fc291715b80 1 client.17394 shutdown 2024-05-27T07:43:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 1 -- 172.21.15.63:0/2637597615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 msgr2=0x55b1a1437c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1437c00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc2680098e0 tx=0x7fc268004e90 comp rx=0 tx=0).stop 2024-05-27T07:43:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 1 -- 172.21.15.63:0/2637597615 shutdown_connections 2024-05-27T07:43:48.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a1447970 0x55b1a1449d90 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 0x55b1a1438140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a14376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 1 --2- 172.21.15.63:0/2637597615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1437c00 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 1 -- 172.21.15.63:0/2637597615 >> 172.21.15.63:0/2637597615 conn(0x55b1a1348390 msgr2=0x55b1a142d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:48.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc266000700 0 client.17394 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:48.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 1 -- 172.21.15.63:0/2637597615 shutdown_connections 2024-05-27T07:43:48.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 1 -- 172.21.15.63:0/2637597615 wait complete. 2024-05-27T07:43:48.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.480+0000 7fc291715b80 20 client.17394 trim_cache size 0 max 16384 2024-05-27T07:43:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.503+0000 7fc291715b80 1 Processor -- start 2024-05-27T07:43:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.504+0000 7fc291715b80 1 -- start start 2024-05-27T07:43:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.504+0000 7fc291715b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a12aee40 0x55b1a1432ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.504+0000 7fc291715b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1436620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.504+0000 7fc291715b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a142de70 0x55b1a1443dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.504+0000 7fc291715b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b4fc0 con 0x55b1a139a9f0 2024-05-27T07:43:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.504+0000 7fc291715b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b2850 con 0x55b1a142de70 2024-05-27T07:43:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.504+0000 7fc291715b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b06c0 con 0x55b1a12aee40 2024-05-27T07:43:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.504+0000 7fc27b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1436620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.504+0000 7fc27b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1436620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37726/0 (socket says 172.21.15.63:37726) 2024-05-27T07:43:48.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.504+0000 7fc27b400700 1 -- 172.21.15.63:0/3035873904 learned_addr learned my addr 172.21.15.63:0/3035873904 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.505+0000 7fc27be00700 1 --2- 172.21.15.63:0/3035873904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a12aee40 0x55b1a1432ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.505+0000 7fc280c00700 1 --2- 172.21.15.63:0/3035873904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a142de70 0x55b1a1443dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.505+0000 7fc27b400700 1 -- 172.21.15.63:0/3035873904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a12aee40 msgr2=0x55b1a1432ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.505+0000 7fc27b400700 1 --2- 172.21.15.63:0/3035873904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a12aee40 0x55b1a1432ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.505+0000 7fc27b400700 1 -- 172.21.15.63:0/3035873904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a142de70 msgr2=0x55b1a1443dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.505+0000 7fc27b400700 1 --2- 172.21.15.63:0/3035873904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a142de70 0x55b1a1443dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.505+0000 7fc27b400700 1 -- 172.21.15.63:0/3035873904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a14384b0 con 0x55b1a139a9f0 2024-05-27T07:43:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.505+0000 7fc27b400700 1 --2- 172.21.15.63:0/3035873904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1436620 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc268022800 tx=0x7fc268009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15bdb3de5c12f1a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.506+0000 7fc270c00700 1 -- 172.21.15.63:0/3035873904 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268027070 con 0x55b1a139a9f0 2024-05-27T07:43:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.506+0000 7fc270c00700 1 -- 172.21.15.63:0/3035873904 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc268022e50 con 0x55b1a139a9f0 2024-05-27T07:43:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.506+0000 7fc270c00700 1 -- 172.21.15.63:0/3035873904 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26800bac0 con 0x55b1a139a9f0 2024-05-27T07:43:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.507+0000 7fc291715b80 1 -- 172.21.15.63:0/3035873904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 msgr2=0x55b1a1436620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.507+0000 7fc291715b80 1 --2- 172.21.15.63:0/3035873904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1436620 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc268022800 tx=0x7fc268009ee0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.507+0000 7fc291715b80 1 -- 172.21.15.63:0/3035873904 shutdown_connections 2024-05-27T07:43:48.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.507+0000 7fc291715b80 1 --2- 172.21.15.63:0/3035873904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a142de70 0x55b1a1443dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.507+0000 7fc291715b80 1 --2- 172.21.15.63:0/3035873904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a12aee40 0x55b1a1432ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.507+0000 7fc291715b80 1 --2- 172.21.15.63:0/3035873904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1436620 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.507+0000 7fc291715b80 1 -- 172.21.15.63:0/3035873904 >> 172.21.15.63:0/3035873904 conn(0x55b1a1348390 msgr2=0x55b1a142eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.507+0000 7fc291715b80 1 -- 172.21.15.63:0/3035873904 shutdown_connections 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.507+0000 7fc291715b80 1 -- 172.21.15.63:0/3035873904 wait complete. 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.508+0000 7fc291715b80 1 -- start start 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.508+0000 7fc291715b80 1 Processor -- start 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.509+0000 7fc291715b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a144d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.509+0000 7fc291715b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1448590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.509+0000 7fc291715b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 0x55b1a1448ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.509+0000 7fc291715b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b0d10 con 0x55b1a139a9f0 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.509+0000 7fc291715b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b2180 con 0x55b1a12aee40 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.509+0000 7fc291715b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b13e0 con 0x55b1a142de70 2024-05-27T07:43:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.509+0000 7fc291715b80 10 client.0 _ll_register_callbacks cb 0x55b1a12f5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.509+0000 7fc27b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1448590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.509+0000 7fc27b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1448590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37736/0 (socket says 172.21.15.63:37736) 2024-05-27T07:43:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.509+0000 7fc27b400700 1 -- 172.21.15.63:0/2380694808 learned_addr learned my addr 172.21.15.63:0/2380694808 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc27be00700 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a144d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc280c00700 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 0x55b1a1448ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc27b400700 1 -- 172.21.15.63:0/2380694808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 msgr2=0x55b1a1448ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc27b400700 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 0x55b1a1448ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc27b400700 1 -- 172.21.15.63:0/2380694808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 msgr2=0x55b1a144d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc27b400700 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a144d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc27b400700 1 -- 172.21.15.63:0/2380694808 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a1449010 con 0x55b1a139a9f0 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc27be00700 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a144d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc280c00700 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 0x55b1a1448ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc27b400700 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1448590 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc2680229c0 tx=0x7fc268004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5f5fb3018caeaf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.510+0000 7fc266000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.511+0000 7fc266000700 1 -- 172.21.15.63:0/2380694808 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268027070 con 0x55b1a139a9f0 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.511+0000 7fc266000700 1 -- 172.21.15.63:0/2380694808 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fc2680051c0 con 0x55b1a139a9f0 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.511+0000 7fc291715b80 1 -- 172.21.15.63:0/2380694808 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc268022e50 con 0x55b1a139a9f0 2024-05-27T07:43:48.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.511+0000 7fc266000700 1 -- 172.21.15.63:0/2380694808 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc268049050 con 0x55b1a139a9f0 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.511+0000 7fc291715b80 10 client.17400 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.511+0000 7fc291715b80 1 -- 172.21.15.63:0/2380694808 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b1a14ba650 con 0x55b1a139a9f0 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.512+0000 7fc266000700 1 -- 172.21.15.63:0/2380694808 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc268022e50 con 0x55b1a139a9f0 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.512+0000 7fc266000700 1 -- 172.21.15.63:0/2380694808 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc268033030 con 0x55b1a139a9f0 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.512+0000 7fc291715b80 10 client.17400 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.512+0000 7fc291715b80 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.512+0000 7fc291715b80 20 client.17400 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.512+0000 7fc291715b80 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a14baca0 0x55b1a14bd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.512+0000 7fc291715b80 4 client.17400 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.512+0000 7fc291715b80 1 -- 172.21.15.63:0/2380694808 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1a1357570 con 0x55b1a14baca0 2024-05-27T07:43:48.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.513+0000 7fc27be00700 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a14baca0 0x55b1a14bd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.514+0000 7fc27be00700 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a14baca0 0x55b1a14bd0c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc7537f4f5b22da server_cookie=b35df54cedb2146f in_seq=0 out_seq=0 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.514+0000 7fc266000700 10 client.17400 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.514+0000 7fc266000700 1 -- 172.21.15.63:0/2380694808 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1a1357570 con 0x55b1a14baca0 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.514+0000 7fc266000700 10 client.17400 handle_command_reply: tid=0 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.514+0000 7fc291715b80 1 client.17400 shutdown 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.514+0000 7fc291715b80 1 -- 172.21.15.63:0/2380694808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 msgr2=0x55b1a1448590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.514+0000 7fc291715b80 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1448590 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc2680229c0 tx=0x7fc268004e90 comp rx=0 tx=0).stop 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.515+0000 7fc291715b80 1 -- 172.21.15.63:0/2380694808 shutdown_connections 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.515+0000 7fc291715b80 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a14baca0 0x55b1a14bd0c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.515+0000 7fc291715b80 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a142de70 0x55b1a1448ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.515+0000 7fc291715b80 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a12aee40 0x55b1a144d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.515+0000 7fc291715b80 1 --2- 172.21.15.63:0/2380694808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a139a9f0 0x55b1a1448590 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.515+0000 7fc291715b80 1 -- 172.21.15.63:0/2380694808 >> 172.21.15.63:0/2380694808 conn(0x55b1a1348390 msgr2=0x55b1a142eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:48.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.515+0000 7fc266000700 0 client.17400 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:48.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.515+0000 7fc291715b80 1 -- 172.21.15.63:0/2380694808 shutdown_connections 2024-05-27T07:43:48.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.515+0000 7fc291715b80 1 -- 172.21.15.63:0/2380694808 wait complete. 2024-05-27T07:43:48.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.515+0000 7fc291715b80 20 client.17400 trim_cache size 0 max 16384 2024-05-27T07:43:48.516 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.516+0000 7fc282c00700 1 -- 172.21.15.63:0/4194256891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2600b29f0 msgr2=0x7fc2600b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.516+0000 7fc282c00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2600b29f0 0x7fc2600b4f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55b1a142fff0 tx=0x7fc27400b040 comp rx=0 tx=0).stop 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.516+0000 7fc282c00700 1 -- 172.21.15.63:0/4194256891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc27c1054d0 msgr2=0x7fc27c19cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.516+0000 7fc282c00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc27c1054d0 0x7fc27c19cf30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00d950 tx=0x7fc26c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.516+0000 7fc282c00700 1 -- 172.21.15.63:0/4194256891 shutdown_connections 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.516+0000 7fc282c00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2600b29f0 0x7fc2600b4f00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.516+0000 7fc282c00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c10e900 0x7fc27c1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.516+0000 7fc282c00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc27c1054d0 0x7fc27c19cf30 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.516+0000 7fc282c00700 1 --2- 172.21.15.63:0/4194256891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc27c105e20 0x7fc27c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.516+0000 7fc282c00700 1 -- 172.21.15.63:0/4194256891 >> 172.21.15.63:0/4194256891 conn(0x7fc27c0fedb0 msgr2=0x7fc27c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.517+0000 7fc282c00700 1 -- 172.21.15.63:0/4194256891 shutdown_connections 2024-05-27T07:43:48.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:48.517+0000 7fc282c00700 1 -- 172.21.15.63:0/4194256891 wait complete. 2024-05-27T07:43:49.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:49 smithi086 ceph-mon[34687]: pgmap v318: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.0 MiB/s wr, 18 op/s 2024-05-27T07:43:49.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:49 smithi149 ceph-mon[35408]: pgmap v318: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.0 MiB/s wr, 18 op/s 2024-05-27T07:43:49.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:49 smithi063 ceph-mon[31613]: pgmap v318: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.0 MiB/s wr, 18 op/s 2024-05-27T07:43:50.541 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:43:50.541 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:43:51.062 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:51.553 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:51 smithi063 ceph-mon[31613]: pgmap v319: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T07:43:51.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:51 smithi086 ceph-mon[34687]: pgmap v319: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T07:43:51.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:51 smithi149 ceph-mon[35408]: pgmap v319: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T07:43:53.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.564+0000 7f363ba00700 1 -- 172.21.15.63:0/3094484336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3634108730 msgr2=0x7f3634110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.564+0000 7f363ba00700 1 --2- 172.21.15.63:0/3094484336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3634108730 0x7f3634110090 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f362c0099c0 tx=0x7f362c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.564+0000 7f363ba00700 1 -- 172.21.15.63:0/3094484336 shutdown_connections 2024-05-27T07:43:53.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.564+0000 7f363ba00700 1 --2- 172.21.15.63:0/3094484336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3634108730 0x7f3634110090 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.564+0000 7f363ba00700 1 --2- 172.21.15.63:0/3094484336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36341034b0 0x7f36341038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.564+0000 7f363ba00700 1 --2- 172.21.15.63:0/3094484336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3634107de0 0x7f36341081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.564+0000 7f363ba00700 1 -- 172.21.15.63:0/3094484336 >> 172.21.15.63:0/3094484336 conn(0x7f36340fedb0 msgr2=0x7f36341011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:53.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.565+0000 7f363ba00700 1 -- 172.21.15.63:0/3094484336 shutdown_connections 2024-05-27T07:43:53.565 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:53 smithi063 ceph-mon[31613]: pgmap v320: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:43:53.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.565+0000 7f363ba00700 1 -- 172.21.15.63:0/3094484336 wait complete. 2024-05-27T07:43:53.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.567+0000 7f363ba00700 1 Processor -- start 2024-05-27T07:43:53.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.567+0000 7f363ba00700 1 -- start start 2024-05-27T07:43:53.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.567+0000 7f363ba00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36341034b0 0x7f363419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.567+0000 7f363ba00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3634107de0 0x7f363419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.567+0000 7f363ba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3634108730 0x7f36341a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.567+0000 7f363ba00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3634113260 con 0x7f3634107de0 2024-05-27T07:43:53.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.567+0000 7f363ba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36341130e0 con 0x7f3634108730 2024-05-27T07:43:53.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.567+0000 7f363ba00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f36341133e0 con 0x7f36341034b0 2024-05-27T07:43:53.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.568+0000 7f3633e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3634107de0 0x7f363419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.568+0000 7f3633e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3634107de0 0x7f363419d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35046/0 (socket says 172.21.15.63:35046) 2024-05-27T07:43:53.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.568+0000 7f3633e00700 1 -- 172.21.15.63:0/3095653955 learned_addr learned my addr 172.21.15.63:0/3095653955 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:53.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.568+0000 7f3639000700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36341034b0 0x7f363419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.568+0000 7f3639a00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3634108730 0x7f36341a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.569+0000 7f3633e00700 1 -- 172.21.15.63:0/3095653955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36341034b0 msgr2=0x7f363419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.569+0000 7f3633e00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36341034b0 0x7f363419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.569+0000 7f3633e00700 1 -- 172.21.15.63:0/3095653955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3634108730 msgr2=0x7f36341a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.569+0000 7f3633e00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3634108730 0x7f36341a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.569+0000 7f3633e00700 1 -- 172.21.15.63:0/3095653955 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36341a1dc0 con 0x7f3634107de0 2024-05-27T07:43:53.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.569+0000 7f3639000700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36341034b0 0x7f363419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:53.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.569+0000 7f3633e00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3634107de0 0x7f363419d490 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f362000d980 tx=0x7f362000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:53.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.570+0000 7f3631600700 1 -- 172.21.15.63:0/3095653955 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3620017070 con 0x7f3634107de0 2024-05-27T07:43:53.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.570+0000 7f3631600700 1 -- 172.21.15.63:0/3095653955 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3620010460 con 0x7f3634107de0 2024-05-27T07:43:53.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.570+0000 7f3631600700 1 -- 172.21.15.63:0/3095653955 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362000de60 con 0x7f3634107de0 2024-05-27T07:43:53.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.570+0000 7f363ba00700 1 -- 172.21.15.63:0/3095653955 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36341a20b0 con 0x7f3634107de0 2024-05-27T07:43:53.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.570+0000 7f363ba00700 1 -- 172.21.15.63:0/3095653955 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36341a99d0 con 0x7f3634107de0 2024-05-27T07:43:53.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.575+0000 7f3631600700 1 -- 172.21.15.63:0/3095653955 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3620010610 con 0x7f3634107de0 2024-05-27T07:43:53.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.575+0000 7f364a4e0b80 1 Processor -- start 2024-05-27T07:43:53.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.576+0000 7f364a4e0b80 1 -- start start 2024-05-27T07:43:53.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.576+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bca5a70 0x55926bd383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.576+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd388f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.576+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bd38ef0 0x55926bd45190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.576+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55926bc66ec0 con 0x55926bca5a70 2024-05-27T07:43:53.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.576+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55926bc56a60 con 0x55926bbb9ec0 2024-05-27T07:43:53.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.576+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55926bc5aaa0 con 0x55926bd38ef0 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.577+0000 7f3639a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bca5a70 0x55926bd383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.577+0000 7f3639a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bca5a70 0x55926bd383b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35058/0 (socket says 172.21.15.63:35058) 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.577+0000 7f3639a00700 1 -- 172.21.15.63:0/2349298063 learned_addr learned my addr 172.21.15.63:0/2349298063 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.577+0000 7f3631600700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f361c0b29f0 0x7f361c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.577+0000 7f3631600700 1 -- 172.21.15.63:0/3095653955 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f36200e2520 con 0x7f3634107de0 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.577+0000 7f3633e00700 1 --2- 172.21.15.63:0/2349298063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bd38ef0 0x55926bd45190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.577+0000 7f3639a00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f361c0b29f0 0x7f361c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.577+0000 7f3639a00700 1 -- 172.21.15.63:0/2349298063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bd38ef0 msgr2=0x55926bd45190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.577+0000 7f3639a00700 1 --2- 172.21.15.63:0/2349298063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bd38ef0 0x55926bd45190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.578+0000 7f3639a00700 1 -- 172.21.15.63:0/2349298063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 msgr2=0x55926bd388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:53.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.578+0000 7f3639a00700 1 --2- 172.21.15.63:0/2349298063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.578+0000 7f3639a00700 1 -- 172.21.15.63:0/2349298063 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926bc1f4a0 con 0x55926bca5a70 2024-05-27T07:43:53.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.578+0000 7f3639a00700 1 --2- 172.21.15.63:0/2349298063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bca5a70 0x55926bd383b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f362c0058e0 tx=0x7f362c0054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d454bdd4bdca2fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:53.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.579+0000 7f3639a00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f361c0b29f0 0x7f361c0b4f00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f362c025000 tx=0x7f362c027000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:53.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.579+0000 7f3626000700 1 -- 172.21.15.63:0/2349298063 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c021050 con 0x55926bca5a70 2024-05-27T07:43:53.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.579+0000 7f3626000700 1 -- 172.21.15.63:0/2349298063 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f362c02a020 con 0x55926bca5a70 2024-05-27T07:43:53.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.579+0000 7f3626000700 1 -- 172.21.15.63:0/2349298063 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c00f6c0 con 0x55926bca5a70 2024-05-27T07:43:53.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.580+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2349298063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bca5a70 msgr2=0x55926bd383b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.580+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2349298063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bca5a70 0x55926bd383b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f362c0058e0 tx=0x7f362c0054f0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.580+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2349298063 shutdown_connections 2024-05-27T07:43:53.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.580+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2349298063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bd38ef0 0x55926bd45190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.580+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2349298063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bca5a70 0x55926bd383b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.580+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2349298063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.580+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2349298063 >> 172.21.15.63:0/2349298063 conn(0x55926bc53410 msgr2=0x55926bd34540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:53.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.580+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2349298063 shutdown_connections 2024-05-27T07:43:53.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.581+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2349298063 wait complete. 2024-05-27T07:43:53.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.581+0000 7f364a4e0b80 1 -- start start 2024-05-27T07:43:53.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.582+0000 7f364a4e0b80 1 Processor -- start 2024-05-27T07:43:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd51370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bca5a70 0x55926bd518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd4b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55926bb5d020 con 0x55926bd38ef0 2024-05-27T07:43:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55926bbf77e0 con 0x55926bbb9ec0 2024-05-27T07:43:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55926bd45ba0 con 0x55926bca5a70 2024-05-27T07:43:53.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f364a4e0b80 10 client.0 _ll_register_callbacks cb 0x55926bc00cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f3639a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd4b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f3639000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd51370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f3639a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd4b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35064/0 (socket says 172.21.15.63:35064) 2024-05-27T07:43:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f3639a00700 1 -- 172.21.15.63:0/2773798413 learned_addr learned my addr 172.21.15.63:0/2773798413 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.583+0000 7f3633e00700 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bca5a70 0x55926bd518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.584+0000 7f3639a00700 1 -- 172.21.15.63:0/2773798413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bca5a70 msgr2=0x55926bd518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.584+0000 7f3639a00700 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bca5a70 0x55926bd518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.584+0000 7f3639a00700 1 -- 172.21.15.63:0/2773798413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 msgr2=0x55926bd51370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.584+0000 7f3639a00700 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd51370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.584+0000 7f3639a00700 1 -- 172.21.15.63:0/2773798413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926bba6950 con 0x55926bd38ef0 2024-05-27T07:43:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.584+0000 7f3639a00700 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd4b770 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f362c009f90 tx=0x7f362c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f59b12c95bff31 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.584+0000 7f361aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.585+0000 7f361aa00700 1 -- 172.21.15.63:0/2773798413 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c00fdf0 con 0x55926bd38ef0 2024-05-27T07:43:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.585+0000 7f361aa00700 1 -- 172.21.15.63:0/2773798413 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f362c038440 con 0x55926bd38ef0 2024-05-27T07:43:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.585+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2773798413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f362c02a020 con 0x55926bd38ef0 2024-05-27T07:43:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.585+0000 7f361aa00700 1 -- 172.21.15.63:0/2773798413 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f362c021020 con 0x55926bd38ef0 2024-05-27T07:43:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.585+0000 7f364a4e0b80 10 client.17412 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.585+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2773798413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55926bc270e0 con 0x55926bd38ef0 2024-05-27T07:43:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.586+0000 7f361aa00700 1 -- 172.21.15.63:0/2773798413 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f362c02a020 con 0x55926bd38ef0 2024-05-27T07:43:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.586+0000 7f361aa00700 1 -- 172.21.15.63:0/2773798413 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f362c040030 con 0x55926bd38ef0 2024-05-27T07:43:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.586+0000 7f364a4e0b80 10 client.17412 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.586+0000 7f364a4e0b80 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.586+0000 7f364a4e0b80 20 client.17412 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.586+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55926bd5b290 0x55926bd5d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.586+0000 7f364a4e0b80 4 client.17412 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.586+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2773798413 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55926bc68e10 con 0x55926bd5b290 2024-05-27T07:43:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.587+0000 7f3639000700 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55926bd5b290 0x55926bd5d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.588+0000 7f3639000700 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55926bd5b290 0x55926bd5d6b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb31bb128f44110 server_cookie=afe6382ca74dc3fb in_seq=0 out_seq=0 2024-05-27T07:43:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.588+0000 7f361aa00700 10 client.17412 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f361aa00700 1 -- 172.21.15.63:0/2773798413 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55926bc68e10 con 0x55926bd5b290 2024-05-27T07:43:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f361aa00700 10 client.17412 handle_command_reply: tid=0 2024-05-27T07:43:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f364a4e0b80 1 client.17412 shutdown 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2773798413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 msgr2=0x55926bd4b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd4b770 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f362c009f90 tx=0x7f362c00f740 comp rx=0 tx=0).stop 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2773798413 shutdown_connections 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55926bd5b290 0x55926bd5d6b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd4b770 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd51370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2773798413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bca5a70 0x55926bd518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2773798413 >> 172.21.15.63:0/2773798413 conn(0x55926bc53410 msgr2=0x55926bd38390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f361aa00700 0 client.17412 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.590+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2773798413 shutdown_connections 2024-05-27T07:43:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.591+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2773798413 wait complete. 2024-05-27T07:43:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.591+0000 7f364a4e0b80 20 client.17412 trim_cache size 0 max 16384 2024-05-27T07:43:53.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.625+0000 7f364a4e0b80 1 Processor -- start 2024-05-27T07:43:53.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.625+0000 7f364a4e0b80 1 -- start start 2024-05-27T07:43:53.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.625+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.625+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bca5a70 0x55926bd3a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.625+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd489d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.625+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36341b0bd0 con 0x55926bd38ef0 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.625+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36341ae460 con 0x55926bbb9ec0 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.625+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f36341ac2d0 con 0x55926bca5a70 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3639a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3639a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35066/0 (socket says 172.21.15.63:35066) 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3639a00700 1 -- 172.21.15.63:0/6192537 learned_addr learned my addr 172.21.15.63:0/6192537 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3639000700 1 --2- 172.21.15.63:0/6192537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3633e00700 1 --2- 172.21.15.63:0/6192537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bca5a70 0x55926bd3a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3639a00700 1 -- 172.21.15.63:0/6192537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bca5a70 msgr2=0x55926bd3a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3639a00700 1 --2- 172.21.15.63:0/6192537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bca5a70 0x55926bd3a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3639a00700 1 -- 172.21.15.63:0/6192537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 msgr2=0x55926bd46f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3639a00700 1 --2- 172.21.15.63:0/6192537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd46f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3639a00700 1 -- 172.21.15.63:0/6192537 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926bd302c0 con 0x55926bd38ef0 2024-05-27T07:43:53.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.626+0000 7f3639000700 1 --2- 172.21.15.63:0/6192537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd46f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:53.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.627+0000 7f3639a00700 1 --2- 172.21.15.63:0/6192537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd489d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f362c02f420 tx=0x7f362c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7227c689da19bf3b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:53.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.627+0000 7f3625600700 1 -- 172.21.15.63:0/6192537 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c04e070 con 0x55926bd38ef0 2024-05-27T07:43:53.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.627+0000 7f3625600700 1 -- 172.21.15.63:0/6192537 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f362c02a040 con 0x55926bd38ef0 2024-05-27T07:43:53.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.627+0000 7f3625600700 1 -- 172.21.15.63:0/6192537 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c038440 con 0x55926bd38ef0 2024-05-27T07:43:53.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.628+0000 7f364a4e0b80 1 -- 172.21.15.63:0/6192537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 msgr2=0x55926bd489d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.628+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/6192537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd489d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f362c02f420 tx=0x7f362c0095f0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.628+0000 7f364a4e0b80 1 -- 172.21.15.63:0/6192537 shutdown_connections 2024-05-27T07:43:53.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.628+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/6192537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd489d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.628+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/6192537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bbb9ec0 0x55926bd46f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.628+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/6192537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bca5a70 0x55926bd3a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.628+0000 7f364a4e0b80 1 -- 172.21.15.63:0/6192537 >> 172.21.15.63:0/6192537 conn(0x55926bc53410 msgr2=0x55926bd39f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:53.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.629+0000 7f364a4e0b80 1 -- 172.21.15.63:0/6192537 shutdown_connections 2024-05-27T07:43:53.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.629+0000 7f364a4e0b80 1 -- 172.21.15.63:0/6192537 wait complete. 2024-05-27T07:43:53.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.629+0000 7f364a4e0b80 1 -- start start 2024-05-27T07:43:53.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.630+0000 7f364a4e0b80 1 Processor -- start 2024-05-27T07:43:53.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.630+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bbb9ec0 0x55926bd56450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.630+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bca5a70 0x55926bd56990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.630+0000 7f364a4e0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd56ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.630+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36341ac920 con 0x55926bd38ef0 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.630+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36341add90 con 0x55926bca5a70 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.630+0000 7f364a4e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f36341acff0 con 0x55926bbb9ec0 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.630+0000 7f364a4e0b80 10 client.0 _ll_register_callbacks cb 0x55926bc00cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.631+0000 7f3639a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd56ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.631+0000 7f3639a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd56ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35070/0 (socket says 172.21.15.63:35070) 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.631+0000 7f3639a00700 1 -- 172.21.15.63:0/2918508544 learned_addr learned my addr 172.21.15.63:0/2918508544 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.631+0000 7f3633e00700 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bca5a70 0x55926bd56990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.631+0000 7f3639000700 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bbb9ec0 0x55926bd56450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.631+0000 7f3639a00700 1 -- 172.21.15.63:0/2918508544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bbb9ec0 msgr2=0x55926bd56450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.631+0000 7f3639a00700 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bbb9ec0 0x55926bd56450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.631+0000 7f3639a00700 1 -- 172.21.15.63:0/2918508544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bca5a70 msgr2=0x55926bd56990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:53.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.631+0000 7f3639a00700 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bca5a70 0x55926bd56990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.631+0000 7f3639a00700 1 -- 172.21.15.63:0/2918508544 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926bdc5b10 con 0x55926bd38ef0 2024-05-27T07:43:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.632+0000 7f3639a00700 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd56ed0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f362c04c040 tx=0x7f362c003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459f6d5e350e052f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.632+0000 7f361aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.632+0000 7f361aa00700 1 -- 172.21.15.63:0/2918508544 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c04e040 con 0x55926bd38ef0 2024-05-27T07:43:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.632+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2918508544 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f362c02a040 con 0x55926bd38ef0 2024-05-27T07:43:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.632+0000 7f361aa00700 1 -- 172.21.15.63:0/2918508544 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f362c00fdf0 con 0x55926bd38ef0 2024-05-27T07:43:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.632+0000 7f361aa00700 1 -- 172.21.15.63:0/2918508544 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f362c02fab0 con 0x55926bd38ef0 2024-05-27T07:43:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.632+0000 7f364a4e0b80 10 client.17418 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.632+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2918508544 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55926bdc6d60 con 0x55926bd38ef0 2024-05-27T07:43:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.633+0000 7f361aa00700 1 -- 172.21.15.63:0/2918508544 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f362c02a040 con 0x55926bd38ef0 2024-05-27T07:43:53.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.633+0000 7f361aa00700 1 -- 172.21.15.63:0/2918508544 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f362c05a030 con 0x55926bd38ef0 2024-05-27T07:43:53.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.633+0000 7f364a4e0b80 10 client.17418 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:53.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.633+0000 7f364a4e0b80 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:53.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.633+0000 7f364a4e0b80 20 client.17418 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:53.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.633+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55926bdc71c0 0x55926bdc95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.633+0000 7f364a4e0b80 4 client.17418 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:43:53.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.633+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2918508544 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55926bc625f0 con 0x55926bdc71c0 2024-05-27T07:43:53.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.634+0000 7f3639000700 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55926bdc71c0 0x55926bdc95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.635+0000 7f3639000700 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55926bdc71c0 0x55926bdc95e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e5b1569424a4252 server_cookie=677ac9eabed30fac in_seq=0 out_seq=0 2024-05-27T07:43:53.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.635+0000 7f361aa00700 10 client.17418 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:53.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f361aa00700 1 -- 172.21.15.63:0/2918508544 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55926bc625f0 con 0x55926bdc71c0 2024-05-27T07:43:53.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f361aa00700 10 client.17418 handle_command_reply: tid=0 2024-05-27T07:43:53.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f364a4e0b80 1 client.17418 shutdown 2024-05-27T07:43:53.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2918508544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 msgr2=0x55926bd56ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd56ed0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f362c04c040 tx=0x7f362c003dd0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2918508544 shutdown_connections 2024-05-27T07:43:53.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55926bdc71c0 0x55926bdc95e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55926bd38ef0 0x55926bd56ed0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55926bbb9ec0 0x55926bd56450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f361aa00700 0 client.17418 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:53.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f364a4e0b80 1 --2- 172.21.15.63:0/2918508544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55926bca5a70 0x55926bd56990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.638+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2918508544 >> 172.21.15.63:0/2918508544 conn(0x55926bc53410 msgr2=0x55926bd45f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:53.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.639+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2918508544 shutdown_connections 2024-05-27T07:43:53.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.639+0000 7f364a4e0b80 1 -- 172.21.15.63:0/2918508544 wait complete. 2024-05-27T07:43:53.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.639+0000 7f364a4e0b80 20 client.17418 trim_cache size 0 max 16384 2024-05-27T07:43:53.638 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:43:53.639 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:43:53.639 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "01f6aa12-6052-4144-a2bc-123f22fc368a", 2024-05-27T07:43:53.639 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:43:53.639 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:43:53.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 -- 172.21.15.63:0/3095653955 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f361c0b29f0 msgr2=0x7f361c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f361c0b29f0 0x7f361c0b4f00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f362c025000 tx=0x7f362c027000 comp rx=0 tx=0).stop 2024-05-27T07:43:53.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 -- 172.21.15.63:0/3095653955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3634107de0 msgr2=0x7f363419d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3634107de0 0x7f363419d490 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f362000d980 tx=0x7f362000dd40 comp rx=0 tx=0).stop 2024-05-27T07:43:53.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 -- 172.21.15.63:0/3095653955 shutdown_connections 2024-05-27T07:43:53.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f361c0b29f0 0x7f361c0b4f00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3634108730 0x7f36341a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36341034b0 0x7f363419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 --2- 172.21.15.63:0/3095653955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3634107de0 0x7f363419d490 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 -- 172.21.15.63:0/3095653955 >> 172.21.15.63:0/3095653955 conn(0x7f36340fedb0 msgr2=0x7f363410fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:53.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 -- 172.21.15.63:0/3095653955 shutdown_connections 2024-05-27T07:43:53.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:53.640+0000 7f363ba00700 1 -- 172.21.15.63:0/3095653955 wait complete. 2024-05-27T07:43:53.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:53 smithi149 ceph-mon[35408]: pgmap v320: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:43:53.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:53 smithi086 ceph-mon[34687]: pgmap v320: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:43:54.719 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:43:54.795 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:43:54.795 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.795 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:43:54.795 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:43:54.795 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:43:54.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:43:54.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:43:54.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:43:54.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.796 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:54 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:43:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:43:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:43:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:43:54.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:43:54.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:43:54.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:43:54.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:54 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:43:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:43:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:43:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:43:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:43:54.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:43:54.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:43:54.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:54.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:54 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:55.221 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:55.630 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:55 smithi063 ceph-mon[31613]: pgmap v321: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:43:55.630 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:43:55.630 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:55.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:55 smithi086 ceph-mon[34687]: pgmap v321: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:43:55.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:43:55.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:55.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:55 smithi149 ceph-mon[35408]: pgmap v321: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:43:55.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:43:55.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:43:56.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:56 smithi063 ceph-mon[31613]: pgmap v322: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:43:56.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:56 smithi086 ceph-mon[34687]: pgmap v322: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:43:56.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:56 smithi149 ceph-mon[35408]: pgmap v322: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:43:57.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.815+0000 7fb30f200700 1 -- 172.21.15.63:0/681857689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3101032e0 msgr2=0x7fb3101036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.815+0000 7fb30f200700 1 --2- 172.21.15.63:0/681857689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3101032e0 0x7fb3101036f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fb3000099c0 tx=0x7fb300009cd0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.815+0000 7fb30f200700 1 -- 172.21.15.63:0/681857689 shutdown_connections 2024-05-27T07:43:57.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.815+0000 7fb30f200700 1 --2- 172.21.15.63:0/681857689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb310108560 0x7fb31010fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.815+0000 7fb30f200700 1 --2- 172.21.15.63:0/681857689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3101032e0 0x7fb3101036f0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.815+0000 7fb30f200700 1 --2- 172.21.15.63:0/681857689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb310107c10 0x7fb310108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.815+0000 7fb30f200700 1 -- 172.21.15.63:0/681857689 >> 172.21.15.63:0/681857689 conn(0x7fb310078af0 msgr2=0x7fb310078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:57.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.815+0000 7fb30f200700 1 -- 172.21.15.63:0/681857689 shutdown_connections 2024-05-27T07:43:57.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.815+0000 7fb30f200700 1 -- 172.21.15.63:0/681857689 wait complete. 2024-05-27T07:43:57.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.816+0000 7fb30f200700 1 Processor -- start 2024-05-27T07:43:57.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.817+0000 7fb30f200700 1 -- start start 2024-05-27T07:43:57.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.817+0000 7fb30f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3101032e0 0x7fb31019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.817+0000 7fb30f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb310107c10 0x7fb31019d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.817+0000 7fb30f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb310108560 0x7fb3101a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.817+0000 7fb30f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb310113090 con 0x7fb310107c10 2024-05-27T07:43:57.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.817+0000 7fb30f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb310112f10 con 0x7fb310108560 2024-05-27T07:43:57.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.817+0000 7fb30f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb310113210 con 0x7fb3101032e0 2024-05-27T07:43:57.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.818+0000 7fb30d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb310107c10 0x7fb31019d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.818+0000 7fb30d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb310107c10 0x7fb31019d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35090/0 (socket says 172.21.15.63:35090) 2024-05-27T07:43:57.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.818+0000 7fb30d400700 1 -- 172.21.15.63:0/2168913720 learned_addr learned my addr 172.21.15.63:0/2168913720 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:57.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.818+0000 7fb30e800700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb310108560 0x7fb3101a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.818+0000 7fb30de00700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3101032e0 0x7fb31019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.818+0000 7fb30d400700 1 -- 172.21.15.63:0/2168913720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3101032e0 msgr2=0x7fb31019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.818+0000 7fb30d400700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3101032e0 0x7fb31019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.818+0000 7fb30d400700 1 -- 172.21.15.63:0/2168913720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb310108560 msgr2=0x7fb3101a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.818+0000 7fb30d400700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb310108560 0x7fb3101a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.818+0000 7fb30d400700 1 -- 172.21.15.63:0/2168913720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3101a1b60 con 0x7fb310107c10 2024-05-27T07:43:57.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.819+0000 7fb30d400700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb310107c10 0x7fb31019d2e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00c9e0 tx=0x7fb2fc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:57.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.819+0000 7fb306a00700 1 -- 172.21.15.63:0/2168913720 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc007aa0 con 0x7fb310107c10 2024-05-27T07:43:57.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.819+0000 7fb306a00700 1 -- 172.21.15.63:0/2168913720 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2fc007c40 con 0x7fb310107c10 2024-05-27T07:43:57.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.819+0000 7fb30f200700 1 -- 172.21.15.63:0/2168913720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3101a1e50 con 0x7fb310107c10 2024-05-27T07:43:57.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.819+0000 7fb306a00700 1 -- 172.21.15.63:0/2168913720 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc018400 con 0x7fb310107c10 2024-05-27T07:43:57.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.820+0000 7fb30f200700 1 -- 172.21.15.63:0/2168913720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3101a2390 con 0x7fb310107c10 2024-05-27T07:43:57.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.825+0000 7fb32355bb80 1 Processor -- start 2024-05-27T07:43:57.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.825+0000 7fb306a00700 1 -- 172.21.15.63:0/2168913720 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb2fc00f450 con 0x7fb310107c10 2024-05-27T07:43:57.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.825+0000 7fb32355bb80 1 -- start start 2024-05-27T07:43:57.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.826+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea982e40 0x5635eab00ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.826+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635eab01410 0x5635eab0bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.826+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.826+0000 7fb32355bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5635eaa2fe40 con 0x5635eab01820 2024-05-27T07:43:57.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.826+0000 7fb32355bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635eaa1f9e0 con 0x5635eab01410 2024-05-27T07:43:57.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.826+0000 7fb32355bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5635eaa23a20 con 0x5635ea982e40 2024-05-27T07:43:57.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.826+0000 7fb30d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.827+0000 7fb30d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35098/0 (socket says 172.21.15.63:35098) 2024-05-27T07:43:57.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.827+0000 7fb30d400700 1 -- 172.21.15.63:0/1687213268 learned_addr learned my addr 172.21.15.63:0/1687213268 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:57.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.827+0000 7fb30de00700 1 --2- 172.21.15.63:0/1687213268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635eab01410 0x5635eab0bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.827+0000 7fb30e800700 1 --2- 172.21.15.63:0/1687213268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea982e40 0x5635eab00ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.827+0000 7fb30d400700 1 -- 172.21.15.63:0/1687213268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea982e40 msgr2=0x5635eab00ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:57.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.827+0000 7fb306a00700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2f40b29f0 0x7fb2f40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.827+0000 7fb30d400700 1 --2- 172.21.15.63:0/1687213268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea982e40 0x5635eab00ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.827+0000 7fb30d400700 1 -- 172.21.15.63:0/1687213268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635eab01410 msgr2=0x5635eab0bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:57.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.827+0000 7fb30d400700 1 --2- 172.21.15.63:0/1687213268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635eab01410 0x5635eab0bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.828+0000 7fb30d400700 1 -- 172.21.15.63:0/1687213268 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ea9e8420 con 0x5635eab01820 2024-05-27T07:43:57.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.828+0000 7fb30e800700 1 --2- 172.21.15.63:0/1687213268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea982e40 0x5635eab00ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:43:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.828+0000 7fb30e800700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2f40b29f0 0x7fb2f40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.828+0000 7fb306a00700 1 -- 172.21.15.63:0/2168913720 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fb2fc0e2900 con 0x7fb310107c10 2024-05-27T07:43:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.828+0000 7fb30d400700 1 --2- 172.21.15.63:0/1687213268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0e020 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0dfc00 tx=0x7fb2fc06ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6270cbd1ace8cc47 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.829+0000 7fb2fb400700 1 -- 172.21.15.63:0/1687213268 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc0a95f0 con 0x5635eab01820 2024-05-27T07:43:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.829+0000 7fb2fb400700 1 -- 172.21.15.63:0/1687213268 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2fc0e4e50 con 0x5635eab01820 2024-05-27T07:43:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.829+0000 7fb2fb400700 1 -- 172.21.15.63:0/1687213268 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc0e0120 con 0x5635eab01820 2024-05-27T07:43:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.830+0000 7fb32355bb80 1 -- 172.21.15.63:0/1687213268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 msgr2=0x5635eab0e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.830+0000 7fb30e800700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2f40b29f0 0x7fb2f40b4f00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb308007fd0 tx=0x7fb308009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.830+0000 7fb32355bb80 1 --2- 172.21.15.63:0/1687213268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0e020 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0dfc00 tx=0x7fb2fc06ed10 comp rx=0 tx=0).stop 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.830+0000 7fb32355bb80 1 -- 172.21.15.63:0/1687213268 shutdown_connections 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.830+0000 7fb32355bb80 1 --2- 172.21.15.63:0/1687213268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0e020 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.830+0000 7fb32355bb80 1 --2- 172.21.15.63:0/1687213268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea982e40 0x5635eab00ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.830+0000 7fb32355bb80 1 --2- 172.21.15.63:0/1687213268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635eab01410 0x5635eab0bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.830+0000 7fb32355bb80 1 -- 172.21.15.63:0/1687213268 >> 172.21.15.63:0/1687213268 conn(0x5635ea9e2fc0 msgr2=0x5635eaafcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.830+0000 7fb32355bb80 1 -- 172.21.15.63:0/1687213268 shutdown_connections 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.830+0000 7fb32355bb80 1 -- 172.21.15.63:0/1687213268 wait complete. 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.831+0000 7fb32355bb80 1 -- start start 2024-05-27T07:43:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.831+0000 7fb32355bb80 1 Processor -- start 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.832+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ea982e40 0x5635eab0b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.832+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 0x5635eab0bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.832+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.832+0000 7fb32355bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5635ea925fa0 con 0x5635eab01820 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.832+0000 7fb32355bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635ea9c0760 con 0x5635ea982e40 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.832+0000 7fb32355bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5635eab0e8d0 con 0x5635eab01410 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.832+0000 7fb32355bb80 10 client.0 _ll_register_callbacks cb 0x5635ea9c9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.833+0000 7fb30e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.833+0000 7fb30e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35108/0 (socket says 172.21.15.63:35108) 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.833+0000 7fb30e800700 1 -- 172.21.15.63:0/3151785094 learned_addr learned my addr 172.21.15.63:0/3151785094 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.833+0000 7fb30d400700 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 0x5635eab0bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.833+0000 7fb30de00700 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ea982e40 0x5635eab0b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.833+0000 7fb30e800700 1 -- 172.21.15.63:0/3151785094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 msgr2=0x5635eab0bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.833+0000 7fb30e800700 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 0x5635eab0bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.833+0000 7fb30e800700 1 -- 172.21.15.63:0/3151785094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ea982e40 msgr2=0x5635eab0b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.833+0000 7fb30e800700 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ea982e40 0x5635eab0b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.833+0000 7fb30e800700 1 -- 172.21.15.63:0/3151785094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ea96f8d0 con 0x5635eab01820 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.834+0000 7fb30e800700 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0c050 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb308016e50 tx=0x7fb308007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e246cd1fa38a3ed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.834+0000 7fb2efe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.834+0000 7fb2efe00700 1 -- 172.21.15.63:0/3151785094 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308026020 con 0x5635eab01820 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.835+0000 7fb2efe00700 1 -- 172.21.15.63:0/3151785094 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fb308015d60 con 0x5635eab01820 2024-05-27T07:43:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.835+0000 7fb32355bb80 1 -- 172.21.15.63:0/3151785094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2fc0e4e50 con 0x5635eab01820 2024-05-27T07:43:57.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.835+0000 7fb2efe00700 1 -- 172.21.15.63:0/3151785094 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb308031050 con 0x5635eab01820 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.835+0000 7fb32355bb80 10 client.17430 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.835+0000 7fb32355bb80 1 -- 172.21.15.63:0/3151785094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5635ea9f0060 con 0x5635eab01820 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.835+0000 7fb2efe00700 1 -- 172.21.15.63:0/3151785094 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2fc0e4e50 con 0x5635eab01820 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.836+0000 7fb2efe00700 1 -- 172.21.15.63:0/3151785094 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb3080213b0 con 0x5635eab01820 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.836+0000 7fb32355bb80 10 client.17430 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.836+0000 7fb32355bb80 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.836+0000 7fb32355bb80 20 client.17430 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.836+0000 7fb32355bb80 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635eab1b990 0x5635eab1ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.836+0000 7fb32355bb80 4 client.17430 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.836+0000 7fb32355bb80 1 -- 172.21.15.63:0/3151785094 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635eaa31d90 con 0x5635eab1b990 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.837+0000 7fb30de00700 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635eab1b990 0x5635eab1ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.837+0000 7fb30de00700 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635eab1b990 0x5635eab1ddb0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4990dd61e52a7663 server_cookie=bd5bfa9acdd6913b in_seq=0 out_seq=0 2024-05-27T07:43:57.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.837+0000 7fb2efe00700 10 client.17430 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:57.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.839+0000 7fb2efe00700 1 -- 172.21.15.63:0/3151785094 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5635eaa31d90 con 0x5635eab1b990 2024-05-27T07:43:57.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.839+0000 7fb2efe00700 10 client.17430 handle_command_reply: tid=0 2024-05-27T07:43:57.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.839+0000 7fb32355bb80 1 client.17430 shutdown 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb32355bb80 1 -- 172.21.15.63:0/3151785094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 msgr2=0x5635eab0c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb32355bb80 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0c050 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb308016e50 tx=0x7fb308007950 comp rx=0 tx=0).stop 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb32355bb80 1 -- 172.21.15.63:0/3151785094 shutdown_connections 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb32355bb80 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635eab1b990 0x5635eab1ddb0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb32355bb80 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635eab01820 0x5635eab0c050 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb32355bb80 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ea982e40 0x5635eab0b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb32355bb80 1 --2- 172.21.15.63:0/3151785094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 0x5635eab0bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb32355bb80 1 -- 172.21.15.63:0/3151785094 >> 172.21.15.63:0/3151785094 conn(0x5635ea9e2fc0 msgr2=0x5635eaafbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb2efe00700 0 client.17430 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb32355bb80 1 -- 172.21.15.63:0/3151785094 shutdown_connections 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.840+0000 7fb32355bb80 1 -- 172.21.15.63:0/3151785094 wait complete. 2024-05-27T07:43:57.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.841+0000 7fb32355bb80 20 client.17430 trim_cache size 0 max 16384 2024-05-27T07:43:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.870+0000 7fb32355bb80 1 Processor -- start 2024-05-27T07:43:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.870+0000 7fb32355bb80 1 -- start start 2024-05-27T07:43:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.871+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea9e2fc0 0x5635eab0bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.871+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab17970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.871+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635eab01410 0x5635eab05010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.871+0000 7fb32355bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb3101b4d80 con 0x5635ea982e40 2024-05-27T07:43:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.871+0000 7fb32355bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3101b2610 con 0x5635eab01410 2024-05-27T07:43:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.871+0000 7fb32355bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb3101b0480 con 0x5635ea9e2fc0 2024-05-27T07:43:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.871+0000 7fb30d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab17970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.871+0000 7fb30d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab17970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35112/0 (socket says 172.21.15.63:35112) 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.871+0000 7fb30d400700 1 -- 172.21.15.63:0/4048693589 learned_addr learned my addr 172.21.15.63:0/4048693589 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.871+0000 7fb30de00700 1 --2- 172.21.15.63:0/4048693589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea9e2fc0 0x5635eab0bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.872+0000 7fb30e800700 1 --2- 172.21.15.63:0/4048693589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635eab01410 0x5635eab05010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.872+0000 7fb30d400700 1 -- 172.21.15.63:0/4048693589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea9e2fc0 msgr2=0x5635eab0bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.872+0000 7fb30d400700 1 --2- 172.21.15.63:0/4048693589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea9e2fc0 0x5635eab0bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.872+0000 7fb30d400700 1 -- 172.21.15.63:0/4048693589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635eab01410 msgr2=0x5635eab05010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.872+0000 7fb30d400700 1 --2- 172.21.15.63:0/4048693589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635eab01410 0x5635eab05010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.872+0000 7fb30d400700 1 -- 172.21.15.63:0/4048693589 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635eaaf71d0 con 0x5635ea982e40 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.872+0000 7fb30d400700 1 --2- 172.21.15.63:0/4048693589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab17970 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0b1000 tx=0x7fb2fc091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0c13e37daef093 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.873+0000 7fb2faa00700 1 -- 172.21.15.63:0/4048693589 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc06f220 con 0x5635ea982e40 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.873+0000 7fb2faa00700 1 -- 172.21.15.63:0/4048693589 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2fc0a9330 con 0x5635ea982e40 2024-05-27T07:43:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.873+0000 7fb2faa00700 1 -- 172.21.15.63:0/4048693589 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc06fc50 con 0x5635ea982e40 2024-05-27T07:43:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.874+0000 7fb32355bb80 1 -- 172.21.15.63:0/4048693589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 msgr2=0x5635eab17970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.874+0000 7fb32355bb80 1 --2- 172.21.15.63:0/4048693589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab17970 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0b1000 tx=0x7fb2fc091040 comp rx=0 tx=0).stop 2024-05-27T07:43:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.874+0000 7fb32355bb80 1 -- 172.21.15.63:0/4048693589 shutdown_connections 2024-05-27T07:43:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.874+0000 7fb32355bb80 1 --2- 172.21.15.63:0/4048693589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635eab01410 0x5635eab05010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.874+0000 7fb32355bb80 1 --2- 172.21.15.63:0/4048693589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635ea9e2fc0 0x5635eab0bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.874+0000 7fb32355bb80 1 --2- 172.21.15.63:0/4048693589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab17970 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.874+0000 7fb32355bb80 1 -- 172.21.15.63:0/4048693589 >> 172.21.15.63:0/4048693589 conn(0x5635eaaf5f60 msgr2=0x5635eab04900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:57.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.874+0000 7fb32355bb80 1 -- 172.21.15.63:0/4048693589 shutdown_connections 2024-05-27T07:43:57.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.874+0000 7fb32355bb80 1 -- 172.21.15.63:0/4048693589 wait complete. 2024-05-27T07:43:57.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.875+0000 7fb32355bb80 1 -- start start 2024-05-27T07:43:57.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.875+0000 7fb32355bb80 1 Processor -- start 2024-05-27T07:43:57.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.876+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab0ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.876+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ea9e2fc0 0x5635eab104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.876+0000 7fb32355bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 0x5635eab85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.876+0000 7fb32355bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb3101b0ad0 con 0x5635ea982e40 2024-05-27T07:43:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.876+0000 7fb32355bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3101b1f40 con 0x5635ea9e2fc0 2024-05-27T07:43:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.876+0000 7fb32355bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb3101b11a0 con 0x5635eab01410 2024-05-27T07:43:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.876+0000 7fb32355bb80 10 client.0 _ll_register_callbacks cb 0x5635ea9c9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.876+0000 7fb30de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab0ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.876+0000 7fb30de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab0ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35124/0 (socket says 172.21.15.63:35124) 2024-05-27T07:43:57.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.876+0000 7fb30de00700 1 -- 172.21.15.63:0/1017159695 learned_addr learned my addr 172.21.15.63:0/1017159695 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:57.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.877+0000 7fb30d400700 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ea9e2fc0 0x5635eab104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.877+0000 7fb30e800700 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 0x5635eab85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.877+0000 7fb30de00700 1 -- 172.21.15.63:0/1017159695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 msgr2=0x5635eab85440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:57.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.877+0000 7fb30de00700 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 0x5635eab85440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.877+0000 7fb30de00700 1 -- 172.21.15.63:0/1017159695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ea9e2fc0 msgr2=0x5635eab104d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:57.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.877+0000 7fb30de00700 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ea9e2fc0 0x5635eab104d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.877+0000 7fb30de00700 1 -- 172.21.15.63:0/1017159695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635eaafe350 con 0x5635ea982e40 2024-05-27T07:43:57.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.877+0000 7fb30e800700 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 0x5635eab85440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:43:57.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.877+0000 7fb30de00700 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab0ff90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb30000b580 tx=0x7fb3000053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97466134be4999f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:57.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.877+0000 7fb2efe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:57.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.878+0000 7fb2efe00700 1 -- 172.21.15.63:0/1017159695 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30000bce0 con 0x5635ea982e40 2024-05-27T07:43:57.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.878+0000 7fb2efe00700 1 -- 172.21.15.63:0/1017159695 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fb300003680 con 0x5635ea982e40 2024-05-27T07:43:57.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.878+0000 7fb32355bb80 1 -- 172.21.15.63:0/1017159695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2fc0a9330 con 0x5635ea982e40 2024-05-27T07:43:57.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.878+0000 7fb2efe00700 1 -- 172.21.15.63:0/1017159695 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb30003a050 con 0x5635ea982e40 2024-05-27T07:43:57.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.878+0000 7fb32355bb80 10 client.17436 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.878+0000 7fb32355bb80 1 -- 172.21.15.63:0/1017159695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5635eab86950 con 0x5635ea982e40 2024-05-27T07:43:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.879+0000 7fb2efe00700 1 -- 172.21.15.63:0/1017159695 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2fc0a9330 con 0x5635ea982e40 2024-05-27T07:43:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.879+0000 7fb2efe00700 1 -- 172.21.15.63:0/1017159695 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb30000f880 con 0x5635ea982e40 2024-05-27T07:43:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.879+0000 7fb32355bb80 10 client.17436 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.879+0000 7fb32355bb80 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.879+0000 7fb32355bb80 20 client.17436 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.879+0000 7fb32355bb80 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635eab01820 0x5635eab890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.879+0000 7fb32355bb80 4 client.17436 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:43:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.879+0000 7fb32355bb80 1 -- 172.21.15.63:0/1017159695 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635eaa2b570 con 0x5635eab01820 2024-05-27T07:43:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.880+0000 7fb30d400700 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635eab01820 0x5635eab890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.881+0000 7fb30d400700 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635eab01820 0x5635eab890a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f50b1221ff7e7151 server_cookie=a617f9ef47d643a5 in_seq=0 out_seq=0 2024-05-27T07:43:57.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.881+0000 7fb2efe00700 10 client.17436 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:57.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.881+0000 7fb2efe00700 1 -- 172.21.15.63:0/1017159695 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5635eaa2b570 con 0x5635eab01820 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.881+0000 7fb2efe00700 10 client.17436 handle_command_reply: tid=0 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.881+0000 7fb32355bb80 1 client.17436 shutdown 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.881+0000 7fb32355bb80 1 -- 172.21.15.63:0/1017159695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 msgr2=0x5635eab0ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.881+0000 7fb32355bb80 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab0ff90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb30000b580 tx=0x7fb3000053e0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.882+0000 7fb32355bb80 1 -- 172.21.15.63:0/1017159695 shutdown_connections 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.882+0000 7fb32355bb80 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635eab01820 0x5635eab890a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.882+0000 7fb32355bb80 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635eab01410 0x5635eab85440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.882+0000 7fb32355bb80 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635ea982e40 0x5635eab0ff90 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.882+0000 7fb32355bb80 1 --2- 172.21.15.63:0/1017159695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635ea9e2fc0 0x5635eab104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.882+0000 7fb32355bb80 1 -- 172.21.15.63:0/1017159695 >> 172.21.15.63:0/1017159695 conn(0x5635eaaf5f60 msgr2=0x5635eab0d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.882+0000 7fb2efe00700 0 client.17436 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.882+0000 7fb32355bb80 1 -- 172.21.15.63:0/1017159695 shutdown_connections 2024-05-27T07:43:57.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.882+0000 7fb32355bb80 1 -- 172.21.15.63:0/1017159695 wait complete. 2024-05-27T07:43:57.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.882+0000 7fb32355bb80 20 client.17436 trim_cache size 0 max 16384 2024-05-27T07:43:57.882 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:43:57.882 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:43:57.882 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:43:57.882 INFO:teuthology.orchestra.run.smithi063.stdout: "01f6aa12-6052-4144-a2bc-123f22fc368a": { 2024-05-27T07:43:57.882 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:43:57.882 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "01f6aa12-6052-4144-a2bc-123f22fc368a", 2024-05-27T07:43:57.882 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:43:57.883 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:43:57.883 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:43:57.883 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:43:57.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 -- 172.21.15.63:0/2168913720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2f40b29f0 msgr2=0x7fb2f40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2f40b29f0 0x7fb2f40b4f00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb308007fd0 tx=0x7fb308009040 comp rx=0 tx=0).stop 2024-05-27T07:43:57.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 -- 172.21.15.63:0/2168913720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb310107c10 msgr2=0x7fb31019d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb310107c10 0x7fb31019d2e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00c9e0 tx=0x7fb2fc00ccf0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 -- 172.21.15.63:0/2168913720 shutdown_connections 2024-05-27T07:43:57.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2f40b29f0 0x7fb2f40b4f00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb310108560 0x7fb3101a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3101032e0 0x7fb31019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 --2- 172.21.15.63:0/2168913720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb310107c10 0x7fb31019d2e0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 -- 172.21.15.63:0/2168913720 >> 172.21.15.63:0/2168913720 conn(0x7fb310078af0 msgr2=0x7fb31010ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:57.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 -- 172.21.15.63:0/2168913720 shutdown_connections 2024-05-27T07:43:57.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:43:57.883+0000 7fb30f200700 1 -- 172.21.15.63:0/2168913720 wait complete. 2024-05-27T07:43:58.736 INFO:tasks.cephfs.filesystem:scrub status for tag:01f6aa12-6052-4144-a2bc-123f22fc368a - {'path': '/', 'tag': '01f6aa12-6052-4144-a2bc-123f22fc368a', 'options': 'recursive,force'} 2024-05-27T07:43:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.881+0000 7fe41b400640 1 -- 172.21.15.63:0/4188581546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe414109c90 msgr2=0x7fe414103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.881+0000 7fe41b400640 1 --2- 172.21.15.63:0/4188581546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe414109c90 0x7fe414103830 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe408009900 tx=0x7fe40802a410 comp rx=0 tx=0).stop 2024-05-27T07:43:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.881+0000 7fe41b400640 1 -- 172.21.15.63:0/4188581546 shutdown_connections 2024-05-27T07:43:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.881+0000 7fe41b400640 1 --2- 172.21.15.63:0/4188581546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4141043e0 0x7fe4141107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.881+0000 7fe41b400640 1 --2- 172.21.15.63:0/4188581546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe41410a130 0x7fe414103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.881+0000 7fe41b400640 1 --2- 172.21.15.63:0/4188581546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe414109c90 0x7fe414103830 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.881+0000 7fe41b400640 1 -- 172.21.15.63:0/4188581546 >> 172.21.15.63:0/4188581546 conn(0x7fe4140ff330 msgr2=0x7fe414101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.881+0000 7fe41b400640 1 -- 172.21.15.63:0/4188581546 shutdown_connections 2024-05-27T07:43:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.881+0000 7fe41b400640 1 -- 172.21.15.63:0/4188581546 wait complete. 2024-05-27T07:43:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.882+0000 7fe41b400640 1 Processor -- start 2024-05-27T07:43:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.882+0000 7fe41b400640 1 -- start start 2024-05-27T07:43:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe41b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4141043e0 0x7fe41419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe41b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe414109c90 0x7fe41419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe41b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe41410a130 0x7fe4141a8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe41b400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe414113d30 con 0x7fe41410a130 2024-05-27T07:43:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe41b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe414113bb0 con 0x7fe4141043e0 2024-05-27T07:43:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe41b400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe414113eb0 con 0x7fe414109c90 2024-05-27T07:43:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe419400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe41410a130 0x7fe4141a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe419400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe41410a130 0x7fe4141a8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35140/0 (socket says 172.21.15.63:35140) 2024-05-27T07:43:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe419400640 1 -- 172.21.15.63:0/1169961857 learned_addr learned my addr 172.21.15.63:0/1169961857 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe413e00640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe414109c90 0x7fe41419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.883+0000 7fe418a00640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4141043e0 0x7fe41419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.884+0000 7fe419400640 1 -- 172.21.15.63:0/1169961857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe414109c90 msgr2=0x7fe41419dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.884+0000 7fe419400640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe414109c90 0x7fe41419dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.884+0000 7fe419400640 1 -- 172.21.15.63:0/1169961857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4141043e0 msgr2=0x7fe41419d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.884+0000 7fe419400640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4141043e0 0x7fe41419d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.884+0000 7fe419400640 1 -- 172.21.15.63:0/1169961857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4141a8bb0 con 0x7fe41410a130 2024-05-27T07:43:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.884+0000 7fe413e00640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe414109c90 0x7fe41419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.884+0000 7fe419400640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe41410a130 0x7fe4141a8670 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00cad0 tx=0x7fe40c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.885+0000 7fe411600640 1 -- 172.21.15.63:0/1169961857 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00ae20 con 0x7fe41410a130 2024-05-27T07:43:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.885+0000 7fe411600640 1 -- 172.21.15.63:0/1169961857 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe40c005020 con 0x7fe41410a130 2024-05-27T07:43:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.885+0000 7fe41b400640 1 -- 172.21.15.63:0/1169961857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4141a8e40 con 0x7fe41410a130 2024-05-27T07:43:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.885+0000 7fe411600640 1 -- 172.21.15.63:0/1169961857 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00f6b0 con 0x7fe41410a130 2024-05-27T07:43:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.885+0000 7fe41b400640 1 -- 172.21.15.63:0/1169961857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4141a92a0 con 0x7fe41410a130 2024-05-27T07:43:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.889+0000 7fe422371740 1 Processor -- start 2024-05-27T07:43:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.889+0000 7fe411600640 1 -- 172.21.15.63:0/1169961857 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe40c0051d0 con 0x7fe41410a130 2024-05-27T07:43:58.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.889+0000 7fe422371740 1 -- start start 2024-05-27T07:43:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.890+0000 7fe422371740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce34860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.890+0000 7fe422371740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca692f0 0x56407ce34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.890+0000 7fe422371740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.890+0000 7fe422371740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56407cda5b90 con 0x56407ca714d0 2024-05-27T07:43:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.890+0000 7fe422371740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56407cd95730 con 0x56407ca921b0 2024-05-27T07:43:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.890+0000 7fe422371740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56407cd99770 con 0x56407ca692f0 2024-05-27T07:43:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.890+0000 7fe413e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.890+0000 7fe413e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce352e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35154/0 (socket says 172.21.15.63:35154) 2024-05-27T07:43:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.890+0000 7fe413e00640 1 -- 172.21.15.63:0/1632939735 learned_addr learned my addr 172.21.15.63:0/1632939735 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.891+0000 7fe413e00640 1 --2- 172.21.15.63:0/1632939735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce34860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.891+0000 7fe418a00640 1 --2- 172.21.15.63:0/1632939735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca692f0 0x56407ce34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.891+0000 7fe413e00640 1 -- 172.21.15.63:0/1632939735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca692f0 msgr2=0x56407ce34da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.891+0000 7fe413e00640 1 --2- 172.21.15.63:0/1632939735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca692f0 0x56407ce34da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.891+0000 7fe413e00640 1 -- 172.21.15.63:0/1632939735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 msgr2=0x56407ce34860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.891+0000 7fe413e00640 1 --2- 172.21.15.63:0/1632939735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce34860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.891+0000 7fe413e00640 1 -- 172.21.15.63:0/1632939735 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56407cd2e8b0 con 0x56407ca714d0 2024-05-27T07:43:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.891+0000 7fe411600640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe3fc0b29d0 0x7fe3fc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.892+0000 7fe411600640 1 -- 172.21.15.63:0/1169961857 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fe40c0e2d70 con 0x7fe41410a130 2024-05-27T07:43:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.892+0000 7fe413e00640 1 --2- 172.21.15.63:0/1632939735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce352e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe4000058c0 tx=0x7fe40000c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2fe81bf5a0beca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.892+0000 7fe418a00640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe3fc0b29d0 0x7fe3fc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.892+0000 7fe405c00640 1 -- 172.21.15.63:0/1632939735 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4000142d0 con 0x56407ca714d0 2024-05-27T07:43:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.892+0000 7fe405c00640 1 -- 172.21.15.63:0/1632939735 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe40000f760 con 0x56407ca714d0 2024-05-27T07:43:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.892+0000 7fe405c00640 1 -- 172.21.15.63:0/1632939735 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4000037f0 con 0x56407ca714d0 2024-05-27T07:43:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.893+0000 7fe422371740 1 -- 172.21.15.63:0/1632939735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 msgr2=0x56407ce352e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.893+0000 7fe418a00640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe3fc0b29d0 0x7fe3fc0b4ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe408004770 tx=0x7fe4080058d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.893+0000 7fe422371740 1 --2- 172.21.15.63:0/1632939735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce352e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe4000058c0 tx=0x7fe40000c4b0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.893+0000 7fe422371740 1 -- 172.21.15.63:0/1632939735 shutdown_connections 2024-05-27T07:43:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.893+0000 7fe422371740 1 --2- 172.21.15.63:0/1632939735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce352e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.893+0000 7fe422371740 1 --2- 172.21.15.63:0/1632939735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca692f0 0x56407ce34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.893+0000 7fe422371740 1 --2- 172.21.15.63:0/1632939735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce34860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.893+0000 7fe422371740 1 -- 172.21.15.63:0/1632939735 >> 172.21.15.63:0/1632939735 conn(0x56407ca86300 msgr2=0x56407ce31010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.894+0000 7fe422371740 1 -- 172.21.15.63:0/1632939735 shutdown_connections 2024-05-27T07:43:58.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.894+0000 7fe422371740 1 -- 172.21.15.63:0/1632939735 wait complete. 2024-05-27T07:43:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.894+0000 7fe422371740 1 -- start start 2024-05-27T07:43:58.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.895+0000 7fe422371740 1 Processor -- start 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.895+0000 7fe422371740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce3ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.895+0000 7fe422371740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 0x56407ce3ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.895+0000 7fe422371740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce4acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.895+0000 7fe422371740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56407ca7b790 con 0x56407ca692f0 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.895+0000 7fe422371740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56407cd3dbc0 con 0x56407ca921b0 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.895+0000 7fe422371740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56407ce41990 con 0x56407ca714d0 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe422371740 10 client.0 _ll_register_callbacks cb 0x56407cd1b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe418a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce3ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe418a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce3ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35168/0 (socket says 172.21.15.63:35168) 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe418a00640 1 -- 172.21.15.63:0/2267990221 learned_addr learned my addr 172.21.15.63:0/2267990221 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe419400640 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce4acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe413e00640 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 0x56407ce3ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe418a00640 1 -- 172.21.15.63:0/2267990221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 msgr2=0x56407ce3ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe418a00640 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 0x56407ce3ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe418a00640 1 -- 172.21.15.63:0/2267990221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 msgr2=0x56407ce4acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe418a00640 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce4acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe418a00640 1 -- 172.21.15.63:0/2267990221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56407caaa210 con 0x56407ca692f0 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.896+0000 7fe419400640 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce4acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.897+0000 7fe418a00640 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce3ea30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe408009900 tx=0x7fe408009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6238a6ac61abdeb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.897+0000 7fe3fa000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.897+0000 7fe3fa000640 1 -- 172.21.15.63:0/2267990221 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408035040 con 0x56407ca692f0 2024-05-27T07:43:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.897+0000 7fe3fa000640 1 -- 172.21.15.63:0/2267990221 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fe4080035f0 con 0x56407ca692f0 2024-05-27T07:43:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.897+0000 7fe422371740 1 -- 172.21.15.63:0/2267990221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe40000f760 con 0x56407ca692f0 2024-05-27T07:43:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe3fa000640 1 -- 172.21.15.63:0/2267990221 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe4080037f0 con 0x56407ca692f0 2024-05-27T07:43:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe422371740 10 client.17448 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe422371740 1 -- 172.21.15.63:0/2267990221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56407cb76620 con 0x56407ca692f0 2024-05-27T07:43:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe3fa000640 1 -- 172.21.15.63:0/2267990221 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe40000f760 con 0x56407ca692f0 2024-05-27T07:43:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe3fa000640 1 -- 172.21.15.63:0/2267990221 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe408044030 con 0x56407ca692f0 2024-05-27T07:43:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe422371740 10 client.17448 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe422371740 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe422371740 20 client.17448 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe422371740 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56407ca50d80 0x56407ce50ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe422371740 4 client.17448 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:43:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.898+0000 7fe422371740 1 -- 172.21.15.63:0/2267990221 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56407cda7ae0 con 0x56407ca50d80 2024-05-27T07:43:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.899+0000 7fe413e00640 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56407ca50d80 0x56407ce50ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.900+0000 7fe413e00640 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56407ca50d80 0x56407ce50ca0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8104c2b8bb31111 server_cookie=3077750223688d97 in_seq=0 out_seq=0 2024-05-27T07:43:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.900+0000 7fe3fa000640 10 client.17448 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe3fa000640 1 -- 172.21.15.63:0/2267990221 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56407cda7ae0 con 0x56407ca50d80 2024-05-27T07:43:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe3fa000640 10 client.17448 handle_command_reply: tid=0 2024-05-27T07:43:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe422371740 1 client.17448 shutdown 2024-05-27T07:43:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe422371740 1 -- 172.21.15.63:0/2267990221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 msgr2=0x56407ce3ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe422371740 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce3ea30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe408009900 tx=0x7fe408009260 comp rx=0 tx=0).stop 2024-05-27T07:43:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe422371740 1 -- 172.21.15.63:0/2267990221 shutdown_connections 2024-05-27T07:43:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe422371740 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56407ca50d80 0x56407ce50ca0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe422371740 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce4acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe422371740 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 0x56407ce3ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe422371740 1 --2- 172.21.15.63:0/2267990221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce3ea30 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe422371740 1 -- 172.21.15.63:0/2267990221 >> 172.21.15.63:0/2267990221 conn(0x56407ca86300 msgr2=0x56407ce36580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.902+0000 7fe3fa000640 0 client.17448 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.903+0000 7fe422371740 1 -- 172.21.15.63:0/2267990221 shutdown_connections 2024-05-27T07:43:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.903+0000 7fe422371740 1 -- 172.21.15.63:0/2267990221 wait complete. 2024-05-27T07:43:58.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.903+0000 7fe422371740 20 client.17448 trim_cache size 0 max 16384 2024-05-27T07:43:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.926+0000 7fe422371740 1 Processor -- start 2024-05-27T07:43:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.926+0000 7fe422371740 1 -- start start 2024-05-27T07:43:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.926+0000 7fe422371740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca692f0 0x56407ce41160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.926+0000 7fe422371740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce3e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.926+0000 7fe422371740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce4aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.926+0000 7fe422371740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b5c60 con 0x56407ca714d0 2024-05-27T07:43:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.926+0000 7fe422371740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b3530 con 0x56407ca921b0 2024-05-27T07:43:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.926+0000 7fe422371740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b13a0 con 0x56407ca692f0 2024-05-27T07:43:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.927+0000 7fe413e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce3e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.927+0000 7fe413e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce3e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48442/0 (socket says 172.21.15.63:48442) 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.927+0000 7fe413e00640 1 -- 172.21.15.63:0/826034652 learned_addr learned my addr 172.21.15.63:0/826034652 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.927+0000 7fe418a00640 1 --2- 172.21.15.63:0/826034652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca692f0 0x56407ce41160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.927+0000 7fe419400640 1 --2- 172.21.15.63:0/826034652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce4aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.927+0000 7fe413e00640 1 -- 172.21.15.63:0/826034652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca692f0 msgr2=0x56407ce41160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.927+0000 7fe413e00640 1 --2- 172.21.15.63:0/826034652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca692f0 0x56407ce41160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.927+0000 7fe413e00640 1 -- 172.21.15.63:0/826034652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 msgr2=0x56407ce4aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.927+0000 7fe413e00640 1 --2- 172.21.15.63:0/826034652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce4aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.927+0000 7fe413e00640 1 -- 172.21.15.63:0/826034652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56407ce3cf80 con 0x56407ca714d0 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.928+0000 7fe413e00640 1 --2- 172.21.15.63:0/826034652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce3e4f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe40000f730 tx=0x7fe400014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed10aa288de7ae83 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.928+0000 7fe405200640 1 -- 172.21.15.63:0/826034652 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400005230 con 0x56407ca714d0 2024-05-27T07:43:58.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.928+0000 7fe405200640 1 -- 172.21.15.63:0/826034652 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4000053d0 con 0x56407ca714d0 2024-05-27T07:43:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.928+0000 7fe405200640 1 -- 172.21.15.63:0/826034652 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400004660 con 0x56407ca714d0 2024-05-27T07:43:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.929+0000 7fe422371740 1 -- 172.21.15.63:0/826034652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 msgr2=0x56407ce3e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.929+0000 7fe422371740 1 --2- 172.21.15.63:0/826034652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce3e4f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe40000f730 tx=0x7fe400014f90 comp rx=0 tx=0).stop 2024-05-27T07:43:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.929+0000 7fe422371740 1 -- 172.21.15.63:0/826034652 shutdown_connections 2024-05-27T07:43:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.929+0000 7fe422371740 1 --2- 172.21.15.63:0/826034652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce4aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.929+0000 7fe422371740 1 --2- 172.21.15.63:0/826034652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca714d0 0x56407ce3e4f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.929+0000 7fe422371740 1 --2- 172.21.15.63:0/826034652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca692f0 0x56407ce41160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.929+0000 7fe422371740 1 -- 172.21.15.63:0/826034652 >> 172.21.15.63:0/826034652 conn(0x56407ca86300 msgr2=0x56407ce39c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.929+0000 7fe422371740 1 -- 172.21.15.63:0/826034652 shutdown_connections 2024-05-27T07:43:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.929+0000 7fe422371740 1 -- 172.21.15.63:0/826034652 wait complete. 2024-05-27T07:43:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.930+0000 7fe422371740 1 -- start start 2024-05-27T07:43:58.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.931+0000 7fe422371740 1 Processor -- start 2024-05-27T07:43:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.931+0000 7fe422371740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce46be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.931+0000 7fe422371740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 0x56407ce47120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.931+0000 7fe422371740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.931+0000 7fe422371740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b19f0 con 0x56407ca692f0 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.931+0000 7fe422371740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b2e60 con 0x56407ca921b0 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.931+0000 7fe422371740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141b20c0 con 0x56407ca714d0 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.931+0000 7fe422371740 10 client.0 _ll_register_callbacks cb 0x7fe408009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.931+0000 7fe418a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce46be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe418a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce46be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48452/0 (socket says 172.21.15.63:48452) 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe418a00640 1 -- 172.21.15.63:0/3988975585 learned_addr learned my addr 172.21.15.63:0/3988975585 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe413e00640 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 0x56407ce47120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe419400640 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe418a00640 1 -- 172.21.15.63:0/3988975585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 msgr2=0x56407ce47120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe418a00640 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 0x56407ce47120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe418a00640 1 -- 172.21.15.63:0/3988975585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 msgr2=0x56407ce421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe418a00640 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe418a00640 1 -- 172.21.15.63:0/3988975585 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56407ce42850 con 0x56407ca692f0 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe413e00640 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 0x56407ce47120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe418a00640 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce46be0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe40802a930 tx=0x7fe408004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcb78519b63d274 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.932+0000 7fe3fa000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.933+0000 7fe3fa000640 1 -- 172.21.15.63:0/3988975585 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408035050 con 0x56407ca692f0 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.933+0000 7fe3fa000640 1 -- 172.21.15.63:0/3988975585 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fe408003ba0 con 0x56407ca692f0 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.933+0000 7fe422371740 1 -- 172.21.15.63:0/3988975585 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4000053d0 con 0x56407ca692f0 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.933+0000 7fe3fa000640 1 -- 172.21.15.63:0/3988975585 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe40806b050 con 0x56407ca692f0 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.933+0000 7fe422371740 10 client.17454 fetch_fsmap learned FSMap version 33 2024-05-27T07:43:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.933+0000 7fe422371740 1 -- 172.21.15.63:0/3988975585 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56407ceb8c70 con 0x56407ca692f0 2024-05-27T07:43:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.934+0000 7fe3fa000640 1 -- 172.21.15.63:0/3988975585 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4000053d0 con 0x56407ca692f0 2024-05-27T07:43:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.934+0000 7fe3fa000640 1 -- 172.21.15.63:0/3988975585 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe408066030 con 0x56407ca692f0 2024-05-27T07:43:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.934+0000 7fe422371740 10 client.17454 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:43:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.934+0000 7fe422371740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:43:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.934+0000 7fe422371740 20 client.17454 populate_metadata read hostname 'smithi063' 2024-05-27T07:43:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.934+0000 7fe422371740 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56407ca261c0 0x56407cebb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.934+0000 7fe422371740 4 client.17454 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:43:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.934+0000 7fe422371740 1 -- 172.21.15.63:0/3988975585 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56407ce3f3a0 con 0x56407ca261c0 2024-05-27T07:43:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.935+0000 7fe413e00640 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56407ca261c0 0x56407cebb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.936+0000 7fe3fa000640 10 client.17454 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.936+0000 7fe413e00640 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56407ca261c0 0x56407cebb4b0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55383c0268f7fe3c server_cookie=cb3d9e6347dfd2a in_seq=0 out_seq=0 2024-05-27T07:43:58.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:43:58 smithi086 ceph-mon[34687]: pgmap v323: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2024-05-27T07:43:58.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:43:58 smithi149 ceph-mon[35408]: pgmap v323: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2024-05-27T07:43:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.964+0000 7fe3fa000640 1 -- 172.21.15.63:0/3988975585 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56407ce3f3a0 con 0x56407ca261c0 2024-05-27T07:43:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.964+0000 7fe3fa000640 10 client.17454 handle_command_reply: tid=0 2024-05-27T07:43:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.964+0000 7fe422371740 1 client.17454 shutdown 2024-05-27T07:43:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.964+0000 7fe422371740 1 -- 172.21.15.63:0/3988975585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 msgr2=0x56407ce46be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.964+0000 7fe422371740 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce46be0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe40802a930 tx=0x7fe408004600 comp rx=0 tx=0).stop 2024-05-27T07:43:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.965+0000 7fe422371740 1 -- 172.21.15.63:0/3988975585 shutdown_connections 2024-05-27T07:43:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.965+0000 7fe422371740 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56407ca261c0 0x56407cebb4b0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.965+0000 7fe422371740 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407ca921b0 0x56407ce421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.965+0000 7fe3fa000640 0 client.17454 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:43:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.965+0000 7fe422371740 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56407ca714d0 0x56407ce47120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.965+0000 7fe422371740 1 --2- 172.21.15.63:0/3988975585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56407ca692f0 0x56407ce46be0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.965+0000 7fe422371740 1 -- 172.21.15.63:0/3988975585 >> 172.21.15.63:0/3988975585 conn(0x56407ca86300 msgr2=0x56407ce40090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.965+0000 7fe422371740 1 -- 172.21.15.63:0/3988975585 shutdown_connections 2024-05-27T07:43:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.965+0000 7fe422371740 1 -- 172.21.15.63:0/3988975585 wait complete. 2024-05-27T07:43:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.965+0000 7fe422371740 20 client.17454 trim_cache size 0 max 16384 2024-05-27T07:43:58.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:43:58 smithi063 ceph-mon[31613]: pgmap v323: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2024-05-27T07:43:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:43:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:43:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:43:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.966+0000 7fe41b400640 1 -- 172.21.15.63:0/1169961857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe3fc0b29d0 msgr2=0x7fe3fc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.966+0000 7fe41b400640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe3fc0b29d0 0x7fe3fc0b4ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe408004770 tx=0x7fe4080058d0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.966+0000 7fe41b400640 1 -- 172.21.15.63:0/1169961857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe41410a130 msgr2=0x7fe4141a8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.966+0000 7fe41b400640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe41410a130 0x7fe4141a8670 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00cad0 tx=0x7fe40c00cf90 comp rx=0 tx=0).stop 2024-05-27T07:43:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.966+0000 7fe41b400640 1 -- 172.21.15.63:0/1169961857 shutdown_connections 2024-05-27T07:43:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.966+0000 7fe41b400640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe3fc0b29d0 0x7fe3fc0b4ee0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.966+0000 7fe41b400640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe41410a130 0x7fe4141a8670 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.966+0000 7fe41b400640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe414109c90 0x7fe41419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.966+0000 7fe41b400640 1 --2- 172.21.15.63:0/1169961857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4141043e0 0x7fe41419d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.966+0000 7fe41b400640 1 -- 172.21.15.63:0/1169961857 >> 172.21.15.63:0/1169961857 conn(0x7fe4140ff330 msgr2=0x7fe414076000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.967+0000 7fe41b400640 1 -- 172.21.15.63:0/1169961857 shutdown_connections 2024-05-27T07:43:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:43:58.967+0000 7fe41b400640 1 -- 172.21.15.63:0/1169961857 wait complete. 2024-05-27T07:44:00.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:00 smithi086 ceph-mon[34687]: pgmap v324: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.7 MiB/s wr, 33 op/s 2024-05-27T07:44:00.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:00 smithi149 ceph-mon[35408]: pgmap v324: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.7 MiB/s wr, 33 op/s 2024-05-27T07:44:00.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:00 smithi063 ceph-mon[31613]: pgmap v324: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.7 MiB/s wr, 33 op/s 2024-05-27T07:44:01.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:01 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:44:01.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:01 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:44:01.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:01 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:44:02.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:44:02.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:44:02.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:02 smithi086 ceph-mon[34687]: pgmap v325: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.3 MiB/s wr, 57 op/s 2024-05-27T07:44:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:44:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:44:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:02 smithi149 ceph-mon[35408]: pgmap v325: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.3 MiB/s wr, 57 op/s 2024-05-27T07:44:02.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:44:02.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:44:02.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:02 smithi063 ceph-mon[31613]: pgmap v325: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.3 MiB/s wr, 57 op/s 2024-05-27T07:44:04.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:04 smithi086 ceph-mon[34687]: pgmap v326: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.0 MiB/s wr, 53 op/s 2024-05-27T07:44:04.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:04 smithi149 ceph-mon[35408]: pgmap v326: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.0 MiB/s wr, 53 op/s 2024-05-27T07:44:04.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:04 smithi063 ceph-mon[31613]: pgmap v326: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.0 MiB/s wr, 53 op/s 2024-05-27T07:44:07.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:06 smithi086 ceph-mon[34687]: pgmap v327: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.0 MiB/s wr, 53 op/s 2024-05-27T07:44:07.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:06 smithi149 ceph-mon[35408]: pgmap v327: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.0 MiB/s wr, 53 op/s 2024-05-27T07:44:07.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:06 smithi063 ceph-mon[31613]: pgmap v327: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.0 MiB/s wr, 53 op/s 2024-05-27T07:44:08.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:07 smithi086 ceph-mon[34687]: osdmap e102: 12 total, 12 up, 12 in 2024-05-27T07:44:08.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:07 smithi149 ceph-mon[35408]: osdmap e102: 12 total, 12 up, 12 in 2024-05-27T07:44:08.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:07 smithi063 ceph-mon[31613]: osdmap e102: 12 total, 12 up, 12 in 2024-05-27T07:44:09.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:08 smithi086 ceph-mon[34687]: pgmap v329: 129 pgs: 129 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:44:09.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:08 smithi149 ceph-mon[35408]: pgmap v329: 129 pgs: 129 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:44:09.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:08 smithi063 ceph-mon[31613]: pgmap v329: 129 pgs: 129 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:44:11.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:10 smithi086 ceph-mon[34687]: pgmap v330: 129 pgs: 129 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:44:11.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:10 smithi149 ceph-mon[35408]: pgmap v330: 129 pgs: 129 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:44:11.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:10 smithi063 ceph-mon[31613]: pgmap v330: 129 pgs: 129 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:44:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.208+0000 7fd421c00640 1 -- 172.21.15.63:0/2160434218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c104bb0 msgr2=0x7fd41c104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.208+0000 7fd421c00640 1 --2- 172.21.15.63:0/2160434218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c104bb0 0x7fd41c104fc0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009900 tx=0x7fd40c02a410 comp rx=0 tx=0).stop 2024-05-27T07:44:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.209+0000 7fd421c00640 1 -- 172.21.15.63:0/2160434218 shutdown_connections 2024-05-27T07:44:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.209+0000 7fd421c00640 1 --2- 172.21.15.63:0/2160434218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd41c10e000 0x7fd41c1104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.209+0000 7fd421c00640 1 --2- 172.21.15.63:0/2160434218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd41c10b610 0x7fd41c10da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.209+0000 7fd421c00640 1 --2- 172.21.15.63:0/2160434218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c104bb0 0x7fd41c104fc0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.209+0000 7fd421c00640 1 -- 172.21.15.63:0/2160434218 >> 172.21.15.63:0/2160434218 conn(0x7fd41c0fe210 msgr2=0x7fd41c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.209+0000 7fd421c00640 1 -- 172.21.15.63:0/2160434218 shutdown_connections 2024-05-27T07:44:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.209+0000 7fd421c00640 1 -- 172.21.15.63:0/2160434218 wait complete. 2024-05-27T07:44:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.210+0000 7fd421c00640 1 Processor -- start 2024-05-27T07:44:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.210+0000 7fd421c00640 1 -- start start 2024-05-27T07:44:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.211+0000 7fd421c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c104bb0 0x7fd41c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.211+0000 7fd421c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd41c10b610 0x7fd41c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.211+0000 7fd421c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd41c10e000 0x7fd41c1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.211+0000 7fd421c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c113b10 con 0x7fd41c104bb0 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.211+0000 7fd421c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c113990 con 0x7fd41c10e000 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.211+0000 7fd421c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c113c90 con 0x7fd41c10b610 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.211+0000 7fd41be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c104bb0 0x7fd41c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.211+0000 7fd41be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c104bb0 0x7fd41c19c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54538/0 (socket says 172.21.15.63:54538) 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.211+0000 7fd41be00640 1 -- 172.21.15.63:0/3853852992 learned_addr learned my addr 172.21.15.63:0/3853852992 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.211+0000 7fd41b400640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd41c10b610 0x7fd41c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.212+0000 7fd421200640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd41c10e000 0x7fd41c1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.212+0000 7fd41be00640 1 -- 172.21.15.63:0/3853852992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd41c10b610 msgr2=0x7fd41c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.212+0000 7fd41be00640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd41c10b610 0x7fd41c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.212+0000 7fd41be00640 1 -- 172.21.15.63:0/3853852992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd41c10e000 msgr2=0x7fd41c1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.212+0000 7fd41be00640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd41c10e000 0x7fd41c1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.212+0000 7fd41be00640 1 -- 172.21.15.63:0/3853852992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c1a1610 con 0x7fd41c104bb0 2024-05-27T07:44:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.212+0000 7fd421200640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd41c10e000 0x7fd41c1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.212+0000 7fd41be00640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c104bb0 0x7fd41c19c7b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009230 tx=0x7fd40c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.212+0000 7fd418c00640 1 -- 172.21.15.63:0/3853852992 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c03f030 con 0x7fd41c104bb0 2024-05-27T07:44:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.213+0000 7fd421c00640 1 -- 172.21.15.63:0/3853852992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41c1a18a0 con 0x7fd41c104bb0 2024-05-27T07:44:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.213+0000 7fd418c00640 1 -- 172.21.15.63:0/3853852992 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd40c003a80 con 0x7fd41c104bb0 2024-05-27T07:44:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.213+0000 7fd418c00640 1 -- 172.21.15.63:0/3853852992 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c035070 con 0x7fd41c104bb0 2024-05-27T07:44:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.213+0000 7fd421c00640 1 -- 172.21.15.63:0/3853852992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41c1a1d90 con 0x7fd41c104bb0 2024-05-27T07:44:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.217+0000 7fd42a206740 1 Processor -- start 2024-05-27T07:44:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.217+0000 7fd42a206740 1 -- start start 2024-05-27T07:44:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.217+0000 7fd42a206740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd38aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.217+0000 7fd42a206740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd38afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.217+0000 7fd42a206740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd38b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.217+0000 7fd42a206740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556acd2fbb90 con 0x556accfc74d0 2024-05-27T07:44:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.217+0000 7fd42a206740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556acd2eb730 con 0x556accfbf2f0 2024-05-27T07:44:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.217+0000 7fd42a206740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556acd2ef770 con 0x556accfe81b0 2024-05-27T07:44:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.217+0000 7fd418c00640 1 -- 172.21.15.63:0/3853852992 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd40c042030 con 0x7fd41c104bb0 2024-05-27T07:44:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.218+0000 7fd41b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd38b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.218+0000 7fd41b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd38b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54554/0 (socket says 172.21.15.63:54554) 2024-05-27T07:44:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.218+0000 7fd41b400640 1 -- 172.21.15.63:0/2573945584 learned_addr learned my addr 172.21.15.63:0/2573945584 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.218+0000 7fd421200640 1 --2- 172.21.15.63:0/2573945584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd38aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.218+0000 7fd41be00640 1 --2- 172.21.15.63:0/2573945584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd38afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.218+0000 7fd41b400640 1 -- 172.21.15.63:0/2573945584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 msgr2=0x556acd38aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.218+0000 7fd41b400640 1 --2- 172.21.15.63:0/2573945584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd38aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.218+0000 7fd41b400640 1 -- 172.21.15.63:0/2573945584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 msgr2=0x556acd38afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.218+0000 7fd41b400640 1 --2- 172.21.15.63:0/2573945584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd38afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.218+0000 7fd41b400640 1 -- 172.21.15.63:0/2573945584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556acd2848b0 con 0x556accfc74d0 2024-05-27T07:44:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.219+0000 7fd41b400640 1 --2- 172.21.15.63:0/2573945584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd38b4e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd40800bc80 tx=0x7fd40800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb038d6b60789d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.219+0000 7fd411600640 1 -- 172.21.15.63:0/2573945584 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408018be0 con 0x556accfc74d0 2024-05-27T07:44:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.219+0000 7fd411600640 1 -- 172.21.15.63:0/2573945584 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd408018d80 con 0x556accfc74d0 2024-05-27T07:44:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.219+0000 7fd411600640 1 -- 172.21.15.63:0/2573945584 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40801f580 con 0x556accfc74d0 2024-05-27T07:44:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.219+0000 7fd418c00640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4000b2950 0x7fd4000b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.220+0000 7fd418c00640 1 -- 172.21.15.63:0/3853852992 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fd40c103020 con 0x7fd41c104bb0 2024-05-27T07:44:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.220+0000 7fd421200640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4000b2950 0x7fd4000b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.221+0000 7fd421200640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4000b2950 0x7fd4000b4e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x556acd387bd0 tx=0x7fd414009680 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.221+0000 7fd42a206740 1 -- 172.21.15.63:0/2573945584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 msgr2=0x556acd38b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.221+0000 7fd42a206740 1 --2- 172.21.15.63:0/2573945584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd38b4e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd40800bc80 tx=0x7fd40800f730 comp rx=0 tx=0).stop 2024-05-27T07:44:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.221+0000 7fd42a206740 1 -- 172.21.15.63:0/2573945584 shutdown_connections 2024-05-27T07:44:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.221+0000 7fd42a206740 1 --2- 172.21.15.63:0/2573945584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd38b4e0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.221+0000 7fd42a206740 1 --2- 172.21.15.63:0/2573945584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd38afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.221+0000 7fd42a206740 1 --2- 172.21.15.63:0/2573945584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd38aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.221+0000 7fd42a206740 1 -- 172.21.15.63:0/2573945584 >> 172.21.15.63:0/2573945584 conn(0x556accfdc300 msgr2=0x556acd387210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.221+0000 7fd42a206740 1 -- 172.21.15.63:0/2573945584 shutdown_connections 2024-05-27T07:44:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.221+0000 7fd42a206740 1 -- 172.21.15.63:0/2573945584 wait complete. 2024-05-27T07:44:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.222+0000 7fd42a206740 1 -- start start 2024-05-27T07:44:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.222+0000 7fd42a206740 1 Processor -- start 2024-05-27T07:44:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd42a206740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd394b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd42a206740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd3950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd42a206740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd395620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd42a206740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556accfd1790 con 0x556accfc74d0 2024-05-27T07:44:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd42a206740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556acd293bc0 con 0x556accfbf2f0 2024-05-27T07:44:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd42a206740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556acd397b90 con 0x556accfe81b0 2024-05-27T07:44:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd42a206740 10 client.0 _ll_register_callbacks cb 0x556acd271400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd41b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd3950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd41b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd3950a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54560/0 (socket says 172.21.15.63:54560) 2024-05-27T07:44:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd41b400640 1 -- 172.21.15.63:0/2241945006 learned_addr learned my addr 172.21.15.63:0/2241945006 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd421200640 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd395620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.223+0000 7fd41be00640 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd394b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.224+0000 7fd41b400640 1 -- 172.21.15.63:0/2241945006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 msgr2=0x556acd395620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.224+0000 7fd41b400640 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd395620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.224+0000 7fd41b400640 1 -- 172.21.15.63:0/2241945006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 msgr2=0x556acd394b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.224+0000 7fd41b400640 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd394b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.224+0000 7fd41b400640 1 -- 172.21.15.63:0/2241945006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556acd000210 con 0x556accfc74d0 2024-05-27T07:44:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.224+0000 7fd41b400640 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd3950a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd408020d50 tx=0x7fd408018c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9b80f881f5d4b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.224+0000 7fd406000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.224+0000 7fd406000640 1 -- 172.21.15.63:0/2241945006 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40802e020 con 0x556accfc74d0 2024-05-27T07:44:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.224+0000 7fd406000640 1 -- 172.21.15.63:0/2241945006 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fd40800bd70 con 0x556accfc74d0 2024-05-27T07:44:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.224+0000 7fd42a206740 1 -- 172.21.15.63:0/2241945006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd408018d80 con 0x556accfc74d0 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.225+0000 7fd406000640 1 -- 172.21.15.63:0/2241945006 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd408039050 con 0x556accfc74d0 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.225+0000 7fd42a206740 10 client.17466 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.225+0000 7fd42a206740 1 -- 172.21.15.63:0/2241945006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556acd0cc620 con 0x556accfc74d0 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.225+0000 7fd406000640 1 -- 172.21.15.63:0/2241945006 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd408018d80 con 0x556accfc74d0 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.225+0000 7fd406000640 1 -- 172.21.15.63:0/2241945006 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd4080207f0 con 0x556accfc74d0 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.225+0000 7fd42a206740 10 client.17466 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.225+0000 7fd42a206740 10 client.17466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.226+0000 7fd42a206740 20 client.17466 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.226+0000 7fd42a206740 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556accfa6d80 0x556acd3a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.226+0000 7fd42a206740 4 client.17466 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.226+0000 7fd42a206740 1 -- 172.21.15.63:0/2241945006 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556acd2fdae0 con 0x556accfa6d80 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.226+0000 7fd41be00640 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556accfa6d80 0x556acd3a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.227+0000 7fd41be00640 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556accfa6d80 0x556acd3a6cc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebcdaec369e65306 server_cookie=18393017442447cb in_seq=0 out_seq=0 2024-05-27T07:44:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.227+0000 7fd406000640 10 client.17466 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.230+0000 7fd406000640 1 -- 172.21.15.63:0/2241945006 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556acd2fdae0 con 0x556accfa6d80 2024-05-27T07:44:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.230+0000 7fd406000640 10 client.17466 handle_command_reply: tid=0 2024-05-27T07:44:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.230+0000 7fd42a206740 1 client.17466 shutdown 2024-05-27T07:44:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 1 -- 172.21.15.63:0/2241945006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 msgr2=0x556acd3950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd3950a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd408020d50 tx=0x7fd408018c50 comp rx=0 tx=0).stop 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 1 -- 172.21.15.63:0/2241945006 shutdown_connections 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556accfa6d80 0x556acd3a6cc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd395620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd3950a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 1 --2- 172.21.15.63:0/2241945006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd394b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd406000640 0 client.17466 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 1 -- 172.21.15.63:0/2241945006 >> 172.21.15.63:0/2241945006 conn(0x556accfdc300 msgr2=0x556acd38c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 1 -- 172.21.15.63:0/2241945006 shutdown_connections 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 1 -- 172.21.15.63:0/2241945006 wait complete. 2024-05-27T07:44:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.231+0000 7fd42a206740 20 client.17466 trim_cache size 0 max 16384 2024-05-27T07:44:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.251+0000 7fd42a206740 1 Processor -- start 2024-05-27T07:44:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.252+0000 7fd42a206740 1 -- start start 2024-05-27T07:44:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.252+0000 7fd42a206740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd393d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.252+0000 7fd42a206740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd395400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.252+0000 7fd42a206740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd38af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.252+0000 7fd42a206740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1b1890 con 0x556accfc74d0 2024-05-27T07:44:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.252+0000 7fd42a206740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1af160 con 0x556accfbf2f0 2024-05-27T07:44:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.252+0000 7fd42a206740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1ad030 con 0x556accfe81b0 2024-05-27T07:44:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd41b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd395400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd41b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd395400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54566/0 (socket says 172.21.15.63:54566) 2024-05-27T07:44:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd41b400640 1 -- 172.21.15.63:0/2294680284 learned_addr learned my addr 172.21.15.63:0/2294680284 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd421200640 1 --2- 172.21.15.63:0/2294680284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd38af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd41be00640 1 --2- 172.21.15.63:0/2294680284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd393d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd41b400640 1 -- 172.21.15.63:0/2294680284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 msgr2=0x556acd38af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd41b400640 1 --2- 172.21.15.63:0/2294680284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd38af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd41b400640 1 -- 172.21.15.63:0/2294680284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 msgr2=0x556acd393d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd41b400640 1 --2- 172.21.15.63:0/2294680284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd393d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd41b400640 1 -- 172.21.15.63:0/2294680284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556acd3930b0 con 0x556accfc74d0 2024-05-27T07:44:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.253+0000 7fd41b400640 1 --2- 172.21.15.63:0/2294680284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd395400 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd408000c00 tx=0x7fd40800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4aabfa8f97a500 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.254+0000 7fd410c00640 1 -- 172.21.15.63:0/2294680284 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40802e040 con 0x556accfc74d0 2024-05-27T07:44:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.254+0000 7fd410c00640 1 -- 172.21.15.63:0/2294680284 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd40800bb00 con 0x556accfc74d0 2024-05-27T07:44:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.254+0000 7fd410c00640 1 -- 172.21.15.63:0/2294680284 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408018cc0 con 0x556accfc74d0 2024-05-27T07:44:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.255+0000 7fd42a206740 1 -- 172.21.15.63:0/2294680284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 msgr2=0x556acd395400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.255+0000 7fd42a206740 1 --2- 172.21.15.63:0/2294680284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd395400 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd408000c00 tx=0x7fd40800f730 comp rx=0 tx=0).stop 2024-05-27T07:44:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.255+0000 7fd42a206740 1 -- 172.21.15.63:0/2294680284 shutdown_connections 2024-05-27T07:44:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.255+0000 7fd42a206740 1 --2- 172.21.15.63:0/2294680284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfe81b0 0x556acd38af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.255+0000 7fd42a206740 1 --2- 172.21.15.63:0/2294680284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfc74d0 0x556acd395400 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.255+0000 7fd42a206740 1 --2- 172.21.15.63:0/2294680284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfbf2f0 0x556acd393d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.255+0000 7fd42a206740 1 -- 172.21.15.63:0/2294680284 >> 172.21.15.63:0/2294680284 conn(0x556accfdc300 msgr2=0x556acd38a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.255+0000 7fd42a206740 1 -- 172.21.15.63:0/2294680284 shutdown_connections 2024-05-27T07:44:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.255+0000 7fd42a206740 1 -- 172.21.15.63:0/2294680284 wait complete. 2024-05-27T07:44:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.256+0000 7fd42a206740 1 -- start start 2024-05-27T07:44:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.256+0000 7fd42a206740 1 Processor -- start 2024-05-27T07:44:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.256+0000 7fd42a206740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfbf2f0 0x556acd399bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.256+0000 7fd42a206740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfc74d0 0x556acd39a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.256+0000 7fd42a206740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfe81b0 0x556acd39a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.256+0000 7fd42a206740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1ad680 con 0x556accfe81b0 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.256+0000 7fd42a206740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1aea90 con 0x556accfc74d0 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.256+0000 7fd42a206740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c1adcf0 con 0x556accfbf2f0 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd42a206740 10 client.0 _ll_register_callbacks cb 0x7fd414009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd421200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfe81b0 0x556acd39a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd421200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfe81b0 0x556acd39a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54580/0 (socket says 172.21.15.63:54580) 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd421200640 1 -- 172.21.15.63:0/3563412683 learned_addr learned my addr 172.21.15.63:0/3563412683 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd41b400640 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfc74d0 0x556acd39a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd41be00640 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfbf2f0 0x556acd399bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd421200640 1 -- 172.21.15.63:0/3563412683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfbf2f0 msgr2=0x556acd399bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd421200640 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfbf2f0 0x556acd399bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd421200640 1 -- 172.21.15.63:0/3563412683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfc74d0 msgr2=0x556acd39a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd421200640 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfc74d0 0x556acd39a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.257+0000 7fd421200640 1 -- 172.21.15.63:0/3563412683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556acd40d190 con 0x556accfe81b0 2024-05-27T07:44:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.258+0000 7fd421200640 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfe81b0 0x556acd39a650 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd414012a80 tx=0x7fd414004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1447c4fb168bb802 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.258+0000 7fd406000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.258+0000 7fd406000640 1 -- 172.21.15.63:0/3563412683 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414021dd0 con 0x556accfe81b0 2024-05-27T07:44:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.258+0000 7fd42a206740 1 -- 172.21.15.63:0/3563412683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd40800bb00 con 0x556accfe81b0 2024-05-27T07:44:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.258+0000 7fd406000640 1 -- 172.21.15.63:0/3563412683 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fd414004b20 con 0x556accfe81b0 2024-05-27T07:44:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd406000640 1 -- 172.21.15.63:0/3563412683 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd414004d20 con 0x556accfe81b0 2024-05-27T07:44:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd42a206740 10 client.17472 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd42a206740 1 -- 172.21.15.63:0/3563412683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556acd40e480 con 0x556accfe81b0 2024-05-27T07:44:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd406000640 1 -- 172.21.15.63:0/3563412683 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd40800bb00 con 0x556accfe81b0 2024-05-27T07:44:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd406000640 1 -- 172.21.15.63:0/3563412683 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd414028070 con 0x556accfe81b0 2024-05-27T07:44:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd42a206740 10 client.17472 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd42a206740 10 client.17472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd42a206740 20 client.17472 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd42a206740 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556accf7c1c0 0x556acd410c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd42a206740 4 client.17472 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:44:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.259+0000 7fd42a206740 1 -- 172.21.15.63:0/3563412683 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556acd2f72c0 con 0x556accf7c1c0 2024-05-27T07:44:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.260+0000 7fd41be00640 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556accf7c1c0 0x556acd410c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.261+0000 7fd41be00640 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556accf7c1c0 0x556acd410c70 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7df55ee4f8a5a2e server_cookie=8feb5e570b522951 in_seq=0 out_seq=0 2024-05-27T07:44:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.261+0000 7fd406000640 10 client.17472 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.298+0000 7fd406000640 1 -- 172.21.15.63:0/3563412683 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556acd2f72c0 con 0x556accf7c1c0 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.298+0000 7fd406000640 10 client.17472 handle_command_reply: tid=0 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.298+0000 7fd42a206740 1 client.17472 shutdown 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd42a206740 1 -- 172.21.15.63:0/3563412683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfe81b0 msgr2=0x556acd39a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd42a206740 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfe81b0 0x556acd39a650 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fd414012a80 tx=0x7fd414004a40 comp rx=0 tx=0).stop 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd42a206740 1 -- 172.21.15.63:0/3563412683 shutdown_connections 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd42a206740 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556accf7c1c0 0x556acd410c70 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd42a206740 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556accfe81b0 0x556acd39a650 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd42a206740 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556accfc74d0 0x556acd39a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd42a206740 1 --2- 172.21.15.63:0/3563412683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556accfbf2f0 0x556acd399bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd42a206740 1 -- 172.21.15.63:0/3563412683 >> 172.21.15.63:0/3563412683 conn(0x556accfdc300 msgr2=0x556acd393a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd406000640 0 client.17472 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd42a206740 1 -- 172.21.15.63:0/3563412683 shutdown_connections 2024-05-27T07:44:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.299+0000 7fd42a206740 1 -- 172.21.15.63:0/3563412683 wait complete. 2024-05-27T07:44:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.300+0000 7fd42a206740 20 client.17472 trim_cache size 0 max 16384 2024-05-27T07:44:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:44:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:44:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:44:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.300+0000 7fd421c00640 1 -- 172.21.15.63:0/3853852992 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4000b2950 msgr2=0x7fd4000b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.300+0000 7fd421c00640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4000b2950 0x7fd4000b4e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x556acd387bd0 tx=0x7fd414009680 comp rx=0 tx=0).stop 2024-05-27T07:44:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.300+0000 7fd421c00640 1 -- 172.21.15.63:0/3853852992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c104bb0 msgr2=0x7fd41c19c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.300+0000 7fd421c00640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c104bb0 0x7fd41c19c7b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009230 tx=0x7fd40c004270 comp rx=0 tx=0).stop 2024-05-27T07:44:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.301+0000 7fd421c00640 1 -- 172.21.15.63:0/3853852992 shutdown_connections 2024-05-27T07:44:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.301+0000 7fd421c00640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4000b2950 0x7fd4000b4e60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.301+0000 7fd421c00640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd41c10e000 0x7fd41c1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.301+0000 7fd421c00640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd41c10b610 0x7fd41c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.301+0000 7fd421c00640 1 --2- 172.21.15.63:0/3853852992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd41c104bb0 0x7fd41c19c7b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.301+0000 7fd421c00640 1 -- 172.21.15.63:0/3853852992 >> 172.21.15.63:0/3853852992 conn(0x7fd41c0fe210 msgr2=0x7fd41c10e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.301+0000 7fd421c00640 1 -- 172.21.15.63:0/3853852992 shutdown_connections 2024-05-27T07:44:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:12.301+0000 7fd421c00640 1 -- 172.21.15.63:0/3853852992 wait complete. 2024-05-27T07:44:13.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:12 smithi086 ceph-mon[34687]: pgmap v331: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.9 MiB/s wr, 9 op/s 2024-05-27T07:44:13.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:12 smithi149 ceph-mon[35408]: pgmap v331: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.9 MiB/s wr, 9 op/s 2024-05-27T07:44:13.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:12 smithi063 ceph-mon[31613]: pgmap v331: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.9 MiB/s wr, 9 op/s 2024-05-27T07:44:15.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:14 smithi086 ceph-mon[34687]: pgmap v332: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.9 MiB/s wr, 9 op/s 2024-05-27T07:44:15.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:14 smithi149 ceph-mon[35408]: pgmap v332: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.9 MiB/s wr, 9 op/s 2024-05-27T07:44:15.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:14 smithi063 ceph-mon[31613]: pgmap v332: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.9 MiB/s wr, 9 op/s 2024-05-27T07:44:16.068 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:15 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:44:16.068 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:15 smithi149 ceph-mon[35408]: osdmap e103: 12 total, 12 up, 12 in 2024-05-27T07:44:16.074 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:15 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:44:16.074 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:15 smithi086 ceph-mon[34687]: osdmap e103: 12 total, 12 up, 12 in 2024-05-27T07:44:16.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:15 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:44:16.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:15 smithi063 ceph-mon[31613]: osdmap e103: 12 total, 12 up, 12 in 2024-05-27T07:44:17.707 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:17 smithi086 ceph-mon[34687]: pgmap v334: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-05-27T07:44:17.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:17 smithi149 ceph-mon[35408]: pgmap v334: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-05-27T07:44:17.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:17 smithi063 ceph-mon[31613]: pgmap v334: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-05-27T07:44:18.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:18 smithi086 ceph-mon[34687]: pgmap v335: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T07:44:18.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:18 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:18 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:18 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:18 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:18 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:18 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:18 smithi149 ceph-mon[35408]: pgmap v335: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T07:44:18.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:18 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:18 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:18 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:18 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:18 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:18 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:18 smithi063 ceph-mon[31613]: pgmap v335: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T07:44:18.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:18 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:18 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:18 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:18 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:18 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:18.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:18 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:19.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T07:44:19.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T07:44:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T07:44:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T07:44:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:44:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:44:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:19.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T07:44:19.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T07:44:19.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T07:44:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T07:44:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:44:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:44:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:19.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T07:44:19.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T07:44:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:44:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T07:44:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T07:44:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:44:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:44:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:44:20.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:20 smithi086 ceph-mon[34687]: pgmap v336: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T07:44:20.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:20 smithi149 ceph-mon[35408]: pgmap v336: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T07:44:20.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:20 smithi063 ceph-mon[31613]: pgmap v336: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T07:44:22.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:22 smithi086 ceph-mon[34687]: pgmap v337: 129 pgs: 129 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 12 op/s 2024-05-27T07:44:22.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:22 smithi149 ceph-mon[35408]: pgmap v337: 129 pgs: 129 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 12 op/s 2024-05-27T07:44:22.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:22 smithi063 ceph-mon[31613]: pgmap v337: 129 pgs: 129 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 12 op/s 2024-05-27T07:44:24.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:24 smithi086 ceph-mon[34687]: pgmap v338: 129 pgs: 129 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 12 op/s 2024-05-27T07:44:24.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:24 smithi149 ceph-mon[35408]: pgmap v338: 129 pgs: 129 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 12 op/s 2024-05-27T07:44:24.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:24 smithi063 ceph-mon[31613]: pgmap v338: 129 pgs: 129 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 12 op/s 2024-05-27T07:44:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.539+0000 7fa6eca00640 1 -- 172.21.15.63:0/2912364260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e8110110 msgr2=0x7fa6e8109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.539+0000 7fa6eca00640 1 --2- 172.21.15.63:0/2912364260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e8110110 0x7fa6e8109ca0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009900 tx=0x7fa6d802a480 comp rx=0 tx=0).stop 2024-05-27T07:44:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.539+0000 7fa6eca00640 1 -- 172.21.15.63:0/2912364260 shutdown_connections 2024-05-27T07:44:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.539+0000 7fa6eca00640 1 --2- 172.21.15.63:0/2912364260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6e810a890 0x7fa6e8116b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.539+0000 7fa6eca00640 1 --2- 172.21.15.63:0/2912364260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6e8110520 0x7fa6e810a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.539+0000 7fa6eca00640 1 --2- 172.21.15.63:0/2912364260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e8110110 0x7fa6e8109ca0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.539+0000 7fa6eca00640 1 -- 172.21.15.63:0/2912364260 >> 172.21.15.63:0/2912364260 conn(0x7fa6e8076070 msgr2=0x7fa6e8078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.539+0000 7fa6eca00640 1 -- 172.21.15.63:0/2912364260 shutdown_connections 2024-05-27T07:44:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.539+0000 7fa6eca00640 1 -- 172.21.15.63:0/2912364260 wait complete. 2024-05-27T07:44:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.540+0000 7fa6eca00640 1 Processor -- start 2024-05-27T07:44:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.540+0000 7fa6eca00640 1 -- start start 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.541+0000 7fa6eca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6e8110110 0x7fa6e81a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.541+0000 7fa6eca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6e8110520 0x7fa6e81a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.541+0000 7fa6eca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e81a4fb0 0x7fa6e81a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.541+0000 7fa6eca00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e811a1d0 con 0x7fa6e81a4fb0 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.541+0000 7fa6eca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e811a050 con 0x7fa6e8110110 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.541+0000 7fa6eca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e811a350 con 0x7fa6e8110520 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.541+0000 7fa6e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e81a4fb0 0x7fa6e81a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.541+0000 7fa6e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e81a4fb0 0x7fa6e81a53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49318/0 (socket says 172.21.15.63:49318) 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.541+0000 7fa6e7e00640 1 -- 172.21.15.63:0/1631622763 learned_addr learned my addr 172.21.15.63:0/1631622763 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.541+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6e8110520 0x7fa6e81a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.542+0000 7fa6e7400640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6e8110110 0x7fa6e81a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.542+0000 7fa6e7e00640 1 -- 172.21.15.63:0/1631622763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6e8110520 msgr2=0x7fa6e81a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.542+0000 7fa6e7e00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6e8110520 0x7fa6e81a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.542+0000 7fa6e7e00640 1 -- 172.21.15.63:0/1631622763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6e8110110 msgr2=0x7fa6e81a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.542+0000 7fa6e7e00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6e8110110 0x7fa6e81a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.542+0000 7fa6e7e00640 1 -- 172.21.15.63:0/1631622763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e81a5a00 con 0x7fa6e81a4fb0 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.542+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6e8110520 0x7fa6e81a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.542+0000 7fa6e7e00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e81a4fb0 0x7fa6e81a53c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000eaa0 tx=0x7fa6e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.543+0000 7fa6dfe00640 1 -- 172.21.15.63:0/1631622763 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e000cdb0 con 0x7fa6e81a4fb0 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.543+0000 7fa6eca00640 1 -- 172.21.15.63:0/1631622763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e81a5cf0 con 0x7fa6e81a4fb0 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.543+0000 7fa6dfe00640 1 -- 172.21.15.63:0/1631622763 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6e0005020 con 0x7fa6e81a4fb0 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.543+0000 7fa6dfe00640 1 -- 172.21.15.63:0/1631622763 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00116d0 con 0x7fa6e81a4fb0 2024-05-27T07:44:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.543+0000 7fa6eca00640 1 -- 172.21.15.63:0/1631622763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e81ad550 con 0x7fa6e81a4fb0 2024-05-27T07:44:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.547+0000 7fa6f5172740 1 Processor -- start 2024-05-27T07:44:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.548+0000 7fa6dfe00640 1 -- 172.21.15.63:0/1631622763 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa6e0004a40 con 0x7fa6e81a4fb0 2024-05-27T07:44:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.548+0000 7fa6f5172740 1 -- start start 2024-05-27T07:44:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.549+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c290b1b0 0x55a2c2cada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.549+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 0x55a2c2cadfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.549+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.549+0000 7fa6f5172740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2c1eb90 con 0x55a2c290b1b0 2024-05-27T07:44:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.549+0000 7fa6f5172740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2c0e730 con 0x55a2c28e22f0 2024-05-27T07:44:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.549+0000 7fa6f5172740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2c12770 con 0x55a2c28ea4d0 2024-05-27T07:44:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.549+0000 7fa6e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c290b1b0 0x55a2c2cada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.549+0000 7fa6e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c290b1b0 0x55a2c2cada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49330/0 (socket says 172.21.15.63:49330) 2024-05-27T07:44:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.549+0000 7fa6e6a00640 1 -- 172.21.15.63:0/4270193800 learned_addr learned my addr 172.21.15.63:0/4270193800 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.549+0000 7fa6e7400640 1 --2- 172.21.15.63:0/4270193800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 0x55a2c2cadfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.550+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/4270193800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.550+0000 7fa6e6a00640 1 -- 172.21.15.63:0/4270193800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28ea4d0 msgr2=0x55a2c2cae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.550+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/4270193800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.550+0000 7fa6e6a00640 1 -- 172.21.15.63:0/4270193800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 msgr2=0x55a2c2cadfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.550+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/4270193800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 0x55a2c2cadfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.550+0000 7fa6e6a00640 1 -- 172.21.15.63:0/4270193800 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c2ba78b0 con 0x55a2c290b1b0 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.550+0000 7fa6dfe00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6cc0b2950 0x7fa6cc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.550+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/4270193800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c290b1b0 0x55a2c2cada60 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400cfd0 tx=0x7fa6d400c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce020b50186498ed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.550+0000 7fa6dfe00640 1 -- 172.21.15.63:0/1631622763 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fa6e00e2bf0 con 0x7fa6e81a4fb0 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.550+0000 7fa6e7400640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6cc0b2950 0x7fa6cc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.551+0000 7fa6d3e00640 1 -- 172.21.15.63:0/4270193800 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4019070 con 0x55a2c290b1b0 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.551+0000 7fa6d3e00640 1 -- 172.21.15.63:0/4270193800 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6d4004230 con 0x55a2c290b1b0 2024-05-27T07:44:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.551+0000 7fa6d3e00640 1 -- 172.21.15.63:0/4270193800 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4011800 con 0x55a2c290b1b0 2024-05-27T07:44:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.552+0000 7fa6f5172740 1 -- 172.21.15.63:0/4270193800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c290b1b0 msgr2=0x55a2c2cada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.552+0000 7fa6f5172740 1 --2- 172.21.15.63:0/4270193800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c290b1b0 0x55a2c2cada60 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400cfd0 tx=0x7fa6d400c720 comp rx=0 tx=0).stop 2024-05-27T07:44:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.552+0000 7fa6e7400640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6cc0b2950 0x7fa6cc0b4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80057c0 tx=0x7fa6d8005a20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.552+0000 7fa6f5172740 1 -- 172.21.15.63:0/4270193800 shutdown_connections 2024-05-27T07:44:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.552+0000 7fa6f5172740 1 --2- 172.21.15.63:0/4270193800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.552+0000 7fa6f5172740 1 --2- 172.21.15.63:0/4270193800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 0x55a2c2cadfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.552+0000 7fa6f5172740 1 --2- 172.21.15.63:0/4270193800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c290b1b0 0x55a2c2cada60 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.552+0000 7fa6f5172740 1 -- 172.21.15.63:0/4270193800 >> 172.21.15.63:0/4270193800 conn(0x55a2c28ff300 msgr2=0x55a2c2caa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.552+0000 7fa6f5172740 1 -- 172.21.15.63:0/4270193800 shutdown_connections 2024-05-27T07:44:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.552+0000 7fa6f5172740 1 -- 172.21.15.63:0/4270193800 wait complete. 2024-05-27T07:44:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.553+0000 7fa6f5172740 1 -- start start 2024-05-27T07:44:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.553+0000 7fa6f5172740 1 Processor -- start 2024-05-27T07:44:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28e22f0 0x55a2c2cb7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cb84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 0x55a2c2cc3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6f5172740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a2c28f4790 con 0x55a2c28ea4d0 2024-05-27T07:44:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6f5172740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2bb6bc0 con 0x55a2c290b1b0 2024-05-27T07:44:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6f5172740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a2c2cbab90 con 0x55a2c28e22f0 2024-05-27T07:44:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6f5172740 10 client.0 _ll_register_callbacks cb 0x55a2c2b94400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cb84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cb84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49340/0 (socket says 172.21.15.63:49340) 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6e6a00640 1 -- 172.21.15.63:0/3758425909 learned_addr learned my addr 172.21.15.63:0/3758425909 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6e7400640 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28e22f0 0x55a2c2cb7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.554+0000 7fa6e7e00640 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 0x55a2c2cc3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6e6a00640 1 -- 172.21.15.63:0/3758425909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28e22f0 msgr2=0x55a2c2cb7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28e22f0 0x55a2c2cb7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6e6a00640 1 -- 172.21.15.63:0/3758425909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 msgr2=0x55a2c2cc3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 0x55a2c2cc3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6e6a00640 1 -- 172.21.15.63:0/3758425909 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c2923210 con 0x55a2c28ea4d0 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6e7e00640 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 0x55a2c2cc3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cb84a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400c2f0 tx=0x7fa6d400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c58b717b756a414 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6d0c00640 1 -- 172.21.15.63:0/3758425909 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4019070 con 0x55a2c28ea4d0 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6d0c00640 1 -- 172.21.15.63:0/3758425909 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fa6d4005040 con 0x55a2c28ea4d0 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.555+0000 7fa6f5172740 1 -- 172.21.15.63:0/3758425909 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6d4004230 con 0x55a2c28ea4d0 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.556+0000 7fa6d0c00640 1 -- 172.21.15.63:0/3758425909 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa6d402f050 con 0x55a2c28ea4d0 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.556+0000 7fa6f5172740 10 client.17484 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.556+0000 7fa6f5172740 1 -- 172.21.15.63:0/3758425909 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a2c29ef620 con 0x55a2c28ea4d0 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.556+0000 7fa6d0c00640 1 -- 172.21.15.63:0/3758425909 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6d4004230 con 0x55a2c28ea4d0 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.556+0000 7fa6d0c00640 1 -- 172.21.15.63:0/3758425909 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa6d4004c70 con 0x55a2c28ea4d0 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.556+0000 7fa6f5172740 10 client.17484 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.556+0000 7fa6f5172740 10 client.17484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.556+0000 7fa6f5172740 20 client.17484 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.556+0000 7fa6f5172740 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a2c28c9d80 0x55a2c2cc9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.557+0000 7fa6f5172740 4 client.17484 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:44:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.557+0000 7fa6f5172740 1 -- 172.21.15.63:0/3758425909 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2c2c20ae0 con 0x55a2c28c9d80 2024-05-27T07:44:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.557+0000 7fa6e7400640 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a2c28c9d80 0x55a2c2cc9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.558+0000 7fa6e7400640 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a2c28c9d80 0x55a2c2cc9e60 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5537e74bbd861dfb server_cookie=f158544cf6af8ef4 in_seq=0 out_seq=0 2024-05-27T07:44:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.558+0000 7fa6d0c00640 10 client.17484 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.560+0000 7fa6d0c00640 1 -- 172.21.15.63:0/3758425909 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a2c2c20ae0 con 0x55a2c28c9d80 2024-05-27T07:44:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.560+0000 7fa6d0c00640 10 client.17484 handle_command_reply: tid=0 2024-05-27T07:44:25.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.560+0000 7fa6f5172740 1 client.17484 shutdown 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 1 -- 172.21.15.63:0/3758425909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 msgr2=0x55a2c2cb84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cb84a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400c2f0 tx=0x7fa6d400cf00 comp rx=0 tx=0).stop 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 1 -- 172.21.15.63:0/3758425909 shutdown_connections 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a2c28c9d80 0x55a2c2cc9e60 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 0x55a2c2cc3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cb84a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 1 --2- 172.21.15.63:0/3758425909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28e22f0 0x55a2c2cb7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6d0c00640 0 client.17484 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 1 -- 172.21.15.63:0/3758425909 >> 172.21.15.63:0/3758425909 conn(0x55a2c28ff300 msgr2=0x55a2c2cad830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 1 -- 172.21.15.63:0/3758425909 shutdown_connections 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 1 -- 172.21.15.63:0/3758425909 wait complete. 2024-05-27T07:44:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.561+0000 7fa6f5172740 20 client.17484 trim_cache size 0 max 16384 2024-05-27T07:44:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.581+0000 7fa6f5172740 1 Processor -- start 2024-05-27T07:44:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.581+0000 7fa6f5172740 1 -- start start 2024-05-27T07:44:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 0x55a2c2cb3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cc3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c290b1b0 0x55a2c2cc4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6f5172740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81b9ec0 con 0x55a2c28ea4d0 2024-05-27T07:44:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6f5172740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81b7790 con 0x55a2c28e22f0 2024-05-27T07:44:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6f5172740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81b5600 con 0x55a2c290b1b0 2024-05-27T07:44:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cc3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cc3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49356/0 (socket says 172.21.15.63:49356) 2024-05-27T07:44:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e6a00640 1 -- 172.21.15.63:0/1346189590 learned_addr learned my addr 172.21.15.63:0/1346189590 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e7400640 1 --2- 172.21.15.63:0/1346189590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 0x55a2c2cb3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e6a00640 1 -- 172.21.15.63:0/1346189590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c290b1b0 msgr2=0x55a2c2cc4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e7e00640 1 --2- 172.21.15.63:0/1346189590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c290b1b0 0x55a2c2cc4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/1346189590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c290b1b0 0x55a2c2cc4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e6a00640 1 -- 172.21.15.63:0/1346189590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 msgr2=0x55a2c2cb3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/1346189590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 0x55a2c2cb3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e6a00640 1 -- 172.21.15.63:0/1346189590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c2cae960 con 0x55a2c28ea4d0 2024-05-27T07:44:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.582+0000 7fa6e7400640 1 --2- 172.21.15.63:0/1346189590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 0x55a2c2cb3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.583+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/1346189590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cc3c30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400ca10 tx=0x7fa6d4013460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afc0112819d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.583+0000 7fa6d3400640 1 -- 172.21.15.63:0/1346189590 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4019050 con 0x55a2c28ea4d0 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.583+0000 7fa6d3400640 1 -- 172.21.15.63:0/1346189590 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6d402b070 con 0x55a2c28ea4d0 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.583+0000 7fa6d3400640 1 -- 172.21.15.63:0/1346189590 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4005330 con 0x55a2c28ea4d0 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.584+0000 7fa6f5172740 1 -- 172.21.15.63:0/1346189590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 msgr2=0x55a2c2cc3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.584+0000 7fa6f5172740 1 --2- 172.21.15.63:0/1346189590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cc3c30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400ca10 tx=0x7fa6d4013460 comp rx=0 tx=0).stop 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.584+0000 7fa6f5172740 1 -- 172.21.15.63:0/1346189590 shutdown_connections 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.584+0000 7fa6f5172740 1 --2- 172.21.15.63:0/1346189590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c290b1b0 0x55a2c2cc4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.584+0000 7fa6f5172740 1 --2- 172.21.15.63:0/1346189590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cc3c30 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.584+0000 7fa6f5172740 1 --2- 172.21.15.63:0/1346189590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c28e22f0 0x55a2c2cb3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.584+0000 7fa6f5172740 1 -- 172.21.15.63:0/1346189590 >> 172.21.15.63:0/1346189590 conn(0x55a2c28ff300 msgr2=0x55a2c2cb9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.585+0000 7fa6f5172740 1 -- 172.21.15.63:0/1346189590 shutdown_connections 2024-05-27T07:44:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.585+0000 7fa6f5172740 1 -- 172.21.15.63:0/1346189590 wait complete. 2024-05-27T07:44:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.585+0000 7fa6f5172740 1 -- start start 2024-05-27T07:44:25.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.585+0000 7fa6f5172740 1 Processor -- start 2024-05-27T07:44:25.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.586+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28e22f0 0x55a2c2cbce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.586+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cbd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.586+0000 7fa6f5172740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 0x55a2c2cbd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.586+0000 7fa6f5172740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81b5c50 con 0x55a2c28ea4d0 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.586+0000 7fa6f5172740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81b70c0 con 0x55a2c290b1b0 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.586+0000 7fa6f5172740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e81b6320 con 0x55a2c28e22f0 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.586+0000 7fa6f5172740 10 client.0 _ll_register_callbacks cb 0x7fa6d40134f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.586+0000 7fa6e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cbd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.586+0000 7fa6e6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cbd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49360/0 (socket says 172.21.15.63:49360) 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.586+0000 7fa6e6a00640 1 -- 172.21.15.63:0/808181939 learned_addr learned my addr 172.21.15.63:0/808181939 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.587+0000 7fa6e7400640 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28e22f0 0x55a2c2cbce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.587+0000 7fa6e7e00640 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 0x55a2c2cbd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.587+0000 7fa6e6a00640 1 -- 172.21.15.63:0/808181939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28e22f0 msgr2=0x55a2c2cbce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.587+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28e22f0 0x55a2c2cbce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.587+0000 7fa6e6a00640 1 -- 172.21.15.63:0/808181939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 msgr2=0x55a2c2cbd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.587+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 0x55a2c2cbd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.587+0000 7fa6e6a00640 1 -- 172.21.15.63:0/808181939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c2d305e0 con 0x55a2c28ea4d0 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.587+0000 7fa6e6a00640 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cbd3b0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4004630 tx=0x7fa6d4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd215e0f5ed2750 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.587+0000 7fa6d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.588+0000 7fa6d0c00640 1 -- 172.21.15.63:0/808181939 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4040060 con 0x55a2c28ea4d0 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.588+0000 7fa6d0c00640 1 -- 172.21.15.63:0/808181939 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fa6d4011e40 con 0x55a2c28ea4d0 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.588+0000 7fa6f5172740 1 -- 172.21.15.63:0/808181939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6d402b070 con 0x55a2c28ea4d0 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.588+0000 7fa6d0c00640 1 -- 172.21.15.63:0/808181939 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa6d4047050 con 0x55a2c28ea4d0 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.588+0000 7fa6f5172740 10 client.17490 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.588+0000 7fa6f5172740 1 -- 172.21.15.63:0/808181939 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a2c2d317e0 con 0x55a2c28ea4d0 2024-05-27T07:44:25.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.588+0000 7fa6d0c00640 1 -- 172.21.15.63:0/808181939 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6d402b070 con 0x55a2c28ea4d0 2024-05-27T07:44:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.589+0000 7fa6d0c00640 1 -- 172.21.15.63:0/808181939 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa6d4004680 con 0x55a2c28ea4d0 2024-05-27T07:44:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.589+0000 7fa6f5172740 10 client.17490 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.589+0000 7fa6f5172740 10 client.17490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.589+0000 7fa6f5172740 20 client.17490 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.589+0000 7fa6f5172740 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a2c289f1c0 0x55a2c2d33ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.589+0000 7fa6f5172740 4 client.17490 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:44:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.589+0000 7fa6f5172740 1 -- 172.21.15.63:0/808181939 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2c2c1a2c0 con 0x55a2c289f1c0 2024-05-27T07:44:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.589+0000 7fa6e7400640 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a2c289f1c0 0x55a2c2d33ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.590+0000 7fa6e7400640 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a2c289f1c0 0x55a2c2d33ef0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f1af925ab8332d server_cookie=46da968ad639b2cf in_seq=0 out_seq=0 2024-05-27T07:44:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.590+0000 7fa6d0c00640 10 client.17490 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.628+0000 7fa6d0c00640 1 -- 172.21.15.63:0/808181939 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a2c2c1a2c0 con 0x55a2c289f1c0 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.628+0000 7fa6d0c00640 10 client.17490 handle_command_reply: tid=0 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.628+0000 7fa6f5172740 1 client.17490 shutdown 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 1 -- 172.21.15.63:0/808181939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 msgr2=0x55a2c2cbd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cbd3b0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4004630 tx=0x7fa6d4011ac0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 1 -- 172.21.15.63:0/808181939 shutdown_connections 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a2c289f1c0 0x55a2c2d33ef0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2c290b1b0 0x55a2c2cbd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a2c28ea4d0 0x55a2c2cbd3b0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 1 --2- 172.21.15.63:0/808181939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a2c28e22f0 0x55a2c2cbce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 1 -- 172.21.15.63:0/808181939 >> 172.21.15.63:0/808181939 conn(0x55a2c28ff300 msgr2=0x55a2c2cb2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6d0c00640 0 client.17490 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 1 -- 172.21.15.63:0/808181939 shutdown_connections 2024-05-27T07:44:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 1 -- 172.21.15.63:0/808181939 wait complete. 2024-05-27T07:44:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.629+0000 7fa6f5172740 20 client.17490 trim_cache size 0 max 16384 2024-05-27T07:44:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:44:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:44:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:44:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.630+0000 7fa6eca00640 1 -- 172.21.15.63:0/1631622763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6cc0b2950 msgr2=0x7fa6cc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.630+0000 7fa6eca00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6cc0b2950 0x7fa6cc0b4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80057c0 tx=0x7fa6d8005a20 comp rx=0 tx=0).stop 2024-05-27T07:44:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.630+0000 7fa6eca00640 1 -- 172.21.15.63:0/1631622763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e81a4fb0 msgr2=0x7fa6e81a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.630+0000 7fa6eca00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e81a4fb0 0x7fa6e81a53c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000eaa0 tx=0x7fa6e000ef60 comp rx=0 tx=0).stop 2024-05-27T07:44:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.631+0000 7fa6eca00640 1 -- 172.21.15.63:0/1631622763 shutdown_connections 2024-05-27T07:44:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.631+0000 7fa6eca00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6cc0b2950 0x7fa6cc0b4e60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.631+0000 7fa6eca00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e81a4fb0 0x7fa6e81a53c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.631+0000 7fa6eca00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6e8110520 0x7fa6e81a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.631+0000 7fa6eca00640 1 --2- 172.21.15.63:0/1631622763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6e8110110 0x7fa6e81a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.631+0000 7fa6eca00640 1 -- 172.21.15.63:0/1631622763 >> 172.21.15.63:0/1631622763 conn(0x7fa6e8076070 msgr2=0x7fa6e8115890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.631+0000 7fa6eca00640 1 -- 172.21.15.63:0/1631622763 shutdown_connections 2024-05-27T07:44:25.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:25.631+0000 7fa6eca00640 1 -- 172.21.15.63:0/1631622763 wait complete. 2024-05-27T07:44:26.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:26 smithi086 ceph-mon[34687]: pgmap v339: 129 pgs: 129 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 12 op/s 2024-05-27T07:44:26.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:26 smithi149 ceph-mon[35408]: pgmap v339: 129 pgs: 129 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 12 op/s 2024-05-27T07:44:26.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:26 smithi063 ceph-mon[31613]: pgmap v339: 129 pgs: 129 active+clean; 2.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 12 op/s 2024-05-27T07:44:28.737 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:44:28.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:28 smithi086 ceph-mon[34687]: pgmap v340: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 17 op/s 2024-05-27T07:44:28.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:28 smithi149 ceph-mon[35408]: pgmap v340: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 17 op/s 2024-05-27T07:44:28.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:28 smithi063 ceph-mon[31613]: pgmap v340: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 17 op/s 2024-05-27T07:44:29.262 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:30.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:30 smithi086 ceph-mon[34687]: pgmap v341: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:44:30.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:30 smithi149 ceph-mon[35408]: pgmap v341: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:44:30.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:30 smithi063 ceph-mon[31613]: pgmap v341: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:44:31.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.646+0000 7f7e7f000700 1 -- 172.21.15.63:0/2065606121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e78103a40 msgr2=0x7f7e78110760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.646+0000 7f7e7f000700 1 --2- 172.21.15.63:0/2065606121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e78103a40 0x7f7e78110760 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009990 tx=0x7f7e64009ca0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.647+0000 7f7e7f000700 1 -- 172.21.15.63:0/2065606121 shutdown_connections 2024-05-27T07:44:31.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.647+0000 7f7e7f000700 1 --2- 172.21.15.63:0/2065606121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e78110ca0 0x7f7e781130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.647+0000 7f7e7f000700 1 --2- 172.21.15.63:0/2065606121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e781030f0 0x7f7e78103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.647+0000 7f7e7f000700 1 --2- 172.21.15.63:0/2065606121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e78103a40 0x7f7e78110760 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.647+0000 7f7e7f000700 1 -- 172.21.15.63:0/2065606121 >> 172.21.15.63:0/2065606121 conn(0x7f7e780fedb0 msgr2=0x7f7e781011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:31.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.647+0000 7f7e7f000700 1 -- 172.21.15.63:0/2065606121 shutdown_connections 2024-05-27T07:44:31.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.648+0000 7f7e7f000700 1 -- 172.21.15.63:0/2065606121 wait complete. 2024-05-27T07:44:31.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.649+0000 7f7e7f000700 1 Processor -- start 2024-05-27T07:44:31.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.650+0000 7f7e7f000700 1 -- start start 2024-05-27T07:44:31.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.650+0000 7f7e7f000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e781030f0 0x7f7e7819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.651+0000 7f7e7f000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e78103a40 0x7f7e7819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.651+0000 7f7e7f000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78110ca0 0x7f7e781a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.651+0000 7f7e7f000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e78116550 con 0x7f7e78103a40 2024-05-27T07:44:31.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.651+0000 7f7e7f000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e781163d0 con 0x7f7e781030f0 2024-05-27T07:44:31.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.651+0000 7f7e7f000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e781166d0 con 0x7f7e78110ca0 2024-05-27T07:44:31.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.651+0000 7f7e77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e781030f0 0x7f7e7819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.651+0000 7f7e77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e781030f0 0x7f7e7819cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56020/0 (socket says 172.21.15.63:56020) 2024-05-27T07:44:31.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.651+0000 7f7e77e00700 1 -- 172.21.15.63:0/4160677164 learned_addr learned my addr 172.21.15.63:0/4160677164 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:31.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.651+0000 7f7e77400700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e78103a40 0x7f7e7819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.652+0000 7f7e7d000700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78110ca0 0x7f7e781a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.652+0000 7f7e77e00700 1 -- 172.21.15.63:0/4160677164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78110ca0 msgr2=0x7f7e781a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.652+0000 7f7e77e00700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78110ca0 0x7f7e781a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.652+0000 7f7e77e00700 1 -- 172.21.15.63:0/4160677164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e78103a40 msgr2=0x7f7e7819d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.652+0000 7f7e77e00700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e78103a40 0x7f7e7819d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.652+0000 7f7e77e00700 1 -- 172.21.15.63:0/4160677164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e781a1e00 con 0x7f7e781030f0 2024-05-27T07:44:31.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.653+0000 7f7e77e00700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e781030f0 0x7f7e7819cf80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d950 tx=0x7f7e6800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:31.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.653+0000 7f7e74c00700 1 -- 172.21.15.63:0/4160677164 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68016070 con 0x7f7e781030f0 2024-05-27T07:44:31.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.653+0000 7f7e7f000700 1 -- 172.21.15.63:0/4160677164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e781a20f0 con 0x7f7e781030f0 2024-05-27T07:44:31.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.653+0000 7f7e7f000700 1 -- 172.21.15.63:0/4160677164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e781a9a10 con 0x7f7e781030f0 2024-05-27T07:44:31.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.654+0000 7f7e74c00700 1 -- 172.21.15.63:0/4160677164 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e68009dd0 con 0x7f7e781030f0 2024-05-27T07:44:31.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.654+0000 7f7e74c00700 1 -- 172.21.15.63:0/4160677164 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68020060 con 0x7f7e781030f0 2024-05-27T07:44:31.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.660+0000 7f7e74c00700 1 -- 172.21.15.63:0/4160677164 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7e68019070 con 0x7f7e781030f0 2024-05-27T07:44:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.661+0000 7f7e8dc32b80 1 Processor -- start 2024-05-27T07:44:31.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.661+0000 7f7e8dc32b80 1 -- start start 2024-05-27T07:44:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.662+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a627ee40 0x5615a64037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.662+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a6403d30 0x5615a63fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.662+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a63fddb0 0x5615a63fe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.662+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5615a632be40 con 0x5615a6403d30 2024-05-27T07:44:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.662+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5615a631b9e0 con 0x5615a63fddb0 2024-05-27T07:44:31.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.662+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5615a631fa20 con 0x5615a627ee40 2024-05-27T07:44:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.662+0000 7f7e77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a6403d30 0x5615a63fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.662+0000 7f7e74c00700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7e5c0b2a70 0x7f7e5c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.662+0000 7f7e77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a6403d30 0x5615a63fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44934/0 (socket says 172.21.15.63:44934) 2024-05-27T07:44:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.663+0000 7f7e77e00700 1 -- 172.21.15.63:0/3155396690 learned_addr learned my addr 172.21.15.63:0/3155396690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.662+0000 7f7e7d000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a627ee40 0x5615a64037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.663+0000 7f7e77400700 1 --2- 172.21.15.63:0/3155396690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a63fddb0 0x5615a63fe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.663+0000 7f7e74c00700 1 -- 172.21.15.63:0/4160677164 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f7e680e2520 con 0x7f7e781030f0 2024-05-27T07:44:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.663+0000 7f7e77e00700 1 -- 172.21.15.63:0/3155396690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a627ee40 msgr2=0x5615a64037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.663+0000 7f7e77e00700 1 --2- 172.21.15.63:0/3155396690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a627ee40 0x5615a64037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.663+0000 7f7e77e00700 1 -- 172.21.15.63:0/3155396690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a63fddb0 msgr2=0x5615a63fe1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.663+0000 7f7e77e00700 1 --2- 172.21.15.63:0/3155396690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a63fddb0 0x5615a63fe1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.663+0000 7f7e77e00700 1 -- 172.21.15.63:0/3155396690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615a62e4420 con 0x5615a6403d30 2024-05-27T07:44:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.663+0000 7f7e7d000700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7e5c0b2a70 0x7f7e5c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.664+0000 7f7e77e00700 1 --2- 172.21.15.63:0/3155396690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a6403d30 0x5615a63fd870 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7e680df7c0 tx=0x7f7e680a8bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9decf7e9eb31cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.664+0000 7f7e6d600700 1 -- 172.21.15.63:0/3155396690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680a8ef0 con 0x5615a6403d30 2024-05-27T07:44:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.665+0000 7f7e7d000700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7e5c0b2a70 0x7f7e5c0b4f80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5615a63feb30 tx=0x7f7e70004b40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.665+0000 7f7e6d600700 1 -- 172.21.15.63:0/3155396690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e680a9ae0 con 0x5615a6403d30 2024-05-27T07:44:31.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.665+0000 7f7e6d600700 1 -- 172.21.15.63:0/3155396690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680a8ef0 con 0x5615a6403d30 2024-05-27T07:44:31.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.666+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/3155396690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a6403d30 msgr2=0x5615a63fd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.666+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/3155396690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a6403d30 0x5615a63fd870 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7e680df7c0 tx=0x7f7e680a8bd0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.666+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/3155396690 shutdown_connections 2024-05-27T07:44:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.666+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/3155396690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a63fddb0 0x5615a63fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.667+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/3155396690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a627ee40 0x5615a64037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.667+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/3155396690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a6403d30 0x5615a63fd870 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.667+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/3155396690 >> 172.21.15.63:0/3155396690 conn(0x5615a62defc0 msgr2=0x5615a63f88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.667+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/3155396690 shutdown_connections 2024-05-27T07:44:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.667+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/3155396690 wait complete. 2024-05-27T07:44:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.667+0000 7f7e8dc32b80 1 -- start start 2024-05-27T07:44:31.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.668+0000 7f7e8dc32b80 1 Processor -- start 2024-05-27T07:44:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.668+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6407490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.668+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a63fddb0 0x5615a64079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.668+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a6403d30 0x5615a6407f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.668+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5615a6221fa0 con 0x5615a627ee40 2024-05-27T07:44:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.668+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5615a62bc760 con 0x5615a63fddb0 2024-05-27T07:44:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.668+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5615a640a800 con 0x5615a6403d30 2024-05-27T07:44:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e8dc32b80 10 client.0 _ll_register_callbacks cb 0x5615a62c5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6407490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6407490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44936/0 (socket says 172.21.15.63:44936) 2024-05-27T07:44:31.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e77e00700 1 -- 172.21.15.63:0/1241610826 learned_addr learned my addr 172.21.15.63:0/1241610826 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e77400700 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a63fddb0 0x5615a64079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e7d000700 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a6403d30 0x5615a6407f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e77e00700 1 -- 172.21.15.63:0/1241610826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a6403d30 msgr2=0x5615a6407f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:31.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e77e00700 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a6403d30 0x5615a6407f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e77e00700 1 -- 172.21.15.63:0/1241610826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a63fddb0 msgr2=0x5615a64079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e77e00700 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a63fddb0 0x5615a64079d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.669+0000 7f7e77e00700 1 -- 172.21.15.63:0/1241610826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615a626b8d0 con 0x5615a627ee40 2024-05-27T07:44:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.670+0000 7f7e77e00700 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6407490 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7e680a90e0 tx=0x7f7e680b22c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c6c8578dfd7877 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.670+0000 7f7e62000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:31.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.670+0000 7f7e62000700 1 -- 172.21.15.63:0/1241610826 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680a9850 con 0x5615a627ee40 2024-05-27T07:44:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.671+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1241610826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e680a9ae0 con 0x5615a627ee40 2024-05-27T07:44:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.671+0000 7f7e62000700 1 -- 172.21.15.63:0/1241610826 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f7e680e2170 con 0x5615a627ee40 2024-05-27T07:44:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.671+0000 7f7e62000700 1 -- 172.21.15.63:0/1241610826 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7e680b1db0 con 0x5615a627ee40 2024-05-27T07:44:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.671+0000 7f7e8dc32b80 10 client.17502 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.671+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1241610826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5615a62ec060 con 0x5615a627ee40 2024-05-27T07:44:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.671+0000 7f7e62000700 1 -- 172.21.15.63:0/1241610826 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e680a9ae0 con 0x5615a627ee40 2024-05-27T07:44:31.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.672+0000 7f7e62000700 1 -- 172.21.15.63:0/1241610826 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7e680a3b00 con 0x5615a627ee40 2024-05-27T07:44:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.672+0000 7f7e8dc32b80 10 client.17502 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.672+0000 7f7e8dc32b80 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.672+0000 7f7e8dc32b80 20 client.17502 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.672+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5615a641fd80 0x5615a64221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.672+0000 7f7e8dc32b80 4 client.17502 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:44:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.672+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1241610826 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615a632dd90 con 0x5615a641fd80 2024-05-27T07:44:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.673+0000 7f7e77400700 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5615a641fd80 0x5615a64221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.674+0000 7f7e77400700 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5615a641fd80 0x5615a64221a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68bb5107209db4b2 server_cookie=b069202dc9dd9971 in_seq=0 out_seq=0 2024-05-27T07:44:31.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.674+0000 7f7e62000700 10 client.17502 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:31.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e62000700 1 -- 172.21.15.63:0/1241610826 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5615a632dd90 con 0x5615a641fd80 2024-05-27T07:44:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e62000700 10 client.17502 handle_command_reply: tid=0 2024-05-27T07:44:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e8dc32b80 1 client.17502 shutdown 2024-05-27T07:44:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1241610826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 msgr2=0x5615a6407490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6407490 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7e680a90e0 tx=0x7f7e680b22c0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1241610826 shutdown_connections 2024-05-27T07:44:31.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5615a641fd80 0x5615a64221a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a6403d30 0x5615a6407f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6407490 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/1241610826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a63fddb0 0x5615a64079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1241610826 >> 172.21.15.63:0/1241610826 conn(0x5615a62defc0 msgr2=0x5615a63f7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:31.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.676+0000 7f7e62000700 0 client.17502 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:31.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.677+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1241610826 shutdown_connections 2024-05-27T07:44:31.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.677+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1241610826 wait complete. 2024-05-27T07:44:31.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.677+0000 7f7e8dc32b80 20 client.17502 trim_cache size 0 max 16384 2024-05-27T07:44:31.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.706+0000 7f7e8dc32b80 1 Processor -- start 2024-05-27T07:44:31.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.706+0000 7f7e8dc32b80 1 -- start start 2024-05-27T07:44:31.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.707+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a627ee40 0x5615a63fd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.707+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a63fddb0 0x5615a641c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.707+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a6403d30 0x5615a63f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.707+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e58011610 con 0x5615a63fddb0 2024-05-27T07:44:31.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.707+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5800eea0 con 0x5615a627ee40 2024-05-27T07:44:31.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.707+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5800cd10 con 0x5615a6403d30 2024-05-27T07:44:31.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.707+0000 7f7e77400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a63fddb0 0x5615a641c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.707+0000 7f7e77400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a63fddb0 0x5615a641c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44944/0 (socket says 172.21.15.63:44944) 2024-05-27T07:44:31.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.707+0000 7f7e77400700 1 -- 172.21.15.63:0/1951875372 learned_addr learned my addr 172.21.15.63:0/1951875372 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.708+0000 7f7e77e00700 1 --2- 172.21.15.63:0/1951875372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a627ee40 0x5615a63fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.708+0000 7f7e7d000700 1 --2- 172.21.15.63:0/1951875372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a6403d30 0x5615a63f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.708+0000 7f7e77e00700 1 -- 172.21.15.63:0/1951875372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a6403d30 msgr2=0x5615a63f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.708+0000 7f7e77e00700 1 --2- 172.21.15.63:0/1951875372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a6403d30 0x5615a63f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.708+0000 7f7e77e00700 1 -- 172.21.15.63:0/1951875372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a63fddb0 msgr2=0x5615a641c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.708+0000 7f7e77e00700 1 --2- 172.21.15.63:0/1951875372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a63fddb0 0x5615a641c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.708+0000 7f7e77e00700 1 -- 172.21.15.63:0/1951875372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615a63fa030 con 0x5615a627ee40 2024-05-27T07:44:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.708+0000 7f7e77400700 1 --2- 172.21.15.63:0/1951875372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a63fddb0 0x5615a641c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:44:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.709+0000 7f7e77e00700 1 --2- 172.21.15.63:0/1951875372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a627ee40 0x5615a63fd870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e680a8cc0 tx=0x7f7e680a8bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee14b8026c2a3e92 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.710+0000 7f7e6cc00700 1 -- 172.21.15.63:0/1951875372 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680a9850 con 0x5615a627ee40 2024-05-27T07:44:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.710+0000 7f7e6cc00700 1 -- 172.21.15.63:0/1951875372 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e680b1f50 con 0x5615a627ee40 2024-05-27T07:44:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.710+0000 7f7e6cc00700 1 -- 172.21.15.63:0/1951875372 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680a8640 con 0x5615a627ee40 2024-05-27T07:44:31.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.711+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1951875372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a627ee40 msgr2=0x5615a63fd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.711+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/1951875372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a627ee40 0x5615a63fd870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e680a8cc0 tx=0x7f7e680a8bd0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.711+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1951875372 shutdown_connections 2024-05-27T07:44:31.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.711+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/1951875372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a6403d30 0x5615a63f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.711+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/1951875372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a627ee40 0x5615a63fd870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.711+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/1951875372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a63fddb0 0x5615a641c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.711+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1951875372 >> 172.21.15.63:0/1951875372 conn(0x5615a62defc0 msgr2=0x5615a6402c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.711+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1951875372 shutdown_connections 2024-05-27T07:44:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.711+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/1951875372 wait complete. 2024-05-27T07:44:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.712+0000 7f7e8dc32b80 1 -- start start 2024-05-27T07:44:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.712+0000 7f7e8dc32b80 1 Processor -- start 2024-05-27T07:44:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.713+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6425a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.713+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a63fddb0 0x5615a6422de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.713+0000 7f7e8dc32b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a6403d30 0x5615a6423480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.713+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5800d360 con 0x5615a627ee40 2024-05-27T07:44:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.713+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5800e7d0 con 0x5615a6403d30 2024-05-27T07:44:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.713+0000 7f7e8dc32b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5800da30 con 0x5615a63fddb0 2024-05-27T07:44:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.713+0000 7f7e8dc32b80 10 client.0 _ll_register_callbacks cb 0x5615a62c5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:31.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.713+0000 7f7e77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6425a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.713+0000 7f7e77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6425a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44958/0 (socket says 172.21.15.63:44958) 2024-05-27T07:44:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.713+0000 7f7e77e00700 1 -- 172.21.15.63:0/828107794 learned_addr learned my addr 172.21.15.63:0/828107794 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.714+0000 7f7e77400700 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a63fddb0 0x5615a6422de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.714+0000 7f7e7d000700 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a6403d30 0x5615a6423480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.714+0000 7f7e77e00700 1 -- 172.21.15.63:0/828107794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a63fddb0 msgr2=0x5615a6422de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.714+0000 7f7e77e00700 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a63fddb0 0x5615a6422de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.714+0000 7f7e77e00700 1 -- 172.21.15.63:0/828107794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a6403d30 msgr2=0x5615a6423480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.714+0000 7f7e77e00700 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a6403d30 0x5615a6423480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.714+0000 7f7e77e00700 1 -- 172.21.15.63:0/828107794 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615a6423c00 con 0x5615a627ee40 2024-05-27T07:44:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.715+0000 7f7e77e00700 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6425a60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7e680a9370 tx=0x7f7e680b2100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4937de67c0f6921e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.715+0000 7f7e62000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.715+0000 7f7e62000700 1 -- 172.21.15.63:0/828107794 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680c7070 con 0x5615a627ee40 2024-05-27T07:44:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.715+0000 7f7e62000700 1 -- 172.21.15.63:0/828107794 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f7e68009940 con 0x5615a627ee40 2024-05-27T07:44:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.715+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/828107794 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e680b1f50 con 0x5615a627ee40 2024-05-27T07:44:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.716+0000 7f7e62000700 1 -- 172.21.15.63:0/828107794 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7e680a3d10 con 0x5615a627ee40 2024-05-27T07:44:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.716+0000 7f7e8dc32b80 10 client.17508 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.716+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/828107794 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5615a6423c00 con 0x5615a627ee40 2024-05-27T07:44:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.716+0000 7f7e62000700 1 -- 172.21.15.63:0/828107794 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e680b1f50 con 0x5615a627ee40 2024-05-27T07:44:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.716+0000 7f7e62000700 1 -- 172.21.15.63:0/828107794 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7e680ca020 con 0x5615a627ee40 2024-05-27T07:44:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.716+0000 7f7e8dc32b80 10 client.17508 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.716+0000 7f7e8dc32b80 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.716+0000 7f7e8dc32b80 20 client.17508 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.716+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5615a648a920 0x5615a648cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.716+0000 7f7e8dc32b80 4 client.17508 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:44:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.717+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/828107794 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615a6327570 con 0x5615a648a920 2024-05-27T07:44:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.717+0000 7f7e77400700 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5615a648a920 0x5615a648cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.718+0000 7f7e77400700 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5615a648a920 0x5615a648cd40 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c80fbedbe350248 server_cookie=33c7df0b3eea143d in_seq=0 out_seq=0 2024-05-27T07:44:31.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.718+0000 7f7e62000700 10 client.17508 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:31.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.719+0000 7f7e62000700 1 -- 172.21.15.63:0/828107794 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5615a6327570 con 0x5615a648a920 2024-05-27T07:44:31.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.719+0000 7f7e62000700 10 client.17508 handle_command_reply: tid=0 2024-05-27T07:44:31.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.719+0000 7f7e8dc32b80 1 client.17508 shutdown 2024-05-27T07:44:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.719+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/828107794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 msgr2=0x5615a6425a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.719+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6425a60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7e680a9370 tx=0x7f7e680b2100 comp rx=0 tx=0).stop 2024-05-27T07:44:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.719+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/828107794 shutdown_connections 2024-05-27T07:44:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.719+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5615a648a920 0x5615a648cd40 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.719+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a6403d30 0x5615a6423480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.720+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615a627ee40 0x5615a6425a60 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.720+0000 7f7e8dc32b80 1 --2- 172.21.15.63:0/828107794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615a63fddb0 0x5615a6422de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.720+0000 7f7e62000700 0 client.17508 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:31.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.720+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/828107794 >> 172.21.15.63:0/828107794 conn(0x5615a62defc0 msgr2=0x5615a6401140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:31.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.720+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/828107794 shutdown_connections 2024-05-27T07:44:31.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.720+0000 7f7e8dc32b80 1 -- 172.21.15.63:0/828107794 wait complete. 2024-05-27T07:44:31.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.720+0000 7f7e8dc32b80 20 client.17508 trim_cache size 0 max 16384 2024-05-27T07:44:31.720 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:44:31.721 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:44:31.721 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:44:31.721 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:44:31.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.721+0000 7f7e7f000700 1 -- 172.21.15.63:0/4160677164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7e5c0b2a70 msgr2=0x7f7e5c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.721+0000 7f7e7f000700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7e5c0b2a70 0x7f7e5c0b4f80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5615a63feb30 tx=0x7f7e70004b40 comp rx=0 tx=0).stop 2024-05-27T07:44:31.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.721+0000 7f7e7f000700 1 -- 172.21.15.63:0/4160677164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e781030f0 msgr2=0x7f7e7819cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.721+0000 7f7e7f000700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e781030f0 0x7f7e7819cf80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d950 tx=0x7f7e6800dc60 comp rx=0 tx=0).stop 2024-05-27T07:44:31.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.722+0000 7f7e7f000700 1 -- 172.21.15.63:0/4160677164 shutdown_connections 2024-05-27T07:44:31.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.722+0000 7f7e7f000700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7e5c0b2a70 0x7f7e5c0b4f80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.722+0000 7f7e7f000700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e78110ca0 0x7f7e781a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.722+0000 7f7e7f000700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e781030f0 0x7f7e7819cf80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.722+0000 7f7e7f000700 1 --2- 172.21.15.63:0/4160677164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e78103a40 0x7f7e7819d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.722+0000 7f7e7f000700 1 -- 172.21.15.63:0/4160677164 >> 172.21.15.63:0/4160677164 conn(0x7f7e780fedb0 msgr2=0x7f7e78112950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:31.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.722+0000 7f7e7f000700 1 -- 172.21.15.63:0/4160677164 shutdown_connections 2024-05-27T07:44:31.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:31.722+0000 7f7e7f000700 1 -- 172.21.15.63:0/4160677164 wait complete. 2024-05-27T07:44:32.723 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:44:32.723 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:44:32.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:32 smithi086 ceph-mon[34687]: pgmap v342: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T07:44:32.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:32 smithi149 ceph-mon[35408]: pgmap v342: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T07:44:32.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:32 smithi063 ceph-mon[31613]: pgmap v342: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T07:44:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:34.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:34 smithi086 ceph-mon[34687]: pgmap v343: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:44:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:34 smithi149 ceph-mon[35408]: pgmap v343: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:44:34.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:34 smithi063 ceph-mon[31613]: pgmap v343: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:44:35.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.822+0000 7ffae3400700 1 -- 172.21.15.63:0/57290500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffadc1045e0 msgr2=0x7ffadc1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.822+0000 7ffae3400700 1 --2- 172.21.15.63:0/57290500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffadc1045e0 0x7ffadc1049f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffad00099c0 tx=0x7ffad0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.822+0000 7ffae3400700 1 -- 172.21.15.63:0/57290500 shutdown_connections 2024-05-27T07:44:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.822+0000 7ffae3400700 1 --2- 172.21.15.63:0/57290500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc109860 0x7ffadc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.822+0000 7ffae3400700 1 --2- 172.21.15.63:0/57290500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffadc1045e0 0x7ffadc1049f0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.822+0000 7ffae3400700 1 --2- 172.21.15.63:0/57290500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffadc108f10 0x7ffadc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.822+0000 7ffae3400700 1 -- 172.21.15.63:0/57290500 >> 172.21.15.63:0/57290500 conn(0x7ffadc100030 msgr2=0x7ffadc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.822+0000 7ffae3400700 1 -- 172.21.15.63:0/57290500 shutdown_connections 2024-05-27T07:44:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.822+0000 7ffae3400700 1 -- 172.21.15.63:0/57290500 wait complete. 2024-05-27T07:44:35.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.824+0000 7ffae3400700 1 Processor -- start 2024-05-27T07:44:35.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.824+0000 7ffae3400700 1 -- start start 2024-05-27T07:44:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffae3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffadc1045e0 0x7ffadc075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffae3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc108f10 0x7ffadc075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffae3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffadc109860 0x7ffadc076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffae3400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc114390 con 0x7ffadc109860 2024-05-27T07:44:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffae3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc114210 con 0x7ffadc108f10 2024-05-27T07:44:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffae3400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc114510 con 0x7ffadc1045e0 2024-05-27T07:44:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffae1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffadc109860 0x7ffadc076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffae1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffadc109860 0x7ffadc076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44972/0 (socket says 172.21.15.63:44972) 2024-05-27T07:44:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffae1400700 1 -- 172.21.15.63:0/468921219 learned_addr learned my addr 172.21.15.63:0/468921219 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffae0a00700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffadc1045e0 0x7ffadc075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:35.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.825+0000 7ffadbe00700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc108f10 0x7ffadc075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:35.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.826+0000 7ffae1400700 1 -- 172.21.15.63:0/468921219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffadc1045e0 msgr2=0x7ffadc075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:35.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.826+0000 7ffae1400700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffadc1045e0 0x7ffadc075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.826+0000 7ffae1400700 1 -- 172.21.15.63:0/468921219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc108f10 msgr2=0x7ffadc075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.826+0000 7ffae1400700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc108f10 0x7ffadc075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.826+0000 7ffae1400700 1 -- 172.21.15.63:0/468921219 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc1a9140 con 0x7ffadc109860 2024-05-27T07:44:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.826+0000 7ffae1400700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffadc109860 0x7ffadc076530 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ffad4007ee0 tx=0x7ffad400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.827+0000 7ffad9600700 1 -- 172.21.15.63:0/468921219 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4004bb0 con 0x7ffadc109860 2024-05-27T07:44:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.827+0000 7ffad9600700 1 -- 172.21.15.63:0/468921219 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffad4004d50 con 0x7ffadc109860 2024-05-27T07:44:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.827+0000 7ffae3400700 1 -- 172.21.15.63:0/468921219 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffadc1a9430 con 0x7ffadc109860 2024-05-27T07:44:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.827+0000 7ffad9600700 1 -- 172.21.15.63:0/468921219 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad40058b0 con 0x7ffadc109860 2024-05-27T07:44:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.827+0000 7ffae3400700 1 -- 172.21.15.63:0/468921219 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffadc1a9920 con 0x7ffadc109860 2024-05-27T07:44:35.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.828+0000 7ffae3400700 1 -- 172.21.15.63:0/468921219 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffadc0512b0 con 0x7ffadc109860 2024-05-27T07:44:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.831+0000 7ffad9600700 1 -- 172.21.15.63:0/468921219 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffad401a020 con 0x7ffadc109860 2024-05-27T07:44:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.834+0000 7ffad9600700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffac40b29f0 0x7ffac40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.834+0000 7ffad9600700 1 -- 172.21.15.63:0/468921219 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7ffad40e20a0 con 0x7ffadc109860 2024-05-27T07:44:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.834+0000 7ffae0a00700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffac40b29f0 0x7ffac40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.835+0000 7ffae0a00700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffac40b29f0 0x7ffac40b4f00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffad0000c00 tx=0x7ffad0005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:35.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:35.838+0000 7ffad9600700 1 -- 172.21.15.63:0/468921219 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffad4010070 con 0x7ffadc109860 2024-05-27T07:44:36.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.171+0000 7ffae3400700 1 -- 172.21.15.63:0/468921219 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffadc02e7a0 con 0x7ffadc109860 2024-05-27T07:44:36.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.173+0000 7ffad9600700 1 -- 172.21.15.63:0/468921219 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7ffad40a9660 con 0x7ffadc109860 2024-05-27T07:44:36.173 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:44:36.174 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:44:36.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.177+0000 7ffae3400700 1 -- 172.21.15.63:0/468921219 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffac40b29f0 msgr2=0x7ffac40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:36.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.177+0000 7ffae3400700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffac40b29f0 0x7ffac40b4f00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffad0000c00 tx=0x7ffad0005fb0 comp rx=0 tx=0).stop 2024-05-27T07:44:36.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.177+0000 7ffae3400700 1 -- 172.21.15.63:0/468921219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffadc109860 msgr2=0x7ffadc076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:36.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.177+0000 7ffae3400700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffadc109860 0x7ffadc076530 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ffad4007ee0 tx=0x7ffad400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:36.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.177+0000 7ffae3400700 1 -- 172.21.15.63:0/468921219 shutdown_connections 2024-05-27T07:44:36.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.177+0000 7ffae3400700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffac40b29f0 0x7ffac40b4f00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.177+0000 7ffae3400700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffadc109860 0x7ffadc076530 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.177+0000 7ffae3400700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffadc1045e0 0x7ffadc075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.177+0000 7ffae3400700 1 --2- 172.21.15.63:0/468921219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffadc108f10 0x7ffadc075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.177+0000 7ffae3400700 1 -- 172.21.15.63:0/468921219 >> 172.21.15.63:0/468921219 conn(0x7ffadc100030 msgr2=0x7ffadc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.178+0000 7ffae3400700 1 -- 172.21.15.63:0/468921219 shutdown_connections 2024-05-27T07:44:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:36.178+0000 7ffae3400700 1 -- 172.21.15.63:0/468921219 wait complete. 2024-05-27T07:44:36.179 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:44:36.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:36 smithi086 ceph-mon[34687]: pgmap v344: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:44:36.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:36 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/468921219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:36.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:36 smithi149 ceph-mon[35408]: pgmap v344: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:44:36.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:36 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/468921219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:36.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:36 smithi063 ceph-mon[31613]: pgmap v344: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:44:36.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:36 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/468921219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:37.034 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:44:37.527 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.884+0000 7fd3af000640 1 -- 172.21.15.63:0/397986093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a810c340 msgr2=0x7fd3a8105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.884+0000 7fd3af000640 1 --2- 172.21.15.63:0/397986093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a810c340 0x7fd3a8105f80 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd394009900 tx=0x7fd39402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.884+0000 7fd3af000640 1 -- 172.21.15.63:0/397986093 shutdown_connections 2024-05-27T07:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.884+0000 7fd3af000640 1 --2- 172.21.15.63:0/397986093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a81065f0 0x7fd3a81129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.884+0000 7fd3af000640 1 --2- 172.21.15.63:0/397986093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a810c340 0x7fd3a8105f80 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.884+0000 7fd3af000640 1 --2- 172.21.15.63:0/397986093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a810bea0 0x7fd3a8105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.884+0000 7fd3af000640 1 -- 172.21.15.63:0/397986093 >> 172.21.15.63:0/397986093 conn(0x7fd3a80771f0 msgr2=0x7fd3a8077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.884+0000 7fd3af000640 1 -- 172.21.15.63:0/397986093 shutdown_connections 2024-05-27T07:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.884+0000 7fd3af000640 1 -- 172.21.15.63:0/397986093 wait complete. 2024-05-27T07:44:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.885+0000 7fd3af000640 1 Processor -- start 2024-05-27T07:44:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.886+0000 7fd3af000640 1 -- start start 2024-05-27T07:44:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.886+0000 7fd3af000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a81065f0 0x7fd3a81a2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.886+0000 7fd3af000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a810bea0 0x7fd3a81a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.886+0000 7fd3af000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a810c340 0x7fd3a819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.886+0000 7fd3af000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8115fa0 con 0x7fd3a810bea0 2024-05-27T07:44:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.886+0000 7fd3af000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8115e20 con 0x7fd3a81065f0 2024-05-27T07:44:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.886+0000 7fd3af000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8116120 con 0x7fd3a810c340 2024-05-27T07:44:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3a7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a810bea0 0x7fd3a81a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3a7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a810bea0 0x7fd3a81a2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44994/0 (socket says 172.21.15.63:44994) 2024-05-27T07:44:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3a7400640 1 -- 172.21.15.63:0/3214300974 learned_addr learned my addr 172.21.15.63:0/3214300974 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a81065f0 0x7fd3a81a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3ad000640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a810c340 0x7fd3a819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3a7400640 1 -- 172.21.15.63:0/3214300974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a810c340 msgr2=0x7fd3a819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3a7400640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a810c340 0x7fd3a819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3a7400640 1 -- 172.21.15.63:0/3214300974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a81065f0 msgr2=0x7fd3a81a2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3a7400640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a81065f0 0x7fd3a81a2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3a7400640 1 -- 172.21.15.63:0/3214300974 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a819cf20 con 0x7fd3a810bea0 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.887+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a81065f0 0x7fd3a81a2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.888+0000 7fd3a7400640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a810bea0 0x7fd3a81a2bb0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd394005940 tx=0x7fd3940040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.888+0000 7fd3a4c00640 1 -- 172.21.15.63:0/3214300974 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39403f030 con 0x7fd3a810bea0 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.888+0000 7fd3a4c00640 1 -- 172.21.15.63:0/3214300974 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd394003a80 con 0x7fd3a810bea0 2024-05-27T07:44:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.888+0000 7fd3af000640 1 -- 172.21.15.63:0/3214300974 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a819d1b0 con 0x7fd3a810bea0 2024-05-27T07:44:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.888+0000 7fd3a4c00640 1 -- 172.21.15.63:0/3214300974 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394035070 con 0x7fd3a810bea0 2024-05-27T07:44:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.888+0000 7fd3af000640 1 -- 172.21.15.63:0/3214300974 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a819d6a0 con 0x7fd3a810bea0 2024-05-27T07:44:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.892+0000 7fd3b5f95740 1 Processor -- start 2024-05-27T07:44:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.892+0000 7fd3b5f95740 1 -- start start 2024-05-27T07:44:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.893+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c5c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.893+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908912f0 0x561790c5cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.893+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908994d0 0x561790c5d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.893+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561790bcdb90 con 0x5617908994d0 2024-05-27T07:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.893+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561790bbd730 con 0x5617908ba1b0 2024-05-27T07:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.893+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561790bc1770 con 0x5617908912f0 2024-05-27T07:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.893+0000 7fd3a4c00640 1 -- 172.21.15.63:0/3214300974 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd394042030 con 0x7fd3a810bea0 2024-05-27T07:44:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.893+0000 7fd3a7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908994d0 0x561790c5d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.894+0000 7fd3a7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908994d0 0x561790c5d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45006/0 (socket says 172.21.15.63:45006) 2024-05-27T07:44:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.894+0000 7fd3a7400640 1 -- 172.21.15.63:0/2994896434 learned_addr learned my addr 172.21.15.63:0/2994896434 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.894+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/2994896434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908912f0 0x561790c5cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.894+0000 7fd3ad000640 1 --2- 172.21.15.63:0/2994896434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c5c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.894+0000 7fd3a7400640 1 -- 172.21.15.63:0/2994896434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908912f0 msgr2=0x561790c5cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.894+0000 7fd3a7400640 1 --2- 172.21.15.63:0/2994896434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908912f0 0x561790c5cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.894+0000 7fd3a7400640 1 -- 172.21.15.63:0/2994896434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 msgr2=0x561790c5c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.894+0000 7fd3a7400640 1 --2- 172.21.15.63:0/2994896434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c5c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.894+0000 7fd3a7400640 1 -- 172.21.15.63:0/2994896434 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561790b568b0 con 0x5617908994d0 2024-05-27T07:44:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.895+0000 7fd3a7400640 1 --2- 172.21.15.63:0/2994896434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908994d0 0x561790c5d2e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd3941002c0 tx=0x7fd39410b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4a920f8c4f9868 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.895+0000 7fd39d600640 1 -- 172.21.15.63:0/2994896434 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39410bd20 con 0x5617908994d0 2024-05-27T07:44:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.895+0000 7fd39d600640 1 -- 172.21.15.63:0/2994896434 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd394110030 con 0x5617908994d0 2024-05-27T07:44:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.895+0000 7fd39d600640 1 -- 172.21.15.63:0/2994896434 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394114630 con 0x5617908994d0 2024-05-27T07:44:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.895+0000 7fd3a4c00640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd38c0b2950 0x7fd38c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.895+0000 7fd3a4c00640 1 -- 172.21.15.63:0/3214300974 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd394102fc0 con 0x7fd3a810bea0 2024-05-27T07:44:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.896+0000 7fd3ad000640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd38c0b2950 0x7fd38c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.896+0000 7fd3b5f95740 1 -- 172.21.15.63:0/2994896434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908994d0 msgr2=0x561790c5d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.896+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/2994896434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908994d0 0x561790c5d2e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd3941002c0 tx=0x7fd39410b320 comp rx=0 tx=0).stop 2024-05-27T07:44:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.896+0000 7fd3ad000640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd38c0b2950 0x7fd38c0b4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x561790c599d0 tx=0x7fd3a0009740 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.897+0000 7fd3b5f95740 1 -- 172.21.15.63:0/2994896434 shutdown_connections 2024-05-27T07:44:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.897+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/2994896434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908994d0 0x561790c5d2e0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.897+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/2994896434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908912f0 0x561790c5cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.897+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/2994896434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c5c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.897+0000 7fd3b5f95740 1 -- 172.21.15.63:0/2994896434 >> 172.21.15.63:0/2994896434 conn(0x5617908ae300 msgr2=0x561790c59010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.897+0000 7fd3b5f95740 1 -- 172.21.15.63:0/2994896434 shutdown_connections 2024-05-27T07:44:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.897+0000 7fd3b5f95740 1 -- 172.21.15.63:0/2994896434 wait complete. 2024-05-27T07:44:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.897+0000 7fd3b5f95740 1 -- start start 2024-05-27T07:44:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.898+0000 7fd3b5f95740 1 Processor -- start 2024-05-27T07:44:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.898+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.898+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908994d0 0x561790c66f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.898+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c72a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.898+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5617908a3790 con 0x5617908912f0 2024-05-27T07:44:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.898+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561790b65bc0 con 0x5617908ba1b0 2024-05-27T07:44:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.898+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561790c69750 con 0x5617908994d0 2024-05-27T07:44:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.898+0000 7fd3b5f95740 10 client.0 _ll_register_callbacks cb 0x561790b43400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.898+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.899+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c669d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38250/0 (socket says 172.21.15.63:38250) 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.899+0000 7fd3a7e00640 1 -- 172.21.15.63:0/1607104452 learned_addr learned my addr 172.21.15.63:0/1607104452 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.899+0000 7fd3ad000640 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c72a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.899+0000 7fd3a7400640 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908994d0 0x561790c66f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.899+0000 7fd3a7e00640 1 -- 172.21.15.63:0/1607104452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908994d0 msgr2=0x561790c66f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.899+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908994d0 0x561790c66f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.899+0000 7fd3a7e00640 1 -- 172.21.15.63:0/1607104452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 msgr2=0x561790c72a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.899+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c72a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.899+0000 7fd3a7e00640 1 -- 172.21.15.63:0/1607104452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617908d2210 con 0x5617908912f0 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.900+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c669d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd3980104d0 tx=0x7fd398010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55abf68bc318b1a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.900+0000 7fd392000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.900+0000 7fd392000640 1 -- 172.21.15.63:0/1607104452 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398018070 con 0x5617908912f0 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.900+0000 7fd392000640 1 -- 172.21.15.63:0/1607104452 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd398014640 con 0x5617908912f0 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.900+0000 7fd3b5f95740 1 -- 172.21.15.63:0/1607104452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd394110030 con 0x5617908912f0 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.900+0000 7fd392000640 1 -- 172.21.15.63:0/1607104452 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd398024050 con 0x5617908912f0 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.900+0000 7fd3b5f95740 10 client.17526 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.900+0000 7fd3b5f95740 1 -- 172.21.15.63:0/1607104452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56179099e620 con 0x5617908912f0 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.901+0000 7fd392000640 1 -- 172.21.15.63:0/1607104452 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd394110030 con 0x5617908912f0 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.901+0000 7fd392000640 1 -- 172.21.15.63:0/1607104452 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd3980128d0 con 0x5617908912f0 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.901+0000 7fd3b5f95740 10 client.17526 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.901+0000 7fd3b5f95740 10 client.17526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.901+0000 7fd3b5f95740 20 client.17526 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.901+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561790878d80 0x561790c78a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.901+0000 7fd3b5f95740 4 client.17526 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:44:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.901+0000 7fd3b5f95740 1 -- 172.21.15.63:0/1607104452 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561790bcfae0 con 0x561790878d80 2024-05-27T07:44:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.902+0000 7fd3a7400640 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561790878d80 0x561790c78a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.903+0000 7fd3a7400640 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561790878d80 0x561790c78a40 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb5b3aa1b77f901 server_cookie=37582c8e1e4a471d in_seq=0 out_seq=0 2024-05-27T07:44:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.903+0000 7fd392000640 10 client.17526 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd392000640 1 -- 172.21.15.63:0/1607104452 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561790bcfae0 con 0x561790878d80 2024-05-27T07:44:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd392000640 10 client.17526 handle_command_reply: tid=0 2024-05-27T07:44:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 client.17526 shutdown 2024-05-27T07:44:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 -- 172.21.15.63:0/1607104452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 msgr2=0x561790c669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c669d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd3980104d0 tx=0x7fd398010990 comp rx=0 tx=0).stop 2024-05-27T07:44:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 -- 172.21.15.63:0/1607104452 shutdown_connections 2024-05-27T07:44:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561790878d80 0x561790c78a40 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c72a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908994d0 0x561790c66f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/1607104452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c669d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 -- 172.21.15.63:0/1607104452 >> 172.21.15.63:0/1607104452 conn(0x5617908ae300 msgr2=0x561790c5e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd392000640 0 client.17526 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 -- 172.21.15.63:0/1607104452 shutdown_connections 2024-05-27T07:44:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.905+0000 7fd3b5f95740 1 -- 172.21.15.63:0/1607104452 wait complete. 2024-05-27T07:44:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.906+0000 7fd3b5f95740 20 client.17526 trim_cache size 0 max 16384 2024-05-27T07:44:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.926+0000 7fd3b5f95740 1 Processor -- start 2024-05-27T07:44:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.926+0000 7fd3b5f95740 1 -- start start 2024-05-27T07:44:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.927+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.927+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908994d0 0x561790c5d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.927+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908ba1b0 0x561790c605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.927+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b5dc0 con 0x5617908912f0 2024-05-27T07:44:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.927+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b3690 con 0x5617908994d0 2024-05-27T07:44:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.927+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b1500 con 0x5617908ba1b0 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.927+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.927+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38258/0 (socket says 172.21.15.63:38258) 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.927+0000 7fd3a7e00640 1 -- 172.21.15.63:0/4220438998 learned_addr learned my addr 172.21.15.63:0/4220438998 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd3a7400640 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908994d0 0x561790c5d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd3ad000640 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908ba1b0 0x561790c605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd3a7e00640 1 -- 172.21.15.63:0/4220438998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908ba1b0 msgr2=0x561790c605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908ba1b0 0x561790c605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd3a7e00640 1 -- 172.21.15.63:0/4220438998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908994d0 msgr2=0x561790c5d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908994d0 0x561790c5d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd3a7e00640 1 -- 172.21.15.63:0/4220438998 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561790878fb0 con 0x5617908912f0 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd3a7400640 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908994d0 0x561790c5d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd3ad000640 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908ba1b0 0x561790c605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c621b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd398000c00 tx=0x7fd3980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f868be3867a1b6c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.928+0000 7fd39cc00640 1 -- 172.21.15.63:0/4220438998 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398018050 con 0x5617908912f0 2024-05-27T07:44:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.929+0000 7fd39cc00640 1 -- 172.21.15.63:0/4220438998 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd39802d070 con 0x5617908912f0 2024-05-27T07:44:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.929+0000 7fd39cc00640 1 -- 172.21.15.63:0/4220438998 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3980149a0 con 0x5617908912f0 2024-05-27T07:44:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.929+0000 7fd3b5f95740 1 -- 172.21.15.63:0/4220438998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 msgr2=0x561790c621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.929+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c621b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd398000c00 tx=0x7fd3980046c0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.930+0000 7fd3b5f95740 1 -- 172.21.15.63:0/4220438998 shutdown_connections 2024-05-27T07:44:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.930+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908ba1b0 0x561790c605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.930+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908994d0 0x561790c5d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.930+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/4220438998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c621b0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.930+0000 7fd3b5f95740 1 -- 172.21.15.63:0/4220438998 >> 172.21.15.63:0/4220438998 conn(0x5617908ae300 msgr2=0x561790c66d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.930+0000 7fd3b5f95740 1 -- 172.21.15.63:0/4220438998 shutdown_connections 2024-05-27T07:44:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.930+0000 7fd3b5f95740 1 -- 172.21.15.63:0/4220438998 wait complete. 2024-05-27T07:44:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.930+0000 7fd3b5f95740 1 -- start start 2024-05-27T07:44:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.930+0000 7fd3b5f95740 1 Processor -- start 2024-05-27T07:44:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.931+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c69960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.931+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908994d0 0x561790c69ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.931+0000 7fd3b5f95740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c6a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.931+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b1b50 con 0x5617908912f0 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.931+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b2fc0 con 0x5617908ba1b0 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.931+0000 7fd3b5f95740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b2220 con 0x5617908994d0 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.931+0000 7fd3b5f95740 10 client.0 _ll_register_callbacks cb 0x7fd398007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.931+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c69960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.931+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c69960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38268/0 (socket says 172.21.15.63:38268) 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.931+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3482052883 learned_addr learned my addr 172.21.15.63:0/3482052883 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.932+0000 7fd3a7400640 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908994d0 0x561790c69ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.932+0000 7fd3ad000640 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c6a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.932+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3482052883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908994d0 msgr2=0x561790c69ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.932+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908994d0 0x561790c69ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.932+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3482052883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 msgr2=0x561790c6a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.932+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c6a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.932+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3482052883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561790cdf150 con 0x5617908912f0 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.932+0000 7fd3ad000640 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c6a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.932+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c69960 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd3980109d0 tx=0x7fd3980126a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ada385f2e2a6ed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.932+0000 7fd392000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.933+0000 7fd392000640 1 -- 172.21.15.63:0/3482052883 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398035060 con 0x5617908912f0 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.933+0000 7fd392000640 1 -- 172.21.15.63:0/3482052883 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd398014720 con 0x5617908912f0 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.933+0000 7fd3b5f95740 1 -- 172.21.15.63:0/3482052883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd39802d070 con 0x5617908912f0 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.933+0000 7fd392000640 1 -- 172.21.15.63:0/3482052883 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd398018040 con 0x5617908912f0 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.933+0000 7fd3b5f95740 10 client.17532 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.933+0000 7fd3b5f95740 1 -- 172.21.15.63:0/3482052883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561790ce0350 con 0x5617908912f0 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.933+0000 7fd392000640 1 -- 172.21.15.63:0/3482052883 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd39802d070 con 0x5617908912f0 2024-05-27T07:44:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.934+0000 7fd392000640 1 -- 172.21.15.63:0/3482052883 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd398031870 con 0x5617908912f0 2024-05-27T07:44:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.934+0000 7fd3b5f95740 10 client.17532 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.934+0000 7fd3b5f95740 10 client.17532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.934+0000 7fd3b5f95740 20 client.17532 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.934+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56179084e1c0 0x561790ce2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.934+0000 7fd3b5f95740 4 client.17532 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:44:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.934+0000 7fd3b5f95740 1 -- 172.21.15.63:0/3482052883 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561790bc92c0 con 0x56179084e1c0 2024-05-27T07:44:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.934+0000 7fd3a7400640 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56179084e1c0 0x561790ce2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.935+0000 7fd392000640 10 client.17532 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.935+0000 7fd3a7400640 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56179084e1c0 0x561790ce2990 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d26a347d2b9bc6d server_cookie=bb8656e3f34e978 in_seq=0 out_seq=0 2024-05-27T07:44:38.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:38 smithi086 ceph-mon[34687]: pgmap v345: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:44:38.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:38 smithi149 ceph-mon[35408]: pgmap v345: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:44:38.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:38 smithi063 ceph-mon[31613]: pgmap v345: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:44:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd392000640 1 -- 172.21.15.63:0/3482052883 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561790bc92c0 con 0x56179084e1c0 2024-05-27T07:44:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd392000640 10 client.17532 handle_command_reply: tid=0 2024-05-27T07:44:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd3b5f95740 1 client.17532 shutdown 2024-05-27T07:44:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd3b5f95740 1 -- 172.21.15.63:0/3482052883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 msgr2=0x561790c69960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c69960 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd3980109d0 tx=0x7fd3980126a0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd3b5f95740 1 -- 172.21.15.63:0/3482052883 shutdown_connections 2024-05-27T07:44:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56179084e1c0 0x561790ce2990 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5617908ba1b0 0x561790c6a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617908994d0 0x561790c69ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd3b5f95740 1 --2- 172.21.15.63:0/3482052883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617908912f0 0x561790c69960 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd392000640 0 client.17532 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.977+0000 7fd3b5f95740 1 -- 172.21.15.63:0/3482052883 >> 172.21.15.63:0/3482052883 conn(0x5617908ae300 msgr2=0x561790c66d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.978+0000 7fd3b5f95740 1 -- 172.21.15.63:0/3482052883 shutdown_connections 2024-05-27T07:44:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.978+0000 7fd3b5f95740 1 -- 172.21.15.63:0/3482052883 wait complete. 2024-05-27T07:44:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.978+0000 7fd3b5f95740 20 client.17532 trim_cache size 0 max 16384 2024-05-27T07:44:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:44:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:44:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 -- 172.21.15.63:0/3214300974 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd38c0b2950 msgr2=0x7fd38c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd38c0b2950 0x7fd38c0b4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x561790c599d0 tx=0x7fd3a0009740 comp rx=0 tx=0).stop 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 -- 172.21.15.63:0/3214300974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a810bea0 msgr2=0x7fd3a81a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a810bea0 0x7fd3a81a2bb0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd394005940 tx=0x7fd3940040e0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 -- 172.21.15.63:0/3214300974 shutdown_connections 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd38c0b2950 0x7fd38c0b4e60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a810c340 0x7fd3a819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a810bea0 0x7fd3a81a2bb0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 --2- 172.21.15.63:0/3214300974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a81065f0 0x7fd3a81a2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 -- 172.21.15.63:0/3214300974 >> 172.21.15.63:0/3214300974 conn(0x7fd3a80771f0 msgr2=0x7fd3a8104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 -- 172.21.15.63:0/3214300974 shutdown_connections 2024-05-27T07:44:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:38.979+0000 7fd3af000640 1 -- 172.21.15.63:0/3214300974 wait complete. 2024-05-27T07:44:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.033+0000 7f6c53200700 1 -- 172.21.15.63:0/589165759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c4c1032e0 msgr2=0x7f6c4c1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.033+0000 7f6c53200700 1 --2- 172.21.15.63:0/589165759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c4c1032e0 0x7f6c4c1036f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6c400099c0 tx=0x7f6c40009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.033+0000 7f6c53200700 1 -- 172.21.15.63:0/589165759 shutdown_connections 2024-05-27T07:44:40.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.033+0000 7f6c53200700 1 --2- 172.21.15.63:0/589165759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4c108560 0x7f6c4c10fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.033+0000 7f6c53200700 1 --2- 172.21.15.63:0/589165759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c4c1032e0 0x7f6c4c1036f0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.033+0000 7f6c53200700 1 --2- 172.21.15.63:0/589165759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107c10 0x7f6c4c108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.033+0000 7f6c53200700 1 -- 172.21.15.63:0/589165759 >> 172.21.15.63:0/589165759 conn(0x7f6c4c078af0 msgr2=0x7f6c4c078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.034+0000 7f6c53200700 1 -- 172.21.15.63:0/589165759 shutdown_connections 2024-05-27T07:44:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.034+0000 7f6c53200700 1 -- 172.21.15.63:0/589165759 wait complete. 2024-05-27T07:44:40.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.035+0000 7f6c53200700 1 Processor -- start 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.035+0000 7f6c53200700 1 -- start start 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c53200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c1032e0 0x7f6c4c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c53200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4c107c10 0x7f6c4c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c53200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c4c108560 0x7f6c4c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c53200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c113040 con 0x7f6c4c108560 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c53200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c112ec0 con 0x7f6c4c1032e0 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c53200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1131c0 con 0x7f6c4c107c10 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c52800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c4c108560 0x7f6c4c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c52800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c4c108560 0x7f6c4c19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38292/0 (socket says 172.21.15.63:38292) 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c52800700 1 -- 172.21.15.63:0/1812084836 learned_addr learned my addr 172.21.15.63:0/1812084836 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c51400700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4c107c10 0x7f6c4c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.036+0000 7f6c51e00700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c1032e0 0x7f6c4c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.037+0000 7f6c52800700 1 -- 172.21.15.63:0/1812084836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4c107c10 msgr2=0x7f6c4c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.037+0000 7f6c52800700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4c107c10 0x7f6c4c1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.037+0000 7f6c52800700 1 -- 172.21.15.63:0/1812084836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c1032e0 msgr2=0x7f6c4c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.037+0000 7f6c52800700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c1032e0 0x7f6c4c1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.037+0000 7f6c52800700 1 -- 172.21.15.63:0/1812084836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c19b020 con 0x7f6c4c108560 2024-05-27T07:44:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.038+0000 7f6c52800700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c4c108560 0x7f6c4c19a970 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6c48007ee0 tx=0x7f6c4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.038+0000 7f6c46a00700 1 -- 172.21.15.63:0/1812084836 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48004bb0 con 0x7f6c4c108560 2024-05-27T07:44:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.038+0000 7f6c46a00700 1 -- 172.21.15.63:0/1812084836 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c48004d50 con 0x7f6c4c108560 2024-05-27T07:44:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.038+0000 7f6c53200700 1 -- 172.21.15.63:0/1812084836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c19b310 con 0x7f6c4c108560 2024-05-27T07:44:40.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.038+0000 7f6c46a00700 1 -- 172.21.15.63:0/1812084836 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c480058b0 con 0x7f6c4c108560 2024-05-27T07:44:40.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.038+0000 7f6c53200700 1 -- 172.21.15.63:0/1812084836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c100400 con 0x7f6c4c108560 2024-05-27T07:44:40.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.043+0000 7f6c46a00700 1 -- 172.21.15.63:0/1812084836 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6c4801a020 con 0x7f6c4c108560 2024-05-27T07:44:40.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.044+0000 7f6c633fcb80 1 Processor -- start 2024-05-27T07:44:40.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.045+0000 7f6c46a00700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6c340b29f0 0x7f6c340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.045+0000 7f6c633fcb80 1 -- start start 2024-05-27T07:44:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.045+0000 7f6c46a00700 1 -- 172.21.15.63:0/1812084836 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f6c480e20a0 con 0x7f6c4c108560 2024-05-27T07:44:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.045+0000 7f6c51400700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6c340b29f0 0x7f6c340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.045+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f8ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.045+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.045+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f90d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.045+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e02f82ee40 con 0x55e02f900410 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.045+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e02f81e9e0 con 0x55e02f781e40 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.045+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e02f822a20 con 0x55e02f900820 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.046+0000 7f6c51e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f8ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.046+0000 7f6c51400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.046+0000 7f6c51e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f8ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:33696/0 (socket says 172.21.15.63:33696) 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.046+0000 7f6c51e00700 1 -- 172.21.15.63:0/1803470890 learned_addr learned my addr 172.21.15.63:0/1803470890 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.046+0000 7f6c52800700 1 --2- 172.21.15.63:0/1803470890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f90d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.046+0000 7f6c51400700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6c340b29f0 0x7f6c340b4f00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0ffdc0 tx=0x7f6c3c0106b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.047+0000 7f6c51400700 1 -- 172.21.15.63:0/1803470890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 msgr2=0x55e02f90d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.047+0000 7f6c51400700 1 --2- 172.21.15.63:0/1803470890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f90d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.047+0000 7f6c51400700 1 -- 172.21.15.63:0/1803470890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 msgr2=0x55e02f8ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.047+0000 7f6c51400700 1 --2- 172.21.15.63:0/1803470890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f8ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.047+0000 7f6c51400700 1 -- 172.21.15.63:0/1803470890 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02f7e7420 con 0x55e02f900410 2024-05-27T07:44:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.047+0000 7f6c51400700 1 --2- 172.21.15.63:0/1803470890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90aad0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c005950 tx=0x7f6c3c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f832e01d4767b1c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.048+0000 7f6c3b400700 1 -- 172.21.15.63:0/1803470890 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c013ba0 con 0x55e02f900410 2024-05-27T07:44:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.048+0000 7f6c3b400700 1 -- 172.21.15.63:0/1803470890 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c3c013d40 con 0x55e02f900410 2024-05-27T07:44:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.048+0000 7f6c3b400700 1 -- 172.21.15.63:0/1803470890 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c005350 con 0x55e02f900410 2024-05-27T07:44:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.049+0000 7f6c633fcb80 1 -- 172.21.15.63:0/1803470890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 msgr2=0x55e02f90aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.049+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/1803470890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90aad0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c005950 tx=0x7f6c3c01b000 comp rx=0 tx=0).stop 2024-05-27T07:44:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.049+0000 7f6c633fcb80 1 -- 172.21.15.63:0/1803470890 shutdown_connections 2024-05-27T07:44:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.049+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/1803470890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f90d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.049+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/1803470890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f8ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.049+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/1803470890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90aad0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.049+0000 7f6c633fcb80 1 -- 172.21.15.63:0/1803470890 >> 172.21.15.63:0/1803470890 conn(0x55e02f7e1fc0 msgr2=0x55e02f8fbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.049+0000 7f6c633fcb80 1 -- 172.21.15.63:0/1803470890 shutdown_connections 2024-05-27T07:44:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.049+0000 7f6c633fcb80 1 -- 172.21.15.63:0/1803470890 wait complete. 2024-05-27T07:44:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.050+0000 7f6c633fcb80 1 -- start start 2024-05-27T07:44:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.050+0000 7f6c633fcb80 1 Processor -- start 2024-05-27T07:44:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.051+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f781e40 0x55e02f90a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.051+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.051+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f900820 0x55e02f90b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.051+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e02f724fa0 con 0x55e02f900410 2024-05-27T07:44:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.051+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e02f7bf760 con 0x55e02f900820 2024-05-27T07:44:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.051+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e02f90d990 con 0x55e02f781e40 2024-05-27T07:44:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.051+0000 7f6c633fcb80 10 client.0 _ll_register_callbacks cb 0x55e02f7c8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.051+0000 7f6c51400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.051+0000 7f6c51400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38318/0 (socket says 172.21.15.63:38318) 2024-05-27T07:44:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.052+0000 7f6c51400700 1 -- 172.21.15.63:0/3379566956 learned_addr learned my addr 172.21.15.63:0/3379566956 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.052+0000 7f6c52800700 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f900820 0x55e02f90b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.052+0000 7f6c51e00700 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f781e40 0x55e02f90a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.052+0000 7f6c51400700 1 -- 172.21.15.63:0/3379566956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f781e40 msgr2=0x55e02f90a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.052+0000 7f6c51400700 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f781e40 0x55e02f90a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.052+0000 7f6c51400700 1 -- 172.21.15.63:0/3379566956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f900820 msgr2=0x55e02f90b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.052+0000 7f6c51400700 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f900820 0x55e02f90b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.052+0000 7f6c51400700 1 -- 172.21.15.63:0/3379566956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02f76e8d0 con 0x55e02f900410 2024-05-27T07:44:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.052+0000 7f6c51e00700 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f781e40 0x55e02f90a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.053+0000 7f6c51400700 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90ab20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c013300 tx=0x7f6c3c013330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c754f8ac1c0e19 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.053+0000 7f6c2fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.053+0000 7f6c2fe00700 1 -- 172.21.15.63:0/3379566956 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c033060 con 0x55e02f900410 2024-05-27T07:44:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.053+0000 7f6c2fe00700 1 -- 172.21.15.63:0/3379566956 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f6c3c005410 con 0x55e02f900410 2024-05-27T07:44:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.053+0000 7f6c633fcb80 1 -- 172.21.15.63:0/3379566956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3c013d40 con 0x55e02f900410 2024-05-27T07:44:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c2fe00700 1 -- 172.21.15.63:0/3379566956 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6c3c03a050 con 0x55e02f900410 2024-05-27T07:44:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c633fcb80 10 client.17544 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c633fcb80 1 -- 172.21.15.63:0/3379566956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e02f7ef060 con 0x55e02f900410 2024-05-27T07:44:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c2fe00700 1 -- 172.21.15.63:0/3379566956 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c3c013d40 con 0x55e02f900410 2024-05-27T07:44:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c2fe00700 1 -- 172.21.15.63:0/3379566956 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6c3c011670 con 0x55e02f900410 2024-05-27T07:44:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c633fcb80 10 client.17544 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c633fcb80 10 client.17544 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:44:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c633fcb80 20 client.17544 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55e02f91a870 0x55e02f91cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c633fcb80 4 client.17544 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:44:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.054+0000 7f6c633fcb80 1 -- 172.21.15.63:0/3379566956 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e02f830d90 con 0x55e02f91a870 2024-05-27T07:44:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.055+0000 7f6c51e00700 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55e02f91a870 0x55e02f91cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.056+0000 7f6c51e00700 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55e02f91a870 0x55e02f91cc90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7b71d91cf14126f server_cookie=ce524a343fc57327 in_seq=0 out_seq=0 2024-05-27T07:44:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.056+0000 7f6c2fe00700 10 client.17544 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:44:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c2fe00700 1 -- 172.21.15.63:0/3379566956 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e02f830d90 con 0x55e02f91a870 2024-05-27T07:44:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c2fe00700 10 client.17544 handle_command_reply: tid=0 2024-05-27T07:44:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 client.17544 shutdown 2024-05-27T07:44:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 -- 172.21.15.63:0/3379566956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 msgr2=0x55e02f90ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90ab20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c013300 tx=0x7f6c3c013330 comp rx=0 tx=0).stop 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 -- 172.21.15.63:0/3379566956 shutdown_connections 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f900820 0x55e02f90b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f781e40 0x55e02f90a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55e02f91a870 0x55e02f91cc90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/3379566956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f90ab20 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 -- 172.21.15.63:0/3379566956 >> 172.21.15.63:0/3379566956 conn(0x55e02f7e1fc0 msgr2=0x55e02f8faff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c2fe00700 0 client.17544 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 -- 172.21.15.63:0/3379566956 shutdown_connections 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.059+0000 7f6c633fcb80 1 -- 172.21.15.63:0/3379566956 wait complete. 2024-05-27T07:44:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.060+0000 7f6c633fcb80 20 client.17544 trim_cache size 0 max 16384 2024-05-27T07:44:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.079+0000 7f6c633fcb80 1 Processor -- start 2024-05-27T07:44:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.079+0000 7f6c633fcb80 1 -- start start 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f90ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f9047a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f91f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b4e90 con 0x55e02f900410 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b2720 con 0x55e02f781e40 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b05b0 con 0x55e02f900820 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c51400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f9047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c51400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f9047a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38330/0 (socket says 172.21.15.63:38330) 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c51400700 1 -- 172.21.15.63:0/765382808 learned_addr learned my addr 172.21.15.63:0/765382808 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c51e00700 1 --2- 172.21.15.63:0/765382808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f90ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c52800700 1 --2- 172.21.15.63:0/765382808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f91f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.080+0000 7f6c51400700 1 -- 172.21.15.63:0/765382808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 msgr2=0x55e02f91f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.081+0000 7f6c51400700 1 --2- 172.21.15.63:0/765382808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f91f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.081+0000 7f6c51400700 1 -- 172.21.15.63:0/765382808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 msgr2=0x55e02f90ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.081+0000 7f6c51400700 1 --2- 172.21.15.63:0/765382808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f90ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.081+0000 7f6c51400700 1 -- 172.21.15.63:0/765382808 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02f8f61d0 con 0x55e02f900410 2024-05-27T07:44:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.081+0000 7f6c51e00700 1 --2- 172.21.15.63:0/765382808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f90ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.081+0000 7f6c51400700 1 --2- 172.21.15.63:0/765382808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f9047a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00bad0 tx=0x7f6c3c015fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc33cb1587f1106 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.081+0000 7f6c3aa00700 1 -- 172.21.15.63:0/765382808 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c033070 con 0x55e02f900410 2024-05-27T07:44:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.081+0000 7f6c3aa00700 1 -- 172.21.15.63:0/765382808 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c3c043070 con 0x55e02f900410 2024-05-27T07:44:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.081+0000 7f6c3aa00700 1 -- 172.21.15.63:0/765382808 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c004690 con 0x55e02f900410 2024-05-27T07:44:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.083+0000 7f6c633fcb80 1 -- 172.21.15.63:0/765382808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 msgr2=0x55e02f9047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.083+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/765382808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f9047a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00bad0 tx=0x7f6c3c015fd0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.083+0000 7f6c633fcb80 1 -- 172.21.15.63:0/765382808 shutdown_connections 2024-05-27T07:44:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.083+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/765382808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f91f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.083+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/765382808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f90ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.083+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/765382808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f9047a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.083+0000 7f6c633fcb80 1 -- 172.21.15.63:0/765382808 >> 172.21.15.63:0/765382808 conn(0x55e02f7e1fc0 msgr2=0x55e02f8fb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.083+0000 7f6c633fcb80 1 -- 172.21.15.63:0/765382808 shutdown_connections 2024-05-27T07:44:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.083+0000 7f6c633fcb80 1 -- 172.21.15.63:0/765382808 wait complete. 2024-05-27T07:44:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.083+0000 7f6c633fcb80 1 -- start start 2024-05-27T07:44:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.084+0000 7f6c633fcb80 1 Processor -- start 2024-05-27T07:44:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.084+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f91cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.084+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f91d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.084+0000 7f6c633fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f98b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.084+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b0c00 con 0x55e02f900410 2024-05-27T07:44:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.084+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b2050 con 0x55e02f781e40 2024-05-27T07:44:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.084+0000 7f6c633fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b12b0 con 0x55e02f900820 2024-05-27T07:44:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c633fcb80 10 client.0 _ll_register_callbacks cb 0x55e02f7c8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c51400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f91d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c51400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f91d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38342/0 (socket says 172.21.15.63:38342) 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c51400700 1 -- 172.21.15.63:0/898743525 learned_addr learned my addr 172.21.15.63:0/898743525 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c51e00700 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f91cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c52800700 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f98b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c51400700 1 -- 172.21.15.63:0/898743525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 msgr2=0x55e02f98b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c51400700 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f98b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c51400700 1 -- 172.21.15.63:0/898743525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 msgr2=0x55e02f91cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c51400700 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f91cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.085+0000 7f6c51400700 1 -- 172.21.15.63:0/898743525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02f98be20 con 0x55e02f900410 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.086+0000 7f6c51400700 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f91d1a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c010 tx=0x7f6c3c012c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f633875d092d868f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.086+0000 7f6c2fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.086+0000 7f6c2fe00700 1 -- 172.21.15.63:0/898743525 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c04c060 con 0x55e02f900410 2024-05-27T07:44:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.086+0000 7f6c2fe00700 1 -- 172.21.15.63:0/898743525 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f6c3c013c20 con 0x55e02f900410 2024-05-27T07:44:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.086+0000 7f6c633fcb80 1 -- 172.21.15.63:0/898743525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3c043070 con 0x55e02f900410 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.087+0000 7f6c2fe00700 1 -- 172.21.15.63:0/898743525 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6c3c053050 con 0x55e02f900410 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.087+0000 7f6c633fcb80 10 client.17550 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.087+0000 7f6c633fcb80 1 -- 172.21.15.63:0/898743525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e02f98ceb0 con 0x55e02f900410 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.087+0000 7f6c2fe00700 1 -- 172.21.15.63:0/898743525 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c3c043070 con 0x55e02f900410 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.087+0000 7f6c2fe00700 1 -- 172.21.15.63:0/898743525 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6c3c047450 con 0x55e02f900410 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.087+0000 7f6c633fcb80 10 client.17550 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.087+0000 7f6c633fcb80 10 client.17550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.088+0000 7f6c633fcb80 20 client.17550 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.088+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55e02f98d530 0x55e02f98f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.088+0000 7f6c633fcb80 4 client.17550 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:44:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.088+0000 7f6c633fcb80 1 -- 172.21.15.63:0/898743525 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e02f82a570 con 0x55e02f98d530 2024-05-27T07:44:40.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.088+0000 7f6c51e00700 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55e02f98d530 0x55e02f98f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.089+0000 7f6c51e00700 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55e02f98d530 0x55e02f98f950 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b00c73cc42dde063 server_cookie=7843fe3c0ec1f4f3 in_seq=0 out_seq=0 2024-05-27T07:44:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.089+0000 7f6c2fe00700 10 client.17550 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:44:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c2fe00700 1 -- 172.21.15.63:0/898743525 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e02f82a570 con 0x55e02f98d530 2024-05-27T07:44:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c2fe00700 10 client.17550 handle_command_reply: tid=0 2024-05-27T07:44:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c633fcb80 1 client.17550 shutdown 2024-05-27T07:44:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c633fcb80 1 -- 172.21.15.63:0/898743525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 msgr2=0x55e02f91d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f91d1a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c010 tx=0x7f6c3c012c70 comp rx=0 tx=0).stop 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c633fcb80 1 -- 172.21.15.63:0/898743525 shutdown_connections 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e02f900820 0x55e02f98b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e02f781e40 0x55e02f91cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55e02f98d530 0x55e02f98f950 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c633fcb80 1 --2- 172.21.15.63:0/898743525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02f900410 0x55e02f91d1a0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c633fcb80 1 -- 172.21.15.63:0/898743525 >> 172.21.15.63:0/898743525 conn(0x55e02f7e1fc0 msgr2=0x55e02f908ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c2fe00700 0 client.17550 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.090+0000 7f6c633fcb80 1 -- 172.21.15.63:0/898743525 shutdown_connections 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.091+0000 7f6c633fcb80 1 -- 172.21.15.63:0/898743525 wait complete. 2024-05-27T07:44:40.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.091+0000 7f6c633fcb80 20 client.17550 trim_cache size 0 max 16384 2024-05-27T07:44:40.093 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:44:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.091+0000 7f6c53200700 1 -- 172.21.15.63:0/1812084836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6c340b29f0 msgr2=0x7f6c340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.091+0000 7f6c53200700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6c340b29f0 0x7f6c340b4f00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0ffdc0 tx=0x7f6c3c0106b0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.091+0000 7f6c53200700 1 -- 172.21.15.63:0/1812084836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c4c108560 msgr2=0x7f6c4c19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.091+0000 7f6c53200700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c4c108560 0x7f6c4c19a970 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6c48007ee0 tx=0x7f6c4800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.092+0000 7f6c53200700 1 -- 172.21.15.63:0/1812084836 shutdown_connections 2024-05-27T07:44:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.092+0000 7f6c53200700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6c340b29f0 0x7f6c340b4f00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.092+0000 7f6c53200700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c4c108560 0x7f6c4c19a970 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.092+0000 7f6c53200700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c1032e0 0x7f6c4c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.092+0000 7f6c53200700 1 --2- 172.21.15.63:0/1812084836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c4c107c10 0x7f6c4c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.092+0000 7f6c53200700 1 -- 172.21.15.63:0/1812084836 >> 172.21.15.63:0/1812084836 conn(0x7f6c4c078af0 msgr2=0x7f6c4c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.092+0000 7f6c53200700 1 -- 172.21.15.63:0/1812084836 shutdown_connections 2024-05-27T07:44:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:40.092+0000 7f6c53200700 1 -- 172.21.15.63:0/1812084836 wait complete. 2024-05-27T07:44:40.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:40 smithi149 ceph-mon[35408]: pgmap v346: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:44:40.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:40 smithi063 ceph-mon[31613]: pgmap v346: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:44:41.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:40 smithi086 ceph-mon[34687]: pgmap v346: 129 pgs: 129 active+clean; 2.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:44:42.477 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:44:43.008 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:43.166 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:42 smithi063 ceph-mon[31613]: pgmap v347: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:44:43.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:42 smithi086 ceph-mon[34687]: pgmap v347: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:44:43.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:42 smithi149 ceph-mon[35408]: pgmap v347: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:44:45.012 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:44 smithi063 ceph-mon[31613]: pgmap v348: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:44:45.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:44 smithi086 ceph-mon[34687]: pgmap v348: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:44:45.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:44 smithi149 ceph-mon[35408]: pgmap v348: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:44:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.402+0000 7fd37a800700 1 -- 172.21.15.63:0/1523363267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3741045e0 msgr2=0x7fd3741049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.402+0000 7fd37a800700 1 --2- 172.21.15.63:0/1523363267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3741045e0 0x7fd3741049f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd3640099c0 tx=0x7fd364009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.403+0000 7fd37a800700 1 -- 172.21.15.63:0/1523363267 shutdown_connections 2024-05-27T07:44:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.403+0000 7fd37a800700 1 --2- 172.21.15.63:0/1523363267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd374109860 0x7fd3741111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.403+0000 7fd37a800700 1 --2- 172.21.15.63:0/1523363267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3741045e0 0x7fd3741049f0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.403+0000 7fd37a800700 1 --2- 172.21.15.63:0/1523363267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd374108f10 0x7fd374109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.403+0000 7fd37a800700 1 -- 172.21.15.63:0/1523363267 >> 172.21.15.63:0/1523363267 conn(0x7fd374100030 msgr2=0x7fd374102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.404+0000 7fd37a800700 1 -- 172.21.15.63:0/1523363267 shutdown_connections 2024-05-27T07:44:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.405+0000 7fd37a800700 1 -- 172.21.15.63:0/1523363267 wait complete. 2024-05-27T07:44:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.406+0000 7fd37a800700 1 Processor -- start 2024-05-27T07:44:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.407+0000 7fd37a800700 1 -- start start 2024-05-27T07:44:45.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.407+0000 7fd37a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3741045e0 0x7fd37419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.407+0000 7fd37a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd374108f10 0x7fd37419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.407+0000 7fd37a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd374109860 0x7fd3741a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.407+0000 7fd37a800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd374114390 con 0x7fd374108f10 2024-05-27T07:44:45.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.407+0000 7fd37a800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd374114210 con 0x7fd3741045e0 2024-05-27T07:44:45.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.407+0000 7fd37a800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd374114510 con 0x7fd374109860 2024-05-27T07:44:45.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd372a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd374108f10 0x7fd37419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd372a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd374108f10 0x7fd37419d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38362/0 (socket says 172.21.15.63:38362) 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd372a00700 1 -- 172.21.15.63:0/296703443 learned_addr learned my addr 172.21.15.63:0/296703443 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd373400700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3741045e0 0x7fd37419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd373e00700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd374109860 0x7fd3741a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd372a00700 1 -- 172.21.15.63:0/296703443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd374109860 msgr2=0x7fd3741a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd372a00700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd374109860 0x7fd3741a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd372a00700 1 -- 172.21.15.63:0/296703443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3741045e0 msgr2=0x7fd37419d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd372a00700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3741045e0 0x7fd37419d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd372a00700 1 -- 172.21.15.63:0/296703443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3741a1ef0 con 0x7fd374108f10 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.408+0000 7fd373e00700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd374109860 0x7fd3741a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:45.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.409+0000 7fd372a00700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd374108f10 0x7fd37419d5c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd36000e970 tx=0x7fd36000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:45.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.409+0000 7fd36be00700 1 -- 172.21.15.63:0/296703443 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36000cbd0 con 0x7fd374108f10 2024-05-27T07:44:45.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.409+0000 7fd37a800700 1 -- 172.21.15.63:0/296703443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3741a21e0 con 0x7fd374108f10 2024-05-27T07:44:45.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.409+0000 7fd36be00700 1 -- 172.21.15.63:0/296703443 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd36000cd70 con 0x7fd374108f10 2024-05-27T07:44:45.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.409+0000 7fd36be00700 1 -- 172.21.15.63:0/296703443 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360005510 con 0x7fd374108f10 2024-05-27T07:44:45.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.409+0000 7fd37a800700 1 -- 172.21.15.63:0/296703443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3741a9b00 con 0x7fd374108f10 2024-05-27T07:44:45.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.415+0000 7fd389395b80 1 Processor -- start 2024-05-27T07:44:45.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.415+0000 7fd389395b80 1 -- start start 2024-05-27T07:44:45.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.415+0000 7fd36be00700 1 -- 172.21.15.63:0/296703443 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd360004750 con 0x7fd374108f10 2024-05-27T07:44:45.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.415+0000 7fd389395b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f326a8e40 0x564f3282d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.416+0000 7fd389395b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f3282dd30 0x564f32827870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.416+0000 7fd389395b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f328281c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.416+0000 7fd389395b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564f32755e40 con 0x564f3282dd30 2024-05-27T07:44:45.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.416+0000 7fd389395b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f327459e0 con 0x564f326a8e40 2024-05-27T07:44:45.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.416+0000 7fd389395b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564f32749a20 con 0x564f32827db0 2024-05-27T07:44:45.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.416+0000 7fd373400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f3282dd30 0x564f32827870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.416+0000 7fd373400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f3282dd30 0x564f32827870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38372/0 (socket says 172.21.15.63:38372) 2024-05-27T07:44:45.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.416+0000 7fd373400700 1 -- 172.21.15.63:0/1469834018 learned_addr learned my addr 172.21.15.63:0/1469834018 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:45.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.416+0000 7fd372a00700 1 --2- 172.21.15.63:0/1469834018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f328281c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.416+0000 7fd373e00700 1 --2- 172.21.15.63:0/1469834018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f326a8e40 0x564f3282d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.417+0000 7fd373400700 1 -- 172.21.15.63:0/1469834018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 msgr2=0x564f328281c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:45.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.417+0000 7fd373400700 1 --2- 172.21.15.63:0/1469834018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f328281c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.417+0000 7fd373400700 1 -- 172.21.15.63:0/1469834018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f326a8e40 msgr2=0x564f3282d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:45.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.417+0000 7fd373400700 1 --2- 172.21.15.63:0/1469834018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f326a8e40 0x564f3282d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.417+0000 7fd373400700 1 -- 172.21.15.63:0/1469834018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f3270e420 con 0x564f3282dd30 2024-05-27T07:44:45.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.417+0000 7fd372a00700 1 --2- 172.21.15.63:0/1469834018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f328281c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:45.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.417+0000 7fd373400700 1 --2- 172.21.15.63:0/1469834018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f3282dd30 0x564f32827870 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd364015000 tx=0x7fd364005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5119a593914df3c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:45.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.418+0000 7fd36be00700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3580b29f0 0x7fd3580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.418+0000 7fd368c00700 1 -- 172.21.15.63:0/1469834018 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364027070 con 0x564f3282dd30 2024-05-27T07:44:45.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.418+0000 7fd368c00700 1 -- 172.21.15.63:0/1469834018 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3640037c0 con 0x564f3282dd30 2024-05-27T07:44:45.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.418+0000 7fd368c00700 1 -- 172.21.15.63:0/1469834018 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36400b990 con 0x564f3282dd30 2024-05-27T07:44:45.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.418+0000 7fd36be00700 1 -- 172.21.15.63:0/296703443 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd3600e2d50 con 0x7fd374108f10 2024-05-27T07:44:45.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.418+0000 7fd373e00700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3580b29f0 0x7fd3580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.419+0000 7fd389395b80 1 -- 172.21.15.63:0/1469834018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f3282dd30 msgr2=0x564f32827870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:45.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.419+0000 7fd389395b80 1 --2- 172.21.15.63:0/1469834018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f3282dd30 0x564f32827870 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd364015000 tx=0x7fd364005040 comp rx=0 tx=0).stop 2024-05-27T07:44:45.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.419+0000 7fd373e00700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3580b29f0 0x7fd3580b4f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x564f32828b30 tx=0x7fd36c009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:45.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.419+0000 7fd389395b80 1 -- 172.21.15.63:0/1469834018 shutdown_connections 2024-05-27T07:44:45.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.419+0000 7fd389395b80 1 --2- 172.21.15.63:0/1469834018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f328281c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.419+0000 7fd389395b80 1 --2- 172.21.15.63:0/1469834018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f326a8e40 0x564f3282d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.419+0000 7fd389395b80 1 --2- 172.21.15.63:0/1469834018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f3282dd30 0x564f32827870 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.419+0000 7fd389395b80 1 -- 172.21.15.63:0/1469834018 >> 172.21.15.63:0/1469834018 conn(0x564f32708fc0 msgr2=0x564f328228d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.419+0000 7fd389395b80 1 -- 172.21.15.63:0/1469834018 shutdown_connections 2024-05-27T07:44:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.420+0000 7fd389395b80 1 -- 172.21.15.63:0/1469834018 wait complete. 2024-05-27T07:44:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.420+0000 7fd389395b80 1 -- start start 2024-05-27T07:44:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.421+0000 7fd389395b80 1 Processor -- start 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.421+0000 7fd389395b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f328314f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.421+0000 7fd389395b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f32831a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.421+0000 7fd389395b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f32831f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.421+0000 7fd389395b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564f3264bfa0 con 0x564f326a8e40 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.421+0000 7fd389395b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f326e6760 con 0x564f3282dd30 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.421+0000 7fd389395b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564f328347a0 con 0x564f32827db0 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.421+0000 7fd389395b80 10 client.0 _ll_register_callbacks cb 0x564f326efc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd373400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f328314f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd373400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f328314f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38382/0 (socket says 172.21.15.63:38382) 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd373400700 1 -- 172.21.15.63:0/3028017312 learned_addr learned my addr 172.21.15.63:0/3028017312 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd373e00700 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f32831f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd372a00700 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f32831a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd373400700 1 -- 172.21.15.63:0/3028017312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 msgr2=0x564f32831a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd373400700 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f32831a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd373400700 1 -- 172.21.15.63:0/3028017312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 msgr2=0x564f32831f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd373400700 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f32831f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd373400700 1 -- 172.21.15.63:0/3028017312 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f326958d0 con 0x564f326a8e40 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.422+0000 7fd373e00700 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f32831f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.423+0000 7fd373400700 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f328314f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd36400b580 tx=0x7fd3640050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21867b628203b7a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.423+0000 7fd35d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.423+0000 7fd35d600700 1 -- 172.21.15.63:0/3028017312 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364027050 con 0x564f326a8e40 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.423+0000 7fd389395b80 1 -- 172.21.15.63:0/3028017312 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3640037c0 con 0x564f326a8e40 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.423+0000 7fd35d600700 1 -- 172.21.15.63:0/3028017312 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd36400b790 con 0x564f326a8e40 2024-05-27T07:44:45.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.423+0000 7fd35d600700 1 -- 172.21.15.63:0/3028017312 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd36403c050 con 0x564f326a8e40 2024-05-27T07:44:45.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.424+0000 7fd389395b80 10 client.17562 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:45.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.424+0000 7fd389395b80 1 -- 172.21.15.63:0/3028017312 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564f32716060 con 0x564f326a8e40 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.424+0000 7fd35d600700 1 -- 172.21.15.63:0/3028017312 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3640037c0 con 0x564f326a8e40 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.424+0000 7fd35d600700 1 -- 172.21.15.63:0/3028017312 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd364021030 con 0x564f326a8e40 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.424+0000 7fd389395b80 10 client.17562 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.424+0000 7fd389395b80 10 client.17562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.425+0000 7fd389395b80 20 client.17562 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.425+0000 7fd389395b80 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564f32841550 0x564f32843970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.425+0000 7fd389395b80 4 client.17562 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.425+0000 7fd389395b80 1 -- 172.21.15.63:0/3028017312 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f32757d90 con 0x564f32841550 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.425+0000 7fd372a00700 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564f32841550 0x564f32843970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.426+0000 7fd372a00700 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564f32841550 0x564f32843970 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92404e44aea60edc server_cookie=9df9dd0606566352 in_seq=0 out_seq=0 2024-05-27T07:44:45.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.426+0000 7fd35d600700 10 client.17562 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:44:45.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.428+0000 7fd35d600700 1 -- 172.21.15.63:0/3028017312 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564f32757d90 con 0x564f32841550 2024-05-27T07:44:45.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.428+0000 7fd35d600700 10 client.17562 handle_command_reply: tid=0 2024-05-27T07:44:45.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.428+0000 7fd389395b80 1 client.17562 shutdown 2024-05-27T07:44:45.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 1 -- 172.21.15.63:0/3028017312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 msgr2=0x564f328314f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:45.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f328314f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd36400b580 tx=0x7fd3640050b0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 1 -- 172.21.15.63:0/3028017312 shutdown_connections 2024-05-27T07:44:45.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f32831f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564f32841550 0x564f32843970 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f328314f0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 1 --2- 172.21.15.63:0/3028017312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f32831a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd35d600700 0 client.17562 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:44:45.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 1 -- 172.21.15.63:0/3028017312 >> 172.21.15.63:0/3028017312 conn(0x564f32708fc0 msgr2=0x564f32822580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:45.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 1 -- 172.21.15.63:0/3028017312 shutdown_connections 2024-05-27T07:44:45.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 1 -- 172.21.15.63:0/3028017312 wait complete. 2024-05-27T07:44:45.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.429+0000 7fd389395b80 20 client.17562 trim_cache size 0 max 16384 2024-05-27T07:44:45.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.451+0000 7fd389395b80 1 Processor -- start 2024-05-27T07:44:45.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.451+0000 7fd389395b80 1 -- start start 2024-05-27T07:44:45.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.451+0000 7fd389395b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f32827870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.451+0000 7fd389395b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f32833990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.451+0000 7fd389395b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f3283da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.451+0000 7fd389395b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b5060 con 0x564f326a8e40 2024-05-27T07:44:45.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.451+0000 7fd389395b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b28f0 con 0x564f3282dd30 2024-05-27T07:44:45.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.451+0000 7fd389395b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b0760 con 0x564f32827db0 2024-05-27T07:44:45.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.452+0000 7fd373400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f32827870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.452+0000 7fd373400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f32827870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38390/0 (socket says 172.21.15.63:38390) 2024-05-27T07:44:45.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.452+0000 7fd373400700 1 -- 172.21.15.63:0/717774751 learned_addr learned my addr 172.21.15.63:0/717774751 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:45.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.452+0000 7fd372a00700 1 --2- 172.21.15.63:0/717774751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f32833990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.452+0000 7fd373e00700 1 --2- 172.21.15.63:0/717774751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f3283da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.452+0000 7fd373400700 1 -- 172.21.15.63:0/717774751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 msgr2=0x564f32833990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:45.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.452+0000 7fd373400700 1 --2- 172.21.15.63:0/717774751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f32833990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.452+0000 7fd373400700 1 -- 172.21.15.63:0/717774751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 msgr2=0x564f3283da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:45.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.452+0000 7fd373400700 1 --2- 172.21.15.63:0/717774751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f3283da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.452+0000 7fd373400700 1 -- 172.21.15.63:0/717774751 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f328325b0 con 0x564f326a8e40 2024-05-27T07:44:45.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.453+0000 7fd373400700 1 --2- 172.21.15.63:0/717774751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f32827870 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd3640098e0 tx=0x7fd364001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf5754b39fc17c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:45.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.453+0000 7fd35fe00700 1 -- 172.21.15.63:0/717774751 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364027070 con 0x564f326a8e40 2024-05-27T07:44:45.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.453+0000 7fd35fe00700 1 -- 172.21.15.63:0/717774751 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd364033070 con 0x564f326a8e40 2024-05-27T07:44:45.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.453+0000 7fd35fe00700 1 -- 172.21.15.63:0/717774751 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36400bde0 con 0x564f326a8e40 2024-05-27T07:44:45.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.454+0000 7fd389395b80 1 -- 172.21.15.63:0/717774751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 msgr2=0x564f32827870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:45.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.454+0000 7fd389395b80 1 --2- 172.21.15.63:0/717774751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f32827870 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd3640098e0 tx=0x7fd364001e50 comp rx=0 tx=0).stop 2024-05-27T07:44:45.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.454+0000 7fd389395b80 1 -- 172.21.15.63:0/717774751 shutdown_connections 2024-05-27T07:44:45.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.454+0000 7fd389395b80 1 --2- 172.21.15.63:0/717774751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f3283da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.454+0000 7fd389395b80 1 --2- 172.21.15.63:0/717774751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f326a8e40 0x564f32827870 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.454+0000 7fd389395b80 1 --2- 172.21.15.63:0/717774751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f32827db0 0x564f32833990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.454+0000 7fd389395b80 1 -- 172.21.15.63:0/717774751 >> 172.21.15.63:0/717774751 conn(0x564f32708fc0 msgr2=0x564f328312d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.455+0000 7fd389395b80 1 -- 172.21.15.63:0/717774751 shutdown_connections 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.455+0000 7fd389395b80 1 -- 172.21.15.63:0/717774751 wait complete. 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.455+0000 7fd389395b80 1 -- start start 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.455+0000 7fd389395b80 1 Processor -- start 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.456+0000 7fd389395b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f326a8e40 0x564f328ab920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.456+0000 7fd389395b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f32827db0 0x564f328abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.456+0000 7fd389395b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f32843f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.456+0000 7fd389395b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b0db0 con 0x564f32827db0 2024-05-27T07:44:45.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.456+0000 7fd389395b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b2220 con 0x564f3282dd30 2024-05-27T07:44:45.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.456+0000 7fd389395b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741b1480 con 0x564f326a8e40 2024-05-27T07:44:45.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.456+0000 7fd389395b80 10 client.0 _ll_register_callbacks cb 0x564f326efc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:45.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.456+0000 7fd372a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f32827db0 0x564f328abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.456+0000 7fd372a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f32827db0 0x564f328abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38394/0 (socket says 172.21.15.63:38394) 2024-05-27T07:44:45.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd372a00700 1 -- 172.21.15.63:0/889788886 learned_addr learned my addr 172.21.15.63:0/889788886 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:45.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd373e00700 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f32843f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd373400700 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f326a8e40 0x564f328ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd372a00700 1 -- 172.21.15.63:0/889788886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f326a8e40 msgr2=0x564f328ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:45.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd372a00700 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f326a8e40 0x564f328ab920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd372a00700 1 -- 172.21.15.63:0/889788886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 msgr2=0x564f32843f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd372a00700 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f32843f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd372a00700 1 -- 172.21.15.63:0/889788886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f32844650 con 0x564f32827db0 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd373400700 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f326a8e40 0x564f328ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd372a00700 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f32827db0 0x564f328abe60 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd3600a6ee0 tx=0x7fd360000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9c5e58b57d3add server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.457+0000 7fd35d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.458+0000 7fd35d600700 1 -- 172.21.15.63:0/889788886 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36008e050 con 0x564f32827db0 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.458+0000 7fd35d600700 1 -- 172.21.15.63:0/889788886 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd3600a9ed0 con 0x564f32827db0 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.458+0000 7fd389395b80 1 -- 172.21.15.63:0/889788886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd364033070 con 0x564f32827db0 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.458+0000 7fd35d600700 1 -- 172.21.15.63:0/889788886 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd3600a5400 con 0x564f32827db0 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.458+0000 7fd389395b80 10 client.17568 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.458+0000 7fd389395b80 1 -- 172.21.15.63:0/889788886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564f328b4020 con 0x564f32827db0 2024-05-27T07:44:45.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.459+0000 7fd35d600700 1 -- 172.21.15.63:0/889788886 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd364033070 con 0x564f32827db0 2024-05-27T07:44:45.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.459+0000 7fd35d600700 1 -- 172.21.15.63:0/889788886 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd3600b2640 con 0x564f32827db0 2024-05-27T07:44:45.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.459+0000 7fd389395b80 10 client.17568 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:45.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.459+0000 7fd389395b80 10 client.17568 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:44:45.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.459+0000 7fd389395b80 20 client.17568 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:45.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.459+0000 7fd389395b80 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564f328b4730 0x564f328b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:45.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.459+0000 7fd389395b80 4 client.17568 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:44:45.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.459+0000 7fd389395b80 1 -- 172.21.15.63:0/889788886 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f32751570 con 0x564f328b4730 2024-05-27T07:44:45.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.460+0000 7fd373400700 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564f328b4730 0x564f328b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:45.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.460+0000 7fd373400700 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564f328b4730 0x564f328b6b50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a121c1bcda0ddad server_cookie=1277556267306a91 in_seq=0 out_seq=0 2024-05-27T07:44:45.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.460+0000 7fd35d600700 10 client.17568 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:44:45.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd35d600700 1 -- 172.21.15.63:0/889788886 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f32751570 con 0x564f328b4730 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd35d600700 10 client.17568 handle_command_reply: tid=0 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd389395b80 1 client.17568 shutdown 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd389395b80 1 -- 172.21.15.63:0/889788886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f32827db0 msgr2=0x564f328abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd389395b80 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f32827db0 0x564f328abe60 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd3600a6ee0 tx=0x7fd360000bc0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd389395b80 1 -- 172.21.15.63:0/889788886 shutdown_connections 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd389395b80 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f3282dd30 0x564f32843f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd389395b80 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f326a8e40 0x564f328ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd389395b80 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564f328b4730 0x564f328b6b50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd389395b80 1 --2- 172.21.15.63:0/889788886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f32827db0 0x564f328abe60 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd389395b80 1 -- 172.21.15.63:0/889788886 >> 172.21.15.63:0/889788886 conn(0x564f32708fc0 msgr2=0x564f32831d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.461+0000 7fd35d600700 0 client.17568 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:44:45.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.462+0000 7fd389395b80 1 -- 172.21.15.63:0/889788886 shutdown_connections 2024-05-27T07:44:45.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.462+0000 7fd389395b80 1 -- 172.21.15.63:0/889788886 wait complete. 2024-05-27T07:44:45.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.462+0000 7fd389395b80 20 client.17568 trim_cache size 0 max 16384 2024-05-27T07:44:45.465 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:44:45.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 -- 172.21.15.63:0/296703443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3580b29f0 msgr2=0x7fd3580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:45.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3580b29f0 0x7fd3580b4f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x564f32828b30 tx=0x7fd36c009040 comp rx=0 tx=0).stop 2024-05-27T07:44:45.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 -- 172.21.15.63:0/296703443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd374108f10 msgr2=0x7fd37419d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:45.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd374108f10 0x7fd37419d5c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd36000e970 tx=0x7fd36000ec80 comp rx=0 tx=0).stop 2024-05-27T07:44:45.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 -- 172.21.15.63:0/296703443 shutdown_connections 2024-05-27T07:44:45.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3580b29f0 0x7fd3580b4f00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd374109860 0x7fd3741a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3741045e0 0x7fd37419d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 --2- 172.21.15.63:0/296703443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd374108f10 0x7fd37419d5c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:45.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 -- 172.21.15.63:0/296703443 >> 172.21.15.63:0/296703443 conn(0x7fd374100030 msgr2=0x7fd37410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:45.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 -- 172.21.15.63:0/296703443 shutdown_connections 2024-05-27T07:44:45.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:45.463+0000 7fd37a800700 1 -- 172.21.15.63:0/296703443 wait complete. 2024-05-27T07:44:46.458 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:44:46.981 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:47.144 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:46 smithi063 ceph-mon[31613]: pgmap v349: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:44:47.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:46 smithi086 ceph-mon[34687]: pgmap v349: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:44:47.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:46 smithi149 ceph-mon[35408]: pgmap v349: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:44:48.960 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:48 smithi063 ceph-mon[31613]: pgmap v350: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:44:49.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:48 smithi086 ceph-mon[34687]: pgmap v350: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:44:49.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:48 smithi149 ceph-mon[35408]: pgmap v350: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:44:49.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.581+0000 7fa3e7200700 1 -- 172.21.15.63:0/693559199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e8104400 msgr2=0x7fa3e8104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.581+0000 7fa3e7200700 1 --2- 172.21.15.63:0/693559199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e8104400 0x7fa3e8104810 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80099c0 tx=0x7fa3d8009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.581+0000 7fa3e7200700 1 -- 172.21.15.63:0/693559199 shutdown_connections 2024-05-27T07:44:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.581+0000 7fa3e7200700 1 --2- 172.21.15.63:0/693559199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e8109680 0x7fa3e8110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.581+0000 7fa3e7200700 1 --2- 172.21.15.63:0/693559199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e8104400 0x7fa3e8104810 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.581+0000 7fa3e7200700 1 --2- 172.21.15.63:0/693559199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e8108d30 0x7fa3e8109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.581+0000 7fa3e7200700 1 -- 172.21.15.63:0/693559199 >> 172.21.15.63:0/693559199 conn(0x7fa3e8079d40 msgr2=0x7fa3e807a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.581+0000 7fa3e7200700 1 -- 172.21.15.63:0/693559199 shutdown_connections 2024-05-27T07:44:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.581+0000 7fa3e7200700 1 -- 172.21.15.63:0/693559199 wait complete. 2024-05-27T07:44:49.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.583+0000 7fa3e7200700 1 Processor -- start 2024-05-27T07:44:49.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.583+0000 7fa3e7200700 1 -- start start 2024-05-27T07:44:49.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e7200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e8104400 0x7fa3e819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e8108d30 0x7fa3e819d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e7200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e8109680 0x7fa3e81a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e7200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81141b0 con 0x7fa3e8109680 2024-05-27T07:44:49.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e8114030 con 0x7fa3e8108d30 2024-05-27T07:44:49.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e7200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e8114330 con 0x7fa3e8104400 2024-05-27T07:44:49.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e8109680 0x7fa3e81a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e8108d30 0x7fa3e819d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e8104400 0x7fa3e819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e8104400 0x7fa3e819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41702/0 (socket says 172.21.15.63:41702) 2024-05-27T07:44:49.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.584+0000 7fa3e5e00700 1 -- 172.21.15.63:0/789184025 learned_addr learned my addr 172.21.15.63:0/789184025 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:49.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.585+0000 7fa3e6800700 1 -- 172.21.15.63:0/789184025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e8104400 msgr2=0x7fa3e819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:49.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.585+0000 7fa3e6800700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e8104400 0x7fa3e819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.585+0000 7fa3e6800700 1 -- 172.21.15.63:0/789184025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e8108d30 msgr2=0x7fa3e819d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:49.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.585+0000 7fa3e6800700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e8108d30 0x7fa3e819d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.585+0000 7fa3e6800700 1 -- 172.21.15.63:0/789184025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e81a1b80 con 0x7fa3e8109680 2024-05-27T07:44:49.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.586+0000 7fa3e6800700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e8109680 0x7fa3e81a1590 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0007f10 tx=0x7fa3e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:49.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.586+0000 7fa3dea00700 1 -- 172.21.15.63:0/789184025 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0004db0 con 0x7fa3e8109680 2024-05-27T07:44:49.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.586+0000 7fa3dea00700 1 -- 172.21.15.63:0/789184025 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3e0009e20 con 0x7fa3e8109680 2024-05-27T07:44:49.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.586+0000 7fa3dea00700 1 -- 172.21.15.63:0/789184025 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0015400 con 0x7fa3e8109680 2024-05-27T07:44:49.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.586+0000 7fa3e7200700 1 -- 172.21.15.63:0/789184025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e81a1e70 con 0x7fa3e8109680 2024-05-27T07:44:49.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.586+0000 7fa3e7200700 1 -- 172.21.15.63:0/789184025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e81a2360 con 0x7fa3e8109680 2024-05-27T07:44:49.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.590+0000 7fa3dea00700 1 -- 172.21.15.63:0/789184025 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa3e0005240 con 0x7fa3e8109680 2024-05-27T07:44:49.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.592+0000 7fa3dea00700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3cc0b2a70 0x7fa3cc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.592+0000 7fa3fb750b80 1 Processor -- start 2024-05-27T07:44:49.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.592+0000 7fa3dea00700 1 -- 172.21.15.63:0/789184025 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fa3e00e22f0 con 0x7fa3e8109680 2024-05-27T07:44:49.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.592+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3cc0b2a70 0x7fa3cc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.593+0000 7fa3fb750b80 1 -- start start 2024-05-27T07:44:49.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.593+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.593+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e8a3410 0x55858e8adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.593+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e8b0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.593+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55858e7d1e40 con 0x55858e724e40 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.593+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55858e7c19e0 con 0x55858e8a3410 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.593+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55858e7c5a20 con 0x55858e8a3820 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3cc0b2a70 0x7fa3cc0b4f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009750 tx=0x7fa3d8005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8a2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50562/0 (socket says 172.21.15.63:50562) 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e5e00700 1 -- 172.21.15.63:0/373623653 learned_addr learned my addr 172.21.15.63:0/373623653 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e5400700 1 --2- 172.21.15.63:0/373623653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e8a3410 0x55858e8adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e6800700 1 --2- 172.21.15.63:0/373623653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e8b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e5e00700 1 -- 172.21.15.63:0/373623653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 msgr2=0x55858e8b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/373623653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e8b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e5e00700 1 -- 172.21.15.63:0/373623653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e8a3410 msgr2=0x55858e8adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:49.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/373623653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e8a3410 0x55858e8adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.594+0000 7fa3e5e00700 1 -- 172.21.15.63:0/373623653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858e78a420 con 0x55858e724e40 2024-05-27T07:44:49.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.595+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/373623653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8a2ed0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005630 tx=0x7fa3d801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbdf518c3566c6d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:49.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.595+0000 7fa3d3400700 1 -- 172.21.15.63:0/373623653 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8005660 con 0x55858e724e40 2024-05-27T07:44:49.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.595+0000 7fa3d3400700 1 -- 172.21.15.63:0/373623653 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3d8003b10 con 0x55858e724e40 2024-05-27T07:44:49.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.595+0000 7fa3d3400700 1 -- 172.21.15.63:0/373623653 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d802f9b0 con 0x55858e724e40 2024-05-27T07:44:49.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.597+0000 7fa3fb750b80 1 -- 172.21.15.63:0/373623653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 msgr2=0x55858e8a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:49.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.597+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/373623653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8a2ed0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005630 tx=0x7fa3d801e000 comp rx=0 tx=0).stop 2024-05-27T07:44:49.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.597+0000 7fa3fb750b80 1 -- 172.21.15.63:0/373623653 shutdown_connections 2024-05-27T07:44:49.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.597+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/373623653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e8b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.597+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/373623653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8a2ed0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.597+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/373623653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e8a3410 0x55858e8adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.597+0000 7fa3fb750b80 1 -- 172.21.15.63:0/373623653 >> 172.21.15.63:0/373623653 conn(0x55858e784fc0 msgr2=0x55858e89ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:49.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.597+0000 7fa3fb750b80 1 -- 172.21.15.63:0/373623653 shutdown_connections 2024-05-27T07:44:49.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.597+0000 7fa3fb750b80 1 -- 172.21.15.63:0/373623653 wait complete. 2024-05-27T07:44:49.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.598+0000 7fa3fb750b80 1 -- start start 2024-05-27T07:44:49.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.598+0000 7fa3fb750b80 1 Processor -- start 2024-05-27T07:44:49.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e724e40 0x55858e8ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e8ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55858e6c7fa0 con 0x55858e8a3410 2024-05-27T07:44:49.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55858e762760 con 0x55858e724e40 2024-05-27T07:44:49.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55858e8b0930 con 0x55858e8a3820 2024-05-27T07:44:49.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3fb750b80 10 client.0 _ll_register_callbacks cb 0x55858e76bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:49.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3e5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3e5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50566/0 (socket says 172.21.15.63:50566) 2024-05-27T07:44:49.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3e5400700 1 -- 172.21.15.63:0/2466808189 learned_addr learned my addr 172.21.15.63:0/2466808189 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:49.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e724e40 0x55858e8ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.599+0000 7fa3e6800700 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e8ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.600+0000 7fa3e5400700 1 -- 172.21.15.63:0/2466808189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 msgr2=0x55858e8ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:49.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.600+0000 7fa3e5400700 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e8ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.600+0000 7fa3e5400700 1 -- 172.21.15.63:0/2466808189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e724e40 msgr2=0x55858e8ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:49.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.600+0000 7fa3e5400700 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e724e40 0x55858e8ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.600+0000 7fa3e5400700 1 -- 172.21.15.63:0/2466808189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858e7118d0 con 0x55858e8a3410 2024-05-27T07:44:49.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.600+0000 7fa3e5400700 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8adb20 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4010a50 tx=0x7fa3d4010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4d8100eb6b2e06 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:49.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.600+0000 7fa3c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.600+0000 7fa3c7e00700 1 -- 172.21.15.63:0/2466808189 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4004320 con 0x55858e8a3410 2024-05-27T07:44:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.601+0000 7fa3c7e00700 1 -- 172.21.15.63:0/2466808189 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fa3d40044c0 con 0x55858e8a3410 2024-05-27T07:44:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.601+0000 7fa3fb750b80 1 -- 172.21.15.63:0/2466808189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d8003b10 con 0x55858e8a3410 2024-05-27T07:44:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.601+0000 7fa3c7e00700 1 -- 172.21.15.63:0/2466808189 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa3d400f890 con 0x55858e8a3410 2024-05-27T07:44:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.601+0000 7fa3fb750b80 10 client.17580 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:49.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.601+0000 7fa3fb750b80 1 -- 172.21.15.63:0/2466808189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55858e792060 con 0x55858e8a3410 2024-05-27T07:44:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.601+0000 7fa3c7e00700 1 -- 172.21.15.63:0/2466808189 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3d8003b10 con 0x55858e8a3410 2024-05-27T07:44:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.601+0000 7fa3c7e00700 1 -- 172.21.15.63:0/2466808189 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa3d4003ca0 con 0x55858e8a3410 2024-05-27T07:44:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.602+0000 7fa3fb750b80 10 client.17580 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.602+0000 7fa3fb750b80 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.602+0000 7fa3fb750b80 20 client.17580 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.602+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55858e8bd9c0 0x55858e8bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.602+0000 7fa3fb750b80 4 client.17580 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:44:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.602+0000 7fa3fb750b80 1 -- 172.21.15.63:0/2466808189 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55858e7d3d90 con 0x55858e8bd9c0 2024-05-27T07:44:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.603+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55858e8bd9c0 0x55858e8bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.604+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55858e8bd9c0 0x55858e8bfe50 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9316243358ef33ce server_cookie=e4cd63173d18f21 in_seq=0 out_seq=0 2024-05-27T07:44:49.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.604+0000 7fa3c7e00700 10 client.17580 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.605+0000 7fa3c7e00700 1 -- 172.21.15.63:0/2466808189 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55858e7d3d90 con 0x55858e8bd9c0 2024-05-27T07:44:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.605+0000 7fa3c7e00700 10 client.17580 handle_command_reply: tid=0 2024-05-27T07:44:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 client.17580 shutdown 2024-05-27T07:44:49.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 -- 172.21.15.63:0/2466808189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 msgr2=0x55858e8adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8adb20 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4010a50 tx=0x7fa3d4010d60 comp rx=0 tx=0).stop 2024-05-27T07:44:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 -- 172.21.15.63:0/2466808189 shutdown_connections 2024-05-27T07:44:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55858e8bd9c0 0x55858e8bfe50 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e8ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e724e40 0x55858e8ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/2466808189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8adb20 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 -- 172.21.15.63:0/2466808189 >> 172.21.15.63:0/2466808189 conn(0x55858e784fc0 msgr2=0x55858e89dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3c7e00700 0 client.17580 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:49.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 -- 172.21.15.63:0/2466808189 shutdown_connections 2024-05-27T07:44:49.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.606+0000 7fa3fb750b80 1 -- 172.21.15.63:0/2466808189 wait complete. 2024-05-27T07:44:49.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.607+0000 7fa3fb750b80 20 client.17580 trim_cache size 0 max 16384 2024-05-27T07:44:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.632+0000 7fa3fb750b80 1 Processor -- start 2024-05-27T07:44:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.632+0000 7fa3fb750b80 1 -- start start 2024-05-27T07:44:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.632+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e724e40 0x55858e8a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.632+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8b9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.632+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e89e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.632+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81b4d70 con 0x55858e8a3410 2024-05-27T07:44:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.632+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81b2600 con 0x55858e724e40 2024-05-27T07:44:49.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.632+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81b0470 con 0x55858e8a3820 2024-05-27T07:44:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.633+0000 7fa3e5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8b9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.633+0000 7fa3e5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8b9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50578/0 (socket says 172.21.15.63:50578) 2024-05-27T07:44:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.633+0000 7fa3e5400700 1 -- 172.21.15.63:0/1855891682 learned_addr learned my addr 172.21.15.63:0/1855891682 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.633+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/1855891682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e724e40 0x55858e8a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.633+0000 7fa3e6800700 1 --2- 172.21.15.63:0/1855891682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e89e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.633+0000 7fa3e5400700 1 -- 172.21.15.63:0/1855891682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 msgr2=0x55858e89e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.633+0000 7fa3e5400700 1 --2- 172.21.15.63:0/1855891682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e89e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.633+0000 7fa3e5400700 1 -- 172.21.15.63:0/1855891682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e724e40 msgr2=0x55858e8a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.633+0000 7fa3e5400700 1 --2- 172.21.15.63:0/1855891682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e724e40 0x55858e8a8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.634+0000 7fa3e5400700 1 -- 172.21.15.63:0/1855891682 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858e8991d0 con 0x55858e8a3410 2024-05-27T07:44:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.634+0000 7fa3e6800700 1 --2- 172.21.15.63:0/1855891682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e89e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.634+0000 7fa3e5400700 1 --2- 172.21.15.63:0/1855891682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8b9a30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4000c00 tx=0x7fa3d400a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e313a3cab081b318 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.634+0000 7fa3d2a00700 1 -- 172.21.15.63:0/1855891682 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d402c070 con 0x55858e8a3410 2024-05-27T07:44:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.634+0000 7fa3d2a00700 1 -- 172.21.15.63:0/1855891682 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3d4004320 con 0x55858e8a3410 2024-05-27T07:44:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.634+0000 7fa3d2a00700 1 -- 172.21.15.63:0/1855891682 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4003680 con 0x55858e8a3410 2024-05-27T07:44:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.635+0000 7fa3fb750b80 1 -- 172.21.15.63:0/1855891682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 msgr2=0x55858e8b9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.635+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/1855891682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8b9a30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4000c00 tx=0x7fa3d400a470 comp rx=0 tx=0).stop 2024-05-27T07:44:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.636+0000 7fa3fb750b80 1 -- 172.21.15.63:0/1855891682 shutdown_connections 2024-05-27T07:44:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.636+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/1855891682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3820 0x55858e89e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.636+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/1855891682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e724e40 0x55858e8a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.636+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/1855891682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e8a3410 0x55858e8b9a30 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.636+0000 7fa3fb750b80 1 -- 172.21.15.63:0/1855891682 >> 172.21.15.63:0/1855891682 conn(0x55858e784fc0 msgr2=0x55858e89dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.636+0000 7fa3fb750b80 1 -- 172.21.15.63:0/1855891682 shutdown_connections 2024-05-27T07:44:49.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.636+0000 7fa3fb750b80 1 -- 172.21.15.63:0/1855891682 wait complete. 2024-05-27T07:44:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.636+0000 7fa3fb750b80 1 -- start start 2024-05-27T07:44:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.637+0000 7fa3fb750b80 1 Processor -- start 2024-05-27T07:44:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.637+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.637+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3410 0x55858e8b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.637+0000 7fa3fb750b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e8a3820 0x55858e926520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.638+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81b0ac0 con 0x55858e724e40 2024-05-27T07:44:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.638+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81b1f30 con 0x55858e8a3820 2024-05-27T07:44:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.638+0000 7fa3fb750b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81b1190 con 0x55858e8a3410 2024-05-27T07:44:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.638+0000 7fa3fb750b80 10 client.0 _ll_register_callbacks cb 0x55858e76bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.638+0000 7fa3e5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.638+0000 7fa3e5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50586/0 (socket says 172.21.15.63:50586) 2024-05-27T07:44:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.638+0000 7fa3e5e00700 1 -- 172.21.15.63:0/3920191684 learned_addr learned my addr 172.21.15.63:0/3920191684 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.638+0000 7fa3e5400700 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3410 0x55858e8b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.638+0000 7fa3e6800700 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e8a3820 0x55858e926520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3e5e00700 1 -- 172.21.15.63:0/3920191684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3410 msgr2=0x55858e8b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3410 0x55858e8b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3e5e00700 1 -- 172.21.15.63:0/3920191684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e8a3820 msgr2=0x55858e926520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e8a3820 0x55858e926520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3e5e00700 1 -- 172.21.15.63:0/3920191684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858e8a0350 con 0x55858e724e40 2024-05-27T07:44:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3e5400700 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3410 0x55858e8b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3e5e00700 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8b11c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005630 tx=0x7fa3d800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c899a7bc9fc8c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3c7e00700 1 -- 172.21.15.63:0/3920191684 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d803c020 con 0x55858e724e40 2024-05-27T07:44:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3c7e00700 1 -- 172.21.15.63:0/3920191684 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fa3d802f8d0 con 0x55858e724e40 2024-05-27T07:44:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.639+0000 7fa3fb750b80 1 -- 172.21.15.63:0/3920191684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d4004320 con 0x55858e724e40 2024-05-27T07:44:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.640+0000 7fa3c7e00700 1 -- 172.21.15.63:0/3920191684 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa3d8047050 con 0x55858e724e40 2024-05-27T07:44:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.640+0000 7fa3fb750b80 10 client.17586 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:49.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.640+0000 7fa3fb750b80 1 -- 172.21.15.63:0/3920191684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55858e927a80 con 0x55858e724e40 2024-05-27T07:44:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.640+0000 7fa3c7e00700 1 -- 172.21.15.63:0/3920191684 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3d4004320 con 0x55858e724e40 2024-05-27T07:44:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.641+0000 7fa3c7e00700 1 -- 172.21.15.63:0/3920191684 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa3d8022070 con 0x55858e724e40 2024-05-27T07:44:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.641+0000 7fa3fb750b80 10 client.17586 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.641+0000 7fa3fb750b80 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.641+0000 7fa3fb750b80 20 client.17586 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.641+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55858e927cb0 0x55858e92a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.641+0000 7fa3fb750b80 4 client.17586 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:44:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.641+0000 7fa3fb750b80 1 -- 172.21.15.63:0/3920191684 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55858e7cd570 con 0x55858e927cb0 2024-05-27T07:44:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.641+0000 7fa3e5400700 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55858e927cb0 0x55858e92a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.642+0000 7fa3e5400700 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55858e927cb0 0x55858e92a0d0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32fa2ee34250afda server_cookie=67974e2e5ebfd5cc in_seq=0 out_seq=0 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.642+0000 7fa3c7e00700 10 client.17586 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3c7e00700 1 -- 172.21.15.63:0/3920191684 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55858e7cd570 con 0x55858e927cb0 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3c7e00700 10 client.17586 handle_command_reply: tid=0 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3fb750b80 1 client.17586 shutdown 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3fb750b80 1 -- 172.21.15.63:0/3920191684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 msgr2=0x55858e8b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8b11c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005630 tx=0x7fa3d800bfd0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3fb750b80 1 -- 172.21.15.63:0/3920191684 shutdown_connections 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55858e927cb0 0x55858e92a0d0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55858e8a3820 0x55858e926520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55858e724e40 0x55858e8b11c0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3fb750b80 1 --2- 172.21.15.63:0/3920191684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55858e8a3410 0x55858e8b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3fb750b80 1 -- 172.21.15.63:0/3920191684 >> 172.21.15.63:0/3920191684 conn(0x55858e784fc0 msgr2=0x55858e89ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:49.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.643+0000 7fa3c7e00700 0 client.17586 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:49.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.644+0000 7fa3fb750b80 1 -- 172.21.15.63:0/3920191684 shutdown_connections 2024-05-27T07:44:49.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.644+0000 7fa3fb750b80 1 -- 172.21.15.63:0/3920191684 wait complete. 2024-05-27T07:44:49.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.644+0000 7fa3fb750b80 20 client.17586 trim_cache size 0 max 16384 2024-05-27T07:44:49.646 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:44:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 -- 172.21.15.63:0/789184025 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3cc0b2a70 msgr2=0x7fa3cc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3cc0b2a70 0x7fa3cc0b4f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009750 tx=0x7fa3d8005fb0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 -- 172.21.15.63:0/789184025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e8109680 msgr2=0x7fa3e81a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e8109680 0x7fa3e81a1590 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0007f10 tx=0x7fa3e000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 -- 172.21.15.63:0/789184025 shutdown_connections 2024-05-27T07:44:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3cc0b2a70 0x7fa3cc0b4f80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e8109680 0x7fa3e81a1590 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e8104400 0x7fa3e819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 --2- 172.21.15.63:0/789184025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e8108d30 0x7fa3e819d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 -- 172.21.15.63:0/789184025 >> 172.21.15.63:0/789184025 conn(0x7fa3e8079d40 msgr2=0x7fa3e81014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 -- 172.21.15.63:0/789184025 shutdown_connections 2024-05-27T07:44:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:49.645+0000 7fa3e7200700 1 -- 172.21.15.63:0/789184025 wait complete. 2024-05-27T07:44:50.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:50 smithi063 ceph-mon[31613]: pgmap v351: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:44:51.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:50 smithi086 ceph-mon[34687]: pgmap v351: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:44:51.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:50 smithi149 ceph-mon[35408]: pgmap v351: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:44:51.637 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:44:51.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:44:52.149 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.237+0000 7fd34d000640 1 -- 172.21.15.63:0/1866545471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd34810d9c0 msgr2=0x7fd34810fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.237+0000 7fd34d000640 1 --2- 172.21.15.63:0/1866545471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd34810d9c0 0x7fd34810fde0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fd334009900 tx=0x7fd33402a480 comp rx=0 tx=0).stop 2024-05-27T07:44:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.238+0000 7fd34d000640 1 -- 172.21.15.63:0/1866545471 shutdown_connections 2024-05-27T07:44:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.238+0000 7fd34d000640 1 --2- 172.21.15.63:0/1866545471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd348110320 0x7fd348112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.238+0000 7fd34d000640 1 --2- 172.21.15.63:0/1866545471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd34810d9c0 0x7fd34810fde0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.238+0000 7fd34d000640 1 --2- 172.21.15.63:0/1866545471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd348075420 0x7fd348075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.238+0000 7fd34d000640 1 -- 172.21.15.63:0/1866545471 >> 172.21.15.63:0/1866545471 conn(0x7fd348079100 msgr2=0x7fd348079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.238+0000 7fd34d000640 1 -- 172.21.15.63:0/1866545471 shutdown_connections 2024-05-27T07:44:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.238+0000 7fd34d000640 1 -- 172.21.15.63:0/1866545471 wait complete. 2024-05-27T07:44:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.239+0000 7fd34d000640 1 Processor -- start 2024-05-27T07:44:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.239+0000 7fd34d000640 1 -- start start 2024-05-27T07:44:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd34d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd348075420 0x7fd3481a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd34d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd34810d9c0 0x7fd3481a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd34d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd348110320 0x7fd34819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd34d000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd348115d90 con 0x7fd348110320 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd34d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd348115c10 con 0x7fd348075420 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd34d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd348115f10 con 0x7fd34810d9c0 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd347e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd348110320 0x7fd34819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd347e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd348110320 0x7fd34819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50600/0 (socket says 172.21.15.63:50600) 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd347e00640 1 -- 172.21.15.63:0/1588749396 learned_addr learned my addr 172.21.15.63:0/1588749396 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd346a00640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd34810d9c0 0x7fd3481a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.240+0000 7fd347400640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd348075420 0x7fd3481a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.241+0000 7fd347e00640 1 -- 172.21.15.63:0/1588749396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd34810d9c0 msgr2=0x7fd3481a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.241+0000 7fd347e00640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd34810d9c0 0x7fd3481a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.241+0000 7fd347e00640 1 -- 172.21.15.63:0/1588749396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd348075420 msgr2=0x7fd3481a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.241+0000 7fd347e00640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd348075420 0x7fd3481a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.241+0000 7fd347e00640 1 -- 172.21.15.63:0/1588749396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34819ccb0 con 0x7fd348110320 2024-05-27T07:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.241+0000 7fd347400640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd348075420 0x7fd3481a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.241+0000 7fd347e00640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd348110320 0x7fd34819c670 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd34000eaa0 tx=0x7fd34000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.241+0000 7fd33fe00640 1 -- 172.21.15.63:0/1588749396 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34000cdb0 con 0x7fd348110320 2024-05-27T07:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.241+0000 7fd34d000640 1 -- 172.21.15.63:0/1588749396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34819cfa0 con 0x7fd348110320 2024-05-27T07:44:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.242+0000 7fd33fe00640 1 -- 172.21.15.63:0/1588749396 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd340005020 con 0x7fd348110320 2024-05-27T07:44:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.242+0000 7fd34d000640 1 -- 172.21.15.63:0/1588749396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34819d4c0 con 0x7fd348110320 2024-05-27T07:44:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.242+0000 7fd33fe00640 1 -- 172.21.15.63:0/1588749396 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3400116d0 con 0x7fd348110320 2024-05-27T07:44:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.245+0000 7fd355780740 1 Processor -- start 2024-05-27T07:44:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.246+0000 7fd33fe00640 1 -- 172.21.15.63:0/1588749396 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd340011870 con 0x7fd348110320 2024-05-27T07:44:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.246+0000 7fd355780740 1 -- start start 2024-05-27T07:44:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd355780740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66ba91b0 0x55ae66f4ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd355780740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f4bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd355780740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd355780740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ae66ebcb90 con 0x55ae66b884d0 2024-05-27T07:44:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd355780740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae66eac730 con 0x55ae66b802f0 2024-05-27T07:44:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd355780740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ae66eb0770 con 0x55ae66ba91b0 2024-05-27T07:44:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd346a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd346a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50616/0 (socket says 172.21.15.63:50616) 2024-05-27T07:44:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd346a00640 1 -- 172.21.15.63:0/3453530494 learned_addr learned my addr 172.21.15.63:0/3453530494 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd346a00640 1 --2- 172.21.15.63:0/3453530494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66ba91b0 0x55ae66f4ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.247+0000 7fd347400640 1 --2- 172.21.15.63:0/3453530494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f4bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.248+0000 7fd346a00640 1 -- 172.21.15.63:0/3453530494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66ba91b0 msgr2=0x55ae66f4ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.248+0000 7fd346a00640 1 --2- 172.21.15.63:0/3453530494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66ba91b0 0x55ae66f4ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.248+0000 7fd33fe00640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd32c0b2950 0x7fd32c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.248+0000 7fd346a00640 1 -- 172.21.15.63:0/3453530494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 msgr2=0x55ae66f4bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.248+0000 7fd346a00640 1 --2- 172.21.15.63:0/3453530494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f4bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.248+0000 7fd346a00640 1 -- 172.21.15.63:0/3453530494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae66e458b0 con 0x55ae66b884d0 2024-05-27T07:44:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.248+0000 7fd33fe00640 1 -- 172.21.15.63:0/1588749396 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd3400e2ff0 con 0x7fd348110320 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.248+0000 7fd347e00640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd32c0b2950 0x7fd32c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.248+0000 7fd346a00640 1 --2- 172.21.15.63:0/3453530494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c4e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd334005760 tx=0x7fd3340056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a47e2d9c1df38c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.249+0000 7fd33cc00640 1 -- 172.21.15.63:0/3453530494 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334035040 con 0x55ae66b884d0 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.249+0000 7fd33cc00640 1 -- 172.21.15.63:0/3453530494 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd33403d440 con 0x55ae66b884d0 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.249+0000 7fd33cc00640 1 -- 172.21.15.63:0/3453530494 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334031600 con 0x55ae66b884d0 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.250+0000 7fd355780740 1 -- 172.21.15.63:0/3453530494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 msgr2=0x55ae66f4c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.250+0000 7fd355780740 1 --2- 172.21.15.63:0/3453530494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c4e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd334005760 tx=0x7fd3340056d0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.250+0000 7fd347e00640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd32c0b2950 0x7fd32c0b4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd3400e0380 tx=0x7fd340009270 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.250+0000 7fd355780740 1 -- 172.21.15.63:0/3453530494 shutdown_connections 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.250+0000 7fd355780740 1 --2- 172.21.15.63:0/3453530494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c4e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.250+0000 7fd355780740 1 --2- 172.21.15.63:0/3453530494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f4bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.250+0000 7fd355780740 1 --2- 172.21.15.63:0/3453530494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66ba91b0 0x55ae66f4ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.250+0000 7fd355780740 1 -- 172.21.15.63:0/3453530494 >> 172.21.15.63:0/3453530494 conn(0x55ae66b9d300 msgr2=0x55ae66f48210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.250+0000 7fd355780740 1 -- 172.21.15.63:0/3453530494 shutdown_connections 2024-05-27T07:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.250+0000 7fd355780740 1 -- 172.21.15.63:0/3453530494 wait complete. 2024-05-27T07:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.251+0000 7fd355780740 1 -- start start 2024-05-27T07:44:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.251+0000 7fd355780740 1 Processor -- start 2024-05-27T07:44:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.251+0000 7fd355780740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66b802f0 0x55ae66f55c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd355780740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f56170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd355780740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66ba91b0 0x55ae66f61e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd355780740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ae66b92790 con 0x55ae66b884d0 2024-05-27T07:44:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd355780740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae66e54bc0 con 0x55ae66ba91b0 2024-05-27T07:44:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd355780740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ae66f58b30 con 0x55ae66b802f0 2024-05-27T07:44:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd355780740 10 client.0 _ll_register_callbacks cb 0x55ae66e32400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd347400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f56170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd347400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f56170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50618/0 (socket says 172.21.15.63:50618) 2024-05-27T07:44:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd347400640 1 -- 172.21.15.63:0/3469379008 learned_addr learned my addr 172.21.15.63:0/3469379008 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd346a00640 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66b802f0 0x55ae66f55c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.252+0000 7fd346a00640 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66ba91b0 0x55ae66f61e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.253+0000 7fd347400640 1 -- 172.21.15.63:0/3469379008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66b802f0 msgr2=0x55ae66f55c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.253+0000 7fd347400640 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66b802f0 0x55ae66f55c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.253+0000 7fd347400640 1 -- 172.21.15.63:0/3469379008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66ba91b0 msgr2=0x55ae66f61e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.253+0000 7fd347400640 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66ba91b0 0x55ae66f61e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.253+0000 7fd347400640 1 -- 172.21.15.63:0/3469379008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae66bc1210 con 0x55ae66b884d0 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.253+0000 7fd347400640 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f56170 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fd338010810 tx=0x7fd338010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627cbf964e1ad0a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.253+0000 7fd331600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.254+0000 7fd331600640 1 -- 172.21.15.63:0/3469379008 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33800ed40 con 0x55ae66b884d0 2024-05-27T07:44:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.254+0000 7fd331600640 1 -- 172.21.15.63:0/3469379008 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd33801c480 con 0x55ae66b884d0 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.254+0000 7fd355780740 1 -- 172.21.15.63:0/3469379008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd33403d440 con 0x55ae66b884d0 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.254+0000 7fd331600640 1 -- 172.21.15.63:0/3469379008 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd338025050 con 0x55ae66b884d0 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.254+0000 7fd355780740 10 client.17598 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.254+0000 7fd355780740 1 -- 172.21.15.63:0/3469379008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ae66c8d620 con 0x55ae66b884d0 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.255+0000 7fd331600640 1 -- 172.21.15.63:0/3469379008 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd33403d440 con 0x55ae66b884d0 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.255+0000 7fd331600640 1 -- 172.21.15.63:0/3469379008 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd3380115e0 con 0x55ae66b884d0 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.255+0000 7fd355780740 10 client.17598 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.255+0000 7fd355780740 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.255+0000 7fd355780740 20 client.17598 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.255+0000 7fd355780740 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ae66b67d80 0x55ae66f67ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.255+0000 7fd355780740 4 client.17598 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.255+0000 7fd355780740 1 -- 172.21.15.63:0/3469379008 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae66ebeae0 con 0x55ae66b67d80 2024-05-27T07:44:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.256+0000 7fd346a00640 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ae66b67d80 0x55ae66f67ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.256+0000 7fd346a00640 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ae66b67d80 0x55ae66f67ea0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67d94526336945 server_cookie=14337e19b06a17e4 in_seq=0 out_seq=0 2024-05-27T07:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.256+0000 7fd331600640 10 client.17598 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.258+0000 7fd331600640 1 -- 172.21.15.63:0/3469379008 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ae66ebeae0 con 0x55ae66b67d80 2024-05-27T07:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.258+0000 7fd331600640 10 client.17598 handle_command_reply: tid=0 2024-05-27T07:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.258+0000 7fd355780740 1 client.17598 shutdown 2024-05-27T07:44:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 1 -- 172.21.15.63:0/3469379008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 msgr2=0x55ae66f56170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f56170 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fd338010810 tx=0x7fd338010cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 1 -- 172.21.15.63:0/3469379008 shutdown_connections 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ae66b67d80 0x55ae66f67ea0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66ba91b0 0x55ae66f61e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f56170 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 1 --2- 172.21.15.63:0/3469379008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66b802f0 0x55ae66f55c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 1 -- 172.21.15.63:0/3469379008 >> 172.21.15.63:0/3469379008 conn(0x55ae66b9d300 msgr2=0x55ae66f4d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd331600640 0 client.17598 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 1 -- 172.21.15.63:0/3469379008 shutdown_connections 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 1 -- 172.21.15.63:0/3469379008 wait complete. 2024-05-27T07:44:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.259+0000 7fd355780740 20 client.17598 trim_cache size 0 max 16384 2024-05-27T07:44:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.279+0000 7fd355780740 1 Processor -- start 2024-05-27T07:44:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.279+0000 7fd355780740 1 -- start start 2024-05-27T07:44:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.279+0000 7fd355780740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f51440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.279+0000 7fd355780740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.279+0000 7fd355780740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66ba91b0 0x55ae66f55bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.279+0000 7fd355780740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481b5b10 con 0x55ae66b884d0 2024-05-27T07:44:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.279+0000 7fd355780740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481b33e0 con 0x55ae66b802f0 2024-05-27T07:44:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.279+0000 7fd355780740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481b1250 con 0x55ae66ba91b0 2024-05-27T07:44:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.280+0000 7fd347400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.280+0000 7fd347400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50630/0 (socket says 172.21.15.63:50630) 2024-05-27T07:44:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.280+0000 7fd347400640 1 -- 172.21.15.63:0/1766187034 learned_addr learned my addr 172.21.15.63:0/1766187034 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.280+0000 7fd346a00640 1 --2- 172.21.15.63:0/1766187034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f51440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.280+0000 7fd346a00640 1 --2- 172.21.15.63:0/1766187034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66ba91b0 0x55ae66f55bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.280+0000 7fd347400640 1 -- 172.21.15.63:0/1766187034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66ba91b0 msgr2=0x55ae66f55bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.280+0000 7fd347400640 1 --2- 172.21.15.63:0/1766187034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66ba91b0 0x55ae66f55bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.280+0000 7fd347400640 1 -- 172.21.15.63:0/1766187034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 msgr2=0x55ae66f51440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.280+0000 7fd347400640 1 --2- 172.21.15.63:0/1766187034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f51440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.280+0000 7fd347400640 1 -- 172.21.15.63:0/1766187034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae66b67fb0 con 0x55ae66b884d0 2024-05-27T07:44:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.281+0000 7fd347400640 1 --2- 172.21.15.63:0/1766187034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c3a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd338011b70 tx=0x7fd33800c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc9016323d31f13 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.281+0000 7fd333e00640 1 -- 172.21.15.63:0/1766187034 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33802d070 con 0x55ae66b884d0 2024-05-27T07:44:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.281+0000 7fd333e00640 1 -- 172.21.15.63:0/1766187034 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd33800eb40 con 0x55ae66b884d0 2024-05-27T07:44:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.281+0000 7fd333e00640 1 -- 172.21.15.63:0/1766187034 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338022040 con 0x55ae66b884d0 2024-05-27T07:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.282+0000 7fd355780740 1 -- 172.21.15.63:0/1766187034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 msgr2=0x55ae66f4c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.282+0000 7fd355780740 1 --2- 172.21.15.63:0/1766187034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c3a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd338011b70 tx=0x7fd33800c650 comp rx=0 tx=0).stop 2024-05-27T07:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.283+0000 7fd355780740 1 -- 172.21.15.63:0/1766187034 shutdown_connections 2024-05-27T07:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.283+0000 7fd355780740 1 --2- 172.21.15.63:0/1766187034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66ba91b0 0x55ae66f55bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.283+0000 7fd355780740 1 --2- 172.21.15.63:0/1766187034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66b884d0 0x55ae66f4c3a0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.283+0000 7fd355780740 1 --2- 172.21.15.63:0/1766187034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f51440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.283+0000 7fd355780740 1 -- 172.21.15.63:0/1766187034 >> 172.21.15.63:0/1766187034 conn(0x55ae66b9d300 msgr2=0x55ae66f554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.283+0000 7fd355780740 1 -- 172.21.15.63:0/1766187034 shutdown_connections 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.283+0000 7fd355780740 1 -- 172.21.15.63:0/1766187034 wait complete. 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.283+0000 7fd355780740 1 -- start start 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.283+0000 7fd355780740 1 Processor -- start 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.284+0000 7fd355780740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f4f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.284+0000 7fd355780740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66b884d0 0x55ae66f4fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.284+0000 7fd355780740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66ba91b0 0x55ae66f50340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.284+0000 7fd355780740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481b18a0 con 0x55ae66ba91b0 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.284+0000 7fd355780740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481b2d10 con 0x55ae66b802f0 2024-05-27T07:44:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.284+0000 7fd355780740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481b1f70 con 0x55ae66b884d0 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.284+0000 7fd355780740 10 client.0 _ll_register_callbacks cb 0x7fd33800c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.284+0000 7fd346a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66ba91b0 0x55ae66f50340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd346a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66ba91b0 0x55ae66f50340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50646/0 (socket says 172.21.15.63:50646) 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd346a00640 1 -- 172.21.15.63:0/3847943922 learned_addr learned my addr 172.21.15.63:0/3847943922 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd347400640 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66b884d0 0x55ae66f4fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd346a00640 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f4f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd346a00640 1 -- 172.21.15.63:0/3847943922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66b884d0 msgr2=0x55ae66f4fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd346a00640 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66b884d0 0x55ae66f4fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd346a00640 1 -- 172.21.15.63:0/3847943922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 msgr2=0x55ae66f4f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd346a00640 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f4f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd346a00640 1 -- 172.21.15.63:0/3847943922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae66f5cdc0 con 0x55ae66ba91b0 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd346a00640 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66ba91b0 0x55ae66f50340 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd334003750 tx=0x7fd33404b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440f3377f3bc9519 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:52.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.285+0000 7fd331600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.286+0000 7fd331600640 1 -- 172.21.15.63:0/3847943922 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334035040 con 0x55ae66ba91b0 2024-05-27T07:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.286+0000 7fd331600640 1 -- 172.21.15.63:0/3847943922 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd334058e00 con 0x55ae66ba91b0 2024-05-27T07:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.286+0000 7fd355780740 1 -- 172.21.15.63:0/3847943922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd33800eb40 con 0x55ae66ba91b0 2024-05-27T07:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.286+0000 7fd331600640 1 -- 172.21.15.63:0/3847943922 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd33403d750 con 0x55ae66ba91b0 2024-05-27T07:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.286+0000 7fd355780740 10 client.17604 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.286+0000 7fd355780740 1 -- 172.21.15.63:0/3847943922 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ae66f5e010 con 0x55ae66ba91b0 2024-05-27T07:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.286+0000 7fd331600640 1 -- 172.21.15.63:0/3847943922 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd33800eb40 con 0x55ae66ba91b0 2024-05-27T07:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.287+0000 7fd331600640 1 -- 172.21.15.63:0/3847943922 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd33403a040 con 0x55ae66ba91b0 2024-05-27T07:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.287+0000 7fd355780740 10 client.17604 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.287+0000 7fd355780740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.287+0000 7fd355780740 20 client.17604 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.287+0000 7fd355780740 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ae66b3d1c0 0x55ae66f5e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.287+0000 7fd355780740 4 client.17604 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.287+0000 7fd355780740 1 -- 172.21.15.63:0/3847943922 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae66eb82c0 con 0x55ae66b3d1c0 2024-05-27T07:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.287+0000 7fd347400640 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ae66b3d1c0 0x55ae66f5e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.288+0000 7fd347400640 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ae66b3d1c0 0x55ae66f5e300 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593472bb9fe0c072 server_cookie=1c94cdef686574b3 in_seq=0 out_seq=0 2024-05-27T07:44:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.288+0000 7fd331600640 10 client.17604 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd331600640 1 -- 172.21.15.63:0/3847943922 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ae66eb82c0 con 0x55ae66b3d1c0 2024-05-27T07:44:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd331600640 10 client.17604 handle_command_reply: tid=0 2024-05-27T07:44:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd355780740 1 client.17604 shutdown 2024-05-27T07:44:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd355780740 1 -- 172.21.15.63:0/3847943922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66ba91b0 msgr2=0x55ae66f50340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd355780740 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66ba91b0 0x55ae66f50340 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd334003750 tx=0x7fd33404b040 comp rx=0 tx=0).stop 2024-05-27T07:44:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd355780740 1 -- 172.21.15.63:0/3847943922 shutdown_connections 2024-05-27T07:44:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd355780740 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ae66b3d1c0 0x55ae66f5e300 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd355780740 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae66ba91b0 0x55ae66f50340 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd355780740 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae66b884d0 0x55ae66f4fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd355780740 1 --2- 172.21.15.63:0/3847943922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae66b802f0 0x55ae66f4f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd331600640 0 client.17604 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.327+0000 7fd355780740 1 -- 172.21.15.63:0/3847943922 >> 172.21.15.63:0/3847943922 conn(0x55ae66b9d300 msgr2=0x55ae66f55490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.328+0000 7fd355780740 1 -- 172.21.15.63:0/3847943922 shutdown_connections 2024-05-27T07:44:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.328+0000 7fd355780740 1 -- 172.21.15.63:0/3847943922 wait complete. 2024-05-27T07:44:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.328+0000 7fd355780740 20 client.17604 trim_cache size 0 max 16384 2024-05-27T07:44:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:44:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:44:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:44:52.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 -- 172.21.15.63:0/1588749396 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd32c0b2950 msgr2=0x7fd32c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:52.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd32c0b2950 0x7fd32c0b4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd3400e0380 tx=0x7fd340009270 comp rx=0 tx=0).stop 2024-05-27T07:44:52.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 -- 172.21.15.63:0/1588749396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd348110320 msgr2=0x7fd34819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd348110320 0x7fd34819c670 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd34000eaa0 tx=0x7fd34000ef60 comp rx=0 tx=0).stop 2024-05-27T07:44:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 -- 172.21.15.63:0/1588749396 shutdown_connections 2024-05-27T07:44:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd32c0b2950 0x7fd32c0b4e60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd348110320 0x7fd34819c670 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd34810d9c0 0x7fd3481a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 --2- 172.21.15.63:0/1588749396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd348075420 0x7fd3481a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 -- 172.21.15.63:0/1588749396 >> 172.21.15.63:0/1588749396 conn(0x7fd348079100 msgr2=0x7fd3481022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.329+0000 7fd34d000640 1 -- 172.21.15.63:0/1588749396 shutdown_connections 2024-05-27T07:44:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:44:52.330+0000 7fd34d000640 1 -- 172.21.15.63:0/1588749396 wait complete. 2024-05-27T07:44:52.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:52 smithi063 ceph-mon[31613]: pgmap v352: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:44:53.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:52 smithi086 ceph-mon[34687]: pgmap v352: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:44:53.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:52 smithi149 ceph-mon[35408]: pgmap v352: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:44:54.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.645+0000 7fdfdfe00700 1 -- 172.21.15.63:0/902953428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe0105300 msgr2=0x7fdfe0105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:54.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.645+0000 7fdfdfe00700 1 --2- 172.21.15.63:0/902953428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe0105300 0x7fdfe0105710 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009990 tx=0x7fdfd0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.645+0000 7fdfdfe00700 1 -- 172.21.15.63:0/902953428 shutdown_connections 2024-05-27T07:44:54.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.645+0000 7fdfdfe00700 1 --2- 172.21.15.63:0/902953428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe010e730 0x7fdfe0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.645+0000 7fdfdfe00700 1 --2- 172.21.15.63:0/902953428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe0105300 0x7fdfe0105710 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.645+0000 7fdfdfe00700 1 --2- 172.21.15.63:0/902953428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe0105c50 0x7fdfe010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.645+0000 7fdfdfe00700 1 -- 172.21.15.63:0/902953428 >> 172.21.15.63:0/902953428 conn(0x7fdfe0078c30 msgr2=0x7fdfe0079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:54.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.645+0000 7fdfdfe00700 1 -- 172.21.15.63:0/902953428 shutdown_connections 2024-05-27T07:44:54.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.645+0000 7fdfdfe00700 1 -- 172.21.15.63:0/902953428 wait complete. 2024-05-27T07:44:54.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.646+0000 7fdfdfe00700 1 Processor -- start 2024-05-27T07:44:54.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.647+0000 7fdfdfe00700 1 -- start start 2024-05-27T07:44:54.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.647+0000 7fdfdfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe0105300 0x7fdfe01a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.647+0000 7fdfdfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe0105c50 0x7fdfe01a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.647+0000 7fdfdfe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe010e730 0x7fdfe019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.647+0000 7fdfdfe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0114180 con 0x7fdfe0105c50 2024-05-27T07:44:54.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.647+0000 7fdfdfe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0114000 con 0x7fdfe0105300 2024-05-27T07:44:54.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.647+0000 7fdfdfe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0114300 con 0x7fdfe010e730 2024-05-27T07:44:54.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.648+0000 7fdfde000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe0105c50 0x7fdfe01a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.648+0000 7fdfde000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe0105c50 0x7fdfe01a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50666/0 (socket says 172.21.15.63:50666) 2024-05-27T07:44:54.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.648+0000 7fdfde000700 1 -- 172.21.15.63:0/815698242 learned_addr learned my addr 172.21.15.63:0/815698242 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:54.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.648+0000 7fdfdea00700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe0105300 0x7fdfe01a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.648+0000 7fdfdf400700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe010e730 0x7fdfe019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.648+0000 7fdfde000700 1 -- 172.21.15.63:0/815698242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe010e730 msgr2=0x7fdfe019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:54.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.648+0000 7fdfde000700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe010e730 0x7fdfe019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.648+0000 7fdfde000700 1 -- 172.21.15.63:0/815698242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe0105300 msgr2=0x7fdfe01a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:54.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.648+0000 7fdfde000700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe0105300 0x7fdfe01a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.648+0000 7fdfde000700 1 -- 172.21.15.63:0/815698242 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe019b070 con 0x7fdfe0105c50 2024-05-27T07:44:54.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.649+0000 7fdfde000700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe0105c50 0x7fdfe01a0ce0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00d950 tx=0x7fdfcc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:54.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.649+0000 7fdfd7400700 1 -- 172.21.15.63:0/815698242 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc016070 con 0x7fdfe0105c50 2024-05-27T07:44:54.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.649+0000 7fdfd7400700 1 -- 172.21.15.63:0/815698242 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfcc009dd0 con 0x7fdfe0105c50 2024-05-27T07:44:54.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.649+0000 7fdfdfe00700 1 -- 172.21.15.63:0/815698242 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe019b360 con 0x7fdfe0105c50 2024-05-27T07:44:54.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.649+0000 7fdfd7400700 1 -- 172.21.15.63:0/815698242 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00f3c0 con 0x7fdfe0105c50 2024-05-27T07:44:54.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.649+0000 7fdfdfe00700 1 -- 172.21.15.63:0/815698242 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe01a7630 con 0x7fdfe0105c50 2024-05-27T07:44:54.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.655+0000 7fdfd7400700 1 -- 172.21.15.63:0/815698242 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdfcc019070 con 0x7fdfe0105c50 2024-05-27T07:44:54.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.655+0000 7fdff4125b80 1 Processor -- start 2024-05-27T07:44:54.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.656+0000 7fdff4125b80 1 -- start start 2024-05-27T07:44:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.656+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.656+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e29d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.656+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e35760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.656+0000 7fdff4125b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559419d56ec0 con 0x559419d95a70 2024-05-27T07:44:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.656+0000 7fdff4125b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559419d46a60 con 0x559419ca9ec0 2024-05-27T07:44:54.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.656+0000 7fdff4125b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559419d4aaa0 con 0x559419e33340 2024-05-27T07:44:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.657+0000 7fdfdf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.657+0000 7fdfdf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e297d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50672/0 (socket says 172.21.15.63:50672) 2024-05-27T07:44:54.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.657+0000 7fdfdf400700 1 -- 172.21.15.63:0/4111854316 learned_addr learned my addr 172.21.15.63:0/4111854316 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.657+0000 7fdfdea00700 1 --2- 172.21.15.63:0/4111854316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e29d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.657+0000 7fdfde000700 1 --2- 172.21.15.63:0/4111854316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e35760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.657+0000 7fdfd7400700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdfc40b2a70 0x7fdfc40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.658+0000 7fdfdea00700 1 -- 172.21.15.63:0/4111854316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 msgr2=0x559419e35760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.658+0000 7fdfdea00700 1 --2- 172.21.15.63:0/4111854316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e35760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.658+0000 7fdfdea00700 1 -- 172.21.15.63:0/4111854316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 msgr2=0x559419e297d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.658+0000 7fdfdea00700 1 --2- 172.21.15.63:0/4111854316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e297d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.658+0000 7fdfdea00700 1 -- 172.21.15.63:0/4111854316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559419d0f4a0 con 0x559419ca9ec0 2024-05-27T07:44:54.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.658+0000 7fdfde000700 1 --2- 172.21.15.63:0/4111854316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e35760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:54.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.658+0000 7fdfd7400700 1 -- 172.21.15.63:0/815698242 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fdfcc0e2a00 con 0x7fdfe0105c50 2024-05-27T07:44:54.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.658+0000 7fdfdf400700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdfc40b2a70 0x7fdfc40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.658+0000 7fdfdea00700 1 --2- 172.21.15.63:0/4111854316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e29d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0006010 tx=0x7fdfd00096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8081950174b6e4a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.659+0000 7fdfdf400700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdfc40b2a70 0x7fdfc40b4f80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x559419e25b10 tx=0x7fdfd8004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.659+0000 7fdfcbe00700 1 -- 172.21.15.63:0/4111854316 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0027070 con 0x559419ca9ec0 2024-05-27T07:44:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.659+0000 7fdfcbe00700 1 -- 172.21.15.63:0/4111854316 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfd00037c0 con 0x559419ca9ec0 2024-05-27T07:44:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.659+0000 7fdfcbe00700 1 -- 172.21.15.63:0/4111854316 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000ba30 con 0x559419ca9ec0 2024-05-27T07:44:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.660+0000 7fdff4125b80 1 -- 172.21.15.63:0/4111854316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 msgr2=0x559419e29d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.660+0000 7fdff4125b80 1 --2- 172.21.15.63:0/4111854316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e29d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0006010 tx=0x7fdfd00096e0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.660+0000 7fdff4125b80 1 -- 172.21.15.63:0/4111854316 shutdown_connections 2024-05-27T07:44:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.660+0000 7fdff4125b80 1 --2- 172.21.15.63:0/4111854316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e35760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.660+0000 7fdff4125b80 1 --2- 172.21.15.63:0/4111854316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.660+0000 7fdff4125b80 1 --2- 172.21.15.63:0/4111854316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e29d10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.660+0000 7fdff4125b80 1 -- 172.21.15.63:0/4111854316 >> 172.21.15.63:0/4111854316 conn(0x559419d43410 msgr2=0x559419e24790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.660+0000 7fdff4125b80 1 -- 172.21.15.63:0/4111854316 shutdown_connections 2024-05-27T07:44:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.660+0000 7fdff4125b80 1 -- 172.21.15.63:0/4111854316 wait complete. 2024-05-27T07:44:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.661+0000 7fdff4125b80 1 -- start start 2024-05-27T07:44:54.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.661+0000 7fdff4125b80 1 Processor -- start 2024-05-27T07:44:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.662+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e3d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.662+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e3d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.662+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e37470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.662+0000 7fdff4125b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559419c4d020 con 0x559419d95a70 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.662+0000 7fdff4125b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559419ce77e0 con 0x559419ca9ec0 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.662+0000 7fdff4125b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559419e35e20 con 0x559419e33340 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.662+0000 7fdff4125b80 10 client.0 _ll_register_callbacks cb 0x559419cf0cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.662+0000 7fdfde000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e3d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.662+0000 7fdfde000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e3d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50684/0 (socket says 172.21.15.63:50684) 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.662+0000 7fdfde000700 1 -- 172.21.15.63:0/2561849435 learned_addr learned my addr 172.21.15.63:0/2561849435 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.663+0000 7fdfdea00700 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e3d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.663+0000 7fdfdf400700 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e37470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.663+0000 7fdfde000700 1 -- 172.21.15.63:0/2561849435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 msgr2=0x559419e37470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.663+0000 7fdfde000700 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e37470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.663+0000 7fdfde000700 1 -- 172.21.15.63:0/2561849435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 msgr2=0x559419e3d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:54.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.663+0000 7fdfde000700 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e3d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.663+0000 7fdfde000700 1 -- 172.21.15.63:0/2561849435 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559419c96950 con 0x559419d95a70 2024-05-27T07:44:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.663+0000 7fdfdea00700 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e3d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.663+0000 7fdfde000700 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e3d5b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc000c00 tx=0x7fdfcc0108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90ff02326805fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.663+0000 7fdfc8c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:54.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.664+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2561849435 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc020070 con 0x559419d95a70 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.664+0000 7fdff4125b80 1 -- 172.21.15.63:0/2561849435 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd00037c0 con 0x559419d95a70 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.664+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2561849435 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fdfcc0dffb0 con 0x559419d95a70 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.664+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2561849435 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdfcc0e01b0 con 0x559419d95a70 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.664+0000 7fdff4125b80 10 client.17616 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.664+0000 7fdff4125b80 1 -- 172.21.15.63:0/2561849435 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559419d170e0 con 0x559419d95a70 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.664+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2561849435 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfd00037c0 con 0x559419d95a70 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.665+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2561849435 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdfcc0a9b20 con 0x559419d95a70 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.665+0000 7fdff4125b80 10 client.17616 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.665+0000 7fdff4125b80 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.665+0000 7fdff4125b80 20 client.17616 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.665+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559419e470a0 0x559419e494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.665+0000 7fdff4125b80 4 client.17616 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:44:54.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.665+0000 7fdff4125b80 1 -- 172.21.15.63:0/2561849435 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559419d58e10 con 0x559419e470a0 2024-05-27T07:44:54.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.666+0000 7fdfdea00700 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559419e470a0 0x559419e494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.667+0000 7fdfc8c00700 10 client.17616 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:54.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.667+0000 7fdfdea00700 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559419e470a0 0x559419e494c0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15476d5eb39d91f1 server_cookie=a86e3ae1180ebc in_seq=0 out_seq=0 2024-05-27T07:44:54.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.669+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2561849435 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559419d58e10 con 0x559419e470a0 2024-05-27T07:44:54.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.669+0000 7fdfc8c00700 10 client.17616 handle_command_reply: tid=0 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.669+0000 7fdff4125b80 1 client.17616 shutdown 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.669+0000 7fdff4125b80 1 -- 172.21.15.63:0/2561849435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 msgr2=0x559419e3d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.669+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e3d5b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc000c00 tx=0x7fdfcc0108d0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.669+0000 7fdff4125b80 1 -- 172.21.15.63:0/2561849435 shutdown_connections 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.669+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559419e470a0 0x559419e494c0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.670+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e37470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.670+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e3d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.670+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2561849435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419d95a70 0x559419e3d5b0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.670+0000 7fdff4125b80 1 -- 172.21.15.63:0/2561849435 >> 172.21.15.63:0/2561849435 conn(0x559419d43410 msgr2=0x559419e24380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.670+0000 7fdfc8c00700 0 client.17616 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:54.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.670+0000 7fdff4125b80 1 -- 172.21.15.63:0/2561849435 shutdown_connections 2024-05-27T07:44:54.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.670+0000 7fdff4125b80 1 -- 172.21.15.63:0/2561849435 wait complete. 2024-05-27T07:44:54.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.670+0000 7fdff4125b80 20 client.17616 trim_cache size 0 max 16384 2024-05-27T07:44:54.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.697+0000 7fdff4125b80 1 Processor -- start 2024-05-27T07:44:54.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.697+0000 7fdff4125b80 1 -- start start 2024-05-27T07:44:54.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.698+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e29e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.698+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419d95a70 0x559419e288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.698+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419e33340 0x559419e31920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.698+0000 7fdff4125b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe01b0a10 con 0x559419e33340 2024-05-27T07:44:54.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.698+0000 7fdff4125b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe01ae2a0 con 0x559419ca9ec0 2024-05-27T07:44:54.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.698+0000 7fdff4125b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe01ac130 con 0x559419d95a70 2024-05-27T07:44:54.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.698+0000 7fdfdf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419e33340 0x559419e31920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.698+0000 7fdfdf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419e33340 0x559419e31920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50690/0 (socket says 172.21.15.63:50690) 2024-05-27T07:44:54.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.698+0000 7fdfdf400700 1 -- 172.21.15.63:0/1510746706 learned_addr learned my addr 172.21.15.63:0/1510746706 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:54.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.699+0000 7fdfdea00700 1 --2- 172.21.15.63:0/1510746706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e29e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.699+0000 7fdfde000700 1 --2- 172.21.15.63:0/1510746706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419d95a70 0x559419e288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.699+0000 7fdfdf400700 1 -- 172.21.15.63:0/1510746706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419d95a70 msgr2=0x559419e288b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.699+0000 7fdfdf400700 1 --2- 172.21.15.63:0/1510746706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419d95a70 0x559419e288b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.699+0000 7fdfdf400700 1 -- 172.21.15.63:0/1510746706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 msgr2=0x559419e29e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.699+0000 7fdfdf400700 1 --2- 172.21.15.63:0/1510746706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e29e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.699+0000 7fdfdf400700 1 -- 172.21.15.63:0/1510746706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559419e24cd0 con 0x559419e33340 2024-05-27T07:44:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.700+0000 7fdfdf400700 1 --2- 172.21.15.63:0/1510746706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419e33340 0x559419e31920 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8017580 tx=0x7fdfd8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74b1a60aa38b05e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.700+0000 7fdfcb400700 1 -- 172.21.15.63:0/1510746706 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd801d070 con 0x559419e33340 2024-05-27T07:44:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.700+0000 7fdfcb400700 1 -- 172.21.15.63:0/1510746706 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfd8028070 con 0x559419e33340 2024-05-27T07:44:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.700+0000 7fdfcb400700 1 -- 172.21.15.63:0/1510746706 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd80166d0 con 0x559419e33340 2024-05-27T07:44:54.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.701+0000 7fdff4125b80 1 -- 172.21.15.63:0/1510746706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419e33340 msgr2=0x559419e31920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:54.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.701+0000 7fdff4125b80 1 --2- 172.21.15.63:0/1510746706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419e33340 0x559419e31920 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8017580 tx=0x7fdfd8015000 comp rx=0 tx=0).stop 2024-05-27T07:44:54.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.701+0000 7fdff4125b80 1 -- 172.21.15.63:0/1510746706 shutdown_connections 2024-05-27T07:44:54.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.701+0000 7fdff4125b80 1 --2- 172.21.15.63:0/1510746706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419e33340 0x559419e31920 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.701+0000 7fdff4125b80 1 --2- 172.21.15.63:0/1510746706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419ca9ec0 0x559419e29e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.701+0000 7fdff4125b80 1 --2- 172.21.15.63:0/1510746706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419d95a70 0x559419e288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.701+0000 7fdff4125b80 1 -- 172.21.15.63:0/1510746706 >> 172.21.15.63:0/1510746706 conn(0x559419d43410 msgr2=0x559419e281a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:54.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.701+0000 7fdff4125b80 1 -- 172.21.15.63:0/1510746706 shutdown_connections 2024-05-27T07:44:54.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.701+0000 7fdff4125b80 1 -- 172.21.15.63:0/1510746706 wait complete. 2024-05-27T07:44:54.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.702+0000 7fdff4125b80 1 -- start start 2024-05-27T07:44:54.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.702+0000 7fdff4125b80 1 Processor -- start 2024-05-27T07:44:54.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419ca9ec0 0x559419e421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419d95a70 0x559419e42710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdff4125b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e42c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdff4125b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe01ac780 con 0x559419ca9ec0 2024-05-27T07:44:54.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdff4125b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe01adbd0 con 0x559419d95a70 2024-05-27T07:44:54.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdff4125b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe01ace30 con 0x559419e33340 2024-05-27T07:44:54.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdff4125b80 10 client.0 _ll_register_callbacks cb 0x559419cf0cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:54.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdfdea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419ca9ec0 0x559419e421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdfdea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419ca9ec0 0x559419e421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50702/0 (socket says 172.21.15.63:50702) 2024-05-27T07:44:54.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdfdea00700 1 -- 172.21.15.63:0/2214065291 learned_addr learned my addr 172.21.15.63:0/2214065291 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:54.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.703+0000 7fdfde000700 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419d95a70 0x559419e42710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.704+0000 7fdfdf400700 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e42c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.704+0000 7fdfdea00700 1 -- 172.21.15.63:0/2214065291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 msgr2=0x559419e42c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:54.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.704+0000 7fdfdea00700 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e42c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.704+0000 7fdfdea00700 1 -- 172.21.15.63:0/2214065291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419d95a70 msgr2=0x559419e42710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:54.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.704+0000 7fdfdea00700 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419d95a70 0x559419e42710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.704+0000 7fdfdea00700 1 -- 172.21.15.63:0/2214065291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559419eb1c50 con 0x559419ca9ec0 2024-05-27T07:44:54.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.704+0000 7fdfdea00700 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419ca9ec0 0x559419e421d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000b580 tx=0x7fdfd00058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c120a6c188c88f06 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:54.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.704+0000 7fdfc8c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:54.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.705+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2214065291 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0034030 con 0x559419ca9ec0 2024-05-27T07:44:54.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.705+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2214065291 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fdfd00050d0 con 0x559419ca9ec0 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.705+0000 7fdff4125b80 1 -- 172.21.15.63:0/2214065291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd8028070 con 0x559419ca9ec0 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.705+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2214065291 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdfd003e050 con 0x559419ca9ec0 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.705+0000 7fdff4125b80 10 client.17622 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.705+0000 7fdff4125b80 1 -- 172.21.15.63:0/2214065291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559419eb2ed0 con 0x559419ca9ec0 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.706+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2214065291 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfd8028070 con 0x559419ca9ec0 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.706+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2214065291 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdfd000f460 con 0x559419ca9ec0 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.706+0000 7fdff4125b80 10 client.17622 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.706+0000 7fdff4125b80 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.706+0000 7fdff4125b80 20 client.17622 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.706+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559419eb3200 0x559419eb5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:54.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.706+0000 7fdff4125b80 4 client.17622 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:44:54.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.706+0000 7fdff4125b80 1 -- 172.21.15.63:0/2214065291 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559419d525f0 con 0x559419eb3200 2024-05-27T07:44:54.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.707+0000 7fdfde000700 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559419eb3200 0x559419eb5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:54.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.708+0000 7fdfc8c00700 10 client.17622 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:54.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.708+0000 7fdfde000700 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559419eb3200 0x559419eb5620 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=833904fe1c0bbd91 server_cookie=a581585b079cd243 in_seq=0 out_seq=0 2024-05-27T07:44:54.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.712+0000 7fdfc8c00700 1 -- 172.21.15.63:0/2214065291 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559419d525f0 con 0x559419eb3200 2024-05-27T07:44:54.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.712+0000 7fdfc8c00700 10 client.17622 handle_command_reply: tid=0 2024-05-27T07:44:54.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.712+0000 7fdff4125b80 1 client.17622 shutdown 2024-05-27T07:44:54.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 1 -- 172.21.15.63:0/2214065291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419ca9ec0 msgr2=0x559419e421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:54.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419ca9ec0 0x559419e421d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000b580 tx=0x7fdfd00058e0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 1 -- 172.21.15.63:0/2214065291 shutdown_connections 2024-05-27T07:44:54.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559419eb3200 0x559419eb5620 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559419e33340 0x559419e42c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559419ca9ec0 0x559419e421d0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 1 --2- 172.21.15.63:0/2214065291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559419d95a70 0x559419e42710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 1 -- 172.21.15.63:0/2214065291 >> 172.21.15.63:0/2214065291 conn(0x559419d43410 msgr2=0x559419e237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdfc8c00700 0 client.17622 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 1 -- 172.21.15.63:0/2214065291 shutdown_connections 2024-05-27T07:44:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 1 -- 172.21.15.63:0/2214065291 wait complete. 2024-05-27T07:44:54.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.713+0000 7fdff4125b80 20 client.17622 trim_cache size 0 max 16384 2024-05-27T07:44:54.714 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:44:54.714 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:44:54.714 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "06c36655-1a65-42b8-a8b5-30f0c6823617", 2024-05-27T07:44:54.715 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:44:54.715 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:44:54.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.714+0000 7fdfdfe00700 1 -- 172.21.15.63:0/815698242 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdfc40b2a70 msgr2=0x7fdfc40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:54.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.714+0000 7fdfdfe00700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdfc40b2a70 0x7fdfc40b4f80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x559419e25b10 tx=0x7fdfd8004ad0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.714+0000 7fdfdfe00700 1 -- 172.21.15.63:0/815698242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe0105c50 msgr2=0x7fdfe01a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:54.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.714+0000 7fdfdfe00700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe0105c50 0x7fdfe01a0ce0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00d950 tx=0x7fdfcc00dc60 comp rx=0 tx=0).stop 2024-05-27T07:44:54.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.715+0000 7fdfdfe00700 1 -- 172.21.15.63:0/815698242 shutdown_connections 2024-05-27T07:44:54.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.715+0000 7fdfdfe00700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdfc40b2a70 0x7fdfc40b4f80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.715+0000 7fdfdfe00700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfe010e730 0x7fdfe019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.715+0000 7fdfdfe00700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe0105300 0x7fdfe01a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.715+0000 7fdfdfe00700 1 --2- 172.21.15.63:0/815698242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfe0105c50 0x7fdfe01a0ce0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:54.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.715+0000 7fdfdfe00700 1 -- 172.21.15.63:0/815698242 >> 172.21.15.63:0/815698242 conn(0x7fdfe0078c30 msgr2=0x7fdfe00ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:54.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.715+0000 7fdfdfe00700 1 -- 172.21.15.63:0/815698242 shutdown_connections 2024-05-27T07:44:54.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:54.715+0000 7fdfdfe00700 1 -- 172.21.15.63:0/815698242 wait complete. 2024-05-27T07:44:54.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:54 smithi063 ceph-mon[31613]: pgmap v353: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:44:55.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:54 smithi086 ceph-mon[34687]: pgmap v353: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:44:55.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:54 smithi149 ceph-mon[35408]: pgmap v353: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:44:55.720 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:44:55.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:44:55.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:55.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:44:55.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:44:55.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:44:55.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:55.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:55.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:44:55.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:55.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:44:55.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:55.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:44:55.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:55.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:44:55.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:55.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.202 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:44:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:44:56.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:44:56.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:44:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:44:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:44:57.593 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:57 smithi063 ceph-mon[31613]: pgmap v354: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:44:57.711 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:57 smithi149 ceph-mon[35408]: pgmap v354: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:44:57.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:57 smithi086 ceph-mon[34687]: pgmap v354: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:44:58.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.591+0000 7f8642600700 1 -- 172.21.15.63:0/2872408745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f863c103870 msgr2=0x7f863c110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:58.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.591+0000 7f8642600700 1 --2- 172.21.15.63:0/2872408745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f863c103870 0x7f863c110590 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8628009990 tx=0x7f8628009ca0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.591+0000 7f8642600700 1 -- 172.21.15.63:0/2872408745 shutdown_connections 2024-05-27T07:44:58.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.591+0000 7f8642600700 1 --2- 172.21.15.63:0/2872408745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c110ad0 0x7f863c112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.591+0000 7f8642600700 1 --2- 172.21.15.63:0/2872408745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f863c102f20 0x7f863c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.591+0000 7f8642600700 1 --2- 172.21.15.63:0/2872408745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f863c103870 0x7f863c110590 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.591+0000 7f8642600700 1 -- 172.21.15.63:0/2872408745 >> 172.21.15.63:0/2872408745 conn(0x7f863c078ab0 msgr2=0x7f863c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:58.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.591+0000 7f8642600700 1 -- 172.21.15.63:0/2872408745 shutdown_connections 2024-05-27T07:44:58.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.592+0000 7f8642600700 1 -- 172.21.15.63:0/2872408745 wait complete. 2024-05-27T07:44:58.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.593+0000 7f8642600700 1 Processor -- start 2024-05-27T07:44:58.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.593+0000 7f8642600700 1 -- start start 2024-05-27T07:44:58.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.594+0000 7f8642600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f863c102f20 0x7f863c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.594+0000 7f8642600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c103870 0x7f863c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.594+0000 7f8642600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f863c110ad0 0x7f863c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.594+0000 7f8642600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f863c116380 con 0x7f863c110ad0 2024-05-27T07:44:58.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.594+0000 7f8642600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f863c116200 con 0x7f863c102f20 2024-05-27T07:44:58.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.594+0000 7f8642600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f863c116500 con 0x7f863c103870 2024-05-27T07:44:58.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.594+0000 7f8641c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f863c110ad0 0x7f863c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.595+0000 7f8641200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f863c102f20 0x7f863c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.595+0000 7f8641c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f863c110ad0 0x7f863c1a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50730/0 (socket says 172.21.15.63:50730) 2024-05-27T07:44:58.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.595+0000 7f863be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c103870 0x7f863c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.595+0000 7f8641c00700 1 -- 172.21.15.63:0/4096273972 learned_addr learned my addr 172.21.15.63:0/4096273972 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:58.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.595+0000 7f8641c00700 1 -- 172.21.15.63:0/4096273972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c103870 msgr2=0x7f863c19d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:58.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.595+0000 7f8641c00700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c103870 0x7f863c19d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.595+0000 7f8641c00700 1 -- 172.21.15.63:0/4096273972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f863c102f20 msgr2=0x7f863c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:58.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.595+0000 7f8641c00700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f863c102f20 0x7f863c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.595+0000 7f8641c00700 1 -- 172.21.15.63:0/4096273972 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c1a1ce0 con 0x7f863c110ad0 2024-05-27T07:44:58.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.596+0000 7f8641c00700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f863c110ad0 0x7f863c1a1640 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8634007ee0 tx=0x7f863400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:58.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.596+0000 7f8639600700 1 -- 172.21.15.63:0/4096273972 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634004bb0 con 0x7f863c110ad0 2024-05-27T07:44:58.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.596+0000 7f8639600700 1 -- 172.21.15.63:0/4096273972 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8634004d50 con 0x7f863c110ad0 2024-05-27T07:44:58.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.596+0000 7f8642600700 1 -- 172.21.15.63:0/4096273972 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c1a1fd0 con 0x7f863c110ad0 2024-05-27T07:44:58.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.596+0000 7f8639600700 1 -- 172.21.15.63:0/4096273972 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340058b0 con 0x7f863c110ad0 2024-05-27T07:44:58.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.596+0000 7f8642600700 1 -- 172.21.15.63:0/4096273972 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c1a98f0 con 0x7f863c110ad0 2024-05-27T07:44:58.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.601+0000 7f8639600700 1 -- 172.21.15.63:0/4096273972 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f863401a020 con 0x7f863c110ad0 2024-05-27T07:44:58.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.602+0000 7f86526b0b80 1 Processor -- start 2024-05-27T07:44:58.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.602+0000 7f86526b0b80 1 -- start start 2024-05-27T07:44:58.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.602+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902ddcee40 0x55902df4ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f8639600700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86240b29f0 0x7f86240b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df57ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 0x55902df5a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f86526b0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55902de7be40 con 0x55902df4d410 2024-05-27T07:44:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f86526b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55902de6b9e0 con 0x55902ddcee40 2024-05-27T07:44:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f86526b0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55902de6fa20 con 0x55902df4d820 2024-05-27T07:44:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f8641200700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86240b29f0 0x7f86240b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f863be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df57ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f8639600700 1 -- 172.21.15.63:0/4096273972 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f86340e2070 con 0x7f863c110ad0 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f863be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df57ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50740/0 (socket says 172.21.15.63:50740) 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f863be00700 1 -- 172.21.15.63:0/2167562772 learned_addr learned my addr 172.21.15.63:0/2167562772 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f863be00700 1 --2- 172.21.15.63:0/2167562772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902ddcee40 0x55902df4ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.603+0000 7f8641c00700 1 --2- 172.21.15.63:0/2167562772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 0x55902df5a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.604+0000 7f863be00700 1 -- 172.21.15.63:0/2167562772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 msgr2=0x55902df5a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.604+0000 7f863be00700 1 --2- 172.21.15.63:0/2167562772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 0x55902df5a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.604+0000 7f863be00700 1 -- 172.21.15.63:0/2167562772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902ddcee40 msgr2=0x55902df4ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.604+0000 7f863be00700 1 --2- 172.21.15.63:0/2167562772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902ddcee40 0x55902df4ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.604+0000 7f863be00700 1 -- 172.21.15.63:0/2167562772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55902de34420 con 0x55902df4d410 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.604+0000 7f8641200700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86240b29f0 0x7f86240b4f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8630005950 tx=0x7f863000b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.604+0000 7f863be00700 1 --2- 172.21.15.63:0/2167562772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df57ad0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8628009990 tx=0x7f862801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f9325952cd6926 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.605+0000 7f862e000700 1 -- 172.21.15.63:0/2167562772 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628030070 con 0x55902df4d410 2024-05-27T07:44:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.605+0000 7f862e000700 1 -- 172.21.15.63:0/2167562772 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8628005500 con 0x55902df4d410 2024-05-27T07:44:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.605+0000 7f862e000700 1 -- 172.21.15.63:0/2167562772 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628003e40 con 0x55902df4d410 2024-05-27T07:44:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.606+0000 7f86526b0b80 1 -- 172.21.15.63:0/2167562772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 msgr2=0x55902df57ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.606+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2167562772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df57ad0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8628009990 tx=0x7f862801a040 comp rx=0 tx=0).stop 2024-05-27T07:44:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.606+0000 7f86526b0b80 1 -- 172.21.15.63:0/2167562772 shutdown_connections 2024-05-27T07:44:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.606+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2167562772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 0x55902df5a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.606+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2167562772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902ddcee40 0x55902df4ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.606+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2167562772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df57ad0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.606+0000 7f86526b0b80 1 -- 172.21.15.63:0/2167562772 >> 172.21.15.63:0/2167562772 conn(0x55902de2efc0 msgr2=0x55902df48d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.606+0000 7f86526b0b80 1 -- 172.21.15.63:0/2167562772 shutdown_connections 2024-05-27T07:44:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.606+0000 7f86526b0b80 1 -- 172.21.15.63:0/2167562772 wait complete. 2024-05-27T07:44:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.607+0000 7f86526b0b80 1 -- start start 2024-05-27T07:44:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.607+0000 7f86526b0b80 1 Processor -- start 2024-05-27T07:44:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.608+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.608+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d410 0x55902df57b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.608+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 0x55902df58060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.608+0000 7f86526b0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55902dd71fa0 con 0x55902ddcee40 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.608+0000 7f86526b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55902de0c760 con 0x55902df4d410 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.608+0000 7f86526b0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55902df5a930 con 0x55902df4d820 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.608+0000 7f86526b0b80 10 client.0 _ll_register_callbacks cb 0x55902de15c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.608+0000 7f8641200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.608+0000 7f8641200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50752/0 (socket says 172.21.15.63:50752) 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.608+0000 7f8641200700 1 -- 172.21.15.63:0/2440711463 learned_addr learned my addr 172.21.15.63:0/2440711463 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.609+0000 7f863be00700 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d410 0x55902df57b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.609+0000 7f8641200700 1 -- 172.21.15.63:0/2440711463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 msgr2=0x55902df58060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.609+0000 7f8641c00700 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 0x55902df58060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.609+0000 7f8641200700 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 0x55902df58060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.609+0000 7f8641200700 1 -- 172.21.15.63:0/2440711463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d410 msgr2=0x55902df57b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:58.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.609+0000 7f8641200700 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d410 0x55902df57b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.609+0000 7f8641200700 1 -- 172.21.15.63:0/2440711463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55902ddbb8d0 con 0x55902ddcee40 2024-05-27T07:44:58.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.609+0000 7f8641c00700 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 0x55902df58060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:58.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.609+0000 7f863be00700 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d410 0x55902df57b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:58.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.610+0000 7f8641200700 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df575e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f86300116f0 tx=0x7f86300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b821e0d20fd93cc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:58.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.610+0000 7f8622a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:58.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.610+0000 7f8622a00700 1 -- 172.21.15.63:0/2440711463 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863001e830 con 0x55902ddcee40 2024-05-27T07:44:58.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.610+0000 7f8622a00700 1 -- 172.21.15.63:0/2440711463 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f863001e9d0 con 0x55902ddcee40 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.610+0000 7f86526b0b80 1 -- 172.21.15.63:0/2440711463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8628005500 con 0x55902ddcee40 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f8622a00700 1 -- 172.21.15.63:0/2440711463 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f863002c050 con 0x55902ddcee40 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f86526b0b80 10 client.17634 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f86526b0b80 1 -- 172.21.15.63:0/2440711463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55902de3c060 con 0x55902ddcee40 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f8622a00700 1 -- 172.21.15.63:0/2440711463 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8628005500 con 0x55902ddcee40 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f8622a00700 1 -- 172.21.15.63:0/2440711463 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8630026bb0 con 0x55902ddcee40 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f86526b0b80 10 client.17634 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f86526b0b80 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f86526b0b80 20 client.17634 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55902df679b0 0x55902df69e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f86526b0b80 4 client.17634 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.611+0000 7f86526b0b80 1 -- 172.21.15.63:0/2440711463 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55902de7dd90 con 0x55902df679b0 2024-05-27T07:44:58.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.612+0000 7f863be00700 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55902df679b0 0x55902df69e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.613+0000 7f8622a00700 10 client.17634 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:58.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.613+0000 7f863be00700 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55902df679b0 0x55902df69e50 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580005e17307c77 server_cookie=3afb3d381cf45ba2 in_seq=0 out_seq=0 2024-05-27T07:44:58.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.615+0000 7f8622a00700 1 -- 172.21.15.63:0/2440711463 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55902de7dd90 con 0x55902df679b0 2024-05-27T07:44:58.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.615+0000 7f8622a00700 10 client.17634 handle_command_reply: tid=0 2024-05-27T07:44:58.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.615+0000 7f86526b0b80 1 client.17634 shutdown 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 1 -- 172.21.15.63:0/2440711463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 msgr2=0x55902df575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df575e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f86300116f0 tx=0x7f86300047c0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 1 -- 172.21.15.63:0/2440711463 shutdown_connections 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55902df679b0 0x55902df69e50 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d820 0x55902df58060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df575e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2440711463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d410 0x55902df57b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 1 -- 172.21.15.63:0/2440711463 >> 172.21.15.63:0/2440711463 conn(0x55902de2efc0 msgr2=0x55902df47ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f8622a00700 0 client.17634 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 1 -- 172.21.15.63:0/2440711463 shutdown_connections 2024-05-27T07:44:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 1 -- 172.21.15.63:0/2440711463 wait complete. 2024-05-27T07:44:58.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.616+0000 7f86526b0b80 20 client.17634 trim_cache size 0 max 16384 2024-05-27T07:44:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.645+0000 7f86526b0b80 1 Processor -- start 2024-05-27T07:44:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.645+0000 7f86526b0b80 1 -- start start 2024-05-27T07:44:58.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.646+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.646+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d410 0x55902df639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.646+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d820 0x55902df57ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.646+0000 7f86526b0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1b0940 con 0x55902ddcee40 2024-05-27T07:44:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.646+0000 7f86526b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1ae1d0 con 0x55902df4d820 2024-05-27T07:44:58.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.646+0000 7f86526b0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1ac040 con 0x55902df4d410 2024-05-27T07:44:58.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.646+0000 7f8641200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.646+0000 7f8641200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50768/0 (socket says 172.21.15.63:50768) 2024-05-27T07:44:58.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.646+0000 7f8641200700 1 -- 172.21.15.63:0/1732485472 learned_addr learned my addr 172.21.15.63:0/1732485472 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:58.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.646+0000 7f863be00700 1 --2- 172.21.15.63:0/1732485472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d410 0x55902df639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.647+0000 7f8641c00700 1 --2- 172.21.15.63:0/1732485472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d820 0x55902df57ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.647+0000 7f8641200700 1 -- 172.21.15.63:0/1732485472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d410 msgr2=0x55902df639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.647+0000 7f8641200700 1 --2- 172.21.15.63:0/1732485472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d410 0x55902df639d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.647+0000 7f8641200700 1 -- 172.21.15.63:0/1732485472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d820 msgr2=0x55902df57ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.647+0000 7f8641200700 1 --2- 172.21.15.63:0/1732485472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d820 0x55902df57ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.647+0000 7f8641200700 1 -- 172.21.15.63:0/1732485472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55902df431d0 con 0x55902ddcee40 2024-05-27T07:44:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.647+0000 7f8641200700 1 --2- 172.21.15.63:0/1732485472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df517a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8630030fd0 tx=0x7f8630004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f2d85ddeec1144 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.648+0000 7f862d600700 1 -- 172.21.15.63:0/1732485472 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863001e830 con 0x55902ddcee40 2024-05-27T07:44:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.648+0000 7f862d600700 1 -- 172.21.15.63:0/1732485472 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8630030390 con 0x55902ddcee40 2024-05-27T07:44:58.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.648+0000 7f862d600700 1 -- 172.21.15.63:0/1732485472 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863001ee60 con 0x55902ddcee40 2024-05-27T07:44:58.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.649+0000 7f86526b0b80 1 -- 172.21.15.63:0/1732485472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 msgr2=0x55902df517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:58.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.649+0000 7f86526b0b80 1 --2- 172.21.15.63:0/1732485472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df517a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8630030fd0 tx=0x7f8630004730 comp rx=0 tx=0).stop 2024-05-27T07:44:58.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.649+0000 7f86526b0b80 1 -- 172.21.15.63:0/1732485472 shutdown_connections 2024-05-27T07:44:58.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.649+0000 7f86526b0b80 1 --2- 172.21.15.63:0/1732485472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d820 0x55902df57ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.649+0000 7f86526b0b80 1 --2- 172.21.15.63:0/1732485472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902ddcee40 0x55902df517a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.649+0000 7f86526b0b80 1 --2- 172.21.15.63:0/1732485472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902df4d410 0x55902df639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.649+0000 7f86526b0b80 1 -- 172.21.15.63:0/1732485472 >> 172.21.15.63:0/1732485472 conn(0x55902de2efc0 msgr2=0x55902df573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:58.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.649+0000 7f86526b0b80 1 -- 172.21.15.63:0/1732485472 shutdown_connections 2024-05-27T07:44:58.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.649+0000 7f86526b0b80 1 -- 172.21.15.63:0/1732485472 wait complete. 2024-05-27T07:44:58.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.650+0000 7f86526b0b80 1 -- start start 2024-05-27T07:44:58.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.650+0000 7f86526b0b80 1 Processor -- start 2024-05-27T07:44:58.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902ddcee40 0x55902df602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df5b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f86526b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d820 0x55902df5bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f86526b0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1ac690 con 0x55902df4d410 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f86526b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1adb00 con 0x55902df4d820 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f86526b0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1acd60 con 0x55902ddcee40 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f86526b0b80 10 client.0 _ll_register_callbacks cb 0x55902de15c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f863be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df5b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f863be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df5b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50784/0 (socket says 172.21.15.63:50784) 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f863be00700 1 -- 172.21.15.63:0/2007240906 learned_addr learned my addr 172.21.15.63:0/2007240906 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.651+0000 7f8641200700 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902ddcee40 0x55902df602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.652+0000 7f8641c00700 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d820 0x55902df5bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.652+0000 7f863be00700 1 -- 172.21.15.63:0/2007240906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902ddcee40 msgr2=0x55902df602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:58.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.652+0000 7f863be00700 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902ddcee40 0x55902df602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.652+0000 7f863be00700 1 -- 172.21.15.63:0/2007240906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d820 msgr2=0x55902df5bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:44:58.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.652+0000 7f863be00700 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d820 0x55902df5bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.652+0000 7f863be00700 1 -- 172.21.15.63:0/2007240906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55902df4a350 con 0x55902df4d410 2024-05-27T07:44:58.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.652+0000 7f863be00700 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df5b6a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8628009ce0 tx=0x7f8628005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f34ccbaea2051e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:58.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.652+0000 7f8622a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:44:58.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.653+0000 7f8622a00700 1 -- 172.21.15.63:0/2007240906 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628030040 con 0x55902df4d410 2024-05-27T07:44:58.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.653+0000 7f86526b0b80 1 -- 172.21.15.63:0/2007240906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8630030390 con 0x55902df4d410 2024-05-27T07:44:58.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.653+0000 7f8622a00700 1 -- 172.21.15.63:0/2007240906 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f86280050c0 con 0x55902df4d410 2024-05-27T07:44:58.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.653+0000 7f8622a00700 1 -- 172.21.15.63:0/2007240906 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8628047050 con 0x55902df4d410 2024-05-27T07:44:58.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.653+0000 7f86526b0b80 10 client.17640 fetch_fsmap learned FSMap version 33 2024-05-27T07:44:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.653+0000 7f86526b0b80 1 -- 172.21.15.63:0/2007240906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55902dfd2260 con 0x55902df4d410 2024-05-27T07:44:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.653+0000 7f8622a00700 1 -- 172.21.15.63:0/2007240906 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8630030390 con 0x55902df4d410 2024-05-27T07:44:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.654+0000 7f8622a00700 1 -- 172.21.15.63:0/2007240906 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f862800b930 con 0x55902df4d410 2024-05-27T07:44:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.654+0000 7f86526b0b80 10 client.17640 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:44:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.654+0000 7f86526b0b80 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:44:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.654+0000 7f86526b0b80 20 client.17640 populate_metadata read hostname 'smithi063' 2024-05-27T07:44:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.654+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55902dfd2430 0x55902dfd4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.654+0000 7f86526b0b80 4 client.17640 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:44:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.654+0000 7f86526b0b80 1 -- 172.21.15.63:0/2007240906 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55902de77570 con 0x55902dfd2430 2024-05-27T07:44:58.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.655+0000 7f8641200700 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55902dfd2430 0x55902dfd4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.656+0000 7f8641200700 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55902dfd2430 0x55902dfd4850 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c79a46cb24d5b4e9 server_cookie=cc5187f8c82a21b in_seq=0 out_seq=0 2024-05-27T07:44:58.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.656+0000 7f8622a00700 10 client.17640 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.656+0000 7f8622a00700 1 -- 172.21.15.63:0/2007240906 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55902de77570 con 0x55902dfd2430 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.656+0000 7f8622a00700 10 client.17640 handle_command_reply: tid=0 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.656+0000 7f86526b0b80 1 client.17640 shutdown 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.656+0000 7f86526b0b80 1 -- 172.21.15.63:0/2007240906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 msgr2=0x55902df5b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.656+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df5b6a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8628009ce0 tx=0x7f8628005920 comp rx=0 tx=0).stop 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.656+0000 7f86526b0b80 1 -- 172.21.15.63:0/2007240906 shutdown_connections 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.657+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55902dfd2430 0x55902dfd4850 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.657+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55902df4d820 0x55902df5bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.657+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55902ddcee40 0x55902df602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.657+0000 7f86526b0b80 1 --2- 172.21.15.63:0/2007240906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55902df4d410 0x55902df5b6a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.657+0000 7f86526b0b80 1 -- 172.21.15.63:0/2007240906 >> 172.21.15.63:0/2007240906 conn(0x55902de2efc0 msgr2=0x55902df57df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.657+0000 7f8622a00700 0 client.17640 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:44:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.657+0000 7f86526b0b80 1 -- 172.21.15.63:0/2007240906 shutdown_connections 2024-05-27T07:44:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.657+0000 7f86526b0b80 1 -- 172.21.15.63:0/2007240906 wait complete. 2024-05-27T07:44:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.657+0000 7f86526b0b80 20 client.17640 trim_cache size 0 max 16384 2024-05-27T07:44:58.658 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:44:58.658 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:44:58.658 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:44:58.659 INFO:teuthology.orchestra.run.smithi063.stdout: "06c36655-1a65-42b8-a8b5-30f0c6823617": { 2024-05-27T07:44:58.659 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:44:58.659 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "06c36655-1a65-42b8-a8b5-30f0c6823617", 2024-05-27T07:44:58.659 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:44:58.659 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:44:58.659 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:44:58.659 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:44:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 -- 172.21.15.63:0/4096273972 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86240b29f0 msgr2=0x7f86240b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86240b29f0 0x7f86240b4f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8630005950 tx=0x7f863000b410 comp rx=0 tx=0).stop 2024-05-27T07:44:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 -- 172.21.15.63:0/4096273972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f863c110ad0 msgr2=0x7f863c1a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f863c110ad0 0x7f863c1a1640 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8634007ee0 tx=0x7f863400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 -- 172.21.15.63:0/4096273972 shutdown_connections 2024-05-27T07:44:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86240b29f0 0x7f86240b4f00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f863c110ad0 0x7f863c1a1640 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f863c102f20 0x7f863c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 --2- 172.21.15.63:0/4096273972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f863c103870 0x7f863c19d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 -- 172.21.15.63:0/4096273972 >> 172.21.15.63:0/4096273972 conn(0x7f863c078ab0 msgr2=0x7f863c111fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 -- 172.21.15.63:0/4096273972 shutdown_connections 2024-05-27T07:44:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:44:58.658+0000 7f8642600700 1 -- 172.21.15.63:0/4096273972 wait complete. 2024-05-27T07:44:59.591 INFO:tasks.cephfs.filesystem:scrub status for tag:06c36655-1a65-42b8-a8b5-30f0c6823617 - {'path': '/', 'tag': '06c36655-1a65-42b8-a8b5-30f0c6823617', 'options': 'recursive,force'} 2024-05-27T07:44:59.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:44:59 smithi086 ceph-mon[34687]: pgmap v355: 129 pgs: 129 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2024-05-27T07:44:59.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:44:59 smithi149 ceph-mon[35408]: pgmap v355: 129 pgs: 129 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2024-05-27T07:44:59.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:44:59 smithi063 ceph-mon[31613]: pgmap v355: 129 pgs: 129 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2024-05-27T07:45:01.768 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:01 smithi063 ceph-mon[31613]: pgmap v356: 129 pgs: 129 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 37 op/s 2024-05-27T07:45:01.768 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:01 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:45:01.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:01 smithi086 ceph-mon[34687]: pgmap v356: 129 pgs: 129 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 37 op/s 2024-05-27T07:45:01.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:01 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:45:01.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:01 smithi149 ceph-mon[35408]: pgmap v356: 129 pgs: 129 active+clean; 2.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 37 op/s 2024-05-27T07:45:01.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:01 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:45:02.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:45:02.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:45:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:45:02.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:45:02.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:45:02.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:45:03.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:03 smithi086 ceph-mon[34687]: pgmap v357: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.8 MiB/s wr, 58 op/s 2024-05-27T07:45:03.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:03 smithi149 ceph-mon[35408]: pgmap v357: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.8 MiB/s wr, 58 op/s 2024-05-27T07:45:03.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:03 smithi063 ceph-mon[31613]: pgmap v357: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.8 MiB/s wr, 58 op/s 2024-05-27T07:45:04.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:04 smithi086 ceph-mon[34687]: pgmap v358: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 54 op/s 2024-05-27T07:45:04.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:04 smithi149 ceph-mon[35408]: pgmap v358: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 54 op/s 2024-05-27T07:45:04.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:04 smithi063 ceph-mon[31613]: pgmap v358: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 54 op/s 2024-05-27T07:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.593+0000 7f017de00640 1 -- 172.21.15.63:0/1558394260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f017810d950 msgr2=0x7f017810fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.593+0000 7f017de00640 1 --2- 172.21.15.63:0/1558394260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f017810d950 0x7f017810fd70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0164009900 tx=0x7f016402a480 comp rx=0 tx=0).stop 2024-05-27T07:45:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.593+0000 7f017de00640 1 -- 172.21.15.63:0/1558394260 shutdown_connections 2024-05-27T07:45:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.593+0000 7f017de00640 1 --2- 172.21.15.63:0/1558394260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01781102b0 0x7f0178112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.593+0000 7f017de00640 1 --2- 172.21.15.63:0/1558394260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f017810d950 0x7f017810fd70 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.593+0000 7f017de00640 1 --2- 172.21.15.63:0/1558394260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0178076770 0x7f0178076b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.593+0000 7f017de00640 1 -- 172.21.15.63:0/1558394260 >> 172.21.15.63:0/1558394260 conn(0x7f01780ff180 msgr2=0x7f01781015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.593+0000 7f017de00640 1 -- 172.21.15.63:0/1558394260 shutdown_connections 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.593+0000 7f017de00640 1 -- 172.21.15.63:0/1558394260 wait complete. 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.595+0000 7f017de00640 1 Processor -- start 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.595+0000 7f017de00640 1 -- start start 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f017de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0178076770 0x7f017819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f017de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f017810d950 0x7f017819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f017de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01781102b0 0x7f01781a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f017de00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0178115d30 con 0x7f01781102b0 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f017de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0178115bb0 con 0x7f017810d950 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f017de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0178115eb0 con 0x7f0178076770 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f017d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01781102b0 0x7f01781a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f017d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01781102b0 0x7f01781a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33404/0 (socket says 172.21.15.63:33404) 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f017d400640 1 -- 172.21.15.63:0/3532509950 learned_addr learned my addr 172.21.15.63:0/3532509950 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f017ca00640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0178076770 0x7f017819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.596+0000 7f0177e00640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f017810d950 0x7f017819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.597+0000 7f017d400640 1 -- 172.21.15.63:0/3532509950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0178076770 msgr2=0x7f017819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.597+0000 7f017d400640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0178076770 0x7f017819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.597+0000 7f017d400640 1 -- 172.21.15.63:0/3532509950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f017810d950 msgr2=0x7f017819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.597+0000 7f017d400640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f017810d950 0x7f017819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.597+0000 7f017d400640 1 -- 172.21.15.63:0/3532509950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01781a1650 con 0x7f01781102b0 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.597+0000 7f017d400640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01781102b0 0x7f01781a1010 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0170009fd0 tx=0x7f017000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.597+0000 7f0175600640 1 -- 172.21.15.63:0/3532509950 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170017070 con 0x7f01781102b0 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.597+0000 7f017de00640 1 -- 172.21.15.63:0/3532509950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01781a1940 con 0x7f01781102b0 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.597+0000 7f0175600640 1 -- 172.21.15.63:0/3532509950 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0170003e00 con 0x7f01781102b0 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.597+0000 7f0175600640 1 -- 172.21.15.63:0/3532509950 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170010460 con 0x7f01781102b0 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.598+0000 7f017de00640 1 -- 172.21.15.63:0/3532509950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01781a92c0 con 0x7f01781102b0 2024-05-27T07:45:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.601+0000 7f0186428740 1 Processor -- start 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.602+0000 7f0175600640 1 -- 172.21.15.63:0/3532509950 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0170003970 con 0x7f01781102b0 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.602+0000 7f0186428740 1 -- start start 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.602+0000 7f0186428740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7c11b0 0x55f69cb63a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.602+0000 7f0186428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 0x55f69cb63fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.602+0000 7f0186428740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.602+0000 7f0186428740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f69cad4b90 con 0x55f69c7a04d0 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.603+0000 7f0186428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f69cac4730 con 0x55f69c7982f0 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.603+0000 7f0186428740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f69cac8770 con 0x55f69c7c11b0 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.603+0000 7f0177e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.603+0000 7f0177e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33412/0 (socket says 172.21.15.63:33412) 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.603+0000 7f0177e00640 1 -- 172.21.15.63:0/1261630039 learned_addr learned my addr 172.21.15.63:0/1261630039 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.603+0000 7f017ca00640 1 --2- 172.21.15.63:0/1261630039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 0x55f69cb63fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.603+0000 7f0177e00640 1 --2- 172.21.15.63:0/1261630039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7c11b0 0x55f69cb63a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.604+0000 7f0177e00640 1 -- 172.21.15.63:0/1261630039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7c11b0 msgr2=0x55f69cb63a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.604+0000 7f0177e00640 1 --2- 172.21.15.63:0/1261630039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7c11b0 0x55f69cb63a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.604+0000 7f0177e00640 1 -- 172.21.15.63:0/1261630039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 msgr2=0x55f69cb63fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.604+0000 7f0177e00640 1 --2- 172.21.15.63:0/1261630039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 0x55f69cb63fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.604+0000 7f0177e00640 1 -- 172.21.15.63:0/1261630039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f69ca5d8b0 con 0x55f69c7a04d0 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.604+0000 7f017ca00640 1 --2- 172.21.15.63:0/1261630039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 0x55f69cb63fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.604+0000 7f0177e00640 1 --2- 172.21.15.63:0/1261630039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb644e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0164005760 tx=0x7f01640056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f17ce620b802ca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.605+0000 7f0169c00640 1 -- 172.21.15.63:0/1261630039 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164035040 con 0x55f69c7a04d0 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.605+0000 7f0169c00640 1 -- 172.21.15.63:0/1261630039 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01640044c0 con 0x55f69c7a04d0 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.605+0000 7f0169c00640 1 -- 172.21.15.63:0/1261630039 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164003ca0 con 0x55f69c7a04d0 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.605+0000 7f0175600640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01600b2950 0x7f01600b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.605+0000 7f0175600640 1 -- 172.21.15.63:0/3532509950 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f01700e2b80 con 0x7f01781102b0 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.606+0000 7f017ca00640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01600b2950 0x7f01600b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.606+0000 7f0186428740 1 -- 172.21.15.63:0/1261630039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 msgr2=0x55f69cb644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.606+0000 7f0186428740 1 --2- 172.21.15.63:0/1261630039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb644e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0164005760 tx=0x7f01640056d0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.606+0000 7f0186428740 1 -- 172.21.15.63:0/1261630039 shutdown_connections 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.606+0000 7f0186428740 1 --2- 172.21.15.63:0/1261630039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb644e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.606+0000 7f0186428740 1 --2- 172.21.15.63:0/1261630039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 0x55f69cb63fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.606+0000 7f0186428740 1 --2- 172.21.15.63:0/1261630039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7c11b0 0x55f69cb63a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.606+0000 7f0186428740 1 -- 172.21.15.63:0/1261630039 >> 172.21.15.63:0/1261630039 conn(0x55f69c7b5300 msgr2=0x55f69cb60210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.606+0000 7f0186428740 1 -- 172.21.15.63:0/1261630039 shutdown_connections 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.606+0000 7f0186428740 1 -- 172.21.15.63:0/1261630039 wait complete. 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.607+0000 7f017ca00640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01600b2950 0x7f01600b4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55f69cb64e10 tx=0x7f016c00b4d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.607+0000 7f0186428740 1 -- start start 2024-05-27T07:45:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.607+0000 7f0186428740 1 Processor -- start 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.608+0000 7f0186428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 0x55f69cb6dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.608+0000 7f0186428740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb6e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.608+0000 7f0186428740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7c11b0 0x55f69cb79e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.608+0000 7f0186428740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f69c7aa790 con 0x55f69c7a04d0 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.608+0000 7f0186428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f69ca6cbc0 con 0x55f69c7982f0 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.608+0000 7f0186428740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f69cb70b30 con 0x55f69c7c11b0 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.608+0000 7f0186428740 10 client.0 _ll_register_callbacks cb 0x55f69ca4a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.608+0000 7f0177e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb6e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.608+0000 7f0177e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb6e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33416/0 (socket says 172.21.15.63:33416) 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.608+0000 7f0177e00640 1 -- 172.21.15.63:0/1761559540 learned_addr learned my addr 172.21.15.63:0/1761559540 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.609+0000 7f017ca00640 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 0x55f69cb6dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.609+0000 7f017d400640 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7c11b0 0x55f69cb79e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.609+0000 7f0177e00640 1 -- 172.21.15.63:0/1761559540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7c11b0 msgr2=0x55f69cb79e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.609+0000 7f0177e00640 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7c11b0 0x55f69cb79e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.609+0000 7f0177e00640 1 -- 172.21.15.63:0/1761559540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 msgr2=0x55f69cb6dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.609+0000 7f0177e00640 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 0x55f69cb6dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.609+0000 7f0177e00640 1 -- 172.21.15.63:0/1761559540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f69c7d9210 con 0x55f69c7a04d0 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.609+0000 7f0177e00640 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb6e1c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f0164031900 tx=0x7f0164031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0118d1d404c595b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.609+0000 7f015e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.610+0000 7f015e000640 1 -- 172.21.15.63:0/1761559540 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164035050 con 0x55f69c7a04d0 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.610+0000 7f015e000640 1 -- 172.21.15.63:0/1761559540 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f0164003c20 con 0x55f69c7a04d0 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.610+0000 7f0186428740 1 -- 172.21.15.63:0/1761559540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f69cad6ae0 con 0x55f69c7a04d0 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.610+0000 7f015e000640 1 -- 172.21.15.63:0/1761559540 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f016405a050 con 0x55f69c7a04d0 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.610+0000 7f0186428740 10 client.17652 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.610+0000 7f0186428740 1 -- 172.21.15.63:0/1761559540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f69c8a5620 con 0x55f69c7a04d0 2024-05-27T07:45:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.610+0000 7f015e000640 1 -- 172.21.15.63:0/1761559540 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f69cad6ae0 con 0x55f69c7a04d0 2024-05-27T07:45:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.611+0000 7f015e000640 1 -- 172.21.15.63:0/1761559540 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f016403a040 con 0x55f69c7a04d0 2024-05-27T07:45:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.611+0000 7f0186428740 10 client.17652 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.611+0000 7f0186428740 10 client.17652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.611+0000 7f0186428740 20 client.17652 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.611+0000 7f0186428740 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f69c77fd80 0x55f69cb7fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.611+0000 7f0186428740 4 client.17652 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.611+0000 7f0186428740 1 -- 172.21.15.63:0/1761559540 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f69cad02c0 con 0x55f69c77fd80 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.612+0000 7f017ca00640 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f69c77fd80 0x55f69cb7fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.612+0000 7f017ca00640 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f69c77fd80 0x55f69cb7fcb0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b8f6fe57dd1108 server_cookie=ccade9e128f7207a in_seq=0 out_seq=0 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.613+0000 7f015e000640 10 client.17652 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.614+0000 7f015e000640 1 -- 172.21.15.63:0/1761559540 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f69cad02c0 con 0x55f69c77fd80 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.614+0000 7f015e000640 10 client.17652 handle_command_reply: tid=0 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.614+0000 7f0186428740 1 client.17652 shutdown 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 1 -- 172.21.15.63:0/1761559540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 msgr2=0x55f69cb6e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb6e1c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f0164031900 tx=0x7f0164031930 comp rx=0 tx=0).stop 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 1 -- 172.21.15.63:0/1761559540 shutdown_connections 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f69c77fd80 0x55f69cb7fcb0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7c11b0 0x55f69cb79e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7a04d0 0x55f69cb6e1c0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 1 --2- 172.21.15.63:0/1761559540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7982f0 0x55f69cb6dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 1 -- 172.21.15.63:0/1761559540 >> 172.21.15.63:0/1761559540 conn(0x55f69c7b5300 msgr2=0x55f69cb65780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f015e000640 0 client.17652 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 1 -- 172.21.15.63:0/1761559540 shutdown_connections 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 1 -- 172.21.15.63:0/1761559540 wait complete. 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.615+0000 7f0186428740 20 client.17652 trim_cache size 0 max 16384 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.632+0000 7f0186428740 1 Processor -- start 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.632+0000 7f0186428740 1 -- start start 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.633+0000 7f0186428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69cb660b0 0x55f69cb703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.633+0000 7f0186428740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7b5300 0x55f69cb79dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.633+0000 7f0186428740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7982f0 0x55f69cb677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.633+0000 7f0186428740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f01781b5b20 con 0x55f69c7b5300 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.633+0000 7f0186428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01781b33f0 con 0x55f69cb660b0 2024-05-27T07:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.633+0000 7f0186428740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01781b1260 con 0x55f69c7982f0 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.633+0000 7f0177e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7b5300 0x55f69cb79dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.633+0000 7f0177e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7b5300 0x55f69cb79dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33428/0 (socket says 172.21.15.63:33428) 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.633+0000 7f0177e00640 1 -- 172.21.15.63:0/545036920 learned_addr learned my addr 172.21.15.63:0/545036920 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.633+0000 7f017d400640 1 --2- 172.21.15.63:0/545036920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7982f0 0x55f69cb677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f017ca00640 1 --2- 172.21.15.63:0/545036920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69cb660b0 0x55f69cb703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f0177e00640 1 -- 172.21.15.63:0/545036920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7982f0 msgr2=0x55f69cb677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f0177e00640 1 --2- 172.21.15.63:0/545036920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7982f0 0x55f69cb677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f0177e00640 1 -- 172.21.15.63:0/545036920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69cb660b0 msgr2=0x55f69cb703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f0177e00640 1 --2- 172.21.15.63:0/545036920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69cb660b0 0x55f69cb703b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f0177e00640 1 -- 172.21.15.63:0/545036920 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f69cb637f0 con 0x55f69c7b5300 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f017ca00640 1 --2- 172.21.15.63:0/545036920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69cb660b0 0x55f69cb703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f0177e00640 1 --2- 172.21.15.63:0/545036920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7b5300 0x55f69cb79dd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f0164000c00 tx=0x7f01640058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e597a6e8f869f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f0169200640 1 -- 172.21.15.63:0/545036920 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164035040 con 0x55f69c7b5300 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f0169200640 1 -- 172.21.15.63:0/545036920 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f016403d480 con 0x55f69c7b5300 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.634+0000 7f0169200640 1 -- 172.21.15.63:0/545036920 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164031e60 con 0x55f69c7b5300 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.635+0000 7f0186428740 1 -- 172.21.15.63:0/545036920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7b5300 msgr2=0x55f69cb79dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.635+0000 7f0186428740 1 --2- 172.21.15.63:0/545036920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7b5300 0x55f69cb79dd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f0164000c00 tx=0x7f01640058f0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.636+0000 7f0186428740 1 -- 172.21.15.63:0/545036920 shutdown_connections 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.636+0000 7f0186428740 1 --2- 172.21.15.63:0/545036920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69c7982f0 0x55f69cb677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.636+0000 7f0186428740 1 --2- 172.21.15.63:0/545036920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7b5300 0x55f69cb79dd0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.636+0000 7f0186428740 1 --2- 172.21.15.63:0/545036920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69cb660b0 0x55f69cb703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.636+0000 7f0186428740 1 -- 172.21.15.63:0/545036920 >> 172.21.15.63:0/545036920 conn(0x55f69cb64610 msgr2=0x55f69cb68ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.636+0000 7f0186428740 1 -- 172.21.15.63:0/545036920 shutdown_connections 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.636+0000 7f0186428740 1 -- 172.21.15.63:0/545036920 wait complete. 2024-05-27T07:45:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.636+0000 7f0186428740 1 -- start start 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.636+0000 7f0186428740 1 Processor -- start 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.637+0000 7f0186428740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7982f0 0x55f69cbdf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.637+0000 7f0186428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7b5300 0x55f69cbdf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.637+0000 7f0186428740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69cb660b0 0x55f69cb80240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.637+0000 7f0186428740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f01781b18b0 con 0x55f69c7982f0 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.637+0000 7f0186428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01781b2d20 con 0x55f69c7b5300 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.637+0000 7f0186428740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01781b1f80 con 0x55f69cb660b0 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.637+0000 7f0186428740 10 client.0 _ll_register_callbacks cb 0x7f016c00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.637+0000 7f017ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7982f0 0x55f69cbdf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.637+0000 7f017ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7982f0 0x55f69cbdf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33444/0 (socket says 172.21.15.63:33444) 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.637+0000 7f017ca00640 1 -- 172.21.15.63:0/500642876 learned_addr learned my addr 172.21.15.63:0/500642876 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.638+0000 7f0177e00640 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7b5300 0x55f69cbdf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.638+0000 7f017d400640 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69cb660b0 0x55f69cb80240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.638+0000 7f017ca00640 1 -- 172.21.15.63:0/500642876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69cb660b0 msgr2=0x55f69cb80240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.638+0000 7f017ca00640 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69cb660b0 0x55f69cb80240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.638+0000 7f017ca00640 1 -- 172.21.15.63:0/500642876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7b5300 msgr2=0x55f69cbdf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.638+0000 7f017ca00640 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7b5300 0x55f69cbdf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.638+0000 7f017ca00640 1 -- 172.21.15.63:0/500642876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f69cb6eba0 con 0x55f69c7982f0 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.638+0000 7f017d400640 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69cb660b0 0x55f69cb80240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.638+0000 7f017ca00640 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7982f0 0x55f69cbdf4a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f016c01f4a0 tx=0x7f016c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31585ba9fa73330e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.638+0000 7f015e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.639+0000 7f015e000640 1 -- 172.21.15.63:0/500642876 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c02d070 con 0x55f69c7982f0 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.639+0000 7f015e000640 1 -- 172.21.15.63:0/500642876 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f016c029710 con 0x55f69c7982f0 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.639+0000 7f0186428740 1 -- 172.21.15.63:0/500642876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f016403d480 con 0x55f69c7982f0 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.639+0000 7f015e000640 1 -- 172.21.15.63:0/500642876 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f016c03b050 con 0x55f69c7982f0 2024-05-27T07:45:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.639+0000 7f0186428740 10 client.17658 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.639+0000 7f0186428740 1 -- 172.21.15.63:0/500642876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f69cbe78d0 con 0x55f69c7982f0 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.639+0000 7f015e000640 1 -- 172.21.15.63:0/500642876 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f016403d480 con 0x55f69c7982f0 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.640+0000 7f015e000640 1 -- 172.21.15.63:0/500642876 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f016c032bb0 con 0x55f69c7982f0 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.640+0000 7f0186428740 10 client.17658 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.640+0000 7f0186428740 10 client.17658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.640+0000 7f0186428740 20 client.17658 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.640+0000 7f0186428740 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f69c7a04d0 0x55f69cbea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.640+0000 7f0186428740 4 client.17658 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.640+0000 7f0186428740 1 -- 172.21.15.63:0/500642876 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f69ca4a4b0 con 0x55f69c7a04d0 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.640+0000 7f0177e00640 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f69c7a04d0 0x55f69cbea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.641+0000 7f0177e00640 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f69c7a04d0 0x55f69cbea060 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c79d8ea0d38ab923 server_cookie=66a3f0698f6b2c86 in_seq=0 out_seq=0 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.641+0000 7f015e000640 10 client.17658 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.678+0000 7f015e000640 1 -- 172.21.15.63:0/500642876 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f69ca4a4b0 con 0x55f69c7a04d0 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.678+0000 7f015e000640 10 client.17658 handle_command_reply: tid=0 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.678+0000 7f0186428740 1 client.17658 shutdown 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 1 -- 172.21.15.63:0/500642876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7982f0 msgr2=0x55f69cbdf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7982f0 0x55f69cbdf4a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f016c01f4a0 tx=0x7f016c0046c0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 1 -- 172.21.15.63:0/500642876 shutdown_connections 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f69c7a04d0 0x55f69cbea060 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f69cb660b0 0x55f69cb80240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f69c7b5300 0x55f69cbdf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 1 --2- 172.21.15.63:0/500642876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f69c7982f0 0x55f69cbdf4a0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 1 -- 172.21.15.63:0/500642876 >> 172.21.15.63:0/500642876 conn(0x55f69cb64610 msgr2=0x55f69cb6d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f015e000640 0 client.17658 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 1 -- 172.21.15.63:0/500642876 shutdown_connections 2024-05-27T07:45:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 1 -- 172.21.15.63:0/500642876 wait complete. 2024-05-27T07:45:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.679+0000 7f0186428740 20 client.17658 trim_cache size 0 max 16384 2024-05-27T07:45:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:45:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:45:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 -- 172.21.15.63:0/3532509950 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01600b2950 msgr2=0x7f01600b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01600b2950 0x7f01600b4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55f69cb64e10 tx=0x7f016c00b4d0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 -- 172.21.15.63:0/3532509950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01781102b0 msgr2=0x7f01781a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01781102b0 0x7f01781a1010 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0170009fd0 tx=0x7f017000b260 comp rx=0 tx=0).stop 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 -- 172.21.15.63:0/3532509950 shutdown_connections 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01600b2950 0x7f01600b4e60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01781102b0 0x7f01781a1010 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f017810d950 0x7f017819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 --2- 172.21.15.63:0/3532509950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0178076770 0x7f017819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 -- 172.21.15.63:0/3532509950 >> 172.21.15.63:0/3532509950 conn(0x7f01780ff180 msgr2=0x7f0178110700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.680+0000 7f017de00640 1 -- 172.21.15.63:0/3532509950 shutdown_connections 2024-05-27T07:45:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:05.681+0000 7f017de00640 1 -- 172.21.15.63:0/3532509950 wait complete. 2024-05-27T07:45:06.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:06 smithi063 ceph-mon[31613]: pgmap v359: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 54 op/s 2024-05-27T07:45:07.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:06 smithi086 ceph-mon[34687]: pgmap v359: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 54 op/s 2024-05-27T07:45:07.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:06 smithi149 ceph-mon[35408]: pgmap v359: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 54 op/s 2024-05-27T07:45:08.109 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:07 smithi086 ceph-mon[34687]: osdmap e104: 12 total, 12 up, 12 in 2024-05-27T07:45:08.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:07 smithi149 ceph-mon[35408]: osdmap e104: 12 total, 12 up, 12 in 2024-05-27T07:45:08.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:07 smithi063 ceph-mon[31613]: osdmap e104: 12 total, 12 up, 12 in 2024-05-27T07:45:09.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:08 smithi086 ceph-mon[34687]: pgmap v361: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 33 op/s 2024-05-27T07:45:10.396 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:08 smithi063 ceph-mon[31613]: pgmap v361: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 33 op/s 2024-05-27T07:45:10.397 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:08 smithi149 ceph-mon[35408]: pgmap v361: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 33 op/s 2024-05-27T07:45:11.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:10 smithi086 ceph-mon[34687]: pgmap v362: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 33 op/s 2024-05-27T07:45:11.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:10 smithi149 ceph-mon[35408]: pgmap v362: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 33 op/s 2024-05-27T07:45:11.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:10 smithi063 ceph-mon[31613]: pgmap v362: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.3 MiB/s wr, 33 op/s 2024-05-27T07:45:13.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:12 smithi086 ceph-mon[34687]: pgmap v363: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:45:13.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:12 smithi149 ceph-mon[35408]: pgmap v363: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:45:13.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:12 smithi063 ceph-mon[31613]: pgmap v363: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:45:15.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:14 smithi086 ceph-mon[34687]: pgmap v364: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:45:15.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:14 smithi149 ceph-mon[35408]: pgmap v364: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:45:15.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:14 smithi063 ceph-mon[31613]: pgmap v364: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:45:16.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:16 smithi086 ceph-mon[34687]: osdmap e105: 12 total, 12 up, 12 in 2024-05-27T07:45:16.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:16 smithi149 ceph-mon[35408]: osdmap e105: 12 total, 12 up, 12 in 2024-05-27T07:45:16.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:16 smithi063 ceph-mon[31613]: osdmap e105: 12 total, 12 up, 12 in 2024-05-27T07:45:17.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:17 smithi086 ceph-mon[34687]: pgmap v366: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-05-27T07:45:17.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:17 smithi149 ceph-mon[35408]: pgmap v366: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-05-27T07:45:17.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:17 smithi063 ceph-mon[31613]: pgmap v366: 129 pgs: 129 active+clean; 2.8 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 MiB/s wr, 4 op/s 2024-05-27T07:45:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.927+0000 7fbcb1800640 1 -- 172.21.15.63:0/453123524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcac104210 msgr2=0x7fbcac1105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.927+0000 7fbcb1800640 1 --2- 172.21.15.63:0/453123524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcac104210 0x7fbcac1105e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009900 tx=0x7fbca402a410 comp rx=0 tx=0).stop 2024-05-27T07:45:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.927+0000 7fbcb1800640 1 -- 172.21.15.63:0/453123524 shutdown_connections 2024-05-27T07:45:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.927+0000 7fbcb1800640 1 --2- 172.21.15.63:0/453123524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcac104210 0x7fbcac1105e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.927+0000 7fbcb1800640 1 --2- 172.21.15.63:0/453123524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcac109f60 0x7fbcac103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.927+0000 7fbcb1800640 1 --2- 172.21.15.63:0/453123524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcac109ac0 0x7fbcac103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.927+0000 7fbcb1800640 1 -- 172.21.15.63:0/453123524 >> 172.21.15.63:0/453123524 conn(0x7fbcac0ff180 msgr2=0x7fbcac1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.927+0000 7fbcb1800640 1 -- 172.21.15.63:0/453123524 shutdown_connections 2024-05-27T07:45:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.927+0000 7fbcb1800640 1 -- 172.21.15.63:0/453123524 wait complete. 2024-05-27T07:45:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.928+0000 7fbcb1800640 1 Processor -- start 2024-05-27T07:45:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.928+0000 7fbcb1800640 1 -- start start 2024-05-27T07:45:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.929+0000 7fbcb1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcac104210 0x7fbcac0760d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.929+0000 7fbcb1800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcac109ac0 0x7fbcac076610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.929+0000 7fbcb1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcac109f60 0x7fbcac076b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.929+0000 7fbcb1800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac113bd0 con 0x7fbcac109f60 2024-05-27T07:45:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.929+0000 7fbcb1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac113a50 con 0x7fbcac104210 2024-05-27T07:45:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.929+0000 7fbcb1800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac113d50 con 0x7fbcac109ac0 2024-05-27T07:45:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.929+0000 7fbcb0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcac109f60 0x7fbcac076b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.929+0000 7fbcb0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcac109f60 0x7fbcac076b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56918/0 (socket says 172.21.15.63:56918) 2024-05-27T07:45:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.929+0000 7fbcb0e00640 1 -- 172.21.15.63:0/3367559871 learned_addr learned my addr 172.21.15.63:0/3367559871 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.930+0000 7fbcabe00640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcac104210 0x7fbcac0760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.930+0000 7fbcab400640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcac109ac0 0x7fbcac076610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.930+0000 7fbcb0e00640 1 -- 172.21.15.63:0/3367559871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcac109ac0 msgr2=0x7fbcac076610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.930+0000 7fbcb0e00640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcac109ac0 0x7fbcac076610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.930+0000 7fbcb0e00640 1 -- 172.21.15.63:0/3367559871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcac104210 msgr2=0x7fbcac0760d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.930+0000 7fbcb0e00640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcac104210 0x7fbcac0760d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.930+0000 7fbcb0e00640 1 -- 172.21.15.63:0/3367559871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcac1a8910 con 0x7fbcac109f60 2024-05-27T07:45:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.931+0000 7fbcb0e00640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcac109f60 0x7fbcac076b50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fbca4004770 tx=0x7fbca4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.931+0000 7fbca8c00640 1 -- 172.21.15.63:0/3367559871 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca403f070 con 0x7fbcac109f60 2024-05-27T07:45:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.931+0000 7fbca8c00640 1 -- 172.21.15.63:0/3367559871 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbca403baf0 con 0x7fbcac109f60 2024-05-27T07:45:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.931+0000 7fbca8c00640 1 -- 172.21.15.63:0/3367559871 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4035030 con 0x7fbcac109f60 2024-05-27T07:45:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.932+0000 7fbcb1800640 1 -- 172.21.15.63:0/3367559871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcac1a8ba0 con 0x7fbcac109f60 2024-05-27T07:45:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.932+0000 7fbcb1800640 1 -- 172.21.15.63:0/3367559871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcac1a9090 con 0x7fbcac109f60 2024-05-27T07:45:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.935+0000 7fbcb9dd7740 1 Processor -- start 2024-05-27T07:45:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.937+0000 7fbca8c00640 1 -- 172.21.15.63:0/3367559871 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbca4042070 con 0x7fbcac109f60 2024-05-27T07:45:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.937+0000 7fbcb9dd7740 1 -- start start 2024-05-27T07:45:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.938+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78271b0 0x55a4d7bc9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.938+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bc9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.938+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78064d0 0x55a4d7bca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.938+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a4d7b3ab90 con 0x55a4d77fe2f0 2024-05-27T07:45:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.938+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a4d7b2a730 con 0x55a4d78064d0 2024-05-27T07:45:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.938+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a4d7b2e770 con 0x55a4d78271b0 2024-05-27T07:45:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.938+0000 7fbcabe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bc9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.938+0000 7fbcabe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bc9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56932/0 (socket says 172.21.15.63:56932) 2024-05-27T07:45:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.938+0000 7fbcabe00640 1 -- 172.21.15.63:0/883324772 learned_addr learned my addr 172.21.15.63:0/883324772 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.939+0000 7fbcab400640 1 --2- 172.21.15.63:0/883324772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78064d0 0x55a4d7bca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.939+0000 7fbcab400640 1 --2- 172.21.15.63:0/883324772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78271b0 0x55a4d7bc9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.939+0000 7fbcabe00640 1 -- 172.21.15.63:0/883324772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78271b0 msgr2=0x55a4d7bc9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.939+0000 7fbcabe00640 1 --2- 172.21.15.63:0/883324772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78271b0 0x55a4d7bc9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.939+0000 7fbcabe00640 1 -- 172.21.15.63:0/883324772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78064d0 msgr2=0x55a4d7bca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.939+0000 7fbcabe00640 1 --2- 172.21.15.63:0/883324772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78064d0 0x55a4d7bca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.939+0000 7fbcabe00640 1 -- 172.21.15.63:0/883324772 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d7ac38b0 con 0x55a4d77fe2f0 2024-05-27T07:45:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.940+0000 7fbca8c00640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc900b2950 0x7fbc900b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.940+0000 7fbca8c00640 1 -- 172.21.15.63:0/3367559871 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fbca4103090 con 0x7fbcac109f60 2024-05-27T07:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.940+0000 7fbcabe00640 1 --2- 172.21.15.63:0/883324772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bc9e90 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c003a20 tx=0x7fbc9c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde88287e004234d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.940+0000 7fbcab400640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc900b2950 0x7fbc900b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.941+0000 7fbca1600640 1 -- 172.21.15.63:0/883324772 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c004b20 con 0x55a4d77fe2f0 2024-05-27T07:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.941+0000 7fbca1600640 1 -- 172.21.15.63:0/883324772 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc9c004cc0 con 0x55a4d77fe2f0 2024-05-27T07:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.941+0000 7fbca1600640 1 -- 172.21.15.63:0/883324772 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0126b0 con 0x55a4d77fe2f0 2024-05-27T07:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.942+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/883324772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 msgr2=0x55a4d7bc9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.942+0000 7fbcab400640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc900b2950 0x7fbc900b4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55a4d7bcab10 tx=0x7fbc980046c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.942+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/883324772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bc9e90 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c003a20 tx=0x7fbc9c009f90 comp rx=0 tx=0).stop 2024-05-27T07:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.942+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/883324772 shutdown_connections 2024-05-27T07:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.942+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/883324772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78064d0 0x55a4d7bca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.942+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/883324772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bc9e90 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.942+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/883324772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78271b0 0x55a4d7bc9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.942+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/883324772 >> 172.21.15.63:0/883324772 conn(0x55a4d781b300 msgr2=0x55a4d7bc6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.943+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/883324772 shutdown_connections 2024-05-27T07:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.943+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/883324772 wait complete. 2024-05-27T07:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.944+0000 7fbcb9dd7740 1 -- start start 2024-05-27T07:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.944+0000 7fbcb9dd7740 1 Processor -- start 2024-05-27T07:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bdc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 0x55a4d7bdc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78271b0 0x55a4d7be8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a4d7810790 con 0x55a4d78271b0 2024-05-27T07:45:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a4d7ad2bc0 con 0x55a4d77fe2f0 2024-05-27T07:45:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a4d7bdf3a0 con 0x55a4d78064d0 2024-05-27T07:45:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcb9dd7740 10 client.0 _ll_register_callbacks cb 0x55a4d7ab0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcb0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78271b0 0x55a4d7be8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcb0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78271b0 0x55a4d7be8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56948/0 (socket says 172.21.15.63:56948) 2024-05-27T07:45:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcb0e00640 1 -- 172.21.15.63:0/3469978034 learned_addr learned my addr 172.21.15.63:0/3469978034 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.945+0000 7fbcabe00640 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bdc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.946+0000 7fbcab400640 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 0x55a4d7bdc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.946+0000 7fbcb0e00640 1 -- 172.21.15.63:0/3469978034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 msgr2=0x55a4d7bdc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.946+0000 7fbcb0e00640 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 0x55a4d7bdc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.946+0000 7fbcb0e00640 1 -- 172.21.15.63:0/3469978034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d77fe2f0 msgr2=0x55a4d7bdc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.946+0000 7fbcb0e00640 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bdc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.946+0000 7fbcb0e00640 1 -- 172.21.15.63:0/3469978034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d783f210 con 0x55a4d78271b0 2024-05-27T07:45:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.947+0000 7fbcb0e00640 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78271b0 0x55a4d7be8700 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fbca40c60e0 tx=0x7fbca4005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616befbae4e16795 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.947+0000 7fbc96000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.947+0000 7fbc96000640 1 -- 172.21.15.63:0/3469978034 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca40e4060 con 0x55a4d78271b0 2024-05-27T07:45:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.947+0000 7fbc96000640 1 -- 172.21.15.63:0/3469978034 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fbca41006a0 con 0x55a4d78271b0 2024-05-27T07:45:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.947+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3469978034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9c004cc0 con 0x55a4d78271b0 2024-05-27T07:45:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.947+0000 7fbc96000640 1 -- 172.21.15.63:0/3469978034 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbca4049040 con 0x55a4d78271b0 2024-05-27T07:45:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.947+0000 7fbcb9dd7740 10 client.17670 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.947+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3469978034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a4d790b620 con 0x55a4d78271b0 2024-05-27T07:45:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.948+0000 7fbc96000640 1 -- 172.21.15.63:0/3469978034 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc9c004cc0 con 0x55a4d78271b0 2024-05-27T07:45:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.948+0000 7fbc96000640 1 -- 172.21.15.63:0/3469978034 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbca40d2980 con 0x55a4d78271b0 2024-05-27T07:45:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.948+0000 7fbcb9dd7740 10 client.17670 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.948+0000 7fbcb9dd7740 10 client.17670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.948+0000 7fbcb9dd7740 20 client.17670 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.948+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4d77e5d80 0x55a4d7bee740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.948+0000 7fbcb9dd7740 4 client.17670 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.948+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3469978034 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4d7b3cae0 con 0x55a4d77e5d80 2024-05-27T07:45:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.949+0000 7fbcabe00640 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4d77e5d80 0x55a4d7bee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.950+0000 7fbcabe00640 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4d77e5d80 0x55a4d7bee740 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69707b6889abda1b server_cookie=114198d90e2d8d14 in_seq=0 out_seq=0 2024-05-27T07:45:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.950+0000 7fbc96000640 10 client.17670 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbc96000640 1 -- 172.21.15.63:0/3469978034 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a4d7b3cae0 con 0x55a4d77e5d80 2024-05-27T07:45:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbc96000640 10 client.17670 handle_command_reply: tid=0 2024-05-27T07:45:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbcb9dd7740 1 client.17670 shutdown 2024-05-27T07:45:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3469978034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78271b0 msgr2=0x55a4d7be8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78271b0 0x55a4d7be8700 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fbca40c60e0 tx=0x7fbca4005ea0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3469978034 shutdown_connections 2024-05-27T07:45:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4d77e5d80 0x55a4d7bee740 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78271b0 0x55a4d7be8700 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 0x55a4d7bdc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3469978034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bdc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbc96000640 0 client.17670 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.952+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3469978034 >> 172.21.15.63:0/3469978034 conn(0x55a4d781b300 msgr2=0x55a4d7bcb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.953+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3469978034 shutdown_connections 2024-05-27T07:45:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.953+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3469978034 wait complete. 2024-05-27T07:45:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.953+0000 7fbcb9dd7740 20 client.17670 trim_cache size 0 max 16384 2024-05-27T07:45:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.974+0000 7fbcb9dd7740 1 Processor -- start 2024-05-27T07:45:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.974+0000 7fbcb9dd7740 1 -- start start 2024-05-27T07:45:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.975+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bd60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.975+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 0x55a4d7bd65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.975+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 0x55a4d7bf3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.975+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac1b16b0 con 0x55a4d77fe2f0 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.975+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac1aef80 con 0x55a4d78271b0 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.975+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac1acdf0 con 0x55a4d78064d0 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.975+0000 7fbcabe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bd60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.975+0000 7fbcabe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bd60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56952/0 (socket says 172.21.15.63:56952) 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.975+0000 7fbcabe00640 1 -- 172.21.15.63:0/136540538 learned_addr learned my addr 172.21.15.63:0/136540538 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.975+0000 7fbcab400640 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 0x55a4d7bd65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.976+0000 7fbcb0e00640 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 0x55a4d7bf3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.976+0000 7fbcabe00640 1 -- 172.21.15.63:0/136540538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 msgr2=0x55a4d7bd65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.976+0000 7fbcabe00640 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 0x55a4d7bd65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.976+0000 7fbcabe00640 1 -- 172.21.15.63:0/136540538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 msgr2=0x55a4d7bf3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.976+0000 7fbcabe00640 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 0x55a4d7bf3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.976+0000 7fbcabe00640 1 -- 172.21.15.63:0/136540538 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d7bc9e90 con 0x55a4d77fe2f0 2024-05-27T07:45:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.976+0000 7fbcab400640 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 0x55a4d7bd65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.976+0000 7fbcb0e00640 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 0x55a4d7bf3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.976+0000 7fbcabe00640 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bd60a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009f60 tx=0x7fbc9c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323c758cf5a1d896 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.976+0000 7fbca0c00640 1 -- 172.21.15.63:0/136540538 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0046e0 con 0x55a4d77fe2f0 2024-05-27T07:45:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.977+0000 7fbca0c00640 1 -- 172.21.15.63:0/136540538 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc9c02d070 con 0x55a4d77fe2f0 2024-05-27T07:45:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.977+0000 7fbca0c00640 1 -- 172.21.15.63:0/136540538 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c02a430 con 0x55a4d77fe2f0 2024-05-27T07:45:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.978+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/136540538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 msgr2=0x55a4d7bd60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.978+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bd60a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009f60 tx=0x7fbc9c005bd0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.978+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/136540538 shutdown_connections 2024-05-27T07:45:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.978+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 0x55a4d7bf3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.978+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d78064d0 0x55a4d7bd65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.978+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/136540538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d77fe2f0 0x55a4d7bd60a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.978+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/136540538 >> 172.21.15.63:0/136540538 conn(0x55a4d781b300 msgr2=0x55a4d7bdc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.978+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/136540538 shutdown_connections 2024-05-27T07:45:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.978+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/136540538 wait complete. 2024-05-27T07:45:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.978+0000 7fbcb9dd7740 1 -- start start 2024-05-27T07:45:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.979+0000 7fbcb9dd7740 1 Processor -- start 2024-05-27T07:45:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.979+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d77fe2f0 0x55a4d7be4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.979+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78064d0 0x55a4d7be4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.979+0000 7fbcb9dd7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 0x55a4d7be1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.979+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac1ad440 con 0x55a4d78064d0 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.979+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac1ae8b0 con 0x55a4d78271b0 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.979+0000 7fbcb9dd7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac1adb10 con 0x55a4d77fe2f0 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.979+0000 7fbcb9dd7740 10 client.0 _ll_register_callbacks cb 0x7fbca4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcab400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78064d0 0x55a4d7be4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcab400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78064d0 0x55a4d7be4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56964/0 (socket says 172.21.15.63:56964) 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcab400640 1 -- 172.21.15.63:0/3023067022 learned_addr learned my addr 172.21.15.63:0/3023067022 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcabe00640 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d77fe2f0 0x55a4d7be4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcb0e00640 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 0x55a4d7be1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcab400640 1 -- 172.21.15.63:0/3023067022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d77fe2f0 msgr2=0x55a4d7be4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcab400640 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d77fe2f0 0x55a4d7be4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcab400640 1 -- 172.21.15.63:0/3023067022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 msgr2=0x55a4d7be1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcab400640 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 0x55a4d7be1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcab400640 1 -- 172.21.15.63:0/3023067022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d7be2530 con 0x55a4d78064d0 2024-05-27T07:45:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.980+0000 7fbcabe00640 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d77fe2f0 0x55a4d7be4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.981+0000 7fbcab400640 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78064d0 0x55a4d7be4ea0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fbcac10a390 tx=0x7fbc98005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dfff9a528cfec9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.981+0000 7fbc96000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.981+0000 7fbc96000640 1 -- 172.21.15.63:0/3023067022 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98019040 con 0x55a4d78064d0 2024-05-27T07:45:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.981+0000 7fbc96000640 1 -- 172.21.15.63:0/3023067022 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fbc98003d10 con 0x55a4d78064d0 2024-05-27T07:45:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.981+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3023067022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9c02d070 con 0x55a4d78064d0 2024-05-27T07:45:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.981+0000 7fbc96000640 1 -- 172.21.15.63:0/3023067022 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbc98021430 con 0x55a4d78064d0 2024-05-27T07:45:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.981+0000 7fbcb9dd7740 10 client.17676 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.981+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3023067022 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a4d7c56850 con 0x55a4d78064d0 2024-05-27T07:45:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.982+0000 7fbc96000640 1 -- 172.21.15.63:0/3023067022 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc9c02d070 con 0x55a4d78064d0 2024-05-27T07:45:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.982+0000 7fbc96000640 1 -- 172.21.15.63:0/3023067022 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbc9802d5e0 con 0x55a4d78064d0 2024-05-27T07:45:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.982+0000 7fbcb9dd7740 10 client.17676 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.982+0000 7fbcb9dd7740 10 client.17676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.982+0000 7fbcb9dd7740 20 client.17676 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.982+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4d77bb1c0 0x55a4d7c590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.982+0000 7fbcb9dd7740 4 client.17676 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:45:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.982+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3023067022 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4d7b362c0 con 0x55a4d77bb1c0 2024-05-27T07:45:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.983+0000 7fbcabe00640 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4d77bb1c0 0x55a4d7c590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.984+0000 7fbcabe00640 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4d77bb1c0 0x55a4d7c590c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b27bc85cf2da7b server_cookie=386625ff12f935dc in_seq=0 out_seq=0 2024-05-27T07:45:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:18.984+0000 7fbc96000640 10 client.17676 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.018+0000 7fbc96000640 1 -- 172.21.15.63:0/3023067022 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a4d7b362c0 con 0x55a4d77bb1c0 2024-05-27T07:45:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.018+0000 7fbc96000640 10 client.17676 handle_command_reply: tid=0 2024-05-27T07:45:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.018+0000 7fbcb9dd7740 1 client.17676 shutdown 2024-05-27T07:45:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.018+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3023067022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78064d0 msgr2=0x55a4d7be4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.018+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78064d0 0x55a4d7be4ea0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fbcac10a390 tx=0x7fbc98005f90 comp rx=0 tx=0).stop 2024-05-27T07:45:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.019+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3023067022 shutdown_connections 2024-05-27T07:45:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.019+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4d77bb1c0 0x55a4d7c590c0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.019+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4d78271b0 0x55a4d7be1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.019+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4d78064d0 0x55a4d7be4ea0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.019+0000 7fbcb9dd7740 1 --2- 172.21.15.63:0/3023067022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4d77fe2f0 0x55a4d7be4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.019+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3023067022 >> 172.21.15.63:0/3023067022 conn(0x55a4d781b300 msgr2=0x55a4d7bddb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.019+0000 7fbc96000640 0 client.17676 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.019+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3023067022 shutdown_connections 2024-05-27T07:45:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.019+0000 7fbcb9dd7740 1 -- 172.21.15.63:0/3023067022 wait complete. 2024-05-27T07:45:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.019+0000 7fbcb9dd7740 20 client.17676 trim_cache size 0 max 16384 2024-05-27T07:45:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:45:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:45:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:45:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 -- 172.21.15.63:0/3367559871 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc900b2950 msgr2=0x7fbc900b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc900b2950 0x7fbc900b4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55a4d7bcab10 tx=0x7fbc980046c0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 -- 172.21.15.63:0/3367559871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcac109f60 msgr2=0x7fbcac076b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcac109f60 0x7fbcac076b50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fbca4004770 tx=0x7fbca4003b90 comp rx=0 tx=0).stop 2024-05-27T07:45:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 -- 172.21.15.63:0/3367559871 shutdown_connections 2024-05-27T07:45:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc900b2950 0x7fbc900b4e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcac109f60 0x7fbcac076b50 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcac109ac0 0x7fbcac076610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 --2- 172.21.15.63:0/3367559871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcac104210 0x7fbcac0760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 -- 172.21.15.63:0/3367559871 >> 172.21.15.63:0/3367559871 conn(0x7fbcac0ff180 msgr2=0x7fbcac101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 -- 172.21.15.63:0/3367559871 shutdown_connections 2024-05-27T07:45:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:19.020+0000 7fbcb1800640 1 -- 172.21.15.63:0/3367559871 wait complete. 2024-05-27T07:45:19.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:19 smithi086 ceph-mon[34687]: pgmap v367: 129 pgs: 129 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:45:19.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:19 smithi149 ceph-mon[35408]: pgmap v367: 129 pgs: 129 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:45:19.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:19 smithi063 ceph-mon[31613]: pgmap v367: 129 pgs: 129 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:45:20.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:20 smithi086 ceph-mon[34687]: pgmap v368: 129 pgs: 129 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:45:20.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:20 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:45:20.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:20 smithi149 ceph-mon[35408]: pgmap v368: 129 pgs: 129 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:45:20.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:20 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:45:20.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:20 smithi063 ceph-mon[31613]: pgmap v368: 129 pgs: 129 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:45:20.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:20 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:45:21.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:21 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:45:21.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:21 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:45:21.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:21 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:45:21.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:21 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:45:21.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:21 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:45:21.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:21 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:45:21.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:21 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:45:21.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:21 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:45:21.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:21 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:45:22.957 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:22 smithi086 ceph-mon[34687]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:45:22.961 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:22 smithi149 ceph-mon[35408]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:45:22.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:22 smithi063 ceph-mon[31613]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:45:24.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:24 smithi149 ceph-mon[35408]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:45:24.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:24 smithi063 ceph-mon[31613]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:45:25.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:24 smithi086 ceph-mon[34687]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:45:26.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:26 smithi063 ceph-mon[31613]: pgmap v371: 129 pgs: 129 active+clean; 2.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T07:45:27.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:26 smithi086 ceph-mon[34687]: pgmap v371: 129 pgs: 129 active+clean; 2.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T07:45:27.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:26 smithi149 ceph-mon[35408]: pgmap v371: 129 pgs: 129 active+clean; 2.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T07:45:29.014 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:28 smithi149 ceph-mon[35408]: pgmap v372: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T07:45:29.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:28 smithi086 ceph-mon[34687]: pgmap v372: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T07:45:29.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:28 smithi063 ceph-mon[31613]: pgmap v372: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T07:45:29.593 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:45:30.119 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:30.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:30 smithi063 ceph-mon[31613]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:45:31.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:30 smithi086 ceph-mon[34687]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:45:31.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:30 smithi149 ceph-mon[35408]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:45:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.263+0000 7fd5af200640 1 -- 172.21.15.63:0/2420785000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5b010c190 msgr2=0x7fd5b0105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.263+0000 7fd5af200640 1 --2- 172.21.15.63:0/2420785000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5b010c190 0x7fd5b0105dd0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009900 tx=0x7fd59c02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.263+0000 7fd5af200640 1 -- 172.21.15.63:0/2420785000 shutdown_connections 2024-05-27T07:45:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.263+0000 7fd5af200640 1 --2- 172.21.15.63:0/2420785000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b0106440 0x7fd5b0112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.263+0000 7fd5af200640 1 --2- 172.21.15.63:0/2420785000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5b010c190 0x7fd5b0105dd0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.263+0000 7fd5af200640 1 --2- 172.21.15.63:0/2420785000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b010bcf0 0x7fd5b0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.263+0000 7fd5af200640 1 -- 172.21.15.63:0/2420785000 >> 172.21.15.63:0/2420785000 conn(0x7fd5b007d510 msgr2=0x7fd5b007d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.263+0000 7fd5af200640 1 -- 172.21.15.63:0/2420785000 shutdown_connections 2024-05-27T07:45:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.263+0000 7fd5af200640 1 -- 172.21.15.63:0/2420785000 wait complete. 2024-05-27T07:45:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.264+0000 7fd5af200640 1 Processor -- start 2024-05-27T07:45:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.265+0000 7fd5af200640 1 -- start start 2024-05-27T07:45:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.265+0000 7fd5af200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b0106440 0x7fd5b01a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.265+0000 7fd5af200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5b010bcf0 0x7fd5b01a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.265+0000 7fd5af200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b010c190 0x7fd5b019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.265+0000 7fd5af200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0115d90 con 0x7fd5b010bcf0 2024-05-27T07:45:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.265+0000 7fd5af200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0115c10 con 0x7fd5b010c190 2024-05-27T07:45:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.265+0000 7fd5af200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0115f10 con 0x7fd5b0106440 2024-05-27T07:45:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ad400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5b010bcf0 0x7fd5b01a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ad400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5b010bcf0 0x7fd5b01a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57114/0 (socket says 172.21.15.63:57114) 2024-05-27T07:45:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ad400640 1 -- 172.21.15.63:0/1776633645 learned_addr learned my addr 172.21.15.63:0/1776633645 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ae800640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b010c190 0x7fd5b019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ade00640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b0106440 0x7fd5b01a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ad400640 1 -- 172.21.15.63:0/1776633645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b0106440 msgr2=0x7fd5b01a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ad400640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b0106440 0x7fd5b01a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ad400640 1 -- 172.21.15.63:0/1776633645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b010c190 msgr2=0x7fd5b019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ad400640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b010c190 0x7fd5b019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ad400640 1 -- 172.21.15.63:0/1776633645 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b019cdc0 con 0x7fd5b010bcf0 2024-05-27T07:45:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.266+0000 7fd5ae800640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b010c190 0x7fd5b019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.267+0000 7fd5ad400640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5b010bcf0 0x7fd5b01a2a50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd59c005940 tx=0x7fd59c0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.267+0000 7fd5a6a00640 1 -- 172.21.15.63:0/1776633645 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c03f030 con 0x7fd5b010bcf0 2024-05-27T07:45:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.267+0000 7fd5a6a00640 1 -- 172.21.15.63:0/1776633645 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd59c003a80 con 0x7fd5b010bcf0 2024-05-27T07:45:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.267+0000 7fd5af200640 1 -- 172.21.15.63:0/1776633645 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b019cff0 con 0x7fd5b010bcf0 2024-05-27T07:45:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.267+0000 7fd5a6a00640 1 -- 172.21.15.63:0/1776633645 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c035070 con 0x7fd5b010bcf0 2024-05-27T07:45:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.267+0000 7fd5af200640 1 -- 172.21.15.63:0/1776633645 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b01a92b0 con 0x7fd5b010bcf0 2024-05-27T07:45:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.272+0000 7fd5bbfce740 1 Processor -- start 2024-05-27T07:45:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.272+0000 7fd5bbfce740 1 -- start start 2024-05-27T07:45:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a48101b0 0x55e2a4bb2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a47e72f0 0x55e2a4bb2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bb34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e2a4b23b90 con 0x55e2a48101b0 2024-05-27T07:45:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2a4b13730 con 0x55e2a47ef4d0 2024-05-27T07:45:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e2a4b17770 con 0x55e2a47e72f0 2024-05-27T07:45:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5ae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a48101b0 0x55e2a4bb2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5a6a00640 1 -- 172.21.15.63:0/1776633645 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd59c042030 con 0x7fd5b010bcf0 2024-05-27T07:45:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5ae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a48101b0 0x55e2a4bb2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57118/0 (socket says 172.21.15.63:57118) 2024-05-27T07:45:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5ae800640 1 -- 172.21.15.63:0/2931303152 learned_addr learned my addr 172.21.15.63:0/2931303152 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.273+0000 7fd5ade00640 1 --2- 172.21.15.63:0/2931303152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a47e72f0 0x55e2a4bb2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.274+0000 7fd5ad400640 1 --2- 172.21.15.63:0/2931303152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bb34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.274+0000 7fd5ae800640 1 -- 172.21.15.63:0/2931303152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a47e72f0 msgr2=0x55e2a4bb2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.274+0000 7fd5ae800640 1 --2- 172.21.15.63:0/2931303152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a47e72f0 0x55e2a4bb2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.274+0000 7fd5ae800640 1 -- 172.21.15.63:0/2931303152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 msgr2=0x55e2a4bb34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.274+0000 7fd5ae800640 1 --2- 172.21.15.63:0/2931303152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bb34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.274+0000 7fd5ae800640 1 -- 172.21.15.63:0/2931303152 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a4aac8b0 con 0x55e2a48101b0 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.275+0000 7fd5ae800640 1 --2- 172.21.15.63:0/2931303152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a48101b0 0x55e2a4bb2a60 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009fb0 tx=0x7fd5a8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76e1c399e490296 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.275+0000 7fd59b400640 1 -- 172.21.15.63:0/2931303152 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8016070 con 0x55e2a48101b0 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.275+0000 7fd59b400640 1 -- 172.21.15.63:0/2931303152 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5a8012ba0 con 0x55e2a48101b0 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.275+0000 7fd59b400640 1 -- 172.21.15.63:0/2931303152 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a801bb40 con 0x55e2a48101b0 2024-05-27T07:45:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.275+0000 7fd5a6a00640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd5940b29d0 0x7fd5940b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.276+0000 7fd5a6a00640 1 -- 172.21.15.63:0/1776633645 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fd59c102f70 con 0x7fd5b010bcf0 2024-05-27T07:45:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.276+0000 7fd5ade00640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd5940b29d0 0x7fd5940b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.276+0000 7fd5bbfce740 1 -- 172.21.15.63:0/2931303152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a48101b0 msgr2=0x55e2a4bb2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.276+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/2931303152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a48101b0 0x55e2a4bb2a60 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009fb0 tx=0x7fd5a8010800 comp rx=0 tx=0).stop 2024-05-27T07:45:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.277+0000 7fd5bbfce740 1 -- 172.21.15.63:0/2931303152 shutdown_connections 2024-05-27T07:45:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.277+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/2931303152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bb34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.277+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/2931303152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a47e72f0 0x55e2a4bb2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.277+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/2931303152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a48101b0 0x55e2a4bb2a60 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.277+0000 7fd5bbfce740 1 -- 172.21.15.63:0/2931303152 >> 172.21.15.63:0/2931303152 conn(0x55e2a4804300 msgr2=0x55e2a4baf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.277+0000 7fd5ade00640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd5940b29d0 0x7fd5940b4ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55e2a4bb3e10 tx=0x7fd5a000b4d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.277+0000 7fd5bbfce740 1 -- 172.21.15.63:0/2931303152 shutdown_connections 2024-05-27T07:45:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.277+0000 7fd5bbfce740 1 -- 172.21.15.63:0/2931303152 wait complete. 2024-05-27T07:45:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.278+0000 7fd5bbfce740 1 -- start start 2024-05-27T07:45:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.278+0000 7fd5bbfce740 1 Processor -- start 2024-05-27T07:45:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.279+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bbcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.279+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bbd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.279+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a48101b0 0x55e2a4bc8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.279+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e2a47f9790 con 0x55e2a47e72f0 2024-05-27T07:45:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.279+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2a4abbbc0 con 0x55e2a47ef4d0 2024-05-27T07:45:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.279+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e2a4bbfb90 con 0x55e2a48101b0 2024-05-27T07:45:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.279+0000 7fd5bbfce740 10 client.0 _ll_register_callbacks cb 0x55e2a4a99400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.279+0000 7fd5ade00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bbcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.279+0000 7fd5ade00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bbcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57134/0 (socket says 172.21.15.63:57134) 2024-05-27T07:45:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.279+0000 7fd5ade00640 1 -- 172.21.15.63:0/962468563 learned_addr learned my addr 172.21.15.63:0/962468563 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd5ad400640 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bbd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd5ade00640 1 -- 172.21.15.63:0/962468563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a48101b0 msgr2=0x55e2a4bc8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd5ade00640 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a48101b0 0x55e2a4bc8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd5ade00640 1 -- 172.21.15.63:0/962468563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 msgr2=0x55e2a4bbd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd5ade00640 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bbd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd5ade00640 1 -- 172.21.15.63:0/962468563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a4828210 con 0x55e2a47e72f0 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd5ad400640 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bbd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd5ade00640 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bbcc70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000bfd0 tx=0x7fd5a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df320ef722263b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd58fe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd58fe00640 1 -- 172.21.15.63:0/962468563 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a000f910 con 0x55e2a47e72f0 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd5bbfce740 1 -- 172.21.15.63:0/962468563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5a8012ba0 con 0x55e2a47e72f0 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.280+0000 7fd58fe00640 1 -- 172.21.15.63:0/962468563 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fd5a000fab0 con 0x55e2a47e72f0 2024-05-27T07:45:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.281+0000 7fd58fe00640 1 -- 172.21.15.63:0/962468563 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd5a0030050 con 0x55e2a47e72f0 2024-05-27T07:45:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.281+0000 7fd5bbfce740 10 client.17688 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.281+0000 7fd5bbfce740 1 -- 172.21.15.63:0/962468563 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e2a48f4620 con 0x55e2a47e72f0 2024-05-27T07:45:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.281+0000 7fd58fe00640 1 -- 172.21.15.63:0/962468563 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5a8012ba0 con 0x55e2a47e72f0 2024-05-27T07:45:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.281+0000 7fd58fe00640 1 -- 172.21.15.63:0/962468563 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd5a001f3f0 con 0x55e2a47e72f0 2024-05-27T07:45:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.282+0000 7fd5bbfce740 10 client.17688 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.282+0000 7fd5bbfce740 10 client.17688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.282+0000 7fd5bbfce740 20 client.17688 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.282+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2a47ced80 0x55e2a4bcefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.282+0000 7fd5bbfce740 4 client.17688 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.282+0000 7fd5bbfce740 1 -- 172.21.15.63:0/962468563 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a4b25ae0 con 0x55e2a47ced80 2024-05-27T07:45:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.283+0000 7fd5ad400640 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2a47ced80 0x55e2a4bcefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.284+0000 7fd5ad400640 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2a47ced80 0x55e2a4bcefc0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6229c28dc0a647 server_cookie=4bbc0b2f1e19340 in_seq=0 out_seq=0 2024-05-27T07:45:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.284+0000 7fd58fe00640 10 client.17688 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd58fe00640 1 -- 172.21.15.63:0/962468563 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e2a4b25ae0 con 0x55e2a47ced80 2024-05-27T07:45:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd58fe00640 10 client.17688 handle_command_reply: tid=0 2024-05-27T07:45:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd5bbfce740 1 client.17688 shutdown 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd5bbfce740 1 -- 172.21.15.63:0/962468563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 msgr2=0x55e2a4bbcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bbcc70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000bfd0 tx=0x7fd5a00046c0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd5bbfce740 1 -- 172.21.15.63:0/962468563 shutdown_connections 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2a47ced80 0x55e2a4bcefc0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a48101b0 0x55e2a4bc8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bbd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/962468563 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bbcc70 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.287+0000 7fd5bbfce740 1 -- 172.21.15.63:0/962468563 >> 172.21.15.63:0/962468563 conn(0x55e2a4804300 msgr2=0x55e2a4bb2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.288+0000 7fd58fe00640 0 client.17688 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.288+0000 7fd5bbfce740 1 -- 172.21.15.63:0/962468563 shutdown_connections 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.288+0000 7fd5bbfce740 1 -- 172.21.15.63:0/962468563 wait complete. 2024-05-27T07:45:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.288+0000 7fd5bbfce740 20 client.17688 trim_cache size 0 max 16384 2024-05-27T07:45:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.310+0000 7fd5bbfce740 1 Processor -- start 2024-05-27T07:45:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.310+0000 7fd5bbfce740 1 -- start start 2024-05-27T07:45:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.311+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bc8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.311+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.311+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a48101b0 0x55e2a4bd47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.311+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b01b5b30 con 0x55e2a47e72f0 2024-05-27T07:45:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.311+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b01b3400 con 0x55e2a47ef4d0 2024-05-27T07:45:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.311+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b01b1270 con 0x55e2a48101b0 2024-05-27T07:45:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.311+0000 7fd5ade00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bc8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.311+0000 7fd5ade00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bc8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57138/0 (socket says 172.21.15.63:57138) 2024-05-27T07:45:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.311+0000 7fd5ade00640 1 -- 172.21.15.63:0/1868695240 learned_addr learned my addr 172.21.15.63:0/1868695240 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.312+0000 7fd5ae800640 1 --2- 172.21.15.63:0/1868695240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a48101b0 0x55e2a4bd47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.312+0000 7fd5ad400640 1 --2- 172.21.15.63:0/1868695240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.312+0000 7fd5ade00640 1 -- 172.21.15.63:0/1868695240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a48101b0 msgr2=0x55e2a4bd47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.312+0000 7fd5ade00640 1 --2- 172.21.15.63:0/1868695240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a48101b0 0x55e2a4bd47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.312+0000 7fd5ade00640 1 -- 172.21.15.63:0/1868695240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 msgr2=0x55e2a4bc91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.312+0000 7fd5ade00640 1 --2- 172.21.15.63:0/1868695240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.312+0000 7fd5ade00640 1 -- 172.21.15.63:0/1868695240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a47cefb0 con 0x55e2a47e72f0 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.312+0000 7fd5ad400640 1 --2- 172.21.15.63:0/1868695240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.312+0000 7fd5ade00640 1 --2- 172.21.15.63:0/1868695240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bc8c60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd5a002e040 tx=0x7fd5a0010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38b6e4adfc15a1a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.313+0000 7fd59aa00640 1 -- 172.21.15.63:0/1868695240 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a001f3f0 con 0x55e2a47e72f0 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.313+0000 7fd59aa00640 1 -- 172.21.15.63:0/1868695240 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5a0038070 con 0x55e2a47e72f0 2024-05-27T07:45:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.313+0000 7fd59aa00640 1 -- 172.21.15.63:0/1868695240 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0027400 con 0x55e2a47e72f0 2024-05-27T07:45:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.314+0000 7fd5bbfce740 1 -- 172.21.15.63:0/1868695240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 msgr2=0x55e2a4bc8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.314+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/1868695240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bc8c60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd5a002e040 tx=0x7fd5a0010eb0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.315+0000 7fd5bbfce740 1 -- 172.21.15.63:0/1868695240 shutdown_connections 2024-05-27T07:45:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.315+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/1868695240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a48101b0 0x55e2a4bd47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.315+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/1868695240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.315+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/1868695240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47e72f0 0x55e2a4bc8c60 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.315+0000 7fd5bbfce740 1 -- 172.21.15.63:0/1868695240 >> 172.21.15.63:0/1868695240 conn(0x55e2a4804300 msgr2=0x55e2a4bbcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.315+0000 7fd5bbfce740 1 -- 172.21.15.63:0/1868695240 shutdown_connections 2024-05-27T07:45:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.315+0000 7fd5bbfce740 1 -- 172.21.15.63:0/1868695240 wait complete. 2024-05-27T07:45:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.315+0000 7fd5bbfce740 1 -- start start 2024-05-27T07:45:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.315+0000 7fd5bbfce740 1 Processor -- start 2024-05-27T07:45:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.316+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a47e72f0 0x55e2a4bc22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.316+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.316+0000 7fd5bbfce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a48101b0 0x55e2a4c35690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.316+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b01b18c0 con 0x55e2a47ef4d0 2024-05-27T07:45:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.316+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b01b2d30 con 0x55e2a48101b0 2024-05-27T07:45:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.316+0000 7fd5bbfce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b01b1f90 con 0x55e2a47e72f0 2024-05-27T07:45:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.316+0000 7fd5bbfce740 10 client.0 _ll_register_callbacks cb 0x7fd5a000b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ad400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ad400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57140/0 (socket says 172.21.15.63:57140) 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ad400640 1 -- 172.21.15.63:0/260552522 learned_addr learned my addr 172.21.15.63:0/260552522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ae800640 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a48101b0 0x55e2a4c35690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ade00640 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a47e72f0 0x55e2a4bc22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ad400640 1 -- 172.21.15.63:0/260552522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a47e72f0 msgr2=0x55e2a4bc22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ad400640 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a47e72f0 0x55e2a4bc22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ad400640 1 -- 172.21.15.63:0/260552522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a48101b0 msgr2=0x55e2a4c35690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ad400640 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a48101b0 0x55e2a4c35690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ad400640 1 -- 172.21.15.63:0/260552522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a4c35bd0 con 0x55e2a47ef4d0 2024-05-27T07:45:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.317+0000 7fd5ae800640 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a48101b0 0x55e2a4c35690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.318+0000 7fd5ad400640 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc2800 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0c9b30 tx=0x7fd59c02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a402c711d4ae34fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.318+0000 7fd58fe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.318+0000 7fd58fe00640 1 -- 172.21.15.63:0/260552522 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c049040 con 0x55e2a47ef4d0 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.318+0000 7fd58fe00640 1 -- 172.21.15.63:0/260552522 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fd59c100520 con 0x55e2a47ef4d0 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.318+0000 7fd5bbfce740 1 -- 172.21.15.63:0/260552522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5a0038070 con 0x55e2a47ef4d0 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.318+0000 7fd58fe00640 1 -- 172.21.15.63:0/260552522 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd59c100720 con 0x55e2a47ef4d0 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.319+0000 7fd5bbfce740 10 client.17694 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.319+0000 7fd5bbfce740 1 -- 172.21.15.63:0/260552522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e2a4c36d40 con 0x55e2a47ef4d0 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.319+0000 7fd58fe00640 1 -- 172.21.15.63:0/260552522 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5a0038070 con 0x55e2a47ef4d0 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.319+0000 7fd58fe00640 1 -- 172.21.15.63:0/260552522 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd59c0d2800 con 0x55e2a47ef4d0 2024-05-27T07:45:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.319+0000 7fd5bbfce740 10 client.17694 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.319+0000 7fd5bbfce740 10 client.17694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.319+0000 7fd5bbfce740 20 client.17694 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.319+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2a47a41c0 0x55e2a4c39480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.319+0000 7fd5bbfce740 4 client.17694 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:45:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.319+0000 7fd5bbfce740 1 -- 172.21.15.63:0/260552522 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2a4b1f2c0 con 0x55e2a47a41c0 2024-05-27T07:45:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.320+0000 7fd5ade00640 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2a47a41c0 0x55e2a4c39480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.321+0000 7fd5ade00640 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2a47a41c0 0x55e2a4c39480 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=760a285051929a18 server_cookie=7c830d7c8dca6373 in_seq=0 out_seq=0 2024-05-27T07:45:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.321+0000 7fd58fe00640 10 client.17694 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd58fe00640 1 -- 172.21.15.63:0/260552522 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e2a4b1f2c0 con 0x55e2a47a41c0 2024-05-27T07:45:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd58fe00640 10 client.17694 handle_command_reply: tid=0 2024-05-27T07:45:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd5bbfce740 1 client.17694 shutdown 2024-05-27T07:45:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd5bbfce740 1 -- 172.21.15.63:0/260552522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47ef4d0 msgr2=0x55e2a4bc2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc2800 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0c9b30 tx=0x7fd59c02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd5bbfce740 1 -- 172.21.15.63:0/260552522 shutdown_connections 2024-05-27T07:45:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2a47a41c0 0x55e2a4c39480 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a48101b0 0x55e2a4c35690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2a47ef4d0 0x55e2a4bc2800 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd5bbfce740 1 --2- 172.21.15.63:0/260552522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2a47e72f0 0x55e2a4bc22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd5bbfce740 1 -- 172.21.15.63:0/260552522 >> 172.21.15.63:0/260552522 conn(0x55e2a4804300 msgr2=0x55e2a4bbe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.353+0000 7fd58fe00640 0 client.17694 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.354+0000 7fd5bbfce740 1 -- 172.21.15.63:0/260552522 shutdown_connections 2024-05-27T07:45:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.354+0000 7fd5bbfce740 1 -- 172.21.15.63:0/260552522 wait complete. 2024-05-27T07:45:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.354+0000 7fd5bbfce740 20 client.17694 trim_cache size 0 max 16384 2024-05-27T07:45:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:45:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:45:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:45:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 -- 172.21.15.63:0/1776633645 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd5940b29d0 msgr2=0x7fd5940b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd5940b29d0 0x7fd5940b4ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55e2a4bb3e10 tx=0x7fd5a000b4d0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 -- 172.21.15.63:0/1776633645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5b010bcf0 msgr2=0x7fd5b01a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5b010bcf0 0x7fd5b01a2a50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd59c005940 tx=0x7fd59c0040e0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 -- 172.21.15.63:0/1776633645 shutdown_connections 2024-05-27T07:45:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd5940b29d0 0x7fd5940b4ee0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5b010c190 0x7fd5b019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5b010bcf0 0x7fd5b01a2a50 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 --2- 172.21.15.63:0/1776633645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5b0106440 0x7fd5b01a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 -- 172.21.15.63:0/1776633645 >> 172.21.15.63:0/1776633645 conn(0x7fd5b007d510 msgr2=0x7fd5b0102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 -- 172.21.15.63:0/1776633645 shutdown_connections 2024-05-27T07:45:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:32.355+0000 7fd5af200640 1 -- 172.21.15.63:0/1776633645 wait complete. 2024-05-27T07:45:32.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.592+0000 7fa097e00700 1 -- 172.21.15.63:0/2702845471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa098108e90 msgr2=0x7fa0981092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.592+0000 7fa097e00700 1 --2- 172.21.15.63:0/2702845471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa098108e90 0x7fa0981092a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fa0840099c0 tx=0x7fa084009cd0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.592+0000 7fa097e00700 1 -- 172.21.15.63:0/2702845471 shutdown_connections 2024-05-27T07:45:32.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.592+0000 7fa097e00700 1 --2- 172.21.15.63:0/2702845471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0981097e0 0x7fa098111140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.592+0000 7fa097e00700 1 --2- 172.21.15.63:0/2702845471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa098102960 0x7fa098102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.592+0000 7fa097e00700 1 --2- 172.21.15.63:0/2702845471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa098108e90 0x7fa0981092a0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.592+0000 7fa097e00700 1 -- 172.21.15.63:0/2702845471 >> 172.21.15.63:0/2702845471 conn(0x7fa098078360 msgr2=0x7fa0980787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.592+0000 7fa097e00700 1 -- 172.21.15.63:0/2702845471 shutdown_connections 2024-05-27T07:45:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.592+0000 7fa097e00700 1 -- 172.21.15.63:0/2702845471 wait complete. 2024-05-27T07:45:32.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.594+0000 7fa097e00700 1 Processor -- start 2024-05-27T07:45:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.594+0000 7fa097e00700 1 -- start start 2024-05-27T07:45:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.594+0000 7fa097e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa098102960 0x7fa09819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.594+0000 7fa097e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa098108e90 0x7fa09819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.594+0000 7fa097e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0981097e0 0x7fa09819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.594+0000 7fa097e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa098114310 con 0x7fa098102960 2024-05-27T07:45:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.594+0000 7fa097e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa098114190 con 0x7fa098108e90 2024-05-27T07:45:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.594+0000 7fa097e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa098114490 con 0x7fa0981097e0 2024-05-27T07:45:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.595+0000 7fa096a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa098102960 0x7fa09819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.595+0000 7fa096a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa098102960 0x7fa09819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57158/0 (socket says 172.21.15.63:57158) 2024-05-27T07:45:32.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.595+0000 7fa096a00700 1 -- 172.21.15.63:0/3584373731 learned_addr learned my addr 172.21.15.63:0/3584373731 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:32.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.595+0000 7fa096000700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa098108e90 0x7fa09819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.595+0000 7fa097400700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0981097e0 0x7fa09819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.596+0000 7fa096a00700 1 -- 172.21.15.63:0/3584373731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0981097e0 msgr2=0x7fa09819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.596+0000 7fa096a00700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0981097e0 0x7fa09819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.596+0000 7fa096a00700 1 -- 172.21.15.63:0/3584373731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa098108e90 msgr2=0x7fa09819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.596+0000 7fa096a00700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa098108e90 0x7fa09819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.596+0000 7fa096a00700 1 -- 172.21.15.63:0/3584373731 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09819fc10 con 0x7fa098102960 2024-05-27T07:45:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.596+0000 7fa096a00700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa098102960 0x7fa09819ad40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fa08800d950 tx=0x7fa08800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.596+0000 7fa08f400700 1 -- 172.21.15.63:0/3584373731 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088017070 con 0x7fa098102960 2024-05-27T07:45:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.597+0000 7fa08f400700 1 -- 172.21.15.63:0/3584373731 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa08800f690 con 0x7fa098102960 2024-05-27T07:45:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.597+0000 7fa097e00700 1 -- 172.21.15.63:0/3584373731 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09819ff00 con 0x7fa098102960 2024-05-27T07:45:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.597+0000 7fa08f400700 1 -- 172.21.15.63:0/3584373731 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0880039e0 con 0x7fa098102960 2024-05-27T07:45:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.597+0000 7fa097e00700 1 -- 172.21.15.63:0/3584373731 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0981a7820 con 0x7fa098102960 2024-05-27T07:45:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.601+0000 7fa08f400700 1 -- 172.21.15.63:0/3584373731 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa088003b80 con 0x7fa098102960 2024-05-27T07:45:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.602+0000 7fa08f400700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa07c0b29f0 0x7fa07c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.603+0000 7fa0ac690b80 1 Processor -- start 2024-05-27T07:45:32.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.603+0000 7fa08f400700 1 -- 172.21.15.63:0/3584373731 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fa0880e24c0 con 0x7fa098102960 2024-05-27T07:45:32.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.603+0000 7fa096000700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa07c0b29f0 0x7fa07c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.603+0000 7fa0ac690b80 1 -- start start 2024-05-27T07:45:32.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.603+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e386b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.603+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 0x555e8e3870b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.603+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e393ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.603+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555e8e2b0e40 con 0x555e8e2ef9f0 2024-05-27T07:45:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.603+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555e8e2a09e0 con 0x555e8e3916c0 2024-05-27T07:45:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.603+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555e8e2a4a20 con 0x555e8e203e40 2024-05-27T07:45:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.604+0000 7fa096a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e386b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.604+0000 7fa096a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e386b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57164/0 (socket says 172.21.15.63:57164) 2024-05-27T07:45:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.604+0000 7fa096a00700 1 -- 172.21.15.63:0/573242309 learned_addr learned my addr 172.21.15.63:0/573242309 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.604+0000 7fa096000700 1 --2- 172.21.15.63:0/573242309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 0x555e8e3870b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.604+0000 7fa097400700 1 --2- 172.21.15.63:0/573242309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e393ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.604+0000 7fa096000700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa07c0b29f0 0x7fa07c0b4f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa084009990 tx=0x7fa084005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.604+0000 7fa096a00700 1 -- 172.21.15.63:0/573242309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 msgr2=0x555e8e3870b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.604+0000 7fa096a00700 1 --2- 172.21.15.63:0/573242309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 0x555e8e3870b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.604+0000 7fa096a00700 1 -- 172.21.15.63:0/573242309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 msgr2=0x555e8e393ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.605+0000 7fa096a00700 1 --2- 172.21.15.63:0/573242309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e393ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.605+0000 7fa096a00700 1 -- 172.21.15.63:0/573242309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8e269420 con 0x555e8e2ef9f0 2024-05-27T07:45:32.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.605+0000 7fa096a00700 1 --2- 172.21.15.63:0/573242309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e386b70 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa0880a5400 tx=0x7fa0880a8c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d1fa6377d3a245 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.605+0000 7fa083e00700 1 -- 172.21.15.63:0/573242309 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0880b21e0 con 0x555e8e2ef9f0 2024-05-27T07:45:32.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.605+0000 7fa083e00700 1 -- 172.21.15.63:0/573242309 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0880e21b0 con 0x555e8e2ef9f0 2024-05-27T07:45:32.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.605+0000 7fa083e00700 1 -- 172.21.15.63:0/573242309 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0880a95e0 con 0x555e8e2ef9f0 2024-05-27T07:45:32.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.606+0000 7fa0ac690b80 1 -- 172.21.15.63:0/573242309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 msgr2=0x555e8e386b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.606+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/573242309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e386b70 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa0880a5400 tx=0x7fa0880a8c20 comp rx=0 tx=0).stop 2024-05-27T07:45:32.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.607+0000 7fa0ac690b80 1 -- 172.21.15.63:0/573242309 shutdown_connections 2024-05-27T07:45:32.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.607+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/573242309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e393ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.607+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/573242309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e386b70 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.607+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/573242309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 0x555e8e3870b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.607+0000 7fa0ac690b80 1 -- 172.21.15.63:0/573242309 >> 172.21.15.63:0/573242309 conn(0x555e8e29d390 msgr2=0x555e8e37e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.607+0000 7fa0ac690b80 1 -- 172.21.15.63:0/573242309 shutdown_connections 2024-05-27T07:45:32.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.607+0000 7fa0ac690b80 1 -- 172.21.15.63:0/573242309 wait complete. 2024-05-27T07:45:32.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.607+0000 7fa0ac690b80 1 -- start start 2024-05-27T07:45:32.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.608+0000 7fa0ac690b80 1 Processor -- start 2024-05-27T07:45:32.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.608+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 0x555e8e38e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.608+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e38ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.608+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e38f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.608+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555e8e1a6fa0 con 0x555e8e2ef9f0 2024-05-27T07:45:32.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.608+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555e8e241760 con 0x555e8e3916c0 2024-05-27T07:45:32.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.608+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555e8e3941d0 con 0x555e8e203e40 2024-05-27T07:45:32.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.609+0000 7fa0ac690b80 10 client.0 _ll_register_callbacks cb 0x555e8e24ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:32.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.609+0000 7fa096000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e38ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.609+0000 7fa096000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e38ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57178/0 (socket says 172.21.15.63:57178) 2024-05-27T07:45:32.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.609+0000 7fa096000700 1 -- 172.21.15.63:0/164409531 learned_addr learned my addr 172.21.15.63:0/164409531 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:32.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.609+0000 7fa096a00700 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 0x555e8e38e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.609+0000 7fa097400700 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e38f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.610+0000 7fa096000700 1 -- 172.21.15.63:0/164409531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 msgr2=0x555e8e38e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.610+0000 7fa096000700 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 0x555e8e38e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.610+0000 7fa096000700 1 -- 172.21.15.63:0/164409531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 msgr2=0x555e8e38f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.610+0000 7fa096000700 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e38f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.610+0000 7fa096000700 1 -- 172.21.15.63:0/164409531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e8e1f08d0 con 0x555e8e2ef9f0 2024-05-27T07:45:32.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.610+0000 7fa096a00700 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 0x555e8e38e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:32.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.610+0000 7fa096000700 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e38ec20 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa098075f20 tx=0x7fa084009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76be670f41c87b07 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.610+0000 7fa080c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:32.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.610+0000 7fa080c00700 1 -- 172.21.15.63:0/164409531 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084003c70 con 0x555e8e2ef9f0 2024-05-27T07:45:32.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.610+0000 7fa0ac690b80 1 -- 172.21.15.63:0/164409531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0880e21b0 con 0x555e8e2ef9f0 2024-05-27T07:45:32.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.611+0000 7fa080c00700 1 -- 172.21.15.63:0/164409531 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fa08400f460 con 0x555e8e2ef9f0 2024-05-27T07:45:32.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.611+0000 7fa080c00700 1 -- 172.21.15.63:0/164409531 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa084003e10 con 0x555e8e2ef9f0 2024-05-27T07:45:32.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.611+0000 7fa0ac690b80 10 client.17706 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:32.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.611+0000 7fa0ac690b80 1 -- 172.21.15.63:0/164409531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555e8e271060 con 0x555e8e2ef9f0 2024-05-27T07:45:32.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.611+0000 7fa080c00700 1 -- 172.21.15.63:0/164409531 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0880e21b0 con 0x555e8e2ef9f0 2024-05-27T07:45:32.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.611+0000 7fa080c00700 1 -- 172.21.15.63:0/164409531 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa084022070 con 0x555e8e2ef9f0 2024-05-27T07:45:32.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.612+0000 7fa0ac690b80 10 client.17706 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:32.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.612+0000 7fa0ac690b80 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:32.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.612+0000 7fa0ac690b80 20 client.17706 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.612+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555e8e3a11c0 0x555e8e3a3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.612+0000 7fa0ac690b80 4 client.17706 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.612+0000 7fa0ac690b80 1 -- 172.21.15.63:0/164409531 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e8e2b2d90 con 0x555e8e3a11c0 2024-05-27T07:45:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.612+0000 7fa096a00700 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555e8e3a11c0 0x555e8e3a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.613+0000 7fa096a00700 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555e8e3a11c0 0x555e8e3a3640 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86fd7b883b7ea352 server_cookie=fc05ba3fc7fa6b26 in_seq=0 out_seq=0 2024-05-27T07:45:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.613+0000 7fa080c00700 10 client.17706 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.615+0000 7fa080c00700 1 -- 172.21.15.63:0/164409531 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555e8e2b2d90 con 0x555e8e3a11c0 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.615+0000 7fa080c00700 10 client.17706 handle_command_reply: tid=0 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.615+0000 7fa0ac690b80 1 client.17706 shutdown 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 1 -- 172.21.15.63:0/164409531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 msgr2=0x555e8e38ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e38ec20 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa098075f20 tx=0x7fa084009cd0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 1 -- 172.21.15.63:0/164409531 shutdown_connections 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555e8e3a11c0 0x555e8e3a3640 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e38f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e203e40 0x555e8e38e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/164409531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e2ef9f0 0x555e8e38ec20 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 1 -- 172.21.15.63:0/164409531 >> 172.21.15.63:0/164409531 conn(0x555e8e29d390 msgr2=0x555e8e386c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa080c00700 0 client.17706 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:32.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 1 -- 172.21.15.63:0/164409531 shutdown_connections 2024-05-27T07:45:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 1 -- 172.21.15.63:0/164409531 wait complete. 2024-05-27T07:45:32.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.616+0000 7fa0ac690b80 20 client.17706 trim_cache size 0 max 16384 2024-05-27T07:45:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.641+0000 7fa0ac690b80 1 Processor -- start 2024-05-27T07:45:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.641+0000 7fa0ac690b80 1 -- start start 2024-05-27T07:45:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.642+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e389cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.642+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 0x555e8e39d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.642+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e3a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.642+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981b09c0 con 0x555e8e203e40 2024-05-27T07:45:32.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.642+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981ae250 con 0x555e8e3916c0 2024-05-27T07:45:32.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.642+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981ac0c0 con 0x555e8e2ef9f0 2024-05-27T07:45:32.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.642+0000 7fa096a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e389cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.642+0000 7fa096a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e389cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57188/0 (socket says 172.21.15.63:57188) 2024-05-27T07:45:32.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.642+0000 7fa096a00700 1 -- 172.21.15.63:0/235272667 learned_addr learned my addr 172.21.15.63:0/235272667 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:32.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.643+0000 7fa096000700 1 --2- 172.21.15.63:0/235272667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 0x555e8e39d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.643+0000 7fa097400700 1 --2- 172.21.15.63:0/235272667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e3a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.643+0000 7fa096a00700 1 -- 172.21.15.63:0/235272667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 msgr2=0x555e8e39d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.643+0000 7fa096a00700 1 --2- 172.21.15.63:0/235272667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 0x555e8e39d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.643+0000 7fa096a00700 1 -- 172.21.15.63:0/235272667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 msgr2=0x555e8e3a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.643+0000 7fa096a00700 1 --2- 172.21.15.63:0/235272667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e3a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.643+0000 7fa096a00700 1 -- 172.21.15.63:0/235272667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8e393e50 con 0x555e8e203e40 2024-05-27T07:45:32.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.643+0000 7fa096000700 1 --2- 172.21.15.63:0/235272667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 0x555e8e39d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:32.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.643+0000 7fa096a00700 1 --2- 172.21.15.63:0/235272667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e389cc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa088010040 tx=0x7fa088000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c36f91d296034c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.644+0000 7fa083400700 1 -- 172.21.15.63:0/235272667 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0880a5610 con 0x555e8e203e40 2024-05-27T07:45:32.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.644+0000 7fa083400700 1 -- 172.21.15.63:0/235272667 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0880dfa70 con 0x555e8e203e40 2024-05-27T07:45:32.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.644+0000 7fa083400700 1 -- 172.21.15.63:0/235272667 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0880a95e0 con 0x555e8e203e40 2024-05-27T07:45:32.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.645+0000 7fa0ac690b80 1 -- 172.21.15.63:0/235272667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 msgr2=0x555e8e389cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.645+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/235272667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e389cc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa088010040 tx=0x7fa088000bc0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.645+0000 7fa0ac690b80 1 -- 172.21.15.63:0/235272667 shutdown_connections 2024-05-27T07:45:32.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.645+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/235272667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e3a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.645+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/235272667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e389cc0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.645+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/235272667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 0x555e8e39d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.645+0000 7fa0ac690b80 1 -- 172.21.15.63:0/235272667 >> 172.21.15.63:0/235272667 conn(0x555e8e29d390 msgr2=0x555e8e3895b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.645+0000 7fa0ac690b80 1 -- 172.21.15.63:0/235272667 shutdown_connections 2024-05-27T07:45:32.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.645+0000 7fa0ac690b80 1 -- 172.21.15.63:0/235272667 wait complete. 2024-05-27T07:45:32.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.646+0000 7fa0ac690b80 1 -- start start 2024-05-27T07:45:32.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.646+0000 7fa0ac690b80 1 Processor -- start 2024-05-27T07:45:32.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e391bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 0x555e8e3920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa0ac690b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e398490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981ac710 con 0x555e8e203e40 2024-05-27T07:45:32.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981adb80 con 0x555e8e3916c0 2024-05-27T07:45:32.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa0ac690b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0981acde0 con 0x555e8e2ef9f0 2024-05-27T07:45:32.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa0ac690b80 10 client.0 _ll_register_callbacks cb 0x555e8e24ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:32.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa096a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e391bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa096a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e391bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57200/0 (socket says 172.21.15.63:57200) 2024-05-27T07:45:32.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa096a00700 1 -- 172.21.15.63:0/3905387633 learned_addr learned my addr 172.21.15.63:0/3905387633 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:32.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.647+0000 7fa096000700 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 0x555e8e3920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.648+0000 7fa097400700 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e398490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.648+0000 7fa096a00700 1 -- 172.21.15.63:0/3905387633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 msgr2=0x555e8e3920f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.648+0000 7fa096a00700 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 0x555e8e3920f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.648+0000 7fa096a00700 1 -- 172.21.15.63:0/3905387633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 msgr2=0x555e8e398490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:32.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.648+0000 7fa096a00700 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e398490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.648+0000 7fa096a00700 1 -- 172.21.15.63:0/3905387633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e8e3928f0 con 0x555e8e203e40 2024-05-27T07:45:32.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.648+0000 7fa096a00700 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e391bb0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa0880a9310 tx=0x7fa0880a9b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8ed71aed5f6cfd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:32.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.648+0000 7fa080c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:32.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.648+0000 7fa080c00700 1 -- 172.21.15.63:0/3905387633 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0880dfc20 con 0x555e8e203e40 2024-05-27T07:45:32.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.649+0000 7fa080c00700 1 -- 172.21.15.63:0/3905387633 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fa088069be0 con 0x555e8e203e40 2024-05-27T07:45:32.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.649+0000 7fa0ac690b80 1 -- 172.21.15.63:0/3905387633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0880dfa70 con 0x555e8e203e40 2024-05-27T07:45:32.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.649+0000 7fa080c00700 1 -- 172.21.15.63:0/3905387633 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa088069de0 con 0x555e8e203e40 2024-05-27T07:45:32.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.649+0000 7fa0ac690b80 10 client.17712 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:32.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.649+0000 7fa0ac690b80 1 -- 172.21.15.63:0/3905387633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555e8e3998e0 con 0x555e8e203e40 2024-05-27T07:45:32.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.650+0000 7fa080c00700 1 -- 172.21.15.63:0/3905387633 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0880dfa70 con 0x555e8e203e40 2024-05-27T07:45:32.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.650+0000 7fa080c00700 1 -- 172.21.15.63:0/3905387633 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa0880b1d50 con 0x555e8e203e40 2024-05-27T07:45:32.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.650+0000 7fa0ac690b80 10 client.17712 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:32.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.650+0000 7fa0ac690b80 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:32.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.650+0000 7fa0ac690b80 20 client.17712 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:32.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.650+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555e8e399bc0 0x555e8e40dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:32.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.650+0000 7fa0ac690b80 4 client.17712 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:45:32.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.650+0000 7fa0ac690b80 1 -- 172.21.15.63:0/3905387633 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e8e2ac570 con 0x555e8e399bc0 2024-05-27T07:45:32.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.651+0000 7fa096000700 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555e8e399bc0 0x555e8e40dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:32.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.652+0000 7fa096000700 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555e8e399bc0 0x555e8e40dd20 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f1dc7771be9e05 server_cookie=36a2b4410d461921 in_seq=0 out_seq=0 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.652+0000 7fa080c00700 10 client.17712 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.652+0000 7fa080c00700 1 -- 172.21.15.63:0/3905387633 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e8e2ac570 con 0x555e8e399bc0 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.652+0000 7fa080c00700 10 client.17712 handle_command_reply: tid=0 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.652+0000 7fa0ac690b80 1 client.17712 shutdown 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.652+0000 7fa0ac690b80 1 -- 172.21.15.63:0/3905387633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 msgr2=0x555e8e391bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.652+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e391bb0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa0880a9310 tx=0x7fa0880a9b50 comp rx=0 tx=0).stop 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.653+0000 7fa0ac690b80 1 -- 172.21.15.63:0/3905387633 shutdown_connections 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.653+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555e8e399bc0 0x555e8e40dd20 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.653+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555e8e3916c0 0x555e8e398490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.653+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555e8e203e40 0x555e8e391bb0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.653+0000 7fa0ac690b80 1 --2- 172.21.15.63:0/3905387633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e8e2ef9f0 0x555e8e3920f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.653+0000 7fa080c00700 0 client.17712 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:32.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.653+0000 7fa0ac690b80 1 -- 172.21.15.63:0/3905387633 >> 172.21.15.63:0/3905387633 conn(0x555e8e29d390 msgr2=0x555e8e388560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.653+0000 7fa0ac690b80 1 -- 172.21.15.63:0/3905387633 shutdown_connections 2024-05-27T07:45:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.653+0000 7fa0ac690b80 1 -- 172.21.15.63:0/3905387633 wait complete. 2024-05-27T07:45:32.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.653+0000 7fa0ac690b80 20 client.17712 trim_cache size 0 max 16384 2024-05-27T07:45:32.662 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:45:32.662 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:45:32.662 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:45:32.662 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:45:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 -- 172.21.15.63:0/3584373731 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa07c0b29f0 msgr2=0x7fa07c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa07c0b29f0 0x7fa07c0b4f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa084009990 tx=0x7fa084005fb0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 -- 172.21.15.63:0/3584373731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa098102960 msgr2=0x7fa09819ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa098102960 0x7fa09819ad40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fa08800d950 tx=0x7fa08800dc60 comp rx=0 tx=0).stop 2024-05-27T07:45:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 -- 172.21.15.63:0/3584373731 shutdown_connections 2024-05-27T07:45:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa07c0b29f0 0x7fa07c0b4f00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0981097e0 0x7fa09819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa098102960 0x7fa09819ad40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 --2- 172.21.15.63:0/3584373731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa098108e90 0x7fa09819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 -- 172.21.15.63:0/3584373731 >> 172.21.15.63:0/3584373731 conn(0x7fa098078360 msgr2=0x7fa098100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 -- 172.21.15.63:0/3584373731 shutdown_connections 2024-05-27T07:45:32.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:32.654+0000 7fa097e00700 1 -- 172.21.15.63:0/3584373731 wait complete. 2024-05-27T07:45:32.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:32 smithi063 ceph-mon[31613]: pgmap v374: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:45:33.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:32 smithi086 ceph-mon[34687]: pgmap v374: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:45:33.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:32 smithi149 ceph-mon[35408]: pgmap v374: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:45:33.637 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:45:33.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:45:34.130 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:34.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:34 smithi063 ceph-mon[31613]: pgmap v375: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:45:35.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:34 smithi086 ceph-mon[34687]: pgmap v375: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:45:35.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:34 smithi149 ceph-mon[35408]: pgmap v375: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:45:36.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.678+0000 7f9cb6200700 1 -- 172.21.15.63:0/3191497214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cb010e6f0 msgr2=0x7f9cb0110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:36.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.678+0000 7f9cb6200700 1 --2- 172.21.15.63:0/3191497214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cb010e6f0 0x7f9cb0110bd0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800b0a0 tx=0x7f9ca800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:45:36.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.678+0000 7f9cb6200700 1 -- 172.21.15.63:0/3191497214 shutdown_connections 2024-05-27T07:45:36.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.678+0000 7f9cb6200700 1 --2- 172.21.15.63:0/3191497214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cb010e6f0 0x7f9cb0110bd0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:36.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.678+0000 7f9cb6200700 1 --2- 172.21.15.63:0/3191497214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0105280 0x7f9cb0105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:36.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.678+0000 7f9cb6200700 1 --2- 172.21.15.63:0/3191497214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb0105bd0 0x7f9cb010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:36.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.678+0000 7f9cb6200700 1 -- 172.21.15.63:0/3191497214 >> 172.21.15.63:0/3191497214 conn(0x7f9cb0078af0 msgr2=0x7f9cb0078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:36.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.678+0000 7f9cb6200700 1 -- 172.21.15.63:0/3191497214 shutdown_connections 2024-05-27T07:45:36.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.678+0000 7f9cb6200700 1 -- 172.21.15.63:0/3191497214 wait complete. 2024-05-27T07:45:36.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.679+0000 7f9cb6200700 1 Processor -- start 2024-05-27T07:45:36.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.680+0000 7f9cb6200700 1 -- start start 2024-05-27T07:45:36.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.680+0000 7f9cb6200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0105280 0x7f9cb01a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:36.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.680+0000 7f9cb6200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cb0105bd0 0x7f9cb01a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:36.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.680+0000 7f9cb6200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb010e6f0 0x7f9cb019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:36.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.680+0000 7f9cb6200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb0114150 con 0x7f9cb0105bd0 2024-05-27T07:45:36.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.680+0000 7f9cb6200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb0113fd0 con 0x7f9cb0105280 2024-05-27T07:45:36.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.680+0000 7f9cb6200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb01142d0 con 0x7f9cb010e6f0 2024-05-27T07:45:36.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.680+0000 7f9cafe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cb0105bd0 0x7f9cb01a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:36.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.681+0000 7f9cafe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cb0105bd0 0x7f9cb01a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57212/0 (socket says 172.21.15.63:57212) 2024-05-27T07:45:36.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.681+0000 7f9cafe00700 1 -- 172.21.15.63:0/3011212194 learned_addr learned my addr 172.21.15.63:0/3011212194 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:36.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.681+0000 7f9cb4e00700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0105280 0x7f9cb01a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:36.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.681+0000 7f9cb5800700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb010e6f0 0x7f9cb019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:36.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.681+0000 7f9cafe00700 1 -- 172.21.15.63:0/3011212194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb010e6f0 msgr2=0x7f9cb019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:36.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.681+0000 7f9cafe00700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb010e6f0 0x7f9cb019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:36.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.681+0000 7f9cafe00700 1 -- 172.21.15.63:0/3011212194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0105280 msgr2=0x7f9cb01a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:36.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.681+0000 7f9cafe00700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0105280 0x7f9cb01a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:36.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.681+0000 7f9cafe00700 1 -- 172.21.15.63:0/3011212194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb019b000 con 0x7f9cb0105bd0 2024-05-27T07:45:36.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.681+0000 7f9cb5800700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb010e6f0 0x7f9cb019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:36.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.682+0000 7f9cafe00700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cb0105bd0 0x7f9cb01a0c70 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00e9a0 tx=0x7f9c9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:36.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.682+0000 7f9cad600700 1 -- 172.21.15.63:0/3011212194 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c00cbd0 con 0x7f9cb0105bd0 2024-05-27T07:45:36.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.682+0000 7f9cad600700 1 -- 172.21.15.63:0/3011212194 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c9c00cd70 con 0x7f9cb0105bd0 2024-05-27T07:45:36.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.682+0000 7f9cb6200700 1 -- 172.21.15.63:0/3011212194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb019b2f0 con 0x7f9cb0105bd0 2024-05-27T07:45:36.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.682+0000 7f9cad600700 1 -- 172.21.15.63:0/3011212194 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c018680 con 0x7f9cb0105bd0 2024-05-27T07:45:36.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.682+0000 7f9cb6200700 1 -- 172.21.15.63:0/3011212194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb01a75c0 con 0x7f9cb0105bd0 2024-05-27T07:45:36.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.684+0000 7f9cb6200700 1 -- 172.21.15.63:0/3011212194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb006c910 con 0x7f9cb0105bd0 2024-05-27T07:45:36.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.688+0000 7f9cad600700 1 -- 172.21.15.63:0/3011212194 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9c9c010430 con 0x7f9cb0105bd0 2024-05-27T07:45:36.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.690+0000 7f9cad600700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c980b29f0 0x7f9c980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:36.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.690+0000 7f9cad600700 1 -- 172.21.15.63:0/3011212194 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f9c9c0e2e00 con 0x7f9cb0105bd0 2024-05-27T07:45:36.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.690+0000 7f9cb4e00700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c980b29f0 0x7f9c980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:36.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.691+0000 7f9cb4e00700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c980b29f0 0x7f9c980b4f00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9cb019ba90 tx=0x7f9ca4009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:36.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.693+0000 7f9cad600700 1 -- 172.21.15.63:0/3011212194 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c9c0aa530 con 0x7f9cb0105bd0 2024-05-27T07:45:36.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:36 smithi063 ceph-mon[31613]: pgmap v376: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:45:36.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.996+0000 7f9cb6200700 1 -- 172.21.15.63:0/3011212194 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cb019be80 con 0x7f9cb0105bd0 2024-05-27T07:45:36.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:36.997+0000 7f9cad600700 1 -- 172.21.15.63:0/3011212194 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f9c9c0a9e30 con 0x7f9cb0105bd0 2024-05-27T07:45:36.996 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:45:36.996 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:45:37.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.000+0000 7f9cb6200700 1 -- 172.21.15.63:0/3011212194 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c980b29f0 msgr2=0x7f9c980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:37.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.000+0000 7f9cb6200700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c980b29f0 0x7f9c980b4f00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9cb019ba90 tx=0x7f9ca4009260 comp rx=0 tx=0).stop 2024-05-27T07:45:37.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.000+0000 7f9cb6200700 1 -- 172.21.15.63:0/3011212194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cb0105bd0 msgr2=0x7f9cb01a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:37.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.000+0000 7f9cb6200700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cb0105bd0 0x7f9cb01a0c70 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00e9a0 tx=0x7f9c9c00ecb0 comp rx=0 tx=0).stop 2024-05-27T07:45:37.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.001+0000 7f9cb6200700 1 -- 172.21.15.63:0/3011212194 shutdown_connections 2024-05-27T07:45:37.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.001+0000 7f9cb6200700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c980b29f0 0x7f9c980b4f00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:37.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.001+0000 7f9cb6200700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cb010e6f0 0x7f9cb019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:37.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.001+0000 7f9cb6200700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0105280 0x7f9cb01a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:37.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.001+0000 7f9cb6200700 1 --2- 172.21.15.63:0/3011212194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cb0105bd0 0x7f9cb01a0c70 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:37.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.001+0000 7f9cb6200700 1 -- 172.21.15.63:0/3011212194 >> 172.21.15.63:0/3011212194 conn(0x7f9cb0078af0 msgr2=0x7f9cb00fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:37.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.001+0000 7f9cb6200700 1 -- 172.21.15.63:0/3011212194 shutdown_connections 2024-05-27T07:45:37.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:37.001+0000 7f9cb6200700 1 -- 172.21.15.63:0/3011212194 wait complete. 2024-05-27T07:45:37.001 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:45:37.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:36 smithi086 ceph-mon[34687]: pgmap v376: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:45:37.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:36 smithi149 ceph-mon[35408]: pgmap v376: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:45:38.001 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:37 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3011212194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:45:38.006 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:45:38.110 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:37 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3011212194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:45:38.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:37 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3011212194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:45:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:38.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:38 smithi063 ceph-mon[31613]: pgmap v377: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:45:39.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:38 smithi086 ceph-mon[34687]: pgmap v377: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:45:39.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:38 smithi149 ceph-mon[35408]: pgmap v377: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:45:40.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.940+0000 7f7033400700 1 -- 172.21.15.63:0/3191747783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c075b50 msgr2=0x7f702c075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:40.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.940+0000 7f7033400700 1 --2- 172.21.15.63:0/3191747783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c075b50 0x7f702c075f60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7020009990 tx=0x7f7020009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.941+0000 7f7033400700 1 -- 172.21.15.63:0/3191747783 shutdown_connections 2024-05-27T07:45:40.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.941+0000 7f7033400700 1 --2- 172.21.15.63:0/3191747783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f702c115360 0x7f702c117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.941+0000 7f7033400700 1 --2- 172.21.15.63:0/3191747783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c075b50 0x7f702c075f60 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.941+0000 7f7033400700 1 --2- 172.21.15.63:0/3191747783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c0764a0 0x7f702c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.941+0000 7f7033400700 1 -- 172.21.15.63:0/3191747783 >> 172.21.15.63:0/3191747783 conn(0x7f702c100030 msgr2=0x7f702c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:40.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.941+0000 7f7033400700 1 -- 172.21.15.63:0/3191747783 shutdown_connections 2024-05-27T07:45:40.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.941+0000 7f7033400700 1 -- 172.21.15.63:0/3191747783 wait complete. 2024-05-27T07:45:40.941 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:40 smithi063 ceph-mon[31613]: pgmap v378: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:45:40.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.942+0000 7f7033400700 1 Processor -- start 2024-05-27T07:45:40.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.942+0000 7f7033400700 1 -- start start 2024-05-27T07:45:40.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.943+0000 7f7033400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c075b50 0x7f702c1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.943+0000 7f7033400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f702c0764a0 0x7f702c1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.943+0000 7f7033400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c115360 0x7f702c1a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.943+0000 7f7033400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f702c11a960 con 0x7f702c075b50 2024-05-27T07:45:40.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.943+0000 7f7033400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f702c11a7e0 con 0x7f702c115360 2024-05-27T07:45:40.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.943+0000 7f7033400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f702c11aae0 con 0x7f702c0764a0 2024-05-27T07:45:40.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.943+0000 7f7030a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c075b50 0x7f702c1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.943+0000 7f7030a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c075b50 0x7f702c1a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60154/0 (socket says 172.21.15.63:60154) 2024-05-27T07:45:40.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.943+0000 7f7030a00700 1 -- 172.21.15.63:0/4019518439 learned_addr learned my addr 172.21.15.63:0/4019518439 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:40.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.944+0000 7f7031400700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c115360 0x7f702c1a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.944+0000 7f702be00700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f702c0764a0 0x7f702c1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.944+0000 7f7030a00700 1 -- 172.21.15.63:0/4019518439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f702c0764a0 msgr2=0x7f702c1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:40.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.944+0000 7f7030a00700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f702c0764a0 0x7f702c1a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.944+0000 7f7030a00700 1 -- 172.21.15.63:0/4019518439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c115360 msgr2=0x7f702c1a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:40.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.944+0000 7f7030a00700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c115360 0x7f702c1a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.944+0000 7f7030a00700 1 -- 172.21.15.63:0/4019518439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c1a6110 con 0x7f702c075b50 2024-05-27T07:45:40.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.945+0000 7f7030a00700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c075b50 0x7f702c1a12f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7020000c00 tx=0x7f70200036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:40.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.945+0000 7f7029600700 1 -- 172.21.15.63:0/4019518439 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020027070 con 0x7f702c075b50 2024-05-27T07:45:40.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.945+0000 7f7029600700 1 -- 172.21.15.63:0/4019518439 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f702000fc80 con 0x7f702c075b50 2024-05-27T07:45:40.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.945+0000 7f7033400700 1 -- 172.21.15.63:0/4019518439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c1a63a0 con 0x7f702c075b50 2024-05-27T07:45:40.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.945+0000 7f7029600700 1 -- 172.21.15.63:0/4019518439 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020022d60 con 0x7f702c075b50 2024-05-27T07:45:40.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.945+0000 7f7033400700 1 -- 172.21.15.63:0/4019518439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c1a68d0 con 0x7f702c075b50 2024-05-27T07:45:40.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.949+0000 7f7029600700 1 -- 172.21.15.63:0/4019518439 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f702002a070 con 0x7f702c075b50 2024-05-27T07:45:40.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.951+0000 7f7041f0bb80 1 Processor -- start 2024-05-27T07:45:40.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.951+0000 7f7041f0bb80 1 -- start start 2024-05-27T07:45:40.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.951+0000 7f7029600700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70140b2a70 0x7f70140b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b71330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b71870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f5e2a9fe40 con 0x55f5e2b71e70 2024-05-27T07:45:40.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5e2a8f9e0 con 0x55f5e2ade9f0 2024-05-27T07:45:40.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f5e2a93a20 con 0x55f5e29f2e40 2024-05-27T07:45:40.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f7029600700 1 -- 172.21.15.63:0/4019518439 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f70200e3a10 con 0x7f702c075b50 2024-05-27T07:45:40.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f7031400700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70140b2a70 0x7f70140b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f7030a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b71330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f702be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b71870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f702be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b71870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:36268/0 (socket says 172.21.15.63:36268) 2024-05-27T07:45:40.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f702be00700 1 -- 172.21.15.63:0/1168194793 learned_addr learned my addr 172.21.15.63:0/1168194793 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:40.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.952+0000 7f7031400700 1 --2- 172.21.15.63:0/1168194793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.953+0000 7f7030a00700 1 -- 172.21.15.63:0/1168194793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 msgr2=0x55f5e2b71870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:40.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.953+0000 7f7030a00700 1 --2- 172.21.15.63:0/1168194793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b71870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.953+0000 7f7030a00700 1 -- 172.21.15.63:0/1168194793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 msgr2=0x55f5e2b7e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:40.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.953+0000 7f7030a00700 1 --2- 172.21.15.63:0/1168194793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.953+0000 7f7030a00700 1 -- 172.21.15.63:0/1168194793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5e2a58420 con 0x55f5e2ade9f0 2024-05-27T07:45:40.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.953+0000 7f7031400700 1 --2- 172.21.15.63:0/1168194793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:45:40.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.954+0000 7f7031400700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70140b2a70 0x7f70140b4f80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7024007fd0 tx=0x7f70240132f0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:40.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.954+0000 7f7030a00700 1 --2- 172.21.15.63:0/1168194793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b71330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f70200e0d40 tx=0x7f70200aa310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0915e056d25c6fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:40.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.954+0000 7f701e000700 1 -- 172.21.15.63:0/1168194793 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70200b3070 con 0x55f5e2ade9f0 2024-05-27T07:45:40.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.954+0000 7f701e000700 1 -- 172.21.15.63:0/1168194793 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70200aa630 con 0x55f5e2ade9f0 2024-05-27T07:45:40.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.954+0000 7f701e000700 1 -- 172.21.15.63:0/1168194793 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70200e0fc0 con 0x55f5e2ade9f0 2024-05-27T07:45:40.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.955+0000 7f7041f0bb80 1 -- 172.21.15.63:0/1168194793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 msgr2=0x55f5e2b71330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:40.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.955+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/1168194793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b71330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f70200e0d40 tx=0x7f70200aa310 comp rx=0 tx=0).stop 2024-05-27T07:45:40.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.955+0000 7f7041f0bb80 1 -- 172.21.15.63:0/1168194793 shutdown_connections 2024-05-27T07:45:40.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.955+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/1168194793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.955+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/1168194793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b71330 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.955+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/1168194793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b71870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.955+0000 7f7041f0bb80 1 -- 172.21.15.63:0/1168194793 >> 172.21.15.63:0/1168194793 conn(0x55f5e2a8c390 msgr2=0x55f5e2b6d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:40.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.955+0000 7f7041f0bb80 1 -- 172.21.15.63:0/1168194793 shutdown_connections 2024-05-27T07:45:40.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.956+0000 7f7041f0bb80 1 -- 172.21.15.63:0/1168194793 wait complete. 2024-05-27T07:45:40.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.956+0000 7f7041f0bb80 1 -- start start 2024-05-27T07:45:40.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.956+0000 7f7041f0bb80 1 Processor -- start 2024-05-27T07:45:40.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.957+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b7b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.957+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.957+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.957+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f5e2995fa0 con 0x55f5e2b71e70 2024-05-27T07:45:40.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.957+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5e2a30760 con 0x55f5e2ade9f0 2024-05-27T07:45:40.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.957+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f5e2b7e940 con 0x55f5e29f2e40 2024-05-27T07:45:40.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.957+0000 7f7041f0bb80 10 client.0 _ll_register_callbacks cb 0x55f5e2a39c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:40.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.957+0000 7f7031400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.957+0000 7f7031400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60184/0 (socket says 172.21.15.63:60184) 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.957+0000 7f7031400700 1 -- 172.21.15.63:0/275857079 learned_addr learned my addr 172.21.15.63:0/275857079 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.958+0000 7f702be00700 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.958+0000 7f7030a00700 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b7b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.958+0000 7f7031400700 1 -- 172.21.15.63:0/275857079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 msgr2=0x55f5e2b7b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.958+0000 7f7031400700 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b7b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.958+0000 7f7031400700 1 -- 172.21.15.63:0/275857079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 msgr2=0x55f5e2b7bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.958+0000 7f7031400700 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b7bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.958+0000 7f7031400700 1 -- 172.21.15.63:0/275857079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5e29df8d0 con 0x55f5e2b71e70 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.958+0000 7f7030a00700 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b7b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.958+0000 7f7031400700 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7c0c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f70240079d0 tx=0x7f702401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a4f3d4d877f17d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:40.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.958+0000 7f7012a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:40.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.959+0000 7f7012a00700 1 -- 172.21.15.63:0/275857079 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702401f040 con 0x55f5e2b71e70 2024-05-27T07:45:40.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.959+0000 7f7012a00700 1 -- 172.21.15.63:0/275857079 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f7024007a00 con 0x55f5e2b71e70 2024-05-27T07:45:40.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.959+0000 7f7041f0bb80 1 -- 172.21.15.63:0/275857079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70200aa630 con 0x55f5e2b71e70 2024-05-27T07:45:40.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.959+0000 7f7012a00700 1 -- 172.21.15.63:0/275857079 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7024031050 con 0x55f5e2b71e70 2024-05-27T07:45:40.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.959+0000 7f7041f0bb80 10 client.17730 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:40.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.959+0000 7f7041f0bb80 1 -- 172.21.15.63:0/275857079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f5e2a60060 con 0x55f5e2b71e70 2024-05-27T07:45:40.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.960+0000 7f7012a00700 1 -- 172.21.15.63:0/275857079 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70200aa630 con 0x55f5e2b71e70 2024-05-27T07:45:40.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.960+0000 7f7012a00700 1 -- 172.21.15.63:0/275857079 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7024016040 con 0x55f5e2b71e70 2024-05-27T07:45:40.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.960+0000 7f7041f0bb80 10 client.17730 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:40.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.960+0000 7f7041f0bb80 10 client.17730 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:45:40.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.960+0000 7f7041f0bb80 20 client.17730 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:40.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.960+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f5e2b8b860 0x55f5e2b8dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.960+0000 7f7041f0bb80 4 client.17730 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:40.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.960+0000 7f7041f0bb80 1 -- 172.21.15.63:0/275857079 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5e2aa1d90 con 0x55f5e2b8b860 2024-05-27T07:45:40.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.961+0000 7f7030a00700 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f5e2b8b860 0x55f5e2b8dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.962+0000 7f7030a00700 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f5e2b8b860 0x55f5e2b8dce0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=660b56894c90cf84 server_cookie=aa01934172d1d102 in_seq=0 out_seq=0 2024-05-27T07:45:40.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.962+0000 7f7012a00700 10 client.17730 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:45:40.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7012a00700 1 -- 172.21.15.63:0/275857079 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f5e2aa1d90 con 0x55f5e2b8b860 2024-05-27T07:45:40.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7012a00700 10 client.17730 handle_command_reply: tid=0 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7041f0bb80 1 client.17730 shutdown 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7041f0bb80 1 -- 172.21.15.63:0/275857079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 msgr2=0x55f5e2b7c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7c0c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f70240079d0 tx=0x7f702401a000 comp rx=0 tx=0).stop 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7041f0bb80 1 -- 172.21.15.63:0/275857079 shutdown_connections 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f5e2b8b860 0x55f5e2b8dce0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2b71e70 0x55f5e2b7c0c0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b7b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/275857079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b7bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7041f0bb80 1 -- 172.21.15.63:0/275857079 >> 172.21.15.63:0/275857079 conn(0x55f5e2a8c390 msgr2=0x55f5e2b71370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.964+0000 7f7012a00700 0 client.17730 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:45:40.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.965+0000 7f7041f0bb80 1 -- 172.21.15.63:0/275857079 shutdown_connections 2024-05-27T07:45:40.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.965+0000 7f7041f0bb80 1 -- 172.21.15.63:0/275857079 wait complete. 2024-05-27T07:45:40.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.965+0000 7f7041f0bb80 20 client.17730 trim_cache size 0 max 16384 2024-05-27T07:45:40.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.986+0000 7f7041f0bb80 1 Processor -- start 2024-05-27T07:45:40.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.986+0000 7f7041f0bb80 1 -- start start 2024-05-27T07:45:40.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.986+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e2a8c390 0x55f5e2b71790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.986+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e29f2e40 0x55f5e2b879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.986+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b7a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.986+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1b91a0 con 0x55f5e29f2e40 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.986+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1b6a30 con 0x55f5e2ade9f0 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.986+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1b48a0 con 0x55f5e2a8c390 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.987+0000 7f702be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e29f2e40 0x55f5e2b879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.987+0000 7f702be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e29f2e40 0x55f5e2b879a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60194/0 (socket says 172.21.15.63:60194) 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.987+0000 7f702be00700 1 -- 172.21.15.63:0/1077011400 learned_addr learned my addr 172.21.15.63:0/1077011400 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.987+0000 7f7031400700 1 --2- 172.21.15.63:0/1077011400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b7a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.987+0000 7f7030a00700 1 --2- 172.21.15.63:0/1077011400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e2a8c390 0x55f5e2b71790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.987+0000 7f702be00700 1 -- 172.21.15.63:0/1077011400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e2a8c390 msgr2=0x55f5e2b71790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.987+0000 7f702be00700 1 --2- 172.21.15.63:0/1077011400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e2a8c390 0x55f5e2b71790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.987+0000 7f702be00700 1 -- 172.21.15.63:0/1077011400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 msgr2=0x55f5e2b7a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:40.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.987+0000 7f702be00700 1 --2- 172.21.15.63:0/1077011400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b7a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.987+0000 7f702be00700 1 -- 172.21.15.63:0/1077011400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5e2b69240 con 0x55f5e29f2e40 2024-05-27T07:45:40.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.988+0000 7f702be00700 1 --2- 172.21.15.63:0/1077011400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e29f2e40 0x55f5e2b879a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f701800bcc0 tx=0x7f7018011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58411f5a55dfc0c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:40.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.988+0000 7f701d600700 1 -- 172.21.15.63:0/1077011400 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018011a40 con 0x55f5e29f2e40 2024-05-27T07:45:40.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.988+0000 7f701d600700 1 -- 172.21.15.63:0/1077011400 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7018011be0 con 0x55f5e29f2e40 2024-05-27T07:45:40.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.988+0000 7f701d600700 1 -- 172.21.15.63:0/1077011400 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701801a3c0 con 0x55f5e29f2e40 2024-05-27T07:45:40.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.989+0000 7f7041f0bb80 1 -- 172.21.15.63:0/1077011400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e29f2e40 msgr2=0x55f5e2b879a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:40.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.989+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/1077011400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e29f2e40 0x55f5e2b879a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f701800bcc0 tx=0x7f7018011440 comp rx=0 tx=0).stop 2024-05-27T07:45:40.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.989+0000 7f7041f0bb80 1 -- 172.21.15.63:0/1077011400 shutdown_connections 2024-05-27T07:45:40.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.989+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/1077011400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2ade9f0 0x55f5e2b7a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.989+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/1077011400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e2a8c390 0x55f5e2b71790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.989+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/1077011400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e29f2e40 0x55f5e2b879a0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.989+0000 7f7041f0bb80 1 -- 172.21.15.63:0/1077011400 >> 172.21.15.63:0/1077011400 conn(0x55f5e2b8e030 msgr2=0x55f5e2b79f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:40.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.989+0000 7f7041f0bb80 1 -- 172.21.15.63:0/1077011400 shutdown_connections 2024-05-27T07:45:40.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.990+0000 7f7041f0bb80 1 -- 172.21.15.63:0/1077011400 wait complete. 2024-05-27T07:45:40.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.990+0000 7f7041f0bb80 1 -- start start 2024-05-27T07:45:40.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.990+0000 7f7041f0bb80 1 Processor -- start 2024-05-27T07:45:40.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.991+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b7ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.991+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2a8c390 0x55f5e2b7f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.991+0000 7f7041f0bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2ade9f0 0x55f5e2bf4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.991+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1b4ef0 con 0x55f5e2ade9f0 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.991+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1b6360 con 0x55f5e2a8c390 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.991+0000 7f7041f0bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1b55c0 con 0x55f5e29f2e40 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.991+0000 7f7041f0bb80 10 client.0 _ll_register_callbacks cb 0x55f5e2a39c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.991+0000 7f7031400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2ade9f0 0x55f5e2bf4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.991+0000 7f7031400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2ade9f0 0x55f5e2bf4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60196/0 (socket says 172.21.15.63:60196) 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.991+0000 7f7031400700 1 -- 172.21.15.63:0/2944397393 learned_addr learned my addr 172.21.15.63:0/2944397393 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.992+0000 7f7030a00700 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b7ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.992+0000 7f702be00700 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2a8c390 0x55f5e2b7f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.992+0000 7f7031400700 1 -- 172.21.15.63:0/2944397393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 msgr2=0x55f5e2b7ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:40.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.992+0000 7f7031400700 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b7ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.992+0000 7f7031400700 1 -- 172.21.15.63:0/2944397393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2a8c390 msgr2=0x55f5e2b7f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:40.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.992+0000 7f7031400700 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2a8c390 0x55f5e2b7f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.992+0000 7f7031400700 1 -- 172.21.15.63:0/2944397393 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5e2b7fa20 con 0x55f5e2ade9f0 2024-05-27T07:45:40.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.992+0000 7f702be00700 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2a8c390 0x55f5e2b7f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:40.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.992+0000 7f7031400700 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2ade9f0 0x55f5e2bf4360 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7024023de0 tx=0x7f7024018370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d2435a053f4399 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:40.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.992+0000 7f7012a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:40.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.993+0000 7f7012a00700 1 -- 172.21.15.63:0/2944397393 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024039020 con 0x55f5e2ade9f0 2024-05-27T07:45:40.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.993+0000 7f7041f0bb80 1 -- 172.21.15.63:0/2944397393 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7018011be0 con 0x55f5e2ade9f0 2024-05-27T07:45:40.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.993+0000 7f7012a00700 1 -- 172.21.15.63:0/2944397393 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f7024019cc0 con 0x55f5e2ade9f0 2024-05-27T07:45:40.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.993+0000 7f7012a00700 1 -- 172.21.15.63:0/2944397393 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7024041050 con 0x55f5e2ade9f0 2024-05-27T07:45:40.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.993+0000 7f7041f0bb80 10 client.17736 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:40.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.993+0000 7f7041f0bb80 1 -- 172.21.15.63:0/2944397393 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f5e2bf58c0 con 0x55f5e2ade9f0 2024-05-27T07:45:40.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.993+0000 7f7012a00700 1 -- 172.21.15.63:0/2944397393 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7018011be0 con 0x55f5e2ade9f0 2024-05-27T07:45:40.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.994+0000 7f7012a00700 1 -- 172.21.15.63:0/2944397393 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7024016050 con 0x55f5e2ade9f0 2024-05-27T07:45:40.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.994+0000 7f7041f0bb80 10 client.17736 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:40.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.994+0000 7f7041f0bb80 10 client.17736 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:45:40.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.994+0000 7f7041f0bb80 20 client.17736 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:40.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.994+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f5e2b71e70 0x55f5e2bf7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:40.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.994+0000 7f7041f0bb80 4 client.17736 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:45:40.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.994+0000 7f7041f0bb80 1 -- 172.21.15.63:0/2944397393 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5e2a9b570 con 0x55f5e2b71e70 2024-05-27T07:45:40.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.995+0000 7f7030a00700 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f5e2b71e70 0x55f5e2bf7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:40.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.995+0000 7f7030a00700 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f5e2b71e70 0x55f5e2bf7fd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=189585299e5d5703 server_cookie=c16c2a2b42db8852 in_seq=0 out_seq=0 2024-05-27T07:45:40.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.995+0000 7f7012a00700 10 client.17736 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:45:40.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7012a00700 1 -- 172.21.15.63:0/2944397393 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5e2a9b570 con 0x55f5e2b71e70 2024-05-27T07:45:40.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7012a00700 10 client.17736 handle_command_reply: tid=0 2024-05-27T07:45:40.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7041f0bb80 1 client.17736 shutdown 2024-05-27T07:45:40.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7041f0bb80 1 -- 172.21.15.63:0/2944397393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2ade9f0 msgr2=0x55f5e2bf4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:40.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2ade9f0 0x55f5e2bf4360 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7024023de0 tx=0x7f7024018370 comp rx=0 tx=0).stop 2024-05-27T07:45:40.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7041f0bb80 1 -- 172.21.15.63:0/2944397393 shutdown_connections 2024-05-27T07:45:40.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f5e2b71e70 0x55f5e2bf7fd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5e2ade9f0 0x55f5e2bf4360 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5e29f2e40 0x55f5e2b7ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7041f0bb80 1 --2- 172.21.15.63:0/2944397393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5e2a8c390 0x55f5e2b7f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:40.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7041f0bb80 1 -- 172.21.15.63:0/2944397393 >> 172.21.15.63:0/2944397393 conn(0x55f5e2b8e030 msgr2=0x55f5e2b7be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:40.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.996+0000 7f7012a00700 0 client.17736 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:45:40.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.997+0000 7f7041f0bb80 1 -- 172.21.15.63:0/2944397393 shutdown_connections 2024-05-27T07:45:40.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.997+0000 7f7041f0bb80 1 -- 172.21.15.63:0/2944397393 wait complete. 2024-05-27T07:45:40.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.997+0000 7f7041f0bb80 20 client.17736 trim_cache size 0 max 16384 2024-05-27T07:45:40.999 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:45:41.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 -- 172.21.15.63:0/4019518439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70140b2a70 msgr2=0x7f70140b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:41.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70140b2a70 0x7f70140b4f80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7024007fd0 tx=0x7f70240132f0 comp rx=0 tx=0).stop 2024-05-27T07:45:41.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 -- 172.21.15.63:0/4019518439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c075b50 msgr2=0x7f702c1a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:41.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c075b50 0x7f702c1a12f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7020000c00 tx=0x7f70200036f0 comp rx=0 tx=0).stop 2024-05-27T07:45:41.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 -- 172.21.15.63:0/4019518439 shutdown_connections 2024-05-27T07:45:41.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70140b2a70 0x7f70140b4f80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:41.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c115360 0x7f702c1a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:41.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f702c075b50 0x7f702c1a12f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:41.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 --2- 172.21.15.63:0/4019518439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f702c0764a0 0x7f702c1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:41.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 -- 172.21.15.63:0/4019518439 >> 172.21.15.63:0/4019518439 conn(0x7f702c100030 msgr2=0x7f702c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:41.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 -- 172.21.15.63:0/4019518439 shutdown_connections 2024-05-27T07:45:41.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:40.998+0000 7f7033400700 1 -- 172.21.15.63:0/4019518439 wait complete. 2024-05-27T07:45:41.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:40 smithi086 ceph-mon[34687]: pgmap v378: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:45:41.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:40 smithi149 ceph-mon[35408]: pgmap v378: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:45:41.978 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:45:42.497 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:42.916 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:42 smithi063 ceph-mon[31613]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:45:43.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:42 smithi086 ceph-mon[34687]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:45:43.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:42 smithi149 ceph-mon[35408]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:45:44.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:44 smithi063 ceph-mon[31613]: pgmap v380: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:45:45.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:44 smithi086 ceph-mon[34687]: pgmap v380: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:45:45.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:44 smithi149 ceph-mon[35408]: pgmap v380: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:45:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.611+0000 7efd32e00640 1 -- 172.21.15.63:0/4179395483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c10d9c0 msgr2=0x7efd2c10fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.611+0000 7efd32e00640 1 --2- 172.21.15.63:0/4179395483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c10d9c0 0x7efd2c10fde0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7efd18009900 tx=0x7efd1802a480 comp rx=0 tx=0).stop 2024-05-27T07:45:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.612+0000 7efd32e00640 1 -- 172.21.15.63:0/4179395483 shutdown_connections 2024-05-27T07:45:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.612+0000 7efd32e00640 1 --2- 172.21.15.63:0/4179395483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd2c110320 0x7efd2c112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.612+0000 7efd32e00640 1 --2- 172.21.15.63:0/4179395483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c10d9c0 0x7efd2c10fde0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.612+0000 7efd32e00640 1 --2- 172.21.15.63:0/4179395483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd2c075420 0x7efd2c075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.612+0000 7efd32e00640 1 -- 172.21.15.63:0/4179395483 >> 172.21.15.63:0/4179395483 conn(0x7efd2c0ff330 msgr2=0x7efd2c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.612+0000 7efd32e00640 1 -- 172.21.15.63:0/4179395483 shutdown_connections 2024-05-27T07:45:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.612+0000 7efd32e00640 1 -- 172.21.15.63:0/4179395483 wait complete. 2024-05-27T07:45:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.613+0000 7efd32e00640 1 Processor -- start 2024-05-27T07:45:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.613+0000 7efd32e00640 1 -- start start 2024-05-27T07:45:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd32e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd2c075420 0x7efd2c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd32e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c10d9c0 0x7efd2c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd32e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd2c110320 0x7efd2c1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd32e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c115d40 con 0x7efd2c10d9c0 2024-05-27T07:45:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd32e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c115bc0 con 0x7efd2c110320 2024-05-27T07:45:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd32e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c115ec0 con 0x7efd2c075420 2024-05-27T07:45:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd31000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c10d9c0 0x7efd2c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd31000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c10d9c0 0x7efd2c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60216/0 (socket says 172.21.15.63:60216) 2024-05-27T07:45:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd31000640 1 -- 172.21.15.63:0/527992258 learned_addr learned my addr 172.21.15.63:0/527992258 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd31a00640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd2c075420 0x7efd2c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.614+0000 7efd32400640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd2c110320 0x7efd2c1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.615+0000 7efd31000640 1 -- 172.21.15.63:0/527992258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd2c075420 msgr2=0x7efd2c19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.615+0000 7efd31000640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd2c075420 0x7efd2c19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.615+0000 7efd31000640 1 -- 172.21.15.63:0/527992258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd2c110320 msgr2=0x7efd2c1a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.615+0000 7efd31000640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd2c110320 0x7efd2c1a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.615+0000 7efd31000640 1 -- 172.21.15.63:0/527992258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c1a15e0 con 0x7efd2c10d9c0 2024-05-27T07:45:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.615+0000 7efd31000640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c10d9c0 0x7efd2c19cd60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7efd18009230 tx=0x7efd180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.616+0000 7efd2a000640 1 -- 172.21.15.63:0/527992258 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1803f030 con 0x7efd2c10d9c0 2024-05-27T07:45:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.616+0000 7efd2a000640 1 -- 172.21.15.63:0/527992258 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd18003c90 con 0x7efd2c10d9c0 2024-05-27T07:45:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.616+0000 7efd32e00640 1 -- 172.21.15.63:0/527992258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2c1a1870 con 0x7efd2c10d9c0 2024-05-27T07:45:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.616+0000 7efd2a000640 1 -- 172.21.15.63:0/527992258 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18035070 con 0x7efd2c10d9c0 2024-05-27T07:45:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.616+0000 7efd32e00640 1 -- 172.21.15.63:0/527992258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2c1a1d60 con 0x7efd2c10d9c0 2024-05-27T07:45:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.620+0000 7efd3b443740 1 Processor -- start 2024-05-27T07:45:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.620+0000 7efd3b443740 1 -- start start 2024-05-27T07:45:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.620+0000 7efd3b443740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffa5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.620+0000 7efd3b443740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 0x556a3ffa5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.620+0000 7efd3b443740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffa64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.620+0000 7efd3b443740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556a3ff16b90 con 0x556a3fc031b0 2024-05-27T07:45:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.620+0000 7efd3b443740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a3ff06730 con 0x556a3fbda2f0 2024-05-27T07:45:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.620+0000 7efd3b443740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556a3ff0a770 con 0x556a3fbe24d0 2024-05-27T07:45:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd32400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffa5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd32400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffa5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60222/0 (socket says 172.21.15.63:60222) 2024-05-27T07:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd32400640 1 -- 172.21.15.63:0/1425448496 learned_addr learned my addr 172.21.15.63:0/1425448496 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd31a00640 1 --2- 172.21.15.63:0/1425448496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 0x556a3ffa5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd2a000640 1 -- 172.21.15.63:0/527992258 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efd18042030 con 0x7efd2c10d9c0 2024-05-27T07:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd32400640 1 -- 172.21.15.63:0/1425448496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 msgr2=0x556a3ffa64e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd32400640 1 --2- 172.21.15.63:0/1425448496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffa64e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd32400640 1 -- 172.21.15.63:0/1425448496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 msgr2=0x556a3ffa5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd32400640 1 --2- 172.21.15.63:0/1425448496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 0x556a3ffa5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd32400640 1 -- 172.21.15.63:0/1425448496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a3fe9f8b0 con 0x556a3fc031b0 2024-05-27T07:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.621+0000 7efd31a00640 1 --2- 172.21.15.63:0/1425448496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 0x556a3ffa5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.622+0000 7efd32400640 1 --2- 172.21.15.63:0/1425448496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffa5a60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7efd2400cb40 tx=0x7efd2400e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55005f6d42f1fbf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.622+0000 7efd1ea00640 1 -- 172.21.15.63:0/1425448496 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24005020 con 0x556a3fc031b0 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.622+0000 7efd1ea00640 1 -- 172.21.15.63:0/1425448496 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd240047a0 con 0x556a3fc031b0 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.622+0000 7efd1ea00640 1 -- 172.21.15.63:0/1425448496 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24010690 con 0x556a3fc031b0 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.623+0000 7efd3b443740 1 -- 172.21.15.63:0/1425448496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 msgr2=0x556a3ffa5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.623+0000 7efd3b443740 1 --2- 172.21.15.63:0/1425448496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffa5a60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7efd2400cb40 tx=0x7efd2400e350 comp rx=0 tx=0).stop 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.623+0000 7efd3b443740 1 -- 172.21.15.63:0/1425448496 shutdown_connections 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.623+0000 7efd3b443740 1 --2- 172.21.15.63:0/1425448496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffa64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.623+0000 7efd3b443740 1 --2- 172.21.15.63:0/1425448496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 0x556a3ffa5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.623+0000 7efd3b443740 1 --2- 172.21.15.63:0/1425448496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffa5a60 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.623+0000 7efd3b443740 1 -- 172.21.15.63:0/1425448496 >> 172.21.15.63:0/1425448496 conn(0x556a3fbf7300 msgr2=0x556a3ffa2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.623+0000 7efd2a000640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd140b2950 0x7efd140b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.623+0000 7efd3b443740 1 -- 172.21.15.63:0/1425448496 shutdown_connections 2024-05-27T07:45:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.624+0000 7efd3b443740 1 -- 172.21.15.63:0/1425448496 wait complete. 2024-05-27T07:45:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.624+0000 7efd2a000640 1 -- 172.21.15.63:0/527992258 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7efd18103070 con 0x7efd2c10d9c0 2024-05-27T07:45:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.624+0000 7efd31a00640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd140b2950 0x7efd140b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.625+0000 7efd3b443740 1 -- start start 2024-05-27T07:45:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.625+0000 7efd31a00640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd140b2950 0x7efd140b4e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x556a3ffa6e10 tx=0x7efd200077b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.625+0000 7efd3b443740 1 Processor -- start 2024-05-27T07:45:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd3b443740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbda2f0 0x556a3ffafc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd3b443740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbe24d0 0x556a3ffb0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd3b443740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffbbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd3b443740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556a3fbec790 con 0x556a3fc031b0 2024-05-27T07:45:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd3b443740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a3feaebc0 con 0x556a3fbe24d0 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd3b443740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556a3ffb2b90 con 0x556a3fbda2f0 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd3b443740 10 client.0 _ll_register_callbacks cb 0x556a3fe8c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd32400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffbbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd32400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffbbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60224/0 (socket says 172.21.15.63:60224) 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd32400640 1 -- 172.21.15.63:0/1628706797 learned_addr learned my addr 172.21.15.63:0/1628706797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd31000640 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbe24d0 0x556a3ffb0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.626+0000 7efd31a00640 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbda2f0 0x556a3ffafc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.627+0000 7efd32400640 1 -- 172.21.15.63:0/1628706797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbda2f0 msgr2=0x556a3ffafc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.627+0000 7efd32400640 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbda2f0 0x556a3ffafc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.627+0000 7efd32400640 1 -- 172.21.15.63:0/1628706797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbe24d0 msgr2=0x556a3ffb0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.627+0000 7efd32400640 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbe24d0 0x556a3ffb0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.627+0000 7efd32400640 1 -- 172.21.15.63:0/1628706797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a3fc1b210 con 0x556a3fc031b0 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.627+0000 7efd32400640 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffbbf20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7efd24000c00 tx=0x7efd2400efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72b75868fb78ba3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.627+0000 7efd13400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.628+0000 7efd13400640 1 -- 172.21.15.63:0/1628706797 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24004240 con 0x556a3fc031b0 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.628+0000 7efd3b443740 1 -- 172.21.15.63:0/1628706797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd240047a0 con 0x556a3fc031b0 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.628+0000 7efd13400640 1 -- 172.21.15.63:0/1628706797 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7efd240043e0 con 0x556a3fc031b0 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.628+0000 7efd13400640 1 -- 172.21.15.63:0/1628706797 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efd24005000 con 0x556a3fc031b0 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.628+0000 7efd3b443740 10 client.17748 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.628+0000 7efd3b443740 1 -- 172.21.15.63:0/1628706797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556a3fce7620 con 0x556a3fc031b0 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.628+0000 7efd13400640 1 -- 172.21.15.63:0/1628706797 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd240047a0 con 0x556a3fc031b0 2024-05-27T07:45:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.629+0000 7efd13400640 1 -- 172.21.15.63:0/1628706797 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7efd24003b90 con 0x556a3fc031b0 2024-05-27T07:45:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.629+0000 7efd3b443740 10 client.17748 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.629+0000 7efd3b443740 10 client.17748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.629+0000 7efd3b443740 20 client.17748 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.629+0000 7efd3b443740 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556a3fbc1d80 0x556a3ffc1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.629+0000 7efd3b443740 4 client.17748 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.629+0000 7efd3b443740 1 -- 172.21.15.63:0/1628706797 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a3ff18ae0 con 0x556a3fbc1d80 2024-05-27T07:45:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.630+0000 7efd31a00640 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556a3fbc1d80 0x556a3ffc1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.631+0000 7efd13400640 10 client.17748 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.631+0000 7efd31a00640 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556a3fbc1d80 0x556a3ffc1e00 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa4a6f32709aea8 server_cookie=a57d918397c51174 in_seq=0 out_seq=0 2024-05-27T07:45:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.633+0000 7efd13400640 1 -- 172.21.15.63:0/1628706797 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556a3ff18ae0 con 0x556a3fbc1d80 2024-05-27T07:45:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.633+0000 7efd13400640 10 client.17748 handle_command_reply: tid=0 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.633+0000 7efd3b443740 1 client.17748 shutdown 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.633+0000 7efd3b443740 1 -- 172.21.15.63:0/1628706797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 msgr2=0x556a3ffbbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.633+0000 7efd3b443740 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffbbf20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7efd24000c00 tx=0x7efd2400efc0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.633+0000 7efd3b443740 1 -- 172.21.15.63:0/1628706797 shutdown_connections 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.633+0000 7efd3b443740 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556a3fbc1d80 0x556a3ffc1e00 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.634+0000 7efd3b443740 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffbbf20 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.634+0000 7efd3b443740 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbe24d0 0x556a3ffb0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.634+0000 7efd3b443740 1 --2- 172.21.15.63:0/1628706797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbda2f0 0x556a3ffafc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.634+0000 7efd13400640 0 client.17748 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.634+0000 7efd3b443740 1 -- 172.21.15.63:0/1628706797 >> 172.21.15.63:0/1628706797 conn(0x556a3fbf7300 msgr2=0x556a3ffa5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.634+0000 7efd3b443740 1 -- 172.21.15.63:0/1628706797 shutdown_connections 2024-05-27T07:45:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.634+0000 7efd3b443740 1 -- 172.21.15.63:0/1628706797 wait complete. 2024-05-27T07:45:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.634+0000 7efd3b443740 20 client.17748 trim_cache size 0 max 16384 2024-05-27T07:45:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.653+0000 7efd3b443740 1 Processor -- start 2024-05-27T07:45:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.653+0000 7efd3b443740 1 -- start start 2024-05-27T07:45:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd3b443740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 0x556a3ffbbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd3b443740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffbc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd3b443740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffc74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd3b443740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c1b5ab0 con 0x556a3fc031b0 2024-05-27T07:45:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd3b443740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c1b3380 con 0x556a3fbda2f0 2024-05-27T07:45:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd3b443740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c1b11f0 con 0x556a3fbe24d0 2024-05-27T07:45:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd32400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffc74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd32400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffc74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60226/0 (socket says 172.21.15.63:60226) 2024-05-27T07:45:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd32400640 1 -- 172.21.15.63:0/3144861666 learned_addr learned my addr 172.21.15.63:0/3144861666 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd31000640 1 --2- 172.21.15.63:0/3144861666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffbc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.654+0000 7efd31a00640 1 --2- 172.21.15.63:0/3144861666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 0x556a3ffbbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.655+0000 7efd32400640 1 -- 172.21.15.63:0/3144861666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 msgr2=0x556a3ffbc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.655+0000 7efd32400640 1 --2- 172.21.15.63:0/3144861666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffbc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.655+0000 7efd32400640 1 -- 172.21.15.63:0/3144861666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 msgr2=0x556a3ffbbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.655+0000 7efd32400640 1 --2- 172.21.15.63:0/3144861666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 0x556a3ffbbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.655+0000 7efd32400640 1 -- 172.21.15.63:0/3144861666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a3ffa2bd0 con 0x556a3fc031b0 2024-05-27T07:45:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.655+0000 7efd32400640 1 --2- 172.21.15.63:0/3144861666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffc74b0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7efd24001eb0 tx=0x7efd2400e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff2f03a8d2a11ab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.655+0000 7efd1e000640 1 -- 172.21.15.63:0/3144861666 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24038070 con 0x556a3fc031b0 2024-05-27T07:45:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.655+0000 7efd1e000640 1 -- 172.21.15.63:0/3144861666 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd2400c6a0 con 0x556a3fc031b0 2024-05-27T07:45:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.655+0000 7efd1e000640 1 -- 172.21.15.63:0/3144861666 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd240056a0 con 0x556a3fc031b0 2024-05-27T07:45:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.656+0000 7efd3b443740 1 -- 172.21.15.63:0/3144861666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 msgr2=0x556a3ffc74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:45.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.656+0000 7efd3b443740 1 --2- 172.21.15.63:0/3144861666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffc74b0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7efd24001eb0 tx=0x7efd2400e350 comp rx=0 tx=0).stop 2024-05-27T07:45:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.656+0000 7efd3b443740 1 -- 172.21.15.63:0/3144861666 shutdown_connections 2024-05-27T07:45:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.656+0000 7efd3b443740 1 --2- 172.21.15.63:0/3144861666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fc031b0 0x556a3ffc74b0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.656+0000 7efd3b443740 1 --2- 172.21.15.63:0/3144861666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffbc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.656+0000 7efd3b443740 1 --2- 172.21.15.63:0/3144861666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fbda2f0 0x556a3ffbbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.656+0000 7efd3b443740 1 -- 172.21.15.63:0/3144861666 >> 172.21.15.63:0/3144861666 conn(0x556a3fbf7300 msgr2=0x556a3ffaff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.657+0000 7efd3b443740 1 -- 172.21.15.63:0/3144861666 shutdown_connections 2024-05-27T07:45:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.657+0000 7efd3b443740 1 -- 172.21.15.63:0/3144861666 wait complete. 2024-05-27T07:45:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.657+0000 7efd3b443740 1 -- start start 2024-05-27T07:45:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.657+0000 7efd3b443740 1 Processor -- start 2024-05-27T07:45:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd3b443740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fbda2f0 0x556a3ffb4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd3b443740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffb5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd3b443740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fc031b0 0x556a3ffb5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd3b443740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c1b1840 con 0x556a3fbda2f0 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd3b443740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c1b2cb0 con 0x556a3fc031b0 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd3b443740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c1b1f10 con 0x556a3fbe24d0 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd3b443740 10 client.0 _ll_register_callbacks cb 0x7efd24014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd31a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fbda2f0 0x556a3ffb4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd31a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fbda2f0 0x556a3ffb4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60238/0 (socket says 172.21.15.63:60238) 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd31a00640 1 -- 172.21.15.63:0/814993032 learned_addr learned my addr 172.21.15.63:0/814993032 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.658+0000 7efd31000640 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffb5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd32400640 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fc031b0 0x556a3ffb5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd31a00640 1 -- 172.21.15.63:0/814993032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 msgr2=0x556a3ffb5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd31a00640 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffb5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd31a00640 1 -- 172.21.15.63:0/814993032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fc031b0 msgr2=0x556a3ffb5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd31a00640 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fc031b0 0x556a3ffb5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd31a00640 1 -- 172.21.15.63:0/814993032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a40028430 con 0x556a3fbda2f0 2024-05-27T07:45:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd31a00640 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fbda2f0 0x556a3ffb4d10 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7efd200076f0 tx=0x7efd20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94947c256e75774a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd13400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd13400640 1 -- 172.21.15.63:0/814993032 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20027250 con 0x556a3fbda2f0 2024-05-27T07:45:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd3b443740 1 -- 172.21.15.63:0/814993032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd2400c6a0 con 0x556a3fbda2f0 2024-05-27T07:45:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.659+0000 7efd13400640 1 -- 172.21.15.63:0/814993032 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7efd200273f0 con 0x556a3fbda2f0 2024-05-27T07:45:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.660+0000 7efd13400640 1 -- 172.21.15.63:0/814993032 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efd20037050 con 0x556a3fbda2f0 2024-05-27T07:45:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.660+0000 7efd3b443740 10 client.17754 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.660+0000 7efd3b443740 1 -- 172.21.15.63:0/814993032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556a40029700 con 0x556a3fbda2f0 2024-05-27T07:45:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.660+0000 7efd13400640 1 -- 172.21.15.63:0/814993032 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd2400c6a0 con 0x556a3fbda2f0 2024-05-27T07:45:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.660+0000 7efd13400640 1 -- 172.21.15.63:0/814993032 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7efd20030790 con 0x556a3fbda2f0 2024-05-27T07:45:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.661+0000 7efd3b443740 10 client.17754 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.661+0000 7efd3b443740 10 client.17754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.661+0000 7efd3b443740 20 client.17754 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.661+0000 7efd3b443740 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556a3fb971c0 0x556a4002bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.661+0000 7efd3b443740 4 client.17754 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:45:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.661+0000 7efd3b443740 1 -- 172.21.15.63:0/814993032 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a3ff122c0 con 0x556a3fb971c0 2024-05-27T07:45:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.661+0000 7efd31000640 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556a3fb971c0 0x556a4002bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:45.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.662+0000 7efd31000640 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556a3fb971c0 0x556a4002bfb0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=532603c17cea7652 server_cookie=fde63feb3747d326 in_seq=0 out_seq=0 2024-05-27T07:45:45.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.662+0000 7efd13400640 10 client.17754 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd13400640 1 -- 172.21.15.63:0/814993032 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556a3ff122c0 con 0x556a3fb971c0 2024-05-27T07:45:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd13400640 10 client.17754 handle_command_reply: tid=0 2024-05-27T07:45:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd3b443740 1 client.17754 shutdown 2024-05-27T07:45:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd3b443740 1 -- 172.21.15.63:0/814993032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fbda2f0 msgr2=0x556a3ffb4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd3b443740 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fbda2f0 0x556a3ffb4d10 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7efd200076f0 tx=0x7efd20004a40 comp rx=0 tx=0).stop 2024-05-27T07:45:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd3b443740 1 -- 172.21.15.63:0/814993032 shutdown_connections 2024-05-27T07:45:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd3b443740 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556a3fb971c0 0x556a4002bfb0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd3b443740 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a3fc031b0 0x556a3ffb5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd3b443740 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556a3fbe24d0 0x556a3ffb5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd3b443740 1 --2- 172.21.15.63:0/814993032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556a3fbda2f0 0x556a3ffb4d10 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd13400640 0 client.17754 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd3b443740 1 -- 172.21.15.63:0/814993032 >> 172.21.15.63:0/814993032 conn(0x556a3fbf7300 msgr2=0x556a3ffaa250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.691+0000 7efd3b443740 1 -- 172.21.15.63:0/814993032 shutdown_connections 2024-05-27T07:45:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.692+0000 7efd3b443740 1 -- 172.21.15.63:0/814993032 wait complete. 2024-05-27T07:45:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.692+0000 7efd3b443740 20 client.17754 trim_cache size 0 max 16384 2024-05-27T07:45:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:45:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:45:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:45:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 -- 172.21.15.63:0/527992258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd140b2950 msgr2=0x7efd140b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd140b2950 0x7efd140b4e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x556a3ffa6e10 tx=0x7efd200077b0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 -- 172.21.15.63:0/527992258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c10d9c0 msgr2=0x7efd2c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c10d9c0 0x7efd2c19cd60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7efd18009230 tx=0x7efd180042f0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 -- 172.21.15.63:0/527992258 shutdown_connections 2024-05-27T07:45:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efd140b2950 0x7efd140b4e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd2c110320 0x7efd2c1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c10d9c0 0x7efd2c19cd60 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 --2- 172.21.15.63:0/527992258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd2c075420 0x7efd2c19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 -- 172.21.15.63:0/527992258 >> 172.21.15.63:0/527992258 conn(0x7efd2c0ff330 msgr2=0x7efd2c110730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 -- 172.21.15.63:0/527992258 shutdown_connections 2024-05-27T07:45:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:45.693+0000 7efd32e00640 1 -- 172.21.15.63:0/527992258 wait complete. 2024-05-27T07:45:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.228+0000 7f60ed400700 1 -- 172.21.15.63:0/1935741528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e8105e20 msgr2=0x7f60e810e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.228+0000 7f60ed400700 1 --2- 172.21.15.63:0/1935741528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e8105e20 0x7f60e810e370 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009990 tx=0x7f60d4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.228+0000 7f60ed400700 1 -- 172.21.15.63:0/1935741528 shutdown_connections 2024-05-27T07:45:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.228+0000 7f60ed400700 1 --2- 172.21.15.63:0/1935741528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e810e900 0x7f60e8110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.228+0000 7f60ed400700 1 --2- 172.21.15.63:0/1935741528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e81054d0 0x7f60e81058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.228+0000 7f60ed400700 1 --2- 172.21.15.63:0/1935741528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e8105e20 0x7f60e810e370 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.228+0000 7f60ed400700 1 -- 172.21.15.63:0/1935741528 >> 172.21.15.63:0/1935741528 conn(0x7f60e80fedb0 msgr2=0x7f60e81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.228+0000 7f60ed400700 1 -- 172.21.15.63:0/1935741528 shutdown_connections 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.228+0000 7f60ed400700 1 -- 172.21.15.63:0/1935741528 wait complete. 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.229+0000 7f60ed400700 1 Processor -- start 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.229+0000 7f60ed400700 1 -- start start 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.230+0000 7f60ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e81054d0 0x7f60e81031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.230+0000 7f60ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e8105e20 0x7f60e8103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.230+0000 7f60ed400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e810e900 0x7f60e8103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.230+0000 7f60ed400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60e8114330 con 0x7f60e81054d0 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.230+0000 7f60ed400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81141b0 con 0x7f60e8105e20 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.230+0000 7f60ed400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81144b0 con 0x7f60e810e900 2024-05-27T07:45:46.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.230+0000 7f60e6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e81054d0 0x7f60e81031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.230+0000 7f60e6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e81054d0 0x7f60e81031c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60258/0 (socket says 172.21.15.63:60258) 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.231+0000 7f60e6800700 1 -- 172.21.15.63:0/799446793 learned_addr learned my addr 172.21.15.63:0/799446793 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.231+0000 7f60e5e00700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e8105e20 0x7f60e8103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.231+0000 7f60e6800700 1 -- 172.21.15.63:0/799446793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e810e900 msgr2=0x7f60e8103c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.231+0000 7f60e7200700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e810e900 0x7f60e8103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.231+0000 7f60e6800700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e810e900 0x7f60e8103c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.231+0000 7f60e6800700 1 -- 172.21.15.63:0/799446793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e8105e20 msgr2=0x7f60e8103700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.231+0000 7f60e6800700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e8105e20 0x7f60e8103700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.231+0000 7f60e6800700 1 -- 172.21.15.63:0/799446793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e81a9140 con 0x7f60e81054d0 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.231+0000 7f60e5e00700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e8105e20 0x7f60e8103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.231+0000 7f60e7200700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e810e900 0x7f60e8103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.232+0000 7f60e6800700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e81054d0 0x7f60e81031c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f60d800d950 tx=0x7f60d800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.232+0000 7f60df400700 1 -- 172.21.15.63:0/799446793 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8016070 con 0x7f60e81054d0 2024-05-27T07:45:46.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.232+0000 7f60df400700 1 -- 172.21.15.63:0/799446793 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60d8010460 con 0x7f60e81054d0 2024-05-27T07:45:46.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.232+0000 7f60ed400700 1 -- 172.21.15.63:0/799446793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e81a9430 con 0x7f60e81054d0 2024-05-27T07:45:46.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.232+0000 7f60ed400700 1 -- 172.21.15.63:0/799446793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e81a9970 con 0x7f60e81054d0 2024-05-27T07:45:46.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.233+0000 7f60df400700 1 -- 172.21.15.63:0/799446793 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d800f620 con 0x7f60e81054d0 2024-05-27T07:45:46.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.236+0000 7f60df400700 1 -- 172.21.15.63:0/799446793 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f60d8019070 con 0x7f60e81054d0 2024-05-27T07:45:46.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.238+0000 7f60df400700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60cc0b29f0 0x7f60cc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.238+0000 7f60df400700 1 -- 172.21.15.63:0/799446793 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f60d80e29c0 con 0x7f60e81054d0 2024-05-27T07:45:46.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.238+0000 7f60fbe96b80 1 Processor -- start 2024-05-27T07:45:46.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.238+0000 7f60e5e00700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60cc0b29f0 0x7f60cc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.239+0000 7f60fbe96b80 1 -- start start 2024-05-27T07:45:46.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.239+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195119f0 0x5586195a4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.239+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195a4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.239+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195a4e70 0x5586195b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.239+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5586194d2e40 con 0x5586195a4e70 2024-05-27T07:45:46.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.239+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586194c29e0 con 0x5586195119f0 2024-05-27T07:45:46.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.239+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5586194c6a20 con 0x558619425e40 2024-05-27T07:45:46.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.239+0000 7f60e5e00700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60cc0b29f0 0x7f60cc0b4f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60d4003760 tx=0x7f60d4005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:46.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.240+0000 7f60e7200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195a4e70 0x5586195b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.240+0000 7f60e7200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195a4e70 0x5586195b1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60262/0 (socket says 172.21.15.63:60262) 2024-05-27T07:45:46.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.240+0000 7f60e7200700 1 -- 172.21.15.63:0/4155996525 learned_addr learned my addr 172.21.15.63:0/4155996525 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.240+0000 7f60e6800700 1 --2- 172.21.15.63:0/4155996525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195119f0 0x5586195a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.240+0000 7f60e5e00700 1 --2- 172.21.15.63:0/4155996525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.240+0000 7f60e7200700 1 -- 172.21.15.63:0/4155996525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 msgr2=0x5586195a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.240+0000 7f60e7200700 1 --2- 172.21.15.63:0/4155996525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.240+0000 7f60e7200700 1 -- 172.21.15.63:0/4155996525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195119f0 msgr2=0x5586195a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.240+0000 7f60e7200700 1 --2- 172.21.15.63:0/4155996525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195119f0 0x5586195a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.241+0000 7f60e7200700 1 -- 172.21.15.63:0/4155996525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55861948b420 con 0x5586195a4e70 2024-05-27T07:45:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.241+0000 7f60e7200700 1 --2- 172.21.15.63:0/4155996525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195a4e70 0x5586195b1110 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f60e0008470 tx=0x7f60e0008780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9149e884d208bf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.241+0000 7f60d3e00700 1 -- 172.21.15.63:0/4155996525 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0018070 con 0x5586195a4e70 2024-05-27T07:45:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.241+0000 7f60d3e00700 1 -- 172.21.15.63:0/4155996525 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60e00119a0 con 0x5586195a4e70 2024-05-27T07:45:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.241+0000 7f60d3e00700 1 -- 172.21.15.63:0/4155996525 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0013a40 con 0x5586195a4e70 2024-05-27T07:45:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.242+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4155996525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195a4e70 msgr2=0x5586195b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.242+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4155996525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195a4e70 0x5586195b1110 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f60e0008470 tx=0x7f60e0008780 comp rx=0 tx=0).stop 2024-05-27T07:45:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.243+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4155996525 shutdown_connections 2024-05-27T07:45:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.243+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4155996525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195a4e70 0x5586195b1110 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.243+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4155996525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195119f0 0x5586195a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.243+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4155996525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.243+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4155996525 >> 172.21.15.63:0/4155996525 conn(0x5586194bf390 msgr2=0x5586195a04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.243+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4155996525 shutdown_connections 2024-05-27T07:45:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.243+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4155996525 wait complete. 2024-05-27T07:45:46.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.244+0000 7f60fbe96b80 1 -- start start 2024-05-27T07:45:46.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.244+0000 7f60fbe96b80 1 Processor -- start 2024-05-27T07:45:46.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5586193c8fa0 con 0x5586195119f0 2024-05-27T07:45:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558619463760 con 0x5586195a4e70 2024-05-27T07:45:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5586195b1b80 con 0x558619425e40 2024-05-27T07:45:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60fbe96b80 10 client.0 _ll_register_callbacks cb 0x55861946cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60e5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60e5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60278/0 (socket says 172.21.15.63:60278) 2024-05-27T07:45:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60e5e00700 1 -- 172.21.15.63:0/4251982477 learned_addr learned my addr 172.21.15.63:0/4251982477 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60e6800700 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.245+0000 7f60e7200700 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.246+0000 7f60e5e00700 1 -- 172.21.15.63:0/4251982477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 msgr2=0x5586195ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.246+0000 7f60e5e00700 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.246+0000 7f60e5e00700 1 -- 172.21.15.63:0/4251982477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 msgr2=0x5586195af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.246+0000 7f60e5e00700 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.246+0000 7f60e5e00700 1 -- 172.21.15.63:0/4251982477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586194128d0 con 0x5586195119f0 2024-05-27T07:45:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.246+0000 7f60e5e00700 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195aeb90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009990 tx=0x7f60d4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906ac0ccdf8701f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.246+0000 7f60d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.247+0000 7f60d0c00700 1 -- 172.21.15.63:0/4251982477 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4034060 con 0x5586195119f0 2024-05-27T07:45:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.247+0000 7f60d0c00700 1 -- 172.21.15.63:0/4251982477 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f60d402f440 con 0x5586195119f0 2024-05-27T07:45:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.247+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4251982477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60e00119a0 con 0x5586195119f0 2024-05-27T07:45:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.247+0000 7f60d0c00700 1 -- 172.21.15.63:0/4251982477 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f60d403b050 con 0x5586195119f0 2024-05-27T07:45:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.247+0000 7f60fbe96b80 10 client.17766 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.247+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4251982477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558619493060 con 0x5586195119f0 2024-05-27T07:45:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.248+0000 7f60d0c00700 1 -- 172.21.15.63:0/4251982477 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60e00119a0 con 0x5586195119f0 2024-05-27T07:45:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.248+0000 7f60d0c00700 1 -- 172.21.15.63:0/4251982477 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f60d40244f0 con 0x5586195119f0 2024-05-27T07:45:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.248+0000 7f60fbe96b80 10 client.17766 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.248+0000 7f60fbe96b80 10 client.17766 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:45:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.248+0000 7f60fbe96b80 20 client.17766 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.248+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5586195beb20 0x5586195c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.248+0000 7f60fbe96b80 4 client.17766 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.248+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4251982477 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586194d4d90 con 0x5586195beb20 2024-05-27T07:45:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.249+0000 7f60e6800700 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5586195beb20 0x5586195c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.250+0000 7f60e6800700 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5586195beb20 0x5586195c0f40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb0b34b0ed372eed server_cookie=83da5ac959956be4 in_seq=0 out_seq=0 2024-05-27T07:45:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.250+0000 7f60d0c00700 10 client.17766 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:45:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60d0c00700 1 -- 172.21.15.63:0/4251982477 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5586194d4d90 con 0x5586195beb20 2024-05-27T07:45:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60d0c00700 10 client.17766 handle_command_reply: tid=0 2024-05-27T07:45:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60fbe96b80 1 client.17766 shutdown 2024-05-27T07:45:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4251982477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 msgr2=0x5586195aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195aeb90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009990 tx=0x7f60d4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4251982477 shutdown_connections 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5586195beb20 0x5586195c0f40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4251982477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195aeb90 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.252+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4251982477 >> 172.21.15.63:0/4251982477 conn(0x5586194bf390 msgr2=0x5586195a43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.253+0000 7f60d0c00700 0 client.17766 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.253+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4251982477 shutdown_connections 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.253+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4251982477 wait complete. 2024-05-27T07:45:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.253+0000 7f60fbe96b80 20 client.17766 trim_cache size 0 max 16384 2024-05-27T07:45:46.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.274+0000 7f60fbe96b80 1 Processor -- start 2024-05-27T07:45:46.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.274+0000 7f60fbe96b80 1 -- start start 2024-05-27T07:45:46.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.275+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.275+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195a4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.275+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195bac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.275+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81b50c0 con 0x5586195119f0 2024-05-27T07:45:46.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.275+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81b2950 con 0x5586195a4e70 2024-05-27T07:45:46.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.275+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81b07c0 con 0x558619425e40 2024-05-27T07:45:46.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.275+0000 7f60e5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.275+0000 7f60e5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60284/0 (socket says 172.21.15.63:60284) 2024-05-27T07:45:46.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.275+0000 7f60e5e00700 1 -- 172.21.15.63:0/2602714119 learned_addr learned my addr 172.21.15.63:0/2602714119 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:46.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.275+0000 7f60e6800700 1 --2- 172.21.15.63:0/2602714119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.276+0000 7f60e7200700 1 --2- 172.21.15.63:0/2602714119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195bac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.276+0000 7f60e5e00700 1 -- 172.21.15.63:0/2602714119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 msgr2=0x5586195a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.276+0000 7f60e5e00700 1 --2- 172.21.15.63:0/2602714119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195a9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.276+0000 7f60e5e00700 1 -- 172.21.15.63:0/2602714119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 msgr2=0x5586195bac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.276+0000 7f60e5e00700 1 --2- 172.21.15.63:0/2602714119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195bac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.276+0000 7f60e5e00700 1 -- 172.21.15.63:0/2602714119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586195a5800 con 0x5586195119f0 2024-05-27T07:45:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.276+0000 7f60e7200700 1 --2- 172.21.15.63:0/2602714119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195bac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.276+0000 7f60e5e00700 1 --2- 172.21.15.63:0/2602714119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195a4830 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f60d40098e0 tx=0x7f60d400bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532d8d97daa111db server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.277+0000 7f60d3400700 1 -- 172.21.15.63:0/2602714119 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4044070 con 0x5586195119f0 2024-05-27T07:45:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.277+0000 7f60d3400700 1 -- 172.21.15.63:0/2602714119 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60d4024d30 con 0x5586195119f0 2024-05-27T07:45:46.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.277+0000 7f60d3400700 1 -- 172.21.15.63:0/2602714119 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4034040 con 0x5586195119f0 2024-05-27T07:45:46.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.278+0000 7f60fbe96b80 1 -- 172.21.15.63:0/2602714119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 msgr2=0x5586195a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.278+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/2602714119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195a4830 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f60d40098e0 tx=0x7f60d400bfc0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.278+0000 7f60fbe96b80 1 -- 172.21.15.63:0/2602714119 shutdown_connections 2024-05-27T07:45:46.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.278+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/2602714119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195bac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.278+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/2602714119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558619425e40 0x5586195a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.278+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/2602714119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586195119f0 0x5586195a4830 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.278+0000 7f60fbe96b80 1 -- 172.21.15.63:0/2602714119 >> 172.21.15.63:0/2602714119 conn(0x5586194bf390 msgr2=0x5586195a4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:46.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.278+0000 7f60fbe96b80 1 -- 172.21.15.63:0/2602714119 shutdown_connections 2024-05-27T07:45:46.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.278+0000 7f60fbe96b80 1 -- 172.21.15.63:0/2602714119 wait complete. 2024-05-27T07:45:46.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.279+0000 7f60fbe96b80 1 -- start start 2024-05-27T07:45:46.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.279+0000 7f60fbe96b80 1 Processor -- start 2024-05-27T07:45:46.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558619425e40 0x5586195a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586195119f0 0x5586195a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60fbe96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81b0e10 con 0x558619425e40 2024-05-27T07:45:46.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81b2280 con 0x5586195a4e70 2024-05-27T07:45:46.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60fbe96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81b14e0 con 0x5586195119f0 2024-05-27T07:45:46.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60fbe96b80 10 client.0 _ll_register_callbacks cb 0x55861946cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:46.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60e6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558619425e40 0x5586195a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60e6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558619425e40 0x5586195a86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60288/0 (socket says 172.21.15.63:60288) 2024-05-27T07:45:46.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60e6800700 1 -- 172.21.15.63:0/4107070715 learned_addr learned my addr 172.21.15.63:0/4107070715 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:46.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60e7200700 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.280+0000 7f60e5e00700 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586195119f0 0x5586195a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.281+0000 7f60e6800700 1 -- 172.21.15.63:0/4107070715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586195119f0 msgr2=0x5586195a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.281+0000 7f60e6800700 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586195119f0 0x5586195a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.281+0000 7f60e6800700 1 -- 172.21.15.63:0/4107070715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 msgr2=0x5586195b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.281+0000 7f60e6800700 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.281+0000 7f60e6800700 1 -- 172.21.15.63:0/4107070715 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586195a9430 con 0x558619425e40 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.281+0000 7f60e6800700 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558619425e40 0x5586195a86c0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f60d800b2f0 tx=0x7f60d800dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43177ed26ae5ba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.281+0000 7f60d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.281+0000 7f60d0c00700 1 -- 172.21.15.63:0/4107070715 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d80c2020 con 0x558619425e40 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.281+0000 7f60d0c00700 1 -- 172.21.15.63:0/4107070715 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f60d80dff70 con 0x558619425e40 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.281+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4107070715 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60d4024d30 con 0x558619425e40 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.282+0000 7f60d0c00700 1 -- 172.21.15.63:0/4107070715 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f60d8020040 con 0x558619425e40 2024-05-27T07:45:46.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.282+0000 7f60fbe96b80 10 client.17772 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:46.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.282+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4107070715 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5586195b7360 con 0x558619425e40 2024-05-27T07:45:46.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.282+0000 7f60d0c00700 1 -- 172.21.15.63:0/4107070715 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60d4024d30 con 0x558619425e40 2024-05-27T07:45:46.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.282+0000 7f60d0c00700 1 -- 172.21.15.63:0/4107070715 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f60d8010930 con 0x558619425e40 2024-05-27T07:45:46.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.282+0000 7f60fbe96b80 10 client.17772 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:46.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.282+0000 7f60fbe96b80 10 client.17772 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:45:46.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.282+0000 7f60fbe96b80 20 client.17772 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:46.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.282+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5586196295d0 0x55861962b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.283+0000 7f60fbe96b80 4 client.17772 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:45:46.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.283+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4107070715 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586194ce570 con 0x5586196295d0 2024-05-27T07:45:46.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.283+0000 7f60e5e00700 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5586196295d0 0x55861962b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.284+0000 7f60d0c00700 10 client.17772 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:45:46.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.284+0000 7f60e5e00700 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5586196295d0 0x55861962b9f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=406659eb7d9a8795 server_cookie=74e8d5a9f0a5814e in_seq=0 out_seq=0 2024-05-27T07:45:46.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.285+0000 7f60d0c00700 1 -- 172.21.15.63:0/4107070715 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586194ce570 con 0x5586196295d0 2024-05-27T07:45:46.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.285+0000 7f60d0c00700 10 client.17772 handle_command_reply: tid=0 2024-05-27T07:45:46.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.285+0000 7f60fbe96b80 1 client.17772 shutdown 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.285+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4107070715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558619425e40 msgr2=0x5586195a86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.285+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558619425e40 0x5586195a86c0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f60d800b2f0 tx=0x7f60d800dfd0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.285+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4107070715 shutdown_connections 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.285+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586195a4e70 0x5586195b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.286+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5586196295d0 0x55861962b9f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.286+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558619425e40 0x5586195a86c0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.286+0000 7f60fbe96b80 1 --2- 172.21.15.63:0/4107070715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586195119f0 0x5586195a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.286+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4107070715 >> 172.21.15.63:0/4107070715 conn(0x5586194bf390 msgr2=0x5586195a5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.286+0000 7f60d0c00700 0 client.17772 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.286+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4107070715 shutdown_connections 2024-05-27T07:45:46.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.286+0000 7f60fbe96b80 1 -- 172.21.15.63:0/4107070715 wait complete. 2024-05-27T07:45:46.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.286+0000 7f60fbe96b80 20 client.17772 trim_cache size 0 max 16384 2024-05-27T07:45:46.286 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.287+0000 7f60ed400700 1 -- 172.21.15.63:0/799446793 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60cc0b29f0 msgr2=0x7f60cc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.287+0000 7f60ed400700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60cc0b29f0 0x7f60cc0b4f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f60d4003760 tx=0x7f60d4005fd0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.287+0000 7f60ed400700 1 -- 172.21.15.63:0/799446793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e81054d0 msgr2=0x7f60e81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.287+0000 7f60ed400700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e81054d0 0x7f60e81031c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f60d800d950 tx=0x7f60d800dd10 comp rx=0 tx=0).stop 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.287+0000 7f60ed400700 1 -- 172.21.15.63:0/799446793 shutdown_connections 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.287+0000 7f60ed400700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60cc0b29f0 0x7f60cc0b4f00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.287+0000 7f60ed400700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e810e900 0x7f60e8103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.287+0000 7f60ed400700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60e81054d0 0x7f60e81031c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.287+0000 7f60ed400700 1 --2- 172.21.15.63:0/799446793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e8105e20 0x7f60e8103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.287+0000 7f60ed400700 1 -- 172.21.15.63:0/799446793 >> 172.21.15.63:0/799446793 conn(0x7f60e80fedb0 msgr2=0x7f60e80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:46.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.288+0000 7f60ed400700 1 -- 172.21.15.63:0/799446793 shutdown_connections 2024-05-27T07:45:46.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:46.288+0000 7f60ed400700 1 -- 172.21.15.63:0/799446793 wait complete. 2024-05-27T07:45:47.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:46 smithi086 ceph-mon[34687]: pgmap v381: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:45:47.208 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:46 smithi063 ceph-mon[31613]: pgmap v381: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:45:47.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:46 smithi149 ceph-mon[35408]: pgmap v381: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:45:47.289 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:45:47.825 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:49.189 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:48 smithi063 ceph-mon[31613]: pgmap v382: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 17 op/s 2024-05-27T07:45:49.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:48 smithi086 ceph-mon[34687]: pgmap v382: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 17 op/s 2024-05-27T07:45:49.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:48 smithi149 ceph-mon[35408]: pgmap v382: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 17 op/s 2024-05-27T07:45:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.146+0000 7fa513200700 1 -- 172.21.15.63:0/1175020232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa514108d20 msgr2=0x7fa514109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.146+0000 7fa513200700 1 --2- 172.21.15.63:0/1175020232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa514108d20 0x7fa514109130 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0099c0 tx=0x7fa4fc009cd0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.147+0000 7fa513200700 1 -- 172.21.15.63:0/1175020232 shutdown_connections 2024-05-27T07:45:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.147+0000 7fa513200700 1 --2- 172.21.15.63:0/1175020232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa514109670 0x7fa514110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.147+0000 7fa513200700 1 --2- 172.21.15.63:0/1175020232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5141043f0 0x7fa514104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.147+0000 7fa513200700 1 --2- 172.21.15.63:0/1175020232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa514108d20 0x7fa514109130 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.147+0000 7fa513200700 1 -- 172.21.15.63:0/1175020232 >> 172.21.15.63:0/1175020232 conn(0x7fa514079d00 msgr2=0x7fa51407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.147+0000 7fa513200700 1 -- 172.21.15.63:0/1175020232 shutdown_connections 2024-05-27T07:45:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.147+0000 7fa513200700 1 -- 172.21.15.63:0/1175020232 wait complete. 2024-05-27T07:45:50.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.149+0000 7fa513200700 1 Processor -- start 2024-05-27T07:45:50.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.149+0000 7fa513200700 1 -- start start 2024-05-27T07:45:50.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.149+0000 7fa513200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5141043f0 0x7fa5141a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.149+0000 7fa513200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa514108d20 0x7fa5141a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa513200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa514109670 0x7fa51419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa513200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa514114180 con 0x7fa5141043f0 2024-05-27T07:45:50.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa513200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa514114000 con 0x7fa514109670 2024-05-27T07:45:50.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa513200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa514114300 con 0x7fa514108d20 2024-05-27T07:45:50.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa511e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5141043f0 0x7fa5141a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa511e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5141043f0 0x7fa5141a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60814/0 (socket says 172.21.15.63:60814) 2024-05-27T07:45:50.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa511e00700 1 -- 172.21.15.63:0/2594194783 learned_addr learned my addr 172.21.15.63:0/2594194783 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:50.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa50be00700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa514108d20 0x7fa5141a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa512800700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa514109670 0x7fa51419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa511e00700 1 -- 172.21.15.63:0/2594194783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa514108d20 msgr2=0x7fa5141a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa511e00700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa514108d20 0x7fa5141a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.150+0000 7fa511e00700 1 -- 172.21.15.63:0/2594194783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa514109670 msgr2=0x7fa51419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.151+0000 7fa511e00700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa514109670 0x7fa51419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.151+0000 7fa511e00700 1 -- 172.21.15.63:0/2594194783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51419b0d0 con 0x7fa5141043f0 2024-05-27T07:45:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.151+0000 7fa512800700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa514109670 0x7fa51419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.151+0000 7fa511e00700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5141043f0 0x7fa5141a0800 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fa50400d950 tx=0x7fa50400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.151+0000 7fa50aa00700 1 -- 172.21.15.63:0/2594194783 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504016070 con 0x7fa5141043f0 2024-05-27T07:45:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.151+0000 7fa50aa00700 1 -- 172.21.15.63:0/2594194783 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa504010460 con 0x7fa5141043f0 2024-05-27T07:45:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.151+0000 7fa513200700 1 -- 172.21.15.63:0/2594194783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51419b3c0 con 0x7fa5141043f0 2024-05-27T07:45:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.152+0000 7fa50aa00700 1 -- 172.21.15.63:0/2594194783 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50400f620 con 0x7fa5141043f0 2024-05-27T07:45:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.152+0000 7fa513200700 1 -- 172.21.15.63:0/2594194783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5141a7690 con 0x7fa5141043f0 2024-05-27T07:45:50.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.156+0000 7fa50aa00700 1 -- 172.21.15.63:0/2594194783 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa504019070 con 0x7fa5141043f0 2024-05-27T07:45:50.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa527506b80 1 Processor -- start 2024-05-27T07:45:50.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa527506b80 1 -- start start 2024-05-27T07:45:50.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa50aa00700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b29f0 0x7fa4f80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa50aa00700 1 -- 172.21.15.63:0/2594194783 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fa5040e2a20 con 0x7fa5141043f0 2024-05-27T07:45:50.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa527506b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546557750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa527506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 0x563546557c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa527506b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635465612c0 0x5635465636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa527506b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563546484e40 con 0x5635464c39f0 2024-05-27T07:45:50.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa527506b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635464749e0 con 0x5635465612c0 2024-05-27T07:45:50.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa527506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563546478a20 con 0x5635463d7e40 2024-05-27T07:45:50.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.158+0000 7fa512800700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b29f0 0x7fa4f80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.159+0000 7fa511e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546557750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.159+0000 7fa511e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546557750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60816/0 (socket says 172.21.15.63:60816) 2024-05-27T07:45:50.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.159+0000 7fa511e00700 1 -- 172.21.15.63:0/3737638850 learned_addr learned my addr 172.21.15.63:0/3737638850 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:50.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.159+0000 7fa512800700 1 --2- 172.21.15.63:0/3737638850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635465612c0 0x5635465636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.159+0000 7fa511e00700 1 -- 172.21.15.63:0/3737638850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 msgr2=0x563546557c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.159+0000 7fa50be00700 1 --2- 172.21.15.63:0/3737638850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 0x563546557c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.160+0000 7fa511e00700 1 --2- 172.21.15.63:0/3737638850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 0x563546557c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.160+0000 7fa511e00700 1 -- 172.21.15.63:0/3737638850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635465612c0 msgr2=0x5635465636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.160+0000 7fa511e00700 1 --2- 172.21.15.63:0/3737638850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635465612c0 0x5635465636e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.160+0000 7fa511e00700 1 -- 172.21.15.63:0/3737638850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354643d420 con 0x5635464c39f0 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.160+0000 7fa512800700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b29f0 0x7fa4f80b4f00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009c10 tx=0x7fa50c009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.160+0000 7fa50be00700 1 --2- 172.21.15.63:0/3737638850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 0x563546557c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.160+0000 7fa511e00700 1 --2- 172.21.15.63:0/3737638850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546557750 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa5040a5960 tx=0x7fa5040a97e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac757a52f7909315 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:50.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.160+0000 7fa503400700 1 -- 172.21.15.63:0/3737638850 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504020030 con 0x5635464c39f0 2024-05-27T07:45:50.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.160+0000 7fa503400700 1 -- 172.21.15.63:0/3737638850 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5040a98e0 con 0x5635464c39f0 2024-05-27T07:45:50.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.160+0000 7fa503400700 1 -- 172.21.15.63:0/3737638850 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5040a5b70 con 0x5635464c39f0 2024-05-27T07:45:50.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.162+0000 7fa527506b80 1 -- 172.21.15.63:0/3737638850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 msgr2=0x563546557750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.162+0000 7fa527506b80 1 --2- 172.21.15.63:0/3737638850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546557750 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa5040a5960 tx=0x7fa5040a97e0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.162+0000 7fa527506b80 1 -- 172.21.15.63:0/3737638850 shutdown_connections 2024-05-27T07:45:50.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.162+0000 7fa527506b80 1 --2- 172.21.15.63:0/3737638850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635465612c0 0x5635465636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.162+0000 7fa527506b80 1 --2- 172.21.15.63:0/3737638850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546557750 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.162+0000 7fa527506b80 1 --2- 172.21.15.63:0/3737638850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 0x563546557c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.162+0000 7fa527506b80 1 -- 172.21.15.63:0/3737638850 >> 172.21.15.63:0/3737638850 conn(0x563546471390 msgr2=0x563546552710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:50.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.162+0000 7fa527506b80 1 -- 172.21.15.63:0/3737638850 shutdown_connections 2024-05-27T07:45:50.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.162+0000 7fa527506b80 1 -- 172.21.15.63:0/3737638850 wait complete. 2024-05-27T07:45:50.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.163+0000 7fa527506b80 1 -- start start 2024-05-27T07:45:50.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.163+0000 7fa527506b80 1 Processor -- start 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa527506b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635463d7e40 0x56354655e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa527506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635464c39f0 0x56354655e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa527506b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354655edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa527506b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56354637afa0 con 0x5635465612c0 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa527506b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563546415760 con 0x5635463d7e40 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa527506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563546563d40 con 0x5635464c39f0 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa527506b80 10 client.0 _ll_register_callbacks cb 0x56354641ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa512800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354655edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa512800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354655edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60826/0 (socket says 172.21.15.63:60826) 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa512800700 1 -- 172.21.15.63:0/983721225 learned_addr learned my addr 172.21.15.63:0/983721225 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa511e00700 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635463d7e40 0x56354655e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.164+0000 7fa50be00700 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635464c39f0 0x56354655e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.165+0000 7fa512800700 1 -- 172.21.15.63:0/983721225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635464c39f0 msgr2=0x56354655e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:50.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.165+0000 7fa512800700 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635464c39f0 0x56354655e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.165+0000 7fa512800700 1 -- 172.21.15.63:0/983721225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635463d7e40 msgr2=0x56354655e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:50.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.165+0000 7fa512800700 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635463d7e40 0x56354655e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.165+0000 7fa512800700 1 -- 172.21.15.63:0/983721225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635463c48d0 con 0x5635465612c0 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.165+0000 7fa512800700 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354655edd0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa50c01db00 tx=0x7fa50c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104fe6851a0f03d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.165+0000 7fa4f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.166+0000 7fa4f7e00700 1 -- 172.21.15.63:0/983721225 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c01d2e0 con 0x5635465612c0 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.166+0000 7fa527506b80 1 -- 172.21.15.63:0/983721225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5040a98e0 con 0x5635465612c0 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.166+0000 7fa4f7e00700 1 -- 172.21.15.63:0/983721225 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fa50c016820 con 0x5635465612c0 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.166+0000 7fa4f7e00700 1 -- 172.21.15.63:0/983721225 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa50c016a20 con 0x5635465612c0 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.166+0000 7fa527506b80 10 client.17784 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.166+0000 7fa527506b80 1 -- 172.21.15.63:0/983721225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563546445060 con 0x5635465612c0 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.167+0000 7fa4f7e00700 1 -- 172.21.15.63:0/983721225 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5040a98e0 con 0x5635465612c0 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.167+0000 7fa4f7e00700 1 -- 172.21.15.63:0/983721225 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa50c01bc80 con 0x5635465612c0 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.167+0000 7fa527506b80 10 client.17784 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.167+0000 7fa527506b80 10 client.17784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.167+0000 7fa527506b80 20 client.17784 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.167+0000 7fa527506b80 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563546570d90 0x5635465731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.167+0000 7fa527506b80 4 client.17784 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.167+0000 7fa527506b80 1 -- 172.21.15.63:0/983721225 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563546486d90 con 0x563546570d90 2024-05-27T07:45:50.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.168+0000 7fa511e00700 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563546570d90 0x5635465731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.169+0000 7fa511e00700 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563546570d90 0x5635465731f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc298fc0e8c9064 server_cookie=404225486c33e752 in_seq=0 out_seq=0 2024-05-27T07:45:50.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.169+0000 7fa4f7e00700 10 client.17784 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:50.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa4f7e00700 1 -- 172.21.15.63:0/983721225 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563546486d90 con 0x563546570d90 2024-05-27T07:45:50.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa4f7e00700 10 client.17784 handle_command_reply: tid=0 2024-05-27T07:45:50.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa527506b80 1 client.17784 shutdown 2024-05-27T07:45:50.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa527506b80 1 -- 172.21.15.63:0/983721225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 msgr2=0x56354655edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa527506b80 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354655edd0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa50c01db00 tx=0x7fa50c004ad0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa527506b80 1 -- 172.21.15.63:0/983721225 shutdown_connections 2024-05-27T07:45:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa527506b80 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563546570d90 0x5635465731f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa527506b80 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354655edd0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa527506b80 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635463d7e40 0x56354655e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa4f7e00700 0 client.17784 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa527506b80 1 --2- 172.21.15.63:0/983721225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635464c39f0 0x56354655e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa527506b80 1 -- 172.21.15.63:0/983721225 >> 172.21.15.63:0/983721225 conn(0x563546471390 msgr2=0x563546557890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.171+0000 7fa527506b80 1 -- 172.21.15.63:0/983721225 shutdown_connections 2024-05-27T07:45:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.172+0000 7fa527506b80 1 -- 172.21.15.63:0/983721225 wait complete. 2024-05-27T07:45:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.172+0000 7fa527506b80 20 client.17784 trim_cache size 0 max 16384 2024-05-27T07:45:50.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.191+0000 7fa527506b80 1 Processor -- start 2024-05-27T07:45:50.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.192+0000 7fa527506b80 1 -- start start 2024-05-27T07:45:50.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.192+0000 7fa527506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 0x56354655ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.192+0000 7fa527506b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635464c39f0 0x563546552740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.192+0000 7fa527506b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354656cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.192+0000 7fa527506b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5141b0ab0 con 0x5635465612c0 2024-05-27T07:45:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.192+0000 7fa527506b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5141ae340 con 0x5635464c39f0 2024-05-27T07:45:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.192+0000 7fa527506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5141ac1b0 con 0x5635463d7e40 2024-05-27T07:45:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.192+0000 7fa512800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354656cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.193+0000 7fa512800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354656cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60834/0 (socket says 172.21.15.63:60834) 2024-05-27T07:45:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.193+0000 7fa512800700 1 -- 172.21.15.63:0/1358467766 learned_addr learned my addr 172.21.15.63:0/1358467766 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.193+0000 7fa511e00700 1 --2- 172.21.15.63:0/1358467766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 0x56354655ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.193+0000 7fa50be00700 1 --2- 172.21.15.63:0/1358467766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635464c39f0 0x563546552740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.193+0000 7fa512800700 1 -- 172.21.15.63:0/1358467766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 msgr2=0x56354655ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.193+0000 7fa512800700 1 --2- 172.21.15.63:0/1358467766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 0x56354655ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.193+0000 7fa512800700 1 -- 172.21.15.63:0/1358467766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635464c39f0 msgr2=0x563546552740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.193+0000 7fa512800700 1 --2- 172.21.15.63:0/1358467766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635464c39f0 0x563546552740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.193+0000 7fa512800700 1 -- 172.21.15.63:0/1358467766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563546558340 con 0x5635465612c0 2024-05-27T07:45:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.194+0000 7fa512800700 1 --2- 172.21.15.63:0/1358467766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354656cde0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa50c01d470 tx=0x7fa50c027420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d2159399c7e26 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:50.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.194+0000 7fa502a00700 1 -- 172.21.15.63:0/1358467766 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0167b0 con 0x5635465612c0 2024-05-27T07:45:50.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.194+0000 7fa502a00700 1 -- 172.21.15.63:0/1358467766 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa50c016bc0 con 0x5635465612c0 2024-05-27T07:45:50.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.194+0000 7fa502a00700 1 -- 172.21.15.63:0/1358467766 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c01b480 con 0x5635465612c0 2024-05-27T07:45:50.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.196+0000 7fa527506b80 1 -- 172.21.15.63:0/1358467766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 msgr2=0x56354656cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.196+0000 7fa527506b80 1 --2- 172.21.15.63:0/1358467766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354656cde0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa50c01d470 tx=0x7fa50c027420 comp rx=0 tx=0).stop 2024-05-27T07:45:50.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.196+0000 7fa527506b80 1 -- 172.21.15.63:0/1358467766 shutdown_connections 2024-05-27T07:45:50.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.196+0000 7fa527506b80 1 --2- 172.21.15.63:0/1358467766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635465612c0 0x56354656cde0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.196+0000 7fa527506b80 1 --2- 172.21.15.63:0/1358467766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635463d7e40 0x56354655ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.196+0000 7fa527506b80 1 --2- 172.21.15.63:0/1358467766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635464c39f0 0x563546552740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.196+0000 7fa527506b80 1 -- 172.21.15.63:0/1358467766 >> 172.21.15.63:0/1358467766 conn(0x563546471390 msgr2=0x563546556790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:50.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.196+0000 7fa527506b80 1 -- 172.21.15.63:0/1358467766 shutdown_connections 2024-05-27T07:45:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.196+0000 7fa527506b80 1 -- 172.21.15.63:0/1358467766 wait complete. 2024-05-27T07:45:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.196+0000 7fa527506b80 1 -- start start 2024-05-27T07:45:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.197+0000 7fa527506b80 1 Processor -- start 2024-05-27T07:45:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.197+0000 7fa527506b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635463d7e40 0x563546564590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.197+0000 7fa527506b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546564af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.197+0000 7fa527506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635465612c0 0x5635465d3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.197+0000 7fa527506b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5141ac800 con 0x5635464c39f0 2024-05-27T07:45:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.197+0000 7fa527506b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5141adc70 con 0x5635463d7e40 2024-05-27T07:45:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.197+0000 7fa527506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5141aced0 con 0x5635465612c0 2024-05-27T07:45:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.197+0000 7fa527506b80 10 client.0 _ll_register_callbacks cb 0x56354641ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa50be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546564af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa50be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546564af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60850/0 (socket says 172.21.15.63:60850) 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa50be00700 1 -- 172.21.15.63:0/3620479567 learned_addr learned my addr 172.21.15.63:0/3620479567 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa511e00700 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635463d7e40 0x563546564590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa512800700 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635465612c0 0x5635465d3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa50be00700 1 -- 172.21.15.63:0/3620479567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635465612c0 msgr2=0x5635465d3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa50be00700 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635465612c0 0x5635465d3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa50be00700 1 -- 172.21.15.63:0/3620479567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635463d7e40 msgr2=0x563546564590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa50be00700 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635463d7e40 0x563546564590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa50be00700 1 -- 172.21.15.63:0/3620479567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635465d3e00 con 0x5635464c39f0 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.198+0000 7fa512800700 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635465612c0 0x5635465d3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.199+0000 7fa50be00700 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546564af0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa51406c3b0 tx=0x7fa4fc009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386b06ac135aaee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.199+0000 7fa4f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:50.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.199+0000 7fa4f7e00700 1 -- 172.21.15.63:0/3620479567 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0095d0 con 0x5635464c39f0 2024-05-27T07:45:50.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.199+0000 7fa4f7e00700 1 -- 172.21.15.63:0/3620479567 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fa4fc003680 con 0x5635464c39f0 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.199+0000 7fa527506b80 1 -- 172.21.15.63:0/3620479567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa50c016bc0 con 0x5635464c39f0 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.200+0000 7fa4f7e00700 1 -- 172.21.15.63:0/3620479567 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa4fc031050 con 0x5635464c39f0 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.200+0000 7fa527506b80 10 client.17790 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.200+0000 7fa527506b80 1 -- 172.21.15.63:0/3620479567 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5635465db110 con 0x5635464c39f0 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.200+0000 7fa4f7e00700 1 -- 172.21.15.63:0/3620479567 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa50c016bc0 con 0x5635464c39f0 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.200+0000 7fa4f7e00700 1 -- 172.21.15.63:0/3620479567 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa4fc00f7d0 con 0x5635464c39f0 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.200+0000 7fa527506b80 10 client.17790 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.200+0000 7fa527506b80 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.200+0000 7fa527506b80 20 client.17790 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.200+0000 7fa527506b80 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635465db8a0 0x5635465ddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.201+0000 7fa527506b80 4 client.17790 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:45:50.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.201+0000 7fa527506b80 1 -- 172.21.15.63:0/3620479567 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563546480570 con 0x5635465db8a0 2024-05-27T07:45:50.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.201+0000 7fa511e00700 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635465db8a0 0x5635465ddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.202+0000 7fa511e00700 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635465db8a0 0x5635465ddcc0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5565f90ddded863e server_cookie=41932506c07c513d in_seq=0 out_seq=0 2024-05-27T07:45:50.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.202+0000 7fa4f7e00700 10 client.17790 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:50.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa4f7e00700 1 -- 172.21.15.63:0/3620479567 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563546480570 con 0x5635465db8a0 2024-05-27T07:45:50.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa4f7e00700 10 client.17790 handle_command_reply: tid=0 2024-05-27T07:45:50.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 client.17790 shutdown 2024-05-27T07:45:50.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 -- 172.21.15.63:0/3620479567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 msgr2=0x563546564af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546564af0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa51406c3b0 tx=0x7fa4fc009fb0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 -- 172.21.15.63:0/3620479567 shutdown_connections 2024-05-27T07:45:50.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635465db8a0 0x5635465ddcc0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635465612c0 0x5635465d3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635463d7e40 0x563546564590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 --2- 172.21.15.63:0/3620479567 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635464c39f0 0x563546564af0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa4f7e00700 0 client.17790 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:50.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 -- 172.21.15.63:0/3620479567 >> 172.21.15.63:0/3620479567 conn(0x563546471390 msgr2=0x56354655cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:50.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 -- 172.21.15.63:0/3620479567 shutdown_connections 2024-05-27T07:45:50.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.203+0000 7fa527506b80 1 -- 172.21.15.63:0/3620479567 wait complete. 2024-05-27T07:45:50.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.204+0000 7fa527506b80 20 client.17790 trim_cache size 0 max 16384 2024-05-27T07:45:50.204 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:45:50.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.204+0000 7fa513200700 1 -- 172.21.15.63:0/2594194783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b29f0 msgr2=0x7fa4f80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.204+0000 7fa513200700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b29f0 0x7fa4f80b4f00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009c10 tx=0x7fa50c009040 comp rx=0 tx=0).stop 2024-05-27T07:45:50.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.204+0000 7fa513200700 1 -- 172.21.15.63:0/2594194783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5141043f0 msgr2=0x7fa5141a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.204+0000 7fa513200700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5141043f0 0x7fa5141a0800 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fa50400d950 tx=0x7fa50400dd10 comp rx=0 tx=0).stop 2024-05-27T07:45:50.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.205+0000 7fa513200700 1 -- 172.21.15.63:0/2594194783 shutdown_connections 2024-05-27T07:45:50.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.205+0000 7fa513200700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b29f0 0x7fa4f80b4f00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.205+0000 7fa513200700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa514109670 0x7fa51419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.205+0000 7fa513200700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5141043f0 0x7fa5141a0800 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.205+0000 7fa513200700 1 --2- 172.21.15.63:0/2594194783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa514108d20 0x7fa5141a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.205+0000 7fa513200700 1 -- 172.21.15.63:0/2594194783 >> 172.21.15.63:0/2594194783 conn(0x7fa514079d00 msgr2=0x7fa514101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:50.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.205+0000 7fa513200700 1 -- 172.21.15.63:0/2594194783 shutdown_connections 2024-05-27T07:45:50.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:50.205+0000 7fa513200700 1 -- 172.21.15.63:0/2594194783 wait complete. 2024-05-27T07:45:50.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:50 smithi063 ceph-mon[31613]: pgmap v383: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:45:51.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:50 smithi086 ceph-mon[34687]: pgmap v383: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:45:51.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:50 smithi149 ceph-mon[35408]: pgmap v383: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:45:52.198 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:45:52.198 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:45:52.696 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:53.104 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:52 smithi063 ceph-mon[31613]: pgmap v384: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:45:53.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:52 smithi086 ceph-mon[34687]: pgmap v384: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:45:53.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:52 smithi149 ceph-mon[35408]: pgmap v384: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:45:54.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:54 smithi063 ceph-mon[31613]: pgmap v385: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:45:55.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:54 smithi086 ceph-mon[34687]: pgmap v385: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:45:55.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:54 smithi149 ceph-mon[35408]: pgmap v385: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:45:55.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.290+0000 7f618aa00700 1 -- 172.21.15.63:0/4284192409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f618410e6f0 msgr2=0x7f6184110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:55.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.290+0000 7f618aa00700 1 --2- 172.21.15.63:0/4284192409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f618410e6f0 0x7f6184110bd0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f617c00b0a0 tx=0x7f617c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.290+0000 7f618aa00700 1 -- 172.21.15.63:0/4284192409 shutdown_connections 2024-05-27T07:45:55.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.290+0000 7f618aa00700 1 --2- 172.21.15.63:0/4284192409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f618410e6f0 0x7f6184110bd0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.290+0000 7f618aa00700 1 --2- 172.21.15.63:0/4284192409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6184105280 0x7f6184105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.290+0000 7f618aa00700 1 --2- 172.21.15.63:0/4284192409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6184105bd0 0x7f618410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.291+0000 7f618aa00700 1 -- 172.21.15.63:0/4284192409 >> 172.21.15.63:0/4284192409 conn(0x7f6184078ab0 msgr2=0x7f6184078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:55.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.291+0000 7f618aa00700 1 -- 172.21.15.63:0/4284192409 shutdown_connections 2024-05-27T07:45:55.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.291+0000 7f618aa00700 1 -- 172.21.15.63:0/4284192409 wait complete. 2024-05-27T07:45:55.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.292+0000 7f618aa00700 1 Processor -- start 2024-05-27T07:45:55.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.292+0000 7f618aa00700 1 -- start start 2024-05-27T07:45:55.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.293+0000 7f618aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6184105280 0x7f6184104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.293+0000 7f618aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6184105bd0 0x7f6184102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.293+0000 7f618aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f618410e6f0 0x7f6184103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.293+0000 7f618aa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6184114100 con 0x7f618410e6f0 2024-05-27T07:45:55.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.293+0000 7f618aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6184113f80 con 0x7f6184105280 2024-05-27T07:45:55.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.293+0000 7f618aa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6184114280 con 0x7f6184105bd0 2024-05-27T07:45:55.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.293+0000 7f618a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f618410e6f0 0x7f6184103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.294+0000 7f618a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f618410e6f0 0x7f6184103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60870/0 (socket says 172.21.15.63:60870) 2024-05-27T07:45:55.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.294+0000 7f618a000700 1 -- 172.21.15.63:0/3227921317 learned_addr learned my addr 172.21.15.63:0/3227921317 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:55.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.294+0000 7f6189600700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6184105280 0x7f6184104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.294+0000 7f6188c00700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6184105bd0 0x7f6184102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.294+0000 7f618a000700 1 -- 172.21.15.63:0/3227921317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6184105bd0 msgr2=0x7f6184102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:55.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.294+0000 7f618a000700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6184105bd0 0x7f6184102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.294+0000 7f618a000700 1 -- 172.21.15.63:0/3227921317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6184105280 msgr2=0x7f6184104b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:55.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.294+0000 7f618a000700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6184105280 0x7f6184104b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.294+0000 7f618a000700 1 -- 172.21.15.63:0/3227921317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6184103d80 con 0x7f618410e6f0 2024-05-27T07:45:55.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.294+0000 7f6188c00700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6184105bd0 0x7f6184102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:55.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.295+0000 7f618a000700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f618410e6f0 0x7f6184103550 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f617c00b830 tx=0x7f617c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:55.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.295+0000 7f6182000700 1 -- 172.21.15.63:0/3227921317 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c021070 con 0x7f618410e6f0 2024-05-27T07:45:55.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.295+0000 7f6182000700 1 -- 172.21.15.63:0/3227921317 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f617c007b80 con 0x7f618410e6f0 2024-05-27T07:45:55.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.295+0000 7f618aa00700 1 -- 172.21.15.63:0/3227921317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61841ad540 con 0x7f618410e6f0 2024-05-27T07:45:55.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.295+0000 7f6182000700 1 -- 172.21.15.63:0/3227921317 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c0052a0 con 0x7f618410e6f0 2024-05-27T07:45:55.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.295+0000 7f618aa00700 1 -- 172.21.15.63:0/3227921317 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61841ad9a0 con 0x7f618410e6f0 2024-05-27T07:45:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.300+0000 7f6182000700 1 -- 172.21.15.63:0/3227921317 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f617c024070 con 0x7f618410e6f0 2024-05-27T07:45:55.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.301+0000 7f619aa5db80 1 Processor -- start 2024-05-27T07:45:55.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.301+0000 7f619aa5db80 1 -- start start 2024-05-27T07:45:55.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.302+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x56139419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.302+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x56139419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.302+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.302+0000 7f619aa5db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5613940ccec0 con 0x5613941a9340 2024-05-27T07:45:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.302+0000 7f619aa5db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5613940bca60 con 0x56139401fec0 2024-05-27T07:45:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.302+0000 7f619aa5db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613940c0aa0 con 0x56139410ba70 2024-05-27T07:45:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.303+0000 7f6182000700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f616c0b29f0 0x7f616c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.303+0000 7f6188c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.303+0000 7f6189600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x56139419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.303+0000 7f6188c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60880/0 (socket says 172.21.15.63:60880) 2024-05-27T07:45:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.303+0000 7f6188c00700 1 -- 172.21.15.63:0/4034028063 learned_addr learned my addr 172.21.15.63:0/4034028063 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.303+0000 7f6188c00700 1 --2- 172.21.15.63:0/4034028063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x56139419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.303+0000 7f6182000700 1 -- 172.21.15.63:0/3227921317 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f617c0e3cb0 con 0x7f618410e6f0 2024-05-27T07:45:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.303+0000 7f618a000700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f616c0b29f0 0x7f616c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.303+0000 7f6189600700 1 -- 172.21.15.63:0/4034028063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 msgr2=0x56139419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.304+0000 7f6189600700 1 --2- 172.21.15.63:0/4034028063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x56139419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.304+0000 7f6189600700 1 -- 172.21.15.63:0/4034028063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 msgr2=0x5613941ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.304+0000 7f6189600700 1 --2- 172.21.15.63:0/4034028063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.304+0000 7f6189600700 1 -- 172.21.15.63:0/4034028063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613940854a0 con 0x56139401fec0 2024-05-27T07:45:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.304+0000 7f6188c00700 1 --2- 172.21.15.63:0/4034028063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:45:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.304+0000 7f6189600700 1 --2- 172.21.15.63:0/4034028063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x56139419fd10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f617800bc80 tx=0x7f617800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a342d30fe354a5a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.304+0000 7f6176a00700 1 -- 172.21.15.63:0/4034028063 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178010ca0 con 0x56139401fec0 2024-05-27T07:45:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.305+0000 7f618a000700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f616c0b29f0 0x7f616c0b4f00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f617c0aab90 tx=0x7f617c01df90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.305+0000 7f6176a00700 1 -- 172.21.15.63:0/4034028063 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6178010e40 con 0x56139401fec0 2024-05-27T07:45:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.305+0000 7f6176a00700 1 -- 172.21.15.63:0/4034028063 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178005020 con 0x56139401fec0 2024-05-27T07:45:55.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.306+0000 7f619aa5db80 1 -- 172.21.15.63:0/4034028063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 msgr2=0x56139419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:55.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.306+0000 7f619aa5db80 1 --2- 172.21.15.63:0/4034028063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x56139419fd10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f617800bc80 tx=0x7f617800bf90 comp rx=0 tx=0).stop 2024-05-27T07:45:55.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.306+0000 7f619aa5db80 1 -- 172.21.15.63:0/4034028063 shutdown_connections 2024-05-27T07:45:55.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.306+0000 7f619aa5db80 1 --2- 172.21.15.63:0/4034028063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.306+0000 7f619aa5db80 1 --2- 172.21.15.63:0/4034028063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x56139419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.306+0000 7f619aa5db80 1 --2- 172.21.15.63:0/4034028063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x56139419fd10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.306+0000 7f619aa5db80 1 -- 172.21.15.63:0/4034028063 >> 172.21.15.63:0/4034028063 conn(0x5613940b9410 msgr2=0x56139419a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.306+0000 7f619aa5db80 1 -- 172.21.15.63:0/4034028063 shutdown_connections 2024-05-27T07:45:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.306+0000 7f619aa5db80 1 -- 172.21.15.63:0/4034028063 wait complete. 2024-05-27T07:45:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.307+0000 7f619aa5db80 1 -- start start 2024-05-27T07:45:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.307+0000 7f619aa5db80 1 Processor -- start 2024-05-27T07:45:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.308+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x5613941a6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.308+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x5613941a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.308+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.308+0000 7f619aa5db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561393fc3020 con 0x5613941a9340 2024-05-27T07:45:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.308+0000 7f619aa5db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56139405d7e0 con 0x56139401fec0 2024-05-27T07:45:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.308+0000 7f619aa5db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613941abe80 con 0x56139410ba70 2024-05-27T07:45:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.308+0000 7f619aa5db80 10 client.0 _ll_register_callbacks cb 0x561394066cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.308+0000 7f6188c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.309+0000 7f6188c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941a6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60882/0 (socket says 172.21.15.63:60882) 2024-05-27T07:45:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.309+0000 7f6188c00700 1 -- 172.21.15.63:0/4095754958 learned_addr learned my addr 172.21.15.63:0/4095754958 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.309+0000 7f6188c00700 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x5613941a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.309+0000 7f6189600700 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x5613941a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.309+0000 7f6188c00700 1 -- 172.21.15.63:0/4095754958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 msgr2=0x5613941a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.309+0000 7f6188c00700 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x5613941a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.309+0000 7f6188c00700 1 -- 172.21.15.63:0/4095754958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 msgr2=0x5613941a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.309+0000 7f6188c00700 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x5613941a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.309+0000 7f6188c00700 1 -- 172.21.15.63:0/4095754958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56139400c950 con 0x5613941a9340 2024-05-27T07:45:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.310+0000 7f6188c00700 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941a6d10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f617000bf40 tx=0x7f61700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2578003b3d085d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.310+0000 7f616b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:55.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.310+0000 7f616b400700 1 -- 172.21.15.63:0/4095754958 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170012c20 con 0x5613941a9340 2024-05-27T07:45:55.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.310+0000 7f616b400700 1 -- 172.21.15.63:0/4095754958 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f6170012dc0 con 0x5613941a9340 2024-05-27T07:45:55.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.310+0000 7f619aa5db80 1 -- 172.21.15.63:0/4095754958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6178010e40 con 0x5613941a9340 2024-05-27T07:45:55.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f616b400700 1 -- 172.21.15.63:0/4095754958 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f617002a050 con 0x5613941a9340 2024-05-27T07:45:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f619aa5db80 10 client.17802 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f619aa5db80 1 -- 172.21.15.63:0/4095754958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56139408d0e0 con 0x5613941a9340 2024-05-27T07:45:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f616b400700 1 -- 172.21.15.63:0/4095754958 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6178010e40 con 0x5613941a9340 2024-05-27T07:45:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f616b400700 1 -- 172.21.15.63:0/4095754958 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6170003b40 con 0x5613941a9340 2024-05-27T07:45:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f619aa5db80 10 client.17802 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f619aa5db80 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f619aa5db80 20 client.17802 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f619aa5db80 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613941b8ca0 0x5613941bb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f619aa5db80 4 client.17802 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.311+0000 7f619aa5db80 1 -- 172.21.15.63:0/4095754958 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613940cee10 con 0x5613941b8ca0 2024-05-27T07:45:55.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.312+0000 7f6189600700 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613941b8ca0 0x5613941bb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.313+0000 7f616b400700 10 client.17802 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:55.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.313+0000 7f6189600700 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613941b8ca0 0x5613941bb0c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c27de942a6dc9f7 server_cookie=4d3c0ee48efc915f in_seq=0 out_seq=0 2024-05-27T07:45:55.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.315+0000 7f616b400700 1 -- 172.21.15.63:0/4095754958 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5613940cee10 con 0x5613941b8ca0 2024-05-27T07:45:55.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.315+0000 7f616b400700 10 client.17802 handle_command_reply: tid=0 2024-05-27T07:45:55.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.315+0000 7f619aa5db80 1 client.17802 shutdown 2024-05-27T07:45:55.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 1 -- 172.21.15.63:0/4095754958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 msgr2=0x5613941a6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:55.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941a6d10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f617000bf40 tx=0x7f61700047c0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 1 -- 172.21.15.63:0/4095754958 shutdown_connections 2024-05-27T07:45:55.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613941b8ca0 0x5613941bb0c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941a6d10 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x5613941a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 1 --2- 172.21.15.63:0/4095754958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x5613941a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 1 -- 172.21.15.63:0/4095754958 >> 172.21.15.63:0/4095754958 conn(0x5613940b9410 msgr2=0x56139419f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:55.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f616b400700 0 client.17802 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:55.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 1 -- 172.21.15.63:0/4095754958 shutdown_connections 2024-05-27T07:45:55.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 1 -- 172.21.15.63:0/4095754958 wait complete. 2024-05-27T07:45:55.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.316+0000 7f619aa5db80 20 client.17802 trim_cache size 0 max 16384 2024-05-27T07:45:55.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.346+0000 7f619aa5db80 1 Processor -- start 2024-05-27T07:45:55.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.346+0000 7f619aa5db80 1 -- start start 2024-05-27T07:45:55.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.347+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139401fec0 0x5613941a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.347+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56139410ba70 0x56139419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.347+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613941a9340 0x5613941b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.347+0000 7f619aa5db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61841b9170 con 0x56139410ba70 2024-05-27T07:45:55.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.347+0000 7f619aa5db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61841b6a00 con 0x5613941a9340 2024-05-27T07:45:55.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.347+0000 7f619aa5db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f61841b4870 con 0x56139401fec0 2024-05-27T07:45:55.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.347+0000 7f6189600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56139410ba70 0x56139419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.347+0000 7f6189600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56139410ba70 0x56139419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60892/0 (socket says 172.21.15.63:60892) 2024-05-27T07:45:55.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.347+0000 7f6189600700 1 -- 172.21.15.63:0/772742449 learned_addr learned my addr 172.21.15.63:0/772742449 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:55.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.347+0000 7f6188c00700 1 --2- 172.21.15.63:0/772742449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139401fec0 0x5613941a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.348+0000 7f6188c00700 1 --2- 172.21.15.63:0/772742449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613941a9340 0x5613941b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.348+0000 7f6189600700 1 -- 172.21.15.63:0/772742449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139401fec0 msgr2=0x5613941a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:55.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.348+0000 7f6189600700 1 --2- 172.21.15.63:0/772742449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139401fec0 0x5613941a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.348+0000 7f6189600700 1 -- 172.21.15.63:0/772742449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613941a9340 msgr2=0x5613941b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:55.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.348+0000 7f6189600700 1 --2- 172.21.15.63:0/772742449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613941a9340 0x5613941b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.348+0000 7f6189600700 1 -- 172.21.15.63:0/772742449 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613941a9160 con 0x56139410ba70 2024-05-27T07:45:55.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.348+0000 7f6188c00700 1 --2- 172.21.15.63:0/772742449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613941a9340 0x5613941b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:55.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.348+0000 7f6189600700 1 --2- 172.21.15.63:0/772742449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56139410ba70 0x56139419a7c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x56139419b9f0 tx=0x7f6178005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6ef524d5857b8c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:55.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.348+0000 7f6176000700 1 -- 172.21.15.63:0/772742449 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61780049d0 con 0x56139410ba70 2024-05-27T07:45:55.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.349+0000 7f6176000700 1 -- 172.21.15.63:0/772742449 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6178010e30 con 0x56139410ba70 2024-05-27T07:45:55.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.349+0000 7f6176000700 1 -- 172.21.15.63:0/772742449 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178011440 con 0x56139410ba70 2024-05-27T07:45:55.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.350+0000 7f619aa5db80 1 -- 172.21.15.63:0/772742449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56139410ba70 msgr2=0x56139419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:55.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.350+0000 7f619aa5db80 1 --2- 172.21.15.63:0/772742449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56139410ba70 0x56139419a7c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x56139419b9f0 tx=0x7f6178005990 comp rx=0 tx=0).stop 2024-05-27T07:45:55.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.350+0000 7f619aa5db80 1 -- 172.21.15.63:0/772742449 shutdown_connections 2024-05-27T07:45:55.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.350+0000 7f619aa5db80 1 --2- 172.21.15.63:0/772742449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613941a9340 0x5613941b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.350+0000 7f619aa5db80 1 --2- 172.21.15.63:0/772742449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139401fec0 0x5613941a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.350+0000 7f619aa5db80 1 --2- 172.21.15.63:0/772742449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56139410ba70 0x56139419a7c0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.350+0000 7f619aa5db80 1 -- 172.21.15.63:0/772742449 >> 172.21.15.63:0/772742449 conn(0x5613940b9410 msgr2=0x56139419e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:55.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.350+0000 7f619aa5db80 1 -- 172.21.15.63:0/772742449 shutdown_connections 2024-05-27T07:45:55.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.350+0000 7f619aa5db80 1 -- 172.21.15.63:0/772742449 wait complete. 2024-05-27T07:45:55.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.350+0000 7f619aa5db80 1 -- start start 2024-05-27T07:45:55.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.351+0000 7f619aa5db80 1 Processor -- start 2024-05-27T07:45:55.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.351+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x5613941b1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.351+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x5613941ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.351+0000 7f619aa5db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.351+0000 7f619aa5db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61841b4ec0 con 0x5613941a9340 2024-05-27T07:45:55.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.351+0000 7f619aa5db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61841b6330 con 0x56139401fec0 2024-05-27T07:45:55.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.351+0000 7f619aa5db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f61841b5590 con 0x56139410ba70 2024-05-27T07:45:55.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.351+0000 7f619aa5db80 10 client.0 _ll_register_callbacks cb 0x561394066cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:55.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.352+0000 7f6188c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.352+0000 7f6188c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60904/0 (socket says 172.21.15.63:60904) 2024-05-27T07:45:55.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.352+0000 7f6188c00700 1 -- 172.21.15.63:0/1276401596 learned_addr learned my addr 172.21.15.63:0/1276401596 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:55.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.352+0000 7f6188c00700 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x5613941b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.352+0000 7f6189600700 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x5613941ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.352+0000 7f6188c00700 1 -- 172.21.15.63:0/1276401596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 msgr2=0x5613941ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:55.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.352+0000 7f6188c00700 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x5613941ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.353+0000 7f6188c00700 1 -- 172.21.15.63:0/1276401596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 msgr2=0x5613941b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.353+0000 7f6188c00700 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x5613941b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.353+0000 7f6188c00700 1 -- 172.21.15.63:0/1276401596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613941ae330 con 0x5613941a9340 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.353+0000 7f6188c00700 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941addf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6170014160 tx=0x7f61700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b3f90e9e7a2aab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.353+0000 7f616b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.353+0000 7f616b400700 1 -- 172.21.15.63:0/1276401596 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61700043d0 con 0x5613941a9340 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.353+0000 7f616b400700 1 -- 172.21.15.63:0/1276401596 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f61700240d0 con 0x5613941a9340 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.353+0000 7f619aa5db80 1 -- 172.21.15.63:0/1276401596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6178010e30 con 0x5613941a9340 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f616b400700 1 -- 172.21.15.63:0/1276401596 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f61700242d0 con 0x5613941a9340 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f619aa5db80 10 client.17808 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f619aa5db80 1 -- 172.21.15.63:0/1276401596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561394224310 con 0x5613941a9340 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f616b400700 1 -- 172.21.15.63:0/1276401596 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6178010e30 con 0x5613941a9340 2024-05-27T07:45:55.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f616b400700 1 -- 172.21.15.63:0/1276401596 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f617003a020 con 0x5613941a9340 2024-05-27T07:45:55.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f619aa5db80 10 client.17808 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:55.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f619aa5db80 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:55.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f619aa5db80 20 client.17808 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:55.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f619aa5db80 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561394224960 0x561394226d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:55.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f619aa5db80 4 client.17808 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:45:55.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.354+0000 7f619aa5db80 1 -- 172.21.15.63:0/1276401596 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5613940c85f0 con 0x561394224960 2024-05-27T07:45:55.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.355+0000 7f6189600700 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561394224960 0x561394226d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:55.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.356+0000 7f616b400700 10 client.17808 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:55.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.356+0000 7f6189600700 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561394224960 0x561394226d80 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab2fd64af67f0f77 server_cookie=a46abe2aa62d6da0 in_seq=0 out_seq=0 2024-05-27T07:45:55.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.359+0000 7f616b400700 1 -- 172.21.15.63:0/1276401596 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5613940c85f0 con 0x561394224960 2024-05-27T07:45:55.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.359+0000 7f616b400700 10 client.17808 handle_command_reply: tid=0 2024-05-27T07:45:55.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.359+0000 7f619aa5db80 1 client.17808 shutdown 2024-05-27T07:45:55.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.359+0000 7f619aa5db80 1 -- 172.21.15.63:0/1276401596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 msgr2=0x5613941addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:55.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.359+0000 7f619aa5db80 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941addf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6170014160 tx=0x7f61700047c0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.359+0000 7f619aa5db80 1 -- 172.21.15.63:0/1276401596 shutdown_connections 2024-05-27T07:45:55.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.359+0000 7f619aa5db80 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561394224960 0x561394226d80 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.360+0000 7f619aa5db80 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613941a9340 0x5613941addf0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.360+0000 7f619aa5db80 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56139401fec0 0x5613941b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.360+0000 7f619aa5db80 1 --2- 172.21.15.63:0/1276401596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56139410ba70 0x5613941ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.360+0000 7f619aa5db80 1 -- 172.21.15.63:0/1276401596 >> 172.21.15.63:0/1276401596 conn(0x5613940b9410 msgr2=0x5613941a6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:55.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.360+0000 7f616b400700 0 client.17808 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:55.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.360+0000 7f619aa5db80 1 -- 172.21.15.63:0/1276401596 shutdown_connections 2024-05-27T07:45:55.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.360+0000 7f619aa5db80 1 -- 172.21.15.63:0/1276401596 wait complete. 2024-05-27T07:45:55.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.360+0000 7f619aa5db80 20 client.17808 trim_cache size 0 max 16384 2024-05-27T07:45:55.361 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:45:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:45:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "df3c052e-d455-4d8f-a422-2f158e74ff24", 2024-05-27T07:45:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:45:55.362 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:45:55.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 -- 172.21.15.63:0/3227921317 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f616c0b29f0 msgr2=0x7f616c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:55.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f616c0b29f0 0x7f616c0b4f00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f617c0aab90 tx=0x7f617c01df90 comp rx=0 tx=0).stop 2024-05-27T07:45:55.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 -- 172.21.15.63:0/3227921317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f618410e6f0 msgr2=0x7f6184103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:55.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f618410e6f0 0x7f6184103550 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f617c00b830 tx=0x7f617c004d00 comp rx=0 tx=0).stop 2024-05-27T07:45:55.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 -- 172.21.15.63:0/3227921317 shutdown_connections 2024-05-27T07:45:55.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f616c0b29f0 0x7f616c0b4f00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f618410e6f0 0x7f6184103550 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6184105280 0x7f6184104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 --2- 172.21.15.63:0/3227921317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6184105bd0 0x7f6184102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:55.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 -- 172.21.15.63:0/3227921317 >> 172.21.15.63:0/3227921317 conn(0x7f6184078ab0 msgr2=0x7f61840fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:55.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.361+0000 7f618aa00700 1 -- 172.21.15.63:0/3227921317 shutdown_connections 2024-05-27T07:45:55.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:55.362+0000 7f618aa00700 1 -- 172.21.15.63:0/3227921317 wait complete. 2024-05-27T07:45:56.198 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:55 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:45:56.199 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.199 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:45:56.199 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:45:56.199 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.199 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:45:56.199 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.199 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:55 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:45:56.199 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:55 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:55 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:45:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:45:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:45:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:45:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:55 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:45:56.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:55 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:55 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:45:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:45:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:45:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:45:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:55 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:45:56.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:55 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:56.275 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:45:56.771 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:57.129 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:56 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:45:57.129 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:56 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.129 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:56 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:45:57.130 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:56 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.130 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:56 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:45:57.130 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:56 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.130 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:56 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.130 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:56 smithi063 ceph-mon[31613]: pgmap v386: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:45:57.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:56 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:45:57.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:56 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:56 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:45:57.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:56 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:56 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:45:57.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:56 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:56 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:56 smithi086 ceph-mon[34687]: pgmap v386: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:45:57.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:56 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:45:57.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:56 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:56 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:45:57.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:56 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:56 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:45:57.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:56 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:56 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:45:57.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:56 smithi149 ceph-mon[35408]: pgmap v386: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:45:58.894 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:45:58 smithi063 ceph-mon[31613]: pgmap v387: 129 pgs: 129 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2024-05-27T07:45:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.958+0000 7f9b4e600640 1 -- 172.21.15.63:0/414986664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b48104250 msgr2=0x7f9b481105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.958+0000 7f9b4e600640 1 --2- 172.21.15.63:0/414986664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b48104250 0x7f9b481105d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ca0 tx=0x7f9b4002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.958+0000 7f9b4e600640 1 -- 172.21.15.63:0/414986664 shutdown_connections 2024-05-27T07:45:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.958+0000 7f9b4e600640 1 --2- 172.21.15.63:0/414986664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b48104250 0x7f9b481105d0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.958+0000 7f9b4e600640 1 --2- 172.21.15.63:0/414986664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b48109f50 0x7f9b48103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.958+0000 7f9b4e600640 1 --2- 172.21.15.63:0/414986664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b48109ab0 0x7f9b481036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.958+0000 7f9b4e600640 1 -- 172.21.15.63:0/414986664 >> 172.21.15.63:0/414986664 conn(0x7f9b48079240 msgr2=0x7f9b48079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.958+0000 7f9b4e600640 1 -- 172.21.15.63:0/414986664 shutdown_connections 2024-05-27T07:45:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.958+0000 7f9b4e600640 1 -- 172.21.15.63:0/414986664 wait complete. 2024-05-27T07:45:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.959+0000 7f9b4e600640 1 Processor -- start 2024-05-27T07:45:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.959+0000 7f9b4e600640 1 -- start start 2024-05-27T07:45:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.960+0000 7f9b4e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b48104250 0x7f9b481a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.960+0000 7f9b4e600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b48109ab0 0x7f9b481a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.960+0000 7f9b4e600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b48109f50 0x7f9b4819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.960+0000 7f9b4e600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48113ba0 con 0x7f9b48109f50 2024-05-27T07:45:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.960+0000 7f9b4e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48113a20 con 0x7f9b48104250 2024-05-27T07:45:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.960+0000 7f9b4e600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48113d20 con 0x7f9b48109ab0 2024-05-27T07:45:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.960+0000 7f9b4dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b48109f50 0x7f9b4819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.960+0000 7f9b4dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b48109f50 0x7f9b4819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55918/0 (socket says 172.21.15.63:55918) 2024-05-27T07:45:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.960+0000 7f9b4dc00640 1 -- 172.21.15.63:0/290968590 learned_addr learned my addr 172.21.15.63:0/290968590 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.961+0000 7f9b3fe00640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b48109ab0 0x7f9b481a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.961+0000 7f9b4d200640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b48104250 0x7f9b481a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.961+0000 7f9b4dc00640 1 -- 172.21.15.63:0/290968590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b48109ab0 msgr2=0x7f9b481a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.961+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b48109ab0 0x7f9b481a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.961+0000 7f9b4dc00640 1 -- 172.21.15.63:0/290968590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b48104250 msgr2=0x7f9b481a2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.961+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b48104250 0x7f9b481a2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.961+0000 7f9b4dc00640 1 -- 172.21.15.63:0/290968590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4819cdb0 con 0x7f9b48109f50 2024-05-27T07:45:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.961+0000 7f9b4d200640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b48104250 0x7f9b481a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.961+0000 7f9b3fe00640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b48109ab0 0x7f9b481a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.961+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b48109f50 0x7f9b4819c680 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9b40005e30 tx=0x7f9b400093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.962+0000 7f9b46000640 1 -- 172.21.15.63:0/290968590 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40032d10 con 0x7f9b48109f50 2024-05-27T07:45:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.962+0000 7f9b46000640 1 -- 172.21.15.63:0/290968590 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b40032eb0 con 0x7f9b48109f50 2024-05-27T07:45:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.962+0000 7f9b4e600640 1 -- 172.21.15.63:0/290968590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4819cfe0 con 0x7f9b48109f50 2024-05-27T07:45:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.962+0000 7f9b46000640 1 -- 172.21.15.63:0/290968590 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40030b20 con 0x7f9b48109f50 2024-05-27T07:45:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.962+0000 7f9b4e600640 1 -- 172.21.15.63:0/290968590 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b481a92a0 con 0x7f9b48109f50 2024-05-27T07:45:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.966+0000 7f9b56b67740 1 Processor -- start 2024-05-27T07:45:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.967+0000 7f9b46000640 1 -- 172.21.15.63:0/290968590 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9b40030cc0 con 0x7f9b48109f50 2024-05-27T07:45:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.967+0000 7f9b56b67740 1 -- start start 2024-05-27T07:45:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.967+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94369d1b0 0x55c943a3fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.967+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9436742f0 0x55c943a3ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.967+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94367c4d0 0x55c943a404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.967+0000 7f9b56b67740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c9439b0b90 con 0x55c9436742f0 2024-05-27T07:45:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.967+0000 7f9b56b67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c9439a0730 con 0x55c94367c4d0 2024-05-27T07:45:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.967+0000 7f9b56b67740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c9439a4770 con 0x55c94369d1b0 2024-05-27T07:45:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.968+0000 7f9b4d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9436742f0 0x55c943a3ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.968+0000 7f9b4d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9436742f0 0x55c943a3ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55934/0 (socket says 172.21.15.63:55934) 2024-05-27T07:45:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.968+0000 7f9b4d200640 1 -- 172.21.15.63:0/963312218 learned_addr learned my addr 172.21.15.63:0/963312218 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.968+0000 7f9b3fe00640 1 --2- 172.21.15.63:0/963312218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94369d1b0 0x55c943a3fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.968+0000 7f9b3fe00640 1 --2- 172.21.15.63:0/963312218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94367c4d0 0x55c943a404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.968+0000 7f9b4d200640 1 -- 172.21.15.63:0/963312218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94369d1b0 msgr2=0x55c943a3fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.968+0000 7f9b4d200640 1 --2- 172.21.15.63:0/963312218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94369d1b0 0x55c943a3fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.969+0000 7f9b4d200640 1 -- 172.21.15.63:0/963312218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94367c4d0 msgr2=0x55c943a404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.969+0000 7f9b4d200640 1 --2- 172.21.15.63:0/963312218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94367c4d0 0x55c943a404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.969+0000 7f9b4d200640 1 -- 172.21.15.63:0/963312218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9439398b0 con 0x55c9436742f0 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.969+0000 7f9b46000640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b2c0b2950 0x7f9b2c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.969+0000 7f9b4d200640 1 --2- 172.21.15.63:0/963312218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9436742f0 0x55c943a3ffa0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9b38005fd0 tx=0x7f9b3800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7c2ed97de14267 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.969+0000 7f9b46000640 1 -- 172.21.15.63:0/290968590 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f9b40103e20 con 0x7f9b48109f50 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.969+0000 7f9b3fe00640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b2c0b2950 0x7f9b2c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.969+0000 7f9b3e000640 1 -- 172.21.15.63:0/963312218 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38016070 con 0x55c9436742f0 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.970+0000 7f9b3e000640 1 -- 172.21.15.63:0/963312218 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b380054b0 con 0x55c9436742f0 2024-05-27T07:45:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.970+0000 7f9b3e000640 1 -- 172.21.15.63:0/963312218 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38004e50 con 0x55c9436742f0 2024-05-27T07:45:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.971+0000 7f9b56b67740 1 -- 172.21.15.63:0/963312218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9436742f0 msgr2=0x55c943a3ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.971+0000 7f9b3fe00640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b2c0b2950 0x7f9b2c0b4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55c943a3cbd0 tx=0x7f9b34005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.971+0000 7f9b56b67740 1 --2- 172.21.15.63:0/963312218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9436742f0 0x55c943a3ffa0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9b38005fd0 tx=0x7f9b3800e6c0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.971+0000 7f9b56b67740 1 -- 172.21.15.63:0/963312218 shutdown_connections 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.971+0000 7f9b56b67740 1 --2- 172.21.15.63:0/963312218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94367c4d0 0x55c943a404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.971+0000 7f9b56b67740 1 --2- 172.21.15.63:0/963312218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9436742f0 0x55c943a3ffa0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.971+0000 7f9b56b67740 1 --2- 172.21.15.63:0/963312218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94369d1b0 0x55c943a3fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.971+0000 7f9b56b67740 1 -- 172.21.15.63:0/963312218 >> 172.21.15.63:0/963312218 conn(0x55c943691300 msgr2=0x55c943a3c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.971+0000 7f9b56b67740 1 -- 172.21.15.63:0/963312218 shutdown_connections 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.971+0000 7f9b56b67740 1 -- 172.21.15.63:0/963312218 wait complete. 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.972+0000 7f9b56b67740 1 -- start start 2024-05-27T07:45:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.972+0000 7f9b56b67740 1 Processor -- start 2024-05-27T07:45:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.972+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9436742f0 0x55c943a49c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.972+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94367c4d0 0x55c943a4a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.972+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a55ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.972+0000 7f9b56b67740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c943686790 con 0x55c94369d1b0 2024-05-27T07:45:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.972+0000 7f9b56b67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c943948bc0 con 0x55c94367c4d0 2024-05-27T07:45:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.972+0000 7f9b56b67740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c943a4cb90 con 0x55c9436742f0 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b56b67740 10 client.0 _ll_register_callbacks cb 0x55c943926400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b4dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a55ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b4dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a55ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55950/0 (socket says 172.21.15.63:55950) 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b4dc00640 1 -- 172.21.15.63:0/2840412531 learned_addr learned my addr 172.21.15.63:0/2840412531 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b4d200640 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9436742f0 0x55c943a49c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b3fe00640 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94367c4d0 0x55c943a4a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b4dc00640 1 -- 172.21.15.63:0/2840412531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9436742f0 msgr2=0x55c943a49c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9436742f0 0x55c943a49c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b4dc00640 1 -- 172.21.15.63:0/2840412531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94367c4d0 msgr2=0x55c943a4a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94367c4d0 0x55c943a4a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.973+0000 7f9b4dc00640 1 -- 172.21.15.63:0/2840412531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9436b5210 con 0x55c94369d1b0 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.974+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a55ec0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9b400caaf0 tx=0x7f9b400cb5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52521f9551e7fcc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.974+0000 7f9b32a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.974+0000 7f9b32a00640 1 -- 172.21.15.63:0/2840412531 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b400d3c50 con 0x55c94369d1b0 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.974+0000 7f9b32a00640 1 -- 172.21.15.63:0/2840412531 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f9b40004040 con 0x55c94369d1b0 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.974+0000 7f9b56b67740 1 -- 172.21.15.63:0/2840412531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b380054b0 con 0x55c94369d1b0 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.974+0000 7f9b32a00640 1 -- 172.21.15.63:0/2840412531 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9b400cb050 con 0x55c94369d1b0 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.974+0000 7f9b56b67740 10 client.17820 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.974+0000 7f9b56b67740 1 -- 172.21.15.63:0/2840412531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c943781620 con 0x55c94369d1b0 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.975+0000 7f9b32a00640 1 -- 172.21.15.63:0/2840412531 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b380054b0 con 0x55c94369d1b0 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.975+0000 7f9b32a00640 1 -- 172.21.15.63:0/2840412531 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9b400ca050 con 0x55c94369d1b0 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.975+0000 7f9b56b67740 10 client.17820 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.975+0000 7f9b56b67740 10 client.17820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.975+0000 7f9b56b67740 20 client.17820 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.975+0000 7f9b56b67740 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c94365bd80 0x55c943a5bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.975+0000 7f9b56b67740 4 client.17820 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.975+0000 7f9b56b67740 1 -- 172.21.15.63:0/2840412531 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9439b2ae0 con 0x55c94365bd80 2024-05-27T07:45:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.976+0000 7f9b4d200640 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c94365bd80 0x55c943a5bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.977+0000 7f9b4d200640 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c94365bd80 0x55c943a5bf80 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513775fa5b68a3d1 server_cookie=f5bbf0a8b47dda0d in_seq=0 out_seq=0 2024-05-27T07:45:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.977+0000 7f9b32a00640 10 client.17820 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b32a00640 1 -- 172.21.15.63:0/2840412531 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c9439b2ae0 con 0x55c94365bd80 2024-05-27T07:45:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b32a00640 10 client.17820 handle_command_reply: tid=0 2024-05-27T07:45:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 client.17820 shutdown 2024-05-27T07:45:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 -- 172.21.15.63:0/2840412531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 msgr2=0x55c943a55ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a55ec0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9b400caaf0 tx=0x7f9b400cb5b0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 -- 172.21.15.63:0/2840412531 shutdown_connections 2024-05-27T07:45:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c94365bd80 0x55c943a5bf80 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a55ec0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c94367c4d0 0x55c943a4a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 --2- 172.21.15.63:0/2840412531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9436742f0 0x55c943a49c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 -- 172.21.15.63:0/2840412531 >> 172.21.15.63:0/2840412531 conn(0x55c943691300 msgr2=0x55c943a3f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b32a00640 0 client.17820 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 -- 172.21.15.63:0/2840412531 shutdown_connections 2024-05-27T07:45:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.979+0000 7f9b56b67740 1 -- 172.21.15.63:0/2840412531 wait complete. 2024-05-27T07:45:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.980+0000 7f9b56b67740 20 client.17820 trim_cache size 0 max 16384 2024-05-27T07:45:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.999+0000 7f9b56b67740 1 Processor -- start 2024-05-27T07:45:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:58.999+0000 7f9b56b67740 1 -- start start 2024-05-27T07:45:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 0x55c943a61740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94367c4d0 0x55c943a61c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a51e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b56b67740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481b5af0 con 0x55c94369d1b0 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b56b67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481b33c0 con 0x55c9436742f0 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b56b67740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481b1230 con 0x55c94367c4d0 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b4dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a51e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b4dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a51e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55954/0 (socket says 172.21.15.63:55954) 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b4dc00640 1 -- 172.21.15.63:0/118485366 learned_addr learned my addr 172.21.15.63:0/118485366 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b3fe00640 1 --2- 172.21.15.63:0/118485366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94367c4d0 0x55c943a61c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.000+0000 7f9b4d200640 1 --2- 172.21.15.63:0/118485366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 0x55c943a61740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.001+0000 7f9b4dc00640 1 -- 172.21.15.63:0/118485366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94367c4d0 msgr2=0x55c943a61c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.001+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/118485366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94367c4d0 0x55c943a61c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.001+0000 7f9b4dc00640 1 -- 172.21.15.63:0/118485366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 msgr2=0x55c943a61740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.001+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/118485366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 0x55c943a61740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.001+0000 7f9b4dc00640 1 -- 172.21.15.63:0/118485366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c943a481b0 con 0x55c94369d1b0 2024-05-27T07:45:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.001+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/118485366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a51e20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f9b401016d0 tx=0x7f9b400c7290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efffa65db20bb7d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.001+0000 7f9b3d600640 1 -- 172.21.15.63:0/118485366 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b400cabc0 con 0x55c94369d1b0 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.001+0000 7f9b3d600640 1 -- 172.21.15.63:0/118485366 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b400e9020 con 0x55c94369d1b0 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.001+0000 7f9b3d600640 1 -- 172.21.15.63:0/118485366 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40004040 con 0x55c94369d1b0 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.002+0000 7f9b56b67740 1 -- 172.21.15.63:0/118485366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 msgr2=0x55c943a51e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.002+0000 7f9b56b67740 1 --2- 172.21.15.63:0/118485366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a51e20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f9b401016d0 tx=0x7f9b400c7290 comp rx=0 tx=0).stop 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.002+0000 7f9b56b67740 1 -- 172.21.15.63:0/118485366 shutdown_connections 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.002+0000 7f9b56b67740 1 --2- 172.21.15.63:0/118485366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943a51e20 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.002+0000 7f9b56b67740 1 --2- 172.21.15.63:0/118485366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94367c4d0 0x55c943a61c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.002+0000 7f9b56b67740 1 --2- 172.21.15.63:0/118485366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 0x55c943a61740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.002+0000 7f9b56b67740 1 -- 172.21.15.63:0/118485366 >> 172.21.15.63:0/118485366 conn(0x55c943691300 msgr2=0x55c943a44e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.002+0000 7f9b56b67740 1 -- 172.21.15.63:0/118485366 shutdown_connections 2024-05-27T07:45:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.003+0000 7f9b56b67740 1 -- 172.21.15.63:0/118485366 wait complete. 2024-05-27T07:45:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.003+0000 7f9b56b67740 1 -- start start 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.003+0000 7f9b56b67740 1 Processor -- start 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.004+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 0x55c943abcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.004+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94367c4d0 0x55c943abd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.004+0000 7f9b56b67740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943ab6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.004+0000 7f9b56b67740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481b1880 con 0x55c94369d1b0 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.004+0000 7f9b56b67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481b2cf0 con 0x55c9436742f0 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.004+0000 7f9b56b67740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9b481b1f50 con 0x55c94367c4d0 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.004+0000 7f9b56b67740 10 client.0 _ll_register_callbacks cb 0x7f9b400099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.004+0000 7f9b4dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943ab6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.004+0000 7f9b4dc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943ab6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55958/0 (socket says 172.21.15.63:55958) 2024-05-27T07:45:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.004+0000 7f9b4dc00640 1 -- 172.21.15.63:0/3562349904 learned_addr learned my addr 172.21.15.63:0/3562349904 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b4d200640 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 0x55c943abcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b3fe00640 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94367c4d0 0x55c943abd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b4dc00640 1 -- 172.21.15.63:0/3562349904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94367c4d0 msgr2=0x55c943abd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94367c4d0 0x55c943abd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b4dc00640 1 -- 172.21.15.63:0/3562349904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 msgr2=0x55c943abcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 0x55c943abcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b4dc00640 1 -- 172.21.15.63:0/3562349904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c943ab75d0 con 0x55c94369d1b0 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b4d200640 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 0x55c943abcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b4dc00640 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943ab6fc0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9b40006010 tx=0x7f9b40101340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224911ef4d4d228d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b32a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.005+0000 7f9b32a00640 1 -- 172.21.15.63:0/3562349904 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b400a9070 con 0x55c94369d1b0 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.006+0000 7f9b32a00640 1 -- 172.21.15.63:0/3562349904 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f9b400c73e0 con 0x55c94369d1b0 2024-05-27T07:45:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.006+0000 7f9b56b67740 1 -- 172.21.15.63:0/3562349904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b400e9020 con 0x55c94369d1b0 2024-05-27T07:45:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.006+0000 7f9b32a00640 1 -- 172.21.15.63:0/3562349904 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9b400cabc0 con 0x55c94369d1b0 2024-05-27T07:45:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.006+0000 7f9b56b67740 10 client.17826 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.006+0000 7f9b56b67740 1 -- 172.21.15.63:0/3562349904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c943ab75d0 con 0x55c94369d1b0 2024-05-27T07:45:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.006+0000 7f9b32a00640 1 -- 172.21.15.63:0/3562349904 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b400e9020 con 0x55c94369d1b0 2024-05-27T07:45:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.006+0000 7f9b32a00640 1 -- 172.21.15.63:0/3562349904 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9b400ac020 con 0x55c94369d1b0 2024-05-27T07:45:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.006+0000 7f9b56b67740 10 client.17826 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.006+0000 7f9b56b67740 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.006+0000 7f9b56b67740 20 client.17826 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.007+0000 7f9b56b67740 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9436311c0 0x55c943ac64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.007+0000 7f9b56b67740 4 client.17826 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:45:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.007+0000 7f9b56b67740 1 -- 172.21.15.63:0/3562349904 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9439ac2c0 con 0x55c9436311c0 2024-05-27T07:45:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.007+0000 7f9b4d200640 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9436311c0 0x55c943ac64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.008+0000 7f9b32a00640 10 client.17826 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.008+0000 7f9b4d200640 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9436311c0 0x55c943ac64e0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f61aec0aea95aac2 server_cookie=886f54f168a879cd in_seq=0 out_seq=0 2024-05-27T07:45:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.045+0000 7f9b32a00640 1 -- 172.21.15.63:0/3562349904 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c9439ac2c0 con 0x55c9436311c0 2024-05-27T07:45:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.045+0000 7f9b32a00640 10 client.17826 handle_command_reply: tid=0 2024-05-27T07:45:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.045+0000 7f9b56b67740 1 client.17826 shutdown 2024-05-27T07:45:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 1 -- 172.21.15.63:0/3562349904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 msgr2=0x55c943ab6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943ab6fc0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9b40006010 tx=0x7f9b40101340 comp rx=0 tx=0).stop 2024-05-27T07:45:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 1 -- 172.21.15.63:0/3562349904 shutdown_connections 2024-05-27T07:45:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9436311c0 0x55c943ac64e0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c94369d1b0 0x55c943ab6fc0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c94367c4d0 0x55c943abd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 1 --2- 172.21.15.63:0/3562349904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9436742f0 0x55c943abcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 1 -- 172.21.15.63:0/3562349904 >> 172.21.15.63:0/3562349904 conn(0x55c943691300 msgr2=0x55c943a60730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b32a00640 0 client.17826 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 1 -- 172.21.15.63:0/3562349904 shutdown_connections 2024-05-27T07:45:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 1 -- 172.21.15.63:0/3562349904 wait complete. 2024-05-27T07:45:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.046+0000 7f9b56b67740 20 client.17826 trim_cache size 0 max 16384 2024-05-27T07:45:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:45:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:45:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:45:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 -- 172.21.15.63:0/290968590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b2c0b2950 msgr2=0x7f9b2c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b2c0b2950 0x7f9b2c0b4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55c943a3cbd0 tx=0x7f9b34005f90 comp rx=0 tx=0).stop 2024-05-27T07:45:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 -- 172.21.15.63:0/290968590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b48109f50 msgr2=0x7f9b4819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b48109f50 0x7f9b4819c680 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9b40005e30 tx=0x7f9b400093d0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 -- 172.21.15.63:0/290968590 shutdown_connections 2024-05-27T07:45:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9b2c0b2950 0x7f9b2c0b4e60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b48109f50 0x7f9b4819c680 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b48109ab0 0x7f9b481a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 --2- 172.21.15.63:0/290968590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b48104250 0x7f9b481a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 -- 172.21.15.63:0/290968590 >> 172.21.15.63:0/290968590 conn(0x7f9b48079240 msgr2=0x7f9b480768f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 -- 172.21.15.63:0/290968590 shutdown_connections 2024-05-27T07:45:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:45:59.048+0000 7f9b4e600640 1 -- 172.21.15.63:0/290968590 wait complete. 2024-05-27T07:45:59.207 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:45:58 smithi086 ceph-mon[34687]: pgmap v387: 129 pgs: 129 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2024-05-27T07:45:59.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:45:58 smithi149 ceph-mon[35408]: pgmap v387: 129 pgs: 129 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.7 MiB/s wr, 40 op/s 2024-05-27T07:45:59.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.277+0000 7f0822400700 1 -- 172.21.15.63:0/4169008532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c105280 msgr2=0x7f081c105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.277+0000 7f0822400700 1 --2- 172.21.15.63:0/4169008532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c105280 0x7f081c105690 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0810009990 tx=0x7f0810009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.277+0000 7f0822400700 1 -- 172.21.15.63:0/4169008532 shutdown_connections 2024-05-27T07:45:59.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.277+0000 7f0822400700 1 --2- 172.21.15.63:0/4169008532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f081c10e6f0 0x7f081c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.277+0000 7f0822400700 1 --2- 172.21.15.63:0/4169008532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c105280 0x7f081c105690 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.277+0000 7f0822400700 1 --2- 172.21.15.63:0/4169008532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081c105bd0 0x7f081c10e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.277+0000 7f0822400700 1 -- 172.21.15.63:0/4169008532 >> 172.21.15.63:0/4169008532 conn(0x7f081c078ab0 msgr2=0x7f081c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:59.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.277+0000 7f0822400700 1 -- 172.21.15.63:0/4169008532 shutdown_connections 2024-05-27T07:45:59.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.277+0000 7f0822400700 1 -- 172.21.15.63:0/4169008532 wait complete. 2024-05-27T07:45:59.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.280+0000 7f0822400700 1 Processor -- start 2024-05-27T07:45:59.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.280+0000 7f0822400700 1 -- start start 2024-05-27T07:45:59.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.281+0000 7f0822400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081c105280 0x7f081c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.281+0000 7f0822400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c105bd0 0x7f081c19d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.281+0000 7f0822400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f081c10e6f0 0x7f081c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.281+0000 7f0822400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f081c114160 con 0x7f081c105bd0 2024-05-27T07:45:59.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.281+0000 7f0822400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f081c113fe0 con 0x7f081c105280 2024-05-27T07:45:59.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.281+0000 7f0822400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1142e0 con 0x7f081c10e6f0 2024-05-27T07:45:59.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.281+0000 7f081be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c105bd0 0x7f081c19d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.282+0000 7f081be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c105bd0 0x7f081c19d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55980/0 (socket says 172.21.15.63:55980) 2024-05-27T07:45:59.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.282+0000 7f081be00700 1 -- 172.21.15.63:0/2599172765 learned_addr learned my addr 172.21.15.63:0/2599172765 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:59.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.282+0000 7f0821000700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081c105280 0x7f081c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.282+0000 7f0821a00700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f081c10e6f0 0x7f081c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.282+0000 7f081be00700 1 -- 172.21.15.63:0/2599172765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f081c10e6f0 msgr2=0x7f081c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.282+0000 7f081be00700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f081c10e6f0 0x7f081c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.282+0000 7f081be00700 1 -- 172.21.15.63:0/2599172765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081c105280 msgr2=0x7f081c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.282+0000 7f081be00700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081c105280 0x7f081c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.282+0000 7f081be00700 1 -- 172.21.15.63:0/2599172765 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c1a1c20 con 0x7f081c105bd0 2024-05-27T07:45:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.283+0000 7f0821a00700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f081c10e6f0 0x7f081c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.283+0000 7f081be00700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c105bd0 0x7f081c19d2f0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f080800d950 tx=0x7f080800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.283+0000 7f0819600700 1 -- 172.21.15.63:0/2599172765 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808016070 con 0x7f081c105bd0 2024-05-27T07:45:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.283+0000 7f0822400700 1 -- 172.21.15.63:0/2599172765 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c1a1f10 con 0x7f081c105bd0 2024-05-27T07:45:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.283+0000 7f0819600700 1 -- 172.21.15.63:0/2599172765 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0808010460 con 0x7f081c105bd0 2024-05-27T07:45:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.283+0000 7f0819600700 1 -- 172.21.15.63:0/2599172765 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080800f620 con 0x7f081c105bd0 2024-05-27T07:45:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.283+0000 7f0822400700 1 -- 172.21.15.63:0/2599172765 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c1a9830 con 0x7f081c105bd0 2024-05-27T07:45:59.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.288+0000 7f0819600700 1 -- 172.21.15.63:0/2599172765 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0808019070 con 0x7f081c105bd0 2024-05-27T07:45:59.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.289+0000 7f08324dbb80 1 Processor -- start 2024-05-27T07:45:59.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.289+0000 7f08324dbb80 1 -- start start 2024-05-27T07:45:59.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.290+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd8f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.290+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd8fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.290+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 0x55cf7fd9b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.290+0000 7f08324dbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cf7fcbce40 con 0x55cf7fc0fe40 2024-05-27T07:45:59.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.290+0000 7f08324dbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf7fcac9e0 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.290+0000 7f08324dbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cf7fcb0a20 con 0x55cf7fd992c0 2024-05-27T07:45:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.290+0000 7f0819600700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f08040b2a70 0x7f08040b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.290+0000 7f0821000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd8fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.290+0000 7f0821000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd8fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55994/0 (socket says 172.21.15.63:55994) 2024-05-27T07:45:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.290+0000 7f0821000700 1 -- 172.21.15.63:0/2832105491 learned_addr learned my addr 172.21.15.63:0/2832105491 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.291+0000 7f0819600700 1 -- 172.21.15.63:0/2599172765 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f08080e24d0 con 0x7f081c105bd0 2024-05-27T07:45:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.291+0000 7f0821a00700 1 --2- 172.21.15.63:0/2832105491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd8f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.291+0000 7f081be00700 1 --2- 172.21.15.63:0/2832105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 0x55cf7fd9b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.291+0000 7f0821a00700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f08040b2a70 0x7f08040b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.291+0000 7f081be00700 1 -- 172.21.15.63:0/2832105491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 msgr2=0x55cf7fd8f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.291+0000 7f081be00700 1 --2- 172.21.15.63:0/2832105491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd8f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.291+0000 7f081be00700 1 -- 172.21.15.63:0/2832105491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 msgr2=0x55cf7fd8fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.291+0000 7f081be00700 1 --2- 172.21.15.63:0/2832105491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd8fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.291+0000 7f081be00700 1 -- 172.21.15.63:0/2832105491 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7fc75420 con 0x55cf7fd992c0 2024-05-27T07:45:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.292+0000 7f0821000700 1 --2- 172.21.15.63:0/2832105491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd8fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:45:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.292+0000 7f0821a00700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f08040b2a70 0x7f08040b4f80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0814009fd0 tx=0x7f08140097e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.292+0000 7f081be00700 1 --2- 172.21.15.63:0/2832105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 0x55cf7fd9b6e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08080df900 tx=0x7f08080a8c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0e9cce8c79cc41 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.292+0000 7f080e000700 1 -- 172.21.15.63:0/2832105491 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808020070 con 0x55cf7fd992c0 2024-05-27T07:45:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.293+0000 7f080e000700 1 -- 172.21.15.63:0/2832105491 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08080a9490 con 0x55cf7fd992c0 2024-05-27T07:45:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.293+0000 7f080e000700 1 -- 172.21.15.63:0/2832105491 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808010930 con 0x55cf7fd992c0 2024-05-27T07:45:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.294+0000 7f08324dbb80 1 -- 172.21.15.63:0/2832105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 msgr2=0x55cf7fd9b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.294+0000 7f08324dbb80 1 --2- 172.21.15.63:0/2832105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 0x55cf7fd9b6e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08080df900 tx=0x7f08080a8c80 comp rx=0 tx=0).stop 2024-05-27T07:45:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.294+0000 7f08324dbb80 1 -- 172.21.15.63:0/2832105491 shutdown_connections 2024-05-27T07:45:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.294+0000 7f08324dbb80 1 --2- 172.21.15.63:0/2832105491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 0x55cf7fd9b6e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.294+0000 7f08324dbb80 1 --2- 172.21.15.63:0/2832105491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd8f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.294+0000 7f08324dbb80 1 --2- 172.21.15.63:0/2832105491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd8fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.294+0000 7f08324dbb80 1 -- 172.21.15.63:0/2832105491 >> 172.21.15.63:0/2832105491 conn(0x55cf7fca9390 msgr2=0x55cf7fd8a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:59.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.294+0000 7f08324dbb80 1 -- 172.21.15.63:0/2832105491 shutdown_connections 2024-05-27T07:45:59.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.294+0000 7f08324dbb80 1 -- 172.21.15.63:0/2832105491 wait complete. 2024-05-27T07:45:59.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.295+0000 7f08324dbb80 1 -- start start 2024-05-27T07:45:59.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.295+0000 7f08324dbb80 1 Processor -- start 2024-05-27T07:45:59.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd96270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd967b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 0x55cf7fd96cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f08324dbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cf7fbb2fa0 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f08324dbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf7fc4d760 con 0x55cf7fc0fe40 2024-05-27T07:45:59.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f08324dbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cf7fd9be00 con 0x55cf7fd992c0 2024-05-27T07:45:59.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f08324dbb80 10 client.0 _ll_register_callbacks cb 0x55cf7fc56c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:59.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f081be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f0821000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd96270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f081be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56004/0 (socket says 172.21.15.63:56004) 2024-05-27T07:45:59.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.296+0000 7f081be00700 1 -- 172.21.15.63:0/3234469291 learned_addr learned my addr 172.21.15.63:0/3234469291 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:59.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.297+0000 7f0821a00700 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 0x55cf7fd96cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.297+0000 7f081be00700 1 -- 172.21.15.63:0/3234469291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 msgr2=0x55cf7fd96cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.297+0000 7f081be00700 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 0x55cf7fd96cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.297+0000 7f081be00700 1 -- 172.21.15.63:0/3234469291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fc0fe40 msgr2=0x55cf7fd96270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.297+0000 7f081be00700 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd96270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.297+0000 7f081be00700 1 -- 172.21.15.63:0/3234469291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7fbfc8d0 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.297+0000 7f081be00700 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd967b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f08080a8d70 tx=0x7f08080e2390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d9fc29ab0cd5e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:59.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.297+0000 7f0802a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:59.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.298+0000 7f0802a00700 1 -- 172.21.15.63:0/3234469291 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808020070 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.298+0000 7f0802a00700 1 -- 172.21.15.63:0/3234469291 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f08080a96f0 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.298+0000 7f08324dbb80 1 -- 172.21.15.63:0/3234469291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08080a9490 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.298+0000 7f0802a00700 1 -- 172.21.15.63:0/3234469291 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f08080c3050 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.298+0000 7f08324dbb80 10 client.17838 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.298+0000 7f08324dbb80 1 -- 172.21.15.63:0/3234469291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cf7fc7d060 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.299+0000 7f0802a00700 1 -- 172.21.15.63:0/3234469291 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08080a9490 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.299+0000 7f0802a00700 1 -- 172.21.15.63:0/3234469291 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f08080b0bc0 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.299+0000 7f08324dbb80 10 client.17838 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.299+0000 7f08324dbb80 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.299+0000 7f08324dbb80 20 client.17838 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.299+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cf7fda8bd0 0x55cf7fdaaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.299+0000 7f08324dbb80 4 client.17838 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.299+0000 7f08324dbb80 1 -- 172.21.15.63:0/3234469291 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf7fcbed90 con 0x55cf7fda8bd0 2024-05-27T07:45:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.300+0000 7f0821000700 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cf7fda8bd0 0x55cf7fdaaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.301+0000 7f0821000700 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cf7fda8bd0 0x55cf7fdaaff0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc454c9e62400fa0 server_cookie=75ade6ca816016fd in_seq=0 out_seq=0 2024-05-27T07:45:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.301+0000 7f0802a00700 10 client.17838 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f0802a00700 1 -- 172.21.15.63:0/3234469291 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cf7fcbed90 con 0x55cf7fda8bd0 2024-05-27T07:45:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f0802a00700 10 client.17838 handle_command_reply: tid=0 2024-05-27T07:45:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f08324dbb80 1 client.17838 shutdown 2024-05-27T07:45:59.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f08324dbb80 1 -- 172.21.15.63:0/3234469291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fcfb9f0 msgr2=0x55cf7fd967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd967b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f08080a8d70 tx=0x7f08080e2390 comp rx=0 tx=0).stop 2024-05-27T07:45:59.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f08324dbb80 1 -- 172.21.15.63:0/3234469291 shutdown_connections 2024-05-27T07:45:59.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cf7fda8bd0 0x55cf7fdaaff0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fd992c0 0x55cf7fd96cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd96270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3234469291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd967b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f08324dbb80 1 -- 172.21.15.63:0/3234469291 >> 172.21.15.63:0/3234469291 conn(0x55cf7fca9390 msgr2=0x55cf7fd8a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:59.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.303+0000 7f0802a00700 0 client.17838 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:59.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.304+0000 7f08324dbb80 1 -- 172.21.15.63:0/3234469291 shutdown_connections 2024-05-27T07:45:59.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.304+0000 7f08324dbb80 1 -- 172.21.15.63:0/3234469291 wait complete. 2024-05-27T07:45:59.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.304+0000 7f08324dbb80 20 client.17838 trim_cache size 0 max 16384 2024-05-27T07:45:59.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.332+0000 7f08324dbb80 1 Processor -- start 2024-05-27T07:45:59.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.332+0000 7f08324dbb80 1 -- start start 2024-05-27T07:45:59.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.332+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd8a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.332+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd95590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.332+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fd992c0 0x55cf7fda50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.332+0000 7f08324dbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1b0b00 con 0x55cf7fd992c0 2024-05-27T07:45:59.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.332+0000 7f08324dbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1ae390 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.332+0000 7f08324dbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1ac200 con 0x55cf7fc0fe40 2024-05-27T07:45:59.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f0821a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fd992c0 0x55cf7fda50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f0821a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fd992c0 0x55cf7fda50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56014/0 (socket says 172.21.15.63:56014) 2024-05-27T07:45:59.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f0821a00700 1 -- 172.21.15.63:0/1002356107 learned_addr learned my addr 172.21.15.63:0/1002356107 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:59.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f0821000700 1 --2- 172.21.15.63:0/1002356107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd8a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f081be00700 1 --2- 172.21.15.63:0/1002356107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd95590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f0821a00700 1 -- 172.21.15.63:0/1002356107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fc0fe40 msgr2=0x55cf7fd8a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f0821a00700 1 --2- 172.21.15.63:0/1002356107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd8a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f0821a00700 1 -- 172.21.15.63:0/1002356107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 msgr2=0x55cf7fd95590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f0821a00700 1 --2- 172.21.15.63:0/1002356107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd95590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f0821a00700 1 -- 172.21.15.63:0/1002356107 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7fd97060 con 0x55cf7fd992c0 2024-05-27T07:45:59.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.333+0000 7f081be00700 1 --2- 172.21.15.63:0/1002356107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd95590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:59.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.334+0000 7f0821a00700 1 --2- 172.21.15.63:0/1002356107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fd992c0 0x55cf7fda50d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f08140178c0 tx=0x7f0814004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa436244d2e0ed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:59.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.334+0000 7f080d600700 1 -- 172.21.15.63:0/1002356107 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814009a90 con 0x55cf7fd992c0 2024-05-27T07:45:59.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.334+0000 7f080d600700 1 -- 172.21.15.63:0/1002356107 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f081400d040 con 0x55cf7fd992c0 2024-05-27T07:45:59.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.334+0000 7f080d600700 1 -- 172.21.15.63:0/1002356107 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140237d0 con 0x55cf7fd992c0 2024-05-27T07:45:59.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.335+0000 7f08324dbb80 1 -- 172.21.15.63:0/1002356107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fd992c0 msgr2=0x55cf7fda50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.335+0000 7f08324dbb80 1 --2- 172.21.15.63:0/1002356107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fd992c0 0x55cf7fda50d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f08140178c0 tx=0x7f0814004ad0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.335+0000 7f08324dbb80 1 -- 172.21.15.63:0/1002356107 shutdown_connections 2024-05-27T07:45:59.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.335+0000 7f08324dbb80 1 --2- 172.21.15.63:0/1002356107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fd992c0 0x55cf7fda50d0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.335+0000 7f08324dbb80 1 --2- 172.21.15.63:0/1002356107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fc0fe40 0x55cf7fd8a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.335+0000 7f08324dbb80 1 --2- 172.21.15.63:0/1002356107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fd95590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.335+0000 7f08324dbb80 1 -- 172.21.15.63:0/1002356107 >> 172.21.15.63:0/1002356107 conn(0x55cf7fca9390 msgr2=0x55cf7fd8e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.336+0000 7f08324dbb80 1 -- 172.21.15.63:0/1002356107 shutdown_connections 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.336+0000 7f08324dbb80 1 -- 172.21.15.63:0/1002356107 wait complete. 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.336+0000 7f08324dbb80 1 -- start start 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.336+0000 7f08324dbb80 1 Processor -- start 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.337+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fdaab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.337+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fdab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.337+0000 7f08324dbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fd992c0 0x55cf7fdab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.337+0000 7f08324dbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1ac850 con 0x55cf7fc0fe40 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.337+0000 7f08324dbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1adcc0 con 0x55cf7fd992c0 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.337+0000 7f08324dbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f081c1acf20 con 0x55cf7fcfb9f0 2024-05-27T07:45:59.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.337+0000 7f08324dbb80 10 client.0 _ll_register_callbacks cb 0x55cf7fc56c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.337+0000 7f0821000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fdaab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.337+0000 7f0821000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fdaab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56022/0 (socket says 172.21.15.63:56022) 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.337+0000 7f0821000700 1 -- 172.21.15.63:0/3495761888 learned_addr learned my addr 172.21.15.63:0/3495761888 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.338+0000 7f081be00700 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fdab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.338+0000 7f0821a00700 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fd992c0 0x55cf7fdab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.338+0000 7f0821000700 1 -- 172.21.15.63:0/3495761888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fcfb9f0 msgr2=0x55cf7fdab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.338+0000 7f0821000700 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fdab080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.338+0000 7f0821000700 1 -- 172.21.15.63:0/3495761888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fd992c0 msgr2=0x55cf7fdab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.338+0000 7f0821000700 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fd992c0 0x55cf7fdab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.338+0000 7f0821000700 1 -- 172.21.15.63:0/3495761888 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7fe11560 con 0x55cf7fc0fe40 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.338+0000 7f0821000700 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fdaab40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0810006010 tx=0x7f0810005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946e7dbef24b7a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.338+0000 7f0802a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:45:59.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.339+0000 7f0802a00700 1 -- 172.21.15.63:0/3495761888 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08100056e0 con 0x55cf7fc0fe40 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.339+0000 7f0802a00700 1 -- 172.21.15.63:0/3495761888 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f08100039c0 con 0x55cf7fc0fe40 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.339+0000 7f08324dbb80 1 -- 172.21.15.63:0/3495761888 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f081400d040 con 0x55cf7fc0fe40 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.339+0000 7f0802a00700 1 -- 172.21.15.63:0/3495761888 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0810003bc0 con 0x55cf7fc0fe40 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.339+0000 7f08324dbb80 10 client.17844 fetch_fsmap learned FSMap version 33 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.339+0000 7f08324dbb80 1 -- 172.21.15.63:0/3495761888 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cf7fe12810 con 0x55cf7fc0fe40 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.340+0000 7f0802a00700 1 -- 172.21.15.63:0/3495761888 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f081400d040 con 0x55cf7fc0fe40 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.340+0000 7f0802a00700 1 -- 172.21.15.63:0/3495761888 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f081000f580 con 0x55cf7fc0fe40 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.340+0000 7f08324dbb80 10 client.17844 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.340+0000 7f08324dbb80 10 client.17844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.340+0000 7f08324dbb80 20 client.17844 populate_metadata read hostname 'smithi063' 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.340+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cf7fe12ec0 0x55cf7fe152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.340+0000 7f08324dbb80 4 client.17844 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:45:59.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.340+0000 7f08324dbb80 1 -- 172.21.15.63:0/3495761888 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf7fd89370 con 0x55cf7fe12ec0 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.341+0000 7f081be00700 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cf7fe12ec0 0x55cf7fe152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.342+0000 7f081be00700 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cf7fe12ec0 0x55cf7fe152e0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57e64bfa419e536 server_cookie=323ad6fea10717e3 in_seq=0 out_seq=0 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.342+0000 7f0802a00700 10 client.17844 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.342+0000 7f0802a00700 1 -- 172.21.15.63:0/3495761888 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cf7fd89370 con 0x55cf7fe12ec0 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.342+0000 7f0802a00700 10 client.17844 handle_command_reply: tid=0 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.342+0000 7f08324dbb80 1 client.17844 shutdown 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.342+0000 7f08324dbb80 1 -- 172.21.15.63:0/3495761888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 msgr2=0x55cf7fdaab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.342+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fdaab40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0810006010 tx=0x7f0810005a90 comp rx=0 tx=0).stop 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.343+0000 7f08324dbb80 1 -- 172.21.15.63:0/3495761888 shutdown_connections 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.343+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cf7fe12ec0 0x55cf7fe152e0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.343+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf7fd992c0 0x55cf7fdab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.343+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cf7fc0fe40 0x55cf7fdaab40 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.343+0000 7f08324dbb80 1 --2- 172.21.15.63:0/3495761888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cf7fcfb9f0 0x55cf7fdab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.343+0000 7f08324dbb80 1 -- 172.21.15.63:0/3495761888 >> 172.21.15.63:0/3495761888 conn(0x55cf7fca9390 msgr2=0x55cf7fd952f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:59.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.343+0000 7f0802a00700 0 client.17844 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:45:59.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.343+0000 7f08324dbb80 1 -- 172.21.15.63:0/3495761888 shutdown_connections 2024-05-27T07:45:59.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.343+0000 7f08324dbb80 1 -- 172.21.15.63:0/3495761888 wait complete. 2024-05-27T07:45:59.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.343+0000 7f08324dbb80 20 client.17844 trim_cache size 0 max 16384 2024-05-27T07:45:59.348 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:45:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:45:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:45:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "df3c052e-d455-4d8f-a422-2f158e74ff24": { 2024-05-27T07:45:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:45:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "df3c052e-d455-4d8f-a422-2f158e74ff24", 2024-05-27T07:45:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:45:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:45:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:45:59.349 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:45:59.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.344+0000 7f0822400700 1 -- 172.21.15.63:0/2599172765 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f08040b2a70 msgr2=0x7f08040b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.344+0000 7f0822400700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f08040b2a70 0x7f08040b4f80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0814009fd0 tx=0x7f08140097e0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.344+0000 7f0822400700 1 -- 172.21.15.63:0/2599172765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c105bd0 msgr2=0x7f081c19d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:59.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.344+0000 7f0822400700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c105bd0 0x7f081c19d2f0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f080800d950 tx=0x7f080800dd10 comp rx=0 tx=0).stop 2024-05-27T07:45:59.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.345+0000 7f0822400700 1 -- 172.21.15.63:0/2599172765 shutdown_connections 2024-05-27T07:45:59.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.345+0000 7f0822400700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f08040b2a70 0x7f08040b4f80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.345+0000 7f0822400700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f081c10e6f0 0x7f081c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.345+0000 7f0822400700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081c105280 0x7f081c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.345+0000 7f0822400700 1 --2- 172.21.15.63:0/2599172765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f081c105bd0 0x7f081c19d2f0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:59.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.345+0000 7f0822400700 1 -- 172.21.15.63:0/2599172765 >> 172.21.15.63:0/2599172765 conn(0x7f081c078ab0 msgr2=0x7f081c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:59.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.345+0000 7f0822400700 1 -- 172.21.15.63:0/2599172765 shutdown_connections 2024-05-27T07:45:59.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:45:59.345+0000 7f0822400700 1 -- 172.21.15.63:0/2599172765 wait complete. 2024-05-27T07:46:00.238 INFO:tasks.cephfs.filesystem:scrub status for tag:df3c052e-d455-4d8f-a422-2f158e74ff24 - {'path': '/', 'tag': 'df3c052e-d455-4d8f-a422-2f158e74ff24', 'options': 'recursive,force'} 2024-05-27T07:46:01.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:00 smithi086 ceph-mon[34687]: pgmap v388: 129 pgs: 129 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 34 op/s 2024-05-27T07:46:01.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:00 smithi149 ceph-mon[35408]: pgmap v388: 129 pgs: 129 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 34 op/s 2024-05-27T07:46:01.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:00 smithi063 ceph-mon[31613]: pgmap v388: 129 pgs: 129 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 34 op/s 2024-05-27T07:46:02.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:01 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:46:02.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:01 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:46:02.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:01 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:46:02.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:01 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:46:02.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:01 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:46:02.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:01 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:46:02.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:02 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:46:02.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:02 smithi063 ceph-mon[31613]: pgmap v389: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.7 MiB/s wr, 58 op/s 2024-05-27T07:46:03.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:02 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:46:03.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:02 smithi086 ceph-mon[34687]: pgmap v389: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.7 MiB/s wr, 58 op/s 2024-05-27T07:46:03.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:02 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:46:03.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:02 smithi149 ceph-mon[35408]: pgmap v389: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.7 MiB/s wr, 58 op/s 2024-05-27T07:46:05.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:04 smithi086 ceph-mon[34687]: pgmap v390: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:46:05.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:04 smithi149 ceph-mon[35408]: pgmap v390: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:46:05.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:04 smithi063 ceph-mon[31613]: pgmap v390: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:46:07.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:06 smithi086 ceph-mon[34687]: pgmap v391: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:46:07.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:06 smithi149 ceph-mon[35408]: pgmap v391: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:46:07.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:06 smithi063 ceph-mon[31613]: pgmap v391: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:46:08.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:07 smithi086 ceph-mon[34687]: osdmap e106: 12 total, 12 up, 12 in 2024-05-27T07:46:08.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:07 smithi149 ceph-mon[35408]: osdmap e106: 12 total, 12 up, 12 in 2024-05-27T07:46:08.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:07 smithi063 ceph-mon[31613]: osdmap e106: 12 total, 12 up, 12 in 2024-05-27T07:46:09.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:08 smithi086 ceph-mon[34687]: pgmap v393: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:46:09.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:08 smithi149 ceph-mon[35408]: pgmap v393: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:46:09.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:08 smithi063 ceph-mon[31613]: pgmap v393: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:46:11.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:10 smithi086 ceph-mon[34687]: pgmap v394: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:46:11.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:10 smithi149 ceph-mon[35408]: pgmap v394: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:46:11.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:10 smithi063 ceph-mon[31613]: pgmap v394: 129 pgs: 129 active+clean; 3.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 33 op/s 2024-05-27T07:46:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.288+0000 7fb177e00640 1 -- 172.21.15.63:0/2391816753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb17810e000 msgr2=0x7fb1781104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.288+0000 7fb177e00640 1 --2- 172.21.15.63:0/2391816753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb17810e000 0x7fb1781104f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb170009900 tx=0x7fb17002a410 comp rx=0 tx=0).stop 2024-05-27T07:46:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.288+0000 7fb177e00640 1 -- 172.21.15.63:0/2391816753 shutdown_connections 2024-05-27T07:46:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.288+0000 7fb177e00640 1 --2- 172.21.15.63:0/2391816753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb17810e000 0x7fb1781104f0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.288+0000 7fb177e00640 1 --2- 172.21.15.63:0/2391816753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb17810b610 0x7fb17810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.288+0000 7fb177e00640 1 --2- 172.21.15.63:0/2391816753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb178104bb0 0x7fb178104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.288+0000 7fb177e00640 1 -- 172.21.15.63:0/2391816753 >> 172.21.15.63:0/2391816753 conn(0x7fb1780fe210 msgr2=0x7fb178100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.289+0000 7fb177e00640 1 -- 172.21.15.63:0/2391816753 shutdown_connections 2024-05-27T07:46:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.289+0000 7fb177e00640 1 -- 172.21.15.63:0/2391816753 wait complete. 2024-05-27T07:46:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.290+0000 7fb177e00640 1 Processor -- start 2024-05-27T07:46:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.290+0000 7fb177e00640 1 -- start start 2024-05-27T07:46:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.290+0000 7fb177e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb178104bb0 0x7fb17819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.290+0000 7fb177e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb17810b610 0x7fb17819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb17810e000 0x7fb17819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb178113b60 con 0x7fb17810e000 2024-05-27T07:46:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1781139e0 con 0x7fb17810b610 2024-05-27T07:46:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb178113ce0 con 0x7fb178104bb0 2024-05-27T07:46:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb17810e000 0x7fb17819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb17810e000 0x7fb17819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50452/0 (socket says 172.21.15.63:50452) 2024-05-27T07:46:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177400640 1 -- 172.21.15.63:0/2035735852 learned_addr learned my addr 172.21.15.63:0/2035735852 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb176a00640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb178104bb0 0x7fb17819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb16e000640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb17810b610 0x7fb17819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177400640 1 -- 172.21.15.63:0/2035735852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb178104bb0 msgr2=0x7fb17819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177400640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb178104bb0 0x7fb17819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177400640 1 -- 172.21.15.63:0/2035735852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb17810b610 msgr2=0x7fb17819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177400640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb17810b610 0x7fb17819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.291+0000 7fb177400640 1 -- 172.21.15.63:0/2035735852 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17819f3c0 con 0x7fb17810e000 2024-05-27T07:46:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.292+0000 7fb176a00640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb178104bb0 0x7fb17819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.292+0000 7fb177400640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb17810e000 0x7fb17819ed80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb170004770 tx=0x7fb170003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.292+0000 7fb17ca00640 1 -- 172.21.15.63:0/2035735852 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17003f070 con 0x7fb17810e000 2024-05-27T07:46:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.292+0000 7fb17ca00640 1 -- 172.21.15.63:0/2035735852 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb17003baf0 con 0x7fb17810e000 2024-05-27T07:46:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.292+0000 7fb177e00640 1 -- 172.21.15.63:0/2035735852 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17819f650 con 0x7fb17810e000 2024-05-27T07:46:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.292+0000 7fb17ca00640 1 -- 172.21.15.63:0/2035735852 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170035030 con 0x7fb17810e000 2024-05-27T07:46:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.292+0000 7fb177e00640 1 -- 172.21.15.63:0/2035735852 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17819fb40 con 0x7fb17810e000 2024-05-27T07:46:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.296+0000 7fb184d94740 1 Processor -- start 2024-05-27T07:46:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.297+0000 7fb17ca00640 1 -- 172.21.15.63:0/2035735852 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb170042070 con 0x7fb17810e000 2024-05-27T07:46:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.297+0000 7fb184d94740 1 -- start start 2024-05-27T07:46:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.298+0000 7fb184d94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 0x557724160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.298+0000 7fb184d94740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x557724160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.298+0000 7fb184d94740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x5577241614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.298+0000 7fb184d94740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5577240d1b90 con 0x557723d952f0 2024-05-27T07:46:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.298+0000 7fb184d94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577240c1730 con 0x557723dbe1b0 2024-05-27T07:46:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.298+0000 7fb184d94740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5577240c5770 con 0x557723d9d4d0 2024-05-27T07:46:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.298+0000 7fb176a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x557724160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.298+0000 7fb176a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x557724160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50468/0 (socket says 172.21.15.63:50468) 2024-05-27T07:46:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.298+0000 7fb176a00640 1 -- 172.21.15.63:0/3096290067 learned_addr learned my addr 172.21.15.63:0/3096290067 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.298+0000 7fb16e000640 1 --2- 172.21.15.63:0/3096290067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x5577241614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb16e000640 1 --2- 172.21.15.63:0/3096290067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 0x557724160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb176a00640 1 -- 172.21.15.63:0/3096290067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 msgr2=0x5577241614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb176a00640 1 --2- 172.21.15.63:0/3096290067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x5577241614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb176a00640 1 -- 172.21.15.63:0/3096290067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 msgr2=0x557724160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb176a00640 1 --2- 172.21.15.63:0/3096290067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 0x557724160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb176a00640 1 -- 172.21.15.63:0/3096290067 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55772405a8b0 con 0x557723d952f0 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb16e000640 1 --2- 172.21.15.63:0/3096290067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 0x557724160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb176a00640 1 --2- 172.21.15.63:0/3096290067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x557724160fa0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb1680104a0 tx=0x7fb168010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da08dec91094b13d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb17ca00640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb15c0b2950 0x7fb15c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb167e00640 1 -- 172.21.15.63:0/3096290067 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168017070 con 0x557723d952f0 2024-05-27T07:46:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.299+0000 7fb17ca00640 1 -- 172.21.15.63:0/2035735852 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9571+0+0 (secure 0 0 0) 0x7fb1701032b0 con 0x7fb17810e000 2024-05-27T07:46:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.300+0000 7fb167e00640 1 -- 172.21.15.63:0/3096290067 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb168011b90 con 0x557723d952f0 2024-05-27T07:46:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.300+0000 7fb16e000640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb15c0b2950 0x7fb15c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.300+0000 7fb167e00640 1 -- 172.21.15.63:0/3096290067 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16800eca0 con 0x557723d952f0 2024-05-27T07:46:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.301+0000 7fb184d94740 1 -- 172.21.15.63:0/3096290067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 msgr2=0x557724160fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.301+0000 7fb184d94740 1 --2- 172.21.15.63:0/3096290067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x557724160fa0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb1680104a0 tx=0x7fb168010960 comp rx=0 tx=0).stop 2024-05-27T07:46:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.301+0000 7fb16e000640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb15c0b2950 0x7fb15c0b4e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55772415dbd0 tx=0x7fb160004730 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.301+0000 7fb184d94740 1 -- 172.21.15.63:0/3096290067 shutdown_connections 2024-05-27T07:46:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.301+0000 7fb184d94740 1 --2- 172.21.15.63:0/3096290067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x5577241614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.301+0000 7fb184d94740 1 --2- 172.21.15.63:0/3096290067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x557724160fa0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.301+0000 7fb184d94740 1 --2- 172.21.15.63:0/3096290067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 0x557724160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.301+0000 7fb184d94740 1 -- 172.21.15.63:0/3096290067 >> 172.21.15.63:0/3096290067 conn(0x557723db2300 msgr2=0x55772415d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.301+0000 7fb184d94740 1 -- 172.21.15.63:0/3096290067 shutdown_connections 2024-05-27T07:46:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.301+0000 7fb184d94740 1 -- 172.21.15.63:0/3096290067 wait complete. 2024-05-27T07:46:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.302+0000 7fb184d94740 1 -- start start 2024-05-27T07:46:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.302+0000 7fb184d94740 1 Processor -- start 2024-05-27T07:46:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.302+0000 7fb184d94740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x55772416ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.302+0000 7fb184d94740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x55772416b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb184d94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 0x55772416b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb184d94740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557723da7790 con 0x557723d952f0 2024-05-27T07:46:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb184d94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557724069bc0 con 0x557723dbe1b0 2024-05-27T07:46:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb184d94740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55772416dbe0 con 0x557723d9d4d0 2024-05-27T07:46:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb184d94740 10 client.0 _ll_register_callbacks cb 0x557724047400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb176a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x55772416ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb176a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x55772416ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50478/0 (socket says 172.21.15.63:50478) 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb176a00640 1 -- 172.21.15.63:0/897648290 learned_addr learned my addr 172.21.15.63:0/897648290 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb16e000640 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x55772416b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb177400640 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 0x55772416b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb176a00640 1 -- 172.21.15.63:0/897648290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 msgr2=0x55772416b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb176a00640 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x55772416b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb176a00640 1 -- 172.21.15.63:0/897648290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 msgr2=0x55772416b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb176a00640 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 0x55772416b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb176a00640 1 -- 172.21.15.63:0/897648290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557723dd6210 con 0x557723d952f0 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.303+0000 7fb177400640 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 0x55772416b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.304+0000 7fb176a00640 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x55772416ab50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb168009600 tx=0x7fb16800e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953b1abee1c7dcd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.304+0000 7fb164c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.304+0000 7fb164c00640 1 -- 172.21.15.63:0/897648290 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168017040 con 0x557723d952f0 2024-05-27T07:46:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.304+0000 7fb164c00640 1 -- 172.21.15.63:0/897648290 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9571+0+0 (secure 0 0 0) 0x7fb168011e00 con 0x557723d952f0 2024-05-27T07:46:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.304+0000 7fb184d94740 1 -- 172.21.15.63:0/897648290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb168011b90 con 0x557723d952f0 2024-05-27T07:46:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.304+0000 7fb164c00640 1 -- 172.21.15.63:0/897648290 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb168032050 con 0x557723d952f0 2024-05-27T07:46:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.304+0000 7fb184d94740 10 client.17856 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.304+0000 7fb184d94740 1 -- 172.21.15.63:0/897648290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557723ea2620 con 0x557723d952f0 2024-05-27T07:46:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.305+0000 7fb164c00640 1 -- 172.21.15.63:0/897648290 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb168011b90 con 0x557723d952f0 2024-05-27T07:46:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.305+0000 7fb164c00640 1 -- 172.21.15.63:0/897648290 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb168024680 con 0x557723d952f0 2024-05-27T07:46:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.305+0000 7fb184d94740 10 client.17856 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.305+0000 7fb184d94740 10 client.17856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.305+0000 7fb184d94740 20 client.17856 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.305+0000 7fb184d94740 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557723d7cd80 0x55772417cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.305+0000 7fb184d94740 4 client.17856 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.305+0000 7fb184d94740 1 -- 172.21.15.63:0/897648290 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577240d3ae0 con 0x557723d7cd80 2024-05-27T07:46:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.306+0000 7fb16e000640 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557723d7cd80 0x55772417cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.307+0000 7fb16e000640 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557723d7cd80 0x55772417cc80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594b5635dce5efcd server_cookie=42ec9dc4ed22738b in_seq=0 out_seq=0 2024-05-27T07:46:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.307+0000 7fb164c00640 10 client.17856 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb164c00640 1 -- 172.21.15.63:0/897648290 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5577240d3ae0 con 0x557723d7cd80 2024-05-27T07:46:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb164c00640 10 client.17856 handle_command_reply: tid=0 2024-05-27T07:46:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb184d94740 1 client.17856 shutdown 2024-05-27T07:46:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb184d94740 1 -- 172.21.15.63:0/897648290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 msgr2=0x55772416ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb184d94740 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x55772416ab50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb168009600 tx=0x7fb16800e430 comp rx=0 tx=0).stop 2024-05-27T07:46:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb184d94740 1 -- 172.21.15.63:0/897648290 shutdown_connections 2024-05-27T07:46:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb184d94740 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557723d7cd80 0x55772417cc80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb184d94740 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723dbe1b0 0x55772416b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb184d94740 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x55772416b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb184d94740 1 --2- 172.21.15.63:0/897648290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d952f0 0x55772416ab50 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb164c00640 0 client.17856 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.309+0000 7fb184d94740 1 -- 172.21.15.63:0/897648290 >> 172.21.15.63:0/897648290 conn(0x557723db2300 msgr2=0x557724160a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.310+0000 7fb184d94740 1 -- 172.21.15.63:0/897648290 shutdown_connections 2024-05-27T07:46:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.310+0000 7fb184d94740 1 -- 172.21.15.63:0/897648290 wait complete. 2024-05-27T07:46:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.310+0000 7fb184d94740 20 client.17856 trim_cache size 0 max 16384 2024-05-27T07:46:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.327+0000 7fb184d94740 1 Processor -- start 2024-05-27T07:46:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.327+0000 7fb184d94740 1 -- start start 2024-05-27T07:46:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.327+0000 7fb184d94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723d952f0 0x55772416d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.327+0000 7fb184d94740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d9d4d0 0x557724182220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.327+0000 7fb184d94740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723dbe1b0 0x557724182760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.327+0000 7fb184d94740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1781b39e0 con 0x557723d9d4d0 2024-05-27T07:46:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.327+0000 7fb184d94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1781b12b0 con 0x557723d952f0 2024-05-27T07:46:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.327+0000 7fb184d94740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb1781af120 con 0x557723dbe1b0 2024-05-27T07:46:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.328+0000 7fb16e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d9d4d0 0x557724182220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.328+0000 7fb16e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d9d4d0 0x557724182220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50488/0 (socket says 172.21.15.63:50488) 2024-05-27T07:46:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.328+0000 7fb16e000640 1 -- 172.21.15.63:0/105636431 learned_addr learned my addr 172.21.15.63:0/105636431 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.328+0000 7fb176a00640 1 --2- 172.21.15.63:0/105636431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723d952f0 0x55772416d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.328+0000 7fb177400640 1 --2- 172.21.15.63:0/105636431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723dbe1b0 0x557724182760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.328+0000 7fb16e000640 1 -- 172.21.15.63:0/105636431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723dbe1b0 msgr2=0x557724182760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.328+0000 7fb16e000640 1 --2- 172.21.15.63:0/105636431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723dbe1b0 0x557724182760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.328+0000 7fb16e000640 1 -- 172.21.15.63:0/105636431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723d952f0 msgr2=0x55772416d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.328+0000 7fb16e000640 1 --2- 172.21.15.63:0/105636431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723d952f0 0x55772416d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.328+0000 7fb16e000640 1 -- 172.21.15.63:0/105636431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55772415de00 con 0x557723d9d4d0 2024-05-27T07:46:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.329+0000 7fb16e000640 1 --2- 172.21.15.63:0/105636431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d9d4d0 0x557724182220 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x55772417d1c0 tx=0x7fb160016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23244ec33ccd34b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.329+0000 7fb167400640 1 -- 172.21.15.63:0/105636431 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16002a660 con 0x557723d9d4d0 2024-05-27T07:46:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.329+0000 7fb167400640 1 -- 172.21.15.63:0/105636431 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb160030070 con 0x557723d9d4d0 2024-05-27T07:46:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.329+0000 7fb167400640 1 -- 172.21.15.63:0/105636431 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160021c40 con 0x557723d9d4d0 2024-05-27T07:46:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.330+0000 7fb184d94740 1 -- 172.21.15.63:0/105636431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d9d4d0 msgr2=0x557724182220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.330+0000 7fb184d94740 1 --2- 172.21.15.63:0/105636431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d9d4d0 0x557724182220 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x55772417d1c0 tx=0x7fb160016000 comp rx=0 tx=0).stop 2024-05-27T07:46:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.330+0000 7fb184d94740 1 -- 172.21.15.63:0/105636431 shutdown_connections 2024-05-27T07:46:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.330+0000 7fb184d94740 1 --2- 172.21.15.63:0/105636431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723dbe1b0 0x557724182760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.330+0000 7fb184d94740 1 --2- 172.21.15.63:0/105636431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723d9d4d0 0x557724182220 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.330+0000 7fb184d94740 1 --2- 172.21.15.63:0/105636431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723d952f0 0x55772416d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.330+0000 7fb184d94740 1 -- 172.21.15.63:0/105636431 >> 172.21.15.63:0/105636431 conn(0x557723db2300 msgr2=0x55772416ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.330+0000 7fb184d94740 1 -- 172.21.15.63:0/105636431 shutdown_connections 2024-05-27T07:46:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.330+0000 7fb184d94740 1 -- 172.21.15.63:0/105636431 wait complete. 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.331+0000 7fb184d94740 1 -- start start 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.331+0000 7fb184d94740 1 Processor -- start 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb184d94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723d952f0 0x557724172f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb184d94740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x5577241734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb184d94740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723dbe1b0 0x557724170360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb184d94740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1781af770 con 0x557723dbe1b0 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb184d94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1781b0be0 con 0x557723d952f0 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb184d94740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb1781afe40 con 0x557723d9d4d0 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb184d94740 10 client.0 _ll_register_callbacks cb 0x7fb168009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb177400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723dbe1b0 0x557724170360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb177400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723dbe1b0 0x557724170360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50496/0 (socket says 172.21.15.63:50496) 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb177400640 1 -- 172.21.15.63:0/1938778065 learned_addr learned my addr 172.21.15.63:0/1938778065 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.332+0000 7fb176a00640 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723d952f0 0x557724172f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb16e000640 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x5577241734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb177400640 1 -- 172.21.15.63:0/1938778065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 msgr2=0x5577241734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb177400640 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x5577241734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb177400640 1 -- 172.21.15.63:0/1938778065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723d952f0 msgr2=0x557724172f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb177400640 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723d952f0 0x557724172f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb177400640 1 -- 172.21.15.63:0/1938778065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557724170a50 con 0x557723dbe1b0 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb16e000640 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x5577241734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb177400640 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723dbe1b0 0x557724170360 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fb170100520 tx=0x7fb170005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417ee9e6e061d9e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb164c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb164c00640 1 -- 172.21.15.63:0/1938778065 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170049070 con 0x557723dbe1b0 2024-05-27T07:46:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb164c00640 1 -- 172.21.15.63:0/1938778065 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9571+0+0 (secure 0 0 0) 0x7fb1700e46d0 con 0x557723dbe1b0 2024-05-27T07:46:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.333+0000 7fb184d94740 1 -- 172.21.15.63:0/1938778065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb160030070 con 0x557723dbe1b0 2024-05-27T07:46:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb164c00640 1 -- 172.21.15.63:0/1938778065 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb1700ca110 con 0x557723dbe1b0 2024-05-27T07:46:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb184d94740 10 client.17862 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb184d94740 1 -- 172.21.15.63:0/1938778065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5577241e8ec0 con 0x557723dbe1b0 2024-05-27T07:46:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb164c00640 1 -- 172.21.15.63:0/1938778065 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb160030070 con 0x557723dbe1b0 2024-05-27T07:46:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb164c00640 1 -- 172.21.15.63:0/1938778065 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb1700c63a0 con 0x557723dbe1b0 2024-05-27T07:46:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb184d94740 10 client.17862 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb184d94740 10 client.17862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb184d94740 20 client.17862 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb184d94740 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557723d521c0 0x5577241eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb184d94740 4 client.17862 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:46:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.334+0000 7fb184d94740 1 -- 172.21.15.63:0/1938778065 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577240cd2c0 con 0x557723d521c0 2024-05-27T07:46:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.335+0000 7fb176a00640 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557723d521c0 0x5577241eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.336+0000 7fb176a00640 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557723d521c0 0x5577241eb6d0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e07d0e60e2f511 server_cookie=e00a1a39ee5ab910 in_seq=0 out_seq=0 2024-05-27T07:46:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.336+0000 7fb164c00640 10 client.17862 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.374+0000 7fb164c00640 1 -- 172.21.15.63:0/1938778065 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5577240cd2c0 con 0x557723d521c0 2024-05-27T07:46:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.374+0000 7fb164c00640 10 client.17862 handle_command_reply: tid=0 2024-05-27T07:46:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.374+0000 7fb184d94740 1 client.17862 shutdown 2024-05-27T07:46:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.374+0000 7fb184d94740 1 -- 172.21.15.63:0/1938778065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723dbe1b0 msgr2=0x557724170360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.374+0000 7fb184d94740 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723dbe1b0 0x557724170360 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fb170100520 tx=0x7fb170005ea0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.375+0000 7fb184d94740 1 -- 172.21.15.63:0/1938778065 shutdown_connections 2024-05-27T07:46:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.375+0000 7fb184d94740 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557723d521c0 0x5577241eb6d0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.375+0000 7fb184d94740 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557723dbe1b0 0x557724170360 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.375+0000 7fb184d94740 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557723d9d4d0 0x5577241734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.375+0000 7fb184d94740 1 --2- 172.21.15.63:0/1938778065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557723d952f0 0x557724172f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.375+0000 7fb184d94740 1 -- 172.21.15.63:0/1938778065 >> 172.21.15.63:0/1938778065 conn(0x557723db2300 msgr2=0x557724175c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.375+0000 7fb164c00640 0 client.17862 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.375+0000 7fb184d94740 1 -- 172.21.15.63:0/1938778065 shutdown_connections 2024-05-27T07:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.375+0000 7fb184d94740 1 -- 172.21.15.63:0/1938778065 wait complete. 2024-05-27T07:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.375+0000 7fb184d94740 20 client.17862 trim_cache size 0 max 16384 2024-05-27T07:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:46:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:46:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:46:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 -- 172.21.15.63:0/2035735852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb15c0b2950 msgr2=0x7fb15c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb15c0b2950 0x7fb15c0b4e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55772415dbd0 tx=0x7fb160004730 comp rx=0 tx=0).stop 2024-05-27T07:46:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 -- 172.21.15.63:0/2035735852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb17810e000 msgr2=0x7fb17819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb17810e000 0x7fb17819ed80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb170004770 tx=0x7fb170003b90 comp rx=0 tx=0).stop 2024-05-27T07:46:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 -- 172.21.15.63:0/2035735852 shutdown_connections 2024-05-27T07:46:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb15c0b2950 0x7fb15c0b4e60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb17810e000 0x7fb17819ed80 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb17810b610 0x7fb17819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 --2- 172.21.15.63:0/2035735852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb178104bb0 0x7fb17819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 -- 172.21.15.63:0/2035735852 >> 172.21.15.63:0/2035735852 conn(0x7fb1780fe210 msgr2=0x7fb17810e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 -- 172.21.15.63:0/2035735852 shutdown_connections 2024-05-27T07:46:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:12.376+0000 7fb177e00640 1 -- 172.21.15.63:0/2035735852 wait complete. 2024-05-27T07:46:13.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:12 smithi086 ceph-mon[34687]: pgmap v395: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:46:13.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:12 smithi149 ceph-mon[35408]: pgmap v395: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:46:13.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:12 smithi063 ceph-mon[31613]: pgmap v395: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:46:15.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:14 smithi086 ceph-mon[34687]: pgmap v396: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:46:15.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:14 smithi149 ceph-mon[35408]: pgmap v396: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:46:15.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:14 smithi063 ceph-mon[31613]: pgmap v396: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:46:16.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:16 smithi086 ceph-mon[34687]: osdmap e107: 12 total, 12 up, 12 in 2024-05-27T07:46:16.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:16 smithi149 ceph-mon[35408]: osdmap e107: 12 total, 12 up, 12 in 2024-05-27T07:46:16.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:16 smithi063 ceph-mon[31613]: osdmap e107: 12 total, 12 up, 12 in 2024-05-27T07:46:17.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:17 smithi086 ceph-mon[34687]: pgmap v398: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-05-27T07:46:17.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:17 smithi149 ceph-mon[35408]: pgmap v398: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-05-27T07:46:17.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:17 smithi063 ceph-mon[31613]: pgmap v398: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-05-27T07:46:18.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:18 smithi086 ceph-mon[34687]: pgmap v399: 129 pgs: 129 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:18.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:18 smithi149 ceph-mon[35408]: pgmap v399: 129 pgs: 129 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:18.964 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:18 smithi063 ceph-mon[31613]: pgmap v399: 129 pgs: 129 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:21.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:20 smithi086 ceph-mon[34687]: pgmap v400: 129 pgs: 129 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:21.211 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:20 smithi149 ceph-mon[35408]: pgmap v400: 129 pgs: 129 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:21.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:20 smithi063 ceph-mon[31613]: pgmap v400: 129 pgs: 129 active+clean; 3.2 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:23.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:22 smithi086 ceph-mon[34687]: pgmap v401: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:23.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:22 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:23.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:22 smithi149 ceph-mon[35408]: pgmap v401: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:23.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:22 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:23.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:22 smithi063 ceph-mon[31613]: pgmap v401: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:23.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:22 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:24.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:23 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:24.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:23 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:24.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:23 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:46:24.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:23 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:24.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:23 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:24.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:23 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:46:24.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:23 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:24.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:23 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:24.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:23 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:46:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.625+0000 7f9183200640 1 -- 172.21.15.63:0/1682393593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f917c108be0 msgr2=0x7f917c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.625+0000 7f9183200640 1 --2- 172.21.15.63:0/1682393593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f917c108be0 0x7f917c101d50 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f916c009900 tx=0x7f916c02a410 comp rx=0 tx=0).stop 2024-05-27T07:46:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.625+0000 7f9183200640 1 -- 172.21.15.63:0/1682393593 shutdown_connections 2024-05-27T07:46:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.625+0000 7f9183200640 1 --2- 172.21.15.63:0/1682393593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917c10e630 0x7f917c110a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.625+0000 7f9183200640 1 --2- 172.21.15.63:0/1682393593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f917c102290 0x7f917c1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.625+0000 7f9183200640 1 --2- 172.21.15.63:0/1682393593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f917c108be0 0x7f917c101d50 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.625+0000 7f9183200640 1 -- 172.21.15.63:0/1682393593 >> 172.21.15.63:0/1682393593 conn(0x7f917c0fd9c0 msgr2=0x7f917c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.626+0000 7f9183200640 1 -- 172.21.15.63:0/1682393593 shutdown_connections 2024-05-27T07:46:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.626+0000 7f9183200640 1 -- 172.21.15.63:0/1682393593 wait complete. 2024-05-27T07:46:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.627+0000 7f9183200640 1 Processor -- start 2024-05-27T07:46:25.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.627+0000 7f9183200640 1 -- start start 2024-05-27T07:46:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.627+0000 7f9183200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f917c102290 0x7f917c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.627+0000 7f9183200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917c108be0 0x7f917c19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.627+0000 7f9183200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f917c10e630 0x7f917c1a8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.627+0000 7f9183200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f917c113de0 con 0x7f917c102290 2024-05-27T07:46:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.627+0000 7f9183200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f917c113c60 con 0x7f917c108be0 2024-05-27T07:46:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.627+0000 7f9183200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f917c113f60 con 0x7f917c10e630 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f917be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f917c102290 0x7f917c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f917be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f917c102290 0x7f917c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47122/0 (socket says 172.21.15.63:47122) 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f917be00640 1 -- 172.21.15.63:0/781668297 learned_addr learned my addr 172.21.15.63:0/781668297 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f917b400640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917c108be0 0x7f917c19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f9181200640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f917c10e630 0x7f917c1a8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f917be00640 1 -- 172.21.15.63:0/781668297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f917c10e630 msgr2=0x7f917c1a8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f917be00640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f917c10e630 0x7f917c1a8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f917be00640 1 -- 172.21.15.63:0/781668297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917c108be0 msgr2=0x7f917c19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f917be00640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917c108be0 0x7f917c19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f917be00640 1 -- 172.21.15.63:0/781668297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917c1a8c60 con 0x7f917c102290 2024-05-27T07:46:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.628+0000 7f9181200640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f917c10e630 0x7f917c1a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.629+0000 7f917be00640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f917c102290 0x7f917c19d6a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f916c005940 tx=0x7f916c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.629+0000 7f9178c00640 1 -- 172.21.15.63:0/781668297 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c03f030 con 0x7f917c102290 2024-05-27T07:46:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.629+0000 7f9178c00640 1 -- 172.21.15.63:0/781668297 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f916c003c90 con 0x7f917c102290 2024-05-27T07:46:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.629+0000 7f9183200640 1 -- 172.21.15.63:0/781668297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917c1a8e30 con 0x7f917c102290 2024-05-27T07:46:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.629+0000 7f9178c00640 1 -- 172.21.15.63:0/781668297 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c035070 con 0x7f917c102290 2024-05-27T07:46:25.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.629+0000 7f9183200640 1 -- 172.21.15.63:0/781668297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f917c1a9290 con 0x7f917c102290 2024-05-27T07:46:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.634+0000 7f918a19a740 1 Processor -- start 2024-05-27T07:46:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.634+0000 7f9178c00640 1 -- 172.21.15.63:0/781668297 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f916c042030 con 0x7f917c102290 2024-05-27T07:46:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.634+0000 7f918a19a740 1 -- start start 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.634+0000 7f918a19a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b653b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.634+0000 7f918a19a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b653bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.634+0000 7f918a19a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 0x5601b653c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.634+0000 7f918a19a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601b64acb90 con 0x5601b61991b0 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.634+0000 7f918a19a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5601b649c730 con 0x5601b61784d0 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.634+0000 7f918a19a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5601b64a0770 con 0x5601b61702f0 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.635+0000 7f9181200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b653b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.635+0000 7f9181200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b653b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47138/0 (socket says 172.21.15.63:47138) 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.635+0000 7f9181200640 1 -- 172.21.15.63:0/426931258 learned_addr learned my addr 172.21.15.63:0/426931258 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.635+0000 7f917b400640 1 --2- 172.21.15.63:0/426931258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 0x5601b653c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.635+0000 7f917be00640 1 --2- 172.21.15.63:0/426931258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b653bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.635+0000 7f9181200640 1 -- 172.21.15.63:0/426931258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 msgr2=0x5601b653bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.635+0000 7f9181200640 1 --2- 172.21.15.63:0/426931258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b653bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.635+0000 7f9181200640 1 -- 172.21.15.63:0/426931258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 msgr2=0x5601b653c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.635+0000 7f9181200640 1 --2- 172.21.15.63:0/426931258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 0x5601b653c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.635+0000 7f9181200640 1 -- 172.21.15.63:0/426931258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b64358b0 con 0x5601b61991b0 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.636+0000 7f917b400640 1 --2- 172.21.15.63:0/426931258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 0x5601b653c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.636+0000 7f9178c00640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f91600b2950 0x7f91600b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.636+0000 7f9181200640 1 --2- 172.21.15.63:0/426931258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b653b860 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9174009e40 tx=0x7f917400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885ece2beeea7425 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.636+0000 7f9178c00640 1 -- 172.21.15.63:0/781668297 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f916c102fe0 con 0x7f917c102290 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.636+0000 7f917be00640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f91600b2950 0x7f91600b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.636+0000 7f9171600640 1 -- 172.21.15.63:0/426931258 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174004d20 con 0x5601b61991b0 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.636+0000 7f9171600640 1 -- 172.21.15.63:0/426931258 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91740054b0 con 0x5601b61991b0 2024-05-27T07:46:25.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.636+0000 7f9171600640 1 -- 172.21.15.63:0/426931258 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174011700 con 0x5601b61991b0 2024-05-27T07:46:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.637+0000 7f918a19a740 1 -- 172.21.15.63:0/426931258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 msgr2=0x5601b653b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.637+0000 7f917be00640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f91600b2950 0x7f91600b4e60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x5601b6539350 tx=0x7f916c009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.637+0000 7f918a19a740 1 --2- 172.21.15.63:0/426931258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b653b860 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9174009e40 tx=0x7f917400f730 comp rx=0 tx=0).stop 2024-05-27T07:46:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.638+0000 7f918a19a740 1 -- 172.21.15.63:0/426931258 shutdown_connections 2024-05-27T07:46:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.638+0000 7f918a19a740 1 --2- 172.21.15.63:0/426931258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 0x5601b653c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.638+0000 7f918a19a740 1 --2- 172.21.15.63:0/426931258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b653bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.638+0000 7f918a19a740 1 --2- 172.21.15.63:0/426931258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b653b860 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.638+0000 7f918a19a740 1 -- 172.21.15.63:0/426931258 >> 172.21.15.63:0/426931258 conn(0x5601b618d300 msgr2=0x5601b6538010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.638+0000 7f918a19a740 1 -- 172.21.15.63:0/426931258 shutdown_connections 2024-05-27T07:46:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.638+0000 7f918a19a740 1 -- 172.21.15.63:0/426931258 wait complete. 2024-05-27T07:46:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.638+0000 7f918a19a740 1 -- start start 2024-05-27T07:46:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.639+0000 7f918a19a740 1 Processor -- start 2024-05-27T07:46:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.639+0000 7f918a19a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b6545a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.639+0000 7f918a19a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61784d0 0x5601b6545f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.639+0000 7f918a19a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61991b0 0x5601b6551d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.639+0000 7f918a19a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601b6182790 con 0x5601b61784d0 2024-05-27T07:46:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.639+0000 7f918a19a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5601b6444bc0 con 0x5601b61991b0 2024-05-27T07:46:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.639+0000 7f918a19a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5601b6548990 con 0x5601b61702f0 2024-05-27T07:46:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.639+0000 7f918a19a740 10 client.0 _ll_register_callbacks cb 0x5601b6422400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f917b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61784d0 0x5601b6545f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f917b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61784d0 0x5601b6545f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47146/0 (socket says 172.21.15.63:47146) 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f917b400640 1 -- 172.21.15.63:0/526155426 learned_addr learned my addr 172.21.15.63:0/526155426 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f9181200640 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b6545a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f9181200640 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61991b0 0x5601b6551d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f917b400640 1 -- 172.21.15.63:0/526155426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 msgr2=0x5601b6545a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f917b400640 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b6545a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f917b400640 1 -- 172.21.15.63:0/526155426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61991b0 msgr2=0x5601b6551d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f917b400640 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61991b0 0x5601b6551d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f917b400640 1 -- 172.21.15.63:0/526155426 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b61b1210 con 0x5601b61784d0 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.640+0000 7f9181200640 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61991b0 0x5601b6551d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.641+0000 7f917b400640 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61784d0 0x5601b6545f70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f916800be90 tx=0x7f91680117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c766f8f3d52bede9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.641+0000 7f9166000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.641+0000 7f9166000640 1 -- 172.21.15.63:0/526155426 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168012650 con 0x5601b61784d0 2024-05-27T07:46:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.641+0000 7f918a19a740 1 -- 172.21.15.63:0/526155426 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91740054b0 con 0x5601b61784d0 2024-05-27T07:46:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.641+0000 7f9166000640 1 -- 172.21.15.63:0/526155426 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f91680127f0 con 0x5601b61784d0 2024-05-27T07:46:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.641+0000 7f9166000640 1 -- 172.21.15.63:0/526155426 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9168012a30 con 0x5601b61784d0 2024-05-27T07:46:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.641+0000 7f918a19a740 10 client.17874 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.641+0000 7f918a19a740 1 -- 172.21.15.63:0/526155426 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5601b627d620 con 0x5601b61784d0 2024-05-27T07:46:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.642+0000 7f9166000640 1 -- 172.21.15.63:0/526155426 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91740054b0 con 0x5601b61784d0 2024-05-27T07:46:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.642+0000 7f9166000640 1 -- 172.21.15.63:0/526155426 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f916801bc30 con 0x5601b61784d0 2024-05-27T07:46:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.642+0000 7f918a19a740 10 client.17874 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.642+0000 7f918a19a740 10 client.17874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.642+0000 7f918a19a740 20 client.17874 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.642+0000 7f918a19a740 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601b6157d80 0x5601b6557dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.642+0000 7f918a19a740 4 client.17874 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.642+0000 7f918a19a740 1 -- 172.21.15.63:0/526155426 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601b64aeae0 con 0x5601b6157d80 2024-05-27T07:46:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.643+0000 7f9181200640 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601b6157d80 0x5601b6557dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.644+0000 7f9181200640 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601b6157d80 0x5601b6557dc0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e3fa981d22e21a server_cookie=fe84a40af24ce363 in_seq=0 out_seq=0 2024-05-27T07:46:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.644+0000 7f9166000640 10 client.17874 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f9166000640 1 -- 172.21.15.63:0/526155426 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5601b64aeae0 con 0x5601b6157d80 2024-05-27T07:46:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f9166000640 10 client.17874 handle_command_reply: tid=0 2024-05-27T07:46:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 client.17874 shutdown 2024-05-27T07:46:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 -- 172.21.15.63:0/526155426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61784d0 msgr2=0x5601b6545f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61784d0 0x5601b6545f70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f916800be90 tx=0x7f91680117b0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 -- 172.21.15.63:0/526155426 shutdown_connections 2024-05-27T07:46:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601b6157d80 0x5601b6557dc0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61991b0 0x5601b6551d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61784d0 0x5601b6545f70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f9166000640 0 client.17874 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 --2- 172.21.15.63:0/526155426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b6545a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 -- 172.21.15.63:0/526155426 >> 172.21.15.63:0/526155426 conn(0x5601b618d300 msgr2=0x5601b653b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 -- 172.21.15.63:0/526155426 shutdown_connections 2024-05-27T07:46:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.646+0000 7f918a19a740 1 -- 172.21.15.63:0/526155426 wait complete. 2024-05-27T07:46:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.647+0000 7f918a19a740 20 client.17874 trim_cache size 0 max 16384 2024-05-27T07:46:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.669+0000 7f918a19a740 1 Processor -- start 2024-05-27T07:46:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.670+0000 7f918a19a740 1 -- start start 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.670+0000 7f918a19a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b6548160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.670+0000 7f918a19a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 0x5601b6537330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.670+0000 7f918a19a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b6551a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.670+0000 7f918a19a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f917c1b5cb0 con 0x5601b61991b0 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.670+0000 7f918a19a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f917c1b3580 con 0x5601b61784d0 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.670+0000 7f918a19a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f917c1b13f0 con 0x5601b61702f0 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.670+0000 7f9181200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b6551a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.670+0000 7f9181200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b6551a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47162/0 (socket says 172.21.15.63:47162) 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.670+0000 7f9181200640 1 -- 172.21.15.63:0/3074744841 learned_addr learned my addr 172.21.15.63:0/3074744841 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f9181200640 1 --2- 172.21.15.63:0/3074744841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b6548160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f917b400640 1 --2- 172.21.15.63:0/3074744841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 0x5601b6537330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f9181200640 1 -- 172.21.15.63:0/3074744841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 msgr2=0x5601b6548160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f9181200640 1 --2- 172.21.15.63:0/3074744841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b6548160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f9181200640 1 -- 172.21.15.63:0/3074744841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 msgr2=0x5601b6537330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f9181200640 1 --2- 172.21.15.63:0/3074744841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 0x5601b6537330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f9181200640 1 -- 172.21.15.63:0/3074744841 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b6532050 con 0x5601b61991b0 2024-05-27T07:46:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f917b400640 1 --2- 172.21.15.63:0/3074744841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 0x5601b6537330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f9181200640 1 --2- 172.21.15.63:0/3074744841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b6551a80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f91740092f0 tx=0x7f917400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a163f7c7ceb368e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f9170c00640 1 -- 172.21.15.63:0/3074744841 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174004c00 con 0x5601b61991b0 2024-05-27T07:46:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f9170c00640 1 -- 172.21.15.63:0/3074744841 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9174004da0 con 0x5601b61991b0 2024-05-27T07:46:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.671+0000 7f9170c00640 1 -- 172.21.15.63:0/3074744841 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174011730 con 0x5601b61991b0 2024-05-27T07:46:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.672+0000 7f918a19a740 1 -- 172.21.15.63:0/3074744841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 msgr2=0x5601b6551a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.672+0000 7f918a19a740 1 --2- 172.21.15.63:0/3074744841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b6551a80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f91740092f0 tx=0x7f917400f730 comp rx=0 tx=0).stop 2024-05-27T07:46:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.672+0000 7f918a19a740 1 -- 172.21.15.63:0/3074744841 shutdown_connections 2024-05-27T07:46:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.672+0000 7f918a19a740 1 --2- 172.21.15.63:0/3074744841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61991b0 0x5601b6551a80 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.672+0000 7f918a19a740 1 --2- 172.21.15.63:0/3074744841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61784d0 0x5601b6537330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.672+0000 7f918a19a740 1 --2- 172.21.15.63:0/3074744841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61702f0 0x5601b6548160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.672+0000 7f918a19a740 1 -- 172.21.15.63:0/3074744841 >> 172.21.15.63:0/3074744841 conn(0x5601b618d300 msgr2=0x5601b6545cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.673+0000 7f918a19a740 1 -- 172.21.15.63:0/3074744841 shutdown_connections 2024-05-27T07:46:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.673+0000 7f918a19a740 1 -- 172.21.15.63:0/3074744841 wait complete. 2024-05-27T07:46:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.673+0000 7f918a19a740 1 -- start start 2024-05-27T07:46:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.674+0000 7f918a19a740 1 Processor -- start 2024-05-27T07:46:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.674+0000 7f918a19a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61702f0 0x5601b6549250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.674+0000 7f918a19a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61784d0 0x5601b6549790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.674+0000 7f918a19a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61991b0 0x5601b65b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.674+0000 7f918a19a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f917c1b1a40 con 0x5601b61702f0 2024-05-27T07:46:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.674+0000 7f918a19a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f917c1b2eb0 con 0x5601b61991b0 2024-05-27T07:46:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.674+0000 7f918a19a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f917c1b2110 con 0x5601b61784d0 2024-05-27T07:46:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.674+0000 7f918a19a740 10 client.0 _ll_register_callbacks cb 0x7f9168005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.675+0000 7f9181200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61702f0 0x5601b6549250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.675+0000 7f9181200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61702f0 0x5601b6549250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47178/0 (socket says 172.21.15.63:47178) 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.675+0000 7f9181200640 1 -- 172.21.15.63:0/1589375354 learned_addr learned my addr 172.21.15.63:0/1589375354 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.675+0000 7f917b400640 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61784d0 0x5601b6549790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.675+0000 7f9181200640 1 -- 172.21.15.63:0/1589375354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61784d0 msgr2=0x5601b6549790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.675+0000 7f9181200640 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61784d0 0x5601b6549790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.675+0000 7f9181200640 1 -- 172.21.15.63:0/1589375354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61991b0 msgr2=0x5601b65b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.675+0000 7f9181200640 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61991b0 0x5601b65b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.675+0000 7f9181200640 1 -- 172.21.15.63:0/1589375354 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b65b8910 con 0x5601b61702f0 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.676+0000 7f9181200640 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61702f0 0x5601b6549250 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f9174009fd0 tx=0x7f9174004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187f77584eeb23dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.676+0000 7f9166000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.676+0000 7f9166000640 1 -- 172.21.15.63:0/1589375354 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174011a80 con 0x5601b61702f0 2024-05-27T07:46:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.676+0000 7f9166000640 1 -- 172.21.15.63:0/1589375354 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f9174011c20 con 0x5601b61702f0 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.676+0000 7f918a19a740 1 -- 172.21.15.63:0/1589375354 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9174004da0 con 0x5601b61702f0 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.676+0000 7f9166000640 1 -- 172.21.15.63:0/1589375354 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9174043050 con 0x5601b61702f0 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.676+0000 7f918a19a740 10 client.17880 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.676+0000 7f918a19a740 1 -- 172.21.15.63:0/1589375354 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5601b65bfb80 con 0x5601b61702f0 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.677+0000 7f9166000640 1 -- 172.21.15.63:0/1589375354 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9174004da0 con 0x5601b61702f0 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.677+0000 7f9166000640 1 -- 172.21.15.63:0/1589375354 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9174032460 con 0x5601b61702f0 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.677+0000 7f918a19a740 10 client.17880 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.677+0000 7f918a19a740 10 client.17880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.677+0000 7f918a19a740 20 client.17880 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.677+0000 7f918a19a740 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601b612d1c0 0x5601b65c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.677+0000 7f918a19a740 4 client.17880 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:46:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.677+0000 7f918a19a740 1 -- 172.21.15.63:0/1589375354 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601b64a82c0 con 0x5601b612d1c0 2024-05-27T07:46:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.678+0000 7f917b400640 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601b612d1c0 0x5601b65c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.679+0000 7f917b400640 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601b612d1c0 0x5601b65c2100 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75deddd91df3ece0 server_cookie=e164eeaecd540346 in_seq=0 out_seq=0 2024-05-27T07:46:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.679+0000 7f9166000640 10 client.17880 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:25.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:25 smithi086 ceph-mon[34687]: pgmap v402: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.710+0000 7f9166000640 1 -- 172.21.15.63:0/1589375354 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5601b64a82c0 con 0x5601b612d1c0 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.710+0000 7f9166000640 10 client.17880 handle_command_reply: tid=0 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.710+0000 7f918a19a740 1 client.17880 shutdown 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f918a19a740 1 -- 172.21.15.63:0/1589375354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61702f0 msgr2=0x5601b6549250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f918a19a740 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61702f0 0x5601b6549250 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f9174009fd0 tx=0x7f9174004cb0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f918a19a740 1 -- 172.21.15.63:0/1589375354 shutdown_connections 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f918a19a740 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601b612d1c0 0x5601b65c2100 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f918a19a740 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601b61991b0 0x5601b65b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f918a19a740 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601b61784d0 0x5601b6549790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f918a19a740 1 --2- 172.21.15.63:0/1589375354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601b61702f0 0x5601b6549250 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f918a19a740 1 -- 172.21.15.63:0/1589375354 >> 172.21.15.63:0/1589375354 conn(0x5601b618d300 msgr2=0x5601b6545a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f9166000640 0 client.17880 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f918a19a740 1 -- 172.21.15.63:0/1589375354 shutdown_connections 2024-05-27T07:46:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.711+0000 7f918a19a740 1 -- 172.21.15.63:0/1589375354 wait complete. 2024-05-27T07:46:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.712+0000 7f918a19a740 20 client.17880 trim_cache size 0 max 16384 2024-05-27T07:46:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:46:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:46:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:46:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.712+0000 7f9183200640 1 -- 172.21.15.63:0/781668297 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f91600b2950 msgr2=0x7f91600b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.712+0000 7f9183200640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f91600b2950 0x7f91600b4e60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x5601b6539350 tx=0x7f916c009260 comp rx=0 tx=0).stop 2024-05-27T07:46:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.713+0000 7f9183200640 1 -- 172.21.15.63:0/781668297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f917c102290 msgr2=0x7f917c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.713+0000 7f9183200640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f917c102290 0x7f917c19d6a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f916c005940 tx=0x7f916c0042f0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.713+0000 7f9183200640 1 -- 172.21.15.63:0/781668297 shutdown_connections 2024-05-27T07:46:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.713+0000 7f9183200640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f91600b2950 0x7f91600b4e60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.713+0000 7f9183200640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f917c10e630 0x7f917c1a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.713+0000 7f9183200640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f917c108be0 0x7f917c19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.713+0000 7f9183200640 1 --2- 172.21.15.63:0/781668297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f917c102290 0x7f917c19d6a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.713+0000 7f9183200640 1 -- 172.21.15.63:0/781668297 >> 172.21.15.63:0/781668297 conn(0x7f917c0fd9c0 msgr2=0x7f917c110060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.713+0000 7f9183200640 1 -- 172.21.15.63:0/781668297 shutdown_connections 2024-05-27T07:46:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:25.713+0000 7f9183200640 1 -- 172.21.15.63:0/781668297 wait complete. 2024-05-27T07:46:25.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:25 smithi063 ceph-mon[31613]: pgmap v402: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:25.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:25 smithi149 ceph-mon[35408]: pgmap v402: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:46:27.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:27 smithi086 ceph-mon[34687]: pgmap v403: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:46:27.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:27 smithi149 ceph-mon[35408]: pgmap v403: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:46:27.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:27 smithi063 ceph-mon[31613]: pgmap v403: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:46:29.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:29 smithi149 ceph-mon[35408]: pgmap v404: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 16 op/s 2024-05-27T07:46:29.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:29 smithi086 ceph-mon[34687]: pgmap v404: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 16 op/s 2024-05-27T07:46:29.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:29 smithi063 ceph-mon[31613]: pgmap v404: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 16 op/s 2024-05-27T07:46:30.239 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:46:30.730 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:31.637 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:31 smithi063 ceph-mon[31613]: pgmap v405: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:46:31.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:31 smithi086 ceph-mon[34687]: pgmap v405: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:46:31.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:31 smithi149 ceph-mon[35408]: pgmap v405: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:46:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.239+0000 7f442fc00700 1 -- 172.21.15.63:0/1535611125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44281054f0 msgr2=0x7f4428105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.239+0000 7f442fc00700 1 --2- 172.21.15.63:0/1535611125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44281054f0 0x7f4428105900 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f441c009990 tx=0x7f441c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.239+0000 7f442fc00700 1 -- 172.21.15.63:0/1535611125 shutdown_connections 2024-05-27T07:46:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.239+0000 7f442fc00700 1 --2- 172.21.15.63:0/1535611125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f442810e8d0 0x7f4428110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.239+0000 7f442fc00700 1 --2- 172.21.15.63:0/1535611125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44281054f0 0x7f4428105900 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.239+0000 7f442fc00700 1 --2- 172.21.15.63:0/1535611125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4428105e40 0x7f442810e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.239+0000 7f442fc00700 1 -- 172.21.15.63:0/1535611125 >> 172.21.15.63:0/1535611125 conn(0x7f44280fedb0 msgr2=0x7f44281011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.239+0000 7f442fc00700 1 -- 172.21.15.63:0/1535611125 shutdown_connections 2024-05-27T07:46:33.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.239+0000 7f442fc00700 1 -- 172.21.15.63:0/1535611125 wait complete. 2024-05-27T07:46:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.240+0000 7f442fc00700 1 Processor -- start 2024-05-27T07:46:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.241+0000 7f442fc00700 1 -- start start 2024-05-27T07:46:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.241+0000 7f442fc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44281054f0 0x7f44281031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.241+0000 7f442fc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4428105e40 0x7f4428103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.241+0000 7f442fc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442810e8d0 0x7f4428103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.241+0000 7f442fc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f44281142d0 con 0x7f442810e8d0 2024-05-27T07:46:33.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.241+0000 7f442fc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4428114150 con 0x7f44281054f0 2024-05-27T07:46:33.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.241+0000 7f442fc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4428114450 con 0x7f4428105e40 2024-05-27T07:46:33.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.242+0000 7f442dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442810e8d0 0x7f4428103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.242+0000 7f442dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442810e8d0 0x7f4428103c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41354/0 (socket says 172.21.15.63:41354) 2024-05-27T07:46:33.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.242+0000 7f442dc00700 1 -- 172.21.15.63:0/774579746 learned_addr learned my addr 172.21.15.63:0/774579746 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:33.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.242+0000 7f442d200700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44281054f0 0x7f44281031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.242+0000 7f4427e00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4428105e40 0x7f4428103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.242+0000 7f442dc00700 1 -- 172.21.15.63:0/774579746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4428105e40 msgr2=0x7f4428103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.242+0000 7f442dc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4428105e40 0x7f4428103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.242+0000 7f442dc00700 1 -- 172.21.15.63:0/774579746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44281054f0 msgr2=0x7f44281031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.242+0000 7f442dc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44281054f0 0x7f44281031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.242+0000 7f442dc00700 1 -- 172.21.15.63:0/774579746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44281a9040 con 0x7f442810e8d0 2024-05-27T07:46:33.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.243+0000 7f442dc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442810e8d0 0x7f4428103c60 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4420007ee0 tx=0x7f442000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:33.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.243+0000 7f4425600700 1 -- 172.21.15.63:0/774579746 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442000dbd0 con 0x7f442810e8d0 2024-05-27T07:46:33.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.243+0000 7f4425600700 1 -- 172.21.15.63:0/774579746 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f442000dd70 con 0x7f442810e8d0 2024-05-27T07:46:33.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.243+0000 7f442fc00700 1 -- 172.21.15.63:0/774579746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44281a9330 con 0x7f442810e8d0 2024-05-27T07:46:33.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.243+0000 7f4425600700 1 -- 172.21.15.63:0/774579746 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44200058d0 con 0x7f442810e8d0 2024-05-27T07:46:33.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.243+0000 7f442fc00700 1 -- 172.21.15.63:0/774579746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44281a9870 con 0x7f442810e8d0 2024-05-27T07:46:33.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.248+0000 7f4425600700 1 -- 172.21.15.63:0/774579746 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f442001a020 con 0x7f442810e8d0 2024-05-27T07:46:33.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.249+0000 7f443e83bb80 1 Processor -- start 2024-05-27T07:46:33.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.250+0000 7f443e83bb80 1 -- start start 2024-05-27T07:46:33.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.250+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f4a09f0 0x55fe9f533330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.250+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f3b4e40 0x55fe9f533870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.250+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f540110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.250+0000 7f443e83bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fe9f461e40 con 0x55fe9f533e70 2024-05-27T07:46:33.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.250+0000 7f443e83bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fe9f4519e0 con 0x55fe9f3b4e40 2024-05-27T07:46:33.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.250+0000 7f443e83bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fe9f455a20 con 0x55fe9f4a09f0 2024-05-27T07:46:33.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.250+0000 7f4425600700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f44100b2a70 0x7f44100b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.251+0000 7f4425600700 1 -- 172.21.15.63:0/774579746 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f44200e2110 con 0x7f442810e8d0 2024-05-27T07:46:33.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.251+0000 7f4427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f540110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.251+0000 7f442d200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f3b4e40 0x55fe9f533870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.251+0000 7f4427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f540110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41370/0 (socket says 172.21.15.63:41370) 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.251+0000 7f4427e00700 1 -- 172.21.15.63:0/2954236052 learned_addr learned my addr 172.21.15.63:0/2954236052 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.251+0000 7f4427e00700 1 --2- 172.21.15.63:0/2954236052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f4a09f0 0x55fe9f533330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.251+0000 7f442dc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f44100b2a70 0x7f44100b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.252+0000 7f4427e00700 1 -- 172.21.15.63:0/2954236052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f4a09f0 msgr2=0x55fe9f533330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.252+0000 7f4427e00700 1 --2- 172.21.15.63:0/2954236052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f4a09f0 0x55fe9f533330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.252+0000 7f4427e00700 1 -- 172.21.15.63:0/2954236052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f3b4e40 msgr2=0x55fe9f533870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.252+0000 7f4427e00700 1 --2- 172.21.15.63:0/2954236052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f3b4e40 0x55fe9f533870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.252+0000 7f4427e00700 1 -- 172.21.15.63:0/2954236052 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe9f41a420 con 0x55fe9f533e70 2024-05-27T07:46:33.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.252+0000 7f4427e00700 1 --2- 172.21.15.63:0/2954236052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f540110 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f44140046c0 tx=0x7f441400c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8596a11cb234d9d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:33.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.252+0000 7f441a000700 1 -- 172.21.15.63:0/2954236052 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44140142d0 con 0x55fe9f533e70 2024-05-27T07:46:33.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.252+0000 7f441a000700 1 -- 172.21.15.63:0/2954236052 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4414011990 con 0x55fe9f533e70 2024-05-27T07:46:33.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.253+0000 7f441a000700 1 -- 172.21.15.63:0/2954236052 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414012650 con 0x55fe9f533e70 2024-05-27T07:46:33.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.254+0000 7f443e83bb80 1 -- 172.21.15.63:0/2954236052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 msgr2=0x55fe9f540110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.254+0000 7f442dc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f44100b2a70 0x7f44100b4f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44200a5050 tx=0x7f442000df20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:33.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.254+0000 7f443e83bb80 1 --2- 172.21.15.63:0/2954236052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f540110 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f44140046c0 tx=0x7f441400c4f0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.254+0000 7f443e83bb80 1 -- 172.21.15.63:0/2954236052 shutdown_connections 2024-05-27T07:46:33.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.254+0000 7f443e83bb80 1 --2- 172.21.15.63:0/2954236052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f540110 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.254+0000 7f443e83bb80 1 --2- 172.21.15.63:0/2954236052 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f4a09f0 0x55fe9f533330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.254+0000 7f443e83bb80 1 --2- 172.21.15.63:0/2954236052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f3b4e40 0x55fe9f533870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.254+0000 7f443e83bb80 1 -- 172.21.15.63:0/2954236052 >> 172.21.15.63:0/2954236052 conn(0x55fe9f44e390 msgr2=0x55fe9f52f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:33.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.254+0000 7f443e83bb80 1 -- 172.21.15.63:0/2954236052 shutdown_connections 2024-05-27T07:46:33.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.254+0000 7f443e83bb80 1 -- 172.21.15.63:0/2954236052 wait complete. 2024-05-27T07:46:33.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.255+0000 7f443e83bb80 1 -- start start 2024-05-27T07:46:33.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.255+0000 7f443e83bb80 1 Processor -- start 2024-05-27T07:46:33.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.256+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f3b4e40 0x55fe9f53d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.256+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f4a09f0 0x55fe9f53db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.256+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f533e70 0x55fe9f53e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.256+0000 7f443e83bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fe9f357fa0 con 0x55fe9f4a09f0 2024-05-27T07:46:33.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.256+0000 7f443e83bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fe9f3f2760 con 0x55fe9f3b4e40 2024-05-27T07:46:33.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.256+0000 7f443e83bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fe9f540b70 con 0x55fe9f533e70 2024-05-27T07:46:33.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.256+0000 7f443e83bb80 10 client.0 _ll_register_callbacks cb 0x55fe9f3fbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:33.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.257+0000 7f442d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f4a09f0 0x55fe9f53db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.257+0000 7f442d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f4a09f0 0x55fe9f53db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41376/0 (socket says 172.21.15.63:41376) 2024-05-27T07:46:33.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.257+0000 7f442d200700 1 -- 172.21.15.63:0/1909588950 learned_addr learned my addr 172.21.15.63:0/1909588950 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:33.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.257+0000 7f4427e00700 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f3b4e40 0x55fe9f53d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.257+0000 7f4427e00700 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f533e70 0x55fe9f53e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.257+0000 7f442d200700 1 -- 172.21.15.63:0/1909588950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f533e70 msgr2=0x55fe9f53e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:33.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.257+0000 7f442d200700 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f533e70 0x55fe9f53e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.257+0000 7f442d200700 1 -- 172.21.15.63:0/1909588950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f3b4e40 msgr2=0x55fe9f53d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:33.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.257+0000 7f442d200700 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f3b4e40 0x55fe9f53d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.257+0000 7f442d200700 1 -- 172.21.15.63:0/1909588950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe9f3a18d0 con 0x55fe9f4a09f0 2024-05-27T07:46:33.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.258+0000 7f442d200700 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f4a09f0 0x55fe9f53db50 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f441c000c00 tx=0x7f441c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a849f6367e856e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:33.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.258+0000 7f440ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:33.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.259+0000 7f440ea00700 1 -- 172.21.15.63:0/1909588950 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c003bd0 con 0x55fe9f4a09f0 2024-05-27T07:46:33.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.260+0000 7f440ea00700 1 -- 172.21.15.63:0/1909588950 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f441c003d70 con 0x55fe9f4a09f0 2024-05-27T07:46:33.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.260+0000 7f443e83bb80 1 -- 172.21.15.63:0/1909588950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4414011990 con 0x55fe9f4a09f0 2024-05-27T07:46:33.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.260+0000 7f440ea00700 1 -- 172.21.15.63:0/1909588950 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f441c031050 con 0x55fe9f4a09f0 2024-05-27T07:46:33.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.260+0000 7f443e83bb80 10 client.17892 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:33.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.260+0000 7f443e83bb80 1 -- 172.21.15.63:0/1909588950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fe9f422060 con 0x55fe9f4a09f0 2024-05-27T07:46:33.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.261+0000 7f440ea00700 1 -- 172.21.15.63:0/1909588950 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4414011990 con 0x55fe9f4a09f0 2024-05-27T07:46:33.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.261+0000 7f440ea00700 1 -- 172.21.15.63:0/1909588950 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f441c023700 con 0x55fe9f4a09f0 2024-05-27T07:46:33.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.261+0000 7f443e83bb80 10 client.17892 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:33.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.261+0000 7f443e83bb80 10 client.17892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:33.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.261+0000 7f443e83bb80 20 client.17892 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:33.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.261+0000 7f443e83bb80 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe9f54da80 0x55fe9f54fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.261+0000 7f443e83bb80 4 client.17892 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:33.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.261+0000 7f443e83bb80 1 -- 172.21.15.63:0/1909588950 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe9f463d90 con 0x55fe9f54da80 2024-05-27T07:46:33.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.262+0000 7f4427e00700 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe9f54da80 0x55fe9f54fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.263+0000 7f4427e00700 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe9f54da80 0x55fe9f54fea0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1b9d23ad674207 server_cookie=d46d0df835af9db8 in_seq=0 out_seq=0 2024-05-27T07:46:33.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.263+0000 7f440ea00700 10 client.17892 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:33.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.265+0000 7f440ea00700 1 -- 172.21.15.63:0/1909588950 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fe9f463d90 con 0x55fe9f54da80 2024-05-27T07:46:33.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.265+0000 7f440ea00700 10 client.17892 handle_command_reply: tid=0 2024-05-27T07:46:33.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.265+0000 7f443e83bb80 1 client.17892 shutdown 2024-05-27T07:46:33.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.265+0000 7f443e83bb80 1 -- 172.21.15.63:0/1909588950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f4a09f0 msgr2=0x55fe9f53db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.265+0000 7f443e83bb80 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f4a09f0 0x55fe9f53db50 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f441c000c00 tx=0x7f441c005770 comp rx=0 tx=0).stop 2024-05-27T07:46:33.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.265+0000 7f443e83bb80 1 -- 172.21.15.63:0/1909588950 shutdown_connections 2024-05-27T07:46:33.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.265+0000 7f443e83bb80 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe9f54da80 0x55fe9f54fea0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.266+0000 7f443e83bb80 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f533e70 0x55fe9f53e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.266+0000 7f443e83bb80 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f3b4e40 0x55fe9f53d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.266+0000 7f443e83bb80 1 --2- 172.21.15.63:0/1909588950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f4a09f0 0x55fe9f53db50 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.266+0000 7f443e83bb80 1 -- 172.21.15.63:0/1909588950 >> 172.21.15.63:0/1909588950 conn(0x55fe9f44e390 msgr2=0x55fe9f533450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:33.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.266+0000 7f440ea00700 0 client.17892 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:33.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.266+0000 7f443e83bb80 1 -- 172.21.15.63:0/1909588950 shutdown_connections 2024-05-27T07:46:33.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.266+0000 7f443e83bb80 1 -- 172.21.15.63:0/1909588950 wait complete. 2024-05-27T07:46:33.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.266+0000 7f443e83bb80 20 client.17892 trim_cache size 0 max 16384 2024-05-27T07:46:33.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.291+0000 7f443e83bb80 1 Processor -- start 2024-05-27T07:46:33.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.291+0000 7f443e83bb80 1 -- start start 2024-05-27T07:46:33.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.291+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f3b4e40 0x55fe9f538e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.291+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f4a09f0 0x55fe9f549c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.291+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f555660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.291+0000 7f443e83bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f44281b4fb0 con 0x55fe9f533e70 2024-05-27T07:46:33.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.291+0000 7f443e83bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44281b2840 con 0x55fe9f4a09f0 2024-05-27T07:46:33.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.291+0000 7f443e83bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f44281b06b0 con 0x55fe9f3b4e40 2024-05-27T07:46:33.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.292+0000 7f4427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f555660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.292+0000 7f4427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f555660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41388/0 (socket says 172.21.15.63:41388) 2024-05-27T07:46:33.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.292+0000 7f4427e00700 1 -- 172.21.15.63:0/625332775 learned_addr learned my addr 172.21.15.63:0/625332775 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:33.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.292+0000 7f442d200700 1 --2- 172.21.15.63:0/625332775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f4a09f0 0x55fe9f549c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.292+0000 7f4427e00700 1 --2- 172.21.15.63:0/625332775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f3b4e40 0x55fe9f538e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.292+0000 7f4427e00700 1 -- 172.21.15.63:0/625332775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f3b4e40 msgr2=0x55fe9f538e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:33.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.292+0000 7f4427e00700 1 --2- 172.21.15.63:0/625332775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f3b4e40 0x55fe9f538e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.292+0000 7f4427e00700 1 -- 172.21.15.63:0/625332775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f4a09f0 msgr2=0x55fe9f549c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:33.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.292+0000 7f4427e00700 1 --2- 172.21.15.63:0/625332775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f4a09f0 0x55fe9f549c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.292+0000 7f4427e00700 1 -- 172.21.15.63:0/625332775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe9f53bb80 con 0x55fe9f533e70 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.293+0000 7f4427e00700 1 --2- 172.21.15.63:0/625332775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f555660 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4414016010 tx=0x7f4414016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c11b4eb8cb529cb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.293+0000 7f4419600700 1 -- 172.21.15.63:0/625332775 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441400c9d0 con 0x55fe9f533e70 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.293+0000 7f4419600700 1 -- 172.21.15.63:0/625332775 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4414011990 con 0x55fe9f533e70 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.293+0000 7f4419600700 1 -- 172.21.15.63:0/625332775 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441402e6b0 con 0x55fe9f533e70 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.294+0000 7f443e83bb80 1 -- 172.21.15.63:0/625332775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 msgr2=0x55fe9f555660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.294+0000 7f443e83bb80 1 --2- 172.21.15.63:0/625332775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f555660 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4414016010 tx=0x7f4414016040 comp rx=0 tx=0).stop 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.294+0000 7f443e83bb80 1 -- 172.21.15.63:0/625332775 shutdown_connections 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.294+0000 7f443e83bb80 1 --2- 172.21.15.63:0/625332775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f533e70 0x55fe9f555660 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.294+0000 7f443e83bb80 1 --2- 172.21.15.63:0/625332775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f3b4e40 0x55fe9f538e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.294+0000 7f443e83bb80 1 --2- 172.21.15.63:0/625332775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f4a09f0 0x55fe9f549c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.294+0000 7f443e83bb80 1 -- 172.21.15.63:0/625332775 >> 172.21.15.63:0/625332775 conn(0x55fe9f44e390 msgr2=0x55fe9f53c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.294+0000 7f443e83bb80 1 -- 172.21.15.63:0/625332775 shutdown_connections 2024-05-27T07:46:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.294+0000 7f443e83bb80 1 -- 172.21.15.63:0/625332775 wait complete. 2024-05-27T07:46:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.295+0000 7f443e83bb80 1 -- start start 2024-05-27T07:46:33.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.295+0000 7f443e83bb80 1 Processor -- start 2024-05-27T07:46:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f3b4e40 0x55fe9f537950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f4a09f0 0x55fe9f537e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f443e83bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f533e70 0x55fe9f544e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f443e83bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f44281b0d00 con 0x55fe9f3b4e40 2024-05-27T07:46:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f443e83bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44281b2170 con 0x55fe9f533e70 2024-05-27T07:46:33.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f443e83bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f44281b13d0 con 0x55fe9f4a09f0 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f443e83bb80 10 client.0 _ll_register_callbacks cb 0x55fe9f3fbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f4427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f3b4e40 0x55fe9f537950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f4427e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f3b4e40 0x55fe9f537950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41394/0 (socket says 172.21.15.63:41394) 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f4427e00700 1 -- 172.21.15.63:0/857613250 learned_addr learned my addr 172.21.15.63:0/857613250 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.296+0000 7f442d200700 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f4a09f0 0x55fe9f537e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.297+0000 7f4427e00700 1 -- 172.21.15.63:0/857613250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f4a09f0 msgr2=0x55fe9f537e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.297+0000 7f4427e00700 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f4a09f0 0x55fe9f537e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.297+0000 7f4427e00700 1 -- 172.21.15.63:0/857613250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f533e70 msgr2=0x55fe9f544e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.297+0000 7f4427e00700 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f533e70 0x55fe9f544e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.297+0000 7f4427e00700 1 -- 172.21.15.63:0/857613250 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe9f545340 con 0x55fe9f3b4e40 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.297+0000 7f4427e00700 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f3b4e40 0x55fe9f537950 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f4414005810 tx=0x7f441400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22e4083cc65482e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.297+0000 7f440ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:33.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.297+0000 7f440ea00700 1 -- 172.21.15.63:0/857613250 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441403f060 con 0x55fe9f3b4e40 2024-05-27T07:46:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.298+0000 7f440ea00700 1 -- 172.21.15.63:0/857613250 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f4414011700 con 0x55fe9f3b4e40 2024-05-27T07:46:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.298+0000 7f443e83bb80 1 -- 172.21.15.63:0/857613250 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4414011990 con 0x55fe9f3b4e40 2024-05-27T07:46:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.298+0000 7f440ea00700 1 -- 172.21.15.63:0/857613250 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4414011e50 con 0x55fe9f3b4e40 2024-05-27T07:46:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.298+0000 7f443e83bb80 10 client.17898 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:33.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.298+0000 7f443e83bb80 1 -- 172.21.15.63:0/857613250 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fe9f5465b0 con 0x55fe9f3b4e40 2024-05-27T07:46:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.298+0000 7f440ea00700 1 -- 172.21.15.63:0/857613250 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4414011990 con 0x55fe9f3b4e40 2024-05-27T07:46:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.299+0000 7f440ea00700 1 -- 172.21.15.63:0/857613250 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f441401f420 con 0x55fe9f3b4e40 2024-05-27T07:46:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.299+0000 7f443e83bb80 10 client.17898 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.299+0000 7f443e83bb80 10 client.17898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.299+0000 7f443e83bb80 20 client.17898 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.299+0000 7f443e83bb80 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe9f5b8530 0x55fe9f5ba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.299+0000 7f443e83bb80 4 client.17898 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:46:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.299+0000 7f443e83bb80 1 -- 172.21.15.63:0/857613250 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe9f45d570 con 0x55fe9f5b8530 2024-05-27T07:46:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.299+0000 7f442d200700 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe9f5b8530 0x55fe9f5ba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.300+0000 7f442d200700 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe9f5b8530 0x55fe9f5ba950 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd8dc2259560c36 server_cookie=46904b233c29f018 in_seq=0 out_seq=0 2024-05-27T07:46:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.300+0000 7f440ea00700 10 client.17898 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f440ea00700 1 -- 172.21.15.63:0/857613250 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fe9f45d570 con 0x55fe9f5b8530 2024-05-27T07:46:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f440ea00700 10 client.17898 handle_command_reply: tid=0 2024-05-27T07:46:33.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f443e83bb80 1 client.17898 shutdown 2024-05-27T07:46:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f443e83bb80 1 -- 172.21.15.63:0/857613250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f3b4e40 msgr2=0x55fe9f537950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f443e83bb80 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f3b4e40 0x55fe9f537950 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f4414005810 tx=0x7f441400cb10 comp rx=0 tx=0).stop 2024-05-27T07:46:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f443e83bb80 1 -- 172.21.15.63:0/857613250 shutdown_connections 2024-05-27T07:46:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f443e83bb80 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe9f5b8530 0x55fe9f5ba950 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f443e83bb80 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe9f533e70 0x55fe9f544e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f443e83bb80 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe9f3b4e40 0x55fe9f537950 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f443e83bb80 1 --2- 172.21.15.63:0/857613250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe9f4a09f0 0x55fe9f537e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f443e83bb80 1 -- 172.21.15.63:0/857613250 >> 172.21.15.63:0/857613250 conn(0x55fe9f44e390 msgr2=0x55fe9f534ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f440ea00700 0 client.17898 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:33.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.301+0000 7f443e83bb80 1 -- 172.21.15.63:0/857613250 shutdown_connections 2024-05-27T07:46:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.302+0000 7f443e83bb80 1 -- 172.21.15.63:0/857613250 wait complete. 2024-05-27T07:46:33.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.302+0000 7f443e83bb80 20 client.17898 trim_cache size 0 max 16384 2024-05-27T07:46:33.304 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:46:33.304 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:46:33.304 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:46:33.304 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:46:33.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 -- 172.21.15.63:0/774579746 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f44100b2a70 msgr2=0x7f44100b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f44100b2a70 0x7f44100b4f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44200a5050 tx=0x7f442000df20 comp rx=0 tx=0).stop 2024-05-27T07:46:33.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 -- 172.21.15.63:0/774579746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442810e8d0 msgr2=0x7f4428103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442810e8d0 0x7f4428103c60 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4420007ee0 tx=0x7f442000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 -- 172.21.15.63:0/774579746 shutdown_connections 2024-05-27T07:46:33.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f44100b2a70 0x7f44100b4f80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442810e8d0 0x7f4428103c60 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44281054f0 0x7f44281031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 --2- 172.21.15.63:0/774579746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4428105e40 0x7f4428103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 -- 172.21.15.63:0/774579746 >> 172.21.15.63:0/774579746 conn(0x7f44280fedb0 msgr2=0x7f44280ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 -- 172.21.15.63:0/774579746 shutdown_connections 2024-05-27T07:46:33.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:33.303+0000 7f442fc00700 1 -- 172.21.15.63:0/774579746 wait complete. 2024-05-27T07:46:33.360 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:33 smithi063 ceph-mon[31613]: pgmap v406: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:46:33.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:33 smithi086 ceph-mon[34687]: pgmap v406: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:46:33.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:33 smithi149 ceph-mon[35408]: pgmap v406: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:46:34.394 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:46:34.395 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:46:34.915 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:35.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:35 smithi063 ceph-mon[31613]: pgmap v407: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:46:35.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:35 smithi086 ceph-mon[34687]: pgmap v407: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:46:35.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:35 smithi149 ceph-mon[35408]: pgmap v407: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:46:37.321 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:37 smithi063 ceph-mon[31613]: pgmap v408: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:46:37.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.468+0000 7f43d1a00700 1 -- 172.21.15.63:0/2683672533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43cc10b890 msgr2=0x7f43cc1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:37.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.468+0000 7f43d1a00700 1 --2- 172.21.15.63:0/2683672533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43cc10b890 0x7f43cc1131f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f43c400b0d0 tx=0x7f43c400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:46:37.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.468+0000 7f43d1a00700 1 -- 172.21.15.63:0/2683672533 shutdown_connections 2024-05-27T07:46:37.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.468+0000 7f43d1a00700 1 --2- 172.21.15.63:0/2683672533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43cc10b890 0x7f43cc1131f0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:37.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.468+0000 7f43d1a00700 1 --2- 172.21.15.63:0/2683672533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43cc106610 0x7f43cc106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:37.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.468+0000 7f43d1a00700 1 --2- 172.21.15.63:0/2683672533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43cc10af40 0x7f43cc10b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:37.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.468+0000 7f43d1a00700 1 -- 172.21.15.63:0/2683672533 >> 172.21.15.63:0/2683672533 conn(0x7f43cc102020 msgr2=0x7f43cc104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:37.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.468+0000 7f43d1a00700 1 -- 172.21.15.63:0/2683672533 shutdown_connections 2024-05-27T07:46:37.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.468+0000 7f43d1a00700 1 -- 172.21.15.63:0/2683672533 wait complete. 2024-05-27T07:46:37.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.469+0000 7f43d1a00700 1 Processor -- start 2024-05-27T07:46:37.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.469+0000 7f43d1a00700 1 -- start start 2024-05-27T07:46:37.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.470+0000 7f43d1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43cc106610 0x7f43cc19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:37.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.470+0000 7f43d1a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43cc10af40 0x7f43cc19d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:37.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.470+0000 7f43d1a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43cc10b890 0x7f43cc1a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:37.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.470+0000 7f43d1a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc1163c0 con 0x7f43cc10b890 2024-05-27T07:46:37.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.470+0000 7f43d1a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc116240 con 0x7f43cc106610 2024-05-27T07:46:37.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.470+0000 7f43d1a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc116540 con 0x7f43cc10af40 2024-05-27T07:46:37.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.470+0000 7f43d1000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43cc10b890 0x7f43cc1a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:37.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.470+0000 7f43d1000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43cc10b890 0x7f43cc1a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41420/0 (socket says 172.21.15.63:41420) 2024-05-27T07:46:37.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.470+0000 7f43d1000700 1 -- 172.21.15.63:0/3547143489 learned_addr learned my addr 172.21.15.63:0/3547143489 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:37.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.471+0000 7f43cb400700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43cc10af40 0x7f43cc19d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:37.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.471+0000 7f43cbe00700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43cc106610 0x7f43cc19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:37.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.471+0000 7f43d1000700 1 -- 172.21.15.63:0/3547143489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43cc10af40 msgr2=0x7f43cc19d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:37.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.471+0000 7f43d1000700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43cc10af40 0x7f43cc19d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:37.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.471+0000 7f43d1000700 1 -- 172.21.15.63:0/3547143489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43cc106610 msgr2=0x7f43cc19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:37.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.471+0000 7f43d1000700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43cc106610 0x7f43cc19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:37.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.471+0000 7f43d1000700 1 -- 172.21.15.63:0/3547143489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc1a1be0 con 0x7f43cc10b890 2024-05-27T07:46:37.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.472+0000 7f43d1000700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43cc10b890 0x7f43cc1a1540 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f43c4007d10 tx=0x7f43c4007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:37.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.472+0000 7f43c8c00700 1 -- 172.21.15.63:0/3547143489 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c401f070 con 0x7f43cc10b890 2024-05-27T07:46:37.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.472+0000 7f43c8c00700 1 -- 172.21.15.63:0/3547143489 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43c4003e10 con 0x7f43cc10b890 2024-05-27T07:46:37.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.472+0000 7f43d1a00700 1 -- 172.21.15.63:0/3547143489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc1a1e70 con 0x7f43cc10b890 2024-05-27T07:46:37.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.472+0000 7f43c8c00700 1 -- 172.21.15.63:0/3547143489 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c401b720 con 0x7f43cc10b890 2024-05-27T07:46:37.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.472+0000 7f43d1a00700 1 -- 172.21.15.63:0/3547143489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc1a2350 con 0x7f43cc10b890 2024-05-27T07:46:37.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.473+0000 7f43d1a00700 1 -- 172.21.15.63:0/3547143489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc0512b0 con 0x7f43cc10b890 2024-05-27T07:46:37.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.477+0000 7f43c8c00700 1 -- 172.21.15.63:0/3547143489 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f43c4022070 con 0x7f43cc10b890 2024-05-27T07:46:37.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.479+0000 7f43c8c00700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43b00b29f0 0x7f43b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:37.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.480+0000 7f43c8c00700 1 -- 172.21.15.63:0/3547143489 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f43c400f020 con 0x7f43cc10b890 2024-05-27T07:46:37.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.480+0000 7f43cbe00700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43b00b29f0 0x7f43b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:37.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.481+0000 7f43cbe00700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43b00b29f0 0x7f43b00b4f00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43bc005950 tx=0x7f43bc00b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:37.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.481+0000 7f43c8c00700 1 -- 172.21.15.63:0/3547143489 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f43c4029020 con 0x7f43cc10b890 2024-05-27T07:46:37.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:37 smithi086 ceph-mon[34687]: pgmap v408: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:46:37.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:37 smithi149 ceph-mon[35408]: pgmap v408: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:46:37.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.808+0000 7f43d1a00700 1 -- 172.21.15.63:0/3547143489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43cc02e5e0 con 0x7f43cc10b890 2024-05-27T07:46:37.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.810+0000 7f43c8c00700 1 -- 172.21.15.63:0/3547143489 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f43c40ab890 con 0x7f43cc10b890 2024-05-27T07:46:37.810 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:46:37.811 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:46:37.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.813+0000 7f43d1a00700 1 -- 172.21.15.63:0/3547143489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43b00b29f0 msgr2=0x7f43b00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:37.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.813+0000 7f43d1a00700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43b00b29f0 0x7f43b00b4f00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43bc005950 tx=0x7f43bc00b410 comp rx=0 tx=0).stop 2024-05-27T07:46:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.813+0000 7f43d1a00700 1 -- 172.21.15.63:0/3547143489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43cc10b890 msgr2=0x7f43cc1a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.813+0000 7f43d1a00700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43cc10b890 0x7f43cc1a1540 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f43c4007d10 tx=0x7f43c4007d40 comp rx=0 tx=0).stop 2024-05-27T07:46:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.813+0000 7f43d1a00700 1 -- 172.21.15.63:0/3547143489 shutdown_connections 2024-05-27T07:46:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.813+0000 7f43d1a00700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43b00b29f0 0x7f43b00b4f00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.813+0000 7f43d1a00700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43cc10b890 0x7f43cc1a1540 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.813+0000 7f43d1a00700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43cc106610 0x7f43cc19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.813+0000 7f43d1a00700 1 --2- 172.21.15.63:0/3547143489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43cc10af40 0x7f43cc19d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.813+0000 7f43d1a00700 1 -- 172.21.15.63:0/3547143489 >> 172.21.15.63:0/3547143489 conn(0x7f43cc102020 msgr2=0x7f43cc10a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.814+0000 7f43d1a00700 1 -- 172.21.15.63:0/3547143489 shutdown_connections 2024-05-27T07:46:37.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:37.814+0000 7f43d1a00700 1 -- 172.21.15.63:0/3547143489 wait complete. 2024-05-27T07:46:37.815 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:46:38.610 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:38 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3547143489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:46:38.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:38 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3547143489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:46:38.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:38 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3547143489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:46:38.804 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:46:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.977+0000 7ff9f6a00640 1 -- 172.21.15.63:0/503143744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f010e1d0 msgr2=0x7ff9f01106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.977+0000 7ff9f6a00640 1 --2- 172.21.15.63:0/503143744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f010e1d0 0x7ff9f01106c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009ca0 tx=0x7ff9e802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.977+0000 7ff9f6a00640 1 -- 172.21.15.63:0/503143744 shutdown_connections 2024-05-27T07:46:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.977+0000 7ff9f6a00640 1 --2- 172.21.15.63:0/503143744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f010e1d0 0x7ff9f01106c0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.977+0000 7ff9f6a00640 1 --2- 172.21.15.63:0/503143744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f010b7e0 0x7ff9f010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.977+0000 7ff9f6a00640 1 --2- 172.21.15.63:0/503143744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f0104d80 0x7ff9f0105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.977+0000 7ff9f6a00640 1 -- 172.21.15.63:0/503143744 >> 172.21.15.63:0/503143744 conn(0x7ff9f00fe380 msgr2=0x7ff9f01007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:38.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.978+0000 7ff9f6a00640 1 -- 172.21.15.63:0/503143744 shutdown_connections 2024-05-27T07:46:38.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.978+0000 7ff9f6a00640 1 -- 172.21.15.63:0/503143744 wait complete. 2024-05-27T07:46:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.979+0000 7ff9f6a00640 1 Processor -- start 2024-05-27T07:46:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.979+0000 7ff9f6a00640 1 -- start start 2024-05-27T07:46:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.980+0000 7ff9f6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0104d80 0x7ff9f01a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.980+0000 7ff9f6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f010b7e0 0x7ff9f01a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.980+0000 7ff9f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f010e1d0 0x7ff9f019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.980+0000 7ff9f6a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0113d30 con 0x7ff9f0104d80 2024-05-27T07:46:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.980+0000 7ff9f6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0113bb0 con 0x7ff9f010e1d0 2024-05-27T07:46:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.980+0000 7ff9f6a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f0113eb0 con 0x7ff9f010b7e0 2024-05-27T07:46:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.980+0000 7ff9efe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0104d80 0x7ff9f01a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.980+0000 7ff9efe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0104d80 0x7ff9f01a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51498/0 (socket says 172.21.15.63:51498) 2024-05-27T07:46:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.980+0000 7ff9efe00640 1 -- 172.21.15.63:0/466134537 learned_addr learned my addr 172.21.15.63:0/466134537 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.980+0000 7ff9f4a00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f010e1d0 0x7ff9f019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.981+0000 7ff9efe00640 1 -- 172.21.15.63:0/466134537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f010b7e0 msgr2=0x7ff9f01a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.981+0000 7ff9efe00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f010b7e0 0x7ff9f01a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.981+0000 7ff9efe00640 1 -- 172.21.15.63:0/466134537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f010e1d0 msgr2=0x7ff9f019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.981+0000 7ff9efe00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f010e1d0 0x7ff9f019c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.981+0000 7ff9efe00640 1 -- 172.21.15.63:0/466134537 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f019cec0 con 0x7ff9f0104d80 2024-05-27T07:46:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.981+0000 7ff9efe00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0104d80 0x7ff9f01a2610 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000d770 tx=0x7ff9e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.982+0000 7ff9ecc00640 1 -- 172.21.15.63:0/466134537 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0017070 con 0x7ff9f0104d80 2024-05-27T07:46:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.982+0000 7ff9ecc00640 1 -- 172.21.15.63:0/466134537 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9e00054b0 con 0x7ff9f0104d80 2024-05-27T07:46:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.982+0000 7ff9f6a00640 1 -- 172.21.15.63:0/466134537 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f019d150 con 0x7ff9f0104d80 2024-05-27T07:46:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.982+0000 7ff9ecc00640 1 -- 172.21.15.63:0/466134537 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0004c60 con 0x7ff9f0104d80 2024-05-27T07:46:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.982+0000 7ff9f6a00640 1 -- 172.21.15.63:0/466134537 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f01a93b0 con 0x7ff9f0104d80 2024-05-27T07:46:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.986+0000 7ff9fdb71740 1 Processor -- start 2024-05-27T07:46:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.986+0000 7ff9fdb71740 1 -- start start 2024-05-27T07:46:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.987+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d481b0 0x559fe60ea860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.987+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 0x559fe60eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.987+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 0x559fe60eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.987+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559fe605bb90 con 0x559fe5d1f2f0 2024-05-27T07:46:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.987+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559fe604b730 con 0x559fe5d274d0 2024-05-27T07:46:38.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.987+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559fe604f770 con 0x559fe5d481b0 2024-05-27T07:46:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.987+0000 7ff9ecc00640 1 -- 172.21.15.63:0/466134537 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff9e0004e00 con 0x7ff9f0104d80 2024-05-27T07:46:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.988+0000 7ff9ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 0x559fe60eb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.988+0000 7ff9ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 0x559fe60eb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:54626/0 (socket says 172.21.15.63:54626) 2024-05-27T07:46:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.988+0000 7ff9ef400640 1 -- 172.21.15.63:0/3431439386 learned_addr learned my addr 172.21.15.63:0/3431439386 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.988+0000 7ff9f4a00640 1 --2- 172.21.15.63:0/3431439386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d481b0 0x559fe60ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.988+0000 7ff9efe00640 1 --2- 172.21.15.63:0/3431439386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 0x559fe60eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.988+0000 7ff9ef400640 1 -- 172.21.15.63:0/3431439386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d481b0 msgr2=0x559fe60ea860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.989+0000 7ff9ef400640 1 --2- 172.21.15.63:0/3431439386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d481b0 0x559fe60ea860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.989+0000 7ff9ef400640 1 -- 172.21.15.63:0/3431439386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 msgr2=0x559fe60eada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.989+0000 7ff9ef400640 1 --2- 172.21.15.63:0/3431439386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 0x559fe60eada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.989+0000 7ff9ef400640 1 -- 172.21.15.63:0/3431439386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe5fe48b0 con 0x559fe5d274d0 2024-05-27T07:46:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.990+0000 7ff9ef400640 1 --2- 172.21.15.63:0/3431439386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 0x559fe60eb2e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009b30 tx=0x7ff9dc00e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=515e5760cbfd3f1a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.990+0000 7ff9e5200640 1 -- 172.21.15.63:0/3431439386 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc00ed80 con 0x559fe5d274d0 2024-05-27T07:46:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.990+0000 7ff9ecc00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2950 0x7ff9d40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.990+0000 7ff9e5200640 1 -- 172.21.15.63:0/3431439386 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9dc0054b0 con 0x559fe5d274d0 2024-05-27T07:46:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.990+0000 7ff9e5200640 1 -- 172.21.15.63:0/3431439386 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc010690 con 0x559fe5d274d0 2024-05-27T07:46:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.991+0000 7ff9ecc00640 1 -- 172.21.15.63:0/466134537 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7ff9e00e2510 con 0x7ff9f0104d80 2024-05-27T07:46:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.991+0000 7ff9f4a00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2950 0x7ff9d40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.992+0000 7ff9fdb71740 1 -- 172.21.15.63:0/3431439386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 msgr2=0x559fe60eb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.992+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/3431439386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 0x559fe60eb2e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009b30 tx=0x7ff9dc00e350 comp rx=0 tx=0).stop 2024-05-27T07:46:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.992+0000 7ff9fdb71740 1 -- 172.21.15.63:0/3431439386 shutdown_connections 2024-05-27T07:46:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.992+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/3431439386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 0x559fe60eb2e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.992+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/3431439386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 0x559fe60eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.992+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/3431439386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d481b0 0x559fe60ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.992+0000 7ff9fdb71740 1 -- 172.21.15.63:0/3431439386 >> 172.21.15.63:0/3431439386 conn(0x559fe5d3c300 msgr2=0x559fe60e7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.993+0000 7ff9f4a00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2950 0x7ff9d40b4e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8004af0 tx=0x7ff9e8005da0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.993+0000 7ff9fdb71740 1 -- 172.21.15.63:0/3431439386 shutdown_connections 2024-05-27T07:46:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.993+0000 7ff9fdb71740 1 -- 172.21.15.63:0/3431439386 wait complete. 2024-05-27T07:46:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.994+0000 7ff9fdb71740 1 -- start start 2024-05-27T07:46:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.994+0000 7ff9fdb71740 1 Processor -- start 2024-05-27T07:46:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.995+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d1f2f0 0x559fe60f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.995+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 0x559fe60f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.995+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe6100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.995+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559fe5d31790 con 0x559fe5d481b0 2024-05-27T07:46:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.995+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559fe5ff3bc0 con 0x559fe5d1f2f0 2024-05-27T07:46:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.995+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559fe60f7990 con 0x559fe5d274d0 2024-05-27T07:46:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.995+0000 7ff9fdb71740 10 client.0 _ll_register_callbacks cb 0x559fe5fd1400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.996+0000 7ff9f4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe6100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.996+0000 7ff9ef400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 0x559fe60f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.996+0000 7ff9ef400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 0x559fe60f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:59352/0 (socket says 172.21.15.63:59352) 2024-05-27T07:46:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.996+0000 7ff9ef400640 1 -- 172.21.15.63:0/2781428913 learned_addr learned my addr 172.21.15.63:0/2781428913 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.996+0000 7ff9efe00640 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d1f2f0 0x559fe60f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.996+0000 7ff9ef400640 1 -- 172.21.15.63:0/2781428913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d1f2f0 msgr2=0x559fe60f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.996+0000 7ff9ef400640 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d1f2f0 0x559fe60f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.997+0000 7ff9ef400640 1 -- 172.21.15.63:0/2781428913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 msgr2=0x559fe6100cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.997+0000 7ff9ef400640 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe6100cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.997+0000 7ff9ef400640 1 -- 172.21.15.63:0/2781428913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe5d60210 con 0x559fe5d274d0 2024-05-27T07:46:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.997+0000 7ff9f4a00640 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe6100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:46:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.997+0000 7ff9ef400640 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 0x559fe60f4f90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00e850 tx=0x7ff9dc005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31792b11ce115f40 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.997+0000 7ff9d9600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:46:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.998+0000 7ff9d9600640 1 -- 172.21.15.63:0/2781428913 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc026020 con 0x559fe5d274d0 2024-05-27T07:46:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.999+0000 7ff9d9600640 1 -- 172.21.15.63:0/2781428913 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7ff9dc018630 con 0x559fe5d274d0 2024-05-27T07:46:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:38.999+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2781428913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9dc0054b0 con 0x559fe5d274d0 2024-05-27T07:46:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.000+0000 7ff9d9600640 1 -- 172.21.15.63:0/2781428913 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff9dc031050 con 0x559fe5d274d0 2024-05-27T07:46:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.000+0000 7ff9fdb71740 10 client.27238 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.000+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2781428913 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559fe5e2c620 con 0x559fe5d274d0 2024-05-27T07:46:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.000+0000 7ff9d9600640 1 -- 172.21.15.63:0/2781428913 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9dc0054b0 con 0x559fe5d274d0 2024-05-27T07:46:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.001+0000 7ff9d9600640 1 -- 172.21.15.63:0/2781428913 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff9dc010430 con 0x559fe5d274d0 2024-05-27T07:46:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.001+0000 7ff9fdb71740 10 client.27238 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.001+0000 7ff9fdb71740 10 client.27238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.001+0000 7ff9fdb71740 20 client.27238 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.001+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fe5d06d80 0x559fe6106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.001+0000 7ff9fdb71740 4 client.27238 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.001+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2781428913 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fe605dae0 con 0x559fe5d06d80 2024-05-27T07:46:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.002+0000 7ff9efe00640 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fe5d06d80 0x559fe6106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.003+0000 7ff9efe00640 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fe5d06d80 0x559fe6106bb0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cfee5a15144239 server_cookie=f91084db72c6085 in_seq=0 out_seq=0 2024-05-27T07:46:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.003+0000 7ff9d9600640 10 client.27238 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9d9600640 1 -- 172.21.15.63:0/2781428913 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559fe605dae0 con 0x559fe5d06d80 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9d9600640 10 client.27238 handle_command_reply: tid=0 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9fdb71740 1 client.27238 shutdown 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2781428913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 msgr2=0x559fe60f4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 0x559fe60f4f90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00e850 tx=0x7ff9dc005250 comp rx=0 tx=0).stop 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2781428913 shutdown_connections 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fe5d06d80 0x559fe6106bb0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe6100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 0x559fe60f4f90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9d9600640 0 client.27238 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2781428913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d1f2f0 0x559fe60f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.005+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2781428913 >> 172.21.15.63:0/2781428913 conn(0x559fe5d3c300 msgr2=0x559fe60ec580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.006+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2781428913 shutdown_connections 2024-05-27T07:46:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.006+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2781428913 wait complete. 2024-05-27T07:46:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.006+0000 7ff9fdb71740 20 client.27238 trim_cache size 0 max 16384 2024-05-27T07:46:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.031+0000 7ff9fdb71740 1 Processor -- start 2024-05-27T07:46:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.031+0000 7ff9fdb71740 1 -- start start 2024-05-27T07:46:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.032+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 0x559fe60f0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.032+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 0x559fe60f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.032+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d481b0 0x559fe60eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.032+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b5d30 con 0x559fe5d1f2f0 2024-05-27T07:46:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.032+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b3600 con 0x559fe5d274d0 2024-05-27T07:46:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.032+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b1470 con 0x559fe5d481b0 2024-05-27T07:46:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.032+0000 7ff9efe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 0x559fe60f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.032+0000 7ff9efe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 0x559fe60f0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51518/0 (socket says 172.21.15.63:51518) 2024-05-27T07:46:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.032+0000 7ff9efe00640 1 -- 172.21.15.63:0/3657443992 learned_addr learned my addr 172.21.15.63:0/3657443992 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.032+0000 7ff9ef400640 1 --2- 172.21.15.63:0/3657443992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 0x559fe60f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.033+0000 7ff9f4a00640 1 --2- 172.21.15.63:0/3657443992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d481b0 0x559fe60eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.033+0000 7ff9efe00640 1 -- 172.21.15.63:0/3657443992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d481b0 msgr2=0x559fe60eb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.033+0000 7ff9efe00640 1 --2- 172.21.15.63:0/3657443992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d481b0 0x559fe60eb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.033+0000 7ff9efe00640 1 -- 172.21.15.63:0/3657443992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 msgr2=0x559fe60f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.033+0000 7ff9efe00640 1 --2- 172.21.15.63:0/3657443992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 0x559fe60f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.033+0000 7ff9efe00640 1 -- 172.21.15.63:0/3657443992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe60f7520 con 0x559fe5d1f2f0 2024-05-27T07:46:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.033+0000 7ff9efe00640 1 --2- 172.21.15.63:0/3657443992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 0x559fe60f0280 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00a9110 tx=0x7ff9e0091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ddc915de02b550 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.034+0000 7ff9dbe00640 1 -- 172.21.15.63:0/3657443992 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0020040 con 0x559fe5d1f2f0 2024-05-27T07:46:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.034+0000 7ff9dbe00640 1 -- 172.21.15.63:0/3657443992 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9e00a8f20 con 0x559fe5d1f2f0 2024-05-27T07:46:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.034+0000 7ff9dbe00640 1 -- 172.21.15.63:0/3657443992 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00a56a0 con 0x559fe5d1f2f0 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.035+0000 7ff9fdb71740 1 -- 172.21.15.63:0/3657443992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 msgr2=0x559fe60f0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.035+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/3657443992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 0x559fe60f0280 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00a9110 tx=0x7ff9e0091040 comp rx=0 tx=0).stop 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.035+0000 7ff9fdb71740 1 -- 172.21.15.63:0/3657443992 shutdown_connections 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.035+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/3657443992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d481b0 0x559fe60eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.035+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/3657443992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d274d0 0x559fe60f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.035+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/3657443992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d1f2f0 0x559fe60f0280 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.035+0000 7ff9fdb71740 1 -- 172.21.15.63:0/3657443992 >> 172.21.15.63:0/3657443992 conn(0x559fe5d3c300 msgr2=0x559fe60f42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.035+0000 7ff9fdb71740 1 -- 172.21.15.63:0/3657443992 shutdown_connections 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.035+0000 7ff9fdb71740 1 -- 172.21.15.63:0/3657443992 wait complete. 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.035+0000 7ff9fdb71740 1 -- start start 2024-05-27T07:46:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.036+0000 7ff9fdb71740 1 Processor -- start 2024-05-27T07:46:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.036+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d1f2f0 0x559fe61070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.036+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 0x559fe6107630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.036+0000 7ff9fdb71740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe616cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.036+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b1ac0 con 0x559fe5d481b0 2024-05-27T07:46:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.036+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b2f30 con 0x559fe5d1f2f0 2024-05-27T07:46:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.036+0000 7ff9fdb71740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01b2190 con 0x559fe5d274d0 2024-05-27T07:46:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.036+0000 7ff9fdb71740 10 client.0 _ll_register_callbacks cb 0x7ff9e00e5ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.037+0000 7ff9f4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe616cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.037+0000 7ff9f4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe616cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51532/0 (socket says 172.21.15.63:51532) 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.037+0000 7ff9f4a00640 1 -- 172.21.15.63:0/2068085265 learned_addr learned my addr 172.21.15.63:0/2068085265 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.037+0000 7ff9efe00640 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d1f2f0 0x559fe61070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.037+0000 7ff9ef400640 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 0x559fe6107630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.037+0000 7ff9f4a00640 1 -- 172.21.15.63:0/2068085265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 msgr2=0x559fe6107630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.037+0000 7ff9f4a00640 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 0x559fe6107630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.037+0000 7ff9f4a00640 1 -- 172.21.15.63:0/2068085265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d1f2f0 msgr2=0x559fe61070f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.037+0000 7ff9f4a00640 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d1f2f0 0x559fe61070f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.037+0000 7ff9f4a00640 1 -- 172.21.15.63:0/2068085265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe616d530 con 0x559fe5d481b0 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.038+0000 7ff9f4a00640 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe616cff0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009ca0 tx=0x7ff9e802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ab423866656c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.038+0000 7ff9d9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.038+0000 7ff9d9600640 1 -- 172.21.15.63:0/2068085265 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8004450 con 0x559fe5d481b0 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.038+0000 7ff9d9600640 1 -- 172.21.15.63:0/2068085265 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7ff9e8042030 con 0x559fe5d481b0 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.038+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2068085265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e00a8f20 con 0x559fe5d481b0 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.039+0000 7ff9d9600640 1 -- 172.21.15.63:0/2068085265 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff9e805d050 con 0x559fe5d481b0 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.039+0000 7ff9fdb71740 10 client.17922 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.039+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2068085265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559fe616e760 con 0x559fe5d481b0 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.039+0000 7ff9d9600640 1 -- 172.21.15.63:0/2068085265 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9e00a8f20 con 0x559fe5d481b0 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.039+0000 7ff9d9600640 1 -- 172.21.15.63:0/2068085265 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff9e8049420 con 0x559fe5d481b0 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.039+0000 7ff9fdb71740 10 client.17922 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.039+0000 7ff9fdb71740 10 client.17922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.040+0000 7ff9fdb71740 20 client.17922 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.040+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fe5cdc1c0 0x559fe6170f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.040+0000 7ff9fdb71740 4 client.17922 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:46:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.040+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2068085265 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fe60572c0 con 0x559fe5cdc1c0 2024-05-27T07:46:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.040+0000 7ff9efe00640 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fe5cdc1c0 0x559fe6170f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.041+0000 7ff9efe00640 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fe5cdc1c0 0x559fe6170f80 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=babaadc500e4d6ac server_cookie=5f098eb4dccb4767 in_seq=0 out_seq=0 2024-05-27T07:46:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.041+0000 7ff9d9600640 10 client.17922 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.073+0000 7ff9d9600640 1 -- 172.21.15.63:0/2068085265 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559fe60572c0 con 0x559fe5cdc1c0 2024-05-27T07:46:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.073+0000 7ff9d9600640 10 client.17922 handle_command_reply: tid=0 2024-05-27T07:46:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.073+0000 7ff9fdb71740 1 client.17922 shutdown 2024-05-27T07:46:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.073+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2068085265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 msgr2=0x559fe616cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.073+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe616cff0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009ca0 tx=0x7ff9e802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.073+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2068085265 shutdown_connections 2024-05-27T07:46:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.074+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fe5cdc1c0 0x559fe6170f80 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.074+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fe5d481b0 0x559fe616cff0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.074+0000 7ff9d9600640 0 client.17922 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.074+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fe5d274d0 0x559fe6107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.074+0000 7ff9fdb71740 1 --2- 172.21.15.63:0/2068085265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fe5d1f2f0 0x559fe61070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.074+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2068085265 >> 172.21.15.63:0/2068085265 conn(0x559fe5d3c300 msgr2=0x559fe60eaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.074+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2068085265 shutdown_connections 2024-05-27T07:46:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.074+0000 7ff9fdb71740 1 -- 172.21.15.63:0/2068085265 wait complete. 2024-05-27T07:46:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.074+0000 7ff9fdb71740 20 client.17922 trim_cache size 0 max 16384 2024-05-27T07:46:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:46:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:46:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:46:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 -- 172.21.15.63:0/466134537 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2950 msgr2=0x7ff9d40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2950 0x7ff9d40b4e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8004af0 tx=0x7ff9e8005da0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 -- 172.21.15.63:0/466134537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0104d80 msgr2=0x7ff9f01a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0104d80 0x7ff9f01a2610 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000d770 tx=0x7ff9e000dc30 comp rx=0 tx=0).stop 2024-05-27T07:46:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 -- 172.21.15.63:0/466134537 shutdown_connections 2024-05-27T07:46:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9d40b2950 0x7ff9d40b4e60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f010e1d0 0x7ff9f019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9f010b7e0 0x7ff9f01a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 --2- 172.21.15.63:0/466134537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9f0104d80 0x7ff9f01a2610 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 -- 172.21.15.63:0/466134537 >> 172.21.15.63:0/466134537 conn(0x7ff9f00fe380 msgr2=0x7ff9f0109e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 -- 172.21.15.63:0/466134537 shutdown_connections 2024-05-27T07:46:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:39.075+0000 7ff9f6a00640 1 -- 172.21.15.63:0/466134537 wait complete. 2024-05-27T07:46:39.297 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:39.623 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:39 smithi063 ceph-mon[31613]: pgmap v409: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:46:39.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:39 smithi086 ceph-mon[34687]: pgmap v409: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:46:39.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:39 smithi149 ceph-mon[35408]: pgmap v409: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:46:41.385 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:41 smithi063 ceph-mon[31613]: pgmap v410: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:46:41.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:41 smithi086 ceph-mon[34687]: pgmap v410: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:46:41.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:41 smithi149 ceph-mon[35408]: pgmap v410: 129 pgs: 129 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:46:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.789+0000 7f1fbac00700 1 -- 172.21.15.63:0/3547839133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb4110b90 msgr2=0x7f1fb4112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.789+0000 7f1fbac00700 1 --2- 172.21.15.63:0/3547839133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb4110b90 0x7f1fb4112ff0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b0a0 tx=0x7f1fac00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.789+0000 7f1fbac00700 1 -- 172.21.15.63:0/3547839133 shutdown_connections 2024-05-27T07:46:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.789+0000 7f1fbac00700 1 --2- 172.21.15.63:0/3547839133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb4110b90 0x7f1fb4112ff0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.789+0000 7f1fbac00700 1 --2- 172.21.15.63:0/3547839133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb4075d20 0x7f1fb4076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.789+0000 7f1fbac00700 1 --2- 172.21.15.63:0/3547839133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb4076670 0x7f1fb4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.789+0000 7f1fbac00700 1 -- 172.21.15.63:0/3547839133 >> 172.21.15.63:0/3547839133 conn(0x7f1fb4100030 msgr2=0x7f1fb4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.789+0000 7f1fbac00700 1 -- 172.21.15.63:0/3547839133 shutdown_connections 2024-05-27T07:46:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.789+0000 7f1fbac00700 1 -- 172.21.15.63:0/3547839133 wait complete. 2024-05-27T07:46:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.791+0000 7f1fbac00700 1 Processor -- start 2024-05-27T07:46:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.791+0000 7f1fbac00700 1 -- start start 2024-05-27T07:46:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fbac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb4075d20 0x7f1fb419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fbac00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb4076670 0x7f1fb419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fbac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb4110b90 0x7f1fb41a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fbac00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb4116520 con 0x7f1fb4076670 2024-05-27T07:46:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fbac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41163a0 con 0x7f1fb4110b90 2024-05-27T07:46:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fbac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41166a0 con 0x7f1fb4075d20 2024-05-27T07:46:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb4076670 0x7f1fb419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb4076670 0x7f1fb419d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51540/0 (socket says 172.21.15.63:51540) 2024-05-27T07:46:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fb3400700 1 -- 172.21.15.63:0/2500081262 learned_addr learned my addr 172.21.15.63:0/2500081262 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb4075d20 0x7f1fb419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.792+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb4110b90 0x7f1fb41a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.793+0000 7f1fb3400700 1 -- 172.21.15.63:0/2500081262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb4075d20 msgr2=0x7f1fb419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.793+0000 7f1fb3400700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb4075d20 0x7f1fb419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.793+0000 7f1fb3400700 1 -- 172.21.15.63:0/2500081262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb4110b90 msgr2=0x7f1fb41a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.793+0000 7f1fb3400700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb4110b90 0x7f1fb41a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.793+0000 7f1fb3400700 1 -- 172.21.15.63:0/2500081262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb41a1e00 con 0x7f1fb4076670 2024-05-27T07:46:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.793+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb4075d20 0x7f1fb419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.793+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb4110b90 0x7f1fb41a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.793+0000 7f1fb3400700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb4076670 0x7f1fb419d4d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000e9a0 tx=0x7f1fa000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.794+0000 7f1fb0c00700 1 -- 172.21.15.63:0/2500081262 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000cbd0 con 0x7f1fb4076670 2024-05-27T07:46:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.794+0000 7f1fb0c00700 1 -- 172.21.15.63:0/2500081262 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fa000cd70 con 0x7f1fb4076670 2024-05-27T07:46:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.794+0000 7f1fbac00700 1 -- 172.21.15.63:0/2500081262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb41a20f0 con 0x7f1fb4076670 2024-05-27T07:46:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.794+0000 7f1fb0c00700 1 -- 172.21.15.63:0/2500081262 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0018680 con 0x7f1fb4076670 2024-05-27T07:46:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.794+0000 7f1fbac00700 1 -- 172.21.15.63:0/2500081262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb41a9a10 con 0x7f1fb4076670 2024-05-27T07:46:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.799+0000 7f1fb0c00700 1 -- 172.21.15.63:0/2500081262 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1fa0010950 con 0x7f1fb4076670 2024-05-27T07:46:41.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.799+0000 7f1fc97d0b80 1 Processor -- start 2024-05-27T07:46:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.800+0000 7f1fc97d0b80 1 -- start start 2024-05-27T07:46:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.800+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4280ce40 0x555f429917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.800+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f4298b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.800+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 0x555f4298c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.800+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555f428b9e40 con 0x555f42991d30 2024-05-27T07:46:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.800+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f428a99e0 con 0x555f4280ce40 2024-05-27T07:46:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.800+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555f428ada20 con 0x555f4298bdb0 2024-05-27T07:46:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.801+0000 7f1fb3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f4298b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.801+0000 7f1fb8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4280ce40 0x555f429917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.801+0000 7f1fb3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f4298b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51552/0 (socket says 172.21.15.63:51552) 2024-05-27T07:46:41.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.801+0000 7f1fb3e00700 1 -- 172.21.15.63:0/3311334901 learned_addr learned my addr 172.21.15.63:0/3311334901 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.801+0000 7f1fb3400700 1 --2- 172.21.15.63:0/3311334901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 0x555f4298c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.802+0000 7f1fb3e00700 1 -- 172.21.15.63:0/3311334901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 msgr2=0x555f4298c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.802+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/3311334901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 0x555f4298c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.802+0000 7f1fb3e00700 1 -- 172.21.15.63:0/3311334901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4280ce40 msgr2=0x555f429917f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.802+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/3311334901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4280ce40 0x555f429917f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.802+0000 7f1fb3e00700 1 -- 172.21.15.63:0/3311334901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f42872420 con 0x555f42991d30 2024-05-27T07:46:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.802+0000 7f1fb0c00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f980b29f0 0x7f1f980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.802+0000 7f1fb0c00700 1 -- 172.21.15.63:0/2500081262 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f1fa00e2cb0 con 0x7f1fb4076670 2024-05-27T07:46:41.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.802+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f980b29f0 0x7f1f980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.803+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/3311334901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f4298b870 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009de0 tx=0x7f1fa40116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b56f16cc4e58a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.803+0000 7f1fa9600700 1 -- 172.21.15.63:0/3311334901 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4011e20 con 0x555f42991d30 2024-05-27T07:46:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.803+0000 7f1fa9600700 1 -- 172.21.15.63:0/3311334901 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fa40128e0 con 0x555f42991d30 2024-05-27T07:46:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.803+0000 7f1fa9600700 1 -- 172.21.15.63:0/3311334901 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400f840 con 0x555f42991d30 2024-05-27T07:46:41.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.803+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f980b29f0 0x7f1f980b4f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1fac006130 tx=0x7f1fac0060a0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.804+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3311334901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 msgr2=0x555f4298b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.804+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/3311334901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f4298b870 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009de0 tx=0x7f1fa40116c0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.805+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3311334901 shutdown_connections 2024-05-27T07:46:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.805+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/3311334901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 0x555f4298c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.805+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/3311334901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4280ce40 0x555f429917f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.805+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/3311334901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f4298b870 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.805+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3311334901 >> 172.21.15.63:0/3311334901 conn(0x555f4286cfc0 msgr2=0x555f429868d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.805+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3311334901 shutdown_connections 2024-05-27T07:46:41.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.805+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3311334901 wait complete. 2024-05-27T07:46:41.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.805+0000 7f1fc97d0b80 1 -- start start 2024-05-27T07:46:41.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.806+0000 7f1fc97d0b80 1 Processor -- start 2024-05-27T07:46:41.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.806+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4280ce40 0x555f42995440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.806+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f4298bdb0 0x555f42995980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.806+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f42991d30 0x555f42995ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.806+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555f427affa0 con 0x555f4298bdb0 2024-05-27T07:46:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.806+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f4284a760 con 0x555f42991d30 2024-05-27T07:46:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.806+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555f42998800 con 0x555f4280ce40 2024-05-27T07:46:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fc97d0b80 10 client.0 _ll_register_callbacks cb 0x555f42853c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f4298bdb0 0x555f42995980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f4298bdb0 0x555f42995980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51562/0 (socket says 172.21.15.63:51562) 2024-05-27T07:46:41.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb3400700 1 -- 172.21.15.63:0/33413039 learned_addr learned my addr 172.21.15.63:0/33413039 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:41.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4280ce40 0x555f42995440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f42991d30 0x555f42995ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb3400700 1 -- 172.21.15.63:0/33413039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4280ce40 msgr2=0x555f42995440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb3400700 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4280ce40 0x555f42995440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb3400700 1 -- 172.21.15.63:0/33413039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f42991d30 msgr2=0x555f42995ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb3400700 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f42991d30 0x555f42995ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb3400700 1 -- 172.21.15.63:0/33413039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f427f98d0 con 0x555f4298bdb0 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.807+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f42991d30 0x555f42995ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.808+0000 7f1fb3400700 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f4298bdb0 0x555f42995980 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00a6e40 tx=0x7f1fa00b2880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66af72eb76dfa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.808+0000 7f1f9e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.808+0000 7f1f9e000700 1 -- 172.21.15.63:0/33413039 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00a99a0 con 0x555f4298bdb0 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.808+0000 7f1f9e000700 1 -- 172.21.15.63:0/33413039 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f1fa0010640 con 0x555f4298bdb0 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.808+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/33413039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa40128e0 con 0x555f4298bdb0 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.808+0000 7f1f9e000700 1 -- 172.21.15.63:0/33413039 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1fa00c3020 con 0x555f4298bdb0 2024-05-27T07:46:41.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.809+0000 7f1fc97d0b80 10 client.17934 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.809+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/33413039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555f4287a060 con 0x555f4298bdb0 2024-05-27T07:46:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.809+0000 7f1f9e000700 1 -- 172.21.15.63:0/33413039 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fa40128e0 con 0x555f4298bdb0 2024-05-27T07:46:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.809+0000 7f1f9e000700 1 -- 172.21.15.63:0/33413039 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1fa00a5040 con 0x555f4298bdb0 2024-05-27T07:46:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.809+0000 7f1fc97d0b80 10 client.17934 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:41.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.809+0000 7f1fc97d0b80 10 client.17934 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:46:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.809+0000 7f1fc97d0b80 20 client.17934 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.809+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x555f429a5690 0x555f429a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.809+0000 7f1fc97d0b80 4 client.17934 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.809+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/33413039 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f428bbd90 con 0x555f429a5690 2024-05-27T07:46:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.810+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x555f429a5690 0x555f429a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.811+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x555f429a5690 0x555f429a7ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b74084e27ca91a22 server_cookie=8552537b2a8666da in_seq=0 out_seq=0 2024-05-27T07:46:41.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.811+0000 7f1f9e000700 10 client.17934 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:46:41.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1f9e000700 1 -- 172.21.15.63:0/33413039 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555f428bbd90 con 0x555f429a5690 2024-05-27T07:46:41.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1f9e000700 10 client.17934 handle_command_reply: tid=0 2024-05-27T07:46:41.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1fc97d0b80 1 client.17934 shutdown 2024-05-27T07:46:41.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/33413039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f4298bdb0 msgr2=0x555f42995980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:41.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f4298bdb0 0x555f42995980 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00a6e40 tx=0x7f1fa00b2880 comp rx=0 tx=0).stop 2024-05-27T07:46:41.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/33413039 shutdown_connections 2024-05-27T07:46:41.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f42991d30 0x555f42995ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4280ce40 0x555f42995440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x555f429a5690 0x555f429a7ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/33413039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f4298bdb0 0x555f42995980 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/33413039 >> 172.21.15.63:0/33413039 conn(0x555f4286cfc0 msgr2=0x555f42986580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:41.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.813+0000 7f1f9e000700 0 client.17934 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:46:41.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.814+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/33413039 shutdown_connections 2024-05-27T07:46:41.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.814+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/33413039 wait complete. 2024-05-27T07:46:41.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.814+0000 7f1fc97d0b80 20 client.17934 trim_cache size 0 max 16384 2024-05-27T07:46:41.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.832+0000 7f1fc97d0b80 1 Processor -- start 2024-05-27T07:46:41.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.832+0000 7f1fc97d0b80 1 -- start start 2024-05-27T07:46:41.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.832+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4280ce40 0x555f4298b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.832+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4298bdb0 0x555f4298fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.832+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.832+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41b5050 con 0x555f42991d30 2024-05-27T07:46:41.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.832+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41b28e0 con 0x555f4298bdb0 2024-05-27T07:46:41.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.832+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41b0750 con 0x555f4280ce40 2024-05-27T07:46:41.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.833+0000 7f1fb8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.833+0000 7f1fb8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51572/0 (socket says 172.21.15.63:51572) 2024-05-27T07:46:41.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.833+0000 7f1fb8c00700 1 -- 172.21.15.63:0/2885154781 learned_addr learned my addr 172.21.15.63:0/2885154781 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:41.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.833+0000 7f1fb3400700 1 --2- 172.21.15.63:0/2885154781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4298bdb0 0x555f4298fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.833+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/2885154781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4280ce40 0x555f4298b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.833+0000 7f1fb8c00700 1 -- 172.21.15.63:0/2885154781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4280ce40 msgr2=0x555f4298b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:41.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.833+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/2885154781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4280ce40 0x555f4298b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.833+0000 7f1fb8c00700 1 -- 172.21.15.63:0/2885154781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4298bdb0 msgr2=0x555f4298fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:41.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.833+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/2885154781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4298bdb0 0x555f4298fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.834+0000 7f1fb8c00700 1 -- 172.21.15.63:0/2885154781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f42996230 con 0x555f42991d30 2024-05-27T07:46:41.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.834+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/2885154781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429978e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b0a0 tx=0x7f1fac01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadcc645bd4a9339 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:41.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.834+0000 7f1fa8c00700 1 -- 172.21.15.63:0/2885154781 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac023070 con 0x555f42991d30 2024-05-27T07:46:41.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.834+0000 7f1fa8c00700 1 -- 172.21.15.63:0/2885154781 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fac004c50 con 0x555f42991d30 2024-05-27T07:46:41.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.834+0000 7f1fa8c00700 1 -- 172.21.15.63:0/2885154781 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0326d0 con 0x555f42991d30 2024-05-27T07:46:41.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.835+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/2885154781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 msgr2=0x555f429978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:41.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.835+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/2885154781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429978e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b0a0 tx=0x7f1fac01f000 comp rx=0 tx=0).stop 2024-05-27T07:46:41.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.836+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/2885154781 shutdown_connections 2024-05-27T07:46:41.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.836+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/2885154781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429978e0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.836+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/2885154781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4280ce40 0x555f4298b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.836+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/2885154781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4298bdb0 0x555f4298fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.836+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/2885154781 >> 172.21.15.63:0/2885154781 conn(0x555f4286cfc0 msgr2=0x555f42990c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:41.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.836+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/2885154781 shutdown_connections 2024-05-27T07:46:41.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.836+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/2885154781 wait complete. 2024-05-27T07:46:41.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.836+0000 7f1fc97d0b80 1 -- start start 2024-05-27T07:46:41.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.837+0000 7f1fc97d0b80 1 Processor -- start 2024-05-27T07:46:41.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.837+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4280ce40 0x555f4299e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.837+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 0x555f429aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.837+0000 7f1fc97d0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.837+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41b0da0 con 0x555f42991d30 2024-05-27T07:46:41.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.837+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41b2210 con 0x555f4280ce40 2024-05-27T07:46:41.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.837+0000 7f1fc97d0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41b1470 con 0x555f4298bdb0 2024-05-27T07:46:41.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.837+0000 7f1fc97d0b80 10 client.0 _ll_register_callbacks cb 0x555f42853c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:41.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51578/0 (socket says 172.21.15.63:51578) 2024-05-27T07:46:41.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb8c00700 1 -- 172.21.15.63:0/3464384121 learned_addr learned my addr 172.21.15.63:0/3464384121 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:41.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4280ce40 0x555f4299e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb3400700 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 0x555f429aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb8c00700 1 -- 172.21.15.63:0/3464384121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 msgr2=0x555f429aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:41.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 0x555f429aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb8c00700 1 -- 172.21.15.63:0/3464384121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4280ce40 msgr2=0x555f4299e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:41.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4280ce40 0x555f4299e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb8c00700 1 -- 172.21.15.63:0/3464384121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f429ad6b0 con 0x555f42991d30 2024-05-27T07:46:41.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.838+0000 7f1fb3400700 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 0x555f429aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:41.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.839+0000 7f1fb8c00700 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429acf80 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0062a0 tx=0x7f1fac007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704cc677bfc5545 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:41.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.839+0000 7f1f9e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:41.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.839+0000 7f1f9e000700 1 -- 172.21.15.63:0/3464384121 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac032680 con 0x555f42991d30 2024-05-27T07:46:41.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.839+0000 7f1f9e000700 1 -- 172.21.15.63:0/3464384121 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f1fac004e00 con 0x555f42991d30 2024-05-27T07:46:41.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.839+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3464384121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fac004c50 con 0x555f42991d30 2024-05-27T07:46:41.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1f9e000700 1 -- 172.21.15.63:0/3464384121 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1fac032820 con 0x555f42991d30 2024-05-27T07:46:41.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1fc97d0b80 10 client.17940 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:41.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3464384121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555f42a0ffb0 con 0x555f42991d30 2024-05-27T07:46:41.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1f9e000700 1 -- 172.21.15.63:0/3464384121 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fac004c50 con 0x555f42991d30 2024-05-27T07:46:41.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1f9e000700 1 -- 172.21.15.63:0/3464384121 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1fac011070 con 0x555f42991d30 2024-05-27T07:46:41.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1fc97d0b80 10 client.17940 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:41.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1fc97d0b80 10 client.17940 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:46:41.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1fc97d0b80 20 client.17940 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:41.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x555f42a10660 0x555f42a12a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:41.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1fc97d0b80 4 client.17940 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:46:41.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.840+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3464384121 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f428b5570 con 0x555f42a10660 2024-05-27T07:46:41.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.841+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x555f42a10660 0x555f42a12a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:41.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.842+0000 7f1fb3e00700 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x555f42a10660 0x555f42a12a80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6dcdd1468f974ac1 server_cookie=d70c528f77d2b90e in_seq=0 out_seq=0 2024-05-27T07:46:41.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.842+0000 7f1f9e000700 10 client.17940 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:46:41.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1f9e000700 1 -- 172.21.15.63:0/3464384121 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f428b5570 con 0x555f42a10660 2024-05-27T07:46:41.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1f9e000700 10 client.17940 handle_command_reply: tid=0 2024-05-27T07:46:41.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1fc97d0b80 1 client.17940 shutdown 2024-05-27T07:46:41.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3464384121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 msgr2=0x555f429acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:41.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429acf80 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0062a0 tx=0x7f1fac007b00 comp rx=0 tx=0).stop 2024-05-27T07:46:41.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3464384121 shutdown_connections 2024-05-27T07:46:41.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x555f42a10660 0x555f42a12a80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f42991d30 0x555f429acf80 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f4280ce40 0x555f4299e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1fc97d0b80 1 --2- 172.21.15.63:0/3464384121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f4298bdb0 0x555f429aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3464384121 >> 172.21.15.63:0/3464384121 conn(0x555f4286cfc0 msgr2=0x555f42995220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:41.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.843+0000 7f1f9e000700 0 client.17940 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:46:41.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.844+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3464384121 shutdown_connections 2024-05-27T07:46:41.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.844+0000 7f1fc97d0b80 1 -- 172.21.15.63:0/3464384121 wait complete. 2024-05-27T07:46:41.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.844+0000 7f1fc97d0b80 20 client.17940 trim_cache size 0 max 16384 2024-05-27T07:46:41.845 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 -- 172.21.15.63:0/2500081262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f980b29f0 msgr2=0x7f1f980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f980b29f0 0x7f1f980b4f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1fac006130 tx=0x7f1fac0060a0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 -- 172.21.15.63:0/2500081262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb4076670 msgr2=0x7f1fb419d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb4076670 0x7f1fb419d4d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000e9a0 tx=0x7f1fa000ed60 comp rx=0 tx=0).stop 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 -- 172.21.15.63:0/2500081262 shutdown_connections 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f980b29f0 0x7f1f980b4f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb4110b90 0x7f1fb41a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb4075d20 0x7f1fb419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 --2- 172.21.15.63:0/2500081262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb4076670 0x7f1fb419d4d0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 -- 172.21.15.63:0/2500081262 >> 172.21.15.63:0/2500081262 conn(0x7f1fb4100030 msgr2=0x7f1fb4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 -- 172.21.15.63:0/2500081262 shutdown_connections 2024-05-27T07:46:41.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:41.845+0000 7f1fbac00700 1 -- 172.21.15.63:0/2500081262 wait complete. 2024-05-27T07:46:42.907 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:46:43.425 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:43.589 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:43 smithi063 ceph-mon[31613]: pgmap v411: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:46:43.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:43 smithi086 ceph-mon[34687]: pgmap v411: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:46:43.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:43 smithi149 ceph-mon[35408]: pgmap v411: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:46:45.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:45 smithi063 ceph-mon[31613]: pgmap v412: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:46:45.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:45 smithi086 ceph-mon[34687]: pgmap v412: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:46:45.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:45 smithi149 ceph-mon[35408]: pgmap v412: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:46:45.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.864+0000 7fd36bc00700 1 -- 172.21.15.63:0/1182613389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd364109860 msgr2=0x7fd3641111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.864+0000 7fd36bc00700 1 --2- 172.21.15.63:0/1182613389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd364109860 0x7fd3641111c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00b0d0 tx=0x7fd35c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.864+0000 7fd36bc00700 1 -- 172.21.15.63:0/1182613389 shutdown_connections 2024-05-27T07:46:45.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.864+0000 7fd36bc00700 1 --2- 172.21.15.63:0/1182613389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd364109860 0x7fd3641111c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.864+0000 7fd36bc00700 1 --2- 172.21.15.63:0/1182613389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3641045e0 0x7fd3641049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.864+0000 7fd36bc00700 1 --2- 172.21.15.63:0/1182613389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd364108f10 0x7fd364109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.864+0000 7fd36bc00700 1 -- 172.21.15.63:0/1182613389 >> 172.21.15.63:0/1182613389 conn(0x7fd364100030 msgr2=0x7fd364102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:45.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.864+0000 7fd36bc00700 1 -- 172.21.15.63:0/1182613389 shutdown_connections 2024-05-27T07:46:45.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.864+0000 7fd36bc00700 1 -- 172.21.15.63:0/1182613389 wait complete. 2024-05-27T07:46:45.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.866+0000 7fd36bc00700 1 Processor -- start 2024-05-27T07:46:45.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.866+0000 7fd36bc00700 1 -- start start 2024-05-27T07:46:45.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.867+0000 7fd36bc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3641045e0 0x7fd36419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.867+0000 7fd36bc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd364108f10 0x7fd36419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.867+0000 7fd36bc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd364109860 0x7fd36419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.867+0000 7fd36bc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd364114390 con 0x7fd364109860 2024-05-27T07:46:45.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.867+0000 7fd36bc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd364114210 con 0x7fd364108f10 2024-05-27T07:46:45.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.867+0000 7fd36bc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd364114510 con 0x7fd3641045e0 2024-05-27T07:46:45.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.867+0000 7fd369c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd364109860 0x7fd36419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.867+0000 7fd369c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd364109860 0x7fd36419f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51586/0 (socket says 172.21.15.63:51586) 2024-05-27T07:46:45.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.867+0000 7fd369c00700 1 -- 172.21.15.63:0/631087960 learned_addr learned my addr 172.21.15.63:0/631087960 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:45.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.868+0000 7fd369200700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3641045e0 0x7fd36419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.868+0000 7fd363e00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd364108f10 0x7fd36419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.868+0000 7fd369c00700 1 -- 172.21.15.63:0/631087960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3641045e0 msgr2=0x7fd36419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.868+0000 7fd369c00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3641045e0 0x7fd36419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.868+0000 7fd369c00700 1 -- 172.21.15.63:0/631087960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd364108f10 msgr2=0x7fd36419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.868+0000 7fd369c00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd364108f10 0x7fd36419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.868+0000 7fd369c00700 1 -- 172.21.15.63:0/631087960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36419fc10 con 0x7fd364109860 2024-05-27T07:46:45.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.869+0000 7fd369c00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd364109860 0x7fd36419f570 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd35c007d10 tx=0x7fd35c007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:45.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.869+0000 7fd361600700 1 -- 172.21.15.63:0/631087960 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c01f070 con 0x7fd364109860 2024-05-27T07:46:45.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.869+0000 7fd361600700 1 -- 172.21.15.63:0/631087960 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd35c003e10 con 0x7fd364109860 2024-05-27T07:46:45.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.869+0000 7fd36bc00700 1 -- 172.21.15.63:0/631087960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36419fea0 con 0x7fd364109860 2024-05-27T07:46:45.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.869+0000 7fd361600700 1 -- 172.21.15.63:0/631087960 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c01b720 con 0x7fd364109860 2024-05-27T07:46:45.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.870+0000 7fd36bc00700 1 -- 172.21.15.63:0/631087960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3641a0380 con 0x7fd364109860 2024-05-27T07:46:45.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.874+0000 7fd361600700 1 -- 172.21.15.63:0/631087960 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd35c022070 con 0x7fd364109860 2024-05-27T07:46:45.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.875+0000 7fd37a7edb80 1 Processor -- start 2024-05-27T07:46:45.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.875+0000 7fd37a7edb80 1 -- start start 2024-05-27T07:46:45.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.876+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x5603713ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.876+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x5603713ef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.876+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603713efc50 0x560371404cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.876+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56037131de40 con 0x56037135c9f0 2024-05-27T07:46:45.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.876+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56037130d9e0 con 0x5603713efc50 2024-05-27T07:46:45.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.876+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560371311a20 con 0x560371270e40 2024-05-27T07:46:45.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.877+0000 7fd363e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x5603713ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.877+0000 7fd363e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x5603713ef1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51588/0 (socket says 172.21.15.63:51588) 2024-05-27T07:46:45.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.877+0000 7fd363e00700 1 -- 172.21.15.63:0/4215560404 learned_addr learned my addr 172.21.15.63:0/4215560404 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.877+0000 7fd363e00700 1 --2- 172.21.15.63:0/4215560404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603713efc50 0x560371404cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.878+0000 7fd361600700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd34c0b29f0 0x7fd34c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.878+0000 7fd369200700 1 --2- 172.21.15.63:0/4215560404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x5603713ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.878+0000 7fd363e00700 1 -- 172.21.15.63:0/4215560404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 msgr2=0x5603713ef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.878+0000 7fd363e00700 1 --2- 172.21.15.63:0/4215560404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x5603713ef710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.878+0000 7fd363e00700 1 -- 172.21.15.63:0/4215560404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603713efc50 msgr2=0x560371404cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.878+0000 7fd363e00700 1 --2- 172.21.15.63:0/4215560404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603713efc50 0x560371404cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.878+0000 7fd361600700 1 -- 172.21.15.63:0/631087960 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7fd35c00f020 con 0x7fd364109860 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.878+0000 7fd363e00700 1 -- 172.21.15.63:0/4215560404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603712d6420 con 0x56037135c9f0 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.878+0000 7fd369c00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd34c0b29f0 0x7fd34c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.879+0000 7fd363e00700 1 --2- 172.21.15.63:0/4215560404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x5603713ef1d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd350012770 tx=0x7fd35000b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd02435d6b23b44e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:45.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.879+0000 7fd369c00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd34c0b29f0 0x7fd34c0b4f00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd35c007f40 tx=0x7fd35c01f970 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:45.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.879+0000 7fd356000700 1 -- 172.21.15.63:0/4215560404 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350012d90 con 0x56037135c9f0 2024-05-27T07:46:45.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.879+0000 7fd356000700 1 -- 172.21.15.63:0/4215560404 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd350004cc0 con 0x56037135c9f0 2024-05-27T07:46:45.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.880+0000 7fd356000700 1 -- 172.21.15.63:0/4215560404 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35000f650 con 0x56037135c9f0 2024-05-27T07:46:45.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.881+0000 7fd37a7edb80 1 -- 172.21.15.63:0/4215560404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 msgr2=0x5603713ef1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.881+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/4215560404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x5603713ef1d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd350012770 tx=0x7fd35000b750 comp rx=0 tx=0).stop 2024-05-27T07:46:45.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.881+0000 7fd37a7edb80 1 -- 172.21.15.63:0/4215560404 shutdown_connections 2024-05-27T07:46:45.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.881+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/4215560404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603713efc50 0x560371404cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.881+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/4215560404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x5603713ef1d0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.881+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/4215560404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x5603713ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.881+0000 7fd37a7edb80 1 -- 172.21.15.63:0/4215560404 >> 172.21.15.63:0/4215560404 conn(0x56037130a390 msgr2=0x5603713eb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:45.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.882+0000 7fd37a7edb80 1 -- 172.21.15.63:0/4215560404 shutdown_connections 2024-05-27T07:46:45.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.882+0000 7fd37a7edb80 1 -- 172.21.15.63:0/4215560404 wait complete. 2024-05-27T07:46:45.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.884+0000 7fd37a7edb80 1 -- start start 2024-05-27T07:46:45.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.885+0000 7fd37a7edb80 1 Processor -- start 2024-05-27T07:46:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.886+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x560371402280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.886+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037135c9f0 0x5603714027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.886+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371402d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.886+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560371213fa0 con 0x5603713efc50 2024-05-27T07:46:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.886+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5603712ae760 con 0x56037135c9f0 2024-05-27T07:46:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.886+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5603714053e0 con 0x560371270e40 2024-05-27T07:46:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.886+0000 7fd37a7edb80 10 client.0 _ll_register_callbacks cb 0x5603712b7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:45.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.887+0000 7fd363e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371402d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.887+0000 7fd363e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x560371402280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.887+0000 7fd363e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371402d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51592/0 (socket says 172.21.15.63:51592) 2024-05-27T07:46:45.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.887+0000 7fd363e00700 1 -- 172.21.15.63:0/2067950757 learned_addr learned my addr 172.21.15.63:0/2067950757 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:45.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.888+0000 7fd369200700 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037135c9f0 0x5603714027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.888+0000 7fd363e00700 1 -- 172.21.15.63:0/2067950757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 msgr2=0x560371402280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:45.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.888+0000 7fd363e00700 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x560371402280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.888+0000 7fd363e00700 1 -- 172.21.15.63:0/2067950757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037135c9f0 msgr2=0x5603714027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:45.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.888+0000 7fd363e00700 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037135c9f0 0x5603714027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.888+0000 7fd363e00700 1 -- 172.21.15.63:0/2067950757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037125d8d0 con 0x5603713efc50 2024-05-27T07:46:45.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.888+0000 7fd363e00700 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371402d00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x560371401b00 tx=0x7fd3500054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e62d8abdd9c0e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:45.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.889+0000 7fd34aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:45.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.889+0000 7fd34aa00700 1 -- 172.21.15.63:0/2067950757 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350023030 con 0x5603713efc50 2024-05-27T07:46:45.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.889+0000 7fd34aa00700 1 -- 172.21.15.63:0/2067950757 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7fd350005210 con 0x5603713efc50 2024-05-27T07:46:45.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.889+0000 7fd37a7edb80 1 -- 172.21.15.63:0/2067950757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd350004cc0 con 0x5603713efc50 2024-05-27T07:46:45.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.889+0000 7fd34aa00700 1 -- 172.21.15.63:0/2067950757 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd350004750 con 0x5603713efc50 2024-05-27T07:46:45.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.890+0000 7fd37a7edb80 10 client.17952 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:45.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.890+0000 7fd37a7edb80 1 -- 172.21.15.63:0/2067950757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5603712de060 con 0x5603713efc50 2024-05-27T07:46:45.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.891+0000 7fd34aa00700 1 -- 172.21.15.63:0/2067950757 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd350004cc0 con 0x5603713efc50 2024-05-27T07:46:45.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.891+0000 7fd34aa00700 1 -- 172.21.15.63:0/2067950757 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd35002b020 con 0x5603713efc50 2024-05-27T07:46:45.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.891+0000 7fd37a7edb80 10 client.17952 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:45.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.891+0000 7fd37a7edb80 10 client.17952 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:46:45.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.891+0000 7fd37a7edb80 20 client.17952 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:45.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.891+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560371412210 0x560371414630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.891+0000 7fd37a7edb80 4 client.17952 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:45.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.891+0000 7fd37a7edb80 1 -- 172.21.15.63:0/2067950757 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56037131fd90 con 0x560371412210 2024-05-27T07:46:45.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.892+0000 7fd369200700 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560371412210 0x560371414630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.893+0000 7fd34aa00700 10 client.17952 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:46:45.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.893+0000 7fd369200700 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560371412210 0x560371414630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aaf6dddd3ee6f570 server_cookie=f92e83aad4d6209 in_seq=0 out_seq=0 2024-05-27T07:46:45.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd34aa00700 1 -- 172.21.15.63:0/2067950757 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56037131fd90 con 0x560371412210 2024-05-27T07:46:45.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd34aa00700 10 client.17952 handle_command_reply: tid=0 2024-05-27T07:46:45.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd37a7edb80 1 client.17952 shutdown 2024-05-27T07:46:45.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd37a7edb80 1 -- 172.21.15.63:0/2067950757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 msgr2=0x560371402d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371402d00 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x560371401b00 tx=0x7fd3500054d0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd37a7edb80 1 -- 172.21.15.63:0/2067950757 shutdown_connections 2024-05-27T07:46:45.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560371412210 0x560371414630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371402d00 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x560371402280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/2067950757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037135c9f0 0x5603714027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd37a7edb80 1 -- 172.21.15.63:0/2067950757 >> 172.21.15.63:0/2067950757 conn(0x56037130a390 msgr2=0x5603713ef150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:45.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.896+0000 7fd34aa00700 0 client.17952 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:46:45.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.897+0000 7fd37a7edb80 1 -- 172.21.15.63:0/2067950757 shutdown_connections 2024-05-27T07:46:45.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.897+0000 7fd37a7edb80 1 -- 172.21.15.63:0/2067950757 wait complete. 2024-05-27T07:46:45.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.897+0000 7fd37a7edb80 20 client.17952 trim_cache size 0 max 16384 2024-05-27T07:46:45.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.916+0000 7fd37a7edb80 1 Processor -- start 2024-05-27T07:46:45.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.917+0000 7fd37a7edb80 1 -- start start 2024-05-27T07:46:45.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.917+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560371270e40 0x560371401520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.917+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x560371404720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.917+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603713efc50 0x5603713f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.917+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641b0b20 con 0x56037135c9f0 2024-05-27T07:46:45.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.917+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641ae3b0 con 0x560371270e40 2024-05-27T07:46:45.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.917+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641ac220 con 0x5603713efc50 2024-05-27T07:46:45.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.918+0000 7fd369200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x560371404720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.918+0000 7fd369200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x560371404720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51602/0 (socket says 172.21.15.63:51602) 2024-05-27T07:46:45.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.918+0000 7fd369200700 1 -- 172.21.15.63:0/3634332685 learned_addr learned my addr 172.21.15.63:0/3634332685 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:45.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.918+0000 7fd363e00700 1 --2- 172.21.15.63:0/3634332685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560371270e40 0x560371401520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.918+0000 7fd363e00700 1 --2- 172.21.15.63:0/3634332685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603713efc50 0x5603713f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.918+0000 7fd369200700 1 -- 172.21.15.63:0/3634332685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603713efc50 msgr2=0x5603713f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:45.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.918+0000 7fd369200700 1 --2- 172.21.15.63:0/3634332685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603713efc50 0x5603713f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.918+0000 7fd369200700 1 -- 172.21.15.63:0/3634332685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560371270e40 msgr2=0x560371401520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:45.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.918+0000 7fd369200700 1 --2- 172.21.15.63:0/3634332685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560371270e40 0x560371401520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.918+0000 7fd369200700 1 -- 172.21.15.63:0/3634332685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037140e480 con 0x56037135c9f0 2024-05-27T07:46:45.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.919+0000 7fd369200700 1 --2- 172.21.15.63:0/3634332685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x560371404720 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd35801dfd0 tx=0x7fd35800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be841a15ab8b9cf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:45.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.919+0000 7fd355600700 1 -- 172.21.15.63:0/3634332685 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358003be0 con 0x56037135c9f0 2024-05-27T07:46:45.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.919+0000 7fd355600700 1 -- 172.21.15.63:0/3634332685 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd358003d80 con 0x56037135c9f0 2024-05-27T07:46:45.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.919+0000 7fd355600700 1 -- 172.21.15.63:0/3634332685 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358005510 con 0x56037135c9f0 2024-05-27T07:46:45.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.920+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3634332685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 msgr2=0x560371404720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.920+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/3634332685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x560371404720 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd35801dfd0 tx=0x7fd35800bdc0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.921+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3634332685 shutdown_connections 2024-05-27T07:46:45.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.921+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/3634332685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603713efc50 0x5603713f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.921+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/3634332685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560371270e40 0x560371401520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.921+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/3634332685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56037135c9f0 0x560371404720 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.921+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3634332685 >> 172.21.15.63:0/3634332685 conn(0x56037130a390 msgr2=0x5603713eae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:45.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.921+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3634332685 shutdown_connections 2024-05-27T07:46:45.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.921+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3634332685 wait complete. 2024-05-27T07:46:45.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.921+0000 7fd37a7edb80 1 -- start start 2024-05-27T07:46:45.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.922+0000 7fd37a7edb80 1 Processor -- start 2024-05-27T07:46:45.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.922+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x560371418bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.922+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371419130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.922+0000 7fd37a7edb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037141a5a0 0x560371409670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.922+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641ac870 con 0x5603713efc50 2024-05-27T07:46:45.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.922+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641adce0 con 0x56037141a5a0 2024-05-27T07:46:45.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.922+0000 7fd37a7edb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641acf40 con 0x560371270e40 2024-05-27T07:46:45.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.922+0000 7fd37a7edb80 10 client.0 _ll_register_callbacks cb 0x5603712b7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:45.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd369200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371419130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd369200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371419130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51616/0 (socket says 172.21.15.63:51616) 2024-05-27T07:46:45.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd369200700 1 -- 172.21.15.63:0/3940397766 learned_addr learned my addr 172.21.15.63:0/3940397766 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:45.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd363e00700 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x560371418bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd363e00700 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037141a5a0 0x560371409670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd369200700 1 -- 172.21.15.63:0/3940397766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 msgr2=0x560371418bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:45.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd369200700 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x560371418bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd369200700 1 -- 172.21.15.63:0/3940397766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037141a5a0 msgr2=0x560371409670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:45.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd369200700 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037141a5a0 0x560371409670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd369200700 1 -- 172.21.15.63:0/3940397766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603714199a0 con 0x5603713efc50 2024-05-27T07:46:45.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.923+0000 7fd363e00700 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037141a5a0 0x560371409670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:45.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.924+0000 7fd369200700 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371419130 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd3580044f0 tx=0x7fd35800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2947886389ac0d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:45.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.924+0000 7fd34aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:45.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.924+0000 7fd34aa00700 1 -- 172.21.15.63:0/3940397766 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358015070 con 0x5603713efc50 2024-05-27T07:46:45.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.924+0000 7fd34aa00700 1 -- 172.21.15.63:0/3940397766 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7fd358005430 con 0x5603713efc50 2024-05-27T07:46:45.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.924+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3940397766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd358003d80 con 0x5603713efc50 2024-05-27T07:46:45.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd34aa00700 1 -- 172.21.15.63:0/3940397766 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd358038050 con 0x5603713efc50 2024-05-27T07:46:45.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd37a7edb80 10 client.17958 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:45.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3940397766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56037140abc0 con 0x5603713efc50 2024-05-27T07:46:45.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd34aa00700 1 -- 172.21.15.63:0/3940397766 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd358003d80 con 0x5603713efc50 2024-05-27T07:46:45.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd34aa00700 1 -- 172.21.15.63:0/3940397766 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd35801d500 con 0x5603713efc50 2024-05-27T07:46:45.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd37a7edb80 10 client.17958 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:45.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd37a7edb80 10 client.17958 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:46:45.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd37a7edb80 20 client.17958 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:45.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x56037147cad0 0x56037147eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd37a7edb80 4 client.17958 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:46:45.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.925+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3940397766 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560371319570 con 0x56037147cad0 2024-05-27T07:46:45.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.926+0000 7fd363e00700 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x56037147cad0 0x56037147eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.927+0000 7fd363e00700 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x56037147cad0 0x56037147eef0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c160f626840386f server_cookie=1580053e39b0d746 in_seq=0 out_seq=0 2024-05-27T07:46:45.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.927+0000 7fd34aa00700 10 client.17958 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:46:45.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd34aa00700 1 -- 172.21.15.63:0/3940397766 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560371319570 con 0x56037147cad0 2024-05-27T07:46:45.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd34aa00700 10 client.17958 handle_command_reply: tid=0 2024-05-27T07:46:45.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 client.17958 shutdown 2024-05-27T07:46:45.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3940397766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 msgr2=0x560371419130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371419130 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd3580044f0 tx=0x7fd35800b8c0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3940397766 shutdown_connections 2024-05-27T07:46:45.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56037141a5a0 0x560371409670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560371270e40 0x560371418bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x56037147cad0 0x56037147eef0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 --2- 172.21.15.63:0/3940397766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603713efc50 0x560371419130 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3940397766 >> 172.21.15.63:0/3940397766 conn(0x56037130a390 msgr2=0x56037140d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:45.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd34aa00700 0 client.17958 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:46:45.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3940397766 shutdown_connections 2024-05-27T07:46:45.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.928+0000 7fd37a7edb80 1 -- 172.21.15.63:0/3940397766 wait complete. 2024-05-27T07:46:45.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.929+0000 7fd37a7edb80 20 client.17958 trim_cache size 0 max 16384 2024-05-27T07:46:45.938 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:46:45.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.929+0000 7fd36bc00700 1 -- 172.21.15.63:0/631087960 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd34c0b29f0 msgr2=0x7fd34c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.929+0000 7fd36bc00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd34c0b29f0 0x7fd34c0b4f00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd35c007f40 tx=0x7fd35c01f970 comp rx=0 tx=0).stop 2024-05-27T07:46:45.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.929+0000 7fd36bc00700 1 -- 172.21.15.63:0/631087960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd364109860 msgr2=0x7fd36419f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.929+0000 7fd36bc00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd364109860 0x7fd36419f570 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd35c007d10 tx=0x7fd35c007d40 comp rx=0 tx=0).stop 2024-05-27T07:46:45.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.930+0000 7fd36bc00700 1 -- 172.21.15.63:0/631087960 shutdown_connections 2024-05-27T07:46:45.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.930+0000 7fd36bc00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd34c0b29f0 0x7fd34c0b4f00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.930+0000 7fd36bc00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd364109860 0x7fd36419f570 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.930+0000 7fd36bc00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3641045e0 0x7fd36419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.930+0000 7fd36bc00700 1 --2- 172.21.15.63:0/631087960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd364108f10 0x7fd36419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.930+0000 7fd36bc00700 1 -- 172.21.15.63:0/631087960 >> 172.21.15.63:0/631087960 conn(0x7fd364100030 msgr2=0x7fd364108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:45.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.930+0000 7fd36bc00700 1 -- 172.21.15.63:0/631087960 shutdown_connections 2024-05-27T07:46:45.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:45.930+0000 7fd36bc00700 1 -- 172.21.15.63:0/631087960 wait complete. 2024-05-27T07:46:46.986 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:46:47.478 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:47.638 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:47 smithi063 ceph-mon[31613]: pgmap v413: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:46:47.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:47 smithi086 ceph-mon[34687]: pgmap v413: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:46:47.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:47 smithi149 ceph-mon[35408]: pgmap v413: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:46:49.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:49 smithi086 ceph-mon[34687]: pgmap v414: 129 pgs: 129 active+clean; 3.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:46:49.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:49 smithi149 ceph-mon[35408]: pgmap v414: 129 pgs: 129 active+clean; 3.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:46:49.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:49 smithi063 ceph-mon[31613]: pgmap v414: 129 pgs: 129 active+clean; 3.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:46:51.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.180+0000 7f9d4aa00700 1 -- 172.21.15.63:0/2161229719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d441034b0 msgr2=0x7f9d441038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.180+0000 7f9d4aa00700 1 --2- 172.21.15.63:0/2161229719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d441034b0 0x7f9d441038c0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f9d340099c0 tx=0x7f9d34009cd0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.180+0000 7f9d4aa00700 1 -- 172.21.15.63:0/2161229719 shutdown_connections 2024-05-27T07:46:51.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.180+0000 7f9d4aa00700 1 --2- 172.21.15.63:0/2161229719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d44108730 0x7f9d44110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.180+0000 7f9d4aa00700 1 --2- 172.21.15.63:0/2161229719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d441034b0 0x7f9d441038c0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.180+0000 7f9d4aa00700 1 --2- 172.21.15.63:0/2161229719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d44107de0 0x7f9d441081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.180+0000 7f9d4aa00700 1 -- 172.21.15.63:0/2161229719 >> 172.21.15.63:0/2161229719 conn(0x7f9d440fedb0 msgr2=0x7f9d441011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.181+0000 7f9d4aa00700 1 -- 172.21.15.63:0/2161229719 shutdown_connections 2024-05-27T07:46:51.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.181+0000 7f9d4aa00700 1 -- 172.21.15.63:0/2161229719 wait complete. 2024-05-27T07:46:51.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.182+0000 7f9d4aa00700 1 Processor -- start 2024-05-27T07:46:51.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.182+0000 7f9d4aa00700 1 -- start start 2024-05-27T07:46:51.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d4aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d441034b0 0x7f9d440ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d4aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d44107de0 0x7f9d440ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d4aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d44108730 0x7f9d440ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d4aa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44113290 con 0x7f9d44108730 2024-05-27T07:46:51.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d4aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44113110 con 0x7f9d441034b0 2024-05-27T07:46:51.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d4aa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44113410 con 0x7f9d44107de0 2024-05-27T07:46:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d48a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d44108730 0x7f9d440ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d48a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d44108730 0x7f9d440ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40658/0 (socket says 172.21.15.63:40658) 2024-05-27T07:46:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d48a00700 1 -- 172.21.15.63:0/3497916268 learned_addr learned my addr 172.21.15.63:0/3497916268 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d43400700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d44107de0 0x7f9d440ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.183+0000 7f9d43e00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d441034b0 0x7f9d440ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.184+0000 7f9d48a00700 1 -- 172.21.15.63:0/3497916268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d44107de0 msgr2=0x7f9d440ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.184+0000 7f9d48a00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d44107de0 0x7f9d440ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.184+0000 7f9d48a00700 1 -- 172.21.15.63:0/3497916268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d441034b0 msgr2=0x7f9d440ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.184+0000 7f9d48a00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d441034b0 0x7f9d440ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.184+0000 7f9d48a00700 1 -- 172.21.15.63:0/3497916268 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d441a9110 con 0x7f9d44108730 2024-05-27T07:46:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.184+0000 7f9d48a00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d44108730 0x7f9d440ffd10 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00b970 tx=0x7f9d3c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.185+0000 7f9d40c00700 1 -- 172.21.15.63:0/3497916268 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00d860 con 0x7f9d44108730 2024-05-27T07:46:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.185+0000 7f9d40c00700 1 -- 172.21.15.63:0/3497916268 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d3c00e430 con 0x7f9d44108730 2024-05-27T07:46:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.185+0000 7f9d4aa00700 1 -- 172.21.15.63:0/3497916268 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d441a9400 con 0x7f9d44108730 2024-05-27T07:46:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.185+0000 7f9d40c00700 1 -- 172.21.15.63:0/3497916268 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c017660 con 0x7f9d44108730 2024-05-27T07:46:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.185+0000 7f9d4aa00700 1 -- 172.21.15.63:0/3497916268 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d441a9940 con 0x7f9d44108730 2024-05-27T07:46:51.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.190+0000 7f9d40c00700 1 -- 172.21.15.63:0/3497916268 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9d3c00da00 con 0x7f9d44108730 2024-05-27T07:46:51.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.191+0000 7f9d59518b80 1 Processor -- start 2024-05-27T07:46:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.191+0000 7f9d59518b80 1 -- start start 2024-05-27T07:46:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.191+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 0x55e706296330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.191+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706117e40 0x55e706296870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.191+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706296e70 0x55e7062a3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.191+0000 7f9d59518b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e7061c4e40 con 0x55e706296e70 2024-05-27T07:46:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.191+0000 7f9d59518b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e7061b49e0 con 0x55e7062039f0 2024-05-27T07:46:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.191+0000 7f9d59518b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e7061b8a20 con 0x55e706117e40 2024-05-27T07:46:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.192+0000 7f9d43400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706296e70 0x55e7062a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.192+0000 7f9d40c00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d280b2a70 0x7f9d280b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.192+0000 7f9d43400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706296e70 0x55e7062a3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40664/0 (socket says 172.21.15.63:40664) 2024-05-27T07:46:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.192+0000 7f9d43400700 1 -- 172.21.15.63:0/3441728514 learned_addr learned my addr 172.21.15.63:0/3441728514 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:51.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.192+0000 7f9d43e00700 1 --2- 172.21.15.63:0/3441728514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706117e40 0x55e706296870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.192+0000 7f9d43400700 1 --2- 172.21.15.63:0/3441728514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 0x55e706296330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.192+0000 7f9d40c00700 1 -- 172.21.15.63:0/3497916268 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f9d3c0e25d0 con 0x7f9d44108730 2024-05-27T07:46:51.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.192+0000 7f9d48a00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d280b2a70 0x7f9d280b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.193+0000 7f9d43400700 1 -- 172.21.15.63:0/3441728514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706117e40 msgr2=0x55e706296870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.193+0000 7f9d43400700 1 --2- 172.21.15.63:0/3441728514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706117e40 0x55e706296870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.193+0000 7f9d43400700 1 -- 172.21.15.63:0/3441728514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 msgr2=0x55e706296330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.193+0000 7f9d43400700 1 --2- 172.21.15.63:0/3441728514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 0x55e706296330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.193+0000 7f9d43400700 1 -- 172.21.15.63:0/3441728514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e70617d420 con 0x55e706296e70 2024-05-27T07:46:51.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.193+0000 7f9d48a00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d280b2a70 0x7f9d280b4f80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0df9c0 tx=0x7f9d3c01b000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.193+0000 7f9d43400700 1 --2- 172.21.15.63:0/3441728514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706296e70 0x55e7062a3110 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x55e706298ff0 tx=0x7f9d30005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c29c86685a2ccdb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.194+0000 7f9d39600700 1 -- 172.21.15.63:0/3441728514 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30012dd0 con 0x55e706296e70 2024-05-27T07:46:51.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.194+0000 7f9d39600700 1 -- 172.21.15.63:0/3441728514 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d30004cc0 con 0x55e706296e70 2024-05-27T07:46:51.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.194+0000 7f9d39600700 1 -- 172.21.15.63:0/3441728514 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3000f3f0 con 0x55e706296e70 2024-05-27T07:46:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.195+0000 7f9d59518b80 1 -- 172.21.15.63:0/3441728514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706296e70 msgr2=0x55e7062a3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.195+0000 7f9d59518b80 1 --2- 172.21.15.63:0/3441728514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706296e70 0x55e7062a3110 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x55e706298ff0 tx=0x7f9d30005d20 comp rx=0 tx=0).stop 2024-05-27T07:46:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.195+0000 7f9d59518b80 1 -- 172.21.15.63:0/3441728514 shutdown_connections 2024-05-27T07:46:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.195+0000 7f9d59518b80 1 --2- 172.21.15.63:0/3441728514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706296e70 0x55e7062a3110 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.195+0000 7f9d59518b80 1 --2- 172.21.15.63:0/3441728514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 0x55e706296330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.195+0000 7f9d59518b80 1 --2- 172.21.15.63:0/3441728514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706117e40 0x55e706296870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.195+0000 7f9d59518b80 1 -- 172.21.15.63:0/3441728514 >> 172.21.15.63:0/3441728514 conn(0x55e7061b1390 msgr2=0x55e7062924c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.195+0000 7f9d59518b80 1 -- 172.21.15.63:0/3441728514 shutdown_connections 2024-05-27T07:46:51.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.195+0000 7f9d59518b80 1 -- 172.21.15.63:0/3441728514 wait complete. 2024-05-27T07:46:51.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.196+0000 7f9d59518b80 1 -- start start 2024-05-27T07:46:51.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.196+0000 7f9d59518b80 1 Processor -- start 2024-05-27T07:46:51.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e7062a0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 0x55e7062a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706296e70 0x55e7062a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d59518b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e7060bafa0 con 0x55e706117e40 2024-05-27T07:46:51.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d59518b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e706155760 con 0x55e7062039f0 2024-05-27T07:46:51.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d59518b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e7062a3b80 con 0x55e706296e70 2024-05-27T07:46:51.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d59518b80 10 client.0 _ll_register_callbacks cb 0x55e70615ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d43400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e7062a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d43400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e7062a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40670/0 (socket says 172.21.15.63:40670) 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d43400700 1 -- 172.21.15.63:0/719325028 learned_addr learned my addr 172.21.15.63:0/719325028 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.197+0000 7f9d43e00700 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 0x55e7062a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.198+0000 7f9d43400700 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706296e70 0x55e7062a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.198+0000 7f9d43400700 1 -- 172.21.15.63:0/719325028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706296e70 msgr2=0x55e7062a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.198+0000 7f9d43400700 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706296e70 0x55e7062a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.198+0000 7f9d43400700 1 -- 172.21.15.63:0/719325028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 msgr2=0x55e7062a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.198+0000 7f9d43400700 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 0x55e7062a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.198+0000 7f9d43400700 1 -- 172.21.15.63:0/719325028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7061048d0 con 0x55e706117e40 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.198+0000 7f9d43e00700 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 0x55e7062a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.198+0000 7f9d43400700 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e7062a0660 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000b9b0 tx=0x7f9d3000b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65614bfc18307b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.198+0000 7f9d2e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.199+0000 7f9d2e000700 1 -- 172.21.15.63:0/719325028 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30004960 con 0x55e706117e40 2024-05-27T07:46:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.199+0000 7f9d59518b80 1 -- 172.21.15.63:0/719325028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d30004cc0 con 0x55e706117e40 2024-05-27T07:46:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.199+0000 7f9d2e000700 1 -- 172.21.15.63:0/719325028 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f9d30004050 con 0x55e706117e40 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.199+0000 7f9d2e000700 1 -- 172.21.15.63:0/719325028 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9d3002f050 con 0x55e706117e40 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.199+0000 7f9d59518b80 10 client.17970 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.199+0000 7f9d59518b80 1 -- 172.21.15.63:0/719325028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e706185060 con 0x55e706117e40 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.199+0000 7f9d2e000700 1 -- 172.21.15.63:0/719325028 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d30004cc0 con 0x55e706117e40 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.200+0000 7f9d2e000700 1 -- 172.21.15.63:0/719325028 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9d30003a60 con 0x55e706117e40 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.200+0000 7f9d59518b80 10 client.17970 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.200+0000 7f9d59518b80 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.200+0000 7f9d59518b80 20 client.17970 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.200+0000 7f9d59518b80 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e7062b09d0 0x55e7062b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.200+0000 7f9d59518b80 4 client.17970 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.200+0000 7f9d59518b80 1 -- 172.21.15.63:0/719325028 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7061c6d90 con 0x55e7062b09d0 2024-05-27T07:46:51.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.201+0000 7f9d43e00700 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e7062b09d0 0x55e7062b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.202+0000 7f9d43e00700 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e7062b09d0 0x55e7062b2df0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8602225e54365e2c server_cookie=e79d782188019de6 in_seq=0 out_seq=0 2024-05-27T07:46:51.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.202+0000 7f9d2e000700 10 client.17970 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:51.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.204+0000 7f9d2e000700 1 -- 172.21.15.63:0/719325028 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e7061c6d90 con 0x55e7062b09d0 2024-05-27T07:46:51.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.204+0000 7f9d2e000700 10 client.17970 handle_command_reply: tid=0 2024-05-27T07:46:51.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.204+0000 7f9d59518b80 1 client.17970 shutdown 2024-05-27T07:46:51.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 1 -- 172.21.15.63:0/719325028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 msgr2=0x55e7062a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e7062a0660 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000b9b0 tx=0x7f9d3000b470 comp rx=0 tx=0).stop 2024-05-27T07:46:51.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 1 -- 172.21.15.63:0/719325028 shutdown_connections 2024-05-27T07:46:51.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e7062b09d0 0x55e7062b2df0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706296e70 0x55e7062a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e7062a0660 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 1 --2- 172.21.15.63:0/719325028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7062039f0 0x55e7062a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 1 -- 172.21.15.63:0/719325028 >> 172.21.15.63:0/719325028 conn(0x55e7061b1390 msgr2=0x55e7062963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d2e000700 0 client.17970 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:51.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 1 -- 172.21.15.63:0/719325028 shutdown_connections 2024-05-27T07:46:51.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 1 -- 172.21.15.63:0/719325028 wait complete. 2024-05-27T07:46:51.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.205+0000 7f9d59518b80 20 client.17970 trim_cache size 0 max 16384 2024-05-27T07:46:51.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.222+0000 7f9d59518b80 1 Processor -- start 2024-05-27T07:46:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.222+0000 7f9d59518b80 1 -- start start 2024-05-27T07:46:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.223+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e70629be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.223+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7062039f0 0x55e706298900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.223+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706296e70 0x55e7062967b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.223+0000 7f9d59518b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b5000 con 0x55e706117e40 2024-05-27T07:46:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.223+0000 7f9d59518b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b2890 con 0x55e706296e70 2024-05-27T07:46:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.223+0000 7f9d59518b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b0700 con 0x55e7062039f0 2024-05-27T07:46:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.223+0000 7f9d43400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e70629be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.223+0000 7f9d43400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e70629be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40682/0 (socket says 172.21.15.63:40682) 2024-05-27T07:46:51.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.223+0000 7f9d43400700 1 -- 172.21.15.63:0/24179310 learned_addr learned my addr 172.21.15.63:0/24179310 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:51.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.224+0000 7f9d43e00700 1 --2- 172.21.15.63:0/24179310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7062039f0 0x55e706298900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.224+0000 7f9d43400700 1 --2- 172.21.15.63:0/24179310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706296e70 0x55e7062967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.224+0000 7f9d43400700 1 -- 172.21.15.63:0/24179310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7062039f0 msgr2=0x55e706298900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.224+0000 7f9d43400700 1 --2- 172.21.15.63:0/24179310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7062039f0 0x55e706298900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.224+0000 7f9d43400700 1 -- 172.21.15.63:0/24179310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706296e70 msgr2=0x55e7062967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.224+0000 7f9d43400700 1 --2- 172.21.15.63:0/24179310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706296e70 0x55e7062967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.224+0000 7f9d43400700 1 -- 172.21.15.63:0/24179310 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7062a1450 con 0x55e706117e40 2024-05-27T07:46:51.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.224+0000 7f9d43400700 1 --2- 172.21.15.63:0/24179310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e70629be30 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000b750 tx=0x7f9d30005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1573b95aeb4d7cf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.225+0000 7f9d38c00700 1 -- 172.21.15.63:0/24179310 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3000bc00 con 0x55e706117e40 2024-05-27T07:46:51.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.225+0000 7f9d38c00700 1 -- 172.21.15.63:0/24179310 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d30016040 con 0x55e706117e40 2024-05-27T07:46:51.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.225+0000 7f9d38c00700 1 -- 172.21.15.63:0/24179310 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30004270 con 0x55e706117e40 2024-05-27T07:46:51.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.226+0000 7f9d59518b80 1 -- 172.21.15.63:0/24179310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 msgr2=0x55e70629be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.226+0000 7f9d59518b80 1 --2- 172.21.15.63:0/24179310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e70629be30 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000b750 tx=0x7f9d30005230 comp rx=0 tx=0).stop 2024-05-27T07:46:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.226+0000 7f9d59518b80 1 -- 172.21.15.63:0/24179310 shutdown_connections 2024-05-27T07:46:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.226+0000 7f9d59518b80 1 --2- 172.21.15.63:0/24179310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706296e70 0x55e7062967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.226+0000 7f9d59518b80 1 --2- 172.21.15.63:0/24179310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e706117e40 0x55e70629be30 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.226+0000 7f9d59518b80 1 --2- 172.21.15.63:0/24179310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7062039f0 0x55e706298900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.226+0000 7f9d59518b80 1 -- 172.21.15.63:0/24179310 >> 172.21.15.63:0/24179310 conn(0x55e7061b1390 msgr2=0x55e70629ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.226+0000 7f9d59518b80 1 -- 172.21.15.63:0/24179310 shutdown_connections 2024-05-27T07:46:51.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.226+0000 7f9d59518b80 1 -- 172.21.15.63:0/24179310 wait complete. 2024-05-27T07:46:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.227+0000 7f9d59518b80 1 -- start start 2024-05-27T07:46:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.227+0000 7f9d59518b80 1 Processor -- start 2024-05-27T07:46:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706117e40 0x55e70629a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7062039f0 0x55e70629aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d59518b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706296e70 0x55e70629afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d59518b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b0d50 con 0x55e7062039f0 2024-05-27T07:46:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d59518b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b21c0 con 0x55e706117e40 2024-05-27T07:46:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d59518b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b1420 con 0x55e706296e70 2024-05-27T07:46:51.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d59518b80 10 client.0 _ll_register_callbacks cb 0x55e70615ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d43e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7062039f0 0x55e70629aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d43e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7062039f0 0x55e70629aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40684/0 (socket says 172.21.15.63:40684) 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d43e00700 1 -- 172.21.15.63:0/1122179811 learned_addr learned my addr 172.21.15.63:0/1122179811 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.228+0000 7f9d43400700 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706117e40 0x55e70629a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.229+0000 7f9d43400700 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706296e70 0x55e70629afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.229+0000 7f9d43e00700 1 -- 172.21.15.63:0/1122179811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706296e70 msgr2=0x55e70629afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.229+0000 7f9d43e00700 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706296e70 0x55e70629afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.229+0000 7f9d43e00700 1 -- 172.21.15.63:0/1122179811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706117e40 msgr2=0x55e70629a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.229+0000 7f9d43e00700 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706117e40 0x55e70629a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.229+0000 7f9d43e00700 1 -- 172.21.15.63:0/1122179811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7062a7f10 con 0x55e7062039f0 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.229+0000 7f9d43400700 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706117e40 0x55e70629a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.229+0000 7f9d43e00700 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7062039f0 0x55e70629aa90 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9d340098e0 tx=0x7f9d34005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589287d595dee5aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.229+0000 7f9d2e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.229+0000 7f9d2e000700 1 -- 172.21.15.63:0/1122179811 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3400baa0 con 0x55e7062039f0 2024-05-27T07:46:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.230+0000 7f9d2e000700 1 -- 172.21.15.63:0/1122179811 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f9d3400bc40 con 0x55e7062039f0 2024-05-27T07:46:51.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.230+0000 7f9d59518b80 1 -- 172.21.15.63:0/1122179811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d30016040 con 0x55e7062039f0 2024-05-27T07:46:51.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.230+0000 7f9d2e000700 1 -- 172.21.15.63:0/1122179811 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9d3400be40 con 0x55e7062039f0 2024-05-27T07:46:51.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.230+0000 7f9d59518b80 10 client.17976 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:51.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.230+0000 7f9d59518b80 1 -- 172.21.15.63:0/1122179811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e7062a91a0 con 0x55e7062039f0 2024-05-27T07:46:51.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.230+0000 7f9d2e000700 1 -- 172.21.15.63:0/1122179811 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d30016040 con 0x55e7062039f0 2024-05-27T07:46:51.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.231+0000 7f9d2e000700 1 -- 172.21.15.63:0/1122179811 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9d3400f760 con 0x55e7062039f0 2024-05-27T07:46:51.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.231+0000 7f9d59518b80 10 client.17976 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.231+0000 7f9d59518b80 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.231+0000 7f9d59518b80 20 client.17976 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.231+0000 7f9d59518b80 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e70631b260 0x55e70631d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.231+0000 7f9d59518b80 4 client.17976 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.231+0000 7f9d59518b80 1 -- 172.21.15.63:0/1122179811 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7061c0570 con 0x55e70631b260 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.231+0000 7f9d43400700 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e70631b260 0x55e70631d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.233+0000 7f9d2e000700 10 client.17976 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.233+0000 7f9d43400700 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e70631b260 0x55e70631d680 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=503fabfc476b4567 server_cookie=45b782f78f11d027 in_seq=0 out_seq=0 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.233+0000 7f9d2e000700 1 -- 172.21.15.63:0/1122179811 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7061c0570 con 0x55e70631b260 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.233+0000 7f9d2e000700 10 client.17976 handle_command_reply: tid=0 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.233+0000 7f9d59518b80 1 client.17976 shutdown 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.233+0000 7f9d59518b80 1 -- 172.21.15.63:0/1122179811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7062039f0 msgr2=0x55e70629aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.233+0000 7f9d59518b80 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7062039f0 0x55e70629aa90 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f9d340098e0 tx=0x7f9d34005720 comp rx=0 tx=0).stop 2024-05-27T07:46:51.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.234+0000 7f9d59518b80 1 -- 172.21.15.63:0/1122179811 shutdown_connections 2024-05-27T07:46:51.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.234+0000 7f9d59518b80 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e70631b260 0x55e70631d680 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.234+0000 7f9d59518b80 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e706296e70 0x55e70629afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.234+0000 7f9d59518b80 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e706117e40 0x55e70629a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.234+0000 7f9d59518b80 1 --2- 172.21.15.63:0/1122179811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7062039f0 0x55e70629aa90 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.234+0000 7f9d59518b80 1 -- 172.21.15.63:0/1122179811 >> 172.21.15.63:0/1122179811 conn(0x55e7061b1390 msgr2=0x55e706298660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.234+0000 7f9d2e000700 0 client.17976 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:51.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.234+0000 7f9d59518b80 1 -- 172.21.15.63:0/1122179811 shutdown_connections 2024-05-27T07:46:51.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.234+0000 7f9d59518b80 1 -- 172.21.15.63:0/1122179811 wait complete. 2024-05-27T07:46:51.237 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:46:51.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.234+0000 7f9d59518b80 20 client.17976 trim_cache size 0 max 16384 2024-05-27T07:46:51.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.235+0000 7f9d4aa00700 1 -- 172.21.15.63:0/3497916268 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d280b2a70 msgr2=0x7f9d280b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.235+0000 7f9d4aa00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d280b2a70 0x7f9d280b4f80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0df9c0 tx=0x7f9d3c01b000 comp rx=0 tx=0).stop 2024-05-27T07:46:51.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.235+0000 7f9d4aa00700 1 -- 172.21.15.63:0/3497916268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d44108730 msgr2=0x7f9d440ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.235+0000 7f9d4aa00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d44108730 0x7f9d440ffd10 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00b970 tx=0x7f9d3c00bc80 comp rx=0 tx=0).stop 2024-05-27T07:46:51.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.235+0000 7f9d4aa00700 1 -- 172.21.15.63:0/3497916268 shutdown_connections 2024-05-27T07:46:51.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.235+0000 7f9d4aa00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9d280b2a70 0x7f9d280b4f80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.235+0000 7f9d4aa00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d44108730 0x7f9d440ffd10 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.235+0000 7f9d4aa00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d441034b0 0x7f9d440ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.235+0000 7f9d4aa00700 1 --2- 172.21.15.63:0/3497916268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d44107de0 0x7f9d440ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.235+0000 7f9d4aa00700 1 -- 172.21.15.63:0/3497916268 >> 172.21.15.63:0/3497916268 conn(0x7f9d440fedb0 msgr2=0x7f9d44106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.236+0000 7f9d4aa00700 1 -- 172.21.15.63:0/3497916268 shutdown_connections 2024-05-27T07:46:51.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:51.236+0000 7f9d4aa00700 1 -- 172.21.15.63:0/3497916268 wait complete. 2024-05-27T07:46:51.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:51 smithi063 ceph-mon[31613]: pgmap v415: 129 pgs: 129 active+clean; 3.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:46:51.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:51 smithi086 ceph-mon[34687]: pgmap v415: 129 pgs: 129 active+clean; 3.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:46:51.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:51 smithi149 ceph-mon[35408]: pgmap v415: 129 pgs: 129 active+clean; 3.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:46:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.333+0000 7f8301a00640 1 -- 172.21.15.63:0/2624378918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82fc075660 msgr2=0x7f82fc075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.333+0000 7f8301a00640 1 --2- 172.21.15.63:0/2624378918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82fc075660 0x7f82fc075a70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009900 tx=0x7f82ec02a480 comp rx=0 tx=0).stop 2024-05-27T07:46:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.333+0000 7f8301a00640 1 -- 172.21.15.63:0/2624378918 shutdown_connections 2024-05-27T07:46:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.333+0000 7f8301a00640 1 --2- 172.21.15.63:0/2624378918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82fc1102a0 0x7f82fc112700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.333+0000 7f8301a00640 1 --2- 172.21.15.63:0/2624378918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc10d940 0x7f82fc10fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.333+0000 7f8301a00640 1 --2- 172.21.15.63:0/2624378918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82fc075660 0x7f82fc075a70 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.333+0000 7f8301a00640 1 -- 172.21.15.63:0/2624378918 >> 172.21.15.63:0/2624378918 conn(0x7f82fc079140 msgr2=0x7f82fc079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.333+0000 7f8301a00640 1 -- 172.21.15.63:0/2624378918 shutdown_connections 2024-05-27T07:46:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.333+0000 7f8301a00640 1 -- 172.21.15.63:0/2624378918 wait complete. 2024-05-27T07:46:52.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.334+0000 7f8301a00640 1 Processor -- start 2024-05-27T07:46:52.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.335+0000 7f8301a00640 1 -- start start 2024-05-27T07:46:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.335+0000 7f8301a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82fc075660 0x7f82fc19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.335+0000 7f8301a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82fc10d940 0x7f82fc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.335+0000 7f8301a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1102a0 0x7f82fc1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.335+0000 7f8301a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc115d20 con 0x7f82fc075660 2024-05-27T07:46:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.335+0000 7f8301a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc115ba0 con 0x7f82fc1102a0 2024-05-27T07:46:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.335+0000 7f8301a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc115ea0 con 0x7f82fc10d940 2024-05-27T07:46:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f82fbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82fc075660 0x7f82fc19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f82fbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82fc075660 0x7f82fc19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40702/0 (socket says 172.21.15.63:40702) 2024-05-27T07:46:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f82fbe00640 1 -- 172.21.15.63:0/3356866667 learned_addr learned my addr 172.21.15.63:0/3356866667 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f8301000640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1102a0 0x7f82fc1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f82fb400640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82fc10d940 0x7f82fc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f82fbe00640 1 -- 172.21.15.63:0/3356866667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82fc10d940 msgr2=0x7f82fc19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f82fbe00640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82fc10d940 0x7f82fc19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f82fbe00640 1 -- 172.21.15.63:0/3356866667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1102a0 msgr2=0x7f82fc1a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f82fbe00640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1102a0 0x7f82fc1a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f82fbe00640 1 -- 172.21.15.63:0/3356866667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc1a15c0 con 0x7f82fc075660 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.336+0000 7f8301000640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1102a0 0x7f82fc1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.337+0000 7f82fbe00640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82fc075660 0x7f82fc19c7e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009a20 tx=0x7f82ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.337+0000 7f82f8c00640 1 -- 172.21.15.63:0/3356866667 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec03f030 con 0x7f82fc075660 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.337+0000 7f8301a00640 1 -- 172.21.15.63:0/3356866667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc1a1850 con 0x7f82fc075660 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.337+0000 7f82f8c00640 1 -- 172.21.15.63:0/3356866667 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82ec003a80 con 0x7f82fc075660 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.337+0000 7f82f8c00640 1 -- 172.21.15.63:0/3356866667 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec035070 con 0x7f82fc075660 2024-05-27T07:46:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.337+0000 7f8301a00640 1 -- 172.21.15.63:0/3356866667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc1a1d90 con 0x7f82fc075660 2024-05-27T07:46:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.341+0000 7f8309fc4740 1 Processor -- start 2024-05-27T07:46:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.342+0000 7f8309fc4740 1 -- start start 2024-05-27T07:46:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.342+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51024f1b0 0x55a5105f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.342+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 0x55a5105f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.342+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a5105f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.342+0000 7f8309fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a510562b90 con 0x55a51022e4d0 2024-05-27T07:46:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.342+0000 7f8309fc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a510552730 con 0x55a51024f1b0 2024-05-27T07:46:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.342+0000 7f8309fc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a510556770 con 0x55a5102262f0 2024-05-27T07:46:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.342+0000 7f82f8c00640 1 -- 172.21.15.63:0/3356866667 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f82ec042030 con 0x7f82fc075660 2024-05-27T07:46:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.342+0000 7f82fb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a5105f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.343+0000 7f82fb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a5105f24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40708/0 (socket says 172.21.15.63:40708) 2024-05-27T07:46:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.343+0000 7f82fb400640 1 -- 172.21.15.63:0/1343230595 learned_addr learned my addr 172.21.15.63:0/1343230595 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.343+0000 7f8301000640 1 --2- 172.21.15.63:0/1343230595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51024f1b0 0x55a5105f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.343+0000 7f82fbe00640 1 --2- 172.21.15.63:0/1343230595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 0x55a5105f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.343+0000 7f82fb400640 1 -- 172.21.15.63:0/1343230595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 msgr2=0x55a5105f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.343+0000 7f82fb400640 1 --2- 172.21.15.63:0/1343230595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 0x55a5105f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.343+0000 7f82fb400640 1 -- 172.21.15.63:0/1343230595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51024f1b0 msgr2=0x55a5105f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.343+0000 7f82fb400640 1 --2- 172.21.15.63:0/1343230595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51024f1b0 0x55a5105f1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.343+0000 7f82fb400640 1 -- 172.21.15.63:0/1343230595 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5104eb8b0 con 0x55a51022e4d0 2024-05-27T07:46:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.344+0000 7f82fbe00640 1 --2- 172.21.15.63:0/1343230595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 0x55a5105f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.344+0000 7f82fb400640 1 --2- 172.21.15.63:0/1343230595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a5105f24e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f82e800cc40 tx=0x7f82e800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3318a71865f52b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.345+0000 7f82f8c00640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82e00b2950 0x7f82e00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.345+0000 7f82f1600640 1 -- 172.21.15.63:0/1343230595 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8014070 con 0x55a51022e4d0 2024-05-27T07:46:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.345+0000 7f82f1600640 1 -- 172.21.15.63:0/1343230595 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82e80047a0 con 0x55a51022e4d0 2024-05-27T07:46:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.345+0000 7f82f1600640 1 -- 172.21.15.63:0/1343230595 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8010690 con 0x55a51022e4d0 2024-05-27T07:46:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.345+0000 7f82f8c00640 1 -- 172.21.15.63:0/3356866667 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f82ec102fc0 con 0x7f82fc075660 2024-05-27T07:46:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.345+0000 7f8301000640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82e00b2950 0x7f82e00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.346+0000 7f8309fc4740 1 -- 172.21.15.63:0/1343230595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 msgr2=0x55a5105f24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.346+0000 7f8309fc4740 1 --2- 172.21.15.63:0/1343230595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a5105f24e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f82e800cc40 tx=0x7f82e800e720 comp rx=0 tx=0).stop 2024-05-27T07:46:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.346+0000 7f8301000640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82e00b2950 0x7f82e00b4e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55a5105eebd0 tx=0x7f82f4009780 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.346+0000 7f8309fc4740 1 -- 172.21.15.63:0/1343230595 shutdown_connections 2024-05-27T07:46:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.346+0000 7f8309fc4740 1 --2- 172.21.15.63:0/1343230595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a5105f24e0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.346+0000 7f8309fc4740 1 --2- 172.21.15.63:0/1343230595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 0x55a5105f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.346+0000 7f8309fc4740 1 --2- 172.21.15.63:0/1343230595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51024f1b0 0x55a5105f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.346+0000 7f8309fc4740 1 -- 172.21.15.63:0/1343230595 >> 172.21.15.63:0/1343230595 conn(0x55a510243300 msgr2=0x55a5105ee210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.346+0000 7f8309fc4740 1 -- 172.21.15.63:0/1343230595 shutdown_connections 2024-05-27T07:46:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.346+0000 7f8309fc4740 1 -- 172.21.15.63:0/1343230595 wait complete. 2024-05-27T07:46:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.347+0000 7f8309fc4740 1 -- start start 2024-05-27T07:46:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.347+0000 7f8309fc4740 1 Processor -- start 2024-05-27T07:46:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.348+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 0x55a5105fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.348+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 0x55a5105fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.348+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51024f1b0 0x55a510607ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.348+0000 7f8309fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a510238790 con 0x55a51024f1b0 2024-05-27T07:46:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.348+0000 7f8309fc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5104fabc0 con 0x55a51022e4d0 2024-05-27T07:46:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.348+0000 7f8309fc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a5105feb90 con 0x55a5102262f0 2024-05-27T07:46:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.348+0000 7f8309fc4740 10 client.0 _ll_register_callbacks cb 0x55a5104d8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f8301000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51024f1b0 0x55a510607ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f8301000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51024f1b0 0x55a510607ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40712/0 (socket says 172.21.15.63:40712) 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f8301000640 1 -- 172.21.15.63:0/2588883128 learned_addr learned my addr 172.21.15.63:0/2588883128 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f82fbe00640 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 0x55a5105fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f82fb400640 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 0x55a5105fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f8301000640 1 -- 172.21.15.63:0/2588883128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 msgr2=0x55a5105fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f8301000640 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 0x55a5105fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f8301000640 1 -- 172.21.15.63:0/2588883128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 msgr2=0x55a5105fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f8301000640 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 0x55a5105fc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f8301000640 1 -- 172.21.15.63:0/2588883128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a510267210 con 0x55a51024f1b0 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.349+0000 7f82fb400640 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 0x55a5105fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.350+0000 7f8301000640 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51024f1b0 0x55a510607ec0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f82f4011ce0 tx=0x7f82f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaee1282bc4cddbe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.350+0000 7f82e6000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.350+0000 7f82e6000640 1 -- 172.21.15.63:0/2588883128 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4010ea0 con 0x55a51024f1b0 2024-05-27T07:46:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.350+0000 7f82e6000640 1 -- 172.21.15.63:0/2588883128 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f82f40203f0 con 0x55a51024f1b0 2024-05-27T07:46:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.350+0000 7f8309fc4740 1 -- 172.21.15.63:0/2588883128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e80047a0 con 0x55a51024f1b0 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.350+0000 7f82e6000640 1 -- 172.21.15.63:0/2588883128 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f82f4028600 con 0x55a51024f1b0 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.350+0000 7f8309fc4740 10 client.17988 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.350+0000 7f8309fc4740 1 -- 172.21.15.63:0/2588883128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a510333620 con 0x55a51024f1b0 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.351+0000 7f82e6000640 1 -- 172.21.15.63:0/2588883128 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82e80047a0 con 0x55a51024f1b0 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.351+0000 7f82e6000640 1 -- 172.21.15.63:0/2588883128 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f82f400f040 con 0x55a51024f1b0 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.351+0000 7f8309fc4740 10 client.17988 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.351+0000 7f8309fc4740 10 client.17988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.351+0000 7f8309fc4740 20 client.17988 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.351+0000 7f8309fc4740 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a51020dd80 0x55a51060dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.351+0000 7f8309fc4740 4 client.17988 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.351+0000 7f8309fc4740 1 -- 172.21.15.63:0/2588883128 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a510564ae0 con 0x55a51020dd80 2024-05-27T07:46:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.352+0000 7f82fbe00640 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a51020dd80 0x55a51060dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.353+0000 7f82e6000640 10 client.17988 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.353+0000 7f82fbe00640 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a51020dd80 0x55a51060dde0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eff8fe0747fb0cf5 server_cookie=4995d43870e993e9 in_seq=0 out_seq=0 2024-05-27T07:46:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f82e6000640 1 -- 172.21.15.63:0/2588883128 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a510564ae0 con 0x55a51020dd80 2024-05-27T07:46:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f82e6000640 10 client.17988 handle_command_reply: tid=0 2024-05-27T07:46:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f8309fc4740 1 client.17988 shutdown 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f8309fc4740 1 -- 172.21.15.63:0/2588883128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51024f1b0 msgr2=0x55a510607ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f8309fc4740 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51024f1b0 0x55a510607ec0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f82f4011ce0 tx=0x7f82f4004a40 comp rx=0 tx=0).stop 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f8309fc4740 1 -- 172.21.15.63:0/2588883128 shutdown_connections 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f8309fc4740 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a51020dd80 0x55a51060dde0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f8309fc4740 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51024f1b0 0x55a510607ec0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f8309fc4740 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 0x55a5105fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f82e6000640 0 client.17988 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f8309fc4740 1 --2- 172.21.15.63:0/2588883128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5102262f0 0x55a5105fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f8309fc4740 1 -- 172.21.15.63:0/2588883128 >> 172.21.15.63:0/2588883128 conn(0x55a510243300 msgr2=0x55a5105f3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.355+0000 7f8309fc4740 1 -- 172.21.15.63:0/2588883128 shutdown_connections 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.356+0000 7f8309fc4740 1 -- 172.21.15.63:0/2588883128 wait complete. 2024-05-27T07:46:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.356+0000 7f8309fc4740 20 client.17988 trim_cache size 0 max 16384 2024-05-27T07:46:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.376+0000 7f8309fc4740 1 Processor -- start 2024-05-27T07:46:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.376+0000 7f8309fc4740 1 -- start start 2024-05-27T07:46:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5102262f0 0x55a5105fdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a510607c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 0x55a510608170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f8309fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc1b5b10 con 0x55a51022e4d0 2024-05-27T07:46:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f8309fc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc1b33e0 con 0x55a5102262f0 2024-05-27T07:46:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f8309fc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc1b1250 con 0x55a51024f1b0 2024-05-27T07:46:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f82fb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a510607c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f82fb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a510607c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40726/0 (socket says 172.21.15.63:40726) 2024-05-27T07:46:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f82fb400640 1 -- 172.21.15.63:0/3701500773 learned_addr learned my addr 172.21.15.63:0/3701500773 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f82fbe00640 1 --2- 172.21.15.63:0/3701500773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5102262f0 0x55a5105fdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.377+0000 7f8301000640 1 --2- 172.21.15.63:0/3701500773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 0x55a510608170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.378+0000 7f82fb400640 1 -- 172.21.15.63:0/3701500773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 msgr2=0x55a510608170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.378+0000 7f82fb400640 1 --2- 172.21.15.63:0/3701500773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 0x55a510608170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.378+0000 7f82fb400640 1 -- 172.21.15.63:0/3701500773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5102262f0 msgr2=0x55a5105fdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.378+0000 7f82fb400640 1 --2- 172.21.15.63:0/3701500773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5102262f0 0x55a5105fdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.378+0000 7f82fb400640 1 -- 172.21.15.63:0/3701500773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5105f2800 con 0x55a51022e4d0 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.378+0000 7f82fb400640 1 --2- 172.21.15.63:0/3701500773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a510607c30 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f82e800cc40 tx=0x7f82e800c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bbfc5d267ccc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.379+0000 7f82f0c00640 1 -- 172.21.15.63:0/3701500773 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8014040 con 0x55a51022e4d0 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.379+0000 7f82f0c00640 1 -- 172.21.15.63:0/3701500773 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82e8005270 con 0x55a51022e4d0 2024-05-27T07:46:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.379+0000 7f82f0c00640 1 -- 172.21.15.63:0/3701500773 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8023690 con 0x55a51022e4d0 2024-05-27T07:46:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.380+0000 7f8309fc4740 1 -- 172.21.15.63:0/3701500773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 msgr2=0x55a510607c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.380+0000 7f8309fc4740 1 --2- 172.21.15.63:0/3701500773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a510607c30 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f82e800cc40 tx=0x7f82e800c370 comp rx=0 tx=0).stop 2024-05-27T07:46:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.380+0000 7f8309fc4740 1 -- 172.21.15.63:0/3701500773 shutdown_connections 2024-05-27T07:46:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.380+0000 7f8309fc4740 1 --2- 172.21.15.63:0/3701500773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 0x55a510608170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.380+0000 7f8309fc4740 1 --2- 172.21.15.63:0/3701500773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a51022e4d0 0x55a510607c30 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.380+0000 7f8309fc4740 1 --2- 172.21.15.63:0/3701500773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5102262f0 0x55a5105fdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.380+0000 7f8309fc4740 1 -- 172.21.15.63:0/3701500773 >> 172.21.15.63:0/3701500773 conn(0x55a510243300 msgr2=0x55a5105fbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.380+0000 7f8309fc4740 1 -- 172.21.15.63:0/3701500773 shutdown_connections 2024-05-27T07:46:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.380+0000 7f8309fc4740 1 -- 172.21.15.63:0/3701500773 wait complete. 2024-05-27T07:46:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.380+0000 7f8309fc4740 1 -- start start 2024-05-27T07:46:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.381+0000 7f8309fc4740 1 Processor -- start 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.381+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5102262f0 0x55a51066e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.381+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 0x55a51066ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.381+0000 7f8309fc4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 0x55a510601400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.381+0000 7f8309fc4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc1b18a0 con 0x55a5102262f0 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.381+0000 7f8309fc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc1b2d10 con 0x55a51022e4d0 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.381+0000 7f8309fc4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc1b1f70 con 0x55a51024f1b0 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.381+0000 7f8309fc4740 10 client.0 _ll_register_callbacks cb 0x7f82f40099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.381+0000 7f82fbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5102262f0 0x55a51066e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f82fbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5102262f0 0x55a51066e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40732/0 (socket says 172.21.15.63:40732) 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f82fbe00640 1 -- 172.21.15.63:0/1099333782 learned_addr learned my addr 172.21.15.63:0/1099333782 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f82fb400640 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 0x55a51066ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f8301000640 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 0x55a510601400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f82fbe00640 1 -- 172.21.15.63:0/1099333782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 msgr2=0x55a510601400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f82fbe00640 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 0x55a510601400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f82fbe00640 1 -- 172.21.15.63:0/1099333782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 msgr2=0x55a51066ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f82fbe00640 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 0x55a51066ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f82fbe00640 1 -- 172.21.15.63:0/1099333782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a510601a40 con 0x55a5102262f0 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f8301000640 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 0x55a510601400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f82fbe00640 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5102262f0 0x55a51066e850 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0c5f60 tx=0x7f82ec003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2916614a56d8d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.382+0000 7f82e6000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.383+0000 7f82e6000640 1 -- 172.21.15.63:0/1099333782 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec049040 con 0x55a5102262f0 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.383+0000 7f82e6000640 1 -- 172.21.15.63:0/1099333782 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f82ec1005d0 con 0x55a5102262f0 2024-05-27T07:46:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.383+0000 7f8309fc4740 1 -- 172.21.15.63:0/1099333782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e8005270 con 0x55a5102262f0 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.383+0000 7f82e6000640 1 -- 172.21.15.63:0/1099333782 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f82ec0ce290 con 0x55a5102262f0 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.383+0000 7f8309fc4740 10 client.17994 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.383+0000 7f8309fc4740 1 -- 172.21.15.63:0/1099333782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a510675f20 con 0x55a5102262f0 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.383+0000 7f82e6000640 1 -- 172.21.15.63:0/1099333782 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82e8005270 con 0x55a5102262f0 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.384+0000 7f82e6000640 1 -- 172.21.15.63:0/1099333782 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f82ec0d28b0 con 0x55a5102262f0 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.384+0000 7f8309fc4740 10 client.17994 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.384+0000 7f8309fc4740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.384+0000 7f8309fc4740 20 client.17994 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.384+0000 7f8309fc4740 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a5101e31c0 0x55a5106786c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.384+0000 7f8309fc4740 4 client.17994 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:46:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.384+0000 7f8309fc4740 1 -- 172.21.15.63:0/1099333782 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a51055e2c0 con 0x55a5101e31c0 2024-05-27T07:46:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.384+0000 7f82fb400640 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a5101e31c0 0x55a5106786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.385+0000 7f82fb400640 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a5101e31c0 0x55a5106786c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4985853f93c954de server_cookie=6803aa52f13967ee in_seq=0 out_seq=0 2024-05-27T07:46:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.385+0000 7f82e6000640 10 client.17994 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.420+0000 7f82e6000640 1 -- 172.21.15.63:0/1099333782 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a51055e2c0 con 0x55a5101e31c0 2024-05-27T07:46:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.420+0000 7f82e6000640 10 client.17994 handle_command_reply: tid=0 2024-05-27T07:46:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 client.17994 shutdown 2024-05-27T07:46:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 -- 172.21.15.63:0/1099333782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5102262f0 msgr2=0x55a51066e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5102262f0 0x55a51066e850 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0c5f60 tx=0x7f82ec003e60 comp rx=0 tx=0).stop 2024-05-27T07:46:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 -- 172.21.15.63:0/1099333782 shutdown_connections 2024-05-27T07:46:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a5101e31c0 0x55a5106786c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a51024f1b0 0x55a510601400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a51022e4d0 0x55a51066ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 --2- 172.21.15.63:0/1099333782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5102262f0 0x55a51066e850 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 -- 172.21.15.63:0/1099333782 >> 172.21.15.63:0/1099333782 conn(0x55a510243300 msgr2=0x55a5105fbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f82e6000640 0 client.17994 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 -- 172.21.15.63:0/1099333782 shutdown_connections 2024-05-27T07:46:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.421+0000 7f8309fc4740 1 -- 172.21.15.63:0/1099333782 wait complete. 2024-05-27T07:46:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.422+0000 7f8309fc4740 20 client.17994 trim_cache size 0 max 16384 2024-05-27T07:46:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:46:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:46:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:46:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 -- 172.21.15.63:0/3356866667 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82e00b2950 msgr2=0x7f82e00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82e00b2950 0x7f82e00b4e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55a5105eebd0 tx=0x7f82f4009780 comp rx=0 tx=0).stop 2024-05-27T07:46:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 -- 172.21.15.63:0/3356866667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82fc075660 msgr2=0x7f82fc19c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82fc075660 0x7f82fc19c7e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009a20 tx=0x7f82ec004270 comp rx=0 tx=0).stop 2024-05-27T07:46:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 -- 172.21.15.63:0/3356866667 shutdown_connections 2024-05-27T07:46:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82e00b2950 0x7f82e00b4e60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82fc1102a0 0x7f82fc1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82fc10d940 0x7f82fc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 --2- 172.21.15.63:0/3356866667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82fc075660 0x7f82fc19c7e0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 -- 172.21.15.63:0/3356866667 >> 172.21.15.63:0/3356866667 conn(0x7f82fc079140 msgr2=0x7f82fc102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 -- 172.21.15.63:0/3356866667 shutdown_connections 2024-05-27T07:46:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:46:52.423+0000 7f8301a00640 1 -- 172.21.15.63:0/3356866667 wait complete. 2024-05-27T07:46:53.242 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:46:53.242 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:46:53.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:53 smithi086 ceph-mon[34687]: pgmap v416: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:46:53.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:53 smithi149 ceph-mon[35408]: pgmap v416: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:46:53.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:53 smithi063 ceph-mon[31613]: pgmap v416: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:46:53.751 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:55.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:55 smithi063 ceph-mon[31613]: pgmap v417: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:46:55.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:55 smithi086 ceph-mon[34687]: pgmap v417: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:46:55.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:55 smithi149 ceph-mon[35408]: pgmap v417: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:46:56.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.345+0000 7f887da00700 1 -- 172.21.15.63:0/178683185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8878108f10 msgr2=0x7f8878109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:56.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.345+0000 7f887da00700 1 --2- 172.21.15.63:0/178683185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8878108f10 0x7f8878109320 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f88640099c0 tx=0x7f8864009cd0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.345+0000 7f887da00700 1 -- 172.21.15.63:0/178683185 shutdown_connections 2024-05-27T07:46:56.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.345+0000 7f887da00700 1 --2- 172.21.15.63:0/178683185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8878109860 0x7f88781111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.345+0000 7f887da00700 1 --2- 172.21.15.63:0/178683185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88781045e0 0x7f88781049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.345+0000 7f887da00700 1 --2- 172.21.15.63:0/178683185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8878108f10 0x7f8878109320 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.345+0000 7f887da00700 1 -- 172.21.15.63:0/178683185 >> 172.21.15.63:0/178683185 conn(0x7f8878100030 msgr2=0x7f8878102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:56.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.345+0000 7f887da00700 1 -- 172.21.15.63:0/178683185 shutdown_connections 2024-05-27T07:46:56.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.345+0000 7f887da00700 1 -- 172.21.15.63:0/178683185 wait complete. 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.347+0000 7f887da00700 1 Processor -- start 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.347+0000 7f887da00700 1 -- start start 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.347+0000 7f887da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88781045e0 0x7f8878075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.347+0000 7f887da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8878108f10 0x7f8878075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.347+0000 7f887da00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8878109860 0x7f8878076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.347+0000 7f887da00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8878114370 con 0x7f88781045e0 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.347+0000 7f887da00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88781141f0 con 0x7f8878108f10 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.347+0000 7f887da00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88781144f0 con 0x7f8878109860 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.348+0000 7f8876a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88781045e0 0x7f8878075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.348+0000 7f8876a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88781045e0 0x7f8878075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40750/0 (socket says 172.21.15.63:40750) 2024-05-27T07:46:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.348+0000 7f8876a00700 1 -- 172.21.15.63:0/2279659301 learned_addr learned my addr 172.21.15.63:0/2279659301 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:56.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.348+0000 7f8876000700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8878108f10 0x7f8878075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.348+0000 7f8877400700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8878109860 0x7f8878076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.349+0000 7f8876a00700 1 -- 172.21.15.63:0/2279659301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8878109860 msgr2=0x7f8878076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.349+0000 7f8876a00700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8878109860 0x7f8878076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.349+0000 7f8876a00700 1 -- 172.21.15.63:0/2279659301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8878108f10 msgr2=0x7f8878075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.349+0000 7f8876a00700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8878108f10 0x7f8878075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.349+0000 7f8876a00700 1 -- 172.21.15.63:0/2279659301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88781a9090 con 0x7f88781045e0 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.349+0000 7f8876000700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8878108f10 0x7f8878075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.349+0000 7f8876a00700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88781045e0 0x7f8878075ab0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f886800c9b0 tx=0x7f886800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.349+0000 7f886f400700 1 -- 172.21.15.63:0/2279659301 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868007b00 con 0x7f88781045e0 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.349+0000 7f887da00700 1 -- 172.21.15.63:0/2279659301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88781a9380 con 0x7f88781045e0 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.350+0000 7f886f400700 1 -- 172.21.15.63:0/2279659301 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8868007ca0 con 0x7f88781045e0 2024-05-27T07:46:56.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.350+0000 7f886f400700 1 -- 172.21.15.63:0/2279659301 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868018660 con 0x7f88781045e0 2024-05-27T07:46:56.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.350+0000 7f887da00700 1 -- 172.21.15.63:0/2279659301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88781a9890 con 0x7f88781045e0 2024-05-27T07:46:56.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.355+0000 7f886f400700 1 -- 172.21.15.63:0/2279659301 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8868004830 con 0x7f88781045e0 2024-05-27T07:46:56.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.357+0000 7f888c4d2b80 1 Processor -- start 2024-05-27T07:46:56.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.357+0000 7f888c4d2b80 1 -- start start 2024-05-27T07:46:56.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.357+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d806aaa70 0x556d8073d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.357+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d805beec0 0x556d8073d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.357+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d8073def0 0x556d8074a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.357+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556d8066bec0 con 0x556d805beec0 2024-05-27T07:46:56.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.357+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d8065ba60 con 0x556d8073def0 2024-05-27T07:46:56.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.357+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556d8065faa0 con 0x556d806aaa70 2024-05-27T07:46:56.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.358+0000 7f8876a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d805beec0 0x556d8073d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.358+0000 7f886f400700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f885c0b29f0 0x7f885c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.358+0000 7f8876a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d805beec0 0x556d8073d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40758/0 (socket says 172.21.15.63:40758) 2024-05-27T07:46:56.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.358+0000 7f8876a00700 1 -- 172.21.15.63:0/3573614862 learned_addr learned my addr 172.21.15.63:0/3573614862 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:56.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.358+0000 7f8877400700 1 --2- 172.21.15.63:0/3573614862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d806aaa70 0x556d8073d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.358+0000 7f8876000700 1 --2- 172.21.15.63:0/3573614862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d8073def0 0x556d8074a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.358+0000 7f886f400700 1 -- 172.21.15.63:0/2279659301 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f88680e2a10 con 0x7f88781045e0 2024-05-27T07:46:56.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.358+0000 7f8877400700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f885c0b29f0 0x7f885c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.359+0000 7f8876a00700 1 -- 172.21.15.63:0/3573614862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d806aaa70 msgr2=0x556d8073d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:56.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.359+0000 7f8876a00700 1 --2- 172.21.15.63:0/3573614862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d806aaa70 0x556d8073d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.359+0000 7f8876a00700 1 -- 172.21.15.63:0/3573614862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d8073def0 msgr2=0x556d8074a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:56.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.359+0000 7f8876a00700 1 --2- 172.21.15.63:0/3573614862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d8073def0 0x556d8074a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.359+0000 7f8876a00700 1 -- 172.21.15.63:0/3573614862 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d806244a0 con 0x556d805beec0 2024-05-27T07:46:56.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.359+0000 7f8876a00700 1 --2- 172.21.15.63:0/3573614862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d805beec0 0x556d8073d8f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f88680a6ba0 tx=0x7f886800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc79f25582793e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:56.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.359+0000 7f8877400700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f885c0b29f0 0x7f885c0b4f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x556d80740070 tx=0x7f8870009100 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:56.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.359+0000 7f8863e00700 1 -- 172.21.15.63:0/3573614862 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88680a9440 con 0x556d805beec0 2024-05-27T07:46:56.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.360+0000 7f8863e00700 1 -- 172.21.15.63:0/3573614862 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f886808fe50 con 0x556d805beec0 2024-05-27T07:46:56.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.360+0000 7f8863e00700 1 -- 172.21.15.63:0/3573614862 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88680a5bc0 con 0x556d805beec0 2024-05-27T07:46:56.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.361+0000 7f888c4d2b80 1 -- 172.21.15.63:0/3573614862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d805beec0 msgr2=0x556d8073d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:56.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.361+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/3573614862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d805beec0 0x556d8073d8f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f88680a6ba0 tx=0x7f886800f000 comp rx=0 tx=0).stop 2024-05-27T07:46:56.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.361+0000 7f888c4d2b80 1 -- 172.21.15.63:0/3573614862 shutdown_connections 2024-05-27T07:46:56.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.361+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/3573614862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d8073def0 0x556d8074a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.361+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/3573614862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d806aaa70 0x556d8073d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.361+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/3573614862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d805beec0 0x556d8073d8f0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.361+0000 7f888c4d2b80 1 -- 172.21.15.63:0/3573614862 >> 172.21.15.63:0/3573614862 conn(0x556d80658410 msgr2=0x556d80739540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:56.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.361+0000 7f888c4d2b80 1 -- 172.21.15.63:0/3573614862 shutdown_connections 2024-05-27T07:46:56.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.361+0000 7f888c4d2b80 1 -- 172.21.15.63:0/3573614862 wait complete. 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.362+0000 7f888c4d2b80 1 -- start start 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.362+0000 7f888c4d2b80 1 Processor -- start 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d805beec0 0x556d807477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d806aaa70 0x556d80747d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d8073def0 0x556d80748260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556d80562020 con 0x556d806aaa70 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d805fc7e0 con 0x556d805beec0 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556d8074ab40 con 0x556d8073def0 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f888c4d2b80 10 client.0 _ll_register_callbacks cb 0x556d80605cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f8876000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d806aaa70 0x556d80747d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f8876000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d806aaa70 0x556d80747d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40764/0 (socket says 172.21.15.63:40764) 2024-05-27T07:46:56.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f8876000700 1 -- 172.21.15.63:0/1484466453 learned_addr learned my addr 172.21.15.63:0/1484466453 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:56.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.363+0000 7f8877400700 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d8073def0 0x556d80748260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.364+0000 7f8876a00700 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d805beec0 0x556d807477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.364+0000 7f8876000700 1 -- 172.21.15.63:0/1484466453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d8073def0 msgr2=0x556d80748260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:56.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.364+0000 7f8876000700 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d8073def0 0x556d80748260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.364+0000 7f8876000700 1 -- 172.21.15.63:0/1484466453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d805beec0 msgr2=0x556d807477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:56.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.364+0000 7f8876000700 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d805beec0 0x556d807477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.364+0000 7f8876000700 1 -- 172.21.15.63:0/1484466453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d805ab950 con 0x556d806aaa70 2024-05-27T07:46:56.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.364+0000 7f8876000700 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d806aaa70 0x556d80747d20 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f88640099c0 tx=0x7f886400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200377eb875ed0b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:56.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.364+0000 7f8860c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:56.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.365+0000 7f8860c00700 1 -- 172.21.15.63:0/1484466453 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88640055a0 con 0x556d806aaa70 2024-05-27T07:46:56.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.365+0000 7f8860c00700 1 -- 172.21.15.63:0/1484466453 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f8864003680 con 0x556d806aaa70 2024-05-27T07:46:56.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.365+0000 7f888c4d2b80 1 -- 172.21.15.63:0/1484466453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f886808fe50 con 0x556d806aaa70 2024-05-27T07:46:56.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.365+0000 7f8860c00700 1 -- 172.21.15.63:0/1484466453 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8864030050 con 0x556d806aaa70 2024-05-27T07:46:56.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.365+0000 7f888c4d2b80 10 client.18006 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:56.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.365+0000 7f888c4d2b80 1 -- 172.21.15.63:0/1484466453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556d8062c0e0 con 0x556d806aaa70 2024-05-27T07:46:56.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.366+0000 7f8860c00700 1 -- 172.21.15.63:0/1484466453 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f886808fe50 con 0x556d806aaa70 2024-05-27T07:46:56.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.366+0000 7f8860c00700 1 -- 172.21.15.63:0/1484466453 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f886400f770 con 0x556d806aaa70 2024-05-27T07:46:56.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.366+0000 7f888c4d2b80 10 client.18006 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:56.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.366+0000 7f888c4d2b80 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:56.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.366+0000 7f888c4d2b80 20 client.18006 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:56.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.366+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d80757ab0 0x556d80759f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.366+0000 7f888c4d2b80 4 client.18006 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:56.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.366+0000 7f888c4d2b80 1 -- 172.21.15.63:0/1484466453 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d8066de10 con 0x556d80757ab0 2024-05-27T07:46:56.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.367+0000 7f8876a00700 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d80757ab0 0x556d80759f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.368+0000 7f8860c00700 10 client.18006 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:56.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.368+0000 7f8876a00700 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d80757ab0 0x556d80759f80 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87542c60b10c8bc0 server_cookie=34355ece66f20dc in_seq=0 out_seq=0 2024-05-27T07:46:56.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f8860c00700 1 -- 172.21.15.63:0/1484466453 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556d8066de10 con 0x556d80757ab0 2024-05-27T07:46:56.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f8860c00700 10 client.18006 handle_command_reply: tid=0 2024-05-27T07:46:56.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f888c4d2b80 1 client.18006 shutdown 2024-05-27T07:46:56.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f888c4d2b80 1 -- 172.21.15.63:0/1484466453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d806aaa70 msgr2=0x556d80747d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:56.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d806aaa70 0x556d80747d20 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f88640099c0 tx=0x7f886400bee0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f888c4d2b80 1 -- 172.21.15.63:0/1484466453 shutdown_connections 2024-05-27T07:46:56.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d80757ab0 0x556d80759f80 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d8073def0 0x556d80748260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d805beec0 0x556d807477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/1484466453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d806aaa70 0x556d80747d20 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f8860c00700 0 client.18006 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:56.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.370+0000 7f888c4d2b80 1 -- 172.21.15.63:0/1484466453 >> 172.21.15.63:0/1484466453 conn(0x556d80658410 msgr2=0x556d8073d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:56.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.371+0000 7f888c4d2b80 1 -- 172.21.15.63:0/1484466453 shutdown_connections 2024-05-27T07:46:56.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.371+0000 7f888c4d2b80 1 -- 172.21.15.63:0/1484466453 wait complete. 2024-05-27T07:46:56.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.371+0000 7f888c4d2b80 20 client.18006 trim_cache size 0 max 16384 2024-05-27T07:46:56.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.397+0000 7f888c4d2b80 1 Processor -- start 2024-05-27T07:46:56.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.397+0000 7f888c4d2b80 1 -- start start 2024-05-27T07:46:56.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.397+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d805beec0 0x556d807481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.397+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d806aaa70 0x556d80742f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.397+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d80753be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.397+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88781b4ed0 con 0x556d8073def0 2024-05-27T07:46:56.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.397+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88781b2760 con 0x556d805beec0 2024-05-27T07:46:56.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.397+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88781b05d0 con 0x556d806aaa70 2024-05-27T07:46:56.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.398+0000 7f8877400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d80753be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.398+0000 7f8877400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d80753be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40774/0 (socket says 172.21.15.63:40774) 2024-05-27T07:46:56.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.398+0000 7f8877400700 1 -- 172.21.15.63:0/4285459324 learned_addr learned my addr 172.21.15.63:0/4285459324 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:56.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.398+0000 7f8876a00700 1 --2- 172.21.15.63:0/4285459324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d805beec0 0x556d807481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.398+0000 7f8876000700 1 --2- 172.21.15.63:0/4285459324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d806aaa70 0x556d80742f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.398+0000 7f8877400700 1 -- 172.21.15.63:0/4285459324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d806aaa70 msgr2=0x556d80742f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.398+0000 7f8877400700 1 --2- 172.21.15.63:0/4285459324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d806aaa70 0x556d80742f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.398+0000 7f8877400700 1 -- 172.21.15.63:0/4285459324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d805beec0 msgr2=0x556d807481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.398+0000 7f8877400700 1 --2- 172.21.15.63:0/4285459324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d805beec0 0x556d807481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.399+0000 7f8877400700 1 -- 172.21.15.63:0/4285459324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d8073aa40 con 0x556d8073def0 2024-05-27T07:46:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.399+0000 7f8877400700 1 --2- 172.21.15.63:0/4285459324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d80753be0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8870019630 tx=0x7f887000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe6a999617a5b16 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.399+0000 7f8863400700 1 -- 172.21.15.63:0/4285459324 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870004bb0 con 0x556d8073def0 2024-05-27T07:46:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.399+0000 7f8863400700 1 -- 172.21.15.63:0/4285459324 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8870004d50 con 0x556d8073def0 2024-05-27T07:46:56.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.399+0000 7f8863400700 1 -- 172.21.15.63:0/4285459324 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870025e00 con 0x556d8073def0 2024-05-27T07:46:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.401+0000 7f888c4d2b80 1 -- 172.21.15.63:0/4285459324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 msgr2=0x556d80753be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.401+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/4285459324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d80753be0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8870019630 tx=0x7f887000b040 comp rx=0 tx=0).stop 2024-05-27T07:46:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.401+0000 7f888c4d2b80 1 -- 172.21.15.63:0/4285459324 shutdown_connections 2024-05-27T07:46:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.401+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/4285459324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d80753be0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.401+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/4285459324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d805beec0 0x556d807481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.401+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/4285459324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d806aaa70 0x556d80742f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.401+0000 7f888c4d2b80 1 -- 172.21.15.63:0/4285459324 >> 172.21.15.63:0/4285459324 conn(0x556d80658410 msgr2=0x556d80746740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.401+0000 7f888c4d2b80 1 -- 172.21.15.63:0/4285459324 shutdown_connections 2024-05-27T07:46:56.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.401+0000 7f888c4d2b80 1 -- 172.21.15.63:0/4285459324 wait complete. 2024-05-27T07:46:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.401+0000 7f888c4d2b80 1 -- start start 2024-05-27T07:46:56.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.402+0000 7f888c4d2b80 1 Processor -- start 2024-05-27T07:46:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.402+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d805beec0 0x556d8074fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.402+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d806aaa70 0x556d80750320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.402+0000 7f888c4d2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d8074ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.402+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88781b0c20 con 0x556d8073def0 2024-05-27T07:46:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.402+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88781b2090 con 0x556d806aaa70 2024-05-27T07:46:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.402+0000 7f888c4d2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88781b12f0 con 0x556d805beec0 2024-05-27T07:46:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.402+0000 7f888c4d2b80 10 client.0 _ll_register_callbacks cb 0x556d807382a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:56.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.403+0000 7f8877400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d8074ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.403+0000 7f8877400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d8074ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40788/0 (socket says 172.21.15.63:40788) 2024-05-27T07:46:56.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.403+0000 7f8877400700 1 -- 172.21.15.63:0/2667836119 learned_addr learned my addr 172.21.15.63:0/2667836119 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:46:56.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.403+0000 7f8876000700 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d806aaa70 0x556d80750320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.403+0000 7f8876a00700 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d805beec0 0x556d8074fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.404+0000 7f8877400700 1 -- 172.21.15.63:0/2667836119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d805beec0 msgr2=0x556d8074fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:56.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.404+0000 7f8877400700 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d805beec0 0x556d8074fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.404+0000 7f8877400700 1 -- 172.21.15.63:0/2667836119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d806aaa70 msgr2=0x556d80750320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:56.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.404+0000 7f8877400700 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d806aaa70 0x556d80750320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.404+0000 7f8877400700 1 -- 172.21.15.63:0/2667836119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d807352c0 con 0x556d8073def0 2024-05-27T07:46:56.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.404+0000 7f8877400700 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d8074ad90 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8870009900 tx=0x7f8870025470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaea2218b72427ae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:56.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.404+0000 7f8860c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:46:56.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.404+0000 7f8860c00700 1 -- 172.21.15.63:0/2667836119 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870025920 con 0x556d8073def0 2024-05-27T07:46:56.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.404+0000 7f8860c00700 1 -- 172.21.15.63:0/2667836119 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f8870012070 con 0x556d8073def0 2024-05-27T07:46:56.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.404+0000 7f888c4d2b80 1 -- 172.21.15.63:0/2667836119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8870004d50 con 0x556d8073def0 2024-05-27T07:46:56.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.405+0000 7f8860c00700 1 -- 172.21.15.63:0/2667836119 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f887003d050 con 0x556d8073def0 2024-05-27T07:46:56.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.405+0000 7f888c4d2b80 10 client.18012 fetch_fsmap learned FSMap version 33 2024-05-27T07:46:56.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.405+0000 7f888c4d2b80 1 -- 172.21.15.63:0/2667836119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556d807c2c40 con 0x556d8073def0 2024-05-27T07:46:56.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.405+0000 7f8860c00700 1 -- 172.21.15.63:0/2667836119 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8870004d50 con 0x556d8073def0 2024-05-27T07:46:56.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.405+0000 7f8860c00700 1 -- 172.21.15.63:0/2667836119 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f887002f4a0 con 0x556d8073def0 2024-05-27T07:46:56.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.405+0000 7f888c4d2b80 10 client.18012 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:46:56.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.405+0000 7f888c4d2b80 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:56.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.406+0000 7f888c4d2b80 20 client.18012 populate_metadata read hostname 'smithi063' 2024-05-27T07:46:56.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.406+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d807c32c0 0x556d807c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:56.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.406+0000 7f888c4d2b80 4 client.18012 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:46:56.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.406+0000 7f888c4d2b80 1 -- 172.21.15.63:0/2667836119 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556d806675f0 con 0x556d807c32c0 2024-05-27T07:46:56.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.406+0000 7f8876a00700 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d807c32c0 0x556d807c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:56.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.407+0000 7f8876a00700 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d807c32c0 0x556d807c56e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd3d8237cda7a3c server_cookie=b2b8176edd375799 in_seq=0 out_seq=0 2024-05-27T07:46:56.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.407+0000 7f8860c00700 10 client.18012 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:56.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.410+0000 7f8860c00700 1 -- 172.21.15.63:0/2667836119 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d806675f0 con 0x556d807c32c0 2024-05-27T07:46:56.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.410+0000 7f8860c00700 10 client.18012 handle_command_reply: tid=0 2024-05-27T07:46:56.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.410+0000 7f888c4d2b80 1 client.18012 shutdown 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.411+0000 7f888c4d2b80 1 -- 172.21.15.63:0/2667836119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 msgr2=0x556d8074ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.411+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d8074ad90 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8870009900 tx=0x7f8870025470 comp rx=0 tx=0).stop 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.411+0000 7f888c4d2b80 1 -- 172.21.15.63:0/2667836119 shutdown_connections 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.411+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d807c32c0 0x556d807c56e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.411+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d8073def0 0x556d8074ad90 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.411+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d805beec0 0x556d8074fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.411+0000 7f888c4d2b80 1 --2- 172.21.15.63:0/2667836119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d806aaa70 0x556d80750320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.411+0000 7f888c4d2b80 1 -- 172.21.15.63:0/2667836119 >> 172.21.15.63:0/2667836119 conn(0x556d80658410 msgr2=0x556d8073ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.411+0000 7f8860c00700 0 client.18012 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.411+0000 7f888c4d2b80 1 -- 172.21.15.63:0/2667836119 shutdown_connections 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.412+0000 7f888c4d2b80 1 -- 172.21.15.63:0/2667836119 wait complete. 2024-05-27T07:46:56.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.412+0000 7f888c4d2b80 20 client.18012 trim_cache size 0 max 16384 2024-05-27T07:46:56.413 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:46:56.413 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:46:56.413 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "8b84186b-c9e7-401f-867d-ea59cef1a7ad", 2024-05-27T07:46:56.413 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:46:56.413 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:46:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 -- 172.21.15.63:0/2279659301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f885c0b29f0 msgr2=0x7f885c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f885c0b29f0 0x7f885c0b4f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x556d80740070 tx=0x7f8870009100 comp rx=0 tx=0).stop 2024-05-27T07:46:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 -- 172.21.15.63:0/2279659301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88781045e0 msgr2=0x7f8878075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88781045e0 0x7f8878075ab0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f886800c9b0 tx=0x7f886800ccc0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 -- 172.21.15.63:0/2279659301 shutdown_connections 2024-05-27T07:46:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f885c0b29f0 0x7f885c0b4f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8878109860 0x7f8878076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88781045e0 0x7f8878075ab0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 --2- 172.21.15.63:0/2279659301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8878108f10 0x7f8878075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 -- 172.21.15.63:0/2279659301 >> 172.21.15.63:0/2279659301 conn(0x7f8878100030 msgr2=0x7f887810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 -- 172.21.15.63:0/2279659301 shutdown_connections 2024-05-27T07:46:56.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:46:56.413+0000 7f887da00700 1 -- 172.21.15.63:0/2279659301 wait complete. 2024-05-27T07:46:57.439 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:46:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: pgmap v418: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:46:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:46:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:46:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:46:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:46:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:46:57.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:46:57.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:46:57.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:46:57.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:57 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: pgmap v418: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:46:57.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:46:57.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:46:57.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:46:57.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:46:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:46:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:46:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:46:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:46:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:57 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: pgmap v418: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:46:57.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:46:57.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:46:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:57 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:57.954 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:58.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:58.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:58.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:59.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:46:59 smithi086 ceph-mon[34687]: pgmap v419: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.4 MiB/s wr, 21 op/s 2024-05-27T07:46:59.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:46:59 smithi149 ceph-mon[35408]: pgmap v419: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.4 MiB/s wr, 21 op/s 2024-05-27T07:46:59.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:46:59 smithi063 ceph-mon[31613]: pgmap v419: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.4 MiB/s wr, 21 op/s 2024-05-27T07:47:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.373+0000 7fe098c00700 1 -- 172.21.15.63:0/2705211599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0941043f0 msgr2=0x7fe094104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.373+0000 7fe098c00700 1 --2- 172.21.15.63:0/2705211599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0941043f0 0x7fe094104800 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fe0840099c0 tx=0x7fe084009cd0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.373+0000 7fe098c00700 1 -- 172.21.15.63:0/2705211599 shutdown_connections 2024-05-27T07:47:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.373+0000 7fe098c00700 1 --2- 172.21.15.63:0/2705211599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe094109670 0x7fe094110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.373+0000 7fe098c00700 1 --2- 172.21.15.63:0/2705211599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0941043f0 0x7fe094104800 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.373+0000 7fe098c00700 1 --2- 172.21.15.63:0/2705211599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe094108d20 0x7fe094109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.373+0000 7fe098c00700 1 -- 172.21.15.63:0/2705211599 >> 172.21.15.63:0/2705211599 conn(0x7fe094079d00 msgr2=0x7fe09407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:00.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.374+0000 7fe098c00700 1 -- 172.21.15.63:0/2705211599 shutdown_connections 2024-05-27T07:47:00.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.374+0000 7fe098c00700 1 -- 172.21.15.63:0/2705211599 wait complete. 2024-05-27T07:47:00.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.375+0000 7fe098c00700 1 Processor -- start 2024-05-27T07:47:00.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.375+0000 7fe098c00700 1 -- start start 2024-05-27T07:47:00.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.376+0000 7fe098c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0941043f0 0x7fe0941a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.376+0000 7fe098c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe094108d20 0x7fe0941a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.376+0000 7fe098c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe094109670 0x7fe09419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.376+0000 7fe098c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe094114150 con 0x7fe094109670 2024-05-27T07:47:00.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.376+0000 7fe098c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe094113fd0 con 0x7fe0941043f0 2024-05-27T07:47:00.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.376+0000 7fe098c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941142d0 con 0x7fe094108d20 2024-05-27T07:47:00.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.376+0000 7fe093e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe094109670 0x7fe09419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.377+0000 7fe093e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe094109670 0x7fe09419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56482/0 (socket says 172.21.15.63:56482) 2024-05-27T07:47:00.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.377+0000 7fe093e00700 1 -- 172.21.15.63:0/3770649714 learned_addr learned my addr 172.21.15.63:0/3770649714 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:00.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.377+0000 7fe093400700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0941043f0 0x7fe0941a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.377+0000 7fe092a00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe094108d20 0x7fe0941a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.377+0000 7fe093e00700 1 -- 172.21.15.63:0/3770649714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe094108d20 msgr2=0x7fe0941a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:00.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.377+0000 7fe093e00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe094108d20 0x7fe0941a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.377+0000 7fe093e00700 1 -- 172.21.15.63:0/3770649714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0941043f0 msgr2=0x7fe0941a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:00.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.377+0000 7fe093e00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0941043f0 0x7fe0941a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.377+0000 7fe093e00700 1 -- 172.21.15.63:0/3770649714 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09419b060 con 0x7fe094109670 2024-05-27T07:47:00.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.378+0000 7fe093e00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe094109670 0x7fe09419a9b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fe08c007ee0 tx=0x7fe08c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:00.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.380+0000 7fe08be00700 1 -- 172.21.15.63:0/3770649714 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c004bb0 con 0x7fe094109670 2024-05-27T07:47:00.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.380+0000 7fe08be00700 1 -- 172.21.15.63:0/3770649714 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe08c004d50 con 0x7fe094109670 2024-05-27T07:47:00.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.380+0000 7fe098c00700 1 -- 172.21.15.63:0/3770649714 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09419b350 con 0x7fe094109670 2024-05-27T07:47:00.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.380+0000 7fe08be00700 1 -- 172.21.15.63:0/3770649714 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0058b0 con 0x7fe094109670 2024-05-27T07:47:00.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.381+0000 7fe098c00700 1 -- 172.21.15.63:0/3770649714 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0941a7620 con 0x7fe094109670 2024-05-27T07:47:00.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.385+0000 7fe08be00700 1 -- 172.21.15.63:0/3770649714 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe08c01a020 con 0x7fe094109670 2024-05-27T07:47:00.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.386+0000 7fe0a8d56b80 1 Processor -- start 2024-05-27T07:47:00.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.386+0000 7fe08be00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe0780b2a70 0x7fe0780b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.386+0000 7fe0a8d56b80 1 -- start start 2024-05-27T07:47:00.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.387+0000 7fe08be00700 1 -- 172.21.15.63:0/3770649714 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7fe08c0e1fa0 con 0x7fe094109670 2024-05-27T07:47:00.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.387+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce868c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.387+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce868cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.387+0000 7fe092a00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe0780b2a70 0x7fe0780b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.387+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce86986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.387+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558ce85b9e40 con 0x558ce85f89f0 2024-05-27T07:47:00.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.387+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558ce85a99e0 con 0x558ce850ce40 2024-05-27T07:47:00.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.387+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ce85ada20 con 0x558ce86962c0 2024-05-27T07:47:00.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.387+0000 7fe093400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce868c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.388+0000 7fe093e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce86986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.388+0000 7fe093400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce868c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56486/0 (socket says 172.21.15.63:56486) 2024-05-27T07:47:00.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.388+0000 7fe093400700 1 -- 172.21.15.63:0/321720415 learned_addr learned my addr 172.21.15.63:0/321720415 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:00.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.388+0000 7fe092a00700 1 --2- 172.21.15.63:0/321720415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce868cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.388+0000 7fe092a00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe0780b2a70 0x7fe0780b4f80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe08000b6c0 tx=0x7fe08000b830 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:00.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.388+0000 7fe093400700 1 -- 172.21.15.63:0/321720415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 msgr2=0x558ce86986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:00.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.388+0000 7fe093400700 1 --2- 172.21.15.63:0/321720415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce86986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.388+0000 7fe093400700 1 -- 172.21.15.63:0/321720415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 msgr2=0x558ce868cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:00.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.388+0000 7fe093400700 1 --2- 172.21.15.63:0/321720415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce868cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.389+0000 7fe093400700 1 -- 172.21.15.63:0/321720415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce8572420 con 0x558ce85f89f0 2024-05-27T07:47:00.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.389+0000 7fe093e00700 1 --2- 172.21.15.63:0/321720415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce86986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:00.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.389+0000 7fe092a00700 1 --2- 172.21.15.63:0/321720415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce868cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:00.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.389+0000 7fe093400700 1 --2- 172.21.15.63:0/321720415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce868c750 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fe08400b580 tx=0x7fe084009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87416faf95136d83 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:00.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.389+0000 7fe088c00700 1 -- 172.21.15.63:0/321720415 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084027070 con 0x558ce85f89f0 2024-05-27T07:47:00.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.389+0000 7fe088c00700 1 -- 172.21.15.63:0/321720415 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe08400be20 con 0x558ce85f89f0 2024-05-27T07:47:00.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.389+0000 7fe088c00700 1 -- 172.21.15.63:0/321720415 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08400fe50 con 0x558ce85f89f0 2024-05-27T07:47:00.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.390+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/321720415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 msgr2=0x558ce868c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:00.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.390+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/321720415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce868c750 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fe08400b580 tx=0x7fe084009510 comp rx=0 tx=0).stop 2024-05-27T07:47:00.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.391+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/321720415 shutdown_connections 2024-05-27T07:47:00.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.391+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/321720415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce86986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.391+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/321720415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce868c750 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.391+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/321720415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce868cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.391+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/321720415 >> 172.21.15.63:0/321720415 conn(0x558ce85a6390 msgr2=0x558ce8687710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:00.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.391+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/321720415 shutdown_connections 2024-05-27T07:47:00.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.391+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/321720415 wait complete. 2024-05-27T07:47:00.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.392+0000 7fe0a8d56b80 1 -- start start 2024-05-27T07:47:00.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.392+0000 7fe0a8d56b80 1 Processor -- start 2024-05-27T07:47:00.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce8693290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce86937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce8693d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558ce84affa0 con 0x558ce85f89f0 2024-05-27T07:47:00.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558ce854a760 con 0x558ce850ce40 2024-05-27T07:47:00.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ce8698da0 con 0x558ce86962c0 2024-05-27T07:47:00.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe0a8d56b80 10 client.0 _ll_register_callbacks cb 0x558ce8553c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:00.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe092a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce86937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe092a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce86937d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56490/0 (socket says 172.21.15.63:56490) 2024-05-27T07:47:00.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe092a00700 1 -- 172.21.15.63:0/2168798739 learned_addr learned my addr 172.21.15.63:0/2168798739 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:00.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe093400700 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce8693290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.393+0000 7fe093e00700 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce8693d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.394+0000 7fe092a00700 1 -- 172.21.15.63:0/2168798739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 msgr2=0x558ce8693d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:00.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.394+0000 7fe092a00700 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce8693d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.394+0000 7fe092a00700 1 -- 172.21.15.63:0/2168798739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 msgr2=0x558ce8693290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:00.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.394+0000 7fe092a00700 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce8693290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.394+0000 7fe092a00700 1 -- 172.21.15.63:0/2168798739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce84f98d0 con 0x558ce85f89f0 2024-05-27T07:47:00.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.394+0000 7fe093400700 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce8693290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:00.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.394+0000 7fe092a00700 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce86937d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fe080005950 tx=0x7fe0800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab062a5397b079f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:00.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.394+0000 7fe07d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:00.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.394+0000 7fe07d600700 1 -- 172.21.15.63:0/2168798739 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08001cce0 con 0x558ce85f89f0 2024-05-27T07:47:00.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.395+0000 7fe07d600700 1 -- 172.21.15.63:0/2168798739 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7fe0800253e0 con 0x558ce85f89f0 2024-05-27T07:47:00.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.395+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/2168798739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe08400be20 con 0x558ce85f89f0 2024-05-27T07:47:00.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.395+0000 7fe07d600700 1 -- 172.21.15.63:0/2168798739 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe08001ce80 con 0x558ce85f89f0 2024-05-27T07:47:00.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.395+0000 7fe0a8d56b80 10 client.18024 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:00.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.395+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/2168798739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558ce857a060 con 0x558ce85f89f0 2024-05-27T07:47:00.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.395+0000 7fe07d600700 1 -- 172.21.15.63:0/2168798739 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe08400be20 con 0x558ce85f89f0 2024-05-27T07:47:00.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.396+0000 7fe07d600700 1 -- 172.21.15.63:0/2168798739 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe080024070 con 0x558ce85f89f0 2024-05-27T07:47:00.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.396+0000 7fe0a8d56b80 10 client.18024 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:00.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.396+0000 7fe0a8d56b80 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:00.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.396+0000 7fe0a8d56b80 20 client.18024 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:00.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.396+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558ce86a5d70 0x558ce86a81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.396+0000 7fe0a8d56b80 4 client.18024 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:00.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.396+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/2168798739 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ce85bbd90 con 0x558ce86a5d70 2024-05-27T07:47:00.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.397+0000 7fe093400700 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558ce86a5d70 0x558ce86a81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.398+0000 7fe093400700 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558ce86a5d70 0x558ce86a81d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cac1cb31c70e7ec server_cookie=580caa976ef63735 in_seq=0 out_seq=0 2024-05-27T07:47:00.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.398+0000 7fe07d600700 10 client.18024 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:00.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe07d600700 1 -- 172.21.15.63:0/2168798739 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558ce85bbd90 con 0x558ce86a5d70 2024-05-27T07:47:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe07d600700 10 client.18024 handle_command_reply: tid=0 2024-05-27T07:47:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 client.18024 shutdown 2024-05-27T07:47:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/2168798739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 msgr2=0x558ce86937d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce86937d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fe080005950 tx=0x7fe0800047c0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/2168798739 shutdown_connections 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558ce86a5d70 0x558ce86a81d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce8693d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce8693290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/2168798739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce85f89f0 0x558ce86937d0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/2168798739 >> 172.21.15.63:0/2168798739 conn(0x558ce85a6390 msgr2=0x558ce86873a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe07d600700 0 client.18024 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/2168798739 shutdown_connections 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.400+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/2168798739 wait complete. 2024-05-27T07:47:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.401+0000 7fe0a8d56b80 20 client.18024 trim_cache size 0 max 16384 2024-05-27T07:47:00.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.424+0000 7fe0a8d56b80 1 Processor -- start 2024-05-27T07:47:00.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.424+0000 7fe0a8d56b80 1 -- start start 2024-05-27T07:47:00.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.425+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce868b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.425+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce85f89f0 0x558ce86a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.425+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce86962c0 0x558ce86ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.425+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941b4e50 con 0x558ce86962c0 2024-05-27T07:47:00.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.425+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941b26e0 con 0x558ce850ce40 2024-05-27T07:47:00.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.425+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941b0550 con 0x558ce85f89f0 2024-05-27T07:47:00.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.425+0000 7fe093e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce86962c0 0x558ce86ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.425+0000 7fe093e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce86962c0 0x558ce86ad9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56506/0 (socket says 172.21.15.63:56506) 2024-05-27T07:47:00.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.425+0000 7fe093e00700 1 -- 172.21.15.63:0/960067211 learned_addr learned my addr 172.21.15.63:0/960067211 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:00.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.425+0000 7fe092a00700 1 --2- 172.21.15.63:0/960067211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce85f89f0 0x558ce86a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.426+0000 7fe093400700 1 --2- 172.21.15.63:0/960067211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce868b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.426+0000 7fe093e00700 1 -- 172.21.15.63:0/960067211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce85f89f0 msgr2=0x558ce86a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:00.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.426+0000 7fe093e00700 1 --2- 172.21.15.63:0/960067211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce85f89f0 0x558ce86a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.426+0000 7fe093e00700 1 -- 172.21.15.63:0/960067211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 msgr2=0x558ce868b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:00.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.426+0000 7fe093e00700 1 --2- 172.21.15.63:0/960067211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce868b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.426+0000 7fe093e00700 1 -- 172.21.15.63:0/960067211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce868bd70 con 0x558ce86962c0 2024-05-27T07:47:00.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.426+0000 7fe093e00700 1 --2- 172.21.15.63:0/960067211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce86962c0 0x558ce86ad9d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0a8b60 tx=0x7fe08c0a89c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a79d4895f2f6f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:00.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.427+0000 7fe07fe00700 1 -- 172.21.15.63:0/960067211 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0a8e60 con 0x558ce86962c0 2024-05-27T07:47:00.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.427+0000 7fe07fe00700 1 -- 172.21.15.63:0/960067211 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe08c0ad210 con 0x558ce86962c0 2024-05-27T07:47:00.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.427+0000 7fe07fe00700 1 -- 172.21.15.63:0/960067211 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0b1830 con 0x558ce86962c0 2024-05-27T07:47:00.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.428+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/960067211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce86962c0 msgr2=0x558ce86ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:00.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.428+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/960067211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce86962c0 0x558ce86ad9d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0a8b60 tx=0x7fe08c0a89c0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.428+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/960067211 shutdown_connections 2024-05-27T07:47:00.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.428+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/960067211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce86962c0 0x558ce86ad9d0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.428+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/960067211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce850ce40 0x558ce868b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.428+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/960067211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce85f89f0 0x558ce86a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.428+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/960067211 >> 172.21.15.63:0/960067211 conn(0x558ce85a6390 msgr2=0x558ce868b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.428+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/960067211 shutdown_connections 2024-05-27T07:47:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.428+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/960067211 wait complete. 2024-05-27T07:47:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.429+0000 7fe0a8d56b80 1 -- start start 2024-05-27T07:47:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.429+0000 7fe0a8d56b80 1 Processor -- start 2024-05-27T07:47:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.429+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce850ce40 0x558ce869e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce85f89f0 0x558ce8699930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe0a8d56b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce8699e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941b0ba0 con 0x558ce850ce40 2024-05-27T07:47:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941b2010 con 0x558ce85f89f0 2024-05-27T07:47:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe0a8d56b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941b1270 con 0x558ce86962c0 2024-05-27T07:47:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe0a8d56b80 10 client.0 _ll_register_callbacks cb 0x558ce86a86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe093400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce850ce40 0x558ce869e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe093400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce850ce40 0x558ce869e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56516/0 (socket says 172.21.15.63:56516) 2024-05-27T07:47:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe093400700 1 -- 172.21.15.63:0/3785820441 learned_addr learned my addr 172.21.15.63:0/3785820441 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe092a00700 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce85f89f0 0x558ce8699930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe093400700 1 -- 172.21.15.63:0/3785820441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 msgr2=0x558ce8699e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe093400700 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce8699e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe093400700 1 -- 172.21.15.63:0/3785820441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce85f89f0 msgr2=0x558ce8699930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.430+0000 7fe093400700 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce85f89f0 0x558ce8699930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.431+0000 7fe093400700 1 -- 172.21.15.63:0/3785820441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce869a480 con 0x558ce850ce40 2024-05-27T07:47:00.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.431+0000 7fe093400700 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce850ce40 0x558ce869e5d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fe084000c00 tx=0x7fe08400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2521895264f75116 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:00.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.431+0000 7fe07d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:00.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.431+0000 7fe07d600700 1 -- 172.21.15.63:0/3785820441 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084034070 con 0x558ce850ce40 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.431+0000 7fe07d600700 1 -- 172.21.15.63:0/3785820441 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7fe08400bdd0 con 0x558ce850ce40 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.431+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/3785820441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe08c0ad210 con 0x558ce850ce40 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.432+0000 7fe07d600700 1 -- 172.21.15.63:0/3785820441 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe084044050 con 0x558ce850ce40 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.432+0000 7fe0a8d56b80 10 client.18030 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.432+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/3785820441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558ce8710650 con 0x558ce850ce40 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.432+0000 7fe07d600700 1 -- 172.21.15.63:0/3785820441 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe08c0ad210 con 0x558ce850ce40 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.432+0000 7fe07d600700 1 -- 172.21.15.63:0/3785820441 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe08400f580 con 0x558ce850ce40 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.432+0000 7fe0a8d56b80 10 client.18030 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.433+0000 7fe0a8d56b80 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.433+0000 7fe0a8d56b80 20 client.18030 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.433+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558ce8710820 0x558ce8712c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.433+0000 7fe0a8d56b80 4 client.18030 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:47:00.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.433+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/3785820441 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ce85b5570 con 0x558ce8710820 2024-05-27T07:47:00.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.433+0000 7fe092a00700 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558ce8710820 0x558ce8712c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:00.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.435+0000 7fe07d600700 10 client.18030 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:00.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.435+0000 7fe092a00700 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558ce8710820 0x558ce8712c40 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9ffe15cf460a95 server_cookie=2b9e17120181913e in_seq=0 out_seq=0 2024-05-27T07:47:00.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.435+0000 7fe07d600700 1 -- 172.21.15.63:0/3785820441 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558ce85b5570 con 0x558ce8710820 2024-05-27T07:47:00.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.435+0000 7fe07d600700 10 client.18030 handle_command_reply: tid=0 2024-05-27T07:47:00.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.435+0000 7fe0a8d56b80 1 client.18030 shutdown 2024-05-27T07:47:00.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.435+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/3785820441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce850ce40 msgr2=0x558ce869e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:00.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.435+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce850ce40 0x558ce869e5d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fe084000c00 tx=0x7fe08400b5b0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.435+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/3785820441 shutdown_connections 2024-05-27T07:47:00.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.435+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558ce8710820 0x558ce8712c40 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.436+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ce86962c0 0x558ce8699e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.436+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558ce850ce40 0x558ce869e5d0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.436+0000 7fe0a8d56b80 1 --2- 172.21.15.63:0/3785820441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ce85f89f0 0x558ce8699930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.436+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/3785820441 >> 172.21.15.63:0/3785820441 conn(0x558ce85a6390 msgr2=0x558ce868e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:00.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.436+0000 7fe07d600700 0 client.18030 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:00.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.436+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/3785820441 shutdown_connections 2024-05-27T07:47:00.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.436+0000 7fe0a8d56b80 1 -- 172.21.15.63:0/3785820441 wait complete. 2024-05-27T07:47:00.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.436+0000 7fe0a8d56b80 20 client.18030 trim_cache size 0 max 16384 2024-05-27T07:47:00.436 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:47:00.436 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:47:00.436 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:47:00.436 INFO:teuthology.orchestra.run.smithi063.stdout: "8b84186b-c9e7-401f-867d-ea59cef1a7ad": { 2024-05-27T07:47:00.437 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:47:00.437 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "8b84186b-c9e7-401f-867d-ea59cef1a7ad", 2024-05-27T07:47:00.437 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:47:00.437 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:47:00.437 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:47:00.437 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:47:00.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 -- 172.21.15.63:0/3770649714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe0780b2a70 msgr2=0x7fe0780b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe0780b2a70 0x7fe0780b4f80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe08000b6c0 tx=0x7fe08000b830 comp rx=0 tx=0).stop 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 -- 172.21.15.63:0/3770649714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe094109670 msgr2=0x7fe09419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe094109670 0x7fe09419a9b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fe08c007ee0 tx=0x7fe08c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 -- 172.21.15.63:0/3770649714 shutdown_connections 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe0780b2a70 0x7fe0780b4f80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe094109670 0x7fe09419a9b0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0941043f0 0x7fe0941a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 --2- 172.21.15.63:0/3770649714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe094108d20 0x7fe0941a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 -- 172.21.15.63:0/3770649714 >> 172.21.15.63:0/3770649714 conn(0x7fe094079d00 msgr2=0x7fe094101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 -- 172.21.15.63:0/3770649714 shutdown_connections 2024-05-27T07:47:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:00.437+0000 7fe098c00700 1 -- 172.21.15.63:0/3770649714 wait complete. 2024-05-27T07:47:01.315 INFO:tasks.cephfs.filesystem:scrub status for tag:8b84186b-c9e7-401f-867d-ea59cef1a7ad - {'path': '/', 'tag': '8b84186b-c9e7-401f-867d-ea59cef1a7ad', 'options': 'recursive,force'} 2024-05-27T07:47:01.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:01 smithi086 ceph-mon[34687]: pgmap v420: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.7 MiB/s wr, 17 op/s 2024-05-27T07:47:01.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:01 smithi149 ceph-mon[35408]: pgmap v420: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.7 MiB/s wr, 17 op/s 2024-05-27T07:47:01.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:01 smithi063 ceph-mon[31613]: pgmap v420: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.7 MiB/s wr, 17 op/s 2024-05-27T07:47:03.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:03 smithi086 ceph-mon[34687]: pgmap v421: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.4 MiB/s wr, 58 op/s 2024-05-27T07:47:03.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:03 smithi149 ceph-mon[35408]: pgmap v421: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.4 MiB/s wr, 58 op/s 2024-05-27T07:47:03.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:03 smithi063 ceph-mon[31613]: pgmap v421: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.4 MiB/s wr, 58 op/s 2024-05-27T07:47:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.676+0000 7f32f9c00640 1 -- 172.21.15.63:0/147378680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f4106420 msgr2=0x7f32f41127f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.676+0000 7f32f9c00640 1 --2- 172.21.15.63:0/147378680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f4106420 0x7f32f41127f0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009ca0 tx=0x7f32ec02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.676+0000 7f32f9c00640 1 -- 172.21.15.63:0/147378680 shutdown_connections 2024-05-27T07:47:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.676+0000 7f32f9c00640 1 --2- 172.21.15.63:0/147378680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f4106420 0x7f32f41127f0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.676+0000 7f32f9c00640 1 --2- 172.21.15.63:0/147378680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f410c170 0x7f32f4105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.676+0000 7f32f9c00640 1 --2- 172.21.15.63:0/147378680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f410bcd0 0x7f32f4105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.676+0000 7f32f9c00640 1 -- 172.21.15.63:0/147378680 >> 172.21.15.63:0/147378680 conn(0x7f32f41013d0 msgr2=0x7f32f4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.677+0000 7f32f9c00640 1 -- 172.21.15.63:0/147378680 shutdown_connections 2024-05-27T07:47:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.677+0000 7f32f9c00640 1 -- 172.21.15.63:0/147378680 wait complete. 2024-05-27T07:47:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.678+0000 7f32f9c00640 1 Processor -- start 2024-05-27T07:47:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.678+0000 7f32f9c00640 1 -- start start 2024-05-27T07:47:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f9c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f4106420 0x7f32f41a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f9c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f410bcd0 0x7f32f41a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f410c170 0x7f32f419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f9c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4115de0 con 0x7f32f4106420 2024-05-27T07:47:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4115c60 con 0x7f32f410c170 2024-05-27T07:47:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f9c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4115f60 con 0x7f32f410bcd0 2024-05-27T07:47:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f4106420 0x7f32f41a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f4106420 0x7f32f41a24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56544/0 (socket says 172.21.15.63:56544) 2024-05-27T07:47:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f3e00640 1 -- 172.21.15.63:0/494937811 learned_addr learned my addr 172.21.15.63:0/494937811 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f9200640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f410c170 0x7f32f419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.679+0000 7f32f3400640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f410bcd0 0x7f32f41a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.680+0000 7f32f3e00640 1 -- 172.21.15.63:0/494937811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f410bcd0 msgr2=0x7f32f41a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.680+0000 7f32f3e00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f410bcd0 0x7f32f41a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.680+0000 7f32f3e00640 1 -- 172.21.15.63:0/494937811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f410c170 msgr2=0x7f32f419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.680+0000 7f32f3e00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f410c170 0x7f32f419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.680+0000 7f32f3e00640 1 -- 172.21.15.63:0/494937811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f419cd90 con 0x7f32f4106420 2024-05-27T07:47:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.680+0000 7f32f3e00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f4106420 0x7f32f41a24e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005fd0 tx=0x7f32e400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.681+0000 7f32f0c00640 1 -- 172.21.15.63:0/494937811 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4017070 con 0x7f32f4106420 2024-05-27T07:47:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.681+0000 7f32f0c00640 1 -- 172.21.15.63:0/494937811 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32e4009c30 con 0x7f32f4106420 2024-05-27T07:47:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.681+0000 7f32f0c00640 1 -- 172.21.15.63:0/494937811 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4003c70 con 0x7f32f4106420 2024-05-27T07:47:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.681+0000 7f32f9c00640 1 -- 172.21.15.63:0/494937811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f419d020 con 0x7f32f4106420 2024-05-27T07:47:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.681+0000 7f32f9c00640 1 -- 172.21.15.63:0/494937811 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f41a9280 con 0x7f32f4106420 2024-05-27T07:47:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.685+0000 7f330219a740 1 Processor -- start 2024-05-27T07:47:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.685+0000 7f330219a740 1 -- start start 2024-05-27T07:47:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.686+0000 7f330219a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce4455a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.686+0000 7f330219a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce408a2f0 0x560ce4455fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.686+0000 7f330219a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce40924d0 0x560ce44564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.686+0000 7f330219a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560ce43c6b90 con 0x560ce40b31b0 2024-05-27T07:47:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.686+0000 7f330219a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ce43b6730 con 0x560ce408a2f0 2024-05-27T07:47:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.686+0000 7f330219a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560ce43ba770 con 0x560ce40924d0 2024-05-27T07:47:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.686+0000 7f32f9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce4455a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.686+0000 7f32f9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce4455a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56556/0 (socket says 172.21.15.63:56556) 2024-05-27T07:47:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.686+0000 7f32f9200640 1 -- 172.21.15.63:0/3571395789 learned_addr learned my addr 172.21.15.63:0/3571395789 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.686+0000 7f32f0c00640 1 -- 172.21.15.63:0/494937811 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f32e4003e10 con 0x7f32f4106420 2024-05-27T07:47:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.687+0000 7f32f9200640 1 -- 172.21.15.63:0/3571395789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce40924d0 msgr2=0x560ce44564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.687+0000 7f32f9200640 1 --2- 172.21.15.63:0/3571395789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce40924d0 0x560ce44564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.687+0000 7f32f9200640 1 -- 172.21.15.63:0/3571395789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce408a2f0 msgr2=0x560ce4455fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.687+0000 7f32f9200640 1 --2- 172.21.15.63:0/3571395789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce408a2f0 0x560ce4455fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.687+0000 7f32f9200640 1 -- 172.21.15.63:0/3571395789 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce434f8b0 con 0x560ce40b31b0 2024-05-27T07:47:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.687+0000 7f32f9200640 1 --2- 172.21.15.63:0/3571395789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce4455a60 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f32ec02aeb0 tx=0x7f32ec031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf71b99a08dc87ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.688+0000 7f32e9600640 1 -- 172.21.15.63:0/3571395789 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec031980 con 0x560ce40b31b0 2024-05-27T07:47:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.688+0000 7f32e9600640 1 -- 172.21.15.63:0/3571395789 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32ec036030 con 0x560ce40b31b0 2024-05-27T07:47:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.688+0000 7f32e9600640 1 -- 172.21.15.63:0/3571395789 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec004320 con 0x560ce40b31b0 2024-05-27T07:47:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.688+0000 7f32f0c00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32d80b2950 0x7f32d80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.689+0000 7f32f0c00640 1 -- 172.21.15.63:0/494937811 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f32e40e26d0 con 0x7f32f4106420 2024-05-27T07:47:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.689+0000 7f32f3e00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32d80b2950 0x7f32d80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.689+0000 7f330219a740 1 -- 172.21.15.63:0/3571395789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 msgr2=0x560ce4455a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.689+0000 7f330219a740 1 --2- 172.21.15.63:0/3571395789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce4455a60 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f32ec02aeb0 tx=0x7f32ec031500 comp rx=0 tx=0).stop 2024-05-27T07:47:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.689+0000 7f330219a740 1 -- 172.21.15.63:0/3571395789 shutdown_connections 2024-05-27T07:47:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.689+0000 7f330219a740 1 --2- 172.21.15.63:0/3571395789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce40924d0 0x560ce44564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.689+0000 7f330219a740 1 --2- 172.21.15.63:0/3571395789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce408a2f0 0x560ce4455fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.689+0000 7f330219a740 1 --2- 172.21.15.63:0/3571395789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce4455a60 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.689+0000 7f330219a740 1 -- 172.21.15.63:0/3571395789 >> 172.21.15.63:0/3571395789 conn(0x560ce40a7300 msgr2=0x560ce4452210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.690+0000 7f330219a740 1 -- 172.21.15.63:0/3571395789 shutdown_connections 2024-05-27T07:47:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.690+0000 7f32f3e00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32d80b2950 0x7f32d80b4e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x560ce4453550 tx=0x7f32e400bf60 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.690+0000 7f330219a740 1 -- 172.21.15.63:0/3571395789 wait complete. 2024-05-27T07:47:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.691+0000 7f330219a740 1 -- start start 2024-05-27T07:47:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.691+0000 7f330219a740 1 Processor -- start 2024-05-27T07:47:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.691+0000 7f330219a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce408a2f0 0x560ce445fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.691+0000 7f330219a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4460110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.691+0000 7f330219a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40b31b0 0x560ce446bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.691+0000 7f330219a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560ce409c790 con 0x560ce40924d0 2024-05-27T07:47:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.691+0000 7f330219a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ce435ebc0 con 0x560ce40b31b0 2024-05-27T07:47:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.691+0000 7f330219a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560ce44629b0 con 0x560ce408a2f0 2024-05-27T07:47:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f330219a740 10 client.0 _ll_register_callbacks cb 0x560ce433c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4460110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4460110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56566/0 (socket says 172.21.15.63:56566) 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f3400640 1 -- 172.21.15.63:0/2266364926 learned_addr learned my addr 172.21.15.63:0/2266364926 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f9200640 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40b31b0 0x560ce446bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f9200640 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce408a2f0 0x560ce445fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f3400640 1 -- 172.21.15.63:0/2266364926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce408a2f0 msgr2=0x560ce445fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f3400640 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce408a2f0 0x560ce445fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f3400640 1 -- 172.21.15.63:0/2266364926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40b31b0 msgr2=0x560ce446bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f3400640 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40b31b0 0x560ce446bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f3400640 1 -- 172.21.15.63:0/2266364926 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce40cb210 con 0x560ce40924d0 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.692+0000 7f32f9200640 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40b31b0 0x560ce446bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.693+0000 7f32f3400640 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4460110 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f32e000bc90 tx=0x7f32e000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3e3c0a98b80e50 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.693+0000 7f32de000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.693+0000 7f32de000640 1 -- 172.21.15.63:0/2266364926 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0015070 con 0x560ce40924d0 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.693+0000 7f32de000640 1 -- 172.21.15.63:0/2266364926 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f32e001a660 con 0x560ce40924d0 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.693+0000 7f330219a740 1 -- 172.21.15.63:0/2266364926 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ec036030 con 0x560ce40924d0 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.693+0000 7f32de000640 1 -- 172.21.15.63:0/2266364926 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f32e0023050 con 0x560ce40924d0 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.694+0000 7f330219a740 10 client.18042 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.694+0000 7f330219a740 1 -- 172.21.15.63:0/2266364926 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560ce4197620 con 0x560ce40924d0 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.694+0000 7f32de000640 1 -- 172.21.15.63:0/2266364926 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32ec036030 con 0x560ce40924d0 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.694+0000 7f32de000640 1 -- 172.21.15.63:0/2266364926 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f32e00118d0 con 0x560ce40924d0 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.694+0000 7f330219a740 10 client.18042 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.694+0000 7f330219a740 10 client.18042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.694+0000 7f330219a740 20 client.18042 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.694+0000 7f330219a740 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560ce4071d80 0x560ce4471cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.694+0000 7f330219a740 4 client.18042 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.694+0000 7f330219a740 1 -- 172.21.15.63:0/2266364926 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ce43c8ae0 con 0x560ce4071d80 2024-05-27T07:47:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.695+0000 7f32f9200640 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560ce4071d80 0x560ce4471cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.696+0000 7f32f9200640 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560ce4071d80 0x560ce4471cc0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e11257a8722c0790 server_cookie=b00c7b4f46b81541 in_seq=0 out_seq=0 2024-05-27T07:47:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.696+0000 7f32de000640 10 client.18042 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.697+0000 7f32de000640 1 -- 172.21.15.63:0/2266364926 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560ce43c8ae0 con 0x560ce4071d80 2024-05-27T07:47:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.697+0000 7f32de000640 10 client.18042 handle_command_reply: tid=0 2024-05-27T07:47:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.697+0000 7f330219a740 1 client.18042 shutdown 2024-05-27T07:47:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 1 -- 172.21.15.63:0/2266364926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 msgr2=0x560ce4460110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4460110 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f32e000bc90 tx=0x7f32e000f730 comp rx=0 tx=0).stop 2024-05-27T07:47:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 1 -- 172.21.15.63:0/2266364926 shutdown_connections 2024-05-27T07:47:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560ce4071d80 0x560ce4471cc0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40b31b0 0x560ce446bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4460110 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 1 --2- 172.21.15.63:0/2266364926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce408a2f0 0x560ce445fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f32de000640 0 client.18042 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 1 -- 172.21.15.63:0/2266364926 >> 172.21.15.63:0/2266364926 conn(0x560ce40a7300 msgr2=0x560ce4457780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 1 -- 172.21.15.63:0/2266364926 shutdown_connections 2024-05-27T07:47:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 1 -- 172.21.15.63:0/2266364926 wait complete. 2024-05-27T07:47:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.698+0000 7f330219a740 20 client.18042 trim_cache size 0 max 16384 2024-05-27T07:47:05.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:05 smithi086 ceph-mon[34687]: pgmap v422: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 54 op/s 2024-05-27T07:47:05.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:05 smithi149 ceph-mon[35408]: pgmap v422: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 54 op/s 2024-05-27T07:47:05.714 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:05 smithi063 ceph-mon[31613]: pgmap v422: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 54 op/s 2024-05-27T07:47:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.719+0000 7f330219a740 1 Processor -- start 2024-05-27T07:47:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.719+0000 7f330219a740 1 -- start start 2024-05-27T07:47:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.720+0000 7f330219a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce408a2f0 0x560ce4460120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.720+0000 7f330219a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4462300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.720+0000 7f330219a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce40b31b0 0x560ce446ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.720+0000 7f330219a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b5b30 con 0x560ce40924d0 2024-05-27T07:47:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.720+0000 7f330219a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b3400 con 0x560ce408a2f0 2024-05-27T07:47:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.720+0000 7f330219a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b1270 con 0x560ce40b31b0 2024-05-27T07:47:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.720+0000 7f32f3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4462300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.720+0000 7f32f3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4462300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56570/0 (socket says 172.21.15.63:56570) 2024-05-27T07:47:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.720+0000 7f32f3400640 1 -- 172.21.15.63:0/1787813179 learned_addr learned my addr 172.21.15.63:0/1787813179 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.720+0000 7f32f9200640 1 --2- 172.21.15.63:0/1787813179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce408a2f0 0x560ce4460120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.721+0000 7f32f9200640 1 --2- 172.21.15.63:0/1787813179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce40b31b0 0x560ce446ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.721+0000 7f32f3400640 1 -- 172.21.15.63:0/1787813179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce40b31b0 msgr2=0x560ce446ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.721+0000 7f32f3400640 1 --2- 172.21.15.63:0/1787813179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce40b31b0 0x560ce446ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.721+0000 7f32f3400640 1 -- 172.21.15.63:0/1787813179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce408a2f0 msgr2=0x560ce4460120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.721+0000 7f32f3400640 1 --2- 172.21.15.63:0/1787813179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce408a2f0 0x560ce4460120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.721+0000 7f32f3400640 1 -- 172.21.15.63:0/1787813179 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce445e120 con 0x560ce40924d0 2024-05-27T07:47:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.721+0000 7f32f3400640 1 --2- 172.21.15.63:0/1787813179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4462300 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f32e0011c40 tx=0x7f32e0012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287e87d974183e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.722+0000 7f32e8c00640 1 -- 172.21.15.63:0/1787813179 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0020040 con 0x560ce40924d0 2024-05-27T07:47:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.722+0000 7f32e8c00640 1 -- 172.21.15.63:0/1787813179 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32e0005040 con 0x560ce40924d0 2024-05-27T07:47:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.722+0000 7f32e8c00640 1 -- 172.21.15.63:0/1787813179 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0015040 con 0x560ce40924d0 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.723+0000 7f330219a740 1 -- 172.21.15.63:0/1787813179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 msgr2=0x560ce4462300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.723+0000 7f330219a740 1 --2- 172.21.15.63:0/1787813179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4462300 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f32e0011c40 tx=0x7f32e0012f40 comp rx=0 tx=0).stop 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.723+0000 7f330219a740 1 -- 172.21.15.63:0/1787813179 shutdown_connections 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.723+0000 7f330219a740 1 --2- 172.21.15.63:0/1787813179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce40b31b0 0x560ce446ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.723+0000 7f330219a740 1 --2- 172.21.15.63:0/1787813179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40924d0 0x560ce4462300 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.723+0000 7f330219a740 1 --2- 172.21.15.63:0/1787813179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce408a2f0 0x560ce4460120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.723+0000 7f330219a740 1 -- 172.21.15.63:0/1787813179 >> 172.21.15.63:0/1787813179 conn(0x560ce40a7300 msgr2=0x560ce4456380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.723+0000 7f330219a740 1 -- 172.21.15.63:0/1787813179 shutdown_connections 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.723+0000 7f330219a740 1 -- 172.21.15.63:0/1787813179 wait complete. 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.723+0000 7f330219a740 1 -- start start 2024-05-27T07:47:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.724+0000 7f330219a740 1 Processor -- start 2024-05-27T07:47:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.724+0000 7f330219a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce408a2f0 0x560ce4463260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.724+0000 7f330219a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40924d0 0x560ce44637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.724+0000 7f330219a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce44d8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.724+0000 7f330219a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b18c0 con 0x560ce40b31b0 2024-05-27T07:47:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.724+0000 7f330219a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b2d30 con 0x560ce40924d0 2024-05-27T07:47:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.724+0000 7f330219a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41b1f90 con 0x560ce408a2f0 2024-05-27T07:47:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.724+0000 7f330219a740 10 client.0 _ll_register_callbacks cb 0x7f32e0005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.724+0000 7f32f9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce44d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.725+0000 7f32f9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce44d8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56582/0 (socket says 172.21.15.63:56582) 2024-05-27T07:47:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.725+0000 7f32f9200640 1 -- 172.21.15.63:0/3407559176 learned_addr learned my addr 172.21.15.63:0/3407559176 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.725+0000 7f32f9200640 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce408a2f0 0x560ce4463260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.725+0000 7f32f3400640 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40924d0 0x560ce44637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.725+0000 7f32f9200640 1 -- 172.21.15.63:0/3407559176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce408a2f0 msgr2=0x560ce4463260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.725+0000 7f32f9200640 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce408a2f0 0x560ce4463260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.725+0000 7f32f9200640 1 -- 172.21.15.63:0/3407559176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40924d0 msgr2=0x560ce44637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.725+0000 7f32f9200640 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40924d0 0x560ce44637a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.725+0000 7f32f9200640 1 -- 172.21.15.63:0/3407559176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce44d8870 con 0x560ce40b31b0 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.725+0000 7f32f9200640 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce44d8330 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f32ec02ae80 tx=0x7f32ec004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8546a6ed2dd9dbc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.726+0000 7f32de000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.726+0000 7f32de000640 1 -- 172.21.15.63:0/3407559176 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec04a040 con 0x560ce40b31b0 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.726+0000 7f330219a740 1 -- 172.21.15.63:0/3407559176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e0005040 con 0x560ce40b31b0 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.726+0000 7f32de000640 1 -- 172.21.15.63:0/3407559176 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f32ec031b80 con 0x560ce40b31b0 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.726+0000 7f32de000640 1 -- 172.21.15.63:0/3407559176 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f32ec068050 con 0x560ce40b31b0 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.726+0000 7f330219a740 10 client.18048 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.726+0000 7f330219a740 1 -- 172.21.15.63:0/3407559176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560ce44d9ad0 con 0x560ce40b31b0 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.727+0000 7f32de000640 1 -- 172.21.15.63:0/3407559176 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32e0005040 con 0x560ce40b31b0 2024-05-27T07:47:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.727+0000 7f32de000640 1 -- 172.21.15.63:0/3407559176 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f32ec043070 con 0x560ce40b31b0 2024-05-27T07:47:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.727+0000 7f330219a740 10 client.18048 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.727+0000 7f330219a740 10 client.18048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.727+0000 7f330219a740 20 client.18048 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.727+0000 7f330219a740 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560ce40471c0 0x560ce44dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.727+0000 7f330219a740 4 client.18048 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:47:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.727+0000 7f330219a740 1 -- 172.21.15.63:0/3407559176 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ce43c22c0 con 0x560ce40471c0 2024-05-27T07:47:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.728+0000 7f32f3400640 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560ce40471c0 0x560ce44dbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.729+0000 7f32f3400640 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560ce40471c0 0x560ce44dbf60 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80870c8cc277d57e server_cookie=b945d3924975eb15 in_seq=0 out_seq=0 2024-05-27T07:47:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.729+0000 7f32de000640 10 client.18048 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.765+0000 7f32de000640 1 -- 172.21.15.63:0/3407559176 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560ce43c22c0 con 0x560ce40471c0 2024-05-27T07:47:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f32de000640 10 client.18048 handle_command_reply: tid=0 2024-05-27T07:47:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 client.18048 shutdown 2024-05-27T07:47:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 -- 172.21.15.63:0/3407559176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 msgr2=0x560ce44d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce44d8330 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f32ec02ae80 tx=0x7f32ec004a80 comp rx=0 tx=0).stop 2024-05-27T07:47:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 -- 172.21.15.63:0/3407559176 shutdown_connections 2024-05-27T07:47:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560ce40471c0 0x560ce44dbf60 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ce40b31b0 0x560ce44d8330 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ce40924d0 0x560ce44637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 --2- 172.21.15.63:0/3407559176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ce408a2f0 0x560ce4463260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f32de000640 0 client.18048 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 -- 172.21.15.63:0/3407559176 >> 172.21.15.63:0/3407559176 conn(0x560ce40a7300 msgr2=0x560ce4450810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 -- 172.21.15.63:0/3407559176 shutdown_connections 2024-05-27T07:47:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.766+0000 7f330219a740 1 -- 172.21.15.63:0/3407559176 wait complete. 2024-05-27T07:47:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.767+0000 7f330219a740 20 client.18048 trim_cache size 0 max 16384 2024-05-27T07:47:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:47:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:47:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:47:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.767+0000 7f32f9c00640 1 -- 172.21.15.63:0/494937811 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32d80b2950 msgr2=0x7f32d80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.767+0000 7f32f9c00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32d80b2950 0x7f32d80b4e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x560ce4453550 tx=0x7f32e400bf60 comp rx=0 tx=0).stop 2024-05-27T07:47:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.767+0000 7f32f9c00640 1 -- 172.21.15.63:0/494937811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f4106420 msgr2=0x7f32f41a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.767+0000 7f32f9c00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f4106420 0x7f32f41a24e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005fd0 tx=0x7f32e400dde0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.767+0000 7f32f9c00640 1 -- 172.21.15.63:0/494937811 shutdown_connections 2024-05-27T07:47:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.767+0000 7f32f9c00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32d80b2950 0x7f32d80b4e60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.767+0000 7f32f9c00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f410c170 0x7f32f419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.768+0000 7f32f9c00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f410bcd0 0x7f32f41a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.768+0000 7f32f9c00640 1 --2- 172.21.15.63:0/494937811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32f4106420 0x7f32f41a24e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.768+0000 7f32f9c00640 1 -- 172.21.15.63:0/494937811 >> 172.21.15.63:0/494937811 conn(0x7f32f41013d0 msgr2=0x7f32f4103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.768+0000 7f32f9c00640 1 -- 172.21.15.63:0/494937811 shutdown_connections 2024-05-27T07:47:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:05.768+0000 7f32f9c00640 1 -- 172.21.15.63:0/494937811 wait complete. 2024-05-27T07:47:07.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:07 smithi086 ceph-mon[34687]: pgmap v423: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 54 op/s 2024-05-27T07:47:07.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:47:07.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:47:07.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:47:07.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:07 smithi086 ceph-mon[34687]: osdmap e108: 12 total, 12 up, 12 in 2024-05-27T07:47:07.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:07 smithi149 ceph-mon[35408]: pgmap v423: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 54 op/s 2024-05-27T07:47:07.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:47:07.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:47:07.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:47:07.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:07 smithi149 ceph-mon[35408]: osdmap e108: 12 total, 12 up, 12 in 2024-05-27T07:47:07.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:07 smithi063 ceph-mon[31613]: pgmap v423: 129 pgs: 129 active+clean; 3.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 54 op/s 2024-05-27T07:47:07.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:47:07.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:47:07.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:47:07.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:07 smithi063 ceph-mon[31613]: osdmap e108: 12 total, 12 up, 12 in 2024-05-27T07:47:09.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:09 smithi086 ceph-mon[34687]: pgmap v425: 129 pgs: 14 active+clean+snaptrim_wait, 115 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 57 op/s 2024-05-27T07:47:09.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:09 smithi149 ceph-mon[35408]: pgmap v425: 129 pgs: 14 active+clean+snaptrim_wait, 115 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 57 op/s 2024-05-27T07:47:09.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:09 smithi063 ceph-mon[31613]: pgmap v425: 129 pgs: 14 active+clean+snaptrim_wait, 115 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 57 op/s 2024-05-27T07:47:11.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:11 smithi086 ceph-mon[34687]: pgmap v426: 129 pgs: 14 active+clean+snaptrim_wait, 115 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 57 op/s 2024-05-27T07:47:11.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:11 smithi149 ceph-mon[35408]: pgmap v426: 129 pgs: 14 active+clean+snaptrim_wait, 115 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 57 op/s 2024-05-27T07:47:11.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:11 smithi063 ceph-mon[31613]: pgmap v426: 129 pgs: 14 active+clean+snaptrim_wait, 115 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 57 op/s 2024-05-27T07:47:13.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:13 smithi086 ceph-mon[34687]: pgmap v427: 129 pgs: 129 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.5 MiB/s wr, 13 op/s 2024-05-27T07:47:13.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:13 smithi149 ceph-mon[35408]: pgmap v427: 129 pgs: 129 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.5 MiB/s wr, 13 op/s 2024-05-27T07:47:13.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:13 smithi063 ceph-mon[31613]: pgmap v427: 129 pgs: 129 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.5 MiB/s wr, 13 op/s 2024-05-27T07:47:15.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:15 smithi086 ceph-mon[34687]: pgmap v428: 129 pgs: 129 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.5 MiB/s wr, 13 op/s 2024-05-27T07:47:15.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:15 smithi149 ceph-mon[35408]: pgmap v428: 129 pgs: 129 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.5 MiB/s wr, 13 op/s 2024-05-27T07:47:15.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:15 smithi063 ceph-mon[31613]: pgmap v428: 129 pgs: 129 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.5 MiB/s wr, 13 op/s 2024-05-27T07:47:16.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:16 smithi086 ceph-mon[34687]: osdmap e109: 12 total, 12 up, 12 in 2024-05-27T07:47:16.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:16 smithi086 ceph-mon[34687]: pgmap v430: 129 pgs: 129 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.8 MiB/s wr, 14 op/s 2024-05-27T07:47:16.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:16 smithi149 ceph-mon[35408]: osdmap e109: 12 total, 12 up, 12 in 2024-05-27T07:47:16.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:16 smithi149 ceph-mon[35408]: pgmap v430: 129 pgs: 129 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.8 MiB/s wr, 14 op/s 2024-05-27T07:47:16.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:16 smithi063 ceph-mon[31613]: osdmap e109: 12 total, 12 up, 12 in 2024-05-27T07:47:16.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:16 smithi063 ceph-mon[31613]: pgmap v430: 129 pgs: 129 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.8 MiB/s wr, 14 op/s 2024-05-27T07:47:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.021+0000 7f48d3200640 1 -- 172.21.15.63:0/846594417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d4109f50 msgr2=0x7f48d4103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.021+0000 7f48d3200640 1 --2- 172.21.15.63:0/846594417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d4109f50 0x7f48d4103be0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009900 tx=0x7f48c002a410 comp rx=0 tx=0).stop 2024-05-27T07:47:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.021+0000 7f48d3200640 1 -- 172.21.15.63:0/846594417 shutdown_connections 2024-05-27T07:47:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.021+0000 7f48d3200640 1 --2- 172.21.15.63:0/846594417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d4104250 0x7f48d41105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.021+0000 7f48d3200640 1 --2- 172.21.15.63:0/846594417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d4109f50 0x7f48d4103be0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.021+0000 7f48d3200640 1 --2- 172.21.15.63:0/846594417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d4109ab0 0x7f48d41036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.021+0000 7f48d3200640 1 -- 172.21.15.63:0/846594417 >> 172.21.15.63:0/846594417 conn(0x7f48d4079240 msgr2=0x7f48d4079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.021+0000 7f48d3200640 1 -- 172.21.15.63:0/846594417 shutdown_connections 2024-05-27T07:47:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.021+0000 7f48d3200640 1 -- 172.21.15.63:0/846594417 wait complete. 2024-05-27T07:47:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.023+0000 7f48d3200640 1 Processor -- start 2024-05-27T07:47:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.023+0000 7f48d3200640 1 -- start start 2024-05-27T07:47:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d4104250 0x7f48d4074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d4109ab0 0x7f48d4075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d4109f50 0x7f48d4075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4113b50 con 0x7f48d4104250 2024-05-27T07:47:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41139d0 con 0x7f48d4109f50 2024-05-27T07:47:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4113cd0 con 0x7f48d4109ab0 2024-05-27T07:47:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d4104250 0x7f48d4074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d4104250 0x7f48d4074fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55990/0 (socket says 172.21.15.63:55990) 2024-05-27T07:47:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d1e00640 1 -- 172.21.15.63:0/2481104421 learned_addr learned my addr 172.21.15.63:0/2481104421 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d1400640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d4109ab0 0x7f48d4075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d2800640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d4109f50 0x7f48d4075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d1e00640 1 -- 172.21.15.63:0/2481104421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d4109ab0 msgr2=0x7f48d4075500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.024+0000 7f48d1e00640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d4109ab0 0x7f48d4075500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.025+0000 7f48d1e00640 1 -- 172.21.15.63:0/2481104421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d4109f50 msgr2=0x7f48d4075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.025+0000 7f48d1e00640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d4109f50 0x7f48d4075a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.025+0000 7f48d1e00640 1 -- 172.21.15.63:0/2481104421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d41a89b0 con 0x7f48d4104250 2024-05-27T07:47:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.025+0000 7f48d1400640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d4109ab0 0x7f48d4075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.025+0000 7f48d2800640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d4109f50 0x7f48d4075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.025+0000 7f48d1e00640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d4104250 0x7f48d4074fc0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f48c400e9c0 tx=0x7f48c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.025+0000 7f48caa00640 1 -- 172.21.15.63:0/2481104421 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c400cd80 con 0x7f48d4104250 2024-05-27T07:47:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.025+0000 7f48caa00640 1 -- 172.21.15.63:0/2481104421 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c40054b0 con 0x7f48d4104250 2024-05-27T07:47:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.025+0000 7f48d3200640 1 -- 172.21.15.63:0/2481104421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d41a8ca0 con 0x7f48d4104250 2024-05-27T07:47:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.025+0000 7f48caa00640 1 -- 172.21.15.63:0/2481104421 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4010430 con 0x7f48d4104250 2024-05-27T07:47:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.026+0000 7f48d3200640 1 -- 172.21.15.63:0/2481104421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d41a9190 con 0x7f48d4104250 2024-05-27T07:47:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.029+0000 7f48e017f740 1 Processor -- start 2024-05-27T07:47:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.029+0000 7f48e017f740 1 -- start start 2024-05-27T07:47:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48e017f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd42a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48e017f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd42fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48e017f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48e017f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563fbccb3b90 con 0x563fbc9772f0 2024-05-27T07:47:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48e017f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563fbcca3730 con 0x563fbc9a01b0 2024-05-27T07:47:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48e017f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563fbcca7770 con 0x563fbc97f4d0 2024-05-27T07:47:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48caa00640 1 -- 172.21.15.63:0/2481104421 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f48c4004f20 con 0x7f48d4104250 2024-05-27T07:47:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48d1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd42fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48d1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd42fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56002/0 (socket says 172.21.15.63:56002) 2024-05-27T07:47:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48d1e00640 1 -- 172.21.15.63:0/1986433234 learned_addr learned my addr 172.21.15.63:0/1986433234 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48d1400640 1 --2- 172.21.15.63:0/1986433234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.030+0000 7f48d2800640 1 --2- 172.21.15.63:0/1986433234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd42a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.031+0000 7f48d1e00640 1 -- 172.21.15.63:0/1986433234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 msgr2=0x563fbcd434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.031+0000 7f48d1e00640 1 --2- 172.21.15.63:0/1986433234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.031+0000 7f48d1e00640 1 -- 172.21.15.63:0/1986433234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 msgr2=0x563fbcd42a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.031+0000 7f48d1e00640 1 --2- 172.21.15.63:0/1986433234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd42a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.031+0000 7f48d1e00640 1 -- 172.21.15.63:0/1986433234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fbcc3c8b0 con 0x563fbc9772f0 2024-05-27T07:47:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.031+0000 7f48d1e00640 1 --2- 172.21.15.63:0/1986433234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd42fa0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f48c40e02d0 tx=0x7f48c40eb570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc591ad32a7f1c17 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.032+0000 7f48bf400640 1 -- 172.21.15.63:0/1986433234 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40f6070 con 0x563fbc9772f0 2024-05-27T07:47:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.032+0000 7f48bf400640 1 -- 172.21.15.63:0/1986433234 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c40e52d0 con 0x563fbc9772f0 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.032+0000 7f48caa00640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48b80b29d0 0x7f48b80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.032+0000 7f48bf400640 1 -- 172.21.15.63:0/1986433234 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40f1660 con 0x563fbc9772f0 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.032+0000 7f48caa00640 1 -- 172.21.15.63:0/2481104421 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f48c40e2fd0 con 0x7f48d4104250 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.032+0000 7f48d2800640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48b80b29d0 0x7f48b80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.033+0000 7f48e017f740 1 -- 172.21.15.63:0/1986433234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 msgr2=0x563fbcd42fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.033+0000 7f48e017f740 1 --2- 172.21.15.63:0/1986433234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd42fa0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f48c40e02d0 tx=0x7f48c40eb570 comp rx=0 tx=0).stop 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.033+0000 7f48d2800640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48b80b29d0 0x7f48b80b4ee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x563fbcd3fbd0 tx=0x7f48cc009700 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.033+0000 7f48e017f740 1 -- 172.21.15.63:0/1986433234 shutdown_connections 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.033+0000 7f48e017f740 1 --2- 172.21.15.63:0/1986433234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.033+0000 7f48e017f740 1 --2- 172.21.15.63:0/1986433234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd42fa0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.033+0000 7f48e017f740 1 --2- 172.21.15.63:0/1986433234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd42a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.033+0000 7f48e017f740 1 -- 172.21.15.63:0/1986433234 >> 172.21.15.63:0/1986433234 conn(0x563fbc994300 msgr2=0x563fbcd3f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.033+0000 7f48e017f740 1 -- 172.21.15.63:0/1986433234 shutdown_connections 2024-05-27T07:47:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.033+0000 7f48e017f740 1 -- 172.21.15.63:0/1986433234 wait complete. 2024-05-27T07:47:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.034+0000 7f48e017f740 1 -- start start 2024-05-27T07:47:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.034+0000 7f48e017f740 1 Processor -- start 2024-05-27T07:47:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.035+0000 7f48e017f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.035+0000 7f48e017f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd4d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.035+0000 7f48e017f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd58ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.035+0000 7f48e017f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563fbc989790 con 0x563fbc9772f0 2024-05-27T07:47:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.035+0000 7f48e017f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563fbcc4bbc0 con 0x563fbc9a01b0 2024-05-27T07:47:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.035+0000 7f48e017f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563fbcd4fb90 con 0x563fbc97f4d0 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.035+0000 7f48e017f740 10 client.0 _ll_register_callbacks cb 0x563fbcc29400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.035+0000 7f48d1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48d1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56016/0 (socket says 172.21.15.63:56016) 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48d1e00640 1 -- 172.21.15.63:0/466041155 learned_addr learned my addr 172.21.15.63:0/466041155 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48d1400640 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd4d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48d2800640 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd58ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48d1e00640 1 -- 172.21.15.63:0/466041155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 msgr2=0x563fbcd4d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48d1e00640 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd4d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48d1e00640 1 -- 172.21.15.63:0/466041155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 msgr2=0x563fbcd58ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48d1e00640 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd58ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48d1e00640 1 -- 172.21.15.63:0/466041155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fbc9b8210 con 0x563fbc9772f0 2024-05-27T07:47:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48d1e00640 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4cc80 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f48c40a9bf0 tx=0x7f48c4009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f447580c9eb1a5d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.036+0000 7f48b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.037+0000 7f48b3e00640 1 -- 172.21.15.63:0/466041155 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40a5db0 con 0x563fbc9772f0 2024-05-27T07:47:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.037+0000 7f48b3e00640 1 -- 172.21.15.63:0/466041155 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f48c40e50d0 con 0x563fbc9772f0 2024-05-27T07:47:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.037+0000 7f48e017f740 1 -- 172.21.15.63:0/466041155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48c40e52d0 con 0x563fbc9772f0 2024-05-27T07:47:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.037+0000 7f48b3e00640 1 -- 172.21.15.63:0/466041155 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f48c40a9660 con 0x563fbc9772f0 2024-05-27T07:47:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.037+0000 7f48e017f740 10 client.18060 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.037+0000 7f48e017f740 1 -- 172.21.15.63:0/466041155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563fbca84620 con 0x563fbc9772f0 2024-05-27T07:47:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.037+0000 7f48b3e00640 1 -- 172.21.15.63:0/466041155 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c40e52d0 con 0x563fbc9772f0 2024-05-27T07:47:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.037+0000 7f48b3e00640 1 -- 172.21.15.63:0/466041155 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f48c40a6180 con 0x563fbc9772f0 2024-05-27T07:47:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.038+0000 7f48e017f740 10 client.18060 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.038+0000 7f48e017f740 10 client.18060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.038+0000 7f48e017f740 20 client.18060 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.038+0000 7f48e017f740 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563fbc95ed80 0x563fbcd5ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.038+0000 7f48e017f740 4 client.18060 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.038+0000 7f48e017f740 1 -- 172.21.15.63:0/466041155 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fbccb5ae0 con 0x563fbc95ed80 2024-05-27T07:47:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.038+0000 7f48d1400640 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563fbc95ed80 0x563fbcd5ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.039+0000 7f48d1400640 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563fbc95ed80 0x563fbcd5ed10 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1cb02eb8767e4b server_cookie=b2ed334399cf20bc in_seq=0 out_seq=0 2024-05-27T07:47:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.039+0000 7f48b3e00640 10 client.18060 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.041+0000 7f48b3e00640 1 -- 172.21.15.63:0/466041155 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563fbccb5ae0 con 0x563fbc95ed80 2024-05-27T07:47:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.041+0000 7f48b3e00640 10 client.18060 handle_command_reply: tid=0 2024-05-27T07:47:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.041+0000 7f48e017f740 1 client.18060 shutdown 2024-05-27T07:47:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 1 -- 172.21.15.63:0/466041155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 msgr2=0x563fbcd4cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4cc80 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f48c40a9bf0 tx=0x7f48c4009770 comp rx=0 tx=0).stop 2024-05-27T07:47:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 1 -- 172.21.15.63:0/466041155 shutdown_connections 2024-05-27T07:47:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563fbc95ed80 0x563fbcd5ed10 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd58ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd4d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 1 --2- 172.21.15.63:0/466041155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4cc80 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 1 -- 172.21.15.63:0/466041155 >> 172.21.15.63:0/466041155 conn(0x563fbc994300 msgr2=0x563fbcd42a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48b3e00640 0 client.18060 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 1 -- 172.21.15.63:0/466041155 shutdown_connections 2024-05-27T07:47:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 1 -- 172.21.15.63:0/466041155 wait complete. 2024-05-27T07:47:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.042+0000 7f48e017f740 20 client.18060 trim_cache size 0 max 16384 2024-05-27T07:47:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.059+0000 7f48e017f740 1 Processor -- start 2024-05-27T07:47:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.059+0000 7f48e017f740 1 -- start start 2024-05-27T07:47:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.059+0000 7f48e017f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48e017f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd58e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48e017f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd4cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48e017f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41b5c20 con 0x563fbc9772f0 2024-05-27T07:47:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48e017f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41b34f0 con 0x563fbc9a01b0 2024-05-27T07:47:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48e017f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41b1360 con 0x563fbc97f4d0 2024-05-27T07:47:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48d1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48d1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56032/0 (socket says 172.21.15.63:56032) 2024-05-27T07:47:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48d1e00640 1 -- 172.21.15.63:0/1173145594 learned_addr learned my addr 172.21.15.63:0/1173145594 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48d1400640 1 --2- 172.21.15.63:0/1173145594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd58e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48d2800640 1 --2- 172.21.15.63:0/1173145594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd4cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48d1e00640 1 -- 172.21.15.63:0/1173145594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 msgr2=0x563fbcd58e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48d1e00640 1 --2- 172.21.15.63:0/1173145594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd58e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48d1e00640 1 -- 172.21.15.63:0/1173145594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 msgr2=0x563fbcd4cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48d1e00640 1 --2- 172.21.15.63:0/1173145594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd4cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.060+0000 7f48d1e00640 1 -- 172.21.15.63:0/1173145594 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fbcd427f0 con 0x563fbc9772f0 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.061+0000 7f48d1e00640 1 --2- 172.21.15.63:0/1173145594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4f3b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f48c400cfd0 tx=0x7f48c40eb570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a4d22ce33534ab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.061+0000 7f48bea00640 1 -- 172.21.15.63:0/1173145594 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40a9840 con 0x563fbc9772f0 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.061+0000 7f48bea00640 1 -- 172.21.15.63:0/1173145594 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c40e68f0 con 0x563fbc9772f0 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.061+0000 7f48bea00640 1 -- 172.21.15.63:0/1173145594 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40a9cc0 con 0x563fbc9772f0 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.062+0000 7f48e017f740 1 -- 172.21.15.63:0/1173145594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 msgr2=0x563fbcd4f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.062+0000 7f48e017f740 1 --2- 172.21.15.63:0/1173145594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4f3b0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f48c400cfd0 tx=0x7f48c40eb570 comp rx=0 tx=0).stop 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.062+0000 7f48e017f740 1 -- 172.21.15.63:0/1173145594 shutdown_connections 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.062+0000 7f48e017f740 1 --2- 172.21.15.63:0/1173145594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9a01b0 0x563fbcd4cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.062+0000 7f48e017f740 1 --2- 172.21.15.63:0/1173145594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc97f4d0 0x563fbcd58e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.062+0000 7f48e017f740 1 --2- 172.21.15.63:0/1173145594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc9772f0 0x563fbcd4f3b0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.062+0000 7f48e017f740 1 -- 172.21.15.63:0/1173145594 >> 172.21.15.63:0/1173145594 conn(0x563fbc994300 msgr2=0x563fbcd4c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.062+0000 7f48e017f740 1 -- 172.21.15.63:0/1173145594 shutdown_connections 2024-05-27T07:47:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.062+0000 7f48e017f740 1 -- 172.21.15.63:0/1173145594 wait complete. 2024-05-27T07:47:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.063+0000 7f48e017f740 1 -- start start 2024-05-27T07:47:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.063+0000 7f48e017f740 1 Processor -- start 2024-05-27T07:47:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48e017f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9772f0 0x563fbcd62450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48e017f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc97f4d0 0x563fbcd607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48e017f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc9a01b0 0x563fbcd60d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48e017f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41b19b0 con 0x563fbc97f4d0 2024-05-27T07:47:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48e017f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41b2e20 con 0x563fbc9772f0 2024-05-27T07:47:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48e017f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41b2080 con 0x563fbc9a01b0 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48e017f740 10 client.0 _ll_register_callbacks cb 0x7f48c40f4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48d1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc97f4d0 0x563fbcd607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48d1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc97f4d0 0x563fbcd607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56044/0 (socket says 172.21.15.63:56044) 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48d1400640 1 -- 172.21.15.63:0/3823483661 learned_addr learned my addr 172.21.15.63:0/3823483661 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48d2800640 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc9a01b0 0x563fbcd60d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.064+0000 7f48d1e00640 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9772f0 0x563fbcd62450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.065+0000 7f48d1400640 1 -- 172.21.15.63:0/3823483661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc9a01b0 msgr2=0x563fbcd60d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.065+0000 7f48d1400640 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc9a01b0 0x563fbcd60d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.065+0000 7f48d1400640 1 -- 172.21.15.63:0/3823483661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9772f0 msgr2=0x563fbcd62450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.065+0000 7f48d1400640 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9772f0 0x563fbcd62450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.065+0000 7f48d1400640 1 -- 172.21.15.63:0/3823483661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fbcd614a0 con 0x563fbc97f4d0 2024-05-27T07:47:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.065+0000 7f48d1400640 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc97f4d0 0x563fbcd607d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f48c002a930 tx=0x7f48c0032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f198e9c415d6a89c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.065+0000 7f48b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.065+0000 7f48b3e00640 1 -- 172.21.15.63:0/3823483661 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0040030 con 0x563fbc97f4d0 2024-05-27T07:47:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.065+0000 7f48b3e00640 1 -- 172.21.15.63:0/3823483661 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f48c0004310 con 0x563fbc97f4d0 2024-05-27T07:47:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.065+0000 7f48e017f740 1 -- 172.21.15.63:0/3823483661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48c40e68f0 con 0x563fbc97f4d0 2024-05-27T07:47:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48b3e00640 1 -- 172.21.15.63:0/3823483661 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f48c0050050 con 0x563fbc97f4d0 2024-05-27T07:47:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48e017f740 10 client.18066 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48e017f740 1 -- 172.21.15.63:0/3823483661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563fbcdc60c0 con 0x563fbc97f4d0 2024-05-27T07:47:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48b3e00640 1 -- 172.21.15.63:0/3823483661 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48c40e68f0 con 0x563fbc97f4d0 2024-05-27T07:47:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48b3e00640 1 -- 172.21.15.63:0/3823483661 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f48c004b030 con 0x563fbc97f4d0 2024-05-27T07:47:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48e017f740 10 client.18066 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48e017f740 10 client.18066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48e017f740 20 client.18066 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48e017f740 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563fbc9341c0 0x563fbcdc8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48e017f740 4 client.18066 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:47:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.066+0000 7f48e017f740 1 -- 172.21.15.63:0/3823483661 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fbcd47260 con 0x563fbc9341c0 2024-05-27T07:47:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.067+0000 7f48d1e00640 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563fbc9341c0 0x563fbcdc8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.068+0000 7f48b3e00640 10 client.18066 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.068+0000 7f48d1e00640 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563fbc9341c0 0x563fbcdc8880 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92dd6381bdecabc5 server_cookie=88d3a8cb5c910392 in_seq=0 out_seq=0 2024-05-27T07:47:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.100+0000 7f48b3e00640 1 -- 172.21.15.63:0/3823483661 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563fbcd47260 con 0x563fbc9341c0 2024-05-27T07:47:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.100+0000 7f48b3e00640 10 client.18066 handle_command_reply: tid=0 2024-05-27T07:47:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.100+0000 7f48e017f740 1 client.18066 shutdown 2024-05-27T07:47:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 1 -- 172.21.15.63:0/3823483661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc97f4d0 msgr2=0x563fbcd607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc97f4d0 0x563fbcd607d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f48c002a930 tx=0x7f48c0032f60 comp rx=0 tx=0).stop 2024-05-27T07:47:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 1 -- 172.21.15.63:0/3823483661 shutdown_connections 2024-05-27T07:47:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563fbc9341c0 0x563fbcdc8880 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563fbc9a01b0 0x563fbcd60d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563fbc97f4d0 0x563fbcd607d0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48b3e00640 0 client.18066 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 1 --2- 172.21.15.63:0/3823483661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fbc9772f0 0x563fbcd62450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 1 -- 172.21.15.63:0/3823483661 >> 172.21.15.63:0/3823483661 conn(0x563fbc994300 msgr2=0x563fbcd4cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 1 -- 172.21.15.63:0/3823483661 shutdown_connections 2024-05-27T07:47:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 1 -- 172.21.15.63:0/3823483661 wait complete. 2024-05-27T07:47:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.101+0000 7f48e017f740 20 client.18066 trim_cache size 0 max 16384 2024-05-27T07:47:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:47:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:47:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:47:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.102+0000 7f48d3200640 1 -- 172.21.15.63:0/2481104421 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48b80b29d0 msgr2=0x7f48b80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.102+0000 7f48d3200640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48b80b29d0 0x7f48b80b4ee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x563fbcd3fbd0 tx=0x7f48cc009700 comp rx=0 tx=0).stop 2024-05-27T07:47:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.102+0000 7f48d3200640 1 -- 172.21.15.63:0/2481104421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d4104250 msgr2=0x7f48d4074fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.102+0000 7f48d3200640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d4104250 0x7f48d4074fc0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f48c400e9c0 tx=0x7f48c400ee80 comp rx=0 tx=0).stop 2024-05-27T07:47:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.102+0000 7f48d3200640 1 -- 172.21.15.63:0/2481104421 shutdown_connections 2024-05-27T07:47:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.102+0000 7f48d3200640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f48b80b29d0 0x7f48b80b4ee0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.102+0000 7f48d3200640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48d4109f50 0x7f48d4075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.102+0000 7f48d3200640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d4109ab0 0x7f48d4075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.102+0000 7f48d3200640 1 --2- 172.21.15.63:0/2481104421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48d4104250 0x7f48d4074fc0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.102+0000 7f48d3200640 1 -- 172.21.15.63:0/2481104421 >> 172.21.15.63:0/2481104421 conn(0x7f48d4079240 msgr2=0x7f48d4102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.103+0000 7f48d3200640 1 -- 172.21.15.63:0/2481104421 shutdown_connections 2024-05-27T07:47:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:19.103+0000 7f48d3200640 1 -- 172.21.15.63:0/2481104421 wait complete. 2024-05-27T07:47:19.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:18 smithi086 ceph-mon[34687]: pgmap v431: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:47:19.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:18 smithi149 ceph-mon[35408]: pgmap v431: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:47:19.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:18 smithi063 ceph-mon[31613]: pgmap v431: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:47:21.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:20 smithi086 ceph-mon[34687]: pgmap v432: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:47:21.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:20 smithi149 ceph-mon[35408]: pgmap v432: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:47:21.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:20 smithi063 ceph-mon[31613]: pgmap v432: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:47:23.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:22 smithi086 ceph-mon[34687]: pgmap v433: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:47:23.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:22 smithi149 ceph-mon[35408]: pgmap v433: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:47:23.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:22 smithi063 ceph-mon[31613]: pgmap v433: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:47:24.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:23 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:47:24.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:23 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:47:24.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:23 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:47:25.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:24 smithi086 ceph-mon[34687]: pgmap v434: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:47:25.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:24 smithi149 ceph-mon[35408]: pgmap v434: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:47:25.214 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:24 smithi063 ceph-mon[31613]: pgmap v434: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:47:26.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:25 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:47:26.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:25 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:47:26.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:25 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:47:26.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:25 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:47:26.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:25 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:47:26.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:25 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:47:26.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:25 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:47:26.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:25 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:47:26.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:25 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:47:27.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:27 smithi086 ceph-mon[34687]: pgmap v435: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.3 MiB/s wr, 10 op/s 2024-05-27T07:47:27.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:27 smithi149 ceph-mon[35408]: pgmap v435: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.3 MiB/s wr, 10 op/s 2024-05-27T07:47:27.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:27 smithi063 ceph-mon[31613]: pgmap v435: 129 pgs: 129 active+clean; 3.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.3 MiB/s wr, 10 op/s 2024-05-27T07:47:29.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:29 smithi086 ceph-mon[34687]: pgmap v436: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:47:29.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:29 smithi149 ceph-mon[35408]: pgmap v436: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:47:29.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:29 smithi063 ceph-mon[31613]: pgmap v436: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:47:31.316 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:47:31.395 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:31 smithi063 ceph-mon[31613]: pgmap v437: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:47:31.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:31 smithi086 ceph-mon[34687]: pgmap v437: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:47:31.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:31 smithi149 ceph-mon[35408]: pgmap v437: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:47:31.818 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.344+0000 7f3284a00640 1 -- 172.21.15.63:0/697469683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328010e1e0 msgr2=0x7f32801106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.344+0000 7f3284a00640 1 --2- 172.21.15.63:0/697469683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328010e1e0 0x7f32801106d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f3278009ca0 tx=0x7f327802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.344+0000 7f3284a00640 1 -- 172.21.15.63:0/697469683 shutdown_connections 2024-05-27T07:47:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.344+0000 7f3284a00640 1 --2- 172.21.15.63:0/697469683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328010e1e0 0x7f32801106d0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.344+0000 7f3284a00640 1 --2- 172.21.15.63:0/697469683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f328010b7f0 0x7f328010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.344+0000 7f3284a00640 1 --2- 172.21.15.63:0/697469683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3280104d90 0x7f32801051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.344+0000 7f3284a00640 1 -- 172.21.15.63:0/697469683 >> 172.21.15.63:0/697469683 conn(0x7f32800782d0 msgr2=0x7f3280078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.344+0000 7f3284a00640 1 -- 172.21.15.63:0/697469683 shutdown_connections 2024-05-27T07:47:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.344+0000 7f3284a00640 1 -- 172.21.15.63:0/697469683 wait complete. 2024-05-27T07:47:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.345+0000 7f3284a00640 1 Processor -- start 2024-05-27T07:47:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.346+0000 7f3284a00640 1 -- start start 2024-05-27T07:47:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.346+0000 7f3284a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3280104d90 0x7f3280105850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.346+0000 7f3284a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328010b7f0 0x7f3280105d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.346+0000 7f3284a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328010e1e0 0x7f32801062d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.346+0000 7f3284a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3280113d40 con 0x7f328010e1e0 2024-05-27T07:47:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.346+0000 7f3284a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3280113bc0 con 0x7f328010b7f0 2024-05-27T07:47:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.346+0000 7f3284a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3280113ec0 con 0x7f3280104d90 2024-05-27T07:47:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328010e1e0 0x7f32801062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328010e1e0 0x7f32801062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51348/0 (socket says 172.21.15.63:51348) 2024-05-27T07:47:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327e800640 1 -- 172.21.15.63:0/2074486777 learned_addr learned my addr 172.21.15.63:0/2074486777 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327de00640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3280104d90 0x7f3280105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327d400640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328010b7f0 0x7f3280105d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327e800640 1 -- 172.21.15.63:0/2074486777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3280104d90 msgr2=0x7f3280105850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327e800640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3280104d90 0x7f3280105850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327e800640 1 -- 172.21.15.63:0/2074486777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328010b7f0 msgr2=0x7f3280105d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327e800640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328010b7f0 0x7f3280105d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327e800640 1 -- 172.21.15.63:0/2074486777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32801acf10 con 0x7f328010e1e0 2024-05-27T07:47:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.347+0000 7f327de00640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3280104d90 0x7f3280105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.348+0000 7f327e800640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328010e1e0 0x7f32801062d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3278005e30 tx=0x7f3278003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.348+0000 7f3276a00640 1 -- 172.21.15.63:0/2074486777 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278032c10 con 0x7f328010e1e0 2024-05-27T07:47:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.349+0000 7f3284a00640 1 -- 172.21.15.63:0/2074486777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32801ad140 con 0x7f328010e1e0 2024-05-27T07:47:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.349+0000 7f3276a00640 1 -- 172.21.15.63:0/2074486777 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3278032db0 con 0x7f328010e1e0 2024-05-27T07:47:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.349+0000 7f3284a00640 1 -- 172.21.15.63:0/2074486777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32801ad6e0 con 0x7f328010e1e0 2024-05-27T07:47:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.349+0000 7f3276a00640 1 -- 172.21.15.63:0/2074486777 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278030ab0 con 0x7f328010e1e0 2024-05-27T07:47:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.353+0000 7f328b9c2740 1 Processor -- start 2024-05-27T07:47:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.355+0000 7f3276a00640 1 -- 172.21.15.63:0/2074486777 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3278030c50 con 0x7f328010e1e0 2024-05-27T07:47:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.355+0000 7f328b9c2740 1 -- start start 2024-05-27T07:47:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.356+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16961b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.356+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692502f0 0x55c16961bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.356+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692584d0 0x55c16961c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.356+0000 7f328b9c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c16958cb90 con 0x55c1692791b0 2024-05-27T07:47:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.356+0000 7f328b9c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c16957c730 con 0x55c1692502f0 2024-05-27T07:47:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.356+0000 7f328b9c2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c169580770 con 0x55c1692584d0 2024-05-27T07:47:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.357+0000 7f327d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16961b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.357+0000 7f327de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692502f0 0x55c16961bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.357+0000 7f327d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16961b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51356/0 (socket says 172.21.15.63:51356) 2024-05-27T07:47:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.357+0000 7f327d400640 1 -- 172.21.15.63:0/66985972 learned_addr learned my addr 172.21.15.63:0/66985972 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.357+0000 7f327d400640 1 --2- 172.21.15.63:0/66985972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692584d0 0x55c16961c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.357+0000 7f327de00640 1 -- 172.21.15.63:0/66985972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692584d0 msgr2=0x55c16961c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.358+0000 7f327de00640 1 --2- 172.21.15.63:0/66985972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692584d0 0x55c16961c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.358+0000 7f327de00640 1 -- 172.21.15.63:0/66985972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 msgr2=0x55c16961b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.358+0000 7f327de00640 1 --2- 172.21.15.63:0/66985972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16961b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.358+0000 7f327de00640 1 -- 172.21.15.63:0/66985972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1695158b0 con 0x55c1692502f0 2024-05-27T07:47:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.358+0000 7f327d400640 1 --2- 172.21.15.63:0/66985972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16961b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.358+0000 7f3276a00640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32640b29d0 0x7f32640b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.358+0000 7f327de00640 1 --2- 172.21.15.63:0/66985972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692502f0 0x55c16961bda0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3270009fd0 tx=0x7f32700118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91b33c30d5a0a5c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.358+0000 7f3276a00640 1 -- 172.21.15.63:0/2074486777 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f3278103bc0 con 0x7f328010e1e0 2024-05-27T07:47:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.358+0000 7f326b000640 1 -- 172.21.15.63:0/66985972 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270004030 con 0x55c1692502f0 2024-05-27T07:47:32.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.359+0000 7f326b000640 1 -- 172.21.15.63:0/66985972 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3270005020 con 0x55c1692502f0 2024-05-27T07:47:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.359+0000 7f326b000640 1 -- 172.21.15.63:0/66985972 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270004c10 con 0x55c1692502f0 2024-05-27T07:47:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.359+0000 7f327d400640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32640b29d0 0x7f32640b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.360+0000 7f328b9c2740 1 -- 172.21.15.63:0/66985972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692502f0 msgr2=0x55c16961bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.360+0000 7f328b9c2740 1 --2- 172.21.15.63:0/66985972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692502f0 0x55c16961bda0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3270009fd0 tx=0x7f32700118d0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.360+0000 7f328b9c2740 1 -- 172.21.15.63:0/66985972 shutdown_connections 2024-05-27T07:47:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.360+0000 7f328b9c2740 1 --2- 172.21.15.63:0/66985972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692584d0 0x55c16961c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.360+0000 7f328b9c2740 1 --2- 172.21.15.63:0/66985972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692502f0 0x55c16961bda0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.360+0000 7f328b9c2740 1 --2- 172.21.15.63:0/66985972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16961b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.360+0000 7f328b9c2740 1 -- 172.21.15.63:0/66985972 >> 172.21.15.63:0/66985972 conn(0x55c16926d300 msgr2=0x55c169618010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.360+0000 7f327d400640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32640b29d0 0x7f32640b4ee0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f326c009fd0 tx=0x7f326c005850 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.360+0000 7f328b9c2740 1 -- 172.21.15.63:0/66985972 shutdown_connections 2024-05-27T07:47:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.360+0000 7f328b9c2740 1 -- 172.21.15.63:0/66985972 wait complete. 2024-05-27T07:47:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.361+0000 7f328b9c2740 1 -- start start 2024-05-27T07:47:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.361+0000 7f328b9c2740 1 Processor -- start 2024-05-27T07:47:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.362+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692502f0 0x55c1696258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.362+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 0x55c169625e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.362+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692791b0 0x55c169626350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.362+0000 7f328b9c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c169262790 con 0x55c1692502f0 2024-05-27T07:47:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.362+0000 7f328b9c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c169524bc0 con 0x55c1692584d0 2024-05-27T07:47:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.362+0000 7f328b9c2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c169628990 con 0x55c1692791b0 2024-05-27T07:47:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.362+0000 7f328b9c2740 10 client.0 _ll_register_callbacks cb 0x55c169502400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.363+0000 7f327de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692502f0 0x55c1696258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.363+0000 7f327de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692502f0 0x55c1696258d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51364/0 (socket says 172.21.15.63:51364) 2024-05-27T07:47:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.363+0000 7f327de00640 1 -- 172.21.15.63:0/2957370533 learned_addr learned my addr 172.21.15.63:0/2957370533 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.363+0000 7f327d400640 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 0x55c169625e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.363+0000 7f327e800640 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692791b0 0x55c169626350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.363+0000 7f327de00640 1 -- 172.21.15.63:0/2957370533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692791b0 msgr2=0x55c169626350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.364+0000 7f327de00640 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692791b0 0x55c169626350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.364+0000 7f327de00640 1 -- 172.21.15.63:0/2957370533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 msgr2=0x55c169625e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.364+0000 7f327de00640 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 0x55c169625e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.364+0000 7f327de00640 1 -- 172.21.15.63:0/2957370533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c169291210 con 0x55c1692502f0 2024-05-27T07:47:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.364+0000 7f327e800640 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692791b0 0x55c169626350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.364+0000 7f327de00640 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692502f0 0x55c1696258d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f32700057c0 tx=0x7f32700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedf4d9b7145d26d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.364+0000 7f325f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.365+0000 7f325f400640 1 -- 172.21.15.63:0/2957370533 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270028020 con 0x55c1692502f0 2024-05-27T07:47:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.365+0000 7f325f400640 1 -- 172.21.15.63:0/2957370533 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f3270016040 con 0x55c1692502f0 2024-05-27T07:47:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.365+0000 7f328b9c2740 1 -- 172.21.15.63:0/2957370533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3270005020 con 0x55c1692502f0 2024-05-27T07:47:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.365+0000 7f325f400640 1 -- 172.21.15.63:0/2957370533 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3270033050 con 0x55c1692502f0 2024-05-27T07:47:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.366+0000 7f328b9c2740 10 client.18078 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.366+0000 7f328b9c2740 1 -- 172.21.15.63:0/2957370533 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c16935d620 con 0x55c1692502f0 2024-05-27T07:47:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.366+0000 7f325f400640 1 -- 172.21.15.63:0/2957370533 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3270005020 con 0x55c1692502f0 2024-05-27T07:47:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.366+0000 7f325f400640 1 -- 172.21.15.63:0/2957370533 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3270003980 con 0x55c1692502f0 2024-05-27T07:47:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.366+0000 7f328b9c2740 10 client.18078 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.366+0000 7f328b9c2740 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.366+0000 7f328b9c2740 20 client.18078 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.366+0000 7f328b9c2740 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c169237d80 0x55c169637b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.366+0000 7f328b9c2740 4 client.18078 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.366+0000 7f328b9c2740 1 -- 172.21.15.63:0/2957370533 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c16958eae0 con 0x55c169237d80 2024-05-27T07:47:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.367+0000 7f327d400640 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c169237d80 0x55c169637b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.369+0000 7f325f400640 10 client.18078 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.369+0000 7f327d400640 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c169237d80 0x55c169637b10 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c3d82b8e3ad3591 server_cookie=b3d56b003174f54d in_seq=0 out_seq=0 2024-05-27T07:47:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f325f400640 1 -- 172.21.15.63:0/2957370533 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c16958eae0 con 0x55c169237d80 2024-05-27T07:47:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f325f400640 10 client.18078 handle_command_reply: tid=0 2024-05-27T07:47:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f328b9c2740 1 client.18078 shutdown 2024-05-27T07:47:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f328b9c2740 1 -- 172.21.15.63:0/2957370533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692502f0 msgr2=0x55c1696258d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f328b9c2740 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692502f0 0x55c1696258d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f32700057c0 tx=0x7f32700042f0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f328b9c2740 1 -- 172.21.15.63:0/2957370533 shutdown_connections 2024-05-27T07:47:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f328b9c2740 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c169237d80 0x55c169637b10 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f328b9c2740 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692791b0 0x55c169626350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f328b9c2740 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 0x55c169625e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f325f400640 0 client.18078 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f328b9c2740 1 --2- 172.21.15.63:0/2957370533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692502f0 0x55c1696258d0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f328b9c2740 1 -- 172.21.15.63:0/2957370533 >> 172.21.15.63:0/2957370533 conn(0x55c16926d300 msgr2=0x55c16961b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.371+0000 7f328b9c2740 1 -- 172.21.15.63:0/2957370533 shutdown_connections 2024-05-27T07:47:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.372+0000 7f328b9c2740 1 -- 172.21.15.63:0/2957370533 wait complete. 2024-05-27T07:47:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.372+0000 7f328b9c2740 20 client.18078 trim_cache size 0 max 16384 2024-05-27T07:47:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.391+0000 7f328b9c2740 1 Processor -- start 2024-05-27T07:47:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.391+0000 7f328b9c2740 1 -- start start 2024-05-27T07:47:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.391+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692502f0 0x55c169627a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.391+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 0x55c169631ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.391+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16963d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.391+0000 7f328b9c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32600127e0 con 0x55c1692791b0 2024-05-27T07:47:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.391+0000 7f328b9c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32600100b0 con 0x55c1692584d0 2024-05-27T07:47:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.391+0000 7f328b9c2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f326000df20 con 0x55c1692502f0 2024-05-27T07:47:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16963d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16963d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51380/0 (socket says 172.21.15.63:51380) 2024-05-27T07:47:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327e800640 1 -- 172.21.15.63:0/3992360305 learned_addr learned my addr 172.21.15.63:0/3992360305 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327d400640 1 --2- 172.21.15.63:0/3992360305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 0x55c169631ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327de00640 1 --2- 172.21.15.63:0/3992360305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692502f0 0x55c169627a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327e800640 1 -- 172.21.15.63:0/3992360305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692502f0 msgr2=0x55c169627a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327e800640 1 --2- 172.21.15.63:0/3992360305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692502f0 0x55c169627a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327e800640 1 -- 172.21.15.63:0/3992360305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 msgr2=0x55c169631ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327e800640 1 --2- 172.21.15.63:0/3992360305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 0x55c169631ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327e800640 1 -- 172.21.15.63:0/3992360305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c169637e80 con 0x55c1692791b0 2024-05-27T07:47:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.392+0000 7f327d400640 1 --2- 172.21.15.63:0/3992360305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 0x55c169631ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.393+0000 7f327e800640 1 --2- 172.21.15.63:0/3992360305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16963d260 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3278031920 tx=0x7f327802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8898194950ea01c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.393+0000 7f326a600640 1 -- 172.21.15.63:0/3992360305 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780e5070 con 0x55c1692791b0 2024-05-27T07:47:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.393+0000 7f326a600640 1 -- 172.21.15.63:0/3992360305 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3278036030 con 0x55c1692791b0 2024-05-27T07:47:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.393+0000 7f326a600640 1 -- 172.21.15.63:0/3992360305 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780cf030 con 0x55c1692791b0 2024-05-27T07:47:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.394+0000 7f328b9c2740 1 -- 172.21.15.63:0/3992360305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 msgr2=0x55c16963d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.394+0000 7f328b9c2740 1 --2- 172.21.15.63:0/3992360305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16963d260 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3278031920 tx=0x7f327802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.394+0000 7f328b9c2740 1 -- 172.21.15.63:0/3992360305 shutdown_connections 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.394+0000 7f328b9c2740 1 --2- 172.21.15.63:0/3992360305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692791b0 0x55c16963d260 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.394+0000 7f328b9c2740 1 --2- 172.21.15.63:0/3992360305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692584d0 0x55c169631ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.394+0000 7f328b9c2740 1 --2- 172.21.15.63:0/3992360305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692502f0 0x55c169627a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.394+0000 7f328b9c2740 1 -- 172.21.15.63:0/3992360305 >> 172.21.15.63:0/3992360305 conn(0x55c16926d300 msgr2=0x55c1696200e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.394+0000 7f328b9c2740 1 -- 172.21.15.63:0/3992360305 shutdown_connections 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.394+0000 7f328b9c2740 1 -- 172.21.15.63:0/3992360305 wait complete. 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.395+0000 7f328b9c2740 1 -- start start 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.395+0000 7f328b9c2740 1 Processor -- start 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.395+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692502f0 0x55c16962aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692584d0 0x55c16962afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f328b9c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692791b0 0x55c16962b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f328b9c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f326000e570 con 0x55c1692584d0 2024-05-27T07:47:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f328b9c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f326000f9e0 con 0x55c1692791b0 2024-05-27T07:47:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f328b9c2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f326000ec40 con 0x55c1692502f0 2024-05-27T07:47:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f328b9c2740 10 client.0 _ll_register_callbacks cb 0x7f326c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692584d0 0x55c16962afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692584d0 0x55c16962afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51392/0 (socket says 172.21.15.63:51392) 2024-05-27T07:47:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327d400640 1 -- 172.21.15.63:0/3244680844 learned_addr learned my addr 172.21.15.63:0/3244680844 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327de00640 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692502f0 0x55c16962aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327e800640 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692791b0 0x55c16962b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327d400640 1 -- 172.21.15.63:0/3244680844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692502f0 msgr2=0x55c16962aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327d400640 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692502f0 0x55c16962aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327d400640 1 -- 172.21.15.63:0/3244680844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692791b0 msgr2=0x55c16962b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327d400640 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692791b0 0x55c16962b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327d400640 1 -- 172.21.15.63:0/3244680844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c16969e130 con 0x55c1692584d0 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.396+0000 7f327e800640 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692791b0 0x55c16962b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.397+0000 7f327d400640 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692584d0 0x55c16962afa0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x55c1696189d0 tx=0x7f326c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ed33bca20d2267 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.397+0000 7f325f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.397+0000 7f325f400640 1 -- 172.21.15.63:0/3244680844 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c005020 con 0x55c1692584d0 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.397+0000 7f325f400640 1 -- 172.21.15.63:0/3244680844 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f326c0051c0 con 0x55c1692584d0 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.397+0000 7f328b9c2740 1 -- 172.21.15.63:0/3244680844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3278036030 con 0x55c1692584d0 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f325f400640 1 -- 172.21.15.63:0/3244680844 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f326c03c050 con 0x55c1692584d0 2024-05-27T07:47:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f328b9c2740 10 client.18084 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f328b9c2740 1 -- 172.21.15.63:0/3244680844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c16969f380 con 0x55c1692584d0 2024-05-27T07:47:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f325f400640 1 -- 172.21.15.63:0/3244680844 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3278036030 con 0x55c1692584d0 2024-05-27T07:47:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f325f400640 1 -- 172.21.15.63:0/3244680844 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f326c0035f0 con 0x55c1692584d0 2024-05-27T07:47:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f328b9c2740 10 client.18084 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f328b9c2740 10 client.18084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f328b9c2740 20 client.18084 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f328b9c2740 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c16920d1c0 0x55c1696a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f328b9c2740 4 client.18084 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:47:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.398+0000 7f328b9c2740 1 -- 172.21.15.63:0/3244680844 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1695882c0 con 0x55c16920d1c0 2024-05-27T07:47:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.399+0000 7f327de00640 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c16920d1c0 0x55c1696a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.400+0000 7f327de00640 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c16920d1c0 0x55c1696a1b70 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1ab3d833c24330 server_cookie=2f8c4a27888059fd in_seq=0 out_seq=0 2024-05-27T07:47:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.400+0000 7f325f400640 10 client.18084 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f325f400640 1 -- 172.21.15.63:0/3244680844 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c1695882c0 con 0x55c16920d1c0 2024-05-27T07:47:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f325f400640 10 client.18084 handle_command_reply: tid=0 2024-05-27T07:47:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f328b9c2740 1 client.18084 shutdown 2024-05-27T07:47:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f328b9c2740 1 -- 172.21.15.63:0/3244680844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692584d0 msgr2=0x55c16962afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f328b9c2740 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692584d0 0x55c16962afa0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x55c1696189d0 tx=0x7f326c004730 comp rx=0 tx=0).stop 2024-05-27T07:47:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f328b9c2740 1 -- 172.21.15.63:0/3244680844 shutdown_connections 2024-05-27T07:47:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f328b9c2740 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c16920d1c0 0x55c1696a1b70 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f328b9c2740 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1692791b0 0x55c16962b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f328b9c2740 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c1692584d0 0x55c16962afa0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f328b9c2740 1 --2- 172.21.15.63:0/3244680844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1692502f0 0x55c16962aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f325f400640 0 client.18084 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.436+0000 7f328b9c2740 1 -- 172.21.15.63:0/3244680844 >> 172.21.15.63:0/3244680844 conn(0x55c16926d300 msgr2=0x55c169627640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.437+0000 7f328b9c2740 1 -- 172.21.15.63:0/3244680844 shutdown_connections 2024-05-27T07:47:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.437+0000 7f328b9c2740 1 -- 172.21.15.63:0/3244680844 wait complete. 2024-05-27T07:47:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.437+0000 7f328b9c2740 20 client.18084 trim_cache size 0 max 16384 2024-05-27T07:47:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:47:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:47:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 -- 172.21.15.63:0/2074486777 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32640b29d0 msgr2=0x7f32640b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32640b29d0 0x7f32640b4ee0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f326c009fd0 tx=0x7f326c005850 comp rx=0 tx=0).stop 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 -- 172.21.15.63:0/2074486777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328010e1e0 msgr2=0x7f32801062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328010e1e0 0x7f32801062d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3278005e30 tx=0x7f3278003f30 comp rx=0 tx=0).stop 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 -- 172.21.15.63:0/2074486777 shutdown_connections 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32640b29d0 0x7f32640b4ee0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328010e1e0 0x7f32801062d0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328010b7f0 0x7f3280105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 --2- 172.21.15.63:0/2074486777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3280104d90 0x7f3280105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 -- 172.21.15.63:0/2074486777 >> 172.21.15.63:0/2074486777 conn(0x7f32800782d0 msgr2=0x7f328010f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 -- 172.21.15.63:0/2074486777 shutdown_connections 2024-05-27T07:47:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:32.438+0000 7f3284a00640 1 -- 172.21.15.63:0/2074486777 wait complete. 2024-05-27T07:47:33.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:33 smithi086 ceph-mon[34687]: pgmap v438: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:47:33.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:33 smithi149 ceph-mon[35408]: pgmap v438: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:47:33.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:33 smithi063 ceph-mon[31613]: pgmap v438: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:47:34.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.247+0000 7f4002e00700 1 -- 172.21.15.63:0/1156941981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc103850 msgr2=0x7f3ffc110570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.248+0000 7f4002e00700 1 --2- 172.21.15.63:0/1156941981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc103850 0x7f3ffc110570 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009990 tx=0x7f3fe8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.248+0000 7f4002e00700 1 -- 172.21.15.63:0/1156941981 shutdown_connections 2024-05-27T07:47:34.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.248+0000 7f4002e00700 1 --2- 172.21.15.63:0/1156941981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ffc110ab0 0x7f3ffc112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.248+0000 7f4002e00700 1 --2- 172.21.15.63:0/1156941981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ffc102f00 0x7f3ffc103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.248+0000 7f4002e00700 1 --2- 172.21.15.63:0/1156941981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc103850 0x7f3ffc110570 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.248+0000 7f4002e00700 1 -- 172.21.15.63:0/1156941981 >> 172.21.15.63:0/1156941981 conn(0x7f3ffc078af0 msgr2=0x7f3ffc078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.248+0000 7f4002e00700 1 -- 172.21.15.63:0/1156941981 shutdown_connections 2024-05-27T07:47:34.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.248+0000 7f4002e00700 1 -- 172.21.15.63:0/1156941981 wait complete. 2024-05-27T07:47:34.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.250+0000 7f4002e00700 1 Processor -- start 2024-05-27T07:47:34.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.250+0000 7f4002e00700 1 -- start start 2024-05-27T07:47:34.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.250+0000 7f4002e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ffc102f00 0x7f3ffc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.250+0000 7f4002e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc103850 0x7f3ffc19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.250+0000 7f4002e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ffc110ab0 0x7f3ffc1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.250+0000 7f4002e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc116340 con 0x7f3ffc103850 2024-05-27T07:47:34.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.250+0000 7f4002e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1161c0 con 0x7f3ffc110ab0 2024-05-27T07:47:34.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.250+0000 7f4002e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1164c0 con 0x7f3ffc102f00 2024-05-27T07:47:34.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.251+0000 7f4001000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc103850 0x7f3ffc19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.251+0000 7f4001000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc103850 0x7f3ffc19d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51414/0 (socket says 172.21.15.63:51414) 2024-05-27T07:47:34.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.251+0000 7f4001000700 1 -- 172.21.15.63:0/2153510493 learned_addr learned my addr 172.21.15.63:0/2153510493 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:34.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.251+0000 7f4001a00700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ffc102f00 0x7f3ffc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.251+0000 7f4002400700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ffc110ab0 0x7f3ffc1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.252+0000 7f4001000700 1 -- 172.21.15.63:0/2153510493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ffc102f00 msgr2=0x7f3ffc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.252+0000 7f4001000700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ffc102f00 0x7f3ffc19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.252+0000 7f4001000700 1 -- 172.21.15.63:0/2153510493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ffc110ab0 msgr2=0x7f3ffc1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.252+0000 7f4001000700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ffc110ab0 0x7f3ffc1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.252+0000 7f4001000700 1 -- 172.21.15.63:0/2153510493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc1a1c00 con 0x7f3ffc103850 2024-05-27T07:47:34.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.252+0000 7f4002400700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ffc110ab0 0x7f3ffc1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:34.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.252+0000 7f4001000700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc103850 0x7f3ffc19d2d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8000c00 tx=0x7f3fe8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.253+0000 7f3ffa000700 1 -- 172.21.15.63:0/2153510493 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8027070 con 0x7f3ffc103850 2024-05-27T07:47:34.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.253+0000 7f4002e00700 1 -- 172.21.15.63:0/2153510493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc1a1e90 con 0x7f3ffc103850 2024-05-27T07:47:34.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.253+0000 7f3ffa000700 1 -- 172.21.15.63:0/2153510493 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fe800fc40 con 0x7f3ffc103850 2024-05-27T07:47:34.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.253+0000 7f3ffa000700 1 -- 172.21.15.63:0/2153510493 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8022bc0 con 0x7f3ffc103850 2024-05-27T07:47:34.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.253+0000 7f4002e00700 1 -- 172.21.15.63:0/2153510493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc1a2370 con 0x7f3ffc103850 2024-05-27T07:47:34.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.257+0000 7f3ffa000700 1 -- 172.21.15.63:0/2153510493 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3fe802a070 con 0x7f3ffc103850 2024-05-27T07:47:34.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.258+0000 7f4012ebcb80 1 Processor -- start 2024-05-27T07:47:34.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.259+0000 7f4012ebcb80 1 -- start start 2024-05-27T07:47:34.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.259+0000 7f3ffa000700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3fe40b29f0 0x7f3fe40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.259+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 0x55a50576aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.259+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b410 0x55a505775ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.259+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b820 0x55a505778020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.259+0000 7f3ffa000700 1 -- 172.21.15.63:0/2153510493 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f3fe80e3a10 con 0x7f3ffc103850 2024-05-27T07:47:34.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.259+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a505699e40 con 0x55a50576b410 2024-05-27T07:47:34.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.259+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5056899e0 con 0x55a5055ece40 2024-05-27T07:47:34.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.259+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a50568da20 con 0x55a50576b820 2024-05-27T07:47:34.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.260+0000 7f4001a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 0x55a50576aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.260+0000 7f4002400700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3fe40b29f0 0x7f3fe40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.260+0000 7f4001a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 0x55a50576aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34032/0 (socket says 172.21.15.63:34032) 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.260+0000 7f4001a00700 1 -- 172.21.15.63:0/4280058817 learned_addr learned my addr 172.21.15.63:0/4280058817 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.260+0000 7f4001000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b410 0x55a505775ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.260+0000 7f4002400700 1 --2- 172.21.15.63:0/4280058817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b820 0x55a505778020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.261+0000 7f4001a00700 1 -- 172.21.15.63:0/4280058817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b820 msgr2=0x55a505778020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.261+0000 7f4001a00700 1 --2- 172.21.15.63:0/4280058817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b820 0x55a505778020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.261+0000 7f4001a00700 1 -- 172.21.15.63:0/4280058817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b410 msgr2=0x55a505775ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.261+0000 7f4001a00700 1 --2- 172.21.15.63:0/4280058817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b410 0x55a505775ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.261+0000 7f4001a00700 1 -- 172.21.15.63:0/4280058817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a505652420 con 0x55a5055ece40 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.261+0000 7f4002400700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3fe40b29f0 0x7f3fe40b4f00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009550 tx=0x7f3ff4009c90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.261+0000 7f4001a00700 1 --2- 172.21.15.63:0/4280058817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 0x55a50576aed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000bd70 tx=0x7f3ff00106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5695e5473ba93c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.262+0000 7f3feea00700 1 -- 172.21.15.63:0/4280058817 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0010e10 con 0x55a5055ece40 2024-05-27T07:47:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.262+0000 7f3feea00700 1 -- 172.21.15.63:0/4280058817 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff00118d0 con 0x55a5055ece40 2024-05-27T07:47:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.262+0000 7f3feea00700 1 -- 172.21.15.63:0/4280058817 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff001a5e0 con 0x55a5055ece40 2024-05-27T07:47:34.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.263+0000 7f4012ebcb80 1 -- 172.21.15.63:0/4280058817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 msgr2=0x55a50576aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.263+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/4280058817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 0x55a50576aed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000bd70 tx=0x7f3ff00106b0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.263+0000 7f4012ebcb80 1 -- 172.21.15.63:0/4280058817 shutdown_connections 2024-05-27T07:47:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.263+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/4280058817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b820 0x55a505778020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.263+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/4280058817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 0x55a50576aed0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.263+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/4280058817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b410 0x55a505775ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.263+0000 7f4012ebcb80 1 -- 172.21.15.63:0/4280058817 >> 172.21.15.63:0/4280058817 conn(0x55a50564cfc0 msgr2=0x55a505766d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.263+0000 7f4012ebcb80 1 -- 172.21.15.63:0/4280058817 shutdown_connections 2024-05-27T07:47:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.263+0000 7f4012ebcb80 1 -- 172.21.15.63:0/4280058817 wait complete. 2024-05-27T07:47:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.264+0000 7f4012ebcb80 1 -- start start 2024-05-27T07:47:34.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.264+0000 7f4012ebcb80 1 Processor -- start 2024-05-27T07:47:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.265+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a5057755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.265+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a50576b410 0x55a505775af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.265+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b820 0x55a505776030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.265+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a50558ffa0 con 0x55a5055ece40 2024-05-27T07:47:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.265+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a50562a760 con 0x55a50576b410 2024-05-27T07:47:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.265+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a505778990 con 0x55a50576b820 2024-05-27T07:47:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.265+0000 7f4012ebcb80 10 client.0 _ll_register_callbacks cb 0x55a505633c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.265+0000 7f4001a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a5057755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.265+0000 7f4001a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a5057755b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51426/0 (socket says 172.21.15.63:51426) 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.265+0000 7f4001a00700 1 -- 172.21.15.63:0/1358389151 learned_addr learned my addr 172.21.15.63:0/1358389151 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.266+0000 7f4002400700 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b820 0x55a505776030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.266+0000 7f4001000700 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a50576b410 0x55a505775af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.266+0000 7f4001a00700 1 -- 172.21.15.63:0/1358389151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b820 msgr2=0x55a505776030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.266+0000 7f4001a00700 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b820 0x55a505776030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.266+0000 7f4001a00700 1 -- 172.21.15.63:0/1358389151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a50576b410 msgr2=0x55a505775af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.266+0000 7f4001a00700 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a50576b410 0x55a505775af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.266+0000 7f4001a00700 1 -- 172.21.15.63:0/1358389151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5055d98d0 con 0x55a5055ece40 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.266+0000 7f4001a00700 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a5057755b0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00059d0 tx=0x7f3ff0010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45039622c918911 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.266+0000 7f3fe3400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.267+0000 7f3fe3400700 1 -- 172.21.15.63:0/1358389151 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0011c80 con 0x55a5055ece40 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.267+0000 7f3fe3400700 1 -- 172.21.15.63:0/1358389151 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f3ff0004020 con 0x55a5055ece40 2024-05-27T07:47:34.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.267+0000 7f4012ebcb80 1 -- 172.21.15.63:0/1358389151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff00118d0 con 0x55a5055ece40 2024-05-27T07:47:34.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.267+0000 7f3fe3400700 1 -- 172.21.15.63:0/1358389151 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3ff0011e20 con 0x55a5055ece40 2024-05-27T07:47:34.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.267+0000 7f4012ebcb80 10 client.18096 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:34.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.267+0000 7f4012ebcb80 1 -- 172.21.15.63:0/1358389151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a50565a060 con 0x55a5055ece40 2024-05-27T07:47:34.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.268+0000 7f3fe3400700 1 -- 172.21.15.63:0/1358389151 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff00118d0 con 0x55a5055ece40 2024-05-27T07:47:34.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.268+0000 7f3fe3400700 1 -- 172.21.15.63:0/1358389151 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3ff0004750 con 0x55a5055ece40 2024-05-27T07:47:34.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.268+0000 7f4012ebcb80 10 client.18096 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:34.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.268+0000 7f4012ebcb80 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:34.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.268+0000 7f4012ebcb80 20 client.18096 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:34.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.268+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a505785840 0x55a505787c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.268+0000 7f4012ebcb80 4 client.18096 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:34.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.268+0000 7f4012ebcb80 1 -- 172.21.15.63:0/1358389151 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a50569bd90 con 0x55a505785840 2024-05-27T07:47:34.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.269+0000 7f4001000700 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a505785840 0x55a505787c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.270+0000 7f4001000700 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a505785840 0x55a505787c60 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46a4a5d4acac1e7 server_cookie=669f04b70962b368 in_seq=0 out_seq=0 2024-05-27T07:47:34.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.270+0000 7f3fe3400700 10 client.18096 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:34.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.272+0000 7f3fe3400700 1 -- 172.21.15.63:0/1358389151 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a50569bd90 con 0x55a505785840 2024-05-27T07:47:34.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f3fe3400700 10 client.18096 handle_command_reply: tid=0 2024-05-27T07:47:34.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 client.18096 shutdown 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 -- 172.21.15.63:0/1358389151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 msgr2=0x55a5057755b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a5057755b0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00059d0 tx=0x7f3ff0010ef0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 -- 172.21.15.63:0/1358389151 shutdown_connections 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a505785840 0x55a505787c60 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b820 0x55a505776030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a5057755b0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/1358389151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a50576b410 0x55a505775af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 -- 172.21.15.63:0/1358389151 >> 172.21.15.63:0/1358389151 conn(0x55a50564cfc0 msgr2=0x55a505765ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f3fe3400700 0 client.18096 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 -- 172.21.15.63:0/1358389151 shutdown_connections 2024-05-27T07:47:34.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.273+0000 7f4012ebcb80 1 -- 172.21.15.63:0/1358389151 wait complete. 2024-05-27T07:47:34.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.274+0000 7f4012ebcb80 20 client.18096 trim_cache size 0 max 16384 2024-05-27T07:47:34.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.302+0000 7f4012ebcb80 1 Processor -- start 2024-05-27T07:47:34.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.303+0000 7f4012ebcb80 1 -- start start 2024-05-27T07:47:34.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.303+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 0x55a5057748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.303+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b410 0x55a505770d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.303+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b820 0x55a505781a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.303+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1b4d60 con 0x55a50576b820 2024-05-27T07:47:34.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.303+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1b25f0 con 0x55a5055ece40 2024-05-27T07:47:34.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.303+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1b0460 con 0x55a50576b410 2024-05-27T07:47:34.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.304+0000 7f4002400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b820 0x55a505781a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.304+0000 7f4002400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b820 0x55a505781a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51428/0 (socket says 172.21.15.63:51428) 2024-05-27T07:47:34.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.304+0000 7f4002400700 1 -- 172.21.15.63:0/3331905544 learned_addr learned my addr 172.21.15.63:0/3331905544 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:34.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.304+0000 7f4001000700 1 --2- 172.21.15.63:0/3331905544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b410 0x55a505770d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.304+0000 7f4001a00700 1 --2- 172.21.15.63:0/3331905544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 0x55a5057748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.304+0000 7f4002400700 1 -- 172.21.15.63:0/3331905544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b410 msgr2=0x55a505770d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.304+0000 7f4002400700 1 --2- 172.21.15.63:0/3331905544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b410 0x55a505770d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.304+0000 7f4002400700 1 -- 172.21.15.63:0/3331905544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 msgr2=0x55a5057748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.304+0000 7f4002400700 1 --2- 172.21.15.63:0/3331905544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 0x55a5057748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.304+0000 7f4002400700 1 -- 172.21.15.63:0/3331905544 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5057766a0 con 0x55a50576b820 2024-05-27T07:47:34.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.305+0000 7f4002400700 1 --2- 172.21.15.63:0/3331905544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b820 0x55a505781a30 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4018910 tx=0x7f3ff401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59bc6133566455 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.305+0000 7f3fee000700 1 -- 172.21.15.63:0/3331905544 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff402b070 con 0x55a50576b820 2024-05-27T07:47:34.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.305+0000 7f3fee000700 1 -- 172.21.15.63:0/3331905544 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff401ac20 con 0x55a50576b820 2024-05-27T07:47:34.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.305+0000 7f3fee000700 1 -- 172.21.15.63:0/3331905544 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4026bf0 con 0x55a50576b820 2024-05-27T07:47:34.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.306+0000 7f4012ebcb80 1 -- 172.21.15.63:0/3331905544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b820 msgr2=0x55a505781a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.306+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/3331905544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b820 0x55a505781a30 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4018910 tx=0x7f3ff401b000 comp rx=0 tx=0).stop 2024-05-27T07:47:34.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.306+0000 7f4012ebcb80 1 -- 172.21.15.63:0/3331905544 shutdown_connections 2024-05-27T07:47:34.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.307+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/3331905544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a50576b820 0x55a505781a30 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.307+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/3331905544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5055ece40 0x55a5057748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.307+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/3331905544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b410 0x55a505770d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.307+0000 7f4012ebcb80 1 -- 172.21.15.63:0/3331905544 >> 172.21.15.63:0/3331905544 conn(0x55a50564cfc0 msgr2=0x55a505775390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.307+0000 7f4012ebcb80 1 -- 172.21.15.63:0/3331905544 shutdown_connections 2024-05-27T07:47:34.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.307+0000 7f4012ebcb80 1 -- 172.21.15.63:0/3331905544 wait complete. 2024-05-27T07:47:34.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.307+0000 7f4012ebcb80 1 -- start start 2024-05-27T07:47:34.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.307+0000 7f4012ebcb80 1 Processor -- start 2024-05-27T07:47:34.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.308+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a50577e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.308+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b410 0x55a5057797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.308+0000 7f4012ebcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a50576b820 0x55a505779d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.308+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1b0ab0 con 0x55a5055ece40 2024-05-27T07:47:34.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.308+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1b1f20 con 0x55a50576b820 2024-05-27T07:47:34.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.308+0000 7f4012ebcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1b1180 con 0x55a50576b410 2024-05-27T07:47:34.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.308+0000 7f4012ebcb80 10 client.0 _ll_register_callbacks cb 0x55a505633c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:34.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.308+0000 7f4001a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a50577e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.308+0000 7f4001a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a50577e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51444/0 (socket says 172.21.15.63:51444) 2024-05-27T07:47:34.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.309+0000 7f4001a00700 1 -- 172.21.15.63:0/65012051 learned_addr learned my addr 172.21.15.63:0/65012051 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:34.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.309+0000 7f4002400700 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a50576b820 0x55a505779d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.309+0000 7f4001000700 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b410 0x55a5057797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.309+0000 7f4001a00700 1 -- 172.21.15.63:0/65012051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b410 msgr2=0x55a5057797b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.309+0000 7f4001a00700 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b410 0x55a5057797b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.309+0000 7f4001a00700 1 -- 172.21.15.63:0/65012051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a50576b820 msgr2=0x55a505779d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.309+0000 7f4001a00700 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a50576b820 0x55a505779d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.309+0000 7f4001a00700 1 -- 172.21.15.63:0/65012051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50577a4f0 con 0x55a5055ece40 2024-05-27T07:47:34.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.310+0000 7f4001a00700 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a50577e3b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00106f0 tx=0x7f3ff0011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce607919061dca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.310+0000 7f3fe3400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:34.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.310+0000 7f3fe3400700 1 -- 172.21.15.63:0/65012051 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0038070 con 0x55a5055ece40 2024-05-27T07:47:34.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.310+0000 7f4012ebcb80 1 -- 172.21.15.63:0/65012051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff401ac20 con 0x55a5055ece40 2024-05-27T07:47:34.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.310+0000 7f3fe3400700 1 -- 172.21.15.63:0/65012051 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f3ff0003c80 con 0x55a5055ece40 2024-05-27T07:47:34.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.310+0000 7f3fe3400700 1 -- 172.21.15.63:0/65012051 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3ff000b410 con 0x55a5055ece40 2024-05-27T07:47:34.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.310+0000 7f4012ebcb80 10 client.18102 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:34.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.310+0000 7f4012ebcb80 1 -- 172.21.15.63:0/65012051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a5057f01a0 con 0x55a5055ece40 2024-05-27T07:47:34.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.311+0000 7f3fe3400700 1 -- 172.21.15.63:0/65012051 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ff401ac20 con 0x55a5055ece40 2024-05-27T07:47:34.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.311+0000 7f3fe3400700 1 -- 172.21.15.63:0/65012051 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3ff0004040 con 0x55a5055ece40 2024-05-27T07:47:34.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.311+0000 7f4012ebcb80 10 client.18102 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:34.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.311+0000 7f4012ebcb80 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:34.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.311+0000 7f4012ebcb80 20 client.18102 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:34.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.311+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a5057f06c0 0x55a5057f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.311+0000 7f4012ebcb80 4 client.18102 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:47:34.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.311+0000 7f4012ebcb80 1 -- 172.21.15.63:0/65012051 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a505695570 con 0x55a5057f06c0 2024-05-27T07:47:34.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.312+0000 7f4001000700 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a5057f06c0 0x55a5057f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.313+0000 7f4001000700 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a5057f06c0 0x55a5057f2ae0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8446ea17864c0485 server_cookie=6d4ed731f0ea82d2 in_seq=0 out_seq=0 2024-05-27T07:47:34.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.313+0000 7f3fe3400700 10 client.18102 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:34.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f3fe3400700 1 -- 172.21.15.63:0/65012051 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a505695570 con 0x55a5057f06c0 2024-05-27T07:47:34.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f3fe3400700 10 client.18102 handle_command_reply: tid=0 2024-05-27T07:47:34.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 client.18102 shutdown 2024-05-27T07:47:34.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 -- 172.21.15.63:0/65012051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 msgr2=0x55a50577e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a50577e3b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00106f0 tx=0x7f3ff0011a40 comp rx=0 tx=0).stop 2024-05-27T07:47:34.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 -- 172.21.15.63:0/65012051 shutdown_connections 2024-05-27T07:47:34.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a5057f06c0 0x55a5057f2ae0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a50576b820 0x55a505779d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5055ece40 0x55a50577e3b0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 --2- 172.21.15.63:0/65012051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a50576b410 0x55a5057797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f3fe3400700 0 client.18102 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:34.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 -- 172.21.15.63:0/65012051 >> 172.21.15.63:0/65012051 conn(0x55a50564cfc0 msgr2=0x55a505774630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 -- 172.21.15.63:0/65012051 shutdown_connections 2024-05-27T07:47:34.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.314+0000 7f4012ebcb80 1 -- 172.21.15.63:0/65012051 wait complete. 2024-05-27T07:47:34.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.315+0000 7f4012ebcb80 20 client.18102 trim_cache size 0 max 16384 2024-05-27T07:47:34.315 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:47:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:47:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:47:34.315 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:47:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.315+0000 7f4002e00700 1 -- 172.21.15.63:0/2153510493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3fe40b29f0 msgr2=0x7f3fe40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.315+0000 7f4002e00700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3fe40b29f0 0x7f3fe40b4f00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009550 tx=0x7f3ff4009c90 comp rx=0 tx=0).stop 2024-05-27T07:47:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.315+0000 7f4002e00700 1 -- 172.21.15.63:0/2153510493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc103850 msgr2=0x7f3ffc19d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.315+0000 7f4002e00700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc103850 0x7f3ffc19d2d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8000c00 tx=0x7f3fe8005ef0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.316+0000 7f4002e00700 1 -- 172.21.15.63:0/2153510493 shutdown_connections 2024-05-27T07:47:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.316+0000 7f4002e00700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3fe40b29f0 0x7f3fe40b4f00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.316+0000 7f4002e00700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ffc110ab0 0x7f3ffc1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.316+0000 7f4002e00700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ffc102f00 0x7f3ffc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.316+0000 7f4002e00700 1 --2- 172.21.15.63:0/2153510493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ffc103850 0x7f3ffc19d2d0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.316+0000 7f4002e00700 1 -- 172.21.15.63:0/2153510493 >> 172.21.15.63:0/2153510493 conn(0x7f3ffc078af0 msgr2=0x7f3ffc111e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.316+0000 7f4002e00700 1 -- 172.21.15.63:0/2153510493 shutdown_connections 2024-05-27T07:47:34.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:34.316+0000 7f4002e00700 1 -- 172.21.15.63:0/2153510493 wait complete. 2024-05-27T07:47:35.250 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:47:35.250 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:47:35.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:35 smithi086 ceph-mon[34687]: pgmap v439: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:47:35.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:35 smithi149 ceph-mon[35408]: pgmap v439: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:47:35.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:35 smithi063 ceph-mon[31613]: pgmap v439: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:47:35.741 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:37.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:37 smithi086 ceph-mon[34687]: pgmap v440: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:47:37.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:37 smithi149 ceph-mon[35408]: pgmap v440: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:47:37.464 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:37 smithi063 ceph-mon[31613]: pgmap v440: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T07:47:38.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.208+0000 7f5011000700 1 -- 172.21.15.63:0/3116529869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c105c30 msgr2=0x7f500c10e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.208+0000 7f5011000700 1 --2- 172.21.15.63:0/3116529869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c105c30 0x7f500c10e180 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009990 tx=0x7f4ff8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.208+0000 7f5011000700 1 -- 172.21.15.63:0/3116529869 shutdown_connections 2024-05-27T07:47:38.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.208+0000 7f5011000700 1 --2- 172.21.15.63:0/3116529869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500c10e710 0x7f500c110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.208+0000 7f5011000700 1 --2- 172.21.15.63:0/3116529869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f500c1052e0 0x7f500c1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.208+0000 7f5011000700 1 --2- 172.21.15.63:0/3116529869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c105c30 0x7f500c10e180 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.208+0000 7f5011000700 1 -- 172.21.15.63:0/3116529869 >> 172.21.15.63:0/3116529869 conn(0x7f500c078af0 msgr2=0x7f500c078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:38.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.209+0000 7f5011000700 1 -- 172.21.15.63:0/3116529869 shutdown_connections 2024-05-27T07:47:38.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.209+0000 7f5011000700 1 -- 172.21.15.63:0/3116529869 wait complete. 2024-05-27T07:47:38.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.210+0000 7f5011000700 1 Processor -- start 2024-05-27T07:47:38.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.210+0000 7f5011000700 1 -- start start 2024-05-27T07:47:38.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.211+0000 7f5011000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c1052e0 0x7f500c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.211+0000 7f5011000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500c105c30 0x7f500c19d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.211+0000 7f5011000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f500c10e710 0x7f500c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.211+0000 7f5011000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f500c114120 con 0x7f500c1052e0 2024-05-27T07:47:38.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.211+0000 7f5011000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f500c113fa0 con 0x7f500c105c30 2024-05-27T07:47:38.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.211+0000 7f5011000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f500c1142a0 con 0x7f500c10e710 2024-05-27T07:47:38.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.211+0000 7f500b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c1052e0 0x7f500c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.211+0000 7f500b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c1052e0 0x7f500c19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51466/0 (socket says 172.21.15.63:51466) 2024-05-27T07:47:38.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.211+0000 7f500b400700 1 -- 172.21.15.63:0/3139857027 learned_addr learned my addr 172.21.15.63:0/3139857027 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:38.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.212+0000 7f500be00700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f500c10e710 0x7f500c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.212+0000 7f500aa00700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500c105c30 0x7f500c19d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.212+0000 7f500b400700 1 -- 172.21.15.63:0/3139857027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f500c10e710 msgr2=0x7f500c1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.212+0000 7f500b400700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f500c10e710 0x7f500c1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.212+0000 7f500b400700 1 -- 172.21.15.63:0/3139857027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500c105c30 msgr2=0x7f500c19d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.212+0000 7f500b400700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500c105c30 0x7f500c19d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.212+0000 7f500b400700 1 -- 172.21.15.63:0/3139857027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c1a1c60 con 0x7f500c1052e0 2024-05-27T07:47:38.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.212+0000 7f500aa00700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500c105c30 0x7f500c19d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:38.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.213+0000 7f500b400700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c1052e0 0x7f500c19cd90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00d950 tx=0x7f4ffc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:38.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.213+0000 7f5003e00700 1 -- 172.21.15.63:0/3139857027 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc016070 con 0x7f500c1052e0 2024-05-27T07:47:38.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.213+0000 7f5011000700 1 -- 172.21.15.63:0/3139857027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500c1a1f50 con 0x7f500c1052e0 2024-05-27T07:47:38.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.213+0000 7f5003e00700 1 -- 172.21.15.63:0/3139857027 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ffc009dd0 con 0x7f500c1052e0 2024-05-27T07:47:38.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.213+0000 7f5003e00700 1 -- 172.21.15.63:0/3139857027 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc00f3c0 con 0x7f500c1052e0 2024-05-27T07:47:38.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.213+0000 7f5011000700 1 -- 172.21.15.63:0/3139857027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500c1a9870 con 0x7f500c1052e0 2024-05-27T07:47:38.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.214+0000 7f5011000700 1 -- 172.21.15.63:0/3139857027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f500c06c8b0 con 0x7f500c1052e0 2024-05-27T07:47:38.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.218+0000 7f5003e00700 1 -- 172.21.15.63:0/3139857027 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4ffc019070 con 0x7f500c1052e0 2024-05-27T07:47:38.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.219+0000 7f5003e00700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ff00b2a70 0x7f4ff00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.220+0000 7f5003e00700 1 -- 172.21.15.63:0/3139857027 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f4ffc0e2600 con 0x7f500c1052e0 2024-05-27T07:47:38.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.220+0000 7f500aa00700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ff00b2a70 0x7f4ff00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.221+0000 7f500aa00700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ff00b2a70 0x7f4ff00b4f80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f500c102640 tx=0x7f4ff800b260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:38.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.223+0000 7f5003e00700 1 -- 172.21.15.63:0/3139857027 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4ffc0a9bc0 con 0x7f500c1052e0 2024-05-27T07:47:38.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.588+0000 7f5011000700 1 -- 172.21.15.63:0/3139857027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f500c0512b0 con 0x7f500c1052e0 2024-05-27T07:47:38.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.590+0000 7f5003e00700 1 -- 172.21.15.63:0/3139857027 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f4ffc0a94c0 con 0x7f500c1052e0 2024-05-27T07:47:38.589 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:47:38.590 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:47:38.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.593+0000 7f5011000700 1 -- 172.21.15.63:0/3139857027 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ff00b2a70 msgr2=0x7f4ff00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.593+0000 7f5011000700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ff00b2a70 0x7f4ff00b4f80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f500c102640 tx=0x7f4ff800b260 comp rx=0 tx=0).stop 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.593+0000 7f5011000700 1 -- 172.21.15.63:0/3139857027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c1052e0 msgr2=0x7f500c19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.593+0000 7f5011000700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c1052e0 0x7f500c19cd90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00d950 tx=0x7f4ffc00dc60 comp rx=0 tx=0).stop 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.594+0000 7f5011000700 1 -- 172.21.15.63:0/3139857027 shutdown_connections 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.594+0000 7f5011000700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ff00b2a70 0x7f4ff00b4f80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.594+0000 7f5011000700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f500c10e710 0x7f500c1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.594+0000 7f5011000700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f500c1052e0 0x7f500c19cd90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.594+0000 7f5011000700 1 --2- 172.21.15.63:0/3139857027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f500c105c30 0x7f500c19d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.594+0000 7f5011000700 1 -- 172.21.15.63:0/3139857027 >> 172.21.15.63:0/3139857027 conn(0x7f500c078af0 msgr2=0x7f500c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.594+0000 7f5011000700 1 -- 172.21.15.63:0/3139857027 shutdown_connections 2024-05-27T07:47:38.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:38.594+0000 7f5011000700 1 -- 172.21.15.63:0/3139857027 wait complete. 2024-05-27T07:47:38.595 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:47:39.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:39 smithi086 ceph-mon[34687]: pgmap v441: 129 pgs: 129 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:47:39.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:39 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3139857027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:47:39.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:39 smithi149 ceph-mon[35408]: pgmap v441: 129 pgs: 129 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:47:39.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:39 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3139857027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:47:39.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:39 smithi063 ceph-mon[31613]: pgmap v441: 129 pgs: 129 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:47:39.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:39 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3139857027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:47:39.583 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:47:40.098 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:41.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:41 smithi086 ceph-mon[34687]: pgmap v442: 129 pgs: 129 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:47:41.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:41 smithi149 ceph-mon[35408]: pgmap v442: 129 pgs: 129 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:47:41.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:41 smithi063 ceph-mon[31613]: pgmap v442: 129 pgs: 129 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:47:42.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.654+0000 7fa60c400700 1 -- 172.21.15.63:0/1959368822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6041045e0 msgr2=0x7fa6041049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.654+0000 7fa60c400700 1 --2- 172.21.15.63:0/1959368822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6041045e0 0x7fa6041049f0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80099c0 tx=0x7fa5f8009cd0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.655+0000 7fa60c400700 1 -- 172.21.15.63:0/1959368822 shutdown_connections 2024-05-27T07:47:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.655+0000 7fa60c400700 1 --2- 172.21.15.63:0/1959368822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa604109860 0x7fa6041111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.655+0000 7fa60c400700 1 --2- 172.21.15.63:0/1959368822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6041045e0 0x7fa6041049f0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.655+0000 7fa60c400700 1 --2- 172.21.15.63:0/1959368822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa604108f10 0x7fa604109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.655+0000 7fa60c400700 1 -- 172.21.15.63:0/1959368822 >> 172.21.15.63:0/1959368822 conn(0x7fa604100030 msgr2=0x7fa604102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.655+0000 7fa60c400700 1 -- 172.21.15.63:0/1959368822 shutdown_connections 2024-05-27T07:47:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.655+0000 7fa60c400700 1 -- 172.21.15.63:0/1959368822 wait complete. 2024-05-27T07:47:42.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.656+0000 7fa60c400700 1 Processor -- start 2024-05-27T07:47:42.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.656+0000 7fa60c400700 1 -- start start 2024-05-27T07:47:42.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.657+0000 7fa60c400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6041045e0 0x7fa60419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.657+0000 7fa60c400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa604108f10 0x7fa60419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.657+0000 7fa60c400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa604109860 0x7fa6041a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.657+0000 7fa60c400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa604114390 con 0x7fa6041045e0 2024-05-27T07:47:42.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.657+0000 7fa60c400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa604114210 con 0x7fa604109860 2024-05-27T07:47:42.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.657+0000 7fa60c400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa604114510 con 0x7fa604108f10 2024-05-27T07:47:42.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.657+0000 7fa609a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6041045e0 0x7fa60419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.657+0000 7fa609a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6041045e0 0x7fa60419d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37306/0 (socket says 172.21.15.63:37306) 2024-05-27T07:47:42.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.657+0000 7fa609a00700 1 -- 172.21.15.63:0/1984935172 learned_addr learned my addr 172.21.15.63:0/1984935172 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:42.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.657+0000 7fa609000700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa604108f10 0x7fa60419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.658+0000 7fa60a400700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa604109860 0x7fa6041a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.658+0000 7fa609a00700 1 -- 172.21.15.63:0/1984935172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa604108f10 msgr2=0x7fa60419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.658+0000 7fa609a00700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa604108f10 0x7fa60419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.658+0000 7fa609a00700 1 -- 172.21.15.63:0/1984935172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa604109860 msgr2=0x7fa6041a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.658+0000 7fa609a00700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa604109860 0x7fa6041a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.658+0000 7fa609a00700 1 -- 172.21.15.63:0/1984935172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6041a1e20 con 0x7fa6041045e0 2024-05-27T07:47:42.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.659+0000 7fa609a00700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6041045e0 0x7fa60419d000 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8000c00 tx=0x7fa5f800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.659+0000 7fa602000700 1 -- 172.21.15.63:0/1984935172 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8027070 con 0x7fa6041045e0 2024-05-27T07:47:42.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.659+0000 7fa60c400700 1 -- 172.21.15.63:0/1984935172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6041a20b0 con 0x7fa6041045e0 2024-05-27T07:47:42.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.659+0000 7fa602000700 1 -- 172.21.15.63:0/1984935172 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5f80056d0 con 0x7fa6041045e0 2024-05-27T07:47:42.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.659+0000 7fa602000700 1 -- 172.21.15.63:0/1984935172 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8022e50 con 0x7fa6041045e0 2024-05-27T07:47:42.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.659+0000 7fa60c400700 1 -- 172.21.15.63:0/1984935172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6041a2590 con 0x7fa6041045e0 2024-05-27T07:47:42.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.663+0000 7fa602000700 1 -- 172.21.15.63:0/1984935172 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa5f802a070 con 0x7fa6041045e0 2024-05-27T07:47:42.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.665+0000 7fa61b032b80 1 Processor -- start 2024-05-27T07:47:42.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.665+0000 7fa61b032b80 1 -- start start 2024-05-27T07:47:42.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4ad5ae40 0x55ca4aedf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aedfd30 0x55ca4aed9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 0x55ca4aeda1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa61b032b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ca4ae07e40 con 0x55ca4ad5ae40 2024-05-27T07:47:42.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa61b032b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca4adf79e0 con 0x55ca4aed9db0 2024-05-27T07:47:42.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa61b032b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ca4adfba20 con 0x55ca4aedfd30 2024-05-27T07:47:42.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa60a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4ad5ae40 0x55ca4aedf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa602000700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa5ec0b2b90 0x7fa5ec0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa60a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4ad5ae40 0x55ca4aedf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37316/0 (socket says 172.21.15.63:37316) 2024-05-27T07:47:42.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa60a400700 1 -- 172.21.15.63:0/2057165398 learned_addr learned my addr 172.21.15.63:0/2057165398 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:42.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa602000700 1 -- 172.21.15.63:0/1984935172 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fa5f80e3a00 con 0x7fa6041045e0 2024-05-27T07:47:42.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.666+0000 7fa609a00700 1 --2- 172.21.15.63:0/2057165398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aedfd30 0x55ca4aed9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.667+0000 7fa609000700 1 --2- 172.21.15.63:0/2057165398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 0x55ca4aeda1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.667+0000 7fa60a400700 1 -- 172.21.15.63:0/2057165398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aedfd30 msgr2=0x55ca4aed9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.667+0000 7fa60a400700 1 --2- 172.21.15.63:0/2057165398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aedfd30 0x55ca4aed9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.667+0000 7fa60a400700 1 -- 172.21.15.63:0/2057165398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 msgr2=0x55ca4aeda1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.667+0000 7fa60a400700 1 --2- 172.21.15.63:0/2057165398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 0x55ca4aeda1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.667+0000 7fa60a400700 1 -- 172.21.15.63:0/2057165398 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4adc0420 con 0x55ca4ad5ae40 2024-05-27T07:47:42.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.667+0000 7fa60a400700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa5ec0b2b90 0x7fa5ec0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.668+0000 7fa60a400700 1 --2- 172.21.15.63:0/2057165398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4ad5ae40 0x55ca4aedf7f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc011530 tx=0x7fa5fc011840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bf7f965fa6a1ed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.668+0000 7fa60a400700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa5ec0b2b90 0x7fa5ec0b50a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc020000 tx=0x7fa5fc01d000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.668+0000 7fa5f6a00700 1 -- 172.21.15.63:0/2057165398 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc019070 con 0x55ca4ad5ae40 2024-05-27T07:47:42.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.668+0000 7fa5f6a00700 1 -- 172.21.15.63:0/2057165398 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5fc01e430 con 0x55ca4ad5ae40 2024-05-27T07:47:42.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.668+0000 7fa5f6a00700 1 -- 172.21.15.63:0/2057165398 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc01fe60 con 0x55ca4ad5ae40 2024-05-27T07:47:42.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.669+0000 7fa61b032b80 1 -- 172.21.15.63:0/2057165398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4ad5ae40 msgr2=0x55ca4aedf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.669+0000 7fa61b032b80 1 --2- 172.21.15.63:0/2057165398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4ad5ae40 0x55ca4aedf7f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc011530 tx=0x7fa5fc011840 comp rx=0 tx=0).stop 2024-05-27T07:47:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.669+0000 7fa61b032b80 1 -- 172.21.15.63:0/2057165398 shutdown_connections 2024-05-27T07:47:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.669+0000 7fa61b032b80 1 --2- 172.21.15.63:0/2057165398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 0x55ca4aeda1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.669+0000 7fa61b032b80 1 --2- 172.21.15.63:0/2057165398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4ad5ae40 0x55ca4aedf7f0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.669+0000 7fa61b032b80 1 --2- 172.21.15.63:0/2057165398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aedfd30 0x55ca4aed9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.669+0000 7fa61b032b80 1 -- 172.21.15.63:0/2057165398 >> 172.21.15.63:0/2057165398 conn(0x55ca4adbafc0 msgr2=0x55ca4aed48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.670+0000 7fa61b032b80 1 -- 172.21.15.63:0/2057165398 shutdown_connections 2024-05-27T07:47:42.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.670+0000 7fa61b032b80 1 -- 172.21.15.63:0/2057165398 wait complete. 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.670+0000 7fa61b032b80 1 -- start start 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.671+0000 7fa61b032b80 1 Processor -- start 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.671+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 0x55ca4aee34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.671+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aed9db0 0x55ca4aee3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.671+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aedfd30 0x55ca4aee3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.671+0000 7fa61b032b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ca4acfdfa0 con 0x55ca4aed9db0 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.671+0000 7fa61b032b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca4ad98760 con 0x55ca4ad5ae40 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.671+0000 7fa61b032b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ca4aee6800 con 0x55ca4aedfd30 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.672+0000 7fa61b032b80 10 client.0 _ll_register_callbacks cb 0x55ca4ada1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.672+0000 7fa609000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aed9db0 0x55ca4aee3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.672+0000 7fa609000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aed9db0 0x55ca4aee3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37328/0 (socket says 172.21.15.63:37328) 2024-05-27T07:47:42.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.672+0000 7fa609000700 1 -- 172.21.15.63:0/564777302 learned_addr learned my addr 172.21.15.63:0/564777302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:42.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.672+0000 7fa609a00700 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 0x55ca4aee34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.672+0000 7fa60a400700 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aedfd30 0x55ca4aee3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.673+0000 7fa609000700 1 -- 172.21.15.63:0/564777302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aedfd30 msgr2=0x55ca4aee3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.673+0000 7fa609000700 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aedfd30 0x55ca4aee3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.673+0000 7fa609000700 1 -- 172.21.15.63:0/564777302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 msgr2=0x55ca4aee34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.673+0000 7fa609000700 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 0x55ca4aee34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.673+0000 7fa609000700 1 -- 172.21.15.63:0/564777302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4ad478d0 con 0x55ca4aed9db0 2024-05-27T07:47:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.673+0000 7fa609a00700 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 0x55ca4aee34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.673+0000 7fa609000700 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aed9db0 0x55ca4aee3a10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0010a50 tx=0x7fa5f0010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5021dd2d7df99072 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.673+0000 7fa5eb400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.674+0000 7fa5eb400700 1 -- 172.21.15.63:0/564777302 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000ae40 con 0x55ca4aed9db0 2024-05-27T07:47:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.674+0000 7fa5eb400700 1 -- 172.21.15.63:0/564777302 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fa5f0004260 con 0x55ca4aed9db0 2024-05-27T07:47:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.674+0000 7fa61b032b80 1 -- 172.21.15.63:0/564777302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5fc01e430 con 0x55ca4aed9db0 2024-05-27T07:47:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.674+0000 7fa5eb400700 1 -- 172.21.15.63:0/564777302 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa5f0004460 con 0x55ca4aed9db0 2024-05-27T07:47:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.674+0000 7fa61b032b80 10 client.18120 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.674+0000 7fa61b032b80 1 -- 172.21.15.63:0/564777302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ca4adc8060 con 0x55ca4aed9db0 2024-05-27T07:47:42.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.675+0000 7fa5eb400700 1 -- 172.21.15.63:0/564777302 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5fc01e430 con 0x55ca4aed9db0 2024-05-27T07:47:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.675+0000 7fa5eb400700 1 -- 172.21.15.63:0/564777302 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa5f0003b10 con 0x55ca4aed9db0 2024-05-27T07:47:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.675+0000 7fa61b032b80 10 client.18120 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.675+0000 7fa61b032b80 10 client.18120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:47:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.675+0000 7fa61b032b80 20 client.18120 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.675+0000 7fa61b032b80 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ca4aef3720 0x55ca4aef5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.675+0000 7fa61b032b80 4 client.18120 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.675+0000 7fa61b032b80 1 -- 172.21.15.63:0/564777302 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca4ae09d90 con 0x55ca4aef3720 2024-05-27T07:47:42.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.676+0000 7fa609a00700 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ca4aef3720 0x55ca4aef5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.677+0000 7fa609a00700 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ca4aef3720 0x55ca4aef5b80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73fc9136c552de1e server_cookie=eb9319ccdebd2009 in_seq=0 out_seq=0 2024-05-27T07:47:42.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.677+0000 7fa5eb400700 10 client.18120 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:47:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa5eb400700 1 -- 172.21.15.63:0/564777302 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ca4ae09d90 con 0x55ca4aef3720 2024-05-27T07:47:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa5eb400700 10 client.18120 handle_command_reply: tid=0 2024-05-27T07:47:42.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa61b032b80 1 client.18120 shutdown 2024-05-27T07:47:42.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa61b032b80 1 -- 172.21.15.63:0/564777302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aed9db0 msgr2=0x55ca4aee3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa61b032b80 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aed9db0 0x55ca4aee3a10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0010a50 tx=0x7fa5f0010d60 comp rx=0 tx=0).stop 2024-05-27T07:47:42.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa61b032b80 1 -- 172.21.15.63:0/564777302 shutdown_connections 2024-05-27T07:47:42.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa61b032b80 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aedfd30 0x55ca4aee3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa61b032b80 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 0x55ca4aee34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa61b032b80 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ca4aef3720 0x55ca4aef5b80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa61b032b80 1 --2- 172.21.15.63:0/564777302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aed9db0 0x55ca4aee3a10 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa61b032b80 1 -- 172.21.15.63:0/564777302 >> 172.21.15.63:0/564777302 conn(0x55ca4adbafc0 msgr2=0x55ca4aeda1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.679+0000 7fa5eb400700 0 client.18120 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:47:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.680+0000 7fa61b032b80 1 -- 172.21.15.63:0/564777302 shutdown_connections 2024-05-27T07:47:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.680+0000 7fa61b032b80 1 -- 172.21.15.63:0/564777302 wait complete. 2024-05-27T07:47:42.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.680+0000 7fa61b032b80 20 client.18120 trim_cache size 0 max 16384 2024-05-27T07:47:42.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.704+0000 7fa61b032b80 1 Processor -- start 2024-05-27T07:47:42.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.704+0000 7fa61b032b80 1 -- start start 2024-05-27T07:47:42.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.704+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 0x55ca4aed9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.705+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aed9db0 0x55ca4aeddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.705+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aee5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.705+0000 7fa61b032b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041b4e50 con 0x55ca4aedfd30 2024-05-27T07:47:42.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.705+0000 7fa61b032b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041b26e0 con 0x55ca4ad5ae40 2024-05-27T07:47:42.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.705+0000 7fa61b032b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041b0550 con 0x55ca4aed9db0 2024-05-27T07:47:42.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.705+0000 7fa60a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aee5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.705+0000 7fa60a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aee5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37332/0 (socket says 172.21.15.63:37332) 2024-05-27T07:47:42.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.705+0000 7fa60a400700 1 -- 172.21.15.63:0/3677607953 learned_addr learned my addr 172.21.15.63:0/3677607953 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:42.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.705+0000 7fa609a00700 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 0x55ca4aed9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.705+0000 7fa609000700 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aed9db0 0x55ca4aeddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa60a400700 1 -- 172.21.15.63:0/3677607953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aed9db0 msgr2=0x55ca4aeddc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa60a400700 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aed9db0 0x55ca4aeddc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa60a400700 1 -- 172.21.15.63:0/3677607953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 msgr2=0x55ca4aed9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa60a400700 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 0x55ca4aed9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa60a400700 1 -- 172.21.15.63:0/3677607953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4aee0140 con 0x55ca4aedfd30 2024-05-27T07:47:42.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa609000700 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aed9db0 0x55ca4aeddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:42.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa609a00700 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 0x55ca4aed9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:42.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa60a400700 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aee5970 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0145a0 tx=0x7fa5fc007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e922e03c3cdca94 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa5f6000700 1 -- 172.21.15.63:0/3677607953 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc019040 con 0x55ca4aedfd30 2024-05-27T07:47:42.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa5f6000700 1 -- 172.21.15.63:0/3677607953 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5fc01fde0 con 0x55ca4aedfd30 2024-05-27T07:47:42.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.706+0000 7fa5f6000700 1 -- 172.21.15.63:0/3677607953 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc01e430 con 0x55ca4aedfd30 2024-05-27T07:47:42.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.707+0000 7fa61b032b80 1 -- 172.21.15.63:0/3677607953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 msgr2=0x55ca4aee5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.707+0000 7fa61b032b80 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aee5970 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0145a0 tx=0x7fa5fc007cb0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.708+0000 7fa61b032b80 1 -- 172.21.15.63:0/3677607953 shutdown_connections 2024-05-27T07:47:42.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.708+0000 7fa61b032b80 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aee5970 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.708+0000 7fa61b032b80 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4ad5ae40 0x55ca4aed9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.708+0000 7fa61b032b80 1 --2- 172.21.15.63:0/3677607953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4aed9db0 0x55ca4aeddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.708+0000 7fa61b032b80 1 -- 172.21.15.63:0/3677607953 >> 172.21.15.63:0/3677607953 conn(0x55ca4adbafc0 msgr2=0x55ca4aede650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.708+0000 7fa61b032b80 1 -- 172.21.15.63:0/3677607953 shutdown_connections 2024-05-27T07:47:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.708+0000 7fa61b032b80 1 -- 172.21.15.63:0/3677607953 wait complete. 2024-05-27T07:47:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.708+0000 7fa61b032b80 1 -- start start 2024-05-27T07:47:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.709+0000 7fa61b032b80 1 Processor -- start 2024-05-27T07:47:42.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.709+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4ad5ae40 0x55ca4aeebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.709+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 0x55ca4aeec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.709+0000 7fa61b032b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aefa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.709+0000 7fa61b032b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041b0ba0 con 0x55ca4aedfd30 2024-05-27T07:47:42.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.709+0000 7fa61b032b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041b2010 con 0x55ca4aed9db0 2024-05-27T07:47:42.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.709+0000 7fa61b032b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041b1270 con 0x55ca4ad5ae40 2024-05-27T07:47:42.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.709+0000 7fa61b032b80 10 client.0 _ll_register_callbacks cb 0x55ca4ada1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:42.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa60a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aefa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa60a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aefa770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37338/0 (socket says 172.21.15.63:37338) 2024-05-27T07:47:42.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa60a400700 1 -- 172.21.15.63:0/2552797646 learned_addr learned my addr 172.21.15.63:0/2552797646 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa609000700 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 0x55ca4aeec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa609a00700 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4ad5ae40 0x55ca4aeebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa60a400700 1 -- 172.21.15.63:0/2552797646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4ad5ae40 msgr2=0x55ca4aeebcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa60a400700 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4ad5ae40 0x55ca4aeebcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa60a400700 1 -- 172.21.15.63:0/2552797646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 msgr2=0x55ca4aeec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa60a400700 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 0x55ca4aeec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa60a400700 1 -- 172.21.15.63:0/2552797646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4aefadb0 con 0x55ca4aedfd30 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.710+0000 7fa609000700 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 0x55ca4aeec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.711+0000 7fa60a400700 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aefa770 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0143f0 tx=0x7fa5fc0074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54bdd061a0aac7a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.711+0000 7fa5eb400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.711+0000 7fa5eb400700 1 -- 172.21.15.63:0/2552797646 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc019040 con 0x55ca4aedfd30 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.711+0000 7fa61b032b80 1 -- 172.21.15.63:0/2552797646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5fc01fde0 con 0x55ca4aedfd30 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.711+0000 7fa5eb400700 1 -- 172.21.15.63:0/2552797646 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fa5fc014a50 con 0x55ca4aedfd30 2024-05-27T07:47:42.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa5eb400700 1 -- 172.21.15.63:0/2552797646 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa5fc043050 con 0x55ca4aedfd30 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa61b032b80 10 client.18126 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa61b032b80 1 -- 172.21.15.63:0/2552797646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ca4af5daf0 con 0x55ca4aedfd30 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa5eb400700 1 -- 172.21.15.63:0/2552797646 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5fc01fde0 con 0x55ca4aedfd30 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa5eb400700 1 -- 172.21.15.63:0/2552797646 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa5fc030020 con 0x55ca4aedfd30 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa61b032b80 10 client.18126 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa61b032b80 10 client.18126 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa61b032b80 20 client.18126 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa61b032b80 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ca4af5ddb0 0x55ca4af601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa61b032b80 4 client.18126 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.712+0000 7fa61b032b80 1 -- 172.21.15.63:0/2552797646 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca4ae03570 con 0x55ca4af5ddb0 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.713+0000 7fa609a00700 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ca4af5ddb0 0x55ca4af601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.714+0000 7fa609a00700 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ca4af5ddb0 0x55ca4af601d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1de93d26ffb7445 server_cookie=e0f4bc22430780b6 in_seq=0 out_seq=0 2024-05-27T07:47:42.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.714+0000 7fa5eb400700 10 client.18126 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:47:42.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.714+0000 7fa5eb400700 1 -- 172.21.15.63:0/2552797646 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca4ae03570 con 0x55ca4af5ddb0 2024-05-27T07:47:42.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.714+0000 7fa5eb400700 10 client.18126 handle_command_reply: tid=0 2024-05-27T07:47:42.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.714+0000 7fa61b032b80 1 client.18126 shutdown 2024-05-27T07:47:42.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.714+0000 7fa61b032b80 1 -- 172.21.15.63:0/2552797646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 msgr2=0x55ca4aefa770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.714+0000 7fa61b032b80 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aefa770 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0143f0 tx=0x7fa5fc0074e0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.715+0000 7fa61b032b80 1 -- 172.21.15.63:0/2552797646 shutdown_connections 2024-05-27T07:47:42.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.715+0000 7fa61b032b80 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ca4af5ddb0 0x55ca4af601d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.715+0000 7fa61b032b80 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca4aedfd30 0x55ca4aefa770 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.715+0000 7fa61b032b80 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca4ad5ae40 0x55ca4aeebcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.715+0000 7fa61b032b80 1 --2- 172.21.15.63:0/2552797646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4aed9db0 0x55ca4aeec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.715+0000 7fa5eb400700 0 client.18126 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:47:42.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.715+0000 7fa61b032b80 1 -- 172.21.15.63:0/2552797646 >> 172.21.15.63:0/2552797646 conn(0x55ca4adbafc0 msgr2=0x55ca4aee32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:42.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.715+0000 7fa61b032b80 1 -- 172.21.15.63:0/2552797646 shutdown_connections 2024-05-27T07:47:42.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.715+0000 7fa61b032b80 1 -- 172.21.15.63:0/2552797646 wait complete. 2024-05-27T07:47:42.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.715+0000 7fa61b032b80 20 client.18126 trim_cache size 0 max 16384 2024-05-27T07:47:42.722 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:47:42.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 -- 172.21.15.63:0/1984935172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa5ec0b2b90 msgr2=0x7fa5ec0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa5ec0b2b90 0x7fa5ec0b50a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc020000 tx=0x7fa5fc01d000 comp rx=0 tx=0).stop 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 -- 172.21.15.63:0/1984935172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6041045e0 msgr2=0x7fa60419d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6041045e0 0x7fa60419d000 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8000c00 tx=0x7fa5f800f690 comp rx=0 tx=0).stop 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 -- 172.21.15.63:0/1984935172 shutdown_connections 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa5ec0b2b90 0x7fa5ec0b50a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa604109860 0x7fa6041a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6041045e0 0x7fa60419d000 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 --2- 172.21.15.63:0/1984935172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa604108f10 0x7fa60419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 -- 172.21.15.63:0/1984935172 >> 172.21.15.63:0/1984935172 conn(0x7fa604100030 msgr2=0x7fa60410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 -- 172.21.15.63:0/1984935172 shutdown_connections 2024-05-27T07:47:42.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:42.716+0000 7fa60c400700 1 -- 172.21.15.63:0/1984935172 wait complete. 2024-05-27T07:47:43.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:43 smithi086 ceph-mon[34687]: pgmap v443: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:47:43.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:43 smithi149 ceph-mon[35408]: pgmap v443: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:47:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:43 smithi063 ceph-mon[31613]: pgmap v443: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:47:43.779 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:47:44.275 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:45.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:45 smithi086 ceph-mon[34687]: pgmap v444: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:47:45.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:45 smithi149 ceph-mon[35408]: pgmap v444: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:47:45.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:45 smithi063 ceph-mon[31613]: pgmap v444: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:47:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.696+0000 7fcf94400640 1 -- 172.21.15.63:0/2458370642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf8c104df0 msgr2=0x7fcf8c105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.696+0000 7fcf94400640 1 --2- 172.21.15.63:0/2458370642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf8c104df0 0x7fcf8c105200 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009ca0 tx=0x7fcf8002a9e0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.697+0000 7fcf94400640 1 -- 172.21.15.63:0/2458370642 shutdown_connections 2024-05-27T07:47:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.697+0000 7fcf94400640 1 --2- 172.21.15.63:0/2458370642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf8c10e200 0x7fcf8c1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.697+0000 7fcf94400640 1 --2- 172.21.15.63:0/2458370642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf8c10b850 0x7fcf8c10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.697+0000 7fcf94400640 1 --2- 172.21.15.63:0/2458370642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf8c104df0 0x7fcf8c105200 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.697+0000 7fcf94400640 1 -- 172.21.15.63:0/2458370642 >> 172.21.15.63:0/2458370642 conn(0x7fcf8c0782d0 msgr2=0x7fcf8c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.697+0000 7fcf94400640 1 -- 172.21.15.63:0/2458370642 shutdown_connections 2024-05-27T07:47:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.697+0000 7fcf94400640 1 -- 172.21.15.63:0/2458370642 wait complete. 2024-05-27T07:47:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.698+0000 7fcf94400640 1 Processor -- start 2024-05-27T07:47:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.699+0000 7fcf94400640 1 -- start start 2024-05-27T07:47:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.699+0000 7fcf94400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf8c104df0 0x7fcf8c1a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.699+0000 7fcf94400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf8c10b850 0x7fcf8c1a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.699+0000 7fcf94400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf8c10e200 0x7fcf8c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.699+0000 7fcf94400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c103b20 con 0x7fcf8c10e200 2024-05-27T07:47:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.699+0000 7fcf94400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c1039a0 con 0x7fcf8c10b850 2024-05-27T07:47:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.699+0000 7fcf94400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c103ca0 con 0x7fcf8c104df0 2024-05-27T07:47:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.700+0000 7fcf92400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf8c10e200 0x7fcf8c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.700+0000 7fcf92400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf8c10e200 0x7fcf8c19c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37362/0 (socket says 172.21.15.63:37362) 2024-05-27T07:47:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.700+0000 7fcf92400640 1 -- 172.21.15.63:0/1479918714 learned_addr learned my addr 172.21.15.63:0/1479918714 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.700+0000 7fcf91a00640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf8c104df0 0x7fcf8c1a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.700+0000 7fcf91000640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf8c10b850 0x7fcf8c1a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.700+0000 7fcf92400640 1 -- 172.21.15.63:0/1479918714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf8c104df0 msgr2=0x7fcf8c1a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.700+0000 7fcf92400640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf8c104df0 0x7fcf8c1a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.700+0000 7fcf92400640 1 -- 172.21.15.63:0/1479918714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf8c10b850 msgr2=0x7fcf8c1a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.700+0000 7fcf92400640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf8c10b850 0x7fcf8c1a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.700+0000 7fcf92400640 1 -- 172.21.15.63:0/1479918714 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c19ce90 con 0x7fcf8c10e200 2024-05-27T07:47:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.701+0000 7fcf92400640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf8c10e200 0x7fcf8c19c7b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400d750 tx=0x7fcf8400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.701+0000 7fcf8a000640 1 -- 172.21.15.63:0/1479918714 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84017070 con 0x7fcf8c10e200 2024-05-27T07:47:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.702+0000 7fcf8a000640 1 -- 172.21.15.63:0/1479918714 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf840054b0 con 0x7fcf8c10e200 2024-05-27T07:47:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.702+0000 7fcf8a000640 1 -- 172.21.15.63:0/1479918714 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84004b90 con 0x7fcf8c10e200 2024-05-27T07:47:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.702+0000 7fcf94400640 1 -- 172.21.15.63:0/1479918714 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8c19d180 con 0x7fcf8c10e200 2024-05-27T07:47:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.702+0000 7fcf94400640 1 -- 172.21.15.63:0/1479918714 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8c1a93d0 con 0x7fcf8c10e200 2024-05-27T07:47:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.706+0000 7fcf9b417740 1 Processor -- start 2024-05-27T07:47:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.706+0000 7fcf8a000640 1 -- 172.21.15.63:0/1479918714 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcf84004d30 con 0x7fcf8c10e200 2024-05-27T07:47:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.706+0000 7fcf9b417740 1 -- start start 2024-05-27T07:47:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.707+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x56060137e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.707+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fb32f0 0x56060137eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.707+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fbb4d0 0x56060137f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.707+0000 7fcf9b417740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5606012efb90 con 0x560600fdc1b0 2024-05-27T07:47:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.707+0000 7fcf9b417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5606012df730 con 0x560600fb32f0 2024-05-27T07:47:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.707+0000 7fcf9b417740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5606012e3770 con 0x560600fbb4d0 2024-05-27T07:47:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.707+0000 7fcf91000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x56060137e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.707+0000 7fcf91000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x56060137e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37366/0 (socket says 172.21.15.63:37366) 2024-05-27T07:47:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.707+0000 7fcf91000640 1 -- 172.21.15.63:0/3948855649 learned_addr learned my addr 172.21.15.63:0/3948855649 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.708+0000 7fcf91a00640 1 --2- 172.21.15.63:0/3948855649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fb32f0 0x56060137eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.708+0000 7fcf91000640 1 -- 172.21.15.63:0/3948855649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fbb4d0 msgr2=0x56060137f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.708+0000 7fcf91000640 1 --2- 172.21.15.63:0/3948855649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fbb4d0 0x56060137f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.708+0000 7fcf91000640 1 -- 172.21.15.63:0/3948855649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fb32f0 msgr2=0x56060137eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.708+0000 7fcf91000640 1 --2- 172.21.15.63:0/3948855649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fb32f0 0x56060137eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.708+0000 7fcf91000640 1 -- 172.21.15.63:0/3948855649 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606012788b0 con 0x560600fdc1b0 2024-05-27T07:47:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.708+0000 7fcf8a000640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcf740b29d0 0x7fcf740b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.708+0000 7fcf91000640 1 --2- 172.21.15.63:0/3948855649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x56060137e860 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800bd00 tx=0x7fcf780127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9541e81fdf1190d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.709+0000 7fcf8a000640 1 -- 172.21.15.63:0/1479918714 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fcf840e2280 con 0x7fcf8c10e200 2024-05-27T07:47:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.709+0000 7fcf91a00640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcf740b29d0 0x7fcf740b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.709+0000 7fcf7ea00640 1 -- 172.21.15.63:0/3948855649 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78005020 con 0x560600fdc1b0 2024-05-27T07:47:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.709+0000 7fcf7ea00640 1 -- 172.21.15.63:0/3948855649 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf78016070 con 0x560600fdc1b0 2024-05-27T07:47:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.709+0000 7fcf7ea00640 1 -- 172.21.15.63:0/3948855649 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78004d90 con 0x560600fdc1b0 2024-05-27T07:47:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.710+0000 7fcf9b417740 1 -- 172.21.15.63:0/3948855649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 msgr2=0x56060137e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.710+0000 7fcf91a00640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcf740b29d0 0x7fcf740b4ee0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c19dae0 tx=0x7fcf80005d80 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.710+0000 7fcf9b417740 1 --2- 172.21.15.63:0/3948855649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x56060137e860 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800bd00 tx=0x7fcf780127c0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.710+0000 7fcf9b417740 1 -- 172.21.15.63:0/3948855649 shutdown_connections 2024-05-27T07:47:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.710+0000 7fcf9b417740 1 --2- 172.21.15.63:0/3948855649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fbb4d0 0x56060137f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.710+0000 7fcf9b417740 1 --2- 172.21.15.63:0/3948855649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fb32f0 0x56060137eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.710+0000 7fcf9b417740 1 --2- 172.21.15.63:0/3948855649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x56060137e860 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.710+0000 7fcf9b417740 1 -- 172.21.15.63:0/3948855649 >> 172.21.15.63:0/3948855649 conn(0x560600fd0300 msgr2=0x56060137b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.710+0000 7fcf9b417740 1 -- 172.21.15.63:0/3948855649 shutdown_connections 2024-05-27T07:47:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.710+0000 7fcf9b417740 1 -- 172.21.15.63:0/3948855649 wait complete. 2024-05-27T07:47:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.711+0000 7fcf9b417740 1 -- start start 2024-05-27T07:47:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.711+0000 7fcf9b417740 1 Processor -- start 2024-05-27T07:47:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x560601388a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x560601388f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 0x560601394cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf9b417740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560600fc5790 con 0x560600fb32f0 2024-05-27T07:47:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf9b417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560601287bc0 con 0x560600fbb4d0 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf9b417740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56060138b990 con 0x560600fdc1b0 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf9b417740 10 client.0 _ll_register_callbacks cb 0x560601265400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf91a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x560601388a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf91a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x560601388a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37376/0 (socket says 172.21.15.63:37376) 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf91a00640 1 -- 172.21.15.63:0/1926417763 learned_addr learned my addr 172.21.15.63:0/1926417763 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf91000640 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x560601388f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.712+0000 7fcf92400640 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 0x560601394cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.713+0000 7fcf91a00640 1 -- 172.21.15.63:0/1926417763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 msgr2=0x560601394cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.713+0000 7fcf91a00640 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 0x560601394cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.713+0000 7fcf91a00640 1 -- 172.21.15.63:0/1926417763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 msgr2=0x560601388f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.713+0000 7fcf91a00640 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x560601388f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.713+0000 7fcf91a00640 1 -- 172.21.15.63:0/1926417763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560600ff4210 con 0x560600fb32f0 2024-05-27T07:47:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.713+0000 7fcf91a00640 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x560601388a30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fcf80035000 tx=0x7fcf800095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0493d41af88649 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.713+0000 7fcf73400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.713+0000 7fcf73400640 1 -- 172.21.15.63:0/1926417763 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf80054060 con 0x560600fb32f0 2024-05-27T07:47:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.714+0000 7fcf9b417740 1 -- 172.21.15.63:0/1926417763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf78016070 con 0x560600fb32f0 2024-05-27T07:47:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.714+0000 7fcf73400640 1 -- 172.21.15.63:0/1926417763 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fcf80004740 con 0x560600fb32f0 2024-05-27T07:47:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.714+0000 7fcf73400640 1 -- 172.21.15.63:0/1926417763 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcf8005b050 con 0x560600fb32f0 2024-05-27T07:47:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.714+0000 7fcf9b417740 10 client.18138 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.714+0000 7fcf9b417740 1 -- 172.21.15.63:0/1926417763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5606010c0620 con 0x560600fb32f0 2024-05-27T07:47:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.714+0000 7fcf73400640 1 -- 172.21.15.63:0/1926417763 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf78016070 con 0x560600fb32f0 2024-05-27T07:47:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.714+0000 7fcf73400640 1 -- 172.21.15.63:0/1926417763 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcf800384e0 con 0x560600fb32f0 2024-05-27T07:47:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.715+0000 7fcf9b417740 10 client.18138 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.715+0000 7fcf9b417740 10 client.18138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.715+0000 7fcf9b417740 20 client.18138 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.715+0000 7fcf9b417740 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560600f9ad80 0x56060139ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.715+0000 7fcf9b417740 4 client.18138 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.715+0000 7fcf9b417740 1 -- 172.21.15.63:0/1926417763 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606012f1ae0 con 0x560600f9ad80 2024-05-27T07:47:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.716+0000 7fcf91000640 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560600f9ad80 0x56060139ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.717+0000 7fcf91000640 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560600f9ad80 0x56060139ac50 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ffb5317da2cb5fd server_cookie=398b2d39da607f17 in_seq=0 out_seq=0 2024-05-27T07:47:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.717+0000 7fcf73400640 10 client.18138 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.719+0000 7fcf73400640 1 -- 172.21.15.63:0/1926417763 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5606012f1ae0 con 0x560600f9ad80 2024-05-27T07:47:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.719+0000 7fcf73400640 10 client.18138 handle_command_reply: tid=0 2024-05-27T07:47:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.719+0000 7fcf9b417740 1 client.18138 shutdown 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 1 -- 172.21.15.63:0/1926417763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 msgr2=0x560601388a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x560601388a30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fcf80035000 tx=0x7fcf800095c0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 1 -- 172.21.15.63:0/1926417763 shutdown_connections 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560600f9ad80 0x56060139ac50 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 0x560601394cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x560601388f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 1 --2- 172.21.15.63:0/1926417763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x560601388a30 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 1 -- 172.21.15.63:0/1926417763 >> 172.21.15.63:0/1926417763 conn(0x560600fd0300 msgr2=0x56060137e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf73400640 0 client.18138 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 1 -- 172.21.15.63:0/1926417763 shutdown_connections 2024-05-27T07:47:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 1 -- 172.21.15.63:0/1926417763 wait complete. 2024-05-27T07:47:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.720+0000 7fcf9b417740 20 client.18138 trim_cache size 0 max 16384 2024-05-27T07:47:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.739+0000 7fcf9b417740 1 Processor -- start 2024-05-27T07:47:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.739+0000 7fcf9b417740 1 -- start start 2024-05-27T07:47:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.739+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x56060138b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.739+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x560601394a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.739+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 0x560601394f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.739+0000 7fcf9b417740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c1b5d30 con 0x560600fb32f0 2024-05-27T07:47:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.739+0000 7fcf9b417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c1b3600 con 0x560600fbb4d0 2024-05-27T07:47:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.739+0000 7fcf9b417740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c1b1470 con 0x560600fdc1b0 2024-05-27T07:47:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf91a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x56060138b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf91a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x56060138b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37382/0 (socket says 172.21.15.63:37382) 2024-05-27T07:47:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf91a00640 1 -- 172.21.15.63:0/2586886526 learned_addr learned my addr 172.21.15.63:0/2586886526 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf91000640 1 --2- 172.21.15.63:0/2586886526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x560601394a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf92400640 1 --2- 172.21.15.63:0/2586886526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 0x560601394f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf91a00640 1 -- 172.21.15.63:0/2586886526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 msgr2=0x560601394f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf91a00640 1 --2- 172.21.15.63:0/2586886526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 0x560601394f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf91a00640 1 -- 172.21.15.63:0/2586886526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 msgr2=0x560601394a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf91a00640 1 --2- 172.21.15.63:0/2586886526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x560601394a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf91a00640 1 -- 172.21.15.63:0/2586886526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560600f9afb0 con 0x560600fb32f0 2024-05-27T07:47:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.740+0000 7fcf92400640 1 --2- 172.21.15.63:0/2586886526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 0x560601394f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.741+0000 7fcf91a00640 1 --2- 172.21.15.63:0/2586886526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x56060138b160 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fcf80004010 tx=0x7fcf8002a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94b25cb24132ca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.741+0000 7fcf7e000640 1 -- 172.21.15.63:0/2586886526 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf80064070 con 0x560600fb32f0 2024-05-27T07:47:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.741+0000 7fcf7e000640 1 -- 172.21.15.63:0/2586886526 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf80038650 con 0x560600fb32f0 2024-05-27T07:47:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.741+0000 7fcf7e000640 1 -- 172.21.15.63:0/2586886526 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf80054040 con 0x560600fb32f0 2024-05-27T07:47:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.742+0000 7fcf9b417740 1 -- 172.21.15.63:0/2586886526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 msgr2=0x56060138b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.742+0000 7fcf9b417740 1 --2- 172.21.15.63:0/2586886526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x56060138b160 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fcf80004010 tx=0x7fcf8002a9e0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.742+0000 7fcf9b417740 1 -- 172.21.15.63:0/2586886526 shutdown_connections 2024-05-27T07:47:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.742+0000 7fcf9b417740 1 --2- 172.21.15.63:0/2586886526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fdc1b0 0x560601394f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.742+0000 7fcf9b417740 1 --2- 172.21.15.63:0/2586886526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x560601394a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.742+0000 7fcf9b417740 1 --2- 172.21.15.63:0/2586886526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fb32f0 0x56060138b160 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.742+0000 7fcf9b417740 1 -- 172.21.15.63:0/2586886526 >> 172.21.15.63:0/2586886526 conn(0x560600fd0300 msgr2=0x560601388cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.742+0000 7fcf9b417740 1 -- 172.21.15.63:0/2586886526 shutdown_connections 2024-05-27T07:47:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.742+0000 7fcf9b417740 1 -- 172.21.15.63:0/2586886526 wait complete. 2024-05-27T07:47:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.743+0000 7fcf9b417740 1 -- start start 2024-05-27T07:47:45.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.743+0000 7fcf9b417740 1 Processor -- start 2024-05-27T07:47:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fb32f0 0x56060138e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x56060138e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf9b417740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x5606013fb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf9b417740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c1b1ac0 con 0x560600fdc1b0 2024-05-27T07:47:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf9b417740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c1b2f30 con 0x560600fbb4d0 2024-05-27T07:47:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf9b417740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c1b2190 con 0x560600fb32f0 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf9b417740 10 client.0 _ll_register_callbacks cb 0x7fcf80009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf92400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x5606013fb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf92400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x5606013fb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37398/0 (socket says 172.21.15.63:37398) 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf92400640 1 -- 172.21.15.63:0/819792630 learned_addr learned my addr 172.21.15.63:0/819792630 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf91a00640 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fb32f0 0x56060138e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.744+0000 7fcf91000640 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x56060138e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf92400640 1 -- 172.21.15.63:0/819792630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fb32f0 msgr2=0x56060138e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf92400640 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fb32f0 0x56060138e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf92400640 1 -- 172.21.15.63:0/819792630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 msgr2=0x56060138e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf92400640 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x56060138e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf92400640 1 -- 172.21.15.63:0/819792630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606013fb7f0 con 0x560600fdc1b0 2024-05-27T07:47:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf91a00640 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fb32f0 0x56060138e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf92400640 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x5606013fb2b0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fcf840a8f50 tx=0x7fcf840dfb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96740d70e070e72 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf73400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf73400640 1 -- 172.21.15.63:0/819792630 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84020050 con 0x560600fdc1b0 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf73400640 1 -- 172.21.15.63:0/819792630 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fcf840c3670 con 0x560600fdc1b0 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.745+0000 7fcf9b417740 1 -- 172.21.15.63:0/819792630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf80038650 con 0x560600fdc1b0 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf73400640 1 -- 172.21.15.63:0/819792630 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcf840c3870 con 0x560600fdc1b0 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf9b417740 10 client.18144 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf9b417740 1 -- 172.21.15.63:0/819792630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560601402a40 con 0x560600fdc1b0 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf73400640 1 -- 172.21.15.63:0/819792630 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf80038650 con 0x560600fdc1b0 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf73400640 1 -- 172.21.15.63:0/819792630 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcf840c3240 con 0x560600fdc1b0 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf9b417740 10 client.18144 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf9b417740 10 client.18144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf9b417740 20 client.18144 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf9b417740 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560600f701c0 0x560601405250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf9b417740 4 client.18144 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:47:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.746+0000 7fcf9b417740 1 -- 172.21.15.63:0/819792630 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606012eb2c0 con 0x560600f701c0 2024-05-27T07:47:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.747+0000 7fcf91a00640 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560600f701c0 0x560601405250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.748+0000 7fcf91a00640 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560600f701c0 0x560601405250 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183eff10bebbc870 server_cookie=e8f56ebceea07aba in_seq=0 out_seq=0 2024-05-27T07:47:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.748+0000 7fcf73400640 10 client.18144 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf73400640 1 -- 172.21.15.63:0/819792630 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5606012eb2c0 con 0x560600f701c0 2024-05-27T07:47:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf73400640 10 client.18144 handle_command_reply: tid=0 2024-05-27T07:47:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf9b417740 1 client.18144 shutdown 2024-05-27T07:47:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf9b417740 1 -- 172.21.15.63:0/819792630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 msgr2=0x5606013fb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf9b417740 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x5606013fb2b0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fcf840a8f50 tx=0x7fcf840dfb40 comp rx=0 tx=0).stop 2024-05-27T07:47:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf9b417740 1 -- 172.21.15.63:0/819792630 shutdown_connections 2024-05-27T07:47:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf9b417740 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560600f701c0 0x560601405250 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf9b417740 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560600fdc1b0 0x5606013fb2b0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf9b417740 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560600fbb4d0 0x56060138e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf9b417740 1 --2- 172.21.15.63:0/819792630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560600fb32f0 0x56060138e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf9b417740 1 -- 172.21.15.63:0/819792630 >> 172.21.15.63:0/819792630 conn(0x560600fd0300 msgr2=0x560601388a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.777+0000 7fcf73400640 0 client.18144 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.778+0000 7fcf9b417740 1 -- 172.21.15.63:0/819792630 shutdown_connections 2024-05-27T07:47:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.778+0000 7fcf9b417740 1 -- 172.21.15.63:0/819792630 wait complete. 2024-05-27T07:47:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.778+0000 7fcf9b417740 20 client.18144 trim_cache size 0 max 16384 2024-05-27T07:47:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:47:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:47:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:47:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 -- 172.21.15.63:0/1479918714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcf740b29d0 msgr2=0x7fcf740b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcf740b29d0 0x7fcf740b4ee0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c19dae0 tx=0x7fcf80005d80 comp rx=0 tx=0).stop 2024-05-27T07:47:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 -- 172.21.15.63:0/1479918714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf8c10e200 msgr2=0x7fcf8c19c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf8c10e200 0x7fcf8c19c7b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400d750 tx=0x7fcf8400dc10 comp rx=0 tx=0).stop 2024-05-27T07:47:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 -- 172.21.15.63:0/1479918714 shutdown_connections 2024-05-27T07:47:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcf740b29d0 0x7fcf740b4ee0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf8c10e200 0x7fcf8c19c7b0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf8c10b850 0x7fcf8c1a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 --2- 172.21.15.63:0/1479918714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf8c104df0 0x7fcf8c1a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 -- 172.21.15.63:0/1479918714 >> 172.21.15.63:0/1479918714 conn(0x7fcf8c0782d0 msgr2=0x7fcf8c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 -- 172.21.15.63:0/1479918714 shutdown_connections 2024-05-27T07:47:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:45.779+0000 7fcf94400640 1 -- 172.21.15.63:0/1479918714 wait complete. 2024-05-27T07:47:46.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.672+0000 7f23c1000700 1 -- 172.21.15.63:0/412689906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23bc105de0 msgr2=0x7f23bc10e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:46.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.672+0000 7f23c1000700 1 --2- 172.21.15.63:0/412689906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23bc105de0 0x7f23bc10e330 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009990 tx=0x7f23a8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.673+0000 7f23c1000700 1 -- 172.21.15.63:0/412689906 shutdown_connections 2024-05-27T07:47:46.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.673+0000 7f23c1000700 1 --2- 172.21.15.63:0/412689906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23bc10e900 0x7f23bc110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.673+0000 7f23c1000700 1 --2- 172.21.15.63:0/412689906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23bc105490 0x7f23bc1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.673+0000 7f23c1000700 1 --2- 172.21.15.63:0/412689906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23bc105de0 0x7f23bc10e330 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.673+0000 7f23c1000700 1 -- 172.21.15.63:0/412689906 >> 172.21.15.63:0/412689906 conn(0x7f23bc0fedb0 msgr2=0x7f23bc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:46.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.673+0000 7f23c1000700 1 -- 172.21.15.63:0/412689906 shutdown_connections 2024-05-27T07:47:46.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.673+0000 7f23c1000700 1 -- 172.21.15.63:0/412689906 wait complete. 2024-05-27T07:47:46.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.674+0000 7f23c1000700 1 Processor -- start 2024-05-27T07:47:46.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.674+0000 7f23c1000700 1 -- start start 2024-05-27T07:47:46.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.675+0000 7f23c1000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23bc105490 0x7f23bc19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.675+0000 7f23c1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23bc105de0 0x7f23bc19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.675+0000 7f23c1000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23bc10e900 0x7f23bc1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.675+0000 7f23c1000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc1142f0 con 0x7f23bc10e900 2024-05-27T07:47:46.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.675+0000 7f23c1000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc114170 con 0x7f23bc105de0 2024-05-27T07:47:46.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.675+0000 7f23c1000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc114470 con 0x7f23bc105490 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.675+0000 7f23ba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23bc10e900 0x7f23bc1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.675+0000 7f23ba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23bc10e900 0x7f23bc1a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37412/0 (socket says 172.21.15.63:37412) 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.675+0000 7f23ba800700 1 -- 172.21.15.63:0/3461188240 learned_addr learned my addr 172.21.15.63:0/3461188240 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.676+0000 7f23b9e00700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23bc105490 0x7f23bc19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.676+0000 7f23b9400700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23bc105de0 0x7f23bc19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.676+0000 7f23ba800700 1 -- 172.21.15.63:0/3461188240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23bc105490 msgr2=0x7f23bc19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.676+0000 7f23ba800700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23bc105490 0x7f23bc19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.676+0000 7f23ba800700 1 -- 172.21.15.63:0/3461188240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23bc105de0 msgr2=0x7f23bc19d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.676+0000 7f23ba800700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23bc105de0 0x7f23bc19d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.676+0000 7f23ba800700 1 -- 172.21.15.63:0/3461188240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc1a1e30 con 0x7f23bc10e900 2024-05-27T07:47:46.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.676+0000 7f23b9400700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23bc105de0 0x7f23bc19d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.676+0000 7f23ba800700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23bc10e900 0x7f23bc1a1790 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f23b4007ee0 tx=0x7f23b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.677+0000 7f23b2a00700 1 -- 172.21.15.63:0/3461188240 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b400dbd0 con 0x7f23bc10e900 2024-05-27T07:47:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.677+0000 7f23b2a00700 1 -- 172.21.15.63:0/3461188240 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23b400dd70 con 0x7f23bc10e900 2024-05-27T07:47:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.677+0000 7f23c1000700 1 -- 172.21.15.63:0/3461188240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23bc1a2120 con 0x7f23bc10e900 2024-05-27T07:47:46.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.677+0000 7f23b2a00700 1 -- 172.21.15.63:0/3461188240 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40058d0 con 0x7f23bc10e900 2024-05-27T07:47:46.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.677+0000 7f23c1000700 1 -- 172.21.15.63:0/3461188240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23bc1a9a40 con 0x7f23bc10e900 2024-05-27T07:47:46.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.681+0000 7f23b2a00700 1 -- 172.21.15.63:0/3461188240 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f23b401a020 con 0x7f23bc10e900 2024-05-27T07:47:46.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.683+0000 7f23b2a00700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23a00b29f0 0x7f23a00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.683+0000 7f23b2a00700 1 -- 172.21.15.63:0/3461188240 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f23b40e2110 con 0x7f23bc10e900 2024-05-27T07:47:46.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.683+0000 7f23b9e00700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23a00b29f0 0x7f23a00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.684+0000 7f23cfb5eb80 1 Processor -- start 2024-05-27T07:47:46.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.684+0000 7f23cfb5eb80 1 -- start start 2024-05-27T07:47:46.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.684+0000 7f23b9e00700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23a00b29f0 0x7f23a00b4f00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f23ac005950 tx=0x7f23ac00b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:46.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 0x55bfe2ea0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ea05c0 0x55bfe2ea09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bfe2dc5e40 con 0x55bfe2d18e40 2024-05-27T07:47:46.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bfe2db59e0 con 0x55bfe2ea05c0 2024-05-27T07:47:46.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bfe2db9a20 con 0x55bfe2ea6540 2024-05-27T07:47:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23b9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23b9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37420/0 (socket says 172.21.15.63:37420) 2024-05-27T07:47:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23b9e00700 1 -- 172.21.15.63:0/1952888121 learned_addr learned my addr 172.21.15.63:0/1952888121 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23b9400700 1 --2- 172.21.15.63:0/1952888121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 0x55bfe2ea0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.685+0000 7f23ba800700 1 --2- 172.21.15.63:0/1952888121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ea05c0 0x55bfe2ea09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.686+0000 7f23b9e00700 1 -- 172.21.15.63:0/1952888121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 msgr2=0x55bfe2ea0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.686+0000 7f23b9e00700 1 --2- 172.21.15.63:0/1952888121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 0x55bfe2ea0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.686+0000 7f23b9e00700 1 -- 172.21.15.63:0/1952888121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ea05c0 msgr2=0x55bfe2ea09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.686+0000 7f23b9e00700 1 --2- 172.21.15.63:0/1952888121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ea05c0 0x55bfe2ea09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.686+0000 7f23b9e00700 1 -- 172.21.15.63:0/1952888121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe2d7e420 con 0x55bfe2d18e40 2024-05-27T07:47:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.686+0000 7f23b9e00700 1 --2- 172.21.15.63:0/1952888121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea6000 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f23ac010fa0 tx=0x7f23ac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f78b85bdf3cc087 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.687+0000 7f23a7400700 1 -- 172.21.15.63:0/1952888121 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac00be60 con 0x55bfe2d18e40 2024-05-27T07:47:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.687+0000 7f23a7400700 1 -- 172.21.15.63:0/1952888121 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23ac011db0 con 0x55bfe2d18e40 2024-05-27T07:47:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.687+0000 7f23a7400700 1 -- 172.21.15.63:0/1952888121 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac01c6b0 con 0x55bfe2d18e40 2024-05-27T07:47:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.688+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1952888121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2d18e40 msgr2=0x55bfe2ea6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.688+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1952888121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea6000 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f23ac010fa0 tx=0x7f23ac013000 comp rx=0 tx=0).stop 2024-05-27T07:47:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.688+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1952888121 shutdown_connections 2024-05-27T07:47:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.688+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1952888121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ea05c0 0x55bfe2ea09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.688+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1952888121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea6000 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.688+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1952888121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 0x55bfe2ea0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.688+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1952888121 >> 172.21.15.63:0/1952888121 conn(0x55bfe2d78fc0 msgr2=0x55bfe2e928d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.688+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1952888121 shutdown_connections 2024-05-27T07:47:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.688+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1952888121 wait complete. 2024-05-27T07:47:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.689+0000 7f23cfb5eb80 1 -- start start 2024-05-27T07:47:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.689+0000 7f23cfb5eb80 1 Processor -- start 2024-05-27T07:47:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.690+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.690+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2eaa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.690+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ea6540 0x55bfe2eaa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.690+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bfe2cbbfa0 con 0x55bfe2ea05c0 2024-05-27T07:47:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.690+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bfe2d56760 con 0x55bfe2ea6540 2024-05-27T07:47:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.690+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bfe2ead010 con 0x55bfe2d18e40 2024-05-27T07:47:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.690+0000 7f23cfb5eb80 10 client.0 _ll_register_callbacks cb 0x55bfe2d5fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.690+0000 7f23b9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2eaa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.690+0000 7f23b9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2eaa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37428/0 (socket says 172.21.15.63:37428) 2024-05-27T07:47:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.690+0000 7f23b9400700 1 -- 172.21.15.63:0/2889280566 learned_addr learned my addr 172.21.15.63:0/2889280566 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:46.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.691+0000 7f23b9e00700 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.691+0000 7f23ba800700 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ea6540 0x55bfe2eaa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.691+0000 7f23b9400700 1 -- 172.21.15.63:0/2889280566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2d18e40 msgr2=0x55bfe2ea9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:46.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.691+0000 7f23b9400700 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.691+0000 7f23b9400700 1 -- 172.21.15.63:0/2889280566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ea6540 msgr2=0x55bfe2eaa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:46.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.691+0000 7f23b9400700 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ea6540 0x55bfe2eaa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.691+0000 7f23b9400700 1 -- 172.21.15.63:0/2889280566 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe2d058d0 con 0x55bfe2ea05c0 2024-05-27T07:47:46.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.692+0000 7f23b9400700 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2eaa170 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f23a8003760 tx=0x7f23a8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896c12f606f3b59d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.692+0000 7f239be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.692+0000 7f239be00700 1 -- 172.21.15.63:0/2889280566 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8028060 con 0x55bfe2ea05c0 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.692+0000 7f239be00700 1 -- 172.21.15.63:0/2889280566 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f23a8003bd0 con 0x55bfe2ea05c0 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.692+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/2889280566 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23ac011db0 con 0x55bfe2ea05c0 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.692+0000 7f239be00700 1 -- 172.21.15.63:0/2889280566 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f23a802f050 con 0x55bfe2ea05c0 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.692+0000 7f23cfb5eb80 10 client.18156 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.692+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/2889280566 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bfe2d86060 con 0x55bfe2ea05c0 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.693+0000 7f239be00700 1 -- 172.21.15.63:0/2889280566 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23ac011db0 con 0x55bfe2ea05c0 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.693+0000 7f239be00700 1 -- 172.21.15.63:0/2889280566 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f23a8023660 con 0x55bfe2ea05c0 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.693+0000 7f23cfb5eb80 10 client.18156 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:46.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.693+0000 7f23cfb5eb80 10 client.18156 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:47:46.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.693+0000 7f23cfb5eb80 20 client.18156 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:46.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.693+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55bfe2eb9f20 0x55bfe2ebc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.693+0000 7f23cfb5eb80 4 client.18156 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:46.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.693+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/2889280566 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfe2dc7d90 con 0x55bfe2eb9f20 2024-05-27T07:47:46.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.694+0000 7f23b9e00700 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55bfe2eb9f20 0x55bfe2ebc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.695+0000 7f23b9e00700 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55bfe2eb9f20 0x55bfe2ebc370 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5e18936fc6e273 server_cookie=6d52a462e403e600 in_seq=0 out_seq=0 2024-05-27T07:47:46.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.695+0000 7f239be00700 10 client.18156 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:47:46.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f239be00700 1 -- 172.21.15.63:0/2889280566 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bfe2dc7d90 con 0x55bfe2eb9f20 2024-05-27T07:47:46.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f239be00700 10 client.18156 handle_command_reply: tid=0 2024-05-27T07:47:46.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f23cfb5eb80 1 client.18156 shutdown 2024-05-27T07:47:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/2889280566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 msgr2=0x55bfe2eaa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2eaa170 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f23a8003760 tx=0x7f23a8009f00 comp rx=0 tx=0).stop 2024-05-27T07:47:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/2889280566 shutdown_connections 2024-05-27T07:47:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ea6540 0x55bfe2eaa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55bfe2eb9f20 0x55bfe2ebc370 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/2889280566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2eaa170 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/2889280566 >> 172.21.15.63:0/2889280566 conn(0x55bfe2d78fc0 msgr2=0x55bfe2e92580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:46.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.697+0000 7f239be00700 0 client.18156 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:47:46.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.698+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/2889280566 shutdown_connections 2024-05-27T07:47:46.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.698+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/2889280566 wait complete. 2024-05-27T07:47:46.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.698+0000 7f23cfb5eb80 20 client.18156 trim_cache size 0 max 16384 2024-05-27T07:47:46.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.722+0000 7f23cfb5eb80 1 Processor -- start 2024-05-27T07:47:46.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.722+0000 7f23cfb5eb80 1 -- start start 2024-05-27T07:47:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.722+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.722+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2ea8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.723+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 0x55bfe2eabf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.723+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f239c011610 con 0x55bfe2ea05c0 2024-05-27T07:47:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.723+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f239c00eea0 con 0x55bfe2d18e40 2024-05-27T07:47:46.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.723+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f239c00cd10 con 0x55bfe2ea6540 2024-05-27T07:47:46.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.723+0000 7f23b9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2ea8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.723+0000 7f23b9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.723+0000 7f23b9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2ea8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37442/0 (socket says 172.21.15.63:37442) 2024-05-27T07:47:46.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.723+0000 7f23b9400700 1 -- 172.21.15.63:0/1396289865 learned_addr learned my addr 172.21.15.63:0/1396289865 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.723+0000 7f23ba800700 1 --2- 172.21.15.63:0/1396289865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 0x55bfe2eabf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.724+0000 7f23b9400700 1 -- 172.21.15.63:0/1396289865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 msgr2=0x55bfe2eabf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.724+0000 7f23b9400700 1 --2- 172.21.15.63:0/1396289865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 0x55bfe2eabf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.724+0000 7f23b9400700 1 -- 172.21.15.63:0/1396289865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2d18e40 msgr2=0x55bfe2ea0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.724+0000 7f23b9400700 1 --2- 172.21.15.63:0/1396289865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.724+0000 7f23b9400700 1 -- 172.21.15.63:0/1396289865 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe2eac4d0 con 0x55bfe2ea05c0 2024-05-27T07:47:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.724+0000 7f23ba800700 1 --2- 172.21.15.63:0/1396289865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 0x55bfe2eabf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.724+0000 7f23b9400700 1 --2- 172.21.15.63:0/1396289865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2ea8f50 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f23a8023bc0 tx=0x7f23a8023610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5f9bdd4a678170 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:46.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.725+0000 7f23a6a00700 1 -- 172.21.15.63:0/1396289865 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8038070 con 0x55bfe2ea05c0 2024-05-27T07:47:46.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.725+0000 7f23a6a00700 1 -- 172.21.15.63:0/1396289865 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23a8003bd0 con 0x55bfe2ea05c0 2024-05-27T07:47:46.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.725+0000 7f23a6a00700 1 -- 172.21.15.63:0/1396289865 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8028040 con 0x55bfe2ea05c0 2024-05-27T07:47:46.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.726+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1396289865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 msgr2=0x55bfe2ea8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:46.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.726+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1396289865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2ea8f50 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f23a8023bc0 tx=0x7f23a8023610 comp rx=0 tx=0).stop 2024-05-27T07:47:46.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.726+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1396289865 shutdown_connections 2024-05-27T07:47:46.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.726+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1396289865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2ea6540 0x55bfe2eabf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.726+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1396289865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2d18e40 0x55bfe2ea0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.726+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1396289865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea05c0 0x55bfe2ea8f50 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.726+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1396289865 >> 172.21.15.63:0/1396289865 conn(0x55bfe2d78fc0 msgr2=0x55bfe2ea3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.726+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1396289865 shutdown_connections 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.726+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1396289865 wait complete. 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.727+0000 7f23cfb5eb80 1 -- start start 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.727+0000 7f23cfb5eb80 1 Processor -- start 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2d18e40 0x55bfe2eb2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea6540 0x55bfe2eb2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23cfb5eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ec0b00 0x55bfe2ec0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f239c00d360 con 0x55bfe2ea6540 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f239c00e7d0 con 0x55bfe2ec0b00 2024-05-27T07:47:46.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23cfb5eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f239c00da30 con 0x55bfe2d18e40 2024-05-27T07:47:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23cfb5eb80 10 client.0 _ll_register_callbacks cb 0x55bfe2ea10c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23b9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea6540 0x55bfe2eb2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23b9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea6540 0x55bfe2eb2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37450/0 (socket says 172.21.15.63:37450) 2024-05-27T07:47:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23b9400700 1 -- 172.21.15.63:0/1117843740 learned_addr learned my addr 172.21.15.63:0/1117843740 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23b9e00700 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2d18e40 0x55bfe2eb2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.728+0000 7f23ba800700 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ec0b00 0x55bfe2ec0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.729+0000 7f23b9400700 1 -- 172.21.15.63:0/1117843740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2d18e40 msgr2=0x55bfe2eb2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.729+0000 7f23b9400700 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2d18e40 0x55bfe2eb2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.729+0000 7f23b9400700 1 -- 172.21.15.63:0/1117843740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ec0b00 msgr2=0x55bfe2ec0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:46.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.729+0000 7f23b9400700 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ec0b00 0x55bfe2ec0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.729+0000 7f23b9400700 1 -- 172.21.15.63:0/1117843740 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe2ec1450 con 0x55bfe2ea6540 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.729+0000 7f23b9400700 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea6540 0x55bfe2eb2840 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f23a80098e0 tx=0x7f23a8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a6709705bc9410 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.729+0000 7f239be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.729+0000 7f239be00700 1 -- 172.21.15.63:0/1117843740 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8038070 con 0x55bfe2ea6540 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.730+0000 7f239be00700 1 -- 172.21.15.63:0/1117843740 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f23a800b980 con 0x55bfe2ea6540 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.730+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1117843740 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a8003bd0 con 0x55bfe2ea6540 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.730+0000 7f239be00700 1 -- 172.21.15.63:0/1117843740 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f23a8023460 con 0x55bfe2ea6540 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.730+0000 7f23cfb5eb80 10 client.18162 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.730+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1117843740 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bfe2f24480 con 0x55bfe2ea6540 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.730+0000 7f239be00700 1 -- 172.21.15.63:0/1117843740 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23a8003bd0 con 0x55bfe2ea6540 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.731+0000 7f239be00700 1 -- 172.21.15.63:0/1117843740 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f23a800fc30 con 0x55bfe2ea6540 2024-05-27T07:47:46.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.731+0000 7f23cfb5eb80 10 client.18162 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:46.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.731+0000 7f23cfb5eb80 10 client.18162 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:47:46.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.731+0000 7f23cfb5eb80 20 client.18162 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:46.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.731+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55bfe2f249c0 0x55bfe2f26de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:46.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.731+0000 7f23cfb5eb80 4 client.18162 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:47:46.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.731+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1117843740 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfe2dc1570 con 0x55bfe2f249c0 2024-05-27T07:47:46.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.732+0000 7f23b9e00700 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55bfe2f249c0 0x55bfe2f26de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:46.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.732+0000 7f23b9e00700 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55bfe2f249c0 0x55bfe2f26de0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48666cd5574dc56e server_cookie=710b218fdf661fd3 in_seq=0 out_seq=0 2024-05-27T07:47:46.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.732+0000 7f239be00700 10 client.18162 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:47:46.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f239be00700 1 -- 172.21.15.63:0/1117843740 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfe2dc1570 con 0x55bfe2f249c0 2024-05-27T07:47:46.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f239be00700 10 client.18162 handle_command_reply: tid=0 2024-05-27T07:47:46.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f23cfb5eb80 1 client.18162 shutdown 2024-05-27T07:47:46.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1117843740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea6540 msgr2=0x55bfe2eb2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:46.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea6540 0x55bfe2eb2840 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f23a80098e0 tx=0x7f23a8005e80 comp rx=0 tx=0).stop 2024-05-27T07:47:46.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1117843740 shutdown_connections 2024-05-27T07:47:46.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfe2ec0b00 0x55bfe2ec0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfe2d18e40 0x55bfe2eb2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55bfe2f249c0 0x55bfe2f26de0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f23cfb5eb80 1 --2- 172.21.15.63:0/1117843740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bfe2ea6540 0x55bfe2eb2840 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1117843740 >> 172.21.15.63:0/1117843740 conn(0x55bfe2d78fc0 msgr2=0x55bfe2ea8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:46.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.733+0000 7f239be00700 0 client.18162 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:47:46.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.734+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1117843740 shutdown_connections 2024-05-27T07:47:46.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.734+0000 7f23cfb5eb80 1 -- 172.21.15.63:0/1117843740 wait complete. 2024-05-27T07:47:46.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.734+0000 7f23cfb5eb80 20 client.18162 trim_cache size 0 max 16384 2024-05-27T07:47:46.738 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:47:46.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 -- 172.21.15.63:0/3461188240 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23a00b29f0 msgr2=0x7f23a00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:46.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23a00b29f0 0x7f23a00b4f00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f23ac005950 tx=0x7f23ac00b410 comp rx=0 tx=0).stop 2024-05-27T07:47:46.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 -- 172.21.15.63:0/3461188240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23bc10e900 msgr2=0x7f23bc1a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23bc10e900 0x7f23bc1a1790 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f23b4007ee0 tx=0x7f23b400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 -- 172.21.15.63:0/3461188240 shutdown_connections 2024-05-27T07:47:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23a00b29f0 0x7f23a00b4f00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23bc10e900 0x7f23bc1a1790 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23bc105490 0x7f23bc19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 --2- 172.21.15.63:0/3461188240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23bc105de0 0x7f23bc19d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 -- 172.21.15.63:0/3461188240 >> 172.21.15.63:0/3461188240 conn(0x7f23bc0fedb0 msgr2=0x7f23bc0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 -- 172.21.15.63:0/3461188240 shutdown_connections 2024-05-27T07:47:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:46.735+0000 7f23c1000700 1 -- 172.21.15.63:0/3461188240 wait complete. 2024-05-27T07:47:47.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:47 smithi086 ceph-mon[34687]: pgmap v445: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:47:47.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:47 smithi149 ceph-mon[35408]: pgmap v445: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:47:47.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:47 smithi063 ceph-mon[31613]: pgmap v445: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:47:47.727 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:47:48.256 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:49.453 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:49 smithi063 ceph-mon[31613]: pgmap v446: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.4 MiB/s wr, 18 op/s 2024-05-27T07:47:49.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:49 smithi086 ceph-mon[34687]: pgmap v446: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.4 MiB/s wr, 18 op/s 2024-05-27T07:47:49.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:49 smithi149 ceph-mon[35408]: pgmap v446: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.4 MiB/s wr, 18 op/s 2024-05-27T07:47:50.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.957+0000 7f162b600700 1 -- 172.21.15.63:0/300056150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1624075d20 msgr2=0x7f1624076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:50.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.957+0000 7f162b600700 1 --2- 172.21.15.63:0/300056150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1624075d20 0x7f1624076130 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f1618009990 tx=0x7f1618009ca0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.957+0000 7f162b600700 1 -- 172.21.15.63:0/300056150 shutdown_connections 2024-05-27T07:47:50.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.957+0000 7f162b600700 1 --2- 172.21.15.63:0/300056150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1624110b90 0x7f1624112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.957+0000 7f162b600700 1 --2- 172.21.15.63:0/300056150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1624075d20 0x7f1624076130 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.957+0000 7f162b600700 1 --2- 172.21.15.63:0/300056150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1624076670 0x7f1624110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.957+0000 7f162b600700 1 -- 172.21.15.63:0/300056150 >> 172.21.15.63:0/300056150 conn(0x7f1624100030 msgr2=0x7f1624102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:50.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.957+0000 7f162b600700 1 -- 172.21.15.63:0/300056150 shutdown_connections 2024-05-27T07:47:50.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.957+0000 7f162b600700 1 -- 172.21.15.63:0/300056150 wait complete. 2024-05-27T07:47:50.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.959+0000 7f162b600700 1 Processor -- start 2024-05-27T07:47:50.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.959+0000 7f162b600700 1 -- start start 2024-05-27T07:47:50.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.959+0000 7f162b600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1624075d20 0x7f162419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.959+0000 7f162b600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1624076670 0x7f162419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.960+0000 7f162b600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1624110b90 0x7f16241a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.960+0000 7f162b600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1624116530 con 0x7f1624110b90 2024-05-27T07:47:50.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.960+0000 7f162b600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16241163b0 con 0x7f1624076670 2024-05-27T07:47:50.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.960+0000 7f162b600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16241166b0 con 0x7f1624075d20 2024-05-27T07:47:50.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.960+0000 7f1629600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1624110b90 0x7f16241a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.960+0000 7f1623e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1624076670 0x7f162419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.960+0000 7f1628c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1624075d20 0x7f162419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.960+0000 7f1629600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1624110b90 0x7f16241a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49170/0 (socket says 172.21.15.63:49170) 2024-05-27T07:47:50.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.960+0000 7f1629600700 1 -- 172.21.15.63:0/829230576 learned_addr learned my addr 172.21.15.63:0/829230576 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:50.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.961+0000 7f1629600700 1 -- 172.21.15.63:0/829230576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1624075d20 msgr2=0x7f162419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:50.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.961+0000 7f1629600700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1624075d20 0x7f162419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.961+0000 7f1629600700 1 -- 172.21.15.63:0/829230576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1624076670 msgr2=0x7f162419d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:50.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.961+0000 7f1629600700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1624076670 0x7f162419d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.961+0000 7f1629600700 1 -- 172.21.15.63:0/829230576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16241a1e50 con 0x7f1624110b90 2024-05-27T07:47:50.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.961+0000 7f1629600700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1624110b90 0x7f16241a17b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f161c0094f0 tx=0x7f161c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:50.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.961+0000 7f1621600700 1 -- 172.21.15.63:0/829230576 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c004db0 con 0x7f1624110b90 2024-05-27T07:47:50.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.962+0000 7f1621600700 1 -- 172.21.15.63:0/829230576 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f161c009e20 con 0x7f1624110b90 2024-05-27T07:47:50.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.962+0000 7f162b600700 1 -- 172.21.15.63:0/829230576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16241a2140 con 0x7f1624110b90 2024-05-27T07:47:50.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.962+0000 7f1621600700 1 -- 172.21.15.63:0/829230576 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c015400 con 0x7f1624110b90 2024-05-27T07:47:50.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.962+0000 7f162b600700 1 -- 172.21.15.63:0/829230576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16241a9a60 con 0x7f1624110b90 2024-05-27T07:47:50.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.966+0000 7f1621600700 1 -- 172.21.15.63:0/829230576 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f161c0155a0 con 0x7f1624110b90 2024-05-27T07:47:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.968+0000 7f1621600700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f160c0b29f0 0x7f160c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.968+0000 7f1621600700 1 -- 172.21.15.63:0/829230576 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f161c0e2400 con 0x7f1624110b90 2024-05-27T07:47:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.968+0000 7f163a23fb80 1 Processor -- start 2024-05-27T07:47:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.968+0000 7f1628c00700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f160c0b29f0 0x7f160c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.968+0000 7f163a23fb80 1 -- start start 2024-05-27T07:47:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.969+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8bd8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.969+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8bd8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.969+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8be5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.969+0000 7f163a23fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d2c8b06e40 con 0x55d2c8bd8e70 2024-05-27T07:47:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.969+0000 7f1628c00700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f160c0b29f0 0x7f160c0b4f00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f161800b5c0 tx=0x7f1618005960 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.969+0000 7f163a23fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2c8af69e0 con 0x55d2c8b459f0 2024-05-27T07:47:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.969+0000 7f163a23fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d2c8afaa20 con 0x55d2c8a59e40 2024-05-27T07:47:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1629600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8be5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1629600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8be5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49174/0 (socket says 172.21.15.63:49174) 2024-05-27T07:47:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1629600700 1 -- 172.21.15.63:0/3896284076 learned_addr learned my addr 172.21.15.63:0/3896284076 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1628c00700 1 --2- 172.21.15.63:0/3896284076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8bd8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1623e00700 1 --2- 172.21.15.63:0/3896284076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8bd8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1629600700 1 -- 172.21.15.63:0/3896284076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 msgr2=0x55d2c8bd8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1629600700 1 --2- 172.21.15.63:0/3896284076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8bd8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1629600700 1 -- 172.21.15.63:0/3896284076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 msgr2=0x55d2c8bd8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1629600700 1 --2- 172.21.15.63:0/3896284076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8bd8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1629600700 1 -- 172.21.15.63:0/3896284076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c8abf420 con 0x55d2c8bd8e70 2024-05-27T07:47:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.970+0000 7f1623e00700 1 --2- 172.21.15.63:0/3896284076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8bd8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.971+0000 7f1629600700 1 --2- 172.21.15.63:0/3896284076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8be5110 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f161c004b80 tx=0x7f161c0a8b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb68a411384e436 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.971+0000 7f1616000700 1 -- 172.21.15.63:0/3896284076 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0ad670 con 0x55d2c8bd8e70 2024-05-27T07:47:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.971+0000 7f1616000700 1 -- 172.21.15.63:0/3896284076 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f161c0b2120 con 0x55d2c8bd8e70 2024-05-27T07:47:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.971+0000 7f1616000700 1 -- 172.21.15.63:0/3896284076 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0b1c90 con 0x55d2c8bd8e70 2024-05-27T07:47:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.972+0000 7f163a23fb80 1 -- 172.21.15.63:0/3896284076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 msgr2=0x55d2c8be5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.972+0000 7f163a23fb80 1 --2- 172.21.15.63:0/3896284076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8be5110 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f161c004b80 tx=0x7f161c0a8b60 comp rx=0 tx=0).stop 2024-05-27T07:47:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.972+0000 7f163a23fb80 1 -- 172.21.15.63:0/3896284076 shutdown_connections 2024-05-27T07:47:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.972+0000 7f163a23fb80 1 --2- 172.21.15.63:0/3896284076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8be5110 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.972+0000 7f163a23fb80 1 --2- 172.21.15.63:0/3896284076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8bd8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.972+0000 7f163a23fb80 1 --2- 172.21.15.63:0/3896284076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8bd8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.972+0000 7f163a23fb80 1 -- 172.21.15.63:0/3896284076 >> 172.21.15.63:0/3896284076 conn(0x55d2c8af3390 msgr2=0x55d2c8bd44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.972+0000 7f163a23fb80 1 -- 172.21.15.63:0/3896284076 shutdown_connections 2024-05-27T07:47:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.972+0000 7f163a23fb80 1 -- 172.21.15.63:0/3896284076 wait complete. 2024-05-27T07:47:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.973+0000 7f163a23fb80 1 -- start start 2024-05-27T07:47:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.973+0000 7f163a23fb80 1 Processor -- start 2024-05-27T07:47:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.974+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8be2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.974+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8b459f0 0x55d2c8be2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.974+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8bd8e70 0x55d2c8be30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.974+0000 7f163a23fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d2c89fcfa0 con 0x55d2c8b459f0 2024-05-27T07:47:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.974+0000 7f163a23fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2c8a97760 con 0x55d2c8bd8e70 2024-05-27T07:47:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.974+0000 7f163a23fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d2c8be5b80 con 0x55d2c8a59e40 2024-05-27T07:47:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.974+0000 7f163a23fb80 10 client.0 _ll_register_callbacks cb 0x55d2c8aa0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.974+0000 7f1623e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8b459f0 0x55d2c8be2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.975+0000 7f1623e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8b459f0 0x55d2c8be2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49188/0 (socket says 172.21.15.63:49188) 2024-05-27T07:47:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.975+0000 7f1623e00700 1 -- 172.21.15.63:0/755995470 learned_addr learned my addr 172.21.15.63:0/755995470 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.975+0000 7f1628c00700 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8be2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.975+0000 7f1629600700 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8bd8e70 0x55d2c8be30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.975+0000 7f1623e00700 1 -- 172.21.15.63:0/755995470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 msgr2=0x55d2c8be2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:50.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.975+0000 7f1623e00700 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8be2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.975+0000 7f1623e00700 1 -- 172.21.15.63:0/755995470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8bd8e70 msgr2=0x55d2c8be30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.975+0000 7f1623e00700 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8bd8e70 0x55d2c8be30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.975+0000 7f1623e00700 1 -- 172.21.15.63:0/755995470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c8a468d0 con 0x55d2c8b459f0 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.976+0000 7f1623e00700 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8b459f0 0x55d2c8be2bb0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f161000bb40 tx=0x7f161000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c675ab1a14951f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.976+0000 7f160aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.976+0000 7f160aa00700 1 -- 172.21.15.63:0/755995470 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610003d80 con 0x55d2c8b459f0 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.976+0000 7f160aa00700 1 -- 172.21.15.63:0/755995470 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f1610004830 con 0x55d2c8b459f0 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.976+0000 7f163a23fb80 1 -- 172.21.15.63:0/755995470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f161c0b2120 con 0x55d2c8b459f0 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.976+0000 7f160aa00700 1 -- 172.21.15.63:0/755995470 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1610004a30 con 0x55d2c8b459f0 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.976+0000 7f163a23fb80 10 client.18174 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.976+0000 7f163a23fb80 1 -- 172.21.15.63:0/755995470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d2c8ac7060 con 0x55d2c8b459f0 2024-05-27T07:47:50.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.977+0000 7f160aa00700 1 -- 172.21.15.63:0/755995470 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f161c0b2120 con 0x55d2c8b459f0 2024-05-27T07:47:50.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.977+0000 7f160aa00700 1 -- 172.21.15.63:0/755995470 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1610019030 con 0x55d2c8b459f0 2024-05-27T07:47:50.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.977+0000 7f163a23fb80 10 client.18174 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:50.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.977+0000 7f163a23fb80 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:50.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.977+0000 7f163a23fb80 20 client.18174 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:50.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.977+0000 7f163a23fb80 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2c8bf2b50 0x55d2c8bf4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:50.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.977+0000 7f163a23fb80 4 client.18174 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:50.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.977+0000 7f163a23fb80 1 -- 172.21.15.63:0/755995470 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2c8b08d90 con 0x55d2c8bf2b50 2024-05-27T07:47:50.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.978+0000 7f1628c00700 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2c8bf2b50 0x55d2c8bf4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:50.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.979+0000 7f1628c00700 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2c8bf2b50 0x55d2c8bf4f70 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad7ae9060ae2d85 server_cookie=2b856d695b8f4cf0 in_seq=0 out_seq=0 2024-05-27T07:47:50.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.979+0000 7f160aa00700 10 client.18174 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:50.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f160aa00700 1 -- 172.21.15.63:0/755995470 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d2c8b08d90 con 0x55d2c8bf2b50 2024-05-27T07:47:50.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f160aa00700 10 client.18174 handle_command_reply: tid=0 2024-05-27T07:47:50.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f163a23fb80 1 client.18174 shutdown 2024-05-27T07:47:50.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f163a23fb80 1 -- 172.21.15.63:0/755995470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8b459f0 msgr2=0x55d2c8be2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:50.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f163a23fb80 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8b459f0 0x55d2c8be2bb0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f161000bb40 tx=0x7f161000be50 comp rx=0 tx=0).stop 2024-05-27T07:47:50.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f163a23fb80 1 -- 172.21.15.63:0/755995470 shutdown_connections 2024-05-27T07:47:50.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f163a23fb80 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2c8bf2b50 0x55d2c8bf4f70 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f163a23fb80 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8bd8e70 0x55d2c8be30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f163a23fb80 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8be2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f163a23fb80 1 --2- 172.21.15.63:0/755995470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8b459f0 0x55d2c8be2bb0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:50.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f160aa00700 0 client.18174 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:50.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.981+0000 7f163a23fb80 1 -- 172.21.15.63:0/755995470 >> 172.21.15.63:0/755995470 conn(0x55d2c8af3390 msgr2=0x55d2c8bd8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:50.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.982+0000 7f163a23fb80 1 -- 172.21.15.63:0/755995470 shutdown_connections 2024-05-27T07:47:50.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.982+0000 7f163a23fb80 1 -- 172.21.15.63:0/755995470 wait complete. 2024-05-27T07:47:50.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:50.982+0000 7f163a23fb80 20 client.18174 trim_cache size 0 max 16384 2024-05-27T07:47:51.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.003+0000 7f163a23fb80 1 Processor -- start 2024-05-27T07:47:51.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.003+0000 7f163a23fb80 1 -- start start 2024-05-27T07:47:51.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.004+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8a59e40 0x55d2c8bdde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.004+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8be15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.004+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8bd8e70 0x55d2c8bd4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.004+0000 7f163a23fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16241b5090 con 0x55d2c8a59e40 2024-05-27T07:47:51.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.004+0000 7f163a23fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16241b2920 con 0x55d2c8b459f0 2024-05-27T07:47:51.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.004+0000 7f163a23fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16241b0790 con 0x55d2c8bd8e70 2024-05-27T07:47:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.004+0000 7f1628c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8a59e40 0x55d2c8bdde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.004+0000 7f1628c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8a59e40 0x55d2c8bdde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49194/0 (socket says 172.21.15.63:49194) 2024-05-27T07:47:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.004+0000 7f1628c00700 1 -- 172.21.15.63:0/2972500198 learned_addr learned my addr 172.21.15.63:0/2972500198 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.005+0000 7f1623e00700 1 --2- 172.21.15.63:0/2972500198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8be15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.005+0000 7f1629600700 1 --2- 172.21.15.63:0/2972500198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8bd8e70 0x55d2c8bd4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.005+0000 7f1628c00700 1 -- 172.21.15.63:0/2972500198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8bd8e70 msgr2=0x55d2c8bd4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.005+0000 7f1628c00700 1 --2- 172.21.15.63:0/2972500198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8bd8e70 0x55d2c8bd4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.005+0000 7f1628c00700 1 -- 172.21.15.63:0/2972500198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 msgr2=0x55d2c8be15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.005+0000 7f1628c00700 1 --2- 172.21.15.63:0/2972500198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8be15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.005+0000 7f1628c00700 1 -- 172.21.15.63:0/2972500198 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c8bd9a50 con 0x55d2c8a59e40 2024-05-27T07:47:51.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.005+0000 7f1628c00700 1 --2- 172.21.15.63:0/2972500198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8a59e40 0x55d2c8bdde90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1618003730 tx=0x7f1618021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2981531aace94308 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:51.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.006+0000 7f1615600700 1 -- 172.21.15.63:0/2972500198 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16180039c0 con 0x55d2c8a59e40 2024-05-27T07:47:51.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.006+0000 7f1615600700 1 -- 172.21.15.63:0/2972500198 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f161800f460 con 0x55d2c8a59e40 2024-05-27T07:47:51.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.006+0000 7f1615600700 1 -- 172.21.15.63:0/2972500198 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618031670 con 0x55d2c8a59e40 2024-05-27T07:47:51.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.007+0000 7f163a23fb80 1 -- 172.21.15.63:0/2972500198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8a59e40 msgr2=0x55d2c8bdde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.007+0000 7f163a23fb80 1 --2- 172.21.15.63:0/2972500198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8a59e40 0x55d2c8bdde90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1618003730 tx=0x7f1618021000 comp rx=0 tx=0).stop 2024-05-27T07:47:51.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.007+0000 7f163a23fb80 1 -- 172.21.15.63:0/2972500198 shutdown_connections 2024-05-27T07:47:51.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.007+0000 7f163a23fb80 1 --2- 172.21.15.63:0/2972500198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8bd8e70 0x55d2c8bd4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.007+0000 7f163a23fb80 1 --2- 172.21.15.63:0/2972500198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8a59e40 0x55d2c8bdde90 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.007+0000 7f163a23fb80 1 --2- 172.21.15.63:0/2972500198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8be15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.007+0000 7f163a23fb80 1 -- 172.21.15.63:0/2972500198 >> 172.21.15.63:0/2972500198 conn(0x55d2c8af3390 msgr2=0x55d2c8bd3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:51.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.007+0000 7f163a23fb80 1 -- 172.21.15.63:0/2972500198 shutdown_connections 2024-05-27T07:47:51.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.007+0000 7f163a23fb80 1 -- 172.21.15.63:0/2972500198 wait complete. 2024-05-27T07:47:51.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.008+0000 7f163a23fb80 1 -- start start 2024-05-27T07:47:51.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.008+0000 7f163a23fb80 1 Processor -- start 2024-05-27T07:47:51.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8be6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8be6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f163a23fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8c55580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f163a23fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16241b0de0 con 0x55d2c8bd8e70 2024-05-27T07:47:51.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f163a23fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16241b2250 con 0x55d2c8b459f0 2024-05-27T07:47:51.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f163a23fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16241b14b0 con 0x55d2c8a59e40 2024-05-27T07:47:51.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f163a23fb80 10 client.0 _ll_register_callbacks cb 0x55d2c8aa0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f1629600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8c55580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f1629600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8c55580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49196/0 (socket says 172.21.15.63:49196) 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f1629600700 1 -- 172.21.15.63:0/646302666 learned_addr learned my addr 172.21.15.63:0/646302666 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f1628c00700 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8be6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.009+0000 7f1623e00700 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8be6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.010+0000 7f1629600700 1 -- 172.21.15.63:0/646302666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 msgr2=0x55d2c8be6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.010+0000 7f1629600700 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8be6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.010+0000 7f1629600700 1 -- 172.21.15.63:0/646302666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 msgr2=0x55d2c8be6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.010+0000 7f1629600700 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8be6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.010+0000 7f1629600700 1 -- 172.21.15.63:0/646302666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c8c55bf0 con 0x55d2c8bd8e70 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.010+0000 7f1629600700 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8c55580 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f161c0a5340 tx=0x7f161c005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87bf27520e9b56d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:51.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.010+0000 7f160aa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:51.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.010+0000 7f160aa00700 1 -- 172.21.15.63:0/646302666 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0a5180 con 0x55d2c8bd8e70 2024-05-27T07:47:51.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.011+0000 7f160aa00700 1 -- 172.21.15.63:0/646302666 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f161c0b1ea0 con 0x55d2c8bd8e70 2024-05-27T07:47:51.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.011+0000 7f163a23fb80 1 -- 172.21.15.63:0/646302666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f161800f460 con 0x55d2c8bd8e70 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.011+0000 7f160aa00700 1 -- 172.21.15.63:0/646302666 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f161c0b2120 con 0x55d2c8bd8e70 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.011+0000 7f163a23fb80 10 client.18180 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.011+0000 7f163a23fb80 1 -- 172.21.15.63:0/646302666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d2c8c5ced0 con 0x55d2c8bd8e70 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.011+0000 7f160aa00700 1 -- 172.21.15.63:0/646302666 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f161800f460 con 0x55d2c8bd8e70 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.011+0000 7f160aa00700 1 -- 172.21.15.63:0/646302666 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f161c012070 con 0x55d2c8bd8e70 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.012+0000 7f163a23fb80 10 client.18180 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.012+0000 7f163a23fb80 10 client.18180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.012+0000 7f163a23fb80 20 client.18180 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.012+0000 7f163a23fb80 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2c8c5d1b0 0x55d2c8c5f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.012+0000 7f163a23fb80 4 client.18180 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:47:51.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.012+0000 7f163a23fb80 1 -- 172.21.15.63:0/646302666 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2c8b02570 con 0x55d2c8c5d1b0 2024-05-27T07:47:51.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.012+0000 7f1628c00700 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2c8c5d1b0 0x55d2c8c5f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.013+0000 7f1628c00700 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2c8c5d1b0 0x55d2c8c5f5d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b89dba861420c4a server_cookie=51a152dba609a047 in_seq=0 out_seq=0 2024-05-27T07:47:51.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.013+0000 7f160aa00700 10 client.18180 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:51.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.014+0000 7f160aa00700 1 -- 172.21.15.63:0/646302666 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2c8b02570 con 0x55d2c8c5d1b0 2024-05-27T07:47:51.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.014+0000 7f160aa00700 10 client.18180 handle_command_reply: tid=0 2024-05-27T07:47:51.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.014+0000 7f163a23fb80 1 client.18180 shutdown 2024-05-27T07:47:51.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.014+0000 7f163a23fb80 1 -- 172.21.15.63:0/646302666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 msgr2=0x55d2c8c55580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.014+0000 7f163a23fb80 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8c55580 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f161c0a5340 tx=0x7f161c005570 comp rx=0 tx=0).stop 2024-05-27T07:47:51.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.014+0000 7f163a23fb80 1 -- 172.21.15.63:0/646302666 shutdown_connections 2024-05-27T07:47:51.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.014+0000 7f163a23fb80 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2c8c5d1b0 0x55d2c8c5f5d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.014+0000 7f163a23fb80 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2c8bd8e70 0x55d2c8c55580 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.014+0000 7f160aa00700 0 client.18180 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:51.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.014+0000 7f163a23fb80 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2c8a59e40 0x55d2c8be6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.015+0000 7f163a23fb80 1 --2- 172.21.15.63:0/646302666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2c8b459f0 0x55d2c8be6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.015+0000 7f163a23fb80 1 -- 172.21.15.63:0/646302666 >> 172.21.15.63:0/646302666 conn(0x55d2c8af3390 msgr2=0x55d2c8be3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:51.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.015+0000 7f163a23fb80 1 -- 172.21.15.63:0/646302666 shutdown_connections 2024-05-27T07:47:51.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.015+0000 7f163a23fb80 1 -- 172.21.15.63:0/646302666 wait complete. 2024-05-27T07:47:51.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.015+0000 7f163a23fb80 20 client.18180 trim_cache size 0 max 16384 2024-05-27T07:47:51.016 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:47:51.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 -- 172.21.15.63:0/829230576 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f160c0b29f0 msgr2=0x7f160c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f160c0b29f0 0x7f160c0b4f00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f161800b5c0 tx=0x7f1618005960 comp rx=0 tx=0).stop 2024-05-27T07:47:51.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 -- 172.21.15.63:0/829230576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1624110b90 msgr2=0x7f16241a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1624110b90 0x7f16241a17b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f161c0094f0 tx=0x7f161c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 -- 172.21.15.63:0/829230576 shutdown_connections 2024-05-27T07:47:51.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f160c0b29f0 0x7f160c0b4f00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1624110b90 0x7f16241a17b0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1624075d20 0x7f162419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 --2- 172.21.15.63:0/829230576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1624076670 0x7f162419d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 -- 172.21.15.63:0/829230576 >> 172.21.15.63:0/829230576 conn(0x7f1624100030 msgr2=0x7f1624100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:51.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 -- 172.21.15.63:0/829230576 shutdown_connections 2024-05-27T07:47:51.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:51.016+0000 7f162b600700 1 -- 172.21.15.63:0/829230576 wait complete. 2024-05-27T07:47:51.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:51 smithi086 ceph-mon[34687]: pgmap v447: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:47:51.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:51 smithi149 ceph-mon[35408]: pgmap v447: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:47:51.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:51 smithi063 ceph-mon[31613]: pgmap v447: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:47:53.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:53 smithi086 ceph-mon[34687]: pgmap v448: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-05-27T07:47:53.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:53 smithi149 ceph-mon[35408]: pgmap v448: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-05-27T07:47:53.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:53 smithi063 ceph-mon[31613]: pgmap v448: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-05-27T07:47:54.556 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:47:54.557 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:47:55.065 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:55.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:55 smithi086 ceph-mon[34687]: pgmap v449: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:47:55.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:55 smithi149 ceph-mon[35408]: pgmap v449: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:47:55.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:55 smithi063 ceph-mon[31613]: pgmap v449: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:47:57.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:57 smithi149 ceph-mon[35408]: pgmap v450: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:47:57.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:57 smithi063 ceph-mon[31613]: pgmap v450: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:47:57.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.515+0000 7f78c2c00700 1 -- 172.21.15.63:0/3912548281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78bc108730 msgr2=0x7f78bc110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:57.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.515+0000 7f78c2c00700 1 --2- 172.21.15.63:0/3912548281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78bc108730 0x7f78bc110090 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f78b400b0d0 tx=0x7f78b400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.515+0000 7f78c2c00700 1 -- 172.21.15.63:0/3912548281 shutdown_connections 2024-05-27T07:47:57.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.515+0000 7f78c2c00700 1 --2- 172.21.15.63:0/3912548281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78bc108730 0x7f78bc110090 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.515+0000 7f78c2c00700 1 --2- 172.21.15.63:0/3912548281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc1034b0 0x7f78bc1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.515+0000 7f78c2c00700 1 --2- 172.21.15.63:0/3912548281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78bc107de0 0x7f78bc1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.515+0000 7f78c2c00700 1 -- 172.21.15.63:0/3912548281 >> 172.21.15.63:0/3912548281 conn(0x7f78bc0fedb0 msgr2=0x7f78bc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:57.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.516+0000 7f78c2c00700 1 -- 172.21.15.63:0/3912548281 shutdown_connections 2024-05-27T07:47:57.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.516+0000 7f78c2c00700 1 -- 172.21.15.63:0/3912548281 wait complete. 2024-05-27T07:47:57.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.517+0000 7f78c2c00700 1 Processor -- start 2024-05-27T07:47:57.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.517+0000 7f78c2c00700 1 -- start start 2024-05-27T07:47:57.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.517+0000 7f78c2c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc1034b0 0x7f78bc19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.518+0000 7f78c2c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78bc107de0 0x7f78bc19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.518+0000 7f78c2c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78bc108730 0x7f78bc1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.518+0000 7f78c2c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc113210 con 0x7f78bc107de0 2024-05-27T07:47:57.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.518+0000 7f78c2c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc113090 con 0x7f78bc1034b0 2024-05-27T07:47:57.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.518+0000 7f78c2c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc113390 con 0x7f78bc108730 2024-05-27T07:47:57.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.518+0000 7f78bb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78bc107de0 0x7f78bc19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.518+0000 7f78bb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78bc107de0 0x7f78bc19d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49214/0 (socket says 172.21.15.63:49214) 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.518+0000 7f78bb400700 1 -- 172.21.15.63:0/3057556784 learned_addr learned my addr 172.21.15.63:0/3057556784 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.518+0000 7f78c0c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78bc108730 0x7f78bc1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.518+0000 7f78bbe00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc1034b0 0x7f78bc19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.519+0000 7f78bb400700 1 -- 172.21.15.63:0/3057556784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78bc108730 msgr2=0x7f78bc1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.519+0000 7f78bb400700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78bc108730 0x7f78bc1a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.519+0000 7f78bb400700 1 -- 172.21.15.63:0/3057556784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc1034b0 msgr2=0x7f78bc19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.519+0000 7f78bb400700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc1034b0 0x7f78bc19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.519+0000 7f78bb400700 1 -- 172.21.15.63:0/3057556784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc1a1de0 con 0x7f78bc107de0 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.519+0000 7f78c0c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78bc108730 0x7f78bc1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:57.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.519+0000 7f78bb400700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78bc107de0 0x7f78bc19d4b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f78a800e970 tx=0x7f78a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.520+0000 7f78b8c00700 1 -- 172.21.15.63:0/3057556784 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a800cbd0 con 0x7f78bc107de0 2024-05-27T07:47:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.520+0000 7f78b8c00700 1 -- 172.21.15.63:0/3057556784 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78a800cd70 con 0x7f78bc107de0 2024-05-27T07:47:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.520+0000 7f78c2c00700 1 -- 172.21.15.63:0/3057556784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc1a20d0 con 0x7f78bc107de0 2024-05-27T07:47:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.520+0000 7f78b8c00700 1 -- 172.21.15.63:0/3057556784 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8005570 con 0x7f78bc107de0 2024-05-27T07:47:57.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.520+0000 7f78c2c00700 1 -- 172.21.15.63:0/3057556784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc1a99f0 con 0x7f78bc107de0 2024-05-27T07:47:57.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.525+0000 7f78d1676b80 1 Processor -- start 2024-05-27T07:47:57.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.525+0000 7f78b8c00700 1 -- 172.21.15.63:0/3057556784 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f78a8004750 con 0x7f78bc107de0 2024-05-27T07:47:57.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.525+0000 7f78d1676b80 1 -- start start 2024-05-27T07:47:57.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.526+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a20a1a70 0x55b9a21343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.526+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a21348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.526+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a2134ef0 0x55b9a2141190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.526+0000 7f78d1676b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2062ec0 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.526+0000 7f78d1676b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2052a60 con 0x55b9a2134ef0 2024-05-27T07:47:57.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.526+0000 7f78d1676b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2056aa0 con 0x55b9a20a1a70 2024-05-27T07:47:57.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.526+0000 7f78bbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a21348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.526+0000 7f78bbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a21348f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49228/0 (socket says 172.21.15.63:49228) 2024-05-27T07:47:57.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.526+0000 7f78bbe00700 1 -- 172.21.15.63:0/3577936954 learned_addr learned my addr 172.21.15.63:0/3577936954 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:57.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.526+0000 7f78c0c00700 1 --2- 172.21.15.63:0/3577936954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a20a1a70 0x55b9a21343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.527+0000 7f78bb400700 1 --2- 172.21.15.63:0/3577936954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a2134ef0 0x55b9a2141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.527+0000 7f78bbe00700 1 -- 172.21.15.63:0/3577936954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a20a1a70 msgr2=0x55b9a21343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:57.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.527+0000 7f78bbe00700 1 --2- 172.21.15.63:0/3577936954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a20a1a70 0x55b9a21343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.527+0000 7f78bbe00700 1 -- 172.21.15.63:0/3577936954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a2134ef0 msgr2=0x55b9a2141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:57.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.527+0000 7f78bbe00700 1 --2- 172.21.15.63:0/3577936954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a2134ef0 0x55b9a2141190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.527+0000 7f78bbe00700 1 -- 172.21.15.63:0/3577936954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a201b4a0 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.527+0000 7f78bb400700 1 --2- 172.21.15.63:0/3577936954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a2134ef0 0x55b9a2141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:57.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.527+0000 7f78bbe00700 1 --2- 172.21.15.63:0/3577936954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a21348f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009d80 tx=0x7f78ac011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caeb9c3f391c033 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:57.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.528+0000 7f78b8c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78a00b29f0 0x7f78a00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.528+0000 7f78b1600700 1 -- 172.21.15.63:0/3577936954 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac004230 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.528+0000 7f78b1600700 1 -- 172.21.15.63:0/3577936954 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78ac0043d0 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.528+0000 7f78b1600700 1 -- 172.21.15.63:0/3577936954 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac0197e0 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.528+0000 7f78b8c00700 1 -- 172.21.15.63:0/3057556784 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f78a80e2c20 con 0x7f78bc107de0 2024-05-27T07:47:57.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.528+0000 7f78c0c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78a00b29f0 0x7f78a00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.529+0000 7f78d1676b80 1 -- 172.21.15.63:0/3577936954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 msgr2=0x55b9a21348f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:57.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.529+0000 7f78c0c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78a00b29f0 0x7f78a00b4f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55b9a2137070 tx=0x7f78b400b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:57.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.529+0000 7f78d1676b80 1 --2- 172.21.15.63:0/3577936954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a21348f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009d80 tx=0x7f78ac011720 comp rx=0 tx=0).stop 2024-05-27T07:47:57.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.529+0000 7f78d1676b80 1 -- 172.21.15.63:0/3577936954 shutdown_connections 2024-05-27T07:47:57.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.529+0000 7f78d1676b80 1 --2- 172.21.15.63:0/3577936954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a2134ef0 0x55b9a2141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.529+0000 7f78d1676b80 1 --2- 172.21.15.63:0/3577936954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a20a1a70 0x55b9a21343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.529+0000 7f78d1676b80 1 --2- 172.21.15.63:0/3577936954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a21348f0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.529+0000 7f78d1676b80 1 -- 172.21.15.63:0/3577936954 >> 172.21.15.63:0/3577936954 conn(0x55b9a204f410 msgr2=0x55b9a2130540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:57.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.530+0000 7f78d1676b80 1 -- 172.21.15.63:0/3577936954 shutdown_connections 2024-05-27T07:47:57.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.530+0000 7f78d1676b80 1 -- 172.21.15.63:0/3577936954 wait complete. 2024-05-27T07:47:57.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.530+0000 7f78d1676b80 1 -- start start 2024-05-27T07:47:57.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.530+0000 7f78d1676b80 1 Processor -- start 2024-05-27T07:47:57.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.531+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a213e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.531+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a20a1a70 0x55b9a213ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.531+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 0x55b9a213f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.531+0000 7f78d1676b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b9a1f59020 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.531+0000 7f78d1676b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b9a1ff37e0 con 0x55b9a20a1a70 2024-05-27T07:47:57.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.531+0000 7f78d1676b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2141c00 con 0x55b9a2134ef0 2024-05-27T07:47:57.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.531+0000 7f78d1676b80 10 client.0 _ll_register_callbacks cb 0x55b9a1ffccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:57.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.531+0000 7f78bbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a213e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78bbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a213e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49232/0 (socket says 172.21.15.63:49232) 2024-05-27T07:47:57.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78bbe00700 1 -- 172.21.15.63:0/4076491961 learned_addr learned my addr 172.21.15.63:0/4076491961 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:57.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78bb400700 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a20a1a70 0x55b9a213ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78bbe00700 1 -- 172.21.15.63:0/4076491961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 msgr2=0x55b9a213f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:57.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78c0c00700 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 0x55b9a213f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78bbe00700 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 0x55b9a213f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78bbe00700 1 -- 172.21.15.63:0/4076491961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a20a1a70 msgr2=0x55b9a213ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:57.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78bbe00700 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a20a1a70 0x55b9a213ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78bbe00700 1 -- 172.21.15.63:0/4076491961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a1fa2950 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78bb400700 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a20a1a70 0x55b9a213ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:57.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.532+0000 7f78c0c00700 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 0x55b9a213f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:57.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.533+0000 7f78bbe00700 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a213e750 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f78ac005270 tx=0x7f78ac005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4f3896ccde0e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:57.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.533+0000 7f78a6000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:57.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.533+0000 7f78a6000700 1 -- 172.21.15.63:0/4076491961 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac027070 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.533+0000 7f78a6000700 1 -- 172.21.15.63:0/4076491961 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f78ac003880 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.533+0000 7f78d1676b80 1 -- 172.21.15.63:0/4076491961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78ac0043d0 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.533+0000 7f78a6000700 1 -- 172.21.15.63:0/4076491961 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f78ac033050 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.533+0000 7f78d1676b80 10 client.18192 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:57.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.533+0000 7f78d1676b80 1 -- 172.21.15.63:0/4076491961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b9a20230e0 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.534+0000 7f78a6000700 1 -- 172.21.15.63:0/4076491961 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78ac0043d0 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.534+0000 7f78a6000700 1 -- 172.21.15.63:0/4076491961 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f78ac004ca0 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.534+0000 7f78d1676b80 10 client.18192 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:57.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.534+0000 7f78d1676b80 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:57.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.534+0000 7f78d1676b80 20 client.18192 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:57.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.534+0000 7f78d1676b80 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b9a214eac0 0x55b9a2150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.534+0000 7f78d1676b80 4 client.18192 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:57.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.534+0000 7f78d1676b80 1 -- 172.21.15.63:0/4076491961 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9a2064e10 con 0x55b9a214eac0 2024-05-27T07:47:57.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.535+0000 7f78bb400700 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b9a214eac0 0x55b9a2150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.536+0000 7f78bb400700 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b9a214eac0 0x55b9a2150ee0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795bfea527e8112 server_cookie=5f70b0c4c1095071 in_seq=0 out_seq=0 2024-05-27T07:47:57.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.536+0000 7f78a6000700 10 client.18192 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:57.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78a6000700 1 -- 172.21.15.63:0/4076491961 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b9a2064e10 con 0x55b9a214eac0 2024-05-27T07:47:57.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78a6000700 10 client.18192 handle_command_reply: tid=0 2024-05-27T07:47:57.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78d1676b80 1 client.18192 shutdown 2024-05-27T07:47:57.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78d1676b80 1 -- 172.21.15.63:0/4076491961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 msgr2=0x55b9a213e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:57.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78d1676b80 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a213e750 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f78ac005270 tx=0x7f78ac005350 comp rx=0 tx=0).stop 2024-05-27T07:47:57.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78d1676b80 1 -- 172.21.15.63:0/4076491961 shutdown_connections 2024-05-27T07:47:57.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78d1676b80 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b9a214eac0 0x55b9a2150ee0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78d1676b80 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 0x55b9a213f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78d1676b80 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a1fb5ec0 0x55b9a213e750 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78d1676b80 1 --2- 172.21.15.63:0/4076491961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a20a1a70 0x55b9a213ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78d1676b80 1 -- 172.21.15.63:0/4076491961 >> 172.21.15.63:0/4076491961 conn(0x55b9a204f410 msgr2=0x55b9a2134390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:57.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.538+0000 7f78a6000700 0 client.18192 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:57.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.539+0000 7f78d1676b80 1 -- 172.21.15.63:0/4076491961 shutdown_connections 2024-05-27T07:47:57.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.539+0000 7f78d1676b80 1 -- 172.21.15.63:0/4076491961 wait complete. 2024-05-27T07:47:57.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.539+0000 7f78d1676b80 20 client.18192 trim_cache size 0 max 16384 2024-05-27T07:47:57.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.567+0000 7f78d1676b80 1 Processor -- start 2024-05-27T07:47:57.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.567+0000 7f78d1676b80 1 -- start start 2024-05-27T07:47:57.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.567+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a1fb5ec0 0x55b9a213d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.567+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a20a1a70 0x55b9a214aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.567+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a2134ef0 0x55b9a2140510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.567+0000 7f78d1676b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1b4fd0 con 0x55b9a2134ef0 2024-05-27T07:47:57.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.567+0000 7f78d1676b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1b2860 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.567+0000 7f78d1676b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1b06d0 con 0x55b9a20a1a70 2024-05-27T07:47:57.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.568+0000 7f78c0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a2134ef0 0x55b9a2140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.568+0000 7f78c0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a2134ef0 0x55b9a2140510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49244/0 (socket says 172.21.15.63:49244) 2024-05-27T07:47:57.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.568+0000 7f78c0c00700 1 -- 172.21.15.63:0/2802915597 learned_addr learned my addr 172.21.15.63:0/2802915597 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:57.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.568+0000 7f78bbe00700 1 --2- 172.21.15.63:0/2802915597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a1fb5ec0 0x55b9a213d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.568+0000 7f78bb400700 1 --2- 172.21.15.63:0/2802915597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a20a1a70 0x55b9a214aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.568+0000 7f78c0c00700 1 -- 172.21.15.63:0/2802915597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a20a1a70 msgr2=0x55b9a214aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:57.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.568+0000 7f78c0c00700 1 --2- 172.21.15.63:0/2802915597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a20a1a70 0x55b9a214aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.568+0000 7f78c0c00700 1 -- 172.21.15.63:0/2802915597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a1fb5ec0 msgr2=0x55b9a213d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:57.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.568+0000 7f78c0c00700 1 --2- 172.21.15.63:0/2802915597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a1fb5ec0 0x55b9a213d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.568+0000 7f78c0c00700 1 -- 172.21.15.63:0/2802915597 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a2131080 con 0x55b9a2134ef0 2024-05-27T07:47:57.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.569+0000 7f78c0c00700 1 --2- 172.21.15.63:0/2802915597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a2134ef0 0x55b9a2140510 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f78b4015000 tx=0x7f78b401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89d1ecf84328529 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:57.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.569+0000 7f78b0c00700 1 -- 172.21.15.63:0/2802915597 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4023070 con 0x55b9a2134ef0 2024-05-27T07:47:57.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.569+0000 7f78b0c00700 1 -- 172.21.15.63:0/2802915597 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78b4007c50 con 0x55b9a2134ef0 2024-05-27T07:47:57.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.569+0000 7f78b0c00700 1 -- 172.21.15.63:0/2802915597 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4029c60 con 0x55b9a2134ef0 2024-05-27T07:47:57.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.570+0000 7f78d1676b80 1 -- 172.21.15.63:0/2802915597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a2134ef0 msgr2=0x55b9a2140510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:57.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.570+0000 7f78d1676b80 1 --2- 172.21.15.63:0/2802915597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a2134ef0 0x55b9a2140510 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f78b4015000 tx=0x7f78b401f000 comp rx=0 tx=0).stop 2024-05-27T07:47:57.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.571+0000 7f78d1676b80 1 -- 172.21.15.63:0/2802915597 shutdown_connections 2024-05-27T07:47:57.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.571+0000 7f78d1676b80 1 --2- 172.21.15.63:0/2802915597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a2134ef0 0x55b9a2140510 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.571+0000 7f78d1676b80 1 --2- 172.21.15.63:0/2802915597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a1fb5ec0 0x55b9a213d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.571+0000 7f78d1676b80 1 --2- 172.21.15.63:0/2802915597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a20a1a70 0x55b9a214aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.571+0000 7f78d1676b80 1 -- 172.21.15.63:0/2802915597 >> 172.21.15.63:0/2802915597 conn(0x55b9a204f410 msgr2=0x55b9a213fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:57.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.571+0000 7f78d1676b80 1 -- 172.21.15.63:0/2802915597 shutdown_connections 2024-05-27T07:47:57.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.571+0000 7f78d1676b80 1 -- 172.21.15.63:0/2802915597 wait complete. 2024-05-27T07:47:57.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.571+0000 7f78d1676b80 1 -- start start 2024-05-27T07:47:57.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.572+0000 7f78d1676b80 1 Processor -- start 2024-05-27T07:47:57.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.572+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a1fb5ec0 0x55b9a2138f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.572+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a20a1a70 0x55b9a2139450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.572+0000 7f78d1676b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 0x55b9a2139990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.572+0000 7f78d1676b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1b0d20 con 0x55b9a20a1a70 2024-05-27T07:47:57.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.572+0000 7f78d1676b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1b2190 con 0x55b9a1fb5ec0 2024-05-27T07:47:57.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.572+0000 7f78d1676b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1b13f0 con 0x55b9a2134ef0 2024-05-27T07:47:57.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.572+0000 7f78d1676b80 10 client.0 _ll_register_callbacks cb 0x55b9a1ffccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:57.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.573+0000 7f78bb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a20a1a70 0x55b9a2139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.573+0000 7f78bb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a20a1a70 0x55b9a2139450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49254/0 (socket says 172.21.15.63:49254) 2024-05-27T07:47:57.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.573+0000 7f78bb400700 1 -- 172.21.15.63:0/955652482 learned_addr learned my addr 172.21.15.63:0/955652482 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.573+0000 7f78bbe00700 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a1fb5ec0 0x55b9a2138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.573+0000 7f78c0c00700 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 0x55b9a2139990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.573+0000 7f78bb400700 1 -- 172.21.15.63:0/955652482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 msgr2=0x55b9a2139990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.573+0000 7f78bb400700 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 0x55b9a2139990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.573+0000 7f78bb400700 1 -- 172.21.15.63:0/955652482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a1fb5ec0 msgr2=0x55b9a2138f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.573+0000 7f78bb400700 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a1fb5ec0 0x55b9a2138f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.573+0000 7f78bb400700 1 -- 172.21.15.63:0/955652482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a2145350 con 0x55b9a20a1a70 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.574+0000 7f78bb400700 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a20a1a70 0x55b9a2139450 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f78a80a9a70 tx=0x7f78a800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322bcde6eb077089 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.574+0000 7f78a6000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.574+0000 7f78a6000700 1 -- 172.21.15.63:0/955652482 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a80b2280 con 0x55b9a20a1a70 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.574+0000 7f78a6000700 1 -- 172.21.15.63:0/955652482 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7f78a80a9f50 con 0x55b9a20a1a70 2024-05-27T07:47:57.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.574+0000 7f78d1676b80 1 -- 172.21.15.63:0/955652482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b4007c50 con 0x55b9a20a1a70 2024-05-27T07:47:57.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78a6000700 1 -- 172.21.15.63:0/955652482 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f78a80a95f0 con 0x55b9a20a1a70 2024-05-27T07:47:57.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78d1676b80 10 client.18198 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:57.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78d1676b80 1 -- 172.21.15.63:0/955652482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b9a21b96a0 con 0x55b9a20a1a70 2024-05-27T07:47:57.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78a6000700 1 -- 172.21.15.63:0/955652482 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78b4007c50 con 0x55b9a20a1a70 2024-05-27T07:47:57.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78a6000700 1 -- 172.21.15.63:0/955652482 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f78a80b24b0 con 0x55b9a20a1a70 2024-05-27T07:47:57.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78d1676b80 10 client.18198 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:57.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78d1676b80 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:57.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78d1676b80 20 client.18198 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:57.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78d1676b80 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b9a21b9d30 0x55b9a21bc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:57.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78d1676b80 4 client.18198 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:47:57.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.575+0000 7f78d1676b80 1 -- 172.21.15.63:0/955652482 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b9a205e5f0 con 0x55b9a21b9d30 2024-05-27T07:47:57.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.576+0000 7f78bbe00700 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b9a21b9d30 0x55b9a21bc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:57.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.577+0000 7f78bbe00700 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b9a21b9d30 0x55b9a21bc150 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35229dbe12ba8cb9 server_cookie=a3aa053dcfd6b158 in_seq=0 out_seq=0 2024-05-27T07:47:57.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.577+0000 7f78a6000700 10 client.18198 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:57.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78a6000700 1 -- 172.21.15.63:0/955652482 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b9a205e5f0 con 0x55b9a21b9d30 2024-05-27T07:47:57.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78a6000700 10 client.18198 handle_command_reply: tid=0 2024-05-27T07:47:57.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78d1676b80 1 client.18198 shutdown 2024-05-27T07:47:57.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78d1676b80 1 -- 172.21.15.63:0/955652482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a20a1a70 msgr2=0x55b9a2139450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:57.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78d1676b80 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a20a1a70 0x55b9a2139450 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f78a80a9a70 tx=0x7f78a800ef50 comp rx=0 tx=0).stop 2024-05-27T07:47:57.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78d1676b80 1 -- 172.21.15.63:0/955652482 shutdown_connections 2024-05-27T07:47:57.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78d1676b80 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b9a21b9d30 0x55b9a21bc150 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78d1676b80 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b9a2134ef0 0x55b9a2139990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78d1676b80 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b9a1fb5ec0 0x55b9a2138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78d1676b80 1 --2- 172.21.15.63:0/955652482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b9a20a1a70 0x55b9a2139450 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78a6000700 0 client.18198 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:57.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.580+0000 7f78d1676b80 1 -- 172.21.15.63:0/955652482 >> 172.21.15.63:0/955652482 conn(0x55b9a204f410 msgr2=0x55b9a213d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:57.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.581+0000 7f78d1676b80 1 -- 172.21.15.63:0/955652482 shutdown_connections 2024-05-27T07:47:57.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.581+0000 7f78d1676b80 1 -- 172.21.15.63:0/955652482 wait complete. 2024-05-27T07:47:57.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.581+0000 7f78d1676b80 20 client.18198 trim_cache size 0 max 16384 2024-05-27T07:47:57.585 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:47:57.585 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:47:57.585 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "960da17b-16d7-48c4-9c56-fc20ee47ba65", 2024-05-27T07:47:57.585 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:47:57.585 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 -- 172.21.15.63:0/3057556784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78a00b29f0 msgr2=0x7f78a00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78a00b29f0 0x7f78a00b4f00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55b9a2137070 tx=0x7f78b400b040 comp rx=0 tx=0).stop 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 -- 172.21.15.63:0/3057556784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78bc107de0 msgr2=0x7f78bc19d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78bc107de0 0x7f78bc19d4b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f78a800e970 tx=0x7f78a800ec80 comp rx=0 tx=0).stop 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 -- 172.21.15.63:0/3057556784 shutdown_connections 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78a00b29f0 0x7f78a00b4f00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78bc108730 0x7f78bc1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc1034b0 0x7f78bc19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 --2- 172.21.15.63:0/3057556784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78bc107de0 0x7f78bc19d4b0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 -- 172.21.15.63:0/3057556784 >> 172.21.15.63:0/3057556784 conn(0x7f78bc0fedb0 msgr2=0x7f78bc06b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 -- 172.21.15.63:0/3057556784 shutdown_connections 2024-05-27T07:47:57.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:47:57.582+0000 7f78c2c00700 1 -- 172.21.15.63:0/3057556784 wait complete. 2024-05-27T07:47:57.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:57 smithi086 ceph-mon[34687]: pgmap v450: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:47:58.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:47:58.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:47:58.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:47:58.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:47:58.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:47:58.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:47:58.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.550 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:47:58.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:47:58.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:47:58.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:47:58.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:47:58.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:47:58.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:47:58.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:58.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:47:58.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.053+0000 7fde41800640 1 -- 172.21.15.63:0/2784699790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10dbc0 msgr2=0x7fde3c10ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.053+0000 7fde41800640 1 --2- 172.21.15.63:0/2784699790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10dbc0 0x7fde3c10ffe0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fde28009900 tx=0x7fde2802a480 comp rx=0 tx=0).stop 2024-05-27T07:47:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.053+0000 7fde41800640 1 -- 172.21.15.63:0/2784699790 shutdown_connections 2024-05-27T07:47:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.053+0000 7fde41800640 1 --2- 172.21.15.63:0/2784699790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde3c110520 0x7fde3c112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.053+0000 7fde41800640 1 --2- 172.21.15.63:0/2784699790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10dbc0 0x7fde3c10ffe0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.053+0000 7fde41800640 1 --2- 172.21.15.63:0/2784699790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde3c075420 0x7fde3c075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.053+0000 7fde41800640 1 -- 172.21.15.63:0/2784699790 >> 172.21.15.63:0/2784699790 conn(0x7fde3c079440 msgr2=0x7fde3c079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.053+0000 7fde41800640 1 -- 172.21.15.63:0/2784699790 shutdown_connections 2024-05-27T07:47:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.054+0000 7fde41800640 1 -- 172.21.15.63:0/2784699790 wait complete. 2024-05-27T07:47:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.055+0000 7fde41800640 1 Processor -- start 2024-05-27T07:47:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.055+0000 7fde41800640 1 -- start start 2024-05-27T07:47:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde41800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde3c075420 0x7fde3c1a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde41800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10dbc0 0x7fde3c1a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde41800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde3c110520 0x7fde3c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde41800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c115f80 con 0x7fde3c10dbc0 2024-05-27T07:47:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde41800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c115e00 con 0x7fde3c075420 2024-05-27T07:47:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde41800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c116100 con 0x7fde3c110520 2024-05-27T07:47:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde3a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10dbc0 0x7fde3c1a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde3a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10dbc0 0x7fde3c1a2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43966/0 (socket says 172.21.15.63:43966) 2024-05-27T07:47:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde3a000640 1 -- 172.21.15.63:0/1610943701 learned_addr learned my addr 172.21.15.63:0/1610943701 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde3aa00640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde3c075420 0x7fde3c1a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.056+0000 7fde3b400640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde3c110520 0x7fde3c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.057+0000 7fde3a000640 1 -- 172.21.15.63:0/1610943701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde3c110520 msgr2=0x7fde3c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.057+0000 7fde3a000640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde3c110520 0x7fde3c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.057+0000 7fde3a000640 1 -- 172.21.15.63:0/1610943701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde3c075420 msgr2=0x7fde3c1a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.057+0000 7fde3a000640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde3c075420 0x7fde3c1a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.057+0000 7fde3a000640 1 -- 172.21.15.63:0/1610943701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c19cf00 con 0x7fde3c10dbc0 2024-05-27T07:47:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.057+0000 7fde3aa00640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde3c075420 0x7fde3c1a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.057+0000 7fde3a000640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10dbc0 0x7fde3c1a2b90 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fde2802a9a0 tx=0x7fde28003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.057+0000 7fde33400640 1 -- 172.21.15.63:0/1610943701 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2803f070 con 0x7fde3c10dbc0 2024-05-27T07:47:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.058+0000 7fde33400640 1 -- 172.21.15.63:0/1610943701 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde2803baf0 con 0x7fde3c10dbc0 2024-05-27T07:47:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.058+0000 7fde41800640 1 -- 172.21.15.63:0/1610943701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3c19d190 con 0x7fde3c10dbc0 2024-05-27T07:47:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.058+0000 7fde33400640 1 -- 172.21.15.63:0/1610943701 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28035030 con 0x7fde3c10dbc0 2024-05-27T07:47:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.058+0000 7fde41800640 1 -- 172.21.15.63:0/1610943701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3c1a94d0 con 0x7fde3c10dbc0 2024-05-27T07:47:59.059 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.062+0000 7fde487ff740 1 Processor -- start 2024-05-27T07:47:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.062+0000 7fde487ff740 1 -- start start 2024-05-27T07:47:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.062+0000 7fde487ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527e41b0 0x557e52b86860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.062+0000 7fde487ff740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b86da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.062+0000 7fde487ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527c34d0 0x557e52b872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.062+0000 7fde487ff740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557e52af7b90 con 0x557e527bb2f0 2024-05-27T07:47:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.062+0000 7fde487ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557e52ae7730 con 0x557e527c34d0 2024-05-27T07:47:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.062+0000 7fde487ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557e52aeb770 con 0x557e527e41b0 2024-05-27T07:47:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.063+0000 7fde3aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b86da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.063+0000 7fde3aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b86da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43974/0 (socket says 172.21.15.63:43974) 2024-05-27T07:47:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.063+0000 7fde3aa00640 1 -- 172.21.15.63:0/474083451 learned_addr learned my addr 172.21.15.63:0/474083451 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.063+0000 7fde3b400640 1 --2- 172.21.15.63:0/474083451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527e41b0 0x557e52b86860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.063+0000 7fde33400640 1 -- 172.21.15.63:0/1610943701 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fde28042070 con 0x7fde3c10dbc0 2024-05-27T07:47:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.063+0000 7fde3aa00640 1 -- 172.21.15.63:0/474083451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527e41b0 msgr2=0x557e52b86860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.063+0000 7fde3aa00640 1 --2- 172.21.15.63:0/474083451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527e41b0 0x557e52b86860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.063+0000 7fde3aa00640 1 -- 172.21.15.63:0/474083451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527c34d0 msgr2=0x557e52b872e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:47:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.063+0000 7fde3aa00640 1 --2- 172.21.15.63:0/474083451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527c34d0 0x557e52b872e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.063+0000 7fde3aa00640 1 -- 172.21.15.63:0/474083451 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e52a808b0 con 0x557e527bb2f0 2024-05-27T07:47:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.064+0000 7fde3aa00640 1 --2- 172.21.15.63:0/474083451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b86da0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009cd0 tx=0x7fde2c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79a3034793f9632 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.064+0000 7fde27e00640 1 -- 172.21.15.63:0/474083451 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c012450 con 0x557e527bb2f0 2024-05-27T07:47:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.064+0000 7fde27e00640 1 -- 172.21.15.63:0/474083451 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde2c012a80 con 0x557e527bb2f0 2024-05-27T07:47:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.064+0000 7fde27e00640 1 -- 172.21.15.63:0/474083451 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00fb50 con 0x557e527bb2f0 2024-05-27T07:47:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.065+0000 7fde487ff740 1 -- 172.21.15.63:0/474083451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 msgr2=0x557e52b86da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.065+0000 7fde487ff740 1 --2- 172.21.15.63:0/474083451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b86da0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009cd0 tx=0x7fde2c0117b0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.065+0000 7fde487ff740 1 -- 172.21.15.63:0/474083451 shutdown_connections 2024-05-27T07:47:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.065+0000 7fde487ff740 1 --2- 172.21.15.63:0/474083451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527c34d0 0x557e52b872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.065+0000 7fde487ff740 1 --2- 172.21.15.63:0/474083451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b86da0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.065+0000 7fde487ff740 1 --2- 172.21.15.63:0/474083451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527e41b0 0x557e52b86860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.065+0000 7fde487ff740 1 -- 172.21.15.63:0/474083451 >> 172.21.15.63:0/474083451 conn(0x557e527d8300 msgr2=0x557e52b83010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.066+0000 7fde487ff740 1 -- 172.21.15.63:0/474083451 shutdown_connections 2024-05-27T07:47:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.066+0000 7fde487ff740 1 -- 172.21.15.63:0/474083451 wait complete. 2024-05-27T07:47:59.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.066+0000 7fde33400640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde200b28e0 0x7fde200b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.066+0000 7fde33400640 1 -- 172.21.15.63:0/1610943701 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fde281030f0 con 0x7fde3c10dbc0 2024-05-27T07:47:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.066+0000 7fde3aa00640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde200b28e0 0x7fde200b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.067+0000 7fde487ff740 1 -- start start 2024-05-27T07:47:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.067+0000 7fde487ff740 1 Processor -- start 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.067+0000 7fde3aa00640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde200b28e0 0x7fde200b4df0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00f7f0 tx=0x7fde2c014040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde487ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527bb2f0 0x557e52b909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde487ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527c34d0 0x557e52b90f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde487ff740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b9cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde487ff740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557e527cd790 con 0x557e527e41b0 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde487ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557e52a8fbc0 con 0x557e527bb2f0 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde487ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557e52b93990 con 0x557e527c34d0 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde487ff740 10 client.0 _ll_register_callbacks cb 0x557e52a6d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde3b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b9cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde3b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b9cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43982/0 (socket says 172.21.15.63:43982) 2024-05-27T07:47:59.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde3b400640 1 -- 172.21.15.63:0/395426958 learned_addr learned my addr 172.21.15.63:0/395426958 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde3a000640 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527c34d0 0x557e52b90f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.068+0000 7fde3aa00640 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527bb2f0 0x557e52b909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.069+0000 7fde3b400640 1 -- 172.21.15.63:0/395426958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527c34d0 msgr2=0x557e52b90f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.069+0000 7fde3b400640 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527c34d0 0x557e52b90f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.069+0000 7fde3b400640 1 -- 172.21.15.63:0/395426958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527bb2f0 msgr2=0x557e52b909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.069+0000 7fde3b400640 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527bb2f0 0x557e52b909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.069+0000 7fde3b400640 1 -- 172.21.15.63:0/395426958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e527fc210 con 0x557e527e41b0 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.069+0000 7fde3aa00640 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527bb2f0 0x557e52b909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.069+0000 7fde3b400640 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b9cd20 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fde34009fc0 tx=0x7fde340044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822a7cffde41316a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.069+0000 7fde24c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.070+0000 7fde24c00640 1 -- 172.21.15.63:0/395426958 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34011900 con 0x557e527e41b0 2024-05-27T07:47:59.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.070+0000 7fde24c00640 1 -- 172.21.15.63:0/395426958 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fde34011aa0 con 0x557e527e41b0 2024-05-27T07:47:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.070+0000 7fde487ff740 1 -- 172.21.15.63:0/395426958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde2c012a80 con 0x557e527e41b0 2024-05-27T07:47:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.070+0000 7fde24c00640 1 -- 172.21.15.63:0/395426958 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde34024050 con 0x557e527e41b0 2024-05-27T07:47:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.070+0000 7fde487ff740 10 client.18210 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.070+0000 7fde487ff740 1 -- 172.21.15.63:0/395426958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557e528c8620 con 0x557e527e41b0 2024-05-27T07:47:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.070+0000 7fde24c00640 1 -- 172.21.15.63:0/395426958 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde2c012a80 con 0x557e527e41b0 2024-05-27T07:47:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.071+0000 7fde24c00640 1 -- 172.21.15.63:0/395426958 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fde34010440 con 0x557e527e41b0 2024-05-27T07:47:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.071+0000 7fde487ff740 10 client.18210 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.071+0000 7fde487ff740 10 client.18210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.071+0000 7fde487ff740 20 client.18210 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.071+0000 7fde487ff740 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557e527a2d80 0x557e52ba2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.071+0000 7fde487ff740 4 client.18210 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.071+0000 7fde487ff740 1 -- 172.21.15.63:0/395426958 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e52af9ae0 con 0x557e527a2d80 2024-05-27T07:47:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.072+0000 7fde3aa00640 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557e527a2d80 0x557e52ba2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.073+0000 7fde3aa00640 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557e527a2d80 0x557e52ba2da0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53669a0c9c7e4942 server_cookie=c4235486c67a9c0e in_seq=0 out_seq=0 2024-05-27T07:47:59.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.073+0000 7fde24c00640 10 client.18210 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde24c00640 1 -- 172.21.15.63:0/395426958 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557e52af9ae0 con 0x557e527a2d80 2024-05-27T07:47:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde24c00640 10 client.18210 handle_command_reply: tid=0 2024-05-27T07:47:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde487ff740 1 client.18210 shutdown 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde487ff740 1 -- 172.21.15.63:0/395426958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 msgr2=0x557e52b9cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde487ff740 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b9cd20 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fde34009fc0 tx=0x7fde340044d0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde487ff740 1 -- 172.21.15.63:0/395426958 shutdown_connections 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde487ff740 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557e527a2d80 0x557e52ba2da0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde487ff740 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b9cd20 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde487ff740 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527c34d0 0x557e52b90f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde487ff740 1 --2- 172.21.15.63:0/395426958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527bb2f0 0x557e52b909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde24c00640 0 client.18210 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde487ff740 1 -- 172.21.15.63:0/395426958 >> 172.21.15.63:0/395426958 conn(0x557e527d8300 msgr2=0x557e52b88580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.075+0000 7fde487ff740 1 -- 172.21.15.63:0/395426958 shutdown_connections 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.076+0000 7fde487ff740 1 -- 172.21.15.63:0/395426958 wait complete. 2024-05-27T07:47:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.076+0000 7fde487ff740 20 client.18210 trim_cache size 0 max 16384 2024-05-27T07:47:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.100+0000 7fde487ff740 1 Processor -- start 2024-05-27T07:47:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.100+0000 7fde487ff740 1 -- start start 2024-05-27T07:47:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.101+0000 7fde487ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527bb2f0 0x557e52b90f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.101+0000 7fde487ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527c34d0 0x557e52b914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.101+0000 7fde487ff740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b8a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.101+0000 7fde487ff740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b5d20 con 0x557e527e41b0 2024-05-27T07:47:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.101+0000 7fde487ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b35f0 con 0x557e527c34d0 2024-05-27T07:47:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.101+0000 7fde487ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b1460 con 0x557e527bb2f0 2024-05-27T07:47:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.101+0000 7fde3b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b8a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.101+0000 7fde3b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b8a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43996/0 (socket says 172.21.15.63:43996) 2024-05-27T07:47:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.101+0000 7fde3b400640 1 -- 172.21.15.63:0/1058669125 learned_addr learned my addr 172.21.15.63:0/1058669125 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.101+0000 7fde3aa00640 1 --2- 172.21.15.63:0/1058669125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527bb2f0 0x557e52b90f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.102+0000 7fde3a000640 1 --2- 172.21.15.63:0/1058669125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527c34d0 0x557e52b914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.102+0000 7fde3b400640 1 -- 172.21.15.63:0/1058669125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527bb2f0 msgr2=0x557e52b90f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.102+0000 7fde3b400640 1 --2- 172.21.15.63:0/1058669125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527bb2f0 0x557e52b90f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.102+0000 7fde3b400640 1 -- 172.21.15.63:0/1058669125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527c34d0 msgr2=0x557e52b914a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.102+0000 7fde3b400640 1 --2- 172.21.15.63:0/1058669125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527c34d0 0x557e52b914a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.102+0000 7fde3b400640 1 -- 172.21.15.63:0/1058669125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e52b8ef60 con 0x557e527e41b0 2024-05-27T07:47:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.102+0000 7fde3a000640 1 --2- 172.21.15.63:0/1058669125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527c34d0 0x557e52b914a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.102+0000 7fde3b400640 1 --2- 172.21.15.63:0/1058669125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b8a480 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fde34009f90 tx=0x7fde34005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2999a1efa47d279 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.103+0000 7fde27400640 1 -- 172.21.15.63:0/1058669125 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34021040 con 0x557e527e41b0 2024-05-27T07:47:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.103+0000 7fde27400640 1 -- 172.21.15.63:0/1058669125 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde34010440 con 0x557e527e41b0 2024-05-27T07:47:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.103+0000 7fde27400640 1 -- 172.21.15.63:0/1058669125 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34011bf0 con 0x557e527e41b0 2024-05-27T07:47:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.104+0000 7fde487ff740 1 -- 172.21.15.63:0/1058669125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 msgr2=0x557e52b8a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.104+0000 7fde487ff740 1 --2- 172.21.15.63:0/1058669125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b8a480 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fde34009f90 tx=0x7fde34005f70 comp rx=0 tx=0).stop 2024-05-27T07:47:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.104+0000 7fde487ff740 1 -- 172.21.15.63:0/1058669125 shutdown_connections 2024-05-27T07:47:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.104+0000 7fde487ff740 1 --2- 172.21.15.63:0/1058669125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527e41b0 0x557e52b8a480 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.104+0000 7fde487ff740 1 --2- 172.21.15.63:0/1058669125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527c34d0 0x557e52b914a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.104+0000 7fde487ff740 1 --2- 172.21.15.63:0/1058669125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527bb2f0 0x557e52b90f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.104+0000 7fde487ff740 1 -- 172.21.15.63:0/1058669125 >> 172.21.15.63:0/1058669125 conn(0x557e527d8300 msgr2=0x557e52b865f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.104+0000 7fde487ff740 1 -- 172.21.15.63:0/1058669125 shutdown_connections 2024-05-27T07:47:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.105+0000 7fde487ff740 1 -- 172.21.15.63:0/1058669125 wait complete. 2024-05-27T07:47:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.105+0000 7fde487ff740 1 -- start start 2024-05-27T07:47:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.105+0000 7fde487ff740 1 Processor -- start 2024-05-27T07:47:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.106+0000 7fde487ff740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b99060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.106+0000 7fde487ff740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527c34d0 0x557e52b943b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.106+0000 7fde487ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527e41b0 0x557e52b948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.106+0000 7fde487ff740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b1ab0 con 0x557e527bb2f0 2024-05-27T07:47:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.106+0000 7fde487ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b2f20 con 0x557e527e41b0 2024-05-27T07:47:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.106+0000 7fde487ff740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1b2180 con 0x557e527c34d0 2024-05-27T07:47:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.106+0000 7fde487ff740 10 client.0 _ll_register_callbacks cb 0x7fde34009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.106+0000 7fde3aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b99060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.107+0000 7fde3aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b99060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44010/0 (socket says 172.21.15.63:44010) 2024-05-27T07:47:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.107+0000 7fde3aa00640 1 -- 172.21.15.63:0/3302730178 learned_addr learned my addr 172.21.15.63:0/3302730178 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:47:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.107+0000 7fde3b400640 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527e41b0 0x557e52b948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.107+0000 7fde3a000640 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527c34d0 0x557e52b943b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.107+0000 7fde3aa00640 1 -- 172.21.15.63:0/3302730178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527c34d0 msgr2=0x557e52b943b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.107+0000 7fde3aa00640 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527c34d0 0x557e52b943b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.107+0000 7fde3aa00640 1 -- 172.21.15.63:0/3302730178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527e41b0 msgr2=0x557e52b948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.107+0000 7fde3aa00640 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527e41b0 0x557e52b948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.107+0000 7fde3aa00640 1 -- 172.21.15.63:0/3302730178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e52b94e30 con 0x557e527bb2f0 2024-05-27T07:47:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.107+0000 7fde3b400640 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527e41b0 0x557e52b948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.108+0000 7fde3aa00640 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b99060 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00f7c0 tx=0x7fde2c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41f149e388c7b64 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.108+0000 7fde24c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:47:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.108+0000 7fde24c00640 1 -- 172.21.15.63:0/3302730178 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c041060 con 0x557e527bb2f0 2024-05-27T07:47:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.108+0000 7fde24c00640 1 -- 172.21.15.63:0/3302730178 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fde2c012510 con 0x557e527bb2f0 2024-05-27T07:47:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.108+0000 7fde487ff740 1 -- 172.21.15.63:0/3302730178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde34010440 con 0x557e527bb2f0 2024-05-27T07:47:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.109+0000 7fde24c00640 1 -- 172.21.15.63:0/3302730178 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde2c048050 con 0x557e527bb2f0 2024-05-27T07:47:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.109+0000 7fde487ff740 10 client.18216 fetch_fsmap learned FSMap version 33 2024-05-27T07:47:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.109+0000 7fde487ff740 1 -- 172.21.15.63:0/3302730178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557e52c0b000 con 0x557e527bb2f0 2024-05-27T07:47:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.109+0000 7fde24c00640 1 -- 172.21.15.63:0/3302730178 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde34010440 con 0x557e527bb2f0 2024-05-27T07:47:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.110+0000 7fde24c00640 1 -- 172.21.15.63:0/3302730178 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fde2c03b070 con 0x557e527bb2f0 2024-05-27T07:47:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.110+0000 7fde487ff740 10 client.18216 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:47:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.110+0000 7fde487ff740 10 client.18216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.110+0000 7fde487ff740 20 client.18216 populate_metadata read hostname 'smithi063' 2024-05-27T07:47:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.110+0000 7fde487ff740 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557e527781c0 0x557e52c0d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.110+0000 7fde487ff740 4 client.18216 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:47:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.110+0000 7fde487ff740 1 -- 172.21.15.63:0/3302730178 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e52af32c0 con 0x557e527781c0 2024-05-27T07:47:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.111+0000 7fde3a000640 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557e527781c0 0x557e52c0d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.112+0000 7fde3a000640 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557e527781c0 0x557e52c0d3f0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8721b5b9cfbeca46 server_cookie=92acd70357ff5382 in_seq=0 out_seq=0 2024-05-27T07:47:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.112+0000 7fde24c00640 10 client.18216 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde24c00640 1 -- 172.21.15.63:0/3302730178 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557e52af32c0 con 0x557e527781c0 2024-05-27T07:47:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde24c00640 10 client.18216 handle_command_reply: tid=0 2024-05-27T07:47:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde487ff740 1 client.18216 shutdown 2024-05-27T07:47:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde487ff740 1 -- 172.21.15.63:0/3302730178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 msgr2=0x557e52b99060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde487ff740 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b99060 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00f7c0 tx=0x7fde2c0117b0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde487ff740 1 -- 172.21.15.63:0/3302730178 shutdown_connections 2024-05-27T07:47:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde487ff740 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557e527781c0 0x557e52c0d3f0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde487ff740 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557e527e41b0 0x557e52b948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde24c00640 0 client.18216 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:47:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde487ff740 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557e527c34d0 0x557e52b943b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde487ff740 1 --2- 172.21.15.63:0/3302730178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557e527bb2f0 0x557e52b99060 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.144+0000 7fde487ff740 1 -- 172.21.15.63:0/3302730178 >> 172.21.15.63:0/3302730178 conn(0x557e527d8300 msgr2=0x557e52b87020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.145+0000 7fde487ff740 1 -- 172.21.15.63:0/3302730178 shutdown_connections 2024-05-27T07:47:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.145+0000 7fde487ff740 1 -- 172.21.15.63:0/3302730178 wait complete. 2024-05-27T07:47:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.145+0000 7fde487ff740 20 client.18216 trim_cache size 0 max 16384 2024-05-27T07:47:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:47:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:47:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:47:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 -- 172.21.15.63:0/1610943701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde200b28e0 msgr2=0x7fde200b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde200b28e0 0x7fde200b4df0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00f7f0 tx=0x7fde2c014040 comp rx=0 tx=0).stop 2024-05-27T07:47:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 -- 172.21.15.63:0/1610943701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10dbc0 msgr2=0x7fde3c1a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10dbc0 0x7fde3c1a2b90 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fde2802a9a0 tx=0x7fde28003b90 comp rx=0 tx=0).stop 2024-05-27T07:47:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 -- 172.21.15.63:0/1610943701 shutdown_connections 2024-05-27T07:47:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde200b28e0 0x7fde200b4df0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde3c110520 0x7fde3c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde3c10dbc0 0x7fde3c1a2b90 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 --2- 172.21.15.63:0/1610943701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde3c075420 0x7fde3c1a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 -- 172.21.15.63:0/1610943701 >> 172.21.15.63:0/1610943701 conn(0x7fde3c079440 msgr2=0x7fde3c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 -- 172.21.15.63:0/1610943701 shutdown_connections 2024-05-27T07:47:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:47:59.146+0000 7fde41800640 1 -- 172.21.15.63:0/1610943701 wait complete. 2024-05-27T07:47:59.553 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:59 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:47:59.553 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:59.553 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:47:59.553 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:59.553 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:47:59 smithi063 ceph-mon[31613]: pgmap v451: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 15 op/s 2024-05-27T07:47:59.614 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:59 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:47:59.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:59.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:47:59.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:59.615 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:47:59 smithi149 ceph-mon[35408]: pgmap v451: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 15 op/s 2024-05-27T07:47:59.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:59 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:47:59.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:59.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:47:59.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:59.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:47:59 smithi086 ceph-mon[34687]: pgmap v451: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 15 op/s 2024-05-27T07:48:01.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.418+0000 7fe6dee00700 1 -- 172.21.15.63:0/4253119120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6d8106560 msgr2=0x7fe6d8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:01.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.418+0000 7fe6dee00700 1 --2- 172.21.15.63:0/4253119120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6d8106560 0x7fe6d8106970 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009990 tx=0x7fe6c4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.418+0000 7fe6dee00700 1 -- 172.21.15.63:0/4253119120 shutdown_connections 2024-05-27T07:48:01.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.418+0000 7fe6dee00700 1 --2- 172.21.15.63:0/4253119120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d8106eb0 0x7fe6d8113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.418+0000 7fe6dee00700 1 --2- 172.21.15.63:0/4253119120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d810c5a0 0x7fe6d810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.418+0000 7fe6dee00700 1 --2- 172.21.15.63:0/4253119120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6d8106560 0x7fe6d8106970 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.418+0000 7fe6dee00700 1 -- 172.21.15.63:0/4253119120 >> 172.21.15.63:0/4253119120 conn(0x7fe6d8102040 msgr2=0x7fe6d8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:01.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.418+0000 7fe6dee00700 1 -- 172.21.15.63:0/4253119120 shutdown_connections 2024-05-27T07:48:01.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.418+0000 7fe6dee00700 1 -- 172.21.15.63:0/4253119120 wait complete. 2024-05-27T07:48:01.421 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:01 smithi063 ceph-mon[31613]: pgmap v452: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T07:48:01.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.420+0000 7fe6dee00700 1 Processor -- start 2024-05-27T07:48:01.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.420+0000 7fe6dee00700 1 -- start start 2024-05-27T07:48:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.421+0000 7fe6dee00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6d8106560 0x7fe6d819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.421+0000 7fe6dee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d8106eb0 0x7fe6d819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.421+0000 7fe6dee00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d810c5a0 0x7fe6d81a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.421+0000 7fe6dee00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8116530 con 0x7fe6d8106560 2024-05-27T07:48:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.421+0000 7fe6dee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81163b0 con 0x7fe6d8106eb0 2024-05-27T07:48:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.421+0000 7fe6dee00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81166b0 con 0x7fe6d810c5a0 2024-05-27T07:48:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.422+0000 7fe6d7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6d8106560 0x7fe6d819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.422+0000 7fe6d7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6d8106560 0x7fe6d819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44028/0 (socket says 172.21.15.63:44028) 2024-05-27T07:48:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.422+0000 7fe6d7e00700 1 -- 172.21.15.63:0/2102461442 learned_addr learned my addr 172.21.15.63:0/2102461442 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.422+0000 7fe6d7400700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d8106eb0 0x7fe6d819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.422+0000 7fe6dce00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d810c5a0 0x7fe6d81a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.423+0000 7fe6d7e00700 1 -- 172.21.15.63:0/2102461442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d810c5a0 msgr2=0x7fe6d81a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.423+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d810c5a0 0x7fe6d81a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.423+0000 7fe6d7e00700 1 -- 172.21.15.63:0/2102461442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d8106eb0 msgr2=0x7fe6d819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.423+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d8106eb0 0x7fe6d819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.423+0000 7fe6d7e00700 1 -- 172.21.15.63:0/2102461442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d81a1dd0 con 0x7fe6d8106560 2024-05-27T07:48:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.423+0000 7fe6d7400700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d8106eb0 0x7fe6d819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.424+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6d8106560 0x7fe6d819cf00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800d980 tx=0x7fe6c800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:01.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.424+0000 7fe6d4c00700 1 -- 172.21.15.63:0/2102461442 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8017070 con 0x7fe6d8106560 2024-05-27T07:48:01.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.425+0000 7fe6d4c00700 1 -- 172.21.15.63:0/2102461442 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c8010460 con 0x7fe6d8106560 2024-05-27T07:48:01.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.425+0000 7fe6dee00700 1 -- 172.21.15.63:0/2102461442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d81a20c0 con 0x7fe6d8106560 2024-05-27T07:48:01.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.425+0000 7fe6d4c00700 1 -- 172.21.15.63:0/2102461442 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800de60 con 0x7fe6d8106560 2024-05-27T07:48:01.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.425+0000 7fe6dee00700 1 -- 172.21.15.63:0/2102461442 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81a99e0 con 0x7fe6d8106560 2024-05-27T07:48:01.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.431+0000 7fe6d4c00700 1 -- 172.21.15.63:0/2102461442 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe6c800f620 con 0x7fe6d8106560 2024-05-27T07:48:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.431+0000 7fe6ed967b80 1 Processor -- start 2024-05-27T07:48:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.432+0000 7fe6ed967b80 1 -- start start 2024-05-27T07:48:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.432+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb2c79f0 0x5594fb35a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.432+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 0x5594fb35a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.432+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb35ae70 0x5594fb367110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.432+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5594fb288e40 con 0x5594fb35ae70 2024-05-27T07:48:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.432+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594fb2789e0 con 0x5594fb2c79f0 2024-05-27T07:48:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.432+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5594fb27ca20 con 0x5594fb1dbe40 2024-05-27T07:48:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.433+0000 7fe6d7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb35ae70 0x5594fb367110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.433+0000 7fe6dce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb2c79f0 0x5594fb35a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.433+0000 7fe6d7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb35ae70 0x5594fb367110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44030/0 (socket says 172.21.15.63:44030) 2024-05-27T07:48:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.433+0000 7fe6d7400700 1 -- 172.21.15.63:0/1323857522 learned_addr learned my addr 172.21.15.63:0/1323857522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.433+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/1323857522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 0x5594fb35a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.433+0000 7fe6d7400700 1 -- 172.21.15.63:0/1323857522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 msgr2=0x5594fb35a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.434+0000 7fe6d4c00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6bc0b2a70 0x7fe6bc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.434+0000 7fe6d7400700 1 --2- 172.21.15.63:0/1323857522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 0x5594fb35a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.434+0000 7fe6d7400700 1 -- 172.21.15.63:0/1323857522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb2c79f0 msgr2=0x5594fb35a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:01.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.434+0000 7fe6d7400700 1 --2- 172.21.15.63:0/1323857522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb2c79f0 0x5594fb35a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.434+0000 7fe6d7400700 1 -- 172.21.15.63:0/1323857522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594fb241420 con 0x5594fb35ae70 2024-05-27T07:48:01.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.434+0000 7fe6dce00700 1 --2- 172.21.15.63:0/1323857522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb2c79f0 0x5594fb35a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:48:01.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.434+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/1323857522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 0x5594fb35a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:01.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.434+0000 7fe6d4c00700 1 -- 172.21.15.63:0/2102461442 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fe6c80e2570 con 0x7fe6d8106560 2024-05-27T07:48:01.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.435+0000 7fe6d7400700 1 --2- 172.21.15.63:0/1323857522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb35ae70 0x5594fb367110 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8075b70 tx=0x7fe6c40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244f63719ed67bda server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:01.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.435+0000 7fe6dce00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6bc0b2a70 0x7fe6bc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.435+0000 7fe6cd600700 1 -- 172.21.15.63:0/1323857522 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40037c0 con 0x5594fb35ae70 2024-05-27T07:48:01.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.435+0000 7fe6cd600700 1 -- 172.21.15.63:0/1323857522 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c4003df0 con 0x5594fb35ae70 2024-05-27T07:48:01.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.435+0000 7fe6cd600700 1 -- 172.21.15.63:0/1323857522 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400ba30 con 0x5594fb35ae70 2024-05-27T07:48:01.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.437+0000 7fe6ed967b80 1 -- 172.21.15.63:0/1323857522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb35ae70 msgr2=0x5594fb367110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:01.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.437+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/1323857522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb35ae70 0x5594fb367110 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8075b70 tx=0x7fe6c40096a0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.437+0000 7fe6dce00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6bc0b2a70 0x7fe6bc0b4f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5594fb35cff0 tx=0x7fe6d0004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:01.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.437+0000 7fe6ed967b80 1 -- 172.21.15.63:0/1323857522 shutdown_connections 2024-05-27T07:48:01.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.437+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/1323857522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb35ae70 0x5594fb367110 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.437+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/1323857522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb2c79f0 0x5594fb35a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.437+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/1323857522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 0x5594fb35a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.437+0000 7fe6ed967b80 1 -- 172.21.15.63:0/1323857522 >> 172.21.15.63:0/1323857522 conn(0x5594fb275390 msgr2=0x5594fb3564c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:01.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.437+0000 7fe6ed967b80 1 -- 172.21.15.63:0/1323857522 shutdown_connections 2024-05-27T07:48:01.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.437+0000 7fe6ed967b80 1 -- 172.21.15.63:0/1323857522 wait complete. 2024-05-27T07:48:01.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.438+0000 7fe6ed967b80 1 -- start start 2024-05-27T07:48:01.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.438+0000 7fe6ed967b80 1 Processor -- start 2024-05-27T07:48:01.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.439+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 0x5594fb3645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.439+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb364b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.439+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb35ae70 0x5594fb365070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.439+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5594fb17efa0 con 0x5594fb2c79f0 2024-05-27T07:48:01.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.439+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594fb219760 con 0x5594fb1dbe40 2024-05-27T07:48:01.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.439+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5594fb367ac0 con 0x5594fb35ae70 2024-05-27T07:48:01.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.439+0000 7fe6ed967b80 10 client.0 _ll_register_callbacks cb 0x5594fb222c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:01.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.440+0000 7fe6d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 0x5594fb3645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.440+0000 7fe6d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 0x5594fb3645f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:50096/0 (socket says 172.21.15.63:50096) 2024-05-27T07:48:01.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.440+0000 7fe6d7e00700 1 -- 172.21.15.63:0/4165941426 learned_addr learned my addr 172.21.15.63:0/4165941426 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:01.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.440+0000 7fe6dce00700 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb35ae70 0x5594fb365070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.440+0000 7fe6d7400700 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb364b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.440+0000 7fe6d7e00700 1 -- 172.21.15.63:0/4165941426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb35ae70 msgr2=0x5594fb365070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:01.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.440+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb35ae70 0x5594fb365070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.440+0000 7fe6d7e00700 1 -- 172.21.15.63:0/4165941426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 msgr2=0x5594fb364b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:01.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.440+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb364b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.440+0000 7fe6d7e00700 1 -- 172.21.15.63:0/4165941426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594fb1c88d0 con 0x5594fb1dbe40 2024-05-27T07:48:01.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.441+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 0x5594fb3645f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80a9130 tx=0x7fe6c80109a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bfcc60bb06a244d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:01.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.441+0000 7fe6c2000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:48:01.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.441+0000 7fe6c2000700 1 -- 172.21.15.63:0/4165941426 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8020070 con 0x5594fb1dbe40 2024-05-27T07:48:01.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.442+0000 7fe6ed967b80 1 -- 172.21.15.63:0/4165941426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c4003df0 con 0x5594fb1dbe40 2024-05-27T07:48:01.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.442+0000 7fe6c2000700 1 -- 172.21.15.63:0/4165941426 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fe6c80a98a0 con 0x5594fb1dbe40 2024-05-27T07:48:01.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.442+0000 7fe6c2000700 1 -- 172.21.15.63:0/4165941426 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe6c80a9b30 con 0x5594fb1dbe40 2024-05-27T07:48:01.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.442+0000 7fe6ed967b80 10 client.37649 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:01.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.442+0000 7fe6ed967b80 1 -- 172.21.15.63:0/4165941426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5594fb249060 con 0x5594fb1dbe40 2024-05-27T07:48:01.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.443+0000 7fe6c2000700 1 -- 172.21.15.63:0/4165941426 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c4003df0 con 0x5594fb1dbe40 2024-05-27T07:48:01.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.443+0000 7fe6c2000700 1 -- 172.21.15.63:0/4165941426 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe6c80b1e00 con 0x5594fb1dbe40 2024-05-27T07:48:01.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.443+0000 7fe6ed967b80 10 client.37649 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:01.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.443+0000 7fe6ed967b80 10 client.37649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:01.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.443+0000 7fe6ed967b80 20 client.37649 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:01.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.443+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5594fb3748c0 0x5594fb376ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.443+0000 7fe6ed967b80 4 client.37649 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:01.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.443+0000 7fe6ed967b80 1 -- 172.21.15.63:0/4165941426 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594fb28ad90 con 0x5594fb3748c0 2024-05-27T07:48:01.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.444+0000 7fe6d7400700 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5594fb3748c0 0x5594fb376ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.445+0000 7fe6d7400700 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5594fb3748c0 0x5594fb376ce0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3908d3840186a8 server_cookie=39d0135fb010785a in_seq=0 out_seq=0 2024-05-27T07:48:01.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.445+0000 7fe6c2000700 10 client.37649 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:01.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6c2000700 1 -- 172.21.15.63:0/4165941426 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5594fb28ad90 con 0x5594fb3748c0 2024-05-27T07:48:01.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6c2000700 10 client.37649 handle_command_reply: tid=0 2024-05-27T07:48:01.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6ed967b80 1 client.37649 shutdown 2024-05-27T07:48:01.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6ed967b80 1 -- 172.21.15.63:0/4165941426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 msgr2=0x5594fb3645f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:01.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 0x5594fb3645f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80a9130 tx=0x7fe6c80109a0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6ed967b80 1 -- 172.21.15.63:0/4165941426 shutdown_connections 2024-05-27T07:48:01.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5594fb3748c0 0x5594fb376ce0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb35ae70 0x5594fb365070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 0x5594fb3645f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/4165941426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb364b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.447+0000 7fe6ed967b80 1 -- 172.21.15.63:0/4165941426 >> 172.21.15.63:0/4165941426 conn(0x5594fb275390 msgr2=0x5594fb35a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:01.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.448+0000 7fe6c2000700 0 client.37649 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:01.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.448+0000 7fe6ed967b80 1 -- 172.21.15.63:0/4165941426 shutdown_connections 2024-05-27T07:48:01.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.448+0000 7fe6ed967b80 1 -- 172.21.15.63:0/4165941426 wait complete. 2024-05-27T07:48:01.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.448+0000 7fe6ed967b80 20 client.37649 trim_cache size 0 max 16384 2024-05-27T07:48:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.478+0000 7fe6ed967b80 1 Processor -- start 2024-05-27T07:48:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.478+0000 7fe6ed967b80 1 -- start start 2024-05-27T07:48:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.479+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 0x5594fb363910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.479+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.479+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb35ae70 0x5594fb3564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.479+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b4eb0 con 0x5594fb2c79f0 2024-05-27T07:48:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.479+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b2740 con 0x5594fb1dbe40 2024-05-27T07:48:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.479+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b05b0 con 0x5594fb35ae70 2024-05-27T07:48:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.479+0000 7fe6d7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.479+0000 7fe6d7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44038/0 (socket says 172.21.15.63:44038) 2024-05-27T07:48:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.479+0000 7fe6d7400700 1 -- 172.21.15.63:0/845085498 learned_addr learned my addr 172.21.15.63:0/845085498 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:01.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.480+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/845085498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 0x5594fb363910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.480+0000 7fe6dce00700 1 --2- 172.21.15.63:0/845085498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb35ae70 0x5594fb3564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.480+0000 7fe6d7400700 1 -- 172.21.15.63:0/845085498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb35ae70 msgr2=0x5594fb3564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.480+0000 7fe6d7400700 1 --2- 172.21.15.63:0/845085498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb35ae70 0x5594fb3564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.480+0000 7fe6d7400700 1 -- 172.21.15.63:0/845085498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 msgr2=0x5594fb363910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.480+0000 7fe6d7400700 1 --2- 172.21.15.63:0/845085498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 0x5594fb363910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.480+0000 7fe6d7400700 1 -- 172.21.15.63:0/845085498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594fb352240 con 0x5594fb2c79f0 2024-05-27T07:48:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.481+0000 7fe6d7400700 1 --2- 172.21.15.63:0/845085498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35fdf0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4000c00 tx=0x7fe6c4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dce7320a2983f4d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.481+0000 7fe6ccc00700 1 -- 172.21.15.63:0/845085498 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40037c0 con 0x5594fb2c79f0 2024-05-27T07:48:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.481+0000 7fe6ccc00700 1 -- 172.21.15.63:0/845085498 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c4003df0 con 0x5594fb2c79f0 2024-05-27T07:48:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.481+0000 7fe6ccc00700 1 -- 172.21.15.63:0/845085498 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400be50 con 0x5594fb2c79f0 2024-05-27T07:48:01.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.482+0000 7fe6ed967b80 1 -- 172.21.15.63:0/845085498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 msgr2=0x5594fb35fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:01.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.482+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/845085498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35fdf0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4000c00 tx=0x7fe6c4005040 comp rx=0 tx=0).stop 2024-05-27T07:48:01.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.483+0000 7fe6ed967b80 1 -- 172.21.15.63:0/845085498 shutdown_connections 2024-05-27T07:48:01.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.483+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/845085498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb35ae70 0x5594fb3564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.483+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/845085498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb1dbe40 0x5594fb363910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.483+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/845085498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35fdf0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.483+0000 7fe6ed967b80 1 -- 172.21.15.63:0/845085498 >> 172.21.15.63:0/845085498 conn(0x5594fb275390 msgr2=0x5594fb365ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.483+0000 7fe6ed967b80 1 -- 172.21.15.63:0/845085498 shutdown_connections 2024-05-27T07:48:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.483+0000 7fe6ed967b80 1 -- 172.21.15.63:0/845085498 wait complete. 2024-05-27T07:48:01.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.484+0000 7fe6ed967b80 1 -- start start 2024-05-27T07:48:01.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.484+0000 7fe6ed967b80 1 Processor -- start 2024-05-27T07:48:01.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 0x5594fb36d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6ed967b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb35ae70 0x5594fb35f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b0c00 con 0x5594fb2c79f0 2024-05-27T07:48:01.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b2070 con 0x5594fb35ae70 2024-05-27T07:48:01.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6ed967b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b12d0 con 0x5594fb1dbe40 2024-05-27T07:48:01.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6ed967b80 10 client.0 _ll_register_callbacks cb 0x5594fb222c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:01.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6d7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6d7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44042/0 (socket says 172.21.15.63:44042) 2024-05-27T07:48:01.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6d7400700 1 -- 172.21.15.63:0/377465190 learned_addr learned my addr 172.21.15.63:0/377465190 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:01.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.485+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 0x5594fb36d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.486+0000 7fe6dce00700 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb35ae70 0x5594fb35f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.486+0000 7fe6d7400700 1 -- 172.21.15.63:0/377465190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 msgr2=0x5594fb36d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.486+0000 7fe6d7400700 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 0x5594fb36d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.486+0000 7fe6d7400700 1 -- 172.21.15.63:0/377465190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb35ae70 msgr2=0x5594fb35f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.486+0000 7fe6d7400700 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb35ae70 0x5594fb35f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.486+0000 7fe6d7400700 1 -- 172.21.15.63:0/377465190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594fb35f6a0 con 0x5594fb2c79f0 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.486+0000 7fe6dce00700 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb35ae70 0x5594fb35f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.486+0000 7fe6d7400700 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35eba0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009d40 tx=0x7fe6c4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b538cd86739a0375 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.486+0000 7fe6c2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.487+0000 7fe6c2000700 1 -- 172.21.15.63:0/377465190 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400bad0 con 0x5594fb2c79f0 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.487+0000 7fe6c2000700 1 -- 172.21.15.63:0/377465190 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9483+0+0 (secure 0 0 0) 0x7fe6c400bc70 con 0x5594fb2c79f0 2024-05-27T07:48:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.487+0000 7fe6ed967b80 1 -- 172.21.15.63:0/377465190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c4003df0 con 0x5594fb2c79f0 2024-05-27T07:48:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.487+0000 7fe6c2000700 1 -- 172.21.15.63:0/377465190 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe6c4047050 con 0x5594fb2c79f0 2024-05-27T07:48:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.487+0000 7fe6ed967b80 10 client.18234 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.487+0000 7fe6ed967b80 1 -- 172.21.15.63:0/377465190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5594fb36baa0 con 0x5594fb2c79f0 2024-05-27T07:48:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.487+0000 7fe6c2000700 1 -- 172.21.15.63:0/377465190 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6c4003df0 con 0x5594fb2c79f0 2024-05-27T07:48:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.488+0000 7fe6c2000700 1 -- 172.21.15.63:0/377465190 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe6c400f840 con 0x5594fb2c79f0 2024-05-27T07:48:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.488+0000 7fe6ed967b80 10 client.18234 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:01.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.488+0000 7fe6ed967b80 10 client.18234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:01.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.488+0000 7fe6ed967b80 20 client.18234 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:01.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.488+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5594fb3df350 0x5594fb3e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:01.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.488+0000 7fe6ed967b80 4 client.18234 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:48:01.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.488+0000 7fe6ed967b80 1 -- 172.21.15.63:0/377465190 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594fb284570 con 0x5594fb3df350 2024-05-27T07:48:01.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.489+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5594fb3df350 0x5594fb3e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:01.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.490+0000 7fe6c2000700 10 client.18234 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:01.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.490+0000 7fe6d7e00700 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5594fb3df350 0x5594fb3e1770 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb924dd6962252be server_cookie=b46b9d49b89b1ba6 in_seq=0 out_seq=0 2024-05-27T07:48:01.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.490+0000 7fe6c2000700 1 -- 172.21.15.63:0/377465190 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5594fb284570 con 0x5594fb3df350 2024-05-27T07:48:01.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.490+0000 7fe6c2000700 10 client.18234 handle_command_reply: tid=0 2024-05-27T07:48:01.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.490+0000 7fe6ed967b80 1 client.18234 shutdown 2024-05-27T07:48:01.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 1 -- 172.21.15.63:0/377465190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 msgr2=0x5594fb35eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:01.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35eba0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009d40 tx=0x7fe6c4009ee0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 1 -- 172.21.15.63:0/377465190 shutdown_connections 2024-05-27T07:48:01.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5594fb3df350 0x5594fb3e1770 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594fb35ae70 0x5594fb35f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5594fb1dbe40 0x5594fb36d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 1 --2- 172.21.15.63:0/377465190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5594fb2c79f0 0x5594fb35eba0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 1 -- 172.21.15.63:0/377465190 >> 172.21.15.63:0/377465190 conn(0x5594fb275390 msgr2=0x5594fb363670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:01.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6c2000700 0 client.18234 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:01.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 1 -- 172.21.15.63:0/377465190 shutdown_connections 2024-05-27T07:48:01.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 1 -- 172.21.15.63:0/377465190 wait complete. 2024-05-27T07:48:01.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.491+0000 7fe6ed967b80 20 client.18234 trim_cache size 0 max 16384 2024-05-27T07:48:01.493 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:48:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:48:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:48:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "960da17b-16d7-48c4-9c56-fc20ee47ba65": { 2024-05-27T07:48:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:48:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "960da17b-16d7-48c4-9c56-fc20ee47ba65", 2024-05-27T07:48:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:48:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:48:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:48:01.494 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:48:01.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.492+0000 7fe6dee00700 1 -- 172.21.15.63:0/2102461442 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6bc0b2a70 msgr2=0x7fe6bc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:01.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.492+0000 7fe6dee00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6bc0b2a70 0x7fe6bc0b4f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5594fb35cff0 tx=0x7fe6d0004ad0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.492+0000 7fe6dee00700 1 -- 172.21.15.63:0/2102461442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6d8106560 msgr2=0x7fe6d819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.492+0000 7fe6dee00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6d8106560 0x7fe6d819cf00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800d980 tx=0x7fe6c800dd40 comp rx=0 tx=0).stop 2024-05-27T07:48:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.492+0000 7fe6dee00700 1 -- 172.21.15.63:0/2102461442 shutdown_connections 2024-05-27T07:48:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.492+0000 7fe6dee00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6bc0b2a70 0x7fe6bc0b4f80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.492+0000 7fe6dee00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d810c5a0 0x7fe6d81a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.492+0000 7fe6dee00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6d8106560 0x7fe6d819cf00 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.492+0000 7fe6dee00700 1 --2- 172.21.15.63:0/2102461442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d8106eb0 0x7fe6d819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.492+0000 7fe6dee00700 1 -- 172.21.15.63:0/2102461442 >> 172.21.15.63:0/2102461442 conn(0x7fe6d8102040 msgr2=0x7fe6d8103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.493+0000 7fe6dee00700 1 -- 172.21.15.63:0/2102461442 shutdown_connections 2024-05-27T07:48:01.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:01.493+0000 7fe6dee00700 1 -- 172.21.15.63:0/2102461442 wait complete. 2024-05-27T07:48:01.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:01 smithi086 ceph-mon[34687]: pgmap v452: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T07:48:01.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:01 smithi149 ceph-mon[35408]: pgmap v452: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T07:48:02.322 INFO:tasks.cephfs.filesystem:scrub status for tag:960da17b-16d7-48c4-9c56-fc20ee47ba65 - {'path': '/', 'tag': '960da17b-16d7-48c4-9c56-fc20ee47ba65', 'options': 'recursive,force'} 2024-05-27T07:48:03.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:03 smithi086 ceph-mon[34687]: pgmap v453: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.5 MiB/s wr, 58 op/s 2024-05-27T07:48:03.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:03 smithi149 ceph-mon[35408]: pgmap v453: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.5 MiB/s wr, 58 op/s 2024-05-27T07:48:03.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:03 smithi063 ceph-mon[31613]: pgmap v453: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.5 MiB/s wr, 58 op/s 2024-05-27T07:48:05.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:05 smithi086 ceph-mon[34687]: pgmap v454: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.8 MiB/s wr, 54 op/s 2024-05-27T07:48:05.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:05 smithi149 ceph-mon[35408]: pgmap v454: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.8 MiB/s wr, 54 op/s 2024-05-27T07:48:05.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:05 smithi063 ceph-mon[31613]: pgmap v454: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.8 MiB/s wr, 54 op/s 2024-05-27T07:48:07.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:07 smithi086 ceph-mon[34687]: pgmap v455: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.8 MiB/s wr, 54 op/s 2024-05-27T07:48:07.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:48:07.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:48:07.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:48:07.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:07 smithi149 ceph-mon[35408]: pgmap v455: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.8 MiB/s wr, 54 op/s 2024-05-27T07:48:07.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:48:07.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:48:07.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:48:07.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:07 smithi063 ceph-mon[31613]: pgmap v455: 129 pgs: 129 active+clean; 3.7 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.8 MiB/s wr, 54 op/s 2024-05-27T07:48:07.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:48:07.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:48:07.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:48:08.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:08 smithi086 ceph-mon[34687]: osdmap e110: 12 total, 12 up, 12 in 2024-05-27T07:48:08.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:08 smithi149 ceph-mon[35408]: osdmap e110: 12 total, 12 up, 12 in 2024-05-27T07:48:08.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:08 smithi063 ceph-mon[31613]: osdmap e110: 12 total, 12 up, 12 in 2024-05-27T07:48:09.611 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:09 smithi086 ceph-mon[34687]: pgmap v457: 129 pgs: 129 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:48:09.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:09 smithi149 ceph-mon[35408]: pgmap v457: 129 pgs: 129 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:48:09.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:09 smithi063 ceph-mon[31613]: pgmap v457: 129 pgs: 129 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:48:11.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:11 smithi086 ceph-mon[34687]: pgmap v458: 129 pgs: 129 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:48:11.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:11 smithi149 ceph-mon[35408]: pgmap v458: 129 pgs: 129 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:48:11.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:11 smithi063 ceph-mon[31613]: pgmap v458: 129 pgs: 129 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:48:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.416+0000 7fbf3b000640 1 -- 172.21.15.63:0/988384195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf3410b610 msgr2=0x7fbf3410da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.416+0000 7fbf3b000640 1 --2- 172.21.15.63:0/988384195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf3410b610 0x7fbf3410da30 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009900 tx=0x7fbf2002a410 comp rx=0 tx=0).stop 2024-05-27T07:48:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.416+0000 7fbf3b000640 1 -- 172.21.15.63:0/988384195 shutdown_connections 2024-05-27T07:48:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.416+0000 7fbf3b000640 1 --2- 172.21.15.63:0/988384195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf3410e000 0x7fbf341104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.416+0000 7fbf3b000640 1 --2- 172.21.15.63:0/988384195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf3410b610 0x7fbf3410da30 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.416+0000 7fbf3b000640 1 --2- 172.21.15.63:0/988384195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf34104bb0 0x7fbf34104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.416+0000 7fbf3b000640 1 -- 172.21.15.63:0/988384195 >> 172.21.15.63:0/988384195 conn(0x7fbf340fe210 msgr2=0x7fbf34100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.417+0000 7fbf3b000640 1 -- 172.21.15.63:0/988384195 shutdown_connections 2024-05-27T07:48:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.417+0000 7fbf3b000640 1 -- 172.21.15.63:0/988384195 wait complete. 2024-05-27T07:48:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.418+0000 7fbf3b000640 1 Processor -- start 2024-05-27T07:48:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.418+0000 7fbf3b000640 1 -- start start 2024-05-27T07:48:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.418+0000 7fbf3b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf34104bb0 0x7fbf3419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.419+0000 7fbf3b000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf3410b610 0x7fbf3419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.419+0000 7fbf3b000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3410e000 0x7fbf341a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.419+0000 7fbf3b000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34113b60 con 0x7fbf3410b610 2024-05-27T07:48:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.419+0000 7fbf3b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341139e0 con 0x7fbf34104bb0 2024-05-27T07:48:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.419+0000 7fbf3b000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34113ce0 con 0x7fbf3410e000 2024-05-27T07:48:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.419+0000 7fbf39200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf3410b610 0x7fbf3419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.419+0000 7fbf39200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf3410b610 0x7fbf3419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48760/0 (socket says 172.21.15.63:48760) 2024-05-27T07:48:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.419+0000 7fbf39200640 1 -- 172.21.15.63:0/628320144 learned_addr learned my addr 172.21.15.63:0/628320144 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.419+0000 7fbf39c00640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf34104bb0 0x7fbf3419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.419+0000 7fbf3a600640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3410e000 0x7fbf341a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.420+0000 7fbf39200640 1 -- 172.21.15.63:0/628320144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3410e000 msgr2=0x7fbf341a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.420+0000 7fbf39200640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3410e000 0x7fbf341a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.420+0000 7fbf39200640 1 -- 172.21.15.63:0/628320144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf34104bb0 msgr2=0x7fbf3419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.420+0000 7fbf39200640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf34104bb0 0x7fbf3419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.420+0000 7fbf39200640 1 -- 172.21.15.63:0/628320144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf341a1660 con 0x7fbf3410b610 2024-05-27T07:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.420+0000 7fbf39200640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf3410b610 0x7fbf3419cd60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbf2002a930 tx=0x7fbf20003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.421+0000 7fbf32000640 1 -- 172.21.15.63:0/628320144 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2003f070 con 0x7fbf3410b610 2024-05-27T07:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.421+0000 7fbf32000640 1 -- 172.21.15.63:0/628320144 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf2003baf0 con 0x7fbf3410b610 2024-05-27T07:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.421+0000 7fbf3b000640 1 -- 172.21.15.63:0/628320144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf341a18f0 con 0x7fbf3410b610 2024-05-27T07:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.421+0000 7fbf32000640 1 -- 172.21.15.63:0/628320144 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20035030 con 0x7fbf3410b610 2024-05-27T07:48:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.421+0000 7fbf3b000640 1 -- 172.21.15.63:0/628320144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf341a1e30 con 0x7fbf3410b610 2024-05-27T07:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.425+0000 7fbf435fc740 1 Processor -- start 2024-05-27T07:48:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.425+0000 7fbf435fc740 1 -- start start 2024-05-27T07:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb824601b0 0x55bb82802a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb82802fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb8243f4d0 0x55bb828034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf435fc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bb82773b90 con 0x55bb8243f4d0 2024-05-27T07:48:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf435fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb82763730 con 0x55bb824601b0 2024-05-27T07:48:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf435fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bb82767770 con 0x55bb824372f0 2024-05-27T07:48:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf32000640 1 -- 172.21.15.63:0/628320144 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbf20042070 con 0x7fbf3410b610 2024-05-27T07:48:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf39200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb8243f4d0 0x55bb828034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf39c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb82802fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf3a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb824601b0 0x55bb82802a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf39200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb8243f4d0 0x55bb828034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48770/0 (socket says 172.21.15.63:48770) 2024-05-27T07:48:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.426+0000 7fbf39200640 1 -- 172.21.15.63:0/3077100519 learned_addr learned my addr 172.21.15.63:0/3077100519 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.427+0000 7fbf39200640 1 -- 172.21.15.63:0/3077100519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 msgr2=0x55bb82802fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.427+0000 7fbf39200640 1 --2- 172.21.15.63:0/3077100519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb82802fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.427+0000 7fbf39200640 1 -- 172.21.15.63:0/3077100519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb824601b0 msgr2=0x55bb82802a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.427+0000 7fbf39200640 1 --2- 172.21.15.63:0/3077100519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb824601b0 0x55bb82802a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.427+0000 7fbf39200640 1 -- 172.21.15.63:0/3077100519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb826fc8b0 con 0x55bb8243f4d0 2024-05-27T07:48:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.428+0000 7fbf39200640 1 --2- 172.21.15.63:0/3077100519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb8243f4d0 0x55bb828034e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbf201002f0 tx=0x7fbf2010b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d884852668613e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.428+0000 7fbf32000640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbf1c0b29d0 0x7fbf1c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.429+0000 7fbf32000640 1 -- 172.21.15.63:0/628320144 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9571+0+0 (secure 0 0 0) 0x7fbf20103050 con 0x7fbf3410b610 2024-05-27T07:48:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.429+0000 7fbf26a00640 1 -- 172.21.15.63:0/3077100519 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf200c5e80 con 0x55bb8243f4d0 2024-05-27T07:48:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.429+0000 7fbf3a600640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbf1c0b29d0 0x7fbf1c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.429+0000 7fbf26a00640 1 -- 172.21.15.63:0/3077100519 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf20110030 con 0x55bb8243f4d0 2024-05-27T07:48:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.429+0000 7fbf26a00640 1 -- 172.21.15.63:0/3077100519 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf200c9730 con 0x55bb8243f4d0 2024-05-27T07:48:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.430+0000 7fbf435fc740 1 -- 172.21.15.63:0/3077100519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb8243f4d0 msgr2=0x55bb828034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.430+0000 7fbf435fc740 1 --2- 172.21.15.63:0/3077100519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb8243f4d0 0x55bb828034e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbf201002f0 tx=0x7fbf2010b360 comp rx=0 tx=0).stop 2024-05-27T07:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.430+0000 7fbf3a600640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbf1c0b29d0 0x7fbf1c0b4ee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55bb827ffbd0 tx=0x7fbf2c0096a0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.430+0000 7fbf435fc740 1 -- 172.21.15.63:0/3077100519 shutdown_connections 2024-05-27T07:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.430+0000 7fbf435fc740 1 --2- 172.21.15.63:0/3077100519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb8243f4d0 0x55bb828034e0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.430+0000 7fbf435fc740 1 --2- 172.21.15.63:0/3077100519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb82802fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.430+0000 7fbf435fc740 1 --2- 172.21.15.63:0/3077100519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb824601b0 0x55bb82802a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.430+0000 7fbf435fc740 1 -- 172.21.15.63:0/3077100519 >> 172.21.15.63:0/3077100519 conn(0x55bb82454300 msgr2=0x55bb827ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.430+0000 7fbf435fc740 1 -- 172.21.15.63:0/3077100519 shutdown_connections 2024-05-27T07:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.430+0000 7fbf435fc740 1 -- 172.21.15.63:0/3077100519 wait complete. 2024-05-27T07:48:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.431+0000 7fbf435fc740 1 -- start start 2024-05-27T07:48:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.431+0000 7fbf435fc740 1 Processor -- start 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.432+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb8280cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.432+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb8280d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.432+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82818bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.432+0000 7fbf435fc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bb82449790 con 0x55bb824601b0 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.432+0000 7fbf435fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb8270bbc0 con 0x55bb8243f4d0 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.432+0000 7fbf435fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bb8280f950 con 0x55bb824372f0 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.432+0000 7fbf435fc740 10 client.0 _ll_register_callbacks cb 0x55bb826e9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.432+0000 7fbf3a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82818bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.432+0000 7fbf3a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82818bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48786/0 (socket says 172.21.15.63:48786) 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.432+0000 7fbf3a600640 1 -- 172.21.15.63:0/3284459729 learned_addr learned my addr 172.21.15.63:0/3284459729 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.433+0000 7fbf39200640 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb8280d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.433+0000 7fbf39c00640 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb8280cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.433+0000 7fbf3a600640 1 -- 172.21.15.63:0/3284459729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 msgr2=0x55bb8280cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.433+0000 7fbf3a600640 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb8280cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.433+0000 7fbf3a600640 1 -- 172.21.15.63:0/3284459729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 msgr2=0x55bb8280d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.433+0000 7fbf3a600640 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb8280d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.433+0000 7fbf3a600640 1 -- 172.21.15.63:0/3284459729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb82478210 con 0x55bb824601b0 2024-05-27T07:48:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.434+0000 7fbf3a600640 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82818bc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c010790 tx=0x7fbf2c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fa9ea1e3a22dd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.434+0000 7fbf1b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.434+0000 7fbf1b400640 1 -- 172.21.15.63:0/3284459729 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c004b20 con 0x55bb824601b0 2024-05-27T07:48:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.434+0000 7fbf1b400640 1 -- 172.21.15.63:0/3284459729 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9571+0+0 (secure 0 0 0) 0x7fbf2c004cc0 con 0x55bb824601b0 2024-05-27T07:48:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.434+0000 7fbf435fc740 1 -- 172.21.15.63:0/3284459729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf20110030 con 0x55bb824601b0 2024-05-27T07:48:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.434+0000 7fbf1b400640 1 -- 172.21.15.63:0/3284459729 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbf2c01f8b0 con 0x55bb824601b0 2024-05-27T07:48:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.435+0000 7fbf435fc740 10 client.18246 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.435+0000 7fbf435fc740 1 -- 172.21.15.63:0/3284459729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bb82544620 con 0x55bb824601b0 2024-05-27T07:48:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.435+0000 7fbf1b400640 1 -- 172.21.15.63:0/3284459729 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf20110030 con 0x55bb824601b0 2024-05-27T07:48:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.435+0000 7fbf1b400640 1 -- 172.21.15.63:0/3284459729 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbf2c026070 con 0x55bb824601b0 2024-05-27T07:48:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.435+0000 7fbf435fc740 10 client.18246 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.435+0000 7fbf435fc740 10 client.18246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.435+0000 7fbf435fc740 20 client.18246 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.435+0000 7fbf435fc740 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bb8241ed80 0x55bb8281ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.435+0000 7fbf435fc740 4 client.18246 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.435+0000 7fbf435fc740 1 -- 172.21.15.63:0/3284459729 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb82775ae0 con 0x55bb8241ed80 2024-05-27T07:48:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.436+0000 7fbf39c00640 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bb8241ed80 0x55bb8281ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.437+0000 7fbf39c00640 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bb8241ed80 0x55bb8281ec80 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9aca48a441ddf4e server_cookie=f7fe5a8f57e16c7f in_seq=0 out_seq=0 2024-05-27T07:48:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.437+0000 7fbf1b400640 10 client.18246 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf1b400640 1 -- 172.21.15.63:0/3284459729 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bb82775ae0 con 0x55bb8241ed80 2024-05-27T07:48:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf1b400640 10 client.18246 handle_command_reply: tid=0 2024-05-27T07:48:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf435fc740 1 client.18246 shutdown 2024-05-27T07:48:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf435fc740 1 -- 172.21.15.63:0/3284459729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 msgr2=0x55bb82818bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf435fc740 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82818bc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c010790 tx=0x7fbf2c004ab0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf435fc740 1 -- 172.21.15.63:0/3284459729 shutdown_connections 2024-05-27T07:48:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf435fc740 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bb8241ed80 0x55bb8281ec80 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf435fc740 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82818bc0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf435fc740 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb8280d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf435fc740 1 --2- 172.21.15.63:0/3284459729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb8280cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf1b400640 0 client.18246 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.439+0000 7fbf435fc740 1 -- 172.21.15.63:0/3284459729 >> 172.21.15.63:0/3284459729 conn(0x55bb82454300 msgr2=0x55bb82804780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.440+0000 7fbf435fc740 1 -- 172.21.15.63:0/3284459729 shutdown_connections 2024-05-27T07:48:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.440+0000 7fbf435fc740 1 -- 172.21.15.63:0/3284459729 wait complete. 2024-05-27T07:48:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.440+0000 7fbf435fc740 20 client.18246 trim_cache size 0 max 16384 2024-05-27T07:48:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.460+0000 7fbf435fc740 1 Processor -- start 2024-05-27T07:48:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.461+0000 7fbf435fc740 1 -- start start 2024-05-27T07:48:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.461+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824372f0 0x55bb828084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.461+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb82803330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.461+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824601b0 0x55bb8280eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.461+0000 7fbf435fc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341b5cb0 con 0x55bb824372f0 2024-05-27T07:48:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.461+0000 7fbf435fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341b3580 con 0x55bb8243f4d0 2024-05-27T07:48:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.461+0000 7fbf435fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341b13f0 con 0x55bb824601b0 2024-05-27T07:48:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.461+0000 7fbf39c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824372f0 0x55bb828084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.462+0000 7fbf39c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824372f0 0x55bb828084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48792/0 (socket says 172.21.15.63:48792) 2024-05-27T07:48:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.462+0000 7fbf39c00640 1 -- 172.21.15.63:0/1504081777 learned_addr learned my addr 172.21.15.63:0/1504081777 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.462+0000 7fbf39200640 1 --2- 172.21.15.63:0/1504081777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb82803330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.462+0000 7fbf3a600640 1 --2- 172.21.15.63:0/1504081777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824601b0 0x55bb8280eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.462+0000 7fbf39c00640 1 -- 172.21.15.63:0/1504081777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824601b0 msgr2=0x55bb8280eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.462+0000 7fbf39c00640 1 --2- 172.21.15.63:0/1504081777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824601b0 0x55bb8280eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.462+0000 7fbf39c00640 1 -- 172.21.15.63:0/1504081777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 msgr2=0x55bb82803330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.462+0000 7fbf39c00640 1 --2- 172.21.15.63:0/1504081777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb82803330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.462+0000 7fbf39c00640 1 -- 172.21.15.63:0/1504081777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb8280b1e0 con 0x55bb824372f0 2024-05-27T07:48:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.462+0000 7fbf39c00640 1 --2- 172.21.15.63:0/1504081777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824372f0 0x55bb828084e0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009dd0 tx=0x7fbf28005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10092f40d5d63de4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.463+0000 7fbf26000640 1 -- 172.21.15.63:0/1504081777 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf28004030 con 0x55bb824372f0 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.463+0000 7fbf26000640 1 -- 172.21.15.63:0/1504081777 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf28005020 con 0x55bb824372f0 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.463+0000 7fbf26000640 1 -- 172.21.15.63:0/1504081777 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf28004c90 con 0x55bb824372f0 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.464+0000 7fbf435fc740 1 -- 172.21.15.63:0/1504081777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824372f0 msgr2=0x55bb828084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.464+0000 7fbf435fc740 1 --2- 172.21.15.63:0/1504081777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824372f0 0x55bb828084e0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009dd0 tx=0x7fbf28005f90 comp rx=0 tx=0).stop 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.464+0000 7fbf435fc740 1 -- 172.21.15.63:0/1504081777 shutdown_connections 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.464+0000 7fbf435fc740 1 --2- 172.21.15.63:0/1504081777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824601b0 0x55bb8280eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.464+0000 7fbf435fc740 1 --2- 172.21.15.63:0/1504081777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb82803330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.464+0000 7fbf435fc740 1 --2- 172.21.15.63:0/1504081777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824372f0 0x55bb828084e0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.464+0000 7fbf435fc740 1 -- 172.21.15.63:0/1504081777 >> 172.21.15.63:0/1504081777 conn(0x55bb82454300 msgr2=0x55bb8280e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.464+0000 7fbf435fc740 1 -- 172.21.15.63:0/1504081777 shutdown_connections 2024-05-27T07:48:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.464+0000 7fbf435fc740 1 -- 172.21.15.63:0/1504081777 wait complete. 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.465+0000 7fbf435fc740 1 -- start start 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.465+0000 7fbf435fc740 1 Processor -- start 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.465+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb828151a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.465+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb828104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.465+0000 7fbf435fc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82810a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf435fc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341b1a40 con 0x55bb824601b0 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf435fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341b2eb0 con 0x55bb8243f4d0 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf435fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341b2110 con 0x55bb824372f0 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf435fc740 10 client.0 _ll_register_callbacks cb 0x7fbf2c009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf3a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82810a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf3a600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82810a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48806/0 (socket says 172.21.15.63:48806) 2024-05-27T07:48:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf3a600640 1 -- 172.21.15.63:0/2494753004 learned_addr learned my addr 172.21.15.63:0/2494753004 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf39c00640 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb828151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf39200640 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb828104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf3a600640 1 -- 172.21.15.63:0/2494753004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 msgr2=0x55bb828151a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.466+0000 7fbf3a600640 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb828151a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.467+0000 7fbf3a600640 1 -- 172.21.15.63:0/2494753004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 msgr2=0x55bb828104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.467+0000 7fbf3a600640 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb828104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.467+0000 7fbf3a600640 1 -- 172.21.15.63:0/2494753004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb82811070 con 0x55bb824601b0 2024-05-27T07:48:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.467+0000 7fbf3a600640 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82810a30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c000f80 tx=0x7fbf2c01f4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c8cc0f3a774e43 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.467+0000 7fbf1b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.468+0000 7fbf1b400640 1 -- 172.21.15.63:0/2494753004 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c03a060 con 0x55bb824601b0 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.468+0000 7fbf1b400640 1 -- 172.21.15.63:0/2494753004 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9571+0+0 (secure 0 0 0) 0x7fbf2c027620 con 0x55bb824601b0 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.468+0000 7fbf435fc740 1 -- 172.21.15.63:0/2494753004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf28005020 con 0x55bb824601b0 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.468+0000 7fbf1b400640 1 -- 172.21.15.63:0/2494753004 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbf2c041050 con 0x55bb824601b0 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.468+0000 7fbf435fc740 10 client.18252 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.468+0000 7fbf435fc740 1 -- 172.21.15.63:0/2494753004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bb82887160 con 0x55bb824601b0 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.468+0000 7fbf1b400640 1 -- 172.21.15.63:0/2494753004 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf28005020 con 0x55bb824601b0 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.469+0000 7fbf1b400640 1 -- 172.21.15.63:0/2494753004 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbf2c026070 con 0x55bb824601b0 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.469+0000 7fbf435fc740 10 client.18252 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.469+0000 7fbf435fc740 10 client.18252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.469+0000 7fbf435fc740 20 client.18252 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.469+0000 7fbf435fc740 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bb823f41c0 0x55bb82889860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.469+0000 7fbf435fc740 4 client.18252 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:48:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.469+0000 7fbf435fc740 1 -- 172.21.15.63:0/2494753004 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb8276f2c0 con 0x55bb823f41c0 2024-05-27T07:48:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.469+0000 7fbf39c00640 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bb823f41c0 0x55bb82889860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.470+0000 7fbf39c00640 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bb823f41c0 0x55bb82889860 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277976be90e07a5b server_cookie=375c84c458afaa63 in_seq=0 out_seq=0 2024-05-27T07:48:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.470+0000 7fbf1b400640 10 client.18252 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf1b400640 1 -- 172.21.15.63:0/2494753004 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bb8276f2c0 con 0x55bb823f41c0 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf1b400640 10 client.18252 handle_command_reply: tid=0 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 client.18252 shutdown 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 -- 172.21.15.63:0/2494753004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 msgr2=0x55bb82810a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82810a30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c000f80 tx=0x7fbf2c01f4f0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 -- 172.21.15.63:0/2494753004 shutdown_connections 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bb823f41c0 0x55bb82889860 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb824601b0 0x55bb82810a30 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb8243f4d0 0x55bb828104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 --2- 172.21.15.63:0/2494753004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bb824372f0 0x55bb828151a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 -- 172.21.15.63:0/2494753004 >> 172.21.15.63:0/2494753004 conn(0x55bb82454300 msgr2=0x55bb82803090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf1b400640 0 client.18252 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 -- 172.21.15.63:0/2494753004 shutdown_connections 2024-05-27T07:48:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.501+0000 7fbf435fc740 1 -- 172.21.15.63:0/2494753004 wait complete. 2024-05-27T07:48:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.502+0000 7fbf435fc740 20 client.18252 trim_cache size 0 max 16384 2024-05-27T07:48:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:48:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:48:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:48:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.502+0000 7fbf3b000640 1 -- 172.21.15.63:0/628320144 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbf1c0b29d0 msgr2=0x7fbf1c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.502+0000 7fbf3b000640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbf1c0b29d0 0x7fbf1c0b4ee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55bb827ffbd0 tx=0x7fbf2c0096a0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.502+0000 7fbf3b000640 1 -- 172.21.15.63:0/628320144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf3410b610 msgr2=0x7fbf3419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.502+0000 7fbf3b000640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf3410b610 0x7fbf3419cd60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbf2002a930 tx=0x7fbf20003b90 comp rx=0 tx=0).stop 2024-05-27T07:48:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.503+0000 7fbf3b000640 1 -- 172.21.15.63:0/628320144 shutdown_connections 2024-05-27T07:48:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.503+0000 7fbf3b000640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbf1c0b29d0 0x7fbf1c0b4ee0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.503+0000 7fbf3b000640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3410e000 0x7fbf341a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.503+0000 7fbf3b000640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf3410b610 0x7fbf3419cd60 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.503+0000 7fbf3b000640 1 --2- 172.21.15.63:0/628320144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf34104bb0 0x7fbf3419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.503+0000 7fbf3b000640 1 -- 172.21.15.63:0/628320144 >> 172.21.15.63:0/628320144 conn(0x7fbf340fe210 msgr2=0x7fbf34109db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.503+0000 7fbf3b000640 1 -- 172.21.15.63:0/628320144 shutdown_connections 2024-05-27T07:48:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:12.503+0000 7fbf3b000640 1 -- 172.21.15.63:0/628320144 wait complete. 2024-05-27T07:48:13.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:13 smithi086 ceph-mon[34687]: pgmap v459: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.3 MiB/s wr, 8 op/s 2024-05-27T07:48:13.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:13 smithi149 ceph-mon[35408]: pgmap v459: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.3 MiB/s wr, 8 op/s 2024-05-27T07:48:13.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:13 smithi063 ceph-mon[31613]: pgmap v459: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.3 MiB/s wr, 8 op/s 2024-05-27T07:48:15.708 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:15 smithi086 ceph-mon[34687]: pgmap v460: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.3 MiB/s wr, 8 op/s 2024-05-27T07:48:15.712 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:15 smithi149 ceph-mon[35408]: pgmap v460: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.3 MiB/s wr, 8 op/s 2024-05-27T07:48:15.715 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:15 smithi063 ceph-mon[31613]: pgmap v460: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.3 MiB/s wr, 8 op/s 2024-05-27T07:48:16.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:16 smithi086 ceph-mon[34687]: osdmap e111: 12 total, 12 up, 12 in 2024-05-27T07:48:16.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:16 smithi086 ceph-mon[34687]: pgmap v462: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-05-27T07:48:16.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:16 smithi149 ceph-mon[35408]: osdmap e111: 12 total, 12 up, 12 in 2024-05-27T07:48:16.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:16 smithi149 ceph-mon[35408]: pgmap v462: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-05-27T07:48:16.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:16 smithi063 ceph-mon[31613]: osdmap e111: 12 total, 12 up, 12 in 2024-05-27T07:48:16.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:16 smithi063 ceph-mon[31613]: pgmap v462: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 MiB/s wr, 4 op/s 2024-05-27T07:48:19.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:18 smithi086 ceph-mon[34687]: pgmap v463: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:19.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:18 smithi149 ceph-mon[35408]: pgmap v463: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:19.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:18 smithi063 ceph-mon[31613]: pgmap v463: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:21.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:20 smithi086 ceph-mon[34687]: pgmap v464: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:21.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:20 smithi149 ceph-mon[35408]: pgmap v464: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:21.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:20 smithi063 ceph-mon[31613]: pgmap v464: 129 pgs: 129 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:23.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:22 smithi086 ceph-mon[34687]: pgmap v465: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:23.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:22 smithi149 ceph-mon[35408]: pgmap v465: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:23.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:22 smithi063 ceph-mon[31613]: pgmap v465: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:25.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:24 smithi086 ceph-mon[34687]: pgmap v466: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:25.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:24 smithi149 ceph-mon[35408]: pgmap v466: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:25.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:24 smithi063 ceph-mon[31613]: pgmap v466: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:48:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.757+0000 7f452da00640 1 -- 172.21.15.63:0/3417952179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4528106600 msgr2=0x7f45281129d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.757+0000 7f452da00640 1 --2- 172.21.15.63:0/3417952179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4528106600 0x7f45281129d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f4520009ca0 tx=0x7f452002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.757+0000 7f452da00640 1 -- 172.21.15.63:0/3417952179 shutdown_connections 2024-05-27T07:48:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.757+0000 7f452da00640 1 --2- 172.21.15.63:0/3417952179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4528106600 0x7f45281129d0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.757+0000 7f452da00640 1 --2- 172.21.15.63:0/3417952179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810c350 0x7f4528105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.757+0000 7f452da00640 1 --2- 172.21.15.63:0/3417952179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f452810beb0 0x7f4528105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.757+0000 7f452da00640 1 -- 172.21.15.63:0/3417952179 >> 172.21.15.63:0/3417952179 conn(0x7f4528101530 msgr2=0x7f4528103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.757+0000 7f452da00640 1 -- 172.21.15.63:0/3417952179 shutdown_connections 2024-05-27T07:48:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.757+0000 7f452da00640 1 -- 172.21.15.63:0/3417952179 wait complete. 2024-05-27T07:48:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.759+0000 7f452da00640 1 Processor -- start 2024-05-27T07:48:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.759+0000 7f452da00640 1 -- start start 2024-05-27T07:48:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f452da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4528106600 0x7f45281a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f452da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f452810beb0 0x7f45281a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f452da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810c350 0x7f452819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f452da00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4528115fc0 con 0x7f4528106600 2024-05-27T07:48:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f452da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4528115e40 con 0x7f452810c350 2024-05-27T07:48:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f452da00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4528116140 con 0x7f452810beb0 2024-05-27T07:48:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f4526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4528106600 0x7f45281a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f4526000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f452810beb0 0x7f45281a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f4526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4528106600 0x7f45281a2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53324/0 (socket says 172.21.15.63:53324) 2024-05-27T07:48:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f4526a00640 1 -- 172.21.15.63:0/45581521 learned_addr learned my addr 172.21.15.63:0/45581521 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.760+0000 7f4527400640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810c350 0x7f452819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.761+0000 7f4526000640 1 -- 172.21.15.63:0/45581521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810c350 msgr2=0x7f452819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.761+0000 7f4526000640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810c350 0x7f452819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.761+0000 7f4526000640 1 -- 172.21.15.63:0/45581521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4528106600 msgr2=0x7f45281a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.761+0000 7f4526000640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4528106600 0x7f45281a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.761+0000 7f4526000640 1 -- 172.21.15.63:0/45581521 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452819ce80 con 0x7f452810beb0 2024-05-27T07:48:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.761+0000 7f4527400640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810c350 0x7f452819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.761+0000 7f4526000640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f452810beb0 0x7f45281a2ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f452806a420 tx=0x7f451400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.762+0000 7f451f400640 1 -- 172.21.15.63:0/45581521 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451400cc40 con 0x7f452810beb0 2024-05-27T07:48:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.762+0000 7f452da00640 1 -- 172.21.15.63:0/45581521 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452819d170 con 0x7f452810beb0 2024-05-27T07:48:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.762+0000 7f451f400640 1 -- 172.21.15.63:0/45581521 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45140044b0 con 0x7f452810beb0 2024-05-27T07:48:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.762+0000 7f451f400640 1 -- 172.21.15.63:0/45581521 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514003e10 con 0x7f452810beb0 2024-05-27T07:48:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.762+0000 7f452da00640 1 -- 172.21.15.63:0/45581521 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45281a93c0 con 0x7f452810beb0 2024-05-27T07:48:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.767+0000 7f4534a78740 1 Processor -- start 2024-05-27T07:48:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.767+0000 7f4534a78740 1 -- start start 2024-05-27T07:48:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.767+0000 7f4534a78740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecf4b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.767+0000 7f4534a78740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf4bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.767+0000 7f4534a78740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf4c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.767+0000 7f4534a78740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5586ecebcb90 con 0x5586ecb802f0 2024-05-27T07:48:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.767+0000 7f4534a78740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586eceac730 con 0x5586ecb884d0 2024-05-27T07:48:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.767+0000 7f4534a78740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5586eceb0770 con 0x5586ecba91b0 2024-05-27T07:48:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.768+0000 7f451f400640 1 -- 172.21.15.63:0/45581521 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4514003fb0 con 0x7f452810beb0 2024-05-27T07:48:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.768+0000 7f4526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf4bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.768+0000 7f4526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf4bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53332/0 (socket says 172.21.15.63:53332) 2024-05-27T07:48:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.768+0000 7f4526a00640 1 -- 172.21.15.63:0/3473778149 learned_addr learned my addr 172.21.15.63:0/3473778149 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.768+0000 7f4527400640 1 --2- 172.21.15.63:0/3473778149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecf4b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.768+0000 7f4526000640 1 --2- 172.21.15.63:0/3473778149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf4c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.769+0000 7f4526a00640 1 -- 172.21.15.63:0/3473778149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 msgr2=0x5586ecf4b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.769+0000 7f4526a00640 1 --2- 172.21.15.63:0/3473778149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecf4b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.769+0000 7f4526a00640 1 -- 172.21.15.63:0/3473778149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 msgr2=0x5586ecf4c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.769+0000 7f4526a00640 1 --2- 172.21.15.63:0/3473778149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf4c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.769+0000 7f4526a00640 1 -- 172.21.15.63:0/3473778149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ece458b0 con 0x5586ecb802f0 2024-05-27T07:48:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.769+0000 7f4526a00640 1 --2- 172.21.15.63:0/3473778149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf4bda0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f4518009d40 tx=0x7f4518011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfe001fc57ee744 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.769+0000 7f4513e00640 1 -- 172.21.15.63:0/3473778149 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451801e070 con 0x5586ecb802f0 2024-05-27T07:48:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.770+0000 7f4513e00640 1 -- 172.21.15.63:0/3473778149 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4518019ac0 con 0x5586ecb802f0 2024-05-27T07:48:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.770+0000 7f4513e00640 1 -- 172.21.15.63:0/3473778149 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451800e8f0 con 0x5586ecb802f0 2024-05-27T07:48:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.770+0000 7f451f400640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f450c0b29d0 0x7f450c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.770+0000 7f451f400640 1 -- 172.21.15.63:0/45581521 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f45140e2db0 con 0x7f452810beb0 2024-05-27T07:48:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.770+0000 7f4527400640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f450c0b29d0 0x7f450c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.771+0000 7f4534a78740 1 -- 172.21.15.63:0/3473778149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 msgr2=0x5586ecf4bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.771+0000 7f4534a78740 1 --2- 172.21.15.63:0/3473778149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf4bda0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f4518009d40 tx=0x7f4518011810 comp rx=0 tx=0).stop 2024-05-27T07:48:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.771+0000 7f4534a78740 1 -- 172.21.15.63:0/3473778149 shutdown_connections 2024-05-27T07:48:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.771+0000 7f4534a78740 1 --2- 172.21.15.63:0/3473778149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf4c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.771+0000 7f4534a78740 1 --2- 172.21.15.63:0/3473778149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf4bda0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.771+0000 7f4534a78740 1 --2- 172.21.15.63:0/3473778149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecf4b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.771+0000 7f4534a78740 1 -- 172.21.15.63:0/3473778149 >> 172.21.15.63:0/3473778149 conn(0x5586ecb9d300 msgr2=0x5586ecf48010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.771+0000 7f4534a78740 1 -- 172.21.15.63:0/3473778149 shutdown_connections 2024-05-27T07:48:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.771+0000 7f4527400640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f450c0b29d0 0x7f450c0b4ee0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5586ecf489d0 tx=0x7f4520009900 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.772+0000 7f4534a78740 1 -- 172.21.15.63:0/3473778149 wait complete. 2024-05-27T07:48:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.772+0000 7f4534a78740 1 -- start start 2024-05-27T07:48:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.772+0000 7f4534a78740 1 Processor -- start 2024-05-27T07:48:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.773+0000 7f4534a78740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf55a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.773+0000 7f4534a78740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf55fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.773+0000 7f4534a78740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecf61cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.773+0000 7f4534a78740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5586ecb92790 con 0x5586ecb802f0 2024-05-27T07:48:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.773+0000 7f4534a78740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586ece54bc0 con 0x5586ecb884d0 2024-05-27T07:48:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.773+0000 7f4534a78740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5586ecf58990 con 0x5586ecba91b0 2024-05-27T07:48:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.773+0000 7f4534a78740 10 client.0 _ll_register_callbacks cb 0x5586ece32400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.773+0000 7f4526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf55a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.773+0000 7f4527400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecf61cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.774+0000 7f4527400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecf61cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:59620/0 (socket says 172.21.15.63:59620) 2024-05-27T07:48:25.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.774+0000 7f4527400640 1 -- 172.21.15.63:0/2619873975 learned_addr learned my addr 172.21.15.63:0/2619873975 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.774+0000 7f4526000640 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf55fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.774+0000 7f4527400640 1 -- 172.21.15.63:0/2619873975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 msgr2=0x5586ecf55fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.774+0000 7f4527400640 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf55fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.774+0000 7f4527400640 1 -- 172.21.15.63:0/2619873975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 msgr2=0x5586ecf55a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.774+0000 7f4527400640 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf55a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.774+0000 7f4527400640 1 -- 172.21.15.63:0/2619873975 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ecbc1210 con 0x5586ecba91b0 2024-05-27T07:48:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.774+0000 7f4526a00640 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf55a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:48:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.775+0000 7f4527400640 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecf61cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4520003e70 tx=0x7f452002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96baf8213f2fc8d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.775+0000 7f4510c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.775+0000 7f4510c00640 1 -- 172.21.15.63:0/2619873975 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520031450 con 0x5586ecba91b0 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.775+0000 7f4534a78740 1 -- 172.21.15.63:0/2619873975 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4518019ac0 con 0x5586ecba91b0 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.775+0000 7f4510c00640 1 -- 172.21.15.63:0/2619873975 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f4520003f60 con 0x5586ecba91b0 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.776+0000 7f4510c00640 1 -- 172.21.15.63:0/2619873975 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4520004160 con 0x5586ecba91b0 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.776+0000 7f4534a78740 10 client.27550 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.776+0000 7f4534a78740 1 -- 172.21.15.63:0/2619873975 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5586ecc8d620 con 0x5586ecba91b0 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.776+0000 7f4510c00640 1 -- 172.21.15.63:0/2619873975 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4518019ac0 con 0x5586ecba91b0 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.776+0000 7f4510c00640 1 -- 172.21.15.63:0/2619873975 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4520051070 con 0x5586ecba91b0 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.777+0000 7f4534a78740 10 client.27550 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.777+0000 7f4534a78740 10 client.27550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.777+0000 7f4534a78740 20 client.27550 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.777+0000 7f4534a78740 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586ecb67d80 0x5586ecf67c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.777+0000 7f4534a78740 4 client.27550 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.777+0000 7f4534a78740 1 -- 172.21.15.63:0/2619873975 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586ecebeae0 con 0x5586ecb67d80 2024-05-27T07:48:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.777+0000 7f4526a00640 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586ecb67d80 0x5586ecf67c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.778+0000 7f4526a00640 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586ecb67d80 0x5586ecf67c60 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e4e0d095350965 server_cookie=f9a491fe604b4146 in_seq=0 out_seq=0 2024-05-27T07:48:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.778+0000 7f4510c00640 10 client.27550 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4510c00640 1 -- 172.21.15.63:0/2619873975 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5586ecebeae0 con 0x5586ecb67d80 2024-05-27T07:48:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4510c00640 10 client.27550 handle_command_reply: tid=0 2024-05-27T07:48:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4534a78740 1 client.27550 shutdown 2024-05-27T07:48:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4534a78740 1 -- 172.21.15.63:0/2619873975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 msgr2=0x5586ecf61cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4534a78740 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecf61cc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4520003e70 tx=0x7f452002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4534a78740 1 -- 172.21.15.63:0/2619873975 shutdown_connections 2024-05-27T07:48:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4534a78740 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586ecb67d80 0x5586ecf67c60 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4534a78740 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecf61cc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4534a78740 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4534a78740 1 --2- 172.21.15.63:0/2619873975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf55a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4510c00640 0 client.27550 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.780+0000 7f4534a78740 1 -- 172.21.15.63:0/2619873975 >> 172.21.15.63:0/2619873975 conn(0x5586ecb9d300 msgr2=0x5586ecf4d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.781+0000 7f4534a78740 1 -- 172.21.15.63:0/2619873975 shutdown_connections 2024-05-27T07:48:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.781+0000 7f4534a78740 1 -- 172.21.15.63:0/2619873975 wait complete. 2024-05-27T07:48:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.781+0000 7f4534a78740 20 client.27550 trim_cache size 0 max 16384 2024-05-27T07:48:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.807+0000 7f4534a78740 1 Processor -- start 2024-05-27T07:48:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.807+0000 7f4534a78740 1 -- start start 2024-05-27T07:48:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4534a78740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecb802f0 0x5586ecf4bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4534a78740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb884d0 0x5586ecf4c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4534a78740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecba91b0 0x5586ecf61a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4534a78740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b5cc0 con 0x5586ecb884d0 2024-05-27T07:48:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4534a78740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b3590 con 0x5586ecba91b0 2024-05-27T07:48:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4534a78740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b1400 con 0x5586ecb802f0 2024-05-27T07:48:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4526000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb884d0 0x5586ecf4c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4526000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb884d0 0x5586ecf4c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53346/0 (socket says 172.21.15.63:53346) 2024-05-27T07:48:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4526000640 1 -- 172.21.15.63:0/3128079441 learned_addr learned my addr 172.21.15.63:0/3128079441 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4526a00640 1 --2- 172.21.15.63:0/3128079441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecb802f0 0x5586ecf4bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.808+0000 7f4527400640 1 --2- 172.21.15.63:0/3128079441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecba91b0 0x5586ecf61a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.809+0000 7f4526000640 1 -- 172.21.15.63:0/3128079441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecb802f0 msgr2=0x5586ecf4bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.809+0000 7f4526000640 1 --2- 172.21.15.63:0/3128079441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecb802f0 0x5586ecf4bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.809+0000 7f4526000640 1 -- 172.21.15.63:0/3128079441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecba91b0 msgr2=0x5586ecf61a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.809+0000 7f4526000640 1 --2- 172.21.15.63:0/3128079441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecba91b0 0x5586ecf61a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.809+0000 7f4526000640 1 -- 172.21.15.63:0/3128079441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ecf42050 con 0x5586ecb884d0 2024-05-27T07:48:25.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.809+0000 7f4526000640 1 --2- 172.21.15.63:0/3128079441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb884d0 0x5586ecf4c240 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f45140a6f40 tx=0x7f45140e6900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f165cabcde31ed9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.809+0000 7f4513400640 1 -- 172.21.15.63:0/3128079441 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45140c1070 con 0x5586ecb884d0 2024-05-27T07:48:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.810+0000 7f4513400640 1 -- 172.21.15.63:0/3128079441 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45140a97e0 con 0x5586ecb884d0 2024-05-27T07:48:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.810+0000 7f4513400640 1 -- 172.21.15.63:0/3128079441 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45140ae080 con 0x5586ecb884d0 2024-05-27T07:48:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.810+0000 7f4534a78740 1 -- 172.21.15.63:0/3128079441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb884d0 msgr2=0x5586ecf4c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.810+0000 7f4534a78740 1 --2- 172.21.15.63:0/3128079441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb884d0 0x5586ecf4c240 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f45140a6f40 tx=0x7f45140e6900 comp rx=0 tx=0).stop 2024-05-27T07:48:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.811+0000 7f4534a78740 1 -- 172.21.15.63:0/3128079441 shutdown_connections 2024-05-27T07:48:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.811+0000 7f4534a78740 1 --2- 172.21.15.63:0/3128079441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecba91b0 0x5586ecf61a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.811+0000 7f4534a78740 1 --2- 172.21.15.63:0/3128079441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb884d0 0x5586ecf4c240 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.811+0000 7f4534a78740 1 --2- 172.21.15.63:0/3128079441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecb802f0 0x5586ecf4bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.811+0000 7f4534a78740 1 -- 172.21.15.63:0/3128079441 >> 172.21.15.63:0/3128079441 conn(0x5586ecb9d300 msgr2=0x5586ecf4b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.811+0000 7f4534a78740 1 -- 172.21.15.63:0/3128079441 shutdown_connections 2024-05-27T07:48:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.811+0000 7f4534a78740 1 -- 172.21.15.63:0/3128079441 wait complete. 2024-05-27T07:48:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.811+0000 7f4534a78740 1 -- start start 2024-05-27T07:48:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.812+0000 7f4534a78740 1 Processor -- start 2024-05-27T07:48:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.812+0000 7f4534a78740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf59240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.812+0000 7f4534a78740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf59780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.812+0000 7f4534a78740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecfce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.812+0000 7f4534a78740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b1a50 con 0x5586ecb802f0 2024-05-27T07:48:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.812+0000 7f4534a78740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b2ec0 con 0x5586ecb884d0 2024-05-27T07:48:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.812+0000 7f4534a78740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45281b2120 con 0x5586ecba91b0 2024-05-27T07:48:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.812+0000 7f4534a78740 10 client.0 _ll_register_callbacks cb 0x7f4518005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf59240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf59240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53362/0 (socket says 172.21.15.63:53362) 2024-05-27T07:48:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4526000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf59780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4526a00640 1 -- 172.21.15.63:0/3295065835 learned_addr learned my addr 172.21.15.63:0/3295065835 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4527400640 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecfce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4526a00640 1 -- 172.21.15.63:0/3295065835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 msgr2=0x5586ecfce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4526a00640 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecfce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4526a00640 1 -- 172.21.15.63:0/3295065835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 msgr2=0x5586ecf59780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4526a00640 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf59780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4526a00640 1 -- 172.21.15.63:0/3295065835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ecfce860 con 0x5586ecb802f0 2024-05-27T07:48:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.813+0000 7f4527400640 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecfce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:48:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.814+0000 7f4526a00640 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf59240 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4518009ad0 tx=0x7f4518009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4c40c1272ed1b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.814+0000 7f4510c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.814+0000 7f4510c00640 1 -- 172.21.15.63:0/3295065835 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451801e040 con 0x5586ecb802f0 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.814+0000 7f4510c00640 1 -- 172.21.15.63:0/3295065835 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f45180196b0 con 0x5586ecb802f0 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.814+0000 7f4534a78740 1 -- 172.21.15.63:0/3295065835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45140a97e0 con 0x5586ecb802f0 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4510c00640 1 -- 172.21.15.63:0/3295065835 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f451803d050 con 0x5586ecb802f0 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4534a78740 10 client.18270 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4534a78740 1 -- 172.21.15.63:0/3295065835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5586ecfcfac0 con 0x5586ecb802f0 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4510c00640 1 -- 172.21.15.63:0/3295065835 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45140a97e0 con 0x5586ecb802f0 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4510c00640 1 -- 172.21.15.63:0/3295065835 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f45180198b0 con 0x5586ecb802f0 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4534a78740 10 client.18270 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4534a78740 10 client.18270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4534a78740 20 client.18270 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4534a78740 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586ecb3d1c0 0x5586ecfd2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4534a78740 4 client.18270 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:48:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.815+0000 7f4534a78740 1 -- 172.21.15.63:0/3295065835 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5586eceb82c0 con 0x5586ecb3d1c0 2024-05-27T07:48:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.816+0000 7f4526000640 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586ecb3d1c0 0x5586ecfd2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.816+0000 7f4510c00640 10 client.18270 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.816+0000 7f4526000640 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586ecb3d1c0 0x5586ecfd2140 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f48b131fdb092aa server_cookie=31b76680b87818f0 in_seq=0 out_seq=0 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.834+0000 7f4510c00640 1 -- 172.21.15.63:0/3295065835 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5586eceb82c0 con 0x5586ecb3d1c0 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.834+0000 7f4510c00640 10 client.18270 handle_command_reply: tid=0 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4534a78740 1 client.18270 shutdown 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4534a78740 1 -- 172.21.15.63:0/3295065835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 msgr2=0x5586ecf59240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4534a78740 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf59240 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4518009ad0 tx=0x7f4518009750 comp rx=0 tx=0).stop 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4534a78740 1 -- 172.21.15.63:0/3295065835 shutdown_connections 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4534a78740 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586ecb3d1c0 0x5586ecfd2140 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4534a78740 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586ecba91b0 0x5586ecfce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4534a78740 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586ecb884d0 0x5586ecf59780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4534a78740 1 --2- 172.21.15.63:0/3295065835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586ecb802f0 0x5586ecf59240 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4534a78740 1 -- 172.21.15.63:0/3295065835 >> 172.21.15.63:0/3295065835 conn(0x5586ecb9d300 msgr2=0x5586ecf4c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4510c00640 0 client.18270 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.835+0000 7f4534a78740 1 -- 172.21.15.63:0/3295065835 shutdown_connections 2024-05-27T07:48:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.836+0000 7f4534a78740 1 -- 172.21.15.63:0/3295065835 wait complete. 2024-05-27T07:48:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.836+0000 7f4534a78740 20 client.18270 trim_cache size 0 max 16384 2024-05-27T07:48:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:48:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:48:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:48:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 -- 172.21.15.63:0/45581521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f450c0b29d0 msgr2=0x7f450c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f450c0b29d0 0x7f450c0b4ee0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5586ecf489d0 tx=0x7f4520009900 comp rx=0 tx=0).stop 2024-05-27T07:48:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 -- 172.21.15.63:0/45581521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f452810beb0 msgr2=0x7f45281a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f452810beb0 0x7f45281a2ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f452806a420 tx=0x7f451400ee50 comp rx=0 tx=0).stop 2024-05-27T07:48:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 -- 172.21.15.63:0/45581521 shutdown_connections 2024-05-27T07:48:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f450c0b29d0 0x7f450c0b4ee0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f452810c350 0x7f452819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f452810beb0 0x7f45281a2ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 --2- 172.21.15.63:0/45581521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4528106600 0x7f45281a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 -- 172.21.15.63:0/45581521 >> 172.21.15.63:0/45581521 conn(0x7f4528101530 msgr2=0x7f4528102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 -- 172.21.15.63:0/45581521 shutdown_connections 2024-05-27T07:48:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:25.837+0000 7f452da00640 1 -- 172.21.15.63:0/45581521 wait complete. 2024-05-27T07:48:26.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:25 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:48:26.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:25 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:48:26.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:25 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:48:27.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:26 smithi086 ceph-mon[34687]: pgmap v467: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:48:27.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:26 smithi149 ceph-mon[35408]: pgmap v467: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:48:27.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:26 smithi063 ceph-mon[31613]: pgmap v467: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:48:28.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:27 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:48:28.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:27 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:48:28.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:27 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:48:28.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:27 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:48:28.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:27 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:48:28.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:27 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:48:28.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:27 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:48:28.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:27 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:48:28.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:27 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:48:29.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:29 smithi086 ceph-mon[34687]: pgmap v468: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:48:29.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:29 smithi149 ceph-mon[35408]: pgmap v468: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:48:29.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:29 smithi063 ceph-mon[31613]: pgmap v468: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:48:31.304 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:31 smithi063 ceph-mon[31613]: pgmap v469: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:48:31.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:31 smithi086 ceph-mon[34687]: pgmap v469: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:48:31.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:31 smithi149 ceph-mon[35408]: pgmap v469: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:48:32.323 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:48:32.868 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:33.277 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:33 smithi063 ceph-mon[31613]: pgmap v470: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:48:33.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:33 smithi086 ceph-mon[34687]: pgmap v470: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:48:33.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:33 smithi149 ceph-mon[35408]: pgmap v470: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:48:35.291 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:35 smithi063 ceph-mon[31613]: pgmap v471: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:48:35.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:35 smithi086 ceph-mon[34687]: pgmap v471: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:48:35.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:35 smithi149 ceph-mon[35408]: pgmap v471: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:48:35.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.467+0000 7eff45c00700 1 -- 172.21.15.63:0/2213106892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff40108d20 msgr2=0x7eff40109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:35.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.467+0000 7eff45c00700 1 --2- 172.21.15.63:0/2213106892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff40108d20 0x7eff40109130 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0099c0 tx=0x7eff2c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.467+0000 7eff45c00700 1 -- 172.21.15.63:0/2213106892 shutdown_connections 2024-05-27T07:48:35.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.467+0000 7eff45c00700 1 --2- 172.21.15.63:0/2213106892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff40109670 0x7eff40110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.467+0000 7eff45c00700 1 --2- 172.21.15.63:0/2213106892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff401043f0 0x7eff40104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.467+0000 7eff45c00700 1 --2- 172.21.15.63:0/2213106892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff40108d20 0x7eff40109130 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.467+0000 7eff45c00700 1 -- 172.21.15.63:0/2213106892 >> 172.21.15.63:0/2213106892 conn(0x7eff40079d00 msgr2=0x7eff4007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:35.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.467+0000 7eff45c00700 1 -- 172.21.15.63:0/2213106892 shutdown_connections 2024-05-27T07:48:35.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.467+0000 7eff45c00700 1 -- 172.21.15.63:0/2213106892 wait complete. 2024-05-27T07:48:35.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.468+0000 7eff45c00700 1 Processor -- start 2024-05-27T07:48:35.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.469+0000 7eff45c00700 1 -- start start 2024-05-27T07:48:35.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.469+0000 7eff45c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff401043f0 0x7eff40075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.469+0000 7eff45c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff40108d20 0x7eff40075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.469+0000 7eff45c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff40109670 0x7eff40076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.469+0000 7eff45c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff40114130 con 0x7eff40108d20 2024-05-27T07:48:35.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.469+0000 7eff45c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff40113fb0 con 0x7eff40109670 2024-05-27T07:48:35.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.469+0000 7eff45c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7eff401142b0 con 0x7eff401043f0 2024-05-27T07:48:35.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.469+0000 7eff3f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff40108d20 0x7eff40075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.470+0000 7eff3f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff40108d20 0x7eff40075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50290/0 (socket says 172.21.15.63:50290) 2024-05-27T07:48:35.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.470+0000 7eff3f400700 1 -- 172.21.15.63:0/3481070090 learned_addr learned my addr 172.21.15.63:0/3481070090 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:35.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.470+0000 7eff3fe00700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff401043f0 0x7eff40075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.470+0000 7eff45200700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff40109670 0x7eff40076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.470+0000 7eff3f400700 1 -- 172.21.15.63:0/3481070090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff401043f0 msgr2=0x7eff40075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:35.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.470+0000 7eff3f400700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff401043f0 0x7eff40075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.470+0000 7eff3f400700 1 -- 172.21.15.63:0/3481070090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff40109670 msgr2=0x7eff40076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.470+0000 7eff3f400700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff40109670 0x7eff40076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.470+0000 7eff3f400700 1 -- 172.21.15.63:0/3481070090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff401a8fc0 con 0x7eff40108d20 2024-05-27T07:48:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.470+0000 7eff45200700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff40109670 0x7eff40076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.471+0000 7eff3f400700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff40108d20 0x7eff40075ff0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00b580 tx=0x7eff2c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.471+0000 7eff3cc00700 1 -- 172.21.15.63:0/3481070090 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c026070 con 0x7eff40108d20 2024-05-27T07:48:35.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.471+0000 7eff3cc00700 1 -- 172.21.15.63:0/3481070090 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff2c005030 con 0x7eff40108d20 2024-05-27T07:48:35.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.471+0000 7eff45c00700 1 -- 172.21.15.63:0/3481070090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff401a9250 con 0x7eff40108d20 2024-05-27T07:48:35.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.471+0000 7eff3cc00700 1 -- 172.21.15.63:0/3481070090 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c0049f0 con 0x7eff40108d20 2024-05-27T07:48:35.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.472+0000 7eff45c00700 1 -- 172.21.15.63:0/3481070090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff401a9700 con 0x7eff40108d20 2024-05-27T07:48:35.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.476+0000 7eff3cc00700 1 -- 172.21.15.63:0/3481070090 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7eff2c029070 con 0x7eff40108d20 2024-05-27T07:48:35.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.477+0000 7eff55decb80 1 Processor -- start 2024-05-27T07:48:35.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.477+0000 7eff55decb80 1 -- start start 2024-05-27T07:48:35.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.477+0000 7eff55decb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 0x5599cbea6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.477+0000 7eff55decb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbeb1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.478+0000 7eff55decb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7820 0x5599cbeb4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.478+0000 7eff55decb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5599cbdd5e40 con 0x5599cbea7410 2024-05-27T07:48:35.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.478+0000 7eff55decb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599cbdc59e0 con 0x5599cbd28e40 2024-05-27T07:48:35.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.478+0000 7eff55decb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5599cbdc9a20 con 0x5599cbea7820 2024-05-27T07:48:35.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.478+0000 7eff3fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbeb1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.478+0000 7eff3fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbeb1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50302/0 (socket says 172.21.15.63:50302) 2024-05-27T07:48:35.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.478+0000 7eff3fe00700 1 -- 172.21.15.63:0/1756278648 learned_addr learned my addr 172.21.15.63:0/1756278648 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:35.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.478+0000 7eff45200700 1 --2- 172.21.15.63:0/1756278648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 0x5599cbea6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.478+0000 7eff3f400700 1 --2- 172.21.15.63:0/1756278648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7820 0x5599cbeb4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.479+0000 7eff3cc00700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff240b2a70 0x7eff240b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.479+0000 7eff3fe00700 1 -- 172.21.15.63:0/1756278648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7820 msgr2=0x5599cbeb4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:35.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.479+0000 7eff3fe00700 1 --2- 172.21.15.63:0/1756278648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7820 0x5599cbeb4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.479+0000 7eff3fe00700 1 -- 172.21.15.63:0/1756278648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 msgr2=0x5599cbea6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:35.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.479+0000 7eff3fe00700 1 --2- 172.21.15.63:0/1756278648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 0x5599cbea6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.479+0000 7eff3fe00700 1 -- 172.21.15.63:0/1756278648 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cbd8e420 con 0x5599cbea7410 2024-05-27T07:48:35.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.479+0000 7eff3cc00700 1 -- 172.21.15.63:0/3481070090 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7eff2c0e3380 con 0x7eff40108d20 2024-05-27T07:48:35.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.480+0000 7eff3fe00700 1 --2- 172.21.15.63:0/1756278648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbeb1ad0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7eff30005e20 tx=0x7eff3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88071752f2fe8e11 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:35.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.480+0000 7eff45200700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff240b2a70 0x7eff240b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.480+0000 7eff35600700 1 -- 172.21.15.63:0/1756278648 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3000ed70 con 0x5599cbea7410 2024-05-27T07:48:35.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.480+0000 7eff35600700 1 -- 172.21.15.63:0/1756278648 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff300108c0 con 0x5599cbea7410 2024-05-27T07:48:35.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.480+0000 7eff35600700 1 -- 172.21.15.63:0/1756278648 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3001a830 con 0x5599cbea7410 2024-05-27T07:48:35.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.481+0000 7eff55decb80 1 -- 172.21.15.63:0/1756278648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 msgr2=0x5599cbeb1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:35.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.481+0000 7eff55decb80 1 --2- 172.21.15.63:0/1756278648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbeb1ad0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7eff30005e20 tx=0x7eff3000cf90 comp rx=0 tx=0).stop 2024-05-27T07:48:35.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.481+0000 7eff45200700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff240b2a70 0x7eff240b4f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5599cbea3690 tx=0x7eff38004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:35.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.482+0000 7eff55decb80 1 -- 172.21.15.63:0/1756278648 shutdown_connections 2024-05-27T07:48:35.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.482+0000 7eff55decb80 1 --2- 172.21.15.63:0/1756278648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7820 0x5599cbeb4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.482+0000 7eff55decb80 1 --2- 172.21.15.63:0/1756278648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 0x5599cbea6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.482+0000 7eff55decb80 1 --2- 172.21.15.63:0/1756278648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbeb1ad0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.482+0000 7eff55decb80 1 -- 172.21.15.63:0/1756278648 >> 172.21.15.63:0/1756278648 conn(0x5599cbd88fc0 msgr2=0x5599cbea2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:35.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.482+0000 7eff55decb80 1 -- 172.21.15.63:0/1756278648 shutdown_connections 2024-05-27T07:48:35.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.482+0000 7eff55decb80 1 -- 172.21.15.63:0/1756278648 wait complete. 2024-05-27T07:48:35.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.483+0000 7eff55decb80 1 -- start start 2024-05-27T07:48:35.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.483+0000 7eff55decb80 1 Processor -- start 2024-05-27T07:48:35.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff55decb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbd28e40 0x5599cbeb1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff55decb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7410 0x5599cbeb1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff55decb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7820 0x5599cbeb1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff55decb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5599cbccbfa0 con 0x5599cbd28e40 2024-05-27T07:48:35.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff55decb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599cbd66760 con 0x5599cbea7820 2024-05-27T07:48:35.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff55decb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5599cbeb4990 con 0x5599cbea7410 2024-05-27T07:48:35.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff55decb80 10 client.0 _ll_register_callbacks cb 0x5599cbd6fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:35.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff3fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbd28e40 0x5599cbeb1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff3fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbd28e40 0x5599cbeb1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50314/0 (socket says 172.21.15.63:50314) 2024-05-27T07:48:35.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff3fe00700 1 -- 172.21.15.63:0/3914085018 learned_addr learned my addr 172.21.15.63:0/3914085018 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:35.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.484+0000 7eff3f400700 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7410 0x5599cbeb1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.485+0000 7eff45200700 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7820 0x5599cbeb1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.485+0000 7eff3fe00700 1 -- 172.21.15.63:0/3914085018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7410 msgr2=0x5599cbeb1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:35.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.485+0000 7eff3fe00700 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7410 0x5599cbeb1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.485+0000 7eff3fe00700 1 -- 172.21.15.63:0/3914085018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7820 msgr2=0x5599cbeb1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:35.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.485+0000 7eff3fe00700 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7820 0x5599cbeb1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.485+0000 7eff3fe00700 1 -- 172.21.15.63:0/3914085018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cbd158d0 con 0x5599cbd28e40 2024-05-27T07:48:35.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.485+0000 7eff3f400700 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7410 0x5599cbeb1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:35.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.485+0000 7eff45200700 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7820 0x5599cbeb1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:35.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.486+0000 7eff3fe00700 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbd28e40 0x5599cbeb1540 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7eff3000eda0 tx=0x7eff3000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af39ba7fb121e848 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:35.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.486+0000 7eff2a000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:35.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.486+0000 7eff2a000700 1 -- 172.21.15.63:0/3914085018 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30027060 con 0x5599cbd28e40 2024-05-27T07:48:35.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.486+0000 7eff2a000700 1 -- 172.21.15.63:0/3914085018 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7eff30005030 con 0x5599cbd28e40 2024-05-27T07:48:35.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.486+0000 7eff55decb80 1 -- 172.21.15.63:0/3914085018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff300108c0 con 0x5599cbd28e40 2024-05-27T07:48:35.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.486+0000 7eff2a000700 1 -- 172.21.15.63:0/3914085018 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7eff30005230 con 0x5599cbd28e40 2024-05-27T07:48:35.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.487+0000 7eff55decb80 10 client.18282 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:35.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.487+0000 7eff55decb80 1 -- 172.21.15.63:0/3914085018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5599cbd96060 con 0x5599cbd28e40 2024-05-27T07:48:35.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.487+0000 7eff2a000700 1 -- 172.21.15.63:0/3914085018 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff300108c0 con 0x5599cbd28e40 2024-05-27T07:48:35.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.487+0000 7eff2a000700 1 -- 172.21.15.63:0/3914085018 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7eff30004870 con 0x5599cbd28e40 2024-05-27T07:48:35.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.487+0000 7eff55decb80 10 client.18282 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:35.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.487+0000 7eff55decb80 10 client.18282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:35.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.487+0000 7eff55decb80 20 client.18282 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:35.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.487+0000 7eff55decb80 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599cbec1840 0x5599cbec3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.487+0000 7eff55decb80 4 client.18282 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:35.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.487+0000 7eff55decb80 1 -- 172.21.15.63:0/3914085018 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599cbdd7d90 con 0x5599cbec1840 2024-05-27T07:48:35.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.488+0000 7eff3f400700 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599cbec1840 0x5599cbec3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.489+0000 7eff3f400700 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599cbec1840 0x5599cbec3c60 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e181c00694eb6f1 server_cookie=d97849f76aa2779a in_seq=0 out_seq=0 2024-05-27T07:48:35.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.489+0000 7eff2a000700 10 client.18282 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:35.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.491+0000 7eff2a000700 1 -- 172.21.15.63:0/3914085018 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5599cbdd7d90 con 0x5599cbec1840 2024-05-27T07:48:35.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.491+0000 7eff2a000700 10 client.18282 handle_command_reply: tid=0 2024-05-27T07:48:35.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.491+0000 7eff55decb80 1 client.18282 shutdown 2024-05-27T07:48:35.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.491+0000 7eff55decb80 1 -- 172.21.15.63:0/3914085018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbd28e40 msgr2=0x5599cbeb1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:35.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.491+0000 7eff55decb80 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbd28e40 0x5599cbeb1540 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7eff3000eda0 tx=0x7eff3000edd0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.492+0000 7eff55decb80 1 -- 172.21.15.63:0/3914085018 shutdown_connections 2024-05-27T07:48:35.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.492+0000 7eff55decb80 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599cbec1840 0x5599cbec3c60 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.492+0000 7eff55decb80 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7820 0x5599cbeb1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.492+0000 7eff55decb80 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbd28e40 0x5599cbeb1540 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.492+0000 7eff55decb80 1 --2- 172.21.15.63:0/3914085018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7410 0x5599cbeb1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.492+0000 7eff55decb80 1 -- 172.21.15.63:0/3914085018 >> 172.21.15.63:0/3914085018 conn(0x5599cbd88fc0 msgr2=0x5599cbea1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:35.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.492+0000 7eff2a000700 0 client.18282 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:35.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.492+0000 7eff55decb80 1 -- 172.21.15.63:0/3914085018 shutdown_connections 2024-05-27T07:48:35.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.492+0000 7eff55decb80 1 -- 172.21.15.63:0/3914085018 wait complete. 2024-05-27T07:48:35.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.492+0000 7eff55decb80 20 client.18282 trim_cache size 0 max 16384 2024-05-27T07:48:35.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.516+0000 7eff55decb80 1 Processor -- start 2024-05-27T07:48:35.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.516+0000 7eff55decb80 1 -- start start 2024-05-27T07:48:35.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.516+0000 7eff55decb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 0x5599cbeacd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.516+0000 7eff55decb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbebdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.516+0000 7eff55decb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7820 0x5599cbeb04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.516+0000 7eff55decb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff401b4de0 con 0x5599cbea7410 2024-05-27T07:48:35.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.516+0000 7eff55decb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff401b2670 con 0x5599cbd28e40 2024-05-27T07:48:35.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.516+0000 7eff55decb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7eff401b04e0 con 0x5599cbea7820 2024-05-27T07:48:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.516+0000 7eff3f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbebdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.517+0000 7eff3f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbebdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50326/0 (socket says 172.21.15.63:50326) 2024-05-27T07:48:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.517+0000 7eff3f400700 1 -- 172.21.15.63:0/171368880 learned_addr learned my addr 172.21.15.63:0/171368880 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:35.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.517+0000 7eff3fe00700 1 --2- 172.21.15.63:0/171368880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 0x5599cbeacd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.517+0000 7eff45200700 1 --2- 172.21.15.63:0/171368880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7820 0x5599cbeb04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.517+0000 7eff3f400700 1 -- 172.21.15.63:0/171368880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7820 msgr2=0x5599cbeb04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.517+0000 7eff3f400700 1 --2- 172.21.15.63:0/171368880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7820 0x5599cbeb04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.517+0000 7eff3f400700 1 -- 172.21.15.63:0/171368880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 msgr2=0x5599cbeacd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.517+0000 7eff3f400700 1 --2- 172.21.15.63:0/171368880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 0x5599cbeacd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.517+0000 7eff3f400700 1 -- 172.21.15.63:0/171368880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cbeb4520 con 0x5599cbea7410 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.517+0000 7eff3fe00700 1 --2- 172.21.15.63:0/171368880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 0x5599cbeacd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.518+0000 7eff3f400700 1 --2- 172.21.15.63:0/171368880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbebdc60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009fd0 tx=0x7eff2c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf264fe67c2e031 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.518+0000 7eff34c00700 1 -- 172.21.15.63:0/171368880 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c0a9de0 con 0x5599cbea7410 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.518+0000 7eff34c00700 1 -- 172.21.15.63:0/171368880 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff2c0aa9d0 con 0x5599cbea7410 2024-05-27T07:48:35.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.518+0000 7eff34c00700 1 -- 172.21.15.63:0/171368880 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c0e0930 con 0x5599cbea7410 2024-05-27T07:48:35.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.519+0000 7eff55decb80 1 -- 172.21.15.63:0/171368880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 msgr2=0x5599cbebdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:35.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.519+0000 7eff55decb80 1 --2- 172.21.15.63:0/171368880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbebdc60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009fd0 tx=0x7eff2c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.519+0000 7eff55decb80 1 -- 172.21.15.63:0/171368880 shutdown_connections 2024-05-27T07:48:35.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.519+0000 7eff55decb80 1 --2- 172.21.15.63:0/171368880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbea7820 0x5599cbeb04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.519+0000 7eff55decb80 1 --2- 172.21.15.63:0/171368880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbd28e40 0x5599cbeacd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.519+0000 7eff55decb80 1 --2- 172.21.15.63:0/171368880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7410 0x5599cbebdc60 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.519+0000 7eff55decb80 1 -- 172.21.15.63:0/171368880 >> 172.21.15.63:0/171368880 conn(0x5599cbd88fc0 msgr2=0x5599cbeb1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:35.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.519+0000 7eff55decb80 1 -- 172.21.15.63:0/171368880 shutdown_connections 2024-05-27T07:48:35.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.519+0000 7eff55decb80 1 -- 172.21.15.63:0/171368880 wait complete. 2024-05-27T07:48:35.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.520+0000 7eff55decb80 1 -- start start 2024-05-27T07:48:35.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.520+0000 7eff55decb80 1 Processor -- start 2024-05-27T07:48:35.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff55decb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbd28e40 0x5599cbec6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff55decb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7410 0x5599cbec41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff55decb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7820 0x5599cbec46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff55decb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff401b0b30 con 0x5599cbea7820 2024-05-27T07:48:35.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff55decb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff401b1fa0 con 0x5599cbea7410 2024-05-27T07:48:35.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff55decb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7eff401b1200 con 0x5599cbd28e40 2024-05-27T07:48:35.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff55decb80 10 client.0 _ll_register_callbacks cb 0x5599cbd6fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:35.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff45200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7820 0x5599cbec46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff45200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7820 0x5599cbec46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50342/0 (socket says 172.21.15.63:50342) 2024-05-27T07:48:35.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff45200700 1 -- 172.21.15.63:0/2388391270 learned_addr learned my addr 172.21.15.63:0/2388391270 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:35.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.521+0000 7eff3fe00700 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbd28e40 0x5599cbec6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.522+0000 7eff3f400700 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7410 0x5599cbec41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.522+0000 7eff45200700 1 -- 172.21.15.63:0/2388391270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbd28e40 msgr2=0x5599cbec6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:35.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.522+0000 7eff45200700 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbd28e40 0x5599cbec6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.522+0000 7eff45200700 1 -- 172.21.15.63:0/2388391270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7410 msgr2=0x5599cbec41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:35.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.522+0000 7eff45200700 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7410 0x5599cbec41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.522+0000 7eff45200700 1 -- 172.21.15.63:0/2388391270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cbec4d30 con 0x5599cbea7820 2024-05-27T07:48:35.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.522+0000 7eff3fe00700 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbd28e40 0x5599cbec6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:35.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.522+0000 7eff45200700 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7820 0x5599cbec46f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7eff38017cd0 tx=0x7eff38009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf69411830d88f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:35.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.522+0000 7eff2a000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:35.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.523+0000 7eff2a000700 1 -- 172.21.15.63:0/2388391270 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38028070 con 0x5599cbea7820 2024-05-27T07:48:35.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.523+0000 7eff2a000700 1 -- 172.21.15.63:0/2388391270 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7eff38016e00 con 0x5599cbea7820 2024-05-27T07:48:35.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.523+0000 7eff55decb80 1 -- 172.21.15.63:0/2388391270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff2c0aa9d0 con 0x5599cbea7820 2024-05-27T07:48:35.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.523+0000 7eff2a000700 1 -- 172.21.15.63:0/2388391270 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7eff38033050 con 0x5599cbea7820 2024-05-27T07:48:35.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.523+0000 7eff55decb80 10 client.18288 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:35.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.523+0000 7eff55decb80 1 -- 172.21.15.63:0/2388391270 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5599cbf2bed0 con 0x5599cbea7820 2024-05-27T07:48:35.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.524+0000 7eff2a000700 1 -- 172.21.15.63:0/2388391270 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff2c0aa9d0 con 0x5599cbea7820 2024-05-27T07:48:35.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.524+0000 7eff2a000700 1 -- 172.21.15.63:0/2388391270 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7eff38023040 con 0x5599cbea7820 2024-05-27T07:48:35.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.524+0000 7eff55decb80 10 client.18288 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:35.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.524+0000 7eff55decb80 10 client.18288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:35.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.524+0000 7eff55decb80 20 client.18288 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:35.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.524+0000 7eff55decb80 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599cbf2c5e0 0x5599cbf2ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:35.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.524+0000 7eff55decb80 4 client.18288 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:48:35.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.524+0000 7eff55decb80 1 -- 172.21.15.63:0/2388391270 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599cbdd1570 con 0x5599cbf2c5e0 2024-05-27T07:48:35.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.525+0000 7eff3fe00700 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599cbf2c5e0 0x5599cbf2ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:35.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.526+0000 7eff3fe00700 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599cbf2c5e0 0x5599cbf2ea00 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a6b080f681c6b9 server_cookie=9db210e2341f3646 in_seq=0 out_seq=0 2024-05-27T07:48:35.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.526+0000 7eff2a000700 10 client.18288 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:35.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.526+0000 7eff2a000700 1 -- 172.21.15.63:0/2388391270 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5599cbdd1570 con 0x5599cbf2c5e0 2024-05-27T07:48:35.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.526+0000 7eff2a000700 10 client.18288 handle_command_reply: tid=0 2024-05-27T07:48:35.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.526+0000 7eff55decb80 1 client.18288 shutdown 2024-05-27T07:48:35.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.526+0000 7eff55decb80 1 -- 172.21.15.63:0/2388391270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7820 msgr2=0x5599cbec46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:35.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.526+0000 7eff55decb80 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7820 0x5599cbec46f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7eff38017cd0 tx=0x7eff38009040 comp rx=0 tx=0).stop 2024-05-27T07:48:35.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.527+0000 7eff55decb80 1 -- 172.21.15.63:0/2388391270 shutdown_connections 2024-05-27T07:48:35.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.527+0000 7eff55decb80 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599cbf2c5e0 0x5599cbf2ea00 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.527+0000 7eff55decb80 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599cbea7820 0x5599cbec46f0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.527+0000 7eff55decb80 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599cbd28e40 0x5599cbec6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.527+0000 7eff55decb80 1 --2- 172.21.15.63:0/2388391270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599cbea7410 0x5599cbec41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.527+0000 7eff2a000700 0 client.18288 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:35.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.527+0000 7eff55decb80 1 -- 172.21.15.63:0/2388391270 >> 172.21.15.63:0/2388391270 conn(0x5599cbd88fc0 msgr2=0x5599cbeb1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:35.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.527+0000 7eff55decb80 1 -- 172.21.15.63:0/2388391270 shutdown_connections 2024-05-27T07:48:35.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.527+0000 7eff55decb80 1 -- 172.21.15.63:0/2388391270 wait complete. 2024-05-27T07:48:35.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.527+0000 7eff55decb80 20 client.18288 trim_cache size 0 max 16384 2024-05-27T07:48:35.530 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:48:35.530 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:48:35.530 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:48:35.530 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 -- 172.21.15.63:0/3481070090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff240b2a70 msgr2=0x7eff240b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff240b2a70 0x7eff240b4f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5599cbea3690 tx=0x7eff38004ad0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 -- 172.21.15.63:0/3481070090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff40108d20 msgr2=0x7eff40075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff40108d20 0x7eff40075ff0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00b580 tx=0x7eff2c00f510 comp rx=0 tx=0).stop 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 -- 172.21.15.63:0/3481070090 shutdown_connections 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff240b2a70 0x7eff240b4f80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff40109670 0x7eff40076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff401043f0 0x7eff40075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 --2- 172.21.15.63:0/3481070090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff40108d20 0x7eff40075ff0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 -- 172.21.15.63:0/3481070090 >> 172.21.15.63:0/3481070090 conn(0x7eff40079d00 msgr2=0x7eff40101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 -- 172.21.15.63:0/3481070090 shutdown_connections 2024-05-27T07:48:35.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:35.528+0000 7eff45c00700 1 -- 172.21.15.63:0/3481070090 wait complete. 2024-05-27T07:48:36.643 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:48:36.643 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:48:37.138 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:37.317 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:37 smithi063 ceph-mon[31613]: pgmap v472: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:48:37.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:37 smithi086 ceph-mon[34687]: pgmap v472: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:48:37.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:37 smithi149 ceph-mon[35408]: pgmap v472: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:48:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.091+0000 7fb469000640 1 -- 172.21.15.63:0/3969837726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb464104c10 msgr2=0x7fb464105020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.091+0000 7fb469000640 1 --2- 172.21.15.63:0/3969837726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb464104c10 0x7fb464105020 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb454009900 tx=0x7fb45402a510 comp rx=0 tx=0).stop 2024-05-27T07:48:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.091+0000 7fb469000640 1 -- 172.21.15.63:0/3969837726 shutdown_connections 2024-05-27T07:48:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.091+0000 7fb469000640 1 --2- 172.21.15.63:0/3969837726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb46410e020 0x7fb464110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.091+0000 7fb469000640 1 --2- 172.21.15.63:0/3969837726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb46410b670 0x7fb46410da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.091+0000 7fb469000640 1 --2- 172.21.15.63:0/3969837726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb464104c10 0x7fb464105020 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.091+0000 7fb469000640 1 -- 172.21.15.63:0/3969837726 >> 172.21.15.63:0/3969837726 conn(0x7fb4640fe210 msgr2=0x7fb464100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.091+0000 7fb469000640 1 -- 172.21.15.63:0/3969837726 shutdown_connections 2024-05-27T07:48:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.091+0000 7fb469000640 1 -- 172.21.15.63:0/3969837726 wait complete. 2024-05-27T07:48:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.093+0000 7fb469000640 1 Processor -- start 2024-05-27T07:48:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.093+0000 7fb469000640 1 -- start start 2024-05-27T07:48:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.093+0000 7fb469000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb464104c10 0x7fb4641024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.093+0000 7fb469000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46410b670 0x7fb464102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.093+0000 7fb469000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb46410e020 0x7fb464102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.093+0000 7fb469000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb464113b30 con 0x7fb46410b670 2024-05-27T07:48:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.093+0000 7fb469000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641139b0 con 0x7fb464104c10 2024-05-27T07:48:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.093+0000 7fb469000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb464113cb0 con 0x7fb46410e020 2024-05-27T07:48:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb45be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46410b670 0x7fb464102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb45be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46410b670 0x7fb464102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43732/0 (socket says 172.21.15.63:43732) 2024-05-27T07:48:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb45be00640 1 -- 172.21.15.63:0/1852035293 learned_addr learned my addr 172.21.15.63:0/1852035293 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb463400640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb464104c10 0x7fb4641024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb463e00640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb46410e020 0x7fb464102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb45be00640 1 -- 172.21.15.63:0/1852035293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb46410e020 msgr2=0x7fb464102f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb45be00640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb46410e020 0x7fb464102f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb45be00640 1 -- 172.21.15.63:0/1852035293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb464104c10 msgr2=0x7fb4641024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb45be00640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb464104c10 0x7fb4641024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb45be00640 1 -- 172.21.15.63:0/1852035293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4641a8930 con 0x7fb46410b670 2024-05-27T07:48:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb463e00640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb46410e020 0x7fb464102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.094+0000 7fb463400640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb464104c10 0x7fb4641024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.095+0000 7fb45be00640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46410b670 0x7fb464102a10 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00e9f0 tx=0x7fb44c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.095+0000 7fb460c00640 1 -- 172.21.15.63:0/1852035293 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00cd80 con 0x7fb46410b670 2024-05-27T07:48:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.095+0000 7fb460c00640 1 -- 172.21.15.63:0/1852035293 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb44c0108c0 con 0x7fb46410b670 2024-05-27T07:48:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.095+0000 7fb460c00640 1 -- 172.21.15.63:0/1852035293 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0187c0 con 0x7fb46410b670 2024-05-27T07:48:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.095+0000 7fb469000640 1 -- 172.21.15.63:0/1852035293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4641a8c20 con 0x7fb46410b670 2024-05-27T07:48:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.095+0000 7fb469000640 1 -- 172.21.15.63:0/1852035293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4641a9120 con 0x7fb46410b670 2024-05-27T07:48:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.099+0000 7fb460c00640 1 -- 172.21.15.63:0/1852035293 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb44c010430 con 0x7fb46410b670 2024-05-27T07:48:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.099+0000 7fb4715c4740 1 Processor -- start 2024-05-27T07:48:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.099+0000 7fb4715c4740 1 -- start start 2024-05-27T07:48:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.100+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c6942a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.100+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6942fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.100+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 0x55f5c69434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.100+0000 7fb4715c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f5c68b3b90 con 0x55f5c65772f0 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.100+0000 7fb4715c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5c68a3730 con 0x55f5c65a01b0 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.100+0000 7fb4715c4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f5c68a7770 con 0x55f5c657f4d0 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.100+0000 7fb463400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6942fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.100+0000 7fb463400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6942fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43746/0 (socket says 172.21.15.63:43746) 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.100+0000 7fb463400640 1 -- 172.21.15.63:0/1986059818 learned_addr learned my addr 172.21.15.63:0/1986059818 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.101+0000 7fb463e00640 1 --2- 172.21.15.63:0/1986059818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c6942a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.101+0000 7fb45be00640 1 --2- 172.21.15.63:0/1986059818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 0x55f5c69434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.101+0000 7fb463400640 1 -- 172.21.15.63:0/1986059818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 msgr2=0x55f5c69434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.101+0000 7fb463400640 1 --2- 172.21.15.63:0/1986059818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 0x55f5c69434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.101+0000 7fb463400640 1 -- 172.21.15.63:0/1986059818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 msgr2=0x55f5c6942a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.101+0000 7fb463400640 1 --2- 172.21.15.63:0/1986059818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c6942a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.101+0000 7fb463400640 1 -- 172.21.15.63:0/1986059818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c683c8b0 con 0x55f5c65772f0 2024-05-27T07:48:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.101+0000 7fb45be00640 1 --2- 172.21.15.63:0/1986059818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 0x55f5c69434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.101+0000 7fb460c00640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4480b29d0 0x7fb4480b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.102+0000 7fb463400640 1 --2- 172.21.15.63:0/1986059818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6942fa0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb454009900 tx=0x7fb454032d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9062174fee74563 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.102+0000 7fb460c00640 1 -- 172.21.15.63:0/1852035293 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fb44c0e2e70 con 0x7fb46410b670 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.102+0000 7fb463e00640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4480b29d0 0x7fb4480b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.102+0000 7fb458c00640 1 -- 172.21.15.63:0/1986059818 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45403f070 con 0x55f5c65772f0 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.102+0000 7fb458c00640 1 -- 172.21.15.63:0/1986059818 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb454005de0 con 0x55f5c65772f0 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.102+0000 7fb458c00640 1 -- 172.21.15.63:0/1986059818 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454035070 con 0x55f5c65772f0 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.103+0000 7fb4715c4740 1 -- 172.21.15.63:0/1986059818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 msgr2=0x55f5c6942fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.103+0000 7fb463e00640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4480b29d0 0x7fb4480b4ee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55f5c693fbd0 tx=0x7fb45c009680 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.103+0000 7fb4715c4740 1 --2- 172.21.15.63:0/1986059818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6942fa0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fb454009900 tx=0x7fb454032d20 comp rx=0 tx=0).stop 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.103+0000 7fb4715c4740 1 -- 172.21.15.63:0/1986059818 shutdown_connections 2024-05-27T07:48:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.103+0000 7fb4715c4740 1 --2- 172.21.15.63:0/1986059818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 0x55f5c69434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.103+0000 7fb4715c4740 1 --2- 172.21.15.63:0/1986059818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6942fa0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.103+0000 7fb4715c4740 1 --2- 172.21.15.63:0/1986059818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c6942a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.103+0000 7fb4715c4740 1 -- 172.21.15.63:0/1986059818 >> 172.21.15.63:0/1986059818 conn(0x55f5c6594300 msgr2=0x55f5c693f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.104+0000 7fb4715c4740 1 -- 172.21.15.63:0/1986059818 shutdown_connections 2024-05-27T07:48:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.104+0000 7fb4715c4740 1 -- 172.21.15.63:0/1986059818 wait complete. 2024-05-27T07:48:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.104+0000 7fb4715c4740 1 -- start start 2024-05-27T07:48:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.104+0000 7fb4715c4740 1 Processor -- start 2024-05-27T07:48:39.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.105+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65772f0 0x55f5c694cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.105+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c694d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.105+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c6958e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.105+0000 7fb4715c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f5c6589790 con 0x55f5c657f4d0 2024-05-27T07:48:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.105+0000 7fb4715c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5c684bbc0 con 0x55f5c65a01b0 2024-05-27T07:48:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.105+0000 7fb4715c4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f5c694fb30 con 0x55f5c65772f0 2024-05-27T07:48:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.105+0000 7fb4715c4740 10 client.0 _ll_register_callbacks cb 0x55f5c6829400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.105+0000 7fb45be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c694d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.105+0000 7fb45be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c694d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43756/0 (socket says 172.21.15.63:43756) 2024-05-27T07:48:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.106+0000 7fb45be00640 1 -- 172.21.15.63:0/884082699 learned_addr learned my addr 172.21.15.63:0/884082699 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.106+0000 7fb463400640 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65772f0 0x55f5c694cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.106+0000 7fb463e00640 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c6958e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.106+0000 7fb45be00640 1 -- 172.21.15.63:0/884082699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65772f0 msgr2=0x55f5c694cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.106+0000 7fb45be00640 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65772f0 0x55f5c694cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.106+0000 7fb45be00640 1 -- 172.21.15.63:0/884082699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 msgr2=0x55f5c6958e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.106+0000 7fb45be00640 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c6958e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.106+0000 7fb45be00640 1 -- 172.21.15.63:0/884082699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c65b8210 con 0x55f5c657f4d0 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.106+0000 7fb45be00640 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c694d150 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0a5db0 tx=0x7fb44c010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f380ac86bb7ebfef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.106+0000 7fb451600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.107+0000 7fb451600640 1 -- 172.21.15.63:0/884082699 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0e0630 con 0x55f5c657f4d0 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.107+0000 7fb451600640 1 -- 172.21.15.63:0/884082699 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fb44c0a8f90 con 0x55f5c657f4d0 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.107+0000 7fb4715c4740 1 -- 172.21.15.63:0/884082699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f5c68b5ae0 con 0x55f5c657f4d0 2024-05-27T07:48:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.107+0000 7fb451600640 1 -- 172.21.15.63:0/884082699 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb44c0a9d30 con 0x55f5c657f4d0 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.107+0000 7fb4715c4740 10 client.18300 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.107+0000 7fb4715c4740 1 -- 172.21.15.63:0/884082699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f5c6684620 con 0x55f5c657f4d0 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.107+0000 7fb451600640 1 -- 172.21.15.63:0/884082699 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb44c0c0030 con 0x55f5c657f4d0 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.108+0000 7fb451600640 1 -- 172.21.15.63:0/884082699 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb44c0a56a0 con 0x55f5c657f4d0 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.108+0000 7fb4715c4740 10 client.18300 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.108+0000 7fb4715c4740 10 client.18300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.108+0000 7fb4715c4740 20 client.18300 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.108+0000 7fb4715c4740 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5c655ed80 0x55f5c695eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.108+0000 7fb4715c4740 4 client.18300 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.108+0000 7fb4715c4740 1 -- 172.21.15.63:0/884082699 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c68af2c0 con 0x55f5c655ed80 2024-05-27T07:48:39.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.108+0000 7fb463400640 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5c655ed80 0x55f5c695eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.109+0000 7fb451600640 10 client.18300 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.109+0000 7fb463400640 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5c655ed80 0x55f5c695eef0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e5cfaa05b4c8cf server_cookie=9adcb1253273cfbc in_seq=0 out_seq=0 2024-05-27T07:48:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.111+0000 7fb451600640 1 -- 172.21.15.63:0/884082699 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f5c68af2c0 con 0x55f5c655ed80 2024-05-27T07:48:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.111+0000 7fb451600640 10 client.18300 handle_command_reply: tid=0 2024-05-27T07:48:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 client.18300 shutdown 2024-05-27T07:48:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 -- 172.21.15.63:0/884082699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 msgr2=0x55f5c694d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c694d150 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0a5db0 tx=0x7fb44c010720 comp rx=0 tx=0).stop 2024-05-27T07:48:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 -- 172.21.15.63:0/884082699 shutdown_connections 2024-05-27T07:48:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5c655ed80 0x55f5c695eef0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c6958e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c694d150 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 --2- 172.21.15.63:0/884082699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65772f0 0x55f5c694cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb451600640 0 client.18300 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 -- 172.21.15.63:0/884082699 >> 172.21.15.63:0/884082699 conn(0x55f5c6594300 msgr2=0x55f5c6942890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 -- 172.21.15.63:0/884082699 shutdown_connections 2024-05-27T07:48:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 1 -- 172.21.15.63:0/884082699 wait complete. 2024-05-27T07:48:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.112+0000 7fb4715c4740 20 client.18300 trim_cache size 0 max 16384 2024-05-27T07:48:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.129+0000 7fb4715c4740 1 Processor -- start 2024-05-27T07:48:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.129+0000 7fb4715c4740 1 -- start start 2024-05-27T07:48:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.130+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6958c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.130+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 0x55f5c6959140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.130+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c69646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.130+0000 7fb4715c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641b5ae0 con 0x55f5c65772f0 2024-05-27T07:48:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.130+0000 7fb4715c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641b33b0 con 0x55f5c65a01b0 2024-05-27T07:48:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.130+0000 7fb4715c4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641b1220 con 0x55f5c657f4d0 2024-05-27T07:48:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.130+0000 7fb463400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6958c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.130+0000 7fb463400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6958c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43760/0 (socket says 172.21.15.63:43760) 2024-05-27T07:48:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.130+0000 7fb463400640 1 -- 172.21.15.63:0/3902521128 learned_addr learned my addr 172.21.15.63:0/3902521128 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb45be00640 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 0x55f5c6959140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb463e00640 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c69646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb463400640 1 -- 172.21.15.63:0/3902521128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 msgr2=0x55f5c6959140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb463400640 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 0x55f5c6959140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb463400640 1 -- 172.21.15.63:0/3902521128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 msgr2=0x55f5c69646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb463400640 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c69646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb463400640 1 -- 172.21.15.63:0/3902521128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c6943810 con 0x55f5c65772f0 2024-05-27T07:48:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb463e00640 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c69646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb45be00640 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 0x55f5c6959140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb463400640 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6958c00 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb454033000 tx=0x7fb4540050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1dcad44968d573 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.131+0000 7fb453e00640 1 -- 172.21.15.63:0/3902521128 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454052070 con 0x55f5c65772f0 2024-05-27T07:48:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.132+0000 7fb453e00640 1 -- 172.21.15.63:0/3902521128 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4540044d0 con 0x55f5c65772f0 2024-05-27T07:48:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.132+0000 7fb453e00640 1 -- 172.21.15.63:0/3902521128 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45403f070 con 0x55f5c65772f0 2024-05-27T07:48:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.133+0000 7fb4715c4740 1 -- 172.21.15.63:0/3902521128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 msgr2=0x55f5c6958c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.133+0000 7fb4715c4740 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6958c00 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb454033000 tx=0x7fb4540050e0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.133+0000 7fb4715c4740 1 -- 172.21.15.63:0/3902521128 shutdown_connections 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.133+0000 7fb4715c4740 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65a01b0 0x55f5c69646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.133+0000 7fb4715c4740 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c657f4d0 0x55f5c6959140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.133+0000 7fb4715c4740 1 --2- 172.21.15.63:0/3902521128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c65772f0 0x55f5c6958c00 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.133+0000 7fb4715c4740 1 -- 172.21.15.63:0/3902521128 >> 172.21.15.63:0/3902521128 conn(0x55f5c6594300 msgr2=0x55f5c694ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.133+0000 7fb4715c4740 1 -- 172.21.15.63:0/3902521128 shutdown_connections 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.133+0000 7fb4715c4740 1 -- 172.21.15.63:0/3902521128 wait complete. 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.133+0000 7fb4715c4740 1 -- start start 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.134+0000 7fb4715c4740 1 Processor -- start 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.134+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65772f0 0x55f5c69550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.134+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c6955630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.134+0000 7fb4715c4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65a01b0 0x55f5c6952550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.134+0000 7fb4715c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641b1870 con 0x55f5c657f4d0 2024-05-27T07:48:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.134+0000 7fb4715c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641b2ce0 con 0x55f5c65772f0 2024-05-27T07:48:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.134+0000 7fb4715c4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641b1f40 con 0x55f5c65a01b0 2024-05-27T07:48:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.134+0000 7fb4715c4740 10 client.0 _ll_register_callbacks cb 0x7fb44c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb45be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c6955630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb45be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c6955630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43776/0 (socket says 172.21.15.63:43776) 2024-05-27T07:48:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb45be00640 1 -- 172.21.15.63:0/2160641820 learned_addr learned my addr 172.21.15.63:0/2160641820 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb463400640 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65772f0 0x55f5c69550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb463e00640 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65a01b0 0x55f5c6952550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb45be00640 1 -- 172.21.15.63:0/2160641820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65a01b0 msgr2=0x55f5c6952550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb45be00640 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65a01b0 0x55f5c6952550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb45be00640 1 -- 172.21.15.63:0/2160641820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65772f0 msgr2=0x55f5c69550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb45be00640 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65772f0 0x55f5c69550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb45be00640 1 -- 172.21.15.63:0/2160641820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c6952cc0 con 0x55f5c657f4d0 2024-05-27T07:48:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.135+0000 7fb463e00640 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65a01b0 0x55f5c6952550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.136+0000 7fb45be00640 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c6955630 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00cfd0 tx=0x7fb44c0e64d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b8cf0d315fd496 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.136+0000 7fb451600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.136+0000 7fb451600640 1 -- 172.21.15.63:0/2160641820 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0a9840 con 0x55f5c657f4d0 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.136+0000 7fb451600640 1 -- 172.21.15.63:0/2160641820 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fb44c0a8a00 con 0x55f5c657f4d0 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.136+0000 7fb4715c4740 1 -- 172.21.15.63:0/2160641820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4540044d0 con 0x55f5c657f4d0 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.136+0000 7fb451600640 1 -- 172.21.15.63:0/2160641820 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb44c0c43f0 con 0x55f5c657f4d0 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.136+0000 7fb4715c4740 10 client.18306 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.136+0000 7fb4715c4740 1 -- 172.21.15.63:0/2160641820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f5c69c6f60 con 0x55f5c657f4d0 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.136+0000 7fb451600640 1 -- 172.21.15.63:0/2160641820 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4540044d0 con 0x55f5c657f4d0 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.137+0000 7fb451600640 1 -- 172.21.15.63:0/2160641820 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb44c0a5040 con 0x55f5c657f4d0 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.137+0000 7fb4715c4740 10 client.18306 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.137+0000 7fb4715c4740 10 client.18306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.137+0000 7fb4715c4740 20 client.18306 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.137+0000 7fb4715c4740 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5c65341c0 0x55f5c69c9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.137+0000 7fb4715c4740 4 client.18306 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:48:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.137+0000 7fb4715c4740 1 -- 172.21.15.63:0/2160641820 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5c68294b0 con 0x55f5c65341c0 2024-05-27T07:48:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.138+0000 7fb463400640 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5c65341c0 0x55f5c69c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.138+0000 7fb463400640 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5c65341c0 0x55f5c69c9660 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c510c04e894c99d server_cookie=56c7544915c3a84 in_seq=0 out_seq=0 2024-05-27T07:48:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.138+0000 7fb451600640 10 client.18306 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb451600640 1 -- 172.21.15.63:0/2160641820 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f5c68294b0 con 0x55f5c65341c0 2024-05-27T07:48:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb451600640 10 client.18306 handle_command_reply: tid=0 2024-05-27T07:48:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb4715c4740 1 client.18306 shutdown 2024-05-27T07:48:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb4715c4740 1 -- 172.21.15.63:0/2160641820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 msgr2=0x55f5c6955630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb4715c4740 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c6955630 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00cfd0 tx=0x7fb44c0e64d0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb4715c4740 1 -- 172.21.15.63:0/2160641820 shutdown_connections 2024-05-27T07:48:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb4715c4740 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f5c65341c0 0x55f5c69c9660 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb4715c4740 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5c65a01b0 0x55f5c6952550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb4715c4740 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5c657f4d0 0x55f5c6955630 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb4715c4740 1 --2- 172.21.15.63:0/2160641820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c65772f0 0x55f5c69550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb4715c4740 1 -- 172.21.15.63:0/2160641820 >> 172.21.15.63:0/2160641820 conn(0x55f5c6594300 msgr2=0x55f5c694cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.169+0000 7fb451600640 0 client.18306 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.170+0000 7fb4715c4740 1 -- 172.21.15.63:0/2160641820 shutdown_connections 2024-05-27T07:48:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.170+0000 7fb4715c4740 1 -- 172.21.15.63:0/2160641820 wait complete. 2024-05-27T07:48:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.170+0000 7fb4715c4740 20 client.18306 trim_cache size 0 max 16384 2024-05-27T07:48:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:48:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:48:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:48:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 -- 172.21.15.63:0/1852035293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4480b29d0 msgr2=0x7fb4480b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4480b29d0 0x7fb4480b4ee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55f5c693fbd0 tx=0x7fb45c009680 comp rx=0 tx=0).stop 2024-05-27T07:48:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 -- 172.21.15.63:0/1852035293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46410b670 msgr2=0x7fb464102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46410b670 0x7fb464102a10 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00e9f0 tx=0x7fb44c00eeb0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 -- 172.21.15.63:0/1852035293 shutdown_connections 2024-05-27T07:48:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb4480b29d0 0x7fb4480b4ee0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb46410e020 0x7fb464102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46410b670 0x7fb464102a10 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 --2- 172.21.15.63:0/1852035293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb464104c10 0x7fb4641024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 -- 172.21.15.63:0/1852035293 >> 172.21.15.63:0/1852035293 conn(0x7fb4640fe210 msgr2=0x7fb46410e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 -- 172.21.15.63:0/1852035293 shutdown_connections 2024-05-27T07:48:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:39.171+0000 7fb469000640 1 -- 172.21.15.63:0/1852035293 wait complete. 2024-05-27T07:48:39.356 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:39 smithi063 ceph-mon[31613]: pgmap v473: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:48:39.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:39 smithi086 ceph-mon[34687]: pgmap v473: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:48:39.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:39 smithi149 ceph-mon[35408]: pgmap v473: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:48:39.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.620+0000 7f11ee000700 1 -- 172.21.15.63:0/4059899228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e8105300 msgr2=0x7f11e8105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.620+0000 7f11ee000700 1 --2- 172.21.15.63:0/4059899228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e8105300 0x7f11e8105710 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009990 tx=0x7f11dc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.620+0000 7f11ee000700 1 -- 172.21.15.63:0/4059899228 shutdown_connections 2024-05-27T07:48:39.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.620+0000 7f11ee000700 1 --2- 172.21.15.63:0/4059899228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11e810e730 0x7f11e8110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.620+0000 7f11ee000700 1 --2- 172.21.15.63:0/4059899228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e8105300 0x7f11e8105710 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.620+0000 7f11ee000700 1 --2- 172.21.15.63:0/4059899228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e8105c50 0x7f11e810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.620+0000 7f11ee000700 1 -- 172.21.15.63:0/4059899228 >> 172.21.15.63:0/4059899228 conn(0x7f11e8078ab0 msgr2=0x7f11e8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:39.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.621+0000 7f11ee000700 1 -- 172.21.15.63:0/4059899228 shutdown_connections 2024-05-27T07:48:39.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.621+0000 7f11ee000700 1 -- 172.21.15.63:0/4059899228 wait complete. 2024-05-27T07:48:39.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.622+0000 7f11ee000700 1 Processor -- start 2024-05-27T07:48:39.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.623+0000 7f11ee000700 1 -- start start 2024-05-27T07:48:39.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.623+0000 7f11ee000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e8105300 0x7f11e81a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.623+0000 7f11ee000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e8105c50 0x7f11e81a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.623+0000 7f11ee000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11e810e730 0x7f11e819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.623+0000 7f11ee000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81141c0 con 0x7f11e8105c50 2024-05-27T07:48:39.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.623+0000 7f11ee000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8114040 con 0x7f11e8105300 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.623+0000 7f11ee000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8114340 con 0x7f11e810e730 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.624+0000 7f11e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e8105c50 0x7f11e81a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.624+0000 7f11e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e8105c50 0x7f11e81a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43800/0 (socket says 172.21.15.63:43800) 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.624+0000 7f11e7e00700 1 -- 172.21.15.63:0/1971877112 learned_addr learned my addr 172.21.15.63:0/1971877112 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.624+0000 7f11ecc00700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e8105300 0x7f11e81a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.624+0000 7f11ed600700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11e810e730 0x7f11e819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.625+0000 7f11e7e00700 1 -- 172.21.15.63:0/1971877112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11e810e730 msgr2=0x7f11e819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.625+0000 7f11e7e00700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11e810e730 0x7f11e819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.625+0000 7f11e7e00700 1 -- 172.21.15.63:0/1971877112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e8105300 msgr2=0x7f11e81a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.625+0000 7f11e7e00700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e8105300 0x7f11e81a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.625+0000 7f11e7e00700 1 -- 172.21.15.63:0/1971877112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e819b090 con 0x7f11e8105c50 2024-05-27T07:48:39.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.625+0000 7f11e7e00700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e8105c50 0x7f11e81a0d00 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f11d400b530 tx=0x7f11d400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:39.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.626+0000 7f11e5600700 1 -- 172.21.15.63:0/1971877112 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4015020 con 0x7f11e8105c50 2024-05-27T07:48:39.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.626+0000 7f11e5600700 1 -- 172.21.15.63:0/1971877112 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11d4011cc0 con 0x7f11e8105c50 2024-05-27T07:48:39.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.626+0000 7f11ee000700 1 -- 172.21.15.63:0/1971877112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e819b380 con 0x7f11e8105c50 2024-05-27T07:48:39.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.626+0000 7f11e5600700 1 -- 172.21.15.63:0/1971877112 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4010c80 con 0x7f11e8105c50 2024-05-27T07:48:39.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.626+0000 7f11ee000700 1 -- 172.21.15.63:0/1971877112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e81a7650 con 0x7f11e8105c50 2024-05-27T07:48:39.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.628+0000 7f11ee000700 1 -- 172.21.15.63:0/1971877112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e80512b0 con 0x7f11e8105c50 2024-05-27T07:48:39.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.633+0000 7f11e5600700 1 -- 172.21.15.63:0/1971877112 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f11d4018020 con 0x7f11e8105c50 2024-05-27T07:48:39.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.635+0000 7f11e5600700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11d00b29f0 0x7f11d00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:39.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.636+0000 7f11e5600700 1 -- 172.21.15.63:0/1971877112 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f11d40e24b0 con 0x7f11e8105c50 2024-05-27T07:48:39.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.636+0000 7f11ecc00700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11d00b29f0 0x7f11d00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:39.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.637+0000 7f11e5600700 1 -- 172.21.15.63:0/1971877112 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f11d401f020 con 0x7f11e8105c50 2024-05-27T07:48:39.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.638+0000 7f11ecc00700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11d00b29f0 0x7f11d00b4f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f11e819bb20 tx=0x7f11dc00b540 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.969+0000 7f11ee000700 1 -- 172.21.15.63:0/1971877112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11e819bf10 con 0x7f11e8105c50 2024-05-27T07:48:39.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.971+0000 7f11e5600700 1 -- 172.21.15.63:0/1971877112 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f11d40a9c80 con 0x7f11e8105c50 2024-05-27T07:48:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:48:39.973 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:48:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.975+0000 7f11ee000700 1 -- 172.21.15.63:0/1971877112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11d00b29f0 msgr2=0x7f11d00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.975+0000 7f11ee000700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11d00b29f0 0x7f11d00b4f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f11e819bb20 tx=0x7f11dc00b540 comp rx=0 tx=0).stop 2024-05-27T07:48:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.975+0000 7f11ee000700 1 -- 172.21.15.63:0/1971877112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e8105c50 msgr2=0x7f11e81a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.975+0000 7f11ee000700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e8105c50 0x7f11e81a0d00 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f11d400b530 tx=0x7f11d400b840 comp rx=0 tx=0).stop 2024-05-27T07:48:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.976+0000 7f11ee000700 1 -- 172.21.15.63:0/1971877112 shutdown_connections 2024-05-27T07:48:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.976+0000 7f11ee000700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11d00b29f0 0x7f11d00b4f00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.976+0000 7f11ee000700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11e810e730 0x7f11e819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.976+0000 7f11ee000700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e8105300 0x7f11e81a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.976+0000 7f11ee000700 1 --2- 172.21.15.63:0/1971877112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11e8105c50 0x7f11e81a0d00 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:39.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.976+0000 7f11ee000700 1 -- 172.21.15.63:0/1971877112 >> 172.21.15.63:0/1971877112 conn(0x7f11e8078ab0 msgr2=0x7f11e80ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:39.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.976+0000 7f11ee000700 1 -- 172.21.15.63:0/1971877112 shutdown_connections 2024-05-27T07:48:39.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:39.976+0000 7f11ee000700 1 -- 172.21.15.63:0/1971877112 wait complete. 2024-05-27T07:48:39.978 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:48:40.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:40 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1971877112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:48:40.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:40 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1971877112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:48:40.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:40 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1971877112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:48:41.039 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:48:41.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:41 smithi086 ceph-mon[34687]: pgmap v474: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-05-27T07:48:41.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:41 smithi149 ceph-mon[35408]: pgmap v474: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-05-27T07:48:41.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:41 smithi063 ceph-mon[31613]: pgmap v474: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-05-27T07:48:41.556 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:43.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:43 smithi086 ceph-mon[34687]: pgmap v475: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:48:43.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:43 smithi149 ceph-mon[35408]: pgmap v475: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:48:43.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:43 smithi063 ceph-mon[31613]: pgmap v475: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:48:44.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.150+0000 7f59e3e00700 1 -- 172.21.15.63:0/3081276701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59e4108d20 msgr2=0x7f59e4109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:44.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.150+0000 7f59e3e00700 1 --2- 172.21.15.63:0/3081276701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59e4108d20 0x7f59e4109130 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f59d00099c0 tx=0x7f59d0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.150+0000 7f59e3e00700 1 -- 172.21.15.63:0/3081276701 shutdown_connections 2024-05-27T07:48:44.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.150+0000 7f59e3e00700 1 --2- 172.21.15.63:0/3081276701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4109670 0x7f59e4110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.150+0000 7f59e3e00700 1 --2- 172.21.15.63:0/3081276701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59e41043f0 0x7f59e4104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.150+0000 7f59e3e00700 1 --2- 172.21.15.63:0/3081276701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59e4108d20 0x7f59e4109130 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.150+0000 7f59e3e00700 1 -- 172.21.15.63:0/3081276701 >> 172.21.15.63:0/3081276701 conn(0x7f59e4079d00 msgr2=0x7f59e407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:44.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.151+0000 7f59e3e00700 1 -- 172.21.15.63:0/3081276701 shutdown_connections 2024-05-27T07:48:44.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.151+0000 7f59e3e00700 1 -- 172.21.15.63:0/3081276701 wait complete. 2024-05-27T07:48:44.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.152+0000 7f59e3e00700 1 Processor -- start 2024-05-27T07:48:44.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.152+0000 7f59e3e00700 1 -- start start 2024-05-27T07:48:44.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.153+0000 7f59e3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59e41043f0 0x7f59e4075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.153+0000 7f59e3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59e4108d20 0x7f59e4075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.153+0000 7f59e3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4109670 0x7f59e4076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.153+0000 7f59e3e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4114180 con 0x7f59e4108d20 2024-05-27T07:48:44.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.153+0000 7f59e3e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4114000 con 0x7f59e4109670 2024-05-27T07:48:44.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.153+0000 7f59e3e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4114300 con 0x7f59e41043f0 2024-05-27T07:48:44.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.153+0000 7f59e2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59e4108d20 0x7f59e4075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.153+0000 7f59e2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59e4108d20 0x7f59e4075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43822/0 (socket says 172.21.15.63:43822) 2024-05-27T07:48:44.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.153+0000 7f59e2000700 1 -- 172.21.15.63:0/1790599672 learned_addr learned my addr 172.21.15.63:0/1790599672 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:44.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.153+0000 7f59e2a00700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59e41043f0 0x7f59e4075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.154+0000 7f59e3400700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4109670 0x7f59e4076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.154+0000 7f59e2000700 1 -- 172.21.15.63:0/1790599672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59e41043f0 msgr2=0x7f59e4075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:44.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.154+0000 7f59e2000700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59e41043f0 0x7f59e4075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.154+0000 7f59e2000700 1 -- 172.21.15.63:0/1790599672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4109670 msgr2=0x7f59e4076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:44.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.154+0000 7f59e2000700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4109670 0x7f59e4076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.154+0000 7f59e2000700 1 -- 172.21.15.63:0/1790599672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e41a8ff0 con 0x7f59e4108d20 2024-05-27T07:48:44.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.154+0000 7f59e2000700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59e4108d20 0x7f59e4075ff0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f59d0005f50 tx=0x7f59d0005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:44.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.155+0000 7f59e8a00700 1 -- 172.21.15.63:0/1790599672 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0026070 con 0x7f59e4108d20 2024-05-27T07:48:44.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.155+0000 7f59e8a00700 1 -- 172.21.15.63:0/1790599672 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59d0003c50 con 0x7f59e4108d20 2024-05-27T07:48:44.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.155+0000 7f59e3e00700 1 -- 172.21.15.63:0/1790599672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e41a9280 con 0x7f59e4108d20 2024-05-27T07:48:44.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.155+0000 7f59e8a00700 1 -- 172.21.15.63:0/1790599672 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d000f990 con 0x7f59e4108d20 2024-05-27T07:48:44.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.155+0000 7f59e3e00700 1 -- 172.21.15.63:0/1790599672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e41a96d0 con 0x7f59e4108d20 2024-05-27T07:48:44.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.160+0000 7f59e8a00700 1 -- 172.21.15.63:0/1790599672 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f59d0029070 con 0x7f59e4108d20 2024-05-27T07:48:44.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.161+0000 7f59f887cb80 1 Processor -- start 2024-05-27T07:48:44.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.162+0000 7f59e8a00700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59c80b29f0 0x7f59c80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.162+0000 7f59f887cb80 1 -- start start 2024-05-27T07:48:44.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.162+0000 7f59e8a00700 1 -- 172.21.15.63:0/1790599672 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f59d00e3280 con 0x7f59e4108d20 2024-05-27T07:48:44.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.162+0000 7f59e3400700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59c80b29f0 0x7f59c80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.162+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153369f0 0x557f153ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.162+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.162+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153d42c0 0x557f153d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.162+0000 7f59f887cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557f152f7e40 con 0x557f1524ae40 2024-05-27T07:48:44.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.162+0000 7f59f887cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f152e79e0 con 0x557f153369f0 2024-05-27T07:48:44.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.162+0000 7f59f887cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557f152eba20 con 0x557f153d42c0 2024-05-27T07:48:44.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43828/0 (socket says 172.21.15.63:43828) 2024-05-27T07:48:44.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2000700 1 -- 172.21.15.63:0/1833781577 learned_addr learned my addr 172.21.15.63:0/1833781577 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:44.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2a00700 1 --2- 172.21.15.63:0/1833781577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153369f0 0x557f153ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e3400700 1 --2- 172.21.15.63:0/1833781577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153d42c0 0x557f153d66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2000700 1 -- 172.21.15.63:0/1833781577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153d42c0 msgr2=0x557f153d66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:44.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2000700 1 --2- 172.21.15.63:0/1833781577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153d42c0 0x557f153d66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2000700 1 -- 172.21.15.63:0/1833781577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153369f0 msgr2=0x557f153ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:44.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2000700 1 --2- 172.21.15.63:0/1833781577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153369f0 0x557f153ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2000700 1 -- 172.21.15.63:0/1833781577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f152b0420 con 0x557f1524ae40 2024-05-27T07:48:44.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2a00700 1 --2- 172.21.15.63:0/1833781577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153369f0 0x557f153ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:44.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e3400700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59c80b29f0 0x7f59c80b4f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0093c0 tx=0x7f59dc009920 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:44.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.163+0000 7f59e2000700 1 --2- 172.21.15.63:0/1833781577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153cac90 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f59d00aa160 tx=0x7f59d00a9de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20a3c4593d0b696 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:44.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.164+0000 7f59d8c00700 1 -- 172.21.15.63:0/1833781577 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d00aa1d0 con 0x557f1524ae40 2024-05-27T07:48:44.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.164+0000 7f59d8c00700 1 -- 172.21.15.63:0/1833781577 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59d00ae580 con 0x557f1524ae40 2024-05-27T07:48:44.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.164+0000 7f59d8c00700 1 -- 172.21.15.63:0/1833781577 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d00aa630 con 0x557f1524ae40 2024-05-27T07:48:44.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.165+0000 7f59f887cb80 1 -- 172.21.15.63:0/1833781577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 msgr2=0x557f153cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:44.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.165+0000 7f59f887cb80 1 --2- 172.21.15.63:0/1833781577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153cac90 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f59d00aa160 tx=0x7f59d00a9de0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.165+0000 7f59f887cb80 1 -- 172.21.15.63:0/1833781577 shutdown_connections 2024-05-27T07:48:44.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.165+0000 7f59f887cb80 1 --2- 172.21.15.63:0/1833781577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153d42c0 0x557f153d66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.165+0000 7f59f887cb80 1 --2- 172.21.15.63:0/1833781577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153369f0 0x557f153ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.165+0000 7f59f887cb80 1 --2- 172.21.15.63:0/1833781577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153cac90 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.165+0000 7f59f887cb80 1 -- 172.21.15.63:0/1833781577 >> 172.21.15.63:0/1833781577 conn(0x557f152e4390 msgr2=0x557f153c5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:44.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.165+0000 7f59f887cb80 1 -- 172.21.15.63:0/1833781577 shutdown_connections 2024-05-27T07:48:44.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.165+0000 7f59f887cb80 1 -- 172.21.15.63:0/1833781577 wait complete. 2024-05-27T07:48:44.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.166+0000 7f59f887cb80 1 -- start start 2024-05-27T07:48:44.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.166+0000 7f59f887cb80 1 Processor -- start 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153d42c0 0x557f153d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59f887cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557f151edfa0 con 0x557f1524ae40 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59f887cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f15288760 con 0x557f153d42c0 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59f887cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557f153d6c20 con 0x557f153369f0 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59f887cb80 10 client.0 _ll_register_callbacks cb 0x557f15291c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59e2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59e2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43832/0 (socket says 172.21.15.63:43832) 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59e2a00700 1 -- 172.21.15.63:0/4274182455 learned_addr learned my addr 172.21.15.63:0/4274182455 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.167+0000 7f59e2000700 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.168+0000 7f59e3400700 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153d42c0 0x557f153d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.168+0000 7f59e2a00700 1 -- 172.21.15.63:0/4274182455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 msgr2=0x557f153d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:44.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.168+0000 7f59e2a00700 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.168+0000 7f59e2a00700 1 -- 172.21.15.63:0/4274182455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153d42c0 msgr2=0x557f153d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:44.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.168+0000 7f59e2a00700 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153d42c0 0x557f153d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.168+0000 7f59e2a00700 1 -- 172.21.15.63:0/4274182455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f152378d0 con 0x557f1524ae40 2024-05-27T07:48:44.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.168+0000 7f59e2a00700 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153d1280 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f59d4012740 tx=0x7f59d4012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f74e7a24a557b2b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:44.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.168+0000 7f59cd600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:44.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.169+0000 7f59cd600700 1 -- 172.21.15.63:0/4274182455 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4004050 con 0x557f1524ae40 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.169+0000 7f59cd600700 1 -- 172.21.15.63:0/4274182455 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f59d40041f0 con 0x557f1524ae40 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.169+0000 7f59f887cb80 1 -- 172.21.15.63:0/4274182455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d00ae580 con 0x557f1524ae40 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.169+0000 7f59cd600700 1 -- 172.21.15.63:0/4274182455 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f59d40043f0 con 0x557f1524ae40 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.169+0000 7f59f887cb80 10 client.18324 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.169+0000 7f59f887cb80 1 -- 172.21.15.63:0/4274182455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557f152b8060 con 0x557f1524ae40 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.170+0000 7f59cd600700 1 -- 172.21.15.63:0/4274182455 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59d00ae580 con 0x557f1524ae40 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.170+0000 7f59cd600700 1 -- 172.21.15.63:0/4274182455 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f59d400f650 con 0x557f1524ae40 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.170+0000 7f59f887cb80 10 client.18324 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.170+0000 7f59f887cb80 10 client.18324 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.170+0000 7f59f887cb80 20 client.18324 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.170+0000 7f59f887cb80 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x557f153e3ad0 0x557f153e5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.170+0000 7f59f887cb80 4 client.18324 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:44.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.170+0000 7f59f887cb80 1 -- 172.21.15.63:0/4274182455 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f152f9d90 con 0x557f153e3ad0 2024-05-27T07:48:44.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.171+0000 7f59e2000700 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x557f153e3ad0 0x557f153e5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.172+0000 7f59e2000700 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x557f153e3ad0 0x557f153e5ef0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e97509d45fe470f server_cookie=5814e242a5d31151 in_seq=0 out_seq=0 2024-05-27T07:48:44.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.172+0000 7f59cd600700 10 client.18324 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:48:44.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59cd600700 1 -- 172.21.15.63:0/4274182455 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557f152f9d90 con 0x557f153e3ad0 2024-05-27T07:48:44.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59cd600700 10 client.18324 handle_command_reply: tid=0 2024-05-27T07:48:44.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59f887cb80 1 client.18324 shutdown 2024-05-27T07:48:44.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59f887cb80 1 -- 172.21.15.63:0/4274182455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 msgr2=0x557f153d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:44.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59f887cb80 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153d1280 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f59d4012740 tx=0x7f59d4012a50 comp rx=0 tx=0).stop 2024-05-27T07:48:44.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59f887cb80 1 -- 172.21.15.63:0/4274182455 shutdown_connections 2024-05-27T07:48:44.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59f887cb80 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f153d42c0 0x557f153d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59f887cb80 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x557f153e3ad0 0x557f153e5ef0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59f887cb80 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f1524ae40 0x557f153d1280 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59f887cb80 1 --2- 172.21.15.63:0/4274182455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59f887cb80 1 -- 172.21.15.63:0/4274182455 >> 172.21.15.63:0/4274182455 conn(0x557f152e4390 msgr2=0x557f153ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:44.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.174+0000 7f59cd600700 0 client.18324 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:48:44.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.175+0000 7f59f887cb80 1 -- 172.21.15.63:0/4274182455 shutdown_connections 2024-05-27T07:48:44.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.175+0000 7f59f887cb80 1 -- 172.21.15.63:0/4274182455 wait complete. 2024-05-27T07:48:44.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.175+0000 7f59f887cb80 20 client.18324 trim_cache size 0 max 16384 2024-05-27T07:48:44.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.194+0000 7f59f887cb80 1 Processor -- start 2024-05-27T07:48:44.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.194+0000 7f59f887cb80 1 -- start start 2024-05-27T07:48:44.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.194+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 0x557f153d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.194+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153c5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.194+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153dfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.194+0000 7f59f887cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b4db0 con 0x557f153d42c0 2024-05-27T07:48:44.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.194+0000 7f59f887cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b2640 con 0x557f1524ae40 2024-05-27T07:48:44.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.194+0000 7f59f887cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b04b0 con 0x557f153369f0 2024-05-27T07:48:44.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.195+0000 7f59e3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153dfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.195+0000 7f59e3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153dfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43836/0 (socket says 172.21.15.63:43836) 2024-05-27T07:48:44.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.195+0000 7f59e3400700 1 -- 172.21.15.63:0/3691469769 learned_addr learned my addr 172.21.15.63:0/3691469769 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:44.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.195+0000 7f59e2a00700 1 --2- 172.21.15.63:0/3691469769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 0x557f153d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.195+0000 7f59e2000700 1 --2- 172.21.15.63:0/3691469769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.195+0000 7f59e3400700 1 -- 172.21.15.63:0/3691469769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 msgr2=0x557f153c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:44.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.195+0000 7f59e3400700 1 --2- 172.21.15.63:0/3691469769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.195+0000 7f59e3400700 1 -- 172.21.15.63:0/3691469769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 msgr2=0x557f153d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:44.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.195+0000 7f59e3400700 1 --2- 172.21.15.63:0/3691469769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 0x557f153d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.195+0000 7f59e3400700 1 -- 172.21.15.63:0/3691469769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f153bf930 con 0x557f153d42c0 2024-05-27T07:48:44.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.196+0000 7f59e3400700 1 --2- 172.21.15.63:0/3691469769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153dfc80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0177f0 tx=0x7f59dc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e04a900efd6ba82 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:44.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.196+0000 7f59cfe00700 1 -- 172.21.15.63:0/3691469769 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc01d070 con 0x557f153d42c0 2024-05-27T07:48:44.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.196+0000 7f59cfe00700 1 -- 172.21.15.63:0/3691469769 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59dc01e040 con 0x557f153d42c0 2024-05-27T07:48:44.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.196+0000 7f59cfe00700 1 -- 172.21.15.63:0/3691469769 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc023ba0 con 0x557f153d42c0 2024-05-27T07:48:44.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.197+0000 7f59f887cb80 1 -- 172.21.15.63:0/3691469769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 msgr2=0x557f153dfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:44.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.197+0000 7f59f887cb80 1 --2- 172.21.15.63:0/3691469769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153dfc80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0177f0 tx=0x7f59dc01b000 comp rx=0 tx=0).stop 2024-05-27T07:48:44.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.197+0000 7f59f887cb80 1 -- 172.21.15.63:0/3691469769 shutdown_connections 2024-05-27T07:48:44.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.197+0000 7f59f887cb80 1 --2- 172.21.15.63:0/3691469769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153dfc80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.198+0000 7f59f887cb80 1 --2- 172.21.15.63:0/3691469769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 0x557f153d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.198+0000 7f59f887cb80 1 --2- 172.21.15.63:0/3691469769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.198+0000 7f59f887cb80 1 -- 172.21.15.63:0/3691469769 >> 172.21.15.63:0/3691469769 conn(0x557f152e4390 msgr2=0x557f153cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:44.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.198+0000 7f59f887cb80 1 -- 172.21.15.63:0/3691469769 shutdown_connections 2024-05-27T07:48:44.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.198+0000 7f59f887cb80 1 -- 172.21.15.63:0/3691469769 wait complete. 2024-05-27T07:48:44.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.198+0000 7f59f887cb80 1 -- start start 2024-05-27T07:48:44.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.199+0000 7f59f887cb80 1 Processor -- start 2024-05-27T07:48:44.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.199+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 0x557f153d6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.199+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153d7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.199+0000 7f59f887cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153d78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.199+0000 7f59f887cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b0b00 con 0x557f153d42c0 2024-05-27T07:48:44.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.199+0000 7f59f887cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b1f70 con 0x557f1524ae40 2024-05-27T07:48:44.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.199+0000 7f59f887cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b11d0 con 0x557f153369f0 2024-05-27T07:48:44.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.199+0000 7f59f887cb80 10 client.0 _ll_register_callbacks cb 0x557f153c4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:44.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 0x557f153d6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153d78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153d78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43846/0 (socket says 172.21.15.63:43846) 2024-05-27T07:48:44.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 0x557f153d6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34516/0 (socket says 172.21.15.63:34516) 2024-05-27T07:48:44.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e3400700 1 -- 172.21.15.63:0/2482595459 learned_addr learned my addr 172.21.15.63:0/2482595459 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:44.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e2000700 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e3400700 1 -- 172.21.15.63:0/2482595459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 msgr2=0x557f153d7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:44.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e3400700 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153d7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e3400700 1 -- 172.21.15.63:0/2482595459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 msgr2=0x557f153d6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:44.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e3400700 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 0x557f153d6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e3400700 1 -- 172.21.15.63:0/2482595459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f1544c590 con 0x557f153d42c0 2024-05-27T07:48:44.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.200+0000 7f59e2000700 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153d7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:44.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.201+0000 7f59e3400700 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153d78b0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009fd0 tx=0x7f59dc023ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec5807135d1f8ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:44.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.201+0000 7f59cd600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:44.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.201+0000 7f59cd600700 1 -- 172.21.15.63:0/2482595459 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc01d070 con 0x557f153d42c0 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.201+0000 7f59f887cb80 1 -- 172.21.15.63:0/2482595459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59dc01e040 con 0x557f153d42c0 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.201+0000 7f59cd600700 1 -- 172.21.15.63:0/2482595459 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f59dc0188e0 con 0x557f153d42c0 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59cd600700 1 -- 172.21.15.63:0/2482595459 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f59dc0233f0 con 0x557f153d42c0 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59f887cb80 10 client.18330 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59f887cb80 1 -- 172.21.15.63:0/2482595459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557f1544d6c0 con 0x557f153d42c0 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59cd600700 1 -- 172.21.15.63:0/2482595459 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59dc01e040 con 0x557f153d42c0 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59cd600700 1 -- 172.21.15.63:0/2482595459 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f59dc032070 con 0x557f153d42c0 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59f887cb80 10 client.18330 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59f887cb80 10 client.18330 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59f887cb80 20 client.18330 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59f887cb80 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x557f1544dd00 0x557f15450120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59f887cb80 4 client.18330 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:44.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.202+0000 7f59f887cb80 1 -- 172.21.15.63:0/2482595459 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f152f3570 con 0x557f1544dd00 2024-05-27T07:48:44.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.203+0000 7f59e2a00700 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x557f1544dd00 0x557f15450120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.204+0000 7f59e2a00700 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x557f1544dd00 0x557f15450120 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a2532fb77d8c1da server_cookie=d39b36f79801f396 in_seq=0 out_seq=0 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.204+0000 7f59cd600700 10 client.18330 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59cd600700 1 -- 172.21.15.63:0/2482595459 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f152f3570 con 0x557f1544dd00 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59cd600700 10 client.18330 handle_command_reply: tid=0 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 client.18330 shutdown 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 -- 172.21.15.63:0/2482595459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 msgr2=0x557f153d78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153d78b0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009fd0 tx=0x7f59dc023ad0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 -- 172.21.15.63:0/2482595459 shutdown_connections 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x557f1544dd00 0x557f15450120 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f153d42c0 0x557f153d78b0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1524ae40 0x557f153d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 --2- 172.21.15.63:0/2482595459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f153369f0 0x557f153d7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 -- 172.21.15.63:0/2482595459 >> 172.21.15.63:0/2482595459 conn(0x557f152e4390 msgr2=0x557f153c9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:44.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59cd600700 0 client.18330 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:48:44.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 -- 172.21.15.63:0/2482595459 shutdown_connections 2024-05-27T07:48:44.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 1 -- 172.21.15.63:0/2482595459 wait complete. 2024-05-27T07:48:44.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.205+0000 7f59f887cb80 20 client.18330 trim_cache size 0 max 16384 2024-05-27T07:48:44.206 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:48:44.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.206+0000 7f59e3e00700 1 -- 172.21.15.63:0/1790599672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59c80b29f0 msgr2=0x7f59c80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:44.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.206+0000 7f59e3e00700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59c80b29f0 0x7f59c80b4f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0093c0 tx=0x7f59dc009920 comp rx=0 tx=0).stop 2024-05-27T07:48:44.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.206+0000 7f59e3e00700 1 -- 172.21.15.63:0/1790599672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59e4108d20 msgr2=0x7f59e4075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:44.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.206+0000 7f59e3e00700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59e4108d20 0x7f59e4075ff0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f59d0005f50 tx=0x7f59d0005280 comp rx=0 tx=0).stop 2024-05-27T07:48:44.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.207+0000 7f59e3e00700 1 -- 172.21.15.63:0/1790599672 shutdown_connections 2024-05-27T07:48:44.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.207+0000 7f59e3e00700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59c80b29f0 0x7f59c80b4f00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.207+0000 7f59e3e00700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4109670 0x7f59e4076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.207+0000 7f59e3e00700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59e41043f0 0x7f59e4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.207+0000 7f59e3e00700 1 --2- 172.21.15.63:0/1790599672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59e4108d20 0x7f59e4075ff0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:44.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.207+0000 7f59e3e00700 1 -- 172.21.15.63:0/1790599672 >> 172.21.15.63:0/1790599672 conn(0x7f59e4079d00 msgr2=0x7f59e4101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:44.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.207+0000 7f59e3e00700 1 -- 172.21.15.63:0/1790599672 shutdown_connections 2024-05-27T07:48:44.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:44.207+0000 7f59e3e00700 1 -- 172.21.15.63:0/1790599672 wait complete. 2024-05-27T07:48:45.251 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:48:45.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:45 smithi086 ceph-mon[34687]: pgmap v476: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:48:45.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:45 smithi149 ceph-mon[35408]: pgmap v476: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:48:45.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:45 smithi063 ceph-mon[31613]: pgmap v476: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:48:45.734 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:47.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:47 smithi086 ceph-mon[34687]: pgmap v477: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:48:47.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:47 smithi149 ceph-mon[35408]: pgmap v477: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:48:47.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:47 smithi063 ceph-mon[31613]: pgmap v477: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:48:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.212+0000 7fc0ee400700 1 -- 172.21.15.63:0/1630624653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0e8108560 msgr2=0x7fc0e810fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.212+0000 7fc0ee400700 1 --2- 172.21.15.63:0/1630624653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0e8108560 0x7fc0e810fec0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000b0d0 tx=0x7fc0e000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.212+0000 7fc0ee400700 1 -- 172.21.15.63:0/1630624653 shutdown_connections 2024-05-27T07:48:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.212+0000 7fc0ee400700 1 --2- 172.21.15.63:0/1630624653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0e8108560 0x7fc0e810fec0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.212+0000 7fc0ee400700 1 --2- 172.21.15.63:0/1630624653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e81032e0 0x7fc0e81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.212+0000 7fc0ee400700 1 --2- 172.21.15.63:0/1630624653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e8107c10 0x7fc0e8108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.212+0000 7fc0ee400700 1 -- 172.21.15.63:0/1630624653 >> 172.21.15.63:0/1630624653 conn(0x7fc0e8078af0 msgr2=0x7fc0e8078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.212+0000 7fc0ee400700 1 -- 172.21.15.63:0/1630624653 shutdown_connections 2024-05-27T07:48:48.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.212+0000 7fc0ee400700 1 -- 172.21.15.63:0/1630624653 wait complete. 2024-05-27T07:48:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.214+0000 7fc0ee400700 1 Processor -- start 2024-05-27T07:48:48.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.214+0000 7fc0ee400700 1 -- start start 2024-05-27T07:48:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0ee400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e81032e0 0x7fc0e819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0ee400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e8107c10 0x7fc0e819d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0ee400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0e8108560 0x7fc0e81a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0ee400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8113090 con 0x7fc0e8108560 2024-05-27T07:48:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0ee400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8112f10 con 0x7fc0e81032e0 2024-05-27T07:48:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0ee400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8113210 con 0x7fc0e8107c10 2024-05-27T07:48:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0eda00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0e8108560 0x7fc0e81a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0eda00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0e8108560 0x7fc0e81a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43864/0 (socket says 172.21.15.63:43864) 2024-05-27T07:48:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0eda00700 1 -- 172.21.15.63:0/2190416145 learned_addr learned my addr 172.21.15.63:0/2190416145 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0ed000700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e81032e0 0x7fc0e819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.215+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e8107c10 0x7fc0e819d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.216+0000 7fc0eda00700 1 -- 172.21.15.63:0/2190416145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e8107c10 msgr2=0x7fc0e819d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.216+0000 7fc0eda00700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e8107c10 0x7fc0e819d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.216+0000 7fc0eda00700 1 -- 172.21.15.63:0/2190416145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e81032e0 msgr2=0x7fc0e819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.216+0000 7fc0eda00700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e81032e0 0x7fc0e819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.216+0000 7fc0eda00700 1 -- 172.21.15.63:0/2190416145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e81a1cb0 con 0x7fc0e8108560 2024-05-27T07:48:48.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.216+0000 7fc0eda00700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0e8108560 0x7fc0e81a1610 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0007d10 tx=0x7fc0e0007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.217+0000 7fc0e5600700 1 -- 172.21.15.63:0/2190416145 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e001f070 con 0x7fc0e8108560 2024-05-27T07:48:48.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.217+0000 7fc0e5600700 1 -- 172.21.15.63:0/2190416145 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0e0003e10 con 0x7fc0e8108560 2024-05-27T07:48:48.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.217+0000 7fc0ee400700 1 -- 172.21.15.63:0/2190416145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e81a1f40 con 0x7fc0e8108560 2024-05-27T07:48:48.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.217+0000 7fc0e5600700 1 -- 172.21.15.63:0/2190416145 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e001b720 con 0x7fc0e8108560 2024-05-27T07:48:48.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.217+0000 7fc0ee400700 1 -- 172.21.15.63:0/2190416145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e81a2420 con 0x7fc0e8108560 2024-05-27T07:48:48.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.222+0000 7fc0e5600700 1 -- 172.21.15.63:0/2190416145 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc0e0022070 con 0x7fc0e8108560 2024-05-27T07:48:48.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.223+0000 7fc0fe5c9b80 1 Processor -- start 2024-05-27T07:48:48.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.223+0000 7fc0fe5c9b80 1 -- start start 2024-05-27T07:48:48.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.224+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c7659ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.224+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a410 0x5580c7664ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.224+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a820 0x5580c7667020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.224+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5580c7588e40 con 0x5580c74dbe40 2024-05-27T07:48:48.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.224+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5580c75789e0 con 0x5580c765a410 2024-05-27T07:48:48.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.224+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5580c757ca20 con 0x5580c765a820 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.224+0000 7fc0e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c7659ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.224+0000 7fc0e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c7659ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43868/0 (socket says 172.21.15.63:43868) 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.224+0000 7fc0e7e00700 1 -- 172.21.15.63:0/3648281072 learned_addr learned my addr 172.21.15.63:0/3648281072 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.224+0000 7fc0ed000700 1 --2- 172.21.15.63:0/3648281072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a410 0x5580c7664ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.225+0000 7fc0e5600700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d00b29f0 0x7fc0d00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.225+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/3648281072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a820 0x5580c7667020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.225+0000 7fc0e7e00700 1 -- 172.21.15.63:0/3648281072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a820 msgr2=0x5580c7667020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.225+0000 7fc0e5600700 1 -- 172.21.15.63:0/2190416145 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fc0e000f020 con 0x7fc0e8108560 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.225+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/3648281072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a820 0x5580c7667020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.225+0000 7fc0e7e00700 1 -- 172.21.15.63:0/3648281072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a410 msgr2=0x5580c7664ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.225+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/3648281072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a410 0x5580c7664ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.225+0000 7fc0e7e00700 1 -- 172.21.15.63:0/3648281072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c7541420 con 0x5580c74dbe40 2024-05-27T07:48:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.225+0000 7fc0eda00700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d00b29f0 0x7fc0d00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.226+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/3648281072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c7659ed0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400b6c0 tx=0x7fc0d400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681b07bd6cc7e84f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.226+0000 7fc0eda00700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d00b29f0 0x7fc0d00b4f00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00e1200 tx=0x7fc0e0005bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.226+0000 7fc0da000700 1 -- 172.21.15.63:0/3648281072 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4004030 con 0x5580c74dbe40 2024-05-27T07:48:48.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.226+0000 7fc0da000700 1 -- 172.21.15.63:0/3648281072 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0d4004830 con 0x5580c74dbe40 2024-05-27T07:48:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.226+0000 7fc0da000700 1 -- 172.21.15.63:0/3648281072 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4005280 con 0x5580c74dbe40 2024-05-27T07:48:48.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.227+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/3648281072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 msgr2=0x5580c7659ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.227+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/3648281072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c7659ed0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400b6c0 tx=0x7fc0d400b7e0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.227+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/3648281072 shutdown_connections 2024-05-27T07:48:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.227+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/3648281072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a820 0x5580c7667020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.227+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/3648281072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c7659ed0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.227+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/3648281072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a410 0x5580c7664ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.227+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/3648281072 >> 172.21.15.63:0/3648281072 conn(0x5580c753bfc0 msgr2=0x5580c7655d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.228+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/3648281072 shutdown_connections 2024-05-27T07:48:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.228+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/3648281072 wait complete. 2024-05-27T07:48:48.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.228+0000 7fc0fe5c9b80 1 -- start start 2024-05-27T07:48:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.229+0000 7fc0fe5c9b80 1 Processor -- start 2024-05-27T07:48:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.229+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c74dbe40 0x5580c7664620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.229+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a410 0x5580c7664b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.229+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a820 0x5580c76650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.229+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5580c747efa0 con 0x5580c765a410 2024-05-27T07:48:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.229+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5580c7519760 con 0x5580c765a820 2024-05-27T07:48:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.229+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5580c7667990 con 0x5580c74dbe40 2024-05-27T07:48:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.229+0000 7fc0fe5c9b80 10 client.0 _ll_register_callbacks cb 0x5580c7522c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0ed000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a410 0x5580c7664b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0ed000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a410 0x5580c7664b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43874/0 (socket says 172.21.15.63:43874) 2024-05-27T07:48:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0ed000700 1 -- 172.21.15.63:0/4083393515 learned_addr learned my addr 172.21.15.63:0/4083393515 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:48.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c74dbe40 0x5580c7664620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a820 0x5580c76650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0ed000700 1 -- 172.21.15.63:0/4083393515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c74dbe40 msgr2=0x5580c7664620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0ed000700 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c74dbe40 0x5580c7664620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0ed000700 1 -- 172.21.15.63:0/4083393515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a820 msgr2=0x5580c76650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0ed000700 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a820 0x5580c76650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0ed000700 1 -- 172.21.15.63:0/4083393515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c74c88d0 con 0x5580c765a410 2024-05-27T07:48:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.230+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a820 0x5580c76650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:48.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.231+0000 7fc0ed000700 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a410 0x5580c7664b60 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00bac0 tx=0x7fc0dc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69e9097add70093 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.231+0000 7fc0cea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.231+0000 7fc0cea00700 1 -- 172.21.15.63:0/4083393515 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc01b440 con 0x5580c765a410 2024-05-27T07:48:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.231+0000 7fc0cea00700 1 -- 172.21.15.63:0/4083393515 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fc0dc01b5e0 con 0x5580c765a410 2024-05-27T07:48:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.231+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/4083393515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d4004830 con 0x5580c765a410 2024-05-27T07:48:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.232+0000 7fc0cea00700 1 -- 172.21.15.63:0/4083393515 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc0dc024050 con 0x5580c765a410 2024-05-27T07:48:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.232+0000 7fc0fe5c9b80 10 client.18342 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:48.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.232+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/4083393515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5580c7549060 con 0x5580c765a410 2024-05-27T07:48:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.232+0000 7fc0cea00700 1 -- 172.21.15.63:0/4083393515 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0d4004830 con 0x5580c765a410 2024-05-27T07:48:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.232+0000 7fc0cea00700 1 -- 172.21.15.63:0/4083393515 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc0dc019020 con 0x5580c765a410 2024-05-27T07:48:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.232+0000 7fc0fe5c9b80 10 client.18342 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.232+0000 7fc0fe5c9b80 10 client.18342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:48:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.233+0000 7fc0fe5c9b80 20 client.18342 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.233+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5580c7674a00 0x5580c7676e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.233+0000 7fc0fe5c9b80 4 client.18342 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:48.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.233+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/4083393515 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580c758ad90 con 0x5580c7674a00 2024-05-27T07:48:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.233+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5580c7674a00 0x5580c7676e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.234+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5580c7674a00 0x5580c7676e50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1083ac402769f26 server_cookie=efeb1807a74490e6 in_seq=0 out_seq=0 2024-05-27T07:48:48.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.234+0000 7fc0cea00700 10 client.18342 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:48:48.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.236+0000 7fc0cea00700 1 -- 172.21.15.63:0/4083393515 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5580c758ad90 con 0x5580c7674a00 2024-05-27T07:48:48.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.236+0000 7fc0cea00700 10 client.18342 handle_command_reply: tid=0 2024-05-27T07:48:48.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.236+0000 7fc0fe5c9b80 1 client.18342 shutdown 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/4083393515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a410 msgr2=0x5580c7664b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a410 0x5580c7664b60 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00bac0 tx=0x7fc0dc00be80 comp rx=0 tx=0).stop 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/4083393515 shutdown_connections 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a820 0x5580c76650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c74dbe40 0x5580c7664620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5580c7674a00 0x5580c7676e50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/4083393515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a410 0x5580c7664b60 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/4083393515 >> 172.21.15.63:0/4083393515 conn(0x5580c753bfc0 msgr2=0x5580c7654ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0cea00700 0 client.18342 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/4083393515 shutdown_connections 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/4083393515 wait complete. 2024-05-27T07:48:48.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.237+0000 7fc0fe5c9b80 20 client.18342 trim_cache size 0 max 16384 2024-05-27T07:48:48.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.258+0000 7fc0fe5c9b80 1 Processor -- start 2024-05-27T07:48:48.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.258+0000 7fc0fe5c9b80 1 -- start start 2024-05-27T07:48:48.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c74dbe40 0x5580c7655ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a410 0x5580c7666a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a820 0x5580c765e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81b4ce0 con 0x5580c765a820 2024-05-27T07:48:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81b2570 con 0x5580c74dbe40 2024-05-27T07:48:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81b03e0 con 0x5580c765a410 2024-05-27T07:48:48.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a820 0x5580c765e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a820 0x5580c765e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43888/0 (socket says 172.21.15.63:43888) 2024-05-27T07:48:48.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0e7e00700 1 -- 172.21.15.63:0/2757086147 learned_addr learned my addr 172.21.15.63:0/2757086147 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:48.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0ed000700 1 --2- 172.21.15.63:0/2757086147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a410 0x5580c7666a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.259+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/2757086147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c74dbe40 0x5580c7655ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.260+0000 7fc0e7e00700 1 -- 172.21.15.63:0/2757086147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a410 msgr2=0x5580c7666a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.260+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/2757086147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a410 0x5580c7666a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.260+0000 7fc0e7e00700 1 -- 172.21.15.63:0/2757086147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c74dbe40 msgr2=0x5580c7655ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.260+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/2757086147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c74dbe40 0x5580c7655ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.260+0000 7fc0e7e00700 1 -- 172.21.15.63:0/2757086147 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c7657350 con 0x5580c765a820 2024-05-27T07:48:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.260+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/2757086147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a820 0x5580c765e7a0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400b690 tx=0x7fc0d400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8906ddba1fad4a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.261+0000 7fc0d9600700 1 -- 172.21.15.63:0/2757086147 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4004c60 con 0x5580c765a820 2024-05-27T07:48:48.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.261+0000 7fc0d9600700 1 -- 172.21.15.63:0/2757086147 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0d4005490 con 0x5580c765a820 2024-05-27T07:48:48.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.261+0000 7fc0d9600700 1 -- 172.21.15.63:0/2757086147 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40106c0 con 0x5580c765a820 2024-05-27T07:48:48.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.262+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/2757086147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a820 msgr2=0x5580c765e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.262+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/2757086147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a820 0x5580c765e7a0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400b690 tx=0x7fc0d400b7e0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.262+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/2757086147 shutdown_connections 2024-05-27T07:48:48.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.262+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/2757086147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c765a820 0x5580c765e7a0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.262+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/2757086147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c74dbe40 0x5580c7655ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.262+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/2757086147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a410 0x5580c7666a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.262+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/2757086147 >> 172.21.15.63:0/2757086147 conn(0x5580c753bfc0 msgr2=0x5580c76557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.262+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/2757086147 shutdown_connections 2024-05-27T07:48:48.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.262+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/2757086147 wait complete. 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.263+0000 7fc0fe5c9b80 1 -- start start 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.263+0000 7fc0fe5c9b80 1 Processor -- start 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c766d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a410 0x5580c7667c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0fe5c9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a820 0x5580c76682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81b0a30 con 0x5580c74dbe40 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81b1ea0 con 0x5580c765a410 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0fe5c9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81b1100 con 0x5580c765a820 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0fe5c9b80 10 client.0 _ll_register_callbacks cb 0x5580c7522c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c766d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c766d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43896/0 (socket says 172.21.15.63:43896) 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0e7e00700 1 -- 172.21.15.63:0/1814458835 learned_addr learned my addr 172.21.15.63:0/1814458835 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0ed000700 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a410 0x5580c7667c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.264+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a820 0x5580c76682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.265+0000 7fc0e7e00700 1 -- 172.21.15.63:0/1814458835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a820 msgr2=0x5580c76682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.265+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a820 0x5580c76682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.265+0000 7fc0e7e00700 1 -- 172.21.15.63:0/1814458835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a410 msgr2=0x5580c7667c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.265+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a410 0x5580c7667c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.265+0000 7fc0e7e00700 1 -- 172.21.15.63:0/1814458835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c76dd560 con 0x5580c74dbe40 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.265+0000 7fc0e7e00700 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c766d4b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4012b80 tx=0x7fc0d400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90997e1357e4cc01 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.265+0000 7fc0cea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.265+0000 7fc0cea00700 1 -- 172.21.15.63:0/1814458835 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4004c60 con 0x5580c74dbe40 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.266+0000 7fc0cea00700 1 -- 172.21.15.63:0/1814458835 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fc0d4004f80 con 0x5580c74dbe40 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.266+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/1814458835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d4005490 con 0x5580c74dbe40 2024-05-27T07:48:48.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.266+0000 7fc0cea00700 1 -- 172.21.15.63:0/1814458835 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc0d403d050 con 0x5580c74dbe40 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.266+0000 7fc0fe5c9b80 10 client.18348 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.266+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/1814458835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5580c76de650 con 0x5580c74dbe40 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.266+0000 7fc0cea00700 1 -- 172.21.15.63:0/1814458835 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0d4005490 con 0x5580c74dbe40 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.267+0000 7fc0cea00700 1 -- 172.21.15.63:0/1814458835 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc0d4010c60 con 0x5580c74dbe40 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.267+0000 7fc0fe5c9b80 10 client.18348 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.267+0000 7fc0fe5c9b80 10 client.18348 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.267+0000 7fc0fe5c9b80 20 client.18348 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.267+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5580c76dea30 0x5580c76e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.267+0000 7fc0fe5c9b80 4 client.18348 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.267+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/1814458835 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580c7584570 con 0x5580c76dea30 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.267+0000 7fc0ed000700 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5580c76dea30 0x5580c76e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.268+0000 7fc0ed000700 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5580c76dea30 0x5580c76e0e50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcddf57fecccea0b server_cookie=d903b72021cbfffd in_seq=0 out_seq=0 2024-05-27T07:48:48.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.268+0000 7fc0cea00700 10 client.18348 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0cea00700 1 -- 172.21.15.63:0/1814458835 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580c7584570 con 0x5580c76dea30 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0cea00700 10 client.18348 handle_command_reply: tid=0 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0fe5c9b80 1 client.18348 shutdown 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/1814458835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 msgr2=0x5580c766d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c766d4b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4012b80 tx=0x7fc0d400b410 comp rx=0 tx=0).stop 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/1814458835 shutdown_connections 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580c765a820 0x5580c76682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5580c76dea30 0x5580c76e0e50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580c74dbe40 0x5580c766d4b0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0fe5c9b80 1 --2- 172.21.15.63:0/1814458835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c765a410 0x5580c7667c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0cea00700 0 client.18348 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.269+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/1814458835 >> 172.21.15.63:0/1814458835 conn(0x5580c753bfc0 msgr2=0x5580c7664400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.270+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/1814458835 shutdown_connections 2024-05-27T07:48:48.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.270+0000 7fc0fe5c9b80 1 -- 172.21.15.63:0/1814458835 wait complete. 2024-05-27T07:48:48.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.270+0000 7fc0fe5c9b80 20 client.18348 trim_cache size 0 max 16384 2024-05-27T07:48:48.275 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:48:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.271+0000 7fc0ee400700 1 -- 172.21.15.63:0/2190416145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d00b29f0 msgr2=0x7fc0d00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.271+0000 7fc0ee400700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d00b29f0 0x7fc0d00b4f00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00e1200 tx=0x7fc0e0005bc0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.271+0000 7fc0ee400700 1 -- 172.21.15.63:0/2190416145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0e8108560 msgr2=0x7fc0e81a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.271+0000 7fc0ee400700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0e8108560 0x7fc0e81a1610 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0007d10 tx=0x7fc0e0007d40 comp rx=0 tx=0).stop 2024-05-27T07:48:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.271+0000 7fc0ee400700 1 -- 172.21.15.63:0/2190416145 shutdown_connections 2024-05-27T07:48:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.271+0000 7fc0ee400700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d00b29f0 0x7fc0d00b4f00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.271+0000 7fc0ee400700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0e8108560 0x7fc0e81a1610 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.271+0000 7fc0ee400700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0e81032e0 0x7fc0e819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.271+0000 7fc0ee400700 1 --2- 172.21.15.63:0/2190416145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0e8107c10 0x7fc0e819d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.271+0000 7fc0ee400700 1 -- 172.21.15.63:0/2190416145 >> 172.21.15.63:0/2190416145 conn(0x7fc0e8078af0 msgr2=0x7fc0e810ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.272+0000 7fc0ee400700 1 -- 172.21.15.63:0/2190416145 shutdown_connections 2024-05-27T07:48:48.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:48.272+0000 7fc0ee400700 1 -- 172.21.15.63:0/2190416145 wait complete. 2024-05-27T07:48:49.315 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:48:49.391 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:49 smithi063 ceph-mon[31613]: pgmap v478: 129 pgs: 129 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:48:49.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:49 smithi086 ceph-mon[34687]: pgmap v478: 129 pgs: 129 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:48:49.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:49 smithi149 ceph-mon[35408]: pgmap v478: 129 pgs: 129 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:48:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:51.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:51 smithi086 ceph-mon[34687]: pgmap v479: 129 pgs: 129 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:48:51.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:51 smithi149 ceph-mon[35408]: pgmap v479: 129 pgs: 129 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:48:51.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:51 smithi063 ceph-mon[31613]: pgmap v479: 129 pgs: 129 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:48:52.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.147+0000 7f132ae00700 1 -- 172.21.15.63:0/4000242469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1324105490 msgr2=0x7f13241058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.147+0000 7f132ae00700 1 --2- 172.21.15.63:0/4000242469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1324105490 0x7f13241058a0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f1314009990 tx=0x7f1314009ca0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.148+0000 7f132ae00700 1 -- 172.21.15.63:0/4000242469 shutdown_connections 2024-05-27T07:48:52.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.148+0000 7f132ae00700 1 --2- 172.21.15.63:0/4000242469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f132410e900 0x7f1324110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.148+0000 7f132ae00700 1 --2- 172.21.15.63:0/4000242469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1324105490 0x7f13241058a0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.148+0000 7f132ae00700 1 --2- 172.21.15.63:0/4000242469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324105de0 0x7f132410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.148+0000 7f132ae00700 1 -- 172.21.15.63:0/4000242469 >> 172.21.15.63:0/4000242469 conn(0x7f13240fedb0 msgr2=0x7f13241011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.148+0000 7f132ae00700 1 -- 172.21.15.63:0/4000242469 shutdown_connections 2024-05-27T07:48:52.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.148+0000 7f132ae00700 1 -- 172.21.15.63:0/4000242469 wait complete. 2024-05-27T07:48:52.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.149+0000 7f132ae00700 1 Processor -- start 2024-05-27T07:48:52.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.150+0000 7f132ae00700 1 -- start start 2024-05-27T07:48:52.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.150+0000 7f132ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324105490 0x7f132419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.150+0000 7f132ae00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1324105de0 0x7f132419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.150+0000 7f132ae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f132410e900 0x7f13241a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.150+0000 7f132ae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1324114350 con 0x7f132410e900 2024-05-27T07:48:52.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.150+0000 7f132ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13241141d0 con 0x7f1324105490 2024-05-27T07:48:52.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.150+0000 7f132ae00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13241144d0 con 0x7f1324105de0 2024-05-27T07:48:52.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.151+0000 7f1328e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f132410e900 0x7f13241a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.151+0000 7f1328e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f132410e900 0x7f13241a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38844/0 (socket says 172.21.15.63:38844) 2024-05-27T07:48:52.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.151+0000 7f1328e00700 1 -- 172.21.15.63:0/2564986756 learned_addr learned my addr 172.21.15.63:0/2564986756 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:52.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.151+0000 7f1323e00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324105490 0x7f132419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.151+0000 7f1323400700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1324105de0 0x7f132419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.151+0000 7f1328e00700 1 -- 172.21.15.63:0/2564986756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1324105de0 msgr2=0x7f132419d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.151+0000 7f1328e00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1324105de0 0x7f132419d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.151+0000 7f1328e00700 1 -- 172.21.15.63:0/2564986756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324105490 msgr2=0x7f132419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.151+0000 7f1328e00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324105490 0x7f132419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.151+0000 7f1328e00700 1 -- 172.21.15.63:0/2564986756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13241a1e00 con 0x7f132410e900 2024-05-27T07:48:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.152+0000 7f1328e00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f132410e900 0x7f13241a1760 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f131c007ee0 tx=0x7f131c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.152+0000 7f1320c00700 1 -- 172.21.15.63:0/2564986756 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c00dbd0 con 0x7f132410e900 2024-05-27T07:48:52.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.152+0000 7f1320c00700 1 -- 172.21.15.63:0/2564986756 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f131c00dd70 con 0x7f132410e900 2024-05-27T07:48:52.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.152+0000 7f132ae00700 1 -- 172.21.15.63:0/2564986756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13241a20f0 con 0x7f132410e900 2024-05-27T07:48:52.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.152+0000 7f132ae00700 1 -- 172.21.15.63:0/2564986756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13241a9a10 con 0x7f132410e900 2024-05-27T07:48:52.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.153+0000 7f1320c00700 1 -- 172.21.15.63:0/2564986756 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c0058d0 con 0x7f132410e900 2024-05-27T07:48:52.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.157+0000 7f1320c00700 1 -- 172.21.15.63:0/2564986756 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f131c01a020 con 0x7f132410e900 2024-05-27T07:48:52.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.158+0000 7f13398bdb80 1 Processor -- start 2024-05-27T07:48:52.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.158+0000 7f13398bdb80 1 -- start start 2024-05-27T07:48:52.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612330799f0 0x56123310c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x56123310c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123310ce70 0x561233119110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f13398bdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56123303ae40 con 0x561232f8de40 2024-05-27T07:48:52.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f13398bdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56123302a9e0 con 0x56123310ce70 2024-05-27T07:48:52.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f13398bdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56123302ea20 con 0x5612330799f0 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f1320c00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13080b29f0 0x7f13080b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f1323e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x56123310c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f1323400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612330799f0 0x56123310c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f1323e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x56123310c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38852/0 (socket says 172.21.15.63:38852) 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f1323e00700 1 -- 172.21.15.63:0/2988874118 learned_addr learned my addr 172.21.15.63:0/2988874118 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.159+0000 7f1328e00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13080b29f0 0x7f13080b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.160+0000 7f1323400700 1 --2- 172.21.15.63:0/2988874118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123310ce70 0x561233119110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.160+0000 7f1320c00700 1 -- 172.21.15.63:0/2564986756 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f131c0e2190 con 0x7f132410e900 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.160+0000 7f1323e00700 1 -- 172.21.15.63:0/2988874118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612330799f0 msgr2=0x56123310c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.160+0000 7f1323e00700 1 --2- 172.21.15.63:0/2988874118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612330799f0 0x56123310c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.160+0000 7f1323e00700 1 -- 172.21.15.63:0/2988874118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123310ce70 msgr2=0x561233119110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.160+0000 7f1323e00700 1 --2- 172.21.15.63:0/2988874118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123310ce70 0x561233119110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.160+0000 7f1323e00700 1 -- 172.21.15.63:0/2988874118 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561232ff3420 con 0x561232f8de40 2024-05-27T07:48:52.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.160+0000 7f1323e00700 1 --2- 172.21.15.63:0/2988874118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x56123310c870 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f1314003730 tx=0x7f1314005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1da521a015f8a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.161+0000 7f1328e00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13080b29f0 0x7f13080b4f00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f131c004ec0 tx=0x7f131c004e10 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.161+0000 7f1319600700 1 -- 172.21.15.63:0/2988874118 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314005390 con 0x561232f8de40 2024-05-27T07:48:52.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.161+0000 7f1319600700 1 -- 172.21.15.63:0/2988874118 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1314005530 con 0x561232f8de40 2024-05-27T07:48:52.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.161+0000 7f1319600700 1 -- 172.21.15.63:0/2988874118 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131400bc30 con 0x561232f8de40 2024-05-27T07:48:52.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.162+0000 7f13398bdb80 1 -- 172.21.15.63:0/2988874118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 msgr2=0x56123310c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.162+0000 7f13398bdb80 1 --2- 172.21.15.63:0/2988874118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x56123310c870 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f1314003730 tx=0x7f1314005f00 comp rx=0 tx=0).stop 2024-05-27T07:48:52.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.162+0000 7f13398bdb80 1 -- 172.21.15.63:0/2988874118 shutdown_connections 2024-05-27T07:48:52.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.162+0000 7f13398bdb80 1 --2- 172.21.15.63:0/2988874118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123310ce70 0x561233119110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.162+0000 7f13398bdb80 1 --2- 172.21.15.63:0/2988874118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612330799f0 0x56123310c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.162+0000 7f13398bdb80 1 --2- 172.21.15.63:0/2988874118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x56123310c870 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.162+0000 7f13398bdb80 1 -- 172.21.15.63:0/2988874118 >> 172.21.15.63:0/2988874118 conn(0x561233027390 msgr2=0x5612331084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.162+0000 7f13398bdb80 1 -- 172.21.15.63:0/2988874118 shutdown_connections 2024-05-27T07:48:52.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.162+0000 7f13398bdb80 1 -- 172.21.15.63:0/2988874118 wait complete. 2024-05-27T07:48:52.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.163+0000 7f13398bdb80 1 -- start start 2024-05-27T07:48:52.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.163+0000 7f13398bdb80 1 Processor -- start 2024-05-27T07:48:52.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.164+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x5612331165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.164+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 0x561233116b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.164+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56123310ce70 0x561233117060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.164+0000 7f13398bdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561232f30fa0 con 0x561232f8de40 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.164+0000 7f13398bdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561232fcb760 con 0x5612330799f0 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.164+0000 7f13398bdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561233119ac0 con 0x56123310ce70 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.164+0000 7f13398bdb80 10 client.0 _ll_register_callbacks cb 0x561232fd4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.164+0000 7f1323400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x5612331165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.165+0000 7f1323400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x5612331165e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38864/0 (socket says 172.21.15.63:38864) 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.165+0000 7f1323400700 1 -- 172.21.15.63:0/3575495422 learned_addr learned my addr 172.21.15.63:0/3575495422 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.165+0000 7f1323e00700 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 0x561233116b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.165+0000 7f1323400700 1 -- 172.21.15.63:0/3575495422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56123310ce70 msgr2=0x561233117060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.165+0000 7f1323400700 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56123310ce70 0x561233117060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.165+0000 7f1323400700 1 -- 172.21.15.63:0/3575495422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 msgr2=0x561233116b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.165+0000 7f1323400700 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 0x561233116b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.165+0000 7f1323400700 1 -- 172.21.15.63:0/3575495422 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561232f7a8d0 con 0x561232f8de40 2024-05-27T07:48:52.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.165+0000 7f1323e00700 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 0x561233116b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:52.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.165+0000 7f1323400700 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x5612331165e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f1310004590 tx=0x7f1310003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18994f15900be5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.166+0000 7f130e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:52.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.166+0000 7f130e000700 1 -- 172.21.15.63:0/3575495422 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13100157f0 con 0x561232f8de40 2024-05-27T07:48:52.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.166+0000 7f13398bdb80 1 -- 172.21.15.63:0/3575495422 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1314005530 con 0x561232f8de40 2024-05-27T07:48:52.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.166+0000 7f130e000700 1 -- 172.21.15.63:0/3575495422 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f1310015990 con 0x561232f8de40 2024-05-27T07:48:52.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.166+0000 7f130e000700 1 -- 172.21.15.63:0/3575495422 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1310028050 con 0x561232f8de40 2024-05-27T07:48:52.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.166+0000 7f13398bdb80 10 client.18360 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:52.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.166+0000 7f13398bdb80 1 -- 172.21.15.63:0/3575495422 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561232ffb060 con 0x561232f8de40 2024-05-27T07:48:52.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.167+0000 7f130e000700 1 -- 172.21.15.63:0/3575495422 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1314005530 con 0x561232f8de40 2024-05-27T07:48:52.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.167+0000 7f130e000700 1 -- 172.21.15.63:0/3575495422 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1310012bc0 con 0x561232f8de40 2024-05-27T07:48:52.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.167+0000 7f13398bdb80 10 client.18360 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:52.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.167+0000 7f13398bdb80 10 client.18360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:52.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.167+0000 7f13398bdb80 20 client.18360 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:52.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.167+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561233126980 0x561233128da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.167+0000 7f13398bdb80 4 client.18360 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:52.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.167+0000 7f13398bdb80 1 -- 172.21.15.63:0/3575495422 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56123303cd90 con 0x561233126980 2024-05-27T07:48:52.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.168+0000 7f1323e00700 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561233126980 0x561233128da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.169+0000 7f1323e00700 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561233126980 0x561233128da0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b156e9ddbccef745 server_cookie=3025e3d1ce890564 in_seq=0 out_seq=0 2024-05-27T07:48:52.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.169+0000 7f130e000700 10 client.18360 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:52.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f130e000700 1 -- 172.21.15.63:0/3575495422 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56123303cd90 con 0x561233126980 2024-05-27T07:48:52.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f130e000700 10 client.18360 handle_command_reply: tid=0 2024-05-27T07:48:52.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 client.18360 shutdown 2024-05-27T07:48:52.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 -- 172.21.15.63:0/3575495422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 msgr2=0x5612331165e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x5612331165e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f1310004590 tx=0x7f1310003be0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 -- 172.21.15.63:0/3575495422 shutdown_connections 2024-05-27T07:48:52.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561233126980 0x561233128da0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56123310ce70 0x561233117060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561232f8de40 0x5612331165e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3575495422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 0x561233116b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 -- 172.21.15.63:0/3575495422 >> 172.21.15.63:0/3575495422 conn(0x561233027390 msgr2=0x56123310c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f130e000700 0 client.18360 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:52.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 -- 172.21.15.63:0/3575495422 shutdown_connections 2024-05-27T07:48:52.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.171+0000 7f13398bdb80 1 -- 172.21.15.63:0/3575495422 wait complete. 2024-05-27T07:48:52.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.172+0000 7f13398bdb80 20 client.18360 trim_cache size 0 max 16384 2024-05-27T07:48:52.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.189+0000 7f13398bdb80 1 Processor -- start 2024-05-27T07:48:52.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.189+0000 7f13398bdb80 1 -- start start 2024-05-27T07:48:52.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.189+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561232f8de40 0x561233129000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.189+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 0x561233111d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.189+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56123310ce70 0x56123310c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.189+0000 7f13398bdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13241b5000 con 0x56123310ce70 2024-05-27T07:48:52.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.189+0000 7f13398bdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13241b2890 con 0x5612330799f0 2024-05-27T07:48:52.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.189+0000 7f13398bdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13241b0700 con 0x561232f8de40 2024-05-27T07:48:52.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56123310ce70 0x56123310c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56123310ce70 0x56123310c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38868/0 (socket says 172.21.15.63:38868) 2024-05-27T07:48:52.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323400700 1 -- 172.21.15.63:0/2700441662 learned_addr learned my addr 172.21.15.63:0/2700441662 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:52.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323e00700 1 --2- 172.21.15.63:0/2700441662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 0x561233111d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323400700 1 --2- 172.21.15.63:0/2700441662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561232f8de40 0x561233129000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323400700 1 -- 172.21.15.63:0/2700441662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561232f8de40 msgr2=0x561233129000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323400700 1 --2- 172.21.15.63:0/2700441662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561232f8de40 0x561233129000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323400700 1 -- 172.21.15.63:0/2700441662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 msgr2=0x561233111d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323400700 1 --2- 172.21.15.63:0/2700441662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 0x561233111d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323400700 1 -- 172.21.15.63:0/2700441662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561233114b50 con 0x56123310ce70 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.190+0000 7f1323e00700 1 --2- 172.21.15.63:0/2700441662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 0x561233111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.191+0000 7f1323400700 1 --2- 172.21.15.63:0/2700441662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56123310ce70 0x56123310c7c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f1310001eb0 tx=0x7f1310017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc87b9567316abac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.191+0000 7f1318c00700 1 -- 172.21.15.63:0/2700441662 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131000c3b0 con 0x56123310ce70 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.191+0000 7f1318c00700 1 -- 172.21.15.63:0/2700441662 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1310034070 con 0x56123310ce70 2024-05-27T07:48:52.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.191+0000 7f1318c00700 1 -- 172.21.15.63:0/2700441662 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310014bb0 con 0x56123310ce70 2024-05-27T07:48:52.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.192+0000 7f13398bdb80 1 -- 172.21.15.63:0/2700441662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56123310ce70 msgr2=0x56123310c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.192+0000 7f13398bdb80 1 --2- 172.21.15.63:0/2700441662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56123310ce70 0x56123310c7c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f1310001eb0 tx=0x7f1310017040 comp rx=0 tx=0).stop 2024-05-27T07:48:52.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.192+0000 7f13398bdb80 1 -- 172.21.15.63:0/2700441662 shutdown_connections 2024-05-27T07:48:52.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.192+0000 7f13398bdb80 1 --2- 172.21.15.63:0/2700441662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56123310ce70 0x56123310c7c0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.192+0000 7f13398bdb80 1 --2- 172.21.15.63:0/2700441662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561232f8de40 0x561233129000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.192+0000 7f13398bdb80 1 --2- 172.21.15.63:0/2700441662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612330799f0 0x561233111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.192+0000 7f13398bdb80 1 -- 172.21.15.63:0/2700441662 >> 172.21.15.63:0/2700441662 conn(0x561233027390 msgr2=0x56123310c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.192+0000 7f13398bdb80 1 -- 172.21.15.63:0/2700441662 shutdown_connections 2024-05-27T07:48:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.193+0000 7f13398bdb80 1 -- 172.21.15.63:0/2700441662 wait complete. 2024-05-27T07:48:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.193+0000 7f13398bdb80 1 -- start start 2024-05-27T07:48:52.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.193+0000 7f13398bdb80 1 Processor -- start 2024-05-27T07:48:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.194+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561232f8de40 0x56123311a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.194+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612330799f0 0x56123311a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.194+0000 7f13398bdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123318d450 0x56123318f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.194+0000 7f13398bdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13241b0d50 con 0x5612330799f0 2024-05-27T07:48:52.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.194+0000 7f13398bdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13241b21c0 con 0x56123318d450 2024-05-27T07:48:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.194+0000 7f13398bdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13241b1420 con 0x561232f8de40 2024-05-27T07:48:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.194+0000 7f13398bdb80 10 client.0 _ll_register_callbacks cb 0x561232fd4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.194+0000 7f1323e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612330799f0 0x56123311a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.195+0000 7f1323e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612330799f0 0x56123311a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38878/0 (socket says 172.21.15.63:38878) 2024-05-27T07:48:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.195+0000 7f1323e00700 1 -- 172.21.15.63:0/3903234183 learned_addr learned my addr 172.21.15.63:0/3903234183 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.195+0000 7f1323400700 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123318d450 0x56123318f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.195+0000 7f1323400700 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561232f8de40 0x56123311a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.195+0000 7f1323e00700 1 -- 172.21.15.63:0/3903234183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561232f8de40 msgr2=0x56123311a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.195+0000 7f1323e00700 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561232f8de40 0x56123311a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.195+0000 7f1323e00700 1 -- 172.21.15.63:0/3903234183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123318d450 msgr2=0x56123318f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.195+0000 7f1323e00700 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123318d450 0x56123318f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.195+0000 7f1323e00700 1 -- 172.21.15.63:0/3903234183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56123318fdb0 con 0x5612330799f0 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.196+0000 7f1323e00700 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612330799f0 0x56123311a880 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f131400b640 tx=0x7f131400b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a931aee4d4de364a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.196+0000 7f130e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.196+0000 7f130e000700 1 -- 172.21.15.63:0/3903234183 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314005410 con 0x5612330799f0 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.196+0000 7f13398bdb80 1 -- 172.21.15.63:0/3903234183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1310034070 con 0x5612330799f0 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.196+0000 7f130e000700 1 -- 172.21.15.63:0/3903234183 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f13140037c0 con 0x5612330799f0 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.196+0000 7f130e000700 1 -- 172.21.15.63:0/3903234183 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f13140039c0 con 0x5612330799f0 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.196+0000 7f13398bdb80 10 client.18366 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.196+0000 7f13398bdb80 1 -- 172.21.15.63:0/3903234183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561233190fb0 con 0x5612330799f0 2024-05-27T07:48:52.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.197+0000 7f130e000700 1 -- 172.21.15.63:0/3903234183 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1310034070 con 0x5612330799f0 2024-05-27T07:48:52.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.197+0000 7f130e000700 1 -- 172.21.15.63:0/3903234183 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f131400f460 con 0x5612330799f0 2024-05-27T07:48:52.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.197+0000 7f13398bdb80 10 client.18366 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:52.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.197+0000 7f13398bdb80 10 client.18366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:52.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.197+0000 7f13398bdb80 20 client.18366 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:52.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.197+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612331911e0 0x561233193600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.197+0000 7f13398bdb80 4 client.18366 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:52.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.197+0000 7f13398bdb80 1 -- 172.21.15.63:0/3903234183 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561233036570 con 0x5612331911e0 2024-05-27T07:48:52.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.198+0000 7f1323400700 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612331911e0 0x561233193600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.199+0000 7f1323400700 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612331911e0 0x561233193600 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6e93453b533ebb0 server_cookie=656663e6b4b98462 in_seq=0 out_seq=0 2024-05-27T07:48:52.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.199+0000 7f130e000700 10 client.18366 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:52.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.199+0000 7f130e000700 1 -- 172.21.15.63:0/3903234183 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561233036570 con 0x5612331911e0 2024-05-27T07:48:52.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.199+0000 7f130e000700 10 client.18366 handle_command_reply: tid=0 2024-05-27T07:48:52.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.199+0000 7f13398bdb80 1 client.18366 shutdown 2024-05-27T07:48:52.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.199+0000 7f13398bdb80 1 -- 172.21.15.63:0/3903234183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612330799f0 msgr2=0x56123311a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.199+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612330799f0 0x56123311a880 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f131400b640 tx=0x7f131400b5d0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.200+0000 7f13398bdb80 1 -- 172.21.15.63:0/3903234183 shutdown_connections 2024-05-27T07:48:52.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.200+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612331911e0 0x561233193600 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.200+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56123318d450 0x56123318f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.200+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561232f8de40 0x56123311a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.200+0000 7f13398bdb80 1 --2- 172.21.15.63:0/3903234183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612330799f0 0x56123311a880 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.200+0000 7f130e000700 0 client.18366 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:52.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.200+0000 7f13398bdb80 1 -- 172.21.15.63:0/3903234183 >> 172.21.15.63:0/3903234183 conn(0x561233027390 msgr2=0x561233117c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.200+0000 7f13398bdb80 1 -- 172.21.15.63:0/3903234183 shutdown_connections 2024-05-27T07:48:52.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.200+0000 7f13398bdb80 1 -- 172.21.15.63:0/3903234183 wait complete. 2024-05-27T07:48:52.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.200+0000 7f13398bdb80 20 client.18366 trim_cache size 0 max 16384 2024-05-27T07:48:52.201 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:48:52.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.201+0000 7f132ae00700 1 -- 172.21.15.63:0/2564986756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13080b29f0 msgr2=0x7f13080b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.201+0000 7f132ae00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13080b29f0 0x7f13080b4f00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f131c004ec0 tx=0x7f131c004e10 comp rx=0 tx=0).stop 2024-05-27T07:48:52.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.201+0000 7f132ae00700 1 -- 172.21.15.63:0/2564986756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f132410e900 msgr2=0x7f13241a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.201+0000 7f132ae00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f132410e900 0x7f13241a1760 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f131c007ee0 tx=0x7f131c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.201+0000 7f132ae00700 1 -- 172.21.15.63:0/2564986756 shutdown_connections 2024-05-27T07:48:52.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.201+0000 7f132ae00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13080b29f0 0x7f13080b4f00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.201+0000 7f132ae00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f132410e900 0x7f13241a1760 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.201+0000 7f132ae00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1324105490 0x7f132419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.201+0000 7f132ae00700 1 --2- 172.21.15.63:0/2564986756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1324105de0 0x7f132419d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.201+0000 7f132ae00700 1 -- 172.21.15.63:0/2564986756 >> 172.21.15.63:0/2564986756 conn(0x7f13240fedb0 msgr2=0x7f13240ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.202+0000 7f132ae00700 1 -- 172.21.15.63:0/2564986756 shutdown_connections 2024-05-27T07:48:52.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:52.202+0000 7f132ae00700 1 -- 172.21.15.63:0/2564986756 wait complete. 2024-05-27T07:48:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.416+0000 7fa225e00640 1 -- 172.21.15.63:0/3256937015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa22010b670 msgr2=0x7fa22010da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.416+0000 7fa225e00640 1 --2- 172.21.15.63:0/3256937015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa22010b670 0x7fa22010da90 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009900 tx=0x7fa20c02a510 comp rx=0 tx=0).stop 2024-05-27T07:48:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.416+0000 7fa225e00640 1 -- 172.21.15.63:0/3256937015 shutdown_connections 2024-05-27T07:48:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.416+0000 7fa225e00640 1 --2- 172.21.15.63:0/3256937015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22010e020 0x7fa220110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.416+0000 7fa225e00640 1 --2- 172.21.15.63:0/3256937015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa22010b670 0x7fa22010da90 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.416+0000 7fa225e00640 1 --2- 172.21.15.63:0/3256937015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa220104c10 0x7fa220105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.416+0000 7fa225e00640 1 -- 172.21.15.63:0/3256937015 >> 172.21.15.63:0/3256937015 conn(0x7fa2200fe210 msgr2=0x7fa220100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.416+0000 7fa225e00640 1 -- 172.21.15.63:0/3256937015 shutdown_connections 2024-05-27T07:48:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.416+0000 7fa225e00640 1 -- 172.21.15.63:0/3256937015 wait complete. 2024-05-27T07:48:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.418+0000 7fa225e00640 1 Processor -- start 2024-05-27T07:48:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.418+0000 7fa225e00640 1 -- start start 2024-05-27T07:48:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.419+0000 7fa225e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa220104c10 0x7fa22019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.419+0000 7fa225e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa22010b670 0x7fa22019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.419+0000 7fa225e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22010e020 0x7fa22019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.419+0000 7fa225e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113b30 con 0x7fa220104c10 2024-05-27T07:48:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.419+0000 7fa225e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201139b0 con 0x7fa22010e020 2024-05-27T07:48:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.419+0000 7fa225e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa220113cb0 con 0x7fa22010b670 2024-05-27T07:48:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.419+0000 7fa224a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa220104c10 0x7fa22019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.419+0000 7fa224a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa220104c10 0x7fa22019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38886/0 (socket says 172.21.15.63:38886) 2024-05-27T07:48:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.419+0000 7fa224a00640 1 -- 172.21.15.63:0/593449004 learned_addr learned my addr 172.21.15.63:0/593449004 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.420+0000 7fa225400640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22010e020 0x7fa22019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.420+0000 7fa21fe00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa22010b670 0x7fa22019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.420+0000 7fa224a00640 1 -- 172.21.15.63:0/593449004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa22010b670 msgr2=0x7fa22019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.420+0000 7fa224a00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa22010b670 0x7fa22019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.420+0000 7fa224a00640 1 -- 172.21.15.63:0/593449004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22010e020 msgr2=0x7fa22019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.420+0000 7fa224a00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22010e020 0x7fa22019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.420+0000 7fa224a00640 1 -- 172.21.15.63:0/593449004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22019f470 con 0x7fa220104c10 2024-05-27T07:48:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.420+0000 7fa224a00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa220104c10 0x7fa22019a610 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fa21400d770 tx=0x7fa21400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.421+0000 7fa21d600640 1 -- 172.21.15.63:0/593449004 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214017070 con 0x7fa220104c10 2024-05-27T07:48:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.421+0000 7fa21d600640 1 -- 172.21.15.63:0/593449004 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2140054b0 con 0x7fa220104c10 2024-05-27T07:48:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.421+0000 7fa21d600640 1 -- 172.21.15.63:0/593449004 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214004c60 con 0x7fa220104c10 2024-05-27T07:48:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.421+0000 7fa225e00640 1 -- 172.21.15.63:0/593449004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22019f760 con 0x7fa220104c10 2024-05-27T07:48:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.421+0000 7fa225e00640 1 -- 172.21.15.63:0/593449004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22019fc60 con 0x7fa220104c10 2024-05-27T07:48:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.425+0000 7fa22e482740 1 Processor -- start 2024-05-27T07:48:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.425+0000 7fa22e482740 1 -- start start 2024-05-27T07:48:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa22e482740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e5be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa22e482740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e5c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa22e482740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 0x55a0c8e5c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa22e482740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a0c8dc8b90 con 0x55a0c8ab51b0 2024-05-27T07:48:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa22e482740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a0c8db8730 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa22e482740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a0c8dbc770 con 0x55a0c8a944d0 2024-05-27T07:48:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa21d600640 1 -- 172.21.15.63:0/593449004 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa214004e00 con 0x7fa220104c10 2024-05-27T07:48:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa225400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e5be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa225400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e5be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38898/0 (socket says 172.21.15.63:38898) 2024-05-27T07:48:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa225400640 1 -- 172.21.15.63:0/3760089838 learned_addr learned my addr 172.21.15.63:0/3760089838 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.426+0000 7fa21fe00640 1 --2- 172.21.15.63:0/3760089838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 0x55a0c8e5c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.427+0000 7fa224a00640 1 --2- 172.21.15.63:0/3760089838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e5c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.427+0000 7fa225400640 1 -- 172.21.15.63:0/3760089838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 msgr2=0x55a0c8e5c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.427+0000 7fa225400640 1 --2- 172.21.15.63:0/3760089838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 0x55a0c8e5c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.427+0000 7fa225400640 1 -- 172.21.15.63:0/3760089838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 msgr2=0x55a0c8e5c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.427+0000 7fa225400640 1 --2- 172.21.15.63:0/3760089838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e5c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.427+0000 7fa225400640 1 -- 172.21.15.63:0/3760089838 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c8d518b0 con 0x55a0c8ab51b0 2024-05-27T07:48:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.427+0000 7fa224a00640 1 --2- 172.21.15.63:0/3760089838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e5c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.427+0000 7fa225400640 1 --2- 172.21.15.63:0/3760089838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e5be70 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fa218009f20 tx=0x7fa21800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e45b894eda3ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.427+0000 7fa211c00640 1 -- 172.21.15.63:0/3760089838 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218015070 con 0x55a0c8ab51b0 2024-05-27T07:48:52.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.428+0000 7fa211c00640 1 -- 172.21.15.63:0/3760089838 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa21801aaf0 con 0x55a0c8ab51b0 2024-05-27T07:48:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.428+0000 7fa211c00640 1 -- 172.21.15.63:0/3760089838 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218011ab0 con 0x55a0c8ab51b0 2024-05-27T07:48:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.428+0000 7fa21d600640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2080b29d0 0x7fa2080b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.428+0000 7fa21d600640 1 -- 172.21.15.63:0/593449004 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fa2140e2500 con 0x7fa220104c10 2024-05-27T07:48:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.428+0000 7fa224a00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2080b29d0 0x7fa2080b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.429+0000 7fa22e482740 1 -- 172.21.15.63:0/3760089838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8ab51b0 msgr2=0x55a0c8e5be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.429+0000 7fa22e482740 1 --2- 172.21.15.63:0/3760089838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e5be70 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fa218009f20 tx=0x7fa21800f7f0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.429+0000 7fa22e482740 1 -- 172.21.15.63:0/3760089838 shutdown_connections 2024-05-27T07:48:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.429+0000 7fa22e482740 1 --2- 172.21.15.63:0/3760089838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 0x55a0c8e5c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.429+0000 7fa22e482740 1 --2- 172.21.15.63:0/3760089838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e5c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.429+0000 7fa22e482740 1 --2- 172.21.15.63:0/3760089838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e5be70 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.429+0000 7fa22e482740 1 -- 172.21.15.63:0/3760089838 >> 172.21.15.63:0/3760089838 conn(0x55a0c8aa9300 msgr2=0x55a0c8e58620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.429+0000 7fa22e482740 1 -- 172.21.15.63:0/3760089838 shutdown_connections 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.429+0000 7fa22e482740 1 -- 172.21.15.63:0/3760089838 wait complete. 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.429+0000 7fa224a00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2080b29d0 0x7fa2080b4ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55a0c8e59960 tx=0x7fa214009f00 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.430+0000 7fa22e482740 1 -- start start 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.430+0000 7fa22e482740 1 Processor -- start 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa22e482740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e65fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa22e482740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a944d0 0x55a0c8e66520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa22e482740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa22e482740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a0c8a9e790 con 0x55a0c8a944d0 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa22e482740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a0c8d60bc0 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa22e482740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a0c8e68fa0 con 0x55a0c8ab51b0 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa22e482740 10 client.0 _ll_register_callbacks cb 0x55a0c8d3e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa21fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a944d0 0x55a0c8e66520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa21fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a944d0 0x55a0c8e66520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38914/0 (socket says 172.21.15.63:38914) 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa21fe00640 1 -- 172.21.15.63:0/504708132 learned_addr learned my addr 172.21.15.63:0/504708132 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa225400640 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e65fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa225400640 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa21fe00640 1 -- 172.21.15.63:0/504708132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 msgr2=0x55a0c8e722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa21fe00640 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e722d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa21fe00640 1 -- 172.21.15.63:0/504708132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 msgr2=0x55a0c8e65fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.431+0000 7fa21fe00640 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e65fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.432+0000 7fa21fe00640 1 -- 172.21.15.63:0/504708132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c8acd210 con 0x55a0c8a944d0 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.432+0000 7fa225400640 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.432+0000 7fa21fe00640 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a944d0 0x55a0c8e66520 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005ab0 tx=0x7fa20c0316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adc1348e17fe27 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.432+0000 7fa206000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.432+0000 7fa206000640 1 -- 172.21.15.63:0/504708132 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c03f070 con 0x55a0c8a944d0 2024-05-27T07:48:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.432+0000 7fa22e482740 1 -- 172.21.15.63:0/504708132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa21801aaf0 con 0x55a0c8a944d0 2024-05-27T07:48:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.432+0000 7fa206000640 1 -- 172.21.15.63:0/504708132 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fa20c004450 con 0x55a0c8a944d0 2024-05-27T07:48:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa206000640 1 -- 172.21.15.63:0/504708132 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa20c035070 con 0x55a0c8a944d0 2024-05-27T07:48:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa22e482740 10 client.18378 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa22e482740 1 -- 172.21.15.63:0/504708132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a0c8b99620 con 0x55a0c8a944d0 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa206000640 1 -- 172.21.15.63:0/504708132 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa21801aaf0 con 0x55a0c8a944d0 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa206000640 1 -- 172.21.15.63:0/504708132 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa20c0049a0 con 0x55a0c8a944d0 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa22e482740 10 client.18378 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa22e482740 10 client.18378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa22e482740 20 client.18378 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa22e482740 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a0c8a73d80 0x55a0c8e78170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa22e482740 4 client.18378 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.433+0000 7fa22e482740 1 -- 172.21.15.63:0/504708132 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0c8dcaae0 con 0x55a0c8a73d80 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.434+0000 7fa225400640 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a0c8a73d80 0x55a0c8e78170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.435+0000 7fa206000640 10 client.18378 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.435+0000 7fa225400640 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a0c8a73d80 0x55a0c8e78170 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5859f44512014c2b server_cookie=171888bfbb1220d8 in_seq=0 out_seq=0 2024-05-27T07:48:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.437+0000 7fa206000640 1 -- 172.21.15.63:0/504708132 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7fa218027e00 con 0x55a0c8a73d80 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.437+0000 7fa206000640 10 client.18378 handle_command_reply: tid=0 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.437+0000 7fa22e482740 1 client.18378 shutdown 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.437+0000 7fa22e482740 1 -- 172.21.15.63:0/504708132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a944d0 msgr2=0x55a0c8e66520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.437+0000 7fa22e482740 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a944d0 0x55a0c8e66520 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005ab0 tx=0x7fa20c0316b0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.438+0000 7fa22e482740 1 -- 172.21.15.63:0/504708132 shutdown_connections 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.438+0000 7fa22e482740 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a0c8a73d80 0x55a0c8e78170 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.438+0000 7fa22e482740 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.438+0000 7fa22e482740 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a944d0 0x55a0c8e66520 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.438+0000 7fa22e482740 1 --2- 172.21.15.63:0/504708132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e65fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.438+0000 7fa22e482740 1 -- 172.21.15.63:0/504708132 >> 172.21.15.63:0/504708132 conn(0x55a0c8aa9300 msgr2=0x55a0c8e5bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.438+0000 7fa206000640 0 client.18378 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.438+0000 7fa22e482740 1 -- 172.21.15.63:0/504708132 shutdown_connections 2024-05-27T07:48:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.438+0000 7fa22e482740 1 -- 172.21.15.63:0/504708132 wait complete. 2024-05-27T07:48:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.438+0000 7fa22e482740 20 client.18378 trim_cache size 0 max 16384 2024-05-27T07:48:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.464+0000 7fa22e482740 1 Processor -- start 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.464+0000 7fa22e482740 1 -- start start 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.465+0000 7fa22e482740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e68710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.465+0000 7fa22e482740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 0x55a0c8e53240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.465+0000 7fa22e482740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.465+0000 7fa22e482740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201b1750 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.465+0000 7fa22e482740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201af020 con 0x55a0c8ab51b0 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.465+0000 7fa22e482740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ace90 con 0x55a0c8a944d0 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.465+0000 7fa225400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e68710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.465+0000 7fa225400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e68710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38916/0 (socket says 172.21.15.63:38916) 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.465+0000 7fa225400640 1 -- 172.21.15.63:0/1967710812 learned_addr learned my addr 172.21.15.63:0/1967710812 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.466+0000 7fa21fe00640 1 --2- 172.21.15.63:0/1967710812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 0x55a0c8e53240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.466+0000 7fa225400640 1 -- 172.21.15.63:0/1967710812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 msgr2=0x55a0c8e53240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.466+0000 7fa225400640 1 --2- 172.21.15.63:0/1967710812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 0x55a0c8e53240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.466+0000 7fa225400640 1 -- 172.21.15.63:0/1967710812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8ab51b0 msgr2=0x55a0c8e612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.466+0000 7fa225400640 1 --2- 172.21.15.63:0/1967710812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.466+0000 7fa225400640 1 -- 172.21.15.63:0/1967710812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c8e66880 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.466+0000 7fa21fe00640 1 --2- 172.21.15.63:0/1967710812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 0x55a0c8e53240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.466+0000 7fa225400640 1 --2- 172.21.15.63:0/1967710812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e68710 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fa218011520 tx=0x7fa218000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5465618048f0a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.467+0000 7fa211200640 1 -- 172.21.15.63:0/1967710812 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218015040 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.467+0000 7fa211200640 1 -- 172.21.15.63:0/1967710812 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa218022040 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.467+0000 7fa211200640 1 -- 172.21.15.63:0/1967710812 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21801aac0 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.468+0000 7fa22e482740 1 -- 172.21.15.63:0/1967710812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 msgr2=0x55a0c8e68710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.468+0000 7fa22e482740 1 --2- 172.21.15.63:0/1967710812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e68710 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fa218011520 tx=0x7fa218000f40 comp rx=0 tx=0).stop 2024-05-27T07:48:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.469+0000 7fa22e482740 1 -- 172.21.15.63:0/1967710812 shutdown_connections 2024-05-27T07:48:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.469+0000 7fa22e482740 1 --2- 172.21.15.63:0/1967710812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.469+0000 7fa22e482740 1 --2- 172.21.15.63:0/1967710812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8a944d0 0x55a0c8e53240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.469+0000 7fa22e482740 1 --2- 172.21.15.63:0/1967710812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e68710 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.469+0000 7fa22e482740 1 -- 172.21.15.63:0/1967710812 >> 172.21.15.63:0/1967710812 conn(0x55a0c8aa9300 msgr2=0x55a0c8e60b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.469+0000 7fa22e482740 1 -- 172.21.15.63:0/1967710812 shutdown_connections 2024-05-27T07:48:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.469+0000 7fa22e482740 1 -- 172.21.15.63:0/1967710812 wait complete. 2024-05-27T07:48:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.470+0000 7fa22e482740 1 -- start start 2024-05-27T07:48:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.470+0000 7fa22e482740 1 Processor -- start 2024-05-27T07:48:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.471+0000 7fa22e482740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e6e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.471+0000 7fa22e482740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a944d0 0x55a0c8e69b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.471+0000 7fa22e482740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e6a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.471+0000 7fa22e482740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ad4e0 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.471+0000 7fa22e482740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201ae950 con 0x55a0c8a944d0 2024-05-27T07:48:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.471+0000 7fa22e482740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa2201adbb0 con 0x55a0c8ab51b0 2024-05-27T07:48:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.471+0000 7fa22e482740 10 client.0 _ll_register_callbacks cb 0x7fa20c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.471+0000 7fa225400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e6e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.471+0000 7fa225400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e6e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38922/0 (socket says 172.21.15.63:38922) 2024-05-27T07:48:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.471+0000 7fa225400640 1 -- 172.21.15.63:0/1763286995 learned_addr learned my addr 172.21.15.63:0/1763286995 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:52.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.472+0000 7fa21fe00640 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a944d0 0x55a0c8e69b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.472+0000 7fa225400640 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e6a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.472+0000 7fa225400640 1 -- 172.21.15.63:0/1763286995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 msgr2=0x55a0c8e6a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.472+0000 7fa225400640 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e6a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.472+0000 7fa225400640 1 -- 172.21.15.63:0/1763286995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a944d0 msgr2=0x55a0c8e69b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.472+0000 7fa225400640 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a944d0 0x55a0c8e69b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.472+0000 7fa225400640 1 -- 172.21.15.63:0/1763286995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c8e6a700 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.472+0000 7fa21fe00640 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a944d0 0x55a0c8e69b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.472+0000 7fa225400640 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e6e830 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fa218009740 tx=0x7fa21800fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606456f935843a18 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.472+0000 7fa206000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.473+0000 7fa206000640 1 -- 172.21.15.63:0/1763286995 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218015040 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.473+0000 7fa206000640 1 -- 172.21.15.63:0/1763286995 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fa218003b70 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.473+0000 7fa22e482740 1 -- 172.21.15.63:0/1763286995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa218022040 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.473+0000 7fa206000640 1 -- 172.21.15.63:0/1763286995 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa21801aad0 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.473+0000 7fa22e482740 10 client.18384 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.473+0000 7fa22e482740 1 -- 172.21.15.63:0/1763286995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a0c8e53fb0 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.474+0000 7fa206000640 1 -- 172.21.15.63:0/1763286995 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa218022040 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.474+0000 7fa206000640 1 -- 172.21.15.63:0/1763286995 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa2180050b0 con 0x55a0c8a8c2f0 2024-05-27T07:48:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.474+0000 7fa22e482740 10 client.18384 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.474+0000 7fa22e482740 10 client.18384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.474+0000 7fa22e482740 20 client.18384 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.474+0000 7fa22e482740 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a0c8a491c0 0x55a0c8e56580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.474+0000 7fa22e482740 4 client.18384 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:48:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.474+0000 7fa22e482740 1 -- 172.21.15.63:0/1763286995 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0c8dc42c0 con 0x55a0c8a491c0 2024-05-27T07:48:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.475+0000 7fa21fe00640 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a0c8a491c0 0x55a0c8e56580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.476+0000 7fa206000640 10 client.18384 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.476+0000 7fa21fe00640 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a0c8a491c0 0x55a0c8e56580 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45d8e9a8308150 server_cookie=d2bc4bf9c5524ceb in_seq=0 out_seq=0 2024-05-27T07:48:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa206000640 1 -- 172.21.15.63:0/1763286995 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a0c8dc42c0 con 0x55a0c8a491c0 2024-05-27T07:48:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa206000640 10 client.18384 handle_command_reply: tid=0 2024-05-27T07:48:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa22e482740 1 client.18384 shutdown 2024-05-27T07:48:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa22e482740 1 -- 172.21.15.63:0/1763286995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 msgr2=0x55a0c8e6e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa22e482740 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e6e830 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fa218009740 tx=0x7fa21800fcf0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa22e482740 1 -- 172.21.15.63:0/1763286995 shutdown_connections 2024-05-27T07:48:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa22e482740 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a0c8a491c0 0x55a0c8e56580 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa22e482740 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0c8ab51b0 0x55a0c8e6a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa22e482740 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a0c8a944d0 0x55a0c8e69b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa22e482740 1 --2- 172.21.15.63:0/1763286995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a0c8a8c2f0 0x55a0c8e6e830 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa206000640 0 client.18384 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa22e482740 1 -- 172.21.15.63:0/1763286995 >> 172.21.15.63:0/1763286995 conn(0x55a0c8aa9300 msgr2=0x55a0c8e52fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.505+0000 7fa22e482740 1 -- 172.21.15.63:0/1763286995 shutdown_connections 2024-05-27T07:48:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.506+0000 7fa22e482740 1 -- 172.21.15.63:0/1763286995 wait complete. 2024-05-27T07:48:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.506+0000 7fa22e482740 20 client.18384 trim_cache size 0 max 16384 2024-05-27T07:48:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:48:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:48:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:48:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.506+0000 7fa225e00640 1 -- 172.21.15.63:0/593449004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2080b29d0 msgr2=0x7fa2080b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2080b29d0 0x7fa2080b4ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55a0c8e59960 tx=0x7fa214009f00 comp rx=0 tx=0).stop 2024-05-27T07:48:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 -- 172.21.15.63:0/593449004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa220104c10 msgr2=0x7fa22019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa220104c10 0x7fa22019a610 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fa21400d770 tx=0x7fa21400dc30 comp rx=0 tx=0).stop 2024-05-27T07:48:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 -- 172.21.15.63:0/593449004 shutdown_connections 2024-05-27T07:48:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2080b29d0 0x7fa2080b4ee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22010e020 0x7fa22019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa22010b670 0x7fa22019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 --2- 172.21.15.63:0/593449004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa220104c10 0x7fa22019a610 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 -- 172.21.15.63:0/593449004 >> 172.21.15.63:0/593449004 conn(0x7fa2200fe210 msgr2=0x7fa22010e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 -- 172.21.15.63:0/593449004 shutdown_connections 2024-05-27T07:48:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:48:52.507+0000 7fa225e00640 1 -- 172.21.15.63:0/593449004 wait complete. 2024-05-27T07:48:53.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:53 smithi086 ceph-mon[34687]: pgmap v480: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:48:53.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:53 smithi149 ceph-mon[35408]: pgmap v480: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:48:53.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:53 smithi063 ceph-mon[31613]: pgmap v480: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:48:54.251 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:48:54.251 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:48:54.761 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:55.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:55 smithi063 ceph-mon[31613]: pgmap v481: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:48:55.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:55 smithi086 ceph-mon[34687]: pgmap v481: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:48:55.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:55 smithi149 ceph-mon[35408]: pgmap v481: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:48:56.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:56 smithi086 ceph-mon[34687]: pgmap v482: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:48:56.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:56 smithi149 ceph-mon[35408]: pgmap v482: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:48:56.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:56 smithi063 ceph-mon[31613]: pgmap v482: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:48:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.604+0000 7fea15800700 1 -- 172.21.15.63:0/1786302355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea10106560 msgr2=0x7fea10106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.604+0000 7fea15800700 1 --2- 172.21.15.63:0/1786302355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea10106560 0x7fea10106970 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009990 tx=0x7fe9fc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.605+0000 7fea15800700 1 -- 172.21.15.63:0/1786302355 shutdown_connections 2024-05-27T07:48:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.605+0000 7fea15800700 1 --2- 172.21.15.63:0/1786302355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea10106eb0 0x7fea10113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.605+0000 7fea15800700 1 --2- 172.21.15.63:0/1786302355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea1010c5a0 0x7fea1010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.605+0000 7fea15800700 1 --2- 172.21.15.63:0/1786302355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea10106560 0x7fea10106970 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.605+0000 7fea15800700 1 -- 172.21.15.63:0/1786302355 >> 172.21.15.63:0/1786302355 conn(0x7fea10102040 msgr2=0x7fea10104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.605+0000 7fea15800700 1 -- 172.21.15.63:0/1786302355 shutdown_connections 2024-05-27T07:48:58.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.605+0000 7fea15800700 1 -- 172.21.15.63:0/1786302355 wait complete. 2024-05-27T07:48:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.607+0000 7fea15800700 1 Processor -- start 2024-05-27T07:48:58.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.607+0000 7fea15800700 1 -- start start 2024-05-27T07:48:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.607+0000 7fea15800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea10106560 0x7fea1019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.607+0000 7fea15800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea10106eb0 0x7fea1019d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.607+0000 7fea15800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea1010c5a0 0x7fea101a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.608+0000 7fea15800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea10116580 con 0x7fea1010c5a0 2024-05-27T07:48:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.608+0000 7fea15800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea10116400 con 0x7fea10106eb0 2024-05-27T07:48:58.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.608+0000 7fea15800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea10116700 con 0x7fea10106560 2024-05-27T07:48:58.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.608+0000 7fea0f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea1010c5a0 0x7fea101a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.608+0000 7fea0f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea1010c5a0 0x7fea101a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38926/0 (socket says 172.21.15.63:38926) 2024-05-27T07:48:58.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.608+0000 7fea0f400700 1 -- 172.21.15.63:0/1952163810 learned_addr learned my addr 172.21.15.63:0/1952163810 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:58.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.608+0000 7fea0ea00700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea10106560 0x7fea1019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.608+0000 7fea0e000700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea10106eb0 0x7fea1019d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.609+0000 7fea0f400700 1 -- 172.21.15.63:0/1952163810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea10106560 msgr2=0x7fea1019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.609+0000 7fea0f400700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea10106560 0x7fea1019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.609+0000 7fea0f400700 1 -- 172.21.15.63:0/1952163810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea10106eb0 msgr2=0x7fea1019d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.609+0000 7fea0f400700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea10106eb0 0x7fea1019d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.609+0000 7fea0f400700 1 -- 172.21.15.63:0/1952163810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea101a1d70 con 0x7fea1010c5a0 2024-05-27T07:48:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.609+0000 7fea0e000700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea10106eb0 0x7fea1019d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.609+0000 7fea0f400700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea1010c5a0 0x7fea101a1780 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fea08007d70 tx=0x7fea0800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.609+0000 7fea07400700 1 -- 172.21.15.63:0/1952163810 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08016020 con 0x7fea1010c5a0 2024-05-27T07:48:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.609+0000 7fea07400700 1 -- 172.21.15.63:0/1952163810 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea0800ad40 con 0x7fea1010c5a0 2024-05-27T07:48:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.609+0000 7fea07400700 1 -- 172.21.15.63:0/1952163810 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea080115e0 con 0x7fea1010c5a0 2024-05-27T07:48:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.610+0000 7fea15800700 1 -- 172.21.15.63:0/1952163810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea101a2060 con 0x7fea1010c5a0 2024-05-27T07:48:58.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.610+0000 7fea15800700 1 -- 172.21.15.63:0/1952163810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea101a2550 con 0x7fea1010c5a0 2024-05-27T07:48:58.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.614+0000 7fea07400700 1 -- 172.21.15.63:0/1952163810 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fea08019020 con 0x7fea1010c5a0 2024-05-27T07:48:58.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.616+0000 7fea24327b80 1 Processor -- start 2024-05-27T07:48:58.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.616+0000 7fea24327b80 1 -- start start 2024-05-27T07:48:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.616+0000 7fea24327b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35dbfa70 0x560f35e523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.616+0000 7fea24327b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35cd3ec0 0x560f35e528f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.616+0000 7fea24327b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e5f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.616+0000 7fea24327b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560f35d80ec0 con 0x560f35e52ef0 2024-05-27T07:48:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.616+0000 7fea24327b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f35d70a60 con 0x560f35cd3ec0 2024-05-27T07:48:58.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.616+0000 7fea24327b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560f35d74aa0 con 0x560f35dbfa70 2024-05-27T07:48:58.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.617+0000 7fea0e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e5f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.617+0000 7fea07400700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9f40b29f0 0x7fe9f40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.617+0000 7fea0ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35cd3ec0 0x560f35e528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.617+0000 7fea0e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e5f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38942/0 (socket says 172.21.15.63:38942) 2024-05-27T07:48:58.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.617+0000 7fea0e000700 1 -- 172.21.15.63:0/517239075 learned_addr learned my addr 172.21.15.63:0/517239075 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:58.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.617+0000 7fea0e000700 1 --2- 172.21.15.63:0/517239075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35dbfa70 0x560f35e523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.617+0000 7fea07400700 1 -- 172.21.15.63:0/1952163810 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fea080e2870 con 0x7fea1010c5a0 2024-05-27T07:48:58.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.617+0000 7fea0f400700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9f40b29f0 0x7fe9f40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.618+0000 7fea0e000700 1 -- 172.21.15.63:0/517239075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35dbfa70 msgr2=0x560f35e523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:58.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.618+0000 7fea0e000700 1 --2- 172.21.15.63:0/517239075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35dbfa70 0x560f35e523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.618+0000 7fea0e000700 1 -- 172.21.15.63:0/517239075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35cd3ec0 msgr2=0x560f35e528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:58.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.618+0000 7fea0e000700 1 --2- 172.21.15.63:0/517239075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35cd3ec0 0x560f35e528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.618+0000 7fea0e000700 1 -- 172.21.15.63:0/517239075 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f35d394a0 con 0x560f35e52ef0 2024-05-27T07:48:58.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.618+0000 7fea0e000700 1 --2- 172.21.15.63:0/517239075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e5f190 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc005f50 tx=0x7fe9fc005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd712d152ccbd83a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:58.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.618+0000 7fea0f400700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9f40b29f0 0x7fe9f40b4f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fea080dfd70 tx=0x7fea080a5040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:58.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.619+0000 7fe9fbe00700 1 -- 172.21.15.63:0/517239075 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc005380 con 0x560f35e52ef0 2024-05-27T07:48:58.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.619+0000 7fe9fbe00700 1 -- 172.21.15.63:0/517239075 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9fc005520 con 0x560f35e52ef0 2024-05-27T07:48:58.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.619+0000 7fe9fbe00700 1 -- 172.21.15.63:0/517239075 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00bd10 con 0x560f35e52ef0 2024-05-27T07:48:58.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.620+0000 7fea24327b80 1 -- 172.21.15.63:0/517239075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 msgr2=0x560f35e5f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:58.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.620+0000 7fea24327b80 1 --2- 172.21.15.63:0/517239075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e5f190 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc005f50 tx=0x7fe9fc005ee0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.620+0000 7fea24327b80 1 -- 172.21.15.63:0/517239075 shutdown_connections 2024-05-27T07:48:58.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.620+0000 7fea24327b80 1 --2- 172.21.15.63:0/517239075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e5f190 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.620+0000 7fea24327b80 1 --2- 172.21.15.63:0/517239075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35dbfa70 0x560f35e523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.620+0000 7fea24327b80 1 --2- 172.21.15.63:0/517239075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35cd3ec0 0x560f35e528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.620+0000 7fea24327b80 1 -- 172.21.15.63:0/517239075 >> 172.21.15.63:0/517239075 conn(0x560f35d6d410 msgr2=0x560f35e4e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:58.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.620+0000 7fea24327b80 1 -- 172.21.15.63:0/517239075 shutdown_connections 2024-05-27T07:48:58.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.620+0000 7fea24327b80 1 -- 172.21.15.63:0/517239075 wait complete. 2024-05-27T07:48:58.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.621+0000 7fea24327b80 1 -- start start 2024-05-27T07:48:58.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.621+0000 7fea24327b80 1 Processor -- start 2024-05-27T07:48:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea24327b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35cd3ec0 0x560f35e5c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea24327b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea24327b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35e52ef0 0x560f35e5d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea24327b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560f35c77020 con 0x560f35dbfa70 2024-05-27T07:48:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea24327b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f35d117e0 con 0x560f35e52ef0 2024-05-27T07:48:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea24327b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560f35e5fba0 con 0x560f35cd3ec0 2024-05-27T07:48:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea24327b80 10 client.0 _ll_register_callbacks cb 0x560f35d1acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea0ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea0ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38954/0 (socket says 172.21.15.63:38954) 2024-05-27T07:48:58.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea0ea00700 1 -- 172.21.15.63:0/298949806 learned_addr learned my addr 172.21.15.63:0/298949806 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:58.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.622+0000 7fea0e000700 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35cd3ec0 0x560f35e5c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.623+0000 7fea0e000700 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35e52ef0 0x560f35e5d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.623+0000 7fea0ea00700 1 -- 172.21.15.63:0/298949806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35cd3ec0 msgr2=0x560f35e5c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.623+0000 7fea0ea00700 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35cd3ec0 0x560f35e5c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.623+0000 7fea0ea00700 1 -- 172.21.15.63:0/298949806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35e52ef0 msgr2=0x560f35e5d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.623+0000 7fea0ea00700 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35e52ef0 0x560f35e5d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.623+0000 7fea0ea00700 1 -- 172.21.15.63:0/298949806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f35cc0950 con 0x560f35dbfa70 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.623+0000 7fea0ea00700 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5cc40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fea0000baf0 tx=0x7fea0000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a1ac01d3c11251 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.623+0000 7fe9f8c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.624+0000 7fe9f8c00700 1 -- 172.21.15.63:0/298949806 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00004830 con 0x560f35dbfa70 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.624+0000 7fea24327b80 1 -- 172.21.15.63:0/298949806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9fc005520 con 0x560f35dbfa70 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.624+0000 7fe9f8c00700 1 -- 172.21.15.63:0/298949806 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fea000049d0 con 0x560f35dbfa70 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.624+0000 7fe9f8c00700 1 -- 172.21.15.63:0/298949806 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fea00023050 con 0x560f35dbfa70 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.624+0000 7fea24327b80 10 client.18396 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:58.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.624+0000 7fea24327b80 1 -- 172.21.15.63:0/298949806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560f35d410e0 con 0x560f35dbfa70 2024-05-27T07:48:58.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.625+0000 7fe9f8c00700 1 -- 172.21.15.63:0/298949806 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9fc005520 con 0x560f35dbfa70 2024-05-27T07:48:58.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.625+0000 7fe9f8c00700 1 -- 172.21.15.63:0/298949806 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fea00019030 con 0x560f35dbfa70 2024-05-27T07:48:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.625+0000 7fea24327b80 10 client.18396 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.625+0000 7fea24327b80 10 client.18396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.625+0000 7fea24327b80 20 client.18396 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.625+0000 7fea24327b80 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560f35e6cb80 0x560f35e6efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.625+0000 7fea24327b80 4 client.18396 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.625+0000 7fea24327b80 1 -- 172.21.15.63:0/298949806 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f35d82e10 con 0x560f35e6cb80 2024-05-27T07:48:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.626+0000 7fea0e000700 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560f35e6cb80 0x560f35e6efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.627+0000 7fe9f8c00700 10 client.18396 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.627+0000 7fea0e000700 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560f35e6cb80 0x560f35e6efd0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb9b960bd8c45119 server_cookie=b7305de8b40c2973 in_seq=0 out_seq=0 2024-05-27T07:48:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fe9f8c00700 1 -- 172.21.15.63:0/298949806 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560f35d82e10 con 0x560f35e6cb80 2024-05-27T07:48:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fe9f8c00700 10 client.18396 handle_command_reply: tid=0 2024-05-27T07:48:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fea24327b80 1 client.18396 shutdown 2024-05-27T07:48:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fea24327b80 1 -- 172.21.15.63:0/298949806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 msgr2=0x560f35e5cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fea24327b80 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5cc40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fea0000baf0 tx=0x7fea0000beb0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fea24327b80 1 -- 172.21.15.63:0/298949806 shutdown_connections 2024-05-27T07:48:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fea24327b80 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560f35e6cb80 0x560f35e6efd0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fea24327b80 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35e52ef0 0x560f35e5d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fea24327b80 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35cd3ec0 0x560f35e5c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fea24327b80 1 --2- 172.21.15.63:0/298949806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5cc40 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fe9f8c00700 0 client.18396 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.629+0000 7fea24327b80 1 -- 172.21.15.63:0/298949806 >> 172.21.15.63:0/298949806 conn(0x560f35d6d410 msgr2=0x560f35e524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.630+0000 7fea24327b80 1 -- 172.21.15.63:0/298949806 shutdown_connections 2024-05-27T07:48:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.630+0000 7fea24327b80 1 -- 172.21.15.63:0/298949806 wait complete. 2024-05-27T07:48:58.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.630+0000 7fea24327b80 20 client.18396 trim_cache size 0 max 16384 2024-05-27T07:48:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.657+0000 7fea24327b80 1 Processor -- start 2024-05-27T07:48:58.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.658+0000 7fea24327b80 1 -- start start 2024-05-27T07:48:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.658+0000 7fea24327b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35cd3ec0 0x560f35e57f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.658+0000 7fea24327b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.658+0000 7fea24327b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35e52ef0 0x560f35e4e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.658+0000 7fea24327b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b4f10 con 0x560f35dbfa70 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.658+0000 7fea24327b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b27a0 con 0x560f35cd3ec0 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.658+0000 7fea24327b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b0610 con 0x560f35e52ef0 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.658+0000 7fea0ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.658+0000 7fea0ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38966/0 (socket says 172.21.15.63:38966) 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.659+0000 7fea0ea00700 1 -- 172.21.15.63:0/1941965527 learned_addr learned my addr 172.21.15.63:0/1941965527 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.659+0000 7fea0e000700 1 --2- 172.21.15.63:0/1941965527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35cd3ec0 0x560f35e57f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.659+0000 7fea0e000700 1 --2- 172.21.15.63:0/1941965527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35e52ef0 0x560f35e4e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.659+0000 7fea0ea00700 1 -- 172.21.15.63:0/1941965527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35e52ef0 msgr2=0x560f35e4e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.659+0000 7fea0ea00700 1 --2- 172.21.15.63:0/1941965527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35e52ef0 0x560f35e4e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.659+0000 7fea0ea00700 1 -- 172.21.15.63:0/1941965527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35cd3ec0 msgr2=0x560f35e57f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.659+0000 7fea0ea00700 1 --2- 172.21.15.63:0/1941965527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35cd3ec0 0x560f35e57f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.659+0000 7fea0ea00700 1 -- 172.21.15.63:0/1941965527 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f35e6a600 con 0x560f35dbfa70 2024-05-27T07:48:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.659+0000 7fea0e000700 1 --2- 172.21.15.63:0/1941965527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35e52ef0 0x560f35e4e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.659+0000 7fea0ea00700 1 --2- 172.21.15.63:0/1941965527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5e7e0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fea00000c00 tx=0x7fea000044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b3d2476097712f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.660+0000 7fe9fb400700 1 -- 172.21.15.63:0/1941965527 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0002c030 con 0x560f35dbfa70 2024-05-27T07:48:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.660+0000 7fe9fb400700 1 -- 172.21.15.63:0/1941965527 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea00011e50 con 0x560f35dbfa70 2024-05-27T07:48:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.660+0000 7fe9fb400700 1 -- 172.21.15.63:0/1941965527 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00020040 con 0x560f35dbfa70 2024-05-27T07:48:58.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.661+0000 7fea24327b80 1 -- 172.21.15.63:0/1941965527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 msgr2=0x560f35e5e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.661+0000 7fea24327b80 1 --2- 172.21.15.63:0/1941965527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5e7e0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fea00000c00 tx=0x7fea000044b0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.661+0000 7fea24327b80 1 -- 172.21.15.63:0/1941965527 shutdown_connections 2024-05-27T07:48:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.661+0000 7fea24327b80 1 --2- 172.21.15.63:0/1941965527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35e52ef0 0x560f35e4e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.661+0000 7fea24327b80 1 --2- 172.21.15.63:0/1941965527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35cd3ec0 0x560f35e57f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.661+0000 7fea24327b80 1 --2- 172.21.15.63:0/1941965527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35dbfa70 0x560f35e5e7e0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.661+0000 7fea24327b80 1 -- 172.21.15.63:0/1941965527 >> 172.21.15.63:0/1941965527 conn(0x560f35d6d410 msgr2=0x560f35e53f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.661+0000 7fea24327b80 1 -- 172.21.15.63:0/1941965527 shutdown_connections 2024-05-27T07:48:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.662+0000 7fea24327b80 1 -- 172.21.15.63:0/1941965527 wait complete. 2024-05-27T07:48:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.662+0000 7fea24327b80 1 -- start start 2024-05-27T07:48:58.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.662+0000 7fea24327b80 1 Processor -- start 2024-05-27T07:48:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.663+0000 7fea24327b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35cd3ec0 0x560f35e5fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.663+0000 7fea24327b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35dbfa70 0x560f35e60390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.663+0000 7fea24327b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e60910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.663+0000 7fea24327b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b0c60 con 0x560f35e52ef0 2024-05-27T07:48:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.663+0000 7fea24327b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b20d0 con 0x560f35dbfa70 2024-05-27T07:48:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.663+0000 7fea24327b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea101b1330 con 0x560f35cd3ec0 2024-05-27T07:48:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.663+0000 7fea24327b80 10 client.0 _ll_register_callbacks cb 0x560f35d1acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:58.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.663+0000 7fea0e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e60910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.663+0000 7fea0e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e60910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38980/0 (socket says 172.21.15.63:38980) 2024-05-27T07:48:58.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.663+0000 7fea0e000700 1 -- 172.21.15.63:0/449407884 learned_addr learned my addr 172.21.15.63:0/449407884 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:48:58.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.664+0000 7fea0ea00700 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35dbfa70 0x560f35e60390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.664+0000 7fea0e000700 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35cd3ec0 0x560f35e5fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.664+0000 7fea0e000700 1 -- 172.21.15.63:0/449407884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35cd3ec0 msgr2=0x560f35e5fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:58.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.664+0000 7fea0e000700 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35cd3ec0 0x560f35e5fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.664+0000 7fea0e000700 1 -- 172.21.15.63:0/449407884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35dbfa70 msgr2=0x560f35e60390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:58.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.664+0000 7fea0e000700 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35dbfa70 0x560f35e60390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.664+0000 7fea0e000700 1 -- 172.21.15.63:0/449407884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f35ed7880 con 0x560f35e52ef0 2024-05-27T07:48:58.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.665+0000 7fea0e000700 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e60910 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0058d0 tx=0x7fe9fc0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccaf1436c2216db server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.665+0000 7fe9f8c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.665+0000 7fe9f8c00700 1 -- 172.21.15.63:0/449407884 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc0054a0 con 0x560f35e52ef0 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.665+0000 7fea24327b80 1 -- 172.21.15.63:0/449407884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea00011e50 con 0x560f35e52ef0 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.665+0000 7fe9f8c00700 1 -- 172.21.15.63:0/449407884 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7fe9fc021040 con 0x560f35e52ef0 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.665+0000 7fe9f8c00700 1 -- 172.21.15.63:0/449407884 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe9fc0039a0 con 0x560f35e52ef0 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.665+0000 7fea24327b80 10 client.18402 fetch_fsmap learned FSMap version 33 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.665+0000 7fea24327b80 1 -- 172.21.15.63:0/449407884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560f35ed8a90 con 0x560f35e52ef0 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.666+0000 7fe9f8c00700 1 -- 172.21.15.63:0/449407884 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea00011e50 con 0x560f35e52ef0 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.666+0000 7fe9f8c00700 1 -- 172.21.15.63:0/449407884 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe9fc0322b0 con 0x560f35e52ef0 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.666+0000 7fea24327b80 10 client.18402 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.666+0000 7fea24327b80 10 client.18402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:58.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.666+0000 7fea24327b80 20 client.18402 populate_metadata read hostname 'smithi063' 2024-05-27T07:48:58.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.666+0000 7fea24327b80 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560f35ed8cc0 0x560f35edb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:58.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.666+0000 7fea24327b80 4 client.18402 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:48:58.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.666+0000 7fea24327b80 1 -- 172.21.15.63:0/449407884 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560f35d7c5f0 con 0x560f35ed8cc0 2024-05-27T07:48:58.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.667+0000 7fea0ea00700 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560f35ed8cc0 0x560f35edb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:58.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.668+0000 7fea0ea00700 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560f35ed8cc0 0x560f35edb0e0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a8a09c48ee8263d server_cookie=55f0fedaaa151b0f in_seq=0 out_seq=0 2024-05-27T07:48:58.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.668+0000 7fe9f8c00700 10 client.18402 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:58.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.670+0000 7fe9f8c00700 1 -- 172.21.15.63:0/449407884 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560f35d7c5f0 con 0x560f35ed8cc0 2024-05-27T07:48:58.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.670+0000 7fe9f8c00700 10 client.18402 handle_command_reply: tid=0 2024-05-27T07:48:58.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.670+0000 7fea24327b80 1 client.18402 shutdown 2024-05-27T07:48:58.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.670+0000 7fea24327b80 1 -- 172.21.15.63:0/449407884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 msgr2=0x560f35e60910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:58.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.670+0000 7fea24327b80 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e60910 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0058d0 tx=0x7fe9fc0095d0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.671+0000 7fea24327b80 1 -- 172.21.15.63:0/449407884 shutdown_connections 2024-05-27T07:48:58.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.671+0000 7fea24327b80 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560f35ed8cc0 0x560f35edb0e0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.671+0000 7fea24327b80 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560f35e52ef0 0x560f35e60910 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.671+0000 7fea24327b80 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560f35cd3ec0 0x560f35e5fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.671+0000 7fea24327b80 1 --2- 172.21.15.63:0/449407884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35dbfa70 0x560f35e60390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.671+0000 7fea24327b80 1 -- 172.21.15.63:0/449407884 >> 172.21.15.63:0/449407884 conn(0x560f35d6d410 msgr2=0x560f35e5e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:58.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.671+0000 7fe9f8c00700 0 client.18402 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:48:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.671+0000 7fea24327b80 1 -- 172.21.15.63:0/449407884 shutdown_connections 2024-05-27T07:48:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.671+0000 7fea24327b80 1 -- 172.21.15.63:0/449407884 wait complete. 2024-05-27T07:48:58.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.671+0000 7fea24327b80 20 client.18402 trim_cache size 0 max 16384 2024-05-27T07:48:58.672 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:48:58.672 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:48:58.672 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "a09a7791-2cf9-4812-ab4f-b3e4c5fef712", 2024-05-27T07:48:58.672 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:48:58.672 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:48:58.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.672+0000 7fea15800700 1 -- 172.21.15.63:0/1952163810 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9f40b29f0 msgr2=0x7fe9f40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:58.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.672+0000 7fea15800700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9f40b29f0 0x7fe9f40b4f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fea080dfd70 tx=0x7fea080a5040 comp rx=0 tx=0).stop 2024-05-27T07:48:58.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.672+0000 7fea15800700 1 -- 172.21.15.63:0/1952163810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea1010c5a0 msgr2=0x7fea101a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:58.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.672+0000 7fea15800700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea1010c5a0 0x7fea101a1780 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fea08007d70 tx=0x7fea0800a340 comp rx=0 tx=0).stop 2024-05-27T07:48:58.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.672+0000 7fea15800700 1 -- 172.21.15.63:0/1952163810 shutdown_connections 2024-05-27T07:48:58.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.672+0000 7fea15800700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe9f40b29f0 0x7fe9f40b4f00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.672+0000 7fea15800700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea1010c5a0 0x7fea101a1780 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.672+0000 7fea15800700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea10106560 0x7fea1019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.672+0000 7fea15800700 1 --2- 172.21.15.63:0/1952163810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea10106eb0 0x7fea1019d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:58.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.672+0000 7fea15800700 1 -- 172.21.15.63:0/1952163810 >> 172.21.15.63:0/1952163810 conn(0x7fea10102040 msgr2=0x7fea101038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:58.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.673+0000 7fea15800700 1 -- 172.21.15.63:0/1952163810 shutdown_connections 2024-05-27T07:48:58.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:48:58.673+0000 7fea15800700 1 -- 172.21.15.63:0/1952163810 wait complete. 2024-05-27T07:48:58.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:58 smithi063 ceph-mon[31613]: pgmap v483: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:48:58.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:48:58.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:58.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:48:58.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:59.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:58 smithi086 ceph-mon[34687]: pgmap v483: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:48:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:48:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:48:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:59.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:58 smithi149 ceph-mon[35408]: pgmap v483: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:48:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:48:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:48:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:59.558 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:48:59.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:48:59.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:59.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:48:59.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:49:00.046 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:00.201 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.202 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:49:00.202 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.202 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:49:00.202 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.202 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:49:00.202 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.202 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:48:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:48:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:49:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:49:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:49:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:49:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:49:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:49:00.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:00.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:48:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:01.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:00 smithi086 ceph-mon[34687]: pgmap v484: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:49:01.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:00 smithi149 ceph-mon[35408]: pgmap v484: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:49:01.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:00 smithi063 ceph-mon[31613]: pgmap v484: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:49:02.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.413+0000 7f4701c00700 1 -- 172.21.15.63:0/3154772020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46fc109690 msgr2=0x7f46fc110ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:02.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.413+0000 7f4701c00700 1 --2- 172.21.15.63:0/3154772020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46fc109690 0x7f46fc110ff0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f46f400b0d0 tx=0x7f46f400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.413+0000 7f4701c00700 1 -- 172.21.15.63:0/3154772020 shutdown_connections 2024-05-27T07:49:02.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.413+0000 7f4701c00700 1 --2- 172.21.15.63:0/3154772020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46fc109690 0x7f46fc110ff0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.413+0000 7f4701c00700 1 --2- 172.21.15.63:0/3154772020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46fc104410 0x7f46fc104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.413+0000 7f4701c00700 1 --2- 172.21.15.63:0/3154772020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc108d40 0x7f46fc109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.413+0000 7f4701c00700 1 -- 172.21.15.63:0/3154772020 >> 172.21.15.63:0/3154772020 conn(0x7f46fc07aca0 msgr2=0x7f46fc07b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:02.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.413+0000 7f4701c00700 1 -- 172.21.15.63:0/3154772020 shutdown_connections 2024-05-27T07:49:02.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.414+0000 7f4701c00700 1 -- 172.21.15.63:0/3154772020 wait complete. 2024-05-27T07:49:02.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.415+0000 7f4701c00700 1 Processor -- start 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.415+0000 7f4701c00700 1 -- start start 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f4701c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46fc104410 0x7f46fc075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f4701c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46fc108d40 0x7f46fc075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f4701c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc109690 0x7f46fc076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f4701c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1141e0 con 0x7f46fc108d40 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f4701c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc114060 con 0x7f46fc104410 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f4701c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc114360 con 0x7f46fc109690 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f46fb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46fc108d40 0x7f46fc075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f46fb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46fc108d40 0x7f46fc075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60526/0 (socket says 172.21.15.63:60526) 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f46fb400700 1 -- 172.21.15.63:0/2293346305 learned_addr learned my addr 172.21.15.63:0/2293346305 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f46fbe00700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46fc104410 0x7f46fc075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.416+0000 7f4701200700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc109690 0x7f46fc076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.417+0000 7f46fb400700 1 -- 172.21.15.63:0/2293346305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc109690 msgr2=0x7f46fc076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:02.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.417+0000 7f46fb400700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc109690 0x7f46fc076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.417+0000 7f46fb400700 1 -- 172.21.15.63:0/2293346305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46fc104410 msgr2=0x7f46fc075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:02.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.417+0000 7f46fb400700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46fc104410 0x7f46fc075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.417+0000 7f46fb400700 1 -- 172.21.15.63:0/2293346305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc1a8f60 con 0x7f46fc108d40 2024-05-27T07:49:02.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.417+0000 7f46fb400700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46fc108d40 0x7f46fc075ff0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f46e800d950 tx=0x7f46e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:02.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.418+0000 7f46f8c00700 1 -- 172.21.15.63:0/2293346305 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8017070 con 0x7f46fc108d40 2024-05-27T07:49:02.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.418+0000 7f46f8c00700 1 -- 172.21.15.63:0/2293346305 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46e8005020 con 0x7f46fc108d40 2024-05-27T07:49:02.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.418+0000 7f46f8c00700 1 -- 172.21.15.63:0/2293346305 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e800f980 con 0x7f46fc108d40 2024-05-27T07:49:02.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.418+0000 7f4701c00700 1 -- 172.21.15.63:0/2293346305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc1a9250 con 0x7f46fc108d40 2024-05-27T07:49:02.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.418+0000 7f4701c00700 1 -- 172.21.15.63:0/2293346305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc1a9790 con 0x7f46fc108d40 2024-05-27T07:49:02.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.424+0000 7f46f8c00700 1 -- 172.21.15.63:0/2293346305 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f46e8004a30 con 0x7f46fc108d40 2024-05-27T07:49:02.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.425+0000 7f4711dcbb80 1 Processor -- start 2024-05-27T07:49:02.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.426+0000 7f4711dcbb80 1 -- start start 2024-05-27T07:49:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.426+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c40a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.426+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c28ae40 0x55ac0c40ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.426+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 0x55ac0c4166e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.426+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ac0c337e40 con 0x55ac0c4142c0 2024-05-27T07:49:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.426+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac0c3279e0 con 0x55ac0c3769f0 2024-05-27T07:49:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.426+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ac0c32ba20 con 0x55ac0c28ae40 2024-05-27T07:49:02.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.427+0000 7f46f8c00700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46e00b2a70 0x7f46e00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.427+0000 7f46fb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 0x55ac0c4166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.427+0000 7f4701200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c40a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.427+0000 7f4701200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c40a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:57738/0 (socket says 172.21.15.63:57738) 2024-05-27T07:49:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.427+0000 7f4701200700 1 -- 172.21.15.63:0/762574185 learned_addr learned my addr 172.21.15.63:0/762574185 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.427+0000 7f46fbe00700 1 --2- 172.21.15.63:0/762574185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c28ae40 0x55ac0c40ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.427+0000 7f46f8c00700 1 -- 172.21.15.63:0/2293346305 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f46e80e20b0 con 0x7f46fc108d40 2024-05-27T07:49:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.428+0000 7f4701200700 1 -- 172.21.15.63:0/762574185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c28ae40 msgr2=0x55ac0c40ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.428+0000 7f4701200700 1 --2- 172.21.15.63:0/762574185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c28ae40 0x55ac0c40ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.428+0000 7f4701200700 1 -- 172.21.15.63:0/762574185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 msgr2=0x55ac0c4166e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.428+0000 7f4701200700 1 --2- 172.21.15.63:0/762574185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 0x55ac0c4166e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.428+0000 7f4701200700 1 -- 172.21.15.63:0/762574185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0c2f0420 con 0x55ac0c3769f0 2024-05-27T07:49:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.428+0000 7f4701200700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46e00b2a70 0x7f46e00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.428+0000 7f4701200700 1 --2- 172.21.15.63:0/762574185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c40a750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009e60 tx=0x7f46f4005aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19827f6ba6ed750a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:02.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.429+0000 7f4701200700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46e00b2a70 0x7f46e00b4f80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f46f402b000 tx=0x7f46f4025000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:02.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.429+0000 7f46f1600700 1 -- 172.21.15.63:0/762574185 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f402e070 con 0x55ac0c3769f0 2024-05-27T07:49:02.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.429+0000 7f46f1600700 1 -- 172.21.15.63:0/762574185 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46f4007a50 con 0x55ac0c3769f0 2024-05-27T07:49:02.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.429+0000 7f46f1600700 1 -- 172.21.15.63:0/762574185 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4005510 con 0x55ac0c3769f0 2024-05-27T07:49:02.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.431+0000 7f4711dcbb80 1 -- 172.21.15.63:0/762574185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 msgr2=0x55ac0c40a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:02.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.431+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/762574185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c40a750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009e60 tx=0x7f46f4005aa0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.431+0000 7f4711dcbb80 1 -- 172.21.15.63:0/762574185 shutdown_connections 2024-05-27T07:49:02.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.431+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/762574185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 0x55ac0c4166e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.431+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/762574185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c40a750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.431+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/762574185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c28ae40 0x55ac0c40ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.431+0000 7f4711dcbb80 1 -- 172.21.15.63:0/762574185 >> 172.21.15.63:0/762574185 conn(0x55ac0c324390 msgr2=0x55ac0c405710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:02.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.431+0000 7f4711dcbb80 1 -- 172.21.15.63:0/762574185 shutdown_connections 2024-05-27T07:49:02.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.431+0000 7f4711dcbb80 1 -- 172.21.15.63:0/762574185 wait complete. 2024-05-27T07:49:02.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.432+0000 7f4711dcbb80 1 -- start start 2024-05-27T07:49:02.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.432+0000 7f4711dcbb80 1 Processor -- start 2024-05-27T07:49:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.433+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c28ae40 0x55ac0c4112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.433+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c3769f0 0x55ac0c411820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.433+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c4142c0 0x55ac0c411d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.433+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ac0c22dfa0 con 0x55ac0c3769f0 2024-05-27T07:49:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.433+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac0c2c8760 con 0x55ac0c28ae40 2024-05-27T07:49:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.433+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ac0c416e00 con 0x55ac0c4142c0 2024-05-27T07:49:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.433+0000 7f4711dcbb80 10 client.0 _ll_register_callbacks cb 0x55ac0c2d1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:02.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.434+0000 7f46fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c28ae40 0x55ac0c4112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.434+0000 7f4701200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c4142c0 0x55ac0c411d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.434+0000 7f4701200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c4142c0 0x55ac0c411d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:59064/0 (socket says 172.21.15.63:59064) 2024-05-27T07:49:02.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.434+0000 7f4701200700 1 -- 172.21.15.63:0/378097494 learned_addr learned my addr 172.21.15.63:0/378097494 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:02.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.434+0000 7f46fb400700 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c3769f0 0x55ac0c411820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.435+0000 7f4701200700 1 -- 172.21.15.63:0/378097494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c28ae40 msgr2=0x55ac0c4112e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:02.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.435+0000 7f4701200700 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c28ae40 0x55ac0c4112e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.435+0000 7f4701200700 1 -- 172.21.15.63:0/378097494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c3769f0 msgr2=0x55ac0c411820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:02.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.435+0000 7f4701200700 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c3769f0 0x55ac0c411820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.435+0000 7f4701200700 1 -- 172.21.15.63:0/378097494 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0c2778d0 con 0x55ac0c4142c0 2024-05-27T07:49:02.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.435+0000 7f4701200700 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c4142c0 0x55ac0c411d60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46f400b820 tx=0x7f46f40075c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e96db5aa9db6abf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:02.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.435+0000 7f46e6000700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:49:02.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.436+0000 7f46e6000700 1 -- 172.21.15.63:0/378097494 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4036020 con 0x55ac0c4142c0 2024-05-27T07:49:02.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.436+0000 7f4711dcbb80 1 -- 172.21.15.63:0/378097494 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f4007a50 con 0x55ac0c4142c0 2024-05-27T07:49:02.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.436+0000 7f46e6000700 1 -- 172.21.15.63:0/378097494 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f46f4005280 con 0x55ac0c4142c0 2024-05-27T07:49:02.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.436+0000 7f46e6000700 1 -- 172.21.15.63:0/378097494 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f46f402e070 con 0x55ac0c4142c0 2024-05-27T07:49:02.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.436+0000 7f4711dcbb80 10 client.27670 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:02.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.436+0000 7f4711dcbb80 1 -- 172.21.15.63:0/378097494 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ac0c2f8060 con 0x55ac0c4142c0 2024-05-27T07:49:02.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.437+0000 7f46e6000700 1 -- 172.21.15.63:0/378097494 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46f4007a50 con 0x55ac0c4142c0 2024-05-27T07:49:02.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.437+0000 7f46e6000700 1 -- 172.21.15.63:0/378097494 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f46f4011070 con 0x55ac0c4142c0 2024-05-27T07:49:02.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.437+0000 7f4711dcbb80 10 client.27670 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:02.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.437+0000 7f4711dcbb80 10 client.27670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:02.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.437+0000 7f4711dcbb80 20 client.27670 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:02.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.437+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac0c423c60 0x55ac0c426080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.438+0000 7f4711dcbb80 4 client.27670 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:02.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.438+0000 7f4711dcbb80 1 -- 172.21.15.63:0/378097494 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac0c339d90 con 0x55ac0c423c60 2024-05-27T07:49:02.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.438+0000 7f46fbe00700 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac0c423c60 0x55ac0c426080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.439+0000 7f46fbe00700 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac0c423c60 0x55ac0c426080 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5f64b2761aa5be1 server_cookie=d56f970e4c91811d in_seq=0 out_seq=0 2024-05-27T07:49:02.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.439+0000 7f46e6000700 10 client.27670 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:02.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.441+0000 7f46e6000700 1 -- 172.21.15.63:0/378097494 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ac0c339d90 con 0x55ac0c423c60 2024-05-27T07:49:02.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.441+0000 7f46e6000700 10 client.27670 handle_command_reply: tid=0 2024-05-27T07:49:02.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.441+0000 7f4711dcbb80 1 client.27670 shutdown 2024-05-27T07:49:02.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.442+0000 7f4711dcbb80 1 -- 172.21.15.63:0/378097494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c4142c0 msgr2=0x55ac0c411d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:02.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.442+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c4142c0 0x55ac0c411d60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46f400b820 tx=0x7f46f40075c0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.442+0000 7f4711dcbb80 1 -- 172.21.15.63:0/378097494 shutdown_connections 2024-05-27T07:49:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.442+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac0c423c60 0x55ac0c426080 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.442+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c4142c0 0x55ac0c411d60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.442+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c28ae40 0x55ac0c4112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.442+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/378097494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c3769f0 0x55ac0c411820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.442+0000 7f4711dcbb80 1 -- 172.21.15.63:0/378097494 >> 172.21.15.63:0/378097494 conn(0x55ac0c324390 msgr2=0x55ac0c40a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.442+0000 7f46e6000700 0 client.27670 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.443+0000 7f4711dcbb80 1 -- 172.21.15.63:0/378097494 shutdown_connections 2024-05-27T07:49:02.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.443+0000 7f4711dcbb80 1 -- 172.21.15.63:0/378097494 wait complete. 2024-05-27T07:49:02.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.443+0000 7f4711dcbb80 20 client.27670 trim_cache size 0 max 16384 2024-05-27T07:49:02.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.469+0000 7f4711dcbb80 1 Processor -- start 2024-05-27T07:49:02.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.469+0000 7f4711dcbb80 1 -- start start 2024-05-27T07:49:02.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.470+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c28ae40 0x55ac0c420100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.470+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c3769f0 0x55ac0c405740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.470+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c4142c0 0x55ac0c4102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.470+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1b4cc0 con 0x55ac0c28ae40 2024-05-27T07:49:02.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.470+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1b2550 con 0x55ac0c4142c0 2024-05-27T07:49:02.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.470+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1b03c0 con 0x55ac0c3769f0 2024-05-27T07:49:02.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.470+0000 7f46fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c28ae40 0x55ac0c420100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.470+0000 7f46fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c28ae40 0x55ac0c420100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60544/0 (socket says 172.21.15.63:60544) 2024-05-27T07:49:02.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.470+0000 7f46fbe00700 1 -- 172.21.15.63:0/1475338546 learned_addr learned my addr 172.21.15.63:0/1475338546 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:02.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.471+0000 7f4701200700 1 --2- 172.21.15.63:0/1475338546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c4142c0 0x55ac0c4102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.471+0000 7f46fb400700 1 --2- 172.21.15.63:0/1475338546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c3769f0 0x55ac0c405740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.471+0000 7f46fbe00700 1 -- 172.21.15.63:0/1475338546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c3769f0 msgr2=0x55ac0c405740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:02.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.471+0000 7f46fbe00700 1 --2- 172.21.15.63:0/1475338546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c3769f0 0x55ac0c405740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.471+0000 7f46fbe00700 1 -- 172.21.15.63:0/1475338546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c4142c0 msgr2=0x55ac0c4102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.471+0000 7f46fbe00700 1 --2- 172.21.15.63:0/1475338546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c4142c0 0x55ac0c4102e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.471+0000 7f46fbe00700 1 -- 172.21.15.63:0/1475338546 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0c40f850 con 0x55ac0c28ae40 2024-05-27T07:49:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.471+0000 7f4701200700 1 --2- 172.21.15.63:0/1475338546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c4142c0 0x55ac0c4102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.471+0000 7f46fbe00700 1 --2- 172.21.15.63:0/1475338546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c28ae40 0x55ac0c420100 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x55ac0c2c8790 tx=0x7f46ec0073d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f611fc78d568b485 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.472+0000 7f46f0c00700 1 -- 172.21.15.63:0/1475338546 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec004230 con 0x55ac0c28ae40 2024-05-27T07:49:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.472+0000 7f46f0c00700 1 -- 172.21.15.63:0/1475338546 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46ec0043d0 con 0x55ac0c28ae40 2024-05-27T07:49:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.472+0000 7f46f0c00700 1 -- 172.21.15.63:0/1475338546 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec01d890 con 0x55ac0c28ae40 2024-05-27T07:49:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.473+0000 7f4711dcbb80 1 -- 172.21.15.63:0/1475338546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c28ae40 msgr2=0x55ac0c420100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:02.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.473+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/1475338546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c28ae40 0x55ac0c420100 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x55ac0c2c8790 tx=0x7f46ec0073d0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.473+0000 7f4711dcbb80 1 -- 172.21.15.63:0/1475338546 shutdown_connections 2024-05-27T07:49:02.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.473+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/1475338546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c4142c0 0x55ac0c4102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.473+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/1475338546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c28ae40 0x55ac0c420100 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.473+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/1475338546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c3769f0 0x55ac0c405740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.473+0000 7f4711dcbb80 1 -- 172.21.15.63:0/1475338546 >> 172.21.15.63:0/1475338546 conn(0x55ac0c324390 msgr2=0x55ac0c40fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:02.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.473+0000 7f4711dcbb80 1 -- 172.21.15.63:0/1475338546 shutdown_connections 2024-05-27T07:49:02.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.474+0000 7f4711dcbb80 1 -- 172.21.15.63:0/1475338546 wait complete. 2024-05-27T07:49:02.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.474+0000 7f4711dcbb80 1 -- start start 2024-05-27T07:49:02.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.474+0000 7f4711dcbb80 1 Processor -- start 2024-05-27T07:49:02.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.475+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c28ae40 0x55ac0c429730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.475+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c426b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.475+0000 7f4711dcbb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 0x55ac0c427040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.475+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1b0a10 con 0x55ac0c4142c0 2024-05-27T07:49:02.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.475+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1b1e80 con 0x55ac0c3769f0 2024-05-27T07:49:02.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.475+0000 7f4711dcbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1b10e0 con 0x55ac0c28ae40 2024-05-27T07:49:02.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.475+0000 7f4711dcbb80 10 client.0 _ll_register_callbacks cb 0x55ac0c2d1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.475+0000 7f4701200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 0x55ac0c427040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.476+0000 7f4701200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 0x55ac0c427040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60546/0 (socket says 172.21.15.63:60546) 2024-05-27T07:49:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.476+0000 7f4701200700 1 -- 172.21.15.63:0/4169050060 learned_addr learned my addr 172.21.15.63:0/4169050060 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.476+0000 7f46fb400700 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c426b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.476+0000 7f46fbe00700 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c28ae40 0x55ac0c429730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.476+0000 7f4701200700 1 -- 172.21.15.63:0/4169050060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c28ae40 msgr2=0x55ac0c429730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:02.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.476+0000 7f4701200700 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c28ae40 0x55ac0c429730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.476+0000 7f4701200700 1 -- 172.21.15.63:0/4169050060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 msgr2=0x55ac0c426b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:02.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.476+0000 7f4701200700 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c426b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.476+0000 7f4701200700 1 -- 172.21.15.63:0/4169050060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0c427730 con 0x55ac0c4142c0 2024-05-27T07:49:02.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.476+0000 7f46fb400700 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c426b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:02.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.477+0000 7f4701200700 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 0x55ac0c427040 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f46f4006260 tx=0x7f46f401e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cffc927a5e1b817 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:02.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.477+0000 7f46e6000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:02.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.477+0000 7f46e6000700 1 -- 172.21.15.63:0/4169050060 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4036070 con 0x55ac0c4142c0 2024-05-27T07:49:02.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.477+0000 7f46e6000700 1 -- 172.21.15.63:0/4169050060 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f46f4005280 con 0x55ac0c4142c0 2024-05-27T07:49:02.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.477+0000 7f4711dcbb80 1 -- 172.21.15.63:0/4169050060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ec0043d0 con 0x55ac0c4142c0 2024-05-27T07:49:02.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.477+0000 7f46e6000700 1 -- 172.21.15.63:0/4169050060 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f46f4005480 con 0x55ac0c4142c0 2024-05-27T07:49:02.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.478+0000 7f4711dcbb80 10 client.18420 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:02.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.478+0000 7f4711dcbb80 1 -- 172.21.15.63:0/4169050060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ac0c496b80 con 0x55ac0c4142c0 2024-05-27T07:49:02.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.478+0000 7f46e6000700 1 -- 172.21.15.63:0/4169050060 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46ec0043d0 con 0x55ac0c4142c0 2024-05-27T07:49:02.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.478+0000 7f46e6000700 1 -- 172.21.15.63:0/4169050060 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f46f4011040 con 0x55ac0c4142c0 2024-05-27T07:49:02.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.478+0000 7f4711dcbb80 10 client.18420 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:02.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.478+0000 7f4711dcbb80 10 client.18420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:02.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.478+0000 7f4711dcbb80 20 client.18420 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:02.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.478+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac0c496fe0 0x55ac0c499400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:02.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.478+0000 7f4711dcbb80 4 client.18420 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:49:02.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.478+0000 7f4711dcbb80 1 -- 172.21.15.63:0/4169050060 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac0c333570 con 0x55ac0c496fe0 2024-05-27T07:49:02.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.479+0000 7f46fbe00700 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac0c496fe0 0x55ac0c499400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:02.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.480+0000 7f46e6000700 10 client.18420 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:02.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.480+0000 7f46fbe00700 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac0c496fe0 0x55ac0c499400 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53401c985f2ba993 server_cookie=554ea09664e8cd1b in_seq=0 out_seq=0 2024-05-27T07:49:02.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f46e6000700 1 -- 172.21.15.63:0/4169050060 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ac0c333570 con 0x55ac0c496fe0 2024-05-27T07:49:02.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f46e6000700 10 client.18420 handle_command_reply: tid=0 2024-05-27T07:49:02.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f4711dcbb80 1 client.18420 shutdown 2024-05-27T07:49:02.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f4711dcbb80 1 -- 172.21.15.63:0/4169050060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 msgr2=0x55ac0c427040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:02.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 0x55ac0c427040 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f46f4006260 tx=0x7f46f401e660 comp rx=0 tx=0).stop 2024-05-27T07:49:02.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f4711dcbb80 1 -- 172.21.15.63:0/4169050060 shutdown_connections 2024-05-27T07:49:02.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac0c496fe0 0x55ac0c499400 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac0c4142c0 0x55ac0c427040 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac0c28ae40 0x55ac0c429730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f4711dcbb80 1 --2- 172.21.15.63:0/4169050060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c3769f0 0x55ac0c426b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f4711dcbb80 1 -- 172.21.15.63:0/4169050060 >> 172.21.15.63:0/4169050060 conn(0x55ac0c324390 msgr2=0x55ac0c410360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:02.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.481+0000 7f46e6000700 0 client.18420 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:02.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.482+0000 7f4711dcbb80 1 -- 172.21.15.63:0/4169050060 shutdown_connections 2024-05-27T07:49:02.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.482+0000 7f4711dcbb80 1 -- 172.21.15.63:0/4169050060 wait complete. 2024-05-27T07:49:02.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.482+0000 7f4711dcbb80 20 client.18420 trim_cache size 0 max 16384 2024-05-27T07:49:02.482 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:49:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:49:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:49:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "a09a7791-2cf9-4812-ab4f-b3e4c5fef712": { 2024-05-27T07:49:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:49:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "a09a7791-2cf9-4812-ab4f-b3e4c5fef712", 2024-05-27T07:49:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:49:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:49:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:49:02.483 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.483+0000 7f4701c00700 1 -- 172.21.15.63:0/2293346305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46e00b2a70 msgr2=0x7f46e00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.483+0000 7f4701c00700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46e00b2a70 0x7f46e00b4f80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f46f402b000 tx=0x7f46f4025000 comp rx=0 tx=0).stop 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.483+0000 7f4701c00700 1 -- 172.21.15.63:0/2293346305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46fc108d40 msgr2=0x7f46fc075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.483+0000 7f4701c00700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46fc108d40 0x7f46fc075ff0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f46e800d950 tx=0x7f46e800dc60 comp rx=0 tx=0).stop 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.483+0000 7f4701c00700 1 -- 172.21.15.63:0/2293346305 shutdown_connections 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.483+0000 7f4701c00700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46e00b2a70 0x7f46e00b4f80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.483+0000 7f4701c00700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc109690 0x7f46fc076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.483+0000 7f4701c00700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46fc104410 0x7f46fc075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.483+0000 7f4701c00700 1 --2- 172.21.15.63:0/2293346305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46fc108d40 0x7f46fc075ff0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.483+0000 7f4701c00700 1 -- 172.21.15.63:0/2293346305 >> 172.21.15.63:0/2293346305 conn(0x7f46fc07aca0 msgr2=0x7f46fc101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:02.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.484+0000 7f4701c00700 1 -- 172.21.15.63:0/2293346305 shutdown_connections 2024-05-27T07:49:02.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:02.484+0000 7f4701c00700 1 -- 172.21.15.63:0/2293346305 wait complete. 2024-05-27T07:49:03.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:02 smithi086 ceph-mon[34687]: pgmap v485: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.1 MiB/s wr, 58 op/s 2024-05-27T07:49:03.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:02 smithi149 ceph-mon[35408]: pgmap v485: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.1 MiB/s wr, 58 op/s 2024-05-27T07:49:03.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:02 smithi063 ceph-mon[31613]: pgmap v485: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.1 MiB/s wr, 58 op/s 2024-05-27T07:49:03.372 INFO:tasks.cephfs.filesystem:scrub status for tag:a09a7791-2cf9-4812-ab4f-b3e4c5fef712 - {'path': '/', 'tag': 'a09a7791-2cf9-4812-ab4f-b3e4c5fef712', 'options': 'recursive,force'} 2024-05-27T07:49:05.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:04 smithi086 ceph-mon[34687]: pgmap v486: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.7 MiB/s wr, 55 op/s 2024-05-27T07:49:05.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:04 smithi149 ceph-mon[35408]: pgmap v486: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.7 MiB/s wr, 55 op/s 2024-05-27T07:49:05.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:04 smithi063 ceph-mon[31613]: pgmap v486: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.7 MiB/s wr, 55 op/s 2024-05-27T07:49:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.752+0000 7f6ee0e00640 1 -- 172.21.15.63:0/1978951105 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc10e000 msgr2=0x7f6edc1104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.752+0000 7f6ee0e00640 1 --2- 172.21.15.63:0/1978951105 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc10e000 0x7f6edc1104f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009900 tx=0x7f6ed402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.753+0000 7f6ee0e00640 1 -- 172.21.15.63:0/1978951105 shutdown_connections 2024-05-27T07:49:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.753+0000 7f6ee0e00640 1 --2- 172.21.15.63:0/1978951105 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc10e000 0x7f6edc1104f0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.753+0000 7f6ee0e00640 1 --2- 172.21.15.63:0/1978951105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc10b610 0x7f6edc10da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.753+0000 7f6ee0e00640 1 --2- 172.21.15.63:0/1978951105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6edc104bb0 0x7f6edc104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.753+0000 7f6ee0e00640 1 -- 172.21.15.63:0/1978951105 >> 172.21.15.63:0/1978951105 conn(0x7f6edc0fe210 msgr2=0x7f6edc100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.753+0000 7f6ee0e00640 1 -- 172.21.15.63:0/1978951105 shutdown_connections 2024-05-27T07:49:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.753+0000 7f6ee0e00640 1 -- 172.21.15.63:0/1978951105 wait complete. 2024-05-27T07:49:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.754+0000 7f6ee0e00640 1 Processor -- start 2024-05-27T07:49:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.754+0000 7f6ee0e00640 1 -- start start 2024-05-27T07:49:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6ee0e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc104bb0 0x7f6edc19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6ee0e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6edc10b610 0x7f6edc19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6ee0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc10e000 0x7f6edc1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6ee0e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc113b10 con 0x7f6edc104bb0 2024-05-27T07:49:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6ee0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc113990 con 0x7f6edc10e000 2024-05-27T07:49:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6ee0e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc113c90 con 0x7f6edc10b610 2024-05-27T07:49:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6edb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc104bb0 0x7f6edc19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6edb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc104bb0 0x7f6edc19c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60562/0 (socket says 172.21.15.63:60562) 2024-05-27T07:49:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6edb400640 1 -- 172.21.15.63:0/3845136882 learned_addr learned my addr 172.21.15.63:0/3845136882 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6edaa00640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6edc10b610 0x7f6edc19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.755+0000 7f6edbe00640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc10e000 0x7f6edc1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.756+0000 7f6edb400640 1 -- 172.21.15.63:0/3845136882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6edc10b610 msgr2=0x7f6edc19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.756+0000 7f6edb400640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6edc10b610 0x7f6edc19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.756+0000 7f6edb400640 1 -- 172.21.15.63:0/3845136882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc10e000 msgr2=0x7f6edc1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.756+0000 7f6edb400640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc10e000 0x7f6edc1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.756+0000 7f6edb400640 1 -- 172.21.15.63:0/3845136882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc1a1610 con 0x7f6edc104bb0 2024-05-27T07:49:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.756+0000 7f6edb400640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc104bb0 0x7f6edc19c7b0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00db80 tx=0x7f6ecc00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.756+0000 7f6ed3e00640 1 -- 172.21.15.63:0/3845136882 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc017070 con 0x7f6edc104bb0 2024-05-27T07:49:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.757+0000 7f6ed3e00640 1 -- 172.21.15.63:0/3845136882 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ecc004020 con 0x7f6edc104bb0 2024-05-27T07:49:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.757+0000 7f6ed3e00640 1 -- 172.21.15.63:0/3845136882 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc00be60 con 0x7f6edc104bb0 2024-05-27T07:49:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.757+0000 7f6ee0e00640 1 -- 172.21.15.63:0/3845136882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6edc1a1900 con 0x7f6edc104bb0 2024-05-27T07:49:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.757+0000 7f6ee0e00640 1 -- 172.21.15.63:0/3845136882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6edc1a1e00 con 0x7f6edc104bb0 2024-05-27T07:49:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.760+0000 7f6ee9566740 1 Processor -- start 2024-05-27T07:49:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6ee9566740 1 -- start start 2024-05-27T07:49:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7542a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c7542fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c717f4d0 0x5570c75434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6ee9566740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5570c74b3b90 con 0x5570c71a01b0 2024-05-27T07:49:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6ee9566740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5570c74a3730 con 0x5570c71772f0 2024-05-27T07:49:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6ee9566740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5570c74a7770 con 0x5570c717f4d0 2024-05-27T07:49:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6edbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7542a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6edbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7542a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60578/0 (socket says 172.21.15.63:60578) 2024-05-27T07:49:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6edbe00640 1 -- 172.21.15.63:0/806148104 learned_addr learned my addr 172.21.15.63:0/806148104 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.761+0000 7f6ed3e00640 1 -- 172.21.15.63:0/3845136882 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6ecc003b70 con 0x7f6edc104bb0 2024-05-27T07:49:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.762+0000 7f6edaa00640 1 --2- 172.21.15.63:0/806148104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c717f4d0 0x5570c75434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.762+0000 7f6edbe00640 1 -- 172.21.15.63:0/806148104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c717f4d0 msgr2=0x5570c75434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.762+0000 7f6edbe00640 1 --2- 172.21.15.63:0/806148104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c717f4d0 0x5570c75434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.762+0000 7f6edbe00640 1 -- 172.21.15.63:0/806148104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 msgr2=0x5570c7542fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.762+0000 7f6edbe00640 1 --2- 172.21.15.63:0/806148104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c7542fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.762+0000 7f6edbe00640 1 -- 172.21.15.63:0/806148104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c743c8b0 con 0x5570c71a01b0 2024-05-27T07:49:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.762+0000 7f6edbe00640 1 --2- 172.21.15.63:0/806148104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7542a60 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009900 tx=0x7f6ed4032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cd52655c8533f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.763+0000 7f6ed0c00640 1 -- 172.21.15.63:0/806148104 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed403f070 con 0x5570c71a01b0 2024-05-27T07:49:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.763+0000 7f6ed0c00640 1 -- 172.21.15.63:0/806148104 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ed4004030 con 0x5570c71a01b0 2024-05-27T07:49:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.763+0000 7f6ed3e00640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6ec00b29d0 0x7f6ec00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.764+0000 7f6ed0c00640 1 -- 172.21.15.63:0/806148104 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4035070 con 0x5570c71a01b0 2024-05-27T07:49:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.764+0000 7f6ed3e00640 1 -- 172.21.15.63:0/3845136882 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f6ecc0e2a90 con 0x7f6edc104bb0 2024-05-27T07:49:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.764+0000 7f6edb400640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6ec00b29d0 0x7f6ec00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.764+0000 7f6ee9566740 1 -- 172.21.15.63:0/806148104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 msgr2=0x5570c7542a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.764+0000 7f6ee9566740 1 --2- 172.21.15.63:0/806148104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7542a60 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009900 tx=0x7f6ed4032bb0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.764+0000 7f6ee9566740 1 -- 172.21.15.63:0/806148104 shutdown_connections 2024-05-27T07:49:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.764+0000 7f6ee9566740 1 --2- 172.21.15.63:0/806148104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c717f4d0 0x5570c75434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.764+0000 7f6ee9566740 1 --2- 172.21.15.63:0/806148104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c7542fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.764+0000 7f6ee9566740 1 --2- 172.21.15.63:0/806148104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7542a60 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.764+0000 7f6ee9566740 1 -- 172.21.15.63:0/806148104 >> 172.21.15.63:0/806148104 conn(0x5570c7194300 msgr2=0x5570c753f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.765+0000 7f6ee9566740 1 -- 172.21.15.63:0/806148104 shutdown_connections 2024-05-27T07:49:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.765+0000 7f6ee9566740 1 -- 172.21.15.63:0/806148104 wait complete. 2024-05-27T07:49:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.765+0000 7f6edb400640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6ec00b29d0 0x7f6ec00b4ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5570c7540550 tx=0x7f6ecc021000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.765+0000 7f6ee9566740 1 -- start start 2024-05-27T07:49:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.766+0000 7f6ee9566740 1 Processor -- start 2024-05-27T07:49:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.766+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c755b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.766+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c717f4d0 0x5570c755ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.766+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7555920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.766+0000 7f6ee9566740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5570c7189790 con 0x5570c71a01b0 2024-05-27T07:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.766+0000 7f6ee9566740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5570c744bbc0 con 0x5570c71772f0 2024-05-27T07:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.766+0000 7f6ee9566740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5570c754fb80 con 0x5570c717f4d0 2024-05-27T07:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.766+0000 7f6ee9566740 10 client.0 _ll_register_callbacks cb 0x5570c7429400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.767+0000 7f6edbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7555920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.767+0000 7f6edbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7555920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60584/0 (socket says 172.21.15.63:60584) 2024-05-27T07:49:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.767+0000 7f6edbe00640 1 -- 172.21.15.63:0/3260380441 learned_addr learned my addr 172.21.15.63:0/3260380441 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.767+0000 7f6edbe00640 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c755b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.767+0000 7f6edaa00640 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c717f4d0 0x5570c755ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.767+0000 7f6edbe00640 1 -- 172.21.15.63:0/3260380441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c717f4d0 msgr2=0x5570c755ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.767+0000 7f6edbe00640 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c717f4d0 0x5570c755ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.767+0000 7f6edbe00640 1 -- 172.21.15.63:0/3260380441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 msgr2=0x5570c755b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.767+0000 7f6edbe00640 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c755b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.767+0000 7f6edbe00640 1 -- 172.21.15.63:0/3260380441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c71b8210 con 0x5570c71a01b0 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.768+0000 7f6edbe00640 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7555920 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x5570c7555180 tx=0x7f6ed40050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b658610d6f38bea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.768+0000 7f6ec5600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.768+0000 7f6ec5600640 1 -- 172.21.15.63:0/3260380441 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4035030 con 0x5570c71a01b0 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.768+0000 7f6ee9566740 1 -- 172.21.15.63:0/3260380441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ed4004030 con 0x5570c71a01b0 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.768+0000 7f6ec5600640 1 -- 172.21.15.63:0/3260380441 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f6ed4031400 con 0x5570c71a01b0 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.768+0000 7f6ec5600640 1 -- 172.21.15.63:0/3260380441 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ed4059050 con 0x5570c71a01b0 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.769+0000 7f6ee9566740 10 client.18432 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.769+0000 7f6ee9566740 1 -- 172.21.15.63:0/3260380441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5570c7284620 con 0x5570c71a01b0 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.769+0000 7f6ec5600640 1 -- 172.21.15.63:0/3260380441 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ed4004030 con 0x5570c71a01b0 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.769+0000 7f6ec5600640 1 -- 172.21.15.63:0/3260380441 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6ed404e020 con 0x5570c71a01b0 2024-05-27T07:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.769+0000 7f6ee9566740 10 client.18432 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.769+0000 7f6ee9566740 10 client.18432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.769+0000 7f6ee9566740 20 client.18432 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.769+0000 7f6ee9566740 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c715ed80 0x5570c7567460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.769+0000 7f6ee9566740 4 client.18432 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.769+0000 7f6ee9566740 1 -- 172.21.15.63:0/3260380441 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570c74b5ae0 con 0x5570c715ed80 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.770+0000 7f6edaa00640 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c715ed80 0x5570c7567460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.771+0000 7f6edaa00640 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c715ed80 0x5570c7567460 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340bd8771b323015 server_cookie=e105794c58aaaceb in_seq=0 out_seq=0 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.771+0000 7f6ec5600640 10 client.18432 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.773+0000 7f6ec5600640 1 -- 172.21.15.63:0/3260380441 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5570c74b5ae0 con 0x5570c715ed80 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.773+0000 7f6ec5600640 10 client.18432 handle_command_reply: tid=0 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.773+0000 7f6ee9566740 1 client.18432 shutdown 2024-05-27T07:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.773+0000 7f6ee9566740 1 -- 172.21.15.63:0/3260380441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 msgr2=0x5570c7555920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.773+0000 7f6ee9566740 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7555920 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x5570c7555180 tx=0x7f6ed40050e0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.774+0000 7f6ee9566740 1 -- 172.21.15.63:0/3260380441 shutdown_connections 2024-05-27T07:49:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.774+0000 7f6ee9566740 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c715ed80 0x5570c7567460 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.774+0000 7f6ee9566740 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7555920 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.774+0000 7f6ee9566740 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c717f4d0 0x5570c755ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.774+0000 7f6ee9566740 1 --2- 172.21.15.63:0/3260380441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c755b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.774+0000 7f6ec5600640 0 client.18432 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.774+0000 7f6ee9566740 1 -- 172.21.15.63:0/3260380441 >> 172.21.15.63:0/3260380441 conn(0x5570c7194300 msgr2=0x5570c7542830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.774+0000 7f6ee9566740 1 -- 172.21.15.63:0/3260380441 shutdown_connections 2024-05-27T07:49:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.774+0000 7f6ee9566740 1 -- 172.21.15.63:0/3260380441 wait complete. 2024-05-27T07:49:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.774+0000 7f6ee9566740 20 client.18432 trim_cache size 0 max 16384 2024-05-27T07:49:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.796+0000 7f6ee9566740 1 Processor -- start 2024-05-27T07:49:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.796+0000 7f6ee9566740 1 -- start start 2024-05-27T07:49:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.796+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71772f0 0x5570c754f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.796+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c717f4d0 0x5570c7544710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.796+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7552b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.796+0000 7f6ee9566740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc1b1700 con 0x5570c71a01b0 2024-05-27T07:49:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.796+0000 7f6ee9566740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc1aefd0 con 0x5570c717f4d0 2024-05-27T07:49:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.796+0000 7f6ee9566740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc1ace80 con 0x5570c71772f0 2024-05-27T07:49:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.797+0000 7f6edbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7552b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.797+0000 7f6edbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7552b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60588/0 (socket says 172.21.15.63:60588) 2024-05-27T07:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.797+0000 7f6edbe00640 1 -- 172.21.15.63:0/2718949867 learned_addr learned my addr 172.21.15.63:0/2718949867 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.797+0000 7f6edaa00640 1 --2- 172.21.15.63:0/2718949867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c717f4d0 0x5570c7544710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.797+0000 7f6edbe00640 1 --2- 172.21.15.63:0/2718949867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71772f0 0x5570c754f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.797+0000 7f6edbe00640 1 -- 172.21.15.63:0/2718949867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71772f0 msgr2=0x5570c754f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.797+0000 7f6edbe00640 1 --2- 172.21.15.63:0/2718949867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71772f0 0x5570c754f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.797+0000 7f6edbe00640 1 -- 172.21.15.63:0/2718949867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c717f4d0 msgr2=0x5570c7544710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.797+0000 7f6edbe00640 1 --2- 172.21.15.63:0/2718949867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c717f4d0 0x5570c7544710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.797+0000 7f6edbe00640 1 -- 172.21.15.63:0/2718949867 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c7543950 con 0x5570c71a01b0 2024-05-27T07:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.798+0000 7f6edbe00640 1 --2- 172.21.15.63:0/2718949867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7552b60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4003650 tx=0x7f6ed4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900ad5b3ae7effae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.798+0000 7f6ec7e00640 1 -- 172.21.15.63:0/2718949867 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4035070 con 0x5570c71a01b0 2024-05-27T07:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.798+0000 7f6ec7e00640 1 -- 172.21.15.63:0/2718949867 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ed4032bb0 con 0x5570c71a01b0 2024-05-27T07:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.798+0000 7f6ec7e00640 1 -- 172.21.15.63:0/2718949867 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4056040 con 0x5570c71a01b0 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.799+0000 7f6ee9566740 1 -- 172.21.15.63:0/2718949867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 msgr2=0x5570c7552b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.799+0000 7f6ee9566740 1 --2- 172.21.15.63:0/2718949867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7552b60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4003650 tx=0x7f6ed4005fd0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.799+0000 7f6ee9566740 1 -- 172.21.15.63:0/2718949867 shutdown_connections 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.799+0000 7f6ee9566740 1 --2- 172.21.15.63:0/2718949867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c71a01b0 0x5570c7552b60 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.799+0000 7f6ee9566740 1 --2- 172.21.15.63:0/2718949867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c717f4d0 0x5570c7544710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.799+0000 7f6ee9566740 1 --2- 172.21.15.63:0/2718949867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71772f0 0x5570c754f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.799+0000 7f6ee9566740 1 -- 172.21.15.63:0/2718949867 >> 172.21.15.63:0/2718949867 conn(0x5570c7194300 msgr2=0x5570c7555be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.799+0000 7f6ee9566740 1 -- 172.21.15.63:0/2718949867 shutdown_connections 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.799+0000 7f6ee9566740 1 -- 172.21.15.63:0/2718949867 wait complete. 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.800+0000 7f6ee9566740 1 -- start start 2024-05-27T07:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.800+0000 7f6ee9566740 1 Processor -- start 2024-05-27T07:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c75c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c717f4d0 0x5570c75c7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6ee9566740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71a01b0 0x5570c7561070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6ee9566740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc1ad4d0 con 0x5570c717f4d0 2024-05-27T07:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6ee9566740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc1ae900 con 0x5570c71772f0 2024-05-27T07:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6ee9566740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc1adb60 con 0x5570c71a01b0 2024-05-27T07:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6ee9566740 10 client.0 _ll_register_callbacks cb 0x7f6ed4032a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6edaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c717f4d0 0x5570c75c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6edaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c717f4d0 0x5570c75c7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60592/0 (socket says 172.21.15.63:60592) 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6edaa00640 1 -- 172.21.15.63:0/529847011 learned_addr learned my addr 172.21.15.63:0/529847011 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6edbe00640 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c75c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.801+0000 7f6edbe00640 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71a01b0 0x5570c7561070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6edaa00640 1 -- 172.21.15.63:0/529847011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71a01b0 msgr2=0x5570c7561070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6edaa00640 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71a01b0 0x5570c7561070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6edaa00640 1 -- 172.21.15.63:0/529847011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 msgr2=0x5570c75c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6edaa00640 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c75c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6edaa00640 1 -- 172.21.15.63:0/529847011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c7561660 con 0x5570c717f4d0 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6edbe00640 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71a01b0 0x5570c7561070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6edaa00640 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c717f4d0 0x5570c75c7350 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x5570c74b3b90 tx=0x7f6ec801b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91a86881485465f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6ec5600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6ec5600640 1 -- 172.21.15.63:0/529847011 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8004230 con 0x5570c717f4d0 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6ec5600640 1 -- 172.21.15.63:0/529847011 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9483+0+0 (secure 0 0 0) 0x7f6ec80043d0 con 0x5570c717f4d0 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.802+0000 7f6ee9566740 1 -- 172.21.15.63:0/529847011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ed4032bb0 con 0x5570c717f4d0 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ec5600640 1 -- 172.21.15.63:0/529847011 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ec802d050 con 0x5570c717f4d0 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ee9566740 10 client.18438 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ee9566740 1 -- 172.21.15.63:0/529847011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5570c75cf5e0 con 0x5570c717f4d0 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ec5600640 1 -- 172.21.15.63:0/529847011 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ed4032bb0 con 0x5570c717f4d0 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ec5600640 1 -- 172.21.15.63:0/529847011 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6ec8022020 con 0x5570c717f4d0 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ee9566740 10 client.18438 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ee9566740 10 client.18438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ee9566740 20 client.18438 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ee9566740 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c71341c0 0x5570c75d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ee9566740 4 client.18438 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:49:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.803+0000 7f6ee9566740 1 -- 172.21.15.63:0/529847011 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570c74af2c0 con 0x5570c71341c0 2024-05-27T07:49:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.804+0000 7f6edbe00640 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c71341c0 0x5570c75d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.805+0000 7f6ec5600640 10 client.18438 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.805+0000 7f6edbe00640 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c71341c0 0x5570c75d1e20 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c90997918558a343 server_cookie=44fcad07bd962807 in_seq=0 out_seq=0 2024-05-27T07:49:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.838+0000 7f6ec5600640 1 -- 172.21.15.63:0/529847011 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5570c74af2c0 con 0x5570c71341c0 2024-05-27T07:49:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.838+0000 7f6ec5600640 10 client.18438 handle_command_reply: tid=0 2024-05-27T07:49:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 client.18438 shutdown 2024-05-27T07:49:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 -- 172.21.15.63:0/529847011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c717f4d0 msgr2=0x5570c75c7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c717f4d0 0x5570c75c7350 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x5570c74b3b90 tx=0x7f6ec801b720 comp rx=0 tx=0).stop 2024-05-27T07:49:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 -- 172.21.15.63:0/529847011 shutdown_connections 2024-05-27T07:49:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5570c71341c0 0x5570c75d1e20 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570c71a01b0 0x5570c7561070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570c717f4d0 0x5570c75c7350 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 --2- 172.21.15.63:0/529847011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5570c71772f0 0x5570c75c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ec5600640 0 client.18438 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 -- 172.21.15.63:0/529847011 >> 172.21.15.63:0/529847011 conn(0x5570c7194300 msgr2=0x5570c75446a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 -- 172.21.15.63:0/529847011 shutdown_connections 2024-05-27T07:49:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.839+0000 7f6ee9566740 1 -- 172.21.15.63:0/529847011 wait complete. 2024-05-27T07:49:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.840+0000 7f6ee9566740 20 client.18438 trim_cache size 0 max 16384 2024-05-27T07:49:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:49:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:49:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:49:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.840+0000 7f6ee0e00640 1 -- 172.21.15.63:0/3845136882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6ec00b29d0 msgr2=0x7f6ec00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.840+0000 7f6ee0e00640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6ec00b29d0 0x7f6ec00b4ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5570c7540550 tx=0x7f6ecc021000 comp rx=0 tx=0).stop 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.840+0000 7f6ee0e00640 1 -- 172.21.15.63:0/3845136882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc104bb0 msgr2=0x7f6edc19c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.840+0000 7f6ee0e00640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc104bb0 0x7f6edc19c7b0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00db80 tx=0x7f6ecc00b630 comp rx=0 tx=0).stop 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.841+0000 7f6ee0e00640 1 -- 172.21.15.63:0/3845136882 shutdown_connections 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.841+0000 7f6ee0e00640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6ec00b29d0 0x7f6ec00b4ee0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.841+0000 7f6ee0e00640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6edc10e000 0x7f6edc1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.841+0000 7f6ee0e00640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6edc10b610 0x7f6edc19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.841+0000 7f6ee0e00640 1 --2- 172.21.15.63:0/3845136882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc104bb0 0x7f6edc19c7b0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.841+0000 7f6ee0e00640 1 -- 172.21.15.63:0/3845136882 >> 172.21.15.63:0/3845136882 conn(0x7f6edc0fe210 msgr2=0x7f6edc109c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.841+0000 7f6ee0e00640 1 -- 172.21.15.63:0/3845136882 shutdown_connections 2024-05-27T07:49:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:05.841+0000 7f6ee0e00640 1 -- 172.21.15.63:0/3845136882 wait complete. 2024-05-27T07:49:07.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:06 smithi086 ceph-mon[34687]: pgmap v487: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.7 MiB/s wr, 55 op/s 2024-05-27T07:49:07.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:06 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:49:07.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:06 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:49:07.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:06 smithi149 ceph-mon[35408]: pgmap v487: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.7 MiB/s wr, 55 op/s 2024-05-27T07:49:07.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:06 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:49:07.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:06 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:49:07.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:06 smithi063 ceph-mon[31613]: pgmap v487: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.7 MiB/s wr, 55 op/s 2024-05-27T07:49:07.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:06 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:49:07.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:06 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:49:08.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:49:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:07 smithi086 ceph-mon[34687]: osdmap e112: 12 total, 12 up, 12 in 2024-05-27T07:49:08.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:49:08.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:07 smithi149 ceph-mon[35408]: osdmap e112: 12 total, 12 up, 12 in 2024-05-27T07:49:08.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:49:08.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:07 smithi063 ceph-mon[31613]: osdmap e112: 12 total, 12 up, 12 in 2024-05-27T07:49:09.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:08 smithi086 ceph-mon[34687]: pgmap v489: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.1 MiB/s wr, 67 op/s 2024-05-27T07:49:09.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:08 smithi149 ceph-mon[35408]: pgmap v489: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.1 MiB/s wr, 67 op/s 2024-05-27T07:49:09.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:08 smithi063 ceph-mon[31613]: pgmap v489: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.1 MiB/s wr, 67 op/s 2024-05-27T07:49:11.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:10 smithi086 ceph-mon[34687]: pgmap v490: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.1 MiB/s wr, 67 op/s 2024-05-27T07:49:11.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:10 smithi149 ceph-mon[35408]: pgmap v490: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.1 MiB/s wr, 67 op/s 2024-05-27T07:49:11.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:10 smithi063 ceph-mon[31613]: pgmap v490: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.1 MiB/s wr, 67 op/s 2024-05-27T07:49:13.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:12 smithi086 ceph-mon[34687]: pgmap v491: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 12 op/s 2024-05-27T07:49:13.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:12 smithi149 ceph-mon[35408]: pgmap v491: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 12 op/s 2024-05-27T07:49:13.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:12 smithi063 ceph-mon[31613]: pgmap v491: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 12 op/s 2024-05-27T07:49:15.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:14 smithi086 ceph-mon[34687]: pgmap v492: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 12 op/s 2024-05-27T07:49:15.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:14 smithi149 ceph-mon[35408]: pgmap v492: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 12 op/s 2024-05-27T07:49:15.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:14 smithi063 ceph-mon[31613]: pgmap v492: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 12 op/s 2024-05-27T07:49:16.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:16 smithi086 ceph-mon[34687]: osdmap e113: 12 total, 12 up, 12 in 2024-05-27T07:49:16.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:16 smithi086 ceph-mon[34687]: pgmap v494: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-05-27T07:49:16.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:16 smithi149 ceph-mon[35408]: osdmap e113: 12 total, 12 up, 12 in 2024-05-27T07:49:16.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:16 smithi149 ceph-mon[35408]: pgmap v494: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-05-27T07:49:16.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:16 smithi063 ceph-mon[31613]: osdmap e113: 12 total, 12 up, 12 in 2024-05-27T07:49:16.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:16 smithi063 ceph-mon[31613]: pgmap v494: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.3 MiB/s wr, 4 op/s 2024-05-27T07:49:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.077+0000 7fdcd6a00640 1 -- 172.21.15.63:0/2433066604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd0103180 msgr2=0x7fdcd010f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.077+0000 7fdcd6a00640 1 --2- 172.21.15.63:0/2433066604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd0103180 0x7fdcd010f4b0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009ca0 tx=0x7fdcc802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.078+0000 7fdcd6a00640 1 -- 172.21.15.63:0/2433066604 shutdown_connections 2024-05-27T07:49:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.078+0000 7fdcd6a00640 1 --2- 172.21.15.63:0/2433066604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd0103180 0x7fdcd010f4b0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.078+0000 7fdcd6a00640 1 --2- 172.21.15.63:0/2433066604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd0108ec0 0x7fdcd0102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.078+0000 7fdcd6a00640 1 --2- 172.21.15.63:0/2433066604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd0108a20 0x7fdcd01025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.078+0000 7fdcd6a00640 1 -- 172.21.15.63:0/2433066604 >> 172.21.15.63:0/2433066604 conn(0x7fdcd0078160 msgr2=0x7fdcd00785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.078+0000 7fdcd6a00640 1 -- 172.21.15.63:0/2433066604 shutdown_connections 2024-05-27T07:49:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.078+0000 7fdcd6a00640 1 -- 172.21.15.63:0/2433066604 wait complete. 2024-05-27T07:49:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.079+0000 7fdcd6a00640 1 Processor -- start 2024-05-27T07:49:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.079+0000 7fdcd6a00640 1 -- start start 2024-05-27T07:49:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd0103180 0x7fdcd019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd0108a20 0x7fdcd019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd0108ec0 0x7fdcd01a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd6a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0113bc0 con 0x7fdcd0108a20 2024-05-27T07:49:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0113a40 con 0x7fdcd0103180 2024-05-27T07:49:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd6a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0113d40 con 0x7fdcd0108ec0 2024-05-27T07:49:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd0108a20 0x7fdcd019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd0108a20 0x7fdcd019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49702/0 (socket says 172.21.15.63:49702) 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd4c00640 1 -- 172.21.15.63:0/3706953365 learned_addr learned my addr 172.21.15.63:0/3706953365 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd6000640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd0108ec0 0x7fdcd01a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.080+0000 7fdcd5600640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd0103180 0x7fdcd019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcd4c00640 1 -- 172.21.15.63:0/3706953365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd0108ec0 msgr2=0x7fdcd01a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd0108ec0 0x7fdcd01a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcd4c00640 1 -- 172.21.15.63:0/3706953365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd0103180 msgr2=0x7fdcd019c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd0103180 0x7fdcd019c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcd4c00640 1 -- 172.21.15.63:0/3706953365 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd01a15c0 con 0x7fdcd0108a20 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd0108a20 0x7fdcd019ccc0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00d740 tx=0x7fdcbc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcce000640 1 -- 172.21.15.63:0/3706953365 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc017070 con 0x7fdcd0108a20 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcce000640 1 -- 172.21.15.63:0/3706953365 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcbc00de50 con 0x7fdcd0108a20 2024-05-27T07:49:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcd6a00640 1 -- 172.21.15.63:0/3706953365 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd01a18b0 con 0x7fdcd0108a20 2024-05-27T07:49:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcce000640 1 -- 172.21.15.63:0/3706953365 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc003bb0 con 0x7fdcd0108a20 2024-05-27T07:49:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.081+0000 7fdcd6a00640 1 -- 172.21.15.63:0/3706953365 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd01a9230 con 0x7fdcd0108a20 2024-05-27T07:49:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.085+0000 7fdcdf01b740 1 Processor -- start 2024-05-27T07:49:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.085+0000 7fdcdf01b740 1 -- start start 2024-05-27T07:49:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcce000640 1 -- 172.21.15.63:0/3706953365 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdcbc003d50 con 0x7fdcd0108a20 2024-05-27T07:49:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f45a41b0 0x55f8f4946a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f457b2f0 0x55f8f4946fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f49474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f8f48b7b90 con 0x55f8f45834d0 2024-05-27T07:49:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8f48a7730 con 0x55f8f45a41b0 2024-05-27T07:49:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f8f48ab770 con 0x55f8f457b2f0 2024-05-27T07:49:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcd4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f49474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcd4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f49474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49718/0 (socket says 172.21.15.63:49718) 2024-05-27T07:49:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcd4c00640 1 -- 172.21.15.63:0/2351207418 learned_addr learned my addr 172.21.15.63:0/2351207418 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcd6000640 1 --2- 172.21.15.63:0/2351207418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f45a41b0 0x55f8f4946a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.086+0000 7fdcd5600640 1 --2- 172.21.15.63:0/2351207418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f457b2f0 0x55f8f4946fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.087+0000 7fdcd4c00640 1 -- 172.21.15.63:0/2351207418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f457b2f0 msgr2=0x55f8f4946fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.087+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/2351207418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f457b2f0 0x55f8f4946fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.087+0000 7fdcd4c00640 1 -- 172.21.15.63:0/2351207418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f45a41b0 msgr2=0x55f8f4946a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.087+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/2351207418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f45a41b0 0x55f8f4946a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.087+0000 7fdcd4c00640 1 -- 172.21.15.63:0/2351207418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8f48408b0 con 0x55f8f45834d0 2024-05-27T07:49:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.088+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/2351207418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f49474e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0df770 tx=0x7fdcbc0e8ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eabf71f1d2c57c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.088+0000 7fdcce000640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcb80b2950 0x7fdcb80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.088+0000 7fdcc2600640 1 -- 172.21.15.63:0/2351207418 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0f5070 con 0x55f8f45834d0 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.088+0000 7fdcce000640 1 -- 172.21.15.63:0/3706953365 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fdcbc0e2470 con 0x7fdcd0108a20 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.088+0000 7fdcc2600640 1 -- 172.21.15.63:0/2351207418 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcbc0a8ea0 con 0x55f8f45834d0 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.088+0000 7fdcc2600640 1 -- 172.21.15.63:0/2351207418 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0a9160 con 0x55f8f45834d0 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.088+0000 7fdcd6000640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcb80b2950 0x7fdcb80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.089+0000 7fdcd6000640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcb80b2950 0x7fdcb80b4e60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8005e30 tx=0x7fdcc8005c50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.089+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2351207418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 msgr2=0x55f8f49474e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.089+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/2351207418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f49474e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0df770 tx=0x7fdcbc0e8ae0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.089+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2351207418 shutdown_connections 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.089+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/2351207418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f49474e0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.089+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/2351207418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f457b2f0 0x55f8f4946fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.089+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/2351207418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f45a41b0 0x55f8f4946a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.089+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2351207418 >> 172.21.15.63:0/2351207418 conn(0x55f8f4598300 msgr2=0x55f8f4943210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.089+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2351207418 shutdown_connections 2024-05-27T07:49:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.090+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2351207418 wait complete. 2024-05-27T07:49:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.090+0000 7fdcdf01b740 1 -- start start 2024-05-27T07:49:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.091+0000 7fdcdf01b740 1 Processor -- start 2024-05-27T07:49:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.091+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f457b2f0 0x55f8f4950c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.091+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45834d0 0x55f8f49511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.091+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45a41b0 0x55f8f495cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.091+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f8f458d790 con 0x55f8f45a41b0 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.091+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8f484fbc0 con 0x55f8f457b2f0 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.091+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f8f4953b90 con 0x55f8f45834d0 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcdf01b740 10 client.0 _ll_register_callbacks cb 0x55f8f482d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcd6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45a41b0 0x55f8f495cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcd6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45a41b0 0x55f8f495cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49720/0 (socket says 172.21.15.63:49720) 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcd6000640 1 -- 172.21.15.63:0/2303300091 learned_addr learned my addr 172.21.15.63:0/2303300091 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcd5600640 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f457b2f0 0x55f8f4950c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45834d0 0x55f8f49511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcd6000640 1 -- 172.21.15.63:0/2303300091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45834d0 msgr2=0x55f8f49511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcd6000640 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45834d0 0x55f8f49511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcd6000640 1 -- 172.21.15.63:0/2303300091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f457b2f0 msgr2=0x55f8f4950c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcd6000640 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f457b2f0 0x55f8f4950c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.092+0000 7fdcd6000640 1 -- 172.21.15.63:0/2303300091 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8f45bc210 con 0x55f8f45a41b0 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.093+0000 7fdcd6000640 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45a41b0 0x55f8f495cec0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009ca0 tx=0x7fdcc802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b76806293f52a09 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.093+0000 7fdcb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.093+0000 7fdcb6a00640 1 -- 172.21.15.63:0/2303300091 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8004040 con 0x55f8f45a41b0 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.093+0000 7fdcb6a00640 1 -- 172.21.15.63:0/2303300091 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fdcc80041e0 con 0x55f8f45a41b0 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.093+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2303300091 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcbc0a8ea0 con 0x55f8f45a41b0 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.093+0000 7fdcb6a00640 1 -- 172.21.15.63:0/2303300091 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdcc80043e0 con 0x55f8f45a41b0 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.094+0000 7fdcdf01b740 10 client.18450 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.094+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2303300091 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f8f4688620 con 0x55f8f45a41b0 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.094+0000 7fdcb6a00640 1 -- 172.21.15.63:0/2303300091 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcbc0a8ea0 con 0x55f8f45a41b0 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.094+0000 7fdcb6a00640 1 -- 172.21.15.63:0/2303300091 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdcc8051070 con 0x55f8f45a41b0 2024-05-27T07:49:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.094+0000 7fdcdf01b740 10 client.18450 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.094+0000 7fdcdf01b740 10 client.18450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.094+0000 7fdcdf01b740 20 client.18450 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.094+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f8f4562d80 0x55f8f4962f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.094+0000 7fdcdf01b740 4 client.18450 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.094+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2303300091 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8f48b9ae0 con 0x55f8f4562d80 2024-05-27T07:49:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.095+0000 7fdcd5600640 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f8f4562d80 0x55f8f4962f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.096+0000 7fdcd5600640 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f8f4562d80 0x55f8f4962f30 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa648c24b50423 server_cookie=319430e0a9d9b88c in_seq=0 out_seq=0 2024-05-27T07:49:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.096+0000 7fdcb6a00640 10 client.18450 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.098+0000 7fdcb6a00640 1 -- 172.21.15.63:0/2303300091 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f8f48b9ae0 con 0x55f8f4562d80 2024-05-27T07:49:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.098+0000 7fdcb6a00640 10 client.18450 handle_command_reply: tid=0 2024-05-27T07:49:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.098+0000 7fdcdf01b740 1 client.18450 shutdown 2024-05-27T07:49:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2303300091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45a41b0 msgr2=0x55f8f495cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45a41b0 0x55f8f495cec0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009ca0 tx=0x7fdcc802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2303300091 shutdown_connections 2024-05-27T07:49:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f8f4562d80 0x55f8f4962f30 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45a41b0 0x55f8f495cec0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45834d0 0x55f8f49511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/2303300091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f457b2f0 0x55f8f4950c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcb6a00640 0 client.18450 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2303300091 >> 172.21.15.63:0/2303300091 conn(0x55f8f4598300 msgr2=0x55f8f4948780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2303300091 shutdown_connections 2024-05-27T07:49:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 1 -- 172.21.15.63:0/2303300091 wait complete. 2024-05-27T07:49:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.099+0000 7fdcdf01b740 20 client.18450 trim_cache size 0 max 16384 2024-05-27T07:49:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.117+0000 7fdcdf01b740 1 Processor -- start 2024-05-27T07:49:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.117+0000 7fdcdf01b740 1 -- start start 2024-05-27T07:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.118+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f457b2f0 0x55f8f494a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.118+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f494ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.118+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45a41b0 0x55f8f495cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.118+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b5b10 con 0x55f8f45834d0 2024-05-27T07:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.118+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b33e0 con 0x55f8f457b2f0 2024-05-27T07:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.118+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b1250 con 0x55f8f45a41b0 2024-05-27T07:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.118+0000 7fdcd4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f494ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.118+0000 7fdcd4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f494ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49732/0 (socket says 172.21.15.63:49732) 2024-05-27T07:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.118+0000 7fdcd4c00640 1 -- 172.21.15.63:0/3175212762 learned_addr learned my addr 172.21.15.63:0/3175212762 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.118+0000 7fdcd5600640 1 --2- 172.21.15.63:0/3175212762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f457b2f0 0x55f8f494a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.119+0000 7fdcd6000640 1 --2- 172.21.15.63:0/3175212762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45a41b0 0x55f8f495cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.119+0000 7fdcd4c00640 1 -- 172.21.15.63:0/3175212762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45a41b0 msgr2=0x55f8f495cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.119+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/3175212762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45a41b0 0x55f8f495cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.119+0000 7fdcd4c00640 1 -- 172.21.15.63:0/3175212762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f457b2f0 msgr2=0x55f8f494a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.119+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/3175212762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f457b2f0 0x55f8f494a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.119+0000 7fdcd4c00640 1 -- 172.21.15.63:0/3175212762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8f49534d0 con 0x55f8f45834d0 2024-05-27T07:49:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.119+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/3175212762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f494ad20 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0dfe00 tx=0x7fdcbc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80571d582d34d2a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.119+0000 7fdcc1c00640 1 -- 172.21.15.63:0/3175212762 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc020040 con 0x55f8f45834d0 2024-05-27T07:49:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.119+0000 7fdcc1c00640 1 -- 172.21.15.63:0/3175212762 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcbc0a8ea0 con 0x55f8f45834d0 2024-05-27T07:49:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.119+0000 7fdcc1c00640 1 -- 172.21.15.63:0/3175212762 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0a8b00 con 0x55f8f45834d0 2024-05-27T07:49:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.120+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3175212762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 msgr2=0x55f8f494ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.120+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/3175212762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f494ad20 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0dfe00 tx=0x7fdcbc009c30 comp rx=0 tx=0).stop 2024-05-27T07:49:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.120+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3175212762 shutdown_connections 2024-05-27T07:49:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.120+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/3175212762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45a41b0 0x55f8f495cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.120+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/3175212762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f45834d0 0x55f8f494ad20 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.120+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/3175212762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f457b2f0 0x55f8f494a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.120+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3175212762 >> 172.21.15.63:0/3175212762 conn(0x55f8f4598300 msgr2=0x55f8f4952dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.121+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3175212762 shutdown_connections 2024-05-27T07:49:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.121+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3175212762 wait complete. 2024-05-27T07:49:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.121+0000 7fdcdf01b740 1 -- start start 2024-05-27T07:49:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.121+0000 7fdcdf01b740 1 Processor -- start 2024-05-27T07:49:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.122+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f457b2f0 0x55f8f49595a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.122+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f45834d0 0x55f8f4954940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.122+0000 7fdcdf01b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45a41b0 0x55f8f4954ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.122+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b18a0 con 0x55f8f457b2f0 2024-05-27T07:49:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.122+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b2d10 con 0x55f8f45834d0 2024-05-27T07:49:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.122+0000 7fdcdf01b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd01b1f70 con 0x55f8f45a41b0 2024-05-27T07:49:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.122+0000 7fdcdf01b740 10 client.0 _ll_register_callbacks cb 0x7fdcc8004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.122+0000 7fdcd5600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f457b2f0 0x55f8f49595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.122+0000 7fdcd5600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f457b2f0 0x55f8f49595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49740/0 (socket says 172.21.15.63:49740) 2024-05-27T07:49:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.122+0000 7fdcd5600640 1 -- 172.21.15.63:0/3038893386 learned_addr learned my addr 172.21.15.63:0/3038893386 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.123+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f45834d0 0x55f8f4954940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.123+0000 7fdcd6000640 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45a41b0 0x55f8f4954ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.123+0000 7fdcd5600640 1 -- 172.21.15.63:0/3038893386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45a41b0 msgr2=0x55f8f4954ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.123+0000 7fdcd5600640 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45a41b0 0x55f8f4954ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.123+0000 7fdcd5600640 1 -- 172.21.15.63:0/3038893386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f45834d0 msgr2=0x55f8f4954940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.123+0000 7fdcd5600640 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f45834d0 0x55f8f4954940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.123+0000 7fdcd5600640 1 -- 172.21.15.63:0/3038893386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8f4955630 con 0x55f8f457b2f0 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.123+0000 7fdcd5600640 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f457b2f0 0x55f8f49595a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x55f8f4944550 tx=0x7fdcc40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430734188fbee3ef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.123+0000 7fdcb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.123+0000 7fdcb6a00640 1 -- 172.21.15.63:0/3038893386 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc401d940 con 0x55f8f457b2f0 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.124+0000 7fdcb6a00640 1 -- 172.21.15.63:0/3038893386 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fdcc401dae0 con 0x55f8f457b2f0 2024-05-27T07:49:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.124+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3038893386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcbc0a8ea0 con 0x55f8f457b2f0 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.124+0000 7fdcb6a00640 1 -- 172.21.15.63:0/3038893386 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdcc402d050 con 0x55f8f457b2f0 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.124+0000 7fdcdf01b740 10 client.18456 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.124+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3038893386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f8f49cb590 con 0x55f8f457b2f0 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.124+0000 7fdcb6a00640 1 -- 172.21.15.63:0/3038893386 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcbc0a8ea0 con 0x55f8f457b2f0 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.125+0000 7fdcb6a00640 1 -- 172.21.15.63:0/3038893386 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdcc8031630 con 0x55f8f457b2f0 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.125+0000 7fdcdf01b740 10 client.18456 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.125+0000 7fdcdf01b740 10 client.18456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.125+0000 7fdcdf01b740 20 client.18456 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.125+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f8f45381c0 0x55f8f49cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.125+0000 7fdcdf01b740 4 client.18456 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:49:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.125+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3038893386 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8f48b32c0 con 0x55f8f45381c0 2024-05-27T07:49:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.125+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f8f45381c0 0x55f8f49cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.126+0000 7fdcb6a00640 10 client.18456 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.126+0000 7fdcd4c00640 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f8f45381c0 0x55f8f49cdcd0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bef35c69baa5c5b3 server_cookie=77c580c62ffe8ffb in_seq=0 out_seq=0 2024-05-27T07:49:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.159+0000 7fdcb6a00640 1 -- 172.21.15.63:0/3038893386 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f8f48b32c0 con 0x55f8f45381c0 2024-05-27T07:49:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcb6a00640 10 client.18456 handle_command_reply: tid=0 2024-05-27T07:49:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 client.18456 shutdown 2024-05-27T07:49:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3038893386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f457b2f0 msgr2=0x55f8f49595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f457b2f0 0x55f8f49595a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x55f8f4944550 tx=0x7fdcc40118d0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3038893386 shutdown_connections 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f8f45381c0 0x55f8f49cdcd0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f8f45a41b0 0x55f8f4954ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8f45834d0 0x55f8f4954940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 --2- 172.21.15.63:0/3038893386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8f457b2f0 0x55f8f49595a0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3038893386 >> 172.21.15.63:0/3038893386 conn(0x55f8f4598300 msgr2=0x55f8f4952ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcb6a00640 0 client.18456 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3038893386 shutdown_connections 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.160+0000 7fdcdf01b740 1 -- 172.21.15.63:0/3038893386 wait complete. 2024-05-27T07:49:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.161+0000 7fdcdf01b740 20 client.18456 trim_cache size 0 max 16384 2024-05-27T07:49:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:49:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:49:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:49:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.161+0000 7fdcd6a00640 1 -- 172.21.15.63:0/3706953365 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcb80b2950 msgr2=0x7fdcb80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.161+0000 7fdcd6a00640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcb80b2950 0x7fdcb80b4e60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8005e30 tx=0x7fdcc8005c50 comp rx=0 tx=0).stop 2024-05-27T07:49:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.161+0000 7fdcd6a00640 1 -- 172.21.15.63:0/3706953365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd0108a20 msgr2=0x7fdcd019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.161+0000 7fdcd6a00640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd0108a20 0x7fdcd019ccc0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00d740 tx=0x7fdcbc00dc00 comp rx=0 tx=0).stop 2024-05-27T07:49:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.162+0000 7fdcd6a00640 1 -- 172.21.15.63:0/3706953365 shutdown_connections 2024-05-27T07:49:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.162+0000 7fdcd6a00640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdcb80b2950 0x7fdcb80b4e60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.162+0000 7fdcd6a00640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcd0108ec0 0x7fdcd01a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.162+0000 7fdcd6a00640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcd0108a20 0x7fdcd019ccc0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.162+0000 7fdcd6a00640 1 --2- 172.21.15.63:0/3706953365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdcd0103180 0x7fdcd019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.162+0000 7fdcd6a00640 1 -- 172.21.15.63:0/3706953365 >> 172.21.15.63:0/3706953365 conn(0x7fdcd0078160 msgr2=0x7fdcd0110230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.162+0000 7fdcd6a00640 1 -- 172.21.15.63:0/3706953365 shutdown_connections 2024-05-27T07:49:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:19.162+0000 7fdcd6a00640 1 -- 172.21.15.63:0/3706953365 wait complete. 2024-05-27T07:49:19.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:18 smithi086 ceph-mon[34687]: pgmap v495: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:49:19.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:18 smithi149 ceph-mon[35408]: pgmap v495: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:49:19.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:18 smithi063 ceph-mon[31613]: pgmap v495: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:49:21.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:20 smithi086 ceph-mon[34687]: pgmap v496: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:49:21.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:20 smithi149 ceph-mon[35408]: pgmap v496: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:49:21.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:20 smithi063 ceph-mon[31613]: pgmap v496: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:49:23.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:22 smithi086 ceph-mon[34687]: pgmap v497: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:49:23.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:22 smithi149 ceph-mon[35408]: pgmap v497: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:49:23.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:22 smithi063 ceph-mon[31613]: pgmap v497: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:49:25.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:24 smithi086 ceph-mon[34687]: pgmap v498: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:49:25.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:24 smithi149 ceph-mon[35408]: pgmap v498: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:49:25.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:24 smithi063 ceph-mon[31613]: pgmap v498: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:49:27.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:26 smithi086 ceph-mon[34687]: pgmap v499: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:49:27.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:26 smithi149 ceph-mon[35408]: pgmap v499: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:49:27.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:26 smithi063 ceph-mon[31613]: pgmap v499: 129 pgs: 129 active+clean; 4.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:49:28.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:27 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:49:28.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:27 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:49:28.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:27 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:49:29.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:28 smithi086 ceph-mon[34687]: pgmap v500: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:49:29.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:28 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:49:29.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:28 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:49:29.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:28 smithi149 ceph-mon[35408]: pgmap v500: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:49:29.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:28 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:49:29.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:28 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:49:29.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:28 smithi063 ceph-mon[31613]: pgmap v500: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:49:29.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:28 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:49:29.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:28 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:49:31.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:30 smithi086 ceph-mon[34687]: pgmap v501: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T07:49:31.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:30 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:49:31.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:30 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:49:31.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:30 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:49:31.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:30 smithi149 ceph-mon[35408]: pgmap v501: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T07:49:31.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:30 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:49:31.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:30 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:49:31.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:30 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:49:31.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:30 smithi063 ceph-mon[31613]: pgmap v501: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T07:49:31.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:30 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:49:31.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:30 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:49:31.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:30 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:49:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.445+0000 7f62fa200640 1 -- 172.21.15.63:0/1967866476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f4109f70 msgr2=0x7f62f4103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.445+0000 7f62fa200640 1 --2- 172.21.15.63:0/1967866476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f4109f70 0x7f62f4103bb0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009900 tx=0x7f62e002a410 comp rx=0 tx=0).stop 2024-05-27T07:49:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.445+0000 7f62fa200640 1 -- 172.21.15.63:0/1967866476 shutdown_connections 2024-05-27T07:49:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.445+0000 7f62fa200640 1 --2- 172.21.15.63:0/1967866476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62f4104220 0x7f62f41105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.445+0000 7f62fa200640 1 --2- 172.21.15.63:0/1967866476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f4109f70 0x7f62f4103bb0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.445+0000 7f62fa200640 1 --2- 172.21.15.63:0/1967866476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f4109ad0 0x7f62f4103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.445+0000 7f62fa200640 1 -- 172.21.15.63:0/1967866476 >> 172.21.15.63:0/1967866476 conn(0x7f62f4079100 msgr2=0x7f62f4079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.445+0000 7f62fa200640 1 -- 172.21.15.63:0/1967866476 shutdown_connections 2024-05-27T07:49:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.446+0000 7f62fa200640 1 -- 172.21.15.63:0/1967866476 wait complete. 2024-05-27T07:49:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.447+0000 7f62fa200640 1 Processor -- start 2024-05-27T07:49:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.447+0000 7f62fa200640 1 -- start start 2024-05-27T07:49:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.447+0000 7f62fa200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f4104220 0x7f62f4074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.447+0000 7f62fa200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f4109ad0 0x7f62f4075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.447+0000 7f62fa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62f4109f70 0x7f62f4075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.447+0000 7f62fa200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4113b70 con 0x7f62f4109ad0 2024-05-27T07:49:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.447+0000 7f62fa200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41139f0 con 0x7f62f4109f70 2024-05-27T07:49:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.447+0000 7f62fa200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62f4113cf0 con 0x7f62f4104220 2024-05-27T07:49:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f4109ad0 0x7f62f4075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f4109ad0 0x7f62f4075500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40262/0 (socket says 172.21.15.63:40262) 2024-05-27T07:49:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f3e00640 1 -- 172.21.15.63:0/742057324 learned_addr learned my addr 172.21.15.63:0/742057324 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f9800640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62f4109f70 0x7f62f4075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f8e00640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f4104220 0x7f62f4074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f3e00640 1 -- 172.21.15.63:0/742057324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f4104220 msgr2=0x7f62f4074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f3e00640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f4104220 0x7f62f4074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f3e00640 1 -- 172.21.15.63:0/742057324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62f4109f70 msgr2=0x7f62f4075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f3e00640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62f4109f70 0x7f62f4075a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f3e00640 1 -- 172.21.15.63:0/742057324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f41a8970 con 0x7f62f4109ad0 2024-05-27T07:49:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f8e00640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f4104220 0x7f62f4074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.448+0000 7f62f9800640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62f4109f70 0x7f62f4075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.449+0000 7f62f3e00640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f4109ad0 0x7f62f4075500 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f62e0005940 tx=0x7f62e00040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.449+0000 7f62f1600640 1 -- 172.21.15.63:0/742057324 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e003f030 con 0x7f62f4109ad0 2024-05-27T07:49:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.449+0000 7f62fa200640 1 -- 172.21.15.63:0/742057324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f41a8c00 con 0x7f62f4109ad0 2024-05-27T07:49:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.449+0000 7f62f1600640 1 -- 172.21.15.63:0/742057324 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62e0003a80 con 0x7f62f4109ad0 2024-05-27T07:49:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.449+0000 7f62f1600640 1 -- 172.21.15.63:0/742057324 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0035070 con 0x7f62f4109ad0 2024-05-27T07:49:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.449+0000 7f62fa200640 1 -- 172.21.15.63:0/742057324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f41a9140 con 0x7f62f4109ad0 2024-05-27T07:49:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.454+0000 7f6302799740 1 Processor -- start 2024-05-27T07:49:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.454+0000 7f6302799740 1 -- start start 2024-05-27T07:49:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.454+0000 7f62f1600640 1 -- 172.21.15.63:0/742057324 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f62e0042030 con 0x7f62f4109ad0 2024-05-27T07:49:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.454+0000 7f6302799740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b652ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.454+0000 7f6302799740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61602f0 0x55b8b652bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.454+0000 7f6302799740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61684d0 0x55b8b652c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.454+0000 7f6302799740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b8b649cb90 con 0x55b8b61891b0 2024-05-27T07:49:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.454+0000 7f6302799740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8b648c730 con 0x55b8b61602f0 2024-05-27T07:49:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.455+0000 7f6302799740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b8b6490770 con 0x55b8b61684d0 2024-05-27T07:49:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.455+0000 7f62f9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b652ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.455+0000 7f62f9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b652ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40270/0 (socket says 172.21.15.63:40270) 2024-05-27T07:49:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.455+0000 7f62f9800640 1 -- 172.21.15.63:0/669207868 learned_addr learned my addr 172.21.15.63:0/669207868 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.455+0000 7f62f8e00640 1 --2- 172.21.15.63:0/669207868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61602f0 0x55b8b652bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.455+0000 7f62f3e00640 1 --2- 172.21.15.63:0/669207868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61684d0 0x55b8b652c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.455+0000 7f62f9800640 1 -- 172.21.15.63:0/669207868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61684d0 msgr2=0x55b8b652c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.455+0000 7f62f9800640 1 --2- 172.21.15.63:0/669207868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61684d0 0x55b8b652c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.455+0000 7f62f9800640 1 -- 172.21.15.63:0/669207868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61602f0 msgr2=0x55b8b652bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.455+0000 7f62f9800640 1 --2- 172.21.15.63:0/669207868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61602f0 0x55b8b652bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.456+0000 7f62f9800640 1 -- 172.21.15.63:0/669207868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8b64258b0 con 0x55b8b61891b0 2024-05-27T07:49:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.456+0000 7f62f9800640 1 --2- 172.21.15.63:0/669207868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b652ba60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009fb0 tx=0x7f62ec010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5377d1a4743c36 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.456+0000 7f62e6000640 1 -- 172.21.15.63:0/669207868 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec016070 con 0x55b8b61891b0 2024-05-27T07:49:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.456+0000 7f62f1600640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62dc0b29d0 0x7f62dc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.456+0000 7f62e6000640 1 -- 172.21.15.63:0/669207868 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62ec012ba0 con 0x55b8b61891b0 2024-05-27T07:49:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.457+0000 7f62f1600640 1 -- 172.21.15.63:0/742057324 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f62e0102fa0 con 0x7f62f4109ad0 2024-05-27T07:49:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.457+0000 7f62e6000640 1 -- 172.21.15.63:0/669207868 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec01bb40 con 0x55b8b61891b0 2024-05-27T07:49:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.457+0000 7f62f8e00640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62dc0b29d0 0x7f62dc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.458+0000 7f6302799740 1 -- 172.21.15.63:0/669207868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 msgr2=0x55b8b652ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.458+0000 7f6302799740 1 --2- 172.21.15.63:0/669207868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b652ba60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009fb0 tx=0x7f62ec010800 comp rx=0 tx=0).stop 2024-05-27T07:49:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.458+0000 7f6302799740 1 -- 172.21.15.63:0/669207868 shutdown_connections 2024-05-27T07:49:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.458+0000 7f6302799740 1 --2- 172.21.15.63:0/669207868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61684d0 0x55b8b652c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.458+0000 7f6302799740 1 --2- 172.21.15.63:0/669207868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61602f0 0x55b8b652bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.458+0000 7f6302799740 1 --2- 172.21.15.63:0/669207868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b652ba60 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.458+0000 7f6302799740 1 -- 172.21.15.63:0/669207868 >> 172.21.15.63:0/669207868 conn(0x55b8b617d300 msgr2=0x55b8b6528210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.458+0000 7f62f8e00640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62dc0b29d0 0x7f62dc0b4ee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005d80 tx=0x7f62e8005cf0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.458+0000 7f6302799740 1 -- 172.21.15.63:0/669207868 shutdown_connections 2024-05-27T07:49:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.458+0000 7f6302799740 1 -- 172.21.15.63:0/669207868 wait complete. 2024-05-27T07:49:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.459+0000 7f6302799740 1 -- start start 2024-05-27T07:49:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.459+0000 7f6302799740 1 Processor -- start 2024-05-27T07:49:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.460+0000 7f6302799740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61602f0 0x55b8b6535b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.460+0000 7f6302799740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61684d0 0x55b8b65360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.460+0000 7f6302799740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61891b0 0x55b8b6541f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.460+0000 7f6302799740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b8b6172790 con 0x55b8b61602f0 2024-05-27T07:49:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.460+0000 7f6302799740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8b6434bc0 con 0x55b8b61684d0 2024-05-27T07:49:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.460+0000 7f6302799740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b8b6538be0 con 0x55b8b61891b0 2024-05-27T07:49:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.460+0000 7f6302799740 10 client.0 _ll_register_callbacks cb 0x55b8b6412400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.461+0000 7f62f8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61602f0 0x55b8b6535b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.461+0000 7f62f8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61602f0 0x55b8b6535b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40276/0 (socket says 172.21.15.63:40276) 2024-05-27T07:49:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.461+0000 7f62f8e00640 1 -- 172.21.15.63:0/4209037775 learned_addr learned my addr 172.21.15.63:0/4209037775 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.461+0000 7f62f3e00640 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61684d0 0x55b8b65360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.461+0000 7f62f9800640 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61891b0 0x55b8b6541f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.461+0000 7f62f8e00640 1 -- 172.21.15.63:0/4209037775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61891b0 msgr2=0x55b8b6541f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.461+0000 7f62f8e00640 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61891b0 0x55b8b6541f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.461+0000 7f62f8e00640 1 -- 172.21.15.63:0/4209037775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61684d0 msgr2=0x55b8b65360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.461+0000 7f62f8e00640 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61684d0 0x55b8b65360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.461+0000 7f62f8e00640 1 -- 172.21.15.63:0/4209037775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8b61a1210 con 0x55b8b61602f0 2024-05-27T07:49:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.462+0000 7f62f8e00640 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61602f0 0x55b8b6535b70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f62f41047b0 tx=0x7f62e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46756c1a74441e08 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.462+0000 7f62daa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.462+0000 7f62daa00640 1 -- 172.21.15.63:0/4209037775 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e801f9b0 con 0x55b8b61602f0 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.462+0000 7f62daa00640 1 -- 172.21.15.63:0/4209037775 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f62e801fb50 con 0x55b8b61602f0 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.462+0000 7f6302799740 1 -- 172.21.15.63:0/4209037775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ec012ba0 con 0x55b8b61602f0 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f62daa00640 1 -- 172.21.15.63:0/4209037775 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f62e801fd50 con 0x55b8b61602f0 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f6302799740 10 client.18468 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f6302799740 1 -- 172.21.15.63:0/4209037775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b8b626d620 con 0x55b8b61602f0 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f62daa00640 1 -- 172.21.15.63:0/4209037775 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62ec012ba0 con 0x55b8b61602f0 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f62daa00640 1 -- 172.21.15.63:0/4209037775 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f62e801d3b0 con 0x55b8b61602f0 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f6302799740 10 client.18468 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f6302799740 10 client.18468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f6302799740 20 client.18468 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f6302799740 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8b6147d80 0x55b8b6547ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f6302799740 4 client.18468 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.463+0000 7f6302799740 1 -- 172.21.15.63:0/4209037775 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8b649eae0 con 0x55b8b6147d80 2024-05-27T07:49:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.464+0000 7f62f3e00640 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8b6147d80 0x55b8b6547ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.465+0000 7f62f3e00640 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8b6147d80 0x55b8b6547ec0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ead5fb3f64aa22 server_cookie=3203512e27813b4 in_seq=0 out_seq=0 2024-05-27T07:49:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.465+0000 7f62daa00640 10 client.18468 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.467+0000 7f62daa00640 1 -- 172.21.15.63:0/4209037775 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b8b649eae0 con 0x55b8b6147d80 2024-05-27T07:49:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.467+0000 7f62daa00640 10 client.18468 handle_command_reply: tid=0 2024-05-27T07:49:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.467+0000 7f6302799740 1 client.18468 shutdown 2024-05-27T07:49:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.467+0000 7f6302799740 1 -- 172.21.15.63:0/4209037775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61602f0 msgr2=0x55b8b6535b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.467+0000 7f6302799740 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61602f0 0x55b8b6535b70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f62f41047b0 tx=0x7f62e80046c0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.468+0000 7f6302799740 1 -- 172.21.15.63:0/4209037775 shutdown_connections 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.468+0000 7f6302799740 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8b6147d80 0x55b8b6547ec0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.468+0000 7f6302799740 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61891b0 0x55b8b6541f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.468+0000 7f6302799740 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61684d0 0x55b8b65360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.468+0000 7f6302799740 1 --2- 172.21.15.63:0/4209037775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61602f0 0x55b8b6535b70 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.468+0000 7f6302799740 1 -- 172.21.15.63:0/4209037775 >> 172.21.15.63:0/4209037775 conn(0x55b8b617d300 msgr2=0x55b8b652b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.468+0000 7f62daa00640 0 client.18468 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.468+0000 7f6302799740 1 -- 172.21.15.63:0/4209037775 shutdown_connections 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.468+0000 7f6302799740 1 -- 172.21.15.63:0/4209037775 wait complete. 2024-05-27T07:49:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.468+0000 7f6302799740 20 client.18468 trim_cache size 0 max 16384 2024-05-27T07:49:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.501+0000 7f6302799740 1 Processor -- start 2024-05-27T07:49:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.502+0000 7f6302799740 1 -- start start 2024-05-27T07:49:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.502+0000 7f6302799740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61602f0 0x55b8b652c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.502+0000 7f6302799740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61684d0 0x55b8b6541d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.502+0000 7f6302799740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b6542240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.502+0000 7f6302799740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41b5c00 con 0x55b8b61891b0 2024-05-27T07:49:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.502+0000 7f6302799740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41b34d0 con 0x55b8b61684d0 2024-05-27T07:49:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.502+0000 7f6302799740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41b1340 con 0x55b8b61602f0 2024-05-27T07:49:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.503+0000 7f62f9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b6542240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.503+0000 7f62f9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b6542240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40284/0 (socket says 172.21.15.63:40284) 2024-05-27T07:49:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.503+0000 7f62f9800640 1 -- 172.21.15.63:0/91647956 learned_addr learned my addr 172.21.15.63:0/91647956 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.503+0000 7f62f3e00640 1 --2- 172.21.15.63:0/91647956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61684d0 0x55b8b6541d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.503+0000 7f62f8e00640 1 --2- 172.21.15.63:0/91647956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61602f0 0x55b8b652c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.503+0000 7f62f9800640 1 -- 172.21.15.63:0/91647956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61602f0 msgr2=0x55b8b652c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.503+0000 7f62f9800640 1 --2- 172.21.15.63:0/91647956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61602f0 0x55b8b652c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.503+0000 7f62f9800640 1 -- 172.21.15.63:0/91647956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61684d0 msgr2=0x55b8b6541d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.503+0000 7f62f9800640 1 --2- 172.21.15.63:0/91647956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61684d0 0x55b8b6541d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.503+0000 7f62f9800640 1 -- 172.21.15.63:0/91647956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8b652cd90 con 0x55b8b61891b0 2024-05-27T07:49:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.504+0000 7f62f9800640 1 --2- 172.21.15.63:0/91647956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b6542240 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f62ec01a000 tx=0x7f62ec010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faea51146f4702e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.504+0000 7f62e5600640 1 -- 172.21.15.63:0/91647956 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec016040 con 0x55b8b61891b0 2024-05-27T07:49:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.504+0000 7f62e5600640 1 -- 172.21.15.63:0/91647956 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62ec02a070 con 0x55b8b61891b0 2024-05-27T07:49:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.504+0000 7f62e5600640 1 -- 172.21.15.63:0/91647956 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0052a0 con 0x55b8b61891b0 2024-05-27T07:49:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.506+0000 7f6302799740 1 -- 172.21.15.63:0/91647956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 msgr2=0x55b8b6542240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.506+0000 7f6302799740 1 --2- 172.21.15.63:0/91647956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b6542240 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f62ec01a000 tx=0x7f62ec010f70 comp rx=0 tx=0).stop 2024-05-27T07:49:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.506+0000 7f6302799740 1 -- 172.21.15.63:0/91647956 shutdown_connections 2024-05-27T07:49:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.506+0000 7f6302799740 1 --2- 172.21.15.63:0/91647956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b6542240 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.506+0000 7f6302799740 1 --2- 172.21.15.63:0/91647956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61684d0 0x55b8b6541d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.506+0000 7f6302799740 1 --2- 172.21.15.63:0/91647956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61602f0 0x55b8b652c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.506+0000 7f6302799740 1 -- 172.21.15.63:0/91647956 >> 172.21.15.63:0/91647956 conn(0x55b8b617d300 msgr2=0x55b8b6535e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.506+0000 7f6302799740 1 -- 172.21.15.63:0/91647956 shutdown_connections 2024-05-27T07:49:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.506+0000 7f6302799740 1 -- 172.21.15.63:0/91647956 wait complete. 2024-05-27T07:49:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.506+0000 7f6302799740 1 -- start start 2024-05-27T07:49:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.507+0000 7f6302799740 1 Processor -- start 2024-05-27T07:49:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.507+0000 7f6302799740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61602f0 0x55b8b653ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.507+0000 7f6302799740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61684d0 0x55b8b653b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.507+0000 7f6302799740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b653b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.507+0000 7f6302799740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41b1990 con 0x55b8b61891b0 2024-05-27T07:49:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.507+0000 7f6302799740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41b2e00 con 0x55b8b61602f0 2024-05-27T07:49:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.507+0000 7f6302799740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62f41b2060 con 0x55b8b61684d0 2024-05-27T07:49:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.507+0000 7f6302799740 10 client.0 _ll_register_callbacks cb 0x7f62e800b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.508+0000 7f62f9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b653b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.508+0000 7f62f9800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b653b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40300/0 (socket says 172.21.15.63:40300) 2024-05-27T07:49:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.508+0000 7f62f9800640 1 -- 172.21.15.63:0/1740934042 learned_addr learned my addr 172.21.15.63:0/1740934042 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.508+0000 7f62f8e00640 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61602f0 0x55b8b653ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.508+0000 7f62f3e00640 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61684d0 0x55b8b653b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.508+0000 7f62f9800640 1 -- 172.21.15.63:0/1740934042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61684d0 msgr2=0x55b8b653b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.508+0000 7f62f9800640 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61684d0 0x55b8b653b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.508+0000 7f62f9800640 1 -- 172.21.15.63:0/1740934042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61602f0 msgr2=0x55b8b653ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.508+0000 7f62f9800640 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61602f0 0x55b8b653ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.508+0000 7f62f9800640 1 -- 172.21.15.63:0/1740934042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8b65ae590 con 0x55b8b61891b0 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.509+0000 7f62f9800640 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b653b700 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f62ec006010 tx=0x7f62ec004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6354955f00aa1d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.509+0000 7f62daa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.509+0000 7f62daa00640 1 -- 172.21.15.63:0/1740934042 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec032030 con 0x55b8b61891b0 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.509+0000 7f62daa00640 1 -- 172.21.15.63:0/1740934042 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f62ec012720 con 0x55b8b61891b0 2024-05-27T07:49:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.509+0000 7f6302799740 1 -- 172.21.15.63:0/1740934042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ec02a070 con 0x55b8b61891b0 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f62daa00640 1 -- 172.21.15.63:0/1740934042 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f62ec03f050 con 0x55b8b61891b0 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f6302799740 10 client.18474 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f6302799740 1 -- 172.21.15.63:0/1740934042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b8b65af7b0 con 0x55b8b61891b0 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f62daa00640 1 -- 172.21.15.63:0/1740934042 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62ec02a070 con 0x55b8b61891b0 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f62daa00640 1 -- 172.21.15.63:0/1740934042 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f62ec00f040 con 0x55b8b61891b0 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f6302799740 10 client.18474 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f6302799740 10 client.18474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f6302799740 20 client.18474 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f6302799740 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8b611d1c0 0x55b8b65b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f6302799740 4 client.18474 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.510+0000 7f6302799740 1 -- 172.21.15.63:0/1740934042 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8b64982c0 con 0x55b8b611d1c0 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.511+0000 7f62f8e00640 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8b611d1c0 0x55b8b65b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.512+0000 7f62daa00640 10 client.18474 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.512+0000 7f62f8e00640 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8b611d1c0 0x55b8b65b1c90 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217ec8870f43b1ae server_cookie=1ce7652cc3a9a54f in_seq=0 out_seq=0 2024-05-27T07:49:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f62daa00640 1 -- 172.21.15.63:0/1740934042 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b8b64982c0 con 0x55b8b611d1c0 2024-05-27T07:49:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f62daa00640 10 client.18474 handle_command_reply: tid=0 2024-05-27T07:49:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f6302799740 1 client.18474 shutdown 2024-05-27T07:49:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f6302799740 1 -- 172.21.15.63:0/1740934042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 msgr2=0x55b8b653b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f6302799740 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b653b700 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f62ec006010 tx=0x7f62ec004cf0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f6302799740 1 -- 172.21.15.63:0/1740934042 shutdown_connections 2024-05-27T07:49:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f6302799740 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8b611d1c0 0x55b8b65b1c90 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f6302799740 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8b61891b0 0x55b8b653b700 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f6302799740 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8b61684d0 0x55b8b653b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f6302799740 1 --2- 172.21.15.63:0/1740934042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8b61602f0 0x55b8b653ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f62daa00640 0 client.18474 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f6302799740 1 -- 172.21.15.63:0/1740934042 >> 172.21.15.63:0/1740934042 conn(0x55b8b617d300 msgr2=0x55b8b6526810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.545+0000 7f6302799740 1 -- 172.21.15.63:0/1740934042 shutdown_connections 2024-05-27T07:49:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.546+0000 7f6302799740 1 -- 172.21.15.63:0/1740934042 wait complete. 2024-05-27T07:49:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.546+0000 7f6302799740 20 client.18474 trim_cache size 0 max 16384 2024-05-27T07:49:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:49:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:49:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:49:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 -- 172.21.15.63:0/742057324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62dc0b29d0 msgr2=0x7f62dc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62dc0b29d0 0x7f62dc0b4ee0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005d80 tx=0x7f62e8005cf0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 -- 172.21.15.63:0/742057324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f4109ad0 msgr2=0x7f62f4075500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f4109ad0 0x7f62f4075500 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f62e0005940 tx=0x7f62e00040e0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 -- 172.21.15.63:0/742057324 shutdown_connections 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62dc0b29d0 0x7f62dc0b4ee0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62f4109f70 0x7f62f4075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f4109ad0 0x7f62f4075500 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 --2- 172.21.15.63:0/742057324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f4104220 0x7f62f4074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 -- 172.21.15.63:0/742057324 >> 172.21.15.63:0/742057324 conn(0x7f62f4079100 msgr2=0x7f62f41022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 -- 172.21.15.63:0/742057324 shutdown_connections 2024-05-27T07:49:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:32.547+0000 7f62fa200640 1 -- 172.21.15.63:0/742057324 wait complete. 2024-05-27T07:49:33.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:32 smithi086 ceph-mon[34687]: pgmap v502: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:49:33.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:32 smithi149 ceph-mon[35408]: pgmap v502: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:49:33.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:32 smithi063 ceph-mon[31613]: pgmap v502: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:49:33.373 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:49:33.867 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:35.117 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:34 smithi063 ceph-mon[31613]: pgmap v503: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:49:35.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:34 smithi086 ceph-mon[34687]: pgmap v503: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:49:35.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:34 smithi149 ceph-mon[35408]: pgmap v503: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:49:36.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.337+0000 7fb7a7200700 1 -- 172.21.15.63:0/2137494168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a8109670 msgr2=0x7fb7a8110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:36.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.337+0000 7fb7a7200700 1 --2- 172.21.15.63:0/2137494168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a8109670 0x7fb7a8110fd0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b0d0 tx=0x7fb7a000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.337+0000 7fb7a7200700 1 -- 172.21.15.63:0/2137494168 shutdown_connections 2024-05-27T07:49:36.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.337+0000 7fb7a7200700 1 --2- 172.21.15.63:0/2137494168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a8109670 0x7fb7a8110fd0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.337+0000 7fb7a7200700 1 --2- 172.21.15.63:0/2137494168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7a81043f0 0x7fb7a8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.337+0000 7fb7a7200700 1 --2- 172.21.15.63:0/2137494168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a8108d20 0x7fb7a8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.337+0000 7fb7a7200700 1 -- 172.21.15.63:0/2137494168 >> 172.21.15.63:0/2137494168 conn(0x7fb7a8079d00 msgr2=0x7fb7a807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:36.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.337+0000 7fb7a7200700 1 -- 172.21.15.63:0/2137494168 shutdown_connections 2024-05-27T07:49:36.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.337+0000 7fb7a7200700 1 -- 172.21.15.63:0/2137494168 wait complete. 2024-05-27T07:49:36.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.338+0000 7fb7a7200700 1 Processor -- start 2024-05-27T07:49:36.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.339+0000 7fb7a7200700 1 -- start start 2024-05-27T07:49:36.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.339+0000 7fb7a7200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a81043f0 0x7fb7a819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.339+0000 7fb7a7200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7a8108d20 0x7fb7a819d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.339+0000 7fb7a7200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a8109670 0x7fb7a81a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.339+0000 7fb7a7200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8114150 con 0x7fb7a8109670 2024-05-27T07:49:36.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.339+0000 7fb7a7200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8113fd0 con 0x7fb7a81043f0 2024-05-27T07:49:36.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.339+0000 7fb7a7200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a81142d0 con 0x7fb7a8108d20 2024-05-27T07:49:36.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.340+0000 7fb7a6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a8109670 0x7fb7a81a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.340+0000 7fb7a6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a8109670 0x7fb7a81a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40312/0 (socket says 172.21.15.63:40312) 2024-05-27T07:49:36.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.340+0000 7fb7a6800700 1 -- 172.21.15.63:0/2030641770 learned_addr learned my addr 172.21.15.63:0/2030641770 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:36.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.340+0000 7fb7a5400700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7a8108d20 0x7fb7a819d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.340+0000 7fb7a5e00700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a81043f0 0x7fb7a819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.341+0000 7fb7a6800700 1 -- 172.21.15.63:0/2030641770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7a8108d20 msgr2=0x7fb7a819d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.341+0000 7fb7a6800700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7a8108d20 0x7fb7a819d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.341+0000 7fb7a6800700 1 -- 172.21.15.63:0/2030641770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a81043f0 msgr2=0x7fb7a819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.341+0000 7fb7a6800700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a81043f0 0x7fb7a819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.341+0000 7fb7a6800700 1 -- 172.21.15.63:0/2030641770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a81a1c10 con 0x7fb7a8109670 2024-05-27T07:49:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.341+0000 7fb7a6800700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a8109670 0x7fb7a81a1570 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0009850 tx=0x7fb7a0007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:36.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.342+0000 7fb79ea00700 1 -- 172.21.15.63:0/2030641770 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a001f070 con 0x7fb7a8109670 2024-05-27T07:49:36.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.342+0000 7fb79ea00700 1 -- 172.21.15.63:0/2030641770 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7a0005500 con 0x7fb7a8109670 2024-05-27T07:49:36.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.342+0000 7fb79ea00700 1 -- 172.21.15.63:0/2030641770 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a001b9a0 con 0x7fb7a8109670 2024-05-27T07:49:36.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.342+0000 7fb7a7200700 1 -- 172.21.15.63:0/2030641770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a81a1ea0 con 0x7fb7a8109670 2024-05-27T07:49:36.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.342+0000 7fb7a7200700 1 -- 172.21.15.63:0/2030641770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a81a2380 con 0x7fb7a8109670 2024-05-27T07:49:36.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.346+0000 7fb79ea00700 1 -- 172.21.15.63:0/2030641770 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb7a0022070 con 0x7fb7a8109670 2024-05-27T07:49:36.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.347+0000 7fb7bb8eeb80 1 Processor -- start 2024-05-27T07:49:36.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.348+0000 7fb79ea00700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb78c0b29f0 0x7fb78c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.348+0000 7fb7bb8eeb80 1 -- start start 2024-05-27T07:49:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.348+0000 7fb79ea00700 1 -- 172.21.15.63:0/2030641770 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fb7a000f020 con 0x7fb7a8109670 2024-05-27T07:49:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.348+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.348+0000 7fb7a5400700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb78c0b29f0 0x7fb78c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.348+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 0x55eb3d3a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.348+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.348+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d2d6e40 con 0x55eb3d3b32c0 2024-05-27T07:49:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.348+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d2c69e0 con 0x55eb3d229e40 2024-05-27T07:49:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.348+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d2caa20 con 0x55eb3d3159f0 2024-05-27T07:49:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.349+0000 7fb7a6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.349+0000 7fb7a6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40320/0 (socket says 172.21.15.63:40320) 2024-05-27T07:49:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.349+0000 7fb7a6800700 1 -- 172.21.15.63:0/2407540324 learned_addr learned my addr 172.21.15.63:0/2407540324 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.349+0000 7fb7a5e00700 1 --2- 172.21.15.63:0/2407540324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.349+0000 7fb7a5400700 1 --2- 172.21.15.63:0/2407540324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 0x55eb3d3a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.349+0000 7fb7a6800700 1 -- 172.21.15.63:0/2407540324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 msgr2=0x55eb3d3a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.349+0000 7fb7a6800700 1 --2- 172.21.15.63:0/2407540324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.349+0000 7fb7a6800700 1 -- 172.21.15.63:0/2407540324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 msgr2=0x55eb3d3a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:36.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.349+0000 7fb7a6800700 1 --2- 172.21.15.63:0/2407540324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 0x55eb3d3a9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.350+0000 7fb7a6800700 1 -- 172.21.15.63:0/2407540324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3d28f420 con 0x55eb3d3b32c0 2024-05-27T07:49:36.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.350+0000 7fb7a5400700 1 --2- 172.21.15.63:0/2407540324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 0x55eb3d3a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:36.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.350+0000 7fb7a5400700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb78c0b29f0 0x7fb78c0b4f00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb79400b6c0 tx=0x7fb794012890 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:36.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.350+0000 7fb7a6800700 1 --2- 172.21.15.63:0/2407540324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b56e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00090a0 tx=0x7fb7a00aa670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf278fec804f939 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:36.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.350+0000 7fb793400700 1 -- 172.21.15.63:0/2407540324 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00aae30 con 0x55eb3d3b32c0 2024-05-27T07:49:36.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.350+0000 7fb793400700 1 -- 172.21.15.63:0/2407540324 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7a00af1e0 con 0x55eb3d3b32c0 2024-05-27T07:49:36.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.351+0000 7fb793400700 1 -- 172.21.15.63:0/2407540324 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00ab2d0 con 0x55eb3d3b32c0 2024-05-27T07:49:36.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.351+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/2407540324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 msgr2=0x55eb3d3b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:36.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.351+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/2407540324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b56e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00090a0 tx=0x7fb7a00aa670 comp rx=0 tx=0).stop 2024-05-27T07:49:36.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.352+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/2407540324 shutdown_connections 2024-05-27T07:49:36.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.352+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/2407540324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b56e0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.352+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/2407540324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.352+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/2407540324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 0x55eb3d3a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.352+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/2407540324 >> 172.21.15.63:0/2407540324 conn(0x55eb3d2c3390 msgr2=0x55eb3d3a4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:36.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.352+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/2407540324 shutdown_connections 2024-05-27T07:49:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.352+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/2407540324 wait complete. 2024-05-27T07:49:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.353+0000 7fb7bb8eeb80 1 -- start start 2024-05-27T07:49:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.353+0000 7fb7bb8eeb80 1 Processor -- start 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.353+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d229e40 0x55eb3d3b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.353+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.353+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.353+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d1ccfa0 con 0x55eb3d229e40 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.353+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d267760 con 0x55eb3d3b32c0 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.353+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55eb3d3b5c20 con 0x55eb3d3159f0 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7bb8eeb80 10 client.0 _ll_register_callbacks cb 0x55eb3d270c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7a5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d229e40 0x55eb3d3b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7a5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d229e40 0x55eb3d3b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40332/0 (socket says 172.21.15.63:40332) 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7a5e00700 1 -- 172.21.15.63:0/362306502 learned_addr learned my addr 172.21.15.63:0/362306502 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:36.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7a5400700 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7a6800700 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7a5e00700 1 -- 172.21.15.63:0/362306502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 msgr2=0x55eb3d3b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7a5e00700 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7a5e00700 1 -- 172.21.15.63:0/362306502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3b32c0 msgr2=0x55eb3d3b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7a5e00700 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.354+0000 7fb7a5e00700 1 -- 172.21.15.63:0/362306502 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3d2168d0 con 0x55eb3d229e40 2024-05-27T07:49:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.355+0000 7fb7a5e00700 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d229e40 0x55eb3d3b01d0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb79800d9b0 tx=0x7fb79800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a864c84e41ce74 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.355+0000 7fb787e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.355+0000 7fb787e00700 1 -- 172.21.15.63:0/362306502 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79801b440 con 0x55eb3d229e40 2024-05-27T07:49:36.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.355+0000 7fb787e00700 1 -- 172.21.15.63:0/362306502 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fb79801b5e0 con 0x55eb3d229e40 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.355+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/362306502 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a00af1e0 con 0x55eb3d229e40 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb787e00700 1 -- 172.21.15.63:0/362306502 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb798024050 con 0x55eb3d229e40 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb7bb8eeb80 10 client.18486 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/362306502 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eb3d297060 con 0x55eb3d229e40 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb787e00700 1 -- 172.21.15.63:0/362306502 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7a00af1e0 con 0x55eb3d229e40 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb787e00700 1 -- 172.21.15.63:0/362306502 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb798019020 con 0x55eb3d229e40 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb7bb8eeb80 10 client.18486 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb7bb8eeb80 10 client.18486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb7bb8eeb80 20 client.18486 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3c2ab0 0x55eb3d3c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb7bb8eeb80 4 client.18486 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:36.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.356+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/362306502 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb3d2d8d90 con 0x55eb3d3c2ab0 2024-05-27T07:49:36.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.357+0000 7fb7a5400700 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3c2ab0 0x55eb3d3c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.358+0000 7fb787e00700 10 client.18486 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:36.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.358+0000 7fb7a5400700 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3c2ab0 0x55eb3d3c4f30 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff20b9ed7ea7dda server_cookie=717cdb4b32508155 in_seq=0 out_seq=0 2024-05-27T07:49:36.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.360+0000 7fb787e00700 1 -- 172.21.15.63:0/362306502 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eb3d2d8d90 con 0x55eb3d3c2ab0 2024-05-27T07:49:36.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.360+0000 7fb787e00700 10 client.18486 handle_command_reply: tid=0 2024-05-27T07:49:36.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.360+0000 7fb7bb8eeb80 1 client.18486 shutdown 2024-05-27T07:49:36.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.360+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/362306502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d229e40 msgr2=0x55eb3d3b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:36.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.360+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d229e40 0x55eb3d3b01d0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb79800d9b0 tx=0x7fb79800dcc0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.361+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/362306502 shutdown_connections 2024-05-27T07:49:36.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.361+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d3c2ab0 0x55eb3d3c4f30 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.361+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.361+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d229e40 0x55eb3d3b01d0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.361+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/362306502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.361+0000 7fb787e00700 0 client.18486 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:36.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.361+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/362306502 >> 172.21.15.63:0/362306502 conn(0x55eb3d2c3390 msgr2=0x55eb3d3a9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:36.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.361+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/362306502 shutdown_connections 2024-05-27T07:49:36.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.361+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/362306502 wait complete. 2024-05-27T07:49:36.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.361+0000 7fb7bb8eeb80 20 client.18486 trim_cache size 0 max 16384 2024-05-27T07:49:36.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.384+0000 7fb7bb8eeb80 1 Processor -- start 2024-05-27T07:49:36.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.384+0000 7fb7bb8eeb80 1 -- start start 2024-05-27T07:49:36.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.385+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d229e40 0x55eb3d3bec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.385+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3159f0 0x55eb3d3ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.385+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.385+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a81b4d20 con 0x55eb3d3b32c0 2024-05-27T07:49:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.385+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a81b25b0 con 0x55eb3d3159f0 2024-05-27T07:49:36.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.385+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a81b0420 con 0x55eb3d229e40 2024-05-27T07:49:36.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.385+0000 7fb7a6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.385+0000 7fb7a5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d229e40 0x55eb3d3bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.385+0000 7fb7a6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40336/0 (socket says 172.21.15.63:40336) 2024-05-27T07:49:36.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.385+0000 7fb7a6800700 1 -- 172.21.15.63:0/3529246606 learned_addr learned my addr 172.21.15.63:0/3529246606 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:36.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.386+0000 7fb7a5400700 1 --2- 172.21.15.63:0/3529246606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3159f0 0x55eb3d3ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.386+0000 7fb7a6800700 1 -- 172.21.15.63:0/3529246606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d229e40 msgr2=0x55eb3d3bec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.386+0000 7fb7a6800700 1 --2- 172.21.15.63:0/3529246606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d229e40 0x55eb3d3bec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.386+0000 7fb7a6800700 1 -- 172.21.15.63:0/3529246606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3159f0 msgr2=0x55eb3d3ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.386+0000 7fb7a6800700 1 --2- 172.21.15.63:0/3529246606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3159f0 0x55eb3d3ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.386+0000 7fb7a6800700 1 -- 172.21.15.63:0/3529246606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3d3b0fc0 con 0x55eb3d3b32c0 2024-05-27T07:49:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.386+0000 7fb7a5400700 1 --2- 172.21.15.63:0/3529246606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3159f0 0x55eb3d3ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:36.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.387+0000 7fb7a6800700 1 --2- 172.21.15.63:0/3529246606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3cac30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00aadc0 tx=0x7fb7a000b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3da03bfed15eb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:36.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.387+0000 7fb792a00700 1 -- 172.21.15.63:0/3529246606 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00af1e0 con 0x55eb3d3b32c0 2024-05-27T07:49:36.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.387+0000 7fb792a00700 1 -- 172.21.15.63:0/3529246606 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7a00b3c00 con 0x55eb3d3b32c0 2024-05-27T07:49:36.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.387+0000 7fb792a00700 1 -- 172.21.15.63:0/3529246606 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00ab090 con 0x55eb3d3b32c0 2024-05-27T07:49:36.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.388+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3529246606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 msgr2=0x55eb3d3cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:36.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.388+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/3529246606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3cac30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00aadc0 tx=0x7fb7a000b9d0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.388+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3529246606 shutdown_connections 2024-05-27T07:49:36.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.388+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/3529246606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3cac30 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.388+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/3529246606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d229e40 0x55eb3d3bec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.388+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/3529246606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d3159f0 0x55eb3d3ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.388+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3529246606 >> 172.21.15.63:0/3529246606 conn(0x55eb3d2c3390 msgr2=0x55eb3d3aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:36.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.388+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3529246606 shutdown_connections 2024-05-27T07:49:36.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.388+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3529246606 wait complete. 2024-05-27T07:49:36.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.389+0000 7fb7bb8eeb80 1 -- start start 2024-05-27T07:49:36.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.389+0000 7fb7bb8eeb80 1 Processor -- start 2024-05-27T07:49:36.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.390+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 0x55eb3d3bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.390+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3b6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.390+0000 7fb7bb8eeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.390+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a81b0a70 con 0x55eb3d3b32c0 2024-05-27T07:49:36.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.390+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a81b1ee0 con 0x55eb3d229e40 2024-05-27T07:49:36.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.390+0000 7fb7bb8eeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a81b1140 con 0x55eb3d3159f0 2024-05-27T07:49:36.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.390+0000 7fb7bb8eeb80 10 client.0 _ll_register_callbacks cb 0x55eb3d270c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:36.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.391+0000 7fb7a5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 0x55eb3d3bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.391+0000 7fb7a5400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.391+0000 7fb7a5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 0x55eb3d3bb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:38264/0 (socket says 172.21.15.63:38264) 2024-05-27T07:49:36.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.391+0000 7fb7a5e00700 1 -- 172.21.15.63:0/3361974821 learned_addr learned my addr 172.21.15.63:0/3361974821 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:36.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.391+0000 7fb7a6800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.391+0000 7fb7a6800700 1 -- 172.21.15.63:0/3361974821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 msgr2=0x55eb3d3b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:36.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.391+0000 7fb7a6800700 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.391+0000 7fb7a6800700 1 -- 172.21.15.63:0/3361974821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 msgr2=0x55eb3d3bb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:36.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.391+0000 7fb7a6800700 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 0x55eb3d3bb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.391+0000 7fb7a6800700 1 -- 172.21.15.63:0/3361974821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3d3b7690 con 0x55eb3d3b32c0 2024-05-27T07:49:36.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.392+0000 7fb7a6800700 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b6ef0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00aab30 tx=0x7fb7a00aa620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5d26d43dcd0a35 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:36.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.392+0000 7fb787e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:36.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.392+0000 7fb787e00700 1 -- 172.21.15.63:0/3361974821 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00cb070 con 0x55eb3d3b32c0 2024-05-27T07:49:36.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.392+0000 7fb787e00700 1 -- 172.21.15.63:0/3361974821 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fb7a00a7420 con 0x55eb3d3b32c0 2024-05-27T07:49:36.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.392+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3361974821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb3d2d2570 con 0x55eb3d3b32c0 2024-05-27T07:49:36.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.392+0000 7fb787e00700 1 -- 172.21.15.63:0/3361974821 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb7a00e16a0 con 0x55eb3d3b32c0 2024-05-27T07:49:36.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.392+0000 7fb7bb8eeb80 10 client.18492 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:36.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.392+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3361974821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eb3d42d560 con 0x55eb3d3b32c0 2024-05-27T07:49:36.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.393+0000 7fb787e00700 1 -- 172.21.15.63:0/3361974821 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7a00b3e50 con 0x55eb3d3b32c0 2024-05-27T07:49:36.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.393+0000 7fb787e00700 1 -- 172.21.15.63:0/3361974821 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb7a00c3020 con 0x55eb3d3b32c0 2024-05-27T07:49:36.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.393+0000 7fb7bb8eeb80 10 client.18492 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:36.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.393+0000 7fb7bb8eeb80 10 client.18492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:36.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.393+0000 7fb7bb8eeb80 20 client.18492 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:36.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.393+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d42d7d0 0x55eb3d42fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:36.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.393+0000 7fb7bb8eeb80 4 client.18492 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:49:36.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.393+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3361974821 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb3d149250 con 0x55eb3d42d7d0 2024-05-27T07:49:36.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.394+0000 7fb7a5e00700 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d42d7d0 0x55eb3d42fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:36.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.395+0000 7fb7a5e00700 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d42d7d0 0x55eb3d42fbf0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca89507ecfe5604e server_cookie=8054f4e991e4972d in_seq=0 out_seq=0 2024-05-27T07:49:36.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.395+0000 7fb787e00700 10 client.18492 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:36.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.395+0000 7fb787e00700 1 -- 172.21.15.63:0/3361974821 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eb3d149250 con 0x55eb3d42d7d0 2024-05-27T07:49:36.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.395+0000 7fb787e00700 10 client.18492 handle_command_reply: tid=0 2024-05-27T07:49:36.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.395+0000 7fb7bb8eeb80 1 client.18492 shutdown 2024-05-27T07:49:36.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3361974821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 msgr2=0x55eb3d3b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:36.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b6ef0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00aab30 tx=0x7fb7a00aa620 comp rx=0 tx=0).stop 2024-05-27T07:49:36.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3361974821 shutdown_connections 2024-05-27T07:49:36.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eb3d42d7d0 0x55eb3d42fbf0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3d3b32c0 0x55eb3d3b6ef0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb3d229e40 0x55eb3d3bb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 1 --2- 172.21.15.63:0/3361974821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eb3d3159f0 0x55eb3d3b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb787e00700 0 client.18492 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:36.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3361974821 >> 172.21.15.63:0/3361974821 conn(0x55eb3d2c3390 msgr2=0x55eb3d3a3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:36.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3361974821 shutdown_connections 2024-05-27T07:49:36.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 1 -- 172.21.15.63:0/3361974821 wait complete. 2024-05-27T07:49:36.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.396+0000 7fb7bb8eeb80 20 client.18492 trim_cache size 0 max 16384 2024-05-27T07:49:36.398 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:49:36.398 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:49:36.398 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:49:36.398 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:49:36.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.397+0000 7fb7a7200700 1 -- 172.21.15.63:0/2030641770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb78c0b29f0 msgr2=0x7fb78c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:36.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.397+0000 7fb7a7200700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb78c0b29f0 0x7fb78c0b4f00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb79400b6c0 tx=0x7fb794012890 comp rx=0 tx=0).stop 2024-05-27T07:49:36.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.397+0000 7fb7a7200700 1 -- 172.21.15.63:0/2030641770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a8109670 msgr2=0x7fb7a81a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:36.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.397+0000 7fb7a7200700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a8109670 0x7fb7a81a1570 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0009850 tx=0x7fb7a0007d90 comp rx=0 tx=0).stop 2024-05-27T07:49:36.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.398+0000 7fb7a7200700 1 -- 172.21.15.63:0/2030641770 shutdown_connections 2024-05-27T07:49:36.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.398+0000 7fb7a7200700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb78c0b29f0 0x7fb78c0b4f00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.398+0000 7fb7a7200700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a8109670 0x7fb7a81a1570 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.398+0000 7fb7a7200700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a81043f0 0x7fb7a819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.398+0000 7fb7a7200700 1 --2- 172.21.15.63:0/2030641770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7a8108d20 0x7fb7a819d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:36.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.398+0000 7fb7a7200700 1 -- 172.21.15.63:0/2030641770 >> 172.21.15.63:0/2030641770 conn(0x7fb7a8079d00 msgr2=0x7fb7a8110d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:36.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.398+0000 7fb7a7200700 1 -- 172.21.15.63:0/2030641770 shutdown_connections 2024-05-27T07:49:36.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:36.398+0000 7fb7a7200700 1 -- 172.21.15.63:0/2030641770 wait complete. 2024-05-27T07:49:37.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:36 smithi086 ceph-mon[34687]: pgmap v504: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:49:37.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:36 smithi149 ceph-mon[35408]: pgmap v504: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:49:37.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:36 smithi063 ceph-mon[31613]: pgmap v504: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:49:37.398 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:49:37.399 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:49:37.898 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:39.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:38 smithi086 ceph-mon[34687]: pgmap v505: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:49:39.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:38 smithi149 ceph-mon[35408]: pgmap v505: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:49:39.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:38 smithi063 ceph-mon[31613]: pgmap v505: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:49:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.447+0000 7f34f4000700 1 -- 172.21.15.63:0/864186780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34ec106eb0 msgr2=0x7f34ec113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.447+0000 7f34f4000700 1 --2- 172.21.15.63:0/864186780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34ec106eb0 0x7f34ec113340 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f34e400b0a0 tx=0x7f34e400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:49:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.447+0000 7f34f4000700 1 -- 172.21.15.63:0/864186780 shutdown_connections 2024-05-27T07:49:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.447+0000 7f34f4000700 1 --2- 172.21.15.63:0/864186780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34ec106eb0 0x7f34ec113340 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.447+0000 7f34f4000700 1 --2- 172.21.15.63:0/864186780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34ec10c5a0 0x7f34ec10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.447+0000 7f34f4000700 1 --2- 172.21.15.63:0/864186780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34ec106560 0x7f34ec106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.447+0000 7f34f4000700 1 -- 172.21.15.63:0/864186780 >> 172.21.15.63:0/864186780 conn(0x7f34ec102040 msgr2=0x7f34ec104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.447+0000 7f34f4000700 1 -- 172.21.15.63:0/864186780 shutdown_connections 2024-05-27T07:49:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.447+0000 7f34f4000700 1 -- 172.21.15.63:0/864186780 wait complete. 2024-05-27T07:49:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.449+0000 7f34f4000700 1 Processor -- start 2024-05-27T07:49:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.449+0000 7f34f4000700 1 -- start start 2024-05-27T07:49:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.449+0000 7f34f4000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34ec106560 0x7f34ec19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.450+0000 7f34f4000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34ec106eb0 0x7f34ec19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.450+0000 7f34f4000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34ec10c5a0 0x7f34ec1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.450+0000 7f34f4000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec1165d0 con 0x7f34ec106eb0 2024-05-27T07:49:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.450+0000 7f34f4000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec116450 con 0x7f34ec10c5a0 2024-05-27T07:49:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.450+0000 7f34f4000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec116750 con 0x7f34ec106560 2024-05-27T07:49:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.450+0000 7f34f0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34ec106eb0 0x7f34ec19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.450+0000 7f34f0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34ec106eb0 0x7f34ec19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54798/0 (socket says 172.21.15.63:54798) 2024-05-27T07:49:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.450+0000 7f34f0c00700 1 -- 172.21.15.63:0/1251946409 learned_addr learned my addr 172.21.15.63:0/1251946409 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.450+0000 7f34f1600700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34ec106560 0x7f34ec19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.450+0000 7f34f2000700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34ec10c5a0 0x7f34ec1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.451+0000 7f34f0c00700 1 -- 172.21.15.63:0/1251946409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34ec106560 msgr2=0x7f34ec19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.451+0000 7f34f0c00700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34ec106560 0x7f34ec19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.451+0000 7f34f0c00700 1 -- 172.21.15.63:0/1251946409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34ec10c5a0 msgr2=0x7f34ec1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.451+0000 7f34f0c00700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34ec10c5a0 0x7f34ec1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.451+0000 7f34f0c00700 1 -- 172.21.15.63:0/1251946409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec1a1dd0 con 0x7f34ec106eb0 2024-05-27T07:49:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.451+0000 7f34f2000700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34ec10c5a0 0x7f34ec1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.451+0000 7f34f0c00700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34ec106eb0 0x7f34ec19d4a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f34d800d950 tx=0x7f34d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.452+0000 7f34ea000700 1 -- 172.21.15.63:0/1251946409 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8017070 con 0x7f34ec106eb0 2024-05-27T07:49:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.452+0000 7f34f4000700 1 -- 172.21.15.63:0/1251946409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ec1a20c0 con 0x7f34ec106eb0 2024-05-27T07:49:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.452+0000 7f34ea000700 1 -- 172.21.15.63:0/1251946409 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34d8009dc0 con 0x7f34ec106eb0 2024-05-27T07:49:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.452+0000 7f34ea000700 1 -- 172.21.15.63:0/1251946409 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d800f3c0 con 0x7f34ec106eb0 2024-05-27T07:49:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.452+0000 7f34f4000700 1 -- 172.21.15.63:0/1251946409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ec1a99e0 con 0x7f34ec106eb0 2024-05-27T07:49:40.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.453+0000 7f34f4000700 1 -- 172.21.15.63:0/1251946409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34cc005330 con 0x7f34ec106eb0 2024-05-27T07:49:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.457+0000 7f34ea000700 1 -- 172.21.15.63:0/1251946409 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f34d801a070 con 0x7f34ec106eb0 2024-05-27T07:49:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.458+0000 7f34ea000700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34d40b29f0 0x7f34d40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.459+0000 7f34ea000700 1 -- 172.21.15.63:0/1251946409 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f34d80e2170 con 0x7f34ec106eb0 2024-05-27T07:49:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.459+0000 7f34f1600700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34d40b29f0 0x7f34d40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.460+0000 7f34f1600700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34d40b29f0 0x7f34d40b4f00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f34e0005950 tx=0x7f34e000a300 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:40.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.462+0000 7f34ea000700 1 -- 172.21.15.63:0/1251946409 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f34d80a9730 con 0x7f34ec106eb0 2024-05-27T07:49:40.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.817+0000 7f34f4000700 1 -- 172.21.15.63:0/1251946409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34cc005620 con 0x7f34ec106eb0 2024-05-27T07:49:40.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.819+0000 7f34ea000700 1 -- 172.21.15.63:0/1251946409 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f34d80a9030 con 0x7f34ec106eb0 2024-05-27T07:49:40.820 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:49:40.820 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:49:40.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.822+0000 7f34f4000700 1 -- 172.21.15.63:0/1251946409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34d40b29f0 msgr2=0x7f34d40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:40.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.822+0000 7f34f4000700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34d40b29f0 0x7f34d40b4f00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f34e0005950 tx=0x7f34e000a300 comp rx=0 tx=0).stop 2024-05-27T07:49:40.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.822+0000 7f34f4000700 1 -- 172.21.15.63:0/1251946409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34ec106eb0 msgr2=0x7f34ec19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:40.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.822+0000 7f34f4000700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34ec106eb0 0x7f34ec19d4a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f34d800d950 tx=0x7f34d800dc60 comp rx=0 tx=0).stop 2024-05-27T07:49:40.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.822+0000 7f34f4000700 1 -- 172.21.15.63:0/1251946409 shutdown_connections 2024-05-27T07:49:40.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.822+0000 7f34f4000700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34d40b29f0 0x7f34d40b4f00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:40.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.822+0000 7f34f4000700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34ec10c5a0 0x7f34ec1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:40.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.822+0000 7f34f4000700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34ec106560 0x7f34ec19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:40.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.822+0000 7f34f4000700 1 --2- 172.21.15.63:0/1251946409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34ec106eb0 0x7f34ec19d4a0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:40.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.822+0000 7f34f4000700 1 -- 172.21.15.63:0/1251946409 >> 172.21.15.63:0/1251946409 conn(0x7f34ec102040 msgr2=0x7f34ec1035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:40.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.823+0000 7f34f4000700 1 -- 172.21.15.63:0/1251946409 shutdown_connections 2024-05-27T07:49:40.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:40.823+0000 7f34f4000700 1 -- 172.21.15.63:0/1251946409 wait complete. 2024-05-27T07:49:40.825 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:49:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:40 smithi086 ceph-mon[34687]: pgmap v506: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:49:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:40 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1251946409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:49:41.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:40 smithi149 ceph-mon[35408]: pgmap v506: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:49:41.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:40 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1251946409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:49:41.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:40 smithi063 ceph-mon[31613]: pgmap v506: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:49:41.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:40 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1251946409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:49:41.810 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:49:42.310 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:43.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:42 smithi086 ceph-mon[34687]: pgmap v507: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:49:43.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:42 smithi149 ceph-mon[35408]: pgmap v507: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:49:43.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:42 smithi063 ceph-mon[31613]: pgmap v507: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:49:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.038+0000 7f9fb1000700 1 -- 172.21.15.63:0/1495340263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fac10e900 msgr2=0x7f9fac110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.038+0000 7f9fb1000700 1 --2- 172.21.15.63:0/1495340263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fac10e900 0x7f9fac110de0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400b0a0 tx=0x7f9fa400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.038+0000 7f9fb1000700 1 -- 172.21.15.63:0/1495340263 shutdown_connections 2024-05-27T07:49:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.038+0000 7f9fb1000700 1 --2- 172.21.15.63:0/1495340263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fac10e900 0x7f9fac110de0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.038+0000 7f9fb1000700 1 --2- 172.21.15.63:0/1495340263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fac1054d0 0x7f9fac1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.038+0000 7f9fb1000700 1 --2- 172.21.15.63:0/1495340263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fac105e20 0x7f9fac10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.038+0000 7f9fb1000700 1 -- 172.21.15.63:0/1495340263 >> 172.21.15.63:0/1495340263 conn(0x7f9fac0fef20 msgr2=0x7f9fac101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.038+0000 7f9fb1000700 1 -- 172.21.15.63:0/1495340263 shutdown_connections 2024-05-27T07:49:45.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.038+0000 7f9fb1000700 1 -- 172.21.15.63:0/1495340263 wait complete. 2024-05-27T07:49:45.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.039+0000 7f9fb1000700 1 Processor -- start 2024-05-27T07:49:45.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.040+0000 7f9fb1000700 1 -- start start 2024-05-27T07:49:45.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.040+0000 7f9fb1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fac1054d0 0x7f9fac19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.040+0000 7f9fb1000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fac105e20 0x7f9fac19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.040+0000 7f9fb1000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fac10e900 0x7f9fac1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.040+0000 7f9fb1000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac114360 con 0x7f9fac105e20 2024-05-27T07:49:45.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.040+0000 7f9fb1000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac1141e0 con 0x7f9fac1054d0 2024-05-27T07:49:45.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.040+0000 7f9fb1000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac1144e0 con 0x7f9fac10e900 2024-05-27T07:49:45.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.040+0000 7f9fa9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fac105e20 0x7f9fac19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.041+0000 7f9fa9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fac105e20 0x7f9fac19d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54820/0 (socket says 172.21.15.63:54820) 2024-05-27T07:49:45.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.041+0000 7f9fa9400700 1 -- 172.21.15.63:0/3659548530 learned_addr learned my addr 172.21.15.63:0/3659548530 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:45.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.041+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fac1054d0 0x7f9fac19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.041+0000 7f9faa800700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fac10e900 0x7f9fac1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.041+0000 7f9fa9400700 1 -- 172.21.15.63:0/3659548530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fac10e900 msgr2=0x7f9fac1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.041+0000 7f9fa9400700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fac10e900 0x7f9fac1a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.041+0000 7f9fa9400700 1 -- 172.21.15.63:0/3659548530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fac1054d0 msgr2=0x7f9fac19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.041+0000 7f9fa9400700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fac1054d0 0x7f9fac19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.041+0000 7f9fa9400700 1 -- 172.21.15.63:0/3659548530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac1a1dc0 con 0x7f9fac105e20 2024-05-27T07:49:45.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.042+0000 7f9fa9400700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fac105e20 0x7f9fac19d490 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800d950 tx=0x7f9f9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.042+0000 7f9fa2a00700 1 -- 172.21.15.63:0/3659548530 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98016070 con 0x7f9fac105e20 2024-05-27T07:49:45.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.042+0000 7f9fa2a00700 1 -- 172.21.15.63:0/3659548530 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f98009dd0 con 0x7f9fac105e20 2024-05-27T07:49:45.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.042+0000 7f9fb1000700 1 -- 172.21.15.63:0/3659548530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac1a20b0 con 0x7f9fac105e20 2024-05-27T07:49:45.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.042+0000 7f9fb1000700 1 -- 172.21.15.63:0/3659548530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac1a99d0 con 0x7f9fac105e20 2024-05-27T07:49:45.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.042+0000 7f9fa2a00700 1 -- 172.21.15.63:0/3659548530 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98020060 con 0x7f9fac105e20 2024-05-27T07:49:45.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.048+0000 7f9fa2a00700 1 -- 172.21.15.63:0/3659548530 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9f98019070 con 0x7f9fac105e20 2024-05-27T07:49:45.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.048+0000 7f9fbfafeb80 1 Processor -- start 2024-05-27T07:49:45.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.048+0000 7f9fbfafeb80 1 -- start start 2024-05-27T07:49:45.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.049+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 0x563ab01297f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.049+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab0123870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.049+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 0x563ab01241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.049+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563ab0051e40 con 0x563ab0129d30 2024-05-27T07:49:45.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.049+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563ab00419e0 con 0x563aaffa4e40 2024-05-27T07:49:45.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.049+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ab0045a20 con 0x563ab0123db0 2024-05-27T07:49:45.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.049+0000 7f9fa9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab0123870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.049+0000 7f9fa9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab0123870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54836/0 (socket says 172.21.15.63:54836) 2024-05-27T07:49:45.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.049+0000 7f9fa9e00700 1 -- 172.21.15.63:0/2119306584 learned_addr learned my addr 172.21.15.63:0/2119306584 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.049+0000 7f9faa800700 1 --2- 172.21.15.63:0/2119306584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 0x563ab01297f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.050+0000 7f9fa9400700 1 --2- 172.21.15.63:0/2119306584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 0x563ab01241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.050+0000 7f9fa9e00700 1 -- 172.21.15.63:0/2119306584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 msgr2=0x563ab01241c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.050+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/2119306584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 0x563ab01241c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.050+0000 7f9fa9e00700 1 -- 172.21.15.63:0/2119306584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 msgr2=0x563ab01297f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.050+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/2119306584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 0x563ab01297f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.050+0000 7f9fa9e00700 1 -- 172.21.15.63:0/2119306584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ab000a420 con 0x563ab0129d30 2024-05-27T07:49:45.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.050+0000 7f9fa2a00700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f900b29f0 0x7f9f900b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.050+0000 7f9faa800700 1 --2- 172.21.15.63:0/2119306584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 0x563ab01297f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:45.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.050+0000 7f9fa2a00700 1 -- 172.21.15.63:0/3659548530 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f9f980e2980 con 0x7f9fac105e20 2024-05-27T07:49:45.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.050+0000 7f9faa800700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f900b29f0 0x7f9f900b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.051+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/2119306584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab0123870 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00bd70 tx=0x7f9f9c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204fe47eb9056c82 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.051+0000 7f9f97400700 1 -- 172.21.15.63:0/2119306584 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c01c480 con 0x563ab0129d30 2024-05-27T07:49:45.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.051+0000 7f9f97400700 1 -- 172.21.15.63:0/2119306584 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f9c01cab0 con 0x563ab0129d30 2024-05-27T07:49:45.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.051+0000 7f9f97400700 1 -- 172.21.15.63:0/2119306584 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c011880 con 0x563ab0129d30 2024-05-27T07:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.052+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2119306584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 msgr2=0x563ab0123870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.052+0000 7f9faa800700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f900b29f0 0x7f9f900b4f00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9fa40062a0 tx=0x7f9fa4006210 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.052+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/2119306584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab0123870 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00bd70 tx=0x7f9f9c0106b0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.053+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2119306584 shutdown_connections 2024-05-27T07:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.053+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/2119306584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 0x563ab01241c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.053+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/2119306584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 0x563ab01297f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.053+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/2119306584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab0123870 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.053+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2119306584 >> 172.21.15.63:0/2119306584 conn(0x563ab0004fc0 msgr2=0x563ab011e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.053+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2119306584 shutdown_connections 2024-05-27T07:49:45.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.053+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2119306584 wait complete. 2024-05-27T07:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.053+0000 7f9fbfafeb80 1 -- start start 2024-05-27T07:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.054+0000 7f9fbfafeb80 1 Processor -- start 2024-05-27T07:49:45.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.054+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 0x563ab012d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.054+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0123db0 0x563ab012daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.054+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0129d30 0x563ab012dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.054+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563aaff47fa0 con 0x563ab0123db0 2024-05-27T07:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.054+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563aaffe2760 con 0x563aaffa4e40 2024-05-27T07:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.054+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ab0130800 con 0x563ab0129d30 2024-05-27T07:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.055+0000 7f9fbfafeb80 10 client.0 _ll_register_callbacks cb 0x563aaffebc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.055+0000 7f9fa9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0123db0 0x563ab012daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.055+0000 7f9fa9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0123db0 0x563ab012daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54838/0 (socket says 172.21.15.63:54838) 2024-05-27T07:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.055+0000 7f9fa9400700 1 -- 172.21.15.63:0/1931222206 learned_addr learned my addr 172.21.15.63:0/1931222206 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:45.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.055+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 0x563ab012d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.055+0000 7f9fa9400700 1 -- 172.21.15.63:0/1931222206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0129d30 msgr2=0x563ab012dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.055+0000 7f9fa9400700 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0129d30 0x563ab012dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.055+0000 7f9fa9400700 1 -- 172.21.15.63:0/1931222206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 msgr2=0x563ab012d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.055+0000 7f9fa9400700 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 0x563ab012d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.055+0000 7f9fa9400700 1 -- 172.21.15.63:0/1931222206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aaff918d0 con 0x563ab0123db0 2024-05-27T07:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.056+0000 7f9fa9400700 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0123db0 0x563ab012daa0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9f980a58c0 tx=0x7f9f980108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ffc7fec94b1c35 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.056+0000 7f9f8be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.056+0000 7f9f8be00700 1 -- 172.21.15.63:0/1931222206 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f980a9f40 con 0x563ab0123db0 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.056+0000 7f9f8be00700 1 -- 172.21.15.63:0/1931222206 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f9f980dff30 con 0x563ab0123db0 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.056+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/1931222206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f9c01cab0 con 0x563ab0123db0 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.056+0000 7f9f8be00700 1 -- 172.21.15.63:0/1931222206 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9f980a9840 con 0x563ab0123db0 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.057+0000 7f9fbfafeb80 10 client.18510 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.057+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/1931222206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ab0012060 con 0x563ab0123db0 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.057+0000 7f9f8be00700 1 -- 172.21.15.63:0/1931222206 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f9c01cab0 con 0x563ab0123db0 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.057+0000 7f9f8be00700 1 -- 172.21.15.63:0/1931222206 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9f980b2210 con 0x563ab0123db0 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.057+0000 7f9fbfafeb80 10 client.18510 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.057+0000 7f9fbfafeb80 10 client.18510 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.057+0000 7f9fbfafeb80 20 client.18510 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.057+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x563ab013d6a0 0x563ab013fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.057+0000 7f9fbfafeb80 4 client.18510 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:45.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.057+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/1931222206 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ab0053d90 con 0x563ab013d6a0 2024-05-27T07:49:45.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.058+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x563ab013d6a0 0x563ab013fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.059+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x563ab013d6a0 0x563ab013fae0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10cd234d545d2fca server_cookie=6b596ad249840d0f in_seq=0 out_seq=0 2024-05-27T07:49:45.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.059+0000 7f9f8be00700 10 client.18510 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:49:45.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.061+0000 7f9f8be00700 1 -- 172.21.15.63:0/1931222206 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563ab0053d90 con 0x563ab013d6a0 2024-05-27T07:49:45.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.061+0000 7f9f8be00700 10 client.18510 handle_command_reply: tid=0 2024-05-27T07:49:45.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.061+0000 7f9fbfafeb80 1 client.18510 shutdown 2024-05-27T07:49:45.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/1931222206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0123db0 msgr2=0x563ab012daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0123db0 0x563ab012daa0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9f980a58c0 tx=0x7f9f980108d0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/1931222206 shutdown_connections 2024-05-27T07:49:45.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0129d30 0x563ab012dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563aaffa4e40 0x563ab012d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x563ab013d6a0 0x563ab013fae0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/1931222206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0123db0 0x563ab012daa0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/1931222206 >> 172.21.15.63:0/1931222206 conn(0x563ab0004fc0 msgr2=0x563ab011e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9f8be00700 0 client.18510 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:49:45.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/1931222206 shutdown_connections 2024-05-27T07:49:45.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/1931222206 wait complete. 2024-05-27T07:49:45.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.062+0000 7f9fbfafeb80 20 client.18510 trim_cache size 0 max 16384 2024-05-27T07:49:45.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.085+0000 7f9fbfafeb80 1 Processor -- start 2024-05-27T07:49:45.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.085+0000 7f9fbfafeb80 1 -- start start 2024-05-27T07:49:45.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.086+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563aaffa4e40 0x563ab0123870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.086+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0123db0 0x563ab012fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.086+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab012dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.086+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac1b5060 con 0x563ab0129d30 2024-05-27T07:49:45.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.086+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac1b28f0 con 0x563ab0123db0 2024-05-27T07:49:45.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.086+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac1b0760 con 0x563aaffa4e40 2024-05-27T07:49:45.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.086+0000 7f9faa800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab012dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.086+0000 7f9faa800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab012dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54846/0 (socket says 172.21.15.63:54846) 2024-05-27T07:49:45.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.086+0000 7f9faa800700 1 -- 172.21.15.63:0/4140384699 learned_addr learned my addr 172.21.15.63:0/4140384699 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:45.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.087+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/4140384699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563aaffa4e40 0x563ab0123870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.087+0000 7f9fa9400700 1 --2- 172.21.15.63:0/4140384699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0123db0 0x563ab012fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.087+0000 7f9faa800700 1 -- 172.21.15.63:0/4140384699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563aaffa4e40 msgr2=0x563ab0123870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.087+0000 7f9faa800700 1 --2- 172.21.15.63:0/4140384699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563aaffa4e40 0x563ab0123870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.087+0000 7f9faa800700 1 -- 172.21.15.63:0/4140384699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0123db0 msgr2=0x563ab012fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.087+0000 7f9faa800700 1 --2- 172.21.15.63:0/4140384699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0123db0 0x563ab012fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.087+0000 7f9faa800700 1 -- 172.21.15.63:0/4140384699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ab012bad0 con 0x563ab0129d30 2024-05-27T07:49:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.087+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/4140384699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563aaffa4e40 0x563ab0123870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.087+0000 7f9faa800700 1 --2- 172.21.15.63:0/4140384699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab012dc30 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4015000 tx=0x7f9fa401e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc41e8b7655693 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.088+0000 7f9f96a00700 1 -- 172.21.15.63:0/4140384699 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa402c070 con 0x563ab0129d30 2024-05-27T07:49:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.088+0000 7f9f96a00700 1 -- 172.21.15.63:0/4140384699 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fa4007c50 con 0x563ab0129d30 2024-05-27T07:49:45.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.088+0000 7f9f96a00700 1 -- 172.21.15.63:0/4140384699 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4022070 con 0x563ab0129d30 2024-05-27T07:49:45.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.089+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/4140384699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 msgr2=0x563ab012dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.089+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/4140384699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab012dc30 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4015000 tx=0x7f9fa401e070 comp rx=0 tx=0).stop 2024-05-27T07:49:45.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.089+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/4140384699 shutdown_connections 2024-05-27T07:49:45.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.089+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/4140384699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ab0129d30 0x563ab012dc30 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.089+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/4140384699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563aaffa4e40 0x563ab0123870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.089+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/4140384699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0123db0 0x563ab012fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.089+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/4140384699 >> 172.21.15.63:0/4140384699 conn(0x563ab0004fc0 msgr2=0x563ab012d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.089+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/4140384699 shutdown_connections 2024-05-27T07:49:45.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.089+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/4140384699 wait complete. 2024-05-27T07:49:45.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.090+0000 7f9fbfafeb80 1 -- start start 2024-05-27T07:49:45.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.090+0000 7f9fbfafeb80 1 Processor -- start 2024-05-27T07:49:45.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563aaffa4e40 0x563ab0135a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 0x563ab0135fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fbfafeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0129d30 0x563ab012e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac1b0db0 con 0x563aaffa4e40 2024-05-27T07:49:45.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac1b2220 con 0x563ab0129d30 2024-05-27T07:49:45.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fbfafeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac1b1480 con 0x563ab0123db0 2024-05-27T07:49:45.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fbfafeb80 10 client.0 _ll_register_callbacks cb 0x563aaffebc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:45.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fa9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563aaffa4e40 0x563ab0135a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fa9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563aaffa4e40 0x563ab0135a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54860/0 (socket says 172.21.15.63:54860) 2024-05-27T07:49:45.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fa9e00700 1 -- 172.21.15.63:0/2253582630 learned_addr learned my addr 172.21.15.63:0/2253582630 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:45.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.091+0000 7f9fa9400700 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 0x563ab0135fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.092+0000 7f9faa800700 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0129d30 0x563ab012e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.092+0000 7f9fa9e00700 1 -- 172.21.15.63:0/2253582630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 msgr2=0x563ab0135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.092+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 0x563ab0135fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.092+0000 7f9fa9e00700 1 -- 172.21.15.63:0/2253582630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0129d30 msgr2=0x563ab012e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.092+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0129d30 0x563ab012e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.092+0000 7f9fa9e00700 1 -- 172.21.15.63:0/2253582630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ab0136590 con 0x563aaffa4e40 2024-05-27T07:49:45.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.092+0000 7f9fa9400700 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 0x563ab0135fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:45.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.092+0000 7f9faa800700 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0129d30 0x563ab012e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:45.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.092+0000 7f9fa9e00700 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563aaffa4e40 0x563ab0135a90 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c000c00 tx=0x7f9f9c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd5c6f7389fb08b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.092+0000 7f9f8be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:45.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.093+0000 7f9f8be00700 1 -- 172.21.15.63:0/2253582630 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c01c890 con 0x563aaffa4e40 2024-05-27T07:49:45.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.093+0000 7f9f8be00700 1 -- 172.21.15.63:0/2253582630 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f9f9c01ca30 con 0x563aaffa4e40 2024-05-27T07:49:45.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.093+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2253582630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa4007c50 con 0x563aaffa4e40 2024-05-27T07:49:45.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.093+0000 7f9f8be00700 1 -- 172.21.15.63:0/2253582630 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9f9c037050 con 0x563aaffa4e40 2024-05-27T07:49:45.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.093+0000 7f9fbfafeb80 10 client.18516 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:45.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.093+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2253582630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ab01afe60 con 0x563aaffa4e40 2024-05-27T07:49:45.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.094+0000 7f9f8be00700 1 -- 172.21.15.63:0/2253582630 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fa4007c50 con 0x563aaffa4e40 2024-05-27T07:49:45.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.094+0000 7f9f8be00700 1 -- 172.21.15.63:0/2253582630 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9f9c0039f0 con 0x563aaffa4e40 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.094+0000 7f9fbfafeb80 10 client.18516 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.094+0000 7f9fbfafeb80 10 client.18516 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.094+0000 7f9fbfafeb80 20 client.18516 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.094+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x563ab01b0340 0x563ab01b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.094+0000 7f9fbfafeb80 4 client.18516 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.094+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2253582630 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ab004d570 con 0x563ab01b0340 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.095+0000 7f9fa9400700 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x563ab01b0340 0x563ab01b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.096+0000 7f9fa9400700 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x563ab01b0340 0x563ab01b2760 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f0d570ccd6838db server_cookie=cca3da4580120afd in_seq=0 out_seq=0 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.096+0000 7f9f8be00700 10 client.18516 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.096+0000 7f9f8be00700 1 -- 172.21.15.63:0/2253582630 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ab004d570 con 0x563ab01b0340 2024-05-27T07:49:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.096+0000 7f9f8be00700 10 client.18516 handle_command_reply: tid=0 2024-05-27T07:49:45.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.096+0000 7f9fbfafeb80 1 client.18516 shutdown 2024-05-27T07:49:45.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.096+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2253582630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563aaffa4e40 msgr2=0x563ab0135a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.096+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563aaffa4e40 0x563ab0135a90 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c000c00 tx=0x7f9f9c00bee0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.097+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2253582630 shutdown_connections 2024-05-27T07:49:45.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.097+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ab0129d30 0x563ab012e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.097+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x563ab01b0340 0x563ab01b2760 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.097+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563aaffa4e40 0x563ab0135a90 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.097+0000 7f9fbfafeb80 1 --2- 172.21.15.63:0/2253582630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ab0123db0 0x563ab0135fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.097+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2253582630 >> 172.21.15.63:0/2253582630 conn(0x563ab0004fc0 msgr2=0x563ab0127140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.097+0000 7f9f8be00700 0 client.18516 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:49:45.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.097+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2253582630 shutdown_connections 2024-05-27T07:49:45.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.097+0000 7f9fbfafeb80 1 -- 172.21.15.63:0/2253582630 wait complete. 2024-05-27T07:49:45.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.097+0000 7f9fbfafeb80 20 client.18516 trim_cache size 0 max 16384 2024-05-27T07:49:45.099 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:49:45.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 -- 172.21.15.63:0/3659548530 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f900b29f0 msgr2=0x7f9f900b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f900b29f0 0x7f9f900b4f00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9fa40062a0 tx=0x7f9fa4006210 comp rx=0 tx=0).stop 2024-05-27T07:49:45.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 -- 172.21.15.63:0/3659548530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fac105e20 msgr2=0x7f9fac19d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fac105e20 0x7f9fac19d490 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800d950 tx=0x7f9f9800dc60 comp rx=0 tx=0).stop 2024-05-27T07:49:45.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 -- 172.21.15.63:0/3659548530 shutdown_connections 2024-05-27T07:49:45.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f900b29f0 0x7f9f900b4f00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fac10e900 0x7f9fac1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fac1054d0 0x7f9fac19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 --2- 172.21.15.63:0/3659548530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fac105e20 0x7f9fac19d490 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 -- 172.21.15.63:0/3659548530 >> 172.21.15.63:0/3659548530 conn(0x7f9fac0fef20 msgr2=0x7f9fac0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.098+0000 7f9fb1000700 1 -- 172.21.15.63:0/3659548530 shutdown_connections 2024-05-27T07:49:45.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:45.099+0000 7f9fb1000700 1 -- 172.21.15.63:0/3659548530 wait complete. 2024-05-27T07:49:45.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:44 smithi086 ceph-mon[34687]: pgmap v508: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:49:45.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:44 smithi149 ceph-mon[35408]: pgmap v508: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:49:45.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:44 smithi063 ceph-mon[31613]: pgmap v508: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:49:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.802+0000 7f6fbee00640 1 -- 172.21.15.63:0/3040913404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fb810beb0 msgr2=0x7f6fb8105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.802+0000 7f6fbee00640 1 --2- 172.21.15.63:0/3040913404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fb810beb0 0x7f6fb8105a50 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009900 tx=0x7f6fa802a410 comp rx=0 tx=0).stop 2024-05-27T07:49:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.802+0000 7f6fbee00640 1 -- 172.21.15.63:0/3040913404 shutdown_connections 2024-05-27T07:49:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.802+0000 7f6fbee00640 1 --2- 172.21.15.63:0/3040913404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb8106600 0x7f6fb81129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.802+0000 7f6fbee00640 1 --2- 172.21.15.63:0/3040913404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb810c350 0x7f6fb8105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.802+0000 7f6fbee00640 1 --2- 172.21.15.63:0/3040913404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fb810beb0 0x7f6fb8105a50 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.802+0000 7f6fbee00640 1 -- 172.21.15.63:0/3040913404 >> 172.21.15.63:0/3040913404 conn(0x7f6fb81016b0 msgr2=0x7f6fb8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.803+0000 7f6fbee00640 1 -- 172.21.15.63:0/3040913404 shutdown_connections 2024-05-27T07:49:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.803+0000 7f6fbee00640 1 -- 172.21.15.63:0/3040913404 wait complete. 2024-05-27T07:49:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.804+0000 7f6fbee00640 1 Processor -- start 2024-05-27T07:49:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.804+0000 7f6fbee00640 1 -- start start 2024-05-27T07:49:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.804+0000 7f6fbee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb8106600 0x7f6fb81a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.804+0000 7f6fbee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb810beb0 0x7f6fb81a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.804+0000 7f6fbee00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fb810c350 0x7f6fb819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.804+0000 7f6fbee00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb8115f60 con 0x7f6fb810c350 2024-05-27T07:49:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.804+0000 7f6fbee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb8115de0 con 0x7f6fb810beb0 2024-05-27T07:49:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.804+0000 7f6fbee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb81160e0 con 0x7f6fb8106600 2024-05-27T07:49:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fbce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fb810c350 0x7f6fb819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fbce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fb810c350 0x7f6fb819c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54878/0 (socket says 172.21.15.63:54878) 2024-05-27T07:49:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fbce00640 1 -- 172.21.15.63:0/3995208531 learned_addr learned my addr 172.21.15.63:0/3995208531 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fb7400640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb810beb0 0x7f6fb81a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb8106600 0x7f6fb81a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fbce00640 1 -- 172.21.15.63:0/3995208531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb8106600 msgr2=0x7f6fb81a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fbce00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb8106600 0x7f6fb81a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fbce00640 1 -- 172.21.15.63:0/3995208531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb810beb0 msgr2=0x7f6fb81a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fbce00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb810beb0 0x7f6fb81a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fbce00640 1 -- 172.21.15.63:0/3995208531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb819cf00 con 0x7f6fb810c350 2024-05-27T07:49:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.805+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb8106600 0x7f6fb81a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.806+0000 7f6fbce00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fb810c350 0x7f6fb819c820 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000eb10 tx=0x7f6fb000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.806+0000 7f6fb4c00640 1 -- 172.21.15.63:0/3995208531 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0004b20 con 0x7f6fb810c350 2024-05-27T07:49:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.806+0000 7f6fb4c00640 1 -- 172.21.15.63:0/3995208531 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fb0004cc0 con 0x7f6fb810c350 2024-05-27T07:49:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.806+0000 7f6fbee00640 1 -- 172.21.15.63:0/3995208531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb819d1f0 con 0x7f6fb810c350 2024-05-27T07:49:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.806+0000 7f6fb4c00640 1 -- 172.21.15.63:0/3995208531 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0010690 con 0x7f6fb810c350 2024-05-27T07:49:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.806+0000 7f6fbee00640 1 -- 172.21.15.63:0/3995208531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb81a9440 con 0x7f6fb810c350 2024-05-27T07:49:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.810+0000 7f6fc5e21740 1 Processor -- start 2024-05-27T07:49:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.811+0000 7f6fb4c00640 1 -- 172.21.15.63:0/3995208531 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6fb0005020 con 0x7f6fb810c350 2024-05-27T07:49:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.812+0000 7f6fc5e21740 1 -- start start 2024-05-27T07:49:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.812+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.812+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.812+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.812+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d3ce11fb90 con 0x55d3cddeb4d0 2024-05-27T07:49:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.812+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3ce10f730 con 0x55d3cde0c1b0 2024-05-27T07:49:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.812+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d3ce113770 con 0x55d3cdde32f0 2024-05-27T07:49:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.813+0000 7f6fb7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.813+0000 7f6fb7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54888/0 (socket says 172.21.15.63:54888) 2024-05-27T07:49:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.813+0000 7f6fb7400640 1 -- 172.21.15.63:0/50322392 learned_addr learned my addr 172.21.15.63:0/50322392 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.813+0000 7f6fb7400640 1 --2- 172.21.15.63:0/50322392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.813+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/50322392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.813+0000 7f6fb7400640 1 -- 172.21.15.63:0/50322392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 msgr2=0x55d3ce1aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.813+0000 7f6fb7400640 1 --2- 172.21.15.63:0/50322392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.813+0000 7f6fb7400640 1 -- 172.21.15.63:0/50322392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 msgr2=0x55d3ce1ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.813+0000 7f6fb7400640 1 --2- 172.21.15.63:0/50322392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.813+0000 7f6fb7400640 1 -- 172.21.15.63:0/50322392 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3ce0a88b0 con 0x55d3cddeb4d0 2024-05-27T07:49:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.814+0000 7f6fb7400640 1 --2- 172.21.15.63:0/50322392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1af2e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x55d3ce1ab9d0 tx=0x7f6fa400c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f09e8c96b077d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.814+0000 7f6fb4c00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f9c0b2950 0x7f6f9c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.814+0000 7f6fad600640 1 -- 172.21.15.63:0/50322392 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40142d0 con 0x55d3cddeb4d0 2024-05-27T07:49:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.814+0000 7f6fb4c00640 1 -- 172.21.15.63:0/3995208531 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f6fb00e30e0 con 0x7f6fb810c350 2024-05-27T07:49:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.814+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f9c0b2950 0x7f6f9c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.814+0000 7f6fad600640 1 -- 172.21.15.63:0/50322392 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fa40054b0 con 0x55d3cddeb4d0 2024-05-27T07:49:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.814+0000 7f6fad600640 1 -- 172.21.15.63:0/50322392 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4004d60 con 0x55d3cddeb4d0 2024-05-27T07:49:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.816+0000 7f6fc5e21740 1 -- 172.21.15.63:0/50322392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 msgr2=0x55d3ce1af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.816+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f9c0b2950 0x7f6f9c0b4e60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80057c0 tx=0x7f6fa8009560 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.816+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/50322392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1af2e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x55d3ce1ab9d0 tx=0x7f6fa400c3b0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.816+0000 7f6fc5e21740 1 -- 172.21.15.63:0/50322392 shutdown_connections 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.816+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/50322392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1af2e0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.816+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/50322392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.816+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/50322392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.816+0000 7f6fc5e21740 1 -- 172.21.15.63:0/50322392 >> 172.21.15.63:0/50322392 conn(0x55d3cde00300 msgr2=0x55d3ce1ab010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.816+0000 7f6fc5e21740 1 -- 172.21.15.63:0/50322392 shutdown_connections 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.816+0000 7f6fc5e21740 1 -- 172.21.15.63:0/50322392 wait complete. 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.817+0000 7f6fc5e21740 1 -- start start 2024-05-27T07:49:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.817+0000 7f6fc5e21740 1 Processor -- start 2024-05-27T07:49:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.817+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.817+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.817+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.817+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d3cddf5790 con 0x55d3cddeb4d0 2024-05-27T07:49:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3ce0b7bc0 con 0x55d3cde0c1b0 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d3ce1bb990 con 0x55d3cdde32f0 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fc5e21740 10 client.0 _ll_register_callbacks cb 0x55d3ce095400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fb7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fb7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1b9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54902/0 (socket says 172.21.15.63:54902) 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fb7400640 1 -- 172.21.15.63:0/1475649474 learned_addr learned my addr 172.21.15.63:0/1475649474 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fbce00640 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fb7400640 1 -- 172.21.15.63:0/1475649474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 msgr2=0x55d3ce1b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fb7400640 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fb7400640 1 -- 172.21.15.63:0/1475649474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 msgr2=0x55d3ce1c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fb7400640 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fb7400640 1 -- 172.21.15.63:0/1475649474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3cde24210 con 0x55d3cddeb4d0 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.818+0000 7f6fbce00640 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.819+0000 7f6fb7400640 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1b9200 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4014f50 tx=0x7f6fa400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88b575225fa466e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.819+0000 7f6fa2000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.819+0000 7f6fa2000640 1 -- 172.21.15.63:0/1475649474 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4005220 con 0x55d3cddeb4d0 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.819+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1475649474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa40054b0 con 0x55d3cddeb4d0 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.819+0000 7f6fa2000640 1 -- 172.21.15.63:0/1475649474 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f6fa4004ae0 con 0x55d3cddeb4d0 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.819+0000 7f6fa2000640 1 -- 172.21.15.63:0/1475649474 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6fa4031050 con 0x55d3cddeb4d0 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.819+0000 7f6fc5e21740 10 client.18528 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.819+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1475649474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d3cdef0620 con 0x55d3cddeb4d0 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.820+0000 7f6fa2000640 1 -- 172.21.15.63:0/1475649474 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fa4025020 con 0x55d3cddeb4d0 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.820+0000 7f6fa2000640 1 -- 172.21.15.63:0/1475649474 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6fa40124e0 con 0x55d3cddeb4d0 2024-05-27T07:49:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.820+0000 7f6fc5e21740 10 client.18528 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.820+0000 7f6fc5e21740 10 client.18528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.820+0000 7f6fc5e21740 20 client.18528 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.820+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3cddcad80 0x55d3ce1cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.820+0000 7f6fc5e21740 4 client.18528 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.820+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1475649474 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3ce121ae0 con 0x55d3cddcad80 2024-05-27T07:49:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.821+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3cddcad80 0x55d3ce1cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.822+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3cddcad80 0x55d3ce1cac10 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7182fbccd6e4ca2 server_cookie=f2f5b751f7d9bf0e in_seq=0 out_seq=0 2024-05-27T07:49:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.822+0000 7f6fa2000640 10 client.18528 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fa2000640 1 -- 172.21.15.63:0/1475649474 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d3ce121ae0 con 0x55d3cddcad80 2024-05-27T07:49:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fa2000640 10 client.18528 handle_command_reply: tid=0 2024-05-27T07:49:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fc5e21740 1 client.18528 shutdown 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1475649474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 msgr2=0x55d3ce1b9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1b9200 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4014f50 tx=0x7f6fa400cd70 comp rx=0 tx=0).stop 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1475649474 shutdown_connections 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3cddcad80 0x55d3ce1cac10 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1b9200 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/1475649474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1475649474 >> 172.21.15.63:0/1475649474 conn(0x55d3cde00300 msgr2=0x55d3ce1b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.824+0000 7f6fa2000640 0 client.18528 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.825+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1475649474 shutdown_connections 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.825+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1475649474 wait complete. 2024-05-27T07:49:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.825+0000 7f6fc5e21740 20 client.18528 trim_cache size 0 max 16384 2024-05-27T07:49:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.842+0000 7f6fc5e21740 1 Processor -- start 2024-05-27T07:49:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.842+0000 7f6fc5e21740 1 -- start start 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.842+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cdde32f0 0x55d3ce1c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.842+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.842+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.842+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb81b5d50 con 0x55d3cdde32f0 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.842+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb81b3620 con 0x55d3cde0c1b0 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.842+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb81b1490 con 0x55d3cddeb4d0 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.843+0000 7f6fb7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cdde32f0 0x55d3ce1c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.843+0000 7f6fb7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cdde32f0 0x55d3ce1c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54908/0 (socket says 172.21.15.63:54908) 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.843+0000 7f6fb7e00640 1 -- 172.21.15.63:0/1487450438 learned_addr learned my addr 172.21.15.63:0/1487450438 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.843+0000 7f6fbce00640 1 --2- 172.21.15.63:0/1487450438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.843+0000 7f6fb7400640 1 --2- 172.21.15.63:0/1487450438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.843+0000 7f6fb7e00640 1 -- 172.21.15.63:0/1487450438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cddeb4d0 msgr2=0x55d3ce1aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.843+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/1487450438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.843+0000 7f6fb7e00640 1 -- 172.21.15.63:0/1487450438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 msgr2=0x55d3ce1af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.843+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/1487450438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.843+0000 7f6fb7e00640 1 -- 172.21.15.63:0/1487450438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3ce1af910 con 0x55d3cdde32f0 2024-05-27T07:49:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.844+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/1487450438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cdde32f0 0x55d3ce1c4c30 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8106b90 tx=0x7f6fa803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462cc14dafdbd77b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.844+0000 7f6facc00640 1 -- 172.21.15.63:0/1487450438 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8055070 con 0x55d3cdde32f0 2024-05-27T07:49:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.844+0000 7f6facc00640 1 -- 172.21.15.63:0/1487450438 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fa80044d0 con 0x55d3cdde32f0 2024-05-27T07:49:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.844+0000 7f6facc00640 1 -- 172.21.15.63:0/1487450438 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8035040 con 0x55d3cdde32f0 2024-05-27T07:49:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.845+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1487450438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cdde32f0 msgr2=0x55d3ce1c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.845+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/1487450438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cdde32f0 0x55d3ce1c4c30 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8106b90 tx=0x7f6fa803e000 comp rx=0 tx=0).stop 2024-05-27T07:49:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.845+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1487450438 shutdown_connections 2024-05-27T07:49:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.845+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/1487450438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.845+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/1487450438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.845+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/1487450438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cdde32f0 0x55d3ce1c4c30 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.845+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1487450438 >> 172.21.15.63:0/1487450438 conn(0x55d3cde00300 msgr2=0x55d3ce1ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.845+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1487450438 shutdown_connections 2024-05-27T07:49:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.845+0000 7f6fc5e21740 1 -- 172.21.15.63:0/1487450438 wait complete. 2024-05-27T07:49:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.846+0000 7f6fc5e21740 1 -- start start 2024-05-27T07:49:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.846+0000 7f6fc5e21740 1 Processor -- start 2024-05-27T07:49:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fc5e21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb81b1ae0 con 0x55d3cddeb4d0 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb81b2f50 con 0x55d3cde0c1b0 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fc5e21740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb81b21b0 con 0x55d3cdde32f0 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fc5e21740 10 client.0 _ll_register_callbacks cb 0x7f6fa400cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fb7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fb7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54916/0 (socket says 172.21.15.63:54916) 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fb7400640 1 -- 172.21.15.63:0/4172394624 learned_addr learned my addr 172.21.15.63:0/4172394624 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.847+0000 7f6fbce00640 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fb7400640 1 -- 172.21.15.63:0/4172394624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 msgr2=0x55d3ce1cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fb7400640 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fb7400640 1 -- 172.21.15.63:0/4172394624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 msgr2=0x55d3ce1cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fb7400640 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fb7400640 1 -- 172.21.15.63:0/4172394624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3ce231280 con 0x55d3cddeb4d0 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fbce00640 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fb7400640 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1cb1a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4014790 tx=0x7f6fa40147c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b8f72dc1f7cd89 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fa2000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fa2000640 1 -- 172.21.15.63:0/4172394624 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa403a060 con 0x55d3cddeb4d0 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fa2000640 1 -- 172.21.15.63:0/4172394624 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f6fa4012450 con 0x55d3cddeb4d0 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.848+0000 7f6fc5e21740 1 -- 172.21.15.63:0/4172394624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa80044d0 con 0x55d3cddeb4d0 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.849+0000 7f6fa2000640 1 -- 172.21.15.63:0/4172394624 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6fa4041050 con 0x55d3cddeb4d0 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.849+0000 7f6fc5e21740 10 client.18534 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.849+0000 7f6fc5e21740 1 -- 172.21.15.63:0/4172394624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d3ce232510 con 0x55d3cddeb4d0 2024-05-27T07:49:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.849+0000 7f6fa2000640 1 -- 172.21.15.63:0/4172394624 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fa80044d0 con 0x55d3cddeb4d0 2024-05-27T07:49:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.849+0000 7f6fa2000640 1 -- 172.21.15.63:0/4172394624 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6fa4034040 con 0x55d3cddeb4d0 2024-05-27T07:49:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.850+0000 7f6fc5e21740 10 client.18534 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.850+0000 7f6fc5e21740 10 client.18534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.850+0000 7f6fc5e21740 20 client.18534 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.850+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3cdda01c0 0x55d3ce234be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.850+0000 7f6fc5e21740 4 client.18534 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:49:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.850+0000 7f6fc5e21740 1 -- 172.21.15.63:0/4172394624 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3ce11b2c0 con 0x55d3cdda01c0 2024-05-27T07:49:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.850+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3cdda01c0 0x55d3ce234be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.851+0000 7f6fb7e00640 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3cdda01c0 0x55d3ce234be0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ac0a1ccee8ccb9 server_cookie=1e070eb7ca92c9a4 in_seq=0 out_seq=0 2024-05-27T07:49:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.851+0000 7f6fa2000640 10 client.18534 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.879+0000 7f6fa2000640 1 -- 172.21.15.63:0/4172394624 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d3ce11b2c0 con 0x55d3cdda01c0 2024-05-27T07:49:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.879+0000 7f6fa2000640 10 client.18534 handle_command_reply: tid=0 2024-05-27T07:49:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.879+0000 7f6fc5e21740 1 client.18534 shutdown 2024-05-27T07:49:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 1 -- 172.21.15.63:0/4172394624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 msgr2=0x55d3ce1cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1cb1a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4014790 tx=0x7f6fa40147c0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 1 -- 172.21.15.63:0/4172394624 shutdown_connections 2024-05-27T07:49:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3cdda01c0 0x55d3ce234be0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3cde0c1b0 0x55d3ce1cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3cddeb4d0 0x55d3ce1cb1a0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 1 --2- 172.21.15.63:0/4172394624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3cdde32f0 0x55d3ce1cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fa2000640 0 client.18534 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 1 -- 172.21.15.63:0/4172394624 >> 172.21.15.63:0/4172394624 conn(0x55d3cde00300 msgr2=0x55d3ce1af160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 1 -- 172.21.15.63:0/4172394624 shutdown_connections 2024-05-27T07:49:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 1 -- 172.21.15.63:0/4172394624 wait complete. 2024-05-27T07:49:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.880+0000 7f6fc5e21740 20 client.18534 trim_cache size 0 max 16384 2024-05-27T07:49:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:49:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:49:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:49:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.881+0000 7f6fbee00640 1 -- 172.21.15.63:0/3995208531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f9c0b2950 msgr2=0x7f6f9c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.881+0000 7f6fbee00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f9c0b2950 0x7f6f9c0b4e60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80057c0 tx=0x7f6fa8009560 comp rx=0 tx=0).stop 2024-05-27T07:49:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.881+0000 7f6fbee00640 1 -- 172.21.15.63:0/3995208531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fb810c350 msgr2=0x7f6fb819c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.881+0000 7f6fbee00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fb810c350 0x7f6fb819c820 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000eb10 tx=0x7f6fb000c2d0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.881+0000 7f6fbee00640 1 -- 172.21.15.63:0/3995208531 shutdown_connections 2024-05-27T07:49:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.881+0000 7f6fbee00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f9c0b2950 0x7f6f9c0b4e60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.881+0000 7f6fbee00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fb810c350 0x7f6fb819c820 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.881+0000 7f6fbee00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb810beb0 0x7f6fb81a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.881+0000 7f6fbee00640 1 --2- 172.21.15.63:0/3995208531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fb8106600 0x7f6fb81a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.882+0000 7f6fbee00640 1 -- 172.21.15.63:0/3995208531 >> 172.21.15.63:0/3995208531 conn(0x7f6fb81016b0 msgr2=0x7f6fb8101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.882+0000 7f6fbee00640 1 -- 172.21.15.63:0/3995208531 shutdown_connections 2024-05-27T07:49:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:45.882+0000 7f6fbee00640 1 -- 172.21.15.63:0/3995208531 wait complete. 2024-05-27T07:49:46.100 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:49:46.607 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:47.031 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:46 smithi063 ceph-mon[31613]: pgmap v509: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:49:47.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:46 smithi086 ceph-mon[34687]: pgmap v509: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:49:47.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:46 smithi149 ceph-mon[35408]: pgmap v509: 129 pgs: 129 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:49:48.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:48 smithi063 ceph-mon[31613]: pgmap v510: 129 pgs: 129 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 17 op/s 2024-05-27T07:49:49.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.111+0000 7fc9b1200700 1 -- 172.21.15.63:0/4047139397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9ac1045e0 msgr2=0x7fc9ac1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:49.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.111+0000 7fc9b1200700 1 --2- 172.21.15.63:0/4047139397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9ac1045e0 0x7fc9ac1049f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0099c0 tx=0x7fc99c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.111+0000 7fc9b1200700 1 -- 172.21.15.63:0/4047139397 shutdown_connections 2024-05-27T07:49:49.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.111+0000 7fc9b1200700 1 --2- 172.21.15.63:0/4047139397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac109860 0x7fc9ac1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.111+0000 7fc9b1200700 1 --2- 172.21.15.63:0/4047139397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9ac1045e0 0x7fc9ac1049f0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.111+0000 7fc9b1200700 1 --2- 172.21.15.63:0/4047139397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac108f10 0x7fc9ac109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.111+0000 7fc9b1200700 1 -- 172.21.15.63:0/4047139397 >> 172.21.15.63:0/4047139397 conn(0x7fc9ac100030 msgr2=0x7fc9ac102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:49.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.111+0000 7fc9b1200700 1 -- 172.21.15.63:0/4047139397 shutdown_connections 2024-05-27T07:49:49.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.111+0000 7fc9b1200700 1 -- 172.21.15.63:0/4047139397 wait complete. 2024-05-27T07:49:49.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.113+0000 7fc9b1200700 1 Processor -- start 2024-05-27T07:49:49.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.113+0000 7fc9b1200700 1 -- start start 2024-05-27T07:49:49.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.114+0000 7fc9b1200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac1045e0 0x7fc9ac19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.114+0000 7fc9b1200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac108f10 0x7fc9ac19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.114+0000 7fc9b1200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9ac109860 0x7fc9ac1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.114+0000 7fc9b1200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac114390 con 0x7fc9ac109860 2024-05-27T07:49:49.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.114+0000 7fc9b1200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac114210 con 0x7fc9ac1045e0 2024-05-27T07:49:49.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.114+0000 7fc9b1200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac114510 con 0x7fc9ac108f10 2024-05-27T07:49:49.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.114+0000 7fc9aa800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9ac109860 0x7fc9ac1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.114+0000 7fc9aa800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9ac109860 0x7fc9ac1a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33760/0 (socket says 172.21.15.63:33760) 2024-05-27T07:49:49.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.114+0000 7fc9aa800700 1 -- 172.21.15.63:0/3659994305 learned_addr learned my addr 172.21.15.63:0/3659994305 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:49.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.114+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac1045e0 0x7fc9ac19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.115+0000 7fc9a9400700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac108f10 0x7fc9ac19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.115+0000 7fc9aa800700 1 -- 172.21.15.63:0/3659994305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac108f10 msgr2=0x7fc9ac19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:49.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.115+0000 7fc9aa800700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac108f10 0x7fc9ac19d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.115+0000 7fc9aa800700 1 -- 172.21.15.63:0/3659994305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac1045e0 msgr2=0x7fc9ac19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:49.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.115+0000 7fc9aa800700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac1045e0 0x7fc9ac19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.115+0000 7fc9aa800700 1 -- 172.21.15.63:0/3659994305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac1a1ed0 con 0x7fc9ac109860 2024-05-27T07:49:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.115+0000 7fc9a9400700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac108f10 0x7fc9ac19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.116+0000 7fc9aa800700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9ac109860 0x7fc9ac1a1830 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4007ee0 tx=0x7fc9a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.116+0000 7fc9a2a00700 1 -- 172.21.15.63:0/3659994305 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a400dbd0 con 0x7fc9ac109860 2024-05-27T07:49:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.116+0000 7fc9b1200700 1 -- 172.21.15.63:0/3659994305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac1a21c0 con 0x7fc9ac109860 2024-05-27T07:49:49.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.116+0000 7fc9b1200700 1 -- 172.21.15.63:0/3659994305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac1a9ae0 con 0x7fc9ac109860 2024-05-27T07:49:49.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.116+0000 7fc9a2a00700 1 -- 172.21.15.63:0/3659994305 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9a400dd70 con 0x7fc9ac109860 2024-05-27T07:49:49.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.116+0000 7fc9a2a00700 1 -- 172.21.15.63:0/3659994305 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40058d0 con 0x7fc9ac109860 2024-05-27T07:49:49.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.121+0000 7fc9a2a00700 1 -- 172.21.15.63:0/3659994305 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc9a401a020 con 0x7fc9ac109860 2024-05-27T07:49:49.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.122+0000 7fc9bfc7db80 1 Processor -- start 2024-05-27T07:49:49.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.122+0000 7fc9bfc7db80 1 -- start start 2024-05-27T07:49:49.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.122+0000 7fc9a2a00700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9900b2a70 0x7fc9900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.123+0000 7fc9a2a00700 1 -- 172.21.15.63:0/3659994305 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fc9a40e1fc0 con 0x7fc9ac109860 2024-05-27T07:49:49.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.123+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 0x55e93f43b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.123+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 0x55e93f435870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.123+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f4361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.123+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e93f363e40 con 0x55e93f43bd30 2024-05-27T07:49:49.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.123+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e93f3539e0 con 0x55e93f2b6e40 2024-05-27T07:49:49.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.123+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e93f357a20 con 0x55e93f435db0 2024-05-27T07:49:49.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.123+0000 7fc9a9400700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9900b2a70 0x7fc9900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9a9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 0x55e93f43b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9a9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 0x55e93f435870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9a9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 0x55e93f43b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34022/0 (socket says 172.21.15.63:34022) 2024-05-27T07:49:49.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9a9e00700 1 -- 172.21.15.63:0/829430539 learned_addr learned my addr 172.21.15.63:0/829430539 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:49.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9aa800700 1 --2- 172.21.15.63:0/829430539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f4361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9a9e00700 1 -- 172.21.15.63:0/829430539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 msgr2=0x55e93f4361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:49.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/829430539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f4361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9a9e00700 1 -- 172.21.15.63:0/829430539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 msgr2=0x55e93f435870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:49.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/829430539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 0x55e93f435870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9a9e00700 1 -- 172.21.15.63:0/829430539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93f31c420 con 0x55e93f2b6e40 2024-05-27T07:49:49.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.124+0000 7fc9a9400700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9900b2a70 0x7fc9900b4f80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc998005950 tx=0x7fc998013980 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:49.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.125+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/829430539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 0x55e93f43b7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0058e0 tx=0x7fc99c005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64dd2494d1a64021 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:49.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.125+0000 7fc997400700 1 -- 172.21.15.63:0/829430539 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c027070 con 0x55e93f2b6e40 2024-05-27T07:49:49.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.125+0000 7fc997400700 1 -- 172.21.15.63:0/829430539 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc99c00b790 con 0x55e93f2b6e40 2024-05-27T07:49:49.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.126+0000 7fc997400700 1 -- 172.21.15.63:0/829430539 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0038e0 con 0x55e93f2b6e40 2024-05-27T07:49:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.127+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/829430539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 msgr2=0x55e93f43b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.127+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/829430539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 0x55e93f43b7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0058e0 tx=0x7fc99c005ea0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.127+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/829430539 shutdown_connections 2024-05-27T07:49:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.127+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/829430539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f4361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.127+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/829430539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 0x55e93f43b7f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.127+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/829430539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 0x55e93f435870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.127+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/829430539 >> 172.21.15.63:0/829430539 conn(0x55e93f316fc0 msgr2=0x55e93f4308d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.127+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/829430539 shutdown_connections 2024-05-27T07:49:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.127+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/829430539 wait complete. 2024-05-27T07:49:49.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.128+0000 7fc9bfc7db80 1 -- start start 2024-05-27T07:49:49.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.128+0000 7fc9bfc7db80 1 Processor -- start 2024-05-27T07:49:49.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 0x55e93f43f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f43f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 0x55e93f43ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e93f259fa0 con 0x55e93f43bd30 2024-05-27T07:49:49.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e93f2f4760 con 0x55e93f2b6e40 2024-05-27T07:49:49.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e93f4427a0 con 0x55e93f435db0 2024-05-27T07:49:49.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9bfc7db80 10 client.0 _ll_register_callbacks cb 0x55e93f2fdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9aa800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 0x55e93f43ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9aa800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 0x55e93f43ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33790/0 (socket says 172.21.15.63:33790) 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9aa800700 1 -- 172.21.15.63:0/2942606258 learned_addr learned my addr 172.21.15.63:0/2942606258 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 0x55e93f43f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.129+0000 7fc9a9400700 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f43f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.130+0000 7fc9aa800700 1 -- 172.21.15.63:0/2942606258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 msgr2=0x55e93f43f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.130+0000 7fc9aa800700 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f43f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.130+0000 7fc9aa800700 1 -- 172.21.15.63:0/2942606258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 msgr2=0x55e93f43f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.130+0000 7fc9aa800700 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 0x55e93f43f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.130+0000 7fc9aa800700 1 -- 172.21.15.63:0/2942606258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93f2a38d0 con 0x55e93f43bd30 2024-05-27T07:49:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.130+0000 7fc9a9400700 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f43f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:49.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.130+0000 7fc9aa800700 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 0x55e93f43ff10 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40df950 tx=0x7fc9a40a9370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8542d6f6b9549afd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:49.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.130+0000 7fc98be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:49.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.131+0000 7fc98be00700 1 -- 172.21.15.63:0/2942606258 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40a95e0 con 0x55e93f43bd30 2024-05-27T07:49:49.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.131+0000 7fc98be00700 1 -- 172.21.15.63:0/2942606258 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fc9a4004ce0 con 0x55e93f43bd30 2024-05-27T07:49:49.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.131+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2942606258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc99c00b790 con 0x55e93f43bd30 2024-05-27T07:49:49.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.131+0000 7fc98be00700 1 -- 172.21.15.63:0/2942606258 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc9a40a8ee0 con 0x55e93f43bd30 2024-05-27T07:49:49.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.131+0000 7fc9bfc7db80 10 client.18546 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:49.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.131+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2942606258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e93f324060 con 0x55e93f43bd30 2024-05-27T07:49:49.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.131+0000 7fc98be00700 1 -- 172.21.15.63:0/2942606258 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc99c00b790 con 0x55e93f43bd30 2024-05-27T07:49:49.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.132+0000 7fc98be00700 1 -- 172.21.15.63:0/2942606258 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc9a40a4850 con 0x55e93f43bd30 2024-05-27T07:49:49.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.132+0000 7fc9bfc7db80 10 client.18546 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:49.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.132+0000 7fc9bfc7db80 10 client.18546 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:49:49.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.132+0000 7fc9bfc7db80 20 client.18546 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:49.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.132+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e93f44f730 0x55e93f451b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.132+0000 7fc9bfc7db80 4 client.18546 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:49.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.132+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2942606258 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e93f365d90 con 0x55e93f44f730 2024-05-27T07:49:49.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.133+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e93f44f730 0x55e93f451b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.134+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e93f44f730 0x55e93f451b50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=938cf2b846f4f5ba server_cookie=d2fc227d028b580a in_seq=0 out_seq=0 2024-05-27T07:49:49.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.134+0000 7fc98be00700 10 client.18546 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:49:49.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc98be00700 1 -- 172.21.15.63:0/2942606258 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e93f365d90 con 0x55e93f44f730 2024-05-27T07:49:49.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc98be00700 10 client.18546 handle_command_reply: tid=0 2024-05-27T07:49:49.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc9bfc7db80 1 client.18546 shutdown 2024-05-27T07:49:49.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2942606258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 msgr2=0x55e93f43ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:49.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 0x55e93f43ff10 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40df950 tx=0x7fc9a40a9370 comp rx=0 tx=0).stop 2024-05-27T07:49:49.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2942606258 shutdown_connections 2024-05-27T07:49:49.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e93f44f730 0x55e93f451b50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f43bd30 0x55e93f43ff10 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f2b6e40 0x55e93f43f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2942606258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f43f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2942606258 >> 172.21.15.63:0/2942606258 conn(0x55e93f316fc0 msgr2=0x55e93f430580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:49.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.136+0000 7fc98be00700 0 client.18546 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:49:49.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.137+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2942606258 shutdown_connections 2024-05-27T07:49:49.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.137+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2942606258 wait complete. 2024-05-27T07:49:49.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.137+0000 7fc9bfc7db80 20 client.18546 trim_cache size 0 max 16384 2024-05-27T07:49:49.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.159+0000 7fc9bfc7db80 1 Processor -- start 2024-05-27T07:49:49.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.159+0000 7fc9bfc7db80 1 -- start start 2024-05-27T07:49:49.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.159+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f435870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.159+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f439c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.159+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f43bd30 0x55e93f44b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.160+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c011640 con 0x55e93f2b6e40 2024-05-27T07:49:49.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.160+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c00eed0 con 0x55e93f43bd30 2024-05-27T07:49:49.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.160+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c00cd40 con 0x55e93f435db0 2024-05-27T07:49:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.160+0000 7fc9a9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f435870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.160+0000 7fc9a9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f435870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33806/0 (socket says 172.21.15.63:33806) 2024-05-27T07:49:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.160+0000 7fc9a9e00700 1 -- 172.21.15.63:0/864136916 learned_addr learned my addr 172.21.15.63:0/864136916 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.160+0000 7fc9a9400700 1 --2- 172.21.15.63:0/864136916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f439c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.160+0000 7fc9aa800700 1 --2- 172.21.15.63:0/864136916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f43bd30 0x55e93f44b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.160+0000 7fc9a9e00700 1 -- 172.21.15.63:0/864136916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 msgr2=0x55e93f439c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.160+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/864136916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f439c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.161+0000 7fc9a9e00700 1 -- 172.21.15.63:0/864136916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f43bd30 msgr2=0x55e93f44b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.161+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/864136916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f43bd30 0x55e93f44b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.161+0000 7fc9a9e00700 1 -- 172.21.15.63:0/864136916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93f44d150 con 0x55e93f2b6e40 2024-05-27T07:49:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.161+0000 7fc9a9400700 1 --2- 172.21.15.63:0/864136916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f439c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.161+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/864136916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f435870 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00b580 tx=0x7fc99c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8876c1fe3a5945 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.161+0000 7fc996a00700 1 -- 172.21.15.63:0/864136916 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c035070 con 0x55e93f2b6e40 2024-05-27T07:49:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.161+0000 7fc996a00700 1 -- 172.21.15.63:0/864136916 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc99c028040 con 0x55e93f2b6e40 2024-05-27T07:49:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.161+0000 7fc996a00700 1 -- 172.21.15.63:0/864136916 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c027040 con 0x55e93f2b6e40 2024-05-27T07:49:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.162+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/864136916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 msgr2=0x55e93f435870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.162+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/864136916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f435870 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00b580 tx=0x7fc99c005f10 comp rx=0 tx=0).stop 2024-05-27T07:49:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.163+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/864136916 shutdown_connections 2024-05-27T07:49:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.163+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/864136916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f43bd30 0x55e93f44b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.163+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/864136916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f435870 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.163+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/864136916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f435db0 0x55e93f439c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.163+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/864136916 >> 172.21.15.63:0/864136916 conn(0x55e93f316fc0 msgr2=0x55e93f43ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.163+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/864136916 shutdown_connections 2024-05-27T07:49:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.163+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/864136916 wait complete. 2024-05-27T07:49:49.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.163+0000 7fc9bfc7db80 1 -- start start 2024-05-27T07:49:49.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.164+0000 7fc9bfc7db80 1 Processor -- start 2024-05-27T07:49:49.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.164+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f456d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.164+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f435db0 0x55e93f457290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.164+0000 7fc9bfc7db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f43bd30 0x55e93f4577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.164+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c00d390 con 0x55e93f2b6e40 2024-05-27T07:49:49.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.164+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c00e800 con 0x55e93f435db0 2024-05-27T07:49:49.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.164+0000 7fc9bfc7db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c00da60 con 0x55e93f43bd30 2024-05-27T07:49:49.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.164+0000 7fc9bfc7db80 10 client.0 _ll_register_callbacks cb 0x55e93f2fdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:49.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9a9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f456d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9a9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f456d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33818/0 (socket says 172.21.15.63:33818) 2024-05-27T07:49:49.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9a9e00700 1 -- 172.21.15.63:0/2746092873 learned_addr learned my addr 172.21.15.63:0/2746092873 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:49.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9a9400700 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f435db0 0x55e93f457290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9aa800700 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f43bd30 0x55e93f4577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9a9e00700 1 -- 172.21.15.63:0/2746092873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f43bd30 msgr2=0x55e93f4577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:49.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f43bd30 0x55e93f4577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9a9e00700 1 -- 172.21.15.63:0/2746092873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f435db0 msgr2=0x55e93f457290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:49.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f435db0 0x55e93f457290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9a9e00700 1 -- 172.21.15.63:0/2746092873 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93f440280 con 0x55e93f2b6e40 2024-05-27T07:49:49.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.165+0000 7fc9a9400700 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f435db0 0x55e93f457290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:49.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.166+0000 7fc9a9e00700 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f456d50 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009350 tx=0x7fc99c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e648ba7bc89c494 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:49.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.166+0000 7fc98be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:49.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.166+0000 7fc98be00700 1 -- 172.21.15.63:0/2746092873 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c03f060 con 0x55e93f2b6e40 2024-05-27T07:49:49.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.166+0000 7fc98be00700 1 -- 172.21.15.63:0/2746092873 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fc99c005300 con 0x55e93f2b6e40 2024-05-27T07:49:49.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.166+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2746092873 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc99c028040 con 0x55e93f2b6e40 2024-05-27T07:49:49.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.167+0000 7fc98be00700 1 -- 172.21.15.63:0/2746092873 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc99c027070 con 0x55e93f2b6e40 2024-05-27T07:49:49.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.167+0000 7fc9bfc7db80 10 client.18552 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:49.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.167+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2746092873 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e93f4ba070 con 0x55e93f2b6e40 2024-05-27T07:49:49.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.167+0000 7fc98be00700 1 -- 172.21.15.63:0/2746092873 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc99c028040 con 0x55e93f2b6e40 2024-05-27T07:49:49.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.168+0000 7fc98be00700 1 -- 172.21.15.63:0/2746092873 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc99c022610 con 0x55e93f2b6e40 2024-05-27T07:49:49.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.168+0000 7fc9bfc7db80 10 client.18552 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:49.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.168+0000 7fc9bfc7db80 10 client.18552 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:49:49.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.168+0000 7fc9bfc7db80 20 client.18552 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:49.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.168+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e93f4ba2f0 0x55e93f4bc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:49.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.168+0000 7fc9bfc7db80 4 client.18552 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:49:49.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.168+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2746092873 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e93f35f570 con 0x55e93f4ba2f0 2024-05-27T07:49:49.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.169+0000 7fc9a9400700 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e93f4ba2f0 0x55e93f4bc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:49.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.170+0000 7fc9a9400700 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e93f4ba2f0 0x55e93f4bc710 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4dfc8d52c78c979 server_cookie=494273742e4b0d46 in_seq=0 out_seq=0 2024-05-27T07:49:49.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.170+0000 7fc98be00700 10 client.18552 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:49:49.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.170+0000 7fc98be00700 1 -- 172.21.15.63:0/2746092873 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e93f35f570 con 0x55e93f4ba2f0 2024-05-27T07:49:49.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.170+0000 7fc98be00700 10 client.18552 handle_command_reply: tid=0 2024-05-27T07:49:49.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.170+0000 7fc9bfc7db80 1 client.18552 shutdown 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.170+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2746092873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 msgr2=0x55e93f456d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.170+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f456d50 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009350 tx=0x7fc99c00b840 comp rx=0 tx=0).stop 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.171+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2746092873 shutdown_connections 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.171+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e93f43bd30 0x55e93f4577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.171+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e93f4ba2f0 0x55e93f4bc710 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.171+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e93f2b6e40 0x55e93f456d50 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.171+0000 7fc9bfc7db80 1 --2- 172.21.15.63:0/2746092873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e93f435db0 0x55e93f457290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.171+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2746092873 >> 172.21.15.63:0/2746092873 conn(0x55e93f316fc0 msgr2=0x55e93f43f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.171+0000 7fc98be00700 0 client.18552 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.171+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2746092873 shutdown_connections 2024-05-27T07:49:49.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.171+0000 7fc9bfc7db80 1 -- 172.21.15.63:0/2746092873 wait complete. 2024-05-27T07:49:49.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.171+0000 7fc9bfc7db80 20 client.18552 trim_cache size 0 max 16384 2024-05-27T07:49:49.173 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:49:49.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.172+0000 7fc9b1200700 1 -- 172.21.15.63:0/3659994305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9900b2a70 msgr2=0x7fc9900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:49.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.172+0000 7fc9b1200700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9900b2a70 0x7fc9900b4f80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc998005950 tx=0x7fc998013980 comp rx=0 tx=0).stop 2024-05-27T07:49:49.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.172+0000 7fc9b1200700 1 -- 172.21.15.63:0/3659994305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9ac109860 msgr2=0x7fc9ac1a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:49.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.172+0000 7fc9b1200700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9ac109860 0x7fc9ac1a1830 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4007ee0 tx=0x7fc9a400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.172+0000 7fc9b1200700 1 -- 172.21.15.63:0/3659994305 shutdown_connections 2024-05-27T07:49:49.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.172+0000 7fc9b1200700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9900b2a70 0x7fc9900b4f80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.172+0000 7fc9b1200700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9ac109860 0x7fc9ac1a1830 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.172+0000 7fc9b1200700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9ac1045e0 0x7fc9ac19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.172+0000 7fc9b1200700 1 --2- 172.21.15.63:0/3659994305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9ac108f10 0x7fc9ac19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:49.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.172+0000 7fc9b1200700 1 -- 172.21.15.63:0/3659994305 >> 172.21.15.63:0/3659994305 conn(0x7fc9ac100030 msgr2=0x7fc9ac10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:49.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.173+0000 7fc9b1200700 1 -- 172.21.15.63:0/3659994305 shutdown_connections 2024-05-27T07:49:49.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:49.173+0000 7fc9b1200700 1 -- 172.21.15.63:0/3659994305 wait complete. 2024-05-27T07:49:49.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:48 smithi086 ceph-mon[34687]: pgmap v510: 129 pgs: 129 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 17 op/s 2024-05-27T07:49:49.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:48 smithi149 ceph-mon[35408]: pgmap v510: 129 pgs: 129 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 17 op/s 2024-05-27T07:49:50.155 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:49:50.665 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:51.089 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:50 smithi063 ceph-mon[31613]: pgmap v511: 129 pgs: 129 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:49:51.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:50 smithi086 ceph-mon[34687]: pgmap v511: 129 pgs: 129 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:49:51.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:50 smithi149 ceph-mon[35408]: pgmap v511: 129 pgs: 129 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:49:53.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.148+0000 7f5d17a00700 1 -- 172.21.15.63:0/1098052629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d101034b0 msgr2=0x7f5d101038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:53.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.148+0000 7f5d17a00700 1 --2- 172.21.15.63:0/1098052629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d101034b0 0x7f5d101038c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5d040099c0 tx=0x7f5d04009cd0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.149+0000 7f5d17a00700 1 -- 172.21.15.63:0/1098052629 shutdown_connections 2024-05-27T07:49:53.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.149+0000 7f5d17a00700 1 --2- 172.21.15.63:0/1098052629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10108730 0x7f5d10110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.149+0000 7f5d17a00700 1 --2- 172.21.15.63:0/1098052629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d101034b0 0x7f5d101038c0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.149+0000 7f5d17a00700 1 --2- 172.21.15.63:0/1098052629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d10107de0 0x7f5d101081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.149+0000 7f5d17a00700 1 -- 172.21.15.63:0/1098052629 >> 172.21.15.63:0/1098052629 conn(0x7f5d100fedb0 msgr2=0x7f5d101011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:53.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.149+0000 7f5d17a00700 1 -- 172.21.15.63:0/1098052629 shutdown_connections 2024-05-27T07:49:53.151 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:52 smithi063 ceph-mon[31613]: pgmap v512: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:49:53.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.149+0000 7f5d17a00700 1 -- 172.21.15.63:0/1098052629 wait complete. 2024-05-27T07:49:53.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.151+0000 7f5d17a00700 1 Processor -- start 2024-05-27T07:49:53.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.151+0000 7f5d17a00700 1 -- start start 2024-05-27T07:49:53.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.151+0000 7f5d17a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d101034b0 0x7f5d1019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.151+0000 7f5d17a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10107de0 0x7f5d1019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.151+0000 7f5d17a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d10108730 0x7f5d101a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.151+0000 7f5d17a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10113260 con 0x7f5d101034b0 2024-05-27T07:49:53.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.151+0000 7f5d17a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101130e0 con 0x7f5d10107de0 2024-05-27T07:49:53.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.151+0000 7f5d17a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101133e0 con 0x7f5d10108730 2024-05-27T07:49:53.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d15000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d101034b0 0x7f5d1019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d15000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d101034b0 0x7f5d1019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33836/0 (socket says 172.21.15.63:33836) 2024-05-27T07:49:53.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d15000700 1 -- 172.21.15.63:0/3197380134 learned_addr learned my addr 172.21.15.63:0/3197380134 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:53.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10107de0 0x7f5d1019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d15a00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d10108730 0x7f5d101a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d15000700 1 -- 172.21.15.63:0/3197380134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d10108730 msgr2=0x7f5d101a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d15000700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d10108730 0x7f5d101a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d15000700 1 -- 172.21.15.63:0/3197380134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10107de0 msgr2=0x7f5d1019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d15000700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10107de0 0x7f5d1019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d15000700 1 -- 172.21.15.63:0/3197380134 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d101a1cf0 con 0x7f5d101034b0 2024-05-27T07:49:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.152+0000 7f5d15a00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d10108730 0x7f5d101a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.153+0000 7f5d15000700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d101034b0 0x7f5d1019ced0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5d04000c00 tx=0x7f5d0400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.153+0000 7f5d0d600700 1 -- 172.21.15.63:0/3197380134 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04027070 con 0x7f5d101034b0 2024-05-27T07:49:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.153+0000 7f5d0d600700 1 -- 172.21.15.63:0/3197380134 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d040056d0 con 0x7f5d101034b0 2024-05-27T07:49:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.153+0000 7f5d0d600700 1 -- 172.21.15.63:0/3197380134 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040173f0 con 0x7f5d101034b0 2024-05-27T07:49:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.154+0000 7f5d17a00700 1 -- 172.21.15.63:0/3197380134 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d101a1f80 con 0x7f5d101034b0 2024-05-27T07:49:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.154+0000 7f5d17a00700 1 -- 172.21.15.63:0/3197380134 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d101a2410 con 0x7f5d101034b0 2024-05-27T07:49:53.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.159+0000 7f5d0d600700 1 -- 172.21.15.63:0/3197380134 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5d0402a070 con 0x7f5d101034b0 2024-05-27T07:49:53.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.159+0000 7f5d26447b80 1 Processor -- start 2024-05-27T07:49:53.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.160+0000 7f5d26447b80 1 -- start start 2024-05-27T07:49:53.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.160+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dc4a9f0 0x55670dcdd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.160+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 0x55670dcdd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.160+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.160+0000 7f5d26447b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55670dc0be40 con 0x55670dcdde70 2024-05-27T07:49:53.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.160+0000 7f5d26447b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55670dbfb9e0 con 0x55670dc4a9f0 2024-05-27T07:49:53.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.160+0000 7f5d26447b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55670dbffa20 con 0x55670db5ee40 2024-05-27T07:49:53.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d15a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dc4a9f0 0x55670dcdd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d0d600700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5cf80b2a70 0x7f5cf80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcea110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33850/0 (socket says 172.21.15.63:33850) 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d15a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dc4a9f0 0x55670dcdd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34078/0 (socket says 172.21.15.63:34078) 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d0fe00700 1 -- 172.21.15.63:0/1298854699 learned_addr learned my addr 172.21.15.63:0/1298854699 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d15000700 1 --2- 172.21.15.63:0/1298854699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 0x55670dcdd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d0d600700 1 -- 172.21.15.63:0/3197380134 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f5d040e3a10 con 0x7f5d101034b0 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d0fe00700 1 -- 172.21.15.63:0/1298854699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 msgr2=0x55670dcdd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/1298854699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 0x55670dcdd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d0fe00700 1 -- 172.21.15.63:0/1298854699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dc4a9f0 msgr2=0x55670dcdd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.161+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/1298854699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dc4a9f0 0x55670dcdd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.162+0000 7f5d0fe00700 1 -- 172.21.15.63:0/1298854699 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55670dbc4420 con 0x55670dcdde70 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.162+0000 7f5d15a00700 1 --2- 172.21.15.63:0/1298854699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dc4a9f0 0x55670dcdd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:53.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.162+0000 7f5d15a00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5cf80b2a70 0x7f5cf80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.162+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/1298854699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcea110 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00ace0 tx=0x7f5cfc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b374fa45bbe4ef81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.162+0000 7f5d02000700 1 -- 172.21.15.63:0/1298854699 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc004020 con 0x55670dcdde70 2024-05-27T07:49:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.163+0000 7f5d02000700 1 -- 172.21.15.63:0/1298854699 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cfc003680 con 0x55670dcdde70 2024-05-27T07:49:53.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.163+0000 7f5d02000700 1 -- 172.21.15.63:0/1298854699 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc018680 con 0x55670dcdde70 2024-05-27T07:49:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.164+0000 7f5d26447b80 1 -- 172.21.15.63:0/1298854699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 msgr2=0x55670dcea110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.164+0000 7f5d26447b80 1 --2- 172.21.15.63:0/1298854699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcea110 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00ace0 tx=0x7f5cfc00f3c0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.164+0000 7f5d15a00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5cf80b2a70 0x7f5cf80b4f80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55670dcdfff0 tx=0x7f5d08004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.164+0000 7f5d26447b80 1 -- 172.21.15.63:0/1298854699 shutdown_connections 2024-05-27T07:49:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.164+0000 7f5d26447b80 1 --2- 172.21.15.63:0/1298854699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcea110 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.164+0000 7f5d26447b80 1 --2- 172.21.15.63:0/1298854699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dc4a9f0 0x55670dcdd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.164+0000 7f5d26447b80 1 --2- 172.21.15.63:0/1298854699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 0x55670dcdd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.164+0000 7f5d26447b80 1 -- 172.21.15.63:0/1298854699 >> 172.21.15.63:0/1298854699 conn(0x55670dbf8390 msgr2=0x55670dcd94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.164+0000 7f5d26447b80 1 -- 172.21.15.63:0/1298854699 shutdown_connections 2024-05-27T07:49:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.164+0000 7f5d26447b80 1 -- 172.21.15.63:0/1298854699 wait complete. 2024-05-27T07:49:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.165+0000 7f5d26447b80 1 -- start start 2024-05-27T07:49:53.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.165+0000 7f5d26447b80 1 Processor -- start 2024-05-27T07:49:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.166+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 0x55670dce7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.166+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dc4a9f0 0x55670dce7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.166+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 0x55670dce80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.166+0000 7f5d26447b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55670db01fa0 con 0x55670dc4a9f0 2024-05-27T07:49:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.166+0000 7f5d26447b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55670db9c760 con 0x55670dcdde70 2024-05-27T07:49:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.166+0000 7f5d26447b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55670dceab80 con 0x55670db5ee40 2024-05-27T07:49:53.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.166+0000 7f5d26447b80 10 client.0 _ll_register_callbacks cb 0x55670dba5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dc4a9f0 0x55670dce7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dc4a9f0 0x55670dce7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33858/0 (socket says 172.21.15.63:33858) 2024-05-27T07:49:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d0fe00700 1 -- 172.21.15.63:0/3841410097 learned_addr learned my addr 172.21.15.63:0/3841410097 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d15000700 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 0x55670dce7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d15a00700 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 0x55670dce80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d0fe00700 1 -- 172.21.15.63:0/3841410097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 msgr2=0x55670dce7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 0x55670dce7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d0fe00700 1 -- 172.21.15.63:0/3841410097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 msgr2=0x55670dce80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 0x55670dce80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d0fe00700 1 -- 172.21.15.63:0/3841410097 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55670db4b8d0 con 0x55670dc4a9f0 2024-05-27T07:49:53.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.167+0000 7f5d15000700 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 0x55670dce7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.168+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dc4a9f0 0x55670dce7bb0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc000c00 tx=0x7f5cfc003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8dc0c0b49ffef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.168+0000 7f5cf6a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.168+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3841410097 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc026020 con 0x55670dc4a9f0 2024-05-27T07:49:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.168+0000 7f5d26447b80 1 -- 172.21.15.63:0/3841410097 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cfc003680 con 0x55670dc4a9f0 2024-05-27T07:49:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.168+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3841410097 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f5cfc004b60 con 0x55670dc4a9f0 2024-05-27T07:49:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.169+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3841410097 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5cfc031050 con 0x55670dc4a9f0 2024-05-27T07:49:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.169+0000 7f5d26447b80 10 client.18564 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.169+0000 7f5d26447b80 1 -- 172.21.15.63:0/3841410097 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55670dbcc060 con 0x55670dc4a9f0 2024-05-27T07:49:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.169+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3841410097 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cfc003680 con 0x55670dc4a9f0 2024-05-27T07:49:53.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.169+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3841410097 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5cfc022460 con 0x55670dc4a9f0 2024-05-27T07:49:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.169+0000 7f5d26447b80 10 client.18564 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.169+0000 7f5d26447b80 10 client.18564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.169+0000 7f5d26447b80 20 client.18564 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.170+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55670dcf7a30 0x55670dcf9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.170+0000 7f5d26447b80 4 client.18564 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.170+0000 7f5d26447b80 1 -- 172.21.15.63:0/3841410097 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55670dc0dd90 con 0x55670dcf7a30 2024-05-27T07:49:53.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.170+0000 7f5d15000700 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55670dcf7a30 0x55670dcf9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.171+0000 7f5d15000700 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55670dcf7a30 0x55670dcf9e50 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588ba956900797ed server_cookie=2e6b46c1d0386cfe in_seq=0 out_seq=0 2024-05-27T07:49:53.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.171+0000 7f5cf6a00700 10 client.18564 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:53.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.173+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3841410097 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55670dc0dd90 con 0x55670dcf7a30 2024-05-27T07:49:53.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.173+0000 7f5cf6a00700 10 client.18564 handle_command_reply: tid=0 2024-05-27T07:49:53.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.173+0000 7f5d26447b80 1 client.18564 shutdown 2024-05-27T07:49:53.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.173+0000 7f5d26447b80 1 -- 172.21.15.63:0/3841410097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dc4a9f0 msgr2=0x55670dce7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.173+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dc4a9f0 0x55670dce7bb0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc000c00 tx=0x7f5cfc003b10 comp rx=0 tx=0).stop 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.174+0000 7f5d26447b80 1 -- 172.21.15.63:0/3841410097 shutdown_connections 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.174+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55670dcf7a30 0x55670dcf9e50 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.174+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 0x55670dce80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.174+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670db5ee40 0x55670dce7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.174+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3841410097 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dc4a9f0 0x55670dce7bb0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.174+0000 7f5d26447b80 1 -- 172.21.15.63:0/3841410097 >> 172.21.15.63:0/3841410097 conn(0x55670dbf8390 msgr2=0x55670dcdd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.174+0000 7f5cf6a00700 0 client.18564 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.174+0000 7f5d26447b80 1 -- 172.21.15.63:0/3841410097 shutdown_connections 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.174+0000 7f5d26447b80 1 -- 172.21.15.63:0/3841410097 wait complete. 2024-05-27T07:49:53.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.174+0000 7f5d26447b80 20 client.18564 trim_cache size 0 max 16384 2024-05-27T07:49:53.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.199+0000 7f5d26447b80 1 Processor -- start 2024-05-27T07:49:53.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.199+0000 7f5d26447b80 1 -- start start 2024-05-27T07:49:53.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.200+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670db5ee40 0x55670dce2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.200+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670dc4a9f0 0x55670dcdd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.200+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcf3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.200+0000 7f5d26447b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b4dc0 con 0x55670dcdde70 2024-05-27T07:49:53.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.200+0000 7f5d26447b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b2650 con 0x55670db5ee40 2024-05-27T07:49:53.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.200+0000 7f5d26447b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b04c0 con 0x55670dc4a9f0 2024-05-27T07:49:53.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.200+0000 7f5d15a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcf3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.200+0000 7f5d15a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcf3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33860/0 (socket says 172.21.15.63:33860) 2024-05-27T07:49:53.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.200+0000 7f5d15a00700 1 -- 172.21.15.63:0/3731685752 learned_addr learned my addr 172.21.15.63:0/3731685752 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:53.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.201+0000 7f5d15000700 1 --2- 172.21.15.63:0/3731685752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670db5ee40 0x55670dce2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.201+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/3731685752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670dc4a9f0 0x55670dcdd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.201+0000 7f5d15a00700 1 -- 172.21.15.63:0/3731685752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670dc4a9f0 msgr2=0x55670dcdd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:53.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.201+0000 7f5d15a00700 1 --2- 172.21.15.63:0/3731685752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670dc4a9f0 0x55670dcdd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.201+0000 7f5d15a00700 1 -- 172.21.15.63:0/3731685752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670db5ee40 msgr2=0x55670dce2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:53.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.201+0000 7f5d15a00700 1 --2- 172.21.15.63:0/3731685752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670db5ee40 0x55670dce2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.201+0000 7f5d15a00700 1 -- 172.21.15.63:0/3731685752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55670dcd5240 con 0x55670dcdde70 2024-05-27T07:49:53.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.202+0000 7f5d15a00700 1 --2- 172.21.15.63:0/3731685752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcf3e20 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5d08015550 tx=0x7f5d08013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6375bbc8ff21d15e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:53.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.202+0000 7f5d01600700 1 -- 172.21.15.63:0/3731685752 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08025070 con 0x55670dcdde70 2024-05-27T07:49:53.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.202+0000 7f5d01600700 1 -- 172.21.15.63:0/3731685752 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d08019040 con 0x55670dcdde70 2024-05-27T07:49:53.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.202+0000 7f5d01600700 1 -- 172.21.15.63:0/3731685752 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d080207f0 con 0x55670dcdde70 2024-05-27T07:49:53.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.203+0000 7f5d26447b80 1 -- 172.21.15.63:0/3731685752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 msgr2=0x55670dcf3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:53.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.203+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3731685752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcf3e20 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5d08015550 tx=0x7f5d08013000 comp rx=0 tx=0).stop 2024-05-27T07:49:53.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.203+0000 7f5d26447b80 1 -- 172.21.15.63:0/3731685752 shutdown_connections 2024-05-27T07:49:53.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.203+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3731685752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670dcdde70 0x55670dcf3e20 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.203+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3731685752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670db5ee40 0x55670dce2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.203+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3731685752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670dc4a9f0 0x55670dcdd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.203+0000 7f5d26447b80 1 -- 172.21.15.63:0/3731685752 >> 172.21.15.63:0/3731685752 conn(0x55670dbf8390 msgr2=0x55670dcdd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:53.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.203+0000 7f5d26447b80 1 -- 172.21.15.63:0/3731685752 shutdown_connections 2024-05-27T07:49:53.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.204+0000 7f5d26447b80 1 -- 172.21.15.63:0/3731685752 wait complete. 2024-05-27T07:49:53.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.204+0000 7f5d26447b80 1 -- start start 2024-05-27T07:49:53.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.204+0000 7f5d26447b80 1 Processor -- start 2024-05-27T07:49:53.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670db5ee40 0x55670dcf7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670dc4a9f0 0x55670dcf7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d26447b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 0x55670dcf84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d26447b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b0b10 con 0x55670db5ee40 2024-05-27T07:49:53.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d26447b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b1f80 con 0x55670dcdde70 2024-05-27T07:49:53.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d26447b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101b11e0 con 0x55670dc4a9f0 2024-05-27T07:49:53.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d26447b80 10 client.0 _ll_register_callbacks cb 0x55670dba5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:53.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d15000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670db5ee40 0x55670dcf7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d15000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670db5ee40 0x55670dcf7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33868/0 (socket says 172.21.15.63:33868) 2024-05-27T07:49:53.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d15000700 1 -- 172.21.15.63:0/3562495664 learned_addr learned my addr 172.21.15.63:0/3562495664 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:53.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.205+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670dc4a9f0 0x55670dcf7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.206+0000 7f5d15a00700 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 0x55670dcf84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.206+0000 7f5d15000700 1 -- 172.21.15.63:0/3562495664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670dc4a9f0 msgr2=0x55670dcf7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:53.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.206+0000 7f5d15000700 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670dc4a9f0 0x55670dcf7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.206+0000 7f5d15000700 1 -- 172.21.15.63:0/3562495664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 msgr2=0x55670dcf84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:53.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.206+0000 7f5d15000700 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 0x55670dcf84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.206+0000 7f5d15000700 1 -- 172.21.15.63:0/3562495664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55670dd60380 con 0x55670db5ee40 2024-05-27T07:49:53.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.206+0000 7f5d15a00700 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 0x55670dcf84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:53.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.206+0000 7f5d15000700 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670db5ee40 0x55670dcf7a30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5d040a7bd0 tx=0x7f5d04075130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8930ef51b315bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:53.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.206+0000 7f5cf6a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:53.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.206+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3562495664 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040aa470 con 0x55670db5ee40 2024-05-27T07:49:53.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.207+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3562495664 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f5d040a6f50 con 0x55670db5ee40 2024-05-27T07:49:53.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.207+0000 7f5d26447b80 1 -- 172.21.15.63:0/3562495664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d08019040 con 0x55670db5ee40 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.207+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3562495664 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5d04070360 con 0x55670db5ee40 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.207+0000 7f5d26447b80 10 client.18570 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.207+0000 7f5d26447b80 1 -- 172.21.15.63:0/3562495664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55670dd61640 con 0x55670db5ee40 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.207+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3562495664 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d08019040 con 0x55670db5ee40 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.208+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3562495664 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5d040e0fc0 con 0x55670db5ee40 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.208+0000 7f5d26447b80 10 client.18570 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.208+0000 7f5d26447b80 10 client.18570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.208+0000 7f5d26447b80 20 client.18570 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.208+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55670dd61d70 0x55670dd64190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.208+0000 7f5d26447b80 4 client.18570 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:49:53.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.208+0000 7f5d26447b80 1 -- 172.21.15.63:0/3562495664 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55670dc07570 con 0x55670dd61d70 2024-05-27T07:49:53.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.208+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55670dd61d70 0x55670dd64190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:53.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.209+0000 7f5d0fe00700 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55670dd61d70 0x55670dd64190 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb6a1d9f79fc248 server_cookie=84f210674c5fc6af in_seq=0 out_seq=0 2024-05-27T07:49:53.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.209+0000 7f5cf6a00700 10 client.18570 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:53.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5cf6a00700 1 -- 172.21.15.63:0/3562495664 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55670dc07570 con 0x55670dd61d70 2024-05-27T07:49:53.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5cf6a00700 10 client.18570 handle_command_reply: tid=0 2024-05-27T07:49:53.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5d26447b80 1 client.18570 shutdown 2024-05-27T07:49:53.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5d26447b80 1 -- 172.21.15.63:0/3562495664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670db5ee40 msgr2=0x55670dcf7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:53.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670db5ee40 0x55670dcf7a30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5d040a7bd0 tx=0x7f5d04075130 comp rx=0 tx=0).stop 2024-05-27T07:49:53.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5d26447b80 1 -- 172.21.15.63:0/3562495664 shutdown_connections 2024-05-27T07:49:53.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55670dd61d70 0x55670dd64190 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55670dcdde70 0x55670dcf84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55670db5ee40 0x55670dcf7a30 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5d26447b80 1 --2- 172.21.15.63:0/3562495664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55670dc4a9f0 0x55670dcf7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5cf6a00700 0 client.18570 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:53.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.210+0000 7f5d26447b80 1 -- 172.21.15.63:0/3562495664 >> 172.21.15.63:0/3562495664 conn(0x55670dbf8390 msgr2=0x55670dce7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:53.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.211+0000 7f5d26447b80 1 -- 172.21.15.63:0/3562495664 shutdown_connections 2024-05-27T07:49:53.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.211+0000 7f5d26447b80 1 -- 172.21.15.63:0/3562495664 wait complete. 2024-05-27T07:49:53.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.211+0000 7f5d26447b80 20 client.18570 trim_cache size 0 max 16384 2024-05-27T07:49:53.213 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:49:53.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 -- 172.21.15.63:0/3197380134 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5cf80b2a70 msgr2=0x7f5cf80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:53.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5cf80b2a70 0x7f5cf80b4f80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55670dcdfff0 tx=0x7f5d08004ad0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 -- 172.21.15.63:0/3197380134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d101034b0 msgr2=0x7f5d1019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:53.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d101034b0 0x7f5d1019ced0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5d04000c00 tx=0x7f5d0400f690 comp rx=0 tx=0).stop 2024-05-27T07:49:53.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 -- 172.21.15.63:0/3197380134 shutdown_connections 2024-05-27T07:49:53.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5cf80b2a70 0x7f5cf80b4f80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d10108730 0x7f5d101a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5d101034b0 0x7f5d1019ced0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 --2- 172.21.15.63:0/3197380134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d10107de0 0x7f5d1019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:53.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 -- 172.21.15.63:0/3197380134 >> 172.21.15.63:0/3197380134 conn(0x7f5d100fedb0 msgr2=0x7f5d101096f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:53.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 -- 172.21.15.63:0/3197380134 shutdown_connections 2024-05-27T07:49:53.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:53.212+0000 7f5d17a00700 1 -- 172.21.15.63:0/3197380134 wait complete. 2024-05-27T07:49:53.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:52 smithi149 ceph-mon[35408]: pgmap v512: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:49:53.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:52 smithi086 ceph-mon[34687]: pgmap v512: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:49:55.232 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:49:55.232 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:49:55.322 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:54 smithi063 ceph-mon[31613]: pgmap v513: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:49:55.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:54 smithi086 ceph-mon[34687]: pgmap v513: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:49:55.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:54 smithi149 ceph-mon[35408]: pgmap v513: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:49:55.734 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:56.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:56 smithi063 ceph-mon[31613]: pgmap v514: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:49:56.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:56 smithi086 ceph-mon[34687]: pgmap v514: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:49:56.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:56 smithi149 ceph-mon[35408]: pgmap v514: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:49:58.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.241+0000 7fc3fa200700 1 -- 172.21.15.63:0/1372138861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3f4105280 msgr2=0x7fc3f4105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:58.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.241+0000 7fc3fa200700 1 --2- 172.21.15.63:0/1372138861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3f4105280 0x7fc3f4105690 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009990 tx=0x7fc3e8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.241+0000 7fc3fa200700 1 -- 172.21.15.63:0/1372138861 shutdown_connections 2024-05-27T07:49:58.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.241+0000 7fc3fa200700 1 --2- 172.21.15.63:0/1372138861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f410e6f0 0x7fc3f4110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.241+0000 7fc3fa200700 1 --2- 172.21.15.63:0/1372138861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3f4105280 0x7fc3f4105690 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.241+0000 7fc3fa200700 1 --2- 172.21.15.63:0/1372138861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f4105bd0 0x7fc3f410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.241+0000 7fc3fa200700 1 -- 172.21.15.63:0/1372138861 >> 172.21.15.63:0/1372138861 conn(0x7fc3f4078ab0 msgr2=0x7fc3f4078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:58.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.242+0000 7fc3fa200700 1 -- 172.21.15.63:0/1372138861 shutdown_connections 2024-05-27T07:49:58.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.242+0000 7fc3fa200700 1 -- 172.21.15.63:0/1372138861 wait complete. 2024-05-27T07:49:58.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.243+0000 7fc3fa200700 1 Processor -- start 2024-05-27T07:49:58.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.243+0000 7fc3fa200700 1 -- start start 2024-05-27T07:49:58.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.244+0000 7fc3fa200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f4105280 0x7fc3f41a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.244+0000 7fc3fa200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f4105bd0 0x7fc3f41a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.244+0000 7fc3fa200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3f410e6f0 0x7fc3f419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.244+0000 7fc3fa200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41140f0 con 0x7fc3f410e6f0 2024-05-27T07:49:58.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.244+0000 7fc3fa200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4113f70 con 0x7fc3f4105bd0 2024-05-27T07:49:58.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.244+0000 7fc3fa200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4114270 con 0x7fc3f4105280 2024-05-27T07:49:58.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3f410e6f0 0x7fc3f419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3f410e6f0 0x7fc3f419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33892/0 (socket says 172.21.15.63:33892) 2024-05-27T07:49:58.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f9800700 1 -- 172.21.15.63:0/2876257409 learned_addr learned my addr 172.21.15.63:0/2876257409 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:58.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f4105280 0x7fc3f41a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f4105bd0 0x7fc3f41a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f9800700 1 -- 172.21.15.63:0/2876257409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f4105280 msgr2=0x7fc3f41a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:58.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f9800700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f4105280 0x7fc3f41a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f9800700 1 -- 172.21.15.63:0/2876257409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f4105bd0 msgr2=0x7fc3f41a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:58.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f9800700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f4105bd0 0x7fc3f41a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f9800700 1 -- 172.21.15.63:0/2876257409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f419aff0 con 0x7fc3f410e6f0 2024-05-27T07:49:58.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.245+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f4105bd0 0x7fc3f41a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:58.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.246+0000 7fc3f9800700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3f410e6f0 0x7fc3f419a9b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00ada0 tx=0x7fc3ec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:58.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.246+0000 7fc3f1600700 1 -- 172.21.15.63:0/2876257409 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec017020 con 0x7fc3f410e6f0 2024-05-27T07:49:58.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.246+0000 7fc3f1600700 1 -- 172.21.15.63:0/2876257409 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3ec00ce10 con 0x7fc3f410e6f0 2024-05-27T07:49:58.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.246+0000 7fc3f1600700 1 -- 172.21.15.63:0/2876257409 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec005280 con 0x7fc3f410e6f0 2024-05-27T07:49:58.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.246+0000 7fc3fa200700 1 -- 172.21.15.63:0/2876257409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f419b2e0 con 0x7fc3f410e6f0 2024-05-27T07:49:58.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.247+0000 7fc3fa200700 1 -- 172.21.15.63:0/2876257409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f4102ea0 con 0x7fc3f410e6f0 2024-05-27T07:49:58.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.252+0000 7fc3f1600700 1 -- 172.21.15.63:0/2876257409 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc3ec00c980 con 0x7fc3f410e6f0 2024-05-27T07:49:58.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.252+0000 7fc40a349b80 1 Processor -- start 2024-05-27T07:49:58.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.252+0000 7fc40a349b80 1 -- start start 2024-05-27T07:49:58.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.253+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f72e2ec0 0x55c9f7460f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.253+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f746bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.253+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f74618a0 0x55c9f746e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.253+0000 7fc40a349b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c9f738fec0 con 0x55c9f7461490 2024-05-27T07:49:58.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.253+0000 7fc40a349b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c9f737fa60 con 0x55c9f74618a0 2024-05-27T07:49:58.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.253+0000 7fc40a349b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c9f7383aa0 con 0x55c9f72e2ec0 2024-05-27T07:49:58.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.253+0000 7fc3f8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f746bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.253+0000 7fc3f8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f746bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33896/0 (socket says 172.21.15.63:33896) 2024-05-27T07:49:58.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.253+0000 7fc3f8e00700 1 -- 172.21.15.63:0/97986977 learned_addr learned my addr 172.21.15.63:0/97986977 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:58.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.253+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/97986977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f72e2ec0 0x55c9f7460f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.254+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/97986977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f74618a0 0x55c9f746e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.254+0000 7fc3f8e00700 1 -- 172.21.15.63:0/97986977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f72e2ec0 msgr2=0x55c9f7460f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:58.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.254+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/97986977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f72e2ec0 0x55c9f7460f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.254+0000 7fc3f8e00700 1 -- 172.21.15.63:0/97986977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f74618a0 msgr2=0x55c9f746e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:58.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.254+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/97986977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f74618a0 0x55c9f746e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.254+0000 7fc3f8e00700 1 -- 172.21.15.63:0/97986977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f73484a0 con 0x55c9f7461490 2024-05-27T07:49:58.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.254+0000 7fc3f1600700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3dc0b2a70 0x7fc3dc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.254+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/97986977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f746bb50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8000c00 tx=0x7fc3e8005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffee6edcfdb3f54 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:58.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.254+0000 7fc3f1600700 1 -- 172.21.15.63:0/2876257409 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fc3ec0e2090 con 0x7fc3f410e6f0 2024-05-27T07:49:58.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.255+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3dc0b2a70 0x7fc3dc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.255+0000 7fc3e6000700 1 -- 172.21.15.63:0/97986977 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8027070 con 0x55c9f7461490 2024-05-27T07:49:58.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.255+0000 7fc3e6000700 1 -- 172.21.15.63:0/97986977 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3e80055b0 con 0x55c9f7461490 2024-05-27T07:49:58.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.255+0000 7fc3e6000700 1 -- 172.21.15.63:0/97986977 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800bcc0 con 0x55c9f7461490 2024-05-27T07:49:58.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.256+0000 7fc40a349b80 1 -- 172.21.15.63:0/97986977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 msgr2=0x55c9f746bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:58.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.256+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3dc0b2a70 0x7fc3dc0b4f80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55c9f7461d30 tx=0x7fc3e0005e30 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:58.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.256+0000 7fc40a349b80 1 --2- 172.21.15.63:0/97986977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f746bb50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8000c00 tx=0x7fc3e8005050 comp rx=0 tx=0).stop 2024-05-27T07:49:58.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.257+0000 7fc40a349b80 1 -- 172.21.15.63:0/97986977 shutdown_connections 2024-05-27T07:49:58.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.257+0000 7fc40a349b80 1 --2- 172.21.15.63:0/97986977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f74618a0 0x55c9f746e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.257+0000 7fc40a349b80 1 --2- 172.21.15.63:0/97986977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f72e2ec0 0x55c9f7460f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.257+0000 7fc40a349b80 1 --2- 172.21.15.63:0/97986977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f746bb50 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.257+0000 7fc40a349b80 1 -- 172.21.15.63:0/97986977 >> 172.21.15.63:0/97986977 conn(0x55c9f7343040 msgr2=0x55c9f745cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:58.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.257+0000 7fc40a349b80 1 -- 172.21.15.63:0/97986977 shutdown_connections 2024-05-27T07:49:58.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.257+0000 7fc40a349b80 1 -- 172.21.15.63:0/97986977 wait complete. 2024-05-27T07:49:58.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.257+0000 7fc40a349b80 1 -- start start 2024-05-27T07:49:58.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.258+0000 7fc40a349b80 1 Processor -- start 2024-05-27T07:49:58.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.258+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f72e2ec0 0x55c9f746b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.258+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f7461490 0x55c9f746bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.258+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f74618a0 0x55c9f746c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.258+0000 7fc40a349b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c9f7286020 con 0x55c9f74618a0 2024-05-27T07:49:58.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.258+0000 7fc40a349b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c9f73207e0 con 0x55c9f72e2ec0 2024-05-27T07:49:58.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.258+0000 7fc40a349b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c9f746e9b0 con 0x55c9f7461490 2024-05-27T07:49:58.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc40a349b80 10 client.0 _ll_register_callbacks cb 0x55c9f7329cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:58.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc3f9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f74618a0 0x55c9f746c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc3f9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f74618a0 0x55c9f746c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33898/0 (socket says 172.21.15.63:33898) 2024-05-27T07:49:58.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc3f9800700 1 -- 172.21.15.63:0/1655600566 learned_addr learned my addr 172.21.15.63:0/1655600566 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:58.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f72e2ec0 0x55c9f746b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f7461490 0x55c9f746bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc3f9800700 1 -- 172.21.15.63:0/1655600566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f7461490 msgr2=0x55c9f746bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:58.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc3f9800700 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f7461490 0x55c9f746bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc3f9800700 1 -- 172.21.15.63:0/1655600566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f72e2ec0 msgr2=0x55c9f746b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:58.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc3f9800700 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f72e2ec0 0x55c9f746b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.259+0000 7fc3f9800700 1 -- 172.21.15.63:0/1655600566 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f72cf950 con 0x55c9f74618a0 2024-05-27T07:49:58.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.260+0000 7fc3f9800700 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f74618a0 0x55c9f746c070 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0a5060 tx=0x7fc3ec00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9969fa2393a4e1ab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:58.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.260+0000 7fc3daa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:58.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.260+0000 7fc3daa00700 1 -- 172.21.15.63:0/1655600566 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0a8af0 con 0x55c9f74618a0 2024-05-27T07:49:58.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.260+0000 7fc3daa00700 1 -- 172.21.15.63:0/1655600566 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fc3ec0b1b30 con 0x55c9f74618a0 2024-05-27T07:49:58.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.260+0000 7fc40a349b80 1 -- 172.21.15.63:0/1655600566 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e80055b0 con 0x55c9f74618a0 2024-05-27T07:49:58.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc3daa00700 1 -- 172.21.15.63:0/1655600566 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc3ec0a96e0 con 0x55c9f74618a0 2024-05-27T07:49:58.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc40a349b80 10 client.18582 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:58.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc40a349b80 1 -- 172.21.15.63:0/1655600566 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9f73500e0 con 0x55c9f74618a0 2024-05-27T07:49:58.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc3daa00700 1 -- 172.21.15.63:0/1655600566 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3e80055b0 con 0x55c9f74618a0 2024-05-27T07:49:58.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc3daa00700 1 -- 172.21.15.63:0/1655600566 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc3ec012070 con 0x55c9f74618a0 2024-05-27T07:49:58.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc40a349b80 10 client.18582 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:58.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc40a349b80 10 client.18582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:58.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc40a349b80 20 client.18582 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:58.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc40a349b80 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9f747bab0 0x55c9f747df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc40a349b80 4 client.18582 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:58.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.261+0000 7fc40a349b80 1 -- 172.21.15.63:0/1655600566 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9f7391e10 con 0x55c9f747bab0 2024-05-27T07:49:58.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.262+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9f747bab0 0x55c9f747df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.263+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9f747bab0 0x55c9f747df80 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e84532e48fd32a server_cookie=e4cd4a2a304eaaf8 in_seq=0 out_seq=0 2024-05-27T07:49:58.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.263+0000 7fc3daa00700 10 client.18582 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:58.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc3daa00700 1 -- 172.21.15.63:0/1655600566 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c9f7391e10 con 0x55c9f747bab0 2024-05-27T07:49:58.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc3daa00700 10 client.18582 handle_command_reply: tid=0 2024-05-27T07:49:58.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc40a349b80 1 client.18582 shutdown 2024-05-27T07:49:58.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc40a349b80 1 -- 172.21.15.63:0/1655600566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f74618a0 msgr2=0x55c9f746c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc40a349b80 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f74618a0 0x55c9f746c070 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0a5060 tx=0x7fc3ec00cce0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc40a349b80 1 -- 172.21.15.63:0/1655600566 shutdown_connections 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc40a349b80 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9f747bab0 0x55c9f747df80 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc40a349b80 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f74618a0 0x55c9f746c070 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc40a349b80 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f72e2ec0 0x55c9f746b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc40a349b80 1 --2- 172.21.15.63:0/1655600566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f7461490 0x55c9f746bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc3daa00700 0 client.18582 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.265+0000 7fc40a349b80 1 -- 172.21.15.63:0/1655600566 >> 172.21.15.63:0/1655600566 conn(0x55c9f7343040 msgr2=0x55c9f745c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.266+0000 7fc40a349b80 1 -- 172.21.15.63:0/1655600566 shutdown_connections 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.266+0000 7fc40a349b80 1 -- 172.21.15.63:0/1655600566 wait complete. 2024-05-27T07:49:58.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.266+0000 7fc40a349b80 20 client.18582 trim_cache size 0 max 16384 2024-05-27T07:49:58.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.289+0000 7fc40a349b80 1 Processor -- start 2024-05-27T07:49:58.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.290+0000 7fc40a349b80 1 -- start start 2024-05-27T07:49:58.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.290+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f72e2ec0 0x55c9f7466e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.290+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f7461490 0x55c9f746da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.290+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f74618a0 0x55c9f746a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.290+0000 7fc40a349b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41b4d30 con 0x55c9f72e2ec0 2024-05-27T07:49:58.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.290+0000 7fc40a349b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41b25c0 con 0x55c9f7461490 2024-05-27T07:49:58.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.290+0000 7fc40a349b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41b0430 con 0x55c9f74618a0 2024-05-27T07:49:58.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.290+0000 7fc3f8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f72e2ec0 0x55c9f7466e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.291+0000 7fc3f8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f72e2ec0 0x55c9f7466e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33906/0 (socket says 172.21.15.63:33906) 2024-05-27T07:49:58.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.291+0000 7fc3f8e00700 1 -- 172.21.15.63:0/3360978211 learned_addr learned my addr 172.21.15.63:0/3360978211 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:58.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.291+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/3360978211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f7461490 0x55c9f746da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.291+0000 7fc3f9800700 1 --2- 172.21.15.63:0/3360978211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f74618a0 0x55c9f746a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.291+0000 7fc3f8e00700 1 -- 172.21.15.63:0/3360978211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f74618a0 msgr2=0x55c9f746a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:58.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.291+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/3360978211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f74618a0 0x55c9f746a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.291+0000 7fc3f8e00700 1 -- 172.21.15.63:0/3360978211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f7461490 msgr2=0x55c9f746da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:58.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.291+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/3360978211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f7461490 0x55c9f746da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.291+0000 7fc3f8e00700 1 -- 172.21.15.63:0/3360978211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f7479480 con 0x55c9f72e2ec0 2024-05-27T07:49:58.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.291+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/3360978211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f7461490 0x55c9f746da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:58.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.292+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/3360978211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f72e2ec0 0x55c9f7466e10 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8025040 tx=0x7fc3e801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b5fd5add5440 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:58.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.292+0000 7fc3e5600700 1 -- 172.21.15.63:0/3360978211 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8032030 con 0x55c9f72e2ec0 2024-05-27T07:49:58.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.292+0000 7fc3e5600700 1 -- 172.21.15.63:0/3360978211 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3e8034070 con 0x55c9f72e2ec0 2024-05-27T07:49:58.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.292+0000 7fc3e5600700 1 -- 172.21.15.63:0/3360978211 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8027040 con 0x55c9f72e2ec0 2024-05-27T07:49:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.293+0000 7fc40a349b80 1 -- 172.21.15.63:0/3360978211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f72e2ec0 msgr2=0x55c9f7466e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.293+0000 7fc40a349b80 1 --2- 172.21.15.63:0/3360978211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f72e2ec0 0x55c9f7466e10 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8025040 tx=0x7fc3e801a040 comp rx=0 tx=0).stop 2024-05-27T07:49:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.293+0000 7fc40a349b80 1 -- 172.21.15.63:0/3360978211 shutdown_connections 2024-05-27T07:49:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.293+0000 7fc40a349b80 1 --2- 172.21.15.63:0/3360978211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f74618a0 0x55c9f746a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.293+0000 7fc40a349b80 1 --2- 172.21.15.63:0/3360978211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f72e2ec0 0x55c9f7466e10 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.294+0000 7fc40a349b80 1 --2- 172.21.15.63:0/3360978211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f7461490 0x55c9f746da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.294+0000 7fc40a349b80 1 -- 172.21.15.63:0/3360978211 >> 172.21.15.63:0/3360978211 conn(0x55c9f7343040 msgr2=0x55c9f746bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.294+0000 7fc40a349b80 1 -- 172.21.15.63:0/3360978211 shutdown_connections 2024-05-27T07:49:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.294+0000 7fc40a349b80 1 -- 172.21.15.63:0/3360978211 wait complete. 2024-05-27T07:49:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.294+0000 7fc40a349b80 1 -- start start 2024-05-27T07:49:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.295+0000 7fc40a349b80 1 Processor -- start 2024-05-27T07:49:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.295+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f72e2ec0 0x55c9f74742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.295+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f7470630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.295+0000 7fc40a349b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f74618a0 0x55c9f7470b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.295+0000 7fc40a349b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41b0a80 con 0x55c9f7461490 2024-05-27T07:49:58.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.295+0000 7fc40a349b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41b1ef0 con 0x55c9f72e2ec0 2024-05-27T07:49:58.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.295+0000 7fc40a349b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41b1150 con 0x55c9f74618a0 2024-05-27T07:49:58.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.295+0000 7fc40a349b80 10 client.0 _ll_register_callbacks cb 0x55c9f74782c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:58.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.296+0000 7fc3f3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f7470630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.296+0000 7fc3f3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f7470630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33918/0 (socket says 172.21.15.63:33918) 2024-05-27T07:49:58.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.296+0000 7fc3f3e00700 1 -- 172.21.15.63:0/3225371681 learned_addr learned my addr 172.21.15.63:0/3225371681 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:58.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.296+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f72e2ec0 0x55c9f74742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.296+0000 7fc3f9800700 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f74618a0 0x55c9f7470b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.296+0000 7fc3f3e00700 1 -- 172.21.15.63:0/3225371681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f74618a0 msgr2=0x55c9f7470b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:58.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.296+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f74618a0 0x55c9f7470b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.296+0000 7fc3f3e00700 1 -- 172.21.15.63:0/3225371681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f72e2ec0 msgr2=0x55c9f74742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:58.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.296+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f72e2ec0 0x55c9f74742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.296+0000 7fc3f3e00700 1 -- 172.21.15.63:0/3225371681 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f74711e0 con 0x55c9f7461490 2024-05-27T07:49:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.297+0000 7fc3f3e00700 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f7470630 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4102710 tx=0x7fc3e000b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47438b0829d6a66 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.297+0000 7fc3daa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.297+0000 7fc3daa00700 1 -- 172.21.15.63:0/3225371681 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0004220 con 0x55c9f7461490 2024-05-27T07:49:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.297+0000 7fc3daa00700 1 -- 172.21.15.63:0/3225371681 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7fc3e00043c0 con 0x55c9f7461490 2024-05-27T07:49:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.297+0000 7fc40a349b80 1 -- 172.21.15.63:0/3225371681 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e8034070 con 0x55c9f7461490 2024-05-27T07:49:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.297+0000 7fc3daa00700 1 -- 172.21.15.63:0/3225371681 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc3e0030050 con 0x55c9f7461490 2024-05-27T07:49:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.298+0000 7fc40a349b80 10 client.18588 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.298+0000 7fc40a349b80 1 -- 172.21.15.63:0/3225371681 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9f74e7460 con 0x55c9f7461490 2024-05-27T07:49:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.298+0000 7fc3daa00700 1 -- 172.21.15.63:0/3225371681 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3e8034070 con 0x55c9f7461490 2024-05-27T07:49:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.298+0000 7fc3daa00700 1 -- 172.21.15.63:0/3225371681 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc3e0018040 con 0x55c9f7461490 2024-05-27T07:49:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.298+0000 7fc40a349b80 10 client.18588 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.298+0000 7fc40a349b80 10 client.18588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.298+0000 7fc40a349b80 20 client.18588 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.298+0000 7fc40a349b80 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9f74e7c20 0x55c9f74ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.298+0000 7fc40a349b80 4 client.18588 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:49:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.298+0000 7fc40a349b80 1 -- 172.21.15.63:0/3225371681 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c9f738b5f0 con 0x55c9f74e7c20 2024-05-27T07:49:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.299+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9f74e7c20 0x55c9f74ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:58.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.300+0000 7fc3f8e00700 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9f74e7c20 0x55c9f74ea040 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c329b17cbffe494 server_cookie=24b2965efe92bd67 in_seq=0 out_seq=0 2024-05-27T07:49:58.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.300+0000 7fc3daa00700 10 client.18588 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:58.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.303+0000 7fc3daa00700 1 -- 172.21.15.63:0/3225371681 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c9f738b5f0 con 0x55c9f74e7c20 2024-05-27T07:49:58.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.303+0000 7fc3daa00700 10 client.18588 handle_command_reply: tid=0 2024-05-27T07:49:58.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.303+0000 7fc40a349b80 1 client.18588 shutdown 2024-05-27T07:49:58.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 1 -- 172.21.15.63:0/3225371681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 msgr2=0x55c9f7470630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:58.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f7470630 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4102710 tx=0x7fc3e000b4c0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 1 -- 172.21.15.63:0/3225371681 shutdown_connections 2024-05-27T07:49:58.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c9f74e7c20 0x55c9f74ea040 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9f74618a0 0x55c9f7470b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc3daa00700 0 client.18588 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:58.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9f72e2ec0 0x55c9f74742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 1 --2- 172.21.15.63:0/3225371681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c9f7461490 0x55c9f7470630 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 1 -- 172.21.15.63:0/3225371681 >> 172.21.15.63:0/3225371681 conn(0x55c9f7343040 msgr2=0x55c9f7462900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:58.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 1 -- 172.21.15.63:0/3225371681 shutdown_connections 2024-05-27T07:49:58.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 1 -- 172.21.15.63:0/3225371681 wait complete. 2024-05-27T07:49:58.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.304+0000 7fc40a349b80 20 client.18588 trim_cache size 0 max 16384 2024-05-27T07:49:58.306 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:49:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:49:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "734944be-d6ef-46f2-b2e1-88af7e6a3bad", 2024-05-27T07:49:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:49:58.307 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:49:58.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.305+0000 7fc3fa200700 1 -- 172.21.15.63:0/2876257409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3dc0b2a70 msgr2=0x7fc3dc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:58.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.305+0000 7fc3fa200700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3dc0b2a70 0x7fc3dc0b4f80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55c9f7461d30 tx=0x7fc3e0005e30 comp rx=0 tx=0).stop 2024-05-27T07:49:58.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.305+0000 7fc3fa200700 1 -- 172.21.15.63:0/2876257409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3f410e6f0 msgr2=0x7fc3f419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:58.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.305+0000 7fc3fa200700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3f410e6f0 0x7fc3f419a9b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00ada0 tx=0x7fc3ec00c330 comp rx=0 tx=0).stop 2024-05-27T07:49:58.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.305+0000 7fc3fa200700 1 -- 172.21.15.63:0/2876257409 shutdown_connections 2024-05-27T07:49:58.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.305+0000 7fc3fa200700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3dc0b2a70 0x7fc3dc0b4f80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.305+0000 7fc3fa200700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3f410e6f0 0x7fc3f419a9b0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.305+0000 7fc3fa200700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f4105280 0x7fc3f41a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.305+0000 7fc3fa200700 1 --2- 172.21.15.63:0/2876257409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3f4105bd0 0x7fc3f41a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:58.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.305+0000 7fc3fa200700 1 -- 172.21.15.63:0/2876257409 >> 172.21.15.63:0/2876257409 conn(0x7fc3f4078ab0 msgr2=0x7fc3f40fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:58.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.306+0000 7fc3fa200700 1 -- 172.21.15.63:0/2876257409 shutdown_connections 2024-05-27T07:49:58.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:49:58.306+0000 7fc3fa200700 1 -- 172.21.15.63:0/2876257409 wait complete. 2024-05-27T07:49:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.135+0000 7f775e200640 1 -- 172.21.15.63:0/4046271926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7758109ad0 msgr2=0x7f7758103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.135+0000 7f775e200640 1 --2- 172.21.15.63:0/4046271926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7758109ad0 0x7f7758103670 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f774c009900 tx=0x7f774c02a410 comp rx=0 tx=0).stop 2024-05-27T07:49:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.135+0000 7f775e200640 1 -- 172.21.15.63:0/4046271926 shutdown_connections 2024-05-27T07:49:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.135+0000 7f775e200640 1 --2- 172.21.15.63:0/4046271926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7758104220 0x7f77581105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.135+0000 7f775e200640 1 --2- 172.21.15.63:0/4046271926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7758109f70 0x7f7758103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.135+0000 7f775e200640 1 --2- 172.21.15.63:0/4046271926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7758109ad0 0x7f7758103670 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.135+0000 7f775e200640 1 -- 172.21.15.63:0/4046271926 >> 172.21.15.63:0/4046271926 conn(0x7f775807a1b0 msgr2=0x7f775807a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.135+0000 7f775e200640 1 -- 172.21.15.63:0/4046271926 shutdown_connections 2024-05-27T07:49:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.135+0000 7f775e200640 1 -- 172.21.15.63:0/4046271926 wait complete. 2024-05-27T07:49:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.137+0000 7f775e200640 1 Processor -- start 2024-05-27T07:49:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.137+0000 7f775e200640 1 -- start start 2024-05-27T07:49:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.137+0000 7f775e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7758104220 0x7f7758074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.137+0000 7f775e200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7758109ad0 0x7f7758075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.137+0000 7f775e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7758109f70 0x7f7758075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.137+0000 7f775e200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7758113b70 con 0x7f7758104220 2024-05-27T07:49:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.137+0000 7f775e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77581139f0 con 0x7f7758109f70 2024-05-27T07:49:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.137+0000 7f775e200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7758113cf0 con 0x7f7758109ad0 2024-05-27T07:49:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f775ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7758104220 0x7f7758074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f775ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7758104220 0x7f7758074fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35418/0 (socket says 172.21.15.63:35418) 2024-05-27T07:49:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f775ce00640 1 -- 172.21.15.63:0/524549701 learned_addr learned my addr 172.21.15.63:0/524549701 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f7757e00640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7758109ad0 0x7f7758075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f775d800640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7758109f70 0x7f7758075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f775ce00640 1 -- 172.21.15.63:0/524549701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7758109ad0 msgr2=0x7f7758075500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f775ce00640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7758109ad0 0x7f7758075500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f775ce00640 1 -- 172.21.15.63:0/524549701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7758109f70 msgr2=0x7f7758075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f775ce00640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7758109f70 0x7f7758075a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f775ce00640 1 -- 172.21.15.63:0/524549701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77581a8950 con 0x7f7758104220 2024-05-27T07:49:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f7757e00640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7758109ad0 0x7f7758075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.138+0000 7f775d800640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7758109f70 0x7f7758075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.139+0000 7f775ce00640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7758104220 0x7f7758074fc0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f774c005940 tx=0x7f774c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.139+0000 7f7755600640 1 -- 172.21.15.63:0/524549701 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c03f030 con 0x7f7758104220 2024-05-27T07:49:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.139+0000 7f7755600640 1 -- 172.21.15.63:0/524549701 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f774c003c90 con 0x7f7758104220 2024-05-27T07:49:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.139+0000 7f775e200640 1 -- 172.21.15.63:0/524549701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77581a8be0 con 0x7f7758104220 2024-05-27T07:49:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.139+0000 7f7755600640 1 -- 172.21.15.63:0/524549701 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c035070 con 0x7f7758104220 2024-05-27T07:49:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.139+0000 7f775e200640 1 -- 172.21.15.63:0/524549701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77581a90d0 con 0x7f7758104220 2024-05-27T07:49:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.143+0000 7f77667a2740 1 Processor -- start 2024-05-27T07:49:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.143+0000 7f77667a2740 1 -- start start 2024-05-27T07:49:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.144+0000 7f77667a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x562779927a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.144+0000 7f77667a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277955c2f0 0x562779927fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.144+0000 7f77667a2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795644d0 0x5627799284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.144+0000 7f77667a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562779898b90 con 0x56277955c2f0 2024-05-27T07:49:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.144+0000 7f77667a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562779888730 con 0x5627795851b0 2024-05-27T07:49:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.144+0000 7f77667a2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56277988c770 con 0x5627795644d0 2024-05-27T07:49:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.144+0000 7f7755600640 1 -- 172.21.15.63:0/524549701 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f774c042030 con 0x7f7758104220 2024-05-27T07:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.144+0000 7f7757e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795644d0 0x5627799284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.144+0000 7f775d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x562779927a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.145+0000 7f7757e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795644d0 0x5627799284e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:42814/0 (socket says 172.21.15.63:42814) 2024-05-27T07:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.145+0000 7f7757e00640 1 -- 172.21.15.63:0/3849039731 learned_addr learned my addr 172.21.15.63:0/3849039731 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.145+0000 7f775ce00640 1 --2- 172.21.15.63:0/3849039731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277955c2f0 0x562779927fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.145+0000 7f7757e00640 1 -- 172.21.15.63:0/3849039731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 msgr2=0x562779927a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.145+0000 7f7757e00640 1 --2- 172.21.15.63:0/3849039731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x562779927a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.145+0000 7f7757e00640 1 -- 172.21.15.63:0/3849039731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277955c2f0 msgr2=0x562779927fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.145+0000 7f7757e00640 1 --2- 172.21.15.63:0/3849039731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277955c2f0 0x562779927fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.145+0000 7f7757e00640 1 -- 172.21.15.63:0/3849039731 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627798218b0 con 0x5627795644d0 2024-05-27T07:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.146+0000 7f7757e00640 1 --2- 172.21.15.63:0/3849039731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795644d0 0x5627799284e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f774400bc60 tx=0x7f774400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9faa11f22e5bbd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.146+0000 7f774a000640 1 -- 172.21.15.63:0/3849039731 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744015070 con 0x5627795644d0 2024-05-27T07:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.146+0000 7f7755600640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77400b2950 0x7f77400b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.147+0000 7f774a000640 1 -- 172.21.15.63:0/3849039731 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f774401a8f0 con 0x5627795644d0 2024-05-27T07:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.147+0000 7f774a000640 1 -- 172.21.15.63:0/3849039731 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744011840 con 0x5627795644d0 2024-05-27T07:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.147+0000 7f7755600640 1 -- 172.21.15.63:0/524549701 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f774c102fe0 con 0x7f7758104220 2024-05-27T07:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.147+0000 7f775d800640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77400b2950 0x7f77400b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.148+0000 7f77667a2740 1 -- 172.21.15.63:0/3849039731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795644d0 msgr2=0x5627799284e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.148+0000 7f77667a2740 1 --2- 172.21.15.63:0/3849039731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795644d0 0x5627799284e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f774400bc60 tx=0x7f774400f730 comp rx=0 tx=0).stop 2024-05-27T07:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.148+0000 7f775d800640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77400b2950 0x7f77400b4e60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x562779924bd0 tx=0x7f7750009800 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.148+0000 7f77667a2740 1 -- 172.21.15.63:0/3849039731 shutdown_connections 2024-05-27T07:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.148+0000 7f77667a2740 1 --2- 172.21.15.63:0/3849039731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795644d0 0x5627799284e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.148+0000 7f77667a2740 1 --2- 172.21.15.63:0/3849039731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277955c2f0 0x562779927fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.148+0000 7f77667a2740 1 --2- 172.21.15.63:0/3849039731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x562779927a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.148+0000 7f77667a2740 1 -- 172.21.15.63:0/3849039731 >> 172.21.15.63:0/3849039731 conn(0x562779579300 msgr2=0x562779924210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.148+0000 7f77667a2740 1 -- 172.21.15.63:0/3849039731 shutdown_connections 2024-05-27T07:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.148+0000 7f77667a2740 1 -- 172.21.15.63:0/3849039731 wait complete. 2024-05-27T07:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.149+0000 7f77667a2740 1 -- start start 2024-05-27T07:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.149+0000 7f77667a2740 1 Processor -- start 2024-05-27T07:49:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f77667a2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277955c2f0 0x562779931b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f77667a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x5627799320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f77667a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x56277993df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f77667a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56277956e790 con 0x5627795644d0 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f77667a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562779830bc0 con 0x5627795851b0 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f77667a2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562779934be0 con 0x56277955c2f0 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f77667a2740 10 client.0 _ll_register_callbacks cb 0x56277980e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f7757e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x5627799320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f7757e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x5627799320d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35426/0 (socket says 172.21.15.63:35426) 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f7757e00640 1 -- 172.21.15.63:0/3901812554 learned_addr learned my addr 172.21.15.63:0/3901812554 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f775ce00640 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277955c2f0 0x562779931b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.150+0000 7f775d800640 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x56277993df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.151+0000 7f7757e00640 1 -- 172.21.15.63:0/3901812554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277955c2f0 msgr2=0x562779931b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.151+0000 7f7757e00640 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277955c2f0 0x562779931b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.151+0000 7f7757e00640 1 -- 172.21.15.63:0/3901812554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 msgr2=0x56277993df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.151+0000 7f7757e00640 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x56277993df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.151+0000 7f7757e00640 1 -- 172.21.15.63:0/3901812554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56277959d210 con 0x5627795644d0 2024-05-27T07:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.151+0000 7f7757e00640 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x5627799320d0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7744000c00 tx=0x7f774400b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7349b82a8c8eba49 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.151+0000 7f773ea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.151+0000 7f773ea00640 1 -- 172.21.15.63:0/3901812554 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744028060 con 0x5627795644d0 2024-05-27T07:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.151+0000 7f773ea00640 1 -- 172.21.15.63:0/3901812554 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f774401aaa0 con 0x5627795644d0 2024-05-27T07:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.151+0000 7f77667a2740 1 -- 172.21.15.63:0/3901812554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f774401a8f0 con 0x5627795644d0 2024-05-27T07:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.152+0000 7f773ea00640 1 -- 172.21.15.63:0/3901812554 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f774402f050 con 0x5627795644d0 2024-05-27T07:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.152+0000 7f77667a2740 10 client.18600 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.152+0000 7f77667a2740 1 -- 172.21.15.63:0/3901812554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562779669620 con 0x5627795644d0 2024-05-27T07:49:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.152+0000 7f773ea00640 1 -- 172.21.15.63:0/3901812554 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f774401a8f0 con 0x5627795644d0 2024-05-27T07:49:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.153+0000 7f773ea00640 1 -- 172.21.15.63:0/3901812554 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7744004890 con 0x5627795644d0 2024-05-27T07:49:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.153+0000 7f77667a2740 10 client.18600 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.153+0000 7f77667a2740 10 client.18600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.153+0000 7f77667a2740 20 client.18600 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.153+0000 7f77667a2740 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562779543d80 0x562779943ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.153+0000 7f77667a2740 4 client.18600 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.153+0000 7f77667a2740 1 -- 172.21.15.63:0/3901812554 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56277989aae0 con 0x562779543d80 2024-05-27T07:49:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.153+0000 7f775ce00640 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562779543d80 0x562779943ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.154+0000 7f775ce00640 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562779543d80 0x562779943ce0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b98ac253a3961b6 server_cookie=281c1d9bfb480f89 in_seq=0 out_seq=0 2024-05-27T07:49:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.154+0000 7f773ea00640 10 client.18600 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.156+0000 7f773ea00640 1 -- 172.21.15.63:0/3901812554 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56277989aae0 con 0x562779543d80 2024-05-27T07:49:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.156+0000 7f773ea00640 10 client.18600 handle_command_reply: tid=0 2024-05-27T07:49:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.156+0000 7f77667a2740 1 client.18600 shutdown 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.156+0000 7f77667a2740 1 -- 172.21.15.63:0/3901812554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 msgr2=0x5627799320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.156+0000 7f77667a2740 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x5627799320d0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7744000c00 tx=0x7f774400b9a0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.157+0000 7f77667a2740 1 -- 172.21.15.63:0/3901812554 shutdown_connections 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.157+0000 7f77667a2740 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562779543d80 0x562779943ce0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.157+0000 7f77667a2740 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x56277993df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.157+0000 7f77667a2740 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x5627799320d0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.157+0000 7f77667a2740 1 --2- 172.21.15.63:0/3901812554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277955c2f0 0x562779931b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.157+0000 7f773ea00640 0 client.18600 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.157+0000 7f77667a2740 1 -- 172.21.15.63:0/3901812554 >> 172.21.15.63:0/3901812554 conn(0x562779579300 msgr2=0x562779929780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.157+0000 7f77667a2740 1 -- 172.21.15.63:0/3901812554 shutdown_connections 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.157+0000 7f77667a2740 1 -- 172.21.15.63:0/3901812554 wait complete. 2024-05-27T07:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.157+0000 7f77667a2740 20 client.18600 trim_cache size 0 max 16384 2024-05-27T07:49:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.176+0000 7f77667a2740 1 Processor -- start 2024-05-27T07:49:59.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.176+0000 7f77667a2740 1 -- start start 2024-05-27T07:49:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.177+0000 7f77667a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277955c2f0 0x56277992d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.177+0000 7f77667a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x56277993de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.177+0000 7f77667a2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795851b0 0x562779933ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.177+0000 7f77667a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77581b5b00 con 0x5627795644d0 2024-05-27T07:49:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.177+0000 7f77667a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77581b33d0 con 0x56277955c2f0 2024-05-27T07:49:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.177+0000 7f77667a2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f77581b1240 con 0x5627795851b0 2024-05-27T07:49:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.177+0000 7f7757e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x56277993de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.177+0000 7f7757e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x56277993de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35434/0 (socket says 172.21.15.63:35434) 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.177+0000 7f7757e00640 1 -- 172.21.15.63:0/4267853588 learned_addr learned my addr 172.21.15.63:0/4267853588 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.178+0000 7f775ce00640 1 --2- 172.21.15.63:0/4267853588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277955c2f0 0x56277992d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.178+0000 7f775d800640 1 --2- 172.21.15.63:0/4267853588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795851b0 0x562779933ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.178+0000 7f7757e00640 1 -- 172.21.15.63:0/4267853588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795851b0 msgr2=0x562779933ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.178+0000 7f7757e00640 1 --2- 172.21.15.63:0/4267853588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795851b0 0x562779933ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.178+0000 7f7757e00640 1 -- 172.21.15.63:0/4267853588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277955c2f0 msgr2=0x56277992d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.178+0000 7f7757e00640 1 --2- 172.21.15.63:0/4267853588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277955c2f0 0x56277992d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.178+0000 7f7757e00640 1 -- 172.21.15.63:0/4267853588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56277991f230 con 0x5627795644d0 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.178+0000 7f775ce00640 1 --2- 172.21.15.63:0/4267853588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277955c2f0 0x56277992d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.178+0000 7f7757e00640 1 --2- 172.21.15.63:0/4267853588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x56277993de80 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f774400b5b0 tx=0x7f7744005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeba81ecf2699c85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.178+0000 7f7749600640 1 -- 172.21.15.63:0/4267853588 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744028070 con 0x5627795644d0 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.179+0000 7f7749600640 1 -- 172.21.15.63:0/4267853588 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7744011440 con 0x5627795644d0 2024-05-27T07:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.179+0000 7f7749600640 1 -- 172.21.15.63:0/4267853588 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744015070 con 0x5627795644d0 2024-05-27T07:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.180+0000 7f77667a2740 1 -- 172.21.15.63:0/4267853588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 msgr2=0x56277993de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.180+0000 7f77667a2740 1 --2- 172.21.15.63:0/4267853588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x56277993de80 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f774400b5b0 tx=0x7f7744005bf0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.180+0000 7f77667a2740 1 -- 172.21.15.63:0/4267853588 shutdown_connections 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.180+0000 7f77667a2740 1 --2- 172.21.15.63:0/4267853588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627795851b0 0x562779933ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.180+0000 7f77667a2740 1 --2- 172.21.15.63:0/4267853588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x56277993de80 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.180+0000 7f77667a2740 1 --2- 172.21.15.63:0/4267853588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277955c2f0 0x56277992d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.180+0000 7f77667a2740 1 -- 172.21.15.63:0/4267853588 >> 172.21.15.63:0/4267853588 conn(0x562779579300 msgr2=0x562779931ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.180+0000 7f77667a2740 1 -- 172.21.15.63:0/4267853588 shutdown_connections 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.180+0000 7f77667a2740 1 -- 172.21.15.63:0/4267853588 wait complete. 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.180+0000 7f77667a2740 1 -- start start 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.181+0000 7f77667a2740 1 Processor -- start 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.181+0000 7f77667a2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277955c2f0 0x562779941cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.181+0000 7f77667a2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x562779942210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.181+0000 7f77667a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x562779942750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.181+0000 7f77667a2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77581b1890 con 0x5627795644d0 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.181+0000 7f77667a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77581b2d00 con 0x5627795851b0 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.181+0000 7f77667a2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f77581b1f60 con 0x56277955c2f0 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.181+0000 7f77667a2740 10 client.0 _ll_register_callbacks cb 0x7f774c1069a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.181+0000 7f7757e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x562779942210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f7757e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x562779942210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35436/0 (socket says 172.21.15.63:35436) 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f7757e00640 1 -- 172.21.15.63:0/2475471707 learned_addr learned my addr 172.21.15.63:0/2475471707 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f775ce00640 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277955c2f0 0x562779941cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f775d800640 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x562779942750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f7757e00640 1 -- 172.21.15.63:0/2475471707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277955c2f0 msgr2=0x562779941cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f7757e00640 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277955c2f0 0x562779941cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f7757e00640 1 -- 172.21.15.63:0/2475471707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 msgr2=0x562779942750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f7757e00640 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x562779942750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f7757e00640 1 -- 172.21.15.63:0/2475471707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627799aa290 con 0x5627795644d0 2024-05-27T07:49:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f7757e00640 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x562779942210 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7744004d80 tx=0x7f7744011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a529ba76189182a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.182+0000 7f773ea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:49:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.183+0000 7f773ea00640 1 -- 172.21.15.63:0/2475471707 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744015070 con 0x5627795644d0 2024-05-27T07:49:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.183+0000 7f773ea00640 1 -- 172.21.15.63:0/2475471707 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7f7744039040 con 0x5627795644d0 2024-05-27T07:49:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.183+0000 7f77667a2740 1 -- 172.21.15.63:0/2475471707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7744011440 con 0x5627795644d0 2024-05-27T07:49:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.183+0000 7f773ea00640 1 -- 172.21.15.63:0/2475471707 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7744048050 con 0x5627795644d0 2024-05-27T07:49:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.183+0000 7f77667a2740 10 client.18606 fetch_fsmap learned FSMap version 33 2024-05-27T07:49:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.183+0000 7f77667a2740 1 -- 172.21.15.63:0/2475471707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5627799ab4e0 con 0x5627795644d0 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.183+0000 7f773ea00640 1 -- 172.21.15.63:0/2475471707 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7744011440 con 0x5627795644d0 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.184+0000 7f773ea00640 1 -- 172.21.15.63:0/2475471707 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7744011a60 con 0x5627795644d0 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.184+0000 7f77667a2740 10 client.18606 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.184+0000 7f77667a2740 10 client.18606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.184+0000 7f77667a2740 20 client.18606 populate_metadata read hostname 'smithi063' 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.184+0000 7f77667a2740 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627795191c0 0x5627799adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.184+0000 7f77667a2740 4 client.18606 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.184+0000 7f77667a2740 1 -- 172.21.15.63:0/2475471707 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627798942c0 con 0x5627795191c0 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.184+0000 7f775ce00640 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627795191c0 0x5627799adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.185+0000 7f775ce00640 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627795191c0 0x5627799adbd0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f82e84708b3baaba server_cookie=f41a21afeece9966 in_seq=0 out_seq=0 2024-05-27T07:49:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.185+0000 7f773ea00640 10 client.18606 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: pgmap v515: 129 pgs: 129 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:49:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:49:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:49:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:49:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:49:59.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:49:59.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:58 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: pgmap v515: 129 pgs: 129 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:49:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:49:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:49:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:49:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:49:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:49:59.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:58 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: pgmap v515: 129 pgs: 129 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:49:59.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:49:59.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:49:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:49:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:49:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:49:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:58 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f773ea00640 1 -- 172.21.15.63:0/2475471707 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5627798942c0 con 0x5627795191c0 2024-05-27T07:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f773ea00640 10 client.18606 handle_command_reply: tid=0 2024-05-27T07:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f77667a2740 1 client.18606 shutdown 2024-05-27T07:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f77667a2740 1 -- 172.21.15.63:0/2475471707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 msgr2=0x562779942210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f77667a2740 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x562779942210 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f7744004d80 tx=0x7f7744011660 comp rx=0 tx=0).stop 2024-05-27T07:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f77667a2740 1 -- 172.21.15.63:0/2475471707 shutdown_connections 2024-05-27T07:49:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f77667a2740 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627795191c0 0x5627799adbd0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f77667a2740 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627795851b0 0x562779942750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f77667a2740 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627795644d0 0x562779942210 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f77667a2740 1 --2- 172.21.15.63:0/2475471707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277955c2f0 0x562779941cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f77667a2740 1 -- 172.21.15.63:0/2475471707 >> 172.21.15.63:0/2475471707 conn(0x562779579300 msgr2=0x562779931ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.215+0000 7f773ea00640 0 client.18606 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.216+0000 7f77667a2740 1 -- 172.21.15.63:0/2475471707 shutdown_connections 2024-05-27T07:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.216+0000 7f77667a2740 1 -- 172.21.15.63:0/2475471707 wait complete. 2024-05-27T07:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.216+0000 7f77667a2740 20 client.18606 trim_cache size 0 max 16384 2024-05-27T07:49:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 -- 172.21.15.63:0/524549701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77400b2950 msgr2=0x7f77400b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77400b2950 0x7f77400b4e60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x562779924bd0 tx=0x7f7750009800 comp rx=0 tx=0).stop 2024-05-27T07:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 -- 172.21.15.63:0/524549701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7758104220 msgr2=0x7f7758074fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7758104220 0x7f7758074fc0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f774c005940 tx=0x7f774c0042f0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 -- 172.21.15.63:0/524549701 shutdown_connections 2024-05-27T07:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77400b2950 0x7f77400b4e60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7758109f70 0x7f7758075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7758109ad0 0x7f7758075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 --2- 172.21.15.63:0/524549701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7758104220 0x7f7758074fc0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 -- 172.21.15.63:0/524549701 >> 172.21.15.63:0/524549701 conn(0x7f775807a1b0 msgr2=0x7f77581001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 -- 172.21.15.63:0/524549701 shutdown_connections 2024-05-27T07:49:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:49:59.217+0000 7f775e200640 1 -- 172.21.15.63:0/524549701 wait complete. 2024-05-27T07:49:59.291 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:49:59.838 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:50:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:50:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:50:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:49:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:50:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:50:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:50:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:49:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:00.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:50:00.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:00.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:50:00.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:00.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:50:00.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:49:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:01.163 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:00 smithi063 ceph-mon[31613]: pgmap v516: 129 pgs: 129 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:50:01.163 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:00 smithi063 ceph-mon[31613]: overall HEALTH_OK 2024-05-27T07:50:01.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:00 smithi086 ceph-mon[34687]: pgmap v516: 129 pgs: 129 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:50:01.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:00 smithi086 ceph-mon[34687]: overall HEALTH_OK 2024-05-27T07:50:01.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:00 smithi149 ceph-mon[35408]: pgmap v516: 129 pgs: 129 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:50:01.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:00 smithi149 ceph-mon[35408]: overall HEALTH_OK 2024-05-27T07:50:02.989 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:02 smithi063 ceph-mon[31613]: pgmap v517: 129 pgs: 129 active+clean; 4.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.7 MiB/s wr, 59 op/s 2024-05-27T07:50:03.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:02 smithi086 ceph-mon[34687]: pgmap v517: 129 pgs: 129 active+clean; 4.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.7 MiB/s wr, 59 op/s 2024-05-27T07:50:03.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:02 smithi149 ceph-mon[35408]: pgmap v517: 129 pgs: 129 active+clean; 4.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.7 MiB/s wr, 59 op/s 2024-05-27T07:50:03.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.360+0000 7faa13200700 1 -- 172.21.15.63:0/4139156564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa14108d20 msgr2=0x7faa14109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:03.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.360+0000 7faa13200700 1 --2- 172.21.15.63:0/4139156564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa14108d20 0x7faa14109130 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7faa000099c0 tx=0x7faa00009cd0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.360+0000 7faa13200700 1 -- 172.21.15.63:0/4139156564 shutdown_connections 2024-05-27T07:50:03.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.360+0000 7faa13200700 1 --2- 172.21.15.63:0/4139156564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa14109670 0x7faa14110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.360+0000 7faa13200700 1 --2- 172.21.15.63:0/4139156564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa141043f0 0x7faa14104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.360+0000 7faa13200700 1 --2- 172.21.15.63:0/4139156564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa14108d20 0x7faa14109130 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.360+0000 7faa13200700 1 -- 172.21.15.63:0/4139156564 >> 172.21.15.63:0/4139156564 conn(0x7faa14079d00 msgr2=0x7faa1407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:03.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.360+0000 7faa13200700 1 -- 172.21.15.63:0/4139156564 shutdown_connections 2024-05-27T07:50:03.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.361+0000 7faa13200700 1 -- 172.21.15.63:0/4139156564 wait complete. 2024-05-27T07:50:03.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.362+0000 7faa13200700 1 Processor -- start 2024-05-27T07:50:03.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.362+0000 7faa13200700 1 -- start start 2024-05-27T07:50:03.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa13200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa141043f0 0x7faa14075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa13200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa14108d20 0x7faa14075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa13200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa14109670 0x7faa14076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa13200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa14114180 con 0x7faa14109670 2024-05-27T07:50:03.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa13200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa14114000 con 0x7faa14108d20 2024-05-27T07:50:03.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa13200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa14114300 con 0x7faa141043f0 2024-05-27T07:50:03.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa12800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa14109670 0x7faa14076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa11e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa141043f0 0x7faa14075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa12800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa14109670 0x7faa14076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35442/0 (socket says 172.21.15.63:35442) 2024-05-27T07:50:03.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa12800700 1 -- 172.21.15.63:0/2689373704 learned_addr learned my addr 172.21.15.63:0/2689373704 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:03.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.363+0000 7faa11400700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa14108d20 0x7faa14075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.364+0000 7faa12800700 1 -- 172.21.15.63:0/2689373704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa141043f0 msgr2=0x7faa14075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:03.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.364+0000 7faa12800700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa141043f0 0x7faa14075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.364+0000 7faa12800700 1 -- 172.21.15.63:0/2689373704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa14108d20 msgr2=0x7faa14075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:03.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.364+0000 7faa12800700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa14108d20 0x7faa14075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.364+0000 7faa12800700 1 -- 172.21.15.63:0/2689373704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa141a8ff0 con 0x7faa14109670 2024-05-27T07:50:03.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.364+0000 7faa12800700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa14109670 0x7faa14076530 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7faa0c007ee0 tx=0x7faa0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:03.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.365+0000 7faa0aa00700 1 -- 172.21.15.63:0/2689373704 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c004bb0 con 0x7faa14109670 2024-05-27T07:50:03.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.365+0000 7faa0aa00700 1 -- 172.21.15.63:0/2689373704 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa0c004d50 con 0x7faa14109670 2024-05-27T07:50:03.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.365+0000 7faa13200700 1 -- 172.21.15.63:0/2689373704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa141a92e0 con 0x7faa14109670 2024-05-27T07:50:03.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.365+0000 7faa0aa00700 1 -- 172.21.15.63:0/2689373704 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0058b0 con 0x7faa14109670 2024-05-27T07:50:03.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.365+0000 7faa13200700 1 -- 172.21.15.63:0/2689373704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa141a97f0 con 0x7faa14109670 2024-05-27T07:50:03.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.369+0000 7faa0aa00700 1 -- 172.21.15.63:0/2689373704 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faa0c01a020 con 0x7faa14109670 2024-05-27T07:50:03.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.370+0000 7faa27de3b80 1 Processor -- start 2024-05-27T07:50:03.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.370+0000 7faa27de3b80 1 -- start start 2024-05-27T07:50:03.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 0x55be4f00f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ee8fe40 0x55be4f00fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f0192c0 0x55be4f01b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa27de3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55be4ef3ce40 con 0x55be4f0192c0 2024-05-27T07:50:03.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa27de3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55be4ef2c9e0 con 0x55be4ef7b9f0 2024-05-27T07:50:03.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa27de3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55be4ef30a20 con 0x55be4ee8fe40 2024-05-27T07:50:03.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa11400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f0192c0 0x55be4f01b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa11400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f0192c0 0x55be4f01b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35446/0 (socket says 172.21.15.63:35446) 2024-05-27T07:50:03.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa11400700 1 -- 172.21.15.63:0/670560722 learned_addr learned my addr 172.21.15.63:0/670560722 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:03.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa11e00700 1 --2- 172.21.15.63:0/670560722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ee8fe40 0x55be4f00fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.371+0000 7faa0aa00700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9f80b2a70 0x7fa9f80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.372+0000 7faa0aa00700 1 -- 172.21.15.63:0/2689373704 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7faa0c0e1ff0 con 0x7faa14109670 2024-05-27T07:50:03.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.372+0000 7faa11400700 1 --2- 172.21.15.63:0/670560722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 0x55be4f00f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.372+0000 7faa12800700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9f80b2a70 0x7fa9f80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.372+0000 7faa11400700 1 -- 172.21.15.63:0/670560722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ee8fe40 msgr2=0x55be4f00fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:03.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.372+0000 7faa11400700 1 --2- 172.21.15.63:0/670560722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ee8fe40 0x55be4f00fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.372+0000 7faa11400700 1 -- 172.21.15.63:0/670560722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 msgr2=0x55be4f00f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:03.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.372+0000 7faa11400700 1 --2- 172.21.15.63:0/670560722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 0x55be4f00f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.372+0000 7faa11400700 1 -- 172.21.15.63:0/670560722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be4eef5420 con 0x55be4f0192c0 2024-05-27T07:50:03.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.373+0000 7faa11400700 1 --2- 172.21.15.63:0/670560722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f0192c0 0x55be4f01b6e0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7faa1406c3b0 tx=0x7faa0001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51aed7d794ff052d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:03.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.373+0000 7faa12800700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9f80b2a70 0x7fa9f80b4f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0a4f90 tx=0x7faa0c004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:03.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.373+0000 7fa9ff400700 1 -- 172.21.15.63:0/670560722 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0000ba90 con 0x55be4f0192c0 2024-05-27T07:50:03.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.373+0000 7fa9ff400700 1 -- 172.21.15.63:0/670560722 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa00005500 con 0x55be4f0192c0 2024-05-27T07:50:03.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.373+0000 7fa9ff400700 1 -- 172.21.15.63:0/670560722 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00003be0 con 0x55be4f0192c0 2024-05-27T07:50:03.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.374+0000 7faa27de3b80 1 -- 172.21.15.63:0/670560722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f0192c0 msgr2=0x55be4f01b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:03.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.374+0000 7faa27de3b80 1 --2- 172.21.15.63:0/670560722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f0192c0 0x55be4f01b6e0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7faa1406c3b0 tx=0x7faa0001a040 comp rx=0 tx=0).stop 2024-05-27T07:50:03.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.375+0000 7faa27de3b80 1 -- 172.21.15.63:0/670560722 shutdown_connections 2024-05-27T07:50:03.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.375+0000 7faa27de3b80 1 --2- 172.21.15.63:0/670560722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f0192c0 0x55be4f01b6e0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.375+0000 7faa27de3b80 1 --2- 172.21.15.63:0/670560722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 0x55be4f00f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.375+0000 7faa27de3b80 1 --2- 172.21.15.63:0/670560722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ee8fe40 0x55be4f00fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.375+0000 7faa27de3b80 1 -- 172.21.15.63:0/670560722 >> 172.21.15.63:0/670560722 conn(0x55be4ef29390 msgr2=0x55be4f00a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:03.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.375+0000 7faa27de3b80 1 -- 172.21.15.63:0/670560722 shutdown_connections 2024-05-27T07:50:03.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.375+0000 7faa27de3b80 1 -- 172.21.15.63:0/670560722 wait complete. 2024-05-27T07:50:03.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.376+0000 7faa27de3b80 1 -- start start 2024-05-27T07:50:03.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.376+0000 7faa27de3b80 1 Processor -- start 2024-05-27T07:50:03.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.377+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4ee8fe40 0x55be4f016280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.377+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 0x55be4f0167c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.377+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4f0192c0 0x55be4f016d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.377+0000 7faa27de3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55be4ee32fa0 con 0x55be4ee8fe40 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.377+0000 7faa27de3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55be4eecd760 con 0x55be4ef7b9f0 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.377+0000 7faa27de3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55be4f01bda0 con 0x55be4f0192c0 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.377+0000 7faa27de3b80 10 client.0 _ll_register_callbacks cb 0x55be4eed6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.377+0000 7faa11400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4ee8fe40 0x55be4f016280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.377+0000 7faa11400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4ee8fe40 0x55be4f016280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35462/0 (socket says 172.21.15.63:35462) 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.377+0000 7faa11400700 1 -- 172.21.15.63:0/1856525763 learned_addr learned my addr 172.21.15.63:0/1856525763 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.378+0000 7faa11400700 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4f0192c0 0x55be4f016d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.378+0000 7faa11e00700 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 0x55be4f0167c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.378+0000 7faa11400700 1 -- 172.21.15.63:0/1856525763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4f0192c0 msgr2=0x55be4f016d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.378+0000 7faa11400700 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4f0192c0 0x55be4f016d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.378+0000 7faa11400700 1 -- 172.21.15.63:0/1856525763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 msgr2=0x55be4f0167c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:03.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.378+0000 7faa11400700 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 0x55be4f0167c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.378+0000 7faa11400700 1 -- 172.21.15.63:0/1856525763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be4ee7c8d0 con 0x55be4ee8fe40 2024-05-27T07:50:03.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.378+0000 7faa11e00700 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 0x55be4f0167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:03.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.379+0000 7faa11400700 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4ee8fe40 0x55be4f016280 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7faa00009d10 tx=0x7faa00005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3af058c007103f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:03.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.379+0000 7fa9f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:03.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.379+0000 7fa9f3e00700 1 -- 172.21.15.63:0/1856525763 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0002c040 con 0x55be4ee8fe40 2024-05-27T07:50:03.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.379+0000 7faa27de3b80 1 -- 172.21.15.63:0/1856525763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be4ef3ed90 con 0x55be4ee8fe40 2024-05-27T07:50:03.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.379+0000 7fa9f3e00700 1 -- 172.21.15.63:0/1856525763 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7faa000052d0 con 0x55be4ee8fe40 2024-05-27T07:50:03.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.380+0000 7fa9f3e00700 1 -- 172.21.15.63:0/1856525763 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faa0003b050 con 0x55be4ee8fe40 2024-05-27T07:50:03.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.380+0000 7faa27de3b80 10 client.18618 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:03.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.380+0000 7faa27de3b80 1 -- 172.21.15.63:0/1856525763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55be4eefd060 con 0x55be4ee8fe40 2024-05-27T07:50:03.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.380+0000 7fa9f3e00700 1 -- 172.21.15.63:0/1856525763 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55be4ef3ed90 con 0x55be4ee8fe40 2024-05-27T07:50:03.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.381+0000 7fa9f3e00700 1 -- 172.21.15.63:0/1856525763 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7faa00021070 con 0x55be4ee8fe40 2024-05-27T07:50:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.382+0000 7faa27de3b80 10 client.18618 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.382+0000 7faa27de3b80 10 client.18618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.382+0000 7faa27de3b80 20 client.18618 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.382+0000 7faa27de3b80 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be4f028ba0 0x55be4f02afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.382+0000 7faa27de3b80 4 client.18618 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.382+0000 7faa27de3b80 1 -- 172.21.15.63:0/1856525763 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be4ef38570 con 0x55be4f028ba0 2024-05-27T07:50:03.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.383+0000 7faa11e00700 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be4f028ba0 0x55be4f02afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.384+0000 7fa9f3e00700 10 client.18618 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:03.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.384+0000 7faa11e00700 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be4f028ba0 0x55be4f02afc0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea3bc39c4a99c0a server_cookie=99fce51b6c59b5f2 in_seq=0 out_seq=0 2024-05-27T07:50:03.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.386+0000 7fa9f3e00700 1 -- 172.21.15.63:0/1856525763 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55be4ef38570 con 0x55be4f028ba0 2024-05-27T07:50:03.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.386+0000 7fa9f3e00700 10 client.18618 handle_command_reply: tid=0 2024-05-27T07:50:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.386+0000 7faa27de3b80 1 client.18618 shutdown 2024-05-27T07:50:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 1 -- 172.21.15.63:0/1856525763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4ee8fe40 msgr2=0x55be4f016280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:03.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4ee8fe40 0x55be4f016280 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7faa00009d10 tx=0x7faa00005ab0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 1 -- 172.21.15.63:0/1856525763 shutdown_connections 2024-05-27T07:50:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be4f028ba0 0x55be4f02afc0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4f0192c0 0x55be4f016d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4ee8fe40 0x55be4f016280 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 1 --2- 172.21.15.63:0/1856525763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4ef7b9f0 0x55be4f0167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7fa9f3e00700 0 client.18618 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 1 -- 172.21.15.63:0/1856525763 >> 172.21.15.63:0/1856525763 conn(0x55be4ef29390 msgr2=0x55be4f00f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 1 -- 172.21.15.63:0/1856525763 shutdown_connections 2024-05-27T07:50:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 1 -- 172.21.15.63:0/1856525763 wait complete. 2024-05-27T07:50:03.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.387+0000 7faa27de3b80 20 client.18618 trim_cache size 0 max 16384 2024-05-27T07:50:03.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.414+0000 7faa27de3b80 1 Processor -- start 2024-05-27T07:50:03.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.414+0000 7faa27de3b80 1 -- start start 2024-05-27T07:50:03.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ef7b9f0 0x55be4f00a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 0x55be4f015280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa27de3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa141b4dc0 con 0x55be4f011840 2024-05-27T07:50:03.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa27de3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa141b2650 con 0x55be4f0192c0 2024-05-27T07:50:03.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa27de3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa141b04c0 con 0x55be4ef7b9f0 2024-05-27T07:50:03.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa11400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa11400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35478/0 (socket says 172.21.15.63:35478) 2024-05-27T07:50:03.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa11400700 1 -- 172.21.15.63:0/1629814851 learned_addr learned my addr 172.21.15.63:0/1629814851 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:03.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa11e00700 1 --2- 172.21.15.63:0/1629814851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 0x55be4f015280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.415+0000 7faa11400700 1 --2- 172.21.15.63:0/1629814851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ef7b9f0 0x55be4f00a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.416+0000 7faa11400700 1 -- 172.21.15.63:0/1629814851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ef7b9f0 msgr2=0x55be4f00a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:03.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.416+0000 7faa11400700 1 --2- 172.21.15.63:0/1629814851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ef7b9f0 0x55be4f00a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.416+0000 7faa11400700 1 -- 172.21.15.63:0/1629814851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 msgr2=0x55be4f015280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:03.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.416+0000 7faa11400700 1 --2- 172.21.15.63:0/1629814851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 0x55be4f015280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.416+0000 7faa11400700 1 -- 172.21.15.63:0/1629814851 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be4f019080 con 0x55be4f011840 2024-05-27T07:50:03.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.416+0000 7faa11400700 1 --2- 172.21.15.63:0/1629814851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02dfe0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7faa00001eb0 tx=0x7faa00003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3430e1099949a214 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:03.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.416+0000 7fa9fea00700 1 -- 172.21.15.63:0/1629814851 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0000bb70 con 0x55be4f011840 2024-05-27T07:50:03.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.416+0000 7fa9fea00700 1 -- 172.21.15.63:0/1629814851 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa000056c0 con 0x55be4f011840 2024-05-27T07:50:03.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.416+0000 7fa9fea00700 1 -- 172.21.15.63:0/1629814851 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000050d0 con 0x55be4f011840 2024-05-27T07:50:03.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.417+0000 7faa27de3b80 1 -- 172.21.15.63:0/1629814851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 msgr2=0x55be4f02dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:03.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.417+0000 7faa27de3b80 1 --2- 172.21.15.63:0/1629814851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02dfe0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7faa00001eb0 tx=0x7faa00003f30 comp rx=0 tx=0).stop 2024-05-27T07:50:03.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.418+0000 7faa27de3b80 1 -- 172.21.15.63:0/1629814851 shutdown_connections 2024-05-27T07:50:03.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.418+0000 7faa27de3b80 1 --2- 172.21.15.63:0/1629814851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02dfe0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.418+0000 7faa27de3b80 1 --2- 172.21.15.63:0/1629814851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ef7b9f0 0x55be4f00a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.418+0000 7faa27de3b80 1 --2- 172.21.15.63:0/1629814851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 0x55be4f015280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.418+0000 7faa27de3b80 1 -- 172.21.15.63:0/1629814851 >> 172.21.15.63:0/1629814851 conn(0x55be4ef29390 msgr2=0x55be4f014b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:03.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.418+0000 7faa27de3b80 1 -- 172.21.15.63:0/1629814851 shutdown_connections 2024-05-27T07:50:03.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.418+0000 7faa27de3b80 1 -- 172.21.15.63:0/1629814851 wait complete. 2024-05-27T07:50:03.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.418+0000 7faa27de3b80 1 -- start start 2024-05-27T07:50:03.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.418+0000 7faa27de3b80 1 Processor -- start 2024-05-27T07:50:03.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.419+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ef7b9f0 0x55be4f02bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.419+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.419+0000 7faa27de3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 0x55be4f02c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.419+0000 7faa27de3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa141b0b10 con 0x55be4f011840 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.419+0000 7faa27de3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa141b1f80 con 0x55be4f0192c0 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.419+0000 7faa27de3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa141b11e0 con 0x55be4ef7b9f0 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.419+0000 7faa27de3b80 10 client.0 _ll_register_callbacks cb 0x55be4eed6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.419+0000 7faa11e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.419+0000 7faa11e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35492/0 (socket says 172.21.15.63:35492) 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.419+0000 7faa11e00700 1 -- 172.21.15.63:0/3584338424 learned_addr learned my addr 172.21.15.63:0/3584338424 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.420+0000 7faa11400700 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ef7b9f0 0x55be4f02bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.420+0000 7faa11400700 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 0x55be4f02c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.420+0000 7faa11e00700 1 -- 172.21.15.63:0/3584338424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ef7b9f0 msgr2=0x55be4f02bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:03.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.420+0000 7faa11e00700 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ef7b9f0 0x55be4f02bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.420+0000 7faa11e00700 1 -- 172.21.15.63:0/3584338424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 msgr2=0x55be4f02c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:03.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.420+0000 7faa11e00700 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 0x55be4f02c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.420+0000 7faa11e00700 1 -- 172.21.15.63:0/3584338424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be4f09ac20 con 0x55be4f011840 2024-05-27T07:50:03.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.420+0000 7faa11400700 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 0x55be4f02c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:03.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.420+0000 7faa11e00700 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02c270 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x55be4f00b210 tx=0x7faa0400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86f03879ee867b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:03.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.420+0000 7fa9f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:03.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.421+0000 7fa9f3e00700 1 -- 172.21.15.63:0/3584338424 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04003de0 con 0x55be4f011840 2024-05-27T07:50:03.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.421+0000 7fa9f3e00700 1 -- 172.21.15.63:0/3584338424 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9483+0+0 (secure 0 0 0) 0x7faa04011440 con 0x55be4f011840 2024-05-27T07:50:03.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.421+0000 7faa27de3b80 1 -- 172.21.15.63:0/3584338424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be4edaf250 con 0x55be4f011840 2024-05-27T07:50:03.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.421+0000 7fa9f3e00700 1 -- 172.21.15.63:0/3584338424 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faa04011640 con 0x55be4f011840 2024-05-27T07:50:03.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.421+0000 7faa27de3b80 10 client.18624 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:03.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.421+0000 7faa27de3b80 1 -- 172.21.15.63:0/3584338424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55be4f09be80 con 0x55be4f011840 2024-05-27T07:50:03.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.422+0000 7fa9f3e00700 1 -- 172.21.15.63:0/3584338424 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55be4edaf250 con 0x55be4f011840 2024-05-27T07:50:03.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.422+0000 7fa9f3e00700 1 -- 172.21.15.63:0/3584338424 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7faa0401d760 con 0x55be4f011840 2024-05-27T07:50:03.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.423+0000 7faa27de3b80 10 client.18624 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:03.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.423+0000 7faa27de3b80 10 client.18624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:03.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.423+0000 7faa27de3b80 20 client.18624 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:03.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.423+0000 7faa27de3b80 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be4f09c610 0x55be4f09ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:03.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.423+0000 7faa27de3b80 4 client.18624 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:50:03.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.423+0000 7faa27de3b80 1 -- 172.21.15.63:0/3584338424 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be4ef00b90 con 0x55be4f09c610 2024-05-27T07:50:03.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.423+0000 7faa11400700 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be4f09c610 0x55be4f09ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:03.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.424+0000 7faa11400700 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be4f09c610 0x55be4f09ea30 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92dd0a35904f63 server_cookie=4ed2ae9d184227d5 in_seq=0 out_seq=0 2024-05-27T07:50:03.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.424+0000 7fa9f3e00700 10 client.18624 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:03.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7fa9f3e00700 1 -- 172.21.15.63:0/3584338424 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55be4ef00b90 con 0x55be4f09c610 2024-05-27T07:50:03.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7fa9f3e00700 10 client.18624 handle_command_reply: tid=0 2024-05-27T07:50:03.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7faa27de3b80 1 client.18624 shutdown 2024-05-27T07:50:03.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7faa27de3b80 1 -- 172.21.15.63:0/3584338424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 msgr2=0x55be4f02c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:03.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7faa27de3b80 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02c270 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x55be4f00b210 tx=0x7faa0400bf00 comp rx=0 tx=0).stop 2024-05-27T07:50:03.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7faa27de3b80 1 -- 172.21.15.63:0/3584338424 shutdown_connections 2024-05-27T07:50:03.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7faa27de3b80 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55be4f09c610 0x55be4f09ea30 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7faa27de3b80 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55be4f0192c0 0x55be4f02c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7faa27de3b80 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55be4ef7b9f0 0x55be4f02bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7faa27de3b80 1 --2- 172.21.15.63:0/3584338424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55be4f011840 0x55be4f02c270 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7fa9f3e00700 0 client.18624 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:03.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.425+0000 7faa27de3b80 1 -- 172.21.15.63:0/3584338424 >> 172.21.15.63:0/3584338424 conn(0x55be4ef29390 msgr2=0x55be4f015300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:03.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.426+0000 7faa27de3b80 1 -- 172.21.15.63:0/3584338424 shutdown_connections 2024-05-27T07:50:03.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.426+0000 7faa27de3b80 1 -- 172.21.15.63:0/3584338424 wait complete. 2024-05-27T07:50:03.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.426+0000 7faa27de3b80 20 client.18624 trim_cache size 0 max 16384 2024-05-27T07:50:03.428 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:50:03.428 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:50:03.428 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:50:03.428 INFO:teuthology.orchestra.run.smithi063.stdout: "734944be-d6ef-46f2-b2e1-88af7e6a3bad": { 2024-05-27T07:50:03.428 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:50:03.428 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "734944be-d6ef-46f2-b2e1-88af7e6a3bad", 2024-05-27T07:50:03.428 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:50:03.429 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:50:03.429 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:50:03.429 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:50:03.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.426+0000 7faa13200700 1 -- 172.21.15.63:0/2689373704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9f80b2a70 msgr2=0x7fa9f80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:03.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.426+0000 7faa13200700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9f80b2a70 0x7fa9f80b4f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0a4f90 tx=0x7faa0c004ad0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.426+0000 7faa13200700 1 -- 172.21.15.63:0/2689373704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa14109670 msgr2=0x7faa14076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:03.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.426+0000 7faa13200700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa14109670 0x7faa14076530 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7faa0c007ee0 tx=0x7faa0c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.427+0000 7faa13200700 1 -- 172.21.15.63:0/2689373704 shutdown_connections 2024-05-27T07:50:03.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.427+0000 7faa13200700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9f80b2a70 0x7fa9f80b4f80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.427+0000 7faa13200700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa14109670 0x7faa14076530 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.427+0000 7faa13200700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa141043f0 0x7faa14075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.427+0000 7faa13200700 1 --2- 172.21.15.63:0/2689373704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa14108d20 0x7faa14075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:03.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.427+0000 7faa13200700 1 -- 172.21.15.63:0/2689373704 >> 172.21.15.63:0/2689373704 conn(0x7faa14079d00 msgr2=0x7faa14101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:03.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.427+0000 7faa13200700 1 -- 172.21.15.63:0/2689373704 shutdown_connections 2024-05-27T07:50:03.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:03.427+0000 7faa13200700 1 -- 172.21.15.63:0/2689373704 wait complete. 2024-05-27T07:50:04.420 INFO:tasks.cephfs.filesystem:scrub status for tag:734944be-d6ef-46f2-b2e1-88af7e6a3bad - {'path': '/', 'tag': '734944be-d6ef-46f2-b2e1-88af7e6a3bad', 'options': 'recursive,force'} 2024-05-27T07:50:05.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:04 smithi086 ceph-mon[34687]: pgmap v518: 129 pgs: 129 active+clean; 4.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.0 MiB/s wr, 55 op/s 2024-05-27T07:50:05.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:04 smithi149 ceph-mon[35408]: pgmap v518: 129 pgs: 129 active+clean; 4.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.0 MiB/s wr, 55 op/s 2024-05-27T07:50:05.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:04 smithi063 ceph-mon[31613]: pgmap v518: 129 pgs: 129 active+clean; 4.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.0 MiB/s wr, 55 op/s 2024-05-27T07:50:07.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:06 smithi086 ceph-mon[34687]: pgmap v519: 129 pgs: 129 active+clean; 4.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.0 MiB/s wr, 55 op/s 2024-05-27T07:50:07.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:06 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:50:07.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:06 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:50:07.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:06 smithi149 ceph-mon[35408]: pgmap v519: 129 pgs: 129 active+clean; 4.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.0 MiB/s wr, 55 op/s 2024-05-27T07:50:07.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:06 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:50:07.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:06 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:50:07.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:06 smithi063 ceph-mon[31613]: pgmap v519: 129 pgs: 129 active+clean; 4.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.0 MiB/s wr, 55 op/s 2024-05-27T07:50:07.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:06 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:50:07.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:06 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:50:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:50:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:07 smithi086 ceph-mon[34687]: osdmap e114: 12 total, 12 up, 12 in 2024-05-27T07:50:08.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:50:08.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:07 smithi149 ceph-mon[35408]: osdmap e114: 12 total, 12 up, 12 in 2024-05-27T07:50:08.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:50:08.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:07 smithi063 ceph-mon[31613]: osdmap e114: 12 total, 12 up, 12 in 2024-05-27T07:50:09.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:08 smithi086 ceph-mon[34687]: pgmap v521: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:50:09.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:08 smithi149 ceph-mon[35408]: pgmap v521: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:50:09.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:08 smithi063 ceph-mon[31613]: pgmap v521: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:50:11.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:10 smithi086 ceph-mon[34687]: pgmap v522: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:50:11.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:10 smithi149 ceph-mon[35408]: pgmap v522: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:50:11.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:10 smithi063 ceph-mon[31613]: pgmap v522: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.1 MiB/s wr, 66 op/s 2024-05-27T07:50:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.483+0000 7fba30a00640 1 -- 172.21.15.63:0/3593880392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba2c10d9c0 msgr2=0x7fba2c10fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.483+0000 7fba30a00640 1 --2- 172.21.15.63:0/3593880392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba2c10d9c0 0x7fba2c10fde0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fba18009900 tx=0x7fba1802a480 comp rx=0 tx=0).stop 2024-05-27T07:50:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.483+0000 7fba30a00640 1 -- 172.21.15.63:0/3593880392 shutdown_connections 2024-05-27T07:50:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.483+0000 7fba30a00640 1 --2- 172.21.15.63:0/3593880392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba2c110320 0x7fba2c112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.483+0000 7fba30a00640 1 --2- 172.21.15.63:0/3593880392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba2c10d9c0 0x7fba2c10fde0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.483+0000 7fba30a00640 1 --2- 172.21.15.63:0/3593880392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c075420 0x7fba2c075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.483+0000 7fba30a00640 1 -- 172.21.15.63:0/3593880392 >> 172.21.15.63:0/3593880392 conn(0x7fba2c07a1b0 msgr2=0x7fba2c07a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.483+0000 7fba30a00640 1 -- 172.21.15.63:0/3593880392 shutdown_connections 2024-05-27T07:50:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.483+0000 7fba30a00640 1 -- 172.21.15.63:0/3593880392 wait complete. 2024-05-27T07:50:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.485+0000 7fba30a00640 1 Processor -- start 2024-05-27T07:50:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.485+0000 7fba30a00640 1 -- start start 2024-05-27T07:50:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.485+0000 7fba30a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c075420 0x7fba2c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.485+0000 7fba30a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba2c10d9c0 0x7fba2c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.485+0000 7fba30a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba2c110320 0x7fba2c1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.485+0000 7fba30a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c115d40 con 0x7fba2c110320 2024-05-27T07:50:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.485+0000 7fba30a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c115bc0 con 0x7fba2c075420 2024-05-27T07:50:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.485+0000 7fba30a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c115ec0 con 0x7fba2c10d9c0 2024-05-27T07:50:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.486+0000 7fba2b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba2c110320 0x7fba2c1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.486+0000 7fba2b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba2c110320 0x7fba2c1a1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51754/0 (socket says 172.21.15.63:51754) 2024-05-27T07:50:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.486+0000 7fba2b200640 1 -- 172.21.15.63:0/2563005599 learned_addr learned my addr 172.21.15.63:0/2563005599 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.486+0000 7fba2a800640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c075420 0x7fba2c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.486+0000 7fba29e00640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba2c10d9c0 0x7fba2c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.486+0000 7fba2b200640 1 -- 172.21.15.63:0/2563005599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba2c10d9c0 msgr2=0x7fba2c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.486+0000 7fba2b200640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba2c10d9c0 0x7fba2c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.486+0000 7fba2b200640 1 -- 172.21.15.63:0/2563005599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c075420 msgr2=0x7fba2c19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.486+0000 7fba2b200640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c075420 0x7fba2c19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.487+0000 7fba2b200640 1 -- 172.21.15.63:0/2563005599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c1a1660 con 0x7fba2c110320 2024-05-27T07:50:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.487+0000 7fba29e00640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba2c10d9c0 0x7fba2c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.487+0000 7fba2b200640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba2c110320 0x7fba2c1a1020 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fba2400d750 tx=0x7fba2400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.487+0000 7fba23400640 1 -- 172.21.15.63:0/2563005599 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24017070 con 0x7fba2c110320 2024-05-27T07:50:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.487+0000 7fba23400640 1 -- 172.21.15.63:0/2563005599 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba240054b0 con 0x7fba2c110320 2024-05-27T07:50:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.487+0000 7fba23400640 1 -- 172.21.15.63:0/2563005599 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24004b90 con 0x7fba2c110320 2024-05-27T07:50:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.487+0000 7fba30a00640 1 -- 172.21.15.63:0/2563005599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2c1a1950 con 0x7fba2c110320 2024-05-27T07:50:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.488+0000 7fba30a00640 1 -- 172.21.15.63:0/2563005599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2c1a92d0 con 0x7fba2c110320 2024-05-27T07:50:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.491+0000 7fba383b0740 1 Processor -- start 2024-05-27T07:50:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.492+0000 7fba23400640 1 -- 172.21.15.63:0/2563005599 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fba24004d30 con 0x7fba2c110320 2024-05-27T07:50:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.492+0000 7fba383b0740 1 -- start start 2024-05-27T07:50:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.493+0000 7fba383b0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525afeaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.493+0000 7fba383b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac1f2f0 0x56525afeafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.493+0000 7fba383b0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525afeb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.493+0000 7fba383b0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56525af5bb90 con 0x56525ac274d0 2024-05-27T07:50:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.493+0000 7fba383b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56525af4b730 con 0x56525ac1f2f0 2024-05-27T07:50:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.493+0000 7fba383b0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56525af4f770 con 0x56525ac481b0 2024-05-27T07:50:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.493+0000 7fba29e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525afeb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.493+0000 7fba29e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525afeb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51770/0 (socket says 172.21.15.63:51770) 2024-05-27T07:50:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.493+0000 7fba29e00640 1 -- 172.21.15.63:0/1515553771 learned_addr learned my addr 172.21.15.63:0/1515553771 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.493+0000 7fba29e00640 1 --2- 172.21.15.63:0/1515553771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525afeaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.494+0000 7fba2a800640 1 --2- 172.21.15.63:0/1515553771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac1f2f0 0x56525afeafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.494+0000 7fba29e00640 1 -- 172.21.15.63:0/1515553771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 msgr2=0x56525afeaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.494+0000 7fba29e00640 1 --2- 172.21.15.63:0/1515553771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525afeaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.494+0000 7fba29e00640 1 -- 172.21.15.63:0/1515553771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac1f2f0 msgr2=0x56525afeafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.494+0000 7fba29e00640 1 --2- 172.21.15.63:0/1515553771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac1f2f0 0x56525afeafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.494+0000 7fba29e00640 1 -- 172.21.15.63:0/1515553771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525aee48b0 con 0x56525ac274d0 2024-05-27T07:50:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.494+0000 7fba23400640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba100b2950 0x7fba100b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.494+0000 7fba23400640 1 -- 172.21.15.63:0/2563005599 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9571+0+0 (secure 0 0 0) 0x7fba240e2430 con 0x7fba2c110320 2024-05-27T07:50:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.495+0000 7fba29e00640 1 --2- 172.21.15.63:0/1515553771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525afeb4e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fba18032fd0 tx=0x7fba180056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9523d76173958b35 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.495+0000 7fba2a800640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba100b2950 0x7fba100b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.495+0000 7fba17400640 1 -- 172.21.15.63:0/1515553771 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18035040 con 0x56525ac274d0 2024-05-27T07:50:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.495+0000 7fba17400640 1 -- 172.21.15.63:0/1515553771 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba180044c0 con 0x56525ac274d0 2024-05-27T07:50:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.495+0000 7fba17400640 1 -- 172.21.15.63:0/1515553771 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18003cb0 con 0x56525ac274d0 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.496+0000 7fba383b0740 1 -- 172.21.15.63:0/1515553771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 msgr2=0x56525afeb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.496+0000 7fba2a800640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba100b2950 0x7fba100b4e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x56525afebe10 tx=0x7fba1c009680 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.496+0000 7fba383b0740 1 --2- 172.21.15.63:0/1515553771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525afeb4e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fba18032fd0 tx=0x7fba180056d0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.497+0000 7fba383b0740 1 -- 172.21.15.63:0/1515553771 shutdown_connections 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.497+0000 7fba383b0740 1 --2- 172.21.15.63:0/1515553771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525afeb4e0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.497+0000 7fba383b0740 1 --2- 172.21.15.63:0/1515553771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac1f2f0 0x56525afeafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.497+0000 7fba383b0740 1 --2- 172.21.15.63:0/1515553771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525afeaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.497+0000 7fba383b0740 1 -- 172.21.15.63:0/1515553771 >> 172.21.15.63:0/1515553771 conn(0x56525ac3c300 msgr2=0x56525afe7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.497+0000 7fba383b0740 1 -- 172.21.15.63:0/1515553771 shutdown_connections 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.497+0000 7fba383b0740 1 -- 172.21.15.63:0/1515553771 wait complete. 2024-05-27T07:50:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.497+0000 7fba383b0740 1 -- start start 2024-05-27T07:50:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.498+0000 7fba383b0740 1 Processor -- start 2024-05-27T07:50:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.498+0000 7fba383b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac1f2f0 0x56525aff4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.498+0000 7fba383b0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525aff5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.498+0000 7fba383b0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525b000e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.498+0000 7fba383b0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56525ac31790 con 0x56525ac274d0 2024-05-27T07:50:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.498+0000 7fba383b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56525aef3bc0 con 0x56525ac1f2f0 2024-05-27T07:50:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.498+0000 7fba383b0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56525aff7b30 con 0x56525ac481b0 2024-05-27T07:50:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba383b0740 10 client.0 _ll_register_callbacks cb 0x56525aed1400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba29e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525aff5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba29e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525aff5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51772/0 (socket says 172.21.15.63:51772) 2024-05-27T07:50:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba29e00640 1 -- 172.21.15.63:0/2472001391 learned_addr learned my addr 172.21.15.63:0/2472001391 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba2a800640 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac1f2f0 0x56525aff4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba2b200640 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525b000e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba29e00640 1 -- 172.21.15.63:0/2472001391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 msgr2=0x56525b000e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba29e00640 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525b000e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba29e00640 1 -- 172.21.15.63:0/2472001391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac1f2f0 msgr2=0x56525aff4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba29e00640 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac1f2f0 0x56525aff4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.499+0000 7fba29e00640 1 -- 172.21.15.63:0/2472001391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525ac60210 con 0x56525ac274d0 2024-05-27T07:50:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.500+0000 7fba29e00640 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525aff5440 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fba1802a9a0 tx=0x7fba18031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cba35aed74dd9c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.500+0000 7fba0be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.500+0000 7fba0be00640 1 -- 172.21.15.63:0/2472001391 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18000e60 con 0x56525ac274d0 2024-05-27T07:50:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.500+0000 7fba0be00640 1 -- 172.21.15.63:0/2472001391 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9571+0+0 (secure 0 0 0) 0x7fba18003c30 con 0x56525ac274d0 2024-05-27T07:50:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.500+0000 7fba383b0740 1 -- 172.21.15.63:0/2472001391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56525af5dae0 con 0x56525ac274d0 2024-05-27T07:50:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba0be00640 1 -- 172.21.15.63:0/2472001391 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fba1804d070 con 0x56525ac274d0 2024-05-27T07:50:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba383b0740 10 client.18636 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba383b0740 1 -- 172.21.15.63:0/2472001391 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56525ad2c620 con 0x56525ac274d0 2024-05-27T07:50:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba0be00640 1 -- 172.21.15.63:0/2472001391 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56525af5dae0 con 0x56525ac274d0 2024-05-27T07:50:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba0be00640 1 -- 172.21.15.63:0/2472001391 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fba1803a040 con 0x56525ac274d0 2024-05-27T07:50:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba383b0740 10 client.18636 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba383b0740 10 client.18636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba383b0740 20 client.18636 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba383b0740 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56525ac06d80 0x56525b006d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba383b0740 4 client.18636 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.501+0000 7fba383b0740 1 -- 172.21.15.63:0/2472001391 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56525af572c0 con 0x56525ac06d80 2024-05-27T07:50:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.502+0000 7fba2a800640 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56525ac06d80 0x56525b006d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.503+0000 7fba2a800640 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56525ac06d80 0x56525b006d40 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474af18375416a3d server_cookie=8d20f760aa203fb3 in_seq=0 out_seq=0 2024-05-27T07:50:12.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.503+0000 7fba0be00640 10 client.18636 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:12.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.505+0000 7fba0be00640 1 -- 172.21.15.63:0/2472001391 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56525af572c0 con 0x56525ac06d80 2024-05-27T07:50:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.505+0000 7fba0be00640 10 client.18636 handle_command_reply: tid=0 2024-05-27T07:50:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.505+0000 7fba383b0740 1 client.18636 shutdown 2024-05-27T07:50:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.505+0000 7fba383b0740 1 -- 172.21.15.63:0/2472001391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 msgr2=0x56525aff5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.505+0000 7fba383b0740 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525aff5440 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fba1802a9a0 tx=0x7fba18031490 comp rx=0 tx=0).stop 2024-05-27T07:50:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.505+0000 7fba383b0740 1 -- 172.21.15.63:0/2472001391 shutdown_connections 2024-05-27T07:50:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.505+0000 7fba383b0740 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56525ac06d80 0x56525b006d40 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.506+0000 7fba383b0740 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525b000e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.506+0000 7fba383b0740 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac274d0 0x56525aff5440 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.506+0000 7fba383b0740 1 --2- 172.21.15.63:0/2472001391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac1f2f0 0x56525aff4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.506+0000 7fba383b0740 1 -- 172.21.15.63:0/2472001391 >> 172.21.15.63:0/2472001391 conn(0x56525ac3c300 msgr2=0x56525afec780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.506+0000 7fba0be00640 0 client.18636 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.506+0000 7fba383b0740 1 -- 172.21.15.63:0/2472001391 shutdown_connections 2024-05-27T07:50:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.506+0000 7fba383b0740 1 -- 172.21.15.63:0/2472001391 wait complete. 2024-05-27T07:50:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.506+0000 7fba383b0740 20 client.18636 trim_cache size 0 max 16384 2024-05-27T07:50:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.526+0000 7fba383b0740 1 Processor -- start 2024-05-27T07:50:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.526+0000 7fba383b0740 1 -- start start 2024-05-27T07:50:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.526+0000 7fba383b0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525aff04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.526+0000 7fba383b0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac274d0 0x56525afeb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.526+0000 7fba383b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac481b0 0x56525aff6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.526+0000 7fba383b0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1b5c80 con 0x56525ac1f2f0 2024-05-27T07:50:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.526+0000 7fba383b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1b3550 con 0x56525ac481b0 2024-05-27T07:50:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.526+0000 7fba383b0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1b13c0 con 0x56525ac274d0 2024-05-27T07:50:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba2a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525aff04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba2a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525aff04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51774/0 (socket says 172.21.15.63:51774) 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba2a800640 1 -- 172.21.15.63:0/2151595364 learned_addr learned my addr 172.21.15.63:0/2151595364 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba2b200640 1 --2- 172.21.15.63:0/2151595364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac481b0 0x56525aff6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba29e00640 1 --2- 172.21.15.63:0/2151595364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac274d0 0x56525afeb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba2a800640 1 -- 172.21.15.63:0/2151595364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac274d0 msgr2=0x56525afeb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba2a800640 1 --2- 172.21.15.63:0/2151595364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac274d0 0x56525afeb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba2a800640 1 -- 172.21.15.63:0/2151595364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac481b0 msgr2=0x56525aff6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba2a800640 1 --2- 172.21.15.63:0/2151595364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac481b0 0x56525aff6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba2a800640 1 -- 172.21.15.63:0/2151595364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525afe5610 con 0x56525ac1f2f0 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.527+0000 7fba2a800640 1 --2- 172.21.15.63:0/2151595364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525aff04e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x56525afe81a0 tx=0x7fba1c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f5957ace7d8cf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.528+0000 7fba16a00640 1 -- 172.21.15.63:0/2151595364 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c004d20 con 0x56525ac1f2f0 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.528+0000 7fba16a00640 1 -- 172.21.15.63:0/2151595364 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba1c01ca50 con 0x56525ac1f2f0 2024-05-27T07:50:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.528+0000 7fba16a00640 1 -- 172.21.15.63:0/2151595364 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c005580 con 0x56525ac1f2f0 2024-05-27T07:50:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.529+0000 7fba383b0740 1 -- 172.21.15.63:0/2151595364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 msgr2=0x56525aff04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.529+0000 7fba383b0740 1 --2- 172.21.15.63:0/2151595364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525aff04e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x56525afe81a0 tx=0x7fba1c016000 comp rx=0 tx=0).stop 2024-05-27T07:50:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.529+0000 7fba383b0740 1 -- 172.21.15.63:0/2151595364 shutdown_connections 2024-05-27T07:50:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.529+0000 7fba383b0740 1 --2- 172.21.15.63:0/2151595364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac481b0 0x56525aff6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.529+0000 7fba383b0740 1 --2- 172.21.15.63:0/2151595364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac274d0 0x56525afeb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.529+0000 7fba383b0740 1 --2- 172.21.15.63:0/2151595364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525aff04e0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.529+0000 7fba383b0740 1 -- 172.21.15.63:0/2151595364 >> 172.21.15.63:0/2151595364 conn(0x56525ac3c300 msgr2=0x56525aff6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.529+0000 7fba383b0740 1 -- 172.21.15.63:0/2151595364 shutdown_connections 2024-05-27T07:50:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.529+0000 7fba383b0740 1 -- 172.21.15.63:0/2151595364 wait complete. 2024-05-27T07:50:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.530+0000 7fba383b0740 1 -- start start 2024-05-27T07:50:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.530+0000 7fba383b0740 1 Processor -- start 2024-05-27T07:50:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.530+0000 7fba383b0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525b006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.530+0000 7fba383b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac274d0 0x56525b007200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.530+0000 7fba383b0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525b007740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.530+0000 7fba383b0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1b1a10 con 0x56525ac1f2f0 2024-05-27T07:50:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.530+0000 7fba383b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1b2e80 con 0x56525ac274d0 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.530+0000 7fba383b0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1b20e0 con 0x56525ac481b0 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.530+0000 7fba383b0740 10 client.0 _ll_register_callbacks cb 0x7fba1c009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba2a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525b006cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba2a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525b006cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51776/0 (socket says 172.21.15.63:51776) 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba2a800640 1 -- 172.21.15.63:0/1583445255 learned_addr learned my addr 172.21.15.63:0/1583445255 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba29e00640 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac274d0 0x56525b007200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba2b200640 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525b007740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba2a800640 1 -- 172.21.15.63:0/1583445255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 msgr2=0x56525b007740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba2a800640 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525b007740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba2a800640 1 -- 172.21.15.63:0/1583445255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac274d0 msgr2=0x56525b007200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba2a800640 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac274d0 0x56525b007200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba2a800640 1 -- 172.21.15.63:0/1583445255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525afea7f0 con 0x56525ac1f2f0 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.531+0000 7fba2b200640 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525b007740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.532+0000 7fba2a800640 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525b006cc0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fba1c01f040 tx=0x7fba1c01cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f659aa471b99e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.532+0000 7fba0be00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.532+0000 7fba0be00640 1 -- 172.21.15.63:0/1583445255 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c038060 con 0x56525ac1f2f0 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.532+0000 7fba0be00640 1 -- 172.21.15.63:0/1583445255 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9571+0+0 (secure 0 0 0) 0x7fba1c01c7d0 con 0x56525ac1f2f0 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.532+0000 7fba383b0740 1 -- 172.21.15.63:0/1583445255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1c01ca50 con 0x56525ac1f2f0 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.532+0000 7fba0be00640 1 -- 172.21.15.63:0/1583445255 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fba1c004a40 con 0x56525ac1f2f0 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.532+0000 7fba383b0740 10 client.18642 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.532+0000 7fba383b0740 1 -- 172.21.15.63:0/1583445255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56525afe8360 con 0x56525ac1f2f0 2024-05-27T07:50:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.533+0000 7fba0be00640 1 -- 172.21.15.63:0/1583445255 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba1c01ca50 con 0x56525ac1f2f0 2024-05-27T07:50:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.533+0000 7fba0be00640 1 -- 172.21.15.63:0/1583445255 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fba1c01a040 con 0x56525ac1f2f0 2024-05-27T07:50:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.533+0000 7fba383b0740 10 client.18642 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.533+0000 7fba383b0740 10 client.18642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.533+0000 7fba383b0740 20 client.18642 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.533+0000 7fba383b0740 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56525abdc1c0 0x56525b070890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.533+0000 7fba383b0740 4 client.18642 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:50:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.533+0000 7fba383b0740 1 -- 172.21.15.63:0/1583445255 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56525aed14b0 con 0x56525abdc1c0 2024-05-27T07:50:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.534+0000 7fba29e00640 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56525abdc1c0 0x56525b070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.535+0000 7fba29e00640 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56525abdc1c0 0x56525b070890 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60be85c975b37c0 server_cookie=30ea71279baca164 in_seq=0 out_seq=0 2024-05-27T07:50:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.535+0000 7fba0be00640 10 client.18642 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba0be00640 1 -- 172.21.15.63:0/1583445255 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56525aed14b0 con 0x56525abdc1c0 2024-05-27T07:50:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba0be00640 10 client.18642 handle_command_reply: tid=0 2024-05-27T07:50:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 client.18642 shutdown 2024-05-27T07:50:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 -- 172.21.15.63:0/1583445255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 msgr2=0x56525b006cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525b006cc0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fba1c01f040 tx=0x7fba1c01cd60 comp rx=0 tx=0).stop 2024-05-27T07:50:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 -- 172.21.15.63:0/1583445255 shutdown_connections 2024-05-27T07:50:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56525abdc1c0 0x56525b070890 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56525ac481b0 0x56525b007740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56525ac274d0 0x56525b007200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 --2- 172.21.15.63:0/1583445255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56525ac1f2f0 0x56525b006cc0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba0be00640 0 client.18642 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 -- 172.21.15.63:0/1583445255 >> 172.21.15.63:0/1583445255 conn(0x56525ac3c300 msgr2=0x56525afeb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 -- 172.21.15.63:0/1583445255 shutdown_connections 2024-05-27T07:50:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.569+0000 7fba383b0740 1 -- 172.21.15.63:0/1583445255 wait complete. 2024-05-27T07:50:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.570+0000 7fba383b0740 20 client.18642 trim_cache size 0 max 16384 2024-05-27T07:50:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:50:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:50:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:50:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.570+0000 7fba30a00640 1 -- 172.21.15.63:0/2563005599 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba100b2950 msgr2=0x7fba100b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.570+0000 7fba30a00640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba100b2950 0x7fba100b4e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x56525afebe10 tx=0x7fba1c009680 comp rx=0 tx=0).stop 2024-05-27T07:50:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.570+0000 7fba30a00640 1 -- 172.21.15.63:0/2563005599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba2c110320 msgr2=0x7fba2c1a1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.570+0000 7fba30a00640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba2c110320 0x7fba2c1a1020 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fba2400d750 tx=0x7fba2400dc10 comp rx=0 tx=0).stop 2024-05-27T07:50:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.571+0000 7fba30a00640 1 -- 172.21.15.63:0/2563005599 shutdown_connections 2024-05-27T07:50:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.571+0000 7fba30a00640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba100b2950 0x7fba100b4e60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.571+0000 7fba30a00640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba2c110320 0x7fba2c1a1020 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.571+0000 7fba30a00640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba2c10d9c0 0x7fba2c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.571+0000 7fba30a00640 1 --2- 172.21.15.63:0/2563005599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c075420 0x7fba2c19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.571+0000 7fba30a00640 1 -- 172.21.15.63:0/2563005599 >> 172.21.15.63:0/2563005599 conn(0x7fba2c07a1b0 msgr2=0x7fba2c1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.571+0000 7fba30a00640 1 -- 172.21.15.63:0/2563005599 shutdown_connections 2024-05-27T07:50:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:12.571+0000 7fba30a00640 1 -- 172.21.15.63:0/2563005599 wait complete. 2024-05-27T07:50:13.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:12 smithi086 ceph-mon[34687]: pgmap v523: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 8 op/s 2024-05-27T07:50:13.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:12 smithi149 ceph-mon[35408]: pgmap v523: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 8 op/s 2024-05-27T07:50:13.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:12 smithi063 ceph-mon[31613]: pgmap v523: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 8 op/s 2024-05-27T07:50:15.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:14 smithi149 ceph-mon[35408]: pgmap v524: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 8 op/s 2024-05-27T07:50:15.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:14 smithi063 ceph-mon[31613]: pgmap v524: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 8 op/s 2024-05-27T07:50:15.458 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:14 smithi086 ceph-mon[34687]: pgmap v524: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 8 op/s 2024-05-27T07:50:16.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:16 smithi086 ceph-mon[34687]: osdmap e115: 12 total, 12 up, 12 in 2024-05-27T07:50:16.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:16 smithi086 ceph-mon[34687]: pgmap v526: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-05-27T07:50:16.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:16 smithi149 ceph-mon[35408]: osdmap e115: 12 total, 12 up, 12 in 2024-05-27T07:50:16.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:16 smithi149 ceph-mon[35408]: pgmap v526: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-05-27T07:50:16.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:16 smithi063 ceph-mon[31613]: osdmap e115: 12 total, 12 up, 12 in 2024-05-27T07:50:16.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:16 smithi063 ceph-mon[31613]: pgmap v526: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-05-27T07:50:19.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:18 smithi086 ceph-mon[34687]: pgmap v527: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:50:19.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:18 smithi149 ceph-mon[35408]: pgmap v527: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:50:19.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:18 smithi063 ceph-mon[31613]: pgmap v527: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:50:21.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:20 smithi086 ceph-mon[34687]: pgmap v528: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:50:21.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:20 smithi149 ceph-mon[35408]: pgmap v528: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:50:21.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:20 smithi063 ceph-mon[31613]: pgmap v528: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:50:23.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:22 smithi086 ceph-mon[34687]: pgmap v529: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:50:23.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:22 smithi149 ceph-mon[35408]: pgmap v529: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:50:23.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:22 smithi063 ceph-mon[31613]: pgmap v529: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:50:25.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:24 smithi086 ceph-mon[34687]: pgmap v530: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:50:25.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:24 smithi149 ceph-mon[35408]: pgmap v530: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:50:25.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:24 smithi063 ceph-mon[31613]: pgmap v530: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:50:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.813+0000 7efe3ec00640 1 -- 172.21.15.63:0/510244968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe3810b610 msgr2=0x7efe3810da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.813+0000 7efe3ec00640 1 --2- 172.21.15.63:0/510244968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe3810b610 0x7efe3810da30 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7efe24009900 tx=0x7efe2402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.813+0000 7efe3ec00640 1 -- 172.21.15.63:0/510244968 shutdown_connections 2024-05-27T07:50:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.813+0000 7efe3ec00640 1 --2- 172.21.15.63:0/510244968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe3810e000 0x7efe381104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.813+0000 7efe3ec00640 1 --2- 172.21.15.63:0/510244968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe3810b610 0x7efe3810da30 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.813+0000 7efe3ec00640 1 --2- 172.21.15.63:0/510244968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe38104bb0 0x7efe38104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.813+0000 7efe3ec00640 1 -- 172.21.15.63:0/510244968 >> 172.21.15.63:0/510244968 conn(0x7efe380fe210 msgr2=0x7efe38100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.813+0000 7efe3ec00640 1 -- 172.21.15.63:0/510244968 shutdown_connections 2024-05-27T07:50:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.814+0000 7efe3ec00640 1 -- 172.21.15.63:0/510244968 wait complete. 2024-05-27T07:50:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.815+0000 7efe3ec00640 1 Processor -- start 2024-05-27T07:50:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.815+0000 7efe3ec00640 1 -- start start 2024-05-27T07:50:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.815+0000 7efe3ec00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe38104bb0 0x7efe3819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.815+0000 7efe3ec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe3810b610 0x7efe3819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3810e000 0x7efe381a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3ec00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe38113b10 con 0x7efe38104bb0 2024-05-27T07:50:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe38113990 con 0x7efe3810e000 2024-05-27T07:50:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3ec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe38113c90 con 0x7efe3810b610 2024-05-27T07:50:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe38104bb0 0x7efe3819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe38104bb0 0x7efe3819c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38726/0 (socket says 172.21.15.63:38726) 2024-05-27T07:50:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3d800640 1 -- 172.21.15.63:0/3625873877 learned_addr learned my addr 172.21.15.63:0/3625873877 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3ce00640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe3810b610 0x7efe3819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3e200640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3810e000 0x7efe381a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3d800640 1 -- 172.21.15.63:0/3625873877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe3810b610 msgr2=0x7efe3819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3d800640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe3810b610 0x7efe3819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3d800640 1 -- 172.21.15.63:0/3625873877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3810e000 msgr2=0x7efe381a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3d800640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3810e000 0x7efe381a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.816+0000 7efe3d800640 1 -- 172.21.15.63:0/3625873877 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe381a15b0 con 0x7efe38104bb0 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.817+0000 7efe3e200640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3810e000 0x7efe381a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.817+0000 7efe3d800640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe38104bb0 0x7efe3819c7d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00d980 tx=0x7efe2c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.817+0000 7efe36000640 1 -- 172.21.15.63:0/3625873877 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c017070 con 0x7efe38104bb0 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.817+0000 7efe36000640 1 -- 172.21.15.63:0/3625873877 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe2c009c30 con 0x7efe38104bb0 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.817+0000 7efe3ec00640 1 -- 172.21.15.63:0/3625873877 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe381a18a0 con 0x7efe38104bb0 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.817+0000 7efe36000640 1 -- 172.21.15.63:0/3625873877 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c00f3c0 con 0x7efe38104bb0 2024-05-27T07:50:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.817+0000 7efe3ec00640 1 -- 172.21.15.63:0/3625873877 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe381a1dc0 con 0x7efe38104bb0 2024-05-27T07:50:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.821+0000 7efe4718b740 1 Processor -- start 2024-05-27T07:50:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.821+0000 7efe4718b740 1 -- start start 2024-05-27T07:50:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.822+0000 7efe4718b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21541b0 0x55efc24f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.822+0000 7efe4718b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc212b2f0 0x55efc24f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.822+0000 7efe4718b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc24f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.822+0000 7efe4718b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55efc2467b90 con 0x55efc21334d0 2024-05-27T07:50:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.822+0000 7efe4718b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55efc2457730 con 0x55efc212b2f0 2024-05-27T07:50:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.822+0000 7efe4718b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55efc245b770 con 0x55efc21541b0 2024-05-27T07:50:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.822+0000 7efe36000640 1 -- 172.21.15.63:0/3625873877 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efe2c010460 con 0x7efe38104bb0 2024-05-27T07:50:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.822+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc24f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.822+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc24f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38730/0 (socket says 172.21.15.63:38730) 2024-05-27T07:50:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.823+0000 7efe3ce00640 1 -- 172.21.15.63:0/1620218727 learned_addr learned my addr 172.21.15.63:0/1620218727 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.823+0000 7efe3e200640 1 --2- 172.21.15.63:0/1620218727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21541b0 0x55efc24f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.823+0000 7efe3d800640 1 --2- 172.21.15.63:0/1620218727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc212b2f0 0x55efc24f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.823+0000 7efe3ce00640 1 -- 172.21.15.63:0/1620218727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21541b0 msgr2=0x55efc24f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.823+0000 7efe3ce00640 1 --2- 172.21.15.63:0/1620218727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21541b0 0x55efc24f6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.823+0000 7efe3ce00640 1 -- 172.21.15.63:0/1620218727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc212b2f0 msgr2=0x55efc24f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.823+0000 7efe3ce00640 1 --2- 172.21.15.63:0/1620218727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc212b2f0 0x55efc24f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.823+0000 7efe3ce00640 1 -- 172.21.15.63:0/1620218727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc23f08b0 con 0x55efc21334d0 2024-05-27T07:50:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.824+0000 7efe3ce00640 1 --2- 172.21.15.63:0/1620218727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc24f74e0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7efe2402a910 tx=0x7efe24032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3675571fc9814a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.824+0000 7efe2aa00640 1 -- 172.21.15.63:0/1620218727 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2403f070 con 0x55efc21334d0 2024-05-27T07:50:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.824+0000 7efe2aa00640 1 -- 172.21.15.63:0/1620218727 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe240044c0 con 0x55efc21334d0 2024-05-27T07:50:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.824+0000 7efe2aa00640 1 -- 172.21.15.63:0/1620218727 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24035070 con 0x55efc21334d0 2024-05-27T07:50:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.824+0000 7efe36000640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efe200b2950 0x7efe200b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.825+0000 7efe36000640 1 -- 172.21.15.63:0/3625873877 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7efe2c0e26a0 con 0x7efe38104bb0 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.825+0000 7efe3e200640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efe200b2950 0x7efe200b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.825+0000 7efe4718b740 1 -- 172.21.15.63:0/1620218727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 msgr2=0x55efc24f74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.825+0000 7efe4718b740 1 --2- 172.21.15.63:0/1620218727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc24f74e0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7efe2402a910 tx=0x7efe24032dd0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.826+0000 7efe4718b740 1 -- 172.21.15.63:0/1620218727 shutdown_connections 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.826+0000 7efe4718b740 1 --2- 172.21.15.63:0/1620218727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc24f74e0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.826+0000 7efe4718b740 1 --2- 172.21.15.63:0/1620218727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc212b2f0 0x55efc24f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.826+0000 7efe4718b740 1 --2- 172.21.15.63:0/1620218727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21541b0 0x55efc24f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.826+0000 7efe4718b740 1 -- 172.21.15.63:0/1620218727 >> 172.21.15.63:0/1620218727 conn(0x55efc2148300 msgr2=0x55efc24f3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.826+0000 7efe4718b740 1 -- 172.21.15.63:0/1620218727 shutdown_connections 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.826+0000 7efe3e200640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efe200b2950 0x7efe200b4e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55efc24f3bd0 tx=0x7efe30008620 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.826+0000 7efe4718b740 1 -- 172.21.15.63:0/1620218727 wait complete. 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.826+0000 7efe4718b740 1 -- start start 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.827+0000 7efe4718b740 1 Processor -- start 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.827+0000 7efe4718b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc212b2f0 0x55efc2500c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.827+0000 7efe4718b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc21334d0 0x55efc25011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.827+0000 7efe4718b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21541b0 0x55efc250ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.827+0000 7efe4718b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55efc213d790 con 0x55efc212b2f0 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.827+0000 7efe4718b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55efc23ffbc0 con 0x55efc21334d0 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.827+0000 7efe4718b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55efc2503b30 con 0x55efc21541b0 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.827+0000 7efe4718b740 10 client.0 _ll_register_callbacks cb 0x55efc23dd400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.828+0000 7efe3d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc212b2f0 0x55efc2500c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.828+0000 7efe3d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc212b2f0 0x55efc2500c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38742/0 (socket says 172.21.15.63:38742) 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.828+0000 7efe3d800640 1 -- 172.21.15.63:0/3618957409 learned_addr learned my addr 172.21.15.63:0/3618957409 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.828+0000 7efe3ce00640 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc21334d0 0x55efc25011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.828+0000 7efe3e200640 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21541b0 0x55efc250ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.828+0000 7efe3d800640 1 -- 172.21.15.63:0/3618957409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21541b0 msgr2=0x55efc250ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.828+0000 7efe3d800640 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21541b0 0x55efc250ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.828+0000 7efe3d800640 1 -- 172.21.15.63:0/3618957409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc21334d0 msgr2=0x55efc25011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.828+0000 7efe3d800640 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc21334d0 0x55efc25011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.828+0000 7efe3d800640 1 -- 172.21.15.63:0/3618957409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc216c210 con 0x55efc212b2f0 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.829+0000 7efe3d800640 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc212b2f0 0x55efc2500c80 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00d950 tx=0x7efe2c0a90d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675b300309ef1c21 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.829+0000 7efe1f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.829+0000 7efe1f400640 1 -- 172.21.15.63:0/3618957409 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c020070 con 0x55efc212b2f0 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.829+0000 7efe1f400640 1 -- 172.21.15.63:0/3618957409 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7efe2c0a5850 con 0x55efc212b2f0 2024-05-27T07:50:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.829+0000 7efe4718b740 1 -- 172.21.15.63:0/3618957409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55efc2469ae0 con 0x55efc212b2f0 2024-05-27T07:50:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.829+0000 7efe1f400640 1 -- 172.21.15.63:0/3618957409 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efe2c0a9cc0 con 0x55efc212b2f0 2024-05-27T07:50:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.829+0000 7efe4718b740 10 client.18654 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.829+0000 7efe4718b740 1 -- 172.21.15.63:0/3618957409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55efc2238620 con 0x55efc212b2f0 2024-05-27T07:50:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.829+0000 7efe1f400640 1 -- 172.21.15.63:0/3618957409 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55efc2469ae0 con 0x55efc212b2f0 2024-05-27T07:50:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.830+0000 7efe1f400640 1 -- 172.21.15.63:0/3618957409 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7efe2c0a3ce0 con 0x55efc212b2f0 2024-05-27T07:50:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.830+0000 7efe4718b740 10 client.18654 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.830+0000 7efe4718b740 10 client.18654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.830+0000 7efe4718b740 20 client.18654 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.830+0000 7efe4718b740 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc2112d80 0x55efc2512e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.830+0000 7efe4718b740 4 client.18654 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.830+0000 7efe4718b740 1 -- 172.21.15.63:0/3618957409 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc24632c0 con 0x55efc2112d80 2024-05-27T07:50:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.831+0000 7efe3ce00640 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc2112d80 0x55efc2512e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.832+0000 7efe3ce00640 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc2112d80 0x55efc2512e70 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11be2d9df2515d86 server_cookie=73aaa76b43022677 in_seq=0 out_seq=0 2024-05-27T07:50:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.832+0000 7efe1f400640 10 client.18654 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe1f400640 1 -- 172.21.15.63:0/3618957409 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55efc24632c0 con 0x55efc2112d80 2024-05-27T07:50:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe1f400640 10 client.18654 handle_command_reply: tid=0 2024-05-27T07:50:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 client.18654 shutdown 2024-05-27T07:50:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 -- 172.21.15.63:0/3618957409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc212b2f0 msgr2=0x55efc2500c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc212b2f0 0x55efc2500c80 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00d950 tx=0x7efe2c0a90d0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 -- 172.21.15.63:0/3618957409 shutdown_connections 2024-05-27T07:50:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc2112d80 0x55efc2512e70 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21541b0 0x55efc250ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc21334d0 0x55efc25011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 --2- 172.21.15.63:0/3618957409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc212b2f0 0x55efc2500c80 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 -- 172.21.15.63:0/3618957409 >> 172.21.15.63:0/3618957409 conn(0x55efc2148300 msgr2=0x55efc24f8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe1f400640 0 client.18654 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 -- 172.21.15.63:0/3618957409 shutdown_connections 2024-05-27T07:50:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.834+0000 7efe4718b740 1 -- 172.21.15.63:0/3618957409 wait complete. 2024-05-27T07:50:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.835+0000 7efe4718b740 20 client.18654 trim_cache size 0 max 16384 2024-05-27T07:50:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.851+0000 7efe4718b740 1 Processor -- start 2024-05-27T07:50:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.851+0000 7efe4718b740 1 -- start start 2024-05-27T07:50:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe4718b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc212b2f0 0x55efc24fc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe4718b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc2500f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe4718b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc21541b0 0x55efc250cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe4718b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b5a40 con 0x55efc21334d0 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe4718b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b3310 con 0x55efc21541b0 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe4718b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b1180 con 0x55efc212b2f0 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc2500f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe3ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc2500f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38756/0 (socket says 172.21.15.63:38756) 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe3ce00640 1 -- 172.21.15.63:0/4183002846 learned_addr learned my addr 172.21.15.63:0/4183002846 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe3d800640 1 --2- 172.21.15.63:0/4183002846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc212b2f0 0x55efc24fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe3ce00640 1 -- 172.21.15.63:0/4183002846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc212b2f0 msgr2=0x55efc24fc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe3ce00640 1 --2- 172.21.15.63:0/4183002846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc212b2f0 0x55efc24fc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe3ce00640 1 -- 172.21.15.63:0/4183002846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc21541b0 msgr2=0x55efc250cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe3ce00640 1 --2- 172.21.15.63:0/4183002846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc21541b0 0x55efc250cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe3ce00640 1 -- 172.21.15.63:0/4183002846 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc24ff1d0 con 0x55efc21334d0 2024-05-27T07:50:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.852+0000 7efe3d800640 1 --2- 172.21.15.63:0/4183002846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc212b2f0 0x55efc24fc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.853+0000 7efe3ce00640 1 --2- 172.21.15.63:0/4183002846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc2500f40 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7efe240057c0 tx=0x7efe24031c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b062a08b74c880a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.853+0000 7efe2a000640 1 -- 172.21.15.63:0/4183002846 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24035030 con 0x55efc21334d0 2024-05-27T07:50:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.853+0000 7efe2a000640 1 -- 172.21.15.63:0/4183002846 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe24031600 con 0x55efc21334d0 2024-05-27T07:50:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.853+0000 7efe2a000640 1 -- 172.21.15.63:0/4183002846 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2403f040 con 0x55efc21334d0 2024-05-27T07:50:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.854+0000 7efe4718b740 1 -- 172.21.15.63:0/4183002846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 msgr2=0x55efc2500f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.854+0000 7efe4718b740 1 --2- 172.21.15.63:0/4183002846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc2500f40 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7efe240057c0 tx=0x7efe24031c00 comp rx=0 tx=0).stop 2024-05-27T07:50:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.854+0000 7efe4718b740 1 -- 172.21.15.63:0/4183002846 shutdown_connections 2024-05-27T07:50:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.854+0000 7efe4718b740 1 --2- 172.21.15.63:0/4183002846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc21541b0 0x55efc250cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.854+0000 7efe4718b740 1 --2- 172.21.15.63:0/4183002846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21334d0 0x55efc2500f40 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.854+0000 7efe4718b740 1 --2- 172.21.15.63:0/4183002846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc212b2f0 0x55efc24fc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.854+0000 7efe4718b740 1 -- 172.21.15.63:0/4183002846 >> 172.21.15.63:0/4183002846 conn(0x55efc2148300 msgr2=0x55efc24f72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.854+0000 7efe4718b740 1 -- 172.21.15.63:0/4183002846 shutdown_connections 2024-05-27T07:50:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.854+0000 7efe4718b740 1 -- 172.21.15.63:0/4183002846 wait complete. 2024-05-27T07:50:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.855+0000 7efe4718b740 1 -- start start 2024-05-27T07:50:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.855+0000 7efe4718b740 1 Processor -- start 2024-05-27T07:50:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe4718b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc212b2f0 0x55efc2503d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe4718b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21334d0 0x55efc25042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe4718b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21541b0 0x55efc2504800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe4718b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b17d0 con 0x55efc21541b0 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe4718b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b2c40 con 0x55efc212b2f0 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe4718b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b1ea0 con 0x55efc21334d0 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe4718b740 10 client.0 _ll_register_callbacks cb 0x7efe2c0a8f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe3e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21541b0 0x55efc2504800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe3e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21541b0 0x55efc2504800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38764/0 (socket says 172.21.15.63:38764) 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe3e200640 1 -- 172.21.15.63:0/2420873555 learned_addr learned my addr 172.21.15.63:0/2420873555 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe3ce00640 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21334d0 0x55efc25042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.856+0000 7efe3d800640 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc212b2f0 0x55efc2503d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.857+0000 7efe3e200640 1 -- 172.21.15.63:0/2420873555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21334d0 msgr2=0x55efc25042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.857+0000 7efe3e200640 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21334d0 0x55efc25042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.857+0000 7efe3e200640 1 -- 172.21.15.63:0/2420873555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc212b2f0 msgr2=0x55efc2503d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.857+0000 7efe3e200640 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc212b2f0 0x55efc2503d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.857+0000 7efe3e200640 1 -- 172.21.15.63:0/2420873555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc25795c0 con 0x55efc21541b0 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.857+0000 7efe3ce00640 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21334d0 0x55efc25042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.857+0000 7efe3e200640 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21541b0 0x55efc2504800 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7efe3001bd60 tx=0x7efe30004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4d7cb40136e722 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.857+0000 7efe1f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.857+0000 7efe1f400640 1 -- 172.21.15.63:0/2420873555 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe300243f0 con 0x55efc21541b0 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.858+0000 7efe4718b740 1 -- 172.21.15.63:0/2420873555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe24031600 con 0x55efc21541b0 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.858+0000 7efe1f400640 1 -- 172.21.15.63:0/2420873555 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7efe30024590 con 0x55efc21541b0 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.858+0000 7efe1f400640 1 -- 172.21.15.63:0/2420873555 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efe30031050 con 0x55efc21541b0 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.858+0000 7efe4718b740 10 client.18660 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.858+0000 7efe4718b740 1 -- 172.21.15.63:0/2420873555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55efc257a810 con 0x55efc21541b0 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.858+0000 7efe1f400640 1 -- 172.21.15.63:0/2420873555 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe24031600 con 0x55efc21541b0 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.858+0000 7efe1f400640 1 -- 172.21.15.63:0/2420873555 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7efe3002c030 con 0x55efc21541b0 2024-05-27T07:50:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.859+0000 7efe4718b740 10 client.18660 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.859+0000 7efe4718b740 10 client.18660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.859+0000 7efe4718b740 20 client.18660 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.859+0000 7efe4718b740 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc20e81c0 0x55efc257d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.859+0000 7efe4718b740 4 client.18660 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:50:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.859+0000 7efe4718b740 1 -- 172.21.15.63:0/2420873555 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55efc23dd4b0 con 0x55efc20e81c0 2024-05-27T07:50:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.860+0000 7efe3d800640 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc20e81c0 0x55efc257d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.861+0000 7efe3d800640 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc20e81c0 0x55efc257d030 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89c067dfc6c7bdf2 server_cookie=fb2dcbf68a9fec89 in_seq=0 out_seq=0 2024-05-27T07:50:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.861+0000 7efe1f400640 10 client.18660 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.892+0000 7efe1f400640 1 -- 172.21.15.63:0/2420873555 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55efc23dd4b0 con 0x55efc20e81c0 2024-05-27T07:50:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.892+0000 7efe1f400640 10 client.18660 handle_command_reply: tid=0 2024-05-27T07:50:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.892+0000 7efe4718b740 1 client.18660 shutdown 2024-05-27T07:50:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.892+0000 7efe4718b740 1 -- 172.21.15.63:0/2420873555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21541b0 msgr2=0x55efc2504800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.892+0000 7efe4718b740 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21541b0 0x55efc2504800 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7efe3001bd60 tx=0x7efe30004ab0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.893+0000 7efe4718b740 1 -- 172.21.15.63:0/2420873555 shutdown_connections 2024-05-27T07:50:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.893+0000 7efe4718b740 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55efc20e81c0 0x55efc257d030 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.893+0000 7efe4718b740 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55efc21541b0 0x55efc2504800 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.893+0000 7efe1f400640 0 client.18660 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.893+0000 7efe4718b740 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc21334d0 0x55efc25042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.893+0000 7efe4718b740 1 --2- 172.21.15.63:0/2420873555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55efc212b2f0 0x55efc2503d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.893+0000 7efe4718b740 1 -- 172.21.15.63:0/2420873555 >> 172.21.15.63:0/2420873555 conn(0x55efc2148300 msgr2=0x55efc24f72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.893+0000 7efe4718b740 1 -- 172.21.15.63:0/2420873555 shutdown_connections 2024-05-27T07:50:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.893+0000 7efe4718b740 1 -- 172.21.15.63:0/2420873555 wait complete. 2024-05-27T07:50:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.894+0000 7efe4718b740 20 client.18660 trim_cache size 0 max 16384 2024-05-27T07:50:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:50:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:50:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:50:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.895+0000 7efe3ec00640 1 -- 172.21.15.63:0/3625873877 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efe200b2950 msgr2=0x7efe200b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.895+0000 7efe3ec00640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efe200b2950 0x7efe200b4e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55efc24f3bd0 tx=0x7efe30008620 comp rx=0 tx=0).stop 2024-05-27T07:50:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.895+0000 7efe3ec00640 1 -- 172.21.15.63:0/3625873877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe38104bb0 msgr2=0x7efe3819c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.895+0000 7efe3ec00640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe38104bb0 0x7efe3819c7d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00d980 tx=0x7efe2c00de40 comp rx=0 tx=0).stop 2024-05-27T07:50:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.895+0000 7efe3ec00640 1 -- 172.21.15.63:0/3625873877 shutdown_connections 2024-05-27T07:50:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.895+0000 7efe3ec00640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efe200b2950 0x7efe200b4e60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.895+0000 7efe3ec00640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe3810e000 0x7efe381a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.895+0000 7efe3ec00640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe3810b610 0x7efe3819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.895+0000 7efe3ec00640 1 --2- 172.21.15.63:0/3625873877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe38104bb0 0x7efe3819c7d0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.895+0000 7efe3ec00640 1 -- 172.21.15.63:0/3625873877 >> 172.21.15.63:0/3625873877 conn(0x7efe380fe210 msgr2=0x7efe38109db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.896+0000 7efe3ec00640 1 -- 172.21.15.63:0/3625873877 shutdown_connections 2024-05-27T07:50:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:25.896+0000 7efe3ec00640 1 -- 172.21.15.63:0/3625873877 wait complete. 2024-05-27T07:50:27.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:26 smithi086 ceph-mon[34687]: pgmap v531: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:50:27.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:26 smithi149 ceph-mon[35408]: pgmap v531: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:50:27.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:26 smithi063 ceph-mon[31613]: pgmap v531: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:50:29.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:28 smithi086 ceph-mon[34687]: pgmap v532: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 16 op/s 2024-05-27T07:50:29.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:28 smithi149 ceph-mon[35408]: pgmap v532: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 16 op/s 2024-05-27T07:50:29.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:28 smithi063 ceph-mon[31613]: pgmap v532: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 16 op/s 2024-05-27T07:50:31.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:30 smithi086 ceph-mon[34687]: pgmap v533: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 MiB/s wr, 9 op/s 2024-05-27T07:50:31.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:30 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:50:31.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:30 smithi149 ceph-mon[35408]: pgmap v533: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 MiB/s wr, 9 op/s 2024-05-27T07:50:31.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:30 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:50:31.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:30 smithi063 ceph-mon[31613]: pgmap v533: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.3 MiB/s wr, 9 op/s 2024-05-27T07:50:31.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:30 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:50:32.920 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.920 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.920 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.920 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.920 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:32 smithi063 ceph-mon[31613]: pgmap v534: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.4 MiB/s wr, 12 op/s 2024-05-27T07:50:32.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:32 smithi086 ceph-mon[34687]: pgmap v534: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.4 MiB/s wr, 12 op/s 2024-05-27T07:50:32.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:32.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:32 smithi149 ceph-mon[35408]: pgmap v534: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.4 MiB/s wr, 12 op/s 2024-05-27T07:50:33.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:33 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:50:33.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:33 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:50:33.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:33 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:33.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:33 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:50:33.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:33 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:50:33.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:33 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:33.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:33 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:50:33.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:33 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:50:33.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:33 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:50:34.421 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:50:34.927 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:34.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:34 smithi086 ceph-mon[34687]: pgmap v535: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:50:34.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:34 smithi149 ceph-mon[35408]: pgmap v535: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:50:34.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:34 smithi063 ceph-mon[31613]: pgmap v535: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:50:37.162 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:36 smithi063 ceph-mon[31613]: pgmap v536: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:50:37.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:36 smithi086 ceph-mon[34687]: pgmap v536: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:50:37.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:36 smithi149 ceph-mon[35408]: pgmap v536: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:50:37.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.609+0000 7fc480c00700 1 -- 172.21.15.63:0/2062635930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc47c105e40 msgr2=0x7fc47c10e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:37.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.609+0000 7fc480c00700 1 --2- 172.21.15.63:0/2062635930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc47c105e40 0x7fc47c10e390 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc468009990 tx=0x7fc468009ca0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.610+0000 7fc480c00700 1 -- 172.21.15.63:0/2062635930 shutdown_connections 2024-05-27T07:50:37.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.610+0000 7fc480c00700 1 --2- 172.21.15.63:0/2062635930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc47c10e920 0x7fc47c110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.610+0000 7fc480c00700 1 --2- 172.21.15.63:0/2062635930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c1054f0 0x7fc47c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.610+0000 7fc480c00700 1 --2- 172.21.15.63:0/2062635930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc47c105e40 0x7fc47c10e390 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.610+0000 7fc480c00700 1 -- 172.21.15.63:0/2062635930 >> 172.21.15.63:0/2062635930 conn(0x7fc47c0fef20 msgr2=0x7fc47c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:37.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.610+0000 7fc480c00700 1 -- 172.21.15.63:0/2062635930 shutdown_connections 2024-05-27T07:50:37.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.610+0000 7fc480c00700 1 -- 172.21.15.63:0/2062635930 wait complete. 2024-05-27T07:50:37.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.611+0000 7fc480c00700 1 Processor -- start 2024-05-27T07:50:37.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.612+0000 7fc480c00700 1 -- start start 2024-05-27T07:50:37.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.612+0000 7fc480c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc47c1054f0 0x7fc47c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.612+0000 7fc480c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc47c105e40 0x7fc47c19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.612+0000 7fc480c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c10e920 0x7fc47c1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.612+0000 7fc480c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c114300 con 0x7fc47c105e40 2024-05-27T07:50:37.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.612+0000 7fc480c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c114180 con 0x7fc47c1054f0 2024-05-27T07:50:37.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.612+0000 7fc480c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c114480 con 0x7fc47c10e920 2024-05-27T07:50:37.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc479400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc47c105e40 0x7fc47c19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc479400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc47c105e40 0x7fc47c19d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44208/0 (socket says 172.21.15.63:44208) 2024-05-27T07:50:37.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc479400700 1 -- 172.21.15.63:0/317067327 learned_addr learned my addr 172.21.15.63:0/317067327 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:37.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc479e00700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc47c1054f0 0x7fc47c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc47a800700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c10e920 0x7fc47c1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc479400700 1 -- 172.21.15.63:0/317067327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c10e920 msgr2=0x7fc47c1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:37.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc479400700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c10e920 0x7fc47c1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc479400700 1 -- 172.21.15.63:0/317067327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc47c1054f0 msgr2=0x7fc47c19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:37.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc479400700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc47c1054f0 0x7fc47c19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc479400700 1 -- 172.21.15.63:0/317067327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c1a1e80 con 0x7fc47c105e40 2024-05-27T07:50:37.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.613+0000 7fc47a800700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c10e920 0x7fc47c1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:37.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.614+0000 7fc479400700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc47c105e40 0x7fc47c19d550 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc468000c00 tx=0x7fc4680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:37.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.614+0000 7fc472a00700 1 -- 172.21.15.63:0/317067327 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468027070 con 0x7fc47c105e40 2024-05-27T07:50:37.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.614+0000 7fc480c00700 1 -- 172.21.15.63:0/317067327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47c1a2110 con 0x7fc47c105e40 2024-05-27T07:50:37.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.614+0000 7fc472a00700 1 -- 172.21.15.63:0/317067327 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc46800fb80 con 0x7fc47c105e40 2024-05-27T07:50:37.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.614+0000 7fc472a00700 1 -- 172.21.15.63:0/317067327 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468022d00 con 0x7fc47c105e40 2024-05-27T07:50:37.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.614+0000 7fc480c00700 1 -- 172.21.15.63:0/317067327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47c1a9a90 con 0x7fc47c105e40 2024-05-27T07:50:37.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.619+0000 7fc472a00700 1 -- 172.21.15.63:0/317067327 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc46802a070 con 0x7fc47c105e40 2024-05-27T07:50:37.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.620+0000 7fc48f6b0b80 1 Processor -- start 2024-05-27T07:50:37.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.621+0000 7fc48f6b0b80 1 -- start start 2024-05-27T07:50:37.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.621+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1e887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.621+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 0x55f3a1e82870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.621+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e831c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.621+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f3a1db0e40 con 0x55f3a1d03e40 2024-05-27T07:50:37.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.621+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3a1da09e0 con 0x55f3a1e82db0 2024-05-27T07:50:37.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.621+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f3a1da4a20 con 0x55f3a1e88d30 2024-05-27T07:50:37.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.621+0000 7fc472a00700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4600b29f0 0x7fc4600b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.621+0000 7fc472a00700 1 -- 172.21.15.63:0/317067327 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7fc4680e3a50 con 0x7fc47c105e40 2024-05-27T07:50:37.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc47a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1e887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc47a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1e887f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44212/0 (socket says 172.21.15.63:44212) 2024-05-27T07:50:37.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc47a800700 1 -- 172.21.15.63:0/3408403758 learned_addr learned my addr 172.21.15.63:0/3408403758 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:37.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc479400700 1 --2- 172.21.15.63:0/3408403758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e831c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc47a800700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4600b29f0 0x7fc4600b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc479e00700 1 --2- 172.21.15.63:0/3408403758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 0x55f3a1e82870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc47a800700 1 -- 172.21.15.63:0/3408403758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 msgr2=0x55f3a1e82870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:37.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc47a800700 1 --2- 172.21.15.63:0/3408403758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 0x55f3a1e82870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc47a800700 1 -- 172.21.15.63:0/3408403758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 msgr2=0x55f3a1e831c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:37.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc47a800700 1 --2- 172.21.15.63:0/3408403758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e831c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.622+0000 7fc47a800700 1 -- 172.21.15.63:0/3408403758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a1d69420 con 0x55f3a1d03e40 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.623+0000 7fc47a800700 1 --2- 172.21.15.63:0/3408403758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1e887f0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fc474007f00 tx=0x7fc474018640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839957fd825c7a4f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.623+0000 7fc47a800700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4600b29f0 0x7fc4600b4f00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc474009710 tx=0x7fc47400b000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.623+0000 7fc467400700 1 -- 172.21.15.63:0/3408403758 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47401e070 con 0x55f3a1d03e40 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.623+0000 7fc467400700 1 -- 172.21.15.63:0/3408403758 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc474009b90 con 0x55f3a1d03e40 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.623+0000 7fc467400700 1 -- 172.21.15.63:0/3408403758 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474005020 con 0x55f3a1d03e40 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.624+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3408403758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 msgr2=0x55f3a1e887f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.624+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/3408403758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1e887f0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fc474007f00 tx=0x7fc474018640 comp rx=0 tx=0).stop 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.625+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3408403758 shutdown_connections 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.625+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/3408403758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e831c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.625+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/3408403758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1e887f0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.625+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/3408403758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 0x55f3a1e82870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.625+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3408403758 >> 172.21.15.63:0/3408403758 conn(0x55f3a1d63fc0 msgr2=0x55f3a1e7d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:37.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.625+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3408403758 shutdown_connections 2024-05-27T07:50:37.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.625+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3408403758 wait complete. 2024-05-27T07:50:37.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.626+0000 7fc48f6b0b80 1 -- start start 2024-05-27T07:50:37.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.626+0000 7fc48f6b0b80 1 Processor -- start 2024-05-27T07:50:37.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1d03e40 0x55f3a1e8c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e82db0 0x55f3a1e8cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f3a1ca6fa0 con 0x55f3a1e88d30 2024-05-27T07:50:37.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3a1d41760 con 0x55f3a1d03e40 2024-05-27T07:50:37.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f3a1e8f740 con 0x55f3a1e82db0 2024-05-27T07:50:37.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc48f6b0b80 10 client.0 _ll_register_callbacks cb 0x55f3a1d4ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:37.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc47a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc479e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1d03e40 0x55f3a1e8c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc479400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e82db0 0x55f3a1e8cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc47a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44218/0 (socket says 172.21.15.63:44218) 2024-05-27T07:50:37.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.627+0000 7fc47a800700 1 -- 172.21.15.63:0/3007061721 learned_addr learned my addr 172.21.15.63:0/3007061721 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:37.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.628+0000 7fc47a800700 1 -- 172.21.15.63:0/3007061721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e82db0 msgr2=0x55f3a1e8cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:37.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.628+0000 7fc47a800700 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e82db0 0x55f3a1e8cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.628+0000 7fc47a800700 1 -- 172.21.15.63:0/3007061721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1d03e40 msgr2=0x55f3a1e8c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:37.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.628+0000 7fc47a800700 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1d03e40 0x55f3a1e8c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.628+0000 7fc47a800700 1 -- 172.21.15.63:0/3007061721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a1cf08d0 con 0x55f3a1e88d30 2024-05-27T07:50:37.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.628+0000 7fc47a800700 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8d010 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fc47401ac10 tx=0x7fc47401ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9432240280a30d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:37.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.628+0000 7fc45be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:37.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.629+0000 7fc45be00700 1 -- 172.21.15.63:0/3007061721 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474031020 con 0x55f3a1e88d30 2024-05-27T07:50:37.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.629+0000 7fc45be00700 1 -- 172.21.15.63:0/3007061721 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7fc4740078e0 con 0x55f3a1e88d30 2024-05-27T07:50:37.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.629+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3007061721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc474009b90 con 0x55f3a1e88d30 2024-05-27T07:50:37.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.629+0000 7fc45be00700 1 -- 172.21.15.63:0/3007061721 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc47403c050 con 0x55f3a1e88d30 2024-05-27T07:50:37.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.629+0000 7fc48f6b0b80 10 client.18672 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:37.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.629+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3007061721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f3a1d71060 con 0x55f3a1e88d30 2024-05-27T07:50:37.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.630+0000 7fc45be00700 1 -- 172.21.15.63:0/3007061721 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc474009b90 con 0x55f3a1e88d30 2024-05-27T07:50:37.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.630+0000 7fc45be00700 1 -- 172.21.15.63:0/3007061721 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc4740224a0 con 0x55f3a1e88d30 2024-05-27T07:50:37.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.630+0000 7fc48f6b0b80 10 client.18672 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:37.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.630+0000 7fc48f6b0b80 10 client.18672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:37.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.630+0000 7fc48f6b0b80 20 client.18672 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:37.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.630+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3a1e9c6d0 0x55f3a1e9eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.630+0000 7fc48f6b0b80 4 client.18672 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:37.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.630+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3007061721 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3a1db2d90 con 0x55f3a1e9c6d0 2024-05-27T07:50:37.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.631+0000 7fc479e00700 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3a1e9c6d0 0x55f3a1e9eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.632+0000 7fc479e00700 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3a1e9c6d0 0x55f3a1e9eaf0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9370dcbedf97eefe server_cookie=8b209df4e8446213 in_seq=0 out_seq=0 2024-05-27T07:50:37.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.632+0000 7fc45be00700 10 client.18672 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:37.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc45be00700 1 -- 172.21.15.63:0/3007061721 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f3a1db2d90 con 0x55f3a1e9c6d0 2024-05-27T07:50:37.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc45be00700 10 client.18672 handle_command_reply: tid=0 2024-05-27T07:50:37.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc48f6b0b80 1 client.18672 shutdown 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3007061721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 msgr2=0x55f3a1e8d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8d010 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fc47401ac10 tx=0x7fc47401ac40 comp rx=0 tx=0).stop 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3007061721 shutdown_connections 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3a1e9c6d0 0x55f3a1e9eaf0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8d010 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1d03e40 0x55f3a1e8c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/3007061721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e82db0 0x55f3a1e8cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3007061721 >> 172.21.15.63:0/3007061721 conn(0x55f3a1d63fc0 msgr2=0x55f3a1e7d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.634+0000 7fc45be00700 0 client.18672 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.635+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3007061721 shutdown_connections 2024-05-27T07:50:37.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.635+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/3007061721 wait complete. 2024-05-27T07:50:37.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.635+0000 7fc48f6b0b80 20 client.18672 trim_cache size 0 max 16384 2024-05-27T07:50:37.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.659+0000 7fc48f6b0b80 1 Processor -- start 2024-05-27T07:50:37.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.660+0000 7fc48f6b0b80 1 -- start start 2024-05-27T07:50:37.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.660+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1d03e40 0x55f3a1e82870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.660+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e8b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.660+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.660+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c1b4f80 con 0x55f3a1e88d30 2024-05-27T07:50:37.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.660+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c1b2810 con 0x55f3a1e82db0 2024-05-27T07:50:37.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.660+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c1b0680 con 0x55f3a1d03e40 2024-05-27T07:50:37.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc47a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc47a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44232/0 (socket says 172.21.15.63:44232) 2024-05-27T07:50:37.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc47a800700 1 -- 172.21.15.63:0/525913532 learned_addr learned my addr 172.21.15.63:0/525913532 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:37.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc479e00700 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1d03e40 0x55f3a1e82870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc479400700 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e8b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc47a800700 1 -- 172.21.15.63:0/525913532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1d03e40 msgr2=0x55f3a1e82870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:37.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc47a800700 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1d03e40 0x55f3a1e82870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc47a800700 1 -- 172.21.15.63:0/525913532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 msgr2=0x55f3a1e8b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:37.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc47a800700 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e8b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc47a800700 1 -- 172.21.15.63:0/525913532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a1e7f310 con 0x55f3a1e88d30 2024-05-27T07:50:37.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc479e00700 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1d03e40 0x55f3a1e82870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:37.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.661+0000 7fc479400700 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e8b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:37.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.662+0000 7fc47a800700 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8cf80 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc474018990 tx=0x7fc474009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89614045fa02beb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:37.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.662+0000 7fc466a00700 1 -- 172.21.15.63:0/525913532 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474031040 con 0x55f3a1e88d30 2024-05-27T07:50:37.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.662+0000 7fc466a00700 1 -- 172.21.15.63:0/525913532 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc47401acb0 con 0x55f3a1e88d30 2024-05-27T07:50:37.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.662+0000 7fc466a00700 1 -- 172.21.15.63:0/525913532 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47401e040 con 0x55f3a1e88d30 2024-05-27T07:50:37.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.663+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/525913532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 msgr2=0x55f3a1e8cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:37.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.663+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8cf80 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc474018990 tx=0x7fc474009db0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.663+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/525913532 shutdown_connections 2024-05-27T07:50:37.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.663+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1e88d30 0x55f3a1e8cf80 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.663+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1d03e40 0x55f3a1e82870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.663+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/525913532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e8b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.663+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/525913532 >> 172.21.15.63:0/525913532 conn(0x55f3a1d63fc0 msgr2=0x55f3a1e86210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:37.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.663+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/525913532 shutdown_connections 2024-05-27T07:50:37.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.663+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/525913532 wait complete. 2024-05-27T07:50:37.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.664+0000 7fc48f6b0b80 1 -- start start 2024-05-27T07:50:37.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.664+0000 7fc48f6b0b80 1 Processor -- start 2024-05-27T07:50:37.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.665+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1ea22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.665+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e9f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.665+0000 7fc48f6b0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 0x55f3a1e9fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.665+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c1b0cd0 con 0x55f3a1d03e40 2024-05-27T07:50:37.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.665+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c1b2140 con 0x55f3a1e82db0 2024-05-27T07:50:37.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.665+0000 7fc48f6b0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c1b13a0 con 0x55f3a1e88d30 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.665+0000 7fc48f6b0b80 10 client.0 _ll_register_callbacks cb 0x55f3a1d4ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.665+0000 7fc479e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1ea22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.665+0000 7fc479e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1ea22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44244/0 (socket says 172.21.15.63:44244) 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.665+0000 7fc479e00700 1 -- 172.21.15.63:0/2705734887 learned_addr learned my addr 172.21.15.63:0/2705734887 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc479400700 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e9f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc47a800700 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 0x55f3a1e9fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc479e00700 1 -- 172.21.15.63:0/2705734887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 msgr2=0x55f3a1e9fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc479e00700 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 0x55f3a1e9fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc479e00700 1 -- 172.21.15.63:0/2705734887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 msgr2=0x55f3a1e9f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc479e00700 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e9f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc479e00700 1 -- 172.21.15.63:0/2705734887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a1ea03b0 con 0x55f3a1d03e40 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc47a800700 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 0x55f3a1e9fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:37.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc479400700 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e9f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:37.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc479e00700 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1ea22a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x55f3a1ca7030 tx=0x7fc46c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f01ae8ad2cdf6a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:37.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.666+0000 7fc45be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.667+0000 7fc45be00700 1 -- 172.21.15.63:0/2705734887 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c004830 con 0x55f3a1d03e40 2024-05-27T07:50:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.667+0000 7fc45be00700 1 -- 172.21.15.63:0/2705734887 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7fc46c0049d0 con 0x55f3a1d03e40 2024-05-27T07:50:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.667+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/2705734887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc47401acb0 con 0x55f3a1d03e40 2024-05-27T07:50:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.667+0000 7fc45be00700 1 -- 172.21.15.63:0/2705734887 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc46c004bd0 con 0x55f3a1d03e40 2024-05-27T07:50:37.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.667+0000 7fc48f6b0b80 10 client.18678 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.667+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/2705734887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f3a1f06f40 con 0x55f3a1d03e40 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.668+0000 7fc45be00700 1 -- 172.21.15.63:0/2705734887 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc47401acb0 con 0x55f3a1d03e40 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.668+0000 7fc45be00700 1 -- 172.21.15.63:0/2705734887 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc46c0275d0 con 0x55f3a1d03e40 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.668+0000 7fc48f6b0b80 10 client.18678 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.668+0000 7fc48f6b0b80 10 client.18678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.668+0000 7fc48f6b0b80 20 client.18678 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.668+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3a1f07650 0x55f3a1f09a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.668+0000 7fc48f6b0b80 4 client.18678 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.668+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/2705734887 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3a1dac570 con 0x55f3a1f07650 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.669+0000 7fc479400700 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3a1f07650 0x55f3a1f09a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.669+0000 7fc479400700 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3a1f07650 0x55f3a1f09a70 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8952a95ffe936e27 server_cookie=1e8c78a0365560e2 in_seq=0 out_seq=0 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.669+0000 7fc45be00700 10 client.18678 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:37.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.670+0000 7fc45be00700 1 -- 172.21.15.63:0/2705734887 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f3a1dac570 con 0x55f3a1f07650 2024-05-27T07:50:37.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.670+0000 7fc45be00700 10 client.18678 handle_command_reply: tid=0 2024-05-27T07:50:37.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.670+0000 7fc48f6b0b80 1 client.18678 shutdown 2024-05-27T07:50:37.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.670+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/2705734887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 msgr2=0x55f3a1ea22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:37.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.670+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1ea22a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x55f3a1ca7030 tx=0x7fc46c00be80 comp rx=0 tx=0).stop 2024-05-27T07:50:37.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.670+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/2705734887 shutdown_connections 2024-05-27T07:50:37.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.670+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3a1f07650 0x55f3a1f09a70 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.670+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3a1e88d30 0x55f3a1e9fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.670+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3a1d03e40 0x55f3a1ea22a0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.670+0000 7fc48f6b0b80 1 --2- 172.21.15.63:0/2705734887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3a1e82db0 0x55f3a1e9f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.671+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/2705734887 >> 172.21.15.63:0/2705734887 conn(0x55f3a1d63fc0 msgr2=0x55f3a1e8cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.671+0000 7fc45be00700 0 client.18678 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.671+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/2705734887 shutdown_connections 2024-05-27T07:50:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.671+0000 7fc48f6b0b80 1 -- 172.21.15.63:0/2705734887 wait complete. 2024-05-27T07:50:37.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.671+0000 7fc48f6b0b80 20 client.18678 trim_cache size 0 max 16384 2024-05-27T07:50:37.674 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:50:37.674 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:50:37.674 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:50:37.674 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:50:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 -- 172.21.15.63:0/317067327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4600b29f0 msgr2=0x7fc4600b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4600b29f0 0x7fc4600b4f00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc474009710 tx=0x7fc47400b000 comp rx=0 tx=0).stop 2024-05-27T07:50:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 -- 172.21.15.63:0/317067327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc47c105e40 msgr2=0x7fc47c19d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc47c105e40 0x7fc47c19d550 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc468000c00 tx=0x7fc4680047b0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 -- 172.21.15.63:0/317067327 shutdown_connections 2024-05-27T07:50:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4600b29f0 0x7fc4600b4f00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc47c10e920 0x7fc47c1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc47c1054f0 0x7fc47c19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 --2- 172.21.15.63:0/317067327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc47c105e40 0x7fc47c19d550 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 -- 172.21.15.63:0/317067327 >> 172.21.15.63:0/317067327 conn(0x7fc47c0fef20 msgr2=0x7fc47c100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 -- 172.21.15.63:0/317067327 shutdown_connections 2024-05-27T07:50:37.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:37.672+0000 7fc480c00700 1 -- 172.21.15.63:0/317067327 wait complete. 2024-05-27T07:50:38.776 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:50:38.777 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:50:38.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:38 smithi063 ceph-mon[31613]: pgmap v537: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:50:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.131+0000 7f0bf9c00640 1 -- 172.21.15.63:0/2498416706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bf410d340 msgr2=0x7f0bf410f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.131+0000 7f0bf9c00640 1 --2- 172.21.15.63:0/2498416706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bf410d340 0x7f0bf410f760 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009ca0 tx=0x7f0bec02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.131+0000 7f0bf9c00640 1 -- 172.21.15.63:0/2498416706 shutdown_connections 2024-05-27T07:50:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.131+0000 7f0bf9c00640 1 --2- 172.21.15.63:0/2498416706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bf410d340 0x7f0bf410f760 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.131+0000 7f0bf9c00640 1 --2- 172.21.15.63:0/2498416706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bf4102ab0 0x7f0bf4102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.131+0000 7f0bf9c00640 1 --2- 172.21.15.63:0/2498416706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf41089e0 0x7f0bf4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.131+0000 7f0bf9c00640 1 -- 172.21.15.63:0/2498416706 >> 172.21.15.63:0/2498416706 conn(0x7f0bf40fe210 msgr2=0x7f0bf4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.131+0000 7f0bf9c00640 1 -- 172.21.15.63:0/2498416706 shutdown_connections 2024-05-27T07:50:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.132+0000 7f0bf9c00640 1 -- 172.21.15.63:0/2498416706 wait complete. 2024-05-27T07:50:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.133+0000 7f0bf9c00640 1 Processor -- start 2024-05-27T07:50:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.133+0000 7f0bf9c00640 1 -- start start 2024-05-27T07:50:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf9c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4102ab0 0x7f0bf419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf9c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bf41089e0 0x7f0bf419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bf410d340 0x7f0bf419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf9c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4112aa0 con 0x7f0bf41089e0 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4112920 con 0x7f0bf410d340 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf9c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4112c20 con 0x7f0bf4102ab0 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bf41089e0 0x7f0bf419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bf41089e0 0x7f0bf419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44816/0 (socket says 172.21.15.63:44816) 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf3400640 1 -- 172.21.15.63:0/3823602745 learned_addr learned my addr 172.21.15.63:0/3823602745 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4102ab0 0x7f0bf419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.134+0000 7f0bf9200640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bf410d340 0x7f0bf419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.135+0000 7f0bf3400640 1 -- 172.21.15.63:0/3823602745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4102ab0 msgr2=0x7f0bf419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.135+0000 7f0bf3400640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4102ab0 0x7f0bf419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.135+0000 7f0bf3400640 1 -- 172.21.15.63:0/3823602745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bf410d340 msgr2=0x7f0bf419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.135+0000 7f0bf3400640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bf410d340 0x7f0bf419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.135+0000 7f0bf3400640 1 -- 172.21.15.63:0/3823602745 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf419f450 con 0x7f0bf41089e0 2024-05-27T07:50:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.135+0000 7f0bf3400640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bf41089e0 0x7f0bf419ab50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f0be000e9f0 tx=0x7f0be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.136+0000 7f0bf0c00640 1 -- 172.21.15.63:0/3823602745 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be000ccc0 con 0x7f0bf41089e0 2024-05-27T07:50:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.136+0000 7f0bf0c00640 1 -- 172.21.15.63:0/3823602745 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0be00044b0 con 0x7f0bf41089e0 2024-05-27T07:50:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.136+0000 7f0bf9c00640 1 -- 172.21.15.63:0/3823602745 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf419f740 con 0x7f0bf41089e0 2024-05-27T07:50:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.136+0000 7f0bf0c00640 1 -- 172.21.15.63:0/3823602745 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0010430 con 0x7f0bf41089e0 2024-05-27T07:50:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.136+0000 7f0bf9c00640 1 -- 172.21.15.63:0/3823602745 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf41a70c0 con 0x7f0bf41089e0 2024-05-27T07:50:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.141+0000 7f0bf0c00640 1 -- 172.21.15.63:0/3823602745 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0be00105d0 con 0x7f0bf41089e0 2024-05-27T07:50:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.141+0000 7f0c0216f740 1 Processor -- start 2024-05-27T07:50:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.142+0000 7f0c0216f740 1 -- start start 2024-05-27T07:50:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.142+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767411b0 0x558276ae3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.142+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767182f0 0x558276ae3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.142+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767204d0 0x558276ae4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.142+0000 7f0c0216f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558276a54b90 con 0x5582767204d0 2024-05-27T07:50:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.142+0000 7f0c0216f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558276a44730 con 0x5582767411b0 2024-05-27T07:50:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.142+0000 7f0c0216f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558276a48770 con 0x5582767182f0 2024-05-27T07:50:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767204d0 0x558276ae4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767204d0 0x558276ae4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44818/0 (socket says 172.21.15.63:44818) 2024-05-27T07:50:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf3400640 1 -- 172.21.15.63:0/2504963664 learned_addr learned my addr 172.21.15.63:0/2504963664 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf9200640 1 --2- 172.21.15.63:0/2504963664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767411b0 0x558276ae3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/2504963664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767182f0 0x558276ae3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf0c00640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bd80b29d0 0x7f0bd80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf3400640 1 -- 172.21.15.63:0/2504963664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767182f0 msgr2=0x558276ae3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf3400640 1 --2- 172.21.15.63:0/2504963664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767182f0 0x558276ae3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf3400640 1 -- 172.21.15.63:0/2504963664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767411b0 msgr2=0x558276ae3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf3400640 1 --2- 172.21.15.63:0/2504963664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767411b0 0x558276ae3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.143+0000 7f0bf3400640 1 -- 172.21.15.63:0/2504963664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582769dd8b0 con 0x5582767204d0 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.144+0000 7f0bf9200640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bd80b29d0 0x7f0bd80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.144+0000 7f0bf0c00640 1 -- 172.21.15.63:0/3823602745 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f0be00e2c50 con 0x7f0bf41089e0 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.145+0000 7f0bf3400640 1 --2- 172.21.15.63:0/2504963664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767204d0 0x558276ae4390 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f0be00dff50 tx=0x7f0be00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568cb8760713137c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.145+0000 7f0bf9200640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bd80b29d0 0x7f0bd80b4ee0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0bec006010 tx=0x7f0bec0095c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.145+0000 7f0be9600640 1 -- 172.21.15.63:0/2504963664 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be00a99b0 con 0x5582767204d0 2024-05-27T07:50:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.145+0000 7f0be9600640 1 -- 172.21.15.63:0/2504963664 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0be00a96f0 con 0x5582767204d0 2024-05-27T07:50:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.145+0000 7f0be9600640 1 -- 172.21.15.63:0/2504963664 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be00a5370 con 0x5582767204d0 2024-05-27T07:50:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.146+0000 7f0c0216f740 1 -- 172.21.15.63:0/2504963664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767204d0 msgr2=0x558276ae4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.146+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2504963664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767204d0 0x558276ae4390 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f0be00dff50 tx=0x7f0be00035f0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.146+0000 7f0c0216f740 1 -- 172.21.15.63:0/2504963664 shutdown_connections 2024-05-27T07:50:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.146+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2504963664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767204d0 0x558276ae4390 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.146+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2504963664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767182f0 0x558276ae3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.146+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2504963664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767411b0 0x558276ae3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.146+0000 7f0c0216f740 1 -- 172.21.15.63:0/2504963664 >> 172.21.15.63:0/2504963664 conn(0x558276735300 msgr2=0x558276ae0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.147+0000 7f0c0216f740 1 -- 172.21.15.63:0/2504963664 shutdown_connections 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.147+0000 7f0c0216f740 1 -- 172.21.15.63:0/2504963664 wait complete. 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.147+0000 7f0c0216f740 1 -- start start 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.148+0000 7f0c0216f740 1 Processor -- start 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.148+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276aee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.148+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767204d0 0x558276aee810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.149+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767411b0 0x558276af2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.149+0000 7f0c0216f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55827672a790 con 0x5582767182f0 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.149+0000 7f0c0216f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5582769ecbc0 con 0x5582767204d0 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.149+0000 7f0c0216f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558276ae6a60 con 0x5582767411b0 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.149+0000 7f0c0216f740 10 client.0 _ll_register_callbacks cb 0x5582769ca400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.149+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276aee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.149+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276aee2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44826/0 (socket says 172.21.15.63:44826) 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.149+0000 7f0bf3e00640 1 -- 172.21.15.63:0/240330087 learned_addr learned my addr 172.21.15.63:0/240330087 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.149+0000 7f0bf9200640 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767411b0 0x558276af2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.149+0000 7f0bf3400640 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767204d0 0x558276aee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.150+0000 7f0bf3e00640 1 -- 172.21.15.63:0/240330087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767411b0 msgr2=0x558276af2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.150+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767411b0 0x558276af2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.150+0000 7f0bf3e00640 1 -- 172.21.15.63:0/240330087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767204d0 msgr2=0x558276aee810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.150+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767204d0 0x558276aee810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.150+0000 7f0bf3e00640 1 -- 172.21.15.63:0/240330087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558276759210 con 0x5582767182f0 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.150+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276aee2d0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0be400bed0 tx=0x7f0be4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621d7b7b72b74f70 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.150+0000 7f0bde000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.150+0000 7f0bde000640 1 -- 172.21.15.63:0/240330087 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4004020 con 0x5582767182f0 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.151+0000 7f0c0216f740 1 -- 172.21.15.63:0/240330087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be00a96f0 con 0x5582767182f0 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.151+0000 7f0bde000640 1 -- 172.21.15.63:0/240330087 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f0be40041c0 con 0x5582767182f0 2024-05-27T07:50:39.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.151+0000 7f0bde000640 1 -- 172.21.15.63:0/240330087 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0be4024050 con 0x5582767182f0 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.151+0000 7f0c0216f740 10 client.18690 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.151+0000 7f0c0216f740 1 -- 172.21.15.63:0/240330087 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558276825620 con 0x5582767182f0 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.151+0000 7f0bde000640 1 -- 172.21.15.63:0/240330087 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0be00a96f0 con 0x5582767182f0 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.151+0000 7f0bde000640 1 -- 172.21.15.63:0/240330087 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0be4004aa0 con 0x5582767182f0 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.152+0000 7f0c0216f740 10 client.18690 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.152+0000 7f0c0216f740 10 client.18690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.152+0000 7f0c0216f740 20 client.18690 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.152+0000 7f0c0216f740 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5582766ffd80 0x558276affeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.152+0000 7f0c0216f740 4 client.18690 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.152+0000 7f0c0216f740 1 -- 172.21.15.63:0/240330087 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558276a56ae0 con 0x5582766ffd80 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.152+0000 7f0bf3400640 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5582766ffd80 0x558276affeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.153+0000 7f0bde000640 10 client.18690 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.153+0000 7f0bf3400640 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5582766ffd80 0x558276affeb0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8866ba1d7ef13c7c server_cookie=384a9d0573ebf121 in_seq=0 out_seq=0 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.162+0000 7f0bde000640 1 -- 172.21.15.63:0/240330087 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558276a56ae0 con 0x5582766ffd80 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.162+0000 7f0bde000640 10 client.18690 handle_command_reply: tid=0 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.162+0000 7f0c0216f740 1 client.18690 shutdown 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.162+0000 7f0c0216f740 1 -- 172.21.15.63:0/240330087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 msgr2=0x558276aee2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.162+0000 7f0c0216f740 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276aee2d0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0be400bed0 tx=0x7f0be4011810 comp rx=0 tx=0).stop 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.163+0000 7f0c0216f740 1 -- 172.21.15.63:0/240330087 shutdown_connections 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.163+0000 7f0c0216f740 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5582766ffd80 0x558276affeb0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.163+0000 7f0c0216f740 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767411b0 0x558276af2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.163+0000 7f0c0216f740 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767204d0 0x558276aee810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.163+0000 7f0c0216f740 1 --2- 172.21.15.63:0/240330087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276aee2d0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.163+0000 7f0bde000640 0 client.18690 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.163+0000 7f0c0216f740 1 -- 172.21.15.63:0/240330087 >> 172.21.15.63:0/240330087 conn(0x558276735300 msgr2=0x558276ae5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.163+0000 7f0c0216f740 1 -- 172.21.15.63:0/240330087 shutdown_connections 2024-05-27T07:50:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.163+0000 7f0c0216f740 1 -- 172.21.15.63:0/240330087 wait complete. 2024-05-27T07:50:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.163+0000 7f0c0216f740 20 client.18690 trim_cache size 0 max 16384 2024-05-27T07:50:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.191+0000 7f0c0216f740 1 Processor -- start 2024-05-27T07:50:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.191+0000 7f0c0216f740 1 -- start start 2024-05-27T07:50:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.192+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276af26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.192+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767204d0 0x558276af2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.192+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767411b0 0x558276ae7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.192+0000 7f0c0216f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41b1810 con 0x5582767182f0 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.192+0000 7f0c0216f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41af0e0 con 0x5582767411b0 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.192+0000 7f0c0216f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41acf50 con 0x5582767204d0 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.192+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276af26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.192+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276af26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44840/0 (socket says 172.21.15.63:44840) 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.192+0000 7f0bf3e00640 1 -- 172.21.15.63:0/2667863729 learned_addr learned my addr 172.21.15.63:0/2667863729 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.193+0000 7f0bf9200640 1 --2- 172.21.15.63:0/2667863729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767411b0 0x558276ae7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.193+0000 7f0bf3400640 1 --2- 172.21.15.63:0/2667863729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767204d0 0x558276af2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.193+0000 7f0bf3e00640 1 -- 172.21.15.63:0/2667863729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767204d0 msgr2=0x558276af2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.193+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/2667863729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767204d0 0x558276af2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.193+0000 7f0bf3e00640 1 -- 172.21.15.63:0/2667863729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767411b0 msgr2=0x558276ae7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.193+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/2667863729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767411b0 0x558276ae7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.193+0000 7f0bf3e00640 1 -- 172.21.15.63:0/2667863729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558276ae3e90 con 0x5582767182f0 2024-05-27T07:50:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.193+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/2667863729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276af26b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0be4000c00 tx=0x7f0be4005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769e0d212a69b21a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.194+0000 7f0be8c00640 1 -- 172.21.15.63:0/2667863729 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4010850 con 0x5582767182f0 2024-05-27T07:50:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.194+0000 7f0be8c00640 1 -- 172.21.15.63:0/2667863729 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0be4004660 con 0x5582767182f0 2024-05-27T07:50:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.194+0000 7f0be8c00640 1 -- 172.21.15.63:0/2667863729 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4005260 con 0x5582767182f0 2024-05-27T07:50:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.195+0000 7f0c0216f740 1 -- 172.21.15.63:0/2667863729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 msgr2=0x558276af26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.195+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2667863729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276af26b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0be4000c00 tx=0x7f0be4005900 comp rx=0 tx=0).stop 2024-05-27T07:50:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.195+0000 7f0c0216f740 1 -- 172.21.15.63:0/2667863729 shutdown_connections 2024-05-27T07:50:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.195+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2667863729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767411b0 0x558276ae7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.195+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2667863729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767204d0 0x558276af2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.195+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2667863729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767182f0 0x558276af26b0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.195+0000 7f0c0216f740 1 -- 172.21.15.63:0/2667863729 >> 172.21.15.63:0/2667863729 conn(0x558276735300 msgr2=0x558276ae93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.196+0000 7f0c0216f740 1 -- 172.21.15.63:0/2667863729 shutdown_connections 2024-05-27T07:50:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.196+0000 7f0c0216f740 1 -- 172.21.15.63:0/2667863729 wait complete. 2024-05-27T07:50:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.196+0000 7f0c0216f740 1 -- start start 2024-05-27T07:50:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.197+0000 7f0c0216f740 1 Processor -- start 2024-05-27T07:50:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.197+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767182f0 0x558276af4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.197+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767204d0 0x558276af53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.197+0000 7f0c0216f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767411b0 0x558276af5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.197+0000 7f0c0216f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41ad5a0 con 0x5582767411b0 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.197+0000 7f0c0216f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41aea10 con 0x5582767182f0 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.197+0000 7f0c0216f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41adc70 con 0x5582767204d0 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.197+0000 7f0c0216f740 10 client.0 _ll_register_callbacks cb 0x7f0be400bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.198+0000 7f0bf9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767411b0 0x558276af5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.198+0000 7f0bf9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767411b0 0x558276af5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44854/0 (socket says 172.21.15.63:44854) 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.198+0000 7f0bf9200640 1 -- 172.21.15.63:0/2650532891 learned_addr learned my addr 172.21.15.63:0/2650532891 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.198+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767182f0 0x558276af4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.198+0000 7f0bf3400640 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767204d0 0x558276af53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.199+0000 7f0bf9200640 1 -- 172.21.15.63:0/2650532891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767204d0 msgr2=0x558276af53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.199+0000 7f0bf9200640 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767204d0 0x558276af53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.199+0000 7f0bf9200640 1 -- 172.21.15.63:0/2650532891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767182f0 msgr2=0x558276af4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.199+0000 7f0bf9200640 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767182f0 0x558276af4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.199+0000 7f0bf9200640 1 -- 172.21.15.63:0/2650532891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558276b66720 con 0x5582767411b0 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.199+0000 7f0bf9200640 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767411b0 0x558276af5930 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0bec035000 tx=0x7f0bec042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32dc048549eb1b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.199+0000 7f0bde000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.199+0000 7f0bde000640 1 -- 172.21.15.63:0/2650532891 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec004450 con 0x5582767411b0 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.199+0000 7f0bde000640 1 -- 172.21.15.63:0/2650532891 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f0bec0045f0 con 0x5582767411b0 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.200+0000 7f0c0216f740 1 -- 172.21.15.63:0/2650532891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be4004660 con 0x5582767411b0 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.200+0000 7f0bde000640 1 -- 172.21.15.63:0/2650532891 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0bec05e050 con 0x5582767411b0 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.200+0000 7f0c0216f740 10 client.18696 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:39.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.200+0000 7f0c0216f740 1 -- 172.21.15.63:0/2650532891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558276b679b0 con 0x5582767411b0 2024-05-27T07:50:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.200+0000 7f0bde000640 1 -- 172.21.15.63:0/2650532891 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0be4004660 con 0x5582767411b0 2024-05-27T07:50:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.201+0000 7f0bde000640 1 -- 172.21.15.63:0/2650532891 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0bec054030 con 0x5582767411b0 2024-05-27T07:50:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.201+0000 7f0c0216f740 10 client.18696 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.201+0000 7f0c0216f740 10 client.18696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.201+0000 7f0c0216f740 20 client.18696 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.201+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5582766d51c0 0x558276b6a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.201+0000 7f0c0216f740 4 client.18696 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:50:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.201+0000 7f0c0216f740 1 -- 172.21.15.63:0/2650532891 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558276a502c0 con 0x5582766d51c0 2024-05-27T07:50:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.202+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5582766d51c0 0x558276b6a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.203+0000 7f0bf3e00640 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5582766d51c0 0x558276b6a170 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b7bd92651ebd679 server_cookie=72edb4f45c512a4f in_seq=0 out_seq=0 2024-05-27T07:50:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.203+0000 7f0bde000640 10 client.18696 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:39.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:38 smithi086 ceph-mon[34687]: pgmap v537: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:50:39.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:38 smithi149 ceph-mon[35408]: pgmap v537: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:50:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.234+0000 7f0bde000640 1 -- 172.21.15.63:0/2650532891 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558276a502c0 con 0x5582766d51c0 2024-05-27T07:50:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.234+0000 7f0bde000640 10 client.18696 handle_command_reply: tid=0 2024-05-27T07:50:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.234+0000 7f0c0216f740 1 client.18696 shutdown 2024-05-27T07:50:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 1 -- 172.21.15.63:0/2650532891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767411b0 msgr2=0x558276af5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767411b0 0x558276af5930 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0bec035000 tx=0x7f0bec042000 comp rx=0 tx=0).stop 2024-05-27T07:50:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 1 -- 172.21.15.63:0/2650532891 shutdown_connections 2024-05-27T07:50:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5582766d51c0 0x558276b6a170 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582767411b0 0x558276af5930 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582767204d0 0x558276af53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 1 --2- 172.21.15.63:0/2650532891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582767182f0 0x558276af4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 1 -- 172.21.15.63:0/2650532891 >> 172.21.15.63:0/2650532891 conn(0x558276735300 msgr2=0x558276af16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0bde000640 0 client.18696 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 1 -- 172.21.15.63:0/2650532891 shutdown_connections 2024-05-27T07:50:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 1 -- 172.21.15.63:0/2650532891 wait complete. 2024-05-27T07:50:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.235+0000 7f0c0216f740 20 client.18696 trim_cache size 0 max 16384 2024-05-27T07:50:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:50:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:50:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:50:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.236+0000 7f0bf9c00640 1 -- 172.21.15.63:0/3823602745 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bd80b29d0 msgr2=0x7f0bd80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.236+0000 7f0bf9c00640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bd80b29d0 0x7f0bd80b4ee0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0bec006010 tx=0x7f0bec0095c0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.236+0000 7f0bf9c00640 1 -- 172.21.15.63:0/3823602745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bf41089e0 msgr2=0x7f0bf419ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.236+0000 7f0bf9c00640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bf41089e0 0x7f0bf419ab50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f0be000e9f0 tx=0x7f0be000eeb0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.236+0000 7f0bf9c00640 1 -- 172.21.15.63:0/3823602745 shutdown_connections 2024-05-27T07:50:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.236+0000 7f0bf9c00640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bd80b29d0 0x7f0bd80b4ee0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.236+0000 7f0bf9c00640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bf410d340 0x7f0bf419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.236+0000 7f0bf9c00640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bf41089e0 0x7f0bf419ab50 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.236+0000 7f0bf9c00640 1 --2- 172.21.15.63:0/3823602745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bf4102ab0 0x7f0bf419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.236+0000 7f0bf9c00640 1 -- 172.21.15.63:0/3823602745 >> 172.21.15.63:0/3823602745 conn(0x7f0bf40fe210 msgr2=0x7f0bf41009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.237+0000 7f0bf9c00640 1 -- 172.21.15.63:0/3823602745 shutdown_connections 2024-05-27T07:50:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:39.237+0000 7f0bf9c00640 1 -- 172.21.15.63:0/3823602745 wait complete. 2024-05-27T07:50:39.310 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:41.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:40 smithi086 ceph-mon[34687]: pgmap v538: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:50:41.211 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:40 smithi063 ceph-mon[31613]: pgmap v538: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:50:41.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:40 smithi149 ceph-mon[35408]: pgmap v538: 129 pgs: 129 active+clean; 4.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 7 op/s 2024-05-27T07:50:41.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.885+0000 7f49dde00700 1 -- 172.21.15.63:0/2748440594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d8105280 msgr2=0x7f49d8105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:41.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.885+0000 7f49dde00700 1 --2- 172.21.15.63:0/2748440594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d8105280 0x7f49d8105690 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009990 tx=0x7f49cc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:50:41.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.885+0000 7f49dde00700 1 -- 172.21.15.63:0/2748440594 shutdown_connections 2024-05-27T07:50:41.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.885+0000 7f49dde00700 1 --2- 172.21.15.63:0/2748440594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d810e6f0 0x7f49d8110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:41.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.885+0000 7f49dde00700 1 --2- 172.21.15.63:0/2748440594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d8105280 0x7f49d8105690 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:41.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.885+0000 7f49dde00700 1 --2- 172.21.15.63:0/2748440594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49d8105bd0 0x7f49d810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:41.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.885+0000 7f49dde00700 1 -- 172.21.15.63:0/2748440594 >> 172.21.15.63:0/2748440594 conn(0x7f49d8078ab0 msgr2=0x7f49d8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:41.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.885+0000 7f49dde00700 1 -- 172.21.15.63:0/2748440594 shutdown_connections 2024-05-27T07:50:41.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.886+0000 7f49dde00700 1 -- 172.21.15.63:0/2748440594 wait complete. 2024-05-27T07:50:41.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.887+0000 7f49dde00700 1 Processor -- start 2024-05-27T07:50:41.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.888+0000 7f49dde00700 1 -- start start 2024-05-27T07:50:41.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.888+0000 7f49dde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d8105280 0x7f49d819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:41.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.888+0000 7f49dde00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8105bd0 0x7f49d819d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:41.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.888+0000 7f49dde00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49d810e6f0 0x7f49d81a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:41.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.888+0000 7f49dde00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f49d81140f0 con 0x7f49d8105280 2024-05-27T07:50:41.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.888+0000 7f49dde00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49d8113f70 con 0x7f49d8105bd0 2024-05-27T07:50:41.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.888+0000 7f49dde00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f49d8114270 con 0x7f49d810e6f0 2024-05-27T07:50:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49dca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d8105280 0x7f49d819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49dca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d8105280 0x7f49d819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44872/0 (socket says 172.21.15.63:44872) 2024-05-27T07:50:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49dca00700 1 -- 172.21.15.63:0/1394777226 learned_addr learned my addr 172.21.15.63:0/1394777226 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49d7e00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8105bd0 0x7f49d819d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49dd400700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49d810e6f0 0x7f49d81a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49dca00700 1 -- 172.21.15.63:0/1394777226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49d810e6f0 msgr2=0x7f49d81a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49dca00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49d810e6f0 0x7f49d81a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49dca00700 1 -- 172.21.15.63:0/1394777226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8105bd0 msgr2=0x7f49d819d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:41.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49dca00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8105bd0 0x7f49d819d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:41.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49dca00700 1 -- 172.21.15.63:0/1394777226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d81a1c10 con 0x7f49d8105280 2024-05-27T07:50:41.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.889+0000 7f49d7e00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8105bd0 0x7f49d819d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:41.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.890+0000 7f49dca00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d8105280 0x7f49d819cd40 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f49cc000c00 tx=0x7f49cc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.890+0000 7f49d5600700 1 -- 172.21.15.63:0/1394777226 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc027070 con 0x7f49d8105280 2024-05-27T07:50:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.890+0000 7f49d5600700 1 -- 172.21.15.63:0/1394777226 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49cc00fc70 con 0x7f49d8105280 2024-05-27T07:50:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.890+0000 7f49d5600700 1 -- 172.21.15.63:0/1394777226 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc022d40 con 0x7f49d8105280 2024-05-27T07:50:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.890+0000 7f49dde00700 1 -- 172.21.15.63:0/1394777226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d81a1ea0 con 0x7f49d8105280 2024-05-27T07:50:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.890+0000 7f49dde00700 1 -- 172.21.15.63:0/1394777226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d81a9820 con 0x7f49d8105280 2024-05-27T07:50:41.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.892+0000 7f49dde00700 1 -- 172.21.15.63:0/1394777226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d80512b0 con 0x7f49d8105280 2024-05-27T07:50:41.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.895+0000 7f49d5600700 1 -- 172.21.15.63:0/1394777226 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f49cc02a070 con 0x7f49d8105280 2024-05-27T07:50:41.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.897+0000 7f49d5600700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49c00b2a70 0x7f49c00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:41.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.897+0000 7f49d5600700 1 -- 172.21.15.63:0/1394777226 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f49cc0e3bc0 con 0x7f49d8105280 2024-05-27T07:50:41.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.897+0000 7f49d7e00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49c00b2a70 0x7f49c00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:41.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.898+0000 7f49d7e00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49c00b2a70 0x7f49c00b4f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005950 tx=0x7f49c40058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:41.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:41.900+0000 7f49d5600700 1 -- 172.21.15.63:0/1394777226 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49cc013070 con 0x7f49d8105280 2024-05-27T07:50:42.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.241+0000 7f49dde00700 1 -- 172.21.15.63:0/1394777226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49d819e0a0 con 0x7f49d8105280 2024-05-27T07:50:42.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.242+0000 7f49d5600700 1 -- 172.21.15.63:0/1394777226 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f49cc0b3030 con 0x7f49d8105280 2024-05-27T07:50:42.243 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:50:42.243 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:50:42.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.245+0000 7f49dde00700 1 -- 172.21.15.63:0/1394777226 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49c00b2a70 msgr2=0x7f49c00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:42.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.245+0000 7f49dde00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49c00b2a70 0x7f49c00b4f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005950 tx=0x7f49c40058e0 comp rx=0 tx=0).stop 2024-05-27T07:50:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.245+0000 7f49dde00700 1 -- 172.21.15.63:0/1394777226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d8105280 msgr2=0x7f49d819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.245+0000 7f49dde00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d8105280 0x7f49d819cd40 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f49cc000c00 tx=0x7f49cc009f90 comp rx=0 tx=0).stop 2024-05-27T07:50:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.246+0000 7f49dde00700 1 -- 172.21.15.63:0/1394777226 shutdown_connections 2024-05-27T07:50:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.246+0000 7f49dde00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49c00b2a70 0x7f49c00b4f80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.246+0000 7f49dde00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49d810e6f0 0x7f49d81a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.246+0000 7f49dde00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49d8105280 0x7f49d819cd40 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.246+0000 7f49dde00700 1 --2- 172.21.15.63:0/1394777226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d8105bd0 0x7f49d819d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.246+0000 7f49dde00700 1 -- 172.21.15.63:0/1394777226 >> 172.21.15.63:0/1394777226 conn(0x7f49d8078ab0 msgr2=0x7f49d8102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:42.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.246+0000 7f49dde00700 1 -- 172.21.15.63:0/1394777226 shutdown_connections 2024-05-27T07:50:42.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:42.246+0000 7f49dde00700 1 -- 172.21.15.63:0/1394777226 wait complete. 2024-05-27T07:50:42.247 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:50:43.160 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:42 smithi063 ceph-mon[31613]: pgmap v539: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:50:43.160 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:42 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1394777226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:50:43.163 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:50:43.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:42 smithi086 ceph-mon[34687]: pgmap v539: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:50:43.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:42 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1394777226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:50:43.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:42 smithi149 ceph-mon[35408]: pgmap v539: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:50:43.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:42 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1394777226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:50:43.677 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:45.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:44 smithi086 ceph-mon[34687]: pgmap v540: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:50:45.209 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:44 smithi063 ceph-mon[31613]: pgmap v540: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:50:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:44 smithi149 ceph-mon[35408]: pgmap v540: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:50:46.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.197+0000 7f9cd2200700 1 -- 172.21.15.63:0/1604982141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ccc10e8c0 msgr2=0x7f9ccc110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:46.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.197+0000 7f9cd2200700 1 --2- 172.21.15.63:0/1604982141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ccc10e8c0 0x7f9ccc110da0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400b0a0 tx=0x7f9cc400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.197+0000 7f9cd2200700 1 -- 172.21.15.63:0/1604982141 shutdown_connections 2024-05-27T07:50:46.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.197+0000 7f9cd2200700 1 --2- 172.21.15.63:0/1604982141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ccc10e8c0 0x7f9ccc110da0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.197+0000 7f9cd2200700 1 --2- 172.21.15.63:0/1604982141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ccc105450 0x7f9ccc105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.197+0000 7f9cd2200700 1 --2- 172.21.15.63:0/1604982141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ccc105da0 0x7f9ccc10e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.197+0000 7f9cd2200700 1 -- 172.21.15.63:0/1604982141 >> 172.21.15.63:0/1604982141 conn(0x7f9ccc0fedb0 msgr2=0x7f9ccc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:46.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.198+0000 7f9cd2200700 1 -- 172.21.15.63:0/1604982141 shutdown_connections 2024-05-27T07:50:46.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.198+0000 7f9cd2200700 1 -- 172.21.15.63:0/1604982141 wait complete. 2024-05-27T07:50:46.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.199+0000 7f9cd2200700 1 Processor -- start 2024-05-27T07:50:46.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.199+0000 7f9cd2200700 1 -- start start 2024-05-27T07:50:46.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.200+0000 7f9cd2200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ccc105450 0x7f9ccc103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.200+0000 7f9cd2200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ccc105da0 0x7f9ccc103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.200+0000 7f9cd2200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ccc10e8c0 0x7f9ccc103bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.200+0000 7f9cd2200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc114320 con 0x7f9ccc105450 2024-05-27T07:50:46.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.200+0000 7f9cd2200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1141a0 con 0x7f9ccc105da0 2024-05-27T07:50:46.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.200+0000 7f9cd2200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1144a0 con 0x7f9ccc10e8c0 2024-05-27T07:50:46.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.200+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ccc105450 0x7f9ccc103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.200+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ccc105450 0x7f9ccc103140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44892/0 (socket says 172.21.15.63:44892) 2024-05-27T07:50:46.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.200+0000 7f9ccb400700 1 -- 172.21.15.63:0/2125048122 learned_addr learned my addr 172.21.15.63:0/2125048122 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:46.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.201+0000 7f9ccaa00700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ccc105da0 0x7f9ccc103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.201+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ccc10e8c0 0x7f9ccc103bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.201+0000 7f9ccb400700 1 -- 172.21.15.63:0/2125048122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ccc10e8c0 msgr2=0x7f9ccc103bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:46.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.201+0000 7f9ccb400700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ccc10e8c0 0x7f9ccc103bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.201+0000 7f9ccb400700 1 -- 172.21.15.63:0/2125048122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ccc105da0 msgr2=0x7f9ccc103680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:46.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.201+0000 7f9ccb400700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ccc105da0 0x7f9ccc103680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.201+0000 7f9ccb400700 1 -- 172.21.15.63:0/2125048122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc1a9050 con 0x7f9ccc105450 2024-05-27T07:50:46.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.201+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ccc10e8c0 0x7f9ccc103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:46.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.201+0000 7f9ccaa00700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ccc105da0 0x7f9ccc103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:46.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.202+0000 7f9ccb400700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ccc105450 0x7f9ccc103140 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00d950 tx=0x7f9cbc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:46.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.202+0000 7f9cc3e00700 1 -- 172.21.15.63:0/2125048122 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc016070 con 0x7f9ccc105450 2024-05-27T07:50:46.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.202+0000 7f9cd2200700 1 -- 172.21.15.63:0/2125048122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc1a92e0 con 0x7f9ccc105450 2024-05-27T07:50:46.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.202+0000 7f9cc3e00700 1 -- 172.21.15.63:0/2125048122 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cbc010460 con 0x7f9ccc105450 2024-05-27T07:50:46.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.202+0000 7f9cc3e00700 1 -- 172.21.15.63:0/2125048122 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00f620 con 0x7f9ccc105450 2024-05-27T07:50:46.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.202+0000 7f9cd2200700 1 -- 172.21.15.63:0/2125048122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc1a9820 con 0x7f9ccc105450 2024-05-27T07:50:46.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.207+0000 7f9cc3e00700 1 -- 172.21.15.63:0/2125048122 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9cbc019070 con 0x7f9ccc105450 2024-05-27T07:50:46.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.208+0000 7f9ce0da8b80 1 Processor -- start 2024-05-27T07:50:46.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.209+0000 7f9ce0da8b80 1 -- start start 2024-05-27T07:50:46.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.209+0000 7f9cc3e00700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9cb00b29f0 0x7f9cb00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.209+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90639f0 0x5653b90f6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.209+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 0x5653b90f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.209+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b9103110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.209+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5653b9024e40 con 0x5653b90639f0 2024-05-27T07:50:46.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.209+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5653b90149e0 con 0x5653b90f6e70 2024-05-27T07:50:46.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.209+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5653b9018a20 con 0x5653b8f77e40 2024-05-27T07:50:46.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.209+0000 7f9cc3e00700 1 -- 172.21.15.63:0/2125048122 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f9cbc0e2620 con 0x7f9ccc105450 2024-05-27T07:50:46.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.209+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9cb00b29f0 0x7f9cb00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90639f0 0x5653b90f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90639f0 0x5653b90f6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44894/0 (socket says 172.21.15.63:44894) 2024-05-27T07:50:46.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccb400700 1 -- 172.21.15.63:0/1070625265 learned_addr learned my addr 172.21.15.63:0/1070625265 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:46.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b9103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccaa00700 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 0x5653b90f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccb400700 1 -- 172.21.15.63:0/1070625265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 msgr2=0x5653b90f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:46.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccb400700 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 0x5653b90f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccb400700 1 -- 172.21.15.63:0/1070625265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 msgr2=0x5653b9103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:46.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccb400700 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b9103110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccb400700 1 -- 172.21.15.63:0/1070625265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653b8fdd420 con 0x5653b90639f0 2024-05-27T07:50:46.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccaa00700 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 0x5653b90f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:46.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.210+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b9103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:46.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.211+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9cb00b29f0 0x7f9cb00b4f00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009e60 tx=0x7f9cc4009380 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:46.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.211+0000 7f9ccb400700 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90639f0 0x5653b90f6330 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0a55c0 tx=0x7f9cbc0a8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afccb8fe93e476e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:46.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.211+0000 7f9cc0c00700 1 -- 172.21.15.63:0/1070625265 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc020070 con 0x5653b90639f0 2024-05-27T07:50:46.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.211+0000 7f9cc0c00700 1 -- 172.21.15.63:0/1070625265 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cbc0a9540 con 0x5653b90639f0 2024-05-27T07:50:46.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.211+0000 7f9cc0c00700 1 -- 172.21.15.63:0/1070625265 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc010ab0 con 0x5653b90639f0 2024-05-27T07:50:46.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.212+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1070625265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90639f0 msgr2=0x5653b90f6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:46.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.212+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90639f0 0x5653b90f6330 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0a55c0 tx=0x7f9cbc0a8de0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.212+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1070625265 shutdown_connections 2024-05-27T07:50:46.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.212+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b9103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.212+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90639f0 0x5653b90f6330 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.212+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1070625265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 0x5653b90f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.212+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1070625265 >> 172.21.15.63:0/1070625265 conn(0x5653b9011390 msgr2=0x5653b90f24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:46.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.213+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1070625265 shutdown_connections 2024-05-27T07:50:46.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.213+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1070625265 wait complete. 2024-05-27T07:50:46.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.213+0000 7f9ce0da8b80 1 -- start start 2024-05-27T07:50:46.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.214+0000 7f9ce0da8b80 1 Processor -- start 2024-05-27T07:50:46.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.214+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b91006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.214+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b90639f0 0x5653b9100be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.214+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b9101120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.214+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5653b8f1afa0 con 0x5653b8f77e40 2024-05-27T07:50:46.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.214+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5653b8fb5760 con 0x5653b90f6e70 2024-05-27T07:50:46.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.214+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5653b9103b80 con 0x5653b90639f0 2024-05-27T07:50:46.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.214+0000 7f9ce0da8b80 10 client.0 _ll_register_callbacks cb 0x5653b8fbec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:46.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.215+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b91006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.215+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b91006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44906/0 (socket says 172.21.15.63:44906) 2024-05-27T07:50:46.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.215+0000 7f9ccb400700 1 -- 172.21.15.63:0/1668242389 learned_addr learned my addr 172.21.15.63:0/1668242389 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:46.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.215+0000 7f9ccaa00700 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b90639f0 0x5653b9100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.215+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b9101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.215+0000 7f9ccb400700 1 -- 172.21.15.63:0/1668242389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b90639f0 msgr2=0x5653b9100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:46.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.215+0000 7f9ccb400700 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b90639f0 0x5653b9100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.215+0000 7f9ccb400700 1 -- 172.21.15.63:0/1668242389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 msgr2=0x5653b9101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:46.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.215+0000 7f9ccb400700 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b9101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.215+0000 7f9ccb400700 1 -- 172.21.15.63:0/1668242389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653b8f648d0 con 0x5653b8f77e40 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.216+0000 7f9ccb400700 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b91006a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0a9240 tx=0x7f9cbc074650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928f87b9cff513f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.216+0000 7f9cb5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.216+0000 7f9cb5600700 1 -- 172.21.15.63:0/1668242389 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc020070 con 0x5653b8f77e40 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.216+0000 7f9cb5600700 1 -- 172.21.15.63:0/1668242389 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f9cbc0a5b30 con 0x5653b8f77e40 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.216+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1668242389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cbc0a9540 con 0x5653b8f77e40 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.216+0000 7f9cb5600700 1 -- 172.21.15.63:0/1668242389 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9cbc074100 con 0x5653b8f77e40 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.216+0000 7f9ce0da8b80 10 client.18714 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.216+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1668242389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5653b8fe5060 con 0x5653b8f77e40 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.217+0000 7f9cb5600700 1 -- 172.21.15.63:0/1668242389 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cbc0a9540 con 0x5653b8f77e40 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.217+0000 7f9cb5600700 1 -- 172.21.15.63:0/1668242389 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9cbc0a9810 con 0x5653b8f77e40 2024-05-27T07:50:46.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.217+0000 7f9ce0da8b80 10 client.18714 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:46.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.217+0000 7f9ce0da8b80 10 client.18714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:50:46.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.217+0000 7f9ce0da8b80 20 client.18714 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:46.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.217+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5653b91109b0 0x5653b9112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.217+0000 7f9ce0da8b80 4 client.18714 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:46.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.217+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1668242389 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653b9026d90 con 0x5653b91109b0 2024-05-27T07:50:46.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.218+0000 7f9ccaa00700 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5653b91109b0 0x5653b9112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.219+0000 7f9ccaa00700 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5653b91109b0 0x5653b9112dd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9f3f160b50b8709 server_cookie=237021779f8a7ab1 in_seq=0 out_seq=0 2024-05-27T07:50:46.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.219+0000 7f9cb5600700 10 client.18714 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:50:46.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9cb5600700 1 -- 172.21.15.63:0/1668242389 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5653b9026d90 con 0x5653b91109b0 2024-05-27T07:50:46.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9cb5600700 10 client.18714 handle_command_reply: tid=0 2024-05-27T07:50:46.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9ce0da8b80 1 client.18714 shutdown 2024-05-27T07:50:46.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1668242389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 msgr2=0x5653b91006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:46.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b91006a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0a9240 tx=0x7f9cbc074650 comp rx=0 tx=0).stop 2024-05-27T07:50:46.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1668242389 shutdown_connections 2024-05-27T07:50:46.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b9101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5653b91109b0 0x5653b9112dd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b91006a0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1668242389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b90639f0 0x5653b9100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9cb5600700 0 client.18714 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:50:46.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.221+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1668242389 >> 172.21.15.63:0/1668242389 conn(0x5653b9011390 msgr2=0x5653b90f6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:46.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.222+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1668242389 shutdown_connections 2024-05-27T07:50:46.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.222+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1668242389 wait complete. 2024-05-27T07:50:46.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.222+0000 7f9ce0da8b80 20 client.18714 trim_cache size 0 max 16384 2024-05-27T07:50:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.240+0000 7f9ce0da8b80 1 Processor -- start 2024-05-27T07:50:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.240+0000 7f9ce0da8b80 1 -- start start 2024-05-27T07:50:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.240+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b90fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.241+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b90639f0 0x5653b90fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.241+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b90fde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.241+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b5010 con 0x5653b8f77e40 2024-05-27T07:50:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.241+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b28a0 con 0x5653b90f6e70 2024-05-27T07:50:46.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.241+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b0710 con 0x5653b90639f0 2024-05-27T07:50:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.241+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b90fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.241+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b90fbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44922/0 (socket says 172.21.15.63:44922) 2024-05-27T07:50:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.241+0000 7f9ccb400700 1 -- 172.21.15.63:0/1230381406 learned_addr learned my addr 172.21.15.63:0/1230381406 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.241+0000 7f9ccaa00700 1 --2- 172.21.15.63:0/1230381406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b90639f0 0x5653b90fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.241+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/1230381406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b90fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.242+0000 7f9ccb400700 1 -- 172.21.15.63:0/1230381406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b90639f0 msgr2=0x5653b90fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.242+0000 7f9ccb400700 1 --2- 172.21.15.63:0/1230381406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b90639f0 0x5653b90fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.242+0000 7f9ccb400700 1 -- 172.21.15.63:0/1230381406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 msgr2=0x5653b90fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.242+0000 7f9ccb400700 1 --2- 172.21.15.63:0/1230381406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b90fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.242+0000 7f9ccb400700 1 -- 172.21.15.63:0/1230381406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653b8fe5060 con 0x5653b8f77e40 2024-05-27T07:50:46.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.242+0000 7f9ccb400700 1 --2- 172.21.15.63:0/1230381406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b90fbe90 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0a5570 tx=0x7f9cbc0dfa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda995481a6c1032 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:46.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.242+0000 7f9cb7e00700 1 -- 172.21.15.63:0/1230381406 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc020070 con 0x5653b8f77e40 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.242+0000 7f9cb7e00700 1 -- 172.21.15.63:0/1230381406 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cbc0a9540 con 0x5653b8f77e40 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.242+0000 7f9cb7e00700 1 -- 172.21.15.63:0/1230381406 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0a8870 con 0x5653b8f77e40 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.243+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1230381406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 msgr2=0x5653b90fbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.243+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1230381406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b90fbe90 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0a5570 tx=0x7f9cbc0dfa60 comp rx=0 tx=0).stop 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.244+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1230381406 shutdown_connections 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.244+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1230381406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90f6e70 0x5653b90fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.244+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1230381406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b8f77e40 0x5653b90fbe90 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.244+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/1230381406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b90639f0 0x5653b90fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.244+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1230381406 >> 172.21.15.63:0/1230381406 conn(0x5653b9011390 msgr2=0x5653b90f2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.244+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1230381406 shutdown_connections 2024-05-27T07:50:46.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.244+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/1230381406 wait complete. 2024-05-27T07:50:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.244+0000 7f9ce0da8b80 1 -- start start 2024-05-27T07:50:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.245+0000 7f9ce0da8b80 1 Processor -- start 2024-05-27T07:50:46.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.245+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 0x5653b9116180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.245+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90639f0 0x5653b9113500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.245+0000 7f9ce0da8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90f6e70 0x5653b9113a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.245+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b0d60 con 0x5653b90f6e70 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.245+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b21d0 con 0x5653b90639f0 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.245+0000 7f9ce0da8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b1430 con 0x5653b8f77e40 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.245+0000 7f9ce0da8b80 10 client.0 _ll_register_callbacks cb 0x5653b8fbec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.246+0000 7f9ccbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90f6e70 0x5653b9113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.246+0000 7f9ccbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90f6e70 0x5653b9113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44934/0 (socket says 172.21.15.63:44934) 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.246+0000 7f9ccbe00700 1 -- 172.21.15.63:0/3085057877 learned_addr learned my addr 172.21.15.63:0/3085057877 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.246+0000 7f9ccb400700 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 0x5653b9116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.246+0000 7f9ccaa00700 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90639f0 0x5653b9113500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.246+0000 7f9ccbe00700 1 -- 172.21.15.63:0/3085057877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 msgr2=0x5653b9116180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.246+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 0x5653b9116180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.246+0000 7f9ccbe00700 1 -- 172.21.15.63:0/3085057877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90639f0 msgr2=0x5653b9113500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.246+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90639f0 0x5653b9113500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.246+0000 7f9ccbe00700 1 -- 172.21.15.63:0/3085057877 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653b9101490 con 0x5653b90f6e70 2024-05-27T07:50:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.247+0000 7f9ccbe00700 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90f6e70 0x5653b9113a40 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400b0a0 tx=0x7f9cc400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea01912771beee16 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.247+0000 7f9cb5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.247+0000 7f9cb5600700 1 -- 172.21.15.63:0/3085057877 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4031030 con 0x5653b90f6e70 2024-05-27T07:50:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.247+0000 7f9cb5600700 1 -- 172.21.15.63:0/3085057877 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f9cc401e730 con 0x5653b90f6e70 2024-05-27T07:50:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.247+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/3085057877 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cbc0a9540 con 0x5653b90f6e70 2024-05-27T07:50:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.248+0000 7f9cb5600700 1 -- 172.21.15.63:0/3085057877 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9cc4026070 con 0x5653b90f6e70 2024-05-27T07:50:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.248+0000 7f9ce0da8b80 10 client.18720 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:46.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.248+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/3085057877 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5653b917abe0 con 0x5653b90f6e70 2024-05-27T07:50:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.248+0000 7f9cb5600700 1 -- 172.21.15.63:0/3085057877 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cbc0a9540 con 0x5653b90f6e70 2024-05-27T07:50:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.248+0000 7f9cb5600700 1 -- 172.21.15.63:0/3085057877 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9cc4011070 con 0x5653b90f6e70 2024-05-27T07:50:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.249+0000 7f9ce0da8b80 10 client.18720 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.249+0000 7f9ce0da8b80 10 client.18720 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:50:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.249+0000 7f9ce0da8b80 20 client.18720 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.249+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5653b917b340 0x5653b917d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.249+0000 7f9ce0da8b80 4 client.18720 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:50:46.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.249+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/3085057877 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653b9020570 con 0x5653b917b340 2024-05-27T07:50:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.249+0000 7f9ccb400700 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5653b917b340 0x5653b917d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.250+0000 7f9cb5600700 10 client.18720 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:50:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.250+0000 7f9ccb400700 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5653b917b340 0x5653b917d760 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cf2098a38da0779 server_cookie=1873c868d1060416 in_seq=0 out_seq=0 2024-05-27T07:50:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.251+0000 7f9cb5600700 1 -- 172.21.15.63:0/3085057877 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653b9020570 con 0x5653b917b340 2024-05-27T07:50:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.251+0000 7f9cb5600700 10 client.18720 handle_command_reply: tid=0 2024-05-27T07:50:46.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.251+0000 7f9ce0da8b80 1 client.18720 shutdown 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.251+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/3085057877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90f6e70 msgr2=0x5653b9113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.251+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90f6e70 0x5653b9113a40 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400b0a0 tx=0x7f9cc400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.252+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/3085057877 shutdown_connections 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.252+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5653b917b340 0x5653b917d760 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.252+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653b90f6e70 0x5653b9113a40 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.252+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653b8f77e40 0x5653b9116180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.252+0000 7f9ce0da8b80 1 --2- 172.21.15.63:0/3085057877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653b90639f0 0x5653b9113500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.252+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/3085057877 >> 172.21.15.63:0/3085057877 conn(0x5653b9011390 msgr2=0x5653b90f2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.252+0000 7f9cb5600700 0 client.18720 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.252+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/3085057877 shutdown_connections 2024-05-27T07:50:46.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.252+0000 7f9ce0da8b80 1 -- 172.21.15.63:0/3085057877 wait complete. 2024-05-27T07:50:46.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.252+0000 7f9ce0da8b80 20 client.18720 trim_cache size 0 max 16384 2024-05-27T07:50:46.253 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:50:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 -- 172.21.15.63:0/2125048122 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9cb00b29f0 msgr2=0x7f9cb00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9cb00b29f0 0x7f9cb00b4f00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009e60 tx=0x7f9cc4009380 comp rx=0 tx=0).stop 2024-05-27T07:50:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 -- 172.21.15.63:0/2125048122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ccc105450 msgr2=0x7f9ccc103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ccc105450 0x7f9ccc103140 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00d950 tx=0x7f9cbc00dd10 comp rx=0 tx=0).stop 2024-05-27T07:50:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 -- 172.21.15.63:0/2125048122 shutdown_connections 2024-05-27T07:50:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9cb00b29f0 0x7f9cb00b4f00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ccc10e8c0 0x7f9ccc103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ccc105450 0x7f9ccc103140 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 --2- 172.21.15.63:0/2125048122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ccc105da0 0x7f9ccc103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:46.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 -- 172.21.15.63:0/2125048122 >> 172.21.15.63:0/2125048122 conn(0x7f9ccc0fedb0 msgr2=0x7f9ccc0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:46.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.253+0000 7f9cd2200700 1 -- 172.21.15.63:0/2125048122 shutdown_connections 2024-05-27T07:50:46.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:46.254+0000 7f9cd2200700 1 -- 172.21.15.63:0/2125048122 wait complete. 2024-05-27T07:50:46.958 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:46 smithi086 ceph-mon[34687]: pgmap v541: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:50:46.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:46 smithi149 ceph-mon[35408]: pgmap v541: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:50:46.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:46 smithi063 ceph-mon[31613]: pgmap v541: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:50:47.228 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:50:47.723 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:49.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:48 smithi086 ceph-mon[34687]: pgmap v542: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:50:49.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:48 smithi149 ceph-mon[35408]: pgmap v542: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:50:49.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:48 smithi063 ceph-mon[31613]: pgmap v542: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:50:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.231+0000 7f3373600700 1 -- 172.21.15.63:0/293961987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f336c105e00 msgr2=0x7f336c10e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.231+0000 7f3373600700 1 --2- 172.21.15.63:0/293961987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f336c105e00 0x7f336c10e350 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f3358009990 tx=0x7f3358009ca0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.231+0000 7f3373600700 1 -- 172.21.15.63:0/293961987 shutdown_connections 2024-05-27T07:50:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.231+0000 7f3373600700 1 --2- 172.21.15.63:0/293961987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c10e8e0 0x7f336c110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.231+0000 7f3373600700 1 --2- 172.21.15.63:0/293961987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f336c1054b0 0x7f336c1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.231+0000 7f3373600700 1 --2- 172.21.15.63:0/293961987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f336c105e00 0x7f336c10e350 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.231+0000 7f3373600700 1 -- 172.21.15.63:0/293961987 >> 172.21.15.63:0/293961987 conn(0x7f336c0fedb0 msgr2=0x7f336c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.231+0000 7f3373600700 1 -- 172.21.15.63:0/293961987 shutdown_connections 2024-05-27T07:50:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.231+0000 7f3373600700 1 -- 172.21.15.63:0/293961987 wait complete. 2024-05-27T07:50:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.233+0000 7f3373600700 1 Processor -- start 2024-05-27T07:50:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.233+0000 7f3373600700 1 -- start start 2024-05-27T07:50:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.233+0000 7f3373600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f336c1054b0 0x7f336c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.233+0000 7f3373600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c105e00 0x7f336c19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.233+0000 7f3373600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f336c10e8e0 0x7f336c1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.233+0000 7f3373600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1142f0 con 0x7f336c10e8e0 2024-05-27T07:50:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.233+0000 7f3373600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f336c114170 con 0x7f336c105e00 2024-05-27T07:50:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.234+0000 7f3373600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f336c114470 con 0x7f336c1054b0 2024-05-27T07:50:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.234+0000 7f3371600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f336c10e8e0 0x7f336c1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.234+0000 7f3371600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f336c10e8e0 0x7f336c1a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41866/0 (socket says 172.21.15.63:41866) 2024-05-27T07:50:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.234+0000 7f3371600700 1 -- 172.21.15.63:0/2982890280 learned_addr learned my addr 172.21.15.63:0/2982890280 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.234+0000 7f3370c00700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f336c1054b0 0x7f336c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.234+0000 7f336be00700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c105e00 0x7f336c19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.235+0000 7f3371600700 1 -- 172.21.15.63:0/2982890280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f336c1054b0 msgr2=0x7f336c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.235+0000 7f3371600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f336c1054b0 0x7f336c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.235+0000 7f3371600700 1 -- 172.21.15.63:0/2982890280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c105e00 msgr2=0x7f336c19d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.235+0000 7f3371600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c105e00 0x7f336c19d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.235+0000 7f3371600700 1 -- 172.21.15.63:0/2982890280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c1a1e30 con 0x7f336c10e8e0 2024-05-27T07:50:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.235+0000 7f3371600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f336c10e8e0 0x7f336c1a1790 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f3364007ee0 tx=0x7f336400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.235+0000 7f3369600700 1 -- 172.21.15.63:0/2982890280 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336400dd20 con 0x7f336c10e8e0 2024-05-27T07:50:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.236+0000 7f3373600700 1 -- 172.21.15.63:0/2982890280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c1a2120 con 0x7f336c10e8e0 2024-05-27T07:50:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.236+0000 7f3369600700 1 -- 172.21.15.63:0/2982890280 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3364004020 con 0x7f336c10e8e0 2024-05-27T07:50:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.236+0000 7f3369600700 1 -- 172.21.15.63:0/2982890280 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364015660 con 0x7f336c10e8e0 2024-05-27T07:50:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.236+0000 7f3373600700 1 -- 172.21.15.63:0/2982890280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c1a9a40 con 0x7f336c10e8e0 2024-05-27T07:50:50.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.241+0000 7f3369600700 1 -- 172.21.15.63:0/2982890280 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3364015800 con 0x7f336c10e8e0 2024-05-27T07:50:50.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.241+0000 7f3382085b80 1 Processor -- start 2024-05-27T07:50:50.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.241+0000 7f3382085b80 1 -- start start 2024-05-27T07:50:50.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.242+0000 7f3382085b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693eeb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.242+0000 7f3382085b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693eebd30 0x55f693ee5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.242+0000 7f3382085b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693ee5db0 0x55f693ee61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.242+0000 7f3382085b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f693e13e40 con 0x55f693d66e40 2024-05-27T07:50:50.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.242+0000 7f3382085b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f693e039e0 con 0x55f693ee5db0 2024-05-27T07:50:50.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.242+0000 7f3382085b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f693e07a20 con 0x55f693eebd30 2024-05-27T07:50:50.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.242+0000 7f336be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693eeb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.242+0000 7f336be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693eeb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41882/0 (socket says 172.21.15.63:41882) 2024-05-27T07:50:50.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.242+0000 7f336be00700 1 -- 172.21.15.63:0/2008034648 learned_addr learned my addr 172.21.15.63:0/2008034648 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:50.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.243+0000 7f3370c00700 1 --2- 172.21.15.63:0/2008034648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693eebd30 0x55f693ee5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.243+0000 7f336be00700 1 --2- 172.21.15.63:0/2008034648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693ee5db0 0x55f693ee61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.243+0000 7f3369600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33540b2a70 0x7f33540b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.243+0000 7f336be00700 1 -- 172.21.15.63:0/2008034648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693eebd30 msgr2=0x55f693ee5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.243+0000 7f336be00700 1 --2- 172.21.15.63:0/2008034648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693eebd30 0x55f693ee5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.243+0000 7f336be00700 1 -- 172.21.15.63:0/2008034648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693ee5db0 msgr2=0x55f693ee61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.243+0000 7f336be00700 1 --2- 172.21.15.63:0/2008034648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693ee5db0 0x55f693ee61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.243+0000 7f336be00700 1 -- 172.21.15.63:0/2008034648 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f693dcc420 con 0x55f693d66e40 2024-05-27T07:50:50.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.243+0000 7f3369600700 1 -- 172.21.15.63:0/2982890280 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f33640e2350 con 0x7f336c10e8e0 2024-05-27T07:50:50.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.243+0000 7f3371600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33540b2a70 0x7f33540b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.244+0000 7f336be00700 1 --2- 172.21.15.63:0/2008034648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693eeb7f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f3358005850 tx=0x7f33580057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3e3c1c8ec0a45e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.244+0000 7f335e000700 1 -- 172.21.15.63:0/2008034648 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335800bdb0 con 0x55f693d66e40 2024-05-27T07:50:50.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.244+0000 7f335e000700 1 -- 172.21.15.63:0/2008034648 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33580055a0 con 0x55f693d66e40 2024-05-27T07:50:50.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.244+0000 7f335e000700 1 -- 172.21.15.63:0/2008034648 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335800f460 con 0x55f693d66e40 2024-05-27T07:50:50.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3382085b80 1 -- 172.21.15.63:0/2008034648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 msgr2=0x55f693eeb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3371600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33540b2a70 0x7f33540b4f80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3364009790 tx=0x7f33640096c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3382085b80 1 --2- 172.21.15.63:0/2008034648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693eeb7f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f3358005850 tx=0x7f33580057e0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3382085b80 1 -- 172.21.15.63:0/2008034648 shutdown_connections 2024-05-27T07:50:50.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3382085b80 1 --2- 172.21.15.63:0/2008034648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693ee5db0 0x55f693ee61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3382085b80 1 --2- 172.21.15.63:0/2008034648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693eeb7f0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3382085b80 1 --2- 172.21.15.63:0/2008034648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693eebd30 0x55f693ee5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3382085b80 1 -- 172.21.15.63:0/2008034648 >> 172.21.15.63:0/2008034648 conn(0x55f693dc6fc0 msgr2=0x55f693ee08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3382085b80 1 -- 172.21.15.63:0/2008034648 shutdown_connections 2024-05-27T07:50:50.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3382085b80 1 -- 172.21.15.63:0/2008034648 wait complete. 2024-05-27T07:50:50.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.246+0000 7f3382085b80 1 -- start start 2024-05-27T07:50:50.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.247+0000 7f3382085b80 1 Processor -- start 2024-05-27T07:50:50.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.247+0000 7f3382085b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693d66e40 0x55f693eef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.247+0000 7f3382085b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693eefae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.247+0000 7f3382085b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693ef0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.247+0000 7f3382085b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f693d09fa0 con 0x55f693eebd30 2024-05-27T07:50:50.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.247+0000 7f3382085b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f693da4760 con 0x55f693d66e40 2024-05-27T07:50:50.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.247+0000 7f3382085b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f693ef27a0 con 0x55f693ee5db0 2024-05-27T07:50:50.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.248+0000 7f3382085b80 10 client.0 _ll_register_callbacks cb 0x55f693dadc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:50.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.248+0000 7f336be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693ef0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.248+0000 7f336be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693ef0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41890/0 (socket says 172.21.15.63:41890) 2024-05-27T07:50:50.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.248+0000 7f336be00700 1 -- 172.21.15.63:0/3441032512 learned_addr learned my addr 172.21.15.63:0/3441032512 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:50.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.248+0000 7f336be00700 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693d66e40 0x55f693eef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.248+0000 7f3370c00700 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693eefae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.249+0000 7f336be00700 1 -- 172.21.15.63:0/3441032512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 msgr2=0x55f693eefae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.249+0000 7f336be00700 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693eefae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.249+0000 7f336be00700 1 -- 172.21.15.63:0/3441032512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693d66e40 msgr2=0x55f693eef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.249+0000 7f336be00700 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693d66e40 0x55f693eef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.249+0000 7f336be00700 1 -- 172.21.15.63:0/3441032512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f693d538d0 con 0x55f693eebd30 2024-05-27T07:50:50.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.249+0000 7f336be00700 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693ef0020 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f3358009350 tx=0x7f335800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e17d92b471f7ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.249+0000 7f3352a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:50.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.250+0000 7f3352a00700 1 -- 172.21.15.63:0/3441032512 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335800bbb0 con 0x55f693eebd30 2024-05-27T07:50:50.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.250+0000 7f3382085b80 1 -- 172.21.15.63:0/3441032512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33580055a0 con 0x55f693eebd30 2024-05-27T07:50:50.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.250+0000 7f3352a00700 1 -- 172.21.15.63:0/3441032512 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f335800bd50 con 0x55f693eebd30 2024-05-27T07:50:50.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.250+0000 7f3352a00700 1 -- 172.21.15.63:0/3441032512 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f335803d050 con 0x55f693eebd30 2024-05-27T07:50:50.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.250+0000 7f3382085b80 10 client.18732 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:50.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.250+0000 7f3382085b80 1 -- 172.21.15.63:0/3441032512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f693dd4060 con 0x55f693eebd30 2024-05-27T07:50:50.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.251+0000 7f3352a00700 1 -- 172.21.15.63:0/3441032512 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33580055a0 con 0x55f693eebd30 2024-05-27T07:50:50.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.251+0000 7f3352a00700 1 -- 172.21.15.63:0/3441032512 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f335800fc10 con 0x55f693eebd30 2024-05-27T07:50:50.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.251+0000 7f3382085b80 10 client.18732 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:50.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.251+0000 7f3382085b80 10 client.18732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:50:50.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.251+0000 7f3382085b80 20 client.18732 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:50.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.251+0000 7f3382085b80 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f693f07dd0 0x55f693f0a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.251+0000 7f3382085b80 4 client.18732 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:50.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.251+0000 7f3382085b80 1 -- 172.21.15.63:0/3441032512 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f693e15d90 con 0x55f693f07dd0 2024-05-27T07:50:50.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.252+0000 7f3370c00700 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f693f07dd0 0x55f693f0a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.253+0000 7f3370c00700 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f693f07dd0 0x55f693f0a1f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc41066231130e44 server_cookie=8ce5bdb64a7f54ad in_seq=0 out_seq=0 2024-05-27T07:50:50.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.253+0000 7f3352a00700 10 client.18732 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:50:50.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.255+0000 7f3352a00700 1 -- 172.21.15.63:0/3441032512 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f693e15d90 con 0x55f693f07dd0 2024-05-27T07:50:50.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.255+0000 7f3352a00700 10 client.18732 handle_command_reply: tid=0 2024-05-27T07:50:50.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.255+0000 7f3382085b80 1 client.18732 shutdown 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.255+0000 7f3382085b80 1 -- 172.21.15.63:0/3441032512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 msgr2=0x55f693ef0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.255+0000 7f3382085b80 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693ef0020 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f3358009350 tx=0x7f335800b860 comp rx=0 tx=0).stop 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.255+0000 7f3382085b80 1 -- 172.21.15.63:0/3441032512 shutdown_connections 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.255+0000 7f3382085b80 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f693f07dd0 0x55f693f0a1f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.256+0000 7f3382085b80 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693ef0020 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.256+0000 7f3382085b80 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693d66e40 0x55f693eef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.256+0000 7f3382085b80 1 --2- 172.21.15.63:0/3441032512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693eefae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.256+0000 7f3382085b80 1 -- 172.21.15.63:0/3441032512 >> 172.21.15.63:0/3441032512 conn(0x55f693dc6fc0 msgr2=0x55f693ee0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.256+0000 7f3352a00700 0 client.18732 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.256+0000 7f3382085b80 1 -- 172.21.15.63:0/3441032512 shutdown_connections 2024-05-27T07:50:50.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.256+0000 7f3382085b80 1 -- 172.21.15.63:0/3441032512 wait complete. 2024-05-27T07:50:50.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.256+0000 7f3382085b80 20 client.18732 trim_cache size 0 max 16384 2024-05-27T07:50:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.273+0000 7f3382085b80 1 Processor -- start 2024-05-27T07:50:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.273+0000 7f3382085b80 1 -- start start 2024-05-27T07:50:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.273+0000 7f3382085b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693ee5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f3382085b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693eee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f3382085b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693eebd30 0x55f693eed150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f3382085b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1b0a90 con 0x55f693d66e40 2024-05-27T07:50:50.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f3382085b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1ae320 con 0x55f693eebd30 2024-05-27T07:50:50.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f3382085b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1ac1d0 con 0x55f693ee5db0 2024-05-27T07:50:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f336be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693ee5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f336be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693ee5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41906/0 (socket says 172.21.15.63:41906) 2024-05-27T07:50:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f336be00700 1 -- 172.21.15.63:0/3391445140 learned_addr learned my addr 172.21.15.63:0/3391445140 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f3370c00700 1 --2- 172.21.15.63:0/3391445140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693eee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f336be00700 1 -- 172.21.15.63:0/3391445140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 msgr2=0x55f693eee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f336be00700 1 --2- 172.21.15.63:0/3391445140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693eee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f336be00700 1 -- 172.21.15.63:0/3391445140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693eebd30 msgr2=0x55f693eed150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f336be00700 1 --2- 172.21.15.63:0/3391445140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693eebd30 0x55f693eed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.274+0000 7f336be00700 1 -- 172.21.15.63:0/3391445140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f693edccf0 con 0x55f693d66e40 2024-05-27T07:50:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.275+0000 7f336be00700 1 --2- 172.21.15.63:0/3391445140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693ee5870 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f3358003760 tx=0x7f3358009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fefb5eb45b812 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.275+0000 7f335d600700 1 -- 172.21.15.63:0/3391445140 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335800b5c0 con 0x55f693d66e40 2024-05-27T07:50:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.275+0000 7f335d600700 1 -- 172.21.15.63:0/3391445140 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3358026960 con 0x55f693d66e40 2024-05-27T07:50:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.275+0000 7f335d600700 1 -- 172.21.15.63:0/3391445140 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358005370 con 0x55f693d66e40 2024-05-27T07:50:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.276+0000 7f3382085b80 1 -- 172.21.15.63:0/3391445140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 msgr2=0x55f693ee5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.276+0000 7f3382085b80 1 --2- 172.21.15.63:0/3391445140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693ee5870 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f3358003760 tx=0x7f3358009290 comp rx=0 tx=0).stop 2024-05-27T07:50:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.276+0000 7f3382085b80 1 -- 172.21.15.63:0/3391445140 shutdown_connections 2024-05-27T07:50:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.276+0000 7f3382085b80 1 --2- 172.21.15.63:0/3391445140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693eebd30 0x55f693eed150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.276+0000 7f3382085b80 1 --2- 172.21.15.63:0/3391445140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693d66e40 0x55f693ee5870 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.276+0000 7f3382085b80 1 --2- 172.21.15.63:0/3391445140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693eee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.276+0000 7f3382085b80 1 -- 172.21.15.63:0/3391445140 >> 172.21.15.63:0/3391445140 conn(0x55f693dc6fc0 msgr2=0x55f693eef560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.277+0000 7f3382085b80 1 -- 172.21.15.63:0/3391445140 shutdown_connections 2024-05-27T07:50:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.277+0000 7f3382085b80 1 -- 172.21.15.63:0/3391445140 wait complete. 2024-05-27T07:50:50.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.277+0000 7f3382085b80 1 -- start start 2024-05-27T07:50:50.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.277+0000 7f3382085b80 1 Processor -- start 2024-05-27T07:50:50.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.278+0000 7f3382085b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693d66e40 0x55f693f09de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.278+0000 7f3382085b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693f0a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.278+0000 7f3382085b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693f0a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.278+0000 7f3382085b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1ac820 con 0x55f693eebd30 2024-05-27T07:50:50.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.278+0000 7f3382085b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1adc50 con 0x55f693d66e40 2024-05-27T07:50:50.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.278+0000 7f3382085b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f336c1aceb0 con 0x55f693ee5db0 2024-05-27T07:50:50.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.278+0000 7f3382085b80 10 client.0 _ll_register_callbacks cb 0x55f693dadc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:50.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.278+0000 7f336be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693f0a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.279+0000 7f336be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693f0a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41910/0 (socket says 172.21.15.63:41910) 2024-05-27T07:50:50.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.279+0000 7f336be00700 1 -- 172.21.15.63:0/1740727879 learned_addr learned my addr 172.21.15.63:0/1740727879 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:50.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.279+0000 7f3370c00700 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693f0a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.279+0000 7f336be00700 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693d66e40 0x55f693f09de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.279+0000 7f336be00700 1 -- 172.21.15.63:0/1740727879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 msgr2=0x55f693f0a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.279+0000 7f336be00700 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693f0a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.279+0000 7f336be00700 1 -- 172.21.15.63:0/1740727879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693d66e40 msgr2=0x55f693f09de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.279+0000 7f336be00700 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693d66e40 0x55f693f09de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.279+0000 7f336be00700 1 -- 172.21.15.63:0/1740727879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f693f70630 con 0x55f693eebd30 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.280+0000 7f336be00700 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693f0a860 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f33580055d0 tx=0x7f33580059e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d689e1869259295 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.280+0000 7f3352a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.280+0000 7f3352a00700 1 -- 172.21.15.63:0/1740727879 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358005320 con 0x55f693eebd30 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.280+0000 7f3352a00700 1 -- 172.21.15.63:0/1740727879 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f3358026b10 con 0x55f693eebd30 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.280+0000 7f3382085b80 1 -- 172.21.15.63:0/1740727879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3358026960 con 0x55f693eebd30 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.280+0000 7f3352a00700 1 -- 172.21.15.63:0/1740727879 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f335800bab0 con 0x55f693eebd30 2024-05-27T07:50:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.281+0000 7f3382085b80 10 client.18738 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:50.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.281+0000 7f3382085b80 1 -- 172.21.15.63:0/1740727879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f693f71810 con 0x55f693eebd30 2024-05-27T07:50:50.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.281+0000 7f3352a00700 1 -- 172.21.15.63:0/1740727879 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3358026960 con 0x55f693eebd30 2024-05-27T07:50:50.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.282+0000 7f3352a00700 1 -- 172.21.15.63:0/1740727879 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3358036430 con 0x55f693eebd30 2024-05-27T07:50:50.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.282+0000 7f3382085b80 10 client.18738 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:50.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.282+0000 7f3382085b80 10 client.18738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:50:50.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.282+0000 7f3382085b80 20 client.18738 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.282+0000 7f3382085b80 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f693f71d60 0x55f693f74180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.282+0000 7f3382085b80 4 client.18738 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:50:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.282+0000 7f3382085b80 1 -- 172.21.15.63:0/1740727879 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f693e0f570 con 0x55f693f71d60 2024-05-27T07:50:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.282+0000 7f3370c00700 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f693f71d60 0x55f693f74180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.283+0000 7f3370c00700 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f693f71d60 0x55f693f74180 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7df9744910d97b1 server_cookie=68dc396d010d1450 in_seq=0 out_seq=0 2024-05-27T07:50:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.283+0000 7f3352a00700 10 client.18738 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:50:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3352a00700 1 -- 172.21.15.63:0/1740727879 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f693e0f570 con 0x55f693f71d60 2024-05-27T07:50:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3352a00700 10 client.18738 handle_command_reply: tid=0 2024-05-27T07:50:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3382085b80 1 client.18738 shutdown 2024-05-27T07:50:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3382085b80 1 -- 172.21.15.63:0/1740727879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 msgr2=0x55f693f0a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3382085b80 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693f0a860 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f33580055d0 tx=0x7f33580059e0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3382085b80 1 -- 172.21.15.63:0/1740727879 shutdown_connections 2024-05-27T07:50:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3382085b80 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f693f71d60 0x55f693f74180 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3382085b80 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f693eebd30 0x55f693f0a860 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3382085b80 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f693d66e40 0x55f693f09de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3382085b80 1 --2- 172.21.15.63:0/1740727879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f693ee5db0 0x55f693f0a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3382085b80 1 -- 172.21.15.63:0/1740727879 >> 172.21.15.63:0/1740727879 conn(0x55f693dc6fc0 msgr2=0x55f693ee9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.284+0000 7f3352a00700 0 client.18738 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:50:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.285+0000 7f3382085b80 1 -- 172.21.15.63:0/1740727879 shutdown_connections 2024-05-27T07:50:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.285+0000 7f3382085b80 1 -- 172.21.15.63:0/1740727879 wait complete. 2024-05-27T07:50:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.285+0000 7f3382085b80 20 client.18738 trim_cache size 0 max 16384 2024-05-27T07:50:50.286 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 -- 172.21.15.63:0/2982890280 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33540b2a70 msgr2=0x7f33540b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33540b2a70 0x7f33540b4f80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3364009790 tx=0x7f33640096c0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 -- 172.21.15.63:0/2982890280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f336c10e8e0 msgr2=0x7f336c1a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f336c10e8e0 0x7f336c1a1790 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f3364007ee0 tx=0x7f336400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 -- 172.21.15.63:0/2982890280 shutdown_connections 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33540b2a70 0x7f33540b4f80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f336c10e8e0 0x7f336c1a1790 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f336c1054b0 0x7f336c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 --2- 172.21.15.63:0/2982890280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f336c105e00 0x7f336c19d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 -- 172.21.15.63:0/2982890280 >> 172.21.15.63:0/2982890280 conn(0x7f336c0fedb0 msgr2=0x7f336c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 -- 172.21.15.63:0/2982890280 shutdown_connections 2024-05-27T07:50:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:50.286+0000 7f3373600700 1 -- 172.21.15.63:0/2982890280 wait complete. 2024-05-27T07:50:50.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:50 smithi063 ceph-mon[31613]: pgmap v543: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:50:51.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:50 smithi086 ceph-mon[34687]: pgmap v543: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:50:51.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:50 smithi149 ceph-mon[35408]: pgmap v543: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:50:51.239 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:50:51.739 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.475+0000 7ff7eb200640 1 -- 172.21.15.63:0/302233399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7e4109f70 msgr2=0x7ff7e4103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.475+0000 7ff7eb200640 1 --2- 172.21.15.63:0/302233399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7e4109f70 0x7ff7e4103bb0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009900 tx=0x7ff7d402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.475+0000 7ff7eb200640 1 -- 172.21.15.63:0/302233399 shutdown_connections 2024-05-27T07:50:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.475+0000 7ff7eb200640 1 --2- 172.21.15.63:0/302233399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7e4104220 0x7ff7e41105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.475+0000 7ff7eb200640 1 --2- 172.21.15.63:0/302233399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7e4109f70 0x7ff7e4103bb0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.475+0000 7ff7eb200640 1 --2- 172.21.15.63:0/302233399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e4109ad0 0x7ff7e4103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.475+0000 7ff7eb200640 1 -- 172.21.15.63:0/302233399 >> 172.21.15.63:0/302233399 conn(0x7ff7e4079100 msgr2=0x7ff7e4079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.475+0000 7ff7eb200640 1 -- 172.21.15.63:0/302233399 shutdown_connections 2024-05-27T07:50:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.475+0000 7ff7eb200640 1 -- 172.21.15.63:0/302233399 wait complete. 2024-05-27T07:50:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.477+0000 7ff7eb200640 1 Processor -- start 2024-05-27T07:50:52.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.477+0000 7ff7eb200640 1 -- start start 2024-05-27T07:50:52.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.477+0000 7ff7eb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e4104220 0x7ff7e4074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.477+0000 7ff7eb200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7e4109ad0 0x7ff7e4075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.477+0000 7ff7eb200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7e4109f70 0x7ff7e4075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.477+0000 7ff7eb200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e4113bc0 con 0x7ff7e4109ad0 2024-05-27T07:50:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.477+0000 7ff7eb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e4113a40 con 0x7ff7e4104220 2024-05-27T07:50:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.477+0000 7ff7eb200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e4113d40 con 0x7ff7e4109f70 2024-05-27T07:50:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.478+0000 7ff7e9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7e4109ad0 0x7ff7e4075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.478+0000 7ff7e9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7e4109ad0 0x7ff7e4075500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41934/0 (socket says 172.21.15.63:41934) 2024-05-27T07:50:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.478+0000 7ff7e9400640 1 -- 172.21.15.63:0/2782844826 learned_addr learned my addr 172.21.15.63:0/2782844826 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:52.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.478+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e4104220 0x7ff7e4074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.478+0000 7ff7ea800640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7e4109f70 0x7ff7e4075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.478+0000 7ff7e9400640 1 -- 172.21.15.63:0/2782844826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7e4109f70 msgr2=0x7ff7e4075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.478+0000 7ff7e9400640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7e4109f70 0x7ff7e4075a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.478+0000 7ff7e9400640 1 -- 172.21.15.63:0/2782844826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e4104220 msgr2=0x7ff7e4074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.478+0000 7ff7e9400640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e4104220 0x7ff7e4074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.478+0000 7ff7e9400640 1 -- 172.21.15.63:0/2782844826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e41a88d0 con 0x7ff7e4109ad0 2024-05-27T07:50:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.479+0000 7ff7e9400640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7e4109ad0 0x7ff7e4075500 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff7d402a910 tx=0x7ff7d4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.479+0000 7ff7dea00640 1 -- 172.21.15.63:0/2782844826 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d403f070 con 0x7ff7e4109ad0 2024-05-27T07:50:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.479+0000 7ff7eb200640 1 -- 172.21.15.63:0/2782844826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e41a8b60 con 0x7ff7e4109ad0 2024-05-27T07:50:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.480+0000 7ff7dea00640 1 -- 172.21.15.63:0/2782844826 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7d4003e10 con 0x7ff7e4109ad0 2024-05-27T07:50:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.480+0000 7ff7dea00640 1 -- 172.21.15.63:0/2782844826 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4035030 con 0x7ff7e4109ad0 2024-05-27T07:50:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.480+0000 7ff7eb200640 1 -- 172.21.15.63:0/2782844826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e41a9050 con 0x7ff7e4109ad0 2024-05-27T07:50:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.484+0000 7ff7f37a9740 1 Processor -- start 2024-05-27T07:50:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.484+0000 7ff7f37a9740 1 -- start start 2024-05-27T07:50:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.484+0000 7ff7dea00640 1 -- 172.21.15.63:0/2782844826 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff7d4042070 con 0x7ff7e4109ad0 2024-05-27T07:50:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4ff1b0 0x56207d8a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4de4d0 0x56207d8a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56207d812b90 con 0x56207d4d62f0 2024-05-27T07:50:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56207d802730 con 0x56207d4de4d0 2024-05-27T07:50:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56207d806770 con 0x56207d4ff1b0 2024-05-27T07:50:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7e9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7e9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41948/0 (socket says 172.21.15.63:41948) 2024-05-27T07:50:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7e9e00640 1 -- 172.21.15.63:0/94809585 learned_addr learned my addr 172.21.15.63:0/94809585 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7ea800640 1 --2- 172.21.15.63:0/94809585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4ff1b0 0x56207d8a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.485+0000 7ff7e9400640 1 --2- 172.21.15.63:0/94809585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4de4d0 0x56207d8a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.486+0000 7ff7e9e00640 1 -- 172.21.15.63:0/94809585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4ff1b0 msgr2=0x56207d8a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.486+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/94809585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4ff1b0 0x56207d8a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.486+0000 7ff7e9e00640 1 -- 172.21.15.63:0/94809585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4de4d0 msgr2=0x56207d8a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.486+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/94809585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4de4d0 0x56207d8a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.486+0000 7ff7e9e00640 1 -- 172.21.15.63:0/94809585 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207d79b8b0 con 0x56207d4d62f0 2024-05-27T07:50:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.486+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/94809585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a1fa0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800cbb0 tx=0x7ff7d800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74f6a832079a2ae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.486+0000 7ff7d3000640 1 -- 172.21.15.63:0/94809585 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8005220 con 0x56207d4d62f0 2024-05-27T07:50:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.487+0000 7ff7dea00640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7cc0b2950 0x7ff7cc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.487+0000 7ff7d3000640 1 -- 172.21.15.63:0/94809585 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7d8004c30 con 0x56207d4d62f0 2024-05-27T07:50:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.487+0000 7ff7d3000640 1 -- 172.21.15.63:0/94809585 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8010690 con 0x56207d4d62f0 2024-05-27T07:50:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.487+0000 7ff7dea00640 1 -- 172.21.15.63:0/2782844826 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7ff7d4103220 con 0x7ff7e4109ad0 2024-05-27T07:50:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.487+0000 7ff7ea800640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7cc0b2950 0x7ff7cc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.488+0000 7ff7f37a9740 1 -- 172.21.15.63:0/94809585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 msgr2=0x56207d8a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.488+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/94809585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a1fa0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800cbb0 tx=0x7ff7d800e720 comp rx=0 tx=0).stop 2024-05-27T07:50:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.488+0000 7ff7ea800640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7cc0b2950 0x7ff7cc0b4e60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x56207d89ebd0 tx=0x7ff7e0009600 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.488+0000 7ff7f37a9740 1 -- 172.21.15.63:0/94809585 shutdown_connections 2024-05-27T07:50:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.488+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/94809585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4de4d0 0x56207d8a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.488+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/94809585 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a1fa0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.488+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/94809585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4ff1b0 0x56207d8a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.488+0000 7ff7f37a9740 1 -- 172.21.15.63:0/94809585 >> 172.21.15.63:0/94809585 conn(0x56207d4f3300 msgr2=0x56207d89e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.488+0000 7ff7f37a9740 1 -- 172.21.15.63:0/94809585 shutdown_connections 2024-05-27T07:50:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.488+0000 7ff7f37a9740 1 -- 172.21.15.63:0/94809585 wait complete. 2024-05-27T07:50:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.489+0000 7ff7f37a9740 1 -- start start 2024-05-27T07:50:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.489+0000 7ff7f37a9740 1 Processor -- start 2024-05-27T07:50:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8abc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4de4d0 0x56207d8ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4ff1b0 0x56207d8c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56207d4e8790 con 0x56207d4d62f0 2024-05-27T07:50:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56207d7aabc0 con 0x56207d4ff1b0 2024-05-27T07:50:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56207d8aeb90 con 0x56207d4de4d0 2024-05-27T07:50:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7f37a9740 10 client.0 _ll_register_callbacks cb 0x56207d788400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7e9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7e9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8abc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41964/0 (socket says 172.21.15.63:41964) 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7e9e00640 1 -- 172.21.15.63:0/2669484917 learned_addr learned my addr 172.21.15.63:0/2669484917 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.490+0000 7ff7e9400640 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4de4d0 0x56207d8ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.491+0000 7ff7ea800640 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4ff1b0 0x56207d8c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.491+0000 7ff7e9e00640 1 -- 172.21.15.63:0/2669484917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4de4d0 msgr2=0x56207d8ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.491+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4de4d0 0x56207d8ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.491+0000 7ff7e9e00640 1 -- 172.21.15.63:0/2669484917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4ff1b0 msgr2=0x56207d8c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.491+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4ff1b0 0x56207d8c0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.491+0000 7ff7e9e00640 1 -- 172.21.15.63:0/2669484917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207d517210 con 0x56207d4d62f0 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.491+0000 7ff7ea800640 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4ff1b0 0x56207d8c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.491+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8abc50 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8005890 tx=0x7ff7d800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1d155f2e58cb04 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.491+0000 7ff7c7400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.491+0000 7ff7c7400640 1 -- 172.21.15.63:0/2669484917 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8029060 con 0x56207d4d62f0 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.492+0000 7ff7c7400640 1 -- 172.21.15.63:0/2669484917 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7ff7d8004250 con 0x56207d4d62f0 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.492+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2669484917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d8004c30 con 0x56207d4d62f0 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.492+0000 7ff7c7400640 1 -- 172.21.15.63:0/2669484917 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff7d8030050 con 0x56207d4d62f0 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.492+0000 7ff7f37a9740 10 client.18750 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.492+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2669484917 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56207d5e3620 con 0x56207d4d62f0 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.492+0000 7ff7c7400640 1 -- 172.21.15.63:0/2669484917 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7d8004c30 con 0x56207d4d62f0 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.492+0000 7ff7c7400640 1 -- 172.21.15.63:0/2669484917 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff7d80037b0 con 0x56207d4d62f0 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.493+0000 7ff7f37a9740 10 client.18750 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.493+0000 7ff7f37a9740 10 client.18750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.493+0000 7ff7f37a9740 20 client.18750 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.493+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56207d4bdd80 0x56207d8c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.493+0000 7ff7f37a9740 4 client.18750 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.493+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2669484917 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56207d814ae0 con 0x56207d4bdd80 2024-05-27T07:50:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.493+0000 7ff7e9400640 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56207d4bdd80 0x56207d8c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.494+0000 7ff7e9400640 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56207d4bdd80 0x56207d8c65f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df19fef6cb182c0 server_cookie=747c8c2067b6e2a0 in_seq=0 out_seq=0 2024-05-27T07:50:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.494+0000 7ff7c7400640 10 client.18750 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.496+0000 7ff7c7400640 1 -- 172.21.15.63:0/2669484917 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56207d814ae0 con 0x56207d4bdd80 2024-05-27T07:50:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.496+0000 7ff7c7400640 10 client.18750 handle_command_reply: tid=0 2024-05-27T07:50:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.496+0000 7ff7f37a9740 1 client.18750 shutdown 2024-05-27T07:50:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2669484917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 msgr2=0x56207d8abc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8abc50 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8005890 tx=0x7ff7d800c900 comp rx=0 tx=0).stop 2024-05-27T07:50:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2669484917 shutdown_connections 2024-05-27T07:50:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56207d4bdd80 0x56207d8c65f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4ff1b0 0x56207d8c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4de4d0 0x56207d8ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2669484917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8abc50 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7c7400640 0 client.18750 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2669484917 >> 172.21.15.63:0/2669484917 conn(0x56207d4f3300 msgr2=0x56207d8a3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2669484917 shutdown_connections 2024-05-27T07:50:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2669484917 wait complete. 2024-05-27T07:50:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.497+0000 7ff7f37a9740 20 client.18750 trim_cache size 0 max 16384 2024-05-27T07:50:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.518+0000 7ff7f37a9740 1 Processor -- start 2024-05-27T07:50:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.518+0000 7ff7f37a9740 1 -- start start 2024-05-27T07:50:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4de4d0 0x56207d8ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4ff1b0 0x56207d8c06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e41b1640 con 0x56207d4d62f0 2024-05-27T07:50:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e41aef10 con 0x56207d4de4d0 2024-05-27T07:50:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e41acdc0 con 0x56207d4ff1b0 2024-05-27T07:50:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7e9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7e9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41974/0 (socket says 172.21.15.63:41974) 2024-05-27T07:50:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7e9e00640 1 -- 172.21.15.63:0/3205995499 learned_addr learned my addr 172.21.15.63:0/3205995499 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7ea800640 1 --2- 172.21.15.63:0/3205995499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4ff1b0 0x56207d8c06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.519+0000 7ff7e9400640 1 --2- 172.21.15.63:0/3205995499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4de4d0 0x56207d8ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.520+0000 7ff7e9e00640 1 -- 172.21.15.63:0/3205995499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4ff1b0 msgr2=0x56207d8c06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.520+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/3205995499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4ff1b0 0x56207d8c06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.520+0000 7ff7e9e00640 1 -- 172.21.15.63:0/3205995499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4de4d0 msgr2=0x56207d8ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.520+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/3205995499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4de4d0 0x56207d8ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.520+0000 7ff7e9e00640 1 -- 172.21.15.63:0/3205995499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207d8cc750 con 0x56207d4d62f0 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.520+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/3205995499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a7440 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e760 tx=0x7ff7d8003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b871168b32ff1c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.520+0000 7ff7d2600640 1 -- 172.21.15.63:0/3205995499 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8029070 con 0x56207d4d62f0 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.520+0000 7ff7d2600640 1 -- 172.21.15.63:0/3205995499 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7d800ca30 con 0x56207d4d62f0 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.520+0000 7ff7d2600640 1 -- 172.21.15.63:0/3205995499 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80108b0 con 0x56207d4d62f0 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.521+0000 7ff7f37a9740 1 -- 172.21.15.63:0/3205995499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 msgr2=0x56207d8a7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.521+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/3205995499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a7440 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e760 tx=0x7ff7d8003fa0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.521+0000 7ff7f37a9740 1 -- 172.21.15.63:0/3205995499 shutdown_connections 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.521+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/3205995499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4ff1b0 0x56207d8c06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.521+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/3205995499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4de4d0 0x56207d8ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.521+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/3205995499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4d62f0 0x56207d8a7440 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.521+0000 7ff7f37a9740 1 -- 172.21.15.63:0/3205995499 >> 172.21.15.63:0/3205995499 conn(0x56207d4f3300 msgr2=0x56207d8adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.522+0000 7ff7f37a9740 1 -- 172.21.15.63:0/3205995499 shutdown_connections 2024-05-27T07:50:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.522+0000 7ff7f37a9740 1 -- 172.21.15.63:0/3205995499 wait complete. 2024-05-27T07:50:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.522+0000 7ff7f37a9740 1 -- start start 2024-05-27T07:50:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.522+0000 7ff7f37a9740 1 Processor -- start 2024-05-27T07:50:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.523+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4d62f0 0x56207d8c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.523+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4de4d0 0x56207d8c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.523+0000 7ff7f37a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4ff1b0 0x56207d8c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.523+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e41ad410 con 0x56207d4ff1b0 2024-05-27T07:50:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.523+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e41ae840 con 0x56207d4d62f0 2024-05-27T07:50:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.523+0000 7ff7f37a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e41adaa0 con 0x56207d4de4d0 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.523+0000 7ff7f37a9740 10 client.0 _ll_register_callbacks cb 0x7ff7d8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.523+0000 7ff7ea800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4ff1b0 0x56207d8c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.523+0000 7ff7ea800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4ff1b0 0x56207d8c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41980/0 (socket says 172.21.15.63:41980) 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.523+0000 7ff7ea800640 1 -- 172.21.15.63:0/2003341869 learned_addr learned my addr 172.21.15.63:0/2003341869 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.524+0000 7ff7e9400640 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4de4d0 0x56207d8c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.524+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4d62f0 0x56207d8c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.524+0000 7ff7ea800640 1 -- 172.21.15.63:0/2003341869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4de4d0 msgr2=0x56207d8c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.524+0000 7ff7ea800640 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4de4d0 0x56207d8c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.524+0000 7ff7ea800640 1 -- 172.21.15.63:0/2003341869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4d62f0 msgr2=0x56207d8c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.524+0000 7ff7ea800640 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4d62f0 0x56207d8c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.524+0000 7ff7ea800640 1 -- 172.21.15.63:0/2003341869 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207d92cbe0 con 0x56207d4ff1b0 2024-05-27T07:50:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.524+0000 7ff7ea800640 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4ff1b0 0x56207d8c7030 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff7e001e790 tx=0x7ff7e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144a032a3d040040 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.524+0000 7ff7c7400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.525+0000 7ff7c7400640 1 -- 172.21.15.63:0/2003341869 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e002c060 con 0x56207d4ff1b0 2024-05-27T07:50:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.525+0000 7ff7c7400640 1 -- 172.21.15.63:0/2003341869 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7ff7e001cc40 con 0x56207d4ff1b0 2024-05-27T07:50:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.525+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2003341869 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d800ca30 con 0x56207d4ff1b0 2024-05-27T07:50:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.525+0000 7ff7c7400640 1 -- 172.21.15.63:0/2003341869 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff7e001ce40 con 0x56207d4ff1b0 2024-05-27T07:50:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.525+0000 7ff7f37a9740 10 client.18756 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.525+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2003341869 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56207d92deb0 con 0x56207d4ff1b0 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.526+0000 7ff7c7400640 1 -- 172.21.15.63:0/2003341869 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7d800ca30 con 0x56207d4ff1b0 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.526+0000 7ff7c7400640 1 -- 172.21.15.63:0/2003341869 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff7e0026070 con 0x56207d4ff1b0 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.526+0000 7ff7f37a9740 10 client.18756 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.526+0000 7ff7f37a9740 10 client.18756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.526+0000 7ff7f37a9740 20 client.18756 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.526+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56207d4931c0 0x56207d9306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.526+0000 7ff7f37a9740 4 client.18756 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.526+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2003341869 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56207d8ac5c0 con 0x56207d4931c0 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.527+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56207d4931c0 0x56207d9306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.528+0000 7ff7e9e00640 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56207d4931c0 0x56207d9306a0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=835426fb86484974 server_cookie=10c3de22934a3052 in_seq=0 out_seq=0 2024-05-27T07:50:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.528+0000 7ff7c7400640 10 client.18756 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7c7400640 1 -- 172.21.15.63:0/2003341869 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56207d8ac5c0 con 0x56207d4931c0 2024-05-27T07:50:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7c7400640 10 client.18756 handle_command_reply: tid=0 2024-05-27T07:50:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7f37a9740 1 client.18756 shutdown 2024-05-27T07:50:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2003341869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4ff1b0 msgr2=0x56207d8c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4ff1b0 0x56207d8c7030 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff7e001e790 tx=0x7ff7e0004a40 comp rx=0 tx=0).stop 2024-05-27T07:50:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2003341869 shutdown_connections 2024-05-27T07:50:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56207d4931c0 0x56207d9306a0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56207d4ff1b0 0x56207d8c7030 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56207d4de4d0 0x56207d8c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7c7400640 0 client.18756 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7f37a9740 1 --2- 172.21.15.63:0/2003341869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207d4d62f0 0x56207d8c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.561+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2003341869 >> 172.21.15.63:0/2003341869 conn(0x56207d4f3300 msgr2=0x56207d8a17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:52.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.562+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2003341869 shutdown_connections 2024-05-27T07:50:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.562+0000 7ff7f37a9740 1 -- 172.21.15.63:0/2003341869 wait complete. 2024-05-27T07:50:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.562+0000 7ff7f37a9740 20 client.18756 trim_cache size 0 max 16384 2024-05-27T07:50:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:50:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:50:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:50:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 -- 172.21.15.63:0/2782844826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7cc0b2950 msgr2=0x7ff7cc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7cc0b2950 0x7ff7cc0b4e60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x56207d89ebd0 tx=0x7ff7e0009600 comp rx=0 tx=0).stop 2024-05-27T07:50:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 -- 172.21.15.63:0/2782844826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7e4109ad0 msgr2=0x7ff7e4075500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7e4109ad0 0x7ff7e4075500 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff7d402a910 tx=0x7ff7d4003c20 comp rx=0 tx=0).stop 2024-05-27T07:50:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 -- 172.21.15.63:0/2782844826 shutdown_connections 2024-05-27T07:50:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7cc0b2950 0x7ff7cc0b4e60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7e4109f70 0x7ff7e4075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7e4109ad0 0x7ff7e4075500 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 --2- 172.21.15.63:0/2782844826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e4104220 0x7ff7e4074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 -- 172.21.15.63:0/2782844826 >> 172.21.15.63:0/2782844826 conn(0x7ff7e4079100 msgr2=0x7ff7e41022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 -- 172.21.15.63:0/2782844826 shutdown_connections 2024-05-27T07:50:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:50:52.563+0000 7ff7eb200640 1 -- 172.21.15.63:0/2782844826 wait complete. 2024-05-27T07:50:53.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:52 smithi086 ceph-mon[34687]: pgmap v544: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:50:53.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:52 smithi149 ceph-mon[35408]: pgmap v544: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:50:53.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:52 smithi063 ceph-mon[31613]: pgmap v544: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:50:54.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.212+0000 7f4e5f200700 1 -- 172.21.15.63:0/1070896124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e5810a7f0 msgr2=0x7f4e5810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.212+0000 7f4e5f200700 1 --2- 172.21.15.63:0/1070896124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e5810a7f0 0x7f4e5810ac00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009990 tx=0x7f4e48009ca0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.212+0000 7f4e5f200700 1 -- 172.21.15.63:0/1070896124 shutdown_connections 2024-05-27T07:50:54.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.212+0000 7f4e5f200700 1 --2- 172.21.15.63:0/1070896124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e5810b140 0x7f4e58117670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.212+0000 7f4e5f200700 1 --2- 172.21.15.63:0/1070896124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e581107d0 0x7f4e58110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.212+0000 7f4e5f200700 1 --2- 172.21.15.63:0/1070896124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e5810a7f0 0x7f4e5810ac00 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.212+0000 7f4e5f200700 1 -- 172.21.15.63:0/1070896124 >> 172.21.15.63:0/1070896124 conn(0x7f4e58076af0 msgr2=0x7f4e58076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.212+0000 7f4e5f200700 1 -- 172.21.15.63:0/1070896124 shutdown_connections 2024-05-27T07:50:54.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.212+0000 7f4e5f200700 1 -- 172.21.15.63:0/1070896124 wait complete. 2024-05-27T07:50:54.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.214+0000 7f4e5f200700 1 Processor -- start 2024-05-27T07:50:54.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.214+0000 7f4e5f200700 1 -- start start 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.214+0000 7f4e5f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e5810a7f0 0x7f4e581a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.214+0000 7f4e5f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5810b140 0x7f4e581a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.214+0000 7f4e5f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e581107d0 0x7f4e581a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.214+0000 7f4e5f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5811a7e0 con 0x7f4e5810a7f0 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.214+0000 7f4e5f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5811a660 con 0x7f4e581107d0 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.214+0000 7f4e5f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5811a960 con 0x7f4e5810b140 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.215+0000 7f4e5de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e5810a7f0 0x7f4e581a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.215+0000 7f4e5de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e5810a7f0 0x7f4e581a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41994/0 (socket says 172.21.15.63:41994) 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.215+0000 7f4e5de00700 1 -- 172.21.15.63:0/2601048165 learned_addr learned my addr 172.21.15.63:0/2601048165 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.215+0000 7f4e5d400700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5810b140 0x7f4e581a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.215+0000 7f4e5e800700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e581107d0 0x7f4e581a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.215+0000 7f4e5de00700 1 -- 172.21.15.63:0/2601048165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5810b140 msgr2=0x7f4e581a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.215+0000 7f4e5de00700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5810b140 0x7f4e581a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.215+0000 7f4e5de00700 1 -- 172.21.15.63:0/2601048165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e581107d0 msgr2=0x7f4e581a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.215+0000 7f4e5de00700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e581107d0 0x7f4e581a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.215+0000 7f4e5de00700 1 -- 172.21.15.63:0/2601048165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e581a60c0 con 0x7f4e5810a7f0 2024-05-27T07:50:54.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.216+0000 7f4e5de00700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e5810a7f0 0x7f4e581a11f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00d980 tx=0x7f4e4c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.216+0000 7f4e52a00700 1 -- 172.21.15.63:0/2601048165 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c017070 con 0x7f4e5810a7f0 2024-05-27T07:50:54.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.216+0000 7f4e52a00700 1 -- 172.21.15.63:0/2601048165 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e4c010460 con 0x7f4e5810a7f0 2024-05-27T07:50:54.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.216+0000 7f4e5f200700 1 -- 172.21.15.63:0/2601048165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e581a63b0 con 0x7f4e5810a7f0 2024-05-27T07:50:54.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.216+0000 7f4e52a00700 1 -- 172.21.15.63:0/2601048165 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00de60 con 0x7f4e5810a7f0 2024-05-27T07:50:54.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.216+0000 7f4e5f200700 1 -- 172.21.15.63:0/2601048165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581adcd0 con 0x7f4e5810a7f0 2024-05-27T07:50:54.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.221+0000 7f4e52a00700 1 -- 172.21.15.63:0/2601048165 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4e4c010610 con 0x7f4e5810a7f0 2024-05-27T07:50:54.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.222+0000 7f4e6f2fab80 1 Processor -- start 2024-05-27T07:50:54.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.222+0000 7f4e6f2fab80 1 -- start start 2024-05-27T07:50:54.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.223+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb2679f0 0x5619bb2fb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.223+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb17be40 0x5619bb2fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.223+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 0x5619bb3076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.223+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5619bb228e40 con 0x5619bb2679f0 2024-05-27T07:50:54.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.223+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5619bb2189e0 con 0x5619bb17be40 2024-05-27T07:50:54.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.223+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5619bb21ca20 con 0x5619bb3052c0 2024-05-27T07:50:54.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.223+0000 7f4e5e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb2679f0 0x5619bb2fb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.223+0000 7f4e5e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb2679f0 0x5619bb2fb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42004/0 (socket says 172.21.15.63:42004) 2024-05-27T07:50:54.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.223+0000 7f4e52a00700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e400b29f0 0x7f4e400b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.223+0000 7f4e5e800700 1 -- 172.21.15.63:0/309280888 learned_addr learned my addr 172.21.15.63:0/309280888 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:54.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e52a00700 1 -- 172.21.15.63:0/2601048165 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f4e4c0e2520 con 0x7f4e5810a7f0 2024-05-27T07:50:54.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e5d400700 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 0x5619bb3076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e5de00700 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb17be40 0x5619bb2fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e5e800700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e400b29f0 0x7f4e400b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e5e800700 1 -- 172.21.15.63:0/309280888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 msgr2=0x5619bb3076e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e5e800700 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 0x5619bb3076e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e5e800700 1 -- 172.21.15.63:0/309280888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb17be40 msgr2=0x5619bb2fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e5e800700 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb17be40 0x5619bb2fbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e5e800700 1 -- 172.21.15.63:0/309280888 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619bb1e1420 con 0x5619bb2679f0 2024-05-27T07:50:54.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e5d400700 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 0x5619bb3076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:54.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.224+0000 7f4e5de00700 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb17be40 0x5619bb2fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:54.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.225+0000 7f4e5e800700 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb2679f0 0x5619bb2fb750 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4e54007940 tx=0x7f4e54007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8469dd8c76b538 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.225+0000 7f4e5e800700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e400b29f0 0x7f4e400b4f00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400c000 tx=0x7f4e5401b000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.225+0000 7f4e47400700 1 -- 172.21.15.63:0/309280888 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54012050 con 0x5619bb2679f0 2024-05-27T07:50:54.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.225+0000 7f4e47400700 1 -- 172.21.15.63:0/309280888 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e5401d070 con 0x5619bb2679f0 2024-05-27T07:50:54.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.225+0000 7f4e47400700 1 -- 172.21.15.63:0/309280888 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54019e10 con 0x5619bb2679f0 2024-05-27T07:50:54.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.227+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/309280888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb2679f0 msgr2=0x5619bb2fb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.227+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb2679f0 0x5619bb2fb750 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4e54007940 tx=0x7f4e54007c50 comp rx=0 tx=0).stop 2024-05-27T07:50:54.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.227+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/309280888 shutdown_connections 2024-05-27T07:50:54.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.227+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 0x5619bb3076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.227+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb2679f0 0x5619bb2fb750 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.227+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/309280888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb17be40 0x5619bb2fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.227+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/309280888 >> 172.21.15.63:0/309280888 conn(0x5619bb215390 msgr2=0x5619bb2f6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.227+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/309280888 shutdown_connections 2024-05-27T07:50:54.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.227+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/309280888 wait complete. 2024-05-27T07:50:54.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.228+0000 7f4e6f2fab80 1 -- start start 2024-05-27T07:50:54.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.228+0000 7f4e6f2fab80 1 Processor -- start 2024-05-27T07:50:54.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.229+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb17be40 0x5619bb3022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.229+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 0x5619bb3027e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.229+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb3052c0 0x5619bb302d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.229+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5619bb11efa0 con 0x5619bb3052c0 2024-05-27T07:50:54.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.229+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5619bb1b9760 con 0x5619bb2679f0 2024-05-27T07:50:54.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.229+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5619bb307da0 con 0x5619bb17be40 2024-05-27T07:50:54.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.229+0000 7f4e6f2fab80 10 client.0 _ll_register_callbacks cb 0x5619bb1c2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:54.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.229+0000 7f4e5e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb3052c0 0x5619bb302d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.229+0000 7f4e5e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb3052c0 0x5619bb302d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42012/0 (socket says 172.21.15.63:42012) 2024-05-27T07:50:54.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.229+0000 7f4e5e800700 1 -- 172.21.15.63:0/1105514580 learned_addr learned my addr 172.21.15.63:0/1105514580 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:54.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.230+0000 7f4e5de00700 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb17be40 0x5619bb3022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.230+0000 7f4e5d400700 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 0x5619bb3027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.230+0000 7f4e5e800700 1 -- 172.21.15.63:0/1105514580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb17be40 msgr2=0x5619bb3022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.230+0000 7f4e5e800700 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb17be40 0x5619bb3022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.230+0000 7f4e5e800700 1 -- 172.21.15.63:0/1105514580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 msgr2=0x5619bb3027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.230+0000 7f4e5e800700 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 0x5619bb3027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.230+0000 7f4e5e800700 1 -- 172.21.15.63:0/1105514580 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619bb1688d0 con 0x5619bb3052c0 2024-05-27T07:50:54.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.231+0000 7f4e5e800700 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb3052c0 0x5619bb302d20 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4e54004d70 tx=0x7f4e54004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c498d8d10b3148b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.231+0000 7f4e3be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:54.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.231+0000 7f4e3be00700 1 -- 172.21.15.63:0/1105514580 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5402d030 con 0x5619bb3052c0 2024-05-27T07:50:54.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.231+0000 7f4e3be00700 1 -- 172.21.15.63:0/1105514580 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f4e54019640 con 0x5619bb3052c0 2024-05-27T07:50:54.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.231+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/1105514580 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e5401d070 con 0x5619bb3052c0 2024-05-27T07:50:54.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.231+0000 7f4e3be00700 1 -- 172.21.15.63:0/1105514580 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4e54037050 con 0x5619bb3052c0 2024-05-27T07:50:54.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.231+0000 7f4e6f2fab80 10 client.18768 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:54.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.231+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/1105514580 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5619bb1e9060 con 0x5619bb3052c0 2024-05-27T07:50:54.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.232+0000 7f4e3be00700 1 -- 172.21.15.63:0/1105514580 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e5401d070 con 0x5619bb3052c0 2024-05-27T07:50:54.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.232+0000 7f4e3be00700 1 -- 172.21.15.63:0/1105514580 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4e54018070 con 0x5619bb3052c0 2024-05-27T07:50:54.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.232+0000 7f4e6f2fab80 10 client.18768 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:54.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.232+0000 7f4e6f2fab80 10 client.18768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:54.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.232+0000 7f4e6f2fab80 20 client.18768 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:54.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.232+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5619bb314d10 0x5619bb317130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.232+0000 7f4e6f2fab80 4 client.18768 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:54.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.232+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/1105514580 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619bb22ad90 con 0x5619bb314d10 2024-05-27T07:50:54.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.233+0000 7f4e5de00700 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5619bb314d10 0x5619bb317130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.234+0000 7f4e3be00700 10 client.18768 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:54.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.234+0000 7f4e5de00700 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5619bb314d10 0x5619bb317130 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9dad570947e0608 server_cookie=6f6c4f04a1c062b in_seq=0 out_seq=0 2024-05-27T07:50:54.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.236+0000 7f4e3be00700 1 -- 172.21.15.63:0/1105514580 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5619bb22ad90 con 0x5619bb314d10 2024-05-27T07:50:54.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.236+0000 7f4e3be00700 10 client.18768 handle_command_reply: tid=0 2024-05-27T07:50:54.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.236+0000 7f4e6f2fab80 1 client.18768 shutdown 2024-05-27T07:50:54.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.236+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/1105514580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb3052c0 msgr2=0x5619bb302d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.236+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb3052c0 0x5619bb302d20 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4e54004d70 tx=0x7f4e54004da0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.237+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/1105514580 shutdown_connections 2024-05-27T07:50:54.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.237+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5619bb314d10 0x5619bb317130 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.237+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb3052c0 0x5619bb302d20 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.237+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb17be40 0x5619bb3022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.237+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/1105514580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 0x5619bb3027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.237+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/1105514580 >> 172.21.15.63:0/1105514580 conn(0x5619bb215390 msgr2=0x5619bb2f6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.237+0000 7f4e3be00700 0 client.18768 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:54.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.237+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/1105514580 shutdown_connections 2024-05-27T07:50:54.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.237+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/1105514580 wait complete. 2024-05-27T07:50:54.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.237+0000 7f4e6f2fab80 20 client.18768 trim_cache size 0 max 16384 2024-05-27T07:50:54.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.256+0000 7f4e6f2fab80 1 Processor -- start 2024-05-27T07:50:54.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.256+0000 7f4e6f2fab80 1 -- start start 2024-05-27T07:50:54.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.257+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb2f6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.257+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 0x5619bb2fab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.257+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 0x5619bb310e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.257+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b9300 con 0x5619bb17be40 2024-05-27T07:50:54.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.257+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b6b90 con 0x5619bb2679f0 2024-05-27T07:50:54.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.257+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b4a00 con 0x5619bb3052c0 2024-05-27T07:50:54.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.257+0000 7f4e5de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb2f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.257+0000 7f4e5de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb2f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42014/0 (socket says 172.21.15.63:42014) 2024-05-27T07:50:54.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.257+0000 7f4e5de00700 1 -- 172.21.15.63:0/312224529 learned_addr learned my addr 172.21.15.63:0/312224529 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:54.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.257+0000 7f4e5d400700 1 --2- 172.21.15.63:0/312224529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 0x5619bb2fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.258+0000 7f4e5e800700 1 --2- 172.21.15.63:0/312224529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 0x5619bb310e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.258+0000 7f4e5de00700 1 -- 172.21.15.63:0/312224529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 msgr2=0x5619bb310e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.258+0000 7f4e5de00700 1 --2- 172.21.15.63:0/312224529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 0x5619bb310e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.258+0000 7f4e5de00700 1 -- 172.21.15.63:0/312224529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 msgr2=0x5619bb2fab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.258+0000 7f4e5de00700 1 --2- 172.21.15.63:0/312224529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 0x5619bb2fab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.258+0000 7f4e5de00700 1 -- 172.21.15.63:0/312224529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619bb2fcd60 con 0x5619bb17be40 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.258+0000 7f4e5d400700 1 --2- 172.21.15.63:0/312224529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 0x5619bb2fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.258+0000 7f4e5de00700 1 --2- 172.21.15.63:0/312224529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb2f6740 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0a90e0 tx=0x7f4e4c0a8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b5a4d537dd5ed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.258+0000 7f4e46a00700 1 -- 172.21.15.63:0/312224529 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0c3070 con 0x5619bb17be40 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.259+0000 7f4e46a00700 1 -- 172.21.15.63:0/312224529 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e4c0b2240 con 0x5619bb17be40 2024-05-27T07:50:54.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.259+0000 7f4e46a00700 1 -- 172.21.15.63:0/312224529 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c020040 con 0x5619bb17be40 2024-05-27T07:50:54.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.259+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/312224529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 msgr2=0x5619bb2f6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.259+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/312224529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb2f6740 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0a90e0 tx=0x7f4e4c0a8de0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.260+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/312224529 shutdown_connections 2024-05-27T07:50:54.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.260+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/312224529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb3052c0 0x5619bb310e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.260+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/312224529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb2f6740 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.260+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/312224529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb2679f0 0x5619bb2fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.260+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/312224529 >> 172.21.15.63:0/312224529 conn(0x5619bb215390 msgr2=0x5619bb2fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.260+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/312224529 shutdown_connections 2024-05-27T07:50:54.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.260+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/312224529 wait complete. 2024-05-27T07:50:54.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.261+0000 7f4e6f2fab80 1 -- start start 2024-05-27T07:50:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.261+0000 7f4e6f2fab80 1 Processor -- start 2024-05-27T07:50:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb30d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb2679f0 0x5619bb30d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e6f2fab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb3052c0 0x5619bb308840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b5050 con 0x5619bb17be40 2024-05-27T07:50:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b64c0 con 0x5619bb3052c0 2024-05-27T07:50:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e6f2fab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b5720 con 0x5619bb2679f0 2024-05-27T07:50:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e6f2fab80 10 client.0 _ll_register_callbacks cb 0x5619bb1c2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:54.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e5de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb30d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e5de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb30d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42028/0 (socket says 172.21.15.63:42028) 2024-05-27T07:50:54.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e5de00700 1 -- 172.21.15.63:0/544951874 learned_addr learned my addr 172.21.15.63:0/544951874 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:54.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.262+0000 7f4e5d400700 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb2679f0 0x5619bb30d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.263+0000 7f4e5e800700 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb3052c0 0x5619bb308840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.263+0000 7f4e5de00700 1 -- 172.21.15.63:0/544951874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb2679f0 msgr2=0x5619bb30d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.263+0000 7f4e5de00700 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb2679f0 0x5619bb30d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.263+0000 7f4e5de00700 1 -- 172.21.15.63:0/544951874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb3052c0 msgr2=0x5619bb308840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.263+0000 7f4e5de00700 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb3052c0 0x5619bb308840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.263+0000 7f4e5de00700 1 -- 172.21.15.63:0/544951874 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619bb303090 con 0x5619bb17be40 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.263+0000 7f4e5de00700 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb30d380 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c010c80 tx=0x7f4e4c0a9b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a505874f87f3687 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.263+0000 7f4e3be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.264+0000 7f4e3be00700 1 -- 172.21.15.63:0/544951874 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c020070 con 0x5619bb17be40 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.264+0000 7f4e3be00700 1 -- 172.21.15.63:0/544951874 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f4e4c0a9840 con 0x5619bb17be40 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.264+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/544951874 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e4c0b2240 con 0x5619bb17be40 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.264+0000 7f4e3be00700 1 -- 172.21.15.63:0/544951874 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4e4c0c3040 con 0x5619bb17be40 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.264+0000 7f4e6f2fab80 10 client.18774 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:54.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.264+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/544951874 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5619bb37ef00 con 0x5619bb17be40 2024-05-27T07:50:54.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.264+0000 7f4e3be00700 1 -- 172.21.15.63:0/544951874 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e4c0b2240 con 0x5619bb17be40 2024-05-27T07:50:54.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.265+0000 7f4e3be00700 1 -- 172.21.15.63:0/544951874 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4e4c0b0b10 con 0x5619bb17be40 2024-05-27T07:50:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.265+0000 7f4e6f2fab80 10 client.18774 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.265+0000 7f4e6f2fab80 10 client.18774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.265+0000 7f4e6f2fab80 20 client.18774 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.265+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5619bb37f520 0x5619bb381940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.265+0000 7f4e6f2fab80 4 client.18774 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:50:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.265+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/544951874 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619bb224570 con 0x5619bb37f520 2024-05-27T07:50:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.265+0000 7f4e5d400700 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5619bb37f520 0x5619bb381940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.267+0000 7f4e5d400700 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5619bb37f520 0x5619bb381940 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=904af9319332bc5c server_cookie=b0ad17c0a533218f in_seq=0 out_seq=0 2024-05-27T07:50:54.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.267+0000 7f4e3be00700 10 client.18774 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.267+0000 7f4e3be00700 1 -- 172.21.15.63:0/544951874 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619bb224570 con 0x5619bb37f520 2024-05-27T07:50:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.267+0000 7f4e3be00700 10 client.18774 handle_command_reply: tid=0 2024-05-27T07:50:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.267+0000 7f4e6f2fab80 1 client.18774 shutdown 2024-05-27T07:50:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.267+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/544951874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 msgr2=0x5619bb30d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.267+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb30d380 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c010c80 tx=0x7f4e4c0a9b90 comp rx=0 tx=0).stop 2024-05-27T07:50:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.267+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/544951874 shutdown_connections 2024-05-27T07:50:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.268+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5619bb37f520 0x5619bb381940 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.268+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5619bb3052c0 0x5619bb308840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.268+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619bb17be40 0x5619bb30d380 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.268+0000 7f4e3be00700 0 client.18774 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.268+0000 7f4e6f2fab80 1 --2- 172.21.15.63:0/544951874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5619bb2679f0 0x5619bb30d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.268+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/544951874 >> 172.21.15.63:0/544951874 conn(0x5619bb215390 msgr2=0x5619bb302080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.268+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/544951874 shutdown_connections 2024-05-27T07:50:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.268+0000 7f4e6f2fab80 1 -- 172.21.15.63:0/544951874 wait complete. 2024-05-27T07:50:54.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.268+0000 7f4e6f2fab80 20 client.18774 trim_cache size 0 max 16384 2024-05-27T07:50:54.270 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:50:54.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 -- 172.21.15.63:0/2601048165 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e400b29f0 msgr2=0x7f4e400b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e400b29f0 0x7f4e400b4f00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400c000 tx=0x7f4e5401b000 comp rx=0 tx=0).stop 2024-05-27T07:50:54.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 -- 172.21.15.63:0/2601048165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e5810a7f0 msgr2=0x7f4e581a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e5810a7f0 0x7f4e581a11f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00d980 tx=0x7f4e4c00dd40 comp rx=0 tx=0).stop 2024-05-27T07:50:54.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 -- 172.21.15.63:0/2601048165 shutdown_connections 2024-05-27T07:50:54.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e400b29f0 0x7f4e400b4f00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e581107d0 0x7f4e581a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e5810a7f0 0x7f4e581a11f0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 --2- 172.21.15.63:0/2601048165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5810b140 0x7f4e581a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 -- 172.21.15.63:0/2601048165 >> 172.21.15.63:0/2601048165 conn(0x7f4e58076af0 msgr2=0x7f4e581089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 -- 172.21.15.63:0/2601048165 shutdown_connections 2024-05-27T07:50:54.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:54.269+0000 7f4e5f200700 1 -- 172.21.15.63:0/2601048165 wait complete. 2024-05-27T07:50:54.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:54 smithi063 ceph-mon[31613]: pgmap v545: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:50:55.208 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:54 smithi086 ceph-mon[34687]: pgmap v545: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:50:55.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:54 smithi149 ceph-mon[35408]: pgmap v545: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:50:56.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:50:56.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:50:56.785 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:57.174 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:56 smithi063 ceph-mon[31613]: pgmap v546: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:50:57.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:56 smithi086 ceph-mon[34687]: pgmap v546: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:50:57.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:56 smithi149 ceph-mon[35408]: pgmap v546: 129 pgs: 129 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:50:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:58 smithi086 ceph-mon[34687]: pgmap v547: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:50:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:58 smithi149 ceph-mon[35408]: pgmap v547: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:50:59.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:58 smithi063 ceph-mon[31613]: pgmap v547: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T07:50:59.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.229+0000 7f185ca00700 1 -- 172.21.15.63:0/2324454013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18581045e0 msgr2=0x7f18581049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.229+0000 7f185ca00700 1 --2- 172.21.15.63:0/2324454013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18581045e0 0x7f18581049f0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f18480099c0 tx=0x7f1848009cd0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.229+0000 7f185ca00700 1 -- 172.21.15.63:0/2324454013 shutdown_connections 2024-05-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.229+0000 7f185ca00700 1 --2- 172.21.15.63:0/2324454013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858109860 0x7f18581111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.229+0000 7f185ca00700 1 --2- 172.21.15.63:0/2324454013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18581045e0 0x7f18581049f0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.229+0000 7f185ca00700 1 --2- 172.21.15.63:0/2324454013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1858108f10 0x7f1858109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.229+0000 7f185ca00700 1 -- 172.21.15.63:0/2324454013 >> 172.21.15.63:0/2324454013 conn(0x7f1858100030 msgr2=0x7f1858102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.230+0000 7f185ca00700 1 -- 172.21.15.63:0/2324454013 shutdown_connections 2024-05-27T07:50:59.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.230+0000 7f185ca00700 1 -- 172.21.15.63:0/2324454013 wait complete. 2024-05-27T07:50:59.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.231+0000 7f185ca00700 1 Processor -- start 2024-05-27T07:50:59.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.231+0000 7f185ca00700 1 -- start start 2024-05-27T07:50:59.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.232+0000 7f185ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18581045e0 0x7f185819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.232+0000 7f185ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1858108f10 0x7f185819d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.232+0000 7f185ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858109860 0x7f18581a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.232+0000 7f185ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1858114390 con 0x7f1858108f10 2024-05-27T07:50:59.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.232+0000 7f185ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1858114210 con 0x7f18581045e0 2024-05-27T07:50:59.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.232+0000 7f185ca00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1858114510 con 0x7f1858109860 2024-05-27T07:50:59.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.232+0000 7f1855400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1858108f10 0x7f185819d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.232+0000 7f1855400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1858108f10 0x7f185819d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43658/0 (socket says 172.21.15.63:43658) 2024-05-27T07:50:59.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.232+0000 7f1855400700 1 -- 172.21.15.63:0/2332861143 learned_addr learned my addr 172.21.15.63:0/2332861143 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:59.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.233+0000 7f1855e00700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18581045e0 0x7f185819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.233+0000 7f1856800700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858109860 0x7f18581a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.233+0000 7f1855400700 1 -- 172.21.15.63:0/2332861143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858109860 msgr2=0x7f18581a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.233+0000 7f1855400700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858109860 0x7f18581a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.233+0000 7f1855400700 1 -- 172.21.15.63:0/2332861143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18581045e0 msgr2=0x7f185819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.233+0000 7f1855400700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18581045e0 0x7f185819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.233+0000 7f1855400700 1 -- 172.21.15.63:0/2332861143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18581a1e70 con 0x7f1858108f10 2024-05-27T07:50:59.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.233+0000 7f1855400700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1858108f10 0x7f185819d540 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f184400d980 tx=0x7f184400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.234+0000 7f184ea00700 1 -- 172.21.15.63:0/2332861143 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844017070 con 0x7f1858108f10 2024-05-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.234+0000 7f184ea00700 1 -- 172.21.15.63:0/2332861143 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1844009dc0 con 0x7f1858108f10 2024-05-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.234+0000 7f184ea00700 1 -- 172.21.15.63:0/2332861143 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184400f3c0 con 0x7f1858108f10 2024-05-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.234+0000 7f185ca00700 1 -- 172.21.15.63:0/2332861143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18581a2160 con 0x7f1858108f10 2024-05-27T07:50:59.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.234+0000 7f185ca00700 1 -- 172.21.15.63:0/2332861143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18581a9a80 con 0x7f1858108f10 2024-05-27T07:50:59.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.239+0000 7f186b48ab80 1 Processor -- start 2024-05-27T07:50:59.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.240+0000 7f186b48ab80 1 -- start start 2024-05-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.240+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699ed65ec0 0x55699eeea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.240+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 0x55699eee48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.240+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eee4e30 0x55699eee5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.240+0000 7f186b48ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55699ee12ec0 con 0x55699eee4e30 2024-05-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.240+0000 7f186b48ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55699ee02a60 con 0x55699eeeadb0 2024-05-27T07:50:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.240+0000 7f186b48ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55699ee06aa0 con 0x55699ed65ec0 2024-05-27T07:50:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.241+0000 7f184ea00700 1 -- 172.21.15.63:0/2332861143 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1844009930 con 0x7f1858108f10 2024-05-27T07:50:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.241+0000 7f1855e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 0x55699eee48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.241+0000 7f1856800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699ed65ec0 0x55699eeea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.241+0000 7f1855e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 0x55699eee48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34142/0 (socket says 172.21.15.63:34142) 2024-05-27T07:50:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.241+0000 7f1855e00700 1 -- 172.21.15.63:0/642415350 learned_addr learned my addr 172.21.15.63:0/642415350 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.241+0000 7f1855400700 1 --2- 172.21.15.63:0/642415350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eee4e30 0x55699eee5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.242+0000 7f1855e00700 1 -- 172.21.15.63:0/642415350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699ed65ec0 msgr2=0x55699eeea870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.242+0000 7f1855e00700 1 --2- 172.21.15.63:0/642415350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699ed65ec0 0x55699eeea870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.242+0000 7f1855e00700 1 -- 172.21.15.63:0/642415350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eee4e30 msgr2=0x55699eee5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:59.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.242+0000 7f1855e00700 1 --2- 172.21.15.63:0/642415350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eee4e30 0x55699eee5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.242+0000 7f1855e00700 1 -- 172.21.15.63:0/642415350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699edcb4a0 con 0x55699eeeadb0 2024-05-27T07:50:59.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.242+0000 7f1855e00700 1 --2- 172.21.15.63:0/642415350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 0x55699eee48f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18480058e0 tx=0x7f1848005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f17175f4da50ed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.243+0000 7f1843400700 1 -- 172.21.15.63:0/642415350 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848027070 con 0x55699eeeadb0 2024-05-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.243+0000 7f1843400700 1 -- 172.21.15.63:0/642415350 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f184800bc20 con 0x55699eeeadb0 2024-05-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.243+0000 7f1843400700 1 -- 172.21.15.63:0/642415350 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480038a0 con 0x55699eeeadb0 2024-05-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.243+0000 7f184ea00700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f183c0b2a70 0x7f183c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.243+0000 7f184ea00700 1 -- 172.21.15.63:0/2332861143 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f18440e24f0 con 0x7f1858108f10 2024-05-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.244+0000 7f1856800700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f183c0b2a70 0x7f183c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.244+0000 7f186b48ab80 1 -- 172.21.15.63:0/642415350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 msgr2=0x55699eee48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.244+0000 7f186b48ab80 1 --2- 172.21.15.63:0/642415350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 0x55699eee48f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18480058e0 tx=0x7f1848005ea0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.244+0000 7f186b48ab80 1 -- 172.21.15.63:0/642415350 shutdown_connections 2024-05-27T07:50:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.244+0000 7f186b48ab80 1 --2- 172.21.15.63:0/642415350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eee4e30 0x55699eee5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.244+0000 7f186b48ab80 1 --2- 172.21.15.63:0/642415350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699ed65ec0 0x55699eeea870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.244+0000 7f186b48ab80 1 --2- 172.21.15.63:0/642415350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 0x55699eee48f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.244+0000 7f186b48ab80 1 -- 172.21.15.63:0/642415350 >> 172.21.15.63:0/642415350 conn(0x55699edc6040 msgr2=0x55699eedf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.245+0000 7f1856800700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f183c0b2a70 0x7f183c0b4f80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55699eee5bb0 tx=0x7f1850009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.245+0000 7f186b48ab80 1 -- 172.21.15.63:0/642415350 shutdown_connections 2024-05-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.245+0000 7f186b48ab80 1 -- 172.21.15.63:0/642415350 wait complete. 2024-05-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.245+0000 7f186b48ab80 1 -- start start 2024-05-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.246+0000 7f186b48ab80 1 Processor -- start 2024-05-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699ed65ec0 0x55699eeee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eee4e30 0x55699eeeeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eeef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f186b48ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55699ed09020 con 0x55699eeeadb0 2024-05-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f186b48ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55699eda37e0 con 0x55699eee4e30 2024-05-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f186b48ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55699eef1820 con 0x55699ed65ec0 2024-05-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f186b48ab80 10 client.0 _ll_register_callbacks cb 0x55699edaccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f1856800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eeef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f1856800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eeef050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43668/0 (socket says 172.21.15.63:43668) 2024-05-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f1856800700 1 -- 172.21.15.63:0/18413382 learned_addr learned my addr 172.21.15.63:0/18413382 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f1855400700 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eee4e30 0x55699eeeeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.247+0000 7f1855e00700 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699ed65ec0 0x55699eeee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.248+0000 7f1856800700 1 -- 172.21.15.63:0/18413382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699ed65ec0 msgr2=0x55699eeee5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.248+0000 7f1856800700 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699ed65ec0 0x55699eeee5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.248+0000 7f1856800700 1 -- 172.21.15.63:0/18413382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eee4e30 msgr2=0x55699eeeeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.248+0000 7f1856800700 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eee4e30 0x55699eeeeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.248+0000 7f1856800700 1 -- 172.21.15.63:0/18413382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699ed52950 con 0x55699eeeadb0 2024-05-27T07:50:59.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.249+0000 7f1856800700 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eeef050 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1850018e70 tx=0x7f1850004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b966c3ff2e16f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:59.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.249+0000 7f1837e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:59.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.249+0000 7f1837e00700 1 -- 172.21.15.63:0/18413382 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185001e050 con 0x55699eeeadb0 2024-05-27T07:50:59.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.249+0000 7f1837e00700 1 -- 172.21.15.63:0/18413382 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f1850004bb0 con 0x55699eeeadb0 2024-05-27T07:50:59.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.249+0000 7f186b48ab80 1 -- 172.21.15.63:0/18413382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f184800bc20 con 0x55699eeeadb0 2024-05-27T07:50:59.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.249+0000 7f1837e00700 1 -- 172.21.15.63:0/18413382 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1850030050 con 0x55699eeeadb0 2024-05-27T07:50:59.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.249+0000 7f186b48ab80 10 client.18786 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:59.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.249+0000 7f186b48ab80 1 -- 172.21.15.63:0/18413382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55699edd30e0 con 0x55699eeeadb0 2024-05-27T07:50:59.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.250+0000 7f1837e00700 1 -- 172.21.15.63:0/18413382 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f184800bc20 con 0x55699eeeadb0 2024-05-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.250+0000 7f1837e00700 1 -- 172.21.15.63:0/18413382 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1850023700 con 0x55699eeeadb0 2024-05-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.250+0000 7f186b48ab80 10 client.18786 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.250+0000 7f186b48ab80 10 client.18786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.250+0000 7f186b48ab80 20 client.18786 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.250+0000 7f186b48ab80 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55699ef07000 0x55699ef094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.250+0000 7f186b48ab80 4 client.18786 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.250+0000 7f186b48ab80 1 -- 172.21.15.63:0/18413382 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55699ee14e10 con 0x55699ef07000 2024-05-27T07:50:59.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.251+0000 7f1855e00700 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55699ef07000 0x55699ef094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.252+0000 7f1855e00700 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55699ef07000 0x55699ef094b0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b71c2571554eb053 server_cookie=1a80370f453d35e in_seq=0 out_seq=0 2024-05-27T07:50:59.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.252+0000 7f1837e00700 10 client.18786 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.255+0000 7f1837e00700 1 -- 172.21.15.63:0/18413382 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55699ee14e10 con 0x55699ef07000 2024-05-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.255+0000 7f1837e00700 10 client.18786 handle_command_reply: tid=0 2024-05-27T07:50:59.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.255+0000 7f186b48ab80 1 client.18786 shutdown 2024-05-27T07:50:59.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 1 -- 172.21.15.63:0/18413382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 msgr2=0x55699eeef050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:59.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eeef050 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1850018e70 tx=0x7f1850004ad0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 1 -- 172.21.15.63:0/18413382 shutdown_connections 2024-05-27T07:50:59.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55699ef07000 0x55699ef094b0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eeef050 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699ed65ec0 0x55699eeee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 1 --2- 172.21.15.63:0/18413382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eee4e30 0x55699eeeeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 1 -- 172.21.15.63:0/18413382 >> 172.21.15.63:0/18413382 conn(0x55699edc6040 msgr2=0x55699eedf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f1837e00700 0 client.18786 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 1 -- 172.21.15.63:0/18413382 shutdown_connections 2024-05-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 1 -- 172.21.15.63:0/18413382 wait complete. 2024-05-27T07:50:59.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.256+0000 7f186b48ab80 20 client.18786 trim_cache size 0 max 16384 2024-05-27T07:50:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.282+0000 7f186b48ab80 1 Processor -- start 2024-05-27T07:50:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.282+0000 7f186b48ab80 1 -- start start 2024-05-27T07:50:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.283+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699ed65ec0 0x55699eee48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.283+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 0x55699eee8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.283+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eef0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.283+0000 7f186b48ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18581b0cb0 con 0x55699eeeadb0 2024-05-27T07:50:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.283+0000 7f186b48ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18581ae540 con 0x55699ed65ec0 2024-05-27T07:50:59.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.283+0000 7f186b48ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18581ac3d0 con 0x55699eee4e30 2024-05-27T07:50:59.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.283+0000 7f1856800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eef0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.283+0000 7f1856800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eef0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43674/0 (socket says 172.21.15.63:43674) 2024-05-27T07:50:59.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.283+0000 7f1856800700 1 -- 172.21.15.63:0/4043189884 learned_addr learned my addr 172.21.15.63:0/4043189884 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:59.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.284+0000 7f1855400700 1 --2- 172.21.15.63:0/4043189884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 0x55699eee8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.284+0000 7f1855e00700 1 --2- 172.21.15.63:0/4043189884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699ed65ec0 0x55699eee48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.284+0000 7f1856800700 1 -- 172.21.15.63:0/4043189884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 msgr2=0x55699eee8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:59.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.284+0000 7f1856800700 1 --2- 172.21.15.63:0/4043189884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 0x55699eee8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.284+0000 7f1856800700 1 -- 172.21.15.63:0/4043189884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699ed65ec0 msgr2=0x55699eee48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:59.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.284+0000 7f1856800700 1 --2- 172.21.15.63:0/4043189884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699ed65ec0 0x55699eee48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.284+0000 7f1856800700 1 -- 172.21.15.63:0/4043189884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699eeef3c0 con 0x55699eeeadb0 2024-05-27T07:50:59.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.284+0000 7f1855400700 1 --2- 172.21.15.63:0/4043189884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 0x55699eee8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:59.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.284+0000 7f1856800700 1 --2- 172.21.15.63:0/4043189884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eef0a70 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f1850009390 tx=0x7f1850005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbdea0047a8a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:59.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.285+0000 7f1842a00700 1 -- 172.21.15.63:0/4043189884 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185001e040 con 0x55699eeeadb0 2024-05-27T07:50:59.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.285+0000 7f1842a00700 1 -- 172.21.15.63:0/4043189884 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f185001f040 con 0x55699eeeadb0 2024-05-27T07:50:59.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.285+0000 7f1842a00700 1 -- 172.21.15.63:0/4043189884 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850016e20 con 0x55699eeeadb0 2024-05-27T07:50:59.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.286+0000 7f186b48ab80 1 -- 172.21.15.63:0/4043189884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 msgr2=0x55699eef0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:59.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.286+0000 7f186b48ab80 1 --2- 172.21.15.63:0/4043189884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eef0a70 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f1850009390 tx=0x7f1850005c60 comp rx=0 tx=0).stop 2024-05-27T07:50:59.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.286+0000 7f186b48ab80 1 -- 172.21.15.63:0/4043189884 shutdown_connections 2024-05-27T07:50:59.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.286+0000 7f186b48ab80 1 --2- 172.21.15.63:0/4043189884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699eeeadb0 0x55699eef0a70 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.286+0000 7f186b48ab80 1 --2- 172.21.15.63:0/4043189884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699ed65ec0 0x55699eee48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.286+0000 7f186b48ab80 1 --2- 172.21.15.63:0/4043189884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 0x55699eee8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.287+0000 7f186b48ab80 1 -- 172.21.15.63:0/4043189884 >> 172.21.15.63:0/4043189884 conn(0x55699edc6040 msgr2=0x55699eeed550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:59.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.287+0000 7f186b48ab80 1 -- 172.21.15.63:0/4043189884 shutdown_connections 2024-05-27T07:50:59.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.287+0000 7f186b48ab80 1 -- 172.21.15.63:0/4043189884 wait complete. 2024-05-27T07:50:59.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.288+0000 7f186b48ab80 1 -- start start 2024-05-27T07:50:59.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.288+0000 7f186b48ab80 1 Processor -- start 2024-05-27T07:50:59.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699ed65ec0 0x55699ef0e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 0x55699ef0ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f186b48ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 0x55699ef0f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f186b48ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18581aca20 con 0x55699ed65ec0 2024-05-27T07:50:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f186b48ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18581ade70 con 0x55699eeeadb0 2024-05-27T07:50:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f186b48ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18581ad0d0 con 0x55699eee4e30 2024-05-27T07:50:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f186b48ab80 10 client.0 _ll_register_callbacks cb 0x55699edaccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f1855e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699ed65ec0 0x55699ef0e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f1855e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699ed65ec0 0x55699ef0e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43678/0 (socket says 172.21.15.63:43678) 2024-05-27T07:50:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f1855e00700 1 -- 172.21.15.63:0/2041562122 learned_addr learned my addr 172.21.15.63:0/2041562122 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:50:59.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.289+0000 7f1855400700 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 0x55699ef0ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.290+0000 7f1855e00700 1 -- 172.21.15.63:0/2041562122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 msgr2=0x55699ef0ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:59.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.290+0000 7f1855e00700 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 0x55699ef0ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.290+0000 7f1855e00700 1 -- 172.21.15.63:0/2041562122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 msgr2=0x55699ef0f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:59.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.290+0000 7f1855e00700 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 0x55699ef0f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.290+0000 7f1855e00700 1 -- 172.21.15.63:0/2041562122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699eefff30 con 0x55699ed65ec0 2024-05-27T07:50:59.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.290+0000 7f1855e00700 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699ed65ec0 0x55699ef0e690 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f184800b580 tx=0x7f1848009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bc49449fedf592 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.290+0000 7f1837e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:50:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.291+0000 7f1837e00700 1 -- 172.21.15.63:0/2041562122 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848035070 con 0x55699ed65ec0 2024-05-27T07:50:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.291+0000 7f1837e00700 1 -- 172.21.15.63:0/2041562122 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f184800b990 con 0x55699ed65ec0 2024-05-27T07:50:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.291+0000 7f186b48ab80 1 -- 172.21.15.63:0/2041562122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185001f040 con 0x55699ed65ec0 2024-05-27T07:50:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.291+0000 7f1837e00700 1 -- 172.21.15.63:0/2041562122 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1848041050 con 0x55699ed65ec0 2024-05-27T07:50:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.291+0000 7f186b48ab80 10 client.18792 fetch_fsmap learned FSMap version 33 2024-05-27T07:50:59.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.291+0000 7f186b48ab80 1 -- 172.21.15.63:0/2041562122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55699ef71c10 con 0x55699ed65ec0 2024-05-27T07:50:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.292+0000 7f1837e00700 1 -- 172.21.15.63:0/2041562122 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f185001f040 con 0x55699ed65ec0 2024-05-27T07:50:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.292+0000 7f1837e00700 1 -- 172.21.15.63:0/2041562122 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1848022480 con 0x55699ed65ec0 2024-05-27T07:50:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.292+0000 7f186b48ab80 10 client.18792 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:50:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.292+0000 7f186b48ab80 10 client.18792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.292+0000 7f186b48ab80 20 client.18792 populate_metadata read hostname 'smithi063' 2024-05-27T07:50:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.292+0000 7f186b48ab80 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55699ef71de0 0x55699ef74200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.292+0000 7f186b48ab80 4 client.18792 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:50:59.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.292+0000 7f186b48ab80 1 -- 172.21.15.63:0/2041562122 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55699ee0e5f0 con 0x55699ef71de0 2024-05-27T07:50:59.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.293+0000 7f1855400700 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55699ef71de0 0x55699ef74200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:59.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.295+0000 7f1855400700 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55699ef71de0 0x55699ef74200 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa5199ea1d83443 server_cookie=60c239cc5bb9c476 in_seq=0 out_seq=0 2024-05-27T07:50:59.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.295+0000 7f1837e00700 10 client.18792 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:59.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.298+0000 7f1837e00700 1 -- 172.21.15.63:0/2041562122 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55699ee0e5f0 con 0x55699ef71de0 2024-05-27T07:50:59.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.298+0000 7f1837e00700 10 client.18792 handle_command_reply: tid=0 2024-05-27T07:50:59.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.298+0000 7f186b48ab80 1 client.18792 shutdown 2024-05-27T07:50:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f186b48ab80 1 -- 172.21.15.63:0/2041562122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699ed65ec0 msgr2=0x55699ef0e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f186b48ab80 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699ed65ec0 0x55699ef0e690 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f184800b580 tx=0x7f1848009310 comp rx=0 tx=0).stop 2024-05-27T07:50:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f186b48ab80 1 -- 172.21.15.63:0/2041562122 shutdown_connections 2024-05-27T07:50:59.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f186b48ab80 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55699ef71de0 0x55699ef74200 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f186b48ab80 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55699eeeadb0 0x55699ef0f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f186b48ab80 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55699ed65ec0 0x55699ef0e690 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f186b48ab80 1 --2- 172.21.15.63:0/2041562122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55699eee4e30 0x55699ef0ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f186b48ab80 1 -- 172.21.15.63:0/2041562122 >> 172.21.15.63:0/2041562122 conn(0x55699edc6040 msgr2=0x55699eeee3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f1837e00700 0 client.18792 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:50:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f186b48ab80 1 -- 172.21.15.63:0/2041562122 shutdown_connections 2024-05-27T07:50:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.299+0000 7f186b48ab80 1 -- 172.21.15.63:0/2041562122 wait complete. 2024-05-27T07:50:59.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.300+0000 7f186b48ab80 20 client.18792 trim_cache size 0 max 16384 2024-05-27T07:50:59.301 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:50:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:50:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "60916075-a1dc-493b-975d-2be43a63a747", 2024-05-27T07:50:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:50:59.301 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:50:59.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 -- 172.21.15.63:0/2332861143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f183c0b2a70 msgr2=0x7f183c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f183c0b2a70 0x7f183c0b4f80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55699eee5bb0 tx=0x7f1850009040 comp rx=0 tx=0).stop 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 -- 172.21.15.63:0/2332861143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1858108f10 msgr2=0x7f185819d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1858108f10 0x7f185819d540 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f184400d980 tx=0x7f184400dc90 comp rx=0 tx=0).stop 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 -- 172.21.15.63:0/2332861143 shutdown_connections 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f183c0b2a70 0x7f183c0b4f80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858109860 0x7f18581a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18581045e0 0x7f185819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 --2- 172.21.15.63:0/2332861143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1858108f10 0x7f185819d540 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 -- 172.21.15.63:0/2332861143 >> 172.21.15.63:0/2332861143 conn(0x7f1858100030 msgr2=0x7f185810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 -- 172.21.15.63:0/2332861143 shutdown_connections 2024-05-27T07:50:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:50:59.301+0000 7f185ca00700 1 -- 172.21.15.63:0/2332861143 wait complete. 2024-05-27T07:50:59.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:59 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:50:59.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:59.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:50:59.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:50:59.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:59.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:50:59.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:59.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:59 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:50:59.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:50:59.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:50:59 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:59 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:51:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:51:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:51:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:51:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:59 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:51:00.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:50:59 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:59 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:51:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:51:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:51:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:51:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:59 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:51:00.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:50:59 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:00.336 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:51:00.854 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:01.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:00 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:51:01.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:00 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:01.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:00 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:51:01.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:00 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:01.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:00 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:51:01.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:00 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:01.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:00 smithi086 ceph-mon[34687]: pgmap v548: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:51:01.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:00 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:51:01.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:00 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:01.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:00 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:51:01.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:00 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:01.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:00 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:51:01.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:00 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:01.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:00 smithi149 ceph-mon[35408]: pgmap v548: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:51:01.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:00 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:51:01.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:00 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:01.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:00 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:51:01.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:00 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:01.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:00 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:51:01.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:00 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:01.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:00 smithi063 ceph-mon[31613]: pgmap v548: 129 pgs: 129 active+clean; 4.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T07:51:03.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:02 smithi086 ceph-mon[34687]: pgmap v549: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.7 MiB/s wr, 60 op/s 2024-05-27T07:51:03.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:02 smithi149 ceph-mon[35408]: pgmap v549: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.7 MiB/s wr, 60 op/s 2024-05-27T07:51:03.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:02 smithi063 ceph-mon[31613]: pgmap v549: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.7 MiB/s wr, 60 op/s 2024-05-27T07:51:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.297+0000 7f6825000700 1 -- 172.21.15.63:0/414262354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105280 msgr2=0x7f6820105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.297+0000 7f6825000700 1 --2- 172.21.15.63:0/414262354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105280 0x7f6820105690 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6810009990 tx=0x7f6810009ca0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.297+0000 7f6825000700 1 -- 172.21.15.63:0/414262354 shutdown_connections 2024-05-27T07:51:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.297+0000 7f6825000700 1 --2- 172.21.15.63:0/414262354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f682010e6f0 0x7f6820110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.297+0000 7f6825000700 1 --2- 172.21.15.63:0/414262354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105280 0x7f6820105690 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.297+0000 7f6825000700 1 --2- 172.21.15.63:0/414262354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6820105bd0 0x7f682010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.297+0000 7f6825000700 1 -- 172.21.15.63:0/414262354 >> 172.21.15.63:0/414262354 conn(0x7f6820078ab0 msgr2=0x7f6820078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.297+0000 7f6825000700 1 -- 172.21.15.63:0/414262354 shutdown_connections 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.297+0000 7f6825000700 1 -- 172.21.15.63:0/414262354 wait complete. 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.298+0000 7f6825000700 1 Processor -- start 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.299+0000 7f6825000700 1 -- start start 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.299+0000 7f6825000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820105280 0x7f682019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.299+0000 7f6825000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105bd0 0x7f682019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.299+0000 7f6825000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f682010e6f0 0x7f68201a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.299+0000 7f6825000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6820114190 con 0x7f6820105bd0 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.299+0000 7f6825000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6820114010 con 0x7f6820105280 2024-05-27T07:51:04.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.299+0000 7f6825000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6820114310 con 0x7f682010e6f0 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.299+0000 7f681ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105bd0 0x7f682019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.299+0000 7f681ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105bd0 0x7f682019d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43710/0 (socket says 172.21.15.63:43710) 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.300+0000 7f681ea00700 1 -- 172.21.15.63:0/1384211531 learned_addr learned my addr 172.21.15.63:0/1384211531 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.300+0000 7f681f400700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820105280 0x7f682019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.300+0000 7f681fe00700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f682010e6f0 0x7f68201a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.300+0000 7f681ea00700 1 -- 172.21.15.63:0/1384211531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f682010e6f0 msgr2=0x7f68201a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.300+0000 7f681ea00700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f682010e6f0 0x7f68201a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.300+0000 7f681ea00700 1 -- 172.21.15.63:0/1384211531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820105280 msgr2=0x7f682019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.300+0000 7f681ea00700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820105280 0x7f682019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.300+0000 7f681ea00700 1 -- 172.21.15.63:0/1384211531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68201a1c50 con 0x7f6820105bd0 2024-05-27T07:51:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.300+0000 7f681f400700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820105280 0x7f682019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.300+0000 7f681ea00700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105bd0 0x7f682019d320 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f680c00d950 tx=0x7f680c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.301+0000 7f6817e00700 1 -- 172.21.15.63:0/1384211531 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c016070 con 0x7f6820105bd0 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.301+0000 7f6817e00700 1 -- 172.21.15.63:0/1384211531 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f680c010460 con 0x7f6820105bd0 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.301+0000 7f6825000700 1 -- 172.21.15.63:0/1384211531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68201a1f40 con 0x7f6820105bd0 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.301+0000 7f6817e00700 1 -- 172.21.15.63:0/1384211531 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c00f620 con 0x7f6820105bd0 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.301+0000 7f6825000700 1 -- 172.21.15.63:0/1384211531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68201a9860 con 0x7f6820105bd0 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.307+0000 7f6835084b80 1 Processor -- start 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.307+0000 7f6835084b80 1 -- start start 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.307+0000 7f6835084b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a91ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.307+0000 7f6835084b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 0x55ed9a928ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.307+0000 7f6835084b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e820 0x55ed9a92b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.307+0000 7f6835084b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ed9a84ce40 con 0x55ed9a79fe40 2024-05-27T07:51:04.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.307+0000 7f6835084b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed9a83c9e0 con 0x55ed9a91e410 2024-05-27T07:51:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.307+0000 7f6835084b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ed9a840a20 con 0x55ed9a91e820 2024-05-27T07:51:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.307+0000 7f6817e00700 1 -- 172.21.15.63:0/1384211531 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f680c019070 con 0x7f6820105bd0 2024-05-27T07:51:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.308+0000 7f681fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a91ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.308+0000 7f681f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 0x55ed9a928ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.308+0000 7f681fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a91ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43720/0 (socket says 172.21.15.63:43720) 2024-05-27T07:51:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.308+0000 7f681fe00700 1 -- 172.21.15.63:0/755856203 learned_addr learned my addr 172.21.15.63:0/755856203 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.308+0000 7f681ea00700 1 --2- 172.21.15.63:0/755856203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e820 0x55ed9a92b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.309+0000 7f681f400700 1 -- 172.21.15.63:0/755856203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e820 msgr2=0x55ed9a92b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.309+0000 7f681f400700 1 --2- 172.21.15.63:0/755856203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e820 0x55ed9a92b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.309+0000 7f681f400700 1 -- 172.21.15.63:0/755856203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 msgr2=0x55ed9a91ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.309+0000 7f681f400700 1 --2- 172.21.15.63:0/755856203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a91ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.309+0000 7f681f400700 1 -- 172.21.15.63:0/755856203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9a805420 con 0x55ed9a91e410 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.309+0000 7f681fe00700 1 --2- 172.21.15.63:0/755856203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a91ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.309+0000 7f681f400700 1 --2- 172.21.15.63:0/755856203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 0x55ed9a928ad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6810000c00 tx=0x7f68100096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b4f7d3212363a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.310+0000 7f6814c00700 1 -- 172.21.15.63:0/755856203 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810026070 con 0x55ed9a91e410 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.310+0000 7f6814c00700 1 -- 172.21.15.63:0/755856203 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f681000bc60 con 0x55ed9a91e410 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.310+0000 7f6814c00700 1 -- 172.21.15.63:0/755856203 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681000fbf0 con 0x55ed9a91e410 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.310+0000 7f6817e00700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68040b2aa0 0x7f68040b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.310+0000 7f6817e00700 1 -- 172.21.15.63:0/1384211531 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f680c0e2680 con 0x7f6820105bd0 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.311+0000 7f681fe00700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68040b2aa0 0x7f68040b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.311+0000 7f6835084b80 1 -- 172.21.15.63:0/755856203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 msgr2=0x55ed9a928ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.311+0000 7f6835084b80 1 --2- 172.21.15.63:0/755856203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 0x55ed9a928ad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6810000c00 tx=0x7f68100096a0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.311+0000 7f681fe00700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68040b2aa0 0x7f68040b4fb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55ed9a91a690 tx=0x7f68180079d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.312+0000 7f6835084b80 1 -- 172.21.15.63:0/755856203 shutdown_connections 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.312+0000 7f6835084b80 1 --2- 172.21.15.63:0/755856203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e820 0x55ed9a92b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.312+0000 7f6835084b80 1 --2- 172.21.15.63:0/755856203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a91ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.312+0000 7f6835084b80 1 --2- 172.21.15.63:0/755856203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 0x55ed9a928ad0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.312+0000 7f6835084b80 1 -- 172.21.15.63:0/755856203 >> 172.21.15.63:0/755856203 conn(0x55ed9a7fffc0 msgr2=0x55ed9a919d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.312+0000 7f6835084b80 1 -- 172.21.15.63:0/755856203 shutdown_connections 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.312+0000 7f6835084b80 1 -- 172.21.15.63:0/755856203 wait complete. 2024-05-27T07:51:04.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.312+0000 7f6835084b80 1 -- start start 2024-05-27T07:51:04.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.313+0000 7f6835084b80 1 Processor -- start 2024-05-27T07:51:04.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.313+0000 7f6835084b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a928520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.313+0000 7f6835084b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 0x55ed9a928a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.313+0000 7f6835084b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e820 0x55ed9a928fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.313+0000 7f6835084b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ed9a742fa0 con 0x55ed9a79fe40 2024-05-27T07:51:04.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.313+0000 7f6835084b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed9a7dd760 con 0x55ed9a91e820 2024-05-27T07:51:04.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.313+0000 7f6835084b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ed9a92b930 con 0x55ed9a91e410 2024-05-27T07:51:04.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f6835084b80 10 client.0 _ll_register_callbacks cb 0x55ed9a7e6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:04.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a928520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a928520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43736/0 (socket says 172.21.15.63:43736) 2024-05-27T07:51:04.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681f400700 1 -- 172.21.15.63:0/3198184902 learned_addr learned my addr 172.21.15.63:0/3198184902 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681ea00700 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 0x55ed9a928a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681fe00700 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e820 0x55ed9a928fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681f400700 1 -- 172.21.15.63:0/3198184902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 msgr2=0x55ed9a928a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681f400700 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 0x55ed9a928a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681f400700 1 -- 172.21.15.63:0/3198184902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e820 msgr2=0x55ed9a928fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681f400700 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e820 0x55ed9a928fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681f400700 1 -- 172.21.15.63:0/3198184902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed9a78c8d0 con 0x55ed9a79fe40 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.314+0000 7f681fe00700 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e820 0x55ed9a928fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.315+0000 7f681f400700 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a928520 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6810009ce0 tx=0x7f681000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b4207290acb60f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.315+0000 7f6809600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:04.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.315+0000 7f6809600700 1 -- 172.21.15.63:0/3198184902 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810026030 con 0x55ed9a79fe40 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.315+0000 7f6835084b80 1 -- 172.21.15.63:0/3198184902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f681000bc60 con 0x55ed9a79fe40 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.315+0000 7f6809600700 1 -- 172.21.15.63:0/3198184902 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f681001e040 con 0x55ed9a79fe40 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6809600700 1 -- 172.21.15.63:0/3198184902 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6810033050 con 0x55ed9a79fe40 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6835084b80 10 client.18804 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6835084b80 1 -- 172.21.15.63:0/3198184902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ed9a80d060 con 0x55ed9a79fe40 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6809600700 1 -- 172.21.15.63:0/3198184902 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f681000bc60 con 0x55ed9a79fe40 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6809600700 1 -- 172.21.15.63:0/3198184902 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6810021bf0 con 0x55ed9a79fe40 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6835084b80 10 client.18804 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6835084b80 10 client.18804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6835084b80 20 client.18804 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6835084b80 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed9a938740 0x55ed9a93ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6835084b80 4 client.18804 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.316+0000 7f6835084b80 1 -- 172.21.15.63:0/3198184902 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed9a84ed90 con 0x55ed9a938740 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.317+0000 7f681ea00700 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed9a938740 0x55ed9a93ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.318+0000 7f681ea00700 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed9a938740 0x55ed9a93ab60 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d2f911ce70381a0 server_cookie=8f33c1b8e11a08e9 in_seq=0 out_seq=0 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.318+0000 7f6809600700 10 client.18804 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.320+0000 7f6809600700 1 -- 172.21.15.63:0/3198184902 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ed9a84ed90 con 0x55ed9a938740 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.320+0000 7f6809600700 10 client.18804 handle_command_reply: tid=0 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.320+0000 7f6835084b80 1 client.18804 shutdown 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.320+0000 7f6835084b80 1 -- 172.21.15.63:0/3198184902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 msgr2=0x55ed9a928520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.320+0000 7f6835084b80 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a928520 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6810009ce0 tx=0x7f681000f740 comp rx=0 tx=0).stop 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.321+0000 7f6835084b80 1 -- 172.21.15.63:0/3198184902 shutdown_connections 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.321+0000 7f6835084b80 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed9a938740 0x55ed9a93ab60 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.321+0000 7f6835084b80 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e820 0x55ed9a928fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.321+0000 7f6835084b80 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a79fe40 0x55ed9a928520 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.321+0000 7f6835084b80 1 --2- 172.21.15.63:0/3198184902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 0x55ed9a928a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.321+0000 7f6809600700 0 client.18804 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.321+0000 7f6835084b80 1 -- 172.21.15.63:0/3198184902 >> 172.21.15.63:0/3198184902 conn(0x55ed9a7fffc0 msgr2=0x55ed9a918ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.321+0000 7f6835084b80 1 -- 172.21.15.63:0/3198184902 shutdown_connections 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.321+0000 7f6835084b80 1 -- 172.21.15.63:0/3198184902 wait complete. 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.321+0000 7f6835084b80 20 client.18804 trim_cache size 0 max 16384 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.350+0000 7f6835084b80 1 Processor -- start 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.351+0000 7f6835084b80 1 -- start start 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.351+0000 7f6835084b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a79fe40 0x55ed9a923cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.351+0000 7f6835084b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 0x55ed9a934c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.351+0000 7f6835084b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a93db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.351+0000 7f6835084b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b4e70 con 0x55ed9a91e820 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.351+0000 7f6835084b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b2700 con 0x55ed9a91e410 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.351+0000 7f6835084b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b0570 con 0x55ed9a79fe40 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.351+0000 7f681fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a93db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.352+0000 7f681fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a93db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43748/0 (socket says 172.21.15.63:43748) 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.352+0000 7f681fe00700 1 -- 172.21.15.63:0/3812311174 learned_addr learned my addr 172.21.15.63:0/3812311174 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.352+0000 7f681ea00700 1 --2- 172.21.15.63:0/3812311174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 0x55ed9a934c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.352+0000 7f681f400700 1 --2- 172.21.15.63:0/3812311174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a79fe40 0x55ed9a923cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.352+0000 7f681fe00700 1 -- 172.21.15.63:0/3812311174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a79fe40 msgr2=0x55ed9a923cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.352+0000 7f681fe00700 1 --2- 172.21.15.63:0/3812311174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a79fe40 0x55ed9a923cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.352+0000 7f681fe00700 1 -- 172.21.15.63:0/3812311174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 msgr2=0x55ed9a934c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.352+0000 7f681fe00700 1 --2- 172.21.15.63:0/3812311174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 0x55ed9a934c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.352+0000 7f681fe00700 1 -- 172.21.15.63:0/3812311174 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9a92b9b0 con 0x55ed9a91e820 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.353+0000 7f681fe00700 1 --2- 172.21.15.63:0/3812311174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a93db80 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6818016760 tx=0x7f6818014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3265089e449e504f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.353+0000 7f680be00700 1 -- 172.21.15.63:0/3812311174 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681801d040 con 0x55ed9a91e820 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.353+0000 7f680be00700 1 -- 172.21.15.63:0/3812311174 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6818027070 con 0x55ed9a91e820 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.353+0000 7f680be00700 1 -- 172.21.15.63:0/3812311174 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818017760 con 0x55ed9a91e820 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.354+0000 7f6835084b80 1 -- 172.21.15.63:0/3812311174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 msgr2=0x55ed9a93db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.354+0000 7f6835084b80 1 --2- 172.21.15.63:0/3812311174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a93db80 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6818016760 tx=0x7f6818014000 comp rx=0 tx=0).stop 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.354+0000 7f6835084b80 1 -- 172.21.15.63:0/3812311174 shutdown_connections 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.354+0000 7f6835084b80 1 --2- 172.21.15.63:0/3812311174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a93db80 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.354+0000 7f6835084b80 1 --2- 172.21.15.63:0/3812311174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a79fe40 0x55ed9a923cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.354+0000 7f6835084b80 1 --2- 172.21.15.63:0/3812311174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a91e410 0x55ed9a934c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.354+0000 7f6835084b80 1 -- 172.21.15.63:0/3812311174 >> 172.21.15.63:0/3812311174 conn(0x55ed9a7fffc0 msgr2=0x55ed9a928d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:04.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.354+0000 7f6835084b80 1 -- 172.21.15.63:0/3812311174 shutdown_connections 2024-05-27T07:51:04.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.355+0000 7f6835084b80 1 -- 172.21.15.63:0/3812311174 wait complete. 2024-05-27T07:51:04.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.355+0000 7f6835084b80 1 -- start start 2024-05-27T07:51:04.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.355+0000 7f6835084b80 1 Processor -- start 2024-05-27T07:51:04.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.356+0000 7f6835084b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a79fe40 0x55ed9a93bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.356+0000 7f6835084b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 0x55ed9a93c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.356+0000 7f6835084b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a9a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.356+0000 7f6835084b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b0bc0 con 0x55ed9a91e820 2024-05-27T07:51:04.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.356+0000 7f6835084b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b2030 con 0x55ed9a79fe40 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.356+0000 7f6835084b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b1290 con 0x55ed9a91e410 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.356+0000 7f6835084b80 10 client.0 _ll_register_callbacks cb 0x55ed9a7e6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.356+0000 7f681fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a9a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.356+0000 7f681fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a9a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43764/0 (socket says 172.21.15.63:43764) 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.356+0000 7f681fe00700 1 -- 172.21.15.63:0/1966180401 learned_addr learned my addr 172.21.15.63:0/1966180401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.357+0000 7f681f400700 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a79fe40 0x55ed9a93bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.357+0000 7f681ea00700 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 0x55ed9a93c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.357+0000 7f681fe00700 1 -- 172.21.15.63:0/1966180401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 msgr2=0x55ed9a93c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.357+0000 7f681fe00700 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 0x55ed9a93c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.357+0000 7f681fe00700 1 -- 172.21.15.63:0/1966180401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a79fe40 msgr2=0x55ed9a93bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.357+0000 7f681fe00700 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a79fe40 0x55ed9a93bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.357+0000 7f681fe00700 1 -- 172.21.15.63:0/1966180401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed9a93cae0 con 0x55ed9a91e820 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.357+0000 7f681ea00700 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 0x55ed9a93c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.357+0000 7f681fe00700 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a9a1fb0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f6818008560 tx=0x7f68180226f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90c5c5790d6e6eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.357+0000 7f6809600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.358+0000 7f6809600700 1 -- 172.21.15.63:0/1966180401 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681802f070 con 0x55ed9a91e820 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.358+0000 7f6809600700 1 -- 172.21.15.63:0/1966180401 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f6818022c50 con 0x55ed9a91e820 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.358+0000 7f6835084b80 1 -- 172.21.15.63:0/1966180401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6818027070 con 0x55ed9a91e820 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.358+0000 7f6809600700 1 -- 172.21.15.63:0/1966180401 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f681803b050 con 0x55ed9a91e820 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.358+0000 7f6835084b80 10 client.18810 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.358+0000 7f6835084b80 1 -- 172.21.15.63:0/1966180401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ed9a9a3570 con 0x55ed9a91e820 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.359+0000 7f6809600700 1 -- 172.21.15.63:0/1966180401 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6818027070 con 0x55ed9a91e820 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.359+0000 7f6809600700 1 -- 172.21.15.63:0/1966180401 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f68180173f0 con 0x55ed9a91e820 2024-05-27T07:51:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.359+0000 7f6835084b80 10 client.18810 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.359+0000 7f6835084b80 10 client.18810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.359+0000 7f6835084b80 20 client.18810 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.359+0000 7f6835084b80 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed9a9a3ba0 0x55ed9a9a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.359+0000 7f6835084b80 4 client.18810 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.359+0000 7f6835084b80 1 -- 172.21.15.63:0/1966180401 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed9a848570 con 0x55ed9a9a3ba0 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.360+0000 7f681f400700 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed9a9a3ba0 0x55ed9a9a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.361+0000 7f681f400700 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed9a9a3ba0 0x55ed9a9a5fc0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e61254dbe3d62b server_cookie=42fdad73ad178933 in_seq=0 out_seq=0 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.361+0000 7f6809600700 10 client.18810 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.361+0000 7f6809600700 1 -- 172.21.15.63:0/1966180401 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ed9a848570 con 0x55ed9a9a3ba0 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.361+0000 7f6809600700 10 client.18810 handle_command_reply: tid=0 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.361+0000 7f6835084b80 1 client.18810 shutdown 2024-05-27T07:51:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.361+0000 7f6835084b80 1 -- 172.21.15.63:0/1966180401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 msgr2=0x55ed9a9a1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.361+0000 7f6835084b80 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a9a1fb0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f6818008560 tx=0x7f68180226f0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.361+0000 7f6835084b80 1 -- 172.21.15.63:0/1966180401 shutdown_connections 2024-05-27T07:51:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.362+0000 7f6835084b80 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed9a9a3ba0 0x55ed9a9a5fc0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.362+0000 7f6835084b80 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed9a91e820 0x55ed9a9a1fb0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.362+0000 7f6835084b80 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed9a79fe40 0x55ed9a93bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.362+0000 7f6835084b80 1 --2- 172.21.15.63:0/1966180401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed9a91e410 0x55ed9a93c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.362+0000 7f6835084b80 1 -- 172.21.15.63:0/1966180401 >> 172.21.15.63:0/1966180401 conn(0x55ed9a7fffc0 msgr2=0x55ed9a928a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.362+0000 7f6809600700 0 client.18810 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.362+0000 7f6835084b80 1 -- 172.21.15.63:0/1966180401 shutdown_connections 2024-05-27T07:51:04.580 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:51:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:51:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:51:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: "60916075-a1dc-493b-975d-2be43a63a747": { 2024-05-27T07:51:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:51:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "60916075-a1dc-493b-975d-2be43a63a747", 2024-05-27T07:51:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:51:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:51:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:51:04.580 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.362+0000 7f6835084b80 1 -- 172.21.15.63:0/1966180401 wait complete. 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.362+0000 7f6835084b80 20 client.18810 trim_cache size 0 max 16384 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 -- 172.21.15.63:0/1384211531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68040b2aa0 msgr2=0x7f68040b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68040b2aa0 0x7f68040b4fb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55ed9a91a690 tx=0x7f68180079d0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 -- 172.21.15.63:0/1384211531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105bd0 msgr2=0x7f682019d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105bd0 0x7f682019d320 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f680c00d950 tx=0x7f680c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 -- 172.21.15.63:0/1384211531 shutdown_connections 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68040b2aa0 0x7f68040b4fb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f682010e6f0 0x7f68201a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820105280 0x7f682019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 --2- 172.21.15.63:0/1384211531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820105bd0 0x7f682019d320 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 -- 172.21.15.63:0/1384211531 >> 172.21.15.63:0/1384211531 conn(0x7f6820078ab0 msgr2=0x7f6820104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 -- 172.21.15.63:0/1384211531 shutdown_connections 2024-05-27T07:51:04.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:03.363+0000 7f6825000700 1 -- 172.21.15.63:0/1384211531 wait complete. 2024-05-27T07:51:05.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:04 smithi086 ceph-mon[34687]: pgmap v550: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:51:05.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:04 smithi149 ceph-mon[35408]: pgmap v550: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:51:05.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:04 smithi063 ceph-mon[31613]: pgmap v550: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:51:05.463 INFO:tasks.cephfs.filesystem:scrub status for tag:60916075-a1dc-493b-975d-2be43a63a747 - {'path': '/', 'tag': '60916075-a1dc-493b-975d-2be43a63a747', 'options': 'recursive,force'} 2024-05-27T07:51:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.805+0000 7f9acee00640 1 -- 172.21.15.63:0/2081640127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ac810bcd0 msgr2=0x7f9ac8105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.805+0000 7f9acee00640 1 --2- 172.21.15.63:0/2081640127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ac810bcd0 0x7f9ac8105870 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009900 tx=0x7f9abc02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.805+0000 7f9acee00640 1 -- 172.21.15.63:0/2081640127 shutdown_connections 2024-05-27T07:51:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.805+0000 7f9acee00640 1 --2- 172.21.15.63:0/2081640127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ac8106420 0x7f9ac81127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.805+0000 7f9acee00640 1 --2- 172.21.15.63:0/2081640127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac810c170 0x7f9ac8105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.805+0000 7f9acee00640 1 --2- 172.21.15.63:0/2081640127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ac810bcd0 0x7f9ac8105870 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.805+0000 7f9acee00640 1 -- 172.21.15.63:0/2081640127 >> 172.21.15.63:0/2081640127 conn(0x7f9ac81013d0 msgr2=0x7f9ac8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.805+0000 7f9acee00640 1 -- 172.21.15.63:0/2081640127 shutdown_connections 2024-05-27T07:51:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.805+0000 7f9acee00640 1 -- 172.21.15.63:0/2081640127 wait complete. 2024-05-27T07:51:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.806+0000 7f9acee00640 1 Processor -- start 2024-05-27T07:51:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.806+0000 7f9acee00640 1 -- start start 2024-05-27T07:51:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.807+0000 7f9acee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ac8106420 0x7f9ac819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.807+0000 7f9acee00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ac810bcd0 0x7f9ac819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.807+0000 7f9acee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac810c170 0x7f9ac81a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.807+0000 7f9acee00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac8115d50 con 0x7f9ac810bcd0 2024-05-27T07:51:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.807+0000 7f9acee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac8115bd0 con 0x7f9ac810c170 2024-05-27T07:51:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.807+0000 7f9acee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac8115ed0 con 0x7f9ac8106420 2024-05-27T07:51:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.807+0000 7f9acd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ac810bcd0 0x7f9ac819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.807+0000 7f9acd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ac810bcd0 0x7f9ac819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43774/0 (socket says 172.21.15.63:43774) 2024-05-27T07:51:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.807+0000 7f9acd000640 1 -- 172.21.15.63:0/692161797 learned_addr learned my addr 172.21.15.63:0/692161797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.808+0000 7f9acda00640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ac8106420 0x7f9ac819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.808+0000 7f9ace400640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac810c170 0x7f9ac81a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.808+0000 7f9acd000640 1 -- 172.21.15.63:0/692161797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ac8106420 msgr2=0x7f9ac819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.808+0000 7f9acd000640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ac8106420 0x7f9ac819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.808+0000 7f9acd000640 1 -- 172.21.15.63:0/692161797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac810c170 msgr2=0x7f9ac81a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.808+0000 7f9acd000640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac810c170 0x7f9ac81a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.808+0000 7f9acd000640 1 -- 172.21.15.63:0/692161797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac81a1650 con 0x7f9ac810bcd0 2024-05-27T07:51:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.808+0000 7f9acd000640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ac810bcd0 0x7f9ac819cd50 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400d740 tx=0x7f9ab400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.808+0000 7f9ac6000640 1 -- 172.21.15.63:0/692161797 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4017070 con 0x7f9ac810bcd0 2024-05-27T07:51:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.809+0000 7f9ac6000640 1 -- 172.21.15.63:0/692161797 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ab400de50 con 0x7f9ac810bcd0 2024-05-27T07:51:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.809+0000 7f9ac6000640 1 -- 172.21.15.63:0/692161797 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4003bb0 con 0x7f9ac810bcd0 2024-05-27T07:51:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.809+0000 7f9acee00640 1 -- 172.21.15.63:0/692161797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac81a1940 con 0x7f9ac810bcd0 2024-05-27T07:51:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.809+0000 7f9acee00640 1 -- 172.21.15.63:0/692161797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac81a1e40 con 0x7f9ac810bcd0 2024-05-27T07:51:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.813+0000 7f9ad7394740 1 Processor -- start 2024-05-27T07:51:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.813+0000 7f9ad7394740 1 -- start start 2024-05-27T07:51:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.813+0000 7f9ac6000640 1 -- 172.21.15.63:0/692161797 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9ab4003d50 con 0x7f9ac810bcd0 2024-05-27T07:51:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.813+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17bb51b0 0x55fe17f57a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.813+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f57fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.813+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17b944d0 0x55fe17f584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.813+0000 7f9ad7394740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fe17ec8b90 con 0x55fe17b944d0 2024-05-27T07:51:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.813+0000 7f9ad7394740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fe17eb8730 con 0x55fe17b8c2f0 2024-05-27T07:51:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.813+0000 7f9ad7394740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fe17ebc770 con 0x55fe17bb51b0 2024-05-27T07:51:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.814+0000 7f9acd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17b944d0 0x55fe17f584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.814+0000 7f9acd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17b944d0 0x55fe17f584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43780/0 (socket says 172.21.15.63:43780) 2024-05-27T07:51:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.814+0000 7f9acd000640 1 -- 172.21.15.63:0/3247001635 learned_addr learned my addr 172.21.15.63:0/3247001635 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.814+0000 7f9acda00640 1 --2- 172.21.15.63:0/3247001635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f57fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.814+0000 7f9ace400640 1 --2- 172.21.15.63:0/3247001635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17bb51b0 0x55fe17f57a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.814+0000 7f9acd000640 1 -- 172.21.15.63:0/3247001635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17bb51b0 msgr2=0x55fe17f57a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.814+0000 7f9acd000640 1 --2- 172.21.15.63:0/3247001635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17bb51b0 0x55fe17f57a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.814+0000 7f9acd000640 1 -- 172.21.15.63:0/3247001635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 msgr2=0x55fe17f57fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.814+0000 7f9acd000640 1 --2- 172.21.15.63:0/3247001635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f57fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.814+0000 7f9acd000640 1 -- 172.21.15.63:0/3247001635 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe17e518b0 con 0x55fe17b944d0 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.815+0000 7f9acd000640 1 --2- 172.21.15.63:0/3247001635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17b944d0 0x55fe17f584e0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40df680 tx=0x7f9ab40eaa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfa3aa236598ea8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.815+0000 7f9ac6000640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ab00b29d0 0x7f9ab00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.815+0000 7f9abaa00640 1 -- 172.21.15.63:0/3247001635 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab40f6070 con 0x55fe17b944d0 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.815+0000 7f9abaa00640 1 -- 172.21.15.63:0/3247001635 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ab40a8db0 con 0x55fe17b944d0 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.816+0000 7f9ac6000640 1 -- 172.21.15.63:0/692161797 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f9ab40e2380 con 0x7f9ac810bcd0 2024-05-27T07:51:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.816+0000 7f9ace400640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ab00b29d0 0x7f9ab00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.816+0000 7f9abaa00640 1 -- 172.21.15.63:0/3247001635 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab40ad8b0 con 0x55fe17b944d0 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.817+0000 7f9ace400640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ab00b29d0 0x7f9ab00b4ee0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55fe17f54bd0 tx=0x7f9ac0009740 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.817+0000 7f9ad7394740 1 -- 172.21.15.63:0/3247001635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17b944d0 msgr2=0x55fe17f584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.817+0000 7f9ad7394740 1 --2- 172.21.15.63:0/3247001635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17b944d0 0x55fe17f584e0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40df680 tx=0x7f9ab40eaa10 comp rx=0 tx=0).stop 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.817+0000 7f9ad7394740 1 -- 172.21.15.63:0/3247001635 shutdown_connections 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.817+0000 7f9ad7394740 1 --2- 172.21.15.63:0/3247001635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17b944d0 0x55fe17f584e0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.817+0000 7f9ad7394740 1 --2- 172.21.15.63:0/3247001635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f57fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.817+0000 7f9ad7394740 1 --2- 172.21.15.63:0/3247001635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17bb51b0 0x55fe17f57a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.817+0000 7f9ad7394740 1 -- 172.21.15.63:0/3247001635 >> 172.21.15.63:0/3247001635 conn(0x55fe17ba9300 msgr2=0x55fe17f54210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.817+0000 7f9ad7394740 1 -- 172.21.15.63:0/3247001635 shutdown_connections 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.817+0000 7f9ad7394740 1 -- 172.21.15.63:0/3247001635 wait complete. 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.818+0000 7f9ad7394740 1 -- start start 2024-05-27T07:51:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.818+0000 7f9ad7394740 1 Processor -- start 2024-05-27T07:51:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f61b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b944d0 0x55fe17f620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f6df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9ad7394740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fe17b9e790 con 0x55fe17bb51b0 2024-05-27T07:51:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9ad7394740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fe17e60bc0 con 0x55fe17b8c2f0 2024-05-27T07:51:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9ad7394740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fe17f64be0 con 0x55fe17b944d0 2024-05-27T07:51:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9ad7394740 10 client.0 _ll_register_callbacks cb 0x55fe17e3e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9ace400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f6df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9ace400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f6df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43788/0 (socket says 172.21.15.63:43788) 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9ace400640 1 -- 172.21.15.63:0/2824281839 learned_addr learned my addr 172.21.15.63:0/2824281839 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9acd000640 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b944d0 0x55fe17f620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.819+0000 7f9acda00640 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f61b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.820+0000 7f9ace400640 1 -- 172.21.15.63:0/2824281839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b944d0 msgr2=0x55fe17f620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.820+0000 7f9ace400640 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b944d0 0x55fe17f620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.820+0000 7f9ace400640 1 -- 172.21.15.63:0/2824281839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 msgr2=0x55fe17f61b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.820+0000 7f9ace400640 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f61b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.820+0000 7f9ace400640 1 -- 172.21.15.63:0/2824281839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe17bcd210 con 0x55fe17bb51b0 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.820+0000 7f9ace400640 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f6df10 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0005c40 tx=0x7f9ac0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92328ab01fc0d86 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.820+0000 7f9aaf400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.821+0000 7f9aaf400640 1 -- 172.21.15.63:0/2824281839 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac001ed40 con 0x55fe17bb51b0 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.821+0000 7f9ad7394740 1 -- 172.21.15.63:0/2824281839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab40a8db0 con 0x55fe17bb51b0 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.821+0000 7f9aaf400640 1 -- 172.21.15.63:0/2824281839 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f9ac001f420 con 0x55fe17bb51b0 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.821+0000 7f9aaf400640 1 -- 172.21.15.63:0/2824281839 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9ac0032050 con 0x55fe17bb51b0 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.821+0000 7f9ad7394740 10 client.18822 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.821+0000 7f9ad7394740 1 -- 172.21.15.63:0/2824281839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fe17c99620 con 0x55fe17bb51b0 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.822+0000 7f9aaf400640 1 -- 172.21.15.63:0/2824281839 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ab40a8db0 con 0x55fe17bb51b0 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.822+0000 7f9aaf400640 1 -- 172.21.15.63:0/2824281839 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9ac000f040 con 0x55fe17bb51b0 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.822+0000 7f9ad7394740 10 client.18822 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.822+0000 7f9ad7394740 10 client.18822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.822+0000 7f9ad7394740 20 client.18822 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.822+0000 7f9ad7394740 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe17b73d80 0x55fe17f73eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.822+0000 7f9ad7394740 4 client.18822 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.822+0000 7f9ad7394740 1 -- 172.21.15.63:0/2824281839 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe17ecaae0 con 0x55fe17b73d80 2024-05-27T07:51:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.823+0000 7f9acda00640 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe17b73d80 0x55fe17f73eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.824+0000 7f9acda00640 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe17b73d80 0x55fe17f73eb0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69e965af0c81a678 server_cookie=fb2fddce9a5adea1 in_seq=0 out_seq=0 2024-05-27T07:51:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.824+0000 7f9aaf400640 10 client.18822 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9aaf400640 1 -- 172.21.15.63:0/2824281839 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fe17ecaae0 con 0x55fe17b73d80 2024-05-27T07:51:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9aaf400640 10 client.18822 handle_command_reply: tid=0 2024-05-27T07:51:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9ad7394740 1 client.18822 shutdown 2024-05-27T07:51:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9ad7394740 1 -- 172.21.15.63:0/2824281839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 msgr2=0x55fe17f6df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9ad7394740 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f6df10 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0005c40 tx=0x7f9ac0004a40 comp rx=0 tx=0).stop 2024-05-27T07:51:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9ad7394740 1 -- 172.21.15.63:0/2824281839 shutdown_connections 2024-05-27T07:51:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9ad7394740 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe17b73d80 0x55fe17f73eb0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9ad7394740 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f6df10 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9ad7394740 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b944d0 0x55fe17f620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9ad7394740 1 --2- 172.21.15.63:0/2824281839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f61b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9ad7394740 1 -- 172.21.15.63:0/2824281839 >> 172.21.15.63:0/2824281839 conn(0x55fe17ba9300 msgr2=0x55fe17f59780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9aaf400640 0 client.18822 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.826+0000 7f9ad7394740 1 -- 172.21.15.63:0/2824281839 shutdown_connections 2024-05-27T07:51:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.827+0000 7f9ad7394740 1 -- 172.21.15.63:0/2824281839 wait complete. 2024-05-27T07:51:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.827+0000 7f9ad7394740 20 client.18822 trim_cache size 0 max 16384 2024-05-27T07:51:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.844+0000 7f9ad7394740 1 Processor -- start 2024-05-27T07:51:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.844+0000 7f9ad7394740 1 -- start start 2024-05-27T07:51:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.844+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.844+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b944d0 0x55fe17f5d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.844+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f54260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.844+0000 7f9ad7394740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81b5c60 con 0x55fe17bb51b0 2024-05-27T07:51:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.844+0000 7f9ad7394740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81b3530 con 0x55fe17b8c2f0 2024-05-27T07:51:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.844+0000 7f9ad7394740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81b13a0 con 0x55fe17b944d0 2024-05-27T07:51:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.845+0000 7f9ace400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f54260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.845+0000 7f9ace400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f54260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43802/0 (socket says 172.21.15.63:43802) 2024-05-27T07:51:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.845+0000 7f9ace400640 1 -- 172.21.15.63:0/2896314322 learned_addr learned my addr 172.21.15.63:0/2896314322 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.845+0000 7f9acda00640 1 --2- 172.21.15.63:0/2896314322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.845+0000 7f9acd000640 1 --2- 172.21.15.63:0/2896314322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b944d0 0x55fe17f5d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.845+0000 7f9ace400640 1 -- 172.21.15.63:0/2896314322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b944d0 msgr2=0x55fe17f5d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.845+0000 7f9ace400640 1 --2- 172.21.15.63:0/2896314322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b944d0 0x55fe17f5d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.845+0000 7f9ace400640 1 -- 172.21.15.63:0/2896314322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 msgr2=0x55fe17f642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.845+0000 7f9ace400640 1 --2- 172.21.15.63:0/2896314322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.845+0000 7f9ace400640 1 -- 172.21.15.63:0/2896314322 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe17f74320 con 0x55fe17bb51b0 2024-05-27T07:51:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.846+0000 7f9ace400640 1 --2- 172.21.15.63:0/2896314322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f54260 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0012d40 tx=0x7f9ac0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c069b2f5b68446 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.846+0000 7f9aba000640 1 -- 172.21.15.63:0/2896314322 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac00105d0 con 0x55fe17bb51b0 2024-05-27T07:51:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.846+0000 7f9aba000640 1 -- 172.21.15.63:0/2896314322 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ac001e780 con 0x55fe17bb51b0 2024-05-27T07:51:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.846+0000 7f9aba000640 1 -- 172.21.15.63:0/2896314322 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac001ce10 con 0x55fe17bb51b0 2024-05-27T07:51:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.847+0000 7f9ad7394740 1 -- 172.21.15.63:0/2896314322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 msgr2=0x55fe17f54260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.847+0000 7f9ad7394740 1 --2- 172.21.15.63:0/2896314322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f54260 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0012d40 tx=0x7f9ac0005fd0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.847+0000 7f9ad7394740 1 -- 172.21.15.63:0/2896314322 shutdown_connections 2024-05-27T07:51:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.847+0000 7f9ad7394740 1 --2- 172.21.15.63:0/2896314322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f54260 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.847+0000 7f9ad7394740 1 --2- 172.21.15.63:0/2896314322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b944d0 0x55fe17f5d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.847+0000 7f9ad7394740 1 --2- 172.21.15.63:0/2896314322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b8c2f0 0x55fe17f642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.847+0000 7f9ad7394740 1 -- 172.21.15.63:0/2896314322 >> 172.21.15.63:0/2896314322 conn(0x55fe17ba9300 msgr2=0x55fe17f577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.848+0000 7f9ad7394740 1 -- 172.21.15.63:0/2896314322 shutdown_connections 2024-05-27T07:51:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.848+0000 7f9ad7394740 1 -- 172.21.15.63:0/2896314322 wait complete. 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.848+0000 7f9ad7394740 1 -- start start 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.848+0000 7f9ad7394740 1 Processor -- start 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b8c2f0 0x55fe17f5cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b944d0 0x55fe17f5b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9ad7394740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f5b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9ad7394740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81b19f0 con 0x55fe17bb51b0 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9ad7394740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81b2e60 con 0x55fe17b944d0 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9ad7394740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81b20c0 con 0x55fe17b8c2f0 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9ad7394740 10 client.0 _ll_register_callbacks cb 0x7f9ac00103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9ace400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f5b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9ace400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f5b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43816/0 (socket says 172.21.15.63:43816) 2024-05-27T07:51:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9ace400640 1 -- 172.21.15.63:0/1041452548 learned_addr learned my addr 172.21.15.63:0/1041452548 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9acda00640 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b8c2f0 0x55fe17f5cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.849+0000 7f9acd000640 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b944d0 0x55fe17f5b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.850+0000 7f9ace400640 1 -- 172.21.15.63:0/1041452548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b8c2f0 msgr2=0x55fe17f5cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.850+0000 7f9ace400640 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b8c2f0 0x55fe17f5cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.850+0000 7f9ace400640 1 -- 172.21.15.63:0/1041452548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b944d0 msgr2=0x55fe17f5b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.850+0000 7f9ace400640 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b944d0 0x55fe17f5b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.850+0000 7f9ace400640 1 -- 172.21.15.63:0/1041452548 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe17f5be00 con 0x55fe17bb51b0 2024-05-27T07:51:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.850+0000 7f9acd000640 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b944d0 0x55fe17f5b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.850+0000 7f9ace400640 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f5b8c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f9ac001c9d0 tx=0x7f9ac0010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692358657a7531ee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.850+0000 7f9aaf400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.850+0000 7f9aaf400640 1 -- 172.21.15.63:0/1041452548 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0044020 con 0x55fe17bb51b0 2024-05-27T07:51:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.851+0000 7f9aaf400640 1 -- 172.21.15.63:0/1041452548 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9483+0+0 (secure 0 0 0) 0x7f9ac001ed40 con 0x55fe17bb51b0 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.851+0000 7f9ad7394740 1 -- 172.21.15.63:0/1041452548 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ac001e780 con 0x55fe17bb51b0 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.851+0000 7f9aaf400640 1 -- 172.21.15.63:0/1041452548 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9ac0004a40 con 0x55fe17bb51b0 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.851+0000 7f9ad7394740 10 client.18828 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.851+0000 7f9ad7394740 1 -- 172.21.15.63:0/1041452548 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fe17f69b70 con 0x55fe17bb51b0 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.851+0000 7f9aaf400640 1 -- 172.21.15.63:0/1041452548 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ac001e780 con 0x55fe17bb51b0 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.851+0000 7f9aaf400640 1 -- 172.21.15.63:0/1041452548 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9ac0005660 con 0x55fe17bb51b0 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.852+0000 7f9ad7394740 10 client.18828 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.852+0000 7f9ad7394740 10 client.18828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.852+0000 7f9ad7394740 20 client.18828 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.852+0000 7f9ad7394740 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe17b491c0 0x55fe17f6a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.852+0000 7f9ad7394740 4 client.18828 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:51:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.852+0000 7f9ad7394740 1 -- 172.21.15.63:0/1041452548 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe17ec42c0 con 0x55fe17b491c0 2024-05-27T07:51:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.852+0000 7f9acda00640 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe17b491c0 0x55fe17f6a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.853+0000 7f9acda00640 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe17b491c0 0x55fe17f6a350 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff0f87fcad6345 server_cookie=d551a71bf2210d65 in_seq=0 out_seq=0 2024-05-27T07:51:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.853+0000 7f9aaf400640 10 client.18828 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9aaf400640 1 -- 172.21.15.63:0/1041452548 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fe17ec42c0 con 0x55fe17b491c0 2024-05-27T07:51:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9aaf400640 10 client.18828 handle_command_reply: tid=0 2024-05-27T07:51:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9ad7394740 1 client.18828 shutdown 2024-05-27T07:51:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9ad7394740 1 -- 172.21.15.63:0/1041452548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 msgr2=0x55fe17f5b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9ad7394740 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f5b8c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f9ac001c9d0 tx=0x7f9ac0010640 comp rx=0 tx=0).stop 2024-05-27T07:51:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9ad7394740 1 -- 172.21.15.63:0/1041452548 shutdown_connections 2024-05-27T07:51:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9ad7394740 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fe17b491c0 0x55fe17f6a350 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9ad7394740 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fe17bb51b0 0x55fe17f5b8c0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9ad7394740 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe17b944d0 0x55fe17f5b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9ad7394740 1 --2- 172.21.15.63:0/1041452548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe17b8c2f0 0x55fe17f5cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9aaf400640 0 client.18828 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.887+0000 7f9ad7394740 1 -- 172.21.15.63:0/1041452548 >> 172.21.15.63:0/1041452548 conn(0x55fe17ba9300 msgr2=0x55fe17f577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.888+0000 7f9ad7394740 1 -- 172.21.15.63:0/1041452548 shutdown_connections 2024-05-27T07:51:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.888+0000 7f9ad7394740 1 -- 172.21.15.63:0/1041452548 wait complete. 2024-05-27T07:51:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.888+0000 7f9ad7394740 20 client.18828 trim_cache size 0 max 16384 2024-05-27T07:51:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:51:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:51:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:51:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 -- 172.21.15.63:0/692161797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ab00b29d0 msgr2=0x7f9ab00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ab00b29d0 0x7f9ab00b4ee0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55fe17f54bd0 tx=0x7f9ac0009740 comp rx=0 tx=0).stop 2024-05-27T07:51:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 -- 172.21.15.63:0/692161797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ac810bcd0 msgr2=0x7f9ac819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ac810bcd0 0x7f9ac819cd50 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400d740 tx=0x7f9ab400dc00 comp rx=0 tx=0).stop 2024-05-27T07:51:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 -- 172.21.15.63:0/692161797 shutdown_connections 2024-05-27T07:51:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ab00b29d0 0x7f9ab00b4ee0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac810c170 0x7f9ac81a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ac810bcd0 0x7f9ac819cd50 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 --2- 172.21.15.63:0/692161797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ac8106420 0x7f9ac819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 -- 172.21.15.63:0/692161797 >> 172.21.15.63:0/692161797 conn(0x7f9ac81013d0 msgr2=0x7f9ac8103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 -- 172.21.15.63:0/692161797 shutdown_connections 2024-05-27T07:51:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:05.889+0000 7f9acee00640 1 -- 172.21.15.63:0/692161797 wait complete. 2024-05-27T07:51:07.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:06 smithi086 ceph-mon[34687]: pgmap v551: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:51:07.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:06 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:51:07.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:06 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:51:07.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:06 smithi149 ceph-mon[35408]: pgmap v551: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:51:07.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:06 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:51:07.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:06 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:51:07.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:06 smithi063 ceph-mon[31613]: pgmap v551: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 55 op/s 2024-05-27T07:51:07.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:06 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:51:07.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:06 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:51:08.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:51:08.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:07 smithi086 ceph-mon[34687]: osdmap e116: 12 total, 12 up, 12 in 2024-05-27T07:51:08.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:51:08.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:07 smithi149 ceph-mon[35408]: osdmap e116: 12 total, 12 up, 12 in 2024-05-27T07:51:08.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:51:08.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:07 smithi063 ceph-mon[31613]: osdmap e116: 12 total, 12 up, 12 in 2024-05-27T07:51:09.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:09 smithi086 ceph-mon[34687]: pgmap v553: 129 pgs: 129 active+clean; 4.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.5 MiB/s wr, 67 op/s 2024-05-27T07:51:09.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:09 smithi149 ceph-mon[35408]: pgmap v553: 129 pgs: 129 active+clean; 4.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.5 MiB/s wr, 67 op/s 2024-05-27T07:51:09.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:08 smithi063 ceph-mon[31613]: pgmap v553: 129 pgs: 129 active+clean; 4.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.5 MiB/s wr, 67 op/s 2024-05-27T07:51:11.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:11 smithi086 ceph-mon[34687]: pgmap v554: 129 pgs: 129 active+clean; 4.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.5 MiB/s wr, 67 op/s 2024-05-27T07:51:11.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:11 smithi149 ceph-mon[35408]: pgmap v554: 129 pgs: 129 active+clean; 4.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.5 MiB/s wr, 67 op/s 2024-05-27T07:51:11.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:11 smithi063 ceph-mon[31613]: pgmap v554: 129 pgs: 129 active+clean; 4.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.5 MiB/s wr, 67 op/s 2024-05-27T07:51:13.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:13 smithi086 ceph-mon[34687]: pgmap v555: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T07:51:13.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:13 smithi149 ceph-mon[35408]: pgmap v555: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T07:51:13.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:13 smithi063 ceph-mon[31613]: pgmap v555: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T07:51:15.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:15 smithi086 ceph-mon[34687]: pgmap v556: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T07:51:15.462 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:15 smithi149 ceph-mon[35408]: pgmap v556: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T07:51:15.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:15 smithi063 ceph-mon[31613]: pgmap v556: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T07:51:16.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:16 smithi086 ceph-mon[34687]: osdmap e117: 12 total, 12 up, 12 in 2024-05-27T07:51:16.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:16 smithi086 ceph-mon[34687]: pgmap v558: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-05-27T07:51:16.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:16 smithi149 ceph-mon[35408]: osdmap e117: 12 total, 12 up, 12 in 2024-05-27T07:51:16.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:16 smithi149 ceph-mon[35408]: pgmap v558: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-05-27T07:51:16.965 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:16 smithi063 ceph-mon[31613]: osdmap e117: 12 total, 12 up, 12 in 2024-05-27T07:51:16.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:16 smithi063 ceph-mon[31613]: pgmap v558: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.7 MiB/s wr, 4 op/s 2024-05-27T07:51:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.130+0000 7ff738c00640 1 -- 172.21.15.63:0/1595770099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734109ab0 msgr2=0x7ff7341036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.130+0000 7ff738c00640 1 --2- 172.21.15.63:0/1595770099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734109ab0 0x7ff7341036a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff724009900 tx=0x7ff72402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.131+0000 7ff738c00640 1 -- 172.21.15.63:0/1595770099 shutdown_connections 2024-05-27T07:51:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.131+0000 7ff738c00640 1 --2- 172.21.15.63:0/1595770099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff734104250 0x7ff7341105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.131+0000 7ff738c00640 1 --2- 172.21.15.63:0/1595770099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff734109f50 0x7ff734103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.131+0000 7ff738c00640 1 --2- 172.21.15.63:0/1595770099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734109ab0 0x7ff7341036a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.131+0000 7ff738c00640 1 -- 172.21.15.63:0/1595770099 >> 172.21.15.63:0/1595770099 conn(0x7ff734079240 msgr2=0x7ff734079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.131+0000 7ff738c00640 1 -- 172.21.15.63:0/1595770099 shutdown_connections 2024-05-27T07:51:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.131+0000 7ff738c00640 1 -- 172.21.15.63:0/1595770099 wait complete. 2024-05-27T07:51:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.132+0000 7ff738c00640 1 Processor -- start 2024-05-27T07:51:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.132+0000 7ff738c00640 1 -- start start 2024-05-27T07:51:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff738c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff734104250 0x7ff73419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff738c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734109ab0 0x7ff73419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff738c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff734109f50 0x7ff7341a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff738c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff734113b50 con 0x7ff734109ab0 2024-05-27T07:51:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff738c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341139d0 con 0x7ff734104250 2024-05-27T07:51:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff738c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff734113cd0 con 0x7ff734109f50 2024-05-27T07:51:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff72aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734109ab0 0x7ff73419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff72aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734109ab0 0x7ff73419db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60478/0 (socket says 172.21.15.63:60478) 2024-05-27T07:51:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff72aa00640 1 -- 172.21.15.63:0/55981592 learned_addr learned my addr 172.21.15.63:0/55981592 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff733400640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff734104250 0x7ff73419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff733e00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff734109f50 0x7ff7341a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff72aa00640 1 -- 172.21.15.63:0/55981592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff734109f50 msgr2=0x7ff7341a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.133+0000 7ff72aa00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff734109f50 0x7ff7341a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.134+0000 7ff72aa00640 1 -- 172.21.15.63:0/55981592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff734104250 msgr2=0x7ff73419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.134+0000 7ff72aa00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff734104250 0x7ff73419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.134+0000 7ff72aa00640 1 -- 172.21.15.63:0/55981592 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7341a89f0 con 0x7ff734109ab0 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.134+0000 7ff72aa00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734109ab0 0x7ff73419db50 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00c970 tx=0x7ff71c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.134+0000 7ff730c00640 1 -- 172.21.15.63:0/55981592 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c007c70 con 0x7ff734109ab0 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.134+0000 7ff730c00640 1 -- 172.21.15.63:0/55981592 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff71c007e10 con 0x7ff734109ab0 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.134+0000 7ff730c00640 1 -- 172.21.15.63:0/55981592 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0186f0 con 0x7ff734109ab0 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.134+0000 7ff738c00640 1 -- 172.21.15.63:0/55981592 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7341a8c20 con 0x7ff734109ab0 2024-05-27T07:51:19.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.134+0000 7ff738c00640 1 -- 172.21.15.63:0/55981592 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7341a90f0 con 0x7ff734109ab0 2024-05-27T07:51:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.138+0000 7ff741202740 1 Processor -- start 2024-05-27T07:51:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.139+0000 7ff741202740 1 -- start start 2024-05-27T07:51:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.139+0000 7ff741202740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc4ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.139+0000 7ff741202740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 0x55dbfdc4efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.139+0000 7ff741202740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc4f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.139+0000 7ff741202740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dbfdbbfb90 con 0x55dbfd88b4d0 2024-05-27T07:51:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.139+0000 7ff741202740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dbfdbaf730 con 0x55dbfd8832f0 2024-05-27T07:51:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.139+0000 7ff741202740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dbfdbb3770 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.140+0000 7ff733400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 0x55dbfdc4efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.140+0000 7ff733e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc4ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.140+0000 7ff733400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 0x55dbfdc4efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:45338/0 (socket says 172.21.15.63:45338) 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.140+0000 7ff733e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc4ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:57518/0 (socket says 172.21.15.63:57518) 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.140+0000 7ff733400640 1 -- 172.21.15.63:0/3723332425 learned_addr learned my addr 172.21.15.63:0/3723332425 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.140+0000 7ff730c00640 1 -- 172.21.15.63:0/55981592 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff71c00fa90 con 0x7ff734109ab0 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.140+0000 7ff72aa00640 1 --2- 172.21.15.63:0/3723332425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc4f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.141+0000 7ff733e00640 1 -- 172.21.15.63:0/3723332425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 msgr2=0x55dbfdc4efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.141+0000 7ff733e00640 1 --2- 172.21.15.63:0/3723332425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 0x55dbfdc4efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.141+0000 7ff733e00640 1 -- 172.21.15.63:0/3723332425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd88b4d0 msgr2=0x55dbfdc4f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.141+0000 7ff733e00640 1 --2- 172.21.15.63:0/3723332425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc4f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.141+0000 7ff733e00640 1 -- 172.21.15.63:0/3723332425 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbfdb488b0 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.141+0000 7ff733400640 1 --2- 172.21.15.63:0/3723332425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 0x55dbfdc4efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.141+0000 7ff733e00640 1 --2- 172.21.15.63:0/3723332425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc4ea60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff72c005fd0 tx=0x7ff72c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=937de6710f52d27f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.142+0000 7ff723e00640 1 -- 172.21.15.63:0/3723332425 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c016070 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.142+0000 7ff723e00640 1 -- 172.21.15.63:0/3723332425 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff72c010e50 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.142+0000 7ff723e00640 1 -- 172.21.15.63:0/3723332425 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0126b0 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.142+0000 7ff730c00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7180b29d0 0x7ff7180b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.142+0000 7ff730c00640 1 -- 172.21.15.63:0/55981592 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7ff71c0e29f0 con 0x7ff734109ab0 2024-05-27T07:51:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.142+0000 7ff733400640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7180b29d0 0x7ff7180b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.143+0000 7ff741202740 1 -- 172.21.15.63:0/3723332425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd8ac1b0 msgr2=0x55dbfdc4ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:19.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.143+0000 7ff741202740 1 --2- 172.21.15.63:0/3723332425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc4ea60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff72c005fd0 tx=0x7ff72c010740 comp rx=0 tx=0).stop 2024-05-27T07:51:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.143+0000 7ff741202740 1 -- 172.21.15.63:0/3723332425 shutdown_connections 2024-05-27T07:51:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.143+0000 7ff741202740 1 --2- 172.21.15.63:0/3723332425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc4f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.143+0000 7ff741202740 1 --2- 172.21.15.63:0/3723332425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 0x55dbfdc4efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.143+0000 7ff741202740 1 --2- 172.21.15.63:0/3723332425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc4ea60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.143+0000 7ff741202740 1 -- 172.21.15.63:0/3723332425 >> 172.21.15.63:0/3723332425 conn(0x55dbfd8a0300 msgr2=0x55dbfdc4b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.143+0000 7ff741202740 1 -- 172.21.15.63:0/3723332425 shutdown_connections 2024-05-27T07:51:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.143+0000 7ff741202740 1 -- 172.21.15.63:0/3723332425 wait complete. 2024-05-27T07:51:19.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.143+0000 7ff733400640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7180b29d0 0x7ff7180b4ee0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55dbfdc4c1a0 tx=0x7ff724009660 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.144+0000 7ff741202740 1 -- start start 2024-05-27T07:51:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.144+0000 7ff741202740 1 Processor -- start 2024-05-27T07:51:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.145+0000 7ff741202740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc58c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.145+0000 7ff741202740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc59190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.145+0000 7ff741202740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc64ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.145+0000 7ff741202740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dbfd895790 con 0x55dbfd8832f0 2024-05-27T07:51:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.145+0000 7ff741202740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dbfdb57bc0 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.145+0000 7ff741202740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dbfdc5bb90 con 0x55dbfd88b4d0 2024-05-27T07:51:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.145+0000 7ff741202740 10 client.0 _ll_register_callbacks cb 0x55dbfdb35400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.145+0000 7ff733400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc58c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.145+0000 7ff733400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc58c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60492/0 (socket says 172.21.15.63:60492) 2024-05-27T07:51:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.145+0000 7ff733400640 1 -- 172.21.15.63:0/638360038 learned_addr learned my addr 172.21.15.63:0/638360038 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.146+0000 7ff72aa00640 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc59190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.146+0000 7ff733e00640 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc64ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.146+0000 7ff733400640 1 -- 172.21.15.63:0/638360038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 msgr2=0x55dbfdc59190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.146+0000 7ff733400640 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc59190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.146+0000 7ff733400640 1 -- 172.21.15.63:0/638360038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8ac1b0 msgr2=0x55dbfdc64ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.146+0000 7ff733400640 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc64ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.146+0000 7ff733400640 1 -- 172.21.15.63:0/638360038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbfd8c4210 con 0x55dbfd8832f0 2024-05-27T07:51:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.146+0000 7ff733400640 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc58c50 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7ff72402a910 tx=0x7ff724009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8895b13694f674e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.146+0000 7ff720c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.147+0000 7ff720c00640 1 -- 172.21.15.63:0/638360038 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72404c030 con 0x55dbfd8832f0 2024-05-27T07:51:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.147+0000 7ff720c00640 1 -- 172.21.15.63:0/638360038 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7ff724003b40 con 0x55dbfd8832f0 2024-05-27T07:51:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.147+0000 7ff741202740 1 -- 172.21.15.63:0/638360038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff72c010e50 con 0x55dbfd8832f0 2024-05-27T07:51:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.147+0000 7ff720c00640 1 -- 172.21.15.63:0/638360038 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff724059050 con 0x55dbfd8832f0 2024-05-27T07:51:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.147+0000 7ff741202740 10 client.18840 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.147+0000 7ff741202740 1 -- 172.21.15.63:0/638360038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dbfd990620 con 0x55dbfd8832f0 2024-05-27T07:51:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.148+0000 7ff720c00640 1 -- 172.21.15.63:0/638360038 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff72c010e50 con 0x55dbfd8832f0 2024-05-27T07:51:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.148+0000 7ff720c00640 1 -- 172.21.15.63:0/638360038 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff72403c440 con 0x55dbfd8832f0 2024-05-27T07:51:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.148+0000 7ff741202740 10 client.18840 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.148+0000 7ff741202740 10 client.18840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.148+0000 7ff741202740 20 client.18840 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.148+0000 7ff741202740 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dbfd86ad80 0x55dbfdc6aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.148+0000 7ff741202740 4 client.18840 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.148+0000 7ff741202740 1 -- 172.21.15.63:0/638360038 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbfdbc1ae0 con 0x55dbfd86ad80 2024-05-27T07:51:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.149+0000 7ff72aa00640 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dbfd86ad80 0x55dbfdc6aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.150+0000 7ff72aa00640 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dbfd86ad80 0x55dbfdc6aee0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d4b600069ec5e1f server_cookie=64e6b19eb795230f in_seq=0 out_seq=0 2024-05-27T07:51:19.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.150+0000 7ff720c00640 10 client.18840 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff720c00640 1 -- 172.21.15.63:0/638360038 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dbfdbc1ae0 con 0x55dbfd86ad80 2024-05-27T07:51:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff720c00640 10 client.18840 handle_command_reply: tid=0 2024-05-27T07:51:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 client.18840 shutdown 2024-05-27T07:51:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 -- 172.21.15.63:0/638360038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 msgr2=0x55dbfdc58c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc58c50 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7ff72402a910 tx=0x7ff724009260 comp rx=0 tx=0).stop 2024-05-27T07:51:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 -- 172.21.15.63:0/638360038 shutdown_connections 2024-05-27T07:51:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dbfd86ad80 0x55dbfdc6aee0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc64ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc59190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 --2- 172.21.15.63:0/638360038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc58c50 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 -- 172.21.15.63:0/638360038 >> 172.21.15.63:0/638360038 conn(0x55dbfd8a0300 msgr2=0x55dbfdc4e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff720c00640 0 client.18840 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 -- 172.21.15.63:0/638360038 shutdown_connections 2024-05-27T07:51:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 1 -- 172.21.15.63:0/638360038 wait complete. 2024-05-27T07:51:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.152+0000 7ff741202740 20 client.18840 trim_cache size 0 max 16384 2024-05-27T07:51:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.170+0000 7ff741202740 1 Processor -- start 2024-05-27T07:51:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.170+0000 7ff741202740 1 -- start start 2024-05-27T07:51:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.171+0000 7ff741202740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc5b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.171+0000 7ff741202740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc64c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.171+0000 7ff741202740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc65170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.171+0000 7ff741202740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b5af0 con 0x55dbfd8832f0 2024-05-27T07:51:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.171+0000 7ff741202740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b33c0 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.171+0000 7ff741202740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b1230 con 0x55dbfd88b4d0 2024-05-27T07:51:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.171+0000 7ff733400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc5b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.171+0000 7ff733400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc5b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60506/0 (socket says 172.21.15.63:60506) 2024-05-27T07:51:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.171+0000 7ff733400640 1 -- 172.21.15.63:0/4128768577 learned_addr learned my addr 172.21.15.63:0/4128768577 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.171+0000 7ff72aa00640 1 --2- 172.21.15.63:0/4128768577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc64c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.172+0000 7ff733e00640 1 --2- 172.21.15.63:0/4128768577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc65170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.172+0000 7ff733400640 1 -- 172.21.15.63:0/4128768577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 msgr2=0x55dbfdc64c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.172+0000 7ff733400640 1 --2- 172.21.15.63:0/4128768577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc64c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.172+0000 7ff733400640 1 -- 172.21.15.63:0/4128768577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8ac1b0 msgr2=0x55dbfdc65170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.172+0000 7ff733400640 1 --2- 172.21.15.63:0/4128768577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc65170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.172+0000 7ff733400640 1 -- 172.21.15.63:0/4128768577 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbfd86afb0 con 0x55dbfd8832f0 2024-05-27T07:51:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.172+0000 7ff72aa00640 1 --2- 172.21.15.63:0/4128768577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc64c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.172+0000 7ff733400640 1 --2- 172.21.15.63:0/4128768577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc5b380 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff72403c860 tx=0x7ff72402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95bdb12f5aac767 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.172+0000 7ff723400640 1 -- 172.21.15.63:0/4128768577 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724062070 con 0x55dbfd8832f0 2024-05-27T07:51:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.173+0000 7ff723400640 1 -- 172.21.15.63:0/4128768577 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff72403ce50 con 0x55dbfd8832f0 2024-05-27T07:51:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.173+0000 7ff723400640 1 -- 172.21.15.63:0/4128768577 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72404c040 con 0x55dbfd8832f0 2024-05-27T07:51:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.174+0000 7ff741202740 1 -- 172.21.15.63:0/4128768577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 msgr2=0x55dbfdc5b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.174+0000 7ff741202740 1 --2- 172.21.15.63:0/4128768577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc5b380 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff72403c860 tx=0x7ff72402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.174+0000 7ff741202740 1 -- 172.21.15.63:0/4128768577 shutdown_connections 2024-05-27T07:51:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.174+0000 7ff741202740 1 --2- 172.21.15.63:0/4128768577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc65170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.174+0000 7ff741202740 1 --2- 172.21.15.63:0/4128768577 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc64c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.174+0000 7ff741202740 1 --2- 172.21.15.63:0/4128768577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8832f0 0x55dbfdc5b380 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.174+0000 7ff741202740 1 -- 172.21.15.63:0/4128768577 >> 172.21.15.63:0/4128768577 conn(0x55dbfd8a0300 msgr2=0x55dbfdc58f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.174+0000 7ff741202740 1 -- 172.21.15.63:0/4128768577 shutdown_connections 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.174+0000 7ff741202740 1 -- 172.21.15.63:0/4128768577 wait complete. 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.174+0000 7ff741202740 1 -- start start 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.175+0000 7ff741202740 1 Processor -- start 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.175+0000 7ff741202740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 0x55dbfdc5de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.175+0000 7ff741202740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc5e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.175+0000 7ff741202740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc5e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.175+0000 7ff741202740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b1880 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.175+0000 7ff741202740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b2cf0 con 0x55dbfd8832f0 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.175+0000 7ff741202740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341b1f50 con 0x55dbfd88b4d0 2024-05-27T07:51:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.175+0000 7ff741202740 10 client.0 _ll_register_callbacks cb 0x7ff724005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.176+0000 7ff733e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc5e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.176+0000 7ff733e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc5e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60512/0 (socket says 172.21.15.63:60512) 2024-05-27T07:51:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.176+0000 7ff733e00640 1 -- 172.21.15.63:0/3968488111 learned_addr learned my addr 172.21.15.63:0/3968488111 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.176+0000 7ff733400640 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 0x55dbfdc5de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.176+0000 7ff72aa00640 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc5e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.176+0000 7ff733e00640 1 -- 172.21.15.63:0/3968488111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 msgr2=0x55dbfdc5e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.176+0000 7ff733e00640 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc5e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.176+0000 7ff733e00640 1 -- 172.21.15.63:0/3968488111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 msgr2=0x55dbfdc5de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.176+0000 7ff733e00640 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 0x55dbfdc5de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.176+0000 7ff733e00640 1 -- 172.21.15.63:0/3968488111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbfdcd17c0 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.177+0000 7ff733e00640 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc5e8e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x55dbfdc4bbd0 tx=0x7ff72c012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a1b06bbe04fd88 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.177+0000 7ff720c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.177+0000 7ff720c00640 1 -- 172.21.15.63:0/3968488111 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c02b060 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.177+0000 7ff741202740 1 -- 172.21.15.63:0/3968488111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff72403ce50 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.177+0000 7ff720c00640 1 -- 172.21.15.63:0/3968488111 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7ff72c012900 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.177+0000 7ff720c00640 1 -- 172.21.15.63:0/3968488111 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff72c032050 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.178+0000 7ff741202740 10 client.18846 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.178+0000 7ff741202740 1 -- 172.21.15.63:0/3968488111 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dbfdcd2a30 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.178+0000 7ff720c00640 1 -- 172.21.15.63:0/3968488111 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff72403ce50 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.178+0000 7ff720c00640 1 -- 172.21.15.63:0/3968488111 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff72c025070 con 0x55dbfd8ac1b0 2024-05-27T07:51:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.178+0000 7ff741202740 10 client.18846 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.178+0000 7ff741202740 10 client.18846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.178+0000 7ff741202740 20 client.18846 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.178+0000 7ff741202740 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dbfd8401c0 0x55dbfdcd4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.178+0000 7ff741202740 4 client.18846 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:51:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.178+0000 7ff741202740 1 -- 172.21.15.63:0/3968488111 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dbfdbbb2c0 con 0x55dbfd8401c0 2024-05-27T07:51:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.179+0000 7ff733400640 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dbfd8401c0 0x55dbfdcd4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.180+0000 7ff720c00640 10 client.18846 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.180+0000 7ff733400640 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dbfd8401c0 0x55dbfdcd4e00 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f3e9aa1514e5da server_cookie=ad398cc005abae60 in_seq=0 out_seq=0 2024-05-27T07:51:19.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:18 smithi086 ceph-mon[34687]: pgmap v559: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:51:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff720c00640 1 -- 172.21.15.63:0/3968488111 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dbfdbbb2c0 con 0x55dbfd8401c0 2024-05-27T07:51:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff720c00640 10 client.18846 handle_command_reply: tid=0 2024-05-27T07:51:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff741202740 1 client.18846 shutdown 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff741202740 1 -- 172.21.15.63:0/3968488111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8ac1b0 msgr2=0x55dbfdc5e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff741202740 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc5e8e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x55dbfdc4bbd0 tx=0x7ff72c012680 comp rx=0 tx=0).stop 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff741202740 1 -- 172.21.15.63:0/3968488111 shutdown_connections 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff741202740 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dbfd8401c0 0x55dbfdcd4e00 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff741202740 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbfd8ac1b0 0x55dbfdc5e8e0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff741202740 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dbfd88b4d0 0x55dbfdc5e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff741202740 1 --2- 172.21.15.63:0/3968488111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dbfd8832f0 0x55dbfdc5de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff741202740 1 -- 172.21.15.63:0/3968488111 >> 172.21.15.63:0/3968488111 conn(0x55dbfd8a0300 msgr2=0x55dbfdc53410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.209+0000 7ff720c00640 0 client.18846 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.210+0000 7ff741202740 1 -- 172.21.15.63:0/3968488111 shutdown_connections 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.210+0000 7ff741202740 1 -- 172.21.15.63:0/3968488111 wait complete. 2024-05-27T07:51:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.210+0000 7ff741202740 20 client.18846 trim_cache size 0 max 16384 2024-05-27T07:51:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:51:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:51:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 -- 172.21.15.63:0/55981592 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7180b29d0 msgr2=0x7ff7180b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7180b29d0 0x7ff7180b4ee0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55dbfdc4c1a0 tx=0x7ff724009660 comp rx=0 tx=0).stop 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 -- 172.21.15.63:0/55981592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734109ab0 msgr2=0x7ff73419db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734109ab0 0x7ff73419db50 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00c970 tx=0x7ff71c00ce30 comp rx=0 tx=0).stop 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 -- 172.21.15.63:0/55981592 shutdown_connections 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff7180b29d0 0x7ff7180b4ee0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff734109f50 0x7ff7341a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff734109ab0 0x7ff73419db50 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 --2- 172.21.15.63:0/55981592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff734104250 0x7ff73419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 -- 172.21.15.63:0/55981592 >> 172.21.15.63:0/55981592 conn(0x7ff734079240 msgr2=0x7ff734102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 -- 172.21.15.63:0/55981592 shutdown_connections 2024-05-27T07:51:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:19.211+0000 7ff738c00640 1 -- 172.21.15.63:0/55981592 wait complete. 2024-05-27T07:51:19.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:18 smithi149 ceph-mon[35408]: pgmap v559: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:51:19.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:18 smithi063 ceph-mon[31613]: pgmap v559: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:51:21.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:20 smithi086 ceph-mon[34687]: pgmap v560: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:51:21.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:20 smithi149 ceph-mon[35408]: pgmap v560: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:51:21.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:20 smithi063 ceph-mon[31613]: pgmap v560: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T07:51:23.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:22 smithi086 ceph-mon[34687]: pgmap v561: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:51:23.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:22 smithi149 ceph-mon[35408]: pgmap v561: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:51:23.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:22 smithi063 ceph-mon[31613]: pgmap v561: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:51:25.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:24 smithi086 ceph-mon[34687]: pgmap v562: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:51:25.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:24 smithi149 ceph-mon[35408]: pgmap v562: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:51:25.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:24 smithi063 ceph-mon[31613]: pgmap v562: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T07:51:27.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:26 smithi086 ceph-mon[34687]: pgmap v563: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:51:27.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:26 smithi149 ceph-mon[35408]: pgmap v563: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:51:27.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:26 smithi063 ceph-mon[31613]: pgmap v563: 129 pgs: 129 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:51:29.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:28 smithi086 ceph-mon[34687]: pgmap v564: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:51:29.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:28 smithi149 ceph-mon[35408]: pgmap v564: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:51:29.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:28 smithi063 ceph-mon[31613]: pgmap v564: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:51:31.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:30 smithi086 ceph-mon[34687]: pgmap v565: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:51:31.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:30 smithi149 ceph-mon[35408]: pgmap v565: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:51:31.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:30 smithi063 ceph-mon[31613]: pgmap v565: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:51:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.472+0000 7f4fc7200640 1 -- 172.21.15.63:0/3160105357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fc810bcd0 msgr2=0x7f4fc8105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.472+0000 7f4fc7200640 1 --2- 172.21.15.63:0/3160105357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fc810bcd0 0x7f4fc8105870 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009900 tx=0x7f4fb802a410 comp rx=0 tx=0).stop 2024-05-27T07:51:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.473+0000 7f4fc7200640 1 -- 172.21.15.63:0/3160105357 shutdown_connections 2024-05-27T07:51:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.473+0000 7f4fc7200640 1 --2- 172.21.15.63:0/3160105357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fc8106420 0x7f4fc81127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.473+0000 7f4fc7200640 1 --2- 172.21.15.63:0/3160105357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc810c170 0x7f4fc8105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.473+0000 7f4fc7200640 1 --2- 172.21.15.63:0/3160105357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fc810bcd0 0x7f4fc8105870 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.473+0000 7f4fc7200640 1 -- 172.21.15.63:0/3160105357 >> 172.21.15.63:0/3160105357 conn(0x7f4fc81013d0 msgr2=0x7f4fc8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.473+0000 7f4fc7200640 1 -- 172.21.15.63:0/3160105357 shutdown_connections 2024-05-27T07:51:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.473+0000 7f4fc7200640 1 -- 172.21.15.63:0/3160105357 wait complete. 2024-05-27T07:51:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.474+0000 7f4fc7200640 1 Processor -- start 2024-05-27T07:51:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.474+0000 7f4fc7200640 1 -- start start 2024-05-27T07:51:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.475+0000 7f4fc7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc8106420 0x7f4fc81a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.475+0000 7f4fc7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fc810bcd0 0x7f4fc81a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.475+0000 7f4fc7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fc810c170 0x7f4fc819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.475+0000 7f4fc7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8115dc0 con 0x7f4fc810bcd0 2024-05-27T07:51:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.475+0000 7f4fc7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8115c40 con 0x7f4fc810c170 2024-05-27T07:51:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.475+0000 7f4fc7200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8115f40 con 0x7f4fc8106420 2024-05-27T07:51:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.476+0000 7f4fc5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fc810bcd0 0x7f4fc81a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.476+0000 7f4fc5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fc810bcd0 0x7f4fc81a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37064/0 (socket says 172.21.15.63:37064) 2024-05-27T07:51:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.476+0000 7f4fc5400640 1 -- 172.21.15.63:0/813835388 learned_addr learned my addr 172.21.15.63:0/813835388 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.476+0000 7f4fc5e00640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc8106420 0x7f4fc81a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.476+0000 7f4fc5400640 1 -- 172.21.15.63:0/813835388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc8106420 msgr2=0x7f4fc81a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.476+0000 7f4fc5400640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc8106420 0x7f4fc81a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.476+0000 7f4fc5400640 1 -- 172.21.15.63:0/813835388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fc810c170 msgr2=0x7f4fc819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.476+0000 7f4fc5400640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fc810c170 0x7f4fc819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.476+0000 7f4fc5400640 1 -- 172.21.15.63:0/813835388 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc819cdd0 con 0x7f4fc810bcd0 2024-05-27T07:51:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.477+0000 7f4fc5400640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fc810bcd0 0x7f4fc81a2a60 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400e9f0 tx=0x7f4fb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.477+0000 7f4fbea00640 1 -- 172.21.15.63:0/813835388 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400ccc0 con 0x7f4fc810bcd0 2024-05-27T07:51:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.477+0000 7f4fc7200640 1 -- 172.21.15.63:0/813835388 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc819d060 con 0x7f4fc810bcd0 2024-05-27T07:51:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.477+0000 7f4fc7200640 1 -- 172.21.15.63:0/813835388 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc81a92c0 con 0x7f4fc810bcd0 2024-05-27T07:51:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.477+0000 7f4fbea00640 1 -- 172.21.15.63:0/813835388 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fb40035f0 con 0x7f4fc810bcd0 2024-05-27T07:51:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.477+0000 7f4fbea00640 1 -- 172.21.15.63:0/813835388 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400ce60 con 0x7f4fc810bcd0 2024-05-27T07:51:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.482+0000 7f4fd3e4e740 1 Processor -- start 2024-05-27T07:51:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.482+0000 7f4fd3e4e740 1 -- start start 2024-05-27T07:51:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.483+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9206b1b0 0x55ac9240da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.483+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac9240dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.483+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac9240e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.483+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ac9237eb90 con 0x55ac9204a4d0 2024-05-27T07:51:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.483+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac9236e730 con 0x55ac920422f0 2024-05-27T07:51:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.483+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ac92372770 con 0x55ac9206b1b0 2024-05-27T07:51:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.483+0000 7f4fc6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9206b1b0 0x55ac9240da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.483+0000 7f4fc5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac9240dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.484+0000 7f4fc6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9206b1b0 0x55ac9240da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:49226/0 (socket says 172.21.15.63:49226) 2024-05-27T07:51:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.484+0000 7f4fc6800640 1 -- 172.21.15.63:0/2752272717 learned_addr learned my addr 172.21.15.63:0/2752272717 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.484+0000 7f4fc5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac9240dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:32978/0 (socket says 172.21.15.63:32978) 2024-05-27T07:51:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.484+0000 7f4fbea00640 1 -- 172.21.15.63:0/813835388 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4fb4010690 con 0x7f4fc810bcd0 2024-05-27T07:51:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.484+0000 7f4fc5400640 1 --2- 172.21.15.63:0/2752272717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac9240e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.484+0000 7f4fc6800640 1 -- 172.21.15.63:0/2752272717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 msgr2=0x55ac9240dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.484+0000 7f4fc6800640 1 --2- 172.21.15.63:0/2752272717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac9240dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.484+0000 7f4fc6800640 1 -- 172.21.15.63:0/2752272717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 msgr2=0x55ac9240e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.485+0000 7f4fc6800640 1 --2- 172.21.15.63:0/2752272717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac9240e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.485+0000 7f4fc6800640 1 -- 172.21.15.63:0/2752272717 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac923078b0 con 0x55ac9206b1b0 2024-05-27T07:51:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.485+0000 7f4fc6800640 1 --2- 172.21.15.63:0/2752272717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9206b1b0 0x55ac9240da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009de0 tx=0x7f4fc000d6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ee14b93f6f5182 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.486+0000 7f4fb3400640 1 -- 172.21.15.63:0/2752272717 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0003b70 con 0x55ac9206b1b0 2024-05-27T07:51:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.486+0000 7f4fb3400640 1 -- 172.21.15.63:0/2752272717 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fc0003d10 con 0x55ac9206b1b0 2024-05-27T07:51:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.486+0000 7f4fb3400640 1 -- 172.21.15.63:0/2752272717 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0010820 con 0x55ac9206b1b0 2024-05-27T07:51:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.486+0000 7f4fbea00640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4fac0b2950 0x7f4fac0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.487+0000 7f4fbea00640 1 -- 172.21.15.63:0/813835388 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f4fb40e2d70 con 0x7f4fc810bcd0 2024-05-27T07:51:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.487+0000 7f4fc5e00640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4fac0b2950 0x7f4fac0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.488+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2752272717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9206b1b0 msgr2=0x55ac9240da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.488+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2752272717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9206b1b0 0x55ac9240da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009de0 tx=0x7f4fc000d6b0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.488+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2752272717 shutdown_connections 2024-05-27T07:51:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.488+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2752272717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac9240e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.488+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2752272717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac9240dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.488+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2752272717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9206b1b0 0x55ac9240da60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.488+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2752272717 >> 172.21.15.63:0/2752272717 conn(0x55ac9205f300 msgr2=0x55ac9240a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.488+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2752272717 shutdown_connections 2024-05-27T07:51:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.488+0000 7f4fc5e00640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4fac0b2950 0x7f4fac0b4e60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80057c0 tx=0x7f4fb8000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.488+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2752272717 wait complete. 2024-05-27T07:51:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.489+0000 7f4fd3e4e740 1 -- start start 2024-05-27T07:51:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.489+0000 7f4fd3e4e740 1 Processor -- start 2024-05-27T07:51:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac92417c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9204a4d0 0x55ac924181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9206b1b0 0x55ac92423ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ac92054790 con 0x55ac9206b1b0 2024-05-27T07:51:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac92316bc0 con 0x55ac920422f0 2024-05-27T07:51:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ac9241ab90 con 0x55ac9204a4d0 2024-05-27T07:51:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fd3e4e740 10 client.0 _ll_register_callbacks cb 0x55ac922f4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fc6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9206b1b0 0x55ac92423ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fc5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac92417c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fc6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9206b1b0 0x55ac92423ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37082/0 (socket says 172.21.15.63:37082) 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.490+0000 7f4fc6800640 1 -- 172.21.15.63:0/2308465938 learned_addr learned my addr 172.21.15.63:0/2308465938 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.491+0000 7f4fc5400640 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9204a4d0 0x55ac924181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.491+0000 7f4fc6800640 1 -- 172.21.15.63:0/2308465938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9204a4d0 msgr2=0x55ac924181a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.491+0000 7f4fc6800640 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9204a4d0 0x55ac924181a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.491+0000 7f4fc6800640 1 -- 172.21.15.63:0/2308465938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 msgr2=0x55ac92417c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.491+0000 7f4fc6800640 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac92417c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.491+0000 7f4fc6800640 1 -- 172.21.15.63:0/2308465938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac92083210 con 0x55ac9206b1b0 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.491+0000 7f4fc5e00640 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac92417c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.492+0000 7f4fc6800640 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9206b1b0 0x55ac92423ec0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006010 tx=0x7f4fc0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795b5d05d504ba76 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.492+0000 7f4fa7e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.492+0000 7f4fa7e00640 1 -- 172.21.15.63:0/2308465938 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0004430 con 0x55ac9206b1b0 2024-05-27T07:51:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.492+0000 7f4fa7e00640 1 -- 172.21.15.63:0/2308465938 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f4fc0004b20 con 0x55ac9206b1b0 2024-05-27T07:51:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.493+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2308465938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc0003d10 con 0x55ac9206b1b0 2024-05-27T07:51:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.493+0000 7f4fa7e00640 1 -- 172.21.15.63:0/2308465938 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4fc0004720 con 0x55ac9206b1b0 2024-05-27T07:51:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.493+0000 7f4fd3e4e740 10 client.18858 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.493+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2308465938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ac9214f620 con 0x55ac9206b1b0 2024-05-27T07:51:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.494+0000 7f4fa7e00640 1 -- 172.21.15.63:0/2308465938 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fc0003d10 con 0x55ac9206b1b0 2024-05-27T07:51:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.494+0000 7f4fa7e00640 1 -- 172.21.15.63:0/2308465938 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4fc0022030 con 0x55ac9206b1b0 2024-05-27T07:51:32.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.494+0000 7f4fd3e4e740 10 client.18858 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.494+0000 7f4fd3e4e740 10 client.18858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.494+0000 7f4fd3e4e740 20 client.18858 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.494+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac92029d80 0x55ac92429e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.494+0000 7f4fd3e4e740 4 client.18858 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.494+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2308465938 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac92380ae0 con 0x55ac92029d80 2024-05-27T07:51:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.495+0000 7f4fc5e00640 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac92029d80 0x55ac92429e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.496+0000 7f4fc5e00640 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac92029d80 0x55ac92429e10 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39fe953d89f65b10 server_cookie=a35bbde52ab3c7f3 in_seq=0 out_seq=0 2024-05-27T07:51:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.496+0000 7f4fa7e00640 10 client.18858 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fa7e00640 1 -- 172.21.15.63:0/2308465938 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ac92380ae0 con 0x55ac92029d80 2024-05-27T07:51:32.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fa7e00640 10 client.18858 handle_command_reply: tid=0 2024-05-27T07:51:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 client.18858 shutdown 2024-05-27T07:51:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2308465938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9206b1b0 msgr2=0x55ac92423ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9206b1b0 0x55ac92423ec0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006010 tx=0x7f4fc0003c20 comp rx=0 tx=0).stop 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2308465938 shutdown_connections 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac92029d80 0x55ac92429e10 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9206b1b0 0x55ac92423ec0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9204a4d0 0x55ac924181a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2308465938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac92417c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2308465938 >> 172.21.15.63:0/2308465938 conn(0x55ac9205f300 msgr2=0x55ac9240d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fa7e00640 0 client.18858 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2308465938 shutdown_connections 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.498+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2308465938 wait complete. 2024-05-27T07:51:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.499+0000 7f4fd3e4e740 20 client.18858 trim_cache size 0 max 16384 2024-05-27T07:51:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.523+0000 7f4fd3e4e740 1 Processor -- start 2024-05-27T07:51:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.524+0000 7f4fd3e4e740 1 -- start start 2024-05-27T07:51:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.524+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 0x55ac92413440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.524+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac92423c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.524+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac92424170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.524+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa80127e0 con 0x55ac9204a4d0 2024-05-27T07:51:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.524+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa80100b0 con 0x55ac920422f0 2024-05-27T07:51:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.524+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa800df20 con 0x55ac9205f300 2024-05-27T07:51:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.525+0000 7f4fc6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac92424170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.525+0000 7f4fc6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac92424170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37094/0 (socket says 172.21.15.63:37094) 2024-05-27T07:51:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.525+0000 7f4fc6800640 1 -- 172.21.15.63:0/2265906775 learned_addr learned my addr 172.21.15.63:0/2265906775 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.525+0000 7f4fc5400640 1 --2- 172.21.15.63:0/2265906775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac92423c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.525+0000 7f4fc5e00640 1 --2- 172.21.15.63:0/2265906775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 0x55ac92413440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.525+0000 7f4fc6800640 1 -- 172.21.15.63:0/2265906775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 msgr2=0x55ac92413440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.525+0000 7f4fc6800640 1 --2- 172.21.15.63:0/2265906775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 0x55ac92413440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.525+0000 7f4fc6800640 1 -- 172.21.15.63:0/2265906775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 msgr2=0x55ac92423c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.525+0000 7f4fc6800640 1 --2- 172.21.15.63:0/2265906775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac92423c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.525+0000 7f4fc6800640 1 -- 172.21.15.63:0/2265906775 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac92029fb0 con 0x55ac9204a4d0 2024-05-27T07:51:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.526+0000 7f4fc6800640 1 --2- 172.21.15.63:0/2265906775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac92424170 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0000f80 tx=0x7f4fc0009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135f67b2eba4e844 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.526+0000 7f4fb2a00640 1 -- 172.21.15.63:0/2265906775 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00099e0 con 0x55ac9204a4d0 2024-05-27T07:51:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.526+0000 7f4fb2a00640 1 -- 172.21.15.63:0/2265906775 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fc0003c90 con 0x55ac9204a4d0 2024-05-27T07:51:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.526+0000 7f4fb2a00640 1 -- 172.21.15.63:0/2265906775 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00044a0 con 0x55ac9204a4d0 2024-05-27T07:51:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.527+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2265906775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 msgr2=0x55ac92424170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.527+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2265906775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac92424170 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0000f80 tx=0x7f4fc0009c60 comp rx=0 tx=0).stop 2024-05-27T07:51:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.527+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2265906775 shutdown_connections 2024-05-27T07:51:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.527+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2265906775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac92424170 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.527+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2265906775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac92423c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.527+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/2265906775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 0x55ac92413440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.527+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2265906775 >> 172.21.15.63:0/2265906775 conn(0x55ac92411fc0 msgr2=0x55ac92419db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.528+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2265906775 shutdown_connections 2024-05-27T07:51:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.528+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/2265906775 wait complete. 2024-05-27T07:51:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.528+0000 7f4fd3e4e740 1 -- start start 2024-05-27T07:51:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.528+0000 7f4fd3e4e740 1 Processor -- start 2024-05-27T07:51:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.529+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac9241cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.529+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac9241d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.529+0000 7f4fd3e4e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 0x55ac9241d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.529+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa800e570 con 0x55ac9204a4d0 2024-05-27T07:51:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.529+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa800f9e0 con 0x55ac920422f0 2024-05-27T07:51:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.529+0000 7f4fd3e4e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa800ec40 con 0x55ac9205f300 2024-05-27T07:51:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.529+0000 7f4fd3e4e740 10 client.0 _ll_register_callbacks cb 0x7f4fb4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.529+0000 7f4fc5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac9241d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.529+0000 7f4fc5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac9241d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37108/0 (socket says 172.21.15.63:37108) 2024-05-27T07:51:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.529+0000 7f4fc5400640 1 -- 172.21.15.63:0/545786686 learned_addr learned my addr 172.21.15.63:0/545786686 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fc6800640 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 0x55ac9241d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fc5400640 1 -- 172.21.15.63:0/545786686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 msgr2=0x55ac9241d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fc5e00640 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac9241cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fc5400640 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 0x55ac9241d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fc5400640 1 -- 172.21.15.63:0/545786686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 msgr2=0x55ac9241cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fc5400640 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac9241cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fc5400640 1 -- 172.21.15.63:0/545786686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac92490400 con 0x55ac9204a4d0 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fc6800640 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 0x55ac9241d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fc5400640 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac9241d020 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40a9c20 tx=0x7f4fb40aa1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169b4ca8811ebb0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fa7e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fa7e00640 1 -- 172.21.15.63:0/545786686 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40b2430 con 0x55ac9204a4d0 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.530+0000 7f4fa7e00640 1 -- 172.21.15.63:0/545786686 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f4fb40a9100 con 0x55ac9204a4d0 2024-05-27T07:51:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/545786686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc0003c90 con 0x55ac9204a4d0 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fa7e00640 1 -- 172.21.15.63:0/545786686 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4fb406e050 con 0x55ac9204a4d0 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fd3e4e740 10 client.18864 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/545786686 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ac924914f0 con 0x55ac9204a4d0 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fa7e00640 1 -- 172.21.15.63:0/545786686 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4fc0003c90 con 0x55ac9204a4d0 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fa7e00640 1 -- 172.21.15.63:0/545786686 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4fb40a8960 con 0x55ac9204a4d0 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fd3e4e740 10 client.18864 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fd3e4e740 10 client.18864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fd3e4e740 20 client.18864 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac9206b1b0 0x55ac92493cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fd3e4e740 4 client.18864 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:51:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.531+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/545786686 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac9237a2c0 con 0x55ac9206b1b0 2024-05-27T07:51:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.532+0000 7f4fc5e00640 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac9206b1b0 0x55ac92493cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.533+0000 7f4fa7e00640 10 client.18864 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.533+0000 7f4fc5e00640 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac9206b1b0 0x55ac92493cb0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc043d5284ca9fe server_cookie=6bfbdf5e9a1da3d3 in_seq=0 out_seq=0 2024-05-27T07:51:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.559+0000 7f4fa7e00640 1 -- 172.21.15.63:0/545786686 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ac9237a2c0 con 0x55ac9206b1b0 2024-05-27T07:51:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.559+0000 7f4fa7e00640 10 client.18864 handle_command_reply: tid=0 2024-05-27T07:51:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.559+0000 7f4fd3e4e740 1 client.18864 shutdown 2024-05-27T07:51:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.559+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/545786686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 msgr2=0x55ac9241d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.559+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac9241d020 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40a9c20 tx=0x7f4fb40aa1f0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.560+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/545786686 shutdown_connections 2024-05-27T07:51:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.560+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac9206b1b0 0x55ac92493cb0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.560+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac9205f300 0x55ac9241d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.560+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac9204a4d0 0x55ac9241d020 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.560+0000 7f4fd3e4e740 1 --2- 172.21.15.63:0/545786686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac920422f0 0x55ac9241cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.560+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/545786686 >> 172.21.15.63:0/545786686 conn(0x55ac92411fc0 msgr2=0x55ac924123d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.560+0000 7f4fa7e00640 0 client.18864 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.560+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/545786686 shutdown_connections 2024-05-27T07:51:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.560+0000 7f4fd3e4e740 1 -- 172.21.15.63:0/545786686 wait complete. 2024-05-27T07:51:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.560+0000 7f4fd3e4e740 20 client.18864 trim_cache size 0 max 16384 2024-05-27T07:51:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:51:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:51:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:51:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.561+0000 7f4fc7200640 1 -- 172.21.15.63:0/813835388 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4fac0b2950 msgr2=0x7f4fac0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.561+0000 7f4fc7200640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4fac0b2950 0x7f4fac0b4e60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80057c0 tx=0x7f4fb8000bc0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.561+0000 7f4fc7200640 1 -- 172.21.15.63:0/813835388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fc810bcd0 msgr2=0x7f4fc81a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.562+0000 7f4fc7200640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fc810bcd0 0x7f4fc81a2a60 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400e9f0 tx=0x7f4fb400eeb0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.562+0000 7f4fc7200640 1 -- 172.21.15.63:0/813835388 shutdown_connections 2024-05-27T07:51:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.562+0000 7f4fc7200640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4fac0b2950 0x7f4fac0b4e60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.562+0000 7f4fc7200640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fc810c170 0x7f4fc819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.562+0000 7f4fc7200640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fc810bcd0 0x7f4fc81a2a60 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.562+0000 7f4fc7200640 1 --2- 172.21.15.63:0/813835388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fc8106420 0x7f4fc81a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.562+0000 7f4fc7200640 1 -- 172.21.15.63:0/813835388 >> 172.21.15.63:0/813835388 conn(0x7f4fc81013d0 msgr2=0x7f4fc81017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.562+0000 7f4fc7200640 1 -- 172.21.15.63:0/813835388 shutdown_connections 2024-05-27T07:51:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:32.562+0000 7f4fc7200640 1 -- 172.21.15.63:0/813835388 wait complete. 2024-05-27T07:51:33.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:32 smithi086 ceph-mon[34687]: pgmap v566: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:51:33.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:32 smithi149 ceph-mon[35408]: pgmap v566: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:51:33.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:32 smithi063 ceph-mon[31613]: pgmap v566: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T07:51:34.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:33 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:51:34.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:33 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:51:34.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:33 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:51:35.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:34 smithi086 ceph-mon[34687]: pgmap v567: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.3 MiB/s wr, 7 op/s 2024-05-27T07:51:35.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:34 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:51:35.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:34 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:51:35.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:34 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:51:35.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:34 smithi149 ceph-mon[35408]: pgmap v567: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.3 MiB/s wr, 7 op/s 2024-05-27T07:51:35.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:34 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:51:35.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:34 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:51:35.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:34 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:51:35.464 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:51:35.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:34 smithi063 ceph-mon[31613]: pgmap v567: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.3 MiB/s wr, 7 op/s 2024-05-27T07:51:35.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:34 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:51:35.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:34 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:51:35.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:34 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:51:35.963 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:37.070 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:36 smithi063 ceph-mon[31613]: pgmap v568: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.3 MiB/s wr, 7 op/s 2024-05-27T07:51:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:36 smithi086 ceph-mon[34687]: pgmap v568: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.3 MiB/s wr, 7 op/s 2024-05-27T07:51:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:36 smithi149 ceph-mon[35408]: pgmap v568: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.3 MiB/s wr, 7 op/s 2024-05-27T07:51:39.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.057+0000 7f0a7be00700 1 -- 172.21.15.63:0/3518592031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a74110fe0 msgr2=0x7f0a74113400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:39.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.057+0000 7f0a7be00700 1 --2- 172.21.15.63:0/3518592031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a74110fe0 0x7f0a74113400 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00b0a0 tx=0x7f0a6c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.058+0000 7f0a7be00700 1 -- 172.21.15.63:0/3518592031 shutdown_connections 2024-05-27T07:51:39.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.058+0000 7f0a7be00700 1 --2- 172.21.15.63:0/3518592031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a74110fe0 0x7f0a74113400 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.058+0000 7f0a7be00700 1 --2- 172.21.15.63:0/3518592031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a741053e0 0x7f0a74102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.058+0000 7f0a7be00700 1 --2- 172.21.15.63:0/3518592031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a74102da0 0x7f0a741031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.058+0000 7f0a7be00700 1 -- 172.21.15.63:0/3518592031 >> 172.21.15.63:0/3518592031 conn(0x7f0a740fe510 msgr2=0x7f0a74100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:39.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.058+0000 7f0a7be00700 1 -- 172.21.15.63:0/3518592031 shutdown_connections 2024-05-27T07:51:39.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.058+0000 7f0a7be00700 1 -- 172.21.15.63:0/3518592031 wait complete. 2024-05-27T07:51:39.061 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:38 smithi063 ceph-mon[31613]: pgmap v569: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:51:39.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.059+0000 7f0a7be00700 1 Processor -- start 2024-05-27T07:51:39.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.060+0000 7f0a7be00700 1 -- start start 2024-05-27T07:51:39.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.060+0000 7f0a7be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a74102da0 0x7f0a74105c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.060+0000 7f0a7be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a741053e0 0x7f0a74106190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.060+0000 7f0a7be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a74110fe0 0x7f0a741066d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.060+0000 7f0a7be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a74116790 con 0x7f0a74110fe0 2024-05-27T07:51:39.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.060+0000 7f0a7be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a74116610 con 0x7f0a741053e0 2024-05-27T07:51:39.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.060+0000 7f0a7be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0a74116910 con 0x7f0a74102da0 2024-05-27T07:51:39.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.061+0000 7f0a79e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a74110fe0 0x7f0a741066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.061+0000 7f0a79e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a74110fe0 0x7f0a741066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41466/0 (socket says 172.21.15.63:41466) 2024-05-27T07:51:39.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.061+0000 7f0a79e00700 1 -- 172.21.15.63:0/4176501622 learned_addr learned my addr 172.21.15.63:0/4176501622 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:39.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.061+0000 7f0a78a00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a741053e0 0x7f0a74106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.061+0000 7f0a79400700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a74102da0 0x7f0a74105c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.061+0000 7f0a79e00700 1 -- 172.21.15.63:0/4176501622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a74102da0 msgr2=0x7f0a74105c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.061+0000 7f0a79e00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a74102da0 0x7f0a74105c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.061+0000 7f0a79e00700 1 -- 172.21.15.63:0/4176501622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a741053e0 msgr2=0x7f0a74106190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.061+0000 7f0a79e00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a741053e0 0x7f0a74106190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.061+0000 7f0a79e00700 1 -- 172.21.15.63:0/4176501622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a741ad6e0 con 0x7f0a74110fe0 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.062+0000 7f0a79e00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a74110fe0 0x7f0a741066d0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00b070 tx=0x7f0a6c004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.062+0000 7f0a72000700 1 -- 172.21.15.63:0/4176501622 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c021070 con 0x7f0a74110fe0 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.062+0000 7f0a72000700 1 -- 172.21.15.63:0/4176501622 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a6c007b80 con 0x7f0a74110fe0 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.062+0000 7f0a72000700 1 -- 172.21.15.63:0/4176501622 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0045f0 con 0x7f0a74110fe0 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.062+0000 7f0a7be00700 1 -- 172.21.15.63:0/4176501622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a741ad910 con 0x7f0a74110fe0 2024-05-27T07:51:39.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.062+0000 7f0a7be00700 1 -- 172.21.15.63:0/4176501622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a741addc0 con 0x7f0a74110fe0 2024-05-27T07:51:39.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.067+0000 7f0a72000700 1 -- 172.21.15.63:0/4176501622 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0a6c003a00 con 0x7f0a74110fe0 2024-05-27T07:51:39.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.068+0000 7f0a8a9e3b80 1 Processor -- start 2024-05-27T07:51:39.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.069+0000 7f0a72000700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0a5c0b2a70 0x7f0a5c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.069+0000 7f0a8a9e3b80 1 -- start start 2024-05-27T07:51:39.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.069+0000 7f0a72000700 1 -- 172.21.15.63:0/4176501622 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f0a6c00f020 con 0x7f0a74110fe0 2024-05-27T07:51:39.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.069+0000 7f0a78a00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0a5c0b2a70 0x7f0a5c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.069+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487268330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.069+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564870e9e40 0x556487268870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.069+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x556487275110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.069+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556487196e40 con 0x5564871d59f0 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.069+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564871869e0 con 0x5564870e9e40 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.069+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55648718aa20 con 0x556487268e70 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.070+0000 7f0a79400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487268330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.070+0000 7f0a79400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487268330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41468/0 (socket says 172.21.15.63:41468) 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.070+0000 7f0a79400700 1 -- 172.21.15.63:0/148046823 learned_addr learned my addr 172.21.15.63:0/148046823 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.070+0000 7f0a79e00700 1 --2- 172.21.15.63:0/148046823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x556487275110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.070+0000 7f0a78a00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0a5c0b2a70 0x7f0a5c0b4f80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a74112eb0 tx=0x7f0a60009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.070+0000 7f0a78a00700 1 --2- 172.21.15.63:0/148046823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564870e9e40 0x556487268870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.071+0000 7f0a79400700 1 -- 172.21.15.63:0/148046823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 msgr2=0x556487275110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.071+0000 7f0a79400700 1 --2- 172.21.15.63:0/148046823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x556487275110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.071+0000 7f0a79400700 1 -- 172.21.15.63:0/148046823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564870e9e40 msgr2=0x556487268870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:39.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.071+0000 7f0a79400700 1 --2- 172.21.15.63:0/148046823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564870e9e40 0x556487268870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.071+0000 7f0a79400700 1 -- 172.21.15.63:0/148046823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648714f420 con 0x5564871d59f0 2024-05-27T07:51:39.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.071+0000 7f0a79400700 1 --2- 172.21.15.63:0/148046823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487268330 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009cf0 tx=0x7f0a680103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aa73b7d27d0273 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:39.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.071+0000 7f0a66a00700 1 -- 172.21.15.63:0/148046823 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68010d00 con 0x5564871d59f0 2024-05-27T07:51:39.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.071+0000 7f0a66a00700 1 -- 172.21.15.63:0/148046823 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a680118d0 con 0x5564871d59f0 2024-05-27T07:51:39.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.071+0000 7f0a66a00700 1 -- 172.21.15.63:0/148046823 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6801a760 con 0x5564871d59f0 2024-05-27T07:51:39.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.072+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/148046823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 msgr2=0x556487268330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:39.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.072+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/148046823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487268330 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009cf0 tx=0x7f0a680103d0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.073+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/148046823 shutdown_connections 2024-05-27T07:51:39.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.073+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/148046823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x556487275110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.073+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/148046823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487268330 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.073+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/148046823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564870e9e40 0x556487268870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.073+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/148046823 >> 172.21.15.63:0/148046823 conn(0x556487183390 msgr2=0x5564872644c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:39.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.073+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/148046823 shutdown_connections 2024-05-27T07:51:39.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.073+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/148046823 wait complete. 2024-05-27T07:51:39.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.074+0000 7f0a8a9e3b80 1 -- start start 2024-05-27T07:51:39.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.074+0000 7f0a8a9e3b80 1 Processor -- start 2024-05-27T07:51:39.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564870e9e40 0x5564872726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 0x556487272c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x556487273140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55648708cfa0 con 0x5564870e9e40 2024-05-27T07:51:39.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556487127760 con 0x5564871d59f0 2024-05-27T07:51:39.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556487275b80 con 0x556487268e70 2024-05-27T07:51:39.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a8a9e3b80 10 client.0 _ll_register_callbacks cb 0x556487130c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:39.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a79400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564870e9e40 0x5564872726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a79400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564870e9e40 0x5564872726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41470/0 (socket says 172.21.15.63:41470) 2024-05-27T07:51:39.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a79400700 1 -- 172.21.15.63:0/1893340593 learned_addr learned my addr 172.21.15.63:0/1893340593 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:39.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.075+0000 7f0a78a00700 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 0x556487272c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.076+0000 7f0a79400700 1 -- 172.21.15.63:0/1893340593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 msgr2=0x556487273140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:39.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.076+0000 7f0a79400700 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x556487273140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.076+0000 7f0a79400700 1 -- 172.21.15.63:0/1893340593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 msgr2=0x556487272c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:39.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.076+0000 7f0a79400700 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 0x556487272c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.076+0000 7f0a79400700 1 -- 172.21.15.63:0/1893340593 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564870d68d0 con 0x5564870e9e40 2024-05-27T07:51:39.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.076+0000 7f0a78a00700 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 0x556487272c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:39.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.076+0000 7f0a79400700 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564870e9e40 0x5564872726c0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009800 tx=0x7f0a680057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80fc515c66d0af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:39.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.076+0000 7f0a5b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:39.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.076+0000 7f0a5b400700 1 -- 172.21.15.63:0/1893340593 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68004450 con 0x5564870e9e40 2024-05-27T07:51:39.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.077+0000 7f0a5b400700 1 -- 172.21.15.63:0/1893340593 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f0a68005050 con 0x5564870e9e40 2024-05-27T07:51:39.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.077+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/1893340593 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a680118d0 con 0x5564870e9e40 2024-05-27T07:51:39.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.077+0000 7f0a5b400700 1 -- 172.21.15.63:0/1893340593 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0a6802f050 con 0x5564870e9e40 2024-05-27T07:51:39.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.077+0000 7f0a8a9e3b80 10 client.18876 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:39.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.077+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/1893340593 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556487157060 con 0x5564870e9e40 2024-05-27T07:51:39.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.077+0000 7f0a5b400700 1 -- 172.21.15.63:0/1893340593 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a680118d0 con 0x5564870e9e40 2024-05-27T07:51:39.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.078+0000 7f0a5b400700 1 -- 172.21.15.63:0/1893340593 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0a68024020 con 0x5564870e9e40 2024-05-27T07:51:39.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.078+0000 7f0a8a9e3b80 10 client.18876 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:39.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.078+0000 7f0a8a9e3b80 10 client.18876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:39.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.078+0000 7f0a8a9e3b80 20 client.18876 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:39.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.078+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556487282ae0 0x556487284f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.078+0000 7f0a8a9e3b80 4 client.18876 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:39.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.078+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/1893340593 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556487198d90 con 0x556487282ae0 2024-05-27T07:51:39.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.079+0000 7f0a78a00700 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556487282ae0 0x556487284f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.080+0000 7f0a78a00700 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556487282ae0 0x556487284f20 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0dfdde50f4aa9b server_cookie=628460aa37bc9c70 in_seq=0 out_seq=0 2024-05-27T07:51:39.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.080+0000 7f0a5b400700 10 client.18876 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:39.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a5b400700 1 -- 172.21.15.63:0/1893340593 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556487198d90 con 0x556487282ae0 2024-05-27T07:51:39.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a5b400700 10 client.18876 handle_command_reply: tid=0 2024-05-27T07:51:39.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a8a9e3b80 1 client.18876 shutdown 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/1893340593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564870e9e40 msgr2=0x5564872726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564870e9e40 0x5564872726c0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009800 tx=0x7f0a680057e0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/1893340593 shutdown_connections 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556487282ae0 0x556487284f20 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x556487273140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564870e9e40 0x5564872726c0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/1893340593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 0x556487272c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/1893340593 >> 172.21.15.63:0/1893340593 conn(0x556487183390 msgr2=0x556487268310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a5b400700 0 client.18876 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.082+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/1893340593 shutdown_connections 2024-05-27T07:51:39.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.083+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/1893340593 wait complete. 2024-05-27T07:51:39.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.083+0000 7f0a8a9e3b80 20 client.18876 trim_cache size 0 max 16384 2024-05-27T07:51:39.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.106+0000 7f0a8a9e3b80 1 Processor -- start 2024-05-27T07:51:39.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.106+0000 7f0a8a9e3b80 1 -- start start 2024-05-27T07:51:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564870e9e40 0x55648726de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 0x5564872716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556487268e70 0x55648728a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a741b92d0 con 0x556487268e70 2024-05-27T07:51:39.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a741b6b60 con 0x5564871d59f0 2024-05-27T07:51:39.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0a741b49d0 con 0x5564870e9e40 2024-05-27T07:51:39.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a79e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556487268e70 0x55648728a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a79e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556487268e70 0x55648728a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41478/0 (socket says 172.21.15.63:41478) 2024-05-27T07:51:39.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a79e00700 1 -- 172.21.15.63:0/3145248457 learned_addr learned my addr 172.21.15.63:0/3145248457 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:39.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a79400700 1 --2- 172.21.15.63:0/3145248457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564870e9e40 0x55648726de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.107+0000 7f0a78a00700 1 --2- 172.21.15.63:0/3145248457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 0x5564872716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.108+0000 7f0a79e00700 1 -- 172.21.15.63:0/3145248457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564870e9e40 msgr2=0x55648726de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:39.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.108+0000 7f0a79e00700 1 --2- 172.21.15.63:0/3145248457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564870e9e40 0x55648726de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.108+0000 7f0a79e00700 1 -- 172.21.15.63:0/3145248457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 msgr2=0x5564872716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:39.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.108+0000 7f0a79e00700 1 --2- 172.21.15.63:0/3145248457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 0x5564872716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.108+0000 7f0a79e00700 1 -- 172.21.15.63:0/3145248457 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556487260240 con 0x556487268e70 2024-05-27T07:51:39.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.108+0000 7f0a78a00700 1 --2- 172.21.15.63:0/3145248457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 0x5564872716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:39.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.108+0000 7f0a79e00700 1 --2- 172.21.15.63:0/3145248457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556487268e70 0x55648728a6e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0a7f30 tx=0x7f0a6c0aa710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90268d82e5915046 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:39.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.109+0000 7f0a66000700 1 -- 172.21.15.63:0/3145248457 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0075c0 con 0x556487268e70 2024-05-27T07:51:39.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.109+0000 7f0a66000700 1 -- 172.21.15.63:0/3145248457 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a6c0b33d0 con 0x556487268e70 2024-05-27T07:51:39.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.109+0000 7f0a66000700 1 -- 172.21.15.63:0/3145248457 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0af070 con 0x556487268e70 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.110+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3145248457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556487268e70 msgr2=0x55648728a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.110+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/3145248457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556487268e70 0x55648728a6e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0a7f30 tx=0x7f0a6c0aa710 comp rx=0 tx=0).stop 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.110+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3145248457 shutdown_connections 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.110+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/3145248457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556487268e70 0x55648728a6e0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.110+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/3145248457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564870e9e40 0x55648726de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.110+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/3145248457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564871d59f0 0x5564872716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.110+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3145248457 >> 172.21.15.63:0/3145248457 conn(0x556487183390 msgr2=0x556487270fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.110+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3145248457 shutdown_connections 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.110+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3145248457 wait complete. 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.111+0000 7f0a8a9e3b80 1 -- start start 2024-05-27T07:51:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.111+0000 7f0a8a9e3b80 1 Processor -- start 2024-05-27T07:51:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.111+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564870e9e40 0x5564872767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.111+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487276d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a8a9e3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x5564872e5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a741b5020 con 0x5564871d59f0 2024-05-27T07:51:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a741b6490 con 0x5564870e9e40 2024-05-27T07:51:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a8a9e3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0a741b56f0 con 0x556487268e70 2024-05-27T07:51:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a8a9e3b80 10 client.0 _ll_register_callbacks cb 0x556487130c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a78a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487276d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a78a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487276d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41490/0 (socket says 172.21.15.63:41490) 2024-05-27T07:51:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a78a00700 1 -- 172.21.15.63:0/3047857737 learned_addr learned my addr 172.21.15.63:0/3047857737 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a79400700 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564870e9e40 0x5564872767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a79e00700 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x5564872e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a78a00700 1 -- 172.21.15.63:0/3047857737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 msgr2=0x5564872e5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a78a00700 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x5564872e5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a78a00700 1 -- 172.21.15.63:0/3047857737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564870e9e40 msgr2=0x5564872767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a78a00700 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564870e9e40 0x5564872767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.112+0000 7f0a78a00700 1 -- 172.21.15.63:0/3047857737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564872734b0 con 0x5564871d59f0 2024-05-27T07:51:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.113+0000 7f0a78a00700 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487276d30 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x556487265a20 tx=0x7f0a600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258d8c4949b491eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.113+0000 7f0a5b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:39.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.113+0000 7f0a5b400700 1 -- 172.21.15.63:0/3047857737 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60004830 con 0x5564871d59f0 2024-05-27T07:51:39.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.113+0000 7f0a5b400700 1 -- 172.21.15.63:0/3047857737 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f0a600049d0 con 0x5564871d59f0 2024-05-27T07:51:39.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.113+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3047857737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556487192570 con 0x5564871d59f0 2024-05-27T07:51:39.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.114+0000 7f0a5b400700 1 -- 172.21.15.63:0/3047857737 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0a60037050 con 0x5564871d59f0 2024-05-27T07:51:39.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.114+0000 7f0a8a9e3b80 10 client.18882 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:39.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.114+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3047857737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5564872ecd60 con 0x5564871d59f0 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.114+0000 7f0a5b400700 1 -- 172.21.15.63:0/3047857737 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556487192570 con 0x5564871d59f0 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.114+0000 7f0a5b400700 1 -- 172.21.15.63:0/3047857737 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0a60028420 con 0x5564871d59f0 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.115+0000 7f0a8a9e3b80 10 client.18882 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.115+0000 7f0a8a9e3b80 10 client.18882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.115+0000 7f0a8a9e3b80 20 client.18882 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.115+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564872ed450 0x5564872ef870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.115+0000 7f0a8a9e3b80 4 client.18882 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.115+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3047857737 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556487009250 con 0x5564872ed450 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.115+0000 7f0a79400700 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564872ed450 0x5564872ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.116+0000 7f0a79400700 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564872ed450 0x5564872ef870 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51cd3d0eab936c32 server_cookie=88e9ff0151239145 in_seq=0 out_seq=0 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.116+0000 7f0a5b400700 10 client.18882 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a5b400700 1 -- 172.21.15.63:0/3047857737 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556487009250 con 0x5564872ed450 2024-05-27T07:51:39.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a5b400700 10 client.18882 handle_command_reply: tid=0 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a8a9e3b80 1 client.18882 shutdown 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3047857737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 msgr2=0x556487276d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487276d30 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x556487265a20 tx=0x7f0a600047c0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3047857737 shutdown_connections 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564872ed450 0x5564872ef870 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556487268e70 0x5564872e5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564870e9e40 0x5564872767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a8a9e3b80 1 --2- 172.21.15.63:0/3047857737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564871d59f0 0x556487276d30 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3047857737 >> 172.21.15.63:0/3047857737 conn(0x556487183390 msgr2=0x556487271740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a5b400700 0 client.18882 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:39.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.117+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3047857737 shutdown_connections 2024-05-27T07:51:39.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.118+0000 7f0a8a9e3b80 1 -- 172.21.15.63:0/3047857737 wait complete. 2024-05-27T07:51:39.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.118+0000 7f0a8a9e3b80 20 client.18882 trim_cache size 0 max 16384 2024-05-27T07:51:39.122 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:51:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:51:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:51:39.122 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:51:39.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.118+0000 7f0a7be00700 1 -- 172.21.15.63:0/4176501622 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0a5c0b2a70 msgr2=0x7f0a5c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:39.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.118+0000 7f0a7be00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0a5c0b2a70 0x7f0a5c0b4f80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a74112eb0 tx=0x7f0a60009260 comp rx=0 tx=0).stop 2024-05-27T07:51:39.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.119+0000 7f0a7be00700 1 -- 172.21.15.63:0/4176501622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a74110fe0 msgr2=0x7f0a741066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:39.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.119+0000 7f0a7be00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a74110fe0 0x7f0a741066d0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00b070 tx=0x7f0a6c004c60 comp rx=0 tx=0).stop 2024-05-27T07:51:39.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.119+0000 7f0a7be00700 1 -- 172.21.15.63:0/4176501622 shutdown_connections 2024-05-27T07:51:39.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.119+0000 7f0a7be00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0a5c0b2a70 0x7f0a5c0b4f80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.119+0000 7f0a7be00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a74110fe0 0x7f0a741066d0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.119+0000 7f0a7be00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a74102da0 0x7f0a74105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.119+0000 7f0a7be00700 1 --2- 172.21.15.63:0/4176501622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a741053e0 0x7f0a74106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:39.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.119+0000 7f0a7be00700 1 -- 172.21.15.63:0/4176501622 >> 172.21.15.63:0/4176501622 conn(0x7f0a740fe510 msgr2=0x7f0a74100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:39.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.119+0000 7f0a7be00700 1 -- 172.21.15.63:0/4176501622 shutdown_connections 2024-05-27T07:51:39.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:39.119+0000 7f0a7be00700 1 -- 172.21.15.63:0/4176501622 wait complete. 2024-05-27T07:51:39.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:38 smithi086 ceph-mon[34687]: pgmap v569: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:51:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:38 smithi149 ceph-mon[35408]: pgmap v569: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:51:40.125 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:51:40.126 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:51:40.629 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:41.040 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:40 smithi063 ceph-mon[31613]: pgmap v570: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:51:41.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:40 smithi086 ceph-mon[34687]: pgmap v570: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:51:41.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:40 smithi149 ceph-mon[35408]: pgmap v570: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:51:43.096 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:42 smithi063 ceph-mon[31613]: pgmap v571: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:51:43.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.256+0000 7ff17f200700 1 -- 172.21.15.63:0/2099457464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff178110c10 msgr2=0x7ff178113030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:43.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.256+0000 7ff17f200700 1 --2- 172.21.15.63:0/2099457464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff178110c10 0x7ff178113030 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7ff17000b0a0 tx=0x7ff17000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.256+0000 7ff17f200700 1 -- 172.21.15.63:0/2099457464 shutdown_connections 2024-05-27T07:51:43.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.256+0000 7ff17f200700 1 --2- 172.21.15.63:0/2099457464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff178110c10 0x7ff178113030 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.256+0000 7ff17f200700 1 --2- 172.21.15.63:0/2099457464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff178075ad0 0x7ff178075ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.256+0000 7ff17f200700 1 --2- 172.21.15.63:0/2099457464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff178076420 0x7ff1781106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.256+0000 7ff17f200700 1 -- 172.21.15.63:0/2099457464 >> 172.21.15.63:0/2099457464 conn(0x7ff178100030 msgr2=0x7ff178102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:43.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.257+0000 7ff17f200700 1 -- 172.21.15.63:0/2099457464 shutdown_connections 2024-05-27T07:51:43.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.257+0000 7ff17f200700 1 -- 172.21.15.63:0/2099457464 wait complete. 2024-05-27T07:51:43.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.258+0000 7ff17f200700 1 Processor -- start 2024-05-27T07:51:43.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.258+0000 7ff17f200700 1 -- start start 2024-05-27T07:51:43.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.258+0000 7ff17f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff178075ad0 0x7ff17819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:43.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.258+0000 7ff17f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff178076420 0x7ff17819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:43.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.258+0000 7ff17f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff178110c10 0x7ff1781a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:43.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.258+0000 7ff17f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff178116550 con 0x7ff178076420 2024-05-27T07:51:43.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.258+0000 7ff17f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1781163d0 con 0x7ff178110c10 2024-05-27T07:51:43.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.258+0000 7ff17f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1781166d0 con 0x7ff178075ad0 2024-05-27T07:51:43.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff177400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff178076420 0x7ff17819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:43.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff177400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff178076420 0x7ff17819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41514/0 (socket says 172.21.15.63:41514) 2024-05-27T07:51:43.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff177400700 1 -- 172.21.15.63:0/4170440866 learned_addr learned my addr 172.21.15.63:0/4170440866 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:43.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff17d200700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff178110c10 0x7ff1781a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:43.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff177e00700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff178075ad0 0x7ff17819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:43.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff177400700 1 -- 172.21.15.63:0/4170440866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff178075ad0 msgr2=0x7ff17819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:43.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff177400700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff178075ad0 0x7ff17819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff177400700 1 -- 172.21.15.63:0/4170440866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff178110c10 msgr2=0x7ff1781a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:43.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff177400700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff178110c10 0x7ff1781a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff177400700 1 -- 172.21.15.63:0/4170440866 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1781a1db0 con 0x7ff178076420 2024-05-27T07:51:43.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.259+0000 7ff177e00700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff178075ad0 0x7ff17819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:43.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.260+0000 7ff177400700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff178076420 0x7ff17819d480 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7ff16400d980 tx=0x7ff16400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:43.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.260+0000 7ff174c00700 1 -- 172.21.15.63:0/4170440866 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164017070 con 0x7ff178076420 2024-05-27T07:51:43.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.260+0000 7ff174c00700 1 -- 172.21.15.63:0/4170440866 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff164010460 con 0x7ff178076420 2024-05-27T07:51:43.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.260+0000 7ff17f200700 1 -- 172.21.15.63:0/4170440866 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1781a20a0 con 0x7ff178076420 2024-05-27T07:51:43.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.260+0000 7ff174c00700 1 -- 172.21.15.63:0/4170440866 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16400de60 con 0x7ff178076420 2024-05-27T07:51:43.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.260+0000 7ff17f200700 1 -- 172.21.15.63:0/4170440866 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1781a99c0 con 0x7ff178076420 2024-05-27T07:51:43.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.262+0000 7ff17f200700 1 -- 172.21.15.63:0/4170440866 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff158005330 con 0x7ff178076420 2024-05-27T07:51:43.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.265+0000 7ff174c00700 1 -- 172.21.15.63:0/4170440866 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff164009b40 con 0x7ff178076420 2024-05-27T07:51:43.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.267+0000 7ff174c00700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff15c0b29f0 0x7ff15c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:43.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.268+0000 7ff174c00700 1 -- 172.21.15.63:0/4170440866 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7ff1640e23a0 con 0x7ff178076420 2024-05-27T07:51:43.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.268+0000 7ff177e00700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff15c0b29f0 0x7ff15c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.269+0000 7ff177e00700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff15c0b29f0 0x7ff15c0b4f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff168005950 tx=0x7ff1680058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:43.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.269+0000 7ff174c00700 1 -- 172.21.15.63:0/4170440866 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff1640a9960 con 0x7ff178076420 2024-05-27T07:51:43.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:43 smithi086 ceph-mon[34687]: pgmap v571: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:51:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:42 smithi149 ceph-mon[35408]: pgmap v571: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:51:43.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.594+0000 7ff17f200700 1 -- 172.21.15.63:0/4170440866 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff158005620 con 0x7ff178076420 2024-05-27T07:51:43.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.595+0000 7ff174c00700 1 -- 172.21.15.63:0/4170440866 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7ff1640a9260 con 0x7ff178076420 2024-05-27T07:51:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:51:43.597 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:51:43.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.599+0000 7ff17f200700 1 -- 172.21.15.63:0/4170440866 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff15c0b29f0 msgr2=0x7ff15c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:43.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.599+0000 7ff17f200700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff15c0b29f0 0x7ff15c0b4f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff168005950 tx=0x7ff1680058e0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.599+0000 7ff17f200700 1 -- 172.21.15.63:0/4170440866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff178076420 msgr2=0x7ff17819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:43.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.599+0000 7ff17f200700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff178076420 0x7ff17819d480 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7ff16400d980 tx=0x7ff16400dd40 comp rx=0 tx=0).stop 2024-05-27T07:51:43.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.599+0000 7ff17f200700 1 -- 172.21.15.63:0/4170440866 shutdown_connections 2024-05-27T07:51:43.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.599+0000 7ff17f200700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff15c0b29f0 0x7ff15c0b4f00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.599+0000 7ff17f200700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff178110c10 0x7ff1781a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.599+0000 7ff17f200700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff178075ad0 0x7ff17819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.600+0000 7ff17f200700 1 --2- 172.21.15.63:0/4170440866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff178076420 0x7ff17819d480 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:43.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.600+0000 7ff17f200700 1 -- 172.21.15.63:0/4170440866 >> 172.21.15.63:0/4170440866 conn(0x7ff178100030 msgr2=0x7ff1781006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:43.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.600+0000 7ff17f200700 1 -- 172.21.15.63:0/4170440866 shutdown_connections 2024-05-27T07:51:43.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:43.600+0000 7ff17f200700 1 -- 172.21.15.63:0/4170440866 wait complete. 2024-05-27T07:51:43.602 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:51:44.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:44 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4170440866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:51:44.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:44 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4170440866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:51:44.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:44 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4170440866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:51:44.592 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:51:45.097 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:45.277 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:45 smithi063 ceph-mon[31613]: pgmap v572: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-05-27T07:51:45.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:45 smithi086 ceph-mon[34687]: pgmap v572: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-05-27T07:51:45.466 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:45 smithi149 ceph-mon[35408]: pgmap v572: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-05-27T07:51:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.814+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1887199610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0075480 msgr2=0x7f2da0075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.814+0000 7f2d9fe00640 1 --2- 172.21.15.63:0/1887199610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0075480 0x7f2da0075890 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009900 tx=0x7f2d9002a480 comp rx=0 tx=0).stop 2024-05-27T07:51:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.814+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1887199610 shutdown_connections 2024-05-27T07:51:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.814+0000 7f2d9fe00640 1 --2- 172.21.15.63:0/1887199610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2da0114a70 0x7f2da0116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.814+0000 7f2d9fe00640 1 --2- 172.21.15.63:0/1887199610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2da0112110 0x7f2da0114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.814+0000 7f2d9fe00640 1 --2- 172.21.15.63:0/1887199610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0075480 0x7f2da0075890 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.814+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1887199610 >> 172.21.15.63:0/1887199610 conn(0x7f2da0079140 msgr2=0x7f2da0079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.814+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1887199610 shutdown_connections 2024-05-27T07:51:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.814+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1887199610 wait complete. 2024-05-27T07:51:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.815+0000 7f2d9fe00640 1 Processor -- start 2024-05-27T07:51:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.816+0000 7f2d9fe00640 1 -- start start 2024-05-27T07:51:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.816+0000 7f2d9fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2da0075480 0x7f2da01a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.816+0000 7f2d9fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0112110 0x7f2da01a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.816+0000 7f2d9fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2da0114a70 0x7f2da01a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.816+0000 7f2d9fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2da011a110 con 0x7f2da0112110 2024-05-27T07:51:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.816+0000 7f2d9fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0119f90 con 0x7f2da0114a70 2024-05-27T07:51:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.816+0000 7f2d9fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2da011a290 con 0x7f2da0075480 2024-05-27T07:51:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.817+0000 7f2d9e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0112110 0x7f2da01a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.817+0000 7f2d9e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0112110 0x7f2da01a6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41540/0 (socket says 172.21.15.63:41540) 2024-05-27T07:51:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.817+0000 7f2d9e000640 1 -- 172.21.15.63:0/1775451690 learned_addr learned my addr 172.21.15.63:0/1775451690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.817+0000 7f2d9f400640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2da0114a70 0x7f2da01a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.817+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2da0075480 0x7f2da01a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.817+0000 7f2d9e000640 1 -- 172.21.15.63:0/1775451690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2da0075480 msgr2=0x7f2da01a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.817+0000 7f2d9e000640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2da0075480 0x7f2da01a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.817+0000 7f2d9e000640 1 -- 172.21.15.63:0/1775451690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2da0114a70 msgr2=0x7f2da01a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.817+0000 7f2d9e000640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2da0114a70 0x7f2da01a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.817+0000 7f2d9e000640 1 -- 172.21.15.63:0/1775451690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da01a1000 con 0x7f2da0112110 2024-05-27T07:51:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.818+0000 7f2d9e000640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0112110 0x7f2da01a6d40 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00c920 tx=0x7f2d8c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.818+0000 7f2d97400640 1 -- 172.21.15.63:0/1775451690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c007c70 con 0x7f2da0112110 2024-05-27T07:51:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.818+0000 7f2d97400640 1 -- 172.21.15.63:0/1775451690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d8c007e10 con 0x7f2da0112110 2024-05-27T07:51:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.818+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1775451690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da01a12f0 con 0x7f2da0112110 2024-05-27T07:51:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.818+0000 7f2d97400640 1 -- 172.21.15.63:0/1775451690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c018710 con 0x7f2da0112110 2024-05-27T07:51:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.818+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1775451690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da01a1840 con 0x7f2da0112110 2024-05-27T07:51:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.822+0000 7f2dac9b0740 1 Processor -- start 2024-05-27T07:51:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.822+0000 7f2dac9b0740 1 -- start start 2024-05-27T07:51:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.822+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bfa01b0 0x55702c342a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.822+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bf772f0 0x55702c342fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf7f4d0 0x55702c3434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55702c2b3b90 con 0x55702bfa01b0 2024-05-27T07:51:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55702c2a3730 con 0x55702bf7f4d0 2024-05-27T07:51:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55702c2a7770 con 0x55702bf772f0 2024-05-27T07:51:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bfa01b0 0x55702c342a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bfa01b0 0x55702c342a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41542/0 (socket says 172.21.15.63:41542) 2024-05-27T07:51:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2d9f400640 1 -- 172.21.15.63:0/980647221 learned_addr learned my addr 172.21.15.63:0/980647221 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2d97400640 1 -- 172.21.15.63:0/1775451690 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2d8c00f450 con 0x7f2da0112110 2024-05-27T07:51:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/980647221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bf772f0 0x55702c342fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2d9f400640 1 -- 172.21.15.63:0/980647221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bf772f0 msgr2=0x55702c342fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2d9f400640 1 --2- 172.21.15.63:0/980647221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bf772f0 0x55702c342fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2d9f400640 1 -- 172.21.15.63:0/980647221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf7f4d0 msgr2=0x55702c3434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2d9f400640 1 --2- 172.21.15.63:0/980647221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf7f4d0 0x55702c3434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.823+0000 7f2d9f400640 1 -- 172.21.15.63:0/980647221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55702c23c8b0 con 0x55702bfa01b0 2024-05-27T07:51:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.824+0000 7f2d9f400640 1 --2- 172.21.15.63:0/980647221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bfa01b0 0x55702c342a60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009e60 tx=0x7f2d98010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa8cb4166ce9dbe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.824+0000 7f2d8be00640 1 -- 172.21.15.63:0/980647221 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98016070 con 0x55702bfa01b0 2024-05-27T07:51:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.824+0000 7f2d8be00640 1 -- 172.21.15.63:0/980647221 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d98004d20 con 0x55702bfa01b0 2024-05-27T07:51:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.824+0000 7f2d8be00640 1 -- 172.21.15.63:0/980647221 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98012650 con 0x55702bfa01b0 2024-05-27T07:51:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.825+0000 7f2d97400640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d840b2950 0x7f2d840b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.825+0000 7f2d97400640 1 -- 172.21.15.63:0/1775451690 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f2d8c0e2cd0 con 0x7f2da0112110 2024-05-27T07:51:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.825+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d840b2950 0x7f2d840b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.825+0000 7f2dac9b0740 1 -- 172.21.15.63:0/980647221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bfa01b0 msgr2=0x55702c342a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.825+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/980647221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bfa01b0 0x55702c342a60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009e60 tx=0x7f2d98010740 comp rx=0 tx=0).stop 2024-05-27T07:51:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.826+0000 7f2dac9b0740 1 -- 172.21.15.63:0/980647221 shutdown_connections 2024-05-27T07:51:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.826+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/980647221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf7f4d0 0x55702c3434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.826+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/980647221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bf772f0 0x55702c342fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.826+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/980647221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bfa01b0 0x55702c342a60 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.826+0000 7f2dac9b0740 1 -- 172.21.15.63:0/980647221 >> 172.21.15.63:0/980647221 conn(0x55702bf94300 msgr2=0x55702c33f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.826+0000 7f2dac9b0740 1 -- 172.21.15.63:0/980647221 shutdown_connections 2024-05-27T07:51:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.826+0000 7f2dac9b0740 1 -- 172.21.15.63:0/980647221 wait complete. 2024-05-27T07:51:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.826+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d840b2950 0x7f2d840b4e60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d90006010 tx=0x7f2d90009640 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.827+0000 7f2dac9b0740 1 -- start start 2024-05-27T07:51:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.827+0000 7f2dac9b0740 1 Processor -- start 2024-05-27T07:51:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.827+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c34ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.827+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf7f4d0 0x55702c34d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bfa01b0 0x55702c358ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55702bf89790 con 0x55702bf772f0 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55702c24bbc0 con 0x55702bf7f4d0 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55702c34fb90 con 0x55702bfa01b0 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2dac9b0740 10 client.0 _ll_register_callbacks cb 0x55702c229400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c34ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c34ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41546/0 (socket says 172.21.15.63:41546) 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9ea00640 1 -- 172.21.15.63:0/3351147742 learned_addr learned my addr 172.21.15.63:0/3351147742 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9e000640 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf7f4d0 0x55702c34d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9f400640 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bfa01b0 0x55702c358ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9ea00640 1 -- 172.21.15.63:0/3351147742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bfa01b0 msgr2=0x55702c358ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bfa01b0 0x55702c358ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9ea00640 1 -- 172.21.15.63:0/3351147742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf7f4d0 msgr2=0x55702c34d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf7f4d0 0x55702c34d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9ea00640 1 -- 172.21.15.63:0/3351147742 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55702bfb8210 con 0x55702bf772f0 2024-05-27T07:51:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.828+0000 7f2d9e000640 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf7f4d0 0x55702c34d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.829+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c34ced0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f2d90033000 tx=0x7f2d90009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9e08b0842229b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.829+0000 7f2d88c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.829+0000 7f2d88c00640 1 -- 172.21.15.63:0/3351147742 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9004b030 con 0x55702bf772f0 2024-05-27T07:51:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.829+0000 7f2dac9b0740 1 -- 172.21.15.63:0/3351147742 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d98004d20 con 0x55702bf772f0 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.829+0000 7f2d88c00640 1 -- 172.21.15.63:0/3351147742 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f2d90003b70 con 0x55702bf772f0 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2d88c00640 1 -- 172.21.15.63:0/3351147742 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2d90058050 con 0x55702bf772f0 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2dac9b0740 10 client.18900 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2dac9b0740 1 -- 172.21.15.63:0/3351147742 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55702c084620 con 0x55702bf772f0 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2d88c00640 1 -- 172.21.15.63:0/3351147742 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d98004d20 con 0x55702bf772f0 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2d88c00640 1 -- 172.21.15.63:0/3351147742 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2d90047ab0 con 0x55702bf772f0 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2dac9b0740 10 client.18900 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2dac9b0740 10 client.18900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2dac9b0740 20 client.18900 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55702bf5ed80 0x55702c35ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2dac9b0740 4 client.18900 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.830+0000 7f2dac9b0740 1 -- 172.21.15.63:0/3351147742 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55702c2b5ae0 con 0x55702bf5ed80 2024-05-27T07:51:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.831+0000 7f2d9e000640 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55702bf5ed80 0x55702c35ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.832+0000 7f2d9e000640 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55702bf5ed80 0x55702c35ef40 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fe50b310cf10a2f server_cookie=e95ba77a5b35da6a in_seq=0 out_seq=0 2024-05-27T07:51:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.832+0000 7f2d88c00640 10 client.18900 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.834+0000 7f2d88c00640 1 -- 172.21.15.63:0/3351147742 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55702c2b5ae0 con 0x55702bf5ed80 2024-05-27T07:51:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.834+0000 7f2d88c00640 10 client.18900 handle_command_reply: tid=0 2024-05-27T07:51:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.834+0000 7f2dac9b0740 1 client.18900 shutdown 2024-05-27T07:51:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 1 -- 172.21.15.63:0/3351147742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 msgr2=0x55702c34ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c34ced0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f2d90033000 tx=0x7f2d90009260 comp rx=0 tx=0).stop 2024-05-27T07:51:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 1 -- 172.21.15.63:0/3351147742 shutdown_connections 2024-05-27T07:51:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55702bf5ed80 0x55702c35ef40 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bfa01b0 0x55702c358ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf7f4d0 0x55702c34d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/3351147742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c34ced0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2d88c00640 0 client.18900 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 1 -- 172.21.15.63:0/3351147742 >> 172.21.15.63:0/3351147742 conn(0x55702bf94300 msgr2=0x55702c344780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 1 -- 172.21.15.63:0/3351147742 shutdown_connections 2024-05-27T07:51:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 1 -- 172.21.15.63:0/3351147742 wait complete. 2024-05-27T07:51:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.835+0000 7f2dac9b0740 20 client.18900 trim_cache size 0 max 16384 2024-05-27T07:51:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.858+0000 7f2dac9b0740 1 Processor -- start 2024-05-27T07:51:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.858+0000 7f2dac9b0740 1 -- start start 2024-05-27T07:51:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.858+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c358c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.858+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bf7f4d0 0x55702c359170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.858+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bfa01b0 0x55702c346710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.858+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01b9ed0 con 0x55702bf772f0 2024-05-27T07:51:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.859+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01b77a0 con 0x55702bfa01b0 2024-05-27T07:51:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.859+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01b5610 con 0x55702bf7f4d0 2024-05-27T07:51:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.859+0000 7f2d9ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c358c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.859+0000 7f2d9ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c358c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41558/0 (socket says 172.21.15.63:41558) 2024-05-27T07:51:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.859+0000 7f2d9ea00640 1 -- 172.21.15.63:0/124401228 learned_addr learned my addr 172.21.15.63:0/124401228 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.859+0000 7f2d9e000640 1 --2- 172.21.15.63:0/124401228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bf7f4d0 0x55702c359170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.859+0000 7f2d9f400640 1 --2- 172.21.15.63:0/124401228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bfa01b0 0x55702c346710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.860+0000 7f2d9ea00640 1 -- 172.21.15.63:0/124401228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bf7f4d0 msgr2=0x55702c359170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.860+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/124401228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bf7f4d0 0x55702c359170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.860+0000 7f2d9ea00640 1 -- 172.21.15.63:0/124401228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bfa01b0 msgr2=0x55702c346710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.860+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/124401228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bfa01b0 0x55702c346710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.860+0000 7f2d9ea00640 1 -- 172.21.15.63:0/124401228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55702c3596b0 con 0x55702bf772f0 2024-05-27T07:51:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.860+0000 7f2d9f400640 1 --2- 172.21.15.63:0/124401228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bfa01b0 0x55702c346710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.860+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/124401228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c358c30 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f2d900057c0 tx=0x7f2d90031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d478b2f131488c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.860+0000 7f2d8b400640 1 -- 172.21.15.63:0/124401228 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90061070 con 0x55702bf772f0 2024-05-27T07:51:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.860+0000 7f2d8b400640 1 -- 172.21.15.63:0/124401228 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d90003bc0 con 0x55702bf772f0 2024-05-27T07:51:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.860+0000 7f2d8b400640 1 -- 172.21.15.63:0/124401228 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9004b070 con 0x55702bf772f0 2024-05-27T07:51:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 -- 172.21.15.63:0/124401228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 msgr2=0x55702c358c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/124401228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c358c30 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f2d900057c0 tx=0x7f2d90031820 comp rx=0 tx=0).stop 2024-05-27T07:51:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 -- 172.21.15.63:0/124401228 shutdown_connections 2024-05-27T07:51:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/124401228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bfa01b0 0x55702c346710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/124401228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bf7f4d0 0x55702c359170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/124401228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf772f0 0x55702c358c30 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 -- 172.21.15.63:0/124401228 >> 172.21.15.63:0/124401228 conn(0x55702bf94300 msgr2=0x55702c34ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 -- 172.21.15.63:0/124401228 shutdown_connections 2024-05-27T07:51:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 -- 172.21.15.63:0/124401228 wait complete. 2024-05-27T07:51:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 -- start start 2024-05-27T07:51:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.862+0000 7f2dac9b0740 1 Processor -- start 2024-05-27T07:51:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.863+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf772f0 0x55702c350440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.863+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf7f4d0 0x55702c350980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.863+0000 7f2dac9b0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bfa01b0 0x55702c3c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.863+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01b5c60 con 0x55702bf7f4d0 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.863+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01b70d0 con 0x55702bf772f0 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.863+0000 7f2dac9b0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01b6330 con 0x55702bfa01b0 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.863+0000 7f2dac9b0740 10 client.0 _ll_register_callbacks cb 0x7f2d90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.864+0000 7f2d9e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf7f4d0 0x55702c350980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.864+0000 7f2d9e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf7f4d0 0x55702c350980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41572/0 (socket says 172.21.15.63:41572) 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.864+0000 7f2d9e000640 1 -- 172.21.15.63:0/1387848486 learned_addr learned my addr 172.21.15.63:0/1387848486 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.864+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf772f0 0x55702c350440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.864+0000 7f2d9f400640 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bfa01b0 0x55702c3c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.864+0000 7f2d9e000640 1 -- 172.21.15.63:0/1387848486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bfa01b0 msgr2=0x55702c3c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.864+0000 7f2d9e000640 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bfa01b0 0x55702c3c5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.864+0000 7f2d9e000640 1 -- 172.21.15.63:0/1387848486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf772f0 msgr2=0x55702c350440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.864+0000 7f2d9e000640 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf772f0 0x55702c350440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.864+0000 7f2d9e000640 1 -- 172.21.15.63:0/1387848486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55702c3c5b40 con 0x55702bf7f4d0 2024-05-27T07:51:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.865+0000 7f2d9e000640 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf7f4d0 0x55702c350980 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00fd80 tx=0x7f2d8c00fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e55c9ec0b589ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.865+0000 7f2d88c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.865+0000 7f2d88c00640 1 -- 172.21.15.63:0/1387848486 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c0c2030 con 0x55702bf7f4d0 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.865+0000 7f2d88c00640 1 -- 172.21.15.63:0/1387848486 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f2d8c0a6180 con 0x55702bf7f4d0 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.865+0000 7f2dac9b0740 1 -- 172.21.15.63:0/1387848486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d90003bc0 con 0x55702bf7f4d0 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.865+0000 7f2d88c00640 1 -- 172.21.15.63:0/1387848486 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2d8c0a9b90 con 0x55702bf7f4d0 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.865+0000 7f2dac9b0740 10 client.18906 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.865+0000 7f2dac9b0740 1 -- 172.21.15.63:0/1387848486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55702c3c6e10 con 0x55702bf7f4d0 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.865+0000 7f2d88c00640 1 -- 172.21.15.63:0/1387848486 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d90003bc0 con 0x55702bf7f4d0 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.866+0000 7f2d88c00640 1 -- 172.21.15.63:0/1387848486 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2d8c0e0280 con 0x55702bf7f4d0 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.866+0000 7f2dac9b0740 10 client.18906 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.866+0000 7f2dac9b0740 10 client.18906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.866+0000 7f2dac9b0740 20 client.18906 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.866+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55702bf341c0 0x55702c3c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.866+0000 7f2dac9b0740 4 client.18906 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:51:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.866+0000 7f2dac9b0740 1 -- 172.21.15.63:0/1387848486 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55702c2af2c0 con 0x55702bf341c0 2024-05-27T07:51:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.866+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55702bf341c0 0x55702c3c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.867+0000 7f2d9ea00640 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55702bf341c0 0x55702c3c94f0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d704a5cb883d834 server_cookie=500e95471a9bd43d in_seq=0 out_seq=0 2024-05-27T07:51:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.867+0000 7f2d88c00640 10 client.18906 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.913+0000 7f2d88c00640 1 -- 172.21.15.63:0/1387848486 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55702c2af2c0 con 0x55702bf341c0 2024-05-27T07:51:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.913+0000 7f2d88c00640 10 client.18906 handle_command_reply: tid=0 2024-05-27T07:51:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.913+0000 7f2dac9b0740 1 client.18906 shutdown 2024-05-27T07:51:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.913+0000 7f2dac9b0740 1 -- 172.21.15.63:0/1387848486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf7f4d0 msgr2=0x55702c350980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.913+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf7f4d0 0x55702c350980 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00fd80 tx=0x7f2d8c00fca0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.914+0000 7f2dac9b0740 1 -- 172.21.15.63:0/1387848486 shutdown_connections 2024-05-27T07:51:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.914+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55702bf341c0 0x55702c3c94f0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.914+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55702bfa01b0 0x55702c3c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.914+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55702bf7f4d0 0x55702c350980 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.914+0000 7f2dac9b0740 1 --2- 172.21.15.63:0/1387848486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55702bf772f0 0x55702c350440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.914+0000 7f2dac9b0740 1 -- 172.21.15.63:0/1387848486 >> 172.21.15.63:0/1387848486 conn(0x55702bf94300 msgr2=0x55702c34c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.914+0000 7f2d88c00640 0 client.18906 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.915+0000 7f2dac9b0740 1 -- 172.21.15.63:0/1387848486 shutdown_connections 2024-05-27T07:51:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.915+0000 7f2dac9b0740 1 -- 172.21.15.63:0/1387848486 wait complete. 2024-05-27T07:51:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.915+0000 7f2dac9b0740 20 client.18906 trim_cache size 0 max 16384 2024-05-27T07:51:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:51:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:51:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:51:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.916+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1775451690 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d840b2950 msgr2=0x7f2d840b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.916+0000 7f2d9fe00640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d840b2950 0x7f2d840b4e60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d90006010 tx=0x7f2d90009640 comp rx=0 tx=0).stop 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.916+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1775451690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0112110 msgr2=0x7f2da01a6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.916+0000 7f2d9fe00640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0112110 0x7f2da01a6d40 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00c920 tx=0x7f2d8c00cde0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.916+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1775451690 shutdown_connections 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.916+0000 7f2d9fe00640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d840b2950 0x7f2d840b4e60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.916+0000 7f2d9fe00640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2da0114a70 0x7f2da01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.916+0000 7f2d9fe00640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2da0112110 0x7f2da01a6d40 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.916+0000 7f2d9fe00640 1 --2- 172.21.15.63:0/1775451690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2da0075480 0x7f2da01a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.916+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1775451690 >> 172.21.15.63:0/1775451690 conn(0x7f2da0079140 msgr2=0x7f2da0112520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.917+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1775451690 shutdown_connections 2024-05-27T07:51:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:45.917+0000 7f2d9fe00640 1 -- 172.21.15.63:0/1775451690 wait complete. 2024-05-27T07:51:47.306 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:47 smithi063 ceph-mon[31613]: pgmap v573: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-05-27T07:51:47.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:47 smithi086 ceph-mon[34687]: pgmap v573: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-05-27T07:51:47.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:47 smithi149 ceph-mon[35408]: pgmap v573: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 9 op/s 2024-05-27T07:51:47.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.716+0000 7fcacd400700 1 -- 172.21.15.63:0/2999901518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac8105300 msgr2=0x7fcac8105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:47.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.716+0000 7fcacd400700 1 --2- 172.21.15.63:0/2999901518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac8105300 0x7fcac8105710 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009990 tx=0x7fcab8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.717+0000 7fcacd400700 1 -- 172.21.15.63:0/2999901518 shutdown_connections 2024-05-27T07:51:47.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.717+0000 7fcacd400700 1 --2- 172.21.15.63:0/2999901518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac810e730 0x7fcac8110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.717+0000 7fcacd400700 1 --2- 172.21.15.63:0/2999901518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac8105300 0x7fcac8105710 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.717+0000 7fcacd400700 1 --2- 172.21.15.63:0/2999901518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8105c50 0x7fcac810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.717+0000 7fcacd400700 1 -- 172.21.15.63:0/2999901518 >> 172.21.15.63:0/2999901518 conn(0x7fcac8078ab0 msgr2=0x7fcac8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.717+0000 7fcacd400700 1 -- 172.21.15.63:0/2999901518 shutdown_connections 2024-05-27T07:51:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.717+0000 7fcacd400700 1 -- 172.21.15.63:0/2999901518 wait complete. 2024-05-27T07:51:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.718+0000 7fcacd400700 1 Processor -- start 2024-05-27T07:51:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.718+0000 7fcacd400700 1 -- start start 2024-05-27T07:51:47.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.719+0000 7fcacd400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac8105300 0x7fcac8102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.719+0000 7fcacd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8105c50 0x7fcac8103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.719+0000 7fcacd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac810e730 0x7fcac8103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.719+0000 7fcacd400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8114170 con 0x7fcac810e730 2024-05-27T07:51:47.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.719+0000 7fcacd400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8113ff0 con 0x7fcac8105c50 2024-05-27T07:51:47.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.719+0000 7fcacd400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81142f0 con 0x7fcac8105300 2024-05-27T07:51:47.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.719+0000 7fcacca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac810e730 0x7fcac8103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.720+0000 7fcacca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac810e730 0x7fcac8103a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41592/0 (socket says 172.21.15.63:41592) 2024-05-27T07:51:47.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.720+0000 7fcacca00700 1 -- 172.21.15.63:0/1658978409 learned_addr learned my addr 172.21.15.63:0/1658978409 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:47.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.720+0000 7fcac7e00700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac8105300 0x7fcac8102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.720+0000 7fcac7400700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8105c50 0x7fcac8103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.720+0000 7fcacca00700 1 -- 172.21.15.63:0/1658978409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac8105300 msgr2=0x7fcac8102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:47.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.720+0000 7fcacca00700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac8105300 0x7fcac8102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.720+0000 7fcacca00700 1 -- 172.21.15.63:0/1658978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8105c50 msgr2=0x7fcac8103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:47.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.720+0000 7fcacca00700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8105c50 0x7fcac8103530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.720+0000 7fcacca00700 1 -- 172.21.15.63:0/1658978409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac81a8e70 con 0x7fcac810e730 2024-05-27T07:51:47.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.721+0000 7fcacca00700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac810e730 0x7fcac8103a70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fcac000b970 tx=0x7fcac000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:47.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.721+0000 7fcac4c00700 1 -- 172.21.15.63:0/1658978409 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0019070 con 0x7fcac810e730 2024-05-27T07:51:47.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.721+0000 7fcac4c00700 1 -- 172.21.15.63:0/1658978409 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcac000cdc0 con 0x7fcac810e730 2024-05-27T07:51:47.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.721+0000 7fcacd400700 1 -- 172.21.15.63:0/1658978409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac81a9160 con 0x7fcac810e730 2024-05-27T07:51:47.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.721+0000 7fcac4c00700 1 -- 172.21.15.63:0/1658978409 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0014e30 con 0x7fcac810e730 2024-05-27T07:51:47.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.721+0000 7fcacd400700 1 -- 172.21.15.63:0/1658978409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac81a9650 con 0x7fcac810e730 2024-05-27T07:51:47.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.726+0000 7fcac4c00700 1 -- 172.21.15.63:0/1658978409 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcac001c070 con 0x7fcac810e730 2024-05-27T07:51:47.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.726+0000 7fcadd5a2b80 1 Processor -- start 2024-05-27T07:51:47.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.727+0000 7fcadd5a2b80 1 -- start start 2024-05-27T07:51:47.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.727+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 0x5571b0bc6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.727+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.727+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 0x5571b0bd4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.727+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5571b0af5e40 con 0x5571b0bc7820 2024-05-27T07:51:47.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.727+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5571b0ae59e0 con 0x5571b0a48e40 2024-05-27T07:51:47.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.727+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5571b0ae9a20 con 0x5571b0bc7410 2024-05-27T07:51:47.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.728+0000 7fcac7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 0x5571b0bc6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.728+0000 7fcac7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.728+0000 7fcac7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 0x5571b0bc6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:53706/0 (socket says 172.21.15.63:53706) 2024-05-27T07:51:47.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.728+0000 7fcac7400700 1 -- 172.21.15.63:0/2348487668 learned_addr learned my addr 172.21.15.63:0/2348487668 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:47.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.728+0000 7fcac7400700 1 --2- 172.21.15.63:0/2348487668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 0x5571b0bd4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.728+0000 7fcac4c00700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaac0b29f0 0x7fcaac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.729+0000 7fcacca00700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaac0b29f0 0x7fcaac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.729+0000 7fcac4c00700 1 -- 172.21.15.63:0/1658978409 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7fcac00e1c80 con 0x7fcac810e730 2024-05-27T07:51:47.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.729+0000 7fcac7400700 1 -- 172.21.15.63:0/2348487668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 msgr2=0x5571b0bd1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:47.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.729+0000 7fcac7400700 1 --2- 172.21.15.63:0/2348487668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.729+0000 7fcac7400700 1 -- 172.21.15.63:0/2348487668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 msgr2=0x5571b0bd4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:47.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.729+0000 7fcac7400700 1 --2- 172.21.15.63:0/2348487668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 0x5571b0bd4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.729+0000 7fcac7400700 1 -- 172.21.15.63:0/2348487668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b0aae420 con 0x5571b0a48e40 2024-05-27T07:51:47.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.730+0000 7fcac7400700 1 --2- 172.21.15.63:0/2348487668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 0x5571b0bc6ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcab400a630 tx=0x7fcab400a7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b44c02b705f241 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:47.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.730+0000 7fcacca00700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaac0b29f0 0x7fcaac0b4f00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcac000bfd0 tx=0x7fcac0004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:47.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.730+0000 7fcabd600700 1 -- 172.21.15.63:0/2348487668 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011e50 con 0x5571b0a48e40 2024-05-27T07:51:47.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.730+0000 7fcabd600700 1 -- 172.21.15.63:0/2348487668 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcab40044b0 con 0x5571b0a48e40 2024-05-27T07:51:47.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.730+0000 7fcabd600700 1 -- 172.21.15.63:0/2348487668 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4005240 con 0x5571b0a48e40 2024-05-27T07:51:47.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.732+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/2348487668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 msgr2=0x5571b0bc6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:47.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.732+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/2348487668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 0x5571b0bc6ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcab400a630 tx=0x7fcab400a7d0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.732+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/2348487668 shutdown_connections 2024-05-27T07:51:47.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.732+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/2348487668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 0x5571b0bd4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.732+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/2348487668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 0x5571b0bc6ed0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.732+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/2348487668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.732+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/2348487668 >> 172.21.15.63:0/2348487668 conn(0x5571b0aa8fc0 msgr2=0x5571b0bc2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:47.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.732+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/2348487668 shutdown_connections 2024-05-27T07:51:47.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.732+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/2348487668 wait complete. 2024-05-27T07:51:47.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.733+0000 7fcadd5a2b80 1 -- start start 2024-05-27T07:51:47.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.733+0000 7fcadd5a2b80 1 Processor -- start 2024-05-27T07:51:47.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.734+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 0x5571b0bd1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.734+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.734+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 0x5571b0bd20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.734+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5571b09ebfa0 con 0x5571b0bc7820 2024-05-27T07:51:47.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.734+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5571b0a86760 con 0x5571b0a48e40 2024-05-27T07:51:47.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.734+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5571b0bd49e0 con 0x5571b0bc7410 2024-05-27T07:51:47.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.734+0000 7fcadd5a2b80 10 client.0 _ll_register_callbacks cb 0x5571b0a8fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:47.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.734+0000 7fcac7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 0x5571b0bd20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.734+0000 7fcac7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 0x5571b0bd20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41608/0 (socket says 172.21.15.63:41608) 2024-05-27T07:51:47.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.734+0000 7fcac7400700 1 -- 172.21.15.63:0/1157475616 learned_addr learned my addr 172.21.15.63:0/1157475616 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:47.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.735+0000 7fcac7400700 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 0x5571b0bd1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.735+0000 7fcac7e00700 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.735+0000 7fcac7400700 1 -- 172.21.15.63:0/1157475616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 msgr2=0x5571b0bd1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:47.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.735+0000 7fcac7400700 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.735+0000 7fcac7400700 1 -- 172.21.15.63:0/1157475616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 msgr2=0x5571b0bd1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:47.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.735+0000 7fcac7400700 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 0x5571b0bd1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.735+0000 7fcac7400700 1 -- 172.21.15.63:0/1157475616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b0a358d0 con 0x5571b0bc7820 2024-05-27T07:51:47.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.735+0000 7fcac7400700 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 0x5571b0bd20a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x5571b0bd0ea0 tx=0x7fcab4004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6eff9d1ae37758b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:47.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.735+0000 7fcab2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:47.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.736+0000 7fcab2000700 1 -- 172.21.15.63:0/1157475616 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4003890 con 0x5571b0bc7820 2024-05-27T07:51:47.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.736+0000 7fcab2000700 1 -- 172.21.15.63:0/1157475616 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7fcab401a040 con 0x5571b0bc7820 2024-05-27T07:51:47.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.736+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1157475616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab40044b0 con 0x5571b0bc7820 2024-05-27T07:51:47.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.736+0000 7fcab2000700 1 -- 172.21.15.63:0/1157475616 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcab4032050 con 0x5571b0bc7820 2024-05-27T07:51:47.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.737+0000 7fcadd5a2b80 10 client.18918 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:47.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.737+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1157475616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5571b0ab6060 con 0x5571b0bc7820 2024-05-27T07:51:47.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.737+0000 7fcab2000700 1 -- 172.21.15.63:0/1157475616 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcab40044b0 con 0x5571b0bc7820 2024-05-27T07:51:47.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.737+0000 7fcab2000700 1 -- 172.21.15.63:0/1157475616 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcab400f9e0 con 0x5571b0bc7820 2024-05-27T07:51:47.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.737+0000 7fcadd5a2b80 10 client.18918 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:47.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.737+0000 7fcadd5a2b80 10 client.18918 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:51:47.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.737+0000 7fcadd5a2b80 20 client.18918 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:47.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.738+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5571b0be17e0 0x5571b0be3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.738+0000 7fcadd5a2b80 4 client.18918 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:47.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.738+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1157475616 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571b0af7d90 con 0x5571b0be17e0 2024-05-27T07:51:47.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.739+0000 7fcac7e00700 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5571b0be17e0 0x5571b0be3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.740+0000 7fcac7e00700 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5571b0be17e0 0x5571b0be3c00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6a61fef2e5f4037 server_cookie=aab03044ffa3ff38 in_seq=0 out_seq=0 2024-05-27T07:51:47.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.740+0000 7fcab2000700 10 client.18918 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:51:47.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.742+0000 7fcab2000700 1 -- 172.21.15.63:0/1157475616 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5571b0af7d90 con 0x5571b0be17e0 2024-05-27T07:51:47.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.742+0000 7fcab2000700 10 client.18918 handle_command_reply: tid=0 2024-05-27T07:51:47.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.742+0000 7fcadd5a2b80 1 client.18918 shutdown 2024-05-27T07:51:47.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.742+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1157475616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 msgr2=0x5571b0bd20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:47.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.742+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 0x5571b0bd20a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x5571b0bd0ea0 tx=0x7fcab4004d30 comp rx=0 tx=0).stop 2024-05-27T07:51:47.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.742+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1157475616 shutdown_connections 2024-05-27T07:51:47.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.743+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5571b0be17e0 0x5571b0be3c00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.743+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0bc7820 0x5571b0bd20a0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.743+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0a48e40 0x5571b0bd1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.743+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1157475616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.743+0000 7fcab2000700 0 client.18918 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:51:47.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.743+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1157475616 >> 172.21.15.63:0/1157475616 conn(0x5571b0aa8fc0 msgr2=0x5571b0bc1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:47.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.743+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1157475616 shutdown_connections 2024-05-27T07:51:47.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.743+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1157475616 wait complete. 2024-05-27T07:51:47.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.743+0000 7fcadd5a2b80 20 client.18918 trim_cache size 0 max 16384 2024-05-27T07:51:47.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.768+0000 7fcadd5a2b80 1 Processor -- start 2024-05-27T07:51:47.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.768+0000 7fcadd5a2b80 1 -- start start 2024-05-27T07:51:47.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.768+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bd2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.769+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bc7410 0x5571b0bcb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.769+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bcced0 0x5571b0bdda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.769+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b4db0 con 0x5571b0a48e40 2024-05-27T07:51:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.769+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b2640 con 0x5571b0bc7410 2024-05-27T07:51:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.769+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b04b0 con 0x5571b0bcced0 2024-05-27T07:51:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.769+0000 7fcac7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bd2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.769+0000 7fcac7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bd2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41616/0 (socket says 172.21.15.63:41616) 2024-05-27T07:51:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.769+0000 7fcac7400700 1 -- 172.21.15.63:0/3111523455 learned_addr learned my addr 172.21.15.63:0/3111523455 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.769+0000 7fcac7e00700 1 --2- 172.21.15.63:0/3111523455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bc7410 0x5571b0bcb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.770+0000 7fcac7400700 1 -- 172.21.15.63:0/3111523455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bcced0 msgr2=0x5571b0bdda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.770+0000 7fcac7400700 1 --2- 172.21.15.63:0/3111523455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bcced0 0x5571b0bdda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.770+0000 7fcac7400700 1 -- 172.21.15.63:0/3111523455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bc7410 msgr2=0x5571b0bcb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.770+0000 7fcac7400700 1 --2- 172.21.15.63:0/3111523455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bc7410 0x5571b0bcb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.770+0000 7fcac7400700 1 -- 172.21.15.63:0/3111523455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b0bc4350 con 0x5571b0a48e40 2024-05-27T07:51:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.770+0000 7fcac7e00700 1 --2- 172.21.15.63:0/3111523455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bc7410 0x5571b0bcb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.770+0000 7fcac7400700 1 --2- 172.21.15.63:0/3111523455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bd2010 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcab40049d0 tx=0x7fcab4011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e80fbc9322300d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.770+0000 7fcabcc00700 1 -- 172.21.15.63:0/3111523455 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab403c070 con 0x5571b0a48e40 2024-05-27T07:51:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.770+0000 7fcabcc00700 1 -- 172.21.15.63:0/3111523455 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcab400ab00 con 0x5571b0a48e40 2024-05-27T07:51:47.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.770+0000 7fcabcc00700 1 -- 172.21.15.63:0/3111523455 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4004ab0 con 0x5571b0a48e40 2024-05-27T07:51:47.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.772+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/3111523455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 msgr2=0x5571b0bd2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:47.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.772+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/3111523455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bd2010 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcab40049d0 tx=0x7fcab4011e40 comp rx=0 tx=0).stop 2024-05-27T07:51:47.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.772+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/3111523455 shutdown_connections 2024-05-27T07:51:47.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.772+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/3111523455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bcced0 0x5571b0bdda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.772+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/3111523455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bd2010 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.772+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/3111523455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bc7410 0x5571b0bcb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.772+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/3111523455 >> 172.21.15.63:0/3111523455 conn(0x5571b0aa8fc0 msgr2=0x5571b0bcfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:47.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.772+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/3111523455 shutdown_connections 2024-05-27T07:51:47.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.772+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/3111523455 wait complete. 2024-05-27T07:51:47.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.773+0000 7fcadd5a2b80 1 -- start start 2024-05-27T07:51:47.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.773+0000 7fcadd5a2b80 1 Processor -- start 2024-05-27T07:51:47.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bda390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcadd5a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bcced0 0x5571b0bd5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b0b00 con 0x5571b0a48e40 2024-05-27T07:51:47.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b1f70 con 0x5571b0bcced0 2024-05-27T07:51:47.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcadd5a2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b11d0 con 0x5571b0bc7410 2024-05-27T07:51:47.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcadd5a2b80 10 client.0 _ll_register_callbacks cb 0x5571b0a8fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:47.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcac7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bda390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcac7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcac7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bda390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41622/0 (socket says 172.21.15.63:41622) 2024-05-27T07:51:47.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.774+0000 7fcac7400700 1 -- 172.21.15.63:0/1428965923 learned_addr learned my addr 172.21.15.63:0/1428965923 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:47.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.775+0000 7fcac7400700 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bcced0 0x5571b0bd5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.775+0000 7fcac7400700 1 -- 172.21.15.63:0/1428965923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 msgr2=0x5571b0bd56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:47.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.775+0000 7fcac7400700 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.775+0000 7fcac7400700 1 -- 172.21.15.63:0/1428965923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bcced0 msgr2=0x5571b0bd5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:47.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.775+0000 7fcac7400700 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bcced0 0x5571b0bd5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.775+0000 7fcac7400700 1 -- 172.21.15.63:0/1428965923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b0bd63d0 con 0x5571b0a48e40 2024-05-27T07:51:47.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.775+0000 7fcac7400700 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bda390 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fcab400b010 tx=0x7fcab400a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e802c75a7cdff85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:47.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.775+0000 7fcab2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:47.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.776+0000 7fcab2000700 1 -- 172.21.15.63:0/1428965923 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab403c070 con 0x5571b0a48e40 2024-05-27T07:51:47.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.776+0000 7fcab2000700 1 -- 172.21.15.63:0/1428965923 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7fcab4003de0 con 0x5571b0a48e40 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.776+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1428965923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab400ab00 con 0x5571b0a48e40 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.776+0000 7fcab2000700 1 -- 172.21.15.63:0/1428965923 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcab401e470 con 0x5571b0a48e40 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.776+0000 7fcadd5a2b80 10 client.18924 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.776+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1428965923 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5571b0c4c010 con 0x5571b0a48e40 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.777+0000 7fcab2000700 1 -- 172.21.15.63:0/1428965923 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcab400ab00 con 0x5571b0a48e40 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.777+0000 7fcab2000700 1 -- 172.21.15.63:0/1428965923 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcab4037600 con 0x5571b0a48e40 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.777+0000 7fcadd5a2b80 10 client.18924 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.777+0000 7fcadd5a2b80 10 client.18924 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.777+0000 7fcadd5a2b80 20 client.18924 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.777+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5571b0c4c550 0x5571b0c4e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.777+0000 7fcadd5a2b80 4 client.18924 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.777+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1428965923 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571b0af1570 con 0x5571b0c4c550 2024-05-27T07:51:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.778+0000 7fcac7e00700 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5571b0c4c550 0x5571b0c4e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:47.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.779+0000 7fcac7e00700 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5571b0c4c550 0x5571b0c4e970 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14e9137cd058f03c server_cookie=b4f20c76aa5b048c in_seq=0 out_seq=0 2024-05-27T07:51:47.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.779+0000 7fcab2000700 10 client.18924 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:51:47.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcab2000700 1 -- 172.21.15.63:0/1428965923 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571b0af1570 con 0x5571b0c4c550 2024-05-27T07:51:47.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcab2000700 10 client.18924 handle_command_reply: tid=0 2024-05-27T07:51:47.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 client.18924 shutdown 2024-05-27T07:51:47.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1428965923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 msgr2=0x5571b0bda390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:47.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bda390 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fcab400b010 tx=0x7fcab400a400 comp rx=0 tx=0).stop 2024-05-27T07:51:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1428965923 shutdown_connections 2024-05-27T07:51:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5571b0bcced0 0x5571b0bd5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5571b0c4c550 0x5571b0c4e970 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5571b0a48e40 0x5571b0bda390 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 --2- 172.21.15.63:0/1428965923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571b0bc7410 0x5571b0bd56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1428965923 >> 172.21.15.63:0/1428965923 conn(0x5571b0aa8fc0 msgr2=0x5571b0bcc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcab2000700 0 client.18924 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:51:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1428965923 shutdown_connections 2024-05-27T07:51:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 1 -- 172.21.15.63:0/1428965923 wait complete. 2024-05-27T07:51:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.780+0000 7fcadd5a2b80 20 client.18924 trim_cache size 0 max 16384 2024-05-27T07:51:47.783 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:51:47.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.781+0000 7fcacd400700 1 -- 172.21.15.63:0/1658978409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaac0b29f0 msgr2=0x7fcaac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:47.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.781+0000 7fcacd400700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaac0b29f0 0x7fcaac0b4f00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcac000bfd0 tx=0x7fcac0004ad0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.781+0000 7fcacd400700 1 -- 172.21.15.63:0/1658978409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac810e730 msgr2=0x7fcac8103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:47.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.781+0000 7fcacd400700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac810e730 0x7fcac8103a70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fcac000b970 tx=0x7fcac000bc80 comp rx=0 tx=0).stop 2024-05-27T07:51:47.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.782+0000 7fcacd400700 1 -- 172.21.15.63:0/1658978409 shutdown_connections 2024-05-27T07:51:47.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.782+0000 7fcacd400700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaac0b29f0 0x7fcaac0b4f00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.782+0000 7fcacd400700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac810e730 0x7fcac8103a70 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.782+0000 7fcacd400700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac8105300 0x7fcac8102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.782+0000 7fcacd400700 1 --2- 172.21.15.63:0/1658978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac8105c50 0x7fcac8103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:47.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.782+0000 7fcacd400700 1 -- 172.21.15.63:0/1658978409 >> 172.21.15.63:0/1658978409 conn(0x7fcac8078ab0 msgr2=0x7fcac80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:47.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.782+0000 7fcacd400700 1 -- 172.21.15.63:0/1658978409 shutdown_connections 2024-05-27T07:51:47.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:47.782+0000 7fcacd400700 1 -- 172.21.15.63:0/1658978409 wait complete. 2024-05-27T07:51:48.760 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:51:49.286 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:49.296 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:49 smithi063 ceph-mon[31613]: pgmap v574: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 16 op/s 2024-05-27T07:51:49.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:49 smithi086 ceph-mon[34687]: pgmap v574: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 16 op/s 2024-05-27T07:51:49.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:49 smithi149 ceph-mon[35408]: pgmap v574: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 16 op/s 2024-05-27T07:51:51.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:51 smithi063 ceph-mon[31613]: pgmap v575: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:51:51.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:51 smithi086 ceph-mon[34687]: pgmap v575: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:51:51.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:51 smithi149 ceph-mon[35408]: pgmap v575: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:51:51.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.672+0000 7f7449200700 1 -- 172.21.15.63:0/2056894076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74441043f0 msgr2=0x7f7444104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:51.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.672+0000 7f7449200700 1 --2- 172.21.15.63:0/2056894076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74441043f0 0x7f7444104800 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f74340099c0 tx=0x7f7434009cd0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.672+0000 7f7449200700 1 -- 172.21.15.63:0/2056894076 shutdown_connections 2024-05-27T07:51:51.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.672+0000 7f7449200700 1 --2- 172.21.15.63:0/2056894076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7444109670 0x7f7444110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.672+0000 7f7449200700 1 --2- 172.21.15.63:0/2056894076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74441043f0 0x7f7444104800 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.672+0000 7f7449200700 1 --2- 172.21.15.63:0/2056894076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7444108d20 0x7f7444109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.672+0000 7f7449200700 1 -- 172.21.15.63:0/2056894076 >> 172.21.15.63:0/2056894076 conn(0x7f7444079d00 msgr2=0x7f744407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:51.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.672+0000 7f7449200700 1 -- 172.21.15.63:0/2056894076 shutdown_connections 2024-05-27T07:51:51.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.672+0000 7f7449200700 1 -- 172.21.15.63:0/2056894076 wait complete. 2024-05-27T07:51:51.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.673+0000 7f7449200700 1 Processor -- start 2024-05-27T07:51:51.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.674+0000 7f7449200700 1 -- start start 2024-05-27T07:51:51.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.674+0000 7f7449200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74441043f0 0x7f7444075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.674+0000 7f7449200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444108d20 0x7f7444075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.674+0000 7f7449200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7444109670 0x7f7444076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.674+0000 7f7449200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74441141a0 con 0x7f7444108d20 2024-05-27T07:51:51.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.674+0000 7f7449200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7444114020 con 0x7f7444109670 2024-05-27T07:51:51.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.674+0000 7f7449200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7444114320 con 0x7f74441043f0 2024-05-27T07:51:51.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.675+0000 7f7442a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444108d20 0x7f7444075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.675+0000 7f7442a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444108d20 0x7f7444075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46292/0 (socket says 172.21.15.63:46292) 2024-05-27T07:51:51.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.675+0000 7f7442a00700 1 -- 172.21.15.63:0/3662268405 learned_addr learned my addr 172.21.15.63:0/3662268405 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:51.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.675+0000 7f7443400700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74441043f0 0x7f7444075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.675+0000 7f7443e00700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7444109670 0x7f7444076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.675+0000 7f7442a00700 1 -- 172.21.15.63:0/3662268405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74441043f0 msgr2=0x7f7444075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:51.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.675+0000 7f7442a00700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74441043f0 0x7f7444075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.675+0000 7f7442a00700 1 -- 172.21.15.63:0/3662268405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7444109670 msgr2=0x7f7444076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:51.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.675+0000 7f7442a00700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7444109670 0x7f7444076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.675+0000 7f7442a00700 1 -- 172.21.15.63:0/3662268405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74441a8f30 con 0x7f7444108d20 2024-05-27T07:51:51.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.676+0000 7f7442a00700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444108d20 0x7f7444075ff0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f743000d980 tx=0x7f743000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:51.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.676+0000 7f743be00700 1 -- 172.21.15.63:0/3662268405 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430017070 con 0x7f7444108d20 2024-05-27T07:51:51.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.676+0000 7f743be00700 1 -- 172.21.15.63:0/3662268405 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7430010460 con 0x7f7444108d20 2024-05-27T07:51:51.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.676+0000 7f743be00700 1 -- 172.21.15.63:0/3662268405 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743000de60 con 0x7f7444108d20 2024-05-27T07:51:51.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.676+0000 7f7449200700 1 -- 172.21.15.63:0/3662268405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74441a9220 con 0x7f7444108d20 2024-05-27T07:51:51.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.676+0000 7f7449200700 1 -- 172.21.15.63:0/3662268405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74441a97b0 con 0x7f7444108d20 2024-05-27T07:51:51.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.682+0000 7f743be00700 1 -- 172.21.15.63:0/3662268405 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7430010610 con 0x7f7444108d20 2024-05-27T07:51:51.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.682+0000 7f7459305b80 1 Processor -- start 2024-05-27T07:51:51.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.683+0000 7f7459305b80 1 -- start start 2024-05-27T07:51:51.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.683+0000 7f7459305b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0359f0 0x5589db0c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.683+0000 7f7459305b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.683+0000 7f7459305b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 0x5589db0d56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.683+0000 7f7459305b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5589daff6e40 con 0x5589daf49e40 2024-05-27T07:51:51.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.683+0000 7f7459305b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589dafe69e0 con 0x5589db0d32c0 2024-05-27T07:51:51.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.683+0000 7f7459305b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5589dafeaa20 con 0x5589db0359f0 2024-05-27T07:51:51.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f7443400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f7443400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46308/0 (socket says 172.21.15.63:46308) 2024-05-27T07:51:51.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f7443400700 1 -- 172.21.15.63:0/4212004350 learned_addr learned my addr 172.21.15.63:0/4212004350 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:51.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f743be00700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f74280b2a70 0x7f74280b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f743be00700 1 -- 172.21.15.63:0/3662268405 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f74300e24c0 con 0x7f7444108d20 2024-05-27T07:51:51.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f7442a00700 1 --2- 172.21.15.63:0/4212004350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 0x5589db0d56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f7443400700 1 -- 172.21.15.63:0/4212004350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0359f0 msgr2=0x5589db0c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:51.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f7443400700 1 --2- 172.21.15.63:0/4212004350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0359f0 0x5589db0c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f7443400700 1 -- 172.21.15.63:0/4212004350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 msgr2=0x5589db0d56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:51.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f7443400700 1 --2- 172.21.15.63:0/4212004350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 0x5589db0d56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f7443e00700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f74280b2a70 0x7f74280b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.684+0000 7f7443400700 1 -- 172.21.15.63:0/4212004350 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589dafaf420 con 0x5589daf49e40 2024-05-27T07:51:51.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.685+0000 7f7443400700 1 --2- 172.21.15.63:0/4212004350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0c9c90 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f74340099c0 tx=0x7f7434009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955f390fe7bc1363 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:51.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.685+0000 7f7438c00700 1 -- 172.21.15.63:0/4212004350 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434026070 con 0x5589daf49e40 2024-05-27T07:51:51.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.685+0000 7f7438c00700 1 -- 172.21.15.63:0/4212004350 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7434005600 con 0x5589daf49e40 2024-05-27T07:51:51.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.685+0000 7f7438c00700 1 -- 172.21.15.63:0/4212004350 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74340038e0 con 0x5589daf49e40 2024-05-27T07:51:51.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.686+0000 7f7459305b80 1 -- 172.21.15.63:0/4212004350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 msgr2=0x5589db0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.686+0000 7f7459305b80 1 --2- 172.21.15.63:0/4212004350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0c9c90 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f74340099c0 tx=0x7f7434009710 comp rx=0 tx=0).stop 2024-05-27T07:51:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.686+0000 7f7443e00700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f74280b2a70 0x7f74280b4f80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f743c009550 tx=0x7f743c009050 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.687+0000 7f7459305b80 1 -- 172.21.15.63:0/4212004350 shutdown_connections 2024-05-27T07:51:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.687+0000 7f7459305b80 1 --2- 172.21.15.63:0/4212004350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 0x5589db0d56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.687+0000 7f7459305b80 1 --2- 172.21.15.63:0/4212004350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0359f0 0x5589db0c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.687+0000 7f7459305b80 1 --2- 172.21.15.63:0/4212004350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0c9c90 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.687+0000 7f7459305b80 1 -- 172.21.15.63:0/4212004350 >> 172.21.15.63:0/4212004350 conn(0x5589dafe3390 msgr2=0x5589db0c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.687+0000 7f7459305b80 1 -- 172.21.15.63:0/4212004350 shutdown_connections 2024-05-27T07:51:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.687+0000 7f7459305b80 1 -- 172.21.15.63:0/4212004350 wait complete. 2024-05-27T07:51:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.688+0000 7f7459305b80 1 -- start start 2024-05-27T07:51:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.688+0000 7f7459305b80 1 Processor -- start 2024-05-27T07:51:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7459305b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589daf49e40 0x5589db0d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7459305b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0359f0 0x5589db0d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7459305b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 0x5589db0d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7459305b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5589daeecfa0 con 0x5589db0359f0 2024-05-27T07:51:51.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7459305b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589daf87760 con 0x5589db0d32c0 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7459305b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5589db0d5da0 con 0x5589daf49e40 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7459305b80 10 client.0 _ll_register_callbacks cb 0x5589daf90c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7442a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0359f0 0x5589db0d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7442a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0359f0 0x5589db0d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46312/0 (socket says 172.21.15.63:46312) 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7442a00700 1 -- 172.21.15.63:0/3606471695 learned_addr learned my addr 172.21.15.63:0/3606471695 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7443400700 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589daf49e40 0x5589db0d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.689+0000 7f7443e00700 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 0x5589db0d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.690+0000 7f7442a00700 1 -- 172.21.15.63:0/3606471695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589daf49e40 msgr2=0x5589db0d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.690+0000 7f7442a00700 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589daf49e40 0x5589db0d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.690+0000 7f7442a00700 1 -- 172.21.15.63:0/3606471695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 msgr2=0x5589db0d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.690+0000 7f7442a00700 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 0x5589db0d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.690+0000 7f7442a00700 1 -- 172.21.15.63:0/3606471695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589daf368d0 con 0x5589db0359f0 2024-05-27T07:51:51.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.690+0000 7f7443e00700 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 0x5589db0d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:51.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.690+0000 7f7442a00700 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0359f0 0x5589db0d07f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f743000d950 tx=0x7f7430010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec5405f14f63dcb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:51.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.690+0000 7f742d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:51.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.690+0000 7f742d600700 1 -- 172.21.15.63:0/3606471695 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430020040 con 0x5589db0359f0 2024-05-27T07:51:51.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.690+0000 7f742d600700 1 -- 172.21.15.63:0/3606471695 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f74300b1fc0 con 0x5589db0359f0 2024-05-27T07:51:51.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.691+0000 7f7459305b80 1 -- 172.21.15.63:0/3606471695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7434005600 con 0x5589db0359f0 2024-05-27T07:51:51.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.691+0000 7f742d600700 1 -- 172.21.15.63:0/3606471695 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f74300a93e0 con 0x5589db0359f0 2024-05-27T07:51:51.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.691+0000 7f7459305b80 10 client.18936 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:51.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.691+0000 7f7459305b80 1 -- 172.21.15.63:0/3606471695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5589dafb7060 con 0x5589db0359f0 2024-05-27T07:51:51.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.691+0000 7f742d600700 1 -- 172.21.15.63:0/3606471695 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7434005600 con 0x5589db0359f0 2024-05-27T07:51:51.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.691+0000 7f742d600700 1 -- 172.21.15.63:0/3606471695 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f74300a8640 con 0x5589db0359f0 2024-05-27T07:51:51.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.692+0000 7f7459305b80 10 client.18936 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:51.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.692+0000 7f7459305b80 10 client.18936 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:51:51.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.692+0000 7f7459305b80 20 client.18936 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:51.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.692+0000 7f7459305b80 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5589db0e2c30 0x5589db0e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.692+0000 7f7459305b80 4 client.18936 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:51.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.692+0000 7f7459305b80 1 -- 172.21.15.63:0/3606471695 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589daff8d90 con 0x5589db0e2c30 2024-05-27T07:51:51.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.692+0000 7f7443400700 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5589db0e2c30 0x5589db0e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.694+0000 7f7443400700 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5589db0e2c30 0x5589db0e5050 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16780752ea71efb1 server_cookie=8733bd070eb18f40 in_seq=0 out_seq=0 2024-05-27T07:51:51.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.694+0000 7f742d600700 10 client.18936 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:51:51.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f742d600700 1 -- 172.21.15.63:0/3606471695 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5589daff8d90 con 0x5589db0e2c30 2024-05-27T07:51:51.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f742d600700 10 client.18936 handle_command_reply: tid=0 2024-05-27T07:51:51.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f7459305b80 1 client.18936 shutdown 2024-05-27T07:51:51.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f7459305b80 1 -- 172.21.15.63:0/3606471695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0359f0 msgr2=0x5589db0d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f7459305b80 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0359f0 0x5589db0d07f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f743000d950 tx=0x7f7430010c10 comp rx=0 tx=0).stop 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f7459305b80 1 -- 172.21.15.63:0/3606471695 shutdown_connections 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f7459305b80 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0d32c0 0x5589db0d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f7459305b80 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589daf49e40 0x5589db0d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f7459305b80 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5589db0e2c30 0x5589db0e5050 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f7459305b80 1 --2- 172.21.15.63:0/3606471695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0359f0 0x5589db0d07f0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f7459305b80 1 -- 172.21.15.63:0/3606471695 >> 172.21.15.63:0/3606471695 conn(0x5589dafe3390 msgr2=0x5589db0c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f742d600700 0 client.18936 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.696+0000 7f7459305b80 1 -- 172.21.15.63:0/3606471695 shutdown_connections 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.697+0000 7f7459305b80 1 -- 172.21.15.63:0/3606471695 wait complete. 2024-05-27T07:51:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.697+0000 7f7459305b80 20 client.18936 trim_cache size 0 max 16384 2024-05-27T07:51:51.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.714+0000 7f7459305b80 1 Processor -- start 2024-05-27T07:51:51.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.714+0000 7f7459305b80 1 -- start start 2024-05-27T07:51:51.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.714+0000 7f7459305b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.715+0000 7f7459305b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 0x5589db0c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.715+0000 7f7459305b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0d32c0 0x5589db0dee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.715+0000 7f7459305b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74441b4e10 con 0x5589daf49e40 2024-05-27T07:51:51.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.715+0000 7f7459305b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74441b26a0 con 0x5589db0359f0 2024-05-27T07:51:51.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.715+0000 7f7459305b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74441b0510 con 0x5589db0d32c0 2024-05-27T07:51:51.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.715+0000 7f7443400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.715+0000 7f7443400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46320/0 (socket says 172.21.15.63:46320) 2024-05-27T07:51:51.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.715+0000 7f7443400700 1 -- 172.21.15.63:0/3765962137 learned_addr learned my addr 172.21.15.63:0/3765962137 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:51.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.715+0000 7f7442a00700 1 --2- 172.21.15.63:0/3765962137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 0x5589db0c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.715+0000 7f7443e00700 1 --2- 172.21.15.63:0/3765962137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0d32c0 0x5589db0dee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.716+0000 7f7443400700 1 -- 172.21.15.63:0/3765962137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0d32c0 msgr2=0x5589db0dee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:51.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.716+0000 7f7443400700 1 --2- 172.21.15.63:0/3765962137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0d32c0 0x5589db0dee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.716+0000 7f7443400700 1 -- 172.21.15.63:0/3765962137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 msgr2=0x5589db0c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:51.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.716+0000 7f7443400700 1 --2- 172.21.15.63:0/3765962137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 0x5589db0c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.716+0000 7f7443400700 1 -- 172.21.15.63:0/3765962137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589db0d30b0 con 0x5589daf49e40 2024-05-27T07:51:51.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.716+0000 7f7443400700 1 --2- 172.21.15.63:0/3765962137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0cf210 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f74340058e0 tx=0x7f74340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36afb35d37936cec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:51.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.717+0000 7f742fe00700 1 -- 172.21.15.63:0/3765962137 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434026040 con 0x5589daf49e40 2024-05-27T07:51:51.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.717+0000 7f742fe00700 1 -- 172.21.15.63:0/3765962137 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7434027040 con 0x5589daf49e40 2024-05-27T07:51:51.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.717+0000 7f742fe00700 1 -- 172.21.15.63:0/3765962137 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434003a80 con 0x5589daf49e40 2024-05-27T07:51:51.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.718+0000 7f7459305b80 1 -- 172.21.15.63:0/3765962137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 msgr2=0x5589db0cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:51.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.718+0000 7f7459305b80 1 --2- 172.21.15.63:0/3765962137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0cf210 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f74340058e0 tx=0x7f74340057f0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.718+0000 7f7459305b80 1 -- 172.21.15.63:0/3765962137 shutdown_connections 2024-05-27T07:51:51.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.718+0000 7f7459305b80 1 --2- 172.21.15.63:0/3765962137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0d32c0 0x5589db0dee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.718+0000 7f7459305b80 1 --2- 172.21.15.63:0/3765962137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589daf49e40 0x5589db0cf210 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.718+0000 7f7459305b80 1 --2- 172.21.15.63:0/3765962137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 0x5589db0c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.718+0000 7f7459305b80 1 -- 172.21.15.63:0/3765962137 >> 172.21.15.63:0/3765962137 conn(0x5589dafe3390 msgr2=0x5589db0d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:51.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.718+0000 7f7459305b80 1 -- 172.21.15.63:0/3765962137 shutdown_connections 2024-05-27T07:51:51.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.718+0000 7f7459305b80 1 -- 172.21.15.63:0/3765962137 wait complete. 2024-05-27T07:51:51.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.719+0000 7f7459305b80 1 -- start start 2024-05-27T07:51:51.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.719+0000 7f7459305b80 1 Processor -- start 2024-05-27T07:51:51.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.720+0000 7f7459305b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 0x5589db0db560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.720+0000 7f7459305b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0d32c0 0x5589db0d6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.720+0000 7f7459305b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0d6e50 0x5589db0d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.720+0000 7f7459305b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74441b0b60 con 0x5589db0d32c0 2024-05-27T07:51:51.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.720+0000 7f7459305b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74441b1fd0 con 0x5589db0359f0 2024-05-27T07:51:51.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.720+0000 7f7459305b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74441b1230 con 0x5589db0d6e50 2024-05-27T07:51:51.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.720+0000 7f7459305b80 10 client.0 _ll_register_callbacks cb 0x5589daf90c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:51.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.720+0000 7f7442a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0d32c0 0x5589db0d6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.720+0000 7f7442a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0d32c0 0x5589db0d6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46336/0 (socket says 172.21.15.63:46336) 2024-05-27T07:51:51.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.720+0000 7f7442a00700 1 -- 172.21.15.63:0/3367471335 learned_addr learned my addr 172.21.15.63:0/3367471335 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:51.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.721+0000 7f7443400700 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 0x5589db0db560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.721+0000 7f7442a00700 1 -- 172.21.15.63:0/3367471335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0d6e50 msgr2=0x5589db0d7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.721+0000 7f7442a00700 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0d6e50 0x5589db0d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.721+0000 7f7442a00700 1 -- 172.21.15.63:0/3367471335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 msgr2=0x5589db0db560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.721+0000 7f7442a00700 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 0x5589db0db560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.721+0000 7f7442a00700 1 -- 172.21.15.63:0/3367471335 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589db14c6b0 con 0x5589db0d32c0 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.721+0000 7f7443400700 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 0x5589db0db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.721+0000 7f7442a00700 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0d32c0 0x5589db0d6910 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f74300b0b30 tx=0x7f74300a9050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c351cab1506cd22 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.721+0000 7f742d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.722+0000 7f742d600700 1 -- 172.21.15.63:0/3367471335 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74300c7030 con 0x5589db0d32c0 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.722+0000 7f742d600700 1 -- 172.21.15.63:0/3367471335 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f74300b1fc0 con 0x5589db0d32c0 2024-05-27T07:51:51.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.722+0000 7f7459305b80 1 -- 172.21.15.63:0/3367471335 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7434027040 con 0x5589db0d32c0 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.722+0000 7f742d600700 1 -- 172.21.15.63:0/3367471335 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f743008a050 con 0x5589db0d32c0 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.722+0000 7f7459305b80 10 client.18942 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.722+0000 7f7459305b80 1 -- 172.21.15.63:0/3367471335 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5589db14d7f0 con 0x5589db0d32c0 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.722+0000 7f742d600700 1 -- 172.21.15.63:0/3367471335 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7434027040 con 0x5589db0d32c0 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.723+0000 7f742d600700 1 -- 172.21.15.63:0/3367471335 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f743008f440 con 0x5589db0d32c0 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.723+0000 7f7459305b80 10 client.18942 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.723+0000 7f7459305b80 10 client.18942 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.723+0000 7f7459305b80 20 client.18942 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.723+0000 7f7459305b80 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5589db14de40 0x5589db150260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.723+0000 7f7459305b80 4 client.18942 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:51:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.723+0000 7f7459305b80 1 -- 172.21.15.63:0/3367471335 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589daff2570 con 0x5589db14de40 2024-05-27T07:51:51.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.723+0000 7f7443400700 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5589db14de40 0x5589db150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:51.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.724+0000 7f742d600700 10 client.18942 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:51:51.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.724+0000 7f7443400700 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5589db14de40 0x5589db150260 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f95248a52cf98238 server_cookie=16889c396b6eede3 in_seq=0 out_seq=0 2024-05-27T07:51:51.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f742d600700 1 -- 172.21.15.63:0/3367471335 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589daff2570 con 0x5589db14de40 2024-05-27T07:51:51.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f742d600700 10 client.18942 handle_command_reply: tid=0 2024-05-27T07:51:51.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f7459305b80 1 client.18942 shutdown 2024-05-27T07:51:51.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f7459305b80 1 -- 172.21.15.63:0/3367471335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0d32c0 msgr2=0x5589db0d6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:51.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f7459305b80 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0d32c0 0x5589db0d6910 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f74300b0b30 tx=0x7f74300a9050 comp rx=0 tx=0).stop 2024-05-27T07:51:51.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f7459305b80 1 -- 172.21.15.63:0/3367471335 shutdown_connections 2024-05-27T07:51:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f7459305b80 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589db0d6e50 0x5589db0d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f7459305b80 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589db0359f0 0x5589db0db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f7459305b80 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5589db14de40 0x5589db150260 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f7459305b80 1 --2- 172.21.15.63:0/3367471335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589db0d32c0 0x5589db0d6910 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f7459305b80 1 -- 172.21.15.63:0/3367471335 >> 172.21.15.63:0/3367471335 conn(0x5589dafe3390 msgr2=0x5589db0c88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.725+0000 7f742d600700 0 client.18942 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:51:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.726+0000 7f7459305b80 1 -- 172.21.15.63:0/3367471335 shutdown_connections 2024-05-27T07:51:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.726+0000 7f7459305b80 1 -- 172.21.15.63:0/3367471335 wait complete. 2024-05-27T07:51:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.726+0000 7f7459305b80 20 client.18942 trim_cache size 0 max 16384 2024-05-27T07:51:51.729 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:51:51.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 -- 172.21.15.63:0/3662268405 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f74280b2a70 msgr2=0x7f74280b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:51.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f74280b2a70 0x7f74280b4f80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f743c009550 tx=0x7f743c009050 comp rx=0 tx=0).stop 2024-05-27T07:51:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 -- 172.21.15.63:0/3662268405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444108d20 msgr2=0x7f7444075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444108d20 0x7f7444075ff0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f743000d980 tx=0x7f743000dd40 comp rx=0 tx=0).stop 2024-05-27T07:51:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 -- 172.21.15.63:0/3662268405 shutdown_connections 2024-05-27T07:51:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f74280b2a70 0x7f74280b4f80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7444109670 0x7f7444076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74441043f0 0x7f7444075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 --2- 172.21.15.63:0/3662268405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444108d20 0x7f7444075ff0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 -- 172.21.15.63:0/3662268405 >> 172.21.15.63:0/3662268405 conn(0x7f7444079d00 msgr2=0x7f7444101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 -- 172.21.15.63:0/3662268405 shutdown_connections 2024-05-27T07:51:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:51.727+0000 7f7449200700 1 -- 172.21.15.63:0/3662268405 wait complete. 2024-05-27T07:51:52.756 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:51:53.279 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:53.443 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:53 smithi063 ceph-mon[31613]: pgmap v576: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:51:53.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:53 smithi086 ceph-mon[34687]: pgmap v576: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:51:53.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:53 smithi149 ceph-mon[35408]: pgmap v576: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:51:55.169 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:55 smithi063 ceph-mon[31613]: pgmap v577: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:51:55.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:55 smithi086 ceph-mon[34687]: pgmap v577: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:51:55.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:55 smithi149 ceph-mon[35408]: pgmap v577: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:51:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.828+0000 7fcbb2c00700 1 -- 172.21.15.63:0/333991783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbac10b2a0 msgr2=0x7fcbac117820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.828+0000 7fcbb2c00700 1 --2- 172.21.15.63:0/333991783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbac10b2a0 0x7fcbac117820 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fcba400b0a0 tx=0x7fcba400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.829+0000 7fcbb2c00700 1 -- 172.21.15.63:0/333991783 shutdown_connections 2024-05-27T07:51:55.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.829+0000 7fcbb2c00700 1 --2- 172.21.15.63:0/333991783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbac10b2a0 0x7fcbac117820 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.829+0000 7fcbb2c00700 1 --2- 172.21.15.63:0/333991783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac110980 0x7fcbac110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.829+0000 7fcbb2c00700 1 --2- 172.21.15.63:0/333991783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbac10a950 0x7fcbac10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.829+0000 7fcbb2c00700 1 -- 172.21.15.63:0/333991783 >> 172.21.15.63:0/333991783 conn(0x7fcbac076d20 msgr2=0x7fcbac077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:55.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.829+0000 7fcbb2c00700 1 -- 172.21.15.63:0/333991783 shutdown_connections 2024-05-27T07:51:55.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.829+0000 7fcbb2c00700 1 -- 172.21.15.63:0/333991783 wait complete. 2024-05-27T07:51:55.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.830+0000 7fcbb2c00700 1 Processor -- start 2024-05-27T07:51:55.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.831+0000 7fcbb2c00700 1 -- start start 2024-05-27T07:51:55.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.831+0000 7fcbb2c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbac10a950 0x7fcbac1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.831+0000 7fcbb2c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbac10b2a0 0x7fcbac1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.831+0000 7fcbb2c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac110980 0x7fcbac1a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.831+0000 7fcbb2c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac11a950 con 0x7fcbac10a950 2024-05-27T07:51:55.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.831+0000 7fcbb2c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac11a7d0 con 0x7fcbac110980 2024-05-27T07:51:55.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.831+0000 7fcbb2c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac11aad0 con 0x7fcbac10b2a0 2024-05-27T07:51:55.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.832+0000 7fcbabe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbac10a950 0x7fcbac1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.832+0000 7fcbabe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbac10a950 0x7fcbac1a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46350/0 (socket says 172.21.15.63:46350) 2024-05-27T07:51:55.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.832+0000 7fcbabe00700 1 -- 172.21.15.63:0/3205226890 learned_addr learned my addr 172.21.15.63:0/3205226890 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.832+0000 7fcbb0c00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac110980 0x7fcbac1a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.832+0000 7fcbab400700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbac10b2a0 0x7fcbac1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.832+0000 7fcbabe00700 1 -- 172.21.15.63:0/3205226890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbac10b2a0 msgr2=0x7fcbac1a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.832+0000 7fcbabe00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbac10b2a0 0x7fcbac1a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.832+0000 7fcbabe00700 1 -- 172.21.15.63:0/3205226890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac110980 msgr2=0x7fcbac1a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.832+0000 7fcbabe00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac110980 0x7fcbac1a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.832+0000 7fcbabe00700 1 -- 172.21.15.63:0/3205226890 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbac1a61f0 con 0x7fcbac10a950 2024-05-27T07:51:55.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.833+0000 7fcbabe00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbac10a950 0x7fcbac1a1320 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00d980 tx=0x7fcb9c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:55.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.833+0000 7fcba8c00700 1 -- 172.21.15.63:0/3205226890 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c017070 con 0x7fcbac10a950 2024-05-27T07:51:55.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.833+0000 7fcbb2c00700 1 -- 172.21.15.63:0/3205226890 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbac1a64e0 con 0x7fcbac10a950 2024-05-27T07:51:55.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.833+0000 7fcba8c00700 1 -- 172.21.15.63:0/3205226890 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb9c010460 con 0x7fcbac10a950 2024-05-27T07:51:55.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.833+0000 7fcbb2c00700 1 -- 172.21.15.63:0/3205226890 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbac1ade00 con 0x7fcbac10a950 2024-05-27T07:51:55.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.835+0000 7fcba8c00700 1 -- 172.21.15.63:0/3205226890 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c020060 con 0x7fcbac10a950 2024-05-27T07:51:55.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.838+0000 7fcba8c00700 1 -- 172.21.15.63:0/3205226890 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcb9c010610 con 0x7fcbac10a950 2024-05-27T07:51:55.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.839+0000 7fcbc17a8b80 1 Processor -- start 2024-05-27T07:51:55.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.839+0000 7fcbc17a8b80 1 -- start start 2024-05-27T07:51:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.840+0000 7fcba8c00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcb900b29f0 0x7fcb900b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.840+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 0x55729ee24330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.840+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729eca5e40 0x55729ee24870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.840+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ee24e70 0x55729ee31110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.840+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55729ed52e40 con 0x55729ee24e70 2024-05-27T07:51:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.840+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55729ed429e0 con 0x55729ed919f0 2024-05-27T07:51:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.840+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55729ed46a20 con 0x55729eca5e40 2024-05-27T07:51:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.840+0000 7fcba8c00700 1 -- 172.21.15.63:0/3205226890 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7fcb9c0e2870 con 0x7fcbac10a950 2024-05-27T07:51:55.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.840+0000 7fcbb0c00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcb900b29f0 0x7fcb900b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.841+0000 7fcbab400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729eca5e40 0x55729ee24870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.841+0000 7fcbb0c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ee24e70 0x55729ee31110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.841+0000 7fcbab400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729eca5e40 0x55729ee24870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:49346/0 (socket says 172.21.15.63:49346) 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.841+0000 7fcbab400700 1 -- 172.21.15.63:0/3290783325 learned_addr learned my addr 172.21.15.63:0/3290783325 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.841+0000 7fcbabe00700 1 --2- 172.21.15.63:0/3290783325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 0x55729ee24330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.841+0000 7fcbab400700 1 -- 172.21.15.63:0/3290783325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 msgr2=0x55729ee24330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.841+0000 7fcbab400700 1 --2- 172.21.15.63:0/3290783325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 0x55729ee24330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.841+0000 7fcbab400700 1 -- 172.21.15.63:0/3290783325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ee24e70 msgr2=0x55729ee31110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.841+0000 7fcbab400700 1 --2- 172.21.15.63:0/3290783325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ee24e70 0x55729ee31110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.841+0000 7fcbab400700 1 -- 172.21.15.63:0/3290783325 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55729ed0b420 con 0x55729eca5e40 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.842+0000 7fcbabe00700 1 --2- 172.21.15.63:0/3290783325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 0x55729ee24330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:55.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.842+0000 7fcbb0c00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcb900b29f0 0x7fcb900b4f00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcba40062a0 tx=0x7fcba4009c40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:55.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.842+0000 7fcbab400700 1 --2- 172.21.15.63:0/3290783325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729eca5e40 0x55729ee24870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800cb90 tx=0x7fcb9800cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5170ba8c3a0df14 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:55.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.843+0000 7fcba1600700 1 -- 172.21.15.63:0/3290783325 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800ec50 con 0x55729eca5e40 2024-05-27T07:51:55.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.843+0000 7fcba1600700 1 -- 172.21.15.63:0/3290783325 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb98016070 con 0x55729eca5e40 2024-05-27T07:51:55.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.843+0000 7fcba1600700 1 -- 172.21.15.63:0/3290783325 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98003c10 con 0x55729eca5e40 2024-05-27T07:51:55.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.844+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/3290783325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729eca5e40 msgr2=0x55729ee24870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:55.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.844+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/3290783325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729eca5e40 0x55729ee24870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800cb90 tx=0x7fcb9800cea0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.844+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/3290783325 shutdown_connections 2024-05-27T07:51:55.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.844+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/3290783325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ee24e70 0x55729ee31110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.844+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/3290783325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 0x55729ee24330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.844+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/3290783325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729eca5e40 0x55729ee24870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.844+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/3290783325 >> 172.21.15.63:0/3290783325 conn(0x55729ed3f390 msgr2=0x55729ee204c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:55.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.844+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/3290783325 shutdown_connections 2024-05-27T07:51:55.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.844+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/3290783325 wait complete. 2024-05-27T07:51:55.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.845+0000 7fcbc17a8b80 1 -- start start 2024-05-27T07:51:55.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.845+0000 7fcbc17a8b80 1 Processor -- start 2024-05-27T07:51:55.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.846+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729eca5e40 0x55729ee2e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.846+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ed919f0 0x55729ee2eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.846+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 0x55729ee2f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.846+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55729ec48fa0 con 0x55729ed919f0 2024-05-27T07:51:55.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.846+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55729ece3760 con 0x55729eca5e40 2024-05-27T07:51:55.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.846+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55729ee31b80 con 0x55729ee24e70 2024-05-27T07:51:55.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.846+0000 7fcbc17a8b80 10 client.0 _ll_register_callbacks cb 0x55729ececc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.847+0000 7fcbb0c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 0x55729ee2f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.847+0000 7fcbb0c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 0x55729ee2f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:49350/0 (socket says 172.21.15.63:49350) 2024-05-27T07:51:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.847+0000 7fcbb0c00700 1 -- 172.21.15.63:0/582892532 learned_addr learned my addr 172.21.15.63:0/582892532 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.847+0000 7fcbabe00700 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729eca5e40 0x55729ee2e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.847+0000 7fcbab400700 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ed919f0 0x55729ee2eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.847+0000 7fcbab400700 1 -- 172.21.15.63:0/582892532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 msgr2=0x55729ee2f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:55.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.847+0000 7fcbab400700 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 0x55729ee2f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.847+0000 7fcbab400700 1 -- 172.21.15.63:0/582892532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729eca5e40 msgr2=0x55729ee2e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:55.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.847+0000 7fcbab400700 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729eca5e40 0x55729ee2e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.847+0000 7fcbab400700 1 -- 172.21.15.63:0/582892532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55729ec928d0 con 0x55729ed919f0 2024-05-27T07:51:55.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.848+0000 7fcbabe00700 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729eca5e40 0x55729ee2e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:55.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.848+0000 7fcbab400700 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ed919f0 0x55729ee2eaf0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x55729ee21790 tx=0x7fcb98010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd02565594de6a0d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:55.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.848+0000 7fcb96000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:55.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.848+0000 7fcb96000700 1 -- 172.21.15.63:0/582892532 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98029060 con 0x55729ed919f0 2024-05-27T07:51:55.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.849+0000 7fcb96000700 1 -- 172.21.15.63:0/582892532 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7fcb98005260 con 0x55729ed919f0 2024-05-27T07:51:55.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.849+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/582892532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb98016070 con 0x55729ed919f0 2024-05-27T07:51:55.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.849+0000 7fcb96000700 1 -- 172.21.15.63:0/582892532 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcb98030050 con 0x55729ed919f0 2024-05-27T07:51:55.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.849+0000 7fcbc17a8b80 10 client.18954 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:55.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.849+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/582892532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55729ed13060 con 0x55729ed919f0 2024-05-27T07:51:55.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.849+0000 7fcb96000700 1 -- 172.21.15.63:0/582892532 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb98016070 con 0x55729ed919f0 2024-05-27T07:51:55.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.850+0000 7fcb96000700 1 -- 172.21.15.63:0/582892532 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcb98004750 con 0x55729ed919f0 2024-05-27T07:51:55.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.850+0000 7fcbc17a8b80 10 client.18954 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:55.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.850+0000 7fcbc17a8b80 10 client.18954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:55.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.850+0000 7fcbc17a8b80 20 client.18954 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:55.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.850+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55729ee3e980 0x55729ee40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.850+0000 7fcbc17a8b80 4 client.18954 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:55.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.850+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/582892532 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55729ed54d90 con 0x55729ee3e980 2024-05-27T07:51:55.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.851+0000 7fcbabe00700 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55729ee3e980 0x55729ee40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.852+0000 7fcbabe00700 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55729ee3e980 0x55729ee40da0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f976bf69604a2305 server_cookie=7c5667c92817e488 in_seq=0 out_seq=0 2024-05-27T07:51:55.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.852+0000 7fcb96000700 10 client.18954 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:55.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcb96000700 1 -- 172.21.15.63:0/582892532 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55729ed54d90 con 0x55729ee3e980 2024-05-27T07:51:55.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcb96000700 10 client.18954 handle_command_reply: tid=0 2024-05-27T07:51:55.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcbc17a8b80 1 client.18954 shutdown 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/582892532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ed919f0 msgr2=0x55729ee2eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ed919f0 0x55729ee2eaf0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x55729ee21790 tx=0x7fcb98010660 comp rx=0 tx=0).stop 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/582892532 shutdown_connections 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55729ee3e980 0x55729ee40da0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcb96000700 0 client.18954 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 0x55729ee2f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729eca5e40 0x55729ee2e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/582892532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729ed919f0 0x55729ee2eaf0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.854+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/582892532 >> 172.21.15.63:0/582892532 conn(0x55729ed3f390 msgr2=0x55729ee24310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.855+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/582892532 shutdown_connections 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.855+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/582892532 wait complete. 2024-05-27T07:51:55.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.855+0000 7fcbc17a8b80 20 client.18954 trim_cache size 0 max 16384 2024-05-27T07:51:55.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.875+0000 7fcbc17a8b80 1 Processor -- start 2024-05-27T07:51:55.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.875+0000 7fcbc17a8b80 1 -- start start 2024-05-27T07:51:55.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.875+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee3ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.875+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ed919f0 0x55729ee29d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.875+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ee24e70 0x55729ee24720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.875+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b9480 con 0x55729eca5e40 2024-05-27T07:51:55.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.875+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b6d10 con 0x55729ee24e70 2024-05-27T07:51:55.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.875+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b4b80 con 0x55729ed919f0 2024-05-27T07:51:55.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.876+0000 7fcbabe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee3ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.876+0000 7fcbabe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee3ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46370/0 (socket says 172.21.15.63:46370) 2024-05-27T07:51:55.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.876+0000 7fcbabe00700 1 -- 172.21.15.63:0/3632110820 learned_addr learned my addr 172.21.15.63:0/3632110820 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:55.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.876+0000 7fcbab400700 1 --2- 172.21.15.63:0/3632110820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ed919f0 0x55729ee29d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.876+0000 7fcbb0c00700 1 --2- 172.21.15.63:0/3632110820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ee24e70 0x55729ee24720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.876+0000 7fcbabe00700 1 -- 172.21.15.63:0/3632110820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ed919f0 msgr2=0x55729ee29d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:55.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.876+0000 7fcbabe00700 1 --2- 172.21.15.63:0/3632110820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ed919f0 0x55729ee29d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.876+0000 7fcbabe00700 1 -- 172.21.15.63:0/3632110820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ee24e70 msgr2=0x55729ee24720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:55.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.876+0000 7fcbabe00700 1 --2- 172.21.15.63:0/3632110820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ee24e70 0x55729ee24720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.876+0000 7fcbabe00700 1 -- 172.21.15.63:0/3632110820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55729ee1c240 con 0x55729eca5e40 2024-05-27T07:51:55.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.877+0000 7fcbabe00700 1 --2- 172.21.15.63:0/3632110820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee3ae20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0a9670 tx=0x7fcb9c091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd5ee4c0cca66c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:55.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.877+0000 7fcba0c00700 1 -- 172.21.15.63:0/3632110820 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0b0e60 con 0x55729eca5e40 2024-05-27T07:51:55.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.877+0000 7fcba0c00700 1 -- 172.21.15.63:0/3632110820 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb9c0a3e50 con 0x55729eca5e40 2024-05-27T07:51:55.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.877+0000 7fcba0c00700 1 -- 172.21.15.63:0/3632110820 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c0dfe20 con 0x55729eca5e40 2024-05-27T07:51:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.878+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/3632110820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 msgr2=0x55729ee3ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.878+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/3632110820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee3ae20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0a9670 tx=0x7fcb9c091040 comp rx=0 tx=0).stop 2024-05-27T07:51:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.878+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/3632110820 shutdown_connections 2024-05-27T07:51:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.878+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/3632110820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ee24e70 0x55729ee24720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.878+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/3632110820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee3ae20 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.878+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/3632110820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ed919f0 0x55729ee29d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.878+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/3632110820 >> 172.21.15.63:0/3632110820 conn(0x55729ed3f390 msgr2=0x55729ee2e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.878+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/3632110820 shutdown_connections 2024-05-27T07:51:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.879+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/3632110820 wait complete. 2024-05-27T07:51:55.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.879+0000 7fcbc17a8b80 1 -- start start 2024-05-27T07:51:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.879+0000 7fcbc17a8b80 1 Processor -- start 2024-05-27T07:51:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.880+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee44660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.880+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 0x55729ee409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.880+0000 7fcbc17a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 0x55729ee41070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.880+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b51d0 con 0x55729eca5e40 2024-05-27T07:51:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.880+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b6640 con 0x55729ed919f0 2024-05-27T07:51:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.880+0000 7fcbc17a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b58a0 con 0x55729ee24e70 2024-05-27T07:51:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.880+0000 7fcbc17a8b80 10 client.0 _ll_register_callbacks cb 0x55729ececc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.880+0000 7fcbabe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee44660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.880+0000 7fcbabe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee44660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46380/0 (socket says 172.21.15.63:46380) 2024-05-27T07:51:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.880+0000 7fcbabe00700 1 -- 172.21.15.63:0/653868364 learned_addr learned my addr 172.21.15.63:0/653868364 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.881+0000 7fcbab400700 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 0x55729ee409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.881+0000 7fcbb0c00700 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 0x55729ee41070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.881+0000 7fcbabe00700 1 -- 172.21.15.63:0/653868364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 msgr2=0x55729ee41070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.881+0000 7fcbabe00700 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 0x55729ee41070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.881+0000 7fcbabe00700 1 -- 172.21.15.63:0/653868364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 msgr2=0x55729ee409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.881+0000 7fcbabe00700 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 0x55729ee409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.881+0000 7fcbabe00700 1 -- 172.21.15.63:0/653868364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55729ee417f0 con 0x55729eca5e40 2024-05-27T07:51:55.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.881+0000 7fcbabe00700 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee44660 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0a9670 tx=0x7fcb9c08d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0170ce17244839f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.881+0000 7fcb96000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.882+0000 7fcb96000700 1 -- 172.21.15.63:0/653868364 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c014020 con 0x55729eca5e40 2024-05-27T07:51:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.882+0000 7fcb96000700 1 -- 172.21.15.63:0/653868364 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7fcb9c0dfe20 con 0x55729eca5e40 2024-05-27T07:51:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.882+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/653868364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb9c0a3e50 con 0x55729eca5e40 2024-05-27T07:51:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.882+0000 7fcb96000700 1 -- 172.21.15.63:0/653868364 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcb9c0e0020 con 0x55729eca5e40 2024-05-27T07:51:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.882+0000 7fcbc17a8b80 10 client.18960 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:55.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.882+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/653868364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55729eea8270 con 0x55729eca5e40 2024-05-27T07:51:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.882+0000 7fcb96000700 1 -- 172.21.15.63:0/653868364 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb9c0a3e50 con 0x55729eca5e40 2024-05-27T07:51:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.883+0000 7fcb96000700 1 -- 172.21.15.63:0/653868364 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcb9c0e1040 con 0x55729eca5e40 2024-05-27T07:51:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.883+0000 7fcbc17a8b80 10 client.18960 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.883+0000 7fcbc17a8b80 10 client.18960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.883+0000 7fcbc17a8b80 20 client.18960 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.883+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55729eea8930 0x55729eeaad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.883+0000 7fcbc17a8b80 4 client.18960 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:51:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.883+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/653868364 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55729ed4e570 con 0x55729eea8930 2024-05-27T07:51:55.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.883+0000 7fcbab400700 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55729eea8930 0x55729eeaad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:55.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.884+0000 7fcb96000700 10 client.18960 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:55.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.884+0000 7fcbab400700 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55729eea8930 0x55729eeaad50 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e34dd6fc8adfbe server_cookie=385dbc10c255cd08 in_seq=0 out_seq=0 2024-05-27T07:51:55.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcb96000700 1 -- 172.21.15.63:0/653868364 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55729ed4e570 con 0x55729eea8930 2024-05-27T07:51:55.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcb96000700 10 client.18960 handle_command_reply: tid=0 2024-05-27T07:51:55.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcbc17a8b80 1 client.18960 shutdown 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/653868364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 msgr2=0x55729ee44660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee44660 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0a9670 tx=0x7fcb9c08d940 comp rx=0 tx=0).stop 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/653868364 shutdown_connections 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55729eea8930 0x55729eeaad50 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55729ee24e70 0x55729ee41070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55729eca5e40 0x55729ee44660 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcbc17a8b80 1 --2- 172.21.15.63:0/653868364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55729ed919f0 0x55729ee409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.889+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/653868364 >> 172.21.15.63:0/653868364 conn(0x55729ed3f390 msgr2=0x55729ee2ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.890+0000 7fcb96000700 0 client.18960 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.890+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/653868364 shutdown_connections 2024-05-27T07:51:55.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.890+0000 7fcbc17a8b80 1 -- 172.21.15.63:0/653868364 wait complete. 2024-05-27T07:51:55.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.890+0000 7fcbc17a8b80 20 client.18960 trim_cache size 0 max 16384 2024-05-27T07:51:55.892 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 -- 172.21.15.63:0/3205226890 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcb900b29f0 msgr2=0x7fcb900b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcb900b29f0 0x7fcb900b4f00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcba40062a0 tx=0x7fcba4009c40 comp rx=0 tx=0).stop 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 -- 172.21.15.63:0/3205226890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbac10a950 msgr2=0x7fcbac1a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbac10a950 0x7fcbac1a1320 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00d980 tx=0x7fcb9c00dd40 comp rx=0 tx=0).stop 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 -- 172.21.15.63:0/3205226890 shutdown_connections 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcb900b29f0 0x7fcb900b4f00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbac110980 0x7fcbac1a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbac10a950 0x7fcbac1a1320 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 --2- 172.21.15.63:0/3205226890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbac10b2a0 0x7fcbac1a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 -- 172.21.15.63:0/3205226890 >> 172.21.15.63:0/3205226890 conn(0x7fcbac076d20 msgr2=0x7fcbac1088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 -- 172.21.15.63:0/3205226890 shutdown_connections 2024-05-27T07:51:55.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:51:55.891+0000 7fcbb2c00700 1 -- 172.21.15.63:0/3205226890 wait complete. 2024-05-27T07:51:57.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:57 smithi086 ceph-mon[34687]: pgmap v578: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:51:57.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:57 smithi149 ceph-mon[35408]: pgmap v578: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:51:57.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:57 smithi063 ceph-mon[31613]: pgmap v578: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:51:58.098 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:51:58.099 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:51:58.618 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.187+0000 7fd39ea00640 1 -- 172.21.15.63:0/175946279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd39810a150 msgr2=0x7fd398103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.187+0000 7fd39ea00640 1 --2- 172.21.15.63:0/175946279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd39810a150 0x7fd398103d90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fd384009900 tx=0x7fd38402a410 comp rx=0 tx=0).stop 2024-05-27T07:51:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.188+0000 7fd39ea00640 1 -- 172.21.15.63:0/175946279 shutdown_connections 2024-05-27T07:51:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.188+0000 7fd39ea00640 1 --2- 172.21.15.63:0/175946279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd398104400 0x7fd3981107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.188+0000 7fd39ea00640 1 --2- 172.21.15.63:0/175946279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd39810a150 0x7fd398103d90 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.188+0000 7fd39ea00640 1 --2- 172.21.15.63:0/175946279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd398109cb0 0x7fd398103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.188+0000 7fd39ea00640 1 -- 172.21.15.63:0/175946279 >> 172.21.15.63:0/175946279 conn(0x7fd3980ff480 msgr2=0x7fd3981018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.188+0000 7fd39ea00640 1 -- 172.21.15.63:0/175946279 shutdown_connections 2024-05-27T07:51:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.188+0000 7fd39ea00640 1 -- 172.21.15.63:0/175946279 wait complete. 2024-05-27T07:51:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.189+0000 7fd39ea00640 1 Processor -- start 2024-05-27T07:51:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.189+0000 7fd39ea00640 1 -- start start 2024-05-27T07:51:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.190+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd398104400 0x7fd39819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.190+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd398109cb0 0x7fd39819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.190+0000 7fd39ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd39810a150 0x7fd3981a8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.190+0000 7fd39ea00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd398113da0 con 0x7fd398109cb0 2024-05-27T07:51:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.190+0000 7fd39ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd398113c20 con 0x7fd39810a150 2024-05-27T07:51:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.190+0000 7fd39ea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd398113f20 con 0x7fd398104400 2024-05-27T07:51:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.190+0000 7fd397400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd398109cb0 0x7fd39819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.191+0000 7fd397400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd398109cb0 0x7fd39819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51784/0 (socket says 172.21.15.63:51784) 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.191+0000 7fd397400640 1 -- 172.21.15.63:0/2741819060 learned_addr learned my addr 172.21.15.63:0/2741819060 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.191+0000 7fd397e00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd398104400 0x7fd39819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.191+0000 7fd39ca00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd39810a150 0x7fd3981a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.191+0000 7fd397400640 1 -- 172.21.15.63:0/2741819060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd398104400 msgr2=0x7fd39819d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.191+0000 7fd397400640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd398104400 0x7fd39819d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.191+0000 7fd397400640 1 -- 172.21.15.63:0/2741819060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd39810a150 msgr2=0x7fd3981a8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.191+0000 7fd397400640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd39810a150 0x7fd3981a8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.191+0000 7fd397400640 1 -- 172.21.15.63:0/2741819060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3981a8c40 con 0x7fd398109cb0 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.191+0000 7fd397e00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd398104400 0x7fd39819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.192+0000 7fd397400640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd398109cb0 0x7fd39819dda0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fd38402a930 tx=0x7fd384003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.192+0000 7fd394c00640 1 -- 172.21.15.63:0/2741819060 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38403f070 con 0x7fd398109cb0 2024-05-27T07:51:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.192+0000 7fd394c00640 1 -- 172.21.15.63:0/2741819060 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd384003e10 con 0x7fd398109cb0 2024-05-27T07:51:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.192+0000 7fd39ea00640 1 -- 172.21.15.63:0/2741819060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3981a8e70 con 0x7fd398109cb0 2024-05-27T07:51:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.192+0000 7fd394c00640 1 -- 172.21.15.63:0/2741819060 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384035030 con 0x7fd398109cb0 2024-05-27T07:51:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.192+0000 7fd39ea00640 1 -- 172.21.15.63:0/2741819060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3981a9280 con 0x7fd398109cb0 2024-05-27T07:51:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.196+0000 7fd3a5a01740 1 Processor -- start 2024-05-27T07:51:59.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.196+0000 7fd3a5a01740 1 -- start start 2024-05-27T07:51:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.197+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 0x561f6654d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.197+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661822f0 0x561f6654dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.197+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f6654e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.197+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561f664beb90 con 0x561f6618a4d0 2024-05-27T07:51:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.197+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f664ae730 con 0x561f661ab1b0 2024-05-27T07:51:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.197+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561f664b2770 con 0x561f661822f0 2024-05-27T07:51:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.197+0000 7fd397e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661822f0 0x561f6654dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.198+0000 7fd39ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 0x561f6654d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.198+0000 7fd397e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661822f0 0x561f6654dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41836/0 (socket says 172.21.15.63:41836) 2024-05-27T07:51:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.198+0000 7fd397e00640 1 -- 172.21.15.63:0/2868681517 learned_addr learned my addr 172.21.15.63:0/2868681517 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.198+0000 7fd397e00640 1 -- 172.21.15.63:0/2868681517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 msgr2=0x561f6654d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.198+0000 7fd397e00640 1 --2- 172.21.15.63:0/2868681517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 0x561f6654d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.198+0000 7fd397e00640 1 -- 172.21.15.63:0/2868681517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 msgr2=0x561f6654e2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:51:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.198+0000 7fd397e00640 1 --2- 172.21.15.63:0/2868681517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f6654e2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.198+0000 7fd397e00640 1 -- 172.21.15.63:0/2868681517 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f664478b0 con 0x561f661822f0 2024-05-27T07:51:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.198+0000 7fd39ca00640 1 --2- 172.21.15.63:0/2868681517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 0x561f6654d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.198+0000 7fd394c00640 1 -- 172.21.15.63:0/2741819060 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd384042070 con 0x7fd398109cb0 2024-05-27T07:51:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.199+0000 7fd397e00640 1 --2- 172.21.15.63:0/2868681517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661822f0 0x561f6654dda0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd38800bcb0 tx=0x7fd38800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f3d6bb0b5de7eed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.199+0000 7fd38d200640 1 -- 172.21.15.63:0/2868681517 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388015070 con 0x561f661822f0 2024-05-27T07:51:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.200+0000 7fd38d200640 1 -- 172.21.15.63:0/2868681517 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3880118d0 con 0x561f661822f0 2024-05-27T07:51:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.200+0000 7fd38d200640 1 -- 172.21.15.63:0/2868681517 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3880107d0 con 0x561f661822f0 2024-05-27T07:51:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.201+0000 7fd394c00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd37c0b29d0 0x7fd37c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.201+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2868681517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661822f0 msgr2=0x561f6654dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.201+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/2868681517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661822f0 0x561f6654dda0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd38800bcb0 tx=0x7fd38800f730 comp rx=0 tx=0).stop 2024-05-27T07:51:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.201+0000 7fd394c00640 1 -- 172.21.15.63:0/2741819060 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7fd3841031f0 con 0x7fd398109cb0 2024-05-27T07:51:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.201+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2868681517 shutdown_connections 2024-05-27T07:51:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.201+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/2868681517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f6654e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.201+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/2868681517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661822f0 0x561f6654dda0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd38800bcb0 tx=0x7fd38800f730 comp rx=0 tx=0).stop 2024-05-27T07:51:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.201+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/2868681517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 0x561f6654d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.201+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2868681517 >> 172.21.15.63:0/2868681517 conn(0x561f6619f300 msgr2=0x561f6654a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.201+0000 7fd39ca00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd37c0b29d0 0x7fd37c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.202+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2868681517 shutdown_connections 2024-05-27T07:51:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.202+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2868681517 wait complete. 2024-05-27T07:51:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.202+0000 7fd39ca00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd37c0b29d0 0x7fd37c0b4ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x561f6654a9d0 tx=0x7fd390009600 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.203+0000 7fd3a5a01740 1 -- start start 2024-05-27T07:51:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.203+0000 7fd3a5a01740 1 Processor -- start 2024-05-27T07:51:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f6618a4d0 0x561f66557d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 0x561f665582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f66169d80 0x561f66563cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561f66194790 con 0x561f66169d80 2024-05-27T07:51:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f66456bc0 con 0x561f6618a4d0 2024-05-27T07:51:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561f6655a990 con 0x561f661ab1b0 2024-05-27T07:51:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd3a5a01740 10 client.0 _ll_register_callbacks cb 0x561f66434400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd39ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f66169d80 0x561f66563cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd39ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f66169d80 0x561f66563cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51800/0 (socket says 172.21.15.63:51800) 2024-05-27T07:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd39ca00640 1 -- 172.21.15.63:0/1419284243 learned_addr learned my addr 172.21.15.63:0/1419284243 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd397e00640 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f6618a4d0 0x561f66557d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.204+0000 7fd397400640 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 0x561f665582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.205+0000 7fd39ca00640 1 -- 172.21.15.63:0/1419284243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 msgr2=0x561f665582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.205+0000 7fd39ca00640 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 0x561f665582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.205+0000 7fd39ca00640 1 -- 172.21.15.63:0/1419284243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f6618a4d0 msgr2=0x561f66557d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.205+0000 7fd39ca00640 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f6618a4d0 0x561f66557d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.205+0000 7fd39ca00640 1 -- 172.21.15.63:0/1419284243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f661c3210 con 0x561f66169d80 2024-05-27T07:51:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.205+0000 7fd397e00640 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f6618a4d0 0x561f66557d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.206+0000 7fd39ca00640 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f66169d80 0x561f66563cf0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fd390011d60 tx=0x7fd390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4144311142d32b75 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.206+0000 7fd381600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.206+0000 7fd381600640 1 -- 172.21.15.63:0/1419284243 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390020620 con 0x561f66169d80 2024-05-27T07:51:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.206+0000 7fd381600640 1 -- 172.21.15.63:0/1419284243 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7fd3900207c0 con 0x561f66169d80 2024-05-27T07:51:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.206+0000 7fd3a5a01740 1 -- 172.21.15.63:0/1419284243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3880118d0 con 0x561f66169d80 2024-05-27T07:51:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.207+0000 7fd381600640 1 -- 172.21.15.63:0/1419284243 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd390033050 con 0x561f66169d80 2024-05-27T07:51:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.207+0000 7fd3a5a01740 10 client.18969 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.207+0000 7fd3a5a01740 1 -- 172.21.15.63:0/1419284243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561f6628f620 con 0x561f66169d80 2024-05-27T07:51:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.207+0000 7fd381600640 1 -- 172.21.15.63:0/1419284243 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3880118d0 con 0x561f66169d80 2024-05-27T07:51:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.208+0000 7fd381600640 1 -- 172.21.15.63:0/1419284243 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd39000f040 con 0x561f66169d80 2024-05-27T07:51:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.208+0000 7fd3a5a01740 10 client.18969 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.208+0000 7fd3a5a01740 10 client.18969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.208+0000 7fd3a5a01740 20 client.18969 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.208+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f6613f1c0 0x561f66569e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.208+0000 7fd3a5a01740 4 client.18969 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.208+0000 7fd3a5a01740 1 -- 172.21.15.63:0/1419284243 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f664c0ae0 con 0x561f6613f1c0 2024-05-27T07:51:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.208+0000 7fd397e00640 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f6613f1c0 0x561f66569e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.209+0000 7fd397e00640 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f6613f1c0 0x561f66569e20 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79c43f241f0bb333 server_cookie=dd45b26920750275 in_seq=0 out_seq=0 2024-05-27T07:51:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.209+0000 7fd381600640 10 client.18969 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.211+0000 7fd381600640 1 -- 172.21.15.63:0/1419284243 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561f664c0ae0 con 0x561f6613f1c0 2024-05-27T07:51:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.211+0000 7fd381600640 10 client.18969 handle_command_reply: tid=0 2024-05-27T07:51:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.211+0000 7fd3a5a01740 1 client.18969 shutdown 2024-05-27T07:51:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 1 -- 172.21.15.63:0/1419284243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f66169d80 msgr2=0x561f66563cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f66169d80 0x561f66563cf0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fd390011d60 tx=0x7fd390004a40 comp rx=0 tx=0).stop 2024-05-27T07:51:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 1 -- 172.21.15.63:0/1419284243 shutdown_connections 2024-05-27T07:51:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f6613f1c0 0x561f66569e20 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f66169d80 0x561f66563cf0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 0x561f665582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/1419284243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f6618a4d0 0x561f66557d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 1 -- 172.21.15.63:0/1419284243 >> 172.21.15.63:0/1419284243 conn(0x561f6619f300 msgr2=0x561f6654f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd381600640 0 client.18969 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 1 -- 172.21.15.63:0/1419284243 shutdown_connections 2024-05-27T07:51:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 1 -- 172.21.15.63:0/1419284243 wait complete. 2024-05-27T07:51:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.212+0000 7fd3a5a01740 20 client.18969 trim_cache size 0 max 16384 2024-05-27T07:51:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.231+0000 7fd3a5a01740 1 Processor -- start 2024-05-27T07:51:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.232+0000 7fd3a5a01740 1 -- start start 2024-05-27T07:51:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.232+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f66169d80 0x561f66553380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.232+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f66549330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.232+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 0x561f6654fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.232+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981b5d80 con 0x561f6618a4d0 2024-05-27T07:51:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.232+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981b3650 con 0x561f661ab1b0 2024-05-27T07:51:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.232+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981b14c0 con 0x561f66169d80 2024-05-27T07:51:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.232+0000 7fd397400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f66549330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.232+0000 7fd397400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f66549330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51812/0 (socket says 172.21.15.63:51812) 2024-05-27T07:51:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.232+0000 7fd397400640 1 -- 172.21.15.63:0/300377825 learned_addr learned my addr 172.21.15.63:0/300377825 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.233+0000 7fd39ca00640 1 --2- 172.21.15.63:0/300377825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 0x561f6654fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.233+0000 7fd397e00640 1 --2- 172.21.15.63:0/300377825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f66169d80 0x561f66553380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.233+0000 7fd397400640 1 -- 172.21.15.63:0/300377825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f66169d80 msgr2=0x561f66553380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.233+0000 7fd397400640 1 --2- 172.21.15.63:0/300377825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f66169d80 0x561f66553380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.233+0000 7fd397400640 1 -- 172.21.15.63:0/300377825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 msgr2=0x561f6654fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.233+0000 7fd397400640 1 --2- 172.21.15.63:0/300377825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 0x561f6654fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.233+0000 7fd397400640 1 -- 172.21.15.63:0/300377825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6613f3f0 con 0x561f6618a4d0 2024-05-27T07:51:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.233+0000 7fd39ca00640 1 --2- 172.21.15.63:0/300377825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 0x561f6654fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.233+0000 7fd397400640 1 --2- 172.21.15.63:0/300377825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f66549330 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fd3840c9db0 tx=0x7fd3840c9bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1ab1947c9a18cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.234+0000 7fd383e00640 1 -- 172.21.15.63:0/300377825 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384049030 con 0x561f6618a4d0 2024-05-27T07:51:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.234+0000 7fd383e00640 1 -- 172.21.15.63:0/300377825 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3840ce460 con 0x561f6618a4d0 2024-05-27T07:51:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.234+0000 7fd383e00640 1 -- 172.21.15.63:0/300377825 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3841007a0 con 0x561f6618a4d0 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.235+0000 7fd3a5a01740 1 -- 172.21.15.63:0/300377825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 msgr2=0x561f66549330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.235+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/300377825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f66549330 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fd3840c9db0 tx=0x7fd3840c9bc0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.235+0000 7fd3a5a01740 1 -- 172.21.15.63:0/300377825 shutdown_connections 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.235+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/300377825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f661ab1b0 0x561f6654fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.235+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/300377825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f66549330 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.235+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/300377825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f66169d80 0x561f66553380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.235+0000 7fd3a5a01740 1 -- 172.21.15.63:0/300377825 >> 172.21.15.63:0/300377825 conn(0x561f6619f300 msgr2=0x561f6654f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.235+0000 7fd3a5a01740 1 -- 172.21.15.63:0/300377825 shutdown_connections 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.235+0000 7fd3a5a01740 1 -- 172.21.15.63:0/300377825 wait complete. 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.236+0000 7fd3a5a01740 1 -- start start 2024-05-27T07:51:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.236+0000 7fd3a5a01740 1 Processor -- start 2024-05-27T07:51:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.236+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f66169d80 0x561f66552ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.236+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f665513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.236+0000 7fd3a5a01740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 0x561f66551900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.236+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981b1b10 con 0x561f6618a4d0 2024-05-27T07:51:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.236+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981b2f80 con 0x561f66169d80 2024-05-27T07:51:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.236+0000 7fd3a5a01740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3981b21e0 con 0x561f661ab1b0 2024-05-27T07:51:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.236+0000 7fd3a5a01740 10 client.0 _ll_register_callbacks cb 0x7fd390009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd397400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f665513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd397400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f665513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51826/0 (socket says 172.21.15.63:51826) 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd397400640 1 -- 172.21.15.63:0/2739663769 learned_addr learned my addr 172.21.15.63:0/2739663769 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd397e00640 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f66169d80 0x561f66552ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd39ca00640 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 0x561f66551900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd397400640 1 -- 172.21.15.63:0/2739663769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 msgr2=0x561f66551900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd397400640 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 0x561f66551900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd397400640 1 -- 172.21.15.63:0/2739663769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f66169d80 msgr2=0x561f66552ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd397400640 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f66169d80 0x561f66552ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd397400640 1 -- 172.21.15.63:0/2739663769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f66551e40 con 0x561f6618a4d0 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd39ca00640 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 0x561f66551900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.237+0000 7fd397e00640 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f66169d80 0x561f66552ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.238+0000 7fd397400640 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f665513c0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fd38402a9d0 tx=0x7fd3840ca170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d67b5e5123cc87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.238+0000 7fd381600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.238+0000 7fd381600640 1 -- 172.21.15.63:0/2739663769 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384049070 con 0x561f6618a4d0 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.238+0000 7fd381600640 1 -- 172.21.15.63:0/2739663769 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7fd3841007a0 con 0x561f6618a4d0 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.238+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2739663769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3840ce460 con 0x561f6618a4d0 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.238+0000 7fd381600640 1 -- 172.21.15.63:0/2739663769 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd3840ab050 con 0x561f6618a4d0 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.238+0000 7fd3a5a01740 10 client.18975 fetch_fsmap learned FSMap version 33 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.238+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2739663769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561f6655f910 con 0x561f6618a4d0 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.239+0000 7fd381600640 1 -- 172.21.15.63:0/2739663769 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3840ce460 con 0x561f6618a4d0 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.239+0000 7fd381600640 1 -- 172.21.15.63:0/2739663769 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd3840d2b80 con 0x561f6618a4d0 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.239+0000 7fd3a5a01740 10 client.18975 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.239+0000 7fd3a5a01740 10 client.18975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.239+0000 7fd3a5a01740 20 client.18975 populate_metadata read hostname 'smithi063' 2024-05-27T07:51:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.239+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f66560010 0x561f66560420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.239+0000 7fd3a5a01740 4 client.18975 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:51:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.239+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2739663769 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f664ba2c0 con 0x561f66560010 2024-05-27T07:51:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.240+0000 7fd397e00640 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f66560010 0x561f66560420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.240+0000 7fd397e00640 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f66560010 0x561f66560420 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2aaa667f87e1685 server_cookie=825c7f0303be562 in_seq=0 out_seq=0 2024-05-27T07:51:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.240+0000 7fd381600640 10 client.18975 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.278+0000 7fd381600640 1 -- 172.21.15.63:0/2739663769 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561f664ba2c0 con 0x561f66560010 2024-05-27T07:51:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd381600640 10 client.18975 handle_command_reply: tid=0 2024-05-27T07:51:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 client.18975 shutdown 2024-05-27T07:51:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2739663769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 msgr2=0x561f665513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f665513c0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fd38402a9d0 tx=0x7fd3840ca170 comp rx=0 tx=0).stop 2024-05-27T07:51:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2739663769 shutdown_connections 2024-05-27T07:51:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f66560010 0x561f66560420 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f661ab1b0 0x561f66551900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f6618a4d0 0x561f665513c0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd381600640 0 client.18975 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:51:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 --2- 172.21.15.63:0/2739663769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f66169d80 0x561f66552ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2739663769 >> 172.21.15.63:0/2739663769 conn(0x561f6619f300 msgr2=0x561f66559210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2739663769 shutdown_connections 2024-05-27T07:51:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.279+0000 7fd3a5a01740 1 -- 172.21.15.63:0/2739663769 wait complete. 2024-05-27T07:51:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.280+0000 7fd3a5a01740 20 client.18975 trim_cache size 0 max 16384 2024-05-27T07:51:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:51:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:51:59.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:51:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.280+0000 7fd39ea00640 1 -- 172.21.15.63:0/2741819060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd37c0b29d0 msgr2=0x7fd37c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.280+0000 7fd39ea00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd37c0b29d0 0x7fd37c0b4ee0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x561f6654a9d0 tx=0x7fd390009600 comp rx=0 tx=0).stop 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.280+0000 7fd39ea00640 1 -- 172.21.15.63:0/2741819060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd398109cb0 msgr2=0x7fd39819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.280+0000 7fd39ea00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd398109cb0 0x7fd39819dda0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fd38402a930 tx=0x7fd384003c20 comp rx=0 tx=0).stop 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.281+0000 7fd39ea00640 1 -- 172.21.15.63:0/2741819060 shutdown_connections 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.281+0000 7fd39ea00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd37c0b29d0 0x7fd37c0b4ee0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.281+0000 7fd39ea00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd39810a150 0x7fd3981a8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.281+0000 7fd39ea00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd398109cb0 0x7fd39819dda0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.281+0000 7fd39ea00640 1 --2- 172.21.15.63:0/2741819060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd398104400 0x7fd39819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.281+0000 7fd39ea00640 1 -- 172.21.15.63:0/2741819060 >> 172.21.15.63:0/2741819060 conn(0x7fd3980ff480 msgr2=0x7fd398074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.281+0000 7fd39ea00640 1 -- 172.21.15.63:0/2741819060 shutdown_connections 2024-05-27T07:51:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:51:59.281+0000 7fd39ea00640 1 -- 172.21.15.63:0/2741819060 wait complete. 2024-05-27T07:51:59.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:51:59 smithi086 ceph-mon[34687]: pgmap v579: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:51:59.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:51:59 smithi149 ceph-mon[35408]: pgmap v579: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:51:59.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:51:59 smithi063 ceph-mon[31613]: pgmap v579: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:52:01.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.186+0000 7f19bb200700 1 -- 172.21.15.63:0/3395376202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b4105470 msgr2=0x7f19b4105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:01.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.186+0000 7f19bb200700 1 --2- 172.21.15.63:0/3395376202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b4105470 0x7f19b4105880 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009990 tx=0x7f19a4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.186+0000 7f19bb200700 1 -- 172.21.15.63:0/3395376202 shutdown_connections 2024-05-27T07:52:01.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.186+0000 7f19bb200700 1 --2- 172.21.15.63:0/3395376202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b410e8e0 0x7f19b4110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.186+0000 7f19bb200700 1 --2- 172.21.15.63:0/3395376202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b4105470 0x7f19b4105880 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.186+0000 7f19bb200700 1 --2- 172.21.15.63:0/3395376202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b4105dc0 0x7f19b410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.186+0000 7f19bb200700 1 -- 172.21.15.63:0/3395376202 >> 172.21.15.63:0/3395376202 conn(0x7f19b40fef20 msgr2=0x7f19b4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:01.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.187+0000 7f19bb200700 1 -- 172.21.15.63:0/3395376202 shutdown_connections 2024-05-27T07:52:01.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.187+0000 7f19bb200700 1 -- 172.21.15.63:0/3395376202 wait complete. 2024-05-27T07:52:01.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.188+0000 7f19bb200700 1 Processor -- start 2024-05-27T07:52:01.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.188+0000 7f19bb200700 1 -- start start 2024-05-27T07:52:01.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19bb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b4105470 0x7f19b419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19bb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b4105dc0 0x7f19b419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19bb200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b410e8e0 0x7f19b41a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19bb200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41142e0 con 0x7f19b4105470 2024-05-27T07:52:01.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19bb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19b4114160 con 0x7f19b4105dc0 2024-05-27T07:52:01.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19bb200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f19b4114460 con 0x7f19b410e8e0 2024-05-27T07:52:01.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19b3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b4105470 0x7f19b419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19b3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b4105470 0x7f19b419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51838/0 (socket says 172.21.15.63:51838) 2024-05-27T07:52:01.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19b3e00700 1 -- 172.21.15.63:0/4286104789 learned_addr learned my addr 172.21.15.63:0/4286104789 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:01.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19b3400700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b4105dc0 0x7f19b419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.189+0000 7f19b9200700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b410e8e0 0x7f19b41a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.190+0000 7f19b3e00700 1 -- 172.21.15.63:0/4286104789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b410e8e0 msgr2=0x7f19b41a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:01.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.190+0000 7f19b3e00700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b410e8e0 0x7f19b41a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.190+0000 7f19b3e00700 1 -- 172.21.15.63:0/4286104789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b4105dc0 msgr2=0x7f19b419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:01.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.190+0000 7f19b3e00700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b4105dc0 0x7f19b419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.190+0000 7f19b3e00700 1 -- 172.21.15.63:0/4286104789 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b41a1de0 con 0x7f19b4105470 2024-05-27T07:52:01.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.190+0000 7f19b3400700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b4105dc0 0x7f19b419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:01.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.190+0000 7f19b3e00700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b4105470 0x7f19b419cf10 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f19a40057c0 tx=0x7f19a4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:01.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.191+0000 7f19b0c00700 1 -- 172.21.15.63:0/4286104789 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4027070 con 0x7f19b4105470 2024-05-27T07:52:01.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.191+0000 7f19bb200700 1 -- 172.21.15.63:0/4286104789 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b41a2070 con 0x7f19b4105470 2024-05-27T07:52:01.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.191+0000 7f19b0c00700 1 -- 172.21.15.63:0/4286104789 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a400fda0 con 0x7f19b4105470 2024-05-27T07:52:01.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.191+0000 7f19b0c00700 1 -- 172.21.15.63:0/4286104789 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4022ca0 con 0x7f19b4105470 2024-05-27T07:52:01.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.191+0000 7f19bb200700 1 -- 172.21.15.63:0/4286104789 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b41a2550 con 0x7f19b4105470 2024-05-27T07:52:01.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.195+0000 7f19b0c00700 1 -- 172.21.15.63:0/4286104789 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f19a402a070 con 0x7f19b4105470 2024-05-27T07:52:01.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.196+0000 7f19c9d30b80 1 Processor -- start 2024-05-27T07:52:01.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.197+0000 7f19c9d30b80 1 -- start start 2024-05-27T07:52:01.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.197+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d202ec0 0x55574d387870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.197+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d387db0 0x55574d3818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.197+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d382240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.197+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55574d2afec0 con 0x55574d381e30 2024-05-27T07:52:01.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.197+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55574d29fa60 con 0x55574d202ec0 2024-05-27T07:52:01.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.197+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55574d2a3aa0 con 0x55574d387db0 2024-05-27T07:52:01.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b0c00700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f19980b29f0 0x7f19980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d382240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d382240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51852/0 (socket says 172.21.15.63:51852) 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b3400700 1 -- 172.21.15.63:0/998893514 learned_addr learned my addr 172.21.15.63:0/998893514 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b0c00700 1 -- 172.21.15.63:0/4286104789 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f19a40e3950 con 0x7f19b4105470 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b9200700 1 --2- 172.21.15.63:0/998893514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d202ec0 0x55574d387870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b9200700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f19980b29f0 0x7f19980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b3e00700 1 --2- 172.21.15.63:0/998893514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d387db0 0x55574d3818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b3400700 1 -- 172.21.15.63:0/998893514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d387db0 msgr2=0x55574d3818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b3400700 1 --2- 172.21.15.63:0/998893514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d387db0 0x55574d3818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b3400700 1 -- 172.21.15.63:0/998893514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d202ec0 msgr2=0x55574d387870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b3400700 1 --2- 172.21.15.63:0/998893514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d202ec0 0x55574d387870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.198+0000 7f19b3400700 1 -- 172.21.15.63:0/998893514 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55574d2684a0 con 0x55574d381e30 2024-05-27T07:52:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.199+0000 7f19b3400700 1 --2- 172.21.15.63:0/998893514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d382240 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f19a000bd60 tx=0x7f19a000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927611c6c13c5a4d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.199+0000 7f19b9200700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f19980b29f0 0x7f19980b4f00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55574d382bb0 tx=0x7f19ac009680 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.199+0000 7f19a9600700 1 -- 172.21.15.63:0/998893514 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a000fda0 con 0x55574d381e30 2024-05-27T07:52:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.199+0000 7f19a9600700 1 -- 172.21.15.63:0/998893514 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a00118d0 con 0x55574d381e30 2024-05-27T07:52:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.200+0000 7f19a9600700 1 -- 172.21.15.63:0/998893514 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a00105f0 con 0x55574d381e30 2024-05-27T07:52:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.200+0000 7f19c9d30b80 1 -- 172.21.15.63:0/998893514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 msgr2=0x55574d382240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.200+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/998893514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d382240 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f19a000bd60 tx=0x7f19a000f640 comp rx=0 tx=0).stop 2024-05-27T07:52:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.201+0000 7f19c9d30b80 1 -- 172.21.15.63:0/998893514 shutdown_connections 2024-05-27T07:52:01.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.201+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/998893514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d382240 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.201+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/998893514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d202ec0 0x55574d387870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.201+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/998893514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d387db0 0x55574d3818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.201+0000 7f19c9d30b80 1 -- 172.21.15.63:0/998893514 >> 172.21.15.63:0/998893514 conn(0x55574d263040 msgr2=0x55574d37c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:01.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.201+0000 7f19c9d30b80 1 -- 172.21.15.63:0/998893514 shutdown_connections 2024-05-27T07:52:01.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.201+0000 7f19c9d30b80 1 -- 172.21.15.63:0/998893514 wait complete. 2024-05-27T07:52:01.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.201+0000 7f19c9d30b80 1 -- start start 2024-05-27T07:52:01.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.202+0000 7f19c9d30b80 1 Processor -- start 2024-05-27T07:52:01.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.202+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 0x55574d38b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.202+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d38bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.202+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d387db0 0x55574d38c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.202+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55574d1a6020 con 0x55574d381e30 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.202+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55574d2407e0 con 0x55574d387db0 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.202+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55574d38e880 con 0x55574d202ec0 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.202+0000 7f19c9d30b80 10 client.0 _ll_register_callbacks cb 0x55574d249cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.203+0000 7f19b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d38bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.203+0000 7f19b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d38bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51862/0 (socket says 172.21.15.63:51862) 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.203+0000 7f19b3400700 1 -- 172.21.15.63:0/4189454985 learned_addr learned my addr 172.21.15.63:0/4189454985 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.203+0000 7f19b3e00700 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 0x55574d38b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.203+0000 7f19b9200700 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d387db0 0x55574d38c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.203+0000 7f19b3400700 1 -- 172.21.15.63:0/4189454985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 msgr2=0x55574d38b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.203+0000 7f19b3400700 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 0x55574d38b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.203+0000 7f19b3400700 1 -- 172.21.15.63:0/4189454985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d387db0 msgr2=0x55574d38c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.203+0000 7f19b3400700 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d387db0 0x55574d38c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.203+0000 7f19b3400700 1 -- 172.21.15.63:0/4189454985 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55574d1ef950 con 0x55574d381e30 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.204+0000 7f19b3e00700 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 0x55574d38b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.204+0000 7f19b3400700 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d38bb20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f19a000fdd0 tx=0x7f19a000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc57ac03ee612740 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.204+0000 7f199e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.204+0000 7f199e000700 1 -- 172.21.15.63:0/4189454985 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0011a80 con 0x55574d381e30 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.205+0000 7f199e000700 1 -- 172.21.15.63:0/4189454985 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f19a0011c20 con 0x55574d381e30 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.205+0000 7f19c9d30b80 1 -- 172.21.15.63:0/4189454985 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a00118d0 con 0x55574d381e30 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.205+0000 7f199e000700 1 -- 172.21.15.63:0/4189454985 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f19a0030050 con 0x55574d381e30 2024-05-27T07:52:01.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.205+0000 7f19c9d30b80 10 client.18987 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:01.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.205+0000 7f19c9d30b80 1 -- 172.21.15.63:0/4189454985 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55574d2700e0 con 0x55574d381e30 2024-05-27T07:52:01.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.205+0000 7f199e000700 1 -- 172.21.15.63:0/4189454985 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a00118d0 con 0x55574d381e30 2024-05-27T07:52:01.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.206+0000 7f199e000700 1 -- 172.21.15.63:0/4189454985 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f19a00049e0 con 0x55574d381e30 2024-05-27T07:52:01.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.206+0000 7f19c9d30b80 10 client.18987 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:01.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.206+0000 7f19c9d30b80 10 client.18987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:01.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.206+0000 7f19c9d30b80 20 client.18987 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:01.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.206+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55574d39b660 0x55574d39da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.206+0000 7f19c9d30b80 4 client.18987 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:01.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.206+0000 7f19c9d30b80 1 -- 172.21.15.63:0/4189454985 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55574d2b1e10 con 0x55574d39b660 2024-05-27T07:52:01.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.207+0000 7f19b3e00700 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55574d39b660 0x55574d39da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.208+0000 7f19b3e00700 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55574d39b660 0x55574d39da80 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b8457b14d4582f server_cookie=ed6f97ce3cf4113b in_seq=0 out_seq=0 2024-05-27T07:52:01.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.208+0000 7f199e000700 10 client.18987 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:01.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.209+0000 7f199e000700 1 -- 172.21.15.63:0/4189454985 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55574d2b1e10 con 0x55574d39b660 2024-05-27T07:52:01.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.209+0000 7f199e000700 10 client.18987 handle_command_reply: tid=0 2024-05-27T07:52:01.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 client.18987 shutdown 2024-05-27T07:52:01.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 -- 172.21.15.63:0/4189454985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 msgr2=0x55574d38bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:01.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d38bb20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f19a000fdd0 tx=0x7f19a000bfd0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 -- 172.21.15.63:0/4189454985 shutdown_connections 2024-05-27T07:52:01.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55574d39b660 0x55574d39da80 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d387db0 0x55574d38c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 0x55574d38b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/4189454985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d38bb20 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 -- 172.21.15.63:0/4189454985 >> 172.21.15.63:0/4189454985 conn(0x55574d263040 msgr2=0x55574d37c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:01.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f199e000700 0 client.18987 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:01.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 -- 172.21.15.63:0/4189454985 shutdown_connections 2024-05-27T07:52:01.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.210+0000 7f19c9d30b80 1 -- 172.21.15.63:0/4189454985 wait complete. 2024-05-27T07:52:01.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.211+0000 7f19c9d30b80 20 client.18987 trim_cache size 0 max 16384 2024-05-27T07:52:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.239+0000 7f19c9d30b80 1 Processor -- start 2024-05-27T07:52:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.239+0000 7f19c9d30b80 1 -- start start 2024-05-27T07:52:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.239+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 0x55574d3818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.239+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d397b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.239+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d387db0 0x55574d385bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.239+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41b4f90 con 0x55574d381e30 2024-05-27T07:52:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.239+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41b2820 con 0x55574d387db0 2024-05-27T07:52:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.239+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41b0690 con 0x55574d202ec0 2024-05-27T07:52:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d397b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d397b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51868/0 (socket says 172.21.15.63:51868) 2024-05-27T07:52:01.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b3400700 1 -- 172.21.15.63:0/3736818704 learned_addr learned my addr 172.21.15.63:0/3736818704 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b9200700 1 --2- 172.21.15.63:0/3736818704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d387db0 0x55574d385bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b3e00700 1 --2- 172.21.15.63:0/3736818704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 0x55574d3818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b3400700 1 -- 172.21.15.63:0/3736818704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 msgr2=0x55574d3818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b3400700 1 --2- 172.21.15.63:0/3736818704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 0x55574d3818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b3400700 1 -- 172.21.15.63:0/3736818704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d387db0 msgr2=0x55574d385bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b3400700 1 --2- 172.21.15.63:0/3736818704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d387db0 0x55574d385bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b3400700 1 -- 172.21.15.63:0/3736818704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55574d38de80 con 0x55574d381e30 2024-05-27T07:52:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.240+0000 7f19b3e00700 1 --2- 172.21.15.63:0/3736818704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 0x55574d3818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.241+0000 7f19b3400700 1 --2- 172.21.15.63:0/3736818704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d397b20 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f19a000bd60 tx=0x7f19a000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ce8e4c3e89646f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:01.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.241+0000 7f19a8c00700 1 -- 172.21.15.63:0/3736818704 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0039070 con 0x55574d381e30 2024-05-27T07:52:01.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.241+0000 7f19a8c00700 1 -- 172.21.15.63:0/3736818704 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a0025470 con 0x55574d381e30 2024-05-27T07:52:01.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.241+0000 7f19a8c00700 1 -- 172.21.15.63:0/3736818704 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0004290 con 0x55574d381e30 2024-05-27T07:52:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.242+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3736818704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 msgr2=0x55574d397b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.242+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/3736818704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d397b20 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f19a000bd60 tx=0x7f19a000f640 comp rx=0 tx=0).stop 2024-05-27T07:52:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.242+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3736818704 shutdown_connections 2024-05-27T07:52:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.242+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/3736818704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d387db0 0x55574d385bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.242+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/3736818704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d202ec0 0x55574d3818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.242+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/3736818704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d397b20 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.242+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3736818704 >> 172.21.15.63:0/3736818704 conn(0x55574d263040 msgr2=0x55574d38ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.242+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3736818704 shutdown_connections 2024-05-27T07:52:01.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.243+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3736818704 wait complete. 2024-05-27T07:52:01.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.243+0000 7f19c9d30b80 1 -- start start 2024-05-27T07:52:01.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.243+0000 7f19c9d30b80 1 Processor -- start 2024-05-27T07:52:01.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.244+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d202ec0 0x55574d3a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.244+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d39f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.244+0000 7f19c9d30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d387db0 0x55574d39f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.244+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41b0ce0 con 0x55574d381e30 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.244+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41b2150 con 0x55574d202ec0 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.244+0000 7f19c9d30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f19b41b13b0 con 0x55574d387db0 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.244+0000 7f19c9d30b80 10 client.0 _ll_register_callbacks cb 0x55574d249cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.244+0000 7f19b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d39f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.244+0000 7f19b3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d39f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51870/0 (socket says 172.21.15.63:51870) 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.244+0000 7f19b3400700 1 -- 172.21.15.63:0/3861014995 learned_addr learned my addr 172.21.15.63:0/3861014995 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.245+0000 7f19b3e00700 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d202ec0 0x55574d3a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.245+0000 7f19b9200700 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d387db0 0x55574d39f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.245+0000 7f19b3400700 1 -- 172.21.15.63:0/3861014995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d387db0 msgr2=0x55574d39f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.245+0000 7f19b3400700 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d387db0 0x55574d39f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.245+0000 7f19b3400700 1 -- 172.21.15.63:0/3861014995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d202ec0 msgr2=0x55574d3a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.245+0000 7f19b3400700 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d202ec0 0x55574d3a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.245+0000 7f19b3400700 1 -- 172.21.15.63:0/3861014995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55574d39fca0 con 0x55574d381e30 2024-05-27T07:52:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.246+0000 7f19b3400700 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d39f220 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f19a0005a80 tx=0x7f19a000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eef6a10147c92a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.246+0000 7f199e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.246+0000 7f199e000700 1 -- 172.21.15.63:0/3861014995 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0039040 con 0x55574d381e30 2024-05-27T07:52:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.246+0000 7f199e000700 1 -- 172.21.15.63:0/3861014995 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f19a0004b00 con 0x55574d381e30 2024-05-27T07:52:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.246+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3861014995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a0025470 con 0x55574d381e30 2024-05-27T07:52:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.246+0000 7f199e000700 1 -- 172.21.15.63:0/3861014995 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f19a0046050 con 0x55574d381e30 2024-05-27T07:52:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.246+0000 7f19c9d30b80 10 client.18993 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.246+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3861014995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55574d40f2f0 con 0x55574d381e30 2024-05-27T07:52:01.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.247+0000 7f199e000700 1 -- 172.21.15.63:0/3861014995 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a0025470 con 0x55574d381e30 2024-05-27T07:52:01.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.247+0000 7f199e000700 1 -- 172.21.15.63:0/3861014995 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f19a0011c30 con 0x55574d381e30 2024-05-27T07:52:01.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.247+0000 7f19c9d30b80 10 client.18993 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.247+0000 7f19c9d30b80 10 client.18993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.247+0000 7f19c9d30b80 20 client.18993 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.247+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55574d40f8b0 0x55574d411cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.247+0000 7f19c9d30b80 4 client.18993 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:52:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.247+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3861014995 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55574d2ab5f0 con 0x55574d40f8b0 2024-05-27T07:52:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.248+0000 7f19b3e00700 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55574d40f8b0 0x55574d411cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.249+0000 7f19b3e00700 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55574d40f8b0 0x55574d411cd0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eba53183a61efc1 server_cookie=4932b4509aeea273 in_seq=0 out_seq=0 2024-05-27T07:52:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.249+0000 7f199e000700 10 client.18993 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.251+0000 7f199e000700 1 -- 172.21.15.63:0/3861014995 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55574d2ab5f0 con 0x55574d40f8b0 2024-05-27T07:52:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.251+0000 7f199e000700 10 client.18993 handle_command_reply: tid=0 2024-05-27T07:52:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.251+0000 7f19c9d30b80 1 client.18993 shutdown 2024-05-27T07:52:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.252+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3861014995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 msgr2=0x55574d39f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.252+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d39f220 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f19a0005a80 tx=0x7f19a000beb0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.252+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3861014995 shutdown_connections 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.252+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55574d40f8b0 0x55574d411cd0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.252+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55574d387db0 0x55574d39f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.252+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55574d202ec0 0x55574d3a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.252+0000 7f19c9d30b80 1 --2- 172.21.15.63:0/3861014995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55574d381e30 0x55574d39f220 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.252+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3861014995 >> 172.21.15.63:0/3861014995 conn(0x55574d263040 msgr2=0x55574d385950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.252+0000 7f199e000700 0 client.18993 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.252+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3861014995 shutdown_connections 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.253+0000 7f19c9d30b80 1 -- 172.21.15.63:0/3861014995 wait complete. 2024-05-27T07:52:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.253+0000 7f19c9d30b80 20 client.18993 trim_cache size 0 max 16384 2024-05-27T07:52:01.255 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:52:01.255 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:52:01.255 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "ffbcd09a-c765-4d00-a8f7-b0dfa6a45bec", 2024-05-27T07:52:01.255 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:52:01.255 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:52:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 -- 172.21.15.63:0/4286104789 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f19980b29f0 msgr2=0x7f19980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f19980b29f0 0x7f19980b4f00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55574d382bb0 tx=0x7f19ac009680 comp rx=0 tx=0).stop 2024-05-27T07:52:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 -- 172.21.15.63:0/4286104789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b4105470 msgr2=0x7f19b419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b4105470 0x7f19b419cf10 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f19a40057c0 tx=0x7f19a4009f90 comp rx=0 tx=0).stop 2024-05-27T07:52:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 -- 172.21.15.63:0/4286104789 shutdown_connections 2024-05-27T07:52:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f19980b29f0 0x7f19980b4f00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b410e8e0 0x7f19b41a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b4105470 0x7f19b419cf10 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 --2- 172.21.15.63:0/4286104789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b4105dc0 0x7f19b419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 -- 172.21.15.63:0/4286104789 >> 172.21.15.63:0/4286104789 conn(0x7f19b40fef20 msgr2=0x7f19b4103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 -- 172.21.15.63:0/4286104789 shutdown_connections 2024-05-27T07:52:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:01.254+0000 7f19bb200700 1 -- 172.21.15.63:0/4286104789 wait complete. 2024-05-27T07:52:01.348 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:01 smithi063 ceph-mon[31613]: pgmap v580: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:52:01.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:01 smithi086 ceph-mon[34687]: pgmap v580: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:52:01.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:01 smithi149 ceph-mon[35408]: pgmap v580: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:52:02.334 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:02 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:52:02.334 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.335 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:02 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:52:02.335 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:02 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:52:02.335 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.335 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:02 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:52:02.335 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.335 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:02 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:52:02.335 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.335 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.338 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:52:02.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:02 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:52:02.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:02 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:52:02.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:02 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:52:02.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:02 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:52:02.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:02 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:52:02.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:02 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:52:02.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:02 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:52:02.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:02 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:52:02.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:02 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:52:02.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:02 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:52:02.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:02.854 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:03.384 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:03 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:52:03.384 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:03 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.384 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:03 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:52:03.384 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:03 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.384 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:03 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:52:03.384 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:03 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.385 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:03 smithi063 ceph-mon[31613]: pgmap v581: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.2 MiB/s wr, 59 op/s 2024-05-27T07:52:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:03 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:52:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:03 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:03 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:52:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:03 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:03 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:52:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:03 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:03 smithi086 ceph-mon[34687]: pgmap v581: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.2 MiB/s wr, 59 op/s 2024-05-27T07:52:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:03 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:52:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:03 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:03 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:52:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:03 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:03 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:52:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:03 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:03 smithi149 ceph-mon[35408]: pgmap v581: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.2 MiB/s wr, 59 op/s 2024-05-27T07:52:05.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.173+0000 7f9fa3400700 1 -- 172.21.15.63:0/3053318814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c106560 msgr2=0x7f9f9c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:05.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.173+0000 7f9fa3400700 1 --2- 172.21.15.63:0/3053318814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c106560 0x7f9f9c106970 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009990 tx=0x7f9f88009ca0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.174+0000 7f9fa3400700 1 -- 172.21.15.63:0/3053318814 shutdown_connections 2024-05-27T07:52:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.174+0000 7f9fa3400700 1 --2- 172.21.15.63:0/3053318814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c106eb0 0x7f9f9c113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.174+0000 7f9fa3400700 1 --2- 172.21.15.63:0/3053318814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c10c5a0 0x7f9f9c10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.174+0000 7f9fa3400700 1 --2- 172.21.15.63:0/3053318814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c106560 0x7f9f9c106970 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.174+0000 7f9fa3400700 1 -- 172.21.15.63:0/3053318814 >> 172.21.15.63:0/3053318814 conn(0x7f9f9c102040 msgr2=0x7f9f9c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.174+0000 7f9fa3400700 1 -- 172.21.15.63:0/3053318814 shutdown_connections 2024-05-27T07:52:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.174+0000 7f9fa3400700 1 -- 172.21.15.63:0/3053318814 wait complete. 2024-05-27T07:52:05.177 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:05 smithi063 ceph-mon[31613]: pgmap v582: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 56 op/s 2024-05-27T07:52:05.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.175+0000 7f9fa3400700 1 Processor -- start 2024-05-27T07:52:05.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.176+0000 7f9fa3400700 1 -- start start 2024-05-27T07:52:05.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.176+0000 7f9fa3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c106560 0x7f9f9c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.176+0000 7f9fa3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c106eb0 0x7f9f9c19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.176+0000 7f9fa3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c10c5a0 0x7f9f9c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.176+0000 7f9fa3400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c116550 con 0x7f9f9c10c5a0 2024-05-27T07:52:05.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.176+0000 7f9fa3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c1163d0 con 0x7f9f9c106560 2024-05-27T07:52:05.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.176+0000 7f9fa3400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c1166d0 con 0x7f9f9c106eb0 2024-05-27T07:52:05.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.177+0000 7f9fa1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c10c5a0 0x7f9f9c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.177+0000 7f9fa0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c106560 0x7f9f9c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.177+0000 7f9fa1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c10c5a0 0x7f9f9c1a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51884/0 (socket says 172.21.15.63:51884) 2024-05-27T07:52:05.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.177+0000 7f9fa1400700 1 -- 172.21.15.63:0/4139037394 learned_addr learned my addr 172.21.15.63:0/4139037394 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:05.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.177+0000 7f9f9be00700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c106eb0 0x7f9f9c19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.178+0000 7f9fa1400700 1 -- 172.21.15.63:0/4139037394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c106eb0 msgr2=0x7f9f9c19d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:05.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.178+0000 7f9fa1400700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c106eb0 0x7f9f9c19d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.178+0000 7f9fa1400700 1 -- 172.21.15.63:0/4139037394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c106560 msgr2=0x7f9f9c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:05.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.178+0000 7f9fa1400700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c106560 0x7f9f9c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.178+0000 7f9fa1400700 1 -- 172.21.15.63:0/4139037394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c1a1e00 con 0x7f9f9c10c5a0 2024-05-27T07:52:05.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.178+0000 7f9fa1400700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c10c5a0 0x7f9f9c1a1760 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400d980 tx=0x7f9f9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:05.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.178+0000 7f9f99600700 1 -- 172.21.15.63:0/4139037394 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94017070 con 0x7f9f9c10c5a0 2024-05-27T07:52:05.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.178+0000 7f9fa3400700 1 -- 172.21.15.63:0/4139037394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9c1a20f0 con 0x7f9f9c10c5a0 2024-05-27T07:52:05.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.179+0000 7f9f99600700 1 -- 172.21.15.63:0/4139037394 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f94005020 con 0x7f9f9c10c5a0 2024-05-27T07:52:05.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.179+0000 7f9f99600700 1 -- 172.21.15.63:0/4139037394 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400f980 con 0x7f9f9c10c5a0 2024-05-27T07:52:05.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.179+0000 7f9fa3400700 1 -- 172.21.15.63:0/4139037394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f9c1a9a10 con 0x7f9f9c10c5a0 2024-05-27T07:52:05.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.183+0000 7f9f99600700 1 -- 172.21.15.63:0/4139037394 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9f94004a30 con 0x7f9f9c10c5a0 2024-05-27T07:52:05.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.184+0000 7f9f99600700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f840b29f0 0x7f9f840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.184+0000 7f9f99600700 1 -- 172.21.15.63:0/4139037394 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f9f940e2820 con 0x7f9f9c10c5a0 2024-05-27T07:52:05.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.185+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f840b29f0 0x7f9f840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.185+0000 7f9fb1fecb80 1 Processor -- start 2024-05-27T07:52:05.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.185+0000 7f9fb1fecb80 1 -- start start 2024-05-27T07:52:05.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 0x561bae0be330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f840b29f0 0x7f9f840b4f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c1079d0 tx=0x7f9f90009180 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:05.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561badf3fe40 0x561bae0be870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bae0bee70 0x561bae0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561badfece40 con 0x561bae0bee70 2024-05-27T07:52:05.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561badfdc9e0 con 0x561bae02b9f0 2024-05-27T07:52:05.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561badfe0a20 con 0x561badf3fe40 2024-05-27T07:52:05.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fa1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bae0bee70 0x561bae0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fa1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bae0bee70 0x561bae0cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51896/0 (socket says 172.21.15.63:51896) 2024-05-27T07:52:05.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fa1400700 1 -- 172.21.15.63:0/2612064626 learned_addr learned my addr 172.21.15.63:0/2612064626 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:05.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9f9be00700 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561badf3fe40 0x561bae0be870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.186+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 0x561bae0be330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.187+0000 7f9fa1400700 1 -- 172.21.15.63:0/2612064626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561badf3fe40 msgr2=0x561bae0be870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.187+0000 7f9fa1400700 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561badf3fe40 0x561bae0be870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.187+0000 7f9fa1400700 1 -- 172.21.15.63:0/2612064626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 msgr2=0x561bae0be330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.187+0000 7f9fa1400700 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 0x561bae0be330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.187+0000 7f9fa1400700 1 -- 172.21.15.63:0/2612064626 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561badfa5420 con 0x561bae0bee70 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.187+0000 7f9f9be00700 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561badf3fe40 0x561bae0be870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.187+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 0x561bae0be330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.187+0000 7f9fa1400700 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bae0bee70 0x561bae0cb110 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9f940a5760 tx=0x7f9f940a91f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6945420fded4fcda server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.188+0000 7f9f8e000700 1 -- 172.21.15.63:0/2612064626 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f940ada90 con 0x561bae0bee70 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.188+0000 7f9f8e000700 1 -- 172.21.15.63:0/2612064626 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f940b2540 con 0x561bae0bee70 2024-05-27T07:52:05.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.188+0000 7f9f8e000700 1 -- 172.21.15.63:0/2612064626 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f940a9ba0 con 0x561bae0bee70 2024-05-27T07:52:05.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.189+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2612064626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bae0bee70 msgr2=0x561bae0cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:05.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.189+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bae0bee70 0x561bae0cb110 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9f940a5760 tx=0x7f9f940a91f0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.189+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2612064626 shutdown_connections 2024-05-27T07:52:05.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.189+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561bae0bee70 0x561bae0cb110 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.189+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 0x561bae0be330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.189+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/2612064626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561badf3fe40 0x561bae0be870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.189+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2612064626 >> 172.21.15.63:0/2612064626 conn(0x561badfd9390 msgr2=0x561bae0ba4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:05.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.189+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2612064626 shutdown_connections 2024-05-27T07:52:05.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.189+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2612064626 wait complete. 2024-05-27T07:52:05.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.190+0000 7f9fb1fecb80 1 -- start start 2024-05-27T07:52:05.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.190+0000 7f9fb1fecb80 1 Processor -- start 2024-05-27T07:52:05.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.191+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.191+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae02b9f0 0x561bae0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.191+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae0bee70 0x561bae0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.191+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561badee2fa0 con 0x561badf3fe40 2024-05-27T07:52:05.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.191+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561badf7d760 con 0x561bae0bee70 2024-05-27T07:52:05.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.191+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561bae0cbb20 con 0x561bae02b9f0 2024-05-27T07:52:05.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.191+0000 7f9fb1fecb80 10 client.0 _ll_register_callbacks cb 0x561badf86c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:05.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.192+0000 7f9fa0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.192+0000 7f9fa0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51900/0 (socket says 172.21.15.63:51900) 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.192+0000 7f9fa0a00700 1 -- 172.21.15.63:0/2171794081 learned_addr learned my addr 172.21.15.63:0/2171794081 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.192+0000 7f9f9be00700 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae02b9f0 0x561bae0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.192+0000 7f9fa1400700 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae0bee70 0x561bae0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.192+0000 7f9fa0a00700 1 -- 172.21.15.63:0/2171794081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae02b9f0 msgr2=0x561bae0c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.192+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae02b9f0 0x561bae0c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.192+0000 7f9fa0a00700 1 -- 172.21.15.63:0/2171794081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae0bee70 msgr2=0x561bae0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.192+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae0bee70 0x561bae0c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.192+0000 7f9fa0a00700 1 -- 172.21.15.63:0/2171794081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561badf2c8d0 con 0x561badf3fe40 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.193+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c8720 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9f90007870 tx=0x7f9f90004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c3d214f71dea8b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.193+0000 7f9f82a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.193+0000 7f9f82a00700 1 -- 172.21.15.63:0/2171794081 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90025030 con 0x561badf3fe40 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.193+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2171794081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f940b2540 con 0x561badf3fe40 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.193+0000 7f9f82a00700 1 -- 172.21.15.63:0/2171794081 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f9f90004bb0 con 0x561badf3fe40 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.194+0000 7f9f82a00700 1 -- 172.21.15.63:0/2171794081 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9f9002f050 con 0x561badf3fe40 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.194+0000 7f9fb1fecb80 10 client.19005 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.194+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2171794081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561badfad060 con 0x561badf3fe40 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.194+0000 7f9f82a00700 1 -- 172.21.15.63:0/2171794081 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f940b2540 con 0x561badf3fe40 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.194+0000 7f9f82a00700 1 -- 172.21.15.63:0/2171794081 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9f900214a0 con 0x561badf3fe40 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.194+0000 7f9fb1fecb80 10 client.19005 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.194+0000 7f9fb1fecb80 10 client.19005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:05.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.194+0000 7f9fb1fecb80 20 client.19005 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:05.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.195+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bae0d8bb0 0x561bae0db010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.195+0000 7f9fb1fecb80 4 client.19005 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:05.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.195+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2171794081 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561badfeed90 con 0x561bae0d8bb0 2024-05-27T07:52:05.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.196+0000 7f9f9be00700 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bae0d8bb0 0x561bae0db010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.197+0000 7f9f9be00700 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bae0d8bb0 0x561bae0db010 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d6cb46e81e880c server_cookie=6b6ad87fb9ad89c8 in_seq=0 out_seq=0 2024-05-27T07:52:05.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.197+0000 7f9f82a00700 10 client.19005 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.198+0000 7f9f82a00700 1 -- 172.21.15.63:0/2171794081 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561badfeed90 con 0x561bae0d8bb0 2024-05-27T07:52:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.198+0000 7f9f82a00700 10 client.19005 handle_command_reply: tid=0 2024-05-27T07:52:05.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9fb1fecb80 1 client.19005 shutdown 2024-05-27T07:52:05.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2171794081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 msgr2=0x561bae0c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:05.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c8720 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9f90007870 tx=0x7f9f90004b40 comp rx=0 tx=0).stop 2024-05-27T07:52:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2171794081 shutdown_connections 2024-05-27T07:52:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bae0d8bb0 0x561bae0db010 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae0bee70 0x561bae0c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c8720 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/2171794081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae02b9f0 0x561bae0c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9f82a00700 0 client.19005 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2171794081 >> 172.21.15.63:0/2171794081 conn(0x561badfd9390 msgr2=0x561bae0be3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.199+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2171794081 shutdown_connections 2024-05-27T07:52:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.200+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/2171794081 wait complete. 2024-05-27T07:52:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.200+0000 7f9fb1fecb80 20 client.19005 trim_cache size 0 max 16384 2024-05-27T07:52:05.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.226+0000 7f9fb1fecb80 1 Processor -- start 2024-05-27T07:52:05.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.226+0000 7f9fb1fecb80 1 -- start start 2024-05-27T07:52:05.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae02b9f0 0x561bae0be7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae0bee70 0x561bae0e0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c1b4f90 con 0x561badf3fe40 2024-05-27T07:52:05.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c1b2820 con 0x561bae0bee70 2024-05-27T07:52:05.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c1b0690 con 0x561bae02b9f0 2024-05-27T07:52:05.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9fa0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9fa0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51912/0 (socket says 172.21.15.63:51912) 2024-05-27T07:52:05.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9fa0a00700 1 -- 172.21.15.63:0/652254242 learned_addr learned my addr 172.21.15.63:0/652254242 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:05.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9f9be00700 1 --2- 172.21.15.63:0/652254242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae02b9f0 0x561bae0be7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.227+0000 7f9fa1400700 1 --2- 172.21.15.63:0/652254242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae0bee70 0x561bae0e0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.228+0000 7f9fa0a00700 1 -- 172.21.15.63:0/652254242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae02b9f0 msgr2=0x561bae0be7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:05.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.228+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/652254242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae02b9f0 0x561bae0be7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.228+0000 7f9fa0a00700 1 -- 172.21.15.63:0/652254242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae0bee70 msgr2=0x561bae0e0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:05.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.228+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/652254242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae0bee70 0x561bae0e0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.228+0000 7f9fa0a00700 1 -- 172.21.15.63:0/652254242 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bae0bb000 con 0x561badf3fe40 2024-05-27T07:52:05.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.228+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/652254242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c3ef0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9f90003d70 tx=0x7f9f90014420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe37fbea225205b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:05.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.228+0000 7f9f8d600700 1 -- 172.21.15.63:0/652254242 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90038070 con 0x561badf3fe40 2024-05-27T07:52:05.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.228+0000 7f9f8d600700 1 -- 172.21.15.63:0/652254242 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f90021d00 con 0x561badf3fe40 2024-05-27T07:52:05.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.228+0000 7f9f8d600700 1 -- 172.21.15.63:0/652254242 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90025070 con 0x561badf3fe40 2024-05-27T07:52:05.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.229+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/652254242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 msgr2=0x561bae0c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:05.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.229+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/652254242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c3ef0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9f90003d70 tx=0x7f9f90014420 comp rx=0 tx=0).stop 2024-05-27T07:52:05.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.230+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/652254242 shutdown_connections 2024-05-27T07:52:05.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.230+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/652254242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae0bee70 0x561bae0e0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.230+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/652254242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0c3ef0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.230+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/652254242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae02b9f0 0x561bae0be7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.230+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/652254242 >> 172.21.15.63:0/652254242 conn(0x561badfd9390 msgr2=0x561bae0be0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:05.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.230+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/652254242 shutdown_connections 2024-05-27T07:52:05.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.230+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/652254242 wait complete. 2024-05-27T07:52:05.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.230+0000 7f9fb1fecb80 1 -- start start 2024-05-27T07:52:05.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.231+0000 7f9fb1fecb80 1 Processor -- start 2024-05-27T07:52:05.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.231+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.231+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 0x561bae0cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.231+0000 7f9fb1fecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae0bee70 0x561bae0cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c1b0ce0 con 0x561badf3fe40 2024-05-27T07:52:05.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c1b2150 con 0x561bae02b9f0 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fb1fecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c1b13b0 con 0x561bae0bee70 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fb1fecb80 10 client.0 _ll_register_callbacks cb 0x561badf86c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fa0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fa0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51916/0 (socket says 172.21.15.63:51916) 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fa0a00700 1 -- 172.21.15.63:0/838055320 learned_addr learned my addr 172.21.15.63:0/838055320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9f9be00700 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 0x561bae0cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fa0a00700 1 -- 172.21.15.63:0/838055320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae0bee70 msgr2=0x561bae0cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae0bee70 0x561bae0cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fa0a00700 1 -- 172.21.15.63:0/838055320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 msgr2=0x561bae0cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 0x561bae0cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9fa0a00700 1 -- 172.21.15.63:0/838055320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bae1426e0 con 0x561badf3fe40 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.232+0000 7f9f9be00700 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 0x561bae0cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:05.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.233+0000 7f9fa0a00700 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0d1650 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9f90014d10 tx=0x7f9f90014fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c40e1bbfe462ba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:05.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.233+0000 7f9f82a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:05.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.233+0000 7f9f82a00700 1 -- 172.21.15.63:0/838055320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90038070 con 0x561badf3fe40 2024-05-27T07:52:05.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.233+0000 7f9f82a00700 1 -- 172.21.15.63:0/838055320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9483+0+0 (secure 0 0 0) 0x7f9f90014650 con 0x561badf3fe40 2024-05-27T07:52:05.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.233+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/838055320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f90021d00 con 0x561badf3fe40 2024-05-27T07:52:05.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9f82a00700 1 -- 172.21.15.63:0/838055320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9f90047050 con 0x561badf3fe40 2024-05-27T07:52:05.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9fb1fecb80 10 client.19011 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:05.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/838055320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561bae1437b0 con 0x561badf3fe40 2024-05-27T07:52:05.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9f82a00700 1 -- 172.21.15.63:0/838055320 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f90021d00 con 0x561badf3fe40 2024-05-27T07:52:05.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9f82a00700 1 -- 172.21.15.63:0/838055320 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9f9003ea30 con 0x561badf3fe40 2024-05-27T07:52:05.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9fb1fecb80 10 client.19011 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:05.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9fb1fecb80 10 client.19011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:05.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9fb1fecb80 20 client.19011 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:05.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bae143da0 0x561bae1461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:05.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9fb1fecb80 4 client.19011 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:52:05.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.234+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/838055320 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561badfe8570 con 0x561bae143da0 2024-05-27T07:52:05.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.235+0000 7f9f9be00700 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bae143da0 0x561bae1461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:05.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.236+0000 7f9f9be00700 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bae143da0 0x561bae1461c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2271c4e62ca5431d server_cookie=9f235beb1844835b in_seq=0 out_seq=0 2024-05-27T07:52:05.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.236+0000 7f9f82a00700 10 client.19011 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:05.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9f82a00700 1 -- 172.21.15.63:0/838055320 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561badfe8570 con 0x561bae143da0 2024-05-27T07:52:05.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9f82a00700 10 client.19011 handle_command_reply: tid=0 2024-05-27T07:52:05.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9fb1fecb80 1 client.19011 shutdown 2024-05-27T07:52:05.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/838055320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 msgr2=0x561bae0d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:05.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0d1650 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9f90014d10 tx=0x7f9f90014fa0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/838055320 shutdown_connections 2024-05-27T07:52:05.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561bae143da0 0x561bae1461c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bae0bee70 0x561bae0cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561badf3fe40 0x561bae0d1650 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9fb1fecb80 1 --2- 172.21.15.63:0/838055320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bae02b9f0 0x561bae0cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9f82a00700 0 client.19011 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:05.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/838055320 >> 172.21.15.63:0/838055320 conn(0x561badfd9390 msgr2=0x561bae0c9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:05.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.237+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/838055320 shutdown_connections 2024-05-27T07:52:05.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.238+0000 7f9fb1fecb80 1 -- 172.21.15.63:0/838055320 wait complete. 2024-05-27T07:52:05.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.238+0000 7f9fb1fecb80 20 client.19011 trim_cache size 0 max 16384 2024-05-27T07:52:05.240 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:52:05.240 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:52:05.240 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:52:05.240 INFO:teuthology.orchestra.run.smithi063.stdout: "ffbcd09a-c765-4d00-a8f7-b0dfa6a45bec": { 2024-05-27T07:52:05.241 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:52:05.241 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "ffbcd09a-c765-4d00-a8f7-b0dfa6a45bec", 2024-05-27T07:52:05.241 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:52:05.241 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:52:05.241 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:52:05.241 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:52:05.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 -- 172.21.15.63:0/4139037394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f840b29f0 msgr2=0x7f9f840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f840b29f0 0x7f9f840b4f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c1079d0 tx=0x7f9f90009180 comp rx=0 tx=0).stop 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 -- 172.21.15.63:0/4139037394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c10c5a0 msgr2=0x7f9f9c1a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c10c5a0 0x7f9f9c1a1760 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400d980 tx=0x7f9f9400dc90 comp rx=0 tx=0).stop 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 -- 172.21.15.63:0/4139037394 shutdown_connections 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9f840b29f0 0x7f9f840b4f00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c10c5a0 0x7f9f9c1a1760 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f9c106560 0x7f9f9c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 --2- 172.21.15.63:0/4139037394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c106eb0 0x7f9f9c19d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 -- 172.21.15.63:0/4139037394 >> 172.21.15.63:0/4139037394 conn(0x7f9f9c102040 msgr2=0x7f9f9c103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 -- 172.21.15.63:0/4139037394 shutdown_connections 2024-05-27T07:52:05.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:05.239+0000 7f9fa3400700 1 -- 172.21.15.63:0/4139037394 wait complete. 2024-05-27T07:52:05.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:05 smithi086 ceph-mon[34687]: pgmap v582: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 56 op/s 2024-05-27T07:52:05.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:05 smithi149 ceph-mon[35408]: pgmap v582: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 56 op/s 2024-05-27T07:52:06.173 INFO:tasks.cephfs.filesystem:scrub status for tag:ffbcd09a-c765-4d00-a8f7-b0dfa6a45bec - {'path': '/', 'tag': 'ffbcd09a-c765-4d00-a8f7-b0dfa6a45bec', 'options': 'recursive,force'} 2024-05-27T07:52:07.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:07 smithi086 ceph-mon[34687]: pgmap v583: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 56 op/s 2024-05-27T07:52:07.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:07 smithi149 ceph-mon[35408]: pgmap v583: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 56 op/s 2024-05-27T07:52:07.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:07 smithi063 ceph-mon[31613]: pgmap v583: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 56 op/s 2024-05-27T07:52:08.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:08 smithi086 ceph-mon[34687]: osdmap e118: 12 total, 12 up, 12 in 2024-05-27T07:52:08.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:08 smithi149 ceph-mon[35408]: osdmap e118: 12 total, 12 up, 12 in 2024-05-27T07:52:08.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:08 smithi063 ceph-mon[31613]: osdmap e118: 12 total, 12 up, 12 in 2024-05-27T07:52:09.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:09 smithi086 ceph-mon[34687]: pgmap v585: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:52:09.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:09 smithi149 ceph-mon[35408]: pgmap v585: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:52:09.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:09 smithi063 ceph-mon[31613]: pgmap v585: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:52:11.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:11 smithi086 ceph-mon[34687]: pgmap v586: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:52:11.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:11 smithi149 ceph-mon[35408]: pgmap v586: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:52:11.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:11 smithi063 ceph-mon[31613]: pgmap v586: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:52:12.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:52:12.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:52:12.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:52:12.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:52:12.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:52:12.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:52:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:52:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:52:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:52:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.532+0000 7f5a0b200640 1 -- 172.21.15.63:0/371279127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a0c10d990 msgr2=0x7f5a0c10fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.532+0000 7f5a0b200640 1 --2- 172.21.15.63:0/371279127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a0c10d990 0x7f5a0c10fdb0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009900 tx=0x7f59f802a480 comp rx=0 tx=0).stop 2024-05-27T07:52:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.532+0000 7f5a0b200640 1 -- 172.21.15.63:0/371279127 shutdown_connections 2024-05-27T07:52:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.532+0000 7f5a0b200640 1 --2- 172.21.15.63:0/371279127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a0c1102f0 0x7f5a0c112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.532+0000 7f5a0b200640 1 --2- 172.21.15.63:0/371279127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a0c10d990 0x7f5a0c10fdb0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.532+0000 7f5a0b200640 1 --2- 172.21.15.63:0/371279127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a0c075480 0x7f5a0c075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.532+0000 7f5a0b200640 1 -- 172.21.15.63:0/371279127 >> 172.21.15.63:0/371279127 conn(0x7f5a0c079240 msgr2=0x7f5a0c079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.532+0000 7f5a0b200640 1 -- 172.21.15.63:0/371279127 shutdown_connections 2024-05-27T07:52:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.533+0000 7f5a0b200640 1 -- 172.21.15.63:0/371279127 wait complete. 2024-05-27T07:52:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.534+0000 7f5a0b200640 1 Processor -- start 2024-05-27T07:52:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.534+0000 7f5a0b200640 1 -- start start 2024-05-27T07:52:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a0c075480 0x7f5a0c1a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a0c10d990 0x7f5a0c1a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a0c1102f0 0x7f5a0c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c115d30 con 0x7f5a0c1102f0 2024-05-27T07:52:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c115bb0 con 0x7f5a0c075480 2024-05-27T07:52:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c115eb0 con 0x7f5a0c10d990 2024-05-27T07:52:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a0c1102f0 0x7f5a0c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a0c1102f0 0x7f5a0c19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53704/0 (socket says 172.21.15.63:53704) 2024-05-27T07:52:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0a800640 1 -- 172.21.15.63:0/575149148 learned_addr learned my addr 172.21.15.63:0/575149148 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a09e00640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a0c075480 0x7f5a0c1a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a09400640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a0c10d990 0x7f5a0c1a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0a800640 1 -- 172.21.15.63:0/575149148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a0c10d990 msgr2=0x7f5a0c1a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0a800640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a0c10d990 0x7f5a0c1a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.535+0000 7f5a0a800640 1 -- 172.21.15.63:0/575149148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a0c075480 msgr2=0x7f5a0c1a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.536+0000 7f5a0a800640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a0c075480 0x7f5a0c1a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.536+0000 7f5a0a800640 1 -- 172.21.15.63:0/575149148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c19ce30 con 0x7f5a0c1102f0 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.536+0000 7f5a0a800640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a0c1102f0 0x7f5a0c19c700 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400eaa0 tx=0x7f5a0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.536+0000 7f5a02a00640 1 -- 172.21.15.63:0/575149148 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0400cdb0 con 0x7f5a0c1102f0 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.536+0000 7f5a0b200640 1 -- 172.21.15.63:0/575149148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0c19d0c0 con 0x7f5a0c1102f0 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.536+0000 7f5a02a00640 1 -- 172.21.15.63:0/575149148 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a04005020 con 0x7f5a0c1102f0 2024-05-27T07:52:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.536+0000 7f5a0b200640 1 -- 172.21.15.63:0/575149148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0c1a9320 con 0x7f5a0c1102f0 2024-05-27T07:52:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.536+0000 7f5a02a00640 1 -- 172.21.15.63:0/575149148 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0401f060 con 0x7f5a0c1102f0 2024-05-27T07:52:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.540+0000 7f5a17bda740 1 Processor -- start 2024-05-27T07:52:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.541+0000 7f5a02a00640 1 -- 172.21.15.63:0/575149148 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5a040051d0 con 0x7f5a0c1102f0 2024-05-27T07:52:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.541+0000 7f5a17bda740 1 -- start start 2024-05-27T07:52:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.542+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a57119fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.542+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a57119ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.542+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 0x55a5711a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.542+0000 7f5a17bda740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a571110b90 con 0x55a570dfd1b0 2024-05-27T07:52:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.542+0000 7f5a17bda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a571100730 con 0x55a570dd42f0 2024-05-27T07:52:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.542+0000 7f5a17bda740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a571104770 con 0x55a570ddc4d0 2024-05-27T07:52:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.542+0000 7f5a09400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a57119fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.542+0000 7f5a09400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a57119fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53720/0 (socket says 172.21.15.63:53720) 2024-05-27T07:52:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.542+0000 7f5a09400640 1 -- 172.21.15.63:0/3988060132 learned_addr learned my addr 172.21.15.63:0/3988060132 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.542+0000 7f5a09e00640 1 --2- 172.21.15.63:0/3988060132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a57119ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.543+0000 7f5a09400640 1 --2- 172.21.15.63:0/3988060132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 0x55a5711a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.543+0000 7f5a09400640 1 -- 172.21.15.63:0/3988060132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 msgr2=0x55a5711a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.543+0000 7f5a09400640 1 --2- 172.21.15.63:0/3988060132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 0x55a5711a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.543+0000 7f5a09400640 1 -- 172.21.15.63:0/3988060132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 msgr2=0x55a57119ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.543+0000 7f5a09400640 1 --2- 172.21.15.63:0/3988060132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a57119ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.543+0000 7f5a09400640 1 -- 172.21.15.63:0/3988060132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5710998b0 con 0x55a570dfd1b0 2024-05-27T07:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.543+0000 7f5a09400640 1 --2- 172.21.15.63:0/3988060132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a57119fa60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f59f8005ab0 tx=0x7f59f8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a848987dcbc2ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.543+0000 7f5a02a00640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59f00b2950 0x7f59f00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.544+0000 7f59f7000640 1 -- 172.21.15.63:0/3988060132 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8035050 con 0x55a570dfd1b0 2024-05-27T07:52:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.544+0000 7f59f7000640 1 -- 172.21.15.63:0/3988060132 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59f8004030 con 0x55a570dfd1b0 2024-05-27T07:52:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.544+0000 7f59f7000640 1 -- 172.21.15.63:0/3988060132 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f80053c0 con 0x55a570dfd1b0 2024-05-27T07:52:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.544+0000 7f5a02a00640 1 -- 172.21.15.63:0/575149148 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9571+0+0 (secure 0 0 0) 0x7f5a040e2bc0 con 0x7f5a0c1102f0 2024-05-27T07:52:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.544+0000 7f5a09e00640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59f00b2950 0x7f59f00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.545+0000 7f5a17bda740 1 -- 172.21.15.63:0/3988060132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 msgr2=0x55a57119fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.545+0000 7f5a17bda740 1 --2- 172.21.15.63:0/3988060132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a57119fa60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f59f8005ab0 tx=0x7f59f8005e50 comp rx=0 tx=0).stop 2024-05-27T07:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.545+0000 7f5a09e00640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59f00b2950 0x7f59f00b4e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55a5711a0e10 tx=0x7f59fc00b4d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.545+0000 7f5a17bda740 1 -- 172.21.15.63:0/3988060132 shutdown_connections 2024-05-27T07:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.545+0000 7f5a17bda740 1 --2- 172.21.15.63:0/3988060132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 0x55a5711a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.545+0000 7f5a17bda740 1 --2- 172.21.15.63:0/3988060132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a57119ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.545+0000 7f5a17bda740 1 --2- 172.21.15.63:0/3988060132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a57119fa60 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.545+0000 7f5a17bda740 1 -- 172.21.15.63:0/3988060132 >> 172.21.15.63:0/3988060132 conn(0x55a570df1300 msgr2=0x55a57119c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.545+0000 7f5a17bda740 1 -- 172.21.15.63:0/3988060132 shutdown_connections 2024-05-27T07:52:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.545+0000 7f5a17bda740 1 -- 172.21.15.63:0/3988060132 wait complete. 2024-05-27T07:52:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.546+0000 7f5a17bda740 1 -- start start 2024-05-27T07:52:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.546+0000 7f5a17bda740 1 Processor -- start 2024-05-27T07:52:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a5711a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 0x55a5711aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a5711b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a17bda740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a570de6790 con 0x55a570dfd1b0 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a17bda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5710a8bc0 con 0x55a570dd42f0 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a17bda740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a5711acb30 con 0x55a570ddc4d0 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a17bda740 10 client.0 _ll_register_callbacks cb 0x55a571086400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a0a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a5711b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a0a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a5711b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53736/0 (socket says 172.21.15.63:53736) 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a0a800640 1 -- 172.21.15.63:0/1993260320 learned_addr learned my addr 172.21.15.63:0/1993260320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a09400640 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 0x55a5711aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.547+0000 7f5a09e00640 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a5711a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f5a0a800640 1 -- 172.21.15.63:0/1993260320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 msgr2=0x55a5711aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f5a0a800640 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 0x55a5711aa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f5a0a800640 1 -- 172.21.15.63:0/1993260320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 msgr2=0x55a5711a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f5a0a800640 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a5711a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f5a0a800640 1 -- 172.21.15.63:0/1993260320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a570e15210 con 0x55a570dfd1b0 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f5a09400640 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 0x55a5711aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f5a0a800640 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a5711b5e90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5a040a9a10 tx=0x7f5a04005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2effa66668cc4d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f59eb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f59eb400640 1 -- 172.21.15.63:0/1993260320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a040ade30 con 0x55a570dfd1b0 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f59eb400640 1 -- 172.21.15.63:0/1993260320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9571+0+0 (secure 0 0 0) 0x7f5a040cc0d0 con 0x55a570dfd1b0 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.548+0000 7f5a17bda740 1 -- 172.21.15.63:0/1993260320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59f8004030 con 0x55a570dfd1b0 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f59eb400640 1 -- 172.21.15.63:0/1993260320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5a040b28e0 con 0x55a570dfd1b0 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f5a17bda740 10 client.19023 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f5a17bda740 1 -- 172.21.15.63:0/1993260320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a570ee1620 con 0x55a570dfd1b0 2024-05-27T07:52:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f59eb400640 1 -- 172.21.15.63:0/1993260320 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59f8004030 con 0x55a570dfd1b0 2024-05-27T07:52:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f59eb400640 1 -- 172.21.15.63:0/1993260320 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5a0400f040 con 0x55a570dfd1b0 2024-05-27T07:52:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f5a17bda740 10 client.19023 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f5a17bda740 10 client.19023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f5a17bda740 20 client.19023 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a570dbbd80 0x55a5711bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f5a17bda740 4 client.19023 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.549+0000 7f5a17bda740 1 -- 172.21.15.63:0/1993260320 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a571112ae0 con 0x55a570dbbd80 2024-05-27T07:52:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.550+0000 7f5a09e00640 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a570dbbd80 0x55a5711bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.551+0000 7f5a09e00640 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a570dbbd80 0x55a5711bbed0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4088a21c9274ba88 server_cookie=f6d561f68ad34414 in_seq=0 out_seq=0 2024-05-27T07:52:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.551+0000 7f59eb400640 10 client.19023 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f59eb400640 1 -- 172.21.15.63:0/1993260320 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a571112ae0 con 0x55a570dbbd80 2024-05-27T07:52:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f59eb400640 10 client.19023 handle_command_reply: tid=0 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 client.19023 shutdown 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 -- 172.21.15.63:0/1993260320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 msgr2=0x55a5711b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a5711b5e90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5a040a9a10 tx=0x7f5a04005720 comp rx=0 tx=0).stop 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 -- 172.21.15.63:0/1993260320 shutdown_connections 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a570dbbd80 0x55a5711bbed0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dfd1b0 0x55a5711b5e90 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570ddc4d0 0x55a5711aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1993260320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a5711a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 -- 172.21.15.63:0/1993260320 >> 172.21.15.63:0/1993260320 conn(0x55a570df1300 msgr2=0x55a57119f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f59eb400640 0 client.19023 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 -- 172.21.15.63:0/1993260320 shutdown_connections 2024-05-27T07:52:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.553+0000 7f5a17bda740 1 -- 172.21.15.63:0/1993260320 wait complete. 2024-05-27T07:52:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.554+0000 7f5a17bda740 20 client.19023 trim_cache size 0 max 16384 2024-05-27T07:52:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.573+0000 7f5a17bda740 1 Processor -- start 2024-05-27T07:52:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.573+0000 7f5a17bda740 1 -- start start 2024-05-27T07:52:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dd42f0 0x55a5711ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570ddc4d0 0x55a57119b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570dfd1b0 0x55a57119ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a17bda740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c1b5af0 con 0x55a570dd42f0 2024-05-27T07:52:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a17bda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c1b33c0 con 0x55a570ddc4d0 2024-05-27T07:52:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a17bda740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c1b1230 con 0x55a570dfd1b0 2024-05-27T07:52:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a09e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dd42f0 0x55a5711ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a09e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dd42f0 0x55a5711ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53738/0 (socket says 172.21.15.63:53738) 2024-05-27T07:52:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a09e00640 1 -- 172.21.15.63:0/1671908586 learned_addr learned my addr 172.21.15.63:0/1671908586 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a09400640 1 --2- 172.21.15.63:0/1671908586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570ddc4d0 0x55a57119b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a0a800640 1 --2- 172.21.15.63:0/1671908586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570dfd1b0 0x55a57119ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a09e00640 1 -- 172.21.15.63:0/1671908586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570dfd1b0 msgr2=0x55a57119ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.574+0000 7f5a09e00640 1 --2- 172.21.15.63:0/1671908586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570dfd1b0 0x55a57119ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.575+0000 7f5a09e00640 1 -- 172.21.15.63:0/1671908586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570ddc4d0 msgr2=0x55a57119b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.575+0000 7f5a09e00640 1 --2- 172.21.15.63:0/1671908586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570ddc4d0 0x55a57119b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.575+0000 7f5a09e00640 1 -- 172.21.15.63:0/1671908586 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5711a81d0 con 0x55a570dd42f0 2024-05-27T07:52:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.575+0000 7f5a09e00640 1 --2- 172.21.15.63:0/1671908586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dd42f0 0x55a5711ac3b0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x55a57119d550 tx=0x7f59fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855bb326f931129c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.575+0000 7f59f6600640 1 -- 172.21.15.63:0/1671908586 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00bea0 con 0x55a570dd42f0 2024-05-27T07:52:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.575+0000 7f59f6600640 1 -- 172.21.15.63:0/1671908586 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59fc02f070 con 0x55a570dd42f0 2024-05-27T07:52:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.575+0000 7f59f6600640 1 -- 172.21.15.63:0/1671908586 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc020a10 con 0x55a570dd42f0 2024-05-27T07:52:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.576+0000 7f5a17bda740 1 -- 172.21.15.63:0/1671908586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dd42f0 msgr2=0x55a5711ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.576+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1671908586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dd42f0 0x55a5711ac3b0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x55a57119d550 tx=0x7f59fc015000 comp rx=0 tx=0).stop 2024-05-27T07:52:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.576+0000 7f5a17bda740 1 -- 172.21.15.63:0/1671908586 shutdown_connections 2024-05-27T07:52:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.576+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1671908586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570dfd1b0 0x55a57119ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.576+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1671908586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570ddc4d0 0x55a57119b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.576+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1671908586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570dd42f0 0x55a5711ac3b0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.576+0000 7f5a17bda740 1 -- 172.21.15.63:0/1671908586 >> 172.21.15.63:0/1671908586 conn(0x55a570df1300 msgr2=0x55a57119f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.577+0000 7f5a17bda740 1 -- 172.21.15.63:0/1671908586 shutdown_connections 2024-05-27T07:52:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.577+0000 7f5a17bda740 1 -- 172.21.15.63:0/1671908586 wait complete. 2024-05-27T07:52:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.577+0000 7f5a17bda740 1 -- start start 2024-05-27T07:52:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.577+0000 7f5a17bda740 1 Processor -- start 2024-05-27T07:52:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.578+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a5711ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.578+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570ddc4d0 0x55a5711ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.578+0000 7f5a17bda740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570dfd1b0 0x55a571222590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.578+0000 7f5a17bda740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c1b1880 con 0x55a570ddc4d0 2024-05-27T07:52:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.578+0000 7f5a17bda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c1b2cf0 con 0x55a570dd42f0 2024-05-27T07:52:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.578+0000 7f5a17bda740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c1b1f50 con 0x55a570dfd1b0 2024-05-27T07:52:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.578+0000 7f5a17bda740 10 client.0 _ll_register_callbacks cb 0x7f5a040a5940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.578+0000 7f5a09400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570ddc4d0 0x55a5711ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.578+0000 7f5a09400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570ddc4d0 0x55a5711ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53746/0 (socket says 172.21.15.63:53746) 2024-05-27T07:52:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.578+0000 7f5a09400640 1 -- 172.21.15.63:0/1059812977 learned_addr learned my addr 172.21.15.63:0/1059812977 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.579+0000 7f5a0a800640 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570dfd1b0 0x55a571222590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.579+0000 7f5a09e00640 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a5711ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.579+0000 7f5a09400640 1 -- 172.21.15.63:0/1059812977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570dfd1b0 msgr2=0x55a571222590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.579+0000 7f5a09400640 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570dfd1b0 0x55a571222590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.579+0000 7f5a09400640 1 -- 172.21.15.63:0/1059812977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 msgr2=0x55a5711ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.579+0000 7f5a09400640 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a5711ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.579+0000 7f5a09400640 1 -- 172.21.15.63:0/1059812977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a571222ad0 con 0x55a570ddc4d0 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.579+0000 7f5a09400640 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570ddc4d0 0x55a5711ad720 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f59f80099f0 tx=0x7f59f80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4c47836ca7615a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.579+0000 7f59eb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.579+0000 7f59eb400640 1 -- 172.21.15.63:0/1059812977 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8035040 con 0x55a570ddc4d0 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.580+0000 7f59eb400640 1 -- 172.21.15.63:0/1059812977 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9571+0+0 (secure 0 0 0) 0x7f59f8003ca0 con 0x55a570ddc4d0 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.580+0000 7f5a17bda740 1 -- 172.21.15.63:0/1059812977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59fc02f070 con 0x55a570ddc4d0 2024-05-27T07:52:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.580+0000 7f59eb400640 1 -- 172.21.15.63:0/1059812977 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f59f8056050 con 0x55a570ddc4d0 2024-05-27T07:52:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.580+0000 7f5a17bda740 10 client.19029 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.580+0000 7f5a17bda740 1 -- 172.21.15.63:0/1059812977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a571223c00 con 0x55a570ddc4d0 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.580+0000 7f59eb400640 1 -- 172.21.15.63:0/1059812977 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59fc02f070 con 0x55a570ddc4d0 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.580+0000 7f59eb400640 1 -- 172.21.15.63:0/1059812977 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f59f8051030 con 0x55a570ddc4d0 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.581+0000 7f5a17bda740 10 client.19029 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.581+0000 7f5a17bda740 10 client.19029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.581+0000 7f5a17bda740 20 client.19029 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.581+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a570d911c0 0x55a571225f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.581+0000 7f5a17bda740 4 client.19029 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.581+0000 7f5a17bda740 1 -- 172.21.15.63:0/1059812977 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a57110c2c0 con 0x55a570d911c0 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.581+0000 7f5a09e00640 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a570d911c0 0x55a571225f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.582+0000 7f59eb400640 10 client.19029 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.582+0000 7f5a09e00640 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a570d911c0 0x55a571225f90 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f411094344bdeac server_cookie=1e5b9a7ca688ffed in_seq=0 out_seq=0 2024-05-27T07:52:12.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f59eb400640 1 -- 172.21.15.63:0/1059812977 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a57110c2c0 con 0x55a570d911c0 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f59eb400640 10 client.19029 handle_command_reply: tid=0 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f5a17bda740 1 client.19029 shutdown 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f5a17bda740 1 -- 172.21.15.63:0/1059812977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570ddc4d0 msgr2=0x55a5711ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570ddc4d0 0x55a5711ad720 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f59f80099f0 tx=0x7f59f80038a0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f5a17bda740 1 -- 172.21.15.63:0/1059812977 shutdown_connections 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a570d911c0 0x55a571225f90 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a570dfd1b0 0x55a571222590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a570ddc4d0 0x55a5711ad720 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f5a17bda740 1 --2- 172.21.15.63:0/1059812977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a570dd42f0 0x55a5711ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f59eb400640 0 client.19029 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.619+0000 7f5a17bda740 1 -- 172.21.15.63:0/1059812977 >> 172.21.15.63:0/1059812977 conn(0x55a570df1300 msgr2=0x55a57119afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.620+0000 7f5a17bda740 1 -- 172.21.15.63:0/1059812977 shutdown_connections 2024-05-27T07:52:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.620+0000 7f5a17bda740 1 -- 172.21.15.63:0/1059812977 wait complete. 2024-05-27T07:52:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.620+0000 7f5a17bda740 20 client.19029 trim_cache size 0 max 16384 2024-05-27T07:52:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:52:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:52:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:52:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 -- 172.21.15.63:0/575149148 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59f00b2950 msgr2=0x7f59f00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59f00b2950 0x7f59f00b4e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55a5711a0e10 tx=0x7f59fc00b4d0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 -- 172.21.15.63:0/575149148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a0c1102f0 msgr2=0x7f5a0c19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a0c1102f0 0x7f5a0c19c700 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400eaa0 tx=0x7f5a0400ef60 comp rx=0 tx=0).stop 2024-05-27T07:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 -- 172.21.15.63:0/575149148 shutdown_connections 2024-05-27T07:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f59f00b2950 0x7f59f00b4e60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a0c1102f0 0x7f5a0c19c700 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a0c10d990 0x7f5a0c1a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 --2- 172.21.15.63:0/575149148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a0c075480 0x7f5a0c1a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 -- 172.21.15.63:0/575149148 >> 172.21.15.63:0/575149148 conn(0x7f5a0c079240 msgr2=0x7f5a0c102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 -- 172.21.15.63:0/575149148 shutdown_connections 2024-05-27T07:52:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:12.621+0000 7f5a0b200640 1 -- 172.21.15.63:0/575149148 wait complete. 2024-05-27T07:52:13.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:13 smithi086 ceph-mon[34687]: pgmap v587: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 8 op/s 2024-05-27T07:52:13.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:13 smithi149 ceph-mon[35408]: pgmap v587: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 8 op/s 2024-05-27T07:52:13.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:13 smithi063 ceph-mon[31613]: pgmap v587: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 8 op/s 2024-05-27T07:52:15.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:15 smithi086 ceph-mon[34687]: pgmap v588: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 8 op/s 2024-05-27T07:52:15.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:15 smithi149 ceph-mon[35408]: pgmap v588: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 8 op/s 2024-05-27T07:52:15.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:15 smithi063 ceph-mon[31613]: pgmap v588: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 8 op/s 2024-05-27T07:52:16.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:16 smithi086 ceph-mon[34687]: osdmap e119: 12 total, 12 up, 12 in 2024-05-27T07:52:16.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:16 smithi086 ceph-mon[34687]: pgmap v590: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.2 MiB/s wr, 4 op/s 2024-05-27T07:52:16.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:16 smithi149 ceph-mon[35408]: osdmap e119: 12 total, 12 up, 12 in 2024-05-27T07:52:16.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:16 smithi149 ceph-mon[35408]: pgmap v590: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.2 MiB/s wr, 4 op/s 2024-05-27T07:52:16.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:16 smithi063 ceph-mon[31613]: osdmap e119: 12 total, 12 up, 12 in 2024-05-27T07:52:16.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:16 smithi063 ceph-mon[31613]: pgmap v590: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.2 MiB/s wr, 4 op/s 2024-05-27T07:52:19.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:18 smithi086 ceph-mon[34687]: pgmap v591: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 11 op/s 2024-05-27T07:52:19.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:18 smithi149 ceph-mon[35408]: pgmap v591: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 11 op/s 2024-05-27T07:52:19.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:18 smithi063 ceph-mon[31613]: pgmap v591: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 11 op/s 2024-05-27T07:52:21.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:20 smithi086 ceph-mon[34687]: pgmap v592: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 11 op/s 2024-05-27T07:52:21.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:20 smithi149 ceph-mon[35408]: pgmap v592: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 11 op/s 2024-05-27T07:52:21.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:20 smithi063 ceph-mon[31613]: pgmap v592: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 11 op/s 2024-05-27T07:52:23.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:22 smithi086 ceph-mon[34687]: pgmap v593: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:52:23.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:22 smithi149 ceph-mon[35408]: pgmap v593: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:52:23.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:22 smithi063 ceph-mon[31613]: pgmap v593: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:52:25.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:24 smithi149 ceph-mon[35408]: pgmap v594: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:52:25.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:24 smithi063 ceph-mon[31613]: pgmap v594: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:52:25.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:24 smithi086 ceph-mon[34687]: pgmap v594: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:52:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.870+0000 7fbbc8e00640 1 -- 172.21.15.63:0/583431452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc410bcd0 msgr2=0x7fbbc4105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.870+0000 7fbbc8e00640 1 --2- 172.21.15.63:0/583431452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc410bcd0 0x7fbbc4105870 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009900 tx=0x7fbbb402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.870+0000 7fbbc8e00640 1 -- 172.21.15.63:0/583431452 shutdown_connections 2024-05-27T07:52:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.870+0000 7fbbc8e00640 1 --2- 172.21.15.63:0/583431452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc4106420 0x7fbbc41127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.870+0000 7fbbc8e00640 1 --2- 172.21.15.63:0/583431452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc410c170 0x7fbbc4105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.870+0000 7fbbc8e00640 1 --2- 172.21.15.63:0/583431452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc410bcd0 0x7fbbc4105870 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.870+0000 7fbbc8e00640 1 -- 172.21.15.63:0/583431452 >> 172.21.15.63:0/583431452 conn(0x7fbbc41013d0 msgr2=0x7fbbc4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.870+0000 7fbbc8e00640 1 -- 172.21.15.63:0/583431452 shutdown_connections 2024-05-27T07:52:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.871+0000 7fbbc8e00640 1 -- 172.21.15.63:0/583431452 wait complete. 2024-05-27T07:52:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.872+0000 7fbbc8e00640 1 Processor -- start 2024-05-27T07:52:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.872+0000 7fbbc8e00640 1 -- start start 2024-05-27T07:52:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc8e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc4106420 0x7fbbc41a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc410bcd0 0x7fbbc41a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc410c170 0x7fbbc419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc8e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4115d50 con 0x7fbbc410c170 2024-05-27T07:52:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc8e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4115bd0 con 0x7fbbc410bcd0 2024-05-27T07:52:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc8e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4115ed0 con 0x7fbbc4106420 2024-05-27T07:52:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc410c170 0x7fbbc419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc410c170 0x7fbbc419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50678/0 (socket says 172.21.15.63:50678) 2024-05-27T07:52:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc3e00640 1 -- 172.21.15.63:0/4242217636 learned_addr learned my addr 172.21.15.63:0/4242217636 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc3400640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc4106420 0x7fbbc41a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.873+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc410bcd0 0x7fbbc41a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.874+0000 7fbbc3e00640 1 -- 172.21.15.63:0/4242217636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc4106420 msgr2=0x7fbbc41a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.874+0000 7fbbc3e00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc4106420 0x7fbbc41a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.874+0000 7fbbc3e00640 1 -- 172.21.15.63:0/4242217636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc410bcd0 msgr2=0x7fbbc41a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.874+0000 7fbbc3e00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc410bcd0 0x7fbbc41a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.874+0000 7fbbc3e00640 1 -- 172.21.15.63:0/4242217636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc419ccd0 con 0x7fbbc410c170 2024-05-27T07:52:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.874+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc410bcd0 0x7fbbc41a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.874+0000 7fbbc3e00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc410c170 0x7fbbc419c690 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009fd0 tx=0x7fbbbc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.875+0000 7fbbbbe00640 1 -- 172.21.15.63:0/4242217636 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc017070 con 0x7fbbc410c170 2024-05-27T07:52:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.875+0000 7fbbbbe00640 1 -- 172.21.15.63:0/4242217636 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbbc003e00 con 0x7fbbc410c170 2024-05-27T07:52:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.875+0000 7fbbbbe00640 1 -- 172.21.15.63:0/4242217636 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc010460 con 0x7fbbc410c170 2024-05-27T07:52:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.875+0000 7fbbc8e00640 1 -- 172.21.15.63:0/4242217636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc419cfc0 con 0x7fbbc410c170 2024-05-27T07:52:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.875+0000 7fbbc8e00640 1 -- 172.21.15.63:0/4242217636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc41a92b0 con 0x7fbbc410c170 2024-05-27T07:52:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.879+0000 7fbbd1413740 1 Processor -- start 2024-05-27T07:52:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.879+0000 7fbbbbe00640 1 -- 172.21.15.63:0/4242217636 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbbbc003970 con 0x7fbbc410c170 2024-05-27T07:52:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.879+0000 7fbbd1413740 1 -- start start 2024-05-27T07:52:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.880+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.880+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f03ee2f0 0x5627f07b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.880+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 0x5627f07ba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.880+0000 7fbbd1413740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5627f072ab90 con 0x5627f04171b0 2024-05-27T07:52:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.880+0000 7fbbd1413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627f071a730 con 0x5627f03f64d0 2024-05-27T07:52:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.880+0000 7fbbd1413740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5627f071e770 con 0x5627f03ee2f0 2024-05-27T07:52:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.880+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.880+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07b9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50684/0 (socket says 172.21.15.63:50684) 2024-05-27T07:52:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.880+0000 7fbbc2a00640 1 -- 172.21.15.63:0/2346848012 learned_addr learned my addr 172.21.15.63:0/2346848012 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.881+0000 7fbbc3400640 1 --2- 172.21.15.63:0/2346848012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f03ee2f0 0x5627f07b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.881+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/2346848012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 0x5627f07ba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.881+0000 7fbbc2a00640 1 -- 172.21.15.63:0/2346848012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f03ee2f0 msgr2=0x5627f07b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.881+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/2346848012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f03ee2f0 0x5627f07b9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.881+0000 7fbbc2a00640 1 -- 172.21.15.63:0/2346848012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 msgr2=0x5627f07ba4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.881+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/2346848012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 0x5627f07ba4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.881+0000 7fbbc2a00640 1 -- 172.21.15.63:0/2346848012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f06b38b0 con 0x5627f04171b0 2024-05-27T07:52:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.881+0000 7fbbbbe00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbba80b29d0 0x7fbba80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.881+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/2346848012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07b9a60 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ce50 tx=0x7fbbb000c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669a3b8a150e8561 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.882+0000 7fbbbbe00640 1 -- 172.21.15.63:0/4242217636 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fbbbc0e2b50 con 0x7fbbc410c170 2024-05-27T07:52:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.882+0000 7fbbc3400640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbba80b29d0 0x7fbba80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.882+0000 7fbbb8c00640 1 -- 172.21.15.63:0/2346848012 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0004230 con 0x5627f04171b0 2024-05-27T07:52:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.882+0000 7fbbb8c00640 1 -- 172.21.15.63:0/2346848012 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbb0016070 con 0x5627f04171b0 2024-05-27T07:52:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.882+0000 7fbbb8c00640 1 -- 172.21.15.63:0/2346848012 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb00126b0 con 0x5627f04171b0 2024-05-27T07:52:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.883+0000 7fbbd1413740 1 -- 172.21.15.63:0/2346848012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 msgr2=0x5627f07b9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.883+0000 7fbbc3400640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbba80b29d0 0x7fbba80b4ee0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4004770 tx=0x7fbbb40058d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.883+0000 7fbbd1413740 1 --2- 172.21.15.63:0/2346848012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07b9a60 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ce50 tx=0x7fbbb000c600 comp rx=0 tx=0).stop 2024-05-27T07:52:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.883+0000 7fbbd1413740 1 -- 172.21.15.63:0/2346848012 shutdown_connections 2024-05-27T07:52:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.883+0000 7fbbd1413740 1 --2- 172.21.15.63:0/2346848012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 0x5627f07ba4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.883+0000 7fbbd1413740 1 --2- 172.21.15.63:0/2346848012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f03ee2f0 0x5627f07b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.883+0000 7fbbd1413740 1 --2- 172.21.15.63:0/2346848012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07b9a60 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.883+0000 7fbbd1413740 1 -- 172.21.15.63:0/2346848012 >> 172.21.15.63:0/2346848012 conn(0x5627f040b300 msgr2=0x5627f07b6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.883+0000 7fbbd1413740 1 -- 172.21.15.63:0/2346848012 shutdown_connections 2024-05-27T07:52:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.884+0000 7fbbd1413740 1 -- 172.21.15.63:0/2346848012 wait complete. 2024-05-27T07:52:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.884+0000 7fbbd1413740 1 -- start start 2024-05-27T07:52:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.884+0000 7fbbd1413740 1 Processor -- start 2024-05-27T07:52:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.885+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03ee2f0 0x5627f07c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.885+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f03f64d0 0x5627f07c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.885+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.885+0000 7fbbd1413740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5627f0400790 con 0x5627f04171b0 2024-05-27T07:52:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.885+0000 7fbbd1413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627f06c2bc0 con 0x5627f03ee2f0 2024-05-27T07:52:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.885+0000 7fbbd1413740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5627f07c6b90 con 0x5627f03f64d0 2024-05-27T07:52:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.885+0000 7fbbd1413740 10 client.0 _ll_register_callbacks cb 0x5627f06a0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.885+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.885+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50700/0 (socket says 172.21.15.63:50700) 2024-05-27T07:52:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.885+0000 7fbbc3e00640 1 -- 172.21.15.63:0/2063293602 learned_addr learned my addr 172.21.15.63:0/2063293602 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.886+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f03f64d0 0x5627f07c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.886+0000 7fbbc3400640 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03ee2f0 0x5627f07c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.886+0000 7fbbc3e00640 1 -- 172.21.15.63:0/2063293602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f03f64d0 msgr2=0x5627f07c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.886+0000 7fbbc3e00640 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f03f64d0 0x5627f07c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.886+0000 7fbbc3e00640 1 -- 172.21.15.63:0/2063293602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03ee2f0 msgr2=0x5627f07c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.886+0000 7fbbc3e00640 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03ee2f0 0x5627f07c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.886+0000 7fbbc3e00640 1 -- 172.21.15.63:0/2063293602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f042f210 con 0x5627f04171b0 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.886+0000 7fbbc3e00640 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07cfec0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00dfd0 tx=0x7fbbbc003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f725f789dee3b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.886+0000 7fbbad600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.887+0000 7fbbad600640 1 -- 172.21.15.63:0/2063293602 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc020070 con 0x5627f04171b0 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.887+0000 7fbbad600640 1 -- 172.21.15.63:0/2063293602 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fbbbc0b23e0 con 0x5627f04171b0 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.887+0000 7fbbd1413740 1 -- 172.21.15.63:0/2063293602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb0016070 con 0x5627f04171b0 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.887+0000 7fbbad600640 1 -- 172.21.15.63:0/2063293602 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbbbc0bee60 con 0x5627f04171b0 2024-05-27T07:52:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.887+0000 7fbbd1413740 10 client.19041 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.887+0000 7fbbd1413740 1 -- 172.21.15.63:0/2063293602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5627f04fb620 con 0x5627f04171b0 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.887+0000 7fbbad600640 1 -- 172.21.15.63:0/2063293602 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbb0016070 con 0x5627f04171b0 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.887+0000 7fbbad600640 1 -- 172.21.15.63:0/2063293602 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbbbc0cc0d0 con 0x5627f04171b0 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.888+0000 7fbbd1413740 10 client.19041 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.888+0000 7fbbd1413740 10 client.19041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.888+0000 7fbbd1413740 20 client.19041 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.888+0000 7fbbd1413740 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627f03d5d80 0x5627f07d5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.888+0000 7fbbd1413740 4 client.19041 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.888+0000 7fbbd1413740 1 -- 172.21.15.63:0/2063293602 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627f072cae0 con 0x5627f03d5d80 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.888+0000 7fbbc3400640 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627f03d5d80 0x5627f07d5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.890+0000 7fbbc3400640 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627f03d5d80 0x5627f07d5fc0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1238d21e582fb50 server_cookie=5e190a3283763137 in_seq=0 out_seq=0 2024-05-27T07:52:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.890+0000 7fbbad600640 10 client.19041 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.891+0000 7fbbad600640 1 -- 172.21.15.63:0/2063293602 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5627f072cae0 con 0x5627f03d5d80 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.891+0000 7fbbad600640 10 client.19041 handle_command_reply: tid=0 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 client.19041 shutdown 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 -- 172.21.15.63:0/2063293602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 msgr2=0x5627f07cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07cfec0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00dfd0 tx=0x7fbbbc003d90 comp rx=0 tx=0).stop 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 -- 172.21.15.63:0/2063293602 shutdown_connections 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627f03d5d80 0x5627f07d5fc0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f04171b0 0x5627f07cfec0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f03f64d0 0x5627f07c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 --2- 172.21.15.63:0/2063293602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03ee2f0 0x5627f07c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbad600640 0 client.19041 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 -- 172.21.15.63:0/2063293602 >> 172.21.15.63:0/2063293602 conn(0x5627f040b300 msgr2=0x5627f07b9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 -- 172.21.15.63:0/2063293602 shutdown_connections 2024-05-27T07:52:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 1 -- 172.21.15.63:0/2063293602 wait complete. 2024-05-27T07:52:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.892+0000 7fbbd1413740 20 client.19041 trim_cache size 0 max 16384 2024-05-27T07:52:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.910+0000 7fbbd1413740 1 Processor -- start 2024-05-27T07:52:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.910+0000 7fbbd1413740 1 -- start start 2024-05-27T07:52:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.910+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03ee2f0 0x5627f07bf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.910+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03f64d0 0x5627f07c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.910+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 0x5627f07c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.910+0000 7fbbd1413740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41b5ac0 con 0x5627f03f64d0 2024-05-27T07:52:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.910+0000 7fbbd1413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41b3390 con 0x5627f03ee2f0 2024-05-27T07:52:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.910+0000 7fbbd1413740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41b1200 con 0x5627f04171b0 2024-05-27T07:52:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03f64d0 0x5627f07c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03f64d0 0x5627f07c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50714/0 (socket says 172.21.15.63:50714) 2024-05-27T07:52:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc2a00640 1 -- 172.21.15.63:0/1311043299 learned_addr learned my addr 172.21.15.63:0/1311043299 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc3400640 1 --2- 172.21.15.63:0/1311043299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03ee2f0 0x5627f07bf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc3e00640 1 --2- 172.21.15.63:0/1311043299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 0x5627f07c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc2a00640 1 -- 172.21.15.63:0/1311043299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 msgr2=0x5627f07c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/1311043299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 0x5627f07c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc2a00640 1 -- 172.21.15.63:0/1311043299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03ee2f0 msgr2=0x5627f07bf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/1311043299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03ee2f0 0x5627f07bf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc2a00640 1 -- 172.21.15.63:0/1311043299 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f07c4830 con 0x5627f03f64d0 2024-05-27T07:52:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.911+0000 7fbbc3e00640 1 --2- 172.21.15.63:0/1311043299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 0x5627f07c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.912+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/1311043299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03f64d0 0x5627f07c5b80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ce50 tx=0x7fbbb0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f66663cd9f202a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.912+0000 7fbbafe00640 1 -- 172.21.15.63:0/1311043299 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0012650 con 0x5627f03f64d0 2024-05-27T07:52:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.912+0000 7fbbafe00640 1 -- 172.21.15.63:0/1311043299 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbb0016070 con 0x5627f03f64d0 2024-05-27T07:52:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.912+0000 7fbbafe00640 1 -- 172.21.15.63:0/1311043299 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0003850 con 0x5627f03f64d0 2024-05-27T07:52:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.913+0000 7fbbd1413740 1 -- 172.21.15.63:0/1311043299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03f64d0 msgr2=0x5627f07c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.913+0000 7fbbd1413740 1 --2- 172.21.15.63:0/1311043299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03f64d0 0x5627f07c5b80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ce50 tx=0x7fbbb0005760 comp rx=0 tx=0).stop 2024-05-27T07:52:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.913+0000 7fbbd1413740 1 -- 172.21.15.63:0/1311043299 shutdown_connections 2024-05-27T07:52:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.913+0000 7fbbd1413740 1 --2- 172.21.15.63:0/1311043299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 0x5627f07c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.913+0000 7fbbd1413740 1 --2- 172.21.15.63:0/1311043299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03f64d0 0x5627f07c5b80 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.913+0000 7fbbd1413740 1 --2- 172.21.15.63:0/1311043299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03ee2f0 0x5627f07bf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.913+0000 7fbbd1413740 1 -- 172.21.15.63:0/1311043299 >> 172.21.15.63:0/1311043299 conn(0x5627f040b300 msgr2=0x5627f07c5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.913+0000 7fbbd1413740 1 -- 172.21.15.63:0/1311043299 shutdown_connections 2024-05-27T07:52:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.913+0000 7fbbd1413740 1 -- 172.21.15.63:0/1311043299 wait complete. 2024-05-27T07:52:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.914+0000 7fbbd1413740 1 -- start start 2024-05-27T07:52:25.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.914+0000 7fbbd1413740 1 Processor -- start 2024-05-27T07:52:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03ee2f0 0x5627f07cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 0x5627f07c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbd1413740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 0x5627f07c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbd1413740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41b1850 con 0x5627f03ee2f0 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbd1413740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41b2cc0 con 0x5627f03f64d0 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbd1413740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41b1f20 con 0x5627f04171b0 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbd1413740 10 client.0 _ll_register_callbacks cb 0x7fbbbc009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbc3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03ee2f0 0x5627f07cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbc3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03ee2f0 0x5627f07cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50718/0 (socket says 172.21.15.63:50718) 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbc3400640 1 -- 172.21.15.63:0/1010245216 learned_addr learned my addr 172.21.15.63:0/1010245216 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.915+0000 7fbbc3e00640 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 0x5627f07c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 0x5627f07c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbc3400640 1 -- 172.21.15.63:0/1010245216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 msgr2=0x5627f07c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbc3400640 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 0x5627f07c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbc3400640 1 -- 172.21.15.63:0/1010245216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 msgr2=0x5627f07c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbc3400640 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 0x5627f07c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbc3400640 1 -- 172.21.15.63:0/1010245216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f07ca030 con 0x5627f03ee2f0 2024-05-27T07:52:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbc3e00640 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 0x5627f07c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 0x5627f07c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbc3400640 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03ee2f0 0x5627f07cc1f0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4000c00 tx=0x7fbbb4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb39a8b5202e0a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbad600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbad600640 1 -- 172.21.15.63:0/1010245216 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb404a020 con 0x5627f03ee2f0 2024-05-27T07:52:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbad600640 1 -- 172.21.15.63:0/1010245216 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fbbb40035f0 con 0x5627f03ee2f0 2024-05-27T07:52:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.916+0000 7fbbd1413740 1 -- 172.21.15.63:0/1010245216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb0016070 con 0x5627f03ee2f0 2024-05-27T07:52:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbad600640 1 -- 172.21.15.63:0/1010245216 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbbb4035070 con 0x5627f03ee2f0 2024-05-27T07:52:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbd1413740 10 client.19047 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbd1413740 1 -- 172.21.15.63:0/1010245216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5627f083e1a0 con 0x5627f03ee2f0 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbad600640 1 -- 172.21.15.63:0/1010245216 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbb0016070 con 0x5627f03ee2f0 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbad600640 1 -- 172.21.15.63:0/1010245216 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbbb4043040 con 0x5627f03ee2f0 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbd1413740 10 client.19047 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbd1413740 10 client.19047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbd1413740 20 client.19047 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbd1413740 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627f03ab1c0 0x5627f0840960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbd1413740 4 client.19047 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.917+0000 7fbbd1413740 1 -- 172.21.15.63:0/1010245216 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627f07262c0 con 0x5627f03ab1c0 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.918+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627f03ab1c0 0x5627f0840960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.919+0000 7fbbc2a00640 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627f03ab1c0 0x5627f0840960 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8daef06579be420 server_cookie=98b63b0164e13b1f in_seq=0 out_seq=0 2024-05-27T07:52:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.919+0000 7fbbad600640 10 client.19047 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:25.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.959+0000 7fbbad600640 1 -- 172.21.15.63:0/1010245216 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5627f07262c0 con 0x5627f03ab1c0 2024-05-27T07:52:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.959+0000 7fbbad600640 10 client.19047 handle_command_reply: tid=0 2024-05-27T07:52:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.959+0000 7fbbd1413740 1 client.19047 shutdown 2024-05-27T07:52:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbd1413740 1 -- 172.21.15.63:0/1010245216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03ee2f0 msgr2=0x5627f07cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbd1413740 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03ee2f0 0x5627f07cc1f0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4000c00 tx=0x7fbbb4009260 comp rx=0 tx=0).stop 2024-05-27T07:52:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbd1413740 1 -- 172.21.15.63:0/1010245216 shutdown_connections 2024-05-27T07:52:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbd1413740 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627f03ab1c0 0x5627f0840960 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbd1413740 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f04171b0 0x5627f07c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbd1413740 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f03f64d0 0x5627f07c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbd1413740 1 --2- 172.21.15.63:0/1010245216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f03ee2f0 0x5627f07cc1f0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbd1413740 1 -- 172.21.15.63:0/1010245216 >> 172.21.15.63:0/1010245216 conn(0x5627f040b300 msgr2=0x5627f07da760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbad600640 0 client.19047 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbd1413740 1 -- 172.21.15.63:0/1010245216 shutdown_connections 2024-05-27T07:52:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.960+0000 7fbbd1413740 1 -- 172.21.15.63:0/1010245216 wait complete. 2024-05-27T07:52:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.961+0000 7fbbd1413740 20 client.19047 trim_cache size 0 max 16384 2024-05-27T07:52:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:52:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:52:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:52:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.961+0000 7fbbc8e00640 1 -- 172.21.15.63:0/4242217636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbba80b29d0 msgr2=0x7fbba80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.961+0000 7fbbc8e00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbba80b29d0 0x7fbba80b4ee0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4004770 tx=0x7fbbb40058d0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.961+0000 7fbbc8e00640 1 -- 172.21.15.63:0/4242217636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc410c170 msgr2=0x7fbbc419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.961+0000 7fbbc8e00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc410c170 0x7fbbc419c690 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009fd0 tx=0x7fbbbc00b260 comp rx=0 tx=0).stop 2024-05-27T07:52:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.962+0000 7fbbc8e00640 1 -- 172.21.15.63:0/4242217636 shutdown_connections 2024-05-27T07:52:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.962+0000 7fbbc8e00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbba80b29d0 0x7fbba80b4ee0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.962+0000 7fbbc8e00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbc410c170 0x7fbbc419c690 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.962+0000 7fbbc8e00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbc410bcd0 0x7fbbc41a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.962+0000 7fbbc8e00640 1 --2- 172.21.15.63:0/4242217636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbc4106420 0x7fbbc41a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.962+0000 7fbbc8e00640 1 -- 172.21.15.63:0/4242217636 >> 172.21.15.63:0/4242217636 conn(0x7fbbc41013d0 msgr2=0x7fbbc41017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.962+0000 7fbbc8e00640 1 -- 172.21.15.63:0/4242217636 shutdown_connections 2024-05-27T07:52:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:25.962+0000 7fbbc8e00640 1 -- 172.21.15.63:0/4242217636 wait complete. 2024-05-27T07:52:27.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:26 smithi063 ceph-mon[31613]: pgmap v595: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:52:27.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:26 smithi086 ceph-mon[34687]: pgmap v595: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:52:27.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:26 smithi149 ceph-mon[35408]: pgmap v595: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:52:29.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:28 smithi149 ceph-mon[35408]: pgmap v596: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T07:52:29.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:28 smithi086 ceph-mon[34687]: pgmap v596: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T07:52:29.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:28 smithi063 ceph-mon[31613]: pgmap v596: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T07:52:31.269 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:30 smithi063 ceph-mon[31613]: pgmap v597: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.4 MiB/s wr, 10 op/s 2024-05-27T07:52:31.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:30 smithi086 ceph-mon[34687]: pgmap v597: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.4 MiB/s wr, 10 op/s 2024-05-27T07:52:31.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:30 smithi149 ceph-mon[35408]: pgmap v597: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.4 MiB/s wr, 10 op/s 2024-05-27T07:52:33.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:32 smithi086 ceph-mon[34687]: pgmap v598: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:52:33.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:32 smithi149 ceph-mon[35408]: pgmap v598: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:52:33.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:32 smithi063 ceph-mon[31613]: pgmap v598: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:52:35.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:34 smithi086 ceph-mon[34687]: pgmap v599: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:52:35.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:34 smithi149 ceph-mon[35408]: pgmap v599: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:52:35.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:34 smithi063 ceph-mon[31613]: pgmap v599: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:52:36.175 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:52:36.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:36 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:52:36.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:36 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:52:36.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:36 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:52:36.694 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:37.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:37 smithi063 ceph-mon[31613]: pgmap v600: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:52:37.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:37 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:52:37.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:37 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:52:37.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:37 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:52:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:37 smithi086 ceph-mon[34687]: pgmap v600: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:52:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:37 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:52:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:37 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:52:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:37 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:52:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:37 smithi149 ceph-mon[35408]: pgmap v600: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:52:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:37 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:52:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:37 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:52:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:37 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:52:39.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:39 smithi063 ceph-mon[31613]: pgmap v601: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 13 op/s 2024-05-27T07:52:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.220+0000 7f57d7200640 1 -- 172.21.15.63:0/4170726026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57d010b670 msgr2=0x7f57d010da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.220+0000 7f57d7200640 1 --2- 172.21.15.63:0/4170726026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57d010b670 0x7f57d010da90 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009860 tx=0x7f57c002a3d0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.220+0000 7f57d7200640 1 -- 172.21.15.63:0/4170726026 shutdown_connections 2024-05-27T07:52:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.220+0000 7f57d7200640 1 --2- 172.21.15.63:0/4170726026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57d010e020 0x7f57d0110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.220+0000 7f57d7200640 1 --2- 172.21.15.63:0/4170726026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57d010b670 0x7f57d010da90 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.220+0000 7f57d7200640 1 --2- 172.21.15.63:0/4170726026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0104c10 0x7f57d0105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.220+0000 7f57d7200640 1 -- 172.21.15.63:0/4170726026 >> 172.21.15.63:0/4170726026 conn(0x7f57d00fe210 msgr2=0x7f57d0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.220+0000 7f57d7200640 1 -- 172.21.15.63:0/4170726026 shutdown_connections 2024-05-27T07:52:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.220+0000 7f57d7200640 1 -- 172.21.15.63:0/4170726026 wait complete. 2024-05-27T07:52:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.221+0000 7f57d7200640 1 Processor -- start 2024-05-27T07:52:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.222+0000 7f57d7200640 1 -- start start 2024-05-27T07:52:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.222+0000 7f57d7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0104c10 0x7f57d01024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.222+0000 7f57d7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57d010b670 0x7f57d0102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.222+0000 7f57d7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57d010e020 0x7f57d0102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.222+0000 7f57d7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57d0113b80 con 0x7f57d010e020 2024-05-27T07:52:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.222+0000 7f57d7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57d0113a00 con 0x7f57d0104c10 2024-05-27T07:52:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.222+0000 7f57d7200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57d0113d00 con 0x7f57d010b670 2024-05-27T07:52:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57d010e020 0x7f57d0102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0104c10 0x7f57d01024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57d010e020 0x7f57d0102f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44766/0 (socket says 172.21.15.63:44766) 2024-05-27T07:52:39.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0104c10 0x7f57d01024d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44888/0 (socket says 172.21.15.63:44888) 2024-05-27T07:52:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d6800640 1 -- 172.21.15.63:0/1279205902 learned_addr learned my addr 172.21.15.63:0/1279205902 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:39.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d5400640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57d010b670 0x7f57d0102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d6800640 1 -- 172.21.15.63:0/1279205902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57d010b670 msgr2=0x7f57d0102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d6800640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57d010b670 0x7f57d0102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d6800640 1 -- 172.21.15.63:0/1279205902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0104c10 msgr2=0x7f57d01024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d6800640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0104c10 0x7f57d01024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.223+0000 7f57d6800640 1 -- 172.21.15.63:0/1279205902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d01a88e0 con 0x7f57d010e020 2024-05-27T07:52:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.224+0000 7f57d6800640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57d010e020 0x7f57d0102f50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00eb10 tx=0x7f57cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.224+0000 7f57caa00640 1 -- 172.21.15.63:0/1279205902 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc004b20 con 0x7f57d010e020 2024-05-27T07:52:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.224+0000 7f57caa00640 1 -- 172.21.15.63:0/1279205902 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57cc004cc0 con 0x7f57d010e020 2024-05-27T07:52:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.224+0000 7f57d7200640 1 -- 172.21.15.63:0/1279205902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d01a8bd0 con 0x7f57d010e020 2024-05-27T07:52:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.224+0000 7f57caa00640 1 -- 172.21.15.63:0/1279205902 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc010690 con 0x7f57d010e020 2024-05-27T07:52:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.224+0000 7f57d7200640 1 -- 172.21.15.63:0/1279205902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d01a90f0 con 0x7f57d010e020 2024-05-27T07:52:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.229+0000 7f57df7cd740 1 Processor -- start 2024-05-27T07:52:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.229+0000 7f57caa00640 1 -- 172.21.15.63:0/1279205902 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f57cc005020 con 0x7f57d010e020 2024-05-27T07:52:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.229+0000 7f57df7cd740 1 -- start start 2024-05-27T07:52:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.230+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 0x55e714a9ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.230+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714a9bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.230+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714a9c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.230+0000 7f57df7cd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e714a0cb90 con 0x55e7146d02f0 2024-05-27T07:52:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.230+0000 7f57df7cd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e7149fc730 con 0x55e7146f91b0 2024-05-27T07:52:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.230+0000 7f57df7cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e714a00770 con 0x55e7146d84d0 2024-05-27T07:52:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.230+0000 7f57d5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714a9bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.230+0000 7f57d5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714a9bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44780/0 (socket says 172.21.15.63:44780) 2024-05-27T07:52:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.230+0000 7f57d5e00640 1 -- 172.21.15.63:0/1163094490 learned_addr learned my addr 172.21.15.63:0/1163094490 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.231+0000 7f57d5400640 1 --2- 172.21.15.63:0/1163094490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 0x55e714a9ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.231+0000 7f57d5400640 1 --2- 172.21.15.63:0/1163094490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714a9c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.231+0000 7f57d5e00640 1 -- 172.21.15.63:0/1163094490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 msgr2=0x55e714a9c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.231+0000 7f57d5e00640 1 --2- 172.21.15.63:0/1163094490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714a9c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.231+0000 7f57d5e00640 1 -- 172.21.15.63:0/1163094490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 msgr2=0x55e714a9ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.231+0000 7f57d5e00640 1 --2- 172.21.15.63:0/1163094490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 0x55e714a9ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.231+0000 7f57d5e00640 1 -- 172.21.15.63:0/1163094490 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7149958b0 con 0x55e7146d02f0 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.231+0000 7f57d5400640 1 --2- 172.21.15.63:0/1163094490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 0x55e714a9ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.231+0000 7f57caa00640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57b80b2950 0x7f57b80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.231+0000 7f57d5e00640 1 --2- 172.21.15.63:0/1163094490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714a9bfa0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009ca0 tx=0x7f57c4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd83003570ffa57 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.232+0000 7f57caa00640 1 -- 172.21.15.63:0/1279205902 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f57cc0e2f60 con 0x7f57d010e020 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.232+0000 7f57d5400640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57b80b2950 0x7f57b80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.232+0000 7f57bf000640 1 -- 172.21.15.63:0/1163094490 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4005220 con 0x55e7146d02f0 2024-05-27T07:52:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.232+0000 7f57bf000640 1 -- 172.21.15.63:0/1163094490 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57c4009e50 con 0x55e7146d02f0 2024-05-27T07:52:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.232+0000 7f57bf000640 1 -- 172.21.15.63:0/1163094490 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4019690 con 0x55e7146d02f0 2024-05-27T07:52:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.233+0000 7f57df7cd740 1 -- 172.21.15.63:0/1163094490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 msgr2=0x55e714a9bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.233+0000 7f57d5400640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57b80b2950 0x7f57b80b4e60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f57c0005cd0 tx=0x7f57c00404e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.233+0000 7f57df7cd740 1 --2- 172.21.15.63:0/1163094490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714a9bfa0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009ca0 tx=0x7f57c4011810 comp rx=0 tx=0).stop 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.233+0000 7f57df7cd740 1 -- 172.21.15.63:0/1163094490 shutdown_connections 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.233+0000 7f57df7cd740 1 --2- 172.21.15.63:0/1163094490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714a9c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.233+0000 7f57df7cd740 1 --2- 172.21.15.63:0/1163094490 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714a9bfa0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.233+0000 7f57df7cd740 1 --2- 172.21.15.63:0/1163094490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 0x55e714a9ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.233+0000 7f57df7cd740 1 -- 172.21.15.63:0/1163094490 >> 172.21.15.63:0/1163094490 conn(0x55e7146ed300 msgr2=0x55e714a98210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.233+0000 7f57df7cd740 1 -- 172.21.15.63:0/1163094490 shutdown_connections 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.234+0000 7f57df7cd740 1 -- 172.21.15.63:0/1163094490 wait complete. 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.234+0000 7f57df7cd740 1 -- start start 2024-05-27T07:52:39.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.234+0000 7f57df7cd740 1 Processor -- start 2024-05-27T07:52:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.235+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 0x55e714aa5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.235+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d84d0 0x55e714aa6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.235+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146f91b0 0x55e714ab1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.235+0000 7f57df7cd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e7146e2790 con 0x55e7146d84d0 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.235+0000 7f57df7cd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e7149a4bc0 con 0x55e7146d02f0 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.235+0000 7f57df7cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e714aa8b90 con 0x55e7146f91b0 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.235+0000 7f57df7cd740 10 client.0 _ll_register_callbacks cb 0x55e714982400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.235+0000 7f57d5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d84d0 0x55e714aa6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.235+0000 7f57d5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d84d0 0x55e714aa6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44786/0 (socket says 172.21.15.63:44786) 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.235+0000 7f57d5400640 1 -- 172.21.15.63:0/4206799150 learned_addr learned my addr 172.21.15.63:0/4206799150 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.236+0000 7f57d5e00640 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 0x55e714aa5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.236+0000 7f57d6800640 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146f91b0 0x55e714ab1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.236+0000 7f57d5400640 1 -- 172.21.15.63:0/4206799150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146f91b0 msgr2=0x55e714ab1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.236+0000 7f57d5400640 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146f91b0 0x55e714ab1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.236+0000 7f57d5400640 1 -- 172.21.15.63:0/4206799150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 msgr2=0x55e714aa5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.236+0000 7f57d5400640 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 0x55e714aa5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.236+0000 7f57d5400640 1 -- 172.21.15.63:0/4206799150 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e714711210 con 0x55e7146d84d0 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.236+0000 7f57d5e00640 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 0x55e714aa5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.236+0000 7f57d5400640 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d84d0 0x55e714aa6110 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f57c0034000 tx=0x7f57c002a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbfa2edffc7a12f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.236+0000 7f57b3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.237+0000 7f57b3400640 1 -- 172.21.15.63:0/4206799150 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c004f070 con 0x55e7146d84d0 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.237+0000 7f57df7cd740 1 -- 172.21.15.63:0/4206799150 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57c4009e50 con 0x55e7146d84d0 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.237+0000 7f57b3400640 1 -- 172.21.15.63:0/4206799150 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f57c0040880 con 0x55e7146d84d0 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.237+0000 7f57b3400640 1 -- 172.21.15.63:0/4206799150 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f57c0004e50 con 0x55e7146d84d0 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.237+0000 7f57df7cd740 10 client.19059 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.237+0000 7f57df7cd740 1 -- 172.21.15.63:0/4206799150 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e7147dd620 con 0x55e7146d84d0 2024-05-27T07:52:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.237+0000 7f57b3400640 1 -- 172.21.15.63:0/4206799150 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57c4009e50 con 0x55e7146d84d0 2024-05-27T07:52:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.238+0000 7f57b3400640 1 -- 172.21.15.63:0/4206799150 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f57c003cac0 con 0x55e7146d84d0 2024-05-27T07:52:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.238+0000 7f57df7cd740 10 client.19059 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.238+0000 7f57df7cd740 10 client.19059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.238+0000 7f57df7cd740 20 client.19059 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.238+0000 7f57df7cd740 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e7146b7d80 0x55e714ab7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.238+0000 7f57df7cd740 4 client.19059 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.238+0000 7f57df7cd740 1 -- 172.21.15.63:0/4206799150 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e714a0eae0 con 0x55e7146b7d80 2024-05-27T07:52:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.239+0000 7f57d5e00640 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e7146b7d80 0x55e714ab7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.240+0000 7f57b3400640 10 client.19059 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.240+0000 7f57d5e00640 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e7146b7d80 0x55e714ab7ea0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b714b20a85789e server_cookie=26163240de223282 in_seq=0 out_seq=0 2024-05-27T07:52:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57b3400640 1 -- 172.21.15.63:0/4206799150 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e714a0eae0 con 0x55e7146b7d80 2024-05-27T07:52:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57b3400640 10 client.19059 handle_command_reply: tid=0 2024-05-27T07:52:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57df7cd740 1 client.19059 shutdown 2024-05-27T07:52:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57df7cd740 1 -- 172.21.15.63:0/4206799150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d84d0 msgr2=0x55e714aa6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57df7cd740 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d84d0 0x55e714aa6110 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f57c0034000 tx=0x7f57c002a3d0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57df7cd740 1 -- 172.21.15.63:0/4206799150 shutdown_connections 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57df7cd740 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e7146b7d80 0x55e714ab7ea0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57df7cd740 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146f91b0 0x55e714ab1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57df7cd740 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d84d0 0x55e714aa6110 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57df7cd740 1 --2- 172.21.15.63:0/4206799150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 0x55e714aa5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.241+0000 7f57b3400640 0 client.19059 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.242+0000 7f57df7cd740 1 -- 172.21.15.63:0/4206799150 >> 172.21.15.63:0/4206799150 conn(0x55e7146ed300 msgr2=0x55e714a9d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.242+0000 7f57df7cd740 1 -- 172.21.15.63:0/4206799150 shutdown_connections 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.242+0000 7f57df7cd740 1 -- 172.21.15.63:0/4206799150 wait complete. 2024-05-27T07:52:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.242+0000 7f57df7cd740 20 client.19059 trim_cache size 0 max 16384 2024-05-27T07:52:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.264+0000 7f57df7cd740 1 Processor -- start 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.264+0000 7f57df7cd740 1 -- start start 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.264+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 0x55e714ab1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.264+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714ab2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.264+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146f91b0 0x55e714a9f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.264+0000 7f57df7cd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01b5a70 con 0x55e7146f91b0 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.264+0000 7f57df7cd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01b3340 con 0x55e7146d02f0 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.264+0000 7f57df7cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01b11b0 con 0x55e7146d84d0 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146f91b0 0x55e714a9f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146f91b0 0x55e714a9f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44800/0 (socket says 172.21.15.63:44800) 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d6800640 1 -- 172.21.15.63:0/3986383915 learned_addr learned my addr 172.21.15.63:0/3986383915 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d5e00640 1 --2- 172.21.15.63:0/3986383915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 0x55e714ab1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d5400640 1 --2- 172.21.15.63:0/3986383915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714ab2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d6800640 1 -- 172.21.15.63:0/3986383915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 msgr2=0x55e714ab2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d6800640 1 --2- 172.21.15.63:0/3986383915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714ab2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d6800640 1 -- 172.21.15.63:0/3986383915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 msgr2=0x55e714ab1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d6800640 1 --2- 172.21.15.63:0/3986383915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 0x55e714ab1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d6800640 1 -- 172.21.15.63:0/3986383915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e714a98b10 con 0x55e7146f91b0 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.265+0000 7f57d5e00640 1 --2- 172.21.15.63:0/3986383915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 0x55e714ab1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.266+0000 7f57d6800640 1 --2- 172.21.15.63:0/3986383915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146f91b0 0x55e714a9f710 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0e02f0 tx=0x7f57cc0a9770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3add2af8b0ec96a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.266+0000 7f57be600640 1 -- 172.21.15.63:0/3986383915 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0aa520 con 0x55e7146f91b0 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.266+0000 7f57be600640 1 -- 172.21.15.63:0/3986383915 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57cc0a9e20 con 0x55e7146f91b0 2024-05-27T07:52:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.266+0000 7f57be600640 1 -- 172.21.15.63:0/3986383915 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0aa0c0 con 0x55e7146f91b0 2024-05-27T07:52:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.267+0000 7f57df7cd740 1 -- 172.21.15.63:0/3986383915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146f91b0 msgr2=0x55e714a9f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.267+0000 7f57df7cd740 1 --2- 172.21.15.63:0/3986383915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146f91b0 0x55e714a9f710 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0e02f0 tx=0x7f57cc0a9770 comp rx=0 tx=0).stop 2024-05-27T07:52:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.267+0000 7f57df7cd740 1 -- 172.21.15.63:0/3986383915 shutdown_connections 2024-05-27T07:52:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.267+0000 7f57df7cd740 1 --2- 172.21.15.63:0/3986383915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146f91b0 0x55e714a9f710 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.267+0000 7f57df7cd740 1 --2- 172.21.15.63:0/3986383915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714ab2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.267+0000 7f57df7cd740 1 --2- 172.21.15.63:0/3986383915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146d02f0 0x55e714ab1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.267+0000 7f57df7cd740 1 -- 172.21.15.63:0/3986383915 >> 172.21.15.63:0/3986383915 conn(0x55e7146ed300 msgr2=0x55e714aa5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.267+0000 7f57df7cd740 1 -- 172.21.15.63:0/3986383915 shutdown_connections 2024-05-27T07:52:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.267+0000 7f57df7cd740 1 -- 172.21.15.63:0/3986383915 wait complete. 2024-05-27T07:52:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.268+0000 7f57df7cd740 1 -- start start 2024-05-27T07:52:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.268+0000 7f57df7cd740 1 Processor -- start 2024-05-27T07:52:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.269+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714aae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.269+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714aae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.269+0000 7f57df7cd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 0x55e714aa95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.269+0000 7f57df7cd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01b1800 con 0x55e7146d02f0 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.269+0000 7f57df7cd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01b2c70 con 0x55e7146f91b0 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.269+0000 7f57df7cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01b1ed0 con 0x55e7146d84d0 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.269+0000 7f57df7cd740 10 client.0 _ll_register_callbacks cb 0x7f57cc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.270+0000 7f57d5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714aae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.270+0000 7f57d5400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714aae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.270+0000 7f57d5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714aae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44816/0 (socket says 172.21.15.63:44816) 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.270+0000 7f57d5400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714aae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:33388/0 (socket says 172.21.15.63:33388) 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.270+0000 7f57d5e00640 1 -- 172.21.15.63:0/852524810 learned_addr learned my addr 172.21.15.63:0/852524810 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.270+0000 7f57d6800640 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 0x55e714aa95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.270+0000 7f57d5e00640 1 -- 172.21.15.63:0/852524810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 msgr2=0x55e714aae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.270+0000 7f57d5e00640 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714aae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.270+0000 7f57d5e00640 1 -- 172.21.15.63:0/852524810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 msgr2=0x55e714aa95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.270+0000 7f57d5e00640 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 0x55e714aa95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.271+0000 7f57d5e00640 1 -- 172.21.15.63:0/852524810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e714aa9d60 con 0x55e7146d02f0 2024-05-27T07:52:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.271+0000 7f57d5e00640 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714aae170 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f57c401b040 tx=0x7f57c4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e623e932e6b9fe4c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.271+0000 7f57b3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.271+0000 7f57b3400640 1 -- 172.21.15.63:0/852524810 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4009e50 con 0x55e7146d02f0 2024-05-27T07:52:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.271+0000 7f57b3400640 1 -- 172.21.15.63:0/852524810 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f57c4004660 con 0x55e7146d02f0 2024-05-27T07:52:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.271+0000 7f57df7cd740 1 -- 172.21.15.63:0/852524810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57cc0a9e20 con 0x55e7146d02f0 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57b3400640 1 -- 172.21.15.63:0/852524810 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f57c403c050 con 0x55e7146d02f0 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57df7cd740 10 client.19065 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57df7cd740 1 -- 172.21.15.63:0/852524810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e714b20100 con 0x55e7146d02f0 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57b3400640 1 -- 172.21.15.63:0/852524810 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57cc0a9e20 con 0x55e7146d02f0 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57b3400640 1 -- 172.21.15.63:0/852524810 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f57c402b630 con 0x55e7146d02f0 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57df7cd740 10 client.19065 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57df7cd740 10 client.19065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57df7cd740 20 client.19065 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57df7cd740 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e71468d1c0 0x55e714b22730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57df7cd740 4 client.19065 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.272+0000 7f57df7cd740 1 -- 172.21.15.63:0/852524810 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e714a082c0 con 0x55e71468d1c0 2024-05-27T07:52:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.273+0000 7f57d5400640 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e71468d1c0 0x55e714b22730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.274+0000 7f57d5400640 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e71468d1c0 0x55e714b22730 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69e0be609c9e12d server_cookie=c2e6860c4d3db8a in_seq=0 out_seq=0 2024-05-27T07:52:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.274+0000 7f57b3400640 10 client.19065 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.305+0000 7f57b3400640 1 -- 172.21.15.63:0/852524810 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e714a082c0 con 0x55e71468d1c0 2024-05-27T07:52:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.305+0000 7f57b3400640 10 client.19065 handle_command_reply: tid=0 2024-05-27T07:52:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.305+0000 7f57df7cd740 1 client.19065 shutdown 2024-05-27T07:52:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57df7cd740 1 -- 172.21.15.63:0/852524810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 msgr2=0x55e714aae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57df7cd740 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714aae170 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f57c401b040 tx=0x7f57c4005040 comp rx=0 tx=0).stop 2024-05-27T07:52:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57df7cd740 1 -- 172.21.15.63:0/852524810 shutdown_connections 2024-05-27T07:52:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57df7cd740 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e71468d1c0 0x55e714b22730 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57df7cd740 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e7146f91b0 0x55e714aa95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57b3400640 0 client.19065 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57df7cd740 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7146d84d0 0x55e714aae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57df7cd740 1 --2- 172.21.15.63:0/852524810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e7146d02f0 0x55e714aae170 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57df7cd740 1 -- 172.21.15.63:0/852524810 >> 172.21.15.63:0/852524810 conn(0x55e7146ed300 msgr2=0x55e714aa7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57df7cd740 1 -- 172.21.15.63:0/852524810 shutdown_connections 2024-05-27T07:52:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.306+0000 7f57df7cd740 1 -- 172.21.15.63:0/852524810 wait complete. 2024-05-27T07:52:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.307+0000 7f57df7cd740 20 client.19065 trim_cache size 0 max 16384 2024-05-27T07:52:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:52:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:52:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:52:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.307+0000 7f57d7200640 1 -- 172.21.15.63:0/1279205902 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57b80b2950 msgr2=0x7f57b80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.307+0000 7f57d7200640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57b80b2950 0x7f57b80b4e60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f57c0005cd0 tx=0x7f57c00404e0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.308+0000 7f57d7200640 1 -- 172.21.15.63:0/1279205902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57d010e020 msgr2=0x7f57d0102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.308+0000 7f57d7200640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57d010e020 0x7f57d0102f50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00eb10 tx=0x7f57cc00c2d0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.308+0000 7f57d7200640 1 -- 172.21.15.63:0/1279205902 shutdown_connections 2024-05-27T07:52:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.308+0000 7f57d7200640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57b80b2950 0x7f57b80b4e60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.308+0000 7f57d7200640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57d010e020 0x7f57d0102f50 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.308+0000 7f57d7200640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57d010b670 0x7f57d0102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.308+0000 7f57d7200640 1 --2- 172.21.15.63:0/1279205902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d0104c10 0x7f57d01024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.308+0000 7f57d7200640 1 -- 172.21.15.63:0/1279205902 >> 172.21.15.63:0/1279205902 conn(0x7f57d00fe210 msgr2=0x7f57d010e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.308+0000 7f57d7200640 1 -- 172.21.15.63:0/1279205902 shutdown_connections 2024-05-27T07:52:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:39.308+0000 7f57d7200640 1 -- 172.21.15.63:0/1279205902 wait complete. 2024-05-27T07:52:39.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:39 smithi086 ceph-mon[34687]: pgmap v601: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 13 op/s 2024-05-27T07:52:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:39 smithi149 ceph-mon[35408]: pgmap v601: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 13 op/s 2024-05-27T07:52:40.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.366+0000 7fa2f7000700 1 -- 172.21.15.63:0/414430015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f0105e60 msgr2=0x7fa2f010e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.366+0000 7fa2f7000700 1 --2- 172.21.15.63:0/414430015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f0105e60 0x7fa2f010e3b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009990 tx=0x7fa2dc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.366+0000 7fa2f7000700 1 -- 172.21.15.63:0/414430015 shutdown_connections 2024-05-27T07:52:40.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.366+0000 7fa2f7000700 1 --2- 172.21.15.63:0/414430015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f010e8f0 0x7fa2f0110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.366+0000 7fa2f7000700 1 --2- 172.21.15.63:0/414430015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2f0105510 0x7fa2f0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.366+0000 7fa2f7000700 1 --2- 172.21.15.63:0/414430015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f0105e60 0x7fa2f010e3b0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.366+0000 7fa2f7000700 1 -- 172.21.15.63:0/414430015 >> 172.21.15.63:0/414430015 conn(0x7fa2f00fedb0 msgr2=0x7fa2f01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.366+0000 7fa2f7000700 1 -- 172.21.15.63:0/414430015 shutdown_connections 2024-05-27T07:52:40.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.366+0000 7fa2f7000700 1 -- 172.21.15.63:0/414430015 wait complete. 2024-05-27T07:52:40.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.367+0000 7fa2f7000700 1 Processor -- start 2024-05-27T07:52:40.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.368+0000 7fa2f7000700 1 -- start start 2024-05-27T07:52:40.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.368+0000 7fa2f7000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f0105510 0x7fa2f019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.368+0000 7fa2f7000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2f0105e60 0x7fa2f019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.368+0000 7fa2f7000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f010e8f0 0x7fa2f01a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.368+0000 7fa2f7000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f01142e0 con 0x7fa2f0105510 2024-05-27T07:52:40.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.368+0000 7fa2f7000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f0114160 con 0x7fa2f0105e60 2024-05-27T07:52:40.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.368+0000 7fa2f7000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f0114460 con 0x7fa2f010e8f0 2024-05-27T07:52:40.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.368+0000 7fa2efe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f0105510 0x7fa2f019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.368+0000 7fa2efe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f0105510 0x7fa2f019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44836/0 (socket says 172.21.15.63:44836) 2024-05-27T07:52:40.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.368+0000 7fa2efe00700 1 -- 172.21.15.63:0/404796951 learned_addr learned my addr 172.21.15.63:0/404796951 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:40.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.369+0000 7fa2ef400700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2f0105e60 0x7fa2f019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.369+0000 7fa2f5000700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f010e8f0 0x7fa2f01a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.369+0000 7fa2efe00700 1 -- 172.21.15.63:0/404796951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f010e8f0 msgr2=0x7fa2f01a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.369+0000 7fa2efe00700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f010e8f0 0x7fa2f01a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.369+0000 7fa2efe00700 1 -- 172.21.15.63:0/404796951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2f0105e60 msgr2=0x7fa2f019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.369+0000 7fa2efe00700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2f0105e60 0x7fa2f019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.369+0000 7fa2efe00700 1 -- 172.21.15.63:0/404796951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f01a1cf0 con 0x7fa2f0105510 2024-05-27T07:52:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.369+0000 7fa2efe00700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f0105510 0x7fa2f019ced0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000d950 tx=0x7fa2e000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.370+0000 7fa2ecc00700 1 -- 172.21.15.63:0/404796951 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0016070 con 0x7fa2f0105510 2024-05-27T07:52:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.370+0000 7fa2ecc00700 1 -- 172.21.15.63:0/404796951 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2e0010460 con 0x7fa2f0105510 2024-05-27T07:52:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.370+0000 7fa2ecc00700 1 -- 172.21.15.63:0/404796951 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e000f620 con 0x7fa2f0105510 2024-05-27T07:52:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.370+0000 7fa2f7000700 1 -- 172.21.15.63:0/404796951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f01a1fe0 con 0x7fa2f0105510 2024-05-27T07:52:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.370+0000 7fa2f7000700 1 -- 172.21.15.63:0/404796951 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f01a24d0 con 0x7fa2f0105510 2024-05-27T07:52:40.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.374+0000 7fa2ecc00700 1 -- 172.21.15.63:0/404796951 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa2e0019070 con 0x7fa2f0105510 2024-05-27T07:52:40.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.376+0000 7fa305b58b80 1 Processor -- start 2024-05-27T07:52:40.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.376+0000 7fa305b58b80 1 -- start start 2024-05-27T07:52:40.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.376+0000 7fa2ecc00700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2d40b29f0 0x7fa2d40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.376+0000 7fa2ecc00700 1 -- 172.21.15.63:0/404796951 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fa2e00e2620 con 0x7fa2f0105510 2024-05-27T07:52:40.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa1a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa14870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa14db0 0x55c33fa151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa305b58b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c33f942e40 con 0x55c33fa1ad30 2024-05-27T07:52:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa305b58b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c33f9329e0 con 0x55c33f895e40 2024-05-27T07:52:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa305b58b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c33f936a20 con 0x55c33fa14db0 2024-05-27T07:52:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa2f5000700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2d40b29f0 0x7fa2d40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa2ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa14870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa2efe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa1a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa2ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa14870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44844/0 (socket says 172.21.15.63:44844) 2024-05-27T07:52:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.377+0000 7fa2ef400700 1 -- 172.21.15.63:0/3801050063 learned_addr learned my addr 172.21.15.63:0/3801050063 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.378+0000 7fa2f5000700 1 --2- 172.21.15.63:0/3801050063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa14db0 0x55c33fa151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.378+0000 7fa2f5000700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2d40b29f0 0x7fa2d40b4f00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80077d0 tx=0x7fa2e800e4f0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.378+0000 7fa2efe00700 1 -- 172.21.15.63:0/3801050063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa14db0 msgr2=0x55c33fa151c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.378+0000 7fa2efe00700 1 --2- 172.21.15.63:0/3801050063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa14db0 0x55c33fa151c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.378+0000 7fa2efe00700 1 -- 172.21.15.63:0/3801050063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 msgr2=0x55c33fa14870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.378+0000 7fa2efe00700 1 --2- 172.21.15.63:0/3801050063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa14870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.378+0000 7fa2efe00700 1 -- 172.21.15.63:0/3801050063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c33f8fb420 con 0x55c33f895e40 2024-05-27T07:52:40.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.379+0000 7fa2efe00700 1 --2- 172.21.15.63:0/3801050063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa1a7f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00df9b0 tx=0x7fa2e00a9050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b857737f89646ba2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.379+0000 7fa2e5600700 1 -- 172.21.15.63:0/3801050063 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0020070 con 0x55c33f895e40 2024-05-27T07:52:40.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.379+0000 7fa2e5600700 1 -- 172.21.15.63:0/3801050063 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2e00ad8f0 con 0x55c33f895e40 2024-05-27T07:52:40.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.379+0000 7fa2e5600700 1 -- 172.21.15.63:0/3801050063 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00e2380 con 0x55c33f895e40 2024-05-27T07:52:40.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.380+0000 7fa305b58b80 1 -- 172.21.15.63:0/3801050063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 msgr2=0x55c33fa1a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.380+0000 7fa305b58b80 1 --2- 172.21.15.63:0/3801050063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa1a7f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00df9b0 tx=0x7fa2e00a9050 comp rx=0 tx=0).stop 2024-05-27T07:52:40.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.381+0000 7fa305b58b80 1 -- 172.21.15.63:0/3801050063 shutdown_connections 2024-05-27T07:52:40.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.381+0000 7fa305b58b80 1 --2- 172.21.15.63:0/3801050063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa14db0 0x55c33fa151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.381+0000 7fa305b58b80 1 --2- 172.21.15.63:0/3801050063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa1a7f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.381+0000 7fa305b58b80 1 --2- 172.21.15.63:0/3801050063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa14870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.381+0000 7fa305b58b80 1 -- 172.21.15.63:0/3801050063 >> 172.21.15.63:0/3801050063 conn(0x55c33f8f5fc0 msgr2=0x55c33fa0f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.381+0000 7fa305b58b80 1 -- 172.21.15.63:0/3801050063 shutdown_connections 2024-05-27T07:52:40.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.381+0000 7fa305b58b80 1 -- 172.21.15.63:0/3801050063 wait complete. 2024-05-27T07:52:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.382+0000 7fa305b58b80 1 -- start start 2024-05-27T07:52:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.382+0000 7fa305b58b80 1 Processor -- start 2024-05-27T07:52:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.382+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33f895e40 0x55c33fa1e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.382+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33fa14db0 0x55c33fa1ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.382+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa1efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.382+0000 7fa305b58b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c33f838fa0 con 0x55c33fa1ad30 2024-05-27T07:52:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.383+0000 7fa305b58b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c33f8d3760 con 0x55c33fa14db0 2024-05-27T07:52:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.383+0000 7fa305b58b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c33fa21800 con 0x55c33f895e40 2024-05-27T07:52:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.383+0000 7fa305b58b80 10 client.0 _ll_register_callbacks cb 0x55c33f8dcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.383+0000 7fa2f5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa1efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.383+0000 7fa2f5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa1efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44856/0 (socket says 172.21.15.63:44856) 2024-05-27T07:52:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.383+0000 7fa2f5000700 1 -- 172.21.15.63:0/2718178248 learned_addr learned my addr 172.21.15.63:0/2718178248 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:40.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.383+0000 7fa2efe00700 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33f895e40 0x55c33fa1e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.383+0000 7fa2ef400700 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33fa14db0 0x55c33fa1ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.384+0000 7fa2f5000700 1 -- 172.21.15.63:0/2718178248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33f895e40 msgr2=0x55c33fa1e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.384+0000 7fa2f5000700 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33f895e40 0x55c33fa1e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.384+0000 7fa2f5000700 1 -- 172.21.15.63:0/2718178248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33fa14db0 msgr2=0x55c33fa1ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.384+0000 7fa2f5000700 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33fa14db0 0x55c33fa1ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.384+0000 7fa2f5000700 1 -- 172.21.15.63:0/2718178248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c33f8828d0 con 0x55c33fa1ad30 2024-05-27T07:52:40.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.384+0000 7fa2f5000700 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa1efb0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8017e20 tx=0x7fa2e8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd43d73f8cc519e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.384+0000 7fa2da000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:40.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.385+0000 7fa2da000700 1 -- 172.21.15.63:0/2718178248 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8025030 con 0x55c33fa1ad30 2024-05-27T07:52:40.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.385+0000 7fa2da000700 1 -- 172.21.15.63:0/2718178248 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fa2e8010030 con 0x55c33fa1ad30 2024-05-27T07:52:40.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.385+0000 7fa305b58b80 1 -- 172.21.15.63:0/2718178248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2e00ad8f0 con 0x55c33fa1ad30 2024-05-27T07:52:40.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.385+0000 7fa2da000700 1 -- 172.21.15.63:0/2718178248 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa2e802f050 con 0x55c33fa1ad30 2024-05-27T07:52:40.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.385+0000 7fa305b58b80 10 client.19077 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:40.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.385+0000 7fa305b58b80 1 -- 172.21.15.63:0/2718178248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c33f903060 con 0x55c33fa1ad30 2024-05-27T07:52:40.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.385+0000 7fa2da000700 1 -- 172.21.15.63:0/2718178248 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2e00ad8f0 con 0x55c33fa1ad30 2024-05-27T07:52:40.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.386+0000 7fa2da000700 1 -- 172.21.15.63:0/2718178248 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa2e80054b0 con 0x55c33fa1ad30 2024-05-27T07:52:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.386+0000 7fa305b58b80 10 client.19077 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.386+0000 7fa305b58b80 10 client.19077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.386+0000 7fa305b58b80 20 client.19077 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.386+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c33fa2e780 0x55c33fa30c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.386+0000 7fa305b58b80 4 client.19077 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.386+0000 7fa305b58b80 1 -- 172.21.15.63:0/2718178248 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c33f944d90 con 0x55c33fa2e780 2024-05-27T07:52:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.387+0000 7fa2efe00700 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c33fa2e780 0x55c33fa30c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.388+0000 7fa2efe00700 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c33fa2e780 0x55c33fa30c20 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89a586757de558f server_cookie=4802ef4bd14e1ec9 in_seq=0 out_seq=0 2024-05-27T07:52:40.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.388+0000 7fa2da000700 10 client.19077 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:40.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.389+0000 7fa2da000700 1 -- 172.21.15.63:0/2718178248 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c33f944d90 con 0x55c33fa2e780 2024-05-27T07:52:40.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.389+0000 7fa2da000700 10 client.19077 handle_command_reply: tid=0 2024-05-27T07:52:40.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 client.19077 shutdown 2024-05-27T07:52:40.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 -- 172.21.15.63:0/2718178248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 msgr2=0x55c33fa1efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa1efb0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8017e20 tx=0x7fa2e8004b40 comp rx=0 tx=0).stop 2024-05-27T07:52:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 -- 172.21.15.63:0/2718178248 shutdown_connections 2024-05-27T07:52:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c33fa2e780 0x55c33fa30c20 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa1efb0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33f895e40 0x55c33fa1e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2718178248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33fa14db0 0x55c33fa1ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa2da000700 0 client.19077 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:40.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 -- 172.21.15.63:0/2718178248 >> 172.21.15.63:0/2718178248 conn(0x55c33f8f5fc0 msgr2=0x55c33fa0f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 -- 172.21.15.63:0/2718178248 shutdown_connections 2024-05-27T07:52:40.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.390+0000 7fa305b58b80 1 -- 172.21.15.63:0/2718178248 wait complete. 2024-05-27T07:52:40.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.391+0000 7fa305b58b80 20 client.19077 trim_cache size 0 max 16384 2024-05-27T07:52:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.415+0000 7fa305b58b80 1 Processor -- start 2024-05-27T07:52:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.416+0000 7fa305b58b80 1 -- start start 2024-05-27T07:52:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.416+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa14870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.416+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa14db0 0x55c33fa1d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.416+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa1ad30 0x55c33fa1ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.416+0000 7fa305b58b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f01b4f60 con 0x55c33fa14db0 2024-05-27T07:52:40.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.416+0000 7fa305b58b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f01b27f0 con 0x55c33f895e40 2024-05-27T07:52:40.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.416+0000 7fa305b58b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f01b0660 con 0x55c33fa1ad30 2024-05-27T07:52:40.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.416+0000 7fa2ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa14db0 0x55c33fa1d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.416+0000 7fa2ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa14db0 0x55c33fa1d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44870/0 (socket says 172.21.15.63:44870) 2024-05-27T07:52:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.417+0000 7fa2ef400700 1 -- 172.21.15.63:0/708538614 learned_addr learned my addr 172.21.15.63:0/708538614 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.417+0000 7fa2efe00700 1 --2- 172.21.15.63:0/708538614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa14870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.417+0000 7fa2f5000700 1 --2- 172.21.15.63:0/708538614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa1ad30 0x55c33fa1ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.417+0000 7fa2ef400700 1 -- 172.21.15.63:0/708538614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa1ad30 msgr2=0x55c33fa1ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.417+0000 7fa2ef400700 1 --2- 172.21.15.63:0/708538614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa1ad30 0x55c33fa1ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.417+0000 7fa2ef400700 1 -- 172.21.15.63:0/708538614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 msgr2=0x55c33fa14870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.417+0000 7fa2ef400700 1 --2- 172.21.15.63:0/708538614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa14870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.417+0000 7fa2ef400700 1 -- 172.21.15.63:0/708538614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c33fa1c440 con 0x55c33fa14db0 2024-05-27T07:52:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.417+0000 7fa2efe00700 1 --2- 172.21.15.63:0/708538614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa14870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:40.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.418+0000 7fa2ef400700 1 --2- 172.21.15.63:0/708538614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa14db0 0x55c33fa1d490 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0059d0 tx=0x7fa2dc005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cb1aafa55e3052 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.418+0000 7fa2e4c00700 1 -- 172.21.15.63:0/708538614 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc029070 con 0x55c33fa14db0 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.418+0000 7fa2e4c00700 1 -- 172.21.15.63:0/708538614 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2dc0056a0 con 0x55c33fa14db0 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.418+0000 7fa2e4c00700 1 -- 172.21.15.63:0/708538614 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc00be00 con 0x55c33fa14db0 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.419+0000 7fa305b58b80 1 -- 172.21.15.63:0/708538614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa14db0 msgr2=0x55c33fa1d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.419+0000 7fa305b58b80 1 --2- 172.21.15.63:0/708538614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa14db0 0x55c33fa1d490 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0059d0 tx=0x7fa2dc005040 comp rx=0 tx=0).stop 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.419+0000 7fa305b58b80 1 -- 172.21.15.63:0/708538614 shutdown_connections 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.419+0000 7fa305b58b80 1 --2- 172.21.15.63:0/708538614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa1ad30 0x55c33fa1ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.419+0000 7fa305b58b80 1 --2- 172.21.15.63:0/708538614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa14870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.419+0000 7fa305b58b80 1 --2- 172.21.15.63:0/708538614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa14db0 0x55c33fa1d490 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.419+0000 7fa305b58b80 1 -- 172.21.15.63:0/708538614 >> 172.21.15.63:0/708538614 conn(0x55c33f8f5fc0 msgr2=0x55c33fa18140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.419+0000 7fa305b58b80 1 -- 172.21.15.63:0/708538614 shutdown_connections 2024-05-27T07:52:40.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.419+0000 7fa305b58b80 1 -- 172.21.15.63:0/708538614 wait complete. 2024-05-27T07:52:40.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.420+0000 7fa305b58b80 1 -- start start 2024-05-27T07:52:40.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.420+0000 7fa305b58b80 1 Processor -- start 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa26a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa14db0 0x55c33fa26f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa305b58b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa23a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa305b58b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f01b0cb0 con 0x55c33fa1ad30 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa305b58b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f01b2120 con 0x55c33f895e40 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa305b58b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f01b1380 con 0x55c33fa14db0 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa305b58b80 10 client.0 _ll_register_callbacks cb 0x55c33f8dcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa2f5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa23a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa2f5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa23a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44872/0 (socket says 172.21.15.63:44872) 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa2f5000700 1 -- 172.21.15.63:0/2497607366 learned_addr learned my addr 172.21.15.63:0/2497607366 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa2ef400700 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa14db0 0x55c33fa26f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.421+0000 7fa2efe00700 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa26a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.422+0000 7fa2f5000700 1 -- 172.21.15.63:0/2497607366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa14db0 msgr2=0x55c33fa26f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.422+0000 7fa2f5000700 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa14db0 0x55c33fa26f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.422+0000 7fa2f5000700 1 -- 172.21.15.63:0/2497607366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 msgr2=0x55c33fa26a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.422+0000 7fa2f5000700 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa26a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.422+0000 7fa2f5000700 1 -- 172.21.15.63:0/2497607366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c33fa27580 con 0x55c33fa1ad30 2024-05-27T07:52:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.422+0000 7fa2f5000700 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa23a20 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800e8f0 tx=0x7fa2e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dd579ef3d312b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.422+0000 7fa2da000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.423+0000 7fa2da000700 1 -- 172.21.15.63:0/2497607366 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8025040 con 0x55c33fa1ad30 2024-05-27T07:52:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.423+0000 7fa2da000700 1 -- 172.21.15.63:0/2497607366 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fa2e8016bd0 con 0x55c33fa1ad30 2024-05-27T07:52:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.423+0000 7fa305b58b80 1 -- 172.21.15.63:0/2497607366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2dc0056a0 con 0x55c33fa1ad30 2024-05-27T07:52:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.423+0000 7fa2da000700 1 -- 172.21.15.63:0/2497607366 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa2e8041050 con 0x55c33fa1ad30 2024-05-27T07:52:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.423+0000 7fa305b58b80 10 client.19083 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.423+0000 7fa305b58b80 1 -- 172.21.15.63:0/2497607366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c33fa997b0 con 0x55c33fa1ad30 2024-05-27T07:52:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.423+0000 7fa2da000700 1 -- 172.21.15.63:0/2497607366 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2dc0056a0 con 0x55c33fa1ad30 2024-05-27T07:52:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.424+0000 7fa2da000700 1 -- 172.21.15.63:0/2497607366 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa2e8021520 con 0x55c33fa1ad30 2024-05-27T07:52:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.424+0000 7fa305b58b80 10 client.19083 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.424+0000 7fa305b58b80 10 client.19083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.424+0000 7fa305b58b80 20 client.19083 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.424+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c33fa99ed0 0x55c33fa9c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.424+0000 7fa305b58b80 4 client.19083 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:52:40.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.424+0000 7fa305b58b80 1 -- 172.21.15.63:0/2497607366 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c33f93e570 con 0x55c33fa99ed0 2024-05-27T07:52:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.425+0000 7fa2efe00700 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c33fa99ed0 0x55c33fa9c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.426+0000 7fa2da000700 10 client.19083 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.426+0000 7fa2efe00700 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c33fa99ed0 0x55c33fa9c2f0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6851c135fdbe724 server_cookie=9a14f39742b759e3 in_seq=0 out_seq=0 2024-05-27T07:52:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.426+0000 7fa2da000700 1 -- 172.21.15.63:0/2497607366 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c33f93e570 con 0x55c33fa99ed0 2024-05-27T07:52:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.426+0000 7fa2da000700 10 client.19083 handle_command_reply: tid=0 2024-05-27T07:52:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.426+0000 7fa305b58b80 1 client.19083 shutdown 2024-05-27T07:52:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.426+0000 7fa305b58b80 1 -- 172.21.15.63:0/2497607366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 msgr2=0x55c33fa23a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.426+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa23a20 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800e8f0 tx=0x7fa2e8019000 comp rx=0 tx=0).stop 2024-05-27T07:52:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.426+0000 7fa305b58b80 1 -- 172.21.15.63:0/2497607366 shutdown_connections 2024-05-27T07:52:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.426+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c33fa99ed0 0x55c33fa9c2f0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.426+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c33fa1ad30 0x55c33fa23a20 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.427+0000 7fa2da000700 0 client.19083 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.427+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c33f895e40 0x55c33fa26a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.427+0000 7fa305b58b80 1 --2- 172.21.15.63:0/2497607366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c33fa14db0 0x55c33fa26f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.427+0000 7fa305b58b80 1 -- 172.21.15.63:0/2497607366 >> 172.21.15.63:0/2497607366 conn(0x55c33f8f5fc0 msgr2=0x55c33fa1ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.427+0000 7fa305b58b80 1 -- 172.21.15.63:0/2497607366 shutdown_connections 2024-05-27T07:52:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.427+0000 7fa305b58b80 1 -- 172.21.15.63:0/2497607366 wait complete. 2024-05-27T07:52:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.427+0000 7fa305b58b80 20 client.19083 trim_cache size 0 max 16384 2024-05-27T07:52:40.429 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:52:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:52:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:52:40.429 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 -- 172.21.15.63:0/404796951 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2d40b29f0 msgr2=0x7fa2d40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2d40b29f0 0x7fa2d40b4f00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80077d0 tx=0x7fa2e800e4f0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 -- 172.21.15.63:0/404796951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f0105510 msgr2=0x7fa2f019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f0105510 0x7fa2f019ced0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000d950 tx=0x7fa2e000dd10 comp rx=0 tx=0).stop 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 -- 172.21.15.63:0/404796951 shutdown_connections 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2d40b29f0 0x7fa2d40b4f00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f010e8f0 0x7fa2f01a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f0105510 0x7fa2f019ced0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 --2- 172.21.15.63:0/404796951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2f0105e60 0x7fa2f019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 -- 172.21.15.63:0/404796951 >> 172.21.15.63:0/404796951 conn(0x7fa2f00fedb0 msgr2=0x7fa2f00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.428+0000 7fa2f7000700 1 -- 172.21.15.63:0/404796951 shutdown_connections 2024-05-27T07:52:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:40.429+0000 7fa2f7000700 1 -- 172.21.15.63:0/404796951 wait complete. 2024-05-27T07:52:41.395 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:41 smithi063 ceph-mon[31613]: pgmap v602: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T07:52:41.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:41 smithi086 ceph-mon[34687]: pgmap v602: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T07:52:41.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:41 smithi149 ceph-mon[35408]: pgmap v602: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T07:52:41.471 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:52:41.472 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:52:41.950 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:43.438 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:43 smithi063 ceph-mon[31613]: pgmap v603: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:52:43.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:43 smithi086 ceph-mon[34687]: pgmap v603: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:52:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:43 smithi149 ceph-mon[35408]: pgmap v603: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:52:44.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.525+0000 7f492de00700 1 -- 172.21.15.63:0/2698489594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4928109670 msgr2=0x7f4928110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:44.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.525+0000 7f492de00700 1 --2- 172.21.15.63:0/2698489594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4928109670 0x7f4928110fd0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f492000b0d0 tx=0x7f492000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:52:44.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.525+0000 7f492de00700 1 -- 172.21.15.63:0/2698489594 shutdown_connections 2024-05-27T07:52:44.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.525+0000 7f492de00700 1 --2- 172.21.15.63:0/2698489594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4928109670 0x7f4928110fd0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:44.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.525+0000 7f492de00700 1 --2- 172.21.15.63:0/2698489594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49281043f0 0x7f4928104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:44.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.525+0000 7f492de00700 1 --2- 172.21.15.63:0/2698489594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4928108d20 0x7f4928109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:44.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.525+0000 7f492de00700 1 -- 172.21.15.63:0/2698489594 >> 172.21.15.63:0/2698489594 conn(0x7f4928079d00 msgr2=0x7f492807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:44.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.525+0000 7f492de00700 1 -- 172.21.15.63:0/2698489594 shutdown_connections 2024-05-27T07:52:44.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.525+0000 7f492de00700 1 -- 172.21.15.63:0/2698489594 wait complete. 2024-05-27T07:52:44.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.527+0000 7f492de00700 1 Processor -- start 2024-05-27T07:52:44.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.527+0000 7f492de00700 1 -- start start 2024-05-27T07:52:44.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.527+0000 7f492de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49281043f0 0x7f4928075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:44.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.527+0000 7f492de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928108d20 0x7f4928075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:44.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.527+0000 7f492de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4928109670 0x7f4928076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:44.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.527+0000 7f492de00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4928114150 con 0x7f49281043f0 2024-05-27T07:52:44.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.527+0000 7f492de00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4928113fd0 con 0x7f4928108d20 2024-05-27T07:52:44.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.527+0000 7f492de00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f49281142d0 con 0x7f4928109670 2024-05-27T07:52:44.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f492ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49281043f0 0x7f4928075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:44.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f492ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49281043f0 0x7f4928075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44898/0 (socket says 172.21.15.63:44898) 2024-05-27T07:52:44.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f492ca00700 1 -- 172.21.15.63:0/1771839694 learned_addr learned my addr 172.21.15.63:0/1771839694 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:44.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f4927e00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928108d20 0x7f4928075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:44.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f492d400700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4928109670 0x7f4928076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:44.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f492ca00700 1 -- 172.21.15.63:0/1771839694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4928109670 msgr2=0x7f4928076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:44.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f492ca00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4928109670 0x7f4928076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:44.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f492ca00700 1 -- 172.21.15.63:0/1771839694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928108d20 msgr2=0x7f4928075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:44.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f492ca00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928108d20 0x7f4928075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:44.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f492ca00700 1 -- 172.21.15.63:0/1771839694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49281a8e80 con 0x7f49281043f0 2024-05-27T07:52:44.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.528+0000 7f4927e00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928108d20 0x7f4928075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:44.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.529+0000 7f492ca00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49281043f0 0x7f4928075ab0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f491c00d950 tx=0x7f491c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:44.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.529+0000 7f4925600700 1 -- 172.21.15.63:0/1771839694 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c016070 con 0x7f49281043f0 2024-05-27T07:52:44.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.529+0000 7f4925600700 1 -- 172.21.15.63:0/1771839694 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f491c010460 con 0x7f49281043f0 2024-05-27T07:52:44.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.529+0000 7f492de00700 1 -- 172.21.15.63:0/1771839694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49281a9170 con 0x7f49281043f0 2024-05-27T07:52:44.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.529+0000 7f4925600700 1 -- 172.21.15.63:0/1771839694 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c00f620 con 0x7f49281043f0 2024-05-27T07:52:44.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.530+0000 7f492de00700 1 -- 172.21.15.63:0/1771839694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49281a9700 con 0x7f49281043f0 2024-05-27T07:52:44.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.531+0000 7f492de00700 1 -- 172.21.15.63:0/1771839694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49281058a0 con 0x7f49281043f0 2024-05-27T07:52:44.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.536+0000 7f4925600700 1 -- 172.21.15.63:0/1771839694 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f491c019070 con 0x7f49281043f0 2024-05-27T07:52:44.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.537+0000 7f4925600700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49100b2a70 0x7f49100b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:44.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.538+0000 7f4925600700 1 -- 172.21.15.63:0/1771839694 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f491c0e2590 con 0x7f49281043f0 2024-05-27T07:52:44.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.538+0000 7f4927e00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49100b2a70 0x7f49100b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:44.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.539+0000 7f4927e00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49100b2a70 0x7f49100b4f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4914005e10 tx=0x7f4914005d60 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:44.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.541+0000 7f4925600700 1 -- 172.21.15.63:0/1771839694 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f491c0a9bb0 con 0x7f49281043f0 2024-05-27T07:52:44.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.845+0000 7f492de00700 1 -- 172.21.15.63:0/1771839694 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4928076fe0 con 0x7f49281043f0 2024-05-27T07:52:44.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.847+0000 7f4925600700 1 -- 172.21.15.63:0/1771839694 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f491c0a94b0 con 0x7f49281043f0 2024-05-27T07:52:44.847 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:52:44.847 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:52:44.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.850+0000 7f492de00700 1 -- 172.21.15.63:0/1771839694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49100b2a70 msgr2=0x7f49100b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:44.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.850+0000 7f492de00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49100b2a70 0x7f49100b4f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4914005e10 tx=0x7f4914005d60 comp rx=0 tx=0).stop 2024-05-27T07:52:44.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.850+0000 7f492de00700 1 -- 172.21.15.63:0/1771839694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49281043f0 msgr2=0x7f4928075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:44.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.850+0000 7f492de00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49281043f0 0x7f4928075ab0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f491c00d950 tx=0x7f491c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:52:44.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.850+0000 7f492de00700 1 -- 172.21.15.63:0/1771839694 shutdown_connections 2024-05-27T07:52:44.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.850+0000 7f492de00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f49100b2a70 0x7f49100b4f80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:44.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.850+0000 7f492de00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4928109670 0x7f4928076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:44.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.850+0000 7f492de00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49281043f0 0x7f4928075ab0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:44.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.850+0000 7f492de00700 1 --2- 172.21.15.63:0/1771839694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4928108d20 0x7f4928075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:44.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.850+0000 7f492de00700 1 -- 172.21.15.63:0/1771839694 >> 172.21.15.63:0/1771839694 conn(0x7f4928079d00 msgr2=0x7f4928101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:44.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.851+0000 7f492de00700 1 -- 172.21.15.63:0/1771839694 shutdown_connections 2024-05-27T07:52:44.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:44.851+0000 7f492de00700 1 -- 172.21.15.63:0/1771839694 wait complete. 2024-05-27T07:52:44.852 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:52:45.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:45 smithi063 ceph-mon[31613]: pgmap v604: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:52:45.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:45 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1771839694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:52:45.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:45 smithi086 ceph-mon[34687]: pgmap v604: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:52:45.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:45 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1771839694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:52:45.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:45 smithi149 ceph-mon[35408]: pgmap v604: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:52:45.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:45 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1771839694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:52:45.816 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:52:46.321 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:47.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:47 smithi063 ceph-mon[31613]: pgmap v605: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T07:52:47.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:47 smithi086 ceph-mon[34687]: pgmap v605: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T07:52:47.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:47 smithi149 ceph-mon[35408]: pgmap v605: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T07:52:48.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.858+0000 7fd360a00700 1 -- 172.21.15.63:0/324042375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd35c108d20 msgr2=0x7fd35c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.858+0000 7fd360a00700 1 --2- 172.21.15.63:0/324042375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd35c108d20 0x7fd35c109130 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd348009990 tx=0x7fd348009ca0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.859+0000 7fd360a00700 1 -- 172.21.15.63:0/324042375 shutdown_connections 2024-05-27T07:52:48.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.859+0000 7fd360a00700 1 --2- 172.21.15.63:0/324042375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd35c109670 0x7fd35c110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.859+0000 7fd360a00700 1 --2- 172.21.15.63:0/324042375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c1043f0 0x7fd35c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.859+0000 7fd360a00700 1 --2- 172.21.15.63:0/324042375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd35c108d20 0x7fd35c109130 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.859+0000 7fd360a00700 1 -- 172.21.15.63:0/324042375 >> 172.21.15.63:0/324042375 conn(0x7fd35c079d00 msgr2=0x7fd35c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.859+0000 7fd360a00700 1 -- 172.21.15.63:0/324042375 shutdown_connections 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.859+0000 7fd360a00700 1 -- 172.21.15.63:0/324042375 wait complete. 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.860+0000 7fd360a00700 1 Processor -- start 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.860+0000 7fd360a00700 1 -- start start 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.861+0000 7fd360a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd35c1043f0 0x7fd35c075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.861+0000 7fd360a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c108d20 0x7fd35c075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.861+0000 7fd360a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd35c109670 0x7fd35c076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.861+0000 7fd360a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c114130 con 0x7fd35c1043f0 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.861+0000 7fd360a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c113fb0 con 0x7fd35c109670 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.861+0000 7fd360a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1142b0 con 0x7fd35c108d20 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.862+0000 7fd35a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd35c1043f0 0x7fd35c075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.862+0000 7fd35a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd35c1043f0 0x7fd35c075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44914/0 (socket says 172.21.15.63:44914) 2024-05-27T07:52:48.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.862+0000 7fd35a800700 1 -- 172.21.15.63:0/1018952752 learned_addr learned my addr 172.21.15.63:0/1018952752 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:48.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.862+0000 7fd35b200700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd35c109670 0x7fd35c076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.862+0000 7fd359e00700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c108d20 0x7fd35c075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.862+0000 7fd35a800700 1 -- 172.21.15.63:0/1018952752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c108d20 msgr2=0x7fd35c075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.862+0000 7fd35a800700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c108d20 0x7fd35c075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.862+0000 7fd35a800700 1 -- 172.21.15.63:0/1018952752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd35c109670 msgr2=0x7fd35c076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.862+0000 7fd35a800700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd35c109670 0x7fd35c076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.862+0000 7fd35a800700 1 -- 172.21.15.63:0/1018952752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c1a8f40 con 0x7fd35c1043f0 2024-05-27T07:52:48.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.863+0000 7fd35a800700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd35c1043f0 0x7fd35c075ab0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00eb30 tx=0x7fd34c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.863+0000 7fd353400700 1 -- 172.21.15.63:0/1018952752 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00cc80 con 0x7fd35c1043f0 2024-05-27T07:52:48.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.863+0000 7fd353400700 1 -- 172.21.15.63:0/1018952752 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd34c00ce20 con 0x7fd35c1043f0 2024-05-27T07:52:48.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.863+0000 7fd360a00700 1 -- 172.21.15.63:0/1018952752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c1a9230 con 0x7fd35c1043f0 2024-05-27T07:52:48.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.864+0000 7fd360a00700 1 -- 172.21.15.63:0/1018952752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c1a97c0 con 0x7fd35c1043f0 2024-05-27T07:52:48.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.865+0000 7fd353400700 1 -- 172.21.15.63:0/1018952752 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c003e10 con 0x7fd35c1043f0 2024-05-27T07:52:48.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.869+0000 7fd353400700 1 -- 172.21.15.63:0/1018952752 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd34c021430 con 0x7fd35c1043f0 2024-05-27T07:52:48.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.869+0000 7fd36fea3b80 1 Processor -- start 2024-05-27T07:52:48.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.869+0000 7fd36fea3b80 1 -- start start 2024-05-27T07:52:48.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.870+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 0x556b8c933750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.870+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 0x556b8c933c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.870+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.870+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556b8c860e40 con 0x556b8c93d2c0 2024-05-27T07:52:48.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.870+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b8c8509e0 con 0x556b8c7b3e40 2024-05-27T07:52:48.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.870+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556b8c854a20 con 0x556b8c89f9f0 2024-05-27T07:52:48.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.870+0000 7fd359e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd359e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44924/0 (socket says 172.21.15.63:44924) 2024-05-27T07:52:48.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd359e00700 1 -- 172.21.15.63:0/3015907344 learned_addr learned my addr 172.21.15.63:0/3015907344 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:48.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd35b200700 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 0x556b8c933750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd35a800700 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 0x556b8c933c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd353400700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3400b2a70 0x7fd3400b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd359e00700 1 -- 172.21.15.63:0/3015907344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 msgr2=0x556b8c933750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd359e00700 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 0x556b8c933750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd359e00700 1 -- 172.21.15.63:0/3015907344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 msgr2=0x556b8c933c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd359e00700 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 0x556b8c933c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd359e00700 1 -- 172.21.15.63:0/3015907344 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8c819420 con 0x556b8c93d2c0 2024-05-27T07:52:48.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd35a800700 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 0x556b8c933c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:48.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.871+0000 7fd35b200700 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 0x556b8c933750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:48.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.872+0000 7fd353400700 1 -- 172.21.15.63:0/1018952752 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fd34c0e2c70 con 0x7fd35c1043f0 2024-05-27T07:52:48.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.872+0000 7fd35b200700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3400b2a70 0x7fd3400b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.872+0000 7fd359e00700 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93f6e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd35c06c3b0 tx=0x7fd3480096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697e13cf82969cef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.872+0000 7fd347e00700 1 -- 172.21.15.63:0/3015907344 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3480054e0 con 0x556b8c93d2c0 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.872+0000 7fd347e00700 1 -- 172.21.15.63:0/3015907344 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd348005680 con 0x556b8c93d2c0 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.872+0000 7fd347e00700 1 -- 172.21.15.63:0/3015907344 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34800bbb0 con 0x556b8c93d2c0 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.873+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3015907344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 msgr2=0x556b8c93f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.873+0000 7fd35b200700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3400b2a70 0x7fd3400b4f80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd354004a60 tx=0x7fd354009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.873+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93f6e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd35c06c3b0 tx=0x7fd3480096a0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.874+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3015907344 shutdown_connections 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.874+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93f6e0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.874+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 0x556b8c933750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.874+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3015907344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 0x556b8c933c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.874+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3015907344 >> 172.21.15.63:0/3015907344 conn(0x556b8c84d390 msgr2=0x556b8c92e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.874+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3015907344 shutdown_connections 2024-05-27T07:52:48.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.874+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3015907344 wait complete. 2024-05-27T07:52:48.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.875+0000 7fd36fea3b80 1 -- start start 2024-05-27T07:52:48.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.875+0000 7fd36fea3b80 1 Processor -- start 2024-05-27T07:52:48.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 0x556b8c93a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c89f9f0 0x556b8c93a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556b8c756fa0 con 0x556b8c93d2c0 2024-05-27T07:52:48.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b8c7f1760 con 0x556b8c89f9f0 2024-05-27T07:52:48.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556b8c93fda0 con 0x556b8c7b3e40 2024-05-27T07:52:48.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd36fea3b80 10 client.0 _ll_register_callbacks cb 0x556b8c7fac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:48.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd35b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd35a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 0x556b8c93a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd359e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c89f9f0 0x556b8c93a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd35b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44934/0 (socket says 172.21.15.63:44934) 2024-05-27T07:52:48.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd35a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 0x556b8c93a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:33498/0 (socket says 172.21.15.63:33498) 2024-05-27T07:52:48.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.876+0000 7fd35b200700 1 -- 172.21.15.63:0/3492410428 learned_addr learned my addr 172.21.15.63:0/3492410428 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:48.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.877+0000 7fd35b200700 1 -- 172.21.15.63:0/3492410428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 msgr2=0x556b8c93a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.877+0000 7fd35b200700 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 0x556b8c93a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.877+0000 7fd35b200700 1 -- 172.21.15.63:0/3492410428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c89f9f0 msgr2=0x556b8c93a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.877+0000 7fd35b200700 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c89f9f0 0x556b8c93a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.877+0000 7fd35b200700 1 -- 172.21.15.63:0/3492410428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8c7a08d0 con 0x556b8c93d2c0 2024-05-27T07:52:48.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.877+0000 7fd35b200700 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93abd0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd354017ee0 tx=0x7fd354004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43feba8f011a66ae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.877+0000 7fd344c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:48.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.878+0000 7fd344c00700 1 -- 172.21.15.63:0/3492410428 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354004bb0 con 0x556b8c93d2c0 2024-05-27T07:52:48.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.878+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3492410428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd348005680 con 0x556b8c93d2c0 2024-05-27T07:52:48.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.878+0000 7fd344c00700 1 -- 172.21.15.63:0/3492410428 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fd354004d50 con 0x556b8c93d2c0 2024-05-27T07:52:48.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.878+0000 7fd344c00700 1 -- 172.21.15.63:0/3492410428 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd354016b40 con 0x556b8c93d2c0 2024-05-27T07:52:48.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.879+0000 7fd36fea3b80 10 client.19101 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:48.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.879+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3492410428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556b8c821060 con 0x556b8c93d2c0 2024-05-27T07:52:48.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.879+0000 7fd344c00700 1 -- 172.21.15.63:0/3492410428 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd348005680 con 0x556b8c93d2c0 2024-05-27T07:52:48.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.879+0000 7fd344c00700 1 -- 172.21.15.63:0/3492410428 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd3540213a0 con 0x556b8c93d2c0 2024-05-27T07:52:48.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.880+0000 7fd36fea3b80 10 client.19101 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:48.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.880+0000 7fd36fea3b80 10 client.19101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:52:48.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.880+0000 7fd36fea3b80 20 client.19101 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:48.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.880+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556b8c94cc10 0x556b8c94f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.880+0000 7fd36fea3b80 4 client.19101 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:48.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.880+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3492410428 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b8c862d90 con 0x556b8c94cc10 2024-05-27T07:52:48.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.880+0000 7fd35a800700 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556b8c94cc10 0x556b8c94f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.881+0000 7fd35a800700 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556b8c94cc10 0x556b8c94f030 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c57dc1a1f07339e5 server_cookie=a5d9919446658817 in_seq=0 out_seq=0 2024-05-27T07:52:48.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.881+0000 7fd344c00700 10 client.19101 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:52:48.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.883+0000 7fd344c00700 1 -- 172.21.15.63:0/3492410428 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556b8c862d90 con 0x556b8c94cc10 2024-05-27T07:52:48.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.883+0000 7fd344c00700 10 client.19101 handle_command_reply: tid=0 2024-05-27T07:52:48.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.883+0000 7fd36fea3b80 1 client.19101 shutdown 2024-05-27T07:52:48.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3492410428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 msgr2=0x556b8c93abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93abd0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd354017ee0 tx=0x7fd354004ad0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3492410428 shutdown_connections 2024-05-27T07:52:48.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556b8c94cc10 0x556b8c94f030 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c93abd0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 0x556b8c93a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3492410428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c89f9f0 0x556b8c93a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3492410428 >> 172.21.15.63:0/3492410428 conn(0x556b8c84d390 msgr2=0x556b8c92e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd344c00700 0 client.19101 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:52:48.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3492410428 shutdown_connections 2024-05-27T07:52:48.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3492410428 wait complete. 2024-05-27T07:52:48.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.884+0000 7fd36fea3b80 20 client.19101 trim_cache size 0 max 16384 2024-05-27T07:52:48.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.909+0000 7fd36fea3b80 1 Processor -- start 2024-05-27T07:52:48.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.909+0000 7fd36fea3b80 1 -- start start 2024-05-27T07:52:48.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.909+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 0x556b8c9390b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.909+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 0x556b8c935950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.909+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c92e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.909+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1b4ed0 con 0x556b8c93d2c0 2024-05-27T07:52:48.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.909+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1b2760 con 0x556b8c7b3e40 2024-05-27T07:52:48.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.909+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1b0550 con 0x556b8c89f9f0 2024-05-27T07:52:48.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.910+0000 7fd35b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c92e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.910+0000 7fd35b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c92e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36256/0 (socket says 172.21.15.63:36256) 2024-05-27T07:52:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.910+0000 7fd35b200700 1 -- 172.21.15.63:0/3000027436 learned_addr learned my addr 172.21.15.63:0/3000027436 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.910+0000 7fd35a800700 1 --2- 172.21.15.63:0/3000027436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 0x556b8c9390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.910+0000 7fd359e00700 1 --2- 172.21.15.63:0/3000027436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 0x556b8c935950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.910+0000 7fd35b200700 1 -- 172.21.15.63:0/3000027436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 msgr2=0x556b8c935950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.910+0000 7fd35b200700 1 --2- 172.21.15.63:0/3000027436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 0x556b8c935950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.910+0000 7fd35b200700 1 -- 172.21.15.63:0/3000027436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 msgr2=0x556b8c9390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.910+0000 7fd35b200700 1 --2- 172.21.15.63:0/3000027436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 0x556b8c9390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.910+0000 7fd35b200700 1 -- 172.21.15.63:0/3000027436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8c9386c0 con 0x556b8c93d2c0 2024-05-27T07:52:48.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.911+0000 7fd35b200700 1 --2- 172.21.15.63:0/3000027436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c92e740 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fd354021790 tx=0x7fd3540215b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fa53b71b3375d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.911+0000 7fd347400700 1 -- 172.21.15.63:0/3000027436 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354016b40 con 0x556b8c93d2c0 2024-05-27T07:52:48.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.911+0000 7fd347400700 1 -- 172.21.15.63:0/3000027436 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd35401b030 con 0x556b8c93d2c0 2024-05-27T07:52:48.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.911+0000 7fd347400700 1 -- 172.21.15.63:0/3000027436 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354022630 con 0x556b8c93d2c0 2024-05-27T07:52:48.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.912+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3000027436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 msgr2=0x556b8c92e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.912+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3000027436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c92e740 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fd354021790 tx=0x7fd3540215b0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.912+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3000027436 shutdown_connections 2024-05-27T07:52:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.912+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3000027436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c93d2c0 0x556b8c92e740 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.912+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3000027436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c7b3e40 0x556b8c9390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.912+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/3000027436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c89f9f0 0x556b8c935950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.912+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3000027436 >> 172.21.15.63:0/3000027436 conn(0x556b8c84d390 msgr2=0x556b8c935240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.912+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3000027436 shutdown_connections 2024-05-27T07:52:48.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.912+0000 7fd36fea3b80 1 -- 172.21.15.63:0/3000027436 wait complete. 2024-05-27T07:52:48.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.913+0000 7fd36fea3b80 1 -- start start 2024-05-27T07:52:48.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.913+0000 7fd36fea3b80 1 Processor -- start 2024-05-27T07:52:48.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 0x556b8c940680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c89f9f0 0x556b8c940bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd36fea3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c93d2c0 0x556b8c9b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1b0c20 con 0x556b8c89f9f0 2024-05-27T07:52:48.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1b2090 con 0x556b8c93d2c0 2024-05-27T07:52:48.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd36fea3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1b12f0 con 0x556b8c7b3e40 2024-05-27T07:52:48.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd36fea3b80 10 client.0 _ll_register_callbacks cb 0x556b8c7fac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:48.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd359e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c89f9f0 0x556b8c940bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd359e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c89f9f0 0x556b8c940bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36264/0 (socket says 172.21.15.63:36264) 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd359e00700 1 -- 172.21.15.63:0/2374909603 learned_addr learned my addr 172.21.15.63:0/2374909603 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.914+0000 7fd35a800700 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 0x556b8c940680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.915+0000 7fd35b200700 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c93d2c0 0x556b8c9b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.915+0000 7fd359e00700 1 -- 172.21.15.63:0/2374909603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 msgr2=0x556b8c940680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.915+0000 7fd359e00700 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 0x556b8c940680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.915+0000 7fd359e00700 1 -- 172.21.15.63:0/2374909603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c93d2c0 msgr2=0x556b8c9b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.915+0000 7fd359e00700 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c93d2c0 0x556b8c9b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.915+0000 7fd359e00700 1 -- 172.21.15.63:0/2374909603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8c9b5ba0 con 0x556b8c89f9f0 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.915+0000 7fd359e00700 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c89f9f0 0x556b8c940bc0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fd35c06c3b0 tx=0x7fd348004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2157b05b8506d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.915+0000 7fd344c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:48.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.916+0000 7fd344c00700 1 -- 172.21.15.63:0/2374909603 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348004ca0 con 0x556b8c89f9f0 2024-05-27T07:52:48.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.916+0000 7fd36fea3b80 1 -- 172.21.15.63:0/2374909603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35401b030 con 0x556b8c89f9f0 2024-05-27T07:52:48.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.916+0000 7fd344c00700 1 -- 172.21.15.63:0/2374909603 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fd34801e040 con 0x556b8c89f9f0 2024-05-27T07:52:48.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.916+0000 7fd344c00700 1 -- 172.21.15.63:0/2374909603 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd34803b050 con 0x556b8c89f9f0 2024-05-27T07:52:48.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.916+0000 7fd36fea3b80 10 client.19107 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:48.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.916+0000 7fd36fea3b80 1 -- 172.21.15.63:0/2374909603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556b8c9b6dd0 con 0x556b8c89f9f0 2024-05-27T07:52:48.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.917+0000 7fd344c00700 1 -- 172.21.15.63:0/2374909603 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd35401b030 con 0x556b8c89f9f0 2024-05-27T07:52:48.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.917+0000 7fd344c00700 1 -- 172.21.15.63:0/2374909603 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd34802d6b0 con 0x556b8c89f9f0 2024-05-27T07:52:48.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.917+0000 7fd36fea3b80 10 client.19107 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:48.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.917+0000 7fd36fea3b80 10 client.19107 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:52:48.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.917+0000 7fd36fea3b80 20 client.19107 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:48.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.917+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556b8c9b6fa0 0x556b8c9b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.917+0000 7fd36fea3b80 4 client.19107 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:52:48.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.917+0000 7fd36fea3b80 1 -- 172.21.15.63:0/2374909603 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b8c85c570 con 0x556b8c9b6fa0 2024-05-27T07:52:48.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.918+0000 7fd35a800700 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556b8c9b6fa0 0x556b8c9b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.919+0000 7fd35a800700 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556b8c9b6fa0 0x556b8c9b93c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14bd2456c8455d13 server_cookie=56039795b39cf230 in_seq=0 out_seq=0 2024-05-27T07:52:48.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.919+0000 7fd344c00700 10 client.19107 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:52:48.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.919+0000 7fd344c00700 1 -- 172.21.15.63:0/2374909603 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b8c85c570 con 0x556b8c9b6fa0 2024-05-27T07:52:48.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.919+0000 7fd344c00700 10 client.19107 handle_command_reply: tid=0 2024-05-27T07:52:48.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.919+0000 7fd36fea3b80 1 client.19107 shutdown 2024-05-27T07:52:48.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd36fea3b80 1 -- 172.21.15.63:0/2374909603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c89f9f0 msgr2=0x556b8c940bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c89f9f0 0x556b8c940bc0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fd35c06c3b0 tx=0x7fd348004870 comp rx=0 tx=0).stop 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd36fea3b80 1 -- 172.21.15.63:0/2374909603 shutdown_connections 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b8c93d2c0 0x556b8c9b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b8c7b3e40 0x556b8c940680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556b8c9b6fa0 0x556b8c9b93c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd36fea3b80 1 --2- 172.21.15.63:0/2374909603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b8c89f9f0 0x556b8c940bc0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd36fea3b80 1 -- 172.21.15.63:0/2374909603 >> 172.21.15.63:0/2374909603 conn(0x556b8c84d390 msgr2=0x556b8c932120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd344c00700 0 client.19107 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd36fea3b80 1 -- 172.21.15.63:0/2374909603 shutdown_connections 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.920+0000 7fd36fea3b80 1 -- 172.21.15.63:0/2374909603 wait complete. 2024-05-27T07:52:48.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.921+0000 7fd36fea3b80 20 client.19107 trim_cache size 0 max 16384 2024-05-27T07:52:48.923 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:52:48.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.921+0000 7fd360a00700 1 -- 172.21.15.63:0/1018952752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3400b2a70 msgr2=0x7fd3400b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.921+0000 7fd360a00700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3400b2a70 0x7fd3400b4f80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd354004a60 tx=0x7fd354009040 comp rx=0 tx=0).stop 2024-05-27T07:52:48.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.921+0000 7fd360a00700 1 -- 172.21.15.63:0/1018952752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd35c1043f0 msgr2=0x7fd35c075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.921+0000 7fd360a00700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd35c1043f0 0x7fd35c075ab0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00eb30 tx=0x7fd34c00eef0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.922+0000 7fd360a00700 1 -- 172.21.15.63:0/1018952752 shutdown_connections 2024-05-27T07:52:48.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.922+0000 7fd360a00700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3400b2a70 0x7fd3400b4f80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.922+0000 7fd360a00700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd35c109670 0x7fd35c076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.922+0000 7fd360a00700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd35c1043f0 0x7fd35c075ab0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.922+0000 7fd360a00700 1 --2- 172.21.15.63:0/1018952752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c108d20 0x7fd35c075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.922+0000 7fd360a00700 1 -- 172.21.15.63:0/1018952752 >> 172.21.15.63:0/1018952752 conn(0x7fd35c079d00 msgr2=0x7fd35c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.922+0000 7fd360a00700 1 -- 172.21.15.63:0/1018952752 shutdown_connections 2024-05-27T07:52:48.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:48.922+0000 7fd360a00700 1 -- 172.21.15.63:0/1018952752 wait complete. 2024-05-27T07:52:49.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:49 smithi063 ceph-mon[31613]: pgmap v606: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:52:49.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:49 smithi086 ceph-mon[34687]: pgmap v606: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:52:49.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:49 smithi149 ceph-mon[35408]: pgmap v606: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:52:49.947 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:52:50.438 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:51.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:51 smithi063 ceph-mon[31613]: pgmap v607: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:52:51.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:51 smithi086 ceph-mon[34687]: pgmap v607: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:52:51.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:51 smithi149 ceph-mon[35408]: pgmap v607: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T07:52:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.575+0000 7fcbe5400640 1 -- 172.21.15.63:0/3081985271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe0104200 msgr2=0x7fcbe01105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.575+0000 7fcbe5400640 1 --2- 172.21.15.63:0/3081985271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe0104200 0x7fcbe01105d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009ca0 tx=0x7fcbd802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.575+0000 7fcbe5400640 1 -- 172.21.15.63:0/3081985271 shutdown_connections 2024-05-27T07:52:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.575+0000 7fcbe5400640 1 --2- 172.21.15.63:0/3081985271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe0104200 0x7fcbe01105d0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.575+0000 7fcbe5400640 1 --2- 172.21.15.63:0/3081985271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe0109f50 0x7fcbe0103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.575+0000 7fcbe5400640 1 --2- 172.21.15.63:0/3081985271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0109ab0 0x7fcbe0103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.575+0000 7fcbe5400640 1 -- 172.21.15.63:0/3081985271 >> 172.21.15.63:0/3081985271 conn(0x7fcbe0079140 msgr2=0x7fcbe0079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.575+0000 7fcbe5400640 1 -- 172.21.15.63:0/3081985271 shutdown_connections 2024-05-27T07:52:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.575+0000 7fcbe5400640 1 -- 172.21.15.63:0/3081985271 wait complete. 2024-05-27T07:52:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.576+0000 7fcbe5400640 1 Processor -- start 2024-05-27T07:52:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.576+0000 7fcbe5400640 1 -- start start 2024-05-27T07:52:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.577+0000 7fcbe5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe0104200 0x7fcbe019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.577+0000 7fcbe5400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0109ab0 0x7fcbe019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.577+0000 7fcbe5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe0109f50 0x7fcbe01a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.577+0000 7fcbe5400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0113bc0 con 0x7fcbe0109f50 2024-05-27T07:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.577+0000 7fcbe5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0113a40 con 0x7fcbe0104200 2024-05-27T07:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.577+0000 7fcbe5400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0113d40 con 0x7fcbe0109ab0 2024-05-27T07:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.577+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe0109f50 0x7fcbe01a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.577+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe0109f50 0x7fcbe01a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36282/0 (socket says 172.21.15.63:36282) 2024-05-27T07:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.577+0000 7fcbe4a00640 1 -- 172.21.15.63:0/773570784 learned_addr learned my addr 172.21.15.63:0/773570784 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.577+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe0104200 0x7fcbe019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.578+0000 7fcbdf400640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0109ab0 0x7fcbe019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.578+0000 7fcbe4a00640 1 -- 172.21.15.63:0/773570784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0109ab0 msgr2=0x7fcbe019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.578+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0109ab0 0x7fcbe019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.578+0000 7fcbe4a00640 1 -- 172.21.15.63:0/773570784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe0104200 msgr2=0x7fcbe019c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.578+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe0104200 0x7fcbe019c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.578+0000 7fcbe4a00640 1 -- 172.21.15.63:0/773570784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe01a1640 con 0x7fcbe0109f50 2024-05-27T07:52:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.578+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe0109f50 0x7fcbe01a1000 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8005cc0 tx=0x7fcbd8032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.579+0000 7fcbdcc00640 1 -- 172.21.15.63:0/773570784 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8032e00 con 0x7fcbe0109f50 2024-05-27T07:52:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.579+0000 7fcbdcc00640 1 -- 172.21.15.63:0/773570784 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbd80044d0 con 0x7fcbe0109f50 2024-05-27T07:52:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.579+0000 7fcbe5400640 1 -- 172.21.15.63:0/773570784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe01a18d0 con 0x7fcbe0109f50 2024-05-27T07:52:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.579+0000 7fcbdcc00640 1 -- 172.21.15.63:0/773570784 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8030b70 con 0x7fcbe0109f50 2024-05-27T07:52:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.579+0000 7fcbe5400640 1 -- 172.21.15.63:0/773570784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe01a1e20 con 0x7fcbe0109f50 2024-05-27T07:52:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.583+0000 7fcbeda12740 1 Processor -- start 2024-05-27T07:52:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.583+0000 7fcbdcc00640 1 -- 172.21.15.63:0/773570784 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcbd8004680 con 0x7fcbe0109f50 2024-05-27T07:52:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.584+0000 7fcbeda12740 1 -- start start 2024-05-27T07:52:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.584+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c391b0 0x5563a0fdba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.584+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c102f0 0x5563a0fdbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.584+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c184d0 0x5563a0fdc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.584+0000 7fcbeda12740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5563a0f4cb90 con 0x5563a0c102f0 2024-05-27T07:52:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.584+0000 7fcbeda12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5563a0f3c730 con 0x5563a0c184d0 2024-05-27T07:52:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.584+0000 7fcbeda12740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5563a0f40770 con 0x5563a0c391b0 2024-05-27T07:52:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.585+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c102f0 0x5563a0fdbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.585+0000 7fcbdfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c102f0 0x5563a0fdbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36292/0 (socket says 172.21.15.63:36292) 2024-05-27T07:52:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.585+0000 7fcbdfe00640 1 -- 172.21.15.63:0/3471681145 learned_addr learned my addr 172.21.15.63:0/3471681145 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.585+0000 7fcbdf400640 1 --2- 172.21.15.63:0/3471681145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c184d0 0x5563a0fdc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.585+0000 7fcbdfe00640 1 -- 172.21.15.63:0/3471681145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c391b0 msgr2=0x5563a0fdba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.585+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/3471681145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c391b0 0x5563a0fdba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.585+0000 7fcbdfe00640 1 -- 172.21.15.63:0/3471681145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c184d0 msgr2=0x5563a0fdc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.585+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/3471681145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c184d0 0x5563a0fdc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.585+0000 7fcbdfe00640 1 -- 172.21.15.63:0/3471681145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a0ed58b0 con 0x5563a0c102f0 2024-05-27T07:52:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.586+0000 7fcbdcc00640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcbc40b2950 0x7fcbc40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.586+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/3471681145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c102f0 0x5563a0fdbfa0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b740 tx=0x7fcbd000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb610dc2b453a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.586+0000 7fcbdcc00640 1 -- 172.21.15.63:0/773570784 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fcbd8103f30 con 0x7fcbe0109f50 2024-05-27T07:52:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.586+0000 7fcbd5200640 1 -- 172.21.15.63:0/3471681145 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0003dd0 con 0x5563a0c102f0 2024-05-27T07:52:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.586+0000 7fcbdf400640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcbc40b2950 0x7fcbc40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.586+0000 7fcbd5200640 1 -- 172.21.15.63:0/3471681145 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbd0015070 con 0x5563a0c102f0 2024-05-27T07:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.586+0000 7fcbd5200640 1 -- 172.21.15.63:0/3471681145 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0004c70 con 0x5563a0c102f0 2024-05-27T07:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.587+0000 7fcbeda12740 1 -- 172.21.15.63:0/3471681145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c102f0 msgr2=0x5563a0fdbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.587+0000 7fcbeda12740 1 --2- 172.21.15.63:0/3471681145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c102f0 0x5563a0fdbfa0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b740 tx=0x7fcbd000bc00 comp rx=0 tx=0).stop 2024-05-27T07:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.587+0000 7fcbdf400640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcbc40b2950 0x7fcbc40b4e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00a540 tx=0x7fcbcc012040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.588+0000 7fcbeda12740 1 -- 172.21.15.63:0/3471681145 shutdown_connections 2024-05-27T07:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.588+0000 7fcbeda12740 1 --2- 172.21.15.63:0/3471681145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c184d0 0x5563a0fdc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.588+0000 7fcbeda12740 1 --2- 172.21.15.63:0/3471681145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c102f0 0x5563a0fdbfa0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.588+0000 7fcbeda12740 1 --2- 172.21.15.63:0/3471681145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c391b0 0x5563a0fdba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.588+0000 7fcbeda12740 1 -- 172.21.15.63:0/3471681145 >> 172.21.15.63:0/3471681145 conn(0x5563a0c2d300 msgr2=0x5563a0fd8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.588+0000 7fcbeda12740 1 -- 172.21.15.63:0/3471681145 shutdown_connections 2024-05-27T07:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.588+0000 7fcbeda12740 1 -- 172.21.15.63:0/3471681145 wait complete. 2024-05-27T07:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.588+0000 7fcbeda12740 1 -- start start 2024-05-27T07:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.589+0000 7fcbeda12740 1 Processor -- start 2024-05-27T07:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.589+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 0x5563a0fe5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.589+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c184d0 0x5563a0fe61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.589+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c391b0 0x5563a0ff1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.589+0000 7fcbeda12740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5563a0c22790 con 0x5563a0c184d0 2024-05-27T07:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.589+0000 7fcbeda12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5563a0ee4bc0 con 0x5563a0c391b0 2024-05-27T07:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.589+0000 7fcbeda12740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5563a0fe8b90 con 0x5563a0c102f0 2024-05-27T07:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbeda12740 10 client.0 _ll_register_callbacks cb 0x5563a0ec2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbdf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c184d0 0x5563a0fe61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbdf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c184d0 0x5563a0fe61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36304/0 (socket says 172.21.15.63:36304) 2024-05-27T07:52:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbdf400640 1 -- 172.21.15.63:0/2273933944 learned_addr learned my addr 172.21.15.63:0/2273933944 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 0x5563a0fe5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c391b0 0x5563a0ff1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbdf400640 1 -- 172.21.15.63:0/2273933944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 msgr2=0x5563a0fe5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbdf400640 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 0x5563a0fe5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbdf400640 1 -- 172.21.15.63:0/2273933944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c391b0 msgr2=0x5563a0ff1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbdf400640 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c391b0 0x5563a0ff1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbdf400640 1 -- 172.21.15.63:0/2273933944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a0c51210 con 0x5563a0c184d0 2024-05-27T07:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c391b0 0x5563a0ff1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.590+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 0x5563a0fe5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.591+0000 7fcbdf400640 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c184d0 0x5563a0fe61a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00aad0 tx=0x7fcbcc00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d643cbaaa0d48b0e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.591+0000 7fcbc9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.591+0000 7fcbc9600640 1 -- 172.21.15.63:0/2273933944 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00f6f0 con 0x5563a0c184d0 2024-05-27T07:52:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.591+0000 7fcbc9600640 1 -- 172.21.15.63:0/2273933944 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fcbcc00f890 con 0x5563a0c184d0 2024-05-27T07:52:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.591+0000 7fcbeda12740 1 -- 172.21.15.63:0/2273933944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd0015070 con 0x5563a0c184d0 2024-05-27T07:52:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.592+0000 7fcbc9600640 1 -- 172.21.15.63:0/2273933944 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcbcc00fa90 con 0x5563a0c184d0 2024-05-27T07:52:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.592+0000 7fcbeda12740 10 client.19119 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.592+0000 7fcbeda12740 1 -- 172.21.15.63:0/2273933944 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5563a0d1d620 con 0x5563a0c184d0 2024-05-27T07:52:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.592+0000 7fcbc9600640 1 -- 172.21.15.63:0/2273933944 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbd0015070 con 0x5563a0c184d0 2024-05-27T07:52:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.592+0000 7fcbc9600640 1 -- 172.21.15.63:0/2273933944 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcbcc01c3f0 con 0x5563a0c184d0 2024-05-27T07:52:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.592+0000 7fcbeda12740 10 client.19119 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.592+0000 7fcbeda12740 10 client.19119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.592+0000 7fcbeda12740 20 client.19119 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.593+0000 7fcbeda12740 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5563a0bf7d80 0x5563a0ff7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.593+0000 7fcbeda12740 4 client.19119 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.593+0000 7fcbeda12740 1 -- 172.21.15.63:0/2273933944 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563a0f4eae0 con 0x5563a0bf7d80 2024-05-27T07:52:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.593+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5563a0bf7d80 0x5563a0ff7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.595+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5563a0bf7d80 0x5563a0ff7f40 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413d6873483453c2 server_cookie=97897c07a838594c in_seq=0 out_seq=0 2024-05-27T07:52:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.595+0000 7fcbc9600640 10 client.19119 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.596+0000 7fcbc9600640 1 -- 172.21.15.63:0/2273933944 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5563a0f4eae0 con 0x5563a0bf7d80 2024-05-27T07:52:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.596+0000 7fcbc9600640 10 client.19119 handle_command_reply: tid=0 2024-05-27T07:52:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 client.19119 shutdown 2024-05-27T07:52:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 -- 172.21.15.63:0/2273933944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c184d0 msgr2=0x5563a0fe61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c184d0 0x5563a0fe61a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00aad0 tx=0x7fcbcc00a2d0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 -- 172.21.15.63:0/2273933944 shutdown_connections 2024-05-27T07:52:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5563a0bf7d80 0x5563a0ff7f40 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c391b0 0x5563a0ff1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c184d0 0x5563a0fe61a0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 --2- 172.21.15.63:0/2273933944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 0x5563a0fe5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbc9600640 0 client.19119 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 -- 172.21.15.63:0/2273933944 >> 172.21.15.63:0/2273933944 conn(0x5563a0c2d300 msgr2=0x5563a0fdb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 -- 172.21.15.63:0/2273933944 shutdown_connections 2024-05-27T07:52:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 1 -- 172.21.15.63:0/2273933944 wait complete. 2024-05-27T07:52:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.597+0000 7fcbeda12740 20 client.19119 trim_cache size 0 max 16384 2024-05-27T07:52:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.617+0000 7fcbeda12740 1 Processor -- start 2024-05-27T07:52:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.617+0000 7fcbeda12740 1 -- start start 2024-05-27T07:52:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.617+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c102f0 0x5563a0fe1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.617+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c184d0 0x5563a0fe7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.617+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0fe82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.617+0000 7fcbeda12740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b5b50 con 0x5563a0c391b0 2024-05-27T07:52:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.617+0000 7fcbeda12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b3420 con 0x5563a0c102f0 2024-05-27T07:52:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.617+0000 7fcbeda12740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b1290 con 0x5563a0c184d0 2024-05-27T07:52:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.617+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0fe82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.618+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0fe82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36308/0 (socket says 172.21.15.63:36308) 2024-05-27T07:52:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.618+0000 7fcbe4a00640 1 -- 172.21.15.63:0/155183206 learned_addr learned my addr 172.21.15.63:0/155183206 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.618+0000 7fcbdf400640 1 --2- 172.21.15.63:0/155183206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c184d0 0x5563a0fe7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.618+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/155183206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c102f0 0x5563a0fe1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.618+0000 7fcbe4a00640 1 -- 172.21.15.63:0/155183206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c184d0 msgr2=0x5563a0fe7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.618+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/155183206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c184d0 0x5563a0fe7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.618+0000 7fcbe4a00640 1 -- 172.21.15.63:0/155183206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c102f0 msgr2=0x5563a0fe1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.618+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/155183206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c102f0 0x5563a0fe1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.618+0000 7fcbe4a00640 1 -- 172.21.15.63:0/155183206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a0fd9340 con 0x5563a0c391b0 2024-05-27T07:52:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.618+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/155183206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0fe82f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80cae90 tx=0x7fcbd80ca910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a63b9df092e6dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.619+0000 7fcbcbe00640 1 -- 172.21.15.63:0/155183206 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd80caf00 con 0x5563a0c391b0 2024-05-27T07:52:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.619+0000 7fcbcbe00640 1 -- 172.21.15.63:0/155183206 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbd8036020 con 0x5563a0c391b0 2024-05-27T07:52:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.619+0000 7fcbcbe00640 1 -- 172.21.15.63:0/155183206 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd80cb160 con 0x5563a0c391b0 2024-05-27T07:52:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.620+0000 7fcbeda12740 1 -- 172.21.15.63:0/155183206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 msgr2=0x5563a0fe82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.620+0000 7fcbeda12740 1 --2- 172.21.15.63:0/155183206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0fe82f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80cae90 tx=0x7fcbd80ca910 comp rx=0 tx=0).stop 2024-05-27T07:52:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.620+0000 7fcbeda12740 1 -- 172.21.15.63:0/155183206 shutdown_connections 2024-05-27T07:52:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.620+0000 7fcbeda12740 1 --2- 172.21.15.63:0/155183206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0fe82f0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.620+0000 7fcbeda12740 1 --2- 172.21.15.63:0/155183206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c184d0 0x5563a0fe7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.620+0000 7fcbeda12740 1 --2- 172.21.15.63:0/155183206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c102f0 0x5563a0fe1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.620+0000 7fcbeda12740 1 -- 172.21.15.63:0/155183206 >> 172.21.15.63:0/155183206 conn(0x5563a0c2d300 msgr2=0x5563a0fe5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.620+0000 7fcbeda12740 1 -- 172.21.15.63:0/155183206 shutdown_connections 2024-05-27T07:52:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.620+0000 7fcbeda12740 1 -- 172.21.15.63:0/155183206 wait complete. 2024-05-27T07:52:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.621+0000 7fcbeda12740 1 -- start start 2024-05-27T07:52:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.621+0000 7fcbeda12740 1 Processor -- start 2024-05-27T07:52:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.621+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 0x5563a0fee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.621+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c184d0 0x5563a0feb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.621+0000 7fcbeda12740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0febcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.621+0000 7fcbeda12740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b18e0 con 0x5563a0c391b0 2024-05-27T07:52:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.621+0000 7fcbeda12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b2d50 con 0x5563a0c184d0 2024-05-27T07:52:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.621+0000 7fcbeda12740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe01b1fb0 con 0x5563a0c102f0 2024-05-27T07:52:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbeda12740 10 client.0 _ll_register_callbacks cb 0x7fcbcc00a940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0febcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbe4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0febcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36316/0 (socket says 172.21.15.63:36316) 2024-05-27T07:52:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbe4a00640 1 -- 172.21.15.63:0/3989332207 learned_addr learned my addr 172.21.15.63:0/3989332207 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbdf400640 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c184d0 0x5563a0feb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 0x5563a0fee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbe4a00640 1 -- 172.21.15.63:0/3989332207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 msgr2=0x5563a0fee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 0x5563a0fee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbe4a00640 1 -- 172.21.15.63:0/3989332207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c184d0 msgr2=0x5563a0feb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c184d0 0x5563a0feb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbe4a00640 1 -- 172.21.15.63:0/3989332207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a0fec310 con 0x5563a0c391b0 2024-05-27T07:52:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.622+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 0x5563a0fee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.623+0000 7fcbe4a00640 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0febcd0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80cac00 tx=0x7fcbd80c73c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c16089a24c64d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.623+0000 7fcbc9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.623+0000 7fcbc9600640 1 -- 172.21.15.63:0/3989332207 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd80caf00 con 0x5563a0c391b0 2024-05-27T07:52:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.623+0000 7fcbc9600640 1 -- 172.21.15.63:0/3989332207 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fcbd80cb160 con 0x5563a0c391b0 2024-05-27T07:52:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.623+0000 7fcbeda12740 1 -- 172.21.15.63:0/3989332207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd8036020 con 0x5563a0c391b0 2024-05-27T07:52:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.623+0000 7fcbc9600640 1 -- 172.21.15.63:0/3989332207 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcbd80cf2b0 con 0x5563a0c391b0 2024-05-27T07:52:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.624+0000 7fcbeda12740 10 client.19125 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.624+0000 7fcbeda12740 1 -- 172.21.15.63:0/3989332207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5563a1060400 con 0x5563a0c391b0 2024-05-27T07:52:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.624+0000 7fcbc9600640 1 -- 172.21.15.63:0/3989332207 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcbd8036020 con 0x5563a0c391b0 2024-05-27T07:52:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.624+0000 7fcbc9600640 1 -- 172.21.15.63:0/3989332207 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fcbd8004b40 con 0x5563a0c391b0 2024-05-27T07:52:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.624+0000 7fcbeda12740 10 client.19125 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.624+0000 7fcbeda12740 10 client.19125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.624+0000 7fcbeda12740 20 client.19125 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.624+0000 7fcbeda12740 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5563a0bcd1c0 0x5563a1062b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.624+0000 7fcbeda12740 4 client.19125 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:52:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.624+0000 7fcbeda12740 1 -- 172.21.15.63:0/3989332207 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563a0f482c0 con 0x5563a0bcd1c0 2024-05-27T07:52:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.625+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5563a0bcd1c0 0x5563a1062b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.626+0000 7fcbdfe00640 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5563a0bcd1c0 0x5563a1062b10 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce8cc0333b0d0c19 server_cookie=19aeff9bf964deb in_seq=0 out_seq=0 2024-05-27T07:52:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.626+0000 7fcbc9600640 10 client.19125 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.659+0000 7fcbc9600640 1 -- 172.21.15.63:0/3989332207 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5563a0f482c0 con 0x5563a0bcd1c0 2024-05-27T07:52:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.659+0000 7fcbc9600640 10 client.19125 handle_command_reply: tid=0 2024-05-27T07:52:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.659+0000 7fcbeda12740 1 client.19125 shutdown 2024-05-27T07:52:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 1 -- 172.21.15.63:0/3989332207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 msgr2=0x5563a0febcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0febcd0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80cac00 tx=0x7fcbd80c73c0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 1 -- 172.21.15.63:0/3989332207 shutdown_connections 2024-05-27T07:52:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5563a0bcd1c0 0x5563a1062b10 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563a0c391b0 0x5563a0febcd0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563a0c184d0 0x5563a0feb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 1 --2- 172.21.15.63:0/3989332207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5563a0c102f0 0x5563a0fee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbc9600640 0 client.19125 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 1 -- 172.21.15.63:0/3989332207 >> 172.21.15.63:0/3989332207 conn(0x5563a0c2d300 msgr2=0x5563a0fe7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 1 -- 172.21.15.63:0/3989332207 shutdown_connections 2024-05-27T07:52:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 1 -- 172.21.15.63:0/3989332207 wait complete. 2024-05-27T07:52:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.660+0000 7fcbeda12740 20 client.19125 trim_cache size 0 max 16384 2024-05-27T07:52:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:52:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:52:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.661+0000 7fcbe5400640 1 -- 172.21.15.63:0/773570784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcbc40b2950 msgr2=0x7fcbc40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.661+0000 7fcbe5400640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcbc40b2950 0x7fcbc40b4e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00a540 tx=0x7fcbcc012040 comp rx=0 tx=0).stop 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.661+0000 7fcbe5400640 1 -- 172.21.15.63:0/773570784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe0109f50 msgr2=0x7fcbe01a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.661+0000 7fcbe5400640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe0109f50 0x7fcbe01a1000 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8005cc0 tx=0x7fcbd8032a10 comp rx=0 tx=0).stop 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.662+0000 7fcbe5400640 1 -- 172.21.15.63:0/773570784 shutdown_connections 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.662+0000 7fcbe5400640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcbc40b2950 0x7fcbc40b4e60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.662+0000 7fcbe5400640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbe0109f50 0x7fcbe01a1000 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.662+0000 7fcbe5400640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0109ab0 0x7fcbe019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.662+0000 7fcbe5400640 1 --2- 172.21.15.63:0/773570784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe0104200 0x7fcbe019c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.662+0000 7fcbe5400640 1 -- 172.21.15.63:0/773570784 >> 172.21.15.63:0/773570784 conn(0x7fcbe0079140 msgr2=0x7fcbe0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.662+0000 7fcbe5400640 1 -- 172.21.15.63:0/773570784 shutdown_connections 2024-05-27T07:52:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:52:52.662+0000 7fcbe5400640 1 -- 172.21.15.63:0/773570784 wait complete. 2024-05-27T07:52:52.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.939+0000 7f7d08c00700 1 -- 172.21.15.63:0/4235951305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d04108730 msgr2=0x7f7d04110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.939+0000 7f7d08c00700 1 --2- 172.21.15.63:0/4235951305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d04108730 0x7f7d04110090 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00b0d0 tx=0x7f7cfc00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.939+0000 7f7d08c00700 1 -- 172.21.15.63:0/4235951305 shutdown_connections 2024-05-27T07:52:52.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.939+0000 7f7d08c00700 1 --2- 172.21.15.63:0/4235951305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d04108730 0x7f7d04110090 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.939+0000 7f7d08c00700 1 --2- 172.21.15.63:0/4235951305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d041034b0 0x7f7d041038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.939+0000 7f7d08c00700 1 --2- 172.21.15.63:0/4235951305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d04107de0 0x7f7d041081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.939+0000 7f7d08c00700 1 -- 172.21.15.63:0/4235951305 >> 172.21.15.63:0/4235951305 conn(0x7f7d040fedb0 msgr2=0x7f7d041011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.940+0000 7f7d08c00700 1 -- 172.21.15.63:0/4235951305 shutdown_connections 2024-05-27T07:52:52.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.940+0000 7f7d08c00700 1 -- 172.21.15.63:0/4235951305 wait complete. 2024-05-27T07:52:52.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.941+0000 7f7d08c00700 1 Processor -- start 2024-05-27T07:52:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.941+0000 7f7d08c00700 1 -- start start 2024-05-27T07:52:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.942+0000 7f7d08c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d041034b0 0x7f7d0419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.942+0000 7f7d08c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d04107de0 0x7f7d0419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.942+0000 7f7d08c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d04108730 0x7f7d041a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.942+0000 7f7d08c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04113260 con 0x7f7d04108730 2024-05-27T07:52:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.942+0000 7f7d08c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041130e0 con 0x7f7d04107de0 2024-05-27T07:52:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.942+0000 7f7d08c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041133e0 con 0x7f7d041034b0 2024-05-27T07:52:52.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.942+0000 7f7d02800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d04108730 0x7f7d041a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.943+0000 7f7d02800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d04108730 0x7f7d041a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36332/0 (socket says 172.21.15.63:36332) 2024-05-27T07:52:52.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.943+0000 7f7d01e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d041034b0 0x7f7d0419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.943+0000 7f7d02800700 1 -- 172.21.15.63:0/1276840470 learned_addr learned my addr 172.21.15.63:0/1276840470 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:52.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.943+0000 7f7d01400700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d04107de0 0x7f7d0419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.943+0000 7f7d02800700 1 -- 172.21.15.63:0/1276840470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d041034b0 msgr2=0x7f7d0419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.944+0000 7f7d02800700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d041034b0 0x7f7d0419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.944+0000 7f7d02800700 1 -- 172.21.15.63:0/1276840470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d04107de0 msgr2=0x7f7d0419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.944+0000 7f7d02800700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d04107de0 0x7f7d0419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.944+0000 7f7d02800700 1 -- 172.21.15.63:0/1276840470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d041a1e00 con 0x7f7d04108730 2024-05-27T07:52:52.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.944+0000 7f7d01400700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d04107de0 0x7f7d0419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:52.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.944+0000 7f7d01e00700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d041034b0 0x7f7d0419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:52.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.944+0000 7f7d02800700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d04108730 0x7f7d041a1760 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0079a0 tx=0x7f7cfc0079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.944+0000 7f7cfaa00700 1 -- 172.21.15.63:0/1276840470 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc01f070 con 0x7f7d04108730 2024-05-27T07:52:52.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.944+0000 7f7d08c00700 1 -- 172.21.15.63:0/1276840470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d041a2090 con 0x7f7d04108730 2024-05-27T07:52:52.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.945+0000 7f7d08c00700 1 -- 172.21.15.63:0/1276840470 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041a9a10 con 0x7f7d04108730 2024-05-27T07:52:52.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.945+0000 7f7cfaa00700 1 -- 172.21.15.63:0/1276840470 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cfc003c00 con 0x7f7d04108730 2024-05-27T07:52:52.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.945+0000 7f7cfaa00700 1 -- 172.21.15.63:0/1276840470 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc01b430 con 0x7f7d04108730 2024-05-27T07:52:52.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.950+0000 7f7cfaa00700 1 -- 172.21.15.63:0/1276840470 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7cfc022070 con 0x7f7d04108730 2024-05-27T07:52:52.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.950+0000 7f7d17805b80 1 Processor -- start 2024-05-27T07:52:52.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.950+0000 7f7d17805b80 1 -- start start 2024-05-27T07:52:52.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.951+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab96fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.951+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96fdd30 0x560ab96f7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.951+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 0x560ab96f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.951+0000 7f7d17805b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560ab9625e40 con 0x560ab9578e40 2024-05-27T07:52:52.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.951+0000 7f7d17805b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ab96159e0 con 0x560ab96f7db0 2024-05-27T07:52:52.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.951+0000 7f7d17805b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560ab9619a20 con 0x560ab96fdd30 2024-05-27T07:52:52.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.951+0000 7f7d01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab96fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.952+0000 7f7d01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab96fd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36340/0 (socket says 172.21.15.63:36340) 2024-05-27T07:52:52.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.952+0000 7f7d01400700 1 -- 172.21.15.63:0/2415665524 learned_addr learned my addr 172.21.15.63:0/2415665524 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:52.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.952+0000 7f7d01e00700 1 --2- 172.21.15.63:0/2415665524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96fdd30 0x560ab96f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.952+0000 7f7d01400700 1 --2- 172.21.15.63:0/2415665524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 0x560ab96f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.952+0000 7f7cfaa00700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7ce80b2a70 0x7f7ce80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.952+0000 7f7d01400700 1 -- 172.21.15.63:0/2415665524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96fdd30 msgr2=0x560ab96f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.952+0000 7f7d01400700 1 --2- 172.21.15.63:0/2415665524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96fdd30 0x560ab96f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.952+0000 7f7d01400700 1 -- 172.21.15.63:0/2415665524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 msgr2=0x560ab96f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.952+0000 7f7d01400700 1 --2- 172.21.15.63:0/2415665524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 0x560ab96f81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.952+0000 7f7d01400700 1 -- 172.21.15.63:0/2415665524 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab95de420 con 0x560ab9578e40 2024-05-27T07:52:52.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.953+0000 7f7cfaa00700 1 -- 172.21.15.63:0/1276840470 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f7cfc00f020 con 0x7f7d04108730 2024-05-27T07:52:52.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.953+0000 7f7d02800700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7ce80b2a70 0x7f7ce80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.953+0000 7f7d01400700 1 --2- 172.21.15.63:0/2415665524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab96fd7f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00046e0 tx=0x7f7cf0007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940cbebd514bfb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.953+0000 7f7cef400700 1 -- 172.21.15.63:0/2415665524 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf001e5f0 con 0x560ab9578e40 2024-05-27T07:52:52.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.953+0000 7f7cef400700 1 -- 172.21.15.63:0/2415665524 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cf00108c0 con 0x560ab9578e40 2024-05-27T07:52:52.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.953+0000 7f7cef400700 1 -- 172.21.15.63:0/2415665524 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00117d0 con 0x560ab9578e40 2024-05-27T07:52:52.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.955+0000 7f7d17805b80 1 -- 172.21.15.63:0/2415665524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 msgr2=0x560ab96fd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.955+0000 7f7d17805b80 1 --2- 172.21.15.63:0/2415665524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab96fd7f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00046e0 tx=0x7f7cf0007b90 comp rx=0 tx=0).stop 2024-05-27T07:52:52.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.955+0000 7f7d02800700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7ce80b2a70 0x7f7ce80b4f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0e0ff0 tx=0x7f7cfc0099e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.955+0000 7f7d17805b80 1 -- 172.21.15.63:0/2415665524 shutdown_connections 2024-05-27T07:52:52.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.955+0000 7f7d17805b80 1 --2- 172.21.15.63:0/2415665524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 0x560ab96f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.955+0000 7f7d17805b80 1 --2- 172.21.15.63:0/2415665524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab96fd7f0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.955+0000 7f7d17805b80 1 --2- 172.21.15.63:0/2415665524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96fdd30 0x560ab96f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.955+0000 7f7d17805b80 1 -- 172.21.15.63:0/2415665524 >> 172.21.15.63:0/2415665524 conn(0x560ab95d8fc0 msgr2=0x560ab96f28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.955+0000 7f7d17805b80 1 -- 172.21.15.63:0/2415665524 shutdown_connections 2024-05-27T07:52:52.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.955+0000 7f7d17805b80 1 -- 172.21.15.63:0/2415665524 wait complete. 2024-05-27T07:52:52.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.956+0000 7f7d17805b80 1 -- start start 2024-05-27T07:52:52.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.956+0000 7f7d17805b80 1 Processor -- start 2024-05-27T07:52:52.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab97014e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96f7db0 0x560ab9701a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96fdd30 0x560ab9701f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d17805b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560ab951bfa0 con 0x560ab9578e40 2024-05-27T07:52:52.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d17805b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ab95b6760 con 0x560ab96fdd30 2024-05-27T07:52:52.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d17805b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560ab9704800 con 0x560ab96f7db0 2024-05-27T07:52:52.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d17805b80 10 client.0 _ll_register_callbacks cb 0x560ab95bfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:52.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab97014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab97014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36356/0 (socket says 172.21.15.63:36356) 2024-05-27T07:52:52.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d01400700 1 -- 172.21.15.63:0/4292775280 learned_addr learned my addr 172.21.15.63:0/4292775280 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:52.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.957+0000 7f7d01e00700 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96f7db0 0x560ab9701a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.958+0000 7f7d01400700 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96fdd30 0x560ab9701f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.958+0000 7f7d01400700 1 -- 172.21.15.63:0/4292775280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96f7db0 msgr2=0x560ab9701a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.958+0000 7f7d01400700 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96f7db0 0x560ab9701a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.958+0000 7f7d01400700 1 -- 172.21.15.63:0/4292775280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96fdd30 msgr2=0x560ab9701f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.958+0000 7f7d01400700 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96fdd30 0x560ab9701f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.958+0000 7f7d01400700 1 -- 172.21.15.63:0/4292775280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab95658d0 con 0x560ab9578e40 2024-05-27T07:52:52.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.958+0000 7f7d01400700 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab97014e0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0007670 tx=0x7f7cf0005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4f3d58054ffbf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.958+0000 7f7ce3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:52.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.959+0000 7f7ce3e00700 1 -- 172.21.15.63:0/4292775280 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf002e630 con 0x560ab9578e40 2024-05-27T07:52:52.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.959+0000 7f7ce3e00700 1 -- 172.21.15.63:0/4292775280 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f7cf002e7d0 con 0x560ab9578e40 2024-05-27T07:52:52.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.959+0000 7f7d17805b80 1 -- 172.21.15.63:0/4292775280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cf00108c0 con 0x560ab9578e40 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.959+0000 7f7ce3e00700 1 -- 172.21.15.63:0/4292775280 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7cf002e9d0 con 0x560ab9578e40 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.959+0000 7f7d17805b80 10 client.19137 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.959+0000 7f7d17805b80 1 -- 172.21.15.63:0/4292775280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560ab95e6060 con 0x560ab9578e40 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.960+0000 7f7ce3e00700 1 -- 172.21.15.63:0/4292775280 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cf00108c0 con 0x560ab9578e40 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.960+0000 7f7ce3e00700 1 -- 172.21.15.63:0/4292775280 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7cf0030c20 con 0x560ab9578e40 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.960+0000 7f7d17805b80 10 client.19137 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.960+0000 7f7d17805b80 10 client.19137 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.960+0000 7f7d17805b80 20 client.19137 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.960+0000 7f7d17805b80 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560ab97115f0 0x560ab9713a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.960+0000 7f7d17805b80 4 client.19137 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:52.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.960+0000 7f7d17805b80 1 -- 172.21.15.63:0/4292775280 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ab9627d90 con 0x560ab97115f0 2024-05-27T07:52:52.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.961+0000 7f7d01e00700 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560ab97115f0 0x560ab9713a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.962+0000 7f7d01e00700 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560ab97115f0 0x560ab9713a10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1e1e55fa3af8378 server_cookie=a140300df5277efe in_seq=0 out_seq=0 2024-05-27T07:52:52.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.962+0000 7f7ce3e00700 10 client.19137 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7ce3e00700 1 -- 172.21.15.63:0/4292775280 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560ab9627d90 con 0x560ab97115f0 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7ce3e00700 10 client.19137 handle_command_reply: tid=0 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7d17805b80 1 client.19137 shutdown 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7d17805b80 1 -- 172.21.15.63:0/4292775280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 msgr2=0x560ab97014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7d17805b80 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab97014e0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0007670 tx=0x7f7cf0005ce0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7d17805b80 1 -- 172.21.15.63:0/4292775280 shutdown_connections 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7d17805b80 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96fdd30 0x560ab9701f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7d17805b80 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560ab97115f0 0x560ab9713a10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7d17805b80 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab97014e0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7d17805b80 1 --2- 172.21.15.63:0/4292775280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96f7db0 0x560ab9701a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7ce3e00700 0 client.19137 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:52:52.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.964+0000 7f7d17805b80 1 -- 172.21.15.63:0/4292775280 >> 172.21.15.63:0/4292775280 conn(0x560ab95d8fc0 msgr2=0x560ab96f2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.965+0000 7f7d17805b80 1 -- 172.21.15.63:0/4292775280 shutdown_connections 2024-05-27T07:52:52.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.965+0000 7f7d17805b80 1 -- 172.21.15.63:0/4292775280 wait complete. 2024-05-27T07:52:52.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.965+0000 7f7d17805b80 20 client.19137 trim_cache size 0 max 16384 2024-05-27T07:52:52.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.989+0000 7f7d17805b80 1 Processor -- start 2024-05-27T07:52:52.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.989+0000 7f7d17805b80 1 -- start start 2024-05-27T07:52:52.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.990+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab9578e40 0x560ab96f7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.990+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 0x560ab96fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.990+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab96fdd30 0x560ab970d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.990+0000 7f7d17805b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041b4f70 con 0x560ab96fdd30 2024-05-27T07:52:52.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.990+0000 7f7d17805b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041b2800 con 0x560ab96f7db0 2024-05-27T07:52:52.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.990+0000 7f7d17805b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041b0670 con 0x560ab9578e40 2024-05-27T07:52:52.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.990+0000 7f7d01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab96fdd30 0x560ab970d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.990+0000 7f7d01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab96fdd30 0x560ab970d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36358/0 (socket says 172.21.15.63:36358) 2024-05-27T07:52:52.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.990+0000 7f7d01400700 1 -- 172.21.15.63:0/1096143084 learned_addr learned my addr 172.21.15.63:0/1096143084 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:52.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.991+0000 7f7d01e00700 1 --2- 172.21.15.63:0/1096143084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 0x560ab96fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.991+0000 7f7d01400700 1 --2- 172.21.15.63:0/1096143084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab9578e40 0x560ab96f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.991+0000 7f7d01e00700 1 -- 172.21.15.63:0/1096143084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab9578e40 msgr2=0x560ab96f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.991+0000 7f7d01e00700 1 --2- 172.21.15.63:0/1096143084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab9578e40 0x560ab96f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.991+0000 7f7d01e00700 1 -- 172.21.15.63:0/1096143084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab96fdd30 msgr2=0x560ab970d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.991+0000 7f7d01e00700 1 --2- 172.21.15.63:0/1096143084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab96fdd30 0x560ab970d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.991+0000 7f7d01e00700 1 -- 172.21.15.63:0/1096143084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab97022d0 con 0x560ab96f7db0 2024-05-27T07:52:52.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.991+0000 7f7d01400700 1 --2- 172.21.15.63:0/1096143084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab96fdd30 0x560ab970d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:52.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.992+0000 7f7d01e00700 1 --2- 172.21.15.63:0/1096143084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 0x560ab96fbb70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x560ab95b68b0 tx=0x7f7cf4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea197a58b533cd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.992+0000 7f7ceea00700 1 -- 172.21.15.63:0/1096143084 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4011440 con 0x560ab96f7db0 2024-05-27T07:52:52.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.992+0000 7f7ceea00700 1 -- 172.21.15.63:0/1096143084 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cf4011a70 con 0x560ab96f7db0 2024-05-27T07:52:52.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.992+0000 7f7ceea00700 1 -- 172.21.15.63:0/1096143084 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf401eaf0 con 0x560ab96f7db0 2024-05-27T07:52:52.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.993+0000 7f7d17805b80 1 -- 172.21.15.63:0/1096143084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 msgr2=0x560ab96fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.993+0000 7f7d17805b80 1 --2- 172.21.15.63:0/1096143084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 0x560ab96fbb70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x560ab95b68b0 tx=0x7f7cf4009260 comp rx=0 tx=0).stop 2024-05-27T07:52:52.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.994+0000 7f7d17805b80 1 -- 172.21.15.63:0/1096143084 shutdown_connections 2024-05-27T07:52:52.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.994+0000 7f7d17805b80 1 --2- 172.21.15.63:0/1096143084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab96fdd30 0x560ab970d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.994+0000 7f7d17805b80 1 --2- 172.21.15.63:0/1096143084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab9578e40 0x560ab96f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.994+0000 7f7d17805b80 1 --2- 172.21.15.63:0/1096143084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96f7db0 0x560ab96fbb70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.994+0000 7f7d17805b80 1 -- 172.21.15.63:0/1096143084 >> 172.21.15.63:0/1096143084 conn(0x560ab95d8fc0 msgr2=0x560ab9700440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.994+0000 7f7d17805b80 1 -- 172.21.15.63:0/1096143084 shutdown_connections 2024-05-27T07:52:52.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.994+0000 7f7d17805b80 1 -- 172.21.15.63:0/1096143084 wait complete. 2024-05-27T07:52:52.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.994+0000 7f7d17805b80 1 -- start start 2024-05-27T07:52:52.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.995+0000 7f7d17805b80 1 Processor -- start 2024-05-27T07:52:52.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.995+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab9709ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.995+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96f7db0 0x560ab970a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.995+0000 7f7d17805b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96fdd30 0x560ab97180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.995+0000 7f7d17805b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041b0cc0 con 0x560ab9578e40 2024-05-27T07:52:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.995+0000 7f7d17805b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041b2130 con 0x560ab96fdd30 2024-05-27T07:52:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.995+0000 7f7d17805b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041b1390 con 0x560ab96f7db0 2024-05-27T07:52:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.995+0000 7f7d17805b80 10 client.0 _ll_register_callbacks cb 0x560ab95bfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.996+0000 7f7d01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab9709ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.996+0000 7f7d01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab9709ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36360/0 (socket says 172.21.15.63:36360) 2024-05-27T07:52:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.996+0000 7f7d01400700 1 -- 172.21.15.63:0/1388276726 learned_addr learned my addr 172.21.15.63:0/1388276726 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.996+0000 7f7d01e00700 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96f7db0 0x560ab970a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.996+0000 7f7d01400700 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96fdd30 0x560ab97180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.996+0000 7f7d01400700 1 -- 172.21.15.63:0/1388276726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96f7db0 msgr2=0x560ab970a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.996+0000 7f7d01400700 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96f7db0 0x560ab970a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.996+0000 7f7d01400700 1 -- 172.21.15.63:0/1388276726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96fdd30 msgr2=0x560ab97180e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.996+0000 7f7d01400700 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96fdd30 0x560ab97180e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.996+0000 7f7d01400700 1 -- 172.21.15.63:0/1388276726 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab9718670 con 0x560ab9578e40 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.997+0000 7f7d01400700 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab9709ac0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0012460 tx=0x7f7cf0030040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd76b2eb21e15c2c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.997+0000 7f7ce3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.997+0000 7f7ce3e00700 1 -- 172.21.15.63:0/1388276726 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf002e630 con 0x560ab9578e40 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.997+0000 7f7ce3e00700 1 -- 172.21.15.63:0/1388276726 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f7cf002e7d0 con 0x560ab9578e40 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.997+0000 7f7d17805b80 1 -- 172.21.15.63:0/1388276726 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cf4011a70 con 0x560ab9578e40 2024-05-27T07:52:52.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.997+0000 7f7ce3e00700 1 -- 172.21.15.63:0/1388276726 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7cf0005110 con 0x560ab9578e40 2024-05-27T07:52:53.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.997+0000 7f7d17805b80 10 client.19143 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:53.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.997+0000 7f7d17805b80 1 -- 172.21.15.63:0/1388276726 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560ab977b6c0 con 0x560ab9578e40 2024-05-27T07:52:53.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.998+0000 7f7ce3e00700 1 -- 172.21.15.63:0/1388276726 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cf4011a70 con 0x560ab9578e40 2024-05-27T07:52:53.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.998+0000 7f7ce3e00700 1 -- 172.21.15.63:0/1388276726 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7cf0003680 con 0x560ab9578e40 2024-05-27T07:52:53.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.998+0000 7f7d17805b80 10 client.19143 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:53.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.998+0000 7f7d17805b80 10 client.19143 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:52:53.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.998+0000 7f7d17805b80 20 client.19143 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:53.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.998+0000 7f7d17805b80 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560ab977bce0 0x560ab977e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:53.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.998+0000 7f7d17805b80 4 client.19143 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:52:53.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.998+0000 7f7d17805b80 1 -- 172.21.15.63:0/1388276726 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ab9621570 con 0x560ab977bce0 2024-05-27T07:52:53.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:52.999+0000 7f7d01e00700 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560ab977bce0 0x560ab977e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:53.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.000+0000 7f7ce3e00700 10 client.19143 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:52:53.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.000+0000 7f7d01e00700 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560ab977bce0 0x560ab977e100 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c561d5c81eb936d server_cookie=49bfc67773ffd74a in_seq=0 out_seq=0 2024-05-27T07:52:53.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.000+0000 7f7ce3e00700 1 -- 172.21.15.63:0/1388276726 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ab9621570 con 0x560ab977bce0 2024-05-27T07:52:53.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.000+0000 7f7ce3e00700 10 client.19143 handle_command_reply: tid=0 2024-05-27T07:52:53.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.000+0000 7f7d17805b80 1 client.19143 shutdown 2024-05-27T07:52:53.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.000+0000 7f7d17805b80 1 -- 172.21.15.63:0/1388276726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 msgr2=0x560ab9709ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.000+0000 7f7d17805b80 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab9709ac0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0012460 tx=0x7f7cf0030040 comp rx=0 tx=0).stop 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.001+0000 7f7d17805b80 1 -- 172.21.15.63:0/1388276726 shutdown_connections 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.001+0000 7f7d17805b80 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab96fdd30 0x560ab97180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.001+0000 7f7d17805b80 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560ab977bce0 0x560ab977e100 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.001+0000 7f7d17805b80 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ab9578e40 0x560ab9709ac0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.001+0000 7f7d17805b80 1 --2- 172.21.15.63:0/1388276726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560ab96f7db0 0x560ab970a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.001+0000 7f7d17805b80 1 -- 172.21.15.63:0/1388276726 >> 172.21.15.63:0/1388276726 conn(0x560ab95d8fc0 msgr2=0x560ab97012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.001+0000 7f7ce3e00700 0 client.19143 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.001+0000 7f7d17805b80 1 -- 172.21.15.63:0/1388276726 shutdown_connections 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.001+0000 7f7d17805b80 1 -- 172.21.15.63:0/1388276726 wait complete. 2024-05-27T07:52:53.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.001+0000 7f7d17805b80 20 client.19143 trim_cache size 0 max 16384 2024-05-27T07:52:53.006 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:52:53.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.002+0000 7f7d08c00700 1 -- 172.21.15.63:0/1276840470 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7ce80b2a70 msgr2=0x7f7ce80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:53.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.002+0000 7f7d08c00700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7ce80b2a70 0x7f7ce80b4f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0e0ff0 tx=0x7f7cfc0099e0 comp rx=0 tx=0).stop 2024-05-27T07:52:53.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.002+0000 7f7d08c00700 1 -- 172.21.15.63:0/1276840470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d04108730 msgr2=0x7f7d041a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:53.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.002+0000 7f7d08c00700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d04108730 0x7f7d041a1760 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0079a0 tx=0x7f7cfc0079d0 comp rx=0 tx=0).stop 2024-05-27T07:52:53.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.002+0000 7f7d08c00700 1 -- 172.21.15.63:0/1276840470 shutdown_connections 2024-05-27T07:52:53.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.002+0000 7f7d08c00700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7ce80b2a70 0x7f7ce80b4f80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:53.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.002+0000 7f7d08c00700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d04108730 0x7f7d041a1760 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:53.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.002+0000 7f7d08c00700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d041034b0 0x7f7d0419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:53.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.002+0000 7f7d08c00700 1 --2- 172.21.15.63:0/1276840470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d04107de0 0x7f7d0419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:53.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.002+0000 7f7d08c00700 1 -- 172.21.15.63:0/1276840470 >> 172.21.15.63:0/1276840470 conn(0x7f7d040fedb0 msgr2=0x7f7d0406b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:53.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.003+0000 7f7d08c00700 1 -- 172.21.15.63:0/1276840470 shutdown_connections 2024-05-27T07:52:53.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:53.003+0000 7f7d08c00700 1 -- 172.21.15.63:0/1276840470 wait complete. 2024-05-27T07:52:53.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:53 smithi063 ceph-mon[31613]: pgmap v608: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:52:53.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:53 smithi086 ceph-mon[34687]: pgmap v608: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:52:53.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:53 smithi149 ceph-mon[35408]: pgmap v608: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:52:53.919 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:52:54.395 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:55.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:55 smithi063 ceph-mon[31613]: pgmap v609: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:52:55.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:55 smithi086 ceph-mon[34687]: pgmap v609: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:52:55.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:55 smithi149 ceph-mon[35408]: pgmap v609: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:52:56.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.854+0000 7f285f200700 1 -- 172.21.15.63:0/2274856255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28601027c0 msgr2=0x7f2860102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:56.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.854+0000 7f285f200700 1 --2- 172.21.15.63:0/2274856255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28601027c0 0x7f2860102bd0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f2848009990 tx=0x7f2848009ca0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.856 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:56 smithi063 ceph-mon[31613]: pgmap v610: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:52:56.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.854+0000 7f285f200700 1 -- 172.21.15.63:0/2274856255 shutdown_connections 2024-05-27T07:52:56.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.854+0000 7f285f200700 1 --2- 172.21.15.63:0/2274856255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2860103110 0x7f286010f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.854+0000 7f285f200700 1 --2- 172.21.15.63:0/2274856255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28601087f0 0x7f2860108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.854+0000 7f285f200700 1 --2- 172.21.15.63:0/2274856255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28601027c0 0x7f2860102bd0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.854+0000 7f285f200700 1 -- 172.21.15.63:0/2274856255 >> 172.21.15.63:0/2274856255 conn(0x7f28600783a0 msgr2=0x7f28600787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:56.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.854+0000 7f285f200700 1 -- 172.21.15.63:0/2274856255 shutdown_connections 2024-05-27T07:52:56.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.855+0000 7f285f200700 1 -- 172.21.15.63:0/2274856255 wait complete. 2024-05-27T07:52:56.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.856+0000 7f285f200700 1 Processor -- start 2024-05-27T07:52:56.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.856+0000 7f285f200700 1 -- start start 2024-05-27T07:52:56.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.857+0000 7f285f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28601027c0 0x7f28601a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.857+0000 7f285f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2860103110 0x7f28601a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.857+0000 7f285f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28601087f0 0x7f286019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.857+0000 7f285f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2860111fa0 con 0x7f28601027c0 2024-05-27T07:52:56.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.857+0000 7f285f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2860111e20 con 0x7f28601087f0 2024-05-27T07:52:56.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.857+0000 7f285f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2860112120 con 0x7f2860103110 2024-05-27T07:52:56.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.857+0000 7f285de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28601027c0 0x7f28601a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.857+0000 7f285de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28601027c0 0x7f28601a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36376/0 (socket says 172.21.15.63:36376) 2024-05-27T07:52:56.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.857+0000 7f285de00700 1 -- 172.21.15.63:0/515064725 learned_addr learned my addr 172.21.15.63:0/515064725 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:56.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.857+0000 7f2855400700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2860103110 0x7f28601a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.858+0000 7f285e800700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28601087f0 0x7f286019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.858+0000 7f285de00700 1 -- 172.21.15.63:0/515064725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2860103110 msgr2=0x7f28601a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:56.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.858+0000 7f285de00700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2860103110 0x7f28601a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.858+0000 7f285de00700 1 -- 172.21.15.63:0/515064725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28601087f0 msgr2=0x7f286019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:56.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.858+0000 7f285de00700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28601087f0 0x7f286019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.858+0000 7f285de00700 1 -- 172.21.15.63:0/515064725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286019b0b0 con 0x7f28601027c0 2024-05-27T07:52:56.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.858+0000 7f2855400700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2860103110 0x7f28601a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:56.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.858+0000 7f285e800700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28601087f0 0x7f286019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:56.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.859+0000 7f285de00700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28601027c0 0x7f28601a07e0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f285000b500 tx=0x7f285000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:56.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.859+0000 7f2857400700 1 -- 172.21.15.63:0/515064725 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850015020 con 0x7f28601027c0 2024-05-27T07:52:56.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.859+0000 7f2857400700 1 -- 172.21.15.63:0/515064725 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2850011cc0 con 0x7f28601027c0 2024-05-27T07:52:56.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.859+0000 7f285f200700 1 -- 172.21.15.63:0/515064725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286019b3a0 con 0x7f28601027c0 2024-05-27T07:52:56.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.859+0000 7f2857400700 1 -- 172.21.15.63:0/515064725 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850010c80 con 0x7f28601027c0 2024-05-27T07:52:56.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.859+0000 7f285f200700 1 -- 172.21.15.63:0/515064725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601a7670 con 0x7f28601027c0 2024-05-27T07:52:56.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.865+0000 7f2857400700 1 -- 172.21.15.63:0/515064725 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2850018020 con 0x7f28601027c0 2024-05-27T07:52:56.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.866+0000 7f287383db80 1 Processor -- start 2024-05-27T07:52:56.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.866+0000 7f287383db80 1 -- start start 2024-05-27T07:52:56.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.866+0000 7f287383db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 0x557ed9ecc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.866+0000 7f287383db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9d4ce40 0x557ed9eccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.867+0000 7f287383db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.867+0000 7f287383db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557ed9df9e40 con 0x557ed9ed62c0 2024-05-27T07:52:56.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.867+0000 7f287383db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ed9de99e0 con 0x557ed9e389f0 2024-05-27T07:52:56.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.867+0000 7f287383db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557ed9deda20 con 0x557ed9d4ce40 2024-05-27T07:52:56.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.867+0000 7f2857400700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28440b2a70 0x7f28440b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.867+0000 7f2855400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.867+0000 7f2855400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36384/0 (socket says 172.21.15.63:36384) 2024-05-27T07:52:56.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.867+0000 7f2855400700 1 -- 172.21.15.63:0/1740923803 learned_addr learned my addr 172.21.15.63:0/1740923803 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:56.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.867+0000 7f285e800700 1 --2- 172.21.15.63:0/1740923803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 0x557ed9ecc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.867+0000 7f285de00700 1 --2- 172.21.15.63:0/1740923803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9d4ce40 0x557ed9eccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.868+0000 7f2857400700 1 -- 172.21.15.63:0/515064725 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f28500e1fe0 con 0x7f28601027c0 2024-05-27T07:52:56.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.868+0000 7f2855400700 1 -- 172.21.15.63:0/1740923803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9d4ce40 msgr2=0x557ed9eccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:56.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.868+0000 7f2855400700 1 --2- 172.21.15.63:0/1740923803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9d4ce40 0x557ed9eccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.868+0000 7f2855400700 1 -- 172.21.15.63:0/1740923803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 msgr2=0x557ed9ecc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:56.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.868+0000 7f2855400700 1 --2- 172.21.15.63:0/1740923803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 0x557ed9ecc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.868+0000 7f2855400700 1 -- 172.21.15.63:0/1740923803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed9db2420 con 0x557ed9ed62c0 2024-05-27T07:52:56.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.868+0000 7f285e800700 1 --2- 172.21.15.63:0/1740923803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 0x557ed9ecc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:56.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.868+0000 7f285e800700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28440b2a70 0x7f28440b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.868+0000 7f2855400700 1 --2- 172.21.15.63:0/1740923803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed86e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f284800b2e0 tx=0x7f28480096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6454515da6040caf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:56.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.869+0000 7f284f400700 1 -- 172.21.15.63:0/1740923803 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284800bbc0 con 0x557ed9ed62c0 2024-05-27T07:52:56.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.869+0000 7f284f400700 1 -- 172.21.15.63:0/1740923803 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f284800bd60 con 0x557ed9ed62c0 2024-05-27T07:52:56.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.869+0000 7f284f400700 1 -- 172.21.15.63:0/1740923803 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848004d90 con 0x557ed9ed62c0 2024-05-27T07:52:56.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.871+0000 7f287383db80 1 -- 172.21.15.63:0/1740923803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 msgr2=0x557ed9ed86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.871+0000 7f287383db80 1 --2- 172.21.15.63:0/1740923803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed86e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f284800b2e0 tx=0x7f28480096a0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.871+0000 7f285e800700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28440b2a70 0x7f28440b4f80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x557ed9ec8a90 tx=0x7f285800c040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.871+0000 7f287383db80 1 -- 172.21.15.63:0/1740923803 shutdown_connections 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.871+0000 7f287383db80 1 --2- 172.21.15.63:0/1740923803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed86e0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.871+0000 7f287383db80 1 --2- 172.21.15.63:0/1740923803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 0x557ed9ecc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.871+0000 7f287383db80 1 --2- 172.21.15.63:0/1740923803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9d4ce40 0x557ed9eccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.871+0000 7f287383db80 1 -- 172.21.15.63:0/1740923803 >> 172.21.15.63:0/1740923803 conn(0x557ed9de6390 msgr2=0x557ed9ec7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.871+0000 7f287383db80 1 -- 172.21.15.63:0/1740923803 shutdown_connections 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.871+0000 7f287383db80 1 -- 172.21.15.63:0/1740923803 wait complete. 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.872+0000 7f287383db80 1 -- start start 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.872+0000 7f287383db80 1 Processor -- start 2024-05-27T07:52:56.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f287383db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9d4ce40 0x557ed9ed3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f287383db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9e389f0 0x557ed9ed3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f287383db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9ed62c0 0x557ed9ed3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f287383db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557ed9ceffa0 con 0x557ed9e389f0 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f287383db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ed9d8a760 con 0x557ed9d4ce40 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f287383db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557ed9edd150 con 0x557ed9ed62c0 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f287383db80 10 client.0 _ll_register_callbacks cb 0x557ed9d93c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f2855400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9e389f0 0x557ed9ed3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f2855400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9e389f0 0x557ed9ed3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36400/0 (socket says 172.21.15.63:36400) 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f2855400700 1 -- 172.21.15.63:0/3458857844 learned_addr learned my addr 172.21.15.63:0/3458857844 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f285de00700 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9d4ce40 0x557ed9ed3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.873+0000 7f285e800700 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9ed62c0 0x557ed9ed3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.874+0000 7f2855400700 1 -- 172.21.15.63:0/3458857844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9ed62c0 msgr2=0x557ed9ed3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.874+0000 7f2855400700 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9ed62c0 0x557ed9ed3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.874+0000 7f2855400700 1 -- 172.21.15.63:0/3458857844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9d4ce40 msgr2=0x557ed9ed3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.874+0000 7f2855400700 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9d4ce40 0x557ed9ed3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.874+0000 7f2855400700 1 -- 172.21.15.63:0/3458857844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed9d398d0 con 0x557ed9e389f0 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.874+0000 7f2855400700 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9e389f0 0x557ed9ed3780 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f2848005540 tx=0x7f2848005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d63243bda33df33 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.874+0000 7f2843e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.875+0000 7f2843e00700 1 -- 172.21.15.63:0/3458857844 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848030060 con 0x557ed9e389f0 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.875+0000 7f2843e00700 1 -- 172.21.15.63:0/3458857844 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f2848004920 con 0x557ed9e389f0 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.875+0000 7f287383db80 1 -- 172.21.15.63:0/3458857844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f284800bd60 con 0x557ed9e389f0 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.875+0000 7f2843e00700 1 -- 172.21.15.63:0/3458857844 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f284800bbc0 con 0x557ed9e389f0 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.875+0000 7f287383db80 10 client.19155 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.875+0000 7f287383db80 1 -- 172.21.15.63:0/3458857844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557ed9dba060 con 0x557ed9e389f0 2024-05-27T07:52:56.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.876+0000 7f2843e00700 1 -- 172.21.15.63:0/3458857844 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f284800bd60 con 0x557ed9e389f0 2024-05-27T07:52:56.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.876+0000 7f2843e00700 1 -- 172.21.15.63:0/3458857844 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f284800fc90 con 0x557ed9e389f0 2024-05-27T07:52:56.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.876+0000 7f287383db80 10 client.19155 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:56.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.876+0000 7f287383db80 10 client.19155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:56.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.876+0000 7f287383db80 20 client.19155 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:56.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.876+0000 7f287383db80 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ed9ee9f40 0x557ed9eec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.876+0000 7f287383db80 4 client.19155 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:56.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.876+0000 7f287383db80 1 -- 172.21.15.63:0/3458857844 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ed9dfbd90 con 0x557ed9ee9f40 2024-05-27T07:52:56.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.877+0000 7f285de00700 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ed9ee9f40 0x557ed9eec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.878+0000 7f2843e00700 10 client.19155 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:56.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.878+0000 7f285de00700 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ed9ee9f40 0x557ed9eec360 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1f1ad8c1bd3863 server_cookie=427de491245b8fb7 in_seq=0 out_seq=0 2024-05-27T07:52:56.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.879+0000 7f2843e00700 1 -- 172.21.15.63:0/3458857844 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557ed9dfbd90 con 0x557ed9ee9f40 2024-05-27T07:52:56.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.879+0000 7f2843e00700 10 client.19155 handle_command_reply: tid=0 2024-05-27T07:52:56.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 client.19155 shutdown 2024-05-27T07:52:56.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 -- 172.21.15.63:0/3458857844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9e389f0 msgr2=0x557ed9ed3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9e389f0 0x557ed9ed3780 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f2848005540 tx=0x7f2848005570 comp rx=0 tx=0).stop 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 -- 172.21.15.63:0/3458857844 shutdown_connections 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ed9ee9f40 0x557ed9eec360 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9ed62c0 0x557ed9ed3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9d4ce40 0x557ed9ed3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 --2- 172.21.15.63:0/3458857844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9e389f0 0x557ed9ed3780 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 -- 172.21.15.63:0/3458857844 >> 172.21.15.63:0/3458857844 conn(0x557ed9de6390 msgr2=0x557ed9ec7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f2843e00700 0 client.19155 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 -- 172.21.15.63:0/3458857844 shutdown_connections 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.880+0000 7f287383db80 1 -- 172.21.15.63:0/3458857844 wait complete. 2024-05-27T07:52:56.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.881+0000 7f287383db80 20 client.19155 trim_cache size 0 max 16384 2024-05-27T07:52:56.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.901+0000 7f287383db80 1 Processor -- start 2024-05-27T07:52:56.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.901+0000 7f287383db80 1 -- start start 2024-05-27T07:52:56.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f287383db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9d4ce40 0x557ed9ec7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f287383db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 0x557ed9ee6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f287383db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f287383db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f283c0116f0 con 0x557ed9ed62c0 2024-05-27T07:52:56.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f287383db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f283c00ef80 con 0x557ed9e389f0 2024-05-27T07:52:56.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f287383db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f283c00cdf0 con 0x557ed9d4ce40 2024-05-27T07:52:56.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f285e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f285e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36408/0 (socket says 172.21.15.63:36408) 2024-05-27T07:52:56.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f285e800700 1 -- 172.21.15.63:0/2045882645 learned_addr learned my addr 172.21.15.63:0/2045882645 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:56.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f285de00700 1 --2- 172.21.15.63:0/2045882645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9d4ce40 0x557ed9ec7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.902+0000 7f2855400700 1 --2- 172.21.15.63:0/2045882645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 0x557ed9ee6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.903+0000 7f285e800700 1 -- 172.21.15.63:0/2045882645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9d4ce40 msgr2=0x557ed9ec7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.903+0000 7f285e800700 1 --2- 172.21.15.63:0/2045882645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9d4ce40 0x557ed9ec7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.903+0000 7f285e800700 1 -- 172.21.15.63:0/2045882645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 msgr2=0x557ed9ee6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.903+0000 7f285e800700 1 --2- 172.21.15.63:0/2045882645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 0x557ed9ee6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.903+0000 7f285e800700 1 -- 172.21.15.63:0/2045882645 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed9ec1930 con 0x557ed9ed62c0 2024-05-27T07:52:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.903+0000 7f285e800700 1 --2- 172.21.15.63:0/2045882645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed04a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f28580178a0 tx=0x7f2858015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9734be2eb2936d3b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.904+0000 7f284ea00700 1 -- 172.21.15.63:0/2045882645 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858013040 con 0x557ed9ed62c0 2024-05-27T07:52:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.904+0000 7f284ea00700 1 -- 172.21.15.63:0/2045882645 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f285801b040 con 0x557ed9ed62c0 2024-05-27T07:52:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.904+0000 7f284ea00700 1 -- 172.21.15.63:0/2045882645 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858005280 con 0x557ed9ed62c0 2024-05-27T07:52:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.905+0000 7f287383db80 1 -- 172.21.15.63:0/2045882645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 msgr2=0x557ed9ed04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.905+0000 7f287383db80 1 --2- 172.21.15.63:0/2045882645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed04a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f28580178a0 tx=0x7f2858015000 comp rx=0 tx=0).stop 2024-05-27T07:52:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.905+0000 7f287383db80 1 -- 172.21.15.63:0/2045882645 shutdown_connections 2024-05-27T07:52:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.905+0000 7f287383db80 1 --2- 172.21.15.63:0/2045882645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9ed04a0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.905+0000 7f287383db80 1 --2- 172.21.15.63:0/2045882645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9d4ce40 0x557ed9ec7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.905+0000 7f287383db80 1 --2- 172.21.15.63:0/2045882645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9e389f0 0x557ed9ee6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.905+0000 7f287383db80 1 -- 172.21.15.63:0/2045882645 >> 172.21.15.63:0/2045882645 conn(0x557ed9de6390 msgr2=0x557ed9eccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.905+0000 7f287383db80 1 -- 172.21.15.63:0/2045882645 shutdown_connections 2024-05-27T07:52:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.905+0000 7f287383db80 1 -- 172.21.15.63:0/2045882645 wait complete. 2024-05-27T07:52:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.906+0000 7f287383db80 1 -- start start 2024-05-27T07:52:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.906+0000 7f287383db80 1 Processor -- start 2024-05-27T07:52:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.906+0000 7f287383db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9d4ce40 0x557ed9eec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.906+0000 7f287383db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9e389f0 0x557ed9eecd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.907+0000 7f287383db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9f527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.907+0000 7f287383db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f283c00d440 con 0x557ed9ed62c0 2024-05-27T07:52:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.907+0000 7f287383db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f283c00e8b0 con 0x557ed9d4ce40 2024-05-27T07:52:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.907+0000 7f287383db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f283c00db10 con 0x557ed9e389f0 2024-05-27T07:52:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.907+0000 7f287383db80 10 client.0 _ll_register_callbacks cb 0x557ed9d93c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.907+0000 7f285e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9f527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.907+0000 7f285e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9f527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36416/0 (socket says 172.21.15.63:36416) 2024-05-27T07:52:56.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.907+0000 7f285e800700 1 -- 172.21.15.63:0/4040782161 learned_addr learned my addr 172.21.15.63:0/4040782161 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:52:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.907+0000 7f285de00700 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9d4ce40 0x557ed9eec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.907+0000 7f2855400700 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9e389f0 0x557ed9eecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.908+0000 7f285e800700 1 -- 172.21.15.63:0/4040782161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9e389f0 msgr2=0x557ed9eecd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.908+0000 7f285e800700 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9e389f0 0x557ed9eecd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.908+0000 7f285e800700 1 -- 172.21.15.63:0/4040782161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9d4ce40 msgr2=0x557ed9eec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.908+0000 7f285e800700 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9d4ce40 0x557ed9eec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.908+0000 7f285e800700 1 -- 172.21.15.63:0/4040782161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed9f52ce0 con 0x557ed9ed62c0 2024-05-27T07:52:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.908+0000 7f285e800700 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9f527a0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f285800c010 tx=0x7f285800aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7763af4e85b22c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.908+0000 7f2843e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:52:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.908+0000 7f2843e00700 1 -- 172.21.15.63:0/4040782161 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858013040 con 0x557ed9ed62c0 2024-05-27T07:52:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.909+0000 7f287383db80 1 -- 172.21.15.63:0/4040782161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f285801b040 con 0x557ed9ed62c0 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.909+0000 7f2843e00700 1 -- 172.21.15.63:0/4040782161 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f2858023630 con 0x557ed9ed62c0 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.909+0000 7f2843e00700 1 -- 172.21.15.63:0/4040782161 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f285803e050 con 0x557ed9ed62c0 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.909+0000 7f287383db80 10 client.19161 fetch_fsmap learned FSMap version 33 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.909+0000 7f287383db80 1 -- 172.21.15.63:0/4040782161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557ed9f53ea0 con 0x557ed9ed62c0 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.909+0000 7f2843e00700 1 -- 172.21.15.63:0/4040782161 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f285801b040 con 0x557ed9ed62c0 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.910+0000 7f2843e00700 1 -- 172.21.15.63:0/4040782161 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2858016050 con 0x557ed9ed62c0 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.910+0000 7f287383db80 10 client.19161 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.910+0000 7f287383db80 10 client.19161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.910+0000 7f287383db80 20 client.19161 populate_metadata read hostname 'smithi063' 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.910+0000 7f287383db80 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ed9f544a0 0x557ed9f568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.910+0000 7f287383db80 4 client.19161 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:52:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.910+0000 7f287383db80 1 -- 172.21.15.63:0/4040782161 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ed9df5570 con 0x557ed9f544a0 2024-05-27T07:52:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.911+0000 7f285de00700 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ed9f544a0 0x557ed9f568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.911+0000 7f2843e00700 10 client.19161 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.911+0000 7f285de00700 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ed9f544a0 0x557ed9f568c0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abf9f95aa0dee64 server_cookie=e99c59c9092790dd in_seq=0 out_seq=0 2024-05-27T07:52:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f2843e00700 1 -- 172.21.15.63:0/4040782161 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ed9df5570 con 0x557ed9f544a0 2024-05-27T07:52:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f2843e00700 10 client.19161 handle_command_reply: tid=0 2024-05-27T07:52:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f287383db80 1 client.19161 shutdown 2024-05-27T07:52:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f287383db80 1 -- 172.21.15.63:0/4040782161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 msgr2=0x557ed9f527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f287383db80 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9f527a0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f285800c010 tx=0x7f285800aa50 comp rx=0 tx=0).stop 2024-05-27T07:52:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f287383db80 1 -- 172.21.15.63:0/4040782161 shutdown_connections 2024-05-27T07:52:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f287383db80 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ed9f544a0 0x557ed9f568c0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f287383db80 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ed9ed62c0 0x557ed9f527a0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f287383db80 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ed9d4ce40 0x557ed9eec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f287383db80 1 --2- 172.21.15.63:0/4040782161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ed9e389f0 0x557ed9eecd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f2843e00700 0 client.19161 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:52:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.912+0000 7f287383db80 1 -- 172.21.15.63:0/4040782161 >> 172.21.15.63:0/4040782161 conn(0x557ed9de6390 msgr2=0x557ed9eccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.913+0000 7f287383db80 1 -- 172.21.15.63:0/4040782161 shutdown_connections 2024-05-27T07:52:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.913+0000 7f287383db80 1 -- 172.21.15.63:0/4040782161 wait complete. 2024-05-27T07:52:56.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.913+0000 7f287383db80 20 client.19161 trim_cache size 0 max 16384 2024-05-27T07:52:56.915 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:52:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 -- 172.21.15.63:0/515064725 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28440b2a70 msgr2=0x7f28440b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28440b2a70 0x7f28440b4f80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x557ed9ec8a90 tx=0x7f285800c040 comp rx=0 tx=0).stop 2024-05-27T07:52:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 -- 172.21.15.63:0/515064725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28601027c0 msgr2=0x7f28601a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28601027c0 0x7f28601a07e0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f285000b500 tx=0x7f285000b8c0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 -- 172.21.15.63:0/515064725 shutdown_connections 2024-05-27T07:52:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28440b2a70 0x7f28440b4f80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28601087f0 0x7f286019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28601027c0 0x7f28601a07e0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 --2- 172.21.15.63:0/515064725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2860103110 0x7f28601a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:56.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 -- 172.21.15.63:0/515064725 >> 172.21.15.63:0/515064725 conn(0x7f28600783a0 msgr2=0x7f2860105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:56.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 -- 172.21.15.63:0/515064725 shutdown_connections 2024-05-27T07:52:56.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:52:56.914+0000 7f285f200700 1 -- 172.21.15.63:0/515064725 wait complete. 2024-05-27T07:52:56.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:56 smithi149 ceph-mon[35408]: pgmap v610: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:52:57.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:56 smithi086 ceph-mon[34687]: pgmap v610: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:52:58.946 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:52:58.946 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:52:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:52:58 smithi149 ceph-mon[35408]: pgmap v611: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:52:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:52:58 smithi063 ceph-mon[31613]: pgmap v611: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:52:59.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:52:58 smithi086 ceph-mon[34687]: pgmap v611: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:52:59.476 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:01.384 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:00 smithi063 ceph-mon[31613]: pgmap v612: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:53:01.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:00 smithi086 ceph-mon[34687]: pgmap v612: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:53:01.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:00 smithi149 ceph-mon[35408]: pgmap v612: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:53:02.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.053+0000 7fa0fdc00700 1 -- 172.21.15.63:0/679711837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0f810a7f0 msgr2=0x7fa0f810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:02.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.053+0000 7fa0fdc00700 1 --2- 172.21.15.63:0/679711837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0f810a7f0 0x7fa0f810ac00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009990 tx=0x7fa0e4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.053+0000 7fa0fdc00700 1 -- 172.21.15.63:0/679711837 shutdown_connections 2024-05-27T07:53:02.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.053+0000 7fa0fdc00700 1 --2- 172.21.15.63:0/679711837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0f810b140 0x7fa0f8117670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.053+0000 7fa0fdc00700 1 --2- 172.21.15.63:0/679711837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f81107d0 0x7fa0f8110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.053+0000 7fa0fdc00700 1 --2- 172.21.15.63:0/679711837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0f810a7f0 0x7fa0f810ac00 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.053+0000 7fa0fdc00700 1 -- 172.21.15.63:0/679711837 >> 172.21.15.63:0/679711837 conn(0x7fa0f8076af0 msgr2=0x7fa0f8076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:02.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.053+0000 7fa0fdc00700 1 -- 172.21.15.63:0/679711837 shutdown_connections 2024-05-27T07:53:02.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.053+0000 7fa0fdc00700 1 -- 172.21.15.63:0/679711837 wait complete. 2024-05-27T07:53:02.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.055+0000 7fa0fdc00700 1 Processor -- start 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.055+0000 7fa0fdc00700 1 -- start start 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.055+0000 7fa0fdc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0f810a7f0 0x7fa0f81a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.055+0000 7fa0fdc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0f810b140 0x7fa0f81a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.055+0000 7fa0fdc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f81107d0 0x7fa0f81a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.055+0000 7fa0fdc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f811a790 con 0x7fa0f810a7f0 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.055+0000 7fa0fdc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f811a610 con 0x7fa0f81107d0 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.055+0000 7fa0fdc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f811a910 con 0x7fa0f810b140 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0f7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0f810a7f0 0x7fa0f81a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0f7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0f810a7f0 0x7fa0f81a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55500/0 (socket says 172.21.15.63:55500) 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0f7e00700 1 -- 172.21.15.63:0/2233618546 learned_addr learned my addr 172.21.15.63:0/2233618546 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:02.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0f7400700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0f810b140 0x7fa0f81a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0fd200700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f81107d0 0x7fa0f81a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0f7e00700 1 -- 172.21.15.63:0/2233618546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0f810b140 msgr2=0x7fa0f81a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0f810b140 0x7fa0f81a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0f7e00700 1 -- 172.21.15.63:0/2233618546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f81107d0 msgr2=0x7fa0f81a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f81107d0 0x7fa0f81a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0f7e00700 1 -- 172.21.15.63:0/2233618546 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f81a6030 con 0x7fa0f810a7f0 2024-05-27T07:53:02.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.056+0000 7fa0fd200700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f81107d0 0x7fa0f81a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:02.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.057+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0f810a7f0 0x7fa0f81a1160 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800d980 tx=0x7fa0e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:02.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.057+0000 7fa0f4c00700 1 -- 172.21.15.63:0/2233618546 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8017070 con 0x7fa0f810a7f0 2024-05-27T07:53:02.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.057+0000 7fa0f4c00700 1 -- 172.21.15.63:0/2233618546 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0e8009dc0 con 0x7fa0f810a7f0 2024-05-27T07:53:02.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.057+0000 7fa0fdc00700 1 -- 172.21.15.63:0/2233618546 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f81a6320 con 0x7fa0f810a7f0 2024-05-27T07:53:02.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.057+0000 7fa0f4c00700 1 -- 172.21.15.63:0/2233618546 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e800f3c0 con 0x7fa0f810a7f0 2024-05-27T07:53:02.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.057+0000 7fa0fdc00700 1 -- 172.21.15.63:0/2233618546 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f81adc40 con 0x7fa0f810a7f0 2024-05-27T07:53:02.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.062+0000 7fa0f4c00700 1 -- 172.21.15.63:0/2233618546 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa0e8009930 con 0x7fa0f810a7f0 2024-05-27T07:53:02.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.063+0000 7fa0f4c00700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa0dc0b2a70 0x7fa0dc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.063+0000 7fa10dd81b80 1 Processor -- start 2024-05-27T07:53:02.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.064+0000 7fa0f4c00700 1 -- 172.21.15.63:0/2233618546 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fa0e80e2480 con 0x7fa0f810a7f0 2024-05-27T07:53:02.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.064+0000 7fa10dd81b80 1 -- start start 2024-05-27T07:53:02.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.064+0000 7fa0f7400700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa0dc0b2a70 0x7fa0dc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.064+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6042f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.064+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f6043490 0x55e2f604db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.064+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f60438a0 0x55e2f60500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.064+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5f71ec0 con 0x55e2f60438a0 2024-05-27T07:53:02.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.064+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5f61a60 con 0x55e2f6043490 2024-05-27T07:53:02.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.064+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5f65aa0 con 0x55e2f5ec4ec0 2024-05-27T07:53:02.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.065+0000 7fa0fd200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f60438a0 0x55e2f60500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.065+0000 7fa0f7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6042f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.065+0000 7fa0f7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6042f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:50034/0 (socket says 172.21.15.63:50034) 2024-05-27T07:53:02.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.065+0000 7fa0fd200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f60438a0 0x55e2f60500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55504/0 (socket says 172.21.15.63:55504) 2024-05-27T07:53:02.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.065+0000 7fa0f7e00700 1 -- 172.21.15.63:0/2428801782 learned_addr learned my addr 172.21.15.63:0/2428801782 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:02.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.065+0000 7fa0f7400700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa0dc0b2a70 0x7fa0dc0b4f80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4006010 tx=0x7fa0e4005960 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:02.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.066+0000 7fa0f7400700 1 --2- 172.21.15.63:0/2428801782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f6043490 0x55e2f604db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.066+0000 7fa0f7e00700 1 -- 172.21.15.63:0/2428801782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f6043490 msgr2=0x55e2f604db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.066+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/2428801782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f6043490 0x55e2f604db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.066+0000 7fa0f7e00700 1 -- 172.21.15.63:0/2428801782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f60438a0 msgr2=0x55e2f60500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.066+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/2428801782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f60438a0 0x55e2f60500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.066+0000 7fa0f7e00700 1 -- 172.21.15.63:0/2428801782 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f5f2a4a0 con 0x55e2f5ec4ec0 2024-05-27T07:53:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.066+0000 7fa0f7400700 1 --2- 172.21.15.63:0/2428801782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f6043490 0x55e2f604db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:02.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.066+0000 7fa0fd200700 1 --2- 172.21.15.63:0/2428801782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f60438a0 0x55e2f60500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:53:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.066+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/2428801782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6042f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80a53c0 tx=0x7fa0e80a8dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40acf8c1354bf3ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.067+0000 7fa0ed600700 1 -- 172.21.15.63:0/2428801782 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8020070 con 0x55e2f5ec4ec0 2024-05-27T07:53:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.067+0000 7fa0ed600700 1 -- 172.21.15.63:0/2428801782 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0e80a9340 con 0x55e2f5ec4ec0 2024-05-27T07:53:02.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.067+0000 7fa0ed600700 1 -- 172.21.15.63:0/2428801782 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80a95a0 con 0x55e2f5ec4ec0 2024-05-27T07:53:02.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.068+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2428801782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 msgr2=0x55e2f6042f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:02.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.068+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/2428801782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6042f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80a53c0 tx=0x7fa0e80a8dc0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.068+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2428801782 shutdown_connections 2024-05-27T07:53:02.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.068+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/2428801782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f60438a0 0x55e2f60500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.068+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/2428801782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6042f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.068+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/2428801782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f6043490 0x55e2f604db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.068+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2428801782 >> 172.21.15.63:0/2428801782 conn(0x55e2f5f25040 msgr2=0x55e2f603edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:02.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.068+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2428801782 shutdown_connections 2024-05-27T07:53:02.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.068+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2428801782 wait complete. 2024-05-27T07:53:02.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.069+0000 7fa10dd81b80 1 -- start start 2024-05-27T07:53:02.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.069+0000 7fa10dd81b80 1 Processor -- start 2024-05-27T07:53:02.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.070+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f604d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.070+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f604dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.070+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 0x55e2f604e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.070+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5e68020 con 0x55e2f6043490 2024-05-27T07:53:02.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.070+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5f027e0 con 0x55e2f60438a0 2024-05-27T07:53:02.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.070+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e2f60509b0 con 0x55e2f5ec4ec0 2024-05-27T07:53:02.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.070+0000 7fa10dd81b80 10 client.0 _ll_register_callbacks cb 0x55e2f5f0bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0f7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f604dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0f7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f604dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55514/0 (socket says 172.21.15.63:55514) 2024-05-27T07:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0f7400700 1 -- 172.21.15.63:0/3178120395 learned_addr learned my addr 172.21.15.63:0/3178120395 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f604d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0fd200700 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 0x55e2f604e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0f7400700 1 -- 172.21.15.63:0/3178120395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 msgr2=0x55e2f604d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0f7400700 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f604d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0f7400700 1 -- 172.21.15.63:0/3178120395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 msgr2=0x55e2f604e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0f7400700 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 0x55e2f604e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0f7400700 1 -- 172.21.15.63:0/3178120395 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f5eb1950 con 0x55e2f6043490 2024-05-27T07:53:02.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.071+0000 7fa0fd200700 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 0x55e2f604e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:02.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.072+0000 7fa0f7400700 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f604dc20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009990 tx=0x7fa0e4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f04acbc1bbf27ee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:02.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.072+0000 7fa0e2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:02.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.072+0000 7fa0e2000700 1 -- 172.21.15.63:0/3178120395 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4003e20 con 0x55e2f6043490 2024-05-27T07:53:02.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.072+0000 7fa0e2000700 1 -- 172.21.15.63:0/3178120395 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fa0e400f460 con 0x55e2f6043490 2024-05-27T07:53:02.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.072+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3178120395 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e80a9340 con 0x55e2f6043490 2024-05-27T07:53:02.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.072+0000 7fa0e2000700 1 -- 172.21.15.63:0/3178120395 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa0e400f660 con 0x55e2f6043490 2024-05-27T07:53:02.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.072+0000 7fa10dd81b80 10 client.19173 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:02.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.072+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3178120395 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e2f5f320e0 con 0x55e2f6043490 2024-05-27T07:53:02.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.073+0000 7fa0e2000700 1 -- 172.21.15.63:0/3178120395 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0e80a9340 con 0x55e2f6043490 2024-05-27T07:53:02.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.073+0000 7fa0e2000700 1 -- 172.21.15.63:0/3178120395 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa0e4022070 con 0x55e2f6043490 2024-05-27T07:53:02.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.073+0000 7fa10dd81b80 10 client.19173 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:02.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.073+0000 7fa10dd81b80 10 client.19173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:02.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.073+0000 7fa10dd81b80 20 client.19173 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:02.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.073+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2f605d930 0x55e2f605fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.073+0000 7fa10dd81b80 4 client.19173 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:02.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.073+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3178120395 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2f5f73e10 con 0x55e2f605d930 2024-05-27T07:53:02.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.074+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2f605d930 0x55e2f605fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.075+0000 7fa0e2000700 10 client.19173 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:02.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.075+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2f605d930 0x55e2f605fd80 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c843098e34ff2f server_cookie=e500da5e2200f635 in_seq=0 out_seq=0 2024-05-27T07:53:02.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa0e2000700 1 -- 172.21.15.63:0/3178120395 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e2f5f73e10 con 0x55e2f605d930 2024-05-27T07:53:02.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa0e2000700 10 client.19173 handle_command_reply: tid=0 2024-05-27T07:53:02.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa10dd81b80 1 client.19173 shutdown 2024-05-27T07:53:02.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3178120395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 msgr2=0x55e2f604dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:02.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f604dc20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009990 tx=0x7fa0e4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3178120395 shutdown_connections 2024-05-27T07:53:02.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2f605d930 0x55e2f605fd80 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 0x55e2f604e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f604d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/3178120395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f604dc20 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3178120395 >> 172.21.15.63:0/3178120395 conn(0x55e2f5f25040 msgr2=0x55e2f603e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:02.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.077+0000 7fa0e2000700 0 client.19173 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:02.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.078+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3178120395 shutdown_connections 2024-05-27T07:53:02.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.078+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3178120395 wait complete. 2024-05-27T07:53:02.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.078+0000 7fa10dd81b80 20 client.19173 trim_cache size 0 max 16384 2024-05-27T07:53:02.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.103+0000 7fa10dd81b80 1 Processor -- start 2024-05-27T07:53:02.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.103+0000 7fa10dd81b80 1 -- start start 2024-05-27T07:53:02.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.104+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f604c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.104+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f6047820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.104+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 0x55e2f6059a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.104+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f81b91b0 con 0x55e2f6043490 2024-05-27T07:53:02.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.104+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f81b6a40 con 0x55e2f60438a0 2024-05-27T07:53:02.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.104+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f81b48b0 con 0x55e2f5ec4ec0 2024-05-27T07:53:02.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.104+0000 7fa0f7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f6047820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.104+0000 7fa0f7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f6047820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55520/0 (socket says 172.21.15.63:55520) 2024-05-27T07:53:02.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.104+0000 7fa0f7400700 1 -- 172.21.15.63:0/3756687666 learned_addr learned my addr 172.21.15.63:0/3756687666 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:02.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.105+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/3756687666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f604c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.105+0000 7fa0fd200700 1 --2- 172.21.15.63:0/3756687666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 0x55e2f6059a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.105+0000 7fa0f7400700 1 -- 172.21.15.63:0/3756687666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 msgr2=0x55e2f604c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:02.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.105+0000 7fa0f7400700 1 --2- 172.21.15.63:0/3756687666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f604c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.105+0000 7fa0f7400700 1 -- 172.21.15.63:0/3756687666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 msgr2=0x55e2f6059a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:02.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.105+0000 7fa0f7400700 1 --2- 172.21.15.63:0/3756687666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 0x55e2f6059a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.105+0000 7fa0f7400700 1 -- 172.21.15.63:0/3756687666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f604fb80 con 0x55e2f6043490 2024-05-27T07:53:02.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.105+0000 7fa0fd200700 1 --2- 172.21.15.63:0/3756687666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 0x55e2f6059a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:02.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.105+0000 7fa0f7400700 1 --2- 172.21.15.63:0/3756687666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f6047820 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fa0e402f8a0 tx=0x7fa0e4005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afdb8bf82e3bff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:02.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.106+0000 7fa0ecc00700 1 -- 172.21.15.63:0/3756687666 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e402fa90 con 0x55e2f6043490 2024-05-27T07:53:02.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.106+0000 7fa0ecc00700 1 -- 172.21.15.63:0/3756687666 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0e4027040 con 0x55e2f6043490 2024-05-27T07:53:02.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.106+0000 7fa0ecc00700 1 -- 172.21.15.63:0/3756687666 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400fdc0 con 0x55e2f6043490 2024-05-27T07:53:02.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.107+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3756687666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 msgr2=0x55e2f6047820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:02.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.107+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/3756687666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f6047820 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fa0e402f8a0 tx=0x7fa0e4005180 comp rx=0 tx=0).stop 2024-05-27T07:53:02.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.107+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3756687666 shutdown_connections 2024-05-27T07:53:02.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.107+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/3756687666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f60438a0 0x55e2f6059a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.107+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/3756687666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f5ec4ec0 0x55e2f604c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.107+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/3756687666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f6047820 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.107+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3756687666 >> 172.21.15.63:0/3756687666 conn(0x55e2f5f25040 msgr2=0x55e2f604def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:02.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.107+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3756687666 shutdown_connections 2024-05-27T07:53:02.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.107+0000 7fa10dd81b80 1 -- 172.21.15.63:0/3756687666 wait complete. 2024-05-27T07:53:02.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.108+0000 7fa10dd81b80 1 -- start start 2024-05-27T07:53:02.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.108+0000 7fa10dd81b80 1 Processor -- start 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.109+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6056080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.109+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f60523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.109+0000 7fa10dd81b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f60438a0 0x55e2f6052930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.109+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f81b4f00 con 0x55e2f6043490 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.109+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f81b6370 con 0x55e2f5ec4ec0 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.109+0000 7fa10dd81b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f81b55d0 con 0x55e2f60438a0 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.109+0000 7fa10dd81b80 10 client.0 _ll_register_callbacks cb 0x55e2f5f0bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.109+0000 7fa0f7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f60523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.109+0000 7fa0f7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f60523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55534/0 (socket says 172.21.15.63:55534) 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.109+0000 7fa0f7400700 1 -- 172.21.15.63:0/2596726051 learned_addr learned my addr 172.21.15.63:0/2596726051 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:02.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6056080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0fd200700 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f60438a0 0x55e2f6052930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0f7400700 1 -- 172.21.15.63:0/2596726051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f60438a0 msgr2=0x55e2f6052930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0f7400700 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f60438a0 0x55e2f6052930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0f7400700 1 -- 172.21.15.63:0/2596726051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f5ec4ec0 msgr2=0x55e2f6056080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0f7400700 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6056080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0f7400700 1 -- 172.21.15.63:0/2596726051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f6052e70 con 0x55e2f6043490 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0fd200700 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f60438a0 0x55e2f6052930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6056080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0f7400700 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f60523f0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4000c00 tx=0x7fa0e400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8f5831bb949798 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.110+0000 7fa0e2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.111+0000 7fa0e2000700 1 -- 172.21.15.63:0/2596726051 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4003e70 con 0x55e2f6043490 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.111+0000 7fa0e2000700 1 -- 172.21.15.63:0/2596726051 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fa0e400fa00 con 0x55e2f6043490 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.111+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2596726051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e4027040 con 0x55e2f6043490 2024-05-27T07:53:02.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.111+0000 7fa0e2000700 1 -- 172.21.15.63:0/2596726051 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa0e400fc00 con 0x55e2f6043490 2024-05-27T07:53:02.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.111+0000 7fa10dd81b80 10 client.19179 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:02.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.111+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2596726051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e2f60c9060 con 0x55e2f6043490 2024-05-27T07:53:02.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.112+0000 7fa0e2000700 1 -- 172.21.15.63:0/2596726051 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0e4027040 con 0x55e2f6043490 2024-05-27T07:53:02.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.112+0000 7fa0e2000700 1 -- 172.21.15.63:0/2596726051 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa0e4022050 con 0x55e2f6043490 2024-05-27T07:53:02.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.112+0000 7fa10dd81b80 10 client.19179 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:02.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.112+0000 7fa10dd81b80 10 client.19179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:02.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.112+0000 7fa10dd81b80 20 client.19179 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:02.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.112+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2f60c9470 0x55e2f60cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:02.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.112+0000 7fa10dd81b80 4 client.19179 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:53:02.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.112+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2596726051 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e2f5f6d5f0 con 0x55e2f60c9470 2024-05-27T07:53:02.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.113+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2f60c9470 0x55e2f60cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:02.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.114+0000 7fa0f7e00700 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2f60c9470 0x55e2f60cb890 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7197d563884eba2 server_cookie=ca9db892c70db22b in_seq=0 out_seq=0 2024-05-27T07:53:02.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.114+0000 7fa0e2000700 10 client.19179 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:02.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.116+0000 7fa0e2000700 1 -- 172.21.15.63:0/2596726051 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e2f5f6d5f0 con 0x55e2f60c9470 2024-05-27T07:53:02.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.116+0000 7fa0e2000700 10 client.19179 handle_command_reply: tid=0 2024-05-27T07:53:02.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.116+0000 7fa10dd81b80 1 client.19179 shutdown 2024-05-27T07:53:02.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2596726051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 msgr2=0x55e2f60523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:02.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f60523f0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4000c00 tx=0x7fa0e400f460 comp rx=0 tx=0).stop 2024-05-27T07:53:02.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2596726051 shutdown_connections 2024-05-27T07:53:02.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e2f60c9470 0x55e2f60cb890 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e2f60438a0 0x55e2f6052930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f5ec4ec0 0x55e2f6056080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 1 --2- 172.21.15.63:0/2596726051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e2f6043490 0x55e2f60523f0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2596726051 >> 172.21.15.63:0/2596726051 conn(0x55e2f5f25040 msgr2=0x55e2f604def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:02.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa0e2000700 0 client.19179 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:02.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2596726051 shutdown_connections 2024-05-27T07:53:02.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 1 -- 172.21.15.63:0/2596726051 wait complete. 2024-05-27T07:53:02.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.117+0000 7fa10dd81b80 20 client.19179 trim_cache size 0 max 16384 2024-05-27T07:53:02.120 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:53:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:53:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "fb86a932-f7db-45e7-b031-6b55cc0a235b", 2024-05-27T07:53:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:53:02.120 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:53:02.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.118+0000 7fa0fdc00700 1 -- 172.21.15.63:0/2233618546 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa0dc0b2a70 msgr2=0x7fa0dc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.118+0000 7fa0fdc00700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa0dc0b2a70 0x7fa0dc0b4f80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4006010 tx=0x7fa0e4005960 comp rx=0 tx=0).stop 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.118+0000 7fa0fdc00700 1 -- 172.21.15.63:0/2233618546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0f810a7f0 msgr2=0x7fa0f81a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.118+0000 7fa0fdc00700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0f810a7f0 0x7fa0f81a1160 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800d980 tx=0x7fa0e800dc90 comp rx=0 tx=0).stop 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.119+0000 7fa0fdc00700 1 -- 172.21.15.63:0/2233618546 shutdown_connections 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.119+0000 7fa0fdc00700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa0dc0b2a70 0x7fa0dc0b4f80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.119+0000 7fa0fdc00700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0f81107d0 0x7fa0f81a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.119+0000 7fa0fdc00700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0f810a7f0 0x7fa0f81a1160 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.119+0000 7fa0fdc00700 1 --2- 172.21.15.63:0/2233618546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0f810b140 0x7fa0f81a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.119+0000 7fa0fdc00700 1 -- 172.21.15.63:0/2233618546 >> 172.21.15.63:0/2233618546 conn(0x7fa0f8076af0 msgr2=0x7fa0f8108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.119+0000 7fa0fdc00700 1 -- 172.21.15.63:0/2233618546 shutdown_connections 2024-05-27T07:53:02.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:02.119+0000 7fa0fdc00700 1 -- 172.21.15.63:0/2233618546 wait complete. 2024-05-27T07:53:02.999 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: pgmap v613: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.8 MiB/s wr, 31 op/s 2024-05-27T07:53:02.999 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:53:02.999 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:02.999 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:53:02.999 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:53:03.000 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.000 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:53:03.000 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.000 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:53:03.000 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.000 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.192 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: pgmap v613: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.8 MiB/s wr, 31 op/s 2024-05-27T07:53:03.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:53:03.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:53:03.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:53:03.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:53:03.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:53:03.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.195 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:53:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: pgmap v613: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.8 MiB/s wr, 31 op/s 2024-05-27T07:53:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:53:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:53:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:53:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:53:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:53:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:03.710 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:04.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:03 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:53:04.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:03 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:04.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:03 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:53:04.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:03 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:04.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:03 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:53:04.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:03 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:04.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:03 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:53:04.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:03 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:04.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:03 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:53:04.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:03 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:04.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:03 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:53:04.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:03 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:04.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:03 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:53:04.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:03 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:04.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:03 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:53:04.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:03 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:04.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:03 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:53:04.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:03 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:05.174 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:04 smithi063 ceph-mon[31613]: pgmap v614: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 28 op/s 2024-05-27T07:53:05.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:04 smithi086 ceph-mon[34687]: pgmap v614: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 28 op/s 2024-05-27T07:53:05.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:04 smithi149 ceph-mon[35408]: pgmap v614: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 28 op/s 2024-05-27T07:53:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.924+0000 7fb212600640 1 -- 172.21.15.63:0/3920019181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c104c50 msgr2=0x7fb20c105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.924+0000 7fb212600640 1 --2- 172.21.15.63:0/3920019181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c104c50 0x7fb20c105060 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fb200009900 tx=0x7fb20002a480 comp rx=0 tx=0).stop 2024-05-27T07:53:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.925+0000 7fb212600640 1 -- 172.21.15.63:0/3920019181 shutdown_connections 2024-05-27T07:53:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.925+0000 7fb212600640 1 --2- 172.21.15.63:0/3920019181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c10e010 0x7fb20c110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.925+0000 7fb212600640 1 --2- 172.21.15.63:0/3920019181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb20c10b6b0 0x7fb20c10dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.925+0000 7fb212600640 1 --2- 172.21.15.63:0/3920019181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c104c50 0x7fb20c105060 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.925+0000 7fb212600640 1 -- 172.21.15.63:0/3920019181 >> 172.21.15.63:0/3920019181 conn(0x7fb20c0fe210 msgr2=0x7fb20c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.925+0000 7fb212600640 1 -- 172.21.15.63:0/3920019181 shutdown_connections 2024-05-27T07:53:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.925+0000 7fb212600640 1 -- 172.21.15.63:0/3920019181 wait complete. 2024-05-27T07:53:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.926+0000 7fb212600640 1 Processor -- start 2024-05-27T07:53:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.926+0000 7fb212600640 1 -- start start 2024-05-27T07:53:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb212600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c104c50 0x7fb20c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb212600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c10b6b0 0x7fb20c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb212600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb20c10e010 0x7fb20c1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb212600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c113b20 con 0x7fb20c104c50 2024-05-27T07:53:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb212600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1139a0 con 0x7fb20c10b6b0 2024-05-27T07:53:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb212600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c113ca0 con 0x7fb20c10e010 2024-05-27T07:53:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb211200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c104c50 0x7fb20c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb211200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c104c50 0x7fb20c19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55554/0 (socket says 172.21.15.63:55554) 2024-05-27T07:53:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb211200640 1 -- 172.21.15.63:0/2106634258 learned_addr learned my addr 172.21.15.63:0/2106634258 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb211c00640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb20c10e010 0x7fb20c1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.927+0000 7fb20be00640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c10b6b0 0x7fb20c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.928+0000 7fb211200640 1 -- 172.21.15.63:0/2106634258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb20c10e010 msgr2=0x7fb20c1a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.928+0000 7fb211200640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb20c10e010 0x7fb20c1a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.928+0000 7fb211200640 1 -- 172.21.15.63:0/2106634258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c10b6b0 msgr2=0x7fb20c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.928+0000 7fb211200640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c10b6b0 0x7fb20c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.928+0000 7fb211200640 1 -- 172.21.15.63:0/2106634258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c1a1640 con 0x7fb20c104c50 2024-05-27T07:53:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.928+0000 7fb211200640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c104c50 0x7fb20c19c7e0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fb200009230 tx=0x7fb200004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.929+0000 7fb209600640 1 -- 172.21.15.63:0/2106634258 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20003f030 con 0x7fb20c104c50 2024-05-27T07:53:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.929+0000 7fb209600640 1 -- 172.21.15.63:0/2106634258 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb200003a80 con 0x7fb20c104c50 2024-05-27T07:53:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.929+0000 7fb212600640 1 -- 172.21.15.63:0/2106634258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c1a18d0 con 0x7fb20c104c50 2024-05-27T07:53:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.929+0000 7fb209600640 1 -- 172.21.15.63:0/2106634258 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200035070 con 0x7fb20c104c50 2024-05-27T07:53:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.929+0000 7fb212600640 1 -- 172.21.15.63:0/2106634258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c1a1e10 con 0x7fb20c104c50 2024-05-27T07:53:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.934+0000 7fb21ac02740 1 Processor -- start 2024-05-27T07:53:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.934+0000 7fb21ac02740 1 -- start start 2024-05-27T07:53:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.934+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.934+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 0x561e957e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.934+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb21ac02740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561e95751b90 con 0x561e9543e1b0 2024-05-27T07:53:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb21ac02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e95741730 con 0x561e954152f0 2024-05-27T07:53:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb21ac02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561e95745770 con 0x561e9541d4d0 2024-05-27T07:53:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb209600640 1 -- 172.21.15.63:0/2106634258 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb200042030 con 0x7fb20c104c50 2024-05-27T07:53:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb211c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb211c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957e0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55560/0 (socket says 172.21.15.63:55560) 2024-05-27T07:53:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb211c00640 1 -- 172.21.15.63:0/726614665 learned_addr learned my addr 172.21.15.63:0/726614665 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb20be00640 1 --2- 172.21.15.63:0/726614665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb211200640 1 --2- 172.21.15.63:0/726614665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 0x561e957e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb211c00640 1 -- 172.21.15.63:0/726614665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 msgr2=0x561e957e14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb211c00640 1 --2- 172.21.15.63:0/726614665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957e14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb211c00640 1 -- 172.21.15.63:0/726614665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 msgr2=0x561e957e0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb211c00640 1 --2- 172.21.15.63:0/726614665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 0x561e957e0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb211c00640 1 -- 172.21.15.63:0/726614665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e956da8b0 con 0x561e9543e1b0 2024-05-27T07:53:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.935+0000 7fb211200640 1 --2- 172.21.15.63:0/726614665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 0x561e957e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.936+0000 7fb211c00640 1 --2- 172.21.15.63:0/726614665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957e0a60 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fb204009e60 tx=0x7fb204010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8e29322a2fe1ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.936+0000 7fb1fe000640 1 -- 172.21.15.63:0/726614665 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204016070 con 0x561e9543e1b0 2024-05-27T07:53:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.936+0000 7fb1fe000640 1 -- 172.21.15.63:0/726614665 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb204004d20 con 0x561e9543e1b0 2024-05-27T07:53:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.936+0000 7fb1fe000640 1 -- 172.21.15.63:0/726614665 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204012690 con 0x561e9543e1b0 2024-05-27T07:53:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.937+0000 7fb209600640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1f40b29d0 0x7fb1f40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.937+0000 7fb209600640 1 -- 172.21.15.63:0/2106634258 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fb200102fe0 con 0x7fb20c104c50 2024-05-27T07:53:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.937+0000 7fb211200640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1f40b29d0 0x7fb1f40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.937+0000 7fb21ac02740 1 -- 172.21.15.63:0/726614665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 msgr2=0x561e957e0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.937+0000 7fb21ac02740 1 --2- 172.21.15.63:0/726614665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957e0a60 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fb204009e60 tx=0x7fb204010740 comp rx=0 tx=0).stop 2024-05-27T07:53:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.938+0000 7fb21ac02740 1 -- 172.21.15.63:0/726614665 shutdown_connections 2024-05-27T07:53:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.938+0000 7fb21ac02740 1 --2- 172.21.15.63:0/726614665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.938+0000 7fb21ac02740 1 --2- 172.21.15.63:0/726614665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 0x561e957e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.938+0000 7fb21ac02740 1 --2- 172.21.15.63:0/726614665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957e0a60 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.938+0000 7fb21ac02740 1 -- 172.21.15.63:0/726614665 >> 172.21.15.63:0/726614665 conn(0x561e95432300 msgr2=0x561e957dd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.938+0000 7fb211200640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1f40b29d0 0x7fb1f40b4ee0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x561e957de550 tx=0x7fb200009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.938+0000 7fb21ac02740 1 -- 172.21.15.63:0/726614665 shutdown_connections 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.938+0000 7fb21ac02740 1 -- 172.21.15.63:0/726614665 wait complete. 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.939+0000 7fb21ac02740 1 -- start start 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.939+0000 7fb21ac02740 1 Processor -- start 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e954152f0 0x561e957eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9541d4d0 0x561e957eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb21ac02740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561e95427790 con 0x561e9543e1b0 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb21ac02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e956e9bc0 con 0x561e9541d4d0 2024-05-27T07:53:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb21ac02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561e957edb90 con 0x561e954152f0 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb21ac02740 10 client.0 _ll_register_callbacks cb 0x561e956c7400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb211c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb211c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55568/0 (socket says 172.21.15.63:55568) 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb211c00640 1 -- 172.21.15.63:0/1128135872 learned_addr learned my addr 172.21.15.63:0/1128135872 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb211c00640 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e954152f0 0x561e957eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.940+0000 7fb20be00640 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9541d4d0 0x561e957eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb211c00640 1 -- 172.21.15.63:0/1128135872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e954152f0 msgr2=0x561e957eabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb211c00640 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e954152f0 0x561e957eabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb211c00640 1 -- 172.21.15.63:0/1128135872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9541d4d0 msgr2=0x561e957eb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb211c00640 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9541d4d0 0x561e957eb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb211c00640 1 -- 172.21.15.63:0/1128135872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e95456210 con 0x561e9543e1b0 2024-05-27T07:53:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb20be00640 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9541d4d0 0x561e957eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb211c00640 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6ec0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x561e957ea410 tx=0x7fb204005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63e279ebc413d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb1f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb1f2a00640 1 -- 172.21.15.63:0/1128135872 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204016030 con 0x561e9543e1b0 2024-05-27T07:53:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb21ac02740 1 -- 172.21.15.63:0/1128135872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb204004d20 con 0x561e9543e1b0 2024-05-27T07:53:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.941+0000 7fb1f2a00640 1 -- 172.21.15.63:0/1128135872 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fb204012cc0 con 0x561e9543e1b0 2024-05-27T07:53:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.942+0000 7fb1f2a00640 1 -- 172.21.15.63:0/1128135872 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb204012700 con 0x561e9543e1b0 2024-05-27T07:53:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.942+0000 7fb21ac02740 10 client.19191 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.942+0000 7fb21ac02740 1 -- 172.21.15.63:0/1128135872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561e95522620 con 0x561e9543e1b0 2024-05-27T07:53:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.942+0000 7fb1f2a00640 1 -- 172.21.15.63:0/1128135872 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb204004d20 con 0x561e9543e1b0 2024-05-27T07:53:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.942+0000 7fb1f2a00640 1 -- 172.21.15.63:0/1128135872 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb20400f040 con 0x561e9543e1b0 2024-05-27T07:53:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.942+0000 7fb21ac02740 10 client.19191 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.942+0000 7fb21ac02740 10 client.19191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.943+0000 7fb21ac02740 20 client.19191 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.943+0000 7fb21ac02740 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561e953fcd80 0x561e957fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.943+0000 7fb21ac02740 4 client.19191 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.943+0000 7fb21ac02740 1 -- 172.21.15.63:0/1128135872 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e95753ae0 con 0x561e953fcd80 2024-05-27T07:53:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.943+0000 7fb20be00640 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561e953fcd80 0x561e957fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.944+0000 7fb20be00640 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561e953fcd80 0x561e957fcc70 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1bc14111ea5436a server_cookie=951ccbf2d6b2b1bd in_seq=0 out_seq=0 2024-05-27T07:53:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.944+0000 7fb1f2a00640 10 client.19191 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.946+0000 7fb1f2a00640 1 -- 172.21.15.63:0/1128135872 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561e95753ae0 con 0x561e953fcd80 2024-05-27T07:53:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.946+0000 7fb1f2a00640 10 client.19191 handle_command_reply: tid=0 2024-05-27T07:53:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.946+0000 7fb21ac02740 1 client.19191 shutdown 2024-05-27T07:53:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 1 -- 172.21.15.63:0/1128135872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 msgr2=0x561e957f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6ec0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x561e957ea410 tx=0x7fb204005990 comp rx=0 tx=0).stop 2024-05-27T07:53:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 1 -- 172.21.15.63:0/1128135872 shutdown_connections 2024-05-27T07:53:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561e953fcd80 0x561e957fcc70 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6ec0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9541d4d0 0x561e957eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 1 --2- 172.21.15.63:0/1128135872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e954152f0 0x561e957eabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb1f2a00640 0 client.19191 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 1 -- 172.21.15.63:0/1128135872 >> 172.21.15.63:0/1128135872 conn(0x561e95432300 msgr2=0x561e957e0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 1 -- 172.21.15.63:0/1128135872 shutdown_connections 2024-05-27T07:53:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 1 -- 172.21.15.63:0/1128135872 wait complete. 2024-05-27T07:53:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.947+0000 7fb21ac02740 20 client.19191 trim_cache size 0 max 16384 2024-05-27T07:53:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.968+0000 7fb21ac02740 1 Processor -- start 2024-05-27T07:53:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.968+0000 7fb21ac02740 1 -- start start 2024-05-27T07:53:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.968+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 0x561e957db810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.969+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957e1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.969+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.969+0000 7fb21ac02740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b5ba0 con 0x561e9543e1b0 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.969+0000 7fb21ac02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b3470 con 0x561e954152f0 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.969+0000 7fb21ac02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b12e0 con 0x561e9541d4d0 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.969+0000 7fb211c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.969+0000 7fb211c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55578/0 (socket says 172.21.15.63:55578) 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.969+0000 7fb211c00640 1 -- 172.21.15.63:0/1092431025 learned_addr learned my addr 172.21.15.63:0/1092431025 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.969+0000 7fb211c00640 1 --2- 172.21.15.63:0/1092431025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 0x561e957db810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.969+0000 7fb20be00640 1 --2- 172.21.15.63:0/1092431025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957e1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.970+0000 7fb211c00640 1 -- 172.21.15.63:0/1092431025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 msgr2=0x561e957e1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.970+0000 7fb211c00640 1 --2- 172.21.15.63:0/1092431025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957e1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.970+0000 7fb211c00640 1 -- 172.21.15.63:0/1092431025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 msgr2=0x561e957db810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.970+0000 7fb211c00640 1 --2- 172.21.15.63:0/1092431025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 0x561e957db810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.970+0000 7fb211c00640 1 -- 172.21.15.63:0/1092431025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e957e1d40 con 0x561e9543e1b0 2024-05-27T07:53:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.970+0000 7fb211c00640 1 --2- 172.21.15.63:0/1092431025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6c30 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x561e957ebfc0 tx=0x7fb204010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b11b1e3e1267beb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.970+0000 7fb1fd600640 1 -- 172.21.15.63:0/1092431025 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20401a5e0 con 0x561e9543e1b0 2024-05-27T07:53:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.970+0000 7fb1fd600640 1 -- 172.21.15.63:0/1092431025 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb20401aa10 con 0x561e9543e1b0 2024-05-27T07:53:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.970+0000 7fb1fd600640 1 -- 172.21.15.63:0/1092431025 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2040388b0 con 0x561e9543e1b0 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.971+0000 7fb21ac02740 1 -- 172.21.15.63:0/1092431025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 msgr2=0x561e957f6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.971+0000 7fb21ac02740 1 --2- 172.21.15.63:0/1092431025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6c30 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x561e957ebfc0 tx=0x7fb204010740 comp rx=0 tx=0).stop 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.971+0000 7fb21ac02740 1 -- 172.21.15.63:0/1092431025 shutdown_connections 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.971+0000 7fb21ac02740 1 --2- 172.21.15.63:0/1092431025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e9543e1b0 0x561e957f6c30 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.971+0000 7fb21ac02740 1 --2- 172.21.15.63:0/1092431025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957e1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.971+0000 7fb21ac02740 1 --2- 172.21.15.63:0/1092431025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e954152f0 0x561e957db810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.972+0000 7fb21ac02740 1 -- 172.21.15.63:0/1092431025 >> 172.21.15.63:0/1092431025 conn(0x561e95432300 msgr2=0x561e957eae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.972+0000 7fb21ac02740 1 -- 172.21.15.63:0/1092431025 shutdown_connections 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.972+0000 7fb21ac02740 1 -- 172.21.15.63:0/1092431025 wait complete. 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.972+0000 7fb21ac02740 1 -- start start 2024-05-27T07:53:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.972+0000 7fb21ac02740 1 Processor -- start 2024-05-27T07:53:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.973+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e954152f0 0x561e957f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.973+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957ee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.973+0000 7fb21ac02740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9543e1b0 0x561e957eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.973+0000 7fb21ac02740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b1930 con 0x561e954152f0 2024-05-27T07:53:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.973+0000 7fb21ac02740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b2da0 con 0x561e9543e1b0 2024-05-27T07:53:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.973+0000 7fb21ac02740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b2000 con 0x561e9541d4d0 2024-05-27T07:53:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.973+0000 7fb21ac02740 10 client.0 _ll_register_callbacks cb 0x7fb204009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.973+0000 7fb211c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e954152f0 0x561e957f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.973+0000 7fb211c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e954152f0 0x561e957f32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55594/0 (socket says 172.21.15.63:55594) 2024-05-27T07:53:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.973+0000 7fb211c00640 1 -- 172.21.15.63:0/2708161164 learned_addr learned my addr 172.21.15.63:0/2708161164 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.974+0000 7fb20be00640 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.974+0000 7fb211c00640 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9543e1b0 0x561e957eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.974+0000 7fb211c00640 1 -- 172.21.15.63:0/2708161164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 msgr2=0x561e957ee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.974+0000 7fb211c00640 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957ee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.974+0000 7fb211c00640 1 -- 172.21.15.63:0/2708161164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9543e1b0 msgr2=0x561e957eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.974+0000 7fb211c00640 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9543e1b0 0x561e957eeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.974+0000 7fb211c00640 1 -- 172.21.15.63:0/2708161164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e957ef080 con 0x561e954152f0 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.974+0000 7fb211c00640 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e954152f0 0x561e957f32b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x561e956c9d80 tx=0x7fb204005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f60707e6b71a1e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.974+0000 7fb1f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.975+0000 7fb1f2a00640 1 -- 172.21.15.63:0/2708161164 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204012cc0 con 0x561e954152f0 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.975+0000 7fb1f2a00640 1 -- 172.21.15.63:0/2708161164 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7fb204048440 con 0x561e954152f0 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.975+0000 7fb21ac02740 1 -- 172.21.15.63:0/2708161164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb20401aa10 con 0x561e954152f0 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.975+0000 7fb1f2a00640 1 -- 172.21.15.63:0/2708161164 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb204051050 con 0x561e954152f0 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.975+0000 7fb21ac02740 10 client.19197 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.975+0000 7fb21ac02740 1 -- 172.21.15.63:0/2708161164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561e95864e40 con 0x561e954152f0 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.975+0000 7fb1f2a00640 1 -- 172.21.15.63:0/2708161164 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb20401aa10 con 0x561e954152f0 2024-05-27T07:53:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.976+0000 7fb1f2a00640 1 -- 172.21.15.63:0/2708161164 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb204046020 con 0x561e954152f0 2024-05-27T07:53:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.976+0000 7fb21ac02740 10 client.19197 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.976+0000 7fb21ac02740 10 client.19197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.976+0000 7fb21ac02740 20 client.19197 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.976+0000 7fb21ac02740 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561e953d21c0 0x561e958674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.976+0000 7fb21ac02740 4 client.19197 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:53:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.976+0000 7fb21ac02740 1 -- 172.21.15.63:0/2708161164 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e9574d2c0 con 0x561e953d21c0 2024-05-27T07:53:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.977+0000 7fb20be00640 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561e953d21c0 0x561e958674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.978+0000 7fb20be00640 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561e953d21c0 0x561e958674f0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac76f7eb3b43a906 server_cookie=ac53936e149ca5c0 in_seq=0 out_seq=0 2024-05-27T07:53:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:05.978+0000 7fb1f2a00640 10 client.19197 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.020+0000 7fb1f2a00640 1 -- 172.21.15.63:0/2708161164 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561e9574d2c0 con 0x561e953d21c0 2024-05-27T07:53:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.020+0000 7fb1f2a00640 10 client.19197 handle_command_reply: tid=0 2024-05-27T07:53:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 client.19197 shutdown 2024-05-27T07:53:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 -- 172.21.15.63:0/2708161164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e954152f0 msgr2=0x561e957f32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e954152f0 0x561e957f32b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x561e956c9d80 tx=0x7fb204005fd0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 -- 172.21.15.63:0/2708161164 shutdown_connections 2024-05-27T07:53:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561e953d21c0 0x561e958674f0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e9543e1b0 0x561e957eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e9541d4d0 0x561e957ee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 --2- 172.21.15.63:0/2708161164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561e954152f0 0x561e957f32b0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 -- 172.21.15.63:0/2708161164 >> 172.21.15.63:0/2708161164 conn(0x561e95432300 msgr2=0x561e957eaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb1f2a00640 0 client.19197 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 -- 172.21.15.63:0/2708161164 shutdown_connections 2024-05-27T07:53:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.021+0000 7fb21ac02740 1 -- 172.21.15.63:0/2708161164 wait complete. 2024-05-27T07:53:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.022+0000 7fb21ac02740 20 client.19197 trim_cache size 0 max 16384 2024-05-27T07:53:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:53:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:53:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:53:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 -- 172.21.15.63:0/2106634258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1f40b29d0 msgr2=0x7fb1f40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1f40b29d0 0x7fb1f40b4ee0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x561e957de550 tx=0x7fb200009260 comp rx=0 tx=0).stop 2024-05-27T07:53:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 -- 172.21.15.63:0/2106634258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c104c50 msgr2=0x7fb20c19c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c104c50 0x7fb20c19c7e0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fb200009230 tx=0x7fb200004270 comp rx=0 tx=0).stop 2024-05-27T07:53:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 -- 172.21.15.63:0/2106634258 shutdown_connections 2024-05-27T07:53:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1f40b29d0 0x7fb1f40b4ee0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb20c10e010 0x7fb20c1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c10b6b0 0x7fb20c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 --2- 172.21.15.63:0/2106634258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb20c104c50 0x7fb20c19c7e0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 -- 172.21.15.63:0/2106634258 >> 172.21.15.63:0/2106634258 conn(0x7fb20c0fe210 msgr2=0x7fb20c109d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 -- 172.21.15.63:0/2106634258 shutdown_connections 2024-05-27T07:53:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:06.023+0000 7fb212600640 1 -- 172.21.15.63:0/2106634258 wait complete. 2024-05-27T07:53:06.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.269+0000 7f382f000700 1 -- 172.21.15.63:0/1890035336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38281043f0 msgr2=0x7f3828104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.269+0000 7f382f000700 1 --2- 172.21.15.63:0/1890035336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38281043f0 0x7f3828104800 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f381c0099c0 tx=0x7f381c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.269+0000 7f382f000700 1 -- 172.21.15.63:0/1890035336 shutdown_connections 2024-05-27T07:53:06.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.269+0000 7f382f000700 1 --2- 172.21.15.63:0/1890035336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3828109670 0x7f3828110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.269+0000 7f382f000700 1 --2- 172.21.15.63:0/1890035336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38281043f0 0x7f3828104800 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.269+0000 7f382f000700 1 --2- 172.21.15.63:0/1890035336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3828108d20 0x7f3828109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.269+0000 7f382f000700 1 -- 172.21.15.63:0/1890035336 >> 172.21.15.63:0/1890035336 conn(0x7f3828079d00 msgr2=0x7f382807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:06.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.269+0000 7f382f000700 1 -- 172.21.15.63:0/1890035336 shutdown_connections 2024-05-27T07:53:06.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.270+0000 7f382f000700 1 -- 172.21.15.63:0/1890035336 wait complete. 2024-05-27T07:53:06.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.271+0000 7f382f000700 1 Processor -- start 2024-05-27T07:53:06.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.271+0000 7f382f000700 1 -- start start 2024-05-27T07:53:06.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.272+0000 7f382f000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281043f0 0x7f38281a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.272+0000 7f382f000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3828108d20 0x7f38281a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.272+0000 7f382f000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3828109670 0x7f382819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.272+0000 7f382f000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3828114170 con 0x7f3828109670 2024-05-27T07:53:06.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.272+0000 7f382f000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3828113ff0 con 0x7f3828108d20 2024-05-27T07:53:06.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.272+0000 7f382f000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f38281142f0 con 0x7f38281043f0 2024-05-27T07:53:06.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.272+0000 7f382e600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3828109670 0x7f382819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.272+0000 7f382e600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3828109670 0x7f382819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55624/0 (socket says 172.21.15.63:55624) 2024-05-27T07:53:06.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.272+0000 7f382e600700 1 -- 172.21.15.63:0/3664937125 learned_addr learned my addr 172.21.15.63:0/3664937125 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:06.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.273+0000 7f382dc00700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281043f0 0x7f38281a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.273+0000 7f382d200700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3828108d20 0x7f38281a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.273+0000 7f382e600700 1 -- 172.21.15.63:0/3664937125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281043f0 msgr2=0x7f38281a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.273+0000 7f382e600700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281043f0 0x7f38281a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.273+0000 7f382e600700 1 -- 172.21.15.63:0/3664937125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3828108d20 msgr2=0x7f38281a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.273+0000 7f382e600700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3828108d20 0x7f38281a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.273+0000 7f382e600700 1 -- 172.21.15.63:0/3664937125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382819b100 con 0x7f3828109670 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.273+0000 7f382dc00700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281043f0 0x7f38281a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.273+0000 7f382e600700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3828109670 0x7f382819aa50 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f382000ef30 tx=0x7f382000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.274+0000 7f3826000700 1 -- 172.21.15.63:0/3664937125 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38200092e0 con 0x7f3828109670 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.274+0000 7f3826000700 1 -- 172.21.15.63:0/3664937125 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3820004bb0 con 0x7f3828109670 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.274+0000 7f382f000700 1 -- 172.21.15.63:0/3664937125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382819b390 con 0x7f3828109670 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.274+0000 7f3826000700 1 -- 172.21.15.63:0/3664937125 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820004020 con 0x7f3828109670 2024-05-27T07:53:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.274+0000 7f382f000700 1 -- 172.21.15.63:0/3664937125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38281a76c0 con 0x7f3828109670 2024-05-27T07:53:06.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.279+0000 7f3826000700 1 -- 172.21.15.63:0/3664937125 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f382001b030 con 0x7f3828109670 2024-05-27T07:53:06.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.281+0000 7f383f1fcb80 1 Processor -- start 2024-05-27T07:53:06.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.281+0000 7f383f1fcb80 1 -- start start 2024-05-27T07:53:06.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.281+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebd31e40 0x5589ebeafd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.281+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebebeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.281+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebeb06d0 0x5589ebec1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.282+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5589ebddee40 con 0x5589ebeb02c0 2024-05-27T07:53:06.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.282+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589ebdce9e0 con 0x5589ebeb06d0 2024-05-27T07:53:06.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.282+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5589ebdd2a20 con 0x5589ebd31e40 2024-05-27T07:53:06.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.282+0000 7f3826000700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f38100b2a70 0x7f38100b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.282+0000 7f3826000700 1 -- 172.21.15.63:0/3664937125 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f38200e2060 con 0x7f3828109670 2024-05-27T07:53:06.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.282+0000 7f382dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebebeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.282+0000 7f382dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebebeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55640/0 (socket says 172.21.15.63:55640) 2024-05-27T07:53:06.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.282+0000 7f382dc00700 1 -- 172.21.15.63:0/1470178243 learned_addr learned my addr 172.21.15.63:0/1470178243 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:06.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.283+0000 7f382d200700 1 --2- 172.21.15.63:0/1470178243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebd31e40 0x5589ebeafd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.283+0000 7f382e600700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f38100b2a70 0x7f38100b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.283+0000 7f382d200700 1 --2- 172.21.15.63:0/1470178243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebeb06d0 0x5589ebec1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.283+0000 7f382dc00700 1 -- 172.21.15.63:0/1470178243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebd31e40 msgr2=0x5589ebeafd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:06.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.283+0000 7f382dc00700 1 --2- 172.21.15.63:0/1470178243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebd31e40 0x5589ebeafd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.283+0000 7f382dc00700 1 -- 172.21.15.63:0/1470178243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebeb06d0 msgr2=0x5589ebec1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:06.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.283+0000 7f382dc00700 1 --2- 172.21.15.63:0/1470178243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebeb06d0 0x5589ebec1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.283+0000 7f382dc00700 1 -- 172.21.15.63:0/1470178243 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ebd97420 con 0x5589ebeb02c0 2024-05-27T07:53:06.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.284+0000 7f382dc00700 1 --2- 172.21.15.63:0/1470178243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebebeee0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f381c00b580 tx=0x7f381c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1259d3ff45735ad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:06.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.284+0000 7f382e600700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f38100b2a70 0x7f38100b4f80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38200df4a0 tx=0x7f3820009480 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:06.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.284+0000 7f381aa00700 1 -- 172.21.15.63:0/1470178243 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c027070 con 0x5589ebeb02c0 2024-05-27T07:53:06.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.284+0000 7f381aa00700 1 -- 172.21.15.63:0/1470178243 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f381c0037c0 con 0x5589ebeb02c0 2024-05-27T07:53:06.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.284+0000 7f381aa00700 1 -- 172.21.15.63:0/1470178243 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c00b990 con 0x5589ebeb02c0 2024-05-27T07:53:06.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.285+0000 7f383f1fcb80 1 -- 172.21.15.63:0/1470178243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 msgr2=0x5589ebebeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.285+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/1470178243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebebeee0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f381c00b580 tx=0x7f381c005040 comp rx=0 tx=0).stop 2024-05-27T07:53:06.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.286+0000 7f383f1fcb80 1 -- 172.21.15.63:0/1470178243 shutdown_connections 2024-05-27T07:53:06.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.286+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/1470178243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebeb06d0 0x5589ebec1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.286+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/1470178243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebd31e40 0x5589ebeafd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.286+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/1470178243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebebeee0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.286+0000 7f383f1fcb80 1 -- 172.21.15.63:0/1470178243 >> 172.21.15.63:0/1470178243 conn(0x5589ebd91fc0 msgr2=0x5589ebeabd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:06.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.286+0000 7f383f1fcb80 1 -- 172.21.15.63:0/1470178243 shutdown_connections 2024-05-27T07:53:06.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.286+0000 7f383f1fcb80 1 -- 172.21.15.63:0/1470178243 wait complete. 2024-05-27T07:53:06.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.288+0000 7f383f1fcb80 1 -- start start 2024-05-27T07:53:06.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.288+0000 7f383f1fcb80 1 Processor -- start 2024-05-27T07:53:06.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebd31e40 0x5589ebec9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebec95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 0x5589ebec3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5589ebcd4fa0 con 0x5589ebeb02c0 2024-05-27T07:53:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589ebd6f760 con 0x5589ebd31e40 2024-05-27T07:53:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5589ebec1d40 con 0x5589ebeb06d0 2024-05-27T07:53:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f383f1fcb80 10 client.0 _ll_register_callbacks cb 0x5589ebd78c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f382dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebec95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f382dc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebec95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55648/0 (socket says 172.21.15.63:55648) 2024-05-27T07:53:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f382dc00700 1 -- 172.21.15.63:0/3071094112 learned_addr learned my addr 172.21.15.63:0/3071094112 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.289+0000 7f382d200700 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 0x5589ebec3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.290+0000 7f382d200700 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebd31e40 0x5589ebec9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.290+0000 7f382dc00700 1 -- 172.21.15.63:0/3071094112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 msgr2=0x5589ebec3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.290+0000 7f382dc00700 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 0x5589ebec3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.290+0000 7f382dc00700 1 -- 172.21.15.63:0/3071094112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebd31e40 msgr2=0x5589ebec9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.290+0000 7f382dc00700 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebd31e40 0x5589ebec9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.290+0000 7f382dc00700 1 -- 172.21.15.63:0/3071094112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ebd1e8d0 con 0x5589ebeb02c0 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.290+0000 7f382d200700 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 0x5589ebec3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.290+0000 7f382dc00700 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebec95c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f381c009d10 tx=0x7f381c003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ba41ef2596bc15 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.290+0000 7f380f400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.291+0000 7f380f400700 1 -- 172.21.15.63:0/3071094112 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c02f020 con 0x5589ebeb02c0 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.291+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3071094112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f381c0037c0 con 0x5589ebeb02c0 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.291+0000 7f380f400700 1 -- 172.21.15.63:0/3071094112 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f381c0053b0 con 0x5589ebeb02c0 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.291+0000 7f380f400700 1 -- 172.21.15.63:0/3071094112 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f381c027040 con 0x5589ebeb02c0 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.291+0000 7f383f1fcb80 10 client.19209 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.291+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3071094112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5589ebd9f060 con 0x5589ebeb02c0 2024-05-27T07:53:06.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.292+0000 7f380f400700 1 -- 172.21.15.63:0/3071094112 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f381c0037c0 con 0x5589ebeb02c0 2024-05-27T07:53:06.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.292+0000 7f380f400700 1 -- 172.21.15.63:0/3071094112 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f381c02a030 con 0x5589ebeb02c0 2024-05-27T07:53:06.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.292+0000 7f383f1fcb80 10 client.19209 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:06.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.292+0000 7f383f1fcb80 10 client.19209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:06.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.292+0000 7f383f1fcb80 20 client.19209 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.292+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589ebed2fa0 0x5589ebed53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.292+0000 7f383f1fcb80 4 client.19209 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.292+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3071094112 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589ebde0d90 con 0x5589ebed2fa0 2024-05-27T07:53:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.293+0000 7f382d200700 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589ebed2fa0 0x5589ebed53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.294+0000 7f382d200700 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589ebed2fa0 0x5589ebed53c0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4313e99114480251 server_cookie=a29c8a649c14228a in_seq=0 out_seq=0 2024-05-27T07:53:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.294+0000 7f380f400700 10 client.19209 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:06.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f380f400700 1 -- 172.21.15.63:0/3071094112 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5589ebde0d90 con 0x5589ebed2fa0 2024-05-27T07:53:06.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f380f400700 10 client.19209 handle_command_reply: tid=0 2024-05-27T07:53:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f383f1fcb80 1 client.19209 shutdown 2024-05-27T07:53:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3071094112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 msgr2=0x5589ebec95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebec95c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f381c009d10 tx=0x7f381c003e10 comp rx=0 tx=0).stop 2024-05-27T07:53:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3071094112 shutdown_connections 2024-05-27T07:53:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589ebed2fa0 0x5589ebed53c0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 0x5589ebec3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebd31e40 0x5589ebec9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3071094112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebeb02c0 0x5589ebec95c0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.297+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3071094112 >> 172.21.15.63:0/3071094112 conn(0x5589ebd91fc0 msgr2=0x5589ebeaaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.298+0000 7f380f400700 0 client.19209 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:06.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.298+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3071094112 shutdown_connections 2024-05-27T07:53:06.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.298+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3071094112 wait complete. 2024-05-27T07:53:06.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.298+0000 7f383f1fcb80 20 client.19209 trim_cache size 0 max 16384 2024-05-27T07:53:06.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.327+0000 7f383f1fcb80 1 Processor -- start 2024-05-27T07:53:06.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.327+0000 7f383f1fcb80 1 -- start start 2024-05-27T07:53:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.327+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebec29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.327+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebeb06d0 0x5589ebeab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.327+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebec3940 0x5589ebeabc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.327+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38281b0a40 con 0x5589ebd31e40 2024-05-27T07:53:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.327+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38281ae2d0 con 0x5589ebeb06d0 2024-05-27T07:53:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.327+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f38281ac160 con 0x5589ebec3940 2024-05-27T07:53:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.328+0000 7f382d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebec29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.328+0000 7f382d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebec29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55662/0 (socket says 172.21.15.63:55662) 2024-05-27T07:53:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.328+0000 7f382d200700 1 -- 172.21.15.63:0/1914504122 learned_addr learned my addr 172.21.15.63:0/1914504122 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.328+0000 7f382dc00700 1 --2- 172.21.15.63:0/1914504122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebeb06d0 0x5589ebeab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.328+0000 7f382d200700 1 --2- 172.21.15.63:0/1914504122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebec3940 0x5589ebeabc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.328+0000 7f382d200700 1 -- 172.21.15.63:0/1914504122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebec3940 msgr2=0x5589ebeabc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.328+0000 7f382d200700 1 --2- 172.21.15.63:0/1914504122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebec3940 0x5589ebeabc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.328+0000 7f382d200700 1 -- 172.21.15.63:0/1914504122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebeb06d0 msgr2=0x5589ebeab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.328+0000 7f382d200700 1 --2- 172.21.15.63:0/1914504122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebeb06d0 0x5589ebeab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.328+0000 7f382d200700 1 -- 172.21.15.63:0/1914504122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ebeb14c0 con 0x5589ebd31e40 2024-05-27T07:53:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.329+0000 7f382d200700 1 --2- 172.21.15.63:0/1914504122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebec29f0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f382807a2a0 tx=0x7f3814009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8979b7e70687f32 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.329+0000 7f381a000700 1 -- 172.21.15.63:0/1914504122 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814004830 con 0x5589ebd31e40 2024-05-27T07:53:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.329+0000 7f381a000700 1 -- 172.21.15.63:0/1914504122 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3814027040 con 0x5589ebd31e40 2024-05-27T07:53:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.329+0000 7f381a000700 1 -- 172.21.15.63:0/1914504122 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814005020 con 0x5589ebd31e40 2024-05-27T07:53:06.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.330+0000 7f383f1fcb80 1 -- 172.21.15.63:0/1914504122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 msgr2=0x5589ebec29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.330+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/1914504122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebec29f0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f382807a2a0 tx=0x7f3814009fd0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.331+0000 7f383f1fcb80 1 -- 172.21.15.63:0/1914504122 shutdown_connections 2024-05-27T07:53:06.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.331+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/1914504122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebec3940 0x5589ebeabc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.331+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/1914504122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebec29f0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.331+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/1914504122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebeb06d0 0x5589ebeab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.331+0000 7f383f1fcb80 1 -- 172.21.15.63:0/1914504122 >> 172.21.15.63:0/1914504122 conn(0x5589ebd91fc0 msgr2=0x5589ebeaaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:06.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.331+0000 7f383f1fcb80 1 -- 172.21.15.63:0/1914504122 shutdown_connections 2024-05-27T07:53:06.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.331+0000 7f383f1fcb80 1 -- 172.21.15.63:0/1914504122 wait complete. 2024-05-27T07:53:06.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.331+0000 7f383f1fcb80 1 -- start start 2024-05-27T07:53:06.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.332+0000 7f383f1fcb80 1 Processor -- start 2024-05-27T07:53:06.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.332+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebf319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.332+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 0x5589ebf31ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.332+0000 7f383f1fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebec3940 0x5589ebf3c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.332+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38281ac7b0 con 0x5589ebd31e40 2024-05-27T07:53:06.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.332+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38281adc00 con 0x5589ebec3940 2024-05-27T07:53:06.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.332+0000 7f383f1fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f38281ace60 con 0x5589ebeb06d0 2024-05-27T07:53:06.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f383f1fcb80 10 client.0 _ll_register_callbacks cb 0x5589ebd78c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:06.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f382d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebf319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f382d200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebf319a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55674/0 (socket says 172.21.15.63:55674) 2024-05-27T07:53:06.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f382d200700 1 -- 172.21.15.63:0/3513005501 learned_addr learned my addr 172.21.15.63:0/3513005501 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:06.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f382dc00700 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 0x5589ebf31ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f382d200700 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebec3940 0x5589ebf3c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f382d200700 1 -- 172.21.15.63:0/3513005501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 msgr2=0x5589ebf31ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:06.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f382d200700 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 0x5589ebf31ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f382d200700 1 -- 172.21.15.63:0/3513005501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebec3940 msgr2=0x5589ebf3c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:06.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f382d200700 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebec3940 0x5589ebf3c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.333+0000 7f382d200700 1 -- 172.21.15.63:0/3513005501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ebf3cde0 con 0x5589ebd31e40 2024-05-27T07:53:06.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.334+0000 7f382d200700 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebf319a0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f38140092a0 tx=0x7f3814005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18baa01925dfaee4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:06.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.334+0000 7f380f400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:06.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.334+0000 7f380f400700 1 -- 172.21.15.63:0/3513005501 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814004050 con 0x5589ebd31e40 2024-05-27T07:53:06.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.334+0000 7f380f400700 1 -- 172.21.15.63:0/3513005501 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9483+0+0 (secure 0 0 0) 0x7f38140041f0 con 0x5589ebd31e40 2024-05-27T07:53:06.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.334+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3513005501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3814027040 con 0x5589ebd31e40 2024-05-27T07:53:06.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.335+0000 7f380f400700 1 -- 172.21.15.63:0/3513005501 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3814040050 con 0x5589ebd31e40 2024-05-27T07:53:06.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.335+0000 7f383f1fcb80 10 client.19215 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:06.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.335+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3513005501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5589ebf3e050 con 0x5589ebd31e40 2024-05-27T07:53:06.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.335+0000 7f380f400700 1 -- 172.21.15.63:0/3513005501 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3814027040 con 0x5589ebd31e40 2024-05-27T07:53:06.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.335+0000 7f380f400700 1 -- 172.21.15.63:0/3513005501 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3814005000 con 0x5589ebd31e40 2024-05-27T07:53:06.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.335+0000 7f383f1fcb80 10 client.19215 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:06.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.335+0000 7f383f1fcb80 10 client.19215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:06.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.335+0000 7f383f1fcb80 20 client.19215 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:06.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.335+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589ebf3e6a0 0x5589ebf40ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:06.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.336+0000 7f383f1fcb80 4 client.19215 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:53:06.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.336+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3513005501 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5589ebdda570 con 0x5589ebf3e6a0 2024-05-27T07:53:06.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.336+0000 7f382dc00700 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589ebf3e6a0 0x5589ebf40ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:06.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.337+0000 7f380f400700 10 client.19215 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:06.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.337+0000 7f382dc00700 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589ebf3e6a0 0x5589ebf40ac0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc4b91e919867eed server_cookie=9f41c7c2b23ef645 in_seq=0 out_seq=0 2024-05-27T07:53:06.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f380f400700 1 -- 172.21.15.63:0/3513005501 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5589ebdda570 con 0x5589ebf3e6a0 2024-05-27T07:53:06.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f380f400700 10 client.19215 handle_command_reply: tid=0 2024-05-27T07:53:06.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f383f1fcb80 1 client.19215 shutdown 2024-05-27T07:53:06.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3513005501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 msgr2=0x5589ebf319a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebf319a0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f38140092a0 tx=0x7f3814005fa0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3513005501 shutdown_connections 2024-05-27T07:53:06.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5589ebf3e6a0 0x5589ebf40ac0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589ebec3940 0x5589ebf3c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589ebd31e40 0x5589ebf319a0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f383f1fcb80 1 --2- 172.21.15.63:0/3513005501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5589ebeb06d0 0x5589ebf31ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3513005501 >> 172.21.15.63:0/3513005501 conn(0x5589ebd91fc0 msgr2=0x5589ebec2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:06.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f380f400700 0 client.19215 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:06.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.338+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3513005501 shutdown_connections 2024-05-27T07:53:06.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.339+0000 7f383f1fcb80 1 -- 172.21.15.63:0/3513005501 wait complete. 2024-05-27T07:53:06.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.339+0000 7f383f1fcb80 20 client.19215 trim_cache size 0 max 16384 2024-05-27T07:53:06.341 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:53:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:53:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:53:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "fb86a932-f7db-45e7-b031-6b55cc0a235b": { 2024-05-27T07:53:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:53:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "fb86a932-f7db-45e7-b031-6b55cc0a235b", 2024-05-27T07:53:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:53:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:53:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:53:06.342 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:53:06.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 -- 172.21.15.63:0/3664937125 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f38100b2a70 msgr2=0x7f38100b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f38100b2a70 0x7f38100b4f80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38200df4a0 tx=0x7f3820009480 comp rx=0 tx=0).stop 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 -- 172.21.15.63:0/3664937125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3828109670 msgr2=0x7f382819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3828109670 0x7f382819aa50 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f382000ef30 tx=0x7f382000c3a0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 -- 172.21.15.63:0/3664937125 shutdown_connections 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f38100b2a70 0x7f38100b4f80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3828109670 0x7f382819aa50 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281043f0 0x7f38281a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 --2- 172.21.15.63:0/3664937125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3828108d20 0x7f38281a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 -- 172.21.15.63:0/3664937125 >> 172.21.15.63:0/3664937125 conn(0x7f3828079d00 msgr2=0x7f3828101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.340+0000 7f382f000700 1 -- 172.21.15.63:0/3664937125 shutdown_connections 2024-05-27T07:53:06.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:06.341+0000 7f382f000700 1 -- 172.21.15.63:0/3664937125 wait complete. 2024-05-27T07:53:07.213 INFO:tasks.cephfs.filesystem:scrub status for tag:fb86a932-f7db-45e7-b031-6b55cc0a235b - {'path': '/', 'tag': 'fb86a932-f7db-45e7-b031-6b55cc0a235b', 'options': 'recursive,force'} 2024-05-27T07:53:07.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:07 smithi086 ceph-mon[34687]: pgmap v615: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 28 op/s 2024-05-27T07:53:07.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:07 smithi149 ceph-mon[35408]: pgmap v615: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 28 op/s 2024-05-27T07:53:07.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:07 smithi063 ceph-mon[31613]: pgmap v615: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.5 MiB/s wr, 28 op/s 2024-05-27T07:53:08.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:08 smithi086 ceph-mon[34687]: osdmap e120: 12 total, 12 up, 12 in 2024-05-27T07:53:08.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:08 smithi149 ceph-mon[35408]: osdmap e120: 12 total, 12 up, 12 in 2024-05-27T07:53:08.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:08 smithi063 ceph-mon[31613]: osdmap e120: 12 total, 12 up, 12 in 2024-05-27T07:53:09.360 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:09 smithi086 ceph-mon[34687]: pgmap v617: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.9 MiB/s wr, 68 op/s 2024-05-27T07:53:09.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:09 smithi149 ceph-mon[35408]: pgmap v617: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.9 MiB/s wr, 68 op/s 2024-05-27T07:53:09.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:09 smithi063 ceph-mon[31613]: pgmap v617: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.9 MiB/s wr, 68 op/s 2024-05-27T07:53:11.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:11 smithi086 ceph-mon[34687]: pgmap v618: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.9 MiB/s wr, 68 op/s 2024-05-27T07:53:11.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:11 smithi149 ceph-mon[35408]: pgmap v618: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.9 MiB/s wr, 68 op/s 2024-05-27T07:53:11.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:11 smithi063 ceph-mon[31613]: pgmap v618: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.9 MiB/s wr, 68 op/s 2024-05-27T07:53:12.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:53:12.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:53:12.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:53:12.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:53:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:53:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:53:13.357 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:13 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:53:13.357 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:13 smithi063 ceph-mon[31613]: pgmap v619: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.5 MiB/s wr, 47 op/s 2024-05-27T07:53:13.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:13 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:53:13.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:13 smithi086 ceph-mon[34687]: pgmap v619: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.5 MiB/s wr, 47 op/s 2024-05-27T07:53:13.466 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:13 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:53:13.466 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:13 smithi149 ceph-mon[35408]: pgmap v619: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.5 MiB/s wr, 47 op/s 2024-05-27T07:53:15.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:15 smithi086 ceph-mon[34687]: pgmap v620: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.5 MiB/s wr, 47 op/s 2024-05-27T07:53:15.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:15 smithi149 ceph-mon[35408]: pgmap v620: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.5 MiB/s wr, 47 op/s 2024-05-27T07:53:15.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:15 smithi063 ceph-mon[31613]: pgmap v620: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.5 MiB/s wr, 47 op/s 2024-05-27T07:53:16.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:16 smithi149 ceph-mon[35408]: osdmap e121: 12 total, 12 up, 12 in 2024-05-27T07:53:16.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:16 smithi149 ceph-mon[35408]: pgmap v622: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-05-27T07:53:16.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:16 smithi063 ceph-mon[31613]: osdmap e121: 12 total, 12 up, 12 in 2024-05-27T07:53:16.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:16 smithi063 ceph-mon[31613]: pgmap v622: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-05-27T07:53:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:16 smithi086 ceph-mon[34687]: osdmap e121: 12 total, 12 up, 12 in 2024-05-27T07:53:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:16 smithi086 ceph-mon[34687]: pgmap v622: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.3 MiB/s wr, 5 op/s 2024-05-27T07:53:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.270+0000 7efe04a00640 1 -- 172.21.15.63:0/320441042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe00114b10 msgr2=0x7efe00116f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.270+0000 7efe04a00640 1 --2- 172.21.15.63:0/320441042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe00114b10 0x7efe00116f30 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009ca0 tx=0x7efdf802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.270+0000 7efe04a00640 1 -- 172.21.15.63:0/320441042 shutdown_connections 2024-05-27T07:53:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.270+0000 7efe04a00640 1 --2- 172.21.15.63:0/320441042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe00114b10 0x7efe00116f30 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.270+0000 7efe04a00640 1 --2- 172.21.15.63:0/320441042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe001121b0 0x7efe001145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.270+0000 7efe04a00640 1 --2- 172.21.15.63:0/320441042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe00078720 0x7efe00078b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.270+0000 7efe04a00640 1 -- 172.21.15.63:0/320441042 >> 172.21.15.63:0/320441042 conn(0x7efe001016b0 msgr2=0x7efe00103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.270+0000 7efe04a00640 1 -- 172.21.15.63:0/320441042 shutdown_connections 2024-05-27T07:53:19.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.270+0000 7efe04a00640 1 -- 172.21.15.63:0/320441042 wait complete. 2024-05-27T07:53:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.272+0000 7efe04a00640 1 Processor -- start 2024-05-27T07:53:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.272+0000 7efe04a00640 1 -- start start 2024-05-27T07:53:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.272+0000 7efe04a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe00078720 0x7efe001a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.272+0000 7efe04a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe001121b0 0x7efe001a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.272+0000 7efe04a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe00114b10 0x7efe001a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.272+0000 7efe04a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe0011a3a0 con 0x7efe001121b0 2024-05-27T07:53:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.272+0000 7efe04a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe0011a220 con 0x7efe00078720 2024-05-27T07:53:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.272+0000 7efe04a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe0011a520 con 0x7efe00114b10 2024-05-27T07:53:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe001121b0 0x7efe001a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe001121b0 0x7efe001a12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44714/0 (socket says 172.21.15.63:44714) 2024-05-27T07:53:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfd400640 1 -- 172.21.15.63:0/2665189486 learned_addr learned my addr 172.21.15.63:0/2665189486 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfde00640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe00078720 0x7efe001a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfe800640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe00114b10 0x7efe001a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfd400640 1 -- 172.21.15.63:0/2665189486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe00114b10 msgr2=0x7efe001a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfd400640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe00114b10 0x7efe001a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfd400640 1 -- 172.21.15.63:0/2665189486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe00078720 msgr2=0x7efe001a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfd400640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe00078720 0x7efe001a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfd400640 1 -- 172.21.15.63:0/2665189486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe001a5be0 con 0x7efe001121b0 2024-05-27T07:53:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.273+0000 7efdfde00640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe00078720 0x7efe001a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.274+0000 7efdfd400640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe001121b0 0x7efe001a12e0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7efdec00e9f0 tx=0x7efdec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.274+0000 7efdf6a00640 1 -- 172.21.15.63:0/2665189486 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec00ccc0 con 0x7efe001121b0 2024-05-27T07:53:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.274+0000 7efdf6a00640 1 -- 172.21.15.63:0/2665189486 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdec0054b0 con 0x7efe001121b0 2024-05-27T07:53:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.274+0000 7efe04a00640 1 -- 172.21.15.63:0/2665189486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe001a5ed0 con 0x7efe001121b0 2024-05-27T07:53:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.274+0000 7efdf6a00640 1 -- 172.21.15.63:0/2665189486 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec010430 con 0x7efe001121b0 2024-05-27T07:53:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.274+0000 7efe04a00640 1 -- 172.21.15.63:0/2665189486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe001a63d0 con 0x7efe001121b0 2024-05-27T07:53:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.278+0000 7efe0bb7c740 1 Processor -- start 2024-05-27T07:53:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.278+0000 7efe0bb7c740 1 -- start start 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.279+0000 7efdf6a00640 1 -- 172.21.15.63:0/2665189486 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efdec0046c0 con 0x7efe001121b0 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.279+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb501b0 0x556c3eef2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.279+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb272f0 0x556c3eef2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.279+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.279+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556c3ee63b90 con 0x556c3eb2f4d0 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.279+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556c3ee53730 con 0x556c3eb501b0 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.279+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556c3ee57770 con 0x556c3eb272f0 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.279+0000 7efdfd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.280+0000 7efdfe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb501b0 0x556c3eef2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.280+0000 7efdfde00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb272f0 0x556c3eef2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.280+0000 7efdfd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44716/0 (socket says 172.21.15.63:44716) 2024-05-27T07:53:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.280+0000 7efdfd400640 1 -- 172.21.15.63:0/100689100 learned_addr learned my addr 172.21.15.63:0/100689100 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.280+0000 7efdfd400640 1 -- 172.21.15.63:0/100689100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb272f0 msgr2=0x556c3eef2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.280+0000 7efdfd400640 1 --2- 172.21.15.63:0/100689100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb272f0 0x556c3eef2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.280+0000 7efdfd400640 1 -- 172.21.15.63:0/100689100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb501b0 msgr2=0x556c3eef2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.280+0000 7efdfd400640 1 --2- 172.21.15.63:0/100689100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb501b0 0x556c3eef2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.280+0000 7efdfd400640 1 -- 172.21.15.63:0/100689100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3edec8b0 con 0x556c3eb2f4d0 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.281+0000 7efdfd400640 1 --2- 172.21.15.63:0/100689100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef32e0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7efdec0e00e0 tx=0x7efdec0e6e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a428a2ba3f06ed57 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.281+0000 7efdeb000640 1 -- 172.21.15.63:0/100689100 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0f6070 con 0x556c3eb2f4d0 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.281+0000 7efdeb000640 1 -- 172.21.15.63:0/100689100 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdec0ebbf0 con 0x556c3eb2f4d0 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.281+0000 7efdeb000640 1 -- 172.21.15.63:0/100689100 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0f05e0 con 0x556c3eb2f4d0 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.281+0000 7efdf6a00640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde40b2950 0x7efde40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.281+0000 7efdf6a00640 1 -- 172.21.15.63:0/2665189486 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7efdec0e2e70 con 0x7efe001121b0 2024-05-27T07:53:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.282+0000 7efdfe800640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde40b2950 0x7efde40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.282+0000 7efe0bb7c740 1 -- 172.21.15.63:0/100689100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 msgr2=0x556c3eef32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.282+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/100689100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef32e0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7efdec0e00e0 tx=0x7efdec0e6e80 comp rx=0 tx=0).stop 2024-05-27T07:53:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.282+0000 7efe0bb7c740 1 -- 172.21.15.63:0/100689100 shutdown_connections 2024-05-27T07:53:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.282+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/100689100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef32e0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.282+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/100689100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb272f0 0x556c3eef2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.282+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/100689100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb501b0 0x556c3eef2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.282+0000 7efe0bb7c740 1 -- 172.21.15.63:0/100689100 >> 172.21.15.63:0/100689100 conn(0x556c3eb44300 msgr2=0x556c3eeef010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.283+0000 7efe0bb7c740 1 -- 172.21.15.63:0/100689100 shutdown_connections 2024-05-27T07:53:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.283+0000 7efdfe800640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde40b2950 0x7efde40b4e60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efdf8004af0 tx=0x7efdf8005da0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.283+0000 7efe0bb7c740 1 -- 172.21.15.63:0/100689100 wait complete. 2024-05-27T07:53:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.283+0000 7efe0bb7c740 1 -- start start 2024-05-27T07:53:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.283+0000 7efe0bb7c740 1 Processor -- start 2024-05-27T07:53:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.284+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3eefccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.284+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eefd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.284+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 0x556c3ef08cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.284+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556c3eb39790 con 0x556c3eb2f4d0 2024-05-27T07:53:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.284+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556c3edfbbc0 con 0x556c3eb272f0 2024-05-27T07:53:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.284+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556c3eeff990 con 0x556c3eb501b0 2024-05-27T07:53:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.284+0000 7efe0bb7c740 10 client.0 _ll_register_callbacks cb 0x556c3edd9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.284+0000 7efdfd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eefd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.284+0000 7efdfd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eefd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44720/0 (socket says 172.21.15.63:44720) 2024-05-27T07:53:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.284+0000 7efdfd400640 1 -- 172.21.15.63:0/2772347061 learned_addr learned my addr 172.21.15.63:0/2772347061 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.285+0000 7efdfde00640 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3eefccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.285+0000 7efdfe800640 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 0x556c3ef08cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.285+0000 7efdfd400640 1 -- 172.21.15.63:0/2772347061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 msgr2=0x556c3ef08cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.285+0000 7efdfd400640 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 0x556c3ef08cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.285+0000 7efdfd400640 1 -- 172.21.15.63:0/2772347061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 msgr2=0x556c3eefccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.285+0000 7efdfd400640 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3eefccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.285+0000 7efdfd400640 1 -- 172.21.15.63:0/2772347061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3eb68210 con 0x556c3eb2f4d0 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.285+0000 7efdfd400640 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eefd210 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7efdec0a9cc0 tx=0x7efdec0f1a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efec0c012463c386 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.285+0000 7efddf400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.285+0000 7efddf400640 1 -- 172.21.15.63:0/2772347061 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0a5bf0 con 0x556c3eb2f4d0 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efddf400640 1 -- 172.21.15.63:0/2772347061 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7efdec0eb9f0 con 0x556c3eb2f4d0 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2772347061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdec0ebbf0 con 0x556c3eb2f4d0 2024-05-27T07:53:19.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efddf400640 1 -- 172.21.15.63:0/2772347061 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efdec0e0630 con 0x556c3eb2f4d0 2024-05-27T07:53:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efe0bb7c740 10 client.19227 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2772347061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556c3ec34620 con 0x556c3eb2f4d0 2024-05-27T07:53:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efddf400640 1 -- 172.21.15.63:0/2772347061 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdec0c0020 con 0x556c3eb2f4d0 2024-05-27T07:53:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efddf400640 1 -- 172.21.15.63:0/2772347061 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7efdec016330 con 0x556c3eb2f4d0 2024-05-27T07:53:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efe0bb7c740 10 client.19227 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efe0bb7c740 10 client.19227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efe0bb7c740 20 client.19227 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.286+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556c3eb0ed80 0x556c3ef0eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.287+0000 7efe0bb7c740 4 client.19227 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.287+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2772347061 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c3ee65ae0 con 0x556c3eb0ed80 2024-05-27T07:53:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.287+0000 7efdfde00640 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556c3eb0ed80 0x556c3ef0eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.288+0000 7efdfde00640 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556c3eb0ed80 0x556c3ef0eb60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15854605eb5ba776 server_cookie=6837122f4e3b01ba in_seq=0 out_seq=0 2024-05-27T07:53:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.288+0000 7efddf400640 10 client.19227 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.290+0000 7efddf400640 1 -- 172.21.15.63:0/2772347061 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556c3ee65ae0 con 0x556c3eb0ed80 2024-05-27T07:53:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.290+0000 7efddf400640 10 client.19227 handle_command_reply: tid=0 2024-05-27T07:53:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.290+0000 7efe0bb7c740 1 client.19227 shutdown 2024-05-27T07:53:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.290+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2772347061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 msgr2=0x556c3eefd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.290+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eefd210 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7efdec0a9cc0 tx=0x7efdec0f1a20 comp rx=0 tx=0).stop 2024-05-27T07:53:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.291+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2772347061 shutdown_connections 2024-05-27T07:53:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.291+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556c3eb0ed80 0x556c3ef0eb60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.291+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 0x556c3ef08cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.291+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eefd210 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.291+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/2772347061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3eefccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.291+0000 7efddf400640 0 client.19227 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.291+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2772347061 >> 172.21.15.63:0/2772347061 conn(0x556c3eb44300 msgr2=0x556c3eef4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.291+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2772347061 shutdown_connections 2024-05-27T07:53:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.291+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2772347061 wait complete. 2024-05-27T07:53:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.291+0000 7efe0bb7c740 20 client.19227 trim_cache size 0 max 16384 2024-05-27T07:53:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.309+0000 7efe0bb7c740 1 Processor -- start 2024-05-27T07:53:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.309+0000 7efe0bb7c740 1 -- start start 2024-05-27T07:53:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.309+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3eef8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 0x556c3eefd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe001ba190 con 0x556c3eb2f4d0 2024-05-27T07:53:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b7a60 con 0x556c3eb272f0 2024-05-27T07:53:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b58d0 con 0x556c3eb501b0 2024-05-27T07:53:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfd400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44730/0 (socket says 172.21.15.63:44730) 2024-05-27T07:53:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfd400640 1 -- 172.21.15.63:0/2616742895 learned_addr learned my addr 172.21.15.63:0/2616742895 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfde00640 1 --2- 172.21.15.63:0/2616742895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3eef8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfe800640 1 --2- 172.21.15.63:0/2616742895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 0x556c3eefd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfd400640 1 -- 172.21.15.63:0/2616742895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 msgr2=0x556c3eefd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfd400640 1 --2- 172.21.15.63:0/2616742895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 0x556c3eefd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfd400640 1 -- 172.21.15.63:0/2616742895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 msgr2=0x556c3eef8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfd400640 1 --2- 172.21.15.63:0/2616742895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3eef8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfd400640 1 -- 172.21.15.63:0/2616742895 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3eef3940 con 0x556c3eb2f4d0 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.310+0000 7efdfe800640 1 --2- 172.21.15.63:0/2616742895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 0x556c3eefd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.311+0000 7efdfd400640 1 --2- 172.21.15.63:0/2616742895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef27e0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7efdec0e63c0 tx=0x7efdec0e6e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655669d80325d0a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.311+0000 7efdea600640 1 -- 172.21.15.63:0/2616742895 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0a94a0 con 0x556c3eb2f4d0 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.311+0000 7efdea600640 1 -- 172.21.15.63:0/2616742895 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdec0c0070 con 0x556c3eb2f4d0 2024-05-27T07:53:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.311+0000 7efdea600640 1 -- 172.21.15.63:0/2616742895 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0a9680 con 0x556c3eb2f4d0 2024-05-27T07:53:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.312+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2616742895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 msgr2=0x556c3eef27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.312+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/2616742895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef27e0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7efdec0e63c0 tx=0x7efdec0e6e80 comp rx=0 tx=0).stop 2024-05-27T07:53:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.312+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2616742895 shutdown_connections 2024-05-27T07:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.312+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/2616742895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb501b0 0x556c3eefd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.312+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/2616742895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb2f4d0 0x556c3eef27e0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.312+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/2616742895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3eef8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.312+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2616742895 >> 172.21.15.63:0/2616742895 conn(0x556c3eb44300 msgr2=0x556c3eefec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.312+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2616742895 shutdown_connections 2024-05-27T07:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.313+0000 7efe0bb7c740 1 -- 172.21.15.63:0/2616742895 wait complete. 2024-05-27T07:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.313+0000 7efe0bb7c740 1 -- start start 2024-05-27T07:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.313+0000 7efe0bb7c740 1 Processor -- start 2024-05-27T07:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.314+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3ef0d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.314+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb2f4d0 0x556c3ef0d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.314+0000 7efe0bb7c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb501b0 0x556c3ef74f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.314+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b5f20 con 0x556c3eb501b0 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.314+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b7390 con 0x556c3eb272f0 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.314+0000 7efe0bb7c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b65f0 con 0x556c3eb2f4d0 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.314+0000 7efe0bb7c740 10 client.0 _ll_register_callbacks cb 0x7efdec0f4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.315+0000 7efdfe800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb501b0 0x556c3ef74f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.315+0000 7efdfe800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb501b0 0x556c3ef74f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44744/0 (socket says 172.21.15.63:44744) 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.315+0000 7efdfe800640 1 -- 172.21.15.63:0/4174620532 learned_addr learned my addr 172.21.15.63:0/4174620532 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.315+0000 7efdfde00640 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3ef0d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.315+0000 7efdfd400640 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb2f4d0 0x556c3ef0d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.315+0000 7efdfe800640 1 -- 172.21.15.63:0/4174620532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb2f4d0 msgr2=0x556c3ef0d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.315+0000 7efdfe800640 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb2f4d0 0x556c3ef0d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.315+0000 7efdfe800640 1 -- 172.21.15.63:0/4174620532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 msgr2=0x556c3ef0d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.315+0000 7efdfe800640 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3ef0d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.315+0000 7efdfe800640 1 -- 172.21.15.63:0/4174620532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3ef754d0 con 0x556c3eb501b0 2024-05-27T07:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.316+0000 7efdfe800640 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb501b0 0x556c3ef74f90 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009ca0 tx=0x7efdf802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1139dcba16ad4f16 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.316+0000 7efddf400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.316+0000 7efddf400640 1 -- 172.21.15.63:0/4174620532 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8057060 con 0x556c3eb501b0 2024-05-27T07:53:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.316+0000 7efddf400640 1 -- 172.21.15.63:0/4174620532 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7efdf8032de0 con 0x556c3eb501b0 2024-05-27T07:53:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.316+0000 7efe0bb7c740 1 -- 172.21.15.63:0/4174620532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdec0c0070 con 0x556c3eb501b0 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.316+0000 7efddf400640 1 -- 172.21.15.63:0/4174620532 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efdf805e050 con 0x556c3eb501b0 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.316+0000 7efe0bb7c740 10 client.19233 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.316+0000 7efe0bb7c740 1 -- 172.21.15.63:0/4174620532 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556c3ef766b0 con 0x556c3eb501b0 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.317+0000 7efddf400640 1 -- 172.21.15.63:0/4174620532 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdec0c0070 con 0x556c3eb501b0 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.317+0000 7efddf400640 1 -- 172.21.15.63:0/4174620532 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7efdf80308c0 con 0x556c3eb501b0 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.317+0000 7efe0bb7c740 10 client.19233 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.317+0000 7efe0bb7c740 10 client.19233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.317+0000 7efe0bb7c740 20 client.19233 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.317+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556c3eae41c0 0x556c3ef78ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.317+0000 7efe0bb7c740 4 client.19233 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.317+0000 7efe0bb7c740 1 -- 172.21.15.63:0/4174620532 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c3ee5f2c0 con 0x556c3eae41c0 2024-05-27T07:53:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.318+0000 7efdfde00640 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556c3eae41c0 0x556c3ef78ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.319+0000 7efdfde00640 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556c3eae41c0 0x556c3ef78ea0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9d8c6804668c50 server_cookie=f709070a873a3811 in_seq=0 out_seq=0 2024-05-27T07:53:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.319+0000 7efddf400640 10 client.19233 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efddf400640 1 -- 172.21.15.63:0/4174620532 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556c3ee5f2c0 con 0x556c3eae41c0 2024-05-27T07:53:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efddf400640 10 client.19233 handle_command_reply: tid=0 2024-05-27T07:53:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 client.19233 shutdown 2024-05-27T07:53:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 -- 172.21.15.63:0/4174620532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb501b0 msgr2=0x556c3ef74f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb501b0 0x556c3ef74f90 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009ca0 tx=0x7efdf802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 -- 172.21.15.63:0/4174620532 shutdown_connections 2024-05-27T07:53:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556c3eae41c0 0x556c3ef78ea0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556c3eb501b0 0x556c3ef74f90 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556c3eb2f4d0 0x556c3ef0d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 --2- 172.21.15.63:0/4174620532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c3eb272f0 0x556c3ef0d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efddf400640 0 client.19233 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 -- 172.21.15.63:0/4174620532 >> 172.21.15.63:0/4174620532 conn(0x556c3eb44300 msgr2=0x556c3eeed320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 -- 172.21.15.63:0/4174620532 shutdown_connections 2024-05-27T07:53:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.359+0000 7efe0bb7c740 1 -- 172.21.15.63:0/4174620532 wait complete. 2024-05-27T07:53:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.360+0000 7efe0bb7c740 20 client.19233 trim_cache size 0 max 16384 2024-05-27T07:53:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:53:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:53:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:53:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 -- 172.21.15.63:0/2665189486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde40b2950 msgr2=0x7efde40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde40b2950 0x7efde40b4e60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efdf8004af0 tx=0x7efdf8005da0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 -- 172.21.15.63:0/2665189486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe001121b0 msgr2=0x7efe001a12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe001121b0 0x7efe001a12e0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7efdec00e9f0 tx=0x7efdec00eeb0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 -- 172.21.15.63:0/2665189486 shutdown_connections 2024-05-27T07:53:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde40b2950 0x7efde40b4e60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe00114b10 0x7efe001a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe001121b0 0x7efe001a12e0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 --2- 172.21.15.63:0/2665189486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe00078720 0x7efe001a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 -- 172.21.15.63:0/2665189486 >> 172.21.15.63:0/2665189486 conn(0x7efe001016b0 msgr2=0x7efe0007b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 -- 172.21.15.63:0/2665189486 shutdown_connections 2024-05-27T07:53:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:19.361+0000 7efe04a00640 1 -- 172.21.15.63:0/2665189486 wait complete. 2024-05-27T07:53:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:18 smithi086 ceph-mon[34687]: pgmap v623: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:53:19.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:18 smithi149 ceph-mon[35408]: pgmap v623: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:53:19.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:18 smithi063 ceph-mon[31613]: pgmap v623: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:53:21.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:20 smithi086 ceph-mon[34687]: pgmap v624: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:53:21.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:20 smithi149 ceph-mon[35408]: pgmap v624: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:53:21.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:20 smithi063 ceph-mon[31613]: pgmap v624: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:53:23.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:22 smithi086 ceph-mon[34687]: pgmap v625: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:53:23.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:22 smithi149 ceph-mon[35408]: pgmap v625: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:53:23.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:22 smithi063 ceph-mon[31613]: pgmap v625: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:53:25.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:24 smithi086 ceph-mon[34687]: pgmap v626: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:53:25.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:24 smithi149 ceph-mon[35408]: pgmap v626: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:53:25.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:24 smithi063 ceph-mon[31613]: pgmap v626: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T07:53:27.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:27 smithi086 ceph-mon[34687]: pgmap v627: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T07:53:27.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:27 smithi149 ceph-mon[35408]: pgmap v627: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T07:53:27.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:27 smithi063 ceph-mon[31613]: pgmap v627: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T07:53:29.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:29 smithi149 ceph-mon[35408]: pgmap v628: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:53:29.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:29 smithi086 ceph-mon[34687]: pgmap v628: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:53:29.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:29 smithi063 ceph-mon[31613]: pgmap v628: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:53:31.269 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:31 smithi063 ceph-mon[31613]: pgmap v629: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:53:31.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:31 smithi086 ceph-mon[34687]: pgmap v629: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:53:31.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:31 smithi149 ceph-mon[35408]: pgmap v629: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:53:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.608+0000 7fa0b9e00640 1 -- 172.21.15.63:0/2710020905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b41027c0 msgr2=0x7fa0b4102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.608+0000 7fa0b9e00640 1 --2- 172.21.15.63:0/2710020905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b41027c0 0x7fa0b4102bd0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009900 tx=0x7fa0a802a410 comp rx=0 tx=0).stop 2024-05-27T07:53:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.608+0000 7fa0b9e00640 1 -- 172.21.15.63:0/2710020905 shutdown_connections 2024-05-27T07:53:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.608+0000 7fa0b9e00640 1 --2- 172.21.15.63:0/2710020905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b4110360 0x7fa0b4112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.608+0000 7fa0b9e00640 1 --2- 172.21.15.63:0/2710020905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b410da00 0x7fa0b410fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.608+0000 7fa0b9e00640 1 --2- 172.21.15.63:0/2710020905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b41027c0 0x7fa0b4102bd0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.608+0000 7fa0b9e00640 1 -- 172.21.15.63:0/2710020905 >> 172.21.15.63:0/2710020905 conn(0x7fa0b40fe210 msgr2=0x7fa0b4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.608+0000 7fa0b9e00640 1 -- 172.21.15.63:0/2710020905 shutdown_connections 2024-05-27T07:53:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.608+0000 7fa0b9e00640 1 -- 172.21.15.63:0/2710020905 wait complete. 2024-05-27T07:53:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.609+0000 7fa0b9e00640 1 Processor -- start 2024-05-27T07:53:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.610+0000 7fa0b9e00640 1 -- start start 2024-05-27T07:53:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.610+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b41027c0 0x7fa0b419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.610+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b410da00 0x7fa0b419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.610+0000 7fa0b9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b4110360 0x7fa0b41a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.610+0000 7fa0b9e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4115d20 con 0x7fa0b4110360 2024-05-27T07:53:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.610+0000 7fa0b9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4115ba0 con 0x7fa0b41027c0 2024-05-27T07:53:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.610+0000 7fa0b9e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4115ea0 con 0x7fa0b410da00 2024-05-27T07:53:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b4110360 0x7fa0b41a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b4110360 0x7fa0b41a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48844/0 (socket says 172.21.15.63:48844) 2024-05-27T07:53:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b9400640 1 -- 172.21.15.63:0/1120443832 learned_addr learned my addr 172.21.15.63:0/1120443832 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b41027c0 0x7fa0b419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b410da00 0x7fa0b419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b9400640 1 -- 172.21.15.63:0/1120443832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b410da00 msgr2=0x7fa0b419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b9400640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b410da00 0x7fa0b419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b9400640 1 -- 172.21.15.63:0/1120443832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b41027c0 msgr2=0x7fa0b419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b9400640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b41027c0 0x7fa0b419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b9400640 1 -- 172.21.15.63:0/1120443832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b41a1620 con 0x7fa0b4110360 2024-05-27T07:53:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b41027c0 0x7fa0b419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.611+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b410da00 0x7fa0b419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.612+0000 7fa0b9400640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b4110360 0x7fa0b41a0fe0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00eaa0 tx=0x7fa0ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.612+0000 7fa0b1600640 1 -- 172.21.15.63:0/1120443832 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00cdb0 con 0x7fa0b4110360 2024-05-27T07:53:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.612+0000 7fa0b1600640 1 -- 172.21.15.63:0/1120443832 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0ac005020 con 0x7fa0b4110360 2024-05-27T07:53:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.612+0000 7fa0b9e00640 1 -- 172.21.15.63:0/1120443832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b41a1910 con 0x7fa0b4110360 2024-05-27T07:53:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.612+0000 7fa0b1600640 1 -- 172.21.15.63:0/1120443832 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac0116d0 con 0x7fa0b4110360 2024-05-27T07:53:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.612+0000 7fa0b9e00640 1 -- 172.21.15.63:0/1120443832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b41a9290 con 0x7fa0b4110360 2024-05-27T07:53:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.616+0000 7fa0c2462740 1 Processor -- start 2024-05-27T07:53:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.617+0000 7fa0b1600640 1 -- 172.21.15.63:0/1120443832 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa0ac0051d0 con 0x7fa0b4110360 2024-05-27T07:53:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.617+0000 7fa0c2462740 1 -- start start 2024-05-27T07:53:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.617+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e8551b0 0x55dc0ebf7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.617+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e82c2f0 0x55dc0ebf7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.617+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ebf84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.617+0000 7fa0c2462740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dc0eb68b90 con 0x55dc0e8344d0 2024-05-27T07:53:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.617+0000 7fa0c2462740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc0eb58730 con 0x55dc0e8551b0 2024-05-27T07:53:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.617+0000 7fa0c2462740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dc0eb5c770 con 0x55dc0e82c2f0 2024-05-27T07:53:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.618+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ebf84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.618+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ebf84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48850/0 (socket says 172.21.15.63:48850) 2024-05-27T07:53:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.618+0000 7fa0b3e00640 1 -- 172.21.15.63:0/4072363953 learned_addr learned my addr 172.21.15.63:0/4072363953 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.618+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/4072363953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e8551b0 0x55dc0ebf7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.618+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/4072363953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e82c2f0 0x55dc0ebf7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.619+0000 7fa0b3e00640 1 -- 172.21.15.63:0/4072363953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e82c2f0 msgr2=0x55dc0ebf7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.619+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/4072363953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e82c2f0 0x55dc0ebf7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.619+0000 7fa0b3e00640 1 -- 172.21.15.63:0/4072363953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e8551b0 msgr2=0x55dc0ebf7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.619+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/4072363953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e8551b0 0x55dc0ebf7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.619+0000 7fa0b3e00640 1 -- 172.21.15.63:0/4072363953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc0eaf18b0 con 0x55dc0e8344d0 2024-05-27T07:53:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.619+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/4072363953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ebf84e0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x55dc0ebf4bd0 tx=0x7fa0a000c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebad8677ec0ec2f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.619+0000 7fa0a5c00640 1 -- 172.21.15.63:0/4072363953 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00142d0 con 0x55dc0e8344d0 2024-05-27T07:53:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.619+0000 7fa0a5c00640 1 -- 172.21.15.63:0/4072363953 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0a00054b0 con 0x55dc0e8344d0 2024-05-27T07:53:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.619+0000 7fa0a5c00640 1 -- 172.21.15.63:0/4072363953 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000fe00 con 0x55dc0e8344d0 2024-05-27T07:53:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.619+0000 7fa0b1600640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa09c0b29d0 0x7fa09c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.620+0000 7fa0b1600640 1 -- 172.21.15.63:0/1120443832 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fa0ac0e2ba0 con 0x7fa0b4110360 2024-05-27T07:53:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.620+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa09c0b29d0 0x7fa09c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.621+0000 7fa0c2462740 1 -- 172.21.15.63:0/4072363953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 msgr2=0x55dc0ebf84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.621+0000 7fa0c2462740 1 --2- 172.21.15.63:0/4072363953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ebf84e0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x55dc0ebf4bd0 tx=0x7fa0a000c570 comp rx=0 tx=0).stop 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.621+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa09c0b29d0 0x7fa09c0b4ee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8000c00 tx=0x7fa0a8009560 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.621+0000 7fa0c2462740 1 -- 172.21.15.63:0/4072363953 shutdown_connections 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.621+0000 7fa0c2462740 1 --2- 172.21.15.63:0/4072363953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ebf84e0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.621+0000 7fa0c2462740 1 --2- 172.21.15.63:0/4072363953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e82c2f0 0x55dc0ebf7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.621+0000 7fa0c2462740 1 --2- 172.21.15.63:0/4072363953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e8551b0 0x55dc0ebf7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.621+0000 7fa0c2462740 1 -- 172.21.15.63:0/4072363953 >> 172.21.15.63:0/4072363953 conn(0x55dc0e849300 msgr2=0x55dc0ebf4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.621+0000 7fa0c2462740 1 -- 172.21.15.63:0/4072363953 shutdown_connections 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.621+0000 7fa0c2462740 1 -- 172.21.15.63:0/4072363953 wait complete. 2024-05-27T07:53:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.622+0000 7fa0c2462740 1 -- start start 2024-05-27T07:53:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.622+0000 7fa0c2462740 1 Processor -- start 2024-05-27T07:53:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.622+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ec01c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.622+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.622+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 0x55dc0ec0de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.622+0000 7fa0c2462740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dc0e83e790 con 0x55dc0e8344d0 2024-05-27T07:53:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.622+0000 7fa0c2462740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc0eb00bc0 con 0x55dc0e82c2f0 2024-05-27T07:53:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.622+0000 7fa0c2462740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dc0ec04b30 con 0x55dc0e8551b0 2024-05-27T07:53:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.623+0000 7fa0c2462740 10 client.0 _ll_register_callbacks cb 0x55dc0eade400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.623+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.623+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48854/0 (socket says 172.21.15.63:48854) 2024-05-27T07:53:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.623+0000 7fa0b3e00640 1 -- 172.21.15.63:0/1004859833 learned_addr learned my addr 172.21.15.63:0/1004859833 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.623+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ec01c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.623+0000 7fa0b9400640 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 0x55dc0ec0de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.624+0000 7fa0b3e00640 1 -- 172.21.15.63:0/1004859833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 msgr2=0x55dc0ec0de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.624+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 0x55dc0ec0de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.624+0000 7fa0b3e00640 1 -- 172.21.15.63:0/1004859833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 msgr2=0x55dc0ec01c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.624+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ec01c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.624+0000 7fa0b3e00640 1 -- 172.21.15.63:0/1004859833 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc0e86d210 con 0x55dc0e8344d0 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.624+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec021c0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000f730 tx=0x7fa0a000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f79db6cd4838ca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.624+0000 7fa09a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.625+0000 7fa09a000640 1 -- 172.21.15.63:0/1004859833 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000fd80 con 0x55dc0e8344d0 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.625+0000 7fa09a000640 1 -- 172.21.15.63:0/1004859833 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fa0a0012450 con 0x55dc0e8344d0 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.625+0000 7fa0c2462740 1 -- 172.21.15.63:0/1004859833 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a00054b0 con 0x55dc0e8344d0 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.625+0000 7fa09a000640 1 -- 172.21.15.63:0/1004859833 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa0a0012650 con 0x55dc0e8344d0 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.625+0000 7fa0c2462740 10 client.19245 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.625+0000 7fa0c2462740 1 -- 172.21.15.63:0/1004859833 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dc0e939620 con 0x55dc0e8344d0 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.625+0000 7fa09a000640 1 -- 172.21.15.63:0/1004859833 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0a00054b0 con 0x55dc0e8344d0 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.626+0000 7fa09a000640 1 -- 172.21.15.63:0/1004859833 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa0a001f640 con 0x55dc0e8344d0 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.626+0000 7fa0c2462740 10 client.19245 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.626+0000 7fa0c2462740 10 client.19245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.626+0000 7fa0c2462740 20 client.19245 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.626+0000 7fa0c2462740 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc0e813d80 0x55dc0ec13d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.626+0000 7fa0c2462740 4 client.19245 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.626+0000 7fa0c2462740 1 -- 172.21.15.63:0/1004859833 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc0eb6aae0 con 0x55dc0e813d80 2024-05-27T07:53:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.627+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc0e813d80 0x55dc0ec13d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.628+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc0e813d80 0x55dc0ec13d10 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb0a97d45cd9e81 server_cookie=be7efe568cdedabe in_seq=0 out_seq=0 2024-05-27T07:53:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.628+0000 7fa09a000640 10 client.19245 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.630+0000 7fa09a000640 1 -- 172.21.15.63:0/1004859833 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dc0eb6aae0 con 0x55dc0e813d80 2024-05-27T07:53:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.630+0000 7fa09a000640 10 client.19245 handle_command_reply: tid=0 2024-05-27T07:53:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.630+0000 7fa0c2462740 1 client.19245 shutdown 2024-05-27T07:53:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.630+0000 7fa0c2462740 1 -- 172.21.15.63:0/1004859833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 msgr2=0x55dc0ec021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.630+0000 7fa0c2462740 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec021c0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000f730 tx=0x7fa0a000cdb0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.630+0000 7fa0c2462740 1 -- 172.21.15.63:0/1004859833 shutdown_connections 2024-05-27T07:53:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.630+0000 7fa0c2462740 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc0e813d80 0x55dc0ec13d10 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.630+0000 7fa0c2462740 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 0x55dc0ec0de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.631+0000 7fa0c2462740 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec021c0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.631+0000 7fa09a000640 0 client.19245 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.631+0000 7fa0c2462740 1 --2- 172.21.15.63:0/1004859833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ec01c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.631+0000 7fa0c2462740 1 -- 172.21.15.63:0/1004859833 >> 172.21.15.63:0/1004859833 conn(0x55dc0e849300 msgr2=0x55dc0ebf9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.631+0000 7fa0c2462740 1 -- 172.21.15.63:0/1004859833 shutdown_connections 2024-05-27T07:53:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.631+0000 7fa0c2462740 1 -- 172.21.15.63:0/1004859833 wait complete. 2024-05-27T07:53:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.631+0000 7fa0c2462740 20 client.19245 trim_cache size 0 max 16384 2024-05-27T07:53:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.655+0000 7fa0c2462740 1 Processor -- start 2024-05-27T07:53:32.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.655+0000 7fa0c2462740 1 -- start start 2024-05-27T07:53:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.656+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ebfd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.656+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec043b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.656+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 0x55dc0ebf8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.656+0000 7fa0c2462740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41b5bd0 con 0x55dc0e8344d0 2024-05-27T07:53:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.656+0000 7fa0c2462740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41b34a0 con 0x55dc0e82c2f0 2024-05-27T07:53:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.656+0000 7fa0c2462740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41b1310 con 0x55dc0e8551b0 2024-05-27T07:53:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.656+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.656+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec043b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48860/0 (socket says 172.21.15.63:48860) 2024-05-27T07:53:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.656+0000 7fa0b3e00640 1 -- 172.21.15.63:0/2783864175 learned_addr learned my addr 172.21.15.63:0/2783864175 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.656+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/2783864175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ebfd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.657+0000 7fa0b9400640 1 --2- 172.21.15.63:0/2783864175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 0x55dc0ebf8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.657+0000 7fa0b3e00640 1 -- 172.21.15.63:0/2783864175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 msgr2=0x55dc0ebf8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.657+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/2783864175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 0x55dc0ebf8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.657+0000 7fa0b3e00640 1 -- 172.21.15.63:0/2783864175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 msgr2=0x55dc0ebfd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.657+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/2783864175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ebfd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.657+0000 7fa0b3e00640 1 -- 172.21.15.63:0/2783864175 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc0ebf77f0 con 0x55dc0e8344d0 2024-05-27T07:53:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.657+0000 7fa0b9400640 1 --2- 172.21.15.63:0/2783864175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 0x55dc0ebf8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.657+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/2783864175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec043b0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000fa50 tx=0x7fa0a000c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b610de40c29e49bc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.657+0000 7fa0a5200640 1 -- 172.21.15.63:0/2783864175 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000faf0 con 0x55dc0e8344d0 2024-05-27T07:53:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.658+0000 7fa0a5200640 1 -- 172.21.15.63:0/2783864175 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0a0023030 con 0x55dc0e8344d0 2024-05-27T07:53:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.658+0000 7fa0a5200640 1 -- 172.21.15.63:0/2783864175 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0012c50 con 0x55dc0e8344d0 2024-05-27T07:53:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.659+0000 7fa0c2462740 1 -- 172.21.15.63:0/2783864175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 msgr2=0x55dc0ec043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.659+0000 7fa0c2462740 1 --2- 172.21.15.63:0/2783864175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec043b0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000fa50 tx=0x7fa0a000c4d0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.659+0000 7fa0c2462740 1 -- 172.21.15.63:0/2783864175 shutdown_connections 2024-05-27T07:53:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.659+0000 7fa0c2462740 1 --2- 172.21.15.63:0/2783864175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8551b0 0x55dc0ebf8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.659+0000 7fa0c2462740 1 --2- 172.21.15.63:0/2783864175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8344d0 0x55dc0ec043b0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.659+0000 7fa0c2462740 1 --2- 172.21.15.63:0/2783864175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ebfd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.659+0000 7fa0c2462740 1 -- 172.21.15.63:0/2783864175 >> 172.21.15.63:0/2783864175 conn(0x55dc0e849300 msgr2=0x55dc0ebf2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.659+0000 7fa0c2462740 1 -- 172.21.15.63:0/2783864175 shutdown_connections 2024-05-27T07:53:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.659+0000 7fa0c2462740 1 -- 172.21.15.63:0/2783864175 wait complete. 2024-05-27T07:53:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.660+0000 7fa0c2462740 1 -- start start 2024-05-27T07:53:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.660+0000 7fa0c2462740 1 Processor -- start 2024-05-27T07:53:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.660+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ec11e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.660+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8344d0 0x55dc0ec12340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.660+0000 7fa0c2462740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8551b0 0x55dc0ec7a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.660+0000 7fa0c2462740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41b1960 con 0x55dc0e8551b0 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.660+0000 7fa0c2462740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41b2dd0 con 0x55dc0e82c2f0 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.660+0000 7fa0c2462740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b41b2030 con 0x55dc0e8344d0 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0c2462740 10 client.0 _ll_register_callbacks cb 0x7fa0a0014470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8551b0 0x55dc0ec7a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8551b0 0x55dc0ec7a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48868/0 (socket says 172.21.15.63:48868) 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b9400640 1 -- 172.21.15.63:0/734508445 learned_addr learned my addr 172.21.15.63:0/734508445 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ec11e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b3e00640 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8344d0 0x55dc0ec12340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b9400640 1 -- 172.21.15.63:0/734508445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8344d0 msgr2=0x55dc0ec12340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b9400640 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8344d0 0x55dc0ec12340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b9400640 1 -- 172.21.15.63:0/734508445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 msgr2=0x55dc0ec11e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b9400640 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ec11e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b9400640 1 -- 172.21.15.63:0/734508445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc0ec12c10 con 0x55dc0e8551b0 2024-05-27T07:53:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.661+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ec11e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.662+0000 7fa0b9400640 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8551b0 0x55dc0ec7a140 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac005bc0 tx=0x7fa0ac0a5260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d519169b73924d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.662+0000 7fa09a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.662+0000 7fa09a000640 1 -- 172.21.15.63:0/734508445 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac01f040 con 0x55dc0e8551b0 2024-05-27T07:53:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.662+0000 7fa09a000640 1 -- 172.21.15.63:0/734508445 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fa0ac0a8ed0 con 0x55dc0e8551b0 2024-05-27T07:53:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.662+0000 7fa0c2462740 1 -- 172.21.15.63:0/734508445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a0023030 con 0x55dc0e8551b0 2024-05-27T07:53:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.662+0000 7fa09a000640 1 -- 172.21.15.63:0/734508445 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa0ac0aa160 con 0x55dc0e8551b0 2024-05-27T07:53:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.662+0000 7fa0c2462740 10 client.19251 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.662+0000 7fa0c2462740 1 -- 172.21.15.63:0/734508445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dc0ec7b6e0 con 0x55dc0e8551b0 2024-05-27T07:53:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.663+0000 7fa09a000640 1 -- 172.21.15.63:0/734508445 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0a0023030 con 0x55dc0e8551b0 2024-05-27T07:53:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.663+0000 7fa09a000640 1 -- 172.21.15.63:0/734508445 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa0ac00f040 con 0x55dc0e8551b0 2024-05-27T07:53:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.663+0000 7fa0c2462740 10 client.19251 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.663+0000 7fa0c2462740 10 client.19251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.663+0000 7fa0c2462740 20 client.19251 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.663+0000 7fa0c2462740 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc0e7e91c0 0x55dc0ec7de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.663+0000 7fa0c2462740 4 client.19251 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:53:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.663+0000 7fa0c2462740 1 -- 172.21.15.63:0/734508445 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc0eb642c0 con 0x55dc0e7e91c0 2024-05-27T07:53:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.664+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc0e7e91c0 0x55dc0ec7de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.665+0000 7fa0b8a00640 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc0e7e91c0 0x55dc0ec7de90 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b752369e8fb9ae8c server_cookie=f8858b4c2ce94e8 in_seq=0 out_seq=0 2024-05-27T07:53:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.665+0000 7fa09a000640 10 client.19251 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.704+0000 7fa09a000640 1 -- 172.21.15.63:0/734508445 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dc0eb642c0 con 0x55dc0e7e91c0 2024-05-27T07:53:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.704+0000 7fa09a000640 10 client.19251 handle_command_reply: tid=0 2024-05-27T07:53:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.704+0000 7fa0c2462740 1 client.19251 shutdown 2024-05-27T07:53:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.704+0000 7fa0c2462740 1 -- 172.21.15.63:0/734508445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8551b0 msgr2=0x55dc0ec7a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.704+0000 7fa0c2462740 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8551b0 0x55dc0ec7a140 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac005bc0 tx=0x7fa0ac0a5260 comp rx=0 tx=0).stop 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.705+0000 7fa0c2462740 1 -- 172.21.15.63:0/734508445 shutdown_connections 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.705+0000 7fa0c2462740 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc0e7e91c0 0x55dc0ec7de90 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.705+0000 7fa0c2462740 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc0e8551b0 0x55dc0ec7a140 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.705+0000 7fa0c2462740 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc0e8344d0 0x55dc0ec12340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.705+0000 7fa0c2462740 1 --2- 172.21.15.63:0/734508445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc0e82c2f0 0x55dc0ec11e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.705+0000 7fa0c2462740 1 -- 172.21.15.63:0/734508445 >> 172.21.15.63:0/734508445 conn(0x55dc0e849300 msgr2=0x55dc0ebf8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.705+0000 7fa09a000640 0 client.19251 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.705+0000 7fa0c2462740 1 -- 172.21.15.63:0/734508445 shutdown_connections 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.705+0000 7fa0c2462740 1 -- 172.21.15.63:0/734508445 wait complete. 2024-05-27T07:53:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.705+0000 7fa0c2462740 20 client.19251 trim_cache size 0 max 16384 2024-05-27T07:53:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:53:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:53:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:53:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 -- 172.21.15.63:0/1120443832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa09c0b29d0 msgr2=0x7fa09c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:32.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa09c0b29d0 0x7fa09c0b4ee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8000c00 tx=0x7fa0a8009560 comp rx=0 tx=0).stop 2024-05-27T07:53:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 -- 172.21.15.63:0/1120443832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b4110360 msgr2=0x7fa0b41a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b4110360 0x7fa0b41a0fe0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00eaa0 tx=0x7fa0ac00ef60 comp rx=0 tx=0).stop 2024-05-27T07:53:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 -- 172.21.15.63:0/1120443832 shutdown_connections 2024-05-27T07:53:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa09c0b29d0 0x7fa09c0b4ee0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0b4110360 0x7fa0b41a0fe0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0b410da00 0x7fa0b419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 --2- 172.21.15.63:0/1120443832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b41027c0 0x7fa0b419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 -- 172.21.15.63:0/1120443832 >> 172.21.15.63:0/1120443832 conn(0x7fa0b40fe210 msgr2=0x7fa0b4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 -- 172.21.15.63:0/1120443832 shutdown_connections 2024-05-27T07:53:32.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:32.706+0000 7fa0b9e00640 1 -- 172.21.15.63:0/1120443832 wait complete. 2024-05-27T07:53:33.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:33 smithi086 ceph-mon[34687]: pgmap v630: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:53:33.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:33 smithi149 ceph-mon[35408]: pgmap v630: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:53:33.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:33 smithi063 ceph-mon[31613]: pgmap v630: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:53:35.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:35 smithi086 ceph-mon[34687]: pgmap v631: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:53:35.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:35 smithi149 ceph-mon[35408]: pgmap v631: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:53:35.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:35 smithi063 ceph-mon[31613]: pgmap v631: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:53:37.214 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:53:37.288 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:37 smithi063 ceph-mon[31613]: pgmap v632: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:53:37.289 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:37 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:53:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:37 smithi086 ceph-mon[34687]: pgmap v632: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:53:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:37 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:53:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:37 smithi149 ceph-mon[35408]: pgmap v632: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T07:53:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:37 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:53:37.746 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:39.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:39 smithi063 ceph-mon[31613]: pgmap v633: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:53:39.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:39 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:53:39.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:39 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:53:39.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:39 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:53:39.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:39 smithi086 ceph-mon[34687]: pgmap v633: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:53:39.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:39 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:53:39.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:39 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:53:39.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:39 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:53:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:39 smithi149 ceph-mon[35408]: pgmap v633: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:53:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:39 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:53:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:39 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:53:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:39 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:53:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.376+0000 7f1c58a00700 1 -- 172.21.15.63:0/1980497550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c541043f0 msgr2=0x7f1c54104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.376+0000 7f1c58a00700 1 --2- 172.21.15.63:0/1980497550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c541043f0 0x7f1c54104800 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f1c440099c0 tx=0x7f1c44009cd0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.377+0000 7f1c58a00700 1 -- 172.21.15.63:0/1980497550 shutdown_connections 2024-05-27T07:53:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.377+0000 7f1c58a00700 1 --2- 172.21.15.63:0/1980497550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54109670 0x7f1c54110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.377+0000 7f1c58a00700 1 --2- 172.21.15.63:0/1980497550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c541043f0 0x7f1c54104800 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.377+0000 7f1c58a00700 1 --2- 172.21.15.63:0/1980497550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c54108d20 0x7f1c54109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.377+0000 7f1c58a00700 1 -- 172.21.15.63:0/1980497550 >> 172.21.15.63:0/1980497550 conn(0x7f1c54079d00 msgr2=0x7f1c5407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.377+0000 7f1c58a00700 1 -- 172.21.15.63:0/1980497550 shutdown_connections 2024-05-27T07:53:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.377+0000 7f1c58a00700 1 -- 172.21.15.63:0/1980497550 wait complete. 2024-05-27T07:53:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.378+0000 7f1c58a00700 1 Processor -- start 2024-05-27T07:53:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.378+0000 7f1c58a00700 1 -- start start 2024-05-27T07:53:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.379+0000 7f1c58a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c541043f0 0x7f1c541a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.379+0000 7f1c58a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c54108d20 0x7f1c541a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.379+0000 7f1c58a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54109670 0x7f1c5419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.379+0000 7f1c58a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541141a0 con 0x7f1c541043f0 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.379+0000 7f1c58a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54114020 con 0x7f1c54109670 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.379+0000 7f1c58a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54114320 con 0x7f1c54108d20 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.379+0000 7f1c53400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c541043f0 0x7f1c541a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.379+0000 7f1c53400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c541043f0 0x7f1c541a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46224/0 (socket says 172.21.15.63:46224) 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.379+0000 7f1c53400700 1 -- 172.21.15.63:0/4000790572 learned_addr learned my addr 172.21.15.63:0/4000790572 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.380+0000 7f1c52a00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c54108d20 0x7f1c541a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.380+0000 7f1c53e00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54109670 0x7f1c5419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.380+0000 7f1c53400700 1 -- 172.21.15.63:0/4000790572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c54108d20 msgr2=0x7f1c541a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.380+0000 7f1c53400700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c54108d20 0x7f1c541a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.380+0000 7f1c53400700 1 -- 172.21.15.63:0/4000790572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54109670 msgr2=0x7f1c5419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.380+0000 7f1c53400700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54109670 0x7f1c5419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.380+0000 7f1c53400700 1 -- 172.21.15.63:0/4000790572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5419b0b0 con 0x7f1c541043f0 2024-05-27T07:53:40.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.380+0000 7f1c53e00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54109670 0x7f1c5419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:40.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.380+0000 7f1c53400700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c541043f0 0x7f1c541a07e0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400b580 tx=0x7f1c4400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:40.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.381+0000 7f1c4be00700 1 -- 172.21.15.63:0/4000790572 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44027070 con 0x7f1c541043f0 2024-05-27T07:53:40.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.381+0000 7f1c4be00700 1 -- 172.21.15.63:0/4000790572 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c4400fdd0 con 0x7f1c541043f0 2024-05-27T07:53:40.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.381+0000 7f1c58a00700 1 -- 172.21.15.63:0/4000790572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5419b340 con 0x7f1c541043f0 2024-05-27T07:53:40.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.381+0000 7f1c58a00700 1 -- 172.21.15.63:0/4000790572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5419b820 con 0x7f1c541043f0 2024-05-27T07:53:40.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.381+0000 7f1c4be00700 1 -- 172.21.15.63:0/4000790572 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c440173f0 con 0x7f1c541043f0 2024-05-27T07:53:40.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.385+0000 7f1c4be00700 1 -- 172.21.15.63:0/4000790572 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1c44022690 con 0x7f1c541043f0 2024-05-27T07:53:40.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.387+0000 7f1c68b67b80 1 Processor -- start 2024-05-27T07:53:40.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.387+0000 7f1c68b67b80 1 -- start start 2024-05-27T07:53:40.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.388+0000 7f1c4be00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1c380b29f0 0x7f1c380b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.388+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31ccad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.388+0000 7f1c4be00700 1 -- 172.21.15.63:0/4000790572 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f1c440e3bc0 con 0x7f1c541043f0 2024-05-27T07:53:40.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.388+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb2a0 0x563f31cd9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.388+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 0x563f31cdc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.388+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563f31bf9e40 con 0x563f31ccb6b0 2024-05-27T07:53:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.388+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f31be99e0 con 0x563f31b4ce40 2024-05-27T07:53:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.388+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563f31beda20 con 0x563f31ccb2a0 2024-05-27T07:53:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.388+0000 7f1c53e00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1c380b29f0 0x7f1c380b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.389+0000 7f1c53400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31ccad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.389+0000 7f1c53400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31ccad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:33642/0 (socket says 172.21.15.63:33642) 2024-05-27T07:53:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.389+0000 7f1c53400700 1 -- 172.21.15.63:0/4051362056 learned_addr learned my addr 172.21.15.63:0/4051362056 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.389+0000 7f1c52a00700 1 --2- 172.21.15.63:0/4051362056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb2a0 0x563f31cd9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.389+0000 7f1c53e00700 1 --2- 172.21.15.63:0/4051362056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 0x563f31cdc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.389+0000 7f1c53400700 1 -- 172.21.15.63:0/4051362056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb2a0 msgr2=0x563f31cd9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:40.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.389+0000 7f1c53400700 1 --2- 172.21.15.63:0/4051362056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb2a0 0x563f31cd9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.389+0000 7f1c53400700 1 -- 172.21.15.63:0/4051362056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 msgr2=0x563f31cdc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:40.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.389+0000 7f1c53400700 1 --2- 172.21.15.63:0/4051362056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 0x563f31cdc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.389+0000 7f1c53400700 1 -- 172.21.15.63:0/4051362056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f31bb2420 con 0x563f31b4ce40 2024-05-27T07:53:40.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.390+0000 7f1c53e00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1c380b29f0 0x7f1c380b4f00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009790 tx=0x7f1c4c0079a0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:40.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.390+0000 7f1c53400700 1 --2- 172.21.15.63:0/4051362056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31ccad60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c440aaaa0 tx=0x7f1c440aa620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c1330614f61ac1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:40.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.390+0000 7f1c48c00700 1 -- 172.21.15.63:0/4051362056 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c440aab10 con 0x563f31b4ce40 2024-05-27T07:53:40.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.390+0000 7f1c48c00700 1 -- 172.21.15.63:0/4051362056 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c440aeec0 con 0x563f31b4ce40 2024-05-27T07:53:40.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.390+0000 7f1c48c00700 1 -- 172.21.15.63:0/4051362056 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c440aaf20 con 0x563f31b4ce40 2024-05-27T07:53:40.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.392+0000 7f1c68b67b80 1 -- 172.21.15.63:0/4051362056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 msgr2=0x563f31ccad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:40.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.392+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/4051362056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31ccad60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c440aaaa0 tx=0x7f1c440aa620 comp rx=0 tx=0).stop 2024-05-27T07:53:40.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.392+0000 7f1c68b67b80 1 -- 172.21.15.63:0/4051362056 shutdown_connections 2024-05-27T07:53:40.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.392+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/4051362056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 0x563f31cdc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.392+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/4051362056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31ccad60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.392+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/4051362056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb2a0 0x563f31cd9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.392+0000 7f1c68b67b80 1 -- 172.21.15.63:0/4051362056 >> 172.21.15.63:0/4051362056 conn(0x563f31bacfc0 msgr2=0x563f31cc6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:40.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.392+0000 7f1c68b67b80 1 -- 172.21.15.63:0/4051362056 shutdown_connections 2024-05-27T07:53:40.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.392+0000 7f1c68b67b80 1 -- 172.21.15.63:0/4051362056 wait complete. 2024-05-27T07:53:40.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.393+0000 7f1c68b67b80 1 -- start start 2024-05-27T07:53:40.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.393+0000 7f1c68b67b80 1 Processor -- start 2024-05-27T07:53:40.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31cd99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31cd9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 0x563f31cda440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563f31aeffa0 con 0x563f31ccb2a0 2024-05-27T07:53:40.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f31b8a760 con 0x563f31b4ce40 2024-05-27T07:53:40.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563f31cdcb60 con 0x563f31ccb6b0 2024-05-27T07:53:40.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c68b67b80 10 client.0 _ll_register_callbacks cb 0x563f31b93c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:40.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c52a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31cd9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c52a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31cd9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46236/0 (socket says 172.21.15.63:46236) 2024-05-27T07:53:40.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c52a00700 1 -- 172.21.15.63:0/3404509736 learned_addr learned my addr 172.21.15.63:0/3404509736 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:40.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.394+0000 7f1c53400700 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31cd99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.395+0000 7f1c53e00700 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 0x563f31cda440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.395+0000 7f1c52a00700 1 -- 172.21.15.63:0/3404509736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 msgr2=0x563f31cda440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:40.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.395+0000 7f1c52a00700 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 0x563f31cda440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.395+0000 7f1c52a00700 1 -- 172.21.15.63:0/3404509736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 msgr2=0x563f31cd99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:40.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.395+0000 7f1c52a00700 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31cd99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.395+0000 7f1c52a00700 1 -- 172.21.15.63:0/3404509736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f31b398d0 con 0x563f31ccb2a0 2024-05-27T07:53:40.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.395+0000 7f1c53400700 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31cd99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:40.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.395+0000 7f1c52a00700 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31cd9f00 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000bf50 tx=0x7f1c400127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f844a684e515db0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:40.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.395+0000 7f1c3d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:40.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.396+0000 7f1c3d600700 1 -- 172.21.15.63:0/3404509736 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40004440 con 0x563f31ccb2a0 2024-05-27T07:53:40.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.396+0000 7f1c3d600700 1 -- 172.21.15.63:0/3404509736 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f1c40003680 con 0x563f31ccb2a0 2024-05-27T07:53:40.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.396+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3404509736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c440aeec0 con 0x563f31ccb2a0 2024-05-27T07:53:40.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.396+0000 7f1c3d600700 1 -- 172.21.15.63:0/3404509736 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1c40003880 con 0x563f31ccb2a0 2024-05-27T07:53:40.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.396+0000 7f1c68b67b80 10 client.19263 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:40.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.396+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3404509736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563f31bba060 con 0x563f31ccb2a0 2024-05-27T07:53:40.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.397+0000 7f1c3d600700 1 -- 172.21.15.63:0/3404509736 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c440aeec0 con 0x563f31ccb2a0 2024-05-27T07:53:40.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.397+0000 7f1c3d600700 1 -- 172.21.15.63:0/3404509736 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1c40019020 con 0x563f31ccb2a0 2024-05-27T07:53:40.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.397+0000 7f1c68b67b80 10 client.19263 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:40.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.397+0000 7f1c68b67b80 10 client.19263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:40.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.397+0000 7f1c68b67b80 20 client.19263 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:40.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.397+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563f31ce9a70 0x563f31cebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.397+0000 7f1c68b67b80 4 client.19263 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:40.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.397+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3404509736 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f31bfbd90 con 0x563f31ce9a70 2024-05-27T07:53:40.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.398+0000 7f1c53400700 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563f31ce9a70 0x563f31cebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.399+0000 7f1c53400700 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563f31ce9a70 0x563f31cebe90 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb373afea15f4c6 server_cookie=ba830496f53df5b in_seq=0 out_seq=0 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.399+0000 7f1c3d600700 10 client.19263 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c3d600700 1 -- 172.21.15.63:0/3404509736 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563f31bfbd90 con 0x563f31ce9a70 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c3d600700 10 client.19263 handle_command_reply: tid=0 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c68b67b80 1 client.19263 shutdown 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3404509736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 msgr2=0x563f31cd9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31cd9f00 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000bf50 tx=0x7f1c400127e0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3404509736 shutdown_connections 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563f31ce9a70 0x563f31cebe90 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 0x563f31cda440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31cd99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3404509736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31cd9f00 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3404509736 >> 172.21.15.63:0/3404509736 conn(0x563f31bacfc0 msgr2=0x563f31cc5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:40.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.401+0000 7f1c3d600700 0 client.19263 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:40.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.402+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3404509736 shutdown_connections 2024-05-27T07:53:40.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.402+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3404509736 wait complete. 2024-05-27T07:53:40.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.402+0000 7f1c68b67b80 20 client.19263 trim_cache size 0 max 16384 2024-05-27T07:53:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.427+0000 7f1c68b67b80 1 Processor -- start 2024-05-27T07:53:40.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.427+0000 7f1c68b67b80 1 -- start start 2024-05-27T07:53:40.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.428+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31cd51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.428+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31ce5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.428+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 0x563f31cf15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.428+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541b0b00 con 0x563f31ccb2a0 2024-05-27T07:53:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.428+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541ae390 con 0x563f31b4ce40 2024-05-27T07:53:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.428+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541ac200 con 0x563f31ccb6b0 2024-05-27T07:53:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.428+0000 7f1c52a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31ce5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.428+0000 7f1c52a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31ce5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46244/0 (socket says 172.21.15.63:46244) 2024-05-27T07:53:40.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.428+0000 7f1c52a00700 1 -- 172.21.15.63:0/3596496035 learned_addr learned my addr 172.21.15.63:0/3596496035 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.428+0000 7f1c53400700 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31cd51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.429+0000 7f1c53e00700 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 0x563f31cf15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.429+0000 7f1c52a00700 1 -- 172.21.15.63:0/3596496035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 msgr2=0x563f31cf15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.429+0000 7f1c52a00700 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 0x563f31cf15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.429+0000 7f1c52a00700 1 -- 172.21.15.63:0/3596496035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 msgr2=0x563f31cd51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.429+0000 7f1c52a00700 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31cd51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.429+0000 7f1c52a00700 1 -- 172.21.15.63:0/3596496035 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f31cc8350 con 0x563f31ccb2a0 2024-05-27T07:53:40.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.429+0000 7f1c53e00700 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 0x563f31cf15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.429+0000 7f1c53400700 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31cd51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.429+0000 7f1c52a00700 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31ce5bc0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000bf20 tx=0x7f1c40005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8142c60a88a7f55 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.430+0000 7f1c3fe00700 1 -- 172.21.15.63:0/3596496035 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40004240 con 0x563f31ccb2a0 2024-05-27T07:53:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.430+0000 7f1c3fe00700 1 -- 172.21.15.63:0/3596496035 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c4002d070 con 0x563f31ccb2a0 2024-05-27T07:53:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.430+0000 7f1c3fe00700 1 -- 172.21.15.63:0/3596496035 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4001b8e0 con 0x563f31ccb2a0 2024-05-27T07:53:40.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.431+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3596496035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 msgr2=0x563f31ce5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:40.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.431+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31ce5bc0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000bf20 tx=0x7f1c40005720 comp rx=0 tx=0).stop 2024-05-27T07:53:40.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.431+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3596496035 shutdown_connections 2024-05-27T07:53:40.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.431+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31ccb6b0 0x563f31cf15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.431+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31b4ce40 0x563f31cd51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.431+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3596496035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb2a0 0x563f31ce5bc0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.431+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3596496035 >> 172.21.15.63:0/3596496035 conn(0x563f31bacfc0 msgr2=0x563f31ccd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:40.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.431+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3596496035 shutdown_connections 2024-05-27T07:53:40.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.431+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3596496035 wait complete. 2024-05-27T07:53:40.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.432+0000 7f1c68b67b80 1 -- start start 2024-05-27T07:53:40.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.432+0000 7f1c68b67b80 1 Processor -- start 2024-05-27T07:53:40.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.433+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31b4ce40 0x563f31cde200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.433+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31ccb2a0 0x563f31cde740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.433+0000 7f1c68b67b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 0x563f31d53480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.433+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541ac850 con 0x563f31ccb6b0 2024-05-27T07:53:40.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.433+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541adcc0 con 0x563f31ccb2a0 2024-05-27T07:53:40.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.433+0000 7f1c68b67b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541acf20 con 0x563f31b4ce40 2024-05-27T07:53:40.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.433+0000 7f1c68b67b80 10 client.0 _ll_register_callbacks cb 0x563f31b93c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:40.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.433+0000 7f1c53e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 0x563f31d53480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.433+0000 7f1c53e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 0x563f31d53480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46246/0 (socket says 172.21.15.63:46246) 2024-05-27T07:53:40.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.433+0000 7f1c53e00700 1 -- 172.21.15.63:0/3301892303 learned_addr learned my addr 172.21.15.63:0/3301892303 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.434+0000 7f1c53400700 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31b4ce40 0x563f31cde200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.434+0000 7f1c52a00700 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31ccb2a0 0x563f31cde740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.434+0000 7f1c53e00700 1 -- 172.21.15.63:0/3301892303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31b4ce40 msgr2=0x563f31cde200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.434+0000 7f1c53e00700 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31b4ce40 0x563f31cde200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.434+0000 7f1c53e00700 1 -- 172.21.15.63:0/3301892303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31ccb2a0 msgr2=0x563f31cde740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.434+0000 7f1c53e00700 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31ccb2a0 0x563f31cde740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.434+0000 7f1c53e00700 1 -- 172.21.15.63:0/3301892303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f31d539c0 con 0x563f31ccb6b0 2024-05-27T07:53:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.435+0000 7f1c53e00700 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 0x563f31d53480 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c01bf40 tx=0x7f1c4c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964577f7fc7afb5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.435+0000 7f1c3d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.435+0000 7f1c3d600700 1 -- 172.21.15.63:0/3301892303 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c028060 con 0x563f31ccb6b0 2024-05-27T07:53:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.435+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3301892303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c4002d070 con 0x563f31ccb6b0 2024-05-27T07:53:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.435+0000 7f1c3d600700 1 -- 172.21.15.63:0/3301892303 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f1c4c003a00 con 0x563f31ccb6b0 2024-05-27T07:53:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.435+0000 7f1c3d600700 1 -- 172.21.15.63:0/3301892303 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1c4c02f050 con 0x563f31ccb6b0 2024-05-27T07:53:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.435+0000 7f1c68b67b80 10 client.19269 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.435+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3301892303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563f31d54b70 con 0x563f31ccb6b0 2024-05-27T07:53:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.436+0000 7f1c3d600700 1 -- 172.21.15.63:0/3301892303 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c4002d070 con 0x563f31ccb6b0 2024-05-27T07:53:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.436+0000 7f1c3d600700 1 -- 172.21.15.63:0/3301892303 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1c4c018070 con 0x563f31ccb6b0 2024-05-27T07:53:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.436+0000 7f1c68b67b80 10 client.19269 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.436+0000 7f1c68b67b80 10 client.19269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.436+0000 7f1c68b67b80 20 client.19269 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.436+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563f31d55200 0x563f31d57620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.437+0000 7f1c68b67b80 4 client.19269 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:53:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.437+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3301892303 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f31bf5570 con 0x563f31d55200 2024-05-27T07:53:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.437+0000 7f1c53400700 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563f31d55200 0x563f31d57620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.438+0000 7f1c3d600700 10 client.19269 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.438+0000 7f1c53400700 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563f31d55200 0x563f31d57620 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3783a80feb0f80de server_cookie=6fb78521342e1960 in_seq=0 out_seq=0 2024-05-27T07:53:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c3d600700 1 -- 172.21.15.63:0/3301892303 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563f31bf5570 con 0x563f31d55200 2024-05-27T07:53:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c3d600700 10 client.19269 handle_command_reply: tid=0 2024-05-27T07:53:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c68b67b80 1 client.19269 shutdown 2024-05-27T07:53:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3301892303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 msgr2=0x563f31d53480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 0x563f31d53480 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c01bf40 tx=0x7f1c4c004b40 comp rx=0 tx=0).stop 2024-05-27T07:53:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3301892303 shutdown_connections 2024-05-27T07:53:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563f31d55200 0x563f31d57620 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f31ccb6b0 0x563f31d53480 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f31b4ce40 0x563f31cde200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c68b67b80 1 --2- 172.21.15.63:0/3301892303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f31ccb2a0 0x563f31cde740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3301892303 >> 172.21.15.63:0/3301892303 conn(0x563f31bacfc0 msgr2=0x563f31cc5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.439+0000 7f1c3d600700 0 client.19269 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.440+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3301892303 shutdown_connections 2024-05-27T07:53:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.440+0000 7f1c68b67b80 1 -- 172.21.15.63:0/3301892303 wait complete. 2024-05-27T07:53:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.440+0000 7f1c68b67b80 20 client.19269 trim_cache size 0 max 16384 2024-05-27T07:53:40.443 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:53:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:53:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:53:40.443 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:53:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 -- 172.21.15.63:0/4000790572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1c380b29f0 msgr2=0x7f1c380b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1c380b29f0 0x7f1c380b4f00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009790 tx=0x7f1c4c0079a0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 -- 172.21.15.63:0/4000790572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c541043f0 msgr2=0x7f1c541a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c541043f0 0x7f1c541a07e0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400b580 tx=0x7f1c4400f510 comp rx=0 tx=0).stop 2024-05-27T07:53:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 -- 172.21.15.63:0/4000790572 shutdown_connections 2024-05-27T07:53:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1c380b29f0 0x7f1c380b4f00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c54109670 0x7f1c5419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c541043f0 0x7f1c541a07e0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 --2- 172.21.15.63:0/4000790572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c54108d20 0x7f1c541a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 -- 172.21.15.63:0/4000790572 >> 172.21.15.63:0/4000790572 conn(0x7f1c54079d00 msgr2=0x7f1c54101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 -- 172.21.15.63:0/4000790572 shutdown_connections 2024-05-27T07:53:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:40.441+0000 7f1c58a00700 1 -- 172.21.15.63:0/4000790572 wait complete. 2024-05-27T07:53:41.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:41 smithi063 ceph-mon[31613]: pgmap v634: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:53:41.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:41 smithi086 ceph-mon[34687]: pgmap v634: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:53:41.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:41 smithi149 ceph-mon[35408]: pgmap v634: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:53:41.478 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:53:41.478 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:53:41.983 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:43.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:43 smithi086 ceph-mon[34687]: pgmap v635: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:53:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:43 smithi149 ceph-mon[35408]: pgmap v635: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:53:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:43 smithi063 ceph-mon[31613]: pgmap v635: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:53:45.308 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:45 smithi063 ceph-mon[31613]: pgmap v636: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:53:45.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:45 smithi086 ceph-mon[34687]: pgmap v636: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:53:45.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:45 smithi149 ceph-mon[35408]: pgmap v636: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:53:45.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.707+0000 7f8682600700 1 -- 172.21.15.63:0/2447909497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f867c109670 msgr2=0x7f867c110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:45.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.707+0000 7f8682600700 1 --2- 172.21.15.63:0/2447909497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f867c109670 0x7f867c110fd0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f867400b0d0 tx=0x7f867400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.707+0000 7f8682600700 1 -- 172.21.15.63:0/2447909497 shutdown_connections 2024-05-27T07:53:45.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.707+0000 7f8682600700 1 --2- 172.21.15.63:0/2447909497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f867c109670 0x7f867c110fd0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.707+0000 7f8682600700 1 --2- 172.21.15.63:0/2447909497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f867c1043f0 0x7f867c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.707+0000 7f8682600700 1 --2- 172.21.15.63:0/2447909497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f867c108d20 0x7f867c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.707+0000 7f8682600700 1 -- 172.21.15.63:0/2447909497 >> 172.21.15.63:0/2447909497 conn(0x7f867c079d00 msgr2=0x7f867c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:45.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.707+0000 7f8682600700 1 -- 172.21.15.63:0/2447909497 shutdown_connections 2024-05-27T07:53:45.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.707+0000 7f8682600700 1 -- 172.21.15.63:0/2447909497 wait complete. 2024-05-27T07:53:45.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.709+0000 7f8682600700 1 Processor -- start 2024-05-27T07:53:45.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.709+0000 7f8682600700 1 -- start start 2024-05-27T07:53:45.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.710+0000 7f8682600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f867c1043f0 0x7f867c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.710+0000 7f8682600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f867c108d20 0x7f867c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.710+0000 7f8682600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f867c109670 0x7f867c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.710+0000 7f8682600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f867c1141a0 con 0x7f867c108d20 2024-05-27T07:53:45.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.710+0000 7f8682600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f867c114020 con 0x7f867c1043f0 2024-05-27T07:53:45.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.710+0000 7f8682600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f867c114320 con 0x7f867c109670 2024-05-27T07:53:45.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.710+0000 7f867be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f867c108d20 0x7f867c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.710+0000 7f867be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f867c108d20 0x7f867c1a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46252/0 (socket says 172.21.15.63:46252) 2024-05-27T07:53:45.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.710+0000 7f867be00700 1 -- 172.21.15.63:0/1827265693 learned_addr learned my addr 172.21.15.63:0/1827265693 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:45.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.710+0000 7f8681200700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f867c1043f0 0x7f867c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.711+0000 7f8681c00700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f867c109670 0x7f867c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.711+0000 7f867be00700 1 -- 172.21.15.63:0/1827265693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f867c109670 msgr2=0x7f867c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:45.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.711+0000 7f867be00700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f867c109670 0x7f867c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.711+0000 7f867be00700 1 -- 172.21.15.63:0/1827265693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f867c1043f0 msgr2=0x7f867c1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:45.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.711+0000 7f867be00700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f867c1043f0 0x7f867c1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.711+0000 7f867be00700 1 -- 172.21.15.63:0/1827265693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c19b090 con 0x7f867c108d20 2024-05-27T07:53:45.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.711+0000 7f867be00700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f867c108d20 0x7f867c1a0cb0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f866800d980 tx=0x7f866800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:45.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.712+0000 7f8679600700 1 -- 172.21.15.63:0/1827265693 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668017070 con 0x7f867c108d20 2024-05-27T07:53:45.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.712+0000 7f8679600700 1 -- 172.21.15.63:0/1827265693 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8668009dc0 con 0x7f867c108d20 2024-05-27T07:53:45.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.712+0000 7f8682600700 1 -- 172.21.15.63:0/1827265693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c19b380 con 0x7f867c108d20 2024-05-27T07:53:45.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.712+0000 7f8679600700 1 -- 172.21.15.63:0/1827265693 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866800f3c0 con 0x7f867c108d20 2024-05-27T07:53:45.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.712+0000 7f8682600700 1 -- 172.21.15.63:0/1827265693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c1a7600 con 0x7f867c108d20 2024-05-27T07:53:45.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.713+0000 7f8682600700 1 -- 172.21.15.63:0/1827265693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867c1058a0 con 0x7f867c108d20 2024-05-27T07:53:45.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.716+0000 7f8679600700 1 -- 172.21.15.63:0/1827265693 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8668009930 con 0x7f867c108d20 2024-05-27T07:53:45.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.718+0000 7f8679600700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86640b29f0 0x7f86640b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.718+0000 7f8679600700 1 -- 172.21.15.63:0/1827265693 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f86680e24d0 con 0x7f867c108d20 2024-05-27T07:53:45.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.718+0000 7f8681200700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86640b29f0 0x7f86640b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.719+0000 7f8681200700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86640b29f0 0x7f86640b4f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f867c102c30 tx=0x7f867000b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:45.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:45.722+0000 7f8679600700 1 -- 172.21.15.63:0/1827265693 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f86680a9a90 con 0x7f867c108d20 2024-05-27T07:53:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.964+0000 7f24b9c00640 1 -- 172.21.15.63:0/1667198265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24b410db90 msgr2=0x7f24b410ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.964+0000 7f24b9c00640 1 --2- 172.21.15.63:0/1667198265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24b410db90 0x7f24b410ffb0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009900 tx=0x7f24a002a480 comp rx=0 tx=0).stop 2024-05-27T07:53:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.965+0000 7f24b9c00640 1 -- 172.21.15.63:0/1667198265 shutdown_connections 2024-05-27T07:53:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.965+0000 7f24b9c00640 1 --2- 172.21.15.63:0/1667198265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b41104f0 0x7f24b4112910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.965+0000 7f24b9c00640 1 --2- 172.21.15.63:0/1667198265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24b410db90 0x7f24b410ffb0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.965+0000 7f24b9c00640 1 --2- 172.21.15.63:0/1667198265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24b4076510 0x7f24b4076920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.965+0000 7f24b9c00640 1 -- 172.21.15.63:0/1667198265 >> 172.21.15.63:0/1667198265 conn(0x7f24b40ff340 msgr2=0x7f24b4101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.965+0000 7f24b9c00640 1 -- 172.21.15.63:0/1667198265 shutdown_connections 2024-05-27T07:53:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.965+0000 7f24b9c00640 1 -- 172.21.15.63:0/1667198265 wait complete. 2024-05-27T07:53:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.966+0000 7f24b9c00640 1 Processor -- start 2024-05-27T07:53:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.967+0000 7f24b9c00640 1 -- start start 2024-05-27T07:53:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.967+0000 7f24b9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b4076510 0x7f24b41a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.967+0000 7f24b9c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24b410db90 0x7f24b41a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.967+0000 7f24b9c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24b41104f0 0x7f24b419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.967+0000 7f24b9c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4115ee0 con 0x7f24b410db90 2024-05-27T07:53:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.967+0000 7f24b9c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4115d60 con 0x7f24b4076510 2024-05-27T07:53:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.967+0000 7f24b9c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4116060 con 0x7f24b41104f0 2024-05-27T07:53:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.968+0000 7f24b2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24b410db90 0x7f24b41a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.968+0000 7f24b2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b4076510 0x7f24b41a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.968+0000 7f24b2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24b410db90 0x7f24b41a2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46270/0 (socket says 172.21.15.63:46270) 2024-05-27T07:53:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.968+0000 7f24b2000640 1 -- 172.21.15.63:0/3601576493 learned_addr learned my addr 172.21.15.63:0/3601576493 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.968+0000 7f24b3400640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24b41104f0 0x7f24b419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.968+0000 7f24b2000640 1 -- 172.21.15.63:0/3601576493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24b41104f0 msgr2=0x7f24b419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.968+0000 7f24b2000640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24b41104f0 0x7f24b419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.968+0000 7f24b2000640 1 -- 172.21.15.63:0/3601576493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b4076510 msgr2=0x7f24b41a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.968+0000 7f24b2000640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b4076510 0x7f24b41a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.968+0000 7f24b2000640 1 -- 172.21.15.63:0/3601576493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b419cf20 con 0x7f24b410db90 2024-05-27T07:53:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.969+0000 7f24b2000640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24b410db90 0x7f24b41a2b50 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f24a0005940 tx=0x7f24a0005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.969+0000 7f24ab400640 1 -- 172.21.15.63:0/3601576493 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a003f030 con 0x7f24b410db90 2024-05-27T07:53:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.969+0000 7f24ab400640 1 -- 172.21.15.63:0/3601576493 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24a00043d0 con 0x7f24b410db90 2024-05-27T07:53:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.969+0000 7f24b9c00640 1 -- 172.21.15.63:0/3601576493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b419d1b0 con 0x7f24b410db90 2024-05-27T07:53:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.969+0000 7f24ab400640 1 -- 172.21.15.63:0/3601576493 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0035070 con 0x7f24b410db90 2024-05-27T07:53:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.970+0000 7f24b9c00640 1 -- 172.21.15.63:0/3601576493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b41a9460 con 0x7f24b410db90 2024-05-27T07:53:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.973+0000 7f24c0c21740 1 Processor -- start 2024-05-27T07:53:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.974+0000 7f24c0c21740 1 -- start start 2024-05-27T07:53:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.974+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136441b0 0x5626139e6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.974+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56261361b2f0 0x5626139e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.974+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626136234d0 0x5626139e72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.974+0000 7f24c0c21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562613957b90 con 0x5626136234d0 2024-05-27T07:53:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.974+0000 7f24c0c21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562613947730 con 0x56261361b2f0 2024-05-27T07:53:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.974+0000 7f24c0c21740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56261394b770 con 0x5626136441b0 2024-05-27T07:53:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.975+0000 7f24ab400640 1 -- 172.21.15.63:0/3601576493 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f24a0042030 con 0x7f24b410db90 2024-05-27T07:53:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.975+0000 7f24b3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136441b0 0x5626139e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.975+0000 7f24b2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56261361b2f0 0x5626139e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.975+0000 7f24b2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56261361b2f0 0x5626139e6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:33714/0 (socket says 172.21.15.63:33714) 2024-05-27T07:53:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.975+0000 7f24b2a00640 1 -- 172.21.15.63:0/2213632231 learned_addr learned my addr 172.21.15.63:0/2213632231 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.976+0000 7f24b2000640 1 --2- 172.21.15.63:0/2213632231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626136234d0 0x5626139e72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.976+0000 7f24b2a00640 1 -- 172.21.15.63:0/2213632231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136441b0 msgr2=0x5626139e6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.976+0000 7f24b2a00640 1 --2- 172.21.15.63:0/2213632231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136441b0 0x5626139e6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.976+0000 7f24b2a00640 1 -- 172.21.15.63:0/2213632231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626136234d0 msgr2=0x5626139e72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.976+0000 7f24b2a00640 1 --2- 172.21.15.63:0/2213632231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626136234d0 0x5626139e72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.976+0000 7f24b2a00640 1 -- 172.21.15.63:0/2213632231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626138e08b0 con 0x56261361b2f0 2024-05-27T07:53:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.976+0000 7f24b2000640 1 --2- 172.21.15.63:0/2213632231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626136234d0 0x5626139e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.977+0000 7f24b2a00640 1 --2- 172.21.15.63:0/2213632231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56261361b2f0 0x5626139e6da0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009ca0 tx=0x7f24a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d7c8a63c0d399fe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.977+0000 7f249fe00640 1 -- 172.21.15.63:0/2213632231 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4005220 con 0x56261361b2f0 2024-05-27T07:53:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.977+0000 7f249fe00640 1 -- 172.21.15.63:0/2213632231 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24a4004c30 con 0x56261361b2f0 2024-05-27T07:53:45.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.977+0000 7f249fe00640 1 -- 172.21.15.63:0/2213632231 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4019690 con 0x56261361b2f0 2024-05-27T07:53:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.978+0000 7f24ab400640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24980b29d0 0x7f24980b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.979+0000 7f24c0c21740 1 -- 172.21.15.63:0/2213632231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56261361b2f0 msgr2=0x5626139e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.979+0000 7f24c0c21740 1 --2- 172.21.15.63:0/2213632231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56261361b2f0 0x5626139e6da0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009ca0 tx=0x7f24a4011810 comp rx=0 tx=0).stop 2024-05-27T07:53:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.979+0000 7f24ab400640 1 -- 172.21.15.63:0/3601576493 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f24a0102d20 con 0x7f24b410db90 2024-05-27T07:53:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.980+0000 7f24c0c21740 1 -- 172.21.15.63:0/2213632231 shutdown_connections 2024-05-27T07:53:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.980+0000 7f24c0c21740 1 --2- 172.21.15.63:0/2213632231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626136234d0 0x5626139e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.980+0000 7f24c0c21740 1 --2- 172.21.15.63:0/2213632231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56261361b2f0 0x5626139e6da0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009ca0 tx=0x7f24a4011810 comp rx=0 tx=0).stop 2024-05-27T07:53:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.980+0000 7f24c0c21740 1 --2- 172.21.15.63:0/2213632231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136441b0 0x5626139e6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.980+0000 7f24c0c21740 1 -- 172.21.15.63:0/2213632231 >> 172.21.15.63:0/2213632231 conn(0x562613638300 msgr2=0x5626139e3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.980+0000 7f24c0c21740 1 -- 172.21.15.63:0/2213632231 shutdown_connections 2024-05-27T07:53:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.980+0000 7f24c0c21740 1 -- 172.21.15.63:0/2213632231 wait complete. 2024-05-27T07:53:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.980+0000 7f24b3400640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24980b29d0 0x7f24980b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.981+0000 7f24c0c21740 1 -- start start 2024-05-27T07:53:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.981+0000 7f24c0c21740 1 Processor -- start 2024-05-27T07:53:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.982+0000 7f24b3400640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24980b29d0 0x7f24980b4ee0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5626139e39d0 tx=0x7f24ac009500 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.983+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136234d0 0x5626139ff460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.983+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136441b0 0x5626139ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.983+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.983+0000 7f24c0c21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56261362d790 con 0x562613602d80 2024-05-27T07:53:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.983+0000 7f24c0c21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5626138efbc0 con 0x5626136234d0 2024-05-27T07:53:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.983+0000 7f24c0c21740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5626139f3990 con 0x5626136441b0 2024-05-27T07:53:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.983+0000 7f24c0c21740 10 client.0 _ll_register_callbacks cb 0x5626138cd400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.983+0000 7f24b3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.983+0000 7f24b3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46296/0 (socket says 172.21.15.63:46296) 2024-05-27T07:53:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.984+0000 7f24b3400640 1 -- 172.21.15.63:0/3323425744 learned_addr learned my addr 172.21.15.63:0/3323425744 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.983+0000 7f24b2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136234d0 0x5626139ff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.984+0000 7f24b3400640 1 -- 172.21.15.63:0/3323425744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136441b0 msgr2=0x5626139ff9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:53:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.984+0000 7f24b3400640 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136441b0 0x5626139ff9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.984+0000 7f24b3400640 1 -- 172.21.15.63:0/3323425744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136234d0 msgr2=0x5626139ff460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.984+0000 7f24b3400640 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136234d0 0x5626139ff460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.984+0000 7f24b3400640 1 -- 172.21.15.63:0/3323425744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261365c210 con 0x562613602d80 2024-05-27T07:53:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.985+0000 7f24b3400640 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f9850 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f24ac01f9e0 tx=0x7f24ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2676c13ee3378251 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.985+0000 7f249cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.985+0000 7f249cc00640 1 -- 172.21.15.63:0/3323425744 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac00eb50 con 0x562613602d80 2024-05-27T07:53:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.985+0000 7f249cc00640 1 -- 172.21.15.63:0/3323425744 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f24ac017030 con 0x562613602d80 2024-05-27T07:53:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.985+0000 7f24c0c21740 1 -- 172.21.15.63:0/3323425744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a4004c30 con 0x562613602d80 2024-05-27T07:53:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.986+0000 7f249cc00640 1 -- 172.21.15.63:0/3323425744 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f24ac032050 con 0x562613602d80 2024-05-27T07:53:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.986+0000 7f24c0c21740 10 client.19287 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.986+0000 7f24c0c21740 1 -- 172.21.15.63:0/3323425744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562613728620 con 0x562613602d80 2024-05-27T07:53:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.986+0000 7f249cc00640 1 -- 172.21.15.63:0/3323425744 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24a4004c30 con 0x562613602d80 2024-05-27T07:53:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.987+0000 7f249cc00640 1 -- 172.21.15.63:0/3323425744 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f24ac0042e0 con 0x562613602d80 2024-05-27T07:53:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.987+0000 7f24c0c21740 10 client.19287 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.987+0000 7f24c0c21740 10 client.19287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.987+0000 7f24c0c21740 20 client.19287 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.987+0000 7f24c0c21740 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626135d81c0 0x562613a0b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.987+0000 7f24c0c21740 4 client.19287 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.987+0000 7f24c0c21740 1 -- 172.21.15.63:0/3323425744 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562613959ae0 con 0x5626135d81c0 2024-05-27T07:53:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.987+0000 7f24b2a00640 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626135d81c0 0x562613a0b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.989+0000 7f249cc00640 10 client.19287 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.989+0000 7f24b2a00640 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626135d81c0 0x562613a0b5f0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0edf04546bd7535 server_cookie=50ce282c5a27bbd9 in_seq=0 out_seq=0 2024-05-27T07:53:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f249cc00640 1 -- 172.21.15.63:0/3323425744 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562613959ae0 con 0x5626135d81c0 2024-05-27T07:53:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f249cc00640 10 client.19287 handle_command_reply: tid=0 2024-05-27T07:53:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f24c0c21740 1 client.19287 shutdown 2024-05-27T07:53:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f24c0c21740 1 -- 172.21.15.63:0/3323425744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 msgr2=0x5626139f9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f24c0c21740 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f9850 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f24ac01f9e0 tx=0x7f24ac004ab0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f24c0c21740 1 -- 172.21.15.63:0/3323425744 shutdown_connections 2024-05-27T07:53:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f24c0c21740 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626135d81c0 0x562613a0b5f0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f24c0c21740 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f9850 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f24c0c21740 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136441b0 0x5626139ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f24c0c21740 1 --2- 172.21.15.63:0/3323425744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136234d0 0x5626139ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f24c0c21740 1 -- 172.21.15.63:0/3323425744 >> 172.21.15.63:0/3323425744 conn(0x562613638300 msgr2=0x5626139e6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f249cc00640 0 client.19287 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.991+0000 7f24c0c21740 1 -- 172.21.15.63:0/3323425744 shutdown_connections 2024-05-27T07:53:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.992+0000 7f24c0c21740 1 -- 172.21.15.63:0/3323425744 wait complete. 2024-05-27T07:53:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:45.992+0000 7f24c0c21740 20 client.19287 trim_cache size 0 max 16384 2024-05-27T07:53:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.013+0000 7f24c0c21740 1 Processor -- start 2024-05-27T07:53:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.013+0000 7f24c0c21740 1 -- start start 2024-05-27T07:53:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.013+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.013+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136234d0 0x5626139e8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.013+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 0x5626139e2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.013+0000 7f24c0c21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41b18e0 con 0x562613602d80 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.013+0000 7f24c0c21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41af1b0 con 0x5626136441b0 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.013+0000 7f24c0c21740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41ad040 con 0x5626136234d0 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46310/0 (socket says 172.21.15.63:46310) 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b2a00640 1 -- 172.21.15.63:0/2672101405 learned_addr learned my addr 172.21.15.63:0/2672101405 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b3400640 1 --2- 172.21.15.63:0/2672101405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 0x5626139e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b2000640 1 --2- 172.21.15.63:0/2672101405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136234d0 0x5626139e8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b2a00640 1 -- 172.21.15.63:0/2672101405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136234d0 msgr2=0x5626139e8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b2a00640 1 --2- 172.21.15.63:0/2672101405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136234d0 0x5626139e8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b2a00640 1 -- 172.21.15.63:0/2672101405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 msgr2=0x5626139e2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b2a00640 1 --2- 172.21.15.63:0/2672101405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 0x5626139e2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b2a00640 1 -- 172.21.15.63:0/2672101405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626139e7990 con 0x562613602d80 2024-05-27T07:53:46.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.014+0000 7f24b3400640 1 --2- 172.21.15.63:0/2672101405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 0x5626139e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.015+0000 7f24b2a00640 1 --2- 172.21.15.63:0/2672101405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f2e60 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009ca0 tx=0x7f24a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19748dcf8042883e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.015+0000 7f249f400640 1 -- 172.21.15.63:0/2672101405 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4033070 con 0x562613602d80 2024-05-27T07:53:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.015+0000 7f249f400640 1 -- 172.21.15.63:0/2672101405 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24a40044d0 con 0x562613602d80 2024-05-27T07:53:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.015+0000 7f249f400640 1 -- 172.21.15.63:0/2672101405 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a402dc50 con 0x562613602d80 2024-05-27T07:53:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.016+0000 7f24c0c21740 1 -- 172.21.15.63:0/2672101405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 msgr2=0x5626139f2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.016+0000 7f24c0c21740 1 --2- 172.21.15.63:0/2672101405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f2e60 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009ca0 tx=0x7f24a4005f90 comp rx=0 tx=0).stop 2024-05-27T07:53:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.016+0000 7f24c0c21740 1 -- 172.21.15.63:0/2672101405 shutdown_connections 2024-05-27T07:53:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.016+0000 7f24c0c21740 1 --2- 172.21.15.63:0/2672101405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 0x5626139e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.016+0000 7f24c0c21740 1 --2- 172.21.15.63:0/2672101405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136234d0 0x5626139e8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.016+0000 7f24c0c21740 1 --2- 172.21.15.63:0/2672101405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x5626139f2e60 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.016+0000 7f24c0c21740 1 -- 172.21.15.63:0/2672101405 >> 172.21.15.63:0/2672101405 conn(0x562613638300 msgr2=0x5626139e7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.016+0000 7f24c0c21740 1 -- 172.21.15.63:0/2672101405 shutdown_connections 2024-05-27T07:53:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.017+0000 7f24c0c21740 1 -- 172.21.15.63:0/2672101405 wait complete. 2024-05-27T07:53:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.017+0000 7f24c0c21740 1 -- start start 2024-05-27T07:53:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.017+0000 7f24c0c21740 1 Processor -- start 2024-05-27T07:53:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.018+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x562613a00810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.018+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136234d0 0x562613a00d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.018+0000 7f24c0c21740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 0x562613a71d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.018+0000 7f24c0c21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41ad690 con 0x562613602d80 2024-05-27T07:53:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.018+0000 7f24c0c21740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41aeae0 con 0x5626136441b0 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.018+0000 7f24c0c21740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24b41add40 con 0x5626136234d0 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.018+0000 7f24c0c21740 10 client.0 _ll_register_callbacks cb 0x7f24ac009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.018+0000 7f24b2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x562613a00810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.018+0000 7f24b2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x562613a00810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46314/0 (socket says 172.21.15.63:46314) 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.018+0000 7f24b2a00640 1 -- 172.21.15.63:0/783591106 learned_addr learned my addr 172.21.15.63:0/783591106 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.019+0000 7f24b2000640 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136234d0 0x562613a00d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.019+0000 7f24b3400640 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 0x562613a71d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.019+0000 7f24b2a00640 1 -- 172.21.15.63:0/783591106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136234d0 msgr2=0x562613a00d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.019+0000 7f24b2a00640 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136234d0 0x562613a00d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.019+0000 7f24b2a00640 1 -- 172.21.15.63:0/783591106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 msgr2=0x562613a71d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.019+0000 7f24b2a00640 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 0x562613a71d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.019+0000 7f24b2a00640 1 -- 172.21.15.63:0/783591106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562613a01640 con 0x562613602d80 2024-05-27T07:53:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.019+0000 7f24b3400640 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 0x562613a71d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.019+0000 7f24b2a00640 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x562613a00810 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009250 tx=0x7f24a4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc44c6769d74184d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.019+0000 7f249cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.020+0000 7f249cc00640 1 -- 172.21.15.63:0/783591106 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4033040 con 0x562613602d80 2024-05-27T07:53:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.020+0000 7f249cc00640 1 -- 172.21.15.63:0/783591106 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f24a402da10 con 0x562613602d80 2024-05-27T07:53:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.020+0000 7f24c0c21740 1 -- 172.21.15.63:0/783591106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a40044d0 con 0x562613602d80 2024-05-27T07:53:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.020+0000 7f249cc00640 1 -- 172.21.15.63:0/783591106 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f24a402dc10 con 0x562613602d80 2024-05-27T07:53:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.020+0000 7f24c0c21740 10 client.19293 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.020+0000 7f24c0c21740 1 -- 172.21.15.63:0/783591106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562613a73300 con 0x562613602d80 2024-05-27T07:53:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.021+0000 7f249cc00640 1 -- 172.21.15.63:0/783591106 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24a40044d0 con 0x562613602d80 2024-05-27T07:53:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.021+0000 7f249cc00640 1 -- 172.21.15.63:0/783591106 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f24a403f030 con 0x562613602d80 2024-05-27T07:53:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.021+0000 7f24c0c21740 10 client.19293 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.021+0000 7f24c0c21740 10 client.19293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.021+0000 7f24c0c21740 20 client.19293 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.021+0000 7f24c0c21740 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562613a737d0 0x562613a75bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.021+0000 7f24c0c21740 4 client.19293 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:53:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.021+0000 7f24c0c21740 1 -- 172.21.15.63:0/783591106 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626139532c0 con 0x562613a737d0 2024-05-27T07:53:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.022+0000 7f24b2000640 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562613a737d0 0x562613a75bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.023+0000 7f24b2000640 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562613a737d0 0x562613a75bf0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80241a6683f4641d server_cookie=680b5f65e51c7874 in_seq=0 out_seq=0 2024-05-27T07:53:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.023+0000 7f249cc00640 10 client.19293 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.053+0000 7f249cc00640 1 -- 172.21.15.63:0/783591106 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5626139532c0 con 0x562613a737d0 2024-05-27T07:53:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.053+0000 7f249cc00640 10 client.19293 handle_command_reply: tid=0 2024-05-27T07:53:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.053+0000 7f24c0c21740 1 client.19293 shutdown 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 1 -- 172.21.15.63:0/783591106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 msgr2=0x562613a00810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x562613a00810 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009250 tx=0x7f24a4011da0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 1 -- 172.21.15.63:0/783591106 shutdown_connections 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562613a737d0 0x562613a75bf0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626136441b0 0x562613a71d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626136234d0 0x562613a00d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 1 --2- 172.21.15.63:0/783591106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562613602d80 0x562613a00810 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f249cc00640 0 client.19293 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 1 -- 172.21.15.63:0/783591106 >> 172.21.15.63:0/783591106 conn(0x562613638300 msgr2=0x5626139e6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 1 -- 172.21.15.63:0/783591106 shutdown_connections 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 1 -- 172.21.15.63:0/783591106 wait complete. 2024-05-27T07:53:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.054+0000 7f24c0c21740 20 client.19293 trim_cache size 0 max 16384 2024-05-27T07:53:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:53:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:53:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.055+0000 7f24b9c00640 1 -- 172.21.15.63:0/3601576493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24980b29d0 msgr2=0x7f24980b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.055+0000 7f24b9c00640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24980b29d0 0x7f24980b4ee0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5626139e39d0 tx=0x7f24ac009500 comp rx=0 tx=0).stop 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.055+0000 7f24b9c00640 1 -- 172.21.15.63:0/3601576493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24b410db90 msgr2=0x7f24b41a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.055+0000 7f24b9c00640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24b410db90 0x7f24b41a2b50 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f24a0005940 tx=0x7f24a0005e30 comp rx=0 tx=0).stop 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.056+0000 7f24b9c00640 1 -- 172.21.15.63:0/3601576493 shutdown_connections 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.056+0000 7f24b9c00640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24980b29d0 0x7f24980b4ee0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.056+0000 7f24b9c00640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24b41104f0 0x7f24b419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.056+0000 7f24b9c00640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24b410db90 0x7f24b41a2b50 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.056+0000 7f24b9c00640 1 --2- 172.21.15.63:0/3601576493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24b4076510 0x7f24b41a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.056+0000 7f24b9c00640 1 -- 172.21.15.63:0/3601576493 >> 172.21.15.63:0/3601576493 conn(0x7f24b40ff340 msgr2=0x7f24b4110a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.056+0000 7f24b9c00640 1 -- 172.21.15.63:0/3601576493 shutdown_connections 2024-05-27T07:53:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:46.056+0000 7f24b9c00640 1 -- 172.21.15.63:0/3601576493 wait complete. 2024-05-27T07:53:46.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.078+0000 7f8682600700 1 -- 172.21.15.63:0/1827265693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f867c19bec0 con 0x7f867c108d20 2024-05-27T07:53:46.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.079+0000 7f8679600700 1 -- 172.21.15.63:0/1827265693 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f86680a9390 con 0x7f867c108d20 2024-05-27T07:53:46.081 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:53:46.081 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:53:46.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.082+0000 7f8682600700 1 -- 172.21.15.63:0/1827265693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86640b29f0 msgr2=0x7f86640b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:46.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.082+0000 7f8682600700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86640b29f0 0x7f86640b4f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f867c102c30 tx=0x7f867000b410 comp rx=0 tx=0).stop 2024-05-27T07:53:46.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.082+0000 7f8682600700 1 -- 172.21.15.63:0/1827265693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f867c108d20 msgr2=0x7f867c1a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:46.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.082+0000 7f8682600700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f867c108d20 0x7f867c1a0cb0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f866800d980 tx=0x7f866800dc90 comp rx=0 tx=0).stop 2024-05-27T07:53:46.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.083+0000 7f8682600700 1 -- 172.21.15.63:0/1827265693 shutdown_connections 2024-05-27T07:53:46.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.083+0000 7f8682600700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f86640b29f0 0x7f86640b4f00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.083+0000 7f8682600700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f867c109670 0x7f867c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.083+0000 7f8682600700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f867c1043f0 0x7f867c1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.083+0000 7f8682600700 1 --2- 172.21.15.63:0/1827265693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f867c108d20 0x7f867c1a0cb0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:46.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.083+0000 7f8682600700 1 -- 172.21.15.63:0/1827265693 >> 172.21.15.63:0/1827265693 conn(0x7f867c079d00 msgr2=0x7f867c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:46.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.083+0000 7f8682600700 1 -- 172.21.15.63:0/1827265693 shutdown_connections 2024-05-27T07:53:46.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:46.083+0000 7f8682600700 1 -- 172.21.15.63:0/1827265693 wait complete. 2024-05-27T07:53:46.086 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:53:47.068 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:53:47.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:47 smithi086 ceph-mon[34687]: pgmap v637: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:53:47.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:47 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1827265693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:53:47.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:47 smithi149 ceph-mon[35408]: pgmap v637: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:53:47.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:47 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1827265693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:53:47.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:47 smithi063 ceph-mon[31613]: pgmap v637: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:53:47.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:47 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1827265693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:53:47.550 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:49.337 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:49 smithi063 ceph-mon[31613]: pgmap v638: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 16 op/s 2024-05-27T07:53:49.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:49 smithi086 ceph-mon[34687]: pgmap v638: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 16 op/s 2024-05-27T07:53:49.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:49 smithi149 ceph-mon[35408]: pgmap v638: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 16 op/s 2024-05-27T07:53:50.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.013+0000 7fbc57000700 1 -- 172.21.15.63:0/816446266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5010e900 msgr2=0x7fbc50110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:50.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.013+0000 7fbc57000700 1 --2- 172.21.15.63:0/816446266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5010e900 0x7fbc50110de0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800b0a0 tx=0x7fbc4800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.013+0000 7fbc57000700 1 -- 172.21.15.63:0/816446266 shutdown_connections 2024-05-27T07:53:50.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.013+0000 7fbc57000700 1 --2- 172.21.15.63:0/816446266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5010e900 0x7fbc50110de0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.013+0000 7fbc57000700 1 --2- 172.21.15.63:0/816446266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc501054d0 0x7fbc501058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.013+0000 7fbc57000700 1 --2- 172.21.15.63:0/816446266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc50105e20 0x7fbc5010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.013+0000 7fbc57000700 1 -- 172.21.15.63:0/816446266 >> 172.21.15.63:0/816446266 conn(0x7fbc500fedb0 msgr2=0x7fbc501011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:50.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.013+0000 7fbc57000700 1 -- 172.21.15.63:0/816446266 shutdown_connections 2024-05-27T07:53:50.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.013+0000 7fbc57000700 1 -- 172.21.15.63:0/816446266 wait complete. 2024-05-27T07:53:50.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.014+0000 7fbc57000700 1 Processor -- start 2024-05-27T07:53:50.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.015+0000 7fbc57000700 1 -- start start 2024-05-27T07:53:50.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.015+0000 7fbc57000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc501054d0 0x7fbc501a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.015+0000 7fbc57000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc50105e20 0x7fbc501a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.015+0000 7fbc57000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5010e900 0x7fbc5019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.015+0000 7fbc57000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50114330 con 0x7fbc5010e900 2024-05-27T07:53:50.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.015+0000 7fbc57000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501141b0 con 0x7fbc501054d0 2024-05-27T07:53:50.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.015+0000 7fbc57000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501144b0 con 0x7fbc50105e20 2024-05-27T07:53:50.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc55000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5010e900 0x7fbc5019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc55000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5010e900 0x7fbc5019ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39226/0 (socket says 172.21.15.63:39226) 2024-05-27T07:53:50.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc55000700 1 -- 172.21.15.63:0/2520200352 learned_addr learned my addr 172.21.15.63:0/2520200352 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:50.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc4fe00700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc501054d0 0x7fbc501a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc47400700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc50105e20 0x7fbc501a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc55000700 1 -- 172.21.15.63:0/2520200352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc50105e20 msgr2=0x7fbc501a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:50.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc55000700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc50105e20 0x7fbc501a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc55000700 1 -- 172.21.15.63:0/2520200352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc501054d0 msgr2=0x7fbc501a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:50.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc55000700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc501054d0 0x7fbc501a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc55000700 1 -- 172.21.15.63:0/2520200352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5019b260 con 0x7fbc5010e900 2024-05-27T07:53:50.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.016+0000 7fbc47400700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc50105e20 0x7fbc501a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:50.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.017+0000 7fbc55000700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5010e900 0x7fbc5019ab30 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800b070 tx=0x7fbc48004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:50.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.017+0000 7fbc4d600700 1 -- 172.21.15.63:0/2520200352 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48021070 con 0x7fbc5010e900 2024-05-27T07:53:50.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.017+0000 7fbc4d600700 1 -- 172.21.15.63:0/2520200352 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc48007b80 con 0x7fbc5010e900 2024-05-27T07:53:50.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.017+0000 7fbc57000700 1 -- 172.21.15.63:0/2520200352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5019b4f0 con 0x7fbc5010e900 2024-05-27T07:53:50.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.017+0000 7fbc4d600700 1 -- 172.21.15.63:0/2520200352 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48004770 con 0x7fbc5010e900 2024-05-27T07:53:50.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.018+0000 7fbc57000700 1 -- 172.21.15.63:0/2520200352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc501a7750 con 0x7fbc5010e900 2024-05-27T07:53:50.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.022+0000 7fbc4d600700 1 -- 172.21.15.63:0/2520200352 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbc48003a00 con 0x7fbc5010e900 2024-05-27T07:53:50.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.024+0000 7fbc65c19b80 1 Processor -- start 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.024+0000 7fbc65c19b80 1 -- start start 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.024+0000 7fbc4d600700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc340b2a70 0x7fbc340b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc4d600700 1 -- 172.21.15.63:0/2520200352 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fbc4800f020 con 0x7fbc5010e900 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc47400700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc340b2a70 0x7fbc340b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c8d99f0 0x55ae2c96c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c96c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c979110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c89ae40 con 0x55ae2c96ce70 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c88a9e0 con 0x55ae2c7ede40 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c88ea20 con 0x55ae2c8d99f0 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc55000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c979110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc55000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c979110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39238/0 (socket says 172.21.15.63:39238) 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc55000700 1 -- 172.21.15.63:0/1707465195 learned_addr learned my addr 172.21.15.63:0/1707465195 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.025+0000 7fbc4fe00700 1 --2- 172.21.15.63:0/1707465195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c8d99f0 0x55ae2c96c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.026+0000 7fbc55000700 1 -- 172.21.15.63:0/1707465195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c8d99f0 msgr2=0x55ae2c96c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.026+0000 7fbc55000700 1 --2- 172.21.15.63:0/1707465195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c8d99f0 0x55ae2c96c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.026+0000 7fbc47400700 1 --2- 172.21.15.63:0/1707465195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c96c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.026+0000 7fbc55000700 1 -- 172.21.15.63:0/1707465195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 msgr2=0x55ae2c96c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.026+0000 7fbc55000700 1 --2- 172.21.15.63:0/1707465195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c96c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.026+0000 7fbc55000700 1 -- 172.21.15.63:0/1707465195 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae2c853420 con 0x55ae2c96ce70 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.026+0000 7fbc47400700 1 --2- 172.21.15.63:0/1707465195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c96c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.026+0000 7fbc47400700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc340b2a70 0x7fbc340b4f80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbc38005e10 tx=0x7fbc3800f360 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:50.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.026+0000 7fbc55000700 1 --2- 172.21.15.63:0/1707465195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c979110 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fbc480aae70 tx=0x7fbc480a6ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5422ffd23d8bf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:50.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.027+0000 7fbc45600700 1 -- 172.21.15.63:0/1707465195 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc480ab5e0 con 0x55ae2c96ce70 2024-05-27T07:53:50.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.027+0000 7fbc45600700 1 -- 172.21.15.63:0/1707465195 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc480aaee0 con 0x55ae2c96ce70 2024-05-27T07:53:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.027+0000 7fbc45600700 1 -- 172.21.15.63:0/1707465195 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc480a71f0 con 0x55ae2c96ce70 2024-05-27T07:53:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.028+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1707465195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 msgr2=0x55ae2c979110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.028+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/1707465195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c979110 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fbc480aae70 tx=0x7fbc480a6ed0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.028+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1707465195 shutdown_connections 2024-05-27T07:53:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.028+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/1707465195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c979110 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.028+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/1707465195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c8d99f0 0x55ae2c96c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.028+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/1707465195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c96c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.028+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1707465195 >> 172.21.15.63:0/1707465195 conn(0x55ae2c887390 msgr2=0x55ae2c9684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:50.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.028+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1707465195 shutdown_connections 2024-05-27T07:53:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.028+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1707465195 wait complete. 2024-05-27T07:53:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.029+0000 7fbc65c19b80 1 -- start start 2024-05-27T07:53:50.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.029+0000 7fbc65c19b80 1 Processor -- start 2024-05-27T07:53:50.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c976720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c976c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 0x55ae2c9771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c790fa0 con 0x55ae2c8d99f0 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c82b760 con 0x55ae2c7ede40 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c97dd50 con 0x55ae2c96ce70 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc65c19b80 10 client.0 _ll_register_callbacks cb 0x55ae2c834c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c976c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c976c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39250/0 (socket says 172.21.15.63:39250) 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc47400700 1 -- 172.21.15.63:0/1155269485 learned_addr learned my addr 172.21.15.63:0/1155269485 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.030+0000 7fbc4fe00700 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c976720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.031+0000 7fbc55000700 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 0x55ae2c9771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.031+0000 7fbc47400700 1 -- 172.21.15.63:0/1155269485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 msgr2=0x55ae2c9771a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.031+0000 7fbc47400700 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 0x55ae2c9771a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.031+0000 7fbc47400700 1 -- 172.21.15.63:0/1155269485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 msgr2=0x55ae2c976720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.031+0000 7fbc47400700 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c976720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.031+0000 7fbc47400700 1 -- 172.21.15.63:0/1155269485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae2c7da8d0 con 0x55ae2c8d99f0 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.032+0000 7fbc47400700 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c976c60 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800ba30 tx=0x7fbc380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b348c88614fba9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.032+0000 7fbc3e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.032+0000 7fbc3e000700 1 -- 172.21.15.63:0/1155269485 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38004230 con 0x55ae2c8d99f0 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.032+0000 7fbc3e000700 1 -- 172.21.15.63:0/1155269485 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fbc380043d0 con 0x55ae2c8d99f0 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.032+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1155269485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc480aaee0 con 0x55ae2c8d99f0 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.032+0000 7fbc3e000700 1 -- 172.21.15.63:0/1155269485 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbc3802d050 con 0x55ae2c8d99f0 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.032+0000 7fbc65c19b80 10 client.19305 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.032+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1155269485 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ae2c85b060 con 0x55ae2c8d99f0 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.033+0000 7fbc3e000700 1 -- 172.21.15.63:0/1155269485 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc480aaee0 con 0x55ae2c8d99f0 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.033+0000 7fbc3e000700 1 -- 172.21.15.63:0/1155269485 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbc3801f550 con 0x55ae2c8d99f0 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.033+0000 7fbc65c19b80 10 client.19305 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:50.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.033+0000 7fbc65c19b80 10 client.19305 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:53:50.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.033+0000 7fbc65c19b80 20 client.19305 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:50.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.034+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ae2c98acd0 0x55ae2c98d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.034+0000 7fbc65c19b80 4 client.19305 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:50.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.034+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1155269485 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae2c89cd90 con 0x55ae2c98acd0 2024-05-27T07:53:50.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.034+0000 7fbc4fe00700 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ae2c98acd0 0x55ae2c98d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.035+0000 7fbc4fe00700 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ae2c98acd0 0x55ae2c98d110 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=818d67174ea2e214 server_cookie=718bbed0550b967c in_seq=0 out_seq=0 2024-05-27T07:53:50.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.035+0000 7fbc3e000700 10 client.19305 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:53:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.037+0000 7fbc3e000700 1 -- 172.21.15.63:0/1155269485 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ae2c89cd90 con 0x55ae2c98acd0 2024-05-27T07:53:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.037+0000 7fbc3e000700 10 client.19305 handle_command_reply: tid=0 2024-05-27T07:53:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.037+0000 7fbc65c19b80 1 client.19305 shutdown 2024-05-27T07:53:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1155269485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 msgr2=0x55ae2c976c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c976c60 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800ba30 tx=0x7fbc380047c0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1155269485 shutdown_connections 2024-05-27T07:53:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 0x55ae2c9771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c976720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ae2c98acd0 0x55ae2c98d110 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/1155269485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c976c60 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1155269485 >> 172.21.15.63:0/1155269485 conn(0x55ae2c887390 msgr2=0x55ae2c96c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc3e000700 0 client.19305 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:53:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1155269485 shutdown_connections 2024-05-27T07:53:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.038+0000 7fbc65c19b80 1 -- 172.21.15.63:0/1155269485 wait complete. 2024-05-27T07:53:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.039+0000 7fbc65c19b80 20 client.19305 trim_cache size 0 max 16384 2024-05-27T07:53:50.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.059+0000 7fbc65c19b80 1 Processor -- start 2024-05-27T07:53:50.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.059+0000 7fbc65c19b80 1 -- start start 2024-05-27T07:53:50.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.059+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c7ede40 0x55ae2c971f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.059+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c8d99f0 0x55ae2c96c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.059+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c9784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.059+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501b0af0 con 0x55ae2c96ce70 2024-05-27T07:53:50.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.059+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501ae380 con 0x55ae2c8d99f0 2024-05-27T07:53:50.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.059+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501ac1f0 con 0x55ae2c7ede40 2024-05-27T07:53:50.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc55000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c9784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc55000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c9784e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39252/0 (socket says 172.21.15.63:39252) 2024-05-27T07:53:50.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc55000700 1 -- 172.21.15.63:0/2425775759 learned_addr learned my addr 172.21.15.63:0/2425775759 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:50.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc4fe00700 1 --2- 172.21.15.63:0/2425775759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c7ede40 0x55ae2c971f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc47400700 1 --2- 172.21.15.63:0/2425775759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c8d99f0 0x55ae2c96c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc55000700 1 -- 172.21.15.63:0/2425775759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c7ede40 msgr2=0x55ae2c971f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:50.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc55000700 1 --2- 172.21.15.63:0/2425775759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c7ede40 0x55ae2c971f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc55000700 1 -- 172.21.15.63:0/2425775759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c8d99f0 msgr2=0x55ae2c96c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:50.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc55000700 1 --2- 172.21.15.63:0/2425775759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c8d99f0 0x55ae2c96c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc55000700 1 -- 172.21.15.63:0/2425775759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae2c974c90 con 0x55ae2c96ce70 2024-05-27T07:53:50.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.060+0000 7fbc47400700 1 --2- 172.21.15.63:0/2425775759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c8d99f0 0x55ae2c96c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:50.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.061+0000 7fbc55000700 1 --2- 172.21.15.63:0/2425775759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c9784e0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fbc480a8150 tx=0x7fbc48009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4c37f3e2aabb52 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:50.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.061+0000 7fbc44c00700 1 -- 172.21.15.63:0/2425775759 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc480b3cb0 con 0x55ae2c96ce70 2024-05-27T07:53:50.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.061+0000 7fbc44c00700 1 -- 172.21.15.63:0/2425775759 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc480ab5e0 con 0x55ae2c96ce70 2024-05-27T07:53:50.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.061+0000 7fbc44c00700 1 -- 172.21.15.63:0/2425775759 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc480ab230 con 0x55ae2c96ce70 2024-05-27T07:53:50.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.062+0000 7fbc65c19b80 1 -- 172.21.15.63:0/2425775759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 msgr2=0x55ae2c9784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:50.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.062+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/2425775759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c9784e0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fbc480a8150 tx=0x7fbc48009e20 comp rx=0 tx=0).stop 2024-05-27T07:53:50.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.062+0000 7fbc65c19b80 1 -- 172.21.15.63:0/2425775759 shutdown_connections 2024-05-27T07:53:50.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.062+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/2425775759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c96ce70 0x55ae2c9784e0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.062+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/2425775759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c7ede40 0x55ae2c971f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.062+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/2425775759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c8d99f0 0x55ae2c96c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.062+0000 7fbc65c19b80 1 -- 172.21.15.63:0/2425775759 >> 172.21.15.63:0/2425775759 conn(0x55ae2c887390 msgr2=0x55ae2c977dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:50.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.063+0000 7fbc65c19b80 1 -- 172.21.15.63:0/2425775759 shutdown_connections 2024-05-27T07:53:50.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.063+0000 7fbc65c19b80 1 -- 172.21.15.63:0/2425775759 wait complete. 2024-05-27T07:53:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.063+0000 7fbc65c19b80 1 -- start start 2024-05-27T07:53:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.063+0000 7fbc65c19b80 1 Processor -- start 2024-05-27T07:53:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.064+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c9835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.064+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c970eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.064+0000 7fbc65c19b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 0x55ae2c9713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.064+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501ac840 con 0x55ae2c8d99f0 2024-05-27T07:53:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.064+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501adcb0 con 0x55ae2c7ede40 2024-05-27T07:53:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.064+0000 7fbc65c19b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501acf10 con 0x55ae2c96ce70 2024-05-27T07:53:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.064+0000 7fbc65c19b80 10 client.0 _ll_register_callbacks cb 0x55ae2c834c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:50.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.064+0000 7fbc47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c970eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.064+0000 7fbc47400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c970eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39256/0 (socket says 172.21.15.63:39256) 2024-05-27T07:53:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.064+0000 7fbc47400700 1 -- 172.21.15.63:0/526341090 learned_addr learned my addr 172.21.15.63:0/526341090 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:50.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc4fe00700 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c9835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc55000700 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 0x55ae2c9713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc47400700 1 -- 172.21.15.63:0/526341090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 msgr2=0x55ae2c9713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc47400700 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 0x55ae2c9713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc47400700 1 -- 172.21.15.63:0/526341090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 msgr2=0x55ae2c9835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc47400700 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c9835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc47400700 1 -- 172.21.15.63:0/526341090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae2c971ae0 con 0x55ae2c8d99f0 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc55000700 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 0x55ae2c9713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc4fe00700 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c9835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc47400700 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c970eb0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fbc38017040 tx=0x7fbc3801fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3245be6a3966a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.065+0000 7fbc3e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.066+0000 7fbc3e000700 1 -- 172.21.15.63:0/526341090 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38004030 con 0x55ae2c8d99f0 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.066+0000 7fbc3e000700 1 -- 172.21.15.63:0/526341090 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fbc380041d0 con 0x55ae2c8d99f0 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.066+0000 7fbc65c19b80 1 -- 172.21.15.63:0/526341090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc480ab5e0 con 0x55ae2c8d99f0 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.066+0000 7fbc3e000700 1 -- 172.21.15.63:0/526341090 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbc38043050 con 0x55ae2c8d99f0 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.066+0000 7fbc65c19b80 10 client.19311 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:50.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.066+0000 7fbc65c19b80 1 -- 172.21.15.63:0/526341090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ae2c9f57e0 con 0x55ae2c8d99f0 2024-05-27T07:53:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.067+0000 7fbc3e000700 1 -- 172.21.15.63:0/526341090 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc480ab5e0 con 0x55ae2c8d99f0 2024-05-27T07:53:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.067+0000 7fbc3e000700 1 -- 172.21.15.63:0/526341090 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbc3800d600 con 0x55ae2c8d99f0 2024-05-27T07:53:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.067+0000 7fbc65c19b80 10 client.19311 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.067+0000 7fbc65c19b80 10 client.19311 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:53:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.067+0000 7fbc65c19b80 20 client.19311 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.067+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ae2c9f5dd0 0x55ae2c9f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.067+0000 7fbc65c19b80 4 client.19311 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:53:50.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.067+0000 7fbc65c19b80 1 -- 172.21.15.63:0/526341090 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae2c896570 con 0x55ae2c9f5dd0 2024-05-27T07:53:50.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.068+0000 7fbc4fe00700 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ae2c9f5dd0 0x55ae2c9f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:50.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.069+0000 7fbc4fe00700 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ae2c9f5dd0 0x55ae2c9f81f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd41fc4b9930f35b server_cookie=424b4316ad17329a in_seq=0 out_seq=0 2024-05-27T07:53:50.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.069+0000 7fbc3e000700 10 client.19311 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.069+0000 7fbc3e000700 1 -- 172.21.15.63:0/526341090 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae2c896570 con 0x55ae2c9f5dd0 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.069+0000 7fbc3e000700 10 client.19311 handle_command_reply: tid=0 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.069+0000 7fbc65c19b80 1 client.19311 shutdown 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 1 -- 172.21.15.63:0/526341090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 msgr2=0x55ae2c970eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c970eb0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fbc38017040 tx=0x7fbc3801fc90 comp rx=0 tx=0).stop 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 1 -- 172.21.15.63:0/526341090 shutdown_connections 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ae2c96ce70 0x55ae2c9713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae2c7ede40 0x55ae2c9835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ae2c9f5dd0 0x55ae2c9f81f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 1 --2- 172.21.15.63:0/526341090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ae2c8d99f0 0x55ae2c970eb0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 1 -- 172.21.15.63:0/526341090 >> 172.21.15.63:0/526341090 conn(0x55ae2c887390 msgr2=0x55ae2c968470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:50.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc3e000700 0 client.19311 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:53:50.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 1 -- 172.21.15.63:0/526341090 shutdown_connections 2024-05-27T07:53:50.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 1 -- 172.21.15.63:0/526341090 wait complete. 2024-05-27T07:53:50.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.070+0000 7fbc65c19b80 20 client.19311 trim_cache size 0 max 16384 2024-05-27T07:53:50.073 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:53:50.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.071+0000 7fbc57000700 1 -- 172.21.15.63:0/2520200352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc340b2a70 msgr2=0x7fbc340b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:50.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.071+0000 7fbc57000700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc340b2a70 0x7fbc340b4f80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbc38005e10 tx=0x7fbc3800f360 comp rx=0 tx=0).stop 2024-05-27T07:53:50.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.071+0000 7fbc57000700 1 -- 172.21.15.63:0/2520200352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5010e900 msgr2=0x7fbc5019ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:50.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.071+0000 7fbc57000700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5010e900 0x7fbc5019ab30 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800b070 tx=0x7fbc48004d50 comp rx=0 tx=0).stop 2024-05-27T07:53:50.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.071+0000 7fbc57000700 1 -- 172.21.15.63:0/2520200352 shutdown_connections 2024-05-27T07:53:50.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.071+0000 7fbc57000700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc340b2a70 0x7fbc340b4f80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.071+0000 7fbc57000700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5010e900 0x7fbc5019ab30 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.071+0000 7fbc57000700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc501054d0 0x7fbc501a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.071+0000 7fbc57000700 1 --2- 172.21.15.63:0/2520200352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc50105e20 0x7fbc501a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:50.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.071+0000 7fbc57000700 1 -- 172.21.15.63:0/2520200352 >> 172.21.15.63:0/2520200352 conn(0x7fbc500fedb0 msgr2=0x7fbc500ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:50.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.072+0000 7fbc57000700 1 -- 172.21.15.63:0/2520200352 shutdown_connections 2024-05-27T07:53:50.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:50.072+0000 7fbc57000700 1 -- 172.21.15.63:0/2520200352 wait complete. 2024-05-27T07:53:51.067 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:53:51.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:51 smithi086 ceph-mon[34687]: pgmap v639: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:53:51.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:51 smithi149 ceph-mon[35408]: pgmap v639: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:53:51.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:51 smithi063 ceph-mon[31613]: pgmap v639: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:53:51.560 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:53.450 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:53 smithi063 ceph-mon[31613]: pgmap v640: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 MiB/s wr, 13 op/s 2024-05-27T07:53:53.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:53 smithi086 ceph-mon[34687]: pgmap v640: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 MiB/s wr, 13 op/s 2024-05-27T07:53:53.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:53 smithi149 ceph-mon[35408]: pgmap v640: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.7 MiB/s wr, 13 op/s 2024-05-27T07:53:54.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.077+0000 7fc373000700 1 -- 172.21.15.63:0/1097118305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc36c10b890 msgr2=0x7fc36c1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.077+0000 7fc373000700 1 --2- 172.21.15.63:0/1097118305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc36c10b890 0x7fc36c1131f0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc3640099c0 tx=0x7fc364009cd0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.077+0000 7fc373000700 1 -- 172.21.15.63:0/1097118305 shutdown_connections 2024-05-27T07:53:54.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.077+0000 7fc373000700 1 --2- 172.21.15.63:0/1097118305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc36c10b890 0x7fc36c1131f0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.077+0000 7fc373000700 1 --2- 172.21.15.63:0/1097118305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c106610 0x7fc36c106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.077+0000 7fc373000700 1 --2- 172.21.15.63:0/1097118305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc36c10af40 0x7fc36c10b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.077+0000 7fc373000700 1 -- 172.21.15.63:0/1097118305 >> 172.21.15.63:0/1097118305 conn(0x7fc36c102020 msgr2=0x7fc36c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:54.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.077+0000 7fc373000700 1 -- 172.21.15.63:0/1097118305 shutdown_connections 2024-05-27T07:53:54.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.077+0000 7fc373000700 1 -- 172.21.15.63:0/1097118305 wait complete. 2024-05-27T07:53:54.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.079+0000 7fc373000700 1 Processor -- start 2024-05-27T07:53:54.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.079+0000 7fc373000700 1 -- start start 2024-05-27T07:53:54.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.079+0000 7fc373000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c106610 0x7fc36c1a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.079+0000 7fc373000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc36c10af40 0x7fc36c1a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.079+0000 7fc373000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc36c10b890 0x7fc36c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.079+0000 7fc373000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c116370 con 0x7fc36c10af40 2024-05-27T07:53:54.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.079+0000 7fc373000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1161f0 con 0x7fc36c10b890 2024-05-27T07:53:54.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.079+0000 7fc373000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1164f0 con 0x7fc36c106610 2024-05-27T07:53:54.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.080+0000 7fc371200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc36c10af40 0x7fc36c1a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.080+0000 7fc371200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc36c10af40 0x7fc36c1a2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39266/0 (socket says 172.21.15.63:39266) 2024-05-27T07:53:54.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.080+0000 7fc371200700 1 -- 172.21.15.63:0/1770349205 learned_addr learned my addr 172.21.15.63:0/1770349205 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:54.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.080+0000 7fc372600700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc36c10b890 0x7fc36c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.080+0000 7fc371c00700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c106610 0x7fc36c1a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.080+0000 7fc371200700 1 -- 172.21.15.63:0/1770349205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c106610 msgr2=0x7fc36c1a29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.080+0000 7fc371200700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c106610 0x7fc36c1a29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.080+0000 7fc371200700 1 -- 172.21.15.63:0/1770349205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc36c10b890 msgr2=0x7fc36c19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.080+0000 7fc371200700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc36c10b890 0x7fc36c19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.080+0000 7fc371200700 1 -- 172.21.15.63:0/1770349205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c19d1f0 con 0x7fc36c10af40 2024-05-27T07:53:54.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.081+0000 7fc371200700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc36c10af40 0x7fc36c1a2f00 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fc35800c9e0 tx=0x7fc35800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:54.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.081+0000 7fc36a000700 1 -- 172.21.15.63:0/1770349205 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358007aa0 con 0x7fc36c10af40 2024-05-27T07:53:54.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.081+0000 7fc36a000700 1 -- 172.21.15.63:0/1770349205 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc358007c40 con 0x7fc36c10af40 2024-05-27T07:53:54.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.081+0000 7fc373000700 1 -- 172.21.15.63:0/1770349205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36c19d4e0 con 0x7fc36c10af40 2024-05-27T07:53:54.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.081+0000 7fc36a000700 1 -- 172.21.15.63:0/1770349205 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358018660 con 0x7fc36c10af40 2024-05-27T07:53:54.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.081+0000 7fc373000700 1 -- 172.21.15.63:0/1770349205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36c1a9770 con 0x7fc36c10af40 2024-05-27T07:53:54.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.086+0000 7fc36a000700 1 -- 172.21.15.63:0/1770349205 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc358018800 con 0x7fc36c10af40 2024-05-27T07:53:54.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.087+0000 7fc38317ab80 1 Processor -- start 2024-05-27T07:53:54.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.087+0000 7fc38317ab80 1 -- start start 2024-05-27T07:53:54.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.087+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9870ade40 0x55e98722bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.088+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 0x55e987236ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.088+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 0x55e987239020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.088+0000 7fc38317ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e98715ae40 con 0x55e9870ade40 2024-05-27T07:53:54.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.088+0000 7fc38317ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e98714a9e0 con 0x55e98722c820 2024-05-27T07:53:54.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.088+0000 7fc38317ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e98714ea20 con 0x55e98722c410 2024-05-27T07:53:54.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.088+0000 7fc372600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9870ade40 0x55e98722bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.088+0000 7fc372600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9870ade40 0x55e98722bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39268/0 (socket says 172.21.15.63:39268) 2024-05-27T07:53:54.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.088+0000 7fc372600700 1 -- 172.21.15.63:0/2679620202 learned_addr learned my addr 172.21.15.63:0/2679620202 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:54.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.088+0000 7fc36a000700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3540b2a70 0x7fc3540b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.088+0000 7fc371c00700 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 0x55e987236ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.089+0000 7fc36a000700 1 -- 172.21.15.63:0/1770349205 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fc3580e28e0 con 0x7fc36c10af40 2024-05-27T07:53:54.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.089+0000 7fc371200700 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 0x55e987239020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.089+0000 7fc372600700 1 -- 172.21.15.63:0/2679620202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 msgr2=0x55e987236ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.089+0000 7fc372600700 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 0x55e987236ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.089+0000 7fc372600700 1 -- 172.21.15.63:0/2679620202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 msgr2=0x55e987239020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.089+0000 7fc372600700 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 0x55e987239020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.089+0000 7fc372600700 1 -- 172.21.15.63:0/2679620202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e987113420 con 0x55e9870ade40 2024-05-27T07:53:54.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.090+0000 7fc371c00700 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 0x55e987236ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:54.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.090+0000 7fc371200700 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 0x55e987239020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:54.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.090+0000 7fc372600700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3540b2a70 0x7fc3540b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.090+0000 7fc372600700 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9870ade40 0x55e98722bed0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fc364015000 tx=0x7fc364009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379796f624128314 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:54.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.091+0000 7fc372600700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3540b2a70 0x7fc3540b4f80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc364029000 tx=0x7fc36402c000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:54.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.092+0000 7fc35ea00700 1 -- 172.21.15.63:0/2679620202 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364003b10 con 0x55e9870ade40 2024-05-27T07:53:54.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.093+0000 7fc35ea00700 1 -- 172.21.15.63:0/2679620202 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc364003e10 con 0x55e9870ade40 2024-05-27T07:53:54.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.093+0000 7fc35ea00700 1 -- 172.21.15.63:0/2679620202 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36402d870 con 0x55e9870ade40 2024-05-27T07:53:54.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.094+0000 7fc38317ab80 1 -- 172.21.15.63:0/2679620202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9870ade40 msgr2=0x55e98722bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.094+0000 7fc38317ab80 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9870ade40 0x55e98722bed0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fc364015000 tx=0x7fc364009510 comp rx=0 tx=0).stop 2024-05-27T07:53:54.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.094+0000 7fc38317ab80 1 -- 172.21.15.63:0/2679620202 shutdown_connections 2024-05-27T07:53:54.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.094+0000 7fc38317ab80 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 0x55e987239020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.094+0000 7fc38317ab80 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9870ade40 0x55e98722bed0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.094+0000 7fc38317ab80 1 --2- 172.21.15.63:0/2679620202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 0x55e987236ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.094+0000 7fc38317ab80 1 -- 172.21.15.63:0/2679620202 >> 172.21.15.63:0/2679620202 conn(0x55e98710dfc0 msgr2=0x55e987227d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:54.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.094+0000 7fc38317ab80 1 -- 172.21.15.63:0/2679620202 shutdown_connections 2024-05-27T07:53:54.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.094+0000 7fc38317ab80 1 -- 172.21.15.63:0/2679620202 wait complete. 2024-05-27T07:53:54.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.095+0000 7fc38317ab80 1 -- start start 2024-05-27T07:53:54.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.095+0000 7fc38317ab80 1 Processor -- start 2024-05-27T07:53:54.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.096+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9870ade40 0x55e987236610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.096+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 0x55e987236b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.096+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c820 0x55e987237090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.096+0000 7fc38317ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e987050fa0 con 0x55e98722c820 2024-05-27T07:53:54.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.096+0000 7fc38317ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e9870eb760 con 0x55e9870ade40 2024-05-27T07:53:54.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.096+0000 7fc38317ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e987239990 con 0x55e98722c410 2024-05-27T07:53:54.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.096+0000 7fc38317ab80 10 client.0 _ll_register_callbacks cb 0x55e9870f4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:54.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.096+0000 7fc372600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c820 0x55e987237090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.097+0000 7fc372600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c820 0x55e987237090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39276/0 (socket says 172.21.15.63:39276) 2024-05-27T07:53:54.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.097+0000 7fc372600700 1 -- 172.21.15.63:0/2159738213 learned_addr learned my addr 172.21.15.63:0/2159738213 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:54.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.097+0000 7fc371200700 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 0x55e987236b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.097+0000 7fc371c00700 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9870ade40 0x55e987236610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.097+0000 7fc372600700 1 -- 172.21.15.63:0/2159738213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 msgr2=0x55e987236b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.097+0000 7fc372600700 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 0x55e987236b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.098+0000 7fc372600700 1 -- 172.21.15.63:0/2159738213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9870ade40 msgr2=0x55e987236610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.098+0000 7fc372600700 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9870ade40 0x55e987236610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.098+0000 7fc372600700 1 -- 172.21.15.63:0/2159738213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98709a8d0 con 0x55e98722c820 2024-05-27T07:53:54.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.098+0000 7fc372600700 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c820 0x55e987237090 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fc36400b580 tx=0x7fc364005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf5d77ecc57ffb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:54.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.098+0000 7fc353400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:54.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.098+0000 7fc353400700 1 -- 172.21.15.63:0/2159738213 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36403a060 con 0x55e98722c820 2024-05-27T07:53:54.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.099+0000 7fc353400700 1 -- 172.21.15.63:0/2159738213 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fc36402d750 con 0x55e98722c820 2024-05-27T07:53:54.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.099+0000 7fc38317ab80 1 -- 172.21.15.63:0/2159738213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc364003e10 con 0x55e98722c820 2024-05-27T07:53:54.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.099+0000 7fc353400700 1 -- 172.21.15.63:0/2159738213 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc364041050 con 0x55e98722c820 2024-05-27T07:53:54.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.099+0000 7fc38317ab80 10 client.19323 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:54.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.099+0000 7fc38317ab80 1 -- 172.21.15.63:0/2159738213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e98711b060 con 0x55e98722c820 2024-05-27T07:53:54.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.100+0000 7fc353400700 1 -- 172.21.15.63:0/2159738213 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc364003e10 con 0x55e98722c820 2024-05-27T07:53:54.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.100+0000 7fc353400700 1 -- 172.21.15.63:0/2159738213 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc36400f6f0 con 0x55e98722c820 2024-05-27T07:53:54.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.100+0000 7fc38317ab80 10 client.19323 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:54.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.100+0000 7fc38317ab80 10 client.19323 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:53:54.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.100+0000 7fc38317ab80 20 client.19323 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:54.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.100+0000 7fc38317ab80 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e987246730 0x55e987248b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.100+0000 7fc38317ab80 4 client.19323 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:54.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.100+0000 7fc38317ab80 1 -- 172.21.15.63:0/2159738213 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e98715cd90 con 0x55e987246730 2024-05-27T07:53:54.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.101+0000 7fc371c00700 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e987246730 0x55e987248b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.102+0000 7fc353400700 10 client.19323 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:53:54.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.102+0000 7fc371c00700 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e987246730 0x55e987248b50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64222848ec7e6d69 server_cookie=c57d07971109d1b6 in_seq=0 out_seq=0 2024-05-27T07:53:54.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.104+0000 7fc353400700 1 -- 172.21.15.63:0/2159738213 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e98715cd90 con 0x55e987246730 2024-05-27T07:53:54.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.104+0000 7fc353400700 10 client.19323 handle_command_reply: tid=0 2024-05-27T07:53:54.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.104+0000 7fc38317ab80 1 client.19323 shutdown 2024-05-27T07:53:54.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 1 -- 172.21.15.63:0/2159738213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c820 msgr2=0x55e987237090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c820 0x55e987237090 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fc36400b580 tx=0x7fc364005810 comp rx=0 tx=0).stop 2024-05-27T07:53:54.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 1 -- 172.21.15.63:0/2159738213 shutdown_connections 2024-05-27T07:53:54.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e987246730 0x55e987248b50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c820 0x55e987237090 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9870ade40 0x55e987236610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 1 --2- 172.21.15.63:0/2159738213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c410 0x55e987236b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 1 -- 172.21.15.63:0/2159738213 >> 172.21.15.63:0/2159738213 conn(0x55e98710dfc0 msgr2=0x55e987226ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:54.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc353400700 0 client.19323 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:53:54.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 1 -- 172.21.15.63:0/2159738213 shutdown_connections 2024-05-27T07:53:54.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 1 -- 172.21.15.63:0/2159738213 wait complete. 2024-05-27T07:53:54.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.105+0000 7fc38317ab80 20 client.19323 trim_cache size 0 max 16384 2024-05-27T07:53:54.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.128+0000 7fc38317ab80 1 Processor -- start 2024-05-27T07:53:54.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.128+0000 7fc38317ab80 1 -- start start 2024-05-27T07:53:54.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.129+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9870ade40 0x55e987231e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.129+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e9872307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.129+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 0x55e987242c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.129+0000 7fc38317ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1b4d70 con 0x55e98722c410 2024-05-27T07:53:54.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.129+0000 7fc38317ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1b2600 con 0x55e98722c820 2024-05-27T07:53:54.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.129+0000 7fc38317ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1b0470 con 0x55e9870ade40 2024-05-27T07:53:54.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.129+0000 7fc371200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e9872307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.129+0000 7fc371200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e9872307a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39286/0 (socket says 172.21.15.63:39286) 2024-05-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.129+0000 7fc371200700 1 -- 172.21.15.63:0/4202003745 learned_addr learned my addr 172.21.15.63:0/4202003745 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.129+0000 7fc371c00700 1 --2- 172.21.15.63:0/4202003745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9870ade40 0x55e987231e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.130+0000 7fc372600700 1 --2- 172.21.15.63:0/4202003745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 0x55e987242c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.130+0000 7fc371200700 1 -- 172.21.15.63:0/4202003745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9870ade40 msgr2=0x55e987231e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.130+0000 7fc371200700 1 --2- 172.21.15.63:0/4202003745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9870ade40 0x55e987231e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.130+0000 7fc371200700 1 -- 172.21.15.63:0/4202003745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 msgr2=0x55e987242c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.130+0000 7fc371200700 1 --2- 172.21.15.63:0/4202003745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 0x55e987242c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.130+0000 7fc371200700 1 -- 172.21.15.63:0/4202003745 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e987229350 con 0x55e98722c410 2024-05-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.130+0000 7fc372600700 1 --2- 172.21.15.63:0/4202003745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 0x55e987242c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:54.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.130+0000 7fc371200700 1 --2- 172.21.15.63:0/4202003745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e9872307a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fc3580a5820 tx=0x7fc3580a8fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38f6bbf6b9b3b4b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:54.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.131+0000 7fc35e000700 1 -- 172.21.15.63:0/4202003745 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3580e00a0 con 0x55e98722c410 2024-05-27T07:53:54.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.131+0000 7fc35e000700 1 -- 172.21.15.63:0/4202003745 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3580a97a0 con 0x55e98722c410 2024-05-27T07:53:54.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.131+0000 7fc35e000700 1 -- 172.21.15.63:0/4202003745 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3580e26c0 con 0x55e98722c410 2024-05-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.132+0000 7fc38317ab80 1 -- 172.21.15.63:0/4202003745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 msgr2=0x55e9872307a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.132+0000 7fc38317ab80 1 --2- 172.21.15.63:0/4202003745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e9872307a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fc3580a5820 tx=0x7fc3580a8fa0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.132+0000 7fc38317ab80 1 -- 172.21.15.63:0/4202003745 shutdown_connections 2024-05-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.132+0000 7fc38317ab80 1 --2- 172.21.15.63:0/4202003745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e98722c820 0x55e987242c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.132+0000 7fc38317ab80 1 --2- 172.21.15.63:0/4202003745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9870ade40 0x55e987231e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.132+0000 7fc38317ab80 1 --2- 172.21.15.63:0/4202003745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e9872307a0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.132+0000 7fc38317ab80 1 -- 172.21.15.63:0/4202003745 >> 172.21.15.63:0/4202003745 conn(0x55e98710dfc0 msgr2=0x55e987236c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.132+0000 7fc38317ab80 1 -- 172.21.15.63:0/4202003745 shutdown_connections 2024-05-27T07:53:54.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.132+0000 7fc38317ab80 1 -- 172.21.15.63:0/4202003745 wait complete. 2024-05-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.133+0000 7fc38317ab80 1 -- start start 2024-05-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.133+0000 7fc38317ab80 1 Processor -- start 2024-05-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.134+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9870ade40 0x55e98724bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.134+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e987249290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.134+0000 7fc38317ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c820 0x55e987249850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.134+0000 7fc38317ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1b0ac0 con 0x55e98722c410 2024-05-27T07:53:54.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.134+0000 7fc38317ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1b1f30 con 0x55e9870ade40 2024-05-27T07:53:54.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.134+0000 7fc38317ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c1b1190 con 0x55e98722c820 2024-05-27T07:53:54.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.134+0000 7fc38317ab80 10 client.0 _ll_register_callbacks cb 0x55e9870f4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.134+0000 7fc371200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e987249290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.134+0000 7fc371200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e987249290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39292/0 (socket says 172.21.15.63:39292) 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.134+0000 7fc371200700 1 -- 172.21.15.63:0/1107620724 learned_addr learned my addr 172.21.15.63:0/1107620724 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.135+0000 7fc371c00700 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9870ade40 0x55e98724bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.135+0000 7fc372600700 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c820 0x55e987249850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.135+0000 7fc371200700 1 -- 172.21.15.63:0/1107620724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c820 msgr2=0x55e987249850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.135+0000 7fc371200700 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c820 0x55e987249850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.135+0000 7fc371200700 1 -- 172.21.15.63:0/1107620724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9870ade40 msgr2=0x55e98724bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.135+0000 7fc371200700 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9870ade40 0x55e98724bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.135+0000 7fc371200700 1 -- 172.21.15.63:0/1107620724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e987249ff0 con 0x55e98722c410 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.135+0000 7fc371200700 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e987249290 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fc3580a94a0 tx=0x7fc3580e2690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3af1a768b5f1df server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:54.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.135+0000 7fc353400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:54.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.136+0000 7fc353400700 1 -- 172.21.15.63:0/1107620724 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3580e00a0 con 0x55e98722c410 2024-05-27T07:53:54.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.136+0000 7fc353400700 1 -- 172.21.15.63:0/1107620724 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fc3580a9c10 con 0x55e98722c410 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.136+0000 7fc38317ab80 1 -- 172.21.15.63:0/1107620724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3580a97a0 con 0x55e98722c410 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.136+0000 7fc353400700 1 -- 172.21.15.63:0/1107620724 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc358083e60 con 0x55e98722c410 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.136+0000 7fc38317ab80 10 client.19329 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.136+0000 7fc38317ab80 1 -- 172.21.15.63:0/1107620724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e9872b96d0 con 0x55e98722c410 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.137+0000 7fc353400700 1 -- 172.21.15.63:0/1107620724 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3580a97a0 con 0x55e98722c410 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.137+0000 7fc353400700 1 -- 172.21.15.63:0/1107620724 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc358020030 con 0x55e98722c410 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.137+0000 7fc38317ab80 10 client.19329 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.137+0000 7fc38317ab80 10 client.19329 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.137+0000 7fc38317ab80 20 client.19329 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.137+0000 7fc38317ab80 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e9872b9d30 0x55e9872bc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.137+0000 7fc38317ab80 4 client.19329 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.137+0000 7fc38317ab80 1 -- 172.21.15.63:0/1107620724 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e987156570 con 0x55e9872b9d30 2024-05-27T07:53:54.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.138+0000 7fc371c00700 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e9872b9d30 0x55e9872bc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.139+0000 7fc371c00700 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e9872b9d30 0x55e9872bc150 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f8c818224e4b03e server_cookie=7801823e761764e8 in_seq=0 out_seq=0 2024-05-27T07:53:54.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.139+0000 7fc353400700 10 client.19329 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.139+0000 7fc353400700 1 -- 172.21.15.63:0/1107620724 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e987156570 con 0x55e9872b9d30 2024-05-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.139+0000 7fc353400700 10 client.19329 handle_command_reply: tid=0 2024-05-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.139+0000 7fc38317ab80 1 client.19329 shutdown 2024-05-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 1 -- 172.21.15.63:0/1107620724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 msgr2=0x55e987249290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e987249290 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fc3580a94a0 tx=0x7fc3580e2690 comp rx=0 tx=0).stop 2024-05-27T07:53:54.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 1 -- 172.21.15.63:0/1107620724 shutdown_connections 2024-05-27T07:53:54.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e98722c820 0x55e987249850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9870ade40 0x55e98724bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e9872b9d30 0x55e9872bc150 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 1 --2- 172.21.15.63:0/1107620724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e98722c410 0x55e987249290 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 1 -- 172.21.15.63:0/1107620724 >> 172.21.15.63:0/1107620724 conn(0x55e98710dfc0 msgr2=0x55e987227a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:54.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc353400700 0 client.19329 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:53:54.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 1 -- 172.21.15.63:0/1107620724 shutdown_connections 2024-05-27T07:53:54.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 1 -- 172.21.15.63:0/1107620724 wait complete. 2024-05-27T07:53:54.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.140+0000 7fc38317ab80 20 client.19329 trim_cache size 0 max 16384 2024-05-27T07:53:54.143 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:53:54.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.141+0000 7fc373000700 1 -- 172.21.15.63:0/1770349205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3540b2a70 msgr2=0x7fc3540b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.141+0000 7fc373000700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3540b2a70 0x7fc3540b4f80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc364029000 tx=0x7fc36402c000 comp rx=0 tx=0).stop 2024-05-27T07:53:54.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.141+0000 7fc373000700 1 -- 172.21.15.63:0/1770349205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc36c10af40 msgr2=0x7fc36c1a2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.141+0000 7fc373000700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc36c10af40 0x7fc36c1a2f00 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fc35800c9e0 tx=0x7fc35800cda0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.141+0000 7fc373000700 1 -- 172.21.15.63:0/1770349205 shutdown_connections 2024-05-27T07:53:54.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.141+0000 7fc373000700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc3540b2a70 0x7fc3540b4f80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.141+0000 7fc373000700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc36c10b890 0x7fc36c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.141+0000 7fc373000700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc36c106610 0x7fc36c1a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.141+0000 7fc373000700 1 --2- 172.21.15.63:0/1770349205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc36c10af40 0x7fc36c1a2f00 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.142+0000 7fc373000700 1 -- 172.21.15.63:0/1770349205 >> 172.21.15.63:0/1770349205 conn(0x7fc36c102020 msgr2=0x7fc36c10a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:54.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.142+0000 7fc373000700 1 -- 172.21.15.63:0/1770349205 shutdown_connections 2024-05-27T07:53:54.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:54.142+0000 7fc373000700 1 -- 172.21.15.63:0/1770349205 wait complete. 2024-05-27T07:53:55.091 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:53:55.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:55 smithi086 ceph-mon[34687]: pgmap v641: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:53:55.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:55 smithi149 ceph-mon[35408]: pgmap v641: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:53:55.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:55 smithi063 ceph-mon[31613]: pgmap v641: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:53:55.617 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:57.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:57 smithi086 ceph-mon[34687]: pgmap v642: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:53:57.462 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:57 smithi063 ceph-mon[31613]: pgmap v642: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:53:57.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:57 smithi149 ceph-mon[35408]: pgmap v642: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 9 op/s 2024-05-27T07:53:58.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.130+0000 7f85ee200700 1 -- 172.21.15.63:0/536467061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e810e6f0 msgr2=0x7f85e8110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:58.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.131+0000 7f85ee200700 1 --2- 172.21.15.63:0/536467061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e810e6f0 0x7f85e8110bd0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f85e000b0a0 tx=0x7f85e000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.131+0000 7f85ee200700 1 -- 172.21.15.63:0/536467061 shutdown_connections 2024-05-27T07:53:58.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.131+0000 7f85ee200700 1 --2- 172.21.15.63:0/536467061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e810e6f0 0x7f85e8110bd0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.131+0000 7f85ee200700 1 --2- 172.21.15.63:0/536467061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85e8105280 0x7f85e8105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.131+0000 7f85ee200700 1 --2- 172.21.15.63:0/536467061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e8105bd0 0x7f85e810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.131+0000 7f85ee200700 1 -- 172.21.15.63:0/536467061 >> 172.21.15.63:0/536467061 conn(0x7f85e8078ab0 msgr2=0x7f85e8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:58.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.131+0000 7f85ee200700 1 -- 172.21.15.63:0/536467061 shutdown_connections 2024-05-27T07:53:58.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.131+0000 7f85ee200700 1 -- 172.21.15.63:0/536467061 wait complete. 2024-05-27T07:53:58.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.132+0000 7f85ee200700 1 Processor -- start 2024-05-27T07:53:58.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.133+0000 7f85ee200700 1 -- start start 2024-05-27T07:53:58.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.134+0000 7f85ee200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e8105280 0x7f85e81a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.134+0000 7f85ee200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85e8105bd0 0x7f85e81a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.134+0000 7f85ee200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e810e6f0 0x7f85e81ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.134+0000 7f85ee200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85e8114120 con 0x7f85e8105280 2024-05-27T07:53:58.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.134+0000 7f85ee200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85e8113fa0 con 0x7f85e810e6f0 2024-05-27T07:53:58.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.134+0000 7f85ee200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81142a0 con 0x7f85e8105bd0 2024-05-27T07:53:58.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.134+0000 7f85ece00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e8105280 0x7f85e81a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.134+0000 7f85ece00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e8105280 0x7f85e81a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39310/0 (socket says 172.21.15.63:39310) 2024-05-27T07:53:58.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.134+0000 7f85ece00700 1 -- 172.21.15.63:0/1114175575 learned_addr learned my addr 172.21.15.63:0/1114175575 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:58.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.135+0000 7f85e7e00700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85e8105bd0 0x7f85e81a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.135+0000 7f85ed800700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e810e6f0 0x7f85e81ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.135+0000 7f85ece00700 1 -- 172.21.15.63:0/1114175575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85e8105bd0 msgr2=0x7f85e81a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:58.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.135+0000 7f85ece00700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85e8105bd0 0x7f85e81a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.135+0000 7f85ece00700 1 -- 172.21.15.63:0/1114175575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e810e6f0 msgr2=0x7f85e81ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:58.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.135+0000 7f85ece00700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e810e6f0 0x7f85e81ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.135+0000 7f85ece00700 1 -- 172.21.15.63:0/1114175575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e81ad3c0 con 0x7f85e8105280 2024-05-27T07:53:58.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.136+0000 7f85ece00700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e8105280 0x7f85e81a1e00 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00d980 tx=0x7f85dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:58.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.136+0000 7f85e5600700 1 -- 172.21.15.63:0/1114175575 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc017070 con 0x7f85e8105280 2024-05-27T07:53:58.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.136+0000 7f85e5600700 1 -- 172.21.15.63:0/1114175575 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85dc005020 con 0x7f85e8105280 2024-05-27T07:53:58.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.136+0000 7f85e5600700 1 -- 172.21.15.63:0/1114175575 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc009ab0 con 0x7f85e8105280 2024-05-27T07:53:58.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.136+0000 7f85ee200700 1 -- 172.21.15.63:0/1114175575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e81ad590 con 0x7f85e8105280 2024-05-27T07:53:58.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.137+0000 7f85ee200700 1 -- 172.21.15.63:0/1114175575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e81ada80 con 0x7f85e8105280 2024-05-27T07:53:58.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.142+0000 7f85e5600700 1 -- 172.21.15.63:0/1114175575 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f85dc0051d0 con 0x7f85e8105280 2024-05-27T07:53:58.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.144+0000 7f85e5600700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85d00b2a70 0x7f85d00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.144+0000 7f85e5600700 1 -- 172.21.15.63:0/1114175575 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f85dc0e28c0 con 0x7f85e8105280 2024-05-27T07:53:58.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.145+0000 7f85fe3fcb80 1 Processor -- start 2024-05-27T07:53:58.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.145+0000 7f85fe3fcb80 1 -- start start 2024-05-27T07:53:58.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.145+0000 7f85e7e00700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85d00b2a70 0x7f85d00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.145+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 0x559e62515ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.145+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62520ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.145+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 0x559e62523020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.145+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559e62444e40 con 0x559e62516410 2024-05-27T07:53:58.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.145+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e624349e0 con 0x559e62397e40 2024-05-27T07:53:58.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.145+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559e62438a20 con 0x559e62516820 2024-05-27T07:53:58.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.146+0000 7f85ece00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 0x559e62515ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.146+0000 7f85ece00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 0x559e62515ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:53052/0 (socket says 172.21.15.63:53052) 2024-05-27T07:53:58.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.146+0000 7f85ece00700 1 -- 172.21.15.63:0/4138857803 learned_addr learned my addr 172.21.15.63:0/4138857803 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:58.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.146+0000 7f85e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62520ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.146+0000 7f85ed800700 1 --2- 172.21.15.63:0/4138857803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 0x559e62523020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.147+0000 7f85e7e00700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85d00b2a70 0x7f85d00b4f80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85e8101f20 tx=0x7f85d400f3c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:58.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.147+0000 7f85ece00700 1 -- 172.21.15.63:0/4138857803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 msgr2=0x559e62523020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:58.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.147+0000 7f85ece00700 1 --2- 172.21.15.63:0/4138857803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 0x559e62523020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.147+0000 7f85ece00700 1 -- 172.21.15.63:0/4138857803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 msgr2=0x559e62520ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:58.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.147+0000 7f85ece00700 1 --2- 172.21.15.63:0/4138857803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62520ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.147+0000 7f85ece00700 1 -- 172.21.15.63:0/4138857803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e623fd420 con 0x559e62397e40 2024-05-27T07:53:58.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.148+0000 7f85ece00700 1 --2- 172.21.15.63:0/4138857803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 0x559e62515ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0a5800 tx=0x7f85dc0a9e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e37a417050d6f5d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:58.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.148+0000 7f85da000700 1 -- 172.21.15.63:0/4138857803 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc009e60 con 0x559e62397e40 2024-05-27T07:53:58.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.149+0000 7f85da000700 1 -- 172.21.15.63:0/4138857803 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85dc0cc0b0 con 0x559e62397e40 2024-05-27T07:53:58.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.149+0000 7f85da000700 1 -- 172.21.15.63:0/4138857803 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0910f0 con 0x559e62397e40 2024-05-27T07:53:58.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.150+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/4138857803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 msgr2=0x559e62515ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:58.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.150+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/4138857803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 0x559e62515ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0a5800 tx=0x7f85dc0a9e80 comp rx=0 tx=0).stop 2024-05-27T07:53:58.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.150+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/4138857803 shutdown_connections 2024-05-27T07:53:58.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.150+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/4138857803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 0x559e62523020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.150+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/4138857803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 0x559e62515ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.150+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/4138857803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62520ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.150+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/4138857803 >> 172.21.15.63:0/4138857803 conn(0x559e623f7fc0 msgr2=0x559e62511d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:58.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.150+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/4138857803 shutdown_connections 2024-05-27T07:53:58.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.150+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/4138857803 wait complete. 2024-05-27T07:53:58.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.151+0000 7f85fe3fcb80 1 -- start start 2024-05-27T07:53:58.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.152+0000 7f85fe3fcb80 1 Processor -- start 2024-05-27T07:53:58.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.152+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 0x559e62520640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.152+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62520b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.152+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516820 0x559e625210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.152+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559e6233afa0 con 0x559e62516410 2024-05-27T07:53:58.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.152+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e623d5760 con 0x559e62516820 2024-05-27T07:53:58.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.152+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559e625238d0 con 0x559e62397e40 2024-05-27T07:53:58.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85fe3fcb80 10 client.0 _ll_register_callbacks cb 0x559e623dec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:58.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62520b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62520b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39330/0 (socket says 172.21.15.63:39330) 2024-05-27T07:53:58.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85e7e00700 1 -- 172.21.15.63:0/3100072641 learned_addr learned my addr 172.21.15.63:0/3100072641 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:58.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85ed800700 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516820 0x559e625210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85ece00700 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 0x559e62520640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85e7e00700 1 -- 172.21.15.63:0/3100072641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 msgr2=0x559e62520640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:58.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85e7e00700 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 0x559e62520640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85e7e00700 1 -- 172.21.15.63:0/3100072641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516820 msgr2=0x559e625210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:58.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85e7e00700 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516820 0x559e625210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.153+0000 7f85e7e00700 1 -- 172.21.15.63:0/3100072641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e623848d0 con 0x559e62516410 2024-05-27T07:53:58.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.154+0000 7f85ece00700 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 0x559e62520640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:58.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.154+0000 7f85ed800700 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516820 0x559e625210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:58.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.154+0000 7f85e7e00700 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62520b80 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f85d4010aa0 tx=0x7f85d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20436af1d3725f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:58.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.154+0000 7f85cea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:58.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.154+0000 7f85cea00700 1 -- 172.21.15.63:0/3100072641 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4018070 con 0x559e62516410 2024-05-27T07:53:58.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.154+0000 7f85cea00700 1 -- 172.21.15.63:0/3100072641 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f85d401cb00 con 0x559e62516410 2024-05-27T07:53:58.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.155+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/3100072641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85dc0cc0b0 con 0x559e62516410 2024-05-27T07:53:58.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.155+0000 7f85cea00700 1 -- 172.21.15.63:0/3100072641 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f85d401cd00 con 0x559e62516410 2024-05-27T07:53:58.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.155+0000 7f85fe3fcb80 10 client.19341 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:58.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.155+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/3100072641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e62405060 con 0x559e62516410 2024-05-27T07:53:58.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.156+0000 7f85cea00700 1 -- 172.21.15.63:0/3100072641 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85dc0cc0b0 con 0x559e62516410 2024-05-27T07:53:58.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.156+0000 7f85cea00700 1 -- 172.21.15.63:0/3100072641 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f85d4003ba0 con 0x559e62516410 2024-05-27T07:53:58.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.156+0000 7f85fe3fcb80 10 client.19341 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:58.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.156+0000 7f85fe3fcb80 10 client.19341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:58.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.156+0000 7f85fe3fcb80 20 client.19341 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:58.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.156+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e62530970 0x559e62532d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.156+0000 7f85fe3fcb80 4 client.19341 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:58.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.156+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/3100072641 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e62446d90 con 0x559e62530970 2024-05-27T07:53:58.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.157+0000 7f85ece00700 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e62530970 0x559e62532d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.158+0000 7f85cea00700 10 client.19341 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:58.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.158+0000 7f85ece00700 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e62530970 0x559e62532d90 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7d0ba8c31d9c39 server_cookie=880f69e137a51525 in_seq=0 out_seq=0 2024-05-27T07:53:58.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.160+0000 7f85cea00700 1 -- 172.21.15.63:0/3100072641 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559e62446d90 con 0x559e62530970 2024-05-27T07:53:58.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.160+0000 7f85cea00700 10 client.19341 handle_command_reply: tid=0 2024-05-27T07:53:58.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85fe3fcb80 1 client.19341 shutdown 2024-05-27T07:53:58.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/3100072641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 msgr2=0x559e62520b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:58.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62520b80 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f85d4010aa0 tx=0x7f85d40047c0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/3100072641 shutdown_connections 2024-05-27T07:53:58.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e62530970 0x559e62532d90 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516820 0x559e625210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 0x559e62520640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/3100072641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62520b80 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85cea00700 0 client.19341 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:58.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/3100072641 >> 172.21.15.63:0/3100072641 conn(0x559e623f7fc0 msgr2=0x559e62510ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:58.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.161+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/3100072641 shutdown_connections 2024-05-27T07:53:58.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.162+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/3100072641 wait complete. 2024-05-27T07:53:58.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.162+0000 7f85fe3fcb80 20 client.19341 trim_cache size 0 max 16384 2024-05-27T07:53:58.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.184+0000 7f85fe3fcb80 1 Processor -- start 2024-05-27T07:53:58.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.184+0000 7f85fe3fcb80 1 -- start start 2024-05-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.185+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 0x559e6251a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.185+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62522ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.185+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 0x559e6252c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.185+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b91b0 con 0x559e62516410 2024-05-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.185+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b6a40 con 0x559e62397e40 2024-05-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.185+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b48b0 con 0x559e62516820 2024-05-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.185+0000 7f85e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62522ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.185+0000 7f85e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62522ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39334/0 (socket says 172.21.15.63:39334) 2024-05-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.185+0000 7f85e7e00700 1 -- 172.21.15.63:0/996425139 learned_addr learned my addr 172.21.15.63:0/996425139 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:58.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.186+0000 7f85ece00700 1 --2- 172.21.15.63:0/996425139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 0x559e6251a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.186+0000 7f85ed800700 1 --2- 172.21.15.63:0/996425139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 0x559e6252c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.186+0000 7f85e7e00700 1 -- 172.21.15.63:0/996425139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 msgr2=0x559e6252c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:58.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.186+0000 7f85e7e00700 1 --2- 172.21.15.63:0/996425139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 0x559e6252c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.186+0000 7f85e7e00700 1 -- 172.21.15.63:0/996425139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 msgr2=0x559e6251a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:58.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.186+0000 7f85e7e00700 1 --2- 172.21.15.63:0/996425139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 0x559e6251a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.186+0000 7f85e7e00700 1 -- 172.21.15.63:0/996425139 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e62513350 con 0x559e62516410 2024-05-27T07:53:58.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.186+0000 7f85ed800700 1 --2- 172.21.15.63:0/996425139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 0x559e6252c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:58.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.186+0000 7f85e7e00700 1 --2- 172.21.15.63:0/996425139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62522ae0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f85d400aa00 tx=0x7f85d401cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651953d4af39a8df server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:58.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.187+0000 7f85d9600700 1 -- 172.21.15.63:0/996425139 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4038070 con 0x559e62516410 2024-05-27T07:53:58.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.187+0000 7f85d9600700 1 -- 172.21.15.63:0/996425139 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d40265b0 con 0x559e62516410 2024-05-27T07:53:58.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.187+0000 7f85d9600700 1 -- 172.21.15.63:0/996425139 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4018040 con 0x559e62516410 2024-05-27T07:53:58.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.188+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/996425139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 msgr2=0x559e62522ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:58.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.188+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/996425139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62522ae0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f85d400aa00 tx=0x7f85d401cc90 comp rx=0 tx=0).stop 2024-05-27T07:53:58.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.188+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/996425139 shutdown_connections 2024-05-27T07:53:58.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.188+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/996425139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62516820 0x559e6252c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.188+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/996425139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62397e40 0x559e6251a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.188+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/996425139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516410 0x559e62522ae0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.188+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/996425139 >> 172.21.15.63:0/996425139 conn(0x559e623f7fc0 msgr2=0x559e62510ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:58.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.188+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/996425139 shutdown_connections 2024-05-27T07:53:58.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.188+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/996425139 wait complete. 2024-05-27T07:53:58.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.189+0000 7f85fe3fcb80 1 -- start start 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.189+0000 7f85fe3fcb80 1 Processor -- start 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 0x559e62528f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516410 0x559e625242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85fe3fcb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516820 0x559e625247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b4f00 con 0x559e62516820 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b6370 con 0x559e62516410 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85fe3fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81b55d0 con 0x559e62397e40 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85fe3fcb80 10 client.0 _ll_register_callbacks cb 0x559e623dec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85ed800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516820 0x559e625247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85ed800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516820 0x559e625247f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39346/0 (socket says 172.21.15.63:39346) 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85ed800700 1 -- 172.21.15.63:0/2951623032 learned_addr learned my addr 172.21.15.63:0/2951623032 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:58.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85ece00700 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 0x559e62528f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.190+0000 7f85e7e00700 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516410 0x559e625242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.191+0000 7f85ed800700 1 -- 172.21.15.63:0/2951623032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 msgr2=0x559e62528f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:58.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.191+0000 7f85ed800700 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 0x559e62528f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.191+0000 7f85ed800700 1 -- 172.21.15.63:0/2951623032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516410 msgr2=0x559e625242b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:58.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.191+0000 7f85ed800700 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516410 0x559e625242b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.191+0000 7f85ed800700 1 -- 172.21.15.63:0/2951623032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e62524d30 con 0x559e62516820 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.191+0000 7f85ed800700 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516820 0x559e625247f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f85e000b0a0 tx=0x7f85e0004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8395cbfb30b0ba12 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.191+0000 7f85cea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.192+0000 7f85cea00700 1 -- 172.21.15.63:0/2951623032 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0004e50 con 0x559e62516820 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.192+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/2951623032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d40265b0 con 0x559e62516820 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.192+0000 7f85cea00700 1 -- 172.21.15.63:0/2951623032 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f85e00077d0 con 0x559e62516820 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.192+0000 7f85cea00700 1 -- 172.21.15.63:0/2951623032 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f85e00079d0 con 0x559e62516820 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.192+0000 7f85fe3fcb80 10 client.19347 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.192+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/2951623032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e6259afc0 con 0x559e62516820 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.192+0000 7f85cea00700 1 -- 172.21.15.63:0/2951623032 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85d40265b0 con 0x559e62516820 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.193+0000 7f85cea00700 1 -- 172.21.15.63:0/2951623032 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f85e0011070 con 0x559e62516820 2024-05-27T07:53:58.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.193+0000 7f85fe3fcb80 10 client.19347 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:58.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.193+0000 7f85fe3fcb80 10 client.19347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:58.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.193+0000 7f85fe3fcb80 20 client.19347 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:58.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.193+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e6259b720 0x559e6259db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:58.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.193+0000 7f85fe3fcb80 4 client.19347 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:53:58.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.193+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/2951623032 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e62440570 con 0x559e6259b720 2024-05-27T07:53:58.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.194+0000 7f85ece00700 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e6259b720 0x559e6259db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:58.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.195+0000 7f85ece00700 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e6259b720 0x559e6259db40 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5011967c25404dc server_cookie=dbbfa6ba97bb54f4 in_seq=0 out_seq=0 2024-05-27T07:53:58.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.195+0000 7f85cea00700 10 client.19347 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:58.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.195+0000 7f85cea00700 1 -- 172.21.15.63:0/2951623032 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e62440570 con 0x559e6259b720 2024-05-27T07:53:58.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.195+0000 7f85cea00700 10 client.19347 handle_command_reply: tid=0 2024-05-27T07:53:58.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.195+0000 7f85fe3fcb80 1 client.19347 shutdown 2024-05-27T07:53:58.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.195+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/2951623032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516820 msgr2=0x559e625247f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.195+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516820 0x559e625247f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f85e000b0a0 tx=0x7f85e0004c60 comp rx=0 tx=0).stop 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.196+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/2951623032 shutdown_connections 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.196+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559e6259b720 0x559e6259db40 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.196+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559e62516820 0x559e625247f0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.196+0000 7f85cea00700 0 client.19347 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.196+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e62397e40 0x559e62528f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.196+0000 7f85fe3fcb80 1 --2- 172.21.15.63:0/2951623032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e62516410 0x559e625242b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.196+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/2951623032 >> 172.21.15.63:0/2951623032 conn(0x559e623f7fc0 msgr2=0x559e62511a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.196+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/2951623032 shutdown_connections 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.196+0000 7f85fe3fcb80 1 -- 172.21.15.63:0/2951623032 wait complete. 2024-05-27T07:53:58.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.196+0000 7f85fe3fcb80 20 client.19347 trim_cache size 0 max 16384 2024-05-27T07:53:58.199 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:53:58.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.197+0000 7f85ee200700 1 -- 172.21.15.63:0/1114175575 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85d00b2a70 msgr2=0x7f85d00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.197+0000 7f85ee200700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85d00b2a70 0x7f85d00b4f80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85e8101f20 tx=0x7f85d400f3c0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.197+0000 7f85ee200700 1 -- 172.21.15.63:0/1114175575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e8105280 msgr2=0x7f85e81a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.197+0000 7f85ee200700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e8105280 0x7f85e81a1e00 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00d980 tx=0x7f85dc00dc90 comp rx=0 tx=0).stop 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.197+0000 7f85ee200700 1 -- 172.21.15.63:0/1114175575 shutdown_connections 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.197+0000 7f85ee200700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f85d00b2a70 0x7f85d00b4f80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.197+0000 7f85ee200700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85e810e6f0 0x7f85e81ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.197+0000 7f85ee200700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85e8105280 0x7f85e81a1e00 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.197+0000 7f85ee200700 1 --2- 172.21.15.63:0/1114175575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85e8105bd0 0x7f85e81a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.197+0000 7f85ee200700 1 -- 172.21.15.63:0/1114175575 >> 172.21.15.63:0/1114175575 conn(0x7f85e8078ab0 msgr2=0x7f85e8104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.198+0000 7f85ee200700 1 -- 172.21.15.63:0/1114175575 shutdown_connections 2024-05-27T07:53:58.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:53:58.198+0000 7f85ee200700 1 -- 172.21.15.63:0/1114175575 wait complete. 2024-05-27T07:53:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.304+0000 7fc16e800640 1 -- 172.21.15.63:0/2658421846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc168104210 msgr2=0x7fc1681105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.304+0000 7fc16e800640 1 --2- 172.21.15.63:0/2658421846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc168104210 0x7fc1681105e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fc160009ca0 tx=0x7fc16002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.304+0000 7fc16e800640 1 -- 172.21.15.63:0/2658421846 shutdown_connections 2024-05-27T07:53:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.304+0000 7fc16e800640 1 --2- 172.21.15.63:0/2658421846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc168104210 0x7fc1681105e0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.304+0000 7fc16e800640 1 --2- 172.21.15.63:0/2658421846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168109f60 0x7fc168103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.304+0000 7fc16e800640 1 --2- 172.21.15.63:0/2658421846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc168109ac0 0x7fc168103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.304+0000 7fc16e800640 1 -- 172.21.15.63:0/2658421846 >> 172.21.15.63:0/2658421846 conn(0x7fc1680ff180 msgr2=0x7fc1681015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.304+0000 7fc16e800640 1 -- 172.21.15.63:0/2658421846 shutdown_connections 2024-05-27T07:53:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.304+0000 7fc16e800640 1 -- 172.21.15.63:0/2658421846 wait complete. 2024-05-27T07:53:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.306+0000 7fc16e800640 1 Processor -- start 2024-05-27T07:53:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.306+0000 7fc16e800640 1 -- start start 2024-05-27T07:53:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.306+0000 7fc16e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc168104210 0x7fc16819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.306+0000 7fc16e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168109ac0 0x7fc16819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.306+0000 7fc16e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc168109f60 0x7fc16819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.306+0000 7fc16e800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc168113bd0 con 0x7fc168109f60 2024-05-27T07:53:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.306+0000 7fc16e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc168113a50 con 0x7fc168109ac0 2024-05-27T07:53:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.306+0000 7fc16e800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc168113d50 con 0x7fc168104210 2024-05-27T07:53:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc168109f60 0x7fc16819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc168109f60 0x7fc16819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56930/0 (socket says 172.21.15.63:56930) 2024-05-27T07:53:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16de00640 1 -- 172.21.15.63:0/1511924466 learned_addr learned my addr 172.21.15.63:0/1511924466 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16ca00640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168109ac0 0x7fc16819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16d400640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc168104210 0x7fc16819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16de00640 1 -- 172.21.15.63:0/1511924466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc168104210 msgr2=0x7fc16819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16de00640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc168104210 0x7fc16819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16de00640 1 -- 172.21.15.63:0/1511924466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168109ac0 msgr2=0x7fc16819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16de00640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168109ac0 0x7fc16819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16de00640 1 -- 172.21.15.63:0/1511924466 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16819f410 con 0x7fc168109f60 2024-05-27T07:53:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16ca00640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168109ac0 0x7fc16819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.307+0000 7fc16d400640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc168104210 0x7fc16819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.308+0000 7fc16de00640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc168109f60 0x7fc16819edd0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fc160031040 tx=0x7fc160032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.308+0000 7fc166000640 1 -- 172.21.15.63:0/1511924466 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160004040 con 0x7fc168109f60 2024-05-27T07:53:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.308+0000 7fc16e800640 1 -- 172.21.15.63:0/1511924466 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16819f6a0 con 0x7fc168109f60 2024-05-27T07:53:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.308+0000 7fc166000640 1 -- 172.21.15.63:0/1511924466 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc160004670 con 0x7fc168109f60 2024-05-27T07:53:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.308+0000 7fc166000640 1 -- 172.21.15.63:0/1511924466 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160030bb0 con 0x7fc168109f60 2024-05-27T07:53:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.308+0000 7fc16e800640 1 -- 172.21.15.63:0/1511924466 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16819fbf0 con 0x7fc168109f60 2024-05-27T07:53:59.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.312+0000 7fc176f6b740 1 Processor -- start 2024-05-27T07:53:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.313+0000 7fc166000640 1 -- 172.21.15.63:0/1511924466 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc1600041e0 con 0x7fc168109f60 2024-05-27T07:53:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.314+0000 7fc176f6b740 1 -- start start 2024-05-27T07:53:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.314+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efab31b0 0x5626efe55930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.314+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa8a2f0 0x5626efe55e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.314+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa924d0 0x5626efe563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.314+0000 7fc176f6b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5626efdc6b90 con 0x5626efa8a2f0 2024-05-27T07:53:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.314+0000 7fc176f6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5626efdb6730 con 0x5626efa924d0 2024-05-27T07:53:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.314+0000 7fc176f6b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5626efdba770 con 0x5626efab31b0 2024-05-27T07:53:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.315+0000 7fc16d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa8a2f0 0x5626efe55e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.315+0000 7fc16d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa8a2f0 0x5626efe55e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56940/0 (socket says 172.21.15.63:56940) 2024-05-27T07:53:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.315+0000 7fc16d400640 1 -- 172.21.15.63:0/891038835 learned_addr learned my addr 172.21.15.63:0/891038835 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.315+0000 7fc16ca00640 1 --2- 172.21.15.63:0/891038835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efab31b0 0x5626efe55930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.315+0000 7fc16ca00640 1 --2- 172.21.15.63:0/891038835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa924d0 0x5626efe563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.315+0000 7fc16d400640 1 -- 172.21.15.63:0/891038835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efab31b0 msgr2=0x5626efe55930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.315+0000 7fc16d400640 1 --2- 172.21.15.63:0/891038835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efab31b0 0x5626efe55930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.315+0000 7fc16d400640 1 -- 172.21.15.63:0/891038835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa924d0 msgr2=0x5626efe563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.315+0000 7fc16d400640 1 --2- 172.21.15.63:0/891038835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa924d0 0x5626efe563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.315+0000 7fc16d400640 1 -- 172.21.15.63:0/891038835 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626efd4f8b0 con 0x5626efa8a2f0 2024-05-27T07:53:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.316+0000 7fc16d400640 1 --2- 172.21.15.63:0/891038835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa8a2f0 0x5626efe55e70 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00bd60 tx=0x7fc15c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475bf6d42fe52513 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.316+0000 7fc15a600640 1 -- 172.21.15.63:0/891038835 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c015070 con 0x5626efa8a2f0 2024-05-27T07:53:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.316+0000 7fc166000640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1500b2950 0x7fc1500b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.316+0000 7fc15a600640 1 -- 172.21.15.63:0/891038835 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc15c0118d0 con 0x5626efa8a2f0 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.316+0000 7fc15a600640 1 -- 172.21.15.63:0/891038835 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c010730 con 0x5626efa8a2f0 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.316+0000 7fc166000640 1 -- 172.21.15.63:0/1511924466 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fc160103e70 con 0x7fc168109f60 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.316+0000 7fc16ca00640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1500b2950 0x7fc1500b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.317+0000 7fc176f6b740 1 -- 172.21.15.63:0/891038835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa8a2f0 msgr2=0x5626efe55e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.317+0000 7fc176f6b740 1 --2- 172.21.15.63:0/891038835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa8a2f0 0x5626efe55e70 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00bd60 tx=0x7fc15c00f730 comp rx=0 tx=0).stop 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.317+0000 7fc16ca00640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1500b2950 0x7fc1500b4e60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc15400cfd0 tx=0x7fc1540046c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.317+0000 7fc176f6b740 1 -- 172.21.15.63:0/891038835 shutdown_connections 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.317+0000 7fc176f6b740 1 --2- 172.21.15.63:0/891038835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa924d0 0x5626efe563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.317+0000 7fc176f6b740 1 --2- 172.21.15.63:0/891038835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa8a2f0 0x5626efe55e70 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.317+0000 7fc176f6b740 1 --2- 172.21.15.63:0/891038835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efab31b0 0x5626efe55930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.318+0000 7fc176f6b740 1 -- 172.21.15.63:0/891038835 >> 172.21.15.63:0/891038835 conn(0x5626efaa7300 msgr2=0x5626efe52210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.318+0000 7fc176f6b740 1 -- 172.21.15.63:0/891038835 shutdown_connections 2024-05-27T07:53:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.318+0000 7fc176f6b740 1 -- 172.21.15.63:0/891038835 wait complete. 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.318+0000 7fc176f6b740 1 -- start start 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.319+0000 7fc176f6b740 1 Processor -- start 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.319+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 0x5626efe63ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe64530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efab31b0 0x5626efe70380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc176f6b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5626efa9c790 con 0x5626efa924d0 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc176f6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5626efd5ebc0 con 0x5626efa8a2f0 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc176f6b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5626efe67050 con 0x5626efab31b0 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc176f6b740 10 client.0 _ll_register_callbacks cb 0x5626efd3c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe64530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16ca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe64530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56942/0 (socket says 172.21.15.63:56942) 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16ca00640 1 -- 172.21.15.63:0/2624730337 learned_addr learned my addr 172.21.15.63:0/2624730337 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16d400640 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 0x5626efe63ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16de00640 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efab31b0 0x5626efe70380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16ca00640 1 -- 172.21.15.63:0/2624730337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efab31b0 msgr2=0x5626efe70380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16ca00640 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efab31b0 0x5626efe70380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16ca00640 1 -- 172.21.15.63:0/2624730337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 msgr2=0x5626efe63ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16ca00640 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 0x5626efe63ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16ca00640 1 -- 172.21.15.63:0/2624730337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626efacb210 con 0x5626efa924d0 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.320+0000 7fc16d400640 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 0x5626efe63ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.321+0000 7fc16ca00640 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe64530 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fc154005bc0 tx=0x7fc15400c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb56115271fe718 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.321+0000 7fc14ea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.321+0000 7fc14ea00640 1 -- 172.21.15.63:0/2624730337 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154024070 con 0x5626efa924d0 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.321+0000 7fc176f6b740 1 -- 172.21.15.63:0/2624730337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc15c0118d0 con 0x5626efa924d0 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.321+0000 7fc14ea00640 1 -- 172.21.15.63:0/2624730337 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fc154005020 con 0x5626efa924d0 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc14ea00640 1 -- 172.21.15.63:0/2624730337 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc15402f050 con 0x5626efa924d0 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc176f6b740 10 client.19359 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc176f6b740 1 -- 172.21.15.63:0/2624730337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5626efb97620 con 0x5626efa924d0 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc14ea00640 1 -- 172.21.15.63:0/2624730337 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc15c0118d0 con 0x5626efa924d0 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc14ea00640 1 -- 172.21.15.63:0/2624730337 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc154020420 con 0x5626efa924d0 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc176f6b740 10 client.19359 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc176f6b740 10 client.19359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc176f6b740 20 client.19359 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc176f6b740 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626efa71d80 0x5626efe763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc176f6b740 4 client.19359 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.322+0000 7fc176f6b740 1 -- 172.21.15.63:0/2624730337 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626efdc8ae0 con 0x5626efa71d80 2024-05-27T07:53:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.323+0000 7fc16d400640 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626efa71d80 0x5626efe763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.324+0000 7fc16d400640 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626efa71d80 0x5626efe763c0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb1625d0e9cba1a server_cookie=22c86a8e9c765fab in_seq=0 out_seq=0 2024-05-27T07:53:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.324+0000 7fc14ea00640 10 client.19359 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.326+0000 7fc14ea00640 1 -- 172.21.15.63:0/2624730337 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5626efdc8ae0 con 0x5626efa71d80 2024-05-27T07:53:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.326+0000 7fc14ea00640 10 client.19359 handle_command_reply: tid=0 2024-05-27T07:53:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.326+0000 7fc176f6b740 1 client.19359 shutdown 2024-05-27T07:53:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.326+0000 7fc176f6b740 1 -- 172.21.15.63:0/2624730337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 msgr2=0x5626efe64530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.326+0000 7fc176f6b740 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe64530 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fc154005bc0 tx=0x7fc15400c470 comp rx=0 tx=0).stop 2024-05-27T07:53:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.327+0000 7fc176f6b740 1 -- 172.21.15.63:0/2624730337 shutdown_connections 2024-05-27T07:53:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.327+0000 7fc176f6b740 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626efa71d80 0x5626efe763c0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.327+0000 7fc176f6b740 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efab31b0 0x5626efe70380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.327+0000 7fc176f6b740 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe64530 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.327+0000 7fc176f6b740 1 --2- 172.21.15.63:0/2624730337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 0x5626efe63ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.327+0000 7fc14ea00640 0 client.19359 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.327+0000 7fc176f6b740 1 -- 172.21.15.63:0/2624730337 >> 172.21.15.63:0/2624730337 conn(0x5626efaa7300 msgr2=0x5626efe58140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.327+0000 7fc176f6b740 1 -- 172.21.15.63:0/2624730337 shutdown_connections 2024-05-27T07:53:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.327+0000 7fc176f6b740 1 -- 172.21.15.63:0/2624730337 wait complete. 2024-05-27T07:53:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.327+0000 7fc176f6b740 20 client.19359 trim_cache size 0 max 16384 2024-05-27T07:53:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.345+0000 7fc176f6b740 1 Processor -- start 2024-05-27T07:53:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.345+0000 7fc176f6b740 1 -- start start 2024-05-27T07:53:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.345+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efaa7300 0x5626efe5f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.345+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa8a2f0 0x5626efe63880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.345+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe5dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.345+0000 7fc176f6b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc1681b1710 con 0x5626efa924d0 2024-05-27T07:53:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.345+0000 7fc176f6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1681aefe0 con 0x5626efaa7300 2024-05-27T07:53:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.345+0000 7fc176f6b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc1681ace50 con 0x5626efa8a2f0 2024-05-27T07:53:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe5dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe5dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56944/0 (socket says 172.21.15.63:56944) 2024-05-27T07:53:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16de00640 1 -- 172.21.15.63:0/1400813467 learned_addr learned my addr 172.21.15.63:0/1400813467 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16d400640 1 --2- 172.21.15.63:0/1400813467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efaa7300 0x5626efe5f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16ca00640 1 --2- 172.21.15.63:0/1400813467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa8a2f0 0x5626efe63880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16de00640 1 -- 172.21.15.63:0/1400813467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa8a2f0 msgr2=0x5626efe63880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16de00640 1 --2- 172.21.15.63:0/1400813467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa8a2f0 0x5626efe63880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16de00640 1 -- 172.21.15.63:0/1400813467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efaa7300 msgr2=0x5626efe5f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16de00640 1 --2- 172.21.15.63:0/1400813467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efaa7300 0x5626efe5f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16de00640 1 -- 172.21.15.63:0/1400813467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626efe52e00 con 0x5626efa924d0 2024-05-27T07:53:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.346+0000 7fc16ca00640 1 --2- 172.21.15.63:0/1400813467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa8a2f0 0x5626efe63880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.347+0000 7fc16de00640 1 --2- 172.21.15.63:0/1400813467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe5dca0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fc1600c6ec0 tx=0x7fc1600ca790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac361d8f78e19e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.347+0000 7fc159c00640 1 -- 172.21.15.63:0/1400813467 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1600cd070 con 0x5626efa924d0 2024-05-27T07:53:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.347+0000 7fc159c00640 1 -- 172.21.15.63:0/1400813467 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc160036040 con 0x5626efa924d0 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.347+0000 7fc159c00640 1 -- 172.21.15.63:0/1400813467 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1600cf1f0 con 0x5626efa924d0 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.348+0000 7fc176f6b740 1 -- 172.21.15.63:0/1400813467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 msgr2=0x5626efe5dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.348+0000 7fc176f6b740 1 --2- 172.21.15.63:0/1400813467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe5dca0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fc1600c6ec0 tx=0x7fc1600ca790 comp rx=0 tx=0).stop 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.348+0000 7fc176f6b740 1 -- 172.21.15.63:0/1400813467 shutdown_connections 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.348+0000 7fc176f6b740 1 --2- 172.21.15.63:0/1400813467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efa924d0 0x5626efe5dca0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.348+0000 7fc176f6b740 1 --2- 172.21.15.63:0/1400813467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa8a2f0 0x5626efe63880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.348+0000 7fc176f6b740 1 --2- 172.21.15.63:0/1400813467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efaa7300 0x5626efe5f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.348+0000 7fc176f6b740 1 -- 172.21.15.63:0/1400813467 >> 172.21.15.63:0/1400813467 conn(0x5626efe56430 msgr2=0x5626efe58020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.348+0000 7fc176f6b740 1 -- 172.21.15.63:0/1400813467 shutdown_connections 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.348+0000 7fc176f6b740 1 -- 172.21.15.63:0/1400813467 wait complete. 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.349+0000 7fc176f6b740 1 -- start start 2024-05-27T07:53:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.349+0000 7fc176f6b740 1 Processor -- start 2024-05-27T07:53:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 0x5626efe6c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa924d0 0x5626efe6c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc176f6b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efaa7300 0x5626efe672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc176f6b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc1681ad4a0 con 0x5626efaa7300 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc176f6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1681ae910 con 0x5626efa8a2f0 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc176f6b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc1681adb70 con 0x5626efa924d0 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc176f6b740 10 client.0 _ll_register_callbacks cb 0x7fc15400cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc16de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efaa7300 0x5626efe672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc16de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efaa7300 0x5626efe672a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56956/0 (socket says 172.21.15.63:56956) 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc16de00640 1 -- 172.21.15.63:0/3246186374 learned_addr learned my addr 172.21.15.63:0/3246186374 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc16ca00640 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa924d0 0x5626efe6c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.350+0000 7fc16d400640 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 0x5626efe6c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.351+0000 7fc16de00640 1 -- 172.21.15.63:0/3246186374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa924d0 msgr2=0x5626efe6c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.351+0000 7fc16de00640 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa924d0 0x5626efe6c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.351+0000 7fc16de00640 1 -- 172.21.15.63:0/3246186374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 msgr2=0x5626efe6c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.351+0000 7fc16de00640 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 0x5626efe6c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.351+0000 7fc16de00640 1 -- 172.21.15.63:0/3246186374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626efe677e0 con 0x5626efaa7300 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.351+0000 7fc16de00640 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efaa7300 0x5626efe672a0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fc160004fd0 tx=0x7fc16002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ffaedfca65e356 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.351+0000 7fc14ea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.351+0000 7fc14ea00640 1 -- 172.21.15.63:0/3246186374 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1600ea060 con 0x5626efaa7300 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.351+0000 7fc14ea00640 1 -- 172.21.15.63:0/3246186374 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7fc160101420 con 0x5626efaa7300 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.351+0000 7fc176f6b740 1 -- 172.21.15.63:0/3246186374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc160036040 con 0x5626efaa7300 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc14ea00640 1 -- 172.21.15.63:0/3246186374 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc160090070 con 0x5626efaa7300 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc176f6b740 10 client.19365 fetch_fsmap learned FSMap version 33 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc176f6b740 1 -- 172.21.15.63:0/3246186374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5626efedf1a0 con 0x5626efaa7300 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc14ea00640 1 -- 172.21.15.63:0/3246186374 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc160036040 con 0x5626efaa7300 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc14ea00640 1 -- 172.21.15.63:0/3246186374 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fc1600cb0a0 con 0x5626efaa7300 2024-05-27T07:53:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc176f6b740 10 client.19365 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:53:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc176f6b740 10 client.19365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc176f6b740 20 client.19365 populate_metadata read hostname 'smithi063' 2024-05-27T07:53:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc176f6b740 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626efab31b0 0x5626efee1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc176f6b740 4 client.19365 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:53:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.352+0000 7fc176f6b740 1 -- 172.21.15.63:0/3246186374 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626efdc22c0 con 0x5626efab31b0 2024-05-27T07:53:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.353+0000 7fc16d400640 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626efab31b0 0x5626efee1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.354+0000 7fc16d400640 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626efab31b0 0x5626efee1820 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb6c6e3c89e464 server_cookie=3f22c9190cf9c42 in_seq=0 out_seq=0 2024-05-27T07:53:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.354+0000 7fc14ea00640 10 client.19365 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc14ea00640 1 -- 172.21.15.63:0/3246186374 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5626efdc22c0 con 0x5626efab31b0 2024-05-27T07:53:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc14ea00640 10 client.19365 handle_command_reply: tid=0 2024-05-27T07:53:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc176f6b740 1 client.19365 shutdown 2024-05-27T07:53:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc176f6b740 1 -- 172.21.15.63:0/3246186374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efaa7300 msgr2=0x5626efe672a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc176f6b740 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efaa7300 0x5626efe672a0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fc160004fd0 tx=0x7fc16002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc176f6b740 1 -- 172.21.15.63:0/3246186374 shutdown_connections 2024-05-27T07:53:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc176f6b740 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626efab31b0 0x5626efee1820 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc176f6b740 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626efaa7300 0x5626efe672a0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc176f6b740 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626efa924d0 0x5626efe6c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc176f6b740 1 --2- 172.21.15.63:0/3246186374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626efa8a2f0 0x5626efe6c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc176f6b740 1 -- 172.21.15.63:0/3246186374 >> 172.21.15.63:0/3246186374 conn(0x5626efe56430 msgr2=0x5626efe52050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.389+0000 7fc14ea00640 0 client.19365 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:53:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.390+0000 7fc176f6b740 1 -- 172.21.15.63:0/3246186374 shutdown_connections 2024-05-27T07:53:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.390+0000 7fc176f6b740 1 -- 172.21.15.63:0/3246186374 wait complete. 2024-05-27T07:53:59.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.390+0000 7fc176f6b740 20 client.19365 trim_cache size 0 max 16384 2024-05-27T07:53:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:53:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:53:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:53:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.391+0000 7fc16e800640 1 -- 172.21.15.63:0/1511924466 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1500b2950 msgr2=0x7fc1500b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.391+0000 7fc16e800640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1500b2950 0x7fc1500b4e60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc15400cfd0 tx=0x7fc1540046c0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.391+0000 7fc16e800640 1 -- 172.21.15.63:0/1511924466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc168109f60 msgr2=0x7fc16819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.391+0000 7fc16e800640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc168109f60 0x7fc16819edd0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fc160031040 tx=0x7fc160032bc0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.392+0000 7fc16e800640 1 -- 172.21.15.63:0/1511924466 shutdown_connections 2024-05-27T07:53:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.392+0000 7fc16e800640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc1500b2950 0x7fc1500b4e60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.392+0000 7fc16e800640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc168109f60 0x7fc16819edd0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.392+0000 7fc16e800640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc168109ac0 0x7fc16819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.392+0000 7fc16e800640 1 --2- 172.21.15.63:0/1511924466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc168104210 0x7fc16819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.392+0000 7fc16e800640 1 -- 172.21.15.63:0/1511924466 >> 172.21.15.63:0/1511924466 conn(0x7fc1680ff180 msgr2=0x7fc1681065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.392+0000 7fc16e800640 1 -- 172.21.15.63:0/1511924466 shutdown_connections 2024-05-27T07:53:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:53:59.392+0000 7fc16e800640 1 -- 172.21.15.63:0/1511924466 wait complete. 2024-05-27T07:53:59.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:53:59 smithi086 ceph-mon[34687]: pgmap v643: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:53:59.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:53:59 smithi149 ceph-mon[35408]: pgmap v643: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:53:59.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:53:59 smithi063 ceph-mon[31613]: pgmap v643: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:54:00.187 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:54:00.187 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:54:00.714 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:01.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:01 smithi063 ceph-mon[31613]: pgmap v644: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:54:01.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:01 smithi086 ceph-mon[34687]: pgmap v644: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:54:01.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:01 smithi149 ceph-mon[35408]: pgmap v644: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:54:03.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.213+0000 7f3423200700 1 -- 172.21.15.63:0/246942125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424109670 msgr2=0x7f3424110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.213+0000 7f3423200700 1 --2- 172.21.15.63:0/246942125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424109670 0x7f3424110fd0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f341c00b0d0 tx=0x7f341c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.213+0000 7f3423200700 1 -- 172.21.15.63:0/246942125 shutdown_connections 2024-05-27T07:54:03.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.213+0000 7f3423200700 1 --2- 172.21.15.63:0/246942125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424109670 0x7f3424110fd0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.213+0000 7f3423200700 1 --2- 172.21.15.63:0/246942125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34241043f0 0x7f3424104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.213+0000 7f3423200700 1 --2- 172.21.15.63:0/246942125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3424108d20 0x7f3424109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.213+0000 7f3423200700 1 -- 172.21.15.63:0/246942125 >> 172.21.15.63:0/246942125 conn(0x7f3424079d00 msgr2=0x7f342407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.213+0000 7f3423200700 1 -- 172.21.15.63:0/246942125 shutdown_connections 2024-05-27T07:54:03.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.213+0000 7f3423200700 1 -- 172.21.15.63:0/246942125 wait complete. 2024-05-27T07:54:03.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:03 smithi063 ceph-mon[31613]: pgmap v645: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T07:54:03.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.215+0000 7f3423200700 1 Processor -- start 2024-05-27T07:54:03.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.215+0000 7f3423200700 1 -- start start 2024-05-27T07:54:03.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.215+0000 7f3423200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34241043f0 0x7f342419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.216+0000 7f3423200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424108d20 0x7f342419d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.216+0000 7f3423200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3424109670 0x7f34241a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.216+0000 7f3423200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3424114150 con 0x7f3424108d20 2024-05-27T07:54:03.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.216+0000 7f3423200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3424113fd0 con 0x7f3424109670 2024-05-27T07:54:03.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.216+0000 7f3423200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34241142d0 con 0x7f34241043f0 2024-05-27T07:54:03.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.216+0000 7f3421400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424108d20 0x7f342419d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.216+0000 7f3421400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424108d20 0x7f342419d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56976/0 (socket says 172.21.15.63:56976) 2024-05-27T07:54:03.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.216+0000 7f3421400700 1 -- 172.21.15.63:0/3034266675 learned_addr learned my addr 172.21.15.63:0/3034266675 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:03.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.216+0000 7f3421e00700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34241043f0 0x7f342419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.216+0000 7f3422800700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3424109670 0x7f34241a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.217+0000 7f3421400700 1 -- 172.21.15.63:0/3034266675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34241043f0 msgr2=0x7f342419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.217+0000 7f3421400700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34241043f0 0x7f342419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.217+0000 7f3421400700 1 -- 172.21.15.63:0/3034266675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3424109670 msgr2=0x7f34241a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.217+0000 7f3421400700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3424109670 0x7f34241a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.217+0000 7f3421400700 1 -- 172.21.15.63:0/3034266675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34241a1bc0 con 0x7f3424108d20 2024-05-27T07:54:03.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.217+0000 7f3421400700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424108d20 0x7f342419d340 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f341000c9e0 tx=0x7f341000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.217+0000 7f341aa00700 1 -- 172.21.15.63:0/3034266675 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410007aa0 con 0x7f3424108d20 2024-05-27T07:54:03.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.218+0000 7f341aa00700 1 -- 172.21.15.63:0/3034266675 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3410007c40 con 0x7f3424108d20 2024-05-27T07:54:03.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.218+0000 7f3423200700 1 -- 172.21.15.63:0/3034266675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34241a1eb0 con 0x7f3424108d20 2024-05-27T07:54:03.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.218+0000 7f341aa00700 1 -- 172.21.15.63:0/3034266675 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410018400 con 0x7f3424108d20 2024-05-27T07:54:03.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.218+0000 7f3423200700 1 -- 172.21.15.63:0/3034266675 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34241a23f0 con 0x7f3424108d20 2024-05-27T07:54:03.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.223+0000 7f3437a81b80 1 Processor -- start 2024-05-27T07:54:03.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.223+0000 7f341aa00700 1 -- 172.21.15.63:0/3034266675 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f341000f450 con 0x7f3424108d20 2024-05-27T07:54:03.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.223+0000 7f3437a81b80 1 -- start start 2024-05-27T07:54:03.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.224+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 0x56401dbdcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.224+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.224+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd8a0 0x56401dbea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.224+0000 7f3437a81b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56401db0bec0 con 0x56401dbdd490 2024-05-27T07:54:03.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.224+0000 7f3437a81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56401dafba60 con 0x56401dbdd8a0 2024-05-27T07:54:03.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.224+0000 7f3437a81b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56401daffaa0 con 0x56401da5eec0 2024-05-27T07:54:03.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.224+0000 7f3421e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f3422800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 0x56401dbdcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f3422800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 0x56401dbdcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:38542/0 (socket says 172.21.15.63:38542) 2024-05-27T07:54:03.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f3422800700 1 -- 172.21.15.63:0/1652333665 learned_addr learned my addr 172.21.15.63:0/1652333665 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:03.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f3421400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd8a0 0x56401dbea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f341aa00700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34080b29f0 0x7f34080b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f3422800700 1 -- 172.21.15.63:0/1652333665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd8a0 msgr2=0x56401dbea0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f3422800700 1 --2- 172.21.15.63:0/1652333665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd8a0 0x56401dbea0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f3422800700 1 -- 172.21.15.63:0/1652333665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 msgr2=0x56401dbe7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f3422800700 1 --2- 172.21.15.63:0/1652333665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f3422800700 1 -- 172.21.15.63:0/1652333665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401dac44a0 con 0x56401da5eec0 2024-05-27T07:54:03.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.225+0000 7f3421e00700 1 --2- 172.21.15.63:0/1652333665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:03.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.226+0000 7f341aa00700 1 -- 172.21.15.63:0/3034266675 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f34100e28d0 con 0x7f3424108d20 2024-05-27T07:54:03.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.226+0000 7f3422800700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34080b29f0 0x7f34080b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.227+0000 7f3422800700 1 --2- 172.21.15.63:0/1652333665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 0x56401dbdcf50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f341c009e40 tx=0x7f341c005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f2152340cb30243 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.227+0000 7f3422800700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34080b29f0 0x7f34080b4f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f341c009610 tx=0x7f341c025000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.227+0000 7f340f400700 1 -- 172.21.15.63:0/1652333665 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c009470 con 0x56401da5eec0 2024-05-27T07:54:03.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.227+0000 7f340f400700 1 -- 172.21.15.63:0/1652333665 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f341c020050 con 0x56401da5eec0 2024-05-27T07:54:03.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.227+0000 7f340f400700 1 -- 172.21.15.63:0/1652333665 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c005790 con 0x56401da5eec0 2024-05-27T07:54:03.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.228+0000 7f3437a81b80 1 -- 172.21.15.63:0/1652333665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 msgr2=0x56401dbdcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.228+0000 7f3437a81b80 1 --2- 172.21.15.63:0/1652333665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 0x56401dbdcf50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f341c009e40 tx=0x7f341c005b00 comp rx=0 tx=0).stop 2024-05-27T07:54:03.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.229+0000 7f3437a81b80 1 -- 172.21.15.63:0/1652333665 shutdown_connections 2024-05-27T07:54:03.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.229+0000 7f3437a81b80 1 --2- 172.21.15.63:0/1652333665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd8a0 0x56401dbea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.229+0000 7f3437a81b80 1 --2- 172.21.15.63:0/1652333665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 0x56401dbdcf50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.229+0000 7f3437a81b80 1 --2- 172.21.15.63:0/1652333665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.229+0000 7f3437a81b80 1 -- 172.21.15.63:0/1652333665 >> 172.21.15.63:0/1652333665 conn(0x56401dabf040 msgr2=0x56401dbd8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.229+0000 7f3437a81b80 1 -- 172.21.15.63:0/1652333665 shutdown_connections 2024-05-27T07:54:03.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.229+0000 7f3437a81b80 1 -- 172.21.15.63:0/1652333665 wait complete. 2024-05-27T07:54:03.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.229+0000 7f3437a81b80 1 -- start start 2024-05-27T07:54:03.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.230+0000 7f3437a81b80 1 Processor -- start 2024-05-27T07:54:03.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.230+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 0x56401dbe7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.230+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.230+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd8a0 0x56401dbe8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.230+0000 7f3437a81b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56401da02020 con 0x56401dbdd490 2024-05-27T07:54:03.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.230+0000 7f3437a81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56401da9c7e0 con 0x56401dbdd8a0 2024-05-27T07:54:03.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.230+0000 7f3437a81b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56401dbea9b0 con 0x56401da5eec0 2024-05-27T07:54:03.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.231+0000 7f3437a81b80 10 client.0 _ll_register_callbacks cb 0x56401daa5cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:03.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.231+0000 7f3421400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.231+0000 7f3421400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56984/0 (socket says 172.21.15.63:56984) 2024-05-27T07:54:03.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.231+0000 7f3421400700 1 -- 172.21.15.63:0/2108711400 learned_addr learned my addr 172.21.15.63:0/2108711400 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:03.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.231+0000 7f3422800700 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd8a0 0x56401dbe8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.231+0000 7f3421400700 1 -- 172.21.15.63:0/2108711400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 msgr2=0x56401dbe7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.231+0000 7f3421400700 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 0x56401dbe7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.231+0000 7f3421400700 1 -- 172.21.15.63:0/2108711400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd8a0 msgr2=0x56401dbe8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.231+0000 7f3421400700 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd8a0 0x56401dbe8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.231+0000 7f3421400700 1 -- 172.21.15.63:0/2108711400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56401da4b950 con 0x56401dbdd490 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.232+0000 7f3421400700 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7bc0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f34100a6a00 tx=0x7f34100e01b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbe7fc38074644f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.232+0000 7f3403e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.232+0000 7f3403e00700 1 -- 172.21.15.63:0/2108711400 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34100a90e0 con 0x56401dbdd490 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.232+0000 7f3403e00700 1 -- 172.21.15.63:0/2108711400 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f34100a5d80 con 0x56401dbdd490 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.232+0000 7f3437a81b80 1 -- 172.21.15.63:0/2108711400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f341c020050 con 0x56401dbdd490 2024-05-27T07:54:03.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.232+0000 7f3403e00700 1 -- 172.21.15.63:0/2108711400 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f34100c3050 con 0x56401dbdd490 2024-05-27T07:54:03.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.233+0000 7f3437a81b80 10 client.19377 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:03.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.233+0000 7f3437a81b80 1 -- 172.21.15.63:0/2108711400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56401dacc0e0 con 0x56401dbdd490 2024-05-27T07:54:03.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.233+0000 7f3403e00700 1 -- 172.21.15.63:0/2108711400 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f341c020050 con 0x56401dbdd490 2024-05-27T07:54:03.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.233+0000 7f3403e00700 1 -- 172.21.15.63:0/2108711400 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f34100bebe0 con 0x56401dbdd490 2024-05-27T07:54:03.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.233+0000 7f3437a81b80 10 client.19377 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:03.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.233+0000 7f3437a81b80 10 client.19377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:03.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.233+0000 7f3437a81b80 20 client.19377 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:03.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.233+0000 7f3437a81b80 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56401dbf7a20 0x56401dbf9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.233+0000 7f3437a81b80 4 client.19377 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:03.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.233+0000 7f3437a81b80 1 -- 172.21.15.63:0/2108711400 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56401db0de10 con 0x56401dbf7a20 2024-05-27T07:54:03.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.234+0000 7f3421e00700 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56401dbf7a20 0x56401dbf9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.235+0000 7f3421e00700 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56401dbf7a20 0x56401dbf9e40 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5553ee589d66608 server_cookie=70e3302b6353eb1d in_seq=0 out_seq=0 2024-05-27T07:54:03.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.235+0000 7f3403e00700 10 client.19377 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:03.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.237+0000 7f3403e00700 1 -- 172.21.15.63:0/2108711400 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56401db0de10 con 0x56401dbf7a20 2024-05-27T07:54:03.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.237+0000 7f3403e00700 10 client.19377 handle_command_reply: tid=0 2024-05-27T07:54:03.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.237+0000 7f3437a81b80 1 client.19377 shutdown 2024-05-27T07:54:03.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 1 -- 172.21.15.63:0/2108711400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 msgr2=0x56401dbe7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7bc0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f34100a6a00 tx=0x7f34100e01b0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 1 -- 172.21.15.63:0/2108711400 shutdown_connections 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56401dbf7a20 0x56401dbf9e40 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd8a0 0x56401dbe8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401da5eec0 0x56401dbe7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 1 --2- 172.21.15.63:0/2108711400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd490 0x56401dbe7bc0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 1 -- 172.21.15.63:0/2108711400 >> 172.21.15.63:0/2108711400 conn(0x56401dabf040 msgr2=0x56401dbd8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3403e00700 0 client.19377 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 1 -- 172.21.15.63:0/2108711400 shutdown_connections 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 1 -- 172.21.15.63:0/2108711400 wait complete. 2024-05-27T07:54:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.238+0000 7f3437a81b80 20 client.19377 trim_cache size 0 max 16384 2024-05-27T07:54:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.262+0000 7f3437a81b80 1 Processor -- start 2024-05-27T07:54:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.262+0000 7f3437a81b80 1 -- start start 2024-05-27T07:54:03.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401da5eec0 0x56401dbe9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401dbdd490 0x56401dbe1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd8a0 0x56401dbff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3437a81b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34241b4d20 con 0x56401dbdd8a0 2024-05-27T07:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3437a81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34241b25b0 con 0x56401da5eec0 2024-05-27T07:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3437a81b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34241b0420 con 0x56401dbdd490 2024-05-27T07:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3422800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd8a0 0x56401dbff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3422800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd8a0 0x56401dbff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56992/0 (socket says 172.21.15.63:56992) 2024-05-27T07:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3422800700 1 -- 172.21.15.63:0/1029172997 learned_addr learned my addr 172.21.15.63:0/1029172997 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:03.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3421400700 1 --2- 172.21.15.63:0/1029172997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401dbdd490 0x56401dbe1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3421e00700 1 --2- 172.21.15.63:0/1029172997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401da5eec0 0x56401dbe9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3422800700 1 -- 172.21.15.63:0/1029172997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401dbdd490 msgr2=0x56401dbe1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.263+0000 7f3422800700 1 --2- 172.21.15.63:0/1029172997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401dbdd490 0x56401dbe1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.264+0000 7f3422800700 1 -- 172.21.15.63:0/1029172997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401da5eec0 msgr2=0x56401dbe9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.264+0000 7f3422800700 1 --2- 172.21.15.63:0/1029172997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401da5eec0 0x56401dbe9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.264+0000 7f3422800700 1 -- 172.21.15.63:0/1029172997 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401dbda3d0 con 0x56401dbdd8a0 2024-05-27T07:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.264+0000 7f3421e00700 1 --2- 172.21.15.63:0/1029172997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401da5eec0 0x56401dbe9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.264+0000 7f3422800700 1 --2- 172.21.15.63:0/1029172997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd8a0 0x56401dbff640 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f341c0098d0 tx=0x7f341c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321850276c0c189a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.264+0000 7f340ea00700 1 -- 172.21.15.63:0/1029172997 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c03d070 con 0x56401dbdd8a0 2024-05-27T07:54:03.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.264+0000 7f340ea00700 1 -- 172.21.15.63:0/1029172997 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f341c035070 con 0x56401dbdd8a0 2024-05-27T07:54:03.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.265+0000 7f340ea00700 1 -- 172.21.15.63:0/1029172997 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c0057e0 con 0x56401dbdd8a0 2024-05-27T07:54:03.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.265+0000 7f3437a81b80 1 -- 172.21.15.63:0/1029172997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd8a0 msgr2=0x56401dbff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.265+0000 7f3437a81b80 1 --2- 172.21.15.63:0/1029172997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd8a0 0x56401dbff640 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f341c0098d0 tx=0x7f341c009860 comp rx=0 tx=0).stop 2024-05-27T07:54:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.266+0000 7f3437a81b80 1 -- 172.21.15.63:0/1029172997 shutdown_connections 2024-05-27T07:54:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.266+0000 7f3437a81b80 1 --2- 172.21.15.63:0/1029172997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401dbdd8a0 0x56401dbff640 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.266+0000 7f3437a81b80 1 --2- 172.21.15.63:0/1029172997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401da5eec0 0x56401dbe9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.266+0000 7f3437a81b80 1 --2- 172.21.15.63:0/1029172997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401dbdd490 0x56401dbe1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.266+0000 7f3437a81b80 1 -- 172.21.15.63:0/1029172997 >> 172.21.15.63:0/1029172997 conn(0x56401dabf040 msgr2=0x56401dbf3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.266+0000 7f3437a81b80 1 -- 172.21.15.63:0/1029172997 shutdown_connections 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.266+0000 7f3437a81b80 1 -- 172.21.15.63:0/1029172997 wait complete. 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.266+0000 7f3437a81b80 1 -- start start 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.267+0000 7f3437a81b80 1 Processor -- start 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.267+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401da5eec0 0x56401dbec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.267+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd490 0x56401dbecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.267+0000 7f3437a81b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401dbdd8a0 0x56401dc5a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.267+0000 7f3437a81b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34241b0a70 con 0x56401da5eec0 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.267+0000 7f3437a81b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34241b1ee0 con 0x56401dbdd490 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.267+0000 7f3437a81b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34241b1140 con 0x56401dbdd8a0 2024-05-27T07:54:03.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.267+0000 7f3437a81b80 10 client.0 _ll_register_callbacks cb 0x56401daa5cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.268+0000 7f3421e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401da5eec0 0x56401dbec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.268+0000 7f3421e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401da5eec0 0x56401dbec660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57000/0 (socket says 172.21.15.63:57000) 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.268+0000 7f3421e00700 1 -- 172.21.15.63:0/3181287273 learned_addr learned my addr 172.21.15.63:0/3181287273 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.268+0000 7f3421400700 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd490 0x56401dbecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.268+0000 7f3422800700 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401dbdd8a0 0x56401dc5a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.268+0000 7f3421e00700 1 -- 172.21.15.63:0/3181287273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401dbdd8a0 msgr2=0x56401dc5a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.268+0000 7f3421e00700 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401dbdd8a0 0x56401dc5a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.268+0000 7f3421e00700 1 -- 172.21.15.63:0/3181287273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd490 msgr2=0x56401dbecba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.268+0000 7f3421e00700 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd490 0x56401dbecba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.268+0000 7f3421e00700 1 -- 172.21.15.63:0/3181287273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56401dc5abb0 con 0x56401da5eec0 2024-05-27T07:54:03.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.269+0000 7f3421e00700 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401da5eec0 0x56401dbec660 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x56401dbe6f00 tx=0x7f3414010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc3e01c8c4dbce5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.269+0000 7f3403e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.269+0000 7f3403e00700 1 -- 172.21.15.63:0/3181287273 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414004520 con 0x56401da5eec0 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.269+0000 7f3403e00700 1 -- 172.21.15.63:0/3181287273 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9483+0+0 (secure 0 0 0) 0x7f3414003680 con 0x56401da5eec0 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.269+0000 7f3437a81b80 1 -- 172.21.15.63:0/3181287273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f341c035070 con 0x56401da5eec0 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.269+0000 7f3403e00700 1 -- 172.21.15.63:0/3181287273 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f341402c050 con 0x56401da5eec0 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.269+0000 7f3437a81b80 10 client.19383 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.269+0000 7f3437a81b80 1 -- 172.21.15.63:0/3181287273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56401dc63020 con 0x56401da5eec0 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.270+0000 7f3403e00700 1 -- 172.21.15.63:0/3181287273 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f341c035070 con 0x56401da5eec0 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.270+0000 7f3403e00700 1 -- 172.21.15.63:0/3181287273 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f341401d950 con 0x56401da5eec0 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.270+0000 7f3437a81b80 10 client.19383 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.270+0000 7f3437a81b80 10 client.19383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.270+0000 7f3437a81b80 20 client.19383 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.270+0000 7f3437a81b80 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56401dc63730 0x56401dc65b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.270+0000 7f3437a81b80 4 client.19383 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.270+0000 7f3437a81b80 1 -- 172.21.15.63:0/3181287273 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56401db075f0 con 0x56401dc63730 2024-05-27T07:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.271+0000 7f3421400700 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56401dc63730 0x56401dc65b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.272+0000 7f3421400700 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56401dc63730 0x56401dc65b50 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca674c86de089421 server_cookie=a7c1e3bfb5998d1f in_seq=0 out_seq=0 2024-05-27T07:54:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.272+0000 7f3403e00700 10 client.19383 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:03.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.275+0000 7f3403e00700 1 -- 172.21.15.63:0/3181287273 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56401db075f0 con 0x56401dc63730 2024-05-27T07:54:03.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.275+0000 7f3403e00700 10 client.19383 handle_command_reply: tid=0 2024-05-27T07:54:03.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.275+0000 7f3437a81b80 1 client.19383 shutdown 2024-05-27T07:54:03.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.276+0000 7f3437a81b80 1 -- 172.21.15.63:0/3181287273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401da5eec0 msgr2=0x56401dbec660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.276+0000 7f3437a81b80 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401da5eec0 0x56401dbec660 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x56401dbe6f00 tx=0x7f3414010990 comp rx=0 tx=0).stop 2024-05-27T07:54:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.276+0000 7f3437a81b80 1 -- 172.21.15.63:0/3181287273 shutdown_connections 2024-05-27T07:54:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.276+0000 7f3437a81b80 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56401dc63730 0x56401dc65b50 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.276+0000 7f3437a81b80 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56401dbdd8a0 0x56401dc5a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.276+0000 7f3437a81b80 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56401da5eec0 0x56401dbec660 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.276+0000 7f3437a81b80 1 --2- 172.21.15.63:0/3181287273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56401dbdd490 0x56401dbecba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.276+0000 7f3403e00700 0 client.19383 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.276+0000 7f3437a81b80 1 -- 172.21.15.63:0/3181287273 >> 172.21.15.63:0/3181287273 conn(0x56401dabf040 msgr2=0x56401dbd8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.276+0000 7f3437a81b80 1 -- 172.21.15.63:0/3181287273 shutdown_connections 2024-05-27T07:54:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.277+0000 7f3437a81b80 1 -- 172.21.15.63:0/3181287273 wait complete. 2024-05-27T07:54:03.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.277+0000 7f3437a81b80 20 client.19383 trim_cache size 0 max 16384 2024-05-27T07:54:03.278 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:54:03.278 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:54:03.278 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "e0c40d31-e6cc-45aa-bf8c-cf183300e15d", 2024-05-27T07:54:03.278 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:54:03.279 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:54:03.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.277+0000 7f3423200700 1 -- 172.21.15.63:0/3034266675 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34080b29f0 msgr2=0x7f34080b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.277+0000 7f3423200700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34080b29f0 0x7f34080b4f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f341c009610 tx=0x7f341c025000 comp rx=0 tx=0).stop 2024-05-27T07:54:03.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.277+0000 7f3423200700 1 -- 172.21.15.63:0/3034266675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424108d20 msgr2=0x7f342419d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.277+0000 7f3423200700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424108d20 0x7f342419d340 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f341000c9e0 tx=0x7f341000ccf0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.278+0000 7f3423200700 1 -- 172.21.15.63:0/3034266675 shutdown_connections 2024-05-27T07:54:03.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.278+0000 7f3423200700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34080b29f0 0x7f34080b4f00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.278+0000 7f3423200700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3424109670 0x7f34241a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.278+0000 7f3423200700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34241043f0 0x7f342419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.278+0000 7f3423200700 1 --2- 172.21.15.63:0/3034266675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3424108d20 0x7f342419d340 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.278+0000 7f3423200700 1 -- 172.21.15.63:0/3034266675 >> 172.21.15.63:0/3034266675 conn(0x7f3424079d00 msgr2=0x7f342410f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.278+0000 7f3423200700 1 -- 172.21.15.63:0/3034266675 shutdown_connections 2024-05-27T07:54:03.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:03.278+0000 7f3423200700 1 -- 172.21.15.63:0/3034266675 wait complete. 2024-05-27T07:54:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:03 smithi086 ceph-mon[34687]: pgmap v645: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T07:54:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:03 smithi149 ceph-mon[35408]: pgmap v645: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T07:54:04.243 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:54:04.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:04 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:54:04.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:04 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:04 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:54:04.462 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:04 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:54:04.462 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:04 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.462 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:04 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:54:04.463 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:04 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.463 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:04 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:54:04.463 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:04 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.463 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:04 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:04 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:54:04.466 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:04 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.467 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:04 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:54:04.467 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:04 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:54:04.467 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:04 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.467 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:04 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:54:04.467 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:04 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.467 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:04 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:54:04.467 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:04 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.468 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:04 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:04 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:54:04.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:04 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:04 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:54:04.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:04 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:54:04.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:04 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:04 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:54:04.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:04 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.469 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:04 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:54:04.469 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:04 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.469 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:04 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:04.786 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:05.202 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:54:05.202 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:05.202 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:54:05.203 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:05.203 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:54:05.203 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:05.203 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:05 smithi063 ceph-mon[31613]: pgmap v646: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 27 op/s 2024-05-27T07:54:05.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:54:05.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:05.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:54:05.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:05.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:54:05.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:05.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:05 smithi086 ceph-mon[34687]: pgmap v646: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 27 op/s 2024-05-27T07:54:05.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:54:05.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:05.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:54:05.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:05.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:54:05.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:05.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:05 smithi149 ceph-mon[35408]: pgmap v646: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 27 op/s 2024-05-27T07:54:07.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:07 smithi063 ceph-mon[31613]: pgmap v647: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 27 op/s 2024-05-27T07:54:07.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.240+0000 7f28dc200700 1 -- 172.21.15.63:0/3462174716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d410c5a0 msgr2=0x7f28d410c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:07.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.240+0000 7f28dc200700 1 --2- 172.21.15.63:0/3462174716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d410c5a0 0x7f28d410c9b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009990 tx=0x7f28c8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.240+0000 7f28dc200700 1 -- 172.21.15.63:0/3462174716 shutdown_connections 2024-05-27T07:54:07.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.240+0000 7f28dc200700 1 --2- 172.21.15.63:0/3462174716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28d4106eb0 0x7f28d4113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.240+0000 7f28dc200700 1 --2- 172.21.15.63:0/3462174716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d410c5a0 0x7f28d410c9b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.240+0000 7f28dc200700 1 --2- 172.21.15.63:0/3462174716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28d4106560 0x7f28d4106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.240+0000 7f28dc200700 1 -- 172.21.15.63:0/3462174716 >> 172.21.15.63:0/3462174716 conn(0x7f28d4102040 msgr2=0x7f28d4104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:07.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.240+0000 7f28dc200700 1 -- 172.21.15.63:0/3462174716 shutdown_connections 2024-05-27T07:54:07.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.240+0000 7f28dc200700 1 -- 172.21.15.63:0/3462174716 wait complete. 2024-05-27T07:54:07.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.241+0000 7f28dc200700 1 Processor -- start 2024-05-27T07:54:07.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.242+0000 7f28dc200700 1 -- start start 2024-05-27T07:54:07.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.242+0000 7f28dc200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106560 0x7f28d419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.242+0000 7f28dc200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28d4106eb0 0x7f28d419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.242+0000 7f28dc200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28d410c5a0 0x7f28d41a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.242+0000 7f28dc200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28d41165b0 con 0x7f28d410c5a0 2024-05-27T07:54:07.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.242+0000 7f28dc200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4116430 con 0x7f28d4106560 2024-05-27T07:54:07.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.242+0000 7f28dc200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4116730 con 0x7f28d4106eb0 2024-05-27T07:54:07.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28da200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28d410c5a0 0x7f28d41a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28d9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106560 0x7f28d419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28d8e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28d4106eb0 0x7f28d419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28d9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106560 0x7f28d419cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56458/0 (socket says 172.21.15.63:56458) 2024-05-27T07:54:07.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28d9800700 1 -- 172.21.15.63:0/2721864691 learned_addr learned my addr 172.21.15.63:0/2721864691 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:07.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28d8e00700 1 -- 172.21.15.63:0/2721864691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106560 msgr2=0x7f28d419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:07.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28d8e00700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106560 0x7f28d419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28d8e00700 1 -- 172.21.15.63:0/2721864691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28d410c5a0 msgr2=0x7f28d41a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:07.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28d8e00700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28d410c5a0 0x7f28d41a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28d8e00700 1 -- 172.21.15.63:0/2721864691 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d41a1e30 con 0x7f28d4106eb0 2024-05-27T07:54:07.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.243+0000 7f28da200700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28d410c5a0 0x7f28d41a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:07.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.244+0000 7f28d9800700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106560 0x7f28d419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:07.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.244+0000 7f28d8e00700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28d4106eb0 0x7f28d419d500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f28d4075b70 tx=0x7f28c000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:07.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.244+0000 7f28d2000700 1 -- 172.21.15.63:0/2721864691 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c000cb70 con 0x7f28d4106eb0 2024-05-27T07:54:07.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.245+0000 7f28dc200700 1 -- 172.21.15.63:0/2721864691 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d41a2120 con 0x7f28d4106eb0 2024-05-27T07:54:07.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.245+0000 7f28dc200700 1 -- 172.21.15.63:0/2721864691 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d41a9a40 con 0x7f28d4106eb0 2024-05-27T07:54:07.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.245+0000 7f28d2000700 1 -- 172.21.15.63:0/2721864691 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28c00044b0 con 0x7f28d4106eb0 2024-05-27T07:54:07.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.245+0000 7f28d2000700 1 -- 172.21.15.63:0/2721864691 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0003ca0 con 0x7f28d4106eb0 2024-05-27T07:54:07.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.249+0000 7f28d2000700 1 -- 172.21.15.63:0/2721864691 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f28c0003e40 con 0x7f28d4106eb0 2024-05-27T07:54:07.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.250+0000 7f28ead4fb80 1 Processor -- start 2024-05-27T07:54:07.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.251+0000 7f28ead4fb80 1 -- start start 2024-05-27T07:54:07.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.251+0000 7f28d2000700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28bc0b2af0 0x7f28bc0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.251+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x5605631887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.251+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563188d30 0x560563182870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.251+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563182db0 0x5605631831c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.251+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5605630b0e40 con 0x560563003e40 2024-05-27T07:54:07.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.251+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5605630a09e0 con 0x560563188d30 2024-05-27T07:54:07.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.251+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5605630a4a20 con 0x560563182db0 2024-05-27T07:54:07.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.251+0000 7f28d2000700 1 -- 172.21.15.63:0/2721864691 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9571+0+0 (secure 0 0 0) 0x7f28c00e2b30 con 0x7f28d4106eb0 2024-05-27T07:54:07.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.252+0000 7f28da200700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28bc0b2af0 0x7f28bc0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.252+0000 7f28d8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563188d30 0x560563182870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.252+0000 7f28d8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563188d30 0x560563182870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56472/0 (socket says 172.21.15.63:56472) 2024-05-27T07:54:07.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.252+0000 7f28d8e00700 1 -- 172.21.15.63:0/3985923542 learned_addr learned my addr 172.21.15.63:0/3985923542 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:07.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.252+0000 7f28d9800700 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x5605631887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.252+0000 7f28da200700 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563182db0 0x5605631831c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.252+0000 7f28d8e00700 1 -- 172.21.15.63:0/3985923542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563182db0 msgr2=0x5605631831c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:07.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.252+0000 7f28d8e00700 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563182db0 0x5605631831c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.252+0000 7f28d8e00700 1 -- 172.21.15.63:0/3985923542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 msgr2=0x5605631887f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:07.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.253+0000 7f28d8e00700 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x5605631887f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.253+0000 7f28d8e00700 1 -- 172.21.15.63:0/3985923542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560563069420 con 0x560563188d30 2024-05-27T07:54:07.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.253+0000 7f28da200700 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563182db0 0x5605631831c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:07.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.253+0000 7f28d9800700 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x5605631887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:07.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.253+0000 7f28d8e00700 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563188d30 0x560563182870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f28c00a5a10 tx=0x7f28c00a9890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4859951ac166021 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:07.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.253+0000 7f28da200700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28bc0b2af0 0x7f28bc0b5000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f28cc015c00 tx=0x7f28cc009920 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:07.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.253+0000 7f28c6a00700 1 -- 172.21.15.63:0/3985923542 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00b2140 con 0x560563188d30 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.253+0000 7f28c6a00700 1 -- 172.21.15.63:0/3985923542 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28c00a94a0 con 0x560563188d30 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.253+0000 7f28c6a00700 1 -- 172.21.15.63:0/3985923542 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00cc0d0 con 0x560563188d30 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.254+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3985923542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563188d30 msgr2=0x560563182870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.254+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563188d30 0x560563182870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f28c00a5a10 tx=0x7f28c00a9890 comp rx=0 tx=0).stop 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.255+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3985923542 shutdown_connections 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.255+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563182db0 0x5605631831c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.255+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x5605631887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.255+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/3985923542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563188d30 0x560563182870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.255+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3985923542 >> 172.21.15.63:0/3985923542 conn(0x560563063fc0 msgr2=0x56056317d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.255+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3985923542 shutdown_connections 2024-05-27T07:54:07.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.255+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3985923542 wait complete. 2024-05-27T07:54:07.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.256+0000 7f28ead4fb80 1 -- start start 2024-05-27T07:54:07.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.256+0000 7f28ead4fb80 1 Processor -- start 2024-05-27T07:54:07.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563003e40 0x56056318c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563182db0 0x56056318ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563188d30 0x56056318cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560562fa6fa0 con 0x560563182db0 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560563041760 con 0x560563003e40 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56056318f800 con 0x560563188d30 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28ead4fb80 10 client.0 _ll_register_callbacks cb 0x56056304ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28d8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563182db0 0x56056318ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28d9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563003e40 0x56056318c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28da200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563188d30 0x56056318cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28d9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563003e40 0x56056318c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56486/0 (socket says 172.21.15.63:56486) 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28d9800700 1 -- 172.21.15.63:0/2475695343 learned_addr learned my addr 172.21.15.63:0/2475695343 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:07.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.257+0000 7f28d8e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563182db0 0x56056318ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57042/0 (socket says 172.21.15.63:57042) 2024-05-27T07:54:07.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.258+0000 7f28d8e00700 1 -- 172.21.15.63:0/2475695343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563188d30 msgr2=0x56056318cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.258+0000 7f28d8e00700 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563188d30 0x56056318cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.258+0000 7f28d8e00700 1 -- 172.21.15.63:0/2475695343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563003e40 msgr2=0x56056318c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.258+0000 7f28d8e00700 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563003e40 0x56056318c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.258+0000 7f28d8e00700 1 -- 172.21.15.63:0/2475695343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560562ff08d0 con 0x560563182db0 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.258+0000 7f28d8e00700 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563182db0 0x56056318ca80 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f28c00a5a10 tx=0x7f28c00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2e7f58269c7d36 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.258+0000 7f28bb400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.259+0000 7f28bb400700 1 -- 172.21.15.63:0/2475695343 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00b2140 con 0x560563182db0 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.259+0000 7f28ead4fb80 1 -- 172.21.15.63:0/2475695343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c00a94a0 con 0x560563182db0 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.259+0000 7f28bb400700 1 -- 172.21.15.63:0/2475695343 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9571+0+0 (secure 0 0 0) 0x7f28c00a9bf0 con 0x560563182db0 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.259+0000 7f28bb400700 1 -- 172.21.15.63:0/2475695343 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f28c00cc2e0 con 0x560563182db0 2024-05-27T07:54:07.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.259+0000 7f28ead4fb80 10 client.19395 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:07.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.259+0000 7f28ead4fb80 1 -- 172.21.15.63:0/2475695343 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560563071060 con 0x560563182db0 2024-05-27T07:54:07.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.259+0000 7f28bb400700 1 -- 172.21.15.63:0/2475695343 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28c00a94a0 con 0x560563182db0 2024-05-27T07:54:07.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.260+0000 7f28bb400700 1 -- 172.21.15.63:0/2475695343 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f28c00b2360 con 0x560563182db0 2024-05-27T07:54:07.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.260+0000 7f28ead4fb80 10 client.19395 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:07.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.260+0000 7f28ead4fb80 10 client.19395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:07.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.260+0000 7f28ead4fb80 20 client.19395 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:07.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.260+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56056319c810 0x56056319ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.260+0000 7f28ead4fb80 4 client.19395 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:07.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.260+0000 7f28ead4fb80 1 -- 172.21.15.63:0/2475695343 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605630b2d90 con 0x56056319c810 2024-05-27T07:54:07.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.261+0000 7f28d9800700 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56056319c810 0x56056319ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.261+0000 7f28d9800700 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56056319c810 0x56056319ec30 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd132896b58cebc server_cookie=57e40f451bbe19b2 in_seq=0 out_seq=0 2024-05-27T07:54:07.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.261+0000 7f28bb400700 10 client.19395 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:07.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.262+0000 7f28bb400700 1 -- 172.21.15.63:0/2475695343 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5605630b2d90 con 0x56056319c810 2024-05-27T07:54:07.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.262+0000 7f28bb400700 10 client.19395 handle_command_reply: tid=0 2024-05-27T07:54:07.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 client.19395 shutdown 2024-05-27T07:54:07.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 -- 172.21.15.63:0/2475695343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563182db0 msgr2=0x56056318ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:07.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563182db0 0x56056318ca80 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f28c00a5a10 tx=0x7f28c00036f0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 -- 172.21.15.63:0/2475695343 shutdown_connections 2024-05-27T07:54:07.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56056319c810 0x56056319ec30 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563188d30 0x56056318cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563003e40 0x56056318c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/2475695343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563182db0 0x56056318ca80 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28bb400700 0 client.19395 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:07.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 -- 172.21.15.63:0/2475695343 >> 172.21.15.63:0/2475695343 conn(0x560563063fc0 msgr2=0x56056317d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:07.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 -- 172.21.15.63:0/2475695343 shutdown_connections 2024-05-27T07:54:07.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.263+0000 7f28ead4fb80 1 -- 172.21.15.63:0/2475695343 wait complete. 2024-05-27T07:54:07.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.264+0000 7f28ead4fb80 20 client.19395 trim_cache size 0 max 16384 2024-05-27T07:54:07.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.292+0000 7f28ead4fb80 1 Processor -- start 2024-05-27T07:54:07.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.292+0000 7f28ead4fb80 1 -- start start 2024-05-27T07:54:07.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.292+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x560563182870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.292+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 0x560563198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.292+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563188d30 0x56056318e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.292+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28b40118d0 con 0x560563003e40 2024-05-27T07:54:07.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.292+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28b400f160 con 0x560563182db0 2024-05-27T07:54:07.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.292+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f28b400cfd0 con 0x560563188d30 2024-05-27T07:54:07.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.292+0000 7f28d9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x560563182870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.293+0000 7f28d9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x560563182870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57052/0 (socket says 172.21.15.63:57052) 2024-05-27T07:54:07.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.293+0000 7f28d9800700 1 -- 172.21.15.63:0/1654907523 learned_addr learned my addr 172.21.15.63:0/1654907523 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:07.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.293+0000 7f28d8e00700 1 --2- 172.21.15.63:0/1654907523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 0x560563198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.293+0000 7f28da200700 1 --2- 172.21.15.63:0/1654907523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563188d30 0x56056318e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.293+0000 7f28d9800700 1 -- 172.21.15.63:0/1654907523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563188d30 msgr2=0x56056318e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:07.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.293+0000 7f28d9800700 1 --2- 172.21.15.63:0/1654907523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563188d30 0x56056318e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.293+0000 7f28d9800700 1 -- 172.21.15.63:0/1654907523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 msgr2=0x560563198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:07.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.293+0000 7f28d9800700 1 --2- 172.21.15.63:0/1654907523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 0x560563198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.293+0000 7f28d9800700 1 -- 172.21.15.63:0/1654907523 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56056318ede0 con 0x560563003e40 2024-05-27T07:54:07.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.293+0000 7f28d8e00700 1 --2- 172.21.15.63:0/1654907523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 0x560563198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:07.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.294+0000 7f28d9800700 1 --2- 172.21.15.63:0/1654907523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x560563182870 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f28c80098e0 tx=0x7f28c8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6aa0653ce91b46 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:07.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.294+0000 7f28c6000700 1 -- 172.21.15.63:0/1654907523 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8005690 con 0x560563003e40 2024-05-27T07:54:07.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.294+0000 7f28c6000700 1 -- 172.21.15.63:0/1654907523 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28c8003b10 con 0x560563003e40 2024-05-27T07:54:07.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.294+0000 7f28c6000700 1 -- 172.21.15.63:0/1654907523 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c800f8a0 con 0x560563003e40 2024-05-27T07:54:07.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.295+0000 7f28ead4fb80 1 -- 172.21.15.63:0/1654907523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 msgr2=0x560563182870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:07.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.295+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/1654907523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x560563182870 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f28c80098e0 tx=0x7f28c8005040 comp rx=0 tx=0).stop 2024-05-27T07:54:07.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.295+0000 7f28ead4fb80 1 -- 172.21.15.63:0/1654907523 shutdown_connections 2024-05-27T07:54:07.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.295+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/1654907523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563188d30 0x56056318e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.295+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/1654907523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563003e40 0x560563182870 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.295+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/1654907523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 0x560563198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.295+0000 7f28ead4fb80 1 -- 172.21.15.63:0/1654907523 >> 172.21.15.63:0/1654907523 conn(0x560563063fc0 msgr2=0x56056318c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:07.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.295+0000 7f28ead4fb80 1 -- 172.21.15.63:0/1654907523 shutdown_connections 2024-05-27T07:54:07.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.296+0000 7f28ead4fb80 1 -- 172.21.15.63:0/1654907523 wait complete. 2024-05-27T07:54:07.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.296+0000 7f28ead4fb80 1 -- start start 2024-05-27T07:54:07.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.296+0000 7f28ead4fb80 1 Processor -- start 2024-05-27T07:54:07.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.297+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563003e40 0x5605631a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.297+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 0x5605631a0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.297+0000 7f28ead4fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563188d30 0x560563206080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.297+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28b400d620 con 0x560563188d30 2024-05-27T07:54:07.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.297+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28b400ea90 con 0x560563182db0 2024-05-27T07:54:07.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.297+0000 7f28ead4fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f28b400dcf0 con 0x560563003e40 2024-05-27T07:54:07.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.297+0000 7f28ead4fb80 10 client.0 _ll_register_callbacks cb 0x56056304ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:07.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.298+0000 7f28da200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563188d30 0x560563206080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.298+0000 7f28da200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563188d30 0x560563206080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57066/0 (socket says 172.21.15.63:57066) 2024-05-27T07:54:07.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.298+0000 7f28da200700 1 -- 172.21.15.63:0/3405409622 learned_addr learned my addr 172.21.15.63:0/3405409622 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:07.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.298+0000 7f28d9800700 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563003e40 0x5605631a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.298+0000 7f28d8e00700 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 0x5605631a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.298+0000 7f28da200700 1 -- 172.21.15.63:0/3405409622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563003e40 msgr2=0x5605631a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:07.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.298+0000 7f28da200700 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563003e40 0x5605631a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.298+0000 7f28da200700 1 -- 172.21.15.63:0/3405409622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 msgr2=0x5605631a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:07.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.298+0000 7f28da200700 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 0x5605631a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.298+0000 7f28da200700 1 -- 172.21.15.63:0/3405409622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605631a0fb0 con 0x560563188d30 2024-05-27T07:54:07.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.299+0000 7f28da200700 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563188d30 0x560563206080 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0156e0 tx=0x7f28cc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926955e4b96964ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:07.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.299+0000 7f28bb400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:07.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.299+0000 7f28bb400700 1 -- 172.21.15.63:0/3405409622 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc029070 con 0x560563188d30 2024-05-27T07:54:07.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.299+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3405409622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c8003b10 con 0x560563188d30 2024-05-27T07:54:07.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.299+0000 7f28bb400700 1 -- 172.21.15.63:0/3405409622 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9571+0+0 (secure 0 0 0) 0x7f28cc007da0 con 0x560563188d30 2024-05-27T07:54:07.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28bb400700 1 -- 172.21.15.63:0/3405409622 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f28cc003a00 con 0x560563188d30 2024-05-27T07:54:07.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28ead4fb80 10 client.19401 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:07.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3405409622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5605632075c0 con 0x560563188d30 2024-05-27T07:54:07.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28bb400700 1 -- 172.21.15.63:0/3405409622 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28c8003b10 con 0x560563188d30 2024-05-27T07:54:07.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28bb400700 1 -- 172.21.15.63:0/3405409622 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f28cc019460 con 0x560563188d30 2024-05-27T07:54:07.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28ead4fb80 10 client.19401 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:07.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28ead4fb80 10 client.19401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:07.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28ead4fb80 20 client.19401 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:07.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560563207d00 0x56056320a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:07.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28ead4fb80 4 client.19401 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:54:07.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.300+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3405409622 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5605630ac570 con 0x560563207d00 2024-05-27T07:54:07.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.301+0000 7f28d9800700 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560563207d00 0x56056320a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:07.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.302+0000 7f28bb400700 10 client.19401 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:07.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.302+0000 7f28d9800700 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560563207d00 0x56056320a120 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a30433fa45f4825 server_cookie=c8bed15bac4db55a in_seq=0 out_seq=0 2024-05-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28bb400700 1 -- 172.21.15.63:0/3405409622 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5605630ac570 con 0x560563207d00 2024-05-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28bb400700 10 client.19401 handle_command_reply: tid=0 2024-05-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28ead4fb80 1 client.19401 shutdown 2024-05-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3405409622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563188d30 msgr2=0x560563206080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563188d30 0x560563206080 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0156e0 tx=0x7f28cc004b40 comp rx=0 tx=0).stop 2024-05-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3405409622 shutdown_connections 2024-05-27T07:54:07.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560563207d00 0x56056320a120 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560563188d30 0x560563206080 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560563003e40 0x5605631a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28ead4fb80 1 --2- 172.21.15.63:0/3405409622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560563182db0 0x5605631a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3405409622 >> 172.21.15.63:0/3405409622 conn(0x560563063fc0 msgr2=0x56056318cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28bb400700 0 client.19401 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.303+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3405409622 shutdown_connections 2024-05-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.304+0000 7f28ead4fb80 1 -- 172.21.15.63:0/3405409622 wait complete. 2024-05-27T07:54:07.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.304+0000 7f28ead4fb80 20 client.19401 trim_cache size 0 max 16384 2024-05-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi063.stdout: "e0c40d31-e6cc-45aa-bf8c-cf183300e15d": { 2024-05-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "e0c40d31-e6cc-45aa-bf8c-cf183300e15d", 2024-05-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:54:07.306 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 -- 172.21.15.63:0/2721864691 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28bc0b2af0 msgr2=0x7f28bc0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28bc0b2af0 0x7f28bc0b5000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f28cc015c00 tx=0x7f28cc009920 comp rx=0 tx=0).stop 2024-05-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 -- 172.21.15.63:0/2721864691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28d4106eb0 msgr2=0x7f28d419d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28d4106eb0 0x7f28d419d500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f28d4075b70 tx=0x7f28c000eca0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 -- 172.21.15.63:0/2721864691 shutdown_connections 2024-05-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f28bc0b2af0 0x7f28bc0b5000 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28d410c5a0 0x7f28d41a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d4106560 0x7f28d419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 --2- 172.21.15.63:0/2721864691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28d4106eb0 0x7f28d419d500 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:07.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 -- 172.21.15.63:0/2721864691 >> 172.21.15.63:0/2721864691 conn(0x7f28d4102040 msgr2=0x7f28d4103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:07.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 -- 172.21.15.63:0/2721864691 shutdown_connections 2024-05-27T07:54:07.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:07.305+0000 7f28dc200700 1 -- 172.21.15.63:0/2721864691 wait complete. 2024-05-27T07:54:07.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:07 smithi086 ceph-mon[34687]: pgmap v647: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 27 op/s 2024-05-27T07:54:07.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:07 smithi149 ceph-mon[35408]: pgmap v647: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 27 op/s 2024-05-27T07:54:08.218 INFO:tasks.cephfs.filesystem:scrub status for tag:e0c40d31-e6cc-45aa-bf8c-cf183300e15d - {'path': '/', 'tag': 'e0c40d31-e6cc-45aa-bf8c-cf183300e15d', 'options': 'recursive,force'} 2024-05-27T07:54:08.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:08 smithi086 ceph-mon[34687]: osdmap e122: 12 total, 12 up, 12 in 2024-05-27T07:54:08.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:08 smithi149 ceph-mon[35408]: osdmap e122: 12 total, 12 up, 12 in 2024-05-27T07:54:08.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:08 smithi063 ceph-mon[31613]: osdmap e122: 12 total, 12 up, 12 in 2024-05-27T07:54:09.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:09 smithi086 ceph-mon[34687]: pgmap v649: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:54:09.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:09 smithi149 ceph-mon[35408]: pgmap v649: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:54:09.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:09 smithi063 ceph-mon[31613]: pgmap v649: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:54:11.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:11 smithi086 ceph-mon[34687]: pgmap v650: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:54:11.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:11 smithi149 ceph-mon[35408]: pgmap v650: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:54:11.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:11 smithi063 ceph-mon[31613]: pgmap v650: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:54:12.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:54:12.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:54:12.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:54:12.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:54:12.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:54:12.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:54:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:54:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:54:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:54:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.636+0000 7f1a8b200640 1 -- 172.21.15.63:0/2881837214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a8c104200 msgr2=0x7f1a8c1105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.636+0000 7f1a8b200640 1 --2- 172.21.15.63:0/2881837214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a8c104200 0x7f1a8c1105d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009ca0 tx=0x7f1a8402a8c0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.636+0000 7f1a8b200640 1 -- 172.21.15.63:0/2881837214 shutdown_connections 2024-05-27T07:54:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.636+0000 7f1a8b200640 1 --2- 172.21.15.63:0/2881837214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a8c104200 0x7f1a8c1105d0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.636+0000 7f1a8b200640 1 --2- 172.21.15.63:0/2881837214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a8c109f50 0x7f1a8c103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.636+0000 7f1a8b200640 1 --2- 172.21.15.63:0/2881837214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c109ab0 0x7f1a8c103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.636+0000 7f1a8b200640 1 -- 172.21.15.63:0/2881837214 >> 172.21.15.63:0/2881837214 conn(0x7f1a8c079140 msgr2=0x7f1a8c079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.636+0000 7f1a8b200640 1 -- 172.21.15.63:0/2881837214 shutdown_connections 2024-05-27T07:54:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.637+0000 7f1a8b200640 1 -- 172.21.15.63:0/2881837214 wait complete. 2024-05-27T07:54:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.638+0000 7f1a8b200640 1 Processor -- start 2024-05-27T07:54:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.638+0000 7f1a8b200640 1 -- start start 2024-05-27T07:54:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.638+0000 7f1a8b200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a8c104200 0x7f1a8c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.638+0000 7f1a8b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a8c109ab0 0x7f1a8c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.638+0000 7f1a8b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c109f50 0x7f1a8c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.638+0000 7f1a8b200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c113bc0 con 0x7f1a8c104200 2024-05-27T07:54:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.638+0000 7f1a8b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c113a40 con 0x7f1a8c109f50 2024-05-27T07:54:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.638+0000 7f1a8b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c113d40 con 0x7f1a8c109ab0 2024-05-27T07:54:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.639+0000 7f1a89e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a8c104200 0x7f1a8c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.639+0000 7f1a89e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a8c104200 0x7f1a8c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38236/0 (socket says 172.21.15.63:38236) 2024-05-27T07:54:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.639+0000 7f1a89e00640 1 -- 172.21.15.63:0/138458989 learned_addr learned my addr 172.21.15.63:0/138458989 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.639+0000 7f1a81400640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a8c109ab0 0x7f1a8c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.639+0000 7f1a8a800640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c109f50 0x7f1a8c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.639+0000 7f1a89e00640 1 -- 172.21.15.63:0/138458989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a8c109ab0 msgr2=0x7f1a8c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.639+0000 7f1a89e00640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a8c109ab0 0x7f1a8c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.639+0000 7f1a89e00640 1 -- 172.21.15.63:0/138458989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c109f50 msgr2=0x7f1a8c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.639+0000 7f1a89e00640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c109f50 0x7f1a8c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.639+0000 7f1a89e00640 1 -- 172.21.15.63:0/138458989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c19f440 con 0x7f1a8c104200 2024-05-27T07:54:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.640+0000 7f1a89e00640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a8c104200 0x7f1a8c19a5e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00e9c0 tx=0x7f1a7c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.640+0000 7f1a83400640 1 -- 172.21.15.63:0/138458989 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c00cd80 con 0x7f1a8c104200 2024-05-27T07:54:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.640+0000 7f1a83400640 1 -- 172.21.15.63:0/138458989 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a7c0054b0 con 0x7f1a8c104200 2024-05-27T07:54:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.640+0000 7f1a8b200640 1 -- 172.21.15.63:0/138458989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c19f730 con 0x7f1a8c104200 2024-05-27T07:54:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.640+0000 7f1a83400640 1 -- 172.21.15.63:0/138458989 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c010430 con 0x7f1a8c104200 2024-05-27T07:54:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.640+0000 7f1a8b200640 1 -- 172.21.15.63:0/138458989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c1a70b0 con 0x7f1a8c104200 2024-05-27T07:54:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.644+0000 7f1a83400640 1 -- 172.21.15.63:0/138458989 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1a7c004f20 con 0x7f1a8c104200 2024-05-27T07:54:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.645+0000 7f1a97c1c740 1 Processor -- start 2024-05-27T07:54:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.645+0000 7f1a97c1c740 1 -- start start 2024-05-27T07:54:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.645+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab3972aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.645+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab3935f2f0 0x55ab3972afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.645+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393674d0 0x55ab3972b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.645+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ab3969bb90 con 0x55ab393881b0 2024-05-27T07:54:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.645+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab3968b730 con 0x55ab393674d0 2024-05-27T07:54:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.645+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ab3968f770 con 0x55ab3935f2f0 2024-05-27T07:54:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.646+0000 7f1a8a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab3972aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.646+0000 7f1a8a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab3972aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38242/0 (socket says 172.21.15.63:38242) 2024-05-27T07:54:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.646+0000 7f1a8a800640 1 -- 172.21.15.63:0/2864173953 learned_addr learned my addr 172.21.15.63:0/2864173953 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.646+0000 7f1a89e00640 1 --2- 172.21.15.63:0/2864173953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab3935f2f0 0x55ab3972afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.646+0000 7f1a81400640 1 --2- 172.21.15.63:0/2864173953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393674d0 0x55ab3972b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.646+0000 7f1a83400640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1a700b29d0 0x7f1a700b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.647+0000 7f1a83400640 1 -- 172.21.15.63:0/138458989 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9571+0+0 (secure 0 0 0) 0x7f1a7c0e3140 con 0x7f1a8c104200 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.647+0000 7f1a8a800640 1 -- 172.21.15.63:0/2864173953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab3935f2f0 msgr2=0x55ab3972afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.647+0000 7f1a8a800640 1 --2- 172.21.15.63:0/2864173953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab3935f2f0 0x55ab3972afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.647+0000 7f1a8a800640 1 -- 172.21.15.63:0/2864173953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393674d0 msgr2=0x55ab3972b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.647+0000 7f1a8a800640 1 --2- 172.21.15.63:0/2864173953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393674d0 0x55ab3972b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.647+0000 7f1a8a800640 1 -- 172.21.15.63:0/2864173953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab396248b0 con 0x55ab393881b0 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.647+0000 7f1a8a800640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1a700b29d0 0x7f1a700b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.647+0000 7f1a8a800640 1 --2- 172.21.15.63:0/2864173953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab3972aa60 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f1a84005e30 tx=0x7f1a84031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4da3bc893d87eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.648+0000 7f1a7b000640 1 -- 172.21.15.63:0/2864173953 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84004040 con 0x55ab393881b0 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.648+0000 7f1a7b000640 1 -- 172.21.15.63:0/2864173953 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a84036070 con 0x55ab393881b0 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.648+0000 7f1a7b000640 1 -- 172.21.15.63:0/2864173953 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a840309f0 con 0x55ab393881b0 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.649+0000 7f1a97c1c740 1 -- 172.21.15.63:0/2864173953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 msgr2=0x55ab3972aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.649+0000 7f1a8a800640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1a700b29d0 0x7f1a700b4ee0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1a84004af0 tx=0x7f1a84048000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.649+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/2864173953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab3972aa60 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f1a84005e30 tx=0x7f1a84031450 comp rx=0 tx=0).stop 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.649+0000 7f1a97c1c740 1 -- 172.21.15.63:0/2864173953 shutdown_connections 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.649+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/2864173953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393674d0 0x55ab3972b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.649+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/2864173953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab3935f2f0 0x55ab3972afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.649+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/2864173953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab3972aa60 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.649+0000 7f1a97c1c740 1 -- 172.21.15.63:0/2864173953 >> 172.21.15.63:0/2864173953 conn(0x55ab3937c300 msgr2=0x55ab39727210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.649+0000 7f1a97c1c740 1 -- 172.21.15.63:0/2864173953 shutdown_connections 2024-05-27T07:54:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.649+0000 7f1a97c1c740 1 -- 172.21.15.63:0/2864173953 wait complete. 2024-05-27T07:54:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.650+0000 7f1a97c1c740 1 -- start start 2024-05-27T07:54:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.650+0000 7f1a97c1c740 1 Processor -- start 2024-05-27T07:54:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.651+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab3935f2f0 0x55ab39734c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.651+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 0x55ab39735160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.651+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab39740d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.651+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ab39371790 con 0x55ab393881b0 2024-05-27T07:54:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.651+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab39633bc0 con 0x55ab3935f2f0 2024-05-27T07:54:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.651+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ab39737a10 con 0x55ab393674d0 2024-05-27T07:54:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.651+0000 7f1a97c1c740 10 client.0 _ll_register_callbacks cb 0x55ab39611400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.651+0000 7f1a8a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab39740d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.651+0000 7f1a8a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab39740d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38252/0 (socket says 172.21.15.63:38252) 2024-05-27T07:54:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.651+0000 7f1a8a800640 1 -- 172.21.15.63:0/3322721129 learned_addr learned my addr 172.21.15.63:0/3322721129 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.652+0000 7f1a81400640 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 0x55ab39735160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.652+0000 7f1a89e00640 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab3935f2f0 0x55ab39734c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.652+0000 7f1a8a800640 1 -- 172.21.15.63:0/3322721129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 msgr2=0x55ab39735160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.652+0000 7f1a8a800640 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 0x55ab39735160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.652+0000 7f1a8a800640 1 -- 172.21.15.63:0/3322721129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab3935f2f0 msgr2=0x55ab39734c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.652+0000 7f1a8a800640 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab3935f2f0 0x55ab39734c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.652+0000 7f1a8a800640 1 -- 172.21.15.63:0/3322721129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab393a0210 con 0x55ab393881b0 2024-05-27T07:54:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.652+0000 7f1a8a800640 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab39740d40 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f1a8402ade0 tx=0x7f1a84030e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445711594f0cdb6c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.652+0000 7f1a6f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.653+0000 7f1a6f400640 1 -- 172.21.15.63:0/3322721129 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84009800 con 0x55ab393881b0 2024-05-27T07:54:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.653+0000 7f1a6f400640 1 -- 172.21.15.63:0/3322721129 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9571+0+0 (secure 0 0 0) 0x7f1a8404d050 con 0x55ab393881b0 2024-05-27T07:54:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.653+0000 7f1a97c1c740 1 -- 172.21.15.63:0/3322721129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a84036070 con 0x55ab393881b0 2024-05-27T07:54:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.653+0000 7f1a6f400640 1 -- 172.21.15.63:0/3322721129 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1a840305c0 con 0x55ab393881b0 2024-05-27T07:54:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.653+0000 7f1a97c1c740 10 client.19413 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.653+0000 7f1a97c1c740 1 -- 172.21.15.63:0/3322721129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ab3946c620 con 0x55ab393881b0 2024-05-27T07:54:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.653+0000 7f1a6f400640 1 -- 172.21.15.63:0/3322721129 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a84036070 con 0x55ab393881b0 2024-05-27T07:54:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.654+0000 7f1a6f400640 1 -- 172.21.15.63:0/3322721129 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1a840516b0 con 0x55ab393881b0 2024-05-27T07:54:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.654+0000 7f1a97c1c740 10 client.19413 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.654+0000 7f1a97c1c740 10 client.19413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.654+0000 7f1a97c1c740 20 client.19413 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.654+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ab39346d80 0x55ab39746b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.654+0000 7f1a97c1c740 4 client.19413 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.654+0000 7f1a97c1c740 1 -- 172.21.15.63:0/3322721129 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab3969dae0 con 0x55ab39346d80 2024-05-27T07:54:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.655+0000 7f1a89e00640 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ab39346d80 0x55ab39746b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.655+0000 7f1a89e00640 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ab39346d80 0x55ab39746b30 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecf4e145ce16045 server_cookie=9216fdb57e7b985c in_seq=0 out_seq=0 2024-05-27T07:54:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.655+0000 7f1a6f400640 10 client.19413 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.657+0000 7f1a6f400640 1 -- 172.21.15.63:0/3322721129 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ab3969dae0 con 0x55ab39346d80 2024-05-27T07:54:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.657+0000 7f1a6f400640 10 client.19413 handle_command_reply: tid=0 2024-05-27T07:54:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.657+0000 7f1a97c1c740 1 client.19413 shutdown 2024-05-27T07:54:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 1 -- 172.21.15.63:0/3322721129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 msgr2=0x55ab39740d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab39740d40 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f1a8402ade0 tx=0x7f1a84030e80 comp rx=0 tx=0).stop 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 1 -- 172.21.15.63:0/3322721129 shutdown_connections 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ab39346d80 0x55ab39746b30 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393881b0 0x55ab39740d40 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 0x55ab39735160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/3322721129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab3935f2f0 0x55ab39734c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 1 -- 172.21.15.63:0/3322721129 >> 172.21.15.63:0/3322721129 conn(0x55ab3937c300 msgr2=0x55ab3972a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a6f400640 0 client.19413 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 1 -- 172.21.15.63:0/3322721129 shutdown_connections 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 1 -- 172.21.15.63:0/3322721129 wait complete. 2024-05-27T07:54:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.658+0000 7f1a97c1c740 20 client.19413 trim_cache size 0 max 16384 2024-05-27T07:54:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.683+0000 7f1a97c1c740 1 Processor -- start 2024-05-27T07:54:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.683+0000 7f1a97c1c740 1 -- start start 2024-05-27T07:54:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.684+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab3935f2f0 0x55ab39735170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.684+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393674d0 0x55ab3974c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.684+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393881b0 0x55ab3974c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.684+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1b3ac0 con 0x55ab393674d0 2024-05-27T07:54:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.684+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1b1390 con 0x55ab3935f2f0 2024-05-27T07:54:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.684+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1af200 con 0x55ab393881b0 2024-05-27T07:54:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.684+0000 7f1a81400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393674d0 0x55ab3974c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.684+0000 7f1a81400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393674d0 0x55ab3974c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38268/0 (socket says 172.21.15.63:38268) 2024-05-27T07:54:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.684+0000 7f1a81400640 1 -- 172.21.15.63:0/700586588 learned_addr learned my addr 172.21.15.63:0/700586588 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.685+0000 7f1a89e00640 1 --2- 172.21.15.63:0/700586588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab3935f2f0 0x55ab39735170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.685+0000 7f1a8a800640 1 --2- 172.21.15.63:0/700586588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393881b0 0x55ab3974c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.685+0000 7f1a81400640 1 -- 172.21.15.63:0/700586588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393881b0 msgr2=0x55ab3974c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.685+0000 7f1a81400640 1 --2- 172.21.15.63:0/700586588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393881b0 0x55ab3974c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.685+0000 7f1a81400640 1 -- 172.21.15.63:0/700586588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab3935f2f0 msgr2=0x55ab39735170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.685+0000 7f1a81400640 1 --2- 172.21.15.63:0/700586588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab3935f2f0 0x55ab39735170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.685+0000 7f1a81400640 1 -- 172.21.15.63:0/700586588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab39727e00 con 0x55ab393674d0 2024-05-27T07:54:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.685+0000 7f1a8a800640 1 --2- 172.21.15.63:0/700586588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393881b0 0x55ab3974c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.685+0000 7f1a81400640 1 --2- 172.21.15.63:0/700586588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393674d0 0x55ab3974c0d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400bc00 tx=0x7f1a74012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4330bf645e08ca6c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.686+0000 7f1a7a600640 1 -- 172.21.15.63:0/700586588 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7400bcf0 con 0x55ab393674d0 2024-05-27T07:54:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.686+0000 7f1a7a600640 1 -- 172.21.15.63:0/700586588 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a740035f0 con 0x55ab393674d0 2024-05-27T07:54:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.686+0000 7f1a7a600640 1 -- 172.21.15.63:0/700586588 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7401b670 con 0x55ab393674d0 2024-05-27T07:54:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.687+0000 7f1a97c1c740 1 -- 172.21.15.63:0/700586588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393674d0 msgr2=0x55ab3974c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.687+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/700586588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393674d0 0x55ab3974c0d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400bc00 tx=0x7f1a74012820 comp rx=0 tx=0).stop 2024-05-27T07:54:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.687+0000 7f1a97c1c740 1 -- 172.21.15.63:0/700586588 shutdown_connections 2024-05-27T07:54:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.687+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/700586588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393881b0 0x55ab3974c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.687+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/700586588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab393674d0 0x55ab3974c0d0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.687+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/700586588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab3935f2f0 0x55ab39735170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.687+0000 7f1a97c1c740 1 -- 172.21.15.63:0/700586588 >> 172.21.15.63:0/700586588 conn(0x55ab3937c300 msgr2=0x55ab3972f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.687+0000 7f1a97c1c740 1 -- 172.21.15.63:0/700586588 shutdown_connections 2024-05-27T07:54:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.687+0000 7f1a97c1c740 1 -- 172.21.15.63:0/700586588 wait complete. 2024-05-27T07:54:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.687+0000 7f1a97c1c740 1 -- start start 2024-05-27T07:54:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.688+0000 7f1a97c1c740 1 Processor -- start 2024-05-27T07:54:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.688+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3935f2f0 0x55ab397a6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.688+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 0x55ab397a6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.688+0000 7f1a97c1c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393881b0 0x55ab3973a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.688+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1af850 con 0x55ab3935f2f0 2024-05-27T07:54:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.688+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1b0cc0 con 0x55ab393881b0 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.688+0000 7f1a97c1c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1aff20 con 0x55ab393674d0 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.688+0000 7f1a97c1c740 10 client.0 _ll_register_callbacks cb 0x7f1a8404a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a89e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3935f2f0 0x55ab397a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a89e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3935f2f0 0x55ab397a6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38280/0 (socket says 172.21.15.63:38280) 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a89e00640 1 -- 172.21.15.63:0/4276162026 learned_addr learned my addr 172.21.15.63:0/4276162026 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a81400640 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 0x55ab397a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a8a800640 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393881b0 0x55ab3973a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a89e00640 1 -- 172.21.15.63:0/4276162026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 msgr2=0x55ab397a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a89e00640 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 0x55ab397a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a89e00640 1 -- 172.21.15.63:0/4276162026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393881b0 msgr2=0x55ab3973a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a89e00640 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393881b0 0x55ab3973a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a89e00640 1 -- 172.21.15.63:0/4276162026 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3973a8b0 con 0x55ab3935f2f0 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a81400640 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 0x55ab397a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a8a800640 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393881b0 0x55ab3973a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a89e00640 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3935f2f0 0x55ab397a6310 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0a6020 tx=0x7f1a7c0e67f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2f068289edadf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.689+0000 7f1a6f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.690+0000 7f1a6f400640 1 -- 172.21.15.63:0/4276162026 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0aa6a0 con 0x55ab3935f2f0 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.690+0000 7f1a6f400640 1 -- 172.21.15.63:0/4276162026 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9571+0+0 (secure 0 0 0) 0x7f1a7c0e0690 con 0x55ab3935f2f0 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.690+0000 7f1a97c1c740 1 -- 172.21.15.63:0/4276162026 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a740035f0 con 0x55ab3935f2f0 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.690+0000 7f1a6f400640 1 -- 172.21.15.63:0/4276162026 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1a7c0e0890 con 0x55ab3935f2f0 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.690+0000 7f1a97c1c740 10 client.19419 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.690+0000 7f1a97c1c740 1 -- 172.21.15.63:0/4276162026 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ab397ae970 con 0x55ab3935f2f0 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.690+0000 7f1a6f400640 1 -- 172.21.15.63:0/4276162026 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a740035f0 con 0x55ab3935f2f0 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.691+0000 7f1a6f400640 1 -- 172.21.15.63:0/4276162026 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1a7c0e50b0 con 0x55ab3935f2f0 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.691+0000 7f1a97c1c740 10 client.19419 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.691+0000 7f1a97c1c740 10 client.19419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.691+0000 7f1a97c1c740 20 client.19419 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.691+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ab3931c1c0 0x55ab397b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.691+0000 7f1a97c1c740 4 client.19419 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:54:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.691+0000 7f1a97c1c740 1 -- 172.21.15.63:0/4276162026 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab396972c0 con 0x55ab3931c1c0 2024-05-27T07:54:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.691+0000 7f1a81400640 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ab3931c1c0 0x55ab397b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.692+0000 7f1a81400640 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ab3931c1c0 0x55ab397b10b0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e51a7310c4e38a server_cookie=a14a0bf6038909be in_seq=0 out_seq=0 2024-05-27T07:54:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.692+0000 7f1a6f400640 10 client.19419 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.733+0000 7f1a6f400640 1 -- 172.21.15.63:0/4276162026 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ab396972c0 con 0x55ab3931c1c0 2024-05-27T07:54:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.733+0000 7f1a6f400640 10 client.19419 handle_command_reply: tid=0 2024-05-27T07:54:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.733+0000 7f1a97c1c740 1 client.19419 shutdown 2024-05-27T07:54:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.733+0000 7f1a97c1c740 1 -- 172.21.15.63:0/4276162026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3935f2f0 msgr2=0x55ab397a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.733+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3935f2f0 0x55ab397a6310 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0a6020 tx=0x7f1a7c0e67f0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.733+0000 7f1a97c1c740 1 -- 172.21.15.63:0/4276162026 shutdown_connections 2024-05-27T07:54:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.733+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ab3931c1c0 0x55ab397b10b0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.734+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab393881b0 0x55ab3973a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.734+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ab393674d0 0x55ab397a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.734+0000 7f1a97c1c740 1 --2- 172.21.15.63:0/4276162026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3935f2f0 0x55ab397a6310 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.734+0000 7f1a6f400640 0 client.19419 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.734+0000 7f1a97c1c740 1 -- 172.21.15.63:0/4276162026 >> 172.21.15.63:0/4276162026 conn(0x55ab3937c300 msgr2=0x55ab39734b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.734+0000 7f1a97c1c740 1 -- 172.21.15.63:0/4276162026 shutdown_connections 2024-05-27T07:54:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.734+0000 7f1a97c1c740 1 -- 172.21.15.63:0/4276162026 wait complete. 2024-05-27T07:54:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.734+0000 7f1a97c1c740 20 client.19419 trim_cache size 0 max 16384 2024-05-27T07:54:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:54:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:54:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:54:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.735+0000 7f1a8b200640 1 -- 172.21.15.63:0/138458989 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1a700b29d0 msgr2=0x7f1a700b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.735+0000 7f1a8b200640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1a700b29d0 0x7f1a700b4ee0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1a84004af0 tx=0x7f1a84048000 comp rx=0 tx=0).stop 2024-05-27T07:54:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.735+0000 7f1a8b200640 1 -- 172.21.15.63:0/138458989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a8c104200 msgr2=0x7f1a8c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.735+0000 7f1a8b200640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a8c104200 0x7f1a8c19a5e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00e9c0 tx=0x7f1a7c00ee80 comp rx=0 tx=0).stop 2024-05-27T07:54:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.735+0000 7f1a8b200640 1 -- 172.21.15.63:0/138458989 shutdown_connections 2024-05-27T07:54:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.735+0000 7f1a8b200640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1a700b29d0 0x7f1a700b4ee0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.735+0000 7f1a8b200640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c109f50 0x7f1a8c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.735+0000 7f1a8b200640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a8c109ab0 0x7f1a8c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.735+0000 7f1a8b200640 1 --2- 172.21.15.63:0/138458989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a8c104200 0x7f1a8c19a5e0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.735+0000 7f1a8b200640 1 -- 172.21.15.63:0/138458989 >> 172.21.15.63:0/138458989 conn(0x7f1a8c079140 msgr2=0x7f1a8c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.736+0000 7f1a8b200640 1 -- 172.21.15.63:0/138458989 shutdown_connections 2024-05-27T07:54:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:12.736+0000 7f1a8b200640 1 -- 172.21.15.63:0/138458989 wait complete. 2024-05-27T07:54:13.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:13 smithi086 ceph-mon[34687]: pgmap v651: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 43 op/s 2024-05-27T07:54:13.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:13 smithi149 ceph-mon[35408]: pgmap v651: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 43 op/s 2024-05-27T07:54:13.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:13 smithi063 ceph-mon[31613]: pgmap v651: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 43 op/s 2024-05-27T07:54:15.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:15 smithi149 ceph-mon[35408]: pgmap v652: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 43 op/s 2024-05-27T07:54:15.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:15 smithi063 ceph-mon[31613]: pgmap v652: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 43 op/s 2024-05-27T07:54:15.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:15 smithi086 ceph-mon[34687]: pgmap v652: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 43 op/s 2024-05-27T07:54:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:16 smithi086 ceph-mon[34687]: osdmap e123: 12 total, 12 up, 12 in 2024-05-27T07:54:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:16 smithi086 ceph-mon[34687]: pgmap v654: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-05-27T07:54:17.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:16 smithi149 ceph-mon[35408]: osdmap e123: 12 total, 12 up, 12 in 2024-05-27T07:54:17.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:16 smithi149 ceph-mon[35408]: pgmap v654: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-05-27T07:54:17.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:16 smithi063 ceph-mon[31613]: osdmap e123: 12 total, 12 up, 12 in 2024-05-27T07:54:17.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:16 smithi063 ceph-mon[31613]: pgmap v654: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.7 MiB/s wr, 5 op/s 2024-05-27T07:54:19.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:19 smithi086 ceph-mon[34687]: pgmap v655: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:19.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:19 smithi149 ceph-mon[35408]: pgmap v655: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:19.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:19 smithi063 ceph-mon[31613]: pgmap v655: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:21.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:21 smithi086 ceph-mon[34687]: pgmap v656: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:21.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:21 smithi149 ceph-mon[35408]: pgmap v656: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:21.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:21 smithi063 ceph-mon[31613]: pgmap v656: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:23.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:23 smithi086 ceph-mon[34687]: pgmap v657: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:23.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:23 smithi149 ceph-mon[35408]: pgmap v657: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:23.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:23 smithi063 ceph-mon[31613]: pgmap v657: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:25.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:25 smithi086 ceph-mon[34687]: pgmap v658: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:25.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:25 smithi149 ceph-mon[35408]: pgmap v658: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:25.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:25 smithi063 ceph-mon[31613]: pgmap v658: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T07:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.996+0000 7f12e3200640 1 -- 172.21.15.63:0/3598764994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12e410a320 msgr2=0x7f12e410a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.996+0000 7f12e3200640 1 --2- 172.21.15.63:0/3598764994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12e410a320 0x7f12e410a730 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009900 tx=0x7f12d402a410 comp rx=0 tx=0).stop 2024-05-27T07:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.997+0000 7f12e3200640 1 -- 172.21.15.63:0/3598764994 shutdown_connections 2024-05-27T07:54:25.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.997+0000 7f12e3200640 1 --2- 172.21.15.63:0/3598764994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12e410f370 0x7f12e4116b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.997+0000 7f12e3200640 1 --2- 172.21.15.63:0/3598764994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e410e8b0 0x7f12e410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.997+0000 7f12e3200640 1 --2- 172.21.15.63:0/3598764994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12e410a320 0x7f12e410a730 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.997+0000 7f12e3200640 1 -- 172.21.15.63:0/3598764994 >> 172.21.15.63:0/3598764994 conn(0x7f12e4076070 msgr2=0x7f12e4076480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.997+0000 7f12e3200640 1 -- 172.21.15.63:0/3598764994 shutdown_connections 2024-05-27T07:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.997+0000 7f12e3200640 1 -- 172.21.15.63:0/3598764994 wait complete. 2024-05-27T07:54:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.998+0000 7f12e3200640 1 Processor -- start 2024-05-27T07:54:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.999+0000 7f12e3200640 1 -- start start 2024-05-27T07:54:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.999+0000 7f12e3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12e410a320 0x7f12e41a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.999+0000 7f12e3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12e410e8b0 0x7f12e41a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.999+0000 7f12e3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e410f370 0x7f12e41a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.999+0000 7f12e3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12e411a150 con 0x7f12e410a320 2024-05-27T07:54:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.999+0000 7f12e3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4119fd0 con 0x7f12e410e8b0 2024-05-27T07:54:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:25.999+0000 7f12e3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f12e411a2d0 con 0x7f12e410f370 2024-05-27T07:54:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12e410a320 0x7f12e41a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12e410a320 0x7f12e41a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41790/0 (socket says 172.21.15.63:41790) 2024-05-27T07:54:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e1e00640 1 -- 172.21.15.63:0/3138540885 learned_addr learned my addr 172.21.15.63:0/3138540885 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e1400640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12e410e8b0 0x7f12e41a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e2800640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e410f370 0x7f12e41a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e1e00640 1 -- 172.21.15.63:0/3138540885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e410f370 msgr2=0x7f12e41a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e1e00640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e410f370 0x7f12e41a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e1e00640 1 -- 172.21.15.63:0/3138540885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12e410e8b0 msgr2=0x7f12e41a6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e1e00640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12e410e8b0 0x7f12e41a6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e1e00640 1 -- 172.21.15.63:0/3138540885 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e41a1170 con 0x7f12e410a320 2024-05-27T07:54:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.000+0000 7f12e1400640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12e410e8b0 0x7f12e41a6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.001+0000 7f12e1e00640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12e410a320 0x7f12e41a68a0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009230 tx=0x7f12d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.001+0000 7f12daa00640 1 -- 172.21.15.63:0/3138540885 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d403f030 con 0x7f12e410a320 2024-05-27T07:54:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.001+0000 7f12daa00640 1 -- 172.21.15.63:0/3138540885 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12d4003a80 con 0x7f12e410a320 2024-05-27T07:54:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.001+0000 7f12daa00640 1 -- 172.21.15.63:0/3138540885 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4035070 con 0x7f12e410a320 2024-05-27T07:54:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.001+0000 7f12e3200640 1 -- 172.21.15.63:0/3138540885 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e41a1400 con 0x7f12e410a320 2024-05-27T07:54:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.002+0000 7f12e3200640 1 -- 172.21.15.63:0/3138540885 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e41a18f0 con 0x7f12e410a320 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.006+0000 7f12effbe740 1 Processor -- start 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.006+0000 7f12effbe740 1 -- start start 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.006+0000 7f12daa00640 1 -- 172.21.15.63:0/3138540885 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f12d4042030 con 0x7f12e410a320 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.006+0000 7f12effbe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e11a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.006+0000 7f12effbe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e11fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.006+0000 7f12effbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a4e4d0 0x55d090e124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.006+0000 7f12effbe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d090d82b90 con 0x55d090a6f1b0 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.006+0000 7f12effbe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d090d72730 con 0x55d090a462f0 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.006+0000 7f12effbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d090d76770 con 0x55d090a4e4d0 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.007+0000 7f12e2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e11a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.007+0000 7f12e2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e11a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41802/0 (socket says 172.21.15.63:41802) 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.007+0000 7f12e2800640 1 -- 172.21.15.63:0/1075354500 learned_addr learned my addr 172.21.15.63:0/1075354500 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.007+0000 7f12e1e00640 1 --2- 172.21.15.63:0/1075354500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e11fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.007+0000 7f12e1400640 1 --2- 172.21.15.63:0/1075354500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a4e4d0 0x55d090e124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.007+0000 7f12e2800640 1 -- 172.21.15.63:0/1075354500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a4e4d0 msgr2=0x55d090e124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.007+0000 7f12e2800640 1 --2- 172.21.15.63:0/1075354500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a4e4d0 0x55d090e124e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.007+0000 7f12e2800640 1 -- 172.21.15.63:0/1075354500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 msgr2=0x55d090e11fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.007+0000 7f12e2800640 1 --2- 172.21.15.63:0/1075354500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e11fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.007+0000 7f12e2800640 1 -- 172.21.15.63:0/1075354500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d090d0b8b0 con 0x55d090a6f1b0 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.008+0000 7f12e2800640 1 --2- 172.21.15.63:0/1075354500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e11a60 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009e60 tx=0x7f12dc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e7db76ce561f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.008+0000 7f12cf400640 1 -- 172.21.15.63:0/1075354500 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc016070 con 0x55d090a6f1b0 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.008+0000 7f12cf400640 1 -- 172.21.15.63:0/1075354500 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12dc004d20 con 0x55d090a6f1b0 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.008+0000 7f12cf400640 1 -- 172.21.15.63:0/1075354500 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc0126b0 con 0x55d090a6f1b0 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.008+0000 7f12daa00640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f12c80b2950 0x7f12c80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.009+0000 7f12daa00640 1 -- 172.21.15.63:0/3138540885 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f12d41030b0 con 0x7f12e410a320 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.009+0000 7f12e1e00640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f12c80b2950 0x7f12c80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.009+0000 7f12effbe740 1 -- 172.21.15.63:0/1075354500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 msgr2=0x55d090e11a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.009+0000 7f12e1e00640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f12c80b2950 0x7f12c80b4e60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55d090e0f550 tx=0x7f12d4009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.009+0000 7f12effbe740 1 --2- 172.21.15.63:0/1075354500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e11a60 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009e60 tx=0x7f12dc010740 comp rx=0 tx=0).stop 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.010+0000 7f12effbe740 1 -- 172.21.15.63:0/1075354500 shutdown_connections 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.010+0000 7f12effbe740 1 --2- 172.21.15.63:0/1075354500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a4e4d0 0x55d090e124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.010+0000 7f12effbe740 1 --2- 172.21.15.63:0/1075354500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e11fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.010+0000 7f12effbe740 1 --2- 172.21.15.63:0/1075354500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e11a60 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.010+0000 7f12effbe740 1 -- 172.21.15.63:0/1075354500 >> 172.21.15.63:0/1075354500 conn(0x55d090a63300 msgr2=0x55d090e0e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.010+0000 7f12effbe740 1 -- 172.21.15.63:0/1075354500 shutdown_connections 2024-05-27T07:54:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.010+0000 7f12effbe740 1 -- 172.21.15.63:0/1075354500 wait complete. 2024-05-27T07:54:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.010+0000 7f12effbe740 1 -- start start 2024-05-27T07:54:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.011+0000 7f12effbe740 1 Processor -- start 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.011+0000 7f12effbe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e1bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.011+0000 7f12effbe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.011+0000 7f12effbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 0x55d090e27ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.011+0000 7f12effbe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d090a58790 con 0x55d090a4e4d0 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.011+0000 7f12effbe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d090d1abc0 con 0x55d090a462f0 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.011+0000 7f12effbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d090e1eb90 con 0x55d090a6f1b0 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.011+0000 7f12effbe740 10 client.0 _ll_register_callbacks cb 0x55d090cf8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.011+0000 7f12e1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.012+0000 7f12e1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41810/0 (socket says 172.21.15.63:41810) 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.012+0000 7f12e1400640 1 -- 172.21.15.63:0/1863810405 learned_addr learned my addr 172.21.15.63:0/1863810405 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:26.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.012+0000 7f12e2800640 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e1bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.012+0000 7f12e2800640 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 0x55d090e27ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.012+0000 7f12e1400640 1 -- 172.21.15.63:0/1863810405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 msgr2=0x55d090e27ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.012+0000 7f12e1400640 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 0x55d090e27ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.012+0000 7f12e1400640 1 -- 172.21.15.63:0/1863810405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 msgr2=0x55d090e1bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.012+0000 7f12e1400640 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e1bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.012+0000 7f12e1400640 1 -- 172.21.15.63:0/1863810405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d090a87210 con 0x55d090a4e4d0 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.012+0000 7f12e2800640 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e1bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.013+0000 7f12e1400640 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1c110 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f12d000be90 tx=0x7f12d00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206dbe3d88131ad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.013+0000 7f12c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.013+0000 7f12c3e00640 1 -- 172.21.15.63:0/1863810405 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0005220 con 0x55d090a4e4d0 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.013+0000 7f12c3e00640 1 -- 172.21.15.63:0/1863810405 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f12d00053c0 con 0x55d090a4e4d0 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.013+0000 7f12effbe740 1 -- 172.21.15.63:0/1863810405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12dc004d20 con 0x55d090a4e4d0 2024-05-27T07:54:26.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.013+0000 7f12c3e00640 1 -- 172.21.15.63:0/1863810405 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f12d00055c0 con 0x55d090a4e4d0 2024-05-27T07:54:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.013+0000 7f12effbe740 10 client.19431 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.013+0000 7f12effbe740 1 -- 172.21.15.63:0/1863810405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d090b53620 con 0x55d090a4e4d0 2024-05-27T07:54:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.014+0000 7f12c3e00640 1 -- 172.21.15.63:0/1863810405 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12dc004d20 con 0x55d090a4e4d0 2024-05-27T07:54:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.014+0000 7f12c3e00640 1 -- 172.21.15.63:0/1863810405 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f12d001b820 con 0x55d090a4e4d0 2024-05-27T07:54:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.014+0000 7f12effbe740 10 client.19431 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.014+0000 7f12effbe740 10 client.19431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.014+0000 7f12effbe740 20 client.19431 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.014+0000 7f12effbe740 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d090a2dd80 0x55d090e2de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.014+0000 7f12effbe740 4 client.19431 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.014+0000 7f12effbe740 1 -- 172.21.15.63:0/1863810405 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d090d84ae0 con 0x55d090a2dd80 2024-05-27T07:54:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.015+0000 7f12e2800640 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d090a2dd80 0x55d090e2de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.016+0000 7f12e2800640 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d090a2dd80 0x55d090e2de50 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fae279abe9be0333 server_cookie=4165198e234e56b9 in_seq=0 out_seq=0 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.016+0000 7f12c3e00640 10 client.19431 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12c3e00640 1 -- 172.21.15.63:0/1863810405 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d090d84ae0 con 0x55d090a2dd80 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12c3e00640 10 client.19431 handle_command_reply: tid=0 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 client.19431 shutdown 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 -- 172.21.15.63:0/1863810405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 msgr2=0x55d090e1c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1c110 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f12d000be90 tx=0x7f12d00117b0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 -- 172.21.15.63:0/1863810405 shutdown_connections 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d090a2dd80 0x55d090e2de50 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 0x55d090e27ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1c110 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 --2- 172.21.15.63:0/1863810405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e1bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12c3e00640 0 client.19431 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:26.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 -- 172.21.15.63:0/1863810405 >> 172.21.15.63:0/1863810405 conn(0x55d090a63300 msgr2=0x55d090e11890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 -- 172.21.15.63:0/1863810405 shutdown_connections 2024-05-27T07:54:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 1 -- 172.21.15.63:0/1863810405 wait complete. 2024-05-27T07:54:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.018+0000 7f12effbe740 20 client.19431 trim_cache size 0 max 16384 2024-05-27T07:54:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.036+0000 7f12effbe740 1 Processor -- start 2024-05-27T07:54:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.036+0000 7f12effbe740 1 -- start start 2024-05-27T07:54:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12effbe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e1e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12effbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a4e4d0 0x55d090e140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12effbe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e16e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12effbe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41b9fb0 con 0x55d090a6f1b0 2024-05-27T07:54:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12effbe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41b7880 con 0x55d090a462f0 2024-05-27T07:54:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12effbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41b56f0 con 0x55d090a4e4d0 2024-05-27T07:54:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12e2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e16e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12e2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e16e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41814/0 (socket says 172.21.15.63:41814) 2024-05-27T07:54:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12e2800640 1 -- 172.21.15.63:0/577290472 learned_addr learned my addr 172.21.15.63:0/577290472 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12e2800640 1 --2- 172.21.15.63:0/577290472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e1e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.037+0000 7f12e1400640 1 --2- 172.21.15.63:0/577290472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a4e4d0 0x55d090e140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.038+0000 7f12e2800640 1 -- 172.21.15.63:0/577290472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a4e4d0 msgr2=0x55d090e140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.038+0000 7f12e2800640 1 --2- 172.21.15.63:0/577290472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a4e4d0 0x55d090e140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.038+0000 7f12e2800640 1 -- 172.21.15.63:0/577290472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 msgr2=0x55d090e1e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.038+0000 7f12e2800640 1 --2- 172.21.15.63:0/577290472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e1e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.038+0000 7f12e2800640 1 -- 172.21.15.63:0/577290472 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d090e1c470 con 0x55d090a6f1b0 2024-05-27T07:54:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.039+0000 7f12e2800640 1 --2- 172.21.15.63:0/577290472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e16e90 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f12dc010c40 tx=0x7f12dc0059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86c04e135bcf0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.039+0000 7f12cea00640 1 -- 172.21.15.63:0/577290472 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc016040 con 0x55d090a6f1b0 2024-05-27T07:54:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.039+0000 7f12cea00640 1 -- 172.21.15.63:0/577290472 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12dc004c60 con 0x55d090a6f1b0 2024-05-27T07:54:26.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.039+0000 7f12cea00640 1 -- 172.21.15.63:0/577290472 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc0285a0 con 0x55d090a6f1b0 2024-05-27T07:54:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.040+0000 7f12effbe740 1 -- 172.21.15.63:0/577290472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 msgr2=0x55d090e16e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.040+0000 7f12effbe740 1 --2- 172.21.15.63:0/577290472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e16e90 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f12dc010c40 tx=0x7f12dc0059b0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.040+0000 7f12effbe740 1 -- 172.21.15.63:0/577290472 shutdown_connections 2024-05-27T07:54:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.040+0000 7f12effbe740 1 --2- 172.21.15.63:0/577290472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a6f1b0 0x55d090e16e90 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.040+0000 7f12effbe740 1 --2- 172.21.15.63:0/577290472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a4e4d0 0x55d090e140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.040+0000 7f12effbe740 1 --2- 172.21.15.63:0/577290472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e1e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.040+0000 7f12effbe740 1 -- 172.21.15.63:0/577290472 >> 172.21.15.63:0/577290472 conn(0x55d090a63300 msgr2=0x55d090e16780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.040+0000 7f12effbe740 1 -- 172.21.15.63:0/577290472 shutdown_connections 2024-05-27T07:54:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.040+0000 7f12effbe740 1 -- 172.21.15.63:0/577290472 wait complete. 2024-05-27T07:54:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.041+0000 7f12effbe740 1 -- start start 2024-05-27T07:54:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.041+0000 7f12effbe740 1 Processor -- start 2024-05-27T07:54:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12effbe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e24270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12effbe740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12effbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 0x55d090e1fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12effbe740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41b5d40 con 0x55d090a4e4d0 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12effbe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41b71b0 con 0x55d090a462f0 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12effbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41b6410 con 0x55d090a6f1b0 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12effbe740 10 client.0 _ll_register_callbacks cb 0x7f12d000ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12e1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12e1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41826/0 (socket says 172.21.15.63:41826) 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12e1400640 1 -- 172.21.15.63:0/899881538 learned_addr learned my addr 172.21.15.63:0/899881538 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.042+0000 7f12e2800640 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 0x55d090e1fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12e2800640 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e24270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12e1400640 1 -- 172.21.15.63:0/899881538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 msgr2=0x55d090e1fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12e1400640 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 0x55d090e1fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12e1400640 1 -- 172.21.15.63:0/899881538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 msgr2=0x55d090e24270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12e1400640 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e24270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12e1400640 1 -- 172.21.15.63:0/899881538 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d090e20040 con 0x55d090a4e4d0 2024-05-27T07:54:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12e2800640 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 0x55d090e1fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12e1400640 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1f5c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f12d001be30 tx=0x7f12d001b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ccd7508968b10 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12c3e00640 1 -- 172.21.15.63:0/899881538 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0033460 con 0x55d090a4e4d0 2024-05-27T07:54:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12c3e00640 1 -- 172.21.15.63:0/899881538 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f12d0012450 con 0x55d090a4e4d0 2024-05-27T07:54:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.043+0000 7f12effbe740 1 -- 172.21.15.63:0/899881538 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12dc004c60 con 0x55d090a4e4d0 2024-05-27T07:54:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12c3e00640 1 -- 172.21.15.63:0/899881538 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f12d0012650 con 0x55d090a4e4d0 2024-05-27T07:54:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12effbe740 10 client.19437 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12effbe740 1 -- 172.21.15.63:0/899881538 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d090e961c0 con 0x55d090a4e4d0 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12c3e00640 1 -- 172.21.15.63:0/899881538 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12dc004c60 con 0x55d090a4e4d0 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12c3e00640 1 -- 172.21.15.63:0/899881538 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f12d0033790 con 0x55d090a4e4d0 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12effbe740 10 client.19437 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12effbe740 10 client.19437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12effbe740 20 client.19437 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12effbe740 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d090a031c0 0x55d090e989b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12effbe740 4 client.19437 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.044+0000 7f12effbe740 1 -- 172.21.15.63:0/899881538 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d090d7e2c0 con 0x55d090a031c0 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.045+0000 7f12e2800640 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d090a031c0 0x55d090e989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.046+0000 7f12c3e00640 10 client.19437 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.046+0000 7f12e2800640 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d090a031c0 0x55d090e989b0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f98b90cc8d99402 server_cookie=93391510c44a2c79 in_seq=0 out_seq=0 2024-05-27T07:54:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.089+0000 7f12c3e00640 1 -- 172.21.15.63:0/899881538 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d090d7e2c0 con 0x55d090a031c0 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.089+0000 7f12c3e00640 10 client.19437 handle_command_reply: tid=0 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.089+0000 7f12effbe740 1 client.19437 shutdown 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 1 -- 172.21.15.63:0/899881538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 msgr2=0x55d090e1f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1f5c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f12d001be30 tx=0x7f12d001b6b0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 1 -- 172.21.15.63:0/899881538 shutdown_connections 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d090a031c0 0x55d090e989b0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d090a6f1b0 0x55d090e1fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d090a4e4d0 0x55d090e1f5c0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 1 --2- 172.21.15.63:0/899881538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d090a462f0 0x55d090e24270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12c3e00640 0 client.19437 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 1 -- 172.21.15.63:0/899881538 >> 172.21.15.63:0/899881538 conn(0x55d090a63300 msgr2=0x55d090e13e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 1 -- 172.21.15.63:0/899881538 shutdown_connections 2024-05-27T07:54:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 1 -- 172.21.15.63:0/899881538 wait complete. 2024-05-27T07:54:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.090+0000 7f12effbe740 20 client.19437 trim_cache size 0 max 16384 2024-05-27T07:54:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:54:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:54:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:54:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.091+0000 7f12e3200640 1 -- 172.21.15.63:0/3138540885 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f12c80b2950 msgr2=0x7f12c80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.091+0000 7f12e3200640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f12c80b2950 0x7f12c80b4e60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55d090e0f550 tx=0x7f12d4009260 comp rx=0 tx=0).stop 2024-05-27T07:54:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.091+0000 7f12e3200640 1 -- 172.21.15.63:0/3138540885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12e410a320 msgr2=0x7f12e41a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.091+0000 7f12e3200640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12e410a320 0x7f12e41a68a0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009230 tx=0x7f12d4004270 comp rx=0 tx=0).stop 2024-05-27T07:54:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.091+0000 7f12e3200640 1 -- 172.21.15.63:0/3138540885 shutdown_connections 2024-05-27T07:54:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.092+0000 7f12e3200640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f12c80b2950 0x7f12c80b4e60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.092+0000 7f12e3200640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12e410f370 0x7f12e41a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.092+0000 7f12e3200640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12e410e8b0 0x7f12e41a6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.092+0000 7f12e3200640 1 --2- 172.21.15.63:0/3138540885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12e410a320 0x7f12e41a68a0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.092+0000 7f12e3200640 1 -- 172.21.15.63:0/3138540885 >> 172.21.15.63:0/3138540885 conn(0x7f12e4076070 msgr2=0x7f12e4107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.092+0000 7f12e3200640 1 -- 172.21.15.63:0/3138540885 shutdown_connections 2024-05-27T07:54:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:26.092+0000 7f12e3200640 1 -- 172.21.15.63:0/3138540885 wait complete. 2024-05-27T07:54:27.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:27 smithi086 ceph-mon[34687]: pgmap v659: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:54:27.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:27 smithi149 ceph-mon[35408]: pgmap v659: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:54:27.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:27 smithi063 ceph-mon[31613]: pgmap v659: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T07:54:29.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:29 smithi086 ceph-mon[34687]: pgmap v660: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:54:29.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:29 smithi149 ceph-mon[35408]: pgmap v660: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:54:29.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:29 smithi063 ceph-mon[31613]: pgmap v660: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:54:31.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:31 smithi086 ceph-mon[34687]: pgmap v661: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:54:31.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:31 smithi149 ceph-mon[35408]: pgmap v661: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:54:31.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:31 smithi063 ceph-mon[31613]: pgmap v661: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:54:33.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:33 smithi086 ceph-mon[34687]: pgmap v662: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:54:33.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:33 smithi149 ceph-mon[35408]: pgmap v662: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:54:33.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:33 smithi063 ceph-mon[31613]: pgmap v662: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:54:35.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:35 smithi086 ceph-mon[34687]: pgmap v663: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:54:35.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:35 smithi149 ceph-mon[35408]: pgmap v663: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:54:35.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:35 smithi063 ceph-mon[31613]: pgmap v663: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:54:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:37 smithi086 ceph-mon[34687]: pgmap v664: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:54:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:37 smithi149 ceph-mon[35408]: pgmap v664: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:54:37.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:37 smithi063 ceph-mon[31613]: pgmap v664: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:54:38.219 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:54:38.742 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:39.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:39 smithi063 ceph-mon[31613]: pgmap v665: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:54:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.332+0000 7f5a9b600640 1 -- 172.21.15.63:0/4089184323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a94114af0 msgr2=0x7f5a94116f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.332+0000 7f5a9b600640 1 --2- 172.21.15.63:0/4089184323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a94114af0 0x7f5a94116f10 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009900 tx=0x7f5a8c02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.332+0000 7f5a9b600640 1 -- 172.21.15.63:0/4089184323 shutdown_connections 2024-05-27T07:54:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.332+0000 7f5a9b600640 1 --2- 172.21.15.63:0/4089184323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a94114af0 0x7f5a94116f10 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.332+0000 7f5a9b600640 1 --2- 172.21.15.63:0/4089184323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a94112190 0x7f5a941145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.332+0000 7f5a9b600640 1 --2- 172.21.15.63:0/4089184323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a94077610 0x7f5a94077a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.332+0000 7f5a9b600640 1 -- 172.21.15.63:0/4089184323 >> 172.21.15.63:0/4089184323 conn(0x7f5a94101520 msgr2=0x7f5a94103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.333+0000 7f5a9b600640 1 -- 172.21.15.63:0/4089184323 shutdown_connections 2024-05-27T07:54:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.333+0000 7f5a9b600640 1 -- 172.21.15.63:0/4089184323 wait complete. 2024-05-27T07:54:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.334+0000 7f5a9b600640 1 Processor -- start 2024-05-27T07:54:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.334+0000 7f5a9b600640 1 -- start start 2024-05-27T07:54:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a9b600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a94077610 0x7f5a941a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a9b600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a94112190 0x7f5a941a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a9b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a94114af0 0x7f5a941a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a9b600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9411a380 con 0x7f5a94112190 2024-05-27T07:54:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a9b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9411a200 con 0x7f5a94114af0 2024-05-27T07:54:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a9b600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9411a500 con 0x7f5a94077610 2024-05-27T07:54:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a93e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a94112190 0x7f5a941a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a93e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a94112190 0x7f5a941a12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34806/0 (socket says 172.21.15.63:34806) 2024-05-27T07:54:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a93e00640 1 -- 172.21.15.63:0/3352681519 learned_addr learned my addr 172.21.15.63:0/3352681519 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a98c00640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a94077610 0x7f5a941a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.335+0000 7f5a99600640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a94114af0 0x7f5a941a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.336+0000 7f5a93e00640 1 -- 172.21.15.63:0/3352681519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a94077610 msgr2=0x7f5a941a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.336+0000 7f5a93e00640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a94077610 0x7f5a941a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.336+0000 7f5a93e00640 1 -- 172.21.15.63:0/3352681519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a94114af0 msgr2=0x7f5a941a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.336+0000 7f5a93e00640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a94114af0 0x7f5a941a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.336+0000 7f5a93e00640 1 -- 172.21.15.63:0/3352681519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a941a5be0 con 0x7f5a94112190 2024-05-27T07:54:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.336+0000 7f5a93e00640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a94112190 0x7f5a941a12e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000e9f0 tx=0x7f5a8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.337+0000 7f5a91600640 1 -- 172.21.15.63:0/3352681519 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000ccc0 con 0x7f5a94112190 2024-05-27T07:54:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.337+0000 7f5a91600640 1 -- 172.21.15.63:0/3352681519 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a800054b0 con 0x7f5a94112190 2024-05-27T07:54:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.337+0000 7f5a91600640 1 -- 172.21.15.63:0/3352681519 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80010430 con 0x7f5a94112190 2024-05-27T07:54:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.337+0000 7f5a9b600640 1 -- 172.21.15.63:0/3352681519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a941a5ed0 con 0x7f5a94112190 2024-05-27T07:54:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.337+0000 7f5a9b600640 1 -- 172.21.15.63:0/3352681519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a941a63d0 con 0x7f5a94112190 2024-05-27T07:54:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.342+0000 7f5aa2619740 1 Processor -- start 2024-05-27T07:54:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.342+0000 7f5aa2619740 1 -- start start 2024-05-27T07:54:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.343+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a309f1b0 0x55b1a3441860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.343+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 0x55b1a3441da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.343+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a307e4d0 0x55b1a34422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.343+0000 7f5aa2619740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b1a33b2b90 con 0x55b1a307e4d0 2024-05-27T07:54:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.343+0000 7f5aa2619740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b1a33a2730 con 0x55b1a309f1b0 2024-05-27T07:54:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.343+0000 7f5aa2619740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b1a33a6770 con 0x55b1a30762f0 2024-05-27T07:54:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.343+0000 7f5a98c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 0x55b1a3441da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.343+0000 7f5a98c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 0x55b1a3441da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:50434/0 (socket says 172.21.15.63:50434) 2024-05-27T07:54:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.343+0000 7f5a98c00640 1 -- 172.21.15.63:0/3033033437 learned_addr learned my addr 172.21.15.63:0/3033033437 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.343+0000 7f5a99600640 1 --2- 172.21.15.63:0/3033033437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a309f1b0 0x55b1a3441860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.344+0000 7f5a91600640 1 -- 172.21.15.63:0/3352681519 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5a800046c0 con 0x7f5a94112190 2024-05-27T07:54:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.344+0000 7f5a98c00640 1 -- 172.21.15.63:0/3033033437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a309f1b0 msgr2=0x55b1a3441860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.344+0000 7f5a98c00640 1 --2- 172.21.15.63:0/3033033437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a309f1b0 0x55b1a3441860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.344+0000 7f5a98c00640 1 -- 172.21.15.63:0/3033033437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a307e4d0 msgr2=0x55b1a34422e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:54:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.344+0000 7f5a98c00640 1 --2- 172.21.15.63:0/3033033437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a307e4d0 0x55b1a34422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.344+0000 7f5a98c00640 1 -- 172.21.15.63:0/3033033437 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a333b8b0 con 0x55b1a30762f0 2024-05-27T07:54:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.345+0000 7f5a98c00640 1 --2- 172.21.15.63:0/3033033437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 0x55b1a3441da0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a88003a20 tx=0x7f5a88003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183e98d70a4ecf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.345+0000 7f5a85c00640 1 -- 172.21.15.63:0/3033033437 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88004860 con 0x55b1a30762f0 2024-05-27T07:54:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.345+0000 7f5a85c00640 1 -- 172.21.15.63:0/3033033437 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a88015030 con 0x55b1a30762f0 2024-05-27T07:54:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.345+0000 7f5a85c00640 1 -- 172.21.15.63:0/3033033437 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88010880 con 0x55b1a30762f0 2024-05-27T07:54:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.346+0000 7f5a91600640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a7c0b29d0 0x7f5a7c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.346+0000 7f5a91600640 1 -- 172.21.15.63:0/3352681519 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f5a800e2c40 con 0x7f5a94112190 2024-05-27T07:54:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.347+0000 7f5a99600640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a7c0b29d0 0x7f5a7c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.347+0000 7f5aa2619740 1 -- 172.21.15.63:0/3033033437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 msgr2=0x55b1a3441da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.347+0000 7f5aa2619740 1 --2- 172.21.15.63:0/3033033437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 0x55b1a3441da0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a88003a20 tx=0x7f5a88003ee0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.347+0000 7f5aa2619740 1 -- 172.21.15.63:0/3033033437 shutdown_connections 2024-05-27T07:54:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.347+0000 7f5aa2619740 1 --2- 172.21.15.63:0/3033033437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a307e4d0 0x55b1a34422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.347+0000 7f5aa2619740 1 --2- 172.21.15.63:0/3033033437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 0x55b1a3441da0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.347+0000 7f5aa2619740 1 --2- 172.21.15.63:0/3033033437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a309f1b0 0x55b1a3441860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.347+0000 7f5aa2619740 1 -- 172.21.15.63:0/3033033437 >> 172.21.15.63:0/3033033437 conn(0x55b1a3093300 msgr2=0x55b1a343e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.347+0000 7f5aa2619740 1 -- 172.21.15.63:0/3033033437 shutdown_connections 2024-05-27T07:54:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.347+0000 7f5aa2619740 1 -- 172.21.15.63:0/3033033437 wait complete. 2024-05-27T07:54:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.348+0000 7f5a99600640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a7c0b29d0 0x7f5a7c0b4ee0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c004770 tx=0x7f5a8c0046c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.348+0000 7f5aa2619740 1 -- start start 2024-05-27T07:54:39.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.348+0000 7f5aa2619740 1 Processor -- start 2024-05-27T07:54:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.349+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 0x55b1a344ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.349+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a307e4d0 0x55b1a344bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.349+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a309f1b0 0x55b1a3457cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.349+0000 7f5aa2619740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b1a3088790 con 0x55b1a30762f0 2024-05-27T07:54:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.349+0000 7f5aa2619740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b1a334abc0 con 0x55b1a309f1b0 2024-05-27T07:54:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.349+0000 7f5aa2619740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b1a344e990 con 0x55b1a307e4d0 2024-05-27T07:54:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.349+0000 7f5aa2619740 10 client.0 _ll_register_callbacks cb 0x55b1a3328400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.349+0000 7f5a98c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 0x55b1a344ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.350+0000 7f5a93e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a307e4d0 0x55b1a344bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.350+0000 7f5a93e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a307e4d0 0x55b1a344bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:50446/0 (socket says 172.21.15.63:50446) 2024-05-27T07:54:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.350+0000 7f5a93e00640 1 -- 172.21.15.63:0/222171697 learned_addr learned my addr 172.21.15.63:0/222171697 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.350+0000 7f5a99600640 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a309f1b0 0x55b1a3457cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.350+0000 7f5a93e00640 1 -- 172.21.15.63:0/222171697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a309f1b0 msgr2=0x55b1a3457cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.350+0000 7f5a93e00640 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a309f1b0 0x55b1a3457cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.350+0000 7f5a93e00640 1 -- 172.21.15.63:0/222171697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 msgr2=0x55b1a344ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.350+0000 7f5a93e00640 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 0x55b1a344ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.350+0000 7f5a93e00640 1 -- 172.21.15.63:0/222171697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a30b7210 con 0x55b1a307e4d0 2024-05-27T07:54:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.350+0000 7f5a98c00640 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 0x55b1a344ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.351+0000 7f5a93e00640 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a307e4d0 0x55b1a344bfa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a800a5be0 tx=0x7f5a800a9570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d09bd3ae0a60a0c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.351+0000 7f5a7a000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:54:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.351+0000 7f5a7a000640 1 -- 172.21.15.63:0/222171697 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a800c3060 con 0x55b1a307e4d0 2024-05-27T07:54:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.351+0000 7f5aa2619740 1 -- 172.21.15.63:0/222171697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a88015030 con 0x55b1a307e4d0 2024-05-27T07:54:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.351+0000 7f5a7a000640 1 -- 172.21.15.63:0/222171697 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f5a800b2740 con 0x55b1a307e4d0 2024-05-27T07:54:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.351+0000 7f5a7a000640 1 -- 172.21.15.63:0/222171697 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5a800c4250 con 0x55b1a307e4d0 2024-05-27T07:54:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.351+0000 7f5aa2619740 10 client.28570 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.351+0000 7f5aa2619740 1 -- 172.21.15.63:0/222171697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b1a3183620 con 0x55b1a307e4d0 2024-05-27T07:54:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.352+0000 7f5a7a000640 1 -- 172.21.15.63:0/222171697 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a88015030 con 0x55b1a307e4d0 2024-05-27T07:54:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.352+0000 7f5a7a000640 1 -- 172.21.15.63:0/222171697 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5a800a54d0 con 0x55b1a307e4d0 2024-05-27T07:54:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.352+0000 7f5aa2619740 10 client.28570 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.352+0000 7f5aa2619740 10 client.28570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.352+0000 7f5aa2619740 20 client.28570 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.352+0000 7f5aa2619740 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a305dd80 0x55b1a345dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.353+0000 7f5aa2619740 4 client.28570 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.353+0000 7f5aa2619740 1 -- 172.21.15.63:0/222171697 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1a33b4ae0 con 0x55b1a305dd80 2024-05-27T07:54:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.353+0000 7f5a98c00640 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a305dd80 0x55b1a345dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.354+0000 7f5a98c00640 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a305dd80 0x55b1a345dcd0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b796e563490302c5 server_cookie=1672369a6ef530d6 in_seq=0 out_seq=0 2024-05-27T07:54:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.354+0000 7f5a7a000640 10 client.28570 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5a7a000640 1 -- 172.21.15.63:0/222171697 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b1a33b4ae0 con 0x55b1a305dd80 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5a7a000640 10 client.28570 handle_command_reply: tid=0 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 client.28570 shutdown 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 -- 172.21.15.63:0/222171697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a307e4d0 msgr2=0x55b1a344bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a307e4d0 0x55b1a344bfa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a800a5be0 tx=0x7f5a800a9570 comp rx=0 tx=0).stop 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 -- 172.21.15.63:0/222171697 shutdown_connections 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a305dd80 0x55b1a345dcd0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a309f1b0 0x55b1a3457cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a307e4d0 0x55b1a344bfa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5a7a000640 0 client.28570 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 --2- 172.21.15.63:0/222171697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 0x55b1a344ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 -- 172.21.15.63:0/222171697 >> 172.21.15.63:0/222171697 conn(0x55b1a3093300 msgr2=0x55b1a3443580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 -- 172.21.15.63:0/222171697 shutdown_connections 2024-05-27T07:54:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.356+0000 7f5aa2619740 1 -- 172.21.15.63:0/222171697 wait complete. 2024-05-27T07:54:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.357+0000 7f5aa2619740 20 client.28570 trim_cache size 0 max 16384 2024-05-27T07:54:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.379+0000 7f5aa2619740 1 Processor -- start 2024-05-27T07:54:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.379+0000 7f5aa2619740 1 -- start start 2024-05-27T07:54:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 0x55b1a344bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 0x55b1a3457a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a309f1b0 0x55b1a3457f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5aa2619740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941ba0d0 con 0x55b1a309f1b0 2024-05-27T07:54:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5aa2619740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941b79a0 con 0x55b1a307e4d0 2024-05-27T07:54:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5aa2619740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941b5810 con 0x55b1a30762f0 2024-05-27T07:54:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5a99600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a309f1b0 0x55b1a3457f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5a99600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a309f1b0 0x55b1a3457f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34832/0 (socket says 172.21.15.63:34832) 2024-05-27T07:54:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5a99600640 1 -- 172.21.15.63:0/2015836406 learned_addr learned my addr 172.21.15.63:0/2015836406 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5a98c00640 1 --2- 172.21.15.63:0/2015836406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 0x55b1a344bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.380+0000 7f5a93e00640 1 --2- 172.21.15.63:0/2015836406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 0x55b1a3457a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.381+0000 7f5a99600640 1 -- 172.21.15.63:0/2015836406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 msgr2=0x55b1a344bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.381+0000 7f5a99600640 1 --2- 172.21.15.63:0/2015836406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 0x55b1a344bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.381+0000 7f5a99600640 1 -- 172.21.15.63:0/2015836406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 msgr2=0x55b1a3457a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.381+0000 7f5a99600640 1 --2- 172.21.15.63:0/2015836406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 0x55b1a3457a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.381+0000 7f5a99600640 1 -- 172.21.15.63:0/2015836406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a3438050 con 0x55b1a309f1b0 2024-05-27T07:54:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.381+0000 7f5a99600640 1 --2- 172.21.15.63:0/2015836406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a309f1b0 0x55b1a3457f70 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0057c0 tx=0x7f5a8c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1467d4a50b21b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.382+0000 7f5a85200640 1 -- 172.21.15.63:0/2015836406 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c035040 con 0x55b1a309f1b0 2024-05-27T07:54:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.382+0000 7f5a85200640 1 -- 172.21.15.63:0/2015836406 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a8c003fa0 con 0x55b1a309f1b0 2024-05-27T07:54:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.382+0000 7f5a85200640 1 -- 172.21.15.63:0/2015836406 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c03d7c0 con 0x55b1a309f1b0 2024-05-27T07:54:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.383+0000 7f5aa2619740 1 -- 172.21.15.63:0/2015836406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a309f1b0 msgr2=0x55b1a3457f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.383+0000 7f5aa2619740 1 --2- 172.21.15.63:0/2015836406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a309f1b0 0x55b1a3457f70 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0057c0 tx=0x7f5a8c040000 comp rx=0 tx=0).stop 2024-05-27T07:54:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.383+0000 7f5aa2619740 1 -- 172.21.15.63:0/2015836406 shutdown_connections 2024-05-27T07:54:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.383+0000 7f5aa2619740 1 --2- 172.21.15.63:0/2015836406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a309f1b0 0x55b1a3457f70 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.383+0000 7f5aa2619740 1 --2- 172.21.15.63:0/2015836406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 0x55b1a3457a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.383+0000 7f5aa2619740 1 --2- 172.21.15.63:0/2015836406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a30762f0 0x55b1a344bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.383+0000 7f5aa2619740 1 -- 172.21.15.63:0/2015836406 >> 172.21.15.63:0/2015836406 conn(0x55b1a3093300 msgr2=0x55b1a3446c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.383+0000 7f5aa2619740 1 -- 172.21.15.63:0/2015836406 shutdown_connections 2024-05-27T07:54:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.383+0000 7f5aa2619740 1 -- 172.21.15.63:0/2015836406 wait complete. 2024-05-27T07:54:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.384+0000 7f5aa2619740 1 -- start start 2024-05-27T07:54:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.384+0000 7f5aa2619740 1 Processor -- start 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 0x55b1a344f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 0x55b1a344f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5aa2619740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a309f1b0 0x55b1a34be210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5aa2619740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941b5e60 con 0x55b1a30762f0 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5aa2619740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941b72d0 con 0x55b1a307e4d0 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5aa2619740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941b6530 con 0x55b1a309f1b0 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5aa2619740 10 client.0 _ll_register_callbacks cb 0x7f5a8000cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5a98c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 0x55b1a344f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5a98c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 0x55b1a344f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34836/0 (socket says 172.21.15.63:34836) 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5a98c00640 1 -- 172.21.15.63:0/3223790075 learned_addr learned my addr 172.21.15.63:0/3223790075 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5a93e00640 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 0x55b1a344f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.385+0000 7f5a99600640 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a309f1b0 0x55b1a34be210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.386+0000 7f5a93e00640 1 -- 172.21.15.63:0/3223790075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a309f1b0 msgr2=0x55b1a34be210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.386+0000 7f5a93e00640 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a309f1b0 0x55b1a34be210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.386+0000 7f5a93e00640 1 -- 172.21.15.63:0/3223790075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 msgr2=0x55b1a344f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.386+0000 7f5a93e00640 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 0x55b1a344f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.386+0000 7f5a93e00640 1 -- 172.21.15.63:0/3223790075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a34be7d0 con 0x55b1a307e4d0 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.386+0000 7f5a93e00640 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 0x55b1a344f970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5a800a9860 tx=0x7f5a80005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269a74cb326a4e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.386+0000 7f5a7a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.387+0000 7f5a7a000640 1 -- 172.21.15.63:0/3223790075 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a800a9930 con 0x55b1a307e4d0 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.387+0000 7f5aa2619740 1 -- 172.21.15.63:0/3223790075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b1a33ae2c0 con 0x55b1a307e4d0 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.387+0000 7f5a7a000640 1 -- 172.21.15.63:0/3223790075 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f5a800a8dc0 con 0x55b1a307e4d0 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.387+0000 7f5a7a000640 1 -- 172.21.15.63:0/3223790075 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5a800a9670 con 0x55b1a307e4d0 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.387+0000 7f5aa2619740 10 client.38714 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.387+0000 7f5aa2619740 1 -- 172.21.15.63:0/3223790075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b1a34c5b50 con 0x55b1a307e4d0 2024-05-27T07:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.387+0000 7f5a7a000640 1 -- 172.21.15.63:0/3223790075 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b1a33ae2c0 con 0x55b1a307e4d0 2024-05-27T07:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.388+0000 7f5a7a000640 1 -- 172.21.15.63:0/3223790075 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5a80004f20 con 0x55b1a307e4d0 2024-05-27T07:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.388+0000 7f5aa2619740 10 client.38714 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.388+0000 7f5aa2619740 10 client.38714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.388+0000 7f5aa2619740 20 client.38714 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.388+0000 7f5aa2619740 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a30331c0 0x55b1a34c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.388+0000 7f5aa2619740 4 client.38714 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:54:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.388+0000 7f5aa2619740 1 -- 172.21.15.63:0/3223790075 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1a33284b0 con 0x55b1a30331c0 2024-05-27T07:54:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.389+0000 7f5a98c00640 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a30331c0 0x55b1a34c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.389+0000 7f5a98c00640 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a30331c0 0x55b1a34c8340 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c390b8f3967d69fa server_cookie=bd434beb255043e7 in_seq=0 out_seq=0 2024-05-27T07:54:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.389+0000 7f5a7a000640 10 client.38714 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.413+0000 7f5a7a000640 1 -- 172.21.15.63:0/3223790075 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b1a33284b0 con 0x55b1a30331c0 2024-05-27T07:54:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.413+0000 7f5a7a000640 10 client.38714 handle_command_reply: tid=0 2024-05-27T07:54:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.413+0000 7f5aa2619740 1 client.38714 shutdown 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 1 -- 172.21.15.63:0/3223790075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 msgr2=0x55b1a344f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 0x55b1a344f970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5a800a9860 tx=0x7f5a80005680 comp rx=0 tx=0).stop 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 1 -- 172.21.15.63:0/3223790075 shutdown_connections 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b1a30331c0 0x55b1a34c8340 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b1a309f1b0 0x55b1a34be210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b1a307e4d0 0x55b1a344f970 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5a7a000640 0 client.38714 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 1 --2- 172.21.15.63:0/3223790075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b1a30762f0 0x55b1a344f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 1 -- 172.21.15.63:0/3223790075 >> 172.21.15.63:0/3223790075 conn(0x55b1a3093300 msgr2=0x55b1a344afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 1 -- 172.21.15.63:0/3223790075 shutdown_connections 2024-05-27T07:54:39.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 1 -- 172.21.15.63:0/3223790075 wait complete. 2024-05-27T07:54:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.414+0000 7f5aa2619740 20 client.38714 trim_cache size 0 max 16384 2024-05-27T07:54:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:54:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:54:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:54:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.415+0000 7f5a9b600640 1 -- 172.21.15.63:0/3352681519 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a7c0b29d0 msgr2=0x7f5a7c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.415+0000 7f5a9b600640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a7c0b29d0 0x7f5a7c0b4ee0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c004770 tx=0x7f5a8c0046c0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.415+0000 7f5a9b600640 1 -- 172.21.15.63:0/3352681519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a94112190 msgr2=0x7f5a941a12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.415+0000 7f5a9b600640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a94112190 0x7f5a941a12e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000e9f0 tx=0x7f5a8000eeb0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.416+0000 7f5a9b600640 1 -- 172.21.15.63:0/3352681519 shutdown_connections 2024-05-27T07:54:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.416+0000 7f5a9b600640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a7c0b29d0 0x7f5a7c0b4ee0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.416+0000 7f5a9b600640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a94114af0 0x7f5a941a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.416+0000 7f5a9b600640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a94112190 0x7f5a941a12e0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.416+0000 7f5a9b600640 1 --2- 172.21.15.63:0/3352681519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a94077610 0x7f5a941a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.416+0000 7f5a9b600640 1 -- 172.21.15.63:0/3352681519 >> 172.21.15.63:0/3352681519 conn(0x7f5a94101520 msgr2=0x7f5a94113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.416+0000 7f5a9b600640 1 -- 172.21.15.63:0/3352681519 shutdown_connections 2024-05-27T07:54:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:39.416+0000 7f5a9b600640 1 -- 172.21.15.63:0/3352681519 wait complete. 2024-05-27T07:54:39.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:39 smithi086 ceph-mon[34687]: pgmap v665: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:54:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:39 smithi149 ceph-mon[35408]: pgmap v665: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:54:40.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:40 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:54:40.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:40 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:54:40.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:40 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:54:41.394 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:41 smithi063 ceph-mon[31613]: pgmap v666: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:54:41.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:41 smithi086 ceph-mon[34687]: pgmap v666: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:54:41.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:41 smithi149 ceph-mon[35408]: pgmap v666: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:54:41.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.392+0000 7fa35e800700 1 -- 172.21.15.63:0/2276692640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa358105300 msgr2=0x7fa358105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:41.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.392+0000 7fa35e800700 1 --2- 172.21.15.63:0/2276692640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa358105300 0x7fa358105710 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009990 tx=0x7fa34c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.392+0000 7fa35e800700 1 -- 172.21.15.63:0/2276692640 shutdown_connections 2024-05-27T07:54:41.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.392+0000 7fa35e800700 1 --2- 172.21.15.63:0/2276692640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa35810e730 0x7fa358110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.392+0000 7fa35e800700 1 --2- 172.21.15.63:0/2276692640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa358105300 0x7fa358105710 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.392+0000 7fa35e800700 1 --2- 172.21.15.63:0/2276692640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa358105c50 0x7fa35810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.392+0000 7fa35e800700 1 -- 172.21.15.63:0/2276692640 >> 172.21.15.63:0/2276692640 conn(0x7fa358078ab0 msgr2=0x7fa358078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.392+0000 7fa35e800700 1 -- 172.21.15.63:0/2276692640 shutdown_connections 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.393+0000 7fa35e800700 1 -- 172.21.15.63:0/2276692640 wait complete. 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.394+0000 7fa35e800700 1 Processor -- start 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.394+0000 7fa35e800700 1 -- start start 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.395+0000 7fa35e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa358105300 0x7fa3581a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.395+0000 7fa35e800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa358105c50 0x7fa3581a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.395+0000 7fa35e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa35810e730 0x7fa35819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.395+0000 7fa35e800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa358116330 con 0x7fa358105300 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.395+0000 7fa35e800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581161b0 con 0x7fa35810e730 2024-05-27T07:54:41.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.395+0000 7fa35e800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581164b0 con 0x7fa358105c50 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.395+0000 7fa35d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa358105300 0x7fa3581a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.395+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa358105c50 0x7fa3581a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.395+0000 7fa35de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa35810e730 0x7fa35819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.395+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa358105c50 0x7fa3581a2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:50476/0 (socket says 172.21.15.63:50476) 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.396+0000 7fa35ca00700 1 -- 172.21.15.63:0/590665706 learned_addr learned my addr 172.21.15.63:0/590665706 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.396+0000 7fa35de00700 1 -- 172.21.15.63:0/590665706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa358105c50 msgr2=0x7fa3581a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.396+0000 7fa35de00700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa358105c50 0x7fa3581a2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.396+0000 7fa35de00700 1 -- 172.21.15.63:0/590665706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa358105300 msgr2=0x7fa3581a29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.396+0000 7fa35de00700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa358105300 0x7fa3581a29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.396+0000 7fa35de00700 1 -- 172.21.15.63:0/590665706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35819d260 con 0x7fa35810e730 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.396+0000 7fa35ca00700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa358105c50 0x7fa3581a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.397+0000 7fa35de00700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa35810e730 0x7fa35819cb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa35000efc0 tx=0x7fa35000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.397+0000 7fa356000700 1 -- 172.21.15.63:0/590665706 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500092e0 con 0x7fa35810e730 2024-05-27T07:54:41.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.397+0000 7fa356000700 1 -- 172.21.15.63:0/590665706 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa350004bb0 con 0x7fa35810e730 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.397+0000 7fa35e800700 1 -- 172.21.15.63:0/590665706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35819d4f0 con 0x7fa35810e730 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.397+0000 7fa356000700 1 -- 172.21.15.63:0/590665706 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350004020 con 0x7fa35810e730 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.397+0000 7fa35e800700 1 -- 172.21.15.63:0/590665706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3581a9750 con 0x7fa35810e730 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.402+0000 7fa356000700 1 -- 172.21.15.63:0/590665706 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa35001b030 con 0x7fa35810e730 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.402+0000 7fa36e9adb80 1 Processor -- start 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.402+0000 7fa36e9adb80 1 -- start start 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.403+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565226f0ce40 0x56522708aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.403+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b410 0x565227095ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.403+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b820 0x565227098020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.403+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x565226fb9e40 con 0x565226f0ce40 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.403+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565226fa99e0 con 0x56522708b820 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.403+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x565226fada20 con 0x56522708b410 2024-05-27T07:54:41.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.403+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565226f0ce40 0x56522708aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.403+0000 7fa35d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b410 0x565227095ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.403+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565226f0ce40 0x56522708aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34864/0 (socket says 172.21.15.63:34864) 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.403+0000 7fa35ca00700 1 -- 172.21.15.63:0/3097581801 learned_addr learned my addr 172.21.15.63:0/3097581801 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.404+0000 7fa35ca00700 1 --2- 172.21.15.63:0/3097581801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b820 0x565227098020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.404+0000 7fa35d400700 1 -- 172.21.15.63:0/3097581801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b820 msgr2=0x565227098020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.404+0000 7fa35d400700 1 --2- 172.21.15.63:0/3097581801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b820 0x565227098020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.404+0000 7fa35d400700 1 -- 172.21.15.63:0/3097581801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565226f0ce40 msgr2=0x56522708aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.404+0000 7fa35d400700 1 --2- 172.21.15.63:0/3097581801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565226f0ce40 0x56522708aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.404+0000 7fa35d400700 1 -- 172.21.15.63:0/3097581801 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565226f72420 con 0x56522708b410 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.404+0000 7fa35d400700 1 --2- 172.21.15.63:0/3097581801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b410 0x565227095ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00b650 tx=0x7fa34c0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa15ff940c286f71 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.405+0000 7fa356000700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3400b29f0 0x7fa3400b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.405+0000 7fa34aa00700 1 -- 172.21.15.63:0/3097581801 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c003680 con 0x56522708b410 2024-05-27T07:54:41.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.405+0000 7fa34aa00700 1 -- 172.21.15.63:0/3097581801 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa34c003cb0 con 0x56522708b410 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.405+0000 7fa34aa00700 1 -- 172.21.15.63:0/3097581801 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00bd00 con 0x56522708b410 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.405+0000 7fa356000700 1 -- 172.21.15.63:0/590665706 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7fa3500e2430 con 0x7fa35810e730 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.405+0000 7fa35ca00700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3400b29f0 0x7fa3400b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.406+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3097581801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b410 msgr2=0x565227095ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.406+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/3097581801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b410 0x565227095ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00b650 tx=0x7fa34c0096a0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.406+0000 7fa35ca00700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3400b29f0 0x7fa3400b4f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa344009880 tx=0x7fa344015040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.406+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3097581801 shutdown_connections 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.406+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/3097581801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b820 0x565227098020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.406+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/3097581801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565226f0ce40 0x56522708aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.406+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/3097581801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b410 0x565227095ad0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.406+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3097581801 >> 172.21.15.63:0/3097581801 conn(0x565226f6cfc0 msgr2=0x565227086d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.407+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3097581801 shutdown_connections 2024-05-27T07:54:41.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.407+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3097581801 wait complete. 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.407+0000 7fa36e9adb80 1 -- start start 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.408+0000 7fa36e9adb80 1 Processor -- start 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.408+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565226f0ce40 0x5652270954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.408+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b410 0x565227095a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.408+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b820 0x565227095f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.408+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x565226eaffa0 con 0x56522708b410 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.408+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565226f4a760 con 0x565226f0ce40 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.408+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5652270988d0 con 0x56522708b820 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.408+0000 7fa36e9adb80 10 client.0 _ll_register_callbacks cb 0x565226f53c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.409+0000 7fa35d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565226f0ce40 0x5652270954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.409+0000 7fa35de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b820 0x565227095f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.409+0000 7fa35d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565226f0ce40 0x5652270954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:58102/0 (socket says 172.21.15.63:58102) 2024-05-27T07:54:41.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.409+0000 7fa35de00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b820 0x565227095f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:50488/0 (socket says 172.21.15.63:50488) 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.409+0000 7fa35d400700 1 -- 172.21.15.63:0/3656761864 learned_addr learned my addr 172.21.15.63:0/3656761864 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.409+0000 7fa35ca00700 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b410 0x565227095a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.410+0000 7fa35d400700 1 -- 172.21.15.63:0/3656761864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b820 msgr2=0x565227095f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.410+0000 7fa35d400700 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b820 0x565227095f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.410+0000 7fa35d400700 1 -- 172.21.15.63:0/3656761864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b410 msgr2=0x565227095a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.410+0000 7fa35d400700 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b410 0x565227095a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.410+0000 7fa35d400700 1 -- 172.21.15.63:0/3656761864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565226ef98d0 con 0x565226f0ce40 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.410+0000 7fa35de00700 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b820 0x565227095f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.410+0000 7fa35d400700 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565226f0ce40 0x5652270954e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005570 tx=0x7fa34c0055a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db339b4b4565f7a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.410+0000 7fa33f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.411+0000 7fa33f400700 1 -- 172.21.15.63:0/3656761864 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c032060 con 0x565226f0ce40 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.411+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3656761864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa34c003cb0 con 0x565226f0ce40 2024-05-27T07:54:41.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.411+0000 7fa33f400700 1 -- 172.21.15.63:0/3656761864 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7fa34c003890 con 0x565226f0ce40 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.411+0000 7fa33f400700 1 -- 172.21.15.63:0/3656761864 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa34c003a90 con 0x565226f0ce40 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.411+0000 7fa36e9adb80 10 client.38726 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.411+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3656761864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x565226f7a060 con 0x565226f0ce40 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.412+0000 7fa33f400700 1 -- 172.21.15.63:0/3656761864 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa34c003cb0 con 0x565226f0ce40 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.412+0000 7fa33f400700 1 -- 172.21.15.63:0/3656761864 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa34c0229e0 con 0x565226f0ce40 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.412+0000 7fa36e9adb80 10 client.38726 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.412+0000 7fa36e9adb80 10 client.38726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.412+0000 7fa36e9adb80 20 client.38726 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.412+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5652270a5630 0x5652270a7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.412+0000 7fa36e9adb80 4 client.38726 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.412+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3656761864 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565226fbbd90 con 0x5652270a5630 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.413+0000 7fa35ca00700 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5652270a5630 0x5652270a7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.414+0000 7fa35ca00700 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5652270a5630 0x5652270a7a50 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=678a694a41e4ffd7 server_cookie=667bca76f8651cc4 in_seq=0 out_seq=0 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.414+0000 7fa33f400700 10 client.38726 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:41.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa33f400700 1 -- 172.21.15.63:0/3656761864 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565226fbbd90 con 0x5652270a5630 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa33f400700 10 client.38726 handle_command_reply: tid=0 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa36e9adb80 1 client.38726 shutdown 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3656761864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565226f0ce40 msgr2=0x5652270954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565226f0ce40 0x5652270954e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005570 tx=0x7fa34c0055a0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3656761864 shutdown_connections 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5652270a5630 0x5652270a7a50 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56522708b820 0x565227095f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565226f0ce40 0x5652270954e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/3656761864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b410 0x565227095a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa33f400700 0 client.38726 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.415+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3656761864 >> 172.21.15.63:0/3656761864 conn(0x565226f6cfc0 msgr2=0x565227085ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:41.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.416+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3656761864 shutdown_connections 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.416+0000 7fa36e9adb80 1 -- 172.21.15.63:0/3656761864 wait complete. 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.416+0000 7fa36e9adb80 20 client.38726 trim_cache size 0 max 16384 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.443+0000 7fa36e9adb80 1 Processor -- start 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.443+0000 7fa36e9adb80 1 -- start start 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.444+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 0x565227097980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.444+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x565227090c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.444+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b820 0x5652270aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.444+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581b4c70 con 0x56522708b820 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.444+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581b2500 con 0x56522708b410 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.444+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581b0370 con 0x565226f0ce40 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.445+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x565227090c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.445+0000 7fa35d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 0x565227097980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.445+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x565227090c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:58104/0 (socket says 172.21.15.63:58104) 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.445+0000 7fa35ca00700 1 -- 172.21.15.63:0/2371145034 learned_addr learned my addr 172.21.15.63:0/2371145034 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.445+0000 7fa35de00700 1 --2- 172.21.15.63:0/2371145034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b820 0x5652270aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.445+0000 7fa35d400700 1 -- 172.21.15.63:0/2371145034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 msgr2=0x565227090c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.445+0000 7fa35d400700 1 --2- 172.21.15.63:0/2371145034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x565227090c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.445+0000 7fa35d400700 1 -- 172.21.15.63:0/2371145034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b820 msgr2=0x5652270aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.445+0000 7fa35d400700 1 --2- 172.21.15.63:0/2371145034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b820 0x5652270aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.445+0000 7fa35d400700 1 -- 172.21.15.63:0/2371145034 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652270965a0 con 0x565226f0ce40 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.446+0000 7fa35d400700 1 --2- 172.21.15.63:0/2371145034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 0x565227097980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0098e0 tx=0x7fa34c00b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1475eaf2c15099e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.446+0000 7fa34a000700 1 -- 172.21.15.63:0/2371145034 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c032070 con 0x565226f0ce40 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.446+0000 7fa34a000700 1 -- 172.21.15.63:0/2371145034 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa34c00b320 con 0x565226f0ce40 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.446+0000 7fa34a000700 1 -- 172.21.15.63:0/2371145034 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00e750 con 0x565226f0ce40 2024-05-27T07:54:41.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.447+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2371145034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 msgr2=0x565227097980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.447+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/2371145034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 0x565227097980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0098e0 tx=0x7fa34c00b6b0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.447+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2371145034 shutdown_connections 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.447+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/2371145034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b820 0x5652270aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.447+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/2371145034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 0x565227097980 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.447+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/2371145034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x565227090c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.447+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2371145034 >> 172.21.15.63:0/2371145034 conn(0x565226f6cfc0 msgr2=0x565227095cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.447+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2371145034 shutdown_connections 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.447+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2371145034 wait complete. 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.448+0000 7fa36e9adb80 1 -- start start 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.448+0000 7fa36e9adb80 1 Processor -- start 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 0x5652270ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x5652270a8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa36e9adb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b820 0x5652270a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581b09c0 con 0x56522708b820 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581b1e30 con 0x56522708b410 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa36e9adb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581b1090 con 0x565226f0ce40 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa36e9adb80 10 client.0 _ll_register_callbacks cb 0x565226f53c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x5652270a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa35d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 0x5652270ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x5652270a8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:58110/0 (socket says 172.21.15.63:58110) 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.449+0000 7fa35ca00700 1 -- 172.21.15.63:0/2260534024 learned_addr learned my addr 172.21.15.63:0/2260534024 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.450+0000 7fa35de00700 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b820 0x5652270a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.450+0000 7fa35ca00700 1 -- 172.21.15.63:0/2260534024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 msgr2=0x5652270ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.450+0000 7fa35ca00700 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 0x5652270ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.450+0000 7fa35ca00700 1 -- 172.21.15.63:0/2260534024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b820 msgr2=0x5652270a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:41.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.450+0000 7fa35ca00700 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b820 0x5652270a8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.450+0000 7fa35ca00700 1 -- 172.21.15.63:0/2260534024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652270a9180 con 0x56522708b410 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.450+0000 7fa35ca00700 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x5652270a8510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x56522708bcb0 tx=0x7fa344009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eed5f9a688a1389 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.450+0000 7fa33f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.451+0000 7fa33f400700 1 -- 172.21.15.63:0/2260534024 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344004550 con 0x56522708b410 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.451+0000 7fa33f400700 1 -- 172.21.15.63:0/2260534024 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7fa344003680 con 0x56522708b410 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.451+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2260534024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa34c00b320 con 0x56522708b410 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.451+0000 7fa33f400700 1 -- 172.21.15.63:0/2260534024 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa34403b050 con 0x56522708b410 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.451+0000 7fa36e9adb80 10 client.38732 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.451+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2260534024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x565227118610 con 0x56522708b410 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.452+0000 7fa33f400700 1 -- 172.21.15.63:0/2260534024 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa34c00b320 con 0x56522708b410 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.452+0000 7fa33f400700 1 -- 172.21.15.63:0/2260534024 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa34402c070 con 0x56522708b410 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.452+0000 7fa36e9adb80 10 client.38732 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:41.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.452+0000 7fa36e9adb80 10 client.38732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.452+0000 7fa36e9adb80 20 client.38732 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.452+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x565227118d80 0x56522711b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.452+0000 7fa36e9adb80 4 client.38732 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.452+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2260534024 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565226fb5570 con 0x565227118d80 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.452+0000 7fa35d400700 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x565227118d80 0x56522711b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.453+0000 7fa35d400700 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x565227118d80 0x56522711b1a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e62cd1cb7873d55 server_cookie=e542aa31e260441b in_seq=0 out_seq=0 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.453+0000 7fa33f400700 10 client.38732 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.453+0000 7fa33f400700 1 -- 172.21.15.63:0/2260534024 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x565226fb5570 con 0x565227118d80 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.453+0000 7fa33f400700 10 client.38732 handle_command_reply: tid=0 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.453+0000 7fa36e9adb80 1 client.38732 shutdown 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2260534024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 msgr2=0x5652270a8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x5652270a8510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x56522708bcb0 tx=0x7fa344009260 comp rx=0 tx=0).stop 2024-05-27T07:54:41.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2260534024 shutdown_connections 2024-05-27T07:54:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x565227118d80 0x56522711b1a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56522708b820 0x5652270a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565226f0ce40 0x5652270ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 1 --2- 172.21.15.63:0/2260534024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56522708b410 0x5652270a8510 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa33f400700 0 client.38732 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2260534024 >> 172.21.15.63:0/2260534024 conn(0x565226f6cfc0 msgr2=0x565227096970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2260534024 shutdown_connections 2024-05-27T07:54:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 1 -- 172.21.15.63:0/2260534024 wait complete. 2024-05-27T07:54:41.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.454+0000 7fa36e9adb80 20 client.38732 trim_cache size 0 max 16384 2024-05-27T07:54:41.793 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:54:41.794 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:54:41.794 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:54:41.794 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:54:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 -- 172.21.15.63:0/590665706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3400b29f0 msgr2=0x7fa3400b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3400b29f0 0x7fa3400b4f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa344009880 tx=0x7fa344015040 comp rx=0 tx=0).stop 2024-05-27T07:54:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 -- 172.21.15.63:0/590665706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa35810e730 msgr2=0x7fa35819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:41.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa35810e730 0x7fa35819cb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa35000efc0 tx=0x7fa35000c3a0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 -- 172.21.15.63:0/590665706 shutdown_connections 2024-05-27T07:54:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3400b29f0 0x7fa3400b4f00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa35810e730 0x7fa35819cb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa358105300 0x7fa3581a29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 --2- 172.21.15.63:0/590665706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa358105c50 0x7fa3581a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 -- 172.21.15.63:0/590665706 >> 172.21.15.63:0/590665706 conn(0x7fa358078ab0 msgr2=0x7fa3580fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 -- 172.21.15.63:0/590665706 shutdown_connections 2024-05-27T07:54:41.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:41.455+0000 7fa35e800700 1 -- 172.21.15.63:0/590665706 wait complete. 2024-05-27T07:54:42.691 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:54:42.691 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:54:43.206 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:43.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:43 smithi086 ceph-mon[34687]: pgmap v667: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:54:43.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:43 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:43 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:43 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:43 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:43 smithi149 ceph-mon[35408]: pgmap v667: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:54:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:43 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:43 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:43 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:43 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:43 smithi063 ceph-mon[31613]: pgmap v667: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:54:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:43 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:43 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:43 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:43 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:44.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:44 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T07:54:44.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:44 smithi063 ceph-mon[31613]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T07:54:44.346 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:44 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T07:54:44.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:44 smithi086 ceph-mon[34687]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T07:54:44.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:44 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T07:54:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:44 smithi149 ceph-mon[35408]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T07:54:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:45.370 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:45 smithi063 ceph-mon[31613]: pgmap v668: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:54:45.370 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:45.370 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:45 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:45.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:45 smithi086 ceph-mon[34687]: pgmap v668: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:54:45.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:45.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:45 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:45.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:45 smithi149 ceph-mon[35408]: pgmap v668: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:54:45.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:45.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:45 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:45.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.655+0000 7f09a3e00700 1 -- 172.21.15.63:0/68805015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f099c108730 msgr2=0x7f099c110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:45.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.655+0000 7f09a3e00700 1 --2- 172.21.15.63:0/68805015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f099c108730 0x7f099c110090 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f099400b0d0 tx=0x7f099400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:54:45.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.655+0000 7f09a3e00700 1 -- 172.21.15.63:0/68805015 shutdown_connections 2024-05-27T07:54:45.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.655+0000 7f09a3e00700 1 --2- 172.21.15.63:0/68805015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f099c108730 0x7f099c110090 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:45.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.655+0000 7f09a3e00700 1 --2- 172.21.15.63:0/68805015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c1034b0 0x7f099c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:45.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.655+0000 7f09a3e00700 1 --2- 172.21.15.63:0/68805015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f099c107de0 0x7f099c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:45.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.655+0000 7f09a3e00700 1 -- 172.21.15.63:0/68805015 >> 172.21.15.63:0/68805015 conn(0x7f099c0fedb0 msgr2=0x7f099c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:45.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.655+0000 7f09a3e00700 1 -- 172.21.15.63:0/68805015 shutdown_connections 2024-05-27T07:54:45.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.656+0000 7f09a3e00700 1 -- 172.21.15.63:0/68805015 wait complete. 2024-05-27T07:54:45.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.657+0000 7f09a3e00700 1 Processor -- start 2024-05-27T07:54:45.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.657+0000 7f09a3e00700 1 -- start start 2024-05-27T07:54:45.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f099c1034b0 0x7f099c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:45.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c107de0 0x7f099c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:45.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f099c108730 0x7f099c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:45.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a3e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f099c113260 con 0x7f099c108730 2024-05-27T07:54:45.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a3e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1130e0 con 0x7f099c107de0 2024-05-27T07:54:45.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a3e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1133e0 con 0x7f099c1034b0 2024-05-27T07:54:45.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f099c108730 0x7f099c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:45.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f099c108730 0x7f099c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34902/0 (socket says 172.21.15.63:34902) 2024-05-27T07:54:45.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a1e00700 1 -- 172.21.15.63:0/2038465756 learned_addr learned my addr 172.21.15.63:0/2038465756 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:45.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a1400700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f099c1034b0 0x7f099c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:45.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.658+0000 7f09a0a00700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c107de0 0x7f099c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:45.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.659+0000 7f09a1e00700 1 -- 172.21.15.63:0/2038465756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f099c1034b0 msgr2=0x7f099c1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:45.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.659+0000 7f09a1e00700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f099c1034b0 0x7f099c1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:45.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.659+0000 7f09a1e00700 1 -- 172.21.15.63:0/2038465756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c107de0 msgr2=0x7f099c1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:45.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.659+0000 7f09a1e00700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c107de0 0x7f099c1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:45.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.659+0000 7f09a1e00700 1 -- 172.21.15.63:0/2038465756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099c19b1b0 con 0x7f099c108730 2024-05-27T07:54:45.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.660+0000 7f09a1e00700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f099c108730 0x7f099c19aad0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f09940079a0 tx=0x7f09940079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:45.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.660+0000 7f099a000700 1 -- 172.21.15.63:0/2038465756 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099401f070 con 0x7f099c108730 2024-05-27T07:54:45.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.660+0000 7f099a000700 1 -- 172.21.15.63:0/2038465756 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0994003c00 con 0x7f099c108730 2024-05-27T07:54:45.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.660+0000 7f09a3e00700 1 -- 172.21.15.63:0/2038465756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099c19b440 con 0x7f099c108730 2024-05-27T07:54:45.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.660+0000 7f09a3e00700 1 -- 172.21.15.63:0/2038465756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f099c10a060 con 0x7f099c108730 2024-05-27T07:54:45.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.660+0000 7f099a000700 1 -- 172.21.15.63:0/2038465756 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099401b430 con 0x7f099c108730 2024-05-27T07:54:45.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.661+0000 7f09a3e00700 1 -- 172.21.15.63:0/2038465756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f099c104960 con 0x7f099c108730 2024-05-27T07:54:45.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.665+0000 7f099a000700 1 -- 172.21.15.63:0/2038465756 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0994022070 con 0x7f099c108730 2024-05-27T07:54:45.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.668+0000 7f099a000700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09840b29f0 0x7f09840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:45.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.668+0000 7f099a000700 1 -- 172.21.15.63:0/2038465756 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f099400f020 con 0x7f099c108730 2024-05-27T07:54:45.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.668+0000 7f09a1400700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09840b29f0 0x7f09840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:45.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.669+0000 7f09a1400700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09840b29f0 0x7f09840b4f00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f099c19bc10 tx=0x7f0990009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:45.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:45.670+0000 7f099a000700 1 -- 172.21.15.63:0/2038465756 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0994029020 con 0x7f099c108730 2024-05-27T07:54:46.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.013+0000 7f09a3e00700 1 -- 172.21.15.63:0/2038465756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f099c19bfc0 con 0x7f099c108730 2024-05-27T07:54:46.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.015+0000 7f099a000700 1 -- 172.21.15.63:0/2038465756 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f09940ab6f0 con 0x7f099c108730 2024-05-27T07:54:46.017 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:54:46.017 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:54:46.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 -- 172.21.15.63:0/2038465756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09840b29f0 msgr2=0x7f09840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:46.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09840b29f0 0x7f09840b4f00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f099c19bc10 tx=0x7f0990009260 comp rx=0 tx=0).stop 2024-05-27T07:54:46.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 -- 172.21.15.63:0/2038465756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f099c108730 msgr2=0x7f099c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:46.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f099c108730 0x7f099c19aad0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f09940079a0 tx=0x7f09940079d0 comp rx=0 tx=0).stop 2024-05-27T07:54:46.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 -- 172.21.15.63:0/2038465756 shutdown_connections 2024-05-27T07:54:46.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09840b29f0 0x7f09840b4f00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:46.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f099c108730 0x7f099c19aad0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:46.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f099c1034b0 0x7f099c1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:46.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 --2- 172.21.15.63:0/2038465756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099c107de0 0x7f099c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:46.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 -- 172.21.15.63:0/2038465756 >> 172.21.15.63:0/2038465756 conn(0x7f099c0fedb0 msgr2=0x7f099c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:46.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 -- 172.21.15.63:0/2038465756 shutdown_connections 2024-05-27T07:54:46.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:46.018+0000 7f09a3e00700 1 -- 172.21.15.63:0/2038465756 wait complete. 2024-05-27T07:54:46.021 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:54:46.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:46 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2038465756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:54:46.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:46 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2038465756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:54:46.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:46 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2038465756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:54:47.010 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:54:47.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:47 smithi063 ceph-mon[31613]: pgmap v669: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:54:47.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:47 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:47 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:47 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:47 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:47 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T07:54:47.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:47 smithi063 ceph-mon[31613]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T07:54:47.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:47 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:54:47.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:47 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:54:47.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:47 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:47 smithi086 ceph-mon[34687]: pgmap v669: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:54:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:47 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:47 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:47 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:47 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:47 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T07:54:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:47 smithi086 ceph-mon[34687]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T07:54:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:47 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:54:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:47 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:54:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:47 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:47.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:47 smithi149 ceph-mon[35408]: pgmap v669: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:54:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:47 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:47 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:47 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:47 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:54:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:47 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T07:54:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:47 smithi149 ceph-mon[35408]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T07:54:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:47 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:54:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:47 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:54:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:47 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:54:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:49.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:49 smithi086 ceph-mon[34687]: pgmap v670: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:54:49.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:49 smithi149 ceph-mon[35408]: pgmap v670: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:54:49.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:49 smithi063 ceph-mon[31613]: pgmap v670: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:54:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.169+0000 7f9aa5000700 1 -- 172.21.15.63:0/2968725956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9aa010b8b0 msgr2=0x7f9aa0113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.169+0000 7f9aa5000700 1 --2- 172.21.15.63:0/2968725956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9aa010b8b0 0x7f9aa0113210 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9a980099c0 tx=0x7f9a98009cd0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.169+0000 7f9aa5000700 1 -- 172.21.15.63:0/2968725956 shutdown_connections 2024-05-27T07:54:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.169+0000 7f9aa5000700 1 --2- 172.21.15.63:0/2968725956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9aa010b8b0 0x7f9aa0113210 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.169+0000 7f9aa5000700 1 --2- 172.21.15.63:0/2968725956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa0106630 0x7f9aa0106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.169+0000 7f9aa5000700 1 --2- 172.21.15.63:0/2968725956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa010af60 0x7f9aa010b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.169+0000 7f9aa5000700 1 -- 172.21.15.63:0/2968725956 >> 172.21.15.63:0/2968725956 conn(0x7f9aa0101ec0 msgr2=0x7f9aa0104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:50.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.169+0000 7f9aa5000700 1 -- 172.21.15.63:0/2968725956 shutdown_connections 2024-05-27T07:54:50.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.170+0000 7f9aa5000700 1 -- 172.21.15.63:0/2968725956 wait complete. 2024-05-27T07:54:50.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.171+0000 7f9aa5000700 1 Processor -- start 2024-05-27T07:54:50.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.171+0000 7f9aa5000700 1 -- start start 2024-05-27T07:54:50.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.172+0000 7f9aa5000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa0106630 0x7f9aa01a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.172+0000 7f9aa5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9aa010af60 0x7f9aa01a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.172+0000 7f9aa5000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa010b8b0 0x7f9aa019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.172+0000 7f9aa5000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa0116390 con 0x7f9aa010af60 2024-05-27T07:54:50.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.172+0000 7f9aa5000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa0116210 con 0x7f9aa0106630 2024-05-27T07:54:50.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.172+0000 7f9aa5000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa0116510 con 0x7f9aa010b8b0 2024-05-27T07:54:50.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.173+0000 7f9a97e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9aa010af60 0x7f9aa01a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.173+0000 7f9a97e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9aa010af60 0x7f9aa01a2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40058/0 (socket says 172.21.15.63:40058) 2024-05-27T07:54:50.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.173+0000 7f9a97e00700 1 -- 172.21.15.63:0/2058376188 learned_addr learned my addr 172.21.15.63:0/2058376188 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:50.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.173+0000 7f9a9fe00700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa010b8b0 0x7f9aa019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.173+0000 7f9a9f400700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa0106630 0x7f9aa01a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.173+0000 7f9a97e00700 1 -- 172.21.15.63:0/2058376188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa010b8b0 msgr2=0x7f9aa019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:50.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.173+0000 7f9a97e00700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa010b8b0 0x7f9aa019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.173+0000 7f9a97e00700 1 -- 172.21.15.63:0/2058376188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa0106630 msgr2=0x7f9aa01a2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:50.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.173+0000 7f9a97e00700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa0106630 0x7f9aa01a2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.174+0000 7f9a97e00700 1 -- 172.21.15.63:0/2058376188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa019d1b0 con 0x7f9aa010af60 2024-05-27T07:54:50.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.174+0000 7f9a97e00700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9aa010af60 0x7f9aa01a2df0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800d980 tx=0x7f9a8800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:50.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.174+0000 7f9a9cc00700 1 -- 172.21.15.63:0/2058376188 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88017070 con 0x7f9aa010af60 2024-05-27T07:54:50.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.174+0000 7f9a9cc00700 1 -- 172.21.15.63:0/2058376188 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a8800fd20 con 0x7f9aa010af60 2024-05-27T07:54:50.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.174+0000 7f9aa5000700 1 -- 172.21.15.63:0/2058376188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa019d4a0 con 0x7f9aa010af60 2024-05-27T07:54:50.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.174+0000 7f9a9cc00700 1 -- 172.21.15.63:0/2058376188 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800e3b0 con 0x7f9aa010af60 2024-05-27T07:54:50.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.175+0000 7f9aa5000700 1 -- 172.21.15.63:0/2058376188 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa01a9740 con 0x7f9aa010af60 2024-05-27T07:54:50.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.180+0000 7f9a9cc00700 1 -- 172.21.15.63:0/2058376188 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9a8800e550 con 0x7f9aa010af60 2024-05-27T07:54:50.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.181+0000 7f9ab5212b80 1 Processor -- start 2024-05-27T07:54:50.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.181+0000 7f9ab5212b80 1 -- start start 2024-05-27T07:54:50.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.181+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x557012610ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.181+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 0x55701261bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.181+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.181+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55701253fe40 con 0x557012611820 2024-05-27T07:54:50.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.181+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55701252f9e0 con 0x557012611410 2024-05-27T07:54:50.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.181+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557012533a20 con 0x557012492e40 2024-05-27T07:54:50.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.182+0000 7f9a97e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.182+0000 7f9a9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 0x55701261bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.182+0000 7f9a97e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40068/0 (socket says 172.21.15.63:40068) 2024-05-27T07:54:50.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.182+0000 7f9a97e00700 1 -- 172.21.15.63:0/3802389012 learned_addr learned my addr 172.21.15.63:0/3802389012 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:50.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.182+0000 7f9a9fe00700 1 --2- 172.21.15.63:0/3802389012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x557012610ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.182+0000 7f9a9cc00700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a840b29f0 0x7f9a840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.183+0000 7f9a9fe00700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a840b29f0 0x7f9a840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.183+0000 7f9a9cc00700 1 -- 172.21.15.63:0/2058376188 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f9a880e2450 con 0x7f9aa010af60 2024-05-27T07:54:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.183+0000 7f9a9f400700 1 -- 172.21.15.63:0/3802389012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 msgr2=0x557012610ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.183+0000 7f9a9f400700 1 --2- 172.21.15.63:0/3802389012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x557012610ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.183+0000 7f9a9f400700 1 -- 172.21.15.63:0/3802389012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 msgr2=0x55701261e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.183+0000 7f9a9f400700 1 --2- 172.21.15.63:0/3802389012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.183+0000 7f9a9f400700 1 -- 172.21.15.63:0/3802389012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570124f8420 con 0x557012611410 2024-05-27T07:54:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.183+0000 7f9a97e00700 1 --2- 172.21.15.63:0/3802389012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.183+0000 7f9a9f400700 1 --2- 172.21.15.63:0/3802389012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 0x55701261bad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9a90003f70 tx=0x7f9a90009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec48775f621f1ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.184+0000 7f9a9fe00700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a840b29f0 0x7f9a840b4f00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009f50 tx=0x7f9a98009cd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:50.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.184+0000 7f9a94c00700 1 -- 172.21.15.63:0/3802389012 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90004610 con 0x557012611410 2024-05-27T07:54:50.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.184+0000 7f9a94c00700 1 -- 172.21.15.63:0/3802389012 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a9001a070 con 0x557012611410 2024-05-27T07:54:50.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.184+0000 7f9a94c00700 1 -- 172.21.15.63:0/3802389012 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a900156e0 con 0x557012611410 2024-05-27T07:54:50.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.185+0000 7f9ab5212b80 1 -- 172.21.15.63:0/3802389012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 msgr2=0x55701261bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:50.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.185+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/3802389012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 0x55701261bad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9a90003f70 tx=0x7f9a90009ba0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.185+0000 7f9ab5212b80 1 -- 172.21.15.63:0/3802389012 shutdown_connections 2024-05-27T07:54:50.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.185+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/3802389012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.185+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/3802389012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x557012610ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.185+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/3802389012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 0x55701261bad0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.185+0000 7f9ab5212b80 1 -- 172.21.15.63:0/3802389012 >> 172.21.15.63:0/3802389012 conn(0x5570124f2fc0 msgr2=0x55701260cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:50.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.186+0000 7f9ab5212b80 1 -- 172.21.15.63:0/3802389012 shutdown_connections 2024-05-27T07:54:50.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.186+0000 7f9ab5212b80 1 -- 172.21.15.63:0/3802389012 wait complete. 2024-05-27T07:54:50.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.186+0000 7f9ab5212b80 1 -- start start 2024-05-27T07:54:50.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.187+0000 7f9ab5212b80 1 Processor -- start 2024-05-27T07:54:50.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.187+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x55701261b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.187+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 0x55701261bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.187+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.187+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557012435fa0 con 0x557012611820 2024-05-27T07:54:50.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.187+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5570124d0760 con 0x557012611410 2024-05-27T07:54:50.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.187+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55701261e990 con 0x557012492e40 2024-05-27T07:54:50.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.187+0000 7f9ab5212b80 10 client.0 _ll_register_callbacks cb 0x5570124d9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:50.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.188+0000 7f9a9fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.188+0000 7f9a9fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40074/0 (socket says 172.21.15.63:40074) 2024-05-27T07:54:50.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.188+0000 7f9a9fe00700 1 -- 172.21.15.63:0/2173046061 learned_addr learned my addr 172.21.15.63:0/2173046061 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:50.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.188+0000 7f9a97e00700 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 0x55701261bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.188+0000 7f9a9f400700 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x55701261b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.188+0000 7f9a9fe00700 1 -- 172.21.15.63:0/2173046061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 msgr2=0x55701261b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:50.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.188+0000 7f9a9fe00700 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x55701261b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.188+0000 7f9a9fe00700 1 -- 172.21.15.63:0/2173046061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 msgr2=0x55701261bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:50.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.188+0000 7f9a9fe00700 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 0x55701261bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.189+0000 7f9a9fe00700 1 -- 172.21.15.63:0/2173046061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55701247f8d0 con 0x557012611820 2024-05-27T07:54:50.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.189+0000 7f9a9fe00700 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261c000 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f9a98015000 tx=0x7f9a98005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a06f10633586c24 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.189+0000 7f9a8d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.189+0000 7f9a8d600700 1 -- 172.21.15.63:0/2173046061 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9800f460 con 0x557012611820 2024-05-27T07:54:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.189+0000 7f9a8d600700 1 -- 172.21.15.63:0/2173046061 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f9a9800f600 con 0x557012611820 2024-05-27T07:54:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.189+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2173046061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a9001a070 con 0x557012611820 2024-05-27T07:54:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.190+0000 7f9a8d600700 1 -- 172.21.15.63:0/2173046061 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9a9803b050 con 0x557012611820 2024-05-27T07:54:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.190+0000 7f9ab5212b80 10 client.19488 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:50.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.190+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2173046061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557012500060 con 0x557012611820 2024-05-27T07:54:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.190+0000 7f9a8d600700 1 -- 172.21.15.63:0/2173046061 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a9001a070 con 0x557012611820 2024-05-27T07:54:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.190+0000 7f9a8d600700 1 -- 172.21.15.63:0/2173046061 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9a980295d0 con 0x557012611820 2024-05-27T07:54:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.190+0000 7f9ab5212b80 10 client.19488 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.190+0000 7f9ab5212b80 10 client.19488 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:54:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.191+0000 7f9ab5212b80 20 client.19488 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.191+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55701262b7d0 0x55701262dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.191+0000 7f9ab5212b80 4 client.19488 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.191+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2173046061 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557012541d90 con 0x55701262b7d0 2024-05-27T07:54:50.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.191+0000 7f9a9f400700 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55701262b7d0 0x55701262dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.192+0000 7f9a9f400700 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55701262b7d0 0x55701262dc60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8676168f4bb007cd server_cookie=9b7b69319883697c in_seq=0 out_seq=0 2024-05-27T07:54:50.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.192+0000 7f9a8d600700 10 client.19488 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:54:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9a8d600700 1 -- 172.21.15.63:0/2173046061 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557012541d90 con 0x55701262b7d0 2024-05-27T07:54:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9a8d600700 10 client.19488 handle_command_reply: tid=0 2024-05-27T07:54:50.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9ab5212b80 1 client.19488 shutdown 2024-05-27T07:54:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2173046061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 msgr2=0x55701261c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261c000 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f9a98015000 tx=0x7f9a98005fb0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2173046061 shutdown_connections 2024-05-27T07:54:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55701262b7d0 0x55701262dc60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611820 0x55701261c000 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x55701261b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2173046061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611410 0x55701261bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2173046061 >> 172.21.15.63:0/2173046061 conn(0x5570124f2fc0 msgr2=0x55701260bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.195+0000 7f9a8d600700 0 client.19488 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:54:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.196+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2173046061 shutdown_connections 2024-05-27T07:54:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.196+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2173046061 wait complete. 2024-05-27T07:54:50.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.196+0000 7f9ab5212b80 20 client.19488 trim_cache size 0 max 16384 2024-05-27T07:54:50.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.221+0000 7f9ab5212b80 1 Processor -- start 2024-05-27T07:54:50.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.221+0000 7f9ab5212b80 1 -- start start 2024-05-27T07:54:50.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.221+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x55701261a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.221+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611410 0x55701261da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.221+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611820 0x557012627a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.221+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01b4dd0 con 0x557012611410 2024-05-27T07:54:50.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.221+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01b2660 con 0x557012611820 2024-05-27T07:54:50.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.221+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01b04d0 con 0x557012492e40 2024-05-27T07:54:50.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.222+0000 7f9a97e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611410 0x55701261da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.222+0000 7f9a97e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611410 0x55701261da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40076/0 (socket says 172.21.15.63:40076) 2024-05-27T07:54:50.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.222+0000 7f9a97e00700 1 -- 172.21.15.63:0/1572973784 learned_addr learned my addr 172.21.15.63:0/1572973784 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:50.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.222+0000 7f9a9f400700 1 --2- 172.21.15.63:0/1572973784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x55701261a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.222+0000 7f9a9fe00700 1 --2- 172.21.15.63:0/1572973784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611820 0x557012627a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.222+0000 7f9a97e00700 1 -- 172.21.15.63:0/1572973784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 msgr2=0x55701261a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:50.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.222+0000 7f9a97e00700 1 --2- 172.21.15.63:0/1572973784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x55701261a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.223+0000 7f9a97e00700 1 -- 172.21.15.63:0/1572973784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611820 msgr2=0x557012627a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:50.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.223+0000 7f9a97e00700 1 --2- 172.21.15.63:0/1572973784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611820 0x557012627a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.223+0000 7f9a97e00700 1 -- 172.21.15.63:0/1572973784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557012619a60 con 0x557012611410 2024-05-27T07:54:50.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.223+0000 7f9a97e00700 1 --2- 172.21.15.63:0/1572973784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611410 0x55701261da20 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9a880e26f0 tx=0x7f9a880a9230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7518b7258070fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:50.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.223+0000 7f9a8fe00700 1 -- 172.21.15.63:0/1572973784 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88020040 con 0x557012611410 2024-05-27T07:54:50.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.223+0000 7f9a8fe00700 1 -- 172.21.15.63:0/1572973784 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a880a8f70 con 0x557012611410 2024-05-27T07:54:50.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.223+0000 7f9a8fe00700 1 -- 172.21.15.63:0/1572973784 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a880b1ce0 con 0x557012611410 2024-05-27T07:54:50.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.224+0000 7f9ab5212b80 1 -- 172.21.15.63:0/1572973784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611410 msgr2=0x55701261da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:50.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.224+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/1572973784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611410 0x55701261da20 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9a880e26f0 tx=0x7f9a880a9230 comp rx=0 tx=0).stop 2024-05-27T07:54:50.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.224+0000 7f9ab5212b80 1 -- 172.21.15.63:0/1572973784 shutdown_connections 2024-05-27T07:54:50.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.224+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/1572973784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611820 0x557012627a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.224+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/1572973784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012492e40 0x55701261a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.225+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/1572973784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012611410 0x55701261da20 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.225+0000 7f9ab5212b80 1 -- 172.21.15.63:0/1572973784 >> 172.21.15.63:0/1572973784 conn(0x5570124f2fc0 msgr2=0x55701261b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:50.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.225+0000 7f9ab5212b80 1 -- 172.21.15.63:0/1572973784 shutdown_connections 2024-05-27T07:54:50.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.225+0000 7f9ab5212b80 1 -- 172.21.15.63:0/1572973784 wait complete. 2024-05-27T07:54:50.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.225+0000 7f9ab5212b80 1 -- start start 2024-05-27T07:54:50.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.226+0000 7f9ab5212b80 1 Processor -- start 2024-05-27T07:54:50.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.226+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012492e40 0x557012624280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.226+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012611410 0x55701261f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.226+0000 7f9ab5212b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611820 0x55701261fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.226+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01b0b20 con 0x557012492e40 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.226+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01b1f90 con 0x557012611820 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.226+0000 7f9ab5212b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01b11f0 con 0x557012611410 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.226+0000 7f9ab5212b80 10 client.0 _ll_register_callbacks cb 0x5570124d9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.227+0000 7f9a9f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012492e40 0x557012624280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.227+0000 7f9a9f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012492e40 0x557012624280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40086/0 (socket says 172.21.15.63:40086) 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.227+0000 7f9a9f400700 1 -- 172.21.15.63:0/2331381252 learned_addr learned my addr 172.21.15.63:0/2331381252 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.227+0000 7f9a9fe00700 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611820 0x55701261fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.227+0000 7f9a97e00700 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012611410 0x55701261f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.227+0000 7f9a9f400700 1 -- 172.21.15.63:0/2331381252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012611410 msgr2=0x55701261f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.227+0000 7f9a9f400700 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012611410 0x55701261f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.227+0000 7f9a9f400700 1 -- 172.21.15.63:0/2331381252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611820 msgr2=0x55701261fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.227+0000 7f9a9f400700 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611820 0x55701261fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.227+0000 7f9a9f400700 1 -- 172.21.15.63:0/2331381252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557012620200 con 0x557012492e40 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.228+0000 7f9a9f400700 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012492e40 0x557012624280 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009ef0 tx=0x7f9a90004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622c54350fe92344 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.228+0000 7f9a8d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.228+0000 7f9a8d600700 1 -- 172.21.15.63:0/2331381252 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90004610 con 0x557012492e40 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.228+0000 7f9a8d600700 1 -- 172.21.15.63:0/2331381252 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f9a90004c00 con 0x557012492e40 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.228+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2331381252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a880a8f70 con 0x557012492e40 2024-05-27T07:54:50.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9a8d600700 1 -- 172.21.15.63:0/2331381252 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9a90004e00 con 0x557012492e40 2024-05-27T07:54:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9ab5212b80 10 client.19494 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2331381252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5570126961b0 con 0x557012492e40 2024-05-27T07:54:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9a8d600700 1 -- 172.21.15.63:0/2331381252 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a880a8f70 con 0x557012492e40 2024-05-27T07:54:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9a8d600700 1 -- 172.21.15.63:0/2331381252 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9a90028650 con 0x557012492e40 2024-05-27T07:54:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9ab5212b80 10 client.19494 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9ab5212b80 10 client.19494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:54:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9ab5212b80 20 client.19494 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5570126965e0 0x557012698a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9ab5212b80 4 client.19494 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:54:50.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.229+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2331381252 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55701253b570 con 0x5570126965e0 2024-05-27T07:54:50.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.230+0000 7f9a97e00700 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5570126965e0 0x557012698a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:50.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.231+0000 7f9a97e00700 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5570126965e0 0x557012698a00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a520d8d48a76217f server_cookie=58f41fb3a2cba7e0 in_seq=0 out_seq=0 2024-05-27T07:54:50.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.231+0000 7f9a8d600700 10 client.19494 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:54:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9a8d600700 1 -- 172.21.15.63:0/2331381252 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55701253b570 con 0x5570126965e0 2024-05-27T07:54:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9a8d600700 10 client.19494 handle_command_reply: tid=0 2024-05-27T07:54:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9ab5212b80 1 client.19494 shutdown 2024-05-27T07:54:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2331381252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012492e40 msgr2=0x557012624280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012492e40 0x557012624280 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009ef0 tx=0x7f9a90004410 comp rx=0 tx=0).stop 2024-05-27T07:54:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2331381252 shutdown_connections 2024-05-27T07:54:50.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557012611820 0x55701261fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5570126965e0 0x557012698a00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557012492e40 0x557012624280 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9ab5212b80 1 --2- 172.21.15.63:0/2331381252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557012611410 0x55701261f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9a8d600700 0 client.19494 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:54:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.232+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2331381252 >> 172.21.15.63:0/2331381252 conn(0x5570124f2fc0 msgr2=0x55701261bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.233+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2331381252 shutdown_connections 2024-05-27T07:54:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.233+0000 7f9ab5212b80 1 -- 172.21.15.63:0/2331381252 wait complete. 2024-05-27T07:54:50.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.233+0000 7f9ab5212b80 20 client.19494 trim_cache size 0 max 16384 2024-05-27T07:54:50.236 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:54:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 -- 172.21.15.63:0/2058376188 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a840b29f0 msgr2=0x7f9a840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a840b29f0 0x7f9a840b4f00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009f50 tx=0x7f9a98009cd0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 -- 172.21.15.63:0/2058376188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9aa010af60 msgr2=0x7f9aa01a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9aa010af60 0x7f9aa01a2df0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800d980 tx=0x7f9a8800dc90 comp rx=0 tx=0).stop 2024-05-27T07:54:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 -- 172.21.15.63:0/2058376188 shutdown_connections 2024-05-27T07:54:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9a840b29f0 0x7f9a840b4f00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa010b8b0 0x7f9aa019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa0106630 0x7f9aa01a2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 --2- 172.21.15.63:0/2058376188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9aa010af60 0x7f9aa01a2df0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 -- 172.21.15.63:0/2058376188 >> 172.21.15.63:0/2058376188 conn(0x7f9aa0101ec0 msgr2=0x7f9aa010a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 -- 172.21.15.63:0/2058376188 shutdown_connections 2024-05-27T07:54:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:50.234+0000 7f9aa5000700 1 -- 172.21.15.63:0/2058376188 wait complete. 2024-05-27T07:54:51.326 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:54:51.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:51 smithi086 ceph-mon[34687]: pgmap v671: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 10 op/s 2024-05-27T07:54:51.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:51 smithi149 ceph-mon[35408]: pgmap v671: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 10 op/s 2024-05-27T07:54:51.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:51 smithi063 ceph-mon[31613]: pgmap v671: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 10 op/s 2024-05-27T07:54:51.872 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.675+0000 7f7ca9400640 1 -- 172.21.15.63:0/242149701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca4108e30 msgr2=0x7f7ca4103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.675+0000 7f7ca9400640 1 --2- 172.21.15.63:0/242149701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca4108e30 0x7f7ca4103100 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009900 tx=0x7f7c9002a410 comp rx=0 tx=0).stop 2024-05-27T07:54:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.675+0000 7f7ca9400640 1 -- 172.21.15.63:0/242149701 shutdown_connections 2024-05-27T07:54:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.675+0000 7f7ca9400640 1 --2- 172.21.15.63:0/242149701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4103790 0x7f7ca410f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.675+0000 7f7ca9400640 1 --2- 172.21.15.63:0/242149701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca4108e30 0x7f7ca4103100 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.675+0000 7f7ca9400640 1 --2- 172.21.15.63:0/242149701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca4108990 0x7f7ca4102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.675+0000 7f7ca9400640 1 -- 172.21.15.63:0/242149701 >> 172.21.15.63:0/242149701 conn(0x7f7ca40fd7e0 msgr2=0x7f7ca40ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.675+0000 7f7ca9400640 1 -- 172.21.15.63:0/242149701 shutdown_connections 2024-05-27T07:54:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.675+0000 7f7ca9400640 1 -- 172.21.15.63:0/242149701 wait complete. 2024-05-27T07:54:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.676+0000 7f7ca9400640 1 Processor -- start 2024-05-27T07:54:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.676+0000 7f7ca9400640 1 -- start start 2024-05-27T07:54:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.677+0000 7f7ca9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4108990 0x7f7ca41a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.677+0000 7f7ca9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca4108e30 0x7f7ca41a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.677+0000 7f7ca9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca410c200 0x7f7ca410c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.677+0000 7f7ca9400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca4113c10 con 0x7f7ca410c200 2024-05-27T07:54:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.677+0000 7f7ca9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca4113a90 con 0x7f7ca4108e30 2024-05-27T07:54:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.677+0000 7f7ca9400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca4113d90 con 0x7f7ca4108990 2024-05-27T07:54:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.677+0000 7f7ca8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca410c200 0x7f7ca410c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.677+0000 7f7ca8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca410c200 0x7f7ca410c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40098/0 (socket says 172.21.15.63:40098) 2024-05-27T07:54:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.677+0000 7f7ca8a00640 1 -- 172.21.15.63:0/2716698781 learned_addr learned my addr 172.21.15.63:0/2716698781 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.677+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4108990 0x7f7ca41a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.678+0000 7f7ca3400640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca4108e30 0x7f7ca41a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.678+0000 7f7ca8a00640 1 -- 172.21.15.63:0/2716698781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4108990 msgr2=0x7f7ca41a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.678+0000 7f7ca8a00640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4108990 0x7f7ca41a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.678+0000 7f7ca8a00640 1 -- 172.21.15.63:0/2716698781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca4108e30 msgr2=0x7f7ca41a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.678+0000 7f7ca8a00640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca4108e30 0x7f7ca41a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.678+0000 7f7ca8a00640 1 -- 172.21.15.63:0/2716698781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca410ccb0 con 0x7f7ca410c200 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.678+0000 7f7ca3400640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca4108e30 0x7f7ca41a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.678+0000 7f7ca8a00640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca410c200 0x7f7ca410c610 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00eaa0 tx=0x7f7c9c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.679+0000 7f7ca0c00640 1 -- 172.21.15.63:0/2716698781 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c004d20 con 0x7f7ca410c200 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.679+0000 7f7ca0c00640 1 -- 172.21.15.63:0/2716698781 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c9c005020 con 0x7f7ca410c200 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.679+0000 7f7ca9400640 1 -- 172.21.15.63:0/2716698781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca410cfa0 con 0x7f7ca410c200 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.679+0000 7f7ca0c00640 1 -- 172.21.15.63:0/2716698781 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0116d0 con 0x7f7ca410c200 2024-05-27T07:54:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.679+0000 7f7ca9400640 1 -- 172.21.15.63:0/2716698781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca41a91f0 con 0x7f7ca410c200 2024-05-27T07:54:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.683+0000 7f7cb19b3740 1 Processor -- start 2024-05-27T07:54:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.684+0000 7f7ca0c00640 1 -- 172.21.15.63:0/2716698781 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7c9c0051d0 con 0x7f7ca410c200 2024-05-27T07:54:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.684+0000 7f7cb19b3740 1 -- start start 2024-05-27T07:54:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.684+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591951a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.684+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915862f0 0x555591951fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.684+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559158e4d0 0x5555919524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.684+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5555918c2b90 con 0x5555915af1b0 2024-05-27T07:54:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.684+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5555918b2730 con 0x5555915862f0 2024-05-27T07:54:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.684+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5555918b6770 con 0x55559158e4d0 2024-05-27T07:54:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.685+0000 7f7ca3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591951a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.685+0000 7f7ca3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591951a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40100/0 (socket says 172.21.15.63:40100) 2024-05-27T07:54:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.685+0000 7f7ca3400640 1 -- 172.21.15.63:0/356515799 learned_addr learned my addr 172.21.15.63:0/356515799 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.685+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/356515799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915862f0 0x555591951fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.685+0000 7f7ca3400640 1 --2- 172.21.15.63:0/356515799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559158e4d0 0x5555919524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.686+0000 7f7ca3400640 1 -- 172.21.15.63:0/356515799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559158e4d0 msgr2=0x5555919524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.686+0000 7f7ca3400640 1 --2- 172.21.15.63:0/356515799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559158e4d0 0x5555919524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.686+0000 7f7ca3400640 1 -- 172.21.15.63:0/356515799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915862f0 msgr2=0x555591951fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.686+0000 7f7ca3400640 1 --2- 172.21.15.63:0/356515799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915862f0 0x555591951fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.686+0000 7f7ca3400640 1 -- 172.21.15.63:0/356515799 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55559184b8b0 con 0x5555915af1b0 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.686+0000 7f7ca0c00640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c880b2950 0x7f7c880b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.686+0000 7f7ca0c00640 1 -- 172.21.15.63:0/2716698781 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f7c9c0e2c00 con 0x7f7ca410c200 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.686+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c880b2950 0x7f7c880b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.686+0000 7f7ca3400640 1 --2- 172.21.15.63:0/356515799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591951a60 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f7ca406bf40 tx=0x7f7c90005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a468bb111fa26e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.687+0000 7f7c99200640 1 -- 172.21.15.63:0/356515799 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90043020 con 0x5555915af1b0 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.687+0000 7f7c99200640 1 -- 172.21.15.63:0/356515799 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c900044c0 con 0x5555915af1b0 2024-05-27T07:54:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.687+0000 7f7c99200640 1 -- 172.21.15.63:0/356515799 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90035070 con 0x5555915af1b0 2024-05-27T07:54:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.688+0000 7f7cb19b3740 1 -- 172.21.15.63:0/356515799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 msgr2=0x555591951a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.688+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/356515799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591951a60 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f7ca406bf40 tx=0x7f7c90005d50 comp rx=0 tx=0).stop 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.688+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c880b2950 0x7f7c880b4e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x555591952e10 tx=0x7f7c9400a2d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.688+0000 7f7cb19b3740 1 -- 172.21.15.63:0/356515799 shutdown_connections 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.688+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/356515799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559158e4d0 0x5555919524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.688+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/356515799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915862f0 0x555591951fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.688+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/356515799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591951a60 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.688+0000 7f7cb19b3740 1 -- 172.21.15.63:0/356515799 >> 172.21.15.63:0/356515799 conn(0x5555915a3300 msgr2=0x55559194e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.688+0000 7f7cb19b3740 1 -- 172.21.15.63:0/356515799 shutdown_connections 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.688+0000 7f7cb19b3740 1 -- 172.21.15.63:0/356515799 wait complete. 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.689+0000 7f7cb19b3740 1 -- start start 2024-05-27T07:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.689+0000 7f7cb19b3740 1 Processor -- start 2024-05-27T07:54:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x55559195bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55559158e4d0 0x55559195c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915af1b0 0x555591967ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555591598790 con 0x5555915862f0 2024-05-27T07:54:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55559185abc0 con 0x55559158e4d0 2024-05-27T07:54:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55559195eb30 con 0x5555915af1b0 2024-05-27T07:54:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7cb19b3740 10 client.0 _ll_register_callbacks cb 0x555591838400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x55559195bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x55559195bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40114/0 (socket says 172.21.15.63:40114) 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7ca3e00640 1 -- 172.21.15.63:0/795786159 learned_addr learned my addr 172.21.15.63:0/795786159 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.690+0000 7f7ca3400640 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55559158e4d0 0x55559195c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.691+0000 7f7ca8a00640 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915af1b0 0x555591967ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.691+0000 7f7ca3e00640 1 -- 172.21.15.63:0/795786159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915af1b0 msgr2=0x555591967ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.691+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915af1b0 0x555591967ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.691+0000 7f7ca3e00640 1 -- 172.21.15.63:0/795786159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55559158e4d0 msgr2=0x55559195c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.691+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55559158e4d0 0x55559195c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.691+0000 7f7ca3e00640 1 -- 172.21.15.63:0/795786159 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555915c7210 con 0x5555915862f0 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.691+0000 7f7ca8a00640 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915af1b0 0x555591967ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.691+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x55559195bed0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7c94011010 tx=0x7f7c94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4556482cb1e0f015 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.691+0000 7f7c8d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.692+0000 7f7c8d600640 1 -- 172.21.15.63:0/795786159 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9402c060 con 0x5555915862f0 2024-05-27T07:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.692+0000 7f7c8d600640 1 -- 172.21.15.63:0/795786159 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f7c94016030 con 0x5555915862f0 2024-05-27T07:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.692+0000 7f7cb19b3740 1 -- 172.21.15.63:0/795786159 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5555918c4ae0 con 0x5555915862f0 2024-05-27T07:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.692+0000 7f7c8d600640 1 -- 172.21.15.63:0/795786159 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7c94033050 con 0x5555915862f0 2024-05-27T07:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.692+0000 7f7cb19b3740 10 client.19506 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.692+0000 7f7cb19b3740 1 -- 172.21.15.63:0/795786159 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555591693620 con 0x5555915862f0 2024-05-27T07:54:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.693+0000 7f7c8d600640 1 -- 172.21.15.63:0/795786159 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5555918c4ae0 con 0x5555915862f0 2024-05-27T07:54:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.693+0000 7f7c8d600640 1 -- 172.21.15.63:0/795786159 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7c9401a3a0 con 0x5555915862f0 2024-05-27T07:54:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.693+0000 7f7cb19b3740 10 client.19506 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.693+0000 7f7cb19b3740 10 client.19506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.693+0000 7f7cb19b3740 20 client.19506 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.693+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55559156dd80 0x55559196df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.693+0000 7f7cb19b3740 4 client.19506 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.693+0000 7f7cb19b3740 1 -- 172.21.15.63:0/795786159 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555918be2c0 con 0x55559156dd80 2024-05-27T07:54:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.694+0000 7f7ca3400640 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55559156dd80 0x55559196df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.695+0000 7f7c8d600640 10 client.19506 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.695+0000 7f7ca3400640 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55559156dd80 0x55559196df80 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53d56d3672c6ccc server_cookie=1a6b372e803b1bb1 in_seq=0 out_seq=0 2024-05-27T07:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7c8d600640 1 -- 172.21.15.63:0/795786159 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5555918be2c0 con 0x55559156dd80 2024-05-27T07:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7c8d600640 10 client.19506 handle_command_reply: tid=0 2024-05-27T07:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7cb19b3740 1 client.19506 shutdown 2024-05-27T07:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7cb19b3740 1 -- 172.21.15.63:0/795786159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 msgr2=0x55559195bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x55559195bed0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7c94011010 tx=0x7f7c94004730 comp rx=0 tx=0).stop 2024-05-27T07:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7cb19b3740 1 -- 172.21.15.63:0/795786159 shutdown_connections 2024-05-27T07:54:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55559156dd80 0x55559196df80 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915af1b0 0x555591967ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55559158e4d0 0x55559195c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/795786159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x55559195bed0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7c8d600640 0 client.19506 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7cb19b3740 1 -- 172.21.15.63:0/795786159 >> 172.21.15.63:0/795786159 conn(0x5555915a3300 msgr2=0x555591951830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.697+0000 7f7cb19b3740 1 -- 172.21.15.63:0/795786159 shutdown_connections 2024-05-27T07:54:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.698+0000 7f7cb19b3740 1 -- 172.21.15.63:0/795786159 wait complete. 2024-05-27T07:54:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.698+0000 7f7cb19b3740 20 client.19506 trim_cache size 0 max 16384 2024-05-27T07:54:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.719+0000 7f7cb19b3740 1 Processor -- start 2024-05-27T07:54:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.720+0000 7f7cb19b3740 1 -- start start 2024-05-27T07:54:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.720+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x555591967c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.720+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559158e4d0 0x555591968170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.720+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915af1b0 0x555591973760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.720+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41b5ac0 con 0x5555915862f0 2024-05-27T07:54:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.720+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41b3390 con 0x5555915af1b0 2024-05-27T07:54:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.720+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41b1200 con 0x55559158e4d0 2024-05-27T07:54:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.720+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x555591967c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.720+0000 7f7ca3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x555591967c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40118/0 (socket says 172.21.15.63:40118) 2024-05-27T07:54:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.720+0000 7f7ca3e00640 1 -- 172.21.15.63:0/2773510209 learned_addr learned my addr 172.21.15.63:0/2773510209 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.721+0000 7f7ca3400640 1 --2- 172.21.15.63:0/2773510209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559158e4d0 0x555591968170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.721+0000 7f7ca8a00640 1 --2- 172.21.15.63:0/2773510209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915af1b0 0x555591973760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.721+0000 7f7ca3e00640 1 -- 172.21.15.63:0/2773510209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559158e4d0 msgr2=0x555591968170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.721+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/2773510209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559158e4d0 0x555591968170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.721+0000 7f7ca3e00640 1 -- 172.21.15.63:0/2773510209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915af1b0 msgr2=0x555591973760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.721+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/2773510209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915af1b0 0x555591973760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.721+0000 7f7ca3e00640 1 -- 172.21.15.63:0/2773510209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555591952e40 con 0x5555915862f0 2024-05-27T07:54:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.721+0000 7f7ca8a00640 1 --2- 172.21.15.63:0/2773510209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915af1b0 0x555591973760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.721+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/2773510209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x555591967c30 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7c94017040 tx=0x7f7c94005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cfcae3cee6178a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.722+0000 7f7c8fe00640 1 -- 172.21.15.63:0/2773510209 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9403b070 con 0x5555915862f0 2024-05-27T07:54:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.722+0000 7f7c8fe00640 1 -- 172.21.15.63:0/2773510209 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c9401a3a0 con 0x5555915862f0 2024-05-27T07:54:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.722+0000 7f7c8fe00640 1 -- 172.21.15.63:0/2773510209 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94027b10 con 0x5555915862f0 2024-05-27T07:54:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.723+0000 7f7cb19b3740 1 -- 172.21.15.63:0/2773510209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 msgr2=0x555591967c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.723+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/2773510209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x555591967c30 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7c94017040 tx=0x7f7c94005b10 comp rx=0 tx=0).stop 2024-05-27T07:54:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.723+0000 7f7cb19b3740 1 -- 172.21.15.63:0/2773510209 shutdown_connections 2024-05-27T07:54:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.723+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/2773510209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555915af1b0 0x555591973760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.723+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/2773510209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559158e4d0 0x555591968170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.723+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/2773510209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915862f0 0x555591967c30 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.723+0000 7f7cb19b3740 1 -- 172.21.15.63:0/2773510209 >> 172.21.15.63:0/2773510209 conn(0x5555915a3300 msgr2=0x55559195de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.723+0000 7f7cb19b3740 1 -- 172.21.15.63:0/2773510209 shutdown_connections 2024-05-27T07:54:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.723+0000 7f7cb19b3740 1 -- 172.21.15.63:0/2773510209 wait complete. 2024-05-27T07:54:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.724+0000 7f7cb19b3740 1 -- start start 2024-05-27T07:54:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.724+0000 7f7cb19b3740 1 Processor -- start 2024-05-27T07:54:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915862f0 0x555591960bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55559158e4d0 0x555591961110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7cb19b3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591961650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41b1850 con 0x5555915af1b0 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41b2cc0 con 0x55559158e4d0 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7cb19b3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41b1f20 con 0x5555915862f0 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7cb19b3740 10 client.0 _ll_register_callbacks cb 0x7f7c9400a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7ca8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591961650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7ca8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591961650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40122/0 (socket says 172.21.15.63:40122) 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7ca8a00640 1 -- 172.21.15.63:0/1655710766 learned_addr learned my addr 172.21.15.63:0/1655710766 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915862f0 0x555591960bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.725+0000 7f7ca3400640 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55559158e4d0 0x555591961110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7ca8a00640 1 -- 172.21.15.63:0/1655710766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915862f0 msgr2=0x555591960bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7ca8a00640 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915862f0 0x555591960bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7ca8a00640 1 -- 172.21.15.63:0/1655710766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55559158e4d0 msgr2=0x555591961110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7ca8a00640 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55559158e4d0 0x555591961110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7ca8a00640 1 -- 172.21.15.63:0/1655710766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555919d4630 con 0x5555915af1b0 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915862f0 0x555591960bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7ca8a00640 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591961650 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0a5b40 tx=0x7f7c9c0a5340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5c367ac08a18f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7c8d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7c8d600640 1 -- 172.21.15.63:0/1655710766 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0c3060 con 0x5555915af1b0 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7c8d600640 1 -- 172.21.15.63:0/1655710766 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f7c9c0a5d50 con 0x5555915af1b0 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.726+0000 7f7cb19b3740 1 -- 172.21.15.63:0/1655710766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c9401a3a0 con 0x5555915af1b0 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7c8d600640 1 -- 172.21.15.63:0/1655710766 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7c9c01f040 con 0x5555915af1b0 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7cb19b3740 10 client.19512 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7cb19b3740 1 -- 172.21.15.63:0/1655710766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5555919d58d0 con 0x5555915af1b0 2024-05-27T07:54:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7c8d600640 1 -- 172.21.15.63:0/1655710766 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c9401a3a0 con 0x5555915af1b0 2024-05-27T07:54:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7c8d600640 1 -- 172.21.15.63:0/1655710766 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7c9c00f040 con 0x5555915af1b0 2024-05-27T07:54:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7cb19b3740 10 client.19512 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7cb19b3740 10 client.19512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7cb19b3740 20 client.19512 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5555915431c0 0x5555919d8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7cb19b3740 4 client.19512 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:54:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.727+0000 7f7cb19b3740 1 -- 172.21.15.63:0/1655710766 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555918384b0 con 0x5555915431c0 2024-05-27T07:54:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.728+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5555915431c0 0x5555919d8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.729+0000 7f7ca3e00640 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5555915431c0 0x5555919d8030 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2cbcb83f825af4 server_cookie=68f6d717276ca58f in_seq=0 out_seq=0 2024-05-27T07:54:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.729+0000 7f7c8d600640 10 client.19512 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.764+0000 7f7c8d600640 1 -- 172.21.15.63:0/1655710766 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5555918384b0 con 0x5555915431c0 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.764+0000 7f7c8d600640 10 client.19512 handle_command_reply: tid=0 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.764+0000 7f7cb19b3740 1 client.19512 shutdown 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 1 -- 172.21.15.63:0/1655710766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 msgr2=0x555591961650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591961650 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0a5b40 tx=0x7f7c9c0a5340 comp rx=0 tx=0).stop 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 1 -- 172.21.15.63:0/1655710766 shutdown_connections 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5555915431c0 0x5555919d8030 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5555915af1b0 0x555591961650 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55559158e4d0 0x555591961110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7c8d600640 0 client.19512 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 1 --2- 172.21.15.63:0/1655710766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5555915862f0 0x555591960bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 1 -- 172.21.15.63:0/1655710766 >> 172.21.15.63:0/1655710766 conn(0x5555915a3300 msgr2=0x55559195b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 1 -- 172.21.15.63:0/1655710766 shutdown_connections 2024-05-27T07:54:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 1 -- 172.21.15.63:0/1655710766 wait complete. 2024-05-27T07:54:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.765+0000 7f7cb19b3740 20 client.19512 trim_cache size 0 max 16384 2024-05-27T07:54:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:54:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:54:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:54:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.766+0000 7f7ca9400640 1 -- 172.21.15.63:0/2716698781 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c880b2950 msgr2=0x7f7c880b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.766+0000 7f7ca9400640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c880b2950 0x7f7c880b4e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x555591952e10 tx=0x7f7c9400a2d0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.766+0000 7f7ca9400640 1 -- 172.21.15.63:0/2716698781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca410c200 msgr2=0x7f7ca410c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.766+0000 7f7ca9400640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca410c200 0x7f7ca410c610 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00eaa0 tx=0x7f7c9c00eea0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.766+0000 7f7ca9400640 1 -- 172.21.15.63:0/2716698781 shutdown_connections 2024-05-27T07:54:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.766+0000 7f7ca9400640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c880b2950 0x7f7c880b4e60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.766+0000 7f7ca9400640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ca410c200 0x7f7ca410c610 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.766+0000 7f7ca9400640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ca4108e30 0x7f7ca41a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.766+0000 7f7ca9400640 1 --2- 172.21.15.63:0/2716698781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca4108990 0x7f7ca41a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.766+0000 7f7ca9400640 1 -- 172.21.15.63:0/2716698781 >> 172.21.15.63:0/2716698781 conn(0x7f7ca40fd7e0 msgr2=0x7f7ca4113170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.767+0000 7f7ca9400640 1 -- 172.21.15.63:0/2716698781 shutdown_connections 2024-05-27T07:54:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:54:52.767+0000 7f7ca9400640 1 -- 172.21.15.63:0/2716698781 wait complete. 2024-05-27T07:54:53.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:53 smithi086 ceph-mon[34687]: pgmap v672: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.0 MiB/s wr, 14 op/s 2024-05-27T07:54:53.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:53 smithi149 ceph-mon[35408]: pgmap v672: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.0 MiB/s wr, 14 op/s 2024-05-27T07:54:53.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:53 smithi063 ceph-mon[31613]: pgmap v672: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.0 MiB/s wr, 14 op/s 2024-05-27T07:54:55.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.248+0000 7f24af600700 1 -- 172.21.15.63:0/3481540468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24a810e8e0 msgr2=0x7f24a8110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:55.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.248+0000 7f24af600700 1 --2- 172.21.15.63:0/3481540468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24a810e8e0 0x7f24a8110dc0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f24a000b0a0 tx=0x7f24a000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.249+0000 7f24af600700 1 -- 172.21.15.63:0/3481540468 shutdown_connections 2024-05-27T07:54:55.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.249+0000 7f24af600700 1 --2- 172.21.15.63:0/3481540468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24a810e8e0 0x7f24a8110dc0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.249+0000 7f24af600700 1 --2- 172.21.15.63:0/3481540468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24a81054b0 0x7f24a81058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.249+0000 7f24af600700 1 --2- 172.21.15.63:0/3481540468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a8105e00 0x7f24a810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.249+0000 7f24af600700 1 -- 172.21.15.63:0/3481540468 >> 172.21.15.63:0/3481540468 conn(0x7f24a80fedb0 msgr2=0x7f24a81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:55.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.249+0000 7f24af600700 1 -- 172.21.15.63:0/3481540468 shutdown_connections 2024-05-27T07:54:55.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.249+0000 7f24af600700 1 -- 172.21.15.63:0/3481540468 wait complete. 2024-05-27T07:54:55.251 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:55 smithi063 ceph-mon[31613]: pgmap v673: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T07:54:55.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.250+0000 7f24af600700 1 Processor -- start 2024-05-27T07:54:55.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.251+0000 7f24af600700 1 -- start start 2024-05-27T07:54:55.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.251+0000 7f24af600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24a81054b0 0x7f24a819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.251+0000 7f24af600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24a8105e00 0x7f24a819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.251+0000 7f24af600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a810e8e0 0x7f24a81a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.251+0000 7f24af600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8114340 con 0x7f24a81054b0 2024-05-27T07:54:55.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.251+0000 7f24af600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81141c0 con 0x7f24a810e8e0 2024-05-27T07:54:55.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.251+0000 7f24af600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81144c0 con 0x7f24a8105e00 2024-05-27T07:54:55.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.252+0000 7f24acc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24a81054b0 0x7f24a819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.252+0000 7f24acc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24a81054b0 0x7f24a819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40146/0 (socket says 172.21.15.63:40146) 2024-05-27T07:54:55.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.252+0000 7f24acc00700 1 -- 172.21.15.63:0/2027142512 learned_addr learned my addr 172.21.15.63:0/2027142512 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:55.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.252+0000 7f24ad600700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a810e8e0 0x7f24a81a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.252+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24a8105e00 0x7f24a819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.252+0000 7f24acc00700 1 -- 172.21.15.63:0/2027142512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24a8105e00 msgr2=0x7f24a819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:55.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.252+0000 7f24acc00700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24a8105e00 0x7f24a819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.252+0000 7f24acc00700 1 -- 172.21.15.63:0/2027142512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a810e8e0 msgr2=0x7f24a81a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:55.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.252+0000 7f24acc00700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a810e8e0 0x7f24a81a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.253+0000 7f24acc00700 1 -- 172.21.15.63:0/2027142512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a81a1dc0 con 0x7f24a81054b0 2024-05-27T07:54:55.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.253+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24a8105e00 0x7f24a819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:55.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.253+0000 7f24acc00700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24a81054b0 0x7f24a819cef0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f249c00d950 tx=0x7f249c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:55.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.253+0000 7f24a5600700 1 -- 172.21.15.63:0/2027142512 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c016070 con 0x7f24a81054b0 2024-05-27T07:54:55.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.254+0000 7f24af600700 1 -- 172.21.15.63:0/2027142512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a81a20b0 con 0x7f24a81054b0 2024-05-27T07:54:55.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.254+0000 7f24a5600700 1 -- 172.21.15.63:0/2027142512 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f249c010460 con 0x7f24a81054b0 2024-05-27T07:54:55.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.254+0000 7f24a5600700 1 -- 172.21.15.63:0/2027142512 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c00f620 con 0x7f24a81054b0 2024-05-27T07:54:55.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.254+0000 7f24af600700 1 -- 172.21.15.63:0/2027142512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a81a99d0 con 0x7f24a81054b0 2024-05-27T07:54:55.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.259+0000 7f24a5600700 1 -- 172.21.15.63:0/2027142512 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f249c019070 con 0x7f24a81054b0 2024-05-27T07:54:55.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.260+0000 7f24a5600700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24900b2a70 0x7f24900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.260+0000 7f24be22bb80 1 Processor -- start 2024-05-27T07:54:55.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.260+0000 7f24a5600700 1 -- 172.21.15.63:0/2027142512 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f249c0e2580 con 0x7f24a81054b0 2024-05-27T07:54:55.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.261+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24900b2a70 0x7f24900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.261+0000 7f24be22bb80 1 -- start start 2024-05-27T07:54:55.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.261+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126a5be40 0x560126be8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.261+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.261+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126be39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.261+0000 7f24be22bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560126b08e40 con 0x560126be94e0 2024-05-27T07:54:55.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.261+0000 7f24be22bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560126af89e0 con 0x560126be35c0 2024-05-27T07:54:55.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.261+0000 7f24be22bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560126afca20 con 0x560126a5be40 2024-05-27T07:54:55.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.262+0000 7f24a7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.262+0000 7f24ad600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126be39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.262+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24900b2a70 0x7f24900b4f80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2494005e10 tx=0x7f2494005d60 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:55.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.262+0000 7f24acc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126a5be40 0x560126be8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.262+0000 7f24a7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40152/0 (socket says 172.21.15.63:40152) 2024-05-27T07:54:55.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.262+0000 7f24a7e00700 1 -- 172.21.15.63:0/2882038511 learned_addr learned my addr 172.21.15.63:0/2882038511 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:55.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.262+0000 7f24a7e00700 1 -- 172.21.15.63:0/2882038511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126a5be40 msgr2=0x560126be8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:55.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.262+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2882038511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126a5be40 0x560126be8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.262+0000 7f24a7e00700 1 -- 172.21.15.63:0/2882038511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 msgr2=0x560126be39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:55.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.262+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2882038511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126be39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.263+0000 7f24a7e00700 1 -- 172.21.15.63:0/2882038511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560126ac1420 con 0x560126be94e0 2024-05-27T07:54:55.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.263+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2882038511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be3080 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f249400fd50 tx=0x7f2494012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0de90d42475af9b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:55.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.263+0000 7f249a000700 1 -- 172.21.15.63:0/2882038511 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494022070 con 0x560126be94e0 2024-05-27T07:54:55.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.263+0000 7f249a000700 1 -- 172.21.15.63:0/2882038511 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2494011bb0 con 0x560126be94e0 2024-05-27T07:54:55.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.263+0000 7f249a000700 1 -- 172.21.15.63:0/2882038511 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24940103f0 con 0x560126be94e0 2024-05-27T07:54:55.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.264+0000 7f24be22bb80 1 -- 172.21.15.63:0/2882038511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 msgr2=0x560126be3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:55.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.264+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2882038511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be3080 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f249400fd50 tx=0x7f2494012000 comp rx=0 tx=0).stop 2024-05-27T07:54:55.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.265+0000 7f24be22bb80 1 -- 172.21.15.63:0/2882038511 shutdown_connections 2024-05-27T07:54:55.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.265+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2882038511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126be39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.265+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2882038511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126a5be40 0x560126be8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.265+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2882038511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be3080 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.265+0000 7f24be22bb80 1 -- 172.21.15.63:0/2882038511 >> 172.21.15.63:0/2882038511 conn(0x560126abbfc0 msgr2=0x560126bd58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:55.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.265+0000 7f24be22bb80 1 -- 172.21.15.63:0/2882038511 shutdown_connections 2024-05-27T07:54:55.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.265+0000 7f24be22bb80 1 -- 172.21.15.63:0/2882038511 wait complete. 2024-05-27T07:54:55.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.266+0000 7f24be22bb80 1 -- start start 2024-05-27T07:54:55.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.266+0000 7f24be22bb80 1 Processor -- start 2024-05-27T07:54:55.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.267+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126a5be40 0x560126becd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.267+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126be35c0 0x560126bed270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.267+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126bed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.267+0000 7f24be22bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601269fefa0 con 0x560126be94e0 2024-05-27T07:54:55.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.267+0000 7f24be22bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560126a99760 con 0x560126a5be40 2024-05-27T07:54:55.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.267+0000 7f24be22bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560126beffc0 con 0x560126be35c0 2024-05-27T07:54:55.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.267+0000 7f24be22bb80 10 client.0 _ll_register_callbacks cb 0x560126aa2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:55.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.267+0000 7f24ad600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126bed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.267+0000 7f24ad600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126bed7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40168/0 (socket says 172.21.15.63:40168) 2024-05-27T07:54:55.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.267+0000 7f24ad600700 1 -- 172.21.15.63:0/2419345621 learned_addr learned my addr 172.21.15.63:0/2419345621 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:55.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.268+0000 7f24acc00700 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126a5be40 0x560126becd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.268+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126be35c0 0x560126bed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.268+0000 7f24ad600700 1 -- 172.21.15.63:0/2419345621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126be35c0 msgr2=0x560126bed270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:55.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.268+0000 7f24ad600700 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126be35c0 0x560126bed270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.268+0000 7f24ad600700 1 -- 172.21.15.63:0/2419345621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126a5be40 msgr2=0x560126becd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:55.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.268+0000 7f24ad600700 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126a5be40 0x560126becd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.268+0000 7f24ad600700 1 -- 172.21.15.63:0/2419345621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560126a488d0 con 0x560126be94e0 2024-05-27T07:54:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.268+0000 7f24ad600700 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126bed7b0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f24a00095f0 tx=0x7f24a0004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db97ac8817252ab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.268+0000 7f248ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.269+0000 7f248ea00700 1 -- 172.21.15.63:0/2419345621 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a00077c0 con 0x560126be94e0 2024-05-27T07:54:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.269+0000 7f248ea00700 1 -- 172.21.15.63:0/2419345621 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f24a0007960 con 0x560126be94e0 2024-05-27T07:54:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.269+0000 7f24be22bb80 1 -- 172.21.15.63:0/2419345621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2494011bb0 con 0x560126be94e0 2024-05-27T07:54:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.269+0000 7f248ea00700 1 -- 172.21.15.63:0/2419345621 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f24a0007b60 con 0x560126be94e0 2024-05-27T07:54:55.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.269+0000 7f24be22bb80 10 client.19524 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:55.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.269+0000 7f24be22bb80 1 -- 172.21.15.63:0/2419345621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560126ac9060 con 0x560126be94e0 2024-05-27T07:54:55.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.270+0000 7f248ea00700 1 -- 172.21.15.63:0/2419345621 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2494011bb0 con 0x560126be94e0 2024-05-27T07:54:55.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.270+0000 7f248ea00700 1 -- 172.21.15.63:0/2419345621 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f24a0011070 con 0x560126be94e0 2024-05-27T07:54:55.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.270+0000 7f24be22bb80 10 client.19524 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:55.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.270+0000 7f24be22bb80 10 client.19524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:54:55.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.270+0000 7f24be22bb80 20 client.19524 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:55.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.270+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560126bfcfa0 0x560126bff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.270+0000 7f24be22bb80 4 client.19524 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:55.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.270+0000 7f24be22bb80 1 -- 172.21.15.63:0/2419345621 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560126b0ad90 con 0x560126bfcfa0 2024-05-27T07:54:55.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.271+0000 7f24acc00700 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560126bfcfa0 0x560126bff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.272+0000 7f24acc00700 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560126bfcfa0 0x560126bff400 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ecbed22214c4940 server_cookie=53ef6afd46f60279 in_seq=0 out_seq=0 2024-05-27T07:54:55.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.272+0000 7f248ea00700 10 client.19524 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:54:55.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.274+0000 7f248ea00700 1 -- 172.21.15.63:0/2419345621 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560126b0ad90 con 0x560126bfcfa0 2024-05-27T07:54:55.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.274+0000 7f248ea00700 10 client.19524 handle_command_reply: tid=0 2024-05-27T07:54:55.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.274+0000 7f24be22bb80 1 client.19524 shutdown 2024-05-27T07:54:55.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f24be22bb80 1 -- 172.21.15.63:0/2419345621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 msgr2=0x560126bed7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:55.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126bed7b0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f24a00095f0 tx=0x7f24a0004a50 comp rx=0 tx=0).stop 2024-05-27T07:54:55.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f24be22bb80 1 -- 172.21.15.63:0/2419345621 shutdown_connections 2024-05-27T07:54:55.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560126bfcfa0 0x560126bff400 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126bed7b0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f248ea00700 0 client.19524 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:54:55.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126a5be40 0x560126becd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2419345621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126be35c0 0x560126bed270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f24be22bb80 1 -- 172.21.15.63:0/2419345621 >> 172.21.15.63:0/2419345621 conn(0x560126abbfc0 msgr2=0x560126bd5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:55.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f24be22bb80 1 -- 172.21.15.63:0/2419345621 shutdown_connections 2024-05-27T07:54:55.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.275+0000 7f24be22bb80 1 -- 172.21.15.63:0/2419345621 wait complete. 2024-05-27T07:54:55.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.276+0000 7f24be22bb80 20 client.19524 trim_cache size 0 max 16384 2024-05-27T07:54:55.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.296+0000 7f24be22bb80 1 Processor -- start 2024-05-27T07:54:55.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.296+0000 7f24be22bb80 1 -- start start 2024-05-27T07:54:55.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.296+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126a5be40 0x560126bda290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.296+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126bf9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.297+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.297+0000 7f24be22bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81b0b50 con 0x560126be94e0 2024-05-27T07:54:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.297+0000 7f24be22bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81ae3e0 con 0x560126be35c0 2024-05-27T07:54:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.297+0000 7f24be22bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81ac2b0 con 0x560126a5be40 2024-05-27T07:54:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.297+0000 7f24ad600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.297+0000 7f24ad600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40172/0 (socket says 172.21.15.63:40172) 2024-05-27T07:54:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.297+0000 7f24ad600700 1 -- 172.21.15.63:0/311791446 learned_addr learned my addr 172.21.15.63:0/311791446 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.297+0000 7f24a7e00700 1 --2- 172.21.15.63:0/311791446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126bf9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.297+0000 7f24acc00700 1 --2- 172.21.15.63:0/311791446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126a5be40 0x560126bda290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.298+0000 7f24ad600700 1 -- 172.21.15.63:0/311791446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126a5be40 msgr2=0x560126bda290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:55.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.298+0000 7f24ad600700 1 --2- 172.21.15.63:0/311791446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126a5be40 0x560126bda290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.298+0000 7f24ad600700 1 -- 172.21.15.63:0/311791446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 msgr2=0x560126bf9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:55.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.298+0000 7f24ad600700 1 --2- 172.21.15.63:0/311791446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126bf9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.298+0000 7f24ad600700 1 -- 172.21.15.63:0/311791446 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560126bef5d0 con 0x560126be94e0 2024-05-27T07:54:55.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.298+0000 7f24ad600700 1 --2- 172.21.15.63:0/311791446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be7070 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009e60 tx=0x7f24a000b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd53dbb943b7bfc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:55.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.298+0000 7f2499600700 1 -- 172.21.15.63:0/311791446 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0007dc0 con 0x560126be94e0 2024-05-27T07:54:55.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.299+0000 7f2499600700 1 -- 172.21.15.63:0/311791446 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24a0005c60 con 0x560126be94e0 2024-05-27T07:54:55.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.299+0000 7f2499600700 1 -- 172.21.15.63:0/311791446 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0005730 con 0x560126be94e0 2024-05-27T07:54:55.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.300+0000 7f24be22bb80 1 -- 172.21.15.63:0/311791446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 msgr2=0x560126be7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:55.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.300+0000 7f24be22bb80 1 --2- 172.21.15.63:0/311791446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be7070 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009e60 tx=0x7f24a000b060 comp rx=0 tx=0).stop 2024-05-27T07:54:55.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.300+0000 7f24be22bb80 1 -- 172.21.15.63:0/311791446 shutdown_connections 2024-05-27T07:54:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.300+0000 7f24be22bb80 1 --2- 172.21.15.63:0/311791446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126be94e0 0x560126be7070 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.300+0000 7f24be22bb80 1 --2- 172.21.15.63:0/311791446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126a5be40 0x560126bda290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.300+0000 7f24be22bb80 1 --2- 172.21.15.63:0/311791446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126bf9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.300+0000 7f24be22bb80 1 -- 172.21.15.63:0/311791446 >> 172.21.15.63:0/311791446 conn(0x560126abbfc0 msgr2=0x560126be6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.300+0000 7f24be22bb80 1 -- 172.21.15.63:0/311791446 shutdown_connections 2024-05-27T07:54:55.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.300+0000 7f24be22bb80 1 -- 172.21.15.63:0/311791446 wait complete. 2024-05-27T07:54:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.301+0000 7f24be22bb80 1 -- start start 2024-05-27T07:54:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.301+0000 7f24be22bb80 1 Processor -- start 2024-05-27T07:54:55.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.301+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126a5be40 0x560126bf5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.301+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126c04190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.301+0000 7f24be22bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126be94e0 0x560126c046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.302+0000 7f24be22bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81ac900 con 0x560126a5be40 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.302+0000 7f24be22bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81add10 con 0x560126be35c0 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.302+0000 7f24be22bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81acf70 con 0x560126be94e0 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.302+0000 7f24be22bb80 10 client.0 _ll_register_callbacks cb 0x560126aa2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.302+0000 7f24acc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126a5be40 0x560126bf5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.302+0000 7f24acc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126a5be40 0x560126bf5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40176/0 (socket says 172.21.15.63:40176) 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.302+0000 7f24acc00700 1 -- 172.21.15.63:0/2602830519 learned_addr learned my addr 172.21.15.63:0/2602830519 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.302+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126c04190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.302+0000 7f24ad600700 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126be94e0 0x560126c046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.303+0000 7f24acc00700 1 -- 172.21.15.63:0/2602830519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126be94e0 msgr2=0x560126c046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:55.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.303+0000 7f24acc00700 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126be94e0 0x560126c046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.303+0000 7f24acc00700 1 -- 172.21.15.63:0/2602830519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 msgr2=0x560126c04190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.303+0000 7f24acc00700 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126c04190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.303+0000 7f24acc00700 1 -- 172.21.15.63:0/2602830519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560126c04c10 con 0x560126a5be40 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.303+0000 7f24acc00700 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126a5be40 0x560126bf5660 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f249c0df9c0 tx=0x7f249c0a8c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369b63fb4d155483 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.303+0000 7f248ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.303+0000 7f248ea00700 1 -- 172.21.15.63:0/2602830519 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c020040 con 0x560126a5be40 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.303+0000 7f248ea00700 1 -- 172.21.15.63:0/2602830519 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f249c0a9700 con 0x560126a5be40 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.303+0000 7f24be22bb80 1 -- 172.21.15.63:0/2602830519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a0005c60 con 0x560126a5be40 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f248ea00700 1 -- 172.21.15.63:0/2602830519 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f249c0b2300 con 0x560126a5be40 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f24be22bb80 10 client.19530 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:55.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f24be22bb80 1 -- 172.21.15.63:0/2602830519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560126c67710 con 0x560126a5be40 2024-05-27T07:54:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f248ea00700 1 -- 172.21.15.63:0/2602830519 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24a0005c60 con 0x560126a5be40 2024-05-27T07:54:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f248ea00700 1 -- 172.21.15.63:0/2602830519 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f249c0b1e70 con 0x560126a5be40 2024-05-27T07:54:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f24be22bb80 10 client.19530 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:55.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f24be22bb80 10 client.19530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:54:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f24be22bb80 20 client.19530 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560126c67d70 0x560126c6a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f24be22bb80 4 client.19530 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:54:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.304+0000 7f24be22bb80 1 -- 172.21.15.63:0/2602830519 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560126b04570 con 0x560126c67d70 2024-05-27T07:54:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.305+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560126c67d70 0x560126c6a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.306+0000 7f24a7e00700 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560126c67d70 0x560126c6a190 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad97b6a25052b1ba server_cookie=cab0450d4a4cfd27 in_seq=0 out_seq=0 2024-05-27T07:54:55.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.306+0000 7f248ea00700 10 client.19530 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:54:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f248ea00700 1 -- 172.21.15.63:0/2602830519 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560126b04570 con 0x560126c67d70 2024-05-27T07:54:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f248ea00700 10 client.19530 handle_command_reply: tid=0 2024-05-27T07:54:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 client.19530 shutdown 2024-05-27T07:54:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 -- 172.21.15.63:0/2602830519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126a5be40 msgr2=0x560126bf5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:55.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126a5be40 0x560126bf5660 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f249c0df9c0 tx=0x7f249c0a8c10 comp rx=0 tx=0).stop 2024-05-27T07:54:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 -- 172.21.15.63:0/2602830519 shutdown_connections 2024-05-27T07:54:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560126be94e0 0x560126c046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x560126c67d70 0x560126c6a190 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560126a5be40 0x560126bf5660 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 --2- 172.21.15.63:0/2602830519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560126be35c0 0x560126c04190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f248ea00700 0 client.19530 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:54:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 -- 172.21.15.63:0/2602830519 >> 172.21.15.63:0/2602830519 conn(0x560126abbfc0 msgr2=0x560126bd9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 -- 172.21.15.63:0/2602830519 shutdown_connections 2024-05-27T07:54:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.307+0000 7f24be22bb80 1 -- 172.21.15.63:0/2602830519 wait complete. 2024-05-27T07:54:55.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.308+0000 7f24be22bb80 20 client.19530 trim_cache size 0 max 16384 2024-05-27T07:54:55.311 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:54:55.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.308+0000 7f24af600700 1 -- 172.21.15.63:0/2027142512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24900b2a70 msgr2=0x7f24900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:55.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.308+0000 7f24af600700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24900b2a70 0x7f24900b4f80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2494005e10 tx=0x7f2494005d60 comp rx=0 tx=0).stop 2024-05-27T07:54:55.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.308+0000 7f24af600700 1 -- 172.21.15.63:0/2027142512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24a81054b0 msgr2=0x7f24a819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:55.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.308+0000 7f24af600700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24a81054b0 0x7f24a819cef0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f249c00d950 tx=0x7f249c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:54:55.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.309+0000 7f24af600700 1 -- 172.21.15.63:0/2027142512 shutdown_connections 2024-05-27T07:54:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.309+0000 7f24af600700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24900b2a70 0x7f24900b4f80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.309+0000 7f24af600700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a810e8e0 0x7f24a81a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.309+0000 7f24af600700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24a81054b0 0x7f24a819cef0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.309+0000 7f24af600700 1 --2- 172.21.15.63:0/2027142512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24a8105e00 0x7f24a819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.309+0000 7f24af600700 1 -- 172.21.15.63:0/2027142512 >> 172.21.15.63:0/2027142512 conn(0x7f24a80fedb0 msgr2=0x7f24a8103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.309+0000 7f24af600700 1 -- 172.21.15.63:0/2027142512 shutdown_connections 2024-05-27T07:54:55.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:55.309+0000 7f24af600700 1 -- 172.21.15.63:0/2027142512 wait complete. 2024-05-27T07:54:55.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:55 smithi086 ceph-mon[34687]: pgmap v673: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T07:54:55.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:55 smithi149 ceph-mon[35408]: pgmap v673: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T07:54:56.243 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:54:56.732 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:57 smithi086 ceph-mon[34687]: pgmap v674: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T07:54:57.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:57 smithi149 ceph-mon[35408]: pgmap v674: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T07:54:57.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:57 smithi063 ceph-mon[31613]: pgmap v674: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T07:54:59.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.186+0000 7f7cf7000700 1 -- 172.21.15.63:0/258402668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf01054d0 msgr2=0x7f7cf01058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.186+0000 7f7cf7000700 1 --2- 172.21.15.63:0/258402668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf01054d0 0x7f7cf01058e0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009990 tx=0x7f7ce0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.186+0000 7f7cf7000700 1 -- 172.21.15.63:0/258402668 shutdown_connections 2024-05-27T07:54:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.186+0000 7f7cf7000700 1 --2- 172.21.15.63:0/258402668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cf010e900 0x7f7cf0110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.186+0000 7f7cf7000700 1 --2- 172.21.15.63:0/258402668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf01054d0 0x7f7cf01058e0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.186+0000 7f7cf7000700 1 --2- 172.21.15.63:0/258402668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf0105e20 0x7f7cf010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.186+0000 7f7cf7000700 1 -- 172.21.15.63:0/258402668 >> 172.21.15.63:0/258402668 conn(0x7f7cf00fef20 msgr2=0x7f7cf0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.186+0000 7f7cf7000700 1 -- 172.21.15.63:0/258402668 shutdown_connections 2024-05-27T07:54:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.186+0000 7f7cf7000700 1 -- 172.21.15.63:0/258402668 wait complete. 2024-05-27T07:54:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.187+0000 7f7cf7000700 1 Processor -- start 2024-05-27T07:54:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.188+0000 7f7cf7000700 1 -- start start 2024-05-27T07:54:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.188+0000 7f7cf7000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf01054d0 0x7f7cf019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.188+0000 7f7cf7000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf0105e20 0x7f7cf019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.188+0000 7f7cf7000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cf010e900 0x7f7cf01a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.188+0000 7f7cf7000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf0114300 con 0x7f7cf0105e20 2024-05-27T07:54:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.188+0000 7f7cf7000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf0114180 con 0x7f7cf01054d0 2024-05-27T07:54:59.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.188+0000 7f7cf7000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf0114480 con 0x7f7cf010e900 2024-05-27T07:54:59.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.189+0000 7f7cef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf0105e20 0x7f7cf019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.189+0000 7f7cef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf0105e20 0x7f7cf019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51440/0 (socket says 172.21.15.63:51440) 2024-05-27T07:54:59.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.189+0000 7f7cef400700 1 -- 172.21.15.63:0/3075560779 learned_addr learned my addr 172.21.15.63:0/3075560779 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:59.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.189+0000 7f7cefe00700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf01054d0 0x7f7cf019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.189+0000 7f7cf5000700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cf010e900 0x7f7cf01a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.189+0000 7f7cef400700 1 -- 172.21.15.63:0/3075560779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cf010e900 msgr2=0x7f7cf01a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:59.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.189+0000 7f7cef400700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cf010e900 0x7f7cf01a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.189+0000 7f7cef400700 1 -- 172.21.15.63:0/3075560779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf01054d0 msgr2=0x7f7cf019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:59.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.189+0000 7f7cef400700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf01054d0 0x7f7cf019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.189+0000 7f7cef400700 1 -- 172.21.15.63:0/3075560779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf01a1d50 con 0x7f7cf0105e20 2024-05-27T07:54:59.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.190+0000 7f7cef400700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf0105e20 0x7f7cf019d4d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c9b0 tx=0x7f7cdc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.191+0000 7f7cecc00700 1 -- 172.21.15.63:0/3075560779 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc007b00 con 0x7f7cf0105e20 2024-05-27T07:54:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.191+0000 7f7cf7000700 1 -- 172.21.15.63:0/3075560779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf01a2040 con 0x7f7cf0105e20 2024-05-27T07:54:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.191+0000 7f7cecc00700 1 -- 172.21.15.63:0/3075560779 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cdc007ca0 con 0x7f7cf0105e20 2024-05-27T07:54:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.191+0000 7f7cecc00700 1 -- 172.21.15.63:0/3075560779 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc018660 con 0x7f7cf0105e20 2024-05-27T07:54:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.191+0000 7f7cf7000700 1 -- 172.21.15.63:0/3075560779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf01a9a10 con 0x7f7cf0105e20 2024-05-27T07:54:59.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.197+0000 7f7cecc00700 1 -- 172.21.15.63:0/3075560779 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7cdc0188c0 con 0x7f7cf0105e20 2024-05-27T07:54:59.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.197+0000 7f7d05c3db80 1 Processor -- start 2024-05-27T07:54:59.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.197+0000 7f7d05c3db80 1 -- start start 2024-05-27T07:54:59.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 0x56373bdd3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bc54e40 0x56373bdd3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bdd3e70 0x56373bde0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56373bd01e40 con 0x56373bdd3e70 2024-05-27T07:54:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56373bcf19e0 con 0x56373bd409f0 2024-05-27T07:54:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56373bcf5a20 con 0x56373bc54e40 2024-05-27T07:54:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7cef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bdd3e70 0x56373bde0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7cef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bdd3e70 0x56373bde0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51454/0 (socket says 172.21.15.63:51454) 2024-05-27T07:54:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7cef400700 1 -- 172.21.15.63:0/1721488109 learned_addr learned my addr 172.21.15.63:0/1721488109 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7cefe00700 1 --2- 172.21.15.63:0/1721488109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bc54e40 0x56373bdd3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.198+0000 7f7cf5000700 1 --2- 172.21.15.63:0/1721488109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 0x56373bdd3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.199+0000 7f7cecc00700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7cd40b2a70 0x7f7cd40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.199+0000 7f7cef400700 1 -- 172.21.15.63:0/1721488109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bc54e40 msgr2=0x56373bdd3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.199+0000 7f7cef400700 1 --2- 172.21.15.63:0/1721488109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bc54e40 0x56373bdd3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.199+0000 7f7cef400700 1 -- 172.21.15.63:0/1721488109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 msgr2=0x56373bdd3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.199+0000 7f7cef400700 1 --2- 172.21.15.63:0/1721488109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 0x56373bdd3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.199+0000 7f7cef400700 1 -- 172.21.15.63:0/1721488109 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373bcba420 con 0x56373bdd3e70 2024-05-27T07:54:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.199+0000 7f7cf5000700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7cd40b2a70 0x7f7cd40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.199+0000 7f7cecc00700 1 -- 172.21.15.63:0/3075560779 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f7cdc0e2930 con 0x7f7cf0105e20 2024-05-27T07:54:59.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.200+0000 7f7cef400700 1 --2- 172.21.15.63:0/1721488109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bdd3e70 0x56373bde0110 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0dfba0 tx=0x7f7cdc06ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f352cbdbb45912e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:59.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.200+0000 7f7ce5600700 1 -- 172.21.15.63:0/1721488109 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc0a9140 con 0x56373bdd3e70 2024-05-27T07:54:59.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.200+0000 7f7ce5600700 1 -- 172.21.15.63:0/1721488109 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cdc007e50 con 0x56373bdd3e70 2024-05-27T07:54:59.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.200+0000 7f7ce5600700 1 -- 172.21.15.63:0/1721488109 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc089350 con 0x56373bdd3e70 2024-05-27T07:54:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.201+0000 7f7d05c3db80 1 -- 172.21.15.63:0/1721488109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bdd3e70 msgr2=0x56373bde0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.201+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/1721488109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bdd3e70 0x56373bde0110 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0dfba0 tx=0x7f7cdc06ece0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.201+0000 7f7cf5000700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7cd40b2a70 0x7f7cd40b4f80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x56373bdd5ff0 tx=0x7f7ce8004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.201+0000 7f7d05c3db80 1 -- 172.21.15.63:0/1721488109 shutdown_connections 2024-05-27T07:54:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.201+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/1721488109 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bdd3e70 0x56373bde0110 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.201+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/1721488109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 0x56373bdd3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.201+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/1721488109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bc54e40 0x56373bdd3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.201+0000 7f7d05c3db80 1 -- 172.21.15.63:0/1721488109 >> 172.21.15.63:0/1721488109 conn(0x56373bcee390 msgr2=0x56373bdcf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.202+0000 7f7d05c3db80 1 -- 172.21.15.63:0/1721488109 shutdown_connections 2024-05-27T07:54:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.202+0000 7f7d05c3db80 1 -- 172.21.15.63:0/1721488109 wait complete. 2024-05-27T07:54:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.202+0000 7f7d05c3db80 1 -- start start 2024-05-27T07:54:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.203+0000 7f7d05c3db80 1 Processor -- start 2024-05-27T07:54:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.203+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bc54e40 0x56373bddd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.203+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bdddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.203+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bdd3e70 0x56373bdde050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.203+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56373bbf7fa0 con 0x56373bd409f0 2024-05-27T07:54:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.203+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56373bc92760 con 0x56373bdd3e70 2024-05-27T07:54:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.203+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56373bde0b20 con 0x56373bc54e40 2024-05-27T07:54:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.203+0000 7f7d05c3db80 10 client.0 _ll_register_callbacks cb 0x56373bc9bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.204+0000 7f7cef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bdddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.204+0000 7f7cef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bdddb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51468/0 (socket says 172.21.15.63:51468) 2024-05-27T07:54:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.204+0000 7f7cef400700 1 -- 172.21.15.63:0/3698464090 learned_addr learned my addr 172.21.15.63:0/3698464090 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.204+0000 7f7cf5000700 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bdd3e70 0x56373bdde050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.204+0000 7f7cefe00700 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bc54e40 0x56373bddd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.204+0000 7f7cef400700 1 -- 172.21.15.63:0/3698464090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bc54e40 msgr2=0x56373bddd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.204+0000 7f7cef400700 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bc54e40 0x56373bddd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.204+0000 7f7cef400700 1 -- 172.21.15.63:0/3698464090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bdd3e70 msgr2=0x56373bdde050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.204+0000 7f7cef400700 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bdd3e70 0x56373bdde050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.204+0000 7f7cef400700 1 -- 172.21.15.63:0/3698464090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373bc418d0 con 0x56373bd409f0 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.205+0000 7f7cef400700 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bdddb10 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00cfd0 tx=0x7f7cdc0a9300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aff30ff43d900d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.205+0000 7f7cda000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.205+0000 7f7cda000700 1 -- 172.21.15.63:0/3698464090 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc089530 con 0x56373bd409f0 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.205+0000 7f7cda000700 1 -- 172.21.15.63:0/3698464090 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f7cdc0dfee0 con 0x56373bd409f0 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.205+0000 7f7d05c3db80 1 -- 172.21.15.63:0/3698464090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cdc007e50 con 0x56373bd409f0 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7cda000700 1 -- 172.21.15.63:0/3698464090 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7cdc0a8f60 con 0x56373bd409f0 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7d05c3db80 10 client.19542 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7d05c3db80 1 -- 172.21.15.63:0/3698464090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56373bcc2060 con 0x56373bd409f0 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7cda000700 1 -- 172.21.15.63:0/3698464090 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cdc007e50 con 0x56373bd409f0 2024-05-27T07:54:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7cda000700 1 -- 172.21.15.63:0/3698464090 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7cdc0b21c0 con 0x56373bd409f0 2024-05-27T07:54:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7d05c3db80 10 client.19542 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7d05c3db80 10 client.19542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7d05c3db80 20 client.19542 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56373bded880 0x56373bdefca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7d05c3db80 4 client.19542 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.206+0000 7f7d05c3db80 1 -- 172.21.15.63:0/3698464090 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56373bd03d90 con 0x56373bded880 2024-05-27T07:54:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.207+0000 7f7cefe00700 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56373bded880 0x56373bdefca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.208+0000 7f7cefe00700 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56373bded880 0x56373bdefca0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f3f4c176c7868c server_cookie=f44bb68edc51bf0f in_seq=0 out_seq=0 2024-05-27T07:54:59.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.208+0000 7f7cda000700 10 client.19542 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:59.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.210+0000 7f7cda000700 1 -- 172.21.15.63:0/3698464090 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56373bd03d90 con 0x56373bded880 2024-05-27T07:54:59.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.210+0000 7f7cda000700 10 client.19542 handle_command_reply: tid=0 2024-05-27T07:54:59.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.210+0000 7f7d05c3db80 1 client.19542 shutdown 2024-05-27T07:54:59.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 1 -- 172.21.15.63:0/3698464090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 msgr2=0x56373bdddb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bdddb10 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00cfd0 tx=0x7f7cdc0a9300 comp rx=0 tx=0).stop 2024-05-27T07:54:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 1 -- 172.21.15.63:0/3698464090 shutdown_connections 2024-05-27T07:54:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56373bded880 0x56373bdefca0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bdd3e70 0x56373bdde050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bc54e40 0x56373bddd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/3698464090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bdddb10 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 1 -- 172.21.15.63:0/3698464090 >> 172.21.15.63:0/3698464090 conn(0x56373bcee390 msgr2=0x56373bdd3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7cda000700 0 client.19542 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 1 -- 172.21.15.63:0/3698464090 shutdown_connections 2024-05-27T07:54:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 1 -- 172.21.15.63:0/3698464090 wait complete. 2024-05-27T07:54:59.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.211+0000 7f7d05c3db80 20 client.19542 trim_cache size 0 max 16384 2024-05-27T07:54:59.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.234+0000 7f7d05c3db80 1 Processor -- start 2024-05-27T07:54:59.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.234+0000 7f7d05c3db80 1 -- start start 2024-05-27T07:54:59.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.234+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bc54e40 0x56373bdd8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.234+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bddc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.234+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bdd3e70 0x56373bde9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.234+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd0011610 con 0x56373bd409f0 2024-05-27T07:54:59.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.234+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd000eea0 con 0x56373bc54e40 2024-05-27T07:54:59.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.234+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd000cd10 con 0x56373bdd3e70 2024-05-27T07:54:59.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.235+0000 7f7cef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bddc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.235+0000 7f7cef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bddc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51474/0 (socket says 172.21.15.63:51474) 2024-05-27T07:54:59.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.235+0000 7f7cef400700 1 -- 172.21.15.63:0/2805403475 learned_addr learned my addr 172.21.15.63:0/2805403475 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:59.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.235+0000 7f7cefe00700 1 --2- 172.21.15.63:0/2805403475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bc54e40 0x56373bdd8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.235+0000 7f7cf5000700 1 --2- 172.21.15.63:0/2805403475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bdd3e70 0x56373bde9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.235+0000 7f7cef400700 1 -- 172.21.15.63:0/2805403475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bdd3e70 msgr2=0x56373bde9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:59.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.235+0000 7f7cef400700 1 --2- 172.21.15.63:0/2805403475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bdd3e70 0x56373bde9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.235+0000 7f7cef400700 1 -- 172.21.15.63:0/2805403475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bc54e40 msgr2=0x56373bdd8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:59.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.235+0000 7f7cef400700 1 --2- 172.21.15.63:0/2805403475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bc54e40 0x56373bdd8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.235+0000 7f7cef400700 1 -- 172.21.15.63:0/2805403475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373bdd4a50 con 0x56373bd409f0 2024-05-27T07:54:59.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.236+0000 7f7cef400700 1 --2- 172.21.15.63:0/2805403475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bddc530 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00fdb0 tx=0x7f7cdc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7198af3377cbbc27 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:59.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.236+0000 7f7ce4c00700 1 -- 172.21.15.63:0/2805403475 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc0b26a0 con 0x56373bd409f0 2024-05-27T07:54:59.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.236+0000 7f7ce4c00700 1 -- 172.21.15.63:0/2805403475 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cdc007e50 con 0x56373bd409f0 2024-05-27T07:54:59.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.236+0000 7f7ce4c00700 1 -- 172.21.15.63:0/2805403475 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc0adba0 con 0x56373bd409f0 2024-05-27T07:54:59.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.237+0000 7f7d05c3db80 1 -- 172.21.15.63:0/2805403475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 msgr2=0x56373bddc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:59.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.237+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/2805403475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bddc530 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00fdb0 tx=0x7f7cdc00ef60 comp rx=0 tx=0).stop 2024-05-27T07:54:59.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.237+0000 7f7d05c3db80 1 -- 172.21.15.63:0/2805403475 shutdown_connections 2024-05-27T07:54:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.237+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/2805403475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bdd3e70 0x56373bde9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.237+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/2805403475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bc54e40 0x56373bdd8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.237+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/2805403475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bd409f0 0x56373bddc530 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.237+0000 7f7d05c3db80 1 -- 172.21.15.63:0/2805403475 >> 172.21.15.63:0/2805403475 conn(0x56373bcee390 msgr2=0x56373bdd30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.238+0000 7f7d05c3db80 1 -- 172.21.15.63:0/2805403475 shutdown_connections 2024-05-27T07:54:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.238+0000 7f7d05c3db80 1 -- 172.21.15.63:0/2805403475 wait complete. 2024-05-27T07:54:59.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.238+0000 7f7d05c3db80 1 -- start start 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.238+0000 7f7d05c3db80 1 Processor -- start 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.239+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bc54e40 0x56373bdf3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.239+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 0x56373bded880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.239+0000 7f7d05c3db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bdd3e70 0x56373bdede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.239+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd000d360 con 0x56373bc54e40 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.239+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd000e7d0 con 0x56373bd409f0 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.239+0000 7f7d05c3db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd000da30 con 0x56373bdd3e70 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.239+0000 7f7d05c3db80 10 client.0 _ll_register_callbacks cb 0x56373bc9bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.239+0000 7f7cefe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bc54e40 0x56373bdf3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.239+0000 7f7cefe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bc54e40 0x56373bdf3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51482/0 (socket says 172.21.15.63:51482) 2024-05-27T07:54:59.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.239+0000 7f7cefe00700 1 -- 172.21.15.63:0/125386475 learned_addr learned my addr 172.21.15.63:0/125386475 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:54:59.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.240+0000 7f7cf5000700 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bdd3e70 0x56373bdede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.240+0000 7f7cef400700 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 0x56373bded880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.240+0000 7f7cefe00700 1 -- 172.21.15.63:0/125386475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bdd3e70 msgr2=0x56373bdede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:59.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.240+0000 7f7cefe00700 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bdd3e70 0x56373bdede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.240+0000 7f7cefe00700 1 -- 172.21.15.63:0/125386475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 msgr2=0x56373bded880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.240+0000 7f7cefe00700 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 0x56373bded880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.240+0000 7f7cefe00700 1 -- 172.21.15.63:0/125386475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373bdee5f0 con 0x56373bc54e40 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.240+0000 7f7cef400700 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 0x56373bded880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.240+0000 7f7cefe00700 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bc54e40 0x56373bdf3060 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009350 tx=0x7f7ce00055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969afd0bd84caf7f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.240+0000 7f7cda000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.241+0000 7f7cda000700 1 -- 172.21.15.63:0/125386475 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0003c90 con 0x56373bc54e40 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.241+0000 7f7d05c3db80 1 -- 172.21.15.63:0/125386475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cdc007e50 con 0x56373bc54e40 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.241+0000 7f7cda000700 1 -- 172.21.15.63:0/125386475 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9483+0+0 (secure 0 0 0) 0x7f7ce000b7d0 con 0x56373bc54e40 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.241+0000 7f7cda000700 1 -- 172.21.15.63:0/125386475 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7ce0038050 con 0x56373bc54e40 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.241+0000 7f7d05c3db80 10 client.19548 fetch_fsmap learned FSMap version 33 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.241+0000 7f7d05c3db80 1 -- 172.21.15.63:0/125386475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56373be57120 con 0x56373bc54e40 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.241+0000 7f7cda000700 1 -- 172.21.15.63:0/125386475 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cdc007e50 con 0x56373bc54e40 2024-05-27T07:54:59.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.242+0000 7f7cda000700 1 -- 172.21.15.63:0/125386475 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7ce000f7b0 con 0x56373bc54e40 2024-05-27T07:54:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.242+0000 7f7d05c3db80 10 client.19548 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:54:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.242+0000 7f7d05c3db80 10 client.19548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.242+0000 7f7d05c3db80 20 client.19548 populate_metadata read hostname 'smithi063' 2024-05-27T07:54:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.242+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56373be57860 0x56373be59c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.242+0000 7f7d05c3db80 4 client.19548 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:54:59.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.242+0000 7f7d05c3db80 1 -- 172.21.15.63:0/125386475 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56373bcfd570 con 0x56373be57860 2024-05-27T07:54:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.243+0000 7f7cef400700 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56373be57860 0x56373be59c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.243+0000 7f7cef400700 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56373be57860 0x56373be59c80 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f59451d7e2c4fe7a server_cookie=df9860ed1ba902ff in_seq=0 out_seq=0 2024-05-27T07:54:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.243+0000 7f7cda000700 10 client.19548 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7cda000700 1 -- 172.21.15.63:0/125386475 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56373bcfd570 con 0x56373be57860 2024-05-27T07:54:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7cda000700 10 client.19548 handle_command_reply: tid=0 2024-05-27T07:54:59.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7d05c3db80 1 client.19548 shutdown 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7d05c3db80 1 -- 172.21.15.63:0/125386475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bc54e40 msgr2=0x56373bdf3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bc54e40 0x56373bdf3060 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009350 tx=0x7f7ce00055b0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7d05c3db80 1 -- 172.21.15.63:0/125386475 shutdown_connections 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56373be57860 0x56373be59c80 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56373bdd3e70 0x56373bdede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56373bc54e40 0x56373bdf3060 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7d05c3db80 1 --2- 172.21.15.63:0/125386475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56373bd409f0 0x56373bded880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7d05c3db80 1 -- 172.21.15.63:0/125386475 >> 172.21.15.63:0/125386475 conn(0x56373bcee390 msgr2=0x56373bdd4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.244+0000 7f7cda000700 0 client.19548 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.245+0000 7f7d05c3db80 1 -- 172.21.15.63:0/125386475 shutdown_connections 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.245+0000 7f7d05c3db80 1 -- 172.21.15.63:0/125386475 wait complete. 2024-05-27T07:54:59.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.245+0000 7f7d05c3db80 20 client.19548 trim_cache size 0 max 16384 2024-05-27T07:54:59.248 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:54:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 -- 172.21.15.63:0/3075560779 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7cd40b2a70 msgr2=0x7f7cd40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7cd40b2a70 0x7f7cd40b4f80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x56373bdd5ff0 tx=0x7f7ce8004ad0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 -- 172.21.15.63:0/3075560779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf0105e20 msgr2=0x7f7cf019d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf0105e20 0x7f7cf019d4d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c9b0 tx=0x7f7cdc00cd70 comp rx=0 tx=0).stop 2024-05-27T07:54:59.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 -- 172.21.15.63:0/3075560779 shutdown_connections 2024-05-27T07:54:59.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7cd40b2a70 0x7f7cd40b4f80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cf010e900 0x7f7cf01a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf01054d0 0x7f7cf019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 --2- 172.21.15.63:0/3075560779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7cf0105e20 0x7f7cf019d4d0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:59.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 -- 172.21.15.63:0/3075560779 >> 172.21.15.63:0/3075560779 conn(0x7f7cf00fef20 msgr2=0x7f7cf01031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:59.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 -- 172.21.15.63:0/3075560779 shutdown_connections 2024-05-27T07:54:59.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:54:59.246+0000 7f7cf7000700 1 -- 172.21.15.63:0/3075560779 wait complete. 2024-05-27T07:54:59.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:54:59 smithi063 ceph-mon[31613]: pgmap v675: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:54:59.517 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:54:59 smithi149 ceph-mon[35408]: pgmap v675: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:54:59.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:54:59 smithi086 ceph-mon[34687]: pgmap v675: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T07:55:01.227 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:55:01.228 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:55:01.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:01 smithi086 ceph-mon[34687]: pgmap v676: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:55:01.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:01 smithi149 ceph-mon[35408]: pgmap v676: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:55:01.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:01 smithi063 ceph-mon[31613]: pgmap v676: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:55:01.718 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:03.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:03 smithi063 ceph-mon[31613]: pgmap v677: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T07:55:03.463 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:03 smithi063 ceph-mon[31613]: osdmap e124: 12 total, 12 up, 12 in 2024-05-27T07:55:03.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:03 smithi086 ceph-mon[34687]: pgmap v677: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T07:55:03.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:03 smithi086 ceph-mon[34687]: osdmap e124: 12 total, 12 up, 12 in 2024-05-27T07:55:03.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:03 smithi149 ceph-mon[35408]: pgmap v677: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T07:55:03.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:03 smithi149 ceph-mon[35408]: osdmap e124: 12 total, 12 up, 12 in 2024-05-27T07:55:04.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.116+0000 7fde3ae00700 1 -- 172.21.15.63:0/1798553391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde341030d0 msgr2=0x7fde341034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:04.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.116+0000 7fde3ae00700 1 --2- 172.21.15.63:0/1798553391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde341030d0 0x7fde341034e0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fde24009990 tx=0x7fde24009ca0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.116+0000 7fde3ae00700 1 -- 172.21.15.63:0/1798553391 shutdown_connections 2024-05-27T07:55:04.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.116+0000 7fde3ae00700 1 --2- 172.21.15.63:0/1798553391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde34110c80 0x7fde341130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.116+0000 7fde3ae00700 1 --2- 172.21.15.63:0/1798553391 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde341030d0 0x7fde341034e0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.116+0000 7fde3ae00700 1 --2- 172.21.15.63:0/1798553391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde34103a20 0x7fde34110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.116+0000 7fde3ae00700 1 -- 172.21.15.63:0/1798553391 >> 172.21.15.63:0/1798553391 conn(0x7fde340fede0 msgr2=0x7fde34101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:04.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.116+0000 7fde3ae00700 1 -- 172.21.15.63:0/1798553391 shutdown_connections 2024-05-27T07:55:04.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.116+0000 7fde3ae00700 1 -- 172.21.15.63:0/1798553391 wait complete. 2024-05-27T07:55:04.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.117+0000 7fde3ae00700 1 Processor -- start 2024-05-27T07:55:04.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.118+0000 7fde3ae00700 1 -- start start 2024-05-27T07:55:04.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.118+0000 7fde3ae00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde341030d0 0x7fde3419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.118+0000 7fde3ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde34103a20 0x7fde3419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.118+0000 7fde3ae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde34110c80 0x7fde341a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.118+0000 7fde3ae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde34116510 con 0x7fde34110c80 2024-05-27T07:55:04.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.118+0000 7fde3ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde34116390 con 0x7fde34103a20 2024-05-27T07:55:04.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.118+0000 7fde3ae00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde34116690 con 0x7fde341030d0 2024-05-27T07:55:04.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.119+0000 7fde38e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde34110c80 0x7fde341a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.119+0000 7fde33e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde341030d0 0x7fde3419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.119+0000 7fde38e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde34110c80 0x7fde341a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51496/0 (socket says 172.21.15.63:51496) 2024-05-27T07:55:04.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.119+0000 7fde38e00700 1 -- 172.21.15.63:0/2485593661 learned_addr learned my addr 172.21.15.63:0/2485593661 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:04.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.119+0000 7fde33400700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde34103a20 0x7fde3419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.120+0000 7fde38e00700 1 -- 172.21.15.63:0/2485593661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde341030d0 msgr2=0x7fde3419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:04.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.120+0000 7fde38e00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde341030d0 0x7fde3419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.120+0000 7fde38e00700 1 -- 172.21.15.63:0/2485593661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde34103a20 msgr2=0x7fde3419d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:04.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.120+0000 7fde38e00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde34103a20 0x7fde3419d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.120+0000 7fde38e00700 1 -- 172.21.15.63:0/2485593661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde341a1d80 con 0x7fde34110c80 2024-05-27T07:55:04.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.120+0000 7fde38e00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde34110c80 0x7fde341a1790 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fde2c007d70 tx=0x7fde2c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:04.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.120+0000 7fde30c00700 1 -- 172.21.15.63:0/2485593661 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c016020 con 0x7fde34110c80 2024-05-27T07:55:04.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.120+0000 7fde30c00700 1 -- 172.21.15.63:0/2485593661 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde2c00ad40 con 0x7fde34110c80 2024-05-27T07:55:04.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.120+0000 7fde30c00700 1 -- 172.21.15.63:0/2485593661 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c0115e0 con 0x7fde34110c80 2024-05-27T07:55:04.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.121+0000 7fde3ae00700 1 -- 172.21.15.63:0/2485593661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde341a2070 con 0x7fde34110c80 2024-05-27T07:55:04.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.121+0000 7fde3ae00700 1 -- 172.21.15.63:0/2485593661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde341a2560 con 0x7fde34110c80 2024-05-27T07:55:04.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.125+0000 7fde30c00700 1 -- 172.21.15.63:0/2485593661 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fde2c019020 con 0x7fde34110c80 2024-05-27T07:55:04.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.127+0000 7fde49982b80 1 Processor -- start 2024-05-27T07:55:04.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.127+0000 7fde49982b80 1 -- start start 2024-05-27T07:55:04.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.127+0000 7fde49982b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1db1dec0 0x55ed1dca2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.127+0000 7fde30c00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde180b29f0 0x7fde180b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.127+0000 7fde49982b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dc9c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.127+0000 7fde49982b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1dc9ce30 0x55ed1dc9d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.128+0000 7fde49982b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ed1dbcaec0 con 0x55ed1dca2db0 2024-05-27T07:55:04.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.128+0000 7fde49982b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed1dbbaa60 con 0x55ed1dc9ce30 2024-05-27T07:55:04.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.128+0000 7fde49982b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ed1dbbeaa0 con 0x55ed1db1dec0 2024-05-27T07:55:04.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.128+0000 7fde30c00700 1 -- 172.21.15.63:0/2485593661 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9571+0+0 (secure 0 0 0) 0x7fde2c0e28c0 con 0x7fde34110c80 2024-05-27T07:55:04.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.128+0000 7fde33e00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde180b29f0 0x7fde180b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.128+0000 7fde33400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1db1dec0 0x55ed1dca2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.128+0000 7fde38e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1dc9ce30 0x55ed1dc9d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.128+0000 7fde33400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1db1dec0 0x55ed1dca2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:39460/0 (socket says 172.21.15.63:39460) 2024-05-27T07:55:04.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.128+0000 7fde38e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1dc9ce30 0x55ed1dc9d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51136/0 (socket says 172.21.15.63:51136) 2024-05-27T07:55:04.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.128+0000 7fde33400700 1 -- 172.21.15.63:0/660269328 learned_addr learned my addr 172.21.15.63:0/660269328 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:04.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.129+0000 7fde33400700 1 --2- 172.21.15.63:0/660269328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dc9c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.129+0000 7fde33400700 1 -- 172.21.15.63:0/660269328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1db1dec0 msgr2=0x55ed1dca2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:04.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.129+0000 7fde33400700 1 --2- 172.21.15.63:0/660269328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1db1dec0 0x55ed1dca2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.129+0000 7fde33400700 1 -- 172.21.15.63:0/660269328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1dc9ce30 msgr2=0x55ed1dc9d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:04.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.129+0000 7fde33400700 1 --2- 172.21.15.63:0/660269328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1dc9ce30 0x55ed1dc9d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.129+0000 7fde33400700 1 -- 172.21.15.63:0/660269328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1db834a0 con 0x55ed1dca2db0 2024-05-27T07:55:04.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.129+0000 7fde33e00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde180b29f0 0x7fde180b4f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fde24006010 tx=0x7fde2400b540 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:04.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.130+0000 7fde33400700 1 --2- 172.21.15.63:0/660269328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dc9c8f0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fde20005d90 tx=0x7fde2000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f818082583215bf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:04.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.130+0000 7fde29600700 1 -- 172.21.15.63:0/660269328 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20004830 con 0x55ed1dca2db0 2024-05-27T07:55:04.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.130+0000 7fde29600700 1 -- 172.21.15.63:0/660269328 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde20005020 con 0x55ed1dca2db0 2024-05-27T07:55:04.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.130+0000 7fde29600700 1 -- 172.21.15.63:0/660269328 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20004e60 con 0x55ed1dca2db0 2024-05-27T07:55:04.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.131+0000 7fde49982b80 1 -- 172.21.15.63:0/660269328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 msgr2=0x55ed1dc9c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:04.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.131+0000 7fde49982b80 1 --2- 172.21.15.63:0/660269328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dc9c8f0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fde20005d90 tx=0x7fde2000b410 comp rx=0 tx=0).stop 2024-05-27T07:55:04.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.131+0000 7fde49982b80 1 -- 172.21.15.63:0/660269328 shutdown_connections 2024-05-27T07:55:04.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.131+0000 7fde49982b80 1 --2- 172.21.15.63:0/660269328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1dc9ce30 0x55ed1dc9d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.131+0000 7fde49982b80 1 --2- 172.21.15.63:0/660269328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1db1dec0 0x55ed1dca2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.131+0000 7fde49982b80 1 --2- 172.21.15.63:0/660269328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dc9c8f0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.131+0000 7fde49982b80 1 -- 172.21.15.63:0/660269328 >> 172.21.15.63:0/660269328 conn(0x55ed1db7e040 msgr2=0x55ed1dc97950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.131+0000 7fde49982b80 1 -- 172.21.15.63:0/660269328 shutdown_connections 2024-05-27T07:55:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.132+0000 7fde49982b80 1 -- 172.21.15.63:0/660269328 wait complete. 2024-05-27T07:55:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.132+0000 7fde49982b80 1 -- start start 2024-05-27T07:55:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.133+0000 7fde49982b80 1 Processor -- start 2024-05-27T07:55:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.133+0000 7fde49982b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dca65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.133+0000 7fde49982b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dca6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.133+0000 7fde49982b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dca2db0 0x55ed1dca7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.133+0000 7fde49982b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ed1dac1020 con 0x55ed1dc9ce30 2024-05-27T07:55:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.133+0000 7fde49982b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed1db5b7e0 con 0x55ed1db1dec0 2024-05-27T07:55:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.133+0000 7fde49982b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ed1dca9880 con 0x55ed1dca2db0 2024-05-27T07:55:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.133+0000 7fde49982b80 10 client.0 _ll_register_callbacks cb 0x55ed1db64cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.134+0000 7fde33400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dca6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.134+0000 7fde33400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dca6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51506/0 (socket says 172.21.15.63:51506) 2024-05-27T07:55:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.134+0000 7fde33400700 1 -- 172.21.15.63:0/1065299821 learned_addr learned my addr 172.21.15.63:0/1065299821 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.134+0000 7fde33e00700 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dca65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.134+0000 7fde38e00700 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dca2db0 0x55ed1dca7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.134+0000 7fde33400700 1 -- 172.21.15.63:0/1065299821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dca2db0 msgr2=0x55ed1dca7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.134+0000 7fde33400700 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dca2db0 0x55ed1dca7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.134+0000 7fde33400700 1 -- 172.21.15.63:0/1065299821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 msgr2=0x55ed1dca65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.134+0000 7fde33400700 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dca65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.134+0000 7fde33400700 1 -- 172.21.15.63:0/1065299821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1db0a950 con 0x55ed1dc9ce30 2024-05-27T07:55:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.135+0000 7fde33400700 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dca6af0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fde20005410 tx=0x7fde20005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50f8a87aa651ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.135+0000 7fde1e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.135+0000 7fde1e000700 1 -- 172.21.15.63:0/1065299821 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20004960 con 0x55ed1dc9ce30 2024-05-27T07:55:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.135+0000 7fde1e000700 1 -- 172.21.15.63:0/1065299821 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9571+0+0 (secure 0 0 0) 0x7fde2001b070 con 0x55ed1dc9ce30 2024-05-27T07:55:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.135+0000 7fde49982b80 1 -- 172.21.15.63:0/1065299821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde20005020 con 0x55ed1dc9ce30 2024-05-27T07:55:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.136+0000 7fde1e000700 1 -- 172.21.15.63:0/1065299821 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde20004b00 con 0x55ed1dc9ce30 2024-05-27T07:55:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.136+0000 7fde49982b80 10 client.19560 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.136+0000 7fde49982b80 1 -- 172.21.15.63:0/1065299821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ed1db8b0e0 con 0x55ed1dc9ce30 2024-05-27T07:55:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.136+0000 7fde1e000700 1 -- 172.21.15.63:0/1065299821 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde20005020 con 0x55ed1dc9ce30 2024-05-27T07:55:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.136+0000 7fde1e000700 1 -- 172.21.15.63:0/1065299821 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fde20003680 con 0x55ed1dc9ce30 2024-05-27T07:55:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.136+0000 7fde49982b80 10 client.19560 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.136+0000 7fde49982b80 10 client.19560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.137+0000 7fde49982b80 20 client.19560 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.137+0000 7fde49982b80 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed1dcb6710 0x55ed1dcb8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.137+0000 7fde49982b80 4 client.19560 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.137+0000 7fde49982b80 1 -- 172.21.15.63:0/1065299821 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed1dbcce10 con 0x55ed1dcb6710 2024-05-27T07:55:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.137+0000 7fde33e00700 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed1dcb6710 0x55ed1dcb8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.138+0000 7fde33e00700 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed1dcb6710 0x55ed1dcb8b30 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269f184193673259 server_cookie=4f99be0daeb330a6 in_seq=0 out_seq=0 2024-05-27T07:55:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.138+0000 7fde1e000700 10 client.19560 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.140+0000 7fde1e000700 1 -- 172.21.15.63:0/1065299821 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ed1dbcce10 con 0x55ed1dcb6710 2024-05-27T07:55:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.140+0000 7fde1e000700 10 client.19560 handle_command_reply: tid=0 2024-05-27T07:55:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 client.19560 shutdown 2024-05-27T07:55:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 -- 172.21.15.63:0/1065299821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 msgr2=0x55ed1dca6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dca6af0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fde20005410 tx=0x7fde20005440 comp rx=0 tx=0).stop 2024-05-27T07:55:04.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 -- 172.21.15.63:0/1065299821 shutdown_connections 2024-05-27T07:55:04.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed1dcb6710 0x55ed1dcb8b30 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dca2db0 0x55ed1dca7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dca65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 --2- 172.21.15.63:0/1065299821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dca6af0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 -- 172.21.15.63:0/1065299821 >> 172.21.15.63:0/1065299821 conn(0x55ed1db7e040 msgr2=0x55ed1dc97600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:04.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde1e000700 0 client.19560 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:04.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 -- 172.21.15.63:0/1065299821 shutdown_connections 2024-05-27T07:55:04.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.141+0000 7fde49982b80 1 -- 172.21.15.63:0/1065299821 wait complete. 2024-05-27T07:55:04.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.142+0000 7fde49982b80 20 client.19560 trim_cache size 0 max 16384 2024-05-27T07:55:04.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.175+0000 7fde49982b80 1 Processor -- start 2024-05-27T07:55:04.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.175+0000 7fde49982b80 1 -- start start 2024-05-27T07:55:04.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.175+0000 7fde49982b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dca8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.175+0000 7fde49982b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dcb2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.175+0000 7fde49982b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dca2db0 0x55ed1dca3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.175+0000 7fde49982b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b5020 con 0x55ed1dc9ce30 2024-05-27T07:55:04.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.175+0000 7fde49982b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b28b0 con 0x55ed1db1dec0 2024-05-27T07:55:04.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.175+0000 7fde49982b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b0720 con 0x55ed1dca2db0 2024-05-27T07:55:04.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.176+0000 7fde33400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dcb2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.176+0000 7fde33400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dcb2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51510/0 (socket says 172.21.15.63:51510) 2024-05-27T07:55:04.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.176+0000 7fde33400700 1 -- 172.21.15.63:0/4230125274 learned_addr learned my addr 172.21.15.63:0/4230125274 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:04.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.176+0000 7fde33e00700 1 --2- 172.21.15.63:0/4230125274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dca8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.176+0000 7fde38e00700 1 --2- 172.21.15.63:0/4230125274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dca2db0 0x55ed1dca3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.177+0000 7fde33400700 1 -- 172.21.15.63:0/4230125274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dca2db0 msgr2=0x55ed1dca3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:04.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.177+0000 7fde33400700 1 --2- 172.21.15.63:0/4230125274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dca2db0 0x55ed1dca3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.177+0000 7fde33400700 1 -- 172.21.15.63:0/4230125274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 msgr2=0x55ed1dca8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:04.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.177+0000 7fde33400700 1 --2- 172.21.15.63:0/4230125274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dca8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.177+0000 7fde33400700 1 -- 172.21.15.63:0/4230125274 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1dca75c0 con 0x55ed1dc9ce30 2024-05-27T07:55:04.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.177+0000 7fde33400700 1 --2- 172.21.15.63:0/4230125274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dcb2b50 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fde20029480 tx=0x7fde2000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d95fab1aca5a895 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:04.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.177+0000 7fde28c00700 1 -- 172.21.15.63:0/4230125274 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2003b070 con 0x55ed1dc9ce30 2024-05-27T07:55:04.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.177+0000 7fde28c00700 1 -- 172.21.15.63:0/4230125274 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde2000bc50 con 0x55ed1dc9ce30 2024-05-27T07:55:04.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.177+0000 7fde28c00700 1 -- 172.21.15.63:0/4230125274 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20027360 con 0x55ed1dc9ce30 2024-05-27T07:55:04.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.178+0000 7fde49982b80 1 -- 172.21.15.63:0/4230125274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 msgr2=0x55ed1dcb2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:04.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.178+0000 7fde49982b80 1 --2- 172.21.15.63:0/4230125274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dcb2b50 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fde20029480 tx=0x7fde2000b410 comp rx=0 tx=0).stop 2024-05-27T07:55:04.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.179+0000 7fde49982b80 1 -- 172.21.15.63:0/4230125274 shutdown_connections 2024-05-27T07:55:04.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.179+0000 7fde49982b80 1 --2- 172.21.15.63:0/4230125274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dca2db0 0x55ed1dca3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.179+0000 7fde49982b80 1 --2- 172.21.15.63:0/4230125274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dca8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.179+0000 7fde49982b80 1 --2- 172.21.15.63:0/4230125274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dc9ce30 0x55ed1dcb2b50 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.179+0000 7fde49982b80 1 -- 172.21.15.63:0/4230125274 >> 172.21.15.63:0/4230125274 conn(0x55ed1db7e040 msgr2=0x55ed1dca34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:04.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.179+0000 7fde49982b80 1 -- 172.21.15.63:0/4230125274 shutdown_connections 2024-05-27T07:55:04.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.179+0000 7fde49982b80 1 -- 172.21.15.63:0/4230125274 wait complete. 2024-05-27T07:55:04.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.179+0000 7fde49982b80 1 -- start start 2024-05-27T07:55:04.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.180+0000 7fde49982b80 1 Processor -- start 2024-05-27T07:55:04.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.180+0000 7fde49982b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dcbbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.180+0000 7fde49982b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dc9ce30 0x55ed1dcba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde49982b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dcba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde49982b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b0d70 con 0x55ed1dca2db0 2024-05-27T07:55:04.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde49982b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b21e0 con 0x55ed1db1dec0 2024-05-27T07:55:04.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde49982b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde341b1440 con 0x55ed1dc9ce30 2024-05-27T07:55:04.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde49982b80 10 client.0 _ll_register_callbacks cb 0x55ed1db64cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:04.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde38e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dcba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde38e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dcba830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51526/0 (socket says 172.21.15.63:51526) 2024-05-27T07:55:04.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde38e00700 1 -- 172.21.15.63:0/1210540664 learned_addr learned my addr 172.21.15.63:0/1210540664 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:04.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde33e00700 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dcbbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde33400700 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dc9ce30 0x55ed1dcba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde38e00700 1 -- 172.21.15.63:0/1210540664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dc9ce30 msgr2=0x55ed1dcba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:04.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.181+0000 7fde38e00700 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dc9ce30 0x55ed1dcba250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.182+0000 7fde38e00700 1 -- 172.21.15.63:0/1210540664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 msgr2=0x55ed1dcbbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.182+0000 7fde38e00700 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dcbbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.182+0000 7fde38e00700 1 -- 172.21.15.63:0/1210540664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1dc911b0 con 0x55ed1dca2db0 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.182+0000 7fde33e00700 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dcbbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.182+0000 7fde38e00700 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dcba830 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0a6aa0 tx=0x7fde2c0a9340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c02bacc7e68313 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.182+0000 7fde1e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.182+0000 7fde1e000700 1 -- 172.21.15.63:0/1210540664 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c0adbe0 con 0x55ed1dca2db0 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.182+0000 7fde1e000700 1 -- 172.21.15.63:0/1210540664 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9571+0+0 (secure 0 0 0) 0x7fde2c0a9a90 con 0x55ed1dca2db0 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.182+0000 7fde49982b80 1 -- 172.21.15.63:0/1210540664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde2000bc50 con 0x55ed1dca2db0 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde1e000700 1 -- 172.21.15.63:0/1210540664 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde2c0b2690 con 0x55ed1dca2db0 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde49982b80 10 client.19566 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:04.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde49982b80 1 -- 172.21.15.63:0/1210540664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ed1dd21dd0 con 0x55ed1dca2db0 2024-05-27T07:55:04.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde1e000700 1 -- 172.21.15.63:0/1210540664 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde2000bc50 con 0x55ed1dca2db0 2024-05-27T07:55:04.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde1e000700 1 -- 172.21.15.63:0/1210540664 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fde2c0e24e0 con 0x55ed1dca2db0 2024-05-27T07:55:04.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde49982b80 10 client.19566 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:04.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde49982b80 10 client.19566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:04.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde49982b80 20 client.19566 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:04.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde49982b80 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed1dd224b0 0x55ed1dd248d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:04.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde49982b80 4 client.19566 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:55:04.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.183+0000 7fde49982b80 1 -- 172.21.15.63:0/1210540664 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ed1dbc65f0 con 0x55ed1dd224b0 2024-05-27T07:55:04.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.184+0000 7fde33e00700 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed1dd224b0 0x55ed1dd248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:04.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.185+0000 7fde33e00700 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed1dd224b0 0x55ed1dd248d0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e68e86859430554 server_cookie=ee3e8cf360391f67 in_seq=0 out_seq=0 2024-05-27T07:55:04.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.185+0000 7fde1e000700 10 client.19566 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde1e000700 1 -- 172.21.15.63:0/1210540664 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ed1dbc65f0 con 0x55ed1dd224b0 2024-05-27T07:55:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde1e000700 10 client.19566 handle_command_reply: tid=0 2024-05-27T07:55:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde49982b80 1 client.19566 shutdown 2024-05-27T07:55:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde49982b80 1 -- 172.21.15.63:0/1210540664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 msgr2=0x55ed1dcba830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:04.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde49982b80 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dcba830 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0a6aa0 tx=0x7fde2c0a9340 comp rx=0 tx=0).stop 2024-05-27T07:55:04.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde49982b80 1 -- 172.21.15.63:0/1210540664 shutdown_connections 2024-05-27T07:55:04.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde49982b80 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ed1dd224b0 0x55ed1dd248d0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde49982b80 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed1dca2db0 0x55ed1dcba830 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde49982b80 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed1db1dec0 0x55ed1dcbbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde49982b80 1 --2- 172.21.15.63:0/1210540664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed1dc9ce30 0x55ed1dcba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde1e000700 0 client.19566 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:04.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.188+0000 7fde49982b80 1 -- 172.21.15.63:0/1210540664 >> 172.21.15.63:0/1210540664 conn(0x55ed1db7e040 msgr2=0x55ed1dca3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:04.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.189+0000 7fde49982b80 1 -- 172.21.15.63:0/1210540664 shutdown_connections 2024-05-27T07:55:04.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.189+0000 7fde49982b80 1 -- 172.21.15.63:0/1210540664 wait complete. 2024-05-27T07:55:04.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.189+0000 7fde49982b80 20 client.19566 trim_cache size 0 max 16384 2024-05-27T07:55:04.191 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:55:04.192 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:55:04.192 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "4913ce03-ada1-4555-8206-97b111286bd4", 2024-05-27T07:55:04.192 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:55:04.192 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:55:04.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 -- 172.21.15.63:0/2485593661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde180b29f0 msgr2=0x7fde180b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:04.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde180b29f0 0x7fde180b4f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fde24006010 tx=0x7fde2400b540 comp rx=0 tx=0).stop 2024-05-27T07:55:04.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 -- 172.21.15.63:0/2485593661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde34110c80 msgr2=0x7fde341a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:04.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde34110c80 0x7fde341a1790 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fde2c007d70 tx=0x7fde2c00a340 comp rx=0 tx=0).stop 2024-05-27T07:55:04.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 -- 172.21.15.63:0/2485593661 shutdown_connections 2024-05-27T07:55:04.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fde180b29f0 0x7fde180b4f00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde34110c80 0x7fde341a1790 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde341030d0 0x7fde3419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 --2- 172.21.15.63:0/2485593661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde34103a20 0x7fde3419d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:04.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 -- 172.21.15.63:0/2485593661 >> 172.21.15.63:0/2485593661 conn(0x7fde340fede0 msgr2=0x7fde34101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:04.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 -- 172.21.15.63:0/2485593661 shutdown_connections 2024-05-27T07:55:04.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:04.190+0000 7fde3ae00700 1 -- 172.21.15.63:0/2485593661 wait complete. 2024-05-27T07:55:05.226 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:55:05.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: pgmap v679: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 33 op/s 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:55:05.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.469 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:55:05.469 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.469 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:55:05.469 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.469 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: pgmap v679: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 33 op/s 2024-05-27T07:55:05.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:55:05.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:55:05.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:55:05.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:55:05.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:55:05.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:55:05.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:55:05.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.713 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:55:05.713 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.713 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: pgmap v679: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 33 op/s 2024-05-27T07:55:05.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:55:05.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:55:05.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:55:05.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:55:05.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:55:05.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:55:05.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:55:05.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:55:05.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.717 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:55:05.744 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.017+0000 7f8c53e00640 1 -- 172.21.15.63:0/1519432644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c54104250 msgr2=0x7f8c541105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.017+0000 7f8c53e00640 1 --2- 172.21.15.63:0/1519432644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c54104250 0x7f8c541105d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009ca0 tx=0x7f8c4c02a8b0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.017+0000 7f8c53e00640 1 -- 172.21.15.63:0/1519432644 shutdown_connections 2024-05-27T07:55:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.017+0000 7f8c53e00640 1 --2- 172.21.15.63:0/1519432644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c54104250 0x7f8c541105d0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.017+0000 7f8c53e00640 1 --2- 172.21.15.63:0/1519432644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54109f50 0x7f8c54103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.017+0000 7f8c53e00640 1 --2- 172.21.15.63:0/1519432644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c54109ab0 0x7f8c541036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.017+0000 7f8c53e00640 1 -- 172.21.15.63:0/1519432644 >> 172.21.15.63:0/1519432644 conn(0x7f8c54079240 msgr2=0x7f8c54079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.017+0000 7f8c53e00640 1 -- 172.21.15.63:0/1519432644 shutdown_connections 2024-05-27T07:55:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.017+0000 7f8c53e00640 1 -- 172.21.15.63:0/1519432644 wait complete. 2024-05-27T07:55:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.018+0000 7f8c53e00640 1 Processor -- start 2024-05-27T07:55:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.018+0000 7f8c53e00640 1 -- start start 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.019+0000 7f8c53e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c54104250 0x7f8c54074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.019+0000 7f8c53e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54109ab0 0x7f8c54075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.019+0000 7f8c53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c54109f50 0x7f8c54075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.019+0000 7f8c53e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54113ba0 con 0x7f8c54104250 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.019+0000 7f8c53e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54113a20 con 0x7f8c54109f50 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.019+0000 7f8c53e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54113d20 con 0x7f8c54109ab0 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.019+0000 7f8c52a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c54104250 0x7f8c54074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.019+0000 7f8c52a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c54104250 0x7f8c54074fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51542/0 (socket says 172.21.15.63:51542) 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.019+0000 7f8c52a00640 1 -- 172.21.15.63:0/1600400621 learned_addr learned my addr 172.21.15.63:0/1600400621 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.019+0000 7f8c52000640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54109ab0 0x7f8c54075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.020+0000 7f8c53400640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c54109f50 0x7f8c54075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.020+0000 7f8c52a00640 1 -- 172.21.15.63:0/1600400621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54109ab0 msgr2=0x7f8c54075500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.020+0000 7f8c52a00640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54109ab0 0x7f8c54075500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.020+0000 7f8c52a00640 1 -- 172.21.15.63:0/1600400621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c54109f50 msgr2=0x7f8c54075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.020+0000 7f8c52a00640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c54109f50 0x7f8c54075a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.020+0000 7f8c52a00640 1 -- 172.21.15.63:0/1600400621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c541a8870 con 0x7f8c54104250 2024-05-27T07:55:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.020+0000 7f8c52a00640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c54104250 0x7f8c54074fc0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400d770 tx=0x7f8c4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.021+0000 7f8c4b400640 1 -- 172.21.15.63:0/1600400621 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44017070 con 0x7f8c54104250 2024-05-27T07:55:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.021+0000 7f8c53e00640 1 -- 172.21.15.63:0/1600400621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c541a8b60 con 0x7f8c54104250 2024-05-27T07:55:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.021+0000 7f8c4b400640 1 -- 172.21.15.63:0/1600400621 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c44009c30 con 0x7f8c54104250 2024-05-27T07:55:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.021+0000 7f8c4b400640 1 -- 172.21.15.63:0/1600400621 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400f620 con 0x7f8c54104250 2024-05-27T07:55:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.021+0000 7f8c53e00640 1 -- 172.21.15.63:0/1600400621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541a90b0 con 0x7f8c54104250 2024-05-27T07:55:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.026+0000 7f8c60bb9740 1 Processor -- start 2024-05-27T07:55:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.026+0000 7f8c60bb9740 1 -- start start 2024-05-27T07:55:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55779452f1b0 0x5577948d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577945062f0 0x5577948d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557794842b90 con 0x55779450e4d0 2024-05-27T07:55:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557794832730 con 0x55779452f1b0 2024-05-27T07:55:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557794836770 con 0x5577945062f0 2024-05-27T07:55:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c4b400640 1 -- 172.21.15.63:0/1600400621 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8c4401a070 con 0x7f8c54104250 2024-05-27T07:55:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c52000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c52000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51556/0 (socket says 172.21.15.63:51556) 2024-05-27T07:55:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c52000640 1 -- 172.21.15.63:0/3816017755 learned_addr learned my addr 172.21.15.63:0/3816017755 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.027+0000 7f8c53400640 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55779452f1b0 0x5577948d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.028+0000 7f8c52a00640 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577945062f0 0x5577948d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.028+0000 7f8c52000640 1 -- 172.21.15.63:0/3816017755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577945062f0 msgr2=0x5577948d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.028+0000 7f8c52000640 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577945062f0 0x5577948d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.028+0000 7f8c52000640 1 -- 172.21.15.63:0/3816017755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55779452f1b0 msgr2=0x5577948d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.028+0000 7f8c52000640 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55779452f1b0 0x5577948d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.028+0000 7f8c52000640 1 -- 172.21.15.63:0/3816017755 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577947cb8b0 con 0x55779450e4d0 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.028+0000 7f8c53400640 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55779452f1b0 0x5577948d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.028+0000 7f8c52a00640 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577945062f0 0x5577948d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.028+0000 7f8c52000640 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948d24e0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f8c40005e30 tx=0x7f8c4000e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6a0d620782b9ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.028+0000 7f8c3fe00640 1 -- 172.21.15.63:0/3816017755 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40004020 con 0x55779450e4d0 2024-05-27T07:55:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.029+0000 7f8c3fe00640 1 -- 172.21.15.63:0/3816017755 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c400054b0 con 0x55779450e4d0 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.029+0000 7f8c3fe00640 1 -- 172.21.15.63:0/3816017755 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40004c00 con 0x55779450e4d0 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.029+0000 7f8c4b400640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8c380b2950 0x7f8c380b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.029+0000 7f8c4b400640 1 -- 172.21.15.63:0/1600400621 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9571+0+0 (secure 0 0 0) 0x7f8c440e25e0 con 0x7f8c54104250 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.029+0000 7f8c53400640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8c380b2950 0x7f8c380b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.030+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3816017755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 msgr2=0x5577948d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.030+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948d24e0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f8c40005e30 tx=0x7f8c4000e350 comp rx=0 tx=0).stop 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.030+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3816017755 shutdown_connections 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.030+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948d24e0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.030+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577945062f0 0x5577948d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.030+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/3816017755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55779452f1b0 0x5577948d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.030+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3816017755 >> 172.21.15.63:0/3816017755 conn(0x557794523300 msgr2=0x5577948ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.030+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3816017755 shutdown_connections 2024-05-27T07:55:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.030+0000 7f8c53400640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8c380b2950 0x7f8c380b4e60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0094e0 tx=0x7f8c4c009800 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.030+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3816017755 wait complete. 2024-05-27T07:55:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.031+0000 7f8c60bb9740 1 -- start start 2024-05-27T07:55:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.031+0000 7f8c60bb9740 1 Processor -- start 2024-05-27T07:55:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 0x5577948e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557794518790 con 0x55779450e4d0 2024-05-27T07:55:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577947dabc0 con 0x5577945062f0 2024-05-27T07:55:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5577948deb90 con 0x55779452f1b0 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c60bb9740 10 client.0 _ll_register_callbacks cb 0x5577947b8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c52000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c52000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948dc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51558/0 (socket says 172.21.15.63:51558) 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c52000640 1 -- 172.21.15.63:0/3726121287 learned_addr learned my addr 172.21.15.63:0/3726121287 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.032+0000 7f8c52a00640 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.033+0000 7f8c53400640 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 0x5577948e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.033+0000 7f8c52000640 1 -- 172.21.15.63:0/3726121287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 msgr2=0x5577948e7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.033+0000 7f8c52000640 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 0x5577948e7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.033+0000 7f8c52000640 1 -- 172.21.15.63:0/3726121287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 msgr2=0x5577948dbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.033+0000 7f8c52000640 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948dbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.033+0000 7f8c52000640 1 -- 172.21.15.63:0/3726121287 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557794547210 con 0x55779450e4d0 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.033+0000 7f8c52000640 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948dc110 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f8c40000c00 tx=0x7f8c40005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59748d207fe9cf40 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.033+0000 7f8c3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.034+0000 7f8c3cc00640 1 -- 172.21.15.63:0/3726121287 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40004640 con 0x55779450e4d0 2024-05-27T07:55:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.034+0000 7f8c3cc00640 1 -- 172.21.15.63:0/3726121287 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9571+0+0 (secure 0 0 0) 0x7f8c4000ed90 con 0x55779450e4d0 2024-05-27T07:55:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.034+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3726121287 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c400054b0 con 0x55779450e4d0 2024-05-27T07:55:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.034+0000 7f8c3cc00640 1 -- 172.21.15.63:0/3726121287 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c4001a460 con 0x55779450e4d0 2024-05-27T07:55:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.034+0000 7f8c60bb9740 10 client.19578 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.034+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3726121287 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557794613620 con 0x55779450e4d0 2024-05-27T07:55:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.035+0000 7f8c3cc00640 1 -- 172.21.15.63:0/3726121287 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c400054b0 con 0x55779450e4d0 2024-05-27T07:55:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.035+0000 7f8c3cc00640 1 -- 172.21.15.63:0/3726121287 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8c40010730 con 0x55779450e4d0 2024-05-27T07:55:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.035+0000 7f8c60bb9740 10 client.19578 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.035+0000 7f8c60bb9740 10 client.19578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.035+0000 7f8c60bb9740 20 client.19578 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.035+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5577944edd80 0x5577948edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.035+0000 7f8c60bb9740 4 client.19578 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.035+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3726121287 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557794844ae0 con 0x5577944edd80 2024-05-27T07:55:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.036+0000 7f8c52a00640 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5577944edd80 0x5577948edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.037+0000 7f8c52a00640 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5577944edd80 0x5577948edc50 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b48558286ab2eeb server_cookie=112059d80153d3a8 in_seq=0 out_seq=0 2024-05-27T07:55:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.037+0000 7f8c3cc00640 10 client.19578 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c3cc00640 1 -- 172.21.15.63:0/3726121287 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557794844ae0 con 0x5577944edd80 2024-05-27T07:55:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c3cc00640 10 client.19578 handle_command_reply: tid=0 2024-05-27T07:55:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 client.19578 shutdown 2024-05-27T07:55:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3726121287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 msgr2=0x5577948dc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948dc110 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f8c40000c00 tx=0x7f8c40005070 comp rx=0 tx=0).stop 2024-05-27T07:55:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3726121287 shutdown_connections 2024-05-27T07:55:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5577944edd80 0x5577948edc50 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 0x5577948e7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779450e4d0 0x5577948dc110 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/3726121287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948dbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3726121287 >> 172.21.15.63:0/3726121287 conn(0x557794523300 msgr2=0x5577948d3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c3cc00640 0 client.19578 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3726121287 shutdown_connections 2024-05-27T07:55:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.039+0000 7f8c60bb9740 1 -- 172.21.15.63:0/3726121287 wait complete. 2024-05-27T07:55:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.040+0000 7f8c60bb9740 20 client.19578 trim_cache size 0 max 16384 2024-05-27T07:55:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.061+0000 7f8c60bb9740 1 Processor -- start 2024-05-27T07:55:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.061+0000 7f8c60bb9740 1 -- start start 2024-05-27T07:55:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948dc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 0x5577948d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577944c31c0 0x5577948e7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b5bf0 con 0x5577944c31c0 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b34c0 con 0x5577945062f0 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b1330 con 0x55779452f1b0 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c53400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577944c31c0 0x5577948e7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c53400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577944c31c0 0x5577948e7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51574/0 (socket says 172.21.15.63:51574) 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c53400640 1 -- 172.21.15.63:0/718528510 learned_addr learned my addr 172.21.15.63:0/718528510 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c52a00640 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948dc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.062+0000 7f8c52000640 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 0x5577948d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.063+0000 7f8c53400640 1 -- 172.21.15.63:0/718528510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 msgr2=0x5577948d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.063+0000 7f8c53400640 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 0x5577948d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.063+0000 7f8c53400640 1 -- 172.21.15.63:0/718528510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 msgr2=0x5577948dc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.063+0000 7f8c53400640 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948dc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.063+0000 7f8c53400640 1 -- 172.21.15.63:0/718528510 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577948d17f0 con 0x5577944c31c0 2024-05-27T07:55:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.063+0000 7f8c52000640 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 0x5577948d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.063+0000 7f8c52a00640 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948dc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.063+0000 7f8c53400640 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577944c31c0 0x5577948e7c30 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02aea0 tx=0x7f8c4c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0993a21752ecfd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.064+0000 7f8c3f400640 1 -- 172.21.15.63:0/718528510 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c030b30 con 0x5577944c31c0 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.064+0000 7f8c3f400640 1 -- 172.21.15.63:0/718528510 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c4c036040 con 0x5577944c31c0 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.064+0000 7f8c3f400640 1 -- 172.21.15.63:0/718528510 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c052460 con 0x5577944c31c0 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.065+0000 7f8c60bb9740 1 -- 172.21.15.63:0/718528510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577944c31c0 msgr2=0x5577948e7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.065+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577944c31c0 0x5577948e7c30 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02aea0 tx=0x7f8c4c040000 comp rx=0 tx=0).stop 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.065+0000 7f8c60bb9740 1 -- 172.21.15.63:0/718528510 shutdown_connections 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.065+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577944c31c0 0x5577948e7c30 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.065+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55779452f1b0 0x5577948d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.065+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/718528510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948dc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.065+0000 7f8c60bb9740 1 -- 172.21.15.63:0/718528510 >> 172.21.15.63:0/718528510 conn(0x557794523300 msgr2=0x5577948d6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.065+0000 7f8c60bb9740 1 -- 172.21.15.63:0/718528510 shutdown_connections 2024-05-27T07:55:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.065+0000 7f8c60bb9740 1 -- 172.21.15.63:0/718528510 wait complete. 2024-05-27T07:55:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.066+0000 7f8c60bb9740 1 -- start start 2024-05-27T07:55:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.066+0000 7f8c60bb9740 1 Processor -- start 2024-05-27T07:55:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.067+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577944c31c0 0x5577948dede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.067+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948df320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.067+0000 7f8c60bb9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779452f1b0 0x5577948df860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.067+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b1980 con 0x55779452f1b0 2024-05-27T07:55:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.067+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b2df0 con 0x5577945062f0 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.067+0000 7f8c60bb9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b2050 con 0x5577944c31c0 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.067+0000 7f8c60bb9740 10 client.0 _ll_register_callbacks cb 0x7f8c40014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.067+0000 7f8c53400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779452f1b0 0x5577948df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.067+0000 7f8c52000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948df320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.068+0000 7f8c52000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948df320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51224/0 (socket says 172.21.15.63:51224) 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.068+0000 7f8c53400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779452f1b0 0x5577948df860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51584/0 (socket says 172.21.15.63:51584) 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.068+0000 7f8c52000640 1 -- 172.21.15.63:0/1030098672 learned_addr learned my addr 172.21.15.63:0/1030098672 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.068+0000 7f8c52a00640 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577944c31c0 0x5577948dede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.068+0000 7f8c53400640 1 -- 172.21.15.63:0/1030098672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577944c31c0 msgr2=0x5577948dede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.068+0000 7f8c53400640 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577944c31c0 0x5577948dede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.068+0000 7f8c53400640 1 -- 172.21.15.63:0/1030098672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 msgr2=0x5577948df320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.068+0000 7f8c53400640 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948df320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.068+0000 7f8c53400640 1 -- 172.21.15.63:0/1030098672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55779495c950 con 0x55779452f1b0 2024-05-27T07:55:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.069+0000 7f8c53400640 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779452f1b0 0x5577948df860 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006010 tx=0x7f8c4c030620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42008deb5d0178a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.069+0000 7f8c3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.070+0000 7f8c3cc00640 1 -- 172.21.15.63:0/1030098672 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c003970 con 0x55779452f1b0 2024-05-27T07:55:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.070+0000 7f8c3cc00640 1 -- 172.21.15.63:0/1030098672 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9571+0+0 (secure 0 0 0) 0x7f8c4c003b10 con 0x55779452f1b0 2024-05-27T07:55:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.070+0000 7f8c60bb9740 1 -- 172.21.15.63:0/1030098672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c036040 con 0x55779452f1b0 2024-05-27T07:55:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.070+0000 7f8c3cc00640 1 -- 172.21.15.63:0/1030098672 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c4c003d10 con 0x55779452f1b0 2024-05-27T07:55:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.070+0000 7f8c60bb9740 10 client.19584 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.070+0000 7f8c60bb9740 1 -- 172.21.15.63:0/1030098672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55779495da80 con 0x55779452f1b0 2024-05-27T07:55:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.071+0000 7f8c3cc00640 1 -- 172.21.15.63:0/1030098672 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c4c036040 con 0x55779452f1b0 2024-05-27T07:55:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.071+0000 7f8c3cc00640 1 -- 172.21.15.63:0/1030098672 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f8c4c044040 con 0x55779452f1b0 2024-05-27T07:55:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.071+0000 7f8c60bb9740 10 client.19584 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.071+0000 7f8c60bb9740 10 client.19584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.071+0000 7f8c60bb9740 20 client.19584 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.071+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55779495e0e0 0x557794960500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.071+0000 7f8c60bb9740 4 client.19584 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:55:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.071+0000 7f8c60bb9740 1 -- 172.21.15.63:0/1030098672 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55779483e2c0 con 0x55779495e0e0 2024-05-27T07:55:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.072+0000 7f8c52a00640 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55779495e0e0 0x557794960500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.073+0000 7f8c52a00640 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55779495e0e0 0x557794960500 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e3edccf69402f5 server_cookie=777df5447eecb61d in_seq=0 out_seq=0 2024-05-27T07:55:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.073+0000 7f8c3cc00640 10 client.19584 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c3cc00640 1 -- 172.21.15.63:0/1030098672 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55779483e2c0 con 0x55779495e0e0 2024-05-27T07:55:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c3cc00640 10 client.19584 handle_command_reply: tid=0 2024-05-27T07:55:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 client.19584 shutdown 2024-05-27T07:55:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 -- 172.21.15.63:0/1030098672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779452f1b0 msgr2=0x5577948df860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779452f1b0 0x5577948df860 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006010 tx=0x7f8c4c030620 comp rx=0 tx=0).stop 2024-05-27T07:55:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 -- 172.21.15.63:0/1030098672 shutdown_connections 2024-05-27T07:55:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55779495e0e0 0x557794960500 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55779452f1b0 0x5577948df860 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577945062f0 0x5577948df320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 --2- 172.21.15.63:0/1030098672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577944c31c0 0x5577948dede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 -- 172.21.15.63:0/1030098672 >> 172.21.15.63:0/1030098672 conn(0x557794523300 msgr2=0x5577948d7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c3cc00640 0 client.19584 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 -- 172.21.15.63:0/1030098672 shutdown_connections 2024-05-27T07:55:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.114+0000 7f8c60bb9740 1 -- 172.21.15.63:0/1030098672 wait complete. 2024-05-27T07:55:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.115+0000 7f8c60bb9740 20 client.19584 trim_cache size 0 max 16384 2024-05-27T07:55:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:55:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:55:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:55:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.115+0000 7f8c53e00640 1 -- 172.21.15.63:0/1600400621 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8c380b2950 msgr2=0x7f8c380b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.115+0000 7f8c53e00640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8c380b2950 0x7f8c380b4e60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0094e0 tx=0x7f8c4c009800 comp rx=0 tx=0).stop 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.115+0000 7f8c53e00640 1 -- 172.21.15.63:0/1600400621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c54104250 msgr2=0x7f8c54074fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.116+0000 7f8c53e00640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c54104250 0x7f8c54074fc0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400d770 tx=0x7f8c4400dc30 comp rx=0 tx=0).stop 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.116+0000 7f8c53e00640 1 -- 172.21.15.63:0/1600400621 shutdown_connections 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.116+0000 7f8c53e00640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8c380b2950 0x7f8c380b4e60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.116+0000 7f8c53e00640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c54109f50 0x7f8c54075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.116+0000 7f8c53e00640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c54109ab0 0x7f8c54075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.116+0000 7f8c53e00640 1 --2- 172.21.15.63:0/1600400621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c54104250 0x7f8c54074fc0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.116+0000 7f8c53e00640 1 -- 172.21.15.63:0/1600400621 >> 172.21.15.63:0/1600400621 conn(0x7f8c54079240 msgr2=0x7f8c54100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.116+0000 7f8c53e00640 1 -- 172.21.15.63:0/1600400621 shutdown_connections 2024-05-27T07:55:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:06.116+0000 7f8c53e00640 1 -- 172.21.15.63:0/1600400621 wait complete. 2024-05-27T07:55:07.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:07 smithi063 ceph-mon[31613]: pgmap v680: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 33 op/s 2024-05-27T07:55:07.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:07 smithi086 ceph-mon[34687]: pgmap v680: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 33 op/s 2024-05-27T07:55:07.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:07 smithi149 ceph-mon[35408]: pgmap v680: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 33 op/s 2024-05-27T07:55:08.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.101+0000 7f240da00700 1 -- 172.21.15.63:0/3316736870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2408105dc0 msgr2=0x7f240810e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:08.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.101+0000 7f240da00700 1 --2- 172.21.15.63:0/3316736870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2408105dc0 0x7f240810e310 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009990 tx=0x7f23f0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.101+0000 7f240da00700 1 -- 172.21.15.63:0/3316736870 shutdown_connections 2024-05-27T07:55:08.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.101+0000 7f240da00700 1 --2- 172.21.15.63:0/3316736870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f240810e8e0 0x7f2408110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.101+0000 7f240da00700 1 --2- 172.21.15.63:0/3316736870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408105470 0x7f2408105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.101+0000 7f240da00700 1 --2- 172.21.15.63:0/3316736870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2408105dc0 0x7f240810e310 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.101+0000 7f240da00700 1 -- 172.21.15.63:0/3316736870 >> 172.21.15.63:0/3316736870 conn(0x7f24080fef20 msgr2=0x7f2408101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:08.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.101+0000 7f240da00700 1 -- 172.21.15.63:0/3316736870 shutdown_connections 2024-05-27T07:55:08.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.102+0000 7f240da00700 1 -- 172.21.15.63:0/3316736870 wait complete. 2024-05-27T07:55:08.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.103+0000 7f240da00700 1 Processor -- start 2024-05-27T07:55:08.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.104+0000 7f240da00700 1 -- start start 2024-05-27T07:55:08.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.104+0000 7f240da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2408105470 0x7f24081a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.104+0000 7f240da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408105dc0 0x7f24081a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.104+0000 7f240da00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f240810e8e0 0x7f240819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.104+0000 7f240da00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24081142f0 con 0x7f2408105470 2024-05-27T07:55:08.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.104+0000 7f240da00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2408114170 con 0x7f2408105dc0 2024-05-27T07:55:08.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.104+0000 7f240da00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2408114470 con 0x7f240810e8e0 2024-05-27T07:55:08.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.105+0000 7f2406a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2408105470 0x7f24081a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.105+0000 7f23fe000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408105dc0 0x7f24081a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.105+0000 7f23fe000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408105dc0 0x7f24081a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51246/0 (socket says 172.21.15.63:51246) 2024-05-27T07:55:08.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.105+0000 7f23fe000700 1 -- 172.21.15.63:0/4253816136 learned_addr learned my addr 172.21.15.63:0/4253816136 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:08.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.105+0000 7f2407400700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f240810e8e0 0x7f240819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.106+0000 7f2406a00700 1 -- 172.21.15.63:0/4253816136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f240810e8e0 msgr2=0x7f240819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:08.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.106+0000 7f2406a00700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f240810e8e0 0x7f240819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.106+0000 7f2406a00700 1 -- 172.21.15.63:0/4253816136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408105dc0 msgr2=0x7f24081a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:08.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.106+0000 7f2406a00700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408105dc0 0x7f24081a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.106+0000 7f2406a00700 1 -- 172.21.15.63:0/4253816136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240819b260 con 0x7f2408105470 2024-05-27T07:55:08.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.106+0000 7f2406a00700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2408105470 0x7f24081a09a0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f23f800d950 tx=0x7f23f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:08.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.107+0000 7f23ffe00700 1 -- 172.21.15.63:0/4253816136 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8017070 con 0x7f2408105470 2024-05-27T07:55:08.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.107+0000 7f23ffe00700 1 -- 172.21.15.63:0/4253816136 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23f8009dc0 con 0x7f2408105470 2024-05-27T07:55:08.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.107+0000 7f240da00700 1 -- 172.21.15.63:0/4253816136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240819b550 con 0x7f2408105470 2024-05-27T07:55:08.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.107+0000 7f240da00700 1 -- 172.21.15.63:0/4253816136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2408103090 con 0x7f2408105470 2024-05-27T07:55:08.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.107+0000 7f23ffe00700 1 -- 172.21.15.63:0/4253816136 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8010cd0 con 0x7f2408105470 2024-05-27T07:55:08.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.112+0000 7f23ffe00700 1 -- 172.21.15.63:0/4253816136 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f23f801a070 con 0x7f2408105470 2024-05-27T07:55:08.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.114+0000 7f23ffe00700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23ec0b29f0 0x7f23ec0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.114+0000 7f241c59db80 1 Processor -- start 2024-05-27T07:55:08.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.115+0000 7f23ffe00700 1 -- 172.21.15.63:0/4253816136 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9571+0+0 (secure 0 0 0) 0x7f23f80e2360 con 0x7f2408105470 2024-05-27T07:55:08.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.115+0000 7f23fe000700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23ec0b29f0 0x7f23ec0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.115+0000 7f241c59db80 1 -- start start 2024-05-27T07:55:08.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.116+0000 7f241c59db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 0x55568c266330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.116+0000 7f241c59db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 0x55568c266870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.116+0000 7f241c59db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c273110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.116+0000 7f241c59db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55568c194e40 con 0x55568c1d39f0 2024-05-27T07:55:08.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.116+0000 7f241c59db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55568c1849e0 con 0x55568c266e70 2024-05-27T07:55:08.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.116+0000 7f241c59db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55568c188a20 con 0x55568c0e7e40 2024-05-27T07:55:08.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.116+0000 7f23fe000700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23ec0b29f0 0x7f23ec0b4f00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009750 tx=0x7f23f0009ee0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:08.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.117+0000 7f23fe000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 0x55568c266870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.117+0000 7f23fe000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 0x55568c266870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:39562/0 (socket says 172.21.15.63:39562) 2024-05-27T07:55:08.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.117+0000 7f23fe000700 1 -- 172.21.15.63:0/2273659683 learned_addr learned my addr 172.21.15.63:0/2273659683 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:08.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.118+0000 7f2407400700 1 --2- 172.21.15.63:0/2273659683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c273110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.118+0000 7f2406a00700 1 --2- 172.21.15.63:0/2273659683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 0x55568c266330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.119+0000 7f23fe000700 1 -- 172.21.15.63:0/2273659683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 msgr2=0x55568c273110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:08.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.119+0000 7f23fe000700 1 --2- 172.21.15.63:0/2273659683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c273110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.119+0000 7f23fe000700 1 -- 172.21.15.63:0/2273659683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 msgr2=0x55568c266330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:08.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.119+0000 7f23fe000700 1 --2- 172.21.15.63:0/2273659683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 0x55568c266330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.119+0000 7f23fe000700 1 -- 172.21.15.63:0/2273659683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55568c14d420 con 0x55568c0e7e40 2024-05-27T07:55:08.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.119+0000 7f2407400700 1 --2- 172.21.15.63:0/2273659683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c273110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:08.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.119+0000 7f23fe000700 1 --2- 172.21.15.63:0/2273659683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 0x55568c266870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2408102540 tx=0x7f23f001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90bcfbc68ba3c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:08.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.120+0000 7f23f7e00700 1 -- 172.21.15.63:0/2273659683 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f000bb60 con 0x55568c0e7e40 2024-05-27T07:55:08.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.121+0000 7f23f7e00700 1 -- 172.21.15.63:0/2273659683 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23f000bd00 con 0x55568c0e7e40 2024-05-27T07:55:08.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.121+0000 7f23f7e00700 1 -- 172.21.15.63:0/2273659683 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f000f7e0 con 0x55568c0e7e40 2024-05-27T07:55:08.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.122+0000 7f241c59db80 1 -- 172.21.15.63:0/2273659683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 msgr2=0x55568c266870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:08.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.122+0000 7f241c59db80 1 --2- 172.21.15.63:0/2273659683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 0x55568c266870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2408102540 tx=0x7f23f001e000 comp rx=0 tx=0).stop 2024-05-27T07:55:08.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.122+0000 7f241c59db80 1 -- 172.21.15.63:0/2273659683 shutdown_connections 2024-05-27T07:55:08.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.122+0000 7f241c59db80 1 --2- 172.21.15.63:0/2273659683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c273110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.122+0000 7f241c59db80 1 --2- 172.21.15.63:0/2273659683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 0x55568c266330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.123+0000 7f241c59db80 1 --2- 172.21.15.63:0/2273659683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 0x55568c266870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.123+0000 7f241c59db80 1 -- 172.21.15.63:0/2273659683 >> 172.21.15.63:0/2273659683 conn(0x55568c181390 msgr2=0x55568c2624c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:08.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.123+0000 7f241c59db80 1 -- 172.21.15.63:0/2273659683 shutdown_connections 2024-05-27T07:55:08.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.123+0000 7f241c59db80 1 -- 172.21.15.63:0/2273659683 wait complete. 2024-05-27T07:55:08.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.124+0000 7f241c59db80 1 -- start start 2024-05-27T07:55:08.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.124+0000 7f241c59db80 1 Processor -- start 2024-05-27T07:55:08.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f241c59db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c0e7e40 0x55568c270650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f241c59db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 0x55568c270b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f241c59db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c2710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f241c59db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55568c08afa0 con 0x55568c0e7e40 2024-05-27T07:55:08.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f241c59db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55568c125760 con 0x55568c266e70 2024-05-27T07:55:08.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f241c59db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55568c273ac0 con 0x55568c1d39f0 2024-05-27T07:55:08.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f241c59db80 10 client.0 _ll_register_callbacks cb 0x55568c12ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:08.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f2406a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c0e7e40 0x55568c270650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f2406a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c0e7e40 0x55568c270650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51612/0 (socket says 172.21.15.63:51612) 2024-05-27T07:55:08.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f2406a00700 1 -- 172.21.15.63:0/2245913265 learned_addr learned my addr 172.21.15.63:0/2245913265 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:08.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f23fe000700 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 0x55568c270b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.125+0000 7f2407400700 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c2710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.126+0000 7f2407400700 1 -- 172.21.15.63:0/2245913265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 msgr2=0x55568c270b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:08.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.126+0000 7f2407400700 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 0x55568c270b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.126+0000 7f2407400700 1 -- 172.21.15.63:0/2245913265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c0e7e40 msgr2=0x55568c270650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:08.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.127+0000 7f2407400700 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c0e7e40 0x55568c270650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.127+0000 7f2407400700 1 -- 172.21.15.63:0/2245913265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55568c0d48d0 con 0x55568c266e70 2024-05-27T07:55:08.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.127+0000 7f2406a00700 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c0e7e40 0x55568c270650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:08.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.128+0000 7f2407400700 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c2710d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2400007b20 tx=0x7f2400007e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a83fc2d1470476 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:08.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.128+0000 7f23f4c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:55:08.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.128+0000 7f23f4c00700 1 -- 172.21.15.63:0/2245913265 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240001a070 con 0x55568c266e70 2024-05-27T07:55:08.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.128+0000 7f241c59db80 1 -- 172.21.15.63:0/2245913265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23f000bd00 con 0x55568c266e70 2024-05-27T07:55:08.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.128+0000 7f23f4c00700 1 -- 172.21.15.63:0/2245913265 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9571+0+0 (secure 0 0 0) 0x7f2400004db0 con 0x55568c266e70 2024-05-27T07:55:08.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.129+0000 7f23f4c00700 1 -- 172.21.15.63:0/2245913265 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2400025050 con 0x55568c266e70 2024-05-27T07:55:08.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.129+0000 7f241c59db80 10 client.38846 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:08.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.129+0000 7f241c59db80 1 -- 172.21.15.63:0/2245913265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55568c155060 con 0x55568c266e70 2024-05-27T07:55:08.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.129+0000 7f23f4c00700 1 -- 172.21.15.63:0/2245913265 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23f000bd00 con 0x55568c266e70 2024-05-27T07:55:08.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.130+0000 7f23f4c00700 1 -- 172.21.15.63:0/2245913265 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2400005520 con 0x55568c266e70 2024-05-27T07:55:08.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.130+0000 7f241c59db80 10 client.38846 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:08.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.130+0000 7f241c59db80 10 client.38846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:08.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.130+0000 7f241c59db80 20 client.38846 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:08.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.130+0000 7f241c59db80 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55568c280a70 0x55568c282f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.130+0000 7f241c59db80 4 client.38846 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:08.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.130+0000 7f241c59db80 1 -- 172.21.15.63:0/2245913265 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55568c196d90 con 0x55568c280a70 2024-05-27T07:55:08.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.131+0000 7f2406a00700 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55568c280a70 0x55568c282f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.132+0000 7f2406a00700 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55568c280a70 0x55568c282f10 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af8a60d88d945b8 server_cookie=7a9b8c9209e71e36 in_seq=0 out_seq=0 2024-05-27T07:55:08.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.132+0000 7f23f4c00700 10 client.38846 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:08.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.133+0000 7f23f4c00700 1 -- 172.21.15.63:0/2245913265 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55568c196d90 con 0x55568c280a70 2024-05-27T07:55:08.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.133+0000 7f23f4c00700 10 client.38846 handle_command_reply: tid=0 2024-05-27T07:55:08.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 client.38846 shutdown 2024-05-27T07:55:08.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 -- 172.21.15.63:0/2245913265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 msgr2=0x55568c2710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:08.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c2710d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2400007b20 tx=0x7f2400007e30 comp rx=0 tx=0).stop 2024-05-27T07:55:08.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 -- 172.21.15.63:0/2245913265 shutdown_connections 2024-05-27T07:55:08.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55568c280a70 0x55568c282f10 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c2710d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c0e7e40 0x55568c270650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 --2- 172.21.15.63:0/2245913265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 0x55568c270b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 -- 172.21.15.63:0/2245913265 >> 172.21.15.63:0/2245913265 conn(0x55568c181390 msgr2=0x55568c266310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:08.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f23f4c00700 0 client.38846 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:08.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 -- 172.21.15.63:0/2245913265 shutdown_connections 2024-05-27T07:55:08.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 1 -- 172.21.15.63:0/2245913265 wait complete. 2024-05-27T07:55:08.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.134+0000 7f241c59db80 20 client.38846 trim_cache size 0 max 16384 2024-05-27T07:55:08.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.159+0000 7f241c59db80 1 Processor -- start 2024-05-27T07:55:08.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.159+0000 7f241c59db80 1 -- start start 2024-05-27T07:55:08.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.159+0000 7f241c59db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 0x55568c26a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.159+0000 7f241c59db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 0x55568c27cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.159+0000 7f241c59db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c288760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.159+0000 7f241c59db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f23e40116e0 con 0x55568c1d39f0 2024-05-27T07:55:08.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.159+0000 7f241c59db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23e400ef70 con 0x55568c266e70 2024-05-27T07:55:08.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.159+0000 7f241c59db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23e400cde0 con 0x55568c0e7e40 2024-05-27T07:55:08.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.160+0000 7f23fe000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 0x55568c27cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.160+0000 7f23fe000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 0x55568c27cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51616/0 (socket says 172.21.15.63:51616) 2024-05-27T07:55:08.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.160+0000 7f23fe000700 1 -- 172.21.15.63:0/3530354746 learned_addr learned my addr 172.21.15.63:0/3530354746 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:08.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.160+0000 7f2407400700 1 --2- 172.21.15.63:0/3530354746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c288760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.160+0000 7f2406a00700 1 --2- 172.21.15.63:0/3530354746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 0x55568c26a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.160+0000 7f23fe000700 1 -- 172.21.15.63:0/3530354746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 msgr2=0x55568c26a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:08.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.160+0000 7f23fe000700 1 --2- 172.21.15.63:0/3530354746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 0x55568c26a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.160+0000 7f23fe000700 1 -- 172.21.15.63:0/3530354746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 msgr2=0x55568c288760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:08.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.160+0000 7f23fe000700 1 --2- 172.21.15.63:0/3530354746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c288760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.160+0000 7f23fe000700 1 -- 172.21.15.63:0/3530354746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55568c25e240 con 0x55568c1d39f0 2024-05-27T07:55:08.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.161+0000 7f23fe000700 1 --2- 172.21.15.63:0/3530354746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 0x55568c27cbc0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f23f000bab0 tx=0x7f23f0005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aca514b8616e885 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:08.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.161+0000 7f23f7400700 1 -- 172.21.15.63:0/3530354746 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f000fae0 con 0x55568c1d39f0 2024-05-27T07:55:08.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.161+0000 7f23f7400700 1 -- 172.21.15.63:0/3530354746 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23f000fc80 con 0x55568c1d39f0 2024-05-27T07:55:08.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.161+0000 7f23f7400700 1 -- 172.21.15.63:0/3530354746 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0003b90 con 0x55568c1d39f0 2024-05-27T07:55:08.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.162+0000 7f241c59db80 1 -- 172.21.15.63:0/3530354746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 msgr2=0x55568c27cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:08.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.162+0000 7f241c59db80 1 --2- 172.21.15.63:0/3530354746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 0x55568c27cbc0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f23f000bab0 tx=0x7f23f0005ee0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.163+0000 7f241c59db80 1 -- 172.21.15.63:0/3530354746 shutdown_connections 2024-05-27T07:55:08.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.163+0000 7f241c59db80 1 --2- 172.21.15.63:0/3530354746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c266e70 0x55568c288760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.163+0000 7f241c59db80 1 --2- 172.21.15.63:0/3530354746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c0e7e40 0x55568c26a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.163+0000 7f241c59db80 1 --2- 172.21.15.63:0/3530354746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c1d39f0 0x55568c27cbc0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.163+0000 7f241c59db80 1 -- 172.21.15.63:0/3530354746 >> 172.21.15.63:0/3530354746 conn(0x55568c181390 msgr2=0x55568c269ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:08.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.163+0000 7f241c59db80 1 -- 172.21.15.63:0/3530354746 shutdown_connections 2024-05-27T07:55:08.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.163+0000 7f241c59db80 1 -- 172.21.15.63:0/3530354746 wait complete. 2024-05-27T07:55:08.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.163+0000 7f241c59db80 1 -- start start 2024-05-27T07:55:08.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.164+0000 7f241c59db80 1 Processor -- start 2024-05-27T07:55:08.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.164+0000 7f241c59db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c0e7e40 0x55568c2790f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.164+0000 7f241c59db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 0x55568c279630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.164+0000 7f241c59db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c266e70 0x55568c274560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.164+0000 7f241c59db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f23e400d430 con 0x55568c266e70 2024-05-27T07:55:08.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.164+0000 7f241c59db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23e400e8a0 con 0x55568c0e7e40 2024-05-27T07:55:08.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.164+0000 7f241c59db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23e400db00 con 0x55568c1d39f0 2024-05-27T07:55:08.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.164+0000 7f241c59db80 10 client.0 _ll_register_callbacks cb 0x55568c12ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:08.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f2407400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c266e70 0x55568c274560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f2407400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c266e70 0x55568c274560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51622/0 (socket says 172.21.15.63:51622) 2024-05-27T07:55:08.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f2407400700 1 -- 172.21.15.63:0/5231599 learned_addr learned my addr 172.21.15.63:0/5231599 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:08.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f2406a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c0e7e40 0x55568c2790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f23fe000700 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 0x55568c279630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f2407400700 1 -- 172.21.15.63:0/5231599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 msgr2=0x55568c279630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:08.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f2407400700 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 0x55568c279630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f2407400700 1 -- 172.21.15.63:0/5231599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c0e7e40 msgr2=0x55568c2790f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:08.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f2407400700 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c0e7e40 0x55568c2790f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f2407400700 1 -- 172.21.15.63:0/5231599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55568c274d00 con 0x55568c266e70 2024-05-27T07:55:08.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.165+0000 7f23fe000700 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 0x55568c279630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:08.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.166+0000 7f2407400700 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c266e70 0x55568c274560 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2400000f80 tx=0x7f2400012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800262c1b93bdf48 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:08.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.166+0000 7f23f4c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:08.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.166+0000 7f23f4c00700 1 -- 172.21.15.63:0/5231599 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240002d020 con 0x55568c266e70 2024-05-27T07:55:08.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.166+0000 7f23f4c00700 1 -- 172.21.15.63:0/5231599 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9571+0+0 (secure 0 0 0) 0x7f2400005fe0 con 0x55568c266e70 2024-05-27T07:55:08.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.166+0000 7f241c59db80 1 -- 172.21.15.63:0/5231599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23f000fc80 con 0x55568c266e70 2024-05-27T07:55:08.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.167+0000 7f23f4c00700 1 -- 172.21.15.63:0/5231599 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2400038050 con 0x55568c266e70 2024-05-27T07:55:08.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.167+0000 7f241c59db80 10 client.19602 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:08.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.167+0000 7f241c59db80 1 -- 172.21.15.63:0/5231599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55568c2eb160 con 0x55568c266e70 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.167+0000 7f23f4c00700 1 -- 172.21.15.63:0/5231599 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23f000fc80 con 0x55568c266e70 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.167+0000 7f23f4c00700 1 -- 172.21.15.63:0/5231599 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2400016540 con 0x55568c266e70 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.167+0000 7f241c59db80 10 client.19602 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.168+0000 7f241c59db80 10 client.19602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.168+0000 7f241c59db80 20 client.19602 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.168+0000 7f241c59db80 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55568c2eb830 0x55568c2edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.168+0000 7f241c59db80 4 client.19602 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.168+0000 7f241c59db80 1 -- 172.21.15.63:0/5231599 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55568c190570 con 0x55568c2eb830 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.168+0000 7f2406a00700 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55568c2eb830 0x55568c2edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.169+0000 7f23f4c00700 10 client.19602 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:08.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.169+0000 7f2406a00700 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55568c2eb830 0x55568c2edc50 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e57d03fb4f72c1 server_cookie=3b1a0942e844b64a in_seq=0 out_seq=0 2024-05-27T07:55:08.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f23f4c00700 1 -- 172.21.15.63:0/5231599 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55568c190570 con 0x55568c2eb830 2024-05-27T07:55:08.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f23f4c00700 10 client.19602 handle_command_reply: tid=0 2024-05-27T07:55:08.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 client.19602 shutdown 2024-05-27T07:55:08.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 -- 172.21.15.63:0/5231599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c266e70 msgr2=0x55568c274560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:08.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c266e70 0x55568c274560 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2400000f80 tx=0x7f2400012ea0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 -- 172.21.15.63:0/5231599 shutdown_connections 2024-05-27T07:55:08.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55568c2eb830 0x55568c2edc50 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55568c266e70 0x55568c274560 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55568c0e7e40 0x55568c2790f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f23f4c00700 0 client.19602 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:08.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 --2- 172.21.15.63:0/5231599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55568c1d39f0 0x55568c279630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 -- 172.21.15.63:0/5231599 >> 172.21.15.63:0/5231599 conn(0x55568c181390 msgr2=0x55568c270f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:08.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 -- 172.21.15.63:0/5231599 shutdown_connections 2024-05-27T07:55:08.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 1 -- 172.21.15.63:0/5231599 wait complete. 2024-05-27T07:55:08.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.170+0000 7f241c59db80 20 client.19602 trim_cache size 0 max 16384 2024-05-27T07:55:08.174 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:55:08.174 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:55:08.174 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:55:08.175 INFO:teuthology.orchestra.run.smithi063.stdout: "4913ce03-ada1-4555-8206-97b111286bd4": { 2024-05-27T07:55:08.175 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:55:08.175 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "4913ce03-ada1-4555-8206-97b111286bd4", 2024-05-27T07:55:08.175 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:55:08.175 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:55:08.175 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:55:08.175 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:55:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.171+0000 7f240da00700 1 -- 172.21.15.63:0/4253816136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23ec0b29f0 msgr2=0x7f23ec0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.171+0000 7f240da00700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23ec0b29f0 0x7f23ec0b4f00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009750 tx=0x7f23f0009ee0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.171+0000 7f240da00700 1 -- 172.21.15.63:0/4253816136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2408105470 msgr2=0x7f24081a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.171+0000 7f240da00700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2408105470 0x7f24081a09a0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f23f800d950 tx=0x7f23f800dc60 comp rx=0 tx=0).stop 2024-05-27T07:55:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.171+0000 7f240da00700 1 -- 172.21.15.63:0/4253816136 shutdown_connections 2024-05-27T07:55:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.171+0000 7f240da00700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f23ec0b29f0 0x7f23ec0b4f00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.172+0000 7f240da00700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f240810e8e0 0x7f240819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.172+0000 7f240da00700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2408105470 0x7f24081a09a0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.172+0000 7f240da00700 1 --2- 172.21.15.63:0/4253816136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408105dc0 0x7f24081a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.172+0000 7f240da00700 1 -- 172.21.15.63:0/4253816136 >> 172.21.15.63:0/4253816136 conn(0x7f24080fef20 msgr2=0x7f24080ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.172+0000 7f240da00700 1 -- 172.21.15.63:0/4253816136 shutdown_connections 2024-05-27T07:55:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:08.172+0000 7f240da00700 1 -- 172.21.15.63:0/4253816136 wait complete. 2024-05-27T07:55:09.094 INFO:tasks.cephfs.filesystem:scrub status for tag:4913ce03-ada1-4555-8206-97b111286bd4 - {'path': '/', 'tag': '4913ce03-ada1-4555-8206-97b111286bd4', 'options': 'recursive,force'} 2024-05-27T07:55:09.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:09 smithi086 ceph-mon[34687]: pgmap v681: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.9 MiB/s wr, 69 op/s 2024-05-27T07:55:09.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:09 smithi149 ceph-mon[35408]: pgmap v681: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.9 MiB/s wr, 69 op/s 2024-05-27T07:55:09.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:09 smithi063 ceph-mon[31613]: pgmap v681: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.9 MiB/s wr, 69 op/s 2024-05-27T07:55:11.593 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:11 smithi149 ceph-mon[35408]: pgmap v682: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.9 MiB/s wr, 69 op/s 2024-05-27T07:55:11.594 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:11 smithi149 ceph-mon[35408]: osdmap e125: 12 total, 12 up, 12 in 2024-05-27T07:55:11.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:11 smithi086 ceph-mon[34687]: pgmap v682: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.9 MiB/s wr, 69 op/s 2024-05-27T07:55:11.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:11 smithi086 ceph-mon[34687]: osdmap e125: 12 total, 12 up, 12 in 2024-05-27T07:55:11.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:11 smithi063 ceph-mon[31613]: pgmap v682: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.9 MiB/s wr, 69 op/s 2024-05-27T07:55:11.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:11 smithi063 ceph-mon[31613]: osdmap e125: 12 total, 12 up, 12 in 2024-05-27T07:55:12.635 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:55:12.635 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:55:12.636 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:55:12.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:55:12.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:55:12.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:55:12.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:55:12.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:55:12.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:55:13.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:13 smithi086 ceph-mon[34687]: pgmap v684: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.0 MiB/s wr, 49 op/s 2024-05-27T07:55:13.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:13 smithi149 ceph-mon[35408]: pgmap v684: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.0 MiB/s wr, 49 op/s 2024-05-27T07:55:13.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:13 smithi063 ceph-mon[31613]: pgmap v684: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.0 MiB/s wr, 49 op/s 2024-05-27T07:55:15.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:15 smithi086 ceph-mon[34687]: pgmap v685: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 48 op/s 2024-05-27T07:55:15.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:15 smithi149 ceph-mon[35408]: pgmap v685: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 48 op/s 2024-05-27T07:55:15.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:15 smithi063 ceph-mon[31613]: pgmap v685: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 48 op/s 2024-05-27T07:55:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:17 smithi086 ceph-mon[34687]: pgmap v686: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 48 op/s 2024-05-27T07:55:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:17 smithi149 ceph-mon[35408]: pgmap v686: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 48 op/s 2024-05-27T07:55:17.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:17 smithi063 ceph-mon[31613]: pgmap v686: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 48 op/s 2024-05-27T07:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.366+0000 7f3378e00640 1 -- 172.21.15.63:0/700743607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f337410f370 msgr2=0x7f3374116b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.366+0000 7f3378e00640 1 --2- 172.21.15.63:0/700743607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f337410f370 0x7f3374116b10 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f336c009ca0 tx=0x7f336c02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.366+0000 7f3378e00640 1 -- 172.21.15.63:0/700743607 shutdown_connections 2024-05-27T07:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.366+0000 7f3378e00640 1 --2- 172.21.15.63:0/700743607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f337410f370 0x7f3374116b10 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.366+0000 7f3378e00640 1 --2- 172.21.15.63:0/700743607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f337410e8b0 0x7f337410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.366+0000 7f3378e00640 1 --2- 172.21.15.63:0/700743607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337410a320 0x7f337410a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.366+0000 7f3378e00640 1 -- 172.21.15.63:0/700743607 >> 172.21.15.63:0/700743607 conn(0x7f3374076070 msgr2=0x7f3374076480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.366+0000 7f3378e00640 1 -- 172.21.15.63:0/700743607 shutdown_connections 2024-05-27T07:55:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.366+0000 7f3378e00640 1 -- 172.21.15.63:0/700743607 wait complete. 2024-05-27T07:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.367+0000 7f3378e00640 1 Processor -- start 2024-05-27T07:55:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.368+0000 7f3378e00640 1 -- start start 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.368+0000 7f3378e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f337410a320 0x7f33741a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.368+0000 7f3378e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337410e8b0 0x7f33741a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.368+0000 7f3378e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f337410f370 0x7f33741a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.368+0000 7f3378e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f337411a180 con 0x7f337410a320 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.368+0000 7f3378e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f337411a000 con 0x7f337410e8b0 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.368+0000 7f3378e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f337411a300 con 0x7f337410f370 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.368+0000 7f3373400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f337410a320 0x7f33741a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.368+0000 7f3373400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f337410a320 0x7f33741a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51394/0 (socket says 172.21.15.63:51394) 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.368+0000 7f3373400640 1 -- 172.21.15.63:0/4093658204 learned_addr learned my addr 172.21.15.63:0/4093658204 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.369+0000 7f3372a00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337410e8b0 0x7f33741a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.369+0000 7f3373e00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f337410f370 0x7f33741a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.369+0000 7f3373400640 1 -- 172.21.15.63:0/4093658204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f337410f370 msgr2=0x7f33741a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.369+0000 7f3373400640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f337410f370 0x7f33741a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.369+0000 7f3373400640 1 -- 172.21.15.63:0/4093658204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337410e8b0 msgr2=0x7f33741a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.369+0000 7f3373400640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337410e8b0 0x7f33741a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.369+0000 7f3373400640 1 -- 172.21.15.63:0/4093658204 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33741a1060 con 0x7f337410a320 2024-05-27T07:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.370+0000 7f3373400640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f337410a320 0x7f33741a67b0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f336400d950 tx=0x7f336400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.370+0000 7f336be00640 1 -- 172.21.15.63:0/4093658204 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364017030 con 0x7f337410a320 2024-05-27T07:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.370+0000 7f336be00640 1 -- 172.21.15.63:0/4093658204 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3364004020 con 0x7f337410a320 2024-05-27T07:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.370+0000 7f336be00640 1 -- 172.21.15.63:0/4093658204 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33640055d0 con 0x7f337410a320 2024-05-27T07:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.370+0000 7f3378e00640 1 -- 172.21.15.63:0/4093658204 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33741a1350 con 0x7f337410a320 2024-05-27T07:55:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.370+0000 7f3378e00640 1 -- 172.21.15.63:0/4093658204 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33741ad630 con 0x7f337410a320 2024-05-27T07:55:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.374+0000 7f33813f1740 1 Processor -- start 2024-05-27T07:55:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.374+0000 7f336be00640 1 -- 172.21.15.63:0/4093658204 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3364009a30 con 0x7f337410a320 2024-05-27T07:55:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.375+0000 7f33813f1740 1 -- start start 2024-05-27T07:55:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.375+0000 7f33813f1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83611b0 0x5622b8703a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.375+0000 7f33813f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 0x5622b8703fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.375+0000 7f33813f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b87044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.375+0000 7f33813f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5622b8674b90 con 0x5622b83404d0 2024-05-27T07:55:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.375+0000 7f33813f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622b8664730 con 0x5622b83382f0 2024-05-27T07:55:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.375+0000 7f33813f1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5622b8668770 con 0x5622b83611b0 2024-05-27T07:55:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b87044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b87044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51396/0 (socket says 172.21.15.63:51396) 2024-05-27T07:55:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3372a00640 1 -- 172.21.15.63:0/294098018 learned_addr learned my addr 172.21.15.63:0/294098018 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3373e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83611b0 0x5622b8703a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3373400640 1 --2- 172.21.15.63:0/294098018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 0x5622b8703fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3372a00640 1 -- 172.21.15.63:0/294098018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83611b0 msgr2=0x5622b8703a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3372a00640 1 --2- 172.21.15.63:0/294098018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83611b0 0x5622b8703a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3372a00640 1 -- 172.21.15.63:0/294098018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 msgr2=0x5622b8703fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3372a00640 1 --2- 172.21.15.63:0/294098018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 0x5622b8703fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3372a00640 1 -- 172.21.15.63:0/294098018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622b85fd8b0 con 0x5622b83404d0 2024-05-27T07:55:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.376+0000 7f3373400640 1 --2- 172.21.15.63:0/294098018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 0x5622b8703fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.377+0000 7f336be00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33580b2950 0x7f33580b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.377+0000 7f3372a00640 1 --2- 172.21.15.63:0/294098018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b87044e0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f33600045e0 tx=0x7f3360004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7476661a407c5cd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.377+0000 7f336be00640 1 -- 172.21.15.63:0/4093658204 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f33640e23d0 con 0x7f337410a320 2024-05-27T07:55:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.377+0000 7f3373e00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33580b2950 0x7f33580b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.377+0000 7f3368c00640 1 -- 172.21.15.63:0/294098018 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360003af0 con 0x5622b83404d0 2024-05-27T07:55:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.377+0000 7f3368c00640 1 -- 172.21.15.63:0/294098018 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3360015070 con 0x5622b83404d0 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.377+0000 7f3368c00640 1 -- 172.21.15.63:0/294098018 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360004c00 con 0x5622b83404d0 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.378+0000 7f33813f1740 1 -- 172.21.15.63:0/294098018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 msgr2=0x5622b87044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.378+0000 7f3373e00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33580b2950 0x7f33580b4e60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f336c004af0 tx=0x7f336c005da0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.378+0000 7f33813f1740 1 --2- 172.21.15.63:0/294098018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b87044e0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f33600045e0 tx=0x7f3360004610 comp rx=0 tx=0).stop 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.378+0000 7f33813f1740 1 -- 172.21.15.63:0/294098018 shutdown_connections 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.378+0000 7f33813f1740 1 --2- 172.21.15.63:0/294098018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b87044e0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.378+0000 7f33813f1740 1 --2- 172.21.15.63:0/294098018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 0x5622b8703fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.378+0000 7f33813f1740 1 --2- 172.21.15.63:0/294098018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83611b0 0x5622b8703a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.378+0000 7f33813f1740 1 -- 172.21.15.63:0/294098018 >> 172.21.15.63:0/294098018 conn(0x5622b8355300 msgr2=0x5622b8700210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.379+0000 7f33813f1740 1 -- 172.21.15.63:0/294098018 shutdown_connections 2024-05-27T07:55:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.379+0000 7f33813f1740 1 -- 172.21.15.63:0/294098018 wait complete. 2024-05-27T07:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.379+0000 7f33813f1740 1 -- start start 2024-05-27T07:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.379+0000 7f33813f1740 1 Processor -- start 2024-05-27T07:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.380+0000 7f33813f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 0x5622b870db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.380+0000 7f33813f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b870e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.380+0000 7f33813f1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83611b0 0x5622b870e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.380+0000 7f33813f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5622b834a790 con 0x5622b83404d0 2024-05-27T07:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.380+0000 7f33813f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622b860cbc0 con 0x5622b83382f0 2024-05-27T07:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.380+0000 7f33813f1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5622b8710be0 con 0x5622b83611b0 2024-05-27T07:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.380+0000 7f33813f1740 10 client.0 _ll_register_callbacks cb 0x5622b85ea400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.380+0000 7f3372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b870e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.380+0000 7f3372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b870e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51398/0 (socket says 172.21.15.63:51398) 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.380+0000 7f3372a00640 1 -- 172.21.15.63:0/1819697732 learned_addr learned my addr 172.21.15.63:0/1819697732 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.381+0000 7f3373400640 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 0x5622b870db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.381+0000 7f3373e00640 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83611b0 0x5622b870e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.381+0000 7f3372a00640 1 -- 172.21.15.63:0/1819697732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83611b0 msgr2=0x5622b870e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.381+0000 7f3372a00640 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83611b0 0x5622b870e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.381+0000 7f3372a00640 1 -- 172.21.15.63:0/1819697732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 msgr2=0x5622b870db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.381+0000 7f3372a00640 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 0x5622b870db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.381+0000 7f3372a00640 1 -- 172.21.15.63:0/1819697732 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622b8379210 con 0x5622b83404d0 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.381+0000 7f3372a00640 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b870e0a0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f336000b620 tx=0x7f3360003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c06543eb71408 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.381+0000 7f335d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.382+0000 7f335d600640 1 -- 172.21.15.63:0/1819697732 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360028060 con 0x5622b83404d0 2024-05-27T07:55:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.382+0000 7f335d600640 1 -- 172.21.15.63:0/1819697732 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f3360011640 con 0x5622b83404d0 2024-05-27T07:55:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.382+0000 7f33813f1740 1 -- 172.21.15.63:0/1819697732 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3360015070 con 0x5622b83404d0 2024-05-27T07:55:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.382+0000 7f335d600640 1 -- 172.21.15.63:0/1819697732 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f336002f050 con 0x5622b83404d0 2024-05-27T07:55:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.382+0000 7f33813f1740 10 client.19614 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.382+0000 7f33813f1740 1 -- 172.21.15.63:0/1819697732 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5622b8445620 con 0x5622b83404d0 2024-05-27T07:55:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.382+0000 7f335d600640 1 -- 172.21.15.63:0/1819697732 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3360015070 con 0x5622b83404d0 2024-05-27T07:55:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.383+0000 7f335d600640 1 -- 172.21.15.63:0/1819697732 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3360022070 con 0x5622b83404d0 2024-05-27T07:55:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.383+0000 7f33813f1740 10 client.19614 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.383+0000 7f33813f1740 10 client.19614 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.383+0000 7f33813f1740 20 client.19614 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.383+0000 7f33813f1740 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622b831fd80 0x5622b871fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.383+0000 7f33813f1740 4 client.19614 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.383+0000 7f33813f1740 1 -- 172.21.15.63:0/1819697732 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622b8676ae0 con 0x5622b831fd80 2024-05-27T07:55:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.384+0000 7f3373400640 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622b831fd80 0x5622b871fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.385+0000 7f3373400640 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622b831fd80 0x5622b871fd10 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9699400e741655b6 server_cookie=638fe09ce788b11b in_seq=0 out_seq=0 2024-05-27T07:55:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.385+0000 7f335d600640 10 client.19614 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f335d600640 1 -- 172.21.15.63:0/1819697732 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5622b8676ae0 con 0x5622b831fd80 2024-05-27T07:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f335d600640 10 client.19614 handle_command_reply: tid=0 2024-05-27T07:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 client.19614 shutdown 2024-05-27T07:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 -- 172.21.15.63:0/1819697732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 msgr2=0x5622b870e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b870e0a0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f336000b620 tx=0x7f3360003b20 comp rx=0 tx=0).stop 2024-05-27T07:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 -- 172.21.15.63:0/1819697732 shutdown_connections 2024-05-27T07:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622b831fd80 0x5622b871fd10 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83611b0 0x5622b870e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b870e0a0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 --2- 172.21.15.63:0/1819697732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83382f0 0x5622b870db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 -- 172.21.15.63:0/1819697732 >> 172.21.15.63:0/1819697732 conn(0x5622b8355300 msgr2=0x5622b8705780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f335d600640 0 client.19614 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 -- 172.21.15.63:0/1819697732 shutdown_connections 2024-05-27T07:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.387+0000 7f33813f1740 1 -- 172.21.15.63:0/1819697732 wait complete. 2024-05-27T07:55:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.388+0000 7f33813f1740 20 client.19614 trim_cache size 0 max 16384 2024-05-27T07:55:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.408+0000 7f33813f1740 1 Processor -- start 2024-05-27T07:55:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.408+0000 7f33813f1740 1 -- start start 2024-05-27T07:55:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.408+0000 7f33813f1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83382f0 0x5622b870fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.408+0000 7f33813f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8719580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.408+0000 7f33813f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 0x5622b870ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.408+0000 7f33813f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b9f30 con 0x5622b83404d0 2024-05-27T07:55:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.408+0000 7f33813f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b7800 con 0x5622b83611b0 2024-05-27T07:55:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.408+0000 7f33813f1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b5670 con 0x5622b83382f0 2024-05-27T07:55:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.409+0000 7f3372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8719580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.409+0000 7f3372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8719580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51404/0 (socket says 172.21.15.63:51404) 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.409+0000 7f3372a00640 1 -- 172.21.15.63:0/3760998477 learned_addr learned my addr 172.21.15.63:0/3760998477 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.409+0000 7f3373e00640 1 --2- 172.21.15.63:0/3760998477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 0x5622b870ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.409+0000 7f3373400640 1 --2- 172.21.15.63:0/3760998477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83382f0 0x5622b870fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.409+0000 7f3372a00640 1 -- 172.21.15.63:0/3760998477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83382f0 msgr2=0x5622b870fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.409+0000 7f3372a00640 1 --2- 172.21.15.63:0/3760998477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83382f0 0x5622b870fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.409+0000 7f3372a00640 1 -- 172.21.15.63:0/3760998477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 msgr2=0x5622b870ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.409+0000 7f3372a00640 1 --2- 172.21.15.63:0/3760998477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 0x5622b870ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.409+0000 7f3372a00640 1 -- 172.21.15.63:0/3760998477 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622b870c0b0 con 0x5622b83404d0 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.410+0000 7f3372a00640 1 --2- 172.21.15.63:0/3760998477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8719580 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3360006010 tx=0x7f336000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2217e43c14b4dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.410+0000 7f335fe00640 1 -- 172.21.15.63:0/3760998477 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360028070 con 0x5622b83404d0 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.410+0000 7f335fe00640 1 -- 172.21.15.63:0/3760998477 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3360015040 con 0x5622b83404d0 2024-05-27T07:55:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.410+0000 7f335fe00640 1 -- 172.21.15.63:0/3760998477 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360005670 con 0x5622b83404d0 2024-05-27T07:55:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.411+0000 7f33813f1740 1 -- 172.21.15.63:0/3760998477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 msgr2=0x5622b8719580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.411+0000 7f33813f1740 1 --2- 172.21.15.63:0/3760998477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8719580 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3360006010 tx=0x7f336000b4a0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.411+0000 7f33813f1740 1 -- 172.21.15.63:0/3760998477 shutdown_connections 2024-05-27T07:55:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.411+0000 7f33813f1740 1 --2- 172.21.15.63:0/3760998477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 0x5622b870ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.411+0000 7f33813f1740 1 --2- 172.21.15.63:0/3760998477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8719580 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.411+0000 7f33813f1740 1 --2- 172.21.15.63:0/3760998477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83382f0 0x5622b870fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.411+0000 7f33813f1740 1 -- 172.21.15.63:0/3760998477 >> 172.21.15.63:0/3760998477 conn(0x5622b8355300 msgr2=0x5622b8718e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.411+0000 7f33813f1740 1 -- 172.21.15.63:0/3760998477 shutdown_connections 2024-05-27T07:55:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.411+0000 7f33813f1740 1 -- 172.21.15.63:0/3760998477 wait complete. 2024-05-27T07:55:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.412+0000 7f33813f1740 1 -- start start 2024-05-27T07:55:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.412+0000 7f33813f1740 1 Processor -- start 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.412+0000 7f33813f1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83382f0 0x5622b8722720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f33813f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8722c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f33813f1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 0x5622b877f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f33813f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b5cc0 con 0x5622b83404d0 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f33813f1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b7130 con 0x5622b83611b0 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f33813f1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f33741b6390 con 0x5622b83382f0 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f33813f1740 10 client.0 _ll_register_callbacks cb 0x7f336c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8722c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3372a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8722c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51406/0 (socket says 172.21.15.63:51406) 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3372a00640 1 -- 172.21.15.63:0/3172700130 learned_addr learned my addr 172.21.15.63:0/3172700130 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:19.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3373400640 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83382f0 0x5622b8722720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3373e00640 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 0x5622b877f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3372a00640 1 -- 172.21.15.63:0/3172700130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83382f0 msgr2=0x5622b8722720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3372a00640 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83382f0 0x5622b8722720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3372a00640 1 -- 172.21.15.63:0/3172700130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 msgr2=0x5622b877f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3372a00640 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 0x5622b877f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3372a00640 1 -- 172.21.15.63:0/3172700130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622b877f830 con 0x5622b83404d0 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.413+0000 7f3373e00640 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 0x5622b877f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.414+0000 7f3372a00640 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8722c60 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3360004370 tx=0x7f3360004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bffdf48873de23 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.414+0000 7f335d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.414+0000 7f335d600640 1 -- 172.21.15.63:0/3172700130 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360028040 con 0x5622b83404d0 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.414+0000 7f33813f1740 1 -- 172.21.15.63:0/3172700130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3360015040 con 0x5622b83404d0 2024-05-27T07:55:19.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.414+0000 7f335d600640 1 -- 172.21.15.63:0/3172700130 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f3360019420 con 0x5622b83404d0 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.414+0000 7f335d600640 1 -- 172.21.15.63:0/3172700130 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3360047050 con 0x5622b83404d0 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.414+0000 7f33813f1740 10 client.19620 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.414+0000 7f33813f1740 1 -- 172.21.15.63:0/3172700130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5622b8787d70 con 0x5622b83404d0 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.415+0000 7f335d600640 1 -- 172.21.15.63:0/3172700130 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3360015040 con 0x5622b83404d0 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.415+0000 7f335d600640 1 -- 172.21.15.63:0/3172700130 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3360022070 con 0x5622b83404d0 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.415+0000 7f33813f1740 10 client.19620 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.415+0000 7f33813f1740 10 client.19620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.415+0000 7f33813f1740 20 client.19620 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.415+0000 7f33813f1740 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622b82f51c0 0x5622b878a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.415+0000 7f33813f1740 4 client.19620 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:55:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.415+0000 7f33813f1740 1 -- 172.21.15.63:0/3172700130 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622b86702c0 con 0x5622b82f51c0 2024-05-27T07:55:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.416+0000 7f3373400640 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622b82f51c0 0x5622b878a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.417+0000 7f3373400640 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622b82f51c0 0x5622b878a4c0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d9c71c61d71d6e8 server_cookie=4cc401617d5ea006 in_seq=0 out_seq=0 2024-05-27T07:55:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.417+0000 7f335d600640 10 client.19620 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.452+0000 7f335d600640 1 -- 172.21.15.63:0/3172700130 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5622b86702c0 con 0x5622b82f51c0 2024-05-27T07:55:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.452+0000 7f335d600640 10 client.19620 handle_command_reply: tid=0 2024-05-27T07:55:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.452+0000 7f33813f1740 1 client.19620 shutdown 2024-05-27T07:55:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.452+0000 7f33813f1740 1 -- 172.21.15.63:0/3172700130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 msgr2=0x5622b8722c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.452+0000 7f33813f1740 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8722c60 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3360004370 tx=0x7f3360004660 comp rx=0 tx=0).stop 2024-05-27T07:55:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.453+0000 7f33813f1740 1 -- 172.21.15.63:0/3172700130 shutdown_connections 2024-05-27T07:55:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.453+0000 7f33813f1740 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622b82f51c0 0x5622b878a4c0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.453+0000 7f33813f1740 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622b83611b0 0x5622b877f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.453+0000 7f33813f1740 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622b83404d0 0x5622b8722c60 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.453+0000 7f335d600640 0 client.19620 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.453+0000 7f33813f1740 1 --2- 172.21.15.63:0/3172700130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622b83382f0 0x5622b8722720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.453+0000 7f33813f1740 1 -- 172.21.15.63:0/3172700130 >> 172.21.15.63:0/3172700130 conn(0x5622b8355300 msgr2=0x5622b8719600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.453+0000 7f33813f1740 1 -- 172.21.15.63:0/3172700130 shutdown_connections 2024-05-27T07:55:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.453+0000 7f33813f1740 1 -- 172.21.15.63:0/3172700130 wait complete. 2024-05-27T07:55:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.453+0000 7f33813f1740 20 client.19620 trim_cache size 0 max 16384 2024-05-27T07:55:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:55:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:55:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:55:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.454+0000 7f3378e00640 1 -- 172.21.15.63:0/4093658204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33580b2950 msgr2=0x7f33580b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.454+0000 7f3378e00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33580b2950 0x7f33580b4e60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f336c004af0 tx=0x7f336c005da0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.454+0000 7f3378e00640 1 -- 172.21.15.63:0/4093658204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f337410a320 msgr2=0x7f33741a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.454+0000 7f3378e00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f337410a320 0x7f33741a67b0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f336400d950 tx=0x7f336400de10 comp rx=0 tx=0).stop 2024-05-27T07:55:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.454+0000 7f3378e00640 1 -- 172.21.15.63:0/4093658204 shutdown_connections 2024-05-27T07:55:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.454+0000 7f3378e00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33580b2950 0x7f33580b4e60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.454+0000 7f3378e00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f337410f370 0x7f33741a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.454+0000 7f3378e00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f337410e8b0 0x7f33741a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.454+0000 7f3378e00640 1 --2- 172.21.15.63:0/4093658204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f337410a320 0x7f33741a67b0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.454+0000 7f3378e00640 1 -- 172.21.15.63:0/4093658204 >> 172.21.15.63:0/4093658204 conn(0x7f3374076070 msgr2=0x7f3374107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.455+0000 7f3378e00640 1 -- 172.21.15.63:0/4093658204 shutdown_connections 2024-05-27T07:55:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:19.455+0000 7f3378e00640 1 -- 172.21.15.63:0/4093658204 wait complete. 2024-05-27T07:55:19.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:19 smithi086 ceph-mon[34687]: pgmap v687: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-05-27T07:55:19.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:19 smithi149 ceph-mon[35408]: pgmap v687: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-05-27T07:55:19.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:19 smithi063 ceph-mon[31613]: pgmap v687: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-05-27T07:55:21.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:21 smithi086 ceph-mon[34687]: pgmap v688: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-05-27T07:55:21.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:21 smithi149 ceph-mon[35408]: pgmap v688: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-05-27T07:55:21.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:21 smithi063 ceph-mon[31613]: pgmap v688: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 7 op/s 2024-05-27T07:55:23.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:23 smithi086 ceph-mon[34687]: pgmap v689: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.1 MiB/s wr, 14 op/s 2024-05-27T07:55:23.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:23 smithi149 ceph-mon[35408]: pgmap v689: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.1 MiB/s wr, 14 op/s 2024-05-27T07:55:23.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:23 smithi063 ceph-mon[31613]: pgmap v689: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.1 MiB/s wr, 14 op/s 2024-05-27T07:55:25.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:25 smithi086 ceph-mon[34687]: pgmap v690: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-05-27T07:55:25.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:25 smithi149 ceph-mon[35408]: pgmap v690: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-05-27T07:55:25.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:25 smithi063 ceph-mon[31613]: pgmap v690: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-05-27T07:55:27.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:27 smithi086 ceph-mon[34687]: pgmap v691: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-05-27T07:55:27.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:27 smithi149 ceph-mon[35408]: pgmap v691: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-05-27T07:55:27.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:27 smithi063 ceph-mon[31613]: pgmap v691: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.1 MiB/s wr, 10 op/s 2024-05-27T07:55:29.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:29 smithi086 ceph-mon[34687]: pgmap v692: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 13 op/s 2024-05-27T07:55:29.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:29 smithi149 ceph-mon[35408]: pgmap v692: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 13 op/s 2024-05-27T07:55:29.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:29 smithi063 ceph-mon[31613]: pgmap v692: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 13 op/s 2024-05-27T07:55:31.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:31 smithi086 ceph-mon[34687]: pgmap v693: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.0 MiB/s wr, 10 op/s 2024-05-27T07:55:31.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:31 smithi149 ceph-mon[35408]: pgmap v693: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.0 MiB/s wr, 10 op/s 2024-05-27T07:55:31.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:31 smithi063 ceph-mon[31613]: pgmap v693: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.0 MiB/s wr, 10 op/s 2024-05-27T07:55:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.700+0000 7f50f4a00640 1 -- 172.21.15.63:0/4149168043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f010beb0 msgr2=0x7f50f0105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.700+0000 7f50f4a00640 1 --2- 172.21.15.63:0/4149168043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f010beb0 0x7f50f0105a50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009900 tx=0x7f50e002a410 comp rx=0 tx=0).stop 2024-05-27T07:55:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.700+0000 7f50f4a00640 1 -- 172.21.15.63:0/4149168043 shutdown_connections 2024-05-27T07:55:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.700+0000 7f50f4a00640 1 --2- 172.21.15.63:0/4149168043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50f0106600 0x7f50f01129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.700+0000 7f50f4a00640 1 --2- 172.21.15.63:0/4149168043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f010c350 0x7f50f0105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.700+0000 7f50f4a00640 1 --2- 172.21.15.63:0/4149168043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f010beb0 0x7f50f0105a50 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.700+0000 7f50f4a00640 1 -- 172.21.15.63:0/4149168043 >> 172.21.15.63:0/4149168043 conn(0x7f50f01016b0 msgr2=0x7f50f0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.700+0000 7f50f4a00640 1 -- 172.21.15.63:0/4149168043 shutdown_connections 2024-05-27T07:55:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.700+0000 7f50f4a00640 1 -- 172.21.15.63:0/4149168043 wait complete. 2024-05-27T07:55:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.702+0000 7f50f4a00640 1 Processor -- start 2024-05-27T07:55:32.703 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.702+0000 7f50f4a00640 1 -- start start 2024-05-27T07:55:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.702+0000 7f50f4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50f0106600 0x7f50f01a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.702+0000 7f50f4a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f010beb0 0x7f50f01a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.702+0000 7f50f4a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f010c350 0x7f50f019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.702+0000 7f50f4a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50f0115f60 con 0x7f50f010c350 2024-05-27T07:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.702+0000 7f50f4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50f0115de0 con 0x7f50f0106600 2024-05-27T07:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.702+0000 7f50f4a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01160e0 con 0x7f50f010beb0 2024-05-27T07:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.703+0000 7f50ee800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f010c350 0x7f50f019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.703+0000 7f50ee800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f010c350 0x7f50f019c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35944/0 (socket says 172.21.15.63:35944) 2024-05-27T07:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.703+0000 7f50ee800640 1 -- 172.21.15.63:0/3614458140 learned_addr learned my addr 172.21.15.63:0/3614458140 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.703+0000 7f50ede00640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50f0106600 0x7f50f01a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.703+0000 7f50ed400640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f010beb0 0x7f50f01a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.704+0000 7f50ee800640 1 -- 172.21.15.63:0/3614458140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f010beb0 msgr2=0x7f50f01a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.704+0000 7f50ee800640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f010beb0 0x7f50f01a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.704+0000 7f50ee800640 1 -- 172.21.15.63:0/3614458140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50f0106600 msgr2=0x7f50f01a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.704+0000 7f50ee800640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50f0106600 0x7f50f01a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.704+0000 7f50ee800640 1 -- 172.21.15.63:0/3614458140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f019cf00 con 0x7f50f010c350 2024-05-27T07:55:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.704+0000 7f50ee800640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f010c350 0x7f50f019c860 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f50e800eaa0 tx=0x7f50e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.704+0000 7f50e6a00640 1 -- 172.21.15.63:0/3614458140 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e800cdb0 con 0x7f50f010c350 2024-05-27T07:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.704+0000 7f50e6a00640 1 -- 172.21.15.63:0/3614458140 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50e8005020 con 0x7f50f010c350 2024-05-27T07:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.704+0000 7f50f4a00640 1 -- 172.21.15.63:0/3614458140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f019d1f0 con 0x7f50f010c350 2024-05-27T07:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.704+0000 7f50e6a00640 1 -- 172.21.15.63:0/3614458140 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e80116d0 con 0x7f50f010c350 2024-05-27T07:55:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.705+0000 7f50f4a00640 1 -- 172.21.15.63:0/3614458140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f01a94d0 con 0x7f50f010c350 2024-05-27T07:55:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.710+0000 7f50fba34740 1 Processor -- start 2024-05-27T07:55:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.710+0000 7f50e6a00640 1 -- 172.21.15.63:0/3614458140 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f50e80051d0 con 0x7f50f010c350 2024-05-27T07:55:32.711 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.710+0000 7f50fba34740 1 -- start start 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50fba34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac961b0 0x5653cb038860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50fba34740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb038da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50fba34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 0x5653cb0392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50fba34740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5653cafa9b90 con 0x5653cac6d2f0 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50fba34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5653caf99730 con 0x5653cac961b0 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50fba34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5653caf9d770 con 0x5653cac754d0 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50ede00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50ede00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb038da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35952/0 (socket says 172.21.15.63:35952) 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50ede00640 1 -- 172.21.15.63:0/3886566365 learned_addr learned my addr 172.21.15.63:0/3886566365 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50ed400640 1 --2- 172.21.15.63:0/3886566365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac961b0 0x5653cb038860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.711+0000 7f50ed400640 1 --2- 172.21.15.63:0/3886566365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 0x5653cb0392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.712+0000 7f50ede00640 1 -- 172.21.15.63:0/3886566365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 msgr2=0x5653cb0392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.712+0000 7f50ede00640 1 --2- 172.21.15.63:0/3886566365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 0x5653cb0392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.712+0000 7f50ede00640 1 -- 172.21.15.63:0/3886566365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac961b0 msgr2=0x5653cb038860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.712+0000 7f50e6a00640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50d40b29d0 0x7f50d40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.712+0000 7f50ede00640 1 --2- 172.21.15.63:0/3886566365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac961b0 0x5653cb038860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.712+0000 7f50ede00640 1 -- 172.21.15.63:0/3886566365 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653caf328b0 con 0x5653cac6d2f0 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.712+0000 7f50e6a00640 1 -- 172.21.15.63:0/3614458140 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f50e80e2aa0 con 0x7f50f010c350 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.712+0000 7f50ed400640 1 --2- 172.21.15.63:0/3886566365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 0x5653cb0392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.712+0000 7f50ee800640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50d40b29d0 0x7f50d40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.712+0000 7f50ede00640 1 --2- 172.21.15.63:0/3886566365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb038da0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f50e0000c00 tx=0x7f50e0031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca7a7c873547547 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.713+0000 7f50db000640 1 -- 172.21.15.63:0/3886566365 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0035030 con 0x5653cac6d2f0 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.713+0000 7f50db000640 1 -- 172.21.15.63:0/3886566365 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50e0040070 con 0x5653cac6d2f0 2024-05-27T07:55:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.713+0000 7f50db000640 1 -- 172.21.15.63:0/3886566365 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0003970 con 0x5653cac6d2f0 2024-05-27T07:55:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.714+0000 7f50fba34740 1 -- 172.21.15.63:0/3886566365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 msgr2=0x5653cb038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.714+0000 7f50ee800640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50d40b29d0 0x7f50d40b4ee0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f50e80a5af0 tx=0x7f50e808ff90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.714+0000 7f50fba34740 1 --2- 172.21.15.63:0/3886566365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb038da0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f50e0000c00 tx=0x7f50e0031bb0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.715+0000 7f50fba34740 1 -- 172.21.15.63:0/3886566365 shutdown_connections 2024-05-27T07:55:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.715+0000 7f50fba34740 1 --2- 172.21.15.63:0/3886566365 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 0x5653cb0392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.715+0000 7f50fba34740 1 --2- 172.21.15.63:0/3886566365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb038da0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.715+0000 7f50fba34740 1 --2- 172.21.15.63:0/3886566365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac961b0 0x5653cb038860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.715+0000 7f50fba34740 1 -- 172.21.15.63:0/3886566365 >> 172.21.15.63:0/3886566365 conn(0x5653cac8a300 msgr2=0x5653cb035010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.715+0000 7f50fba34740 1 -- 172.21.15.63:0/3886566365 shutdown_connections 2024-05-27T07:55:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.715+0000 7f50fba34740 1 -- 172.21.15.63:0/3886566365 wait complete. 2024-05-27T07:55:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.716+0000 7f50fba34740 1 -- start start 2024-05-27T07:55:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.716+0000 7f50fba34740 1 Processor -- start 2024-05-27T07:55:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50fba34740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb042a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50fba34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 0x5653cb042f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50fba34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac961b0 0x5653cb04ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50fba34740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5653cac7f790 con 0x5653cac6d2f0 2024-05-27T07:55:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50fba34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5653caf41bc0 con 0x5653cac961b0 2024-05-27T07:55:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50fba34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5653cb045930 con 0x5653cac754d0 2024-05-27T07:55:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50fba34740 10 client.0 _ll_register_callbacks cb 0x5653caf1f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50ed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb042a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50ed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb042a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35960/0 (socket says 172.21.15.63:35960) 2024-05-27T07:55:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50ed400640 1 -- 172.21.15.63:0/2462187871 learned_addr learned my addr 172.21.15.63:0/2462187871 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.717+0000 7f50ede00640 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 0x5653cb042f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.718+0000 7f50ed400640 1 -- 172.21.15.63:0/2462187871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 msgr2=0x5653cb042f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.718+0000 7f50ed400640 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 0x5653cb042f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.718+0000 7f50ed400640 1 -- 172.21.15.63:0/2462187871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac961b0 msgr2=0x5653cb04ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.718+0000 7f50ed400640 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac961b0 0x5653cb04ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.718+0000 7f50ed400640 1 -- 172.21.15.63:0/2462187871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cacae210 con 0x5653cac6d2f0 2024-05-27T07:55:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.718+0000 7f50ed400640 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb042a00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f50dc005940 tx=0x7f50dc010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af524c91cb1d72e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.718+0000 7f50cf400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.718+0000 7f50cf400640 1 -- 172.21.15.63:0/2462187871 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc005220 con 0x5653cac6d2f0 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.718+0000 7f50fba34740 1 -- 172.21.15.63:0/2462187871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e0040070 con 0x5653cac6d2f0 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.718+0000 7f50cf400640 1 -- 172.21.15.63:0/2462187871 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f50dc0053c0 con 0x5653cac6d2f0 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.719+0000 7f50cf400640 1 -- 172.21.15.63:0/2462187871 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f50dc026050 con 0x5653cac6d2f0 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.719+0000 7f50fba34740 10 client.19632 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.719+0000 7f50fba34740 1 -- 172.21.15.63:0/2462187871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5653cad7a620 con 0x5653cac6d2f0 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.719+0000 7f50cf400640 1 -- 172.21.15.63:0/2462187871 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50e0040070 con 0x5653cac6d2f0 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.719+0000 7f50cf400640 1 -- 172.21.15.63:0/2462187871 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f50dc011700 con 0x5653cac6d2f0 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.720+0000 7f50fba34740 10 client.19632 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.720+0000 7f50fba34740 10 client.19632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.720+0000 7f50fba34740 20 client.19632 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.720+0000 7f50fba34740 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5653cac54d80 0x5653cb054c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.720+0000 7f50fba34740 4 client.19632 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.720+0000 7f50fba34740 1 -- 172.21.15.63:0/2462187871 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653cafabae0 con 0x5653cac54d80 2024-05-27T07:55:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.720+0000 7f50ede00640 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5653cac54d80 0x5653cb054c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.722+0000 7f50ede00640 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5653cac54d80 0x5653cb054c90 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f11bc908e9f1670 server_cookie=4b35b043b6b8d1c1 in_seq=0 out_seq=0 2024-05-27T07:55:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.722+0000 7f50cf400640 10 client.19632 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.723+0000 7f50cf400640 1 -- 172.21.15.63:0/2462187871 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5653cafabae0 con 0x5653cac54d80 2024-05-27T07:55:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.723+0000 7f50cf400640 10 client.19632 handle_command_reply: tid=0 2024-05-27T07:55:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.723+0000 7f50fba34740 1 client.19632 shutdown 2024-05-27T07:55:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 1 -- 172.21.15.63:0/2462187871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 msgr2=0x5653cb042a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb042a00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f50dc005940 tx=0x7f50dc010860 comp rx=0 tx=0).stop 2024-05-27T07:55:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 1 -- 172.21.15.63:0/2462187871 shutdown_connections 2024-05-27T07:55:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5653cac54d80 0x5653cb054c90 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac961b0 0x5653cb04ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac754d0 0x5653cb042f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 1 --2- 172.21.15.63:0/2462187871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb042a00 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50cf400640 0 client.19632 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 1 -- 172.21.15.63:0/2462187871 >> 172.21.15.63:0/2462187871 conn(0x5653cac8a300 msgr2=0x5653cb03a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 1 -- 172.21.15.63:0/2462187871 shutdown_connections 2024-05-27T07:55:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 1 -- 172.21.15.63:0/2462187871 wait complete. 2024-05-27T07:55:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.724+0000 7f50fba34740 20 client.19632 trim_cache size 0 max 16384 2024-05-27T07:55:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.746+0000 7f50fba34740 1 Processor -- start 2024-05-27T07:55:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.746+0000 7f50fba34740 1 -- start start 2024-05-27T07:55:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.746+0000 7f50fba34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac6d2f0 0x5653cb045130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.746+0000 7f50fba34740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac754d0 0x5653cb04ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.746+0000 7f50fba34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac961b0 0x5653cb04ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.746+0000 7f50fba34740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01b5d70 con 0x5653cac754d0 2024-05-27T07:55:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.746+0000 7f50fba34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01b3640 con 0x5653cac6d2f0 2024-05-27T07:55:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.746+0000 7f50fba34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01b14b0 con 0x5653cac961b0 2024-05-27T07:55:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.747+0000 7f50ede00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac754d0 0x5653cb04ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.747+0000 7f50ede00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac754d0 0x5653cb04ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35966/0 (socket says 172.21.15.63:35966) 2024-05-27T07:55:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.747+0000 7f50ede00640 1 -- 172.21.15.63:0/3182512256 learned_addr learned my addr 172.21.15.63:0/3182512256 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.747+0000 7f50ed400640 1 --2- 172.21.15.63:0/3182512256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac6d2f0 0x5653cb045130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.747+0000 7f50ede00640 1 -- 172.21.15.63:0/3182512256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac961b0 msgr2=0x5653cb04ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.747+0000 7f50ede00640 1 --2- 172.21.15.63:0/3182512256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac961b0 0x5653cb04ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.747+0000 7f50ede00640 1 -- 172.21.15.63:0/3182512256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac6d2f0 msgr2=0x5653cb045130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:32.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.747+0000 7f50ede00640 1 --2- 172.21.15.63:0/3182512256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac6d2f0 0x5653cb045130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.747+0000 7f50ede00640 1 -- 172.21.15.63:0/3182512256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cb039c40 con 0x5653cac754d0 2024-05-27T07:55:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.747+0000 7f50ed400640 1 --2- 172.21.15.63:0/3182512256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac6d2f0 0x5653cb045130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.748+0000 7f50ede00640 1 --2- 172.21.15.63:0/3182512256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac754d0 0x5653cb04ea00 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009900 tx=0x7f50e002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca5c3d9db23ec86 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.748+0000 7f50da600640 1 -- 172.21.15.63:0/3182512256 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0055070 con 0x5653cac754d0 2024-05-27T07:55:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.748+0000 7f50da600640 1 -- 172.21.15.63:0/3182512256 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50e0040040 con 0x5653cac754d0 2024-05-27T07:55:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.748+0000 7f50da600640 1 -- 172.21.15.63:0/3182512256 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0031e60 con 0x5653cac754d0 2024-05-27T07:55:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.749+0000 7f50fba34740 1 -- 172.21.15.63:0/3182512256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac754d0 msgr2=0x5653cb04ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.749+0000 7f50fba34740 1 --2- 172.21.15.63:0/3182512256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac754d0 0x5653cb04ea00 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009900 tx=0x7f50e002a410 comp rx=0 tx=0).stop 2024-05-27T07:55:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.749+0000 7f50fba34740 1 -- 172.21.15.63:0/3182512256 shutdown_connections 2024-05-27T07:55:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.749+0000 7f50fba34740 1 --2- 172.21.15.63:0/3182512256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac961b0 0x5653cb04ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.749+0000 7f50fba34740 1 --2- 172.21.15.63:0/3182512256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac754d0 0x5653cb04ea00 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.749+0000 7f50fba34740 1 --2- 172.21.15.63:0/3182512256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac6d2f0 0x5653cb045130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.749+0000 7f50fba34740 1 -- 172.21.15.63:0/3182512256 >> 172.21.15.63:0/3182512256 conn(0x5653cac8a300 msgr2=0x5653cb03dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.750+0000 7f50fba34740 1 -- 172.21.15.63:0/3182512256 shutdown_connections 2024-05-27T07:55:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.750+0000 7f50fba34740 1 -- 172.21.15.63:0/3182512256 wait complete. 2024-05-27T07:55:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.750+0000 7f50fba34740 1 -- start start 2024-05-27T07:55:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.750+0000 7f50fba34740 1 Processor -- start 2024-05-27T07:55:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.751+0000 7f50fba34740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb04b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.751+0000 7f50fba34740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac754d0 0x5653cb0464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.751+0000 7f50fba34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac961b0 0x5653cb0469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.751+0000 7f50fba34740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01b1b00 con 0x5653cac6d2f0 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.751+0000 7f50fba34740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01b2f70 con 0x5653cac754d0 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.751+0000 7f50fba34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50f01b21d0 con 0x5653cac961b0 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.751+0000 7f50fba34740 10 client.0 _ll_register_callbacks cb 0x7f50dc00bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.751+0000 7f50ed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb04b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.751+0000 7f50ed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb04b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35976/0 (socket says 172.21.15.63:35976) 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.751+0000 7f50ed400640 1 -- 172.21.15.63:0/2829752310 learned_addr learned my addr 172.21.15.63:0/2829752310 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.752+0000 7f50ede00640 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac754d0 0x5653cb0464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.752+0000 7f50ed400640 1 -- 172.21.15.63:0/2829752310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac961b0 msgr2=0x5653cb0469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.752+0000 7f50ed400640 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac961b0 0x5653cb0469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.752+0000 7f50ed400640 1 -- 172.21.15.63:0/2829752310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac754d0 msgr2=0x5653cb0464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.752+0000 7f50ed400640 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac754d0 0x5653cb0464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.752+0000 7f50ed400640 1 -- 172.21.15.63:0/2829752310 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cb046f30 con 0x5653cac6d2f0 2024-05-27T07:55:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.752+0000 7f50ed400640 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb04b110 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00bcf0 tx=0x7f50dc010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c98d8c57b28eb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.752+0000 7f50cf400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.753+0000 7f50cf400640 1 -- 172.21.15.63:0/2829752310 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc030070 con 0x5653cac6d2f0 2024-05-27T07:55:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.753+0000 7f50fba34740 1 -- 172.21.15.63:0/2829752310 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e0040040 con 0x5653cac6d2f0 2024-05-27T07:55:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.753+0000 7f50cf400640 1 -- 172.21.15.63:0/2829752310 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f50dc011710 con 0x5653cac6d2f0 2024-05-27T07:55:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.753+0000 7f50cf400640 1 -- 172.21.15.63:0/2829752310 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f50dc00ba00 con 0x5653cac6d2f0 2024-05-27T07:55:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.753+0000 7f50fba34740 10 client.19638 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:32.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.753+0000 7f50fba34740 1 -- 172.21.15.63:0/2829752310 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5653cb0bd030 con 0x5653cac6d2f0 2024-05-27T07:55:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.753+0000 7f50cf400640 1 -- 172.21.15.63:0/2829752310 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50e0040040 con 0x5653cac6d2f0 2024-05-27T07:55:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.753+0000 7f50cf400640 1 -- 172.21.15.63:0/2829752310 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f50dc004aa0 con 0x5653cac6d2f0 2024-05-27T07:55:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.754+0000 7f50fba34740 10 client.19638 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.754+0000 7f50fba34740 10 client.19638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.754+0000 7f50fba34740 20 client.19638 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.754+0000 7f50fba34740 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5653cac2a1c0 0x5653cb0bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.754+0000 7f50fba34740 4 client.19638 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:55:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.754+0000 7f50fba34740 1 -- 172.21.15.63:0/2829752310 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653cafa52c0 con 0x5653cac2a1c0 2024-05-27T07:55:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.754+0000 7f50ede00640 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5653cac2a1c0 0x5653cb0bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.755+0000 7f50ede00640 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5653cac2a1c0 0x5653cb0bf7a0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=577a82687a268c09 server_cookie=ada46dcb55b99ea1 in_seq=0 out_seq=0 2024-05-27T07:55:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.755+0000 7f50cf400640 10 client.19638 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.787+0000 7f50cf400640 1 -- 172.21.15.63:0/2829752310 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5653cafa52c0 con 0x5653cac2a1c0 2024-05-27T07:55:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.787+0000 7f50cf400640 10 client.19638 handle_command_reply: tid=0 2024-05-27T07:55:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.787+0000 7f50fba34740 1 client.19638 shutdown 2024-05-27T07:55:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.787+0000 7f50fba34740 1 -- 172.21.15.63:0/2829752310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 msgr2=0x5653cb04b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.787+0000 7f50fba34740 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb04b110 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00bcf0 tx=0x7f50dc010dd0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.787+0000 7f50fba34740 1 -- 172.21.15.63:0/2829752310 shutdown_connections 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.787+0000 7f50fba34740 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5653cac2a1c0 0x5653cb0bf7a0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.788+0000 7f50fba34740 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653cac961b0 0x5653cb0469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.788+0000 7f50fba34740 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653cac754d0 0x5653cb0464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.788+0000 7f50fba34740 1 --2- 172.21.15.63:0/2829752310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5653cac6d2f0 0x5653cb04b110 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.788+0000 7f50cf400640 0 client.19638 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.788+0000 7f50fba34740 1 -- 172.21.15.63:0/2829752310 >> 172.21.15.63:0/2829752310 conn(0x5653cac8a300 msgr2=0x5653cb044090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.788+0000 7f50fba34740 1 -- 172.21.15.63:0/2829752310 shutdown_connections 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.788+0000 7f50fba34740 1 -- 172.21.15.63:0/2829752310 wait complete. 2024-05-27T07:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.788+0000 7f50fba34740 20 client.19638 trim_cache size 0 max 16384 2024-05-27T07:55:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:55:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:55:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 -- 172.21.15.63:0/3614458140 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50d40b29d0 msgr2=0x7f50d40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50d40b29d0 0x7f50d40b4ee0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f50e80a5af0 tx=0x7f50e808ff90 comp rx=0 tx=0).stop 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 -- 172.21.15.63:0/3614458140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f010c350 msgr2=0x7f50f019c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f010c350 0x7f50f019c860 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f50e800eaa0 tx=0x7f50e800ef60 comp rx=0 tx=0).stop 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 -- 172.21.15.63:0/3614458140 shutdown_connections 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50d40b29d0 0x7f50d40b4ee0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f010c350 0x7f50f019c860 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50f010beb0 0x7f50f01a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 --2- 172.21.15.63:0/3614458140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50f0106600 0x7f50f01a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 -- 172.21.15.63:0/3614458140 >> 172.21.15.63:0/3614458140 conn(0x7f50f01016b0 msgr2=0x7f50f0101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 -- 172.21.15.63:0/3614458140 shutdown_connections 2024-05-27T07:55:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:32.789+0000 7f50f4a00640 1 -- 172.21.15.63:0/3614458140 wait complete. 2024-05-27T07:55:33.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:33 smithi086 ceph-mon[34687]: pgmap v694: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:55:33.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:33 smithi149 ceph-mon[35408]: pgmap v694: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:55:33.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:33 smithi063 ceph-mon[31613]: pgmap v694: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 13 op/s 2024-05-27T07:55:35.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:35 smithi086 ceph-mon[34687]: pgmap v695: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:55:35.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:35 smithi149 ceph-mon[35408]: pgmap v695: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:55:35.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:35 smithi063 ceph-mon[31613]: pgmap v695: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:55:37.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:37 smithi086 ceph-mon[34687]: pgmap v696: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:55:37.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:37 smithi149 ceph-mon[35408]: pgmap v696: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:55:37.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:37 smithi063 ceph-mon[31613]: pgmap v696: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:55:39.095 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:55:39.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:39 smithi063 ceph-mon[31613]: pgmap v697: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:55:39.601 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:39.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:39 smithi086 ceph-mon[34687]: pgmap v697: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:55:39.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:39 smithi149 ceph-mon[35408]: pgmap v697: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:55:41.673 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:41 smithi063 ceph-mon[31613]: pgmap v698: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:55:41.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:41 smithi086 ceph-mon[34687]: pgmap v698: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:55:41.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:41 smithi149 ceph-mon[35408]: pgmap v698: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:55:42.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.185+0000 7f83dfe00700 1 -- 172.21.15.63:0/3226260396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83e01043f0 msgr2=0x7f83e0104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:42.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.185+0000 7f83dfe00700 1 --2- 172.21.15.63:0/3226260396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83e01043f0 0x7f83e0104800 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f83d00099c0 tx=0x7f83d0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.185+0000 7f83dfe00700 1 -- 172.21.15.63:0/3226260396 shutdown_connections 2024-05-27T07:55:42.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.185+0000 7f83dfe00700 1 --2- 172.21.15.63:0/3226260396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83e0109670 0x7f83e0110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.185+0000 7f83dfe00700 1 --2- 172.21.15.63:0/3226260396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83e01043f0 0x7f83e0104800 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.185+0000 7f83dfe00700 1 --2- 172.21.15.63:0/3226260396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e0108d20 0x7f83e0109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.185+0000 7f83dfe00700 1 -- 172.21.15.63:0/3226260396 >> 172.21.15.63:0/3226260396 conn(0x7f83e0079d00 msgr2=0x7f83e007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:42.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.185+0000 7f83dfe00700 1 -- 172.21.15.63:0/3226260396 shutdown_connections 2024-05-27T07:55:42.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.185+0000 7f83dfe00700 1 -- 172.21.15.63:0/3226260396 wait complete. 2024-05-27T07:55:42.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.186+0000 7f83dfe00700 1 Processor -- start 2024-05-27T07:55:42.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.187+0000 7f83dfe00700 1 -- start start 2024-05-27T07:55:42.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.187+0000 7f83dfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83e01043f0 0x7f83e019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.187+0000 7f83dfe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83e0108d20 0x7f83e019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.187+0000 7f83dfe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e0109670 0x7f83e01a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.187+0000 7f83dfe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01141a0 con 0x7f83e01043f0 2024-05-27T07:55:42.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.187+0000 7f83dfe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0114020 con 0x7f83e0108d20 2024-05-27T07:55:42.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.187+0000 7f83dfe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0114320 con 0x7f83e0109670 2024-05-27T07:55:42.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.188+0000 7f83dea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83e01043f0 0x7f83e019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.188+0000 7f83dea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83e01043f0 0x7f83e019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59836/0 (socket says 172.21.15.63:59836) 2024-05-27T07:55:42.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.188+0000 7f83dea00700 1 -- 172.21.15.63:0/1612565612 learned_addr learned my addr 172.21.15.63:0/1612565612 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:42.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.188+0000 7f83de000700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83e0108d20 0x7f83e019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.188+0000 7f83df400700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e0109670 0x7f83e01a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.188+0000 7f83dea00700 1 -- 172.21.15.63:0/1612565612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e0109670 msgr2=0x7f83e01a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.188+0000 7f83dea00700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e0109670 0x7f83e01a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.188+0000 7f83dea00700 1 -- 172.21.15.63:0/1612565612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83e0108d20 msgr2=0x7f83e019d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.188+0000 7f83dea00700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83e0108d20 0x7f83e019d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.188+0000 7f83dea00700 1 -- 172.21.15.63:0/1612565612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e01a1c80 con 0x7f83e01043f0 2024-05-27T07:55:42.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.189+0000 7f83dea00700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83e01043f0 0x7f83e019cdb0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f83d0000c00 tx=0x7f83d000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.189+0000 7f83d7400700 1 -- 172.21.15.63:0/1612565612 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0027070 con 0x7f83e01043f0 2024-05-27T07:55:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.189+0000 7f83dfe00700 1 -- 172.21.15.63:0/1612565612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e01a1f10 con 0x7f83e01043f0 2024-05-27T07:55:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.189+0000 7f83d7400700 1 -- 172.21.15.63:0/1612565612 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83d000fda0 con 0x7f83e01043f0 2024-05-27T07:55:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.189+0000 7f83d7400700 1 -- 172.21.15.63:0/1612565612 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0022e50 con 0x7f83e01043f0 2024-05-27T07:55:42.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.190+0000 7f83dfe00700 1 -- 172.21.15.63:0/1612565612 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e01a23f0 con 0x7f83e01043f0 2024-05-27T07:55:42.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.194+0000 7f83d7400700 1 -- 172.21.15.63:0/1612565612 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f83d002a070 con 0x7f83e01043f0 2024-05-27T07:55:42.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.196+0000 7f83f4216b80 1 Processor -- start 2024-05-27T07:55:42.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.196+0000 7f83f4216b80 1 -- start start 2024-05-27T07:55:42.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.196+0000 7f83d7400700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83c40b2b90 0x7f83c40b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.196+0000 7f83d7400700 1 -- 172.21.15.63:0/1612565612 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f83d00e3620 con 0x7f83e01043f0 2024-05-27T07:55:42.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 0x55aac7a38750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac78b8e40 0x55aac7a38c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83f4216b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aac7965e40 con 0x55aac7a422c0 2024-05-27T07:55:42.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83f4216b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aac79559e0 con 0x55aac79a49f0 2024-05-27T07:55:42.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83f4216b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55aac7959a20 con 0x55aac78b8e40 2024-05-27T07:55:42.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83df400700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83c40b2b90 0x7f83c40b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59848/0 (socket says 172.21.15.63:59848) 2024-05-27T07:55:42.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83df400700 1 -- 172.21.15.63:0/1514154313 learned_addr learned my addr 172.21.15.63:0/1514154313 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:42.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83dea00700 1 --2- 172.21.15.63:0/1514154313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 0x55aac7a38750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.197+0000 7f83de000700 1 --2- 172.21.15.63:0/1514154313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac78b8e40 0x55aac7a38c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.198+0000 7f83df400700 1 -- 172.21.15.63:0/1514154313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac78b8e40 msgr2=0x55aac7a38c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:42.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.198+0000 7f83df400700 1 --2- 172.21.15.63:0/1514154313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac78b8e40 0x55aac7a38c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.198+0000 7f83df400700 1 -- 172.21.15.63:0/1514154313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 msgr2=0x55aac7a38750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:42.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.198+0000 7f83df400700 1 --2- 172.21.15.63:0/1514154313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 0x55aac7a38750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.198+0000 7f83df400700 1 -- 172.21.15.63:0/1514154313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aac791e420 con 0x55aac7a422c0 2024-05-27T07:55:42.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.198+0000 7f83dea00700 1 --2- 172.21.15.63:0/1514154313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 0x55aac7a38750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:42.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.198+0000 7f83df400700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83c40b2b90 0x7f83c40b50a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f83d80076f0 tx=0x7f83d8009f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:42.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.198+0000 7f83df400700 1 --2- 172.21.15.63:0/1514154313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a446e0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009260 tx=0x7f83d801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9623ca11b6f9aa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:42.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.199+0000 7f83cbe00700 1 -- 172.21.15.63:0/1514154313 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8028070 con 0x55aac7a422c0 2024-05-27T07:55:42.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.199+0000 7f83cbe00700 1 -- 172.21.15.63:0/1514154313 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83d8023af0 con 0x55aac7a422c0 2024-05-27T07:55:42.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.199+0000 7f83cbe00700 1 -- 172.21.15.63:0/1514154313 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8012040 con 0x55aac7a422c0 2024-05-27T07:55:42.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.200+0000 7f83f4216b80 1 -- 172.21.15.63:0/1514154313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 msgr2=0x55aac7a446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:42.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.200+0000 7f83f4216b80 1 --2- 172.21.15.63:0/1514154313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a446e0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009260 tx=0x7f83d801d000 comp rx=0 tx=0).stop 2024-05-27T07:55:42.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.201+0000 7f83f4216b80 1 -- 172.21.15.63:0/1514154313 shutdown_connections 2024-05-27T07:55:42.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.201+0000 7f83f4216b80 1 --2- 172.21.15.63:0/1514154313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a446e0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.201+0000 7f83f4216b80 1 --2- 172.21.15.63:0/1514154313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 0x55aac7a38750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.201+0000 7f83f4216b80 1 --2- 172.21.15.63:0/1514154313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac78b8e40 0x55aac7a38c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.201+0000 7f83f4216b80 1 -- 172.21.15.63:0/1514154313 >> 172.21.15.63:0/1514154313 conn(0x55aac7952390 msgr2=0x55aac7a33710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:42.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.201+0000 7f83f4216b80 1 -- 172.21.15.63:0/1514154313 shutdown_connections 2024-05-27T07:55:42.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.201+0000 7f83f4216b80 1 -- 172.21.15.63:0/1514154313 wait complete. 2024-05-27T07:55:42.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.201+0000 7f83f4216b80 1 -- start start 2024-05-27T07:55:42.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.202+0000 7f83f4216b80 1 Processor -- start 2024-05-27T07:55:42.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.202+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac78b8e40 0x55aac7a3f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.202+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac79a49f0 0x55aac7a3f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.202+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a3fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.202+0000 7f83f4216b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aac785bfa0 con 0x55aac7a422c0 2024-05-27T07:55:42.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.202+0000 7f83f4216b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aac78f6760 con 0x55aac78b8e40 2024-05-27T07:55:42.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.202+0000 7f83f4216b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55aac7a44da0 con 0x55aac79a49f0 2024-05-27T07:55:42.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.202+0000 7f83f4216b80 10 client.0 _ll_register_callbacks cb 0x55aac78ffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:42.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.203+0000 7f83df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a3fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.203+0000 7f83dea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac78b8e40 0x55aac7a3f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.203+0000 7f83df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a3fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59850/0 (socket says 172.21.15.63:59850) 2024-05-27T07:55:42.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.203+0000 7f83df400700 1 -- 172.21.15.63:0/4185208720 learned_addr learned my addr 172.21.15.63:0/4185208720 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:42.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.203+0000 7f83de000700 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac79a49f0 0x55aac7a3f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.203+0000 7f83df400700 1 -- 172.21.15.63:0/4185208720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac79a49f0 msgr2=0x55aac7a3f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:42.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.203+0000 7f83df400700 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac79a49f0 0x55aac7a3f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.203+0000 7f83df400700 1 -- 172.21.15.63:0/4185208720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac78b8e40 msgr2=0x55aac7a3f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:42.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.203+0000 7f83df400700 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac78b8e40 0x55aac7a3f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.203+0000 7f83df400700 1 -- 172.21.15.63:0/4185208720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aac78a58d0 con 0x55aac7a422c0 2024-05-27T07:55:42.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.204+0000 7f83df400700 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a3fd10 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009160 tx=0x7f83d8023690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde54c7bda6ef388 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:42.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.204+0000 7f83c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:42.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.204+0000 7f83c8c00700 1 -- 172.21.15.63:0/4185208720 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8028040 con 0x55aac7a422c0 2024-05-27T07:55:42.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.204+0000 7f83c8c00700 1 -- 172.21.15.63:0/4185208720 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f83d8004a50 con 0x55aac7a422c0 2024-05-27T07:55:42.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.204+0000 7f83f4216b80 1 -- 172.21.15.63:0/4185208720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d8023af0 con 0x55aac7a422c0 2024-05-27T07:55:42.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.205+0000 7f83c8c00700 1 -- 172.21.15.63:0/4185208720 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f83d803a050 con 0x55aac7a422c0 2024-05-27T07:55:42.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.205+0000 7f83f4216b80 10 client.19650 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:42.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.205+0000 7f83f4216b80 1 -- 172.21.15.63:0/4185208720 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aac7926060 con 0x55aac7a422c0 2024-05-27T07:55:42.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.205+0000 7f83c8c00700 1 -- 172.21.15.63:0/4185208720 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83d8023af0 con 0x55aac7a422c0 2024-05-27T07:55:42.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.205+0000 7f83c8c00700 1 -- 172.21.15.63:0/4185208720 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f83d8018040 con 0x55aac7a422c0 2024-05-27T07:55:42.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.205+0000 7f83f4216b80 10 client.19650 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:42.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.205+0000 7f83f4216b80 10 client.19650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:42.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.205+0000 7f83f4216b80 20 client.19650 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:42.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.205+0000 7f83f4216b80 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55aac7a51ba0 0x55aac7a53fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.206+0000 7f83f4216b80 4 client.19650 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:42.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.206+0000 7f83f4216b80 1 -- 172.21.15.63:0/4185208720 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aac7967d90 con 0x55aac7a51ba0 2024-05-27T07:55:42.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.206+0000 7f83dea00700 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55aac7a51ba0 0x55aac7a53fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.207+0000 7f83dea00700 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55aac7a51ba0 0x55aac7a53fc0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=687120cd947e5747 server_cookie=7cb17cccde2cc268 in_seq=0 out_seq=0 2024-05-27T07:55:42.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.207+0000 7f83c8c00700 10 client.19650 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:42.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.209+0000 7f83c8c00700 1 -- 172.21.15.63:0/4185208720 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55aac7967d90 con 0x55aac7a51ba0 2024-05-27T07:55:42.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.209+0000 7f83c8c00700 10 client.19650 handle_command_reply: tid=0 2024-05-27T07:55:42.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.209+0000 7f83f4216b80 1 client.19650 shutdown 2024-05-27T07:55:42.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.209+0000 7f83f4216b80 1 -- 172.21.15.63:0/4185208720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 msgr2=0x55aac7a3fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:42.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.209+0000 7f83f4216b80 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a3fd10 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009160 tx=0x7f83d8023690 comp rx=0 tx=0).stop 2024-05-27T07:55:42.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.209+0000 7f83f4216b80 1 -- 172.21.15.63:0/4185208720 shutdown_connections 2024-05-27T07:55:42.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.210+0000 7f83f4216b80 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55aac7a51ba0 0x55aac7a53fc0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.210+0000 7f83f4216b80 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a3fd10 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.210+0000 7f83f4216b80 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac78b8e40 0x55aac7a3f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.210+0000 7f83f4216b80 1 --2- 172.21.15.63:0/4185208720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac79a49f0 0x55aac7a3f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.210+0000 7f83c8c00700 0 client.19650 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:42.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.210+0000 7f83f4216b80 1 -- 172.21.15.63:0/4185208720 >> 172.21.15.63:0/4185208720 conn(0x55aac7952390 msgr2=0x55aac7a387f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:42.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.210+0000 7f83f4216b80 1 -- 172.21.15.63:0/4185208720 shutdown_connections 2024-05-27T07:55:42.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.210+0000 7f83f4216b80 1 -- 172.21.15.63:0/4185208720 wait complete. 2024-05-27T07:55:42.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.210+0000 7f83f4216b80 20 client.19650 trim_cache size 0 max 16384 2024-05-27T07:55:42.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.233+0000 7f83f4216b80 1 Processor -- start 2024-05-27T07:55:42.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.233+0000 7f83f4216b80 1 -- start start 2024-05-27T07:55:42.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.234+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac78b8e40 0x55aac7a33740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.234+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 0x55aac7a4de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.234+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a37120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.234+0000 7f83f4216b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01b4d80 con 0x55aac7a422c0 2024-05-27T07:55:42.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.234+0000 7f83f4216b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01b2610 con 0x55aac79a49f0 2024-05-27T07:55:42.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.234+0000 7f83f4216b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01b0480 con 0x55aac78b8e40 2024-05-27T07:55:42.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.234+0000 7f83df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a37120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.234+0000 7f83df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a37120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59862/0 (socket says 172.21.15.63:59862) 2024-05-27T07:55:42.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.234+0000 7f83df400700 1 -- 172.21.15.63:0/2757841518 learned_addr learned my addr 172.21.15.63:0/2757841518 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.235+0000 7f83dea00700 1 --2- 172.21.15.63:0/2757841518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac78b8e40 0x55aac7a33740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.235+0000 7f83de000700 1 --2- 172.21.15.63:0/2757841518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 0x55aac7a4de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.235+0000 7f83df400700 1 -- 172.21.15.63:0/2757841518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac78b8e40 msgr2=0x55aac7a33740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.235+0000 7f83df400700 1 --2- 172.21.15.63:0/2757841518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac78b8e40 0x55aac7a33740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.235+0000 7f83df400700 1 -- 172.21.15.63:0/2757841518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 msgr2=0x55aac7a4de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.235+0000 7f83df400700 1 --2- 172.21.15.63:0/2757841518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 0x55aac7a4de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.235+0000 7f83df400700 1 -- 172.21.15.63:0/2757841518 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aac7a39d60 con 0x55aac7a422c0 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.235+0000 7f83df400700 1 --2- 172.21.15.63:0/2757841518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a37120 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009010 tx=0x7f83d80186b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7264ce2d26a65d97 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.236+0000 7f83cb400700 1 -- 172.21.15.63:0/2757841518 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8043070 con 0x55aac7a422c0 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.236+0000 7f83cb400700 1 -- 172.21.15.63:0/2757841518 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83d8031070 con 0x55aac7a422c0 2024-05-27T07:55:42.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.236+0000 7f83cb400700 1 -- 172.21.15.63:0/2757841518 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8028040 con 0x55aac7a422c0 2024-05-27T07:55:42.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.237+0000 7f83f4216b80 1 -- 172.21.15.63:0/2757841518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 msgr2=0x55aac7a37120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:42.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.237+0000 7f83f4216b80 1 --2- 172.21.15.63:0/2757841518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a37120 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009010 tx=0x7f83d80186b0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.237+0000 7f83f4216b80 1 -- 172.21.15.63:0/2757841518 shutdown_connections 2024-05-27T07:55:42.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.237+0000 7f83f4216b80 1 --2- 172.21.15.63:0/2757841518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a37120 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.237+0000 7f83f4216b80 1 --2- 172.21.15.63:0/2757841518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac78b8e40 0x55aac7a33740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.237+0000 7f83f4216b80 1 --2- 172.21.15.63:0/2757841518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac79a49f0 0x55aac7a4de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.237+0000 7f83f4216b80 1 -- 172.21.15.63:0/2757841518 >> 172.21.15.63:0/2757841518 conn(0x55aac7952390 msgr2=0x55aac7a3faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:42.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.237+0000 7f83f4216b80 1 -- 172.21.15.63:0/2757841518 shutdown_connections 2024-05-27T07:55:42.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.237+0000 7f83f4216b80 1 -- 172.21.15.63:0/2757841518 wait complete. 2024-05-27T07:55:42.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.238+0000 7f83f4216b80 1 -- start start 2024-05-27T07:55:42.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.238+0000 7f83f4216b80 1 Processor -- start 2024-05-27T07:55:42.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac78b8e40 0x55aac7a45de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac79a49f0 0x55aac7a46320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83f4216b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a46860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83f4216b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01b0ad0 con 0x55aac7a422c0 2024-05-27T07:55:42.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83f4216b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01b1f40 con 0x55aac78b8e40 2024-05-27T07:55:42.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83f4216b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01b11a0 con 0x55aac79a49f0 2024-05-27T07:55:42.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83f4216b80 10 client.0 _ll_register_callbacks cb 0x55aac78ffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:42.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a46860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a46860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59878/0 (socket says 172.21.15.63:59878) 2024-05-27T07:55:42.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83df400700 1 -- 172.21.15.63:0/2561421896 learned_addr learned my addr 172.21.15.63:0/2561421896 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:42.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83de000700 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac79a49f0 0x55aac7a46320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.239+0000 7f83dea00700 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac78b8e40 0x55aac7a45de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.240+0000 7f83df400700 1 -- 172.21.15.63:0/2561421896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac79a49f0 msgr2=0x55aac7a46320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:42.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.240+0000 7f83df400700 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac79a49f0 0x55aac7a46320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.240+0000 7f83df400700 1 -- 172.21.15.63:0/2561421896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac78b8e40 msgr2=0x55aac7a45de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:42.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.240+0000 7f83df400700 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac78b8e40 0x55aac7a45de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.240+0000 7f83df400700 1 -- 172.21.15.63:0/2561421896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aac7a39390 con 0x55aac7a422c0 2024-05-27T07:55:42.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.240+0000 7f83df400700 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a46860 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009160 tx=0x7f83d8005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6a2a6cde67d46c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:42.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.240+0000 7f83c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:42.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.241+0000 7f83c8c00700 1 -- 172.21.15.63:0/2561421896 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8043030 con 0x55aac7a422c0 2024-05-27T07:55:42.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.241+0000 7f83c8c00700 1 -- 172.21.15.63:0/2561421896 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f83d8004de0 con 0x55aac7a422c0 2024-05-27T07:55:42.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.241+0000 7f83f4216b80 1 -- 172.21.15.63:0/2561421896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d8031070 con 0x55aac7a422c0 2024-05-27T07:55:42.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.241+0000 7f83c8c00700 1 -- 172.21.15.63:0/2561421896 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f83d8050050 con 0x55aac7a422c0 2024-05-27T07:55:42.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.241+0000 7f83f4216b80 10 client.19656 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:42.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.241+0000 7f83f4216b80 1 -- 172.21.15.63:0/2561421896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aac7abc590 con 0x55aac7a422c0 2024-05-27T07:55:42.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.241+0000 7f83c8c00700 1 -- 172.21.15.63:0/2561421896 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83d8031070 con 0x55aac7a422c0 2024-05-27T07:55:42.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.242+0000 7f83c8c00700 1 -- 172.21.15.63:0/2561421896 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f83d8023b90 con 0x55aac7a422c0 2024-05-27T07:55:42.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.242+0000 7f83f4216b80 10 client.19656 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:42.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.242+0000 7f83f4216b80 10 client.19656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:42.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.242+0000 7f83f4216b80 20 client.19656 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:42.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.242+0000 7f83f4216b80 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55aac7abcbd0 0x55aac7abeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:42.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.242+0000 7f83f4216b80 4 client.19656 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:55:42.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.242+0000 7f83f4216b80 1 -- 172.21.15.63:0/2561421896 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aac7961570 con 0x55aac7abcbd0 2024-05-27T07:55:42.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.242+0000 7f83dea00700 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55aac7abcbd0 0x55aac7abeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.243+0000 7f83dea00700 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55aac7abcbd0 0x55aac7abeff0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca111978a8fd6617 server_cookie=367feee23839b02d in_seq=0 out_seq=0 2024-05-27T07:55:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.243+0000 7f83c8c00700 10 client.19656 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83c8c00700 1 -- 172.21.15.63:0/2561421896 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aac7961570 con 0x55aac7abcbd0 2024-05-27T07:55:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83c8c00700 10 client.19656 handle_command_reply: tid=0 2024-05-27T07:55:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83f4216b80 1 client.19656 shutdown 2024-05-27T07:55:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83f4216b80 1 -- 172.21.15.63:0/2561421896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 msgr2=0x55aac7a46860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83f4216b80 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a46860 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009160 tx=0x7f83d8005f80 comp rx=0 tx=0).stop 2024-05-27T07:55:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83f4216b80 1 -- 172.21.15.63:0/2561421896 shutdown_connections 2024-05-27T07:55:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83f4216b80 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55aac7abcbd0 0x55aac7abeff0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83f4216b80 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aac7a422c0 0x55aac7a46860 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83f4216b80 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aac78b8e40 0x55aac7a45de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83f4216b80 1 --2- 172.21.15.63:0/2561421896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aac79a49f0 0x55aac7a46320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83c8c00700 0 client.19656 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:42.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.244+0000 7f83f4216b80 1 -- 172.21.15.63:0/2561421896 >> 172.21.15.63:0/2561421896 conn(0x55aac7952390 msgr2=0x55aac7a3faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:42.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.245+0000 7f83f4216b80 1 -- 172.21.15.63:0/2561421896 shutdown_connections 2024-05-27T07:55:42.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.245+0000 7f83f4216b80 1 -- 172.21.15.63:0/2561421896 wait complete. 2024-05-27T07:55:42.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.245+0000 7f83f4216b80 20 client.19656 trim_cache size 0 max 16384 2024-05-27T07:55:42.248 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:55:42.248 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:55:42.248 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:55:42.248 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:55:42.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.246+0000 7f83dfe00700 1 -- 172.21.15.63:0/1612565612 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83c40b2b90 msgr2=0x7f83c40b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:42.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.246+0000 7f83dfe00700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83c40b2b90 0x7f83c40b50a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f83d80076f0 tx=0x7f83d8009f90 comp rx=0 tx=0).stop 2024-05-27T07:55:42.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.246+0000 7f83dfe00700 1 -- 172.21.15.63:0/1612565612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83e01043f0 msgr2=0x7f83e019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:42.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.246+0000 7f83dfe00700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83e01043f0 0x7f83e019cdb0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f83d0000c00 tx=0x7f83d000f460 comp rx=0 tx=0).stop 2024-05-27T07:55:42.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.246+0000 7f83dfe00700 1 -- 172.21.15.63:0/1612565612 shutdown_connections 2024-05-27T07:55:42.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.246+0000 7f83dfe00700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83c40b2b90 0x7f83c40b50a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.246+0000 7f83dfe00700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83e0109670 0x7f83e01a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.246+0000 7f83dfe00700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83e01043f0 0x7f83e019cdb0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.246+0000 7f83dfe00700 1 --2- 172.21.15.63:0/1612565612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83e0108d20 0x7f83e019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:42.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.246+0000 7f83dfe00700 1 -- 172.21.15.63:0/1612565612 >> 172.21.15.63:0/1612565612 conn(0x7f83e0079d00 msgr2=0x7f83e0110d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:42.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.247+0000 7f83dfe00700 1 -- 172.21.15.63:0/1612565612 shutdown_connections 2024-05-27T07:55:42.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:42.247+0000 7f83dfe00700 1 -- 172.21.15.63:0/1612565612 wait complete. 2024-05-27T07:55:43.188 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:55:43.189 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:55:43.694 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:43.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:43 smithi086 ceph-mon[34687]: pgmap v699: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:55:43.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:43 smithi149 ceph-mon[35408]: pgmap v699: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:55:43.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:43 smithi063 ceph-mon[31613]: pgmap v699: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:55:45.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:45 smithi063 ceph-mon[31613]: pgmap v700: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:55:45.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:45 smithi086 ceph-mon[34687]: pgmap v700: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:55:45.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:45 smithi149 ceph-mon[35408]: pgmap v700: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:55:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.046+0000 7f710a800640 1 -- 172.21.15.63:0/1553550788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710410d340 msgr2=0x7f710410f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.046+0000 7f710a800640 1 --2- 172.21.15.63:0/1553550788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710410d340 0x7f710410f760 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009ca0 tx=0x7f70fc02a8e0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.046+0000 7f710a800640 1 -- 172.21.15.63:0/1553550788 shutdown_connections 2024-05-27T07:55:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.046+0000 7f710a800640 1 --2- 172.21.15.63:0/1553550788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f710410d340 0x7f710410f760 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.046+0000 7f710a800640 1 --2- 172.21.15.63:0/1553550788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7104102ab0 0x7f7104102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.046+0000 7f710a800640 1 --2- 172.21.15.63:0/1553550788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71041089e0 0x7f7104102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.046+0000 7f710a800640 1 -- 172.21.15.63:0/1553550788 >> 172.21.15.63:0/1553550788 conn(0x7f71040fe210 msgr2=0x7f7104100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.046+0000 7f710a800640 1 -- 172.21.15.63:0/1553550788 shutdown_connections 2024-05-27T07:55:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.046+0000 7f710a800640 1 -- 172.21.15.63:0/1553550788 wait complete. 2024-05-27T07:55:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.047+0000 7f710a800640 1 Processor -- start 2024-05-27T07:55:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.048+0000 7f710a800640 1 -- start start 2024-05-27T07:55:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.048+0000 7f710a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7104102ab0 0x7f71041a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.048+0000 7f710a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71041089e0 0x7f71041a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.048+0000 7f710a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710410d340 0x7f710419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.048+0000 7f710a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7104112ad0 con 0x7f7104102ab0 2024-05-27T07:55:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.048+0000 7f710a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7104112950 con 0x7f710410d340 2024-05-27T07:55:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.048+0000 7f710a800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7104112c50 con 0x7f71041089e0 2024-05-27T07:55:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.049+0000 7f7109400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7104102ab0 0x7f71041a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.049+0000 7f7109400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7104102ab0 0x7f71041a2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59896/0 (socket says 172.21.15.63:59896) 2024-05-27T07:55:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.049+0000 7f7109400640 1 -- 172.21.15.63:0/2961396975 learned_addr learned my addr 172.21.15.63:0/2961396975 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.049+0000 7f7109e00640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710410d340 0x7f710419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.049+0000 7f7108a00640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71041089e0 0x7f71041a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.049+0000 7f7109400640 1 -- 172.21.15.63:0/2961396975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71041089e0 msgr2=0x7f71041a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.049+0000 7f7109400640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71041089e0 0x7f71041a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.049+0000 7f7109400640 1 -- 172.21.15.63:0/2961396975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710410d340 msgr2=0x7f710419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.049+0000 7f7109400640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710410d340 0x7f710419c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.049+0000 7f7109400640 1 -- 172.21.15.63:0/2961396975 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710419cd30 con 0x7f7104102ab0 2024-05-27T07:55:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.050+0000 7f7109400640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7104102ab0 0x7f71041a2490 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f70f800d770 tx=0x7f70f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.050+0000 7f7102000640 1 -- 172.21.15.63:0/2961396975 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8017070 con 0x7f7104102ab0 2024-05-27T07:55:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.050+0000 7f7102000640 1 -- 172.21.15.63:0/2961396975 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70f8009c30 con 0x7f7104102ab0 2024-05-27T07:55:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.050+0000 7f7102000640 1 -- 172.21.15.63:0/2961396975 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f800f620 con 0x7f7104102ab0 2024-05-27T07:55:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.050+0000 7f710a800640 1 -- 172.21.15.63:0/2961396975 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710419d020 con 0x7f7104102ab0 2024-05-27T07:55:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.050+0000 7f710a800640 1 -- 172.21.15.63:0/2961396975 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71041a9230 con 0x7f7104102ab0 2024-05-27T07:55:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.054+0000 7f7112db7740 1 Processor -- start 2024-05-27T07:55:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.054+0000 7f7112db7740 1 -- start start 2024-05-27T07:55:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.055+0000 7f7112db7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6740061b0 0x55c6743a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.055+0000 7f7112db7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6743a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.055+0000 7f7112db7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 0x55c6743a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.055+0000 7f7112db7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c674319b90 con 0x55c673fdd2f0 2024-05-27T07:55:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.055+0000 7f7112db7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c674309730 con 0x55c6740061b0 2024-05-27T07:55:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.055+0000 7f7112db7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c67430d770 con 0x55c673fe54d0 2024-05-27T07:55:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.055+0000 7f7102000640 1 -- 172.21.15.63:0/2961396975 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f70f801a070 con 0x7f7104102ab0 2024-05-27T07:55:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7108a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 0x55c6743a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7109e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6740061b0 0x55c6743a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7109400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6743a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7108a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 0x55c6743a94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:56028/0 (socket says 172.21.15.63:56028) 2024-05-27T07:55:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7109e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6740061b0 0x55c6743a8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44824/0 (socket says 172.21.15.63:44824) 2024-05-27T07:55:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7108a00640 1 -- 172.21.15.63:0/1811387222 learned_addr learned my addr 172.21.15.63:0/1811387222 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7109400640 1 -- 172.21.15.63:0/1811387222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 msgr2=0x55c6743a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7109400640 1 --2- 172.21.15.63:0/1811387222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 0x55c6743a94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7109400640 1 -- 172.21.15.63:0/1811387222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6740061b0 msgr2=0x55c6743a8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7109400640 1 --2- 172.21.15.63:0/1811387222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6740061b0 0x55c6743a8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.056+0000 7f7109400640 1 -- 172.21.15.63:0/1811387222 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6742a28b0 con 0x55c673fdd2f0 2024-05-27T07:55:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.057+0000 7f7109400640 1 --2- 172.21.15.63:0/1811387222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6743a8fa0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f70f80df910 tx=0x7f70f80e8bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da47f809ecbec4a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.057+0000 7f70f6600640 1 -- 172.21.15.63:0/1811387222 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f80f5070 con 0x55c673fdd2f0 2024-05-27T07:55:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.057+0000 7f70f6600640 1 -- 172.21.15.63:0/1811387222 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70f80e8e40 con 0x55c673fdd2f0 2024-05-27T07:55:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.057+0000 7f70f6600640 1 -- 172.21.15.63:0/1811387222 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f80e4410 con 0x55c673fdd2f0 2024-05-27T07:55:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.057+0000 7f7102000640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70ec0b2950 0x7f70ec0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.058+0000 7f7109e00640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70ec0b2950 0x7f70ec0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.058+0000 7f7102000640 1 -- 172.21.15.63:0/2961396975 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f70f80e2580 con 0x7f7104102ab0 2024-05-27T07:55:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.058+0000 7f7112db7740 1 -- 172.21.15.63:0/1811387222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 msgr2=0x55c6743a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.058+0000 7f7112db7740 1 --2- 172.21.15.63:0/1811387222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6743a8fa0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f70f80df910 tx=0x7f70f80e8bf0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.058+0000 7f7112db7740 1 -- 172.21.15.63:0/1811387222 shutdown_connections 2024-05-27T07:55:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.058+0000 7f7112db7740 1 --2- 172.21.15.63:0/1811387222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 0x55c6743a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.058+0000 7f7112db7740 1 --2- 172.21.15.63:0/1811387222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6743a8fa0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.058+0000 7f7112db7740 1 --2- 172.21.15.63:0/1811387222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6740061b0 0x55c6743a8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.059+0000 7f7112db7740 1 -- 172.21.15.63:0/1811387222 >> 172.21.15.63:0/1811387222 conn(0x55c673ffa300 msgr2=0x55c6743a5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.059+0000 7f7109e00640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70ec0b2950 0x7f70ec0b4e60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55c6743a5bd0 tx=0x7f70fc005c30 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.059+0000 7f7112db7740 1 -- 172.21.15.63:0/1811387222 shutdown_connections 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.059+0000 7f7112db7740 1 -- 172.21.15.63:0/1811387222 wait complete. 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.059+0000 7f7112db7740 1 -- start start 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.060+0000 7f7112db7740 1 Processor -- start 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.060+0000 7f7112db7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fdd2f0 0x55c6743b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.060+0000 7f7112db7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fe54d0 0x55c6743b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.060+0000 7f7112db7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743beec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.060+0000 7f7112db7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c673fef790 con 0x55c6740061b0 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.060+0000 7f7112db7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6742b1bc0 con 0x55c673fe54d0 2024-05-27T07:55:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.060+0000 7f7112db7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c6743b5b90 con 0x55c673fdd2f0 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.060+0000 7f7112db7740 10 client.0 _ll_register_callbacks cb 0x55c67428f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.061+0000 7f7109e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743beec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.061+0000 7f7109e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743beec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59926/0 (socket says 172.21.15.63:59926) 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.061+0000 7f7109e00640 1 -- 172.21.15.63:0/4221188886 learned_addr learned my addr 172.21.15.63:0/4221188886 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.061+0000 7f7108a00640 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fe54d0 0x55c6743b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.061+0000 7f7109400640 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fdd2f0 0x55c6743b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.061+0000 7f7109e00640 1 -- 172.21.15.63:0/4221188886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fdd2f0 msgr2=0x55c6743b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.061+0000 7f7109e00640 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fdd2f0 0x55c6743b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.061+0000 7f7109e00640 1 -- 172.21.15.63:0/4221188886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fe54d0 msgr2=0x55c6743b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.061+0000 7f7109e00640 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fe54d0 0x55c6743b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.061+0000 7f7109e00640 1 -- 172.21.15.63:0/4221188886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67401e210 con 0x55c6740061b0 2024-05-27T07:55:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.062+0000 7f7109e00640 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743beec0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f70fc035000 tx=0x7f70fc02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ad76fbc00967c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.062+0000 7f70eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.062+0000 7f70eaa00640 1 -- 172.21.15.63:0/4221188886 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0047d0 con 0x55c6740061b0 2024-05-27T07:55:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.062+0000 7f70eaa00640 1 -- 172.21.15.63:0/4221188886 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f70fc0305e0 con 0x55c6740061b0 2024-05-27T07:55:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.062+0000 7f7112db7740 1 -- 172.21.15.63:0/4221188886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c67431bae0 con 0x55c6740061b0 2024-05-27T07:55:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.063+0000 7f70eaa00640 1 -- 172.21.15.63:0/4221188886 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f70fc0307e0 con 0x55c6740061b0 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.063+0000 7f7112db7740 10 client.19668 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.063+0000 7f7112db7740 1 -- 172.21.15.63:0/4221188886 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c6740ea620 con 0x55c6740061b0 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.063+0000 7f70eaa00640 1 -- 172.21.15.63:0/4221188886 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c67431bae0 con 0x55c6740061b0 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.063+0000 7f70eaa00640 1 -- 172.21.15.63:0/4221188886 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f70fc050070 con 0x55c6740061b0 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.063+0000 7f7112db7740 10 client.19668 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.063+0000 7f7112db7740 10 client.19668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.064+0000 7f7112db7740 20 client.19668 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.064+0000 7f7112db7740 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c673fc4d80 0x55c6743c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.064+0000 7f7112db7740 4 client.19668 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.064+0000 7f7112db7740 1 -- 172.21.15.63:0/4221188886 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6743152c0 con 0x55c673fc4d80 2024-05-27T07:55:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.064+0000 7f7109400640 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c673fc4d80 0x55c6743c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.065+0000 7f7109400640 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c673fc4d80 0x55c6743c4f40 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f59fa03daf31570 server_cookie=34759df659d585dd in_seq=0 out_seq=0 2024-05-27T07:55:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.065+0000 7f70eaa00640 10 client.19668 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.067+0000 7f70eaa00640 1 -- 172.21.15.63:0/4221188886 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c6743152c0 con 0x55c673fc4d80 2024-05-27T07:55:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.067+0000 7f70eaa00640 10 client.19668 handle_command_reply: tid=0 2024-05-27T07:55:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.067+0000 7f7112db7740 1 client.19668 shutdown 2024-05-27T07:55:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.068+0000 7f7112db7740 1 -- 172.21.15.63:0/4221188886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 msgr2=0x55c6743beec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.068+0000 7f7112db7740 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743beec0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f70fc035000 tx=0x7f70fc02a8e0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.069+0000 7f7112db7740 1 -- 172.21.15.63:0/4221188886 shutdown_connections 2024-05-27T07:55:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.069+0000 7f7112db7740 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c673fc4d80 0x55c6743c4f40 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.069+0000 7f7112db7740 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743beec0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.069+0000 7f7112db7740 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fe54d0 0x55c6743b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.069+0000 7f7112db7740 1 --2- 172.21.15.63:0/4221188886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fdd2f0 0x55c6743b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.069+0000 7f7112db7740 1 -- 172.21.15.63:0/4221188886 >> 172.21.15.63:0/4221188886 conn(0x55c673ffa300 msgr2=0x55c6743aa780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.069+0000 7f70eaa00640 0 client.19668 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.069+0000 7f7112db7740 1 -- 172.21.15.63:0/4221188886 shutdown_connections 2024-05-27T07:55:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.069+0000 7f7112db7740 1 -- 172.21.15.63:0/4221188886 wait complete. 2024-05-27T07:55:46.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.069+0000 7f7112db7740 20 client.19668 trim_cache size 0 max 16384 2024-05-27T07:55:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.091+0000 7f7112db7740 1 Processor -- start 2024-05-27T07:55:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.091+0000 7f7112db7740 1 -- start start 2024-05-27T07:55:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.091+0000 7f7112db7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fdd2f0 0x55c6743b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.091+0000 7f7112db7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 0x55c6743bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.091+0000 7f7112db7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7112db7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f71041b5b60 con 0x55c6740061b0 2024-05-27T07:55:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7112db7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71041b3430 con 0x55c673fdd2f0 2024-05-27T07:55:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7112db7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f71041b12a0 con 0x55c673fe54d0 2024-05-27T07:55:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7109e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7109e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743bf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59932/0 (socket says 172.21.15.63:59932) 2024-05-27T07:55:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7109e00640 1 -- 172.21.15.63:0/2019632876 learned_addr learned my addr 172.21.15.63:0/2019632876 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7109400640 1 --2- 172.21.15.63:0/2019632876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fdd2f0 0x55c6743b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7108a00640 1 --2- 172.21.15.63:0/2019632876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 0x55c6743bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7109e00640 1 -- 172.21.15.63:0/2019632876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 msgr2=0x55c6743bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7109e00640 1 --2- 172.21.15.63:0/2019632876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 0x55c6743bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7109e00640 1 -- 172.21.15.63:0/2019632876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fdd2f0 msgr2=0x55c6743b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7109e00640 1 --2- 172.21.15.63:0/2019632876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fdd2f0 0x55c6743b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.092+0000 7f7109e00640 1 -- 172.21.15.63:0/2019632876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6743a87f0 con 0x55c6740061b0 2024-05-27T07:55:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.093+0000 7f7109e00640 1 --2- 172.21.15.63:0/2019632876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743bf170 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f70fc006010 tx=0x7f70fc003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913901032601345f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.093+0000 7f70f5c00640 1 -- 172.21.15.63:0/2019632876 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc031470 con 0x55c6740061b0 2024-05-27T07:55:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.093+0000 7f70f5c00640 1 -- 172.21.15.63:0/2019632876 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70fc036040 con 0x55c6740061b0 2024-05-27T07:55:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.093+0000 7f70f5c00640 1 -- 172.21.15.63:0/2019632876 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc030980 con 0x55c6740061b0 2024-05-27T07:55:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.094+0000 7f7112db7740 1 -- 172.21.15.63:0/2019632876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 msgr2=0x55c6743bf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.094+0000 7f7112db7740 1 --2- 172.21.15.63:0/2019632876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743bf170 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f70fc006010 tx=0x7f70fc003ec0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.095+0000 7f7112db7740 1 -- 172.21.15.63:0/2019632876 shutdown_connections 2024-05-27T07:55:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.095+0000 7f7112db7740 1 --2- 172.21.15.63:0/2019632876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c6740061b0 0x55c6743bf170 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.095+0000 7f7112db7740 1 --2- 172.21.15.63:0/2019632876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c673fe54d0 0x55c6743bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.095+0000 7f7112db7740 1 --2- 172.21.15.63:0/2019632876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fdd2f0 0x55c6743b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.095+0000 7f7112db7740 1 -- 172.21.15.63:0/2019632876 >> 172.21.15.63:0/2019632876 conn(0x55c673ffa300 msgr2=0x55c6743b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.095+0000 7f7112db7740 1 -- 172.21.15.63:0/2019632876 shutdown_connections 2024-05-27T07:55:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.095+0000 7f7112db7740 1 -- 172.21.15.63:0/2019632876 wait complete. 2024-05-27T07:55:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.095+0000 7f7112db7740 1 -- start start 2024-05-27T07:55:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.096+0000 7f7112db7740 1 Processor -- start 2024-05-27T07:55:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.096+0000 7f7112db7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6744259c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.096+0000 7f7112db7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fe54d0 0x55c674425f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.096+0000 7f7112db7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6740061b0 0x55c6743b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.096+0000 7f7112db7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f71041b18f0 con 0x55c673fdd2f0 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.096+0000 7f7112db7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71041b2d60 con 0x55c673fe54d0 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.096+0000 7f7112db7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f71041b1fc0 con 0x55c6740061b0 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7112db7740 10 client.0 _ll_register_callbacks cb 0x7f70fc031660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7109400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6744259c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7109400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6744259c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59936/0 (socket says 172.21.15.63:59936) 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7109400640 1 -- 172.21.15.63:0/1904719690 learned_addr learned my addr 172.21.15.63:0/1904719690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7108a00640 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fe54d0 0x55c674425f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7109e00640 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6740061b0 0x55c6743b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7109400640 1 -- 172.21.15.63:0/1904719690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6740061b0 msgr2=0x55c6743b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7109400640 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6740061b0 0x55c6743b63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7109400640 1 -- 172.21.15.63:0/1904719690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fe54d0 msgr2=0x55c674425f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7109400640 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fe54d0 0x55c674425f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.097+0000 7f7109400640 1 -- 172.21.15.63:0/1904719690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6743b69d0 con 0x55c673fdd2f0 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.098+0000 7f7109400640 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6744259c0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f70f80df9c0 tx=0x7f70f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5330a9167298bfef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.098+0000 7f70eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.098+0000 7f70eaa00640 1 -- 172.21.15.63:0/1904719690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8021040 con 0x55c673fdd2f0 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.098+0000 7f70eaa00640 1 -- 172.21.15.63:0/1904719690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f70f80dfb90 con 0x55c673fdd2f0 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.098+0000 7f7112db7740 1 -- 172.21.15.63:0/1904719690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70fc036040 con 0x55c673fdd2f0 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.098+0000 7f70eaa00640 1 -- 172.21.15.63:0/1904719690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f70f80a5360 con 0x55c673fdd2f0 2024-05-27T07:55:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.098+0000 7f7112db7740 10 client.19674 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.098+0000 7f7112db7740 1 -- 172.21.15.63:0/1904719690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c67442cff0 con 0x55c673fdd2f0 2024-05-27T07:55:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.099+0000 7f70eaa00640 1 -- 172.21.15.63:0/1904719690 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70fc036040 con 0x55c673fdd2f0 2024-05-27T07:55:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.099+0000 7f70eaa00640 1 -- 172.21.15.63:0/1904719690 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f70f80a5730 con 0x55c673fdd2f0 2024-05-27T07:55:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.099+0000 7f7112db7740 10 client.19674 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.099+0000 7f7112db7740 10 client.19674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.099+0000 7f7112db7740 20 client.19674 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.099+0000 7f7112db7740 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c673f9a1c0 0x55c67442f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.099+0000 7f7112db7740 4 client.19674 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:55:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.099+0000 7f7112db7740 1 -- 172.21.15.63:0/1904719690 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c67428f4b0 con 0x55c673f9a1c0 2024-05-27T07:55:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.100+0000 7f7108a00640 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c673f9a1c0 0x55c67442f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.101+0000 7f7108a00640 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c673f9a1c0 0x55c67442f2e0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=119f7b29b4202a39 server_cookie=61652e416dec4def in_seq=0 out_seq=0 2024-05-27T07:55:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.101+0000 7f70eaa00640 10 client.19674 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f70eaa00640 1 -- 172.21.15.63:0/1904719690 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c67428f4b0 con 0x55c673f9a1c0 2024-05-27T07:55:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f70eaa00640 10 client.19674 handle_command_reply: tid=0 2024-05-27T07:55:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 client.19674 shutdown 2024-05-27T07:55:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 -- 172.21.15.63:0/1904719690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 msgr2=0x55c6744259c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6744259c0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f70f80df9c0 tx=0x7f70f80057f0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 -- 172.21.15.63:0/1904719690 shutdown_connections 2024-05-27T07:55:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c673f9a1c0 0x55c67442f2e0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c6740061b0 0x55c6743b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c673fe54d0 0x55c674425f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 --2- 172.21.15.63:0/1904719690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c673fdd2f0 0x55c6744259c0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 -- 172.21.15.63:0/1904719690 >> 172.21.15.63:0/1904719690 conn(0x55c673ffa300 msgr2=0x55c6743a89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f70eaa00640 0 client.19674 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 -- 172.21.15.63:0/1904719690 shutdown_connections 2024-05-27T07:55:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.133+0000 7f7112db7740 1 -- 172.21.15.63:0/1904719690 wait complete. 2024-05-27T07:55:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.134+0000 7f7112db7740 20 client.19674 trim_cache size 0 max 16384 2024-05-27T07:55:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:55:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:55:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:55:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.134+0000 7f710a800640 1 -- 172.21.15.63:0/2961396975 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70ec0b2950 msgr2=0x7f70ec0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.134+0000 7f710a800640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70ec0b2950 0x7f70ec0b4e60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55c6743a5bd0 tx=0x7f70fc005c30 comp rx=0 tx=0).stop 2024-05-27T07:55:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.134+0000 7f710a800640 1 -- 172.21.15.63:0/2961396975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7104102ab0 msgr2=0x7f71041a2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.134+0000 7f710a800640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7104102ab0 0x7f71041a2490 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f70f800d770 tx=0x7f70f800dc30 comp rx=0 tx=0).stop 2024-05-27T07:55:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.135+0000 7f710a800640 1 -- 172.21.15.63:0/2961396975 shutdown_connections 2024-05-27T07:55:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.135+0000 7f710a800640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f70ec0b2950 0x7f70ec0b4e60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.135+0000 7f710a800640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f710410d340 0x7f710419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.135+0000 7f710a800640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71041089e0 0x7f71041a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.135+0000 7f710a800640 1 --2- 172.21.15.63:0/2961396975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7104102ab0 0x7f71041a2490 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.135+0000 7f710a800640 1 -- 172.21.15.63:0/2961396975 >> 172.21.15.63:0/2961396975 conn(0x7f71040fe210 msgr2=0x7f71041007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.135+0000 7f710a800640 1 -- 172.21.15.63:0/2961396975 shutdown_connections 2024-05-27T07:55:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:46.135+0000 7f710a800640 1 -- 172.21.15.63:0/2961396975 wait complete. 2024-05-27T07:55:46.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.339+0000 7f1f0ee00700 1 -- 172.21.15.63:0/867890169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0810b860 msgr2=0x7f1f081131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.339+0000 7f1f0ee00700 1 --2- 172.21.15.63:0/867890169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0810b860 0x7f1f081131c0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000b0d0 tx=0x7f1f0000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.340+0000 7f1f0ee00700 1 -- 172.21.15.63:0/867890169 shutdown_connections 2024-05-27T07:55:46.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.340+0000 7f1f0ee00700 1 --2- 172.21.15.63:0/867890169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0810b860 0x7f1f081131c0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.340+0000 7f1f0ee00700 1 --2- 172.21.15.63:0/867890169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f08106630 0x7f1f08106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.340+0000 7f1f0ee00700 1 --2- 172.21.15.63:0/867890169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f0810af10 0x7f1f0810b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.340+0000 7f1f0ee00700 1 -- 172.21.15.63:0/867890169 >> 172.21.15.63:0/867890169 conn(0x7f1f08102020 msgr2=0x7f1f08104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:46.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.340+0000 7f1f0ee00700 1 -- 172.21.15.63:0/867890169 shutdown_connections 2024-05-27T07:55:46.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.340+0000 7f1f0ee00700 1 -- 172.21.15.63:0/867890169 wait complete. 2024-05-27T07:55:46.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.341+0000 7f1f0ee00700 1 Processor -- start 2024-05-27T07:55:46.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.341+0000 7f1f0ee00700 1 -- start start 2024-05-27T07:55:46.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.342+0000 7f1f0ee00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08106630 0x7f1f0819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.342+0000 7f1f0ee00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f0810af10 0x7f1f0819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.342+0000 7f1f0ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0810b860 0x7f1f081a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.342+0000 7f1f0ee00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08116400 con 0x7f1f08106630 2024-05-27T07:55:46.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.342+0000 7f1f0ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08116280 con 0x7f1f0810b860 2024-05-27T07:55:46.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.342+0000 7f1f0ee00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08116580 con 0x7f1f0810af10 2024-05-27T07:55:46.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.342+0000 7f1f0da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08106630 0x7f1f0819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.342+0000 7f1f0da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08106630 0x7f1f0819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59956/0 (socket says 172.21.15.63:59956) 2024-05-27T07:55:46.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.342+0000 7f1f0da00700 1 -- 172.21.15.63:0/2539492693 learned_addr learned my addr 172.21.15.63:0/2539492693 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:46.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.343+0000 7f1f0d000700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f0810af10 0x7f1f0819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.343+0000 7f1f0e400700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0810b860 0x7f1f081a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.343+0000 7f1f0da00700 1 -- 172.21.15.63:0/2539492693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f0810af10 msgr2=0x7f1f0819d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.343+0000 7f1f0da00700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f0810af10 0x7f1f0819d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.343+0000 7f1f0da00700 1 -- 172.21.15.63:0/2539492693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0810b860 msgr2=0x7f1f081a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.343+0000 7f1f0da00700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0810b860 0x7f1f081a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.343+0000 7f1f0da00700 1 -- 172.21.15.63:0/2539492693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f081a1bb0 con 0x7f1f08106630 2024-05-27T07:55:46.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.343+0000 7f1f0e400700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0810b860 0x7f1f081a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:46.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.344+0000 7f1f0da00700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08106630 0x7f1f0819cd90 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00c9b0 tx=0x7f1efc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:46.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.344+0000 7f1f06000700 1 -- 172.21.15.63:0/2539492693 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc007b00 con 0x7f1f08106630 2024-05-27T07:55:46.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.344+0000 7f1f0ee00700 1 -- 172.21.15.63:0/2539492693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f081a1ea0 con 0x7f1f08106630 2024-05-27T07:55:46.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.344+0000 7f1f06000700 1 -- 172.21.15.63:0/2539492693 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1efc007ca0 con 0x7f1f08106630 2024-05-27T07:55:46.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.344+0000 7f1f06000700 1 -- 172.21.15.63:0/2539492693 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc018660 con 0x7f1f08106630 2024-05-27T07:55:46.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.344+0000 7f1f0ee00700 1 -- 172.21.15.63:0/2539492693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f081a23e0 con 0x7f1f08106630 2024-05-27T07:55:46.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.346+0000 7f1f0ee00700 1 -- 172.21.15.63:0/2539492693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f080512b0 con 0x7f1f08106630 2024-05-27T07:55:46.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.349+0000 7f1f06000700 1 -- 172.21.15.63:0/2539492693 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1efc00f450 con 0x7f1f08106630 2024-05-27T07:55:46.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.351+0000 7f1f06000700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1ef00b29f0 0x7f1ef00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:46.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.351+0000 7f1f06000700 1 -- 172.21.15.63:0/2539492693 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f1efc0e2dd0 con 0x7f1f08106630 2024-05-27T07:55:46.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.352+0000 7f1f0d000700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1ef00b29f0 0x7f1ef00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:46.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.352+0000 7f1f0d000700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1ef00b29f0 0x7f1ef00b4f00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4006fd0 tx=0x7f1ef4008040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:46.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.354+0000 7f1f06000700 1 -- 172.21.15.63:0/2539492693 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1efc014030 con 0x7f1f08106630 2024-05-27T07:55:46.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.681+0000 7f1f0ee00700 1 -- 172.21.15.63:0/2539492693 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f0802e5e0 con 0x7f1f08106630 2024-05-27T07:55:46.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.682+0000 7f1f06000700 1 -- 172.21.15.63:0/2539492693 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7f1efc0aa420 con 0x7f1f08106630 2024-05-27T07:55:46.683 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:55:46.683 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:55:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.686+0000 7f1f0ee00700 1 -- 172.21.15.63:0/2539492693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1ef00b29f0 msgr2=0x7f1ef00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.686+0000 7f1f0ee00700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1ef00b29f0 0x7f1ef00b4f00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4006fd0 tx=0x7f1ef4008040 comp rx=0 tx=0).stop 2024-05-27T07:55:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.686+0000 7f1f0ee00700 1 -- 172.21.15.63:0/2539492693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08106630 msgr2=0x7f1f0819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.686+0000 7f1f0ee00700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08106630 0x7f1f0819cd90 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00c9b0 tx=0x7f1efc00cd70 comp rx=0 tx=0).stop 2024-05-27T07:55:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.687+0000 7f1f0ee00700 1 -- 172.21.15.63:0/2539492693 shutdown_connections 2024-05-27T07:55:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.687+0000 7f1f0ee00700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1ef00b29f0 0x7f1ef00b4f00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.687+0000 7f1f0ee00700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f0810b860 0x7f1f081a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.687+0000 7f1f0ee00700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f08106630 0x7f1f0819cd90 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.687+0000 7f1f0ee00700 1 --2- 172.21.15.63:0/2539492693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f0810af10 0x7f1f0819d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.687+0000 7f1f0ee00700 1 -- 172.21.15.63:0/2539492693 >> 172.21.15.63:0/2539492693 conn(0x7f1f08102020 msgr2=0x7f1f0810a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.687+0000 7f1f0ee00700 1 -- 172.21.15.63:0/2539492693 shutdown_connections 2024-05-27T07:55:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:46.687+0000 7f1f0ee00700 1 -- 172.21.15.63:0/2539492693 wait complete. 2024-05-27T07:55:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:55:47.632 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:55:47.697 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:47 smithi063 ceph-mon[31613]: pgmap v701: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:55:47.697 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:47 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:55:47.697 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:47 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2539492693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:55:47.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:47 smithi086 ceph-mon[34687]: pgmap v701: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:55:47.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:47 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:55:47.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:47 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2539492693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:55:47.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:47 smithi149 ceph-mon[35408]: pgmap v701: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:55:47.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:47 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:55:47.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:47 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2539492693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:55:48.126 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:48.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:48 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:55:48.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:48 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:55:48.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:48 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:55:48.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:48 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:55:48.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:48 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:55:48.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:48 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:55:48.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:48 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:55:48.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:48 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:55:48.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:48 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:55:49.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:49 smithi086 ceph-mon[34687]: pgmap v702: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.7 MiB/s wr, 16 op/s 2024-05-27T07:55:49.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:49 smithi149 ceph-mon[35408]: pgmap v702: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.7 MiB/s wr, 16 op/s 2024-05-27T07:55:49.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:49 smithi063 ceph-mon[31613]: pgmap v702: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.7 MiB/s wr, 16 op/s 2024-05-27T07:55:50.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.819+0000 7f5c5ca00700 1 -- 172.21.15.63:0/2198928270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c581043f0 msgr2=0x7f5c58104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.819+0000 7f5c5ca00700 1 --2- 172.21.15.63:0/2198928270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c581043f0 0x7f5c58104800 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f5c480099c0 tx=0x7f5c48009cd0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.819+0000 7f5c5ca00700 1 -- 172.21.15.63:0/2198928270 shutdown_connections 2024-05-27T07:55:50.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.819+0000 7f5c5ca00700 1 --2- 172.21.15.63:0/2198928270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58109670 0x7f5c58110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.819+0000 7f5c5ca00700 1 --2- 172.21.15.63:0/2198928270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c581043f0 0x7f5c58104800 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.819+0000 7f5c5ca00700 1 --2- 172.21.15.63:0/2198928270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58108d20 0x7f5c58109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.819+0000 7f5c5ca00700 1 -- 172.21.15.63:0/2198928270 >> 172.21.15.63:0/2198928270 conn(0x7f5c58079d00 msgr2=0x7f5c5807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:50.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.819+0000 7f5c5ca00700 1 -- 172.21.15.63:0/2198928270 shutdown_connections 2024-05-27T07:55:50.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.819+0000 7f5c5ca00700 1 -- 172.21.15.63:0/2198928270 wait complete. 2024-05-27T07:55:50.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.821+0000 7f5c5ca00700 1 Processor -- start 2024-05-27T07:55:50.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.821+0000 7f5c5ca00700 1 -- start start 2024-05-27T07:55:50.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.821+0000 7f5c5ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c581043f0 0x7f5c5819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.822+0000 7f5c5ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58108d20 0x7f5c5819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.822+0000 7f5c5ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58109670 0x7f5c581a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.822+0000 7f5c5ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581141a0 con 0x7f5c581043f0 2024-05-27T07:55:50.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.822+0000 7f5c5ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58114020 con 0x7f5c58109670 2024-05-27T07:55:50.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.822+0000 7f5c5ca00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58114320 con 0x7f5c58108d20 2024-05-27T07:55:50.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.822+0000 7f5c57400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c581043f0 0x7f5c5819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.822+0000 7f5c57400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c581043f0 0x7f5c5819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57172/0 (socket says 172.21.15.63:57172) 2024-05-27T07:55:50.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.822+0000 7f5c57400700 1 -- 172.21.15.63:0/1391311207 learned_addr learned my addr 172.21.15.63:0/1391311207 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:50.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.822+0000 7f5c56a00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58108d20 0x7f5c5819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.822+0000 7f5c57e00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58109670 0x7f5c581a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.823+0000 7f5c57400700 1 -- 172.21.15.63:0/1391311207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58108d20 msgr2=0x7f5c5819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.823+0000 7f5c57400700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58108d20 0x7f5c5819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.823+0000 7f5c57400700 1 -- 172.21.15.63:0/1391311207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58109670 msgr2=0x7f5c581a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.823+0000 7f5c57400700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58109670 0x7f5c581a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.823+0000 7f5c57400700 1 -- 172.21.15.63:0/1391311207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c581a1d20 con 0x7f5c581043f0 2024-05-27T07:55:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.823+0000 7f5c57400700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c581043f0 0x7f5c5819ce50 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800b580 tx=0x7f5c480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.824+0000 7f5c4fe00700 1 -- 172.21.15.63:0/1391311207 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48027070 con 0x7f5c581043f0 2024-05-27T07:55:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.824+0000 7f5c4fe00700 1 -- 172.21.15.63:0/1391311207 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c4800fc80 con 0x7f5c581043f0 2024-05-27T07:55:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.824+0000 7f5c5ca00700 1 -- 172.21.15.63:0/1391311207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c581a1fb0 con 0x7f5c581043f0 2024-05-27T07:55:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.824+0000 7f5c4fe00700 1 -- 172.21.15.63:0/1391311207 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48022d30 con 0x7f5c581043f0 2024-05-27T07:55:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.824+0000 7f5c5ca00700 1 -- 172.21.15.63:0/1391311207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c581a9930 con 0x7f5c581043f0 2024-05-27T07:55:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.828+0000 7f5c4fe00700 1 -- 172.21.15.63:0/1391311207 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5c4802a070 con 0x7f5c581043f0 2024-05-27T07:55:50.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.830+0000 7f5c6cbf4b80 1 Processor -- start 2024-05-27T07:55:50.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.830+0000 7f5c6cbf4b80 1 -- start start 2024-05-27T07:55:50.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.830+0000 7f5c4fe00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c3c0b2a70 0x7f5c3c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.830+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc41c99f0 0x558fc425d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.830+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc425dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.830+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc42696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.830+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558fc418ae40 con 0x558fc42672c0 2024-05-27T07:55:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.830+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fc417a9e0 con 0x558fc41c99f0 2024-05-27T07:55:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.830+0000 7f5c4fe00700 1 -- 172.21.15.63:0/1391311207 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f5c480e3710 con 0x7f5c581043f0 2024-05-27T07:55:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.830+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558fc417ea20 con 0x558fc40dde40 2024-05-27T07:55:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.831+0000 7f5c57e00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c3c0b2a70 0x7f5c3c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.831+0000 7f5c57e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc42696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.831+0000 7f5c57e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc42696e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57178/0 (socket says 172.21.15.63:57178) 2024-05-27T07:55:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.831+0000 7f5c57e00700 1 -- 172.21.15.63:0/2654971083 learned_addr learned my addr 172.21.15.63:0/2654971083 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.831+0000 7f5c56a00700 1 --2- 172.21.15.63:0/2654971083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc425dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.831+0000 7f5c57400700 1 --2- 172.21.15.63:0/2654971083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc41c99f0 0x558fc425d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.832+0000 7f5c57e00700 1 -- 172.21.15.63:0/2654971083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 msgr2=0x558fc425dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.832+0000 7f5c57e00700 1 --2- 172.21.15.63:0/2654971083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc425dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.832+0000 7f5c57e00700 1 -- 172.21.15.63:0/2654971083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc41c99f0 msgr2=0x558fc425d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.832+0000 7f5c57e00700 1 --2- 172.21.15.63:0/2654971083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc41c99f0 0x558fc425d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.832+0000 7f5c57e00700 1 -- 172.21.15.63:0/2654971083 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc4143420 con 0x558fc42672c0 2024-05-27T07:55:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.832+0000 7f5c57e00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c3c0b2a70 0x7f5c3c0b4f80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009fd0 tx=0x7f5c50009d00 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.832+0000 7f5c57e00700 1 --2- 172.21.15.63:0/2654971083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc42696e0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f5c50007fc0 tx=0x7f5c5001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26500ab1889c9668 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.833+0000 7f5c4cc00700 1 -- 172.21.15.63:0/2654971083 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50009a50 con 0x558fc42672c0 2024-05-27T07:55:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.833+0000 7f5c4cc00700 1 -- 172.21.15.63:0/2654971083 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c5001f070 con 0x558fc42672c0 2024-05-27T07:55:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.833+0000 7f5c4cc00700 1 -- 172.21.15.63:0/2654971083 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50023af0 con 0x558fc42672c0 2024-05-27T07:55:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.834+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2654971083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 msgr2=0x558fc42696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.834+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/2654971083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc42696e0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f5c50007fc0 tx=0x7f5c5001c000 comp rx=0 tx=0).stop 2024-05-27T07:55:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.834+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2654971083 shutdown_connections 2024-05-27T07:55:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.834+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/2654971083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc42696e0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.834+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/2654971083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc41c99f0 0x558fc425d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.834+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/2654971083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc425dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.834+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2654971083 >> 172.21.15.63:0/2654971083 conn(0x558fc4177390 msgr2=0x558fc4258710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.834+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2654971083 shutdown_connections 2024-05-27T07:55:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.835+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2654971083 wait complete. 2024-05-27T07:55:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.835+0000 7f5c6cbf4b80 1 -- start start 2024-05-27T07:55:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.836+0000 7f5c6cbf4b80 1 Processor -- start 2024-05-27T07:55:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.836+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc4264230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.836+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc41c99f0 0x558fc4264770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.836+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc4264cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.836+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558fc4080fa0 con 0x558fc42672c0 2024-05-27T07:55:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.836+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fc411b760 con 0x558fc41c99f0 2024-05-27T07:55:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.836+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558fc4269da0 con 0x558fc40dde40 2024-05-27T07:55:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.836+0000 7f5c6cbf4b80 10 client.0 _ll_register_callbacks cb 0x558fc4124c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.837+0000 7f5c57e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc4264cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.837+0000 7f5c57e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc4264cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57182/0 (socket says 172.21.15.63:57182) 2024-05-27T07:55:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.837+0000 7f5c57e00700 1 -- 172.21.15.63:0/2838748627 learned_addr learned my addr 172.21.15.63:0/2838748627 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.837+0000 7f5c56a00700 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc41c99f0 0x558fc4264770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.837+0000 7f5c57400700 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc4264230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.838+0000 7f5c57e00700 1 -- 172.21.15.63:0/2838748627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 msgr2=0x558fc4264230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.838+0000 7f5c57e00700 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc4264230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.838+0000 7f5c57e00700 1 -- 172.21.15.63:0/2838748627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc41c99f0 msgr2=0x558fc4264770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.838+0000 7f5c57e00700 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc41c99f0 0x558fc4264770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.838+0000 7f5c57e00700 1 -- 172.21.15.63:0/2838748627 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc40ca8d0 con 0x558fc42672c0 2024-05-27T07:55:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.838+0000 7f5c57e00700 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc4264cb0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f5c50004f50 tx=0x7f5c50023760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b679b8f688d8631e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.838+0000 7f5c41600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.839+0000 7f5c41600700 1 -- 172.21.15.63:0/2838748627 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50033070 con 0x558fc42672c0 2024-05-27T07:55:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.839+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2838748627 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c5001f070 con 0x558fc42672c0 2024-05-27T07:55:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.839+0000 7f5c41600700 1 -- 172.21.15.63:0/2838748627 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f5c50023c80 con 0x558fc42672c0 2024-05-27T07:55:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.839+0000 7f5c41600700 1 -- 172.21.15.63:0/2838748627 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5c50009a50 con 0x558fc42672c0 2024-05-27T07:55:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.839+0000 7f5c6cbf4b80 10 client.19692 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.839+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2838748627 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558fc414b060 con 0x558fc42672c0 2024-05-27T07:55:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.839+0000 7f5c41600700 1 -- 172.21.15.63:0/2838748627 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c5001f070 con 0x558fc42672c0 2024-05-27T07:55:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.840+0000 7f5c41600700 1 -- 172.21.15.63:0/2838748627 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5c50018040 con 0x558fc42672c0 2024-05-27T07:55:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.840+0000 7f5c6cbf4b80 10 client.19692 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.840+0000 7f5c6cbf4b80 10 client.19692 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:55:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.840+0000 7f5c6cbf4b80 20 client.19692 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.840+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x558fc4276be0 0x558fc4279000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.840+0000 7f5c6cbf4b80 4 client.19692 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.840+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2838748627 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fc418cd90 con 0x558fc4276be0 2024-05-27T07:55:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.841+0000 7f5c57400700 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x558fc4276be0 0x558fc4279000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.842+0000 7f5c57400700 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x558fc4276be0 0x558fc4279000 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5068ba22d5c51f2 server_cookie=87dd723c177a1040 in_seq=0 out_seq=0 2024-05-27T07:55:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.842+0000 7f5c41600700 10 client.19692 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:55:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c41600700 1 -- 172.21.15.63:0/2838748627 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558fc418cd90 con 0x558fc4276be0 2024-05-27T07:55:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c41600700 10 client.19692 handle_command_reply: tid=0 2024-05-27T07:55:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c6cbf4b80 1 client.19692 shutdown 2024-05-27T07:55:50.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2838748627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 msgr2=0x558fc4264cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc4264cb0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f5c50004f50 tx=0x7f5c50023760 comp rx=0 tx=0).stop 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2838748627 shutdown_connections 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x558fc4276be0 0x558fc4279000 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc42672c0 0x558fc4264cb0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc4264230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/2838748627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc41c99f0 0x558fc4264770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2838748627 >> 172.21.15.63:0/2838748627 conn(0x558fc4177390 msgr2=0x558fc425d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.844+0000 7f5c41600700 0 client.19692 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.845+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2838748627 shutdown_connections 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.845+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/2838748627 wait complete. 2024-05-27T07:55:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.845+0000 7f5c6cbf4b80 20 client.19692 trim_cache size 0 max 16384 2024-05-27T07:55:50.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.865+0000 7f5c6cbf4b80 1 Processor -- start 2024-05-27T07:55:50.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.865+0000 7f5c6cbf4b80 1 -- start start 2024-05-27T07:55:50.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.866+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc40dde40 0x558fc4263190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.866+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc4273070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.866+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc42672c0 0x558fc4258740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.866+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581b4d50 con 0x558fc41c99f0 2024-05-27T07:55:50.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.866+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581b25e0 con 0x558fc40dde40 2024-05-27T07:55:50.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.866+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581b0450 con 0x558fc42672c0 2024-05-27T07:55:50.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.866+0000 7f5c56a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc4273070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.866+0000 7f5c56a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc4273070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57186/0 (socket says 172.21.15.63:57186) 2024-05-27T07:55:50.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.866+0000 7f5c56a00700 1 -- 172.21.15.63:0/1111913320 learned_addr learned my addr 172.21.15.63:0/1111913320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:50.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.867+0000 7f5c57400700 1 --2- 172.21.15.63:0/1111913320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc40dde40 0x558fc4263190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.867+0000 7f5c57e00700 1 --2- 172.21.15.63:0/1111913320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc42672c0 0x558fc4258740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.867+0000 7f5c56a00700 1 -- 172.21.15.63:0/1111913320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc42672c0 msgr2=0x558fc4258740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:50.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.867+0000 7f5c56a00700 1 --2- 172.21.15.63:0/1111913320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc42672c0 0x558fc4258740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.867+0000 7f5c56a00700 1 -- 172.21.15.63:0/1111913320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc40dde40 msgr2=0x558fc4263190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:50.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.867+0000 7f5c56a00700 1 --2- 172.21.15.63:0/1111913320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc40dde40 0x558fc4263190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.867+0000 7f5c56a00700 1 -- 172.21.15.63:0/1111913320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc4259bf0 con 0x558fc41c99f0 2024-05-27T07:55:50.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.867+0000 7f5c56a00700 1 --2- 172.21.15.63:0/1111913320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc4273070 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5c5806c3b0 tx=0x7f5c4400baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84f1a98a17d7a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:50.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.868+0000 7f5c43e00700 1 -- 172.21.15.63:0/1111913320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44004830 con 0x558fc41c99f0 2024-05-27T07:55:50.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.868+0000 7f5c43e00700 1 -- 172.21.15.63:0/1111913320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c44005020 con 0x558fc41c99f0 2024-05-27T07:55:50.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.868+0000 7f5c43e00700 1 -- 172.21.15.63:0/1111913320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44003980 con 0x558fc41c99f0 2024-05-27T07:55:50.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.869+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1111913320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 msgr2=0x558fc4273070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.869+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/1111913320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc4273070 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5c5806c3b0 tx=0x7f5c4400baf0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.869+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1111913320 shutdown_connections 2024-05-27T07:55:50.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.869+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/1111913320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc42672c0 0x558fc4258740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.869+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/1111913320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc40dde40 0x558fc4263190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.869+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/1111913320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc4273070 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.869+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1111913320 >> 172.21.15.63:0/1111913320 conn(0x558fc4177390 msgr2=0x558fc425f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:50.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.869+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1111913320 shutdown_connections 2024-05-27T07:55:50.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.869+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1111913320 wait complete. 2024-05-27T07:55:50.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.870+0000 7f5c6cbf4b80 1 -- start start 2024-05-27T07:55:50.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.870+0000 7f5c6cbf4b80 1 Processor -- start 2024-05-27T07:55:50.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc427c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc427c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c6cbf4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc42672c0 0x558fc4279550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581b0aa0 con 0x558fc41c99f0 2024-05-27T07:55:50.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581b1f10 con 0x558fc42672c0 2024-05-27T07:55:50.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c6cbf4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581b1170 con 0x558fc40dde40 2024-05-27T07:55:50.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c6cbf4b80 10 client.0 _ll_register_callbacks cb 0x558fc4124c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:50.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c56a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc427c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c56a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc427c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57202/0 (socket says 172.21.15.63:57202) 2024-05-27T07:55:50.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c56a00700 1 -- 172.21.15.63:0/1837163375 learned_addr learned my addr 172.21.15.63:0/1837163375 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:50.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.871+0000 7f5c57400700 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc427c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.872+0000 7f5c57e00700 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc42672c0 0x558fc4279550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.872+0000 7f5c56a00700 1 -- 172.21.15.63:0/1837163375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 msgr2=0x558fc427c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.872+0000 7f5c56a00700 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc427c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.872+0000 7f5c56a00700 1 -- 172.21.15.63:0/1837163375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc42672c0 msgr2=0x558fc4279550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.872+0000 7f5c56a00700 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc42672c0 0x558fc4279550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.872+0000 7f5c56a00700 1 -- 172.21.15.63:0/1837163375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc4279b90 con 0x558fc41c99f0 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.872+0000 7f5c57e00700 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc42672c0 0x558fc4279550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.872+0000 7f5c56a00700 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc427c580 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5c440046e0 tx=0x7f5c440054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750916067ae771ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.872+0000 7f5c41600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.872+0000 7f5c41600700 1 -- 172.21.15.63:0/1837163375 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44004960 con 0x558fc41c99f0 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.873+0000 7f5c41600700 1 -- 172.21.15.63:0/1837163375 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f5c44018040 con 0x558fc41c99f0 2024-05-27T07:55:50.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.873+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1837163375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c44005020 con 0x558fc41c99f0 2024-05-27T07:55:50.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.873+0000 7f5c41600700 1 -- 172.21.15.63:0/1837163375 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5c44004b00 con 0x558fc41c99f0 2024-05-27T07:55:50.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.873+0000 7f5c6cbf4b80 10 client.19698 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:50.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.873+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1837163375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558fc42e0df0 con 0x558fc41c99f0 2024-05-27T07:55:50.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.873+0000 7f5c41600700 1 -- 172.21.15.63:0/1837163375 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c44005020 con 0x558fc41c99f0 2024-05-27T07:55:50.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.874+0000 7f5c41600700 1 -- 172.21.15.63:0/1837163375 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f5c4401c600 con 0x558fc41c99f0 2024-05-27T07:55:50.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.874+0000 7f5c6cbf4b80 10 client.19698 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:50.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.874+0000 7f5c6cbf4b80 10 client.19698 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:55:50.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.874+0000 7f5c6cbf4b80 20 client.19698 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:50.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.874+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x558fc42e1470 0x558fc42e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.874+0000 7f5c6cbf4b80 4 client.19698 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:55:50.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.874+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1837163375 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558fc4186570 con 0x558fc42e1470 2024-05-27T07:55:50.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.874+0000 7f5c57400700 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x558fc42e1470 0x558fc42e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.875+0000 7f5c57400700 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x558fc42e1470 0x558fc42e3890 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b064f5a0a6f61fb server_cookie=95860d9051b34230 in_seq=0 out_seq=0 2024-05-27T07:55:50.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.875+0000 7f5c41600700 10 client.19698 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:55:50.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c41600700 1 -- 172.21.15.63:0/1837163375 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558fc4186570 con 0x558fc42e1470 2024-05-27T07:55:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c41600700 10 client.19698 handle_command_reply: tid=0 2024-05-27T07:55:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 client.19698 shutdown 2024-05-27T07:55:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1837163375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 msgr2=0x558fc427c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc427c580 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5c440046e0 tx=0x7f5c440054b0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1837163375 shutdown_connections 2024-05-27T07:55:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc42672c0 0x558fc4279550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fc40dde40 0x558fc427c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x558fc42e1470 0x558fc42e3890 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 --2- 172.21.15.63:0/1837163375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fc41c99f0 0x558fc427c580 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1837163375 >> 172.21.15.63:0/1837163375 conn(0x558fc4177390 msgr2=0x558fc42656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c41600700 0 client.19698 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:55:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1837163375 shutdown_connections 2024-05-27T07:55:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.876+0000 7f5c6cbf4b80 1 -- 172.21.15.63:0/1837163375 wait complete. 2024-05-27T07:55:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.877+0000 7f5c6cbf4b80 20 client.19698 trim_cache size 0 max 16384 2024-05-27T07:55:50.879 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:55:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.877+0000 7f5c5ca00700 1 -- 172.21.15.63:0/1391311207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c3c0b2a70 msgr2=0x7f5c3c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.877+0000 7f5c5ca00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c3c0b2a70 0x7f5c3c0b4f80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009fd0 tx=0x7f5c50009d00 comp rx=0 tx=0).stop 2024-05-27T07:55:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.877+0000 7f5c5ca00700 1 -- 172.21.15.63:0/1391311207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c581043f0 msgr2=0x7f5c5819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.877+0000 7f5c5ca00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c581043f0 0x7f5c5819ce50 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800b580 tx=0x7f5c480047b0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.878+0000 7f5c5ca00700 1 -- 172.21.15.63:0/1391311207 shutdown_connections 2024-05-27T07:55:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.878+0000 7f5c5ca00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c3c0b2a70 0x7f5c3c0b4f80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.878+0000 7f5c5ca00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c58109670 0x7f5c581a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.878+0000 7f5c5ca00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c581043f0 0x7f5c5819ce50 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.878+0000 7f5c5ca00700 1 --2- 172.21.15.63:0/1391311207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c58108d20 0x7f5c5819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.878+0000 7f5c5ca00700 1 -- 172.21.15.63:0/1391311207 >> 172.21.15.63:0/1391311207 conn(0x7f5c58079d00 msgr2=0x7f5c58101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:50.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.878+0000 7f5c5ca00700 1 -- 172.21.15.63:0/1391311207 shutdown_connections 2024-05-27T07:55:50.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:50.878+0000 7f5c5ca00700 1 -- 172.21.15.63:0/1391311207 wait complete. 2024-05-27T07:55:51.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:51 smithi086 ceph-mon[34687]: pgmap v703: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:55:51.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:51 smithi149 ceph-mon[35408]: pgmap v703: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:55:51.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:51 smithi063 ceph-mon[31613]: pgmap v703: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:55:51.774 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:55:52.261 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:53.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:53 smithi086 ceph-mon[34687]: pgmap v704: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:55:53.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:53 smithi149 ceph-mon[35408]: pgmap v704: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:55:53.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:53 smithi063 ceph-mon[31613]: pgmap v704: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:55:54.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.767+0000 7fe79c800700 1 -- 172.21.15.63:0/2447913917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe794109860 msgr2=0x7fe7941111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:54.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.767+0000 7fe79c800700 1 --2- 172.21.15.63:0/2447913917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe794109860 0x7fe7941111c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe7900099c0 tx=0x7fe790009cd0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.767+0000 7fe79c800700 1 -- 172.21.15.63:0/2447913917 shutdown_connections 2024-05-27T07:55:54.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.767+0000 7fe79c800700 1 --2- 172.21.15.63:0/2447913917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe794109860 0x7fe7941111c0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.767+0000 7fe79c800700 1 --2- 172.21.15.63:0/2447913917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7941045e0 0x7fe7941049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.767+0000 7fe79c800700 1 --2- 172.21.15.63:0/2447913917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe794108f10 0x7fe794109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.767+0000 7fe79c800700 1 -- 172.21.15.63:0/2447913917 >> 172.21.15.63:0/2447913917 conn(0x7fe794100030 msgr2=0x7fe794102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:54.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.767+0000 7fe79c800700 1 -- 172.21.15.63:0/2447913917 shutdown_connections 2024-05-27T07:55:54.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.768+0000 7fe79c800700 1 -- 172.21.15.63:0/2447913917 wait complete. 2024-05-27T07:55:54.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.769+0000 7fe79c800700 1 Processor -- start 2024-05-27T07:55:54.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.769+0000 7fe79c800700 1 -- start start 2024-05-27T07:55:54.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.770+0000 7fe79c800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7941045e0 0x7fe79419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.770+0000 7fe79c800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe794108f10 0x7fe79419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.770+0000 7fe79c800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe794109860 0x7fe7941a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.770+0000 7fe79c800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe794114340 con 0x7fe7941045e0 2024-05-27T07:55:54.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.770+0000 7fe79c800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7941141c0 con 0x7fe794109860 2024-05-27T07:55:54.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.770+0000 7fe79c800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7941144c0 con 0x7fe794108f10 2024-05-27T07:55:54.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.770+0000 7fe799e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7941045e0 0x7fe79419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.771+0000 7fe799e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7941045e0 0x7fe79419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57234/0 (socket says 172.21.15.63:57234) 2024-05-27T07:55:54.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.771+0000 7fe799e00700 1 -- 172.21.15.63:0/3540840367 learned_addr learned my addr 172.21.15.63:0/3540840367 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:54.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.771+0000 7fe79a800700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe794109860 0x7fe7941a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.771+0000 7fe799400700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe794108f10 0x7fe79419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.771+0000 7fe799e00700 1 -- 172.21.15.63:0/3540840367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe794108f10 msgr2=0x7fe79419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:54.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.771+0000 7fe799e00700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe794108f10 0x7fe79419d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.771+0000 7fe799e00700 1 -- 172.21.15.63:0/3540840367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe794109860 msgr2=0x7fe7941a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:54.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.771+0000 7fe799e00700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe794109860 0x7fe7941a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.771+0000 7fe799e00700 1 -- 172.21.15.63:0/3540840367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7941a1d10 con 0x7fe7941045e0 2024-05-27T07:55:54.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.772+0000 7fe799e00700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7941045e0 0x7fe79419cef0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00ef10 tx=0x7fe78c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:54.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.772+0000 7fe78aa00700 1 -- 172.21.15.63:0/3540840367 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00c9c0 con 0x7fe7941045e0 2024-05-27T07:55:54.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.772+0000 7fe78aa00700 1 -- 172.21.15.63:0/3540840367 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe78c019070 con 0x7fe7941045e0 2024-05-27T07:55:54.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.772+0000 7fe78aa00700 1 -- 172.21.15.63:0/3540840367 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c014480 con 0x7fe7941045e0 2024-05-27T07:55:54.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.772+0000 7fe79c800700 1 -- 172.21.15.63:0/3540840367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7941a1fa0 con 0x7fe7941045e0 2024-05-27T07:55:54.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.772+0000 7fe79c800700 1 -- 172.21.15.63:0/3540840367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7941a24e0 con 0x7fe7941045e0 2024-05-27T07:55:54.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.777+0000 7fe78aa00700 1 -- 172.21.15.63:0/3540840367 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe78c012050 con 0x7fe7941045e0 2024-05-27T07:55:54.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.778+0000 7fe7ab2dfb80 1 Processor -- start 2024-05-27T07:55:54.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.778+0000 7fe7ab2dfb80 1 -- start start 2024-05-27T07:55:54.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556560b8e40 0x55565623d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656237870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555656237db0 0x5556562381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555656165e40 con 0x55565623dd30 2024-05-27T07:55:54.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5556561559e0 con 0x5556560b8e40 2024-05-27T07:55:54.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555656159a20 con 0x555656237db0 2024-05-27T07:55:54.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe799e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656237870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe799e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656237870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57236/0 (socket says 172.21.15.63:57236) 2024-05-27T07:55:54.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe799e00700 1 -- 172.21.15.63:0/2796432333 learned_addr learned my addr 172.21.15.63:0/2796432333 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:54.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe78aa00700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7800b29f0 0x7fe7800b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe79a800700 1 --2- 172.21.15.63:0/2796432333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556560b8e40 0x55565623d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.779+0000 7fe799400700 1 --2- 172.21.15.63:0/2796432333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555656237db0 0x5556562381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.780+0000 7fe799e00700 1 -- 172.21.15.63:0/2796432333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555656237db0 msgr2=0x5556562381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:54.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.780+0000 7fe799e00700 1 --2- 172.21.15.63:0/2796432333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555656237db0 0x5556562381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.780+0000 7fe799e00700 1 -- 172.21.15.63:0/2796432333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556560b8e40 msgr2=0x55565623d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:54.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.780+0000 7fe799e00700 1 --2- 172.21.15.63:0/2796432333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556560b8e40 0x55565623d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.780+0000 7fe78aa00700 1 -- 172.21.15.63:0/3540840367 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7fe78c0e2960 con 0x7fe7941045e0 2024-05-27T07:55:54.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.780+0000 7fe799e00700 1 -- 172.21.15.63:0/2796432333 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55565611e420 con 0x55565623dd30 2024-05-27T07:55:54.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.780+0000 7fe79a800700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7800b29f0 0x7fe7800b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.780+0000 7fe799e00700 1 --2- 172.21.15.63:0/2796432333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656237870 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0a6a90 tx=0x7fe78c009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5994a9e6fb6cbb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:54.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.781+0000 7fe77f400700 1 -- 172.21.15.63:0/2796432333 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0a95f0 con 0x55565623dd30 2024-05-27T07:55:54.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.781+0000 7fe77f400700 1 -- 172.21.15.63:0/2796432333 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe78c0a9330 con 0x55565623dd30 2024-05-27T07:55:54.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.782+0000 7fe77f400700 1 -- 172.21.15.63:0/2796432333 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0a8f90 con 0x55565623dd30 2024-05-27T07:55:54.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.782+0000 7fe79a800700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7800b29f0 0x7fe7800b4f00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x555656238b30 tx=0x7fe79001a040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:54.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.782+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/2796432333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 msgr2=0x555656237870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:54.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.782+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/2796432333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656237870 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0a6a90 tx=0x7fe78c009eb0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.782+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/2796432333 shutdown_connections 2024-05-27T07:55:54.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.782+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/2796432333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555656237db0 0x5556562381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.782+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/2796432333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556560b8e40 0x55565623d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.782+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/2796432333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656237870 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.782+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/2796432333 >> 172.21.15.63:0/2796432333 conn(0x555656118fc0 msgr2=0x5556562328d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:54.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.783+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/2796432333 shutdown_connections 2024-05-27T07:55:54.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.783+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/2796432333 wait complete. 2024-05-27T07:55:54.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.783+0000 7fe7ab2dfb80 1 -- start start 2024-05-27T07:55:54.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.784+0000 7fe7ab2dfb80 1 Processor -- start 2024-05-27T07:55:54.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.784+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x5556562414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.784+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555656237db0 0x555656241a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.784+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656241f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.784+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55565605bfa0 con 0x55565623dd30 2024-05-27T07:55:54.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.784+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5556560f6760 con 0x555656237db0 2024-05-27T07:55:54.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.784+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555656244740 con 0x5556560b8e40 2024-05-27T07:55:54.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.784+0000 7fe7ab2dfb80 10 client.0 _ll_register_callbacks cb 0x5556560ffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:54.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.785+0000 7fe79a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656241f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.785+0000 7fe799400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555656237db0 0x555656241a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.785+0000 7fe79a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656241f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57252/0 (socket says 172.21.15.63:57252) 2024-05-27T07:55:54.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.785+0000 7fe79a800700 1 -- 172.21.15.63:0/847394780 learned_addr learned my addr 172.21.15.63:0/847394780 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:54.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.785+0000 7fe799e00700 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x5556562414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.786+0000 7fe79a800700 1 -- 172.21.15.63:0/847394780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 msgr2=0x5556562414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:54.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.786+0000 7fe79a800700 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x5556562414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.786+0000 7fe79a800700 1 -- 172.21.15.63:0/847394780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555656237db0 msgr2=0x555656241a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:54.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.786+0000 7fe79a800700 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555656237db0 0x555656241a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.786+0000 7fe79a800700 1 -- 172.21.15.63:0/847394780 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556560a58d0 con 0x55565623dd30 2024-05-27T07:55:54.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.786+0000 7fe79a800700 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656241f50 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fe790015000 tx=0x7fe790009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c418f525f4aca89f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:54.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.786+0000 7fe773e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:54.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.786+0000 7fe773e00700 1 -- 172.21.15.63:0/847394780 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790034060 con 0x55565623dd30 2024-05-27T07:55:54.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.787+0000 7fe773e00700 1 -- 172.21.15.63:0/847394780 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7fe79000bd60 con 0x55565623dd30 2024-05-27T07:55:54.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.787+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/847394780 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe78c0a9330 con 0x55565623dd30 2024-05-27T07:55:54.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.787+0000 7fe773e00700 1 -- 172.21.15.63:0/847394780 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe790024470 con 0x55565623dd30 2024-05-27T07:55:54.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.787+0000 7fe7ab2dfb80 10 client.19710 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:54.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.787+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/847394780 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555656126060 con 0x55565623dd30 2024-05-27T07:55:54.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.787+0000 7fe773e00700 1 -- 172.21.15.63:0/847394780 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe78c0a9330 con 0x55565623dd30 2024-05-27T07:55:54.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.788+0000 7fe773e00700 1 -- 172.21.15.63:0/847394780 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe79000f560 con 0x55565623dd30 2024-05-27T07:55:54.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.788+0000 7fe7ab2dfb80 10 client.19710 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:54.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.788+0000 7fe7ab2dfb80 10 client.19710 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:55:54.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.788+0000 7fe7ab2dfb80 20 client.19710 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:54.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.788+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x555656251580 0x5556562539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.788+0000 7fe7ab2dfb80 4 client.19710 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:54.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.788+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/847394780 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555656167d90 con 0x555656251580 2024-05-27T07:55:54.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.789+0000 7fe799e00700 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x555656251580 0x5556562539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.790+0000 7fe799e00700 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x555656251580 0x5556562539d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=979abd2e818be1fe server_cookie=1b514fe601642828 in_seq=0 out_seq=0 2024-05-27T07:55:54.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.790+0000 7fe773e00700 10 client.19710 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:55:54.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe773e00700 1 -- 172.21.15.63:0/847394780 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555656167d90 con 0x555656251580 2024-05-27T07:55:54.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe773e00700 10 client.19710 handle_command_reply: tid=0 2024-05-27T07:55:54.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe7ab2dfb80 1 client.19710 shutdown 2024-05-27T07:55:54.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/847394780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 msgr2=0x555656241f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:54.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656241f50 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fe790015000 tx=0x7fe790009310 comp rx=0 tx=0).stop 2024-05-27T07:55:54.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/847394780 shutdown_connections 2024-05-27T07:55:54.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x555656251580 0x5556562539d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x555656241f50 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x5556562414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/847394780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555656237db0 0x555656241a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/847394780 >> 172.21.15.63:0/847394780 conn(0x555656118fc0 msgr2=0x555656232580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:54.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.792+0000 7fe773e00700 0 client.19710 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:55:54.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.793+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/847394780 shutdown_connections 2024-05-27T07:55:54.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.793+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/847394780 wait complete. 2024-05-27T07:55:54.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.793+0000 7fe7ab2dfb80 20 client.19710 trim_cache size 0 max 16384 2024-05-27T07:55:54.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.818+0000 7fe7ab2dfb80 1 Processor -- start 2024-05-27T07:55:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.818+0000 7fe7ab2dfb80 1 -- start start 2024-05-27T07:55:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.818+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x555656237870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.818+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555656237db0 0x55565623f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.818+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55565623dd30 0x555656243970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7941b5030 con 0x555656237db0 2024-05-27T07:55:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7941b28c0 con 0x55565623dd30 2024-05-27T07:55:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7941b0730 con 0x5556560b8e40 2024-05-27T07:55:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe799400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555656237db0 0x55565623f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe799400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555656237db0 0x55565623f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57262/0 (socket says 172.21.15.63:57262) 2024-05-27T07:55:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe799400700 1 -- 172.21.15.63:0/1178183152 learned_addr learned my addr 172.21.15.63:0/1178183152 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:54.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe799e00700 1 --2- 172.21.15.63:0/1178183152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x555656237870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe79a800700 1 --2- 172.21.15.63:0/1178183152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55565623dd30 0x555656243970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe799400700 1 -- 172.21.15.63:0/1178183152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 msgr2=0x555656237870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:54.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe799400700 1 --2- 172.21.15.63:0/1178183152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x555656237870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe799400700 1 -- 172.21.15.63:0/1178183152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55565623dd30 msgr2=0x555656243970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:54.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe799400700 1 --2- 172.21.15.63:0/1178183152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55565623dd30 0x555656243970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.819+0000 7fe799400700 1 -- 172.21.15.63:0/1178183152 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55565623d320 con 0x555656237db0 2024-05-27T07:55:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.820+0000 7fe799400700 1 --2- 172.21.15.63:0/1178183152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555656237db0 0x55565623f1a0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fe78400ad30 tx=0x7fe784012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94ae027805c60e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.820+0000 7fe77ea00700 1 -- 172.21.15.63:0/1178183152 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784004830 con 0x555656237db0 2024-05-27T07:55:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.820+0000 7fe77ea00700 1 -- 172.21.15.63:0/1178183152 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe784005020 con 0x555656237db0 2024-05-27T07:55:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.820+0000 7fe77ea00700 1 -- 172.21.15.63:0/1178183152 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78400f890 con 0x555656237db0 2024-05-27T07:55:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.821+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1178183152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555656237db0 msgr2=0x55565623f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.821+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/1178183152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555656237db0 0x55565623f1a0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fe78400ad30 tx=0x7fe784012730 comp rx=0 tx=0).stop 2024-05-27T07:55:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.822+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1178183152 shutdown_connections 2024-05-27T07:55:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.822+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/1178183152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55565623dd30 0x555656243970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.822+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/1178183152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x555656237870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.822+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/1178183152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555656237db0 0x55565623f1a0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.822+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1178183152 >> 172.21.15.63:0/1178183152 conn(0x555656118fc0 msgr2=0x55565623c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:54.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.822+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1178183152 shutdown_connections 2024-05-27T07:55:54.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.822+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1178183152 wait complete. 2024-05-27T07:55:54.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.822+0000 7fe7ab2dfb80 1 -- start start 2024-05-27T07:55:54.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.823+0000 7fe7ab2dfb80 1 Processor -- start 2024-05-27T07:55:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.823+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x5556562499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.823+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555656237db0 0x555656249f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.823+0000 7fe7ab2dfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x5556562581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.823+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7941b0d80 con 0x55565623dd30 2024-05-27T07:55:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.823+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7941b21f0 con 0x555656237db0 2024-05-27T07:55:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.823+0000 7fe7ab2dfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe7941b1450 con 0x5556560b8e40 2024-05-27T07:55:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.823+0000 7fe7ab2dfb80 10 client.0 _ll_register_callbacks cb 0x5556560ffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.824+0000 7fe79a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x5556562581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.824+0000 7fe79a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x5556562581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57274/0 (socket says 172.21.15.63:57274) 2024-05-27T07:55:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.824+0000 7fe79a800700 1 -- 172.21.15.63:0/1798818156 learned_addr learned my addr 172.21.15.63:0/1798818156 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.824+0000 7fe799e00700 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x5556562499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.824+0000 7fe799400700 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555656237db0 0x555656249f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.824+0000 7fe79a800700 1 -- 172.21.15.63:0/1798818156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 msgr2=0x5556562499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.824+0000 7fe79a800700 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x5556562499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.824+0000 7fe79a800700 1 -- 172.21.15.63:0/1798818156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555656237db0 msgr2=0x555656249f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.824+0000 7fe79a800700 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555656237db0 0x555656249f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.824+0000 7fe79a800700 1 -- 172.21.15.63:0/1798818156 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55565624a550 con 0x55565623dd30 2024-05-27T07:55:54.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.825+0000 7fe79a800700 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x5556562581c0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe7900098e0 tx=0x7fe79000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f86c53cd41162e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:54.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.825+0000 7fe773e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.825+0000 7fe773e00700 1 -- 172.21.15.63:0/1798818156 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790034070 con 0x55565623dd30 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.825+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1798818156 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe784005020 con 0x55565623dd30 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.825+0000 7fe773e00700 1 -- 172.21.15.63:0/1798818156 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7fe7900246b0 con 0x55565623dd30 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.825+0000 7fe773e00700 1 -- 172.21.15.63:0/1798818156 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe790046050 con 0x55565623dd30 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.825+0000 7fe7ab2dfb80 10 client.19716 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.825+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1798818156 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5556562bb5d0 con 0x55565623dd30 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.826+0000 7fe773e00700 1 -- 172.21.15.63:0/1798818156 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe784005020 con 0x55565623dd30 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.826+0000 7fe773e00700 1 -- 172.21.15.63:0/1798818156 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe79002f700 con 0x55565623dd30 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.826+0000 7fe7ab2dfb80 10 client.19716 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.826+0000 7fe7ab2dfb80 10 client.19716 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:55:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.826+0000 7fe7ab2dfb80 20 client.19716 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.826+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5556562bbbf0 0x5556562be010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.826+0000 7fe7ab2dfb80 4 client.19716 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:55:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.826+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1798818156 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555656161570 con 0x5556562bbbf0 2024-05-27T07:55:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.827+0000 7fe799e00700 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5556562bbbf0 0x5556562be010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:54.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.828+0000 7fe799e00700 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5556562bbbf0 0x5556562be010 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa939136d1f99969 server_cookie=ac05476ded16ad70 in_seq=0 out_seq=0 2024-05-27T07:55:54.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.828+0000 7fe773e00700 10 client.19716 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:55:54.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe773e00700 1 -- 172.21.15.63:0/1798818156 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555656161570 con 0x5556562bbbf0 2024-05-27T07:55:54.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe773e00700 10 client.19716 handle_command_reply: tid=0 2024-05-27T07:55:54.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 client.19716 shutdown 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1798818156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 msgr2=0x5556562581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x5556562581c0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe7900098e0 tx=0x7fe79000bf40 comp rx=0 tx=0).stop 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1798818156 shutdown_connections 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5556562bbbf0 0x5556562be010 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55565623dd30 0x5556562581c0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5556560b8e40 0x5556562499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 --2- 172.21.15.63:0/1798818156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555656237db0 0x555656249f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe773e00700 0 client.19716 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1798818156 >> 172.21.15.63:0/1798818156 conn(0x555656118fc0 msgr2=0x555656231b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1798818156 shutdown_connections 2024-05-27T07:55:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.829+0000 7fe7ab2dfb80 1 -- 172.21.15.63:0/1798818156 wait complete. 2024-05-27T07:55:54.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.830+0000 7fe7ab2dfb80 20 client.19716 trim_cache size 0 max 16384 2024-05-27T07:55:54.832 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:55:54.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.830+0000 7fe79c800700 1 -- 172.21.15.63:0/3540840367 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7800b29f0 msgr2=0x7fe7800b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:54.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.830+0000 7fe79c800700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7800b29f0 0x7fe7800b4f00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x555656238b30 tx=0x7fe79001a040 comp rx=0 tx=0).stop 2024-05-27T07:55:54.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.830+0000 7fe79c800700 1 -- 172.21.15.63:0/3540840367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7941045e0 msgr2=0x7fe79419cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:54.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.830+0000 7fe79c800700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7941045e0 0x7fe79419cef0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00ef10 tx=0x7fe78c00c3a0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.831+0000 7fe79c800700 1 -- 172.21.15.63:0/3540840367 shutdown_connections 2024-05-27T07:55:54.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.831+0000 7fe79c800700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7800b29f0 0x7fe7800b4f00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.831+0000 7fe79c800700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe794109860 0x7fe7941a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.831+0000 7fe79c800700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7941045e0 0x7fe79419cef0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.831+0000 7fe79c800700 1 --2- 172.21.15.63:0/3540840367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe794108f10 0x7fe79419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:54.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.831+0000 7fe79c800700 1 -- 172.21.15.63:0/3540840367 >> 172.21.15.63:0/3540840367 conn(0x7fe794100030 msgr2=0x7fe794110e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:54.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.831+0000 7fe79c800700 1 -- 172.21.15.63:0/3540840367 shutdown_connections 2024-05-27T07:55:54.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:54.831+0000 7fe79c800700 1 -- 172.21.15.63:0/3540840367 wait complete. 2024-05-27T07:55:55.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:55 smithi086 ceph-mon[34687]: pgmap v705: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-05-27T07:55:55.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:55 smithi149 ceph-mon[35408]: pgmap v705: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-05-27T07:55:55.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:55 smithi063 ceph-mon[31613]: pgmap v705: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-05-27T07:55:55.858 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:55:56.355 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:57.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:57 smithi086 ceph-mon[34687]: pgmap v706: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-05-27T07:55:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:57 smithi149 ceph-mon[35408]: pgmap v706: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-05-27T07:55:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:57 smithi063 ceph-mon[31613]: pgmap v706: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.0 MiB/s wr, 10 op/s 2024-05-27T07:55:58.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.878+0000 7f6f97000700 1 -- 172.21.15.63:0/612361772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f90102f20 msgr2=0x7f6f90103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:58.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.878+0000 7f6f97000700 1 --2- 172.21.15.63:0/612361772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f90102f20 0x7f6f90103330 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009990 tx=0x7f6f84009ca0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.878+0000 7f6f97000700 1 -- 172.21.15.63:0/612361772 shutdown_connections 2024-05-27T07:55:58.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.878+0000 7f6f97000700 1 --2- 172.21.15.63:0/612361772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f90110ad0 0x7f6f90112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.878+0000 7f6f97000700 1 --2- 172.21.15.63:0/612361772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f90102f20 0x7f6f90103330 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.878+0000 7f6f97000700 1 --2- 172.21.15.63:0/612361772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90103870 0x7f6f90110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.878+0000 7f6f97000700 1 -- 172.21.15.63:0/612361772 >> 172.21.15.63:0/612361772 conn(0x7f6f90078ab0 msgr2=0x7f6f90078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:58.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.878+0000 7f6f97000700 1 -- 172.21.15.63:0/612361772 shutdown_connections 2024-05-27T07:55:58.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.878+0000 7f6f97000700 1 -- 172.21.15.63:0/612361772 wait complete. 2024-05-27T07:55:58.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.880+0000 7f6f97000700 1 Processor -- start 2024-05-27T07:55:58.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.880+0000 7f6f97000700 1 -- start start 2024-05-27T07:55:58.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.880+0000 7f6f97000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f90102f20 0x7f6f9019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.880+0000 7f6f97000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f90103870 0x7f6f9019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.880+0000 7f6f97000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90110ad0 0x7f6f901a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.880+0000 7f6f97000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90116410 con 0x7f6f90103870 2024-05-27T07:55:58.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.880+0000 7f6f97000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90116290 con 0x7f6f90102f20 2024-05-27T07:55:58.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.880+0000 7f6f97000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90116590 con 0x7f6f90110ad0 2024-05-27T07:55:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.881+0000 7f6f95200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f90103870 0x7f6f9019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.881+0000 7f6f95200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f90103870 0x7f6f9019d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57298/0 (socket says 172.21.15.63:57298) 2024-05-27T07:55:58.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.881+0000 7f6f95200700 1 -- 172.21.15.63:0/3379606854 learned_addr learned my addr 172.21.15.63:0/3379606854 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.881+0000 7f6f95c00700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f90102f20 0x7f6f9019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.881+0000 7f6f96600700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90110ad0 0x7f6f901a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.881+0000 7f6f95200700 1 -- 172.21.15.63:0/3379606854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90110ad0 msgr2=0x7f6f901a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.881+0000 7f6f95200700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90110ad0 0x7f6f901a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.881+0000 7f6f95200700 1 -- 172.21.15.63:0/3379606854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f90102f20 msgr2=0x7f6f9019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.881+0000 7f6f95200700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f90102f20 0x7f6f9019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.881+0000 7f6f95200700 1 -- 172.21.15.63:0/3379606854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f901a1cf0 con 0x7f6f90103870 2024-05-27T07:55:58.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.882+0000 7f6f95200700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f90103870 0x7f6f9019d3c0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00b4d0 tx=0x7f6f7c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:58.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.882+0000 7f6f8e000700 1 -- 172.21.15.63:0/3379606854 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c015020 con 0x7f6f90103870 2024-05-27T07:55:58.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.882+0000 7f6f8e000700 1 -- 172.21.15.63:0/3379606854 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f7c011cc0 con 0x7f6f90103870 2024-05-27T07:55:58.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.882+0000 7f6f97000700 1 -- 172.21.15.63:0/3379606854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f901a1fe0 con 0x7f6f90103870 2024-05-27T07:55:58.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.882+0000 7f6f8e000700 1 -- 172.21.15.63:0/3379606854 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c010c30 con 0x7f6f90103870 2024-05-27T07:55:58.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.883+0000 7f6f97000700 1 -- 172.21.15.63:0/3379606854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f901a9900 con 0x7f6f90103870 2024-05-27T07:55:58.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.887+0000 7f6f8e000700 1 -- 172.21.15.63:0/3379606854 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6f7c018020 con 0x7f6f90103870 2024-05-27T07:55:58.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.888+0000 7f6fa718eb80 1 Processor -- start 2024-05-27T07:55:58.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.888+0000 7f6fa718eb80 1 -- start start 2024-05-27T07:55:58.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.889+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348656eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.889+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348656f410 0x563486579ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.889+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f820 0x56348657c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.889+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56348649de40 con 0x5634863f0e40 2024-05-27T07:55:58.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.889+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56348648d9e0 con 0x56348656f820 2024-05-27T07:55:58.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.889+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563486491a20 con 0x56348656f410 2024-05-27T07:55:58.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.889+0000 7f6f8e000700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f780b29f0 0x7f6f780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f96600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348656eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f95c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348656f410 0x563486579ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f96600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348656eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57308/0 (socket says 172.21.15.63:57308) 2024-05-27T07:55:58.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f96600700 1 -- 172.21.15.63:0/2089827447 learned_addr learned my addr 172.21.15.63:0/2089827447 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:58.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f8e000700 1 -- 172.21.15.63:0/3379606854 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f6f7c0e2080 con 0x7f6f90103870 2024-05-27T07:55:58.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f96600700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f780b29f0 0x7f6f780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f95200700 1 --2- 172.21.15.63:0/2089827447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f820 0x56348657c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f96600700 1 -- 172.21.15.63:0/2089827447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348656f410 msgr2=0x563486579ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f96600700 1 --2- 172.21.15.63:0/2089827447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348656f410 0x563486579ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f96600700 1 -- 172.21.15.63:0/2089827447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f820 msgr2=0x56348657c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f96600700 1 --2- 172.21.15.63:0/2089827447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f820 0x56348657c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.890+0000 7f6f96600700 1 -- 172.21.15.63:0/2089827447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563486456420 con 0x5634863f0e40 2024-05-27T07:55:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.891+0000 7f6f96600700 1 --2- 172.21.15.63:0/2089827447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348656eed0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f6f88017490 tx=0x7f6f880177a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4e1b3d52861dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.891+0000 7f6f96600700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f780b29f0 0x7f6f780b4f00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6f880060b0 tx=0x7f6f8801b000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.891+0000 7f6f82a00700 1 -- 172.21.15.63:0/2089827447 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800ab00 con 0x5634863f0e40 2024-05-27T07:55:58.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.891+0000 7f6f82a00700 1 -- 172.21.15.63:0/2089827447 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f8800a090 con 0x5634863f0e40 2024-05-27T07:55:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.891+0000 7f6f82a00700 1 -- 172.21.15.63:0/2089827447 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88003d60 con 0x5634863f0e40 2024-05-27T07:55:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.892+0000 7f6fa718eb80 1 -- 172.21.15.63:0/2089827447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 msgr2=0x56348656eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.892+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/2089827447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348656eed0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f6f88017490 tx=0x7f6f880177a0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.893+0000 7f6fa718eb80 1 -- 172.21.15.63:0/2089827447 shutdown_connections 2024-05-27T07:55:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.893+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/2089827447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f820 0x56348657c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.893+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/2089827447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348656eed0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.893+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/2089827447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348656f410 0x563486579ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.893+0000 7f6fa718eb80 1 -- 172.21.15.63:0/2089827447 >> 172.21.15.63:0/2089827447 conn(0x563486450fc0 msgr2=0x56348656ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.893+0000 7f6fa718eb80 1 -- 172.21.15.63:0/2089827447 shutdown_connections 2024-05-27T07:55:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.893+0000 7f6fa718eb80 1 -- 172.21.15.63:0/2089827447 wait complete. 2024-05-27T07:55:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.893+0000 7f6fa718eb80 1 -- start start 2024-05-27T07:55:58.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.894+0000 7f6fa718eb80 1 Processor -- start 2024-05-27T07:55:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.894+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634863f0e40 0x563486579630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.894+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x563486579b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.895+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x56348657a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.895+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563486393fa0 con 0x56348656f820 2024-05-27T07:55:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.895+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56348642e760 con 0x56348656f410 2024-05-27T07:55:58.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.895+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56348657c930 con 0x5634863f0e40 2024-05-27T07:55:58.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.895+0000 7f6fa718eb80 10 client.0 _ll_register_callbacks cb 0x563486437c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:58.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.895+0000 7f6f95c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634863f0e40 0x563486579630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.895+0000 7f6f96600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x56348657a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.895+0000 7f6f96600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x56348657a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57312/0 (socket says 172.21.15.63:57312) 2024-05-27T07:55:58.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.895+0000 7f6f96600700 1 -- 172.21.15.63:0/4030710473 learned_addr learned my addr 172.21.15.63:0/4030710473 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:58.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.895+0000 7f6f95200700 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x563486579b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.896+0000 7f6f96600700 1 -- 172.21.15.63:0/4030710473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634863f0e40 msgr2=0x563486579630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:58.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.896+0000 7f6f96600700 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634863f0e40 0x563486579630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.896+0000 7f6f96600700 1 -- 172.21.15.63:0/4030710473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 msgr2=0x563486579b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:58.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.896+0000 7f6f96600700 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x563486579b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.896+0000 7f6f96600700 1 -- 172.21.15.63:0/4030710473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634863dd8d0 con 0x56348656f820 2024-05-27T07:55:58.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.896+0000 7f6f96600700 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x56348657a0b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f6f88007750 tx=0x7f6f88007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce816ca4ec053b02 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:58.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.896+0000 7f6f77400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:58.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.897+0000 7f6f77400700 1 -- 172.21.15.63:0/4030710473 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800a3c0 con 0x56348656f820 2024-05-27T07:55:58.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.897+0000 7f6f77400700 1 -- 172.21.15.63:0/4030710473 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f6f88022450 con 0x56348656f820 2024-05-27T07:55:58.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.897+0000 7f6fa718eb80 1 -- 172.21.15.63:0/4030710473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f8800a090 con 0x56348656f820 2024-05-27T07:55:58.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.897+0000 7f6f77400700 1 -- 172.21.15.63:0/4030710473 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6f88022670 con 0x56348656f820 2024-05-27T07:55:58.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.897+0000 7f6fa718eb80 10 client.19728 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:58.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.897+0000 7f6fa718eb80 1 -- 172.21.15.63:0/4030710473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56348645e060 con 0x56348656f820 2024-05-27T07:55:58.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.898+0000 7f6f77400700 1 -- 172.21.15.63:0/4030710473 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f8800a090 con 0x56348656f820 2024-05-27T07:55:58.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.898+0000 7f6f77400700 1 -- 172.21.15.63:0/4030710473 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6f8802b760 con 0x56348656f820 2024-05-27T07:55:58.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.898+0000 7f6fa718eb80 10 client.19728 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:58.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.898+0000 7f6fa718eb80 10 client.19728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:58.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.898+0000 7f6fa718eb80 20 client.19728 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:58.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.898+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563486589860 0x56348658bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.898+0000 7f6fa718eb80 4 client.19728 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:58.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.898+0000 7f6fa718eb80 1 -- 172.21.15.63:0/4030710473 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56348649fd90 con 0x563486589860 2024-05-27T07:55:58.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.899+0000 7f6f95c00700 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563486589860 0x56348658bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.900+0000 7f6f95c00700 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563486589860 0x56348658bc80 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4c3fec1ec1e316 server_cookie=75b1b614758ce065 in_seq=0 out_seq=0 2024-05-27T07:55:58.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.900+0000 7f6f77400700 10 client.19728 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:58.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6f77400700 1 -- 172.21.15.63:0/4030710473 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56348649fd90 con 0x563486589860 2024-05-27T07:55:58.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6f77400700 10 client.19728 handle_command_reply: tid=0 2024-05-27T07:55:58.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6fa718eb80 1 client.19728 shutdown 2024-05-27T07:55:58.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6fa718eb80 1 -- 172.21.15.63:0/4030710473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 msgr2=0x56348657a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:58.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x56348657a0b0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f6f88007750 tx=0x7f6f88007780 comp rx=0 tx=0).stop 2024-05-27T07:55:58.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6fa718eb80 1 -- 172.21.15.63:0/4030710473 shutdown_connections 2024-05-27T07:55:58.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563486589860 0x56348658bc80 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x56348657a0b0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634863f0e40 0x563486579630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/4030710473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x563486579b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6fa718eb80 1 -- 172.21.15.63:0/4030710473 >> 172.21.15.63:0/4030710473 conn(0x563486450fc0 msgr2=0x563486569ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:58.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.902+0000 7f6f77400700 0 client.19728 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:58.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.903+0000 7f6fa718eb80 1 -- 172.21.15.63:0/4030710473 shutdown_connections 2024-05-27T07:55:58.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.903+0000 7f6fa718eb80 1 -- 172.21.15.63:0/4030710473 wait complete. 2024-05-27T07:55:58.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.903+0000 7f6fa718eb80 20 client.19728 trim_cache size 0 max 16384 2024-05-27T07:55:58.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.923+0000 7f6fa718eb80 1 Processor -- start 2024-05-27T07:55:58.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.923+0000 7f6fa718eb80 1 -- start start 2024-05-27T07:55:58.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.923+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634863f0e40 0x563486578950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.923+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x563486574e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.924+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x563486585bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.924+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901b4ed0 con 0x56348656f820 2024-05-27T07:55:58.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.924+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901b2760 con 0x56348656f410 2024-05-27T07:55:58.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.924+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901b05d0 con 0x5634863f0e40 2024-05-27T07:55:58.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.924+0000 7f6f96600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x563486585bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.924+0000 7f6f96600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x563486585bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47076/0 (socket says 172.21.15.63:47076) 2024-05-27T07:55:58.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.924+0000 7f6f96600700 1 -- 172.21.15.63:0/2918953871 learned_addr learned my addr 172.21.15.63:0/2918953871 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:58.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.924+0000 7f6f95c00700 1 --2- 172.21.15.63:0/2918953871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634863f0e40 0x563486578950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.924+0000 7f6f95200700 1 --2- 172.21.15.63:0/2918953871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x563486574e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.925+0000 7f6f96600700 1 -- 172.21.15.63:0/2918953871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634863f0e40 msgr2=0x563486578950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:58.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.925+0000 7f6f96600700 1 --2- 172.21.15.63:0/2918953871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634863f0e40 0x563486578950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.925+0000 7f6f96600700 1 -- 172.21.15.63:0/2918953871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 msgr2=0x563486574e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:58.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.925+0000 7f6f96600700 1 --2- 172.21.15.63:0/2918953871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x563486574e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.925+0000 7f6f96600700 1 -- 172.21.15.63:0/2918953871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634865701d0 con 0x56348656f820 2024-05-27T07:55:58.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.925+0000 7f6f96600700 1 --2- 172.21.15.63:0/2918953871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x563486585bd0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800a5f0 tx=0x7f6f88007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd9079ce5a1e9a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:58.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.925+0000 7f6f82000700 1 -- 172.21.15.63:0/2918953871 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88022450 con 0x56348656f820 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.926+0000 7f6f82000700 1 -- 172.21.15.63:0/2918953871 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f88030070 con 0x56348656f820 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.926+0000 7f6f82000700 1 -- 172.21.15.63:0/2918953871 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88004b00 con 0x56348656f820 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.927+0000 7f6fa718eb80 1 -- 172.21.15.63:0/2918953871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 msgr2=0x563486585bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.927+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/2918953871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x563486585bd0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800a5f0 tx=0x7f6f88007650 comp rx=0 tx=0).stop 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.927+0000 7f6fa718eb80 1 -- 172.21.15.63:0/2918953871 shutdown_connections 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.927+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/2918953871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56348656f820 0x563486585bd0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.927+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/2918953871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5634863f0e40 0x563486578950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.927+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/2918953871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x563486574e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.927+0000 7f6fa718eb80 1 -- 172.21.15.63:0/2918953871 >> 172.21.15.63:0/2918953871 conn(0x563486450fc0 msgr2=0x563486579410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.927+0000 7f6fa718eb80 1 -- 172.21.15.63:0/2918953871 shutdown_connections 2024-05-27T07:55:58.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.927+0000 7f6fa718eb80 1 -- 172.21.15.63:0/2918953871 wait complete. 2024-05-27T07:55:58.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.928+0000 7f6fa718eb80 1 -- start start 2024-05-27T07:55:58.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.928+0000 7f6fa718eb80 1 Processor -- start 2024-05-27T07:55:58.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348658e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x56348658ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6fa718eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348656f820 0x56348658bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901b0c20 con 0x5634863f0e40 2024-05-27T07:55:58.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901b2090 con 0x56348656f410 2024-05-27T07:55:58.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6fa718eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901b12f0 con 0x56348656f820 2024-05-27T07:55:58.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6fa718eb80 10 client.0 _ll_register_callbacks cb 0x563486437c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:58.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6f95c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348658e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6f95c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348658e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47092/0 (socket says 172.21.15.63:47092) 2024-05-27T07:55:58.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6f95c00700 1 -- 172.21.15.63:0/1793824494 learned_addr learned my addr 172.21.15.63:0/1793824494 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:58.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6f95200700 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x56348658ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.929+0000 7f6f96600700 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348656f820 0x56348658bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.930+0000 7f6f95c00700 1 -- 172.21.15.63:0/1793824494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348656f820 msgr2=0x56348658bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:58.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.930+0000 7f6f95c00700 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348656f820 0x56348658bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.930+0000 7f6f95c00700 1 -- 172.21.15.63:0/1793824494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 msgr2=0x56348658ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:58.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.930+0000 7f6f95c00700 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x56348658ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.930+0000 7f6f95c00700 1 -- 172.21.15.63:0/1793824494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634865717b0 con 0x5634863f0e40 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.930+0000 7f6f95c00700 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348658e810 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009990 tx=0x7f6f84003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6bcdf54b8fe1f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.930+0000 7f6f77400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.930+0000 7f6f77400700 1 -- 172.21.15.63:0/1793824494 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84005370 con 0x5634863f0e40 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.930+0000 7f6fa718eb80 1 -- 172.21.15.63:0/1793824494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f88030070 con 0x5634863f0e40 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.930+0000 7f6f77400700 1 -- 172.21.15.63:0/1793824494 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f6f84005510 con 0x5634863f0e40 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.931+0000 7f6f77400700 1 -- 172.21.15.63:0/1793824494 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6f84038050 con 0x5634863f0e40 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.931+0000 7f6fa718eb80 10 client.19734 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.931+0000 7f6fa718eb80 1 -- 172.21.15.63:0/1793824494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56348656c350 con 0x5634863f0e40 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.931+0000 7f6f77400700 1 -- 172.21.15.63:0/1793824494 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f88030070 con 0x5634863f0e40 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.931+0000 7f6f77400700 1 -- 172.21.15.63:0/1793824494 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6f8402f670 con 0x5634863f0e40 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.932+0000 7f6fa718eb80 10 client.19734 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:58.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.932+0000 7f6fa718eb80 10 client.19734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:58.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.932+0000 7f6fa718eb80 20 client.19734 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:58.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.932+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5634865f3d70 0x5634865f6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:58.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.932+0000 7f6fa718eb80 4 client.19734 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:55:58.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.932+0000 7f6fa718eb80 1 -- 172.21.15.63:0/1793824494 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563486499570 con 0x5634865f3d70 2024-05-27T07:55:58.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.932+0000 7f6f95200700 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5634865f3d70 0x5634865f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:58.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6f77400700 10 client.19734 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:58.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6f95200700 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5634865f3d70 0x5634865f6190 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f22fbde8726932 server_cookie=d59f910e6760143 in_seq=0 out_seq=0 2024-05-27T07:55:58.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6f77400700 1 -- 172.21.15.63:0/1793824494 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563486499570 con 0x5634865f3d70 2024-05-27T07:55:58.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6f77400700 10 client.19734 handle_command_reply: tid=0 2024-05-27T07:55:58.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6fa718eb80 1 client.19734 shutdown 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6fa718eb80 1 -- 172.21.15.63:0/1793824494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 msgr2=0x56348658e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348658e810 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009990 tx=0x7f6f84003d60 comp rx=0 tx=0).stop 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6fa718eb80 1 -- 172.21.15.63:0/1793824494 shutdown_connections 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5634865f3d70 0x5634865f6190 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56348656f820 0x56348658bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5634863f0e40 0x56348658e810 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6fa718eb80 1 --2- 172.21.15.63:0/1793824494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56348656f410 0x56348658ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6fa718eb80 1 -- 172.21.15.63:0/1793824494 >> 172.21.15.63:0/1793824494 conn(0x563486450fc0 msgr2=0x5634865786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.934+0000 7f6f77400700 0 client.19734 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.935+0000 7f6fa718eb80 1 -- 172.21.15.63:0/1793824494 shutdown_connections 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.935+0000 7f6fa718eb80 1 -- 172.21.15.63:0/1793824494 wait complete. 2024-05-27T07:55:58.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.935+0000 7f6fa718eb80 20 client.19734 trim_cache size 0 max 16384 2024-05-27T07:55:58.937 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:55:58.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 -- 172.21.15.63:0/3379606854 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f780b29f0 msgr2=0x7f6f780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:58.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f780b29f0 0x7f6f780b4f00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6f880060b0 tx=0x7f6f8801b000 comp rx=0 tx=0).stop 2024-05-27T07:55:58.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 -- 172.21.15.63:0/3379606854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f90103870 msgr2=0x7f6f9019d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:58.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f90103870 0x7f6f9019d3c0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00b4d0 tx=0x7f6f7c00b7e0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 -- 172.21.15.63:0/3379606854 shutdown_connections 2024-05-27T07:55:58.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f780b29f0 0x7f6f780b4f00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90110ad0 0x7f6f901a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f90102f20 0x7f6f9019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 --2- 172.21.15.63:0/3379606854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f90103870 0x7f6f9019d3c0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:58.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 -- 172.21.15.63:0/3379606854 >> 172.21.15.63:0/3379606854 conn(0x7f6f90078ab0 msgr2=0x7f6f90111ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:58.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 -- 172.21.15.63:0/3379606854 shutdown_connections 2024-05-27T07:55:58.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:55:58.936+0000 7f6f97000700 1 -- 172.21.15.63:0/3379606854 wait complete. 2024-05-27T07:55:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.409+0000 7f79c2e00640 1 -- 172.21.15.63:0/1459831929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79bc1103c0 msgr2=0x7f79bc1127e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.409+0000 7f79c2e00640 1 --2- 172.21.15.63:0/1459831929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79bc1103c0 0x7f79bc1127e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009ca0 tx=0x7f79b402a8c0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.409+0000 7f79c2e00640 1 -- 172.21.15.63:0/1459831929 shutdown_connections 2024-05-27T07:55:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.409+0000 7f79c2e00640 1 --2- 172.21.15.63:0/1459831929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79bc1103c0 0x7f79bc1127e0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.409+0000 7f79c2e00640 1 --2- 172.21.15.63:0/1459831929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79bc10da60 0x7f79bc10fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.409+0000 7f79c2e00640 1 --2- 172.21.15.63:0/1459831929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79bc102820 0x7f79bc102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.409+0000 7f79c2e00640 1 -- 172.21.15.63:0/1459831929 >> 172.21.15.63:0/1459831929 conn(0x7f79bc0fe210 msgr2=0x7f79bc100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.409+0000 7f79c2e00640 1 -- 172.21.15.63:0/1459831929 shutdown_connections 2024-05-27T07:55:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.410+0000 7f79c2e00640 1 -- 172.21.15.63:0/1459831929 wait complete. 2024-05-27T07:55:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.411+0000 7f79c2e00640 1 Processor -- start 2024-05-27T07:55:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.411+0000 7f79c2e00640 1 -- start start 2024-05-27T07:55:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.411+0000 7f79c2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79bc102820 0x7f79bc19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.411+0000 7f79c2e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79bc10da60 0x7f79bc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.411+0000 7f79c2e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79bc1103c0 0x7f79bc1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.411+0000 7f79c2e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc115ce0 con 0x7f79bc10da60 2024-05-27T07:55:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.411+0000 7f79c2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc115b60 con 0x7f79bc102820 2024-05-27T07:55:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.411+0000 7f79c2e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc115e60 con 0x7f79bc1103c0 2024-05-27T07:55:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.412+0000 7f79c1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79bc10da60 0x7f79bc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.412+0000 7f79c1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79bc10da60 0x7f79bc19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47118/0 (socket says 172.21.15.63:47118) 2024-05-27T07:55:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.412+0000 7f79c1000640 1 -- 172.21.15.63:0/1039741489 learned_addr learned my addr 172.21.15.63:0/1039741489 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.412+0000 7f79c1a00640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79bc102820 0x7f79bc19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.412+0000 7f79c1000640 1 -- 172.21.15.63:0/1039741489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79bc1103c0 msgr2=0x7f79bc1a1000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:55:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.412+0000 7f79c1000640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79bc1103c0 0x7f79bc1a1000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.412+0000 7f79c1000640 1 -- 172.21.15.63:0/1039741489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79bc102820 msgr2=0x7f79bc19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.412+0000 7f79c1000640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79bc102820 0x7f79bc19c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.412+0000 7f79c1000640 1 -- 172.21.15.63:0/1039741489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79bc1a1640 con 0x7f79bc10da60 2024-05-27T07:55:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.412+0000 7f79c1a00640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79bc102820 0x7f79bc19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.413+0000 7f79c1000640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79bc10da60 0x7f79bc19cd40 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f79a800d770 tx=0x7f79a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.413+0000 7f79ba000640 1 -- 172.21.15.63:0/1039741489 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8017070 con 0x7f79bc10da60 2024-05-27T07:55:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.413+0000 7f79ba000640 1 -- 172.21.15.63:0/1039741489 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79a8009c30 con 0x7f79bc10da60 2024-05-27T07:55:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.413+0000 7f79ba000640 1 -- 172.21.15.63:0/1039741489 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a800f620 con 0x7f79bc10da60 2024-05-27T07:55:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.413+0000 7f79c2e00640 1 -- 172.21.15.63:0/1039741489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79bc1a1930 con 0x7f79bc10da60 2024-05-27T07:55:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.413+0000 7f79c2e00640 1 -- 172.21.15.63:0/1039741489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79bc1a1e30 con 0x7f79bc10da60 2024-05-27T07:55:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.417+0000 7f79cb570740 1 Processor -- start 2024-05-27T07:55:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.418+0000 7f79cb570740 1 -- start start 2024-05-27T07:55:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.418+0000 7f79cb570740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7621b0 0x5623ceb04a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.418+0000 7f79cb570740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7392f0 0x5623ceb04fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.418+0000 7f79cb570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7414d0 0x5623ceb054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.418+0000 7f79cb570740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5623cea75b90 con 0x5623ce7621b0 2024-05-27T07:55:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.418+0000 7f79cb570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623cea65730 con 0x5623ce7414d0 2024-05-27T07:55:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.418+0000 7f79cb570740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5623cea69770 con 0x5623ce7392f0 2024-05-27T07:55:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.418+0000 7f79ba000640 1 -- 172.21.15.63:0/1039741489 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f79a801a070 con 0x7f79bc10da60 2024-05-27T07:55:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.418+0000 7f79c2400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7621b0 0x5623ceb04a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.419+0000 7f79c2400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7621b0 0x5623ceb04a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47122/0 (socket says 172.21.15.63:47122) 2024-05-27T07:55:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.419+0000 7f79c2400640 1 -- 172.21.15.63:0/183094701 learned_addr learned my addr 172.21.15.63:0/183094701 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.419+0000 7f79c1a00640 1 --2- 172.21.15.63:0/183094701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7392f0 0x5623ceb04fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.419+0000 7f79c1000640 1 --2- 172.21.15.63:0/183094701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7414d0 0x5623ceb054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.419+0000 7f79c2400640 1 -- 172.21.15.63:0/183094701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7392f0 msgr2=0x5623ceb04fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.419+0000 7f79c2400640 1 --2- 172.21.15.63:0/183094701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7392f0 0x5623ceb04fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.419+0000 7f79c2400640 1 -- 172.21.15.63:0/183094701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7414d0 msgr2=0x5623ceb054e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.419+0000 7f79c2400640 1 --2- 172.21.15.63:0/183094701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7414d0 0x5623ceb054e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.419+0000 7f79c2400640 1 -- 172.21.15.63:0/183094701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623ce9fe8b0 con 0x5623ce7621b0 2024-05-27T07:55:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.420+0000 7f79c2400640 1 --2- 172.21.15.63:0/183094701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7621b0 0x5623ceb04a60 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f79b4006010 tx=0x7f79b40098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d6e3fb1ec7ab81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.420+0000 7f79aea00640 1 -- 172.21.15.63:0/183094701 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b40316d0 con 0x5623ce7621b0 2024-05-27T07:55:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.421+0000 7f79aea00640 1 -- 172.21.15.63:0/183094701 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79b4036070 con 0x5623ce7621b0 2024-05-27T07:55:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.421+0000 7f79aea00640 1 -- 172.21.15.63:0/183094701 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b40042a0 con 0x5623ce7621b0 2024-05-27T07:55:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.421+0000 7f79ba000640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f79a40b2a50 0x7f79a40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.421+0000 7f79ba000640 1 -- 172.21.15.63:0/1039741489 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f79a80e2550 con 0x7f79bc10da60 2024-05-27T07:55:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.421+0000 7f79c1a00640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f79a40b2a50 0x7f79a40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.422+0000 7f79cb570740 1 -- 172.21.15.63:0/183094701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7621b0 msgr2=0x5623ceb04a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.422+0000 7f79cb570740 1 --2- 172.21.15.63:0/183094701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7621b0 0x5623ceb04a60 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f79b4006010 tx=0x7f79b40098b0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.422+0000 7f79cb570740 1 -- 172.21.15.63:0/183094701 shutdown_connections 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.422+0000 7f79cb570740 1 --2- 172.21.15.63:0/183094701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7414d0 0x5623ceb054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.422+0000 7f79cb570740 1 --2- 172.21.15.63:0/183094701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7392f0 0x5623ceb04fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.422+0000 7f79cb570740 1 --2- 172.21.15.63:0/183094701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7621b0 0x5623ceb04a60 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.422+0000 7f79cb570740 1 -- 172.21.15.63:0/183094701 >> 172.21.15.63:0/183094701 conn(0x5623ce756300 msgr2=0x5623ceb01210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.422+0000 7f79cb570740 1 -- 172.21.15.63:0/183094701 shutdown_connections 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.422+0000 7f79cb570740 1 -- 172.21.15.63:0/183094701 wait complete. 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.422+0000 7f79c1a00640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f79a40b2a50 0x7f79a40b4f60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5623ceb05e10 tx=0x7f79b000b5a0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.423+0000 7f79cb570740 1 -- start start 2024-05-27T07:55:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.423+0000 7f79cb570740 1 Processor -- start 2024-05-27T07:55:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79cb570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7392f0 0x5623ceb0ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79cb570740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb0f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79cb570740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 0x5623ceb1ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79cb570740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5623ce74b790 con 0x5623ce7414d0 2024-05-27T07:55:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79cb570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623cea0dbc0 con 0x5623ce7392f0 2024-05-27T07:55:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79cb570740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5623ceb11b30 con 0x5623ce7621b0 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79cb570740 10 client.0 _ll_register_callbacks cb 0x5623ce9eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79c1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb0f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79c1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb0f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47126/0 (socket says 172.21.15.63:47126) 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79c1000640 1 -- 172.21.15.63:0/64634165 learned_addr learned my addr 172.21.15.63:0/64634165 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.424+0000 7f79c1a00640 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7392f0 0x5623ceb0ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.425+0000 7f79c2400640 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 0x5623ceb1ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.425+0000 7f79c1000640 1 -- 172.21.15.63:0/64634165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 msgr2=0x5623ceb1ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.425+0000 7f79c1000640 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 0x5623ceb1ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.425+0000 7f79c1000640 1 -- 172.21.15.63:0/64634165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7392f0 msgr2=0x5623ceb0ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.425+0000 7f79c1000640 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7392f0 0x5623ceb0ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.425+0000 7f79c1000640 1 -- 172.21.15.63:0/64634165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623ce77a210 con 0x5623ce7414d0 2024-05-27T07:55:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.425+0000 7f79c2400640 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 0x5623ceb1ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.425+0000 7f79c1000640 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb0f150 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f79a8000c00 tx=0x7f79a800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaa818173917343 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.425+0000 7f79a3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.426+0000 7f79a3400640 1 -- 172.21.15.63:0/64634165 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8021030 con 0x5623ce7414d0 2024-05-27T07:55:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.426+0000 7f79a3400640 1 -- 172.21.15.63:0/64634165 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f79a80b0d50 con 0x5623ce7414d0 2024-05-27T07:55:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.426+0000 7f79cb570740 1 -- 172.21.15.63:0/64634165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b4036070 con 0x5623ce7414d0 2024-05-27T07:55:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.426+0000 7f79a3400640 1 -- 172.21.15.63:0/64634165 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f79a80a8d60 con 0x5623ce7414d0 2024-05-27T07:55:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.426+0000 7f79cb570740 10 client.19746 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.426+0000 7f79cb570740 1 -- 172.21.15.63:0/64634165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5623ce846620 con 0x5623ce7414d0 2024-05-27T07:55:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.426+0000 7f79a3400640 1 -- 172.21.15.63:0/64634165 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79b4036070 con 0x5623ce7414d0 2024-05-27T07:55:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.427+0000 7f79a3400640 1 -- 172.21.15.63:0/64634165 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f79a80e2160 con 0x5623ce7414d0 2024-05-27T07:55:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.427+0000 7f79cb570740 10 client.19746 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.427+0000 7f79cb570740 10 client.19746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.427+0000 7f79cb570740 20 client.19746 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.427+0000 7f79cb570740 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623ce720d80 0x5623ceb20e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.427+0000 7f79cb570740 4 client.19746 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.427+0000 7f79cb570740 1 -- 172.21.15.63:0/64634165 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623cea77ae0 con 0x5623ce720d80 2024-05-27T07:55:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.427+0000 7f79c1a00640 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623ce720d80 0x5623ceb20e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.428+0000 7f79c1a00640 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623ce720d80 0x5623ceb20e10 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83d331d72dc9f62 server_cookie=2a35054b55099745 in_seq=0 out_seq=0 2024-05-27T07:55:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.428+0000 7f79a3400640 10 client.19746 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.430+0000 7f79a3400640 1 -- 172.21.15.63:0/64634165 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5623cea77ae0 con 0x5623ce720d80 2024-05-27T07:55:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.430+0000 7f79a3400640 10 client.19746 handle_command_reply: tid=0 2024-05-27T07:55:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.430+0000 7f79cb570740 1 client.19746 shutdown 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 1 -- 172.21.15.63:0/64634165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 msgr2=0x5623ceb0f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb0f150 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f79a8000c00 tx=0x7f79a800b480 comp rx=0 tx=0).stop 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 1 -- 172.21.15.63:0/64634165 shutdown_connections 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623ce720d80 0x5623ceb20e10 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 0x5623ceb1ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb0f150 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 1 --2- 172.21.15.63:0/64634165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7392f0 0x5623ceb0ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 1 -- 172.21.15.63:0/64634165 >> 172.21.15.63:0/64634165 conn(0x5623ce756300 msgr2=0x5623ceb04890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79a3400640 0 client.19746 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 1 -- 172.21.15.63:0/64634165 shutdown_connections 2024-05-27T07:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 1 -- 172.21.15.63:0/64634165 wait complete. 2024-05-27T07:55:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.431+0000 7f79cb570740 20 client.19746 trim_cache size 0 max 16384 2024-05-27T07:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.455+0000 7f79cb570740 1 Processor -- start 2024-05-27T07:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.455+0000 7f79cb570740 1 -- start start 2024-05-27T07:55:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.455+0000 7f79cb570740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7392f0 0x5623ceb10e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.455+0000 7f79cb570740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb16dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79cb570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7621b0 0x5623ceb17300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79cb570740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1b5b60 con 0x5623ce7414d0 2024-05-27T07:55:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79cb570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1b3430 con 0x5623ce7621b0 2024-05-27T07:55:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79cb570740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1b12a0 con 0x5623ce7392f0 2024-05-27T07:55:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79c1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb16dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79c1000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb16dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47130/0 (socket says 172.21.15.63:47130) 2024-05-27T07:55:59.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79c1000640 1 -- 172.21.15.63:0/2821850680 learned_addr learned my addr 172.21.15.63:0/2821850680 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79c1a00640 1 --2- 172.21.15.63:0/2821850680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7392f0 0x5623ceb10e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79c2400640 1 --2- 172.21.15.63:0/2821850680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7621b0 0x5623ceb17300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79c1000640 1 -- 172.21.15.63:0/2821850680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7392f0 msgr2=0x5623ceb10e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79c1000640 1 --2- 172.21.15.63:0/2821850680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7392f0 0x5623ceb10e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79c1000640 1 -- 172.21.15.63:0/2821850680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7621b0 msgr2=0x5623ceb17300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79c1000640 1 --2- 172.21.15.63:0/2821850680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7621b0 0x5623ceb17300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.456+0000 7f79c1000640 1 -- 172.21.15.63:0/2821850680 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623ceb1bbd0 con 0x5623ce7414d0 2024-05-27T07:55:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.457+0000 7f79c2400640 1 --2- 172.21.15.63:0/2821850680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7621b0 0x5623ceb17300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:59.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.457+0000 7f79c1000640 1 --2- 172.21.15.63:0/2821850680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb16dc0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f79a80a9110 tx=0x7f79a80910f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a8786df656da95 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.460+0000 7f79ae000640 1 -- 172.21.15.63:0/2821850680 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8021050 con 0x5623ce7414d0 2024-05-27T07:55:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.460+0000 7f79ae000640 1 -- 172.21.15.63:0/2821850680 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79a8014030 con 0x5623ce7414d0 2024-05-27T07:55:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.460+0000 7f79ae000640 1 -- 172.21.15.63:0/2821850680 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a80a8b80 con 0x5623ce7414d0 2024-05-27T07:55:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.461+0000 7f79cb570740 1 -- 172.21.15.63:0/2821850680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 msgr2=0x5623ceb16dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.461+0000 7f79cb570740 1 --2- 172.21.15.63:0/2821850680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb16dc0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f79a80a9110 tx=0x7f79a80910f0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.461+0000 7f79cb570740 1 -- 172.21.15.63:0/2821850680 shutdown_connections 2024-05-27T07:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.461+0000 7f79cb570740 1 --2- 172.21.15.63:0/2821850680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7621b0 0x5623ceb17300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.461+0000 7f79cb570740 1 --2- 172.21.15.63:0/2821850680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7414d0 0x5623ceb16dc0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.461+0000 7f79cb570740 1 --2- 172.21.15.63:0/2821850680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7392f0 0x5623ceb10e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.461+0000 7f79cb570740 1 -- 172.21.15.63:0/2821850680 >> 172.21.15.63:0/2821850680 conn(0x5623ce756300 msgr2=0x5623ceb0eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.461+0000 7f79cb570740 1 -- 172.21.15.63:0/2821850680 shutdown_connections 2024-05-27T07:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.461+0000 7f79cb570740 1 -- 172.21.15.63:0/2821850680 wait complete. 2024-05-27T07:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.462+0000 7f79cb570740 1 -- start start 2024-05-27T07:55:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.462+0000 7f79cb570740 1 Processor -- start 2024-05-27T07:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79cb570740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7392f0 0x5623ceb169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79cb570740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7414d0 0x5623ceb16f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79cb570740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 0x5623ceb80850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79cb570740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1b18f0 con 0x5623ce7392f0 2024-05-27T07:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79cb570740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1b2d60 con 0x5623ce7414d0 2024-05-27T07:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79cb570740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1b1fc0 con 0x5623ce7621b0 2024-05-27T07:55:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79cb570740 10 client.0 _ll_register_callbacks cb 0x7f79a80dfd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79c1a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7392f0 0x5623ceb169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79c1a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7392f0 0x5623ceb169f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47146/0 (socket says 172.21.15.63:47146) 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79c1a00640 1 -- 172.21.15.63:0/2058226401 learned_addr learned my addr 172.21.15.63:0/2058226401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79c1000640 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7414d0 0x5623ceb16f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79c2400640 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 0x5623ceb80850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79c1a00640 1 -- 172.21.15.63:0/2058226401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 msgr2=0x5623ceb80850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79c1a00640 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 0x5623ceb80850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79c1a00640 1 -- 172.21.15.63:0/2058226401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7414d0 msgr2=0x5623ceb16f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.463+0000 7f79c1a00640 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7414d0 0x5623ceb16f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.464+0000 7f79c1a00640 1 -- 172.21.15.63:0/2058226401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623ceb80e10 con 0x5623ce7392f0 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.464+0000 7f79c1a00640 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7392f0 0x5623ceb169f0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x5623ceb021a0 tx=0x7f79b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7079458e91ab597 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.464+0000 7f79a3400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:55:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.464+0000 7f79a3400640 1 -- 172.21.15.63:0/2058226401 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b00305f0 con 0x5623ce7392f0 2024-05-27T07:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.464+0000 7f79a3400640 1 -- 172.21.15.63:0/2058226401 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7f79b0030790 con 0x5623ce7392f0 2024-05-27T07:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.464+0000 7f79cb570740 1 -- 172.21.15.63:0/2058226401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a8014030 con 0x5623ce7392f0 2024-05-27T07:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79a3400640 1 -- 172.21.15.63:0/2058226401 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f79b0038050 con 0x5623ce7392f0 2024-05-27T07:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79cb570740 10 client.19752 fetch_fsmap learned FSMap version 33 2024-05-27T07:55:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79cb570740 1 -- 172.21.15.63:0/2058226401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5623ceb89320 con 0x5623ce7392f0 2024-05-27T07:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79a3400640 1 -- 172.21.15.63:0/2058226401 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79a8014030 con 0x5623ce7392f0 2024-05-27T07:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79a3400640 1 -- 172.21.15.63:0/2058226401 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f79b001fa50 con 0x5623ce7392f0 2024-05-27T07:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79cb570740 10 client.19752 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79cb570740 10 client.19752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79cb570740 20 client.19752 populate_metadata read hostname 'smithi063' 2024-05-27T07:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79cb570740 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623ce6f61c0 0x5623ceb8bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79cb570740 4 client.19752 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:55:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.465+0000 7f79cb570740 1 -- 172.21.15.63:0/2058226401 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623ceb05420 con 0x5623ce6f61c0 2024-05-27T07:55:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.466+0000 7f79c1000640 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623ce6f61c0 0x5623ceb8bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.467+0000 7f79c1000640 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623ce6f61c0 0x5623ceb8bae0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb7192e14d1258f server_cookie=cfb8e22b2730c6fd in_seq=0 out_seq=0 2024-05-27T07:55:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.467+0000 7f79a3400640 10 client.19752 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.499+0000 7f79a3400640 1 -- 172.21.15.63:0/2058226401 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5623ceb05420 con 0x5623ce6f61c0 2024-05-27T07:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.499+0000 7f79a3400640 10 client.19752 handle_command_reply: tid=0 2024-05-27T07:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.499+0000 7f79cb570740 1 client.19752 shutdown 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.499+0000 7f79cb570740 1 -- 172.21.15.63:0/2058226401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7392f0 msgr2=0x5623ceb169f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.499+0000 7f79cb570740 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7392f0 0x5623ceb169f0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x5623ceb021a0 tx=0x7f79b00046c0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.500+0000 7f79cb570740 1 -- 172.21.15.63:0/2058226401 shutdown_connections 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.500+0000 7f79cb570740 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623ce6f61c0 0x5623ceb8bae0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.500+0000 7f79cb570740 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623ce7621b0 0x5623ceb80850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.500+0000 7f79cb570740 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ce7414d0 0x5623ceb16f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.500+0000 7f79cb570740 1 --2- 172.21.15.63:0/2058226401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623ce7392f0 0x5623ceb169f0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.500+0000 7f79a3400640 0 client.19752 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.500+0000 7f79cb570740 1 -- 172.21.15.63:0/2058226401 >> 172.21.15.63:0/2058226401 conn(0x5623ce756300 msgr2=0x5623ceb093e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.500+0000 7f79cb570740 1 -- 172.21.15.63:0/2058226401 shutdown_connections 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.500+0000 7f79cb570740 1 -- 172.21.15.63:0/2058226401 wait complete. 2024-05-27T07:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.500+0000 7f79cb570740 20 client.19752 trim_cache size 0 max 16384 2024-05-27T07:55:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:55:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:55:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 -- 172.21.15.63:0/1039741489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f79a40b2a50 msgr2=0x7f79a40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f79a40b2a50 0x7f79a40b4f60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5623ceb05e10 tx=0x7f79b000b5a0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 -- 172.21.15.63:0/1039741489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79bc10da60 msgr2=0x7f79bc19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79bc10da60 0x7f79bc19cd40 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f79a800d770 tx=0x7f79a800dc30 comp rx=0 tx=0).stop 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 -- 172.21.15.63:0/1039741489 shutdown_connections 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f79a40b2a50 0x7f79a40b4f60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79bc1103c0 0x7f79bc1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79bc10da60 0x7f79bc19cd40 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 --2- 172.21.15.63:0/1039741489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79bc102820 0x7f79bc19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 -- 172.21.15.63:0/1039741489 >> 172.21.15.63:0/1039741489 conn(0x7f79bc0fe210 msgr2=0x7f79bc100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 -- 172.21.15.63:0/1039741489 shutdown_connections 2024-05-27T07:55:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:55:59.501+0000 7f79c2e00640 1 -- 172.21.15.63:0/1039741489 wait complete. 2024-05-27T07:55:59.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:55:59 smithi086 ceph-mon[34687]: pgmap v707: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.7 MiB/s wr, 14 op/s 2024-05-27T07:55:59.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:55:59 smithi149 ceph-mon[35408]: pgmap v707: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.7 MiB/s wr, 14 op/s 2024-05-27T07:55:59.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:55:59 smithi063 ceph-mon[31613]: pgmap v707: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.7 MiB/s wr, 14 op/s 2024-05-27T07:56:00.935 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:56:00.936 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:56:01.468 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:01.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:01 smithi149 ceph-mon[35408]: pgmap v708: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 7 op/s 2024-05-27T07:56:01.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:01 smithi063 ceph-mon[31613]: pgmap v708: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 7 op/s 2024-05-27T07:56:01.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:01 smithi086 ceph-mon[34687]: pgmap v708: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 7 op/s 2024-05-27T07:56:03.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:03 smithi063 ceph-mon[31613]: pgmap v709: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.5 MiB/s wr, 31 op/s 2024-05-27T07:56:03.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:03 smithi086 ceph-mon[34687]: pgmap v709: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.5 MiB/s wr, 31 op/s 2024-05-27T07:56:03.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:03 smithi149 ceph-mon[35408]: pgmap v709: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.5 MiB/s wr, 31 op/s 2024-05-27T07:56:04.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.076+0000 7fbc83e00700 1 -- 172.21.15.63:0/401629903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc84103070 msgr2=0x7fbc84103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:04.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.076+0000 7fbc83e00700 1 --2- 172.21.15.63:0/401629903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc84103070 0x7fbc84103480 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400b0a0 tx=0x7fbc7400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.076+0000 7fbc83e00700 1 -- 172.21.15.63:0/401629903 shutdown_connections 2024-05-27T07:56:04.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.076+0000 7fbc83e00700 1 --2- 172.21.15.63:0/401629903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc841084c0 0x7fbc84108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.076+0000 7fbc83e00700 1 --2- 172.21.15.63:0/401629903 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc84103070 0x7fbc84103480 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.076+0000 7fbc83e00700 1 --2- 172.21.15.63:0/401629903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc841039c0 0x7fbc84107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.076+0000 7fbc83e00700 1 -- 172.21.15.63:0/401629903 >> 172.21.15.63:0/401629903 conn(0x7fbc84078ab0 msgr2=0x7fbc84078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:04.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.076+0000 7fbc83e00700 1 -- 172.21.15.63:0/401629903 shutdown_connections 2024-05-27T07:56:04.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.076+0000 7fbc83e00700 1 -- 172.21.15.63:0/401629903 wait complete. 2024-05-27T07:56:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.078+0000 7fbc83e00700 1 Processor -- start 2024-05-27T07:56:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.078+0000 7fbc83e00700 1 -- start start 2024-05-27T07:56:04.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc83e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc84103070 0x7fbc841a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc83e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc841039c0 0x7fbc841a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc83e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc841084c0 0x7fbc841a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc83e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8411a750 con 0x7fbc841084c0 2024-05-27T07:56:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc83e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8411a5d0 con 0x7fbc84103070 2024-05-27T07:56:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc83e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8411a8d0 con 0x7fbc841039c0 2024-05-27T07:56:04.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc83400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc841084c0 0x7fbc841a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc83400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc841084c0 0x7fbc841a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47160/0 (socket says 172.21.15.63:47160) 2024-05-27T07:56:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc83400700 1 -- 172.21.15.63:0/3059490258 learned_addr learned my addr 172.21.15.63:0/3059490258 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc7a000700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc841039c0 0x7fbc841a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.079+0000 7fbc82a00700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc84103070 0x7fbc841a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.080+0000 7fbc83400700 1 -- 172.21.15.63:0/3059490258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc841039c0 msgr2=0x7fbc841a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.080+0000 7fbc83400700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc841039c0 0x7fbc841a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.080+0000 7fbc83400700 1 -- 172.21.15.63:0/3059490258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc84103070 msgr2=0x7fbc841a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.080+0000 7fbc83400700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc84103070 0x7fbc841a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.080+0000 7fbc83400700 1 -- 172.21.15.63:0/3059490258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc841a6080 con 0x7fbc841084c0 2024-05-27T07:56:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.080+0000 7fbc83400700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc841084c0 0x7fbc841a5a90 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c900 tx=0x7fbc7c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.081+0000 7fbc7be00700 1 -- 172.21.15.63:0/3059490258 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0078d0 con 0x7fbc841084c0 2024-05-27T07:56:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.081+0000 7fbc7be00700 1 -- 172.21.15.63:0/3059490258 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc7c00f450 con 0x7fbc841084c0 2024-05-27T07:56:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.081+0000 7fbc83e00700 1 -- 172.21.15.63:0/3059490258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc841a6310 con 0x7fbc841084c0 2024-05-27T07:56:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.081+0000 7fbc7be00700 1 -- 172.21.15.63:0/3059490258 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c018660 con 0x7fbc841084c0 2024-05-27T07:56:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.081+0000 7fbc83e00700 1 -- 172.21.15.63:0/3059490258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc841a6800 con 0x7fbc841084c0 2024-05-27T07:56:04.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.086+0000 7fbc7be00700 1 -- 172.21.15.63:0/3059490258 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbc7c007a70 con 0x7fbc841084c0 2024-05-27T07:56:04.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.086+0000 7fbc98218b80 1 Processor -- start 2024-05-27T07:56:04.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.086+0000 7fbc98218b80 1 -- start start 2024-05-27T07:56:04.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.087+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c90a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.087+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c78aec0 0x56408c90ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.087+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c914340 0x56408c916760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.087+0000 7fbc98218b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56408c837ec0 con 0x56408c876a70 2024-05-27T07:56:04.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.087+0000 7fbc98218b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56408c827a60 con 0x56408c914340 2024-05-27T07:56:04.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.087+0000 7fbc98218b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56408c82baa0 con 0x56408c78aec0 2024-05-27T07:56:04.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.087+0000 7fbc7a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c90a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.087+0000 7fbc7a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c90a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47174/0 (socket says 172.21.15.63:47174) 2024-05-27T07:56:04.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.087+0000 7fbc7a000700 1 -- 172.21.15.63:0/1589653257 learned_addr learned my addr 172.21.15.63:0/1589653257 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:04.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.088+0000 7fbc7a000700 1 --2- 172.21.15.63:0/1589653257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c914340 0x56408c916760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.088+0000 7fbc7a000700 1 -- 172.21.15.63:0/1589653257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c78aec0 msgr2=0x56408c90ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:04.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.088+0000 7fbc7a000700 1 --2- 172.21.15.63:0/1589653257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c78aec0 0x56408c90ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.088+0000 7fbc7a000700 1 -- 172.21.15.63:0/1589653257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c914340 msgr2=0x56408c916760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:04.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.088+0000 7fbc7a000700 1 --2- 172.21.15.63:0/1589653257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c914340 0x56408c916760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.088+0000 7fbc7a000700 1 -- 172.21.15.63:0/1589653257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408c7f04a0 con 0x56408c876a70 2024-05-27T07:56:04.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.088+0000 7fbc7be00700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc680b29f0 0x7fbc680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.088+0000 7fbc7a000700 1 --2- 172.21.15.63:0/1589653257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c90a7d0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c012620 tx=0x7fbc6c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af85ed5b452f302f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:04.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.089+0000 7fbc7be00700 1 -- 172.21.15.63:0/3059490258 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7fbc7c0e2820 con 0x7fbc841084c0 2024-05-27T07:56:04.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.089+0000 7fbc82a00700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc680b29f0 0x7fbc680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.089+0000 7fbc73e00700 1 -- 172.21.15.63:0/1589653257 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c004030 con 0x56408c876a70 2024-05-27T07:56:04.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.089+0000 7fbc73e00700 1 -- 172.21.15.63:0/1589653257 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc6c004830 con 0x56408c876a70 2024-05-27T07:56:04.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.089+0000 7fbc73e00700 1 -- 172.21.15.63:0/1589653257 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c005280 con 0x56408c876a70 2024-05-27T07:56:04.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.090+0000 7fbc98218b80 1 -- 172.21.15.63:0/1589653257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 msgr2=0x56408c90a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:04.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.090+0000 7fbc82a00700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc680b29f0 0x7fbc680b4f00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009fd0 tx=0x7fbc74006000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:04.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.090+0000 7fbc98218b80 1 --2- 172.21.15.63:0/1589653257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c90a7d0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c012620 tx=0x7fbc6c00b8c0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.090+0000 7fbc98218b80 1 -- 172.21.15.63:0/1589653257 shutdown_connections 2024-05-27T07:56:04.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.091+0000 7fbc98218b80 1 --2- 172.21.15.63:0/1589653257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c914340 0x56408c916760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.091+0000 7fbc98218b80 1 --2- 172.21.15.63:0/1589653257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c90a7d0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.091+0000 7fbc98218b80 1 --2- 172.21.15.63:0/1589653257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c78aec0 0x56408c90ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.091+0000 7fbc98218b80 1 -- 172.21.15.63:0/1589653257 >> 172.21.15.63:0/1589653257 conn(0x56408c824410 msgr2=0x56408c905790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:04.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.091+0000 7fbc98218b80 1 -- 172.21.15.63:0/1589653257 shutdown_connections 2024-05-27T07:56:04.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.091+0000 7fbc98218b80 1 -- 172.21.15.63:0/1589653257 wait complete. 2024-05-27T07:56:04.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.092+0000 7fbc98218b80 1 -- start start 2024-05-27T07:56:04.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.092+0000 7fbc98218b80 1 Processor -- start 2024-05-27T07:56:04.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.092+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c911360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.092+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c9118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.093+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c914340 0x56408c911de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.093+0000 7fbc98218b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56408c72e020 con 0x56408c876a70 2024-05-27T07:56:04.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.093+0000 7fbc98218b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56408c7c87e0 con 0x56408c78aec0 2024-05-27T07:56:04.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.093+0000 7fbc98218b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56408c916e80 con 0x56408c914340 2024-05-27T07:56:04.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.093+0000 7fbc98218b80 10 client.0 _ll_register_callbacks cb 0x56408c7d1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:04.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.093+0000 7fbc7a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c9118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.093+0000 7fbc7a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c9118a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47188/0 (socket says 172.21.15.63:47188) 2024-05-27T07:56:04.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.093+0000 7fbc7a000700 1 -- 172.21.15.63:0/310594220 learned_addr learned my addr 172.21.15.63:0/310594220 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:04.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.093+0000 7fbc82a00700 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c911360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.093+0000 7fbc83400700 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c914340 0x56408c911de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.094+0000 7fbc7a000700 1 -- 172.21.15.63:0/310594220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c914340 msgr2=0x56408c911de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:04.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.094+0000 7fbc7a000700 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c914340 0x56408c911de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.094+0000 7fbc7a000700 1 -- 172.21.15.63:0/310594220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 msgr2=0x56408c911360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:04.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.094+0000 7fbc7a000700 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c911360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.094+0000 7fbc7a000700 1 -- 172.21.15.63:0/310594220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408c777950 con 0x56408c876a70 2024-05-27T07:56:04.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.094+0000 7fbc7a000700 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c9118a0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c004ca0 tx=0x7fbc6c004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157077d1bfd37059 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:04.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.094+0000 7fbc70c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:04.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.095+0000 7fbc70c00700 1 -- 172.21.15.63:0/310594220 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c004240 con 0x56408c876a70 2024-05-27T07:56:04.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.095+0000 7fbc98218b80 1 -- 172.21.15.63:0/310594220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc6c004830 con 0x56408c876a70 2024-05-27T07:56:04.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.095+0000 7fbc70c00700 1 -- 172.21.15.63:0/310594220 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7fbc6c0043e0 con 0x56408c876a70 2024-05-27T07:56:04.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.095+0000 7fbc70c00700 1 -- 172.21.15.63:0/310594220 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbc6c02f050 con 0x56408c876a70 2024-05-27T07:56:04.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.095+0000 7fbc98218b80 10 client.19764 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:04.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.096+0000 7fbc98218b80 1 -- 172.21.15.63:0/310594220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56408c7f80e0 con 0x56408c876a70 2024-05-27T07:56:04.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.096+0000 7fbc70c00700 1 -- 172.21.15.63:0/310594220 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc6c004830 con 0x56408c876a70 2024-05-27T07:56:04.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.096+0000 7fbc70c00700 1 -- 172.21.15.63:0/310594220 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbc6c0038b0 con 0x56408c876a70 2024-05-27T07:56:04.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.096+0000 7fbc98218b80 10 client.19764 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:04.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.096+0000 7fbc98218b80 10 client.19764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:04.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.097+0000 7fbc98218b80 20 client.19764 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:04.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.097+0000 7fbc98218b80 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408c923d70 0x56408c926190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.097+0000 7fbc98218b80 4 client.19764 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:04.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.097+0000 7fbc98218b80 1 -- 172.21.15.63:0/310594220 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56408c839e10 con 0x56408c923d70 2024-05-27T07:56:04.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.097+0000 7fbc82a00700 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408c923d70 0x56408c926190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.098+0000 7fbc82a00700 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408c923d70 0x56408c926190 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26692ba16bbaebe2 server_cookie=98f4b7247077067d in_seq=0 out_seq=0 2024-05-27T07:56:04.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.098+0000 7fbc70c00700 10 client.19764 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:04.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.100+0000 7fbc70c00700 1 -- 172.21.15.63:0/310594220 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56408c839e10 con 0x56408c923d70 2024-05-27T07:56:04.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.100+0000 7fbc70c00700 10 client.19764 handle_command_reply: tid=0 2024-05-27T07:56:04.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.101+0000 7fbc98218b80 1 client.19764 shutdown 2024-05-27T07:56:04.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.101+0000 7fbc98218b80 1 -- 172.21.15.63:0/310594220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 msgr2=0x56408c9118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:04.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.101+0000 7fbc98218b80 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c9118a0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c004ca0 tx=0x7fbc6c004cd0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.101+0000 7fbc98218b80 1 -- 172.21.15.63:0/310594220 shutdown_connections 2024-05-27T07:56:04.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.101+0000 7fbc98218b80 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408c923d70 0x56408c926190 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.101+0000 7fbc98218b80 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c914340 0x56408c911de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.101+0000 7fbc98218b80 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c911360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.101+0000 7fbc98218b80 1 --2- 172.21.15.63:0/310594220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c9118a0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.101+0000 7fbc98218b80 1 -- 172.21.15.63:0/310594220 >> 172.21.15.63:0/310594220 conn(0x56408c824410 msgr2=0x56408c9053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:04.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.101+0000 7fbc70c00700 0 client.19764 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:04.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.102+0000 7fbc98218b80 1 -- 172.21.15.63:0/310594220 shutdown_connections 2024-05-27T07:56:04.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.102+0000 7fbc98218b80 1 -- 172.21.15.63:0/310594220 wait complete. 2024-05-27T07:56:04.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.102+0000 7fbc98218b80 20 client.19764 trim_cache size 0 max 16384 2024-05-27T07:56:04.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.132+0000 7fbc98218b80 1 Processor -- start 2024-05-27T07:56:04.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.132+0000 7fbc98218b80 1 -- start start 2024-05-27T07:56:04.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.133+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c90cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.133+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c876a70 0x56408c9057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.133+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c914340 0x56408c928da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.133+0000 7fbc98218b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841b9130 con 0x56408c914340 2024-05-27T07:56:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.133+0000 7fbc98218b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841b69c0 con 0x56408c78aec0 2024-05-27T07:56:04.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.133+0000 7fbc98218b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841b4830 con 0x56408c876a70 2024-05-27T07:56:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.133+0000 7fbc83400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c914340 0x56408c928da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.133+0000 7fbc83400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c914340 0x56408c928da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47190/0 (socket says 172.21.15.63:47190) 2024-05-27T07:56:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.133+0000 7fbc83400700 1 -- 172.21.15.63:0/2499250901 learned_addr learned my addr 172.21.15.63:0/2499250901 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.134+0000 7fbc82a00700 1 --2- 172.21.15.63:0/2499250901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c90cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.134+0000 7fbc7a000700 1 --2- 172.21.15.63:0/2499250901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c876a70 0x56408c9057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.134+0000 7fbc83400700 1 -- 172.21.15.63:0/2499250901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c876a70 msgr2=0x56408c9057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:04.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.134+0000 7fbc83400700 1 --2- 172.21.15.63:0/2499250901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c876a70 0x56408c9057c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.134+0000 7fbc83400700 1 -- 172.21.15.63:0/2499250901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 msgr2=0x56408c90cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.134+0000 7fbc83400700 1 --2- 172.21.15.63:0/2499250901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c90cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.134+0000 7fbc83400700 1 -- 172.21.15.63:0/2499250901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408c913c00 con 0x56408c914340 2024-05-27T07:56:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.134+0000 7fbc82a00700 1 --2- 172.21.15.63:0/2499250901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c90cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.134+0000 7fbc83400700 1 --2- 172.21.15.63:0/2499250901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c914340 0x56408c928da0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c8d0 tx=0x7fbc7c007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793b00d63c7dc61a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.135+0000 7fbc73400700 1 -- 172.21.15.63:0/2499250901 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0a96e0 con 0x56408c914340 2024-05-27T07:56:04.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.135+0000 7fbc73400700 1 -- 172.21.15.63:0/2499250901 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc7c0ada90 con 0x56408c914340 2024-05-27T07:56:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.135+0000 7fbc73400700 1 -- 172.21.15.63:0/2499250901 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0dff10 con 0x56408c914340 2024-05-27T07:56:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.136+0000 7fbc98218b80 1 -- 172.21.15.63:0/2499250901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c914340 msgr2=0x56408c928da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.136+0000 7fbc98218b80 1 --2- 172.21.15.63:0/2499250901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c914340 0x56408c928da0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c8d0 tx=0x7fbc7c007340 comp rx=0 tx=0).stop 2024-05-27T07:56:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.136+0000 7fbc98218b80 1 -- 172.21.15.63:0/2499250901 shutdown_connections 2024-05-27T07:56:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.136+0000 7fbc98218b80 1 --2- 172.21.15.63:0/2499250901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c914340 0x56408c928da0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.136+0000 7fbc98218b80 1 --2- 172.21.15.63:0/2499250901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c90cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.136+0000 7fbc98218b80 1 --2- 172.21.15.63:0/2499250901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c876a70 0x56408c9057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.136+0000 7fbc98218b80 1 -- 172.21.15.63:0/2499250901 >> 172.21.15.63:0/2499250901 conn(0x56408c824410 msgr2=0x56408c909810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.136+0000 7fbc98218b80 1 -- 172.21.15.63:0/2499250901 shutdown_connections 2024-05-27T07:56:04.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.136+0000 7fbc98218b80 1 -- 172.21.15.63:0/2499250901 wait complete. 2024-05-27T07:56:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.137+0000 7fbc98218b80 1 -- start start 2024-05-27T07:56:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.137+0000 7fbc98218b80 1 Processor -- start 2024-05-27T07:56:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c926c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c927190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc98218b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c914340 0x56408c9276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc98218b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841b4e80 con 0x56408c876a70 2024-05-27T07:56:04.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc98218b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841b62f0 con 0x56408c78aec0 2024-05-27T07:56:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc98218b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841b5550 con 0x56408c914340 2024-05-27T07:56:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc98218b80 10 client.0 _ll_register_callbacks cb 0x56408c7d1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc7a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c927190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc7a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c927190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47192/0 (socket says 172.21.15.63:47192) 2024-05-27T07:56:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc7a000700 1 -- 172.21.15.63:0/540069624 learned_addr learned my addr 172.21.15.63:0/540069624 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc82a00700 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c926c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc83400700 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c914340 0x56408c9276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc7a000700 1 -- 172.21.15.63:0/540069624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c914340 msgr2=0x56408c9276d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.138+0000 7fbc7a000700 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c914340 0x56408c9276d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.139+0000 7fbc7a000700 1 -- 172.21.15.63:0/540069624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 msgr2=0x56408c926c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.139+0000 7fbc7a000700 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c926c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.139+0000 7fbc7a000700 1 -- 172.21.15.63:0/540069624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408c98d660 con 0x56408c876a70 2024-05-27T07:56:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.139+0000 7fbc7a000700 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c927190 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x56408c90a3d0 tx=0x7fbc6c012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25175c872d1a7d5e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:04.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.139+0000 7fbc70c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:04.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.139+0000 7fbc70c00700 1 -- 172.21.15.63:0/540069624 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c037020 con 0x56408c876a70 2024-05-27T07:56:04.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.139+0000 7fbc98218b80 1 -- 172.21.15.63:0/540069624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc7c0ada90 con 0x56408c876a70 2024-05-27T07:56:04.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.139+0000 7fbc70c00700 1 -- 172.21.15.63:0/540069624 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9483+0+0 (secure 0 0 0) 0x7fbc6c00b810 con 0x56408c876a70 2024-05-27T07:56:04.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.140+0000 7fbc70c00700 1 -- 172.21.15.63:0/540069624 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbc6c010460 con 0x56408c876a70 2024-05-27T07:56:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.140+0000 7fbc98218b80 10 client.19770 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.140+0000 7fbc98218b80 1 -- 172.21.15.63:0/540069624 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56408c98e900 con 0x56408c876a70 2024-05-27T07:56:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.140+0000 7fbc70c00700 1 -- 172.21.15.63:0/540069624 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc7c0ada90 con 0x56408c876a70 2024-05-27T07:56:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.141+0000 7fbc70c00700 1 -- 172.21.15.63:0/540069624 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fbc6c010780 con 0x56408c876a70 2024-05-27T07:56:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.141+0000 7fbc98218b80 10 client.19770 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.141+0000 7fbc98218b80 10 client.19770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.141+0000 7fbc98218b80 20 client.19770 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.141+0000 7fbc98218b80 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408c98f000 0x56408c991420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.141+0000 7fbc98218b80 4 client.19770 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:56:04.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.141+0000 7fbc98218b80 1 -- 172.21.15.63:0/540069624 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56408c8335f0 con 0x56408c98f000 2024-05-27T07:56:04.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.141+0000 7fbc82a00700 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408c98f000 0x56408c991420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:04.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.143+0000 7fbc82a00700 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408c98f000 0x56408c991420 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a6a9ae1503ded7 server_cookie=b3a926ab7db1b58d in_seq=0 out_seq=0 2024-05-27T07:56:04.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.143+0000 7fbc70c00700 10 client.19770 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.145+0000 7fbc70c00700 1 -- 172.21.15.63:0/540069624 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56408c8335f0 con 0x56408c98f000 2024-05-27T07:56:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.145+0000 7fbc70c00700 10 client.19770 handle_command_reply: tid=0 2024-05-27T07:56:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.145+0000 7fbc98218b80 1 client.19770 shutdown 2024-05-27T07:56:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.145+0000 7fbc98218b80 1 -- 172.21.15.63:0/540069624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 msgr2=0x56408c927190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.145+0000 7fbc98218b80 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c927190 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x56408c90a3d0 tx=0x7fbc6c012d70 comp rx=0 tx=0).stop 2024-05-27T07:56:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.145+0000 7fbc98218b80 1 -- 172.21.15.63:0/540069624 shutdown_connections 2024-05-27T07:56:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.145+0000 7fbc98218b80 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408c98f000 0x56408c991420 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.146+0000 7fbc98218b80 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408c914340 0x56408c9276d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.146+0000 7fbc98218b80 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408c78aec0 0x56408c926c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.146+0000 7fbc98218b80 1 --2- 172.21.15.63:0/540069624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408c876a70 0x56408c927190 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.146+0000 7fbc98218b80 1 -- 172.21.15.63:0/540069624 >> 172.21.15.63:0/540069624 conn(0x56408c824410 msgr2=0x56408c909810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:04.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.146+0000 7fbc70c00700 0 client.19770 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:04.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.146+0000 7fbc98218b80 1 -- 172.21.15.63:0/540069624 shutdown_connections 2024-05-27T07:56:04.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.146+0000 7fbc98218b80 1 -- 172.21.15.63:0/540069624 wait complete. 2024-05-27T07:56:04.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.146+0000 7fbc98218b80 20 client.19770 trim_cache size 0 max 16384 2024-05-27T07:56:04.148 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:56:04.148 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:56:04.148 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "404d86f3-5893-4f14-9dd6-0be1ba5207e2", 2024-05-27T07:56:04.148 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:56:04.148 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:56:04.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.147+0000 7fbc83e00700 1 -- 172.21.15.63:0/3059490258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc680b29f0 msgr2=0x7fbc680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:04.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.147+0000 7fbc83e00700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc680b29f0 0x7fbc680b4f00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009fd0 tx=0x7fbc74006000 comp rx=0 tx=0).stop 2024-05-27T07:56:04.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.147+0000 7fbc83e00700 1 -- 172.21.15.63:0/3059490258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc841084c0 msgr2=0x7fbc841a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:04.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.147+0000 7fbc83e00700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc841084c0 0x7fbc841a5a90 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c900 tx=0x7fbc7c00cc10 comp rx=0 tx=0).stop 2024-05-27T07:56:04.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.147+0000 7fbc83e00700 1 -- 172.21.15.63:0/3059490258 shutdown_connections 2024-05-27T07:56:04.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.147+0000 7fbc83e00700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc680b29f0 0x7fbc680b4f00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.147+0000 7fbc83e00700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc841084c0 0x7fbc841a5a90 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.147+0000 7fbc83e00700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc84103070 0x7fbc841a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.147+0000 7fbc83e00700 1 --2- 172.21.15.63:0/3059490258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc841039c0 0x7fbc841a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:04.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.147+0000 7fbc83e00700 1 -- 172.21.15.63:0/3059490258 >> 172.21.15.63:0/3059490258 conn(0x7fbc84078ab0 msgr2=0x7fbc84105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:04.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.148+0000 7fbc83e00700 1 -- 172.21.15.63:0/3059490258 shutdown_connections 2024-05-27T07:56:04.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:04.148+0000 7fbc83e00700 1 -- 172.21.15.63:0/3059490258 wait complete. 2024-05-27T07:56:05.165 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:56:05.655 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:05.809 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: pgmap v710: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.1 MiB/s wr, 27 op/s 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:56:05.810 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:56:05.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:56:05.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.811 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:05 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: pgmap v710: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.1 MiB/s wr, 27 op/s 2024-05-27T07:56:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:56:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:56:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:56:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:05 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: pgmap v710: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.1 MiB/s wr, 27 op/s 2024-05-27T07:56:05.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:56:05.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:56:05.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:56:05.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:56:05.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:56:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:56:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:56:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:56:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:05 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:07.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:07 smithi086 ceph-mon[34687]: pgmap v711: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.1 MiB/s wr, 27 op/s 2024-05-27T07:56:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:07 smithi086 ceph-mon[34687]: osdmap e126: 12 total, 12 up, 12 in 2024-05-27T07:56:07.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:07 smithi149 ceph-mon[35408]: pgmap v711: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.1 MiB/s wr, 27 op/s 2024-05-27T07:56:07.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:07 smithi149 ceph-mon[35408]: osdmap e126: 12 total, 12 up, 12 in 2024-05-27T07:56:07.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:07 smithi063 ceph-mon[31613]: pgmap v711: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.1 MiB/s wr, 27 op/s 2024-05-27T07:56:07.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:07 smithi063 ceph-mon[31613]: osdmap e126: 12 total, 12 up, 12 in 2024-05-27T07:56:09.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.675+0000 7f043e400700 1 -- 172.21.15.63:0/3402439239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0438075af0 msgr2=0x7f0438075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:09.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.675+0000 7f043e400700 1 --2- 172.21.15.63:0/3402439239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0438075af0 0x7f0438075f00 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f0428009990 tx=0x7f0428009ca0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.675+0000 7f043e400700 1 -- 172.21.15.63:0/3402439239 shutdown_connections 2024-05-27T07:56:09.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.675+0000 7f043e400700 1 --2- 172.21.15.63:0/3402439239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438110bf0 0x7f0438113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.675+0000 7f043e400700 1 --2- 172.21.15.63:0/3402439239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0438075af0 0x7f0438075f00 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.675+0000 7f043e400700 1 --2- 172.21.15.63:0/3402439239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0438076440 0x7f04381106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.675+0000 7f043e400700 1 -- 172.21.15.63:0/3402439239 >> 172.21.15.63:0/3402439239 conn(0x7f0438100030 msgr2=0x7f0438102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:09.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.675+0000 7f043e400700 1 -- 172.21.15.63:0/3402439239 shutdown_connections 2024-05-27T07:56:09.677 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:09 smithi063 ceph-mon[31613]: pgmap v713: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:56:09.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.676+0000 7f043e400700 1 -- 172.21.15.63:0/3402439239 wait complete. 2024-05-27T07:56:09.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.677+0000 7f043e400700 1 Processor -- start 2024-05-27T07:56:09.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.678+0000 7f043e400700 1 -- start start 2024-05-27T07:56:09.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.678+0000 7f043e400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0438075af0 0x7f043819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.678+0000 7f043e400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438076440 0x7f043819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.678+0000 7f043e400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0438110bf0 0x7f04381a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.678+0000 7f043e400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04381165d0 con 0x7f0438110bf0 2024-05-27T07:56:09.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.678+0000 7f043e400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0438116450 con 0x7f0438076440 2024-05-27T07:56:09.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.678+0000 7f043e400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0438116750 con 0x7f0438075af0 2024-05-27T07:56:09.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.679+0000 7f0437e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0438110bf0 0x7f04381a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.679+0000 7f0437e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0438110bf0 0x7f04381a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37300/0 (socket says 172.21.15.63:37300) 2024-05-27T07:56:09.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.679+0000 7f0437e00700 1 -- 172.21.15.63:0/1489761855 learned_addr learned my addr 172.21.15.63:0/1489761855 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:09.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.679+0000 7f0436a00700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438076440 0x7f043819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.680+0000 7f0437400700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0438075af0 0x7f043819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.680+0000 7f0437e00700 1 -- 172.21.15.63:0/1489761855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0438075af0 msgr2=0x7f043819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:09.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.680+0000 7f0437e00700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0438075af0 0x7f043819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.680+0000 7f0437e00700 1 -- 172.21.15.63:0/1489761855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438076440 msgr2=0x7f043819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:09.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.680+0000 7f0437e00700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438076440 0x7f043819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.680+0000 7f0437e00700 1 -- 172.21.15.63:0/1489761855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04381a1e00 con 0x7f0438110bf0 2024-05-27T07:56:09.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.680+0000 7f0437400700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0438075af0 0x7f043819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:09.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.681+0000 7f0437e00700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0438110bf0 0x7f04381a1810 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f0430007d40 tx=0x7f043000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:09.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.681+0000 7f042fe00700 1 -- 172.21.15.63:0/1489761855 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430016020 con 0x7f0438110bf0 2024-05-27T07:56:09.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.681+0000 7f042fe00700 1 -- 172.21.15.63:0/1489761855 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f043000ad40 con 0x7f0438110bf0 2024-05-27T07:56:09.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.681+0000 7f042fe00700 1 -- 172.21.15.63:0/1489761855 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430012e60 con 0x7f0438110bf0 2024-05-27T07:56:09.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.681+0000 7f043e400700 1 -- 172.21.15.63:0/1489761855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04381a20f0 con 0x7f0438110bf0 2024-05-27T07:56:09.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.682+0000 7f043e400700 1 -- 172.21.15.63:0/1489761855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04381a25e0 con 0x7f0438110bf0 2024-05-27T07:56:09.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.687+0000 7f042fe00700 1 -- 172.21.15.63:0/1489761855 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0430019020 con 0x7f0438110bf0 2024-05-27T07:56:09.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.688+0000 7f044ceb6b80 1 Processor -- start 2024-05-27T07:56:09.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.688+0000 7f044ceb6b80 1 -- start start 2024-05-27T07:56:09.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.688+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8da9e40 0x5613b8f2e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.688+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8f2ed30 0x5613b8f28870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.688+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.689+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5613b8e56e40 con 0x5613b8f2ed30 2024-05-27T07:56:09.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.689+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5613b8e469e0 con 0x5613b8da9e40 2024-05-27T07:56:09.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.689+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613b8e4aa20 con 0x5613b8f28db0 2024-05-27T07:56:09.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.689+0000 7f0437400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8f2ed30 0x5613b8f28870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.689+0000 7f0437400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8f2ed30 0x5613b8f28870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37314/0 (socket says 172.21.15.63:37314) 2024-05-27T07:56:09.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.689+0000 7f0437400700 1 -- 172.21.15.63:0/1344409226 learned_addr learned my addr 172.21.15.63:0/1344409226 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:09.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.689+0000 7f0436a00700 1 --2- 172.21.15.63:0/1344409226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8da9e40 0x5613b8f2e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.689+0000 7f042fe00700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f041c0b2a70 0x7f041c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.690+0000 7f0436a00700 1 --2- 172.21.15.63:0/1344409226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.690+0000 7f0437400700 1 -- 172.21.15.63:0/1344409226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 msgr2=0x5613b8f291c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:09.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.690+0000 7f042fe00700 1 -- 172.21.15.63:0/1489761855 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9571+0+0 (secure 0 0 0) 0x7f04300e26c0 con 0x7f0438110bf0 2024-05-27T07:56:09.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.690+0000 7f0437400700 1 --2- 172.21.15.63:0/1344409226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f291c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.690+0000 7f0437400700 1 -- 172.21.15.63:0/1344409226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8da9e40 msgr2=0x5613b8f2e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:09.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.690+0000 7f0437400700 1 --2- 172.21.15.63:0/1344409226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8da9e40 0x5613b8f2e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.690+0000 7f0437400700 1 -- 172.21.15.63:0/1344409226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b8e0f420 con 0x5613b8f2ed30 2024-05-27T07:56:09.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.690+0000 7f0436a00700 1 --2- 172.21.15.63:0/1344409226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:09.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.690+0000 7f0437e00700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f041c0b2a70 0x7f041c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.690+0000 7f0437400700 1 --2- 172.21.15.63:0/1344409226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8f2ed30 0x5613b8f28870 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f0428006010 tx=0x7f04280096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50cc0ebbc6c03a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:09.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.691+0000 7f042cc00700 1 -- 172.21.15.63:0/1344409226 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428005220 con 0x5613b8f2ed30 2024-05-27T07:56:09.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.691+0000 7f042cc00700 1 -- 172.21.15.63:0/1344409226 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04280037c0 con 0x5613b8f2ed30 2024-05-27T07:56:09.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.691+0000 7f042cc00700 1 -- 172.21.15.63:0/1344409226 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042800bbf0 con 0x5613b8f2ed30 2024-05-27T07:56:09.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.692+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1344409226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8f2ed30 msgr2=0x5613b8f28870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:09.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.692+0000 7f0437e00700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f041c0b2a70 0x7f041c0b4f80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f04300a9390 tx=0x7f043000af70 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:09.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.692+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1344409226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8f2ed30 0x5613b8f28870 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f0428006010 tx=0x7f04280096a0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.692+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1344409226 shutdown_connections 2024-05-27T07:56:09.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.692+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1344409226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.692+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1344409226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8da9e40 0x5613b8f2e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.692+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1344409226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8f2ed30 0x5613b8f28870 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.692+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1344409226 >> 172.21.15.63:0/1344409226 conn(0x5613b8e09fc0 msgr2=0x5613b8f238d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:09.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.693+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1344409226 shutdown_connections 2024-05-27T07:56:09.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.693+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1344409226 wait complete. 2024-05-27T07:56:09.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.693+0000 7f044ceb6b80 1 -- start start 2024-05-27T07:56:09.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.694+0000 7f044ceb6b80 1 Processor -- start 2024-05-27T07:56:09.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.694+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.694+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f32a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.694+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f2ed30 0x5613b8f32f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.695+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5613b8d4cfa0 con 0x5613b8da9e40 2024-05-27T07:56:09.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.695+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5613b8de7760 con 0x5613b8f2ed30 2024-05-27T07:56:09.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.695+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613b8f357a0 con 0x5613b8f28db0 2024-05-27T07:56:09.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.695+0000 7f044ceb6b80 10 client.0 _ll_register_callbacks cb 0x5613b8df0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:09.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.695+0000 7f0436a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.695+0000 7f0437400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f32a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.695+0000 7f0436a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f324d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37326/0 (socket says 172.21.15.63:37326) 2024-05-27T07:56:09.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.695+0000 7f0436a00700 1 -- 172.21.15.63:0/1852193054 learned_addr learned my addr 172.21.15.63:0/1852193054 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:09.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.696+0000 7f0436a00700 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f2ed30 0x5613b8f32f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.696+0000 7f0436a00700 1 -- 172.21.15.63:0/1852193054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 msgr2=0x5613b8f32a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:09.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.696+0000 7f0436a00700 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f32a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.696+0000 7f0436a00700 1 -- 172.21.15.63:0/1852193054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f2ed30 msgr2=0x5613b8f32f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:09.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.696+0000 7f0436a00700 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f2ed30 0x5613b8f32f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.696+0000 7f0436a00700 1 -- 172.21.15.63:0/1852193054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b8d968d0 con 0x5613b8da9e40 2024-05-27T07:56:09.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.697+0000 7f0436a00700 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f324d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f0438077de0 tx=0x7f0424014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a1cb5a95d7e6c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:09.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.697+0000 7f0421600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:09.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.697+0000 7f0421600700 1 -- 172.21.15.63:0/1852193054 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424004260 con 0x5613b8da9e40 2024-05-27T07:56:09.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.697+0000 7f0421600700 1 -- 172.21.15.63:0/1852193054 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9571+0+0 (secure 0 0 0) 0x7f042401b040 con 0x5613b8da9e40 2024-05-27T07:56:09.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.697+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1852193054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04280037c0 con 0x5613b8da9e40 2024-05-27T07:56:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.697+0000 7f0421600700 1 -- 172.21.15.63:0/1852193054 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0424004400 con 0x5613b8da9e40 2024-05-27T07:56:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.697+0000 7f044ceb6b80 10 client.19782 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.697+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1852193054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5613b8e17060 con 0x5613b8da9e40 2024-05-27T07:56:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.698+0000 7f0421600700 1 -- 172.21.15.63:0/1852193054 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04280037c0 con 0x5613b8da9e40 2024-05-27T07:56:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.698+0000 7f0421600700 1 -- 172.21.15.63:0/1852193054 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0424003780 con 0x5613b8da9e40 2024-05-27T07:56:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.698+0000 7f044ceb6b80 10 client.19782 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:09.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.698+0000 7f044ceb6b80 10 client.19782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.698+0000 7f044ceb6b80 20 client.19782 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.698+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613b8f426e0 0x5613b8f44b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.698+0000 7f044ceb6b80 4 client.19782 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.698+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1852193054 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613b8e58d90 con 0x5613b8f426e0 2024-05-27T07:56:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.699+0000 7f0437400700 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613b8f426e0 0x5613b8f44b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.700+0000 7f0437400700 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613b8f426e0 0x5613b8f44b90 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e835472862da332 server_cookie=af94e033efc62c53 in_seq=0 out_seq=0 2024-05-27T07:56:09.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.700+0000 7f0421600700 10 client.19782 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:09.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f0421600700 1 -- 172.21.15.63:0/1852193054 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5613b8e58d90 con 0x5613b8f426e0 2024-05-27T07:56:09.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f0421600700 10 client.19782 handle_command_reply: tid=0 2024-05-27T07:56:09.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f044ceb6b80 1 client.19782 shutdown 2024-05-27T07:56:09.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1852193054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 msgr2=0x5613b8f324d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f324d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f0438077de0 tx=0x7f0424014040 comp rx=0 tx=0).stop 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1852193054 shutdown_connections 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613b8f426e0 0x5613b8f44b90 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f2ed30 0x5613b8f32f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f324d0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1852193054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f32a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1852193054 >> 172.21.15.63:0/1852193054 conn(0x5613b8e09fc0 msgr2=0x5613b8f23580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.702+0000 7f0421600700 0 client.19782 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.703+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1852193054 shutdown_connections 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.703+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1852193054 wait complete. 2024-05-27T07:56:09.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.703+0000 7f044ceb6b80 20 client.19782 trim_cache size 0 max 16384 2024-05-27T07:56:09.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.729+0000 7f044ceb6b80 1 Processor -- start 2024-05-27T07:56:09.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.729+0000 7f044ceb6b80 1 -- start start 2024-05-27T07:56:09.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.729+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f28870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.729+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f28db0 0x5613b8f34970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.730+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f2ed30 0x5613b8f3e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.730+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0418011760 con 0x5613b8da9e40 2024-05-27T07:56:09.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.730+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f041800eff0 con 0x5613b8f28db0 2024-05-27T07:56:09.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.730+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f041800ce60 con 0x5613b8f2ed30 2024-05-27T07:56:09.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.730+0000 7f0436a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f28870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.730+0000 7f0436a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f28870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37336/0 (socket says 172.21.15.63:37336) 2024-05-27T07:56:09.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.730+0000 7f0436a00700 1 -- 172.21.15.63:0/1672931340 learned_addr learned my addr 172.21.15.63:0/1672931340 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:09.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.730+0000 7f0437400700 1 --2- 172.21.15.63:0/1672931340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f28db0 0x5613b8f34970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.730+0000 7f0436a00700 1 --2- 172.21.15.63:0/1672931340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f2ed30 0x5613b8f3e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.731+0000 7f0436a00700 1 -- 172.21.15.63:0/1672931340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f2ed30 msgr2=0x5613b8f3e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:09.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.731+0000 7f0436a00700 1 --2- 172.21.15.63:0/1672931340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f2ed30 0x5613b8f3e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.731+0000 7f0436a00700 1 -- 172.21.15.63:0/1672931340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f28db0 msgr2=0x5613b8f34970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:09.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.731+0000 7f0436a00700 1 --2- 172.21.15.63:0/1672931340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f28db0 0x5613b8f34970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.731+0000 7f0436a00700 1 -- 172.21.15.63:0/1672931340 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b8f332c0 con 0x5613b8da9e40 2024-05-27T07:56:09.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.731+0000 7f0436a00700 1 --2- 172.21.15.63:0/1672931340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f28870 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f042400c010 tx=0x7f042400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fc2e9704bf8796 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:09.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.731+0000 7f0423e00700 1 -- 172.21.15.63:0/1672931340 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424012110 con 0x5613b8da9e40 2024-05-27T07:56:09.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.732+0000 7f0423e00700 1 -- 172.21.15.63:0/1672931340 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0424005030 con 0x5613b8da9e40 2024-05-27T07:56:09.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.732+0000 7f0423e00700 1 -- 172.21.15.63:0/1672931340 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04240108b0 con 0x5613b8da9e40 2024-05-27T07:56:09.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.733+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1672931340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 msgr2=0x5613b8f28870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:09.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.733+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1672931340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f28870 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f042400c010 tx=0x7f042400b630 comp rx=0 tx=0).stop 2024-05-27T07:56:09.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.733+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1672931340 shutdown_connections 2024-05-27T07:56:09.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.733+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1672931340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f2ed30 0x5613b8f3e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.733+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1672931340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f28870 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.733+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/1672931340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f28db0 0x5613b8f34970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.733+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1672931340 >> 172.21.15.63:0/1672931340 conn(0x5613b8e09fc0 msgr2=0x5613b8f2c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:09.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.733+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1672931340 shutdown_connections 2024-05-27T07:56:09.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.733+0000 7f044ceb6b80 1 -- 172.21.15.63:0/1672931340 wait complete. 2024-05-27T07:56:09.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.733+0000 7f044ceb6b80 1 -- start start 2024-05-27T07:56:09.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.734+0000 7f044ceb6b80 1 Processor -- start 2024-05-27T07:56:09.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.734+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f3ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.734+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f3b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.734+0000 7f044ceb6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f2ed30 0x5613b8f49540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.734+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f041800d4b0 con 0x5613b8da9e40 2024-05-27T07:56:09.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.734+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f041800e920 con 0x5613b8f2ed30 2024-05-27T07:56:09.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.734+0000 7f044ceb6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f041800db80 con 0x5613b8f28db0 2024-05-27T07:56:09.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.734+0000 7f044ceb6b80 10 client.0 _ll_register_callbacks cb 0x5613b8df0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:09.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.735+0000 7f0436a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f3ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.735+0000 7f0436a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f3ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37346/0 (socket says 172.21.15.63:37346) 2024-05-27T07:56:09.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.735+0000 7f0436a00700 1 -- 172.21.15.63:0/3944474024 learned_addr learned my addr 172.21.15.63:0/3944474024 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:09.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.735+0000 7f0437400700 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f3b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.735+0000 7f0436a00700 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f2ed30 0x5613b8f49540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.736+0000 7f0436a00700 1 -- 172.21.15.63:0/3944474024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 msgr2=0x5613b8f3b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:09.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.736+0000 7f0436a00700 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f3b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.736+0000 7f0436a00700 1 -- 172.21.15.63:0/3944474024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f2ed30 msgr2=0x5613b8f49540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:09.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.736+0000 7f0436a00700 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f2ed30 0x5613b8f49540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.736+0000 7f0436a00700 1 -- 172.21.15.63:0/3944474024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b8f49b30 con 0x5613b8da9e40 2024-05-27T07:56:09.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.736+0000 7f0437400700 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f3b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:56:09.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.736+0000 7f0436a00700 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f3ac90 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0424012a70 tx=0x7f042400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d77943315a6966 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:09.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.736+0000 7f0421600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:09.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.737+0000 7f0421600700 1 -- 172.21.15.63:0/3944474024 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424036040 con 0x5613b8da9e40 2024-05-27T07:56:09.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.737+0000 7f0421600700 1 -- 172.21.15.63:0/3944474024 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9571+0+0 (secure 0 0 0) 0x7f04240051e0 con 0x5613b8da9e40 2024-05-27T07:56:09.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.737+0000 7f044ceb6b80 1 -- 172.21.15.63:0/3944474024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0424005030 con 0x5613b8da9e40 2024-05-27T07:56:09.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.737+0000 7f0421600700 1 -- 172.21.15.63:0/3944474024 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0424012240 con 0x5613b8da9e40 2024-05-27T07:56:09.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.737+0000 7f044ceb6b80 10 client.19788 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:09.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.737+0000 7f044ceb6b80 1 -- 172.21.15.63:0/3944474024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5613b8facab0 con 0x5613b8da9e40 2024-05-27T07:56:09.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.738+0000 7f0421600700 1 -- 172.21.15.63:0/3944474024 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0424005030 con 0x5613b8da9e40 2024-05-27T07:56:09.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.738+0000 7f0421600700 1 -- 172.21.15.63:0/3944474024 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f042403c020 con 0x5613b8da9e40 2024-05-27T07:56:09.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.738+0000 7f044ceb6b80 10 client.19788 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:09.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.738+0000 7f044ceb6b80 10 client.19788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:09.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.738+0000 7f044ceb6b80 20 client.19788 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:09.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.738+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613b8facf70 0x5613b8faf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:09.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.738+0000 7f044ceb6b80 4 client.19788 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:56:09.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.738+0000 7f044ceb6b80 1 -- 172.21.15.63:0/3944474024 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5613b8e52570 con 0x5613b8facf70 2024-05-27T07:56:09.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.739+0000 7f0437400700 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613b8facf70 0x5613b8faf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:09.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.740+0000 7f0437400700 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613b8facf70 0x5613b8faf390 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=643d040bb2b6bce3 server_cookie=288b4b93fff141ab in_seq=0 out_seq=0 2024-05-27T07:56:09.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.740+0000 7f0421600700 10 client.19788 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:09.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.740+0000 7f0421600700 1 -- 172.21.15.63:0/3944474024 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5613b8e52570 con 0x5613b8facf70 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.740+0000 7f0421600700 10 client.19788 handle_command_reply: tid=0 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.740+0000 7f044ceb6b80 1 client.19788 shutdown 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.740+0000 7f044ceb6b80 1 -- 172.21.15.63:0/3944474024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 msgr2=0x5613b8f3ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.740+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f3ac90 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0424012a70 tx=0x7f042400b410 comp rx=0 tx=0).stop 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.741+0000 7f044ceb6b80 1 -- 172.21.15.63:0/3944474024 shutdown_connections 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.741+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5613b8facf70 0x5613b8faf390 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.741+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5613b8f2ed30 0x5613b8f49540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.741+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5613b8da9e40 0x5613b8f3ac90 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.741+0000 7f044ceb6b80 1 --2- 172.21.15.63:0/3944474024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613b8f28db0 0x5613b8f3b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.741+0000 7f044ceb6b80 1 -- 172.21.15.63:0/3944474024 >> 172.21.15.63:0/3944474024 conn(0x5613b8e09fc0 msgr2=0x5613b8f322b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.741+0000 7f0421600700 0 client.19788 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:09.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.741+0000 7f044ceb6b80 1 -- 172.21.15.63:0/3944474024 shutdown_connections 2024-05-27T07:56:09.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.741+0000 7f044ceb6b80 1 -- 172.21.15.63:0/3944474024 wait complete. 2024-05-27T07:56:09.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.741+0000 7f044ceb6b80 20 client.19788 trim_cache size 0 max 16384 2024-05-27T07:56:09.743 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:56:09.743 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:56:09.744 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:56:09.744 INFO:teuthology.orchestra.run.smithi063.stdout: "404d86f3-5893-4f14-9dd6-0be1ba5207e2": { 2024-05-27T07:56:09.744 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:56:09.744 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "404d86f3-5893-4f14-9dd6-0be1ba5207e2", 2024-05-27T07:56:09.744 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:56:09.744 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:56:09.744 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:56:09.744 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.742+0000 7f043e400700 1 -- 172.21.15.63:0/1489761855 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f041c0b2a70 msgr2=0x7f041c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.742+0000 7f043e400700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f041c0b2a70 0x7f041c0b4f80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f04300a9390 tx=0x7f043000af70 comp rx=0 tx=0).stop 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.742+0000 7f043e400700 1 -- 172.21.15.63:0/1489761855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0438110bf0 msgr2=0x7f04381a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.742+0000 7f043e400700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0438110bf0 0x7f04381a1810 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f0430007d40 tx=0x7f043000a340 comp rx=0 tx=0).stop 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.742+0000 7f043e400700 1 -- 172.21.15.63:0/1489761855 shutdown_connections 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.742+0000 7f043e400700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f041c0b2a70 0x7f041c0b4f80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.742+0000 7f043e400700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0438110bf0 0x7f04381a1810 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.742+0000 7f043e400700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0438075af0 0x7f043819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.742+0000 7f043e400700 1 --2- 172.21.15.63:0/1489761855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0438076440 0x7f043819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.742+0000 7f043e400700 1 -- 172.21.15.63:0/1489761855 >> 172.21.15.63:0/1489761855 conn(0x7f0438100030 msgr2=0x7f0438100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.743+0000 7f043e400700 1 -- 172.21.15.63:0/1489761855 shutdown_connections 2024-05-27T07:56:09.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:09.743+0000 7f043e400700 1 -- 172.21.15.63:0/1489761855 wait complete. 2024-05-27T07:56:09.762 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:09 smithi086 ceph-mon[34687]: pgmap v713: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:56:09.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:09 smithi149 ceph-mon[35408]: pgmap v713: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:56:10.673 INFO:tasks.cephfs.filesystem:scrub status for tag:404d86f3-5893-4f14-9dd6-0be1ba5207e2 - {'path': '/', 'tag': '404d86f3-5893-4f14-9dd6-0be1ba5207e2', 'options': 'recursive,force'} 2024-05-27T07:56:11.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:11 smithi086 ceph-mon[34687]: pgmap v714: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:56:11.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:11 smithi149 ceph-mon[35408]: pgmap v714: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:56:11.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:11 smithi063 ceph-mon[31613]: pgmap v714: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.5 MiB/s wr, 68 op/s 2024-05-27T07:56:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.739+0000 7f337fe00640 1 -- 172.21.15.63:0/4098565826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f338010e000 msgr2=0x7f33801104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.739+0000 7f337fe00640 1 --2- 172.21.15.63:0/4098565826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f338010e000 0x7f33801104f0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f3378009ca0 tx=0x7f337802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.739+0000 7f337fe00640 1 -- 172.21.15.63:0/4098565826 shutdown_connections 2024-05-27T07:56:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.740+0000 7f337fe00640 1 --2- 172.21.15.63:0/4098565826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f338010e000 0x7f33801104f0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.740+0000 7f337fe00640 1 --2- 172.21.15.63:0/4098565826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338010b610 0x7f338010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.740+0000 7f337fe00640 1 --2- 172.21.15.63:0/4098565826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3380104bb0 0x7f3380104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.740+0000 7f337fe00640 1 -- 172.21.15.63:0/4098565826 >> 172.21.15.63:0/4098565826 conn(0x7f33800fe210 msgr2=0x7f3380100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.740+0000 7f337fe00640 1 -- 172.21.15.63:0/4098565826 shutdown_connections 2024-05-27T07:56:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.740+0000 7f337fe00640 1 -- 172.21.15.63:0/4098565826 wait complete. 2024-05-27T07:56:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.741+0000 7f337fe00640 1 Processor -- start 2024-05-27T07:56:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.741+0000 7f337fe00640 1 -- start start 2024-05-27T07:56:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.742+0000 7f337fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3380104bb0 0x7f338019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.742+0000 7f337fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338010b610 0x7f338019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.742+0000 7f337fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f338010e000 0x7f338019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.742+0000 7f337fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3380113b60 con 0x7f3380104bb0 2024-05-27T07:56:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.742+0000 7f337fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33801139e0 con 0x7f338010b610 2024-05-27T07:56:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.742+0000 7f337fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3380113ce0 con 0x7f338010e000 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.742+0000 7f337ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3380104bb0 0x7f338019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.742+0000 7f337ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3380104bb0 0x7f338019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37362/0 (socket says 172.21.15.63:37362) 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.742+0000 7f337ea00640 1 -- 172.21.15.63:0/3156502904 learned_addr learned my addr 172.21.15.63:0/3156502904 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.742+0000 7f337e000640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338010b610 0x7f338019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f337f400640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f338010e000 0x7f338019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f337ea00640 1 -- 172.21.15.63:0/3156502904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f338010e000 msgr2=0x7f338019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f337ea00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f338010e000 0x7f338019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f337ea00640 1 -- 172.21.15.63:0/3156502904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338010b610 msgr2=0x7f338019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f337ea00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338010b610 0x7f338019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f337ea00640 1 -- 172.21.15.63:0/3156502904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338019f390 con 0x7f3380104bb0 2024-05-27T07:56:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f337f400640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f338010e000 0x7f338019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f337ea00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3380104bb0 0x7f338019a580 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f337000d950 tx=0x7f337000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f3377400640 1 -- 172.21.15.63:0/3156502904 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370017030 con 0x7f3380104bb0 2024-05-27T07:56:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f3377400640 1 -- 172.21.15.63:0/3156502904 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3370004020 con 0x7f3380104bb0 2024-05-27T07:56:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.743+0000 7f3377400640 1 -- 172.21.15.63:0/3156502904 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700055d0 con 0x7f3380104bb0 2024-05-27T07:56:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.744+0000 7f337fe00640 1 -- 172.21.15.63:0/3156502904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338019f680 con 0x7f3380104bb0 2024-05-27T07:56:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.744+0000 7f337fe00640 1 -- 172.21.15.63:0/3156502904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338019fb80 con 0x7f3380104bb0 2024-05-27T07:56:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.747+0000 7f338c795740 1 Processor -- start 2024-05-27T07:56:12.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.748+0000 7f338c795740 1 -- start start 2024-05-27T07:56:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.748+0000 7f338c795740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f4c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.748+0000 7f338c795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b812f0 0x5586a5f4ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.748+0000 7f338c795740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b894d0 0x5586a5f4d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.748+0000 7f338c795740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5586a5ebdb90 con 0x5586a5baa1b0 2024-05-27T07:56:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.748+0000 7f338c795740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586a5ead730 con 0x5586a5b812f0 2024-05-27T07:56:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.748+0000 7f338c795740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5586a5eb1770 con 0x5586a5b894d0 2024-05-27T07:56:12.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.748+0000 7f3377400640 1 -- 172.21.15.63:0/3156502904 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3370009a30 con 0x7f3380104bb0 2024-05-27T07:56:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.749+0000 7f337f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f4c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.749+0000 7f337f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f4c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37364/0 (socket says 172.21.15.63:37364) 2024-05-27T07:56:12.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.749+0000 7f337f400640 1 -- 172.21.15.63:0/294080524 learned_addr learned my addr 172.21.15.63:0/294080524 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.749+0000 7f337e000640 1 --2- 172.21.15.63:0/294080524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b894d0 0x5586a5f4d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.749+0000 7f337f400640 1 -- 172.21.15.63:0/294080524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b894d0 msgr2=0x5586a5f4d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.749+0000 7f337f400640 1 --2- 172.21.15.63:0/294080524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b894d0 0x5586a5f4d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.749+0000 7f337f400640 1 -- 172.21.15.63:0/294080524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b812f0 msgr2=0x5586a5f4ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.749+0000 7f337f400640 1 --2- 172.21.15.63:0/294080524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b812f0 0x5586a5f4ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.749+0000 7f337f400640 1 -- 172.21.15.63:0/294080524 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586a5e468b0 con 0x5586a5baa1b0 2024-05-27T07:56:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.750+0000 7f337f400640 1 --2- 172.21.15.63:0/294080524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f4c930 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f3378031530 tx=0x7f3378031560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b82031bfb5ac0b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.750+0000 7f336be00640 1 -- 172.21.15.63:0/294080524 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378031b80 con 0x5586a5baa1b0 2024-05-27T07:56:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.750+0000 7f336be00640 1 -- 172.21.15.63:0/294080524 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f337803f070 con 0x5586a5baa1b0 2024-05-27T07:56:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.750+0000 7f336be00640 1 -- 172.21.15.63:0/294080524 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378004600 con 0x5586a5baa1b0 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.751+0000 7f3377400640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33640b29d0 0x7f33640b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.751+0000 7f3377400640 1 -- 172.21.15.63:0/3156502904 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9571+0+0 (secure 0 0 0) 0x7f33700e2500 con 0x7f3380104bb0 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.751+0000 7f337ea00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33640b29d0 0x7f33640b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.751+0000 7f338c795740 1 -- 172.21.15.63:0/294080524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 msgr2=0x5586a5f4c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.751+0000 7f338c795740 1 --2- 172.21.15.63:0/294080524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f4c930 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f3378031530 tx=0x7f3378031560 comp rx=0 tx=0).stop 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.751+0000 7f338c795740 1 -- 172.21.15.63:0/294080524 shutdown_connections 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.751+0000 7f338c795740 1 --2- 172.21.15.63:0/294080524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b894d0 0x5586a5f4d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.751+0000 7f338c795740 1 --2- 172.21.15.63:0/294080524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b812f0 0x5586a5f4ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.751+0000 7f338c795740 1 --2- 172.21.15.63:0/294080524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f4c930 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.751+0000 7f338c795740 1 -- 172.21.15.63:0/294080524 >> 172.21.15.63:0/294080524 conn(0x5586a5b9e300 msgr2=0x5586a5f49210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.752+0000 7f338c795740 1 -- 172.21.15.63:0/294080524 shutdown_connections 2024-05-27T07:56:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.752+0000 7f338c795740 1 -- 172.21.15.63:0/294080524 wait complete. 2024-05-27T07:56:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.752+0000 7f337ea00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33640b29d0 0x7f33640b4ee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5586a5f4a2c0 tx=0x7f3370005770 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.752+0000 7f338c795740 1 -- start start 2024-05-27T07:56:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.753+0000 7f338c795740 1 Processor -- start 2024-05-27T07:56:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.753+0000 7f338c795740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b812f0 0x5586a5f5b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.753+0000 7f338c795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b894d0 0x5586a5f5b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.753+0000 7f338c795740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f67320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.753+0000 7f338c795740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5586a5b93790 con 0x5586a5baa1b0 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.753+0000 7f338c795740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586a5e55bc0 con 0x5586a5b894d0 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.753+0000 7f338c795740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5586a5f5dff0 con 0x5586a5b812f0 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.753+0000 7f338c795740 10 client.0 _ll_register_callbacks cb 0x5586a5e33400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.754+0000 7f337f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f67320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.754+0000 7f337f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f67320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37378/0 (socket says 172.21.15.63:37378) 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.754+0000 7f337f400640 1 -- 172.21.15.63:0/1094850930 learned_addr learned my addr 172.21.15.63:0/1094850930 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.754+0000 7f337e000640 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b894d0 0x5586a5f5b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.754+0000 7f337f400640 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b812f0 0x5586a5f5b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.754+0000 7f337f400640 1 -- 172.21.15.63:0/1094850930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b812f0 msgr2=0x5586a5f5b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.754+0000 7f337f400640 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b812f0 0x5586a5f5b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.754+0000 7f337f400640 1 -- 172.21.15.63:0/1094850930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b894d0 msgr2=0x5586a5f5b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.754+0000 7f337f400640 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b894d0 0x5586a5f5b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.754+0000 7f337f400640 1 -- 172.21.15.63:0/1094850930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586a5bc2210 con 0x5586a5baa1b0 2024-05-27T07:56:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.755+0000 7f337f400640 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f67320 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f3378009960 tx=0x7f3378009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c7c0a66156527b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.755+0000 7f3368c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.755+0000 7f3368c00640 1 -- 172.21.15.63:0/1094850930 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378057060 con 0x5586a5baa1b0 2024-05-27T07:56:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.755+0000 7f338c795740 1 -- 172.21.15.63:0/1094850930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337803f070 con 0x5586a5baa1b0 2024-05-27T07:56:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.755+0000 7f3368c00640 1 -- 172.21.15.63:0/1094850930 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9571+0+0 (secure 0 0 0) 0x7f3378003970 con 0x5586a5baa1b0 2024-05-27T07:56:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.755+0000 7f3368c00640 1 -- 172.21.15.63:0/1094850930 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3378003b70 con 0x5586a5baa1b0 2024-05-27T07:56:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.755+0000 7f338c795740 10 client.19800 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.755+0000 7f338c795740 1 -- 172.21.15.63:0/1094850930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5586a5c8e620 con 0x5586a5baa1b0 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.756+0000 7f3368c00640 1 -- 172.21.15.63:0/1094850930 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f337803f070 con 0x5586a5baa1b0 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.756+0000 7f3368c00640 1 -- 172.21.15.63:0/1094850930 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3378030a80 con 0x5586a5baa1b0 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.756+0000 7f338c795740 10 client.19800 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.756+0000 7f338c795740 10 client.19800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.756+0000 7f338c795740 20 client.19800 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.756+0000 7f338c795740 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586a5b68d80 0x5586a5f6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.756+0000 7f338c795740 4 client.19800 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.756+0000 7f338c795740 1 -- 172.21.15.63:0/1094850930 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586a5ebfae0 con 0x5586a5b68d80 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.757+0000 7f337e000640 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586a5b68d80 0x5586a5f6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.758+0000 7f337e000640 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586a5b68d80 0x5586a5f6d140 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b026f4ceda8e034 server_cookie=b345b5ed7e41e43b in_seq=0 out_seq=0 2024-05-27T07:56:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.758+0000 7f3368c00640 10 client.19800 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.760+0000 7f3368c00640 1 -- 172.21.15.63:0/1094850930 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5586a5ebfae0 con 0x5586a5b68d80 2024-05-27T07:56:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.760+0000 7f3368c00640 10 client.19800 handle_command_reply: tid=0 2024-05-27T07:56:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.760+0000 7f338c795740 1 client.19800 shutdown 2024-05-27T07:56:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.760+0000 7f338c795740 1 -- 172.21.15.63:0/1094850930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 msgr2=0x5586a5f67320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.760+0000 7f338c795740 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f67320 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f3378009960 tx=0x7f3378009560 comp rx=0 tx=0).stop 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.761+0000 7f338c795740 1 -- 172.21.15.63:0/1094850930 shutdown_connections 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.761+0000 7f338c795740 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586a5b68d80 0x5586a5f6d140 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.761+0000 7f338c795740 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5baa1b0 0x5586a5f67320 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.761+0000 7f338c795740 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b894d0 0x5586a5f5b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.761+0000 7f338c795740 1 --2- 172.21.15.63:0/1094850930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b812f0 0x5586a5f5b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.761+0000 7f3368c00640 0 client.19800 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.761+0000 7f338c795740 1 -- 172.21.15.63:0/1094850930 >> 172.21.15.63:0/1094850930 conn(0x5586a5b9e300 msgr2=0x5586a5f4f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.761+0000 7f338c795740 1 -- 172.21.15.63:0/1094850930 shutdown_connections 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.761+0000 7f338c795740 1 -- 172.21.15.63:0/1094850930 wait complete. 2024-05-27T07:56:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.761+0000 7f338c795740 20 client.19800 trim_cache size 0 max 16384 2024-05-27T07:56:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.784+0000 7f338c795740 1 Processor -- start 2024-05-27T07:56:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.784+0000 7f338c795740 1 -- start start 2024-05-27T07:56:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f338c795740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b9e300 0x5586a5f56890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f338c795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b812f0 0x5586a5f5b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f338c795740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f4e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f338c795740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33801b1850 con 0x5586a5b894d0 2024-05-27T07:56:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f338c795740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33801af120 con 0x5586a5b812f0 2024-05-27T07:56:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f338c795740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f33801acf90 con 0x5586a5b9e300 2024-05-27T07:56:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f337f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f4e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f337f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f4e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37388/0 (socket says 172.21.15.63:37388) 2024-05-27T07:56:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f337f400640 1 -- 172.21.15.63:0/2543801415 learned_addr learned my addr 172.21.15.63:0/2543801415 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f337e000640 1 --2- 172.21.15.63:0/2543801415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b812f0 0x5586a5f5b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.785+0000 7f337f400640 1 --2- 172.21.15.63:0/2543801415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b9e300 0x5586a5f56890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.786+0000 7f337f400640 1 -- 172.21.15.63:0/2543801415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b9e300 msgr2=0x5586a5f56890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.786+0000 7f337f400640 1 --2- 172.21.15.63:0/2543801415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b9e300 0x5586a5f56890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.786+0000 7f337f400640 1 -- 172.21.15.63:0/2543801415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b812f0 msgr2=0x5586a5f5b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.786+0000 7f337f400640 1 --2- 172.21.15.63:0/2543801415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b812f0 0x5586a5f5b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.786+0000 7f337f400640 1 -- 172.21.15.63:0/2543801415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586a5f49bf0 con 0x5586a5b894d0 2024-05-27T07:56:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.786+0000 7f337e000640 1 --2- 172.21.15.63:0/2543801415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b812f0 0x5586a5f5b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.786+0000 7f337f400640 1 --2- 172.21.15.63:0/2543801415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f4e5f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f3378031ab0 tx=0x7f33780098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ff65972cd4a90 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.786+0000 7f336b400640 1 -- 172.21.15.63:0/2543801415 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378057040 con 0x5586a5b894d0 2024-05-27T07:56:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.786+0000 7f336b400640 1 -- 172.21.15.63:0/2543801415 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f337803f070 con 0x5586a5b894d0 2024-05-27T07:56:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.786+0000 7f336b400640 1 -- 172.21.15.63:0/2543801415 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33780446a0 con 0x5586a5b894d0 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.787+0000 7f338c795740 1 -- 172.21.15.63:0/2543801415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 msgr2=0x5586a5f4e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.787+0000 7f338c795740 1 --2- 172.21.15.63:0/2543801415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f4e5f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f3378031ab0 tx=0x7f33780098a0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.788+0000 7f338c795740 1 -- 172.21.15.63:0/2543801415 shutdown_connections 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.788+0000 7f338c795740 1 --2- 172.21.15.63:0/2543801415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f4e5f0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.788+0000 7f338c795740 1 --2- 172.21.15.63:0/2543801415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b812f0 0x5586a5f5b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.788+0000 7f338c795740 1 --2- 172.21.15.63:0/2543801415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b9e300 0x5586a5f56890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.788+0000 7f338c795740 1 -- 172.21.15.63:0/2543801415 >> 172.21.15.63:0/2543801415 conn(0x5586a5f4d430 msgr2=0x5586a5f49050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.788+0000 7f338c795740 1 -- 172.21.15.63:0/2543801415 shutdown_connections 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.788+0000 7f338c795740 1 -- 172.21.15.63:0/2543801415 wait complete. 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.788+0000 7f338c795740 1 -- start start 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.788+0000 7f338c795740 1 Processor -- start 2024-05-27T07:56:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.789+0000 7f338c795740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b812f0 0x5586a5f703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.789+0000 7f338c795740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f70900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.789+0000 7f338c795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b9e300 0x5586a5f6b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.789+0000 7f338c795740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33801ad5e0 con 0x5586a5b894d0 2024-05-27T07:56:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.789+0000 7f338c795740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33801aea50 con 0x5586a5b9e300 2024-05-27T07:56:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.789+0000 7f338c795740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f33801adcb0 con 0x5586a5b812f0 2024-05-27T07:56:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.789+0000 7f338c795740 10 client.0 _ll_register_callbacks cb 0x7f3378031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.789+0000 7f337e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f70900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.789+0000 7f337e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f70900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37392/0 (socket says 172.21.15.63:37392) 2024-05-27T07:56:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.789+0000 7f337e000640 1 -- 172.21.15.63:0/1008453781 learned_addr learned my addr 172.21.15.63:0/1008453781 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f337f400640 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b812f0 0x5586a5f703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f337f400640 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b9e300 0x5586a5f6b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f337e000640 1 -- 172.21.15.63:0/1008453781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b812f0 msgr2=0x5586a5f703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f337e000640 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b812f0 0x5586a5f703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f337e000640 1 -- 172.21.15.63:0/1008453781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b9e300 msgr2=0x5586a5f6b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f337e000640 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b9e300 0x5586a5f6b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f337e000640 1 -- 172.21.15.63:0/1008453781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586a5f6bec0 con 0x5586a5b894d0 2024-05-27T07:56:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f337f400640 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b9e300 0x5586a5f6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f337e000640 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f70900 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f336c00bc20 tx=0x7f336c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a320fe5f09b6d8fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f3368c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.790+0000 7f3368c00640 1 -- 172.21.15.63:0/1008453781 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c01e390 con 0x5586a5b894d0 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.791+0000 7f3368c00640 1 -- 172.21.15.63:0/1008453781 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9571+0+0 (secure 0 0 0) 0x7f336c01e530 con 0x5586a5b894d0 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.791+0000 7f338c795740 1 -- 172.21.15.63:0/1008453781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337803f070 con 0x5586a5b894d0 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.791+0000 7f3368c00640 1 -- 172.21.15.63:0/1008453781 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f336c02f050 con 0x5586a5b894d0 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.791+0000 7f338c795740 10 client.19806 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.791+0000 7f338c795740 1 -- 172.21.15.63:0/1008453781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5586a5fd5010 con 0x5586a5b894d0 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.791+0000 7f3368c00640 1 -- 172.21.15.63:0/1008453781 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f337803f070 con 0x5586a5b894d0 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.792+0000 7f3368c00640 1 -- 172.21.15.63:0/1008453781 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f336c012aa0 con 0x5586a5b894d0 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.792+0000 7f338c795740 10 client.19806 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.792+0000 7f338c795740 10 client.19806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.792+0000 7f338c795740 20 client.19806 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.792+0000 7f338c795740 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586a5baa1b0 0x5586a5fd7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.792+0000 7f338c795740 4 client.19806 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:56:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.792+0000 7f338c795740 1 -- 172.21.15.63:0/1008453781 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5586a5eb92c0 con 0x5586a5baa1b0 2024-05-27T07:56:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.792+0000 7f337f400640 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586a5baa1b0 0x5586a5fd7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.793+0000 7f337f400640 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586a5baa1b0 0x5586a5fd7800 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=400368d93fa4a1fc server_cookie=736d83e7e82997b5 in_seq=0 out_seq=0 2024-05-27T07:56:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.793+0000 7f3368c00640 10 client.19806 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.834+0000 7f3368c00640 1 -- 172.21.15.63:0/1008453781 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5586a5eb92c0 con 0x5586a5baa1b0 2024-05-27T07:56:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.834+0000 7f3368c00640 10 client.19806 handle_command_reply: tid=0 2024-05-27T07:56:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.834+0000 7f338c795740 1 client.19806 shutdown 2024-05-27T07:56:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f338c795740 1 -- 172.21.15.63:0/1008453781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 msgr2=0x5586a5f70900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f338c795740 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f70900 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f336c00bc20 tx=0x7f336c004410 comp rx=0 tx=0).stop 2024-05-27T07:56:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f338c795740 1 -- 172.21.15.63:0/1008453781 shutdown_connections 2024-05-27T07:56:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f338c795740 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5586a5baa1b0 0x5586a5fd7800 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f338c795740 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586a5b9e300 0x5586a5f6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f338c795740 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586a5b894d0 0x5586a5f70900 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f338c795740 1 --2- 172.21.15.63:0/1008453781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5586a5b812f0 0x5586a5f703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f3368c00640 0 client.19806 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f338c795740 1 -- 172.21.15.63:0/1008453781 >> 172.21.15.63:0/1008453781 conn(0x5586a5f4d430 msgr2=0x5586a5f49050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f338c795740 1 -- 172.21.15.63:0/1008453781 shutdown_connections 2024-05-27T07:56:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.835+0000 7f338c795740 1 -- 172.21.15.63:0/1008453781 wait complete. 2024-05-27T07:56:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.836+0000 7f338c795740 20 client.19806 trim_cache size 0 max 16384 2024-05-27T07:56:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:56:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:56:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:56:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 -- 172.21.15.63:0/3156502904 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33640b29d0 msgr2=0x7f33640b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33640b29d0 0x7f33640b4ee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5586a5f4a2c0 tx=0x7f3370005770 comp rx=0 tx=0).stop 2024-05-27T07:56:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 -- 172.21.15.63:0/3156502904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3380104bb0 msgr2=0x7f338019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3380104bb0 0x7f338019a580 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f337000d950 tx=0x7f337000de10 comp rx=0 tx=0).stop 2024-05-27T07:56:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 -- 172.21.15.63:0/3156502904 shutdown_connections 2024-05-27T07:56:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f33640b29d0 0x7f33640b4ee0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f338010e000 0x7f338019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f338010b610 0x7f338019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 --2- 172.21.15.63:0/3156502904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3380104bb0 0x7f338019a580 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 -- 172.21.15.63:0/3156502904 >> 172.21.15.63:0/3156502904 conn(0x7f33800fe210 msgr2=0x7f338010e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 -- 172.21.15.63:0/3156502904 shutdown_connections 2024-05-27T07:56:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:12.837+0000 7f337fe00640 1 -- 172.21.15.63:0/3156502904 wait complete. 2024-05-27T07:56:12.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:56:12.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:56:12.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:12 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:56:12.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:56:12.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:56:12.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:12 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:56:12.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:56:12.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:56:12.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:12 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:56:13.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:13 smithi086 ceph-mon[34687]: pgmap v715: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.9 MiB/s wr, 43 op/s 2024-05-27T07:56:13.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:13 smithi149 ceph-mon[35408]: pgmap v715: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.9 MiB/s wr, 43 op/s 2024-05-27T07:56:13.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:13 smithi063 ceph-mon[31613]: pgmap v715: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.9 MiB/s wr, 43 op/s 2024-05-27T07:56:15.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:15 smithi086 ceph-mon[34687]: pgmap v716: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.9 MiB/s wr, 43 op/s 2024-05-27T07:56:15.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:15 smithi149 ceph-mon[35408]: pgmap v716: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.9 MiB/s wr, 43 op/s 2024-05-27T07:56:15.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:15 smithi063 ceph-mon[31613]: pgmap v716: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.9 MiB/s wr, 43 op/s 2024-05-27T07:56:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:16 smithi086 ceph-mon[34687]: osdmap e127: 12 total, 12 up, 12 in 2024-05-27T07:56:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:16 smithi086 ceph-mon[34687]: pgmap v718: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.2 MiB/s wr, 46 op/s 2024-05-27T07:56:17.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:16 smithi149 ceph-mon[35408]: osdmap e127: 12 total, 12 up, 12 in 2024-05-27T07:56:17.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:16 smithi149 ceph-mon[35408]: pgmap v718: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.2 MiB/s wr, 46 op/s 2024-05-27T07:56:17.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:16 smithi063 ceph-mon[31613]: osdmap e127: 12 total, 12 up, 12 in 2024-05-27T07:56:17.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:16 smithi063 ceph-mon[31613]: pgmap v718: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.2 MiB/s wr, 46 op/s 2024-05-27T07:56:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:19 smithi086 ceph-mon[34687]: pgmap v719: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:19.466 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:19 smithi149 ceph-mon[35408]: pgmap v719: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:19.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:19 smithi063 ceph-mon[31613]: pgmap v719: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:21.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:21 smithi086 ceph-mon[34687]: pgmap v720: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:21.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:21 smithi149 ceph-mon[35408]: pgmap v720: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:21.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:21 smithi063 ceph-mon[31613]: pgmap v720: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:23.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:23 smithi086 ceph-mon[34687]: pgmap v721: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:23.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:23 smithi149 ceph-mon[35408]: pgmap v721: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:23.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:23 smithi063 ceph-mon[31613]: pgmap v721: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:25.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:25 smithi086 ceph-mon[34687]: pgmap v722: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:25.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:25 smithi149 ceph-mon[35408]: pgmap v722: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:25.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:25 smithi063 ceph-mon[31613]: pgmap v722: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 13 op/s 2024-05-27T07:56:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.108+0000 7f113fe00640 1 -- 172.21.15.63:0/1643583895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f114010da00 msgr2=0x7f114010fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.108+0000 7f113fe00640 1 --2- 172.21.15.63:0/1643583895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f114010da00 0x7f114010fe20 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f1128009900 tx=0x7f112802a410 comp rx=0 tx=0).stop 2024-05-27T07:56:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.109+0000 7f113fe00640 1 -- 172.21.15.63:0/1643583895 shutdown_connections 2024-05-27T07:56:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.109+0000 7f113fe00640 1 --2- 172.21.15.63:0/1643583895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140110360 0x7f1140112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.109+0000 7f113fe00640 1 --2- 172.21.15.63:0/1643583895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f114010da00 0x7f114010fe20 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.109+0000 7f113fe00640 1 --2- 172.21.15.63:0/1643583895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11401027c0 0x7f1140102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.109+0000 7f113fe00640 1 -- 172.21.15.63:0/1643583895 >> 172.21.15.63:0/1643583895 conn(0x7f11400fe210 msgr2=0x7f1140100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.109+0000 7f113fe00640 1 -- 172.21.15.63:0/1643583895 shutdown_connections 2024-05-27T07:56:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.109+0000 7f113fe00640 1 -- 172.21.15.63:0/1643583895 wait complete. 2024-05-27T07:56:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.110+0000 7f113fe00640 1 Processor -- start 2024-05-27T07:56:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.110+0000 7f113fe00640 1 -- start start 2024-05-27T07:56:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.111+0000 7f113fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11401027c0 0x7f11401a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.111+0000 7f113fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f114010da00 0x7f11401a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.111+0000 7f113fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140110360 0x7f114019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.111+0000 7f113fe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1140115d70 con 0x7f11401027c0 2024-05-27T07:56:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.111+0000 7f113fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1140115bf0 con 0x7f1140110360 2024-05-27T07:56:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.111+0000 7f113fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1140115ef0 con 0x7f114010da00 2024-05-27T07:56:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.111+0000 7f113ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11401027c0 0x7f11401a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.111+0000 7f113ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11401027c0 0x7f11401a24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54184/0 (socket says 172.21.15.63:54184) 2024-05-27T07:56:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.111+0000 7f113ea00640 1 -- 172.21.15.63:0/3727161371 learned_addr learned my addr 172.21.15.63:0/3727161371 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.112+0000 7f1137e00640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f114010da00 0x7f11401a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.112+0000 7f113f400640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140110360 0x7f114019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.112+0000 7f113ea00640 1 -- 172.21.15.63:0/3727161371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f114010da00 msgr2=0x7f11401a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.112+0000 7f113ea00640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f114010da00 0x7f11401a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.112+0000 7f113ea00640 1 -- 172.21.15.63:0/3727161371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140110360 msgr2=0x7f114019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.112+0000 7f113ea00640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140110360 0x7f114019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.112+0000 7f113ea00640 1 -- 172.21.15.63:0/3727161371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114019cda0 con 0x7f11401027c0 2024-05-27T07:56:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.112+0000 7f113f400640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140110360 0x7f114019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.113+0000 7f113ea00640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11401027c0 0x7f11401a24f0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f113000d950 tx=0x7f113000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.113+0000 7f1137400640 1 -- 172.21.15.63:0/3727161371 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130017070 con 0x7f11401027c0 2024-05-27T07:56:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.113+0000 7f1137400640 1 -- 172.21.15.63:0/3727161371 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1130009c30 con 0x7f11401027c0 2024-05-27T07:56:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.113+0000 7f113fe00640 1 -- 172.21.15.63:0/3727161371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114019d030 con 0x7f11401027c0 2024-05-27T07:56:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.113+0000 7f1137400640 1 -- 172.21.15.63:0/3727161371 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113000f620 con 0x7f11401027c0 2024-05-27T07:56:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.113+0000 7f113fe00640 1 -- 172.21.15.63:0/3727161371 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11401a9290 con 0x7f11401027c0 2024-05-27T07:56:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.117+0000 7f114c869740 1 Processor -- start 2024-05-27T07:56:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.117+0000 7f114c869740 1 -- start start 2024-05-27T07:56:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.117+0000 7f114c869740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc9bc1b0 0x5585ccd5ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.117+0000 7f114c869740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9932f0 0x5585ccd5efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.117+0000 7f114c869740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 0x5585ccd5f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.117+0000 7f114c869740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5585ccccfb90 con 0x5585cc9932f0 2024-05-27T07:56:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.117+0000 7f114c869740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5585cccbf730 con 0x5585cc99b4d0 2024-05-27T07:56:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.117+0000 7f114c869740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5585cccc3770 con 0x5585cc9bc1b0 2024-05-27T07:56:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.118+0000 7f1137e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 0x5585ccd5f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.118+0000 7f1137400640 1 -- 172.21.15.63:0/3727161371 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1130010460 con 0x7f11401027c0 2024-05-27T07:56:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.118+0000 7f1137e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 0x5585ccd5f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:47382/0 (socket says 172.21.15.63:47382) 2024-05-27T07:56:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.118+0000 7f1137e00640 1 -- 172.21.15.63:0/2698440155 learned_addr learned my addr 172.21.15.63:0/2698440155 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.118+0000 7f113f400640 1 --2- 172.21.15.63:0/2698440155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc9bc1b0 0x5585ccd5ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.119+0000 7f113ea00640 1 --2- 172.21.15.63:0/2698440155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9932f0 0x5585ccd5efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.119+0000 7f1137e00640 1 -- 172.21.15.63:0/2698440155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc9bc1b0 msgr2=0x5585ccd5ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.119+0000 7f1137e00640 1 --2- 172.21.15.63:0/2698440155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc9bc1b0 0x5585ccd5ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.119+0000 7f1137e00640 1 -- 172.21.15.63:0/2698440155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9932f0 msgr2=0x5585ccd5efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.119+0000 7f1137e00640 1 --2- 172.21.15.63:0/2698440155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9932f0 0x5585ccd5efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.119+0000 7f1137e00640 1 -- 172.21.15.63:0/2698440155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ccc588b0 con 0x5585cc99b4d0 2024-05-27T07:56:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.120+0000 7f1137e00640 1 --2- 172.21.15.63:0/2698440155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 0x5585ccd5f4e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1128009900 tx=0x7f1128005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff2e6abe84103bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.120+0000 7f1137400640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11240b2950 0x7f11240b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.120+0000 7f112fe00640 1 -- 172.21.15.63:0/2698440155 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128040070 con 0x5585cc99b4d0 2024-05-27T07:56:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.120+0000 7f1137400640 1 -- 172.21.15.63:0/3727161371 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f11300e2670 con 0x7f11401027c0 2024-05-27T07:56:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.120+0000 7f112fe00640 1 -- 172.21.15.63:0/2698440155 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1128004490 con 0x5585cc99b4d0 2024-05-27T07:56:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.120+0000 7f112fe00640 1 -- 172.21.15.63:0/2698440155 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128035040 con 0x5585cc99b4d0 2024-05-27T07:56:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.120+0000 7f113f400640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11240b2950 0x7f11240b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.121+0000 7f113f400640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11240b2950 0x7f11240b4e60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5585ccd5bbd0 tx=0x7f1138009820 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.121+0000 7f114c869740 1 -- 172.21.15.63:0/2698440155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 msgr2=0x5585ccd5f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.121+0000 7f114c869740 1 --2- 172.21.15.63:0/2698440155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 0x5585ccd5f4e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1128009900 tx=0x7f1128005f30 comp rx=0 tx=0).stop 2024-05-27T07:56:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.122+0000 7f114c869740 1 -- 172.21.15.63:0/2698440155 shutdown_connections 2024-05-27T07:56:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.122+0000 7f114c869740 1 --2- 172.21.15.63:0/2698440155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 0x5585ccd5f4e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.122+0000 7f114c869740 1 --2- 172.21.15.63:0/2698440155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9932f0 0x5585ccd5efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.122+0000 7f114c869740 1 --2- 172.21.15.63:0/2698440155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc9bc1b0 0x5585ccd5ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.122+0000 7f114c869740 1 -- 172.21.15.63:0/2698440155 >> 172.21.15.63:0/2698440155 conn(0x5585cc9b0300 msgr2=0x5585ccd5b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.122+0000 7f114c869740 1 -- 172.21.15.63:0/2698440155 shutdown_connections 2024-05-27T07:56:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.122+0000 7f114c869740 1 -- 172.21.15.63:0/2698440155 wait complete. 2024-05-27T07:56:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.123+0000 7f114c869740 1 -- start start 2024-05-27T07:56:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.123+0000 7f114c869740 1 Processor -- start 2024-05-27T07:56:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f114c869740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc9932f0 0x5585ccd68bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f114c869740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc99b4d0 0x5585ccd69100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f114c869740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd74e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f114c869740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5585cc9a5790 con 0x5585cc9bc1b0 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f114c869740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5585ccc67bc0 con 0x5585cc9932f0 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f114c869740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5585ccd6bb30 con 0x5585cc99b4d0 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f114c869740 10 client.0 _ll_register_callbacks cb 0x5585ccc45400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f113f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd74e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f113f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd74e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54202/0 (socket says 172.21.15.63:54202) 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f113f400640 1 -- 172.21.15.63:0/352865704 learned_addr learned my addr 172.21.15.63:0/352865704 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f1137e00640 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc99b4d0 0x5585ccd69100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.124+0000 7f113ea00640 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc9932f0 0x5585ccd68bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.125+0000 7f113f400640 1 -- 172.21.15.63:0/352865704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc99b4d0 msgr2=0x5585ccd69100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.125+0000 7f113f400640 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc99b4d0 0x5585ccd69100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.125+0000 7f113f400640 1 -- 172.21.15.63:0/352865704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc9932f0 msgr2=0x5585ccd68bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.125+0000 7f113f400640 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc9932f0 0x5585ccd68bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.125+0000 7f113f400640 1 -- 172.21.15.63:0/352865704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585cc9d4210 con 0x5585cc9bc1b0 2024-05-27T07:56:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.125+0000 7f113f400640 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd74e60 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f1138010a10 tx=0x7f1138004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed28c4fe6ad56a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.125+0000 7f112cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.125+0000 7f112cc00640 1 -- 172.21.15.63:0/352865704 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138004d20 con 0x5585cc9bc1b0 2024-05-27T07:56:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.126+0000 7f114c869740 1 -- 172.21.15.63:0/352865704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1128004490 con 0x5585cc9bc1b0 2024-05-27T07:56:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.126+0000 7f112cc00640 1 -- 172.21.15.63:0/352865704 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f113801d6f0 con 0x5585cc9bc1b0 2024-05-27T07:56:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.126+0000 7f112cc00640 1 -- 172.21.15.63:0/352865704 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f113801d8f0 con 0x5585cc9bc1b0 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.126+0000 7f114c869740 10 client.19818 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.126+0000 7f114c869740 1 -- 172.21.15.63:0/352865704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5585ccaa0620 con 0x5585cc9bc1b0 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.126+0000 7f112cc00640 1 -- 172.21.15.63:0/352865704 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1128004490 con 0x5585cc9bc1b0 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.127+0000 7f112cc00640 1 -- 172.21.15.63:0/352865704 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1138005690 con 0x5585cc9bc1b0 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.127+0000 7f114c869740 10 client.19818 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.127+0000 7f114c869740 10 client.19818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.127+0000 7f114c869740 20 client.19818 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.127+0000 7f114c869740 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5585cc97ad80 0x5585ccd7ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.127+0000 7f114c869740 4 client.19818 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.127+0000 7f114c869740 1 -- 172.21.15.63:0/352865704 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585cccd1ae0 con 0x5585cc97ad80 2024-05-27T07:56:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.127+0000 7f113ea00640 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5585cc97ad80 0x5585ccd7ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.128+0000 7f112cc00640 10 client.19818 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.128+0000 7f113ea00640 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5585cc97ad80 0x5585ccd7ae10 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a931267f0eead873 server_cookie=ee4b220ad4273ebf in_seq=0 out_seq=0 2024-05-27T07:56:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.130+0000 7f112cc00640 1 -- 172.21.15.63:0/352865704 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5585cccd1ae0 con 0x5585cc97ad80 2024-05-27T07:56:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.130+0000 7f112cc00640 10 client.19818 handle_command_reply: tid=0 2024-05-27T07:56:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.130+0000 7f114c869740 1 client.19818 shutdown 2024-05-27T07:56:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 1 -- 172.21.15.63:0/352865704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 msgr2=0x5585ccd74e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd74e60 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f1138010a10 tx=0x7f1138004a40 comp rx=0 tx=0).stop 2024-05-27T07:56:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 1 -- 172.21.15.63:0/352865704 shutdown_connections 2024-05-27T07:56:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5585cc97ad80 0x5585ccd7ae10 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd74e60 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc99b4d0 0x5585ccd69100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 1 --2- 172.21.15.63:0/352865704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc9932f0 0x5585ccd68bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f112cc00640 0 client.19818 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 1 -- 172.21.15.63:0/352865704 >> 172.21.15.63:0/352865704 conn(0x5585cc9b0300 msgr2=0x5585ccd60780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 1 -- 172.21.15.63:0/352865704 shutdown_connections 2024-05-27T07:56:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 1 -- 172.21.15.63:0/352865704 wait complete. 2024-05-27T07:56:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.131+0000 7f114c869740 20 client.19818 trim_cache size 0 max 16384 2024-05-27T07:56:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.153+0000 7f114c869740 1 Processor -- start 2024-05-27T07:56:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.153+0000 7f114c869740 1 -- start start 2024-05-27T07:56:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.153+0000 7f114c869740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc9932f0 0x5585ccd69110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.153+0000 7f114c869740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc99b4d0 0x5585ccd6ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.153+0000 7f114c869740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd5f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.153+0000 7f114c869740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b5b00 con 0x5585cc9bc1b0 2024-05-27T07:56:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.153+0000 7f114c869740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b33d0 con 0x5585cc9932f0 2024-05-27T07:56:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.153+0000 7f114c869740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b1240 con 0x5585cc99b4d0 2024-05-27T07:56:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.154+0000 7f113f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd5f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.154+0000 7f113f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd5f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54210/0 (socket says 172.21.15.63:54210) 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.154+0000 7f113f400640 1 -- 172.21.15.63:0/3496999736 learned_addr learned my addr 172.21.15.63:0/3496999736 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.154+0000 7f113ea00640 1 --2- 172.21.15.63:0/3496999736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc9932f0 0x5585ccd69110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.154+0000 7f1137e00640 1 --2- 172.21.15.63:0/3496999736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc99b4d0 0x5585ccd6ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.154+0000 7f113f400640 1 -- 172.21.15.63:0/3496999736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc99b4d0 msgr2=0x5585ccd6ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.154+0000 7f113f400640 1 --2- 172.21.15.63:0/3496999736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc99b4d0 0x5585ccd6ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.154+0000 7f113f400640 1 -- 172.21.15.63:0/3496999736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc9932f0 msgr2=0x5585ccd69110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.154+0000 7f113f400640 1 --2- 172.21.15.63:0/3496999736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc9932f0 0x5585ccd69110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.154+0000 7f113f400640 1 -- 172.21.15.63:0/3496999736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ccd61170 con 0x5585cc9bc1b0 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.155+0000 7f113f400640 1 --2- 172.21.15.63:0/3496999736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd5f0f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f1138006010 tx=0x7f1138010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcb25e8bd1ff0fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.155+0000 7f112f400640 1 -- 172.21.15.63:0/3496999736 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113801da90 con 0x5585cc9bc1b0 2024-05-27T07:56:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.155+0000 7f112f400640 1 -- 172.21.15.63:0/3496999736 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f113801dc30 con 0x5585cc9bc1b0 2024-05-27T07:56:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.155+0000 7f112f400640 1 -- 172.21.15.63:0/3496999736 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138036690 con 0x5585cc9bc1b0 2024-05-27T07:56:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.156+0000 7f114c869740 1 -- 172.21.15.63:0/3496999736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 msgr2=0x5585ccd5f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.156+0000 7f114c869740 1 --2- 172.21.15.63:0/3496999736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd5f0f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f1138006010 tx=0x7f1138010ed0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.156+0000 7f114c869740 1 -- 172.21.15.63:0/3496999736 shutdown_connections 2024-05-27T07:56:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.156+0000 7f114c869740 1 --2- 172.21.15.63:0/3496999736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccd5f0f0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.156+0000 7f114c869740 1 --2- 172.21.15.63:0/3496999736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc99b4d0 0x5585ccd6ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.156+0000 7f114c869740 1 --2- 172.21.15.63:0/3496999736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc9932f0 0x5585ccd69110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.156+0000 7f114c869740 1 -- 172.21.15.63:0/3496999736 >> 172.21.15.63:0/3496999736 conn(0x5585cc9b0300 msgr2=0x5585ccd5e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.156+0000 7f114c869740 1 -- 172.21.15.63:0/3496999736 shutdown_connections 2024-05-27T07:56:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.156+0000 7f114c869740 1 -- 172.21.15.63:0/3496999736 wait complete. 2024-05-27T07:56:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.157+0000 7f114c869740 1 -- start start 2024-05-27T07:56:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.157+0000 7f114c869740 1 Processor -- start 2024-05-27T07:56:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f114c869740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc9932f0 0x5585ccd6c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f114c869740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 0x5585ccd6c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f114c869740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccde1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f114c869740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b1890 con 0x5585cc9bc1b0 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f114c869740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b2d00 con 0x5585cc99b4d0 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f114c869740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f11401b1f60 con 0x5585cc9932f0 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f114c869740 10 client.0 _ll_register_callbacks cb 0x7f1138009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f113f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccde1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f113f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccde1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54218/0 (socket says 172.21.15.63:54218) 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f113f400640 1 -- 172.21.15.63:0/314694362 learned_addr learned my addr 172.21.15.63:0/314694362 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f1137e00640 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 0x5585ccd6c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f113ea00640 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc9932f0 0x5585ccd6c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f113f400640 1 -- 172.21.15.63:0/314694362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc9932f0 msgr2=0x5585ccd6c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.158+0000 7f113f400640 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc9932f0 0x5585ccd6c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.159+0000 7f113f400640 1 -- 172.21.15.63:0/314694362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 msgr2=0x5585ccd6c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.159+0000 7f113f400640 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 0x5585ccd6c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.159+0000 7f113f400640 1 -- 172.21.15.63:0/314694362 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ccde19c0 con 0x5585cc9bc1b0 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.159+0000 7f113f400640 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccde1480 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f1138017040 tx=0x7f1138010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9286cec759b47b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.159+0000 7f112cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.159+0000 7f112cc00640 1 -- 172.21.15.63:0/314694362 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138044020 con 0x5585cc9bc1b0 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.159+0000 7f112cc00640 1 -- 172.21.15.63:0/314694362 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f1138005250 con 0x5585cc9bc1b0 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.159+0000 7f114c869740 1 -- 172.21.15.63:0/314694362 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f113801dc30 con 0x5585cc9bc1b0 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.160+0000 7f112cc00640 1 -- 172.21.15.63:0/314694362 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f113804f050 con 0x5585cc9bc1b0 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.160+0000 7f114c869740 10 client.19824 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.160+0000 7f114c869740 1 -- 172.21.15.63:0/314694362 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5585ccde2a90 con 0x5585cc9bc1b0 2024-05-27T07:56:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.160+0000 7f112cc00640 1 -- 172.21.15.63:0/314694362 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f113801dc30 con 0x5585cc9bc1b0 2024-05-27T07:56:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.160+0000 7f112cc00640 1 -- 172.21.15.63:0/314694362 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1138028cc0 con 0x5585cc9bc1b0 2024-05-27T07:56:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.160+0000 7f114c869740 10 client.19824 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.160+0000 7f114c869740 10 client.19824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.161+0000 7f114c869740 20 client.19824 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.161+0000 7f114c869740 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5585cc9501c0 0x5585ccde51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.161+0000 7f114c869740 4 client.19824 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:56:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.161+0000 7f114c869740 1 -- 172.21.15.63:0/314694362 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585ccccb2c0 con 0x5585cc9501c0 2024-05-27T07:56:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.161+0000 7f113ea00640 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5585cc9501c0 0x5585ccde51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.162+0000 7f112cc00640 10 client.19824 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.162+0000 7f113ea00640 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5585cc9501c0 0x5585ccde51b0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98064ea72c14da67 server_cookie=9d9e105a721176e1 in_seq=0 out_seq=0 2024-05-27T07:56:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f112cc00640 1 -- 172.21.15.63:0/314694362 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5585ccccb2c0 con 0x5585cc9501c0 2024-05-27T07:56:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f112cc00640 10 client.19824 handle_command_reply: tid=0 2024-05-27T07:56:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f114c869740 1 client.19824 shutdown 2024-05-27T07:56:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f114c869740 1 -- 172.21.15.63:0/314694362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 msgr2=0x5585ccde1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f114c869740 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccde1480 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f1138017040 tx=0x7f1138010640 comp rx=0 tx=0).stop 2024-05-27T07:56:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f114c869740 1 -- 172.21.15.63:0/314694362 shutdown_connections 2024-05-27T07:56:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f114c869740 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5585cc9501c0 0x5585ccde51b0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f114c869740 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585cc9bc1b0 0x5585ccde1480 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f114c869740 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5585cc99b4d0 0x5585ccd6c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f114c869740 1 --2- 172.21.15.63:0/314694362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5585cc9932f0 0x5585ccd6c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f114c869740 1 -- 172.21.15.63:0/314694362 >> 172.21.15.63:0/314694362 conn(0x5585cc9b0300 msgr2=0x5585ccd6a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f112cc00640 0 client.19824 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.198+0000 7f114c869740 1 -- 172.21.15.63:0/314694362 shutdown_connections 2024-05-27T07:56:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.199+0000 7f114c869740 1 -- 172.21.15.63:0/314694362 wait complete. 2024-05-27T07:56:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.199+0000 7f114c869740 20 client.19824 trim_cache size 0 max 16384 2024-05-27T07:56:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:56:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:56:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:56:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 -- 172.21.15.63:0/3727161371 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11240b2950 msgr2=0x7f11240b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11240b2950 0x7f11240b4e60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5585ccd5bbd0 tx=0x7f1138009820 comp rx=0 tx=0).stop 2024-05-27T07:56:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 -- 172.21.15.63:0/3727161371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11401027c0 msgr2=0x7f11401a24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11401027c0 0x7f11401a24f0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f113000d950 tx=0x7f113000de10 comp rx=0 tx=0).stop 2024-05-27T07:56:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 -- 172.21.15.63:0/3727161371 shutdown_connections 2024-05-27T07:56:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11240b2950 0x7f11240b4e60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140110360 0x7f114019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f114010da00 0x7f11401a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 --2- 172.21.15.63:0/3727161371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11401027c0 0x7f11401a24f0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 -- 172.21.15.63:0/3727161371 >> 172.21.15.63:0/3727161371 conn(0x7f11400fe210 msgr2=0x7f1140100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 -- 172.21.15.63:0/3727161371 shutdown_connections 2024-05-27T07:56:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:26.200+0000 7f113fe00640 1 -- 172.21.15.63:0/3727161371 wait complete. 2024-05-27T07:56:27.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:27 smithi086 ceph-mon[34687]: pgmap v723: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:56:27.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:27 smithi149 ceph-mon[35408]: pgmap v723: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:56:27.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:27 smithi063 ceph-mon[31613]: pgmap v723: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-05-27T07:56:29.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:29 smithi086 ceph-mon[34687]: pgmap v724: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T07:56:29.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:29 smithi149 ceph-mon[35408]: pgmap v724: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T07:56:29.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:29 smithi063 ceph-mon[31613]: pgmap v724: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T07:56:31.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:31 smithi086 ceph-mon[34687]: pgmap v725: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:56:31.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:31 smithi149 ceph-mon[35408]: pgmap v725: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:56:31.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:31 smithi063 ceph-mon[31613]: pgmap v725: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:56:33.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:33 smithi086 ceph-mon[34687]: pgmap v726: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:56:33.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:33 smithi149 ceph-mon[35408]: pgmap v726: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:56:33.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:33 smithi063 ceph-mon[31613]: pgmap v726: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:56:35.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:35 smithi086 ceph-mon[34687]: pgmap v727: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:56:35.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:35 smithi149 ceph-mon[35408]: pgmap v727: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:56:35.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:35 smithi063 ceph-mon[31613]: pgmap v727: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:56:37.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:37 smithi086 ceph-mon[34687]: pgmap v728: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:56:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:37 smithi149 ceph-mon[35408]: pgmap v728: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:56:37.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:37 smithi063 ceph-mon[31613]: pgmap v728: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:56:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.450+0000 7f99d7e00640 1 -- 172.21.15.63:0/4052409648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d810e8b0 msgr2=0x7f99d810ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.450+0000 7f99d7e00640 1 --2- 172.21.15.63:0/4052409648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d810e8b0 0x7f99d810ece0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009900 tx=0x7f99c402a410 comp rx=0 tx=0).stop 2024-05-27T07:56:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.450+0000 7f99d7e00640 1 -- 172.21.15.63:0/4052409648 shutdown_connections 2024-05-27T07:56:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.450+0000 7f99d7e00640 1 --2- 172.21.15.63:0/4052409648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d810f370 0x7f99d8116b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.450+0000 7f99d7e00640 1 --2- 172.21.15.63:0/4052409648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d810e8b0 0x7f99d810ece0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.450+0000 7f99d7e00640 1 --2- 172.21.15.63:0/4052409648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99d810a320 0x7f99d810a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.450+0000 7f99d7e00640 1 -- 172.21.15.63:0/4052409648 >> 172.21.15.63:0/4052409648 conn(0x7f99d8076070 msgr2=0x7f99d8076480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.450+0000 7f99d7e00640 1 -- 172.21.15.63:0/4052409648 shutdown_connections 2024-05-27T07:56:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.450+0000 7f99d7e00640 1 -- 172.21.15.63:0/4052409648 wait complete. 2024-05-27T07:56:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.451+0000 7f99d7e00640 1 Processor -- start 2024-05-27T07:56:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.452+0000 7f99d7e00640 1 -- start start 2024-05-27T07:56:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.452+0000 7f99d7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99d810a320 0x7f99d81a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.452+0000 7f99d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d810e8b0 0x7f99d81a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.452+0000 7f99d7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d810f370 0x7f99d81a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.452+0000 7f99d7e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99d811a120 con 0x7f99d810f370 2024-05-27T07:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.452+0000 7f99d7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8119fa0 con 0x7f99d810e8b0 2024-05-27T07:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.452+0000 7f99d7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99d811a2a0 con 0x7f99d810a320 2024-05-27T07:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d810f370 0x7f99d81a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d810f370 0x7f99d81a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40416/0 (socket says 172.21.15.63:40416) 2024-05-27T07:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d7400640 1 -- 172.21.15.63:0/1548077801 learned_addr learned my addr 172.21.15.63:0/1548077801 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d6a00640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99d810a320 0x7f99d81a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d6000640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d810e8b0 0x7f99d81a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d7400640 1 -- 172.21.15.63:0/1548077801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99d810a320 msgr2=0x7f99d81a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d7400640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99d810a320 0x7f99d81a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d7400640 1 -- 172.21.15.63:0/1548077801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d810e8b0 msgr2=0x7f99d81a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d7400640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d810e8b0 0x7f99d81a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d7400640 1 -- 172.21.15.63:0/1548077801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d81a10a0 con 0x7f99d810f370 2024-05-27T07:56:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.453+0000 7f99d6a00640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99d810a320 0x7f99d81a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.454+0000 7f99d7400640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d810f370 0x7f99d81a0a00 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f99d000eaa0 tx=0x7f99d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.454+0000 7f99cf400640 1 -- 172.21.15.63:0/1548077801 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d000cdb0 con 0x7f99d810f370 2024-05-27T07:56:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.454+0000 7f99cf400640 1 -- 172.21.15.63:0/1548077801 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d0005020 con 0x7f99d810f370 2024-05-27T07:56:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.454+0000 7f99d7e00640 1 -- 172.21.15.63:0/1548077801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d81a1390 con 0x7f99d810f370 2024-05-27T07:56:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.454+0000 7f99cf400640 1 -- 172.21.15.63:0/1548077801 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d00116d0 con 0x7f99d810f370 2024-05-27T07:56:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.454+0000 7f99d7e00640 1 -- 172.21.15.63:0/1548077801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d81ad670 con 0x7f99d810f370 2024-05-27T07:56:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.458+0000 7f99e4838740 1 Processor -- start 2024-05-27T07:56:39.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:39 smithi086 ceph-mon[34687]: pgmap v729: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.4 MiB/s wr, 12 op/s 2024-05-27T07:56:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.459+0000 7f99cf400640 1 -- 172.21.15.63:0/1548077801 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f99d00051d0 con 0x7f99d810f370 2024-05-27T07:56:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.459+0000 7f99e4838740 1 -- start start 2024-05-27T07:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.460+0000 7f99e4838740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d980b1b0 0x5636d9bada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.460+0000 7f99e4838740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97e22f0 0x5636d9badfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.460+0000 7f99e4838740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 0x5636d9bae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.460+0000 7f99e4838740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5636d9b1eb90 con 0x5636d97e22f0 2024-05-27T07:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.460+0000 7f99e4838740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5636d9b0e730 con 0x5636d980b1b0 2024-05-27T07:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.460+0000 7f99e4838740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5636d9b12770 con 0x5636d97ea4d0 2024-05-27T07:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.460+0000 7f99d6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97e22f0 0x5636d9badfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.460+0000 7f99d6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97e22f0 0x5636d9badfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40424/0 (socket says 172.21.15.63:40424) 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.460+0000 7f99d6a00640 1 -- 172.21.15.63:0/3723795163 learned_addr learned my addr 172.21.15.63:0/3723795163 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.461+0000 7f99d6000640 1 --2- 172.21.15.63:0/3723795163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d980b1b0 0x5636d9bada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.461+0000 7f99d6000640 1 --2- 172.21.15.63:0/3723795163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 0x5636d9bae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.461+0000 7f99d6a00640 1 -- 172.21.15.63:0/3723795163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 msgr2=0x5636d9bae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.461+0000 7f99d6a00640 1 --2- 172.21.15.63:0/3723795163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 0x5636d9bae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.461+0000 7f99d6a00640 1 -- 172.21.15.63:0/3723795163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d980b1b0 msgr2=0x5636d9bada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.461+0000 7f99d6a00640 1 --2- 172.21.15.63:0/3723795163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d980b1b0 0x5636d9bada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.461+0000 7f99d6a00640 1 -- 172.21.15.63:0/3723795163 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636d9aa78b0 con 0x5636d97e22f0 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.461+0000 7f99d6000640 1 --2- 172.21.15.63:0/3723795163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 0x5636d9bae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.461+0000 7f99cf400640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99bc0b29d0 0x7f99bc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.461+0000 7f99d6a00640 1 --2- 172.21.15.63:0/3723795163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97e22f0 0x5636d9badfa0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f99c800bd30 tx=0x7f99c800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccff8f55883b91fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:39.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:39 smithi149 ceph-mon[35408]: pgmap v729: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.4 MiB/s wr, 12 op/s 2024-05-27T07:56:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.462+0000 7f99cf400640 1 -- 172.21.15.63:0/1548077801 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f99d00e2e60 con 0x7f99d810f370 2024-05-27T07:56:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.462+0000 7f99d6000640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99bc0b29d0 0x7f99bc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.462+0000 7f99c3e00640 1 -- 172.21.15.63:0/3723795163 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8015070 con 0x5636d97e22f0 2024-05-27T07:56:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.462+0000 7f99c3e00640 1 -- 172.21.15.63:0/3723795163 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99c800be20 con 0x5636d97e22f0 2024-05-27T07:56:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.462+0000 7f99c3e00640 1 -- 172.21.15.63:0/3723795163 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8011440 con 0x5636d97e22f0 2024-05-27T07:56:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.463+0000 7f99e4838740 1 -- 172.21.15.63:0/3723795163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97e22f0 msgr2=0x5636d9badfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.463+0000 7f99e4838740 1 --2- 172.21.15.63:0/3723795163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97e22f0 0x5636d9badfa0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f99c800bd30 tx=0x7f99c800f730 comp rx=0 tx=0).stop 2024-05-27T07:56:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.463+0000 7f99d6000640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99bc0b29d0 0x7f99bc0b4ee0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f99c4004770 tx=0x7f99c4005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.463+0000 7f99e4838740 1 -- 172.21.15.63:0/3723795163 shutdown_connections 2024-05-27T07:56:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.463+0000 7f99e4838740 1 --2- 172.21.15.63:0/3723795163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 0x5636d9bae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.463+0000 7f99e4838740 1 --2- 172.21.15.63:0/3723795163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97e22f0 0x5636d9badfa0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.463+0000 7f99e4838740 1 --2- 172.21.15.63:0/3723795163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d980b1b0 0x5636d9bada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.463+0000 7f99e4838740 1 -- 172.21.15.63:0/3723795163 >> 172.21.15.63:0/3723795163 conn(0x5636d97ff300 msgr2=0x5636d9baa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.463+0000 7f99e4838740 1 -- 172.21.15.63:0/3723795163 shutdown_connections 2024-05-27T07:56:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.464+0000 7f99e4838740 1 -- 172.21.15.63:0/3723795163 wait complete. 2024-05-27T07:56:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.464+0000 7f99e4838740 1 -- start start 2024-05-27T07:56:39.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:39 smithi063 ceph-mon[31613]: pgmap v729: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.4 MiB/s wr, 12 op/s 2024-05-27T07:56:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.464+0000 7f99e4838740 1 Processor -- start 2024-05-27T07:56:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.465+0000 7f99e4838740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bb7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.465+0000 7f99e4838740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bb80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.465+0000 7f99e4838740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 0x5636d9bc3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.465+0000 7f99e4838740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5636d97f4790 con 0x5636d97ea4d0 2024-05-27T07:56:39.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.465+0000 7f99e4838740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5636d9ab6bc0 con 0x5636d97e22f0 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.465+0000 7f99e4838740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5636d9bbab90 con 0x5636d980b1b0 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.465+0000 7f99e4838740 10 client.0 _ll_register_callbacks cb 0x5636d9a94400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.465+0000 7f99d6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bb80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.465+0000 7f99d6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bb80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40438/0 (socket says 172.21.15.63:40438) 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.465+0000 7f99d6000640 1 -- 172.21.15.63:0/1369416004 learned_addr learned my addr 172.21.15.63:0/1369416004 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.466+0000 7f99d6a00640 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bb7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.466+0000 7f99d7400640 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 0x5636d9bc3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.466+0000 7f99d6000640 1 -- 172.21.15.63:0/1369416004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 msgr2=0x5636d9bc3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.466+0000 7f99d6000640 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 0x5636d9bc3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.466+0000 7f99d6000640 1 -- 172.21.15.63:0/1369416004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 msgr2=0x5636d9bb7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.466+0000 7f99d6000640 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bb7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.466+0000 7f99d6000640 1 -- 172.21.15.63:0/1369416004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d9823210 con 0x5636d97ea4d0 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.466+0000 7f99d6a00640 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bb7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.466+0000 7f99d6000640 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bb80d0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009900 tx=0x7f99c40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88abca1d6a17dcc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.466+0000 7f99c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.467+0000 7f99c0c00640 1 -- 172.21.15.63:0/1369416004 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4035040 con 0x5636d97ea4d0 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.467+0000 7f99e4838740 1 -- 172.21.15.63:0/1369416004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c800be20 con 0x5636d97ea4d0 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.467+0000 7f99c0c00640 1 -- 172.21.15.63:0/1369416004 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f99c40035f0 con 0x5636d97ea4d0 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.467+0000 7f99c0c00640 1 -- 172.21.15.63:0/1369416004 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f99c405a050 con 0x5636d97ea4d0 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.467+0000 7f99e4838740 10 client.19836 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.467+0000 7f99e4838740 1 -- 172.21.15.63:0/1369416004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5636d98ef620 con 0x5636d97ea4d0 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.468+0000 7f99c0c00640 1 -- 172.21.15.63:0/1369416004 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99c800be20 con 0x5636d97ea4d0 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.468+0000 7f99c0c00640 1 -- 172.21.15.63:0/1369416004 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f99c403d720 con 0x5636d97ea4d0 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.468+0000 7f99e4838740 10 client.19836 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.468+0000 7f99e4838740 10 client.19836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.468+0000 7f99e4838740 20 client.19836 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.468+0000 7f99e4838740 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5636d97c9d80 0x5636d9bc9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.468+0000 7f99e4838740 4 client.19836 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.468+0000 7f99e4838740 1 -- 172.21.15.63:0/1369416004 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636d9b20ae0 con 0x5636d97c9d80 2024-05-27T07:56:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.469+0000 7f99d6a00640 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5636d97c9d80 0x5636d9bc9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.470+0000 7f99d6a00640 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5636d97c9d80 0x5636d9bc9e40 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ca61552a505d82 server_cookie=7845c0c4f5d04b8f in_seq=0 out_seq=0 2024-05-27T07:56:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.470+0000 7f99c0c00640 10 client.19836 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.471+0000 7f99c0c00640 1 -- 172.21.15.63:0/1369416004 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5636d9b20ae0 con 0x5636d97c9d80 2024-05-27T07:56:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.471+0000 7f99c0c00640 10 client.19836 handle_command_reply: tid=0 2024-05-27T07:56:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 client.19836 shutdown 2024-05-27T07:56:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 -- 172.21.15.63:0/1369416004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 msgr2=0x5636d9bb80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bb80d0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009900 tx=0x7f99c40058d0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 -- 172.21.15.63:0/1369416004 shutdown_connections 2024-05-27T07:56:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5636d97c9d80 0x5636d9bc9e40 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 0x5636d9bc3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bb80d0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99c0c00640 0 client.19836 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 --2- 172.21.15.63:0/1369416004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bb7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 -- 172.21.15.63:0/1369416004 >> 172.21.15.63:0/1369416004 conn(0x5636d97ff300 msgr2=0x5636d9baf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 -- 172.21.15.63:0/1369416004 shutdown_connections 2024-05-27T07:56:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 1 -- 172.21.15.63:0/1369416004 wait complete. 2024-05-27T07:56:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.472+0000 7f99e4838740 20 client.19836 trim_cache size 0 max 16384 2024-05-27T07:56:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.494+0000 7f99e4838740 1 Processor -- start 2024-05-27T07:56:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.494+0000 7f99e4838740 1 -- start start 2024-05-27T07:56:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.495+0000 7f99e4838740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bc3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.495+0000 7f99e4838740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bc41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.495+0000 7f99e4838740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 0x5636d9bb1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.495+0000 7f99e4838740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99d81b9eb0 con 0x5636d97ea4d0 2024-05-27T07:56:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.495+0000 7f99e4838740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99d81b7780 con 0x5636d97e22f0 2024-05-27T07:56:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.495+0000 7f99e4838740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99d81b55f0 con 0x5636d980b1b0 2024-05-27T07:56:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.495+0000 7f99d6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bc41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.495+0000 7f99d6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bc41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40454/0 (socket says 172.21.15.63:40454) 2024-05-27T07:56:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.495+0000 7f99d6000640 1 -- 172.21.15.63:0/541307551 learned_addr learned my addr 172.21.15.63:0/541307551 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.496+0000 7f99d6a00640 1 --2- 172.21.15.63:0/541307551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bc3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.496+0000 7f99d7400640 1 --2- 172.21.15.63:0/541307551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 0x5636d9bb1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.496+0000 7f99d6000640 1 -- 172.21.15.63:0/541307551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 msgr2=0x5636d9bb1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.496+0000 7f99d6000640 1 --2- 172.21.15.63:0/541307551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 0x5636d9bb1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.496+0000 7f99d6000640 1 -- 172.21.15.63:0/541307551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 msgr2=0x5636d9bc3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.496+0000 7f99d6000640 1 --2- 172.21.15.63:0/541307551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bc3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.496+0000 7f99d6000640 1 -- 172.21.15.63:0/541307551 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636d9baab10 con 0x5636d97ea4d0 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.496+0000 7f99d7400640 1 --2- 172.21.15.63:0/541307551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 0x5636d9bb1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.496+0000 7f99d6000640 1 --2- 172.21.15.63:0/541307551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bc41a0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f99c4033000 tx=0x7f99c402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c708e9a09824068b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.497+0000 7f99c3400640 1 -- 172.21.15.63:0/541307551 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4063070 con 0x5636d97ea4d0 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.497+0000 7f99c3400640 1 -- 172.21.15.63:0/541307551 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99c403d8f0 con 0x5636d97ea4d0 2024-05-27T07:56:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.497+0000 7f99c3400640 1 -- 172.21.15.63:0/541307551 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4003b20 con 0x5636d97ea4d0 2024-05-27T07:56:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.498+0000 7f99e4838740 1 -- 172.21.15.63:0/541307551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 msgr2=0x5636d9bc41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.498+0000 7f99e4838740 1 --2- 172.21.15.63:0/541307551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bc41a0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f99c4033000 tx=0x7f99c402a410 comp rx=0 tx=0).stop 2024-05-27T07:56:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.498+0000 7f99e4838740 1 -- 172.21.15.63:0/541307551 shutdown_connections 2024-05-27T07:56:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.498+0000 7f99e4838740 1 --2- 172.21.15.63:0/541307551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d980b1b0 0x5636d9bb1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.498+0000 7f99e4838740 1 --2- 172.21.15.63:0/541307551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d97ea4d0 0x5636d9bc41a0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.498+0000 7f99e4838740 1 --2- 172.21.15.63:0/541307551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bc3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.498+0000 7f99e4838740 1 -- 172.21.15.63:0/541307551 >> 172.21.15.63:0/541307551 conn(0x5636d97ff300 msgr2=0x5636d9bae220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.498+0000 7f99e4838740 1 -- 172.21.15.63:0/541307551 shutdown_connections 2024-05-27T07:56:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.498+0000 7f99e4838740 1 -- 172.21.15.63:0/541307551 wait complete. 2024-05-27T07:56:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.499+0000 7f99e4838740 1 -- start start 2024-05-27T07:56:39.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.499+0000 7f99e4838740 1 Processor -- start 2024-05-27T07:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99e4838740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bc01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99e4838740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 0x5636d9bbb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99e4838740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d980b1b0 0x5636d9bbba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99e4838740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99d81b5c40 con 0x5636d980b1b0 2024-05-27T07:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99e4838740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99d81b70b0 con 0x5636d97e22f0 2024-05-27T07:56:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99e4838740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99d81b6310 con 0x5636d97ea4d0 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99e4838740 10 client.0 _ll_register_callbacks cb 0x7f99c4032f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99d7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d980b1b0 0x5636d9bbba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99d7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d980b1b0 0x5636d9bbba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40470/0 (socket says 172.21.15.63:40470) 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99d7400640 1 -- 172.21.15.63:0/1411090447 learned_addr learned my addr 172.21.15.63:0/1411090447 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99d6a00640 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bc01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99d6000640 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 0x5636d9bbb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99d7400640 1 -- 172.21.15.63:0/1411090447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 msgr2=0x5636d9bbb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.500+0000 7f99d7400640 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 0x5636d9bbb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.501+0000 7f99d7400640 1 -- 172.21.15.63:0/1411090447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 msgr2=0x5636d9bc01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.501+0000 7f99d7400640 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bc01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.501+0000 7f99d7400640 1 -- 172.21.15.63:0/1411090447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d9bbbfb0 con 0x5636d980b1b0 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.501+0000 7f99d6a00640 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bc01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.501+0000 7f99d7400640 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d980b1b0 0x5636d9bbba70 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f99d00a9a80 tx=0x7f99d00a9950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4274b6456f0349b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.501+0000 7f99c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.501+0000 7f99c0c00640 1 -- 172.21.15.63:0/1411090447 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d001f030 con 0x5636d980b1b0 2024-05-27T07:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.501+0000 7f99c0c00640 1 -- 172.21.15.63:0/1411090447 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f99d00a6370 con 0x5636d980b1b0 2024-05-27T07:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.501+0000 7f99e4838740 1 -- 172.21.15.63:0/1411090447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c403d8f0 con 0x5636d980b1b0 2024-05-27T07:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99c0c00640 1 -- 172.21.15.63:0/1411090447 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f99d00ae130 con 0x5636d980b1b0 2024-05-27T07:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99e4838740 10 client.19842 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99e4838740 1 -- 172.21.15.63:0/1411090447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5636d9c32010 con 0x5636d980b1b0 2024-05-27T07:56:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99c0c00640 1 -- 172.21.15.63:0/1411090447 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99c403d8f0 con 0x5636d980b1b0 2024-05-27T07:56:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99c0c00640 1 -- 172.21.15.63:0/1411090447 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f99d000f040 con 0x5636d980b1b0 2024-05-27T07:56:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99e4838740 10 client.19842 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99e4838740 10 client.19842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99e4838740 20 client.19842 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99e4838740 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5636d979f1c0 0x5636d9c347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99e4838740 4 client.19842 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:56:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.502+0000 7f99e4838740 1 -- 172.21.15.63:0/1411090447 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636d9b1a2c0 con 0x5636d979f1c0 2024-05-27T07:56:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.503+0000 7f99d6a00640 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5636d979f1c0 0x5636d9c347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.504+0000 7f99d6a00640 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5636d979f1c0 0x5636d9c347a0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb96fcd6fd3df77 server_cookie=62574f4c1ad27d16 in_seq=0 out_seq=0 2024-05-27T07:56:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.504+0000 7f99c0c00640 10 client.19842 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.545+0000 7f99c0c00640 1 -- 172.21.15.63:0/1411090447 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5636d9b1a2c0 con 0x5636d979f1c0 2024-05-27T07:56:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.545+0000 7f99c0c00640 10 client.19842 handle_command_reply: tid=0 2024-05-27T07:56:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 client.19842 shutdown 2024-05-27T07:56:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 -- 172.21.15.63:0/1411090447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d980b1b0 msgr2=0x5636d9bbba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d980b1b0 0x5636d9bbba70 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f99d00a9a80 tx=0x7f99d00a9950 comp rx=0 tx=0).stop 2024-05-27T07:56:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 -- 172.21.15.63:0/1411090447 shutdown_connections 2024-05-27T07:56:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5636d979f1c0 0x5636d9c347a0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636d980b1b0 0x5636d9bbba70 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5636d97ea4d0 0x5636d9bbb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 --2- 172.21.15.63:0/1411090447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d97e22f0 0x5636d9bc01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 -- 172.21.15.63:0/1411090447 >> 172.21.15.63:0/1411090447 conn(0x5636d97ff300 msgr2=0x5636d9bb70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99c0c00640 0 client.19842 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 -- 172.21.15.63:0/1411090447 shutdown_connections 2024-05-27T07:56:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 1 -- 172.21.15.63:0/1411090447 wait complete. 2024-05-27T07:56:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.546+0000 7f99e4838740 20 client.19842 trim_cache size 0 max 16384 2024-05-27T07:56:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:56:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:56:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:56:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.547+0000 7f99d7e00640 1 -- 172.21.15.63:0/1548077801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99bc0b29d0 msgr2=0x7f99bc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.547+0000 7f99d7e00640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99bc0b29d0 0x7f99bc0b4ee0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f99c4004770 tx=0x7f99c4005fd0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.547+0000 7f99d7e00640 1 -- 172.21.15.63:0/1548077801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d810f370 msgr2=0x7f99d81a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.547+0000 7f99d7e00640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d810f370 0x7f99d81a0a00 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f99d000eaa0 tx=0x7f99d000ef60 comp rx=0 tx=0).stop 2024-05-27T07:56:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.547+0000 7f99d7e00640 1 -- 172.21.15.63:0/1548077801 shutdown_connections 2024-05-27T07:56:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.547+0000 7f99d7e00640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99bc0b29d0 0x7f99bc0b4ee0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.547+0000 7f99d7e00640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99d810f370 0x7f99d81a0a00 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.547+0000 7f99d7e00640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99d810e8b0 0x7f99d81a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.547+0000 7f99d7e00640 1 --2- 172.21.15.63:0/1548077801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99d810a320 0x7f99d81a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.547+0000 7f99d7e00640 1 -- 172.21.15.63:0/1548077801 >> 172.21.15.63:0/1548077801 conn(0x7f99d8076070 msgr2=0x7f99d8107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.548+0000 7f99d7e00640 1 -- 172.21.15.63:0/1548077801 shutdown_connections 2024-05-27T07:56:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:39.548+0000 7f99d7e00640 1 -- 172.21.15.63:0/1548077801 wait complete. 2024-05-27T07:56:40.674 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:56:41.180 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:41.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:41 smithi086 ceph-mon[34687]: pgmap v730: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 MiB/s wr, 6 op/s 2024-05-27T07:56:41.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:41 smithi149 ceph-mon[35408]: pgmap v730: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 MiB/s wr, 6 op/s 2024-05-27T07:56:41.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:41 smithi063 ceph-mon[31613]: pgmap v730: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.3 MiB/s wr, 6 op/s 2024-05-27T07:56:43.418 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:43 smithi063 ceph-mon[31613]: pgmap v731: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:56:43.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:43 smithi086 ceph-mon[34687]: pgmap v731: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:56:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:43 smithi149 ceph-mon[35408]: pgmap v731: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:56:43.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.687+0000 7f2f56a00700 1 -- 172.21.15.63:0/250024420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f50105450 msgr2=0x7f2f50105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:43.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.687+0000 7f2f56a00700 1 --2- 172.21.15.63:0/250024420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f50105450 0x7f2f50105860 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009990 tx=0x7f2f40009ca0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.687+0000 7f2f56a00700 1 -- 172.21.15.63:0/250024420 shutdown_connections 2024-05-27T07:56:43.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.687+0000 7f2f56a00700 1 --2- 172.21.15.63:0/250024420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f5010e8c0 0x7f2f50110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.687+0000 7f2f56a00700 1 --2- 172.21.15.63:0/250024420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f50105450 0x7f2f50105860 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.687+0000 7f2f56a00700 1 --2- 172.21.15.63:0/250024420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50105da0 0x7f2f5010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.687+0000 7f2f56a00700 1 -- 172.21.15.63:0/250024420 >> 172.21.15.63:0/250024420 conn(0x7f2f500fedb0 msgr2=0x7f2f501011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:43.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.688+0000 7f2f56a00700 1 -- 172.21.15.63:0/250024420 shutdown_connections 2024-05-27T07:56:43.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.688+0000 7f2f56a00700 1 -- 172.21.15.63:0/250024420 wait complete. 2024-05-27T07:56:43.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.690+0000 7f2f56a00700 1 Processor -- start 2024-05-27T07:56:43.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.690+0000 7f2f56a00700 1 -- start start 2024-05-27T07:56:43.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.690+0000 7f2f56a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f50105450 0x7f2f5019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.690+0000 7f2f56a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50105da0 0x7f2f5019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.691+0000 7f2f56a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f5010e8c0 0x7f2f501a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.691+0000 7f2f56a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50114310 con 0x7f2f50105450 2024-05-27T07:56:43.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.691+0000 7f2f56a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50114190 con 0x7f2f5010e8c0 2024-05-27T07:56:43.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.691+0000 7f2f56a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50114490 con 0x7f2f50105da0 2024-05-27T07:56:43.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.691+0000 7f2f4fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f50105450 0x7f2f5019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.691+0000 7f2f4fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f50105450 0x7f2f5019cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40488/0 (socket says 172.21.15.63:40488) 2024-05-27T07:56:43.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.691+0000 7f2f4fe00700 1 -- 172.21.15.63:0/1353512588 learned_addr learned my addr 172.21.15.63:0/1353512588 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:43.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.691+0000 7f2f4f400700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50105da0 0x7f2f5019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.691+0000 7f2f54a00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f5010e8c0 0x7f2f501a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.692+0000 7f2f4fe00700 1 -- 172.21.15.63:0/1353512588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50105da0 msgr2=0x7f2f5019d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.692+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50105da0 0x7f2f5019d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.692+0000 7f2f4fe00700 1 -- 172.21.15.63:0/1353512588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f5010e8c0 msgr2=0x7f2f501a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.692+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f5010e8c0 0x7f2f501a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.692+0000 7f2f4fe00700 1 -- 172.21.15.63:0/1353512588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f501a1dd0 con 0x7f2f50105450 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.692+0000 7f2f4f400700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50105da0 0x7f2f5019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.692+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f50105450 0x7f2f5019cf00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f2f40005910 tx=0x7f2f400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.693+0000 7f2f4cc00700 1 -- 172.21.15.63:0/1353512588 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40027070 con 0x7f2f50105450 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.693+0000 7f2f4cc00700 1 -- 172.21.15.63:0/1353512588 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f4000fcc0 con 0x7f2f50105450 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.693+0000 7f2f56a00700 1 -- 172.21.15.63:0/1353512588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f501a2060 con 0x7f2f50105450 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.693+0000 7f2f4cc00700 1 -- 172.21.15.63:0/1353512588 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40022cf0 con 0x7f2f50105450 2024-05-27T07:56:43.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.693+0000 7f2f56a00700 1 -- 172.21.15.63:0/1353512588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f501a2540 con 0x7f2f50105450 2024-05-27T07:56:43.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.698+0000 7f2f4cc00700 1 -- 172.21.15.63:0/1353512588 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2f4002a070 con 0x7f2f50105450 2024-05-27T07:56:43.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.699+0000 7f2f655f1b80 1 Processor -- start 2024-05-27T07:56:43.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.699+0000 7f2f655f1b80 1 -- start start 2024-05-27T07:56:43.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.699+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56007884c9f0 0x5600788e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.699+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560078760e40 0x5600788e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.700+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600788e8670 0x5600788f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.700+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56007880de40 con 0x56007884c9f0 2024-05-27T07:56:43.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.700+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5600787fd9e0 con 0x560078760e40 2024-05-27T07:56:43.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.700+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560078801a20 con 0x5600788e8670 2024-05-27T07:56:43.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.700+0000 7f2f54a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56007884c9f0 0x5600788e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.700+0000 7f2f54a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56007884c9f0 0x5600788e7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40496/0 (socket says 172.21.15.63:40496) 2024-05-27T07:56:43.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.700+0000 7f2f54a00700 1 -- 172.21.15.63:0/1583084865 learned_addr learned my addr 172.21.15.63:0/1583084865 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:43.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.700+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/1583084865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560078760e40 0x5600788e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.700+0000 7f2f4f400700 1 --2- 172.21.15.63:0/1583084865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600788e8670 0x5600788f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.701+0000 7f2f4cc00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f340b29f0 0x7f2f340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.701+0000 7f2f54a00700 1 -- 172.21.15.63:0/1583084865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600788e8670 msgr2=0x5600788f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:43.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.701+0000 7f2f54a00700 1 --2- 172.21.15.63:0/1583084865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600788e8670 0x5600788f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.701+0000 7f2f54a00700 1 -- 172.21.15.63:0/1583084865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560078760e40 msgr2=0x5600788e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:43.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.701+0000 7f2f54a00700 1 --2- 172.21.15.63:0/1583084865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560078760e40 0x5600788e8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.701+0000 7f2f54a00700 1 -- 172.21.15.63:0/1583084865 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600787c6420 con 0x56007884c9f0 2024-05-27T07:56:43.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.701+0000 7f2f4cc00700 1 -- 172.21.15.63:0/1353512588 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f2f400e3850 con 0x7f2f50105450 2024-05-27T07:56:43.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.702+0000 7f2f54a00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f340b29f0 0x7f2f340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.702+0000 7f2f54a00700 1 --2- 172.21.15.63:0/1583084865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56007884c9f0 0x5600788e7b30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009d20 tx=0x7f2f480077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee04f6ab57c1dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:43.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.703+0000 7f2f45600700 1 -- 172.21.15.63:0/1583084865 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48014530 con 0x56007884c9f0 2024-05-27T07:56:43.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.703+0000 7f2f54a00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f340b29f0 0x7f2f340b4f00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2f4801e000 tx=0x7f2f4801f000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:43.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.703+0000 7f2f45600700 1 -- 172.21.15.63:0/1583084865 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f48021ac0 con 0x56007884c9f0 2024-05-27T07:56:43.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.703+0000 7f2f45600700 1 -- 172.21.15.63:0/1583084865 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4801f990 con 0x56007884c9f0 2024-05-27T07:56:43.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.705+0000 7f2f655f1b80 1 -- 172.21.15.63:0/1583084865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56007884c9f0 msgr2=0x5600788e7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:43.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.705+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/1583084865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56007884c9f0 0x5600788e7b30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009d20 tx=0x7f2f480077e0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.705+0000 7f2f655f1b80 1 -- 172.21.15.63:0/1583084865 shutdown_connections 2024-05-27T07:56:43.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.705+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/1583084865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600788e8670 0x5600788f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.705+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/1583084865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56007884c9f0 0x5600788e7b30 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.705+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/1583084865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560078760e40 0x5600788e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.705+0000 7f2f655f1b80 1 -- 172.21.15.63:0/1583084865 >> 172.21.15.63:0/1583084865 conn(0x5600787fa390 msgr2=0x5600788db380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:43.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.705+0000 7f2f655f1b80 1 -- 172.21.15.63:0/1583084865 shutdown_connections 2024-05-27T07:56:43.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.705+0000 7f2f655f1b80 1 -- 172.21.15.63:0/1583084865 wait complete. 2024-05-27T07:56:43.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.706+0000 7f2f655f1b80 1 -- start start 2024-05-27T07:56:43.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.706+0000 7f2f655f1b80 1 Processor -- start 2024-05-27T07:56:43.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.707+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.707+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 0x5600788f2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.707+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600788e8670 0x5600788f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.707+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560078703fa0 con 0x560078760e40 2024-05-27T07:56:43.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.707+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56007879e760 con 0x56007884c9f0 2024-05-27T07:56:43.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.707+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5600788f53a0 con 0x5600788e8670 2024-05-27T07:56:43.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.707+0000 7f2f655f1b80 10 client.0 _ll_register_callbacks cb 0x5600787a7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:43.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f4fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f4fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40508/0 (socket says 172.21.15.63:40508) 2024-05-27T07:56:43.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f4fe00700 1 -- 172.21.15.63:0/3559532522 learned_addr learned my addr 172.21.15.63:0/3559532522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:43.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f4f400700 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 0x5600788f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f54a00700 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600788e8670 0x5600788f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f4fe00700 1 -- 172.21.15.63:0/3559532522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600788e8670 msgr2=0x5600788f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:43.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600788e8670 0x5600788f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f4fe00700 1 -- 172.21.15.63:0/3559532522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 msgr2=0x5600788f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:43.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 0x5600788f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f4fe00700 1 -- 172.21.15.63:0/3559532522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56007874d8d0 con 0x560078760e40 2024-05-27T07:56:43.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.708+0000 7f2f4f400700 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 0x5600788f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:43.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.709+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788f1ff0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000c00 tx=0x7f2f40009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8452e8a7ad074a31 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:43.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.709+0000 7f2f3a000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:43.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.709+0000 7f2f3a000700 1 -- 172.21.15.63:0/3559532522 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f400a6660 con 0x560078760e40 2024-05-27T07:56:43.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.709+0000 7f2f3a000700 1 -- 172.21.15.63:0/3559532522 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f2f4000f830 con 0x560078760e40 2024-05-27T07:56:43.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.710+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3559532522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f48021ac0 con 0x560078760e40 2024-05-27T07:56:43.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.710+0000 7f2f3a000700 1 -- 172.21.15.63:0/3559532522 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2f400e9020 con 0x560078760e40 2024-05-27T07:56:43.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.710+0000 7f2f655f1b80 10 client.19854 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:43.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.710+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3559532522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5600787ce060 con 0x560078760e40 2024-05-27T07:56:43.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.711+0000 7f2f3a000700 1 -- 172.21.15.63:0/3559532522 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f48021ac0 con 0x560078760e40 2024-05-27T07:56:43.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.711+0000 7f2f3a000700 1 -- 172.21.15.63:0/3559532522 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2f400afce0 con 0x560078760e40 2024-05-27T07:56:43.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.711+0000 7f2f655f1b80 10 client.19854 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:43.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.711+0000 7f2f655f1b80 10 client.19854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:43.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.711+0000 7f2f655f1b80 20 client.19854 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:43.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.711+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5600789022e0 0x560078904770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.711+0000 7f2f655f1b80 4 client.19854 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:43.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.711+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3559532522 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56007880fd90 con 0x5600789022e0 2024-05-27T07:56:43.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.712+0000 7f2f4f400700 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5600789022e0 0x560078904770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.713+0000 7f2f3a000700 10 client.19854 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:43.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.713+0000 7f2f4f400700 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5600789022e0 0x560078904770 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d3e8f9097c2fd3 server_cookie=b0b7109fc26f70b5 in_seq=0 out_seq=0 2024-05-27T07:56:43.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.715+0000 7f2f3a000700 1 -- 172.21.15.63:0/3559532522 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56007880fd90 con 0x5600789022e0 2024-05-27T07:56:43.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.715+0000 7f2f3a000700 10 client.19854 handle_command_reply: tid=0 2024-05-27T07:56:43.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.715+0000 7f2f655f1b80 1 client.19854 shutdown 2024-05-27T07:56:43.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3559532522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 msgr2=0x5600788f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:43.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788f1ff0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000c00 tx=0x7f2f40009ca0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3559532522 shutdown_connections 2024-05-27T07:56:43.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5600789022e0 0x560078904770 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5600788e8670 0x5600788f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788f1ff0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3559532522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 0x5600788f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3559532522 >> 172.21.15.63:0/3559532522 conn(0x5600787fa390 msgr2=0x5600788e7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:43.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f3a000700 0 client.19854 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:43.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3559532522 shutdown_connections 2024-05-27T07:56:43.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3559532522 wait complete. 2024-05-27T07:56:43.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.716+0000 7f2f655f1b80 20 client.19854 trim_cache size 0 max 16384 2024-05-27T07:56:43.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.749+0000 7f2f655f1b80 1 Processor -- start 2024-05-27T07:56:43.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.749+0000 7f2f655f1b80 1 -- start start 2024-05-27T07:56:43.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.749+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560078760e40 0x5600788f1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.749+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 0x5600788ed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.749+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600788e8670 0x5600788fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.749+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501b0ae0 con 0x5600788e8670 2024-05-27T07:56:43.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.749+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501ae370 con 0x56007884c9f0 2024-05-27T07:56:43.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.749+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501ac220 con 0x560078760e40 2024-05-27T07:56:43.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.750+0000 7f2f54a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600788e8670 0x5600788fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.750+0000 7f2f54a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600788e8670 0x5600788fe490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40510/0 (socket says 172.21.15.63:40510) 2024-05-27T07:56:43.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.750+0000 7f2f54a00700 1 -- 172.21.15.63:0/2324104161 learned_addr learned my addr 172.21.15.63:0/2324104161 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:43.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.750+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/2324104161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560078760e40 0x5600788f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.750+0000 7f2f4f400700 1 --2- 172.21.15.63:0/2324104161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 0x5600788ed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.751+0000 7f2f54a00700 1 -- 172.21.15.63:0/2324104161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560078760e40 msgr2=0x5600788f1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:43.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.751+0000 7f2f54a00700 1 --2- 172.21.15.63:0/2324104161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560078760e40 0x5600788f1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.751+0000 7f2f54a00700 1 -- 172.21.15.63:0/2324104161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 msgr2=0x5600788ed7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:43.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.751+0000 7f2f54a00700 1 --2- 172.21.15.63:0/2324104161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 0x5600788ed7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.751+0000 7f2f54a00700 1 -- 172.21.15.63:0/2324104161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600788d7240 con 0x5600788e8670 2024-05-27T07:56:43.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.751+0000 7f2f54a00700 1 --2- 172.21.15.63:0/2324104161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600788e8670 0x5600788fe490 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x5600788ea800 tx=0x7f2f48009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80350531b0fe23dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:43.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.752+0000 7f2f44c00700 1 -- 172.21.15.63:0/2324104161 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48030020 con 0x5600788e8670 2024-05-27T07:56:43.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.752+0000 7f2f44c00700 1 -- 172.21.15.63:0/2324104161 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f48027040 con 0x5600788e8670 2024-05-27T07:56:43.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.752+0000 7f2f44c00700 1 -- 172.21.15.63:0/2324104161 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48026070 con 0x5600788e8670 2024-05-27T07:56:43.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.753+0000 7f2f655f1b80 1 -- 172.21.15.63:0/2324104161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600788e8670 msgr2=0x5600788fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:43.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.753+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/2324104161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600788e8670 0x5600788fe490 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x5600788ea800 tx=0x7f2f48009620 comp rx=0 tx=0).stop 2024-05-27T07:56:43.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.753+0000 7f2f655f1b80 1 -- 172.21.15.63:0/2324104161 shutdown_connections 2024-05-27T07:56:43.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.753+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/2324104161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600788e8670 0x5600788fe490 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.753+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/2324104161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560078760e40 0x5600788f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.753+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/2324104161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56007884c9f0 0x5600788ed7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.753+0000 7f2f655f1b80 1 -- 172.21.15.63:0/2324104161 >> 172.21.15.63:0/2324104161 conn(0x5600787fa390 msgr2=0x5600788e9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:43.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.753+0000 7f2f655f1b80 1 -- 172.21.15.63:0/2324104161 shutdown_connections 2024-05-27T07:56:43.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.753+0000 7f2f655f1b80 1 -- 172.21.15.63:0/2324104161 wait complete. 2024-05-27T07:56:43.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.754+0000 7f2f655f1b80 1 -- start start 2024-05-27T07:56:43.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.754+0000 7f2f655f1b80 1 Processor -- start 2024-05-27T07:56:43.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788fad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56007884c9f0 0x5600788ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f655f1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600788e8670 0x5600788ecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501ac870 con 0x560078760e40 2024-05-27T07:56:43.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501adca0 con 0x5600788e8670 2024-05-27T07:56:43.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f655f1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501acf00 con 0x56007884c9f0 2024-05-27T07:56:43.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f655f1b80 10 client.0 _ll_register_callbacks cb 0x5600787a7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:43.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f4fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788fad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f4fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788fad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40524/0 (socket says 172.21.15.63:40524) 2024-05-27T07:56:43.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f4fe00700 1 -- 172.21.15.63:0/3403174678 learned_addr learned my addr 172.21.15.63:0/3403174678 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:43.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.755+0000 7f2f4f400700 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56007884c9f0 0x5600788ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.756+0000 7f2f54a00700 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600788e8670 0x5600788ecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.756+0000 7f2f4fe00700 1 -- 172.21.15.63:0/3403174678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56007884c9f0 msgr2=0x5600788ec7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:43.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.756+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56007884c9f0 0x5600788ec7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.756+0000 7f2f4fe00700 1 -- 172.21.15.63:0/3403174678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600788e8670 msgr2=0x5600788ecd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:43.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.756+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600788e8670 0x5600788ecd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.756+0000 7f2f4fe00700 1 -- 172.21.15.63:0/3403174678 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600788f2de0 con 0x560078760e40 2024-05-27T07:56:43.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.756+0000 7f2f4fe00700 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788fad90 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009660 tx=0x7f2f40009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809e6451bb210388 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:43.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.756+0000 7f2f3a000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:43.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.757+0000 7f2f3a000700 1 -- 172.21.15.63:0/3403174678 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f400cd070 con 0x560078760e40 2024-05-27T07:56:43.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.757+0000 7f2f3a000700 1 -- 172.21.15.63:0/3403174678 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f2f400a9a00 con 0x560078760e40 2024-05-27T07:56:43.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.757+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3403174678 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f48027040 con 0x560078760e40 2024-05-27T07:56:43.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.757+0000 7f2f3a000700 1 -- 172.21.15.63:0/3403174678 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2f400a9f10 con 0x560078760e40 2024-05-27T07:56:43.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.757+0000 7f2f655f1b80 10 client.19860 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:43.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.757+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3403174678 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5600788f2de0 con 0x560078760e40 2024-05-27T07:56:43.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.758+0000 7f2f3a000700 1 -- 172.21.15.63:0/3403174678 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f48027040 con 0x560078760e40 2024-05-27T07:56:43.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.758+0000 7f2f3a000700 1 -- 172.21.15.63:0/3403174678 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f2f400afce0 con 0x560078760e40 2024-05-27T07:56:43.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.758+0000 7f2f655f1b80 10 client.19860 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:43.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.758+0000 7f2f655f1b80 10 client.19860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:43.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.758+0000 7f2f655f1b80 20 client.19860 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:43.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.758+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56007896cfc0 0x56007896f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:43.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.758+0000 7f2f655f1b80 4 client.19860 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:56:43.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.758+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3403174678 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560078809570 con 0x56007896cfc0 2024-05-27T07:56:43.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.759+0000 7f2f4f400700 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56007896cfc0 0x56007896f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:43.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.760+0000 7f2f4f400700 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56007896cfc0 0x56007896f3e0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f045229cd4b620fa server_cookie=751e2bd7cb29f4d1 in_seq=0 out_seq=0 2024-05-27T07:56:43.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.760+0000 7f2f3a000700 10 client.19860 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:43.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.760+0000 7f2f3a000700 1 -- 172.21.15.63:0/3403174678 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560078809570 con 0x56007896cfc0 2024-05-27T07:56:43.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.760+0000 7f2f3a000700 10 client.19860 handle_command_reply: tid=0 2024-05-27T07:56:43.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.760+0000 7f2f655f1b80 1 client.19860 shutdown 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3403174678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 msgr2=0x5600788fad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788fad90 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009660 tx=0x7f2f40009910 comp rx=0 tx=0).stop 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3403174678 shutdown_connections 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56007896cfc0 0x56007896f3e0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600788e8670 0x5600788ecd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560078760e40 0x5600788fad90 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f655f1b80 1 --2- 172.21.15.63:0/3403174678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56007884c9f0 0x5600788ec7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3403174678 >> 172.21.15.63:0/3403174678 conn(0x5600787fa390 msgr2=0x5600788f1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f3a000700 0 client.19860 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3403174678 shutdown_connections 2024-05-27T07:56:43.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.761+0000 7f2f655f1b80 1 -- 172.21.15.63:0/3403174678 wait complete. 2024-05-27T07:56:43.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.762+0000 7f2f655f1b80 20 client.19860 trim_cache size 0 max 16384 2024-05-27T07:56:43.764 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:56:43.764 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:56:43.764 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:56:43.764 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:56:43.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.762+0000 7f2f56a00700 1 -- 172.21.15.63:0/1353512588 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f340b29f0 msgr2=0x7f2f340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:43.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.762+0000 7f2f56a00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f340b29f0 0x7f2f340b4f00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2f4801e000 tx=0x7f2f4801f000 comp rx=0 tx=0).stop 2024-05-27T07:56:43.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.762+0000 7f2f56a00700 1 -- 172.21.15.63:0/1353512588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f50105450 msgr2=0x7f2f5019cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:43.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.762+0000 7f2f56a00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f50105450 0x7f2f5019cf00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f2f40005910 tx=0x7f2f400047b0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.763+0000 7f2f56a00700 1 -- 172.21.15.63:0/1353512588 shutdown_connections 2024-05-27T07:56:43.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.763+0000 7f2f56a00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2f340b29f0 0x7f2f340b4f00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.763+0000 7f2f56a00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f5010e8c0 0x7f2f501a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.763+0000 7f2f56a00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f50105450 0x7f2f5019cf00 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.763+0000 7f2f56a00700 1 --2- 172.21.15.63:0/1353512588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50105da0 0x7f2f5019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:43.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.763+0000 7f2f56a00700 1 -- 172.21.15.63:0/1353512588 >> 172.21.15.63:0/1353512588 conn(0x7f2f500fedb0 msgr2=0x7f2f500ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:43.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.763+0000 7f2f56a00700 1 -- 172.21.15.63:0/1353512588 shutdown_connections 2024-05-27T07:56:43.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:43.763+0000 7f2f56a00700 1 -- 172.21.15.63:0/1353512588 wait complete. 2024-05-27T07:56:44.801 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:56:44.801 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:56:45.306 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:45.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:45 smithi086 ceph-mon[34687]: pgmap v732: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:56:45.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:45 smithi149 ceph-mon[35408]: pgmap v732: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:56:45.464 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:45 smithi063 ceph-mon[31613]: pgmap v732: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:56:47.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:47 smithi086 ceph-mon[34687]: pgmap v733: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:56:47.464 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:47 smithi063 ceph-mon[31613]: pgmap v733: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:56:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:47 smithi149 ceph-mon[35408]: pgmap v733: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:56:47.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.839+0000 7ff8d0800700 1 -- 172.21.15.63:0/2987406833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8c81034b0 msgr2=0x7ff8c81038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:47.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.839+0000 7ff8d0800700 1 --2- 172.21.15.63:0/2987406833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8c81034b0 0x7ff8c81038c0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0099c0 tx=0x7ff8bc009cd0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.839+0000 7ff8d0800700 1 -- 172.21.15.63:0/2987406833 shutdown_connections 2024-05-27T07:56:47.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.839+0000 7ff8d0800700 1 --2- 172.21.15.63:0/2987406833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c8108730 0x7ff8c8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.839+0000 7ff8d0800700 1 --2- 172.21.15.63:0/2987406833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8c81034b0 0x7ff8c81038c0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.839+0000 7ff8d0800700 1 --2- 172.21.15.63:0/2987406833 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c8107de0 0x7ff8c81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.839+0000 7ff8d0800700 1 -- 172.21.15.63:0/2987406833 >> 172.21.15.63:0/2987406833 conn(0x7ff8c80fedb0 msgr2=0x7ff8c81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:47.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.839+0000 7ff8d0800700 1 -- 172.21.15.63:0/2987406833 shutdown_connections 2024-05-27T07:56:47.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.839+0000 7ff8d0800700 1 -- 172.21.15.63:0/2987406833 wait complete. 2024-05-27T07:56:47.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.841+0000 7ff8d0800700 1 Processor -- start 2024-05-27T07:56:47.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.841+0000 7ff8d0800700 1 -- start start 2024-05-27T07:56:47.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.841+0000 7ff8d0800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c81034b0 0x7ff8c819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:47.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.841+0000 7ff8d0800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8c8107de0 0x7ff8c819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:47.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.841+0000 7ff8d0800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c8108730 0x7ff8c81a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:47.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.842+0000 7ff8d0800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c806c2b0 con 0x7ff8c8107de0 2024-05-27T07:56:47.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.842+0000 7ff8d0800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c806c130 con 0x7ff8c81034b0 2024-05-27T07:56:47.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.842+0000 7ff8d0800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c806c430 con 0x7ff8c8108730 2024-05-27T07:56:47.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.842+0000 7ff8cd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8c8107de0 0x7ff8c819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:47.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.842+0000 7ff8cd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8c8107de0 0x7ff8c819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40538/0 (socket says 172.21.15.63:40538) 2024-05-27T07:56:47.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.842+0000 7ff8cd400700 1 -- 172.21.15.63:0/3751310301 learned_addr learned my addr 172.21.15.63:0/3751310301 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:47.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.842+0000 7ff8cde00700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c81034b0 0x7ff8c819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:47.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.842+0000 7ff8ce800700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c8108730 0x7ff8c81a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:47.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.843+0000 7ff8cd400700 1 -- 172.21.15.63:0/3751310301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c8108730 msgr2=0x7ff8c81a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:47.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.843+0000 7ff8cd400700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c8108730 0x7ff8c81a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.843+0000 7ff8cd400700 1 -- 172.21.15.63:0/3751310301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c81034b0 msgr2=0x7ff8c819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:47.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.843+0000 7ff8cd400700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c81034b0 0x7ff8c819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.843+0000 7ff8cd400700 1 -- 172.21.15.63:0/3751310301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c81a1ea0 con 0x7ff8c8107de0 2024-05-27T07:56:47.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.843+0000 7ff8cd400700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8c8107de0 0x7ff8c819d570 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800d980 tx=0x7ff8b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:47.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.844+0000 7ff8c2a00700 1 -- 172.21.15.63:0/3751310301 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8017070 con 0x7ff8c8107de0 2024-05-27T07:56:47.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.844+0000 7ff8c2a00700 1 -- 172.21.15.63:0/3751310301 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8b800fd20 con 0x7ff8c8107de0 2024-05-27T07:56:47.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.844+0000 7ff8d0800700 1 -- 172.21.15.63:0/3751310301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c81a2190 con 0x7ff8c8107de0 2024-05-27T07:56:47.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.844+0000 7ff8c2a00700 1 -- 172.21.15.63:0/3751310301 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800e3b0 con 0x7ff8c8107de0 2024-05-27T07:56:47.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.844+0000 7ff8d0800700 1 -- 172.21.15.63:0/3751310301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c81a9ab0 con 0x7ff8c8107de0 2024-05-27T07:56:47.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.845+0000 7ff8d0800700 1 -- 172.21.15.63:0/3751310301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c80512b0 con 0x7ff8c8107de0 2024-05-27T07:56:47.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.850+0000 7ff8c2a00700 1 -- 172.21.15.63:0/3751310301 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff8b800f890 con 0x7ff8c8107de0 2024-05-27T07:56:47.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.851+0000 7ff8c2a00700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8b00b29f0 0x7ff8b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:47.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.852+0000 7ff8c2a00700 1 -- 172.21.15.63:0/3751310301 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7ff8b80e24d0 con 0x7ff8c8107de0 2024-05-27T07:56:47.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.852+0000 7ff8cde00700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8b00b29f0 0x7ff8b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:47.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.853+0000 7ff8cde00700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8b00b29f0 0x7ff8b00b4f00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc000c00 tx=0x7ff8bc005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:47.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:47.854+0000 7ff8c2a00700 1 -- 172.21.15.63:0/3751310301 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8b80a9a90 con 0x7ff8c8107de0 2024-05-27T07:56:48.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.202+0000 7ff8d0800700 1 -- 172.21.15.63:0/3751310301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8c81a9c80 con 0x7ff8c8107de0 2024-05-27T07:56:48.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.203+0000 7ff8c2a00700 1 -- 172.21.15.63:0/3751310301 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7ff8b80a9390 con 0x7ff8c8107de0 2024-05-27T07:56:48.206 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:56:48.206 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:56:48.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.208+0000 7ff8d0800700 1 -- 172.21.15.63:0/3751310301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8b00b29f0 msgr2=0x7ff8b00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.208+0000 7ff8d0800700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8b00b29f0 0x7ff8b00b4f00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc000c00 tx=0x7ff8bc005fb0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.208+0000 7ff8d0800700 1 -- 172.21.15.63:0/3751310301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8c8107de0 msgr2=0x7ff8c819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.208+0000 7ff8d0800700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8c8107de0 0x7ff8c819d570 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800d980 tx=0x7ff8b800dc90 comp rx=0 tx=0).stop 2024-05-27T07:56:48.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.208+0000 7ff8d0800700 1 -- 172.21.15.63:0/3751310301 shutdown_connections 2024-05-27T07:56:48.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.208+0000 7ff8d0800700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8b00b29f0 0x7ff8b00b4f00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.208+0000 7ff8d0800700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8c8108730 0x7ff8c81a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.208+0000 7ff8d0800700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8c81034b0 0x7ff8c819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.208+0000 7ff8d0800700 1 --2- 172.21.15.63:0/3751310301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8c8107de0 0x7ff8c819d570 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.208+0000 7ff8d0800700 1 -- 172.21.15.63:0/3751310301 >> 172.21.15.63:0/3751310301 conn(0x7ff8c80fedb0 msgr2=0x7ff8c81096f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:48.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.209+0000 7ff8d0800700 1 -- 172.21.15.63:0/3751310301 shutdown_connections 2024-05-27T07:56:48.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:48.209+0000 7ff8d0800700 1 -- 172.21.15.63:0/3751310301 wait complete. 2024-05-27T07:56:48.212 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:56:49.197 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:56:49.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:49 smithi086 ceph-mon[34687]: pgmap v734: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:56:49.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:49 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3751310301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:56:49.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:49 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:56:49.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:49 smithi149 ceph-mon[35408]: pgmap v734: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:56:49.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:49 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3751310301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:56:49.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:49 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:56:49.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:49 smithi063 ceph-mon[31613]: pgmap v734: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T07:56:49.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:49 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3751310301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:56:49.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:49 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:56:49.730 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:51.192 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:51 smithi063 ceph-mon[31613]: pgmap v735: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:56:51.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:51 smithi086 ceph-mon[34687]: pgmap v735: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:56:51.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:51 smithi149 ceph-mon[35408]: pgmap v735: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:56:52.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.509+0000 7f0d84a00700 1 -- 172.21.15.63:0/3158148529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d80106eb0 msgr2=0x7f0d80113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.509+0000 7f0d84a00700 1 --2- 172.21.15.63:0/3158148529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d80106eb0 0x7f0d80113340 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800b0a0 tx=0x7f0d7800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.510+0000 7f0d84a00700 1 -- 172.21.15.63:0/3158148529 shutdown_connections 2024-05-27T07:56:52.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.510+0000 7f0d84a00700 1 --2- 172.21.15.63:0/3158148529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d80106eb0 0x7f0d80113340 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.510+0000 7f0d84a00700 1 --2- 172.21.15.63:0/3158148529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8010c5a0 0x7f0d8010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.510+0000 7f0d84a00700 1 --2- 172.21.15.63:0/3158148529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d80106560 0x7f0d80106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.510+0000 7f0d84a00700 1 -- 172.21.15.63:0/3158148529 >> 172.21.15.63:0/3158148529 conn(0x7f0d80102080 msgr2=0x7f0d801044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.510+0000 7f0d84a00700 1 -- 172.21.15.63:0/3158148529 shutdown_connections 2024-05-27T07:56:52.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.510+0000 7f0d84a00700 1 -- 172.21.15.63:0/3158148529 wait complete. 2024-05-27T07:56:52.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.512+0000 7f0d84a00700 1 Processor -- start 2024-05-27T07:56:52.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.512+0000 7f0d84a00700 1 -- start start 2024-05-27T07:56:52.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.512+0000 7f0d84a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d80106560 0x7f0d8019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.512+0000 7f0d84a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d80106eb0 0x7f0d8019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.513+0000 7f0d84a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8010c5a0 0x7f0d801a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.513+0000 7f0d84a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801165d0 con 0x7f0d80106560 2024-05-27T07:56:52.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.513+0000 7f0d84a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d80116450 con 0x7f0d8010c5a0 2024-05-27T07:56:52.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.513+0000 7f0d84a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0d80116750 con 0x7f0d80106eb0 2024-05-27T07:56:52.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.513+0000 7f0d7de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d80106560 0x7f0d8019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.513+0000 7f0d7de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d80106560 0x7f0d8019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38474/0 (socket says 172.21.15.63:38474) 2024-05-27T07:56:52.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.513+0000 7f0d7de00700 1 -- 172.21.15.63:0/1085720463 learned_addr learned my addr 172.21.15.63:0/1085720463 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:52.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.513+0000 7f0d7e800700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8010c5a0 0x7f0d801a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.513+0000 7f0d7d400700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d80106eb0 0x7f0d8019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.514+0000 7f0d7de00700 1 -- 172.21.15.63:0/1085720463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d80106eb0 msgr2=0x7f0d8019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.514+0000 7f0d7de00700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d80106eb0 0x7f0d8019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.514+0000 7f0d7de00700 1 -- 172.21.15.63:0/1085720463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8010c5a0 msgr2=0x7f0d801a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.514+0000 7f0d7de00700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8010c5a0 0x7f0d801a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.514+0000 7f0d7de00700 1 -- 172.21.15.63:0/1085720463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d801a1d90 con 0x7f0d80106560 2024-05-27T07:56:52.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.514+0000 7f0d7de00700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d80106560 0x7f0d8019cec0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000e970 tx=0x7f0d7000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.514+0000 7f0d76a00700 1 -- 172.21.15.63:0/1085720463 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70004a30 con 0x7f0d80106560 2024-05-27T07:56:52.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.515+0000 7f0d76a00700 1 -- 172.21.15.63:0/1085720463 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d7000cc70 con 0x7f0d80106560 2024-05-27T07:56:52.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.515+0000 7f0d84a00700 1 -- 172.21.15.63:0/1085720463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d801a2020 con 0x7f0d80106560 2024-05-27T07:56:52.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.515+0000 7f0d76a00700 1 -- 172.21.15.63:0/1085720463 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70005520 con 0x7f0d80106560 2024-05-27T07:56:52.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.515+0000 7f0d84a00700 1 -- 172.21.15.63:0/1085720463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d801a99a0 con 0x7f0d80106560 2024-05-27T07:56:52.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.520+0000 7f0d93506b80 1 Processor -- start 2024-05-27T07:56:52.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.520+0000 7f0d76a00700 1 -- 172.21.15.63:0/1085720463 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0d70021440 con 0x7f0d80106560 2024-05-27T07:56:52.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.520+0000 7f0d93506b80 1 -- start start 2024-05-27T07:56:52.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 0x55faac3af330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3af870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac3afe70 0x55faac3bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d93506b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55faac2dde40 con 0x55faac230e40 2024-05-27T07:56:52.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d93506b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55faac2cd9e0 con 0x55faac3afe70 2024-05-27T07:56:52.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d93506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55faac2d1a20 con 0x55faac31c9f0 2024-05-27T07:56:52.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d7de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d7de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38486/0 (socket says 172.21.15.63:38486) 2024-05-27T07:56:52.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d7de00700 1 -- 172.21.15.63:0/3847707718 learned_addr learned my addr 172.21.15.63:0/3847707718 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:52.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d7e800700 1 --2- 172.21.15.63:0/3847707718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 0x55faac3af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.521+0000 7f0d7d400700 1 --2- 172.21.15.63:0/3847707718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac3afe70 0x55faac3bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.522+0000 7f0d7de00700 1 -- 172.21.15.63:0/3847707718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 msgr2=0x55faac3af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.522+0000 7f0d7de00700 1 --2- 172.21.15.63:0/3847707718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 0x55faac3af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.522+0000 7f0d7de00700 1 -- 172.21.15.63:0/3847707718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac3afe70 msgr2=0x55faac3bc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.522+0000 7f0d7de00700 1 --2- 172.21.15.63:0/3847707718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac3afe70 0x55faac3bc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.522+0000 7f0d7de00700 1 -- 172.21.15.63:0/3847707718 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faac296420 con 0x55faac230e40 2024-05-27T07:56:52.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.522+0000 7f0d7e800700 1 --2- 172.21.15.63:0/3847707718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 0x55faac3af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:52.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.522+0000 7f0d76a00700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0d640b29f0 0x7f0d640b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.522+0000 7f0d76a00700 1 -- 172.21.15.63:0/1085720463 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f0d700e2be0 con 0x7f0d80106560 2024-05-27T07:56:52.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.522+0000 7f0d7e800700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0d640b29f0 0x7f0d640b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.523+0000 7f0d7de00700 1 --2- 172.21.15.63:0/3847707718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3af870 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f0d700dfe50 tx=0x7f0d700e6bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b82466cddd4bc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.523+0000 7f0d6b400700 1 -- 172.21.15.63:0/3847707718 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700a5960 con 0x55faac230e40 2024-05-27T07:56:52.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.523+0000 7f0d6b400700 1 -- 172.21.15.63:0/3847707718 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d700a95b0 con 0x55faac230e40 2024-05-27T07:56:52.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.523+0000 7f0d6b400700 1 -- 172.21.15.63:0/3847707718 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700a9210 con 0x55faac230e40 2024-05-27T07:56:52.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.524+0000 7f0d93506b80 1 -- 172.21.15.63:0/3847707718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 msgr2=0x55faac3af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.524+0000 7f0d93506b80 1 --2- 172.21.15.63:0/3847707718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3af870 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f0d700dfe50 tx=0x7f0d700e6bc0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.524+0000 7f0d7e800700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0d640b29f0 0x7f0d640b4f00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55faac3b1ff0 tx=0x7f0d78006210 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.524+0000 7f0d93506b80 1 -- 172.21.15.63:0/3847707718 shutdown_connections 2024-05-27T07:56:52.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.524+0000 7f0d93506b80 1 --2- 172.21.15.63:0/3847707718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac3afe70 0x55faac3bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.524+0000 7f0d93506b80 1 --2- 172.21.15.63:0/3847707718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 0x55faac3af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.524+0000 7f0d93506b80 1 --2- 172.21.15.63:0/3847707718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3af870 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.524+0000 7f0d93506b80 1 -- 172.21.15.63:0/3847707718 >> 172.21.15.63:0/3847707718 conn(0x55faac2ca390 msgr2=0x55faac3ab4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.525+0000 7f0d93506b80 1 -- 172.21.15.63:0/3847707718 shutdown_connections 2024-05-27T07:56:52.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.525+0000 7f0d93506b80 1 -- 172.21.15.63:0/3847707718 wait complete. 2024-05-27T07:56:52.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.525+0000 7f0d93506b80 1 -- start start 2024-05-27T07:56:52.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.526+0000 7f0d93506b80 1 Processor -- start 2024-05-27T07:56:52.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.526+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 0x55faac3b9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.526+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 0x55faac3b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.526+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac3afe70 0x55faac3ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.526+0000 7f0d93506b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55faac1d3fa0 con 0x55faac3afe70 2024-05-27T07:56:52.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.526+0000 7f0d93506b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55faac26e760 con 0x55faac230e40 2024-05-27T07:56:52.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.526+0000 7f0d93506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55faac3bcb20 con 0x55faac31c9f0 2024-05-27T07:56:52.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.526+0000 7f0d93506b80 10 client.0 _ll_register_callbacks cb 0x55faac277c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:52.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.527+0000 7f0d7e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac3afe70 0x55faac3ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.527+0000 7f0d7e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac3afe70 0x55faac3ba1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38496/0 (socket says 172.21.15.63:38496) 2024-05-27T07:56:52.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.527+0000 7f0d7e800700 1 -- 172.21.15.63:0/4041609102 learned_addr learned my addr 172.21.15.63:0/4041609102 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:52.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.527+0000 7f0d7de00700 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 0x55faac3b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.527+0000 7f0d7d400700 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 0x55faac3b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.527+0000 7f0d7de00700 1 -- 172.21.15.63:0/4041609102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 msgr2=0x55faac3b9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.527+0000 7f0d7de00700 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 0x55faac3b9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.527+0000 7f0d7de00700 1 -- 172.21.15.63:0/4041609102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac3afe70 msgr2=0x55faac3ba1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.527+0000 7f0d7de00700 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac3afe70 0x55faac3ba1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.527+0000 7f0d7de00700 1 -- 172.21.15.63:0/4041609102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faac21d8d0 con 0x55faac230e40 2024-05-27T07:56:52.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.528+0000 7f0d7d400700 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 0x55faac3b9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:52.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.528+0000 7f0d7de00700 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 0x55faac3b9740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55faac3aca20 tx=0x7f0d70004e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6320c8666e38c85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.528+0000 7f0d5fe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.528+0000 7f0d5fe00700 1 -- 172.21.15.63:0/4041609102 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700a93f0 con 0x55faac230e40 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.528+0000 7f0d93506b80 1 -- 172.21.15.63:0/4041609102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d700a95b0 con 0x55faac230e40 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.529+0000 7f0d5fe00700 1 -- 172.21.15.63:0/4041609102 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f0d700a9d00 con 0x55faac230e40 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.529+0000 7f0d5fe00700 1 -- 172.21.15.63:0/4041609102 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d700a9870 con 0x55faac230e40 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.529+0000 7f0d93506b80 10 client.39092 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.529+0000 7f0d93506b80 1 -- 172.21.15.63:0/4041609102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55faac29e060 con 0x55faac230e40 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.529+0000 7f0d5fe00700 1 -- 172.21.15.63:0/4041609102 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d700a95b0 con 0x55faac230e40 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.530+0000 7f0d5fe00700 1 -- 172.21.15.63:0/4041609102 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0d700b2470 con 0x55faac230e40 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.530+0000 7f0d93506b80 10 client.39092 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.530+0000 7f0d93506b80 10 client.39092 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:56:52.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.530+0000 7f0d93506b80 20 client.39092 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:52.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.530+0000 7f0d93506b80 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55faac3c9920 0x55faac3cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.530+0000 7f0d93506b80 4 client.39092 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:52.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.530+0000 7f0d93506b80 1 -- 172.21.15.63:0/4041609102 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faac2dfd90 con 0x55faac3c9920 2024-05-27T07:56:52.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.530+0000 7f0d7d400700 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55faac3c9920 0x55faac3cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.531+0000 7f0d7d400700 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55faac3c9920 0x55faac3cbd40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bf98a6167f39baf server_cookie=75880c3197929b9c in_seq=0 out_seq=0 2024-05-27T07:56:52.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.531+0000 7f0d5fe00700 10 client.39092 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:56:52.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.532+0000 7f0d5fe00700 1 -- 172.21.15.63:0/4041609102 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55faac2dfd90 con 0x55faac3c9920 2024-05-27T07:56:52.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.532+0000 7f0d5fe00700 10 client.39092 handle_command_reply: tid=0 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.532+0000 7f0d93506b80 1 client.39092 shutdown 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d93506b80 1 -- 172.21.15.63:0/4041609102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 msgr2=0x55faac3b9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d93506b80 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 0x55faac3b9740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55faac3aca20 tx=0x7f0d70004e00 comp rx=0 tx=0).stop 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d93506b80 1 -- 172.21.15.63:0/4041609102 shutdown_connections 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d93506b80 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55faac3c9920 0x55faac3cbd40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d93506b80 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac3afe70 0x55faac3ba1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d93506b80 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 0x55faac3b9740 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d93506b80 1 --2- 172.21.15.63:0/4041609102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac31c9f0 0x55faac3b9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d93506b80 1 -- 172.21.15.63:0/4041609102 >> 172.21.15.63:0/4041609102 conn(0x55faac2ca390 msgr2=0x55faac3af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d5fe00700 0 client.39092 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d93506b80 1 -- 172.21.15.63:0/4041609102 shutdown_connections 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.533+0000 7f0d93506b80 1 -- 172.21.15.63:0/4041609102 wait complete. 2024-05-27T07:56:52.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.534+0000 7f0d93506b80 20 client.39092 trim_cache size 0 max 16384 2024-05-27T07:56:52.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.557+0000 7f0d93506b80 1 Processor -- start 2024-05-27T07:56:52.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.557+0000 7f0d93506b80 1 -- start start 2024-05-27T07:56:52.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.558+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 0x55faac3b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.558+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac31c9f0 0x55faac3b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.558+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac3afe70 0x55faac3c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.558+0000 7f0d93506b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801b4fe0 con 0x55faac31c9f0 2024-05-27T07:56:52.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.558+0000 7f0d93506b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801b2870 con 0x55faac230e40 2024-05-27T07:56:52.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.558+0000 7f0d93506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801b06e0 con 0x55faac3afe70 2024-05-27T07:56:52.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.558+0000 7f0d7d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac31c9f0 0x55faac3b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.558+0000 7f0d7d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac31c9f0 0x55faac3b4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38508/0 (socket says 172.21.15.63:38508) 2024-05-27T07:56:52.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.558+0000 7f0d7d400700 1 -- 172.21.15.63:0/3672798665 learned_addr learned my addr 172.21.15.63:0/3672798665 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:52.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.559+0000 7f0d7de00700 1 --2- 172.21.15.63:0/3672798665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 0x55faac3b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.559+0000 7f0d7e800700 1 --2- 172.21.15.63:0/3672798665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac3afe70 0x55faac3c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.559+0000 7f0d7d400700 1 -- 172.21.15.63:0/3672798665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac3afe70 msgr2=0x55faac3c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.559+0000 7f0d7d400700 1 --2- 172.21.15.63:0/3672798665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac3afe70 0x55faac3c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.559+0000 7f0d7d400700 1 -- 172.21.15.63:0/3672798665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 msgr2=0x55faac3b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.559+0000 7f0d7d400700 1 --2- 172.21.15.63:0/3672798665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 0x55faac3b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.559+0000 7f0d7d400700 1 -- 172.21.15.63:0/3672798665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faac3bc750 con 0x55faac31c9f0 2024-05-27T07:56:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.559+0000 7f0d7d400700 1 --2- 172.21.15.63:0/3672798665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac31c9f0 0x55faac3b4ef0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c005950 tx=0x7f0d6c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5435b1105121fd3f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.560+0000 7f0d6aa00700 1 -- 172.21.15.63:0/3672798665 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0039e0 con 0x55faac31c9f0 2024-05-27T07:56:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.560+0000 7f0d6aa00700 1 -- 172.21.15.63:0/3672798665 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d6c004830 con 0x55faac31c9f0 2024-05-27T07:56:52.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.560+0000 7f0d6aa00700 1 -- 172.21.15.63:0/3672798665 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0054e0 con 0x55faac31c9f0 2024-05-27T07:56:52.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.561+0000 7f0d93506b80 1 -- 172.21.15.63:0/3672798665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac31c9f0 msgr2=0x55faac3b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.561+0000 7f0d93506b80 1 --2- 172.21.15.63:0/3672798665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac31c9f0 0x55faac3b4ef0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c005950 tx=0x7f0d6c00b410 comp rx=0 tx=0).stop 2024-05-27T07:56:52.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.561+0000 7f0d93506b80 1 -- 172.21.15.63:0/3672798665 shutdown_connections 2024-05-27T07:56:52.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.561+0000 7f0d93506b80 1 --2- 172.21.15.63:0/3672798665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac3afe70 0x55faac3c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.561+0000 7f0d93506b80 1 --2- 172.21.15.63:0/3672798665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac230e40 0x55faac3b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.561+0000 7f0d93506b80 1 --2- 172.21.15.63:0/3672798665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac31c9f0 0x55faac3b4ef0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.561+0000 7f0d93506b80 1 -- 172.21.15.63:0/3672798665 >> 172.21.15.63:0/3672798665 conn(0x55faac2ca390 msgr2=0x55faac3badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.561+0000 7f0d93506b80 1 -- 172.21.15.63:0/3672798665 shutdown_connections 2024-05-27T07:56:52.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.561+0000 7f0d93506b80 1 -- 172.21.15.63:0/3672798665 wait complete. 2024-05-27T07:56:52.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.562+0000 7f0d93506b80 1 -- start start 2024-05-27T07:56:52.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.562+0000 7f0d93506b80 1 Processor -- start 2024-05-27T07:56:52.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac31c9f0 0x55faac3cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d93506b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac3afe70 0x55faac3ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d93506b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801b0d30 con 0x55faac230e40 2024-05-27T07:56:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d93506b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801b21a0 con 0x55faac31c9f0 2024-05-27T07:56:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d93506b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801b1400 con 0x55faac3afe70 2024-05-27T07:56:52.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d93506b80 10 client.0 _ll_register_callbacks cb 0x55faac277c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d7de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d7de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38518/0 (socket says 172.21.15.63:38518) 2024-05-27T07:56:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d7de00700 1 -- 172.21.15.63:0/624344526 learned_addr learned my addr 172.21.15.63:0/624344526 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d7d400700 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac31c9f0 0x55faac3cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.563+0000 7f0d7e800700 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac3afe70 0x55faac3ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.564+0000 7f0d7de00700 1 -- 172.21.15.63:0/624344526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac3afe70 msgr2=0x55faac3ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.564+0000 7f0d7de00700 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac3afe70 0x55faac3ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.564+0000 7f0d7de00700 1 -- 172.21.15.63:0/624344526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac31c9f0 msgr2=0x55faac3cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.564+0000 7f0d7de00700 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac31c9f0 0x55faac3cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.564+0000 7f0d7de00700 1 -- 172.21.15.63:0/624344526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faac3caa60 con 0x55faac230e40 2024-05-27T07:56:52.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.564+0000 7f0d7d400700 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac31c9f0 0x55faac3cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:52.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.564+0000 7f0d7de00700 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3ced80 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f0d70004800 tx=0x7f0d700c42a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff9f9fdc930f887 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.564+0000 7f0d5fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:52.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.564+0000 7f0d5fe00700 1 -- 172.21.15.63:0/624344526 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700c9070 con 0x55faac230e40 2024-05-27T07:56:52.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.565+0000 7f0d5fe00700 1 -- 172.21.15.63:0/624344526 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f0d700a9d00 con 0x55faac230e40 2024-05-27T07:56:52.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.565+0000 7f0d93506b80 1 -- 172.21.15.63:0/624344526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6c004830 con 0x55faac230e40 2024-05-27T07:56:52.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.565+0000 7f0d5fe00700 1 -- 172.21.15.63:0/624344526 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d700a93f0 con 0x55faac230e40 2024-05-27T07:56:52.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.565+0000 7f0d93506b80 10 client.19884 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:52.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.565+0000 7f0d93506b80 1 -- 172.21.15.63:0/624344526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55faac433b00 con 0x55faac230e40 2024-05-27T07:56:52.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.565+0000 7f0d5fe00700 1 -- 172.21.15.63:0/624344526 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d6c004830 con 0x55faac230e40 2024-05-27T07:56:52.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.565+0000 7f0d5fe00700 1 -- 172.21.15.63:0/624344526 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f0d700a8d00 con 0x55faac230e40 2024-05-27T07:56:52.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.566+0000 7f0d93506b80 10 client.19884 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:52.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.566+0000 7f0d93506b80 10 client.19884 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:56:52.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.566+0000 7f0d93506b80 20 client.19884 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:52.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.566+0000 7f0d93506b80 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55faac433ff0 0x55faac436410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.566+0000 7f0d93506b80 4 client.19884 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:56:52.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.566+0000 7f0d93506b80 1 -- 172.21.15.63:0/624344526 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55faac2d9570 con 0x55faac433ff0 2024-05-27T07:56:52.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.566+0000 7f0d7d400700 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55faac433ff0 0x55faac436410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.568+0000 7f0d7d400700 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55faac433ff0 0x55faac436410 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f85621ef5c472e99 server_cookie=14614b782dbe8f3e in_seq=0 out_seq=0 2024-05-27T07:56:52.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.568+0000 7f0d5fe00700 10 client.19884 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.568+0000 7f0d5fe00700 1 -- 172.21.15.63:0/624344526 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55faac2d9570 con 0x55faac433ff0 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.568+0000 7f0d5fe00700 10 client.19884 handle_command_reply: tid=0 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.568+0000 7f0d93506b80 1 client.19884 shutdown 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 1 -- 172.21.15.63:0/624344526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 msgr2=0x55faac3ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3ced80 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f0d70004800 tx=0x7f0d700c42a0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 1 -- 172.21.15.63:0/624344526 shutdown_connections 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55faac3afe70 0x55faac3ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55faac433ff0 0x55faac436410 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55faac230e40 0x55faac3ced80 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 1 --2- 172.21.15.63:0/624344526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55faac31c9f0 0x55faac3cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 1 -- 172.21.15.63:0/624344526 >> 172.21.15.63:0/624344526 conn(0x55faac2ca390 msgr2=0x55faac3b9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d5fe00700 0 client.19884 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:56:52.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 1 -- 172.21.15.63:0/624344526 shutdown_connections 2024-05-27T07:56:52.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 1 -- 172.21.15.63:0/624344526 wait complete. 2024-05-27T07:56:52.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.569+0000 7f0d93506b80 20 client.19884 trim_cache size 0 max 16384 2024-05-27T07:56:52.572 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:56:52.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.570+0000 7f0d84a00700 1 -- 172.21.15.63:0/1085720463 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0d640b29f0 msgr2=0x7f0d640b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.570+0000 7f0d84a00700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0d640b29f0 0x7f0d640b4f00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55faac3b1ff0 tx=0x7f0d78006210 comp rx=0 tx=0).stop 2024-05-27T07:56:52.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.570+0000 7f0d84a00700 1 -- 172.21.15.63:0/1085720463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d80106560 msgr2=0x7f0d8019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.570+0000 7f0d84a00700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d80106560 0x7f0d8019cec0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000e970 tx=0x7f0d7000ec80 comp rx=0 tx=0).stop 2024-05-27T07:56:52.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.570+0000 7f0d84a00700 1 -- 172.21.15.63:0/1085720463 shutdown_connections 2024-05-27T07:56:52.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.570+0000 7f0d84a00700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0d640b29f0 0x7f0d640b4f00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.570+0000 7f0d84a00700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d8010c5a0 0x7f0d801a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.570+0000 7f0d84a00700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d80106560 0x7f0d8019cec0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.570+0000 7f0d84a00700 1 --2- 172.21.15.63:0/1085720463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d80106eb0 0x7f0d8019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.570+0000 7f0d84a00700 1 -- 172.21.15.63:0/1085720463 >> 172.21.15.63:0/1085720463 conn(0x7f0d80102080 msgr2=0x7f0d801047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.571+0000 7f0d84a00700 1 -- 172.21.15.63:0/1085720463 shutdown_connections 2024-05-27T07:56:52.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:52.571+0000 7f0d84a00700 1 -- 172.21.15.63:0/1085720463 wait complete. 2024-05-27T07:56:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.788+0000 7faaef200640 1 -- 172.21.15.63:0/4109922324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faaf010c160 msgr2=0x7faaf0105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.788+0000 7faaef200640 1 --2- 172.21.15.63:0/4109922324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faaf010c160 0x7faaf0105da0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7faadc009900 tx=0x7faadc02a410 comp rx=0 tx=0).stop 2024-05-27T07:56:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.788+0000 7faaef200640 1 -- 172.21.15.63:0/4109922324 shutdown_connections 2024-05-27T07:56:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.788+0000 7faaef200640 1 --2- 172.21.15.63:0/4109922324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaf0106410 0x7faaf01127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.788+0000 7faaef200640 1 --2- 172.21.15.63:0/4109922324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faaf010c160 0x7faaf0105da0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.788+0000 7faaef200640 1 --2- 172.21.15.63:0/4109922324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faaf010bcc0 0x7faaf0105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.788+0000 7faaef200640 1 -- 172.21.15.63:0/4109922324 >> 172.21.15.63:0/4109922324 conn(0x7faaf0101380 msgr2=0x7faaf01037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.789+0000 7faaef200640 1 -- 172.21.15.63:0/4109922324 shutdown_connections 2024-05-27T07:56:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.789+0000 7faaef200640 1 -- 172.21.15.63:0/4109922324 wait complete. 2024-05-27T07:56:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.790+0000 7faaef200640 1 Processor -- start 2024-05-27T07:56:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.790+0000 7faaef200640 1 -- start start 2024-05-27T07:56:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.790+0000 7faaef200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faaf0106410 0x7faaf019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaef200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaf010bcc0 0x7faaf019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaef200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faaf010c160 0x7faaf01a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaef200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0115d90 con 0x7faaf010c160 2024-05-27T07:56:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaef200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0115c10 con 0x7faaf0106410 2024-05-27T07:56:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaef200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0115f10 con 0x7faaf010bcc0 2024-05-27T07:56:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaee800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faaf010c160 0x7faaf01a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaee800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faaf010c160 0x7faaf01a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38542/0 (socket says 172.21.15.63:38542) 2024-05-27T07:56:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaee800640 1 -- 172.21.15.63:0/3196529228 learned_addr learned my addr 172.21.15.63:0/3196529228 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaede00640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faaf0106410 0x7faaf019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaed400640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaf010bcc0 0x7faaf019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaee800640 1 -- 172.21.15.63:0/3196529228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaf010bcc0 msgr2=0x7faaf019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.791+0000 7faaee800640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaf010bcc0 0x7faaf019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.792+0000 7faaee800640 1 -- 172.21.15.63:0/3196529228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faaf0106410 msgr2=0x7faaf019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.792+0000 7faaee800640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faaf0106410 0x7faaf019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.792+0000 7faaee800640 1 -- 172.21.15.63:0/3196529228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf01a1650 con 0x7faaf010c160 2024-05-27T07:56:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.792+0000 7faaed400640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaf010bcc0 0x7faaf019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.792+0000 7faaee800640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faaf010c160 0x7faaf01a1010 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7faae800eaa0 tx=0x7faae800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.792+0000 7faae6a00640 1 -- 172.21.15.63:0/3196529228 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae800cdb0 con 0x7faaf010c160 2024-05-27T07:56:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.792+0000 7faae6a00640 1 -- 172.21.15.63:0/3196529228 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faae8005020 con 0x7faaf010c160 2024-05-27T07:56:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.792+0000 7faae6a00640 1 -- 172.21.15.63:0/3196529228 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae80116d0 con 0x7faaf010c160 2024-05-27T07:56:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.792+0000 7faaef200640 1 -- 172.21.15.63:0/3196529228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf01a1940 con 0x7faaf010c160 2024-05-27T07:56:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.793+0000 7faaef200640 1 -- 172.21.15.63:0/3196529228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf01a1e10 con 0x7faaf010c160 2024-05-27T07:56:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.797+0000 7faafbe35740 1 Processor -- start 2024-05-27T07:56:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.797+0000 7faae6a00640 1 -- 172.21.15.63:0/3196529228 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faae80051d0 con 0x7faaf010c160 2024-05-27T07:56:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.798+0000 7faafbe35740 1 -- start start 2024-05-27T07:56:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.798+0000 7faafbe35740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f85a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.798+0000 7faafbe35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bba2f0 0x55ba21f85fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.798+0000 7faafbe35740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f864e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.799+0000 7faafbe35740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ba21ef6b90 con 0x55ba21bc24d0 2024-05-27T07:56:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.799+0000 7faafbe35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba21ee6730 con 0x55ba21bba2f0 2024-05-27T07:56:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.799+0000 7faafbe35740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ba21eea770 con 0x55ba21be31b0 2024-05-27T07:56:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.799+0000 7faaed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f864e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.799+0000 7faaed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f864e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38550/0 (socket says 172.21.15.63:38550) 2024-05-27T07:56:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.799+0000 7faaed400640 1 -- 172.21.15.63:0/764297265 learned_addr learned my addr 172.21.15.63:0/764297265 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.799+0000 7faaed400640 1 --2- 172.21.15.63:0/764297265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f85a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.799+0000 7faaede00640 1 --2- 172.21.15.63:0/764297265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bba2f0 0x55ba21f85fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.800+0000 7faaed400640 1 -- 172.21.15.63:0/764297265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 msgr2=0x55ba21f85a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.800+0000 7faaed400640 1 --2- 172.21.15.63:0/764297265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f85a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.800+0000 7faaed400640 1 -- 172.21.15.63:0/764297265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bba2f0 msgr2=0x55ba21f85fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.800+0000 7faaed400640 1 --2- 172.21.15.63:0/764297265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bba2f0 0x55ba21f85fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.800+0000 7faaed400640 1 -- 172.21.15.63:0/764297265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba21e7f8b0 con 0x55ba21bc24d0 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.800+0000 7faae6a00640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faad40b2950 0x7faad40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.800+0000 7faae6a00640 1 -- 172.21.15.63:0/3196529228 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7faae80e2c00 con 0x7faaf010c160 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.800+0000 7faaed400640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faad40b2950 0x7faad40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.800+0000 7faaed400640 1 --2- 172.21.15.63:0/764297265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f864e0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7faadc005f70 tx=0x7faadc009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1015f763e42b30b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.801+0000 7faadb400640 1 -- 172.21.15.63:0/764297265 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc035040 con 0x55ba21bc24d0 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.801+0000 7faadb400640 1 -- 172.21.15.63:0/764297265 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faadc0054b0 con 0x55ba21bc24d0 2024-05-27T07:56:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.801+0000 7faadb400640 1 -- 172.21.15.63:0/764297265 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc004c00 con 0x55ba21bc24d0 2024-05-27T07:56:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.802+0000 7faafbe35740 1 -- 172.21.15.63:0/764297265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 msgr2=0x55ba21f864e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.802+0000 7faafbe35740 1 --2- 172.21.15.63:0/764297265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f864e0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7faadc005f70 tx=0x7faadc009560 comp rx=0 tx=0).stop 2024-05-27T07:56:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.802+0000 7faaed400640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faad40b2950 0x7faad40b4e60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55ba21f87510 tx=0x7faadc048000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.802+0000 7faafbe35740 1 -- 172.21.15.63:0/764297265 shutdown_connections 2024-05-27T07:56:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.802+0000 7faafbe35740 1 --2- 172.21.15.63:0/764297265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f864e0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.802+0000 7faafbe35740 1 --2- 172.21.15.63:0/764297265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bba2f0 0x55ba21f85fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.802+0000 7faafbe35740 1 --2- 172.21.15.63:0/764297265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f85a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.802+0000 7faafbe35740 1 -- 172.21.15.63:0/764297265 >> 172.21.15.63:0/764297265 conn(0x55ba21bd7300 msgr2=0x55ba21f82210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.802+0000 7faafbe35740 1 -- 172.21.15.63:0/764297265 shutdown_connections 2024-05-27T07:56:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.802+0000 7faafbe35740 1 -- 172.21.15.63:0/764297265 wait complete. 2024-05-27T07:56:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.803+0000 7faafbe35740 1 -- start start 2024-05-27T07:56:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.803+0000 7faafbe35740 1 Processor -- start 2024-05-27T07:56:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faafbe35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bba2f0 0x55ba21f8fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faafbe35740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f90170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faafbe35740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f9bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faafbe35740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ba21bcc790 con 0x55ba21bc24d0 2024-05-27T07:56:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faafbe35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba21e8ebc0 con 0x55ba21bba2f0 2024-05-27T07:56:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faafbe35740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ba21f92b90 con 0x55ba21be31b0 2024-05-27T07:56:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faafbe35740 10 client.0 _ll_register_callbacks cb 0x55ba21e6c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faaed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f90170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faaed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f90170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38560/0 (socket says 172.21.15.63:38560) 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faaed400640 1 -- 172.21.15.63:0/1325114387 learned_addr learned my addr 172.21.15.63:0/1325114387 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.804+0000 7faaede00640 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bba2f0 0x55ba21f8fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.805+0000 7faaee800640 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f9bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.805+0000 7faaed400640 1 -- 172.21.15.63:0/1325114387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 msgr2=0x55ba21f9bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.805+0000 7faaed400640 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f9bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.805+0000 7faaed400640 1 -- 172.21.15.63:0/1325114387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bba2f0 msgr2=0x55ba21f8fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.805+0000 7faaed400640 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bba2f0 0x55ba21f8fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.805+0000 7faaed400640 1 -- 172.21.15.63:0/1325114387 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba21bfb210 con 0x55ba21bc24d0 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.805+0000 7faaed400640 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f90170 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7faadc005e70 tx=0x7faadc0042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248b1d04c0105995 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.805+0000 7faacfe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.806+0000 7faacfe00640 1 -- 172.21.15.63:0/1325114387 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc035070 con 0x55ba21bc24d0 2024-05-27T07:56:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.806+0000 7faafbe35740 1 -- 172.21.15.63:0/1325114387 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faadc0054b0 con 0x55ba21bc24d0 2024-05-27T07:56:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.806+0000 7faacfe00640 1 -- 172.21.15.63:0/1325114387 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7faadc004c10 con 0x55ba21bc24d0 2024-05-27T07:56:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.806+0000 7faacfe00640 1 -- 172.21.15.63:0/1325114387 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faadc063050 con 0x55ba21bc24d0 2024-05-27T07:56:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.806+0000 7faafbe35740 10 client.19896 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.806+0000 7faafbe35740 1 -- 172.21.15.63:0/1325114387 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ba21cc7620 con 0x55ba21bc24d0 2024-05-27T07:56:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.806+0000 7faacfe00640 1 -- 172.21.15.63:0/1325114387 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faadc0054b0 con 0x55ba21bc24d0 2024-05-27T07:56:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.807+0000 7faacfe00640 1 -- 172.21.15.63:0/1325114387 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7faadc03a040 con 0x55ba21bc24d0 2024-05-27T07:56:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.807+0000 7faafbe35740 10 client.19896 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.807+0000 7faafbe35740 10 client.19896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.807+0000 7faafbe35740 20 client.19896 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.807+0000 7faafbe35740 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba21ba1d80 0x55ba21fa1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.807+0000 7faafbe35740 4 client.19896 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.807+0000 7faafbe35740 1 -- 172.21.15.63:0/1325114387 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba21ef8ae0 con 0x55ba21ba1d80 2024-05-27T07:56:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.807+0000 7faaede00640 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba21ba1d80 0x55ba21fa1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.808+0000 7faaede00640 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba21ba1d80 0x55ba21fa1d30 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8993534c283850a1 server_cookie=77e37b330fce78f6 in_seq=0 out_seq=0 2024-05-27T07:56:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.808+0000 7faacfe00640 10 client.19896 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.809+0000 7faacfe00640 1 -- 172.21.15.63:0/1325114387 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ba21ef8ae0 con 0x55ba21ba1d80 2024-05-27T07:56:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.809+0000 7faacfe00640 10 client.19896 handle_command_reply: tid=0 2024-05-27T07:56:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.809+0000 7faafbe35740 1 client.19896 shutdown 2024-05-27T07:56:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 1 -- 172.21.15.63:0/1325114387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 msgr2=0x55ba21f90170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f90170 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7faadc005e70 tx=0x7faadc0042b0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 1 -- 172.21.15.63:0/1325114387 shutdown_connections 2024-05-27T07:56:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba21ba1d80 0x55ba21fa1d30 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f9bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21f90170 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 1 --2- 172.21.15.63:0/1325114387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bba2f0 0x55ba21f8fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 1 -- 172.21.15.63:0/1325114387 >> 172.21.15.63:0/1325114387 conn(0x55ba21bd7300 msgr2=0x55ba21f87780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faacfe00640 0 client.19896 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 1 -- 172.21.15.63:0/1325114387 shutdown_connections 2024-05-27T07:56:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 1 -- 172.21.15.63:0/1325114387 wait complete. 2024-05-27T07:56:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.810+0000 7faafbe35740 20 client.19896 trim_cache size 0 max 16384 2024-05-27T07:56:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.829+0000 7faafbe35740 1 Processor -- start 2024-05-27T07:56:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.830+0000 7faafbe35740 1 -- start start 2024-05-27T07:56:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.830+0000 7faafbe35740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bba2f0 0x55ba21f90180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.830+0000 7faafbe35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bc24d0 0x55ba21f897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.830+0000 7faafbe35740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f89d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.830+0000 7faafbe35740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faaf01b5ac0 con 0x55ba21bba2f0 2024-05-27T07:56:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.830+0000 7faafbe35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faaf01b3390 con 0x55ba21bc24d0 2024-05-27T07:56:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.830+0000 7faafbe35740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faaf01b1200 con 0x55ba21be31b0 2024-05-27T07:56:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.830+0000 7faaede00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bba2f0 0x55ba21f90180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.830+0000 7faaede00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bba2f0 0x55ba21f90180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38566/0 (socket says 172.21.15.63:38566) 2024-05-27T07:56:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.830+0000 7faaede00640 1 -- 172.21.15.63:0/3476364511 learned_addr learned my addr 172.21.15.63:0/3476364511 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.831+0000 7faaed400640 1 --2- 172.21.15.63:0/3476364511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bc24d0 0x55ba21f897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.831+0000 7faaee800640 1 --2- 172.21.15.63:0/3476364511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f89d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.831+0000 7faaede00640 1 -- 172.21.15.63:0/3476364511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 msgr2=0x55ba21f89d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.831+0000 7faaede00640 1 --2- 172.21.15.63:0/3476364511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f89d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.831+0000 7faaede00640 1 -- 172.21.15.63:0/3476364511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bc24d0 msgr2=0x55ba21f897c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.831+0000 7faaede00640 1 --2- 172.21.15.63:0/3476364511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bc24d0 0x55ba21f897c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.831+0000 7faaede00640 1 -- 172.21.15.63:0/3476364511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba21f82de0 con 0x55ba21bba2f0 2024-05-27T07:56:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.831+0000 7faaede00640 1 --2- 172.21.15.63:0/3476364511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bba2f0 0x55ba21f90180 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7faae000b850 tx=0x7faae000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34076161d0f804ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.832+0000 7faadaa00640 1 -- 172.21.15.63:0/3476364511 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0003d40 con 0x55ba21bba2f0 2024-05-27T07:56:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.832+0000 7faadaa00640 1 -- 172.21.15.63:0/3476364511 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faae00054b0 con 0x55ba21bba2f0 2024-05-27T07:56:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.832+0000 7faadaa00640 1 -- 172.21.15.63:0/3476364511 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0011440 con 0x55ba21bba2f0 2024-05-27T07:56:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.833+0000 7faafbe35740 1 -- 172.21.15.63:0/3476364511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bba2f0 msgr2=0x55ba21f90180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.833+0000 7faafbe35740 1 --2- 172.21.15.63:0/3476364511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bba2f0 0x55ba21f90180 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7faae000b850 tx=0x7faae000b390 comp rx=0 tx=0).stop 2024-05-27T07:56:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.833+0000 7faafbe35740 1 -- 172.21.15.63:0/3476364511 shutdown_connections 2024-05-27T07:56:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.833+0000 7faafbe35740 1 --2- 172.21.15.63:0/3476364511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21be31b0 0x55ba21f89d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.833+0000 7faafbe35740 1 --2- 172.21.15.63:0/3476364511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21bc24d0 0x55ba21f897c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.833+0000 7faafbe35740 1 --2- 172.21.15.63:0/3476364511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bba2f0 0x55ba21f90180 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.833+0000 7faafbe35740 1 -- 172.21.15.63:0/3476364511 >> 172.21.15.63:0/3476364511 conn(0x55ba21bd7300 msgr2=0x55ba21f91350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.833+0000 7faafbe35740 1 -- 172.21.15.63:0/3476364511 shutdown_connections 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.833+0000 7faafbe35740 1 -- 172.21.15.63:0/3476364511 wait complete. 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.833+0000 7faafbe35740 1 -- start start 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.834+0000 7faafbe35740 1 Processor -- start 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.834+0000 7faafbe35740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21bba2f0 0x55ba21fa22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.834+0000 7faafbe35740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21fa2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.834+0000 7faafbe35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21be31b0 0x55ba22008170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.834+0000 7faafbe35740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faaf01b1850 con 0x55ba21bc24d0 2024-05-27T07:56:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.834+0000 7faafbe35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faaf01b2cc0 con 0x55ba21be31b0 2024-05-27T07:56:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.834+0000 7faafbe35740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faaf01b1f20 con 0x55ba21bba2f0 2024-05-27T07:56:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faafbe35740 10 client.0 _ll_register_callbacks cb 0x7faae000b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faaed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21fa2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faaed400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21fa2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38568/0 (socket says 172.21.15.63:38568) 2024-05-27T07:56:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faaed400640 1 -- 172.21.15.63:0/241155715 learned_addr learned my addr 172.21.15.63:0/241155715 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faaee800640 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21be31b0 0x55ba22008170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faaede00640 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21bba2f0 0x55ba21fa22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faaed400640 1 -- 172.21.15.63:0/241155715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21bba2f0 msgr2=0x55ba21fa22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faaed400640 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21bba2f0 0x55ba21fa22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faaed400640 1 -- 172.21.15.63:0/241155715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21be31b0 msgr2=0x55ba22008170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faaed400640 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21be31b0 0x55ba22008170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.835+0000 7faaed400640 1 -- 172.21.15.63:0/241155715 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba220086e0 con 0x55ba21bc24d0 2024-05-27T07:56:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.836+0000 7faaed400640 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21fa2810 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7faadc0057c0 tx=0x7faadc02ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5386be9e187f4ff2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.836+0000 7faacfe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.836+0000 7faacfe00640 1 -- 172.21.15.63:0/241155715 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc041020 con 0x55ba21bc24d0 2024-05-27T07:56:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.836+0000 7faacfe00640 1 -- 172.21.15.63:0/241155715 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7faadc03d480 con 0x55ba21bc24d0 2024-05-27T07:56:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.836+0000 7faafbe35740 1 -- 172.21.15.63:0/241155715 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae00054b0 con 0x55ba21bc24d0 2024-05-27T07:56:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.836+0000 7faacfe00640 1 -- 172.21.15.63:0/241155715 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faadc035040 con 0x55ba21bc24d0 2024-05-27T07:56:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.836+0000 7faafbe35740 10 client.19902 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.836+0000 7faafbe35740 1 -- 172.21.15.63:0/241155715 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ba22009950 con 0x55ba21bc24d0 2024-05-27T07:56:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.837+0000 7faacfe00640 1 -- 172.21.15.63:0/241155715 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faae00054b0 con 0x55ba21bc24d0 2024-05-27T07:56:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.837+0000 7faacfe00640 1 -- 172.21.15.63:0/241155715 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7faadc004c10 con 0x55ba21bc24d0 2024-05-27T07:56:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.837+0000 7faafbe35740 10 client.19902 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.837+0000 7faafbe35740 10 client.19902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.837+0000 7faafbe35740 20 client.19902 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.837+0000 7faafbe35740 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba21b771c0 0x55ba2200bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.837+0000 7faafbe35740 4 client.19902 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:56:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.837+0000 7faafbe35740 1 -- 172.21.15.63:0/241155715 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba21ef22c0 con 0x55ba21b771c0 2024-05-27T07:56:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.837+0000 7faaede00640 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba21b771c0 0x55ba2200bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.838+0000 7faaede00640 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba21b771c0 0x55ba2200bfc0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34c542c332301071 server_cookie=747087ddbe0f9fdb in_seq=0 out_seq=0 2024-05-27T07:56:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.838+0000 7faacfe00640 10 client.19902 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.872+0000 7faacfe00640 1 -- 172.21.15.63:0/241155715 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ba21ef22c0 con 0x55ba21b771c0 2024-05-27T07:56:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.872+0000 7faacfe00640 10 client.19902 handle_command_reply: tid=0 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.872+0000 7faafbe35740 1 client.19902 shutdown 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.872+0000 7faafbe35740 1 -- 172.21.15.63:0/241155715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 msgr2=0x55ba21fa2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.872+0000 7faafbe35740 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21fa2810 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7faadc0057c0 tx=0x7faadc02ab20 comp rx=0 tx=0).stop 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.873+0000 7faafbe35740 1 -- 172.21.15.63:0/241155715 shutdown_connections 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.873+0000 7faafbe35740 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba21b771c0 0x55ba2200bfc0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.873+0000 7faafbe35740 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba21be31b0 0x55ba22008170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.873+0000 7faafbe35740 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba21bc24d0 0x55ba21fa2810 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.873+0000 7faafbe35740 1 --2- 172.21.15.63:0/241155715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba21bba2f0 0x55ba21fa22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.873+0000 7faafbe35740 1 -- 172.21.15.63:0/241155715 >> 172.21.15.63:0/241155715 conn(0x55ba21bd7300 msgr2=0x55ba21f8fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.873+0000 7faacfe00640 0 client.19902 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.873+0000 7faafbe35740 1 -- 172.21.15.63:0/241155715 shutdown_connections 2024-05-27T07:56:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.873+0000 7faafbe35740 1 -- 172.21.15.63:0/241155715 wait complete. 2024-05-27T07:56:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.873+0000 7faafbe35740 20 client.19902 trim_cache size 0 max 16384 2024-05-27T07:56:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:56:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:56:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:56:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 -- 172.21.15.63:0/3196529228 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faad40b2950 msgr2=0x7faad40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faad40b2950 0x7faad40b4e60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55ba21f87510 tx=0x7faadc048000 comp rx=0 tx=0).stop 2024-05-27T07:56:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 -- 172.21.15.63:0/3196529228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faaf010c160 msgr2=0x7faaf01a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faaf010c160 0x7faaf01a1010 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7faae800eaa0 tx=0x7faae800ef60 comp rx=0 tx=0).stop 2024-05-27T07:56:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 -- 172.21.15.63:0/3196529228 shutdown_connections 2024-05-27T07:56:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faad40b2950 0x7faad40b4e60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faaf010c160 0x7faaf01a1010 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaf010bcc0 0x7faaf019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 --2- 172.21.15.63:0/3196529228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faaf0106410 0x7faaf019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 -- 172.21.15.63:0/3196529228 >> 172.21.15.63:0/3196529228 conn(0x7faaf0101380 msgr2=0x7faaf01119f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.874+0000 7faaef200640 1 -- 172.21.15.63:0/3196529228 shutdown_connections 2024-05-27T07:56:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:56:52.875+0000 7faaef200640 1 -- 172.21.15.63:0/3196529228 wait complete. 2024-05-27T07:56:53.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:53 smithi086 ceph-mon[34687]: pgmap v736: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:56:53.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:53 smithi149 ceph-mon[35408]: pgmap v736: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:56:53.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:53 smithi063 ceph-mon[31613]: pgmap v736: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T07:56:53.791 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:56:54.323 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:55.265 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:55 smithi063 ceph-mon[31613]: pgmap v737: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:56:55.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:55 smithi086 ceph-mon[34687]: pgmap v737: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:56:55.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:55 smithi149 ceph-mon[35408]: pgmap v737: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:56:56.843 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:56 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:56.844 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:56 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:56.844 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:56 smithi063 ceph-mon[31613]: pgmap v738: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:56:56.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:56 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:56.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:56 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:56.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:56 smithi086 ceph-mon[34687]: pgmap v738: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:56:56.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:56 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:56 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:56 smithi149 ceph-mon[35408]: pgmap v738: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T07:56:57.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.017+0000 7f6e07e00700 1 -- 172.21.15.63:0/3166128413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e001045e0 msgr2=0x7f6e001049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.017+0000 7f6e07e00700 1 --2- 172.21.15.63:0/3166128413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e001045e0 0x7f6e001049f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f6df40099c0 tx=0x7f6df4009cd0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.018+0000 7f6e07e00700 1 -- 172.21.15.63:0/3166128413 shutdown_connections 2024-05-27T07:56:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.018+0000 7f6e07e00700 1 --2- 172.21.15.63:0/3166128413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e00109860 0x7f6e001111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.018+0000 7f6e07e00700 1 --2- 172.21.15.63:0/3166128413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e001045e0 0x7f6e001049f0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.018+0000 7f6e07e00700 1 --2- 172.21.15.63:0/3166128413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e00108f10 0x7f6e00109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.018+0000 7f6e07e00700 1 -- 172.21.15.63:0/3166128413 >> 172.21.15.63:0/3166128413 conn(0x7f6e00100030 msgr2=0x7f6e00102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.018+0000 7f6e07e00700 1 -- 172.21.15.63:0/3166128413 shutdown_connections 2024-05-27T07:56:57.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.018+0000 7f6e07e00700 1 -- 172.21.15.63:0/3166128413 wait complete. 2024-05-27T07:56:57.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.019+0000 7f6e07e00700 1 Processor -- start 2024-05-27T07:56:57.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.019+0000 7f6e07e00700 1 -- start start 2024-05-27T07:56:57.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.020+0000 7f6e07e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e001045e0 0x7f6e0019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.020+0000 7f6e07e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e00108f10 0x7f6e0019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.020+0000 7f6e07e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e00109860 0x7f6e001a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.020+0000 7f6e07e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00114390 con 0x7f6e001045e0 2024-05-27T07:56:57.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.020+0000 7f6e07e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00114210 con 0x7f6e00109860 2024-05-27T07:56:57.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.020+0000 7f6e07e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6e00114510 con 0x7f6e00108f10 2024-05-27T07:56:57.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.020+0000 7f6e05400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e001045e0 0x7f6e0019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.020+0000 7f6e05400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e001045e0 0x7f6e0019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38592/0 (socket says 172.21.15.63:38592) 2024-05-27T07:56:57.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.020+0000 7f6e05400700 1 -- 172.21.15.63:0/1762334795 learned_addr learned my addr 172.21.15.63:0/1762334795 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:57.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.020+0000 7f6e04a00700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e00108f10 0x7f6e0019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.021+0000 7f6e05e00700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e00109860 0x7f6e001a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.021+0000 7f6e05400700 1 -- 172.21.15.63:0/1762334795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e00108f10 msgr2=0x7f6e0019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:57.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.021+0000 7f6e05400700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e00108f10 0x7f6e0019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.021+0000 7f6e05400700 1 -- 172.21.15.63:0/1762334795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e00109860 msgr2=0x7f6e001a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:57.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.021+0000 7f6e05400700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e00109860 0x7f6e001a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.021+0000 7f6e05400700 1 -- 172.21.15.63:0/1762334795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e001a1ee0 con 0x7f6e001045e0 2024-05-27T07:56:57.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.022+0000 7f6e05400700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e001045e0 0x7f6e0019d060 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f6df4000c00 tx=0x7f6df400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:57.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.022+0000 7f6dfe000700 1 -- 172.21.15.63:0/1762334795 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4027070 con 0x7f6e001045e0 2024-05-27T07:56:57.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.022+0000 7f6dfe000700 1 -- 172.21.15.63:0/1762334795 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6df400fcf0 con 0x7f6e001045e0 2024-05-27T07:56:57.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.022+0000 7f6e07e00700 1 -- 172.21.15.63:0/1762334795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e001a2170 con 0x7f6e001045e0 2024-05-27T07:56:57.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.022+0000 7f6dfe000700 1 -- 172.21.15.63:0/1762334795 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4022dc0 con 0x7f6e001045e0 2024-05-27T07:56:57.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.022+0000 7f6e07e00700 1 -- 172.21.15.63:0/1762334795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e001a2650 con 0x7f6e001045e0 2024-05-27T07:56:57.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.026+0000 7f6dfe000700 1 -- 172.21.15.63:0/1762334795 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6df402a070 con 0x7f6e001045e0 2024-05-27T07:56:57.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.028+0000 7f6dfe000700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6de80b2b40 0x7f6de80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.029+0000 7f6e16891b80 1 Processor -- start 2024-05-27T07:56:57.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.029+0000 7f6dfe000700 1 -- 172.21.15.63:0/1762334795 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f6df40e36f0 con 0x7f6e001045e0 2024-05-27T07:56:57.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.029+0000 7f6e16891b80 1 -- start start 2024-05-27T07:56:57.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.029+0000 7f6e04a00700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6de80b2b40 0x7f6de80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c50000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 0x562869c4a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c4a5c0 0x562869c4a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e16891b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562869b6fe40 con 0x562869ac2e40 2024-05-27T07:56:57.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e16891b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562869b5f9e0 con 0x562869c4a5c0 2024-05-27T07:56:57.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e16891b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562869b63a20 con 0x562869c50540 2024-05-27T07:56:57.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e04a00700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6de80b2b40 0x7f6de80b5050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6e00107e30 tx=0x7f6dec00b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:57.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e05400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c50000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e04a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 0x562869c4a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e05e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c4a5c0 0x562869c4a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e05400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c50000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38596/0 (socket says 172.21.15.63:38596) 2024-05-27T07:56:57.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e05400700 1 -- 172.21.15.63:0/34852013 learned_addr learned my addr 172.21.15.63:0/34852013 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:57.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.030+0000 7f6e04a00700 1 --2- 172.21.15.63:0/34852013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 0x562869c4a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:60316/0 (socket says 172.21.15.63:60316) 2024-05-27T07:56:57.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.031+0000 7f6e04a00700 1 -- 172.21.15.63:0/34852013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c4a5c0 msgr2=0x562869c4a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:57.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.031+0000 7f6e04a00700 1 --2- 172.21.15.63:0/34852013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c4a5c0 0x562869c4a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.031+0000 7f6e04a00700 1 -- 172.21.15.63:0/34852013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 msgr2=0x562869c50000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:57.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.031+0000 7f6e04a00700 1 --2- 172.21.15.63:0/34852013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c50000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.031+0000 7f6e04a00700 1 -- 172.21.15.63:0/34852013 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562869b28420 con 0x562869c50540 2024-05-27T07:56:57.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.031+0000 7f6e05400700 1 --2- 172.21.15.63:0/34852013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c50000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:56:57.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.032+0000 7f6e04a00700 1 --2- 172.21.15.63:0/34852013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 0x562869c4a080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e0006c670 tx=0x7f6dec013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b05ea5b56a2a8911 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.032+0000 7f6df2a00700 1 -- 172.21.15.63:0/34852013 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec011bb0 con 0x562869c50540 2024-05-27T07:56:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.032+0000 7f6df2a00700 1 -- 172.21.15.63:0/34852013 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dec011d50 con 0x562869c50540 2024-05-27T07:56:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.032+0000 7f6df2a00700 1 -- 172.21.15.63:0/34852013 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec01d610 con 0x562869c50540 2024-05-27T07:56:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.033+0000 7f6e16891b80 1 -- 172.21.15.63:0/34852013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 msgr2=0x562869c4a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.033+0000 7f6e16891b80 1 --2- 172.21.15.63:0/34852013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 0x562869c4a080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6e0006c670 tx=0x7f6dec013000 comp rx=0 tx=0).stop 2024-05-27T07:56:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.034+0000 7f6e16891b80 1 -- 172.21.15.63:0/34852013 shutdown_connections 2024-05-27T07:56:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.034+0000 7f6e16891b80 1 --2- 172.21.15.63:0/34852013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c4a5c0 0x562869c4a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.034+0000 7f6e16891b80 1 --2- 172.21.15.63:0/34852013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c50000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.034+0000 7f6e16891b80 1 --2- 172.21.15.63:0/34852013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 0x562869c4a080 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.034+0000 7f6e16891b80 1 -- 172.21.15.63:0/34852013 >> 172.21.15.63:0/34852013 conn(0x562869b22fc0 msgr2=0x562869c3c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.034+0000 7f6e16891b80 1 -- 172.21.15.63:0/34852013 shutdown_connections 2024-05-27T07:56:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.034+0000 7f6e16891b80 1 -- 172.21.15.63:0/34852013 wait complete. 2024-05-27T07:56:57.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.035+0000 7f6e16891b80 1 -- start start 2024-05-27T07:56:57.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.035+0000 7f6e16891b80 1 Processor -- start 2024-05-27T07:56:57.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.036+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c53cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.036+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c4a5c0 0x562869c541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.036+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 0x562869c54730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.036+0000 7f6e16891b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562869a65fa0 con 0x562869ac2e40 2024-05-27T07:56:57.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.036+0000 7f6e16891b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562869b00760 con 0x562869c4a5c0 2024-05-27T07:56:57.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.036+0000 7f6e16891b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562869c57010 con 0x562869c50540 2024-05-27T07:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.036+0000 7f6e16891b80 10 client.0 _ll_register_callbacks cb 0x562869b09c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.036+0000 7f6e05400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c53cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.036+0000 7f6e05400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c53cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38604/0 (socket says 172.21.15.63:38604) 2024-05-27T07:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.037+0000 7f6e05400700 1 -- 172.21.15.63:0/2052902608 learned_addr learned my addr 172.21.15.63:0/2052902608 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.037+0000 7f6e04a00700 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c4a5c0 0x562869c541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.037+0000 7f6e05e00700 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 0x562869c54730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.037+0000 7f6e05400700 1 -- 172.21.15.63:0/2052902608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 msgr2=0x562869c54730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.037+0000 7f6e05400700 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 0x562869c54730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.037+0000 7f6e05400700 1 -- 172.21.15.63:0/2052902608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c4a5c0 msgr2=0x562869c541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.037+0000 7f6e05400700 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c4a5c0 0x562869c541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.037+0000 7f6e05400700 1 -- 172.21.15.63:0/2052902608 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562869aaf8d0 con 0x562869ac2e40 2024-05-27T07:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.038+0000 7f6e05400700 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c53cb0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f6df40e0990 tx=0x7f6df40aab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97e7cfe3888d678 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.038+0000 7f6de7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.038+0000 7f6de7400700 1 -- 172.21.15.63:0/2052902608 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40e2070 con 0x562869ac2e40 2024-05-27T07:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.038+0000 7f6de7400700 1 -- 172.21.15.63:0/2052902608 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f6df40afb80 con 0x562869ac2e40 2024-05-27T07:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.038+0000 7f6e16891b80 1 -- 172.21.15.63:0/2052902608 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dec011d50 con 0x562869ac2e40 2024-05-27T07:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.038+0000 7f6de7400700 1 -- 172.21.15.63:0/2052902608 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6df40095c0 con 0x562869ac2e40 2024-05-27T07:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.038+0000 7f6e16891b80 10 client.19914 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.038+0000 7f6e16891b80 1 -- 172.21.15.63:0/2052902608 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562869b30060 con 0x562869ac2e40 2024-05-27T07:56:57.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.039+0000 7f6de7400700 1 -- 172.21.15.63:0/2052902608 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6dec011d50 con 0x562869ac2e40 2024-05-27T07:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.039+0000 7f6de7400700 1 -- 172.21.15.63:0/2052902608 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6df40b1ce0 con 0x562869ac2e40 2024-05-27T07:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.039+0000 7f6e16891b80 10 client.19914 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.039+0000 7f6e16891b80 10 client.19914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.039+0000 7f6e16891b80 20 client.19914 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.039+0000 7f6e16891b80 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x562869c63e80 0x562869c66310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.039+0000 7f6e16891b80 4 client.19914 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:57.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.039+0000 7f6e16891b80 1 -- 172.21.15.63:0/2052902608 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562869b71d90 con 0x562869c63e80 2024-05-27T07:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.040+0000 7f6e04a00700 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x562869c63e80 0x562869c66310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.041+0000 7f6e04a00700 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x562869c63e80 0x562869c66310 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7608960e03464b5d server_cookie=70f2f50bf9a1b592 in_seq=0 out_seq=0 2024-05-27T07:56:57.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.041+0000 7f6de7400700 10 client.19914 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:56:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.043+0000 7f6de7400700 1 -- 172.21.15.63:0/2052902608 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562869b71d90 con 0x562869c63e80 2024-05-27T07:56:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.043+0000 7f6de7400700 10 client.19914 handle_command_reply: tid=0 2024-05-27T07:56:57.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.043+0000 7f6e16891b80 1 client.19914 shutdown 2024-05-27T07:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.043+0000 7f6e16891b80 1 -- 172.21.15.63:0/2052902608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 msgr2=0x562869c53cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.043+0000 7f6e16891b80 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c53cb0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f6df40e0990 tx=0x7f6df40aab80 comp rx=0 tx=0).stop 2024-05-27T07:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.044+0000 7f6e16891b80 1 -- 172.21.15.63:0/2052902608 shutdown_connections 2024-05-27T07:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.044+0000 7f6e16891b80 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c50540 0x562869c54730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.044+0000 7f6e16891b80 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x562869c63e80 0x562869c66310 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.044+0000 7f6e16891b80 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c53cb0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.044+0000 7f6e16891b80 1 --2- 172.21.15.63:0/2052902608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c4a5c0 0x562869c541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.044+0000 7f6e16891b80 1 -- 172.21.15.63:0/2052902608 >> 172.21.15.63:0/2052902608 conn(0x562869b22fc0 msgr2=0x562869c3c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.044+0000 7f6de7400700 0 client.19914 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:56:57.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.044+0000 7f6e16891b80 1 -- 172.21.15.63:0/2052902608 shutdown_connections 2024-05-27T07:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.044+0000 7f6e16891b80 1 -- 172.21.15.63:0/2052902608 wait complete. 2024-05-27T07:56:57.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.044+0000 7f6e16891b80 20 client.19914 trim_cache size 0 max 16384 2024-05-27T07:56:57.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.063+0000 7f6e16891b80 1 Processor -- start 2024-05-27T07:56:57.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.064+0000 7f6e16891b80 1 -- start start 2024-05-27T07:56:57.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.064+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869ac2e40 0x562869c4a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.064+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 0x562869c4e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.064+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869c50540 0x562869c56010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.064+0000 7f6e16891b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001b0a20 con 0x562869c50540 2024-05-27T07:56:57.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.064+0000 7f6e16891b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001ae2b0 con 0x562869ac2e40 2024-05-27T07:56:57.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.064+0000 7f6e16891b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001ac120 con 0x562869c4a5c0 2024-05-27T07:56:57.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e05e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869c50540 0x562869c56010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e05e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869c50540 0x562869c56010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38618/0 (socket says 172.21.15.63:38618) 2024-05-27T07:56:57.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e05e00700 1 -- 172.21.15.63:0/469835428 learned_addr learned my addr 172.21.15.63:0/469835428 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e05400700 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869ac2e40 0x562869c4a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e04a00700 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 0x562869c4e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e05e00700 1 -- 172.21.15.63:0/469835428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 msgr2=0x562869c4e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e05e00700 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 0x562869c4e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e05e00700 1 -- 172.21.15.63:0/469835428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869ac2e40 msgr2=0x562869c4a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e05e00700 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869ac2e40 0x562869c4a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e05e00700 1 -- 172.21.15.63:0/469835428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562869c56550 con 0x562869c50540 2024-05-27T07:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e05400700 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869ac2e40 0x562869c4a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.065+0000 7f6e04a00700 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 0x562869c4e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.066+0000 7f6e05e00700 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869c50540 0x562869c56010 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009ee0 tx=0x7f6df8009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8174373be4bfe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.066+0000 7f6df2000700 1 -- 172.21.15.63:0/469835428 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8004bb0 con 0x562869c50540 2024-05-27T07:56:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.066+0000 7f6df2000700 1 -- 172.21.15.63:0/469835428 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6df801b070 con 0x562869c50540 2024-05-27T07:56:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.066+0000 7f6df2000700 1 -- 172.21.15.63:0/469835428 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8005020 con 0x562869c50540 2024-05-27T07:56:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.067+0000 7f6e16891b80 1 -- 172.21.15.63:0/469835428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869c50540 msgr2=0x562869c56010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:57.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.067+0000 7f6e16891b80 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869c50540 0x562869c56010 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009ee0 tx=0x7f6df8009f10 comp rx=0 tx=0).stop 2024-05-27T07:56:57.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.068+0000 7f6e16891b80 1 -- 172.21.15.63:0/469835428 shutdown_connections 2024-05-27T07:56:57.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.068+0000 7f6e16891b80 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869c50540 0x562869c56010 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.068+0000 7f6e16891b80 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869ac2e40 0x562869c4a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.068+0000 7f6e16891b80 1 --2- 172.21.15.63:0/469835428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 0x562869c4e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.068+0000 7f6e16891b80 1 -- 172.21.15.63:0/469835428 >> 172.21.15.63:0/469835428 conn(0x562869b22fc0 msgr2=0x562869c4f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:57.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.068+0000 7f6e16891b80 1 -- 172.21.15.63:0/469835428 shutdown_connections 2024-05-27T07:56:57.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.068+0000 7f6e16891b80 1 -- 172.21.15.63:0/469835428 wait complete. 2024-05-27T07:56:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.068+0000 7f6e16891b80 1 -- start start 2024-05-27T07:56:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.069+0000 7f6e16891b80 1 Processor -- start 2024-05-27T07:56:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.069+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c6b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.069+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 0x562869c6bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.069+0000 7f6e16891b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c50540 0x562869c6c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.069+0000 7f6e16891b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001ac770 con 0x562869ac2e40 2024-05-27T07:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.069+0000 7f6e16891b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001adbe0 con 0x562869c50540 2024-05-27T07:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.069+0000 7f6e16891b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6e001ace40 con 0x562869c4a5c0 2024-05-27T07:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.069+0000 7f6e16891b80 10 client.0 _ll_register_callbacks cb 0x562869b09c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.069+0000 7f6e05400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c6b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6e05400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c6b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38634/0 (socket says 172.21.15.63:38634) 2024-05-27T07:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6e05400700 1 -- 172.21.15.63:0/803388290 learned_addr learned my addr 172.21.15.63:0/803388290 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6e04a00700 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 0x562869c6bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6e05e00700 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c50540 0x562869c6c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6e05400700 1 -- 172.21.15.63:0/803388290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 msgr2=0x562869c6bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:57.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6e05400700 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 0x562869c6bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6e05400700 1 -- 172.21.15.63:0/803388290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c50540 msgr2=0x562869c6c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:57.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6e05400700 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c50540 0x562869c6c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6e05400700 1 -- 172.21.15.63:0/803388290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562869ccd9d0 con 0x562869ac2e40 2024-05-27T07:56:57.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6e05400700 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c6b6a0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f6df40afdd0 tx=0x7f6df4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6e1c87b06f4d22 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.070+0000 7f6de7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:56:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.071+0000 7f6de7400700 1 -- 172.21.15.63:0/803388290 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df40e2070 con 0x562869ac2e40 2024-05-27T07:56:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.071+0000 7f6de7400700 1 -- 172.21.15.63:0/803388290 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f6df40a6c30 con 0x562869ac2e40 2024-05-27T07:56:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.071+0000 7f6e16891b80 1 -- 172.21.15.63:0/803388290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df801b070 con 0x562869ac2e40 2024-05-27T07:56:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.071+0000 7f6de7400700 1 -- 172.21.15.63:0/803388290 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6df40e34c0 con 0x562869ac2e40 2024-05-27T07:56:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.071+0000 7f6e16891b80 10 client.19920 fetch_fsmap learned FSMap version 33 2024-05-27T07:56:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.071+0000 7f6e16891b80 1 -- 172.21.15.63:0/803388290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562869cceb80 con 0x562869ac2e40 2024-05-27T07:56:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.072+0000 7f6de7400700 1 -- 172.21.15.63:0/803388290 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6df801b070 con 0x562869ac2e40 2024-05-27T07:56:57.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.072+0000 7f6de7400700 1 -- 172.21.15.63:0/803388290 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f6df40b1ce0 con 0x562869ac2e40 2024-05-27T07:56:57.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.072+0000 7f6e16891b80 10 client.19920 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:56:57.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.072+0000 7f6e16891b80 10 client.19920 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:56:57.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.072+0000 7f6e16891b80 20 client.19920 populate_metadata read hostname 'smithi063' 2024-05-27T07:56:57.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.072+0000 7f6e16891b80 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x562869ccf0d0 0x562869cd14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:57.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.072+0000 7f6e16891b80 4 client.19920 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:56:57.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.072+0000 7f6e16891b80 1 -- 172.21.15.63:0/803388290 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562869b6b570 con 0x562869ccf0d0 2024-05-27T07:56:57.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.073+0000 7f6e04a00700 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x562869ccf0d0 0x562869cd14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:57.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.074+0000 7f6e04a00700 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x562869ccf0d0 0x562869cd14f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61159a309e270f59 server_cookie=ed4d42ef21996ba2 in_seq=0 out_seq=0 2024-05-27T07:56:57.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.074+0000 7f6de7400700 10 client.19920 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:56:57.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.074+0000 7f6de7400700 1 -- 172.21.15.63:0/803388290 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562869b6b570 con 0x562869ccf0d0 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.074+0000 7f6de7400700 10 client.19920 handle_command_reply: tid=0 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.074+0000 7f6e16891b80 1 client.19920 shutdown 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.074+0000 7f6e16891b80 1 -- 172.21.15.63:0/803388290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 msgr2=0x562869c6b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.074+0000 7f6e16891b80 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c6b6a0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f6df40afdd0 tx=0x7f6df4005830 comp rx=0 tx=0).stop 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.075+0000 7f6e16891b80 1 -- 172.21.15.63:0/803388290 shutdown_connections 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.075+0000 7f6e16891b80 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562869c50540 0x562869c6c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.075+0000 7f6e16891b80 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x562869ccf0d0 0x562869cd14f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.075+0000 7f6e16891b80 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562869ac2e40 0x562869c6b6a0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.075+0000 7f6e16891b80 1 --2- 172.21.15.63:0/803388290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562869c4a5c0 0x562869c6bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.075+0000 7f6e16891b80 1 -- 172.21.15.63:0/803388290 >> 172.21.15.63:0/803388290 conn(0x562869b22fc0 msgr2=0x562869c53a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:57.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.075+0000 7f6de7400700 0 client.19920 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:56:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.075+0000 7f6e16891b80 1 -- 172.21.15.63:0/803388290 shutdown_connections 2024-05-27T07:56:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.075+0000 7f6e16891b80 1 -- 172.21.15.63:0/803388290 wait complete. 2024-05-27T07:56:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.075+0000 7f6e16891b80 20 client.19920 trim_cache size 0 max 16384 2024-05-27T07:56:57.078 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:56:57.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.076+0000 7f6e07e00700 1 -- 172.21.15.63:0/1762334795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6de80b2b40 msgr2=0x7f6de80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:57.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.076+0000 7f6e07e00700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6de80b2b40 0x7f6de80b5050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6e00107e30 tx=0x7f6dec00b410 comp rx=0 tx=0).stop 2024-05-27T07:56:57.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.076+0000 7f6e07e00700 1 -- 172.21.15.63:0/1762334795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e001045e0 msgr2=0x7f6e0019d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:57.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.076+0000 7f6e07e00700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e001045e0 0x7f6e0019d060 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f6df4000c00 tx=0x7f6df400f460 comp rx=0 tx=0).stop 2024-05-27T07:56:57.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.076+0000 7f6e07e00700 1 -- 172.21.15.63:0/1762334795 shutdown_connections 2024-05-27T07:56:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.076+0000 7f6e07e00700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6de80b2b40 0x7f6de80b5050 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.076+0000 7f6e07e00700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e00109860 0x7f6e001a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.076+0000 7f6e07e00700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e001045e0 0x7f6e0019d060 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.076+0000 7f6e07e00700 1 --2- 172.21.15.63:0/1762334795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e00108f10 0x7f6e0019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.076+0000 7f6e07e00700 1 -- 172.21.15.63:0/1762334795 >> 172.21.15.63:0/1762334795 conn(0x7f6e00100030 msgr2=0x7f6e0010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.077+0000 7f6e07e00700 1 -- 172.21.15.63:0/1762334795 shutdown_connections 2024-05-27T07:56:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:56:57.077+0000 7f6e07e00700 1 -- 172.21.15.63:0/1762334795 wait complete. 2024-05-27T07:56:58.484 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:56:58.996 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:59.157 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:58 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:59.157 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:58 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:59.157 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:58 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:56:59.157 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:58 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:56:59.157 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:58 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:59.158 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:56:58 smithi063 ceph-mon[31613]: pgmap v739: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.0 MiB/s wr, 14 op/s 2024-05-27T07:56:59.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:58 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:59.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:58 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:59.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:58 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:56:59.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:58 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:56:59.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:58 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:59.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:56:58 smithi086 ceph-mon[34687]: pgmap v739: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.0 MiB/s wr, 14 op/s 2024-05-27T07:56:59.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:58 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:59.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:58 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:59.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:58 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:56:59.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:58 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:56:59.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:58 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:56:59.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:56:58 smithi149 ceph-mon[35408]: pgmap v739: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.0 MiB/s wr, 14 op/s 2024-05-27T07:57:01.201 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:01 smithi063 ceph-mon[31613]: pgmap v740: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.4 MiB/s wr, 11 op/s 2024-05-27T07:57:01.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:01 smithi086 ceph-mon[34687]: pgmap v740: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.4 MiB/s wr, 11 op/s 2024-05-27T07:57:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.459+0000 7ff4dda00700 1 -- 172.21.15.63:0/2560688020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d8105bd0 msgr2=0x7ff4d810e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.459+0000 7ff4dda00700 1 --2- 172.21.15.63:0/2560688020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d8105bd0 0x7ff4d810e120 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009990 tx=0x7ff4c4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.459+0000 7ff4dda00700 1 -- 172.21.15.63:0/2560688020 shutdown_connections 2024-05-27T07:57:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.459+0000 7ff4dda00700 1 --2- 172.21.15.63:0/2560688020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810e6f0 0x7ff4d8110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.459+0000 7ff4dda00700 1 --2- 172.21.15.63:0/2560688020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d8105280 0x7ff4d8105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.459+0000 7ff4dda00700 1 --2- 172.21.15.63:0/2560688020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d8105bd0 0x7ff4d810e120 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.459+0000 7ff4dda00700 1 -- 172.21.15.63:0/2560688020 >> 172.21.15.63:0/2560688020 conn(0x7ff4d8078ab0 msgr2=0x7ff4d8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.460+0000 7ff4dda00700 1 -- 172.21.15.63:0/2560688020 shutdown_connections 2024-05-27T07:57:01.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.460+0000 7ff4dda00700 1 -- 172.21.15.63:0/2560688020 wait complete. 2024-05-27T07:57:01.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.462+0000 7ff4dda00700 1 Processor -- start 2024-05-27T07:57:01.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.462+0000 7ff4dda00700 1 -- start start 2024-05-27T07:57:01.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:01 smithi149 ceph-mon[35408]: pgmap v740: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.4 MiB/s wr, 11 op/s 2024-05-27T07:57:01.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.462+0000 7ff4dda00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d8105280 0x7ff4d8106190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.462+0000 7ff4dda00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d8105bd0 0x7ff4d81066d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.462+0000 7ff4dda00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810e6f0 0x7ff4d8106c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.462+0000 7ff4dda00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d8114150 con 0x7ff4d8105280 2024-05-27T07:57:01.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.462+0000 7ff4dda00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d8113fd0 con 0x7ff4d8105bd0 2024-05-27T07:57:01.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.462+0000 7ff4dda00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81142d0 con 0x7ff4d810e6f0 2024-05-27T07:57:01.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.463+0000 7ff4d7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d8105280 0x7ff4d8106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.463+0000 7ff4d7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d8105280 0x7ff4d8106190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45436/0 (socket says 172.21.15.63:45436) 2024-05-27T07:57:01.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.463+0000 7ff4d7e00700 1 -- 172.21.15.63:0/1845611355 learned_addr learned my addr 172.21.15.63:0/1845611355 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:01.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.463+0000 7ff4d7400700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d8105bd0 0x7ff4d81066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.463+0000 7ff4dd000700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810e6f0 0x7ff4d8106c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.464+0000 7ff4d7e00700 1 -- 172.21.15.63:0/1845611355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810e6f0 msgr2=0x7ff4d8106c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:01.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.464+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810e6f0 0x7ff4d8106c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.464+0000 7ff4d7e00700 1 -- 172.21.15.63:0/1845611355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d8105bd0 msgr2=0x7ff4d81066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:01.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.464+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d8105bd0 0x7ff4d81066d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.464+0000 7ff4d7e00700 1 -- 172.21.15.63:0/1845611355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d81ad320 con 0x7ff4d8105280 2024-05-27T07:57:01.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.464+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d8105280 0x7ff4d8106190 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800d950 tx=0x7ff4c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:01.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.464+0000 7ff4d4c00700 1 -- 172.21.15.63:0/1845611355 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8016070 con 0x7ff4d8105280 2024-05-27T07:57:01.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.465+0000 7ff4d4c00700 1 -- 172.21.15.63:0/1845611355 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4c800fd20 con 0x7ff4d8105280 2024-05-27T07:57:01.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.465+0000 7ff4dda00700 1 -- 172.21.15.63:0/1845611355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d81ad5b0 con 0x7ff4d8105280 2024-05-27T07:57:01.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.465+0000 7ff4d4c00700 1 -- 172.21.15.63:0/1845611355 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c800e3b0 con 0x7ff4d8105280 2024-05-27T07:57:01.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.465+0000 7ff4dda00700 1 -- 172.21.15.63:0/1845611355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d81adb40 con 0x7ff4d8105280 2024-05-27T07:57:01.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.471+0000 7ff4d4c00700 1 -- 172.21.15.63:0/1845611355 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff4c8019070 con 0x7ff4d8105280 2024-05-27T07:57:01.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.471+0000 7ff4edae0b80 1 Processor -- start 2024-05-27T07:57:01.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.471+0000 7ff4edae0b80 1 -- start start 2024-05-27T07:57:01.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.472+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be72ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.472+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 0x56070be7dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.472+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be80020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.472+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56070bda1e40 con 0x56070be73820 2024-05-27T07:57:01.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.472+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56070bd919e0 con 0x56070bcf4e40 2024-05-27T07:57:01.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.472+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56070bd95a20 con 0x56070be73410 2024-05-27T07:57:01.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.472+0000 7ff4d7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be80020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4d7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be80020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45450/0 (socket says 172.21.15.63:45450) 2024-05-27T07:57:01.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4d7400700 1 -- 172.21.15.63:0/35473550 learned_addr learned my addr 172.21.15.63:0/35473550 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:01.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4dd000700 1 --2- 172.21.15.63:0/35473550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be72ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/35473550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 0x56070be7dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4d7400700 1 -- 172.21.15.63:0/35473550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 msgr2=0x56070be7dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:01.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4d7400700 1 --2- 172.21.15.63:0/35473550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 0x56070be7dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4d7400700 1 -- 172.21.15.63:0/35473550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 msgr2=0x56070be72ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:01.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4d7400700 1 --2- 172.21.15.63:0/35473550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be72ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4d7400700 1 -- 172.21.15.63:0/35473550 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56070bd5a420 con 0x56070be73820 2024-05-27T07:57:01.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/35473550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 0x56070be7dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:01.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.473+0000 7ff4d4c00700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4bc0b2a70 0x7ff4bc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.474+0000 7ff4d4c00700 1 -- 172.21.15.63:0/1845611355 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7ff4c80e27f0 con 0x7ff4d8105280 2024-05-27T07:57:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.474+0000 7ff4d7400700 1 --2- 172.21.15.63:0/35473550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be80020 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4015000 tx=0x7ff4c4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c039f080b6d49c4f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.474+0000 7ff4dd000700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4bc0b2a70 0x7ff4bc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.474+0000 7ff4cd600700 1 -- 172.21.15.63:0/35473550 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4005390 con 0x56070be73820 2024-05-27T07:57:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.474+0000 7ff4cd600700 1 -- 172.21.15.63:0/35473550 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4c4005530 con 0x56070be73820 2024-05-27T07:57:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.475+0000 7ff4cd600700 1 -- 172.21.15.63:0/35473550 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400bc30 con 0x56070be73820 2024-05-27T07:57:01.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.476+0000 7ff4edae0b80 1 -- 172.21.15.63:0/35473550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 msgr2=0x56070be80020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:01.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.476+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/35473550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be80020 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4015000 tx=0x7ff4c4005f00 comp rx=0 tx=0).stop 2024-05-27T07:57:01.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.476+0000 7ff4dd000700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4bc0b2a70 0x7ff4bc0b4f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x56070be6f690 tx=0x7ff4d0009100 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:01.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.476+0000 7ff4edae0b80 1 -- 172.21.15.63:0/35473550 shutdown_connections 2024-05-27T07:57:01.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.476+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/35473550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be80020 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.476+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/35473550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be72ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.476+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/35473550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 0x56070be7dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.476+0000 7ff4edae0b80 1 -- 172.21.15.63:0/35473550 >> 172.21.15.63:0/35473550 conn(0x56070bd54fc0 msgr2=0x56070be6ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.476+0000 7ff4edae0b80 1 -- 172.21.15.63:0/35473550 shutdown_connections 2024-05-27T07:57:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.476+0000 7ff4edae0b80 1 -- 172.21.15.63:0/35473550 wait complete. 2024-05-27T07:57:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.477+0000 7ff4edae0b80 1 -- start start 2024-05-27T07:57:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.477+0000 7ff4edae0b80 1 Processor -- start 2024-05-27T07:57:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.478+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be7d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.478+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73410 0x56070be7d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.478+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73820 0x56070be7df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.478+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56070bc97fa0 con 0x56070be73410 2024-05-27T07:57:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.478+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56070bd32760 con 0x56070bcf4e40 2024-05-27T07:57:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.478+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56070be80920 con 0x56070be73820 2024-05-27T07:57:01.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.478+0000 7ff4edae0b80 10 client.0 _ll_register_callbacks cb 0x56070bd3bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:01.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.478+0000 7ff4d7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73410 0x56070be7d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.479+0000 7ff4d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be7d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.479+0000 7ff4d7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be7d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:50460/0 (socket says 172.21.15.63:50460) 2024-05-27T07:57:01.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.479+0000 7ff4d7e00700 1 -- 172.21.15.63:0/1333798656 learned_addr learned my addr 172.21.15.63:0/1333798656 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:01.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.479+0000 7ff4dd000700 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73820 0x56070be7df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.479+0000 7ff4d7400700 1 -- 172.21.15.63:0/1333798656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73820 msgr2=0x56070be7df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.479+0000 7ff4d7400700 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73820 0x56070be7df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.479+0000 7ff4d7400700 1 -- 172.21.15.63:0/1333798656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 msgr2=0x56070be7d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.479+0000 7ff4d7400700 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be7d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.479+0000 7ff4d7400700 1 -- 172.21.15.63:0/1333798656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56070bce18d0 con 0x56070be73410 2024-05-27T07:57:01.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.480+0000 7ff4d7400700 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73410 0x56070be7d9c0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4024040 tx=0x7ff4c4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4d8f28b4b0deb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:01.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.480+0000 7ff4c2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:01.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.480+0000 7ff4c2000700 1 -- 172.21.15.63:0/1333798656 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c402f020 con 0x56070be73410 2024-05-27T07:57:01.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.480+0000 7ff4c2000700 1 -- 172.21.15.63:0/1333798656 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7ff4c4003a50 con 0x56070be73410 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.480+0000 7ff4edae0b80 1 -- 172.21.15.63:0/1333798656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4c4005530 con 0x56070be73410 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.481+0000 7ff4c2000700 1 -- 172.21.15.63:0/1333798656 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff4c4003c50 con 0x56070be73410 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.481+0000 7ff4edae0b80 10 client.19932 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.481+0000 7ff4edae0b80 1 -- 172.21.15.63:0/1333798656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56070bd62060 con 0x56070be73410 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.481+0000 7ff4c2000700 1 -- 172.21.15.63:0/1333798656 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4c4005530 con 0x56070be73410 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.481+0000 7ff4c2000700 1 -- 172.21.15.63:0/1333798656 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff4c400f790 con 0x56070be73410 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.482+0000 7ff4edae0b80 10 client.19932 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.482+0000 7ff4edae0b80 10 client.19932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.482+0000 7ff4edae0b80 20 client.19932 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.482+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56070be8d620 0x56070be8fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.482+0000 7ff4edae0b80 4 client.19932 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:01.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.482+0000 7ff4edae0b80 1 -- 172.21.15.63:0/1333798656 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56070bda3d90 con 0x56070be8d620 2024-05-27T07:57:01.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.482+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56070be8d620 0x56070be8fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.483+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56070be8d620 0x56070be8fa40 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a31e8ce7c91507d server_cookie=3824ff52a95a03e2 in_seq=0 out_seq=0 2024-05-27T07:57:01.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.483+0000 7ff4c2000700 10 client.19932 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:01.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.485+0000 7ff4c2000700 1 -- 172.21.15.63:0/1333798656 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56070bda3d90 con 0x56070be8d620 2024-05-27T07:57:01.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.485+0000 7ff4c2000700 10 client.19932 handle_command_reply: tid=0 2024-05-27T07:57:01.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.485+0000 7ff4edae0b80 1 client.19932 shutdown 2024-05-27T07:57:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 1 -- 172.21.15.63:0/1333798656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73410 msgr2=0x56070be7d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73410 0x56070be7d9c0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4024040 tx=0x7ff4c4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 1 -- 172.21.15.63:0/1333798656 shutdown_connections 2024-05-27T07:57:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56070be8d620 0x56070be8fa40 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73820 0x56070be7df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be7d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/1333798656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73410 0x56070be7d9c0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 1 -- 172.21.15.63:0/1333798656 >> 172.21.15.63:0/1333798656 conn(0x56070bd54fc0 msgr2=0x56070be6dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4c2000700 0 client.19932 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 1 -- 172.21.15.63:0/1333798656 shutdown_connections 2024-05-27T07:57:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 1 -- 172.21.15.63:0/1333798656 wait complete. 2024-05-27T07:57:01.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.486+0000 7ff4edae0b80 20 client.19932 trim_cache size 0 max 16384 2024-05-27T07:57:01.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.507+0000 7ff4edae0b80 1 Processor -- start 2024-05-27T07:57:01.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.507+0000 7ff4edae0b80 1 -- start start 2024-05-27T07:57:01.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.508+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070bcf4e40 0x56070be89bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.508+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070be73410 0x56070be7f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.508+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be78c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.508+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b9100 con 0x56070be73820 2024-05-27T07:57:01.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.508+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b6990 con 0x56070be73410 2024-05-27T07:57:01.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.508+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b4800 con 0x56070bcf4e40 2024-05-27T07:57:01.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.508+0000 7ff4dd000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be78c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.509+0000 7ff4dd000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be78c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45466/0 (socket says 172.21.15.63:45466) 2024-05-27T07:57:01.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.509+0000 7ff4dd000700 1 -- 172.21.15.63:0/3123680260 learned_addr learned my addr 172.21.15.63:0/3123680260 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:01.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.509+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/3123680260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070bcf4e40 0x56070be89bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.509+0000 7ff4d7400700 1 --2- 172.21.15.63:0/3123680260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070be73410 0x56070be7f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.509+0000 7ff4dd000700 1 -- 172.21.15.63:0/3123680260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070bcf4e40 msgr2=0x56070be89bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:01.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.509+0000 7ff4dd000700 1 --2- 172.21.15.63:0/3123680260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070bcf4e40 0x56070be89bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.509+0000 7ff4dd000700 1 -- 172.21.15.63:0/3123680260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070be73410 msgr2=0x56070be7f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:01.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.509+0000 7ff4dd000700 1 --2- 172.21.15.63:0/3123680260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070be73410 0x56070be7f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.509+0000 7ff4dd000700 1 -- 172.21.15.63:0/3123680260 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56070be691d0 con 0x56070be73820 2024-05-27T07:57:01.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.510+0000 7ff4dd000700 1 --2- 172.21.15.63:0/3123680260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be78c50 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00045a0 tx=0x7ff4d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bb5ec291aabe05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:01.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.510+0000 7ff4ccc00700 1 -- 172.21.15.63:0/3123680260 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0007b90 con 0x56070be73820 2024-05-27T07:57:01.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.510+0000 7ff4ccc00700 1 -- 172.21.15.63:0/3123680260 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4d001c030 con 0x56070be73820 2024-05-27T07:57:01.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.510+0000 7ff4ccc00700 1 -- 172.21.15.63:0/3123680260 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d002c400 con 0x56070be73820 2024-05-27T07:57:01.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.511+0000 7ff4edae0b80 1 -- 172.21.15.63:0/3123680260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 msgr2=0x56070be78c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:01.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.511+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/3123680260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be78c50 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00045a0 tx=0x7ff4d0016000 comp rx=0 tx=0).stop 2024-05-27T07:57:01.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.511+0000 7ff4edae0b80 1 -- 172.21.15.63:0/3123680260 shutdown_connections 2024-05-27T07:57:01.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.511+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/3123680260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be78c50 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.511+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/3123680260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070bcf4e40 0x56070be89bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.511+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/3123680260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070be73410 0x56070be7f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.511+0000 7ff4edae0b80 1 -- 172.21.15.63:0/3123680260 >> 172.21.15.63:0/3123680260 conn(0x56070bd54fc0 msgr2=0x56070be7d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:01.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.512+0000 7ff4edae0b80 1 -- 172.21.15.63:0/3123680260 shutdown_connections 2024-05-27T07:57:01.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.512+0000 7ff4edae0b80 1 -- 172.21.15.63:0/3123680260 wait complete. 2024-05-27T07:57:01.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.512+0000 7ff4edae0b80 1 -- start start 2024-05-27T07:57:01.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.512+0000 7ff4edae0b80 1 Processor -- start 2024-05-27T07:57:01.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.513+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be8f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.513+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 0x56070be8fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.513+0000 7ff4edae0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be8ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.513+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b4e50 con 0x56070be73820 2024-05-27T07:57:01.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.513+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b62c0 con 0x56070bcf4e40 2024-05-27T07:57:01.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.513+0000 7ff4edae0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b5520 con 0x56070be73410 2024-05-27T07:57:01.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.513+0000 7ff4edae0b80 10 client.0 _ll_register_callbacks cb 0x56070bd3bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:01.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.514+0000 7ff4dd000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be8ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.514+0000 7ff4dd000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be8ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45480/0 (socket says 172.21.15.63:45480) 2024-05-27T07:57:01.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.514+0000 7ff4dd000700 1 -- 172.21.15.63:0/2629167025 learned_addr learned my addr 172.21.15.63:0/2629167025 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:01.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.514+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be8f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.514+0000 7ff4d7400700 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 0x56070be8fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.514+0000 7ff4dd000700 1 -- 172.21.15.63:0/2629167025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 msgr2=0x56070be8fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:01.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.514+0000 7ff4dd000700 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 0x56070be8fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.514+0000 7ff4dd000700 1 -- 172.21.15.63:0/2629167025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 msgr2=0x56070be8f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:01.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.514+0000 7ff4dd000700 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be8f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.514+0000 7ff4dd000700 1 -- 172.21.15.63:0/2629167025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56070bef5ed0 con 0x56070be73820 2024-05-27T07:57:01.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.515+0000 7ff4dd000700 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be8ff50 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009710 tx=0x7ff4d0003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3876b89c92ca2ff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:01.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.515+0000 7ff4c2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:01.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.515+0000 7ff4c2000700 1 -- 172.21.15.63:0/2629167025 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0016d10 con 0x56070be73820 2024-05-27T07:57:01.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.515+0000 7ff4c2000700 1 -- 172.21.15.63:0/2629167025 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7ff4d0004ad0 con 0x56070be73820 2024-05-27T07:57:01.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.515+0000 7ff4edae0b80 1 -- 172.21.15.63:0/2629167025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d001c030 con 0x56070be73820 2024-05-27T07:57:01.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.515+0000 7ff4c2000700 1 -- 172.21.15.63:0/2629167025 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff4d0004cd0 con 0x56070be73820 2024-05-27T07:57:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.516+0000 7ff4edae0b80 10 client.19938 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.516+0000 7ff4edae0b80 1 -- 172.21.15.63:0/2629167025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56070bef7130 con 0x56070be73820 2024-05-27T07:57:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.516+0000 7ff4c2000700 1 -- 172.21.15.63:0/2629167025 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4d001c030 con 0x56070be73820 2024-05-27T07:57:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.516+0000 7ff4c2000700 1 -- 172.21.15.63:0/2629167025 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff4d00053d0 con 0x56070be73820 2024-05-27T07:57:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.516+0000 7ff4edae0b80 10 client.19938 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.516+0000 7ff4edae0b80 10 client.19938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.516+0000 7ff4edae0b80 20 client.19938 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.516+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56070bef7780 0x56070bef9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.516+0000 7ff4edae0b80 4 client.19938 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:57:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.516+0000 7ff4edae0b80 1 -- 172.21.15.63:0/2629167025 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56070bd9d570 con 0x56070bef7780 2024-05-27T07:57:01.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.517+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56070bef7780 0x56070bef9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:01.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.518+0000 7ff4d7e00700 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56070bef7780 0x56070bef9ba0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ea6d1f299d1201 server_cookie=824c2af023ff8a92 in_seq=0 out_seq=0 2024-05-27T07:57:01.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.518+0000 7ff4c2000700 10 client.19938 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:01.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.518+0000 7ff4c2000700 1 -- 172.21.15.63:0/2629167025 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56070bd9d570 con 0x56070bef7780 2024-05-27T07:57:01.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.518+0000 7ff4c2000700 10 client.19938 handle_command_reply: tid=0 2024-05-27T07:57:01.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.518+0000 7ff4edae0b80 1 client.19938 shutdown 2024-05-27T07:57:01.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 1 -- 172.21.15.63:0/2629167025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 msgr2=0x56070be8ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:01.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be8ff50 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009710 tx=0x7ff4d0003d70 comp rx=0 tx=0).stop 2024-05-27T07:57:01.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 1 -- 172.21.15.63:0/2629167025 shutdown_connections 2024-05-27T07:57:01.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56070bef7780 0x56070bef9ba0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56070be73820 0x56070be8ff50 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56070bcf4e40 0x56070be8f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 1 --2- 172.21.15.63:0/2629167025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56070be73410 0x56070be8fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 1 -- 172.21.15.63:0/2629167025 >> 172.21.15.63:0/2629167025 conn(0x56070bd54fc0 msgr2=0x56070be6eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:01.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4c2000700 0 client.19938 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:01.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 1 -- 172.21.15.63:0/2629167025 shutdown_connections 2024-05-27T07:57:01.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 1 -- 172.21.15.63:0/2629167025 wait complete. 2024-05-27T07:57:01.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.519+0000 7ff4edae0b80 20 client.19938 trim_cache size 0 max 16384 2024-05-27T07:57:01.522 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:57:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 -- 172.21.15.63:0/1845611355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4bc0b2a70 msgr2=0x7ff4bc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4bc0b2a70 0x7ff4bc0b4f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x56070be6f690 tx=0x7ff4d0009100 comp rx=0 tx=0).stop 2024-05-27T07:57:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 -- 172.21.15.63:0/1845611355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d8105280 msgr2=0x7ff4d8106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d8105280 0x7ff4d8106190 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800d950 tx=0x7ff4c800dc60 comp rx=0 tx=0).stop 2024-05-27T07:57:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 -- 172.21.15.63:0/1845611355 shutdown_connections 2024-05-27T07:57:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4bc0b2a70 0x7ff4bc0b4f80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d810e6f0 0x7ff4d8106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d8105280 0x7ff4d8106190 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 --2- 172.21.15.63:0/1845611355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4d8105bd0 0x7ff4d81066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:01.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 -- 172.21.15.63:0/1845611355 >> 172.21.15.63:0/1845611355 conn(0x7ff4d8078ab0 msgr2=0x7ff4d8102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:01.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 -- 172.21.15.63:0/1845611355 shutdown_connections 2024-05-27T07:57:01.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:01.520+0000 7ff4dda00700 1 -- 172.21.15.63:0/1845611355 wait complete. 2024-05-27T07:57:03.381 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:03 smithi063 ceph-mon[31613]: pgmap v741: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.4 MiB/s wr, 32 op/s 2024-05-27T07:57:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:03 smithi086 ceph-mon[34687]: pgmap v741: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.4 MiB/s wr, 32 op/s 2024-05-27T07:57:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:03 smithi149 ceph-mon[35408]: pgmap v741: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.4 MiB/s wr, 32 op/s 2024-05-27T07:57:03.505 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:57:03.505 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:57:04.030 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:05.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:05 smithi086 ceph-mon[34687]: pgmap v742: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 28 op/s 2024-05-27T07:57:05.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:05 smithi149 ceph-mon[35408]: pgmap v742: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 28 op/s 2024-05-27T07:57:05.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:05 smithi063 ceph-mon[31613]: pgmap v742: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 28 op/s 2024-05-27T07:57:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.133+0000 7f55e6800640 1 -- 172.21.15.63:0/4055871878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55e010e1e0 msgr2=0x7f55e01106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.133+0000 7f55e6800640 1 --2- 172.21.15.63:0/4055871878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55e010e1e0 0x7f55e01106d0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009ca0 tx=0x7f55d802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.133+0000 7f55e6800640 1 -- 172.21.15.63:0/4055871878 shutdown_connections 2024-05-27T07:57:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.133+0000 7f55e6800640 1 --2- 172.21.15.63:0/4055871878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55e010e1e0 0x7f55e01106d0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.133+0000 7f55e6800640 1 --2- 172.21.15.63:0/4055871878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55e010b7f0 0x7f55e010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.133+0000 7f55e6800640 1 --2- 172.21.15.63:0/4055871878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e0104d90 0x7f55e01051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.133+0000 7f55e6800640 1 -- 172.21.15.63:0/4055871878 >> 172.21.15.63:0/4055871878 conn(0x7f55e00782d0 msgr2=0x7f55e0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.134+0000 7f55e6800640 1 -- 172.21.15.63:0/4055871878 shutdown_connections 2024-05-27T07:57:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.134+0000 7f55e6800640 1 -- 172.21.15.63:0/4055871878 wait complete. 2024-05-27T07:57:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.135+0000 7f55e6800640 1 Processor -- start 2024-05-27T07:57:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.135+0000 7f55e6800640 1 -- start start 2024-05-27T07:57:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.136+0000 7f55e6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55e0104d90 0x7f55e0102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.136+0000 7f55e6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e010b7f0 0x7f55e0102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.136+0000 7f55e6800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55e010e1e0 0x7f55e01030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.136+0000 7f55e6800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0113d40 con 0x7f55e0104d90 2024-05-27T07:57:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.136+0000 7f55e6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0113bc0 con 0x7f55e010b7f0 2024-05-27T07:57:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.136+0000 7f55e6800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0113ec0 con 0x7f55e010e1e0 2024-05-27T07:57:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.136+0000 7f55df400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55e0104d90 0x7f55e0102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.136+0000 7f55df400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55e0104d90 0x7f55e0102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45492/0 (socket says 172.21.15.63:45492) 2024-05-27T07:57:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.136+0000 7f55df400640 1 -- 172.21.15.63:0/1865577548 learned_addr learned my addr 172.21.15.63:0/1865577548 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.137+0000 7f55dea00640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e010b7f0 0x7f55e0102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.137+0000 7f55dfe00640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55e010e1e0 0x7f55e01030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.137+0000 7f55df400640 1 -- 172.21.15.63:0/1865577548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55e010e1e0 msgr2=0x7f55e01030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.137+0000 7f55df400640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55e010e1e0 0x7f55e01030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.137+0000 7f55df400640 1 -- 172.21.15.63:0/1865577548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e010b7f0 msgr2=0x7f55e0102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.137+0000 7f55df400640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e010b7f0 0x7f55e0102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.137+0000 7f55df400640 1 -- 172.21.15.63:0/1865577548 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e01a8a80 con 0x7f55e0104d90 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.137+0000 7f55dea00640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e010b7f0 0x7f55e0102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.137+0000 7f55df400640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55e0104d90 0x7f55e0102650 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f55d000e9f0 tx=0x7f55d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.138+0000 7f55d7e00640 1 -- 172.21.15.63:0/1865577548 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d000cd80 con 0x7f55e0104d90 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.138+0000 7f55e6800640 1 -- 172.21.15.63:0/1865577548 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e01a8d70 con 0x7f55e0104d90 2024-05-27T07:57:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.138+0000 7f55d7e00640 1 -- 172.21.15.63:0/1865577548 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55d00108c0 con 0x7f55e0104d90 2024-05-27T07:57:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.138+0000 7f55d7e00640 1 -- 172.21.15.63:0/1865577548 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00187c0 con 0x7f55e0104d90 2024-05-27T07:57:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.138+0000 7f55e6800640 1 -- 172.21.15.63:0/1865577548 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e01a9280 con 0x7f55e0104d90 2024-05-27T07:57:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.142+0000 7f55ed797740 1 Processor -- start 2024-05-27T07:57:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.143+0000 7f55ed797740 1 -- start start 2024-05-27T07:57:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.143+0000 7f55ed797740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920701b0 0x564b92412860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.143+0000 7f55ed797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 0x564b92412da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.143+0000 7f55ed797740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924132e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.143+0000 7f55ed797740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564b92383b90 con 0x564b9204f4d0 2024-05-27T07:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.143+0000 7f55ed797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564b92373730 con 0x564b920472f0 2024-05-27T07:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.143+0000 7f55ed797740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564b92377770 con 0x564b920701b0 2024-05-27T07:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.144+0000 7f55dea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.144+0000 7f55dea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924132e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45504/0 (socket says 172.21.15.63:45504) 2024-05-27T07:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.144+0000 7f55dea00640 1 -- 172.21.15.63:0/640071668 learned_addr learned my addr 172.21.15.63:0/640071668 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.144+0000 7f55dfe00640 1 --2- 172.21.15.63:0/640071668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920701b0 0x564b92412860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.144+0000 7f55d7e00640 1 -- 172.21.15.63:0/1865577548 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f55d0010430 con 0x7f55e0104d90 2024-05-27T07:57:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.145+0000 7f55dea00640 1 -- 172.21.15.63:0/640071668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920701b0 msgr2=0x564b92412860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.145+0000 7f55dea00640 1 --2- 172.21.15.63:0/640071668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920701b0 0x564b92412860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.145+0000 7f55dea00640 1 -- 172.21.15.63:0/640071668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 msgr2=0x564b92412da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.145+0000 7f55dea00640 1 --2- 172.21.15.63:0/640071668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 0x564b92412da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.145+0000 7f55dea00640 1 -- 172.21.15.63:0/640071668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9230c8b0 con 0x564b9204f4d0 2024-05-27T07:57:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.145+0000 7f55dea00640 1 --2- 172.21.15.63:0/640071668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924132e0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009c70 tx=0x7f55cc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a5c46d1571fa25 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.145+0000 7f55d4c00640 1 -- 172.21.15.63:0/640071668 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc012450 con 0x564b9204f4d0 2024-05-27T07:57:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.146+0000 7f55d4c00640 1 -- 172.21.15.63:0/640071668 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55cc012a80 con 0x564b9204f4d0 2024-05-27T07:57:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.146+0000 7f55d4c00640 1 -- 172.21.15.63:0/640071668 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc00f9b0 con 0x564b9204f4d0 2024-05-27T07:57:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.146+0000 7f55d7e00640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55c40b29d0 0x7f55c40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.147+0000 7f55d7e00640 1 -- 172.21.15.63:0/1865577548 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f55d00e2e60 con 0x7f55e0104d90 2024-05-27T07:57:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.147+0000 7f55dfe00640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55c40b29d0 0x7f55c40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.147+0000 7f55ed797740 1 -- 172.21.15.63:0/640071668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 msgr2=0x564b924132e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.147+0000 7f55ed797740 1 --2- 172.21.15.63:0/640071668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924132e0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009c70 tx=0x7f55cc0117b0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.147+0000 7f55ed797740 1 -- 172.21.15.63:0/640071668 shutdown_connections 2024-05-27T07:57:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.147+0000 7f55ed797740 1 --2- 172.21.15.63:0/640071668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924132e0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.147+0000 7f55ed797740 1 --2- 172.21.15.63:0/640071668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 0x564b92412da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.147+0000 7f55ed797740 1 --2- 172.21.15.63:0/640071668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920701b0 0x564b92412860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.147+0000 7f55ed797740 1 -- 172.21.15.63:0/640071668 >> 172.21.15.63:0/640071668 conn(0x564b92064300 msgr2=0x564b9240f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.147+0000 7f55ed797740 1 -- 172.21.15.63:0/640071668 shutdown_connections 2024-05-27T07:57:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.148+0000 7f55ed797740 1 -- 172.21.15.63:0/640071668 wait complete. 2024-05-27T07:57:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.148+0000 7f55dfe00640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55c40b29d0 0x7f55c40b4ee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f55d8005e30 tx=0x7f55d8005d80 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.148+0000 7f55ed797740 1 -- start start 2024-05-27T07:57:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.149+0000 7f55ed797740 1 Processor -- start 2024-05-27T07:57:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.149+0000 7f55ed797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 0x564b9241c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.149+0000 7f55ed797740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b9204f4d0 0x564b9241cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.149+0000 7f55ed797740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920701b0 0x564b92428cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.149+0000 7f55ed797740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564b92059790 con 0x564b920701b0 2024-05-27T07:57:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.149+0000 7f55ed797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564b9231bbc0 con 0x564b920472f0 2024-05-27T07:57:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.149+0000 7f55ed797740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564b9241f990 con 0x564b9204f4d0 2024-05-27T07:57:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.150+0000 7f55ed797740 10 client.0 _ll_register_callbacks cb 0x564b922f9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.150+0000 7f55df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 0x564b9241c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.150+0000 7f55dfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920701b0 0x564b92428cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.150+0000 7f55df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 0x564b9241c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:50512/0 (socket says 172.21.15.63:50512) 2024-05-27T07:57:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.150+0000 7f55df400640 1 -- 172.21.15.63:0/2371215270 learned_addr learned my addr 172.21.15.63:0/2371215270 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.150+0000 7f55dea00640 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b9204f4d0 0x564b9241cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.151+0000 7f55df400640 1 -- 172.21.15.63:0/2371215270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b9204f4d0 msgr2=0x564b9241cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.151+0000 7f55df400640 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b9204f4d0 0x564b9241cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.151+0000 7f55df400640 1 -- 172.21.15.63:0/2371215270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920701b0 msgr2=0x564b92428cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.151+0000 7f55df400640 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920701b0 0x564b92428cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.151+0000 7f55df400640 1 -- 172.21.15.63:0/2371215270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b92088210 con 0x564b920472f0 2024-05-27T07:57:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.151+0000 7f55df400640 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 0x564b9241c9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f55d00e02a0 tx=0x7f55d0010810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36d606975f3e29d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.151+0000 7f55c9600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T07:57:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.152+0000 7f55c9600640 1 -- 172.21.15.63:0/2371215270 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00ae130 con 0x564b920472f0 2024-05-27T07:57:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.152+0000 7f55c9600640 1 -- 172.21.15.63:0/2371215270 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f55d00a6370 con 0x564b920472f0 2024-05-27T07:57:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.152+0000 7f55ed797740 1 -- 172.21.15.63:0/2371215270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55cc012a80 con 0x564b920472f0 2024-05-27T07:57:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.153+0000 7f55c9600640 1 -- 172.21.15.63:0/2371215270 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55d00b2be0 con 0x564b920472f0 2024-05-27T07:57:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.153+0000 7f55ed797740 10 client.39161 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.153+0000 7f55ed797740 1 -- 172.21.15.63:0/2371215270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564b92154620 con 0x564b920472f0 2024-05-27T07:57:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.153+0000 7f55c9600640 1 -- 172.21.15.63:0/2371215270 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55cc012a80 con 0x564b920472f0 2024-05-27T07:57:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.154+0000 7f55c9600640 1 -- 172.21.15.63:0/2371215270 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f55d00c4160 con 0x564b920472f0 2024-05-27T07:57:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.154+0000 7f55ed797740 10 client.39161 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.154+0000 7f55ed797740 10 client.39161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.154+0000 7f55ed797740 20 client.39161 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.154+0000 7f55ed797740 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564b9202ed80 0x564b9242ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.154+0000 7f55ed797740 4 client.39161 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.154+0000 7f55ed797740 1 -- 172.21.15.63:0/2371215270 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b92385ae0 con 0x564b9202ed80 2024-05-27T07:57:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.155+0000 7f55dea00640 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564b9202ed80 0x564b9242ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.156+0000 7f55dea00640 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564b9202ed80 0x564b9242ec00 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=543e40f7b53f0cb3 server_cookie=85f09b73571b1d2c in_seq=0 out_seq=0 2024-05-27T07:57:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.156+0000 7f55c9600640 10 client.39161 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.158+0000 7f55c9600640 1 -- 172.21.15.63:0/2371215270 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564b92385ae0 con 0x564b9202ed80 2024-05-27T07:57:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.158+0000 7f55c9600640 10 client.39161 handle_command_reply: tid=0 2024-05-27T07:57:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.158+0000 7f55ed797740 1 client.39161 shutdown 2024-05-27T07:57:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.158+0000 7f55ed797740 1 -- 172.21.15.63:0/2371215270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 msgr2=0x564b9241c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.158+0000 7f55ed797740 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 0x564b9241c9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f55d00e02a0 tx=0x7f55d0010810 comp rx=0 tx=0).stop 2024-05-27T07:57:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.159+0000 7f55ed797740 1 -- 172.21.15.63:0/2371215270 shutdown_connections 2024-05-27T07:57:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.159+0000 7f55ed797740 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564b9202ed80 0x564b9242ec00 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.159+0000 7f55ed797740 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920701b0 0x564b92428cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.159+0000 7f55ed797740 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b9204f4d0 0x564b9241cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.159+0000 7f55c9600640 0 client.39161 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.159+0000 7f55ed797740 1 --2- 172.21.15.63:0/2371215270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920472f0 0x564b9241c9f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.159+0000 7f55ed797740 1 -- 172.21.15.63:0/2371215270 >> 172.21.15.63:0/2371215270 conn(0x564b92064300 msgr2=0x564b92414580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.159+0000 7f55ed797740 1 -- 172.21.15.63:0/2371215270 shutdown_connections 2024-05-27T07:57:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.159+0000 7f55ed797740 1 -- 172.21.15.63:0/2371215270 wait complete. 2024-05-27T07:57:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.159+0000 7f55ed797740 20 client.39161 trim_cache size 0 max 16384 2024-05-27T07:57:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.182+0000 7f55ed797740 1 Processor -- start 2024-05-27T07:57:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.182+0000 7f55ed797740 1 -- start start 2024-05-27T07:57:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.183+0000 7f55ed797740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920472f0 0x564b924181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.183+0000 7f55ed797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b9204f4d0 0x564b9240e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.183+0000 7f55ed797740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920701b0 0x564b924131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.183+0000 7f55ed797740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b5d90 con 0x564b920472f0 2024-05-27T07:57:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.183+0000 7f55ed797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b3660 con 0x564b9204f4d0 2024-05-27T07:57:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.183+0000 7f55ed797740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b14b0 con 0x564b920701b0 2024-05-27T07:57:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.183+0000 7f55df400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920472f0 0x564b924181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.183+0000 7f55df400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920472f0 0x564b924181f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45530/0 (socket says 172.21.15.63:45530) 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.183+0000 7f55df400640 1 -- 172.21.15.63:0/1654580667 learned_addr learned my addr 172.21.15.63:0/1654580667 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.184+0000 7f55dfe00640 1 --2- 172.21.15.63:0/1654580667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920701b0 0x564b924131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.184+0000 7f55dea00640 1 --2- 172.21.15.63:0/1654580667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b9204f4d0 0x564b9240e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.184+0000 7f55df400640 1 -- 172.21.15.63:0/1654580667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920701b0 msgr2=0x564b924131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.184+0000 7f55df400640 1 --2- 172.21.15.63:0/1654580667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920701b0 0x564b924131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.184+0000 7f55df400640 1 -- 172.21.15.63:0/1654580667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b9204f4d0 msgr2=0x564b9240e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.184+0000 7f55df400640 1 --2- 172.21.15.63:0/1654580667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b9204f4d0 0x564b9240e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.184+0000 7f55df400640 1 -- 172.21.15.63:0/1654580667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b92410130 con 0x564b920472f0 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.184+0000 7f55df400640 1 --2- 172.21.15.63:0/1654580667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920472f0 0x564b924181f0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f55d00e07f0 tx=0x7f55d00a9f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53a5365bb8d1b2a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.184+0000 7f55cbe00640 1 -- 172.21.15.63:0/1654580667 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00ae130 con 0x564b920472f0 2024-05-27T07:57:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.185+0000 7f55cbe00640 1 -- 172.21.15.63:0/1654580667 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55d00a6370 con 0x564b920472f0 2024-05-27T07:57:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.185+0000 7f55cbe00640 1 -- 172.21.15.63:0/1654580667 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00a5a70 con 0x564b920472f0 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.186+0000 7f55ed797740 1 -- 172.21.15.63:0/1654580667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920472f0 msgr2=0x564b924181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.186+0000 7f55ed797740 1 --2- 172.21.15.63:0/1654580667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920472f0 0x564b924181f0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f55d00e07f0 tx=0x7f55d00a9f00 comp rx=0 tx=0).stop 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.186+0000 7f55ed797740 1 -- 172.21.15.63:0/1654580667 shutdown_connections 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.186+0000 7f55ed797740 1 --2- 172.21.15.63:0/1654580667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920701b0 0x564b924131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.186+0000 7f55ed797740 1 --2- 172.21.15.63:0/1654580667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b9204f4d0 0x564b9240e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.186+0000 7f55ed797740 1 --2- 172.21.15.63:0/1654580667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b920472f0 0x564b924181f0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.186+0000 7f55ed797740 1 -- 172.21.15.63:0/1654580667 >> 172.21.15.63:0/1654580667 conn(0x564b92064300 msgr2=0x564b9241cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.186+0000 7f55ed797740 1 -- 172.21.15.63:0/1654580667 shutdown_connections 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.186+0000 7f55ed797740 1 -- 172.21.15.63:0/1654580667 wait complete. 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.188+0000 7f55ed797740 1 -- start start 2024-05-27T07:57:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.188+0000 7f55ed797740 1 Processor -- start 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.189+0000 7f55ed797740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920472f0 0x564b92416d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.189+0000 7f55ed797740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.189+0000 7f55ed797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920701b0 0x564b92423c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.189+0000 7f55ed797740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b1b20 con 0x564b9204f4d0 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.189+0000 7f55ed797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b2f90 con 0x564b920701b0 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.189+0000 7f55ed797740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f55e01b21f0 con 0x564b920472f0 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.189+0000 7f55ed797740 10 client.0 _ll_register_callbacks cb 0x7f55d000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.189+0000 7f55dea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.190+0000 7f55dea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924172c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45532/0 (socket says 172.21.15.63:45532) 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.190+0000 7f55dea00640 1 -- 172.21.15.63:0/1544882598 learned_addr learned my addr 172.21.15.63:0/1544882598 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.190+0000 7f55df400640 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920472f0 0x564b92416d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.190+0000 7f55dfe00640 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920701b0 0x564b92423c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.190+0000 7f55dea00640 1 -- 172.21.15.63:0/1544882598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920472f0 msgr2=0x564b92416d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.190+0000 7f55dea00640 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920472f0 0x564b92416d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.190+0000 7f55dea00640 1 -- 172.21.15.63:0/1544882598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920701b0 msgr2=0x564b92423c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.190+0000 7f55dea00640 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920701b0 0x564b92423c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.190+0000 7f55dea00640 1 -- 172.21.15.63:0/1544882598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b92424160 con 0x564b9204f4d0 2024-05-27T07:57:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.190+0000 7f55df400640 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920472f0 0x564b92416d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.191+0000 7f55dea00640 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924172c0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f55cc01c040 tx=0x7f55cc011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445c86ef1bd031f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.191+0000 7f55c9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.191+0000 7f55c9600640 1 -- 172.21.15.63:0/1544882598 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc009530 con 0x564b9204f4d0 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.191+0000 7f55c9600640 1 -- 172.21.15.63:0/1544882598 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7f55cc0050b0 con 0x564b9204f4d0 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.191+0000 7f55ed797740 1 -- 172.21.15.63:0/1544882598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d00a6370 con 0x564b9204f4d0 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.191+0000 7f55c9600640 1 -- 172.21.15.63:0/1544882598 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55cc03f050 con 0x564b9204f4d0 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.191+0000 7f55ed797740 10 client.19956 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.191+0000 7f55ed797740 1 -- 172.21.15.63:0/1544882598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564b924253f0 con 0x564b9204f4d0 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.192+0000 7f55c9600640 1 -- 172.21.15.63:0/1544882598 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55d00a6370 con 0x564b9204f4d0 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.192+0000 7f55c9600640 1 -- 172.21.15.63:0/1544882598 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f55cc00f6b0 con 0x564b9204f4d0 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.192+0000 7f55ed797740 10 client.19956 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.192+0000 7f55ed797740 10 client.19956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.192+0000 7f55ed797740 20 client.19956 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.192+0000 7f55ed797740 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564b920041c0 0x564b92499250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.192+0000 7f55ed797740 4 client.19956 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:57:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.192+0000 7f55ed797740 1 -- 172.21.15.63:0/1544882598 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b9237f2c0 con 0x564b920041c0 2024-05-27T07:57:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.193+0000 7f55df400640 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564b920041c0 0x564b92499250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.194+0000 7f55df400640 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564b920041c0 0x564b92499250 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9a113339566b994 server_cookie=28518ce762adde13 in_seq=0 out_seq=0 2024-05-27T07:57:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.194+0000 7f55c9600640 10 client.19956 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55c9600640 1 -- 172.21.15.63:0/1544882598 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564b9237f2c0 con 0x564b920041c0 2024-05-27T07:57:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55c9600640 10 client.19956 handle_command_reply: tid=0 2024-05-27T07:57:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 client.19956 shutdown 2024-05-27T07:57:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 -- 172.21.15.63:0/1544882598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 msgr2=0x564b924172c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924172c0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f55cc01c040 tx=0x7f55cc011df0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 -- 172.21.15.63:0/1544882598 shutdown_connections 2024-05-27T07:57:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564b920041c0 0x564b92499250 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564b920701b0 0x564b92423c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b9204f4d0 0x564b924172c0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55c9600640 0 client.19956 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 --2- 172.21.15.63:0/1544882598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564b920472f0 0x564b92416d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 -- 172.21.15.63:0/1544882598 >> 172.21.15.63:0/1544882598 conn(0x564b92064300 msgr2=0x564b92413160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 -- 172.21.15.63:0/1544882598 shutdown_connections 2024-05-27T07:57:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.234+0000 7f55ed797740 1 -- 172.21.15.63:0/1544882598 wait complete. 2024-05-27T07:57:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.235+0000 7f55ed797740 20 client.19956 trim_cache size 0 max 16384 2024-05-27T07:57:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:57:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:57:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:57:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.235+0000 7f55e6800640 1 -- 172.21.15.63:0/1865577548 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55c40b29d0 msgr2=0x7f55c40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.235+0000 7f55e6800640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55c40b29d0 0x7f55c40b4ee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f55d8005e30 tx=0x7f55d8005d80 comp rx=0 tx=0).stop 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.235+0000 7f55e6800640 1 -- 172.21.15.63:0/1865577548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55e0104d90 msgr2=0x7f55e0102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.235+0000 7f55e6800640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55e0104d90 0x7f55e0102650 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f55d000e9f0 tx=0x7f55d000eeb0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.236+0000 7f55e6800640 1 -- 172.21.15.63:0/1865577548 shutdown_connections 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.236+0000 7f55e6800640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55c40b29d0 0x7f55c40b4ee0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.236+0000 7f55e6800640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55e010e1e0 0x7f55e01030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.236+0000 7f55e6800640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e010b7f0 0x7f55e0102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.236+0000 7f55e6800640 1 --2- 172.21.15.63:0/1865577548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55e0104d90 0x7f55e0102650 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.236+0000 7f55e6800640 1 -- 172.21.15.63:0/1865577548 >> 172.21.15.63:0/1865577548 conn(0x7f55e00782d0 msgr2=0x7f55e01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.236+0000 7f55e6800640 1 -- 172.21.15.63:0/1865577548 shutdown_connections 2024-05-27T07:57:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:06.236+0000 7f55e6800640 1 -- 172.21.15.63:0/1865577548 wait complete. 2024-05-27T07:57:06.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.557+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1204829318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e8110ab0 msgr2=0x7ff5e8112ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.557+0000 7ff5e7e00700 1 --2- 172.21.15.63:0/1204829318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e8110ab0 0x7ff5e8112ed0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009990 tx=0x7ff5e0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.558+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1204829318 shutdown_connections 2024-05-27T07:57:06.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.558+0000 7ff5e7e00700 1 --2- 172.21.15.63:0/1204829318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e8110ab0 0x7ff5e8112ed0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.558+0000 7ff5e7e00700 1 --2- 172.21.15.63:0/1204829318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5e8102f00 0x7ff5e8103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.558+0000 7ff5e7e00700 1 --2- 172.21.15.63:0/1204829318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e8103850 0x7ff5e8110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.558+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1204829318 >> 172.21.15.63:0/1204829318 conn(0x7ff5e8078af0 msgr2=0x7ff5e8078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.558+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1204829318 shutdown_connections 2024-05-27T07:57:06.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.558+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1204829318 wait complete. 2024-05-27T07:57:06.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.559+0000 7ff5e7e00700 1 Processor -- start 2024-05-27T07:57:06.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.559+0000 7ff5e7e00700 1 -- start start 2024-05-27T07:57:06.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.560+0000 7ff5e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e8102f00 0x7ff5e81a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.560+0000 7ff5e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e8103850 0x7ff5e81a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.560+0000 7ff5e7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5e8110ab0 0x7ff5e819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.560+0000 7ff5e7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e8116360 con 0x7ff5e8103850 2024-05-27T07:57:06.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.560+0000 7ff5e7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81161e0 con 0x7ff5e8102f00 2024-05-27T07:57:06.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.560+0000 7ff5e7e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81164e0 con 0x7ff5e8110ab0 2024-05-27T07:57:06.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.561+0000 7ff5d6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e8103850 0x7ff5e81a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.561+0000 7ff5e6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e8102f00 0x7ff5e81a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.561+0000 7ff5d6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e8103850 0x7ff5e81a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45540/0 (socket says 172.21.15.63:45540) 2024-05-27T07:57:06.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.561+0000 7ff5d6000700 1 -- 172.21.15.63:0/1515594326 learned_addr learned my addr 172.21.15.63:0/1515594326 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:06.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.561+0000 7ff5e7400700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5e8110ab0 0x7ff5e819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.561+0000 7ff5d6000700 1 -- 172.21.15.63:0/1515594326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5e8110ab0 msgr2=0x7ff5e819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.561+0000 7ff5d6000700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5e8110ab0 0x7ff5e819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.561+0000 7ff5d6000700 1 -- 172.21.15.63:0/1515594326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e8102f00 msgr2=0x7ff5e81a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.561+0000 7ff5d6000700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e8102f00 0x7ff5e81a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.561+0000 7ff5d6000700 1 -- 172.21.15.63:0/1515594326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e819d1e0 con 0x7ff5e8103850 2024-05-27T07:57:06.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.562+0000 7ff5d6000700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e8103850 0x7ff5e81a2e20 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000d950 tx=0x7ff5d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.562+0000 7ff5dfe00700 1 -- 172.21.15.63:0/1515594326 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0016070 con 0x7ff5e8103850 2024-05-27T07:57:06.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.562+0000 7ff5dfe00700 1 -- 172.21.15.63:0/1515594326 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5d0004830 con 0x7ff5e8103850 2024-05-27T07:57:06.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.562+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1515594326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e819d4d0 con 0x7ff5e8103850 2024-05-27T07:57:06.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.562+0000 7ff5dfe00700 1 -- 172.21.15.63:0/1515594326 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00053e0 con 0x7ff5e8103850 2024-05-27T07:57:06.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.563+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1515594326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e81a9770 con 0x7ff5e8103850 2024-05-27T07:57:06.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.567+0000 7ff5dfe00700 1 -- 172.21.15.63:0/1515594326 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff5d0005580 con 0x7ff5e8103850 2024-05-27T07:57:06.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.568+0000 7ff5fc83cb80 1 Processor -- start 2024-05-27T07:57:06.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.568+0000 7ff5fc83cb80 1 -- start start 2024-05-27T07:57:06.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.569+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1c59a70 0x5606f1ced7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.569+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cedd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.569+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1cf7340 0x5606f1cf9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.569+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5606f1c1aec0 con 0x5606f1b6dec0 2024-05-27T07:57:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.569+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5606f1c0aa60 con 0x5606f1c59a70 2024-05-27T07:57:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.569+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5606f1c0eaa0 con 0x5606f1cf7340 2024-05-27T07:57:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.569+0000 7ff5e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cedd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.569+0000 7ff5dfe00700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff5c80b29f0 0x7ff5c80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.569+0000 7ff5e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cedd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45552/0 (socket says 172.21.15.63:45552) 2024-05-27T07:57:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.569+0000 7ff5e6a00700 1 -- 172.21.15.63:0/642466884 learned_addr learned my addr 172.21.15.63:0/642466884 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.570+0000 7ff5d6000700 1 --2- 172.21.15.63:0/642466884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1cf7340 0x5606f1cf9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.570+0000 7ff5e7400700 1 --2- 172.21.15.63:0/642466884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1c59a70 0x5606f1ced7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.570+0000 7ff5dfe00700 1 -- 172.21.15.63:0/1515594326 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7ff5d00e27a0 con 0x7ff5e8103850 2024-05-27T07:57:06.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.570+0000 7ff5e6a00700 1 -- 172.21.15.63:0/642466884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1cf7340 msgr2=0x5606f1cf9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.570+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/642466884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1cf7340 0x5606f1cf9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.570+0000 7ff5e7400700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff5c80b29f0 0x7ff5c80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.570+0000 7ff5e6a00700 1 -- 172.21.15.63:0/642466884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1c59a70 msgr2=0x5606f1ced7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.570+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/642466884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1c59a70 0x5606f1ced7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.570+0000 7ff5e6a00700 1 -- 172.21.15.63:0/642466884 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f1bd34a0 con 0x5606f1b6dec0 2024-05-27T07:57:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.570+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/642466884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cedd10 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8011600 tx=0x7ff5d8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e08053ca901d638 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.571+0000 7ff5dcc00700 1 -- 172.21.15.63:0/642466884 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8019030 con 0x5606f1b6dec0 2024-05-27T07:57:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.571+0000 7ff5dcc00700 1 -- 172.21.15.63:0/642466884 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5d8004020 con 0x5606f1b6dec0 2024-05-27T07:57:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.571+0000 7ff5dcc00700 1 -- 172.21.15.63:0/642466884 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80093a0 con 0x5606f1b6dec0 2024-05-27T07:57:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.572+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/642466884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 msgr2=0x5606f1cedd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.572+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/642466884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cedd10 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8011600 tx=0x7ff5d8009920 comp rx=0 tx=0).stop 2024-05-27T07:57:06.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.572+0000 7ff5e7400700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff5c80b29f0 0x7ff5c80b4f00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5606f1ce9b10 tx=0x7ff5e0005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.572+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/642466884 shutdown_connections 2024-05-27T07:57:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.572+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/642466884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1cf7340 0x5606f1cf9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.572+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/642466884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1c59a70 0x5606f1ced7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.572+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/642466884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cedd10 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.573+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/642466884 >> 172.21.15.63:0/642466884 conn(0x5606f1c07410 msgr2=0x5606f1ce8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.573+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/642466884 shutdown_connections 2024-05-27T07:57:06.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.573+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/642466884 wait complete. 2024-05-27T07:57:06.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.573+0000 7ff5fc83cb80 1 -- start start 2024-05-27T07:57:06.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.574+0000 7ff5fc83cb80 1 Processor -- start 2024-05-27T07:57:06.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.574+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cf4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.574+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1cf47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.574+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1cf4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.574+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5606f1b11020 con 0x5606f1b6dec0 2024-05-27T07:57:06.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.574+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5606f1bab7e0 con 0x5606f1cf7340 2024-05-27T07:57:06.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.574+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5606f1cf9e80 con 0x5606f1c59a70 2024-05-27T07:57:06.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.574+0000 7ff5fc83cb80 10 client.0 _ll_register_callbacks cb 0x5606f1bb4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:06.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.575+0000 7ff5e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cf4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.575+0000 7ff5e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cf4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45564/0 (socket says 172.21.15.63:45564) 2024-05-27T07:57:06.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.575+0000 7ff5e6a00700 1 -- 172.21.15.63:0/2410721014 learned_addr learned my addr 172.21.15.63:0/2410721014 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:06.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.575+0000 7ff5d6000700 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1cf47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.575+0000 7ff5e7400700 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1cf4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.575+0000 7ff5e6a00700 1 -- 172.21.15.63:0/2410721014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 msgr2=0x5606f1cf47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.575+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1cf47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.575+0000 7ff5e6a00700 1 -- 172.21.15.63:0/2410721014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 msgr2=0x5606f1cf4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.575+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1cf4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.575+0000 7ff5e6a00700 1 -- 172.21.15.63:0/2410721014 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f1b5a950 con 0x5606f1b6dec0 2024-05-27T07:57:06.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.576+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cf4280 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8004930 tx=0x7ff5d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cffe70e756bc16 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.576+0000 7ff5d4c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:06.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.576+0000 7ff5d4c00700 1 -- 172.21.15.63:0/2410721014 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80093a0 con 0x5606f1b6dec0 2024-05-27T07:57:06.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.576+0000 7ff5d4c00700 1 -- 172.21.15.63:0/2410721014 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7ff5d8007500 con 0x5606f1b6dec0 2024-05-27T07:57:06.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.576+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2410721014 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5d8004020 con 0x5606f1b6dec0 2024-05-27T07:57:06.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.577+0000 7ff5d4c00700 1 -- 172.21.15.63:0/2410721014 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff5d8030050 con 0x5606f1b6dec0 2024-05-27T07:57:06.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.577+0000 7ff5fc83cb80 10 client.19968 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:06.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.577+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2410721014 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5606f1bdb0e0 con 0x5606f1b6dec0 2024-05-27T07:57:06.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.577+0000 7ff5d4c00700 1 -- 172.21.15.63:0/2410721014 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5d8004020 con 0x5606f1b6dec0 2024-05-27T07:57:06.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.577+0000 7ff5d4c00700 1 -- 172.21.15.63:0/2410721014 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff5d80050e0 con 0x5606f1b6dec0 2024-05-27T07:57:06.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.578+0000 7ff5fc83cb80 10 client.19968 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:06.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.578+0000 7ff5fc83cb80 10 client.19968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:06.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.578+0000 7ff5fc83cb80 20 client.19968 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:06.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.578+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606f1d06bc0 0x5606f1d08fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.578+0000 7ff5fc83cb80 4 client.19968 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:06.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.578+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2410721014 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606f1c1ce10 con 0x5606f1d06bc0 2024-05-27T07:57:06.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.579+0000 7ff5d6000700 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606f1d06bc0 0x5606f1d08fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.584+0000 7ff5d6000700 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606f1d06bc0 0x5606f1d08fe0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89afb6eeb7694537 server_cookie=772bcc1ddc4f19 in_seq=0 out_seq=0 2024-05-27T07:57:06.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.584+0000 7ff5d4c00700 10 client.19968 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:06.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.586+0000 7ff5d4c00700 1 -- 172.21.15.63:0/2410721014 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5606f1c1ce10 con 0x5606f1d06bc0 2024-05-27T07:57:06.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.586+0000 7ff5d4c00700 10 client.19968 handle_command_reply: tid=0 2024-05-27T07:57:06.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.586+0000 7ff5fc83cb80 1 client.19968 shutdown 2024-05-27T07:57:06.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2410721014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 msgr2=0x5606f1cf4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cf4280 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8004930 tx=0x7ff5d80046b0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2410721014 shutdown_connections 2024-05-27T07:57:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606f1d06bc0 0x5606f1d08fe0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1cf4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1cf4280 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/2410721014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1cf47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2410721014 >> 172.21.15.63:0/2410721014 conn(0x5606f1c07410 msgr2=0x5606f1ce8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5d4c00700 0 client.19968 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2410721014 shutdown_connections 2024-05-27T07:57:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.587+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2410721014 wait complete. 2024-05-27T07:57:06.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.588+0000 7ff5fc83cb80 20 client.19968 trim_cache size 0 max 16384 2024-05-27T07:57:06.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.613+0000 7ff5fc83cb80 1 Processor -- start 2024-05-27T07:57:06.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.613+0000 7ff5fc83cb80 1 -- start start 2024-05-27T07:57:06.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.614+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d03120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.614+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1cecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.614+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1d0c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.614+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81b4e50 con 0x5606f1b6dec0 2024-05-27T07:57:06.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.614+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81b26e0 con 0x5606f1cf7340 2024-05-27T07:57:06.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.614+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81b0550 con 0x5606f1c59a70 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.614+0000 7ff5e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d03120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.614+0000 7ff5e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d03120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45568/0 (socket says 172.21.15.63:45568) 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.614+0000 7ff5e6a00700 1 -- 172.21.15.63:0/2977134018 learned_addr learned my addr 172.21.15.63:0/2977134018 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.615+0000 7ff5d6000700 1 --2- 172.21.15.63:0/2977134018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1cecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.615+0000 7ff5e7400700 1 --2- 172.21.15.63:0/2977134018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1d0c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.615+0000 7ff5e6a00700 1 -- 172.21.15.63:0/2977134018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 msgr2=0x5606f1cecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.615+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/2977134018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1cecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.615+0000 7ff5e6a00700 1 -- 172.21.15.63:0/2977134018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 msgr2=0x5606f1d0c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.615+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/2977134018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1d0c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.615+0000 7ff5e6a00700 1 -- 172.21.15.63:0/2977134018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f1ceede0 con 0x5606f1b6dec0 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.615+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/2977134018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d03120 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80061e0 tx=0x7ff5d8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd61da9e714fd8f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.616+0000 7ff5d7e00700 1 -- 172.21.15.63:0/2977134018 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8019040 con 0x5606f1b6dec0 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.616+0000 7ff5d7e00700 1 -- 172.21.15.63:0/2977134018 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5d8003fd0 con 0x5606f1b6dec0 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.616+0000 7ff5d7e00700 1 -- 172.21.15.63:0/2977134018 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8007500 con 0x5606f1b6dec0 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.617+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2977134018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 msgr2=0x5606f1d03120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.617+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/2977134018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d03120 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80061e0 tx=0x7ff5d8009920 comp rx=0 tx=0).stop 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.617+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2977134018 shutdown_connections 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.617+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/2977134018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1d0c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.617+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/2977134018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d03120 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.617+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/2977134018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1cecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.617+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2977134018 >> 172.21.15.63:0/2977134018 conn(0x5606f1c07410 msgr2=0x5606f1cf31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.617+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2977134018 shutdown_connections 2024-05-27T07:57:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.617+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/2977134018 wait complete. 2024-05-27T07:57:06.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.618+0000 7ff5fc83cb80 1 -- start start 2024-05-27T07:57:06.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.618+0000 7ff5fc83cb80 1 Processor -- start 2024-05-27T07:57:06.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d0a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1d0a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5fc83cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1d70430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81b0ba0 con 0x5606f1b6dec0 2024-05-27T07:57:06.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81b2010 con 0x5606f1cf7340 2024-05-27T07:57:06.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5fc83cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e81b1270 con 0x5606f1c59a70 2024-05-27T07:57:06.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5fc83cb80 10 client.0 _ll_register_callbacks cb 0x5606f1bb4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d0a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d0a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45582/0 (socket says 172.21.15.63:45582) 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5e6a00700 1 -- 172.21.15.63:0/3434785411 learned_addr learned my addr 172.21.15.63:0/3434785411 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5e7400700 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1d70430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.619+0000 7ff5d6000700 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1d0a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.620+0000 7ff5e6a00700 1 -- 172.21.15.63:0/3434785411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 msgr2=0x5606f1d0a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.620+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1d0a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.620+0000 7ff5e6a00700 1 -- 172.21.15.63:0/3434785411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 msgr2=0x5606f1d70430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.620+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1d70430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.620+0000 7ff5e6a00700 1 -- 172.21.15.63:0/3434785411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f1d70970 con 0x5606f1b6dec0 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.620+0000 7ff5e6a00700 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d0a4a0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80053e0 tx=0x7ff5d8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af52b23dff293c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.620+0000 7ff5d4c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.620+0000 7ff5d4c00700 1 -- 172.21.15.63:0/3434785411 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8019070 con 0x5606f1b6dec0 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.621+0000 7ff5d4c00700 1 -- 172.21.15.63:0/3434785411 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9483+0+0 (secure 0 0 0) 0x7ff5d8004ee0 con 0x5606f1b6dec0 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.621+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/3434785411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5d8003fd0 con 0x5606f1b6dec0 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.621+0000 7ff5d4c00700 1 -- 172.21.15.63:0/3434785411 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff5d8046050 con 0x5606f1b6dec0 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.621+0000 7ff5fc83cb80 10 client.19974 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.621+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/3434785411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5606f1d71b90 con 0x5606f1b6dec0 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.621+0000 7ff5d4c00700 1 -- 172.21.15.63:0/3434785411 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5d8003fd0 con 0x5606f1b6dec0 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.622+0000 7ff5d4c00700 1 -- 172.21.15.63:0/3434785411 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff5d803c030 con 0x5606f1b6dec0 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.622+0000 7ff5fc83cb80 10 client.19974 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.622+0000 7ff5fc83cb80 10 client.19974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.622+0000 7ff5fc83cb80 20 client.19974 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.622+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606f1d721c0 0x5606f1d745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.622+0000 7ff5fc83cb80 4 client.19974 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:57:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.622+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/3434785411 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5606f1c165f0 con 0x5606f1d721c0 2024-05-27T07:57:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.622+0000 7ff5d6000700 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606f1d721c0 0x5606f1d745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.623+0000 7ff5d6000700 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606f1d721c0 0x5606f1d745e0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e4077a2fbf5662 server_cookie=75fb123014e39323 in_seq=0 out_seq=0 2024-05-27T07:57:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.623+0000 7ff5d4c00700 10 client.19974 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:06.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.626+0000 7ff5d4c00700 1 -- 172.21.15.63:0/3434785411 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5606f1c165f0 con 0x5606f1d721c0 2024-05-27T07:57:06.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.626+0000 7ff5d4c00700 10 client.19974 handle_command_reply: tid=0 2024-05-27T07:57:06.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.626+0000 7ff5fc83cb80 1 client.19974 shutdown 2024-05-27T07:57:06.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/3434785411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 msgr2=0x5606f1d0a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d0a4a0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80053e0 tx=0x7ff5d8004760 comp rx=0 tx=0).stop 2024-05-27T07:57:06.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/3434785411 shutdown_connections 2024-05-27T07:57:06.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606f1d721c0 0x5606f1d745e0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606f1cf7340 0x5606f1d70430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1b6dec0 0x5606f1d0a4a0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5fc83cb80 1 --2- 172.21.15.63:0/3434785411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5606f1c59a70 0x5606f1d0a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5d4c00700 0 client.19974 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:06.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/3434785411 >> 172.21.15.63:0/3434785411 conn(0x5606f1c07410 msgr2=0x5606f1cf4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/3434785411 shutdown_connections 2024-05-27T07:57:06.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.627+0000 7ff5fc83cb80 1 -- 172.21.15.63:0/3434785411 wait complete. 2024-05-27T07:57:06.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.628+0000 7ff5fc83cb80 20 client.19974 trim_cache size 0 max 16384 2024-05-27T07:57:06.630 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:57:06.631 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:57:06.631 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "112f95b4-20c5-47cc-825a-cfc325bc0069", 2024-05-27T07:57:06.631 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:57:06.631 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:57:06.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.628+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1515594326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff5c80b29f0 msgr2=0x7ff5c80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.628+0000 7ff5e7e00700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff5c80b29f0 0x7ff5c80b4f00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5606f1ce9b10 tx=0x7ff5e0005fb0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.628+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1515594326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e8103850 msgr2=0x7ff5e81a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.628+0000 7ff5e7e00700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e8103850 0x7ff5e81a2e20 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000d950 tx=0x7ff5d000dc60 comp rx=0 tx=0).stop 2024-05-27T07:57:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.629+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1515594326 shutdown_connections 2024-05-27T07:57:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.629+0000 7ff5e7e00700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff5c80b29f0 0x7ff5c80b4f00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.629+0000 7ff5e7e00700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5e8110ab0 0x7ff5e819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.629+0000 7ff5e7e00700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e8102f00 0x7ff5e81a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.629+0000 7ff5e7e00700 1 --2- 172.21.15.63:0/1515594326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e8103850 0x7ff5e81a2e20 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.629+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1515594326 >> 172.21.15.63:0/1515594326 conn(0x7ff5e8078af0 msgr2=0x7ff5e8111db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.629+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1515594326 shutdown_connections 2024-05-27T07:57:06.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:06.629+0000 7ff5e7e00700 1 -- 172.21.15.63:0/1515594326 wait complete. 2024-05-27T07:57:07.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:07 smithi063 ceph-mon[31613]: pgmap v743: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 28 op/s 2024-05-27T07:57:07.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:07 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:57:07.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:07 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:07.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:07 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:57:07.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:07 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:07.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:07 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:57:07.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:07 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:07.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:07 smithi086 ceph-mon[34687]: pgmap v743: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 28 op/s 2024-05-27T07:57:07.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:07 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:57:07.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:07 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:07.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:07 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:57:07.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:07 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:07.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:07 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:57:07.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:07 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:07.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:07 smithi149 ceph-mon[35408]: pgmap v743: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 28 op/s 2024-05-27T07:57:07.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:07 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:57:07.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:07 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:07.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:07 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:57:07.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:07 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:07.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:07 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:57:07.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:07 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:07.499 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:57:08.023 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:08.414 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:57:08.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:57:08.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:57:08.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:57:08.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:57:08.415 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.416 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.416 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:08 smithi063 ceph-mon[31613]: osdmap e128: 12 total, 12 up, 12 in 2024-05-27T07:57:08.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:57:08.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:57:08.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:57:08.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:57:08.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:57:08.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:08 smithi086 ceph-mon[34687]: osdmap e128: 12 total, 12 up, 12 in 2024-05-27T07:57:08.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:57:08.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:57:08.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:57:08.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:57:08.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:57:08.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:08.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:08 smithi149 ceph-mon[35408]: osdmap e128: 12 total, 12 up, 12 in 2024-05-27T07:57:09.431 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:09 smithi063 ceph-mon[31613]: pgmap v745: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T07:57:09.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:09 smithi086 ceph-mon[34687]: pgmap v745: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T07:57:09.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:09 smithi149 ceph-mon[35408]: pgmap v745: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T07:57:10.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.498+0000 7f790fe00700 1 -- 172.21.15.63:0/365091304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7910076670 msgr2=0x7f7910110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:10.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.498+0000 7f790fe00700 1 --2- 172.21.15.63:0/365091304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7910076670 0x7f7910110460 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009990 tx=0x7f78fc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.499+0000 7f790fe00700 1 -- 172.21.15.63:0/365091304 shutdown_connections 2024-05-27T07:57:10.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.499+0000 7f790fe00700 1 --2- 172.21.15.63:0/365091304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79101109a0 0x7f7910112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.499+0000 7f790fe00700 1 --2- 172.21.15.63:0/365091304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910075d20 0x7f7910076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.499+0000 7f790fe00700 1 --2- 172.21.15.63:0/365091304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7910076670 0x7f7910110460 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.499+0000 7f790fe00700 1 -- 172.21.15.63:0/365091304 >> 172.21.15.63:0/365091304 conn(0x7f7910079d00 msgr2=0x7f791007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:10.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.499+0000 7f790fe00700 1 -- 172.21.15.63:0/365091304 shutdown_connections 2024-05-27T07:57:10.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.499+0000 7f790fe00700 1 -- 172.21.15.63:0/365091304 wait complete. 2024-05-27T07:57:10.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.500+0000 7f790fe00700 1 Processor -- start 2024-05-27T07:57:10.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.500+0000 7f790fe00700 1 -- start start 2024-05-27T07:57:10.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.501+0000 7f790fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910075d20 0x7f791019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.501+0000 7f790fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7910076670 0x7f791019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.501+0000 7f790fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79101109a0 0x7f79101a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.501+0000 7f790fe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79101163a0 con 0x7f79101109a0 2024-05-27T07:57:10.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.501+0000 7f790fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7910116220 con 0x7f7910075d20 2024-05-27T07:57:10.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.501+0000 7f790fe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7910116520 con 0x7f7910076670 2024-05-27T07:57:10.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.501+0000 7f790f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79101109a0 0x7f79101a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.502+0000 7f790ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910075d20 0x7f791019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.502+0000 7f790f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79101109a0 0x7f79101a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59940/0 (socket says 172.21.15.63:59940) 2024-05-27T07:57:10.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.502+0000 7f790f400700 1 -- 172.21.15.63:0/3508815425 learned_addr learned my addr 172.21.15.63:0/3508815425 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:10.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.502+0000 7f790e000700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7910076670 0x7f791019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.502+0000 7f790f400700 1 -- 172.21.15.63:0/3508815425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7910076670 msgr2=0x7f791019d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.502+0000 7f790f400700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7910076670 0x7f791019d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.502+0000 7f790f400700 1 -- 172.21.15.63:0/3508815425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910075d20 msgr2=0x7f791019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.502+0000 7f790f400700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910075d20 0x7f791019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.502+0000 7f790f400700 1 -- 172.21.15.63:0/3508815425 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79101a1c80 con 0x7f79101109a0 2024-05-27T07:57:10.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.502+0000 7f790e000700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7910076670 0x7f791019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.503+0000 7f790f400700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79101109a0 0x7f79101a15e0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f79080094f0 tx=0x7f790800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.503+0000 7f7914a00700 1 -- 172.21.15.63:0/3508815425 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790800ddd0 con 0x7f79101109a0 2024-05-27T07:57:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.503+0000 7f790fe00700 1 -- 172.21.15.63:0/3508815425 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79101a1f70 con 0x7f79101109a0 2024-05-27T07:57:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.503+0000 7f7914a00700 1 -- 172.21.15.63:0/3508815425 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7908009e20 con 0x7f79101109a0 2024-05-27T07:57:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.503+0000 7f7914a00700 1 -- 172.21.15.63:0/3508815425 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908015400 con 0x7f79101109a0 2024-05-27T07:57:10.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.503+0000 7f790fe00700 1 -- 172.21.15.63:0/3508815425 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101a9890 con 0x7f79101109a0 2024-05-27T07:57:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.508+0000 7f7914a00700 1 -- 172.21.15.63:0/3508815425 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7908005020 con 0x7f79101109a0 2024-05-27T07:57:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.508+0000 7f792491bb80 1 Processor -- start 2024-05-27T07:57:10.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.509+0000 7f792491bb80 1 -- start start 2024-05-27T07:57:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.509+0000 7f792491bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e703e40 0x559a8e881ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.509+0000 7f792491bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e88cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.509+0000 7f792491bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882820 0x559a8e88f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.509+0000 7f792491bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559a8e7b0e40 con 0x559a8e703e40 2024-05-27T07:57:10.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.509+0000 7f792491bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559a8e7a09e0 con 0x559a8e882410 2024-05-27T07:57:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.509+0000 7f792491bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559a8e7a4a20 con 0x559a8e882820 2024-05-27T07:57:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.510+0000 7f790e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e703e40 0x559a8e881ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.510+0000 7f790e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e703e40 0x559a8e881ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59942/0 (socket says 172.21.15.63:59942) 2024-05-27T07:57:10.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.510+0000 7f790ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e88cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.510+0000 7f790e000700 1 -- 172.21.15.63:0/1521051443 learned_addr learned my addr 172.21.15.63:0/1521051443 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.510+0000 7f790e000700 1 --2- 172.21.15.63:0/1521051443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882820 0x559a8e88f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.511+0000 7f7914a00700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78f40b2a70 0x7f78f40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.511+0000 7f790ea00700 1 -- 172.21.15.63:0/1521051443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882820 msgr2=0x559a8e88f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.511+0000 7f790ea00700 1 --2- 172.21.15.63:0/1521051443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882820 0x559a8e88f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.511+0000 7f790ea00700 1 -- 172.21.15.63:0/1521051443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e703e40 msgr2=0x559a8e881ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.511+0000 7f790ea00700 1 --2- 172.21.15.63:0/1521051443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e703e40 0x559a8e881ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.511+0000 7f790ea00700 1 -- 172.21.15.63:0/1521051443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8e769420 con 0x559a8e882410 2024-05-27T07:57:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.511+0000 7f790e000700 1 --2- 172.21.15.63:0/1521051443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e703e40 0x559a8e881ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:10.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.511+0000 7f7914a00700 1 -- 172.21.15.63:0/3508815425 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9571+0+0 (secure 0 0 0) 0x7f79080e2450 con 0x7f79101109a0 2024-05-27T07:57:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.511+0000 7f790f400700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78f40b2a70 0x7f78f40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.511+0000 7f790ea00700 1 --2- 172.21.15.63:0/1521051443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e88cad0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7900007bf0 tx=0x7f7900007f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9473a7e198594f80 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.512+0000 7f7904c00700 1 -- 172.21.15.63:0/1521051443 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900010c40 con 0x559a8e882410 2024-05-27T07:57:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.512+0000 7f790f400700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78f40b2a70 0x7f78f40b4f80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7908005c30 tx=0x7f790800b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.512+0000 7f7904c00700 1 -- 172.21.15.63:0/1521051443 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7900015070 con 0x559a8e882410 2024-05-27T07:57:10.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.512+0000 7f7904c00700 1 -- 172.21.15.63:0/1521051443 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900004d60 con 0x559a8e882410 2024-05-27T07:57:10.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.513+0000 7f792491bb80 1 -- 172.21.15.63:0/1521051443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 msgr2=0x559a8e88cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:10.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.513+0000 7f792491bb80 1 --2- 172.21.15.63:0/1521051443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e88cad0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7900007bf0 tx=0x7f7900007f00 comp rx=0 tx=0).stop 2024-05-27T07:57:10.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.513+0000 7f792491bb80 1 -- 172.21.15.63:0/1521051443 shutdown_connections 2024-05-27T07:57:10.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.513+0000 7f792491bb80 1 --2- 172.21.15.63:0/1521051443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882820 0x559a8e88f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.513+0000 7f792491bb80 1 --2- 172.21.15.63:0/1521051443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e703e40 0x559a8e881ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.513+0000 7f792491bb80 1 --2- 172.21.15.63:0/1521051443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e88cad0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.513+0000 7f792491bb80 1 -- 172.21.15.63:0/1521051443 >> 172.21.15.63:0/1521051443 conn(0x559a8e763fc0 msgr2=0x559a8e87dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:10.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.514+0000 7f792491bb80 1 -- 172.21.15.63:0/1521051443 shutdown_connections 2024-05-27T07:57:10.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.514+0000 7f792491bb80 1 -- 172.21.15.63:0/1521051443 wait complete. 2024-05-27T07:57:10.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.514+0000 7f792491bb80 1 -- start start 2024-05-27T07:57:10.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.515+0000 7f792491bb80 1 Processor -- start 2024-05-27T07:57:10.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.515+0000 7f792491bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 0x559a8e88c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.515+0000 7f792491bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882410 0x559a8e88cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.515+0000 7f792491bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.515+0000 7f792491bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559a8e6a6fa0 con 0x559a8e882820 2024-05-27T07:57:10.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.515+0000 7f792491bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559a8e741760 con 0x559a8e703e40 2024-05-27T07:57:10.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.515+0000 7f792491bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559a8e88f990 con 0x559a8e882410 2024-05-27T07:57:10.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.516+0000 7f792491bb80 10 client.0 _ll_register_callbacks cb 0x559a8e74ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:10.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.516+0000 7f790e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 0x559a8e88c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.516+0000 7f790e000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 0x559a8e88c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:58190/0 (socket says 172.21.15.63:58190) 2024-05-27T07:57:10.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.516+0000 7f790e000700 1 -- 172.21.15.63:0/2377655245 learned_addr learned my addr 172.21.15.63:0/2377655245 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:10.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.516+0000 7f790e000700 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.516+0000 7f790ea00700 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882410 0x559a8e88cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.517+0000 7f790e000700 1 -- 172.21.15.63:0/2377655245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882410 msgr2=0x559a8e88cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:10.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.517+0000 7f790e000700 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882410 0x559a8e88cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.517+0000 7f790e000700 1 -- 172.21.15.63:0/2377655245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 msgr2=0x559a8e88c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:10.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.517+0000 7f790e000700 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 0x559a8e88c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.517+0000 7f790e000700 1 -- 172.21.15.63:0/2377655245 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8e6f08d0 con 0x559a8e882820 2024-05-27T07:57:10.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.517+0000 7f790e000700 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88d080 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x559a8e88be80 tx=0x7f78fc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81aa4eb8a1857b61 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:10.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.517+0000 7f78f9600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:10.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.517+0000 7f78f9600700 1 -- 172.21.15.63:0/2377655245 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc021070 con 0x559a8e882820 2024-05-27T07:57:10.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.517+0000 7f78f9600700 1 -- 172.21.15.63:0/2377655245 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9571+0+0 (secure 0 0 0) 0x7f78fc00fb30 con 0x559a8e882820 2024-05-27T07:57:10.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.517+0000 7f792491bb80 1 -- 172.21.15.63:0/2377655245 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7900015070 con 0x559a8e882820 2024-05-27T07:57:10.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.518+0000 7f78f9600700 1 -- 172.21.15.63:0/2377655245 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f78fc00fd30 con 0x559a8e882820 2024-05-27T07:57:10.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.518+0000 7f792491bb80 10 client.19986 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:10.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.518+0000 7f792491bb80 1 -- 172.21.15.63:0/2377655245 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a8e771060 con 0x559a8e882820 2024-05-27T07:57:10.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.518+0000 7f78f9600700 1 -- 172.21.15.63:0/2377655245 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7900015070 con 0x559a8e882820 2024-05-27T07:57:10.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.519+0000 7f78f9600700 1 -- 172.21.15.63:0/2377655245 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f78fc027070 con 0x559a8e882820 2024-05-27T07:57:10.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.519+0000 7f792491bb80 10 client.19986 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:10.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.519+0000 7f792491bb80 10 client.19986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:10.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.519+0000 7f792491bb80 20 client.19986 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:10.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.519+0000 7f792491bb80 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559a8e89c9b0 0x559a8e89edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.519+0000 7f792491bb80 4 client.19986 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:10.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.519+0000 7f792491bb80 1 -- 172.21.15.63:0/2377655245 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a8e7b2d90 con 0x559a8e89c9b0 2024-05-27T07:57:10.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.520+0000 7f790ea00700 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559a8e89c9b0 0x559a8e89edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.520+0000 7f790ea00700 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559a8e89c9b0 0x559a8e89edd0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53336cc85d896766 server_cookie=bd4b74b4d48b2aae in_seq=0 out_seq=0 2024-05-27T07:57:10.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.520+0000 7f78f9600700 10 client.19986 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:10.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.522+0000 7f78f9600700 1 -- 172.21.15.63:0/2377655245 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559a8e7b2d90 con 0x559a8e89c9b0 2024-05-27T07:57:10.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.522+0000 7f78f9600700 10 client.19986 handle_command_reply: tid=0 2024-05-27T07:57:10.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.522+0000 7f792491bb80 1 client.19986 shutdown 2024-05-27T07:57:10.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 1 -- 172.21.15.63:0/2377655245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 msgr2=0x559a8e88d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:10.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88d080 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x559a8e88be80 tx=0x7f78fc005e80 comp rx=0 tx=0).stop 2024-05-27T07:57:10.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 1 -- 172.21.15.63:0/2377655245 shutdown_connections 2024-05-27T07:57:10.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559a8e89c9b0 0x559a8e89edd0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88d080 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 0x559a8e88c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 1 --2- 172.21.15.63:0/2377655245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882410 0x559a8e88cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 1 -- 172.21.15.63:0/2377655245 >> 172.21.15.63:0/2377655245 conn(0x559a8e763fc0 msgr2=0x559a8e87cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:10.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f78f9600700 0 client.19986 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:10.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 1 -- 172.21.15.63:0/2377655245 shutdown_connections 2024-05-27T07:57:10.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 1 -- 172.21.15.63:0/2377655245 wait complete. 2024-05-27T07:57:10.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.523+0000 7f792491bb80 20 client.19986 trim_cache size 0 max 16384 2024-05-27T07:57:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.557+0000 7f792491bb80 1 Processor -- start 2024-05-27T07:57:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.557+0000 7f792491bb80 1 -- start start 2024-05-27T07:57:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.557+0000 7f792491bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e703e40 0x559a8e87de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.557+0000 7f792491bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e887e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.557+0000 7f792491bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.557+0000 7f792491bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b4f70 con 0x559a8e882820 2024-05-27T07:57:10.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.557+0000 7f792491bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b2800 con 0x559a8e882410 2024-05-27T07:57:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.557+0000 7f792491bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b0670 con 0x559a8e703e40 2024-05-27T07:57:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e887e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e887e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:58196/0 (socket says 172.21.15.63:58196) 2024-05-27T07:57:10.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59958/0 (socket says 172.21.15.63:59958) 2024-05-27T07:57:10.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790ea00700 1 -- 172.21.15.63:0/2432789377 learned_addr learned my addr 172.21.15.63:0/2432789377 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:10.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790e000700 1 --2- 172.21.15.63:0/2432789377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e703e40 0x559a8e87de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790e000700 1 -- 172.21.15.63:0/2432789377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e703e40 msgr2=0x559a8e87de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:10.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790e000700 1 --2- 172.21.15.63:0/2432789377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e703e40 0x559a8e87de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790e000700 1 -- 172.21.15.63:0/2432789377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 msgr2=0x559a8e887e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:10.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790e000700 1 --2- 172.21.15.63:0/2432789377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e887e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790e000700 1 -- 172.21.15.63:0/2432789377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8e87f350 con 0x559a8e882820 2024-05-27T07:57:10.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.558+0000 7f790ea00700 1 --2- 172.21.15.63:0/2432789377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e887e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:10.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.559+0000 7f790e000700 1 --2- 172.21.15.63:0/2432789377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88cff0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f78fc017490 tx=0x7f78fc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708fe549abee9f73 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:10.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.559+0000 7f78fbe00700 1 -- 172.21.15.63:0/2432789377 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc021030 con 0x559a8e882820 2024-05-27T07:57:10.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.559+0000 7f78fbe00700 1 -- 172.21.15.63:0/2432789377 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78fc022040 con 0x559a8e882820 2024-05-27T07:57:10.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.559+0000 7f78fbe00700 1 -- 172.21.15.63:0/2432789377 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc02a680 con 0x559a8e882820 2024-05-27T07:57:10.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.561+0000 7f792491bb80 1 -- 172.21.15.63:0/2432789377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 msgr2=0x559a8e88cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:10.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.561+0000 7f792491bb80 1 --2- 172.21.15.63:0/2432789377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88cff0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f78fc017490 tx=0x7f78fc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.561+0000 7f792491bb80 1 -- 172.21.15.63:0/2432789377 shutdown_connections 2024-05-27T07:57:10.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.561+0000 7f792491bb80 1 --2- 172.21.15.63:0/2432789377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882820 0x559a8e88cff0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.561+0000 7f792491bb80 1 --2- 172.21.15.63:0/2432789377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e703e40 0x559a8e87de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.561+0000 7f792491bb80 1 --2- 172.21.15.63:0/2432789377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e882410 0x559a8e887e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.561+0000 7f792491bb80 1 -- 172.21.15.63:0/2432789377 >> 172.21.15.63:0/2432789377 conn(0x559a8e763fc0 msgr2=0x559a8e885900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:10.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.561+0000 7f792491bb80 1 -- 172.21.15.63:0/2432789377 shutdown_connections 2024-05-27T07:57:10.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.561+0000 7f792491bb80 1 -- 172.21.15.63:0/2432789377 wait complete. 2024-05-27T07:57:10.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.562+0000 7f792491bb80 1 -- start start 2024-05-27T07:57:10.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.562+0000 7f792491bb80 1 Processor -- start 2024-05-27T07:57:10.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f792491bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 0x559a8e8954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f792491bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882410 0x559a8e890820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f792491bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882820 0x559a8e890e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f792491bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b0cc0 con 0x559a8e882410 2024-05-27T07:57:10.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f792491bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b2130 con 0x559a8e703e40 2024-05-27T07:57:10.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f792491bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b1390 con 0x559a8e882820 2024-05-27T07:57:10.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f792491bb80 10 client.0 _ll_register_callbacks cb 0x559a8e74ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:10.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f790ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882410 0x559a8e890820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f790ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882410 0x559a8e890820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59970/0 (socket says 172.21.15.63:59970) 2024-05-27T07:57:10.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f790ea00700 1 -- 172.21.15.63:0/3284637480 learned_addr learned my addr 172.21.15.63:0/3284637480 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:10.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f790e000700 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 0x559a8e8954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.563+0000 7f790e000700 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882820 0x559a8e890e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.564+0000 7f790ea00700 1 -- 172.21.15.63:0/3284637480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882820 msgr2=0x559a8e890e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:10.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.564+0000 7f790ea00700 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882820 0x559a8e890e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.564+0000 7f790ea00700 1 -- 172.21.15.63:0/3284637480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 msgr2=0x559a8e8954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:10.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.564+0000 7f790ea00700 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 0x559a8e8954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.564+0000 7f790ea00700 1 -- 172.21.15.63:0/3284637480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8e87e870 con 0x559a8e882410 2024-05-27T07:57:10.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.564+0000 7f790ea00700 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882410 0x559a8e890820 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x559a8e87f050 tx=0x7f7900010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2ac5e97da785c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:10.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.564+0000 7f78f9600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:10.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.565+0000 7f78f9600700 1 -- 172.21.15.63:0/3284637480 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900005320 con 0x559a8e882410 2024-05-27T07:57:10.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.565+0000 7f78f9600700 1 -- 172.21.15.63:0/3284637480 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9571+0+0 (secure 0 0 0) 0x7f79000054c0 con 0x559a8e882410 2024-05-27T07:57:10.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.565+0000 7f792491bb80 1 -- 172.21.15.63:0/3284637480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78fc022040 con 0x559a8e882410 2024-05-27T07:57:10.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.565+0000 7f78f9600700 1 -- 172.21.15.63:0/3284637480 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7900036050 con 0x559a8e882410 2024-05-27T07:57:10.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.565+0000 7f792491bb80 10 client.19992 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:10.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.565+0000 7f792491bb80 1 -- 172.21.15.63:0/3284637480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a8e907240 con 0x559a8e882410 2024-05-27T07:57:10.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.566+0000 7f78f9600700 1 -- 172.21.15.63:0/3284637480 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78fc022040 con 0x559a8e882410 2024-05-27T07:57:10.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.566+0000 7f78f9600700 1 -- 172.21.15.63:0/3284637480 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f79000115e0 con 0x559a8e882410 2024-05-27T07:57:10.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.566+0000 7f792491bb80 10 client.19992 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:10.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.566+0000 7f792491bb80 10 client.19992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:10.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.566+0000 7f792491bb80 20 client.19992 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:10.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.566+0000 7f792491bb80 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559a8e9075f0 0x559a8e909a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:10.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.566+0000 7f792491bb80 4 client.19992 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:57:10.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.566+0000 7f792491bb80 1 -- 172.21.15.63:0/3284637480 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a8e7ac570 con 0x559a8e9075f0 2024-05-27T07:57:10.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.567+0000 7f790e000700 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559a8e9075f0 0x559a8e909a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:10.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.567+0000 7f790e000700 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559a8e9075f0 0x559a8e909a10 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421b5caa1c55118c server_cookie=6a975dba44bac6f in_seq=0 out_seq=0 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.567+0000 7f78f9600700 10 client.19992 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f78f9600700 1 -- 172.21.15.63:0/3284637480 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559a8e7ac570 con 0x559a8e9075f0 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f78f9600700 10 client.19992 handle_command_reply: tid=0 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f792491bb80 1 client.19992 shutdown 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f792491bb80 1 -- 172.21.15.63:0/3284637480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882410 msgr2=0x559a8e890820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f792491bb80 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882410 0x559a8e890820 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x559a8e87f050 tx=0x7f7900010b30 comp rx=0 tx=0).stop 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f792491bb80 1 -- 172.21.15.63:0/3284637480 shutdown_connections 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f792491bb80 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559a8e9075f0 0x559a8e909a10 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f792491bb80 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a8e882820 0x559a8e890e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f792491bb80 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a8e703e40 0x559a8e8954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f792491bb80 1 --2- 172.21.15.63:0/3284637480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559a8e882410 0x559a8e890820 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.568+0000 7f792491bb80 1 -- 172.21.15.63:0/3284637480 >> 172.21.15.63:0/3284637480 conn(0x559a8e763fc0 msgr2=0x559a8e88cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:10.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.569+0000 7f78f9600700 0 client.19992 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:10.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.569+0000 7f792491bb80 1 -- 172.21.15.63:0/3284637480 shutdown_connections 2024-05-27T07:57:10.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.569+0000 7f792491bb80 1 -- 172.21.15.63:0/3284637480 wait complete. 2024-05-27T07:57:10.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.569+0000 7f792491bb80 20 client.19992 trim_cache size 0 max 16384 2024-05-27T07:57:10.572 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:57:10.572 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:57:10.572 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:57:10.573 INFO:teuthology.orchestra.run.smithi063.stdout: "112f95b4-20c5-47cc-825a-cfc325bc0069": { 2024-05-27T07:57:10.573 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:57:10.573 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "112f95b4-20c5-47cc-825a-cfc325bc0069", 2024-05-27T07:57:10.573 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:57:10.573 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:57:10.573 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:57:10.573 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 -- 172.21.15.63:0/3508815425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78f40b2a70 msgr2=0x7f78f40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78f40b2a70 0x7f78f40b4f80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7908005c30 tx=0x7f790800b040 comp rx=0 tx=0).stop 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 -- 172.21.15.63:0/3508815425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79101109a0 msgr2=0x7f79101a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79101109a0 0x7f79101a15e0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f79080094f0 tx=0x7f790800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 -- 172.21.15.63:0/3508815425 shutdown_connections 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78f40b2a70 0x7f78f40b4f80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79101109a0 0x7f79101a15e0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7910075d20 0x7f791019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 --2- 172.21.15.63:0/3508815425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7910076670 0x7f791019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 -- 172.21.15.63:0/3508815425 >> 172.21.15.63:0/3508815425 conn(0x7f7910079d00 msgr2=0x7f7910100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:10.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 -- 172.21.15.63:0/3508815425 shutdown_connections 2024-05-27T07:57:10.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:10.570+0000 7f790fe00700 1 -- 172.21.15.63:0/3508815425 wait complete. 2024-05-27T07:57:11.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:11 smithi086 ceph-mon[34687]: pgmap v746: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T07:57:11.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:11 smithi149 ceph-mon[35408]: pgmap v746: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T07:57:11.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:11 smithi063 ceph-mon[31613]: pgmap v746: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T07:57:13.358 INFO:tasks.cephfs.filesystem:scrub status for tag:112f95b4-20c5-47cc-825a-cfc325bc0069 - {'path': '/', 'tag': '112f95b4-20c5-47cc-825a-cfc325bc0069', 'options': 'recursive,force'} 2024-05-27T07:57:13.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:13 smithi086 ceph-mon[34687]: pgmap v747: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 49 op/s 2024-05-27T07:57:13.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:13 smithi149 ceph-mon[35408]: pgmap v747: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 49 op/s 2024-05-27T07:57:13.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:13 smithi063 ceph-mon[31613]: pgmap v747: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 49 op/s 2024-05-27T07:57:15.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:15 smithi086 ceph-mon[34687]: pgmap v748: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 49 op/s 2024-05-27T07:57:15.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:15 smithi149 ceph-mon[35408]: pgmap v748: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 49 op/s 2024-05-27T07:57:15.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:15 smithi063 ceph-mon[31613]: pgmap v748: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 49 op/s 2024-05-27T07:57:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:16 smithi086 ceph-mon[34687]: osdmap e129: 12 total, 12 up, 12 in 2024-05-27T07:57:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:16 smithi086 ceph-mon[34687]: pgmap v750: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.6 MiB/s wr, 45 op/s 2024-05-27T07:57:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:16 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:57:17.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:16 smithi149 ceph-mon[35408]: osdmap e129: 12 total, 12 up, 12 in 2024-05-27T07:57:17.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:16 smithi149 ceph-mon[35408]: pgmap v750: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.6 MiB/s wr, 45 op/s 2024-05-27T07:57:17.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:16 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:57:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:16 smithi063 ceph-mon[31613]: osdmap e129: 12 total, 12 up, 12 in 2024-05-27T07:57:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:16 smithi063 ceph-mon[31613]: pgmap v750: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.6 MiB/s wr, 45 op/s 2024-05-27T07:57:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:16 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:57:18.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:57:18.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:57:18.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:57:18.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:57:18.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:57:18.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:57:19.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:18 smithi086 ceph-mon[34687]: pgmap v751: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.6 MiB/s wr, 44 op/s 2024-05-27T07:57:19.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:18 smithi149 ceph-mon[35408]: pgmap v751: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.6 MiB/s wr, 44 op/s 2024-05-27T07:57:19.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:18 smithi063 ceph-mon[31613]: pgmap v751: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.6 MiB/s wr, 44 op/s 2024-05-27T07:57:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.474+0000 7f1054000640 1 -- 172.21.15.63:0/1207827197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f104c106610 msgr2=0x7f104c1129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.474+0000 7f1054000640 1 --2- 172.21.15.63:0/1207827197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f104c106610 0x7f104c1129e0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f1044009ca0 tx=0x7f104402a8c0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.474+0000 7f1054000640 1 -- 172.21.15.63:0/1207827197 shutdown_connections 2024-05-27T07:57:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.474+0000 7f1054000640 1 --2- 172.21.15.63:0/1207827197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f104c106610 0x7f104c1129e0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.474+0000 7f1054000640 1 --2- 172.21.15.63:0/1207827197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f104c10c360 0x7f104c105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.474+0000 7f1054000640 1 --2- 172.21.15.63:0/1207827197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f104c10bec0 0x7f104c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.474+0000 7f1054000640 1 -- 172.21.15.63:0/1207827197 >> 172.21.15.63:0/1207827197 conn(0x7f104c1016b0 msgr2=0x7f104c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.474+0000 7f1054000640 1 -- 172.21.15.63:0/1207827197 shutdown_connections 2024-05-27T07:57:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.474+0000 7f1054000640 1 -- 172.21.15.63:0/1207827197 wait complete. 2024-05-27T07:57:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.476+0000 7f1054000640 1 Processor -- start 2024-05-27T07:57:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.476+0000 7f1054000640 1 -- start start 2024-05-27T07:57:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1054000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f104c106610 0x7f104c1a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1054000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f104c10bec0 0x7f104c1a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1054000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f104c10c360 0x7f104c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1054000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f104c115fb0 con 0x7f104c10c360 2024-05-27T07:57:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1054000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f104c115e30 con 0x7f104c10bec0 2024-05-27T07:57:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1054000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f104c116130 con 0x7f104c106610 2024-05-27T07:57:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1052000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f104c10c360 0x7f104c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1052000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f104c10c360 0x7f104c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48872/0 (socket says 172.21.15.63:48872) 2024-05-27T07:57:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1052000640 1 -- 172.21.15.63:0/1121501178 learned_addr learned my addr 172.21.15.63:0/1121501178 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1051600640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f104c106610 0x7f104c1a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.477+0000 7f1050c00640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f104c10bec0 0x7f104c1a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.478+0000 7f1052000640 1 -- 172.21.15.63:0/1121501178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f104c106610 msgr2=0x7f104c1a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.478+0000 7f1052000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f104c106610 0x7f104c1a25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.478+0000 7f1052000640 1 -- 172.21.15.63:0/1121501178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f104c10bec0 msgr2=0x7f104c1a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.478+0000 7f1052000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f104c10bec0 0x7f104c1a2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.478+0000 7f1052000640 1 -- 172.21.15.63:0/1121501178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c19ced0 con 0x7f104c10c360 2024-05-27T07:57:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.478+0000 7f1052000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f104c10c360 0x7f104c19c7f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f1044031040 tx=0x7f1044003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.478+0000 7f104a000640 1 -- 172.21.15.63:0/1121501178 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044032c90 con 0x7f104c10c360 2024-05-27T07:57:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.478+0000 7f1054000640 1 -- 172.21.15.63:0/1121501178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104c19d160 con 0x7f104c10c360 2024-05-27T07:57:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.478+0000 7f104a000640 1 -- 172.21.15.63:0/1121501178 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1044032e30 con 0x7f104c10c360 2024-05-27T07:57:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.479+0000 7f104a000640 1 -- 172.21.15.63:0/1121501178 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044030b30 con 0x7f104c10c360 2024-05-27T07:57:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.479+0000 7f1054000640 1 -- 172.21.15.63:0/1121501178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104c1a9410 con 0x7f104c10c360 2024-05-27T07:57:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.483+0000 7f105af8d740 1 Processor -- start 2024-05-27T07:57:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.483+0000 7f104a000640 1 -- 172.21.15.63:0/1121501178 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1044030cd0 con 0x7f104c10c360 2024-05-27T07:57:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.483+0000 7f105af8d740 1 -- start start 2024-05-27T07:57:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.484+0000 7f105af8d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1f61b0 0x56137b598860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.484+0000 7f105af8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b598da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.484+0000 7f105af8d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1d54d0 0x56137b5992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.484+0000 7f105af8d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56137b509b90 con 0x56137b1f61b0 2024-05-27T07:57:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.484+0000 7f105af8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56137b4f9730 con 0x56137b1cd2f0 2024-05-27T07:57:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.484+0000 7f105af8d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56137b4fd770 con 0x56137b1d54d0 2024-05-27T07:57:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.484+0000 7f1050c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1f61b0 0x56137b598860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.484+0000 7f1050c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1f61b0 0x56137b598860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48882/0 (socket says 172.21.15.63:48882) 2024-05-27T07:57:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.484+0000 7f1050c00640 1 -- 172.21.15.63:0/2250526248 learned_addr learned my addr 172.21.15.63:0/2250526248 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.485+0000 7f1051600640 1 --2- 172.21.15.63:0/2250526248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b598da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.485+0000 7f1050c00640 1 --2- 172.21.15.63:0/2250526248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1d54d0 0x56137b5992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.485+0000 7f104a000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10340b2950 0x7f10340b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.485+0000 7f1050c00640 1 -- 172.21.15.63:0/2250526248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1d54d0 msgr2=0x56137b5992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.485+0000 7f1050c00640 1 --2- 172.21.15.63:0/2250526248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1d54d0 0x56137b5992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.485+0000 7f1050c00640 1 -- 172.21.15.63:0/2250526248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 msgr2=0x56137b598da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.485+0000 7f1050c00640 1 --2- 172.21.15.63:0/2250526248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b598da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.485+0000 7f1050c00640 1 -- 172.21.15.63:0/2250526248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137b4928b0 con 0x56137b1f61b0 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.485+0000 7f104a000640 1 -- 172.21.15.63:0/1121501178 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f1044103e70 con 0x7f104c10c360 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.485+0000 7f1052000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10340b2950 0x7f10340b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.486+0000 7f1050c00640 1 --2- 172.21.15.63:0/2250526248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1f61b0 0x56137b598860 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f10380113b0 tx=0x7f103800b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9df16a7c316fe5a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.486+0000 7f103e600640 1 -- 172.21.15.63:0/2250526248 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038019030 con 0x56137b1f61b0 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.486+0000 7f103e600640 1 -- 172.21.15.63:0/2250526248 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10380110d0 con 0x56137b1f61b0 2024-05-27T07:57:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.486+0000 7f103e600640 1 -- 172.21.15.63:0/2250526248 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038003c70 con 0x56137b1f61b0 2024-05-27T07:57:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.487+0000 7f105af8d740 1 -- 172.21.15.63:0/2250526248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1f61b0 msgr2=0x56137b598860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.487+0000 7f105af8d740 1 --2- 172.21.15.63:0/2250526248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1f61b0 0x56137b598860 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f10380113b0 tx=0x7f103800b570 comp rx=0 tx=0).stop 2024-05-27T07:57:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.487+0000 7f1052000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10340b2950 0x7f10340b4e60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f10440094e0 tx=0x7f10440095c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.487+0000 7f105af8d740 1 -- 172.21.15.63:0/2250526248 shutdown_connections 2024-05-27T07:57:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.487+0000 7f105af8d740 1 --2- 172.21.15.63:0/2250526248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1d54d0 0x56137b5992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.487+0000 7f105af8d740 1 --2- 172.21.15.63:0/2250526248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b598da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.487+0000 7f105af8d740 1 --2- 172.21.15.63:0/2250526248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1f61b0 0x56137b598860 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.487+0000 7f105af8d740 1 -- 172.21.15.63:0/2250526248 >> 172.21.15.63:0/2250526248 conn(0x56137b1ea300 msgr2=0x56137b595010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.488+0000 7f105af8d740 1 -- 172.21.15.63:0/2250526248 shutdown_connections 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.488+0000 7f105af8d740 1 -- 172.21.15.63:0/2250526248 wait complete. 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.488+0000 7f105af8d740 1 -- start start 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.488+0000 7f105af8d740 1 Processor -- start 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.489+0000 7f105af8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b5a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.489+0000 7f105af8d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.489+0000 7f105af8d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.489+0000 7f105af8d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56137b1df790 con 0x56137b1d54d0 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.489+0000 7f105af8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56137b4a1bc0 con 0x56137b1cd2f0 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.489+0000 7f105af8d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56137b5a5930 con 0x56137b1f61b0 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.489+0000 7f105af8d740 10 client.0 _ll_register_callbacks cb 0x56137b47f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.489+0000 7f1051600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.489+0000 7f1051600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48890/0 (socket says 172.21.15.63:48890) 2024-05-27T07:57:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.489+0000 7f1051600640 1 -- 172.21.15.63:0/584615314 learned_addr learned my addr 172.21.15.63:0/584615314 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.490+0000 7f1050c00640 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b5a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.490+0000 7f1050c00640 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.490+0000 7f1051600640 1 -- 172.21.15.63:0/584615314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 msgr2=0x56137b5aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.490+0000 7f1051600640 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.490+0000 7f1051600640 1 -- 172.21.15.63:0/584615314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 msgr2=0x56137b5a29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.490+0000 7f1051600640 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b5a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.490+0000 7f1051600640 1 -- 172.21.15.63:0/584615314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137b20e210 con 0x56137b1d54d0 2024-05-27T07:57:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.490+0000 7f1050c00640 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.490+0000 7f1051600640 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a2f30 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f10400126d0 tx=0x7f1040012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbad1712d2da3231 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.490+0000 7f1032a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.491+0000 7f1032a00640 1 -- 172.21.15.63:0/584615314 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104000f8c0 con 0x56137b1d54d0 2024-05-27T07:57:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.491+0000 7f1032a00640 1 -- 172.21.15.63:0/584615314 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f104000fa60 con 0x56137b1d54d0 2024-05-27T07:57:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.491+0000 7f105af8d740 1 -- 172.21.15.63:0/584615314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10380110d0 con 0x56137b1d54d0 2024-05-27T07:57:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.491+0000 7f1032a00640 1 -- 172.21.15.63:0/584615314 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f104000fc60 con 0x56137b1d54d0 2024-05-27T07:57:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.491+0000 7f105af8d740 10 client.20004 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.491+0000 7f105af8d740 1 -- 172.21.15.63:0/584615314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56137b2da620 con 0x56137b1d54d0 2024-05-27T07:57:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.491+0000 7f1032a00640 1 -- 172.21.15.63:0/584615314 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10380110d0 con 0x56137b1d54d0 2024-05-27T07:57:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.492+0000 7f1032a00640 1 -- 172.21.15.63:0/584615314 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1040010430 con 0x56137b1d54d0 2024-05-27T07:57:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.492+0000 7f105af8d740 10 client.20004 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.492+0000 7f105af8d740 10 client.20004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.492+0000 7f105af8d740 20 client.20004 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.492+0000 7f105af8d740 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56137b1b4d80 0x56137b5b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.492+0000 7f105af8d740 4 client.20004 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.492+0000 7f105af8d740 1 -- 172.21.15.63:0/584615314 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56137b50bae0 con 0x56137b1b4d80 2024-05-27T07:57:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.493+0000 7f1050c00640 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56137b1b4d80 0x56137b5b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.493+0000 7f1050c00640 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56137b1b4d80 0x56137b5b4c60 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd21a914e54e338 server_cookie=c6215f2059563af8 in_seq=0 out_seq=0 2024-05-27T07:57:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.493+0000 7f1032a00640 10 client.20004 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.495+0000 7f1032a00640 1 -- 172.21.15.63:0/584615314 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56137b50bae0 con 0x56137b1b4d80 2024-05-27T07:57:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.495+0000 7f1032a00640 10 client.20004 handle_command_reply: tid=0 2024-05-27T07:57:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.495+0000 7f105af8d740 1 client.20004 shutdown 2024-05-27T07:57:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 1 -- 172.21.15.63:0/584615314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 msgr2=0x56137b5a2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a2f30 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f10400126d0 tx=0x7f1040012b90 comp rx=0 tx=0).stop 2024-05-27T07:57:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 1 -- 172.21.15.63:0/584615314 shutdown_connections 2024-05-27T07:57:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56137b1b4d80 0x56137b5b4c60 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a2f30 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 1 --2- 172.21.15.63:0/584615314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b5a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 1 -- 172.21.15.63:0/584615314 >> 172.21.15.63:0/584615314 conn(0x56137b1ea300 msgr2=0x56137b598630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f1032a00640 0 client.20004 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 1 -- 172.21.15.63:0/584615314 shutdown_connections 2024-05-27T07:57:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 1 -- 172.21.15.63:0/584615314 wait complete. 2024-05-27T07:57:19.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.496+0000 7f105af8d740 20 client.20004 trim_cache size 0 max 16384 2024-05-27T07:57:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.515+0000 7f105af8d740 1 Processor -- start 2024-05-27T07:57:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.515+0000 7f105af8d740 1 -- start start 2024-05-27T07:57:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.515+0000 7f105af8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b59cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.515+0000 7f105af8d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.515+0000 7f105af8d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.515+0000 7f105af8d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f104c1b5d00 con 0x56137b1d54d0 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.515+0000 7f105af8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f104c1b35d0 con 0x56137b1cd2f0 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.515+0000 7f105af8d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f104c1b1440 con 0x56137b1f61b0 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1051600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1051600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48906/0 (socket says 172.21.15.63:48906) 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1051600640 1 -- 172.21.15.63:0/250495409 learned_addr learned my addr 172.21.15.63:0/250495409 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1050c00640 1 --2- 172.21.15.63:0/250495409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b59cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1050c00640 1 --2- 172.21.15.63:0/250495409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1051600640 1 -- 172.21.15.63:0/250495409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 msgr2=0x56137b5ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1051600640 1 --2- 172.21.15.63:0/250495409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1051600640 1 -- 172.21.15.63:0/250495409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 msgr2=0x56137b59cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1051600640 1 --2- 172.21.15.63:0/250495409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b59cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1051600640 1 -- 172.21.15.63:0/250495409 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137b590080 con 0x56137b1d54d0 2024-05-27T07:57:19.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.516+0000 7f1050c00640 1 --2- 172.21.15.63:0/250495409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b59cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.517+0000 7f1051600640 1 --2- 172.21.15.63:0/250495409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a4820 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f10400108d0 tx=0x7f104000fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac87d5a2e198e848 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.517+0000 7f103dc00640 1 -- 172.21.15.63:0/250495409 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040000e60 con 0x56137b1d54d0 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.517+0000 7f103dc00640 1 -- 172.21.15.63:0/250495409 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10400056b0 con 0x56137b1d54d0 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.517+0000 7f103dc00640 1 -- 172.21.15.63:0/250495409 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104001c560 con 0x56137b1d54d0 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.518+0000 7f105af8d740 1 -- 172.21.15.63:0/250495409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 msgr2=0x56137b5a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.518+0000 7f105af8d740 1 --2- 172.21.15.63:0/250495409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a4820 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f10400108d0 tx=0x7f104000fe80 comp rx=0 tx=0).stop 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.518+0000 7f105af8d740 1 -- 172.21.15.63:0/250495409 shutdown_connections 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.518+0000 7f105af8d740 1 --2- 172.21.15.63:0/250495409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.518+0000 7f105af8d740 1 --2- 172.21.15.63:0/250495409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b1d54d0 0x56137b5a4820 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.518+0000 7f105af8d740 1 --2- 172.21.15.63:0/250495409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b59cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.518+0000 7f105af8d740 1 -- 172.21.15.63:0/250495409 >> 172.21.15.63:0/250495409 conn(0x56137b1ea300 msgr2=0x56137b5a4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.518+0000 7f105af8d740 1 -- 172.21.15.63:0/250495409 shutdown_connections 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.518+0000 7f105af8d740 1 -- 172.21.15.63:0/250495409 wait complete. 2024-05-27T07:57:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.519+0000 7f105af8d740 1 -- start start 2024-05-27T07:57:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.519+0000 7f105af8d740 1 Processor -- start 2024-05-27T07:57:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f105af8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b5a7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f105af8d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f105af8d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b18a1c0 0x56137b5a8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f105af8d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f104c1b1a90 con 0x56137b18a1c0 2024-05-27T07:57:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f105af8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f104c1b2f00 con 0x56137b1cd2f0 2024-05-27T07:57:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f105af8d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f104c1b2160 con 0x56137b1f61b0 2024-05-27T07:57:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f105af8d740 10 client.0 _ll_register_callbacks cb 0x7f1040009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f1050c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b18a1c0 0x56137b5a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f1050c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b18a1c0 0x56137b5a8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48920/0 (socket says 172.21.15.63:48920) 2024-05-27T07:57:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f1050c00640 1 -- 172.21.15.63:0/865333785 learned_addr learned my addr 172.21.15.63:0/865333785 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f1050c00640 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b5a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.520+0000 7f1051600640 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.521+0000 7f1050c00640 1 -- 172.21.15.63:0/865333785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 msgr2=0x56137b5a7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.521+0000 7f1050c00640 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5a7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.521+0000 7f1050c00640 1 -- 172.21.15.63:0/865333785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 msgr2=0x56137b5a7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.521+0000 7f1050c00640 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b5a7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.521+0000 7f1050c00640 1 -- 172.21.15.63:0/865333785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137b61b340 con 0x56137b18a1c0 2024-05-27T07:57:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.521+0000 7f1050c00640 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b18a1c0 0x56137b5a8300 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f10380058c0 tx=0x7f103800b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93902aa26d7fdd61 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.521+0000 7f1032a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.522+0000 7f1032a00640 1 -- 172.21.15.63:0/865333785 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103801f450 con 0x56137b18a1c0 2024-05-27T07:57:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.522+0000 7f1032a00640 1 -- 172.21.15.63:0/865333785 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f1038028040 con 0x56137b18a1c0 2024-05-27T07:57:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.522+0000 7f105af8d740 1 -- 172.21.15.63:0/865333785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10400056b0 con 0x56137b18a1c0 2024-05-27T07:57:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.522+0000 7f1032a00640 1 -- 172.21.15.63:0/865333785 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f103801f5f0 con 0x56137b18a1c0 2024-05-27T07:57:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.522+0000 7f105af8d740 10 client.20010 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.522+0000 7f105af8d740 1 -- 172.21.15.63:0/865333785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56137b61c410 con 0x56137b18a1c0 2024-05-27T07:57:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.522+0000 7f1032a00640 1 -- 172.21.15.63:0/865333785 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10400056b0 con 0x56137b18a1c0 2024-05-27T07:57:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.522+0000 7f1032a00640 1 -- 172.21.15.63:0/865333785 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1038028440 con 0x56137b18a1c0 2024-05-27T07:57:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.523+0000 7f105af8d740 10 client.20010 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.523+0000 7f105af8d740 10 client.20010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.523+0000 7f105af8d740 20 client.20010 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.523+0000 7f105af8d740 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56137b61c890 0x56137b61ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.523+0000 7f105af8d740 4 client.20010 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:57:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.523+0000 7f105af8d740 1 -- 172.21.15.63:0/865333785 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56137b5052c0 con 0x56137b61c890 2024-05-27T07:57:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.523+0000 7f1051600640 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56137b61c890 0x56137b61ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.524+0000 7f1032a00640 10 client.20010 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.524+0000 7f1051600640 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56137b61c890 0x56137b61ecb0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c014748bff33b0 server_cookie=175ed8fbca740678 in_seq=0 out_seq=0 2024-05-27T07:57:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.563+0000 7f1032a00640 1 -- 172.21.15.63:0/865333785 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56137b5052c0 con 0x56137b61c890 2024-05-27T07:57:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.563+0000 7f1032a00640 10 client.20010 handle_command_reply: tid=0 2024-05-27T07:57:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.563+0000 7f105af8d740 1 client.20010 shutdown 2024-05-27T07:57:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.563+0000 7f105af8d740 1 -- 172.21.15.63:0/865333785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b18a1c0 msgr2=0x56137b5a8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.563+0000 7f105af8d740 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b18a1c0 0x56137b5a8300 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f10380058c0 tx=0x7f103800b570 comp rx=0 tx=0).stop 2024-05-27T07:57:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.564+0000 7f105af8d740 1 -- 172.21.15.63:0/865333785 shutdown_connections 2024-05-27T07:57:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.564+0000 7f105af8d740 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56137b61c890 0x56137b61ecb0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.564+0000 7f105af8d740 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56137b18a1c0 0x56137b5a8300 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.564+0000 7f1032a00640 0 client.20010 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.564+0000 7f105af8d740 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56137b1f61b0 0x56137b5a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.564+0000 7f105af8d740 1 --2- 172.21.15.63:0/865333785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56137b1cd2f0 0x56137b5a7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.564+0000 7f105af8d740 1 -- 172.21.15.63:0/865333785 >> 172.21.15.63:0/865333785 conn(0x56137b1ea300 msgr2=0x56137b5a4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.564+0000 7f105af8d740 1 -- 172.21.15.63:0/865333785 shutdown_connections 2024-05-27T07:57:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.564+0000 7f105af8d740 1 -- 172.21.15.63:0/865333785 wait complete. 2024-05-27T07:57:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.564+0000 7f105af8d740 20 client.20010 trim_cache size 0 max 16384 2024-05-27T07:57:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:57:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:57:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:57:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 -- 172.21.15.63:0/1121501178 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10340b2950 msgr2=0x7f10340b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10340b2950 0x7f10340b4e60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f10440094e0 tx=0x7f10440095c0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 -- 172.21.15.63:0/1121501178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f104c10c360 msgr2=0x7f104c19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f104c10c360 0x7f104c19c7f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f1044031040 tx=0x7f1044003f80 comp rx=0 tx=0).stop 2024-05-27T07:57:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 -- 172.21.15.63:0/1121501178 shutdown_connections 2024-05-27T07:57:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10340b2950 0x7f10340b4e60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f104c10c360 0x7f104c19c7f0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f104c10bec0 0x7f104c1a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 --2- 172.21.15.63:0/1121501178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f104c106610 0x7f104c1a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 -- 172.21.15.63:0/1121501178 >> 172.21.15.63:0/1121501178 conn(0x7f104c1016b0 msgr2=0x7f104c103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 -- 172.21.15.63:0/1121501178 shutdown_connections 2024-05-27T07:57:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:19.566+0000 7f1054000640 1 -- 172.21.15.63:0/1121501178 wait complete. 2024-05-27T07:57:21.433 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:21 smithi086 ceph-mon[34687]: pgmap v752: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.6 MiB/s wr, 44 op/s 2024-05-27T07:57:21.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:21 smithi149 ceph-mon[35408]: pgmap v752: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.6 MiB/s wr, 44 op/s 2024-05-27T07:57:21.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:21 smithi063 ceph-mon[31613]: pgmap v752: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.6 MiB/s wr, 44 op/s 2024-05-27T07:57:23.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:23 smithi086 ceph-mon[34687]: pgmap v753: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:57:23.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:23 smithi149 ceph-mon[35408]: pgmap v753: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:57:23.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:23 smithi063 ceph-mon[31613]: pgmap v753: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:57:25.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:25 smithi086 ceph-mon[34687]: pgmap v754: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:57:25.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:25 smithi149 ceph-mon[35408]: pgmap v754: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:57:25.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:25 smithi063 ceph-mon[31613]: pgmap v754: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 11 op/s 2024-05-27T07:57:27.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:27 smithi086 ceph-mon[34687]: pgmap v755: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T07:57:27.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:27 smithi149 ceph-mon[35408]: pgmap v755: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T07:57:27.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:27 smithi063 ceph-mon[31613]: pgmap v755: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T07:57:29.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:29 smithi086 ceph-mon[34687]: pgmap v756: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:29.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:29 smithi149 ceph-mon[35408]: pgmap v756: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:29.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:29 smithi063 ceph-mon[31613]: pgmap v756: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:31.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:31 smithi086 ceph-mon[34687]: pgmap v757: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:57:31.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:31 smithi149 ceph-mon[35408]: pgmap v757: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:57:31.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:31 smithi063 ceph-mon[31613]: pgmap v757: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T07:57:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.826+0000 7f7599a00640 1 -- 172.21.15.63:0/2814203744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7594109ab0 msgr2=0x7f7594103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.826+0000 7f7599a00640 1 --2- 172.21.15.63:0/2814203744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7594109ab0 0x7f7594103650 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f7584009900 tx=0x7f758402a400 comp rx=0 tx=0).stop 2024-05-27T07:57:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.826+0000 7f7599a00640 1 -- 172.21.15.63:0/2814203744 shutdown_connections 2024-05-27T07:57:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.826+0000 7f7599a00640 1 --2- 172.21.15.63:0/2814203744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7594104200 0x7f75941105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.826+0000 7f7599a00640 1 --2- 172.21.15.63:0/2814203744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7594109f50 0x7f7594103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.826+0000 7f7599a00640 1 --2- 172.21.15.63:0/2814203744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7594109ab0 0x7f7594103650 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.826+0000 7f7599a00640 1 -- 172.21.15.63:0/2814203744 >> 172.21.15.63:0/2814203744 conn(0x7f7594079140 msgr2=0x7f7594079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.826+0000 7f7599a00640 1 -- 172.21.15.63:0/2814203744 shutdown_connections 2024-05-27T07:57:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.826+0000 7f7599a00640 1 -- 172.21.15.63:0/2814203744 wait complete. 2024-05-27T07:57:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.828+0000 7f7599a00640 1 Processor -- start 2024-05-27T07:57:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.828+0000 7f7599a00640 1 -- start start 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.828+0000 7f7599a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7594104200 0x7f759419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.828+0000 7f7599a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7594109ab0 0x7f759419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.828+0000 7f7599a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7594109f50 0x7f75941a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.828+0000 7f7599a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7594113b50 con 0x7f7594109f50 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.828+0000 7f7599a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75941139d0 con 0x7f7594109ab0 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.828+0000 7f7599a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7594113cd0 con 0x7f7594104200 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7599000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7594109f50 0x7f75941a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7599000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7594109f50 0x7f75941a84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32892/0 (socket says 172.21.15.63:32892) 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7599000640 1 -- 172.21.15.63:0/1885901066 learned_addr learned my addr 172.21.15.63:0/1885901066 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7593e00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7594104200 0x7f759419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7593400640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7594109ab0 0x7f759419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7599000640 1 -- 172.21.15.63:0/1885901066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7594104200 msgr2=0x7f759419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7599000640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7594104200 0x7f759419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7599000640 1 -- 172.21.15.63:0/1885901066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7594109ab0 msgr2=0x7f759419db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7599000640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7594109ab0 0x7f759419db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7599000640 1 -- 172.21.15.63:0/1885901066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75941a89f0 con 0x7f7594109f50 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.829+0000 7f7593e00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7594104200 0x7f759419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.830+0000 7f7599000640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7594109f50 0x7f75941a84b0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f758c00eaa0 tx=0x7f758c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.830+0000 7f7590c00640 1 -- 172.21.15.63:0/1885901066 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c00cdb0 con 0x7f7594109f50 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.830+0000 7f7590c00640 1 -- 172.21.15.63:0/1885901066 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f758c005020 con 0x7f7594109f50 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.830+0000 7f7590c00640 1 -- 172.21.15.63:0/1885901066 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c0116d0 con 0x7f7594109f50 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.830+0000 7f7599a00640 1 -- 172.21.15.63:0/1885901066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75941a8c80 con 0x7f7594109f50 2024-05-27T07:57:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.830+0000 7f7599a00640 1 -- 172.21.15.63:0/1885901066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75941a9150 con 0x7f7594109f50 2024-05-27T07:57:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.834+0000 7f75a1fc5740 1 Processor -- start 2024-05-27T07:57:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.835+0000 7f7590c00640 1 -- 172.21.15.63:0/1885901066 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f758c0051d0 con 0x7f7594109f50 2024-05-27T07:57:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.835+0000 7f75a1fc5740 1 -- start start 2024-05-27T07:57:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.835+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.835+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f60162f0 0x5601f63e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.835+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f63e24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.835+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601f6352b90 con 0x5601f603f1b0 2024-05-27T07:57:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.835+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5601f6342730 con 0x5601f601e4d0 2024-05-27T07:57:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.835+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5601f6346770 con 0x5601f60162f0 2024-05-27T07:57:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.836+0000 7f7593400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.836+0000 7f7593400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63e1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32908/0 (socket says 172.21.15.63:32908) 2024-05-27T07:57:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.836+0000 7f7593400640 1 -- 172.21.15.63:0/1600223123 learned_addr learned my addr 172.21.15.63:0/1600223123 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.836+0000 7f7593e00640 1 --2- 172.21.15.63:0/1600223123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f60162f0 0x5601f63e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.836+0000 7f7593400640 1 --2- 172.21.15.63:0/1600223123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f63e24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.836+0000 7f7593400640 1 -- 172.21.15.63:0/1600223123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f60162f0 msgr2=0x5601f63e1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.836+0000 7f7593400640 1 --2- 172.21.15.63:0/1600223123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f60162f0 0x5601f63e1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.836+0000 7f7593400640 1 -- 172.21.15.63:0/1600223123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 msgr2=0x5601f63e24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.836+0000 7f7593400640 1 --2- 172.21.15.63:0/1600223123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f63e24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.836+0000 7f7593400640 1 -- 172.21.15.63:0/1600223123 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601f62db8b0 con 0x5601f603f1b0 2024-05-27T07:57:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.837+0000 7f7593400640 1 --2- 172.21.15.63:0/1600223123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63e1a60 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f758000be60 tx=0x7f758000b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c5400a48cd492c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.837+0000 7f7590c00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75780b29d0 0x7f75780b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.837+0000 7f7590c00640 1 -- 172.21.15.63:0/1885901066 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f758c0e2c10 con 0x7f7594109f50 2024-05-27T07:57:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.837+0000 7f7589600640 1 -- 172.21.15.63:0/1600223123 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580011e50 con 0x5601f603f1b0 2024-05-27T07:57:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.837+0000 7f7589600640 1 -- 172.21.15.63:0/1600223123 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f758000f880 con 0x5601f603f1b0 2024-05-27T07:57:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.837+0000 7f7589600640 1 -- 172.21.15.63:0/1600223123 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580012750 con 0x5601f603f1b0 2024-05-27T07:57:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.837+0000 7f7593e00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75780b29d0 0x7f75780b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.838+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1600223123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 msgr2=0x5601f63e1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.838+0000 7f7593e00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75780b29d0 0x7f75780b4ee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7584006010 tx=0x7f75840058b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.838+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/1600223123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63e1a60 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f758000be60 tx=0x7f758000b5a0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.839+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1600223123 shutdown_connections 2024-05-27T07:57:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.839+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/1600223123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f63e24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.839+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/1600223123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f60162f0 0x5601f63e1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.839+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/1600223123 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63e1a60 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.839+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1600223123 >> 172.21.15.63:0/1600223123 conn(0x5601f6033300 msgr2=0x5601f63de210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.839+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1600223123 shutdown_connections 2024-05-27T07:57:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.839+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1600223123 wait complete. 2024-05-27T07:57:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.839+0000 7f75a1fc5740 1 -- start start 2024-05-27T07:57:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.840+0000 7f75a1fc5740 1 Processor -- start 2024-05-27T07:57:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.840+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f60162f0 0x5601f63ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.840+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f63ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.840+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f603f1b0 0x5601f63f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.840+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601f6028790 con 0x5601f60162f0 2024-05-27T07:57:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.840+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5601f62eabc0 con 0x5601f601e4d0 2024-05-27T07:57:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.840+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5601f63eeb30 con 0x5601f603f1b0 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f75a1fc5740 10 client.0 _ll_register_callbacks cb 0x5601f62c8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f7593e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f60162f0 0x5601f63ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f7593e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f60162f0 0x5601f63ebbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32916/0 (socket says 172.21.15.63:32916) 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f7593e00640 1 -- 172.21.15.63:0/1894733555 learned_addr learned my addr 172.21.15.63:0/1894733555 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f7593400640 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f63ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f7599000640 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f603f1b0 0x5601f63f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f7593e00640 1 -- 172.21.15.63:0/1894733555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f603f1b0 msgr2=0x5601f63f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f7593e00640 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f603f1b0 0x5601f63f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f7593e00640 1 -- 172.21.15.63:0/1894733555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 msgr2=0x5601f63ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f7593e00640 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f63ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.841+0000 7f7593e00640 1 -- 172.21.15.63:0/1894733555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601f6057210 con 0x5601f60162f0 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.842+0000 7f7593e00640 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f60162f0 0x5601f63ebbb0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f7584009900 tx=0x7f7584009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741dd6fb55a4968d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.842+0000 7f757e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.842+0000 7f757e000640 1 -- 172.21.15.63:0/1894733555 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758404b070 con 0x5601f60162f0 2024-05-27T07:57:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.842+0000 7f757e000640 1 -- 172.21.15.63:0/1894733555 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f7584005500 con 0x5601f60162f0 2024-05-27T07:57:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.842+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1894733555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f758000f880 con 0x5601f60162f0 2024-05-27T07:57:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.842+0000 7f757e000640 1 -- 172.21.15.63:0/1894733555 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7584057050 con 0x5601f60162f0 2024-05-27T07:57:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.842+0000 7f75a1fc5740 10 client.20022 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.842+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1894733555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5601f6123620 con 0x5601f60162f0 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.843+0000 7f757e000640 1 -- 172.21.15.63:0/1894733555 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f758000f880 con 0x5601f60162f0 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.843+0000 7f757e000640 1 -- 172.21.15.63:0/1894733555 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f7584003940 con 0x5601f60162f0 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.843+0000 7f75a1fc5740 10 client.20022 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.843+0000 7f75a1fc5740 10 client.20022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.843+0000 7f75a1fc5740 20 client.20022 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.843+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601f5ffdd80 0x5601f63fdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.843+0000 7f75a1fc5740 4 client.20022 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.843+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1894733555 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601f6354ae0 con 0x5601f5ffdd80 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.844+0000 7f7593400640 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601f5ffdd80 0x5601f63fdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.845+0000 7f7593400640 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601f5ffdd80 0x5601f63fdd70 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b711bb177ddfe444 server_cookie=e757bfe84694f07f in_seq=0 out_seq=0 2024-05-27T07:57:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.845+0000 7f757e000640 10 client.20022 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f757e000640 1 -- 172.21.15.63:0/1894733555 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5601f6354ae0 con 0x5601f5ffdd80 2024-05-27T07:57:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f757e000640 10 client.20022 handle_command_reply: tid=0 2024-05-27T07:57:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f75a1fc5740 1 client.20022 shutdown 2024-05-27T07:57:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1894733555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f60162f0 msgr2=0x5601f63ebbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f60162f0 0x5601f63ebbb0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f7584009900 tx=0x7f7584009260 comp rx=0 tx=0).stop 2024-05-27T07:57:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1894733555 shutdown_connections 2024-05-27T07:57:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601f5ffdd80 0x5601f63fdd70 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f603f1b0 0x5601f63f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f63ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/1894733555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f60162f0 0x5601f63ebbb0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1894733555 >> 172.21.15.63:0/1894733555 conn(0x5601f6033300 msgr2=0x5601f63e1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.847+0000 7f757e000640 0 client.20022 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.848+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1894733555 shutdown_connections 2024-05-27T07:57:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.848+0000 7f75a1fc5740 1 -- 172.21.15.63:0/1894733555 wait complete. 2024-05-27T07:57:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.848+0000 7f75a1fc5740 20 client.20022 trim_cache size 0 max 16384 2024-05-27T07:57:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.867+0000 7f75a1fc5740 1 Processor -- start 2024-05-27T07:57:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.867+0000 7f75a1fc5740 1 -- start start 2024-05-27T07:57:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.867+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f60162f0 0x5601f63dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.867+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f601e4d0 0x5601f63ee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.867+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f603f1b0 0x5601f63ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.867+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75941b5ae0 con 0x5601f601e4d0 2024-05-27T07:57:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.867+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75941b33b0 con 0x5601f60162f0 2024-05-27T07:57:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.867+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f75941b1220 con 0x5601f603f1b0 2024-05-27T07:57:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.868+0000 7f7593400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f601e4d0 0x5601f63ee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.868+0000 7f7593400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f601e4d0 0x5601f63ee2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32922/0 (socket says 172.21.15.63:32922) 2024-05-27T07:57:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.868+0000 7f7593400640 1 -- 172.21.15.63:0/3524914550 learned_addr learned my addr 172.21.15.63:0/3524914550 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.868+0000 7f7599000640 1 --2- 172.21.15.63:0/3524914550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f603f1b0 0x5601f63ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.868+0000 7f7593e00640 1 --2- 172.21.15.63:0/3524914550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f60162f0 0x5601f63dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.868+0000 7f7593400640 1 -- 172.21.15.63:0/3524914550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f603f1b0 msgr2=0x5601f63ebb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.868+0000 7f7593400640 1 --2- 172.21.15.63:0/3524914550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f603f1b0 0x5601f63ebb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.868+0000 7f7593400640 1 -- 172.21.15.63:0/3524914550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f60162f0 msgr2=0x5601f63dd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.868+0000 7f7593400640 1 --2- 172.21.15.63:0/3524914550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f60162f0 0x5601f63dd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.868+0000 7f7593400640 1 -- 172.21.15.63:0/3524914550 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601f63dec00 con 0x5601f601e4d0 2024-05-27T07:57:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.869+0000 7f7593400640 1 --2- 172.21.15.63:0/3524914550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f601e4d0 0x5601f63ee2e0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f7580011be0 tx=0x7f7580014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abea47221421b35c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.869+0000 7f7588c00640 1 -- 172.21.15.63:0/3524914550 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580005240 con 0x5601f601e4d0 2024-05-27T07:57:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.869+0000 7f7588c00640 1 -- 172.21.15.63:0/3524914550 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75800053e0 con 0x5601f601e4d0 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.869+0000 7f7588c00640 1 -- 172.21.15.63:0/3524914550 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580029430 con 0x5601f601e4d0 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.870+0000 7f75a1fc5740 1 -- 172.21.15.63:0/3524914550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f601e4d0 msgr2=0x5601f63ee2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.870+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/3524914550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f601e4d0 0x5601f63ee2e0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f7580011be0 tx=0x7f7580014040 comp rx=0 tx=0).stop 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.870+0000 7f75a1fc5740 1 -- 172.21.15.63:0/3524914550 shutdown_connections 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.870+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/3524914550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f603f1b0 0x5601f63ebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.870+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/3524914550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f601e4d0 0x5601f63ee2e0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.870+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/3524914550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f60162f0 0x5601f63dd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.870+0000 7f75a1fc5740 1 -- 172.21.15.63:0/3524914550 >> 172.21.15.63:0/3524914550 conn(0x5601f6033300 msgr2=0x5601f63eb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.871+0000 7f75a1fc5740 1 -- 172.21.15.63:0/3524914550 shutdown_connections 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.871+0000 7f75a1fc5740 1 -- 172.21.15.63:0/3524914550 wait complete. 2024-05-27T07:57:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.871+0000 7f75a1fc5740 1 -- start start 2024-05-27T07:57:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.871+0000 7f75a1fc5740 1 Processor -- start 2024-05-27T07:57:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f60162f0 0x5601f6466f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f64674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f75a1fc5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63ef380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75941b1870 con 0x5601f603f1b0 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75941b2ce0 con 0x5601f601e4d0 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f75a1fc5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f75941b1f40 con 0x5601f60162f0 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f75a1fc5740 10 client.0 _ll_register_callbacks cb 0x7f7584009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f7599000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f7599000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63ef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:32934/0 (socket says 172.21.15.63:32934) 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f7599000640 1 -- 172.21.15.63:0/2042368666 learned_addr learned my addr 172.21.15.63:0/2042368666 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.872+0000 7f7593e00640 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f60162f0 0x5601f6466f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f7593400640 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f64674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f7599000640 1 -- 172.21.15.63:0/2042368666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f60162f0 msgr2=0x5601f6466f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f7599000640 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f60162f0 0x5601f6466f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f7599000640 1 -- 172.21.15.63:0/2042368666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 msgr2=0x5601f64674b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f7599000640 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f64674b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f7599000640 1 -- 172.21.15.63:0/2042368666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601f63ef970 con 0x5601f603f1b0 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f7593400640 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f64674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f7599000640 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63ef380 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f758c0a9a60 tx=0x7f758c0b2270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf24e0711fad02d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f757e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f757e000640 1 -- 172.21.15.63:0/2042368666 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c01f040 con 0x5601f603f1b0 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f757e000640 1 -- 172.21.15.63:0/2042368666 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f758c0aae00 con 0x5601f603f1b0 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.873+0000 7f75a1fc5740 1 -- 172.21.15.63:0/2042368666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75800053e0 con 0x5601f603f1b0 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f757e000640 1 -- 172.21.15.63:0/2042368666 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f758c0a9ad0 con 0x5601f603f1b0 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f75a1fc5740 10 client.20028 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:32.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f75a1fc5740 1 -- 172.21.15.63:0/2042368666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5601f646e5c0 con 0x5601f603f1b0 2024-05-27T07:57:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f757e000640 1 -- 172.21.15.63:0/2042368666 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75800053e0 con 0x5601f603f1b0 2024-05-27T07:57:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f757e000640 1 -- 172.21.15.63:0/2042368666 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f758c00f040 con 0x5601f603f1b0 2024-05-27T07:57:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f75a1fc5740 10 client.20028 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f75a1fc5740 10 client.20028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f75a1fc5740 20 client.20028 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601f5fd31c0 0x5601f6470d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f75a1fc5740 4 client.20028 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:57:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.874+0000 7f75a1fc5740 1 -- 172.21.15.63:0/2042368666 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601f634e2c0 con 0x5601f5fd31c0 2024-05-27T07:57:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.875+0000 7f7593e00640 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601f5fd31c0 0x5601f6470d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.876+0000 7f7593e00640 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601f5fd31c0 0x5601f6470d80 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d962a66651c1bb9 server_cookie=7fe98a1bfb300b8b in_seq=0 out_seq=0 2024-05-27T07:57:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.876+0000 7f757e000640 10 client.20028 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f757e000640 1 -- 172.21.15.63:0/2042368666 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5601f634e2c0 con 0x5601f5fd31c0 2024-05-27T07:57:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f757e000640 10 client.20028 handle_command_reply: tid=0 2024-05-27T07:57:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f75a1fc5740 1 client.20028 shutdown 2024-05-27T07:57:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f75a1fc5740 1 -- 172.21.15.63:0/2042368666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 msgr2=0x5601f63ef380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63ef380 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f758c0a9a60 tx=0x7f758c0b2270 comp rx=0 tx=0).stop 2024-05-27T07:57:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f75a1fc5740 1 -- 172.21.15.63:0/2042368666 shutdown_connections 2024-05-27T07:57:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5601f5fd31c0 0x5601f6470d80 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601f603f1b0 0x5601f63ef380 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5601f601e4d0 0x5601f64674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f75a1fc5740 1 --2- 172.21.15.63:0/2042368666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601f60162f0 0x5601f6466f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f757e000640 0 client.20028 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.911+0000 7f75a1fc5740 1 -- 172.21.15.63:0/2042368666 >> 172.21.15.63:0/2042368666 conn(0x5601f6033300 msgr2=0x5601f63dcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.912+0000 7f75a1fc5740 1 -- 172.21.15.63:0/2042368666 shutdown_connections 2024-05-27T07:57:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.912+0000 7f75a1fc5740 1 -- 172.21.15.63:0/2042368666 wait complete. 2024-05-27T07:57:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.912+0000 7f75a1fc5740 20 client.20028 trim_cache size 0 max 16384 2024-05-27T07:57:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:57:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:57:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:57:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.913+0000 7f7599a00640 1 -- 172.21.15.63:0/1885901066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75780b29d0 msgr2=0x7f75780b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.913+0000 7f7599a00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75780b29d0 0x7f75780b4ee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7584006010 tx=0x7f75840058b0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.913+0000 7f7599a00640 1 -- 172.21.15.63:0/1885901066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7594109f50 msgr2=0x7f75941a84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.913+0000 7f7599a00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7594109f50 0x7f75941a84b0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f758c00eaa0 tx=0x7f758c00ef60 comp rx=0 tx=0).stop 2024-05-27T07:57:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.913+0000 7f7599a00640 1 -- 172.21.15.63:0/1885901066 shutdown_connections 2024-05-27T07:57:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.913+0000 7f7599a00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75780b29d0 0x7f75780b4ee0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.913+0000 7f7599a00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7594109f50 0x7f75941a84b0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.913+0000 7f7599a00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7594109ab0 0x7f759419db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.913+0000 7f7599a00640 1 --2- 172.21.15.63:0/1885901066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7594104200 0x7f759419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.913+0000 7f7599a00640 1 -- 172.21.15.63:0/1885901066 >> 172.21.15.63:0/1885901066 conn(0x7f7594079140 msgr2=0x7f7594102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.914+0000 7f7599a00640 1 -- 172.21.15.63:0/1885901066 shutdown_connections 2024-05-27T07:57:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:32.914+0000 7f7599a00640 1 -- 172.21.15.63:0/1885901066 wait complete. 2024-05-27T07:57:33.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:33 smithi086 ceph-mon[34687]: pgmap v758: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:33.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:33 smithi149 ceph-mon[35408]: pgmap v758: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:33.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:33 smithi063 ceph-mon[31613]: pgmap v758: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:35.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:35 smithi086 ceph-mon[34687]: pgmap v759: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:57:35.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:35 smithi149 ceph-mon[35408]: pgmap v759: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:57:35.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:35 smithi063 ceph-mon[31613]: pgmap v759: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:57:37.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:37 smithi086 ceph-mon[34687]: pgmap v760: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:57:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:37 smithi149 ceph-mon[35408]: pgmap v760: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:57:37.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:37 smithi063 ceph-mon[31613]: pgmap v760: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T07:57:39.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:39 smithi086 ceph-mon[34687]: pgmap v761: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:39 smithi149 ceph-mon[35408]: pgmap v761: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:39.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:39 smithi063 ceph-mon[31613]: pgmap v761: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:41.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:41 smithi086 ceph-mon[34687]: pgmap v762: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:57:41.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:41 smithi149 ceph-mon[35408]: pgmap v762: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:57:41.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:41 smithi063 ceph-mon[31613]: pgmap v762: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T07:57:43.360 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:57:43.447 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:43 smithi063 ceph-mon[31613]: pgmap v763: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:43.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:43 smithi086 ceph-mon[34687]: pgmap v763: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:43 smithi149 ceph-mon[35408]: pgmap v763: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:57:43.867 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:45.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:45 smithi086 ceph-mon[34687]: pgmap v764: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:57:45.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:45 smithi149 ceph-mon[35408]: pgmap v764: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:57:45.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:45 smithi063 ceph-mon[31613]: pgmap v764: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:57:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.165+0000 7fd982e00640 1 -- 172.21.15.63:0/492923352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd97c10b6e0 msgr2=0x7fd97c10db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.165+0000 7fd982e00640 1 --2- 172.21.15.63:0/492923352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd97c10b6e0 0x7fd97c10db00 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fd968009900 tx=0x7fd96802a480 comp rx=0 tx=0).stop 2024-05-27T07:57:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.165+0000 7fd982e00640 1 -- 172.21.15.63:0/492923352 shutdown_connections 2024-05-27T07:57:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.165+0000 7fd982e00640 1 --2- 172.21.15.63:0/492923352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c10e040 0x7fd97c110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.165+0000 7fd982e00640 1 --2- 172.21.15.63:0/492923352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd97c10b6e0 0x7fd97c10db00 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.165+0000 7fd982e00640 1 --2- 172.21.15.63:0/492923352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd97c104c80 0x7fd97c105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.165+0000 7fd982e00640 1 -- 172.21.15.63:0/492923352 >> 172.21.15.63:0/492923352 conn(0x7fd97c0790c0 msgr2=0x7fd97c0794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.165+0000 7fd982e00640 1 -- 172.21.15.63:0/492923352 shutdown_connections 2024-05-27T07:57:46.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.165+0000 7fd982e00640 1 -- 172.21.15.63:0/492923352 wait complete. 2024-05-27T07:57:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.167+0000 7fd982e00640 1 Processor -- start 2024-05-27T07:57:46.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.167+0000 7fd982e00640 1 -- start start 2024-05-27T07:57:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.167+0000 7fd982e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd97c104c80 0x7fd97c19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.167+0000 7fd982e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd97c10b6e0 0x7fd97c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.167+0000 7fd982e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c10e040 0x7fd97c1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.167+0000 7fd982e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c113b50 con 0x7fd97c10b6e0 2024-05-27T07:57:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.167+0000 7fd982e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c1139d0 con 0x7fd97c10e040 2024-05-27T07:57:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.167+0000 7fd982e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c113cd0 con 0x7fd97c104c80 2024-05-27T07:57:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.168+0000 7fd981000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd97c10b6e0 0x7fd97c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.168+0000 7fd981000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd97c10b6e0 0x7fd97c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52812/0 (socket says 172.21.15.63:52812) 2024-05-27T07:57:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.168+0000 7fd981000640 1 -- 172.21.15.63:0/2968996202 learned_addr learned my addr 172.21.15.63:0/2968996202 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:46.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.168+0000 7fd982400640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c10e040 0x7fd97c1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.168+0000 7fd981a00640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd97c104c80 0x7fd97c19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.168+0000 7fd981000640 1 -- 172.21.15.63:0/2968996202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd97c104c80 msgr2=0x7fd97c19c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.168+0000 7fd981000640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd97c104c80 0x7fd97c19c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.168+0000 7fd981000640 1 -- 172.21.15.63:0/2968996202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c10e040 msgr2=0x7fd97c1a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.169+0000 7fd981000640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c10e040 0x7fd97c1a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.169+0000 7fd981000640 1 -- 172.21.15.63:0/2968996202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c1a1630 con 0x7fd97c10b6e0 2024-05-27T07:57:46.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.169+0000 7fd981000640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd97c10b6e0 0x7fd97c19cd30 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fd968009230 tx=0x7fd968004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.170+0000 7fd97a000640 1 -- 172.21.15.63:0/2968996202 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96803f030 con 0x7fd97c10b6e0 2024-05-27T07:57:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.170+0000 7fd97a000640 1 -- 172.21.15.63:0/2968996202 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd968003a80 con 0x7fd97c10b6e0 2024-05-27T07:57:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.170+0000 7fd97a000640 1 -- 172.21.15.63:0/2968996202 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968035070 con 0x7fd97c10b6e0 2024-05-27T07:57:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.170+0000 7fd982e00640 1 -- 172.21.15.63:0/2968996202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c1a18c0 con 0x7fd97c10b6e0 2024-05-27T07:57:46.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.170+0000 7fd982e00640 1 -- 172.21.15.63:0/2968996202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c1a1d60 con 0x7fd97c10b6e0 2024-05-27T07:57:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.175+0000 7fd97a000640 1 -- 172.21.15.63:0/2968996202 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd968042030 con 0x7fd97c10b6e0 2024-05-27T07:57:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.175+0000 7fd98b408740 1 Processor -- start 2024-05-27T07:57:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.176+0000 7fd98b408740 1 -- start start 2024-05-27T07:57:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.176+0000 7fd98b408740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f87119a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.176+0000 7fd98b408740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 0x561f87119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.176+0000 7fd98b408740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f8711a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.176+0000 7fd98b408740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561f8708ab90 con 0x561f86d771b0 2024-05-27T07:57:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.176+0000 7fd98b408740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f8707a730 con 0x561f86d564d0 2024-05-27T07:57:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.176+0000 7fd98b408740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561f8707e770 con 0x561f86d4e2f0 2024-05-27T07:57:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd982400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f87119a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd981a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 0x561f87119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd982400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f87119a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52826/0 (socket says 172.21.15.63:52826) 2024-05-27T07:57:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd982400640 1 -- 172.21.15.63:0/3020665161 learned_addr learned my addr 172.21.15.63:0/3020665161 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd981000640 1 --2- 172.21.15.63:0/3020665161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f8711a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd97a000640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9640b29d0 0x7fd9640b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd982400640 1 -- 172.21.15.63:0/3020665161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 msgr2=0x561f87119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd982400640 1 --2- 172.21.15.63:0/3020665161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 0x561f87119fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd982400640 1 -- 172.21.15.63:0/3020665161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 msgr2=0x561f8711a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd982400640 1 --2- 172.21.15.63:0/3020665161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f8711a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.177+0000 7fd982400640 1 -- 172.21.15.63:0/3020665161 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f870138b0 con 0x561f86d771b0 2024-05-27T07:57:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.178+0000 7fd97a000640 1 -- 172.21.15.63:0/2968996202 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7fd968102e70 con 0x7fd97c10b6e0 2024-05-27T07:57:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.178+0000 7fd982400640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9640b29d0 0x7fd9640b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.178+0000 7fd982400640 1 --2- 172.21.15.63:0/3020665161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f87119a60 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fd974009f70 tx=0x7fd974010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65be5091cc78e07 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.179+0000 7fd96ea00640 1 -- 172.21.15.63:0/3020665161 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974022070 con 0x561f86d771b0 2024-05-27T07:57:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.179+0000 7fd96ea00640 1 -- 172.21.15.63:0/3020665161 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9740113e0 con 0x561f86d771b0 2024-05-27T07:57:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.179+0000 7fd96ea00640 1 -- 172.21.15.63:0/3020665161 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974016070 con 0x561f86d771b0 2024-05-27T07:57:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.180+0000 7fd98b408740 1 -- 172.21.15.63:0/3020665161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 msgr2=0x561f87119a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.180+0000 7fd982400640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9640b29d0 0x7fd9640b4ee0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd97401f000 tx=0x7fd974022000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.180+0000 7fd98b408740 1 --2- 172.21.15.63:0/3020665161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f87119a60 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fd974009f70 tx=0x7fd974010800 comp rx=0 tx=0).stop 2024-05-27T07:57:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.180+0000 7fd98b408740 1 -- 172.21.15.63:0/3020665161 shutdown_connections 2024-05-27T07:57:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.180+0000 7fd98b408740 1 --2- 172.21.15.63:0/3020665161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f8711a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.180+0000 7fd98b408740 1 --2- 172.21.15.63:0/3020665161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 0x561f87119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.180+0000 7fd98b408740 1 --2- 172.21.15.63:0/3020665161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f87119a60 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.180+0000 7fd98b408740 1 -- 172.21.15.63:0/3020665161 >> 172.21.15.63:0/3020665161 conn(0x561f86d6b300 msgr2=0x561f87116210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.181+0000 7fd98b408740 1 -- 172.21.15.63:0/3020665161 shutdown_connections 2024-05-27T07:57:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.181+0000 7fd98b408740 1 -- 172.21.15.63:0/3020665161 wait complete. 2024-05-27T07:57:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.181+0000 7fd98b408740 1 -- start start 2024-05-27T07:57:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.182+0000 7fd98b408740 1 Processor -- start 2024-05-27T07:57:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.182+0000 7fd98b408740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d4e2f0 0x561f87123ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.182+0000 7fd98b408740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f87124420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd98b408740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d771b0 0x561f8712fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd98b408740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561f86d60790 con 0x561f86d4e2f0 2024-05-27T07:57:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd98b408740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f87022bc0 con 0x561f86d564d0 2024-05-27T07:57:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd98b408740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561f87126b90 con 0x561f86d771b0 2024-05-27T07:57:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd98b408740 10 client.0 _ll_register_callbacks cb 0x561f87000400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd981a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d4e2f0 0x561f87123ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd981a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d4e2f0 0x561f87123ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52836/0 (socket says 172.21.15.63:52836) 2024-05-27T07:57:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd981a00640 1 -- 172.21.15.63:0/1427354421 learned_addr learned my addr 172.21.15.63:0/1427354421 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd981000640 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f87124420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd981a00640 1 -- 172.21.15.63:0/1427354421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d771b0 msgr2=0x561f8712fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd981a00640 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d771b0 0x561f8712fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd981a00640 1 -- 172.21.15.63:0/1427354421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 msgr2=0x561f87124420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd981a00640 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f87124420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.183+0000 7fd981a00640 1 -- 172.21.15.63:0/1427354421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f86d8f210 con 0x561f86d4e2f0 2024-05-27T07:57:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.184+0000 7fd981a00640 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d4e2f0 0x561f87123ee0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b990 tx=0x7fd97000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafb485982ac798a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.184+0000 7fd963400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.184+0000 7fd963400640 1 -- 172.21.15.63:0/1427354421 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970003e60 con 0x561f86d4e2f0 2024-05-27T07:57:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.184+0000 7fd98b408740 1 -- 172.21.15.63:0/1427354421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9740113e0 con 0x561f86d4e2f0 2024-05-27T07:57:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.184+0000 7fd963400640 1 -- 172.21.15.63:0/1427354421 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7fd970005020 con 0x561f86d4e2f0 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd963400640 1 -- 172.21.15.63:0/1427354421 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd970024050 con 0x561f86d4e2f0 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd98b408740 10 client.20040 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd98b408740 1 -- 172.21.15.63:0/1427354421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561f86e5b620 con 0x561f86d4e2f0 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd963400640 1 -- 172.21.15.63:0/1427354421 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9740113e0 con 0x561f86d4e2f0 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd963400640 1 -- 172.21.15.63:0/1427354421 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd97001b470 con 0x561f86d4e2f0 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd98b408740 10 client.20040 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd98b408740 10 client.20040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd98b408740 20 client.20040 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd98b408740 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f86d35d80 0x561f87136000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd98b408740 4 client.20040 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.185+0000 7fd98b408740 1 -- 172.21.15.63:0/1427354421 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f8708cae0 con 0x561f86d35d80 2024-05-27T07:57:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.186+0000 7fd981000640 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f86d35d80 0x561f87136000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.187+0000 7fd981000640 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f86d35d80 0x561f87136000 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a747c01e1218d6d server_cookie=2236c60f6b2f93ba in_seq=0 out_seq=0 2024-05-27T07:57:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.187+0000 7fd963400640 10 client.20040 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.189+0000 7fd963400640 1 -- 172.21.15.63:0/1427354421 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561f8708cae0 con 0x561f86d35d80 2024-05-27T07:57:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.189+0000 7fd963400640 10 client.20040 handle_command_reply: tid=0 2024-05-27T07:57:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.189+0000 7fd98b408740 1 client.20040 shutdown 2024-05-27T07:57:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 1 -- 172.21.15.63:0/1427354421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d4e2f0 msgr2=0x561f87123ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d4e2f0 0x561f87123ee0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b990 tx=0x7fd97000be50 comp rx=0 tx=0).stop 2024-05-27T07:57:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 1 -- 172.21.15.63:0/1427354421 shutdown_connections 2024-05-27T07:57:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f86d35d80 0x561f87136000 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d771b0 0x561f8712fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f87124420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 1 --2- 172.21.15.63:0/1427354421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d4e2f0 0x561f87123ee0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 1 -- 172.21.15.63:0/1427354421 >> 172.21.15.63:0/1427354421 conn(0x561f86d6b300 msgr2=0x561f87119890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd963400640 0 client.20040 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 1 -- 172.21.15.63:0/1427354421 shutdown_connections 2024-05-27T07:57:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 1 -- 172.21.15.63:0/1427354421 wait complete. 2024-05-27T07:57:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.190+0000 7fd98b408740 20 client.20040 trim_cache size 0 max 16384 2024-05-27T07:57:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.212+0000 7fd98b408740 1 Processor -- start 2024-05-27T07:57:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.212+0000 7fd98b408740 1 -- start start 2024-05-27T07:57:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.212+0000 7fd98b408740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 0x561f8711f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.212+0000 7fd98b408740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f87123c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.212+0000 7fd98b408740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f871241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.212+0000 7fd98b408740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c1b5b30 con 0x561f86d771b0 2024-05-27T07:57:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.212+0000 7fd98b408740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c1b3400 con 0x561f86d564d0 2024-05-27T07:57:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.212+0000 7fd98b408740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c1b1270 con 0x561f86d4e2f0 2024-05-27T07:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd982400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f871241b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd982400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f871241b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52846/0 (socket says 172.21.15.63:52846) 2024-05-27T07:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd982400640 1 -- 172.21.15.63:0/1358451284 learned_addr learned my addr 172.21.15.63:0/1358451284 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd981000640 1 --2- 172.21.15.63:0/1358451284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f87123c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd981a00640 1 --2- 172.21.15.63:0/1358451284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 0x561f8711f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd982400640 1 -- 172.21.15.63:0/1358451284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 msgr2=0x561f8711f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd982400640 1 --2- 172.21.15.63:0/1358451284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 0x561f8711f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd982400640 1 -- 172.21.15.63:0/1358451284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 msgr2=0x561f87123c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd982400640 1 --2- 172.21.15.63:0/1358451284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f87123c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd982400640 1 -- 172.21.15.63:0/1358451284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f871193f0 con 0x561f86d771b0 2024-05-27T07:57:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.213+0000 7fd981a00640 1 --2- 172.21.15.63:0/1358451284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 0x561f8711f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.214+0000 7fd982400640 1 --2- 172.21.15.63:0/1358451284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f871241b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fd974011010 tx=0x7fd974009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa0bc77e4e94b60 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.214+0000 7fd96e000640 1 -- 172.21.15.63:0/1358451284 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974016030 con 0x561f86d771b0 2024-05-27T07:57:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.214+0000 7fd96e000640 1 -- 172.21.15.63:0/1358451284 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd974003970 con 0x561f86d771b0 2024-05-27T07:57:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.214+0000 7fd96e000640 1 -- 172.21.15.63:0/1358451284 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9740042a0 con 0x561f86d771b0 2024-05-27T07:57:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.215+0000 7fd98b408740 1 -- 172.21.15.63:0/1358451284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 msgr2=0x561f871241b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.215+0000 7fd98b408740 1 --2- 172.21.15.63:0/1358451284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f871241b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fd974011010 tx=0x7fd974009670 comp rx=0 tx=0).stop 2024-05-27T07:57:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.215+0000 7fd98b408740 1 -- 172.21.15.63:0/1358451284 shutdown_connections 2024-05-27T07:57:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.215+0000 7fd98b408740 1 --2- 172.21.15.63:0/1358451284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d771b0 0x561f871241b0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.215+0000 7fd98b408740 1 --2- 172.21.15.63:0/1358451284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d564d0 0x561f87123c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.215+0000 7fd98b408740 1 --2- 172.21.15.63:0/1358451284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d4e2f0 0x561f8711f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.215+0000 7fd98b408740 1 -- 172.21.15.63:0/1358451284 >> 172.21.15.63:0/1358451284 conn(0x561f86d6b300 msgr2=0x561f87123560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.215+0000 7fd98b408740 1 -- 172.21.15.63:0/1358451284 shutdown_connections 2024-05-27T07:57:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.215+0000 7fd98b408740 1 -- 172.21.15.63:0/1358451284 wait complete. 2024-05-27T07:57:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.216+0000 7fd98b408740 1 -- start start 2024-05-27T07:57:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.216+0000 7fd98b408740 1 Processor -- start 2024-05-27T07:57:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.217+0000 7fd98b408740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d4e2f0 0x561f871293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.217+0000 7fd98b408740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d564d0 0x561f87129900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.217+0000 7fd98b408740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d771b0 0x561f871966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.217+0000 7fd98b408740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c1b18c0 con 0x561f86d564d0 2024-05-27T07:57:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.217+0000 7fd98b408740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c1b2d30 con 0x561f86d4e2f0 2024-05-27T07:57:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.217+0000 7fd98b408740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c1b1f90 con 0x561f86d771b0 2024-05-27T07:57:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.217+0000 7fd98b408740 10 client.0 _ll_register_callbacks cb 0x7fd9700042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.217+0000 7fd981000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d564d0 0x561f87129900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.217+0000 7fd981000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d564d0 0x561f87129900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52850/0 (socket says 172.21.15.63:52850) 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.217+0000 7fd981000640 1 -- 172.21.15.63:0/666165328 learned_addr learned my addr 172.21.15.63:0/666165328 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.218+0000 7fd981a00640 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d4e2f0 0x561f871293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.218+0000 7fd982400640 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d771b0 0x561f871966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.218+0000 7fd981000640 1 -- 172.21.15.63:0/666165328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d771b0 msgr2=0x561f871966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.218+0000 7fd981000640 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d771b0 0x561f871966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.218+0000 7fd981000640 1 -- 172.21.15.63:0/666165328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d4e2f0 msgr2=0x561f871293c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.218+0000 7fd981000640 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d4e2f0 0x561f871293c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.218+0000 7fd981000640 1 -- 172.21.15.63:0/666165328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f87196be0 con 0x561f86d564d0 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.218+0000 7fd981000640 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d564d0 0x561f87129900 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fd9680c9bd0 tx=0x7fd96802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a557fdffd284c11 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.218+0000 7fd963400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.219+0000 7fd963400640 1 -- 172.21.15.63:0/666165328 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968049040 con 0x561f86d564d0 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.219+0000 7fd963400640 1 -- 172.21.15.63:0/666165328 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7fd9680035f0 con 0x561f86d564d0 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.219+0000 7fd98b408740 1 -- 172.21.15.63:0/666165328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd974003970 con 0x561f86d564d0 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.219+0000 7fd963400640 1 -- 172.21.15.63:0/666165328 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd9680ce0e0 con 0x561f86d564d0 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.219+0000 7fd98b408740 10 client.20046 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.219+0000 7fd98b408740 1 -- 172.21.15.63:0/666165328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561f8719df20 con 0x561f86d564d0 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.219+0000 7fd963400640 1 -- 172.21.15.63:0/666165328 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9680e0070 con 0x561f86d564d0 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.220+0000 7fd963400640 1 -- 172.21.15.63:0/666165328 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd9681004b0 con 0x561f86d564d0 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.220+0000 7fd98b408740 10 client.20046 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.220+0000 7fd98b408740 10 client.20046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.220+0000 7fd98b408740 20 client.20046 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.220+0000 7fd98b408740 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f86d0b1c0 0x561f871a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.220+0000 7fd98b408740 4 client.20046 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:57:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.220+0000 7fd98b408740 1 -- 172.21.15.63:0/666165328 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f870862c0 con 0x561f86d0b1c0 2024-05-27T07:57:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.220+0000 7fd981a00640 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f86d0b1c0 0x561f871a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.222+0000 7fd981a00640 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f86d0b1c0 0x561f871a06e0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b66cbe946447e9 server_cookie=c2b067570be2bc57 in_seq=0 out_seq=0 2024-05-27T07:57:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.222+0000 7fd963400640 10 client.20046 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.256+0000 7fd963400640 1 -- 172.21.15.63:0/666165328 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561f870862c0 con 0x561f86d0b1c0 2024-05-27T07:57:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.256+0000 7fd963400640 10 client.20046 handle_command_reply: tid=0 2024-05-27T07:57:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.256+0000 7fd98b408740 1 client.20046 shutdown 2024-05-27T07:57:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.256+0000 7fd98b408740 1 -- 172.21.15.63:0/666165328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d564d0 msgr2=0x561f87129900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.256+0000 7fd98b408740 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d564d0 0x561f87129900 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fd9680c9bd0 tx=0x7fd96802a480 comp rx=0 tx=0).stop 2024-05-27T07:57:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.257+0000 7fd98b408740 1 -- 172.21.15.63:0/666165328 shutdown_connections 2024-05-27T07:57:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.257+0000 7fd98b408740 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561f86d0b1c0 0x561f871a06e0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.257+0000 7fd98b408740 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f86d771b0 0x561f871966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.257+0000 7fd98b408740 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f86d564d0 0x561f87129900 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.257+0000 7fd98b408740 1 --2- 172.21.15.63:0/666165328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f86d4e2f0 0x561f871293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.257+0000 7fd98b408740 1 -- 172.21.15.63:0/666165328 >> 172.21.15.63:0/666165328 conn(0x561f86d6b300 msgr2=0x561f87123f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.257+0000 7fd963400640 0 client.20046 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.257+0000 7fd98b408740 1 -- 172.21.15.63:0/666165328 shutdown_connections 2024-05-27T07:57:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.257+0000 7fd98b408740 1 -- 172.21.15.63:0/666165328 wait complete. 2024-05-27T07:57:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.257+0000 7fd98b408740 20 client.20046 trim_cache size 0 max 16384 2024-05-27T07:57:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:57:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:57:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:57:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.258+0000 7fd982e00640 1 -- 172.21.15.63:0/2968996202 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9640b29d0 msgr2=0x7fd9640b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.258+0000 7fd982e00640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9640b29d0 0x7fd9640b4ee0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd97401f000 tx=0x7fd974022000 comp rx=0 tx=0).stop 2024-05-27T07:57:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.258+0000 7fd982e00640 1 -- 172.21.15.63:0/2968996202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd97c10b6e0 msgr2=0x7fd97c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.258+0000 7fd982e00640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd97c10b6e0 0x7fd97c19cd30 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fd968009230 tx=0x7fd968004270 comp rx=0 tx=0).stop 2024-05-27T07:57:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.258+0000 7fd982e00640 1 -- 172.21.15.63:0/2968996202 shutdown_connections 2024-05-27T07:57:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.258+0000 7fd982e00640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9640b29d0 0x7fd9640b4ee0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.258+0000 7fd982e00640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd97c10e040 0x7fd97c1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.258+0000 7fd982e00640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd97c10b6e0 0x7fd97c19cd30 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.258+0000 7fd982e00640 1 --2- 172.21.15.63:0/2968996202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd97c104c80 0x7fd97c19c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.258+0000 7fd982e00640 1 -- 172.21.15.63:0/2968996202 >> 172.21.15.63:0/2968996202 conn(0x7fd97c0790c0 msgr2=0x7fd97c10a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.259+0000 7fd982e00640 1 -- 172.21.15.63:0/2968996202 shutdown_connections 2024-05-27T07:57:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:46.259+0000 7fd982e00640 1 -- 172.21.15.63:0/2968996202 wait complete. 2024-05-27T07:57:46.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.562+0000 7f3eb0e00700 1 -- 172.21.15.63:0/2687967394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3eac10a3d0 msgr2=0x7f3eac10a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.562+0000 7f3eb0e00700 1 --2- 172.21.15.63:0/2687967394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3eac10a3d0 0x7f3eac10a7e0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009990 tx=0x7f3e9c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.562+0000 7f3eb0e00700 1 -- 172.21.15.63:0/2687967394 shutdown_connections 2024-05-27T07:57:46.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.562+0000 7f3eb0e00700 1 --2- 172.21.15.63:0/2687967394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eac103120 0x7f3eac111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.562+0000 7f3eb0e00700 1 --2- 172.21.15.63:0/2687967394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3eac10a3d0 0x7f3eac10a7e0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.562+0000 7f3eb0e00700 1 --2- 172.21.15.63:0/2687967394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3eac1027d0 0x7f3eac102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.562+0000 7f3eb0e00700 1 -- 172.21.15.63:0/2687967394 >> 172.21.15.63:0/2687967394 conn(0x7f3eac0fe6e0 msgr2=0x7f3eac100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.562+0000 7f3eb0e00700 1 -- 172.21.15.63:0/2687967394 shutdown_connections 2024-05-27T07:57:46.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.562+0000 7f3eb0e00700 1 -- 172.21.15.63:0/2687967394 wait complete. 2024-05-27T07:57:46.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.563+0000 7f3eb0e00700 1 Processor -- start 2024-05-27T07:57:46.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.564+0000 7f3eb0e00700 1 -- start start 2024-05-27T07:57:46.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.564+0000 7f3eb0e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eac1027d0 0x7f3eac19d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.564+0000 7f3eb0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3eac103120 0x7f3eac19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.564+0000 7f3eb0e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3eac10a3d0 0x7f3eac1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.564+0000 7f3eb0e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac114370 con 0x7f3eac103120 2024-05-27T07:57:46.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.564+0000 7f3eb0e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac1141f0 con 0x7f3eac1027d0 2024-05-27T07:57:46.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.564+0000 7f3eb0e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac1144f0 con 0x7f3eac10a3d0 2024-05-27T07:57:46.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.564+0000 7f3ea9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3eac103120 0x7f3eac19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.565+0000 7f3ea9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3eac103120 0x7f3eac19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52878/0 (socket says 172.21.15.63:52878) 2024-05-27T07:57:46.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.565+0000 7f3ea9400700 1 -- 172.21.15.63:0/1640375239 learned_addr learned my addr 172.21.15.63:0/1640375239 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:46.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.565+0000 7f3ea9e00700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eac1027d0 0x7f3eac19d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.565+0000 7f3eaa800700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3eac10a3d0 0x7f3eac1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.565+0000 7f3ea9400700 1 -- 172.21.15.63:0/1640375239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3eac10a3d0 msgr2=0x7f3eac1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.565+0000 7f3ea9400700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3eac10a3d0 0x7f3eac1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.565+0000 7f3ea9400700 1 -- 172.21.15.63:0/1640375239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eac1027d0 msgr2=0x7f3eac19d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.565+0000 7f3ea9400700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eac1027d0 0x7f3eac19d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.565+0000 7f3ea9400700 1 -- 172.21.15.63:0/1640375239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eac1a1f70 con 0x7f3eac103120 2024-05-27T07:57:46.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.566+0000 7f3ea9400700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3eac103120 0x7f3eac19d640 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800d980 tx=0x7f3e9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.566+0000 7f3ea2a00700 1 -- 172.21.15.63:0/1640375239 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98017070 con 0x7f3eac103120 2024-05-27T07:57:46.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.566+0000 7f3ea2a00700 1 -- 172.21.15.63:0/1640375239 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e98009dc0 con 0x7f3eac103120 2024-05-27T07:57:46.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.566+0000 7f3eb0e00700 1 -- 172.21.15.63:0/1640375239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eac1a2260 con 0x7f3eac103120 2024-05-27T07:57:46.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.566+0000 7f3ea2a00700 1 -- 172.21.15.63:0/1640375239 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800f3c0 con 0x7f3eac103120 2024-05-27T07:57:46.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.566+0000 7f3eb0e00700 1 -- 172.21.15.63:0/1640375239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eac1a9b80 con 0x7f3eac103120 2024-05-27T07:57:46.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.572+0000 7f3ebf89eb80 1 Processor -- start 2024-05-27T07:57:46.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.572+0000 7f3ebf89eb80 1 -- start start 2024-05-27T07:57:46.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.573+0000 7f3ea2a00700 1 -- 172.21.15.63:0/1640375239 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3e98009930 con 0x7f3eac103120 2024-05-27T07:57:46.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.573+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c3e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.573+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c38870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.573+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.573+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d6d3b66e40 con 0x55d6d3c38db0 2024-05-27T07:57:46.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.573+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d6d3b569e0 con 0x55d6d3c3ed30 2024-05-27T07:57:46.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.573+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d6d3b5aa20 con 0x55d6d3ab9e40 2024-05-27T07:57:46.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.573+0000 7f3ea9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.574+0000 7f3ea9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c38870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.574+0000 7f3ea9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c391c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52892/0 (socket says 172.21.15.63:52892) 2024-05-27T07:57:46.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.574+0000 7f3ea9400700 1 -- 172.21.15.63:0/4186849902 learned_addr learned my addr 172.21.15.63:0/4186849902 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:46.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.574+0000 7f3eaa800700 1 --2- 172.21.15.63:0/4186849902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c3e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.574+0000 7f3ea9400700 1 -- 172.21.15.63:0/4186849902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3ab9e40 msgr2=0x55d6d3c3e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.574+0000 7f3ea9400700 1 --2- 172.21.15.63:0/4186849902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c3e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.574+0000 7f3ea9400700 1 -- 172.21.15.63:0/4186849902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3c3ed30 msgr2=0x55d6d3c38870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.574+0000 7f3ea9400700 1 --2- 172.21.15.63:0/4186849902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c38870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.574+0000 7f3ea9400700 1 -- 172.21.15.63:0/4186849902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6d3b1f420 con 0x55d6d3c38db0 2024-05-27T07:57:46.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.575+0000 7f3ea9400700 1 --2- 172.21.15.63:0/4186849902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c391c0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f3e980df030 tx=0x7f3e980e8710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f519db5d168916 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.575+0000 7f3ea2a00700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3e900b2b40 0x7f3e900b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.575+0000 7f3e97400700 1 -- 172.21.15.63:0/4186849902 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980a8470 con 0x55d6d3c38db0 2024-05-27T07:57:46.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.575+0000 7f3ea2a00700 1 -- 172.21.15.63:0/1640375239 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f3e980e1dc0 con 0x7f3eac103120 2024-05-27T07:57:46.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.576+0000 7f3e97400700 1 -- 172.21.15.63:0/4186849902 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e980a9400 con 0x55d6d3c38db0 2024-05-27T07:57:46.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.576+0000 7f3eaa800700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3e900b2b40 0x7f3e900b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.576+0000 7f3e97400700 1 -- 172.21.15.63:0/4186849902 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980a8650 con 0x55d6d3c38db0 2024-05-27T07:57:46.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.577+0000 7f3eaa800700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3e900b2b40 0x7f3e900b5050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55d6d3c39b30 tx=0x7f3ea4009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.577+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/4186849902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 msgr2=0x55d6d3c391c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.577+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/4186849902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c391c0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f3e980df030 tx=0x7f3e980e8710 comp rx=0 tx=0).stop 2024-05-27T07:57:46.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.577+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/4186849902 shutdown_connections 2024-05-27T07:57:46.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.577+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/4186849902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c391c0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.577+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/4186849902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c3e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.577+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/4186849902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c38870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.577+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/4186849902 >> 172.21.15.63:0/4186849902 conn(0x55d6d3b19fc0 msgr2=0x55d6d3c338d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.577+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/4186849902 shutdown_connections 2024-05-27T07:57:46.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.577+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/4186849902 wait complete. 2024-05-27T07:57:46.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.578+0000 7f3ebf89eb80 1 -- start start 2024-05-27T07:57:46.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.578+0000 7f3ebf89eb80 1 Processor -- start 2024-05-27T07:57:46.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.579+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c42540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.579+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c38db0 0x55d6d3c42a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.579+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c42fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.579+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d6d3a5cfa0 con 0x55d6d3ab9e40 2024-05-27T07:57:46.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.579+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d6d3af7760 con 0x55d6d3c3ed30 2024-05-27T07:57:46.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.579+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d6d3c45800 con 0x55d6d3c38db0 2024-05-27T07:57:46.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.579+0000 7f3ebf89eb80 10 client.0 _ll_register_callbacks cb 0x55d6d3b00c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:46.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.580+0000 7f3ea9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c42540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.580+0000 7f3ea9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c42540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52902/0 (socket says 172.21.15.63:52902) 2024-05-27T07:57:46.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.580+0000 7f3ea9e00700 1 -- 172.21.15.63:0/3771932351 learned_addr learned my addr 172.21.15.63:0/3771932351 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:46.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.580+0000 7f3ea9400700 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c38db0 0x55d6d3c42a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.580+0000 7f3eaa800700 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c42fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.580+0000 7f3ea9e00700 1 -- 172.21.15.63:0/3771932351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c38db0 msgr2=0x55d6d3c42a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.580+0000 7f3ea9e00700 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c38db0 0x55d6d3c42a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.580+0000 7f3ea9e00700 1 -- 172.21.15.63:0/3771932351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3c3ed30 msgr2=0x55d6d3c42fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.580+0000 7f3ea9e00700 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c42fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.580+0000 7f3ea9e00700 1 -- 172.21.15.63:0/3771932351 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6d3aa68d0 con 0x55d6d3ab9e40 2024-05-27T07:57:46.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.581+0000 7f3ea9e00700 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c42540 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c005810 tx=0x7f3e9c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5a537308d0fe2c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.581+0000 7f3e8be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:46.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.582+0000 7f3e8be00700 1 -- 172.21.15.63:0/3771932351 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c003bd0 con 0x55d6d3ab9e40 2024-05-27T07:57:46.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.582+0000 7f3e8be00700 1 -- 172.21.15.63:0/3771932351 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f3e9c003d70 con 0x55d6d3ab9e40 2024-05-27T07:57:46.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.582+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/3771932351 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e980a9400 con 0x55d6d3ab9e40 2024-05-27T07:57:46.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.582+0000 7f3e8be00700 1 -- 172.21.15.63:0/3771932351 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e9c00b7d0 con 0x55d6d3ab9e40 2024-05-27T07:57:46.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.582+0000 7f3ebf89eb80 10 client.20058 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:46.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.582+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/3771932351 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d6d3b27060 con 0x55d6d3ab9e40 2024-05-27T07:57:46.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.583+0000 7f3e8be00700 1 -- 172.21.15.63:0/3771932351 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e980a9400 con 0x55d6d3ab9e40 2024-05-27T07:57:46.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.583+0000 7f3e8be00700 1 -- 172.21.15.63:0/3771932351 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3e9c023970 con 0x55d6d3ab9e40 2024-05-27T07:57:46.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.583+0000 7f3ebf89eb80 10 client.20058 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:46.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.583+0000 7f3ebf89eb80 10 client.20058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:46.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.583+0000 7f3ebf89eb80 20 client.20058 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:46.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.583+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d6d3c52790 0x55d6d3c54bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.583+0000 7f3ebf89eb80 4 client.20058 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:46.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.583+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/3771932351 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6d3b68d90 con 0x55d6d3c52790 2024-05-27T07:57:46.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.584+0000 7f3ea9400700 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d6d3c52790 0x55d6d3c54bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.585+0000 7f3ea9400700 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d6d3c52790 0x55d6d3c54bd0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740b0f07b3e52f97 server_cookie=2d9199ceadcc14f in_seq=0 out_seq=0 2024-05-27T07:57:46.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.585+0000 7f3e8be00700 10 client.20058 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:46.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.587+0000 7f3e8be00700 1 -- 172.21.15.63:0/3771932351 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d6d3b68d90 con 0x55d6d3c52790 2024-05-27T07:57:46.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.587+0000 7f3e8be00700 10 client.20058 handle_command_reply: tid=0 2024-05-27T07:57:46.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.587+0000 7f3ebf89eb80 1 client.20058 shutdown 2024-05-27T07:57:46.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.587+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/3771932351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3ab9e40 msgr2=0x55d6d3c42540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.587+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c42540 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c005810 tx=0x7f3e9c009ee0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.587+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/3771932351 shutdown_connections 2024-05-27T07:57:46.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.587+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d6d3c52790 0x55d6d3c54bd0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.587+0000 7f3e8be00700 0 client.20058 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:46.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.587+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c42fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.588+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c42540 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.588+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/3771932351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c38db0 0x55d6d3c42a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.588+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/3771932351 >> 172.21.15.63:0/3771932351 conn(0x55d6d3b19fc0 msgr2=0x55d6d3c33580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.588+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/3771932351 shutdown_connections 2024-05-27T07:57:46.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.588+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/3771932351 wait complete. 2024-05-27T07:57:46.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.588+0000 7f3ebf89eb80 20 client.20058 trim_cache size 0 max 16384 2024-05-27T07:57:46.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.622+0000 7f3ebf89eb80 1 Processor -- start 2024-05-27T07:57:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.622+0000 7f3ebf89eb80 1 -- start start 2024-05-27T07:57:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.622+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c38870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.622+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c38db0 0x55d6d3c3cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.622+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.622+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac1b4f60 con 0x55d6d3c3ed30 2024-05-27T07:57:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.622+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac1b27f0 con 0x55d6d3ab9e40 2024-05-27T07:57:46.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.622+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac1b0660 con 0x55d6d3c38db0 2024-05-27T07:57:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.623+0000 7f3eaa800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.623+0000 7f3eaa800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52916/0 (socket says 172.21.15.63:52916) 2024-05-27T07:57:46.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.623+0000 7f3eaa800700 1 -- 172.21.15.63:0/2380288487 learned_addr learned my addr 172.21.15.63:0/2380288487 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.623+0000 7f3ea9e00700 1 --2- 172.21.15.63:0/2380288487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c38870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.623+0000 7f3ea9400700 1 --2- 172.21.15.63:0/2380288487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c38db0 0x55d6d3c3cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.623+0000 7f3eaa800700 1 -- 172.21.15.63:0/2380288487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c38db0 msgr2=0x55d6d3c3cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.623+0000 7f3eaa800700 1 --2- 172.21.15.63:0/2380288487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c38db0 0x55d6d3c3cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.623+0000 7f3eaa800700 1 -- 172.21.15.63:0/2380288487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 msgr2=0x55d6d3c38870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.623+0000 7f3eaa800700 1 --2- 172.21.15.63:0/2380288487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c38870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.624+0000 7f3eaa800700 1 -- 172.21.15.63:0/2380288487 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6d3c3e1c0 con 0x55d6d3c3ed30 2024-05-27T07:57:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.624+0000 7f3eaa800700 1 --2- 172.21.15.63:0/2380288487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c448a0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4018570 tx=0x7f3ea4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b8eb45c7dbede4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.625+0000 7f3e96a00700 1 -- 172.21.15.63:0/2380288487 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea401e030 con 0x55d6d3c3ed30 2024-05-27T07:57:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.625+0000 7f3e96a00700 1 -- 172.21.15.63:0/2380288487 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea4029070 con 0x55d6d3c3ed30 2024-05-27T07:57:46.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.625+0000 7f3e96a00700 1 -- 172.21.15.63:0/2380288487 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4016680 con 0x55d6d3c3ed30 2024-05-27T07:57:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.626+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/2380288487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c3ed30 msgr2=0x55d6d3c448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.626+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/2380288487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c448a0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4018570 tx=0x7f3ea4015000 comp rx=0 tx=0).stop 2024-05-27T07:57:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.626+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/2380288487 shutdown_connections 2024-05-27T07:57:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.626+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/2380288487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c448a0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.626+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/2380288487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c38870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.626+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/2380288487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c38db0 0x55d6d3c3cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.626+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/2380288487 >> 172.21.15.63:0/2380288487 conn(0x55d6d3b19fc0 msgr2=0x55d6d3c42320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.626+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/2380288487 shutdown_connections 2024-05-27T07:57:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.627+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/2380288487 wait complete. 2024-05-27T07:57:46.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.627+0000 7f3ebf89eb80 1 -- start start 2024-05-27T07:57:46.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.628+0000 7f3ebf89eb80 1 Processor -- start 2024-05-27T07:57:46.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.629+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c59fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.629+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c5a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.629+0000 7f3ebf89eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c5aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.629+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac1b0cb0 con 0x55d6d3c38db0 2024-05-27T07:57:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.629+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac1b2120 con 0x55d6d3ab9e40 2024-05-27T07:57:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.629+0000 7f3ebf89eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac1b1380 con 0x55d6d3c3ed30 2024-05-27T07:57:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.629+0000 7f3ebf89eb80 10 client.0 _ll_register_callbacks cb 0x55d6d3b00c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.630+0000 7f3ea9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c5a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.630+0000 7f3ea9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c59fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.630+0000 7f3ea9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c5a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52922/0 (socket says 172.21.15.63:52922) 2024-05-27T07:57:46.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.630+0000 7f3ea9400700 1 -- 172.21.15.63:0/1723749328 learned_addr learned my addr 172.21.15.63:0/1723749328 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:46.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.630+0000 7f3eaa800700 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c5aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.630+0000 7f3ea9400700 1 -- 172.21.15.63:0/1723749328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c3ed30 msgr2=0x55d6d3c5aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.630+0000 7f3ea9400700 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c5aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.631+0000 7f3ea9400700 1 -- 172.21.15.63:0/1723749328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 msgr2=0x55d6d3c59fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.631+0000 7f3ea9400700 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c59fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.631+0000 7f3ea9400700 1 -- 172.21.15.63:0/1723749328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6d3cbc2d0 con 0x55d6d3c38db0 2024-05-27T07:57:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.631+0000 7f3ea9e00700 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c59fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.631+0000 7f3ea9400700 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c5a4e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f3e980a8a00 tx=0x7f3e98005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90abddd9c4370ef5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.631+0000 7f3e8be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.632+0000 7f3e8be00700 1 -- 172.21.15.63:0/1723749328 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98020070 con 0x55d6d3c38db0 2024-05-27T07:57:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.632+0000 7f3e8be00700 1 -- 172.21.15.63:0/1723749328 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f3e980c9e00 con 0x55d6d3c38db0 2024-05-27T07:57:46.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.632+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/1723749328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea4029070 con 0x55d6d3c38db0 2024-05-27T07:57:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.632+0000 7f3e8be00700 1 -- 172.21.15.63:0/1723749328 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3e980a8840 con 0x55d6d3c38db0 2024-05-27T07:57:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.632+0000 7f3ebf89eb80 10 client.20064 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.632+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/1723749328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d6d3cc5d30 con 0x55d6d3c38db0 2024-05-27T07:57:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.633+0000 7f3e8be00700 1 -- 172.21.15.63:0/1723749328 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea4029070 con 0x55d6d3c38db0 2024-05-27T07:57:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.633+0000 7f3e8be00700 1 -- 172.21.15.63:0/1723749328 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f3e980a4f90 con 0x55d6d3c38db0 2024-05-27T07:57:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.633+0000 7f3ebf89eb80 10 client.20064 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:46.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.633+0000 7f3ebf89eb80 10 client.20064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:46.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.633+0000 7f3ebf89eb80 20 client.20064 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:46.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.633+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d6d3c52790 0x55d6d3cc7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.633+0000 7f3ebf89eb80 4 client.20064 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:57:46.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.633+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/1723749328 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d6d3b62570 con 0x55d6d3c52790 2024-05-27T07:57:46.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.634+0000 7f3ea9e00700 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d6d3c52790 0x55d6d3cc7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.636+0000 7f3e8be00700 10 client.20064 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:46.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.636+0000 7f3ea9e00700 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d6d3c52790 0x55d6d3cc7f70 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc92b01192bb548 server_cookie=789e1b8245c97f31 in_seq=0 out_seq=0 2024-05-27T07:57:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.636+0000 7f3e8be00700 1 -- 172.21.15.63:0/1723749328 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d6d3b62570 con 0x55d6d3c52790 2024-05-27T07:57:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.636+0000 7f3e8be00700 10 client.20064 handle_command_reply: tid=0 2024-05-27T07:57:46.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.636+0000 7f3ebf89eb80 1 client.20064 shutdown 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.636+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/1723749328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 msgr2=0x55d6d3c5a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.636+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c5a4e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f3e980a8a00 tx=0x7f3e98005dc0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.637+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/1723749328 shutdown_connections 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.637+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d6d3c52790 0x55d6d3cc7f70 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.637+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d6d3c3ed30 0x55d6d3c5aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.637+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d6d3ab9e40 0x55d6d3c59fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.637+0000 7f3ebf89eb80 1 --2- 172.21.15.63:0/1723749328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d6d3c38db0 0x55d6d3c5a4e0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.637+0000 7f3e8be00700 0 client.20064 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.637+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/1723749328 >> 172.21.15.63:0/1723749328 conn(0x55d6d3b19fc0 msgr2=0x55d6d3c3c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.637+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/1723749328 shutdown_connections 2024-05-27T07:57:46.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.637+0000 7f3ebf89eb80 1 -- 172.21.15.63:0/1723749328 wait complete. 2024-05-27T07:57:46.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.637+0000 7f3ebf89eb80 20 client.20064 trim_cache size 0 max 16384 2024-05-27T07:57:46.639 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:57:46.639 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:57:46.639 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:57:46.640 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:57:46.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 -- 172.21.15.63:0/1640375239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3e900b2b40 msgr2=0x7f3e900b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3e900b2b40 0x7f3e900b5050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55d6d3c39b30 tx=0x7f3ea4009040 comp rx=0 tx=0).stop 2024-05-27T07:57:46.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 -- 172.21.15.63:0/1640375239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3eac103120 msgr2=0x7f3eac19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3eac103120 0x7f3eac19d640 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800d980 tx=0x7f3e9800dc90 comp rx=0 tx=0).stop 2024-05-27T07:57:46.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 -- 172.21.15.63:0/1640375239 shutdown_connections 2024-05-27T07:57:46.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3e900b2b40 0x7f3e900b5050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3eac10a3d0 0x7f3eac1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eac1027d0 0x7f3eac19d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 --2- 172.21.15.63:0/1640375239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3eac103120 0x7f3eac19d640 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 -- 172.21.15.63:0/1640375239 >> 172.21.15.63:0/1640375239 conn(0x7f3eac0fe6e0 msgr2=0x7f3eac10f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 -- 172.21.15.63:0/1640375239 shutdown_connections 2024-05-27T07:57:46.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:46.639+0000 7f3eb0e00700 1 -- 172.21.15.63:0/1640375239 wait complete. 2024-05-27T07:57:47.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:47 smithi063 ceph-mon[31613]: pgmap v765: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:57:47.643 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:57:47.644 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:57:47.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:47 smithi086 ceph-mon[34687]: pgmap v765: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:57:47.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:47 smithi149 ceph-mon[35408]: pgmap v765: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:57:48.171 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:49.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:49 smithi086 ceph-mon[34687]: pgmap v766: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:57:49.712 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:49 smithi063 ceph-mon[31613]: pgmap v766: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:57:49.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:49 smithi149 ceph-mon[35408]: pgmap v766: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T07:57:50.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.739+0000 7fbc47200700 1 -- 172.21.15.63:0/1867121099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc48105280 msgr2=0x7fbc48105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:50.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.739+0000 7fbc47200700 1 --2- 172.21.15.63:0/1867121099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc48105280 0x7fbc48105690 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009990 tx=0x7fbc38009ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:50.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.739+0000 7fbc47200700 1 -- 172.21.15.63:0/1867121099 shutdown_connections 2024-05-27T07:57:50.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.739+0000 7fbc47200700 1 --2- 172.21.15.63:0/1867121099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4810e6f0 0x7fbc48110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:50.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.739+0000 7fbc47200700 1 --2- 172.21.15.63:0/1867121099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc48105280 0x7fbc48105690 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:50.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.739+0000 7fbc47200700 1 --2- 172.21.15.63:0/1867121099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48105bd0 0x7fbc4810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:50.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.739+0000 7fbc47200700 1 -- 172.21.15.63:0/1867121099 >> 172.21.15.63:0/1867121099 conn(0x7fbc48078ab0 msgr2=0x7fbc48078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:50.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.739+0000 7fbc47200700 1 -- 172.21.15.63:0/1867121099 shutdown_connections 2024-05-27T07:57:50.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.740+0000 7fbc47200700 1 -- 172.21.15.63:0/1867121099 wait complete. 2024-05-27T07:57:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.741+0000 7fbc47200700 1 Processor -- start 2024-05-27T07:57:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.741+0000 7fbc47200700 1 -- start start 2024-05-27T07:57:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.741+0000 7fbc47200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc48105280 0x7fbc4819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.742+0000 7fbc47200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48105bd0 0x7fbc4819d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.742+0000 7fbc47200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4810e6f0 0x7fbc481a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.742+0000 7fbc47200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481140f0 con 0x7fbc48105280 2024-05-27T07:57:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.742+0000 7fbc47200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48113f70 con 0x7fbc4810e6f0 2024-05-27T07:57:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.742+0000 7fbc47200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48114270 con 0x7fbc48105bd0 2024-05-27T07:57:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.742+0000 7fbc45e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc48105280 0x7fbc4819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.742+0000 7fbc45e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc48105280 0x7fbc4819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50302/0 (socket says 172.21.15.63:50302) 2024-05-27T07:57:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.742+0000 7fbc45e00700 1 -- 172.21.15.63:0/2940897225 learned_addr learned my addr 172.21.15.63:0/2940897225 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.742+0000 7fbc3fe00700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48105bd0 0x7fbc4819d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.742+0000 7fbc46800700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4810e6f0 0x7fbc481a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.743+0000 7fbc45e00700 1 -- 172.21.15.63:0/2940897225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48105bd0 msgr2=0x7fbc4819d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.743+0000 7fbc45e00700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48105bd0 0x7fbc4819d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.743+0000 7fbc45e00700 1 -- 172.21.15.63:0/2940897225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4810e6f0 msgr2=0x7fbc481a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.743+0000 7fbc45e00700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4810e6f0 0x7fbc481a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:50.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.743+0000 7fbc45e00700 1 -- 172.21.15.63:0/2940897225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc481a1c10 con 0x7fbc48105280 2024-05-27T07:57:50.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.743+0000 7fbc45e00700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc48105280 0x7fbc4819cd40 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fbc38000c00 tx=0x7fbc380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:50.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.743+0000 7fbc3ea00700 1 -- 172.21.15.63:0/2940897225 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38027070 con 0x7fbc48105280 2024-05-27T07:57:50.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.744+0000 7fbc3ea00700 1 -- 172.21.15.63:0/2940897225 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc3800fb00 con 0x7fbc48105280 2024-05-27T07:57:50.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.744+0000 7fbc47200700 1 -- 172.21.15.63:0/2940897225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc481a1ea0 con 0x7fbc48105280 2024-05-27T07:57:50.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.744+0000 7fbc3ea00700 1 -- 172.21.15.63:0/2940897225 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38022c40 con 0x7fbc48105280 2024-05-27T07:57:50.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.744+0000 7fbc47200700 1 -- 172.21.15.63:0/2940897225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc481a9820 con 0x7fbc48105280 2024-05-27T07:57:50.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.745+0000 7fbc47200700 1 -- 172.21.15.63:0/2940897225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc480512b0 con 0x7fbc48105280 2024-05-27T07:57:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.748+0000 7fbc3ea00700 1 -- 172.21.15.63:0/2940897225 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbc38022470 con 0x7fbc48105280 2024-05-27T07:57:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.750+0000 7fbc3ea00700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc2c0b2a70 0x7fbc2c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.750+0000 7fbc3ea00700 1 -- 172.21.15.63:0/2940897225 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7fbc380e3eb0 con 0x7fbc48105280 2024-05-27T07:57:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.751+0000 7fbc3fe00700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc2c0b2a70 0x7fbc2c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:50.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.752+0000 7fbc3fe00700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc2c0b2a70 0x7fbc2c0b4f80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc48102510 tx=0x7fbc3000b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:50.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:50.753+0000 7fbc3ea00700 1 -- 172.21.15.63:0/2940897225 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc38030070 con 0x7fbc48105280 2024-05-27T07:57:51.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.065+0000 7fbc47200700 1 -- 172.21.15.63:0/2940897225 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc4819e0a0 con 0x7fbc48105280 2024-05-27T07:57:51.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.066+0000 7fbc3ea00700 1 -- 172.21.15.63:0/2940897225 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11862 (secure 0 0 0) 0x7fbc38013070 con 0x7fbc48105280 2024-05-27T07:57:51.067 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:57:51.068 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:38:01.253677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_35654":{"gid":35654,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3714867941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3714867941},{"type":"v1","addr":"172.21.15.86:6833","nonce":3714867941}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34454,34484,34520]},"id":1}]} 2024-05-27T07:57:51.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 -- 172.21.15.63:0/2940897225 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc2c0b2a70 msgr2=0x7fbc2c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc2c0b2a70 0x7fbc2c0b4f80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc48102510 tx=0x7fbc3000b410 comp rx=0 tx=0).stop 2024-05-27T07:57:51.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 -- 172.21.15.63:0/2940897225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc48105280 msgr2=0x7fbc4819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc48105280 0x7fbc4819cd40 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fbc38000c00 tx=0x7fbc380047b0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 -- 172.21.15.63:0/2940897225 shutdown_connections 2024-05-27T07:57:51.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbc2c0b2a70 0x7fbc2c0b4f80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc4810e6f0 0x7fbc481a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc48105280 0x7fbc4819cd40 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 --2- 172.21.15.63:0/2940897225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc48105bd0 0x7fbc4819d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 -- 172.21.15.63:0/2940897225 >> 172.21.15.63:0/2940897225 conn(0x7fbc48078ab0 msgr2=0x7fbc48102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:51.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 -- 172.21.15.63:0/2940897225 shutdown_connections 2024-05-27T07:57:51.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:51.070+0000 7fbc47200700 1 -- 172.21.15.63:0/2940897225 wait complete. 2024-05-27T07:57:51.073 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-05-27T07:57:51.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:51 smithi063 ceph-mon[31613]: pgmap v767: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:57:51.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:51 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2940897225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:57:51.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:51 smithi086 ceph-mon[34687]: pgmap v767: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:57:51.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:51 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2940897225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:57:51.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:51 smithi149 ceph-mon[35408]: pgmap v767: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T07:57:51.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:51 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2940897225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:57:52.040 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:57:52.582 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:53.246 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:53 smithi063 ceph-mon[31613]: pgmap v768: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:57:53.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:53 smithi086 ceph-mon[34687]: pgmap v768: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:57:53.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:53 smithi149 ceph-mon[35408]: pgmap v768: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T07:57:55.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.070+0000 7faf21c00700 1 -- 172.21.15.63:0/2318146623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf1c108d20 msgr2=0x7faf1c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:55.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.070+0000 7faf21c00700 1 --2- 172.21.15.63:0/2318146623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf1c108d20 0x7faf1c109130 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7faf080099c0 tx=0x7faf08009cd0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.070+0000 7faf21c00700 1 -- 172.21.15.63:0/2318146623 shutdown_connections 2024-05-27T07:57:55.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.070+0000 7faf21c00700 1 --2- 172.21.15.63:0/2318146623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf1c109670 0x7faf1c110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.070+0000 7faf21c00700 1 --2- 172.21.15.63:0/2318146623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf1c1043f0 0x7faf1c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.070+0000 7faf21c00700 1 --2- 172.21.15.63:0/2318146623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf1c108d20 0x7faf1c109130 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.070+0000 7faf21c00700 1 -- 172.21.15.63:0/2318146623 >> 172.21.15.63:0/2318146623 conn(0x7faf1c079d00 msgr2=0x7faf1c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:55.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.071+0000 7faf21c00700 1 -- 172.21.15.63:0/2318146623 shutdown_connections 2024-05-27T07:57:55.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.071+0000 7faf21c00700 1 -- 172.21.15.63:0/2318146623 wait complete. 2024-05-27T07:57:55.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.072+0000 7faf21c00700 1 Processor -- start 2024-05-27T07:57:55.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.072+0000 7faf21c00700 1 -- start start 2024-05-27T07:57:55.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf21c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf1c1043f0 0x7faf1c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf21c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf1c108d20 0x7faf1c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf21c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf1c109670 0x7faf1c1a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf21c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c114180 con 0x7faf1c109670 2024-05-27T07:57:55.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf21c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c114000 con 0x7faf1c108d20 2024-05-27T07:57:55.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf21c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c114300 con 0x7faf1c1043f0 2024-05-27T07:57:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf21200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf1c109670 0x7faf1c1a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf1be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf1c1043f0 0x7faf1c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf21200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf1c109670 0x7faf1c1a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50328/0 (socket says 172.21.15.63:50328) 2024-05-27T07:57:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf21200700 1 -- 172.21.15.63:0/3078656807 learned_addr learned my addr 172.21.15.63:0/3078656807 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.073+0000 7faf1b400700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf1c108d20 0x7faf1c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.074+0000 7faf1be00700 1 -- 172.21.15.63:0/3078656807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf1c108d20 msgr2=0x7faf1c19d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:55.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.074+0000 7faf1be00700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf1c108d20 0x7faf1c19d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.074+0000 7faf1be00700 1 -- 172.21.15.63:0/3078656807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf1c109670 msgr2=0x7faf1c1a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:55.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.074+0000 7faf1be00700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf1c109670 0x7faf1c1a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.074+0000 7faf1be00700 1 -- 172.21.15.63:0/3078656807 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c1a1ce0 con 0x7faf1c1043f0 2024-05-27T07:57:55.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.074+0000 7faf21200700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf1c109670 0x7faf1c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:55.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.075+0000 7faf1be00700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf1c1043f0 0x7faf1c19ce10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00d950 tx=0x7faf0c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:55.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.075+0000 7faf18c00700 1 -- 172.21.15.63:0/3078656807 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c016070 con 0x7faf1c1043f0 2024-05-27T07:57:55.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.075+0000 7faf21c00700 1 -- 172.21.15.63:0/3078656807 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c1a1fd0 con 0x7faf1c1043f0 2024-05-27T07:57:55.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.075+0000 7faf18c00700 1 -- 172.21.15.63:0/3078656807 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf0c00fd20 con 0x7faf1c1043f0 2024-05-27T07:57:55.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.076+0000 7faf18c00700 1 -- 172.21.15.63:0/3078656807 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00e3b0 con 0x7faf1c1043f0 2024-05-27T07:57:55.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.076+0000 7faf21c00700 1 -- 172.21.15.63:0/3078656807 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c1a98f0 con 0x7faf1c1043f0 2024-05-27T07:57:55.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.081+0000 7faf31cb1b80 1 Processor -- start 2024-05-27T07:57:55.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.081+0000 7faf31cb1b80 1 -- start start 2024-05-27T07:57:55.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.081+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c8ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.081+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d410 0x55b0f7c97ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.082+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d820 0x55b0f7c9a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.082+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b0f7bbbe40 con 0x55b0f7c8d820 2024-05-27T07:57:55.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.082+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0f7bab9e0 con 0x55b0f7b0ee40 2024-05-27T07:57:55.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.082+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b0f7bafa20 con 0x55b0f7c8d410 2024-05-27T07:57:55.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.082+0000 7faf18c00700 1 -- 172.21.15.63:0/3078656807 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faf0c019070 con 0x7faf1c1043f0 2024-05-27T07:57:55.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.082+0000 7faf1b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d820 0x55b0f7c9a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.082+0000 7faf1b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d820 0x55b0f7c9a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50340/0 (socket says 172.21.15.63:50340) 2024-05-27T07:57:55.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.082+0000 7faf1b400700 1 -- 172.21.15.63:0/4155631100 learned_addr learned my addr 172.21.15.63:0/4155631100 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:55.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.082+0000 7faf21200700 1 --2- 172.21.15.63:0/4155631100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c8ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.083+0000 7faf1be00700 1 --2- 172.21.15.63:0/4155631100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d410 0x55b0f7c97ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.083+0000 7faf1b400700 1 -- 172.21.15.63:0/4155631100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d410 msgr2=0x55b0f7c97ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:55.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.083+0000 7faf1b400700 1 --2- 172.21.15.63:0/4155631100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d410 0x55b0f7c97ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.083+0000 7faf1b400700 1 -- 172.21.15.63:0/4155631100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7b0ee40 msgr2=0x55b0f7c8ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:55.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.083+0000 7faf1b400700 1 --2- 172.21.15.63:0/4155631100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c8ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.083+0000 7faf1b400700 1 -- 172.21.15.63:0/4155631100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f7b74420 con 0x55b0f7c8d820 2024-05-27T07:57:55.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.083+0000 7faf1b400700 1 --2- 172.21.15.63:0/4155631100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d820 0x55b0f7c9a020 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7faf080099c0 tx=0x7faf080096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320597cdd24d0228 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:55.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.084+0000 7faf11600700 1 -- 172.21.15.63:0/4155631100 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08027070 con 0x55b0f7c8d820 2024-05-27T07:57:55.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.084+0000 7faf11600700 1 -- 172.21.15.63:0/4155631100 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf0800b790 con 0x55b0f7c8d820 2024-05-27T07:57:55.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.084+0000 7faf11600700 1 -- 172.21.15.63:0/4155631100 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080038c0 con 0x55b0f7c8d820 2024-05-27T07:57:55.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.084+0000 7faf18c00700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf000b29f0 0x7faf000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.084+0000 7faf18c00700 1 -- 172.21.15.63:0/3078656807 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7faf0c0e23c0 con 0x7faf1c1043f0 2024-05-27T07:57:55.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.085+0000 7faf21200700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf000b29f0 0x7faf000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.085+0000 7faf31cb1b80 1 -- 172.21.15.63:0/4155631100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d820 msgr2=0x55b0f7c9a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:55.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.085+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/4155631100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d820 0x55b0f7c9a020 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7faf080099c0 tx=0x7faf080096e0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.085+0000 7faf31cb1b80 1 -- 172.21.15.63:0/4155631100 shutdown_connections 2024-05-27T07:57:55.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.085+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/4155631100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d820 0x55b0f7c9a020 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.085+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/4155631100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c8ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.085+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/4155631100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d410 0x55b0f7c97ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.085+0000 7faf31cb1b80 1 -- 172.21.15.63:0/4155631100 >> 172.21.15.63:0/4155631100 conn(0x55b0f7b6efc0 msgr2=0x55b0f7c88d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:55.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.085+0000 7faf31cb1b80 1 -- 172.21.15.63:0/4155631100 shutdown_connections 2024-05-27T07:57:55.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.086+0000 7faf31cb1b80 1 -- 172.21.15.63:0/4155631100 wait complete. 2024-05-27T07:57:55.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.086+0000 7faf21200700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf000b29f0 0x7faf000b4f00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55b0f7c89690 tx=0x7faf14009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:55.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.086+0000 7faf31cb1b80 1 -- start start 2024-05-27T07:57:55.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.087+0000 7faf31cb1b80 1 Processor -- start 2024-05-27T07:57:55.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.087+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c97560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.087+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7c97aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.087+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 0x55b0f7c97fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.087+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b0f7ab1fa0 con 0x55b0f7c8d410 2024-05-27T07:57:55.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.087+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0f7b4c760 con 0x55b0f7b0ee40 2024-05-27T07:57:55.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.087+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b0f7c9a930 con 0x55b0f7c8d820 2024-05-27T07:57:55.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf31cb1b80 10 client.0 _ll_register_callbacks cb 0x55b0f7b55c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf1b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7c97aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf1b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7c97aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50352/0 (socket says 172.21.15.63:50352) 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf1b400700 1 -- 172.21.15.63:0/417380128 learned_addr learned my addr 172.21.15.63:0/417380128 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf1be00700 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c97560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf21200700 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 0x55b0f7c97fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf1b400700 1 -- 172.21.15.63:0/417380128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 msgr2=0x55b0f7c97fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf1b400700 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 0x55b0f7c97fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf1b400700 1 -- 172.21.15.63:0/417380128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7b0ee40 msgr2=0x55b0f7c97560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf1b400700 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c97560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.088+0000 7faf1b400700 1 -- 172.21.15.63:0/417380128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f7afb8d0 con 0x55b0f7c8d410 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.089+0000 7faf1b400700 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7c97aa0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7faf08009990 tx=0x7faf0800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c032a46178e6565 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:55.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.089+0000 7faf06000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:55.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.089+0000 7faf06000700 1 -- 172.21.15.63:0/417380128 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08030020 con 0x55b0f7c8d410 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.089+0000 7faf06000700 1 -- 172.21.15.63:0/417380128 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7faf080053d0 con 0x55b0f7c8d410 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.089+0000 7faf31cb1b80 1 -- 172.21.15.63:0/417380128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0800b790 con 0x55b0f7c8d410 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.090+0000 7faf06000700 1 -- 172.21.15.63:0/417380128 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faf08027040 con 0x55b0f7c8d410 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.090+0000 7faf31cb1b80 10 client.20082 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.090+0000 7faf31cb1b80 1 -- 172.21.15.63:0/417380128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b0f7b7c060 con 0x55b0f7c8d410 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.090+0000 7faf06000700 1 -- 172.21.15.63:0/417380128 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf0800b790 con 0x55b0f7c8d410 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.091+0000 7faf06000700 1 -- 172.21.15.63:0/417380128 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7faf08022480 con 0x55b0f7c8d410 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.091+0000 7faf31cb1b80 10 client.20082 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.091+0000 7faf31cb1b80 10 client.20082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.091+0000 7faf31cb1b80 20 client.20082 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.091+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0f7ca76e0 0x55b0f7ca9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.091+0000 7faf31cb1b80 4 client.20082 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:55.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.091+0000 7faf31cb1b80 1 -- 172.21.15.63:0/417380128 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0f7bbdd90 con 0x55b0f7ca76e0 2024-05-27T07:57:55.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.091+0000 7faf1be00700 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0f7ca76e0 0x55b0f7ca9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.092+0000 7faf1be00700 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0f7ca76e0 0x55b0f7ca9b00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75ab4823296ec5b5 server_cookie=97b7a6595538498a in_seq=0 out_seq=0 2024-05-27T07:57:55.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.092+0000 7faf06000700 10 client.20082 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:57:55.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.094+0000 7faf06000700 1 -- 172.21.15.63:0/417380128 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b0f7bbdd90 con 0x55b0f7ca76e0 2024-05-27T07:57:55.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.094+0000 7faf06000700 10 client.20082 handle_command_reply: tid=0 2024-05-27T07:57:55.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 client.20082 shutdown 2024-05-27T07:57:55.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 -- 172.21.15.63:0/417380128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 msgr2=0x55b0f7c97aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:55.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7c97aa0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7faf08009990 tx=0x7faf0800bbe0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 -- 172.21.15.63:0/417380128 shutdown_connections 2024-05-27T07:57:55.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 0x55b0f7c97fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c97560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0f7ca76e0 0x55b0f7ca9b00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/417380128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7c97aa0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 -- 172.21.15.63:0/417380128 >> 172.21.15.63:0/417380128 conn(0x55b0f7b6efc0 msgr2=0x55b0f7c87ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:55.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf06000700 0 client.20082 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:57:55.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 -- 172.21.15.63:0/417380128 shutdown_connections 2024-05-27T07:57:55.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.095+0000 7faf31cb1b80 1 -- 172.21.15.63:0/417380128 wait complete. 2024-05-27T07:57:55.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.096+0000 7faf31cb1b80 20 client.20082 trim_cache size 0 max 16384 2024-05-27T07:57:55.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.116+0000 7faf31cb1b80 1 Processor -- start 2024-05-27T07:57:55.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.116+0000 7faf31cb1b80 1 -- start start 2024-05-27T07:57:55.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c99a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d410 0x55b0f7ca3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 0x55b0f7c96430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c1b4e00 con 0x55b0f7b0ee40 2024-05-27T07:57:55.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c1b2690 con 0x55b0f7c8d410 2024-05-27T07:57:55.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c1b0500 con 0x55b0f7c8d820 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf1be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c99a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf1be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c99a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50354/0 (socket says 172.21.15.63:50354) 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf1be00700 1 -- 172.21.15.63:0/3607263810 learned_addr learned my addr 172.21.15.63:0/3607263810 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf1b400700 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d410 0x55b0f7ca3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.117+0000 7faf21200700 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 0x55b0f7c96430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf1be00700 1 -- 172.21.15.63:0/3607263810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 msgr2=0x55b0f7c96430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf1be00700 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 0x55b0f7c96430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf1be00700 1 -- 172.21.15.63:0/3607263810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d410 msgr2=0x55b0f7ca3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf1be00700 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d410 0x55b0f7ca3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf1be00700 1 -- 172.21.15.63:0/3607263810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f7c831d0 con 0x55b0f7b0ee40 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf1b400700 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d410 0x55b0f7ca3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf21200700 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 0x55b0f7c96430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf1be00700 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c99a00 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0df7b0 tx=0x7faf0c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be87e9bca003470 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:55.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf10c00700 1 -- 172.21.15.63:0/3607263810 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c020040 con 0x55b0f7b0ee40 2024-05-27T07:57:55.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf10c00700 1 -- 172.21.15.63:0/3607263810 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf0c0a8df0 con 0x55b0f7b0ee40 2024-05-27T07:57:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.118+0000 7faf10c00700 1 -- 172.21.15.63:0/3607263810 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0b2140 con 0x55b0f7b0ee40 2024-05-27T07:57:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.119+0000 7faf31cb1b80 1 -- 172.21.15.63:0/3607263810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7b0ee40 msgr2=0x55b0f7c99a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.119+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c99a00 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0df7b0 tx=0x7faf0c00bf90 comp rx=0 tx=0).stop 2024-05-27T07:57:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.119+0000 7faf31cb1b80 1 -- 172.21.15.63:0/3607263810 shutdown_connections 2024-05-27T07:57:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.119+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7c8d820 0x55b0f7c96430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.119+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7b0ee40 0x55b0f7c99a00 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.119+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/3607263810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d410 0x55b0f7ca3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.119+0000 7faf31cb1b80 1 -- 172.21.15.63:0/3607263810 >> 172.21.15.63:0/3607263810 conn(0x55b0f7b6efc0 msgr2=0x55b0f7c95d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.120+0000 7faf31cb1b80 1 -- 172.21.15.63:0/3607263810 shutdown_connections 2024-05-27T07:57:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.120+0000 7faf31cb1b80 1 -- 172.21.15.63:0/3607263810 wait complete. 2024-05-27T07:57:55.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.120+0000 7faf31cb1b80 1 -- start start 2024-05-27T07:57:55.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.121+0000 7faf31cb1b80 1 Processor -- start 2024-05-27T07:57:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.121+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7b0ee40 0x55b0f7cacd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.121+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7caa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.121+0000 7faf31cb1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d820 0x55b0f7caa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.121+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c1b0b50 con 0x55b0f7c8d410 2024-05-27T07:57:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.121+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c1b1fc0 con 0x55b0f7c8d820 2024-05-27T07:57:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.121+0000 7faf31cb1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c1b1220 con 0x55b0f7b0ee40 2024-05-27T07:57:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.122+0000 7faf31cb1b80 10 client.0 _ll_register_callbacks cb 0x55b0f7b55c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.122+0000 7faf1b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7caa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.122+0000 7faf1b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7caa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50366/0 (socket says 172.21.15.63:50366) 2024-05-27T07:57:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.122+0000 7faf1b400700 1 -- 172.21.15.63:0/1458329358 learned_addr learned my addr 172.21.15.63:0/1458329358 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.122+0000 7faf1be00700 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7b0ee40 0x55b0f7cacd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.122+0000 7faf21200700 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d820 0x55b0f7caa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.123+0000 7faf1b400700 1 -- 172.21.15.63:0/1458329358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7b0ee40 msgr2=0x55b0f7cacd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.123+0000 7faf1b400700 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7b0ee40 0x55b0f7cacd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.123+0000 7faf1b400700 1 -- 172.21.15.63:0/1458329358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d820 msgr2=0x55b0f7caa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.123+0000 7faf1b400700 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d820 0x55b0f7caa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.123+0000 7faf1b400700 1 -- 172.21.15.63:0/1458329358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f7c8e420 con 0x55b0f7c8d410 2024-05-27T07:57:55.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.123+0000 7faf1be00700 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7b0ee40 0x55b0f7cacd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:55.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.123+0000 7faf1b400700 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7caa090 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b580 tx=0x7faf08022740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adcd12102d91637 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:55.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.123+0000 7faf06000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:55.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.124+0000 7faf06000700 1 -- 172.21.15.63:0/1458329358 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08030070 con 0x55b0f7c8d410 2024-05-27T07:57:55.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.124+0000 7faf06000700 1 -- 172.21.15.63:0/1458329358 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7faf080051d0 con 0x55b0f7c8d410 2024-05-27T07:57:55.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.124+0000 7faf31cb1b80 1 -- 172.21.15.63:0/1458329358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0c0a8df0 con 0x55b0f7c8d410 2024-05-27T07:57:55.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.124+0000 7faf06000700 1 -- 172.21.15.63:0/1458329358 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faf08047050 con 0x55b0f7c8d410 2024-05-27T07:57:55.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.124+0000 7faf31cb1b80 10 client.20088 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:55.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.124+0000 7faf31cb1b80 1 -- 172.21.15.63:0/1458329358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b0f7c8a350 con 0x55b0f7c8d410 2024-05-27T07:57:55.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.125+0000 7faf06000700 1 -- 172.21.15.63:0/1458329358 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf0c0a8df0 con 0x55b0f7c8d410 2024-05-27T07:57:55.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.125+0000 7faf06000700 1 -- 172.21.15.63:0/1458329358 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7faf08003c60 con 0x55b0f7c8d410 2024-05-27T07:57:55.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.125+0000 7faf31cb1b80 10 client.20088 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:55.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.125+0000 7faf31cb1b80 10 client.20088 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:57:55.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.125+0000 7faf31cb1b80 20 client.20088 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:55.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.125+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0f7d1a600 0x55b0f7d1ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:55.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.125+0000 7faf31cb1b80 4 client.20088 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:57:55.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.125+0000 7faf31cb1b80 1 -- 172.21.15.63:0/1458329358 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0f7bb7570 con 0x55b0f7d1a600 2024-05-27T07:57:55.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.126+0000 7faf1be00700 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0f7d1a600 0x55b0f7d1ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:55.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.127+0000 7faf1be00700 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0f7d1a600 0x55b0f7d1ca20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e137f050ba4a3b9 server_cookie=bd90f305f845215f in_seq=0 out_seq=0 2024-05-27T07:57:55.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.127+0000 7faf06000700 10 client.20088 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:57:55.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.127+0000 7faf06000700 1 -- 172.21.15.63:0/1458329358 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0f7bb7570 con 0x55b0f7d1a600 2024-05-27T07:57:55.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.127+0000 7faf06000700 10 client.20088 handle_command_reply: tid=0 2024-05-27T07:57:55.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.127+0000 7faf31cb1b80 1 client.20088 shutdown 2024-05-27T07:57:55.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 1 -- 172.21.15.63:0/1458329358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 msgr2=0x55b0f7caa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:55.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7caa090 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b580 tx=0x7faf08022740 comp rx=0 tx=0).stop 2024-05-27T07:57:55.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 1 -- 172.21.15.63:0/1458329358 shutdown_connections 2024-05-27T07:57:55.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0f7c8d820 0x55b0f7caa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0f7b0ee40 0x55b0f7cacd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0f7d1a600 0x55b0f7d1ca20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 1 --2- 172.21.15.63:0/1458329358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0f7c8d410 0x55b0f7caa090 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 1 -- 172.21.15.63:0/1458329358 >> 172.21.15.63:0/1458329358 conn(0x55b0f7b6efc0 msgr2=0x55b0f7ca2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:55.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf06000700 0 client.20088 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:57:55.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 1 -- 172.21.15.63:0/1458329358 shutdown_connections 2024-05-27T07:57:55.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 1 -- 172.21.15.63:0/1458329358 wait complete. 2024-05-27T07:57:55.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.128+0000 7faf31cb1b80 20 client.20088 trim_cache size 0 max 16384 2024-05-27T07:57:55.130 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:57:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.129+0000 7faf21c00700 1 -- 172.21.15.63:0/3078656807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf000b29f0 msgr2=0x7faf000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.129+0000 7faf21c00700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf000b29f0 0x7faf000b4f00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55b0f7c89690 tx=0x7faf14009040 comp rx=0 tx=0).stop 2024-05-27T07:57:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.129+0000 7faf21c00700 1 -- 172.21.15.63:0/3078656807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf1c1043f0 msgr2=0x7faf1c19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.129+0000 7faf21c00700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf1c1043f0 0x7faf1c19ce10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00d950 tx=0x7faf0c00dc60 comp rx=0 tx=0).stop 2024-05-27T07:57:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.129+0000 7faf21c00700 1 -- 172.21.15.63:0/3078656807 shutdown_connections 2024-05-27T07:57:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.129+0000 7faf21c00700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf000b29f0 0x7faf000b4f00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.129+0000 7faf21c00700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf1c109670 0x7faf1c1a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.129+0000 7faf21c00700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf1c1043f0 0x7faf1c19ce10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.129+0000 7faf21c00700 1 --2- 172.21.15.63:0/3078656807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf1c108d20 0x7faf1c19d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.129+0000 7faf21c00700 1 -- 172.21.15.63:0/3078656807 >> 172.21.15.63:0/3078656807 conn(0x7faf1c079d00 msgr2=0x7faf1c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.130+0000 7faf21c00700 1 -- 172.21.15.63:0/3078656807 shutdown_connections 2024-05-27T07:57:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:55.130+0000 7faf21c00700 1 -- 172.21.15.63:0/3078656807 wait complete. 2024-05-27T07:57:55.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:55 smithi063 ceph-mon[31613]: pgmap v769: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:57:55.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:55 smithi086 ceph-mon[34687]: pgmap v769: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:57:55.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:55 smithi149 ceph-mon[35408]: pgmap v769: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:57:56.213 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:57:56.730 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:57.682 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:57 smithi063 ceph-mon[31613]: pgmap v770: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:57:57.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:57 smithi086 ceph-mon[34687]: pgmap v770: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:57:57.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:57 smithi149 ceph-mon[35408]: pgmap v770: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T07:57:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.134+0000 7f4a9ae00700 1 -- 172.21.15.63:0/610211494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a94105280 msgr2=0x7f4a94105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.134+0000 7f4a9ae00700 1 --2- 172.21.15.63:0/610211494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a94105280 0x7f4a94105690 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009990 tx=0x7f4a88009ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.134+0000 7f4a9ae00700 1 -- 172.21.15.63:0/610211494 shutdown_connections 2024-05-27T07:57:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.134+0000 7f4a9ae00700 1 --2- 172.21.15.63:0/610211494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a9410e6f0 0x7f4a94110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.134+0000 7f4a9ae00700 1 --2- 172.21.15.63:0/610211494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a94105280 0x7f4a94105690 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.134+0000 7f4a9ae00700 1 --2- 172.21.15.63:0/610211494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a94105bd0 0x7f4a9410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.134+0000 7f4a9ae00700 1 -- 172.21.15.63:0/610211494 >> 172.21.15.63:0/610211494 conn(0x7f4a94078ab0 msgr2=0x7f4a94078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.134+0000 7f4a9ae00700 1 -- 172.21.15.63:0/610211494 shutdown_connections 2024-05-27T07:57:59.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.134+0000 7f4a9ae00700 1 -- 172.21.15.63:0/610211494 wait complete. 2024-05-27T07:57:59.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.136+0000 7f4a9ae00700 1 Processor -- start 2024-05-27T07:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.136+0000 7f4a9ae00700 1 -- start start 2024-05-27T07:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a9ae00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a94105280 0x7f4a941a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a9ae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a94105bd0 0x7f4a941a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a9ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9410e6f0 0x7f4a9419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a9ae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94114140 con 0x7f4a94105bd0 2024-05-27T07:57:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a9ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94113fc0 con 0x7f4a9410e6f0 2024-05-27T07:57:59.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a9ae00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941142c0 con 0x7f4a94105280 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a99000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a94105bd0 0x7f4a941a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a99a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a94105280 0x7f4a941a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a99a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a94105280 0x7f4a941a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:51154/0 (socket says 172.21.15.63:51154) 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a99a00700 1 -- 172.21.15.63:0/2666627804 learned_addr learned my addr 172.21.15.63:0/2666627804 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a99000700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a94105bd0 0x7f4a941a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54660/0 (socket says 172.21.15.63:54660) 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.137+0000 7f4a9a400700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9410e6f0 0x7f4a9419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.138+0000 7f4a99000700 1 -- 172.21.15.63:0/2666627804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a94105280 msgr2=0x7f4a941a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.138+0000 7f4a99000700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a94105280 0x7f4a941a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.138+0000 7f4a99000700 1 -- 172.21.15.63:0/2666627804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9410e6f0 msgr2=0x7f4a9419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.138+0000 7f4a99000700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9410e6f0 0x7f4a9419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.138+0000 7f4a99000700 1 -- 172.21.15.63:0/2666627804 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9419b050 con 0x7f4a94105bd0 2024-05-27T07:57:59.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.138+0000 7f4a99a00700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a94105280 0x7f4a941a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:59.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.138+0000 7f4a99000700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a94105bd0 0x7f4a941a0cc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000d950 tx=0x7f4a8000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.139+0000 7f4a92000700 1 -- 172.21.15.63:0/2666627804 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80016070 con 0x7f4a94105bd0 2024-05-27T07:57:59.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.139+0000 7f4a92000700 1 -- 172.21.15.63:0/2666627804 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a8000fd20 con 0x7f4a94105bd0 2024-05-27T07:57:59.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.139+0000 7f4a9ae00700 1 -- 172.21.15.63:0/2666627804 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9419b340 con 0x7f4a94105bd0 2024-05-27T07:57:59.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.139+0000 7f4a9ae00700 1 -- 172.21.15.63:0/2666627804 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a94102ea0 con 0x7f4a94105bd0 2024-05-27T07:57:59.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.139+0000 7f4a92000700 1 -- 172.21.15.63:0/2666627804 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8000e3b0 con 0x7f4a94105bd0 2024-05-27T07:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.144+0000 7f4aaaeffb80 1 Processor -- start 2024-05-27T07:57:59.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.145+0000 7f4aaaeffb80 1 -- start start 2024-05-27T07:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.145+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aadbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.145+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8aadc410 0x563b8aae6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.145+0000 7f4a92000700 1 -- 172.21.15.63:0/2666627804 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4a80019070 con 0x7f4a94105bd0 2024-05-27T07:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.145+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 0x563b8aae9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.145+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563b8aa0ae40 con 0x563b8a95de40 2024-05-27T07:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.145+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b8a9fa9e0 con 0x563b8aadc410 2024-05-27T07:57:59.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.145+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563b8a9fea20 con 0x563b8aadc820 2024-05-27T07:57:59.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.146+0000 7f4a99000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 0x563b8aae9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.146+0000 7f4a99000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 0x563b8aae9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:51156/0 (socket says 172.21.15.63:51156) 2024-05-27T07:57:59.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.146+0000 7f4a99000700 1 -- 172.21.15.63:0/2749090058 learned_addr learned my addr 172.21.15.63:0/2749090058 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:59.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.146+0000 7f4a9a400700 1 --2- 172.21.15.63:0/2749090058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aadbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.146+0000 7f4a99a00700 1 --2- 172.21.15.63:0/2749090058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8aadc410 0x563b8aae6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.146+0000 7f4a9a400700 1 -- 172.21.15.63:0/2749090058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 msgr2=0x563b8aae9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.146+0000 7f4a9a400700 1 --2- 172.21.15.63:0/2749090058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 0x563b8aae9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.146+0000 7f4a9a400700 1 -- 172.21.15.63:0/2749090058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8aadc410 msgr2=0x563b8aae6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.146+0000 7f4a9a400700 1 --2- 172.21.15.63:0/2749090058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8aadc410 0x563b8aae6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.146+0000 7f4a9a400700 1 -- 172.21.15.63:0/2749090058 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b8a9c3420 con 0x563b8a95de40 2024-05-27T07:57:59.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.147+0000 7f4a9a400700 1 --2- 172.21.15.63:0/2749090058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aadbed0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c003e80 tx=0x7f4a8c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df459cefa971a2e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.147+0000 7f4a86a00700 1 -- 172.21.15.63:0/2749090058 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c004bb0 con 0x563b8a95de40 2024-05-27T07:57:59.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.148+0000 7f4a86a00700 1 -- 172.21.15.63:0/2749090058 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a8c004800 con 0x563b8a95de40 2024-05-27T07:57:59.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.148+0000 7f4a86a00700 1 -- 172.21.15.63:0/2749090058 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c007c10 con 0x563b8a95de40 2024-05-27T07:57:59.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.148+0000 7f4a92000700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4a7c0b2a70 0x7f4a7c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.148+0000 7f4a92000700 1 -- 172.21.15.63:0/2666627804 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f4a800e2650 con 0x7f4a94105bd0 2024-05-27T07:57:59.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.149+0000 7f4a99a00700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4a7c0b2a70 0x7f4a7c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.149+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/2749090058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 msgr2=0x563b8aadbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.149+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/2749090058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aadbed0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c003e80 tx=0x7f4a8c009ba0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.149+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/2749090058 shutdown_connections 2024-05-27T07:57:59.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.149+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/2749090058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 0x563b8aae9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.149+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/2749090058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aadbed0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.149+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/2749090058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8aadc410 0x563b8aae6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.149+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/2749090058 >> 172.21.15.63:0/2749090058 conn(0x563b8a9bdfc0 msgr2=0x563b8aad7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.149+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/2749090058 shutdown_connections 2024-05-27T07:57:59.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.149+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/2749090058 wait complete. 2024-05-27T07:57:59.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.150+0000 7f4a99a00700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4a7c0b2a70 0x7f4a7c0b4f80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x563b8aadd430 tx=0x7f4a88005040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.150+0000 7f4aaaeffb80 1 -- start start 2024-05-27T07:57:59.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.151+0000 7f4aaaeffb80 1 Processor -- start 2024-05-27T07:57:59.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.151+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 0x563b8aaf5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.151+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8aadc410 0x563b8aaf56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.151+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 0x563b8aaef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.151+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563b8a900fa0 con 0x563b8aadc410 2024-05-27T07:57:59.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.151+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b8a99b760 con 0x563b8a95de40 2024-05-27T07:57:59.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.151+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563b8aae9990 con 0x563b8aadc820 2024-05-27T07:57:59.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.151+0000 7f4aaaeffb80 10 client.0 _ll_register_callbacks cb 0x563b8a9a4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:59.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.152+0000 7f4a99000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8aadc410 0x563b8aaf56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.152+0000 7f4a99000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8aadc410 0x563b8aaf56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54674/0 (socket says 172.21.15.63:54674) 2024-05-27T07:57:59.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.152+0000 7f4a99000700 1 -- 172.21.15.63:0/3616733444 learned_addr learned my addr 172.21.15.63:0/3616733444 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:59.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.152+0000 7f4a9a400700 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 0x563b8aaef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.152+0000 7f4a99a00700 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 0x563b8aaf5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a99000700 1 -- 172.21.15.63:0/3616733444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 msgr2=0x563b8aaef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a99000700 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 0x563b8aaef590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a99000700 1 -- 172.21.15.63:0/3616733444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 msgr2=0x563b8aaf5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a99000700 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 0x563b8aaf5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a99000700 1 -- 172.21.15.63:0/3616733444 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b8a94a8d0 con 0x563b8aadc410 2024-05-27T07:57:59.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a9a400700 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 0x563b8aaef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:59.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a99a00700 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 0x563b8aaf5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:59.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a99000700 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8aadc410 0x563b8aaf56d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f4a800a5590 tx=0x7f4a800108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cff336bf928390 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a7b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:59.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a7b400700 1 -- 172.21.15.63:0/3616733444 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80020030 con 0x563b8aadc410 2024-05-27T07:57:59.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.153+0000 7f4a7b400700 1 -- 172.21.15.63:0/3616733444 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f4a800e21d0 con 0x563b8aadc410 2024-05-27T07:57:59.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.154+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3616733444 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a8c004800 con 0x563b8aadc410 2024-05-27T07:57:59.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.154+0000 7f4a7b400700 1 -- 172.21.15.63:0/3616733444 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4a800a8c80 con 0x563b8aadc410 2024-05-27T07:57:59.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.154+0000 7f4aaaeffb80 10 client.20100 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:59.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.154+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3616733444 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563b8a9cb060 con 0x563b8aadc410 2024-05-27T07:57:59.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.155+0000 7f4a7b400700 1 -- 172.21.15.63:0/3616733444 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a8c004800 con 0x563b8aadc410 2024-05-27T07:57:59.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.155+0000 7f4a7b400700 1 -- 172.21.15.63:0/3616733444 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4a800a9770 con 0x563b8aadc410 2024-05-27T07:57:59.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.155+0000 7f4aaaeffb80 10 client.20100 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:59.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.155+0000 7f4aaaeffb80 10 client.20100 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:57:59.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.155+0000 7f4aaaeffb80 20 client.20100 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:59.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.155+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563b8aaff160 0x563b8ab01580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.155+0000 7f4aaaeffb80 4 client.20100 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:59.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.155+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3616733444 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b8aa0cd90 con 0x563b8aaff160 2024-05-27T07:57:59.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.156+0000 7f4a99a00700 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563b8aaff160 0x563b8ab01580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.157+0000 7f4a99a00700 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563b8aaff160 0x563b8ab01580 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1943d28882571086 server_cookie=f8ffe60aecb3b02 in_seq=0 out_seq=0 2024-05-27T07:57:59.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.157+0000 7f4a7b400700 10 client.20100 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:57:59.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.158+0000 7f4a7b400700 1 -- 172.21.15.63:0/3616733444 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563b8aa0cd90 con 0x563b8aaff160 2024-05-27T07:57:59.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.158+0000 7f4a7b400700 10 client.20100 handle_command_reply: tid=0 2024-05-27T07:57:59.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.159+0000 7f4aaaeffb80 1 client.20100 shutdown 2024-05-27T07:57:59.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.159+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3616733444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8aadc410 msgr2=0x563b8aaf56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.159+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8aadc410 0x563b8aaf56d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f4a800a5590 tx=0x7f4a800108e0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.159+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3616733444 shutdown_connections 2024-05-27T07:57:59.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.159+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc820 0x563b8aaef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.159+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 0x563b8aaf5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.159+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563b8aaff160 0x563b8ab01580 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.159+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3616733444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8aadc410 0x563b8aaf56d0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.159+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3616733444 >> 172.21.15.63:0/3616733444 conn(0x563b8a9bdfc0 msgr2=0x563b8aad7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.159+0000 7f4a7b400700 0 client.20100 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:57:59.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.160+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3616733444 shutdown_connections 2024-05-27T07:57:59.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.160+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3616733444 wait complete. 2024-05-27T07:57:59.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.160+0000 7f4aaaeffb80 20 client.20100 trim_cache size 0 max 16384 2024-05-27T07:57:59.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.184+0000 7f4aaaeffb80 1 Processor -- start 2024-05-27T07:57:59.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.184+0000 7f4aaaeffb80 1 -- start start 2024-05-27T07:57:59.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.185+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aaf5800 0x563b8aade020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.185+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a9bdfc0 0x563b8ab06d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.185+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 0x563b8ab07270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.185+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74011610 con 0x563b8a9bdfc0 2024-05-27T07:57:59.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.185+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400eea0 con 0x563b8a95de40 2024-05-27T07:57:59.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.185+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400cd10 con 0x563b8aaf5800 2024-05-27T07:57:59.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a99a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aaf5800 0x563b8aade020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a99a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aaf5800 0x563b8aade020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:51176/0 (socket says 172.21.15.63:51176) 2024-05-27T07:57:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a99a00700 1 -- 172.21.15.63:0/2057149553 learned_addr learned my addr 172.21.15.63:0/2057149553 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a99000700 1 --2- 172.21.15.63:0/2057149553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a9bdfc0 0x563b8ab06d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a9a400700 1 --2- 172.21.15.63:0/2057149553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 0x563b8ab07270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a99000700 1 -- 172.21.15.63:0/2057149553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aaf5800 msgr2=0x563b8aade020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a99000700 1 --2- 172.21.15.63:0/2057149553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aaf5800 0x563b8aade020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a99000700 1 -- 172.21.15.63:0/2057149553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 msgr2=0x563b8ab07270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a99000700 1 --2- 172.21.15.63:0/2057149553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 0x563b8ab07270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a99000700 1 -- 172.21.15.63:0/2057149553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b8aadba40 con 0x563b8a9bdfc0 2024-05-27T07:57:59.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.186+0000 7f4a9a400700 1 --2- 172.21.15.63:0/2057149553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 0x563b8ab07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:59.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.187+0000 7f4a99000700 1 --2- 172.21.15.63:0/2057149553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a9bdfc0 0x563b8ab06d30 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f4a80000c00 tx=0x7f4a800e28d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08f3078ef364aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.187+0000 7f4a86000700 1 -- 172.21.15.63:0/2057149553 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a800c2070 con 0x563b8a9bdfc0 2024-05-27T07:57:59.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.187+0000 7f4a86000700 1 -- 172.21.15.63:0/2057149553 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a800a9020 con 0x563b8a9bdfc0 2024-05-27T07:57:59.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.187+0000 7f4a86000700 1 -- 172.21.15.63:0/2057149553 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80020070 con 0x563b8a9bdfc0 2024-05-27T07:57:59.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.188+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/2057149553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a9bdfc0 msgr2=0x563b8ab06d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.188+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/2057149553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a9bdfc0 0x563b8ab06d30 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f4a80000c00 tx=0x7f4a800e28d0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.189+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/2057149553 shutdown_connections 2024-05-27T07:57:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.189+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/2057149553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8a95de40 0x563b8ab07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.189+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/2057149553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aaf5800 0x563b8aade020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.189+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/2057149553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a9bdfc0 0x563b8ab06d30 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.189+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/2057149553 >> 172.21.15.63:0/2057149553 conn(0x563b8aad0f60 msgr2=0x563b8aadd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.189+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/2057149553 shutdown_connections 2024-05-27T07:57:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.189+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/2057149553 wait complete. 2024-05-27T07:57:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.189+0000 7f4aaaeffb80 1 -- start start 2024-05-27T07:57:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.190+0000 7f4aaaeffb80 1 Processor -- start 2024-05-27T07:57:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.190+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aaf92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.190+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8aaf5800 0x563b8aaf97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.190+0000 7f4aaaeffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc410 0x563b8aaf9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.190+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400d360 con 0x563b8a95de40 2024-05-27T07:57:59.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.190+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400e7d0 con 0x563b8aaf5800 2024-05-27T07:57:59.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.190+0000 7f4aaaeffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400da30 con 0x563b8aadc410 2024-05-27T07:57:59.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.190+0000 7f4aaaeffb80 10 client.0 _ll_register_callbacks cb 0x563b8a9a4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:59.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a99a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aaf92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a99a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aaf92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54690/0 (socket says 172.21.15.63:54690) 2024-05-27T07:57:59.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a99a00700 1 -- 172.21.15.63:0/3288050056 learned_addr learned my addr 172.21.15.63:0/3288050056 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:59.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a99000700 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8aaf5800 0x563b8aaf97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a9a400700 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc410 0x563b8aaf9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a99a00700 1 -- 172.21.15.63:0/3288050056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc410 msgr2=0x563b8aaf9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a99a00700 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc410 0x563b8aaf9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a99a00700 1 -- 172.21.15.63:0/3288050056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8aaf5800 msgr2=0x563b8aaf97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a99a00700 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8aaf5800 0x563b8aaf97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a99a00700 1 -- 172.21.15.63:0/3288050056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b8ab68d80 con 0x563b8a95de40 2024-05-27T07:57:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.191+0000 7f4a9a400700 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc410 0x563b8aaf9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.192+0000 7f4a99a00700 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aaf92b0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f4a88005910 tx=0x7f4a8800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbcc5b445bdec7a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.192+0000 7f4a7b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.192+0000 7f4a7b400700 1 -- 172.21.15.63:0/3288050056 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88025480 con 0x563b8a95de40 2024-05-27T07:57:59.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.192+0000 7f4a7b400700 1 -- 172.21.15.63:0/3288050056 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7f4a88025620 con 0x563b8a95de40 2024-05-27T07:57:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.192+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3288050056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a800a9020 con 0x563b8a95de40 2024-05-27T07:57:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.193+0000 7f4a7b400700 1 -- 172.21.15.63:0/3288050056 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4a88046050 con 0x563b8a95de40 2024-05-27T07:57:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.193+0000 7f4aaaeffb80 10 client.20106 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.193+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3288050056 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563b8ab69fc0 con 0x563b8a95de40 2024-05-27T07:57:59.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.193+0000 7f4a7b400700 1 -- 172.21.15.63:0/3288050056 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a800a9020 con 0x563b8a95de40 2024-05-27T07:57:59.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.193+0000 7f4a7b400700 1 -- 172.21.15.63:0/3288050056 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4a8800f820 con 0x563b8a95de40 2024-05-27T07:57:59.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.194+0000 7f4aaaeffb80 10 client.20106 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:59.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.194+0000 7f4aaaeffb80 10 client.20106 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:57:59.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.194+0000 7f4aaaeffb80 20 client.20106 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:59.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.194+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563b8aadc820 0x563b8ab6c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.194+0000 7f4aaaeffb80 4 client.20106 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:57:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.194+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3288050056 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b8aa06570 con 0x563b8aadc820 2024-05-27T07:57:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.194+0000 7f4a99000700 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563b8aadc820 0x563b8ab6c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.195+0000 7f4a99000700 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563b8aadc820 0x563b8ab6c740 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3aaf5d1833553214 server_cookie=d2d20ecdee4f1a49 in_seq=0 out_seq=0 2024-05-27T07:57:59.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.195+0000 7f4a7b400700 10 client.20106 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:57:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4a7b400700 1 -- 172.21.15.63:0/3288050056 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b8aa06570 con 0x563b8aadc820 2024-05-27T07:57:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4a7b400700 10 client.20106 handle_command_reply: tid=0 2024-05-27T07:57:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 client.20106 shutdown 2024-05-27T07:57:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3288050056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 msgr2=0x563b8aaf92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aaf92b0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f4a88005910 tx=0x7f4a8800b540 comp rx=0 tx=0).stop 2024-05-27T07:57:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3288050056 shutdown_connections 2024-05-27T07:57:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b8aadc410 0x563b8aaf9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563b8aadc820 0x563b8ab6c740 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b8a95de40 0x563b8aaf92b0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 --2- 172.21.15.63:0/3288050056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b8aaf5800 0x563b8aaf97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3288050056 >> 172.21.15.63:0/3288050056 conn(0x563b8aad0f60 msgr2=0x563b8aad6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4a7b400700 0 client.20106 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:57:59.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3288050056 shutdown_connections 2024-05-27T07:57:59.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.196+0000 7f4aaaeffb80 1 -- 172.21.15.63:0/3288050056 wait complete. 2024-05-27T07:57:59.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.197+0000 7f4aaaeffb80 20 client.20106 trim_cache size 0 max 16384 2024-05-27T07:57:59.212 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:57:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 -- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4a7c0b2a70 msgr2=0x7f4a7c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4a7c0b2a70 0x7f4a7c0b4f80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x563b8aadd430 tx=0x7f4a88005040 comp rx=0 tx=0).stop 2024-05-27T07:57:59.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 -- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a94105bd0 msgr2=0x7f4a941a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a94105bd0 0x7f4a941a0cc0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000d950 tx=0x7f4a8000dc60 comp rx=0 tx=0).stop 2024-05-27T07:57:59.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 -- 172.21.15.63:0/2666627804 shutdown_connections 2024-05-27T07:57:59.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4a7c0b2a70 0x7f4a7c0b4f80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a9410e6f0 0x7f4a9419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a94105280 0x7f4a941a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 --2- 172.21.15.63:0/2666627804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a94105bd0 0x7f4a941a0cc0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 -- 172.21.15.63:0/2666627804 >> 172.21.15.63:0/2666627804 conn(0x7f4a94078ab0 msgr2=0x7f4a940fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 -- 172.21.15.63:0/2666627804 shutdown_connections 2024-05-27T07:57:59.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:57:59.198+0000 7f4a9ae00700 1 -- 172.21.15.63:0/2666627804 wait complete. 2024-05-27T07:57:59.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:59 smithi063 ceph-mon[31613]: pgmap v771: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:57:59.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:57:59 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:57:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.504+0000 7fb001600640 1 -- 172.21.15.63:0/196388435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc109f50 msgr2=0x7faffc103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.504+0000 7fb001600640 1 --2- 172.21.15.63:0/196388435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc109f50 0x7faffc103be0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009900 tx=0x7fafe802a3f0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.505+0000 7fb001600640 1 -- 172.21.15.63:0/196388435 shutdown_connections 2024-05-27T07:57:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.505+0000 7fb001600640 1 --2- 172.21.15.63:0/196388435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faffc104250 0x7faffc1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.505+0000 7fb001600640 1 --2- 172.21.15.63:0/196388435 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc109f50 0x7faffc103be0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.505+0000 7fb001600640 1 --2- 172.21.15.63:0/196388435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faffc109ab0 0x7faffc1036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.505+0000 7fb001600640 1 -- 172.21.15.63:0/196388435 >> 172.21.15.63:0/196388435 conn(0x7faffc079240 msgr2=0x7faffc079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.505+0000 7fb001600640 1 -- 172.21.15.63:0/196388435 shutdown_connections 2024-05-27T07:57:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.505+0000 7fb001600640 1 -- 172.21.15.63:0/196388435 wait complete. 2024-05-27T07:57:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.506+0000 7fb001600640 1 Processor -- start 2024-05-27T07:57:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.507+0000 7fb001600640 1 -- start start 2024-05-27T07:57:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.507+0000 7fb001600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc104250 0x7faffc19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.507+0000 7fb001600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faffc109ab0 0x7faffc19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.507+0000 7fb001600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faffc19ef30 0x7faffc19f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.507+0000 7fb001600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faffc113bb0 con 0x7faffc104250 2024-05-27T07:57:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.507+0000 7fb001600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faffc113a30 con 0x7faffc19ef30 2024-05-27T07:57:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.507+0000 7fb001600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faffc113d30 con 0x7faffc109ab0 2024-05-27T07:57:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.508+0000 7faffbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc104250 0x7faffc19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.508+0000 7faffbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc104250 0x7faffc19a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54720/0 (socket says 172.21.15.63:54720) 2024-05-27T07:57:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.508+0000 7faffbe00640 1 -- 172.21.15.63:0/1987553522 learned_addr learned my addr 172.21.15.63:0/1987553522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.508+0000 7faffb400640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faffc109ab0 0x7faffc19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.508+0000 7fb000c00640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faffc19ef30 0x7faffc19f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.508+0000 7faffbe00640 1 -- 172.21.15.63:0/1987553522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faffc109ab0 msgr2=0x7faffc19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.508+0000 7faffbe00640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faffc109ab0 0x7faffc19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.508+0000 7faffbe00640 1 -- 172.21.15.63:0/1987553522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faffc19ef30 msgr2=0x7faffc19f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.508+0000 7faffbe00640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faffc19ef30 0x7faffc19f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.508+0000 7faffbe00640 1 -- 172.21.15.63:0/1987553522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc19fa40 con 0x7faffc104250 2024-05-27T07:57:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.509+0000 7faffbe00640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc104250 0x7faffc19a7d0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fafec00d770 tx=0x7fafec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.509+0000 7faff8c00640 1 -- 172.21.15.63:0/1987553522 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec017070 con 0x7faffc104250 2024-05-27T07:57:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.509+0000 7faff8c00640 1 -- 172.21.15.63:0/1987553522 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafec009c30 con 0x7faffc104250 2024-05-27T07:57:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.509+0000 7fb001600640 1 -- 172.21.15.63:0/1987553522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc19fd30 con 0x7faffc104250 2024-05-27T07:57:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.509+0000 7faff8c00640 1 -- 172.21.15.63:0/1987553522 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec00f620 con 0x7faffc104250 2024-05-27T07:57:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.510+0000 7fb001600640 1 -- 172.21.15.63:0/1987553522 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc1a7530 con 0x7faffc104250 2024-05-27T07:57:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.514+0000 7fb009b8f740 1 Processor -- start 2024-05-27T07:57:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.514+0000 7fb009b8f740 1 -- start start 2024-05-27T07:57:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7faff8c00640 1 -- 172.21.15.63:0/1987553522 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fafec01a070 con 0x7faffc104250 2024-05-27T07:57:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 0x564c5e3a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7fb009b8f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564c5e318b90 con 0x564c5dfe44d0 2024-05-27T07:57:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7fb009b8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564c5e308730 con 0x564c5e0051b0 2024-05-27T07:57:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7fb009b8f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564c5e30c770 con 0x564c5dfdc2f0 2024-05-27T07:57:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7faffb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7faffb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3a83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54734/0 (socket says 172.21.15.63:54734) 2024-05-27T07:57:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7faffb400640 1 -- 172.21.15.63:0/1607604280 learned_addr learned my addr 172.21.15.63:0/1607604280 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.515+0000 7fb000c00640 1 --2- 172.21.15.63:0/1607604280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.516+0000 7faffbe00640 1 --2- 172.21.15.63:0/1607604280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 0x564c5e3a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.516+0000 7faffb400640 1 -- 172.21.15.63:0/1607604280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 msgr2=0x564c5e3a7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.516+0000 7faffb400640 1 --2- 172.21.15.63:0/1607604280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 0x564c5e3a7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.516+0000 7faffb400640 1 -- 172.21.15.63:0/1607604280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 msgr2=0x564c5e3a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.516+0000 7faffb400640 1 --2- 172.21.15.63:0/1607604280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3a7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.516+0000 7faffb400640 1 -- 172.21.15.63:0/1607604280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c5e2a18b0 con 0x564c5dfe44d0 2024-05-27T07:57:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.517+0000 7faffb400640 1 --2- 172.21.15.63:0/1607604280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3a83d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fafe8005940 tx=0x7fafe8031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7cb1137ea9813e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.519 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:59 smithi149 ceph-mon[35408]: pgmap v771: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:57:59.520 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:57:59 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:57:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.517+0000 7faff8c00640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fafe00b2aa0 0x7fafe00b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.517+0000 7faff8c00640 1 -- 172.21.15.63:0/1987553522 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7fafec0e2440 con 0x7faffc104250 2024-05-27T07:57:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.517+0000 7faff1200640 1 -- 172.21.15.63:0/1607604280 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8040070 con 0x564c5dfe44d0 2024-05-27T07:57:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.517+0000 7faff1200640 1 -- 172.21.15.63:0/1607604280 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafe8004020 con 0x564c5dfe44d0 2024-05-27T07:57:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.517+0000 7faff1200640 1 -- 172.21.15.63:0/1607604280 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8035040 con 0x564c5dfe44d0 2024-05-27T07:57:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.517+0000 7fb000c00640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fafe00b2aa0 0x7fafe00b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.518+0000 7fb009b8f740 1 -- 172.21.15.63:0/1607604280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 msgr2=0x564c5e3a83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.518+0000 7fb009b8f740 1 --2- 172.21.15.63:0/1607604280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3a83d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fafe8005940 tx=0x7fafe8031dc0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.518+0000 7fb000c00640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fafe00b2aa0 0x7fafe00b4fb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x564c5e3a9750 tx=0x7faff400e040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.519+0000 7fb009b8f740 1 -- 172.21.15.63:0/1607604280 shutdown_connections 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.519+0000 7fb009b8f740 1 --2- 172.21.15.63:0/1607604280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3a83d0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.519+0000 7fb009b8f740 1 --2- 172.21.15.63:0/1607604280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 0x564c5e3a7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.519+0000 7fb009b8f740 1 --2- 172.21.15.63:0/1607604280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.519+0000 7fb009b8f740 1 -- 172.21.15.63:0/1607604280 >> 172.21.15.63:0/1607604280 conn(0x564c5dff9300 msgr2=0x564c5e3a4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.519+0000 7fb009b8f740 1 -- 172.21.15.63:0/1607604280 shutdown_connections 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.519+0000 7fb009b8f740 1 -- 172.21.15.63:0/1607604280 wait complete. 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.519+0000 7fb009b8f740 1 -- start start 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.520+0000 7fb009b8f740 1 Processor -- start 2024-05-27T07:57:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.520+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 0x564c5e3b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.520+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3b6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.520+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3c2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.520+0000 7fb009b8f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564c5dfee790 con 0x564c5dfe44d0 2024-05-27T07:57:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.520+0000 7fb009b8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564c5e2b0bc0 con 0x564c5e0051b0 2024-05-27T07:57:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.520+0000 7fb009b8f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564c5e3b8f40 con 0x564c5dfdc2f0 2024-05-27T07:57:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.521+0000 7fb009b8f740 10 client.0 _ll_register_callbacks cb 0x564c5e28e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.521+0000 7faffb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.521+0000 7fb000c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.521+0000 7faffbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 0x564c5e3b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.521+0000 7fb000c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3c2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:59710/0 (socket says 172.21.15.63:59710) 2024-05-27T07:57:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.521+0000 7fb000c00640 1 -- 172.21.15.63:0/3093028792 learned_addr learned my addr 172.21.15.63:0/3093028792 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.522+0000 7faffbe00640 1 -- 172.21.15.63:0/3093028792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 msgr2=0x564c5e3c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.522+0000 7faffbe00640 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.522+0000 7faffbe00640 1 -- 172.21.15.63:0/3093028792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 msgr2=0x564c5e3b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.522+0000 7faffbe00640 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.522+0000 7faffbe00640 1 -- 172.21.15.63:0/3093028792 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c5e01d210 con 0x564c5dfdc2f0 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.522+0000 7fb000c00640 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.522+0000 7faffb400640 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.523+0000 7faffbe00640 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 0x564c5e3b6010 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fafec0df770 tx=0x7fafec0a52d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1313a6e4921580c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.523+0000 7fafe5600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.523+0000 7fafe5600640 1 -- 172.21.15.63:0/3093028792 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec021070 con 0x564c5dfdc2f0 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.523+0000 7fafe5600640 1 -- 172.21.15.63:0/3093028792 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7fafec0e2170 con 0x564c5dfdc2f0 2024-05-27T07:57:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.524+0000 7fb009b8f740 1 -- 172.21.15.63:0/3093028792 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe8004020 con 0x564c5dfdc2f0 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.524+0000 7fafe5600640 1 -- 172.21.15.63:0/3093028792 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fafec0dfc00 con 0x564c5dfdc2f0 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.524+0000 7fb009b8f740 10 client.29173 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.524+0000 7fb009b8f740 1 -- 172.21.15.63:0/3093028792 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564c5e0e9620 con 0x564c5dfdc2f0 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.524+0000 7fafe5600640 1 -- 172.21.15.63:0/3093028792 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafe8004020 con 0x564c5dfdc2f0 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.525+0000 7fafe5600640 1 -- 172.21.15.63:0/3093028792 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fafec0a4c70 con 0x564c5dfdc2f0 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.525+0000 7fb009b8f740 10 client.29173 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.525+0000 7fb009b8f740 10 client.29173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.525+0000 7fb009b8f740 20 client.29173 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.525+0000 7fb009b8f740 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564c5dfc3d80 0x564c5e3c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.525+0000 7fb009b8f740 4 client.29173 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.525+0000 7fb009b8f740 1 -- 172.21.15.63:0/3093028792 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c5e31aae0 con 0x564c5dfc3d80 2024-05-27T07:57:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.526+0000 7faffb400640 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564c5dfc3d80 0x564c5e3c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.526+0000 7faffb400640 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564c5dfc3d80 0x564c5e3c8160 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e72287737ba5af server_cookie=1fb9594c84469974 in_seq=0 out_seq=0 2024-05-27T07:57:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.527+0000 7fafe5600640 10 client.29173 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.528+0000 7fafe5600640 1 -- 172.21.15.63:0/3093028792 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564c5e31aae0 con 0x564c5dfc3d80 2024-05-27T07:57:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.528+0000 7fafe5600640 10 client.29173 handle_command_reply: tid=0 2024-05-27T07:57:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 client.29173 shutdown 2024-05-27T07:57:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 -- 172.21.15.63:0/3093028792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 msgr2=0x564c5e3b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 0x564c5e3b6010 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fafec0df770 tx=0x7fafec0a52d0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 -- 172.21.15.63:0/3093028792 shutdown_connections 2024-05-27T07:57:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564c5dfc3d80 0x564c5e3c8160 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fafe5600640 0 client.29173 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 --2- 172.21.15.63:0/3093028792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfdc2f0 0x564c5e3b6010 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 -- 172.21.15.63:0/3093028792 >> 172.21.15.63:0/3093028792 conn(0x564c5dff9300 msgr2=0x564c5e3a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 -- 172.21.15.63:0/3093028792 shutdown_connections 2024-05-27T07:57:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.529+0000 7fb009b8f740 1 -- 172.21.15.63:0/3093028792 wait complete. 2024-05-27T07:57:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.530+0000 7fb009b8f740 20 client.29173 trim_cache size 0 max 16384 2024-05-27T07:57:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.551+0000 7fb009b8f740 1 Processor -- start 2024-05-27T07:57:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.551+0000 7fb009b8f740 1 -- start start 2024-05-27T07:57:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.552+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5dfdc2f0 0x564c5e3a9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.552+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.552+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5e0051b0 0x564c5e3c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.552+0000 7fb009b8f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1b1dd0 con 0x564c5dfe44d0 2024-05-27T07:57:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.552+0000 7fb009b8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1af6a0 con 0x564c5dfdc2f0 2024-05-27T07:57:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.552+0000 7fb009b8f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1ad510 con 0x564c5e0051b0 2024-05-27T07:57:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.552+0000 7faffb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.553+0000 7faffb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54754/0 (socket says 172.21.15.63:54754) 2024-05-27T07:57:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.553+0000 7faffb400640 1 -- 172.21.15.63:0/3539991850 learned_addr learned my addr 172.21.15.63:0/3539991850 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.553+0000 7fb000c00640 1 --2- 172.21.15.63:0/3539991850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5e0051b0 0x564c5e3c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.553+0000 7faffbe00640 1 --2- 172.21.15.63:0/3539991850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5dfdc2f0 0x564c5e3a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.553+0000 7faffb400640 1 -- 172.21.15.63:0/3539991850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5e0051b0 msgr2=0x564c5e3c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.553+0000 7faffb400640 1 --2- 172.21.15.63:0/3539991850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5e0051b0 0x564c5e3c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.553+0000 7faffb400640 1 -- 172.21.15.63:0/3539991850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5dfdc2f0 msgr2=0x564c5e3a9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.553+0000 7faffb400640 1 --2- 172.21.15.63:0/3539991850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5dfdc2f0 0x564c5e3a9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.553+0000 7faffb400640 1 -- 172.21.15.63:0/3539991850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c5e3b1890 con 0x564c5dfe44d0 2024-05-27T07:57:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.553+0000 7faffbe00640 1 --2- 172.21.15.63:0/3539991850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5dfdc2f0 0x564c5e3a9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.554+0000 7faffb400640 1 --2- 172.21.15.63:0/3539991850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3c1fe0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fafe80057c0 tx=0x7fafe802aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43164f3aceff21d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.554+0000 7fafe7e00640 1 -- 172.21.15.63:0/3539991850 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8035040 con 0x564c5dfe44d0 2024-05-27T07:57:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.554+0000 7fafe7e00640 1 -- 172.21.15.63:0/3539991850 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafe8031600 con 0x564c5dfe44d0 2024-05-27T07:57:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.554+0000 7fafe7e00640 1 -- 172.21.15.63:0/3539991850 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8040040 con 0x564c5dfe44d0 2024-05-27T07:57:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.555+0000 7fb009b8f740 1 -- 172.21.15.63:0/3539991850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 msgr2=0x564c5e3c1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.555+0000 7fb009b8f740 1 --2- 172.21.15.63:0/3539991850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3c1fe0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fafe80057c0 tx=0x7fafe802aaf0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.555+0000 7fb009b8f740 1 -- 172.21.15.63:0/3539991850 shutdown_connections 2024-05-27T07:57:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.555+0000 7fb009b8f740 1 --2- 172.21.15.63:0/3539991850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5e0051b0 0x564c5e3c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.555+0000 7fb009b8f740 1 --2- 172.21.15.63:0/3539991850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfe44d0 0x564c5e3c1fe0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.555+0000 7fb009b8f740 1 --2- 172.21.15.63:0/3539991850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5dfdc2f0 0x564c5e3a9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.555+0000 7fb009b8f740 1 -- 172.21.15.63:0/3539991850 >> 172.21.15.63:0/3539991850 conn(0x564c5dff9300 msgr2=0x564c5e3b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.556+0000 7fb009b8f740 1 -- 172.21.15.63:0/3539991850 shutdown_connections 2024-05-27T07:57:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.556+0000 7fb009b8f740 1 -- 172.21.15.63:0/3539991850 wait complete. 2024-05-27T07:57:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.556+0000 7fb009b8f740 1 -- start start 2024-05-27T07:57:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.556+0000 7fb009b8f740 1 Processor -- start 2024-05-27T07:57:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.557+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfdc2f0 0x564c5e3bc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.557+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfe44d0 0x564c5e3bc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.557+0000 7fb009b8f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.557+0000 7fb009b8f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1adb60 con 0x564c5dfdc2f0 2024-05-27T07:57:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.557+0000 7fb009b8f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1aefd0 con 0x564c5e0051b0 2024-05-27T07:57:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.557+0000 7fb009b8f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faffc1ae230 con 0x564c5dfe44d0 2024-05-27T07:57:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.557+0000 7fb009b8f740 10 client.0 _ll_register_callbacks cb 0x7fafec010ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.558+0000 7faffbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfdc2f0 0x564c5e3bc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.558+0000 7faffbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfdc2f0 0x564c5e3bc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54770/0 (socket says 172.21.15.63:54770) 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.558+0000 7faffbe00640 1 -- 172.21.15.63:0/2261218479 learned_addr learned my addr 172.21.15.63:0/2261218479 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.558+0000 7fb000c00640 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.558+0000 7faffb400640 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfe44d0 0x564c5e3bc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.558+0000 7faffbe00640 1 -- 172.21.15.63:0/2261218479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfe44d0 msgr2=0x564c5e3bc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.558+0000 7faffbe00640 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfe44d0 0x564c5e3bc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.558+0000 7faffbe00640 1 -- 172.21.15.63:0/2261218479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 msgr2=0x564c5e3af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.558+0000 7faffbe00640 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.558+0000 7faffbe00640 1 -- 172.21.15.63:0/2261218479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c5e3bcfc0 con 0x564c5dfdc2f0 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.559+0000 7faffbe00640 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfdc2f0 0x564c5e3bc270 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fafec0a9000 tx=0x7fafec0a51c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0d7dda794df088 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.559+0000 7fafe5600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:57:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.559+0000 7fafe5600640 1 -- 172.21.15.63:0/2261218479 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec021070 con 0x564c5dfdc2f0 2024-05-27T07:57:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.560+0000 7fafe5600640 1 -- 172.21.15.63:0/2261218479 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9483+0+0 (secure 0 0 0) 0x7fafec0a8770 con 0x564c5dfdc2f0 2024-05-27T07:57:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.560+0000 7fb009b8f740 1 -- 172.21.15.63:0/2261218479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe8031600 con 0x564c5dfdc2f0 2024-05-27T07:57:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.560+0000 7fafe5600640 1 -- 172.21.15.63:0/2261218479 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fafec0a90d0 con 0x564c5dfdc2f0 2024-05-27T07:57:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.560+0000 7fb009b8f740 10 client.20124 fetch_fsmap learned FSMap version 33 2024-05-27T07:57:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.560+0000 7fb009b8f740 1 -- 172.21.15.63:0/2261218479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564c5e3b0d90 con 0x564c5dfdc2f0 2024-05-27T07:57:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.561+0000 7fafe5600640 1 -- 172.21.15.63:0/2261218479 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafe8031600 con 0x564c5dfdc2f0 2024-05-27T07:57:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.561+0000 7fafe5600640 1 -- 172.21.15.63:0/2261218479 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fafec0a7fd0 con 0x564c5dfdc2f0 2024-05-27T07:57:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.561+0000 7fb009b8f740 10 client.20124 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:57:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.561+0000 7fb009b8f740 10 client.20124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.561+0000 7fb009b8f740 20 client.20124 populate_metadata read hostname 'smithi063' 2024-05-27T07:57:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.561+0000 7fb009b8f740 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564c5df991c0 0x564c5e3b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.561+0000 7fb009b8f740 4 client.20124 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:57:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.561+0000 7fb009b8f740 1 -- 172.21.15.63:0/2261218479 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c5e3142c0 con 0x564c5df991c0 2024-05-27T07:57:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.561+0000 7faffb400640 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564c5df991c0 0x564c5e3b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.562+0000 7faffb400640 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564c5df991c0 0x564c5e3b1310 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b102e54aa62e29cb server_cookie=aa7bcd7b21f2129e in_seq=0 out_seq=0 2024-05-27T07:57:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.562+0000 7fafe5600640 10 client.20124 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fafe5600640 1 -- 172.21.15.63:0/2261218479 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564c5e3142c0 con 0x564c5df991c0 2024-05-27T07:57:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fafe5600640 10 client.20124 handle_command_reply: tid=0 2024-05-27T07:57:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fb009b8f740 1 client.20124 shutdown 2024-05-27T07:57:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fb009b8f740 1 -- 172.21.15.63:0/2261218479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfdc2f0 msgr2=0x564c5e3bc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fb009b8f740 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfdc2f0 0x564c5e3bc270 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fafec0a9000 tx=0x7fafec0a51c0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fb009b8f740 1 -- 172.21.15.63:0/2261218479 shutdown_connections 2024-05-27T07:57:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fb009b8f740 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564c5df991c0 0x564c5e3b1310 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fb009b8f740 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c5e0051b0 0x564c5e3af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fb009b8f740 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564c5dfe44d0 0x564c5e3bc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fb009b8f740 1 --2- 172.21.15.63:0/2261218479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564c5dfdc2f0 0x564c5e3bc270 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fb009b8f740 1 -- 172.21.15.63:0/2261218479 >> 172.21.15.63:0/2261218479 conn(0x564c5dff9300 msgr2=0x564c5e3b76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fafe5600640 0 client.20124 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:57:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.593+0000 7fb009b8f740 1 -- 172.21.15.63:0/2261218479 shutdown_connections 2024-05-27T07:57:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.594+0000 7fb009b8f740 1 -- 172.21.15.63:0/2261218479 wait complete. 2024-05-27T07:57:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.594+0000 7fb009b8f740 20 client.20124 trim_cache size 0 max 16384 2024-05-27T07:57:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:57:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:57:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:57:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.594+0000 7fb001600640 1 -- 172.21.15.63:0/1987553522 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fafe00b2aa0 msgr2=0x7fafe00b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.594+0000 7fb001600640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fafe00b2aa0 0x7fafe00b4fb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x564c5e3a9750 tx=0x7faff400e040 comp rx=0 tx=0).stop 2024-05-27T07:57:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.594+0000 7fb001600640 1 -- 172.21.15.63:0/1987553522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc104250 msgr2=0x7faffc19a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.594+0000 7fb001600640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc104250 0x7faffc19a7d0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fafec00d770 tx=0x7fafec00dc30 comp rx=0 tx=0).stop 2024-05-27T07:57:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.595+0000 7fb001600640 1 -- 172.21.15.63:0/1987553522 shutdown_connections 2024-05-27T07:57:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.595+0000 7fb001600640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fafe00b2aa0 0x7fafe00b4fb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.595+0000 7fb001600640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faffc19ef30 0x7faffc19f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.595+0000 7fb001600640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faffc109ab0 0x7faffc19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.595+0000 7fb001600640 1 --2- 172.21.15.63:0/1987553522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc104250 0x7faffc19a7d0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.595+0000 7fb001600640 1 -- 172.21.15.63:0/1987553522 >> 172.21.15.63:0/1987553522 conn(0x7faffc079240 msgr2=0x7faffc100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.595+0000 7fb001600640 1 -- 172.21.15.63:0/1987553522 shutdown_connections 2024-05-27T07:57:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:57:59.595+0000 7fb001600640 1 -- 172.21.15.63:0/1987553522 wait complete. 2024-05-27T07:57:59.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:59 smithi086 ceph-mon[34687]: pgmap v771: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T07:57:59.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:57:59 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:58:00.089 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:58:00.645 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:01.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:01 smithi063 ceph-mon[31613]: pgmap v772: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:58:01.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:01 smithi149 ceph-mon[35408]: pgmap v772: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:58:01.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:01 smithi086 ceph-mon[34687]: pgmap v772: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T07:58:03.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.148+0000 7f84d3400700 1 -- 172.21.15.63:0/3357381575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc1054b0 msgr2=0x7f84cc1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:03.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.148+0000 7f84d3400700 1 --2- 172.21.15.63:0/3357381575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc1054b0 0x7f84cc1058c0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009990 tx=0x7f84c0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.148+0000 7f84d3400700 1 -- 172.21.15.63:0/3357381575 shutdown_connections 2024-05-27T07:58:03.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.148+0000 7f84d3400700 1 --2- 172.21.15.63:0/3357381575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84cc10e8e0 0x7f84cc110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.148+0000 7f84d3400700 1 --2- 172.21.15.63:0/3357381575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc1054b0 0x7f84cc1058c0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.148+0000 7f84d3400700 1 --2- 172.21.15.63:0/3357381575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84cc105e00 0x7f84cc10e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.148+0000 7f84d3400700 1 -- 172.21.15.63:0/3357381575 >> 172.21.15.63:0/3357381575 conn(0x7f84cc0fedb0 msgr2=0x7f84cc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:03.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.148+0000 7f84d3400700 1 -- 172.21.15.63:0/3357381575 shutdown_connections 2024-05-27T07:58:03.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.148+0000 7f84d3400700 1 -- 172.21.15.63:0/3357381575 wait complete. 2024-05-27T07:58:03.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.150+0000 7f84d3400700 1 Processor -- start 2024-05-27T07:58:03.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.150+0000 7f84d3400700 1 -- start start 2024-05-27T07:58:03.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84d3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84cc1054b0 0x7f84cc1031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84d3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc105e00 0x7f84cc1036e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84d3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84cc10e8e0 0x7f84cc103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84d3400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1142e0 con 0x7f84cc105e00 2024-05-27T07:58:03.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84d3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc114160 con 0x7f84cc1054b0 2024-05-27T07:58:03.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84d3400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc114460 con 0x7f84cc10e8e0 2024-05-27T07:58:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84cbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc105e00 0x7f84cc1036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84cbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc105e00 0x7f84cc1036e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54778/0 (socket says 172.21.15.63:54778) 2024-05-27T07:58:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84cbe00700 1 -- 172.21.15.63:0/1567916660 learned_addr learned my addr 172.21.15.63:0/1567916660 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84d0a00700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84cc1054b0 0x7f84cc1031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.151+0000 7f84d1400700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84cc10e8e0 0x7f84cc103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.152+0000 7f84cbe00700 1 -- 172.21.15.63:0/1567916660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84cc10e8e0 msgr2=0x7f84cc103c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.152+0000 7f84cbe00700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84cc10e8e0 0x7f84cc103c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.152+0000 7f84cbe00700 1 -- 172.21.15.63:0/1567916660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84cc1054b0 msgr2=0x7f84cc1031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.152+0000 7f84cbe00700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84cc1054b0 0x7f84cc1031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.152+0000 7f84cbe00700 1 -- 172.21.15.63:0/1567916660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84cc1a9090 con 0x7f84cc105e00 2024-05-27T07:58:03.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.153+0000 7f84cbe00700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc105e00 0x7f84cc1036e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f84b800d950 tx=0x7f84b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:03.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.153+0000 7f84c9600700 1 -- 172.21.15.63:0/1567916660 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8016070 con 0x7f84cc105e00 2024-05-27T07:58:03.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.153+0000 7f84c9600700 1 -- 172.21.15.63:0/1567916660 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84b800fd20 con 0x7f84cc105e00 2024-05-27T07:58:03.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.153+0000 7f84d3400700 1 -- 172.21.15.63:0/1567916660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84cc1a9380 con 0x7f84cc105e00 2024-05-27T07:58:03.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.153+0000 7f84c9600700 1 -- 172.21.15.63:0/1567916660 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b800e3b0 con 0x7f84cc105e00 2024-05-27T07:58:03.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.153+0000 7f84d3400700 1 -- 172.21.15.63:0/1567916660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84cc1a98c0 con 0x7f84cc105e00 2024-05-27T07:58:03.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.158+0000 7f84c9600700 1 -- 172.21.15.63:0/1567916660 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f84b8019070 con 0x7f84cc105e00 2024-05-27T07:58:03.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.159+0000 7f84e1f9fb80 1 Processor -- start 2024-05-27T07:58:03.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.159+0000 7f84e1f9fb80 1 -- start start 2024-05-27T07:58:03.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.160+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53f91de40 0x55c53faa27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.160+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53fa9c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.160+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53fa9cdb0 0x55c53fa9d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.160+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c53f9cae40 con 0x55c53f91de40 2024-05-27T07:58:03.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.160+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c53f9ba9e0 con 0x55c53fa9cdb0 2024-05-27T07:58:03.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.160+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c53f9bea20 con 0x55c53faa2d30 2024-05-27T07:58:03.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.161+0000 7f84d1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53f91de40 0x55c53faa27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.161+0000 7f84d1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53f91de40 0x55c53faa27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54788/0 (socket says 172.21.15.63:54788) 2024-05-27T07:58:03.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.161+0000 7f84d1400700 1 -- 172.21.15.63:0/4081559815 learned_addr learned my addr 172.21.15.63:0/4081559815 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:03.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.161+0000 7f84c9600700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f84b40b29f0 0x7f84b40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.161+0000 7f84d0a00700 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53fa9c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.161+0000 7f84cbe00700 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53fa9cdb0 0x55c53fa9d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.161+0000 7f84c9600700 1 -- 172.21.15.63:0/1567916660 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9571+0+0 (secure 0 0 0) 0x7f84b80e2570 con 0x7f84cc105e00 2024-05-27T07:58:03.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.161+0000 7f84d1400700 1 -- 172.21.15.63:0/4081559815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 msgr2=0x55c53fa9c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:03.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.162+0000 7f84d1400700 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53fa9c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.162+0000 7f84d1400700 1 -- 172.21.15.63:0/4081559815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53fa9cdb0 msgr2=0x55c53fa9d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:03.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.162+0000 7f84d1400700 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53fa9cdb0 0x55c53fa9d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.162+0000 7f84d1400700 1 -- 172.21.15.63:0/4081559815 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c53f983420 con 0x55c53f91de40 2024-05-27T07:58:03.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.162+0000 7f84cbe00700 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53fa9cdb0 0x55c53fa9d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:03.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.162+0000 7f84d1400700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f84b40b29f0 0x7f84b40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.162+0000 7f84d0a00700 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53fa9c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:03.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.162+0000 7f84d1400700 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53f91de40 0x55c53faa27f0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f84c40099c0 tx=0x7f84c4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a1881eeb5b45da server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:03.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.163+0000 7f84be000700 1 -- 172.21.15.63:0/4081559815 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4018440 con 0x55c53f91de40 2024-05-27T07:58:03.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.163+0000 7f84be000700 1 -- 172.21.15.63:0/4081559815 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84c401c070 con 0x55c53f91de40 2024-05-27T07:58:03.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.163+0000 7f84be000700 1 -- 172.21.15.63:0/4081559815 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c40226c0 con 0x55c53f91de40 2024-05-27T07:58:03.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.165+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/4081559815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53f91de40 msgr2=0x55c53faa27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:03.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.165+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53f91de40 0x55c53faa27f0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f84c40099c0 tx=0x7f84c4009cd0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.165+0000 7f84d1400700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f84b40b29f0 0x7f84b40b4f00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f84c4021000 tx=0x7f84c401e000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:03.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.165+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/4081559815 shutdown_connections 2024-05-27T07:58:03.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.165+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53fa9cdb0 0x55c53fa9d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.165+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53f91de40 0x55c53faa27f0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.165+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/4081559815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53fa9c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.165+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/4081559815 >> 172.21.15.63:0/4081559815 conn(0x55c53f97dfc0 msgr2=0x55c53fa978d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:03.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.165+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/4081559815 shutdown_connections 2024-05-27T07:58:03.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.165+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/4081559815 wait complete. 2024-05-27T07:58:03.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.166+0000 7f84e1f9fb80 1 -- start start 2024-05-27T07:58:03.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.166+0000 7f84e1f9fb80 1 Processor -- start 2024-05-27T07:58:03.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53faa65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53fa9cdb0 0x55c53faa6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53faa2d30 0x55c53faa7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c53f8c0fa0 con 0x55c53faa2d30 2024-05-27T07:58:03.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c53f95b760 con 0x55c53f91de40 2024-05-27T07:58:03.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c53faa9800 con 0x55c53fa9cdb0 2024-05-27T07:58:03.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84e1f9fb80 10 client.0 _ll_register_callbacks cb 0x55c53f964c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:03.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84cbe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53fa9cdb0 0x55c53faa6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84d1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53faa2d30 0x55c53faa7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84cbe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53fa9cdb0 0x55c53faa6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:51254/0 (socket says 172.21.15.63:51254) 2024-05-27T07:58:03.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84cbe00700 1 -- 172.21.15.63:0/56978409 learned_addr learned my addr 172.21.15.63:0/56978409 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:03.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.167+0000 7f84d0a00700 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53faa65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.168+0000 7f84cbe00700 1 -- 172.21.15.63:0/56978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 msgr2=0x55c53faa65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:03.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.168+0000 7f84cbe00700 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53faa65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.168+0000 7f84cbe00700 1 -- 172.21.15.63:0/56978409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53faa2d30 msgr2=0x55c53faa7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:03.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.168+0000 7f84cbe00700 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53faa2d30 0x55c53faa7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.168+0000 7f84cbe00700 1 -- 172.21.15.63:0/56978409 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c53f90a8d0 con 0x55c53fa9cdb0 2024-05-27T07:58:03.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.168+0000 7f84cbe00700 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53fa9cdb0 0x55c53faa6ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f84b80a9360 tx=0x7f84b8016400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95a03eb84095f7a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:03.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.168+0000 7f84b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T07:58:03.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.168+0000 7f84b2a00700 1 -- 172.21.15.63:0/56978409 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8016210 con 0x55c53fa9cdb0 2024-05-27T07:58:03.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.169+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/56978409 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84c401c070 con 0x55c53fa9cdb0 2024-05-27T07:58:03.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.169+0000 7f84b2a00700 1 -- 172.21.15.63:0/56978409 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9571+0+0 (secure 0 0 0) 0x7f84b8015da0 con 0x55c53fa9cdb0 2024-05-27T07:58:03.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.169+0000 7f84b2a00700 1 -- 172.21.15.63:0/56978409 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f84b80a93d0 con 0x55c53fa9cdb0 2024-05-27T07:58:03.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.169+0000 7f84e1f9fb80 10 client.29191 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:03.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.169+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/56978409 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c53f98b060 con 0x55c53fa9cdb0 2024-05-27T07:58:03.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.170+0000 7f84b2a00700 1 -- 172.21.15.63:0/56978409 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84c401c070 con 0x55c53fa9cdb0 2024-05-27T07:58:03.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.170+0000 7f84b2a00700 1 -- 172.21.15.63:0/56978409 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f84b80a9630 con 0x55c53fa9cdb0 2024-05-27T07:58:03.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.170+0000 7f84e1f9fb80 10 client.29191 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:03.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.170+0000 7f84e1f9fb80 10 client.29191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:03.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.170+0000 7f84e1f9fb80 20 client.29191 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:03.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.170+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c53fab6540 0x55c53fab8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.170+0000 7f84e1f9fb80 4 client.29191 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:03.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.170+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/56978409 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c53f9ccd90 con 0x55c53fab6540 2024-05-27T07:58:03.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.171+0000 7f84d0a00700 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c53fab6540 0x55c53fab8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.172+0000 7f84d0a00700 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c53fab6540 0x55c53fab8960 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18383348c27d8899 server_cookie=b79d01f8eb67c682 in_seq=0 out_seq=0 2024-05-27T07:58:03.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.172+0000 7f84b2a00700 10 client.29191 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.174+0000 7f84b2a00700 1 -- 172.21.15.63:0/56978409 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c53f9ccd90 con 0x55c53fab6540 2024-05-27T07:58:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.174+0000 7f84b2a00700 10 client.29191 handle_command_reply: tid=0 2024-05-27T07:58:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.174+0000 7f84e1f9fb80 1 client.29191 shutdown 2024-05-27T07:58:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.174+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/56978409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53fa9cdb0 msgr2=0x55c53faa6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:03.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.174+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53fa9cdb0 0x55c53faa6ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f84b80a9360 tx=0x7f84b8016400 comp rx=0 tx=0).stop 2024-05-27T07:58:03.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.175+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/56978409 shutdown_connections 2024-05-27T07:58:03.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.175+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c53fab6540 0x55c53fab8960 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.175+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53faa2d30 0x55c53faa7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.175+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53faa65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.175+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/56978409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53fa9cdb0 0x55c53faa6ae0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.175+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/56978409 >> 172.21.15.63:0/56978409 conn(0x55c53f97dfc0 msgr2=0x55c53fa97580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:03.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.175+0000 7f84b2a00700 0 client.29191 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:03.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.175+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/56978409 shutdown_connections 2024-05-27T07:58:03.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.175+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/56978409 wait complete. 2024-05-27T07:58:03.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.175+0000 7f84e1f9fb80 20 client.29191 trim_cache size 0 max 16384 2024-05-27T07:58:03.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.194+0000 7f84e1f9fb80 1 Processor -- start 2024-05-27T07:58:03.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.194+0000 7f84e1f9fb80 1 -- start start 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.194+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53fa9c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.194+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53faa89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.194+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53faa0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.194+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b5100 con 0x55c53fa9cdb0 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.194+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b2990 con 0x55c53f91de40 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.194+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b0800 con 0x55c53faa2d30 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.194+0000 7f84cbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53faa89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.195+0000 7f84cbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53faa89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54802/0 (socket says 172.21.15.63:54802) 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.195+0000 7f84cbe00700 1 -- 172.21.15.63:0/2770592104 learned_addr learned my addr 172.21.15.63:0/2770592104 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.195+0000 7f84d0a00700 1 --2- 172.21.15.63:0/2770592104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53fa9c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.195+0000 7f84d1400700 1 --2- 172.21.15.63:0/2770592104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53faa0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.195+0000 7f84cbe00700 1 -- 172.21.15.63:0/2770592104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 msgr2=0x55c53faa0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:03.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.195+0000 7f84cbe00700 1 --2- 172.21.15.63:0/2770592104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53faa0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.195+0000 7f84cbe00700 1 -- 172.21.15.63:0/2770592104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 msgr2=0x55c53fa9c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:03.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.195+0000 7f84cbe00700 1 --2- 172.21.15.63:0/2770592104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53fa9c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.195+0000 7f84cbe00700 1 -- 172.21.15.63:0/2770592104 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c53fa9d1c0 con 0x55c53fa9cdb0 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.195+0000 7f84cbe00700 1 --2- 172.21.15.63:0/2770592104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53faa89f0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f84b8091110 tx=0x7f84b80108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645ec8a3841358f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.196+0000 7f84bd600700 1 -- 172.21.15.63:0/2770592104 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8016210 con 0x55c53fa9cdb0 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.196+0000 7f84bd600700 1 -- 172.21.15.63:0/2770592104 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84b80ad780 con 0x55c53fa9cdb0 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.196+0000 7f84bd600700 1 -- 172.21.15.63:0/2770592104 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80a9630 con 0x55c53fa9cdb0 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.197+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2770592104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 msgr2=0x55c53faa89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.197+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/2770592104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53faa89f0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f84b8091110 tx=0x7f84b80108b0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.197+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2770592104 shutdown_connections 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.197+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/2770592104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53faa0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.197+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/2770592104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53fa9c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.197+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/2770592104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53faa89f0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.197+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2770592104 >> 172.21.15.63:0/2770592104 conn(0x55c53f97dfc0 msgr2=0x55c53faa6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:03.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.197+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2770592104 shutdown_connections 2024-05-27T07:58:03.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.197+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2770592104 wait complete. 2024-05-27T07:58:03.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.198+0000 7f84e1f9fb80 1 -- start start 2024-05-27T07:58:03.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.198+0000 7f84e1f9fb80 1 Processor -- start 2024-05-27T07:58:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.198+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53fabddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.198+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53fabe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84e1f9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53fabe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b0e50 con 0x55c53fa9cdb0 2024-05-27T07:58:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b22c0 con 0x55c53f91de40 2024-05-27T07:58:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84e1f9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b1520 con 0x55c53faa2d30 2024-05-27T07:58:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84e1f9fb80 10 client.0 _ll_register_callbacks cb 0x55c53f964c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84cbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53fabe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84cbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53fabe300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54818/0 (socket says 172.21.15.63:54818) 2024-05-27T07:58:03.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84cbe00700 1 -- 172.21.15.63:0/2367120690 learned_addr learned my addr 172.21.15.63:0/2367120690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:03.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84d0a00700 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53fabddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84d1400700 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53fabe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84cbe00700 1 -- 172.21.15.63:0/2367120690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 msgr2=0x55c53fabe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84cbe00700 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53fabe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84cbe00700 1 -- 172.21.15.63:0/2367120690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 msgr2=0x55c53fabddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84cbe00700 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53fabddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.199+0000 7f84cbe00700 1 -- 172.21.15.63:0/2367120690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c53fb20080 con 0x55c53fa9cdb0 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.200+0000 7f84cbe00700 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53fabe300 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f84b8010fb0 tx=0x7f84b80b2230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c8398d6357741a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.200+0000 7f84b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.200+0000 7f84b2a00700 1 -- 172.21.15.63:0/2367120690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b80c8060 con 0x55c53fa9cdb0 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.200+0000 7f84b2a00700 1 -- 172.21.15.63:0/2367120690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9571+0+0 (secure 0 0 0) 0x7f84b800f890 con 0x55c53fa9cdb0 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.200+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2367120690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b80ad780 con 0x55c53fa9cdb0 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.200+0000 7f84b2a00700 1 -- 172.21.15.63:0/2367120690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f84b800fad0 con 0x55c53fa9cdb0 2024-05-27T07:58:03.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.200+0000 7f84e1f9fb80 10 client.20142 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:03.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.200+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2367120690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c53fb212e0 con 0x55c53fa9cdb0 2024-05-27T07:58:03.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.201+0000 7f84b2a00700 1 -- 172.21.15.63:0/2367120690 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84b808a080 con 0x55c53fa9cdb0 2024-05-27T07:58:03.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.201+0000 7f84b2a00700 1 -- 172.21.15.63:0/2367120690 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f84b80a4d40 con 0x55c53fa9cdb0 2024-05-27T07:58:03.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.201+0000 7f84e1f9fb80 10 client.20142 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:03.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.201+0000 7f84e1f9fb80 10 client.20142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:03.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.201+0000 7f84e1f9fb80 20 client.20142 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:03.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.201+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c53fb21750 0x55c53fb23b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:03.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.201+0000 7f84e1f9fb80 4 client.20142 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:58:03.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.201+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2367120690 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c53f9c6570 con 0x55c53fb21750 2024-05-27T07:58:03.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.202+0000 7f84d0a00700 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c53fb21750 0x55c53fb23b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:03.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.203+0000 7f84d0a00700 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c53fb21750 0x55c53fb23b70 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f042d1c9a51edc server_cookie=316697df475ffb14 in_seq=0 out_seq=0 2024-05-27T07:58:03.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.203+0000 7f84b2a00700 10 client.20142 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:03.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84b2a00700 1 -- 172.21.15.63:0/2367120690 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c53f9c6570 con 0x55c53fb21750 2024-05-27T07:58:03.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84b2a00700 10 client.20142 handle_command_reply: tid=0 2024-05-27T07:58:03.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 client.20142 shutdown 2024-05-27T07:58:03.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2367120690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 msgr2=0x55c53fabe300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:03.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53fabe300 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f84b8010fb0 tx=0x7f84b80b2230 comp rx=0 tx=0).stop 2024-05-27T07:58:03.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2367120690 shutdown_connections 2024-05-27T07:58:03.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c53fb21750 0x55c53fb23b70 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c53faa2d30 0x55c53fabe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c53f91de40 0x55c53fabddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 --2- 172.21.15.63:0/2367120690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c53fa9cdb0 0x55c53fabe300 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84b2a00700 0 client.20142 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:03.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2367120690 >> 172.21.15.63:0/2367120690 conn(0x55c53f97dfc0 msgr2=0x55c53faa6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:03.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2367120690 shutdown_connections 2024-05-27T07:58:03.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 1 -- 172.21.15.63:0/2367120690 wait complete. 2024-05-27T07:58:03.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.204+0000 7f84e1f9fb80 20 client.20142 trim_cache size 0 max 16384 2024-05-27T07:58:03.207 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:58:03.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 -- 172.21.15.63:0/1567916660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f84b40b29f0 msgr2=0x7f84b40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:03.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f84b40b29f0 0x7f84b40b4f00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f84c4021000 tx=0x7f84c401e000 comp rx=0 tx=0).stop 2024-05-27T07:58:03.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 -- 172.21.15.63:0/1567916660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc105e00 msgr2=0x7f84cc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:03.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc105e00 0x7f84cc1036e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f84b800d950 tx=0x7f84b800dc60 comp rx=0 tx=0).stop 2024-05-27T07:58:03.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 -- 172.21.15.63:0/1567916660 shutdown_connections 2024-05-27T07:58:03.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f84b40b29f0 0x7f84b40b4f00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84cc10e8e0 0x7f84cc103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84cc1054b0 0x7f84cc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 --2- 172.21.15.63:0/1567916660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc105e00 0x7f84cc1036e0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:03.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 -- 172.21.15.63:0/1567916660 >> 172.21.15.63:0/1567916660 conn(0x7f84cc0fedb0 msgr2=0x7f84cc0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:03.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.205+0000 7f84d3400700 1 -- 172.21.15.63:0/1567916660 shutdown_connections 2024-05-27T07:58:03.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:03.206+0000 7f84d3400700 1 -- 172.21.15.63:0/1567916660 wait complete. 2024-05-27T07:58:03.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:03 smithi063 ceph-mon[31613]: pgmap v773: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.4 MiB/s wr, 31 op/s 2024-05-27T07:58:03.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:03 smithi063 ceph-mon[31613]: osdmap e130: 12 total, 12 up, 12 in 2024-05-27T07:58:03.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:03 smithi086 ceph-mon[34687]: pgmap v773: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.4 MiB/s wr, 31 op/s 2024-05-27T07:58:03.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:03 smithi086 ceph-mon[34687]: osdmap e130: 12 total, 12 up, 12 in 2024-05-27T07:58:03.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:03 smithi149 ceph-mon[35408]: pgmap v773: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.4 MiB/s wr, 31 op/s 2024-05-27T07:58:03.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:03 smithi149 ceph-mon[35408]: osdmap e130: 12 total, 12 up, 12 in 2024-05-27T07:58:05.213 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:58:05.214 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:58:05.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:05 smithi149 ceph-mon[35408]: pgmap v775: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T07:58:05.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:05 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:58:05.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:05 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:58:05.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:05 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:58:05.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:05 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:58:05.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:05 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:58:05.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:05 smithi086 ceph-mon[34687]: pgmap v775: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T07:58:05.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:05 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:58:05.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:05 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:58:05.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:05 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:58:05.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:05 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:58:05.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:05 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:58:05.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:05 smithi063 ceph-mon[31613]: pgmap v775: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T07:58:05.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:05 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:58:05.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:05 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:58:05.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:05 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:58:05.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:05 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:58:05.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:05 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:58:05.756 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:07.649 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:07 smithi063 ceph-mon[31613]: pgmap v776: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T07:58:07.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:07 smithi086 ceph-mon[34687]: pgmap v776: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T07:58:07.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:07 smithi149 ceph-mon[35408]: pgmap v776: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T07:58:08.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.308+0000 7fb73ea00700 1 -- 172.21.15.63:0/229610735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb738105bd0 msgr2=0x7fb73810e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:08.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.308+0000 7fb73ea00700 1 --2- 172.21.15.63:0/229610735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb738105bd0 0x7fb73810e120 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fb724009990 tx=0x7fb724009ca0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.308+0000 7fb73ea00700 1 -- 172.21.15.63:0/229610735 shutdown_connections 2024-05-27T07:58:08.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.308+0000 7fb73ea00700 1 --2- 172.21.15.63:0/229610735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb73810e6f0 0x7fb738110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.308+0000 7fb73ea00700 1 --2- 172.21.15.63:0/229610735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb738105280 0x7fb738105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.308+0000 7fb73ea00700 1 --2- 172.21.15.63:0/229610735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb738105bd0 0x7fb73810e120 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.308+0000 7fb73ea00700 1 -- 172.21.15.63:0/229610735 >> 172.21.15.63:0/229610735 conn(0x7fb738078ab0 msgr2=0x7fb738078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:08.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.308+0000 7fb73ea00700 1 -- 172.21.15.63:0/229610735 shutdown_connections 2024-05-27T07:58:08.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.309+0000 7fb73ea00700 1 -- 172.21.15.63:0/229610735 wait complete. 2024-05-27T07:58:08.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.310+0000 7fb73ea00700 1 Processor -- start 2024-05-27T07:58:08.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.311+0000 7fb73ea00700 1 -- start start 2024-05-27T07:58:08.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.311+0000 7fb73ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb738105280 0x7fb73819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.311+0000 7fb73ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb738105bd0 0x7fb73819d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.311+0000 7fb73ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb73810e6f0 0x7fb7381a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73ea00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb738114150 con 0x7fb738105280 2024-05-27T07:58:08.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73ea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb738113fd0 con 0x7fb738105bd0 2024-05-27T07:58:08.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73ea00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381142d0 con 0x7fb73810e6f0 2024-05-27T07:58:08.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb738105280 0x7fb73819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb738105280 0x7fb73819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54824/0 (socket says 172.21.15.63:54824) 2024-05-27T07:58:08.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73d600700 1 -- 172.21.15.63:0/3481257360 learned_addr learned my addr 172.21.15.63:0/3481257360 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:08.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73cc00700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb738105bd0 0x7fb73819d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73e000700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb73810e6f0 0x7fb7381a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73d600700 1 -- 172.21.15.63:0/3481257360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb73810e6f0 msgr2=0x7fb7381a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:08.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73d600700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb73810e6f0 0x7fb7381a15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73d600700 1 -- 172.21.15.63:0/3481257360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb738105bd0 msgr2=0x7fb73819d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:08.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73d600700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb738105bd0 0x7fb73819d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.312+0000 7fb73d600700 1 -- 172.21.15.63:0/3481257360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7381a1c90 con 0x7fb738105280 2024-05-27T07:58:08.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.313+0000 7fb73cc00700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb738105bd0 0x7fb73819d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:08.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.313+0000 7fb73d600700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb738105280 0x7fb73819cdc0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00d950 tx=0x7fb72c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:08.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.313+0000 7fb736000700 1 -- 172.21.15.63:0/3481257360 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c016070 con 0x7fb738105280 2024-05-27T07:58:08.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.313+0000 7fb73ea00700 1 -- 172.21.15.63:0/3481257360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7381a1f80 con 0x7fb738105280 2024-05-27T07:58:08.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.313+0000 7fb736000700 1 -- 172.21.15.63:0/3481257360 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb72c010460 con 0x7fb738105280 2024-05-27T07:58:08.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.313+0000 7fb736000700 1 -- 172.21.15.63:0/3481257360 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00f620 con 0x7fb738105280 2024-05-27T07:58:08.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.314+0000 7fb73ea00700 1 -- 172.21.15.63:0/3481257360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7381a98a0 con 0x7fb738105280 2024-05-27T07:58:08.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.319+0000 7fb736000700 1 -- 172.21.15.63:0/3481257360 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb72c019070 con 0x7fb738105280 2024-05-27T07:58:08.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.319+0000 7fb74eb25b80 1 Processor -- start 2024-05-27T07:58:08.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.320+0000 7fb74eb25b80 1 -- start start 2024-05-27T07:58:08.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.320+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.320+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 0x55d2675bdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.320+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 0x55d2675c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.320+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d2674eaec0 con 0x55d267529a70 2024-05-27T07:58:08.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.320+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2674daa60 con 0x55d26743dec0 2024-05-27T07:58:08.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.320+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d2674deaa0 con 0x55d2675c7340 2024-05-27T07:58:08.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.321+0000 7fb736000700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7200b29f0 0x7fb7200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.321+0000 7fb73e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.321+0000 7fb73e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675bd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54840/0 (socket says 172.21.15.63:54840) 2024-05-27T07:58:08.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.321+0000 7fb73e000700 1 -- 172.21.15.63:0/1221636335 learned_addr learned my addr 172.21.15.63:0/1221636335 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:08.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.321+0000 7fb73cc00700 1 --2- 172.21.15.63:0/1221636335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 0x55d2675c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.321+0000 7fb73d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 0x55d2675bdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.321+0000 7fb736000700 1 -- 172.21.15.63:0/3481257360 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9571+0+0 (secure 0 0 0) 0x7fb72c0e2680 con 0x7fb738105280 2024-05-27T07:58:08.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.321+0000 7fb73e000700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7200b29f0 0x7fb7200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.322+0000 7fb73cc00700 1 -- 172.21.15.63:0/1221636335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 msgr2=0x55d2675bdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.322+0000 7fb73cc00700 1 --2- 172.21.15.63:0/1221636335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 0x55d2675bdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.322+0000 7fb73cc00700 1 -- 172.21.15.63:0/1221636335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 msgr2=0x55d2675bd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.322+0000 7fb73cc00700 1 --2- 172.21.15.63:0/1221636335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675bd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.322+0000 7fb73cc00700 1 -- 172.21.15.63:0/1221636335 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2674a34a0 con 0x55d2675c7340 2024-05-27T07:58:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.322+0000 7fb73e000700 1 --2- 172.21.15.63:0/1221636335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675bd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:58:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.322+0000 7fb73cc00700 1 --2- 172.21.15.63:0/1221636335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 0x55d2675c9760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb738102530 tx=0x7fb7240096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc494cc12f7b6be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.322+0000 7fb73e000700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7200b29f0 0x7fb7200b4f00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55d2675b9b10 tx=0x7fb730004b40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.323+0000 7fb72aa00700 1 -- 172.21.15.63:0/1221636335 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7240037c0 con 0x55d2675c7340 2024-05-27T07:58:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.323+0000 7fb72aa00700 1 -- 172.21.15.63:0/1221636335 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb724003df0 con 0x55d2675c7340 2024-05-27T07:58:08.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.323+0000 7fb72aa00700 1 -- 172.21.15.63:0/1221636335 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72400ba30 con 0x55d2675c7340 2024-05-27T07:58:08.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.324+0000 7fb74eb25b80 1 -- 172.21.15.63:0/1221636335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 msgr2=0x55d2675c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:08.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.324+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/1221636335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 0x55d2675c9760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb738102530 tx=0x7fb7240096a0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.324+0000 7fb74eb25b80 1 -- 172.21.15.63:0/1221636335 shutdown_connections 2024-05-27T07:58:08.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.324+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/1221636335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 0x55d2675c9760 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.324+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/1221636335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675bd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.324+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/1221636335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 0x55d2675bdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.324+0000 7fb74eb25b80 1 -- 172.21.15.63:0/1221636335 >> 172.21.15.63:0/1221636335 conn(0x55d2674d7410 msgr2=0x55d2675b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:08.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.325+0000 7fb74eb25b80 1 -- 172.21.15.63:0/1221636335 shutdown_connections 2024-05-27T07:58:08.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.325+0000 7fb74eb25b80 1 -- 172.21.15.63:0/1221636335 wait complete. 2024-05-27T07:58:08.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.325+0000 7fb74eb25b80 1 -- start start 2024-05-27T07:58:08.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.326+0000 7fb74eb25b80 1 Processor -- start 2024-05-27T07:58:08.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.326+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 0x55d2675c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.326+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d267529a70 0x55d2675c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.326+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2675c7340 0x55d2675c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.326+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d2673e1020 con 0x55d2675c7340 2024-05-27T07:58:08.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.326+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d26747b7e0 con 0x55d26743dec0 2024-05-27T07:58:08.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.326+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d2675c9dc0 con 0x55d267529a70 2024-05-27T07:58:08.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.326+0000 7fb74eb25b80 10 client.0 _ll_register_callbacks cb 0x55d267484cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:08.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2675c7340 0x55d2675c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2675c7340 0x55d2675c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54848/0 (socket says 172.21.15.63:54848) 2024-05-27T07:58:08.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73e000700 1 -- 172.21.15.63:0/3773621027 learned_addr learned my addr 172.21.15.63:0/3773621027 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:08.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73cc00700 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d267529a70 0x55d2675c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73d600700 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 0x55d2675c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73e000700 1 -- 172.21.15.63:0/3773621027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d267529a70 msgr2=0x55d2675c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:08.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73e000700 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d267529a70 0x55d2675c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73e000700 1 -- 172.21.15.63:0/3773621027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 msgr2=0x55d2675c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:08.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73e000700 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 0x55d2675c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73e000700 1 -- 172.21.15.63:0/3773621027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d26742a950 con 0x55d2675c7340 2024-05-27T07:58:08.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.327+0000 7fb73d600700 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 0x55d2675c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:08.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.328+0000 7fb73e000700 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2675c7340 0x55d2675c4c90 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fb730015ee0 tx=0x7fb730004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb98c693e7caf6fe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:08.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.328+0000 7fb71f400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:08.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.328+0000 7fb71f400700 1 -- 172.21.15.63:0/3773621027 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730012050 con 0x55d2675c7340 2024-05-27T07:58:08.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.328+0000 7fb71f400700 1 -- 172.21.15.63:0/3773621027 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9571+0+0 (secure 0 0 0) 0x7fb730004bb0 con 0x55d2675c7340 2024-05-27T07:58:08.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.328+0000 7fb74eb25b80 1 -- 172.21.15.63:0/3773621027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb724003df0 con 0x55d2675c7340 2024-05-27T07:58:08.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.329+0000 7fb71f400700 1 -- 172.21.15.63:0/3773621027 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb73002e050 con 0x55d2675c7340 2024-05-27T07:58:08.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.329+0000 7fb74eb25b80 10 client.20154 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:08.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.329+0000 7fb74eb25b80 1 -- 172.21.15.63:0/3773621027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d2674ab0e0 con 0x55d2675c7340 2024-05-27T07:58:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.329+0000 7fb71f400700 1 -- 172.21.15.63:0/3773621027 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb724003df0 con 0x55d2675c7340 2024-05-27T07:58:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.330+0000 7fb71f400700 1 -- 172.21.15.63:0/3773621027 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb7300041b0 con 0x55d2675c7340 2024-05-27T07:58:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.330+0000 7fb74eb25b80 10 client.20154 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.330+0000 7fb74eb25b80 10 client.20154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.330+0000 7fb74eb25b80 20 client.20154 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.330+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2675d6ce0 0x55d2675d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.330+0000 7fb74eb25b80 4 client.20154 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:08.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.330+0000 7fb74eb25b80 1 -- 172.21.15.63:0/3773621027 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2674ece10 con 0x55d2675d6ce0 2024-05-27T07:58:08.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.331+0000 7fb73d600700 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2675d6ce0 0x55d2675d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.332+0000 7fb73d600700 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2675d6ce0 0x55d2675d9160 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70771ac1f520bdaf server_cookie=34d0e896ab68f87a in_seq=0 out_seq=0 2024-05-27T07:58:08.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.332+0000 7fb71f400700 10 client.20154 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:08.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.333+0000 7fb71f400700 1 -- 172.21.15.63:0/3773621027 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d2674ece10 con 0x55d2675d6ce0 2024-05-27T07:58:08.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.333+0000 7fb71f400700 10 client.20154 handle_command_reply: tid=0 2024-05-27T07:58:08.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 client.20154 shutdown 2024-05-27T07:58:08.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 -- 172.21.15.63:0/3773621027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2675c7340 msgr2=0x55d2675c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:08.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2675c7340 0x55d2675c4c90 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fb730015ee0 tx=0x7fb730004ad0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 -- 172.21.15.63:0/3773621027 shutdown_connections 2024-05-27T07:58:08.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2675d6ce0 0x55d2675d9160 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d2675c7340 0x55d2675c4c90 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d26743dec0 0x55d2675c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/3773621027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d267529a70 0x55d2675c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb71f400700 0 client.20154 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:08.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 -- 172.21.15.63:0/3773621027 >> 172.21.15.63:0/3773621027 conn(0x55d2674d7410 msgr2=0x55d2675b82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:08.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 -- 172.21.15.63:0/3773621027 shutdown_connections 2024-05-27T07:58:08.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.334+0000 7fb74eb25b80 1 -- 172.21.15.63:0/3773621027 wait complete. 2024-05-27T07:58:08.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.335+0000 7fb74eb25b80 20 client.20154 trim_cache size 0 max 16384 2024-05-27T07:58:08.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.365+0000 7fb74eb25b80 1 Processor -- start 2024-05-27T07:58:08.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.365+0000 7fb74eb25b80 1 -- start start 2024-05-27T07:58:08.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.365+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d26743dec0 0x55d2675bcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.365+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.365+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2675c7340 0x55d2675de920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.366+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381b4ee0 con 0x55d267529a70 2024-05-27T07:58:08.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.366+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381b2770 con 0x55d2675c7340 2024-05-27T07:58:08.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.366+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381b05e0 con 0x55d26743dec0 2024-05-27T07:58:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.366+0000 7fb73cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.366+0000 7fb73cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675d2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54858/0 (socket says 172.21.15.63:54858) 2024-05-27T07:58:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.366+0000 7fb73cc00700 1 -- 172.21.15.63:0/1463448303 learned_addr learned my addr 172.21.15.63:0/1463448303 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.366+0000 7fb73d600700 1 --2- 172.21.15.63:0/1463448303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d26743dec0 0x55d2675bcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.366+0000 7fb73e000700 1 --2- 172.21.15.63:0/1463448303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2675c7340 0x55d2675de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.366+0000 7fb73cc00700 1 -- 172.21.15.63:0/1463448303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d26743dec0 msgr2=0x55d2675bcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.366+0000 7fb73cc00700 1 --2- 172.21.15.63:0/1463448303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d26743dec0 0x55d2675bcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.367+0000 7fb73cc00700 1 -- 172.21.15.63:0/1463448303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2675c7340 msgr2=0x55d2675de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.367+0000 7fb73cc00700 1 --2- 172.21.15.63:0/1463448303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2675c7340 0x55d2675de920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.367+0000 7fb73cc00700 1 -- 172.21.15.63:0/1463448303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2675c2780 con 0x55d267529a70 2024-05-27T07:58:08.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.367+0000 7fb73e000700 1 --2- 172.21.15.63:0/1463448303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2675c7340 0x55d2675de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:08.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.367+0000 7fb73cc00700 1 --2- 172.21.15.63:0/1463448303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675d2e60 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fb724000c00 tx=0x7fb724003950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3c393f9ab7cf4f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:08.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.367+0000 7fb72a000700 1 -- 172.21.15.63:0/1463448303 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724003ac0 con 0x55d267529a70 2024-05-27T07:58:08.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.367+0000 7fb72a000700 1 -- 172.21.15.63:0/1463448303 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb724003c60 con 0x55d267529a70 2024-05-27T07:58:08.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.367+0000 7fb72a000700 1 -- 172.21.15.63:0/1463448303 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72400ba50 con 0x55d267529a70 2024-05-27T07:58:08.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.368+0000 7fb74eb25b80 1 -- 172.21.15.63:0/1463448303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 msgr2=0x55d2675d2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:08.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.368+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/1463448303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675d2e60 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fb724000c00 tx=0x7fb724003950 comp rx=0 tx=0).stop 2024-05-27T07:58:08.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.369+0000 7fb74eb25b80 1 -- 172.21.15.63:0/1463448303 shutdown_connections 2024-05-27T07:58:08.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.369+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/1463448303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2675c7340 0x55d2675de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.369+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/1463448303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d26743dec0 0x55d2675bcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.369+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/1463448303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d267529a70 0x55d2675d2e60 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.369+0000 7fb74eb25b80 1 -- 172.21.15.63:0/1463448303 >> 172.21.15.63:0/1463448303 conn(0x55d2674d7410 msgr2=0x55d2675bde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:08.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.369+0000 7fb74eb25b80 1 -- 172.21.15.63:0/1463448303 shutdown_connections 2024-05-27T07:58:08.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.369+0000 7fb74eb25b80 1 -- 172.21.15.63:0/1463448303 wait complete. 2024-05-27T07:58:08.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.370+0000 7fb74eb25b80 1 -- start start 2024-05-27T07:58:08.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.370+0000 7fb74eb25b80 1 Processor -- start 2024-05-27T07:58:08.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.370+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d26743dec0 0x55d2675cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.370+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d267529a70 0x55d2675cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.371+0000 7fb74eb25b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 0x55d2675cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.371+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381b0c30 con 0x55d26743dec0 2024-05-27T07:58:08.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.371+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381b20a0 con 0x55d267529a70 2024-05-27T07:58:08.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.371+0000 7fb74eb25b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381b1300 con 0x55d2675c7340 2024-05-27T07:58:08.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.371+0000 7fb74eb25b80 10 client.0 _ll_register_callbacks cb 0x55d267484cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:08.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.371+0000 7fb73d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d26743dec0 0x55d2675cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.371+0000 7fb73d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d26743dec0 0x55d2675cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54872/0 (socket says 172.21.15.63:54872) 2024-05-27T07:58:08.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.371+0000 7fb73d600700 1 -- 172.21.15.63:0/56040381 learned_addr learned my addr 172.21.15.63:0/56040381 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:08.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.371+0000 7fb73cc00700 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d267529a70 0x55d2675cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.371+0000 7fb73e000700 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 0x55d2675cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb73d600700 1 -- 172.21.15.63:0/56040381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 msgr2=0x55d2675cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:08.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb73d600700 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 0x55d2675cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb73d600700 1 -- 172.21.15.63:0/56040381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d267529a70 msgr2=0x55d2675cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:08.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb73d600700 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d267529a70 0x55d2675cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb73d600700 1 -- 172.21.15.63:0/56040381 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2675cbca0 con 0x55d26743dec0 2024-05-27T07:58:08.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb73cc00700 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d267529a70 0x55d2675cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:08.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb73d600700 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d26743dec0 0x55d2675cf090 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0a67b0 tx=0x7fb72c056740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4113522cb2a3cfb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:08.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb71f400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:08.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb71f400700 1 -- 172.21.15.63:0/56040381 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0e2040 con 0x55d26743dec0 2024-05-27T07:58:08.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb71f400700 1 -- 172.21.15.63:0/56040381 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9571+0+0 (secure 0 0 0) 0x7fb72c0a97a0 con 0x55d26743dec0 2024-05-27T07:58:08.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.372+0000 7fb74eb25b80 1 -- 172.21.15.63:0/56040381 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb724003c60 con 0x55d26743dec0 2024-05-27T07:58:08.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb71f400700 1 -- 172.21.15.63:0/56040381 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb72c020040 con 0x55d26743dec0 2024-05-27T07:58:08.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb74eb25b80 10 client.20160 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:08.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb74eb25b80 1 -- 172.21.15.63:0/56040381 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d2676421a0 con 0x55d26743dec0 2024-05-27T07:58:08.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb71f400700 1 -- 172.21.15.63:0/56040381 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb724003c60 con 0x55d26743dec0 2024-05-27T07:58:08.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb71f400700 1 -- 172.21.15.63:0/56040381 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb72c0b1fa0 con 0x55d26743dec0 2024-05-27T07:58:08.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb74eb25b80 10 client.20160 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:08.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb74eb25b80 10 client.20160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:08.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb74eb25b80 20 client.20160 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:08.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2676428b0 0x55d267644cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:08.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb74eb25b80 4 client.20160 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:58:08.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.373+0000 7fb74eb25b80 1 -- 172.21.15.63:0/56040381 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d2674e65f0 con 0x55d2676428b0 2024-05-27T07:58:08.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.374+0000 7fb73cc00700 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2676428b0 0x55d267644cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:08.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.375+0000 7fb73cc00700 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2676428b0 0x55d267644cd0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785a0f89dffddf96 server_cookie=477d68aed6d2ddde in_seq=0 out_seq=0 2024-05-27T07:58:08.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.375+0000 7fb71f400700 10 client.20160 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:08.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb71f400700 1 -- 172.21.15.63:0/56040381 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d2674e65f0 con 0x55d2676428b0 2024-05-27T07:58:08.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb71f400700 10 client.20160 handle_command_reply: tid=0 2024-05-27T07:58:08.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb74eb25b80 1 client.20160 shutdown 2024-05-27T07:58:08.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb74eb25b80 1 -- 172.21.15.63:0/56040381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d26743dec0 msgr2=0x55d2675cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d26743dec0 0x55d2675cf090 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0a67b0 tx=0x7fb72c056740 comp rx=0 tx=0).stop 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb74eb25b80 1 -- 172.21.15.63:0/56040381 shutdown_connections 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d2676428b0 0x55d267644cd0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2675c7340 0x55d2675cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d26743dec0 0x55d2675cf090 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb74eb25b80 1 --2- 172.21.15.63:0/56040381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d267529a70 0x55d2675cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb74eb25b80 1 -- 172.21.15.63:0/56040381 >> 172.21.15.63:0/56040381 conn(0x55d2674d7410 msgr2=0x55d2675c3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.378+0000 7fb71f400700 0 client.20160 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.379+0000 7fb74eb25b80 1 -- 172.21.15.63:0/56040381 shutdown_connections 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.379+0000 7fb74eb25b80 1 -- 172.21.15.63:0/56040381 wait complete. 2024-05-27T07:58:08.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.379+0000 7fb74eb25b80 20 client.20160 trim_cache size 0 max 16384 2024-05-27T07:58:08.381 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:58:08.381 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:58:08.381 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "f2ce2ff7-4c09-43e8-8242-9459922c7545", 2024-05-27T07:58:08.381 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:58:08.381 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:58:08.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 -- 172.21.15.63:0/3481257360 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7200b29f0 msgr2=0x7fb7200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:08.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7200b29f0 0x7fb7200b4f00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55d2675b9b10 tx=0x7fb730004b40 comp rx=0 tx=0).stop 2024-05-27T07:58:08.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 -- 172.21.15.63:0/3481257360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb738105280 msgr2=0x7fb73819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:08.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb738105280 0x7fb73819cdc0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00d950 tx=0x7fb72c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:58:08.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 -- 172.21.15.63:0/3481257360 shutdown_connections 2024-05-27T07:58:08.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb7200b29f0 0x7fb7200b4f00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb73810e6f0 0x7fb7381a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb738105280 0x7fb73819cdc0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 --2- 172.21.15.63:0/3481257360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb738105bd0 0x7fb73819d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:08.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 -- 172.21.15.63:0/3481257360 >> 172.21.15.63:0/3481257360 conn(0x7fb738078ab0 msgr2=0x7fb738102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:08.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 -- 172.21.15.63:0/3481257360 shutdown_connections 2024-05-27T07:58:08.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:08.380+0000 7fb73ea00700 1 -- 172.21.15.63:0/3481257360 wait complete. 2024-05-27T07:58:09.220 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:58:09.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: pgmap v777: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.3 MiB/s wr, 34 op/s 2024-05-27T07:58:09.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:58:09.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.703 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:09.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: pgmap v777: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.3 MiB/s wr, 34 op/s 2024-05-27T07:58:09.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:58:09.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:58:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: pgmap v777: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.3 MiB/s wr, 34 op/s 2024-05-27T07:58:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:58:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:58:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:11.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:11 smithi063 ceph-mon[31613]: pgmap v778: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.3 MiB/s wr, 34 op/s 2024-05-27T07:58:11.430 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:11 smithi063 ceph-mon[31613]: osdmap e131: 12 total, 12 up, 12 in 2024-05-27T07:58:11.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:11 smithi086 ceph-mon[34687]: pgmap v778: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.3 MiB/s wr, 34 op/s 2024-05-27T07:58:11.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:11 smithi086 ceph-mon[34687]: osdmap e131: 12 total, 12 up, 12 in 2024-05-27T07:58:11.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:11 smithi149 ceph-mon[35408]: pgmap v778: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.3 MiB/s wr, 34 op/s 2024-05-27T07:58:11.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:11 smithi149 ceph-mon[35408]: osdmap e131: 12 total, 12 up, 12 in 2024-05-27T07:58:12.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.061+0000 7f97a7400700 1 -- 172.21.15.63:0/2191853154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97a0108730 msgr2=0x7f97a0110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.061+0000 7f97a7400700 1 --2- 172.21.15.63:0/2191853154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97a0108730 0x7f97a0110090 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f979800b0d0 tx=0x7f979800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.061+0000 7f97a7400700 1 -- 172.21.15.63:0/2191853154 shutdown_connections 2024-05-27T07:58:12.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.061+0000 7f97a7400700 1 --2- 172.21.15.63:0/2191853154 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97a0108730 0x7f97a0110090 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.061+0000 7f97a7400700 1 --2- 172.21.15.63:0/2191853154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a01034b0 0x7f97a01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.061+0000 7f97a7400700 1 --2- 172.21.15.63:0/2191853154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97a0107de0 0x7f97a01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.061+0000 7f97a7400700 1 -- 172.21.15.63:0/2191853154 >> 172.21.15.63:0/2191853154 conn(0x7f97a00fedb0 msgr2=0x7f97a01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.061+0000 7f97a7400700 1 -- 172.21.15.63:0/2191853154 shutdown_connections 2024-05-27T07:58:12.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.062+0000 7f97a7400700 1 -- 172.21.15.63:0/2191853154 wait complete. 2024-05-27T07:58:12.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.063+0000 7f97a7400700 1 Processor -- start 2024-05-27T07:58:12.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.063+0000 7f97a7400700 1 -- start start 2024-05-27T07:58:12.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.064+0000 7f97a7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a01034b0 0x7f97a019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.064+0000 7f97a7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97a0107de0 0x7f97a019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.064+0000 7f97a7400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97a0108730 0x7f97a01a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.064+0000 7f97a7400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0113210 con 0x7f97a0107de0 2024-05-27T07:58:12.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.064+0000 7f97a7400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0113090 con 0x7f97a01034b0 2024-05-27T07:58:12.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.064+0000 7f97a7400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0113390 con 0x7f97a0108730 2024-05-27T07:58:12.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.064+0000 7f979fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97a0107de0 0x7f97a019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.065+0000 7f979fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97a0107de0 0x7f97a019d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49616/0 (socket says 172.21.15.63:49616) 2024-05-27T07:58:12.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.065+0000 7f979fe00700 1 -- 172.21.15.63:0/1737937683 learned_addr learned my addr 172.21.15.63:0/1737937683 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:12.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.065+0000 7f97a5400700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97a0108730 0x7f97a01a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.065+0000 7f97a4a00700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a01034b0 0x7f97a019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.065+0000 7f979fe00700 1 -- 172.21.15.63:0/1737937683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97a0108730 msgr2=0x7f97a01a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.065+0000 7f979fe00700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97a0108730 0x7f97a01a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.065+0000 7f979fe00700 1 -- 172.21.15.63:0/1737937683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a01034b0 msgr2=0x7f97a019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.065+0000 7f979fe00700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a01034b0 0x7f97a019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.065+0000 7f979fe00700 1 -- 172.21.15.63:0/1737937683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a01a1de0 con 0x7f97a0107de0 2024-05-27T07:58:12.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.065+0000 7f97a4a00700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a01034b0 0x7f97a019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:12.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.066+0000 7f979fe00700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97a0107de0 0x7f97a019d4b0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f978c00d980 tx=0x7f978c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.066+0000 7f979d600700 1 -- 172.21.15.63:0/1737937683 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c017070 con 0x7f97a0107de0 2024-05-27T07:58:12.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.066+0000 7f979d600700 1 -- 172.21.15.63:0/1737937683 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f978c010460 con 0x7f97a0107de0 2024-05-27T07:58:12.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.066+0000 7f97a7400700 1 -- 172.21.15.63:0/1737937683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a01a20d0 con 0x7f97a0107de0 2024-05-27T07:58:12.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.066+0000 7f979d600700 1 -- 172.21.15.63:0/1737937683 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c00de60 con 0x7f97a0107de0 2024-05-27T07:58:12.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.066+0000 7f97a7400700 1 -- 172.21.15.63:0/1737937683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a01a99f0 con 0x7f97a0107de0 2024-05-27T07:58:12.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.070+0000 7f979d600700 1 -- 172.21.15.63:0/1737937683 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f978c010610 con 0x7f97a0107de0 2024-05-27T07:58:12.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.072+0000 7f979d600700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f97880b29f0 0x7f97880b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.072+0000 7f979d600700 1 -- 172.21.15.63:0/1737937683 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f978c0e2520 con 0x7f97a0107de0 2024-05-27T07:58:12.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.072+0000 7f97a4a00700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f97880b29f0 0x7f97880b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.073+0000 7f97b5f96b80 1 Processor -- start 2024-05-27T07:58:12.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.073+0000 7f97b5f96b80 1 -- start start 2024-05-27T07:58:12.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.073+0000 7f97a4a00700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f97880b29f0 0x7f97880b4f00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f979400bc10 tx=0x7f979400b5f0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854e6ce40 0x556854ff17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 0x556854feb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854febdb0 0x556854fec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556854f19e40 con 0x556854febdb0 2024-05-27T07:58:12.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556854f099e0 con 0x556854ff1d30 2024-05-27T07:58:12.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556854f0da20 con 0x556854e6ce40 2024-05-27T07:58:12.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f97a5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854febdb0 0x556854fec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f97a5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854febdb0 0x556854fec1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49630/0 (socket says 172.21.15.63:49630) 2024-05-27T07:58:12.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f97a5400700 1 -- 172.21.15.63:0/4272496358 learned_addr learned my addr 172.21.15.63:0/4272496358 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:12.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f979fe00700 1 --2- 172.21.15.63:0/4272496358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 0x556854feb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.074+0000 7f97a4a00700 1 --2- 172.21.15.63:0/4272496358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854e6ce40 0x556854ff17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.075+0000 7f97a5400700 1 -- 172.21.15.63:0/4272496358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854e6ce40 msgr2=0x556854ff17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.075+0000 7f97a5400700 1 --2- 172.21.15.63:0/4272496358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854e6ce40 0x556854ff17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.075+0000 7f97a5400700 1 -- 172.21.15.63:0/4272496358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 msgr2=0x556854feb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.075+0000 7f97a5400700 1 --2- 172.21.15.63:0/4272496358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 0x556854feb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.075+0000 7f97a5400700 1 -- 172.21.15.63:0/4272496358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556854ed2420 con 0x556854febdb0 2024-05-27T07:58:12.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.075+0000 7f979fe00700 1 --2- 172.21.15.63:0/4272496358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 0x556854feb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:12.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.075+0000 7f97a5400700 1 --2- 172.21.15.63:0/4272496358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854febdb0 0x556854fec1c0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f9798009bb0 tx=0x7f9798009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a0c01144c2ebeb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.076+0000 7f9792000700 1 -- 172.21.15.63:0/4272496358 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798020030 con 0x556854febdb0 2024-05-27T07:58:12.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.076+0000 7f9792000700 1 -- 172.21.15.63:0/4272496358 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9798004a50 con 0x556854febdb0 2024-05-27T07:58:12.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.076+0000 7f9792000700 1 -- 172.21.15.63:0/4272496358 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798007c20 con 0x556854febdb0 2024-05-27T07:58:12.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.077+0000 7f97b5f96b80 1 -- 172.21.15.63:0/4272496358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854febdb0 msgr2=0x556854fec1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.077+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/4272496358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854febdb0 0x556854fec1c0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f9798009bb0 tx=0x7f9798009be0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.077+0000 7f97b5f96b80 1 -- 172.21.15.63:0/4272496358 shutdown_connections 2024-05-27T07:58:12.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.077+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/4272496358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854febdb0 0x556854fec1c0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.077+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/4272496358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854e6ce40 0x556854ff17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.077+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/4272496358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 0x556854feb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.077+0000 7f97b5f96b80 1 -- 172.21.15.63:0/4272496358 >> 172.21.15.63:0/4272496358 conn(0x556854eccfc0 msgr2=0x556854fe68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.077+0000 7f97b5f96b80 1 -- 172.21.15.63:0/4272496358 shutdown_connections 2024-05-27T07:58:12.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.078+0000 7f97b5f96b80 1 -- 172.21.15.63:0/4272496358 wait complete. 2024-05-27T07:58:12.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.078+0000 7f97b5f96b80 1 -- start start 2024-05-27T07:58:12.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.078+0000 7f97b5f96b80 1 Processor -- start 2024-05-27T07:58:12.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.079+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854e6ce40 0x556854ff5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.079+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x556854ff5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.079+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556854ff5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.079+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556854e0ffa0 con 0x556854ff1d30 2024-05-27T07:58:12.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.079+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556854eaa760 con 0x556854e6ce40 2024-05-27T07:58:12.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.079+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556854ff87a0 con 0x556854febdb0 2024-05-27T07:58:12.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.079+0000 7f97b5f96b80 10 client.0 _ll_register_callbacks cb 0x556854eb3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:12.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.079+0000 7f97a5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556854ff5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.080+0000 7f97a5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556854ff5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49642/0 (socket says 172.21.15.63:49642) 2024-05-27T07:58:12.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.080+0000 7f97a5400700 1 -- 172.21.15.63:0/2306420802 learned_addr learned my addr 172.21.15.63:0/2306420802 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:12.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.080+0000 7f97a4a00700 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854e6ce40 0x556854ff5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.080+0000 7f979fe00700 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x556854ff5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.080+0000 7f97a5400700 1 -- 172.21.15.63:0/2306420802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 msgr2=0x556854ff5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.080+0000 7f97a5400700 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x556854ff5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.080+0000 7f97a5400700 1 -- 172.21.15.63:0/2306420802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854e6ce40 msgr2=0x556854ff5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.080+0000 7f97a5400700 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854e6ce40 0x556854ff5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.080+0000 7f97a5400700 1 -- 172.21.15.63:0/2306420802 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556854e598d0 con 0x556854ff1d30 2024-05-27T07:58:12.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.081+0000 7f97a5400700 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556854ff5fe0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f9798007600 tx=0x7f97980092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16f532801bee8e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.081+0000 7f9786a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:12.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.081+0000 7f9786a00700 1 -- 172.21.15.63:0/2306420802 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979802b070 con 0x556854ff1d30 2024-05-27T07:58:12.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.081+0000 7f9786a00700 1 -- 172.21.15.63:0/2306420802 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f9798007a80 con 0x556854ff1d30 2024-05-27T07:58:12.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.081+0000 7f97b5f96b80 1 -- 172.21.15.63:0/2306420802 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9798004a50 con 0x556854ff1d30 2024-05-27T07:58:12.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.081+0000 7f9786a00700 1 -- 172.21.15.63:0/2306420802 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9798020040 con 0x556854ff1d30 2024-05-27T07:58:12.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.082+0000 7f97b5f96b80 10 client.20172 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:12.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.082+0000 7f97b5f96b80 1 -- 172.21.15.63:0/2306420802 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556854eda060 con 0x556854ff1d30 2024-05-27T07:58:12.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.082+0000 7f9786a00700 1 -- 172.21.15.63:0/2306420802 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9798004a50 con 0x556854ff1d30 2024-05-27T07:58:12.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.082+0000 7f9786a00700 1 -- 172.21.15.63:0/2306420802 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9798011070 con 0x556854ff1d30 2024-05-27T07:58:12.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.083+0000 7f97b5f96b80 10 client.20172 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:12.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.083+0000 7f97b5f96b80 10 client.20172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:12.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.083+0000 7f97b5f96b80 20 client.20172 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:12.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.083+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556855005580 0x5568550079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.083+0000 7f97b5f96b80 4 client.20172 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:12.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.083+0000 7f97b5f96b80 1 -- 172.21.15.63:0/2306420802 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556854f1bd90 con 0x556855005580 2024-05-27T07:58:12.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.084+0000 7f97a4a00700 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556855005580 0x5568550079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.085+0000 7f97a4a00700 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556855005580 0x5568550079a0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484a8276064d68dc server_cookie=17d477f61ccde934 in_seq=0 out_seq=0 2024-05-27T07:58:12.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.085+0000 7f9786a00700 10 client.20172 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:12.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.086+0000 7f9786a00700 1 -- 172.21.15.63:0/2306420802 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556854f1bd90 con 0x556855005580 2024-05-27T07:58:12.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.086+0000 7f9786a00700 10 client.20172 handle_command_reply: tid=0 2024-05-27T07:58:12.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 client.20172 shutdown 2024-05-27T07:58:12.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 -- 172.21.15.63:0/2306420802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 msgr2=0x556854ff5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556854ff5fe0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f9798007600 tx=0x7f97980092e0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 -- 172.21.15.63:0/2306420802 shutdown_connections 2024-05-27T07:58:12.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556855005580 0x5568550079a0 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556854ff5fe0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854e6ce40 0x556854ff5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/2306420802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x556854ff5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 -- 172.21.15.63:0/2306420802 >> 172.21.15.63:0/2306420802 conn(0x556854eccfc0 msgr2=0x556854fe6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f9786a00700 0 client.20172 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:12.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 -- 172.21.15.63:0/2306420802 shutdown_connections 2024-05-27T07:58:12.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.087+0000 7f97b5f96b80 1 -- 172.21.15.63:0/2306420802 wait complete. 2024-05-27T07:58:12.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.088+0000 7f97b5f96b80 20 client.20172 trim_cache size 0 max 16384 2024-05-27T07:58:12.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.113+0000 7f97b5f96b80 1 Processor -- start 2024-05-27T07:58:12.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.113+0000 7f97b5f96b80 1 -- start start 2024-05-27T07:58:12.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.113+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854e6ce40 0x556854feb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.113+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x556854ff7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.113+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 0x556854fefb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.113+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01b4ee0 con 0x556854e6ce40 2024-05-27T07:58:12.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.114+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01b2770 con 0x556854ff1d30 2024-05-27T07:58:12.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.114+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01b05e0 con 0x556854febdb0 2024-05-27T07:58:12.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.114+0000 7f97a4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854e6ce40 0x556854feb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.114+0000 7f97a4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854e6ce40 0x556854feb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49658/0 (socket says 172.21.15.63:49658) 2024-05-27T07:58:12.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.114+0000 7f97a4a00700 1 -- 172.21.15.63:0/1361638704 learned_addr learned my addr 172.21.15.63:0/1361638704 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:12.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.114+0000 7f979fe00700 1 --2- 172.21.15.63:0/1361638704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x556854ff7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.114+0000 7f97a5400700 1 --2- 172.21.15.63:0/1361638704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 0x556854fefb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.114+0000 7f97a4a00700 1 -- 172.21.15.63:0/1361638704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 msgr2=0x556854ff7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.115+0000 7f97a4a00700 1 --2- 172.21.15.63:0/1361638704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x556854ff7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.115+0000 7f97a4a00700 1 -- 172.21.15.63:0/1361638704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 msgr2=0x556854fefb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.115+0000 7f97a4a00700 1 --2- 172.21.15.63:0/1361638704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 0x556854fefb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.115+0000 7f97a4a00700 1 -- 172.21.15.63:0/1361638704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556854fe8320 con 0x556854e6ce40 2024-05-27T07:58:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.115+0000 7f97a4a00700 1 --2- 172.21.15.63:0/1361638704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854e6ce40 0x556854feb870 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f979400bfd0 tx=0x7f9794015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceef15a4a932d3cd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.115+0000 7f9791600700 1 -- 172.21.15.63:0/1361638704 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794021c70 con 0x556854e6ce40 2024-05-27T07:58:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.115+0000 7f9791600700 1 -- 172.21.15.63:0/1361638704 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9794021e10 con 0x556854e6ce40 2024-05-27T07:58:12.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.115+0000 7f9791600700 1 -- 172.21.15.63:0/1361638704 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794003dc0 con 0x556854e6ce40 2024-05-27T07:58:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.116+0000 7f97b5f96b80 1 -- 172.21.15.63:0/1361638704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854e6ce40 msgr2=0x556854feb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.116+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/1361638704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854e6ce40 0x556854feb870 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f979400bfd0 tx=0x7f9794015000 comp rx=0 tx=0).stop 2024-05-27T07:58:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.117+0000 7f97b5f96b80 1 -- 172.21.15.63:0/1361638704 shutdown_connections 2024-05-27T07:58:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.117+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/1361638704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854ff1d30 0x556854fefb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.117+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/1361638704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854e6ce40 0x556854feb870 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.117+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/1361638704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x556854ff7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.117+0000 7f97b5f96b80 1 -- 172.21.15.63:0/1361638704 >> 172.21.15.63:0/1361638704 conn(0x556854eccfc0 msgr2=0x556854ff5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.117+0000 7f97b5f96b80 1 -- 172.21.15.63:0/1361638704 shutdown_connections 2024-05-27T07:58:12.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.117+0000 7f97b5f96b80 1 -- 172.21.15.63:0/1361638704 wait complete. 2024-05-27T07:58:12.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.117+0000 7f97b5f96b80 1 -- start start 2024-05-27T07:58:12.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.118+0000 7f97b5f96b80 1 Processor -- start 2024-05-27T07:58:12.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.118+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854e6ce40 0x556855007d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.118+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x5568550082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.118+0000 7f97b5f96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556855067db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.118+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01b0c30 con 0x556854ff1d30 2024-05-27T07:58:12.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.118+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01b20a0 con 0x556854e6ce40 2024-05-27T07:58:12.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.118+0000 7f97b5f96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01b1300 con 0x556854febdb0 2024-05-27T07:58:12.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.118+0000 7f97b5f96b80 10 client.0 _ll_register_callbacks cb 0x556854eb3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:12.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.119+0000 7f97a5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556855067db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.119+0000 7f97a5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556855067db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49668/0 (socket says 172.21.15.63:49668) 2024-05-27T07:58:12.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.119+0000 7f97a5400700 1 -- 172.21.15.63:0/482808210 learned_addr learned my addr 172.21.15.63:0/482808210 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:12.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.119+0000 7f97a4a00700 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854e6ce40 0x556855007d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.119+0000 7f979fe00700 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x5568550082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.119+0000 7f97a5400700 1 -- 172.21.15.63:0/482808210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 msgr2=0x5568550082f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.119+0000 7f97a5400700 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x5568550082f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.119+0000 7f97a5400700 1 -- 172.21.15.63:0/482808210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854e6ce40 msgr2=0x556855007d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.119+0000 7f97a5400700 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854e6ce40 0x556855007d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.119+0000 7f97a5400700 1 -- 172.21.15.63:0/482808210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568550683f0 con 0x556854ff1d30 2024-05-27T07:58:12.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.120+0000 7f97a5400700 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556855067db0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f9798008fd0 tx=0x7f979800b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cfa9071988d085 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.120+0000 7f9786a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:12.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.120+0000 7f9786a00700 1 -- 172.21.15.63:0/482808210 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979802b070 con 0x556854ff1d30 2024-05-27T07:58:12.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.120+0000 7f97b5f96b80 1 -- 172.21.15.63:0/482808210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9794021e10 con 0x556854ff1d30 2024-05-27T07:58:12.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.120+0000 7f9786a00700 1 -- 172.21.15.63:0/482808210 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f9798007a50 con 0x556854ff1d30 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f9786a00700 1 -- 172.21.15.63:0/482808210 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9798043050 con 0x556854ff1d30 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f97b5f96b80 10 client.20178 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f97b5f96b80 1 -- 172.21.15.63:0/482808210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55685506f720 con 0x556854ff1d30 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f9786a00700 1 -- 172.21.15.63:0/482808210 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9794021e10 con 0x556854ff1d30 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f9786a00700 1 -- 172.21.15.63:0/482808210 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9798011040 con 0x556854ff1d30 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f97b5f96b80 10 client.20178 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f97b5f96b80 10 client.20178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f97b5f96b80 20 client.20178 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55685506fdd0 0x5568550721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f97b5f96b80 4 client.20178 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.121+0000 7f97b5f96b80 1 -- 172.21.15.63:0/482808210 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556854f15570 con 0x55685506fdd0 2024-05-27T07:58:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.122+0000 7f97a4a00700 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55685506fdd0 0x5568550721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.123+0000 7f97a4a00700 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55685506fdd0 0x5568550721f0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c83a50d5b85665e server_cookie=36ea59db4bba0794 in_seq=0 out_seq=0 2024-05-27T07:58:12.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.123+0000 7f9786a00700 10 client.20178 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:12.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f9786a00700 1 -- 172.21.15.63:0/482808210 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556854f15570 con 0x55685506fdd0 2024-05-27T07:58:12.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f9786a00700 10 client.20178 handle_command_reply: tid=0 2024-05-27T07:58:12.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 client.20178 shutdown 2024-05-27T07:58:12.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 -- 172.21.15.63:0/482808210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 msgr2=0x556855067db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556855067db0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f9798008fd0 tx=0x7f979800b090 comp rx=0 tx=0).stop 2024-05-27T07:58:12.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 -- 172.21.15.63:0/482808210 shutdown_connections 2024-05-27T07:58:12.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55685506fdd0 0x5568550721f0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556854ff1d30 0x556855067db0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556854e6ce40 0x556855007d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 --2- 172.21.15.63:0/482808210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556854febdb0 0x5568550082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 -- 172.21.15.63:0/482808210 >> 172.21.15.63:0/482808210 conn(0x556854eccfc0 msgr2=0x556854ff2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f9786a00700 0 client.20178 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:12.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 -- 172.21.15.63:0/482808210 shutdown_connections 2024-05-27T07:58:12.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 1 -- 172.21.15.63:0/482808210 wait complete. 2024-05-27T07:58:12.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.124+0000 7f97b5f96b80 20 client.20178 trim_cache size 0 max 16384 2024-05-27T07:58:12.127 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:58:12.127 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:58:12.128 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:58:12.128 INFO:teuthology.orchestra.run.smithi063.stdout: "f2ce2ff7-4c09-43e8-8242-9459922c7545": { 2024-05-27T07:58:12.128 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:58:12.128 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "f2ce2ff7-4c09-43e8-8242-9459922c7545", 2024-05-27T07:58:12.128 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:58:12.128 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:58:12.128 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:58:12.128 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.125+0000 7f97a7400700 1 -- 172.21.15.63:0/1737937683 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f97880b29f0 msgr2=0x7f97880b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.125+0000 7f97a7400700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f97880b29f0 0x7f97880b4f00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f979400bc10 tx=0x7f979400b5f0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.125+0000 7f97a7400700 1 -- 172.21.15.63:0/1737937683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97a0107de0 msgr2=0x7f97a019d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.125+0000 7f97a7400700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97a0107de0 0x7f97a019d4b0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f978c00d980 tx=0x7f978c00dd40 comp rx=0 tx=0).stop 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.125+0000 7f97a7400700 1 -- 172.21.15.63:0/1737937683 shutdown_connections 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.125+0000 7f97a7400700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f97880b29f0 0x7f97880b4f00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.125+0000 7f97a7400700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97a0108730 0x7f97a01a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.125+0000 7f97a7400700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97a01034b0 0x7f97a019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.125+0000 7f97a7400700 1 --2- 172.21.15.63:0/1737937683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97a0107de0 0x7f97a019d4b0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.125+0000 7f97a7400700 1 -- 172.21.15.63:0/1737937683 >> 172.21.15.63:0/1737937683 conn(0x7f97a00fedb0 msgr2=0x7f97a006b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.126+0000 7f97a7400700 1 -- 172.21.15.63:0/1737937683 shutdown_connections 2024-05-27T07:58:12.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:12.126+0000 7f97a7400700 1 -- 172.21.15.63:0/1737937683 wait complete. 2024-05-27T07:58:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.845+0000 7f9386a00640 1 -- 172.21.15.63:0/2815153384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9380104220 msgr2=0x7f93801105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.845+0000 7f9386a00640 1 --2- 172.21.15.63:0/2815153384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9380104220 0x7f93801105f0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f9378009ca0 tx=0x7f937802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.845+0000 7f9386a00640 1 -- 172.21.15.63:0/2815153384 shutdown_connections 2024-05-27T07:58:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.845+0000 7f9386a00640 1 --2- 172.21.15.63:0/2815153384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9380104220 0x7f93801105f0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.845+0000 7f9386a00640 1 --2- 172.21.15.63:0/2815153384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380109f70 0x7f9380103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.845+0000 7f9386a00640 1 --2- 172.21.15.63:0/2815153384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380109ad0 0x7f9380103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.845+0000 7f9386a00640 1 -- 172.21.15.63:0/2815153384 >> 172.21.15.63:0/2815153384 conn(0x7f938007a1b0 msgr2=0x7f938007a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.845+0000 7f9386a00640 1 -- 172.21.15.63:0/2815153384 shutdown_connections 2024-05-27T07:58:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.846+0000 7f9386a00640 1 -- 172.21.15.63:0/2815153384 wait complete. 2024-05-27T07:58:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.847+0000 7f9386a00640 1 Processor -- start 2024-05-27T07:58:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.848+0000 7f9386a00640 1 -- start start 2024-05-27T07:58:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.848+0000 7f9386a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380104220 0x7f93801a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.848+0000 7f9386a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9380109ad0 0x7f93801a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.848+0000 7f9386a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380109f70 0x7f938019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.848+0000 7f9386a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9380113be0 con 0x7f9380109ad0 2024-05-27T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.848+0000 7f9386a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9380113a60 con 0x7f9380109f70 2024-05-27T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.848+0000 7f9386a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9380113d60 con 0x7f9380104220 2024-05-27T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.849+0000 7f9384c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9380109ad0 0x7f93801a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.849+0000 7f9384c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9380109ad0 0x7f93801a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49688/0 (socket says 172.21.15.63:49688) 2024-05-27T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.849+0000 7f9384c00640 1 -- 172.21.15.63:0/980371053 learned_addr learned my addr 172.21.15.63:0/980371053 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.849+0000 7f9385600640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380104220 0x7f93801a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.849+0000 7f9386000640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380109f70 0x7f938019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.850+0000 7f9384c00640 1 -- 172.21.15.63:0/980371053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380104220 msgr2=0x7f93801a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.850+0000 7f9384c00640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380104220 0x7f93801a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.850+0000 7f9384c00640 1 -- 172.21.15.63:0/980371053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380109f70 msgr2=0x7f938019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.850+0000 7f9384c00640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380109f70 0x7f938019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.850+0000 7f9384c00640 1 -- 172.21.15.63:0/980371053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938019cd90 con 0x7f9380109ad0 2024-05-27T07:58:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.850+0000 7f9384c00640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9380109ad0 0x7f93801a2a20 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f936c005fd0 tx=0x7f936c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.851+0000 7f937e000640 1 -- 172.21.15.63:0/980371053 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c017070 con 0x7f9380109ad0 2024-05-27T07:58:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.851+0000 7f937e000640 1 -- 172.21.15.63:0/980371053 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f936c009c30 con 0x7f9380109ad0 2024-05-27T07:58:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.851+0000 7f937e000640 1 -- 172.21.15.63:0/980371053 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c003c90 con 0x7f9380109ad0 2024-05-27T07:58:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.851+0000 7f9386a00640 1 -- 172.21.15.63:0/980371053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938019d020 con 0x7f9380109ad0 2024-05-27T07:58:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.851+0000 7f9386a00640 1 -- 172.21.15.63:0/980371053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93801a9280 con 0x7f9380109ad0 2024-05-27T07:58:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.855+0000 7f938efce740 1 Processor -- start 2024-05-27T07:58:12.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.856+0000 7f938efce740 1 -- start start 2024-05-27T07:58:12.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.856+0000 7f938efce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65221b0 0x5623f68c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.856+0000 7f938efce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64f92f0 0x5623f68c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.856+0000 7f938efce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.856+0000 7f938efce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5623f6835b90 con 0x5623f65014d0 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.856+0000 7f938efce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623f6825730 con 0x5623f64f92f0 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.856+0000 7f938efce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5623f6829770 con 0x5623f65221b0 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.856+0000 7f937e000640 1 -- 172.21.15.63:0/980371053 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f936c003e30 con 0x7f9380109ad0 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.857+0000 7f9384c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.857+0000 7f9384c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49690/0 (socket says 172.21.15.63:49690) 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.857+0000 7f9384c00640 1 -- 172.21.15.63:0/2581396341 learned_addr learned my addr 172.21.15.63:0/2581396341 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.857+0000 7f9386000640 1 --2- 172.21.15.63:0/2581396341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65221b0 0x5623f68c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.857+0000 7f9385600640 1 --2- 172.21.15.63:0/2581396341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64f92f0 0x5623f68c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.857+0000 7f9384c00640 1 -- 172.21.15.63:0/2581396341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65221b0 msgr2=0x5623f68c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.857+0000 7f9384c00640 1 --2- 172.21.15.63:0/2581396341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65221b0 0x5623f68c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.857+0000 7f9384c00640 1 -- 172.21.15.63:0/2581396341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64f92f0 msgr2=0x5623f68c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.857+0000 7f9384c00640 1 --2- 172.21.15.63:0/2581396341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64f92f0 0x5623f68c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.857+0000 7f9384c00640 1 -- 172.21.15.63:0/2581396341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f67be8b0 con 0x5623f65014d0 2024-05-27T07:58:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.858+0000 7f9384c00640 1 --2- 172.21.15.63:0/2581396341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68c54e0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f936c0df7c0 tx=0x7f936c0e8b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d933e53e57dd6b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.858+0000 7f9372600640 1 -- 172.21.15.63:0/2581396341 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c0f5070 con 0x5623f65014d0 2024-05-27T07:58:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.858+0000 7f9372600640 1 -- 172.21.15.63:0/2581396341 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f936c0e2d20 con 0x5623f65014d0 2024-05-27T07:58:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.858+0000 7f9372600640 1 -- 172.21.15.63:0/2581396341 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c0f0a40 con 0x5623f65014d0 2024-05-27T07:58:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.859+0000 7f937e000640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93680b29d0 0x7f93680b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.859+0000 7f937e000640 1 -- 172.21.15.63:0/980371053 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f936c0e2490 con 0x7f9380109ad0 2024-05-27T07:58:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.859+0000 7f9386000640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93680b29d0 0x7f93680b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.859+0000 7f938efce740 1 -- 172.21.15.63:0/2581396341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 msgr2=0x5623f68c54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.859+0000 7f938efce740 1 --2- 172.21.15.63:0/2581396341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68c54e0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f936c0df7c0 tx=0x7f936c0e8b00 comp rx=0 tx=0).stop 2024-05-27T07:58:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.860+0000 7f938efce740 1 -- 172.21.15.63:0/2581396341 shutdown_connections 2024-05-27T07:58:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.860+0000 7f938efce740 1 --2- 172.21.15.63:0/2581396341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68c54e0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.860+0000 7f938efce740 1 --2- 172.21.15.63:0/2581396341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64f92f0 0x5623f68c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.860+0000 7f938efce740 1 --2- 172.21.15.63:0/2581396341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65221b0 0x5623f68c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.860+0000 7f938efce740 1 -- 172.21.15.63:0/2581396341 >> 172.21.15.63:0/2581396341 conn(0x5623f6516300 msgr2=0x5623f68c1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.860+0000 7f938efce740 1 -- 172.21.15.63:0/2581396341 shutdown_connections 2024-05-27T07:58:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.860+0000 7f938efce740 1 -- 172.21.15.63:0/2581396341 wait complete. 2024-05-27T07:58:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.860+0000 7f9386000640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93680b29d0 0x7f93680b4ee0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9378006010 tx=0x7f9378009410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.861+0000 7f938efce740 1 -- start start 2024-05-27T07:58:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.861+0000 7f938efce740 1 Processor -- start 2024-05-27T07:58:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f938efce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f64f92f0 0x5623f68cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f938efce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65014d0 0x5623f68cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f938efce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f65221b0 0x5623f68daec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f938efce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5623f650b790 con 0x5623f64f92f0 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f938efce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623f67cdbc0 con 0x5623f65221b0 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f938efce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5623f68d1b90 con 0x5623f65014d0 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f938efce740 10 client.0 _ll_register_callbacks cb 0x5623f67ab400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f9385600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f64f92f0 0x5623f68cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f9385600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f64f92f0 0x5623f68cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49704/0 (socket says 172.21.15.63:49704) 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f9385600640 1 -- 172.21.15.63:0/1380244265 learned_addr learned my addr 172.21.15.63:0/1380244265 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f9384c00640 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65014d0 0x5623f68cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.862+0000 7f9386000640 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f65221b0 0x5623f68daec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.863+0000 7f9385600640 1 -- 172.21.15.63:0/1380244265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65014d0 msgr2=0x5623f68cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.863+0000 7f9385600640 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65014d0 0x5623f68cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.863+0000 7f9385600640 1 -- 172.21.15.63:0/1380244265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f65221b0 msgr2=0x5623f68daec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.863+0000 7f9385600640 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f65221b0 0x5623f68daec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.863+0000 7f9385600640 1 -- 172.21.15.63:0/1380244265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f653a210 con 0x5623f64f92f0 2024-05-27T07:58:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.863+0000 7f9385600640 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f64f92f0 0x5623f68cec80 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f937400b960 tx=0x7f937400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4592479bf78eda server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.863+0000 7f9366a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.863+0000 7f9366a00640 1 -- 172.21.15.63:0/1380244265 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374005020 con 0x5623f64f92f0 2024-05-27T07:58:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.863+0000 7f938efce740 1 -- 172.21.15.63:0/1380244265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936c0e2d20 con 0x5623f64f92f0 2024-05-27T07:58:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.863+0000 7f9366a00640 1 -- 172.21.15.63:0/1380244265 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f93740051c0 con 0x5623f64f92f0 2024-05-27T07:58:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f9366a00640 1 -- 172.21.15.63:0/1380244265 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9374024050 con 0x5623f64f92f0 2024-05-27T07:58:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f938efce740 10 client.20190 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f938efce740 1 -- 172.21.15.63:0/1380244265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5623f6606620 con 0x5623f64f92f0 2024-05-27T07:58:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f9366a00640 1 -- 172.21.15.63:0/1380244265 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f936c0e2d20 con 0x5623f64f92f0 2024-05-27T07:58:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f9366a00640 1 -- 172.21.15.63:0/1380244265 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f937401b470 con 0x5623f64f92f0 2024-05-27T07:58:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f938efce740 10 client.20190 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f938efce740 10 client.20190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f938efce740 20 client.20190 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f938efce740 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623f64e0d80 0x5623f68e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f938efce740 4 client.20190 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.864+0000 7f938efce740 1 -- 172.21.15.63:0/1380244265 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623f6837ae0 con 0x5623f64e0d80 2024-05-27T07:58:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.865+0000 7f9384c00640 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623f64e0d80 0x5623f68e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.866+0000 7f9384c00640 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623f64e0d80 0x5623f68e0f40 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=907028bba5461866 server_cookie=4bb495a54ac804f2 in_seq=0 out_seq=0 2024-05-27T07:58:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.866+0000 7f9366a00640 10 client.20190 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f9366a00640 1 -- 172.21.15.63:0/1380244265 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5623f6837ae0 con 0x5623f64e0d80 2024-05-27T07:58:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f9366a00640 10 client.20190 handle_command_reply: tid=0 2024-05-27T07:58:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f938efce740 1 client.20190 shutdown 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f938efce740 1 -- 172.21.15.63:0/1380244265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f64f92f0 msgr2=0x5623f68cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f938efce740 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f64f92f0 0x5623f68cec80 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f937400b960 tx=0x7f937400be20 comp rx=0 tx=0).stop 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f938efce740 1 -- 172.21.15.63:0/1380244265 shutdown_connections 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f938efce740 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623f64e0d80 0x5623f68e0f40 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f938efce740 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f65221b0 0x5623f68daec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f938efce740 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65014d0 0x5623f68cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f938efce740 1 --2- 172.21.15.63:0/1380244265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f64f92f0 0x5623f68cec80 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f9366a00640 0 client.20190 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.868+0000 7f938efce740 1 -- 172.21.15.63:0/1380244265 >> 172.21.15.63:0/1380244265 conn(0x5623f6516300 msgr2=0x5623f68c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.869+0000 7f938efce740 1 -- 172.21.15.63:0/1380244265 shutdown_connections 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.869+0000 7f938efce740 1 -- 172.21.15.63:0/1380244265 wait complete. 2024-05-27T07:58:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.869+0000 7f938efce740 20 client.20190 trim_cache size 0 max 16384 2024-05-27T07:58:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.891+0000 7f938efce740 1 Processor -- start 2024-05-27T07:58:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.891+0000 7f938efce740 1 -- start start 2024-05-27T07:58:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.891+0000 7f938efce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.891+0000 7f938efce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65221b0 0x5623f68dac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.892+0000 7f938efce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64b61c0 0x5623f68db170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.892+0000 7f938efce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b5b70 con 0x5623f65014d0 2024-05-27T07:58:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.892+0000 7f938efce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b3440 con 0x5623f64b61c0 2024-05-27T07:58:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.892+0000 7f938efce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b12b0 con 0x5623f65221b0 2024-05-27T07:58:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.892+0000 7f9385600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.892+0000 7f9385600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49710/0 (socket says 172.21.15.63:49710) 2024-05-27T07:58:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.892+0000 7f9385600640 1 -- 172.21.15.63:0/2162770730 learned_addr learned my addr 172.21.15.63:0/2162770730 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.892+0000 7f9384c00640 1 --2- 172.21.15.63:0/2162770730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65221b0 0x5623f68dac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.892+0000 7f9386000640 1 --2- 172.21.15.63:0/2162770730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64b61c0 0x5623f68db170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.893+0000 7f9385600640 1 -- 172.21.15.63:0/2162770730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65221b0 msgr2=0x5623f68dac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.893+0000 7f9385600640 1 --2- 172.21.15.63:0/2162770730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65221b0 0x5623f68dac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.893+0000 7f9385600640 1 -- 172.21.15.63:0/2162770730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64b61c0 msgr2=0x5623f68db170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.893+0000 7f9385600640 1 --2- 172.21.15.63:0/2162770730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64b61c0 0x5623f68db170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.893+0000 7f9385600640 1 -- 172.21.15.63:0/2162770730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f68db6b0 con 0x5623f65014d0 2024-05-27T07:58:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.893+0000 7f9385600640 1 --2- 172.21.15.63:0/2162770730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68ca480 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f9374000c00 tx=0x7f9374005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72188ed98c3454a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.893+0000 7f9371c00640 1 -- 172.21.15.63:0/2162770730 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374021040 con 0x5623f65014d0 2024-05-27T07:58:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.894+0000 7f9371c00640 1 -- 172.21.15.63:0/2162770730 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f937401b470 con 0x5623f65014d0 2024-05-27T07:58:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.894+0000 7f9371c00640 1 -- 172.21.15.63:0/2162770730 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93740055e0 con 0x5623f65014d0 2024-05-27T07:58:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.895+0000 7f938efce740 1 -- 172.21.15.63:0/2162770730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 msgr2=0x5623f68ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.895+0000 7f938efce740 1 --2- 172.21.15.63:0/2162770730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68ca480 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f9374000c00 tx=0x7f9374005f30 comp rx=0 tx=0).stop 2024-05-27T07:58:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.895+0000 7f938efce740 1 -- 172.21.15.63:0/2162770730 shutdown_connections 2024-05-27T07:58:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.895+0000 7f938efce740 1 --2- 172.21.15.63:0/2162770730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64b61c0 0x5623f68db170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.895+0000 7f938efce740 1 --2- 172.21.15.63:0/2162770730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65221b0 0x5623f68dac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.895+0000 7f938efce740 1 --2- 172.21.15.63:0/2162770730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65014d0 0x5623f68ca480 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.895+0000 7f938efce740 1 -- 172.21.15.63:0/2162770730 >> 172.21.15.63:0/2162770730 conn(0x5623f6516300 msgr2=0x5623f68d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.895+0000 7f938efce740 1 -- 172.21.15.63:0/2162770730 shutdown_connections 2024-05-27T07:58:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.895+0000 7f938efce740 1 -- 172.21.15.63:0/2162770730 wait complete. 2024-05-27T07:58:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.896+0000 7f938efce740 1 -- start start 2024-05-27T07:58:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.896+0000 7f938efce740 1 Processor -- start 2024-05-27T07:58:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f938efce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64b61c0 0x5623f68d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f938efce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65014d0 0x5623f68d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f938efce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65221b0 0x5623f68d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f938efce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b1900 con 0x5623f65221b0 2024-05-27T07:58:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f938efce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b2d70 con 0x5623f64b61c0 2024-05-27T07:58:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f938efce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93801b1fd0 con 0x5623f65014d0 2024-05-27T07:58:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f938efce740 10 client.0 _ll_register_callbacks cb 0x7f93740042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f9386000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65221b0 0x5623f68d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f9386000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65221b0 0x5623f68d2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49722/0 (socket says 172.21.15.63:49722) 2024-05-27T07:58:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f9386000640 1 -- 172.21.15.63:0/2166427148 learned_addr learned my addr 172.21.15.63:0/2166427148 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f9384c00640 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65014d0 0x5623f68d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.897+0000 7f9385600640 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64b61c0 0x5623f68d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.898+0000 7f9386000640 1 -- 172.21.15.63:0/2166427148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65014d0 msgr2=0x5623f68d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.898+0000 7f9386000640 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65014d0 0x5623f68d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.898+0000 7f9386000640 1 -- 172.21.15.63:0/2166427148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64b61c0 msgr2=0x5623f68d7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.898+0000 7f9386000640 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64b61c0 0x5623f68d7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.898+0000 7f9386000640 1 -- 172.21.15.63:0/2166427148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f68d3350 con 0x5623f65221b0 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.898+0000 7f9386000640 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65221b0 0x5623f68d2ce0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f9378009ca0 tx=0x7f937802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8973285b28f293d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.898+0000 7f9366a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.899+0000 7f9366a00640 1 -- 172.21.15.63:0/2166427148 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378030d20 con 0x5623f65221b0 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.899+0000 7f9366a00640 1 -- 172.21.15.63:0/2166427148 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f9378042030 con 0x5623f65221b0 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.899+0000 7f938efce740 1 -- 172.21.15.63:0/2166427148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f937401b470 con 0x5623f65221b0 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.899+0000 7f9366a00640 1 -- 172.21.15.63:0/2166427148 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9378003970 con 0x5623f65221b0 2024-05-27T07:58:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.899+0000 7f938efce740 10 client.20196 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.899+0000 7f938efce740 1 -- 172.21.15.63:0/2166427148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5623f6949430 con 0x5623f65221b0 2024-05-27T07:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.899+0000 7f9366a00640 1 -- 172.21.15.63:0/2166427148 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f937401b470 con 0x5623f65221b0 2024-05-27T07:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.900+0000 7f9366a00640 1 -- 172.21.15.63:0/2166427148 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f9378054420 con 0x5623f65221b0 2024-05-27T07:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.900+0000 7f938efce740 10 client.20196 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.900+0000 7f938efce740 10 client.20196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.900+0000 7f938efce740 20 client.20196 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.900+0000 7f938efce740 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623f6949be0 0x5623f694c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.900+0000 7f938efce740 4 client.20196 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.900+0000 7f938efce740 1 -- 172.21.15.63:0/2166427148 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623f68312c0 con 0x5623f6949be0 2024-05-27T07:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.901+0000 7f9385600640 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623f6949be0 0x5623f694c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.902+0000 7f9385600640 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623f6949be0 0x5623f694c020 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a4231797698c3a server_cookie=1434213504100eb2 in_seq=0 out_seq=0 2024-05-27T07:58:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.902+0000 7f9366a00640 10 client.20196 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f9366a00640 1 -- 172.21.15.63:0/2166427148 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5623f68312c0 con 0x5623f6949be0 2024-05-27T07:58:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f9366a00640 10 client.20196 handle_command_reply: tid=0 2024-05-27T07:58:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f938efce740 1 client.20196 shutdown 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f938efce740 1 -- 172.21.15.63:0/2166427148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65221b0 msgr2=0x5623f68d2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f938efce740 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65221b0 0x5623f68d2ce0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f9378009ca0 tx=0x7f937802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f938efce740 1 -- 172.21.15.63:0/2166427148 shutdown_connections 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f938efce740 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5623f6949be0 0x5623f694c020 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f938efce740 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5623f65221b0 0x5623f68d2ce0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f938efce740 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623f65014d0 0x5623f68d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f938efce740 1 --2- 172.21.15.63:0/2166427148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623f64b61c0 0x5623f68d7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f9366a00640 0 client.20196 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.948+0000 7f938efce740 1 -- 172.21.15.63:0/2166427148 >> 172.21.15.63:0/2166427148 conn(0x5623f6516300 msgr2=0x5623f68c47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.949+0000 7f938efce740 1 -- 172.21.15.63:0/2166427148 shutdown_connections 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.949+0000 7f938efce740 1 -- 172.21.15.63:0/2166427148 wait complete. 2024-05-27T07:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.949+0000 7f938efce740 20 client.20196 trim_cache size 0 max 16384 2024-05-27T07:58:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:58:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:58:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 -- 172.21.15.63:0/980371053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93680b29d0 msgr2=0x7f93680b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93680b29d0 0x7f93680b4ee0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9378006010 tx=0x7f9378009410 comp rx=0 tx=0).stop 2024-05-27T07:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 -- 172.21.15.63:0/980371053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9380109ad0 msgr2=0x7f93801a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9380109ad0 0x7f93801a2a20 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f936c005fd0 tx=0x7f936c00dde0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 -- 172.21.15.63:0/980371053 shutdown_connections 2024-05-27T07:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93680b29d0 0x7f93680b4ee0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9380109f70 0x7f938019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9380109ad0 0x7f93801a2a20 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 --2- 172.21.15.63:0/980371053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9380104220 0x7f93801a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 -- 172.21.15.63:0/980371053 >> 172.21.15.63:0/980371053 conn(0x7f938007a1b0 msgr2=0x7f93800768f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 -- 172.21.15.63:0/980371053 shutdown_connections 2024-05-27T07:58:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:12.950+0000 7f9386a00640 1 -- 172.21.15.63:0/980371053 wait complete. 2024-05-27T07:58:13.070 INFO:tasks.cephfs.filesystem:scrub status for tag:f2ce2ff7-4c09-43e8-8242-9459922c7545 - {'path': '/', 'tag': 'f2ce2ff7-4c09-43e8-8242-9459922c7545', 'options': 'recursive,force'} 2024-05-27T07:58:13.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:13 smithi086 ceph-mon[34687]: pgmap v780: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.1 MiB/s wr, 46 op/s 2024-05-27T07:58:13.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:13 smithi149 ceph-mon[35408]: pgmap v780: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.1 MiB/s wr, 46 op/s 2024-05-27T07:58:13.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:13 smithi063 ceph-mon[31613]: pgmap v780: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.1 MiB/s wr, 46 op/s 2024-05-27T07:58:15.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:15 smithi086 ceph-mon[34687]: pgmap v781: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:58:15.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:15 smithi149 ceph-mon[35408]: pgmap v781: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:58:15.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:15 smithi063 ceph-mon[31613]: pgmap v781: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:58:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:17 smithi086 ceph-mon[34687]: pgmap v782: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:58:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:58:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:58:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:58:17.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:17 smithi149 ceph-mon[35408]: pgmap v782: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:58:17.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:58:17.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:58:17.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:58:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:17 smithi063 ceph-mon[31613]: pgmap v782: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:58:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:58:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:58:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:58:19.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:19 smithi086 ceph-mon[34687]: pgmap v783: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 48 op/s 2024-05-27T07:58:19.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:19 smithi149 ceph-mon[35408]: pgmap v783: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 48 op/s 2024-05-27T07:58:19.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:19 smithi063 ceph-mon[31613]: pgmap v783: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 48 op/s 2024-05-27T07:58:21.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:21 smithi086 ceph-mon[34687]: pgmap v784: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 48 op/s 2024-05-27T07:58:21.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:21 smithi149 ceph-mon[35408]: pgmap v784: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 48 op/s 2024-05-27T07:58:21.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:21 smithi063 ceph-mon[31613]: pgmap v784: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.6 MiB/s wr, 48 op/s 2024-05-27T07:58:23.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:23 smithi086 ceph-mon[34687]: pgmap v785: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.8 MiB/s wr, 47 op/s 2024-05-27T07:58:23.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:23 smithi149 ceph-mon[35408]: pgmap v785: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.8 MiB/s wr, 47 op/s 2024-05-27T07:58:23.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:23 smithi063 ceph-mon[31613]: pgmap v785: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.8 MiB/s wr, 47 op/s 2024-05-27T07:58:25.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:25 smithi086 ceph-mon[34687]: pgmap v786: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T07:58:25.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:25 smithi149 ceph-mon[35408]: pgmap v786: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T07:58:25.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:25 smithi063 ceph-mon[31613]: pgmap v786: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T07:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.191+0000 7fba70400640 1 -- 172.21.15.63:0/1607957073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba68109040 msgr2=0x7fba68102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.191+0000 7fba70400640 1 --2- 172.21.15.63:0/1607957073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba68109040 0x7fba68102c80 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fba54009900 tx=0x7fba5402a400 comp rx=0 tx=0).stop 2024-05-27T07:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.191+0000 7fba70400640 1 -- 172.21.15.63:0/1607957073 shutdown_connections 2024-05-27T07:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.191+0000 7fba70400640 1 --2- 172.21.15.63:0/1607957073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba681032f0 0x7fba6810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.191+0000 7fba70400640 1 --2- 172.21.15.63:0/1607957073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba68109040 0x7fba68102c80 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.191+0000 7fba70400640 1 --2- 172.21.15.63:0/1607957073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba68108ba0 0x7fba68102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.191+0000 7fba70400640 1 -- 172.21.15.63:0/1607957073 >> 172.21.15.63:0/1607957073 conn(0x7fba68078270 msgr2=0x7fba680786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.191+0000 7fba70400640 1 -- 172.21.15.63:0/1607957073 shutdown_connections 2024-05-27T07:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.191+0000 7fba70400640 1 -- 172.21.15.63:0/1607957073 wait complete. 2024-05-27T07:58:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.193+0000 7fba70400640 1 Processor -- start 2024-05-27T07:58:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.193+0000 7fba70400640 1 -- start start 2024-05-27T07:58:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.193+0000 7fba70400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba681032f0 0x7fba681a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.193+0000 7fba70400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba68108ba0 0x7fba681a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.193+0000 7fba70400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba68109040 0x7fba6819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.193+0000 7fba70400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba68112c40 con 0x7fba68109040 2024-05-27T07:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.193+0000 7fba70400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba68112ac0 con 0x7fba68108ba0 2024-05-27T07:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.193+0000 7fba70400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba68112dc0 con 0x7fba681032f0 2024-05-27T07:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba68109040 0x7fba6819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba68109040 0x7fba6819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45580/0 (socket says 172.21.15.63:45580) 2024-05-27T07:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6e400640 1 -- 172.21.15.63:0/3144651020 learned_addr learned my addr 172.21.15.63:0/3144651020 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6da00640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba681032f0 0x7fba681a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6d000640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba68108ba0 0x7fba681a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6e400640 1 -- 172.21.15.63:0/3144651020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba681032f0 msgr2=0x7fba681a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6e400640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba681032f0 0x7fba681a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6e400640 1 -- 172.21.15.63:0/3144651020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba68108ba0 msgr2=0x7fba681a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6e400640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba68108ba0 0x7fba681a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6e400640 1 -- 172.21.15.63:0/3144651020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6819cef0 con 0x7fba68109040 2024-05-27T07:58:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.194+0000 7fba6e400640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba68109040 0x7fba6819c810 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fba6000d750 tx=0x7fba6000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.195+0000 7fba66000640 1 -- 172.21.15.63:0/3144651020 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60017070 con 0x7fba68109040 2024-05-27T07:58:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.195+0000 7fba70400640 1 -- 172.21.15.63:0/3144651020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba6819d1e0 con 0x7fba68109040 2024-05-27T07:58:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.195+0000 7fba66000640 1 -- 172.21.15.63:0/3144651020 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba600054b0 con 0x7fba68109040 2024-05-27T07:58:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.195+0000 7fba66000640 1 -- 172.21.15.63:0/3144651020 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60004b90 con 0x7fba68109040 2024-05-27T07:58:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.195+0000 7fba70400640 1 -- 172.21.15.63:0/3144651020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba681a9430 con 0x7fba68109040 2024-05-27T07:58:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.199+0000 7fba77484740 1 Processor -- start 2024-05-27T07:58:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.200+0000 7fba66000640 1 -- 172.21.15.63:0/3144651020 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fba60004d30 con 0x7fba68109040 2024-05-27T07:58:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.200+0000 7fba77484740 1 -- start start 2024-05-27T07:58:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba77484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f85a181b0 0x555f85dba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba77484740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859ef2f0 0x555f85dbada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba77484740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859f74d0 0x555f85dbb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba77484740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555f85d2bb90 con 0x555f859f74d0 2024-05-27T07:58:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba77484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f85d1b730 con 0x555f85a181b0 2024-05-27T07:58:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba77484740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555f85d1f770 con 0x555f859ef2f0 2024-05-27T07:58:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba6d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859f74d0 0x555f85dbb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba6d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859f74d0 0x555f85dbb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45590/0 (socket says 172.21.15.63:45590) 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba6d000640 1 -- 172.21.15.63:0/436961286 learned_addr learned my addr 172.21.15.63:0/436961286 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba6da00640 1 --2- 172.21.15.63:0/436961286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859ef2f0 0x555f85dbada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.201+0000 7fba6d000640 1 --2- 172.21.15.63:0/436961286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f85a181b0 0x555f85dba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.202+0000 7fba6d000640 1 -- 172.21.15.63:0/436961286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859ef2f0 msgr2=0x555f85dbada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.202+0000 7fba6d000640 1 --2- 172.21.15.63:0/436961286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859ef2f0 0x555f85dbada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.202+0000 7fba6d000640 1 -- 172.21.15.63:0/436961286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f85a181b0 msgr2=0x555f85dba860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.202+0000 7fba6d000640 1 --2- 172.21.15.63:0/436961286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f85a181b0 0x555f85dba860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.202+0000 7fba6d000640 1 -- 172.21.15.63:0/436961286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f85cb48b0 con 0x555f859f74d0 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.202+0000 7fba66000640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba500b2950 0x7fba500b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.202+0000 7fba6d000640 1 --2- 172.21.15.63:0/436961286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859f74d0 0x555f85dbb2e0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fba54005940 tx=0x7fba540058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95c80b00a9aa1e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.202+0000 7fba66000640 1 -- 172.21.15.63:0/3144651020 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7fba600e2570 con 0x7fba68109040 2024-05-27T07:58:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.202+0000 7fba6da00640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba500b2950 0x7fba500b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.203+0000 7fba5aa00640 1 -- 172.21.15.63:0/436961286 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54035040 con 0x555f859f74d0 2024-05-27T07:58:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.203+0000 7fba5aa00640 1 -- 172.21.15.63:0/436961286 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba540044c0 con 0x555f859f74d0 2024-05-27T07:58:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.203+0000 7fba5aa00640 1 -- 172.21.15.63:0/436961286 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54003b50 con 0x555f859f74d0 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.204+0000 7fba77484740 1 -- 172.21.15.63:0/436961286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859f74d0 msgr2=0x555f85dbb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.204+0000 7fba77484740 1 --2- 172.21.15.63:0/436961286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859f74d0 0x555f85dbb2e0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fba54005940 tx=0x7fba540058b0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.204+0000 7fba6da00640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba500b2950 0x7fba500b4e60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x555f85dbbc10 tx=0x7fba5c009760 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.204+0000 7fba77484740 1 -- 172.21.15.63:0/436961286 shutdown_connections 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.204+0000 7fba77484740 1 --2- 172.21.15.63:0/436961286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859f74d0 0x555f85dbb2e0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.204+0000 7fba77484740 1 --2- 172.21.15.63:0/436961286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859ef2f0 0x555f85dbada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.204+0000 7fba77484740 1 --2- 172.21.15.63:0/436961286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f85a181b0 0x555f85dba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.204+0000 7fba77484740 1 -- 172.21.15.63:0/436961286 >> 172.21.15.63:0/436961286 conn(0x555f85a0c300 msgr2=0x555f85db7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.204+0000 7fba77484740 1 -- 172.21.15.63:0/436961286 shutdown_connections 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.204+0000 7fba77484740 1 -- 172.21.15.63:0/436961286 wait complete. 2024-05-27T07:58:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.205+0000 7fba77484740 1 -- start start 2024-05-27T07:58:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.205+0000 7fba77484740 1 Processor -- start 2024-05-27T07:58:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba77484740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859ef2f0 0x555f85dc4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba77484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 0x555f85dc5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba77484740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f85a181b0 0x555f85dd0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba77484740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555f85a01790 con 0x555f859ef2f0 2024-05-27T07:58:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba77484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f85cc3bc0 con 0x555f859f74d0 2024-05-27T07:58:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba77484740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555f85dc7930 con 0x555f85a181b0 2024-05-27T07:58:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba77484740 10 client.0 _ll_register_callbacks cb 0x555f85ca1400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba6da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859ef2f0 0x555f85dc4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba6da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859ef2f0 0x555f85dc4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45594/0 (socket says 172.21.15.63:45594) 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba6da00640 1 -- 172.21.15.63:0/1013433773 learned_addr learned my addr 172.21.15.63:0/1013433773 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba6e400640 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f85a181b0 0x555f85dd0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.206+0000 7fba6d000640 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 0x555f85dc5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.207+0000 7fba6da00640 1 -- 172.21.15.63:0/1013433773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f85a181b0 msgr2=0x555f85dd0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.207+0000 7fba6da00640 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f85a181b0 0x555f85dd0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.207+0000 7fba6da00640 1 -- 172.21.15.63:0/1013433773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 msgr2=0x555f85dc5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.207+0000 7fba6da00640 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 0x555f85dc5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.207+0000 7fba6da00640 1 -- 172.21.15.63:0/1013433773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f85a30210 con 0x555f859ef2f0 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.207+0000 7fba6da00640 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859ef2f0 0x555f85dc4d00 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fba5c01d680 tx=0x7fba5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843bed9ae3eb1157 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.207+0000 7fba4f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.207+0000 7fba4f400640 1 -- 172.21.15.63:0/1013433773 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c01b870 con 0x555f859ef2f0 2024-05-27T07:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.207+0000 7fba4f400640 1 -- 172.21.15.63:0/1013433773 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7fba5c01ba10 con 0x555f859ef2f0 2024-05-27T07:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.207+0000 7fba77484740 1 -- 172.21.15.63:0/1013433773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f85d2dae0 con 0x555f859ef2f0 2024-05-27T07:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.208+0000 7fba4f400640 1 -- 172.21.15.63:0/1013433773 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fba5c01bc10 con 0x555f859ef2f0 2024-05-27T07:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.208+0000 7fba77484740 10 client.20208 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.208+0000 7fba77484740 1 -- 172.21.15.63:0/1013433773 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555f85afc620 con 0x555f859ef2f0 2024-05-27T07:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.208+0000 7fba4f400640 1 -- 172.21.15.63:0/1013433773 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x555f85d2dae0 con 0x555f859ef2f0 2024-05-27T07:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.208+0000 7fba4f400640 1 -- 172.21.15.63:0/1013433773 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fba5c024070 con 0x555f859ef2f0 2024-05-27T07:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.208+0000 7fba77484740 10 client.20208 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.208+0000 7fba77484740 10 client.20208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.209+0000 7fba77484740 20 client.20208 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.209+0000 7fba77484740 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555f859d6d80 0x555f85dd6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.209+0000 7fba77484740 4 client.20208 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.209+0000 7fba77484740 1 -- 172.21.15.63:0/1013433773 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f85d272c0 con 0x555f859d6d80 2024-05-27T07:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.209+0000 7fba6d000640 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555f859d6d80 0x555f85dd6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.210+0000 7fba6d000640 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555f859d6d80 0x555f85dd6d80 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c081f3d62876e server_cookie=c302af0b3e5c92b3 in_seq=0 out_seq=0 2024-05-27T07:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.210+0000 7fba4f400640 10 client.20208 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.212+0000 7fba4f400640 1 -- 172.21.15.63:0/1013433773 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555f85d272c0 con 0x555f859d6d80 2024-05-27T07:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.212+0000 7fba4f400640 10 client.20208 handle_command_reply: tid=0 2024-05-27T07:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.212+0000 7fba77484740 1 client.20208 shutdown 2024-05-27T07:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.212+0000 7fba77484740 1 -- 172.21.15.63:0/1013433773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859ef2f0 msgr2=0x555f85dc4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba77484740 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859ef2f0 0x555f85dc4d00 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fba5c01d680 tx=0x7fba5c004a40 comp rx=0 tx=0).stop 2024-05-27T07:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba77484740 1 -- 172.21.15.63:0/1013433773 shutdown_connections 2024-05-27T07:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba77484740 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555f859d6d80 0x555f85dd6d80 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba77484740 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f85a181b0 0x555f85dd0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba77484740 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 0x555f85dc5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba77484740 1 --2- 172.21.15.63:0/1013433773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f859ef2f0 0x555f85dc4d00 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba4f400640 0 client.20208 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba77484740 1 -- 172.21.15.63:0/1013433773 >> 172.21.15.63:0/1013433773 conn(0x555f85a0c300 msgr2=0x555f85dbc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba77484740 1 -- 172.21.15.63:0/1013433773 shutdown_connections 2024-05-27T07:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba77484740 1 -- 172.21.15.63:0/1013433773 wait complete. 2024-05-27T07:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.213+0000 7fba77484740 20 client.20208 trim_cache size 0 max 16384 2024-05-27T07:58:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.234+0000 7fba77484740 1 Processor -- start 2024-05-27T07:58:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.234+0000 7fba77484740 1 -- start start 2024-05-27T07:58:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.234+0000 7fba77484740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859ef2f0 0x555f85dc6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.234+0000 7fba77484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 0x555f85dd09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.234+0000 7fba77484740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dd0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.234+0000 7fba77484740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba681b5d00 con 0x555f85a181b0 2024-05-27T07:58:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.234+0000 7fba77484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba681b35d0 con 0x555f859f74d0 2024-05-27T07:58:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.234+0000 7fba77484740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba681b1440 con 0x555f859ef2f0 2024-05-27T07:58:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.234+0000 7fba6e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dd0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dd0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45596/0 (socket says 172.21.15.63:45596) 2024-05-27T07:58:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6e400640 1 -- 172.21.15.63:0/3850366366 learned_addr learned my addr 172.21.15.63:0/3850366366 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6da00640 1 --2- 172.21.15.63:0/3850366366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859ef2f0 0x555f85dc6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6d000640 1 --2- 172.21.15.63:0/3850366366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 0x555f85dd09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6e400640 1 -- 172.21.15.63:0/3850366366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859ef2f0 msgr2=0x555f85dc6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6e400640 1 --2- 172.21.15.63:0/3850366366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859ef2f0 0x555f85dc6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6e400640 1 -- 172.21.15.63:0/3850366366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 msgr2=0x555f85dd09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6e400640 1 --2- 172.21.15.63:0/3850366366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 0x555f85dd09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6e400640 1 -- 172.21.15.63:0/3850366366 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f85dbbc40 con 0x555f85a181b0 2024-05-27T07:58:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6d000640 1 --2- 172.21.15.63:0/3850366366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 0x555f85dd09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.235+0000 7fba6e400640 1 --2- 172.21.15.63:0/3850366366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dd0f10 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fba600a66a0 tx=0x7fba600a8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d68ab947010f305 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.236+0000 7fba5a000640 1 -- 172.21.15.63:0/3850366366 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba600c3070 con 0x555f85a181b0 2024-05-27T07:58:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.236+0000 7fba5a000640 1 -- 172.21.15.63:0/3850366366 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba600a9430 con 0x555f85a181b0 2024-05-27T07:58:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.236+0000 7fba5a000640 1 -- 172.21.15.63:0/3850366366 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60020040 con 0x555f85a181b0 2024-05-27T07:58:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.237+0000 7fba77484740 1 -- 172.21.15.63:0/3850366366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 msgr2=0x555f85dd0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.237+0000 7fba77484740 1 --2- 172.21.15.63:0/3850366366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dd0f10 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fba600a66a0 tx=0x7fba600a8c60 comp rx=0 tx=0).stop 2024-05-27T07:58:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.237+0000 7fba77484740 1 -- 172.21.15.63:0/3850366366 shutdown_connections 2024-05-27T07:58:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.237+0000 7fba77484740 1 --2- 172.21.15.63:0/3850366366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dd0f10 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.237+0000 7fba77484740 1 --2- 172.21.15.63:0/3850366366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859f74d0 0x555f85dd09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.237+0000 7fba77484740 1 --2- 172.21.15.63:0/3850366366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859ef2f0 0x555f85dc6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.237+0000 7fba77484740 1 -- 172.21.15.63:0/3850366366 >> 172.21.15.63:0/3850366366 conn(0x555f85a0c300 msgr2=0x555f85dbb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.237+0000 7fba77484740 1 -- 172.21.15.63:0/3850366366 shutdown_connections 2024-05-27T07:58:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.237+0000 7fba77484740 1 -- 172.21.15.63:0/3850366366 wait complete. 2024-05-27T07:58:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.238+0000 7fba77484740 1 -- start start 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.238+0000 7fba77484740 1 Processor -- start 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.238+0000 7fba77484740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859ef2f0 0x555f85dc7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.238+0000 7fba77484740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859f74d0 0x555f85dc80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.238+0000 7fba77484740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dc8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba77484740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba681b1a90 con 0x555f85a181b0 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba77484740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba681b2f00 con 0x555f859ef2f0 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba77484740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba681b2160 con 0x555f859f74d0 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba77484740 10 client.0 _ll_register_callbacks cb 0x7fba5c003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba6e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dc8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba6e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dc8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45606/0 (socket says 172.21.15.63:45606) 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba6e400640 1 -- 172.21.15.63:0/512584356 learned_addr learned my addr 172.21.15.63:0/512584356 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba6da00640 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859ef2f0 0x555f85dc7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba6d000640 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859f74d0 0x555f85dc80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba6e400640 1 -- 172.21.15.63:0/512584356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859f74d0 msgr2=0x555f85dc80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba6e400640 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859f74d0 0x555f85dc80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.239+0000 7fba6e400640 1 -- 172.21.15.63:0/512584356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859ef2f0 msgr2=0x555f85dc7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.240+0000 7fba6e400640 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859ef2f0 0x555f85dc7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.240+0000 7fba6e400640 1 -- 172.21.15.63:0/512584356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f85e3d490 con 0x555f85a181b0 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.240+0000 7fba6e400640 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dc8600 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fba600a9130 tx=0x7fba600dfd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06bf6df9e8c6cd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.240+0000 7fba4f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.240+0000 7fba4f400640 1 -- 172.21.15.63:0/512584356 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60020070 con 0x555f85a181b0 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.240+0000 7fba4f400640 1 -- 172.21.15.63:0/512584356 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7fba600e20c0 con 0x555f85a181b0 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.240+0000 7fba77484740 1 -- 172.21.15.63:0/512584356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba600a9430 con 0x555f85a181b0 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.240+0000 7fba4f400640 1 -- 172.21.15.63:0/512584356 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fba600c3070 con 0x555f85a181b0 2024-05-27T07:58:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.240+0000 7fba77484740 10 client.20214 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.240+0000 7fba77484740 1 -- 172.21.15.63:0/512584356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555f85e3e680 con 0x555f85a181b0 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.241+0000 7fba4f400640 1 -- 172.21.15.63:0/512584356 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba600a9430 con 0x555f85a181b0 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.241+0000 7fba4f400640 1 -- 172.21.15.63:0/512584356 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fba600ca300 con 0x555f85a181b0 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.241+0000 7fba77484740 10 client.20214 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.241+0000 7fba77484740 10 client.20214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.241+0000 7fba77484740 20 client.20214 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.241+0000 7fba77484740 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555f859ac1c0 0x555f85e40d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.241+0000 7fba77484740 4 client.20214 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.241+0000 7fba77484740 1 -- 172.21.15.63:0/512584356 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f85ca14b0 con 0x555f859ac1c0 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.242+0000 7fba6da00640 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555f859ac1c0 0x555f85e40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.243+0000 7fba6da00640 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555f859ac1c0 0x555f85e40d00 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caeb31ee028c996e server_cookie=92aefd04f2111473 in_seq=0 out_seq=0 2024-05-27T07:58:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.243+0000 7fba4f400640 10 client.20214 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba4f400640 1 -- 172.21.15.63:0/512584356 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555f85ca14b0 con 0x555f859ac1c0 2024-05-27T07:58:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba4f400640 10 client.20214 handle_command_reply: tid=0 2024-05-27T07:58:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba77484740 1 client.20214 shutdown 2024-05-27T07:58:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba77484740 1 -- 172.21.15.63:0/512584356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 msgr2=0x555f85dc8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba77484740 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dc8600 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fba600a9130 tx=0x7fba600dfd30 comp rx=0 tx=0).stop 2024-05-27T07:58:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba77484740 1 -- 172.21.15.63:0/512584356 shutdown_connections 2024-05-27T07:58:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba77484740 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555f859ac1c0 0x555f85e40d00 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba77484740 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555f85a181b0 0x555f85dc8600 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba77484740 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f859f74d0 0x555f85dc80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba77484740 1 --2- 172.21.15.63:0/512584356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f859ef2f0 0x555f85dc7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.291+0000 7fba4f400640 0 client.20214 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.292+0000 7fba77484740 1 -- 172.21.15.63:0/512584356 >> 172.21.15.63:0/512584356 conn(0x555f85a0c300 msgr2=0x555f85dbb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.292+0000 7fba77484740 1 -- 172.21.15.63:0/512584356 shutdown_connections 2024-05-27T07:58:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.292+0000 7fba77484740 1 -- 172.21.15.63:0/512584356 wait complete. 2024-05-27T07:58:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.292+0000 7fba77484740 20 client.20214 trim_cache size 0 max 16384 2024-05-27T07:58:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:58:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:58:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:58:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.293+0000 7fba70400640 1 -- 172.21.15.63:0/3144651020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba500b2950 msgr2=0x7fba500b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.293+0000 7fba70400640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba500b2950 0x7fba500b4e60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x555f85dbbc10 tx=0x7fba5c009760 comp rx=0 tx=0).stop 2024-05-27T07:58:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.293+0000 7fba70400640 1 -- 172.21.15.63:0/3144651020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba68109040 msgr2=0x7fba6819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.293+0000 7fba70400640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba68109040 0x7fba6819c810 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fba6000d750 tx=0x7fba6000dc10 comp rx=0 tx=0).stop 2024-05-27T07:58:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.293+0000 7fba70400640 1 -- 172.21.15.63:0/3144651020 shutdown_connections 2024-05-27T07:58:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.293+0000 7fba70400640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba500b2950 0x7fba500b4e60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.293+0000 7fba70400640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba68109040 0x7fba6819c810 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.293+0000 7fba70400640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba68108ba0 0x7fba681a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.293+0000 7fba70400640 1 --2- 172.21.15.63:0/3144651020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba681032f0 0x7fba681a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.293+0000 7fba70400640 1 -- 172.21.15.63:0/3144651020 >> 172.21.15.63:0/3144651020 conn(0x7fba68078270 msgr2=0x7fba68101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.294+0000 7fba70400640 1 -- 172.21.15.63:0/3144651020 shutdown_connections 2024-05-27T07:58:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:26.294+0000 7fba70400640 1 -- 172.21.15.63:0/3144651020 wait complete. 2024-05-27T07:58:27.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:26 smithi086 ceph-mon[34687]: pgmap v787: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T07:58:27.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:26 smithi149 ceph-mon[35408]: pgmap v787: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T07:58:27.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:26 smithi063 ceph-mon[31613]: pgmap v787: 129 pgs: 129 active+clean; 7.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T07:58:29.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:29 smithi086 ceph-mon[34687]: pgmap v788: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 18 op/s 2024-05-27T07:58:29.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:29 smithi149 ceph-mon[35408]: pgmap v788: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 18 op/s 2024-05-27T07:58:29.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:29 smithi063 ceph-mon[31613]: pgmap v788: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 18 op/s 2024-05-27T07:58:30.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:29 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[69422]: *** Caught signal (Segmentation fault) ** 2024-05-27T07:58:30.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:29 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[69422]: in thread 7faeea000700 thread_name:msgr-worker-0 2024-05-27T07:58:30.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:30 smithi086 podman[76114]: 2024-05-27 07:58:30.028739556 +0000 UTC m=+0.038272572 container died 864f5397b10eefc55b6aa14400c32e308f0cf81101b58e384f8abf68dd06b034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-27T07:58:30.960 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:30 smithi086 podman[76114]: 2024-05-27 07:58:30.702233101 +0000 UTC m=+0.711766146 container remove 864f5397b10eefc55b6aa14400c32e308f0cf81101b58e384f8abf68dd06b034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True) 2024-05-27T07:58:30.960 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:30 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Main process exited, code=exited, status=139/n/a 2024-05-27T07:58:31.460 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:31 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-05-27T07:58:31.460 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:31 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Consumed 8.760s CPU time. 2024-05-27T07:58:31.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:31 smithi086 ceph-mon[34687]: pgmap v789: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:58:31.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:31 smithi149 ceph-mon[35408]: pgmap v789: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:58:31.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:31 smithi063 ceph-mon[31613]: pgmap v789: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:58:33.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:33 smithi086 ceph-mon[34687]: pgmap v790: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T07:58:33.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:33 smithi149 ceph-mon[35408]: pgmap v790: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T07:58:33.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:33 smithi063 ceph-mon[31613]: pgmap v790: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T07:58:35.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:35 smithi086 ceph-mon[34687]: pgmap v791: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:58:35.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:35 smithi149 ceph-mon[35408]: pgmap v791: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:58:35.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:35 smithi063 ceph-mon[31613]: pgmap v791: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:58:37.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:37 smithi086 ceph-mon[34687]: pgmap v792: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:58:37.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:37 smithi149 ceph-mon[35408]: pgmap v792: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:58:37.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:37 smithi063 ceph-mon[31613]: pgmap v792: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:58:39.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:39 smithi086 ceph-mon[34687]: pgmap v793: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:58:39.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:39 smithi149 ceph-mon[35408]: pgmap v793: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:58:39.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:39 smithi063 ceph-mon[31613]: pgmap v793: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T07:58:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.554+0000 7f0509000640 1 -- 172.21.15.63:0/2506807921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f050410da00 msgr2=0x7f050410fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.554+0000 7f0509000640 1 --2- 172.21.15.63:0/2506807921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f050410da00 0x7f050410fe20 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f002a3f0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.554+0000 7f0509000640 1 -- 172.21.15.63:0/2506807921 shutdown_connections 2024-05-27T07:58:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.554+0000 7f0509000640 1 --2- 172.21.15.63:0/2506807921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0504110360 0x7f0504112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.554+0000 7f0509000640 1 --2- 172.21.15.63:0/2506807921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f050410da00 0x7f050410fe20 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.554+0000 7f0509000640 1 --2- 172.21.15.63:0/2506807921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05041027c0 0x7f0504102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.554+0000 7f0509000640 1 -- 172.21.15.63:0/2506807921 >> 172.21.15.63:0/2506807921 conn(0x7f05040fe210 msgr2=0x7f0504100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.554+0000 7f0509000640 1 -- 172.21.15.63:0/2506807921 shutdown_connections 2024-05-27T07:58:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.554+0000 7f0509000640 1 -- 172.21.15.63:0/2506807921 wait complete. 2024-05-27T07:58:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.555+0000 7f0509000640 1 Processor -- start 2024-05-27T07:58:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.556+0000 7f0509000640 1 -- start start 2024-05-27T07:58:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.556+0000 7f0509000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05041027c0 0x7f05041a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.556+0000 7f0509000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050410da00 0x7f05041a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0509000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0504110360 0x7f050419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0509000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0504115d20 con 0x7f05041027c0 2024-05-27T07:58:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0509000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0504115ba0 con 0x7f050410da00 2024-05-27T07:58:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0509000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0504115ea0 con 0x7f0504110360 2024-05-27T07:58:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0503400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05041027c0 0x7f05041a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0503400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05041027c0 0x7f05041a25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58168/0 (socket says 172.21.15.63:58168) 2024-05-27T07:58:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0503400640 1 -- 172.21.15.63:0/2732057007 learned_addr learned my addr 172.21.15.63:0/2732057007 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0503e00640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0504110360 0x7f050419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0502a00640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050410da00 0x7f05041a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0503400640 1 -- 172.21.15.63:0/2732057007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0504110360 msgr2=0x7f050419c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0503400640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0504110360 0x7f050419c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0503400640 1 -- 172.21.15.63:0/2732057007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050410da00 msgr2=0x7f05041a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0503400640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050410da00 0x7f05041a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.557+0000 7f0503400640 1 -- 172.21.15.63:0/2732057007 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050419ce60 con 0x7f05041027c0 2024-05-27T07:58:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.558+0000 7f0503400640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05041027c0 0x7f05041a25b0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f04f400d950 tx=0x7f04f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.558+0000 7f04fbe00640 1 -- 172.21.15.63:0/2732057007 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4017070 con 0x7f05041027c0 2024-05-27T07:58:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.558+0000 7f04fbe00640 1 -- 172.21.15.63:0/2732057007 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04f4009c30 con 0x7f05041027c0 2024-05-27T07:58:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.558+0000 7f0509000640 1 -- 172.21.15.63:0/2732057007 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050419d0f0 con 0x7f05041027c0 2024-05-27T07:58:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.558+0000 7f04fbe00640 1 -- 172.21.15.63:0/2732057007 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f400f620 con 0x7f05041027c0 2024-05-27T07:58:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.558+0000 7f0509000640 1 -- 172.21.15.63:0/2732057007 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05041a9350 con 0x7f05041027c0 2024-05-27T07:58:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.562+0000 7f04fbe00640 1 -- 172.21.15.63:0/2732057007 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f04f4010460 con 0x7f05041027c0 2024-05-27T07:58:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.563+0000 7f051165c740 1 Processor -- start 2024-05-27T07:58:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.563+0000 7f051165c740 1 -- start start 2024-05-27T07:58:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.563+0000 7f051165c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e39a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.563+0000 7f051165c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 0x557eb9e39fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.563+0000 7f051165c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e3a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.563+0000 7f051165c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557eb9daab90 con 0x557eb9a971b0 2024-05-27T07:58:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.563+0000 7f051165c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557eb9d9a730 con 0x557eb9a764d0 2024-05-27T07:58:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.563+0000 7f051165c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557eb9d9e770 con 0x557eb9a6e2f0 2024-05-27T07:58:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0503e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e39a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0503e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e39a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58170/0 (socket says 172.21.15.63:58170) 2024-05-27T07:58:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0503e00640 1 -- 172.21.15.63:0/2454610239 learned_addr learned my addr 172.21.15.63:0/2454610239 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0502a00640 1 --2- 172.21.15.63:0/2454610239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e3a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0503400640 1 --2- 172.21.15.63:0/2454610239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 0x557eb9e39fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0503e00640 1 -- 172.21.15.63:0/2454610239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 msgr2=0x557eb9e39fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0503e00640 1 --2- 172.21.15.63:0/2454610239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 0x557eb9e39fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0503e00640 1 -- 172.21.15.63:0/2454610239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 msgr2=0x557eb9e3a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0503e00640 1 --2- 172.21.15.63:0/2454610239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e3a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0503e00640 1 -- 172.21.15.63:0/2454610239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb9d338b0 con 0x557eb9a971b0 2024-05-27T07:58:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f04fbe00640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04e80b29d0 0x7f04e80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.564+0000 7f0503400640 1 --2- 172.21.15.63:0/2454610239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 0x557eb9e39fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.565+0000 7f04fbe00640 1 -- 172.21.15.63:0/2732057007 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f04f40e26a0 con 0x7f05041027c0 2024-05-27T07:58:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.565+0000 7f0503400640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04e80b29d0 0x7f04e80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.565+0000 7f0503e00640 1 --2- 172.21.15.63:0/2454610239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e39a60 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009e60 tx=0x7f04fc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1916e1cdf65c0c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.565+0000 7f04f8c00640 1 -- 172.21.15.63:0/2454610239 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc016070 con 0x557eb9a971b0 2024-05-27T07:58:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.566+0000 7f04f8c00640 1 -- 172.21.15.63:0/2454610239 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04fc004d20 con 0x557eb9a971b0 2024-05-27T07:58:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.566+0000 7f04f8c00640 1 -- 172.21.15.63:0/2454610239 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0126b0 con 0x557eb9a971b0 2024-05-27T07:58:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.567+0000 7f051165c740 1 -- 172.21.15.63:0/2454610239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 msgr2=0x557eb9e39a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.567+0000 7f0503400640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04e80b29d0 0x7f04e80b4ee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x557eb9e37550 tx=0x7f04f400bf60 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.567+0000 7f051165c740 1 --2- 172.21.15.63:0/2454610239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e39a60 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009e60 tx=0x7f04fc010740 comp rx=0 tx=0).stop 2024-05-27T07:58:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.567+0000 7f051165c740 1 -- 172.21.15.63:0/2454610239 shutdown_connections 2024-05-27T07:58:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.567+0000 7f051165c740 1 --2- 172.21.15.63:0/2454610239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e3a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.567+0000 7f051165c740 1 --2- 172.21.15.63:0/2454610239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 0x557eb9e39fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.567+0000 7f051165c740 1 --2- 172.21.15.63:0/2454610239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e39a60 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.567+0000 7f051165c740 1 -- 172.21.15.63:0/2454610239 >> 172.21.15.63:0/2454610239 conn(0x557eb9a8b300 msgr2=0x557eb9e36210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.567+0000 7f051165c740 1 -- 172.21.15.63:0/2454610239 shutdown_connections 2024-05-27T07:58:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.567+0000 7f051165c740 1 -- 172.21.15.63:0/2454610239 wait complete. 2024-05-27T07:58:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.568+0000 7f051165c740 1 -- start start 2024-05-27T07:58:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.568+0000 7f051165c740 1 Processor -- start 2024-05-27T07:58:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.568+0000 7f051165c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 0x557eb9e43c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.568+0000 7f051165c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e44180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.568+0000 7f051165c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.569+0000 7f051165c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557eb9a80790 con 0x557eb9a971b0 2024-05-27T07:58:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.569+0000 7f051165c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557eb9d42bc0 con 0x557eb9a764d0 2024-05-27T07:58:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.569+0000 7f051165c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557eb9e46b90 con 0x557eb9a6e2f0 2024-05-27T07:58:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.569+0000 7f051165c740 10 client.0 _ll_register_callbacks cb 0x557eb9d20400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.569+0000 7f0503e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.569+0000 7f0503e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58180/0 (socket says 172.21.15.63:58180) 2024-05-27T07:58:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.569+0000 7f0503e00640 1 -- 172.21.15.63:0/2914892784 learned_addr learned my addr 172.21.15.63:0/2914892784 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.569+0000 7f0503e00640 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 0x557eb9e43c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.569+0000 7f0502a00640 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e44180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.570+0000 7f0503e00640 1 -- 172.21.15.63:0/2914892784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 msgr2=0x557eb9e43c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.570+0000 7f0503e00640 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 0x557eb9e43c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.570+0000 7f0503e00640 1 -- 172.21.15.63:0/2914892784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 msgr2=0x557eb9e44180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.570+0000 7f0503e00640 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e44180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.570+0000 7f0503e00640 1 -- 172.21.15.63:0/2914892784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb9aaf210 con 0x557eb9a971b0 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.570+0000 7f0503e00640 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4fec0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f04fc004cf0 tx=0x7f04fc0059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725a1fcbac5410dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.570+0000 7f04ed600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.570+0000 7f04ed600640 1 -- 172.21.15.63:0/2914892784 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc016070 con 0x557eb9a971b0 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.570+0000 7f04ed600640 1 -- 172.21.15.63:0/2914892784 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f04fc012ae0 con 0x557eb9a971b0 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.570+0000 7f051165c740 1 -- 172.21.15.63:0/2914892784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04fc004d20 con 0x557eb9a971b0 2024-05-27T07:58:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.571+0000 7f04ed600640 1 -- 172.21.15.63:0/2914892784 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f04fc004ed0 con 0x557eb9a971b0 2024-05-27T07:58:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.571+0000 7f051165c740 10 client.20226 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.571+0000 7f051165c740 1 -- 172.21.15.63:0/2914892784 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557eb9b7b620 con 0x557eb9a971b0 2024-05-27T07:58:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.571+0000 7f04ed600640 1 -- 172.21.15.63:0/2914892784 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04fc004d20 con 0x557eb9a971b0 2024-05-27T07:58:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.571+0000 7f04ed600640 1 -- 172.21.15.63:0/2914892784 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f04fc012ce0 con 0x557eb9a971b0 2024-05-27T07:58:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.571+0000 7f051165c740 10 client.20226 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.571+0000 7f051165c740 10 client.20226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.571+0000 7f051165c740 20 client.20226 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.572+0000 7f051165c740 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557eb9a55d80 0x557eb9e55da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.572+0000 7f051165c740 4 client.20226 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.572+0000 7f051165c740 1 -- 172.21.15.63:0/2914892784 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eb9dacae0 con 0x557eb9a55d80 2024-05-27T07:58:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.572+0000 7f0502a00640 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557eb9a55d80 0x557eb9e55da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.573+0000 7f0502a00640 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557eb9a55d80 0x557eb9e55da0 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277b90d824d7123c server_cookie=a83f6c3816b8a9f5 in_seq=0 out_seq=0 2024-05-27T07:58:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.573+0000 7f04ed600640 10 client.20226 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.575+0000 7f04ed600640 1 -- 172.21.15.63:0/2914892784 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557eb9dacae0 con 0x557eb9a55d80 2024-05-27T07:58:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.575+0000 7f04ed600640 10 client.20226 handle_command_reply: tid=0 2024-05-27T07:58:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.575+0000 7f051165c740 1 client.20226 shutdown 2024-05-27T07:58:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 1 -- 172.21.15.63:0/2914892784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 msgr2=0x557eb9e4fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4fec0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f04fc004cf0 tx=0x7f04fc0059b0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 1 -- 172.21.15.63:0/2914892784 shutdown_connections 2024-05-27T07:58:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557eb9a55d80 0x557eb9e55da0 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4fec0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e44180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 1 --2- 172.21.15.63:0/2914892784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a6e2f0 0x557eb9e43c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 1 -- 172.21.15.63:0/2914892784 >> 172.21.15.63:0/2914892784 conn(0x557eb9a8b300 msgr2=0x557eb9e39890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f04ed600640 0 client.20226 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 1 -- 172.21.15.63:0/2914892784 shutdown_connections 2024-05-27T07:58:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 1 -- 172.21.15.63:0/2914892784 wait complete. 2024-05-27T07:58:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.576+0000 7f051165c740 20 client.20226 trim_cache size 0 max 16384 2024-05-27T07:58:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.601+0000 7f051165c740 1 Processor -- start 2024-05-27T07:58:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.601+0000 7f051165c740 1 -- start start 2024-05-27T07:58:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.602+0000 7f051165c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a6e2f0 0x557eb9e4fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.602+0000 7f051165c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a764d0 0x557eb9e50170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.602+0000 7f051165c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.602+0000 7f051165c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05041b5be0 con 0x557eb9a971b0 2024-05-27T07:58:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.602+0000 7f051165c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05041b34b0 con 0x557eb9a6e2f0 2024-05-27T07:58:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.602+0000 7f051165c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05041b1320 con 0x557eb9a764d0 2024-05-27T07:58:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.602+0000 7f0503e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.602+0000 7f0503e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58192/0 (socket says 172.21.15.63:58192) 2024-05-27T07:58:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.602+0000 7f0503e00640 1 -- 172.21.15.63:0/1917355735 learned_addr learned my addr 172.21.15.63:0/1917355735 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.602+0000 7f0502a00640 1 --2- 172.21.15.63:0/1917355735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a764d0 0x557eb9e50170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.603+0000 7f0503e00640 1 --2- 172.21.15.63:0/1917355735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a6e2f0 0x557eb9e4fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.603+0000 7f0503e00640 1 -- 172.21.15.63:0/1917355735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a764d0 msgr2=0x557eb9e50170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.603+0000 7f0503e00640 1 --2- 172.21.15.63:0/1917355735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a764d0 0x557eb9e50170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.603+0000 7f0503e00640 1 -- 172.21.15.63:0/1917355735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a6e2f0 msgr2=0x557eb9e4fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.603+0000 7f0503e00640 1 --2- 172.21.15.63:0/1917355735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a6e2f0 0x557eb9e4fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.603+0000 7f0503e00640 1 -- 172.21.15.63:0/1917355735 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb9e3adf0 con 0x557eb9a971b0 2024-05-27T07:58:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.603+0000 7f0503e00640 1 --2- 172.21.15.63:0/1917355735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4be20 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f04fc004cc0 tx=0x7f04fc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba4f3970656654f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.604+0000 7f04efe00640 1 -- 172.21.15.63:0/1917355735 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc016070 con 0x557eb9a971b0 2024-05-27T07:58:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.604+0000 7f04efe00640 1 -- 172.21.15.63:0/1917355735 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04fc010c40 con 0x557eb9a971b0 2024-05-27T07:58:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.604+0000 7f04efe00640 1 -- 172.21.15.63:0/1917355735 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc01a8a0 con 0x557eb9a971b0 2024-05-27T07:58:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.605+0000 7f051165c740 1 -- 172.21.15.63:0/1917355735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 msgr2=0x557eb9e4be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.605+0000 7f051165c740 1 --2- 172.21.15.63:0/1917355735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4be20 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f04fc004cc0 tx=0x7f04fc010740 comp rx=0 tx=0).stop 2024-05-27T07:58:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.605+0000 7f051165c740 1 -- 172.21.15.63:0/1917355735 shutdown_connections 2024-05-27T07:58:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.605+0000 7f051165c740 1 --2- 172.21.15.63:0/1917355735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a971b0 0x557eb9e4be20 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.605+0000 7f051165c740 1 --2- 172.21.15.63:0/1917355735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a764d0 0x557eb9e50170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.605+0000 7f051165c740 1 --2- 172.21.15.63:0/1917355735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a6e2f0 0x557eb9e4fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.605+0000 7f051165c740 1 -- 172.21.15.63:0/1917355735 >> 172.21.15.63:0/1917355735 conn(0x557eb9a8b300 msgr2=0x557eb9e3ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.605+0000 7f051165c740 1 -- 172.21.15.63:0/1917355735 shutdown_connections 2024-05-27T07:58:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.605+0000 7f051165c740 1 -- 172.21.15.63:0/1917355735 wait complete. 2024-05-27T07:58:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.605+0000 7f051165c740 1 -- start start 2024-05-27T07:58:39.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.606+0000 7f051165c740 1 Processor -- start 2024-05-27T07:58:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.606+0000 7f051165c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a6e2f0 0x557eb9e4b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.606+0000 7f051165c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.606+0000 7f051165c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a971b0 0x557eb9eb52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.606+0000 7f051165c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05041b1970 con 0x557eb9a6e2f0 2024-05-27T07:58:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.606+0000 7f051165c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05041b2de0 con 0x557eb9a764d0 2024-05-27T07:58:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.606+0000 7f051165c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05041b2040 con 0x557eb9a971b0 2024-05-27T07:58:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.606+0000 7f051165c740 10 client.0 _ll_register_callbacks cb 0x7f04f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.606+0000 7f0503e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a6e2f0 0x557eb9e4b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.607+0000 7f0503e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a6e2f0 0x557eb9e4b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58198/0 (socket says 172.21.15.63:58198) 2024-05-27T07:58:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.607+0000 7f0503e00640 1 -- 172.21.15.63:0/2853351297 learned_addr learned my addr 172.21.15.63:0/2853351297 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.607+0000 7f0502a00640 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.607+0000 7f0503e00640 1 -- 172.21.15.63:0/2853351297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a971b0 msgr2=0x557eb9eb52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.607+0000 7f0503e00640 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a971b0 0x557eb9eb52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.607+0000 7f0503e00640 1 -- 172.21.15.63:0/2853351297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 msgr2=0x557eb9e4bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.607+0000 7f0503e00640 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e4bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.607+0000 7f0503e00640 1 -- 172.21.15.63:0/2853351297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb9eb5840 con 0x557eb9a6e2f0 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.607+0000 7f0503e00640 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a6e2f0 0x557eb9e4b640 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009c20 tx=0x7f04fc0126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9d08f4a0fd689b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.607+0000 7f04ed600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.608+0000 7f04ed600640 1 -- 172.21.15.63:0/2853351297 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc016040 con 0x557eb9a6e2f0 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.608+0000 7f04ed600640 1 -- 172.21.15.63:0/2853351297 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9483+0+0 (secure 0 0 0) 0x7f04fc012ce0 con 0x557eb9a6e2f0 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.608+0000 7f051165c740 1 -- 172.21.15.63:0/2853351297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04fc010c40 con 0x557eb9a6e2f0 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.608+0000 7f04ed600640 1 -- 172.21.15.63:0/2853351297 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f04fc04a050 con 0x557eb9a6e2f0 2024-05-27T07:58:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.608+0000 7f051165c740 10 client.20232 fetch_fsmap learned FSMap version 33 2024-05-27T07:58:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.608+0000 7f051165c740 1 -- 172.21.15.63:0/2853351297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557eb9ebdd30 con 0x557eb9a6e2f0 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.608+0000 7f04ed600640 1 -- 172.21.15.63:0/2853351297 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04fc010c40 con 0x557eb9a6e2f0 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.609+0000 7f04ed600640 1 -- 172.21.15.63:0/2853351297 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f04fc042020 con 0x557eb9a6e2f0 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.609+0000 7f051165c740 10 client.20232 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.609+0000 7f051165c740 10 client.20232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.609+0000 7f051165c740 20 client.20232 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.609+0000 7f051165c740 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557eb9a2b1c0 0x557eb9ec0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.609+0000 7f051165c740 4 client.20232 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.609+0000 7f051165c740 1 -- 172.21.15.63:0/2853351297 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557eb9da62c0 con 0x557eb9a2b1c0 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.609+0000 7f0502a00640 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557eb9a2b1c0 0x557eb9ec0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.610+0000 7f0502a00640 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557eb9a2b1c0 0x557eb9ec0390 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9d187a9ef1cb99 server_cookie=58800de69c225981 in_seq=0 out_seq=0 2024-05-27T07:58:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.610+0000 7f04ed600640 10 client.20232 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.644+0000 7f04ed600640 1 -- 172.21.15.63:0/2853351297 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557eb9da62c0 con 0x557eb9a2b1c0 2024-05-27T07:58:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.644+0000 7f04ed600640 10 client.20232 handle_command_reply: tid=0 2024-05-27T07:58:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.644+0000 7f051165c740 1 client.20232 shutdown 2024-05-27T07:58:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.644+0000 7f051165c740 1 -- 172.21.15.63:0/2853351297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a6e2f0 msgr2=0x557eb9e4b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.644+0000 7f051165c740 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a6e2f0 0x557eb9e4b640 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009c20 tx=0x7f04fc0126d0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.644+0000 7f051165c740 1 -- 172.21.15.63:0/2853351297 shutdown_connections 2024-05-27T07:58:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.644+0000 7f051165c740 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557eb9a2b1c0 0x557eb9ec0390 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.645+0000 7f051165c740 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557eb9a971b0 0x557eb9eb52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.645+0000 7f051165c740 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557eb9a764d0 0x557eb9e4bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.645+0000 7f051165c740 1 --2- 172.21.15.63:0/2853351297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557eb9a6e2f0 0x557eb9e4b640 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.645+0000 7f051165c740 1 -- 172.21.15.63:0/2853351297 >> 172.21.15.63:0/2853351297 conn(0x557eb9a8b300 msgr2=0x557eb9e4ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.645+0000 7f04ed600640 0 client.20232 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.645+0000 7f051165c740 1 -- 172.21.15.63:0/2853351297 shutdown_connections 2024-05-27T07:58:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.645+0000 7f051165c740 1 -- 172.21.15.63:0/2853351297 wait complete. 2024-05-27T07:58:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.645+0000 7f051165c740 20 client.20232 trim_cache size 0 max 16384 2024-05-27T07:58:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:58:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:58:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:58:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.646+0000 7f0509000640 1 -- 172.21.15.63:0/2732057007 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04e80b29d0 msgr2=0x7f04e80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.646+0000 7f0509000640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04e80b29d0 0x7f04e80b4ee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x557eb9e37550 tx=0x7f04f400bf60 comp rx=0 tx=0).stop 2024-05-27T07:58:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.646+0000 7f0509000640 1 -- 172.21.15.63:0/2732057007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05041027c0 msgr2=0x7f05041a25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.646+0000 7f0509000640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05041027c0 0x7f05041a25b0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f04f400d950 tx=0x7f04f400de10 comp rx=0 tx=0).stop 2024-05-27T07:58:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.646+0000 7f0509000640 1 -- 172.21.15.63:0/2732057007 shutdown_connections 2024-05-27T07:58:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.646+0000 7f0509000640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04e80b29d0 0x7f04e80b4ee0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.646+0000 7f0509000640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0504110360 0x7f050419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.646+0000 7f0509000640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f050410da00 0x7f05041a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.646+0000 7f0509000640 1 --2- 172.21.15.63:0/2732057007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05041027c0 0x7f05041a25b0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.646+0000 7f0509000640 1 -- 172.21.15.63:0/2732057007 >> 172.21.15.63:0/2732057007 conn(0x7f05040fe210 msgr2=0x7f05041111a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.647+0000 7f0509000640 1 -- 172.21.15.63:0/2732057007 shutdown_connections 2024-05-27T07:58:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:39.647+0000 7f0509000640 1 -- 172.21.15.63:0/2732057007 wait complete. 2024-05-27T07:58:41.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:41 smithi086 ceph-mon[34687]: pgmap v794: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 5 op/s 2024-05-27T07:58:41.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:41 smithi149 ceph-mon[35408]: pgmap v794: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 5 op/s 2024-05-27T07:58:41.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:41 smithi063 ceph-mon[31613]: pgmap v794: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 5 op/s 2024-05-27T07:58:41.851 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:41 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 2. 2024-05-27T07:58:41.851 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:41 smithi086 systemd[1]: Stopped Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:58:41.851 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:41 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Consumed 8.760s CPU time. 2024-05-27T07:58:41.851 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:41 smithi086 systemd[1]: Starting Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T07:58:42.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:41 smithi086 podman[76205]: 2024-05-27 07:58:41.755059699 +0000 UTC m=+0.036230848 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:58:42.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:41 smithi086 podman[76205]: 2024-05-27 07:58:41.894930383 +0000 UTC m=+0.176101571 container create 7af8edee5dccf7a0c451139f94be84fc3cab96a0b2315797c20f02f0610944a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:58:42.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:42 smithi086 podman[76205]: 2024-05-27 07:58:42.207574255 +0000 UTC m=+0.488745423 container init 7af8edee5dccf7a0c451139f94be84fc3cab96a0b2315797c20f02f0610944a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T07:58:42.460 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:42 smithi086 podman[76205]: 2024-05-27 07:58:42.21743561 +0000 UTC m=+0.498606802 container start 7af8edee5dccf7a0c451139f94be84fc3cab96a0b2315797c20f02f0610944a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T07:58:42.461 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:42 smithi086 bash[76205]: 7af8edee5dccf7a0c451139f94be84fc3cab96a0b2315797c20f02f0610944a2 2024-05-27T07:58:42.461 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:42 smithi086 systemd[1]: Started Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T07:58:42.461 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:42 smithi086 ceph-mds[76223]: -- 172.21.15.86:0/655640988 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e7ff33a40 con 0x558e7ff7b180 2024-05-27T07:58:42.461 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 07:58:42 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: starting mds.k at 2024-05-27T07:58:43.071 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:58:43.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:43 smithi086 ceph-mon[34687]: pgmap v795: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:58:43.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:43 smithi086 ceph-mon[34687]: daemon mds.k restarted 2024-05-27T07:58:43.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:43 smithi149 ceph-mon[35408]: pgmap v795: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:58:43.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:43 smithi149 ceph-mon[35408]: daemon mds.k restarted 2024-05-27T07:58:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:43 smithi063 ceph-mon[31613]: pgmap v795: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 10 op/s 2024-05-27T07:58:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:43 smithi063 ceph-mon[31613]: daemon mds.k restarted 2024-05-27T07:58:43.604 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:44.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:44 smithi086 ceph-mon[34687]: osdmap e132: 12 total, 12 up, 12 in 2024-05-27T07:58:44.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:44 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6832/3062218679,v1:172.21.15.86:6833/3062218679] up:boot 2024-05-27T07:58:44.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:44 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T07:58:44.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:58:44.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:44 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:58:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:44 smithi149 ceph-mon[35408]: osdmap e132: 12 total, 12 up, 12 in 2024-05-27T07:58:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:44 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6832/3062218679,v1:172.21.15.86:6833/3062218679] up:boot 2024-05-27T07:58:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:44 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T07:58:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:58:44.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:44 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:58:44.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:44 smithi063 ceph-mon[31613]: osdmap e132: 12 total, 12 up, 12 in 2024-05-27T07:58:44.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:44 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6832/3062218679,v1:172.21.15.86:6833/3062218679] up:boot 2024-05-27T07:58:44.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:44 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T07:58:44.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:58:44.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:44 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T07:58:45.572 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:45 smithi063 ceph-mon[31613]: pgmap v797: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:58:45.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:45 smithi086 ceph-mon[34687]: pgmap v797: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:58:45.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:45 smithi149 ceph-mon[35408]: pgmap v797: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:58:45.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.985+0000 7fa29f200700 1 -- 172.21.15.63:0/766904652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a0105c50 msgr2=0x7fa2a010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:45.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.985+0000 7fa29f200700 1 --2- 172.21.15.63:0/766904652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a0105c50 0x7fa2a010e1a0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fa288009990 tx=0x7fa288009ca0 comp rx=0 tx=0).stop 2024-05-27T07:58:45.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.985+0000 7fa29f200700 1 -- 172.21.15.63:0/766904652 shutdown_connections 2024-05-27T07:58:45.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.985+0000 7fa29f200700 1 --2- 172.21.15.63:0/766904652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2a010e730 0x7fa2a0110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:45.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.985+0000 7fa29f200700 1 --2- 172.21.15.63:0/766904652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2a0105300 0x7fa2a0105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:45.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.985+0000 7fa29f200700 1 --2- 172.21.15.63:0/766904652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a0105c50 0x7fa2a010e1a0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:45.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.985+0000 7fa29f200700 1 -- 172.21.15.63:0/766904652 >> 172.21.15.63:0/766904652 conn(0x7fa2a0078ab0 msgr2=0x7fa2a0078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:45.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.985+0000 7fa29f200700 1 -- 172.21.15.63:0/766904652 shutdown_connections 2024-05-27T07:58:45.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.985+0000 7fa29f200700 1 -- 172.21.15.63:0/766904652 wait complete. 2024-05-27T07:58:45.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.986+0000 7fa29f200700 1 Processor -- start 2024-05-27T07:58:45.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.987+0000 7fa29f200700 1 -- start start 2024-05-27T07:58:45.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.987+0000 7fa29f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a0105300 0x7fa2a0102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:45.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.987+0000 7fa29f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2a0105c50 0x7fa2a0103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:45.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.987+0000 7fa29f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2a010e730 0x7fa2a0103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:45.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.987+0000 7fa29f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0114100 con 0x7fa2a0105300 2024-05-27T07:58:45.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.987+0000 7fa29f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0113f80 con 0x7fa2a0105c50 2024-05-27T07:58:45.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.987+0000 7fa29f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0114280 con 0x7fa2a010e730 2024-05-27T07:58:45.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.987+0000 7fa29de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a0105300 0x7fa2a0102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:45.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.988+0000 7fa29de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a0105300 0x7fa2a0102ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58220/0 (socket says 172.21.15.63:58220) 2024-05-27T07:58:45.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.988+0000 7fa29de00700 1 -- 172.21.15.63:0/190815362 learned_addr learned my addr 172.21.15.63:0/190815362 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:45.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.988+0000 7fa295400700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2a0105c50 0x7fa2a0103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:45.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.988+0000 7fa29e800700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2a010e730 0x7fa2a0103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.988+0000 7fa29de00700 1 -- 172.21.15.63:0/190815362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2a010e730 msgr2=0x7fa2a0103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.988+0000 7fa29de00700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2a010e730 0x7fa2a0103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.988+0000 7fa29de00700 1 -- 172.21.15.63:0/190815362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2a0105c50 msgr2=0x7fa2a0103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.988+0000 7fa29de00700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2a0105c50 0x7fa2a0103530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.988+0000 7fa29de00700 1 -- 172.21.15.63:0/190815362 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a01a8ea0 con 0x7fa2a0105300 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.989+0000 7fa29de00700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a0105300 0x7fa2a0102ff0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fa29000d950 tx=0x7fa29000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.989+0000 7fa297400700 1 -- 172.21.15.63:0/190815362 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290016070 con 0x7fa2a0105300 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.989+0000 7fa297400700 1 -- 172.21.15.63:0/190815362 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa290009dd0 con 0x7fa2a0105300 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.989+0000 7fa297400700 1 -- 172.21.15.63:0/190815362 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29000f3c0 con 0x7fa2a0105300 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.989+0000 7fa29f200700 1 -- 172.21.15.63:0/190815362 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a01a9190 con 0x7fa2a0105300 2024-05-27T07:58:45.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.989+0000 7fa29f200700 1 -- 172.21.15.63:0/190815362 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a01a9650 con 0x7fa2a0105300 2024-05-27T07:58:45.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.993+0000 7fa297400700 1 -- 172.21.15.63:0/190815362 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa290019070 con 0x7fa2a0105300 2024-05-27T07:58:45.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.994+0000 7fa2b3b18b80 1 Processor -- start 2024-05-27T07:58:45.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.995+0000 7fa2b3b18b80 1 -- start start 2024-05-27T07:58:45.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.995+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244314eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:45.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.995+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x562443159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:45.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.995+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x56244315c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:45.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.995+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56244307de40 con 0x562442fd0e40 2024-05-27T07:58:45.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.995+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56244306d9e0 con 0x56244314f410 2024-05-27T07:58:45.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.995+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562443071a20 con 0x56244314f820 2024-05-27T07:58:45.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.996+0000 7fa297400700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2840b29f0 0x7fa2840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:45.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.996+0000 7fa29e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244314eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.996+0000 7fa29e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244314eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58236/0 (socket says 172.21.15.63:58236) 2024-05-27T07:58:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.996+0000 7fa29e800700 1 -- 172.21.15.63:0/4079518996 learned_addr learned my addr 172.21.15.63:0/4079518996 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.996+0000 7fa29de00700 1 --2- 172.21.15.63:0/4079518996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x562443159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.996+0000 7fa297400700 1 -- 172.21.15.63:0/190815362 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7fa2900e2660 con 0x7fa2a0105300 2024-05-27T07:58:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.996+0000 7fa295400700 1 --2- 172.21.15.63:0/4079518996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x56244315c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:45.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.996+0000 7fa29e800700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2840b29f0 0x7fa2840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.997+0000 7fa29de00700 1 -- 172.21.15.63:0/4079518996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 msgr2=0x56244315c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.997+0000 7fa29de00700 1 --2- 172.21.15.63:0/4079518996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x56244315c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.997+0000 7fa29de00700 1 -- 172.21.15.63:0/4079518996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 msgr2=0x56244314eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.997+0000 7fa29de00700 1 --2- 172.21.15.63:0/4079518996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244314eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.997+0000 7fa29de00700 1 -- 172.21.15.63:0/4079518996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562443036420 con 0x56244314f410 2024-05-27T07:58:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.997+0000 7fa29de00700 1 --2- 172.21.15.63:0/4079518996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x562443159ad0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa2900df9e0 tx=0x7fa29009ddf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d617940ff39868b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.997+0000 7fa29e800700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2840b29f0 0x7fa2840b4f00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x56244314b690 tx=0x7fa298004b40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.998+0000 7fa28f400700 1 -- 172.21.15.63:0/4079518996 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900a9c70 con 0x56244314f410 2024-05-27T07:58:46.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.998+0000 7fa28f400700 1 -- 172.21.15.63:0/4079518996 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa290016210 con 0x56244314f410 2024-05-27T07:58:46.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.998+0000 7fa28f400700 1 -- 172.21.15.63:0/4079518996 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29001ed50 con 0x56244314f410 2024-05-27T07:58:46.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.999+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/4079518996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 msgr2=0x562443159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.999+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/4079518996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x562443159ad0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa2900df9e0 tx=0x7fa29009ddf0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.999+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/4079518996 shutdown_connections 2024-05-27T07:58:46.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.999+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/4079518996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x56244315c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.999+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/4079518996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244314eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.999+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/4079518996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x562443159ad0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.999+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/4079518996 >> 172.21.15.63:0/4079518996 conn(0x562443030fc0 msgr2=0x56244314ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.999+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/4079518996 shutdown_connections 2024-05-27T07:58:46.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:45.999+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/4079518996 wait complete. 2024-05-27T07:58:46.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.000+0000 7fa2b3b18b80 1 -- start start 2024-05-27T07:58:46.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.000+0000 7fa2b3b18b80 1 Processor -- start 2024-05-27T07:58:46.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.001+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562442fd0e40 0x562443159570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.001+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x562443159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.001+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f820 0x562443159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.001+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562442f73fa0 con 0x56244314f820 2024-05-27T07:58:46.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.001+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56244300e760 con 0x56244314f410 2024-05-27T07:58:46.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.001+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56244315c990 con 0x562442fd0e40 2024-05-27T07:58:46.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.001+0000 7fa2b3b18b80 10 client.0 _ll_register_callbacks cb 0x562443017c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:46.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.001+0000 7fa29e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f820 0x562443159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.002+0000 7fa29e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f820 0x562443159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58250/0 (socket says 172.21.15.63:58250) 2024-05-27T07:58:46.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.002+0000 7fa29e800700 1 -- 172.21.15.63:0/2852923209 learned_addr learned my addr 172.21.15.63:0/2852923209 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:46.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.002+0000 7fa29de00700 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562442fd0e40 0x562443159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.002+0000 7fa295400700 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x562443159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.002+0000 7fa29e800700 1 -- 172.21.15.63:0/2852923209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562442fd0e40 msgr2=0x562443159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.002+0000 7fa29e800700 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562442fd0e40 0x562443159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.002+0000 7fa29e800700 1 -- 172.21.15.63:0/2852923209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 msgr2=0x562443159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.002+0000 7fa29e800700 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x562443159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.002+0000 7fa29e800700 1 -- 172.21.15.63:0/2852923209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562442fbd8d0 con 0x56244314f820 2024-05-27T07:58:46.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.003+0000 7fa29e800700 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f820 0x562443159ff0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fa298012a60 tx=0x7fa298004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5ed752679b3b1d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.003+0000 7fa283e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:46.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.003+0000 7fa283e00700 1 -- 172.21.15.63:0/2852923209 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298029060 con 0x56244314f820 2024-05-27T07:58:46.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.003+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2852923209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa290016210 con 0x56244314f820 2024-05-27T07:58:46.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.003+0000 7fa283e00700 1 -- 172.21.15.63:0/2852923209 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7fa298004350 con 0x56244314f820 2024-05-27T07:58:46.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.004+0000 7fa283e00700 1 -- 172.21.15.63:0/2852923209 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa298030050 con 0x56244314f820 2024-05-27T07:58:46.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.004+0000 7fa2b3b18b80 10 client.20244 fetch_fsmap learned FSMap version 35 2024-05-27T07:58:46.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.004+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2852923209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56244303e060 con 0x56244314f820 2024-05-27T07:58:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.004+0000 7fa283e00700 1 -- 172.21.15.63:0/2852923209 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa290016210 con 0x56244314f820 2024-05-27T07:58:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.004+0000 7fa283e00700 1 -- 172.21.15.63:0/2852923209 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa298024b90 con 0x56244314f820 2024-05-27T07:58:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.004+0000 7fa2b3b18b80 10 client.20244 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:58:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.004+0000 7fa2b3b18b80 10 client.20244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.004+0000 7fa2b3b18b80 20 client.20244 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.005+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624431720e0 0x562443174500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.005+0000 7fa2b3b18b80 4 client.20244 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:46.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.005+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2852923209 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56244307fd90 con 0x5624431720e0 2024-05-27T07:58:46.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.005+0000 7fa29de00700 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624431720e0 0x562443174500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.006+0000 7fa29de00700 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624431720e0 0x562443174500 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e431d0217735b4 server_cookie=31f68918196b9f7c in_seq=0 out_seq=0 2024-05-27T07:58:46.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.006+0000 7fa283e00700 10 client.20244 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.008+0000 7fa283e00700 1 -- 172.21.15.63:0/2852923209 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56244307fd90 con 0x5624431720e0 2024-05-27T07:58:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.008+0000 7fa283e00700 10 client.20244 handle_command_reply: tid=0 2024-05-27T07:58:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.008+0000 7fa2b3b18b80 1 client.20244 shutdown 2024-05-27T07:58:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2852923209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f820 msgr2=0x562443159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f820 0x562443159ff0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fa298012a60 tx=0x7fa298004ad0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2852923209 shutdown_connections 2024-05-27T07:58:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624431720e0 0x562443174500 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f820 0x562443159ff0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562442fd0e40 0x562443159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/2852923209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x562443159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa283e00700 0 client.20244 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:46.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2852923209 >> 172.21.15.63:0/2852923209 conn(0x562443030fc0 msgr2=0x562443149ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2852923209 shutdown_connections 2024-05-27T07:58:46.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2852923209 wait complete. 2024-05-27T07:58:46.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.009+0000 7fa2b3b18b80 20 client.20244 trim_cache size 0 max 16384 2024-05-27T07:58:46.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.036+0000 7fa2b3b18b80 1 Processor -- start 2024-05-27T07:58:46.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.036+0000 7fa2b3b18b80 1 -- start start 2024-05-27T07:58:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562442fd0e40 0x5624431537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f410 0x56244315ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x562443165a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01b0950 con 0x56244314f410 2024-05-27T07:58:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01ae1e0 con 0x562442fd0e40 2024-05-27T07:58:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01ac030 con 0x56244314f820 2024-05-27T07:58:46.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa295400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f410 0x56244315ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa295400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f410 0x56244315ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58254/0 (socket says 172.21.15.63:58254) 2024-05-27T07:58:46.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa295400700 1 -- 172.21.15.63:0/2625244167 learned_addr learned my addr 172.21.15.63:0/2625244167 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa29de00700 1 --2- 172.21.15.63:0/2625244167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562442fd0e40 0x5624431537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.037+0000 7fa29e800700 1 --2- 172.21.15.63:0/2625244167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x562443165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.038+0000 7fa295400700 1 -- 172.21.15.63:0/2625244167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 msgr2=0x562443165a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.038+0000 7fa295400700 1 --2- 172.21.15.63:0/2625244167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x562443165a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.038+0000 7fa295400700 1 -- 172.21.15.63:0/2625244167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562442fd0e40 msgr2=0x5624431537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.038+0000 7fa295400700 1 --2- 172.21.15.63:0/2625244167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562442fd0e40 0x5624431537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.038+0000 7fa295400700 1 -- 172.21.15.63:0/2625244167 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562443158fa0 con 0x56244314f410 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.038+0000 7fa295400700 1 --2- 172.21.15.63:0/2625244167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f410 0x56244315ba10 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa288000c00 tx=0x7fa288005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db9efa23d61d9ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.039+0000 7fa28ea00700 1 -- 172.21.15.63:0/2625244167 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288027020 con 0x56244314f410 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.039+0000 7fa28ea00700 1 -- 172.21.15.63:0/2625244167 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa288005570 con 0x56244314f410 2024-05-27T07:58:46.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.039+0000 7fa28ea00700 1 -- 172.21.15.63:0/2625244167 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28800bc00 con 0x56244314f410 2024-05-27T07:58:46.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.040+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2625244167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f410 msgr2=0x56244315ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.040+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/2625244167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f410 0x56244315ba10 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa288000c00 tx=0x7fa288005960 comp rx=0 tx=0).stop 2024-05-27T07:58:46.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.040+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2625244167 shutdown_connections 2024-05-27T07:58:46.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.040+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/2625244167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x562443165a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.040+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/2625244167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562442fd0e40 0x5624431537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.040+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/2625244167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56244314f410 0x56244315ba10 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.040+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2625244167 >> 172.21.15.63:0/2625244167 conn(0x562443030fc0 msgr2=0x562443158890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.040+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2625244167 shutdown_connections 2024-05-27T07:58:46.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.040+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/2625244167 wait complete. 2024-05-27T07:58:46.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.041+0000 7fa2b3b18b80 1 -- start start 2024-05-27T07:58:46.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.041+0000 7fa2b3b18b80 1 Processor -- start 2024-05-27T07:58:46.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244315dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x56244315e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa2b3b18b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x56244315e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01ac6a0 con 0x562442fd0e40 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01adb10 con 0x56244314f410 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa2b3b18b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01acd70 con 0x56244314f820 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa2b3b18b80 10 client.0 _ll_register_callbacks cb 0x562443017c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa29de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244315dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa29de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244315dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58258/0 (socket says 172.21.15.63:58258) 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa29de00700 1 -- 172.21.15.63:0/194590787 learned_addr learned my addr 172.21.15.63:0/194590787 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa29e800700 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x56244315e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa295400700 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x56244315e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa29de00700 1 -- 172.21.15.63:0/194590787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 msgr2=0x56244315e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.042+0000 7fa29de00700 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x56244315e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.043+0000 7fa29de00700 1 -- 172.21.15.63:0/194590787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 msgr2=0x56244315e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.043+0000 7fa29de00700 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x56244315e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.043+0000 7fa29de00700 1 -- 172.21.15.63:0/194590787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624431dbd30 con 0x562442fd0e40 2024-05-27T07:58:46.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.043+0000 7fa295400700 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x56244315e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.043+0000 7fa29de00700 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244315dba0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fa290010fd0 tx=0x7fa2900a5560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203308fc6c5a312b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.043+0000 7fa283e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.043+0000 7fa283e00700 1 -- 172.21.15.63:0/194590787 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900c2070 con 0x562442fd0e40 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.043+0000 7fa283e00700 1 -- 172.21.15.63:0/194590787 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7fa29006e680 con 0x562442fd0e40 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.043+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/194590787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa288005570 con 0x562442fd0e40 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa283e00700 1 -- 172.21.15.63:0/194590787 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa290020040 con 0x562442fd0e40 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa2b3b18b80 10 client.20250 fetch_fsmap learned FSMap version 35 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/194590787 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5624431dcfa0 con 0x562442fd0e40 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa283e00700 1 -- 172.21.15.63:0/194590787 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa288005570 con 0x562442fd0e40 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa283e00700 1 -- 172.21.15.63:0/194590787 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa2900e2210 con 0x562442fd0e40 2024-05-27T07:58:46.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa2b3b18b80 10 client.20250 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:58:46.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa2b3b18b80 10 client.20250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:46.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa2b3b18b80 20 client.20250 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:46.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624431dd170 0x5624431df590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa2b3b18b80 4 client.20250 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:58:46.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.044+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/194590787 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562443079570 con 0x5624431dd170 2024-05-27T07:58:46.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.045+0000 7fa295400700 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624431dd170 0x5624431df590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.047+0000 7fa295400700 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624431dd170 0x5624431df590 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67774403f071271 server_cookie=44b69af1ce13ab63 in_seq=0 out_seq=0 2024-05-27T07:58:46.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.047+0000 7fa283e00700 10 client.20250 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.047+0000 7fa283e00700 1 -- 172.21.15.63:0/194590787 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562443079570 con 0x5624431dd170 2024-05-27T07:58:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.047+0000 7fa283e00700 10 client.20250 handle_command_reply: tid=0 2024-05-27T07:58:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.047+0000 7fa2b3b18b80 1 client.20250 shutdown 2024-05-27T07:58:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/194590787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 msgr2=0x56244315dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244315dba0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fa290010fd0 tx=0x7fa2900a5560 comp rx=0 tx=0).stop 2024-05-27T07:58:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/194590787 shutdown_connections 2024-05-27T07:58:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5624431dd170 0x5624431df590 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56244314f820 0x56244315e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562442fd0e40 0x56244315dba0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 1 --2- 172.21.15.63:0/194590787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56244314f410 0x56244315e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/194590787 >> 172.21.15.63:0/194590787 conn(0x562443030fc0 msgr2=0x5624431585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa283e00700 0 client.20250 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/194590787 shutdown_connections 2024-05-27T07:58:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 1 -- 172.21.15.63:0/194590787 wait complete. 2024-05-27T07:58:46.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.048+0000 7fa2b3b18b80 20 client.20250 trim_cache size 0 max 16384 2024-05-27T07:58:46.051 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:58:46.051 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:58:46.052 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:58:46.052 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:58:46.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.049+0000 7fa29f200700 1 -- 172.21.15.63:0/190815362 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2840b29f0 msgr2=0x7fa2840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.049+0000 7fa29f200700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2840b29f0 0x7fa2840b4f00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x56244314b690 tx=0x7fa298004b40 comp rx=0 tx=0).stop 2024-05-27T07:58:46.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.049+0000 7fa29f200700 1 -- 172.21.15.63:0/190815362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a0105300 msgr2=0x7fa2a0102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.049+0000 7fa29f200700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a0105300 0x7fa2a0102ff0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fa29000d950 tx=0x7fa29000dc60 comp rx=0 tx=0).stop 2024-05-27T07:58:46.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.049+0000 7fa29f200700 1 -- 172.21.15.63:0/190815362 shutdown_connections 2024-05-27T07:58:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.049+0000 7fa29f200700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa2840b29f0 0x7fa2840b4f00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.049+0000 7fa29f200700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2a010e730 0x7fa2a0103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.049+0000 7fa29f200700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a0105300 0x7fa2a0102ff0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.049+0000 7fa29f200700 1 --2- 172.21.15.63:0/190815362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2a0105c50 0x7fa2a0103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.049+0000 7fa29f200700 1 -- 172.21.15.63:0/190815362 >> 172.21.15.63:0/190815362 conn(0x7fa2a0078ab0 msgr2=0x7fa2a00fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.050+0000 7fa29f200700 1 -- 172.21.15.63:0/190815362 shutdown_connections 2024-05-27T07:58:46.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:46.050+0000 7fa29f200700 1 -- 172.21.15.63:0/190815362 wait complete. 2024-05-27T07:58:47.013 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:58:47.013 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:58:47.529 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:47.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:47 smithi063 ceph-mon[31613]: pgmap v798: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:58:47.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:47 smithi086 ceph-mon[34687]: pgmap v798: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:58:47.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:47 smithi149 ceph-mon[35408]: pgmap v798: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.4 MiB/s wr, 9 op/s 2024-05-27T07:58:49.427 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:49 smithi063 ceph-mon[31613]: pgmap v799: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 11 op/s 2024-05-27T07:58:49.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:49 smithi086 ceph-mon[34687]: pgmap v799: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 11 op/s 2024-05-27T07:58:49.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:49 smithi149 ceph-mon[35408]: pgmap v799: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 11 op/s 2024-05-27T07:58:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.037+0000 7ffa13e00700 1 -- 172.21.15.63:0/1818362312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa14105c90 msgr2=0x7ffa1410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.037+0000 7ffa13e00700 1 --2- 172.21.15.63:0/1818362312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa14105c90 0x7ffa1410e1e0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009990 tx=0x7ff9fc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.037+0000 7ffa13e00700 1 -- 172.21.15.63:0/1818362312 shutdown_connections 2024-05-27T07:58:50.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.037+0000 7ffa13e00700 1 --2- 172.21.15.63:0/1818362312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa1410e720 0x7ffa14110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.037+0000 7ffa13e00700 1 --2- 172.21.15.63:0/1818362312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa14105340 0x7ffa14105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.037+0000 7ffa13e00700 1 --2- 172.21.15.63:0/1818362312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa14105c90 0x7ffa1410e1e0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.037+0000 7ffa13e00700 1 -- 172.21.15.63:0/1818362312 >> 172.21.15.63:0/1818362312 conn(0x7ffa14078ab0 msgr2=0x7ffa14078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.037+0000 7ffa13e00700 1 -- 172.21.15.63:0/1818362312 shutdown_connections 2024-05-27T07:58:50.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.038+0000 7ffa13e00700 1 -- 172.21.15.63:0/1818362312 wait complete. 2024-05-27T07:58:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.039+0000 7ffa13e00700 1 Processor -- start 2024-05-27T07:58:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.039+0000 7ffa13e00700 1 -- start start 2024-05-27T07:58:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa13e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa14105340 0x7ffa1419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa13e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa14105c90 0x7ffa1419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa13e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1410e720 0x7ffa141a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa13e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14114150 con 0x7ffa1410e720 2024-05-27T07:58:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa13e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14113fd0 con 0x7ffa14105340 2024-05-27T07:58:50.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa13e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141142d0 con 0x7ffa14105c90 2024-05-27T07:58:50.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa13400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1410e720 0x7ffa141a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:50.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa13400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1410e720 0x7ffa141a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60428/0 (socket says 172.21.15.63:60428) 2024-05-27T07:58:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa13400700 1 -- 172.21.15.63:0/150350309 learned_addr learned my addr 172.21.15.63:0/150350309 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa12a00700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa14105340 0x7ffa1419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.040+0000 7ffa0a000700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa14105c90 0x7ffa1419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.041+0000 7ffa13400700 1 -- 172.21.15.63:0/150350309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa14105c90 msgr2=0x7ffa1419d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.041+0000 7ffa13400700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa14105c90 0x7ffa1419d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.041+0000 7ffa13400700 1 -- 172.21.15.63:0/150350309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa14105340 msgr2=0x7ffa1419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.041+0000 7ffa13400700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa14105340 0x7ffa1419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.041+0000 7ffa13400700 1 -- 172.21.15.63:0/150350309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa141a1cb0 con 0x7ffa1410e720 2024-05-27T07:58:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.041+0000 7ffa13400700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1410e720 0x7ffa141a1610 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c007ee0 tx=0x7ffa0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:50.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.042+0000 7ffa18a00700 1 -- 172.21.15.63:0/150350309 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c004bb0 con 0x7ffa1410e720 2024-05-27T07:58:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.042+0000 7ffa13e00700 1 -- 172.21.15.63:0/150350309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa141a1fa0 con 0x7ffa1410e720 2024-05-27T07:58:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.042+0000 7ffa18a00700 1 -- 172.21.15.63:0/150350309 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa0c004d50 con 0x7ffa1410e720 2024-05-27T07:58:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.042+0000 7ffa18a00700 1 -- 172.21.15.63:0/150350309 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0058b0 con 0x7ffa1410e720 2024-05-27T07:58:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.042+0000 7ffa13e00700 1 -- 172.21.15.63:0/150350309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa141a98c0 con 0x7ffa1410e720 2024-05-27T07:58:50.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.043+0000 7ffa13e00700 1 -- 172.21.15.63:0/150350309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa140512b0 con 0x7ffa1410e720 2024-05-27T07:58:50.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.047+0000 7ffa18a00700 1 -- 172.21.15.63:0/150350309 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffa0c01a020 con 0x7ffa1410e720 2024-05-27T07:58:50.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.049+0000 7ffa18a00700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9f80b29f0 0x7ff9f80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:50.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.049+0000 7ffa18a00700 1 -- 172.21.15.63:0/150350309 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7ffa0c0e26e0 con 0x7ffa1410e720 2024-05-27T07:58:50.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.050+0000 7ffa12a00700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9f80b29f0 0x7ff9f80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:50.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.050+0000 7ffa12a00700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9f80b29f0 0x7ff9f80b4f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009c60 tx=0x7ffa04009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:50.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.051+0000 7ffa18a00700 1 -- 172.21.15.63:0/150350309 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa0c010070 con 0x7ffa1410e720 2024-05-27T07:58:50.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.372+0000 7ffa13e00700 1 -- 172.21.15.63:0/150350309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa1402e5e0 con 0x7ffa1410e720 2024-05-27T07:58:50.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.375+0000 7ffa18a00700 1 -- 172.21.15.63:0/150350309 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7ffa0c0a9cf0 con 0x7ffa1410e720 2024-05-27T07:58:50.378 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:58:50.378 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T07:58:50.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.379+0000 7ffa13e00700 1 -- 172.21.15.63:0/150350309 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9f80b29f0 msgr2=0x7ff9f80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:50.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.379+0000 7ffa13e00700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9f80b29f0 0x7ff9f80b4f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009c60 tx=0x7ffa04009260 comp rx=0 tx=0).stop 2024-05-27T07:58:50.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.379+0000 7ffa13e00700 1 -- 172.21.15.63:0/150350309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1410e720 msgr2=0x7ffa141a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:50.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.379+0000 7ffa13e00700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1410e720 0x7ffa141a1610 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c007ee0 tx=0x7ffa0c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.379+0000 7ffa13e00700 1 -- 172.21.15.63:0/150350309 shutdown_connections 2024-05-27T07:58:50.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.379+0000 7ffa13e00700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff9f80b29f0 0x7ff9f80b4f00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.379+0000 7ffa13e00700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa1410e720 0x7ffa141a1610 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.379+0000 7ffa13e00700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa14105340 0x7ffa1419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.379+0000 7ffa13e00700 1 --2- 172.21.15.63:0/150350309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa14105c90 0x7ffa1419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:50.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.379+0000 7ffa13e00700 1 -- 172.21.15.63:0/150350309 >> 172.21.15.63:0/150350309 conn(0x7ffa14078ab0 msgr2=0x7ffa14104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:50.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.380+0000 7ffa13e00700 1 -- 172.21.15.63:0/150350309 shutdown_connections 2024-05-27T07:58:50.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:50.380+0000 7ffa13e00700 1 -- 172.21.15.63:0/150350309 wait complete. 2024-05-27T07:58:50.383 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T07:58:51.391 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:58:51.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:51 smithi086 ceph-mon[34687]: pgmap v800: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 11 op/s 2024-05-27T07:58:51.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:51 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/150350309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:58:51.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:51 smithi149 ceph-mon[35408]: pgmap v800: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 11 op/s 2024-05-27T07:58:51.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:51 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/150350309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:58:51.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:51 smithi063 ceph-mon[31613]: pgmap v800: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.2 MiB/s wr, 11 op/s 2024-05-27T07:58:51.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:51 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/150350309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:58:51.896 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.898+0000 7f901ee00640 1 -- 172.21.15.63:0/4056356180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9018104bb0 msgr2=0x7f9018104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.898+0000 7f901ee00640 1 --2- 172.21.15.63:0/4056356180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9018104bb0 0x7f9018104fc0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f900c009900 tx=0x7f900c02a410 comp rx=0 tx=0).stop 2024-05-27T07:58:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.898+0000 7f901ee00640 1 -- 172.21.15.63:0/4056356180 shutdown_connections 2024-05-27T07:58:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.898+0000 7f901ee00640 1 --2- 172.21.15.63:0/4056356180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f901810e000 0x7f90181104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.898+0000 7f901ee00640 1 --2- 172.21.15.63:0/4056356180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f901810b610 0x7f901810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.898+0000 7f901ee00640 1 --2- 172.21.15.63:0/4056356180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9018104bb0 0x7f9018104fc0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.898+0000 7f901ee00640 1 -- 172.21.15.63:0/4056356180 >> 172.21.15.63:0/4056356180 conn(0x7f90180fe210 msgr2=0x7f9018100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.898+0000 7f901ee00640 1 -- 172.21.15.63:0/4056356180 shutdown_connections 2024-05-27T07:58:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.898+0000 7f901ee00640 1 -- 172.21.15.63:0/4056356180 wait complete. 2024-05-27T07:58:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.899+0000 7f901ee00640 1 Processor -- start 2024-05-27T07:58:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.900+0000 7f901ee00640 1 -- start start 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.900+0000 7f901ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9018104bb0 0x7f9018102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.900+0000 7f901ee00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f901810b610 0x7f90181029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.900+0000 7f901ee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f901810e000 0x7f9018102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.900+0000 7f901ee00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9018113b10 con 0x7f901810b610 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.900+0000 7f901ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9018113990 con 0x7f9018104bb0 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.900+0000 7f901ee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9018113c90 con 0x7f901810e000 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.901+0000 7f901d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f901810b610 0x7f90181029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.901+0000 7f901d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f901810b610 0x7f90181029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60446/0 (socket says 172.21.15.63:60446) 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.901+0000 7f901d000640 1 -- 172.21.15.63:0/2752798353 learned_addr learned my addr 172.21.15.63:0/2752798353 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.901+0000 7f901da00640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9018104bb0 0x7f9018102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.901+0000 7f901e400640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f901810e000 0x7f9018102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.901+0000 7f901d000640 1 -- 172.21.15.63:0/2752798353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f901810e000 msgr2=0x7f9018102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.901+0000 7f901d000640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f901810e000 0x7f9018102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.901+0000 7f901d000640 1 -- 172.21.15.63:0/2752798353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9018104bb0 msgr2=0x7f9018102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.901+0000 7f901d000640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9018104bb0 0x7f9018102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.901+0000 7f901d000640 1 -- 172.21.15.63:0/2752798353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90181a8930 con 0x7f901810b610 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.902+0000 7f901d000640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f901810b610 0x7f90181029b0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f900400d770 tx=0x7f900400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.902+0000 7f9016000640 1 -- 172.21.15.63:0/2752798353 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004017070 con 0x7f901810b610 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.902+0000 7f9016000640 1 -- 172.21.15.63:0/2752798353 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90040054b0 con 0x7f901810b610 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.903+0000 7f901ee00640 1 -- 172.21.15.63:0/2752798353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90181a8c20 con 0x7f901810b610 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.903+0000 7f9016000640 1 -- 172.21.15.63:0/2752798353 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004004c60 con 0x7f901810b610 2024-05-27T07:58:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.903+0000 7f901ee00640 1 -- 172.21.15.63:0/2752798353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90181a9170 con 0x7f901810b610 2024-05-27T07:58:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.906+0000 7f902742d740 1 Processor -- start 2024-05-27T07:58:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.907+0000 7f902742d740 1 -- start start 2024-05-27T07:58:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.907+0000 7f902742d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277618f1b0 0x562776531a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.907+0000 7f902742d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627761662f0 0x562776531fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.907+0000 7f902742d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277616e4d0 0x5627765324e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.907+0000 7f902742d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5627764a2b90 con 0x56277618f1b0 2024-05-27T07:58:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.907+0000 7f902742d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562776492730 con 0x5627761662f0 2024-05-27T07:58:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.907+0000 7f902742d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562776496770 con 0x56277616e4d0 2024-05-27T07:58:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.907+0000 7f901e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277618f1b0 0x562776531a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.907+0000 7f901e400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277618f1b0 0x562776531a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60452/0 (socket says 172.21.15.63:60452) 2024-05-27T07:58:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.907+0000 7f901e400640 1 -- 172.21.15.63:0/2256265651 learned_addr learned my addr 172.21.15.63:0/2256265651 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.908+0000 7f901da00640 1 --2- 172.21.15.63:0/2256265651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627761662f0 0x562776531fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.908+0000 7f901e400640 1 -- 172.21.15.63:0/2256265651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277616e4d0 msgr2=0x5627765324e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.908+0000 7f901e400640 1 --2- 172.21.15.63:0/2256265651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277616e4d0 0x5627765324e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.908+0000 7f901e400640 1 -- 172.21.15.63:0/2256265651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627761662f0 msgr2=0x562776531fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.908+0000 7f901e400640 1 --2- 172.21.15.63:0/2256265651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627761662f0 0x562776531fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.908+0000 7f901e400640 1 -- 172.21.15.63:0/2256265651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56277642b8b0 con 0x56277618f1b0 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.908+0000 7f901e400640 1 --2- 172.21.15.63:0/2256265651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277618f1b0 0x562776531a60 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f9010009f70 tx=0x7f9010010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67cc18b571dc153 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.909+0000 7f900aa00640 1 -- 172.21.15.63:0/2256265651 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010016070 con 0x56277618f1b0 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.909+0000 7f900aa00640 1 -- 172.21.15.63:0/2256265651 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9010004d20 con 0x56277618f1b0 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.909+0000 7f900aa00640 1 -- 172.21.15.63:0/2256265651 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010012720 con 0x56277618f1b0 2024-05-27T07:58:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.909+0000 7f9016000640 1 -- 172.21.15.63:0/2752798353 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9004004e00 con 0x7f901810b610 2024-05-27T07:58:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.910+0000 7f902742d740 1 -- 172.21.15.63:0/2256265651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277618f1b0 msgr2=0x562776531a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.910+0000 7f902742d740 1 --2- 172.21.15.63:0/2256265651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277618f1b0 0x562776531a60 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f9010009f70 tx=0x7f9010010800 comp rx=0 tx=0).stop 2024-05-27T07:58:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.910+0000 7f902742d740 1 -- 172.21.15.63:0/2256265651 shutdown_connections 2024-05-27T07:58:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.910+0000 7f902742d740 1 --2- 172.21.15.63:0/2256265651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277616e4d0 0x5627765324e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.910+0000 7f902742d740 1 --2- 172.21.15.63:0/2256265651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627761662f0 0x562776531fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.910+0000 7f902742d740 1 --2- 172.21.15.63:0/2256265651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277618f1b0 0x562776531a60 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.910+0000 7f902742d740 1 -- 172.21.15.63:0/2256265651 >> 172.21.15.63:0/2256265651 conn(0x562776183300 msgr2=0x56277652e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.910+0000 7f902742d740 1 -- 172.21.15.63:0/2256265651 shutdown_connections 2024-05-27T07:58:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.910+0000 7f902742d740 1 -- 172.21.15.63:0/2256265651 wait complete. 2024-05-27T07:58:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.911+0000 7f902742d740 1 -- start start 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.912+0000 7f902742d740 1 Processor -- start 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.912+0000 7f9016000640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90000b28e0 0x7f90000b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.912+0000 7f9016000640 1 -- 172.21.15.63:0/2752798353 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f90040e22c0 con 0x7f901810b610 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.912+0000 7f901e400640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90000b28e0 0x7f90000b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.912+0000 7f902742d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x56277653bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.912+0000 7f902742d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277616e4d0 0x56277653c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.912+0000 7f902742d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277618f1b0 0x562776547ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.912+0000 7f902742d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562776178790 con 0x5627761662f0 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.912+0000 7f902742d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56277643abc0 con 0x56277618f1b0 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.912+0000 7f902742d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56277653eb90 con 0x56277616e4d0 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f902742d740 10 client.0 _ll_register_callbacks cb 0x562776418400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x56277653bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x56277653bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60454/0 (socket says 172.21.15.63:60454) 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901da00640 1 -- 172.21.15.63:0/197045031 learned_addr learned my addr 172.21.15.63:0/197045031 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901d000640 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277616e4d0 0x56277653c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901e400640 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277618f1b0 0x562776547ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901e400640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90000b28e0 0x7f90000b4df0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9010000f80 tx=0x7f9010010fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901da00640 1 -- 172.21.15.63:0/197045031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277616e4d0 msgr2=0x56277653c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901da00640 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277616e4d0 0x56277653c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901da00640 1 -- 172.21.15.63:0/197045031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277618f1b0 msgr2=0x562776547ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901da00640 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277618f1b0 0x562776547ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.913+0000 7f901da00640 1 -- 172.21.15.63:0/197045031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627761a7210 con 0x5627761662f0 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.914+0000 7f901da00640 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x56277653bee0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f900c02a930 tx=0x7f900c031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a250e2e6cf803dd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.914+0000 7f8fff400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.914+0000 7f8fff400640 1 -- 172.21.15.63:0/197045031 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c035070 con 0x5627761662f0 2024-05-27T07:58:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.914+0000 7f902742d740 1 -- 172.21.15.63:0/197045031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9010004d20 con 0x5627761662f0 2024-05-27T07:58:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.914+0000 7f8fff400640 1 -- 172.21.15.63:0/197045031 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f900c004040 con 0x5627761662f0 2024-05-27T07:58:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f8fff400640 1 -- 172.21.15.63:0/197045031 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f900c04e050 con 0x5627761662f0 2024-05-27T07:58:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f902742d740 10 client.20268 fetch_fsmap learned FSMap version 35 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f902742d740 1 -- 172.21.15.63:0/197045031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562776273620 con 0x5627761662f0 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f8fff400640 1 -- 172.21.15.63:0/197045031 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9010004d20 con 0x5627761662f0 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f8fff400640 1 -- 172.21.15.63:0/197045031 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f900c043020 con 0x5627761662f0 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f902742d740 10 client.20268 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f902742d740 10 client.20268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f902742d740 20 client.20268 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f902742d740 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56277614dd80 0x56277654df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f902742d740 4 client.20268 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.915+0000 7f902742d740 1 -- 172.21.15.63:0/197045031 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627764a4ae0 con 0x56277614dd80 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.916+0000 7f901d000640 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56277614dd80 0x56277654df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.917+0000 7f901d000640 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56277614dd80 0x56277654df40 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af76adc9733f19eb server_cookie=76e10076804887f0 in_seq=0 out_seq=0 2024-05-27T07:58:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.917+0000 7f8fff400640 10 client.20268 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f8fff400640 1 -- 172.21.15.63:0/197045031 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5627764a4ae0 con 0x56277614dd80 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f8fff400640 10 client.20268 handle_command_reply: tid=0 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 client.20268 shutdown 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 -- 172.21.15.63:0/197045031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 msgr2=0x56277653bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x56277653bee0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f900c02a930 tx=0x7f900c031de0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 -- 172.21.15.63:0/197045031 shutdown_connections 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56277614dd80 0x56277654df40 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277618f1b0 0x562776547ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277616e4d0 0x56277653c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 --2- 172.21.15.63:0/197045031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x56277653bee0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f8fff400640 0 client.20268 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 -- 172.21.15.63:0/197045031 >> 172.21.15.63:0/197045031 conn(0x562776183300 msgr2=0x562776533780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 -- 172.21.15.63:0/197045031 shutdown_connections 2024-05-27T07:58:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.919+0000 7f902742d740 1 -- 172.21.15.63:0/197045031 wait complete. 2024-05-27T07:58:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.920+0000 7f902742d740 20 client.20268 trim_cache size 0 max 16384 2024-05-27T07:58:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.943+0000 7f902742d740 1 Processor -- start 2024-05-27T07:58:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.943+0000 7f902742d740 1 -- start start 2024-05-27T07:58:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.943+0000 7f902742d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x562776547c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.943+0000 7f902742d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277616e4d0 0x562776548170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.943+0000 7f902742d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277618f1b0 0x562776553700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.943+0000 7f902742d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90181b5a00 con 0x5627761662f0 2024-05-27T07:58:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.943+0000 7f902742d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90181b32d0 con 0x56277616e4d0 2024-05-27T07:58:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.943+0000 7f902742d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90181b1120 con 0x56277618f1b0 2024-05-27T07:58:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x562776547c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x562776547c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60468/0 (socket says 172.21.15.63:60468) 2024-05-27T07:58:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901da00640 1 -- 172.21.15.63:0/3392234081 learned_addr learned my addr 172.21.15.63:0/3392234081 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901e400640 1 --2- 172.21.15.63:0/3392234081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277618f1b0 0x562776553700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901d000640 1 --2- 172.21.15.63:0/3392234081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277616e4d0 0x562776548170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901da00640 1 -- 172.21.15.63:0/3392234081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277618f1b0 msgr2=0x562776553700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901da00640 1 --2- 172.21.15.63:0/3392234081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277618f1b0 0x562776553700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901da00640 1 -- 172.21.15.63:0/3392234081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277616e4d0 msgr2=0x562776548170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901da00640 1 --2- 172.21.15.63:0/3392234081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277616e4d0 0x562776548170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901da00640 1 -- 172.21.15.63:0/3392234081 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627765361f0 con 0x5627761662f0 2024-05-27T07:58:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.944+0000 7f901d000640 1 --2- 172.21.15.63:0/3392234081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277616e4d0 0x562776548170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.945+0000 7f901da00640 1 --2- 172.21.15.63:0/3392234081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x562776547c30 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f900c003df0 tx=0x7f900c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b765b0ee0d05357 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.945+0000 7f900a000640 1 -- 172.21.15.63:0/3392234081 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c057070 con 0x5627761662f0 2024-05-27T07:58:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.945+0000 7f900a000640 1 -- 172.21.15.63:0/3392234081 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f900c004470 con 0x5627761662f0 2024-05-27T07:58:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.945+0000 7f900a000640 1 -- 172.21.15.63:0/3392234081 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c035040 con 0x5627761662f0 2024-05-27T07:58:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.946+0000 7f902742d740 1 -- 172.21.15.63:0/3392234081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 msgr2=0x562776547c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.946+0000 7f902742d740 1 --2- 172.21.15.63:0/3392234081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x562776547c30 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f900c003df0 tx=0x7f900c02a410 comp rx=0 tx=0).stop 2024-05-27T07:58:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.946+0000 7f902742d740 1 -- 172.21.15.63:0/3392234081 shutdown_connections 2024-05-27T07:58:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.946+0000 7f902742d740 1 --2- 172.21.15.63:0/3392234081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277618f1b0 0x562776553700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.946+0000 7f902742d740 1 --2- 172.21.15.63:0/3392234081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56277616e4d0 0x562776548170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.946+0000 7f902742d740 1 --2- 172.21.15.63:0/3392234081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627761662f0 0x562776547c30 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.946+0000 7f902742d740 1 -- 172.21.15.63:0/3392234081 >> 172.21.15.63:0/3392234081 conn(0x562776183300 msgr2=0x56277653bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.947+0000 7f902742d740 1 -- 172.21.15.63:0/3392234081 shutdown_connections 2024-05-27T07:58:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.947+0000 7f902742d740 1 -- 172.21.15.63:0/3392234081 wait complete. 2024-05-27T07:58:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.947+0000 7f902742d740 1 -- start start 2024-05-27T07:58:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.947+0000 7f902742d740 1 Processor -- start 2024-05-27T07:58:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f902742d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627761662f0 0x5627765442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f902742d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277616e4d0 0x56277653f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f902742d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277618f1b0 0x56277653fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f902742d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90181b1790 con 0x56277616e4d0 2024-05-27T07:58:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f902742d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90181b2c00 con 0x5627761662f0 2024-05-27T07:58:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f902742d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90181b1e60 con 0x56277618f1b0 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f902742d740 10 client.0 _ll_register_callbacks cb 0x7f900c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f901d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277616e4d0 0x56277653f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f901d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277616e4d0 0x56277653f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60484/0 (socket says 172.21.15.63:60484) 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f901d000640 1 -- 172.21.15.63:0/1005369120 learned_addr learned my addr 172.21.15.63:0/1005369120 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.948+0000 7f901da00640 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627761662f0 0x5627765442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f901e400640 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277618f1b0 0x56277653fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f901d000640 1 -- 172.21.15.63:0/1005369120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277618f1b0 msgr2=0x56277653fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f901d000640 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277618f1b0 0x56277653fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f901d000640 1 -- 172.21.15.63:0/1005369120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627761662f0 msgr2=0x5627765442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f901d000640 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627761662f0 0x5627765442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f901d000640 1 -- 172.21.15.63:0/1005369120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562776540090 con 0x56277616e4d0 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f901d000640 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277616e4d0 0x56277653f610 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f90040dfca0 tx=0x7f9004005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f52302ddca15e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f8fff400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f8fff400640 1 -- 172.21.15.63:0/1005369120 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90040c3060 con 0x56277616e4d0 2024-05-27T07:58:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f8fff400640 1 -- 172.21.15.63:0/1005369120 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f90040df920 con 0x56277616e4d0 2024-05-27T07:58:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.949+0000 7f902742d740 1 -- 172.21.15.63:0/1005369120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f900c004470 con 0x56277616e4d0 2024-05-27T07:58:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f8fff400640 1 -- 172.21.15.63:0/1005369120 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9004020040 con 0x56277616e4d0 2024-05-27T07:58:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f902742d740 10 client.20274 fetch_fsmap learned FSMap version 35 2024-05-27T07:58:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f902742d740 1 -- 172.21.15.63:0/1005369120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5627765b62b0 con 0x56277616e4d0 2024-05-27T07:58:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f8fff400640 1 -- 172.21.15.63:0/1005369120 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f900c004470 con 0x56277616e4d0 2024-05-27T07:58:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f8fff400640 1 -- 172.21.15.63:0/1005369120 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f90040a5570 con 0x56277616e4d0 2024-05-27T07:58:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f902742d740 10 client.20274 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:58:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f902742d740 10 client.20274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f902742d740 20 client.20274 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f902742d740 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627761231c0 0x5627765b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f902742d740 4 client.20274 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:58:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.950+0000 7f902742d740 1 -- 172.21.15.63:0/1005369120 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56277649e2c0 con 0x5627761231c0 2024-05-27T07:58:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.951+0000 7f901da00640 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627761231c0 0x5627765b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.952+0000 7f8fff400640 10 client.20274 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.952+0000 7f901da00640 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627761231c0 0x5627765b89d0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eaeaacfdfe789e7 server_cookie=c15d091c80918e92 in_seq=0 out_seq=0 2024-05-27T07:58:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.992+0000 7f8fff400640 1 -- 172.21.15.63:0/1005369120 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56277649e2c0 con 0x5627761231c0 2024-05-27T07:58:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.992+0000 7f8fff400640 10 client.20274 handle_command_reply: tid=0 2024-05-27T07:58:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.992+0000 7f902742d740 1 client.20274 shutdown 2024-05-27T07:58:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f902742d740 1 -- 172.21.15.63:0/1005369120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277616e4d0 msgr2=0x56277653f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f902742d740 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277616e4d0 0x56277653f610 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f90040dfca0 tx=0x7f9004005660 comp rx=0 tx=0).stop 2024-05-27T07:58:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f902742d740 1 -- 172.21.15.63:0/1005369120 shutdown_connections 2024-05-27T07:58:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f902742d740 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5627761231c0 0x5627765b89d0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f902742d740 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56277618f1b0 0x56277653fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f902742d740 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56277616e4d0 0x56277653f610 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f902742d740 1 --2- 172.21.15.63:0/1005369120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627761662f0 0x5627765442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f902742d740 1 -- 172.21.15.63:0/1005369120 >> 172.21.15.63:0/1005369120 conn(0x562776183300 msgr2=0x5627765317f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f8fff400640 0 client.20274 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:58:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f902742d740 1 -- 172.21.15.63:0/1005369120 shutdown_connections 2024-05-27T07:58:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.993+0000 7f902742d740 1 -- 172.21.15.63:0/1005369120 wait complete. 2024-05-27T07:58:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.994+0000 7f902742d740 20 client.20274 trim_cache size 0 max 16384 2024-05-27T07:58:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:58:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:58:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:58:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 -- 172.21.15.63:0/2752798353 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90000b28e0 msgr2=0x7f90000b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90000b28e0 0x7f90000b4df0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9010000f80 tx=0x7f9010010fd0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 -- 172.21.15.63:0/2752798353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f901810b610 msgr2=0x7f90181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f901810b610 0x7f90181029b0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f900400d770 tx=0x7f900400dc30 comp rx=0 tx=0).stop 2024-05-27T07:58:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 -- 172.21.15.63:0/2752798353 shutdown_connections 2024-05-27T07:58:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90000b28e0 0x7f90000b4df0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f901810e000 0x7f9018102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f901810b610 0x7f90181029b0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 --2- 172.21.15.63:0/2752798353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9018104bb0 0x7f9018102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 -- 172.21.15.63:0/2752798353 >> 172.21.15.63:0/2752798353 conn(0x7f90180fe210 msgr2=0x7f901810e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 -- 172.21.15.63:0/2752798353 shutdown_connections 2024-05-27T07:58:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:58:52.995+0000 7f901ee00640 1 -- 172.21.15.63:0/2752798353 wait complete. 2024-05-27T07:58:53.689 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:53 smithi063 ceph-mon[31613]: pgmap v801: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 9 op/s 2024-05-27T07:58:53.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:53 smithi086 ceph-mon[34687]: pgmap v801: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 9 op/s 2024-05-27T07:58:53.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:53 smithi149 ceph-mon[35408]: pgmap v801: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 9 op/s 2024-05-27T07:58:55.353 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:55 smithi063 ceph-mon[31613]: pgmap v802: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 8 op/s 2024-05-27T07:58:55.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:55 smithi086 ceph-mon[34687]: pgmap v802: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 8 op/s 2024-05-27T07:58:55.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:55 smithi149 ceph-mon[35408]: pgmap v802: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.9 MiB/s wr, 8 op/s 2024-05-27T07:58:55.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.759+0000 7f4d6fe00700 1 -- 172.21.15.63:0/148321384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d70105280 msgr2=0x7f4d70105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.759+0000 7f4d6fe00700 1 --2- 172.21.15.63:0/148321384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d70105280 0x7f4d70105690 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009990 tx=0x7f4d60009ca0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.759+0000 7f4d6fe00700 1 -- 172.21.15.63:0/148321384 shutdown_connections 2024-05-27T07:58:55.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.759+0000 7f4d6fe00700 1 --2- 172.21.15.63:0/148321384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d7010e6f0 0x7f4d70110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.759+0000 7f4d6fe00700 1 --2- 172.21.15.63:0/148321384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d70105280 0x7f4d70105690 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.759+0000 7f4d6fe00700 1 --2- 172.21.15.63:0/148321384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d70105bd0 0x7f4d7010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.759+0000 7f4d6fe00700 1 -- 172.21.15.63:0/148321384 >> 172.21.15.63:0/148321384 conn(0x7f4d70078ab0 msgr2=0x7f4d70078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.760+0000 7f4d6fe00700 1 -- 172.21.15.63:0/148321384 shutdown_connections 2024-05-27T07:58:55.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.760+0000 7f4d6fe00700 1 -- 172.21.15.63:0/148321384 wait complete. 2024-05-27T07:58:55.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.761+0000 7f4d6fe00700 1 Processor -- start 2024-05-27T07:58:55.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.761+0000 7f4d6fe00700 1 -- start start 2024-05-27T07:58:55.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.762+0000 7f4d6fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d70105280 0x7f4d70102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.762+0000 7f4d6fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70105bd0 0x7f4d701034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.762+0000 7f4d6fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7010e6f0 0x7f4d701039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.762+0000 7f4d6fe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d70114140 con 0x7f4d70105280 2024-05-27T07:58:55.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.762+0000 7f4d6fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d70113fc0 con 0x7f4d70105bd0 2024-05-27T07:58:55.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.762+0000 7f4d6fe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701142c0 con 0x7f4d7010e6f0 2024-05-27T07:58:55.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.762+0000 7f4d6ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d70105280 0x7f4d70102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.762+0000 7f4d6ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d70105280 0x7f4d70102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60496/0 (socket says 172.21.15.63:60496) 2024-05-27T07:58:55.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.762+0000 7f4d6ea00700 1 -- 172.21.15.63:0/3365662512 learned_addr learned my addr 172.21.15.63:0/3365662512 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:55.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.763+0000 7f4d6e000700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70105bd0 0x7f4d701034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.763+0000 7f4d6f400700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7010e6f0 0x7f4d701039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.763+0000 7f4d6ea00700 1 -- 172.21.15.63:0/3365662512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7010e6f0 msgr2=0x7f4d701039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.763+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7010e6f0 0x7f4d701039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.763+0000 7f4d6ea00700 1 -- 172.21.15.63:0/3365662512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70105bd0 msgr2=0x7f4d701034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.763+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70105bd0 0x7f4d701034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.763+0000 7f4d6ea00700 1 -- 172.21.15.63:0/3365662512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d701a8f10 con 0x7f4d70105280 2024-05-27T07:58:55.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.763+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d70105280 0x7f4d70102f70 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4d60005910 tx=0x7f4d600047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.764+0000 7f4d67400700 1 -- 172.21.15.63:0/3365662512 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60004830 con 0x7f4d70105280 2024-05-27T07:58:55.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.764+0000 7f4d6fe00700 1 -- 172.21.15.63:0/3365662512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d701a91a0 con 0x7f4d70105280 2024-05-27T07:58:55.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.764+0000 7f4d67400700 1 -- 172.21.15.63:0/3365662512 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d60005020 con 0x7f4d70105280 2024-05-27T07:58:55.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.764+0000 7f4d67400700 1 -- 172.21.15.63:0/3365662512 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000f960 con 0x7f4d70105280 2024-05-27T07:58:55.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.764+0000 7f4d6fe00700 1 -- 172.21.15.63:0/3365662512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d701a95a0 con 0x7f4d70105280 2024-05-27T07:58:55.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.768+0000 7f4d67400700 1 -- 172.21.15.63:0/3365662512 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4d600054d0 con 0x7f4d70105280 2024-05-27T07:58:55.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.769+0000 7f4d84106b80 1 Processor -- start 2024-05-27T07:58:55.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.769+0000 7f4d84106b80 1 -- start start 2024-05-27T07:58:55.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.770+0000 7f4d67400700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4d540b29f0 0x7f4d540b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.770+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.770+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.770+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 0x5649bd5db6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.770+0000 7f4d84106b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5649bd4fce40 con 0x5649bd53b9f0 2024-05-27T07:58:55.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.770+0000 7f4d84106b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5649bd4ec9e0 con 0x5649bd5d92c0 2024-05-27T07:58:55.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.770+0000 7f4d84106b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5649bd4f0a20 con 0x5649bd44fe40 2024-05-27T07:58:55.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.770+0000 7f4d67400700 1 -- 172.21.15.63:0/3365662512 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f4d600e37e0 con 0x7f4d70105280 2024-05-27T07:58:55.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60500/0 (socket says 172.21.15.63:60500) 2024-05-27T07:58:55.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6ea00700 1 -- 172.21.15.63:0/3476912870 learned_addr learned my addr 172.21.15.63:0/3476912870 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:55.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6e000700 1 --2- 172.21.15.63:0/3476912870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6f400700 1 --2- 172.21.15.63:0/3476912870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 0x5649bd5db6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6f400700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4d540b29f0 0x7f4d540b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6ea00700 1 -- 172.21.15.63:0/3476912870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 msgr2=0x5649bd5cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/3476912870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6ea00700 1 -- 172.21.15.63:0/3476912870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 msgr2=0x5649bd5db6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/3476912870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 0x5649bd5db6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.771+0000 7f4d6ea00700 1 -- 172.21.15.63:0/3476912870 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649bd4b5420 con 0x5649bd53b9f0 2024-05-27T07:58:55.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.772+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/3476912870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5cf750 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f4d600a6860 tx=0x7f4d600a67d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18674106e41fbbb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.772+0000 7f4d6f400700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4d540b29f0 0x7f4d540b4f00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x5649bd5d1210 tx=0x7f4d68009680 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.772+0000 7f4d5be00700 1 -- 172.21.15.63:0/3476912870 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d600aa2f0 con 0x5649bd53b9f0 2024-05-27T07:58:55.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.772+0000 7f4d5be00700 1 -- 172.21.15.63:0/3476912870 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d600aaee0 con 0x5649bd53b9f0 2024-05-27T07:58:55.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.772+0000 7f4d5be00700 1 -- 172.21.15.63:0/3476912870 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d600b35b0 con 0x5649bd53b9f0 2024-05-27T07:58:55.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.773+0000 7f4d84106b80 1 -- 172.21.15.63:0/3476912870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 msgr2=0x5649bd5cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.773+0000 7f4d84106b80 1 --2- 172.21.15.63:0/3476912870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5cf750 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f4d600a6860 tx=0x7f4d600a67d0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.773+0000 7f4d84106b80 1 -- 172.21.15.63:0/3476912870 shutdown_connections 2024-05-27T07:58:55.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.773+0000 7f4d84106b80 1 --2- 172.21.15.63:0/3476912870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 0x5649bd5db6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.773+0000 7f4d84106b80 1 --2- 172.21.15.63:0/3476912870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5cf750 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.773+0000 7f4d84106b80 1 --2- 172.21.15.63:0/3476912870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.773+0000 7f4d84106b80 1 -- 172.21.15.63:0/3476912870 >> 172.21.15.63:0/3476912870 conn(0x5649bd4e9390 msgr2=0x5649bd5ca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.774+0000 7f4d84106b80 1 -- 172.21.15.63:0/3476912870 shutdown_connections 2024-05-27T07:58:55.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.774+0000 7f4d84106b80 1 -- 172.21.15.63:0/3476912870 wait complete. 2024-05-27T07:58:55.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.775+0000 7f4d84106b80 1 -- start start 2024-05-27T07:58:55.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.775+0000 7f4d84106b80 1 Processor -- start 2024-05-27T07:58:55.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd53b9f0 0x5649bd5d6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d84106b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5649bd3f2fa0 con 0x5649bd5d92c0 2024-05-27T07:58:55.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d84106b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5649bd48d760 con 0x5649bd53b9f0 2024-05-27T07:58:55.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d84106b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5649bd5dbc20 con 0x5649bd44fe40 2024-05-27T07:58:55.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d84106b80 10 client.0 _ll_register_callbacks cb 0x5649bd496c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d6f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d6f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5d6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60508/0 (socket says 172.21.15.63:60508) 2024-05-27T07:58:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d6f400700 1 -- 172.21.15.63:0/2391419996 learned_addr learned my addr 172.21.15.63:0/2391419996 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d6e000700 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd53b9f0 0x5649bd5d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.776+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.777+0000 7f4d6f400700 1 -- 172.21.15.63:0/2391419996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 msgr2=0x5649bd5d62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.777+0000 7f4d6f400700 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5d62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.777+0000 7f4d6f400700 1 -- 172.21.15.63:0/2391419996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd53b9f0 msgr2=0x5649bd5d6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.777+0000 7f4d6f400700 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd53b9f0 0x5649bd5d6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.777+0000 7f4d6f400700 1 -- 172.21.15.63:0/2391419996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649bd43c8d0 con 0x5649bd5d92c0 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.777+0000 7f4d6f400700 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5d6d70 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f4d68018a50 tx=0x7f4d68004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cb0ac473356b5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.777+0000 7f4d58c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.778+0000 7f4d58c00700 1 -- 172.21.15.63:0/2391419996 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680146f0 con 0x5649bd5d92c0 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.778+0000 7f4d58c00700 1 -- 172.21.15.63:0/2391419996 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f4d68014890 con 0x5649bd5d92c0 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.778+0000 7f4d84106b80 1 -- 172.21.15.63:0/2391419996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d600aaee0 con 0x5649bd5d92c0 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.778+0000 7f4d58c00700 1 -- 172.21.15.63:0/2391419996 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4d68030050 con 0x5649bd5d92c0 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.778+0000 7f4d84106b80 10 client.20286 fetch_fsmap learned FSMap version 35 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.778+0000 7f4d84106b80 1 -- 172.21.15.63:0/2391419996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5649bd4bd060 con 0x5649bd5d92c0 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.778+0000 7f4d58c00700 1 -- 172.21.15.63:0/2391419996 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d600aaee0 con 0x5649bd5d92c0 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.779+0000 7f4d58c00700 1 -- 172.21.15.63:0/2391419996 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4d680229d0 con 0x5649bd5d92c0 2024-05-27T07:58:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.779+0000 7f4d84106b80 10 client.20286 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:58:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.779+0000 7f4d84106b80 10 client.20286 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:58:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.779+0000 7f4d84106b80 20 client.20286 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.779+0000 7f4d84106b80 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5649bd5e8b80 0x5649bd5eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.779+0000 7f4d84106b80 4 client.20286 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.779+0000 7f4d84106b80 1 -- 172.21.15.63:0/2391419996 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649bd4fed90 con 0x5649bd5e8b80 2024-05-27T07:58:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.780+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5649bd5e8b80 0x5649bd5eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.781+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5649bd5e8b80 0x5649bd5eafd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=590253416b47f755 server_cookie=de8b76f3323271a7 in_seq=0 out_seq=0 2024-05-27T07:58:55.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.781+0000 7f4d58c00700 10 client.20286 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:58:55.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d58c00700 1 -- 172.21.15.63:0/2391419996 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5649bd4fed90 con 0x5649bd5e8b80 2024-05-27T07:58:55.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d58c00700 10 client.20286 handle_command_reply: tid=0 2024-05-27T07:58:55.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d84106b80 1 client.20286 shutdown 2024-05-27T07:58:55.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d84106b80 1 -- 172.21.15.63:0/2391419996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 msgr2=0x5649bd5d6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d84106b80 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5d6d70 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f4d68018a50 tx=0x7f4d68004b40 comp rx=0 tx=0).stop 2024-05-27T07:58:55.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d84106b80 1 -- 172.21.15.63:0/2391419996 shutdown_connections 2024-05-27T07:58:55.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d84106b80 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5649bd5e8b80 0x5649bd5eafd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d84106b80 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5d6d70 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d84106b80 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d84106b80 1 --2- 172.21.15.63:0/2391419996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd53b9f0 0x5649bd5d6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d84106b80 1 -- 172.21.15.63:0/2391419996 >> 172.21.15.63:0/2391419996 conn(0x5649bd4e9390 msgr2=0x5649bd5cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.783+0000 7f4d58c00700 0 client.20286 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:58:55.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.784+0000 7f4d84106b80 1 -- 172.21.15.63:0/2391419996 shutdown_connections 2024-05-27T07:58:55.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.784+0000 7f4d84106b80 1 -- 172.21.15.63:0/2391419996 wait complete. 2024-05-27T07:58:55.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.784+0000 7f4d84106b80 20 client.20286 trim_cache size 0 max 16384 2024-05-27T07:58:55.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.807+0000 7f4d84106b80 1 Processor -- start 2024-05-27T07:58:55.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.807+0000 7f4d84106b80 1 -- start start 2024-05-27T07:58:55.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.807+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd44fe40 0x5649bd5ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.807+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd53b9f0 0x5649bd5e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.807+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5f0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.808+0000 7f4d84106b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701b4cd0 con 0x5649bd5d92c0 2024-05-27T07:58:55.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.808+0000 7f4d84106b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701b2560 con 0x5649bd44fe40 2024-05-27T07:58:55.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.808+0000 7f4d84106b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701b03b0 con 0x5649bd53b9f0 2024-05-27T07:58:55.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.808+0000 7f4d6f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.808+0000 7f4d6f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5f0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60512/0 (socket says 172.21.15.63:60512) 2024-05-27T07:58:55.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.808+0000 7f4d6f400700 1 -- 172.21.15.63:0/2215709436 learned_addr learned my addr 172.21.15.63:0/2215709436 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:55.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.808+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/2215709436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd44fe40 0x5649bd5ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.808+0000 7f4d6e000700 1 --2- 172.21.15.63:0/2215709436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd53b9f0 0x5649bd5e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.809+0000 7f4d6f400700 1 -- 172.21.15.63:0/2215709436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd53b9f0 msgr2=0x5649bd5e4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.809+0000 7f4d6f400700 1 --2- 172.21.15.63:0/2215709436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd53b9f0 0x5649bd5e4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.809+0000 7f4d6f400700 1 -- 172.21.15.63:0/2215709436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd44fe40 msgr2=0x5649bd5ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.809+0000 7f4d6f400700 1 --2- 172.21.15.63:0/2215709436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd44fe40 0x5649bd5ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.809+0000 7f4d6f400700 1 -- 172.21.15.63:0/2215709436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649bd5d4860 con 0x5649bd5d92c0 2024-05-27T07:58:55.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.809+0000 7f4d6e000700 1 --2- 172.21.15.63:0/2215709436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd53b9f0 0x5649bd5e4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:55.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.809+0000 7f4d6f400700 1 --2- 172.21.15.63:0/2215709436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5f0790 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f4d68009260 tx=0x7f4d68006130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c429aa9a067cce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.809+0000 7f4d5b400700 1 -- 172.21.15.63:0/2215709436 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6803a030 con 0x5649bd5d92c0 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.809+0000 7f4d5b400700 1 -- 172.21.15.63:0/2215709436 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d68027070 con 0x5649bd5d92c0 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.810+0000 7f4d5b400700 1 -- 172.21.15.63:0/2215709436 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680223f0 con 0x5649bd5d92c0 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.810+0000 7f4d84106b80 1 -- 172.21.15.63:0/2215709436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 msgr2=0x5649bd5f0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.810+0000 7f4d84106b80 1 --2- 172.21.15.63:0/2215709436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5f0790 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f4d68009260 tx=0x7f4d68006130 comp rx=0 tx=0).stop 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.811+0000 7f4d84106b80 1 -- 172.21.15.63:0/2215709436 shutdown_connections 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.811+0000 7f4d84106b80 1 --2- 172.21.15.63:0/2215709436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd5d92c0 0x5649bd5f0790 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.811+0000 7f4d84106b80 1 --2- 172.21.15.63:0/2215709436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd44fe40 0x5649bd5ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.811+0000 7f4d84106b80 1 --2- 172.21.15.63:0/2215709436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd53b9f0 0x5649bd5e4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.811+0000 7f4d84106b80 1 -- 172.21.15.63:0/2215709436 >> 172.21.15.63:0/2215709436 conn(0x5649bd4e9390 msgr2=0x5649bd5d4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.811+0000 7f4d84106b80 1 -- 172.21.15.63:0/2215709436 shutdown_connections 2024-05-27T07:58:55.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.811+0000 7f4d84106b80 1 -- 172.21.15.63:0/2215709436 wait complete. 2024-05-27T07:58:55.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.811+0000 7f4d84106b80 1 -- start start 2024-05-27T07:58:55.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.812+0000 7f4d84106b80 1 Processor -- start 2024-05-27T07:58:55.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.812+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.812+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.812+0000 7f4d84106b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 0x5649bd5dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.812+0000 7f4d84106b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701b0a20 con 0x5649bd53b9f0 2024-05-27T07:58:55.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.812+0000 7f4d84106b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701b1e90 con 0x5649bd5d92c0 2024-05-27T07:58:55.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.812+0000 7f4d84106b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4d701b10f0 con 0x5649bd44fe40 2024-05-27T07:58:55.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.812+0000 7f4d84106b80 10 client.0 _ll_register_callbacks cb 0x5649bd496c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:55.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5dc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60524/0 (socket says 172.21.15.63:60524) 2024-05-27T07:58:55.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6e000700 1 -- 172.21.15.63:0/3748912504 learned_addr learned my addr 172.21.15.63:0/3748912504 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:55.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6f400700 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 0x5649bd5dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6e000700 1 -- 172.21.15.63:0/3748912504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 msgr2=0x5649bd5dbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6e000700 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5dbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6e000700 1 -- 172.21.15.63:0/3748912504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 msgr2=0x5649bd5dc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6e000700 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 0x5649bd5dc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6e000700 1 -- 172.21.15.63:0/3748912504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649bd659e60 con 0x5649bd53b9f0 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.813+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.814+0000 7f4d6f400700 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 0x5649bd5dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.814+0000 7f4d6e000700 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5dc160 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c005950 tx=0x7f4d5c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6cc7fe75cc7fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.814+0000 7f4d58c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.814+0000 7f4d58c00700 1 -- 172.21.15.63:0/3748912504 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c003e60 con 0x5649bd53b9f0 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.814+0000 7f4d58c00700 1 -- 172.21.15.63:0/3748912504 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f4d5c004830 con 0x5649bd53b9f0 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.814+0000 7f4d84106b80 1 -- 172.21.15.63:0/3748912504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d68027070 con 0x5649bd53b9f0 2024-05-27T07:58:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d58c00700 1 -- 172.21.15.63:0/3748912504 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c029050 con 0x5649bd53b9f0 2024-05-27T07:58:55.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d84106b80 10 client.20292 fetch_fsmap learned FSMap version 35 2024-05-27T07:58:55.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d84106b80 1 -- 172.21.15.63:0/3748912504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5649bd65afc0 con 0x5649bd53b9f0 2024-05-27T07:58:55.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d58c00700 1 -- 172.21.15.63:0/3748912504 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d68027070 con 0x5649bd53b9f0 2024-05-27T07:58:55.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d58c00700 1 -- 172.21.15.63:0/3748912504 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4d5c0114d0 con 0x5649bd53b9f0 2024-05-27T07:58:55.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d84106b80 10 client.20292 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:58:55.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d84106b80 10 client.20292 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:58:55.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d84106b80 20 client.20292 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:55.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d84106b80 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5649bd65b780 0x5649bd65dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d84106b80 4 client.20292 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:58:55.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.815+0000 7f4d84106b80 1 -- 172.21.15.63:0/3748912504 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649bd4f8570 con 0x5649bd65b780 2024-05-27T07:58:55.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.816+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5649bd65b780 0x5649bd65dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.817+0000 7f4d6ea00700 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5649bd65b780 0x5649bd65dba0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=edcace2a84231dd6 server_cookie=b2ff155145a7a2db in_seq=0 out_seq=0 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.817+0000 7f4d58c00700 10 client.20292 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d58c00700 1 -- 172.21.15.63:0/3748912504 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649bd4f8570 con 0x5649bd65b780 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d58c00700 10 client.20292 handle_command_reply: tid=0 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 client.20292 shutdown 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 -- 172.21.15.63:0/3748912504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 msgr2=0x5649bd5dc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5dc160 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c005950 tx=0x7f4d5c0046a0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 -- 172.21.15.63:0/3748912504 shutdown_connections 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649bd5d92c0 0x5649bd5dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5649bd44fe40 0x5649bd5dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5649bd65b780 0x5649bd65dba0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 --2- 172.21.15.63:0/3748912504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649bd53b9f0 0x5649bd5dc160 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 -- 172.21.15.63:0/3748912504 >> 172.21.15.63:0/3748912504 conn(0x5649bd4e9390 msgr2=0x5649bd5d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d58c00700 0 client.20292 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:58:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 -- 172.21.15.63:0/3748912504 shutdown_connections 2024-05-27T07:58:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 1 -- 172.21.15.63:0/3748912504 wait complete. 2024-05-27T07:58:55.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.818+0000 7f4d84106b80 20 client.20292 trim_cache size 0 max 16384 2024-05-27T07:58:55.822 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:58:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.819+0000 7f4d6fe00700 1 -- 172.21.15.63:0/3365662512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4d540b29f0 msgr2=0x7f4d540b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.819+0000 7f4d6fe00700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4d540b29f0 0x7f4d540b4f00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x5649bd5d1210 tx=0x7f4d68009680 comp rx=0 tx=0).stop 2024-05-27T07:58:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.819+0000 7f4d6fe00700 1 -- 172.21.15.63:0/3365662512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d70105280 msgr2=0x7f4d70102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.819+0000 7f4d6fe00700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d70105280 0x7f4d70102f70 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4d60005910 tx=0x7f4d600047b0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.819+0000 7f4d6fe00700 1 -- 172.21.15.63:0/3365662512 shutdown_connections 2024-05-27T07:58:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.819+0000 7f4d6fe00700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4d540b29f0 0x7f4d540b4f00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.820+0000 7f4d6fe00700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d7010e6f0 0x7f4d701039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.820+0000 7f4d6fe00700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d70105280 0x7f4d70102f70 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.820+0000 7f4d6fe00700 1 --2- 172.21.15.63:0/3365662512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d70105bd0 0x7f4d701034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.820+0000 7f4d6fe00700 1 -- 172.21.15.63:0/3365662512 >> 172.21.15.63:0/3365662512 conn(0x7f4d70078ab0 msgr2=0x7f4d700fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.820+0000 7f4d6fe00700 1 -- 172.21.15.63:0/3365662512 shutdown_connections 2024-05-27T07:58:55.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:55.820+0000 7f4d6fe00700 1 -- 172.21.15.63:0/3365662512 wait complete. 2024-05-27T07:58:56.799 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:58:57.287 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:57.623 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:57 smithi063 ceph-mon[31613]: pgmap v803: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:58:57.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:57 smithi086 ceph-mon[34687]: pgmap v803: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:58:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:57 smithi149 ceph-mon[35408]: pgmap v803: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 7 op/s 2024-05-27T07:58:59.402 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:58:59 smithi063 ceph-mon[31613]: pgmap v804: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:58:59.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.669+0000 7f0388c00700 1 -- 172.21.15.63:0/1967092469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0384102ee0 msgr2=0x7f0384110f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:59.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.669+0000 7f0388c00700 1 --2- 172.21.15.63:0/1967092469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0384102ee0 0x7f0384110f70 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f037c00b0a0 tx=0x7f037c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.669+0000 7f0388c00700 1 -- 172.21.15.63:0/1967092469 shutdown_connections 2024-05-27T07:58:59.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.669+0000 7f0388c00700 1 --2- 172.21.15.63:0/1967092469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0384102ee0 0x7f0384110f70 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.669+0000 7f0388c00700 1 --2- 172.21.15.63:0/1967092469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0384109100 0x7f0384109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.669+0000 7f0388c00700 1 --2- 172.21.15.63:0/1967092469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0384102590 0x7f03841029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.669+0000 7f0388c00700 1 -- 172.21.15.63:0/1967092469 >> 172.21.15.63:0/1967092469 conn(0x7f03840783a0 msgr2=0x7f03840787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:59.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.669+0000 7f0388c00700 1 -- 172.21.15.63:0/1967092469 shutdown_connections 2024-05-27T07:58:59.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.669+0000 7f0388c00700 1 -- 172.21.15.63:0/1967092469 wait complete. 2024-05-27T07:58:59.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.671+0000 7f0388c00700 1 Processor -- start 2024-05-27T07:58:59.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.671+0000 7f0388c00700 1 -- start start 2024-05-27T07:58:59.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0388c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0384102590 0x7f0384103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0388c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0384102ee0 0x7f0384103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0388c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0384109100 0x7f03841041f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0388c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03841142a0 con 0x7f0384102ee0 2024-05-27T07:58:59.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0388c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0384114120 con 0x7f0384109100 2024-05-27T07:58:59.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0388c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0384114420 con 0x7f0384102590 2024-05-27T07:58:59.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0382a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0384102ee0 0x7f0384103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0382a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0384102ee0 0x7f0384103cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52454/0 (socket says 172.21.15.63:52454) 2024-05-27T07:58:59.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0382a00700 1 -- 172.21.15.63:0/3348683875 learned_addr learned my addr 172.21.15.63:0/3348683875 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:59.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0383400700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0384102590 0x7f0384103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.672+0000 7f0383e00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0384109100 0x7f03841041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.673+0000 7f0382a00700 1 -- 172.21.15.63:0/3348683875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0384102590 msgr2=0x7f0384103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.673+0000 7f0382a00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0384102590 0x7f0384103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.673+0000 7f0382a00700 1 -- 172.21.15.63:0/3348683875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0384109100 msgr2=0x7f03841041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.673+0000 7f0382a00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0384109100 0x7f03841041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.673+0000 7f0382a00700 1 -- 172.21.15.63:0/3348683875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03841a8f50 con 0x7f0384102ee0 2024-05-27T07:58:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.673+0000 7f0383e00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0384109100 0x7f03841041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.673+0000 7f0382a00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0384102ee0 0x7f0384103cb0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f037000d980 tx=0x7f037000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.674+0000 7f037be00700 1 -- 172.21.15.63:0/3348683875 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370017070 con 0x7f0384102ee0 2024-05-27T07:58:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.674+0000 7f037be00700 1 -- 172.21.15.63:0/3348683875 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0370010460 con 0x7f0384102ee0 2024-05-27T07:58:59.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.674+0000 7f037be00700 1 -- 172.21.15.63:0/3348683875 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037000de60 con 0x7f0384102ee0 2024-05-27T07:58:59.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.674+0000 7f0388c00700 1 -- 172.21.15.63:0/3348683875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03841a9240 con 0x7f0384102ee0 2024-05-27T07:58:59.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.674+0000 7f0388c00700 1 -- 172.21.15.63:0/3348683875 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03841a9730 con 0x7f0384102ee0 2024-05-27T07:58:59.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.679+0000 7f0398dbeb80 1 Processor -- start 2024-05-27T07:58:59.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.680+0000 7f0398dbeb80 1 -- start start 2024-05-27T07:58:59.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.680+0000 7f037be00700 1 -- 172.21.15.63:0/3348683875 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0370010610 con 0x7f0384102ee0 2024-05-27T07:58:59.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.680+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x56400628aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.680+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x564006295ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.680+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 0x564006298020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.680+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5640061b9e40 con 0x56400628b410 2024-05-27T07:58:59.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.680+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640061a99e0 con 0x56400610ce40 2024-05-27T07:58:59.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.680+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5640061ada20 con 0x56400628b820 2024-05-27T07:58:59.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.681+0000 7f0383400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x564006295ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.681+0000 7f0383400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x564006295ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52456/0 (socket says 172.21.15.63:52456) 2024-05-27T07:58:59.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.681+0000 7f0383400700 1 -- 172.21.15.63:0/2116188915 learned_addr learned my addr 172.21.15.63:0/2116188915 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:59.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.681+0000 7f0383e00700 1 --2- 172.21.15.63:0/2116188915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x56400628aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.681+0000 7f0382a00700 1 --2- 172.21.15.63:0/2116188915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 0x564006298020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.681+0000 7f0383400700 1 -- 172.21.15.63:0/2116188915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 msgr2=0x564006298020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:59.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.681+0000 7f0383400700 1 --2- 172.21.15.63:0/2116188915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 0x564006298020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.681+0000 7f0383400700 1 -- 172.21.15.63:0/2116188915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 msgr2=0x56400628aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:59.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.681+0000 7f0383400700 1 --2- 172.21.15.63:0/2116188915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x56400628aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.681+0000 7f0383400700 1 -- 172.21.15.63:0/2116188915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564006172420 con 0x56400628b410 2024-05-27T07:58:59.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.682+0000 7f0383400700 1 --2- 172.21.15.63:0/2116188915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x564006295ad0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f037400bc60 tx=0x7f037400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a591ecd7886373d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:59.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.682+0000 7f0378c00700 1 -- 172.21.15.63:0/2116188915 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374015070 con 0x56400628b410 2024-05-27T07:58:59.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.683+0000 7f0378c00700 1 -- 172.21.15.63:0/2116188915 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0374011440 con 0x56400628b410 2024-05-27T07:58:59.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.683+0000 7f0378c00700 1 -- 172.21.15.63:0/2116188915 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374010630 con 0x56400628b410 2024-05-27T07:58:59.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.683+0000 7f037be00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03680b2a70 0x7f03680b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.683+0000 7f037be00700 1 -- 172.21.15.63:0/3348683875 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f03700e24a0 con 0x7f0384102ee0 2024-05-27T07:58:59.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.683+0000 7f0383e00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03680b2a70 0x7f03680b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.684+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2116188915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 msgr2=0x564006295ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:59.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.684+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/2116188915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x564006295ad0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f037400bc60 tx=0x7f037400bf70 comp rx=0 tx=0).stop 2024-05-27T07:58:59.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.684+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2116188915 shutdown_connections 2024-05-27T07:58:59.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.684+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/2116188915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 0x564006298020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.684+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/2116188915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x56400628aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.684+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/2116188915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x564006295ad0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.684+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2116188915 >> 172.21.15.63:0/2116188915 conn(0x56400616cfc0 msgr2=0x564006286d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:59.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.684+0000 7f0383e00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03680b2a70 0x7f03680b4f80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x564006287690 tx=0x7f037c009780 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:59.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.684+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2116188915 shutdown_connections 2024-05-27T07:58:59.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.685+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2116188915 wait complete. 2024-05-27T07:58:59.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.685+0000 7f0398dbeb80 1 -- start start 2024-05-27T07:58:59.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.685+0000 7f0398dbeb80 1 Processor -- start 2024-05-27T07:58:59.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.686+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x564006295520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.686+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b410 0x564006295a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.686+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x564006295fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.686+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5640060affa0 con 0x56400628b820 2024-05-27T07:58:59.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.686+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56400614a760 con 0x56400610ce40 2024-05-27T07:58:59.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.686+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564006298990 con 0x56400628b410 2024-05-27T07:58:59.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.686+0000 7f0398dbeb80 10 client.0 _ll_register_callbacks cb 0x564006153c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:59.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.687+0000 7f0383e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x564006295fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.687+0000 7f0383400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x564006295520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.687+0000 7f0383400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x564006295520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46294/0 (socket says 172.21.15.63:46294) 2024-05-27T07:58:59.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.687+0000 7f0383e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x564006295fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52462/0 (socket says 172.21.15.63:52462) 2024-05-27T07:58:59.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.687+0000 7f0383400700 1 -- 172.21.15.63:0/2932728823 learned_addr learned my addr 172.21.15.63:0/2932728823 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:59.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.687+0000 7f0382a00700 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b410 0x564006295a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.688+0000 7f0383e00700 1 -- 172.21.15.63:0/2932728823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b410 msgr2=0x564006295a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:59.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.688+0000 7f0383e00700 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b410 0x564006295a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.688+0000 7f0383e00700 1 -- 172.21.15.63:0/2932728823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 msgr2=0x564006295520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:59.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.688+0000 7f0383e00700 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x564006295520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.688+0000 7f0383e00700 1 -- 172.21.15.63:0/2932728823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640060f98d0 con 0x56400628b820 2024-05-27T07:58:59.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.688+0000 7f0383e00700 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x564006295fa0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f037c00b830 tx=0x7f037c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd615449ac475ce7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:59.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.688+0000 7f036d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:59.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.689+0000 7f036d600700 1 -- 172.21.15.63:0/2932728823 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c02d070 con 0x56400628b820 2024-05-27T07:58:59.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.689+0000 7f036d600700 1 -- 172.21.15.63:0/2932728823 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f037c005040 con 0x56400628b820 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.689+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2932728823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0374011440 con 0x56400628b820 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.689+0000 7f036d600700 1 -- 172.21.15.63:0/2932728823 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f037c03a050 con 0x56400628b820 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.689+0000 7f0398dbeb80 10 client.20304 fetch_fsmap learned FSMap version 35 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.689+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2932728823 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56400617a060 con 0x56400628b820 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.690+0000 7f036d600700 1 -- 172.21.15.63:0/2932728823 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0374011440 con 0x56400628b820 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.690+0000 7f036d600700 1 -- 172.21.15.63:0/2932728823 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f037c011070 con 0x56400628b820 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.690+0000 7f0398dbeb80 10 client.20304 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.690+0000 7f0398dbeb80 10 client.20304 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.690+0000 7f0398dbeb80 20 client.20304 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.690+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5640062a5920 0x5640062a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.690+0000 7f0398dbeb80 4 client.20304 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:59.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.690+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2932728823 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640061bbd90 con 0x5640062a5920 2024-05-27T07:58:59.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.691+0000 7f0383400700 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5640062a5920 0x5640062a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.692+0000 7f036d600700 10 client.20304 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:58:59.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.692+0000 7f0383400700 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5640062a5920 0x5640062a7d40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4777ebcd0681ff0 server_cookie=49f8d6acb7c9a5d6 in_seq=0 out_seq=0 2024-05-27T07:58:59.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f036d600700 1 -- 172.21.15.63:0/2932728823 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5640061bbd90 con 0x5640062a5920 2024-05-27T07:58:59.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f036d600700 10 client.20304 handle_command_reply: tid=0 2024-05-27T07:58:59.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f0398dbeb80 1 client.20304 shutdown 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2932728823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 msgr2=0x564006295fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x564006295fa0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f037c00b830 tx=0x7f037c0099e0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2932728823 shutdown_connections 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5640062a5920 0x5640062a7d40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x564006295fa0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x564006295520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/2932728823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b410 0x564006295a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f036d600700 0 client.20304 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.694+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2932728823 >> 172.21.15.63:0/2932728823 conn(0x56400616cfc0 msgr2=0x564006285ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.695+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2932728823 shutdown_connections 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.695+0000 7f0398dbeb80 1 -- 172.21.15.63:0/2932728823 wait complete. 2024-05-27T07:58:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.695+0000 7f0398dbeb80 20 client.20304 trim_cache size 0 max 16384 2024-05-27T07:58:59.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:58:59 smithi086 ceph-mon[34687]: pgmap v804: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:58:59.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:58:59 smithi149 ceph-mon[35408]: pgmap v804: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T07:58:59.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.720+0000 7f0398dbeb80 1 Processor -- start 2024-05-27T07:58:59.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.720+0000 7f0398dbeb80 1 -- start start 2024-05-27T07:58:59.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.720+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400610ce40 0x564006294480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.720+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400628b410 0x564006290cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.721+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x5640062a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.721+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b4c90 con 0x56400628b820 2024-05-27T07:58:59.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.721+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b2520 con 0x56400628b410 2024-05-27T07:58:59.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.721+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b0370 con 0x56400610ce40 2024-05-27T07:58:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.721+0000 7f0383e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x5640062a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.721+0000 7f0383e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x5640062a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52478/0 (socket says 172.21.15.63:52478) 2024-05-27T07:58:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.721+0000 7f0383e00700 1 -- 172.21.15.63:0/1888702730 learned_addr learned my addr 172.21.15.63:0/1888702730 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.721+0000 7f0383400700 1 --2- 172.21.15.63:0/1888702730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400610ce40 0x564006294480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.721+0000 7f0382a00700 1 --2- 172.21.15.63:0/1888702730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400628b410 0x564006290cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.722+0000 7f0383e00700 1 -- 172.21.15.63:0/1888702730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400610ce40 msgr2=0x564006294480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.722+0000 7f0383e00700 1 --2- 172.21.15.63:0/1888702730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400610ce40 0x564006294480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.722+0000 7f0383e00700 1 -- 172.21.15.63:0/1888702730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400628b410 msgr2=0x564006290cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.722+0000 7f0383e00700 1 --2- 172.21.15.63:0/1888702730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400628b410 0x564006290cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.722+0000 7f0383e00700 1 -- 172.21.15.63:0/1888702730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564006287910 con 0x56400628b820 2024-05-27T07:58:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.722+0000 7f0383e00700 1 --2- 172.21.15.63:0/1888702730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x5640062a1ab0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f037c00b000 tx=0x7f037c01df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a4005f4f994ee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.722+0000 7f036fe00700 1 -- 172.21.15.63:0/1888702730 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c02d040 con 0x56400628b820 2024-05-27T07:58:59.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.723+0000 7f036fe00700 1 -- 172.21.15.63:0/1888702730 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f037c01dd00 con 0x56400628b820 2024-05-27T07:58:59.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.723+0000 7f036fe00700 1 -- 172.21.15.63:0/1888702730 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c023070 con 0x56400628b820 2024-05-27T07:58:59.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.724+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1888702730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 msgr2=0x5640062a1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:59.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.724+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/1888702730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x5640062a1ab0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f037c00b000 tx=0x7f037c01df70 comp rx=0 tx=0).stop 2024-05-27T07:58:59.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.724+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1888702730 shutdown_connections 2024-05-27T07:58:59.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.724+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/1888702730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b820 0x5640062a1ab0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.724+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/1888702730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400610ce40 0x564006294480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.724+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/1888702730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400628b410 0x564006290cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.724+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1888702730 >> 172.21.15.63:0/1888702730 conn(0x56400616cfc0 msgr2=0x5640062954e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:59.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.724+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1888702730 shutdown_connections 2024-05-27T07:58:59.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.724+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1888702730 wait complete. 2024-05-27T07:58:59.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.725+0000 7f0398dbeb80 1 -- start start 2024-05-27T07:58:59.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.725+0000 7f0398dbeb80 1 Processor -- start 2024-05-27T07:58:59.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.725+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x56400629e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x5640062995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0398dbeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 0x564006299b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b09e0 con 0x56400628b410 2024-05-27T07:58:59.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b1e50 con 0x56400610ce40 2024-05-27T07:58:59.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0398dbeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03841b10b0 con 0x56400628b820 2024-05-27T07:58:59.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0398dbeb80 10 client.0 _ll_register_callbacks cb 0x564006153c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:59.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0382a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x5640062995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0382a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x5640062995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52488/0 (socket says 172.21.15.63:52488) 2024-05-27T07:58:59.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0382a00700 1 -- 172.21.15.63:0/1118045079 learned_addr learned my addr 172.21.15.63:0/1118045079 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:58:59.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0383400700 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x56400629e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.726+0000 7f0383e00700 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 0x564006299b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f0382a00700 1 -- 172.21.15.63:0/1118045079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 msgr2=0x564006299b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:59.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f0382a00700 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 0x564006299b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f0382a00700 1 -- 172.21.15.63:0/1118045079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 msgr2=0x56400629e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:59.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f0382a00700 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x56400629e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f0382a00700 1 -- 172.21.15.63:0/1118045079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400629a100 con 0x56400628b410 2024-05-27T07:58:59.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f0383e00700 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 0x564006299b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:59.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f0382a00700 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x5640062995d0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f03700a6680 tx=0x7f03700dfd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fc15b703fd41e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:59.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f036d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:58:59.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f036d600700 1 -- 172.21.15.63:0/1118045079 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370020040 con 0x56400628b410 2024-05-27T07:58:59.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f036d600700 1 -- 172.21.15.63:0/1118045079 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9395+0+0 (secure 0 0 0) 0x7f03700a9670 con 0x56400628b410 2024-05-27T07:58:59.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.727+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1118045079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f037c01dd00 con 0x56400628b410 2024-05-27T07:58:59.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.728+0000 7f036d600700 1 -- 172.21.15.63:0/1118045079 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f03700a9b10 con 0x56400628b410 2024-05-27T07:58:59.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.728+0000 7f0398dbeb80 10 client.20310 fetch_fsmap learned FSMap version 35 2024-05-27T07:58:59.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.728+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1118045079 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564006310190 con 0x56400628b410 2024-05-27T07:58:59.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.728+0000 7f036d600700 1 -- 172.21.15.63:0/1118045079 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f037c01dd00 con 0x56400628b410 2024-05-27T07:58:59.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.728+0000 7f036d600700 1 -- 172.21.15.63:0/1118045079 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f03700b1de0 con 0x56400628b410 2024-05-27T07:58:59.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.728+0000 7f0398dbeb80 10 client.20310 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:58:59.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.728+0000 7f0398dbeb80 10 client.20310 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:58:59.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.728+0000 7f0398dbeb80 20 client.20310 populate_metadata read hostname 'smithi063' 2024-05-27T07:58:59.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.728+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564006310870 0x564006312c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:59.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.729+0000 7f0398dbeb80 4 client.20310 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:58:59.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.729+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1118045079 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640061b5570 con 0x564006310870 2024-05-27T07:58:59.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.729+0000 7f0383400700 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564006310870 0x564006312c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:59.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.731+0000 7f0383400700 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564006310870 0x564006312c90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e45611760bd42a76 server_cookie=c2cd7e8df72c975b in_seq=0 out_seq=0 2024-05-27T07:58:59.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.731+0000 7f036d600700 10 client.20310 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T07:58:59.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.731+0000 7f036d600700 1 -- 172.21.15.63:0/1118045079 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640061b5570 con 0x564006310870 2024-05-27T07:58:59.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.731+0000 7f036d600700 10 client.20310 handle_command_reply: tid=0 2024-05-27T07:58:59.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.731+0000 7f0398dbeb80 1 client.20310 shutdown 2024-05-27T07:58:59.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.731+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1118045079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 msgr2=0x5640062995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:59.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.731+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x5640062995d0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f03700a6680 tx=0x7f03700dfd80 comp rx=0 tx=0).stop 2024-05-27T07:58:59.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.732+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1118045079 shutdown_connections 2024-05-27T07:58:59.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.732+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56400628b820 0x564006299b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.732+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56400610ce40 0x56400629e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.732+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x564006310870 0x564006312c90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.732+0000 7f0398dbeb80 1 --2- 172.21.15.63:0/1118045079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56400628b410 0x5640062995d0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.732+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1118045079 >> 172.21.15.63:0/1118045079 conn(0x56400616cfc0 msgr2=0x564006296960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:59.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.732+0000 7f036d600700 0 client.20310 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T07:58:59.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.732+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1118045079 shutdown_connections 2024-05-27T07:58:59.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.732+0000 7f0398dbeb80 1 -- 172.21.15.63:0/1118045079 wait complete. 2024-05-27T07:58:59.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.732+0000 7f0398dbeb80 20 client.20310 trim_cache size 0 max 16384 2024-05-27T07:58:59.735 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:58:59.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 -- 172.21.15.63:0/3348683875 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03680b2a70 msgr2=0x7f03680b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:59.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03680b2a70 0x7f03680b4f80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x564006287690 tx=0x7f037c009780 comp rx=0 tx=0).stop 2024-05-27T07:58:59.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 -- 172.21.15.63:0/3348683875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0384102ee0 msgr2=0x7f0384103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:59.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0384102ee0 0x7f0384103cb0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f037000d980 tx=0x7f037000dd40 comp rx=0 tx=0).stop 2024-05-27T07:58:59.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 -- 172.21.15.63:0/3348683875 shutdown_connections 2024-05-27T07:58:59.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03680b2a70 0x7f03680b4f80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0384109100 0x7f03841041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0384102590 0x7f0384103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 --2- 172.21.15.63:0/3348683875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0384102ee0 0x7f0384103cb0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:59.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 -- 172.21.15.63:0/3348683875 >> 172.21.15.63:0/3348683875 conn(0x7f03840783a0 msgr2=0x7f03840ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:59.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.733+0000 7f0388c00700 1 -- 172.21.15.63:0/3348683875 shutdown_connections 2024-05-27T07:58:59.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:58:59.734+0000 7f0388c00700 1 -- 172.21.15.63:0/3348683875 wait complete. 2024-05-27T07:59:00.589 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:59:01.057 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:01.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:01 smithi063 ceph-mon[31613]: pgmap v805: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:59:01.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:01 smithi086 ceph-mon[34687]: pgmap v805: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:59:01.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:01 smithi149 ceph-mon[35408]: pgmap v805: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T07:59:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.583+0000 7f07b7e00700 1 -- 172.21.15.63:0/2296273908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b810e710 msgr2=0x7f07b8110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.583+0000 7f07b7e00700 1 --2- 172.21.15.63:0/2296273908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b810e710 0x7f07b8110bd0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b0a0 tx=0x7f07b000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.583+0000 7f07b7e00700 1 -- 172.21.15.63:0/2296273908 shutdown_connections 2024-05-27T07:59:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.583+0000 7f07b7e00700 1 --2- 172.21.15.63:0/2296273908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b810e710 0x7f07b8110bd0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.583+0000 7f07b7e00700 1 --2- 172.21.15.63:0/2296273908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b81052e0 0x7f07b81056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.583+0000 7f07b7e00700 1 --2- 172.21.15.63:0/2296273908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8105c30 0x7f07b810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.583+0000 7f07b7e00700 1 -- 172.21.15.63:0/2296273908 >> 172.21.15.63:0/2296273908 conn(0x7f07b8078ab0 msgr2=0x7f07b8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.583+0000 7f07b7e00700 1 -- 172.21.15.63:0/2296273908 shutdown_connections 2024-05-27T07:59:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.583+0000 7f07b7e00700 1 -- 172.21.15.63:0/2296273908 wait complete. 2024-05-27T07:59:03.587 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:03 smithi063 ceph-mon[31613]: pgmap v806: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.1 MiB/s wr, 32 op/s 2024-05-27T07:59:03.587 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:03 smithi063 ceph-mon[31613]: osdmap e133: 12 total, 12 up, 12 in 2024-05-27T07:59:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.585+0000 7f07b7e00700 1 Processor -- start 2024-05-27T07:59:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.585+0000 7f07b7e00700 1 -- start start 2024-05-27T07:59:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.585+0000 7f07b7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b81052e0 0x7f07b81a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.585+0000 7f07b7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8105c30 0x7f07b81a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.585+0000 7f07b7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b810e710 0x7f07b81aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.586+0000 7f07b7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8114150 con 0x7f07b810e710 2024-05-27T07:59:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.586+0000 7f07b7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8113fd0 con 0x7f07b8105c30 2024-05-27T07:59:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.586+0000 7f07b7e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81142d0 con 0x7f07b81052e0 2024-05-27T07:59:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.586+0000 7f07b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b810e710 0x7f07b81aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.586+0000 7f07b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b810e710 0x7f07b81aceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52518/0 (socket says 172.21.15.63:52518) 2024-05-27T07:59:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.586+0000 7f07b7400700 1 -- 172.21.15.63:0/3845562413 learned_addr learned my addr 172.21.15.63:0/3845562413 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.586+0000 7f07b6000700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8105c30 0x7f07b81a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.586+0000 7f07b6a00700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b81052e0 0x7f07b81a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.587+0000 7f07b7400700 1 -- 172.21.15.63:0/3845562413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b81052e0 msgr2=0x7f07b81a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:03.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.587+0000 7f07b7400700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b81052e0 0x7f07b81a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.587+0000 7f07b7400700 1 -- 172.21.15.63:0/3845562413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8105c30 msgr2=0x7f07b81a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.587+0000 7f07b7400700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8105c30 0x7f07b81a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.587+0000 7f07b7400700 1 -- 172.21.15.63:0/3845562413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b81ad3f0 con 0x7f07b810e710 2024-05-27T07:59:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.587+0000 7f07b7400700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b810e710 0x7f07b81aceb0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b070 tx=0x7f07b0004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.588+0000 7f07af400700 1 -- 172.21.15.63:0/3845562413 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0021070 con 0x7f07b810e710 2024-05-27T07:59:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.588+0000 7f07af400700 1 -- 172.21.15.63:0/3845562413 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07b0007b80 con 0x7f07b810e710 2024-05-27T07:59:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.588+0000 7f07af400700 1 -- 172.21.15.63:0/3845562413 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0004640 con 0x7f07b810e710 2024-05-27T07:59:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.588+0000 7f07b7e00700 1 -- 172.21.15.63:0/3845562413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b81ad5c0 con 0x7f07b810e710 2024-05-27T07:59:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.588+0000 7f07b7e00700 1 -- 172.21.15.63:0/3845562413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b81ada10 con 0x7f07b810e710 2024-05-27T07:59:03.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.592+0000 7f07af400700 1 -- 172.21.15.63:0/3845562413 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f07b001d450 con 0x7f07b810e710 2024-05-27T07:59:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.594+0000 7f07cc527b80 1 Processor -- start 2024-05-27T07:59:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.594+0000 7f07cc527b80 1 -- start start 2024-05-27T07:59:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.594+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e2f2e40 0x55a95e470ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.595+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e47bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.595+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 0x55a95e47e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.595+0000 7f07cc527b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a95e39fe40 con 0x55a95e471410 2024-05-27T07:59:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.595+0000 7f07cc527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a95e38f9e0 con 0x55a95e471820 2024-05-27T07:59:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.595+0000 7f07cc527b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a95e393a20 con 0x55a95e2f2e40 2024-05-27T07:59:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.595+0000 7f07b6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e47bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.595+0000 7f07af400700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f079c0b29f0 0x7f079c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.595+0000 7f07b6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e47bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52530/0 (socket says 172.21.15.63:52530) 2024-05-27T07:59:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.595+0000 7f07b6a00700 1 -- 172.21.15.63:0/423178512 learned_addr learned my addr 172.21.15.63:0/423178512 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.595+0000 7f07af400700 1 -- 172.21.15.63:0/3845562413 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9571+0+0 (secure 0 0 0) 0x7f07b000f020 con 0x7f07b810e710 2024-05-27T07:59:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.596+0000 7f07b6000700 1 --2- 172.21.15.63:0/423178512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e2f2e40 0x55a95e470ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.596+0000 7f07b7400700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f079c0b29f0 0x7f079c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.596+0000 7f07b6000700 1 --2- 172.21.15.63:0/423178512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 0x55a95e47e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.596+0000 7f07b6a00700 1 -- 172.21.15.63:0/423178512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e2f2e40 msgr2=0x55a95e470ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.596+0000 7f07b6a00700 1 --2- 172.21.15.63:0/423178512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e2f2e40 0x55a95e470ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.596+0000 7f07b6a00700 1 -- 172.21.15.63:0/423178512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 msgr2=0x55a95e47e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.596+0000 7f07b6a00700 1 --2- 172.21.15.63:0/423178512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 0x55a95e47e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.596+0000 7f07b6a00700 1 -- 172.21.15.63:0/423178512 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a95e358420 con 0x55a95e471410 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.596+0000 7f07b6000700 1 --2- 172.21.15.63:0/423178512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 0x55a95e47e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.597+0000 7f07b7400700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f079c0b29f0 0x7f079c0b4f00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f07b00a6ed0 tx=0x7f07b0009b80 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.597+0000 7f07b6a00700 1 --2- 172.21.15.63:0/423178512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e47bad0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f07a800bd70 tx=0x7f07a80106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5749136ba010ffb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.597+0000 7f07a3e00700 1 -- 172.21.15.63:0/423178512 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a801c480 con 0x55a95e471410 2024-05-27T07:59:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.597+0000 7f07a3e00700 1 -- 172.21.15.63:0/423178512 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a801cab0 con 0x55a95e471410 2024-05-27T07:59:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.597+0000 7f07a3e00700 1 -- 172.21.15.63:0/423178512 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8011880 con 0x55a95e471410 2024-05-27T07:59:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.598+0000 7f07cc527b80 1 -- 172.21.15.63:0/423178512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 msgr2=0x55a95e47bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.598+0000 7f07cc527b80 1 --2- 172.21.15.63:0/423178512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e47bad0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f07a800bd70 tx=0x7f07a80106b0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.599+0000 7f07cc527b80 1 -- 172.21.15.63:0/423178512 shutdown_connections 2024-05-27T07:59:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.599+0000 7f07cc527b80 1 --2- 172.21.15.63:0/423178512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 0x55a95e47e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.599+0000 7f07cc527b80 1 --2- 172.21.15.63:0/423178512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e2f2e40 0x55a95e470ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.599+0000 7f07cc527b80 1 --2- 172.21.15.63:0/423178512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e47bad0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.599+0000 7f07cc527b80 1 -- 172.21.15.63:0/423178512 >> 172.21.15.63:0/423178512 conn(0x55a95e352fc0 msgr2=0x55a95e46cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.599+0000 7f07cc527b80 1 -- 172.21.15.63:0/423178512 shutdown_connections 2024-05-27T07:59:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.599+0000 7f07cc527b80 1 -- 172.21.15.63:0/423178512 wait complete. 2024-05-27T07:59:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.600+0000 7f07cc527b80 1 -- start start 2024-05-27T07:59:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.600+0000 7f07cc527b80 1 Processor -- start 2024-05-27T07:59:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e2f2e40 0x55a95e47b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e471410 0x55a95e47ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 0x55a95e47bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07cc527b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a95e295fa0 con 0x55a95e2f2e40 2024-05-27T07:59:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07cc527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a95e330760 con 0x55a95e471820 2024-05-27T07:59:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07cc527b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a95e47e990 con 0x55a95e471410 2024-05-27T07:59:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07cc527b80 10 client.0 _ll_register_callbacks cb 0x55a95e339c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07b6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e2f2e40 0x55a95e47b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07b6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e471410 0x55a95e47ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07b6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e2f2e40 0x55a95e47b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52532/0 (socket says 172.21.15.63:52532) 2024-05-27T07:59:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.601+0000 7f07b6000700 1 -- 172.21.15.63:0/226187663 learned_addr learned my addr 172.21.15.63:0/226187663 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.602+0000 7f07b6000700 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 0x55a95e47bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.602+0000 7f07b6000700 1 -- 172.21.15.63:0/226187663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e471410 msgr2=0x55a95e47ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.602+0000 7f07b6000700 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e471410 0x55a95e47ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.602+0000 7f07b6000700 1 -- 172.21.15.63:0/226187663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 msgr2=0x55a95e47bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.602+0000 7f07b6000700 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 0x55a95e47bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.602+0000 7f07b6000700 1 -- 172.21.15.63:0/226187663 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a95e2df8d0 con 0x55a95e2f2e40 2024-05-27T07:59:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.603+0000 7f07b6000700 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e2f2e40 0x55a95e47b550 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f07a4014d00 tx=0x7f07a4005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca7f5449b114f1f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.603+0000 7f07a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.603+0000 7f07a0c00700 1 -- 172.21.15.63:0/226187663 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4018030 con 0x55a95e2f2e40 2024-05-27T07:59:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.603+0000 7f07a0c00700 1 -- 172.21.15.63:0/226187663 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9571+0+0 (secure 0 0 0) 0x7f07a4014120 con 0x55a95e2f2e40 2024-05-27T07:59:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.603+0000 7f07cc527b80 1 -- 172.21.15.63:0/226187663 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a801cab0 con 0x55a95e2f2e40 2024-05-27T07:59:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.604+0000 7f07a0c00700 1 -- 172.21.15.63:0/226187663 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f07a4028050 con 0x55a95e2f2e40 2024-05-27T07:59:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.604+0000 7f07cc527b80 10 client.20322 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.604+0000 7f07cc527b80 1 -- 172.21.15.63:0/226187663 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a95e360060 con 0x55a95e2f2e40 2024-05-27T07:59:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.604+0000 7f07a0c00700 1 -- 172.21.15.63:0/226187663 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a801cab0 con 0x55a95e2f2e40 2024-05-27T07:59:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.604+0000 7f07a0c00700 1 -- 172.21.15.63:0/226187663 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f07a4011770 con 0x55a95e2f2e40 2024-05-27T07:59:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.604+0000 7f07cc527b80 10 client.20322 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.604+0000 7f07cc527b80 10 client.20322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.604+0000 7f07cc527b80 20 client.20322 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.605+0000 7f07cc527b80 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a95e48b880 0x55a95e48dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.605+0000 7f07cc527b80 4 client.20322 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.605+0000 7f07cc527b80 1 -- 172.21.15.63:0/226187663 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a95e3a1d90 con 0x55a95e48b880 2024-05-27T07:59:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.605+0000 7f07b6a00700 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a95e48b880 0x55a95e48dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.607+0000 7f07b6a00700 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a95e48b880 0x55a95e48dca0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d501568941b5b7c server_cookie=d30cfe5927ec79b3 in_seq=0 out_seq=0 2024-05-27T07:59:03.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.607+0000 7f07a0c00700 10 client.20322 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:03.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.608+0000 7f07a0c00700 1 -- 172.21.15.63:0/226187663 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a95e3a1d90 con 0x55a95e48b880 2024-05-27T07:59:03.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07a0c00700 10 client.20322 handle_command_reply: tid=0 2024-05-27T07:59:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07cc527b80 1 client.20322 shutdown 2024-05-27T07:59:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07cc527b80 1 -- 172.21.15.63:0/226187663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e2f2e40 msgr2=0x55a95e47b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07cc527b80 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e2f2e40 0x55a95e47b550 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f07a4014d00 tx=0x7f07a4005f60 comp rx=0 tx=0).stop 2024-05-27T07:59:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07cc527b80 1 -- 172.21.15.63:0/226187663 shutdown_connections 2024-05-27T07:59:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07cc527b80 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a95e48b880 0x55a95e48dca0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07cc527b80 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471820 0x55a95e47bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07cc527b80 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e2f2e40 0x55a95e47b550 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07cc527b80 1 --2- 172.21.15.63:0/226187663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e471410 0x55a95e47ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07a0c00700 0 client.20322 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.609+0000 7f07cc527b80 1 -- 172.21.15.63:0/226187663 >> 172.21.15.63:0/226187663 conn(0x55a95e352fc0 msgr2=0x55a95e46bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.610+0000 7f07cc527b80 1 -- 172.21.15.63:0/226187663 shutdown_connections 2024-05-27T07:59:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.610+0000 7f07cc527b80 1 -- 172.21.15.63:0/226187663 wait complete. 2024-05-27T07:59:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.610+0000 7f07cc527b80 20 client.20322 trim_cache size 0 max 16384 2024-05-27T07:59:03.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.632+0000 7f07cc527b80 1 Processor -- start 2024-05-27T07:59:03.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.632+0000 7f07cc527b80 1 -- start start 2024-05-27T07:59:03.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e2f2e40 0x55a95e47a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471410 0x55a95e47d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471820 0x55a95e487a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07cc527b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b92e0 con 0x55a95e471820 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07cc527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b6b70 con 0x55a95e471410 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07cc527b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b49e0 con 0x55a95e2f2e40 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07b6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471820 0x55a95e487a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07b6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471820 0x55a95e487a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52538/0 (socket says 172.21.15.63:52538) 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07b6000700 1 -- 172.21.15.63:0/3625852935 learned_addr learned my addr 172.21.15.63:0/3625852935 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07b6000700 1 --2- 172.21.15.63:0/3625852935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e2f2e40 0x55a95e47a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.633+0000 7f07b6a00700 1 --2- 172.21.15.63:0/3625852935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471410 0x55a95e47d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.634+0000 7f07b6000700 1 -- 172.21.15.63:0/3625852935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e2f2e40 msgr2=0x55a95e47a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:03.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.634+0000 7f07b6000700 1 --2- 172.21.15.63:0/3625852935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e2f2e40 0x55a95e47a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.634+0000 7f07b6000700 1 -- 172.21.15.63:0/3625852935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471410 msgr2=0x55a95e47d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:03.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.634+0000 7f07b6000700 1 --2- 172.21.15.63:0/3625852935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471410 0x55a95e47d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.634+0000 7f07b6000700 1 -- 172.21.15.63:0/3625852935 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a95e47c340 con 0x55a95e471820 2024-05-27T07:59:03.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.634+0000 7f07b6000700 1 --2- 172.21.15.63:0/3625852935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471820 0x55a95e487a90 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f07a4011b70 tx=0x7f07a4016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e17d69745e9b8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:03.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.634+0000 7f07a3400700 1 -- 172.21.15.63:0/3625852935 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4018050 con 0x55a95e471820 2024-05-27T07:59:03.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.635+0000 7f07a3400700 1 -- 172.21.15.63:0/3625852935 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a40074e0 con 0x55a95e471820 2024-05-27T07:59:03.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.635+0000 7f07a3400700 1 -- 172.21.15.63:0/3625852935 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a401b040 con 0x55a95e471820 2024-05-27T07:59:03.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.636+0000 7f07cc527b80 1 -- 172.21.15.63:0/3625852935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471820 msgr2=0x55a95e487a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:03.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.636+0000 7f07cc527b80 1 --2- 172.21.15.63:0/3625852935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471820 0x55a95e487a90 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f07a4011b70 tx=0x7f07a4016040 comp rx=0 tx=0).stop 2024-05-27T07:59:03.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.636+0000 7f07cc527b80 1 -- 172.21.15.63:0/3625852935 shutdown_connections 2024-05-27T07:59:03.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.636+0000 7f07cc527b80 1 --2- 172.21.15.63:0/3625852935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471820 0x55a95e487a90 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.636+0000 7f07cc527b80 1 --2- 172.21.15.63:0/3625852935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e2f2e40 0x55a95e47a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.636+0000 7f07cc527b80 1 --2- 172.21.15.63:0/3625852935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e471410 0x55a95e47d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.636+0000 7f07cc527b80 1 -- 172.21.15.63:0/3625852935 >> 172.21.15.63:0/3625852935 conn(0x55a95e352fc0 msgr2=0x55a95e472880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:03.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.636+0000 7f07cc527b80 1 -- 172.21.15.63:0/3625852935 shutdown_connections 2024-05-27T07:59:03.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.636+0000 7f07cc527b80 1 -- 172.21.15.63:0/3625852935 wait complete. 2024-05-27T07:59:03.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.637+0000 7f07cc527b80 1 -- start start 2024-05-27T07:59:03.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.637+0000 7f07cc527b80 1 Processor -- start 2024-05-27T07:59:03.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e2f2e40 0x55a95e483d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e4842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07cc527b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e471820 0x55a95e47f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07cc527b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b5030 con 0x55a95e471410 2024-05-27T07:59:03.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07cc527b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b64a0 con 0x55a95e2f2e40 2024-05-27T07:59:03.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07cc527b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b5700 con 0x55a95e471820 2024-05-27T07:59:03.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07cc527b80 10 client.0 _ll_register_callbacks cb 0x55a95e339c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:03.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07b6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e4842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07b6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e4842b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52540/0 (socket says 172.21.15.63:52540) 2024-05-27T07:59:03.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07b6a00700 1 -- 172.21.15.63:0/2394912438 learned_addr learned my addr 172.21.15.63:0/2394912438 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:03.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07b6000700 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e471820 0x55a95e47f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.638+0000 7f07b6000700 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e2f2e40 0x55a95e483d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.639+0000 7f07b6a00700 1 -- 172.21.15.63:0/2394912438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e471820 msgr2=0x55a95e47f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:03.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.639+0000 7f07b6a00700 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e471820 0x55a95e47f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.639+0000 7f07b6a00700 1 -- 172.21.15.63:0/2394912438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e2f2e40 msgr2=0x55a95e483d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:03.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.639+0000 7f07b6a00700 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e2f2e40 0x55a95e483d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.639+0000 7f07b6a00700 1 -- 172.21.15.63:0/2394912438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a95e47f930 con 0x55a95e471410 2024-05-27T07:59:03.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.639+0000 7f07b6a00700 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e4842b0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f07a800b890 tx=0x7f07a8010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d58575c703f5f0e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:03.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.639+0000 7f07a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:03.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.639+0000 7f07a0c00700 1 -- 172.21.15.63:0/2394912438 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8023040 con 0x55a95e471410 2024-05-27T07:59:03.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.639+0000 7f07a0c00700 1 -- 172.21.15.63:0/2394912438 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9571+0+0 (secure 0 0 0) 0x7f07a80052c0 con 0x55a95e471410 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.639+0000 7f07cc527b80 1 -- 172.21.15.63:0/2394912438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a40074e0 con 0x55a95e471410 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.640+0000 7f07a0c00700 1 -- 172.21.15.63:0/2394912438 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f07a8036050 con 0x55a95e471410 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.640+0000 7f07cc527b80 10 client.20328 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.640+0000 7f07cc527b80 1 -- 172.21.15.63:0/2394912438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a95e4f5cf0 con 0x55a95e471410 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.640+0000 7f07a0c00700 1 -- 172.21.15.63:0/2394912438 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a40074e0 con 0x55a95e471410 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.640+0000 7f07a0c00700 1 -- 172.21.15.63:0/2394912438 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f07a8003680 con 0x55a95e471410 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.641+0000 7f07cc527b80 10 client.20328 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.641+0000 7f07cc527b80 10 client.20328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.641+0000 7f07cc527b80 20 client.20328 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.641+0000 7f07cc527b80 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a95e4f6150 0x55a95e4f8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.641+0000 7f07cc527b80 4 client.20328 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.641+0000 7f07cc527b80 1 -- 172.21.15.63:0/2394912438 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a95e39b570 con 0x55a95e4f6150 2024-05-27T07:59:03.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.641+0000 7f07b6000700 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a95e4f6150 0x55a95e4f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:03.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.643+0000 7f07b6000700 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a95e4f6150 0x55a95e4f8570 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7930705fd183604e server_cookie=edd3e6a82e4db2f in_seq=0 out_seq=0 2024-05-27T07:59:03.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.643+0000 7f07a0c00700 10 client.20328 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:03.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.643+0000 7f07a0c00700 1 -- 172.21.15.63:0/2394912438 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a95e39b570 con 0x55a95e4f6150 2024-05-27T07:59:03.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.643+0000 7f07a0c00700 10 client.20328 handle_command_reply: tid=0 2024-05-27T07:59:03.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.643+0000 7f07cc527b80 1 client.20328 shutdown 2024-05-27T07:59:03.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 1 -- 172.21.15.63:0/2394912438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 msgr2=0x55a95e4842b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:03.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e4842b0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f07a800b890 tx=0x7f07a8010bf0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 1 -- 172.21.15.63:0/2394912438 shutdown_connections 2024-05-27T07:59:03.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a95e4f6150 0x55a95e4f8570 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a95e471820 0x55a95e47f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a95e2f2e40 0x55a95e483d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 1 --2- 172.21.15.63:0/2394912438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a95e471410 0x55a95e4842b0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 1 -- 172.21.15.63:0/2394912438 >> 172.21.15.63:0/2394912438 conn(0x55a95e352fc0 msgr2=0x55a95e47bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:03.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07a0c00700 0 client.20328 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:03.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 1 -- 172.21.15.63:0/2394912438 shutdown_connections 2024-05-27T07:59:03.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 1 -- 172.21.15.63:0/2394912438 wait complete. 2024-05-27T07:59:03.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.644+0000 7f07cc527b80 20 client.20328 trim_cache size 0 max 16384 2024-05-27T07:59:03.647 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:59:03.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 -- 172.21.15.63:0/3845562413 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f079c0b29f0 msgr2=0x7f079c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:03.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f079c0b29f0 0x7f079c0b4f00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f07b00a6ed0 tx=0x7f07b0009b80 comp rx=0 tx=0).stop 2024-05-27T07:59:03.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 -- 172.21.15.63:0/3845562413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b810e710 msgr2=0x7f07b81aceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:03.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b810e710 0x7f07b81aceb0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b070 tx=0x7f07b0004d50 comp rx=0 tx=0).stop 2024-05-27T07:59:03.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 -- 172.21.15.63:0/3845562413 shutdown_connections 2024-05-27T07:59:03.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f079c0b29f0 0x7f079c0b4f00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07b810e710 0x7f07b81aceb0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b81052e0 0x7f07b81a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 --2- 172.21.15.63:0/3845562413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b8105c30 0x7f07b81a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:03.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 -- 172.21.15.63:0/3845562413 >> 172.21.15.63:0/3845562413 conn(0x7f07b8078ab0 msgr2=0x7f07b80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:03.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.645+0000 7f07b7e00700 1 -- 172.21.15.63:0/3845562413 shutdown_connections 2024-05-27T07:59:03.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:03.646+0000 7f07b7e00700 1 -- 172.21.15.63:0/3845562413 wait complete. 2024-05-27T07:59:03.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:03 smithi086 ceph-mon[34687]: pgmap v806: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.1 MiB/s wr, 32 op/s 2024-05-27T07:59:03.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:03 smithi086 ceph-mon[34687]: osdmap e133: 12 total, 12 up, 12 in 2024-05-27T07:59:03.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:03 smithi149 ceph-mon[35408]: pgmap v806: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.1 MiB/s wr, 32 op/s 2024-05-27T07:59:03.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:03 smithi149 ceph-mon[35408]: osdmap e133: 12 total, 12 up, 12 in 2024-05-27T07:59:05.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:05 smithi086 ceph-mon[34687]: pgmap v808: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.1 MiB/s wr, 34 op/s 2024-05-27T07:59:05.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:05 smithi149 ceph-mon[35408]: pgmap v808: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.1 MiB/s wr, 34 op/s 2024-05-27T07:59:05.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:05 smithi063 ceph-mon[31613]: pgmap v808: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.1 MiB/s wr, 34 op/s 2024-05-27T07:59:05.725 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:59:05.725 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:59:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.268+0000 7fba45400640 1 -- 172.21.15.63:0/3559679191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4010a170 msgr2=0x7fba40103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.268+0000 7fba45400640 1 --2- 172.21.15.63:0/3559679191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4010a170 0x7fba40103db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009860 tx=0x7fba2c02a340 comp rx=0 tx=0).stop 2024-05-27T07:59:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.268+0000 7fba45400640 1 -- 172.21.15.63:0/3559679191 shutdown_connections 2024-05-27T07:59:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.268+0000 7fba45400640 1 --2- 172.21.15.63:0/3559679191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba40104420 0x7fba401107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.268+0000 7fba45400640 1 --2- 172.21.15.63:0/3559679191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4010a170 0x7fba40103db0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.268+0000 7fba45400640 1 --2- 172.21.15.63:0/3559679191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba40109cd0 0x7fba40103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.268+0000 7fba45400640 1 -- 172.21.15.63:0/3559679191 >> 172.21.15.63:0/3559679191 conn(0x7fba40079440 msgr2=0x7fba40079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.268+0000 7fba45400640 1 -- 172.21.15.63:0/3559679191 shutdown_connections 2024-05-27T07:59:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.268+0000 7fba45400640 1 -- 172.21.15.63:0/3559679191 wait complete. 2024-05-27T07:59:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.270+0000 7fba45400640 1 Processor -- start 2024-05-27T07:59:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.270+0000 7fba45400640 1 -- start start 2024-05-27T07:59:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba45400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba40104420 0x7fba401a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba45400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba40109cd0 0x7fba401a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba45400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4010a170 0x7fba4019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba45400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba40113d70 con 0x7fba40109cd0 2024-05-27T07:59:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba45400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba40113bf0 con 0x7fba40104420 2024-05-27T07:59:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba45400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba40113ef0 con 0x7fba4010a170 2024-05-27T07:59:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba3de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba40109cd0 0x7fba401a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba3de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba40109cd0 0x7fba401a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52550/0 (socket says 172.21.15.63:52550) 2024-05-27T07:59:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba3de00640 1 -- 172.21.15.63:0/3205931716 learned_addr learned my addr 172.21.15.63:0/3205931716 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba3e800640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba40104420 0x7fba401a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.271+0000 7fba3f200640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4010a170 0x7fba4019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.272+0000 7fba3de00640 1 -- 172.21.15.63:0/3205931716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4010a170 msgr2=0x7fba4019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.272+0000 7fba3de00640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4010a170 0x7fba4019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.272+0000 7fba3de00640 1 -- 172.21.15.63:0/3205931716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba40104420 msgr2=0x7fba401a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.272+0000 7fba3de00640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba40104420 0x7fba401a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.272+0000 7fba3de00640 1 -- 172.21.15.63:0/3205931716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4019adc0 con 0x7fba40109cd0 2024-05-27T07:59:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.273+0000 7fba3de00640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba40109cd0 0x7fba401a0a50 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fba2c005940 tx=0x7fba2c030d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.273+0000 7fba37400640 1 -- 172.21.15.63:0/3205931716 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c040020 con 0x7fba40109cd0 2024-05-27T07:59:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.273+0000 7fba37400640 1 -- 172.21.15.63:0/3205931716 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba2c032df0 con 0x7fba40109cd0 2024-05-27T07:59:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.273+0000 7fba45400640 1 -- 172.21.15.63:0/3205931716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4019aff0 con 0x7fba40109cd0 2024-05-27T07:59:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.273+0000 7fba37400640 1 -- 172.21.15.63:0/3205931716 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c037040 con 0x7fba40109cd0 2024-05-27T07:59:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.273+0000 7fba45400640 1 -- 172.21.15.63:0/3205931716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4019b4e0 con 0x7fba40109cd0 2024-05-27T07:59:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.278+0000 7fba4c416740 1 Processor -- start 2024-05-27T07:59:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.279+0000 7fba4c416740 1 -- start start 2024-05-27T07:59:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.279+0000 7fba4c416740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 0x56452b893860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.279+0000 7fba4c416740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b893da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.279+0000 7fba4c416740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4d04d0 0x56452b8942e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.279+0000 7fba4c416740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56452b804b90 con 0x56452b4c82f0 2024-05-27T07:59:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.279+0000 7fba4c416740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56452b7f4730 con 0x56452b4d04d0 2024-05-27T07:59:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.279+0000 7fba4c416740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56452b7f8770 con 0x56452b4f11b0 2024-05-27T07:59:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.280+0000 7fba3f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 0x56452b893860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.280+0000 7fba3e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b893da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.280+0000 7fba3f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 0x56452b893860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:59584/0 (socket says 172.21.15.63:59584) 2024-05-27T07:59:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.280+0000 7fba3f200640 1 -- 172.21.15.63:0/3403693991 learned_addr learned my addr 172.21.15.63:0/3403693991 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.281+0000 7fba3f200640 1 -- 172.21.15.63:0/3403693991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4d04d0 msgr2=0x56452b8942e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:59:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.281+0000 7fba3f200640 1 --2- 172.21.15.63:0/3403693991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4d04d0 0x56452b8942e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.281+0000 7fba3f200640 1 -- 172.21.15.63:0/3403693991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 msgr2=0x56452b893da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.281+0000 7fba3f200640 1 --2- 172.21.15.63:0/3403693991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b893da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.281+0000 7fba3f200640 1 -- 172.21.15.63:0/3403693991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56452b78d8b0 con 0x56452b4f11b0 2024-05-27T07:59:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.281+0000 7fba37400640 1 -- 172.21.15.63:0/3205931716 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fba2c043020 con 0x7fba40109cd0 2024-05-27T07:59:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.281+0000 7fba3f200640 1 --2- 172.21.15.63:0/3403693991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 0x56452b893860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba38010370 tx=0x7fba38010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5bb21692a331a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.282+0000 7fba2b400640 1 -- 172.21.15.63:0/3403693991 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba38016070 con 0x56452b4f11b0 2024-05-27T07:59:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.282+0000 7fba2b400640 1 -- 172.21.15.63:0/3403693991 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba38004d20 con 0x56452b4f11b0 2024-05-27T07:59:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.282+0000 7fba2b400640 1 -- 172.21.15.63:0/3403693991 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba380052c0 con 0x56452b4f11b0 2024-05-27T07:59:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.283+0000 7fba4c416740 1 -- 172.21.15.63:0/3403693991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 msgr2=0x56452b893860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.283+0000 7fba4c416740 1 --2- 172.21.15.63:0/3403693991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 0x56452b893860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba38010370 tx=0x7fba38010830 comp rx=0 tx=0).stop 2024-05-27T07:59:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.283+0000 7fba37400640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba240b2aa0 0x7fba240b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.284+0000 7fba4c416740 1 -- 172.21.15.63:0/3403693991 shutdown_connections 2024-05-27T07:59:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.284+0000 7fba37400640 1 -- 172.21.15.63:0/3205931716 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9571+0+0 (secure 0 0 0) 0x7fba2c102940 con 0x7fba40109cd0 2024-05-27T07:59:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.284+0000 7fba4c416740 1 --2- 172.21.15.63:0/3403693991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4d04d0 0x56452b8942e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.284+0000 7fba4c416740 1 --2- 172.21.15.63:0/3403693991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b893da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.284+0000 7fba4c416740 1 --2- 172.21.15.63:0/3403693991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 0x56452b893860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.284+0000 7fba4c416740 1 -- 172.21.15.63:0/3403693991 >> 172.21.15.63:0/3403693991 conn(0x56452b4e5300 msgr2=0x56452b890010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.284+0000 7fba3f200640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba240b2aa0 0x7fba240b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.284+0000 7fba4c416740 1 -- 172.21.15.63:0/3403693991 shutdown_connections 2024-05-27T07:59:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.284+0000 7fba4c416740 1 -- 172.21.15.63:0/3403693991 wait complete. 2024-05-27T07:59:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.285+0000 7fba3f200640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba240b2aa0 0x7fba240b4fb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fba38009db0 tx=0x7fba38009bb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.285+0000 7fba4c416740 1 -- start start 2024-05-27T07:59:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.286+0000 7fba4c416740 1 Processor -- start 2024-05-27T07:59:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.286+0000 7fba4c416740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.287+0000 7fba4c416740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4d04d0 0x56452b89df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.287+0000 7fba4c416740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 0x56452b8a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.287+0000 7fba4c416740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56452b4da790 con 0x56452b4c82f0 2024-05-27T07:59:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.287+0000 7fba4c416740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56452b79cbc0 con 0x56452b4d04d0 2024-05-27T07:59:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.287+0000 7fba4c416740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56452b8a0990 con 0x56452b4f11b0 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.287+0000 7fba3e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.287+0000 7fba3e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52574/0 (socket says 172.21.15.63:52574) 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.287+0000 7fba3e800640 1 -- 172.21.15.63:0/3507202380 learned_addr learned my addr 172.21.15.63:0/3507202380 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.287+0000 7fba4c416740 10 client.0 _ll_register_callbacks cb 0x56452b77a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.287+0000 7fba3f200640 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 0x56452b8a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.288+0000 7fba3de00640 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4d04d0 0x56452b89df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.288+0000 7fba3e800640 1 -- 172.21.15.63:0/3507202380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 msgr2=0x56452b8a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.288+0000 7fba3e800640 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 0x56452b8a9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.288+0000 7fba3e800640 1 -- 172.21.15.63:0/3507202380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4d04d0 msgr2=0x56452b89df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.288+0000 7fba3e800640 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4d04d0 0x56452b89df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.288+0000 7fba3e800640 1 -- 172.21.15.63:0/3507202380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56452b509210 con 0x56452b4c82f0 2024-05-27T07:59:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.288+0000 7fba3de00640 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4d04d0 0x56452b89df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.288+0000 7fba3e800640 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89d9c0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fba401049b0 tx=0x7fba30003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad027504449ebed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.288+0000 7fba1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.289+0000 7fba1fe00640 1 -- 172.21.15.63:0/3507202380 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3001f030 con 0x56452b4c82f0 2024-05-27T07:59:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.289+0000 7fba1fe00640 1 -- 172.21.15.63:0/3507202380 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9571+0+0 (secure 0 0 0) 0x7fba30005020 con 0x56452b4c82f0 2024-05-27T07:59:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.289+0000 7fba4c416740 1 -- 172.21.15.63:0/3507202380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba38004d20 con 0x56452b4c82f0 2024-05-27T07:59:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.289+0000 7fba1fe00640 1 -- 172.21.15.63:0/3507202380 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fba30029050 con 0x56452b4c82f0 2024-05-27T07:59:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.289+0000 7fba4c416740 10 client.20340 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.289+0000 7fba4c416740 1 -- 172.21.15.63:0/3507202380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56452b5d5620 con 0x56452b4c82f0 2024-05-27T07:59:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.290+0000 7fba1fe00640 1 -- 172.21.15.63:0/3507202380 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba38004d20 con 0x56452b4c82f0 2024-05-27T07:59:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.290+0000 7fba1fe00640 1 -- 172.21.15.63:0/3507202380 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fba300047a0 con 0x56452b4c82f0 2024-05-27T07:59:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.290+0000 7fba4c416740 10 client.20340 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.290+0000 7fba4c416740 10 client.20340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.290+0000 7fba4c416740 20 client.20340 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.290+0000 7fba4c416740 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56452b4afd80 0x56452b8afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.290+0000 7fba4c416740 4 client.20340 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.290+0000 7fba4c416740 1 -- 172.21.15.63:0/3507202380 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56452b806ae0 con 0x56452b4afd80 2024-05-27T07:59:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.291+0000 7fba3de00640 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56452b4afd80 0x56452b8afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.292+0000 7fba3de00640 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56452b4afd80 0x56452b8afde0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1bc64c9080b64ba server_cookie=fe16086838afbe79 in_seq=0 out_seq=0 2024-05-27T07:59:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.292+0000 7fba1fe00640 10 client.20340 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.294+0000 7fba1fe00640 1 -- 172.21.15.63:0/3507202380 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56452b806ae0 con 0x56452b4afd80 2024-05-27T07:59:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.294+0000 7fba1fe00640 10 client.20340 handle_command_reply: tid=0 2024-05-27T07:59:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.295+0000 7fba4c416740 1 client.20340 shutdown 2024-05-27T07:59:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.295+0000 7fba4c416740 1 -- 172.21.15.63:0/3507202380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 msgr2=0x56452b89d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.295+0000 7fba4c416740 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89d9c0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fba401049b0 tx=0x7fba30003a80 comp rx=0 tx=0).stop 2024-05-27T07:59:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.295+0000 7fba4c416740 1 -- 172.21.15.63:0/3507202380 shutdown_connections 2024-05-27T07:59:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.295+0000 7fba4c416740 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56452b4afd80 0x56452b8afde0 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.295+0000 7fba4c416740 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4f11b0 0x56452b8a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.295+0000 7fba1fe00640 0 client.20340 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.295+0000 7fba4c416740 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4d04d0 0x56452b89df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.295+0000 7fba4c416740 1 --2- 172.21.15.63:0/3507202380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89d9c0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.295+0000 7fba4c416740 1 -- 172.21.15.63:0/3507202380 >> 172.21.15.63:0/3507202380 conn(0x56452b4e5300 msgr2=0x56452b893690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.296+0000 7fba4c416740 1 -- 172.21.15.63:0/3507202380 shutdown_connections 2024-05-27T07:59:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.296+0000 7fba4c416740 1 -- 172.21.15.63:0/3507202380 wait complete. 2024-05-27T07:59:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.296+0000 7fba4c416740 20 client.20340 trim_cache size 0 max 16384 2024-05-27T07:59:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.331+0000 7fba4c416740 1 Processor -- start 2024-05-27T07:59:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.331+0000 7fba4c416740 1 -- start start 2024-05-27T07:59:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba4c416740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4afd80 0x56452b899210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba4c416740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba4c416740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4d04d0 0x56452b89d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba4c416740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba200129d0 con 0x56452b4c82f0 2024-05-27T07:59:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba4c416740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba200102a0 con 0x56452b4afd80 2024-05-27T07:59:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba4c416740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba2000e110 con 0x56452b4d04d0 2024-05-27T07:59:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba3de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba3e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4afd80 0x56452b899210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba3de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52584/0 (socket says 172.21.15.63:52584) 2024-05-27T07:59:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba3f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4d04d0 0x56452b89d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.332+0000 7fba3de00640 1 -- 172.21.15.63:0/266406608 learned_addr learned my addr 172.21.15.63:0/266406608 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.333+0000 7fba3e800640 1 -- 172.21.15.63:0/266406608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4d04d0 msgr2=0x56452b89d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.333+0000 7fba3e800640 1 --2- 172.21.15.63:0/266406608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4d04d0 0x56452b89d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.333+0000 7fba3e800640 1 -- 172.21.15.63:0/266406608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 msgr2=0x56452b89fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.333+0000 7fba3e800640 1 --2- 172.21.15.63:0/266406608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.333+0000 7fba3e800640 1 -- 172.21.15.63:0/266406608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56452b894bf0 con 0x56452b4afd80 2024-05-27T07:59:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.333+0000 7fba3de00640 1 --2- 172.21.15.63:0/266406608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.334+0000 7fba3e800640 1 --2- 172.21.15.63:0/266406608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4afd80 0x56452b899210 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba30000c00 tx=0x7fba300092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dea9777b15669d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.334+0000 7fba2aa00640 1 -- 172.21.15.63:0/266406608 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30031020 con 0x56452b4afd80 2024-05-27T07:59:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.334+0000 7fba2aa00640 1 -- 172.21.15.63:0/266406608 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba30004880 con 0x56452b4afd80 2024-05-27T07:59:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.334+0000 7fba2aa00640 1 -- 172.21.15.63:0/266406608 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3001f040 con 0x56452b4afd80 2024-05-27T07:59:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.336+0000 7fba4c416740 1 -- 172.21.15.63:0/266406608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4afd80 msgr2=0x56452b899210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.336+0000 7fba4c416740 1 --2- 172.21.15.63:0/266406608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4afd80 0x56452b899210 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba30000c00 tx=0x7fba300092c0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.336+0000 7fba4c416740 1 -- 172.21.15.63:0/266406608 shutdown_connections 2024-05-27T07:59:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.336+0000 7fba4c416740 1 --2- 172.21.15.63:0/266406608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4d04d0 0x56452b89d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.336+0000 7fba4c416740 1 --2- 172.21.15.63:0/266406608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4c82f0 0x56452b89fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.336+0000 7fba4c416740 1 --2- 172.21.15.63:0/266406608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4afd80 0x56452b899210 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.336+0000 7fba4c416740 1 -- 172.21.15.63:0/266406608 >> 172.21.15.63:0/266406608 conn(0x56452b4e5300 msgr2=0x56452b89d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.336+0000 7fba4c416740 1 -- 172.21.15.63:0/266406608 shutdown_connections 2024-05-27T07:59:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.336+0000 7fba4c416740 1 -- 172.21.15.63:0/266406608 wait complete. 2024-05-27T07:59:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.337+0000 7fba4c416740 1 -- start start 2024-05-27T07:59:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.337+0000 7fba4c416740 1 Processor -- start 2024-05-27T07:59:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.338+0000 7fba4c416740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4afd80 0x56452b8a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.338+0000 7fba4c416740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4c82f0 0x56452b8a6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.338+0000 7fba4c416740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4d04d0 0x56452b8a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.338+0000 7fba4c416740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba2000e760 con 0x56452b4afd80 2024-05-27T07:59:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.338+0000 7fba4c416740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba2000fbd0 con 0x56452b4c82f0 2024-05-27T07:59:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.338+0000 7fba4c416740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba2000ee30 con 0x56452b4d04d0 2024-05-27T07:59:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.338+0000 7fba4c416740 10 client.0 _ll_register_callbacks cb 0x7fba3000fcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.339+0000 7fba3e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4afd80 0x56452b8a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.339+0000 7fba3e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4afd80 0x56452b8a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52598/0 (socket says 172.21.15.63:52598) 2024-05-27T07:59:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.339+0000 7fba3e800640 1 -- 172.21.15.63:0/203365219 learned_addr learned my addr 172.21.15.63:0/203365219 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.339+0000 7fba3de00640 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4c82f0 0x56452b8a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.339+0000 7fba3f200640 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4d04d0 0x56452b8a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.339+0000 7fba3e800640 1 -- 172.21.15.63:0/203365219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4d04d0 msgr2=0x56452b8a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.339+0000 7fba3e800640 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4d04d0 0x56452b8a31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.339+0000 7fba3e800640 1 -- 172.21.15.63:0/203365219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4c82f0 msgr2=0x56452b8a6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.339+0000 7fba3e800640 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4c82f0 0x56452b8a6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.339+0000 7fba3e800640 1 -- 172.21.15.63:0/203365219 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56452b8a37b0 con 0x56452b4afd80 2024-05-27T07:59:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.340+0000 7fba3e800640 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4afd80 0x56452b8a5cc0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fba30005d60 tx=0x7fba30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7d646d3ca69232 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.340+0000 7fba1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.340+0000 7fba1fe00640 1 -- 172.21.15.63:0/203365219 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30031030 con 0x56452b4afd80 2024-05-27T07:59:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.340+0000 7fba1fe00640 1 -- 172.21.15.63:0/203365219 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9571+0+0 (secure 0 0 0) 0x7fba30009c00 con 0x56452b4afd80 2024-05-27T07:59:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.340+0000 7fba4c416740 1 -- 172.21.15.63:0/203365219 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba30004880 con 0x56452b4afd80 2024-05-27T07:59:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.341+0000 7fba1fe00640 1 -- 172.21.15.63:0/203365219 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fba30046050 con 0x56452b4afd80 2024-05-27T07:59:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.341+0000 7fba4c416740 10 client.20346 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.341+0000 7fba4c416740 1 -- 172.21.15.63:0/203365219 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56452b917e40 con 0x56452b4afd80 2024-05-27T07:59:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.341+0000 7fba1fe00640 1 -- 172.21.15.63:0/203365219 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba30004880 con 0x56452b4afd80 2024-05-27T07:59:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.342+0000 7fba1fe00640 1 -- 172.21.15.63:0/203365219 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fba3001b450 con 0x56452b4afd80 2024-05-27T07:59:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.342+0000 7fba4c416740 10 client.20346 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.342+0000 7fba4c416740 10 client.20346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.342+0000 7fba4c416740 20 client.20346 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.342+0000 7fba4c416740 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56452b4f11b0 0x56452b91a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.342+0000 7fba4c416740 4 client.20346 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:59:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.342+0000 7fba4c416740 1 -- 172.21.15.63:0/203365219 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56452b8002c0 con 0x56452b4f11b0 2024-05-27T07:59:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.343+0000 7fba3de00640 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56452b4f11b0 0x56452b91a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.344+0000 7fba1fe00640 10 client.20346 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.344+0000 7fba3de00640 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56452b4f11b0 0x56452b91a490 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc69e49aa6c0ab9 server_cookie=63a34f095ea8220c in_seq=0 out_seq=0 2024-05-27T07:59:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba1fe00640 1 -- 172.21.15.63:0/203365219 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56452b8002c0 con 0x56452b4f11b0 2024-05-27T07:59:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba1fe00640 10 client.20346 handle_command_reply: tid=0 2024-05-27T07:59:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba4c416740 1 client.20346 shutdown 2024-05-27T07:59:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba4c416740 1 -- 172.21.15.63:0/203365219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4afd80 msgr2=0x56452b8a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba4c416740 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4afd80 0x56452b8a5cc0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fba30005d60 tx=0x7fba30004730 comp rx=0 tx=0).stop 2024-05-27T07:59:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba4c416740 1 -- 172.21.15.63:0/203365219 shutdown_connections 2024-05-27T07:59:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba4c416740 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56452b4f11b0 0x56452b91a490 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba4c416740 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452b4d04d0 0x56452b8a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba4c416740 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452b4c82f0 0x56452b8a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba4c416740 1 --2- 172.21.15.63:0/203365219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452b4afd80 0x56452b8a5cc0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.379+0000 7fba4c416740 1 -- 172.21.15.63:0/203365219 >> 172.21.15.63:0/203365219 conn(0x56452b4e5300 msgr2=0x56452b89ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.380+0000 7fba1fe00640 0 client.20346 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.380+0000 7fba4c416740 1 -- 172.21.15.63:0/203365219 shutdown_connections 2024-05-27T07:59:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.380+0000 7fba4c416740 1 -- 172.21.15.63:0/203365219 wait complete. 2024-05-27T07:59:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.380+0000 7fba4c416740 20 client.20346 trim_cache size 0 max 16384 2024-05-27T07:59:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:59:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:59:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:59:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 -- 172.21.15.63:0/3205931716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba240b2aa0 msgr2=0x7fba240b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba240b2aa0 0x7fba240b4fb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fba38009db0 tx=0x7fba38009bb0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 -- 172.21.15.63:0/3205931716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba40109cd0 msgr2=0x7fba401a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba40109cd0 0x7fba401a0a50 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fba2c005940 tx=0x7fba2c030d60 comp rx=0 tx=0).stop 2024-05-27T07:59:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 -- 172.21.15.63:0/3205931716 shutdown_connections 2024-05-27T07:59:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba240b2aa0 0x7fba240b4fb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba4010a170 0x7fba4019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba40109cd0 0x7fba401a0a50 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 --2- 172.21.15.63:0/3205931716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba40104420 0x7fba401a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 -- 172.21.15.63:0/3205931716 >> 172.21.15.63:0/3205931716 conn(0x7fba40079440 msgr2=0x7fba40101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 -- 172.21.15.63:0/3205931716 shutdown_connections 2024-05-27T07:59:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:06.381+0000 7fba45400640 1 -- 172.21.15.63:0/3205931716 wait complete. 2024-05-27T07:59:06.587 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:06 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:59:06.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:06 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:59:06.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:06 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:59:07.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:07 smithi086 ceph-mon[34687]: pgmap v809: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.1 MiB/s wr, 34 op/s 2024-05-27T07:59:07.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:07 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:59:07.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:07 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:59:07.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:07 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:59:07.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:07 smithi149 ceph-mon[35408]: pgmap v809: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.1 MiB/s wr, 34 op/s 2024-05-27T07:59:07.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:07 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:59:07.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:07 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:59:07.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:07 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:59:07.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:07 smithi063 ceph-mon[31613]: pgmap v809: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.1 MiB/s wr, 34 op/s 2024-05-27T07:59:07.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:07 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:59:07.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:07 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:59:07.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:07 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T07:59:08.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.705+0000 7fb31b200700 1 -- 172.21.15.63:0/737197181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb31c105c50 msgr2=0x7fb31c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:08.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.705+0000 7fb31b200700 1 --2- 172.21.15.63:0/737197181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb31c105c50 0x7fb31c10e1a0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fb308009990 tx=0x7fb308009ca0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.705+0000 7fb31b200700 1 -- 172.21.15.63:0/737197181 shutdown_connections 2024-05-27T07:59:08.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.705+0000 7fb31b200700 1 --2- 172.21.15.63:0/737197181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb31c10e730 0x7fb31c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.705+0000 7fb31b200700 1 --2- 172.21.15.63:0/737197181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb31c105300 0x7fb31c105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.705+0000 7fb31b200700 1 --2- 172.21.15.63:0/737197181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb31c105c50 0x7fb31c10e1a0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.705+0000 7fb31b200700 1 -- 172.21.15.63:0/737197181 >> 172.21.15.63:0/737197181 conn(0x7fb31c078ab0 msgr2=0x7fb31c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:08.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.706+0000 7fb31b200700 1 -- 172.21.15.63:0/737197181 shutdown_connections 2024-05-27T07:59:08.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.706+0000 7fb31b200700 1 -- 172.21.15.63:0/737197181 wait complete. 2024-05-27T07:59:08.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.707+0000 7fb31b200700 1 Processor -- start 2024-05-27T07:59:08.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.707+0000 7fb31b200700 1 -- start start 2024-05-27T07:59:08.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.708+0000 7fb31b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb31c105300 0x7fb31c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.708+0000 7fb31b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb31c105c50 0x7fb31c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.708+0000 7fb31b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb31c10e730 0x7fb31c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.708+0000 7fb31b200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c114100 con 0x7fb31c105c50 2024-05-27T07:59:08.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.708+0000 7fb31b200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c113f80 con 0x7fb31c105300 2024-05-27T07:59:08.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.708+0000 7fb31b200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c114280 con 0x7fb31c10e730 2024-05-27T07:59:08.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.709+0000 7fb319400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb31c105c50 0x7fb31c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.709+0000 7fb319e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb31c105300 0x7fb31c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.709+0000 7fb319400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb31c105c50 0x7fb31c1a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52612/0 (socket says 172.21.15.63:52612) 2024-05-27T07:59:08.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.709+0000 7fb319400700 1 -- 172.21.15.63:0/3538851277 learned_addr learned my addr 172.21.15.63:0/3538851277 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:08.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.709+0000 7fb31a800700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb31c10e730 0x7fb31c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.709+0000 7fb319400700 1 -- 172.21.15.63:0/3538851277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb31c10e730 msgr2=0x7fb31c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:08.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.709+0000 7fb319400700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb31c10e730 0x7fb31c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.709+0000 7fb319400700 1 -- 172.21.15.63:0/3538851277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb31c105300 msgr2=0x7fb31c1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:08.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.709+0000 7fb319400700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb31c105300 0x7fb31c1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.709+0000 7fb319400700 1 -- 172.21.15.63:0/3538851277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c19b0b0 con 0x7fb31c105c50 2024-05-27T07:59:08.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.710+0000 7fb319400700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb31c105c50 0x7fb31c1a0d20 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fb3080057c0 tx=0x7fb3080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:08.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.711+0000 7fb312a00700 1 -- 172.21.15.63:0/3538851277 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308027070 con 0x7fb31c105c50 2024-05-27T07:59:08.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.711+0000 7fb312a00700 1 -- 172.21.15.63:0/3538851277 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb30800fb00 con 0x7fb31c105c50 2024-05-27T07:59:08.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.711+0000 7fb312a00700 1 -- 172.21.15.63:0/3538851277 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308022ad0 con 0x7fb31c105c50 2024-05-27T07:59:08.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.711+0000 7fb31b200700 1 -- 172.21.15.63:0/3538851277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c19b340 con 0x7fb31c105c50 2024-05-27T07:59:08.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.711+0000 7fb31b200700 1 -- 172.21.15.63:0/3538851277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c19b7d0 con 0x7fb31c105c50 2024-05-27T07:59:08.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.716+0000 7fb312a00700 1 -- 172.21.15.63:0/3538851277 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb30802a070 con 0x7fb31c105c50 2024-05-27T07:59:08.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.716+0000 7fb32f607b80 1 Processor -- start 2024-05-27T07:59:08.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.716+0000 7fb32f607b80 1 -- start start 2024-05-27T07:59:08.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.717+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841a5ea70 0x556841af27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.717+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.717+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841afe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.717+0000 7fb32f607b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556841a1fec0 con 0x556841972ec0 2024-05-27T07:59:08.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.717+0000 7fb32f607b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556841a0fa60 con 0x556841a5ea70 2024-05-27T07:59:08.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.717+0000 7fb32f607b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556841a13aa0 con 0x556841afc340 2024-05-27T07:59:08.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52620/0 (socket says 172.21.15.63:52620) 2024-05-27T07:59:08.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb319e00700 1 -- 172.21.15.63:0/3770233456 learned_addr learned my addr 172.21.15.63:0/3770233456 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:08.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb31a800700 1 --2- 172.21.15.63:0/3770233456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841a5ea70 0x556841af27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb319400700 1 --2- 172.21.15.63:0/3770233456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841afe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb319e00700 1 -- 172.21.15.63:0/3770233456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 msgr2=0x556841afe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:08.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb319e00700 1 --2- 172.21.15.63:0/3770233456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841afe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb319e00700 1 -- 172.21.15.63:0/3770233456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841a5ea70 msgr2=0x556841af27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:08.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb319e00700 1 --2- 172.21.15.63:0/3770233456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841a5ea70 0x556841af27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb319e00700 1 -- 172.21.15.63:0/3770233456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568419d84a0 con 0x556841972ec0 2024-05-27T07:59:08.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.718+0000 7fb312a00700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3000b2a70 0x7fb3000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.719+0000 7fb312a00700 1 -- 172.21.15.63:0/3538851277 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9571+0+0 (secure 0 0 0) 0x7fb3080e3a40 con 0x7fb31c105c50 2024-05-27T07:59:08.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.719+0000 7fb319e00700 1 --2- 172.21.15.63:0/3770233456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af2d10 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00bd30 tx=0x7fb30c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb32a95c0951d5cd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:08.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.719+0000 7fb31a800700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3000b2a70 0x7fb3000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.719+0000 7fb307400700 1 -- 172.21.15.63:0/3770233456 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c010e10 con 0x556841972ec0 2024-05-27T07:59:08.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.719+0000 7fb307400700 1 -- 172.21.15.63:0/3770233456 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb30c01c910 con 0x556841972ec0 2024-05-27T07:59:08.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.719+0000 7fb307400700 1 -- 172.21.15.63:0/3770233456 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0117c0 con 0x556841972ec0 2024-05-27T07:59:08.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.721+0000 7fb32f607b80 1 -- 172.21.15.63:0/3770233456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 msgr2=0x556841af2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:08.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.721+0000 7fb32f607b80 1 --2- 172.21.15.63:0/3770233456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af2d10 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00bd30 tx=0x7fb30c0106b0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.721+0000 7fb31a800700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3000b2a70 0x7fb3000b4f80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x556841aeeb10 tx=0x7fb314009100 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:08.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.721+0000 7fb32f607b80 1 -- 172.21.15.63:0/3770233456 shutdown_connections 2024-05-27T07:59:08.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.721+0000 7fb32f607b80 1 --2- 172.21.15.63:0/3770233456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841afe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.721+0000 7fb32f607b80 1 --2- 172.21.15.63:0/3770233456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841a5ea70 0x556841af27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.721+0000 7fb32f607b80 1 --2- 172.21.15.63:0/3770233456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af2d10 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.721+0000 7fb32f607b80 1 -- 172.21.15.63:0/3770233456 >> 172.21.15.63:0/3770233456 conn(0x556841a0c410 msgr2=0x556841aed790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:08.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.721+0000 7fb32f607b80 1 -- 172.21.15.63:0/3770233456 shutdown_connections 2024-05-27T07:59:08.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.721+0000 7fb32f607b80 1 -- 172.21.15.63:0/3770233456 wait complete. 2024-05-27T07:59:08.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.722+0000 7fb32f607b80 1 -- start start 2024-05-27T07:59:08.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.722+0000 7fb32f607b80 1 Processor -- start 2024-05-27T07:59:08.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841972ec0 0x556841af92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841a5ea70 0x556841af97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841af9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb32f607b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556841916020 con 0x556841a5ea70 2024-05-27T07:59:08.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb32f607b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568419b07e0 con 0x556841972ec0 2024-05-27T07:59:08.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb32f607b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556841afee80 con 0x556841afc340 2024-05-27T07:59:08.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb32f607b80 10 client.0 _ll_register_callbacks cb 0x5568419b9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:08.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb319400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841a5ea70 0x556841af97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb319400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841a5ea70 0x556841af97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52626/0 (socket says 172.21.15.63:52626) 2024-05-27T07:59:08.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb319400700 1 -- 172.21.15.63:0/390173039 learned_addr learned my addr 172.21.15.63:0/390173039 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:08.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.723+0000 7fb319e00700 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841972ec0 0x556841af92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.724+0000 7fb31a800700 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841af9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.724+0000 7fb319400700 1 -- 172.21.15.63:0/390173039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 msgr2=0x556841af9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:08.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.724+0000 7fb319400700 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841af9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.724+0000 7fb319400700 1 -- 172.21.15.63:0/390173039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841972ec0 msgr2=0x556841af92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:08.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.724+0000 7fb319400700 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841972ec0 0x556841af92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.724+0000 7fb319400700 1 -- 172.21.15.63:0/390173039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55684195f950 con 0x556841a5ea70 2024-05-27T07:59:08.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.724+0000 7fb31a800700 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841af9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:08.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.724+0000 7fb319e00700 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841972ec0 0x556841af92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:08.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.724+0000 7fb319400700 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841a5ea70 0x556841af97e0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fb3080e0dd0 tx=0x7fb308009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395da9f4030bbaac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:08.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.724+0000 7fb2fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:08.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.725+0000 7fb2fbe00700 1 -- 172.21.15.63:0/390173039 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3080a6120 con 0x556841a5ea70 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.725+0000 7fb2fbe00700 1 -- 172.21.15.63:0/390173039 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9571+0+0 (secure 0 0 0) 0x7fb3080b3040 con 0x556841a5ea70 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.725+0000 7fb32f607b80 1 -- 172.21.15.63:0/390173039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb30c01c910 con 0x556841a5ea70 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.725+0000 7fb2fbe00700 1 -- 172.21.15.63:0/390173039 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb3080aa4a0 con 0x556841a5ea70 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.725+0000 7fb32f607b80 10 client.20358 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.725+0000 7fb32f607b80 1 -- 172.21.15.63:0/390173039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5568419e00e0 con 0x556841a5ea70 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.725+0000 7fb2fbe00700 1 -- 172.21.15.63:0/390173039 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb30c01c910 con 0x556841a5ea70 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.726+0000 7fb2fbe00700 1 -- 172.21.15.63:0/390173039 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb3080b1070 con 0x556841a5ea70 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.726+0000 7fb32f607b80 10 client.20358 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.726+0000 7fb32f607b80 10 client.20358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.726+0000 7fb32f607b80 20 client.20358 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.726+0000 7fb32f607b80 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556841b0bc50 0x556841b0e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.726+0000 7fb32f607b80 4 client.20358 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:08.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.726+0000 7fb32f607b80 1 -- 172.21.15.63:0/390173039 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556841a21e10 con 0x556841b0bc50 2024-05-27T07:59:08.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.727+0000 7fb319e00700 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556841b0bc50 0x556841b0e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.728+0000 7fb319e00700 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556841b0bc50 0x556841b0e0f0 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d0b09bf6b54963 server_cookie=8b0bb00bac791ffe in_seq=0 out_seq=0 2024-05-27T07:59:08.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.728+0000 7fb2fbe00700 10 client.20358 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb2fbe00700 1 -- 172.21.15.63:0/390173039 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556841a21e10 con 0x556841b0bc50 2024-05-27T07:59:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb2fbe00700 10 client.20358 handle_command_reply: tid=0 2024-05-27T07:59:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb32f607b80 1 client.20358 shutdown 2024-05-27T07:59:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb32f607b80 1 -- 172.21.15.63:0/390173039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841a5ea70 msgr2=0x556841af97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb32f607b80 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841a5ea70 0x556841af97e0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fb3080e0dd0 tx=0x7fb308009ca0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb32f607b80 1 -- 172.21.15.63:0/390173039 shutdown_connections 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb32f607b80 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556841b0bc50 0x556841b0e0f0 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb32f607b80 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841af9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb32f607b80 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841972ec0 0x556841af92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb32f607b80 1 --2- 172.21.15.63:0/390173039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841a5ea70 0x556841af97e0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb2fbe00700 0 client.20358 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb32f607b80 1 -- 172.21.15.63:0/390173039 >> 172.21.15.63:0/390173039 conn(0x556841a0c410 msgr2=0x556841aed380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.730+0000 7fb32f607b80 1 -- 172.21.15.63:0/390173039 shutdown_connections 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.731+0000 7fb32f607b80 1 -- 172.21.15.63:0/390173039 wait complete. 2024-05-27T07:59:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.731+0000 7fb32f607b80 20 client.20358 trim_cache size 0 max 16384 2024-05-27T07:59:08.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.756+0000 7fb32f607b80 1 Processor -- start 2024-05-27T07:59:08.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.756+0000 7fb32f607b80 1 -- start start 2024-05-27T07:59:08.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.757+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.757+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841a5ea70 0x556841b07f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.757+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841b138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.757+0000 7fb32f607b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1b4e80 con 0x556841972ec0 2024-05-27T07:59:08.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.757+0000 7fb32f607b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1b2710 con 0x556841a5ea70 2024-05-27T07:59:08.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.757+0000 7fb32f607b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1b05a0 con 0x556841afc340 2024-05-27T07:59:08.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.757+0000 7fb319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.757+0000 7fb319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52638/0 (socket says 172.21.15.63:52638) 2024-05-27T07:59:08.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.757+0000 7fb319e00700 1 -- 172.21.15.63:0/621389524 learned_addr learned my addr 172.21.15.63:0/621389524 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:08.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.758+0000 7fb319400700 1 --2- 172.21.15.63:0/621389524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841a5ea70 0x556841b07f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.758+0000 7fb31a800700 1 --2- 172.21.15.63:0/621389524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841b138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.758+0000 7fb319e00700 1 -- 172.21.15.63:0/621389524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 msgr2=0x556841b138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:08.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.758+0000 7fb319e00700 1 --2- 172.21.15.63:0/621389524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841b138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.758+0000 7fb319e00700 1 -- 172.21.15.63:0/621389524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841a5ea70 msgr2=0x556841b07f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:08.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.758+0000 7fb319e00700 1 --2- 172.21.15.63:0/621389524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841a5ea70 0x556841b07f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.758+0000 7fb319e00700 1 -- 172.21.15.63:0/621389524 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556841af7810 con 0x556841972ec0 2024-05-27T07:59:08.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.758+0000 7fb319e00700 1 --2- 172.21.15.63:0/621389524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af1bd0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00bc50 tx=0x7fb30c00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a4a53910244033 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:08.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.759+0000 7fb306a00700 1 -- 172.21.15.63:0/621389524 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c023040 con 0x556841972ec0 2024-05-27T07:59:08.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.759+0000 7fb306a00700 1 -- 172.21.15.63:0/621389524 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb30c003680 con 0x556841972ec0 2024-05-27T07:59:08.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.759+0000 7fb306a00700 1 -- 172.21.15.63:0/621389524 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c028e00 con 0x556841972ec0 2024-05-27T07:59:08.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.760+0000 7fb32f607b80 1 -- 172.21.15.63:0/621389524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 msgr2=0x556841af1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:08.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.760+0000 7fb32f607b80 1 --2- 172.21.15.63:0/621389524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af1bd0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00bc50 tx=0x7fb30c00bec0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.760+0000 7fb32f607b80 1 -- 172.21.15.63:0/621389524 shutdown_connections 2024-05-27T07:59:08.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.760+0000 7fb32f607b80 1 --2- 172.21.15.63:0/621389524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841afc340 0x556841b138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.760+0000 7fb32f607b80 1 --2- 172.21.15.63:0/621389524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841af1bd0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.760+0000 7fb32f607b80 1 --2- 172.21.15.63:0/621389524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841a5ea70 0x556841b07f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.760+0000 7fb32f607b80 1 -- 172.21.15.63:0/621389524 >> 172.21.15.63:0/621389524 conn(0x556841a0c410 msgr2=0x556841af4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:08.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.760+0000 7fb32f607b80 1 -- 172.21.15.63:0/621389524 shutdown_connections 2024-05-27T07:59:08.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.760+0000 7fb32f607b80 1 -- 172.21.15.63:0/621389524 wait complete. 2024-05-27T07:59:08.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.761+0000 7fb32f607b80 1 -- start start 2024-05-27T07:59:08.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.761+0000 7fb32f607b80 1 Processor -- start 2024-05-27T07:59:08.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841b6eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841a5ea70 0x556841b6eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb32f607b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841afc340 0x556841b00810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb32f607b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1b0bf0 con 0x556841972ec0 2024-05-27T07:59:08.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb32f607b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1b2040 con 0x556841afc340 2024-05-27T07:59:08.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb32f607b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c1b12a0 con 0x556841a5ea70 2024-05-27T07:59:08.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb32f607b80 10 client.0 _ll_register_callbacks cb 0x5568419b9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:08.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841b6eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841b6eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52640/0 (socket says 172.21.15.63:52640) 2024-05-27T07:59:08.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb319e00700 1 -- 172.21.15.63:0/4202639242 learned_addr learned my addr 172.21.15.63:0/4202639242 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:08.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb319400700 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841a5ea70 0x556841b6eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.762+0000 7fb31a800700 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841afc340 0x556841b00810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.763+0000 7fb319e00700 1 -- 172.21.15.63:0/4202639242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841a5ea70 msgr2=0x556841b6eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:08.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.763+0000 7fb319e00700 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841a5ea70 0x556841b6eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.763+0000 7fb319e00700 1 -- 172.21.15.63:0/4202639242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841afc340 msgr2=0x556841b00810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:08.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.763+0000 7fb319e00700 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841afc340 0x556841b00810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.763+0000 7fb319e00700 1 -- 172.21.15.63:0/4202639242 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556841b00e80 con 0x556841972ec0 2024-05-27T07:59:08.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.763+0000 7fb31a800700 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841afc340 0x556841b00810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:08.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.763+0000 7fb319e00700 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841b6eab0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fb30c010a00 tx=0x7fb30c010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6251f437dbaaaba3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:08.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.763+0000 7fb2fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:08.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.763+0000 7fb2fbe00700 1 -- 172.21.15.63:0/4202639242 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c023040 con 0x556841972ec0 2024-05-27T07:59:08.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.764+0000 7fb2fbe00700 1 -- 172.21.15.63:0/4202639242 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9571+0+0 (secure 0 0 0) 0x7fb30c003c70 con 0x556841972ec0 2024-05-27T07:59:08.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.764+0000 7fb32f607b80 1 -- 172.21.15.63:0/4202639242 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb30c003680 con 0x556841972ec0 2024-05-27T07:59:08.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.764+0000 7fb2fbe00700 1 -- 172.21.15.63:0/4202639242 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb30c03a050 con 0x556841972ec0 2024-05-27T07:59:08.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.764+0000 7fb32f607b80 10 client.20364 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:08.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.764+0000 7fb32f607b80 1 -- 172.21.15.63:0/4202639242 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556841b772d0 con 0x556841972ec0 2024-05-27T07:59:08.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.764+0000 7fb2fbe00700 1 -- 172.21.15.63:0/4202639242 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb30c003680 con 0x556841972ec0 2024-05-27T07:59:08.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.764+0000 7fb2fbe00700 1 -- 172.21.15.63:0/4202639242 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb30c026020 con 0x556841972ec0 2024-05-27T07:59:08.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.765+0000 7fb32f607b80 10 client.20364 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:08.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.765+0000 7fb32f607b80 10 client.20364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:08.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.765+0000 7fb32f607b80 20 client.20364 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:08.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.765+0000 7fb32f607b80 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556841b77730 0x556841b79b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:08.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.765+0000 7fb32f607b80 4 client.20364 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:59:08.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.765+0000 7fb32f607b80 1 -- 172.21.15.63:0/4202639242 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556841a1b5f0 con 0x556841b77730 2024-05-27T07:59:08.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.765+0000 7fb319400700 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556841b77730 0x556841b79b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:08.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.766+0000 7fb319400700 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556841b77730 0x556841b79b50 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98113d804776f02c server_cookie=1012dabe4b6308b5 in_seq=0 out_seq=0 2024-05-27T07:59:08.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.766+0000 7fb2fbe00700 10 client.20364 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:08.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb2fbe00700 1 -- 172.21.15.63:0/4202639242 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556841a1b5f0 con 0x556841b77730 2024-05-27T07:59:08.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb2fbe00700 10 client.20364 handle_command_reply: tid=0 2024-05-27T07:59:08.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb32f607b80 1 client.20364 shutdown 2024-05-27T07:59:08.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb32f607b80 1 -- 172.21.15.63:0/4202639242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 msgr2=0x556841b6eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:08.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb32f607b80 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841b6eab0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fb30c010a00 tx=0x7fb30c010d60 comp rx=0 tx=0).stop 2024-05-27T07:59:08.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb32f607b80 1 -- 172.21.15.63:0/4202639242 shutdown_connections 2024-05-27T07:59:08.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb32f607b80 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556841b77730 0x556841b79b50 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb2fbe00700 0 client.20364 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:08.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb32f607b80 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556841afc340 0x556841b00810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb32f607b80 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556841972ec0 0x556841b6eab0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb32f607b80 1 --2- 172.21.15.63:0/4202639242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556841a5ea70 0x556841b6eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.769+0000 7fb32f607b80 1 -- 172.21.15.63:0/4202639242 >> 172.21.15.63:0/4202639242 conn(0x556841a0c410 msgr2=0x556841af4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:08.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.770+0000 7fb32f607b80 1 -- 172.21.15.63:0/4202639242 shutdown_connections 2024-05-27T07:59:08.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.770+0000 7fb32f607b80 1 -- 172.21.15.63:0/4202639242 wait complete. 2024-05-27T07:59:08.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.770+0000 7fb32f607b80 20 client.20364 trim_cache size 0 max 16384 2024-05-27T07:59:08.772 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:59:08.772 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T07:59:08.772 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "bf08952d-8017-4796-bd04-fd42b8d03d84", 2024-05-27T07:59:08.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T07:59:08.772 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:59:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.771+0000 7fb31b200700 1 -- 172.21.15.63:0/3538851277 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3000b2a70 msgr2=0x7fb3000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.771+0000 7fb31b200700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3000b2a70 0x7fb3000b4f80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x556841aeeb10 tx=0x7fb314009100 comp rx=0 tx=0).stop 2024-05-27T07:59:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.771+0000 7fb31b200700 1 -- 172.21.15.63:0/3538851277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb31c105c50 msgr2=0x7fb31c1a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.771+0000 7fb31b200700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb31c105c50 0x7fb31c1a0d20 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fb3080057c0 tx=0x7fb3080057f0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.771+0000 7fb31b200700 1 -- 172.21.15.63:0/3538851277 shutdown_connections 2024-05-27T07:59:08.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.771+0000 7fb31b200700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb3000b2a70 0x7fb3000b4f80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.771+0000 7fb31b200700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb31c10e730 0x7fb31c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.771+0000 7fb31b200700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb31c105300 0x7fb31c1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.771+0000 7fb31b200700 1 --2- 172.21.15.63:0/3538851277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb31c105c50 0x7fb31c1a0d20 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:08.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.771+0000 7fb31b200700 1 -- 172.21.15.63:0/3538851277 >> 172.21.15.63:0/3538851277 conn(0x7fb31c078ab0 msgr2=0x7fb31c101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:08.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.772+0000 7fb31b200700 1 -- 172.21.15.63:0/3538851277 shutdown_connections 2024-05-27T07:59:08.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:08.772+0000 7fb31b200700 1 -- 172.21.15.63:0/3538851277 wait complete. 2024-05-27T07:59:09.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: pgmap v810: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.9 MiB/s wr, 35 op/s 2024-05-27T07:59:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:59:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:59:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:59:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:59:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:59:09.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.664 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:59:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: pgmap v810: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.9 MiB/s wr, 35 op/s 2024-05-27T07:59:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:59:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:59:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:59:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:59:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:59:09.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: pgmap v810: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.9 MiB/s wr, 35 op/s 2024-05-27T07:59:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:59:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:59:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:59:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:59:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:59:09.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:10.152 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:10.573 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:10 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T07:59:10.573 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:10.573 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:10 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:59:10.573 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:10 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T07:59:10.573 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:10.573 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:10.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:10 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T07:59:10.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:10.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:10 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:59:10.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:10 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T07:59:10.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:10.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:10.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:10 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T07:59:10.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:10.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:10 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:59:10.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:10 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T07:59:10.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:10.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T07:59:11.623 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:11 smithi063 ceph-mon[31613]: pgmap v811: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.9 MiB/s wr, 35 op/s 2024-05-27T07:59:11.623 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:11 smithi063 ceph-mon[31613]: osdmap e134: 12 total, 12 up, 12 in 2024-05-27T07:59:11.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:11 smithi086 ceph-mon[34687]: pgmap v811: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.9 MiB/s wr, 35 op/s 2024-05-27T07:59:11.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:11 smithi086 ceph-mon[34687]: osdmap e134: 12 total, 12 up, 12 in 2024-05-27T07:59:11.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:11 smithi149 ceph-mon[35408]: pgmap v811: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.9 MiB/s wr, 35 op/s 2024-05-27T07:59:11.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:11 smithi149 ceph-mon[35408]: osdmap e134: 12 total, 12 up, 12 in 2024-05-27T07:59:12.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.641+0000 7f3bd9200700 1 -- 172.21.15.63:0/3492457120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bd4108f10 msgr2=0x7f3bd4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:12.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.641+0000 7f3bd9200700 1 --2- 172.21.15.63:0/3492457120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bd4108f10 0x7f3bd4109320 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00099c0 tx=0x7f3bc0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.641+0000 7f3bd9200700 1 -- 172.21.15.63:0/3492457120 shutdown_connections 2024-05-27T07:59:12.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.641+0000 7f3bd9200700 1 --2- 172.21.15.63:0/3492457120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd4109860 0x7f3bd41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.641+0000 7f3bd9200700 1 --2- 172.21.15.63:0/3492457120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bd41045e0 0x7f3bd41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.641+0000 7f3bd9200700 1 --2- 172.21.15.63:0/3492457120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bd4108f10 0x7f3bd4109320 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.641+0000 7f3bd9200700 1 -- 172.21.15.63:0/3492457120 >> 172.21.15.63:0/3492457120 conn(0x7f3bd4100030 msgr2=0x7f3bd4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:12.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.642+0000 7f3bd9200700 1 -- 172.21.15.63:0/3492457120 shutdown_connections 2024-05-27T07:59:12.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.642+0000 7f3bd9200700 1 -- 172.21.15.63:0/3492457120 wait complete. 2024-05-27T07:59:12.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.643+0000 7f3bd9200700 1 Processor -- start 2024-05-27T07:59:12.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.643+0000 7f3bd9200700 1 -- start start 2024-05-27T07:59:12.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd9200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bd41045e0 0x7f3bd419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd9200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bd4108f10 0x7f3bd419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd4109860 0x7f3bd41a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd9200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd4114370 con 0x7f3bd4108f10 2024-05-27T07:59:12.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd9200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd41141f0 con 0x7f3bd4109860 2024-05-27T07:59:12.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd9200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd41144f0 con 0x7f3bd41045e0 2024-05-27T07:59:12.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bd4108f10 0x7f3bd419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bd4108f10 0x7f3bd419d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54364/0 (socket says 172.21.15.63:54364) 2024-05-27T07:59:12.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd1400700 1 -- 172.21.15.63:0/2738446233 learned_addr learned my addr 172.21.15.63:0/2738446233 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:12.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bd41045e0 0x7f3bd419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.644+0000 7f3bd2800700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd4109860 0x7f3bd41a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.645+0000 7f3bd1400700 1 -- 172.21.15.63:0/2738446233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bd41045e0 msgr2=0x7f3bd419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:12.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.645+0000 7f3bd1400700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bd41045e0 0x7f3bd419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.645+0000 7f3bd1400700 1 -- 172.21.15.63:0/2738446233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd4109860 msgr2=0x7f3bd41a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:12.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.645+0000 7f3bd1400700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd4109860 0x7f3bd41a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.645+0000 7f3bd1400700 1 -- 172.21.15.63:0/2738446233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd41a1e90 con 0x7f3bd4108f10 2024-05-27T07:59:12.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.646+0000 7f3bd1400700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bd4108f10 0x7f3bd419d560 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00098e0 tx=0x7f3bc00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:12.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.646+0000 7f3bcaa00700 1 -- 172.21.15.63:0/2738446233 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0027070 con 0x7f3bd4108f10 2024-05-27T07:59:12.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.646+0000 7f3bcaa00700 1 -- 172.21.15.63:0/2738446233 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bc000fd30 con 0x7f3bd4108f10 2024-05-27T07:59:12.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.646+0000 7f3bd9200700 1 -- 172.21.15.63:0/2738446233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd41a2120 con 0x7f3bd4108f10 2024-05-27T07:59:12.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.647+0000 7f3bd9200700 1 -- 172.21.15.63:0/2738446233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd41a2600 con 0x7f3bd4108f10 2024-05-27T07:59:12.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.647+0000 7f3bcaa00700 1 -- 172.21.15.63:0/2738446233 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0022dc0 con 0x7f3bd4108f10 2024-05-27T07:59:12.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.651+0000 7f3be7dd9b80 1 Processor -- start 2024-05-27T07:59:12.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.652+0000 7f3be7dd9b80 1 -- start start 2024-05-27T07:59:12.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.652+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6eb79f0 0x5557a6f4a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.652+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 0x5557a6f4a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.652+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6f4ae70 0x5557a6f57110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.653+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5557a6e78e40 con 0x5557a6f4ae70 2024-05-27T07:59:12.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.653+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557a6e689e0 con 0x5557a6eb79f0 2024-05-27T07:59:12.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.653+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5557a6e6ca20 con 0x5557a6dcbe40 2024-05-27T07:59:12.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.653+0000 7f3bd2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6eb79f0 0x5557a6f4a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.653+0000 7f3bd2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6eb79f0 0x5557a6f4a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:52402/0 (socket says 172.21.15.63:52402) 2024-05-27T07:59:12.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.653+0000 7f3bd2800700 1 -- 172.21.15.63:0/460453225 learned_addr learned my addr 172.21.15.63:0/460453225 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:12.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.653+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/460453225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 0x5557a6f4a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.653+0000 7f3bcaa00700 1 -- 172.21.15.63:0/2738446233 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3bc0022590 con 0x7f3bd4108f10 2024-05-27T07:59:12.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.654+0000 7f3bd1400700 1 --2- 172.21.15.63:0/460453225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6f4ae70 0x5557a6f57110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.654+0000 7f3bd1e00700 1 -- 172.21.15.63:0/460453225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6eb79f0 msgr2=0x5557a6f4a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:12.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.654+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/460453225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6eb79f0 0x5557a6f4a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.654+0000 7f3bd1e00700 1 -- 172.21.15.63:0/460453225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6f4ae70 msgr2=0x5557a6f57110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:12.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.654+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/460453225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6f4ae70 0x5557a6f57110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.654+0000 7f3bd1e00700 1 -- 172.21.15.63:0/460453225 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a6e31420 con 0x5557a6dcbe40 2024-05-27T07:59:12.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.654+0000 7f3bd1400700 1 --2- 172.21.15.63:0/460453225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6f4ae70 0x5557a6f57110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:12.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.655+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/460453225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 0x5557a6f4a870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400bc70 tx=0x7f3bc400bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d661c49400036be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:12.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.656+0000 7f3bcaa00700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bb80b2a70 0x7f3bb80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.656+0000 7f3bbf400700 1 -- 172.21.15.63:0/460453225 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4015070 con 0x5557a6dcbe40 2024-05-27T07:59:12.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.656+0000 7f3bbf400700 1 -- 172.21.15.63:0/460453225 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bc400fc20 con 0x5557a6dcbe40 2024-05-27T07:59:12.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.656+0000 7f3bbf400700 1 -- 172.21.15.63:0/460453225 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40116a0 con 0x5557a6dcbe40 2024-05-27T07:59:12.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.656+0000 7f3bcaa00700 1 -- 172.21.15.63:0/2738446233 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f3bc00e39c0 con 0x7f3bd4108f10 2024-05-27T07:59:12.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.656+0000 7f3bd2800700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bb80b2a70 0x7f3bb80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.657+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/460453225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 msgr2=0x5557a6f4a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:12.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.657+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/460453225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 0x5557a6f4a870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400bc70 tx=0x7f3bc400bf80 comp rx=0 tx=0).stop 2024-05-27T07:59:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.657+0000 7f3bd2800700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bb80b2a70 0x7f3bb80b4f80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x5557a6f4cff0 tx=0x7f3bcc009100 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.657+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/460453225 shutdown_connections 2024-05-27T07:59:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.657+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/460453225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6f4ae70 0x5557a6f57110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.657+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/460453225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6eb79f0 0x5557a6f4a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.657+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/460453225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 0x5557a6f4a870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.657+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/460453225 >> 172.21.15.63:0/460453225 conn(0x5557a6e65390 msgr2=0x5557a6f464c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.657+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/460453225 shutdown_connections 2024-05-27T07:59:12.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.658+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/460453225 wait complete. 2024-05-27T07:59:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.658+0000 7f3be7dd9b80 1 -- start start 2024-05-27T07:59:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.659+0000 7f3be7dd9b80 1 Processor -- start 2024-05-27T07:59:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.659+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 0x5557a6f54620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.659+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6eb79f0 0x5557a6f54b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.659+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f4ae70 0x5557a6f550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.659+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5557a6d6efa0 con 0x5557a6eb79f0 2024-05-27T07:59:12.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.659+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557a6e09760 con 0x5557a6f4ae70 2024-05-27T07:59:12.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.659+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5557a6f57b80 con 0x5557a6dcbe40 2024-05-27T07:59:12.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.659+0000 7f3be7dd9b80 10 client.0 _ll_register_callbacks cb 0x5557a6e12c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:12.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.660+0000 7f3bd1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6eb79f0 0x5557a6f54b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.660+0000 7f3bd1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6eb79f0 0x5557a6f54b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54382/0 (socket says 172.21.15.63:54382) 2024-05-27T07:59:12.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.660+0000 7f3bd1400700 1 -- 172.21.15.63:0/1307813124 learned_addr learned my addr 172.21.15.63:0/1307813124 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:12.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.660+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 0x5557a6f54620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.660+0000 7f3bd2800700 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f4ae70 0x5557a6f550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.661+0000 7f3bd1400700 1 -- 172.21.15.63:0/1307813124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 msgr2=0x5557a6f54620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:12.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.661+0000 7f3bd1400700 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 0x5557a6f54620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.661+0000 7f3bd1400700 1 -- 172.21.15.63:0/1307813124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f4ae70 msgr2=0x5557a6f550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:12.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.661+0000 7f3bd1400700 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f4ae70 0x5557a6f550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.661+0000 7f3bd1400700 1 -- 172.21.15.63:0/1307813124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a6db88d0 con 0x5557a6eb79f0 2024-05-27T07:59:12.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.661+0000 7f3bd1400700 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6eb79f0 0x5557a6f54b60 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00a7b80 tx=0x7f3bc00aae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20367e93b286b57a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:12.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.661+0000 7f3bb3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:12.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.662+0000 7f3bb3e00700 1 -- 172.21.15.63:0/1307813124 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00a67d0 con 0x5557a6eb79f0 2024-05-27T07:59:12.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.662+0000 7f3bb3e00700 1 -- 172.21.15.63:0/1307813124 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f3bc00b10b0 con 0x5557a6eb79f0 2024-05-27T07:59:12.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.662+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/1307813124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bc400fc20 con 0x5557a6eb79f0 2024-05-27T07:59:12.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.662+0000 7f3bb3e00700 1 -- 172.21.15.63:0/1307813124 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3bc00a60a0 con 0x5557a6eb79f0 2024-05-27T07:59:12.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.662+0000 7f3be7dd9b80 10 client.20376 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:12.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.662+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/1307813124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5557a6e39060 con 0x5557a6eb79f0 2024-05-27T07:59:12.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.663+0000 7f3bb3e00700 1 -- 172.21.15.63:0/1307813124 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bc400fc20 con 0x5557a6eb79f0 2024-05-27T07:59:12.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.663+0000 7f3bb3e00700 1 -- 172.21.15.63:0/1307813124 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3bc00cbb30 con 0x5557a6eb79f0 2024-05-27T07:59:12.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.663+0000 7f3be7dd9b80 10 client.20376 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:12.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.663+0000 7f3be7dd9b80 10 client.20376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:12.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.663+0000 7f3be7dd9b80 20 client.20376 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:12.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.664+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557a6f64a80 0x5557a6f66ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.664+0000 7f3be7dd9b80 4 client.20376 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:12.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.664+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/1307813124 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557a6e7ad90 con 0x5557a6f64a80 2024-05-27T07:59:12.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.664+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557a6f64a80 0x5557a6f66ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.665+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557a6f64a80 0x5557a6f66ed0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a424722518f4a341 server_cookie=85bb18116567fca1 in_seq=0 out_seq=0 2024-05-27T07:59:12.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.665+0000 7f3bb3e00700 10 client.20376 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:12.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.667+0000 7f3bb3e00700 1 -- 172.21.15.63:0/1307813124 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5557a6e7ad90 con 0x5557a6f64a80 2024-05-27T07:59:12.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.667+0000 7f3bb3e00700 10 client.20376 handle_command_reply: tid=0 2024-05-27T07:59:12.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.667+0000 7f3be7dd9b80 1 client.20376 shutdown 2024-05-27T07:59:12.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/1307813124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6eb79f0 msgr2=0x5557a6f54b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:12.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6eb79f0 0x5557a6f54b60 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00a7b80 tx=0x7f3bc00aae50 comp rx=0 tx=0).stop 2024-05-27T07:59:12.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/1307813124 shutdown_connections 2024-05-27T07:59:12.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557a6f64a80 0x5557a6f66ed0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f4ae70 0x5557a6f550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6dcbe40 0x5557a6f54620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/1307813124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6eb79f0 0x5557a6f54b60 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3bb3e00700 0 client.20376 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:12.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/1307813124 >> 172.21.15.63:0/1307813124 conn(0x5557a6e65390 msgr2=0x5557a6f4a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:12.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/1307813124 shutdown_connections 2024-05-27T07:59:12.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.668+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/1307813124 wait complete. 2024-05-27T07:59:12.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.669+0000 7f3be7dd9b80 20 client.20376 trim_cache size 0 max 16384 2024-05-27T07:59:12.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.697+0000 7f3be7dd9b80 1 Processor -- start 2024-05-27T07:59:12.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.697+0000 7f3be7dd9b80 1 -- start start 2024-05-27T07:59:12.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.697+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f4fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.697+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6f4ae70 0x5557a6f60c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.697+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f563e0 0x5557a6f6c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.697+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd41b0b90 con 0x5557a6dcbe40 2024-05-27T07:59:12.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.697+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd41ae420 con 0x5557a6f563e0 2024-05-27T07:59:12.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.697+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd41ac290 con 0x5557a6f4ae70 2024-05-27T07:59:12.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.698+0000 7f3bd1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f4fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.698+0000 7f3bd1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f4fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54384/0 (socket says 172.21.15.63:54384) 2024-05-27T07:59:12.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.698+0000 7f3bd1e00700 1 -- 172.21.15.63:0/849304114 learned_addr learned my addr 172.21.15.63:0/849304114 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:12.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.698+0000 7f3bd1400700 1 --2- 172.21.15.63:0/849304114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6f4ae70 0x5557a6f60c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.698+0000 7f3bd2800700 1 --2- 172.21.15.63:0/849304114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f563e0 0x5557a6f6c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.698+0000 7f3bd1e00700 1 -- 172.21.15.63:0/849304114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6f4ae70 msgr2=0x5557a6f60c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:12.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.698+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/849304114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6f4ae70 0x5557a6f60c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.699+0000 7f3bd1e00700 1 -- 172.21.15.63:0/849304114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f563e0 msgr2=0x5557a6f6c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:12.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.699+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/849304114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f563e0 0x5557a6f6c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.699+0000 7f3bd1e00700 1 -- 172.21.15.63:0/849304114 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a6f52b90 con 0x5557a6dcbe40 2024-05-27T07:59:12.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.699+0000 7f3bd2800700 1 --2- 172.21.15.63:0/849304114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f563e0 0x5557a6f6c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:12.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.699+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/849304114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f4fe90 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400bc70 tx=0x7f3bc400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f847e85f90b5daea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:12.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.700+0000 7f3bbea00700 1 -- 172.21.15.63:0/849304114 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4015040 con 0x5557a6dcbe40 2024-05-27T07:59:12.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.700+0000 7f3bbea00700 1 -- 172.21.15.63:0/849304114 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bc40050e0 con 0x5557a6dcbe40 2024-05-27T07:59:12.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.700+0000 7f3bbea00700 1 -- 172.21.15.63:0/849304114 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc401a6c0 con 0x5557a6dcbe40 2024-05-27T07:59:12.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.701+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/849304114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 msgr2=0x5557a6f4fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:12.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.701+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/849304114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f4fe90 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400bc70 tx=0x7f3bc400fc60 comp rx=0 tx=0).stop 2024-05-27T07:59:12.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.701+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/849304114 shutdown_connections 2024-05-27T07:59:12.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.701+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/849304114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f563e0 0x5557a6f6c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.701+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/849304114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f4fe90 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.701+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/849304114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6f4ae70 0x5557a6f60c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.701+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/849304114 >> 172.21.15.63:0/849304114 conn(0x5557a6e65390 msgr2=0x5557a6f55cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:12.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.701+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/849304114 shutdown_connections 2024-05-27T07:59:12.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.701+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/849304114 wait complete. 2024-05-27T07:59:12.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.702+0000 7f3be7dd9b80 1 -- start start 2024-05-27T07:59:12.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.702+0000 7f3be7dd9b80 1 Processor -- start 2024-05-27T07:59:12.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.703+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f5d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.703+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f4ae70 0x5557a6f57e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.703+0000 7f3be7dd9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6f563e0 0x5557a6f58370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.703+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd41ac8e0 con 0x5557a6dcbe40 2024-05-27T07:59:12.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.703+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd41add50 con 0x5557a6f4ae70 2024-05-27T07:59:12.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.703+0000 7f3be7dd9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd41acfb0 con 0x5557a6f563e0 2024-05-27T07:59:12.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.703+0000 7f3be7dd9b80 10 client.0 _ll_register_callbacks cb 0x5557a6e12c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:12.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.703+0000 7f3bd1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f5d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.704+0000 7f3bd1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f5d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54396/0 (socket says 172.21.15.63:54396) 2024-05-27T07:59:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.704+0000 7f3bd1e00700 1 -- 172.21.15.63:0/2982594268 learned_addr learned my addr 172.21.15.63:0/2982594268 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.704+0000 7f3bd1400700 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f4ae70 0x5557a6f57e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.704+0000 7f3bd2800700 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6f563e0 0x5557a6f58370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.704+0000 7f3bd1e00700 1 -- 172.21.15.63:0/2982594268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6f563e0 msgr2=0x5557a6f58370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.704+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6f563e0 0x5557a6f58370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.704+0000 7f3bd1e00700 1 -- 172.21.15.63:0/2982594268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f4ae70 msgr2=0x5557a6f57e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.704+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f4ae70 0x5557a6f57e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.704+0000 7f3bd1e00700 1 -- 172.21.15.63:0/2982594268 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a6f588b0 con 0x5557a6dcbe40 2024-05-27T07:59:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.705+0000 7f3bd1e00700 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f5d280 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b5c0 tx=0x7f3bc40054c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890778e56f08baf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:12.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.705+0000 7f3bb3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:12.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.705+0000 7f3bb3e00700 1 -- 172.21.15.63:0/2982594268 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4015070 con 0x5557a6dcbe40 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.705+0000 7f3bb3e00700 1 -- 172.21.15.63:0/2982594268 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f3bc4004450 con 0x5557a6dcbe40 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.705+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/2982594268 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bc40050e0 con 0x5557a6dcbe40 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.705+0000 7f3bb3e00700 1 -- 172.21.15.63:0/2982594268 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3bc4042050 con 0x5557a6dcbe40 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.705+0000 7f3be7dd9b80 10 client.20382 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.705+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/2982594268 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5557a6fd00e0 con 0x5557a6dcbe40 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.706+0000 7f3bb3e00700 1 -- 172.21.15.63:0/2982594268 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bc40050e0 con 0x5557a6dcbe40 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.706+0000 7f3bb3e00700 1 -- 172.21.15.63:0/2982594268 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3bc402f070 con 0x5557a6dcbe40 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.706+0000 7f3be7dd9b80 10 client.20382 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.706+0000 7f3be7dd9b80 10 client.20382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.706+0000 7f3be7dd9b80 20 client.20382 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.706+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557a6fd0530 0x5557a6fd2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:12.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.706+0000 7f3be7dd9b80 4 client.20382 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:59:12.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.706+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/2982594268 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557a6e74570 con 0x5557a6fd0530 2024-05-27T07:59:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.707+0000 7f3bd1400700 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557a6fd0530 0x5557a6fd2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.708+0000 7f3bd1400700 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557a6fd0530 0x5557a6fd2950 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f57b4bfd0d9d77c server_cookie=1b78e7824f6b110e in_seq=0 out_seq=0 2024-05-27T07:59:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.708+0000 7f3bb3e00700 10 client.20382 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.708+0000 7f3bb3e00700 1 -- 172.21.15.63:0/2982594268 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5557a6e74570 con 0x5557a6fd0530 2024-05-27T07:59:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.708+0000 7f3bb3e00700 10 client.20382 handle_command_reply: tid=0 2024-05-27T07:59:12.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.708+0000 7f3be7dd9b80 1 client.20382 shutdown 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/2982594268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 msgr2=0x5557a6f5d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f5d280 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b5c0 tx=0x7f3bc40054c0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/2982594268 shutdown_connections 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557a6fd0530 0x5557a6fd2950 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557a6f563e0 0x5557a6f58370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557a6dcbe40 0x5557a6f5d280 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3be7dd9b80 1 --2- 172.21.15.63:0/2982594268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557a6f4ae70 0x5557a6f57e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/2982594268 >> 172.21.15.63:0/2982594268 conn(0x5557a6e65390 msgr2=0x5557a6f4bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3bb3e00700 0 client.20382 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/2982594268 shutdown_connections 2024-05-27T07:59:12.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.709+0000 7f3be7dd9b80 1 -- 172.21.15.63:0/2982594268 wait complete. 2024-05-27T07:59:12.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.710+0000 7f3be7dd9b80 20 client.20382 trim_cache size 0 max 16384 2024-05-27T07:59:12.712 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:59:12.712 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:59:12.712 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T07:59:12.712 INFO:teuthology.orchestra.run.smithi063.stdout: "bf08952d-8017-4796-bd04-fd42b8d03d84": { 2024-05-27T07:59:12.712 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T07:59:12.713 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "bf08952d-8017-4796-bd04-fd42b8d03d84", 2024-05-27T07:59:12.713 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T07:59:12.713 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:59:12.713 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T07:59:12.713 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:59:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.710+0000 7f3bd9200700 1 -- 172.21.15.63:0/2738446233 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bb80b2a70 msgr2=0x7f3bb80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.710+0000 7f3bd9200700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bb80b2a70 0x7f3bb80b4f80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x5557a6f4cff0 tx=0x7f3bcc009100 comp rx=0 tx=0).stop 2024-05-27T07:59:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.710+0000 7f3bd9200700 1 -- 172.21.15.63:0/2738446233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bd4108f10 msgr2=0x7f3bd419d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.710+0000 7f3bd9200700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bd4108f10 0x7f3bd419d560 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00098e0 tx=0x7f3bc00047b0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.711+0000 7f3bd9200700 1 -- 172.21.15.63:0/2738446233 shutdown_connections 2024-05-27T07:59:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.711+0000 7f3bd9200700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bb80b2a70 0x7f3bb80b4f80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.711+0000 7f3bd9200700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd4109860 0x7f3bd41a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.711+0000 7f3bd9200700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bd41045e0 0x7f3bd419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.711+0000 7f3bd9200700 1 --2- 172.21.15.63:0/2738446233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bd4108f10 0x7f3bd419d560 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.711+0000 7f3bd9200700 1 -- 172.21.15.63:0/2738446233 >> 172.21.15.63:0/2738446233 conn(0x7f3bd4100030 msgr2=0x7f3bd410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.711+0000 7f3bd9200700 1 -- 172.21.15.63:0/2738446233 shutdown_connections 2024-05-27T07:59:12.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:12.711+0000 7f3bd9200700 1 -- 172.21.15.63:0/2738446233 wait complete. 2024-05-27T07:59:13.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:13 smithi063 ceph-mon[31613]: pgmap v813: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.8 MiB/s wr, 51 op/s 2024-05-27T07:59:13.582 INFO:tasks.cephfs.filesystem:scrub status for tag:bf08952d-8017-4796-bd04-fd42b8d03d84 - {'path': '/', 'tag': 'bf08952d-8017-4796-bd04-fd42b8d03d84', 'options': 'recursive,force'} 2024-05-27T07:59:13.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:13 smithi086 ceph-mon[34687]: pgmap v813: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.8 MiB/s wr, 51 op/s 2024-05-27T07:59:13.733 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:13 smithi149 ceph-mon[35408]: pgmap v813: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.8 MiB/s wr, 51 op/s 2024-05-27T07:59:15.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:15 smithi086 ceph-mon[34687]: pgmap v814: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.6 MiB/s wr, 50 op/s 2024-05-27T07:59:15.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:15 smithi149 ceph-mon[35408]: pgmap v814: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.6 MiB/s wr, 50 op/s 2024-05-27T07:59:15.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:15 smithi063 ceph-mon[31613]: pgmap v814: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.6 MiB/s wr, 50 op/s 2024-05-27T07:59:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:17 smithi086 ceph-mon[34687]: pgmap v815: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.6 MiB/s wr, 50 op/s 2024-05-27T07:59:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:59:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:59:17.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T07:59:17.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:17 smithi149 ceph-mon[35408]: pgmap v815: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.6 MiB/s wr, 50 op/s 2024-05-27T07:59:17.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:59:17.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:59:17.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T07:59:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:17 smithi063 ceph-mon[31613]: pgmap v815: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.6 MiB/s wr, 50 op/s 2024-05-27T07:59:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:59:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:59:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T07:59:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.624+0000 7fdb11600640 1 -- 172.21.15.63:0/3082562548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb0c10ea80 msgr2=0x7fdb0c10eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.624+0000 7fdb11600640 1 --2- 172.21.15.63:0/3082562548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb0c10ea80 0x7fdb0c10eeb0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009900 tx=0x7fdaf802a3f0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.625+0000 7fdb11600640 1 -- 172.21.15.63:0/3082562548 shutdown_connections 2024-05-27T07:59:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.625+0000 7fdb11600640 1 --2- 172.21.15.63:0/3082562548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10f540 0x7fdb0c116ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.625+0000 7fdb11600640 1 --2- 172.21.15.63:0/3082562548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb0c10ea80 0x7fdb0c10eeb0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.625+0000 7fdb11600640 1 --2- 172.21.15.63:0/3082562548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c10a500 0x7fdb0c10a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.625+0000 7fdb11600640 1 -- 172.21.15.63:0/3082562548 >> 172.21.15.63:0/3082562548 conn(0x7fdb0c077190 msgr2=0x7fdb0c0775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.625+0000 7fdb11600640 1 -- 172.21.15.63:0/3082562548 shutdown_connections 2024-05-27T07:59:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.625+0000 7fdb11600640 1 -- 172.21.15.63:0/3082562548 wait complete. 2024-05-27T07:59:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.626+0000 7fdb11600640 1 Processor -- start 2024-05-27T07:59:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.626+0000 7fdb11600640 1 -- start start 2024-05-27T07:59:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb11600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb0c10a500 0x7fdb0c1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb11600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c10ea80 0x7fdb0c1a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb11600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10f540 0x7fdb0c1a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb11600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c11a2f0 con 0x7fdb0c10a500 2024-05-27T07:59:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb11600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c11a170 con 0x7fdb0c10ea80 2024-05-27T07:59:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb11600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c11a470 con 0x7fdb0c10f540 2024-05-27T07:59:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb0aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb0c10a500 0x7fdb0c1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb0aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb0c10a500 0x7fdb0c1a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59166/0 (socket says 172.21.15.63:59166) 2024-05-27T07:59:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb0aa00640 1 -- 172.21.15.63:0/2053475486 learned_addr learned my addr 172.21.15.63:0/2053475486 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb0a000640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c10ea80 0x7fdb0c1a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.627+0000 7fdb0b400640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10f540 0x7fdb0c1a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb0aa00640 1 -- 172.21.15.63:0/2053475486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10f540 msgr2=0x7fdb0c1a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb0aa00640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10f540 0x7fdb0c1a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb0aa00640 1 -- 172.21.15.63:0/2053475486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c10ea80 msgr2=0x7fdb0c1a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb0aa00640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c10ea80 0x7fdb0c1a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb0aa00640 1 -- 172.21.15.63:0/2053475486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c1a5b50 con 0x7fdb0c10a500 2024-05-27T07:59:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb0b400640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10f540 0x7fdb0c1a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb0a000640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c10ea80 0x7fdb0c1a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb0aa00640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb0c10a500 0x7fdb0c1a0d70 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00c950 tx=0x7fdafc00ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb03400640 1 -- 172.21.15.63:0/2053475486 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc007c70 con 0x7fdb0c10a500 2024-05-27T07:59:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb03400640 1 -- 172.21.15.63:0/2053475486 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdafc007e10 con 0x7fdb0c10a500 2024-05-27T07:59:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb03400640 1 -- 172.21.15.63:0/2053475486 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc018760 con 0x7fdb0c10a500 2024-05-27T07:59:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.628+0000 7fdb11600640 1 -- 172.21.15.63:0/2053475486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0c1a5e40 con 0x7fdb0c10a500 2024-05-27T07:59:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.629+0000 7fdb11600640 1 -- 172.21.15.63:0/2053475486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0c1a6340 con 0x7fdb0c10a500 2024-05-27T07:59:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.632+0000 7fdb18618740 1 Processor -- start 2024-05-27T07:59:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb18618740 1 -- start start 2024-05-27T07:59:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb18618740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb03400640 1 -- 172.21.15.63:0/2053475486 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdafc00f450 con 0x7fdb0c10a500 2024-05-27T07:59:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb18618740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30e22f0 0x55ead34adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb18618740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30ea4d0 0x55ead34ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb18618740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ead341eb90 con 0x55ead310b1b0 2024-05-27T07:59:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb18618740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ead340e730 con 0x55ead30ea4d0 2024-05-27T07:59:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb18618740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ead3412770 con 0x55ead30e22f0 2024-05-27T07:59:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb0b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb0b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59178/0 (socket says 172.21.15.63:59178) 2024-05-27T07:59:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.633+0000 7fdb0b400640 1 -- 172.21.15.63:0/2104283149 learned_addr learned my addr 172.21.15.63:0/2104283149 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.634+0000 7fdb0aa00640 1 --2- 172.21.15.63:0/2104283149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30e22f0 0x55ead34adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.634+0000 7fdb0a000640 1 --2- 172.21.15.63:0/2104283149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30ea4d0 0x55ead34ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.634+0000 7fdb0b400640 1 -- 172.21.15.63:0/2104283149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30e22f0 msgr2=0x55ead34adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.634+0000 7fdb0b400640 1 --2- 172.21.15.63:0/2104283149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30e22f0 0x55ead34adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.634+0000 7fdb0b400640 1 -- 172.21.15.63:0/2104283149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30ea4d0 msgr2=0x55ead34ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.634+0000 7fdb0b400640 1 --2- 172.21.15.63:0/2104283149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30ea4d0 0x55ead34ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.634+0000 7fdb0b400640 1 -- 172.21.15.63:0/2104283149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ead33a78b0 con 0x55ead310b1b0 2024-05-27T07:59:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.634+0000 7fdb0b400640 1 --2- 172.21.15.63:0/2104283149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34ad860 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009e40 tx=0x7fdb0400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7b506073c96eff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.635+0000 7fdaf7400640 1 -- 172.21.15.63:0/2104283149 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04004d20 con 0x55ead310b1b0 2024-05-27T07:59:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.635+0000 7fdaf7400640 1 -- 172.21.15.63:0/2104283149 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb040054b0 con 0x55ead310b1b0 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.635+0000 7fdaf7400640 1 -- 172.21.15.63:0/2104283149 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04011700 con 0x55ead310b1b0 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.635+0000 7fdb03400640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdaf00b29d0 0x7fdaf00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.635+0000 7fdb03400640 1 -- 172.21.15.63:0/2053475486 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7fdafc0e2b40 con 0x7fdb0c10a500 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.635+0000 7fdb0aa00640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdaf00b29d0 0x7fdaf00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.636+0000 7fdb18618740 1 -- 172.21.15.63:0/2104283149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 msgr2=0x55ead34ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.636+0000 7fdb18618740 1 --2- 172.21.15.63:0/2104283149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34ad860 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fdb04009e40 tx=0x7fdb0400f730 comp rx=0 tx=0).stop 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.636+0000 7fdb18618740 1 -- 172.21.15.63:0/2104283149 shutdown_connections 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.636+0000 7fdb18618740 1 --2- 172.21.15.63:0/2104283149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30ea4d0 0x55ead34ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.636+0000 7fdb18618740 1 --2- 172.21.15.63:0/2104283149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30e22f0 0x55ead34adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.636+0000 7fdb18618740 1 --2- 172.21.15.63:0/2104283149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34ad860 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.636+0000 7fdb18618740 1 -- 172.21.15.63:0/2104283149 >> 172.21.15.63:0/2104283149 conn(0x55ead30ff300 msgr2=0x55ead34aa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.636+0000 7fdb0aa00640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdaf00b29d0 0x7fdaf00b4ee0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0dff80 tx=0x7fdafc0e2e10 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.636+0000 7fdb18618740 1 -- 172.21.15.63:0/2104283149 shutdown_connections 2024-05-27T07:59:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.636+0000 7fdb18618740 1 -- 172.21.15.63:0/2104283149 wait complete. 2024-05-27T07:59:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.637+0000 7fdb18618740 1 -- start start 2024-05-27T07:59:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.637+0000 7fdb18618740 1 Processor -- start 2024-05-27T07:59:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.638+0000 7fdb18618740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 0x55ead34b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.638+0000 7fdb18618740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30ea4d0 0x55ead34b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.638+0000 7fdb18618740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34c3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.638+0000 7fdb18618740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ead30f4790 con 0x55ead310b1b0 2024-05-27T07:59:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.638+0000 7fdb18618740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ead33b6bc0 con 0x55ead30e22f0 2024-05-27T07:59:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.638+0000 7fdb18618740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ead34ba990 con 0x55ead30ea4d0 2024-05-27T07:59:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.638+0000 7fdb18618740 10 client.0 _ll_register_callbacks cb 0x55ead3394400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.638+0000 7fdb0b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34c3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.639+0000 7fdb0b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34c3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59190/0 (socket says 172.21.15.63:59190) 2024-05-27T07:59:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.639+0000 7fdb0b400640 1 -- 172.21.15.63:0/1026638001 learned_addr learned my addr 172.21.15.63:0/1026638001 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.639+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 0x55ead34b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.639+0000 7fdb0a000640 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30ea4d0 0x55ead34b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.639+0000 7fdb0b400640 1 -- 172.21.15.63:0/1026638001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30ea4d0 msgr2=0x55ead34b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.639+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30ea4d0 0x55ead34b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.639+0000 7fdb0b400640 1 -- 172.21.15.63:0/1026638001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 msgr2=0x55ead34b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.639+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 0x55ead34b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.639+0000 7fdb0b400640 1 -- 172.21.15.63:0/1026638001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ead3123210 con 0x55ead310b1b0 2024-05-27T07:59:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.640+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34c3cc0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fdb040092f0 tx=0x7fdb0400ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1580a29f8a66f850 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.640+0000 7fdaebe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.640+0000 7fdaebe00640 1 -- 172.21.15.63:0/1026638001 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0402c060 con 0x55ead310b1b0 2024-05-27T07:59:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.640+0000 7fdb18618740 1 -- 172.21.15.63:0/1026638001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb040054b0 con 0x55ead310b1b0 2024-05-27T07:59:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.640+0000 7fdaebe00640 1 -- 172.21.15.63:0/1026638001 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7fdb04005660 con 0x55ead310b1b0 2024-05-27T07:59:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdaebe00640 1 -- 172.21.15.63:0/1026638001 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdb04004cf0 con 0x55ead310b1b0 2024-05-27T07:59:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdb18618740 10 client.20394 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdb18618740 1 -- 172.21.15.63:0/1026638001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ead31ef620 con 0x55ead310b1b0 2024-05-27T07:59:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdaebe00640 1 -- 172.21.15.63:0/1026638001 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb040054b0 con 0x55ead310b1b0 2024-05-27T07:59:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdaebe00640 1 -- 172.21.15.63:0/1026638001 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fdb040194a0 con 0x55ead310b1b0 2024-05-27T07:59:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdb18618740 10 client.20394 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdb18618740 10 client.20394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdb18618740 20 client.20394 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdb18618740 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ead30c9d80 0x55ead34c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdb18618740 4 client.20394 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.641+0000 7fdb18618740 1 -- 172.21.15.63:0/1026638001 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ead3420ae0 con 0x55ead30c9d80 2024-05-27T07:59:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.642+0000 7fdb0a000640 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ead30c9d80 0x55ead34c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.643+0000 7fdb0a000640 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ead30c9d80 0x55ead34c9bc0 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef55054cb04d753e server_cookie=4b9a1c2cb6f83e82 in_seq=0 out_seq=0 2024-05-27T07:59:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.643+0000 7fdaebe00640 10 client.20394 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdaebe00640 1 -- 172.21.15.63:0/1026638001 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ead3420ae0 con 0x55ead30c9d80 2024-05-27T07:59:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdaebe00640 10 client.20394 handle_command_reply: tid=0 2024-05-27T07:59:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdb18618740 1 client.20394 shutdown 2024-05-27T07:59:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdb18618740 1 -- 172.21.15.63:0/1026638001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 msgr2=0x55ead34c3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdb18618740 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34c3cc0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fdb040092f0 tx=0x7fdb0400ff00 comp rx=0 tx=0).stop 2024-05-27T07:59:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdb18618740 1 -- 172.21.15.63:0/1026638001 shutdown_connections 2024-05-27T07:59:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdb18618740 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ead30c9d80 0x55ead34c9bc0 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdb18618740 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead310b1b0 0x55ead34c3cc0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdb18618740 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30ea4d0 0x55ead34b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdb18618740 1 --2- 172.21.15.63:0/1026638001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 0x55ead34b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdaebe00640 0 client.20394 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.645+0000 7fdb18618740 1 -- 172.21.15.63:0/1026638001 >> 172.21.15.63:0/1026638001 conn(0x55ead30ff300 msgr2=0x55ead34ad690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.646+0000 7fdb18618740 1 -- 172.21.15.63:0/1026638001 shutdown_connections 2024-05-27T07:59:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.646+0000 7fdb18618740 1 -- 172.21.15.63:0/1026638001 wait complete. 2024-05-27T07:59:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.646+0000 7fdb18618740 20 client.20394 trim_cache size 0 max 16384 2024-05-27T07:59:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.663+0000 7fdb18618740 1 Processor -- start 2024-05-27T07:59:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.663+0000 7fdb18618740 1 -- start start 2024-05-27T07:59:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.663+0000 7fdb18618740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30e22f0 0x55ead34b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.663+0000 7fdb18618740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30ea4d0 0x55ead34cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.663+0000 7fdb18618740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead310b1b0 0x55ead34cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.663+0000 7fdb18618740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1ba100 con 0x55ead30e22f0 2024-05-27T07:59:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.663+0000 7fdb18618740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1b79d0 con 0x55ead310b1b0 2024-05-27T07:59:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.663+0000 7fdb18618740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1b5840 con 0x55ead30ea4d0 2024-05-27T07:59:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.664+0000 7fdb0b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30e22f0 0x55ead34b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.664+0000 7fdb0b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30e22f0 0x55ead34b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59194/0 (socket says 172.21.15.63:59194) 2024-05-27T07:59:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.664+0000 7fdb0b400640 1 -- 172.21.15.63:0/1504098957 learned_addr learned my addr 172.21.15.63:0/1504098957 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.664+0000 7fdb0a000640 1 --2- 172.21.15.63:0/1504098957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30ea4d0 0x55ead34cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.664+0000 7fdb0b400640 1 -- 172.21.15.63:0/1504098957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30ea4d0 msgr2=0x55ead34cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.664+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1504098957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30ea4d0 0x55ead34cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.664+0000 7fdb0b400640 1 -- 172.21.15.63:0/1504098957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead310b1b0 msgr2=0x55ead34cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.664+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1504098957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead310b1b0 0x55ead34cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.664+0000 7fdb0b400640 1 -- 172.21.15.63:0/1504098957 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ead34b8f20 con 0x55ead30e22f0 2024-05-27T07:59:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.664+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1504098957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30e22f0 0x55ead34b7fb0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400a010 tx=0x7fdb04004eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b70975a2bff7ddf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.665+0000 7fdaf6a00640 1 -- 172.21.15.63:0/1504098957 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0402c040 con 0x55ead30e22f0 2024-05-27T07:59:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.665+0000 7fdaf6a00640 1 -- 172.21.15.63:0/1504098957 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb04019cb0 con 0x55ead30e22f0 2024-05-27T07:59:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.665+0000 7fdaf6a00640 1 -- 172.21.15.63:0/1504098957 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04019650 con 0x55ead30e22f0 2024-05-27T07:59:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.666+0000 7fdb18618740 1 -- 172.21.15.63:0/1504098957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30e22f0 msgr2=0x55ead34b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.666+0000 7fdb18618740 1 --2- 172.21.15.63:0/1504098957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30e22f0 0x55ead34b7fb0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400a010 tx=0x7fdb04004eb0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.666+0000 7fdb18618740 1 -- 172.21.15.63:0/1504098957 shutdown_connections 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.666+0000 7fdb18618740 1 --2- 172.21.15.63:0/1504098957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead310b1b0 0x55ead34cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.666+0000 7fdb18618740 1 --2- 172.21.15.63:0/1504098957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead30ea4d0 0x55ead34cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.666+0000 7fdb18618740 1 --2- 172.21.15.63:0/1504098957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30e22f0 0x55ead34b7fb0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.666+0000 7fdb18618740 1 -- 172.21.15.63:0/1504098957 >> 172.21.15.63:0/1504098957 conn(0x55ead30ff300 msgr2=0x55ead34b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.666+0000 7fdb18618740 1 -- 172.21.15.63:0/1504098957 shutdown_connections 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.666+0000 7fdb18618740 1 -- 172.21.15.63:0/1504098957 wait complete. 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.666+0000 7fdb18618740 1 -- start start 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.667+0000 7fdb18618740 1 Processor -- start 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.667+0000 7fdb18618740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 0x55ead34bcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.667+0000 7fdb18618740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30ea4d0 0x55ead34bd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.667+0000 7fdb18618740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead310b1b0 0x55ead3530060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.667+0000 7fdb18618740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1b5e90 con 0x55ead30ea4d0 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.667+0000 7fdb18618740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1b7300 con 0x55ead30e22f0 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.667+0000 7fdb18618740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c1b6560 con 0x55ead310b1b0 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.667+0000 7fdb18618740 10 client.0 _ll_register_callbacks cb 0x7fdb04009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30ea4d0 0x55ead34bd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30ea4d0 0x55ead34bd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59202/0 (socket says 172.21.15.63:59202) 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0a000640 1 -- 172.21.15.63:0/1086600505 learned_addr learned my addr 172.21.15.63:0/1086600505 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 0x55ead34bcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0a000640 1 -- 172.21.15.63:0/1086600505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead310b1b0 msgr2=0x55ead3530060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0a000640 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead310b1b0 0x55ead3530060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0a000640 1 -- 172.21.15.63:0/1086600505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 msgr2=0x55ead34bcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0a000640 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 0x55ead34bcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0a000640 1 -- 172.21.15.63:0/1086600505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ead34bdad0 con 0x55ead30ea4d0 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 0x55ead34bcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdb0a000640 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30ea4d0 0x55ead34bd280 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fdaf802a910 tx=0x7fdaf802aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4766441ffb1a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.668+0000 7fdaebe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.669+0000 7fdaebe00640 1 -- 172.21.15.63:0/1086600505 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8035040 con 0x55ead30ea4d0 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.669+0000 7fdaebe00640 1 -- 172.21.15.63:0/1086600505 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7fdaf8004390 con 0x55ead30ea4d0 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.669+0000 7fdb18618740 1 -- 172.21.15.63:0/1086600505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb04019cb0 con 0x55ead30ea4d0 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.669+0000 7fdaebe00640 1 -- 172.21.15.63:0/1086600505 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdaf8050050 con 0x55ead30ea4d0 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.669+0000 7fdb18618740 10 client.20400 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.669+0000 7fdb18618740 1 -- 172.21.15.63:0/1086600505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ead35315c0 con 0x55ead30ea4d0 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.669+0000 7fdaebe00640 1 -- 172.21.15.63:0/1086600505 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb04019cb0 con 0x55ead30ea4d0 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.670+0000 7fdaebe00640 1 -- 172.21.15.63:0/1086600505 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fdaf8004bf0 con 0x55ead30ea4d0 2024-05-27T07:59:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.670+0000 7fdb18618740 10 client.20400 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.670+0000 7fdb18618740 10 client.20400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.670+0000 7fdb18618740 20 client.20400 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.670+0000 7fdb18618740 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ead309f1c0 0x55ead3533e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.670+0000 7fdb18618740 4 client.20400 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:59:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.670+0000 7fdb18618740 1 -- 172.21.15.63:0/1086600505 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ead341a2c0 con 0x55ead309f1c0 2024-05-27T07:59:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.670+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ead309f1c0 0x55ead3533e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.671+0000 7fdb0b400640 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ead309f1c0 0x55ead3533e00 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0137438da4409c server_cookie=6eeeb2f636955724 in_seq=0 out_seq=0 2024-05-27T07:59:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.671+0000 7fdaebe00640 10 client.20400 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:19 smithi086 ceph-mon[34687]: pgmap v816: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:59:19.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:19 smithi149 ceph-mon[35408]: pgmap v816: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:59:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.715+0000 7fdaebe00640 1 -- 172.21.15.63:0/1086600505 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ead341a2c0 con 0x55ead309f1c0 2024-05-27T07:59:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.715+0000 7fdaebe00640 10 client.20400 handle_command_reply: tid=0 2024-05-27T07:59:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.715+0000 7fdb18618740 1 client.20400 shutdown 2024-05-27T07:59:19.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:19 smithi063 ceph-mon[31613]: pgmap v816: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.716+0000 7fdb18618740 1 -- 172.21.15.63:0/1086600505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30ea4d0 msgr2=0x55ead34bd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.716+0000 7fdb18618740 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30ea4d0 0x55ead34bd280 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fdaf802a910 tx=0x7fdaf802aa70 comp rx=0 tx=0).stop 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.716+0000 7fdb18618740 1 -- 172.21.15.63:0/1086600505 shutdown_connections 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.716+0000 7fdb18618740 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ead309f1c0 0x55ead3533e00 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.716+0000 7fdb18618740 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ead310b1b0 0x55ead3530060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.716+0000 7fdb18618740 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ead30ea4d0 0x55ead34bd280 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.716+0000 7fdb18618740 1 --2- 172.21.15.63:0/1086600505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ead30e22f0 0x55ead34bcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.716+0000 7fdb18618740 1 -- 172.21.15.63:0/1086600505 >> 172.21.15.63:0/1086600505 conn(0x55ead30ff300 msgr2=0x55ead34b79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.716+0000 7fdaebe00640 0 client.20400 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.716+0000 7fdb18618740 1 -- 172.21.15.63:0/1086600505 shutdown_connections 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.717+0000 7fdb18618740 1 -- 172.21.15.63:0/1086600505 wait complete. 2024-05-27T07:59:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.717+0000 7fdb18618740 20 client.20400 trim_cache size 0 max 16384 2024-05-27T07:59:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:59:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:59:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.718+0000 7fdb11600640 1 -- 172.21.15.63:0/2053475486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdaf00b29d0 msgr2=0x7fdaf00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.718+0000 7fdb11600640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdaf00b29d0 0x7fdaf00b4ee0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0dff80 tx=0x7fdafc0e2e10 comp rx=0 tx=0).stop 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.718+0000 7fdb11600640 1 -- 172.21.15.63:0/2053475486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb0c10a500 msgr2=0x7fdb0c1a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.718+0000 7fdb11600640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb0c10a500 0x7fdb0c1a0d70 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00c950 tx=0x7fdafc00ce10 comp rx=0 tx=0).stop 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.718+0000 7fdb11600640 1 -- 172.21.15.63:0/2053475486 shutdown_connections 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.718+0000 7fdb11600640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdaf00b29d0 0x7fdaf00b4ee0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.718+0000 7fdb11600640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb0c10f540 0x7fdb0c1a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.718+0000 7fdb11600640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0c10ea80 0x7fdb0c1a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.718+0000 7fdb11600640 1 --2- 172.21.15.63:0/2053475486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb0c10a500 0x7fdb0c1a0d70 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.718+0000 7fdb11600640 1 -- 172.21.15.63:0/2053475486 >> 172.21.15.63:0/2053475486 conn(0x7fdb0c077190 msgr2=0x7fdb0c07a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.719+0000 7fdb11600640 1 -- 172.21.15.63:0/2053475486 shutdown_connections 2024-05-27T07:59:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:19.719+0000 7fdb11600640 1 -- 172.21.15.63:0/2053475486 wait complete. 2024-05-27T07:59:21.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:21 smithi086 ceph-mon[34687]: pgmap v817: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:59:21.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:21 smithi149 ceph-mon[35408]: pgmap v817: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:59:21.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:21 smithi063 ceph-mon[31613]: pgmap v817: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.0 MiB/s wr, 45 op/s 2024-05-27T07:59:23.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:23 smithi086 ceph-mon[34687]: pgmap v818: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.5 MiB/s wr, 48 op/s 2024-05-27T07:59:23.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:23 smithi149 ceph-mon[35408]: pgmap v818: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.5 MiB/s wr, 48 op/s 2024-05-27T07:59:23.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:23 smithi063 ceph-mon[31613]: pgmap v818: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.5 MiB/s wr, 48 op/s 2024-05-27T07:59:25.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:25 smithi086 ceph-mon[34687]: pgmap v819: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:59:25.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:25 smithi149 ceph-mon[35408]: pgmap v819: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:59:25.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:25 smithi063 ceph-mon[31613]: pgmap v819: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:59:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:27 smithi086 ceph-mon[34687]: pgmap v820: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:59:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:27 smithi149 ceph-mon[35408]: pgmap v820: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:59:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:27 smithi063 ceph-mon[31613]: pgmap v820: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T07:59:29.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:29 smithi086 ceph-mon[34687]: pgmap v821: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T07:59:29.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:29 smithi149 ceph-mon[35408]: pgmap v821: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T07:59:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:29 smithi063 ceph-mon[31613]: pgmap v821: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T07:59:31.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:31 smithi086 ceph-mon[34687]: pgmap v822: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:59:31.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:31 smithi149 ceph-mon[35408]: pgmap v822: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:59:31.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:31 smithi063 ceph-mon[31613]: pgmap v822: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:59:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.990+0000 7f42e9000640 1 -- 172.21.15.63:0/3673030090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e410e000 msgr2=0x7f42e41104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.990+0000 7f42e9000640 1 --2- 172.21.15.63:0/3673030090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e410e000 0x7f42e41104f0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009ca0 tx=0x7f42dc02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:59:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.990+0000 7f42e9000640 1 -- 172.21.15.63:0/3673030090 shutdown_connections 2024-05-27T07:59:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.990+0000 7f42e9000640 1 --2- 172.21.15.63:0/3673030090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e410e000 0x7f42e41104f0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.990+0000 7f42e9000640 1 --2- 172.21.15.63:0/3673030090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e410b610 0x7f42e410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.990+0000 7f42e9000640 1 --2- 172.21.15.63:0/3673030090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e4104bb0 0x7f42e4104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.990+0000 7f42e9000640 1 -- 172.21.15.63:0/3673030090 >> 172.21.15.63:0/3673030090 conn(0x7f42e40fe210 msgr2=0x7f42e4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.990+0000 7f42e9000640 1 -- 172.21.15.63:0/3673030090 shutdown_connections 2024-05-27T07:59:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.991+0000 7f42e9000640 1 -- 172.21.15.63:0/3673030090 wait complete. 2024-05-27T07:59:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.992+0000 7f42e9000640 1 Processor -- start 2024-05-27T07:59:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.992+0000 7f42e9000640 1 -- start start 2024-05-27T07:59:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.993+0000 7f42e9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e4104bb0 0x7f42e419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.993+0000 7f42e9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e410b610 0x7f42e419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.993+0000 7f42e9000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e410e000 0x7f42e419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.993+0000 7f42e9000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4113b60 con 0x7f42e410b610 2024-05-27T07:59:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.993+0000 7f42e9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41139e0 con 0x7f42e4104bb0 2024-05-27T07:59:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.993+0000 7f42e9000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4113ce0 con 0x7f42e410e000 2024-05-27T07:59:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.993+0000 7f42e2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e410b610 0x7f42e419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.993+0000 7f42e2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e410b610 0x7f42e419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43698/0 (socket says 172.21.15.63:43698) 2024-05-27T07:59:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.993+0000 7f42e2a00640 1 -- 172.21.15.63:0/3358837521 learned_addr learned my addr 172.21.15.63:0/3358837521 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.994+0000 7f42e3e00640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e410e000 0x7f42e419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.994+0000 7f42e3400640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e4104bb0 0x7f42e419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.994+0000 7f42e2a00640 1 -- 172.21.15.63:0/3358837521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e410e000 msgr2=0x7f42e419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.994+0000 7f42e2a00640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e410e000 0x7f42e419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.994+0000 7f42e2a00640 1 -- 172.21.15.63:0/3358837521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e4104bb0 msgr2=0x7f42e419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.994+0000 7f42e2a00640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e4104bb0 0x7f42e419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.994+0000 7f42e2a00640 1 -- 172.21.15.63:0/3358837521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e419f3e0 con 0x7f42e410b610 2024-05-27T07:59:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.995+0000 7f42e2a00640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e410b610 0x7f42e419aae0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f42d000d950 tx=0x7f42d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.995+0000 7f42dbe00640 1 -- 172.21.15.63:0/3358837521 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0017070 con 0x7f42e410b610 2024-05-27T07:59:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.995+0000 7f42dbe00640 1 -- 172.21.15.63:0/3358837521 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42d0009c30 con 0x7f42e410b610 2024-05-27T07:59:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.995+0000 7f42e9000640 1 -- 172.21.15.63:0/3358837521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e419f6d0 con 0x7f42e410b610 2024-05-27T07:59:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.995+0000 7f42dbe00640 1 -- 172.21.15.63:0/3358837521 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d000f620 con 0x7f42e410b610 2024-05-27T07:59:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:32.995+0000 7f42e9000640 1 -- 172.21.15.63:0/3358837521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e41a7050 con 0x7f42e410b610 2024-05-27T07:59:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.000+0000 7f42f1598740 1 Processor -- start 2024-05-27T07:59:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.000+0000 7f42f1598740 1 -- start start 2024-05-27T07:59:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.001+0000 7f42f1598740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb48a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.001+0000 7f42f1598740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd77d2f0 0x5592bdb48fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.001+0000 7f42f1598740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd7854d0 0x5592bdb494e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.001+0000 7f42f1598740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5592bdab9b90 con 0x5592bd7a61b0 2024-05-27T07:59:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.001+0000 7f42f1598740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5592bdaa9730 con 0x5592bd7854d0 2024-05-27T07:59:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.001+0000 7f42f1598740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5592bdaad770 con 0x5592bd77d2f0 2024-05-27T07:59:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.002+0000 7f42e3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd77d2f0 0x5592bdb48fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.002+0000 7f42dbe00640 1 -- 172.21.15.63:0/3358837521 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f42d0010460 con 0x7f42e410b610 2024-05-27T07:59:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.002+0000 7f42e3400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd77d2f0 0x5592bdb48fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:39426/0 (socket says 172.21.15.63:39426) 2024-05-27T07:59:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.002+0000 7f42e3400640 1 -- 172.21.15.63:0/669290484 learned_addr learned my addr 172.21.15.63:0/669290484 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.003+0000 7f42e3e00640 1 --2- 172.21.15.63:0/669290484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb48a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.003+0000 7f42e3400640 1 -- 172.21.15.63:0/669290484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd7854d0 msgr2=0x5592bdb494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.003+0000 7f42e3400640 1 --2- 172.21.15.63:0/669290484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd7854d0 0x5592bdb494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.003+0000 7f42e3400640 1 -- 172.21.15.63:0/669290484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 msgr2=0x5592bdb48a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.003+0000 7f42e3400640 1 --2- 172.21.15.63:0/669290484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb48a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.003+0000 7f42e3400640 1 -- 172.21.15.63:0/669290484 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592bda428b0 con 0x5592bd77d2f0 2024-05-27T07:59:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.004+0000 7f42e3400640 1 --2- 172.21.15.63:0/669290484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd77d2f0 0x5592bdb48fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f42d400dcb0 tx=0x7f42d400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e04833cb476391df server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.005+0000 7f42d8c00640 1 -- 172.21.15.63:0/669290484 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4011440 con 0x5592bd77d2f0 2024-05-27T07:59:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.005+0000 7f42d8c00640 1 -- 172.21.15.63:0/669290484 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42d4011a70 con 0x5592bd77d2f0 2024-05-27T07:59:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.005+0000 7f42d8c00640 1 -- 172.21.15.63:0/669290484 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d401b960 con 0x5592bd77d2f0 2024-05-27T07:59:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.005+0000 7f42dbe00640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f42c80b2950 0x7f42c80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.005+0000 7f42dbe00640 1 -- 172.21.15.63:0/3358837521 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f42d00e2630 con 0x7f42e410b610 2024-05-27T07:59:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.005+0000 7f42e3e00640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f42c80b2950 0x7f42c80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.006+0000 7f42f1598740 1 -- 172.21.15.63:0/669290484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd77d2f0 msgr2=0x5592bdb48fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.006+0000 7f42f1598740 1 --2- 172.21.15.63:0/669290484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd77d2f0 0x5592bdb48fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f42d400dcb0 tx=0x7f42d400f730 comp rx=0 tx=0).stop 2024-05-27T07:59:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.006+0000 7f42f1598740 1 -- 172.21.15.63:0/669290484 shutdown_connections 2024-05-27T07:59:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.006+0000 7f42f1598740 1 --2- 172.21.15.63:0/669290484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd7854d0 0x5592bdb494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.006+0000 7f42f1598740 1 --2- 172.21.15.63:0/669290484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd77d2f0 0x5592bdb48fa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.006+0000 7f42f1598740 1 --2- 172.21.15.63:0/669290484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb48a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.006+0000 7f42f1598740 1 -- 172.21.15.63:0/669290484 >> 172.21.15.63:0/669290484 conn(0x5592bd79a300 msgr2=0x5592bdb45210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.006+0000 7f42e3e00640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f42c80b2950 0x7f42c80b4e60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f42dc005e30 tx=0x7f42dc005c50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.006+0000 7f42f1598740 1 -- 172.21.15.63:0/669290484 shutdown_connections 2024-05-27T07:59:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.007+0000 7f42f1598740 1 -- 172.21.15.63:0/669290484 wait complete. 2024-05-27T07:59:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.008+0000 7f42f1598740 1 -- start start 2024-05-27T07:59:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.008+0000 7f42f1598740 1 Processor -- start 2024-05-27T07:59:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42f1598740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd77d2f0 0x5592bdb52ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42f1598740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb53400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42f1598740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd7a61b0 0x5592bdb632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42f1598740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5592bd78f790 con 0x5592bd77d2f0 2024-05-27T07:59:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42f1598740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5592bda51bc0 con 0x5592bd7a61b0 2024-05-27T07:59:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42f1598740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5592bdb59fa0 con 0x5592bd7854d0 2024-05-27T07:59:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42f1598740 10 client.0 _ll_register_callbacks cb 0x5592bda2f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42e3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd77d2f0 0x5592bdb52ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42e3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd77d2f0 0x5592bdb52ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43702/0 (socket says 172.21.15.63:43702) 2024-05-27T07:59:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42e3400640 1 -- 172.21.15.63:0/9538774 learned_addr learned my addr 172.21.15.63:0/9538774 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.009+0000 7f42e2a00640 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb53400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.010+0000 7f42e3e00640 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd7a61b0 0x5592bdb632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.010+0000 7f42e3400640 1 -- 172.21.15.63:0/9538774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 msgr2=0x5592bdb53400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.010+0000 7f42e3400640 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb53400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.010+0000 7f42e3400640 1 -- 172.21.15.63:0/9538774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd7a61b0 msgr2=0x5592bdb632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.010+0000 7f42e3400640 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd7a61b0 0x5592bdb632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.010+0000 7f42e3400640 1 -- 172.21.15.63:0/9538774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592bd7be210 con 0x5592bd77d2f0 2024-05-27T07:59:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.011+0000 7f42e3400640 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd77d2f0 0x5592bdb52ec0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f42d401a000 tx=0x7f42d401b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b5bef6334271a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.011+0000 7f42cd600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.011+0000 7f42cd600640 1 -- 172.21.15.63:0/9538774 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4004240 con 0x5592bd77d2f0 2024-05-27T07:59:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.011+0000 7f42cd600640 1 -- 172.21.15.63:0/9538774 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f42d40043e0 con 0x5592bd77d2f0 2024-05-27T07:59:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.011+0000 7f42f1598740 1 -- 172.21.15.63:0/9538774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d4011a70 con 0x5592bd77d2f0 2024-05-27T07:59:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.012+0000 7f42cd600640 1 -- 172.21.15.63:0/9538774 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f42d4032050 con 0x5592bd77d2f0 2024-05-27T07:59:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.012+0000 7f42f1598740 10 client.20412 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.012+0000 7f42f1598740 1 -- 172.21.15.63:0/9538774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5592bd88a620 con 0x5592bd77d2f0 2024-05-27T07:59:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.012+0000 7f42cd600640 1 -- 172.21.15.63:0/9538774 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42d4011a70 con 0x5592bd77d2f0 2024-05-27T07:59:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.013+0000 7f42cd600640 1 -- 172.21.15.63:0/9538774 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f42d402d030 con 0x5592bd77d2f0 2024-05-27T07:59:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.013+0000 7f42f1598740 10 client.20412 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.013+0000 7f42f1598740 10 client.20412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.013+0000 7f42f1598740 20 client.20412 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.013+0000 7f42f1598740 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5592bd764d80 0x5592bdb69210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.013+0000 7f42f1598740 4 client.20412 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.013+0000 7f42f1598740 1 -- 172.21.15.63:0/9538774 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592bdabbae0 con 0x5592bd764d80 2024-05-27T07:59:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.014+0000 7f42e2a00640 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5592bd764d80 0x5592bdb69210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.016+0000 7f42e2a00640 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5592bd764d80 0x5592bdb69210 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc8bf89620aeb431 server_cookie=f5d0eb2876bd817d in_seq=0 out_seq=0 2024-05-27T07:59:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.016+0000 7f42cd600640 10 client.20412 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.017+0000 7f42cd600640 1 -- 172.21.15.63:0/9538774 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5592bdabbae0 con 0x5592bd764d80 2024-05-27T07:59:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42cd600640 10 client.20412 handle_command_reply: tid=0 2024-05-27T07:59:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42f1598740 1 client.20412 shutdown 2024-05-27T07:59:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42f1598740 1 -- 172.21.15.63:0/9538774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd77d2f0 msgr2=0x5592bdb52ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42f1598740 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd77d2f0 0x5592bdb52ec0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f42d401a000 tx=0x7f42d401b480 comp rx=0 tx=0).stop 2024-05-27T07:59:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42f1598740 1 -- 172.21.15.63:0/9538774 shutdown_connections 2024-05-27T07:59:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42f1598740 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5592bd764d80 0x5592bdb69210 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42f1598740 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd7a61b0 0x5592bdb632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42f1598740 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb53400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42f1598740 1 --2- 172.21.15.63:0/9538774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd77d2f0 0x5592bdb52ec0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42cd600640 0 client.20412 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42f1598740 1 -- 172.21.15.63:0/9538774 >> 172.21.15.63:0/9538774 conn(0x5592bd79a300 msgr2=0x5592bdb4a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.018+0000 7f42f1598740 1 -- 172.21.15.63:0/9538774 shutdown_connections 2024-05-27T07:59:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.019+0000 7f42f1598740 1 -- 172.21.15.63:0/9538774 wait complete. 2024-05-27T07:59:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.019+0000 7f42f1598740 20 client.20412 trim_cache size 0 max 16384 2024-05-27T07:59:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.042+0000 7f42f1598740 1 Processor -- start 2024-05-27T07:59:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.042+0000 7f42f1598740 1 -- start start 2024-05-27T07:59:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.042+0000 7f42f1598740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 0x5592bdb49920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.042+0000 7f42f1598740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb63240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.042+0000 7f42f1598740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb4c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.042+0000 7f42f1598740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41b1830 con 0x5592bd7a61b0 2024-05-27T07:59:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.042+0000 7f42f1598740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41af100 con 0x5592bd77d2f0 2024-05-27T07:59:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.042+0000 7f42f1598740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41acf70 con 0x5592bd7854d0 2024-05-27T07:59:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.043+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb4c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.043+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb4c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43704/0 (socket says 172.21.15.63:43704) 2024-05-27T07:59:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.043+0000 7f42e3e00640 1 -- 172.21.15.63:0/1767682051 learned_addr learned my addr 172.21.15.63:0/1767682051 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.043+0000 7f42e3400640 1 --2- 172.21.15.63:0/1767682051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 0x5592bdb49920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.043+0000 7f42e2a00640 1 --2- 172.21.15.63:0/1767682051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb63240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.044+0000 7f42e3e00640 1 -- 172.21.15.63:0/1767682051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 msgr2=0x5592bdb63240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.044+0000 7f42e3e00640 1 --2- 172.21.15.63:0/1767682051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb63240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.044+0000 7f42e3e00640 1 -- 172.21.15.63:0/1767682051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 msgr2=0x5592bdb49920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.044+0000 7f42e3e00640 1 --2- 172.21.15.63:0/1767682051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 0x5592bdb49920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.044+0000 7f42e3e00640 1 -- 172.21.15.63:0/1767682051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592bdb3f210 con 0x5592bd7a61b0 2024-05-27T07:59:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.044+0000 7f42e3400640 1 --2- 172.21.15.63:0/1767682051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 0x5592bdb49920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.044+0000 7f42e3e00640 1 --2- 172.21.15.63:0/1767682051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb4c7c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009ca0 tx=0x7f42dc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bebcfdbbd8ef47 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.044+0000 7f42cfe00640 1 -- 172.21.15.63:0/1767682051 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc031a40 con 0x5592bd7a61b0 2024-05-27T07:59:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.045+0000 7f42cfe00640 1 -- 172.21.15.63:0/1767682051 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42dc036040 con 0x5592bd7a61b0 2024-05-27T07:59:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.045+0000 7f42cfe00640 1 -- 172.21.15.63:0/1767682051 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0047c0 con 0x5592bd7a61b0 2024-05-27T07:59:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.046+0000 7f42f1598740 1 -- 172.21.15.63:0/1767682051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 msgr2=0x5592bdb4c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.046+0000 7f42f1598740 1 --2- 172.21.15.63:0/1767682051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb4c7c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009ca0 tx=0x7f42dc041000 comp rx=0 tx=0).stop 2024-05-27T07:59:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.046+0000 7f42f1598740 1 -- 172.21.15.63:0/1767682051 shutdown_connections 2024-05-27T07:59:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.046+0000 7f42f1598740 1 --2- 172.21.15.63:0/1767682051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb4c7c0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.046+0000 7f42f1598740 1 --2- 172.21.15.63:0/1767682051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb63240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.046+0000 7f42f1598740 1 --2- 172.21.15.63:0/1767682051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 0x5592bdb49920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.046+0000 7f42f1598740 1 -- 172.21.15.63:0/1767682051 >> 172.21.15.63:0/1767682051 conn(0x5592bd79a300 msgr2=0x5592bdb543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.046+0000 7f42f1598740 1 -- 172.21.15.63:0/1767682051 shutdown_connections 2024-05-27T07:59:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.046+0000 7f42f1598740 1 -- 172.21.15.63:0/1767682051 wait complete. 2024-05-27T07:59:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.046+0000 7f42f1598740 1 -- start start 2024-05-27T07:59:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.047+0000 7f42f1598740 1 Processor -- start 2024-05-27T07:59:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.047+0000 7f42f1598740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 0x5592bdb6c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.047+0000 7f42f1598740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb6c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.047+0000 7f42f1598740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb67800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.047+0000 7f42f1598740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41ad5c0 con 0x5592bd7a61b0 2024-05-27T07:59:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.047+0000 7f42f1598740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41aea30 con 0x5592bd77d2f0 2024-05-27T07:59:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.047+0000 7f42f1598740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41adc90 con 0x5592bd7854d0 2024-05-27T07:59:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.047+0000 7f42f1598740 10 client.0 _ll_register_callbacks cb 0x7f42d4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.048+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb67800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.048+0000 7f42e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 0x5592bdb6c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.048+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb67800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43708/0 (socket says 172.21.15.63:43708) 2024-05-27T07:59:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.048+0000 7f42e3e00640 1 -- 172.21.15.63:0/2422745825 learned_addr learned my addr 172.21.15.63:0/2422745825 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.048+0000 7f42e2a00640 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb6c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.048+0000 7f42e3e00640 1 -- 172.21.15.63:0/2422745825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 msgr2=0x5592bdb6c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.048+0000 7f42e3e00640 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb6c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.048+0000 7f42e3e00640 1 -- 172.21.15.63:0/2422745825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 msgr2=0x5592bdb6c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.048+0000 7f42e3e00640 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 0x5592bdb6c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.048+0000 7f42e3e00640 1 -- 172.21.15.63:0/2422745825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592bdb67ef0 con 0x5592bd7a61b0 2024-05-27T07:59:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.049+0000 7f42e3e00640 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb67800 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f42dc02a900 tx=0x7f42dc0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ce0c6a95ed7dfb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.049+0000 7f42cd600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.049+0000 7f42cd600640 1 -- 172.21.15.63:0/2422745825 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc04d070 con 0x5592bd7a61b0 2024-05-27T07:59:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.049+0000 7f42cd600640 1 -- 172.21.15.63:0/2422745825 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f42dc042040 con 0x5592bd7a61b0 2024-05-27T07:59:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.049+0000 7f42f1598740 1 -- 172.21.15.63:0/2422745825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42dc036040 con 0x5592bd7a61b0 2024-05-27T07:59:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.050+0000 7f42cd600640 1 -- 172.21.15.63:0/2422745825 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f42dc0041d0 con 0x5592bd7a61b0 2024-05-27T07:59:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.050+0000 7f42f1598740 10 client.20418 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.050+0000 7f42f1598740 1 -- 172.21.15.63:0/2422745825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5592bdbd0f80 con 0x5592bd7a61b0 2024-05-27T07:59:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.050+0000 7f42cd600640 1 -- 172.21.15.63:0/2422745825 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42dc036040 con 0x5592bd7a61b0 2024-05-27T07:59:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.050+0000 7f42cd600640 1 -- 172.21.15.63:0/2422745825 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f42dc05f4b0 con 0x5592bd7a61b0 2024-05-27T07:59:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.050+0000 7f42f1598740 10 client.20418 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.051+0000 7f42f1598740 10 client.20418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.051+0000 7f42f1598740 20 client.20418 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.051+0000 7f42f1598740 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5592bd73a1c0 0x5592bdbd35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.051+0000 7f42f1598740 4 client.20418 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:59:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.051+0000 7f42f1598740 1 -- 172.21.15.63:0/2422745825 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592bdab52c0 con 0x5592bd73a1c0 2024-05-27T07:59:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.051+0000 7f42e3400640 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5592bd73a1c0 0x5592bdbd35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.052+0000 7f42e3400640 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5592bd73a1c0 0x5592bdbd35e0 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d6c09ee0a51b357 server_cookie=53b9af00b83b75c5 in_seq=0 out_seq=0 2024-05-27T07:59:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.052+0000 7f42cd600640 10 client.20418 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42cd600640 1 -- 172.21.15.63:0/2422745825 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5592bdab52c0 con 0x5592bd73a1c0 2024-05-27T07:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42cd600640 10 client.20418 handle_command_reply: tid=0 2024-05-27T07:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42f1598740 1 client.20418 shutdown 2024-05-27T07:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42f1598740 1 -- 172.21.15.63:0/2422745825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 msgr2=0x5592bdb67800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42f1598740 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb67800 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f42dc02a900 tx=0x7f42dc0316e0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42f1598740 1 -- 172.21.15.63:0/2422745825 shutdown_connections 2024-05-27T07:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42f1598740 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5592bd73a1c0 0x5592bdbd35e0 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42f1598740 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5592bd7a61b0 0x5592bdb67800 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42f1598740 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bd7854d0 0x5592bdb6c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42f1598740 1 --2- 172.21.15.63:0/2422745825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5592bd77d2f0 0x5592bdb6c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.085+0000 7f42f1598740 1 -- 172.21.15.63:0/2422745825 >> 172.21.15.63:0/2422745825 conn(0x5592bd79a300 msgr2=0x5592bdb54d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.086+0000 7f42cd600640 0 client.20418 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.086+0000 7f42f1598740 1 -- 172.21.15.63:0/2422745825 shutdown_connections 2024-05-27T07:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.086+0000 7f42f1598740 1 -- 172.21.15.63:0/2422745825 wait complete. 2024-05-27T07:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.086+0000 7f42f1598740 20 client.20418 trim_cache size 0 max 16384 2024-05-27T07:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:59:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 -- 172.21.15.63:0/3358837521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f42c80b2950 msgr2=0x7f42c80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f42c80b2950 0x7f42c80b4e60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f42dc005e30 tx=0x7f42dc005c50 comp rx=0 tx=0).stop 2024-05-27T07:59:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 -- 172.21.15.63:0/3358837521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e410b610 msgr2=0x7f42e419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e410b610 0x7f42e419aae0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f42d000d950 tx=0x7f42d000de10 comp rx=0 tx=0).stop 2024-05-27T07:59:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 -- 172.21.15.63:0/3358837521 shutdown_connections 2024-05-27T07:59:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f42c80b2950 0x7f42c80b4e60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e410e000 0x7f42e419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42e410b610 0x7f42e419aae0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 --2- 172.21.15.63:0/3358837521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e4104bb0 0x7f42e419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 -- 172.21.15.63:0/3358837521 >> 172.21.15.63:0/3358837521 conn(0x7f42e40fe210 msgr2=0x7f42e410e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 -- 172.21.15.63:0/3358837521 shutdown_connections 2024-05-27T07:59:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:33.087+0000 7f42e9000640 1 -- 172.21.15.63:0/3358837521 wait complete. 2024-05-27T07:59:33.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:33 smithi086 ceph-mon[34687]: pgmap v823: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:59:33.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:33 smithi149 ceph-mon[35408]: pgmap v823: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:59:33.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:33 smithi063 ceph-mon[31613]: pgmap v823: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T07:59:35.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:35 smithi086 ceph-mon[34687]: pgmap v824: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:59:35.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:35 smithi149 ceph-mon[35408]: pgmap v824: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:59:35.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:35 smithi063 ceph-mon[31613]: pgmap v824: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:59:37.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:37 smithi086 ceph-mon[34687]: pgmap v825: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:59:37.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:37 smithi149 ceph-mon[35408]: pgmap v825: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:59:37.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:37 smithi063 ceph-mon[31613]: pgmap v825: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T07:59:39.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:39 smithi086 ceph-mon[34687]: pgmap v826: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:59:39.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:39 smithi149 ceph-mon[35408]: pgmap v826: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:59:39.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:39 smithi063 ceph-mon[31613]: pgmap v826: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:59:41.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:41 smithi086 ceph-mon[34687]: pgmap v827: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:59:41.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:41 smithi149 ceph-mon[35408]: pgmap v827: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:59:41.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:41 smithi063 ceph-mon[31613]: pgmap v827: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T07:59:43.584 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:59:43.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:43 smithi149 ceph-mon[35408]: pgmap v828: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:59:43.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:43 smithi063 ceph-mon[31613]: pgmap v828: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:59:43.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:43 smithi086 ceph-mon[34687]: pgmap v828: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T07:59:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:45.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:45 smithi063 ceph-mon[31613]: pgmap v829: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:59:45.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:45 smithi086 ceph-mon[34687]: pgmap v829: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:59:45.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:45 smithi149 ceph-mon[35408]: pgmap v829: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:59:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.340+0000 7ff518a00640 1 -- 172.21.15.63:0/3572200215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff514109ab0 msgr2=0x7ff5141036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.340+0000 7ff518a00640 1 --2- 172.21.15.63:0/3572200215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff514109ab0 0x7ff5141036a0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff504009980 tx=0x7ff50402e820 comp rx=0 tx=0).stop 2024-05-27T07:59:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.340+0000 7ff518a00640 1 -- 172.21.15.63:0/3572200215 shutdown_connections 2024-05-27T07:59:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.340+0000 7ff518a00640 1 --2- 172.21.15.63:0/3572200215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514104250 0x7ff5141105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.340+0000 7ff518a00640 1 --2- 172.21.15.63:0/3572200215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff514109f50 0x7ff514103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.340+0000 7ff518a00640 1 --2- 172.21.15.63:0/3572200215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff514109ab0 0x7ff5141036a0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.340+0000 7ff518a00640 1 -- 172.21.15.63:0/3572200215 >> 172.21.15.63:0/3572200215 conn(0x7ff514079240 msgr2=0x7ff514079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.340+0000 7ff518a00640 1 -- 172.21.15.63:0/3572200215 shutdown_connections 2024-05-27T07:59:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.340+0000 7ff518a00640 1 -- 172.21.15.63:0/3572200215 wait complete. 2024-05-27T07:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.342+0000 7ff518a00640 1 Processor -- start 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.342+0000 7ff518a00640 1 -- start start 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.342+0000 7ff518a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514104250 0x7ff514198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.342+0000 7ff518a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff514109ab0 0x7ff514198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.342+0000 7ff518a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff514109f50 0x7ff51419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.342+0000 7ff518a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141119c0 con 0x7ff514109f50 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.342+0000 7ff518a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff514111840 con 0x7ff514109ab0 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.342+0000 7ff518a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff514111b40 con 0x7ff514104250 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff513e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff514109f50 0x7ff51419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff513e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff514109f50 0x7ff51419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57918/0 (socket says 172.21.15.63:57918) 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff513e00640 1 -- 172.21.15.63:0/2910698029 learned_addr learned my addr 172.21.15.63:0/2910698029 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff512a00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff514109ab0 0x7ff514198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff513400640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514104250 0x7ff514198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff513e00640 1 -- 172.21.15.63:0/2910698029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514104250 msgr2=0x7ff514198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff513e00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514104250 0x7ff514198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff513e00640 1 -- 172.21.15.63:0/2910698029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff514109ab0 msgr2=0x7ff514198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff513e00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff514109ab0 0x7ff514198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff513e00640 1 -- 172.21.15.63:0/2910698029 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51419d290 con 0x7ff514109f50 2024-05-27T07:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.343+0000 7ff512a00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff514109ab0 0x7ff514198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.344+0000 7ff513e00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff514109f50 0x7ff51419cc50 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00eaa0 tx=0x7ff50c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.344+0000 7ff50be00640 1 -- 172.21.15.63:0/2910698029 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c00cdb0 con 0x7ff514109f50 2024-05-27T07:59:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.344+0000 7ff50be00640 1 -- 172.21.15.63:0/2910698029 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff50c005020 con 0x7ff514109f50 2024-05-27T07:59:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.344+0000 7ff518a00640 1 -- 172.21.15.63:0/2910698029 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51419d580 con 0x7ff514109f50 2024-05-27T07:59:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.344+0000 7ff50be00640 1 -- 172.21.15.63:0/2910698029 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0116d0 con 0x7ff514109f50 2024-05-27T07:59:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.344+0000 7ff518a00640 1 -- 172.21.15.63:0/2910698029 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a4f00 con 0x7ff514109f50 2024-05-27T07:59:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.348+0000 7ff520ff0740 1 Processor -- start 2024-05-27T07:59:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.350+0000 7ff50be00640 1 -- 172.21.15.63:0/2910698029 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff50c0051d0 con 0x7ff514109f50 2024-05-27T07:59:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.350+0000 7ff520ff0740 1 -- start start 2024-05-27T07:59:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.350+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d109102a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.350+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 0x55d109102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d3f4d0 0x55d1091034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff520ff0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d109073b90 con 0x55d108d601b0 2024-05-27T07:59:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff520ff0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d109063730 con 0x55d108d3f4d0 2024-05-27T07:59:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff520ff0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d109067770 con 0x55d108d372f0 2024-05-27T07:59:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff512a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d109102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff512a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d109102a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57932/0 (socket says 172.21.15.63:57932) 2024-05-27T07:59:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff512a00640 1 -- 172.21.15.63:0/1499580476 learned_addr learned my addr 172.21.15.63:0/1499580476 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff513400640 1 --2- 172.21.15.63:0/1499580476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 0x55d109102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff512a00640 1 -- 172.21.15.63:0/1499580476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 msgr2=0x55d109102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff512a00640 1 --2- 172.21.15.63:0/1499580476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 0x55d109102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff512a00640 1 -- 172.21.15.63:0/1499580476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d3f4d0 msgr2=0x55d1091034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.351+0000 7ff512a00640 1 --2- 172.21.15.63:0/1499580476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d3f4d0 0x55d1091034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.352+0000 7ff512a00640 1 -- 172.21.15.63:0/1499580476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d108ffc8b0 con 0x55d108d601b0 2024-05-27T07:59:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.352+0000 7ff513400640 1 --2- 172.21.15.63:0/1499580476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 0x55d109102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.352+0000 7ff50be00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4f80b29d0 0x7ff4f80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.352+0000 7ff512a00640 1 --2- 172.21.15.63:0/1499580476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d109102a60 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff50000bcd0 tx=0x7ff500011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8584387b43e103 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.352+0000 7ff50be00640 1 -- 172.21.15.63:0/2910698029 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7ff50c0e2bb0 con 0x7ff514109f50 2024-05-27T07:59:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.352+0000 7ff513400640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4f80b29d0 0x7ff4f80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.352+0000 7ff4ffe00640 1 -- 172.21.15.63:0/1499580476 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500004020 con 0x55d108d601b0 2024-05-27T07:59:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.353+0000 7ff4ffe00640 1 -- 172.21.15.63:0/1499580476 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5000041c0 con 0x55d108d601b0 2024-05-27T07:59:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.353+0000 7ff4ffe00640 1 -- 172.21.15.63:0/1499580476 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500012650 con 0x55d108d601b0 2024-05-27T07:59:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.354+0000 7ff520ff0740 1 -- 172.21.15.63:0/1499580476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 msgr2=0x55d109102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.354+0000 7ff513400640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4f80b29d0 0x7ff4f80b4ee0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff50402ee10 tx=0x7ff504005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.354+0000 7ff520ff0740 1 --2- 172.21.15.63:0/1499580476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d109102a60 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff50000bcd0 tx=0x7ff500011750 comp rx=0 tx=0).stop 2024-05-27T07:59:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.354+0000 7ff520ff0740 1 -- 172.21.15.63:0/1499580476 shutdown_connections 2024-05-27T07:59:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.354+0000 7ff520ff0740 1 --2- 172.21.15.63:0/1499580476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d3f4d0 0x55d1091034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.354+0000 7ff520ff0740 1 --2- 172.21.15.63:0/1499580476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 0x55d109102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.354+0000 7ff520ff0740 1 --2- 172.21.15.63:0/1499580476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d109102a60 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.354+0000 7ff520ff0740 1 -- 172.21.15.63:0/1499580476 >> 172.21.15.63:0/1499580476 conn(0x55d108d54300 msgr2=0x55d1090ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.354+0000 7ff520ff0740 1 -- 172.21.15.63:0/1499580476 shutdown_connections 2024-05-27T07:59:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.354+0000 7ff520ff0740 1 -- 172.21.15.63:0/1499580476 wait complete. 2024-05-27T07:59:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.355+0000 7ff520ff0740 1 -- start start 2024-05-27T07:59:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.355+0000 7ff520ff0740 1 Processor -- start 2024-05-27T07:59:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.356+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 0x55d10910cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.356+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d3f4d0 0x55d10910d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.356+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 0x55d109118ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.356+0000 7ff520ff0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d108d49790 con 0x55d108d3f4d0 2024-05-27T07:59:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.356+0000 7ff520ff0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d10900bbc0 con 0x55d108d601b0 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.356+0000 7ff520ff0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d10910fb90 con 0x55d108d372f0 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.356+0000 7ff520ff0740 10 client.0 _ll_register_callbacks cb 0x55d108fe9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.356+0000 7ff512a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d3f4d0 0x55d10910d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.356+0000 7ff512a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d3f4d0 0x55d10910d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57938/0 (socket says 172.21.15.63:57938) 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.356+0000 7ff512a00640 1 -- 172.21.15.63:0/3328109260 learned_addr learned my addr 172.21.15.63:0/3328109260 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.357+0000 7ff513e00640 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 0x55d109118ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.357+0000 7ff513400640 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 0x55d10910cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.357+0000 7ff512a00640 1 -- 172.21.15.63:0/3328109260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 msgr2=0x55d10910cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.357+0000 7ff512a00640 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 0x55d10910cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.357+0000 7ff512a00640 1 -- 172.21.15.63:0/3328109260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 msgr2=0x55d109118ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.357+0000 7ff512a00640 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 0x55d109118ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.357+0000 7ff512a00640 1 -- 172.21.15.63:0/3328109260 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d108d78210 con 0x55d108d3f4d0 2024-05-27T07:59:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.357+0000 7ff512a00640 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d3f4d0 0x55d10910d440 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7ff500011c50 tx=0x7ff50000b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60545406ff618a14 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.357+0000 7ff4fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.357+0000 7ff4fcc00640 1 -- 172.21.15.63:0/3328109260 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500003a00 con 0x55d108d3f4d0 2024-05-27T07:59:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.358+0000 7ff4fcc00640 1 -- 172.21.15.63:0/3328109260 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7ff500026450 con 0x55d108d3f4d0 2024-05-27T07:59:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.358+0000 7ff520ff0740 1 -- 172.21.15.63:0/3328109260 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5000041c0 con 0x55d108d3f4d0 2024-05-27T07:59:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.358+0000 7ff4fcc00640 1 -- 172.21.15.63:0/3328109260 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff500003ba0 con 0x55d108d3f4d0 2024-05-27T07:59:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.358+0000 7ff520ff0740 10 client.20430 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.358+0000 7ff520ff0740 1 -- 172.21.15.63:0/3328109260 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d108e44620 con 0x55d108d3f4d0 2024-05-27T07:59:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.358+0000 7ff4fcc00640 1 -- 172.21.15.63:0/3328109260 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5000041c0 con 0x55d108d3f4d0 2024-05-27T07:59:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.358+0000 7ff4fcc00640 1 -- 172.21.15.63:0/3328109260 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff50001a490 con 0x55d108d3f4d0 2024-05-27T07:59:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.359+0000 7ff520ff0740 10 client.20430 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.359+0000 7ff520ff0740 10 client.20430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.359+0000 7ff520ff0740 20 client.20430 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.359+0000 7ff520ff0740 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d108d1ed80 0x55d10911eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.359+0000 7ff520ff0740 4 client.20430 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.359+0000 7ff520ff0740 1 -- 172.21.15.63:0/3328109260 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d109075ae0 con 0x55d108d1ed80 2024-05-27T07:59:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.359+0000 7ff513400640 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d108d1ed80 0x55d10911eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.360+0000 7ff513400640 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d108d1ed80 0x55d10911eeb0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb12c84e0ff193cf server_cookie=f463c54db2d1bf5e in_seq=0 out_seq=0 2024-05-27T07:59:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.360+0000 7ff4fcc00640 10 client.20430 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.362+0000 7ff4fcc00640 1 -- 172.21.15.63:0/3328109260 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d109075ae0 con 0x55d108d1ed80 2024-05-27T07:59:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.362+0000 7ff4fcc00640 10 client.20430 handle_command_reply: tid=0 2024-05-27T07:59:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.362+0000 7ff520ff0740 1 client.20430 shutdown 2024-05-27T07:59:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 1 -- 172.21.15.63:0/3328109260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d3f4d0 msgr2=0x55d10910d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d3f4d0 0x55d10910d440 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7ff500011c50 tx=0x7ff50000b9c0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 1 -- 172.21.15.63:0/3328109260 shutdown_connections 2024-05-27T07:59:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d108d1ed80 0x55d10911eeb0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 0x55d109118ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d3f4d0 0x55d10910d440 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 1 --2- 172.21.15.63:0/3328109260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d372f0 0x55d10910cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff4fcc00640 0 client.20430 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 1 -- 172.21.15.63:0/3328109260 >> 172.21.15.63:0/3328109260 conn(0x55d108d54300 msgr2=0x55d109102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 1 -- 172.21.15.63:0/3328109260 shutdown_connections 2024-05-27T07:59:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 1 -- 172.21.15.63:0/3328109260 wait complete. 2024-05-27T07:59:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.363+0000 7ff520ff0740 20 client.20430 trim_cache size 0 max 16384 2024-05-27T07:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.381+0000 7ff520ff0740 1 Processor -- start 2024-05-27T07:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.381+0000 7ff520ff0740 1 -- start start 2024-05-27T07:59:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d372f0 0x55d1090fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d3f4d0 0x55d109106f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 0x55d109118c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff520ff0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141b1750 con 0x55d108d372f0 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff520ff0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141af020 con 0x55d108d601b0 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff520ff0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141ace90 con 0x55d108d3f4d0 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff513400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d372f0 0x55d1090fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff513400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d372f0 0x55d1090fe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57942/0 (socket says 172.21.15.63:57942) 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff513400640 1 -- 172.21.15.63:0/935782419 learned_addr learned my addr 172.21.15.63:0/935782419 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff512a00640 1 --2- 172.21.15.63:0/935782419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d3f4d0 0x55d109106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.382+0000 7ff513e00640 1 --2- 172.21.15.63:0/935782419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 0x55d109118c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.383+0000 7ff513400640 1 -- 172.21.15.63:0/935782419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d3f4d0 msgr2=0x55d109106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.383+0000 7ff513400640 1 --2- 172.21.15.63:0/935782419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d3f4d0 0x55d109106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.383+0000 7ff513400640 1 -- 172.21.15.63:0/935782419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 msgr2=0x55d109118c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.383+0000 7ff513400640 1 --2- 172.21.15.63:0/935782419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 0x55d109118c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.383+0000 7ff513400640 1 -- 172.21.15.63:0/935782419 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1090f9210 con 0x55d108d372f0 2024-05-27T07:59:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.383+0000 7ff513e00640 1 --2- 172.21.15.63:0/935782419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 0x55d109118c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.383+0000 7ff513400640 1 --2- 172.21.15.63:0/935782419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d372f0 0x55d1090fe2e0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7ff5040096c0 tx=0x7ff504041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb14eb8268aa0ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.383+0000 7ff4ff400640 1 -- 172.21.15.63:0/935782419 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504005590 con 0x55d108d372f0 2024-05-27T07:59:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.384+0000 7ff4ff400640 1 -- 172.21.15.63:0/935782419 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff504059070 con 0x55d108d372f0 2024-05-27T07:59:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.384+0000 7ff4ff400640 1 -- 172.21.15.63:0/935782419 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040536c0 con 0x55d108d372f0 2024-05-27T07:59:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.385+0000 7ff520ff0740 1 -- 172.21.15.63:0/935782419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d372f0 msgr2=0x55d1090fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.385+0000 7ff520ff0740 1 --2- 172.21.15.63:0/935782419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d372f0 0x55d1090fe2e0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7ff5040096c0 tx=0x7ff504041000 comp rx=0 tx=0).stop 2024-05-27T07:59:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.385+0000 7ff520ff0740 1 -- 172.21.15.63:0/935782419 shutdown_connections 2024-05-27T07:59:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.385+0000 7ff520ff0740 1 --2- 172.21.15.63:0/935782419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d601b0 0x55d109118c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.385+0000 7ff520ff0740 1 --2- 172.21.15.63:0/935782419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d3f4d0 0x55d109106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.385+0000 7ff520ff0740 1 --2- 172.21.15.63:0/935782419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d372f0 0x55d1090fe2e0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.385+0000 7ff520ff0740 1 -- 172.21.15.63:0/935782419 >> 172.21.15.63:0/935782419 conn(0x55d108d54300 msgr2=0x55d10910e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.385+0000 7ff520ff0740 1 -- 172.21.15.63:0/935782419 shutdown_connections 2024-05-27T07:59:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.385+0000 7ff520ff0740 1 -- 172.21.15.63:0/935782419 wait complete. 2024-05-27T07:59:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.386+0000 7ff520ff0740 1 -- start start 2024-05-27T07:59:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.386+0000 7ff520ff0740 1 Processor -- start 2024-05-27T07:59:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.386+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d372f0 0x55d109110440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.386+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d3f4d0 0x55d109110980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.386+0000 7ff520ff0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d10917f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.386+0000 7ff520ff0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141ad4e0 con 0x55d108d601b0 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.386+0000 7ff520ff0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141ae950 con 0x55d108d372f0 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.386+0000 7ff520ff0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141adbb0 con 0x55d108d3f4d0 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff520ff0740 10 client.0 _ll_register_callbacks cb 0x7ff50000bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff513e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d10917f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff513e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d10917f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57956/0 (socket says 172.21.15.63:57956) 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff513e00640 1 -- 172.21.15.63:0/4131125737 learned_addr learned my addr 172.21.15.63:0/4131125737 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff513400640 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d372f0 0x55d109110440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff512a00640 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d3f4d0 0x55d109110980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff513e00640 1 -- 172.21.15.63:0/4131125737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d3f4d0 msgr2=0x55d109110980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff513e00640 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d3f4d0 0x55d109110980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff513e00640 1 -- 172.21.15.63:0/4131125737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d372f0 msgr2=0x55d109110440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff513e00640 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d372f0 0x55d109110440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.387+0000 7ff513e00640 1 -- 172.21.15.63:0/4131125737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d10917f880 con 0x55d108d601b0 2024-05-27T07:59:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.388+0000 7ff513e00640 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d10917f340 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0a6d40 tx=0x7ff50c0b2270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430450071434025d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.388+0000 7ff4fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.388+0000 7ff4fcc00640 1 -- 172.21.15.63:0/4131125737 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0c3060 con 0x55d108d601b0 2024-05-27T07:59:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.388+0000 7ff520ff0740 1 -- 172.21.15.63:0/4131125737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff504059070 con 0x55d108d601b0 2024-05-27T07:59:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.388+0000 7ff4fcc00640 1 -- 172.21.15.63:0/4131125737 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7ff50c0a60c0 con 0x55d108d601b0 2024-05-27T07:59:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.388+0000 7ff4fcc00640 1 -- 172.21.15.63:0/4131125737 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff50c01f040 con 0x55d108d601b0 2024-05-27T07:59:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.388+0000 7ff520ff0740 10 client.20436 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.388+0000 7ff520ff0740 1 -- 172.21.15.63:0/4131125737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d109186b30 con 0x55d108d601b0 2024-05-27T07:59:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.389+0000 7ff4fcc00640 1 -- 172.21.15.63:0/4131125737 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff504059070 con 0x55d108d601b0 2024-05-27T07:59:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.389+0000 7ff4fcc00640 1 -- 172.21.15.63:0/4131125737 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff50c00f040 con 0x55d108d601b0 2024-05-27T07:59:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.389+0000 7ff520ff0740 10 client.20436 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.389+0000 7ff520ff0740 10 client.20436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.389+0000 7ff520ff0740 20 client.20436 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.389+0000 7ff520ff0740 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d108cf41c0 0x55d109189240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.389+0000 7ff520ff0740 4 client.20436 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:59:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.389+0000 7ff520ff0740 1 -- 172.21.15.63:0/4131125737 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d10906f2c0 con 0x55d108cf41c0 2024-05-27T07:59:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.390+0000 7ff513400640 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d108cf41c0 0x55d109189240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.391+0000 7ff513400640 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d108cf41c0 0x55d109189240 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dbbee03031f187d server_cookie=a1e85247af45126f in_seq=0 out_seq=0 2024-05-27T07:59:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.391+0000 7ff4fcc00640 10 client.20436 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff4fcc00640 1 -- 172.21.15.63:0/4131125737 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d10906f2c0 con 0x55d108cf41c0 2024-05-27T07:59:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff4fcc00640 10 client.20436 handle_command_reply: tid=0 2024-05-27T07:59:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff520ff0740 1 client.20436 shutdown 2024-05-27T07:59:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff520ff0740 1 -- 172.21.15.63:0/4131125737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 msgr2=0x55d10917f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff520ff0740 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d10917f340 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0a6d40 tx=0x7ff50c0b2270 comp rx=0 tx=0).stop 2024-05-27T07:59:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff520ff0740 1 -- 172.21.15.63:0/4131125737 shutdown_connections 2024-05-27T07:59:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff520ff0740 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d108cf41c0 0x55d109189240 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff520ff0740 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d108d601b0 0x55d10917f340 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff520ff0740 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d108d3f4d0 0x55d109110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff520ff0740 1 --2- 172.21.15.63:0/4131125737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d108d372f0 0x55d109110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff520ff0740 1 -- 172.21.15.63:0/4131125737 >> 172.21.15.63:0/4131125737 conn(0x55d108d54300 msgr2=0x55d109106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.418+0000 7ff4fcc00640 0 client.20436 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.419+0000 7ff520ff0740 1 -- 172.21.15.63:0/4131125737 shutdown_connections 2024-05-27T07:59:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.419+0000 7ff520ff0740 1 -- 172.21.15.63:0/4131125737 wait complete. 2024-05-27T07:59:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.419+0000 7ff520ff0740 20 client.20436 trim_cache size 0 max 16384 2024-05-27T07:59:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:59:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:59:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:59:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 -- 172.21.15.63:0/2910698029 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4f80b29d0 msgr2=0x7ff4f80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4f80b29d0 0x7ff4f80b4ee0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff50402ee10 tx=0x7ff504005fd0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 -- 172.21.15.63:0/2910698029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff514109f50 msgr2=0x7ff51419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff514109f50 0x7ff51419cc50 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00eaa0 tx=0x7ff50c00ef60 comp rx=0 tx=0).stop 2024-05-27T07:59:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 -- 172.21.15.63:0/2910698029 shutdown_connections 2024-05-27T07:59:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4f80b29d0 0x7ff4f80b4ee0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff514109f50 0x7ff51419cc50 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff514109ab0 0x7ff514198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 --2- 172.21.15.63:0/2910698029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff514104250 0x7ff514198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 -- 172.21.15.63:0/2910698029 >> 172.21.15.63:0/2910698029 conn(0x7ff514079240 msgr2=0x7ff514105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 -- 172.21.15.63:0/2910698029 shutdown_connections 2024-05-27T07:59:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:46.420+0000 7ff518a00640 1 -- 172.21.15.63:0/2910698029 wait complete. 2024-05-27T07:59:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.684+0000 7ff5aae00700 1 -- 172.21.15.63:0/3112067625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a41052a0 msgr2=0x7ff5a41056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.684+0000 7ff5aae00700 1 --2- 172.21.15.63:0/3112067625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a41052a0 0x7ff5a41056b0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7ff598009990 tx=0x7ff598009ca0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.684+0000 7ff5aae00700 1 -- 172.21.15.63:0/3112067625 shutdown_connections 2024-05-27T07:59:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.684+0000 7ff5aae00700 1 --2- 172.21.15.63:0/3112067625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a410e710 0x7ff5a4110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.684+0000 7ff5aae00700 1 --2- 172.21.15.63:0/3112067625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a41052a0 0x7ff5a41056b0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.684+0000 7ff5aae00700 1 --2- 172.21.15.63:0/3112067625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a4105bf0 0x7ff5a410e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.684+0000 7ff5aae00700 1 -- 172.21.15.63:0/3112067625 >> 172.21.15.63:0/3112067625 conn(0x7ff5a4078c30 msgr2=0x7ff5a4079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.684+0000 7ff5aae00700 1 -- 172.21.15.63:0/3112067625 shutdown_connections 2024-05-27T07:59:46.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.685+0000 7ff5aae00700 1 -- 172.21.15.63:0/3112067625 wait complete. 2024-05-27T07:59:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.686+0000 7ff5aae00700 1 Processor -- start 2024-05-27T07:59:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.686+0000 7ff5aae00700 1 -- start start 2024-05-27T07:59:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.686+0000 7ff5aae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a41052a0 0x7ff5a419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.686+0000 7ff5aae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a4105bf0 0x7ff5a419d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5aae00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a410e710 0x7ff5a41a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5aae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4114160 con 0x7ff5a41052a0 2024-05-27T07:59:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5aae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4113fe0 con 0x7ff5a4105bf0 2024-05-27T07:59:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5aae00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41142e0 con 0x7ff5a410e710 2024-05-27T07:59:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5a9a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a41052a0 0x7ff5a419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5a9a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a41052a0 0x7ff5a419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57972/0 (socket says 172.21.15.63:57972) 2024-05-27T07:59:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5a9a00700 1 -- 172.21.15.63:0/4181633660 learned_addr learned my addr 172.21.15.63:0/4181633660 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:46.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5a9000700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a4105bf0 0x7ff5a419d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5aa400700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a410e710 0x7ff5a41a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5a9a00700 1 -- 172.21.15.63:0/4181633660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a410e710 msgr2=0x7ff5a41a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5a9a00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a410e710 0x7ff5a41a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5a9a00700 1 -- 172.21.15.63:0/4181633660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a4105bf0 msgr2=0x7ff5a419d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.687+0000 7ff5a9a00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a4105bf0 0x7ff5a419d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.688+0000 7ff5a9a00700 1 -- 172.21.15.63:0/4181633660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a41a1c20 con 0x7ff5a41052a0 2024-05-27T07:59:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.688+0000 7ff5aa400700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a410e710 0x7ff5a41a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.688+0000 7ff5a9000700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a4105bf0 0x7ff5a419d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.688+0000 7ff5a9a00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a41052a0 0x7ff5a419cd50 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7ff598000c00 tx=0x7ff598005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.688+0000 7ff5a2000700 1 -- 172.21.15.63:0/4181633660 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598027070 con 0x7ff5a41052a0 2024-05-27T07:59:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.688+0000 7ff5a2000700 1 -- 172.21.15.63:0/4181633660 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff59800fb80 con 0x7ff5a41052a0 2024-05-27T07:59:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.688+0000 7ff5aae00700 1 -- 172.21.15.63:0/4181633660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a41a1eb0 con 0x7ff5a41052a0 2024-05-27T07:59:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.688+0000 7ff5a2000700 1 -- 172.21.15.63:0/4181633660 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598022c30 con 0x7ff5a41052a0 2024-05-27T07:59:46.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.689+0000 7ff5aae00700 1 -- 172.21.15.63:0/4181633660 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a41a2390 con 0x7ff5a41052a0 2024-05-27T07:59:46.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.695+0000 7ff5a2000700 1 -- 172.21.15.63:0/4181633660 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff59802a070 con 0x7ff5a41052a0 2024-05-27T07:59:46.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.695+0000 7ff5baeb2b80 1 Processor -- start 2024-05-27T07:59:46.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.695+0000 7ff5baeb2b80 1 -- start start 2024-05-27T07:59:46.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.696+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1333f59f0 0x55a133489750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.696+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a133489c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.696+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1334932c0 0x55a1334956e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.696+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a1333b6e40 con 0x55a1333f59f0 2024-05-27T07:59:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.696+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1333a69e0 con 0x55a133309e40 2024-05-27T07:59:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.696+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a1333aaa20 con 0x55a1334932c0 2024-05-27T07:59:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.696+0000 7ff5aa400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1333f59f0 0x55a133489750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.696+0000 7ff5aa400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1333f59f0 0x55a133489750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57976/0 (socket says 172.21.15.63:57976) 2024-05-27T07:59:46.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.696+0000 7ff5aa400700 1 -- 172.21.15.63:0/1906191603 learned_addr learned my addr 172.21.15.63:0/1906191603 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:46.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.697+0000 7ff5a9a00700 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a133489c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.697+0000 7ff5a9000700 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1334932c0 0x55a1334956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.697+0000 7ff5aa400700 1 -- 172.21.15.63:0/1906191603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1334932c0 msgr2=0x55a1334956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.697+0000 7ff5aa400700 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1334932c0 0x55a1334956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.697+0000 7ff5aa400700 1 -- 172.21.15.63:0/1906191603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 msgr2=0x55a133489c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.697+0000 7ff5aa400700 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a133489c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.697+0000 7ff5aa400700 1 -- 172.21.15.63:0/1906191603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a13336f420 con 0x55a1333f59f0 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.697+0000 7ff5a9000700 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1334932c0 0x55a1334956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.697+0000 7ff5a9a00700 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a133489c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.698+0000 7ff5a2000700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff58c0b29f0 0x7ff58c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.698+0000 7ff5aa400700 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1333f59f0 0x55a133489750 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009cd0 tx=0x7ff59c007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea8a3d5889900af server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.698+0000 7ff5a9a00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff58c0b29f0 0x7ff58c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.698+0000 7ff5a2000700 1 -- 172.21.15.63:0/4181633660 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7ff5980e39d0 con 0x7ff5a41052a0 2024-05-27T07:59:46.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.698+0000 7ff596a00700 1 -- 172.21.15.63:0/1906191603 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c019070 con 0x55a1333f59f0 2024-05-27T07:59:46.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.698+0000 7ff596a00700 1 -- 172.21.15.63:0/1906191603 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff59c007e40 con 0x55a1333f59f0 2024-05-27T07:59:46.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.698+0000 7ff596a00700 1 -- 172.21.15.63:0/1906191603 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c013830 con 0x55a1333f59f0 2024-05-27T07:59:46.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.700+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/1906191603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1333f59f0 msgr2=0x55a133489750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.700+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1333f59f0 0x55a133489750 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009cd0 tx=0x7ff59c007500 comp rx=0 tx=0).stop 2024-05-27T07:59:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.700+0000 7ff5a9a00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff58c0b29f0 0x7ff58c0b4f00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55a133485970 tx=0x7ff5980a6040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.700+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/1906191603 shutdown_connections 2024-05-27T07:59:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.700+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1334932c0 0x55a1334956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.700+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1333f59f0 0x55a133489750 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.700+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/1906191603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a133489c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.700+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/1906191603 >> 172.21.15.63:0/1906191603 conn(0x55a1333a3390 msgr2=0x55a133484710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.700+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/1906191603 shutdown_connections 2024-05-27T07:59:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.700+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/1906191603 wait complete. 2024-05-27T07:59:46.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.701+0000 7ff5baeb2b80 1 -- start start 2024-05-27T07:59:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.701+0000 7ff5baeb2b80 1 Processor -- start 2024-05-27T07:59:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a133309e40 0x55a133490310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1333f59f0 0x55a133490850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a133490d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a1332acfa0 con 0x55a1334932c0 2024-05-27T07:59:46.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a133347760 con 0x55a1333f59f0 2024-05-27T07:59:46.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a133495e00 con 0x55a133309e40 2024-05-27T07:59:46.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5baeb2b80 10 client.0 _ll_register_callbacks cb 0x55a133350c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:46.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5a9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1333f59f0 0x55a133490850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5a9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1333f59f0 0x55a133490850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:36902/0 (socket says 172.21.15.63:36902) 2024-05-27T07:59:46.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5a9000700 1 -- 172.21.15.63:0/2182277251 learned_addr learned my addr 172.21.15.63:0/2182277251 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:46.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.702+0000 7ff5aa400700 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a133490d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.703+0000 7ff5aa400700 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a133309e40 0x55a133490310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.703+0000 7ff5aa400700 1 -- 172.21.15.63:0/2182277251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a133309e40 msgr2=0x55a133490310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.703+0000 7ff5aa400700 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a133309e40 0x55a133490310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.703+0000 7ff5aa400700 1 -- 172.21.15.63:0/2182277251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1333f59f0 msgr2=0x55a133490850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.703+0000 7ff5aa400700 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1333f59f0 0x55a133490850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.703+0000 7ff5aa400700 1 -- 172.21.15.63:0/2182277251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1332f68d0 con 0x55a1334932c0 2024-05-27T07:59:46.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.704+0000 7ff5aa400700 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a133490d90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7ff59c015fd0 tx=0x7ff59c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf30fcd555a0e68 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.704+0000 7ff58b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:46.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.704+0000 7ff58b400700 1 -- 172.21.15.63:0/2182277251 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c019070 con 0x55a1334932c0 2024-05-27T07:59:46.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.704+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/2182277251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1333b8d90 con 0x55a1334932c0 2024-05-27T07:59:46.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.704+0000 7ff58b400700 1 -- 172.21.15.63:0/2182277251 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7ff59c014830 con 0x55a1334932c0 2024-05-27T07:59:46.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.705+0000 7ff58b400700 1 -- 172.21.15.63:0/2182277251 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff59c014c40 con 0x55a1334932c0 2024-05-27T07:59:46.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.705+0000 7ff5baeb2b80 10 client.20448 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:46.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.705+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/2182277251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a133377060 con 0x55a1334932c0 2024-05-27T07:59:46.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.705+0000 7ff58b400700 1 -- 172.21.15.63:0/2182277251 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a1333b8d90 con 0x55a1334932c0 2024-05-27T07:59:46.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.705+0000 7ff58b400700 1 -- 172.21.15.63:0/2182277251 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff59c02c030 con 0x55a1334932c0 2024-05-27T07:59:46.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.706+0000 7ff5baeb2b80 10 client.20448 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:46.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.706+0000 7ff5baeb2b80 10 client.20448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:46.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.706+0000 7ff5baeb2b80 20 client.20448 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:46.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.706+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a1334a2d40 0x55a1334a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.706+0000 7ff5baeb2b80 4 client.20448 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:46.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.706+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/2182277251 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1333b2570 con 0x55a1334a2d40 2024-05-27T07:59:46.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.707+0000 7ff5a9000700 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a1334a2d40 0x55a1334a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.708+0000 7ff5a9000700 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a1334a2d40 0x55a1334a5160 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ea4564573cb0b server_cookie=f3878e79c31b6934 in_seq=0 out_seq=0 2024-05-27T07:59:46.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.708+0000 7ff58b400700 10 client.20448 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:46.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.709+0000 7ff58b400700 1 -- 172.21.15.63:0/2182277251 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a1333b2570 con 0x55a1334a2d40 2024-05-27T07:59:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.709+0000 7ff58b400700 10 client.20448 handle_command_reply: tid=0 2024-05-27T07:59:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 client.20448 shutdown 2024-05-27T07:59:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/2182277251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 msgr2=0x55a133490d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a133490d90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7ff59c015fd0 tx=0x7ff59c00b040 comp rx=0 tx=0).stop 2024-05-27T07:59:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/2182277251 shutdown_connections 2024-05-27T07:59:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a1334a2d40 0x55a1334a5160 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a133490d90 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a133309e40 0x55a133490310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/2182277251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1333f59f0 0x55a133490850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/2182277251 >> 172.21.15.63:0/2182277251 conn(0x55a1333a3390 msgr2=0x55a133484300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff58b400700 0 client.20448 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/2182277251 shutdown_connections 2024-05-27T07:59:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.710+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/2182277251 wait complete. 2024-05-27T07:59:46.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.711+0000 7ff5baeb2b80 20 client.20448 trim_cache size 0 max 16384 2024-05-27T07:59:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.738+0000 7ff5baeb2b80 1 Processor -- start 2024-05-27T07:59:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.738+0000 7ff5baeb2b80 1 -- start start 2024-05-27T07:59:46.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.738+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a133484740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.738+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1333f59f0 0x55a133488b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.738+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a13349eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.738+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41b4e60 con 0x55a1334932c0 2024-05-27T07:59:46.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.738+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41b26f0 con 0x55a133309e40 2024-05-27T07:59:46.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.738+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41b0500 con 0x55a1333f59f0 2024-05-27T07:59:46.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.739+0000 7ff5aa400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a13349eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.739+0000 7ff5aa400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a13349eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57992/0 (socket says 172.21.15.63:57992) 2024-05-27T07:59:46.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.739+0000 7ff5aa400700 1 -- 172.21.15.63:0/201714072 learned_addr learned my addr 172.21.15.63:0/201714072 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:46.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.739+0000 7ff5a9000700 1 --2- 172.21.15.63:0/201714072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1333f59f0 0x55a133488b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.739+0000 7ff5aa400700 1 --2- 172.21.15.63:0/201714072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a133484740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.739+0000 7ff5aa400700 1 -- 172.21.15.63:0/201714072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1333f59f0 msgr2=0x55a133488b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.739+0000 7ff5aa400700 1 --2- 172.21.15.63:0/201714072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1333f59f0 0x55a133488b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.739+0000 7ff5aa400700 1 -- 172.21.15.63:0/201714072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 msgr2=0x55a133484740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.739+0000 7ff5aa400700 1 --2- 172.21.15.63:0/201714072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a133484740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.739+0000 7ff5aa400700 1 -- 172.21.15.63:0/201714072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a13347e930 con 0x55a1334932c0 2024-05-27T07:59:46.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.740+0000 7ff5aa400700 1 --2- 172.21.15.63:0/201714072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a13349eea0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x55a13344e410 tx=0x7ff59c014de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d949c163901b14 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.740+0000 7ff596000700 1 -- 172.21.15.63:0/201714072 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c019070 con 0x55a1334932c0 2024-05-27T07:59:46.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.740+0000 7ff596000700 1 -- 172.21.15.63:0/201714072 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff59c014610 con 0x55a1334932c0 2024-05-27T07:59:46.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.740+0000 7ff596000700 1 -- 172.21.15.63:0/201714072 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0133a0 con 0x55a1334932c0 2024-05-27T07:59:46.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.741+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/201714072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 msgr2=0x55a13349eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.741+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/201714072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a13349eea0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x55a13344e410 tx=0x7ff59c014de0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.742+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/201714072 shutdown_connections 2024-05-27T07:59:46.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.742+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/201714072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a13349eea0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.742+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/201714072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a133484740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.742+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/201714072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1333f59f0 0x55a133488b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.742+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/201714072 >> 172.21.15.63:0/201714072 conn(0x55a1333a3390 msgr2=0x55a13348b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.742+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/201714072 shutdown_connections 2024-05-27T07:59:46.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.742+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/201714072 wait complete. 2024-05-27T07:59:46.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.743+0000 7ff5baeb2b80 1 -- start start 2024-05-27T07:59:46.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.743+0000 7ff5baeb2b80 1 Processor -- start 2024-05-27T07:59:46.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.743+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a1335059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1333f59f0 0x55a133505ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5baeb2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a1334965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41b0bb0 con 0x55a1334932c0 2024-05-27T07:59:46.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41b2020 con 0x55a133309e40 2024-05-27T07:59:46.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5baeb2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41b1280 con 0x55a1333f59f0 2024-05-27T07:59:46.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5baeb2b80 10 client.0 _ll_register_callbacks cb 0x55a133350c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5aa400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a1334965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5aa400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a1334965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58006/0 (socket says 172.21.15.63:58006) 2024-05-27T07:59:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5aa400700 1 -- 172.21.15.63:0/3359489564 learned_addr learned my addr 172.21.15.63:0/3359489564 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5aa400700 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a1335059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.744+0000 7ff5a9000700 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1333f59f0 0x55a133505ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.745+0000 7ff5aa400700 1 -- 172.21.15.63:0/3359489564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1333f59f0 msgr2=0x55a133505ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.745+0000 7ff5aa400700 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1333f59f0 0x55a133505ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.745+0000 7ff5aa400700 1 -- 172.21.15.63:0/3359489564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 msgr2=0x55a1335059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.745+0000 7ff5aa400700 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a1335059a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.745+0000 7ff5aa400700 1 -- 172.21.15.63:0/3359489564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a133496c30 con 0x55a1334932c0 2024-05-27T07:59:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.745+0000 7ff5aa400700 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a1334965f0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x55a13349be50 tx=0x7ff59c0092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde0d20a2e7b8a9e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:46.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.745+0000 7ff58b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.745+0000 7ff58b400700 1 -- 172.21.15.63:0/3359489564 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c019040 con 0x55a1334932c0 2024-05-27T07:59:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.746+0000 7ff58b400700 1 -- 172.21.15.63:0/3359489564 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7ff59c024610 con 0x55a1334932c0 2024-05-27T07:59:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.746+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/3359489564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff59c014610 con 0x55a1334932c0 2024-05-27T07:59:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.746+0000 7ff58b400700 1 -- 172.21.15.63:0/3359489564 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff59c024810 con 0x55a1334932c0 2024-05-27T07:59:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.746+0000 7ff5baeb2b80 10 client.20454 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.746+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/3359489564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a13350d020 con 0x55a1334932c0 2024-05-27T07:59:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.746+0000 7ff58b400700 1 -- 172.21.15.63:0/3359489564 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff59c014610 con 0x55a1334932c0 2024-05-27T07:59:46.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.746+0000 7ff58b400700 1 -- 172.21.15.63:0/3359489564 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff59c02c040 con 0x55a1334932c0 2024-05-27T07:59:46.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.747+0000 7ff5baeb2b80 10 client.20454 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:46.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.747+0000 7ff5baeb2b80 10 client.20454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:46.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.747+0000 7ff5baeb2b80 20 client.20454 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:46.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.747+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a13350d3e0 0x55a13350f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:46.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.747+0000 7ff5baeb2b80 4 client.20454 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:59:46.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.747+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/3359489564 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a133229250 con 0x55a13350d3e0 2024-05-27T07:59:46.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.747+0000 7ff5a9000700 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a13350d3e0 0x55a13350f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:46.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.749+0000 7ff5a9000700 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a13350d3e0 0x55a13350f800 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157c8a16f0aa0757 server_cookie=e6cd139628512a3d in_seq=0 out_seq=0 2024-05-27T07:59:46.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.749+0000 7ff58b400700 10 client.20454 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:46.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.749+0000 7ff58b400700 1 -- 172.21.15.63:0/3359489564 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a133229250 con 0x55a13350d3e0 2024-05-27T07:59:46.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.749+0000 7ff58b400700 10 client.20454 handle_command_reply: tid=0 2024-05-27T07:59:46.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.749+0000 7ff5baeb2b80 1 client.20454 shutdown 2024-05-27T07:59:46.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.750+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/3359489564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 msgr2=0x55a1334965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.750+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a1334965f0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x55a13349be50 tx=0x7ff59c0092b0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.750+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/3359489564 shutdown_connections 2024-05-27T07:59:46.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.750+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a13350d3e0 0x55a13350f800 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.750+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a1334932c0 0x55a1334965f0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.750+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a133309e40 0x55a1335059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.750+0000 7ff5baeb2b80 1 --2- 172.21.15.63:0/3359489564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a1333f59f0 0x55a133505ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.750+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/3359489564 >> 172.21.15.63:0/3359489564 conn(0x55a1333a3390 msgr2=0x55a1334888b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.750+0000 7ff58b400700 0 client.20454 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:46.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.751+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/3359489564 shutdown_connections 2024-05-27T07:59:46.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.751+0000 7ff5baeb2b80 1 -- 172.21.15.63:0/3359489564 wait complete. 2024-05-27T07:59:46.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.751+0000 7ff5baeb2b80 20 client.20454 trim_cache size 0 max 16384 2024-05-27T07:59:46.753 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T07:59:46.753 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T07:59:46.753 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T07:59:46.753 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T07:59:46.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 -- 172.21.15.63:0/4181633660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff58c0b29f0 msgr2=0x7ff58c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff58c0b29f0 0x7ff58c0b4f00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55a133485970 tx=0x7ff5980a6040 comp rx=0 tx=0).stop 2024-05-27T07:59:46.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 -- 172.21.15.63:0/4181633660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a41052a0 msgr2=0x7ff5a419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:46.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a41052a0 0x7ff5a419cd50 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7ff598000c00 tx=0x7ff598005ef0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 -- 172.21.15.63:0/4181633660 shutdown_connections 2024-05-27T07:59:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff58c0b29f0 0x7ff58c0b4f00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a410e710 0x7ff5a41a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a41052a0 0x7ff5a419cd50 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 --2- 172.21.15.63:0/4181633660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5a4105bf0 0x7ff5a419d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 -- 172.21.15.63:0/4181633660 >> 172.21.15.63:0/4181633660 conn(0x7ff5a4078c30 msgr2=0x7ff5a4101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 -- 172.21.15.63:0/4181633660 shutdown_connections 2024-05-27T07:59:46.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:46.752+0000 7ff5aae00700 1 -- 172.21.15.63:0/4181633660 wait complete. 2024-05-27T07:59:47.707 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:59:47.708 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:59:47.786 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:47 smithi063 ceph-mon[31613]: pgmap v830: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:59:47.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:47 smithi086 ceph-mon[34687]: pgmap v830: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:59:47.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:47 smithi149 ceph-mon[35408]: pgmap v830: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T07:59:48.215 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:49.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:49 smithi063 ceph-mon[31613]: pgmap v831: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 17 op/s 2024-05-27T07:59:49.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:49 smithi086 ceph-mon[34687]: pgmap v831: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 17 op/s 2024-05-27T07:59:49.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:49 smithi149 ceph-mon[35408]: pgmap v831: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 17 op/s 2024-05-27T07:59:50.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.777+0000 7fc243000700 1 -- 172.21.15.63:0/2485847147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c109860 msgr2=0x7fc23c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:50.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.777+0000 7fc243000700 1 --2- 172.21.15.63:0/2485847147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c109860 0x7fc23c1111c0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fc23400b0d0 tx=0x7fc23400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.777+0000 7fc243000700 1 -- 172.21.15.63:0/2485847147 shutdown_connections 2024-05-27T07:59:50.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.777+0000 7fc243000700 1 --2- 172.21.15.63:0/2485847147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c109860 0x7fc23c1111c0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.777+0000 7fc243000700 1 --2- 172.21.15.63:0/2485847147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc23c1045e0 0x7fc23c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.777+0000 7fc243000700 1 --2- 172.21.15.63:0/2485847147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc23c108f10 0x7fc23c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.777+0000 7fc243000700 1 -- 172.21.15.63:0/2485847147 >> 172.21.15.63:0/2485847147 conn(0x7fc23c100030 msgr2=0x7fc23c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:50.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.777+0000 7fc243000700 1 -- 172.21.15.63:0/2485847147 shutdown_connections 2024-05-27T07:59:50.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.777+0000 7fc243000700 1 -- 172.21.15.63:0/2485847147 wait complete. 2024-05-27T07:59:50.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.779+0000 7fc243000700 1 Processor -- start 2024-05-27T07:59:50.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.779+0000 7fc243000700 1 -- start start 2024-05-27T07:59:50.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.779+0000 7fc243000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c1045e0 0x7fc23c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.779+0000 7fc243000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc23c108f10 0x7fc23c19d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.779+0000 7fc243000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc23c109860 0x7fc23c1a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.779+0000 7fc243000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c114390 con 0x7fc23c1045e0 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.779+0000 7fc243000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c114210 con 0x7fc23c108f10 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.779+0000 7fc243000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c114510 con 0x7fc23c109860 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc23be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c1045e0 0x7fc23c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc23be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c1045e0 0x7fc23c19cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44636/0 (socket says 172.21.15.63:44636) 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc23be00700 1 -- 172.21.15.63:0/4038312729 learned_addr learned my addr 172.21.15.63:0/4038312729 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc23b400700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc23c108f10 0x7fc23c19d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc241000700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc23c109860 0x7fc23c1a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc23be00700 1 -- 172.21.15.63:0/4038312729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc23c109860 msgr2=0x7fc23c1a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc23be00700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc23c109860 0x7fc23c1a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc23be00700 1 -- 172.21.15.63:0/4038312729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc23c108f10 msgr2=0x7fc23c19d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:50.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc23be00700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc23c108f10 0x7fc23c19d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc23be00700 1 -- 172.21.15.63:0/4038312729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c1a1e10 con 0x7fc23c1045e0 2024-05-27T07:59:50.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc23b400700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc23c108f10 0x7fc23c19d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:50.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.780+0000 7fc241000700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc23c109860 0x7fc23c1a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:50.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.781+0000 7fc23be00700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c1045e0 0x7fc23c19cf40 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d980 tx=0x7fc22c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:50.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.781+0000 7fc238c00700 1 -- 172.21.15.63:0/4038312729 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c016070 con 0x7fc23c1045e0 2024-05-27T07:59:50.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.781+0000 7fc238c00700 1 -- 172.21.15.63:0/4038312729 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc22c004020 con 0x7fc23c1045e0 2024-05-27T07:59:50.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.781+0000 7fc238c00700 1 -- 172.21.15.63:0/4038312729 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c004bb0 con 0x7fc23c1045e0 2024-05-27T07:59:50.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.781+0000 7fc243000700 1 -- 172.21.15.63:0/4038312729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c1a2100 con 0x7fc23c1045e0 2024-05-27T07:59:50.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.782+0000 7fc243000700 1 -- 172.21.15.63:0/4038312729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c1a9a20 con 0x7fc23c1045e0 2024-05-27T07:59:50.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.783+0000 7fc243000700 1 -- 172.21.15.63:0/4038312729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c0512b0 con 0x7fc23c1045e0 2024-05-27T07:59:50.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.787+0000 7fc238c00700 1 -- 172.21.15.63:0/4038312729 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc22c00bb60 con 0x7fc23c1045e0 2024-05-27T07:59:50.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.788+0000 7fc238c00700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2200b29f0 0x7fc2200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.789+0000 7fc238c00700 1 -- 172.21.15.63:0/4038312729 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7fc22c0e2de0 con 0x7fc23c1045e0 2024-05-27T07:59:50.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.789+0000 7fc23b400700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2200b29f0 0x7fc2200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.790+0000 7fc23b400700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2200b29f0 0x7fc2200b4f00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc228005fd0 tx=0x7fc228005db0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:50.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:50.791+0000 7fc238c00700 1 -- 172.21.15.63:0/4038312729 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc22c01f020 con 0x7fc23c1045e0 2024-05-27T07:59:51.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.114+0000 7fc243000700 1 -- 172.21.15.63:0/4038312729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc23c1a9bf0 con 0x7fc23c1045e0 2024-05-27T07:59:51.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.115+0000 7fc238c00700 1 -- 172.21.15.63:0/4038312729 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7fc22c0aa3a0 con 0x7fc23c1045e0 2024-05-27T07:59:51.117 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T07:59:51.117 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T07:59:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.119+0000 7fc243000700 1 -- 172.21.15.63:0/4038312729 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2200b29f0 msgr2=0x7fc2200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.119+0000 7fc243000700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2200b29f0 0x7fc2200b4f00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc228005fd0 tx=0x7fc228005db0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.119+0000 7fc243000700 1 -- 172.21.15.63:0/4038312729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c1045e0 msgr2=0x7fc23c19cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.119+0000 7fc243000700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c1045e0 0x7fc23c19cf40 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d980 tx=0x7fc22c00dd40 comp rx=0 tx=0).stop 2024-05-27T07:59:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.119+0000 7fc243000700 1 -- 172.21.15.63:0/4038312729 shutdown_connections 2024-05-27T07:59:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.119+0000 7fc243000700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc2200b29f0 0x7fc2200b4f00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.119+0000 7fc243000700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc23c109860 0x7fc23c1a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.119+0000 7fc243000700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c1045e0 0x7fc23c19cf40 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.119+0000 7fc243000700 1 --2- 172.21.15.63:0/4038312729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc23c108f10 0x7fc23c19d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.119+0000 7fc243000700 1 -- 172.21.15.63:0/4038312729 >> 172.21.15.63:0/4038312729 conn(0x7fc23c100030 msgr2=0x7fc23c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:51.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.120+0000 7fc243000700 1 -- 172.21.15.63:0/4038312729 shutdown_connections 2024-05-27T07:59:51.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:51.120+0000 7fc243000700 1 -- 172.21.15.63:0/4038312729 wait complete. 2024-05-27T07:59:51.122 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T07:59:51.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:51 smithi086 ceph-mon[34687]: pgmap v832: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 9 op/s 2024-05-27T07:59:51.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:51 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4038312729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:59:51.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:51 smithi149 ceph-mon[35408]: pgmap v832: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 9 op/s 2024-05-27T07:59:51.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:51 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4038312729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:59:51.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:51 smithi063 ceph-mon[31613]: pgmap v832: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 9 op/s 2024-05-27T07:59:51.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:51 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4038312729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:59:52.059 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:59:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:53.798 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:53 smithi063 ceph-mon[31613]: pgmap v833: 129 pgs: 129 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:59:53.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:53 smithi086 ceph-mon[34687]: pgmap v833: 129 pgs: 129 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:59:53.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:53 smithi149 ceph-mon[35408]: pgmap v833: 129 pgs: 129 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T07:59:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.116+0000 7fa9cac00700 1 -- 172.21.15.63:0/790859755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9c410b2d0 msgr2=0x7fa9c4117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.116+0000 7fa9cac00700 1 --2- 172.21.15.63:0/790859755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9c410b2d0 0x7fa9c4117800 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00b0a0 tx=0x7fa9bc00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.116+0000 7fa9cac00700 1 -- 172.21.15.63:0/790859755 shutdown_connections 2024-05-27T07:59:55.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.116+0000 7fa9cac00700 1 --2- 172.21.15.63:0/790859755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9c410b2d0 0x7fa9c4117800 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.116+0000 7fa9cac00700 1 --2- 172.21.15.63:0/790859755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9c4110960 0x7fa9c4110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.116+0000 7fa9cac00700 1 --2- 172.21.15.63:0/790859755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c410a980 0x7fa9c410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.116+0000 7fa9cac00700 1 -- 172.21.15.63:0/790859755 >> 172.21.15.63:0/790859755 conn(0x7fa9c4077c70 msgr2=0x7fa9c40780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.117+0000 7fa9cac00700 1 -- 172.21.15.63:0/790859755 shutdown_connections 2024-05-27T07:59:55.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.117+0000 7fa9cac00700 1 -- 172.21.15.63:0/790859755 wait complete. 2024-05-27T07:59:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.118+0000 7fa9cac00700 1 Processor -- start 2024-05-27T07:59:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.118+0000 7fa9cac00700 1 -- start start 2024-05-27T07:59:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9cac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9c410a980 0x7fa9c41a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9cac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c410b2d0 0x7fa9c41a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9cac00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9c4110960 0x7fa9c41a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9cac00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c411a980 con 0x7fa9c4110960 2024-05-27T07:59:55.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9cac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c411a800 con 0x7fa9c410b2d0 2024-05-27T07:59:55.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9cac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c411ab00 con 0x7fa9c410a980 2024-05-27T07:59:55.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9c8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9c4110960 0x7fa9c41a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9c8c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9c4110960 0x7fa9c41a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44668/0 (socket says 172.21.15.63:44668) 2024-05-27T07:59:55.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9c8c00700 1 -- 172.21.15.63:0/879148739 learned_addr learned my addr 172.21.15.63:0/879148739 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9c410a980 0x7fa9c41a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.119+0000 7fa9c3400700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c410b2d0 0x7fa9c41a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.120+0000 7fa9c8c00700 1 -- 172.21.15.63:0/879148739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9c410a980 msgr2=0x7fa9c41a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.120+0000 7fa9c8c00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9c410a980 0x7fa9c41a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.120+0000 7fa9c8c00700 1 -- 172.21.15.63:0/879148739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c410b2d0 msgr2=0x7fa9c41a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.120+0000 7fa9c8c00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c410b2d0 0x7fa9c41a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.120+0000 7fa9c8c00700 1 -- 172.21.15.63:0/879148739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c41a62e0 con 0x7fa9c4110960 2024-05-27T07:59:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.120+0000 7fa9c8c00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9c4110960 0x7fa9c41a5c40 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00b070 tx=0x7fa9bc004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.121+0000 7fa9c0c00700 1 -- 172.21.15.63:0/879148739 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc021070 con 0x7fa9c4110960 2024-05-27T07:59:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.121+0000 7fa9c0c00700 1 -- 172.21.15.63:0/879148739 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9bc007b80 con 0x7fa9c4110960 2024-05-27T07:59:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.121+0000 7fa9cac00700 1 -- 172.21.15.63:0/879148739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c41a6570 con 0x7fa9c4110960 2024-05-27T07:59:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.121+0000 7fa9c0c00700 1 -- 172.21.15.63:0/879148739 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc004640 con 0x7fa9c4110960 2024-05-27T07:59:55.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.121+0000 7fa9cac00700 1 -- 172.21.15.63:0/879148739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c41adef0 con 0x7fa9c4110960 2024-05-27T07:59:55.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.126+0000 7fa9c0c00700 1 -- 172.21.15.63:0/879148739 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa9bc003a00 con 0x7fa9c4110960 2024-05-27T07:59:55.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.127+0000 7fa9d9807b80 1 Processor -- start 2024-05-27T07:59:55.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.127+0000 7fa9d9807b80 1 -- start start 2024-05-27T07:59:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.127+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 0x55f1ef54c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef54ce70 0x55f1ef559110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f1ef47ae40 con 0x55f1ef3cde40 2024-05-27T07:59:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f1ef46a9e0 con 0x55f1ef4b99f0 2024-05-27T07:59:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f1ef46ea20 con 0x55f1ef54ce70 2024-05-27T07:59:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9c0c00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9a80b2a70 0x7fa9a80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9c3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9c3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44680/0 (socket says 172.21.15.63:44680) 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9c3e00700 1 -- 172.21.15.63:0/1696347309 learned_addr learned my addr 172.21.15.63:0/1696347309 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9c0c00700 1 -- 172.21.15.63:0/879148739 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7fa9bc00f020 con 0x7fa9c4110960 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9c8c00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9a80b2a70 0x7fa9a80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9c3400700 1 --2- 172.21.15.63:0/1696347309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 0x55f1ef54c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.128+0000 7fa9c3400700 1 --2- 172.21.15.63:0/1696347309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef54ce70 0x55f1ef559110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.129+0000 7fa9c3e00700 1 -- 172.21.15.63:0/1696347309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef54ce70 msgr2=0x55f1ef559110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.129+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/1696347309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef54ce70 0x55f1ef559110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.129+0000 7fa9c3e00700 1 -- 172.21.15.63:0/1696347309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 msgr2=0x55f1ef54c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.129+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/1696347309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 0x55f1ef54c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.129+0000 7fa9c3e00700 1 -- 172.21.15.63:0/1696347309 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1ef433420 con 0x55f1ef3cde40 2024-05-27T07:59:55.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.129+0000 7fa9c3400700 1 --2- 172.21.15.63:0/1696347309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 0x55f1ef54c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:55.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.129+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/1696347309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c870 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400bcb0 tx=0x7fa9b40103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b034556136d412f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.130+0000 7fa9c8c00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9a80b2a70 0x7fa9a80b4f80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc012000 tx=0x7fa9bc00b3b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.130+0000 7fa9b9600700 1 -- 172.21.15.63:0/1696347309 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4010d00 con 0x55f1ef3cde40 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.130+0000 7fa9b9600700 1 -- 172.21.15.63:0/1696347309 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9b40044c0 con 0x55f1ef3cde40 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.130+0000 7fa9b9600700 1 -- 172.21.15.63:0/1696347309 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4011440 con 0x55f1ef3cde40 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.131+0000 7fa9d9807b80 1 -- 172.21.15.63:0/1696347309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 msgr2=0x55f1ef54c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.131+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/1696347309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c870 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400bcb0 tx=0x7fa9b40103d0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.131+0000 7fa9d9807b80 1 -- 172.21.15.63:0/1696347309 shutdown_connections 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.131+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/1696347309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef54ce70 0x55f1ef559110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.131+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/1696347309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 0x55f1ef54c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.131+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/1696347309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c870 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.131+0000 7fa9d9807b80 1 -- 172.21.15.63:0/1696347309 >> 172.21.15.63:0/1696347309 conn(0x55f1ef467390 msgr2=0x55f1ef5484c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.131+0000 7fa9d9807b80 1 -- 172.21.15.63:0/1696347309 shutdown_connections 2024-05-27T07:59:55.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.131+0000 7fa9d9807b80 1 -- 172.21.15.63:0/1696347309 wait complete. 2024-05-27T07:59:55.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.132+0000 7fa9d9807b80 1 -- start start 2024-05-27T07:59:55.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.132+0000 7fa9d9807b80 1 Processor -- start 2024-05-27T07:59:55.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.133+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef3cde40 0x55f1ef556680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.133+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef4b99f0 0x55f1ef556bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.133+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef54ce70 0x55f1ef557100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.133+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f1ef370fa0 con 0x55f1ef4b99f0 2024-05-27T07:59:55.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.133+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f1ef40b760 con 0x55f1ef54ce70 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.133+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f1ef559b80 con 0x55f1ef3cde40 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.133+0000 7fa9d9807b80 10 client.0 _ll_register_callbacks cb 0x55f1ef414c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.133+0000 7fa9c3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef4b99f0 0x55f1ef556bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.133+0000 7fa9c3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef4b99f0 0x55f1ef556bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44686/0 (socket says 172.21.15.63:44686) 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.133+0000 7fa9c3e00700 1 -- 172.21.15.63:0/940577783 learned_addr learned my addr 172.21.15.63:0/940577783 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.134+0000 7fa9c3400700 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef3cde40 0x55f1ef556680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.134+0000 7fa9c3400700 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef54ce70 0x55f1ef557100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.134+0000 7fa9c3e00700 1 -- 172.21.15.63:0/940577783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef3cde40 msgr2=0x55f1ef556680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.134+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef3cde40 0x55f1ef556680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.134+0000 7fa9c3e00700 1 -- 172.21.15.63:0/940577783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef54ce70 msgr2=0x55f1ef557100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.134+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef54ce70 0x55f1ef557100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.134+0000 7fa9c3e00700 1 -- 172.21.15.63:0/940577783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ef3ba8d0 con 0x55f1ef4b99f0 2024-05-27T07:59:55.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.134+0000 7fa9c3400700 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef3cde40 0x55f1ef556680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:55.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.134+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef4b99f0 0x55f1ef556bc0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4000c00 tx=0x7fa9b4010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8d9b2f575bef03 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.134+0000 7fa9ae000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:55.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.135+0000 7fa9ae000700 1 -- 172.21.15.63:0/940577783 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4005440 con 0x55f1ef4b99f0 2024-05-27T07:59:55.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.135+0000 7fa9ae000700 1 -- 172.21.15.63:0/940577783 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7fa9b40055e0 con 0x55f1ef4b99f0 2024-05-27T07:59:55.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.135+0000 7fa9d9807b80 1 -- 172.21.15.63:0/940577783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b40044c0 con 0x55f1ef4b99f0 2024-05-27T07:59:55.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.135+0000 7fa9ae000700 1 -- 172.21.15.63:0/940577783 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa9b402e050 con 0x55f1ef4b99f0 2024-05-27T07:59:55.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.135+0000 7fa9d9807b80 10 client.20472 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.135+0000 7fa9d9807b80 1 -- 172.21.15.63:0/940577783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f1ef43b060 con 0x55f1ef4b99f0 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.136+0000 7fa9ae000700 1 -- 172.21.15.63:0/940577783 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9b40044c0 con 0x55f1ef4b99f0 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.136+0000 7fa9ae000700 1 -- 172.21.15.63:0/940577783 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa9b4023020 con 0x55f1ef4b99f0 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.136+0000 7fa9d9807b80 10 client.20472 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.136+0000 7fa9d9807b80 10 client.20472 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.136+0000 7fa9d9807b80 20 client.20472 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.136+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f1ef566a50 0x55f1ef568e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.136+0000 7fa9d9807b80 4 client.20472 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.136+0000 7fa9d9807b80 1 -- 172.21.15.63:0/940577783 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1ef47cd90 con 0x55f1ef566a50 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.137+0000 7fa9c3400700 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f1ef566a50 0x55f1ef568e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.138+0000 7fa9c3400700 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f1ef566a50 0x55f1ef568e70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f97be52e844800 server_cookie=73ed31dde316c354 in_seq=0 out_seq=0 2024-05-27T07:59:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.138+0000 7fa9ae000700 10 client.20472 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.140+0000 7fa9ae000700 1 -- 172.21.15.63:0/940577783 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f1ef47cd90 con 0x55f1ef566a50 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.140+0000 7fa9ae000700 10 client.20472 handle_command_reply: tid=0 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.140+0000 7fa9d9807b80 1 client.20472 shutdown 2024-05-27T07:59:55.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.140+0000 7fa9d9807b80 1 -- 172.21.15.63:0/940577783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef4b99f0 msgr2=0x55f1ef556bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.140+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef4b99f0 0x55f1ef556bc0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4000c00 tx=0x7fa9b4010de0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.141+0000 7fa9d9807b80 1 -- 172.21.15.63:0/940577783 shutdown_connections 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.141+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef54ce70 0x55f1ef557100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.141+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef3cde40 0x55f1ef556680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.141+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f1ef566a50 0x55f1ef568e70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.141+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/940577783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef4b99f0 0x55f1ef556bc0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.141+0000 7fa9d9807b80 1 -- 172.21.15.63:0/940577783 >> 172.21.15.63:0/940577783 conn(0x55f1ef467390 msgr2=0x55f1ef54c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.141+0000 7fa9ae000700 0 client.20472 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.141+0000 7fa9d9807b80 1 -- 172.21.15.63:0/940577783 shutdown_connections 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.141+0000 7fa9d9807b80 1 -- 172.21.15.63:0/940577783 wait complete. 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.141+0000 7fa9d9807b80 20 client.20472 trim_cache size 0 max 16384 2024-05-27T07:59:55.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.164+0000 7fa9d9807b80 1 Processor -- start 2024-05-27T07:59:55.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.164+0000 7fa9d9807b80 1 -- start start 2024-05-27T07:59:55.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.164+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.164+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef4b99f0 0x55f1ef551e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.164+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef54ce70 0x55f1ef550030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.164+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41b9390 con 0x55f1ef3cde40 2024-05-27T07:59:55.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.164+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41b6c20 con 0x55f1ef54ce70 2024-05-27T07:59:55.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.164+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41b4a90 con 0x55f1ef4b99f0 2024-05-27T07:59:55.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.165+0000 7fa9c3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.165+0000 7fa9c3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44702/0 (socket says 172.21.15.63:44702) 2024-05-27T07:59:55.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.165+0000 7fa9c3400700 1 -- 172.21.15.63:0/418180643 learned_addr learned my addr 172.21.15.63:0/418180643 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:55.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.165+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/418180643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef4b99f0 0x55f1ef551e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.165+0000 7fa9c3400700 1 -- 172.21.15.63:0/418180643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef4b99f0 msgr2=0x55f1ef551e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.165+0000 7fa9c3400700 1 --2- 172.21.15.63:0/418180643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef4b99f0 0x55f1ef551e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.165+0000 7fa9c3400700 1 -- 172.21.15.63:0/418180643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef54ce70 msgr2=0x55f1ef550030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.165+0000 7fa9c3400700 1 --2- 172.21.15.63:0/418180643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef54ce70 0x55f1ef550030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.165+0000 7fa9c3400700 1 -- 172.21.15.63:0/418180643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1ef549000 con 0x55f1ef3cde40 2024-05-27T07:59:55.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.166+0000 7fa9c3400700 1 --2- 172.21.15.63:0/418180643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c830 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005950 tx=0x7fa9b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69522d007ec9f7ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.166+0000 7fa9b8c00700 1 -- 172.21.15.63:0/418180643 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b00128d0 con 0x55f1ef3cde40 2024-05-27T07:59:55.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.166+0000 7fa9b8c00700 1 -- 172.21.15.63:0/418180643 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9b0003680 con 0x55f1ef3cde40 2024-05-27T07:59:55.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.166+0000 7fa9b8c00700 1 -- 172.21.15.63:0/418180643 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0020350 con 0x55f1ef3cde40 2024-05-27T07:59:55.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.167+0000 7fa9d9807b80 1 -- 172.21.15.63:0/418180643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 msgr2=0x55f1ef54c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.167+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/418180643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c830 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005950 tx=0x7fa9b00047c0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.167+0000 7fa9d9807b80 1 -- 172.21.15.63:0/418180643 shutdown_connections 2024-05-27T07:59:55.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.167+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/418180643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef54ce70 0x55f1ef550030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.167+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/418180643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef54c830 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.167+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/418180643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef4b99f0 0x55f1ef551e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.167+0000 7fa9d9807b80 1 -- 172.21.15.63:0/418180643 >> 172.21.15.63:0/418180643 conn(0x55f1ef467390 msgr2=0x55f1ef54c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.168+0000 7fa9d9807b80 1 -- 172.21.15.63:0/418180643 shutdown_connections 2024-05-27T07:59:55.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.168+0000 7fa9d9807b80 1 -- 172.21.15.63:0/418180643 wait complete. 2024-05-27T07:59:55.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.168+0000 7fa9d9807b80 1 -- start start 2024-05-27T07:59:55.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.169+0000 7fa9d9807b80 1 Processor -- start 2024-05-27T07:59:55.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.169+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef56c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.169+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 0x55f1ef56c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.169+0000 7fa9d9807b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef54ce70 0x55f1ef567650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.169+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41b50e0 con 0x55f1ef3cde40 2024-05-27T07:59:55.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.169+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41b6550 con 0x55f1ef4b99f0 2024-05-27T07:59:55.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.169+0000 7fa9d9807b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41b57b0 con 0x55f1ef54ce70 2024-05-27T07:59:55.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.170+0000 7fa9d9807b80 10 client.0 _ll_register_callbacks cb 0x55f1ef414c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:55.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.170+0000 7fa9c3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef56c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.170+0000 7fa9c3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef56c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44704/0 (socket says 172.21.15.63:44704) 2024-05-27T07:59:55.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.170+0000 7fa9c3400700 1 -- 172.21.15.63:0/4248861957 learned_addr learned my addr 172.21.15.63:0/4248861957 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:55.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.170+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 0x55f1ef56c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.170+0000 7fa9c3400700 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef54ce70 0x55f1ef567650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.171+0000 7fa9c3400700 1 -- 172.21.15.63:0/4248861957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef54ce70 msgr2=0x55f1ef567650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.171+0000 7fa9c3400700 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef54ce70 0x55f1ef567650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.171+0000 7fa9c3400700 1 -- 172.21.15.63:0/4248861957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 msgr2=0x55f1ef56c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.171+0000 7fa9c3400700 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 0x55f1ef56c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.171+0000 7fa9c3400700 1 -- 172.21.15.63:0/4248861957 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ef567df0 con 0x55f1ef3cde40 2024-05-27T07:59:55.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.171+0000 7fa9c3400700 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef56c1e0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0004790 tx=0x7fa9b0003830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba95609e5fcca93 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.171+0000 7fa9ae000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.171+0000 7fa9ae000700 1 -- 172.21.15.63:0/4248861957 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0012de0 con 0x55f1ef3cde40 2024-05-27T07:59:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.171+0000 7fa9ae000700 1 -- 172.21.15.63:0/4248861957 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7fa9b002d470 con 0x55f1ef3cde40 2024-05-27T07:59:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.171+0000 7fa9d9807b80 1 -- 172.21.15.63:0/4248861957 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b0003680 con 0x55f1ef3cde40 2024-05-27T07:59:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9ae000700 1 -- 172.21.15.63:0/4248861957 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa9b002d670 con 0x55f1ef3cde40 2024-05-27T07:59:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9d9807b80 10 client.20478 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:55.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9d9807b80 1 -- 172.21.15.63:0/4248861957 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f1ef5d0dd0 con 0x55f1ef3cde40 2024-05-27T07:59:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9ae000700 1 -- 172.21.15.63:0/4248861957 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9b0003680 con 0x55f1ef3cde40 2024-05-27T07:59:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9ae000700 1 -- 172.21.15.63:0/4248861957 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa9b00286f0 con 0x55f1ef3cde40 2024-05-27T07:59:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9d9807b80 10 client.20478 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9d9807b80 10 client.20478 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:59:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9d9807b80 20 client.20478 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f1ef5d13a0 0x55f1ef5d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9d9807b80 4 client.20478 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:59:55.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.172+0000 7fa9d9807b80 1 -- 172.21.15.63:0/4248861957 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1ef476570 con 0x55f1ef5d13a0 2024-05-27T07:59:55.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.173+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f1ef5d13a0 0x55f1ef5d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.174+0000 7fa9c3e00700 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f1ef5d13a0 0x55f1ef5d37c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb13469910b61405 server_cookie=d0a5dc57afb5de4d in_seq=0 out_seq=0 2024-05-27T07:59:55.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.174+0000 7fa9ae000700 10 client.20478 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T07:59:55.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9ae000700 1 -- 172.21.15.63:0/4248861957 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1ef476570 con 0x55f1ef5d13a0 2024-05-27T07:59:55.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9ae000700 10 client.20478 handle_command_reply: tid=0 2024-05-27T07:59:55.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 client.20478 shutdown 2024-05-27T07:59:55.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 -- 172.21.15.63:0/4248861957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 msgr2=0x55f1ef56c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef56c1e0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0004790 tx=0x7fa9b0003830 comp rx=0 tx=0).stop 2024-05-27T07:59:55.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 -- 172.21.15.63:0/4248861957 shutdown_connections 2024-05-27T07:59:55.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f1ef54ce70 0x55f1ef567650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55f1ef5d13a0 0x55f1ef5d37c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f1ef3cde40 0x55f1ef56c1e0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 --2- 172.21.15.63:0/4248861957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1ef4b99f0 0x55f1ef56c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 -- 172.21.15.63:0/4248861957 >> 172.21.15.63:0/4248861957 conn(0x55f1ef467390 msgr2=0x55f1ef557cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9ae000700 0 client.20478 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T07:59:55.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 -- 172.21.15.63:0/4248861957 shutdown_connections 2024-05-27T07:59:55.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.175+0000 7fa9d9807b80 1 -- 172.21.15.63:0/4248861957 wait complete. 2024-05-27T07:59:55.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.176+0000 7fa9d9807b80 20 client.20478 trim_cache size 0 max 16384 2024-05-27T07:59:55.179 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T07:59:55.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.176+0000 7fa9cac00700 1 -- 172.21.15.63:0/879148739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9a80b2a70 msgr2=0x7fa9a80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.176+0000 7fa9cac00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9a80b2a70 0x7fa9a80b4f80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc012000 tx=0x7fa9bc00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.176+0000 7fa9cac00700 1 -- 172.21.15.63:0/879148739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9c4110960 msgr2=0x7fa9c41a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.176+0000 7fa9cac00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9c4110960 0x7fa9c41a5c40 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00b070 tx=0x7fa9bc004d50 comp rx=0 tx=0).stop 2024-05-27T07:59:55.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.177+0000 7fa9cac00700 1 -- 172.21.15.63:0/879148739 shutdown_connections 2024-05-27T07:59:55.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.177+0000 7fa9cac00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa9a80b2a70 0x7fa9a80b4f80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.177+0000 7fa9cac00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9c4110960 0x7fa9c41a5c40 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.177+0000 7fa9cac00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9c410a980 0x7fa9c41a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.177+0000 7fa9cac00700 1 --2- 172.21.15.63:0/879148739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c410b2d0 0x7fa9c41a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.177+0000 7fa9cac00700 1 -- 172.21.15.63:0/879148739 >> 172.21.15.63:0/879148739 conn(0x7fa9c4077c70 msgr2=0x7fa9c407b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.177+0000 7fa9cac00700 1 -- 172.21.15.63:0/879148739 shutdown_connections 2024-05-27T07:59:55.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T07:59:55.177+0000 7fa9cac00700 1 -- 172.21.15.63:0/879148739 wait complete. 2024-05-27T07:59:55.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:55 smithi086 ceph-mon[34687]: pgmap v834: 129 pgs: 129 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:59:55.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:55 smithi149 ceph-mon[35408]: pgmap v834: 129 pgs: 129 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:59:55.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:55 smithi063 ceph-mon[31613]: pgmap v834: 129 pgs: 129 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:59:57.354 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:59:57.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:57 smithi063 ceph-mon[31613]: pgmap v835: 129 pgs: 129 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:59:57.834 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:57.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:57 smithi086 ceph-mon[34687]: pgmap v835: 129 pgs: 129 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:59:57.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:57 smithi149 ceph-mon[35408]: pgmap v835: 129 pgs: 129 active+clean; 7.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T07:59:59.655 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 07:59:59 smithi063 ceph-mon[31613]: pgmap v836: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:59:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.693+0000 7f0073600640 1 -- 172.21.15.63:0/4156169409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f006c1104d0 msgr2=0x7f006c1128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.693+0000 7f0073600640 1 --2- 172.21.15.63:0/4156169409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f006c1104d0 0x7f006c1128f0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f0064009ca0 tx=0x7f006402a950 comp rx=0 tx=0).stop 2024-05-27T07:59:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.693+0000 7f0073600640 1 -- 172.21.15.63:0/4156169409 shutdown_connections 2024-05-27T07:59:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.693+0000 7f0073600640 1 --2- 172.21.15.63:0/4156169409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f006c1104d0 0x7f006c1128f0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.693+0000 7f0073600640 1 --2- 172.21.15.63:0/4156169409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f006c10db70 0x7f006c10ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.693+0000 7f0073600640 1 --2- 172.21.15.63:0/4156169409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c076530 0x7f006c076940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.693+0000 7f0073600640 1 -- 172.21.15.63:0/4156169409 >> 172.21.15.63:0/4156169409 conn(0x7f006c0ff340 msgr2=0x7f006c101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.693+0000 7f0073600640 1 -- 172.21.15.63:0/4156169409 shutdown_connections 2024-05-27T07:59:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.693+0000 7f0073600640 1 -- 172.21.15.63:0/4156169409 wait complete. 2024-05-27T07:59:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.694+0000 7f0073600640 1 Processor -- start 2024-05-27T07:59:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.695+0000 7f0073600640 1 -- start start 2024-05-27T07:59:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.695+0000 7f0073600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c076530 0x7f006c1a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.695+0000 7f0073600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f006c10db70 0x7f006c1a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.695+0000 7f0073600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f006c1104d0 0x7f006c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.695+0000 7f0073600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f006c115f50 con 0x7f006c10db70 2024-05-27T07:59:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.695+0000 7f0073600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f006c115dd0 con 0x7f006c076530 2024-05-27T07:59:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.695+0000 7f0073600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1160d0 con 0x7f006c1104d0 2024-05-27T07:59:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.695+0000 7f006be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f006c10db70 0x7f006c1a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.695+0000 7f006be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f006c10db70 0x7f006c1a2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40898/0 (socket says 172.21.15.63:40898) 2024-05-27T07:59:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.695+0000 7f006be00640 1 -- 172.21.15.63:0/204759515 learned_addr learned my addr 172.21.15.63:0/204759515 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.696+0000 7f0070c00640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c076530 0x7f006c1a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.696+0000 7f0071600640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f006c1104d0 0x7f006c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.696+0000 7f006be00640 1 -- 172.21.15.63:0/204759515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f006c1104d0 msgr2=0x7f006c19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.696+0000 7f006be00640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f006c1104d0 0x7f006c19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.696+0000 7f006be00640 1 -- 172.21.15.63:0/204759515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c076530 msgr2=0x7f006c1a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.696+0000 7f006be00640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c076530 0x7f006c1a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.696+0000 7f006be00640 1 -- 172.21.15.63:0/204759515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c19cec0 con 0x7f006c10db70 2024-05-27T07:59:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.696+0000 7f006be00640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f006c10db70 0x7f006c1a2b50 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f005800e9f0 tx=0x7f005800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.697+0000 7f0069600640 1 -- 172.21.15.63:0/204759515 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005800ccc0 con 0x7f006c10db70 2024-05-27T07:59:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.697+0000 7f0069600640 1 -- 172.21.15.63:0/204759515 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00580054b0 con 0x7f006c10db70 2024-05-27T07:59:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.697+0000 7f0073600640 1 -- 172.21.15.63:0/204759515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c19d150 con 0x7f006c10db70 2024-05-27T07:59:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.697+0000 7f0069600640 1 -- 172.21.15.63:0/204759515 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058010430 con 0x7f006c10db70 2024-05-27T07:59:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.697+0000 7f0073600640 1 -- 172.21.15.63:0/204759515 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c1a93b0 con 0x7f006c10db70 2024-05-27T07:59:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.701+0000 7f007a591740 1 Processor -- start 2024-05-27T07:59:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.701+0000 7f007a591740 1 -- start start 2024-05-27T07:59:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.701+0000 7f007a591740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1ba2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.701+0000 7f007a591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 0x5560d1ba2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.701+0000 7f007a591740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17df4d0 0x5560d1ba32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.701+0000 7f007a591740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5560d1b13b90 con 0x5560d17df4d0 2024-05-27T07:59:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.701+0000 7f007a591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560d1b03730 con 0x5560d17d72f0 2024-05-27T07:59:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.701+0000 7f007a591740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5560d1b07770 con 0x5560d18001b0 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.702+0000 7f0069600640 1 -- 172.21.15.63:0/204759515 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f00580046c0 con 0x7f006c10db70 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.702+0000 7f0070c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 0x5560d1ba2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.702+0000 7f0070c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 0x5560d1ba2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:38628/0 (socket says 172.21.15.63:38628) 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.702+0000 7f0070c00640 1 -- 172.21.15.63:0/1605973863 learned_addr learned my addr 172.21.15.63:0/1605973863 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.702+0000 7f0071600640 1 --2- 172.21.15.63:0/1605973863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1ba2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.702+0000 7f006be00640 1 --2- 172.21.15.63:0/1605973863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17df4d0 0x5560d1ba32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.703+0000 7f0070c00640 1 -- 172.21.15.63:0/1605973863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 msgr2=0x5560d1ba2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.703+0000 7f0070c00640 1 --2- 172.21.15.63:0/1605973863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1ba2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.703+0000 7f0070c00640 1 -- 172.21.15.63:0/1605973863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17df4d0 msgr2=0x5560d1ba32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.703+0000 7f0070c00640 1 --2- 172.21.15.63:0/1605973863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17df4d0 0x5560d1ba32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.703+0000 7f0070c00640 1 -- 172.21.15.63:0/1605973863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d1a9c8b0 con 0x5560d17d72f0 2024-05-27T07:59:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.703+0000 7f006be00640 1 --2- 172.21.15.63:0/1605973863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17df4d0 0x5560d1ba32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.703+0000 7f0070c00640 1 --2- 172.21.15.63:0/1605973863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 0x5560d1ba2da0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0060009cf0 tx=0x7f0060011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43bfb214ad8b4384 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.704+0000 7f005dc00640 1 -- 172.21.15.63:0/1605973863 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060004230 con 0x5560d17d72f0 2024-05-27T07:59:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.704+0000 7f005dc00640 1 -- 172.21.15.63:0/1605973863 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00600043d0 con 0x5560d17d72f0 2024-05-27T07:59:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.704+0000 7f005dc00640 1 -- 172.21.15.63:0/1605973863 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060019750 con 0x5560d17d72f0 2024-05-27T07:59:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f007a591740 1 -- 172.21.15.63:0/1605973863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 msgr2=0x5560d1ba2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f007a591740 1 --2- 172.21.15.63:0/1605973863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 0x5560d1ba2da0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0060009cf0 tx=0x7f0060011810 comp rx=0 tx=0).stop 2024-05-27T07:59:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f0069600640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f00540b2950 0x7f00540b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f007a591740 1 -- 172.21.15.63:0/1605973863 shutdown_connections 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f007a591740 1 --2- 172.21.15.63:0/1605973863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17df4d0 0x5560d1ba32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f007a591740 1 --2- 172.21.15.63:0/1605973863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 0x5560d1ba2da0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f007a591740 1 --2- 172.21.15.63:0/1605973863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1ba2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f007a591740 1 -- 172.21.15.63:0/1605973863 >> 172.21.15.63:0/1605973863 conn(0x5560d17f4300 msgr2=0x5560d1b9f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f0069600640 1 -- 172.21.15.63:0/204759515 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f00580e2d70 con 0x7f006c10db70 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f0071600640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f00540b2950 0x7f00540b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f007a591740 1 -- 172.21.15.63:0/1605973863 shutdown_connections 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.705+0000 7f007a591740 1 -- 172.21.15.63:0/1605973863 wait complete. 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.706+0000 7f007a591740 1 -- start start 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.706+0000 7f0071600640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f00540b2950 0x7f00540b4e60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5560d1b9f9d0 tx=0x7f0064004a40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.706+0000 7f007a591740 1 Processor -- start 2024-05-27T07:59:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f007a591740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f007a591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17df4d0 0x5560d1bacee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f007a591740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1bb8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f007a591740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5560d17e9790 con 0x5560d17d72f0 2024-05-27T07:59:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f007a591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560d1aabbc0 con 0x5560d17df4d0 2024-05-27T07:59:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f007a591740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5560d1baf990 con 0x5560d18001b0 2024-05-27T07:59:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f007a591740 10 client.0 _ll_register_callbacks cb 0x5560d1a89400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f0070c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f0070c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bac9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40914/0 (socket says 172.21.15.63:40914) 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f0070c00640 1 -- 172.21.15.63:0/3506548602 learned_addr learned my addr 172.21.15.63:0/3506548602 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f0071600640 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1bb8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.707+0000 7f006be00640 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17df4d0 0x5560d1bacee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.708+0000 7f0070c00640 1 -- 172.21.15.63:0/3506548602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 msgr2=0x5560d1bb8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.708+0000 7f0070c00640 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1bb8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.708+0000 7f0070c00640 1 -- 172.21.15.63:0/3506548602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17df4d0 msgr2=0x5560d1bacee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.708+0000 7f0070c00640 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17df4d0 0x5560d1bacee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.708+0000 7f0070c00640 1 -- 172.21.15.63:0/3506548602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d1818210 con 0x5560d17d72f0 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.708+0000 7f0070c00640 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bac9a0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f0060003f30 tx=0x7f00600038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc6da4c9332ed9e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.708+0000 7f0052000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.708+0000 7f0052000640 1 -- 172.21.15.63:0/3506548602 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060027070 con 0x5560d17d72f0 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.709+0000 7f0052000640 1 -- 172.21.15.63:0/3506548602 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f0060005240 con 0x5560d17d72f0 2024-05-27T07:59:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.709+0000 7f007a591740 1 -- 172.21.15.63:0/3506548602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00600043d0 con 0x5560d17d72f0 2024-05-27T07:59:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.709+0000 7f0052000640 1 -- 172.21.15.63:0/3506548602 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0060033050 con 0x5560d17d72f0 2024-05-27T07:59:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.709+0000 7f007a591740 10 client.20490 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.709+0000 7f007a591740 1 -- 172.21.15.63:0/3506548602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5560d18e4620 con 0x5560d17d72f0 2024-05-27T07:59:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.709+0000 7f0052000640 1 -- 172.21.15.63:0/3506548602 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00600043d0 con 0x5560d17d72f0 2024-05-27T07:59:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.709+0000 7f0052000640 1 -- 172.21.15.63:0/3506548602 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0060004b50 con 0x5560d17d72f0 2024-05-27T07:59:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.710+0000 7f007a591740 10 client.20490 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.710+0000 7f007a591740 10 client.20490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.710+0000 7f007a591740 20 client.20490 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.710+0000 7f007a591740 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5560d17bed80 0x5560d1bbeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.710+0000 7f007a591740 4 client.20490 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.710+0000 7f007a591740 1 -- 172.21.15.63:0/3506548602 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560d1b15ae0 con 0x5560d17bed80 2024-05-27T07:59:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.710+0000 7f006be00640 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5560d17bed80 0x5560d1bbeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.711+0000 7f006be00640 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5560d17bed80 0x5560d1bbeb30 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b6c4196d2dda59e server_cookie=8fcdeeae72aca8fe in_seq=0 out_seq=0 2024-05-27T07:59:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.711+0000 7f0052000640 10 client.20490 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.713+0000 7f0052000640 1 -- 172.21.15.63:0/3506548602 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5560d1b15ae0 con 0x5560d17bed80 2024-05-27T07:59:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.713+0000 7f0052000640 10 client.20490 handle_command_reply: tid=0 2024-05-27T07:59:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 client.20490 shutdown 2024-05-27T07:59:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 -- 172.21.15.63:0/3506548602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 msgr2=0x5560d1bac9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bac9a0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f0060003f30 tx=0x7f00600038a0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 -- 172.21.15.63:0/3506548602 shutdown_connections 2024-05-27T07:59:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5560d17bed80 0x5560d1bbeb30 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1bb8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17df4d0 0x5560d1bacee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 --2- 172.21.15.63:0/3506548602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bac9a0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f0052000640 0 client.20490 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 -- 172.21.15.63:0/3506548602 >> 172.21.15.63:0/3506548602 conn(0x5560d17f4300 msgr2=0x5560d1ba2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 -- 172.21.15.63:0/3506548602 shutdown_connections 2024-05-27T07:59:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 1 -- 172.21.15.63:0/3506548602 wait complete. 2024-05-27T07:59:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.714+0000 7f007a591740 20 client.20490 trim_cache size 0 max 16384 2024-05-27T07:59:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.734+0000 7f007a591740 1 Processor -- start 2024-05-27T07:59:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.734+0000 7f007a591740 1 -- start start 2024-05-27T07:59:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.734+0000 7f007a591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 0x5560d1ba81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.734+0000 7f007a591740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d17df4d0 0x5560d1baeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.734+0000 7f007a591740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d18001b0 0x5560d1bac940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f007a591740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1b5cc0 con 0x5560d18001b0 2024-05-27T07:59:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f007a591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1b3590 con 0x5560d17d72f0 2024-05-27T07:59:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f007a591740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1b1400 con 0x5560d17df4d0 2024-05-27T07:59:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f0071600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d18001b0 0x5560d1bac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f0071600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d18001b0 0x5560d1bac940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40930/0 (socket says 172.21.15.63:40930) 2024-05-27T07:59:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f0071600640 1 -- 172.21.15.63:0/251402001 learned_addr learned my addr 172.21.15.63:0/251402001 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f006be00640 1 --2- 172.21.15.63:0/251402001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d17df4d0 0x5560d1baeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f0070c00640 1 --2- 172.21.15.63:0/251402001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 0x5560d1ba81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f0071600640 1 -- 172.21.15.63:0/251402001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d17df4d0 msgr2=0x5560d1baeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f0071600640 1 --2- 172.21.15.63:0/251402001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d17df4d0 0x5560d1baeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f0071600640 1 -- 172.21.15.63:0/251402001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 msgr2=0x5560d1ba81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f0071600640 1 --2- 172.21.15.63:0/251402001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 0x5560d1ba81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.735+0000 7f0071600640 1 -- 172.21.15.63:0/251402001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d1bbeea0 con 0x5560d18001b0 2024-05-27T07:59:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.736+0000 7f0071600640 1 --2- 172.21.15.63:0/251402001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d18001b0 0x5560d1bac940 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f0064035000 tx=0x7f0064040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad99a890cfafac2d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.736+0000 7f005d200640 1 -- 172.21.15.63:0/251402001 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064034040 con 0x5560d18001b0 2024-05-27T07:59:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.736+0000 7f005d200640 1 -- 172.21.15.63:0/251402001 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f006402fdb0 con 0x5560d18001b0 2024-05-27T07:59:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.736+0000 7f005d200640 1 -- 172.21.15.63:0/251402001 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064030440 con 0x5560d18001b0 2024-05-27T07:59:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.737+0000 7f007a591740 1 -- 172.21.15.63:0/251402001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d18001b0 msgr2=0x5560d1bac940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.737+0000 7f007a591740 1 --2- 172.21.15.63:0/251402001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d18001b0 0x5560d1bac940 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f0064035000 tx=0x7f0064040000 comp rx=0 tx=0).stop 2024-05-27T07:59:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.737+0000 7f007a591740 1 -- 172.21.15.63:0/251402001 shutdown_connections 2024-05-27T07:59:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.737+0000 7f007a591740 1 --2- 172.21.15.63:0/251402001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d18001b0 0x5560d1bac940 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.737+0000 7f007a591740 1 --2- 172.21.15.63:0/251402001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d17df4d0 0x5560d1baeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.737+0000 7f007a591740 1 --2- 172.21.15.63:0/251402001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17d72f0 0x5560d1ba81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.737+0000 7f007a591740 1 -- 172.21.15.63:0/251402001 >> 172.21.15.63:0/251402001 conn(0x5560d17f4300 msgr2=0x5560d1bac230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.737+0000 7f007a591740 1 -- 172.21.15.63:0/251402001 shutdown_connections 2024-05-27T07:59:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.737+0000 7f007a591740 1 -- 172.21.15.63:0/251402001 wait complete. 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.738+0000 7f007a591740 1 -- start start 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.738+0000 7f007a591740 1 Processor -- start 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f007a591740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bb1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f007a591740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17df4d0 0x5560d1bb2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f007a591740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1c25150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f007a591740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1b1a50 con 0x5560d17d72f0 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f007a591740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1b2ec0 con 0x5560d17df4d0 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f007a591740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1b2120 con 0x5560d18001b0 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f007a591740 10 client.0 _ll_register_callbacks cb 0x7f0060005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f0070c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bb1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f0070c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bb1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40936/0 (socket says 172.21.15.63:40936) 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f0070c00640 1 -- 172.21.15.63:0/451993231 learned_addr learned my addr 172.21.15.63:0/451993231 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T07:59:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f006be00640 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17df4d0 0x5560d1bb2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.739+0000 7f0071600640 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1c25150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.740+0000 7f0070c00640 1 -- 172.21.15.63:0/451993231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 msgr2=0x5560d1c25150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.740+0000 7f0070c00640 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1c25150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.740+0000 7f0070c00640 1 -- 172.21.15.63:0/451993231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17df4d0 msgr2=0x5560d1bb2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.740+0000 7f0070c00640 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17df4d0 0x5560d1bb2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.740+0000 7f0070c00640 1 -- 172.21.15.63:0/451993231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d1bb2c20 con 0x5560d17d72f0 2024-05-27T07:59:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.740+0000 7f0070c00640 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bb1e00 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f0060011850 tx=0x7f0060005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7a00ee87bef407 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:59.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.740+0000 7f0052000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T07:59:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.740+0000 7f0052000640 1 -- 172.21.15.63:0/451993231 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060027070 con 0x5560d17d72f0 2024-05-27T07:59:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.740+0000 7f0052000640 1 -- 172.21.15.63:0/451993231 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f0060019c00 con 0x5560d17d72f0 2024-05-27T07:59:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.740+0000 7f007a591740 1 -- 172.21.15.63:0/451993231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006402fdb0 con 0x5560d17d72f0 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f0052000640 1 -- 172.21.15.63:0/451993231 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0060009b70 con 0x5560d17d72f0 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f007a591740 10 client.20496 fetch_fsmap learned FSMap version 35 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f007a591740 1 -- 172.21.15.63:0/451993231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5560d1c26730 con 0x5560d17d72f0 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f0052000640 1 -- 172.21.15.63:0/451993231 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f006402fdb0 con 0x5560d17d72f0 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f0052000640 1 -- 172.21.15.63:0/451993231 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0060023460 con 0x5560d17d72f0 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f007a591740 10 client.20496 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f007a591740 10 client.20496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f007a591740 20 client.20496 populate_metadata read hostname 'smithi063' 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f007a591740 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5560d17941c0 0x5560d1c28e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f007a591740 4 client.20496 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.741+0000 7f007a591740 1 -- 172.21.15.63:0/451993231 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560d1b0f2c0 con 0x5560d17941c0 2024-05-27T07:59:59.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.742+0000 7f006be00640 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5560d17941c0 0x5560d1c28e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.743+0000 7f006be00640 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5560d17941c0 0x5560d1c28e30 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b921cbe3d242a0 server_cookie=fd5ba8372398668 in_seq=0 out_seq=0 2024-05-27T07:59:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.743+0000 7f0052000640 10 client.20496 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:59.769 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 07:59:59 smithi149 ceph-mon[35408]: pgmap v836: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T07:59:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.777+0000 7f0052000640 1 -- 172.21.15.63:0/451993231 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5560d1b0f2c0 con 0x5560d17941c0 2024-05-27T07:59:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.777+0000 7f0052000640 10 client.20496 handle_command_reply: tid=0 2024-05-27T07:59:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.777+0000 7f007a591740 1 client.20496 shutdown 2024-05-27T07:59:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f007a591740 1 -- 172.21.15.63:0/451993231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 msgr2=0x5560d1bb1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f007a591740 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bb1e00 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f0060011850 tx=0x7f0060005dc0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f007a591740 1 -- 172.21.15.63:0/451993231 shutdown_connections 2024-05-27T07:59:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f007a591740 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5560d17941c0 0x5560d1c28e30 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f007a591740 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560d18001b0 0x5560d1c25150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f007a591740 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d17df4d0 0x5560d1bb2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f007a591740 1 --2- 172.21.15.63:0/451993231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560d17d72f0 0x5560d1bb1e00 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f007a591740 1 -- 172.21.15.63:0/451993231 >> 172.21.15.63:0/451993231 conn(0x5560d17f4300 msgr2=0x5560d1bacc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f0052000640 0 client.20496 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T07:59:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f007a591740 1 -- 172.21.15.63:0/451993231 shutdown_connections 2024-05-27T07:59:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.778+0000 7f007a591740 1 -- 172.21.15.63:0/451993231 wait complete. 2024-05-27T07:59:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.779+0000 7f007a591740 20 client.20496 trim_cache size 0 max 16384 2024-05-27T07:59:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T07:59:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T07:59:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T07:59:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 -- 172.21.15.63:0/204759515 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f00540b2950 msgr2=0x7f00540b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f00540b2950 0x7f00540b4e60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5560d1b9f9d0 tx=0x7f0064004a40 comp rx=0 tx=0).stop 2024-05-27T07:59:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 -- 172.21.15.63:0/204759515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f006c10db70 msgr2=0x7f006c1a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f006c10db70 0x7f006c1a2b50 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f005800e9f0 tx=0x7f005800eeb0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 -- 172.21.15.63:0/204759515 shutdown_connections 2024-05-27T07:59:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f00540b2950 0x7f00540b4e60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f006c1104d0 0x7f006c19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f006c10db70 0x7f006c1a2b50 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 --2- 172.21.15.63:0/204759515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c076530 0x7f006c1a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 -- 172.21.15.63:0/204759515 >> 172.21.15.63:0/204759515 conn(0x7f006c0ff340 msgr2=0x7f006c1108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 -- 172.21.15.63:0/204759515 shutdown_connections 2024-05-27T07:59:59.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T07:59:59.780+0000 7f0073600640 1 -- 172.21.15.63:0/204759515 wait complete. 2024-05-27T07:59:59.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 07:59:59 smithi086 ceph-mon[34687]: pgmap v836: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 13 op/s 2024-05-27T08:00:00.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.279+0000 7f53ef400700 1 -- 172.21.15.63:0/3966036387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8105e90 msgr2=0x7f53e810e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:00.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.279+0000 7f53ef400700 1 --2- 172.21.15.63:0/3966036387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8105e90 0x7f53e810e3e0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009990 tx=0x7f53d4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.279+0000 7f53ef400700 1 -- 172.21.15.63:0/3966036387 shutdown_connections 2024-05-27T08:00:00.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.279+0000 7f53ef400700 1 --2- 172.21.15.63:0/3966036387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e810e920 0x7f53e8110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.279+0000 7f53ef400700 1 --2- 172.21.15.63:0/3966036387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8105540 0x7f53e8105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.279+0000 7f53ef400700 1 --2- 172.21.15.63:0/3966036387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8105e90 0x7f53e810e3e0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.279+0000 7f53ef400700 1 -- 172.21.15.63:0/3966036387 >> 172.21.15.63:0/3966036387 conn(0x7f53e8078df0 msgr2=0x7f53e8079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:00.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.279+0000 7f53ef400700 1 -- 172.21.15.63:0/3966036387 shutdown_connections 2024-05-27T08:00:00.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.279+0000 7f53ef400700 1 -- 172.21.15.63:0/3966036387 wait complete. 2024-05-27T08:00:00.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.281+0000 7f53ef400700 1 Processor -- start 2024-05-27T08:00:00.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.281+0000 7f53ef400700 1 -- start start 2024-05-27T08:00:00.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8105540 0x7f53e819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53ef400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8105e90 0x7f53e819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53ef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e810e920 0x7f53e819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53ef400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8114320 con 0x7f53e810e920 2024-05-27T08:00:00.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53ef400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81141a0 con 0x7f53e8105540 2024-05-27T08:00:00.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53ef400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81144a0 con 0x7f53e8105e90 2024-05-27T08:00:00.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e810e920 0x7f53e819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e810e920 0x7f53e819f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40962/0 (socket says 172.21.15.63:40962) 2024-05-27T08:00:00.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53ed400700 1 -- 172.21.15.63:0/2177417868 learned_addr learned my addr 172.21.15.63:0/2177417868 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:00.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53eca00700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8105540 0x7f53e819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.282+0000 7f53e7e00700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8105e90 0x7f53e819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.283+0000 7f53ed400700 1 -- 172.21.15.63:0/2177417868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8105e90 msgr2=0x7f53e819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:00.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.283+0000 7f53ed400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8105e90 0x7f53e819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.283+0000 7f53ed400700 1 -- 172.21.15.63:0/2177417868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8105540 msgr2=0x7f53e819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:00.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.283+0000 7f53ed400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8105540 0x7f53e819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.283+0000 7f53ed400700 1 -- 172.21.15.63:0/2177417868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e819fc60 con 0x7f53e810e920 2024-05-27T08:00:00.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.283+0000 7f53ed400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e810e920 0x7f53e819f5c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f53e0007fa0 tx=0x7f53e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:00.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.284+0000 7f53e5600700 1 -- 172.21.15.63:0/2177417868 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0004bb0 con 0x7f53e810e920 2024-05-27T08:00:00.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.284+0000 7f53e5600700 1 -- 172.21.15.63:0/2177417868 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53e0004d50 con 0x7f53e810e920 2024-05-27T08:00:00.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.284+0000 7f53e5600700 1 -- 172.21.15.63:0/2177417868 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0005940 con 0x7f53e810e920 2024-05-27T08:00:00.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.284+0000 7f53ef400700 1 -- 172.21.15.63:0/2177417868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e819ff50 con 0x7f53e810e920 2024-05-27T08:00:00.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.284+0000 7f53ef400700 1 -- 172.21.15.63:0/2177417868 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e81a7870 con 0x7f53e810e920 2024-05-27T08:00:00.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.289+0000 7f53e5600700 1 -- 172.21.15.63:0/2177417868 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f53e001a020 con 0x7f53e810e920 2024-05-27T08:00:00.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.289+0000 7f53fdf74b80 1 Processor -- start 2024-05-27T08:00:00.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.290+0000 7f53fdf74b80 1 -- start start 2024-05-27T08:00:00.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.290+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0c2ae40 0x5587e0daf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.290+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0dafd30 0x5587e0da9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.290+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0daa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.290+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5587e0cd7e40 con 0x5587e0da9db0 2024-05-27T08:00:00.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.290+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5587e0cc79e0 con 0x5587e0dafd30 2024-05-27T08:00:00.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.290+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5587e0ccba20 con 0x5587e0c2ae40 2024-05-27T08:00:00.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.291+0000 7f53e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0daa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.291+0000 7f53e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0daa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40974/0 (socket says 172.21.15.63:40974) 2024-05-27T08:00:00.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.291+0000 7f53e7e00700 1 -- 172.21.15.63:0/865221363 learned_addr learned my addr 172.21.15.63:0/865221363 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:00.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.291+0000 7f53eca00700 1 --2- 172.21.15.63:0/865221363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0dafd30 0x5587e0da9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.291+0000 7f53e7e00700 1 --2- 172.21.15.63:0/865221363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0c2ae40 0x5587e0daf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.291+0000 7f53e5600700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 0x7f53d00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.292+0000 7f53e7e00700 1 -- 172.21.15.63:0/865221363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0c2ae40 msgr2=0x5587e0daf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:00.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.292+0000 7f53e7e00700 1 --2- 172.21.15.63:0/865221363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0c2ae40 0x5587e0daf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.292+0000 7f53e7e00700 1 -- 172.21.15.63:0/865221363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0dafd30 msgr2=0x5587e0da9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:00.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.292+0000 7f53e7e00700 1 --2- 172.21.15.63:0/865221363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0dafd30 0x5587e0da9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.292+0000 7f53e7e00700 1 -- 172.21.15.63:0/865221363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e0c90420 con 0x5587e0da9db0 2024-05-27T08:00:00.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.292+0000 7f53e5600700 1 -- 172.21.15.63:0/2177417868 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f53e00e20d0 con 0x7f53e810e920 2024-05-27T08:00:00.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.292+0000 7f53ed400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 0x7f53d00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.292+0000 7f53e7e00700 1 --2- 172.21.15.63:0/865221363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0daa1c0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f53d4000c00 tx=0x7f53d401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3c855097bb954 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:00.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.292+0000 7f53da000700 1 -- 172.21.15.63:0/865221363 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d400bbb0 con 0x5587e0da9db0 2024-05-27T08:00:00.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.293+0000 7f53da000700 1 -- 172.21.15.63:0/865221363 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53d400bd50 con 0x5587e0da9db0 2024-05-27T08:00:00.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.293+0000 7f53da000700 1 -- 172.21.15.63:0/865221363 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4003d10 con 0x5587e0da9db0 2024-05-27T08:00:00.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.294+0000 7f53fdf74b80 1 -- 172.21.15.63:0/865221363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 msgr2=0x5587e0daa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:00.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.294+0000 7f53ed400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 0x7f53d00b4f00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f53e0019ff0 tx=0x7f53e000b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:00.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.294+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/865221363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0daa1c0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f53d4000c00 tx=0x7f53d401a040 comp rx=0 tx=0).stop 2024-05-27T08:00:00.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.294+0000 7f53fdf74b80 1 -- 172.21.15.63:0/865221363 shutdown_connections 2024-05-27T08:00:00.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.294+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/865221363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0daa1c0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.294+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/865221363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0c2ae40 0x5587e0daf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.294+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/865221363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0dafd30 0x5587e0da9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.294+0000 7f53fdf74b80 1 -- 172.21.15.63:0/865221363 >> 172.21.15.63:0/865221363 conn(0x5587e0c8afc0 msgr2=0x5587e0da48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:00.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.294+0000 7f53fdf74b80 1 -- 172.21.15.63:0/865221363 shutdown_connections 2024-05-27T08:00:00.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.294+0000 7f53fdf74b80 1 -- 172.21.15.63:0/865221363 wait complete. 2024-05-27T08:00:00.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.295+0000 7f53fdf74b80 1 -- start start 2024-05-27T08:00:00.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.295+0000 7f53fdf74b80 1 Processor -- start 2024-05-27T08:00:00.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0c2ae40 0x5587e0dbd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 0x5587e0dbdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0dafd30 0x5587e0db7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5587e0bcdfa0 con 0x5587e0dafd30 2024-05-27T08:00:00.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5587e0c68760 con 0x5587e0da9db0 2024-05-27T08:00:00.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5587e0db67a0 con 0x5587e0c2ae40 2024-05-27T08:00:00.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53fdf74b80 10 client.0 _ll_register_callbacks cb 0x5587e0c71c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:00.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0dafd30 0x5587e0db7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0dafd30 0x5587e0db7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40978/0 (socket says 172.21.15.63:40978) 2024-05-27T08:00:00.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53e7e00700 1 -- 172.21.15.63:0/533499709 learned_addr learned my addr 172.21.15.63:0/533499709 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:00.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53eca00700 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 0x5587e0dbdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.296+0000 7f53e7e00700 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0c2ae40 0x5587e0dbd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.297+0000 7f53e7e00700 1 -- 172.21.15.63:0/533499709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0c2ae40 msgr2=0x5587e0dbd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:00.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.297+0000 7f53e7e00700 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0c2ae40 0x5587e0dbd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.297+0000 7f53e7e00700 1 -- 172.21.15.63:0/533499709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 msgr2=0x5587e0dbdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:00.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.297+0000 7f53e7e00700 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 0x5587e0dbdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.297+0000 7f53e7e00700 1 -- 172.21.15.63:0/533499709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e0c178d0 con 0x5587e0dafd30 2024-05-27T08:00:00.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.297+0000 7f53e7e00700 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0dafd30 0x5587e0db7dd0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f53d4006010 tx=0x7f53d40038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa840902daeb365 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:00.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.297+0000 7f53cea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:00.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.298+0000 7f53cea00700 1 -- 172.21.15.63:0/533499709 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d400bbb0 con 0x5587e0dafd30 2024-05-27T08:00:00.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.298+0000 7f53fdf74b80 1 -- 172.21.15.63:0/533499709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d400bd50 con 0x5587e0dafd30 2024-05-27T08:00:00.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.298+0000 7f53cea00700 1 -- 172.21.15.63:0/533499709 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f53d400fb70 con 0x5587e0dafd30 2024-05-27T08:00:00.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.298+0000 7f53cea00700 1 -- 172.21.15.63:0/533499709 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f53d4040050 con 0x5587e0dafd30 2024-05-27T08:00:00.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.298+0000 7f53fdf74b80 10 client.20508 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:00.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.298+0000 7f53fdf74b80 1 -- 172.21.15.63:0/533499709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5587e0c98060 con 0x5587e0dafd30 2024-05-27T08:00:00.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.298+0000 7f53cea00700 1 -- 172.21.15.63:0/533499709 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53d400bd50 con 0x5587e0dafd30 2024-05-27T08:00:00.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.299+0000 7f53cea00700 1 -- 172.21.15.63:0/533499709 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f53d4032040 con 0x5587e0dafd30 2024-05-27T08:00:00.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.299+0000 7f53fdf74b80 10 client.20508 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:00.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.299+0000 7f53fdf74b80 10 client.20508 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:00:00.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.299+0000 7f53fdf74b80 20 client.20508 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:00.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.299+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5587e0dc78f0 0x5587e0dc9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.299+0000 7f53fdf74b80 4 client.20508 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:00.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.299+0000 7f53fdf74b80 1 -- 172.21.15.63:0/533499709 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587e0cd9d90 con 0x5587e0dc78f0 2024-05-27T08:00:00.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.300+0000 7f53eca00700 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5587e0dc78f0 0x5587e0dc9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.301+0000 7f53eca00700 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5587e0dc78f0 0x5587e0dc9d10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52229f3ff5812359 server_cookie=1f81b662fbd3c396 in_seq=0 out_seq=0 2024-05-27T08:00:00.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.301+0000 7f53cea00700 10 client.20508 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:00:00.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53cea00700 1 -- 172.21.15.63:0/533499709 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5587e0cd9d90 con 0x5587e0dc78f0 2024-05-27T08:00:00.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53cea00700 10 client.20508 handle_command_reply: tid=0 2024-05-27T08:00:00.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 client.20508 shutdown 2024-05-27T08:00:00.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 -- 172.21.15.63:0/533499709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0dafd30 msgr2=0x5587e0db7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:00.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0dafd30 0x5587e0db7dd0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f53d4006010 tx=0x7f53d40038f0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 -- 172.21.15.63:0/533499709 shutdown_connections 2024-05-27T08:00:00.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5587e0dc78f0 0x5587e0dc9d10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0dafd30 0x5587e0db7dd0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0c2ae40 0x5587e0dbd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/533499709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 0x5587e0dbdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 -- 172.21.15.63:0/533499709 >> 172.21.15.63:0/533499709 conn(0x5587e0c8afc0 msgr2=0x5587e0da4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:00.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53cea00700 0 client.20508 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:00:00.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 -- 172.21.15.63:0/533499709 shutdown_connections 2024-05-27T08:00:00.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.303+0000 7f53fdf74b80 1 -- 172.21.15.63:0/533499709 wait complete. 2024-05-27T08:00:00.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.304+0000 7f53fdf74b80 20 client.20508 trim_cache size 0 max 16384 2024-05-27T08:00:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.323+0000 7f53fdf74b80 1 Processor -- start 2024-05-27T08:00:00.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.323+0000 7f53fdf74b80 1 -- start start 2024-05-27T08:00:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.323+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0c2ae40 0x5587e0da9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.323+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 0x5587e0db58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.323+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0dafd30 0x5587e0db5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.323+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81b0ac0 con 0x5587e0c2ae40 2024-05-27T08:00:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.323+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81ae350 con 0x5587e0da9db0 2024-05-27T08:00:00.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.323+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81ac1c0 con 0x5587e0dafd30 2024-05-27T08:00:00.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.324+0000 7f53e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0c2ae40 0x5587e0da9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.324+0000 7f53e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0c2ae40 0x5587e0da9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40986/0 (socket says 172.21.15.63:40986) 2024-05-27T08:00:00.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.324+0000 7f53e7e00700 1 -- 172.21.15.63:0/2573207099 learned_addr learned my addr 172.21.15.63:0/2573207099 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:00.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.324+0000 7f53eca00700 1 --2- 172.21.15.63:0/2573207099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 0x5587e0db58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.324+0000 7f53e7e00700 1 -- 172.21.15.63:0/2573207099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0dafd30 msgr2=0x5587e0db5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.324+0000 7f53e7e00700 1 --2- 172.21.15.63:0/2573207099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0dafd30 0x5587e0db5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.324+0000 7f53e7e00700 1 -- 172.21.15.63:0/2573207099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 msgr2=0x5587e0db58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.324+0000 7f53e7e00700 1 --2- 172.21.15.63:0/2573207099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 0x5587e0db58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.324+0000 7f53e7e00700 1 -- 172.21.15.63:0/2573207099 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e0db6320 con 0x5587e0c2ae40 2024-05-27T08:00:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.325+0000 7f53eca00700 1 --2- 172.21.15.63:0/2573207099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 0x5587e0db58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.325+0000 7f53e7e00700 1 --2- 172.21.15.63:0/2573207099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0c2ae40 0x5587e0da9870 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009540 tx=0x7f53d4005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e41493da75a469c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.325+0000 7f53d9600700 1 -- 172.21.15.63:0/2573207099 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4005110 con 0x5587e0c2ae40 2024-05-27T08:00:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.325+0000 7f53d9600700 1 -- 172.21.15.63:0/2573207099 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53d40052b0 con 0x5587e0c2ae40 2024-05-27T08:00:00.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.325+0000 7f53d9600700 1 -- 172.21.15.63:0/2573207099 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4003c50 con 0x5587e0c2ae40 2024-05-27T08:00:00.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.326+0000 7f53fdf74b80 1 -- 172.21.15.63:0/2573207099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0c2ae40 msgr2=0x5587e0da9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:00.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.326+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/2573207099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0c2ae40 0x5587e0da9870 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009540 tx=0x7f53d4005ab0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.326+0000 7f53fdf74b80 1 -- 172.21.15.63:0/2573207099 shutdown_connections 2024-05-27T08:00:00.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.326+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/2573207099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0dafd30 0x5587e0db5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.326+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/2573207099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0c2ae40 0x5587e0da9870 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.326+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/2573207099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0da9db0 0x5587e0db58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.327+0000 7f53fdf74b80 1 -- 172.21.15.63:0/2573207099 >> 172.21.15.63:0/2573207099 conn(0x5587e0c8afc0 msgr2=0x5587e0daec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:00.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.327+0000 7f53fdf74b80 1 -- 172.21.15.63:0/2573207099 shutdown_connections 2024-05-27T08:00:00.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.327+0000 7f53fdf74b80 1 -- 172.21.15.63:0/2573207099 wait complete. 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.327+0000 7f53fdf74b80 1 -- start start 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.328+0000 7f53fdf74b80 1 Processor -- start 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.328+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0c2ae40 0x5587e0dc4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.328+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0dc0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.328+0000 7f53fdf74b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0dafd30 0x5587e0dc0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.328+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81ac810 con 0x5587e0da9db0 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.328+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81adc80 con 0x5587e0c2ae40 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.328+0000 7f53fdf74b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81acee0 con 0x5587e0dafd30 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.328+0000 7f53fdf74b80 10 client.0 _ll_register_callbacks cb 0x5587e0c71c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53eca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0dc0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53eca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0dc0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40992/0 (socket says 172.21.15.63:40992) 2024-05-27T08:00:00.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53eca00700 1 -- 172.21.15.63:0/1675814498 learned_addr learned my addr 172.21.15.63:0/1675814498 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:00.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53e7e00700 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0c2ae40 0x5587e0dc4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53e7e00700 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0dafd30 0x5587e0dc0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53eca00700 1 -- 172.21.15.63:0/1675814498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0dafd30 msgr2=0x5587e0dc0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53eca00700 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0dafd30 0x5587e0dc0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53eca00700 1 -- 172.21.15.63:0/1675814498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0c2ae40 msgr2=0x5587e0dc4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53eca00700 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0c2ae40 0x5587e0dc4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53eca00700 1 -- 172.21.15.63:0/1675814498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e0dc14b0 con 0x5587e0da9db0 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.329+0000 7f53e7e00700 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0dafd30 0x5587e0dc0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.330+0000 7f53eca00700 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0dc0690 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x5587e0cd7f00 tx=0x7f53dc00bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbf90d0947df35b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.330+0000 7f53cea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.330+0000 7f53cea00700 1 -- 172.21.15.63:0/1675814498 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc01d480 con 0x5587e0da9db0 2024-05-27T08:00:00.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.330+0000 7f53cea00700 1 -- 172.21.15.63:0/1675814498 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f53dc01d620 con 0x5587e0da9db0 2024-05-27T08:00:00.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.330+0000 7f53fdf74b80 1 -- 172.21.15.63:0/1675814498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53d40052b0 con 0x5587e0da9db0 2024-05-27T08:00:00.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.331+0000 7f53cea00700 1 -- 172.21.15.63:0/1675814498 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f53dc030050 con 0x5587e0da9db0 2024-05-27T08:00:00.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.331+0000 7f53fdf74b80 10 client.20514 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:00.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.331+0000 7f53fdf74b80 1 -- 172.21.15.63:0/1675814498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5587e0e31270 con 0x5587e0da9db0 2024-05-27T08:00:00.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.331+0000 7f53cea00700 1 -- 172.21.15.63:0/1675814498 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53d40052b0 con 0x5587e0da9db0 2024-05-27T08:00:00.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.331+0000 7f53cea00700 1 -- 172.21.15.63:0/1675814498 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f53dc026030 con 0x5587e0da9db0 2024-05-27T08:00:00.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.331+0000 7f53fdf74b80 10 client.20514 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:00.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.331+0000 7f53fdf74b80 10 client.20514 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:00:00.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.331+0000 7f53fdf74b80 20 client.20514 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:00.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.331+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5587e0e31960 0x5587e0e33d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:00.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.332+0000 7f53fdf74b80 4 client.20514 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:00:00.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.332+0000 7f53fdf74b80 1 -- 172.21.15.63:0/1675814498 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587e0cd3570 con 0x5587e0e31960 2024-05-27T08:00:00.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.332+0000 7f53e7e00700 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5587e0e31960 0x5587e0e33d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:00.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.333+0000 7f53e7e00700 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5587e0e31960 0x5587e0e33d80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e516f513f80fd0e8 server_cookie=611464bbff72fe2b in_seq=0 out_seq=0 2024-05-27T08:00:00.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.333+0000 7f53cea00700 10 client.20514 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:00:00.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53cea00700 1 -- 172.21.15.63:0/1675814498 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587e0cd3570 con 0x5587e0e31960 2024-05-27T08:00:00.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53cea00700 10 client.20514 handle_command_reply: tid=0 2024-05-27T08:00:00.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53fdf74b80 1 client.20514 shutdown 2024-05-27T08:00:00.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53fdf74b80 1 -- 172.21.15.63:0/1675814498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 msgr2=0x5587e0dc0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:00.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0dc0690 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x5587e0cd7f00 tx=0x7f53dc00bd90 comp rx=0 tx=0).stop 2024-05-27T08:00:00.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53fdf74b80 1 -- 172.21.15.63:0/1675814498 shutdown_connections 2024-05-27T08:00:00.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587e0dafd30 0x5587e0dc0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587e0c2ae40 0x5587e0dc4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5587e0e31960 0x5587e0e33d80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53fdf74b80 1 --2- 172.21.15.63:0/1675814498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5587e0da9db0 0x5587e0dc0690 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53cea00700 0 client.20514 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:00:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.334+0000 7f53fdf74b80 1 -- 172.21.15.63:0/1675814498 >> 172.21.15.63:0/1675814498 conn(0x5587e0c8afc0 msgr2=0x5587e0da3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.335+0000 7f53fdf74b80 1 -- 172.21.15.63:0/1675814498 shutdown_connections 2024-05-27T08:00:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.335+0000 7f53fdf74b80 1 -- 172.21.15.63:0/1675814498 wait complete. 2024-05-27T08:00:00.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.335+0000 7f53fdf74b80 20 client.20514 trim_cache size 0 max 16384 2024-05-27T08:00:00.338 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:00:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 -- 172.21.15.63:0/2177417868 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 msgr2=0x7f53d00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 0x7f53d00b4f00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f53e0019ff0 tx=0x7f53e000b040 comp rx=0 tx=0).stop 2024-05-27T08:00:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 -- 172.21.15.63:0/2177417868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e810e920 msgr2=0x7f53e819f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e810e920 0x7f53e819f5c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f53e0007fa0 tx=0x7f53e000d3b0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 -- 172.21.15.63:0/2177417868 shutdown_connections 2024-05-27T08:00:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 0x7f53d00b4f00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e810e920 0x7f53e819f5c0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8105540 0x7f53e819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 --2- 172.21.15.63:0/2177417868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8105e90 0x7f53e819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:00.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 -- 172.21.15.63:0/2177417868 >> 172.21.15.63:0/2177417868 conn(0x7f53e8078df0 msgr2=0x7f53e81012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:00.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 -- 172.21.15.63:0/2177417868 shutdown_connections 2024-05-27T08:00:00.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:00.336+0000 7f53ef400700 1 -- 172.21.15.63:0/2177417868 wait complete. 2024-05-27T08:00:00.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:00 smithi063 ceph-mon[31613]: overall HEALTH_OK 2024-05-27T08:00:00.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:00 smithi086 ceph-mon[34687]: overall HEALTH_OK 2024-05-27T08:00:00.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:00 smithi149 ceph-mon[35408]: overall HEALTH_OK 2024-05-27T08:00:01.280 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:00:01.780 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:00:01.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:01 smithi063 ceph-mon[31613]: pgmap v837: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T08:00:01.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:01 smithi086 ceph-mon[34687]: pgmap v837: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T08:00:01.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:01 smithi149 ceph-mon[35408]: pgmap v837: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T08:00:03.880 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:03 smithi063 ceph-mon[31613]: pgmap v838: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T08:00:03.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:03 smithi086 ceph-mon[34687]: pgmap v838: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T08:00:03.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:03 smithi149 ceph-mon[35408]: pgmap v838: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T08:00:04.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.366+0000 7f65c7e00700 1 -- 172.21.15.63:0/1431666990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c810e710 msgr2=0x7f65c8110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.366+0000 7f65c7e00700 1 --2- 172.21.15.63:0/1431666990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c810e710 0x7f65c8110bf0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f65c000b0a0 tx=0x7f65c000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.367+0000 7f65c7e00700 1 -- 172.21.15.63:0/1431666990 shutdown_connections 2024-05-27T08:00:04.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.367+0000 7f65c7e00700 1 --2- 172.21.15.63:0/1431666990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c810e710 0x7f65c8110bf0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.367+0000 7f65c7e00700 1 --2- 172.21.15.63:0/1431666990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c81052a0 0x7f65c81056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.367+0000 7f65c7e00700 1 --2- 172.21.15.63:0/1431666990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c8105bf0 0x7f65c810e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.367+0000 7f65c7e00700 1 -- 172.21.15.63:0/1431666990 >> 172.21.15.63:0/1431666990 conn(0x7f65c8078c30 msgr2=0x7f65c8079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.367+0000 7f65c7e00700 1 -- 172.21.15.63:0/1431666990 shutdown_connections 2024-05-27T08:00:04.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.367+0000 7f65c7e00700 1 -- 172.21.15.63:0/1431666990 wait complete. 2024-05-27T08:00:04.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.368+0000 7f65c7e00700 1 Processor -- start 2024-05-27T08:00:04.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.369+0000 7f65c7e00700 1 -- start start 2024-05-27T08:00:04.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.369+0000 7f65c7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c81052a0 0x7f65c819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.369+0000 7f65c7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c8105bf0 0x7f65c819d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.369+0000 7f65c7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c810e710 0x7f65c81a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.369+0000 7f65c7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65c8114170 con 0x7f65c810e710 2024-05-27T08:00:04.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.369+0000 7f65c7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65c8113ff0 con 0x7f65c81052a0 2024-05-27T08:00:04.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.369+0000 7f65c7e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81142f0 con 0x7f65c8105bf0 2024-05-27T08:00:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.370+0000 7f65c7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c810e710 0x7f65c81a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.370+0000 7f65c7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c810e710 0x7f65c81a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41016/0 (socket says 172.21.15.63:41016) 2024-05-27T08:00:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.370+0000 7f65c7400700 1 -- 172.21.15.63:0/1072320394 learned_addr learned my addr 172.21.15.63:0/1072320394 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.370+0000 7f65c6a00700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c81052a0 0x7f65c819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.370+0000 7f65c6000700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c8105bf0 0x7f65c819d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.370+0000 7f65c7400700 1 -- 172.21.15.63:0/1072320394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c8105bf0 msgr2=0x7f65c819d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.370+0000 7f65c7400700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c8105bf0 0x7f65c819d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.370+0000 7f65c7400700 1 -- 172.21.15.63:0/1072320394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c81052a0 msgr2=0x7f65c819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.371+0000 7f65c7400700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c81052a0 0x7f65c819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.371+0000 7f65c7400700 1 -- 172.21.15.63:0/1072320394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c81a1bf0 con 0x7f65c810e710 2024-05-27T08:00:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.371+0000 7f65c7400700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c810e710 0x7f65c81a1550 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f65c000b070 tx=0x7f65c0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.371+0000 7f65bf400700 1 -- 172.21.15.63:0/1072320394 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0021070 con 0x7f65c810e710 2024-05-27T08:00:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.372+0000 7f65c7e00700 1 -- 172.21.15.63:0/1072320394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c81a1e80 con 0x7f65c810e710 2024-05-27T08:00:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.372+0000 7f65bf400700 1 -- 172.21.15.63:0/1072320394 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65c0007b80 con 0x7f65c810e710 2024-05-27T08:00:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.372+0000 7f65bf400700 1 -- 172.21.15.63:0/1072320394 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0004650 con 0x7f65c810e710 2024-05-27T08:00:04.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.372+0000 7f65c7e00700 1 -- 172.21.15.63:0/1072320394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c81a9800 con 0x7f65c810e710 2024-05-27T08:00:04.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.376+0000 7f65bf400700 1 -- 172.21.15.63:0/1072320394 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f65c0003a00 con 0x7f65c810e710 2024-05-27T08:00:04.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.377+0000 7f65dc736b80 1 Processor -- start 2024-05-27T08:00:04.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.377+0000 7f65dc736b80 1 -- start start 2024-05-27T08:00:04.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.378+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be7be9f0 0x5642be852750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.378+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be6d2e40 0x5642be852c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.378+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 0x5642be85e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.378+0000 7f65dc736b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5642be77fe40 con 0x5642be85c2c0 2024-05-27T08:00:04.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.378+0000 7f65dc736b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642be76f9e0 con 0x5642be6d2e40 2024-05-27T08:00:04.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.378+0000 7f65dc736b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5642be773a20 con 0x5642be7be9f0 2024-05-27T08:00:04.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.378+0000 7f65c6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 0x5642be85e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.378+0000 7f65c6000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be7be9f0 0x5642be852750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65c6000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be7be9f0 0x5642be852750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53614/0 (socket says 172.21.15.63:53614) 2024-05-27T08:00:04.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65c6000700 1 -- 172.21.15.63:0/1213588140 learned_addr learned my addr 172.21.15.63:0/1213588140 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:04.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65bf400700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65ac0b2a70 0x7f65ac0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65c6a00700 1 --2- 172.21.15.63:0/1213588140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be6d2e40 0x5642be852c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65bf400700 1 -- 172.21.15.63:0/1072320394 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f65c000f020 con 0x7f65c810e710 2024-05-27T08:00:04.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65c7400700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65ac0b2a70 0x7f65ac0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65c6000700 1 -- 172.21.15.63:0/1213588140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be6d2e40 msgr2=0x5642be852c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65c6000700 1 --2- 172.21.15.63:0/1213588140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be6d2e40 0x5642be852c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65c6000700 1 -- 172.21.15.63:0/1213588140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 msgr2=0x5642be85e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65c6000700 1 --2- 172.21.15.63:0/1213588140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 0x5642be85e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.379+0000 7f65c6000700 1 -- 172.21.15.63:0/1213588140 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642be738420 con 0x5642be7be9f0 2024-05-27T08:00:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.380+0000 7f65c6000700 1 --2- 172.21.15.63:0/1213588140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be7be9f0 0x5642be852750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5642be854210 tx=0x7f65b4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6595d133875d83d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.380+0000 7f65c7400700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65ac0b2a70 0x7f65ac0b4f80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f65c001d020 tx=0x7f65c0004f70 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.380+0000 7f65b3e00700 1 -- 172.21.15.63:0/1213588140 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4013110 con 0x5642be7be9f0 2024-05-27T08:00:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.381+0000 7f65b3e00700 1 -- 172.21.15.63:0/1213588140 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65b40132b0 con 0x5642be7be9f0 2024-05-27T08:00:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.381+0000 7f65b3e00700 1 -- 172.21.15.63:0/1213588140 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4005450 con 0x5642be7be9f0 2024-05-27T08:00:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.382+0000 7f65dc736b80 1 -- 172.21.15.63:0/1213588140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be7be9f0 msgr2=0x5642be852750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.382+0000 7f65dc736b80 1 --2- 172.21.15.63:0/1213588140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be7be9f0 0x5642be852750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5642be854210 tx=0x7f65b4009260 comp rx=0 tx=0).stop 2024-05-27T08:00:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.382+0000 7f65dc736b80 1 -- 172.21.15.63:0/1213588140 shutdown_connections 2024-05-27T08:00:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.382+0000 7f65dc736b80 1 --2- 172.21.15.63:0/1213588140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 0x5642be85e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.382+0000 7f65dc736b80 1 --2- 172.21.15.63:0/1213588140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be7be9f0 0x5642be852750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.382+0000 7f65dc736b80 1 --2- 172.21.15.63:0/1213588140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be6d2e40 0x5642be852c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.382+0000 7f65dc736b80 1 -- 172.21.15.63:0/1213588140 >> 172.21.15.63:0/1213588140 conn(0x5642be76c390 msgr2=0x5642be84d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.382+0000 7f65dc736b80 1 -- 172.21.15.63:0/1213588140 shutdown_connections 2024-05-27T08:00:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.382+0000 7f65dc736b80 1 -- 172.21.15.63:0/1213588140 wait complete. 2024-05-27T08:00:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.383+0000 7f65dc736b80 1 -- start start 2024-05-27T08:00:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.383+0000 7f65dc736b80 1 Processor -- start 2024-05-27T08:00:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.384+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be6d2e40 0x5642be859240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.384+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be7be9f0 0x5642be859780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.384+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be85c2c0 0x5642be859cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.384+0000 7f65dc736b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5642be675fa0 con 0x5642be6d2e40 2024-05-27T08:00:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.384+0000 7f65dc736b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642be710760 con 0x5642be7be9f0 2024-05-27T08:00:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.384+0000 7f65dc736b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5642be85eda0 con 0x5642be85c2c0 2024-05-27T08:00:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.384+0000 7f65dc736b80 10 client.0 _ll_register_callbacks cb 0x5642be719c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.385+0000 7f65c6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be6d2e40 0x5642be859240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.385+0000 7f65c6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be7be9f0 0x5642be859780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.385+0000 7f65c6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be6d2e40 0x5642be859240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41034/0 (socket says 172.21.15.63:41034) 2024-05-27T08:00:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.385+0000 7f65c6000700 1 -- 172.21.15.63:0/2630909164 learned_addr learned my addr 172.21.15.63:0/2630909164 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.385+0000 7f65c6000700 1 -- 172.21.15.63:0/2630909164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be85c2c0 msgr2=0x5642be859cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.385+0000 7f65c6000700 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be85c2c0 0x5642be859cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.385+0000 7f65c6000700 1 -- 172.21.15.63:0/2630909164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be7be9f0 msgr2=0x5642be859780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.385+0000 7f65c6000700 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be7be9f0 0x5642be859780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.385+0000 7f65c6000700 1 -- 172.21.15.63:0/2630909164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642be6bf8d0 con 0x5642be6d2e40 2024-05-27T08:00:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.386+0000 7f65c6000700 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be6d2e40 0x5642be859240 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009400 tx=0x7f65b40096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b5a49b03bd3ec0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.386+0000 7f65b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.386+0000 7f65b0c00700 1 -- 172.21.15.63:0/2630909164 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4005000 con 0x5642be6d2e40 2024-05-27T08:00:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.386+0000 7f65b0c00700 1 -- 172.21.15.63:0/2630909164 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f65b4004960 con 0x5642be6d2e40 2024-05-27T08:00:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.386+0000 7f65dc736b80 1 -- 172.21.15.63:0/2630909164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b40132b0 con 0x5642be6d2e40 2024-05-27T08:00:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.386+0000 7f65b0c00700 1 -- 172.21.15.63:0/2630909164 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f65b4004b60 con 0x5642be6d2e40 2024-05-27T08:00:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.386+0000 7f65dc736b80 10 client.20526 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.386+0000 7f65dc736b80 1 -- 172.21.15.63:0/2630909164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5642be740060 con 0x5642be6d2e40 2024-05-27T08:00:04.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.387+0000 7f65b0c00700 1 -- 172.21.15.63:0/2630909164 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65b40132b0 con 0x5642be6d2e40 2024-05-27T08:00:04.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.387+0000 7f65b0c00700 1 -- 172.21.15.63:0/2630909164 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f65b4011440 con 0x5642be6d2e40 2024-05-27T08:00:04.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.387+0000 7f65dc736b80 10 client.20526 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:04.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.387+0000 7f65dc736b80 10 client.20526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:04.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.387+0000 7f65dc736b80 20 client.20526 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:04.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.387+0000 7f65dc736b80 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642be874480 0x5642be8768a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.387+0000 7f65dc736b80 4 client.20526 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:04.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.387+0000 7f65dc736b80 1 -- 172.21.15.63:0/2630909164 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642be781d90 con 0x5642be874480 2024-05-27T08:00:04.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.388+0000 7f65c6a00700 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642be874480 0x5642be8768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.389+0000 7f65c6a00700 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642be874480 0x5642be8768a0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab8fa74e1a4af12 server_cookie=25ceceb8c190d4e9 in_seq=0 out_seq=0 2024-05-27T08:00:04.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.389+0000 7f65b0c00700 10 client.20526 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:04.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65b0c00700 1 -- 172.21.15.63:0/2630909164 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5642be781d90 con 0x5642be874480 2024-05-27T08:00:04.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65b0c00700 10 client.20526 handle_command_reply: tid=0 2024-05-27T08:00:04.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65dc736b80 1 client.20526 shutdown 2024-05-27T08:00:04.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65dc736b80 1 -- 172.21.15.63:0/2630909164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be6d2e40 msgr2=0x5642be859240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65dc736b80 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be6d2e40 0x5642be859240 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009400 tx=0x7f65b40096f0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65dc736b80 1 -- 172.21.15.63:0/2630909164 shutdown_connections 2024-05-27T08:00:04.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65dc736b80 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642be874480 0x5642be8768a0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65dc736b80 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be85c2c0 0x5642be859cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65dc736b80 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be6d2e40 0x5642be859240 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65dc736b80 1 --2- 172.21.15.63:0/2630909164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be7be9f0 0x5642be859780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65dc736b80 1 -- 172.21.15.63:0/2630909164 >> 172.21.15.63:0/2630909164 conn(0x5642be76c390 msgr2=0x5642be852820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.391+0000 7f65b0c00700 0 client.20526 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:04.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.392+0000 7f65dc736b80 1 -- 172.21.15.63:0/2630909164 shutdown_connections 2024-05-27T08:00:04.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.392+0000 7f65dc736b80 1 -- 172.21.15.63:0/2630909164 wait complete. 2024-05-27T08:00:04.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.392+0000 7f65dc736b80 20 client.20526 trim_cache size 0 max 16384 2024-05-27T08:00:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.409+0000 7f65dc736b80 1 Processor -- start 2024-05-27T08:00:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.409+0000 7f65dc736b80 1 -- start start 2024-05-27T08:00:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.409+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be6d2e40 0x5642be870930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.409+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be7be9f0 0x5642be8581a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.409+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 0x5642be84d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.409+0000 7f65dc736b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81b0970 con 0x5642be85c2c0 2024-05-27T08:00:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.409+0000 7f65dc736b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81ae200 con 0x5642be7be9f0 2024-05-27T08:00:04.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.409+0000 7f65dc736b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81ac090 con 0x5642be6d2e40 2024-05-27T08:00:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.410+0000 7f65c6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 0x5642be84d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.410+0000 7f65c6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 0x5642be84d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41040/0 (socket says 172.21.15.63:41040) 2024-05-27T08:00:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.410+0000 7f65c6000700 1 -- 172.21.15.63:0/2712850706 learned_addr learned my addr 172.21.15.63:0/2712850706 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.410+0000 7f65c6000700 1 --2- 172.21.15.63:0/2712850706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be6d2e40 0x5642be870930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.410+0000 7f65c6a00700 1 --2- 172.21.15.63:0/2712850706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be7be9f0 0x5642be8581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.411+0000 7f65c6000700 1 -- 172.21.15.63:0/2712850706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be6d2e40 msgr2=0x5642be870930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.411+0000 7f65c6000700 1 --2- 172.21.15.63:0/2712850706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be6d2e40 0x5642be870930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.411+0000 7f65c6000700 1 -- 172.21.15.63:0/2712850706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be7be9f0 msgr2=0x5642be8581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.411+0000 7f65c6000700 1 --2- 172.21.15.63:0/2712850706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be7be9f0 0x5642be8581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.411+0000 7f65c6000700 1 -- 172.21.15.63:0/2712850706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642be853d60 con 0x5642be85c2c0 2024-05-27T08:00:04.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.411+0000 7f65c6000700 1 --2- 172.21.15.63:0/2712850706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 0x5642be84d740 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x5642be870e70 tx=0x7f65b4013560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72222d457b0d63e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.411+0000 7f65b3400700 1 -- 172.21.15.63:0/2712850706 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4004960 con 0x5642be85c2c0 2024-05-27T08:00:04.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.411+0000 7f65b3400700 1 -- 172.21.15.63:0/2712850706 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65b4013250 con 0x5642be85c2c0 2024-05-27T08:00:04.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.412+0000 7f65b3400700 1 -- 172.21.15.63:0/2712850706 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4020440 con 0x5642be85c2c0 2024-05-27T08:00:04.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.412+0000 7f65dc736b80 1 -- 172.21.15.63:0/2712850706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 msgr2=0x5642be84d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.412+0000 7f65dc736b80 1 --2- 172.21.15.63:0/2712850706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 0x5642be84d740 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x5642be870e70 tx=0x7f65b4013560 comp rx=0 tx=0).stop 2024-05-27T08:00:04.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.412+0000 7f65dc736b80 1 -- 172.21.15.63:0/2712850706 shutdown_connections 2024-05-27T08:00:04.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.412+0000 7f65dc736b80 1 --2- 172.21.15.63:0/2712850706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be85c2c0 0x5642be84d740 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.412+0000 7f65dc736b80 1 --2- 172.21.15.63:0/2712850706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be6d2e40 0x5642be870930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.412+0000 7f65dc736b80 1 --2- 172.21.15.63:0/2712850706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be7be9f0 0x5642be8581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.412+0000 7f65dc736b80 1 -- 172.21.15.63:0/2712850706 >> 172.21.15.63:0/2712850706 conn(0x5642be76c390 msgr2=0x5642be854070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.413+0000 7f65dc736b80 1 -- 172.21.15.63:0/2712850706 shutdown_connections 2024-05-27T08:00:04.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.413+0000 7f65dc736b80 1 -- 172.21.15.63:0/2712850706 wait complete. 2024-05-27T08:00:04.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.414+0000 7f65dc736b80 1 -- start start 2024-05-27T08:00:04.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.414+0000 7f65dc736b80 1 Processor -- start 2024-05-27T08:00:04.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be6d2e40 0x5642be876e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be7be9f0 0x5642be8773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65dc736b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be85c2c0 0x5642be8d6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65dc736b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81ac6e0 con 0x5642be7be9f0 2024-05-27T08:00:04.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65dc736b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81adb30 con 0x5642be85c2c0 2024-05-27T08:00:04.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65dc736b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81acd90 con 0x5642be6d2e40 2024-05-27T08:00:04.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65dc736b80 10 client.0 _ll_register_callbacks cb 0x5642be719c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:04.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65c6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be7be9f0 0x5642be8773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65c6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be7be9f0 0x5642be8773a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41048/0 (socket says 172.21.15.63:41048) 2024-05-27T08:00:04.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65c6a00700 1 -- 172.21.15.63:0/420217142 learned_addr learned my addr 172.21.15.63:0/420217142 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:04.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.415+0000 7f65c6000700 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be6d2e40 0x5642be876e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.416+0000 7f65c6000700 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be85c2c0 0x5642be8d6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.416+0000 7f65c6a00700 1 -- 172.21.15.63:0/420217142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be6d2e40 msgr2=0x5642be876e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.416+0000 7f65c6a00700 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be6d2e40 0x5642be876e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.416+0000 7f65c6a00700 1 -- 172.21.15.63:0/420217142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be85c2c0 msgr2=0x5642be8d6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.416+0000 7f65c6a00700 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be85c2c0 0x5642be8d6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.416+0000 7f65c6a00700 1 -- 172.21.15.63:0/420217142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642be8d7220 con 0x5642be7be9f0 2024-05-27T08:00:04.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.416+0000 7f65c6a00700 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be7be9f0 0x5642be8773a0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f65b8004790 tx=0x7f65b800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21da31dc70c96a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.416+0000 7f65b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:04.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.417+0000 7f65b0c00700 1 -- 172.21.15.63:0/420217142 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b801d380 con 0x5642be7be9f0 2024-05-27T08:00:04.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.417+0000 7f65b0c00700 1 -- 172.21.15.63:0/420217142 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9483+0+0 (secure 0 0 0) 0x7f65b801d520 con 0x5642be7be9f0 2024-05-27T08:00:04.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.417+0000 7f65dc736b80 1 -- 172.21.15.63:0/420217142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b4013250 con 0x5642be7be9f0 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.417+0000 7f65b0c00700 1 -- 172.21.15.63:0/420217142 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f65b802f050 con 0x5642be7be9f0 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.417+0000 7f65dc736b80 10 client.20532 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.417+0000 7f65dc736b80 1 -- 172.21.15.63:0/420217142 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5642be8de560 con 0x5642be7be9f0 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.418+0000 7f65b0c00700 1 -- 172.21.15.63:0/420217142 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65b4013250 con 0x5642be7be9f0 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.418+0000 7f65b0c00700 1 -- 172.21.15.63:0/420217142 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f65b80269a0 con 0x5642be7be9f0 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.418+0000 7f65dc736b80 10 client.20532 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.418+0000 7f65dc736b80 10 client.20532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.418+0000 7f65dc736b80 20 client.20532 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.418+0000 7f65dc736b80 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642be8dec50 0x5642be8e1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.418+0000 7f65dc736b80 4 client.20532 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:00:04.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.418+0000 7f65dc736b80 1 -- 172.21.15.63:0/420217142 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642be77b570 con 0x5642be8dec50 2024-05-27T08:00:04.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.419+0000 7f65c6000700 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642be8dec50 0x5642be8e1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.420+0000 7f65c6000700 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642be8dec50 0x5642be8e1070 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf046cffbb44bdb server_cookie=b25db8fd04c3b448 in_seq=0 out_seq=0 2024-05-27T08:00:04.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.420+0000 7f65b0c00700 10 client.20532 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:04.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65b0c00700 1 -- 172.21.15.63:0/420217142 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642be77b570 con 0x5642be8dec50 2024-05-27T08:00:04.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65b0c00700 10 client.20532 handle_command_reply: tid=0 2024-05-27T08:00:04.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 client.20532 shutdown 2024-05-27T08:00:04.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 -- 172.21.15.63:0/420217142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be7be9f0 msgr2=0x5642be8773a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be7be9f0 0x5642be8773a0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f65b8004790 tx=0x7f65b800be80 comp rx=0 tx=0).stop 2024-05-27T08:00:04.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 -- 172.21.15.63:0/420217142 shutdown_connections 2024-05-27T08:00:04.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642be8dec50 0x5642be8e1070 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642be85c2c0 0x5642be8d6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642be6d2e40 0x5642be876e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 --2- 172.21.15.63:0/420217142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642be7be9f0 0x5642be8773a0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65b0c00700 0 client.20532 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:04.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 -- 172.21.15.63:0/420217142 >> 172.21.15.63:0/420217142 conn(0x5642be76c390 msgr2=0x5642be854800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 -- 172.21.15.63:0/420217142 shutdown_connections 2024-05-27T08:00:04.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 1 -- 172.21.15.63:0/420217142 wait complete. 2024-05-27T08:00:04.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.421+0000 7f65dc736b80 20 client.20532 trim_cache size 0 max 16384 2024-05-27T08:00:04.425 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:00:04.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.422+0000 7f65c7e00700 1 -- 172.21.15.63:0/1072320394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65ac0b2a70 msgr2=0x7f65ac0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.422+0000 7f65c7e00700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65ac0b2a70 0x7f65ac0b4f80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f65c001d020 tx=0x7f65c0004f70 comp rx=0 tx=0).stop 2024-05-27T08:00:04.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.422+0000 7f65c7e00700 1 -- 172.21.15.63:0/1072320394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c810e710 msgr2=0x7f65c81a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.422+0000 7f65c7e00700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c810e710 0x7f65c81a1550 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f65c000b070 tx=0x7f65c0004c80 comp rx=0 tx=0).stop 2024-05-27T08:00:04.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.422+0000 7f65c7e00700 1 -- 172.21.15.63:0/1072320394 shutdown_connections 2024-05-27T08:00:04.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.422+0000 7f65c7e00700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65ac0b2a70 0x7f65ac0b4f80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.422+0000 7f65c7e00700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65c810e710 0x7f65c81a1550 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.422+0000 7f65c7e00700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c81052a0 0x7f65c819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.422+0000 7f65c7e00700 1 --2- 172.21.15.63:0/1072320394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65c8105bf0 0x7f65c819d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.422+0000 7f65c7e00700 1 -- 172.21.15.63:0/1072320394 >> 172.21.15.63:0/1072320394 conn(0x7f65c8078c30 msgr2=0x7f65c8102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.423+0000 7f65c7e00700 1 -- 172.21.15.63:0/1072320394 shutdown_connections 2024-05-27T08:00:04.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:04.423+0000 7f65c7e00700 1 -- 172.21.15.63:0/1072320394 wait complete. 2024-05-27T08:00:05.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:05 smithi063 ceph-mon[31613]: pgmap v839: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 27 op/s 2024-05-27T08:00:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:05 smithi086 ceph-mon[34687]: pgmap v839: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 27 op/s 2024-05-27T08:00:05.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:05 smithi149 ceph-mon[35408]: pgmap v839: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 27 op/s 2024-05-27T08:00:06.397 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:00:06.397 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:00:06.958 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:00:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:07 smithi086 ceph-mon[34687]: pgmap v840: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 27 op/s 2024-05-27T08:00:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:07 smithi086 ceph-mon[34687]: osdmap e135: 12 total, 12 up, 12 in 2024-05-27T08:00:07.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:07 smithi149 ceph-mon[35408]: pgmap v840: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 27 op/s 2024-05-27T08:00:07.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:07 smithi149 ceph-mon[35408]: osdmap e135: 12 total, 12 up, 12 in 2024-05-27T08:00:07.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:07 smithi063 ceph-mon[31613]: pgmap v840: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 27 op/s 2024-05-27T08:00:07.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:07 smithi063 ceph-mon[31613]: osdmap e135: 12 total, 12 up, 12 in 2024-05-27T08:00:08.852 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:08 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:00:08.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:08 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:00:08.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:08 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:00:09.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.467+0000 7f4b35a00700 1 -- 172.21.15.63:0/3288562031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b301054b0 msgr2=0x7f4b301058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:09.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.467+0000 7f4b35a00700 1 --2- 172.21.15.63:0/3288562031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b301054b0 0x7f4b301058c0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009990 tx=0x7f4b20009ca0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.467+0000 7f4b35a00700 1 -- 172.21.15.63:0/3288562031 shutdown_connections 2024-05-27T08:00:09.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.467+0000 7f4b35a00700 1 --2- 172.21.15.63:0/3288562031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b3010e8e0 0x7f4b30110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.467+0000 7f4b35a00700 1 --2- 172.21.15.63:0/3288562031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b301054b0 0x7f4b301058c0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.467+0000 7f4b35a00700 1 --2- 172.21.15.63:0/3288562031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b30105e00 0x7f4b3010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.467+0000 7f4b35a00700 1 -- 172.21.15.63:0/3288562031 >> 172.21.15.63:0/3288562031 conn(0x7f4b300fedb0 msgr2=0x7f4b301011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:09.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.467+0000 7f4b35a00700 1 -- 172.21.15.63:0/3288562031 shutdown_connections 2024-05-27T08:00:09.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.468+0000 7f4b35a00700 1 -- 172.21.15.63:0/3288562031 wait complete. 2024-05-27T08:00:09.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.469+0000 7f4b35a00700 1 Processor -- start 2024-05-27T08:00:09.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.469+0000 7f4b35a00700 1 -- start start 2024-05-27T08:00:09.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.470+0000 7f4b35a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b301054b0 0x7f4b3019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.470+0000 7f4b35a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b30105e00 0x7f4b3019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.470+0000 7f4b35a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b3010e8e0 0x7f4b301a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.470+0000 7f4b35a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30114330 con 0x7f4b30105e00 2024-05-27T08:00:09.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.470+0000 7f4b35a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301141b0 con 0x7f4b301054b0 2024-05-27T08:00:09.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.470+0000 7f4b35a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301144b0 con 0x7f4b3010e8e0 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.470+0000 7f4b2e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b30105e00 0x7f4b3019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.470+0000 7f4b2e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b30105e00 0x7f4b3019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49904/0 (socket says 172.21.15.63:49904) 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.471+0000 7f4b2e000700 1 -- 172.21.15.63:0/3685341120 learned_addr learned my addr 172.21.15.63:0/3685341120 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.471+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b301054b0 0x7f4b3019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.471+0000 7f4b2f400700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b3010e8e0 0x7f4b301a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.471+0000 7f4b2e000700 1 -- 172.21.15.63:0/3685341120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b3010e8e0 msgr2=0x7f4b301a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.471+0000 7f4b2e000700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b3010e8e0 0x7f4b301a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.471+0000 7f4b2e000700 1 -- 172.21.15.63:0/3685341120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b301054b0 msgr2=0x7f4b3019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.471+0000 7f4b2e000700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b301054b0 0x7f4b3019cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.471+0000 7f4b2e000700 1 -- 172.21.15.63:0/3685341120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b301a1dd0 con 0x7f4b30105e00 2024-05-27T08:00:09.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.472+0000 7f4b2e000700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b30105e00 0x7f4b3019d4a0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00d950 tx=0x7f4b1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.472+0000 7f4b27400700 1 -- 172.21.15.63:0/3685341120 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c016070 con 0x7f4b30105e00 2024-05-27T08:00:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.472+0000 7f4b27400700 1 -- 172.21.15.63:0/3685341120 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b1c00fd20 con 0x7f4b30105e00 2024-05-27T08:00:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.472+0000 7f4b27400700 1 -- 172.21.15.63:0/3685341120 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00e3b0 con 0x7f4b30105e00 2024-05-27T08:00:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.472+0000 7f4b35a00700 1 -- 172.21.15.63:0/3685341120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b301a20c0 con 0x7f4b30105e00 2024-05-27T08:00:09.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.472+0000 7f4b35a00700 1 -- 172.21.15.63:0/3685341120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b301a99e0 con 0x7f4b30105e00 2024-05-27T08:00:09.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.478+0000 7f4b4458bb80 1 Processor -- start 2024-05-27T08:00:09.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.478+0000 7f4b4458bb80 1 -- start start 2024-05-27T08:00:09.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.478+0000 7f4b27400700 1 -- 172.21.15.63:0/3685341120 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4b1c019070 con 0x7f4b30105e00 2024-05-27T08:00:09.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.478+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93a12a70 0x55ac93aa53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.478+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93926ec0 0x55ac93aa58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.478+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.478+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ac939d3ec0 con 0x55ac93aa5ef0 2024-05-27T08:00:09.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.479+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac939c3a60 con 0x55ac93926ec0 2024-05-27T08:00:09.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.479+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ac939c7aa0 con 0x55ac93a12a70 2024-05-27T08:00:09.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.479+0000 7f4b2e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.479+0000 7f4b2e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49918/0 (socket says 172.21.15.63:49918) 2024-05-27T08:00:09.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.479+0000 7f4b2f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93a12a70 0x55ac93aa53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.479+0000 7f4b2e000700 1 -- 172.21.15.63:0/2733350199 learned_addr learned my addr 172.21.15.63:0/2733350199 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:09.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.479+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/2733350199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93926ec0 0x55ac93aa58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.480+0000 7f4b2e000700 1 -- 172.21.15.63:0/2733350199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93a12a70 msgr2=0x55ac93aa53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:09.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.480+0000 7f4b2e000700 1 --2- 172.21.15.63:0/2733350199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93a12a70 0x55ac93aa53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.480+0000 7f4b2e000700 1 -- 172.21.15.63:0/2733350199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93926ec0 msgr2=0x55ac93aa58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:09.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.480+0000 7f4b2e000700 1 --2- 172.21.15.63:0/2733350199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93926ec0 0x55ac93aa58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.480+0000 7f4b2e000700 1 -- 172.21.15.63:0/2733350199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9398c4a0 con 0x55ac93aa5ef0 2024-05-27T08:00:09.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.481+0000 7f4b2e000700 1 --2- 172.21.15.63:0/2733350199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2190 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0df6d0 tx=0x7f4b1c0e88f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a3ad8a85205b15 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:09.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.481+0000 7f4b1be00700 1 -- 172.21.15.63:0/2733350199 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0f6070 con 0x55ac93aa5ef0 2024-05-27T08:00:09.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.481+0000 7f4b27400700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b140b2b40 0x7f4b140b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.481+0000 7f4b1be00700 1 -- 172.21.15.63:0/2733350199 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b1c0e2cf0 con 0x55ac93aa5ef0 2024-05-27T08:00:09.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.481+0000 7f4b1be00700 1 -- 172.21.15.63:0/2733350199 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0a5940 con 0x55ac93aa5ef0 2024-05-27T08:00:09.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.481+0000 7f4b27400700 1 -- 172.21.15.63:0/3685341120 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9571+0+0 (secure 0 0 0) 0x7f4b1c0e2430 con 0x7f4b30105e00 2024-05-27T08:00:09.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.482+0000 7f4b2f400700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b140b2b40 0x7f4b140b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.483+0000 7f4b4458bb80 1 -- 172.21.15.63:0/2733350199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 msgr2=0x55ac93ab2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:09.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.483+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/2733350199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2190 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0df6d0 tx=0x7f4b1c0e88f0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.483+0000 7f4b2f400700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b140b2b40 0x7f4b140b5050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009790 tx=0x7f4b28009130 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:09.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.483+0000 7f4b4458bb80 1 -- 172.21.15.63:0/2733350199 shutdown_connections 2024-05-27T08:00:09.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.483+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/2733350199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2190 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.483+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/2733350199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93a12a70 0x55ac93aa53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.483+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/2733350199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93926ec0 0x55ac93aa58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.483+0000 7f4b4458bb80 1 -- 172.21.15.63:0/2733350199 >> 172.21.15.63:0/2733350199 conn(0x55ac939c0410 msgr2=0x55ac93aa1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:09.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.483+0000 7f4b4458bb80 1 -- 172.21.15.63:0/2733350199 shutdown_connections 2024-05-27T08:00:09.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.483+0000 7f4b4458bb80 1 -- 172.21.15.63:0/2733350199 wait complete. 2024-05-27T08:00:09.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.484+0000 7f4b4458bb80 1 -- start start 2024-05-27T08:00:09.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.484+0000 7f4b4458bb80 1 Processor -- start 2024-05-27T08:00:09.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.485+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93926ec0 0x55ac93aaf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.485+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93a12a70 0x55ac93aafc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.485+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.485+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ac938ca020 con 0x55ac93926ec0 2024-05-27T08:00:09.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.485+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac939647e0 con 0x55ac93a12a70 2024-05-27T08:00:09.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.485+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ac93ab2c00 con 0x55ac93aa5ef0 2024-05-27T08:00:09.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.485+0000 7f4b4458bb80 10 client.0 _ll_register_callbacks cb 0x55ac9396dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:09.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.486+0000 7f4b2ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93926ec0 0x55ac93aaf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.486+0000 7f4b2ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93926ec0 0x55ac93aaf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49928/0 (socket says 172.21.15.63:49928) 2024-05-27T08:00:09.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.486+0000 7f4b2ea00700 1 -- 172.21.15.63:0/3216964723 learned_addr learned my addr 172.21.15.63:0/3216964723 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:09.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.486+0000 7f4b2e000700 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93a12a70 0x55ac93aafc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.486+0000 7f4b2f400700 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.487+0000 7f4b2ea00700 1 -- 172.21.15.63:0/3216964723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93aa5ef0 msgr2=0x55ac93ab0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:09.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.487+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.487+0000 7f4b2ea00700 1 -- 172.21.15.63:0/3216964723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93a12a70 msgr2=0x55ac93aafc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:09.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.487+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93a12a70 0x55ac93aafc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.487+0000 7f4b2ea00700 1 -- 172.21.15.63:0/3216964723 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac93913950 con 0x55ac93926ec0 2024-05-27T08:00:09.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.487+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93926ec0 0x55ac93aaf6e0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009990 tx=0x7f4b200057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410026c0dde0fe6a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:09.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.487+0000 7f4b18c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:09.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.487+0000 7f4b18c00700 1 -- 172.21.15.63:0/3216964723 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20003bd0 con 0x55ac93926ec0 2024-05-27T08:00:09.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.488+0000 7f4b18c00700 1 -- 172.21.15.63:0/3216964723 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9571+0+0 (secure 0 0 0) 0x7f4b20003d70 con 0x55ac93926ec0 2024-05-27T08:00:09.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.488+0000 7f4b4458bb80 1 -- 172.21.15.63:0/3216964723 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1c0e2cf0 con 0x55ac93926ec0 2024-05-27T08:00:09.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.488+0000 7f4b18c00700 1 -- 172.21.15.63:0/3216964723 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4b20031050 con 0x55ac93926ec0 2024-05-27T08:00:09.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.488+0000 7f4b4458bb80 10 client.20544 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:09.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.488+0000 7f4b4458bb80 1 -- 172.21.15.63:0/3216964723 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ac939940e0 con 0x55ac93926ec0 2024-05-27T08:00:09.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.489+0000 7f4b18c00700 1 -- 172.21.15.63:0/3216964723 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b1c0e2cf0 con 0x55ac93926ec0 2024-05-27T08:00:09.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.489+0000 7f4b18c00700 1 -- 172.21.15.63:0/3216964723 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4b20023660 con 0x55ac93926ec0 2024-05-27T08:00:09.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.489+0000 7f4b4458bb80 10 client.20544 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:09.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.489+0000 7f4b4458bb80 10 client.20544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:09.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.489+0000 7f4b4458bb80 20 client.20544 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:09.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.489+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac93abfc40 0x55ac93ac20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.489+0000 7f4b4458bb80 4 client.20544 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:09.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.489+0000 7f4b4458bb80 1 -- 172.21.15.63:0/3216964723 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac939d5e10 con 0x55ac93abfc40 2024-05-27T08:00:09.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.490+0000 7f4b2e000700 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac93abfc40 0x55ac93ac20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.491+0000 7f4b2e000700 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac93abfc40 0x55ac93ac20e0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67994fdae62fd8d1 server_cookie=1f0fbb62076ff37d in_seq=0 out_seq=0 2024-05-27T08:00:09.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.491+0000 7f4b18c00700 10 client.20544 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:09.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.493+0000 7f4b18c00700 1 -- 172.21.15.63:0/3216964723 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ac939d5e10 con 0x55ac93abfc40 2024-05-27T08:00:09.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.493+0000 7f4b18c00700 10 client.20544 handle_command_reply: tid=0 2024-05-27T08:00:09.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.493+0000 7f4b4458bb80 1 client.20544 shutdown 2024-05-27T08:00:09.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.493+0000 7f4b4458bb80 1 -- 172.21.15.63:0/3216964723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93926ec0 msgr2=0x55ac93aaf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:09.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.493+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93926ec0 0x55ac93aaf6e0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009990 tx=0x7f4b200057e0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.494+0000 7f4b4458bb80 1 -- 172.21.15.63:0/3216964723 shutdown_connections 2024-05-27T08:00:09.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.494+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac93abfc40 0x55ac93ac20e0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.494+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.494+0000 7f4b18c00700 0 client.20544 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:09.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.494+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93926ec0 0x55ac93aaf6e0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.494+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/3216964723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93a12a70 0x55ac93aafc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.494+0000 7f4b4458bb80 1 -- 172.21.15.63:0/3216964723 >> 172.21.15.63:0/3216964723 conn(0x55ac939c0410 msgr2=0x55ac93aa5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:09.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.494+0000 7f4b4458bb80 1 -- 172.21.15.63:0/3216964723 shutdown_connections 2024-05-27T08:00:09.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.494+0000 7f4b4458bb80 1 -- 172.21.15.63:0/3216964723 wait complete. 2024-05-27T08:00:09.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.495+0000 7f4b4458bb80 20 client.20544 trim_cache size 0 max 16384 2024-05-27T08:00:09.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.526+0000 7f4b4458bb80 1 Processor -- start 2024-05-27T08:00:09.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.526+0000 7f4b4458bb80 1 -- start start 2024-05-27T08:00:09.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93a12a70 0x55ac93aa7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93aa5ef0 0x55ac93abbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93aa90b0 0x55ac93aa94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b5080 con 0x55ac93a12a70 2024-05-27T08:00:09.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b2910 con 0x55ac93aa90b0 2024-05-27T08:00:09.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b0780 con 0x55ac93aa5ef0 2024-05-27T08:00:09.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b2ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93a12a70 0x55ac93aa7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b2ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93a12a70 0x55ac93aa7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49940/0 (socket says 172.21.15.63:49940) 2024-05-27T08:00:09.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b2ea00700 1 -- 172.21.15.63:0/4273598827 learned_addr learned my addr 172.21.15.63:0/4273598827 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:09.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b2f400700 1 --2- 172.21.15.63:0/4273598827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93aa90b0 0x55ac93aa94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.527+0000 7f4b2e000700 1 --2- 172.21.15.63:0/4273598827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93aa5ef0 0x55ac93abbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.528+0000 7f4b2ea00700 1 -- 172.21.15.63:0/4273598827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93aa5ef0 msgr2=0x55ac93abbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:09.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.528+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/4273598827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93aa5ef0 0x55ac93abbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.528+0000 7f4b2ea00700 1 -- 172.21.15.63:0/4273598827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93aa90b0 msgr2=0x55ac93aa94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:09.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.528+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/4273598827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93aa90b0 0x55ac93aa94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.528+0000 7f4b2ea00700 1 -- 172.21.15.63:0/4273598827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac93aa0b90 con 0x55ac93a12a70 2024-05-27T08:00:09.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.528+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/4273598827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93a12a70 0x55ac93aa7980 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009ce0 tx=0x7f4b20009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ea6c86eb4e24cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:09.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.529+0000 7f4b1b400700 1 -- 172.21.15.63:0/4273598827 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20023470 con 0x55ac93a12a70 2024-05-27T08:00:09.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.529+0000 7f4b1b400700 1 -- 172.21.15.63:0/4273598827 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b2003a070 con 0x55ac93a12a70 2024-05-27T08:00:09.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.529+0000 7f4b1b400700 1 -- 172.21.15.63:0/4273598827 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000ba60 con 0x55ac93a12a70 2024-05-27T08:00:09.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.530+0000 7f4b4458bb80 1 -- 172.21.15.63:0/4273598827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93a12a70 msgr2=0x55ac93aa7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:09.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.530+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/4273598827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93a12a70 0x55ac93aa7980 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009ce0 tx=0x7f4b20009fd0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.530+0000 7f4b4458bb80 1 -- 172.21.15.63:0/4273598827 shutdown_connections 2024-05-27T08:00:09.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.530+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/4273598827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93aa90b0 0x55ac93aa94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.530+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/4273598827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93a12a70 0x55ac93aa7980 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.530+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/4273598827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93aa5ef0 0x55ac93abbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.530+0000 7f4b4458bb80 1 -- 172.21.15.63:0/4273598827 >> 172.21.15.63:0/4273598827 conn(0x55ac939c0410 msgr2=0x55ac93aae640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:09.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.530+0000 7f4b4458bb80 1 -- 172.21.15.63:0/4273598827 shutdown_connections 2024-05-27T08:00:09.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.531+0000 7f4b4458bb80 1 -- 172.21.15.63:0/4273598827 wait complete. 2024-05-27T08:00:09.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.531+0000 7f4b4458bb80 1 -- start start 2024-05-27T08:00:09.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.531+0000 7f4b4458bb80 1 Processor -- start 2024-05-27T08:00:09.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.532+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93a12a70 0x55ac93ab83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.532+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.532+0000 7f4b4458bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93aa90b0 0x55ac93ab33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.532+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b0dd0 con 0x55ac93aa5ef0 2024-05-27T08:00:09.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.532+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b2240 con 0x55ac93aa90b0 2024-05-27T08:00:09.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.532+0000 7f4b4458bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b14a0 con 0x55ac93a12a70 2024-05-27T08:00:09.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.532+0000 7f4b4458bb80 10 client.0 _ll_register_callbacks cb 0x55ac9396dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:09.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.532+0000 7f4b2e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.532+0000 7f4b2e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49954/0 (socket says 172.21.15.63:49954) 2024-05-27T08:00:09.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.533+0000 7f4b2e000700 1 -- 172.21.15.63:0/683788455 learned_addr learned my addr 172.21.15.63:0/683788455 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:09.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.533+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93a12a70 0x55ac93ab83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.533+0000 7f4b2f400700 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93aa90b0 0x55ac93ab33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.533+0000 7f4b2e000700 1 -- 172.21.15.63:0/683788455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93a12a70 msgr2=0x55ac93ab83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:09.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.533+0000 7f4b2e000700 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93a12a70 0x55ac93ab83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.533+0000 7f4b2e000700 1 -- 172.21.15.63:0/683788455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93aa90b0 msgr2=0x55ac93ab33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:09.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.533+0000 7f4b2e000700 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93aa90b0 0x55ac93ab33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.533+0000 7f4b2e000700 1 -- 172.21.15.63:0/683788455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac93ab3900 con 0x55ac93aa5ef0 2024-05-27T08:00:09.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.534+0000 7f4b2e000700 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2e80 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0a5370 tx=0x7f4b1c0e8c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da919c63b12775e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:09.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.534+0000 7f4b18c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:09.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.534+0000 7f4b18c00700 1 -- 172.21.15.63:0/683788455 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0a8c40 con 0x55ac93aa5ef0 2024-05-27T08:00:09.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.534+0000 7f4b18c00700 1 -- 172.21.15.63:0/683788455 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9571+0+0 (secure 0 0 0) 0x7f4b1c0106a0 con 0x55ac93aa5ef0 2024-05-27T08:00:09.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.534+0000 7f4b4458bb80 1 -- 172.21.15.63:0/683788455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2003a070 con 0x55ac93aa5ef0 2024-05-27T08:00:09.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.535+0000 7f4b18c00700 1 -- 172.21.15.63:0/683788455 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4b1c0a8e30 con 0x55ac93aa5ef0 2024-05-27T08:00:09.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.535+0000 7f4b4458bb80 10 client.20550 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:09.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.535+0000 7f4b4458bb80 1 -- 172.21.15.63:0/683788455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ac93b33c90 con 0x55ac93aa5ef0 2024-05-27T08:00:09.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.535+0000 7f4b18c00700 1 -- 172.21.15.63:0/683788455 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b2003a070 con 0x55ac93aa5ef0 2024-05-27T08:00:09.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.535+0000 7f4b18c00700 1 -- 172.21.15.63:0/683788455 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4b1c0df9e0 con 0x55ac93aa5ef0 2024-05-27T08:00:09.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.535+0000 7f4b4458bb80 10 client.20550 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:09.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.535+0000 7f4b4458bb80 10 client.20550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:09.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.536+0000 7f4b4458bb80 20 client.20550 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:09.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.536+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac93b33e60 0x55ac93b36280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:09.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.536+0000 7f4b4458bb80 4 client.20550 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:00:09.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.536+0000 7f4b4458bb80 1 -- 172.21.15.63:0/683788455 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ac939cf5f0 con 0x55ac93b33e60 2024-05-27T08:00:09.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.536+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac93b33e60 0x55ac93b36280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:09.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.537+0000 7f4b2ea00700 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac93b33e60 0x55ac93b36280 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1b9f569887089d server_cookie=d179b9de574ffd2d in_seq=0 out_seq=0 2024-05-27T08:00:09.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.537+0000 7f4b18c00700 10 client.20550 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:09.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.540+0000 7f4b18c00700 1 -- 172.21.15.63:0/683788455 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ac939cf5f0 con 0x55ac93b33e60 2024-05-27T08:00:09.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.540+0000 7f4b18c00700 10 client.20550 handle_command_reply: tid=0 2024-05-27T08:00:09.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.540+0000 7f4b4458bb80 1 client.20550 shutdown 2024-05-27T08:00:09.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.540+0000 7f4b4458bb80 1 -- 172.21.15.63:0/683788455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 msgr2=0x55ac93ab2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:09.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.540+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2e80 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0a5370 tx=0x7f4b1c0e8c40 comp rx=0 tx=0).stop 2024-05-27T08:00:09.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.540+0000 7f4b4458bb80 1 -- 172.21.15.63:0/683788455 shutdown_connections 2024-05-27T08:00:09.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.540+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ac93b33e60 0x55ac93b36280 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.541+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac93aa90b0 0x55ac93ab33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.541+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac93a12a70 0x55ac93ab83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.541+0000 7f4b4458bb80 1 --2- 172.21.15.63:0/683788455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ac93aa5ef0 0x55ac93ab2e80 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.541+0000 7f4b4458bb80 1 -- 172.21.15.63:0/683788455 >> 172.21.15.63:0/683788455 conn(0x55ac939c0410 msgr2=0x55ac93aa80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:09.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.541+0000 7f4b18c00700 0 client.20550 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:09.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.541+0000 7f4b4458bb80 1 -- 172.21.15.63:0/683788455 shutdown_connections 2024-05-27T08:00:09.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.541+0000 7f4b4458bb80 1 -- 172.21.15.63:0/683788455 wait complete. 2024-05-27T08:00:09.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.541+0000 7f4b4458bb80 20 client.20550 trim_cache size 0 max 16384 2024-05-27T08:00:09.544 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:00:09.544 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:00:09.545 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "f063ae70-aa3a-4f08-81d3-b4bafc842384", 2024-05-27T08:00:09.545 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:00:09.545 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:00:09.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.542+0000 7f4b35a00700 1 -- 172.21.15.63:0/3685341120 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b140b2b40 msgr2=0x7f4b140b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:09.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.542+0000 7f4b35a00700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b140b2b40 0x7f4b140b5050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009790 tx=0x7f4b28009130 comp rx=0 tx=0).stop 2024-05-27T08:00:09.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.542+0000 7f4b35a00700 1 -- 172.21.15.63:0/3685341120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b30105e00 msgr2=0x7f4b3019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:09.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.542+0000 7f4b35a00700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b30105e00 0x7f4b3019d4a0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00d950 tx=0x7f4b1c00dc60 comp rx=0 tx=0).stop 2024-05-27T08:00:09.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.543+0000 7f4b35a00700 1 -- 172.21.15.63:0/3685341120 shutdown_connections 2024-05-27T08:00:09.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.543+0000 7f4b35a00700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b140b2b40 0x7f4b140b5050 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.543+0000 7f4b35a00700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b3010e8e0 0x7f4b301a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.543+0000 7f4b35a00700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b301054b0 0x7f4b3019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.543+0000 7f4b35a00700 1 --2- 172.21.15.63:0/3685341120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b30105e00 0x7f4b3019d4a0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:09.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.543+0000 7f4b35a00700 1 -- 172.21.15.63:0/3685341120 >> 172.21.15.63:0/3685341120 conn(0x7f4b300fedb0 msgr2=0x7f4b30103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:09.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.543+0000 7f4b35a00700 1 -- 172.21.15.63:0/3685341120 shutdown_connections 2024-05-27T08:00:09.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:09.543+0000 7f4b35a00700 1 -- 172.21.15.63:0/3685341120 wait complete. 2024-05-27T08:00:09.893 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:09 smithi063 ceph-mon[31613]: pgmap v842: 129 pgs: 1 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 111 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T08:00:09.893 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:09 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:00:09.893 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:09 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:00:09.893 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:09 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:00:09.893 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:09 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:09.893 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:09 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:00:09.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:09 smithi086 ceph-mon[34687]: pgmap v842: 129 pgs: 1 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 111 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T08:00:09.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:09 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:00:09.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:09 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:00:09.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:09 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:00:09.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:09 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:09.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:09 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:00:09.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:09 smithi149 ceph-mon[35408]: pgmap v842: 129 pgs: 1 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 111 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T08:00:09.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:09 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:00:09.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:09 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:00:09.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:09 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:00:09.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:09 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:09.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:09 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:00:10.369 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:00:10.870 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:00:10.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:00:10.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:00:10.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:00:10.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:00:10.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:00:10.963 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.963 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:00:10.963 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.963 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.963 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:00:10.963 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:00:10.963 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:00:10.963 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:10 smithi086 ceph-mon[34687]: pgmap v843: 129 pgs: 1 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 111 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T08:00:10.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:00:10.966 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:00:10.966 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:00:10.966 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:10 smithi149 ceph-mon[35408]: pgmap v843: 129 pgs: 1 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 111 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T08:00:10.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:00:10.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:00:10.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:00:10.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:00:10.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:00:10.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:00:10.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:10.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:00:10.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:00:10.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:00:10.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:10 smithi063 ceph-mon[31613]: pgmap v843: 129 pgs: 1 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 111 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 33 op/s 2024-05-27T08:00:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.048+0000 7f9473600640 1 -- 172.21.15.63:0/1840303065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946c104400 msgr2=0x7f946c1107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.048+0000 7f9473600640 1 --2- 172.21.15.63:0/1840303065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946c104400 0x7f946c1107d0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f9464009ca0 tx=0x7f946402a8b0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.049+0000 7f9473600640 1 -- 172.21.15.63:0/1840303065 shutdown_connections 2024-05-27T08:00:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.049+0000 7f9473600640 1 --2- 172.21.15.63:0/1840303065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946c104400 0x7f946c1107d0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.049+0000 7f9473600640 1 --2- 172.21.15.63:0/1840303065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f946c10a150 0x7f946c103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.049+0000 7f9473600640 1 --2- 172.21.15.63:0/1840303065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c109cb0 0x7f946c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.049+0000 7f9473600640 1 -- 172.21.15.63:0/1840303065 >> 172.21.15.63:0/1840303065 conn(0x7f946c0ff480 msgr2=0x7f946c1018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.049+0000 7f9473600640 1 -- 172.21.15.63:0/1840303065 shutdown_connections 2024-05-27T08:00:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.049+0000 7f9473600640 1 -- 172.21.15.63:0/1840303065 wait complete. 2024-05-27T08:00:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.050+0000 7f9473600640 1 Processor -- start 2024-05-27T08:00:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.051+0000 7f9473600640 1 -- start start 2024-05-27T08:00:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.051+0000 7f9473600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946c104400 0x7f946c074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.051+0000 7f9473600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c109cb0 0x7f946c075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.051+0000 7f9473600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f946c10a150 0x7f946c075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.051+0000 7f9473600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f946c113da0 con 0x7f946c104400 2024-05-27T08:00:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.051+0000 7f9473600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f946c113c20 con 0x7f946c109cb0 2024-05-27T08:00:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.051+0000 7f9473600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f946c113f20 con 0x7f946c10a150 2024-05-27T08:00:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.052+0000 7f9470c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946c104400 0x7f946c074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.052+0000 7f9470c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946c104400 0x7f946c074fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49984/0 (socket says 172.21.15.63:49984) 2024-05-27T08:00:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.052+0000 7f9470c00640 1 -- 172.21.15.63:0/3838321372 learned_addr learned my addr 172.21.15.63:0/3838321372 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.052+0000 7f946be00640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c109cb0 0x7f946c075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.052+0000 7f9471600640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f946c10a150 0x7f946c075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.052+0000 7f9470c00640 1 -- 172.21.15.63:0/3838321372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f946c10a150 msgr2=0x7f946c075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.052+0000 7f9470c00640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f946c10a150 0x7f946c075a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.052+0000 7f9470c00640 1 -- 172.21.15.63:0/3838321372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c109cb0 msgr2=0x7f946c075500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.052+0000 7f9470c00640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c109cb0 0x7f946c075500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.052+0000 7f9470c00640 1 -- 172.21.15.63:0/3838321372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c1a8a70 con 0x7f946c104400 2024-05-27T08:00:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.053+0000 7f9470c00640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946c104400 0x7f946c074fc0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f946000d770 tx=0x7f946000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.053+0000 7f9469600640 1 -- 172.21.15.63:0/3838321372 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460017070 con 0x7f946c104400 2024-05-27T08:00:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.053+0000 7f9469600640 1 -- 172.21.15.63:0/3838321372 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9460009c30 con 0x7f946c104400 2024-05-27T08:00:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.053+0000 7f9473600640 1 -- 172.21.15.63:0/3838321372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946c1a8d60 con 0x7f946c104400 2024-05-27T08:00:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.053+0000 7f9469600640 1 -- 172.21.15.63:0/3838321372 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946000f620 con 0x7f946c104400 2024-05-27T08:00:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.053+0000 7f9473600640 1 -- 172.21.15.63:0/3838321372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946c1a9270 con 0x7f946c104400 2024-05-27T08:00:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.057+0000 7f947a5c6740 1 Processor -- start 2024-05-27T08:00:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.057+0000 7f947a5c6740 1 -- start start 2024-05-27T08:00:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.058+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae6a1b0 0x55cb5b20c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.058+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae412f0 0x55cb5b20cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.058+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae494d0 0x55cb5b20d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.058+0000 7f947a5c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b17db90 con 0x55cb5ae494d0 2024-05-27T08:00:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.058+0000 7f947a5c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b16d730 con 0x55cb5ae412f0 2024-05-27T08:00:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.058+0000 7f947a5c6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b171770 con 0x55cb5ae6a1b0 2024-05-27T08:00:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.058+0000 7f946be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae494d0 0x55cb5b20d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.058+0000 7f946be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae494d0 0x55cb5b20d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49990/0 (socket says 172.21.15.63:49990) 2024-05-27T08:00:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.058+0000 7f946be00640 1 -- 172.21.15.63:0/776444337 learned_addr learned my addr 172.21.15.63:0/776444337 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.058+0000 7f9471600640 1 --2- 172.21.15.63:0/776444337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae6a1b0 0x55cb5b20c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.059+0000 7f946be00640 1 -- 172.21.15.63:0/776444337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae6a1b0 msgr2=0x55cb5b20c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.059+0000 7f946be00640 1 --2- 172.21.15.63:0/776444337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae6a1b0 0x55cb5b20c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.059+0000 7f946be00640 1 -- 172.21.15.63:0/776444337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae412f0 msgr2=0x55cb5b20cda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:00:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.059+0000 7f946be00640 1 --2- 172.21.15.63:0/776444337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae412f0 0x55cb5b20cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.059+0000 7f946be00640 1 -- 172.21.15.63:0/776444337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5b1068b0 con 0x55cb5ae494d0 2024-05-27T08:00:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.059+0000 7f946be00640 1 --2- 172.21.15.63:0/776444337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae494d0 0x55cb5b20d2e0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f9458005e30 tx=0x7f945800e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc21977e123e543c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.060+0000 7f945dc00640 1 -- 172.21.15.63:0/776444337 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458004020 con 0x55cb5ae494d0 2024-05-27T08:00:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.060+0000 7f9469600640 1 -- 172.21.15.63:0/3838321372 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f946001a070 con 0x7f946c104400 2024-05-27T08:00:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.060+0000 7f945dc00640 1 -- 172.21.15.63:0/776444337 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94580054b0 con 0x55cb5ae494d0 2024-05-27T08:00:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.060+0000 7f945dc00640 1 -- 172.21.15.63:0/776444337 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458004b50 con 0x55cb5ae494d0 2024-05-27T08:00:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.061+0000 7f947a5c6740 1 -- 172.21.15.63:0/776444337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae494d0 msgr2=0x55cb5b20d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.061+0000 7f947a5c6740 1 --2- 172.21.15.63:0/776444337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae494d0 0x55cb5b20d2e0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f9458005e30 tx=0x7f945800e350 comp rx=0 tx=0).stop 2024-05-27T08:00:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.061+0000 7f947a5c6740 1 -- 172.21.15.63:0/776444337 shutdown_connections 2024-05-27T08:00:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.061+0000 7f947a5c6740 1 --2- 172.21.15.63:0/776444337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae494d0 0x55cb5b20d2e0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.061+0000 7f947a5c6740 1 --2- 172.21.15.63:0/776444337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae412f0 0x55cb5b20cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.061+0000 7f947a5c6740 1 --2- 172.21.15.63:0/776444337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae6a1b0 0x55cb5b20c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.061+0000 7f947a5c6740 1 -- 172.21.15.63:0/776444337 >> 172.21.15.63:0/776444337 conn(0x55cb5ae5e300 msgr2=0x55cb5b209010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.061+0000 7f947a5c6740 1 -- 172.21.15.63:0/776444337 shutdown_connections 2024-05-27T08:00:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.061+0000 7f947a5c6740 1 -- 172.21.15.63:0/776444337 wait complete. 2024-05-27T08:00:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.062+0000 7f947a5c6740 1 -- start start 2024-05-27T08:00:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.062+0000 7f9469600640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f94540b28e0 0x7f94540b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.062+0000 7f947a5c6740 1 Processor -- start 2024-05-27T08:00:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.062+0000 7f9469600640 1 -- 172.21.15.63:0/3838321372 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9571+0+0 (secure 0 0 0) 0x7f94600e2590 con 0x7f946c104400 2024-05-27T08:00:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.063+0000 7f9471600640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f94540b28e0 0x7f94540b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.063+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b216a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.063+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b216fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.063+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae6a1b0 0x55cb5b222cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.063+0000 7f947a5c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cb5ae53790 con 0x55cb5ae412f0 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.063+0000 7f947a5c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b115bc0 con 0x55cb5ae6a1b0 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.063+0000 7f947a5c6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b219990 con 0x55cb5ae494d0 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.063+0000 7f947a5c6740 10 client.0 _ll_register_callbacks cb 0x55cb5b0f3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.063+0000 7f9470c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b216a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f9470c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b216a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50006/0 (socket says 172.21.15.63:50006) 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f9470c00640 1 -- 172.21.15.63:0/4062528024 learned_addr learned my addr 172.21.15.63:0/4062528024 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f9471600640 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae6a1b0 0x55cb5b222cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f946be00640 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b216fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f9471600640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f94540b28e0 0x7f94540b4df0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55cb5b2099d0 tx=0x7f9464009380 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f9470c00640 1 -- 172.21.15.63:0/4062528024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 msgr2=0x55cb5b216fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f9470c00640 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b216fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f9470c00640 1 -- 172.21.15.63:0/4062528024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae6a1b0 msgr2=0x55cb5b222cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f9470c00640 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae6a1b0 0x55cb5b222cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f9470c00640 1 -- 172.21.15.63:0/4062528024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb5ae82210 con 0x55cb5ae412f0 2024-05-27T08:00:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.064+0000 7f946be00640 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b216fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:00:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.065+0000 7f9470c00640 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b216a70 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f946000d740 tx=0x7f9460010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ce79568dca20ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.065+0000 7f9452000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.065+0000 7f9452000640 1 -- 172.21.15.63:0/4062528024 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460021050 con 0x55cb5ae412f0 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.065+0000 7f9452000640 1 -- 172.21.15.63:0/4062528024 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9571+0+0 (secure 0 0 0) 0x7f94600e2150 con 0x55cb5ae412f0 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.065+0000 7f947a5c6740 1 -- 172.21.15.63:0/4062528024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94580054b0 con 0x55cb5ae412f0 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.065+0000 7f9452000640 1 -- 172.21.15.63:0/4062528024 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9460009de0 con 0x55cb5ae412f0 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.065+0000 7f947a5c6740 10 client.20562 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.066+0000 7f947a5c6740 1 -- 172.21.15.63:0/4062528024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cb5af4e620 con 0x55cb5ae412f0 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.066+0000 7f9452000640 1 -- 172.21.15.63:0/4062528024 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94580054b0 con 0x55cb5ae412f0 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.066+0000 7f9452000640 1 -- 172.21.15.63:0/4062528024 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f94600a86b0 con 0x55cb5ae412f0 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.066+0000 7f947a5c6740 10 client.20562 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.066+0000 7f947a5c6740 10 client.20562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.066+0000 7f947a5c6740 20 client.20562 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.066+0000 7f947a5c6740 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb5ae28d80 0x55cb5b228c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.066+0000 7f947a5c6740 4 client.20562 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.066+0000 7f947a5c6740 1 -- 172.21.15.63:0/4062528024 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb5b17fae0 con 0x55cb5ae28d80 2024-05-27T08:00:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.067+0000 7f946be00640 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb5ae28d80 0x55cb5b228c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.068+0000 7f946be00640 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb5ae28d80 0x55cb5b228c80 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96b5c34f937e899d server_cookie=d5d78dac1dd1638e in_seq=0 out_seq=0 2024-05-27T08:00:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.068+0000 7f9452000640 10 client.20562 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.070+0000 7f9452000640 1 -- 172.21.15.63:0/4062528024 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cb5b17fae0 con 0x55cb5ae28d80 2024-05-27T08:00:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.070+0000 7f9452000640 10 client.20562 handle_command_reply: tid=0 2024-05-27T08:00:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.070+0000 7f947a5c6740 1 client.20562 shutdown 2024-05-27T08:00:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 1 -- 172.21.15.63:0/4062528024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 msgr2=0x55cb5b216a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b216a70 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f946000d740 tx=0x7f9460010f00 comp rx=0 tx=0).stop 2024-05-27T08:00:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 1 -- 172.21.15.63:0/4062528024 shutdown_connections 2024-05-27T08:00:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb5ae28d80 0x55cb5b228c80 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae6a1b0 0x55cb5b222cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b216fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 1 --2- 172.21.15.63:0/4062528024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b216a70 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 1 -- 172.21.15.63:0/4062528024 >> 172.21.15.63:0/4062528024 conn(0x55cb5ae5e300 msgr2=0x55cb5b20e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f9452000640 0 client.20562 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 1 -- 172.21.15.63:0/4062528024 shutdown_connections 2024-05-27T08:00:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 1 -- 172.21.15.63:0/4062528024 wait complete. 2024-05-27T08:00:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.071+0000 7f947a5c6740 20 client.20562 trim_cache size 0 max 16384 2024-05-27T08:00:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.092+0000 7f947a5c6740 1 Processor -- start 2024-05-27T08:00:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.092+0000 7f947a5c6740 1 -- start start 2024-05-27T08:00:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae5e300 0x55cb5b216530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b222a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b222f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f947a5c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1b5dc0 con 0x55cb5ae412f0 2024-05-27T08:00:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f947a5c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1b3690 con 0x55cb5ae5e300 2024-05-27T08:00:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f947a5c6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1b1500 con 0x55cb5ae494d0 2024-05-27T08:00:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f946be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b222a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f946be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b222a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50020/0 (socket says 172.21.15.63:50020) 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f946be00640 1 -- 172.21.15.63:0/4176465849 learned_addr learned my addr 172.21.15.63:0/4176465849 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f9471600640 1 --2- 172.21.15.63:0/4176465849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b222f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.093+0000 7f9470c00640 1 --2- 172.21.15.63:0/4176465849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae5e300 0x55cb5b216530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.094+0000 7f946be00640 1 -- 172.21.15.63:0/4176465849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 msgr2=0x55cb5b222f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.094+0000 7f946be00640 1 --2- 172.21.15.63:0/4176465849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b222f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.094+0000 7f946be00640 1 -- 172.21.15.63:0/4176465849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae5e300 msgr2=0x55cb5b216530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.094+0000 7f946be00640 1 --2- 172.21.15.63:0/4176465849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae5e300 0x55cb5b216530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.094+0000 7f946be00640 1 -- 172.21.15.63:0/4176465849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5b2173e0 con 0x55cb5ae412f0 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.094+0000 7f946be00640 1 --2- 172.21.15.63:0/4176465849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b222a30 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f945800c920 tx=0x7f945800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc483884a1d9134 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.094+0000 7f945d200640 1 -- 172.21.15.63:0/4176465849 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945801c950 con 0x55cb5ae412f0 2024-05-27T08:00:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.095+0000 7f945d200640 1 -- 172.21.15.63:0/4176465849 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f945801a460 con 0x55cb5ae412f0 2024-05-27T08:00:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.095+0000 7f945d200640 1 -- 172.21.15.63:0/4176465849 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458010690 con 0x55cb5ae412f0 2024-05-27T08:00:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.096+0000 7f947a5c6740 1 -- 172.21.15.63:0/4176465849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 msgr2=0x55cb5b222a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.096+0000 7f947a5c6740 1 --2- 172.21.15.63:0/4176465849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b222a30 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f945800c920 tx=0x7f945800ef20 comp rx=0 tx=0).stop 2024-05-27T08:00:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.096+0000 7f947a5c6740 1 -- 172.21.15.63:0/4176465849 shutdown_connections 2024-05-27T08:00:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.096+0000 7f947a5c6740 1 --2- 172.21.15.63:0/4176465849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b222f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.096+0000 7f947a5c6740 1 --2- 172.21.15.63:0/4176465849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae412f0 0x55cb5b222a30 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.096+0000 7f947a5c6740 1 --2- 172.21.15.63:0/4176465849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae5e300 0x55cb5b216530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.096+0000 7f947a5c6740 1 -- 172.21.15.63:0/4176465849 >> 172.21.15.63:0/4176465849 conn(0x55cb5b20ee20 msgr2=0x55cb5b218bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.096+0000 7f947a5c6740 1 -- 172.21.15.63:0/4176465849 shutdown_connections 2024-05-27T08:00:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.096+0000 7f947a5c6740 1 -- 172.21.15.63:0/4176465849 wait complete. 2024-05-27T08:00:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.096+0000 7f947a5c6740 1 -- start start 2024-05-27T08:00:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.097+0000 7f947a5c6740 1 Processor -- start 2024-05-27T08:00:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.097+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae412f0 0x55cb5b21a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.097+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b21a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.098+0000 7f947a5c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae5e300 0x55cb5b28f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.098+0000 7f947a5c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1b1b50 con 0x55cb5ae5e300 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.098+0000 7f947a5c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1b2fc0 con 0x55cb5ae412f0 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.098+0000 7f947a5c6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f946c1b2220 con 0x55cb5ae494d0 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.098+0000 7f947a5c6740 10 client.0 _ll_register_callbacks cb 0x7f9460005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.098+0000 7f9471600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae5e300 0x55cb5b28f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.098+0000 7f9471600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae5e300 0x55cb5b28f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50034/0 (socket says 172.21.15.63:50034) 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.098+0000 7f9471600640 1 -- 172.21.15.63:0/817145106 learned_addr learned my addr 172.21.15.63:0/817145106 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.098+0000 7f9470c00640 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae412f0 0x55cb5b21a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.098+0000 7f946be00640 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b21a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.099+0000 7f9471600640 1 -- 172.21.15.63:0/817145106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 msgr2=0x55cb5b21a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.099+0000 7f9471600640 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b21a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.099+0000 7f9471600640 1 -- 172.21.15.63:0/817145106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae412f0 msgr2=0x55cb5b21a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.099+0000 7f9471600640 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae412f0 0x55cb5b21a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.099+0000 7f9471600640 1 -- 172.21.15.63:0/817145106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb5b28f860 con 0x55cb5ae5e300 2024-05-27T08:00:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.099+0000 7f9471600640 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae5e300 0x55cb5b28f320 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f9464009ca0 tx=0x7f946402a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75fc81ae5196846 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.099+0000 7f9452000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.100+0000 7f9452000640 1 -- 172.21.15.63:0/817145106 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464003d00 con 0x55cb5ae5e300 2024-05-27T08:00:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.100+0000 7f9452000640 1 -- 172.21.15.63:0/817145106 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9571+0+0 (secure 0 0 0) 0x7f9464042030 con 0x55cb5ae5e300 2024-05-27T08:00:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.100+0000 7f947a5c6740 1 -- 172.21.15.63:0/817145106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945801a460 con 0x55cb5ae5e300 2024-05-27T08:00:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.100+0000 7f9452000640 1 -- 172.21.15.63:0/817145106 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f946405d050 con 0x55cb5ae5e300 2024-05-27T08:00:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.100+0000 7f947a5c6740 10 client.20568 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.100+0000 7f947a5c6740 1 -- 172.21.15.63:0/817145106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cb5b290a60 con 0x55cb5ae5e300 2024-05-27T08:00:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.100+0000 7f9452000640 1 -- 172.21.15.63:0/817145106 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f945801a460 con 0x55cb5ae5e300 2024-05-27T08:00:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.101+0000 7f9452000640 1 -- 172.21.15.63:0/817145106 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9464054580 con 0x55cb5ae5e300 2024-05-27T08:00:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.101+0000 7f947a5c6740 10 client.20568 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.101+0000 7f947a5c6740 10 client.20568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.101+0000 7f947a5c6740 20 client.20568 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.101+0000 7f947a5c6740 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb5ae6a1b0 0x55cb5b2931f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.101+0000 7f947a5c6740 4 client.20568 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:00:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.101+0000 7f947a5c6740 1 -- 172.21.15.63:0/817145106 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb5b1792c0 con 0x55cb5ae6a1b0 2024-05-27T08:00:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.102+0000 7f9470c00640 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb5ae6a1b0 0x55cb5b2931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.103+0000 7f9470c00640 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb5ae6a1b0 0x55cb5b2931f0 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88940d143d7f5b6b server_cookie=22058c3eac5f2bc3 in_seq=0 out_seq=0 2024-05-27T08:00:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.103+0000 7f9452000640 10 client.20568 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.144+0000 7f9452000640 1 -- 172.21.15.63:0/817145106 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cb5b1792c0 con 0x55cb5ae6a1b0 2024-05-27T08:00:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.144+0000 7f9452000640 10 client.20568 handle_command_reply: tid=0 2024-05-27T08:00:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.144+0000 7f947a5c6740 1 client.20568 shutdown 2024-05-27T08:00:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 1 -- 172.21.15.63:0/817145106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae5e300 msgr2=0x55cb5b28f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae5e300 0x55cb5b28f320 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f9464009ca0 tx=0x7f946402a8b0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 1 -- 172.21.15.63:0/817145106 shutdown_connections 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb5ae6a1b0 0x55cb5b2931f0 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb5ae5e300 0x55cb5b28f320 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb5ae494d0 0x55cb5b21a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 1 --2- 172.21.15.63:0/817145106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb5ae412f0 0x55cb5b21a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 1 -- 172.21.15.63:0/817145106 >> 172.21.15.63:0/817145106 conn(0x55cb5b20ee20 msgr2=0x55cb5b20c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f9452000640 0 client.20568 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 1 -- 172.21.15.63:0/817145106 shutdown_connections 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 1 -- 172.21.15.63:0/817145106 wait complete. 2024-05-27T08:00:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.145+0000 7f947a5c6740 20 client.20568 trim_cache size 0 max 16384 2024-05-27T08:00:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:00:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:00:13.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.146+0000 7f9473600640 1 -- 172.21.15.63:0/3838321372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f94540b28e0 msgr2=0x7f94540b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.146+0000 7f9473600640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f94540b28e0 0x7f94540b4df0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55cb5b2099d0 tx=0x7f9464009380 comp rx=0 tx=0).stop 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.146+0000 7f9473600640 1 -- 172.21.15.63:0/3838321372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946c104400 msgr2=0x7f946c074fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.146+0000 7f9473600640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946c104400 0x7f946c074fc0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f946000d770 tx=0x7f946000dc30 comp rx=0 tx=0).stop 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.146+0000 7f9473600640 1 -- 172.21.15.63:0/3838321372 shutdown_connections 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.146+0000 7f9473600640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f94540b28e0 0x7f94540b4df0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.146+0000 7f9473600640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f946c10a150 0x7f946c075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.146+0000 7f9473600640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f946c109cb0 0x7f946c075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.146+0000 7f9473600640 1 --2- 172.21.15.63:0/3838321372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f946c104400 0x7f946c074fc0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.146+0000 7f9473600640 1 -- 172.21.15.63:0/3838321372 >> 172.21.15.63:0/3838321372 conn(0x7f946c0ff480 msgr2=0x7f946c101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.147+0000 7f9473600640 1 -- 172.21.15.63:0/3838321372 shutdown_connections 2024-05-27T08:00:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:13.147+0000 7f9473600640 1 -- 172.21.15.63:0/3838321372 wait complete. 2024-05-27T08:00:13.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.321+0000 7fc541000700 1 -- 172.21.15.63:0/3481668167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc53c10e6f0 msgr2=0x7fc53c110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.321+0000 7fc541000700 1 --2- 172.21.15.63:0/3481668167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc53c10e6f0 0x7fc53c110bd0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fc53400b0a0 tx=0x7fc53400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.322+0000 7fc541000700 1 -- 172.21.15.63:0/3481668167 shutdown_connections 2024-05-27T08:00:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.322+0000 7fc541000700 1 --2- 172.21.15.63:0/3481668167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc53c10e6f0 0x7fc53c110bd0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.322+0000 7fc541000700 1 --2- 172.21.15.63:0/3481668167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc53c105280 0x7fc53c105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.322+0000 7fc541000700 1 --2- 172.21.15.63:0/3481668167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc53c105bd0 0x7fc53c10e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.322+0000 7fc541000700 1 -- 172.21.15.63:0/3481668167 >> 172.21.15.63:0/3481668167 conn(0x7fc53c078ab0 msgr2=0x7fc53c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.322+0000 7fc541000700 1 -- 172.21.15.63:0/3481668167 shutdown_connections 2024-05-27T08:00:13.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.322+0000 7fc541000700 1 -- 172.21.15.63:0/3481668167 wait complete. 2024-05-27T08:00:13.326 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:13 smithi063 ceph-mon[31613]: pgmap v844: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.3 MiB/s wr, 46 op/s 2024-05-27T08:00:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.323+0000 7fc541000700 1 Processor -- start 2024-05-27T08:00:13.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.323+0000 7fc541000700 1 -- start start 2024-05-27T08:00:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.324+0000 7fc541000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc53c105280 0x7fc53c1060f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.324+0000 7fc541000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc53c105bd0 0x7fc53c106630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.324+0000 7fc541000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc53c10e6f0 0x7fc53c106b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.324+0000 7fc541000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c114100 con 0x7fc53c10e6f0 2024-05-27T08:00:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.324+0000 7fc541000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c113f80 con 0x7fc53c105bd0 2024-05-27T08:00:13.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.324+0000 7fc541000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c114280 con 0x7fc53c105280 2024-05-27T08:00:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.325+0000 7fc53aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc53c105bd0 0x7fc53c106630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.325+0000 7fc53aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc53c105bd0 0x7fc53c106630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:40604/0 (socket says 172.21.15.63:40604) 2024-05-27T08:00:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.325+0000 7fc53aa00700 1 -- 172.21.15.63:0/75080080 learned_addr learned my addr 172.21.15.63:0/75080080 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.325+0000 7fc53b400700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc53c105280 0x7fc53c1060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.325+0000 7fc53be00700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc53c10e6f0 0x7fc53c106b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.326+0000 7fc53aa00700 1 -- 172.21.15.63:0/75080080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc53c105280 msgr2=0x7fc53c1060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.326+0000 7fc53aa00700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc53c105280 0x7fc53c1060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.326+0000 7fc53aa00700 1 -- 172.21.15.63:0/75080080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc53c10e6f0 msgr2=0x7fc53c106b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.326+0000 7fc53aa00700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc53c10e6f0 0x7fc53c106b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.326+0000 7fc53aa00700 1 -- 172.21.15.63:0/75080080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c1ad330 con 0x7fc53c105bd0 2024-05-27T08:00:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.326+0000 7fc53b400700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc53c105280 0x7fc53c1060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:13.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.327+0000 7fc53aa00700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc53c105bd0 0x7fc53c106630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc52800d950 tx=0x7fc52800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.327+0000 7fc533e00700 1 -- 172.21.15.63:0/75080080 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528016070 con 0x7fc53c105bd0 2024-05-27T08:00:13.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.327+0000 7fc541000700 1 -- 172.21.15.63:0/75080080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c1ad560 con 0x7fc53c105bd0 2024-05-27T08:00:13.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.327+0000 7fc533e00700 1 -- 172.21.15.63:0/75080080 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc528009dd0 con 0x7fc53c105bd0 2024-05-27T08:00:13.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.327+0000 7fc533e00700 1 -- 172.21.15.63:0/75080080 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52800f3c0 con 0x7fc53c105bd0 2024-05-27T08:00:13.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.327+0000 7fc541000700 1 -- 172.21.15.63:0/75080080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c1adaf0 con 0x7fc53c105bd0 2024-05-27T08:00:13.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.333+0000 7fc551139b80 1 Processor -- start 2024-05-27T08:00:13.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.333+0000 7fc533e00700 1 -- 172.21.15.63:0/75080080 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc528019070 con 0x7fc53c105bd0 2024-05-27T08:00:13.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.334+0000 7fc551139b80 1 -- start start 2024-05-27T08:00:13.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.334+0000 7fc551139b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d57ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.334+0000 7fc551139b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d62ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.334+0000 7fc551139b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 0x55a3f5d65020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.334+0000 7fc551139b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a3f5c86e40 con 0x55a3f5d58410 2024-05-27T08:00:13.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.334+0000 7fc551139b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a3f5c769e0 con 0x55a3f5d58820 2024-05-27T08:00:13.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.334+0000 7fc551139b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a3f5c7aa20 con 0x55a3f5bd9e40 2024-05-27T08:00:13.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.335+0000 7fc53be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d57ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.335+0000 7fc53b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d62ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.335+0000 7fc53aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 0x55a3f5d65020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.335+0000 7fc53be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d57ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:47684/0 (socket says 172.21.15.63:47684) 2024-05-27T08:00:13.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.335+0000 7fc53be00700 1 -- 172.21.15.63:0/1898862746 learned_addr learned my addr 172.21.15.63:0/1898862746 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:13.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.336+0000 7fc53aa00700 1 -- 172.21.15.63:0/1898862746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 msgr2=0x55a3f5d57ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.336+0000 7fc53aa00700 1 --2- 172.21.15.63:0/1898862746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d57ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.336+0000 7fc53aa00700 1 -- 172.21.15.63:0/1898862746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 msgr2=0x55a3f5d62ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.336+0000 7fc53aa00700 1 --2- 172.21.15.63:0/1898862746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d62ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.336+0000 7fc533e00700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc5200b2a70 0x7fc5200b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.336+0000 7fc53aa00700 1 -- 172.21.15.63:0/1898862746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f5c3f420 con 0x55a3f5d58820 2024-05-27T08:00:13.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.336+0000 7fc53be00700 1 --2- 172.21.15.63:0/1898862746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d57ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:13.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.337+0000 7fc533e00700 1 -- 172.21.15.63:0/75080080 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9571+0+0 (secure 0 0 0) 0x7fc5280e25f0 con 0x7fc53c105bd0 2024-05-27T08:00:13.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.337+0000 7fc53aa00700 1 --2- 172.21.15.63:0/1898862746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 0x55a3f5d65020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc5280df920 tx=0x7fc5280a5b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f394c5eb8a6cd45 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.337+0000 7fc53be00700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc5200b2a70 0x7fc5200b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.338+0000 7fc530c00700 1 -- 172.21.15.63:0/1898862746 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528020030 con 0x55a3f5d58820 2024-05-27T08:00:13.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.338+0000 7fc530c00700 1 -- 172.21.15.63:0/1898862746 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5280a8fc0 con 0x55a3f5d58820 2024-05-27T08:00:13.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.338+0000 7fc530c00700 1 -- 172.21.15.63:0/1898862746 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5280a8e00 con 0x55a3f5d58820 2024-05-27T08:00:13.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.339+0000 7fc551139b80 1 -- 172.21.15.63:0/1898862746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 msgr2=0x55a3f5d65020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.339+0000 7fc551139b80 1 --2- 172.21.15.63:0/1898862746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 0x55a3f5d65020 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc5280df920 tx=0x7fc5280a5b00 comp rx=0 tx=0).stop 2024-05-27T08:00:13.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.339+0000 7fc53be00700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc5200b2a70 0x7fc5200b4f80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55a3f5d54690 tx=0x7fc53400b7d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.339+0000 7fc551139b80 1 -- 172.21.15.63:0/1898862746 shutdown_connections 2024-05-27T08:00:13.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.339+0000 7fc551139b80 1 --2- 172.21.15.63:0/1898862746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 0x55a3f5d65020 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.339+0000 7fc551139b80 1 --2- 172.21.15.63:0/1898862746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d57ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.339+0000 7fc551139b80 1 --2- 172.21.15.63:0/1898862746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d62ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.339+0000 7fc551139b80 1 -- 172.21.15.63:0/1898862746 >> 172.21.15.63:0/1898862746 conn(0x55a3f5c39fc0 msgr2=0x55a3f5d53d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.339+0000 7fc551139b80 1 -- 172.21.15.63:0/1898862746 shutdown_connections 2024-05-27T08:00:13.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.339+0000 7fc551139b80 1 -- 172.21.15.63:0/1898862746 wait complete. 2024-05-27T08:00:13.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.340+0000 7fc551139b80 1 -- start start 2024-05-27T08:00:13.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.340+0000 7fc551139b80 1 Processor -- start 2024-05-27T08:00:13.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.341+0000 7fc551139b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.341+0000 7fc551139b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d62b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.341+0000 7fc551139b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 0x55a3f5d63050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.341+0000 7fc551139b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a3f5b7cfa0 con 0x55a3f5d58410 2024-05-27T08:00:13.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.341+0000 7fc551139b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a3f5c17760 con 0x55a3f5bd9e40 2024-05-27T08:00:13.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.341+0000 7fc551139b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a3f5d65990 con 0x55a3f5d58820 2024-05-27T08:00:13.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.341+0000 7fc551139b80 10 client.0 _ll_register_callbacks cb 0x55a3f5c20c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:13.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.342+0000 7fc53aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d62b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.342+0000 7fc53aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d62b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50070/0 (socket says 172.21.15.63:50070) 2024-05-27T08:00:13.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.342+0000 7fc53aa00700 1 -- 172.21.15.63:0/1039950954 learned_addr learned my addr 172.21.15.63:0/1039950954 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:13.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.342+0000 7fc53b400700 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.342+0000 7fc53be00700 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 0x55a3f5d63050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.342+0000 7fc53aa00700 1 -- 172.21.15.63:0/1039950954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 msgr2=0x55a3f5d63050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.342+0000 7fc53aa00700 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 0x55a3f5d63050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.342+0000 7fc53aa00700 1 -- 172.21.15.63:0/1039950954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5bd9e40 msgr2=0x55a3f5d625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.342+0000 7fc53aa00700 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.342+0000 7fc53aa00700 1 -- 172.21.15.63:0/1039950954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f5bc68d0 con 0x55a3f5d58410 2024-05-27T08:00:13.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.343+0000 7fc53aa00700 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d62b10 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fc5280dff20 tx=0x7fc5280e8790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917754be1cde19ad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.343+0000 7fc525600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:13.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.343+0000 7fc525600700 1 -- 172.21.15.63:0/1039950954 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528020050 con 0x55a3f5d58410 2024-05-27T08:00:13.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.343+0000 7fc525600700 1 -- 172.21.15.63:0/1039950954 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9571+0+0 (secure 0 0 0) 0x7fc5280dfba0 con 0x55a3f5d58410 2024-05-27T08:00:13.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.343+0000 7fc551139b80 1 -- 172.21.15.63:0/1039950954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5280a8fc0 con 0x55a3f5d58410 2024-05-27T08:00:13.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.344+0000 7fc525600700 1 -- 172.21.15.63:0/1039950954 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc5280a9bb0 con 0x55a3f5d58410 2024-05-27T08:00:13.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.344+0000 7fc551139b80 10 client.20580 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:13.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.344+0000 7fc551139b80 1 -- 172.21.15.63:0/1039950954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a3f5c47060 con 0x55a3f5d58410 2024-05-27T08:00:13.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.344+0000 7fc525600700 1 -- 172.21.15.63:0/1039950954 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5280a8fc0 con 0x55a3f5d58410 2024-05-27T08:00:13.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.345+0000 7fc525600700 1 -- 172.21.15.63:0/1039950954 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc5280aa7e0 con 0x55a3f5d58410 2024-05-27T08:00:13.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.345+0000 7fc551139b80 10 client.20580 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:13.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.345+0000 7fc551139b80 10 client.20580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:13.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.345+0000 7fc551139b80 20 client.20580 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:13.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.345+0000 7fc551139b80 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a3f5d72760 0x55a3f5d74b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.345+0000 7fc551139b80 4 client.20580 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.345+0000 7fc551139b80 1 -- 172.21.15.63:0/1039950954 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3f5c88d90 con 0x55a3f5d72760 2024-05-27T08:00:13.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.346+0000 7fc53b400700 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a3f5d72760 0x55a3f5d74b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.348+0000 7fc53b400700 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a3f5d72760 0x55a3f5d74b80 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e465c26217e6547 server_cookie=3c0806e175d8228f in_seq=0 out_seq=0 2024-05-27T08:00:13.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.348+0000 7fc525600700 10 client.20580 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:13.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.351+0000 7fc525600700 1 -- 172.21.15.63:0/1039950954 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a3f5c88d90 con 0x55a3f5d72760 2024-05-27T08:00:13.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.351+0000 7fc525600700 10 client.20580 handle_command_reply: tid=0 2024-05-27T08:00:13.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.351+0000 7fc551139b80 1 client.20580 shutdown 2024-05-27T08:00:13.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.352+0000 7fc551139b80 1 -- 172.21.15.63:0/1039950954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 msgr2=0x55a3f5d62b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.352+0000 7fc551139b80 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d62b10 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fc5280dff20 tx=0x7fc5280e8790 comp rx=0 tx=0).stop 2024-05-27T08:00:13.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.352+0000 7fc551139b80 1 -- 172.21.15.63:0/1039950954 shutdown_connections 2024-05-27T08:00:13.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.352+0000 7fc551139b80 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a3f5d72760 0x55a3f5d74b80 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.352+0000 7fc551139b80 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 0x55a3f5d63050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.352+0000 7fc551139b80 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.352+0000 7fc551139b80 1 --2- 172.21.15.63:0/1039950954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d62b10 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.352+0000 7fc551139b80 1 -- 172.21.15.63:0/1039950954 >> 172.21.15.63:0/1039950954 conn(0x55a3f5c39fc0 msgr2=0x55a3f5d52ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.352+0000 7fc525600700 0 client.20580 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:13.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.353+0000 7fc551139b80 1 -- 172.21.15.63:0/1039950954 shutdown_connections 2024-05-27T08:00:13.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.353+0000 7fc551139b80 1 -- 172.21.15.63:0/1039950954 wait complete. 2024-05-27T08:00:13.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.353+0000 7fc551139b80 20 client.20580 trim_cache size 0 max 16384 2024-05-27T08:00:13.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.387+0000 7fc551139b80 1 Processor -- start 2024-05-27T08:00:13.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.387+0000 7fc551139b80 1 -- start start 2024-05-27T08:00:13.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.387+0000 7fc551139b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d5c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.387+0000 7fc551139b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d53e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.387+0000 7fc551139b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 0x55a3f5d62fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.387+0000 7fc551139b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b92f0 con 0x55a3f5d58410 2024-05-27T08:00:13.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.387+0000 7fc551139b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b6b80 con 0x55a3f5bd9e40 2024-05-27T08:00:13.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.387+0000 7fc551139b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b49f0 con 0x55a3f5d58820 2024-05-27T08:00:13.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d53e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d53e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50080/0 (socket says 172.21.15.63:50080) 2024-05-27T08:00:13.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53aa00700 1 -- 172.21.15.63:0/805231043 learned_addr learned my addr 172.21.15.63:0/805231043 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:13.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53b400700 1 --2- 172.21.15.63:0/805231043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d5c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53be00700 1 --2- 172.21.15.63:0/805231043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 0x55a3f5d62fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53aa00700 1 -- 172.21.15.63:0/805231043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 msgr2=0x55a3f5d62fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53aa00700 1 --2- 172.21.15.63:0/805231043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 0x55a3f5d62fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53aa00700 1 -- 172.21.15.63:0/805231043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5bd9e40 msgr2=0x55a3f5d5c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53aa00700 1 --2- 172.21.15.63:0/805231043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d5c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53aa00700 1 -- 172.21.15.63:0/805231043 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f5d55350 con 0x55a3f5d58410 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.388+0000 7fc53be00700 1 --2- 172.21.15.63:0/805231043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 0x55a3f5d62fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.389+0000 7fc53aa00700 1 --2- 172.21.15.63:0/805231043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d53e30 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fc5280b2360 tx=0x7fc5280a9a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a4dac4f39bbb6d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.389+0000 7fc527e00700 1 -- 172.21.15.63:0/805231043 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528020070 con 0x55a3f5d58410 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.389+0000 7fc527e00700 1 -- 172.21.15.63:0/805231043 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5280e8320 con 0x55a3f5d58410 2024-05-27T08:00:13.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.389+0000 7fc527e00700 1 -- 172.21.15.63:0/805231043 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5280aa5e0 con 0x55a3f5d58410 2024-05-27T08:00:13.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.390+0000 7fc551139b80 1 -- 172.21.15.63:0/805231043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 msgr2=0x55a3f5d53e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.390+0000 7fc551139b80 1 --2- 172.21.15.63:0/805231043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d53e30 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fc5280b2360 tx=0x7fc5280a9a90 comp rx=0 tx=0).stop 2024-05-27T08:00:13.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.390+0000 7fc551139b80 1 -- 172.21.15.63:0/805231043 shutdown_connections 2024-05-27T08:00:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.390+0000 7fc551139b80 1 --2- 172.21.15.63:0/805231043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5d58820 0x55a3f5d62fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.390+0000 7fc551139b80 1 --2- 172.21.15.63:0/805231043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d5c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.390+0000 7fc551139b80 1 --2- 172.21.15.63:0/805231043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d53e30 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.390+0000 7fc551139b80 1 -- 172.21.15.63:0/805231043 >> 172.21.15.63:0/805231043 conn(0x55a3f5c39fc0 msgr2=0x55a3f5d63b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.390+0000 7fc551139b80 1 -- 172.21.15.63:0/805231043 shutdown_connections 2024-05-27T08:00:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.391+0000 7fc551139b80 1 -- 172.21.15.63:0/805231043 wait complete. 2024-05-27T08:00:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.391+0000 7fc551139b80 1 -- start start 2024-05-27T08:00:13.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.391+0000 7fc551139b80 1 Processor -- start 2024-05-27T08:00:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.392+0000 7fc551139b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d77b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.392+0000 7fc551139b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d74f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.392+0000 7fc551139b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 0x55a3f5d75450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.392+0000 7fc551139b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b5040 con 0x55a3f5d58410 2024-05-27T08:00:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.392+0000 7fc551139b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b64b0 con 0x55a3f5d58820 2024-05-27T08:00:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.392+0000 7fc551139b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b5710 con 0x55a3f5bd9e40 2024-05-27T08:00:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.392+0000 7fc551139b80 10 client.0 _ll_register_callbacks cb 0x55a3f5c20c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.392+0000 7fc53aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d74f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.393+0000 7fc53aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d74f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50086/0 (socket says 172.21.15.63:50086) 2024-05-27T08:00:13.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.393+0000 7fc53aa00700 1 -- 172.21.15.63:0/378487587 learned_addr learned my addr 172.21.15.63:0/378487587 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:13.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.393+0000 7fc53be00700 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 0x55a3f5d75450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.393+0000 7fc53b400700 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d77b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.393+0000 7fc53aa00700 1 -- 172.21.15.63:0/378487587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 msgr2=0x55a3f5d77b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.393+0000 7fc53aa00700 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d77b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.393+0000 7fc53aa00700 1 -- 172.21.15.63:0/378487587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 msgr2=0x55a3f5d75450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.393+0000 7fc53aa00700 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 0x55a3f5d75450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.393+0000 7fc53aa00700 1 -- 172.21.15.63:0/378487587 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f5d75990 con 0x55a3f5d58410 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.393+0000 7fc53be00700 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 0x55a3f5d75450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.394+0000 7fc53aa00700 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d74f10 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fc528010fd0 tx=0x7fc5280e86f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf427778ecaacf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.394+0000 7fc525600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.394+0000 7fc525600700 1 -- 172.21.15.63:0/378487587 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528020050 con 0x55a3f5d58410 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.394+0000 7fc525600700 1 -- 172.21.15.63:0/378487587 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 9571+0+0 (secure 0 0 0) 0x7fc5280aa7e0 con 0x55a3f5d58410 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.394+0000 7fc551139b80 1 -- 172.21.15.63:0/378487587 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5280e8320 con 0x55a3f5d58410 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.394+0000 7fc525600700 1 -- 172.21.15.63:0/378487587 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc5280a8c20 con 0x55a3f5d58410 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.394+0000 7fc551139b80 10 client.20586 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.394+0000 7fc551139b80 1 -- 172.21.15.63:0/378487587 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a3f5d75990 con 0x55a3f5d58410 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.395+0000 7fc525600700 1 -- 172.21.15.63:0/378487587 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5280e8320 con 0x55a3f5d58410 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.395+0000 7fc525600700 1 -- 172.21.15.63:0/378487587 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc5280a3c60 con 0x55a3f5d58410 2024-05-27T08:00:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.395+0000 7fc551139b80 10 client.20586 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:13.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.395+0000 7fc551139b80 10 client.20586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:13.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.395+0000 7fc551139b80 20 client.20586 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:13.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.395+0000 7fc551139b80 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a3f5ddce60 0x55a3f5ddf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:13.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.395+0000 7fc551139b80 4 client.20586 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:00:13.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.395+0000 7fc551139b80 1 -- 172.21.15.63:0/378487587 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a3f5c82570 con 0x55a3f5ddce60 2024-05-27T08:00:13.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.396+0000 7fc53b400700 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a3f5ddce60 0x55a3f5ddf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.397+0000 7fc53b400700 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a3f5ddce60 0x55a3f5ddf280 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f377eee73fef514b server_cookie=10b3adb245d2dfe2 in_seq=0 out_seq=0 2024-05-27T08:00:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.397+0000 7fc525600700 10 client.20586 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc525600700 1 -- 172.21.15.63:0/378487587 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a3f5c82570 con 0x55a3f5ddce60 2024-05-27T08:00:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc525600700 10 client.20586 handle_command_reply: tid=0 2024-05-27T08:00:13.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 client.20586 shutdown 2024-05-27T08:00:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 -- 172.21.15.63:0/378487587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 msgr2=0x55a3f5d74f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d74f10 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fc528010fd0 tx=0x7fc5280e86f0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 -- 172.21.15.63:0/378487587 shutdown_connections 2024-05-27T08:00:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a3f5ddce60 0x55a3f5ddf280 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3f5d58820 0x55a3f5d75450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3f5bd9e40 0x55a3f5d77b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 --2- 172.21.15.63:0/378487587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a3f5d58410 0x55a3f5d74f10 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc525600700 0 client.20586 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 -- 172.21.15.63:0/378487587 >> 172.21.15.63:0/378487587 conn(0x55a3f5c39fc0 msgr2=0x55a3f5d62d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 -- 172.21.15.63:0/378487587 shutdown_connections 2024-05-27T08:00:13.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.398+0000 7fc551139b80 1 -- 172.21.15.63:0/378487587 wait complete. 2024-05-27T08:00:13.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.399+0000 7fc551139b80 20 client.20586 trim_cache size 0 max 16384 2024-05-27T08:00:13.402 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:00:13.402 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:00:13.402 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:00:13.402 INFO:teuthology.orchestra.run.smithi063.stdout: "f063ae70-aa3a-4f08-81d3-b4bafc842384": { 2024-05-27T08:00:13.402 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:00:13.402 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "f063ae70-aa3a-4f08-81d3-b4bafc842384", 2024-05-27T08:00:13.402 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:00:13.403 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:00:13.403 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:00:13.403 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:00:13.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 -- 172.21.15.63:0/75080080 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc5200b2a70 msgr2=0x7fc5200b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc5200b2a70 0x7fc5200b4f80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55a3f5d54690 tx=0x7fc53400b7d0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 -- 172.21.15.63:0/75080080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc53c105bd0 msgr2=0x7fc53c106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:13.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc53c105bd0 0x7fc53c106630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc52800d950 tx=0x7fc52800dc60 comp rx=0 tx=0).stop 2024-05-27T08:00:13.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 -- 172.21.15.63:0/75080080 shutdown_connections 2024-05-27T08:00:13.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc5200b2a70 0x7fc5200b4f80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc53c10e6f0 0x7fc53c106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc53c105280 0x7fc53c1060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 --2- 172.21.15.63:0/75080080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc53c105bd0 0x7fc53c106630 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:13.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 -- 172.21.15.63:0/75080080 >> 172.21.15.63:0/75080080 conn(0x7fc53c078ab0 msgr2=0x7fc53c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:13.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 -- 172.21.15.63:0/75080080 shutdown_connections 2024-05-27T08:00:13.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:13.400+0000 7fc541000700 1 -- 172.21.15.63:0/75080080 wait complete. 2024-05-27T08:00:13.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:13 smithi086 ceph-mon[34687]: pgmap v844: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.3 MiB/s wr, 46 op/s 2024-05-27T08:00:13.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:13 smithi149 ceph-mon[35408]: pgmap v844: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.3 MiB/s wr, 46 op/s 2024-05-27T08:00:14.341 INFO:tasks.cephfs.filesystem:scrub status for tag:f063ae70-aa3a-4f08-81d3-b4bafc842384 - {'path': '/', 'tag': 'f063ae70-aa3a-4f08-81d3-b4bafc842384', 'options': 'recursive,force'} 2024-05-27T08:00:15.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:15 smithi086 ceph-mon[34687]: pgmap v845: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.3 MiB/s wr, 46 op/s 2024-05-27T08:00:15.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:15 smithi149 ceph-mon[35408]: pgmap v845: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.3 MiB/s wr, 46 op/s 2024-05-27T08:00:15.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:15 smithi063 ceph-mon[31613]: pgmap v845: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.3 MiB/s wr, 46 op/s 2024-05-27T08:00:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:16 smithi086 ceph-mon[34687]: osdmap e136: 12 total, 12 up, 12 in 2024-05-27T08:00:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:16 smithi086 ceph-mon[34687]: pgmap v847: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.6 MiB/s wr, 48 op/s 2024-05-27T08:00:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:16 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:00:17.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:16 smithi149 ceph-mon[35408]: osdmap e136: 12 total, 12 up, 12 in 2024-05-27T08:00:17.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:16 smithi149 ceph-mon[35408]: pgmap v847: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.6 MiB/s wr, 48 op/s 2024-05-27T08:00:17.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:16 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:00:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:16 smithi063 ceph-mon[31613]: osdmap e136: 12 total, 12 up, 12 in 2024-05-27T08:00:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:16 smithi063 ceph-mon[31613]: pgmap v847: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.6 MiB/s wr, 48 op/s 2024-05-27T08:00:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:16 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:00:18.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:00:18.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:00:18.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:00:18.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:00:18.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:00:18.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:00:19.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:18 smithi086 ceph-mon[34687]: pgmap v848: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:00:19.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:18 smithi149 ceph-mon[35408]: pgmap v848: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:00:19.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:18 smithi063 ceph-mon[31613]: pgmap v848: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:00:21.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:21 smithi086 ceph-mon[34687]: pgmap v849: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:00:21.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:21 smithi149 ceph-mon[35408]: pgmap v849: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:00:21.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:21 smithi063 ceph-mon[31613]: pgmap v849: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:00:23.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:23 smithi086 ceph-mon[34687]: pgmap v850: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:00:23.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:23 smithi149 ceph-mon[35408]: pgmap v850: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:00:23.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:23 smithi063 ceph-mon[31613]: pgmap v850: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:00:25.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:25 smithi086 ceph-mon[34687]: pgmap v851: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:00:25.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:25 smithi149 ceph-mon[35408]: pgmap v851: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:00:25.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:25 smithi063 ceph-mon[31613]: pgmap v851: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:00:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.394+0000 7fc0efe00640 1 -- 172.21.15.63:0/2253912872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f0109f50 msgr2=0x7fc0f0103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.394+0000 7fc0efe00640 1 --2- 172.21.15.63:0/2253912872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f0109f50 0x7fc0f0103be0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009900 tx=0x7fc0dc02a410 comp rx=0 tx=0).stop 2024-05-27T08:00:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.395+0000 7fc0efe00640 1 -- 172.21.15.63:0/2253912872 shutdown_connections 2024-05-27T08:00:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.395+0000 7fc0efe00640 1 --2- 172.21.15.63:0/2253912872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0104250 0x7fc0f01105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.395+0000 7fc0efe00640 1 --2- 172.21.15.63:0/2253912872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f0109f50 0x7fc0f0103be0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.395+0000 7fc0efe00640 1 --2- 172.21.15.63:0/2253912872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0f0109ab0 0x7fc0f01036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.395+0000 7fc0efe00640 1 -- 172.21.15.63:0/2253912872 >> 172.21.15.63:0/2253912872 conn(0x7fc0f0079240 msgr2=0x7fc0f0079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.395+0000 7fc0efe00640 1 -- 172.21.15.63:0/2253912872 shutdown_connections 2024-05-27T08:00:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.395+0000 7fc0efe00640 1 -- 172.21.15.63:0/2253912872 wait complete. 2024-05-27T08:00:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.396+0000 7fc0efe00640 1 Processor -- start 2024-05-27T08:00:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.396+0000 7fc0efe00640 1 -- start start 2024-05-27T08:00:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.397+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0f0104250 0x7fc0f019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.397+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f0109ab0 0x7fc0f019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.397+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0109f50 0x7fc0f019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.397+0000 7fc0efe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f0113b50 con 0x7fc0f0109ab0 2024-05-27T08:00:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.397+0000 7fc0efe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01139d0 con 0x7fc0f0109f50 2024-05-27T08:00:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.397+0000 7fc0efe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f0113cd0 con 0x7fc0f0104250 2024-05-27T08:00:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.397+0000 7fc0ee000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f0109ab0 0x7fc0f019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.397+0000 7fc0ee000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f0109ab0 0x7fc0f019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54796/0 (socket says 172.21.15.63:54796) 2024-05-27T08:00:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.397+0000 7fc0ee000640 1 -- 172.21.15.63:0/2840537964 learned_addr learned my addr 172.21.15.63:0/2840537964 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.397+0000 7fc0eea00640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0f0104250 0x7fc0f019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.398+0000 7fc0ef400640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0109f50 0x7fc0f019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.398+0000 7fc0ee000640 1 -- 172.21.15.63:0/2840537964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0f0104250 msgr2=0x7fc0f019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.398+0000 7fc0ee000640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0f0104250 0x7fc0f019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.398+0000 7fc0ee000640 1 -- 172.21.15.63:0/2840537964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0109f50 msgr2=0x7fc0f019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.398+0000 7fc0ee000640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0109f50 0x7fc0f019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.398+0000 7fc0ee000640 1 -- 172.21.15.63:0/2840537964 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f019f430 con 0x7fc0f0109ab0 2024-05-27T08:00:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.398+0000 7fc0ee000640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f0109ab0 0x7fc0f019abb0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009230 tx=0x7fc0dc0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.399+0000 7fc0e7400640 1 -- 172.21.15.63:0/2840537964 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc03f030 con 0x7fc0f0109ab0 2024-05-27T08:00:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.399+0000 7fc0e7400640 1 -- 172.21.15.63:0/2840537964 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0dc003a80 con 0x7fc0f0109ab0 2024-05-27T08:00:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.399+0000 7fc0e7400640 1 -- 172.21.15.63:0/2840537964 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc035070 con 0x7fc0f0109ab0 2024-05-27T08:00:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.399+0000 7fc0efe00640 1 -- 172.21.15.63:0/2840537964 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f019f6c0 con 0x7fc0f0109ab0 2024-05-27T08:00:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.399+0000 7fc0efe00640 1 -- 172.21.15.63:0/2840537964 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f019fc00 con 0x7fc0f0109ab0 2024-05-27T08:00:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.403+0000 7fc0fca1a740 1 Processor -- start 2024-05-27T08:00:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.403+0000 7fc0fca1a740 1 -- start start 2024-05-27T08:00:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecd32f0 0x55e47f09ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0e7400640 1 -- 172.21.15.63:0/2840537964 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc0dc042030 con 0x7fc0f0109ab0 2024-05-27T08:00:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecdb4d0 0x55e47f09f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e47f00fb90 con 0x55e47ecfc1b0 2024-05-27T08:00:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e47efff730 con 0x55e47ecd32f0 2024-05-27T08:00:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e47f003770 con 0x55e47ecdb4d0 2024-05-27T08:00:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0ef400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0ef400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54804/0 (socket says 172.21.15.63:54804) 2024-05-27T08:00:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0ef400640 1 -- 172.21.15.63:0/3982298302 learned_addr learned my addr 172.21.15.63:0/3982298302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.404+0000 7fc0ee000640 1 --2- 172.21.15.63:0/3982298302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecdb4d0 0x55e47f09f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.405+0000 7fc0eea00640 1 --2- 172.21.15.63:0/3982298302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecd32f0 0x55e47f09ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.405+0000 7fc0ef400640 1 -- 172.21.15.63:0/3982298302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecdb4d0 msgr2=0x55e47f09f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.405+0000 7fc0ef400640 1 --2- 172.21.15.63:0/3982298302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecdb4d0 0x55e47f09f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.405+0000 7fc0ef400640 1 -- 172.21.15.63:0/3982298302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecd32f0 msgr2=0x55e47f09ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.405+0000 7fc0ef400640 1 --2- 172.21.15.63:0/3982298302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecd32f0 0x55e47f09ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.405+0000 7fc0ef400640 1 -- 172.21.15.63:0/3982298302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47ef988b0 con 0x55e47ecfc1b0 2024-05-27T08:00:26.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.405+0000 7fc0ef400640 1 --2- 172.21.15.63:0/3982298302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09e8f0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8007e50 tx=0x7fc0e800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8512a49704a143 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:26.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.406+0000 7fc0dbe00640 1 -- 172.21.15.63:0/3982298302 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8014070 con 0x55e47ecfc1b0 2024-05-27T08:00:26.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.406+0000 7fc0dbe00640 1 -- 172.21.15.63:0/3982298302 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0e800ee90 con 0x55e47ecfc1b0 2024-05-27T08:00:26.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.406+0000 7fc0e7400640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d40b29d0 0x7fc0d40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.406+0000 7fc0dbe00640 1 -- 172.21.15.63:0/3982298302 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8005310 con 0x55e47ecfc1b0 2024-05-27T08:00:26.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.406+0000 7fc0e7400640 1 -- 172.21.15.63:0/2840537964 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7fc0dc102ea0 con 0x7fc0f0109ab0 2024-05-27T08:00:26.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.406+0000 7fc0eea00640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d40b29d0 0x7fc0d40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.407+0000 7fc0eea00640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d40b29d0 0x7fc0d40b4ee0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55e47f09c2c0 tx=0x7fc0e000b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.407+0000 7fc0fca1a740 1 -- 172.21.15.63:0/3982298302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 msgr2=0x55e47f09e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.407+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/3982298302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09e8f0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8007e50 tx=0x7fc0e800e720 comp rx=0 tx=0).stop 2024-05-27T08:00:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.407+0000 7fc0fca1a740 1 -- 172.21.15.63:0/3982298302 shutdown_connections 2024-05-27T08:00:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.407+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/3982298302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecdb4d0 0x55e47f09f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.407+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/3982298302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecd32f0 0x55e47f09ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.407+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/3982298302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09e8f0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.407+0000 7fc0fca1a740 1 -- 172.21.15.63:0/3982298302 >> 172.21.15.63:0/3982298302 conn(0x55e47ecf0300 msgr2=0x55e47f09b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.407+0000 7fc0fca1a740 1 -- 172.21.15.63:0/3982298302 shutdown_connections 2024-05-27T08:00:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.407+0000 7fc0fca1a740 1 -- 172.21.15.63:0/3982298302 wait complete. 2024-05-27T08:00:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.408+0000 7fc0fca1a740 1 -- start start 2024-05-27T08:00:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.408+0000 7fc0fca1a740 1 Processor -- start 2024-05-27T08:00:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecd32f0 0x55e47f0ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecdb4d0 0x55e47f0ad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f0b9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e47ece5790 con 0x55e47ecfc1b0 2024-05-27T08:00:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e47efa7bc0 con 0x55e47ecdb4d0 2024-05-27T08:00:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e47f0b0000 con 0x55e47ecd32f0 2024-05-27T08:00:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0fca1a740 10 client.0 _ll_register_callbacks cb 0x55e47ef85400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0ef400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f0b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0ef400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f0b9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54814/0 (socket says 172.21.15.63:54814) 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0ef400640 1 -- 172.21.15.63:0/2631448954 learned_addr learned my addr 172.21.15.63:0/2631448954 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0eea00640 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecd32f0 0x55e47f0ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.409+0000 7fc0ee000640 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecdb4d0 0x55e47f0ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.410+0000 7fc0ef400640 1 -- 172.21.15.63:0/2631448954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecd32f0 msgr2=0x55e47f0ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.410+0000 7fc0ef400640 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecd32f0 0x55e47f0ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.410+0000 7fc0ef400640 1 -- 172.21.15.63:0/2631448954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecdb4d0 msgr2=0x55e47f0ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.410+0000 7fc0ef400640 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecdb4d0 0x55e47f0ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.410+0000 7fc0ef400640 1 -- 172.21.15.63:0/2631448954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47ed14210 con 0x55e47ecfc1b0 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.410+0000 7fc0ef400640 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f0b9330 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8000f80 tx=0x7fc0e8010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bca89036ca92624 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.410+0000 7fc0d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.410+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2631448954 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8027060 con 0x55e47ecfc1b0 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.410+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2631448954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e800ee90 con 0x55e47ecfc1b0 2024-05-27T08:00:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.410+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2631448954 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7fc0e80109c0 con 0x55e47ecfc1b0 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.411+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2631448954 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc0e802e050 con 0x55e47ecfc1b0 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.411+0000 7fc0fca1a740 10 client.20598 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.411+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2631448954 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e47ede0620 con 0x55e47ecfc1b0 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.411+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2631448954 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0e800ee90 con 0x55e47ecfc1b0 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.411+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2631448954 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc0e80184c0 con 0x55e47ecfc1b0 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.411+0000 7fc0fca1a740 10 client.20598 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.411+0000 7fc0fca1a740 10 client.20598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.412+0000 7fc0fca1a740 20 client.20598 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.412+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e47ecbad80 0x55e47f0bf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.412+0000 7fc0fca1a740 4 client.20598 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.412+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2631448954 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e47f011ae0 con 0x55e47ecbad80 2024-05-27T08:00:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.412+0000 7fc0eea00640 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e47ecbad80 0x55e47f0bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.413+0000 7fc0eea00640 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e47ecbad80 0x55e47f0bf290 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad85911e81e14d7f server_cookie=78963a6dbb54f1bc in_seq=0 out_seq=0 2024-05-27T08:00:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.413+0000 7fc0d8c00640 10 client.20598 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.415+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2631448954 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e47f011ae0 con 0x55e47ecbad80 2024-05-27T08:00:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.415+0000 7fc0d8c00640 10 client.20598 handle_command_reply: tid=0 2024-05-27T08:00:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.415+0000 7fc0fca1a740 1 client.20598 shutdown 2024-05-27T08:00:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2631448954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 msgr2=0x55e47f0b9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f0b9330 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8000f80 tx=0x7fc0e8010640 comp rx=0 tx=0).stop 2024-05-27T08:00:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2631448954 shutdown_connections 2024-05-27T08:00:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e47ecbad80 0x55e47f0bf290 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f0b9330 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecdb4d0 0x55e47f0ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2631448954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecd32f0 0x55e47f0ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2631448954 >> 172.21.15.63:0/2631448954 conn(0x55e47ecf0300 msgr2=0x55e47f0a06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0d8c00640 0 client.20598 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2631448954 shutdown_connections 2024-05-27T08:00:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2631448954 wait complete. 2024-05-27T08:00:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.416+0000 7fc0fca1a740 20 client.20598 trim_cache size 0 max 16384 2024-05-27T08:00:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.434+0000 7fc0fca1a740 1 Processor -- start 2024-05-27T08:00:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.434+0000 7fc0fca1a740 1 -- start start 2024-05-27T08:00:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.434+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecd32f0 0x55e47f0af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.434+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecdb4d0 0x55e47f0b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.434+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.434+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01b1760 con 0x55e47ecfc1b0 2024-05-27T08:00:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.434+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01af030 con 0x55e47ecdb4d0 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.434+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01acea0 con 0x55e47ecd32f0 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.435+0000 7fc0ef400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.435+0000 7fc0ef400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54828/0 (socket says 172.21.15.63:54828) 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.435+0000 7fc0ef400640 1 -- 172.21.15.63:0/2955425195 learned_addr learned my addr 172.21.15.63:0/2955425195 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.435+0000 7fc0ee000640 1 --2- 172.21.15.63:0/2955425195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecdb4d0 0x55e47f0b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.435+0000 7fc0eea00640 1 --2- 172.21.15.63:0/2955425195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecd32f0 0x55e47f0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.435+0000 7fc0ef400640 1 -- 172.21.15.63:0/2955425195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecd32f0 msgr2=0x55e47f0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.435+0000 7fc0ef400640 1 --2- 172.21.15.63:0/2955425195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecd32f0 0x55e47f0af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.435+0000 7fc0ef400640 1 -- 172.21.15.63:0/2955425195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecdb4d0 msgr2=0x55e47f0b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.435+0000 7fc0ef400640 1 --2- 172.21.15.63:0/2955425195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecdb4d0 0x55e47f0b92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.435+0000 7fc0ef400640 1 -- 172.21.15.63:0/2955425195 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47f09c0f0 con 0x55e47ecfc1b0 2024-05-27T08:00:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.436+0000 7fc0ef400640 1 --2- 172.21.15.63:0/2955425195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09ef00 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80188c0 tx=0x7fc0e8019f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01b62266b27db0e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.436+0000 7fc0db400640 1 -- 172.21.15.63:0/2955425195 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8037070 con 0x55e47ecfc1b0 2024-05-27T08:00:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.436+0000 7fc0db400640 1 -- 172.21.15.63:0/2955425195 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0e8003e00 con 0x55e47ecfc1b0 2024-05-27T08:00:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.436+0000 7fc0db400640 1 -- 172.21.15.63:0/2955425195 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8027040 con 0x55e47ecfc1b0 2024-05-27T08:00:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.437+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2955425195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 msgr2=0x55e47f09ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.437+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2955425195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09ef00 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80188c0 tx=0x7fc0e8019f40 comp rx=0 tx=0).stop 2024-05-27T08:00:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.437+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2955425195 shutdown_connections 2024-05-27T08:00:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.437+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2955425195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecfc1b0 0x55e47f09ef00 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.437+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2955425195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecdb4d0 0x55e47f0b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.437+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2955425195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecd32f0 0x55e47f0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.437+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2955425195 >> 172.21.15.63:0/2955425195 conn(0x55e47ecf0300 msgr2=0x55e47f09e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.437+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2955425195 shutdown_connections 2024-05-27T08:00:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.437+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2955425195 wait complete. 2024-05-27T08:00:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.438+0000 7fc0fca1a740 1 -- start start 2024-05-27T08:00:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.438+0000 7fc0fca1a740 1 Processor -- start 2024-05-27T08:00:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.439+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecd32f0 0x55e47f0beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.439+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecdb4d0 0x55e47f0bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.439+0000 7fc0fca1a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecfc1b0 0x55e47f0bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.439+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01ad4f0 con 0x55e47ecdb4d0 2024-05-27T08:00:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.439+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01ae960 con 0x55e47ecd32f0 2024-05-27T08:00:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.439+0000 7fc0fca1a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01adbc0 con 0x55e47ecfc1b0 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.439+0000 7fc0fca1a740 10 client.0 _ll_register_callbacks cb 0x7fc0e8007ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.439+0000 7fc0ee000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecdb4d0 0x55e47f0bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.439+0000 7fc0ee000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecdb4d0 0x55e47f0bf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54832/0 (socket says 172.21.15.63:54832) 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.439+0000 7fc0ee000640 1 -- 172.21.15.63:0/2962744389 learned_addr learned my addr 172.21.15.63:0/2962744389 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.440+0000 7fc0eea00640 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecd32f0 0x55e47f0beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.440+0000 7fc0ef400640 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecfc1b0 0x55e47f0bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.440+0000 7fc0ee000640 1 -- 172.21.15.63:0/2962744389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecfc1b0 msgr2=0x55e47f0bfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.440+0000 7fc0ee000640 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecfc1b0 0x55e47f0bfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.440+0000 7fc0ee000640 1 -- 172.21.15.63:0/2962744389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecd32f0 msgr2=0x55e47f0beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.440+0000 7fc0ee000640 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecd32f0 0x55e47f0beff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.440+0000 7fc0ee000640 1 -- 172.21.15.63:0/2962744389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47f0a1670 con 0x55e47ecdb4d0 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.440+0000 7fc0ef400640 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecfc1b0 0x55e47f0bfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:00:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.440+0000 7fc0ee000640 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecdb4d0 0x55e47f0bf530 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc005940 tx=0x7fc0dc0c5ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b3b0f3477e13ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.440+0000 7fc0d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.441+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2962744389 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc049040 con 0x55e47ecdb4d0 2024-05-27T08:00:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.441+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2962744389 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7fc0dc0e43e0 con 0x55e47ecdb4d0 2024-05-27T08:00:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.441+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2962744389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e8003e00 con 0x55e47ecdb4d0 2024-05-27T08:00:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.441+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2962744389 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc0dc0c94d0 con 0x55e47ecdb4d0 2024-05-27T08:00:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.441+0000 7fc0fca1a740 10 client.20604 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.441+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2962744389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e47f126710 con 0x55e47ecdb4d0 2024-05-27T08:00:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.441+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2962744389 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0e8003e00 con 0x55e47ecdb4d0 2024-05-27T08:00:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.441+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2962744389 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc0dc0c9fc0 con 0x55e47ecdb4d0 2024-05-27T08:00:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.442+0000 7fc0fca1a740 10 client.20604 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.442+0000 7fc0fca1a740 10 client.20604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.442+0000 7fc0fca1a740 20 client.20604 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.442+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e47ec901c0 0x55e47f128ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.442+0000 7fc0fca1a740 4 client.20604 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:00:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.442+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2962744389 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e47f00b2c0 con 0x55e47ec901c0 2024-05-27T08:00:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.442+0000 7fc0eea00640 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e47ec901c0 0x55e47f128ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.443+0000 7fc0d8c00640 10 client.20604 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.443+0000 7fc0eea00640 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e47ec901c0 0x55e47f128ed0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c681e05a575c441f server_cookie=382454cdd1e6aa6f in_seq=0 out_seq=0 2024-05-27T08:00:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0d8c00640 1 -- 172.21.15.63:0/2962744389 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e47f00b2c0 con 0x55e47ec901c0 2024-05-27T08:00:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0d8c00640 10 client.20604 handle_command_reply: tid=0 2024-05-27T08:00:26.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0fca1a740 1 client.20604 shutdown 2024-05-27T08:00:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2962744389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecdb4d0 msgr2=0x55e47f0bf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecdb4d0 0x55e47f0bf530 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc005940 tx=0x7fc0dc0c5ce0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2962744389 shutdown_connections 2024-05-27T08:00:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e47ec901c0 0x55e47f128ed0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e47ecfc1b0 0x55e47f0bfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e47ecdb4d0 0x55e47f0bf530 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0fca1a740 1 --2- 172.21.15.63:0/2962744389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e47ecd32f0 0x55e47f0beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2962744389 >> 172.21.15.63:0/2962744389 conn(0x55e47ecf0300 msgr2=0x55e47f09f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.482+0000 7fc0d8c00640 0 client.20604 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.483+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2962744389 shutdown_connections 2024-05-27T08:00:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.483+0000 7fc0fca1a740 1 -- 172.21.15.63:0/2962744389 wait complete. 2024-05-27T08:00:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.483+0000 7fc0fca1a740 20 client.20604 trim_cache size 0 max 16384 2024-05-27T08:00:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:00:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:00:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:00:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 -- 172.21.15.63:0/2840537964 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d40b29d0 msgr2=0x7fc0d40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d40b29d0 0x7fc0d40b4ee0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55e47f09c2c0 tx=0x7fc0e000b390 comp rx=0 tx=0).stop 2024-05-27T08:00:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 -- 172.21.15.63:0/2840537964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f0109ab0 msgr2=0x7fc0f019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f0109ab0 0x7fc0f019abb0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009230 tx=0x7fc0dc0040e0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 -- 172.21.15.63:0/2840537964 shutdown_connections 2024-05-27T08:00:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0d40b29d0 0x7fc0d40b4ee0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0f0109f50 0x7fc0f019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f0109ab0 0x7fc0f019abb0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 --2- 172.21.15.63:0/2840537964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0f0104250 0x7fc0f019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 -- 172.21.15.63:0/2840537964 >> 172.21.15.63:0/2840537964 conn(0x7fc0f0079240 msgr2=0x7fc0f0102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.484+0000 7fc0efe00640 1 -- 172.21.15.63:0/2840537964 shutdown_connections 2024-05-27T08:00:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:26.485+0000 7fc0efe00640 1 -- 172.21.15.63:0/2840537964 wait complete. 2024-05-27T08:00:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:27 smithi086 ceph-mon[34687]: pgmap v852: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:00:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:27 smithi149 ceph-mon[35408]: pgmap v852: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:00:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:27 smithi063 ceph-mon[31613]: pgmap v852: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:00:29.518 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:29 smithi149 ceph-mon[35408]: pgmap v853: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T08:00:29.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:29 smithi086 ceph-mon[34687]: pgmap v853: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T08:00:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:29 smithi063 ceph-mon[31613]: pgmap v853: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 17 op/s 2024-05-27T08:00:31.520 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:31 smithi063 ceph-mon[31613]: pgmap v854: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T08:00:31.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:31 smithi086 ceph-mon[34687]: pgmap v854: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T08:00:31.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:31 smithi149 ceph-mon[35408]: pgmap v854: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.3 MiB/s wr, 10 op/s 2024-05-27T08:00:33.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:33 smithi086 ceph-mon[34687]: pgmap v855: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T08:00:33.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:33 smithi149 ceph-mon[35408]: pgmap v855: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T08:00:33.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:33 smithi063 ceph-mon[31613]: pgmap v855: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T08:00:35.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:35 smithi086 ceph-mon[34687]: pgmap v856: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:00:35.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:35 smithi149 ceph-mon[35408]: pgmap v856: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:00:35.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:35 smithi063 ceph-mon[31613]: pgmap v856: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:00:37.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:37 smithi086 ceph-mon[34687]: pgmap v857: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:00:37.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:37 smithi149 ceph-mon[35408]: pgmap v857: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:00:37.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:37 smithi063 ceph-mon[31613]: pgmap v857: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:00:39.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:39 smithi086 ceph-mon[34687]: pgmap v858: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:00:39.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:39 smithi149 ceph-mon[35408]: pgmap v858: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:00:39.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:39 smithi063 ceph-mon[31613]: pgmap v858: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:00:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.731+0000 7f4e63e00640 1 -- 172.21.15.63:0/3569792642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e6410f3c0 msgr2=0x7f4e64116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.731+0000 7f4e63e00640 1 --2- 172.21.15.63:0/3569792642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e6410f3c0 0x7f4e64116b60 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009ca0 tx=0x7f4e5c02a8c0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.732+0000 7f4e63e00640 1 -- 172.21.15.63:0/3569792642 shutdown_connections 2024-05-27T08:00:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.732+0000 7f4e63e00640 1 --2- 172.21.15.63:0/3569792642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e6410f3c0 0x7f4e64116b60 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.732+0000 7f4e63e00640 1 --2- 172.21.15.63:0/3569792642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6410e900 0x7f4e6410ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.732+0000 7f4e63e00640 1 --2- 172.21.15.63:0/3569792642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e6410a390 0x7f4e6410a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.732+0000 7f4e63e00640 1 -- 172.21.15.63:0/3569792642 >> 172.21.15.63:0/3569792642 conn(0x7f4e64076070 msgr2=0x7f4e64076480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.732+0000 7f4e63e00640 1 -- 172.21.15.63:0/3569792642 shutdown_connections 2024-05-27T08:00:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.732+0000 7f4e63e00640 1 -- 172.21.15.63:0/3569792642 wait complete. 2024-05-27T08:00:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.733+0000 7f4e63e00640 1 Processor -- start 2024-05-27T08:00:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.733+0000 7f4e63e00640 1 -- start start 2024-05-27T08:00:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.733+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e6410a390 0x7f4e641a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e6410e900 0x7f4e641a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6410f3c0 0x7f4e641a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e63e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6411a1e0 con 0x7f4e6410a390 2024-05-27T08:00:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e63e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6411a060 con 0x7f4e6410e900 2024-05-27T08:00:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e63e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6411a360 con 0x7f4e6410f3c0 2024-05-27T08:00:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e62a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e6410a390 0x7f4e641a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e62a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e6410a390 0x7f4e641a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36624/0 (socket says 172.21.15.63:36624) 2024-05-27T08:00:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e62a00640 1 -- 172.21.15.63:0/249559859 learned_addr learned my addr 172.21.15.63:0/249559859 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e5a000640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e6410e900 0x7f4e641a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e63400640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6410f3c0 0x7f4e641a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e62a00640 1 -- 172.21.15.63:0/249559859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6410f3c0 msgr2=0x7f4e641a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e62a00640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6410f3c0 0x7f4e641a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e62a00640 1 -- 172.21.15.63:0/249559859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e6410e900 msgr2=0x7f4e641a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e62a00640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e6410e900 0x7f4e641a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.734+0000 7f4e62a00640 1 -- 172.21.15.63:0/249559859 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e641a1060 con 0x7f4e6410a390 2024-05-27T08:00:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.735+0000 7f4e63400640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6410f3c0 0x7f4e641a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:00:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.735+0000 7f4e62a00640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e6410a390 0x7f4e641a67b0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400d770 tx=0x7f4e5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.735+0000 7f4e5be00640 1 -- 172.21.15.63:0/249559859 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54017070 con 0x7f4e6410a390 2024-05-27T08:00:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.735+0000 7f4e5be00640 1 -- 172.21.15.63:0/249559859 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e54009c30 con 0x7f4e6410a390 2024-05-27T08:00:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.735+0000 7f4e63e00640 1 -- 172.21.15.63:0/249559859 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e641a1350 con 0x7f4e6410a390 2024-05-27T08:00:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.735+0000 7f4e5be00640 1 -- 172.21.15.63:0/249559859 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400f620 con 0x7f4e6410a390 2024-05-27T08:00:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.735+0000 7f4e63e00640 1 -- 172.21.15.63:0/249559859 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e641ad630 con 0x7f4e6410a390 2024-05-27T08:00:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.739+0000 7f4e5be00640 1 -- 172.21.15.63:0/249559859 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4e5401a070 con 0x7f4e6410a390 2024-05-27T08:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.740+0000 7f4e70722740 1 Processor -- start 2024-05-27T08:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.740+0000 7f4e70722740 1 -- start start 2024-05-27T08:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.740+0000 7f4e70722740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 0x5561647a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.740+0000 7f4e70722740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.740+0000 7f4e70722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.740+0000 7f4e70722740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556164711b90 con 0x5561643d52f0 2024-05-27T08:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.740+0000 7f4e70722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556164701730 con 0x5561643dd4d0 2024-05-27T08:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.740+0000 7f4e70722740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556164705770 con 0x5561643fe1b0 2024-05-27T08:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.741+0000 7f4e62a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.741+0000 7f4e62a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36638/0 (socket says 172.21.15.63:36638) 2024-05-27T08:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.741+0000 7f4e62a00640 1 -- 172.21.15.63:0/1712736101 learned_addr learned my addr 172.21.15.63:0/1712736101 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.741+0000 7f4e5a000640 1 --2- 172.21.15.63:0/1712736101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.741+0000 7f4e63400640 1 --2- 172.21.15.63:0/1712736101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 0x5561647a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.742+0000 7f4e62a00640 1 -- 172.21.15.63:0/1712736101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 msgr2=0x5561647a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.742+0000 7f4e62a00640 1 --2- 172.21.15.63:0/1712736101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 0x5561647a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.742+0000 7f4e62a00640 1 -- 172.21.15.63:0/1712736101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 msgr2=0x5561647a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.742+0000 7f4e62a00640 1 --2- 172.21.15.63:0/1712736101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.742+0000 7f4e62a00640 1 -- 172.21.15.63:0/1712736101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55616469a8b0 con 0x5561643d52f0 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.742+0000 7f4e5be00640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e480b29d0 0x7f4e480b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.742+0000 7f4e62a00640 1 --2- 172.21.15.63:0/1712736101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647a0fa0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f4e540df7e0 tx=0x7f4e540e8b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0faeea2f0e0fdd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.742+0000 7f4e63400640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e480b29d0 0x7f4e480b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.742+0000 7f4e5be00640 1 -- 172.21.15.63:0/249559859 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f4e540e2570 con 0x7f4e6410a390 2024-05-27T08:00:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.743+0000 7f4e53e00640 1 -- 172.21.15.63:0/1712736101 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e540a52f0 con 0x5561643d52f0 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.743+0000 7f4e53e00640 1 -- 172.21.15.63:0/1712736101 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e54010e40 con 0x5561643d52f0 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.743+0000 7f4e53e00640 1 -- 172.21.15.63:0/1712736101 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e540a8ba0 con 0x5561643d52f0 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.744+0000 7f4e70722740 1 -- 172.21.15.63:0/1712736101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 msgr2=0x5561647a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.744+0000 7f4e63400640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e480b29d0 0x7f4e480b4ee0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c006010 tx=0x7f4e5c004a40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.744+0000 7f4e70722740 1 --2- 172.21.15.63:0/1712736101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647a0fa0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f4e540df7e0 tx=0x7f4e540e8b40 comp rx=0 tx=0).stop 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.744+0000 7f4e70722740 1 -- 172.21.15.63:0/1712736101 shutdown_connections 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.744+0000 7f4e70722740 1 --2- 172.21.15.63:0/1712736101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.744+0000 7f4e70722740 1 --2- 172.21.15.63:0/1712736101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647a0fa0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.744+0000 7f4e70722740 1 --2- 172.21.15.63:0/1712736101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 0x5561647a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.744+0000 7f4e70722740 1 -- 172.21.15.63:0/1712736101 >> 172.21.15.63:0/1712736101 conn(0x5561643f2300 msgr2=0x55616479d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.744+0000 7f4e70722740 1 -- 172.21.15.63:0/1712736101 shutdown_connections 2024-05-27T08:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.744+0000 7f4e70722740 1 -- 172.21.15.63:0/1712736101 wait complete. 2024-05-27T08:00:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.745+0000 7f4e70722740 1 -- start start 2024-05-27T08:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.745+0000 7f4e70722740 1 Processor -- start 2024-05-27T08:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e70722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643d52f0 0x5561647aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e70722740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643dd4d0 0x5561647ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e70722740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e70722740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5561643e7790 con 0x5561643fe1b0 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e70722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561646a9bc0 con 0x5561643d52f0 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e70722740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5561647adb90 con 0x5561643dd4d0 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e70722740 10 client.0 _ll_register_callbacks cb 0x556164687400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e63400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e63400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36654/0 (socket says 172.21.15.63:36654) 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e63400640 1 -- 172.21.15.63:0/1831149560 learned_addr learned my addr 172.21.15.63:0/1831149560 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e5a000640 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643dd4d0 0x5561647ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.746+0000 7f4e62a00640 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643d52f0 0x5561647aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.747+0000 7f4e63400640 1 -- 172.21.15.63:0/1831149560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643dd4d0 msgr2=0x5561647ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.747+0000 7f4e63400640 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643dd4d0 0x5561647ab1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.747+0000 7f4e63400640 1 -- 172.21.15.63:0/1831149560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643d52f0 msgr2=0x5561647aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.747+0000 7f4e63400640 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643d52f0 0x5561647aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.747+0000 7f4e63400640 1 -- 172.21.15.63:0/1831149560 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556164416210 con 0x5561643fe1b0 2024-05-27T08:00:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.747+0000 7f4e63400640 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b6ec0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c035000 tx=0x7f4e5c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc655f49d51337d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.747+0000 7f4e50c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.747+0000 7f4e50c00640 1 -- 172.21.15.63:0/1831149560 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c004310 con 0x5561643fe1b0 2024-05-27T08:00:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.748+0000 7f4e50c00640 1 -- 172.21.15.63:0/1831149560 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f4e5c0044b0 con 0x5561643fe1b0 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.748+0000 7f4e70722740 1 -- 172.21.15.63:0/1831149560 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556164713ae0 con 0x5561643fe1b0 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.748+0000 7f4e50c00640 1 -- 172.21.15.63:0/1831149560 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4e5c0046b0 con 0x5561643fe1b0 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.748+0000 7f4e70722740 10 client.20616 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.748+0000 7f4e70722740 1 -- 172.21.15.63:0/1831149560 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5561644e2620 con 0x5561643fe1b0 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.748+0000 7f4e50c00640 1 -- 172.21.15.63:0/1831149560 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556164713ae0 con 0x5561643fe1b0 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.749+0000 7f4e50c00640 1 -- 172.21.15.63:0/1831149560 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4e5c050070 con 0x5561643fe1b0 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.749+0000 7f4e70722740 10 client.20616 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.749+0000 7f4e70722740 10 client.20616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.749+0000 7f4e70722740 20 client.20616 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.749+0000 7f4e70722740 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561643bcd80 0x5561647bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.749+0000 7f4e70722740 4 client.20616 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.749+0000 7f4e70722740 1 -- 172.21.15.63:0/1831149560 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55616470d2c0 con 0x5561643bcd80 2024-05-27T08:00:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.749+0000 7f4e62a00640 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561643bcd80 0x5561647bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.750+0000 7f4e62a00640 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561643bcd80 0x5561647bcf40 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18d8d8fea5b991da server_cookie=ea2bc6cc9765def3 in_seq=0 out_seq=0 2024-05-27T08:00:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.750+0000 7f4e50c00640 10 client.20616 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.752+0000 7f4e50c00640 1 -- 172.21.15.63:0/1831149560 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55616470d2c0 con 0x5561643bcd80 2024-05-27T08:00:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.752+0000 7f4e50c00640 10 client.20616 handle_command_reply: tid=0 2024-05-27T08:00:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.752+0000 7f4e70722740 1 client.20616 shutdown 2024-05-27T08:00:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 1 -- 172.21.15.63:0/1831149560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 msgr2=0x5561647b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b6ec0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c035000 tx=0x7f4e5c02a8c0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 1 -- 172.21.15.63:0/1831149560 shutdown_connections 2024-05-27T08:00:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561643bcd80 0x5561647bcf40 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b6ec0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643dd4d0 0x5561647ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 1 --2- 172.21.15.63:0/1831149560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643d52f0 0x5561647aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e50c00640 0 client.20616 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 1 -- 172.21.15.63:0/1831149560 >> 172.21.15.63:0/1831149560 conn(0x5561643f2300 msgr2=0x5561647a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 1 -- 172.21.15.63:0/1831149560 shutdown_connections 2024-05-27T08:00:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 1 -- 172.21.15.63:0/1831149560 wait complete. 2024-05-27T08:00:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.753+0000 7f4e70722740 20 client.20616 trim_cache size 0 max 16384 2024-05-27T08:00:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.779+0000 7f4e70722740 1 Processor -- start 2024-05-27T08:00:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.779+0000 7f4e70722740 1 -- start start 2024-05-27T08:00:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.779+0000 7f4e70722740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643d52f0 0x5561647ad390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.779+0000 7f4e70722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.779+0000 7f4e70722740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.779+0000 7f4e70722740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641ba010 con 0x5561643fe1b0 2024-05-27T08:00:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.779+0000 7f4e70722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641b78e0 con 0x5561643dd4d0 2024-05-27T08:00:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.779+0000 7f4e70722740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641b5750 con 0x5561643d52f0 2024-05-27T08:00:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e63400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e63400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36668/0 (socket says 172.21.15.63:36668) 2024-05-27T08:00:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e63400640 1 -- 172.21.15.63:0/9185578 learned_addr learned my addr 172.21.15.63:0/9185578 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e62a00640 1 --2- 172.21.15.63:0/9185578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643d52f0 0x5561647ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e5a000640 1 --2- 172.21.15.63:0/9185578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e63400640 1 -- 172.21.15.63:0/9185578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643d52f0 msgr2=0x5561647ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e63400640 1 --2- 172.21.15.63:0/9185578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643d52f0 0x5561647ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e63400640 1 -- 172.21.15.63:0/9185578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 msgr2=0x5561647b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e63400640 1 --2- 172.21.15.63:0/9185578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e63400640 1 -- 172.21.15.63:0/9185578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561647ac120 con 0x5561643fe1b0 2024-05-27T08:00:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.780+0000 7f4e63400640 1 --2- 172.21.15.63:0/9185578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b7170 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c052590 tx=0x7f4e5c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f199857765b08500 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.781+0000 7f4e53400640 1 -- 172.21.15.63:0/9185578 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c009360 con 0x5561643fe1b0 2024-05-27T08:00:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.781+0000 7f4e53400640 1 -- 172.21.15.63:0/9185578 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e5c036050 con 0x5561643fe1b0 2024-05-27T08:00:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.781+0000 7f4e53400640 1 -- 172.21.15.63:0/9185578 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c030c90 con 0x5561643fe1b0 2024-05-27T08:00:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.782+0000 7f4e70722740 1 -- 172.21.15.63:0/9185578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 msgr2=0x5561647b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.782+0000 7f4e70722740 1 --2- 172.21.15.63:0/9185578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b7170 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c052590 tx=0x7f4e5c003f70 comp rx=0 tx=0).stop 2024-05-27T08:00:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.782+0000 7f4e70722740 1 -- 172.21.15.63:0/9185578 shutdown_connections 2024-05-27T08:00:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.782+0000 7f4e70722740 1 --2- 172.21.15.63:0/9185578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643fe1b0 0x5561647b7170 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.782+0000 7f4e70722740 1 --2- 172.21.15.63:0/9185578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.782+0000 7f4e70722740 1 --2- 172.21.15.63:0/9185578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643d52f0 0x5561647ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.782+0000 7f4e70722740 1 -- 172.21.15.63:0/9185578 >> 172.21.15.63:0/9185578 conn(0x5561643f2300 msgr2=0x5561647aaf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.782+0000 7f4e70722740 1 -- 172.21.15.63:0/9185578 shutdown_connections 2024-05-27T08:00:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.782+0000 7f4e70722740 1 -- 172.21.15.63:0/9185578 wait complete. 2024-05-27T08:00:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.783+0000 7f4e70722740 1 -- start start 2024-05-27T08:00:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.783+0000 7f4e70722740 1 Processor -- start 2024-05-27T08:00:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e70722740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e70722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e70722740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 0x5561647b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e70722740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641b5da0 con 0x5561643d52f0 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e70722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641b7210 con 0x5561643dd4d0 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e70722740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e641b6470 con 0x5561643fe1b0 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e70722740 10 client.0 _ll_register_callbacks cb 0x7f4e5c031ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e62a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e62a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36682/0 (socket says 172.21.15.63:36682) 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e62a00640 1 -- 172.21.15.63:0/2245759166 learned_addr learned my addr 172.21.15.63:0/2245759166 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e5a000640 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.784+0000 7f4e63400640 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 0x5561647b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e62a00640 1 -- 172.21.15.63:0/2245759166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 msgr2=0x5561647b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e62a00640 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 0x5561647b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e62a00640 1 -- 172.21.15.63:0/2245759166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 msgr2=0x5561647b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e62a00640 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e62a00640 1 -- 172.21.15.63:0/2245759166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561647b09f0 con 0x5561643d52f0 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e63400640 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 0x5561647b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e5a000640 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e62a00640 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647b2ef0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f4e540a9130 tx=0x7f4e540df890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa0deb37677a313 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e50c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e50c00640 1 -- 172.21.15.63:0/2245759166 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54021040 con 0x5561643d52f0 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e50c00640 1 -- 172.21.15.63:0/2245759166 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f4e540e8150 con 0x5561643d52f0 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.785+0000 7f4e70722740 1 -- 172.21.15.63:0/2245759166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e5c036050 con 0x5561643d52f0 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e50c00640 1 -- 172.21.15.63:0/2245759166 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4e540a8d80 con 0x5561643d52f0 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e70722740 10 client.20622 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e70722740 1 -- 172.21.15.63:0/2245759166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556164825000 con 0x5561643d52f0 2024-05-27T08:00:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e50c00640 1 -- 172.21.15.63:0/2245759166 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e5c036050 con 0x5561643d52f0 2024-05-27T08:00:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e50c00640 1 -- 172.21.15.63:0/2245759166 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4e540ca2f0 con 0x5561643d52f0 2024-05-27T08:00:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e70722740 10 client.20622 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e70722740 10 client.20622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e70722740 20 client.20622 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e70722740 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561643921c0 0x556164827290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e70722740 4 client.20622 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:00:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.786+0000 7f4e70722740 1 -- 172.21.15.63:0/2245759166 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561646874b0 con 0x5561643921c0 2024-05-27T08:00:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.787+0000 7f4e5a000640 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561643921c0 0x556164827290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.788+0000 7f4e5a000640 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561643921c0 0x556164827290 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1564598e08d5e4a4 server_cookie=79bf7beb2dc519b4 in_seq=0 out_seq=0 2024-05-27T08:00:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.788+0000 7f4e50c00640 10 client.20622 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.835+0000 7f4e50c00640 1 -- 172.21.15.63:0/2245759166 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5561646874b0 con 0x5561643921c0 2024-05-27T08:00:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.835+0000 7f4e50c00640 10 client.20622 handle_command_reply: tid=0 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.835+0000 7f4e70722740 1 client.20622 shutdown 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.835+0000 7f4e70722740 1 -- 172.21.15.63:0/2245759166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 msgr2=0x5561647b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.835+0000 7f4e70722740 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647b2ef0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f4e540a9130 tx=0x7f4e540df890 comp rx=0 tx=0).stop 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.836+0000 7f4e70722740 1 -- 172.21.15.63:0/2245759166 shutdown_connections 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.836+0000 7f4e70722740 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561643921c0 0x556164827290 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.836+0000 7f4e70722740 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561643fe1b0 0x5561647b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.836+0000 7f4e70722740 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561643dd4d0 0x5561647b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.836+0000 7f4e50c00640 0 client.20622 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.836+0000 7f4e70722740 1 --2- 172.21.15.63:0/2245759166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561643d52f0 0x5561647b2ef0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.836+0000 7f4e70722740 1 -- 172.21.15.63:0/2245759166 >> 172.21.15.63:0/2245759166 conn(0x5561643f2300 msgr2=0x5561647aac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.836+0000 7f4e70722740 1 -- 172.21.15.63:0/2245759166 shutdown_connections 2024-05-27T08:00:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.836+0000 7f4e70722740 1 -- 172.21.15.63:0/2245759166 wait complete. 2024-05-27T08:00:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.836+0000 7f4e70722740 20 client.20622 trim_cache size 0 max 16384 2024-05-27T08:00:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:00:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:00:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:00:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.837+0000 7f4e63e00640 1 -- 172.21.15.63:0/249559859 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e480b29d0 msgr2=0x7f4e480b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.837+0000 7f4e63e00640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e480b29d0 0x7f4e480b4ee0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c006010 tx=0x7f4e5c004a40 comp rx=0 tx=0).stop 2024-05-27T08:00:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.837+0000 7f4e63e00640 1 -- 172.21.15.63:0/249559859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e6410a390 msgr2=0x7f4e641a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.837+0000 7f4e63e00640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e6410a390 0x7f4e641a67b0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400d770 tx=0x7f4e5400dc30 comp rx=0 tx=0).stop 2024-05-27T08:00:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.837+0000 7f4e63e00640 1 -- 172.21.15.63:0/249559859 shutdown_connections 2024-05-27T08:00:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.837+0000 7f4e63e00640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e480b29d0 0x7f4e480b4ee0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.837+0000 7f4e63e00640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e6410f3c0 0x7f4e641a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.837+0000 7f4e63e00640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e6410e900 0x7f4e641a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.837+0000 7f4e63e00640 1 --2- 172.21.15.63:0/249559859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e6410a390 0x7f4e641a67b0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.837+0000 7f4e63e00640 1 -- 172.21.15.63:0/249559859 >> 172.21.15.63:0/249559859 conn(0x7f4e64076070 msgr2=0x7f4e64107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.838+0000 7f4e63e00640 1 -- 172.21.15.63:0/249559859 shutdown_connections 2024-05-27T08:00:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:39.838+0000 7f4e63e00640 1 -- 172.21.15.63:0/249559859 wait complete. 2024-05-27T08:00:41.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:41 smithi086 ceph-mon[34687]: pgmap v859: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T08:00:41.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:41 smithi149 ceph-mon[35408]: pgmap v859: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T08:00:41.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:41 smithi063 ceph-mon[31613]: pgmap v859: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T08:00:43.557 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:43 smithi063 ceph-mon[31613]: pgmap v860: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:00:43.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:43 smithi086 ceph-mon[34687]: pgmap v860: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:00:43.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:43 smithi149 ceph-mon[35408]: pgmap v860: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:00:44.343 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:00:44.837 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:00:45.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:45 smithi086 ceph-mon[34687]: pgmap v861: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:00:45.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:45 smithi149 ceph-mon[35408]: pgmap v861: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:00:45.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:45 smithi063 ceph-mon[31613]: pgmap v861: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:00:47.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.333+0000 7fa659800700 1 -- 172.21.15.63:0/2360230158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa654110bf0 msgr2=0x7fa654113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:47.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.333+0000 7fa659800700 1 --2- 172.21.15.63:0/2360230158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa654110bf0 0x7fa654113010 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00b0a0 tx=0x7fa64c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.336 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:47 smithi063 ceph-mon[31613]: pgmap v862: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:00:47.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.333+0000 7fa659800700 1 -- 172.21.15.63:0/2360230158 shutdown_connections 2024-05-27T08:00:47.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.333+0000 7fa659800700 1 --2- 172.21.15.63:0/2360230158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa654110bf0 0x7fa654113010 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.333+0000 7fa659800700 1 --2- 172.21.15.63:0/2360230158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa654075af0 0x7fa654075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.333+0000 7fa659800700 1 --2- 172.21.15.63:0/2360230158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa654076440 0x7fa6541106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.333+0000 7fa659800700 1 -- 172.21.15.63:0/2360230158 >> 172.21.15.63:0/2360230158 conn(0x7fa654100030 msgr2=0x7fa654102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:47.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.333+0000 7fa659800700 1 -- 172.21.15.63:0/2360230158 shutdown_connections 2024-05-27T08:00:47.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.333+0000 7fa659800700 1 -- 172.21.15.63:0/2360230158 wait complete. 2024-05-27T08:00:47.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.334+0000 7fa659800700 1 Processor -- start 2024-05-27T08:00:47.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.335+0000 7fa659800700 1 -- start start 2024-05-27T08:00:47.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.335+0000 7fa659800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa654075af0 0x7fa65419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.335+0000 7fa659800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa654076440 0x7fa65419d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.335+0000 7fa659800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa654110bf0 0x7fa6541a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.335+0000 7fa659800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541165b0 con 0x7fa654076440 2024-05-27T08:00:47.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.335+0000 7fa659800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa654116430 con 0x7fa654075af0 2024-05-27T08:00:47.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.335+0000 7fa659800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa654116730 con 0x7fa654110bf0 2024-05-27T08:00:47.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa652000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa654076440 0x7fa65419d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa652000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa654076440 0x7fa65419d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36690/0 (socket says 172.21.15.63:36690) 2024-05-27T08:00:47.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa652000700 1 -- 172.21.15.63:0/1584723216 learned_addr learned my addr 172.21.15.63:0/1584723216 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:47.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa652a00700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa654075af0 0x7fa65419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa653400700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa654110bf0 0x7fa6541a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa652000700 1 -- 172.21.15.63:0/1584723216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa654110bf0 msgr2=0x7fa6541a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:47.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa652000700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa654110bf0 0x7fa6541a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa652000700 1 -- 172.21.15.63:0/1584723216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa654075af0 msgr2=0x7fa65419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:47.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa652000700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa654075af0 0x7fa65419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa652000700 1 -- 172.21.15.63:0/1584723216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6541a1e10 con 0x7fa654076440 2024-05-27T08:00:47.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.336+0000 7fa653400700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa654110bf0 0x7fa6541a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:47.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.337+0000 7fa652000700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa654076440 0x7fa65419d4e0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fa64000d980 tx=0x7fa64000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:47.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.337+0000 7fa64b400700 1 -- 172.21.15.63:0/1584723216 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640017070 con 0x7fa654076440 2024-05-27T08:00:47.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.337+0000 7fa64b400700 1 -- 172.21.15.63:0/1584723216 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa640009dc0 con 0x7fa654076440 2024-05-27T08:00:47.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.337+0000 7fa64b400700 1 -- 172.21.15.63:0/1584723216 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64000f3c0 con 0x7fa654076440 2024-05-27T08:00:47.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.337+0000 7fa659800700 1 -- 172.21.15.63:0/1584723216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6541a2100 con 0x7fa654076440 2024-05-27T08:00:47.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.338+0000 7fa659800700 1 -- 172.21.15.63:0/1584723216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6541a9a20 con 0x7fa654076440 2024-05-27T08:00:47.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.342+0000 7fa64b400700 1 -- 172.21.15.63:0/1584723216 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa640009930 con 0x7fa654076440 2024-05-27T08:00:47.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.343+0000 7fa668283b80 1 Processor -- start 2024-05-27T08:00:47.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.343+0000 7fa668283b80 1 -- start start 2024-05-27T08:00:47.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa668283b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 0x5626e6a257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa668283b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a1f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa668283b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e6a1fdb0 0x5626e6a201c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa668283b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5626e694de40 con 0x5626e6a25d30 2024-05-27T08:00:47.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa668283b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5626e693d9e0 con 0x5626e6a1fdb0 2024-05-27T08:00:47.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa668283b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5626e6941a20 con 0x5626e68a0e40 2024-05-27T08:00:47.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa64b400700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6380b2a70 0x7fa6380b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa652a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a1f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa652a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a1f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36702/0 (socket says 172.21.15.63:36702) 2024-05-27T08:00:47.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa652a00700 1 -- 172.21.15.63:0/2726229916 learned_addr learned my addr 172.21.15.63:0/2726229916 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:47.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.344+0000 7fa64b400700 1 -- 172.21.15.63:0/1584723216 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7fa6400e2480 con 0x7fa654076440 2024-05-27T08:00:47.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.345+0000 7fa652000700 1 --2- 172.21.15.63:0/2726229916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e6a1fdb0 0x5626e6a201c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.345+0000 7fa653400700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6380b2a70 0x7fa6380b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.345+0000 7fa653400700 1 --2- 172.21.15.63:0/2726229916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 0x5626e6a257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.345+0000 7fa652a00700 1 -- 172.21.15.63:0/2726229916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 msgr2=0x5626e6a257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:47.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.345+0000 7fa652a00700 1 --2- 172.21.15.63:0/2726229916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 0x5626e6a257f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.345+0000 7fa652a00700 1 -- 172.21.15.63:0/2726229916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e6a1fdb0 msgr2=0x5626e6a201c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:47.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.345+0000 7fa652a00700 1 --2- 172.21.15.63:0/2726229916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e6a1fdb0 0x5626e6a201c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.345+0000 7fa652a00700 1 -- 172.21.15.63:0/2726229916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e6906420 con 0x5626e6a25d30 2024-05-27T08:00:47.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.345+0000 7fa653400700 1 --2- 172.21.15.63:0/2726229916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 0x5626e6a257f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:00:47.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.346+0000 7fa652a00700 1 --2- 172.21.15.63:0/2726229916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a1f870 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa64400bd70 tx=0x7fa6440106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9120756a0f798f0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:47.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.346+0000 7fa653400700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6380b2a70 0x7fa6380b4f80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x5626e6a20b30 tx=0x7fa64c009780 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:47.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.346+0000 7fa63fe00700 1 -- 172.21.15.63:0/2726229916 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644010e10 con 0x5626e6a25d30 2024-05-27T08:00:47.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.347+0000 7fa63fe00700 1 -- 172.21.15.63:0/2726229916 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa64401c910 con 0x5626e6a25d30 2024-05-27T08:00:47.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.347+0000 7fa63fe00700 1 -- 172.21.15.63:0/2726229916 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440117a0 con 0x5626e6a25d30 2024-05-27T08:00:47.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.348+0000 7fa668283b80 1 -- 172.21.15.63:0/2726229916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 msgr2=0x5626e6a1f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:47.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.348+0000 7fa668283b80 1 --2- 172.21.15.63:0/2726229916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a1f870 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa64400bd70 tx=0x7fa6440106b0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.348+0000 7fa668283b80 1 -- 172.21.15.63:0/2726229916 shutdown_connections 2024-05-27T08:00:47.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.348+0000 7fa668283b80 1 --2- 172.21.15.63:0/2726229916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e6a1fdb0 0x5626e6a201c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.348+0000 7fa668283b80 1 --2- 172.21.15.63:0/2726229916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 0x5626e6a257f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.348+0000 7fa668283b80 1 --2- 172.21.15.63:0/2726229916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a1f870 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.348+0000 7fa668283b80 1 -- 172.21.15.63:0/2726229916 >> 172.21.15.63:0/2726229916 conn(0x5626e6900fc0 msgr2=0x5626e6a1a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:47.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.348+0000 7fa668283b80 1 -- 172.21.15.63:0/2726229916 shutdown_connections 2024-05-27T08:00:47.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.348+0000 7fa668283b80 1 -- 172.21.15.63:0/2726229916 wait complete. 2024-05-27T08:00:47.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.349+0000 7fa668283b80 1 -- start start 2024-05-27T08:00:47.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.349+0000 7fa668283b80 1 Processor -- start 2024-05-27T08:00:47.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.349+0000 7fa668283b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 0x5626e6a29530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.349+0000 7fa668283b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e6a1fdb0 0x5626e6a29a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.350+0000 7fa668283b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a29fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.350+0000 7fa668283b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5626e6843fa0 con 0x5626e6a25d30 2024-05-27T08:00:47.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.350+0000 7fa668283b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5626e68de760 con 0x5626e6a1fdb0 2024-05-27T08:00:47.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.350+0000 7fa668283b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5626e6a2c800 con 0x5626e68a0e40 2024-05-27T08:00:47.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.350+0000 7fa668283b80 10 client.0 _ll_register_callbacks cb 0x5626e68e7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:47.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.350+0000 7fa653400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a29fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.350+0000 7fa653400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a29fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36704/0 (socket says 172.21.15.63:36704) 2024-05-27T08:00:47.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.350+0000 7fa653400700 1 -- 172.21.15.63:0/3844215654 learned_addr learned my addr 172.21.15.63:0/3844215654 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:47.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.350+0000 7fa652a00700 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 0x5626e6a29530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.350+0000 7fa652000700 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e6a1fdb0 0x5626e6a29a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.351+0000 7fa653400700 1 -- 172.21.15.63:0/3844215654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 msgr2=0x5626e6a29530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:47.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.351+0000 7fa653400700 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 0x5626e6a29530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.351+0000 7fa653400700 1 -- 172.21.15.63:0/3844215654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e6a1fdb0 msgr2=0x5626e6a29a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:47.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.351+0000 7fa653400700 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e6a1fdb0 0x5626e6a29a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.351+0000 7fa653400700 1 -- 172.21.15.63:0/3844215654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e688d8d0 con 0x5626e6a25d30 2024-05-27T08:00:47.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.351+0000 7fa653400700 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a29fb0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00b0a0 tx=0x7fa64c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3e041cd20e8b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:47.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.351+0000 7fa63cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:47.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.352+0000 7fa63cc00700 1 -- 172.21.15.63:0/3844215654 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c02e020 con 0x5626e6a25d30 2024-05-27T08:00:47.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.352+0000 7fa63cc00700 1 -- 172.21.15.63:0/3844215654 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7fa64c02a470 con 0x5626e6a25d30 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.352+0000 7fa668283b80 1 -- 172.21.15.63:0/3844215654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa64401c910 con 0x5626e6a25d30 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.352+0000 7fa63cc00700 1 -- 172.21.15.63:0/3844215654 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa64c02a670 con 0x5626e6a25d30 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.352+0000 7fa668283b80 10 client.20634 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.352+0000 7fa668283b80 1 -- 172.21.15.63:0/3844215654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5626e690e060 con 0x5626e6a25d30 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.353+0000 7fa63cc00700 1 -- 172.21.15.63:0/3844215654 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa64401c910 con 0x5626e6a25d30 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.353+0000 7fa63cc00700 1 -- 172.21.15.63:0/3844215654 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa64c011070 con 0x5626e6a25d30 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.353+0000 7fa668283b80 10 client.20634 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.353+0000 7fa668283b80 10 client.20634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.353+0000 7fa668283b80 20 client.20634 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.353+0000 7fa668283b80 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626e6a41ea0 0x5626e6a44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.353+0000 7fa668283b80 4 client.20634 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:47.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.353+0000 7fa668283b80 1 -- 172.21.15.63:0/3844215654 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626e694fd90 con 0x5626e6a41ea0 2024-05-27T08:00:47.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.354+0000 7fa652a00700 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626e6a41ea0 0x5626e6a44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.355+0000 7fa652a00700 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626e6a41ea0 0x5626e6a44350 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb2dbceb02ed901 server_cookie=db445dbb5dbd3a8 in_seq=0 out_seq=0 2024-05-27T08:00:47.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.355+0000 7fa63cc00700 10 client.20634 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:47.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa63cc00700 1 -- 172.21.15.63:0/3844215654 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5626e694fd90 con 0x5626e6a41ea0 2024-05-27T08:00:47.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa63cc00700 10 client.20634 handle_command_reply: tid=0 2024-05-27T08:00:47.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 client.20634 shutdown 2024-05-27T08:00:47.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 -- 172.21.15.63:0/3844215654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 msgr2=0x5626e6a29fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:47.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a29fb0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00b0a0 tx=0x7fa64c0099e0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 -- 172.21.15.63:0/3844215654 shutdown_connections 2024-05-27T08:00:47.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626e6a41ea0 0x5626e6a44350 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a29fb0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e68a0e40 0x5626e6a29530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 --2- 172.21.15.63:0/3844215654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e6a1fdb0 0x5626e6a29a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 -- 172.21.15.63:0/3844215654 >> 172.21.15.63:0/3844215654 conn(0x5626e6900fc0 msgr2=0x5626e6a1a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:47.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa63cc00700 0 client.20634 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:47.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 -- 172.21.15.63:0/3844215654 shutdown_connections 2024-05-27T08:00:47.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.357+0000 7fa668283b80 1 -- 172.21.15.63:0/3844215654 wait complete. 2024-05-27T08:00:47.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.358+0000 7fa668283b80 20 client.20634 trim_cache size 0 max 16384 2024-05-27T08:00:47.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.381+0000 7fa668283b80 1 Processor -- start 2024-05-27T08:00:47.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.381+0000 7fa668283b80 1 -- start start 2024-05-27T08:00:47.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.381+0000 7fa668283b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 0x5626e6a1f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.381+0000 7fa668283b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a1fdb0 0x5626e6a2b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.381+0000 7fa668283b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a25d30 0x5626e6a3e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.381+0000 7fa668283b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541b0d50 con 0x5626e6a1fdb0 2024-05-27T08:00:47.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.381+0000 7fa668283b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541ae5e0 con 0x5626e68a0e40 2024-05-27T08:00:47.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.381+0000 7fa668283b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541ac450 con 0x5626e6a25d30 2024-05-27T08:00:47.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.382+0000 7fa652000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a1fdb0 0x5626e6a2b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.382+0000 7fa652000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a1fdb0 0x5626e6a2b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36706/0 (socket says 172.21.15.63:36706) 2024-05-27T08:00:47.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.382+0000 7fa652000700 1 -- 172.21.15.63:0/3207869288 learned_addr learned my addr 172.21.15.63:0/3207869288 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.382+0000 7fa652a00700 1 --2- 172.21.15.63:0/3207869288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 0x5626e6a1f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.382+0000 7fa653400700 1 --2- 172.21.15.63:0/3207869288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a25d30 0x5626e6a3e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.382+0000 7fa652000700 1 -- 172.21.15.63:0/3207869288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a25d30 msgr2=0x5626e6a3e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.382+0000 7fa652000700 1 --2- 172.21.15.63:0/3207869288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a25d30 0x5626e6a3e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.382+0000 7fa652000700 1 -- 172.21.15.63:0/3207869288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 msgr2=0x5626e6a1f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.382+0000 7fa652000700 1 --2- 172.21.15.63:0/3207869288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 0x5626e6a1f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.382+0000 7fa652000700 1 -- 172.21.15.63:0/3207869288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e6a201c0 con 0x5626e6a1fdb0 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.383+0000 7fa652000700 1 --2- 172.21.15.63:0/3207869288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a1fdb0 0x5626e6a2b9d0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fa6400a9040 tx=0x7fa640009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f882121c64f626b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.383+0000 7fa63f400700 1 -- 172.21.15.63:0/3207869288 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640020040 con 0x5626e6a1fdb0 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.383+0000 7fa63f400700 1 -- 172.21.15.63:0/3207869288 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6400a9340 con 0x5626e6a1fdb0 2024-05-27T08:00:47.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.383+0000 7fa63f400700 1 -- 172.21.15.63:0/3207869288 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400a95a0 con 0x5626e6a1fdb0 2024-05-27T08:00:47.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.384+0000 7fa668283b80 1 -- 172.21.15.63:0/3207869288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a1fdb0 msgr2=0x5626e6a2b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:47.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.384+0000 7fa668283b80 1 --2- 172.21.15.63:0/3207869288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a1fdb0 0x5626e6a2b9d0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fa6400a9040 tx=0x7fa640009f70 comp rx=0 tx=0).stop 2024-05-27T08:00:47.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.384+0000 7fa668283b80 1 -- 172.21.15.63:0/3207869288 shutdown_connections 2024-05-27T08:00:47.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.384+0000 7fa668283b80 1 --2- 172.21.15.63:0/3207869288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a25d30 0x5626e6a3e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.384+0000 7fa668283b80 1 --2- 172.21.15.63:0/3207869288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 0x5626e6a1f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.384+0000 7fa668283b80 1 --2- 172.21.15.63:0/3207869288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a1fdb0 0x5626e6a2b9d0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.384+0000 7fa668283b80 1 -- 172.21.15.63:0/3207869288 >> 172.21.15.63:0/3207869288 conn(0x5626e6900fc0 msgr2=0x5626e6a29310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:47.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.385+0000 7fa668283b80 1 -- 172.21.15.63:0/3207869288 shutdown_connections 2024-05-27T08:00:47.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.385+0000 7fa668283b80 1 -- 172.21.15.63:0/3207869288 wait complete. 2024-05-27T08:00:47.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.385+0000 7fa668283b80 1 -- start start 2024-05-27T08:00:47.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.385+0000 7fa668283b80 1 Processor -- start 2024-05-27T08:00:47.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.386+0000 7fa668283b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 0x5626e6a3a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.386+0000 7fa668283b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a1fdb0 0x5626e6a3a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.386+0000 7fa668283b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.386+0000 7fa668283b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541acaa0 con 0x5626e6a25d30 2024-05-27T08:00:47.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.386+0000 7fa668283b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541adf10 con 0x5626e68a0e40 2024-05-27T08:00:47.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.386+0000 7fa668283b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541ad170 con 0x5626e6a1fdb0 2024-05-27T08:00:47.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.386+0000 7fa668283b80 10 client.0 _ll_register_callbacks cb 0x5626e68e7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:47.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.386+0000 7fa653400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.386+0000 7fa653400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a372f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36716/0 (socket says 172.21.15.63:36716) 2024-05-27T08:00:47.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.387+0000 7fa653400700 1 -- 172.21.15.63:0/827124598 learned_addr learned my addr 172.21.15.63:0/827124598 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:47.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.387+0000 7fa652a00700 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 0x5626e6a3a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.387+0000 7fa652000700 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a1fdb0 0x5626e6a3a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.387+0000 7fa653400700 1 -- 172.21.15.63:0/827124598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a1fdb0 msgr2=0x5626e6a3a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:47.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.387+0000 7fa653400700 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a1fdb0 0x5626e6a3a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.387+0000 7fa653400700 1 -- 172.21.15.63:0/827124598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 msgr2=0x5626e6a3a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:47.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.387+0000 7fa653400700 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 0x5626e6a3a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.387+0000 7fa653400700 1 -- 172.21.15.63:0/827124598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e6a37830 con 0x5626e6a25d30 2024-05-27T08:00:47.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.387+0000 7fa652000700 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a1fdb0 0x5626e6a3a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:47.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.387+0000 7fa652a00700 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 0x5626e6a3a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:47.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.388+0000 7fa653400700 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a372f0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00b3f0 tx=0x7fa64c005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8b68f9b5081e87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:47.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.388+0000 7fa63cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:47.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.388+0000 7fa63cc00700 1 -- 172.21.15.63:0/827124598 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c042060 con 0x5626e6a25d30 2024-05-27T08:00:47.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.388+0000 7fa63cc00700 1 -- 172.21.15.63:0/827124598 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7fa64c02a810 con 0x5626e6a25d30 2024-05-27T08:00:47.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.388+0000 7fa668283b80 1 -- 172.21.15.63:0/827124598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6400a9340 con 0x5626e6a25d30 2024-05-27T08:00:47.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.388+0000 7fa63cc00700 1 -- 172.21.15.63:0/827124598 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa64c049050 con 0x5626e6a25d30 2024-05-27T08:00:47.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.388+0000 7fa668283b80 10 client.20640 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:47.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.388+0000 7fa668283b80 1 -- 172.21.15.63:0/827124598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5626e6aad170 con 0x5626e6a25d30 2024-05-27T08:00:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.389+0000 7fa63cc00700 1 -- 172.21.15.63:0/827124598 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6400a9340 con 0x5626e6a25d30 2024-05-27T08:00:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.389+0000 7fa63cc00700 1 -- 172.21.15.63:0/827124598 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa64c011040 con 0x5626e6a25d30 2024-05-27T08:00:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.389+0000 7fa668283b80 10 client.20640 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.389+0000 7fa668283b80 10 client.20640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.389+0000 7fa668283b80 20 client.20640 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.389+0000 7fa668283b80 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626e6aad7f0 0x5626e6aafc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.389+0000 7fa668283b80 4 client.20640 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:00:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.389+0000 7fa668283b80 1 -- 172.21.15.63:0/827124598 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5626e6949570 con 0x5626e6aad7f0 2024-05-27T08:00:47.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.390+0000 7fa652a00700 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626e6aad7f0 0x5626e6aafc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:47.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.391+0000 7fa652a00700 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626e6aad7f0 0x5626e6aafc10 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a852c7820407eb6d server_cookie=b75de6190de61781 in_seq=0 out_seq=0 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.391+0000 7fa63cc00700 10 client.20640 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.391+0000 7fa63cc00700 1 -- 172.21.15.63:0/827124598 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5626e6949570 con 0x5626e6aad7f0 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.391+0000 7fa63cc00700 10 client.20640 handle_command_reply: tid=0 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.391+0000 7fa668283b80 1 client.20640 shutdown 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.391+0000 7fa668283b80 1 -- 172.21.15.63:0/827124598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 msgr2=0x5626e6a372f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.391+0000 7fa668283b80 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a372f0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00b3f0 tx=0x7fa64c005640 comp rx=0 tx=0).stop 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.392+0000 7fa668283b80 1 -- 172.21.15.63:0/827124598 shutdown_connections 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.392+0000 7fa668283b80 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5626e6aad7f0 0x5626e6aafc10 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.392+0000 7fa668283b80 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626e6a25d30 0x5626e6a372f0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.392+0000 7fa668283b80 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5626e68a0e40 0x5626e6a3a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.392+0000 7fa668283b80 1 --2- 172.21.15.63:0/827124598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626e6a1fdb0 0x5626e6a3a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.392+0000 7fa668283b80 1 -- 172.21.15.63:0/827124598 >> 172.21.15.63:0/827124598 conn(0x5626e6900fc0 msgr2=0x5626e6a29d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.392+0000 7fa63cc00700 0 client.20640 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.392+0000 7fa668283b80 1 -- 172.21.15.63:0/827124598 shutdown_connections 2024-05-27T08:00:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.392+0000 7fa668283b80 1 -- 172.21.15.63:0/827124598 wait complete. 2024-05-27T08:00:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.392+0000 7fa668283b80 20 client.20640 trim_cache size 0 max 16384 2024-05-27T08:00:47.396 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:00:47.396 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:00:47.396 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:00:47.396 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:00:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 -- 172.21.15.63:0/1584723216 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6380b2a70 msgr2=0x7fa6380b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6380b2a70 0x7fa6380b4f80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x5626e6a20b30 tx=0x7fa64c009780 comp rx=0 tx=0).stop 2024-05-27T08:00:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 -- 172.21.15.63:0/1584723216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa654076440 msgr2=0x7fa65419d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa654076440 0x7fa65419d4e0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fa64000d980 tx=0x7fa64000dc90 comp rx=0 tx=0).stop 2024-05-27T08:00:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 -- 172.21.15.63:0/1584723216 shutdown_connections 2024-05-27T08:00:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6380b2a70 0x7fa6380b4f80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa654110bf0 0x7fa6541a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa654075af0 0x7fa65419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 --2- 172.21.15.63:0/1584723216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa654076440 0x7fa65419d4e0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 -- 172.21.15.63:0/1584723216 >> 172.21.15.63:0/1584723216 conn(0x7fa654100030 msgr2=0x7fa654100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:47.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 -- 172.21.15.63:0/1584723216 shutdown_connections 2024-05-27T08:00:47.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:47.393+0000 7fa659800700 1 -- 172.21.15.63:0/1584723216 wait complete. 2024-05-27T08:00:47.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:47 smithi086 ceph-mon[34687]: pgmap v862: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:00:47.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:47 smithi149 ceph-mon[35408]: pgmap v862: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:00:48.227 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:00:48.228 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:00:48.735 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:00:49.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:49 smithi086 ceph-mon[34687]: pgmap v863: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:00:49.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:49 smithi149 ceph-mon[35408]: pgmap v863: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:00:49.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:49 smithi063 ceph-mon[31613]: pgmap v863: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:00:51.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.240+0000 7f7c74400700 1 -- 172.21.15.63:0/3491779161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c6c10e900 msgr2=0x7f7c6c110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:51.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.240+0000 7f7c74400700 1 --2- 172.21.15.63:0/3491779161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c6c10e900 0x7f7c6c110de0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400b0a0 tx=0x7f7c6400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:00:51.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.240+0000 7f7c74400700 1 -- 172.21.15.63:0/3491779161 shutdown_connections 2024-05-27T08:00:51.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.240+0000 7f7c74400700 1 --2- 172.21.15.63:0/3491779161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c6c10e900 0x7f7c6c110de0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:51.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.240+0000 7f7c74400700 1 --2- 172.21.15.63:0/3491779161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c1054d0 0x7f7c6c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:51.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.240+0000 7f7c74400700 1 --2- 172.21.15.63:0/3491779161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c105e20 0x7f7c6c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:51.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.240+0000 7f7c74400700 1 -- 172.21.15.63:0/3491779161 >> 172.21.15.63:0/3491779161 conn(0x7f7c6c0fedb0 msgr2=0x7f7c6c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:51.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.240+0000 7f7c74400700 1 -- 172.21.15.63:0/3491779161 shutdown_connections 2024-05-27T08:00:51.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.240+0000 7f7c74400700 1 -- 172.21.15.63:0/3491779161 wait complete. 2024-05-27T08:00:51.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.241+0000 7f7c74400700 1 Processor -- start 2024-05-27T08:00:51.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.242+0000 7f7c74400700 1 -- start start 2024-05-27T08:00:51.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.242+0000 7f7c74400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c1054d0 0x7f7c6c1031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:51.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.242+0000 7f7c74400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c105e20 0x7f7c6c103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:51.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.242+0000 7f7c74400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c6c10e900 0x7f7c6c103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:51.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.242+0000 7f7c74400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c114330 con 0x7f7c6c10e900 2024-05-27T08:00:51.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.242+0000 7f7c74400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c1141b0 con 0x7f7c6c105e20 2024-05-27T08:00:51.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.242+0000 7f7c74400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c1144b0 con 0x7f7c6c1054d0 2024-05-27T08:00:51.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.243+0000 7f7c72400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c6c10e900 0x7f7c6c103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:51.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.243+0000 7f7c72400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c6c10e900 0x7f7c6c103c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42804/0 (socket says 172.21.15.63:42804) 2024-05-27T08:00:51.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.243+0000 7f7c72400700 1 -- 172.21.15.63:0/1138126382 learned_addr learned my addr 172.21.15.63:0/1138126382 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:51.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.243+0000 7f7c71a00700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c1054d0 0x7f7c6c1031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:51.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.243+0000 7f7c71000700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c105e20 0x7f7c6c103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:51.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.243+0000 7f7c72400700 1 -- 172.21.15.63:0/1138126382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c1054d0 msgr2=0x7f7c6c1031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:51.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.243+0000 7f7c72400700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c1054d0 0x7f7c6c1031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:51.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.243+0000 7f7c72400700 1 -- 172.21.15.63:0/1138126382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c105e20 msgr2=0x7f7c6c103700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:51.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.243+0000 7f7c72400700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c105e20 0x7f7c6c103700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:51.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.243+0000 7f7c72400700 1 -- 172.21.15.63:0/1138126382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c1a9040 con 0x7f7c6c10e900 2024-05-27T08:00:51.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.244+0000 7f7c72400700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c6c10e900 0x7f7c6c103c40 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400b070 tx=0x7f7c64004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:51.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.244+0000 7f7c6a000700 1 -- 172.21.15.63:0/1138126382 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64021070 con 0x7f7c6c10e900 2024-05-27T08:00:51.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.244+0000 7f7c6a000700 1 -- 172.21.15.63:0/1138126382 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c64007b80 con 0x7f7c6c10e900 2024-05-27T08:00:51.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.244+0000 7f7c74400700 1 -- 172.21.15.63:0/1138126382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6c1a92d0 con 0x7f7c6c10e900 2024-05-27T08:00:51.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.244+0000 7f7c6a000700 1 -- 172.21.15.63:0/1138126382 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64004640 con 0x7f7c6c10e900 2024-05-27T08:00:51.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.245+0000 7f7c74400700 1 -- 172.21.15.63:0/1138126382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6c1a9800 con 0x7f7c6c10e900 2024-05-27T08:00:51.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.246+0000 7f7c74400700 1 -- 172.21.15.63:0/1138126382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c6c0512b0 con 0x7f7c6c10e900 2024-05-27T08:00:51.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.249+0000 7f7c6a000700 1 -- 172.21.15.63:0/1138126382 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7c64003a00 con 0x7f7c6c10e900 2024-05-27T08:00:51.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.252+0000 7f7c6a000700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c540b2a70 0x7f7c540b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:51.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.252+0000 7f7c6a000700 1 -- 172.21.15.63:0/1138126382 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f7c6400f020 con 0x7f7c6c10e900 2024-05-27T08:00:51.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.252+0000 7f7c71a00700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c540b2a70 0x7f7c540b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:51.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.253+0000 7f7c71a00700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c540b2a70 0x7f7c540b4f80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7c60005950 tx=0x7f7c6000b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:51.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.254+0000 7f7c6a000700 1 -- 172.21.15.63:0/1138126382 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c6402a020 con 0x7f7c6c10e900 2024-05-27T08:00:51.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.581+0000 7f7c74400700 1 -- 172.21.15.63:0/1138126382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c6c1046f0 con 0x7f7c6c10e900 2024-05-27T08:00:51.584 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:51 smithi063 ceph-mon[31613]: pgmap v864: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:00:51.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.584+0000 7f7c6a000700 1 -- 172.21.15.63:0/1138126382 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7f7c640ab870 con 0x7f7c6c10e900 2024-05-27T08:00:51.586 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:00:51.586 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:00:51.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.588+0000 7f7c74400700 1 -- 172.21.15.63:0/1138126382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c540b2a70 msgr2=0x7f7c540b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:51.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.588+0000 7f7c74400700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c540b2a70 0x7f7c540b4f80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7c60005950 tx=0x7f7c6000b410 comp rx=0 tx=0).stop 2024-05-27T08:00:51.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.588+0000 7f7c74400700 1 -- 172.21.15.63:0/1138126382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c6c10e900 msgr2=0x7f7c6c103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:51.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.588+0000 7f7c74400700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c6c10e900 0x7f7c6c103c40 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400b070 tx=0x7f7c64004d50 comp rx=0 tx=0).stop 2024-05-27T08:00:51.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.588+0000 7f7c74400700 1 -- 172.21.15.63:0/1138126382 shutdown_connections 2024-05-27T08:00:51.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.588+0000 7f7c74400700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7c540b2a70 0x7f7c540b4f80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:51.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.588+0000 7f7c74400700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c6c10e900 0x7f7c6c103c40 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:51.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.588+0000 7f7c74400700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7c6c1054d0 0x7f7c6c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:51.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.588+0000 7f7c74400700 1 --2- 172.21.15.63:0/1138126382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c105e20 0x7f7c6c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:51.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.588+0000 7f7c74400700 1 -- 172.21.15.63:0/1138126382 >> 172.21.15.63:0/1138126382 conn(0x7f7c6c0fedb0 msgr2=0x7f7c6c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:51.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.589+0000 7f7c74400700 1 -- 172.21.15.63:0/1138126382 shutdown_connections 2024-05-27T08:00:51.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:51.589+0000 7f7c74400700 1 -- 172.21.15.63:0/1138126382 wait complete. 2024-05-27T08:00:51.593 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:00:51.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:51 smithi086 ceph-mon[34687]: pgmap v864: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:00:51.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:51 smithi149 ceph-mon[35408]: pgmap v864: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:00:52.548 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:00:52.630 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:52 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1138126382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:00:52.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:52 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1138126382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:00:52.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:52 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1138126382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:00:53.045 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:00:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.107+0000 7f76d6e00640 1 -- 172.21.15.63:0/4125883962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76d0102a20 msgr2=0x7f76d0102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.107+0000 7f76d6e00640 1 --2- 172.21.15.63:0/4125883962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76d0102a20 0x7f76d0102e30 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009900 tx=0x7f76c002a410 comp rx=0 tx=0).stop 2024-05-27T08:00:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.107+0000 7f76d6e00640 1 -- 172.21.15.63:0/4125883962 shutdown_connections 2024-05-27T08:00:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.108+0000 7f76d6e00640 1 --2- 172.21.15.63:0/4125883962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d01105c0 0x7f76d01129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.108+0000 7f76d6e00640 1 --2- 172.21.15.63:0/4125883962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76d010dc60 0x7f76d0110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.108+0000 7f76d6e00640 1 --2- 172.21.15.63:0/4125883962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76d0102a20 0x7f76d0102e30 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.108+0000 7f76d6e00640 1 -- 172.21.15.63:0/4125883962 >> 172.21.15.63:0/4125883962 conn(0x7f76d00782d0 msgr2=0x7f76d0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.108+0000 7f76d6e00640 1 -- 172.21.15.63:0/4125883962 shutdown_connections 2024-05-27T08:00:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.108+0000 7f76d6e00640 1 -- 172.21.15.63:0/4125883962 wait complete. 2024-05-27T08:00:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.109+0000 7f76d6e00640 1 Processor -- start 2024-05-27T08:00:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.109+0000 7f76d6e00640 1 -- start start 2024-05-27T08:00:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76d6e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76d0102a20 0x7f76d01a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76d6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d010dc60 0x7f76d01a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76d6e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76d01105c0 0x7f76d019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76d6e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0115f50 con 0x7f76d0102a20 2024-05-27T08:00:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76d6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0115dd0 con 0x7f76d010dc60 2024-05-27T08:00:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76d6e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01160d0 con 0x7f76d01105c0 2024-05-27T08:00:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76d0102a20 0x7f76d01a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76d0102a20 0x7f76d01a2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42828/0 (socket says 172.21.15.63:42828) 2024-05-27T08:00:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76cfe00640 1 -- 172.21.15.63:0/765889471 learned_addr learned my addr 172.21.15.63:0/765889471 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76d4e00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76d01105c0 0x7f76d019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.110+0000 7f76cf400640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d010dc60 0x7f76d01a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.111+0000 7f76cfe00640 1 -- 172.21.15.63:0/765889471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76d01105c0 msgr2=0x7f76d019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.111+0000 7f76cfe00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76d01105c0 0x7f76d019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.111+0000 7f76cfe00640 1 -- 172.21.15.63:0/765889471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d010dc60 msgr2=0x7f76d01a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.111+0000 7f76cfe00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d010dc60 0x7f76d01a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.111+0000 7f76cfe00640 1 -- 172.21.15.63:0/765889471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d019cf30 con 0x7f76d0102a20 2024-05-27T08:00:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.111+0000 7f76cfe00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76d0102a20 0x7f76d01a2680 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f76c002a930 tx=0x7f76c0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.111+0000 7f76ccc00640 1 -- 172.21.15.63:0/765889471 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c003f070 con 0x7f76d0102a20 2024-05-27T08:00:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.112+0000 7f76ccc00640 1 -- 172.21.15.63:0/765889471 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76c0003e40 con 0x7f76d0102a20 2024-05-27T08:00:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.112+0000 7f76d6e00640 1 -- 172.21.15.63:0/765889471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d019d1c0 con 0x7f76d0102a20 2024-05-27T08:00:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.112+0000 7f76ccc00640 1 -- 172.21.15.63:0/765889471 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0035030 con 0x7f76d0102a20 2024-05-27T08:00:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.112+0000 7f76d6e00640 1 -- 172.21.15.63:0/765889471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d019d6b0 con 0x7f76d0102a20 2024-05-27T08:00:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.117+0000 7f76ddda6740 1 Processor -- start 2024-05-27T08:00:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.117+0000 7f76ccc00640 1 -- 172.21.15.63:0/765889471 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f76c0042070 con 0x7f76d0102a20 2024-05-27T08:00:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.117+0000 7f76ddda6740 1 -- start start 2024-05-27T08:00:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.118+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 0x562a667f7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.118+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a6642c2f0 0x562a667f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.118+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a667f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.118+0000 7f76ddda6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562a66768b90 con 0x562a664344d0 2024-05-27T08:00:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.118+0000 7f76ddda6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562a66758730 con 0x562a664551b0 2024-05-27T08:00:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.118+0000 7f76ddda6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562a6675c770 con 0x562a6642c2f0 2024-05-27T08:00:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.118+0000 7f76cf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a667f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.118+0000 7f76cf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a667f81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42832/0 (socket says 172.21.15.63:42832) 2024-05-27T08:00:53.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.118+0000 7f76cf400640 1 -- 172.21.15.63:0/2515611046 learned_addr learned my addr 172.21.15.63:0/2515611046 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.119+0000 7f76d4e00640 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 0x562a667f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.119+0000 7f76cfe00640 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a6642c2f0 0x562a667f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.119+0000 7f76cf400640 1 -- 172.21.15.63:0/2515611046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a6642c2f0 msgr2=0x562a667f7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.119+0000 7f76cf400640 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a6642c2f0 0x562a667f7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.119+0000 7f76cf400640 1 -- 172.21.15.63:0/2515611046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 msgr2=0x562a667f7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.119+0000 7f76cf400640 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 0x562a667f7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.119+0000 7f76cf400640 1 -- 172.21.15.63:0/2515611046 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a666f18b0 con 0x562a664344d0 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.119+0000 7f76d4e00640 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 0x562a667f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.119+0000 7f76cfe00640 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a6642c2f0 0x562a667f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.119+0000 7f76cf400640 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a667f81d0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00bbe0 tx=0x7f76bc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac913f46358aef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.120+0000 7f76ccc00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76b40b2950 0x7f76b40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.120+0000 7f76c5600640 1 -- 172.21.15.63:0/2515611046 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc015070 con 0x562a664344d0 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.120+0000 7f76c5600640 1 -- 172.21.15.63:0/2515611046 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76bc005020 con 0x562a664344d0 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.120+0000 7f76c5600640 1 -- 172.21.15.63:0/2515611046 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc004a00 con 0x562a664344d0 2024-05-27T08:00:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.120+0000 7f76ccc00640 1 -- 172.21.15.63:0/765889471 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f76c0103390 con 0x7f76d0102a20 2024-05-27T08:00:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.120+0000 7f76d4e00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76b40b2950 0x7f76b40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.121+0000 7f76ddda6740 1 -- 172.21.15.63:0/2515611046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 msgr2=0x562a667f81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.121+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a667f81d0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00bbe0 tx=0x7f76bc00f730 comp rx=0 tx=0).stop 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.121+0000 7f76d4e00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76b40b2950 0x7f76b40b4e60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f76c80092f0 tx=0x7f76c8009280 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.121+0000 7f76ddda6740 1 -- 172.21.15.63:0/2515611046 shutdown_connections 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.121+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a667f81d0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.121+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a6642c2f0 0x562a667f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.121+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2515611046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 0x562a667f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.121+0000 7f76ddda6740 1 -- 172.21.15.63:0/2515611046 >> 172.21.15.63:0/2515611046 conn(0x562a66449300 msgr2=0x562a667f4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.121+0000 7f76ddda6740 1 -- 172.21.15.63:0/2515611046 shutdown_connections 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.121+0000 7f76ddda6740 1 -- 172.21.15.63:0/2515611046 wait complete. 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.122+0000 7f76ddda6740 1 -- start start 2024-05-27T08:00:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.122+0000 7f76ddda6740 1 Processor -- start 2024-05-27T08:00:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.123+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a6642c2f0 0x562a6680a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.123+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a6680a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.123+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664551b0 0x562a66816530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.123+0000 7f76ddda6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562a6643e790 con 0x562a664344d0 2024-05-27T08:00:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.123+0000 7f76ddda6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562a66700bc0 con 0x562a6642c2f0 2024-05-27T08:00:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.123+0000 7f76ddda6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562a6680d1a0 con 0x562a664551b0 2024-05-27T08:00:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.123+0000 7f76ddda6740 10 client.0 _ll_register_callbacks cb 0x562a666de400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.123+0000 7f76cf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a6680a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.123+0000 7f76cf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a6680a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42842/0 (socket says 172.21.15.63:42842) 2024-05-27T08:00:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.123+0000 7f76cf400640 1 -- 172.21.15.63:0/2950988992 learned_addr learned my addr 172.21.15.63:0/2950988992 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.124+0000 7f76d4e00640 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664551b0 0x562a66816530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.124+0000 7f76cfe00640 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a6642c2f0 0x562a6680a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.124+0000 7f76cf400640 1 -- 172.21.15.63:0/2950988992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664551b0 msgr2=0x562a66816530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.124+0000 7f76cf400640 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664551b0 0x562a66816530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.124+0000 7f76cf400640 1 -- 172.21.15.63:0/2950988992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a6642c2f0 msgr2=0x562a6680a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.124+0000 7f76cf400640 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a6642c2f0 0x562a6680a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.124+0000 7f76cf400640 1 -- 172.21.15.63:0/2950988992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a6646d210 con 0x562a664344d0 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.124+0000 7f76cf400640 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a6680a6f0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00bf70 tx=0x7f76bc00bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4d60c806e72027 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.124+0000 7f76ba000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.125+0000 7f76ba000640 1 -- 172.21.15.63:0/2950988992 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc015040 con 0x562a664344d0 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.125+0000 7f76ddda6740 1 -- 172.21.15.63:0/2950988992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76bc005020 con 0x562a664344d0 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.125+0000 7f76ba000640 1 -- 172.21.15.63:0/2950988992 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f76bc0054e0 con 0x562a664344d0 2024-05-27T08:00:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.125+0000 7f76ba000640 1 -- 172.21.15.63:0/2950988992 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f76bc0046c0 con 0x562a664344d0 2024-05-27T08:00:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.125+0000 7f76ddda6740 10 client.20658 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.125+0000 7f76ddda6740 1 -- 172.21.15.63:0/2950988992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562a66539620 con 0x562a664344d0 2024-05-27T08:00:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.125+0000 7f76ba000640 1 -- 172.21.15.63:0/2950988992 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76bc005020 con 0x562a664344d0 2024-05-27T08:00:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.126+0000 7f76ba000640 1 -- 172.21.15.63:0/2950988992 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f76bc011900 con 0x562a664344d0 2024-05-27T08:00:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.126+0000 7f76ddda6740 10 client.20658 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.126+0000 7f76ddda6740 10 client.20658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.126+0000 7f76ddda6740 20 client.20658 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.126+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a66413d80 0x562a6681c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.126+0000 7f76ddda6740 4 client.20658 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.126+0000 7f76ddda6740 1 -- 172.21.15.63:0/2950988992 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a6676aae0 con 0x562a66413d80 2024-05-27T08:00:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.127+0000 7f76cfe00640 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a66413d80 0x562a6681c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.127+0000 7f76cfe00640 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a66413d80 0x562a6681c370 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac899da64b764227 server_cookie=ee7c17006fa7ae68 in_seq=0 out_seq=0 2024-05-27T08:00:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.127+0000 7f76ba000640 10 client.20658 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.129+0000 7f76ba000640 1 -- 172.21.15.63:0/2950988992 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562a6676aae0 con 0x562a66413d80 2024-05-27T08:00:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.129+0000 7f76ba000640 10 client.20658 handle_command_reply: tid=0 2024-05-27T08:00:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.129+0000 7f76ddda6740 1 client.20658 shutdown 2024-05-27T08:00:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 1 -- 172.21.15.63:0/2950988992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 msgr2=0x562a6680a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a6680a6f0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00bf70 tx=0x7f76bc00bfa0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 1 -- 172.21.15.63:0/2950988992 shutdown_connections 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a66413d80 0x562a6681c370 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664551b0 0x562a66816530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a664344d0 0x562a6680a6f0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2950988992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a6642c2f0 0x562a6680a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 1 -- 172.21.15.63:0/2950988992 >> 172.21.15.63:0/2950988992 conn(0x562a66449300 msgr2=0x562a667f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ba000640 0 client.20658 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 1 -- 172.21.15.63:0/2950988992 shutdown_connections 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 1 -- 172.21.15.63:0/2950988992 wait complete. 2024-05-27T08:00:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.130+0000 7f76ddda6740 20 client.20658 trim_cache size 0 max 16384 2024-05-27T08:00:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.151+0000 7f76ddda6740 1 Processor -- start 2024-05-27T08:00:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.151+0000 7f76ddda6740 1 -- start start 2024-05-27T08:00:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.151+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.151+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664344d0 0x562a667f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.151+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664551b0 0x562a66803ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.151+0000 7f76ddda6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01b18e0 con 0x562a6642c2f0 2024-05-27T08:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.151+0000 7f76ddda6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01af1b0 con 0x562a664344d0 2024-05-27T08:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.151+0000 7f76ddda6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01ad060 con 0x562a664551b0 2024-05-27T08:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.152+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.152+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42846/0 (socket says 172.21.15.63:42846) 2024-05-27T08:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.152+0000 7f76cfe00640 1 -- 172.21.15.63:0/3351200636 learned_addr learned my addr 172.21.15.63:0/3351200636 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.152+0000 7f76cf400640 1 --2- 172.21.15.63:0/3351200636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664344d0 0x562a667f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.152+0000 7f76d4e00640 1 --2- 172.21.15.63:0/3351200636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664551b0 0x562a66803ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.152+0000 7f76cfe00640 1 -- 172.21.15.63:0/3351200636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664551b0 msgr2=0x562a66803ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.152+0000 7f76cfe00640 1 --2- 172.21.15.63:0/3351200636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664551b0 0x562a66803ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.153+0000 7f76cfe00640 1 -- 172.21.15.63:0/3351200636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664344d0 msgr2=0x562a667f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.153+0000 7f76cfe00640 1 --2- 172.21.15.63:0/3351200636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664344d0 0x562a667f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.153+0000 7f76cfe00640 1 -- 172.21.15.63:0/3351200636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a6680d1f0 con 0x562a6642c2f0 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.153+0000 7f76cfe00640 1 --2- 172.21.15.63:0/3351200636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680c300 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f76c00ca1e0 tx=0x7f76c0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8948a155dd66ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.153+0000 7f76c4c00640 1 -- 172.21.15.63:0/3351200636 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0049040 con 0x562a6642c2f0 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.153+0000 7f76c4c00640 1 -- 172.21.15.63:0/3351200636 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76c00ca250 con 0x562a6642c2f0 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.153+0000 7f76c4c00640 1 -- 172.21.15.63:0/3351200636 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0100940 con 0x562a6642c2f0 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.154+0000 7f76ddda6740 1 -- 172.21.15.63:0/3351200636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 msgr2=0x562a6680c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.154+0000 7f76ddda6740 1 --2- 172.21.15.63:0/3351200636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680c300 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f76c00ca1e0 tx=0x7f76c0005de0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.154+0000 7f76ddda6740 1 -- 172.21.15.63:0/3351200636 shutdown_connections 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.154+0000 7f76ddda6740 1 --2- 172.21.15.63:0/3351200636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664551b0 0x562a66803ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.154+0000 7f76ddda6740 1 --2- 172.21.15.63:0/3351200636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664344d0 0x562a667f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.154+0000 7f76ddda6740 1 --2- 172.21.15.63:0/3351200636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680c300 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.154+0000 7f76ddda6740 1 -- 172.21.15.63:0/3351200636 >> 172.21.15.63:0/3351200636 conn(0x562a66449300 msgr2=0x562a667f75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.154+0000 7f76ddda6740 1 -- 172.21.15.63:0/3351200636 shutdown_connections 2024-05-27T08:00:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.155+0000 7f76ddda6740 1 -- 172.21.15.63:0/3351200636 wait complete. 2024-05-27T08:00:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.155+0000 7f76ddda6740 1 -- start start 2024-05-27T08:00:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.155+0000 7f76ddda6740 1 Processor -- start 2024-05-27T08:00:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.156+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.156+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664344d0 0x562a6680d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.156+0000 7f76ddda6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 0x562a6680ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.156+0000 7f76ddda6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01ad6b0 con 0x562a6642c2f0 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.156+0000 7f76ddda6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01aeae0 con 0x562a664551b0 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.156+0000 7f76ddda6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01add40 con 0x562a664344d0 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.156+0000 7f76ddda6740 10 client.0 _ll_register_callbacks cb 0x7f76bc00ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.156+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.156+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42852/0 (socket says 172.21.15.63:42852) 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.156+0000 7f76cfe00640 1 -- 172.21.15.63:0/2657975736 learned_addr learned my addr 172.21.15.63:0/2657975736 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76cf400640 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664344d0 0x562a6680d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76d4e00640 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 0x562a6680ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76cfe00640 1 -- 172.21.15.63:0/2657975736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664344d0 msgr2=0x562a6680d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76cfe00640 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664344d0 0x562a6680d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76cfe00640 1 -- 172.21.15.63:0/2657975736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 msgr2=0x562a6680ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76cfe00640 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 0x562a6680ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76cfe00640 1 -- 172.21.15.63:0/2657975736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a66882a40 con 0x562a6642c2f0 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76cfe00640 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680d450 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f76c00c62d0 tx=0x7f76c01032c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169b9d612a84bb87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76ba000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76ba000640 1 -- 172.21.15.63:0/2657975736 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0049040 con 0x562a6642c2f0 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76ba000640 1 -- 172.21.15.63:0/2657975736 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f76c0031400 con 0x562a6642c2f0 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.157+0000 7f76ddda6740 1 -- 172.21.15.63:0/2657975736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c00ca250 con 0x562a6642c2f0 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ba000640 1 -- 172.21.15.63:0/2657975736 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f76c00e1080 con 0x562a6642c2f0 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ddda6740 10 client.20664 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ddda6740 1 -- 172.21.15.63:0/2657975736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562a66883cb0 con 0x562a6642c2f0 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ba000640 1 -- 172.21.15.63:0/2657975736 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76c00ca250 con 0x562a6642c2f0 2024-05-27T08:00:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ba000640 1 -- 172.21.15.63:0/2657975736 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f76c00ca520 con 0x562a6642c2f0 2024-05-27T08:00:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ddda6740 10 client.20664 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ddda6740 10 client.20664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ddda6740 20 client.20664 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a663e91c0 0x562a66886280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ddda6740 4 client.20664 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:00:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.158+0000 7f76ddda6740 1 -- 172.21.15.63:0/2657975736 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a667642c0 con 0x562a663e91c0 2024-05-27T08:00:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.159+0000 7f76cf400640 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a663e91c0 0x562a66886280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.160+0000 7f76cf400640 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a663e91c0 0x562a66886280 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ee45a058813db8c server_cookie=b024d4ca438be3e8 in_seq=0 out_seq=0 2024-05-27T08:00:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.160+0000 7f76ba000640 10 client.20664 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.199+0000 7f76ba000640 1 -- 172.21.15.63:0/2657975736 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562a667642c0 con 0x562a663e91c0 2024-05-27T08:00:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.199+0000 7f76ba000640 10 client.20664 handle_command_reply: tid=0 2024-05-27T08:00:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.199+0000 7f76ddda6740 1 client.20664 shutdown 2024-05-27T08:00:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.199+0000 7f76ddda6740 1 -- 172.21.15.63:0/2657975736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 msgr2=0x562a6680d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.199+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680d450 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f76c00c62d0 tx=0x7f76c01032c0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.199+0000 7f76ddda6740 1 -- 172.21.15.63:0/2657975736 shutdown_connections 2024-05-27T08:00:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.199+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a663e91c0 0x562a66886280 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.200+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a664551b0 0x562a6680ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.200+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a664344d0 0x562a6680d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.200+0000 7f76ddda6740 1 --2- 172.21.15.63:0/2657975736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a6642c2f0 0x562a6680d450 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.200+0000 7f76ba000640 0 client.20664 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:00:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.200+0000 7f76ddda6740 1 -- 172.21.15.63:0/2657975736 >> 172.21.15.63:0/2657975736 conn(0x562a66449300 msgr2=0x562a6680bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.200+0000 7f76ddda6740 1 -- 172.21.15.63:0/2657975736 shutdown_connections 2024-05-27T08:00:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.200+0000 7f76ddda6740 1 -- 172.21.15.63:0/2657975736 wait complete. 2024-05-27T08:00:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.200+0000 7f76ddda6740 20 client.20664 trim_cache size 0 max 16384 2024-05-27T08:00:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:00:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:00:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:00:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 -- 172.21.15.63:0/765889471 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76b40b2950 msgr2=0x7f76b40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76b40b2950 0x7f76b40b4e60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f76c80092f0 tx=0x7f76c8009280 comp rx=0 tx=0).stop 2024-05-27T08:00:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 -- 172.21.15.63:0/765889471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76d0102a20 msgr2=0x7f76d01a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76d0102a20 0x7f76d01a2680 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f76c002a930 tx=0x7f76c0003c20 comp rx=0 tx=0).stop 2024-05-27T08:00:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 -- 172.21.15.63:0/765889471 shutdown_connections 2024-05-27T08:00:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f76b40b2950 0x7f76b40b4e60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76d01105c0 0x7f76d019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d010dc60 0x7f76d01a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 --2- 172.21.15.63:0/765889471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76d0102a20 0x7f76d01a2680 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 -- 172.21.15.63:0/765889471 >> 172.21.15.63:0/765889471 conn(0x7f76d00782d0 msgr2=0x7f76d01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 -- 172.21.15.63:0/765889471 shutdown_connections 2024-05-27T08:00:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:00:53.201+0000 7f76d6e00640 1 -- 172.21.15.63:0/765889471 wait complete. 2024-05-27T08:00:53.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:53 smithi063 ceph-mon[31613]: pgmap v865: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:00:53.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:53 smithi086 ceph-mon[34687]: pgmap v865: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:00:53.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:53 smithi149 ceph-mon[35408]: pgmap v865: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.529+0000 7f655fe00700 1 -- 172.21.15.63:0/143795709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6560109670 msgr2=0x7f6560110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.529+0000 7f655fe00700 1 --2- 172.21.15.63:0/143795709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6560109670 0x7f6560110fd0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f655800b0d0 tx=0x7f655800b3e0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.529+0000 7f655fe00700 1 -- 172.21.15.63:0/143795709 shutdown_connections 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.529+0000 7f655fe00700 1 --2- 172.21.15.63:0/143795709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6560109670 0x7f6560110fd0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.529+0000 7f655fe00700 1 --2- 172.21.15.63:0/143795709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65601043f0 0x7f6560104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.529+0000 7f655fe00700 1 --2- 172.21.15.63:0/143795709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6560108d20 0x7f6560109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.529+0000 7f655fe00700 1 -- 172.21.15.63:0/143795709 >> 172.21.15.63:0/143795709 conn(0x7f6560079d00 msgr2=0x7f656007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.529+0000 7f655fe00700 1 -- 172.21.15.63:0/143795709 shutdown_connections 2024-05-27T08:00:55.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.529+0000 7f655fe00700 1 -- 172.21.15.63:0/143795709 wait complete. 2024-05-27T08:00:55.532 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:55 smithi063 ceph-mon[31613]: pgmap v866: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.531+0000 7f655fe00700 1 Processor -- start 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.531+0000 7f655fe00700 1 -- start start 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.531+0000 7f655fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65601043f0 0x7f6560075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.531+0000 7f655fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6560108d20 0x7f6560075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.531+0000 7f655fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6560109670 0x7f6560076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.531+0000 7f655fe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65601141a0 con 0x7f65601043f0 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.531+0000 7f655fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6560114020 con 0x7f6560108d20 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.531+0000 7f655fe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6560114320 con 0x7f6560109670 2024-05-27T08:00:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.532+0000 7f655ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65601043f0 0x7f6560075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.532+0000 7f655ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65601043f0 0x7f6560075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42870/0 (socket says 172.21.15.63:42870) 2024-05-27T08:00:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.532+0000 7f655ea00700 1 -- 172.21.15.63:0/2735041440 learned_addr learned my addr 172.21.15.63:0/2735041440 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:55.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.532+0000 7f655e000700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6560108d20 0x7f6560075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.532+0000 7f655f400700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6560109670 0x7f6560076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.533+0000 7f655ea00700 1 -- 172.21.15.63:0/2735041440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6560109670 msgr2=0x7f6560076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.533+0000 7f655ea00700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6560109670 0x7f6560076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.533+0000 7f655ea00700 1 -- 172.21.15.63:0/2735041440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6560108d20 msgr2=0x7f6560075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.533+0000 7f655ea00700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6560108d20 0x7f6560075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.533+0000 7f655ea00700 1 -- 172.21.15.63:0/2735041440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65601a8f90 con 0x7f65601043f0 2024-05-27T08:00:55.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.533+0000 7f655ea00700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65601043f0 0x7f6560075ab0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f655000d950 tx=0x7f655000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.533+0000 7f6557400700 1 -- 172.21.15.63:0/2735041440 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550016070 con 0x7f65601043f0 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.533+0000 7f6557400700 1 -- 172.21.15.63:0/2735041440 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f655000fd20 con 0x7f65601043f0 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.533+0000 7f655fe00700 1 -- 172.21.15.63:0/2735041440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65601a9280 con 0x7f65601043f0 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.534+0000 7f6557400700 1 -- 172.21.15.63:0/2735041440 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655000e3b0 con 0x7f65601043f0 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.534+0000 7f655fe00700 1 -- 172.21.15.63:0/2735041440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65601a97c0 con 0x7f65601043f0 2024-05-27T08:00:55.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.538+0000 7f6557400700 1 -- 172.21.15.63:0/2735041440 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6550019070 con 0x7f65601043f0 2024-05-27T08:00:55.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.540+0000 7f6574358b80 1 Processor -- start 2024-05-27T08:00:55.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.540+0000 7f6574358b80 1 -- start start 2024-05-27T08:00:55.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.540+0000 7f6557400700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65440b29f0 0x7f65440b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f6557400700 1 -- 172.21.15.63:0/2735041440 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f65500e24a0 con 0x7f65601043f0 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f6574358b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34379f0 0x5654b34cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f6574358b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b334be40 0x5654b34cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f6574358b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b34d52c0 0x5654b34d76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f6574358b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5654b33f8e40 con 0x5654b34379f0 2024-05-27T08:00:55.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f6574358b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654b33e89e0 con 0x5654b34d52c0 2024-05-27T08:00:55.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f6574358b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5654b33eca20 con 0x5654b334be40 2024-05-27T08:00:55.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f655ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34379f0 0x5654b34cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f655ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34379f0 0x5654b34cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42876/0 (socket says 172.21.15.63:42876) 2024-05-27T08:00:55.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f655ea00700 1 -- 172.21.15.63:0/1450861633 learned_addr learned my addr 172.21.15.63:0/1450861633 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:55.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.541+0000 7f655f400700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65440b29f0 0x7f65440b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.542+0000 7f655e000700 1 --2- 172.21.15.63:0/1450861633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b334be40 0x5654b34cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.542+0000 7f655ea00700 1 -- 172.21.15.63:0/1450861633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b334be40 msgr2=0x5654b34cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.542+0000 7f655ea00700 1 --2- 172.21.15.63:0/1450861633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b334be40 0x5654b34cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.542+0000 7f655ea00700 1 -- 172.21.15.63:0/1450861633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b34d52c0 msgr2=0x5654b34d76e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:00:55.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.542+0000 7f655ea00700 1 --2- 172.21.15.63:0/1450861633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b34d52c0 0x5654b34d76e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.542+0000 7f655ea00700 1 -- 172.21.15.63:0/1450861633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b33b1420 con 0x5654b34379f0 2024-05-27T08:00:55.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.542+0000 7f655ea00700 1 --2- 172.21.15.63:0/1450861633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34379f0 0x5654b34cb750 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f65500a9350 tx=0x7f65500a8f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c846546fe1ee06 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.542+0000 7f654be00700 1 -- 172.21.15.63:0/1450861633 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550020070 con 0x5654b34379f0 2024-05-27T08:00:55.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.543+0000 7f654be00700 1 -- 172.21.15.63:0/1450861633 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65500ad770 con 0x5654b34379f0 2024-05-27T08:00:55.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.543+0000 7f654be00700 1 -- 172.21.15.63:0/1450861633 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65500a9880 con 0x5654b34379f0 2024-05-27T08:00:55.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.543+0000 7f655f400700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65440b29f0 0x7f65440b4f00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x5654b34cd210 tx=0x7f655800b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.544+0000 7f6574358b80 1 -- 172.21.15.63:0/1450861633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34379f0 msgr2=0x5654b34cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.544+0000 7f6574358b80 1 --2- 172.21.15.63:0/1450861633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34379f0 0x5654b34cb750 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f65500a9350 tx=0x7f65500a8f50 comp rx=0 tx=0).stop 2024-05-27T08:00:55.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.544+0000 7f6574358b80 1 -- 172.21.15.63:0/1450861633 shutdown_connections 2024-05-27T08:00:55.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.544+0000 7f6574358b80 1 --2- 172.21.15.63:0/1450861633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b34d52c0 0x5654b34d76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.544+0000 7f6574358b80 1 --2- 172.21.15.63:0/1450861633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34379f0 0x5654b34cb750 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.544+0000 7f6574358b80 1 --2- 172.21.15.63:0/1450861633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b334be40 0x5654b34cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.544+0000 7f6574358b80 1 -- 172.21.15.63:0/1450861633 >> 172.21.15.63:0/1450861633 conn(0x5654b33e5390 msgr2=0x5654b34c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.544+0000 7f6574358b80 1 -- 172.21.15.63:0/1450861633 shutdown_connections 2024-05-27T08:00:55.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.544+0000 7f6574358b80 1 -- 172.21.15.63:0/1450861633 wait complete. 2024-05-27T08:00:55.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.545+0000 7f6574358b80 1 -- start start 2024-05-27T08:00:55.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.545+0000 7f6574358b80 1 Processor -- start 2024-05-27T08:00:55.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.546+0000 7f6574358b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b334be40 0x5654b34d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.546+0000 7f6574358b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b34379f0 0x5654b34d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.546+0000 7f6574358b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.546+0000 7f6574358b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5654b32eefa0 con 0x5654b34d52c0 2024-05-27T08:00:55.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.546+0000 7f6574358b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654b3389760 con 0x5654b34379f0 2024-05-27T08:00:55.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.546+0000 7f6574358b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5654b34d7d40 con 0x5654b334be40 2024-05-27T08:00:55.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.546+0000 7f6574358b80 10 client.0 _ll_register_callbacks cb 0x5654b3392c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:55.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.546+0000 7f655f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.547+0000 7f655f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42882/0 (socket says 172.21.15.63:42882) 2024-05-27T08:00:55.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.547+0000 7f655f400700 1 -- 172.21.15.63:0/2474258527 learned_addr learned my addr 172.21.15.63:0/2474258527 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:55.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.547+0000 7f655e000700 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b34379f0 0x5654b34d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.547+0000 7f655f400700 1 -- 172.21.15.63:0/2474258527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b334be40 msgr2=0x5654b34d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.547+0000 7f655f400700 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b334be40 0x5654b34d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.547+0000 7f655f400700 1 -- 172.21.15.63:0/2474258527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b34379f0 msgr2=0x5654b34d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.547+0000 7f655f400700 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b34379f0 0x5654b34d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.547+0000 7f655f400700 1 -- 172.21.15.63:0/2474258527 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b33388d0 con 0x5654b34d52c0 2024-05-27T08:00:55.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.547+0000 7f655e000700 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b34379f0 0x5654b34d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:55.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.548+0000 7f655f400700 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d2d20 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f6558007910 tx=0x7f655800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f21269cd68a980 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.548+0000 7f6548c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:55.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.548+0000 7f6548c00700 1 -- 172.21.15.63:0/2474258527 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558022070 con 0x5654b34d52c0 2024-05-27T08:00:55.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.548+0000 7f6574358b80 1 -- 172.21.15.63:0/2474258527 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65500ad770 con 0x5654b34d52c0 2024-05-27T08:00:55.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.548+0000 7f6548c00700 1 -- 172.21.15.63:0/2474258527 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f6558028830 con 0x5654b34d52c0 2024-05-27T08:00:55.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.548+0000 7f6548c00700 1 -- 172.21.15.63:0/2474258527 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6558038050 con 0x5654b34d52c0 2024-05-27T08:00:55.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.548+0000 7f6574358b80 10 client.20676 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:55.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.548+0000 7f6574358b80 1 -- 172.21.15.63:0/2474258527 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654b33b9060 con 0x5654b34d52c0 2024-05-27T08:00:55.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.549+0000 7f6548c00700 1 -- 172.21.15.63:0/2474258527 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65500ad770 con 0x5654b34d52c0 2024-05-27T08:00:55.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.549+0000 7f6548c00700 1 -- 172.21.15.63:0/2474258527 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6558011070 con 0x5654b34d52c0 2024-05-27T08:00:55.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.549+0000 7f6574358b80 10 client.20676 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:55.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.549+0000 7f6574358b80 10 client.20676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:00:55.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.549+0000 7f6574358b80 20 client.20676 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:55.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.550+0000 7f6574358b80 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5654b34e4c50 0x5654b34e70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.550+0000 7f6574358b80 4 client.20676 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:55.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.550+0000 7f6574358b80 1 -- 172.21.15.63:0/2474258527 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654b33fad90 con 0x5654b34e4c50 2024-05-27T08:00:55.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.550+0000 7f655ea00700 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5654b34e4c50 0x5654b34e70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.551+0000 7f655ea00700 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5654b34e4c50 0x5654b34e70f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2442edf13b1bc328 server_cookie=606d3c8dc591948e in_seq=0 out_seq=0 2024-05-27T08:00:55.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.551+0000 7f6548c00700 10 client.20676 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:00:55.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.553+0000 7f6548c00700 1 -- 172.21.15.63:0/2474258527 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5654b33fad90 con 0x5654b34e4c50 2024-05-27T08:00:55.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.553+0000 7f6548c00700 10 client.20676 handle_command_reply: tid=0 2024-05-27T08:00:55.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.553+0000 7f6574358b80 1 client.20676 shutdown 2024-05-27T08:00:55.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 1 -- 172.21.15.63:0/2474258527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 msgr2=0x5654b34d2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d2d20 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f6558007910 tx=0x7f655800b3e0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 1 -- 172.21.15.63:0/2474258527 shutdown_connections 2024-05-27T08:00:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5654b34e4c50 0x5654b34e70f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d2d20 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b334be40 0x5654b34d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 1 --2- 172.21.15.63:0/2474258527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b34379f0 0x5654b34d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 1 -- 172.21.15.63:0/2474258527 >> 172.21.15.63:0/2474258527 conn(0x5654b33e5390 msgr2=0x5654b34c6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6548c00700 0 client.20676 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:00:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 1 -- 172.21.15.63:0/2474258527 shutdown_connections 2024-05-27T08:00:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 1 -- 172.21.15.63:0/2474258527 wait complete. 2024-05-27T08:00:55.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.554+0000 7f6574358b80 20 client.20676 trim_cache size 0 max 16384 2024-05-27T08:00:55.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.582+0000 7f6574358b80 1 Processor -- start 2024-05-27T08:00:55.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.582+0000 7f6574358b80 1 -- start start 2024-05-27T08:00:55.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.583+0000 7f6574358b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 0x5654b34cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.583+0000 7f6574358b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b34379f0 0x5654b34e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.583+0000 7f6574358b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.583+0000 7f6574358b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b4e70 con 0x5654b34d52c0 2024-05-27T08:00:55.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.583+0000 7f6574358b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b2700 con 0x5654b334be40 2024-05-27T08:00:55.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.583+0000 7f6574358b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b0570 con 0x5654b34379f0 2024-05-27T08:00:55.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 0x5654b34cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 0x5654b34cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46646/0 (socket says 172.21.15.63:46646) 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655ea00700 1 -- 172.21.15.63:0/2573557439 learned_addr learned my addr 172.21.15.63:0/2573557439 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655e000700 1 --2- 172.21.15.63:0/2573557439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b34379f0 0x5654b34e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655f400700 1 --2- 172.21.15.63:0/2573557439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655ea00700 1 -- 172.21.15.63:0/2573557439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b34379f0 msgr2=0x5654b34e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655ea00700 1 --2- 172.21.15.63:0/2573557439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b34379f0 0x5654b34e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655ea00700 1 -- 172.21.15.63:0/2573557439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 msgr2=0x5654b34ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655ea00700 1 --2- 172.21.15.63:0/2573557439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655ea00700 1 -- 172.21.15.63:0/2573557439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b34d3090 con 0x5654b334be40 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.584+0000 7f655f400700 1 --2- 172.21.15.63:0/2573557439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:55.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.585+0000 7f655ea00700 1 --2- 172.21.15.63:0/2573557439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 0x5654b34cbdc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65500a90c0 tx=0x7f65500a9540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6868e337dad33fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.585+0000 7f654b400700 1 -- 172.21.15.63:0/2573557439 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550020070 con 0x5654b334be40 2024-05-27T08:00:55.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.586+0000 7f654b400700 1 -- 172.21.15.63:0/2573557439 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65500106a0 con 0x5654b334be40 2024-05-27T08:00:55.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.586+0000 7f654b400700 1 -- 172.21.15.63:0/2573557439 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65500b1ff0 con 0x5654b334be40 2024-05-27T08:00:55.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.587+0000 7f6574358b80 1 -- 172.21.15.63:0/2573557439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 msgr2=0x5654b34cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.587+0000 7f6574358b80 1 --2- 172.21.15.63:0/2573557439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 0x5654b34cbdc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65500a90c0 tx=0x7f65500a9540 comp rx=0 tx=0).stop 2024-05-27T08:00:55.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.587+0000 7f6574358b80 1 -- 172.21.15.63:0/2573557439 shutdown_connections 2024-05-27T08:00:55.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.587+0000 7f6574358b80 1 --2- 172.21.15.63:0/2573557439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.587+0000 7f6574358b80 1 --2- 172.21.15.63:0/2573557439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 0x5654b34cbdc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.587+0000 7f6574358b80 1 --2- 172.21.15.63:0/2573557439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b34379f0 0x5654b34e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.587+0000 7f6574358b80 1 -- 172.21.15.63:0/2573557439 >> 172.21.15.63:0/2573557439 conn(0x5654b33e5390 msgr2=0x5654b34ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.587+0000 7f6574358b80 1 -- 172.21.15.63:0/2573557439 shutdown_connections 2024-05-27T08:00:55.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.588+0000 7f6574358b80 1 -- 172.21.15.63:0/2573557439 wait complete. 2024-05-27T08:00:55.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.588+0000 7f6574358b80 1 -- start start 2024-05-27T08:00:55.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.589+0000 7f6574358b80 1 Processor -- start 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.589+0000 7f6574358b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 0x5654b34dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.589+0000 7f6574358b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b34379f0 0x5654b34d87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.589+0000 7f6574358b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.589+0000 7f6574358b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b0bc0 con 0x5654b34d52c0 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.589+0000 7f6574358b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b2030 con 0x5654b334be40 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.589+0000 7f6574358b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b1290 con 0x5654b34379f0 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.589+0000 7f6574358b80 10 client.0 _ll_register_callbacks cb 0x5654b3392c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.590+0000 7f655f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.590+0000 7f655ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 0x5654b34dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.590+0000 7f655f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42910/0 (socket says 172.21.15.63:42910) 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.590+0000 7f655f400700 1 -- 172.21.15.63:0/654005538 learned_addr learned my addr 172.21.15.63:0/654005538 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.590+0000 7f655e000700 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b34379f0 0x5654b34d87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.590+0000 7f655f400700 1 -- 172.21.15.63:0/654005538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b34379f0 msgr2=0x5654b34d87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.590+0000 7f655f400700 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b34379f0 0x5654b34d87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.590+0000 7f655f400700 1 -- 172.21.15.63:0/654005538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 msgr2=0x5654b34dd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.590+0000 7f655f400700 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 0x5654b34dd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.591+0000 7f655f400700 1 -- 172.21.15.63:0/654005538 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b34d9260 con 0x5654b34d52c0 2024-05-27T08:00:55.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.591+0000 7f655f400700 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d8d20 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f6558009fd0 tx=0x7f6558005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20696d14d14ba9f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.591+0000 7f6548c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.591+0000 7f6548c00700 1 -- 172.21.15.63:0/654005538 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655801c410 con 0x5654b34d52c0 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.591+0000 7f6548c00700 1 -- 172.21.15.63:0/654005538 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f655801c5b0 con 0x5654b34d52c0 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.591+0000 7f6574358b80 1 -- 172.21.15.63:0/654005538 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65500106a0 con 0x5654b34d52c0 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.592+0000 7f6548c00700 1 -- 172.21.15.63:0/654005538 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f655801c7b0 con 0x5654b34d52c0 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.592+0000 7f6574358b80 10 client.20682 fetch_fsmap learned FSMap version 35 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.592+0000 7f6574358b80 1 -- 172.21.15.63:0/654005538 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654b3557c50 con 0x5654b34d52c0 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.592+0000 7f6548c00700 1 -- 172.21.15.63:0/654005538 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65500106a0 con 0x5654b34d52c0 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.592+0000 7f6548c00700 1 -- 172.21.15.63:0/654005538 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6558011040 con 0x5654b34d52c0 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.592+0000 7f6574358b80 10 client.20682 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.592+0000 7f6574358b80 10 client.20682 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.592+0000 7f6574358b80 20 client.20682 populate_metadata read hostname 'smithi063' 2024-05-27T08:00:55.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.592+0000 7f6574358b80 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5654b3558290 0x5654b355a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.593+0000 7f6574358b80 4 client.20682 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:00:55.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.593+0000 7f6574358b80 1 -- 172.21.15.63:0/654005538 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654b33f4570 con 0x5654b3558290 2024-05-27T08:00:55.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.593+0000 7f655ea00700 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5654b3558290 0x5654b355a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.594+0000 7f655ea00700 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5654b3558290 0x5654b355a6b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=720b6df5509426b1 server_cookie=58b32d59540149f4 in_seq=0 out_seq=0 2024-05-27T08:00:55.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.594+0000 7f6548c00700 10 client.20682 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:00:55.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6548c00700 1 -- 172.21.15.63:0/654005538 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654b33f4570 con 0x5654b3558290 2024-05-27T08:00:55.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6548c00700 10 client.20682 handle_command_reply: tid=0 2024-05-27T08:00:55.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6574358b80 1 client.20682 shutdown 2024-05-27T08:00:55.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6574358b80 1 -- 172.21.15.63:0/654005538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 msgr2=0x5654b34d8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6574358b80 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d8d20 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f6558009fd0 tx=0x7f6558005f10 comp rx=0 tx=0).stop 2024-05-27T08:00:55.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6574358b80 1 -- 172.21.15.63:0/654005538 shutdown_connections 2024-05-27T08:00:55.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6574358b80 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5654b3558290 0x5654b355a6b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6574358b80 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654b34d52c0 0x5654b34d8d20 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6574358b80 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b334be40 0x5654b34dd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6574358b80 1 --2- 172.21.15.63:0/654005538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5654b34379f0 0x5654b34d87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6574358b80 1 -- 172.21.15.63:0/654005538 >> 172.21.15.63:0/654005538 conn(0x5654b33e5390 msgr2=0x5654b34d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6548c00700 0 client.20682 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:00:55.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.595+0000 7f6574358b80 1 -- 172.21.15.63:0/654005538 shutdown_connections 2024-05-27T08:00:55.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.596+0000 7f6574358b80 1 -- 172.21.15.63:0/654005538 wait complete. 2024-05-27T08:00:55.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.596+0000 7f6574358b80 20 client.20682 trim_cache size 0 max 16384 2024-05-27T08:00:55.598 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 -- 172.21.15.63:0/2735041440 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65440b29f0 msgr2=0x7f65440b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65440b29f0 0x7f65440b4f00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x5654b34cd210 tx=0x7f655800b040 comp rx=0 tx=0).stop 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 -- 172.21.15.63:0/2735041440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65601043f0 msgr2=0x7f6560075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65601043f0 0x7f6560075ab0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f655000d950 tx=0x7f655000dc60 comp rx=0 tx=0).stop 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 -- 172.21.15.63:0/2735041440 shutdown_connections 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f65440b29f0 0x7f65440b4f00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6560109670 0x7f6560076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65601043f0 0x7f6560075ab0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 --2- 172.21.15.63:0/2735041440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6560108d20 0x7f6560075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 -- 172.21.15.63:0/2735041440 >> 172.21.15.63:0/2735041440 conn(0x7f6560079d00 msgr2=0x7f6560101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 -- 172.21.15.63:0/2735041440 shutdown_connections 2024-05-27T08:00:55.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:00:55.597+0000 7f655fe00700 1 -- 172.21.15.63:0/2735041440 wait complete. 2024-05-27T08:00:55.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:55 smithi086 ceph-mon[34687]: pgmap v866: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T08:00:55.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:55 smithi149 ceph-mon[35408]: pgmap v866: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T08:00:56.530 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:00:57.031 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:00:57.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:56 smithi086 ceph-mon[34687]: pgmap v867: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T08:00:57.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:56 smithi149 ceph-mon[35408]: pgmap v867: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T08:00:57.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:56 smithi063 ceph-mon[31613]: pgmap v867: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 7 op/s 2024-05-27T08:00:59.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:00:59 smithi086 ceph-mon[34687]: pgmap v868: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T08:00:59.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:00:59 smithi149 ceph-mon[35408]: pgmap v868: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T08:00:59.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:00:59 smithi063 ceph-mon[31613]: pgmap v868: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T08:01:00.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.960+0000 7fe6a3400700 1 -- 172.21.15.63:0/4005899680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe69c105510 msgr2=0x7fe69c105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:00.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.960+0000 7fe6a3400700 1 --2- 172.21.15.63:0/4005899680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe69c105510 0x7fe69c105920 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fe69000b0a0 tx=0x7fe69000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.960+0000 7fe6a3400700 1 -- 172.21.15.63:0/4005899680 shutdown_connections 2024-05-27T08:01:00.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.960+0000 7fe6a3400700 1 --2- 172.21.15.63:0/4005899680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe69c10e8f0 0x7fe69c110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.960+0000 7fe6a3400700 1 --2- 172.21.15.63:0/4005899680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe69c105510 0x7fe69c105920 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.960+0000 7fe6a3400700 1 --2- 172.21.15.63:0/4005899680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe69c105e60 0x7fe69c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.960+0000 7fe6a3400700 1 -- 172.21.15.63:0/4005899680 >> 172.21.15.63:0/4005899680 conn(0x7fe69c0fedb0 msgr2=0x7fe69c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:00.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.961+0000 7fe6a3400700 1 -- 172.21.15.63:0/4005899680 shutdown_connections 2024-05-27T08:01:00.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.961+0000 7fe6a3400700 1 -- 172.21.15.63:0/4005899680 wait complete. 2024-05-27T08:01:00.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.962+0000 7fe6a3400700 1 Processor -- start 2024-05-27T08:01:00.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.963+0000 7fe6a3400700 1 -- start start 2024-05-27T08:01:00.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.963+0000 7fe6a3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe69c105510 0x7fe69c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.963+0000 7fe6a3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe69c105e60 0x7fe69c19d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.963+0000 7fe6a3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe69c10e8f0 0x7fe69c1a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.963+0000 7fe6a3400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c103c90 con 0x7fe69c105e60 2024-05-27T08:01:00.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.963+0000 7fe6a3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c103b10 con 0x7fe69c10e8f0 2024-05-27T08:01:00.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.963+0000 7fe6a3400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c103e10 con 0x7fe69c105510 2024-05-27T08:01:00.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe69be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe69c105e60 0x7fe69c19d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe69be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe69c105e60 0x7fe69c19d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41124/0 (socket says 172.21.15.63:41124) 2024-05-27T08:01:00.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe69be00700 1 -- 172.21.15.63:0/3672240998 learned_addr learned my addr 172.21.15.63:0/3672240998 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:00.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe69c105510 0x7fe69c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe6a1400700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe69c10e8f0 0x7fe69c1a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe69be00700 1 -- 172.21.15.63:0/3672240998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe69c105510 msgr2=0x7fe69c19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:00.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe69be00700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe69c105510 0x7fe69c19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe69be00700 1 -- 172.21.15.63:0/3672240998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe69c10e8f0 msgr2=0x7fe69c1a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:00.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe69be00700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe69c10e8f0 0x7fe69c1a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe69be00700 1 -- 172.21.15.63:0/3672240998 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c1a1e10 con 0x7fe69c105e60 2024-05-27T08:01:00.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.964+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe69c105510 0x7fe69c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:00.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.965+0000 7fe69be00700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe69c105e60 0x7fe69c19d4e0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fe68800d950 tx=0x7fe68800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:00.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.965+0000 7fe699600700 1 -- 172.21.15.63:0/3672240998 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688016070 con 0x7fe69c105e60 2024-05-27T08:01:00.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.965+0000 7fe6a3400700 1 -- 172.21.15.63:0/3672240998 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69c1a2100 con 0x7fe69c105e60 2024-05-27T08:01:00.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.965+0000 7fe6a3400700 1 -- 172.21.15.63:0/3672240998 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69c1a9a20 con 0x7fe69c105e60 2024-05-27T08:01:00.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.965+0000 7fe699600700 1 -- 172.21.15.63:0/3672240998 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe688010460 con 0x7fe69c105e60 2024-05-27T08:01:00.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.966+0000 7fe699600700 1 -- 172.21.15.63:0/3672240998 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688020060 con 0x7fe69c105e60 2024-05-27T08:01:00.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.970+0000 7fe699600700 1 -- 172.21.15.63:0/3672240998 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe688019070 con 0x7fe69c105e60 2024-05-27T08:01:00.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.971+0000 7fe6b1f68b80 1 Processor -- start 2024-05-27T08:01:00.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.971+0000 7fe6b1f68b80 1 -- start start 2024-05-27T08:01:00.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.972+0000 7fe699600700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6840b29f0 0x7fe6840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.972+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc889330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.972+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc889870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.972+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc896110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.972+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5615cc7b7e40 con 0x5615cc889e70 2024-05-27T08:01:00.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.972+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5615cc7a79e0 con 0x5615cc70ae40 2024-05-27T08:01:00.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.972+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5615cc7aba20 con 0x5615cc7f69f0 2024-05-27T08:01:00.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.972+0000 7fe699600700 1 -- 172.21.15.63:0/3672240998 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7fe6880e24e0 con 0x7fe69c105e60 2024-05-27T08:01:00.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.972+0000 7fe6a1400700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6840b29f0 0x7fe6840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.973+0000 7fe6a0a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc889330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.973+0000 7fe6a0a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc889330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:33546/0 (socket says 172.21.15.63:33546) 2024-05-27T08:01:00.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.973+0000 7fe6a0a00700 1 -- 172.21.15.63:0/4158144599 learned_addr learned my addr 172.21.15.63:0/4158144599 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:00.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.973+0000 7fe6a1400700 1 --2- 172.21.15.63:0/4158144599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc896110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.973+0000 7fe69be00700 1 --2- 172.21.15.63:0/4158144599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc889870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.974+0000 7fe6a1400700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6840b29f0 0x7fe6840b4f00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe694013400 tx=0x7fe6940137c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:00.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.974+0000 7fe6a0a00700 1 -- 172.21.15.63:0/4158144599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 msgr2=0x5615cc889870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:00.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.974+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/4158144599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc889870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.974+0000 7fe6a0a00700 1 -- 172.21.15.63:0/4158144599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 msgr2=0x5615cc896110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:00.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.974+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/4158144599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc896110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.974+0000 7fe6a0a00700 1 -- 172.21.15.63:0/4158144599 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615cc770420 con 0x5615cc7f69f0 2024-05-27T08:01:00.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.974+0000 7fe6a1400700 1 --2- 172.21.15.63:0/4158144599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc896110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:01:00.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.974+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/4158144599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc889330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe690009dd0 tx=0x7fe690009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20ce32e6ffa790b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:00.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.975+0000 7fe68e000700 1 -- 172.21.15.63:0/4158144599 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690022070 con 0x5615cc7f69f0 2024-05-27T08:01:00.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.975+0000 7fe68e000700 1 -- 172.21.15.63:0/4158144599 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe690009480 con 0x5615cc7f69f0 2024-05-27T08:01:00.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.975+0000 7fe68e000700 1 -- 172.21.15.63:0/4158144599 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69001c440 con 0x5615cc7f69f0 2024-05-27T08:01:00.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.976+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/4158144599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 msgr2=0x5615cc889330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:00.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.976+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/4158144599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc889330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe690009dd0 tx=0x7fe690009e00 comp rx=0 tx=0).stop 2024-05-27T08:01:00.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.976+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/4158144599 shutdown_connections 2024-05-27T08:01:00.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.976+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/4158144599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc896110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.976+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/4158144599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc889330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.976+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/4158144599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc889870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.976+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/4158144599 >> 172.21.15.63:0/4158144599 conn(0x5615cc7a4390 msgr2=0x5615cc8854c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:00.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.976+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/4158144599 shutdown_connections 2024-05-27T08:01:00.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.976+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/4158144599 wait complete. 2024-05-27T08:01:00.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.977+0000 7fe6b1f68b80 1 -- start start 2024-05-27T08:01:00.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.977+0000 7fe6b1f68b80 1 Processor -- start 2024-05-27T08:01:00.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.978+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc8935a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.978+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc893ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.978+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc894020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.978+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5615cc6adfa0 con 0x5615cc889e70 2024-05-27T08:01:00.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.978+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5615cc748760 con 0x5615cc70ae40 2024-05-27T08:01:00.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.978+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5615cc896b20 con 0x5615cc7f69f0 2024-05-27T08:01:00.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.978+0000 7fe6b1f68b80 10 client.0 _ll_register_callbacks cb 0x5615cc751c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:00.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.979+0000 7fe6a1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc894020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.979+0000 7fe6a1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc894020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41154/0 (socket says 172.21.15.63:41154) 2024-05-27T08:01:00.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.979+0000 7fe6a1400700 1 -- 172.21.15.63:0/2944730702 learned_addr learned my addr 172.21.15.63:0/2944730702 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:00.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.979+0000 7fe69be00700 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc893ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.979+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc8935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.979+0000 7fe6a1400700 1 -- 172.21.15.63:0/2944730702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 msgr2=0x5615cc893ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:00.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.979+0000 7fe6a1400700 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc893ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.979+0000 7fe6a1400700 1 -- 172.21.15.63:0/2944730702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 msgr2=0x5615cc8935a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:00.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.979+0000 7fe6a1400700 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc8935a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.979+0000 7fe6a1400700 1 -- 172.21.15.63:0/2944730702 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615cc6f78d0 con 0x5615cc889e70 2024-05-27T08:01:00.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.980+0000 7fe6a1400700 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc894020 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fe694013fd0 tx=0x7fe6940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f9d3730f6d60b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:00.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.980+0000 7fe682a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:00.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.980+0000 7fe682a00700 1 -- 172.21.15.63:0/2944730702 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69402b680 con 0x5615cc889e70 2024-05-27T08:01:00.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.980+0000 7fe682a00700 1 -- 172.21.15.63:0/2944730702 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7fe694020450 con 0x5615cc889e70 2024-05-27T08:01:00.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.980+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2944730702 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615cc7b9d90 con 0x5615cc889e70 2024-05-27T08:01:00.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.981+0000 7fe682a00700 1 -- 172.21.15.63:0/2944730702 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe69402b820 con 0x5615cc889e70 2024-05-27T08:01:00.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.981+0000 7fe6b1f68b80 10 client.20694 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:00.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.981+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2944730702 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5615cc778060 con 0x5615cc889e70 2024-05-27T08:01:00.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.981+0000 7fe682a00700 1 -- 172.21.15.63:0/2944730702 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5615cc7b9d90 con 0x5615cc889e70 2024-05-27T08:01:00.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.982+0000 7fe682a00700 1 -- 172.21.15.63:0/2944730702 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe694020b50 con 0x5615cc889e70 2024-05-27T08:01:00.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.982+0000 7fe6b1f68b80 10 client.20694 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:00.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.982+0000 7fe6b1f68b80 10 client.20694 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:01:00.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.982+0000 7fe6b1f68b80 20 client.20694 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:00.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.982+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5615cc8a39f0 0x5615cc8a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:00.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.982+0000 7fe6b1f68b80 4 client.20694 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:00.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.982+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2944730702 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615cc7b3570 con 0x5615cc8a39f0 2024-05-27T08:01:00.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.983+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5615cc8a39f0 0x5615cc8a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:00.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.984+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5615cc8a39f0 0x5615cc8a5e50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d59ebfe8fc1b1ff6 server_cookie=fb817d42768b10d7 in_seq=0 out_seq=0 2024-05-27T08:01:00.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.984+0000 7fe682a00700 10 client.20694 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:01:00.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe682a00700 1 -- 172.21.15.63:0/2944730702 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5615cc7b3570 con 0x5615cc8a39f0 2024-05-27T08:01:00.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe682a00700 10 client.20694 handle_command_reply: tid=0 2024-05-27T08:01:00.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe6b1f68b80 1 client.20694 shutdown 2024-05-27T08:01:00.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2944730702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 msgr2=0x5615cc894020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:00.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc894020 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fe694013fd0 tx=0x7fe6940047c0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2944730702 shutdown_connections 2024-05-27T08:01:00.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5615cc8a39f0 0x5615cc8a5e50 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc894020 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc8935a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2944730702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc893ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:00.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.987+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2944730702 >> 172.21.15.63:0/2944730702 conn(0x5615cc7a4390 msgr2=0x5615cc889310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:00.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.988+0000 7fe682a00700 0 client.20694 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:01:00.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.988+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2944730702 shutdown_connections 2024-05-27T08:01:00.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.988+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2944730702 wait complete. 2024-05-27T08:01:00.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:00.988+0000 7fe6b1f68b80 20 client.20694 trim_cache size 0 max 16384 2024-05-27T08:01:01.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.015+0000 7fe6b1f68b80 1 Processor -- start 2024-05-27T08:01:01.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.015+0000 7fe6b1f68b80 1 -- start start 2024-05-27T08:01:01.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc70ae40 0x5615cc8a6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:01.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc7f69f0 0x5615cc88ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:01.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc889e70 0x5615cc885590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:01.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c011610 con 0x5615cc7f69f0 2024-05-27T08:01:01.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c00eea0 con 0x5615cc889e70 2024-05-27T08:01:01.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c00cd10 con 0x5615cc70ae40 2024-05-27T08:01:01.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe69be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc7f69f0 0x5615cc88ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:01.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe69be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc7f69f0 0x5615cc88ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41168/0 (socket says 172.21.15.63:41168) 2024-05-27T08:01:01.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe69be00700 1 -- 172.21.15.63:0/3281863000 learned_addr learned my addr 172.21.15.63:0/3281863000 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:01.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/3281863000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc70ae40 0x5615cc8a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:01.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.016+0000 7fe6a1400700 1 --2- 172.21.15.63:0/3281863000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc889e70 0x5615cc885590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:01.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.017+0000 7fe69be00700 1 -- 172.21.15.63:0/3281863000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc70ae40 msgr2=0x5615cc8a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:01.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.017+0000 7fe69be00700 1 --2- 172.21.15.63:0/3281863000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc70ae40 0x5615cc8a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.017+0000 7fe69be00700 1 -- 172.21.15.63:0/3281863000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc889e70 msgr2=0x5615cc885590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:01.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.017+0000 7fe69be00700 1 --2- 172.21.15.63:0/3281863000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc889e70 0x5615cc885590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.017+0000 7fe69be00700 1 -- 172.21.15.63:0/3281863000 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615cc891b10 con 0x5615cc7f69f0 2024-05-27T08:01:01.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.017+0000 7fe69be00700 1 --2- 172.21.15.63:0/3281863000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc7f69f0 0x5615cc88ed50 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fe6880a5480 tx=0x7fe688091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9da05e8ea167e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:01.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.018+0000 7fe68d600700 1 -- 172.21.15.63:0/3281863000 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6880a9400 con 0x5615cc7f69f0 2024-05-27T08:01:01.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.018+0000 7fe68d600700 1 -- 172.21.15.63:0/3281863000 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6880ad7b0 con 0x5615cc7f69f0 2024-05-27T08:01:01.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.018+0000 7fe68d600700 1 -- 172.21.15.63:0/3281863000 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6880b2030 con 0x5615cc7f69f0 2024-05-27T08:01:01.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.019+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/3281863000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc7f69f0 msgr2=0x5615cc88ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:01.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.019+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/3281863000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc7f69f0 0x5615cc88ed50 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fe6880a5480 tx=0x7fe688091040 comp rx=0 tx=0).stop 2024-05-27T08:01:01.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.019+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/3281863000 shutdown_connections 2024-05-27T08:01:01.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.019+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/3281863000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc889e70 0x5615cc885590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.019+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/3281863000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc70ae40 0x5615cc8a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.019+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/3281863000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc7f69f0 0x5615cc88ed50 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.019+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/3281863000 >> 172.21.15.63:0/3281863000 conn(0x5615cc7a4390 msgr2=0x5615cc884e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:01.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.020+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/3281863000 shutdown_connections 2024-05-27T08:01:01.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.020+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/3281863000 wait complete. 2024-05-27T08:01:01.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.021+0000 7fe6b1f68b80 1 -- start start 2024-05-27T08:01:01.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.021+0000 7fe6b1f68b80 1 Processor -- start 2024-05-27T08:01:01.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.021+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc88d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:01.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc88dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:01.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe6b1f68b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc89ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:01.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c00d360 con 0x5615cc889e70 2024-05-27T08:01:01.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c00e7d0 con 0x5615cc70ae40 2024-05-27T08:01:01.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe6b1f68b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c00da30 con 0x5615cc7f69f0 2024-05-27T08:01:01.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe6b1f68b80 10 client.0 _ll_register_callbacks cb 0x5615cc751c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:01.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe6a1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc89ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:01.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe6a1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc89ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41180/0 (socket says 172.21.15.63:41180) 2024-05-27T08:01:01.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe6a1400700 1 -- 172.21.15.63:0/2535917276 learned_addr learned my addr 172.21.15.63:0/2535917276 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:01.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe69be00700 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc88dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:01.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.022+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc88d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:01.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.023+0000 7fe6a1400700 1 -- 172.21.15.63:0/2535917276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 msgr2=0x5615cc88dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:01.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.023+0000 7fe6a1400700 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc88dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.023+0000 7fe6a1400700 1 -- 172.21.15.63:0/2535917276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 msgr2=0x5615cc88d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:01.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.023+0000 7fe6a1400700 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc88d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.023+0000 7fe6a1400700 1 -- 172.21.15.63:0/2535917276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615cc89b320 con 0x5615cc889e70 2024-05-27T08:01:01.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.023+0000 7fe6a1400700 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc89ade0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fe69401fcc0 tx=0x7fe69402bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85f503f51a2eae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:01.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.023+0000 7fe682a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:01.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.024+0000 7fe682a00700 1 -- 172.21.15.63:0/2535917276 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69403c020 con 0x5615cc889e70 2024-05-27T08:01:01.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.024+0000 7fe682a00700 1 -- 172.21.15.63:0/2535917276 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7fe6940119f0 con 0x5615cc889e70 2024-05-27T08:01:01.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.024+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2535917276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6880ad7b0 con 0x5615cc889e70 2024-05-27T08:01:01.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.024+0000 7fe682a00700 1 -- 172.21.15.63:0/2535917276 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe694047050 con 0x5615cc889e70 2024-05-27T08:01:01.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.024+0000 7fe6b1f68b80 10 client.20700 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:01.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.024+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2535917276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5615cc89c580 con 0x5615cc889e70 2024-05-27T08:01:01.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.025+0000 7fe682a00700 1 -- 172.21.15.63:0/2535917276 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6880ad7b0 con 0x5615cc889e70 2024-05-27T08:01:01.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.025+0000 7fe682a00700 1 -- 172.21.15.63:0/2535917276 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe694020970 con 0x5615cc889e70 2024-05-27T08:01:01.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.025+0000 7fe6b1f68b80 10 client.20700 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:01.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.025+0000 7fe6b1f68b80 10 client.20700 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:01:01.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.025+0000 7fe6b1f68b80 20 client.20700 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:01.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.025+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5615cc90e520 0x5615cc910940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:01.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.025+0000 7fe6b1f68b80 4 client.20700 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:01:01.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.025+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2535917276 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615cc62a250 con 0x5615cc90e520 2024-05-27T08:01:01.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.026+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5615cc90e520 0x5615cc910940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:01.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.027+0000 7fe682a00700 10 client.20700 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:01:01.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.027+0000 7fe6a0a00700 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5615cc90e520 0x5615cc910940 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3487d606b1f647c server_cookie=e916913c6cdbb237 in_seq=0 out_seq=0 2024-05-27T08:01:01.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.028+0000 7fe682a00700 1 -- 172.21.15.63:0/2535917276 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615cc62a250 con 0x5615cc90e520 2024-05-27T08:01:01.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.028+0000 7fe682a00700 10 client.20700 handle_command_reply: tid=0 2024-05-27T08:01:01.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.028+0000 7fe6b1f68b80 1 client.20700 shutdown 2024-05-27T08:01:01.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.028+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2535917276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 msgr2=0x5615cc89ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.028+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc89ade0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fe69401fcc0 tx=0x7fe69402bc50 comp rx=0 tx=0).stop 2024-05-27T08:01:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.028+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2535917276 shutdown_connections 2024-05-27T08:01:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.028+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5615cc90e520 0x5615cc910940 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.028+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5615cc889e70 0x5615cc89ade0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.028+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615cc70ae40 0x5615cc88d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.028+0000 7fe6b1f68b80 1 --2- 172.21.15.63:0/2535917276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615cc7f69f0 0x5615cc88dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.029+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2535917276 >> 172.21.15.63:0/2535917276 conn(0x5615cc7a4390 msgr2=0x5615cc88aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:01.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.029+0000 7fe682a00700 0 client.20700 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:01:01.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.029+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2535917276 shutdown_connections 2024-05-27T08:01:01.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.029+0000 7fe6b1f68b80 1 -- 172.21.15.63:0/2535917276 wait complete. 2024-05-27T08:01:01.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.030+0000 7fe6b1f68b80 20 client.20700 trim_cache size 0 max 16384 2024-05-27T08:01:01.031 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:01:01.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.030+0000 7fe6a3400700 1 -- 172.21.15.63:0/3672240998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6840b29f0 msgr2=0x7fe6840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:01.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.030+0000 7fe6a3400700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6840b29f0 0x7fe6840b4f00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe694013400 tx=0x7fe6940137c0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.030+0000 7fe6a3400700 1 -- 172.21.15.63:0/3672240998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe69c105e60 msgr2=0x7fe69c19d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:01.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.030+0000 7fe6a3400700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe69c105e60 0x7fe69c19d4e0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fe68800d950 tx=0x7fe68800dd10 comp rx=0 tx=0).stop 2024-05-27T08:01:01.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.031+0000 7fe6a3400700 1 -- 172.21.15.63:0/3672240998 shutdown_connections 2024-05-27T08:01:01.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.031+0000 7fe6a3400700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe6840b29f0 0x7fe6840b4f00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.031+0000 7fe6a3400700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe69c10e8f0 0x7fe69c1a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.031+0000 7fe6a3400700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe69c105510 0x7fe69c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.031+0000 7fe6a3400700 1 --2- 172.21.15.63:0/3672240998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe69c105e60 0x7fe69c19d4e0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:01.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.031+0000 7fe6a3400700 1 -- 172.21.15.63:0/3672240998 >> 172.21.15.63:0/3672240998 conn(0x7fe69c0fedb0 msgr2=0x7fe69c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:01.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.031+0000 7fe6a3400700 1 -- 172.21.15.63:0/3672240998 shutdown_connections 2024-05-27T08:01:01.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:01.031+0000 7fe6a3400700 1 -- 172.21.15.63:0/3672240998 wait complete. 2024-05-27T08:01:01.365 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:01 smithi063 ceph-mon[31613]: pgmap v869: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:01:01.527 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:01 smithi086 ceph-mon[34687]: pgmap v869: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:01:01.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:01 smithi149 ceph-mon[35408]: pgmap v869: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:01:02.026 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:01:02.525 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:03.629 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:03 smithi063 ceph-mon[31613]: pgmap v870: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.5 MiB/s wr, 31 op/s 2024-05-27T08:01:03.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:03 smithi086 ceph-mon[34687]: pgmap v870: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.5 MiB/s wr, 31 op/s 2024-05-27T08:01:03.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:03 smithi149 ceph-mon[35408]: pgmap v870: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.5 MiB/s wr, 31 op/s 2024-05-27T08:01:05.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.092+0000 7f5c14e00700 1 -- 172.21.15.63:0/552162872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c10107de0 msgr2=0x7f5c101081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:05.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.092+0000 7f5c14e00700 1 --2- 172.21.15.63:0/552162872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c10107de0 0x7f5c101081f0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0099c0 tx=0x7f5bfc009cd0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.092+0000 7f5c14e00700 1 -- 172.21.15.63:0/552162872 shutdown_connections 2024-05-27T08:01:05.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.092+0000 7f5c14e00700 1 --2- 172.21.15.63:0/552162872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c10108730 0x7f5c10110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.092+0000 7f5c14e00700 1 --2- 172.21.15.63:0/552162872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c101034b0 0x7f5c101038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.092+0000 7f5c14e00700 1 --2- 172.21.15.63:0/552162872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c10107de0 0x7f5c101081f0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.092+0000 7f5c14e00700 1 -- 172.21.15.63:0/552162872 >> 172.21.15.63:0/552162872 conn(0x7f5c100fedb0 msgr2=0x7f5c101011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:05.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.092+0000 7f5c14e00700 1 -- 172.21.15.63:0/552162872 shutdown_connections 2024-05-27T08:01:05.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.092+0000 7f5c14e00700 1 -- 172.21.15.63:0/552162872 wait complete. 2024-05-27T08:01:05.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.094+0000 7f5c14e00700 1 Processor -- start 2024-05-27T08:01:05.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.094+0000 7f5c14e00700 1 -- start start 2024-05-27T08:01:05.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.094+0000 7f5c14e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c101034b0 0x7f5c100ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.094+0000 7f5c14e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c10107de0 0x7f5c100ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.094+0000 7f5c14e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c10108730 0x7f5c100ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.094+0000 7f5c14e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101131f0 con 0x7f5c10108730 2024-05-27T08:01:05.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.094+0000 7f5c14e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10113070 con 0x7f5c10107de0 2024-05-27T08:01:05.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.094+0000 7f5c14e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10113370 con 0x7f5c101034b0 2024-05-27T08:01:05.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.095+0000 7f5c0e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c10108730 0x7f5c100ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.095+0000 7f5c0e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c10108730 0x7f5c100ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41186/0 (socket says 172.21.15.63:41186) 2024-05-27T08:01:05.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.095+0000 7f5c0e800700 1 -- 172.21.15.63:0/2626845770 learned_addr learned my addr 172.21.15.63:0/2626845770 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:05.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.095+0000 7f5c0de00700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c101034b0 0x7f5c100ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.095+0000 7f5c0d400700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c10107de0 0x7f5c100ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.096+0000 7f5c0e800700 1 -- 172.21.15.63:0/2626845770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c101034b0 msgr2=0x7f5c100ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:05.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.096+0000 7f5c0e800700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c101034b0 0x7f5c100ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.096+0000 7f5c0e800700 1 -- 172.21.15.63:0/2626845770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c10107de0 msgr2=0x7f5c100ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:05.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.096+0000 7f5c0e800700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c10107de0 0x7f5c100ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.096+0000 7f5c0e800700 1 -- 172.21.15.63:0/2626845770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c101a9050 con 0x7f5c10108730 2024-05-27T08:01:05.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.096+0000 7f5c0e800700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c10108730 0x7f5c100ffd10 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f5c08007ee0 tx=0x7f5c0800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:05.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.096+0000 7f5c06a00700 1 -- 172.21.15.63:0/2626845770 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08004bb0 con 0x7f5c10108730 2024-05-27T08:01:05.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.096+0000 7f5c06a00700 1 -- 172.21.15.63:0/2626845770 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c08004d50 con 0x7f5c10108730 2024-05-27T08:01:05.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.097+0000 7f5c14e00700 1 -- 172.21.15.63:0/2626845770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c101a9340 con 0x7f5c10108730 2024-05-27T08:01:05.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.097+0000 7f5c06a00700 1 -- 172.21.15.63:0/2626845770 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c080058b0 con 0x7f5c10108730 2024-05-27T08:01:05.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.097+0000 7f5c14e00700 1 -- 172.21.15.63:0/2626845770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c101a9850 con 0x7f5c10108730 2024-05-27T08:01:05.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.102+0000 7f5c238a3b80 1 Processor -- start 2024-05-27T08:01:05.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.102+0000 7f5c06a00700 1 -- 172.21.15.63:0/2626845770 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5c0801a020 con 0x7f5c10108730 2024-05-27T08:01:05.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.102+0000 7f5c238a3b80 1 -- start start 2024-05-27T08:01:05.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.103+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19439330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.103+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c192bae40 0x559c19439870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.103+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 0x559c19446110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.103+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559c19367e40 con 0x559c193a69f0 2024-05-27T08:01:05.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.103+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c193579e0 con 0x559c192bae40 2024-05-27T08:01:05.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.103+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559c1935ba20 con 0x559c19439e70 2024-05-27T08:01:05.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.103+0000 7f5c0d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19439330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.103+0000 7f5c0d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19439330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41198/0 (socket says 172.21.15.63:41198) 2024-05-27T08:01:05.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.103+0000 7f5c0d400700 1 -- 172.21.15.63:0/3520499141 learned_addr learned my addr 172.21.15.63:0/3520499141 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:05.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.103+0000 7f5c0de00700 1 --2- 172.21.15.63:0/3520499141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c192bae40 0x559c19439870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.104+0000 7f5c0d400700 1 --2- 172.21.15.63:0/3520499141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 0x559c19446110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.104+0000 7f5c0d400700 1 -- 172.21.15.63:0/3520499141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 msgr2=0x559c19446110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:05.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.104+0000 7f5c0d400700 1 --2- 172.21.15.63:0/3520499141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 0x559c19446110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.104+0000 7f5c0d400700 1 -- 172.21.15.63:0/3520499141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c192bae40 msgr2=0x559c19439870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.104+0000 7f5c0d400700 1 --2- 172.21.15.63:0/3520499141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c192bae40 0x559c19439870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.104+0000 7f5c0d400700 1 -- 172.21.15.63:0/3520499141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c19320420 con 0x559c193a69f0 2024-05-27T08:01:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.104+0000 7f5c06a00700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bf40b2a70 0x7f5bf40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.104+0000 7f5c0d400700 1 --2- 172.21.15.63:0/3520499141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19439330 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f5c10076aa0 tx=0x7f5bfc009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe7d680fbcac5ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.105+0000 7f5c06a00700 1 -- 172.21.15.63:0/2626845770 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f5c080e1fe0 con 0x7f5c10108730 2024-05-27T08:01:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.105+0000 7f5bfb400700 1 -- 172.21.15.63:0/3520499141 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc005310 con 0x559c193a69f0 2024-05-27T08:01:05.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.105+0000 7f5bfb400700 1 -- 172.21.15.63:0/3520499141 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bfc0054b0 con 0x559c193a69f0 2024-05-27T08:01:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.105+0000 7f5bfb400700 1 -- 172.21.15.63:0/3520499141 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0039a0 con 0x559c193a69f0 2024-05-27T08:01:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.105+0000 7f5c0de00700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bf40b2a70 0x7f5bf40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.106+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3520499141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 msgr2=0x559c19439330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.106+0000 7f5c0de00700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bf40b2a70 0x7f5bf40b4f80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x559c19436790 tx=0x7f5c0000b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.106+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/3520499141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19439330 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f5c10076aa0 tx=0x7f5bfc009f10 comp rx=0 tx=0).stop 2024-05-27T08:01:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.107+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3520499141 shutdown_connections 2024-05-27T08:01:05.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.107+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/3520499141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 0x559c19446110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.107+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/3520499141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19439330 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.107+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/3520499141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c192bae40 0x559c19439870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.107+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3520499141 >> 172.21.15.63:0/3520499141 conn(0x559c19354390 msgr2=0x559c194354c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.107+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3520499141 shutdown_connections 2024-05-27T08:01:05.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.107+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3520499141 wait complete. 2024-05-27T08:01:05.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.108+0000 7f5c238a3b80 1 -- start start 2024-05-27T08:01:05.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.108+0000 7f5c238a3b80 1 Processor -- start 2024-05-27T08:01:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c192bae40 0x559c19443740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19443c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 0x559c194441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559c1925dfa0 con 0x559c193a69f0 2024-05-27T08:01:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c192f8760 con 0x559c19439e70 2024-05-27T08:01:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559c19446b20 con 0x559c192bae40 2024-05-27T08:01:05.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c238a3b80 10 client.0 _ll_register_callbacks cb 0x559c19301c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:05.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c0d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19443c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c0d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19443c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41204/0 (socket says 172.21.15.63:41204) 2024-05-27T08:01:05.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c0d400700 1 -- 172.21.15.63:0/2457082505 learned_addr learned my addr 172.21.15.63:0/2457082505 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:05.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c0de00700 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c192bae40 0x559c19443740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.109+0000 7f5c0e800700 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 0x559c194441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.110+0000 7f5c0d400700 1 -- 172.21.15.63:0/2457082505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c192bae40 msgr2=0x559c19443740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:05.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.110+0000 7f5c0d400700 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c192bae40 0x559c19443740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.110+0000 7f5c0d400700 1 -- 172.21.15.63:0/2457082505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 msgr2=0x559c194441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:05.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.110+0000 7f5c0d400700 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 0x559c194441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.110+0000 7f5c0d400700 1 -- 172.21.15.63:0/2457082505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c192a78d0 con 0x559c193a69f0 2024-05-27T08:01:05.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.110+0000 7f5c0e800700 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 0x559c194441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:05.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.110+0000 7f5c0d400700 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19443c80 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0098e0 tx=0x7f5bfc00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536efc5a69389dd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:05.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.110+0000 7f5befe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:05.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.111+0000 7f5befe00700 1 -- 172.21.15.63:0/2457082505 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc02f070 con 0x559c193a69f0 2024-05-27T08:01:05.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.111+0000 7f5befe00700 1 -- 172.21.15.63:0/2457082505 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f5bfc003680 con 0x559c193a69f0 2024-05-27T08:01:05.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.111+0000 7f5c238a3b80 1 -- 172.21.15.63:0/2457082505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bfc0054b0 con 0x559c193a69f0 2024-05-27T08:01:05.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.111+0000 7f5befe00700 1 -- 172.21.15.63:0/2457082505 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5bfc03b050 con 0x559c193a69f0 2024-05-27T08:01:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.111+0000 7f5c238a3b80 10 client.20712 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.111+0000 7f5c238a3b80 1 -- 172.21.15.63:0/2457082505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559c19328060 con 0x559c193a69f0 2024-05-27T08:01:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.112+0000 7f5befe00700 1 -- 172.21.15.63:0/2457082505 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bfc0054b0 con 0x559c193a69f0 2024-05-27T08:01:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.112+0000 7f5befe00700 1 -- 172.21.15.63:0/2457082505 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5bfc00fac0 con 0x559c193a69f0 2024-05-27T08:01:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.112+0000 7f5c238a3b80 10 client.20712 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.112+0000 7f5c238a3b80 10 client.20712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.112+0000 7f5c238a3b80 20 client.20712 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.112+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c194538f0 0x559c19455d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.112+0000 7f5c238a3b80 4 client.20712 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:05.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.112+0000 7f5c238a3b80 1 -- 172.21.15.63:0/2457082505 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c19369d90 con 0x559c194538f0 2024-05-27T08:01:05.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.113+0000 7f5c0de00700 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c194538f0 0x559c19455d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.114+0000 7f5c0de00700 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c194538f0 0x559c19455d10 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=119d7902ac7195f1 server_cookie=62e90327d3f9cd9e in_seq=0 out_seq=0 2024-05-27T08:01:05.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.114+0000 7f5befe00700 10 client.20712 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.116+0000 7f5befe00700 1 -- 172.21.15.63:0/2457082505 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559c19369d90 con 0x559c194538f0 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.116+0000 7f5befe00700 10 client.20712 handle_command_reply: tid=0 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.116+0000 7f5c238a3b80 1 client.20712 shutdown 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 1 -- 172.21.15.63:0/2457082505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 msgr2=0x559c19443c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19443c80 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0098e0 tx=0x7f5bfc00b6b0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 1 -- 172.21.15.63:0/2457082505 shutdown_connections 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c194538f0 0x559c19455d10 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 0x559c194441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c192bae40 0x559c19443740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/2457082505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c19443c80 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 1 -- 172.21.15.63:0/2457082505 >> 172.21.15.63:0/2457082505 conn(0x559c19354390 msgr2=0x559c19439450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5befe00700 0 client.20712 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 1 -- 172.21.15.63:0/2457082505 shutdown_connections 2024-05-27T08:01:05.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 1 -- 172.21.15.63:0/2457082505 wait complete. 2024-05-27T08:01:05.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.117+0000 7f5c238a3b80 20 client.20712 trim_cache size 0 max 16384 2024-05-27T08:01:05.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.143+0000 7f5c238a3b80 1 Processor -- start 2024-05-27T08:01:05.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.143+0000 7f5c238a3b80 1 -- start start 2024-05-27T08:01:05.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.143+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c192bae40 0x559c1943ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.143+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c1944fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.144+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 0x559c19435540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.144+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101b4e90 con 0x559c193a69f0 2024-05-27T08:01:05.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.144+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101b2720 con 0x559c19439e70 2024-05-27T08:01:05.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.144+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101b0590 con 0x559c192bae40 2024-05-27T08:01:05.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.144+0000 7f5c0d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c1944fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.144+0000 7f5c0d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c1944fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41216/0 (socket says 172.21.15.63:41216) 2024-05-27T08:01:05.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.144+0000 7f5c0d400700 1 -- 172.21.15.63:0/680048825 learned_addr learned my addr 172.21.15.63:0/680048825 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:05.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.144+0000 7f5c0de00700 1 --2- 172.21.15.63:0/680048825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c192bae40 0x559c1943ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.145+0000 7f5c0e800700 1 --2- 172.21.15.63:0/680048825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 0x559c19435540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.145+0000 7f5c0d400700 1 -- 172.21.15.63:0/680048825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c192bae40 msgr2=0x559c1943ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:05.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.145+0000 7f5c0d400700 1 --2- 172.21.15.63:0/680048825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c192bae40 0x559c1943ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.145+0000 7f5c0d400700 1 -- 172.21.15.63:0/680048825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 msgr2=0x559c19435540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:05.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.145+0000 7f5c0d400700 1 --2- 172.21.15.63:0/680048825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 0x559c19435540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.145+0000 7f5c0d400700 1 -- 172.21.15.63:0/680048825 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c19442a60 con 0x559c193a69f0 2024-05-27T08:01:05.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.145+0000 7f5c0d400700 1 --2- 172.21.15.63:0/680048825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c1944fdc0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc005b10 tx=0x7f5bfc009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796000e4b63735f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:05.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.146+0000 7f5bfaa00700 1 -- 172.21.15.63:0/680048825 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc02f040 con 0x559c193a69f0 2024-05-27T08:01:05.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.146+0000 7f5bfaa00700 1 -- 172.21.15.63:0/680048825 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bfc005490 con 0x559c193a69f0 2024-05-27T08:01:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.146+0000 7f5bfaa00700 1 -- 172.21.15.63:0/680048825 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc027d70 con 0x559c193a69f0 2024-05-27T08:01:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.147+0000 7f5c238a3b80 1 -- 172.21.15.63:0/680048825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 msgr2=0x559c1944fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.147+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/680048825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c1944fdc0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc005b10 tx=0x7f5bfc009710 comp rx=0 tx=0).stop 2024-05-27T08:01:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.147+0000 7f5c238a3b80 1 -- 172.21.15.63:0/680048825 shutdown_connections 2024-05-27T08:01:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.147+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/680048825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c19439e70 0x559c19435540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.147+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/680048825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c192bae40 0x559c1943ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.147+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/680048825 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c193a69f0 0x559c1944fdc0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.147+0000 7f5c238a3b80 1 -- 172.21.15.63:0/680048825 >> 172.21.15.63:0/680048825 conn(0x559c19354390 msgr2=0x559c19434e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.148+0000 7f5c238a3b80 1 -- 172.21.15.63:0/680048825 shutdown_connections 2024-05-27T08:01:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.148+0000 7f5c238a3b80 1 -- 172.21.15.63:0/680048825 wait complete. 2024-05-27T08:01:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.148+0000 7f5c238a3b80 1 -- start start 2024-05-27T08:01:05.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.148+0000 7f5c238a3b80 1 Processor -- start 2024-05-27T08:01:05.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.149+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c192bae40 0x559c19458d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.149+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c193a69f0 0x559c19459280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.149+0000 7f5c238a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 0x559c19456250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.149+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101b0be0 con 0x559c192bae40 2024-05-27T08:01:05.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.149+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101b2050 con 0x559c193a69f0 2024-05-27T08:01:05.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.149+0000 7f5c238a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101b12b0 con 0x559c19439e70 2024-05-27T08:01:05.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.149+0000 7f5c238a3b80 10 client.0 _ll_register_callbacks cb 0x559c19301c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:05.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c192bae40 0x559c19458d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c192bae40 0x559c19458d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41230/0 (socket says 172.21.15.63:41230) 2024-05-27T08:01:05.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0de00700 1 -- 172.21.15.63:0/3698033143 learned_addr learned my addr 172.21.15.63:0/3698033143 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0e800700 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 0x559c19456250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0d400700 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c193a69f0 0x559c19459280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0de00700 1 -- 172.21.15.63:0/3698033143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 msgr2=0x559c19456250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0de00700 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 0x559c19456250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0de00700 1 -- 172.21.15.63:0/3698033143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c193a69f0 msgr2=0x559c19459280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0de00700 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c193a69f0 0x559c19459280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0de00700 1 -- 172.21.15.63:0/3698033143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c19456860 con 0x559c192bae40 2024-05-27T08:01:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0e800700 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 0x559c19456250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.150+0000 7f5c0d400700 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c193a69f0 0x559c19459280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:01:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.151+0000 7f5c0de00700 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c192bae40 0x559c19458d40 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x559c19436a20 tx=0x7f5c00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a007c4da3bfe2d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.151+0000 7f5befe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.151+0000 7f5befe00700 1 -- 172.21.15.63:0/3698033143 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0002e070 con 0x559c192bae40 2024-05-27T08:01:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.151+0000 7f5befe00700 1 -- 172.21.15.63:0/3698033143 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f5c00003e00 con 0x559c192bae40 2024-05-27T08:01:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.151+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3698033143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bfc005490 con 0x559c192bae40 2024-05-27T08:01:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.152+0000 7f5befe00700 1 -- 172.21.15.63:0/3698033143 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5c0003a050 con 0x559c192bae40 2024-05-27T08:01:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.152+0000 7f5c238a3b80 10 client.20718 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.152+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3698033143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559c194bdb40 con 0x559c192bae40 2024-05-27T08:01:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.152+0000 7f5befe00700 1 -- 172.21.15.63:0/3698033143 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bfc005490 con 0x559c192bae40 2024-05-27T08:01:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.153+0000 7f5befe00700 1 -- 172.21.15.63:0/3698033143 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5c0001d410 con 0x559c192bae40 2024-05-27T08:01:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.153+0000 7f5c238a3b80 10 client.20718 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.153+0000 7f5c238a3b80 10 client.20718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.153+0000 7f5c238a3b80 20 client.20718 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.153+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c194be280 0x559c194c06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:05.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.153+0000 7f5c238a3b80 4 client.20718 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:01:05.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.153+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3698033143 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c19363570 con 0x559c194be280 2024-05-27T08:01:05.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.154+0000 7f5c0d400700 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c194be280 0x559c194c06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:05.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.155+0000 7f5c0d400700 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c194be280 0x559c194c06a0 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=742efa0e9f8b731c server_cookie=4352d6015975aaf1 in_seq=0 out_seq=0 2024-05-27T08:01:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.155+0000 7f5befe00700 10 client.20718 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.155+0000 7f5befe00700 1 -- 172.21.15.63:0/3698033143 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c19363570 con 0x559c194be280 2024-05-27T08:01:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.155+0000 7f5befe00700 10 client.20718 handle_command_reply: tid=0 2024-05-27T08:01:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.155+0000 7f5c238a3b80 1 client.20718 shutdown 2024-05-27T08:01:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3698033143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c192bae40 msgr2=0x559c19458d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c192bae40 0x559c19458d40 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x559c19436a20 tx=0x7f5c00004750 comp rx=0 tx=0).stop 2024-05-27T08:01:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3698033143 shutdown_connections 2024-05-27T08:01:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c194be280 0x559c194c06a0 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c19439e70 0x559c19456250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c192bae40 0x559c19458d40 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 1 --2- 172.21.15.63:0/3698033143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c193a69f0 0x559c19459280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5befe00700 0 client.20718 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3698033143 >> 172.21.15.63:0/3698033143 conn(0x559c19354390 msgr2=0x559c19444d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3698033143 shutdown_connections 2024-05-27T08:01:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 1 -- 172.21.15.63:0/3698033143 wait complete. 2024-05-27T08:01:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.156+0000 7f5c238a3b80 20 client.20718 trim_cache size 0 max 16384 2024-05-27T08:01:05.159 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:01:05.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.157+0000 7f5c14e00700 1 -- 172.21.15.63:0/2626845770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bf40b2a70 msgr2=0x7f5bf40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:05.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.157+0000 7f5c14e00700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bf40b2a70 0x7f5bf40b4f80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x559c19436790 tx=0x7f5c0000b410 comp rx=0 tx=0).stop 2024-05-27T08:01:05.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.157+0000 7f5c14e00700 1 -- 172.21.15.63:0/2626845770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c10108730 msgr2=0x7f5c100ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.157+0000 7f5c14e00700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c10108730 0x7f5c100ffd10 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f5c08007ee0 tx=0x7f5c0800d3b0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.158+0000 7f5c14e00700 1 -- 172.21.15.63:0/2626845770 shutdown_connections 2024-05-27T08:01:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.158+0000 7f5c14e00700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5bf40b2a70 0x7f5bf40b4f80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.158+0000 7f5c14e00700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c10108730 0x7f5c100ffd10 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.158+0000 7f5c14e00700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c101034b0 0x7f5c100ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.158+0000 7f5c14e00700 1 --2- 172.21.15.63:0/2626845770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c10107de0 0x7f5c100ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.158+0000 7f5c14e00700 1 -- 172.21.15.63:0/2626845770 >> 172.21.15.63:0/2626845770 conn(0x7f5c100fedb0 msgr2=0x7f5c1006b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.158+0000 7f5c14e00700 1 -- 172.21.15.63:0/2626845770 shutdown_connections 2024-05-27T08:01:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:05.158+0000 7f5c14e00700 1 -- 172.21.15.63:0/2626845770 wait complete. 2024-05-27T08:01:05.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:05 smithi063 ceph-mon[31613]: pgmap v871: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-05-27T08:01:05.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:05 smithi086 ceph-mon[34687]: pgmap v871: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-05-27T08:01:05.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:05 smithi149 ceph-mon[35408]: pgmap v871: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-05-27T08:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.453+0000 7f69a2800640 1 -- 172.21.15.63:0/3204163449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c102b10 msgr2=0x7f699c102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.453+0000 7f69a2800640 1 --2- 172.21.15.63:0/3204163449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c102b10 0x7f699c102f20 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f6988009900 tx=0x7f698802a3f0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.454+0000 7f69a2800640 1 -- 172.21.15.63:0/3204163449 shutdown_connections 2024-05-27T08:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.454+0000 7f69a2800640 1 --2- 172.21.15.63:0/3204163449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c10d350 0x7f699c10f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.454+0000 7f69a2800640 1 --2- 172.21.15.63:0/3204163449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c102b10 0x7f699c102f20 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.454+0000 7f69a2800640 1 --2- 172.21.15.63:0/3204163449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f699c1089f0 0x7f699c1025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.454+0000 7f69a2800640 1 -- 172.21.15.63:0/3204163449 >> 172.21.15.63:0/3204163449 conn(0x7f699c0fe210 msgr2=0x7f699c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.454+0000 7f69a2800640 1 -- 172.21.15.63:0/3204163449 shutdown_connections 2024-05-27T08:01:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.454+0000 7f69a2800640 1 -- 172.21.15.63:0/3204163449 wait complete. 2024-05-27T08:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.455+0000 7f69a2800640 1 Processor -- start 2024-05-27T08:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.455+0000 7f69a2800640 1 -- start start 2024-05-27T08:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f699c102b10 0x7f699c1101b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a2800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c1089f0 0x7f699c1106f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a2800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c10d350 0x7f699c110c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a2800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f699c113bb0 con 0x7f699c10d350 2024-05-27T08:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f699c113a30 con 0x7f699c102b10 2024-05-27T08:01:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a2800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f699c113d30 con 0x7f699c1089f0 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c10d350 0x7f699c110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c10d350 0x7f699c110c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41256/0 (socket says 172.21.15.63:41256) 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a1e00640 1 -- 172.21.15.63:0/482695555 learned_addr learned my addr 172.21.15.63:0/482695555 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a1400640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f699c102b10 0x7f699c1101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.456+0000 7f69a0a00640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c1089f0 0x7f699c1106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.457+0000 7f69a1e00640 1 -- 172.21.15.63:0/482695555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c1089f0 msgr2=0x7f699c1106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.457+0000 7f69a1e00640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c1089f0 0x7f699c1106f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.457+0000 7f69a1e00640 1 -- 172.21.15.63:0/482695555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f699c102b10 msgr2=0x7f699c1101b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.457+0000 7f69a1e00640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f699c102b10 0x7f699c1101b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.457+0000 7f69a1e00640 1 -- 172.21.15.63:0/482695555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c1a8950 con 0x7f699c10d350 2024-05-27T08:01:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.457+0000 7f69a1e00640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c10d350 0x7f699c110c30 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f699400eaa0 tx=0x7f699400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.457+0000 7f699a000640 1 -- 172.21.15.63:0/482695555 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699400cdb0 con 0x7f699c10d350 2024-05-27T08:01:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.457+0000 7f699a000640 1 -- 172.21.15.63:0/482695555 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6994005020 con 0x7f699c10d350 2024-05-27T08:01:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.457+0000 7f69a2800640 1 -- 172.21.15.63:0/482695555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c1a8c40 con 0x7f699c10d350 2024-05-27T08:01:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.458+0000 7f699a000640 1 -- 172.21.15.63:0/482695555 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69940116d0 con 0x7f699c10d350 2024-05-27T08:01:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.458+0000 7f69a2800640 1 -- 172.21.15.63:0/482695555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c1a9160 con 0x7f699c10d350 2024-05-27T08:01:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.461+0000 7f69aadb9740 1 Processor -- start 2024-05-27T08:01:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.462+0000 7f699a000640 1 -- 172.21.15.63:0/482695555 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f69940051d0 con 0x7f699c10d350 2024-05-27T08:01:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.462+0000 7f69aadb9740 1 -- start start 2024-05-27T08:01:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.462+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6f121b0 0x55e4f72b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.462+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6ee92f0 0x55e4f72b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.463+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.463+0000 7f69aadb9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e4f7225b90 con 0x55e4f6f121b0 2024-05-27T08:01:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.463+0000 7f69aadb9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e4f7215730 con 0x55e4f6ef14d0 2024-05-27T08:01:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.463+0000 7f69aadb9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e4f7219770 con 0x55e4f6ee92f0 2024-05-27T08:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.463+0000 7f69a0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6f121b0 0x55e4f72b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.463+0000 7f69a0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6f121b0 0x55e4f72b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41270/0 (socket says 172.21.15.63:41270) 2024-05-27T08:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.463+0000 7f69a0a00640 1 -- 172.21.15.63:0/302754384 learned_addr learned my addr 172.21.15.63:0/302754384 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.463+0000 7f69a1400640 1 --2- 172.21.15.63:0/302754384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6ee92f0 0x55e4f72b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.463+0000 7f69a0a00640 1 --2- 172.21.15.63:0/302754384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.464+0000 7f69a0a00640 1 -- 172.21.15.63:0/302754384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6ee92f0 msgr2=0x55e4f72b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.464+0000 7f69a0a00640 1 --2- 172.21.15.63:0/302754384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6ee92f0 0x55e4f72b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.464+0000 7f699a000640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69840b29d0 0x7f69840b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.464+0000 7f69a0a00640 1 -- 172.21.15.63:0/302754384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 msgr2=0x55e4f72b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.464+0000 7f69a0a00640 1 --2- 172.21.15.63:0/302754384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.464+0000 7f69a0a00640 1 -- 172.21.15.63:0/302754384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4f71ae8b0 con 0x55e4f6f121b0 2024-05-27T08:01:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.464+0000 7f699a000640 1 -- 172.21.15.63:0/482695555 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f69940e2a90 con 0x7f699c10d350 2024-05-27T08:01:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.464+0000 7f69a0a00640 1 --2- 172.21.15.63:0/302754384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6f121b0 0x55e4f72b4a60 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f6988005ab0 tx=0x7f6988005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811fbd0690e9475f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.464+0000 7f69a1e00640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69840b29d0 0x7f69840b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.465+0000 7f698e600640 1 -- 172.21.15.63:0/302754384 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988035070 con 0x55e4f6f121b0 2024-05-27T08:01:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.465+0000 7f698e600640 1 -- 172.21.15.63:0/302754384 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6988003c80 con 0x55e4f6f121b0 2024-05-27T08:01:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.465+0000 7f698e600640 1 -- 172.21.15.63:0/302754384 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698803bbb0 con 0x55e4f6f121b0 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.466+0000 7f69aadb9740 1 -- 172.21.15.63:0/302754384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6f121b0 msgr2=0x55e4f72b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.466+0000 7f69aadb9740 1 --2- 172.21.15.63:0/302754384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6f121b0 0x55e4f72b4a60 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f6988005ab0 tx=0x7f6988005ed0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.466+0000 7f69a1e00640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69840b29d0 0x7f69840b4ee0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f69940092f0 tx=0x7f6994005b20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.466+0000 7f69aadb9740 1 -- 172.21.15.63:0/302754384 shutdown_connections 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.466+0000 7f69aadb9740 1 --2- 172.21.15.63:0/302754384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.466+0000 7f69aadb9740 1 --2- 172.21.15.63:0/302754384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6ee92f0 0x55e4f72b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.466+0000 7f69aadb9740 1 --2- 172.21.15.63:0/302754384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6f121b0 0x55e4f72b4a60 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.466+0000 7f69aadb9740 1 -- 172.21.15.63:0/302754384 >> 172.21.15.63:0/302754384 conn(0x55e4f6f06300 msgr2=0x55e4f72b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.466+0000 7f69aadb9740 1 -- 172.21.15.63:0/302754384 shutdown_connections 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.466+0000 7f69aadb9740 1 -- 172.21.15.63:0/302754384 wait complete. 2024-05-27T08:01:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.467+0000 7f69aadb9740 1 -- start start 2024-05-27T08:01:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.467+0000 7f69aadb9740 1 Processor -- start 2024-05-27T08:01:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72beec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72bf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69aadb9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e4f6efb790 con 0x55e4f6ee92f0 2024-05-27T08:01:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69aadb9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e4f71bdbc0 con 0x55e4f6ef14d0 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69aadb9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e4f72c1b30 con 0x55e4f6f121b0 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69aadb9740 10 client.0 _ll_register_callbacks cb 0x55e4f719b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69a0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72beec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69a0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72beec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41282/0 (socket says 172.21.15.63:41282) 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69a0a00640 1 -- 172.21.15.63:0/3677713012 learned_addr learned my addr 172.21.15.63:0/3677713012 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69a1400640 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72bf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.468+0000 7f69a0a00640 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.469+0000 7f69a0a00640 1 -- 172.21.15.63:0/3677713012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 msgr2=0x55e4f72cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.469+0000 7f69a0a00640 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.469+0000 7f69a0a00640 1 -- 172.21.15.63:0/3677713012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 msgr2=0x55e4f72bf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.469+0000 7f69a0a00640 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72bf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.469+0000 7f69a0a00640 1 -- 172.21.15.63:0/3677713012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4f6f2a210 con 0x55e4f6ee92f0 2024-05-27T08:01:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.469+0000 7f69a0a00640 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72beec0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f6988004650 tx=0x7f6988000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd49cb637ac1f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.469+0000 7f6982a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.469+0000 7f6982a00640 1 -- 172.21.15.63:0/3677713012 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988053020 con 0x55e4f6ee92f0 2024-05-27T08:01:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.469+0000 7f6982a00640 1 -- 172.21.15.63:0/3677713012 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f6988003a00 con 0x55e4f6ee92f0 2024-05-27T08:01:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.469+0000 7f69aadb9740 1 -- 172.21.15.63:0/3677713012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6988003c80 con 0x55e4f6ee92f0 2024-05-27T08:01:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.470+0000 7f6982a00640 1 -- 172.21.15.63:0/3677713012 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6988035070 con 0x55e4f6ee92f0 2024-05-27T08:01:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.470+0000 7f69aadb9740 10 client.20730 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.470+0000 7f69aadb9740 1 -- 172.21.15.63:0/3677713012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e4f6ff6620 con 0x55e4f6ee92f0 2024-05-27T08:01:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.470+0000 7f6982a00640 1 -- 172.21.15.63:0/3677713012 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6988003c80 con 0x55e4f6ee92f0 2024-05-27T08:01:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.470+0000 7f6982a00640 1 -- 172.21.15.63:0/3677713012 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f69880497c0 con 0x55e4f6ee92f0 2024-05-27T08:01:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.470+0000 7f69aadb9740 10 client.20730 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.470+0000 7f69aadb9740 10 client.20730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.470+0000 7f69aadb9740 20 client.20730 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.471+0000 7f69aadb9740 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4f6ed0d80 0x55e4f72d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.471+0000 7f69aadb9740 4 client.20730 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.471+0000 7f69aadb9740 1 -- 172.21.15.63:0/3677713012 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4f7227ae0 con 0x55e4f6ed0d80 2024-05-27T08:01:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.471+0000 7f69a1400640 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4f6ed0d80 0x55e4f72d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.472+0000 7f69a1400640 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4f6ed0d80 0x55e4f72d0cf0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8ae351127a320e server_cookie=80528647529646e in_seq=0 out_seq=0 2024-05-27T08:01:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.472+0000 7f6982a00640 10 client.20730 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.474+0000 7f6982a00640 1 -- 172.21.15.63:0/3677713012 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e4f7227ae0 con 0x55e4f6ed0d80 2024-05-27T08:01:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.474+0000 7f6982a00640 10 client.20730 handle_command_reply: tid=0 2024-05-27T08:01:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.474+0000 7f69aadb9740 1 client.20730 shutdown 2024-05-27T08:01:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 1 -- 172.21.15.63:0/3677713012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 msgr2=0x55e4f72beec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72beec0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f6988004650 tx=0x7f6988000bc0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 1 -- 172.21.15.63:0/3677713012 shutdown_connections 2024-05-27T08:01:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4f6ed0d80 0x55e4f72d0cf0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 1 --2- 172.21.15.63:0/3677713012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72beec0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f6982a00640 0 client.20730 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 1 -- 172.21.15.63:0/3677713012 >> 172.21.15.63:0/3677713012 conn(0x55e4f6f06300 msgr2=0x55e4f72b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 1 -- 172.21.15.63:0/3677713012 shutdown_connections 2024-05-27T08:01:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 1 -- 172.21.15.63:0/3677713012 wait complete. 2024-05-27T08:01:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.475+0000 7f69aadb9740 20 client.20730 trim_cache size 0 max 16384 2024-05-27T08:01:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.498+0000 7f69aadb9740 1 Processor -- start 2024-05-27T08:01:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.498+0000 7f69aadb9740 1 -- start start 2024-05-27T08:01:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.499+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.499+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72ba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.499+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.499+0000 7f69aadb9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b5860 con 0x55e4f6ee92f0 2024-05-27T08:01:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.499+0000 7f69aadb9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b3130 con 0x55e4f6ef14d0 2024-05-27T08:01:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.499+0000 7f69aadb9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b0fa0 con 0x55e4f6f121b0 2024-05-27T08:01:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72ba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72ba4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:48098/0 (socket says 172.21.15.63:48098) 2024-05-27T08:01:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a1400640 1 -- 172.21.15.63:0/151087461 learned_addr learned my addr 172.21.15.63:0/151087461 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41292/0 (socket says 172.21.15.63:41292) 2024-05-27T08:01:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a0a00640 1 --2- 172.21.15.63:0/151087461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a1400640 1 -- 172.21.15.63:0/151087461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 msgr2=0x55e4f72cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a1400640 1 --2- 172.21.15.63:0/151087461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a1400640 1 -- 172.21.15.63:0/151087461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 msgr2=0x55e4f72b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a1400640 1 --2- 172.21.15.63:0/151087461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.500+0000 7f69a1400640 1 -- 172.21.15.63:0/151087461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4f72b1bd0 con 0x55e4f6ef14d0 2024-05-27T08:01:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.501+0000 7f69a1400640 1 --2- 172.21.15.63:0/151087461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72ba4e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55e4f72b21a0 tx=0x7f699000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1542eb6d63fe8fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.502+0000 7f698dc00640 1 -- 172.21.15.63:0/151087461 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990003be0 con 0x55e4f6ef14d0 2024-05-27T08:01:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.502+0000 7f698dc00640 1 -- 172.21.15.63:0/151087461 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6990003d80 con 0x55e4f6ef14d0 2024-05-27T08:01:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.502+0000 7f698dc00640 1 -- 172.21.15.63:0/151087461 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990027680 con 0x55e4f6ef14d0 2024-05-27T08:01:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.503+0000 7f69aadb9740 1 -- 172.21.15.63:0/151087461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 msgr2=0x55e4f72ba4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.503+0000 7f69aadb9740 1 --2- 172.21.15.63:0/151087461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72ba4e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55e4f72b21a0 tx=0x7f699000b390 comp rx=0 tx=0).stop 2024-05-27T08:01:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.503+0000 7f69aadb9740 1 -- 172.21.15.63:0/151087461 shutdown_connections 2024-05-27T08:01:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.503+0000 7f69aadb9740 1 --2- 172.21.15.63:0/151087461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.503+0000 7f69aadb9740 1 --2- 172.21.15.63:0/151087461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f72ba4e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.503+0000 7f69aadb9740 1 --2- 172.21.15.63:0/151087461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f72b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.503+0000 7f69aadb9740 1 -- 172.21.15.63:0/151087461 >> 172.21.15.63:0/151087461 conn(0x55e4f6f06300 msgr2=0x55e4f72b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.503+0000 7f69aadb9740 1 -- 172.21.15.63:0/151087461 shutdown_connections 2024-05-27T08:01:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.503+0000 7f69aadb9740 1 -- 172.21.15.63:0/151087461 wait complete. 2024-05-27T08:01:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.504+0000 7f69aadb9740 1 -- start start 2024-05-27T08:01:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.504+0000 7f69aadb9740 1 Processor -- start 2024-05-27T08:01:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f7330540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f7330a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69aadb9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69aadb9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b15f0 con 0x55e4f6ee92f0 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69aadb9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b2a60 con 0x55e4f6ef14d0 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69aadb9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b1cc0 con 0x55e4f6f121b0 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69aadb9740 10 client.0 _ll_register_callbacks cb 0x7f699000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69a0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f7330540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69a0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f7330540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41294/0 (socket says 172.21.15.63:41294) 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69a0a00640 1 -- 172.21.15.63:0/1773102806 learned_addr learned my addr 172.21.15.63:0/1773102806 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69a1400640 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f7330a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.505+0000 7f69a0a00640 1 -- 172.21.15.63:0/1773102806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 msgr2=0x55e4f72d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.506+0000 7f69a0a00640 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.506+0000 7f69a0a00640 1 -- 172.21.15.63:0/1773102806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 msgr2=0x55e4f7330a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.506+0000 7f69a0a00640 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f7330a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.506+0000 7f69a0a00640 1 -- 172.21.15.63:0/1773102806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4f72d1ac0 con 0x55e4f6ee92f0 2024-05-27T08:01:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.506+0000 7f69a0a00640 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f7330540 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f6988049c30 tx=0x7f698802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4aa8c885f38ed68 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.506+0000 7f6982a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.506+0000 7f6982a00640 1 -- 172.21.15.63:0/1773102806 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988053070 con 0x55e4f6ee92f0 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.506+0000 7f6982a00640 1 -- 172.21.15.63:0/1773102806 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9483+0+0 (secure 0 0 0) 0x7f6988003ac0 con 0x55e4f6ee92f0 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.506+0000 7f69aadb9740 1 -- 172.21.15.63:0/1773102806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6990003d80 con 0x55e4f6ee92f0 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.507+0000 7f6982a00640 1 -- 172.21.15.63:0/1773102806 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f698806d050 con 0x55e4f6ee92f0 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.507+0000 7f69aadb9740 10 client.20736 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.507+0000 7f69aadb9740 1 -- 172.21.15.63:0/1773102806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e4f7338b60 con 0x55e4f6ee92f0 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.507+0000 7f6982a00640 1 -- 172.21.15.63:0/1773102806 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6990003d80 con 0x55e4f6ee92f0 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.507+0000 7f6982a00640 1 -- 172.21.15.63:0/1773102806 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f698804c5c0 con 0x55e4f6ee92f0 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.507+0000 7f69aadb9740 10 client.20736 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.507+0000 7f69aadb9740 10 client.20736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.507+0000 7f69aadb9740 20 client.20736 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.508+0000 7f69aadb9740 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4f6ea61c0 0x55e4f733b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.508+0000 7f69aadb9740 4 client.20736 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:01:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.508+0000 7f69aadb9740 1 -- 172.21.15.63:0/1773102806 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4f72212c0 con 0x55e4f6ea61c0 2024-05-27T08:01:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.508+0000 7f69a1400640 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4f6ea61c0 0x55e4f733b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.509+0000 7f69a1400640 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4f6ea61c0 0x55e4f733b2f0 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad6bd95b51696607 server_cookie=49c57a1a60922d6e in_seq=0 out_seq=0 2024-05-27T08:01:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.509+0000 7f6982a00640 10 client.20736 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.561+0000 7f6982a00640 1 -- 172.21.15.63:0/1773102806 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e4f72212c0 con 0x55e4f6ea61c0 2024-05-27T08:01:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.561+0000 7f6982a00640 10 client.20736 handle_command_reply: tid=0 2024-05-27T08:01:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.561+0000 7f69aadb9740 1 client.20736 shutdown 2024-05-27T08:01:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f69aadb9740 1 -- 172.21.15.63:0/1773102806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 msgr2=0x55e4f7330540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f69aadb9740 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f7330540 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f6988049c30 tx=0x7f698802ab90 comp rx=0 tx=0).stop 2024-05-27T08:01:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f69aadb9740 1 -- 172.21.15.63:0/1773102806 shutdown_connections 2024-05-27T08:01:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f69aadb9740 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4f6ea61c0 0x55e4f733b2f0 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f69aadb9740 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e4f6f121b0 0x55e4f72d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f6982a00640 0 client.20736 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f69aadb9740 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e4f6ef14d0 0x55e4f7330a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f69aadb9740 1 --2- 172.21.15.63:0/1773102806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e4f6ee92f0 0x55e4f7330540 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f69aadb9740 1 -- 172.21.15.63:0/1773102806 >> 172.21.15.63:0/1773102806 conn(0x55e4f6f06300 msgr2=0x55e4f72c0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f69aadb9740 1 -- 172.21.15.63:0/1773102806 shutdown_connections 2024-05-27T08:01:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.562+0000 7f69aadb9740 1 -- 172.21.15.63:0/1773102806 wait complete. 2024-05-27T08:01:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.563+0000 7f69aadb9740 20 client.20736 trim_cache size 0 max 16384 2024-05-27T08:01:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:01:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:01:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:01:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.563+0000 7f69a2800640 1 -- 172.21.15.63:0/482695555 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69840b29d0 msgr2=0x7f69840b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.563+0000 7f69a2800640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69840b29d0 0x7f69840b4ee0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f69940092f0 tx=0x7f6994005b20 comp rx=0 tx=0).stop 2024-05-27T08:01:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.564+0000 7f69a2800640 1 -- 172.21.15.63:0/482695555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c10d350 msgr2=0x7f699c110c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.564+0000 7f69a2800640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c10d350 0x7f699c110c30 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f699400eaa0 tx=0x7f699400ef60 comp rx=0 tx=0).stop 2024-05-27T08:01:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.564+0000 7f69a2800640 1 -- 172.21.15.63:0/482695555 shutdown_connections 2024-05-27T08:01:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.564+0000 7f69a2800640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69840b29d0 0x7f69840b4ee0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.564+0000 7f69a2800640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c10d350 0x7f699c110c30 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.564+0000 7f69a2800640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c1089f0 0x7f699c1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.564+0000 7f69a2800640 1 --2- 172.21.15.63:0/482695555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f699c102b10 0x7f699c1101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.564+0000 7f69a2800640 1 -- 172.21.15.63:0/482695555 >> 172.21.15.63:0/482695555 conn(0x7f699c0fe210 msgr2=0x7f699c104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.564+0000 7f69a2800640 1 -- 172.21.15.63:0/482695555 shutdown_connections 2024-05-27T08:01:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:06.564+0000 7f69a2800640 1 -- 172.21.15.63:0/482695555 wait complete. 2024-05-27T08:01:07.264 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:01:07.264 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:01:07.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:07 smithi086 ceph-mon[34687]: pgmap v872: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-05-27T08:01:07.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:07 smithi149 ceph-mon[35408]: pgmap v872: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-05-27T08:01:07.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:07 smithi063 ceph-mon[31613]: pgmap v872: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 27 op/s 2024-05-27T08:01:07.780 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:08.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:08 smithi086 ceph-mon[34687]: osdmap e137: 12 total, 12 up, 12 in 2024-05-27T08:01:08.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:08 smithi149 ceph-mon[35408]: osdmap e137: 12 total, 12 up, 12 in 2024-05-27T08:01:08.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:08 smithi063 ceph-mon[31613]: osdmap e137: 12 total, 12 up, 12 in 2024-05-27T08:01:09.612 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:09 smithi086 ceph-mon[34687]: pgmap v874: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:01:09.660 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:09 smithi063 ceph-mon[31613]: pgmap v874: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:01:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:09 smithi149 ceph-mon[35408]: pgmap v874: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:01:10.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.338+0000 7f19bec00700 1 -- 172.21.15.63:0/2420785316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b8108f10 msgr2=0x7f19b8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:10.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.338+0000 7f19bec00700 1 --2- 172.21.15.63:0/2420785316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b8108f10 0x7f19b8109320 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f19a40099c0 tx=0x7f19a4009cd0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.338+0000 7f19bec00700 1 -- 172.21.15.63:0/2420785316 shutdown_connections 2024-05-27T08:01:10.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.338+0000 7f19bec00700 1 --2- 172.21.15.63:0/2420785316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b8109860 0x7f19b81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.338+0000 7f19bec00700 1 --2- 172.21.15.63:0/2420785316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b81045e0 0x7f19b81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.338+0000 7f19bec00700 1 --2- 172.21.15.63:0/2420785316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b8108f10 0x7f19b8109320 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.338+0000 7f19bec00700 1 -- 172.21.15.63:0/2420785316 >> 172.21.15.63:0/2420785316 conn(0x7f19b8100030 msgr2=0x7f19b8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:10.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.338+0000 7f19bec00700 1 -- 172.21.15.63:0/2420785316 shutdown_connections 2024-05-27T08:01:10.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.338+0000 7f19bec00700 1 -- 172.21.15.63:0/2420785316 wait complete. 2024-05-27T08:01:10.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.339+0000 7f19bec00700 1 Processor -- start 2024-05-27T08:01:10.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.340+0000 7f19bec00700 1 -- start start 2024-05-27T08:01:10.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.340+0000 7f19bec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b81045e0 0x7f19b819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.340+0000 7f19bec00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b8108f10 0x7f19b819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.340+0000 7f19bec00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b8109860 0x7f19b81a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.340+0000 7f19bec00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8114390 con 0x7f19b8108f10 2024-05-27T08:01:10.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.340+0000 7f19bec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8114210 con 0x7f19b81045e0 2024-05-27T08:01:10.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.340+0000 7f19bec00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8114510 con 0x7f19b8109860 2024-05-27T08:01:10.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.340+0000 7f19b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b8108f10 0x7f19b819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.341+0000 7f19b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b8108f10 0x7f19b819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41088/0 (socket says 172.21.15.63:41088) 2024-05-27T08:01:10.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.341+0000 7f19b7400700 1 -- 172.21.15.63:0/2469551826 learned_addr learned my addr 172.21.15.63:0/2469551826 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:10.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.341+0000 7f19b7e00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b81045e0 0x7f19b819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.341+0000 7f19b7400700 1 -- 172.21.15.63:0/2469551826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b8109860 msgr2=0x7f19b81a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:10.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.341+0000 7f19b7400700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b8109860 0x7f19b81a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.341+0000 7f19b7400700 1 -- 172.21.15.63:0/2469551826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b81045e0 msgr2=0x7f19b819d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:10.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.341+0000 7f19b7400700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b81045e0 0x7f19b819d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.341+0000 7f19b7400700 1 -- 172.21.15.63:0/2469551826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b81a1e60 con 0x7f19b8108f10 2024-05-27T08:01:10.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.341+0000 7f19b7e00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b81045e0 0x7f19b819d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:10.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.342+0000 7f19b7400700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b8108f10 0x7f19b819d5e0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f19a400b580 tx=0x7f19a400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.342+0000 7f19b4c00700 1 -- 172.21.15.63:0/2469551826 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4027070 con 0x7f19b8108f10 2024-05-27T08:01:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.342+0000 7f19b4c00700 1 -- 172.21.15.63:0/2469551826 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a400fda0 con 0x7f19b8108f10 2024-05-27T08:01:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.342+0000 7f19bec00700 1 -- 172.21.15.63:0/2469551826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b81a20f0 con 0x7f19b8108f10 2024-05-27T08:01:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.343+0000 7f19b4c00700 1 -- 172.21.15.63:0/2469551826 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a40173f0 con 0x7f19b8108f10 2024-05-27T08:01:10.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.343+0000 7f19bec00700 1 -- 172.21.15.63:0/2469551826 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b81a25d0 con 0x7f19b8108f10 2024-05-27T08:01:10.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.348+0000 7f19b4c00700 1 -- 172.21.15.63:0/2469551826 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f19a402a070 con 0x7f19b8108f10 2024-05-27T08:01:10.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.349+0000 7f19cd766b80 1 Processor -- start 2024-05-27T08:01:10.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.349+0000 7f19cd766b80 1 -- start start 2024-05-27T08:01:10.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.350+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c90d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.350+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 0x560d3c9078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.350+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c908240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.350+0000 7f19cd766b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560d3c835ec0 con 0x560d3c90ddb0 2024-05-27T08:01:10.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.350+0000 7f19cd766b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d3c825a60 con 0x560d3c907e30 2024-05-27T08:01:10.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.350+0000 7f19cd766b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560d3c829aa0 con 0x560d3c788ec0 2024-05-27T08:01:10.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.350+0000 7f19b4c00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f199c0b2b40 0x7f199c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.351+0000 7f19bcc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c90d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.351+0000 7f19bcc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c90d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:59976/0 (socket says 172.21.15.63:59976) 2024-05-27T08:01:10.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.351+0000 7f19bcc00700 1 -- 172.21.15.63:0/2123204840 learned_addr learned my addr 172.21.15.63:0/2123204840 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:10.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.351+0000 7f19b7400700 1 --2- 172.21.15.63:0/2123204840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c908240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.351+0000 7f19b7e00700 1 --2- 172.21.15.63:0/2123204840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 0x560d3c9078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.351+0000 7f19b4c00700 1 -- 172.21.15.63:0/2469551826 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9571+0+0 (secure 0 0 0) 0x7f19a40e3a10 con 0x7f19b8108f10 2024-05-27T08:01:10.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.351+0000 7f19bcc00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f199c0b2b40 0x7f199c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.352+0000 7f19b7400700 1 -- 172.21.15.63:0/2123204840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 msgr2=0x560d3c90d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:10.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.352+0000 7f19b7400700 1 --2- 172.21.15.63:0/2123204840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c90d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.352+0000 7f19b7400700 1 -- 172.21.15.63:0/2123204840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 msgr2=0x560d3c9078f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:10.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.352+0000 7f19b7400700 1 --2- 172.21.15.63:0/2123204840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 0x560d3c9078f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.352+0000 7f19b7400700 1 -- 172.21.15.63:0/2123204840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d3c7ee4a0 con 0x560d3c907e30 2024-05-27T08:01:10.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.352+0000 7f19b7400700 1 --2- 172.21.15.63:0/2123204840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c908240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f19a40e0ce0 tx=0x7f19a409f180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84872759f836c180 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:10.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.353+0000 7f19bcc00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f199c0b2b40 0x7f199c0b5050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x560d3c908bb0 tx=0x7f19b0006080 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:10.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.353+0000 7f19ad600700 1 -- 172.21.15.63:0/2123204840 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a40a6090 con 0x560d3c907e30 2024-05-27T08:01:10.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.353+0000 7f19ad600700 1 -- 172.21.15.63:0/2123204840 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a4028260 con 0x560d3c907e30 2024-05-27T08:01:10.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.353+0000 7f19ad600700 1 -- 172.21.15.63:0/2123204840 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a40aecb0 con 0x560d3c907e30 2024-05-27T08:01:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.354+0000 7f19cd766b80 1 -- 172.21.15.63:0/2123204840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 msgr2=0x560d3c908240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.354+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2123204840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c908240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f19a40e0ce0 tx=0x7f19a409f180 comp rx=0 tx=0).stop 2024-05-27T08:01:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.355+0000 7f19cd766b80 1 -- 172.21.15.63:0/2123204840 shutdown_connections 2024-05-27T08:01:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.355+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2123204840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c908240 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.355+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2123204840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c90d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.355+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2123204840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 0x560d3c9078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.355+0000 7f19cd766b80 1 -- 172.21.15.63:0/2123204840 >> 172.21.15.63:0/2123204840 conn(0x560d3c7e9040 msgr2=0x560d3c902950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.355+0000 7f19cd766b80 1 -- 172.21.15.63:0/2123204840 shutdown_connections 2024-05-27T08:01:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.355+0000 7f19cd766b80 1 -- 172.21.15.63:0/2123204840 wait complete. 2024-05-27T08:01:10.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.356+0000 7f19cd766b80 1 -- start start 2024-05-27T08:01:10.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.357+0000 7f19cd766b80 1 Processor -- start 2024-05-27T08:01:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.357+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c911530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.357+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c907e30 0x560d3c911a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.357+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c90ddb0 0x560d3c911fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.357+0000 7f19cd766b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560d3c72c020 con 0x560d3c907e30 2024-05-27T08:01:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.357+0000 7f19cd766b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d3c7c67e0 con 0x560d3c90ddb0 2024-05-27T08:01:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.357+0000 7f19cd766b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560d3c914640 con 0x560d3c788ec0 2024-05-27T08:01:10.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.358+0000 7f19cd766b80 10 client.0 _ll_register_callbacks cb 0x560d3c7cfcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.358+0000 7f19b7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c911530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.358+0000 7f19b7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c911530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:59990/0 (socket says 172.21.15.63:59990) 2024-05-27T08:01:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.358+0000 7f19b7e00700 1 -- 172.21.15.63:0/2150516522 learned_addr learned my addr 172.21.15.63:0/2150516522 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:10.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.359+0000 7f19bcc00700 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c90ddb0 0x560d3c911fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.359+0000 7f19b7e00700 1 -- 172.21.15.63:0/2150516522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c90ddb0 msgr2=0x560d3c911fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:10.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.359+0000 7f19b7e00700 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c90ddb0 0x560d3c911fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.359+0000 7f19b7e00700 1 -- 172.21.15.63:0/2150516522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c907e30 msgr2=0x560d3c911a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:10.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.359+0000 7f19b7e00700 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c907e30 0x560d3c911a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.359+0000 7f19b7e00700 1 -- 172.21.15.63:0/2150516522 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d3c775950 con 0x560d3c788ec0 2024-05-27T08:01:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.360+0000 7f19b7e00700 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c911530 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f19a80126d0 tx=0x7f19a80129e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b72f1afa0e905d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.360+0000 7f19a2000700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:01:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.361+0000 7f19a2000700 1 -- 172.21.15.63:0/2150516522 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a801f060 con 0x560d3c788ec0 2024-05-27T08:01:10.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.361+0000 7f19cd766b80 1 -- 172.21.15.63:0/2150516522 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a4028260 con 0x560d3c788ec0 2024-05-27T08:01:10.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.361+0000 7f19a2000700 1 -- 172.21.15.63:0/2150516522 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9571+0+0 (secure 0 0 0) 0x7f19a80107e0 con 0x560d3c788ec0 2024-05-27T08:01:10.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.361+0000 7f19a2000700 1 -- 172.21.15.63:0/2150516522 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f19a80109e0 con 0x560d3c788ec0 2024-05-27T08:01:10.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.361+0000 7f19cd766b80 10 client.29737 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:10.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.361+0000 7f19cd766b80 1 -- 172.21.15.63:0/2150516522 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560d3c7f60e0 con 0x560d3c788ec0 2024-05-27T08:01:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.362+0000 7f19a2000700 1 -- 172.21.15.63:0/2150516522 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a4028260 con 0x560d3c788ec0 2024-05-27T08:01:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.362+0000 7f19a2000700 1 -- 172.21.15.63:0/2150516522 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f19a8019030 con 0x560d3c788ec0 2024-05-27T08:01:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.362+0000 7f19cd766b80 10 client.29737 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.362+0000 7f19cd766b80 10 client.29737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.363+0000 7f19cd766b80 20 client.29737 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.363+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d3c929c30 0x560d3c92c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.363+0000 7f19cd766b80 4 client.29737 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:10.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.363+0000 7f19cd766b80 1 -- 172.21.15.63:0/2150516522 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d3c837e10 con 0x560d3c929c30 2024-05-27T08:01:10.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.364+0000 7f19b7400700 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d3c929c30 0x560d3c92c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.365+0000 7f19a2000700 10 client.29737 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:10.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.365+0000 7f19b7400700 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d3c929c30 0x560d3c92c0d0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f469a9f8b89a885 server_cookie=688d1fadf002e5c9 in_seq=0 out_seq=0 2024-05-27T08:01:10.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19a2000700 1 -- 172.21.15.63:0/2150516522 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560d3c837e10 con 0x560d3c929c30 2024-05-27T08:01:10.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19a2000700 10 client.29737 handle_command_reply: tid=0 2024-05-27T08:01:10.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19cd766b80 1 client.29737 shutdown 2024-05-27T08:01:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19cd766b80 1 -- 172.21.15.63:0/2150516522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 msgr2=0x560d3c911530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c911530 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f19a80126d0 tx=0x7f19a80129e0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19cd766b80 1 -- 172.21.15.63:0/2150516522 shutdown_connections 2024-05-27T08:01:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d3c929c30 0x560d3c92c0d0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c90ddb0 0x560d3c911fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c788ec0 0x560d3c911530 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2150516522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c907e30 0x560d3c911a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19a2000700 0 client.29737 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:10.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19cd766b80 1 -- 172.21.15.63:0/2150516522 >> 172.21.15.63:0/2150516522 conn(0x560d3c7e9040 msgr2=0x560d3c902600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:10.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.367+0000 7f19cd766b80 1 -- 172.21.15.63:0/2150516522 shutdown_connections 2024-05-27T08:01:10.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.368+0000 7f19cd766b80 1 -- 172.21.15.63:0/2150516522 wait complete. 2024-05-27T08:01:10.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.368+0000 7f19cd766b80 20 client.29737 trim_cache size 0 max 16384 2024-05-27T08:01:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.394+0000 7f19cd766b80 1 Processor -- start 2024-05-27T08:01:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.394+0000 7f19cd766b80 1 -- start start 2024-05-27T08:01:10.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.394+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c788ec0 0x560d3c9078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.394+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c925e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.394+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c90ddb0 0x560d3c90b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.394+0000 7f19cd766b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81b0be0 con 0x560d3c788ec0 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.394+0000 7f19cd766b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81ae470 con 0x560d3c907e30 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.394+0000 7f19cd766b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81ac2e0 con 0x560d3c90ddb0 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.395+0000 7f19b7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c788ec0 0x560d3c9078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.395+0000 7f19b7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c788ec0 0x560d3c9078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41110/0 (socket says 172.21.15.63:41110) 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.395+0000 7f19b7e00700 1 -- 172.21.15.63:0/2581951828 learned_addr learned my addr 172.21.15.63:0/2581951828 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.395+0000 7f19b7400700 1 --2- 172.21.15.63:0/2581951828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c925e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.395+0000 7f19bcc00700 1 --2- 172.21.15.63:0/2581951828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c90ddb0 0x560d3c90b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.395+0000 7f19b7e00700 1 -- 172.21.15.63:0/2581951828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c90ddb0 msgr2=0x560d3c90b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.396+0000 7f19b7e00700 1 --2- 172.21.15.63:0/2581951828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c90ddb0 0x560d3c90b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.396+0000 7f19b7e00700 1 -- 172.21.15.63:0/2581951828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 msgr2=0x560d3c925e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.396+0000 7f19b7e00700 1 --2- 172.21.15.63:0/2581951828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c925e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.396+0000 7f19b7e00700 1 -- 172.21.15.63:0/2581951828 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d3c90cd70 con 0x560d3c788ec0 2024-05-27T08:01:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.396+0000 7f19b7400700 1 --2- 172.21.15.63:0/2581951828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c925e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.396+0000 7f19b7e00700 1 --2- 172.21.15.63:0/2581951828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c788ec0 0x560d3c9078f0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f19a801ad80 tx=0x7f19a80107f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7059de70f6cfdb27 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.396+0000 7f19acc00700 1 -- 172.21.15.63:0/2581951828 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8010c00 con 0x560d3c788ec0 2024-05-27T08:01:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.397+0000 7f19acc00700 1 -- 172.21.15.63:0/2581951828 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a8010da0 con 0x560d3c788ec0 2024-05-27T08:01:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.397+0000 7f19acc00700 1 -- 172.21.15.63:0/2581951828 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a801a950 con 0x560d3c788ec0 2024-05-27T08:01:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.398+0000 7f19cd766b80 1 -- 172.21.15.63:0/2581951828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c788ec0 msgr2=0x560d3c9078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.398+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2581951828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c788ec0 0x560d3c9078f0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f19a801ad80 tx=0x7f19a80107f0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.398+0000 7f19cd766b80 1 -- 172.21.15.63:0/2581951828 shutdown_connections 2024-05-27T08:01:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.398+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2581951828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c90ddb0 0x560d3c90b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.398+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2581951828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c788ec0 0x560d3c9078f0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.398+0000 7f19cd766b80 1 --2- 172.21.15.63:0/2581951828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c925e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.398+0000 7f19cd766b80 1 -- 172.21.15.63:0/2581951828 >> 172.21.15.63:0/2581951828 conn(0x560d3c7e9040 msgr2=0x560d3c911d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.398+0000 7f19cd766b80 1 -- 172.21.15.63:0/2581951828 shutdown_connections 2024-05-27T08:01:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.398+0000 7f19cd766b80 1 -- 172.21.15.63:0/2581951828 wait complete. 2024-05-27T08:01:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.399+0000 7f19cd766b80 1 -- start start 2024-05-27T08:01:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.399+0000 7f19cd766b80 1 Processor -- start 2024-05-27T08:01:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c9227e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 0x560d3c91f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19cd766b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c91f560 0x560d3c91f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19cd766b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81ac930 con 0x560d3c90ddb0 2024-05-27T08:01:10.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19cd766b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81adda0 con 0x560d3c907e30 2024-05-27T08:01:10.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19cd766b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81ad000 con 0x560d3c91f560 2024-05-27T08:01:10.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19cd766b80 10 client.0 _ll_register_callbacks cb 0x560d3c7cfcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:10.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 0x560d3c91f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19b7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 0x560d3c91f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41116/0 (socket says 172.21.15.63:41116) 2024-05-27T08:01:10.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19b7400700 1 -- 172.21.15.63:0/163464288 learned_addr learned my addr 172.21.15.63:0/163464288 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:10.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19b7e00700 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c9227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19bcc00700 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c91f560 0x560d3c91f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.400+0000 7f19b7400700 1 -- 172.21.15.63:0/163464288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c91f560 msgr2=0x560d3c91f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19b7400700 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c91f560 0x560d3c91f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19b7400700 1 -- 172.21.15.63:0/163464288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 msgr2=0x560d3c9227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19b7400700 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c9227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19b7400700 1 -- 172.21.15.63:0/163464288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d3c994880 con 0x560d3c90ddb0 2024-05-27T08:01:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19bcc00700 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c91f560 0x560d3c91f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:01:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19b7e00700 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c9227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:10.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19b7400700 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 0x560d3c91f020 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f19a40aa890 tx=0x7f19a40a68f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fb05c99688844b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:10.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19a2000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:10.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19a2000700 1 -- 172.21.15.63:0/163464288 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a40c4060 con 0x560d3c90ddb0 2024-05-27T08:01:10.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19a2000700 1 -- 172.21.15.63:0/163464288 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9571+0+0 (secure 0 0 0) 0x7f19a4022470 con 0x560d3c90ddb0 2024-05-27T08:01:10.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.401+0000 7f19cd766b80 1 -- 172.21.15.63:0/163464288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19a8010da0 con 0x560d3c90ddb0 2024-05-27T08:01:10.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.402+0000 7f19a2000700 1 -- 172.21.15.63:0/163464288 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f19a40b36d0 con 0x560d3c90ddb0 2024-05-27T08:01:10.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.402+0000 7f19cd766b80 10 client.20751 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:10.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.402+0000 7f19cd766b80 1 -- 172.21.15.63:0/163464288 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560d3c995b20 con 0x560d3c90ddb0 2024-05-27T08:01:10.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.402+0000 7f19a2000700 1 -- 172.21.15.63:0/163464288 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f19a8010da0 con 0x560d3c90ddb0 2024-05-27T08:01:10.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.402+0000 7f19a2000700 1 -- 172.21.15.63:0/163464288 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f19a40e0f60 con 0x560d3c90ddb0 2024-05-27T08:01:10.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.403+0000 7f19cd766b80 10 client.20751 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:10.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.403+0000 7f19cd766b80 10 client.20751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:10.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.403+0000 7f19cd766b80 20 client.20751 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:10.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.403+0000 7f19cd766b80 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d3c9960a0 0x560d3c9984c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:10.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.403+0000 7f19cd766b80 4 client.20751 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:01:10.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.403+0000 7f19cd766b80 1 -- 172.21.15.63:0/163464288 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560d3c8315f0 con 0x560d3c9960a0 2024-05-27T08:01:10.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.403+0000 7f19b7e00700 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d3c9960a0 0x560d3c9984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:10.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.405+0000 7f19b7e00700 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d3c9960a0 0x560d3c9984c0 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871848ed7a58f345 server_cookie=63cd57de6d53aec9 in_seq=0 out_seq=0 2024-05-27T08:01:10.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.405+0000 7f19a2000700 10 client.20751 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:10.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.407+0000 7f19a2000700 1 -- 172.21.15.63:0/163464288 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560d3c8315f0 con 0x560d3c9960a0 2024-05-27T08:01:10.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.407+0000 7f19a2000700 10 client.20751 handle_command_reply: tid=0 2024-05-27T08:01:10.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.407+0000 7f19cd766b80 1 client.20751 shutdown 2024-05-27T08:01:10.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19cd766b80 1 -- 172.21.15.63:0/163464288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 msgr2=0x560d3c91f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19cd766b80 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 0x560d3c91f020 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f19a40aa890 tx=0x7f19a40a68f0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19cd766b80 1 -- 172.21.15.63:0/163464288 shutdown_connections 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19cd766b80 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d3c9960a0 0x560d3c9984c0 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19cd766b80 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d3c91f560 0x560d3c91f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19cd766b80 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d3c907e30 0x560d3c9227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19cd766b80 1 --2- 172.21.15.63:0/163464288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d3c90ddb0 0x560d3c91f020 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19cd766b80 1 -- 172.21.15.63:0/163464288 >> 172.21.15.63:0/163464288 conn(0x560d3c7e9040 msgr2=0x560d3c911d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19a2000700 0 client.20751 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19cd766b80 1 -- 172.21.15.63:0/163464288 shutdown_connections 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.408+0000 7f19cd766b80 1 -- 172.21.15.63:0/163464288 wait complete. 2024-05-27T08:01:10.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.409+0000 7f19cd766b80 20 client.20751 trim_cache size 0 max 16384 2024-05-27T08:01:10.411 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:01:10.411 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:01:10.411 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "46482219-a7e7-4b53-b46f-66f3f179ade4", 2024-05-27T08:01:10.411 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:01:10.411 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:01:10.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.409+0000 7f19bec00700 1 -- 172.21.15.63:0/2469551826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f199c0b2b40 msgr2=0x7f199c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:10.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.409+0000 7f19bec00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f199c0b2b40 0x7f199c0b5050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x560d3c908bb0 tx=0x7f19b0006080 comp rx=0 tx=0).stop 2024-05-27T08:01:10.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.409+0000 7f19bec00700 1 -- 172.21.15.63:0/2469551826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b8108f10 msgr2=0x7f19b819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:10.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.409+0000 7f19bec00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b8108f10 0x7f19b819d5e0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f19a400b580 tx=0x7f19a400f460 comp rx=0 tx=0).stop 2024-05-27T08:01:10.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.410+0000 7f19bec00700 1 -- 172.21.15.63:0/2469551826 shutdown_connections 2024-05-27T08:01:10.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.410+0000 7f19bec00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f199c0b2b40 0x7f199c0b5050 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.410+0000 7f19bec00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b8109860 0x7f19b81a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.410+0000 7f19bec00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19b81045e0 0x7f19b819d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.410+0000 7f19bec00700 1 --2- 172.21.15.63:0/2469551826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19b8108f10 0x7f19b819d5e0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:10.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.410+0000 7f19bec00700 1 -- 172.21.15.63:0/2469551826 >> 172.21.15.63:0/2469551826 conn(0x7f19b8100030 msgr2=0x7f19b810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:10.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.410+0000 7f19bec00700 1 -- 172.21.15.63:0/2469551826 shutdown_connections 2024-05-27T08:01:10.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:10.410+0000 7f19bec00700 1 -- 172.21.15.63:0/2469551826 wait complete. 2024-05-27T08:01:11.382 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:01:11.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: pgmap v875: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:01:11.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:01:11.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:01:11.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:01:11.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:01:11.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:01:11.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:01:11.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:01:11.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:01:11.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:01:11.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:11 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: pgmap v875: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:01:11.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:01:11.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:01:11.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:11 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: pgmap v875: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:01:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:01:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:01:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:01:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:01:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:01:11.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:01:11.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:01:11.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:01:11.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:01:11.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:11 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:01:11.877 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:12.545 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:12 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.545 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:12 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.545 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:12 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.545 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:12 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:12 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:12 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:12 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:12 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:12 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:12 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:12 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:12.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:12 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:13.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:13 smithi086 ceph-mon[34687]: pgmap v876: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.7 MiB/s wr, 51 op/s 2024-05-27T08:01:13.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:13 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:01:13.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:13 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:01:13.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:13 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:13.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:13 smithi149 ceph-mon[35408]: pgmap v876: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.7 MiB/s wr, 51 op/s 2024-05-27T08:01:13.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:13 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:01:13.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:13 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:01:13.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:13 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:13.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:13 smithi063 ceph-mon[31613]: pgmap v876: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.7 MiB/s wr, 51 op/s 2024-05-27T08:01:13.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:13 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:01:13.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:13 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:01:13.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:13 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:01:14.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.384+0000 7f82a4a00700 1 -- 172.21.15.63:0/424874638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a0107de0 msgr2=0x7f82a01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:14.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.384+0000 7f82a4a00700 1 --2- 172.21.15.63:0/424874638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a0107de0 0x7f82a01081f0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f82880099c0 tx=0x7f8288009cd0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.384+0000 7f82a4a00700 1 -- 172.21.15.63:0/424874638 shutdown_connections 2024-05-27T08:01:14.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.384+0000 7f82a4a00700 1 --2- 172.21.15.63:0/424874638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a0108730 0x7f82a0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.384+0000 7f82a4a00700 1 --2- 172.21.15.63:0/424874638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82a01034b0 0x7f82a01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.384+0000 7f82a4a00700 1 --2- 172.21.15.63:0/424874638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a0107de0 0x7f82a01081f0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.384+0000 7f82a4a00700 1 -- 172.21.15.63:0/424874638 >> 172.21.15.63:0/424874638 conn(0x7f82a00fedb0 msgr2=0x7f82a01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.385+0000 7f82a4a00700 1 -- 172.21.15.63:0/424874638 shutdown_connections 2024-05-27T08:01:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.385+0000 7f82a4a00700 1 -- 172.21.15.63:0/424874638 wait complete. 2024-05-27T08:01:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.386+0000 7f82a4a00700 1 Processor -- start 2024-05-27T08:01:14.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.386+0000 7f82a4a00700 1 -- start start 2024-05-27T08:01:14.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.387+0000 7f82a4a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a01034b0 0x7f82a019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.387+0000 7f82a4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a0107de0 0x7f82a019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.387+0000 7f82a4a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82a0108730 0x7f82a019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.387+0000 7f82a4a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82a01131f0 con 0x7f82a0107de0 2024-05-27T08:01:14.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.387+0000 7f82a4a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82a0113070 con 0x7f82a01034b0 2024-05-27T08:01:14.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.387+0000 7f82a4a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f82a0113370 con 0x7f82a0108730 2024-05-27T08:01:14.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.387+0000 7f829d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a0107de0 0x7f82a019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.387+0000 7f829d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a0107de0 0x7f82a019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41136/0 (socket says 172.21.15.63:41136) 2024-05-27T08:01:14.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.387+0000 7f829d400700 1 -- 172.21.15.63:0/4210059009 learned_addr learned my addr 172.21.15.63:0/4210059009 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:14.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.388+0000 7f829de00700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a01034b0 0x7f82a019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.388+0000 7f829e800700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82a0108730 0x7f82a019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.388+0000 7f829d400700 1 -- 172.21.15.63:0/4210059009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82a0108730 msgr2=0x7f82a019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:14.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.388+0000 7f829d400700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82a0108730 0x7f82a019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.388+0000 7f829d400700 1 -- 172.21.15.63:0/4210059009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a01034b0 msgr2=0x7f82a019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:14.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.388+0000 7f829d400700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a01034b0 0x7f82a019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.388+0000 7f829d400700 1 -- 172.21.15.63:0/4210059009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a019fcc0 con 0x7f82a0107de0 2024-05-27T08:01:14.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.389+0000 7f829d400700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a0107de0 0x7f82a019b390 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f828800b580 tx=0x7f8288004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:14.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.389+0000 7f8296a00700 1 -- 172.21.15.63:0/4210059009 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828800f8a0 con 0x7f82a0107de0 2024-05-27T08:01:14.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.389+0000 7f8296a00700 1 -- 172.21.15.63:0/4210059009 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8288004830 con 0x7f82a0107de0 2024-05-27T08:01:14.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.389+0000 7f8296a00700 1 -- 172.21.15.63:0/4210059009 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288005350 con 0x7f82a0107de0 2024-05-27T08:01:14.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.389+0000 7f82a4a00700 1 -- 172.21.15.63:0/4210059009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a019ff50 con 0x7f82a0107de0 2024-05-27T08:01:14.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.389+0000 7f82a4a00700 1 -- 172.21.15.63:0/4210059009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a01a03d0 con 0x7f82a0107de0 2024-05-27T08:01:14.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.394+0000 7f8296a00700 1 -- 172.21.15.63:0/4210059009 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8288029060 con 0x7f82a0107de0 2024-05-27T08:01:14.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.395+0000 7f82b3557b80 1 Processor -- start 2024-05-27T08:01:14.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.396+0000 7f82b3557b80 1 -- start start 2024-05-27T08:01:14.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.396+0000 7f8296a00700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82840b29f0 0x7f82840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.396+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063680fe40 0x5606369947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.396+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560636994d30 0x56063698e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.396+0000 7f8296a00700 1 -- 172.21.15.63:0/4210059009 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9571+0+0 (secure 0 0 0) 0x7f82880e3860 con 0x7f82a0107de0 2024-05-27T08:01:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.396+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x56063698f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.396+0000 7f82b3557b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5606368bce40 con 0x56063698edb0 2024-05-27T08:01:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.396+0000 7f82b3557b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5606368ac9e0 con 0x560636994d30 2024-05-27T08:01:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.396+0000 7f82b3557b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5606368b0a20 con 0x56063680fe40 2024-05-27T08:01:14.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.397+0000 7f829e800700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82840b29f0 0x7f82840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.397+0000 7f829e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x56063698f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.397+0000 7f829e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x56063698f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41146/0 (socket says 172.21.15.63:41146) 2024-05-27T08:01:14.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.397+0000 7f829e800700 1 -- 172.21.15.63:0/1306389508 learned_addr learned my addr 172.21.15.63:0/1306389508 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:14.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.397+0000 7f829de00700 1 --2- 172.21.15.63:0/1306389508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063680fe40 0x5606369947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.397+0000 7f829d400700 1 --2- 172.21.15.63:0/1306389508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560636994d30 0x56063698e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.398+0000 7f829e800700 1 -- 172.21.15.63:0/1306389508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063680fe40 msgr2=0x5606369947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:14.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.398+0000 7f829e800700 1 --2- 172.21.15.63:0/1306389508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063680fe40 0x5606369947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.398+0000 7f829e800700 1 -- 172.21.15.63:0/1306389508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560636994d30 msgr2=0x56063698e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:14.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.398+0000 7f829e800700 1 --2- 172.21.15.63:0/1306389508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560636994d30 0x56063698e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.398+0000 7f829e800700 1 -- 172.21.15.63:0/1306389508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560636875420 con 0x56063698edb0 2024-05-27T08:01:14.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.398+0000 7f829e800700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82840b29f0 0x7f82840b4f00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8298009710 tx=0x7f8298009c70 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:14.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.398+0000 7f829e800700 1 --2- 172.21.15.63:0/1306389508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x56063698f1c0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f8298018ac0 tx=0x7f829801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25580d895a61ae34 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.399+0000 7f828f400700 1 -- 172.21.15.63:0/1306389508 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298004bb0 con 0x56063698edb0 2024-05-27T08:01:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.399+0000 7f828f400700 1 -- 172.21.15.63:0/1306389508 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f829801e070 con 0x56063698edb0 2024-05-27T08:01:14.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.399+0000 7f828f400700 1 -- 172.21.15.63:0/1306389508 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298022460 con 0x56063698edb0 2024-05-27T08:01:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.400+0000 7f82b3557b80 1 -- 172.21.15.63:0/1306389508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 msgr2=0x56063698f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.400+0000 7f82b3557b80 1 --2- 172.21.15.63:0/1306389508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x56063698f1c0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f8298018ac0 tx=0x7f829801b000 comp rx=0 tx=0).stop 2024-05-27T08:01:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.400+0000 7f82b3557b80 1 -- 172.21.15.63:0/1306389508 shutdown_connections 2024-05-27T08:01:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.400+0000 7f82b3557b80 1 --2- 172.21.15.63:0/1306389508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x56063698f1c0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.400+0000 7f82b3557b80 1 --2- 172.21.15.63:0/1306389508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063680fe40 0x5606369947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.400+0000 7f82b3557b80 1 --2- 172.21.15.63:0/1306389508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560636994d30 0x56063698e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.400+0000 7f82b3557b80 1 -- 172.21.15.63:0/1306389508 >> 172.21.15.63:0/1306389508 conn(0x56063686ffc0 msgr2=0x5606369898d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.400+0000 7f82b3557b80 1 -- 172.21.15.63:0/1306389508 shutdown_connections 2024-05-27T08:01:14.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.400+0000 7f82b3557b80 1 -- 172.21.15.63:0/1306389508 wait complete. 2024-05-27T08:01:14.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.401+0000 7f82b3557b80 1 -- start start 2024-05-27T08:01:14.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.402+0000 7f82b3557b80 1 Processor -- start 2024-05-27T08:01:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.402+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063680fe40 0x560636998560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.402+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063698edb0 0x560636998aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.402+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560636994d30 0x560636998fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.402+0000 7f82b3557b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5606367b2fa0 con 0x56063680fe40 2024-05-27T08:01:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.402+0000 7f82b3557b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56063684d760 con 0x560636994d30 2024-05-27T08:01:14.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.402+0000 7f82b3557b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56063699b7a0 con 0x56063698edb0 2024-05-27T08:01:14.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.403+0000 7f82b3557b80 10 client.0 _ll_register_callbacks cb 0x560636856c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:14.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.403+0000 7f829de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063680fe40 0x560636998560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.403+0000 7f829de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063680fe40 0x560636998560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41160/0 (socket says 172.21.15.63:41160) 2024-05-27T08:01:14.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.403+0000 7f829de00700 1 -- 172.21.15.63:0/1654574108 learned_addr learned my addr 172.21.15.63:0/1654574108 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:14.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.403+0000 7f829e800700 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560636994d30 0x560636998fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.403+0000 7f829d400700 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063698edb0 0x560636998aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.403+0000 7f829de00700 1 -- 172.21.15.63:0/1654574108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063698edb0 msgr2=0x560636998aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:14.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.404+0000 7f829de00700 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063698edb0 0x560636998aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.404+0000 7f829de00700 1 -- 172.21.15.63:0/1654574108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560636994d30 msgr2=0x560636998fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:14.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.404+0000 7f829de00700 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560636994d30 0x560636998fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.404+0000 7f829de00700 1 -- 172.21.15.63:0/1654574108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606367fc8d0 con 0x56063680fe40 2024-05-27T08:01:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.404+0000 7f829d400700 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063698edb0 0x560636998aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:01:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.404+0000 7f829de00700 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063680fe40 0x560636998560 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f829000bb70 tx=0x7f829000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14fe9d365c438d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:14.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.404+0000 7f8283e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:14.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.405+0000 7f8283e00700 1 -- 172.21.15.63:0/1654574108 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8290003ca0 con 0x56063680fe40 2024-05-27T08:01:14.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.405+0000 7f8283e00700 1 -- 172.21.15.63:0/1654574108 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9571+0+0 (secure 0 0 0) 0x7f8290004830 con 0x56063680fe40 2024-05-27T08:01:14.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.405+0000 7f82b3557b80 1 -- 172.21.15.63:0/1654574108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f829801e070 con 0x56063680fe40 2024-05-27T08:01:14.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.405+0000 7f8283e00700 1 -- 172.21.15.63:0/1654574108 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8290024050 con 0x56063680fe40 2024-05-27T08:01:14.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.405+0000 7f82b3557b80 10 client.20763 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:14.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.405+0000 7f82b3557b80 1 -- 172.21.15.63:0/1654574108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56063687d060 con 0x56063680fe40 2024-05-27T08:01:14.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.405+0000 7f8283e00700 1 -- 172.21.15.63:0/1654574108 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f829801e070 con 0x56063680fe40 2024-05-27T08:01:14.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.406+0000 7f8283e00700 1 -- 172.21.15.63:0/1654574108 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8290011440 con 0x56063680fe40 2024-05-27T08:01:14.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.406+0000 7f82b3557b80 10 client.20763 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:14.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.406+0000 7f82b3557b80 10 client.20763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:14.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.406+0000 7f82b3557b80 20 client.20763 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:14.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.406+0000 7f82b3557b80 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606369a8750 0x5606369aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.406+0000 7f82b3557b80 4 client.20763 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:14.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.406+0000 7f82b3557b80 1 -- 172.21.15.63:0/1654574108 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606368bed90 con 0x5606369a8750 2024-05-27T08:01:14.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.407+0000 7f829d400700 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606369a8750 0x5606369aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.408+0000 7f829d400700 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606369a8750 0x5606369aabd0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b880cdc03e87c597 server_cookie=1cff0b8d64a31ecc in_seq=0 out_seq=0 2024-05-27T08:01:14.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.408+0000 7f8283e00700 10 client.20763 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:14.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.409+0000 7f8283e00700 1 -- 172.21.15.63:0/1654574108 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5606368bed90 con 0x5606369a8750 2024-05-27T08:01:14.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.409+0000 7f8283e00700 10 client.20763 handle_command_reply: tid=0 2024-05-27T08:01:14.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 client.20763 shutdown 2024-05-27T08:01:14.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 -- 172.21.15.63:0/1654574108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063680fe40 msgr2=0x560636998560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:14.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063680fe40 0x560636998560 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f829000bb70 tx=0x7f829000be80 comp rx=0 tx=0).stop 2024-05-27T08:01:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 -- 172.21.15.63:0/1654574108 shutdown_connections 2024-05-27T08:01:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5606369a8750 0x5606369aabd0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560636994d30 0x560636998fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063680fe40 0x560636998560 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 --2- 172.21.15.63:0/1654574108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063698edb0 0x560636998aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 -- 172.21.15.63:0/1654574108 >> 172.21.15.63:0/1654574108 conn(0x56063686ffc0 msgr2=0x560636989580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f8283e00700 0 client.20763 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 -- 172.21.15.63:0/1654574108 shutdown_connections 2024-05-27T08:01:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.410+0000 7f82b3557b80 1 -- 172.21.15.63:0/1654574108 wait complete. 2024-05-27T08:01:14.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.411+0000 7f82b3557b80 20 client.20763 trim_cache size 0 max 16384 2024-05-27T08:01:14.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.435+0000 7f82b3557b80 1 Processor -- start 2024-05-27T08:01:14.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.435+0000 7f82b3557b80 1 -- start start 2024-05-27T08:01:14.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063680fe40 0x56063698e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063698edb0 0x56063699a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560636994d30 0x560636992850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f82b3557b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82a01b4f90 con 0x560636994d30 2024-05-27T08:01:14.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f82b3557b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82a01b2820 con 0x56063698edb0 2024-05-27T08:01:14.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f82b3557b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f82a01b06b0 con 0x56063680fe40 2024-05-27T08:01:14.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f829e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560636994d30 0x560636992850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f829e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560636994d30 0x560636992850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41170/0 (socket says 172.21.15.63:41170) 2024-05-27T08:01:14.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f829e800700 1 -- 172.21.15.63:0/3038359690 learned_addr learned my addr 172.21.15.63:0/3038359690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:14.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f829de00700 1 --2- 172.21.15.63:0/3038359690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063680fe40 0x56063698e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.436+0000 7f829d400700 1 --2- 172.21.15.63:0/3038359690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063698edb0 0x56063699a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.437+0000 7f829e800700 1 -- 172.21.15.63:0/3038359690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063680fe40 msgr2=0x56063698e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:14.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.437+0000 7f829e800700 1 --2- 172.21.15.63:0/3038359690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063680fe40 0x56063698e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.437+0000 7f829e800700 1 -- 172.21.15.63:0/3038359690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063698edb0 msgr2=0x56063699a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:14.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.437+0000 7f829e800700 1 --2- 172.21.15.63:0/3038359690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063698edb0 0x56063699a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.437+0000 7f829e800700 1 -- 172.21.15.63:0/3038359690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560636997830 con 0x560636994d30 2024-05-27T08:01:14.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.437+0000 7f829e800700 1 --2- 172.21.15.63:0/3038359690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560636994d30 0x560636992850 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f82980186f0 tx=0x7f829801af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568046c3a54bde56 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:14.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.439+0000 7f828ea00700 1 -- 172.21.15.63:0/3038359690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298032070 con 0x560636994d30 2024-05-27T08:01:14.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.439+0000 7f828ea00700 1 -- 172.21.15.63:0/3038359690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f829801e040 con 0x560636994d30 2024-05-27T08:01:14.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.439+0000 7f828ea00700 1 -- 172.21.15.63:0/3038359690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829801ac20 con 0x560636994d30 2024-05-27T08:01:14.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.440+0000 7f82b3557b80 1 -- 172.21.15.63:0/3038359690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560636994d30 msgr2=0x560636992850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:14.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.440+0000 7f82b3557b80 1 --2- 172.21.15.63:0/3038359690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560636994d30 0x560636992850 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f82980186f0 tx=0x7f829801af80 comp rx=0 tx=0).stop 2024-05-27T08:01:14.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.440+0000 7f82b3557b80 1 -- 172.21.15.63:0/3038359690 shutdown_connections 2024-05-27T08:01:14.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.440+0000 7f82b3557b80 1 --2- 172.21.15.63:0/3038359690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560636994d30 0x560636992850 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.440+0000 7f82b3557b80 1 --2- 172.21.15.63:0/3038359690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56063680fe40 0x56063698e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.440+0000 7f82b3557b80 1 --2- 172.21.15.63:0/3038359690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063698edb0 0x56063699a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.440+0000 7f82b3557b80 1 -- 172.21.15.63:0/3038359690 >> 172.21.15.63:0/3038359690 conn(0x56063686ffc0 msgr2=0x560636992140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:14.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.440+0000 7f82b3557b80 1 -- 172.21.15.63:0/3038359690 shutdown_connections 2024-05-27T08:01:14.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.441+0000 7f82b3557b80 1 -- 172.21.15.63:0/3038359690 wait complete. 2024-05-27T08:01:14.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.441+0000 7f82b3557b80 1 -- start start 2024-05-27T08:01:14.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.441+0000 7f82b3557b80 1 Processor -- start 2024-05-27T08:01:14.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.442+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063680fe40 0x5606369a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.442+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x5606369af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.442+0000 7f82b3557b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560636994d30 0x5606369afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.442+0000 7f82b3557b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82a01b0d00 con 0x56063698edb0 2024-05-27T08:01:14.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.442+0000 7f82b3557b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82a01b2150 con 0x56063680fe40 2024-05-27T08:01:14.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.442+0000 7f82b3557b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f82a01b13b0 con 0x560636994d30 2024-05-27T08:01:14.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.442+0000 7f82b3557b80 10 client.0 _ll_register_callbacks cb 0x560636856c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:14.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.443+0000 7f829d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x5606369af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.443+0000 7f829d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x5606369af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41172/0 (socket says 172.21.15.63:41172) 2024-05-27T08:01:14.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.443+0000 7f829d400700 1 -- 172.21.15.63:0/2614806207 learned_addr learned my addr 172.21.15.63:0/2614806207 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:14.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.443+0000 7f829de00700 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063680fe40 0x5606369a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.443+0000 7f829e800700 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560636994d30 0x5606369afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.443+0000 7f829d400700 1 -- 172.21.15.63:0/2614806207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560636994d30 msgr2=0x5606369afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.443+0000 7f829d400700 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560636994d30 0x5606369afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.443+0000 7f829d400700 1 -- 172.21.15.63:0/2614806207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063680fe40 msgr2=0x5606369a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.443+0000 7f829d400700 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063680fe40 0x5606369a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.443+0000 7f829d400700 1 -- 172.21.15.63:0/2614806207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606369b03e0 con 0x56063698edb0 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.444+0000 7f829d400700 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x5606369af960 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f8288009d10 tx=0x7f82880cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cbb9866ad4d60f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.444+0000 7f8283e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.444+0000 7f8283e00700 1 -- 172.21.15.63:0/2614806207 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828806b070 con 0x56063698edb0 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.444+0000 7f82b3557b80 1 -- 172.21.15.63:0/2614806207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f829801e040 con 0x56063698edb0 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.444+0000 7f8283e00700 1 -- 172.21.15.63:0/2614806207 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9571+0+0 (secure 0 0 0) 0x7f82880e0db0 con 0x56063698edb0 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.444+0000 7f8283e00700 1 -- 172.21.15.63:0/2614806207 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f82880c3070 con 0x56063698edb0 2024-05-27T08:01:14.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.444+0000 7f82b3557b80 10 client.20769 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:14.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.444+0000 7f82b3557b80 1 -- 172.21.15.63:0/2614806207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560636a1b6f0 con 0x56063698edb0 2024-05-27T08:01:14.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.445+0000 7f8283e00700 1 -- 172.21.15.63:0/2614806207 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f829801e040 con 0x56063698edb0 2024-05-27T08:01:14.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.445+0000 7f8283e00700 1 -- 172.21.15.63:0/2614806207 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f828806e030 con 0x56063698edb0 2024-05-27T08:01:14.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.445+0000 7f82b3557b80 10 client.20769 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:14.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.445+0000 7f82b3557b80 10 client.20769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:14.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.445+0000 7f82b3557b80 20 client.20769 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:14.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.445+0000 7f82b3557b80 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560636a1bd80 0x560636a1e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:14.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.445+0000 7f82b3557b80 4 client.20769 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:01:14.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.445+0000 7f82b3557b80 1 -- 172.21.15.63:0/2614806207 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606368b8570 con 0x560636a1bd80 2024-05-27T08:01:14.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.446+0000 7f829de00700 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560636a1bd80 0x560636a1e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:14.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.447+0000 7f829de00700 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560636a1bd80 0x560636a1e1a0 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3abef4eab2bc1cd server_cookie=885f17ce94ade49e in_seq=0 out_seq=0 2024-05-27T08:01:14.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.447+0000 7f8283e00700 10 client.20769 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:14.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f8283e00700 1 -- 172.21.15.63:0/2614806207 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5606368b8570 con 0x560636a1bd80 2024-05-27T08:01:14.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f8283e00700 10 client.20769 handle_command_reply: tid=0 2024-05-27T08:01:14.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 client.20769 shutdown 2024-05-27T08:01:14.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 -- 172.21.15.63:0/2614806207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 msgr2=0x5606369af960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:14.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x5606369af960 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f8288009d10 tx=0x7f82880cd040 comp rx=0 tx=0).stop 2024-05-27T08:01:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 -- 172.21.15.63:0/2614806207 shutdown_connections 2024-05-27T08:01:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560636a1bd80 0x560636a1e1a0 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560636994d30 0x5606369afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56063680fe40 0x5606369a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 --2- 172.21.15.63:0/2614806207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56063698edb0 0x5606369af960 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 -- 172.21.15.63:0/2614806207 >> 172.21.15.63:0/2614806207 conn(0x56063686ffc0 msgr2=0x560636998d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f8283e00700 0 client.20769 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:14.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 -- 172.21.15.63:0/2614806207 shutdown_connections 2024-05-27T08:01:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.448+0000 7f82b3557b80 1 -- 172.21.15.63:0/2614806207 wait complete. 2024-05-27T08:01:14.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.449+0000 7f82b3557b80 20 client.20769 trim_cache size 0 max 16384 2024-05-27T08:01:14.451 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:01:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:01:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:01:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "46482219-a7e7-4b53-b46f-66f3f179ade4": { 2024-05-27T08:01:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:01:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "46482219-a7e7-4b53-b46f-66f3f179ade4", 2024-05-27T08:01:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:01:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:01:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:01:14.452 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:01:14.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.449+0000 7f82a4a00700 1 -- 172.21.15.63:0/4210059009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82840b29f0 msgr2=0x7f82840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.449+0000 7f82a4a00700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82840b29f0 0x7f82840b4f00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8298009710 tx=0x7f8298009c70 comp rx=0 tx=0).stop 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.449+0000 7f82a4a00700 1 -- 172.21.15.63:0/4210059009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a0107de0 msgr2=0x7f82a019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.449+0000 7f82a4a00700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a0107de0 0x7f82a019b390 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f828800b580 tx=0x7f8288004790 comp rx=0 tx=0).stop 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.450+0000 7f82a4a00700 1 -- 172.21.15.63:0/4210059009 shutdown_connections 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.450+0000 7f82a4a00700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f82840b29f0 0x7f82840b4f00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.450+0000 7f82a4a00700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82a0108730 0x7f82a019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.450+0000 7f82a4a00700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82a01034b0 0x7f82a019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.450+0000 7f82a4a00700 1 --2- 172.21.15.63:0/4210059009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a0107de0 0x7f82a019b390 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.450+0000 7f82a4a00700 1 -- 172.21.15.63:0/4210059009 >> 172.21.15.63:0/4210059009 conn(0x7f82a00fedb0 msgr2=0x7f82a006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.450+0000 7f82a4a00700 1 -- 172.21.15.63:0/4210059009 shutdown_connections 2024-05-27T08:01:14.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:14.450+0000 7f82a4a00700 1 -- 172.21.15.63:0/4210059009 wait complete. 2024-05-27T08:01:15.318 INFO:tasks.cephfs.filesystem:scrub status for tag:46482219-a7e7-4b53-b46f-66f3f179ade4 - {'path': '/', 'tag': '46482219-a7e7-4b53-b46f-66f3f179ade4', 'options': 'recursive,force'} 2024-05-27T08:01:15.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:15 smithi086 ceph-mon[34687]: pgmap v877: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.7 MiB/s wr, 51 op/s 2024-05-27T08:01:15.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:15 smithi149 ceph-mon[35408]: pgmap v877: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.7 MiB/s wr, 51 op/s 2024-05-27T08:01:15.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:15 smithi063 ceph-mon[31613]: pgmap v877: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 6.7 MiB/s wr, 51 op/s 2024-05-27T08:01:17.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:16 smithi149 ceph-mon[35408]: osdmap e138: 12 total, 12 up, 12 in 2024-05-27T08:01:17.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:16 smithi149 ceph-mon[35408]: pgmap v879: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 3.6 MiB/s wr, 46 op/s 2024-05-27T08:01:17.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:16 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:01:17.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:16 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:01:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:16 smithi063 ceph-mon[31613]: osdmap e138: 12 total, 12 up, 12 in 2024-05-27T08:01:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:16 smithi063 ceph-mon[31613]: pgmap v879: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 3.6 MiB/s wr, 46 op/s 2024-05-27T08:01:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:16 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:01:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:16 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:01:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:16 smithi086 ceph-mon[34687]: osdmap e138: 12 total, 12 up, 12 in 2024-05-27T08:01:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:16 smithi086 ceph-mon[34687]: pgmap v879: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 3.6 MiB/s wr, 46 op/s 2024-05-27T08:01:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:16 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:01:17.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:16 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:01:18.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:17 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:01:18.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:17 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:01:18.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:17 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:01:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:18 smithi086 ceph-mon[34687]: pgmap v880: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-05-27T08:01:19.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:18 smithi149 ceph-mon[35408]: pgmap v880: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-05-27T08:01:19.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:18 smithi063 ceph-mon[31613]: pgmap v880: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-05-27T08:01:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.817+0000 7f90de400640 1 -- 172.21.15.63:0/2579115428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d8109f50 msgr2=0x7f90d8103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.817+0000 7f90de400640 1 --2- 172.21.15.63:0/2579115428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d8109f50 0x7f90d8103be0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009900 tx=0x7f90c002a410 comp rx=0 tx=0).stop 2024-05-27T08:01:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.817+0000 7f90de400640 1 -- 172.21.15.63:0/2579115428 shutdown_connections 2024-05-27T08:01:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.817+0000 7f90de400640 1 --2- 172.21.15.63:0/2579115428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d8104250 0x7f90d81105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.817+0000 7f90de400640 1 --2- 172.21.15.63:0/2579115428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d8109f50 0x7f90d8103be0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.817+0000 7f90de400640 1 --2- 172.21.15.63:0/2579115428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d8109ab0 0x7f90d81036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.817+0000 7f90de400640 1 -- 172.21.15.63:0/2579115428 >> 172.21.15.63:0/2579115428 conn(0x7f90d8079240 msgr2=0x7f90d8079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.817+0000 7f90de400640 1 -- 172.21.15.63:0/2579115428 shutdown_connections 2024-05-27T08:01:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.817+0000 7f90de400640 1 -- 172.21.15.63:0/2579115428 wait complete. 2024-05-27T08:01:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.818+0000 7f90de400640 1 Processor -- start 2024-05-27T08:01:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.819+0000 7f90de400640 1 -- start start 2024-05-27T08:01:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.819+0000 7f90de400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d8104250 0x7f90d819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.819+0000 7f90de400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d8109ab0 0x7f90d819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.819+0000 7f90de400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d8109f50 0x7f90d819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.819+0000 7f90de400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90d8113b50 con 0x7f90d8109ab0 2024-05-27T08:01:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.819+0000 7f90de400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81139d0 con 0x7f90d8109f50 2024-05-27T08:01:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.819+0000 7f90de400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90d8113cd0 con 0x7f90d8104250 2024-05-27T08:01:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d8109ab0 0x7f90d819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90cfe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d8109ab0 0x7f90d819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36450/0 (socket says 172.21.15.63:36450) 2024-05-27T08:01:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90cfe00640 1 -- 172.21.15.63:0/850549136 learned_addr learned my addr 172.21.15.63:0/850549136 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90dd000640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d8104250 0x7f90d819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90dda00640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d8109f50 0x7f90d819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90cfe00640 1 -- 172.21.15.63:0/850549136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d8104250 msgr2=0x7f90d819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90cfe00640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d8104250 0x7f90d819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90cfe00640 1 -- 172.21.15.63:0/850549136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d8109f50 msgr2=0x7f90d819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90cfe00640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d8109f50 0x7f90d819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90cfe00640 1 -- 172.21.15.63:0/850549136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d819f470 con 0x7f90d8109ab0 2024-05-27T08:01:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.820+0000 7f90dda00640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d8109f50 0x7f90d819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.821+0000 7f90cfe00640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d8109ab0 0x7f90d819ab70 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f90c0005940 tx=0x7f90c00040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.821+0000 7f90d6000640 1 -- 172.21.15.63:0/850549136 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c003f030 con 0x7f90d8109ab0 2024-05-27T08:01:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.821+0000 7f90d6000640 1 -- 172.21.15.63:0/850549136 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90c0003a80 con 0x7f90d8109ab0 2024-05-27T08:01:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.821+0000 7f90d6000640 1 -- 172.21.15.63:0/850549136 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0035070 con 0x7f90d8109ab0 2024-05-27T08:01:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.821+0000 7f90de400640 1 -- 172.21.15.63:0/850549136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d819f700 con 0x7f90d8109ab0 2024-05-27T08:01:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.821+0000 7f90de400640 1 -- 172.21.15.63:0/850549136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d819fba0 con 0x7f90d8109ab0 2024-05-27T08:01:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.825+0000 7f90e6975740 1 Processor -- start 2024-05-27T08:01:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.826+0000 7f90e6975740 1 -- start start 2024-05-27T08:01:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.826+0000 7f90d6000640 1 -- 172.21.15.63:0/850549136 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f90c0042030 con 0x7f90d8109ab0 2024-05-27T08:01:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.826+0000 7f90e6975740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de82930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.826+0000 7f90e6975740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dab72f0 0x55f95de82e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.826+0000 7f90e6975740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dabf4d0 0x55f95de833b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.826+0000 7f90e6975740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f95ddf3b90 con 0x55f95dae01b0 2024-05-27T08:01:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.826+0000 7f90e6975740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f95dde3730 con 0x55f95dabf4d0 2024-05-27T08:01:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.826+0000 7f90e6975740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f95dde7770 con 0x55f95dab72f0 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.827+0000 7f90dda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de82930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.827+0000 7f90dda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de82930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36456/0 (socket says 172.21.15.63:36456) 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.827+0000 7f90dda00640 1 -- 172.21.15.63:0/2003853880 learned_addr learned my addr 172.21.15.63:0/2003853880 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.827+0000 7f90cfe00640 1 --2- 172.21.15.63:0/2003853880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dabf4d0 0x55f95de833b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.827+0000 7f90dd000640 1 --2- 172.21.15.63:0/2003853880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dab72f0 0x55f95de82e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.827+0000 7f90dda00640 1 -- 172.21.15.63:0/2003853880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dab72f0 msgr2=0x55f95de82e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.827+0000 7f90dda00640 1 --2- 172.21.15.63:0/2003853880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dab72f0 0x55f95de82e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.827+0000 7f90dda00640 1 -- 172.21.15.63:0/2003853880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dabf4d0 msgr2=0x55f95de833b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.827+0000 7f90dda00640 1 --2- 172.21.15.63:0/2003853880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dabf4d0 0x55f95de833b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.827+0000 7f90dda00640 1 -- 172.21.15.63:0/2003853880 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95dd7c8b0 con 0x55f95dae01b0 2024-05-27T08:01:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.828+0000 7f90dda00640 1 --2- 172.21.15.63:0/2003853880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de82930 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f90d0010370 tx=0x7f90d0010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f0c46ce41eefc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.828+0000 7f90ce000640 1 -- 172.21.15.63:0/2003853880 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0016070 con 0x55f95dae01b0 2024-05-27T08:01:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.828+0000 7f90ce000640 1 -- 172.21.15.63:0/2003853880 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90d0012ba0 con 0x55f95dae01b0 2024-05-27T08:01:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.828+0000 7f90ce000640 1 -- 172.21.15.63:0/2003853880 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0011ab0 con 0x55f95dae01b0 2024-05-27T08:01:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.828+0000 7f90d6000640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90bc0b29d0 0x7f90bc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.829+0000 7f90d6000640 1 -- 172.21.15.63:0/850549136 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f90c0102fa0 con 0x7f90d8109ab0 2024-05-27T08:01:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.829+0000 7f90dd000640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90bc0b29d0 0x7f90bc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.829+0000 7f90e6975740 1 -- 172.21.15.63:0/2003853880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 msgr2=0x55f95de82930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.829+0000 7f90e6975740 1 --2- 172.21.15.63:0/2003853880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de82930 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f90d0010370 tx=0x7f90d0010830 comp rx=0 tx=0).stop 2024-05-27T08:01:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.829+0000 7f90e6975740 1 -- 172.21.15.63:0/2003853880 shutdown_connections 2024-05-27T08:01:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.829+0000 7f90e6975740 1 --2- 172.21.15.63:0/2003853880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dabf4d0 0x55f95de833b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.829+0000 7f90e6975740 1 --2- 172.21.15.63:0/2003853880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dab72f0 0x55f95de82e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.829+0000 7f90e6975740 1 --2- 172.21.15.63:0/2003853880 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de82930 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.829+0000 7f90e6975740 1 -- 172.21.15.63:0/2003853880 >> 172.21.15.63:0/2003853880 conn(0x55f95dad4300 msgr2=0x55f95de7f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.830+0000 7f90e6975740 1 -- 172.21.15.63:0/2003853880 shutdown_connections 2024-05-27T08:01:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.830+0000 7f90e6975740 1 -- 172.21.15.63:0/2003853880 wait complete. 2024-05-27T08:01:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.830+0000 7f90dd000640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90bc0b29d0 0x7f90bc0b4ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f90c8005fd0 tx=0x7f90c800b5b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.830+0000 7f90e6975740 1 -- start start 2024-05-27T08:01:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.831+0000 7f90e6975740 1 Processor -- start 2024-05-27T08:01:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.831+0000 7f90e6975740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dab72f0 0x55f95de91070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.831+0000 7f90e6975740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.831+0000 7f90e6975740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dae01b0 0x55f95de9d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.831+0000 7f90e6975740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f95dac9790 con 0x55f95dab72f0 2024-05-27T08:01:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.831+0000 7f90e6975740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f95dd8bbc0 con 0x55f95dae01b0 2024-05-27T08:01:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.831+0000 7f90e6975740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f95de94050 con 0x55f95dabf4d0 2024-05-27T08:01:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.831+0000 7f90e6975740 10 client.0 _ll_register_callbacks cb 0x55f95dd69400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90dd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dab72f0 0x55f95de91070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90dd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dab72f0 0x55f95de91070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36468/0 (socket says 172.21.15.63:36468) 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90dd000640 1 -- 172.21.15.63:0/2877478186 learned_addr learned my addr 172.21.15.63:0/2877478186 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90dda00640 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dae01b0 0x55f95de9d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90cfe00640 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90dd000640 1 -- 172.21.15.63:0/2877478186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 msgr2=0x55f95de915b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90dd000640 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de915b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90dd000640 1 -- 172.21.15.63:0/2877478186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dae01b0 msgr2=0x55f95de9d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90dd000640 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dae01b0 0x55f95de9d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90dd000640 1 -- 172.21.15.63:0/2877478186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95daf8210 con 0x55f95dab72f0 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.832+0000 7f90dda00640 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dae01b0 0x55f95de9d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.833+0000 7f90dd000640 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dab72f0 0x55f95de91070 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f90c80126d0 tx=0x7f90c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de35bf39bc6e141 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.833+0000 7f90c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.833+0000 7f90c6a00640 1 -- 172.21.15.63:0/2877478186 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8005220 con 0x55f95dab72f0 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.833+0000 7f90e6975740 1 -- 172.21.15.63:0/2877478186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d0012ba0 con 0x55f95dab72f0 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.833+0000 7f90c6a00640 1 -- 172.21.15.63:0/2877478186 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f90c80053c0 con 0x55f95dab72f0 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.833+0000 7f90c6a00640 1 -- 172.21.15.63:0/2877478186 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f90c802f050 con 0x55f95dab72f0 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.833+0000 7f90e6975740 10 client.20781 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.833+0000 7f90e6975740 1 -- 172.21.15.63:0/2877478186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f95dbc4620 con 0x55f95dab72f0 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.834+0000 7f90c6a00640 1 -- 172.21.15.63:0/2877478186 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90d0012ba0 con 0x55f95dab72f0 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.834+0000 7f90c6a00640 1 -- 172.21.15.63:0/2877478186 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f90c801f910 con 0x55f95dab72f0 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.834+0000 7f90e6975740 10 client.20781 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.834+0000 7f90e6975740 10 client.20781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.834+0000 7f90e6975740 20 client.20781 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.834+0000 7f90e6975740 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f95da9ed80 0x55f95dea34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.834+0000 7f90e6975740 4 client.20781 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.834+0000 7f90e6975740 1 -- 172.21.15.63:0/2877478186 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f95ddf5ae0 con 0x55f95da9ed80 2024-05-27T08:01:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.835+0000 7f90cfe00640 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f95da9ed80 0x55f95dea34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.836+0000 7f90cfe00640 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f95da9ed80 0x55f95dea34c0 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3097f88cd01eb4e7 server_cookie=2ac54afa5266496f in_seq=0 out_seq=0 2024-05-27T08:01:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.836+0000 7f90c6a00640 10 client.20781 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90c6a00640 1 -- 172.21.15.63:0/2877478186 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f95ddf5ae0 con 0x55f95da9ed80 2024-05-27T08:01:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90c6a00640 10 client.20781 handle_command_reply: tid=0 2024-05-27T08:01:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90e6975740 1 client.20781 shutdown 2024-05-27T08:01:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90e6975740 1 -- 172.21.15.63:0/2877478186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dab72f0 msgr2=0x55f95de91070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90e6975740 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dab72f0 0x55f95de91070 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f90c80126d0 tx=0x7f90c80046c0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90e6975740 1 -- 172.21.15.63:0/2877478186 shutdown_connections 2024-05-27T08:01:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90e6975740 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f95da9ed80 0x55f95dea34c0 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90e6975740 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dae01b0 0x55f95de9d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90e6975740 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de915b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90e6975740 1 --2- 172.21.15.63:0/2877478186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dab72f0 0x55f95de91070 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90e6975740 1 -- 172.21.15.63:0/2877478186 >> 172.21.15.63:0/2877478186 conn(0x55f95dad4300 msgr2=0x55f95de85140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.838+0000 7f90c6a00640 0 client.20781 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.839+0000 7f90e6975740 1 -- 172.21.15.63:0/2877478186 shutdown_connections 2024-05-27T08:01:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.839+0000 7f90e6975740 1 -- 172.21.15.63:0/2877478186 wait complete. 2024-05-27T08:01:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.839+0000 7f90e6975740 20 client.20781 trim_cache size 0 max 16384 2024-05-27T08:01:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.864+0000 7f90e6975740 1 Processor -- start 2024-05-27T08:01:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.864+0000 7f90e6975740 1 -- start start 2024-05-27T08:01:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.864+0000 7f90e6975740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dab72f0 0x55f95de937a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.864+0000 7f90e6975740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.864+0000 7f90e6975740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.864+0000 7f90e6975740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81b17d0 con 0x55f95dae01b0 2024-05-27T08:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.864+0000 7f90e6975740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81af0a0 con 0x55f95dab72f0 2024-05-27T08:01:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.864+0000 7f90e6975740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81acf10 con 0x55f95dabf4d0 2024-05-27T08:01:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.865+0000 7f90dda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.865+0000 7f90dda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36478/0 (socket says 172.21.15.63:36478) 2024-05-27T08:01:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.865+0000 7f90dda00640 1 -- 172.21.15.63:0/3333909294 learned_addr learned my addr 172.21.15.63:0/3333909294 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.865+0000 7f90dd000640 1 --2- 172.21.15.63:0/3333909294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dab72f0 0x55f95de937a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.865+0000 7f90cfe00640 1 --2- 172.21.15.63:0/3333909294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.865+0000 7f90dda00640 1 -- 172.21.15.63:0/3333909294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 msgr2=0x55f95de832c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.865+0000 7f90dda00640 1 --2- 172.21.15.63:0/3333909294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de832c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.865+0000 7f90dda00640 1 -- 172.21.15.63:0/3333909294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dab72f0 msgr2=0x55f95de937a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.865+0000 7f90dda00640 1 --2- 172.21.15.63:0/3333909294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dab72f0 0x55f95de937a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.865+0000 7f90dda00640 1 -- 172.21.15.63:0/3333909294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95da9efb0 con 0x55f95dae01b0 2024-05-27T08:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.866+0000 7f90dda00640 1 --2- 172.21.15.63:0/3333909294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8c800 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009f00 tx=0x7f90d0009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f8c0979075886e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.866+0000 7f90cd600640 1 -- 172.21.15.63:0/3333909294 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d002a070 con 0x55f95dae01b0 2024-05-27T08:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.866+0000 7f90cd600640 1 -- 172.21.15.63:0/3333909294 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90d0004c00 con 0x55f95dae01b0 2024-05-27T08:01:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.866+0000 7f90cd600640 1 -- 172.21.15.63:0/3333909294 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0016050 con 0x55f95dae01b0 2024-05-27T08:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.867+0000 7f90e6975740 1 -- 172.21.15.63:0/3333909294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 msgr2=0x55f95de8c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.867+0000 7f90e6975740 1 --2- 172.21.15.63:0/3333909294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8c800 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009f00 tx=0x7f90d0009f30 comp rx=0 tx=0).stop 2024-05-27T08:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.868+0000 7f90e6975740 1 -- 172.21.15.63:0/3333909294 shutdown_connections 2024-05-27T08:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.868+0000 7f90e6975740 1 --2- 172.21.15.63:0/3333909294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8c800 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.868+0000 7f90e6975740 1 --2- 172.21.15.63:0/3333909294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de832c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.868+0000 7f90e6975740 1 --2- 172.21.15.63:0/3333909294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dab72f0 0x55f95de937a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.868+0000 7f90e6975740 1 -- 172.21.15.63:0/3333909294 >> 172.21.15.63:0/3333909294 conn(0x55f95dad4300 msgr2=0x55f95de85140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.868+0000 7f90e6975740 1 -- 172.21.15.63:0/3333909294 shutdown_connections 2024-05-27T08:01:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.868+0000 7f90e6975740 1 -- 172.21.15.63:0/3333909294 wait complete. 2024-05-27T08:01:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.868+0000 7f90e6975740 1 -- start start 2024-05-27T08:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.868+0000 7f90e6975740 1 Processor -- start 2024-05-27T08:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.869+0000 7f90e6975740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dab72f0 0x55f95de8c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.869+0000 7f90e6975740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de8aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.869+0000 7f90e6975740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.869+0000 7f90e6975740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81ad560 con 0x55f95dae01b0 2024-05-27T08:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.869+0000 7f90e6975740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81ae9d0 con 0x55f95dab72f0 2024-05-27T08:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.869+0000 7f90e6975740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81adc30 con 0x55f95dabf4d0 2024-05-27T08:01:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.869+0000 7f90e6975740 10 client.0 _ll_register_callbacks cb 0x7f90c800b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.869+0000 7f90dda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.869+0000 7f90dda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36482/0 (socket says 172.21.15.63:36482) 2024-05-27T08:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.869+0000 7f90dda00640 1 -- 172.21.15.63:0/2439647009 learned_addr learned my addr 172.21.15.63:0/2439647009 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.870+0000 7f90dd000640 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dab72f0 0x55f95de8c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.870+0000 7f90cfe00640 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de8aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.870+0000 7f90dda00640 1 -- 172.21.15.63:0/2439647009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 msgr2=0x55f95de8aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.870+0000 7f90dda00640 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de8aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.870+0000 7f90dda00640 1 -- 172.21.15.63:0/2439647009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dab72f0 msgr2=0x55f95de8c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.870+0000 7f90dda00640 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dab72f0 0x55f95de8c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.870+0000 7f90dda00640 1 -- 172.21.15.63:0/2439647009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95de8b740 con 0x55f95dae01b0 2024-05-27T08:01:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.870+0000 7f90dda00640 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8b000 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f90d0010fe0 tx=0x7f90d00127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51b4cc65231926f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.870+0000 7f90c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.871+0000 7f90c6a00640 1 -- 172.21.15.63:0/2439647009 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d002a040 con 0x55f95dae01b0 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.871+0000 7f90e6975740 1 -- 172.21.15.63:0/2439647009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d0004c00 con 0x55f95dae01b0 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.871+0000 7f90c6a00640 1 -- 172.21.15.63:0/2439647009 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f90d00129b0 con 0x55f95dae01b0 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.871+0000 7f90c6a00640 1 -- 172.21.15.63:0/2439647009 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f90d0016020 con 0x55f95dae01b0 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.871+0000 7f90e6975740 10 client.20787 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.871+0000 7f90e6975740 1 -- 172.21.15.63:0/2439647009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f95de99210 con 0x55f95dae01b0 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.871+0000 7f90c6a00640 1 -- 172.21.15.63:0/2439647009 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90d0004c00 con 0x55f95dae01b0 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.872+0000 7f90c6a00640 1 -- 172.21.15.63:0/2439647009 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f90d002d050 con 0x55f95dae01b0 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.872+0000 7f90e6975740 10 client.20787 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.872+0000 7f90e6975740 10 client.20787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.872+0000 7f90e6975740 20 client.20787 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.872+0000 7f90e6975740 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f95da741c0 0x55f95de99790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.872+0000 7f90e6975740 4 client.20787 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.872+0000 7f90e6975740 1 -- 172.21.15.63:0/2439647009 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f95ddef2c0 con 0x55f95da741c0 2024-05-27T08:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.872+0000 7f90dd000640 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f95da741c0 0x55f95de99790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.873+0000 7f90dd000640 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f95da741c0 0x55f95de99790 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ae3332708ffcde server_cookie=a95b8acd4c7fc12a in_seq=0 out_seq=0 2024-05-27T08:01:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.873+0000 7f90c6a00640 10 client.20787 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.914+0000 7f90c6a00640 1 -- 172.21.15.63:0/2439647009 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f95ddef2c0 con 0x55f95da741c0 2024-05-27T08:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.914+0000 7f90c6a00640 10 client.20787 handle_command_reply: tid=0 2024-05-27T08:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.914+0000 7f90e6975740 1 client.20787 shutdown 2024-05-27T08:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.914+0000 7f90e6975740 1 -- 172.21.15.63:0/2439647009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 msgr2=0x55f95de8b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.914+0000 7f90e6975740 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8b000 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f90d0010fe0 tx=0x7f90d00127c0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.915+0000 7f90e6975740 1 -- 172.21.15.63:0/2439647009 shutdown_connections 2024-05-27T08:01:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.915+0000 7f90e6975740 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f95da741c0 0x55f95de99790 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.915+0000 7f90e6975740 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f95dae01b0 0x55f95de8b000 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.915+0000 7f90e6975740 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f95dabf4d0 0x55f95de8aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.915+0000 7f90e6975740 1 --2- 172.21.15.63:0/2439647009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f95dab72f0 0x55f95de8c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.915+0000 7f90e6975740 1 -- 172.21.15.63:0/2439647009 >> 172.21.15.63:0/2439647009 conn(0x55f95dad4300 msgr2=0x55f95de91330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.915+0000 7f90c6a00640 0 client.20787 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.915+0000 7f90e6975740 1 -- 172.21.15.63:0/2439647009 shutdown_connections 2024-05-27T08:01:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.915+0000 7f90e6975740 1 -- 172.21.15.63:0/2439647009 wait complete. 2024-05-27T08:01:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.915+0000 7f90e6975740 20 client.20787 trim_cache size 0 max 16384 2024-05-27T08:01:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:01:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:01:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.916+0000 7f90de400640 1 -- 172.21.15.63:0/850549136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90bc0b29d0 msgr2=0x7f90bc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.916+0000 7f90de400640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90bc0b29d0 0x7f90bc0b4ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f90c8005fd0 tx=0x7f90c800b5b0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.916+0000 7f90de400640 1 -- 172.21.15.63:0/850549136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d8109ab0 msgr2=0x7f90d819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.916+0000 7f90de400640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d8109ab0 0x7f90d819ab70 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f90c0005940 tx=0x7f90c00040e0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.917+0000 7f90de400640 1 -- 172.21.15.63:0/850549136 shutdown_connections 2024-05-27T08:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.917+0000 7f90de400640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90bc0b29d0 0x7f90bc0b4ee0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.917+0000 7f90de400640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d8109f50 0x7f90d819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.917+0000 7f90de400640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d8109ab0 0x7f90d819ab70 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.917+0000 7f90de400640 1 --2- 172.21.15.63:0/850549136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d8104250 0x7f90d819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.917+0000 7f90de400640 1 -- 172.21.15.63:0/850549136 >> 172.21.15.63:0/850549136 conn(0x7f90d8079240 msgr2=0x7f90d8102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.917+0000 7f90de400640 1 -- 172.21.15.63:0/850549136 shutdown_connections 2024-05-27T08:01:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:19.917+0000 7f90de400640 1 -- 172.21.15.63:0/850549136 wait complete. 2024-05-27T08:01:21.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:21 smithi086 ceph-mon[34687]: pgmap v881: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-05-27T08:01:21.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:21 smithi149 ceph-mon[35408]: pgmap v881: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-05-27T08:01:21.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:21 smithi063 ceph-mon[31613]: pgmap v881: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.8 MiB/s wr, 46 op/s 2024-05-27T08:01:23.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:23 smithi086 ceph-mon[34687]: pgmap v882: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:01:23.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:23 smithi149 ceph-mon[35408]: pgmap v882: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:01:23.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:23 smithi063 ceph-mon[31613]: pgmap v882: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:01:25.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:25 smithi086 ceph-mon[34687]: pgmap v883: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:01:25.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:25 smithi149 ceph-mon[35408]: pgmap v883: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:01:25.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:25 smithi063 ceph-mon[31613]: pgmap v883: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:01:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:27 smithi086 ceph-mon[34687]: pgmap v884: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 13 op/s 2024-05-27T08:01:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:27 smithi149 ceph-mon[35408]: pgmap v884: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 13 op/s 2024-05-27T08:01:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:27 smithi063 ceph-mon[31613]: pgmap v884: 129 pgs: 129 active+clean; 8.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 13 op/s 2024-05-27T08:01:29.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:29 smithi086 ceph-mon[34687]: pgmap v885: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 15 op/s 2024-05-27T08:01:29.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:29 smithi149 ceph-mon[35408]: pgmap v885: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 15 op/s 2024-05-27T08:01:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:29 smithi063 ceph-mon[31613]: pgmap v885: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 15 op/s 2024-05-27T08:01:31.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:31 smithi086 ceph-mon[34687]: pgmap v886: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T08:01:31.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:31 smithi149 ceph-mon[35408]: pgmap v886: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T08:01:31.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:31 smithi063 ceph-mon[31613]: pgmap v886: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T08:01:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.150+0000 7faadc600640 1 -- 172.21.15.63:0/3797602641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad4075420 msgr2=0x7faad4075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.150+0000 7faadc600640 1 --2- 172.21.15.63:0/3797602641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad4075420 0x7faad4075830 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7faac8009900 tx=0x7faac802a480 comp rx=0 tx=0).stop 2024-05-27T08:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.150+0000 7faadc600640 1 -- 172.21.15.63:0/3797602641 shutdown_connections 2024-05-27T08:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.150+0000 7faadc600640 1 --2- 172.21.15.63:0/3797602641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faad4110520 0x7faad4112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.150+0000 7faadc600640 1 --2- 172.21.15.63:0/3797602641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faad410dbc0 0x7faad410ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.150+0000 7faadc600640 1 --2- 172.21.15.63:0/3797602641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad4075420 0x7faad4075830 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.150+0000 7faadc600640 1 -- 172.21.15.63:0/3797602641 >> 172.21.15.63:0/3797602641 conn(0x7faad4079440 msgr2=0x7faad4079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.150+0000 7faadc600640 1 -- 172.21.15.63:0/3797602641 shutdown_connections 2024-05-27T08:01:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.150+0000 7faadc600640 1 -- 172.21.15.63:0/3797602641 wait complete. 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.151+0000 7faadc600640 1 Processor -- start 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.151+0000 7faadc600640 1 -- start start 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.152+0000 7faadc600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad4075420 0x7faad41a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.152+0000 7faadc600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faad410dbc0 0x7faad41a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.152+0000 7faadc600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faad4110520 0x7faad419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.152+0000 7faadc600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faad4115f50 con 0x7faad4075420 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.152+0000 7faadc600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faad4115dd0 con 0x7faad410dbc0 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.152+0000 7faadc600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faad41160d0 con 0x7faad4110520 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.152+0000 7faad9c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad4075420 0x7faad41a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.152+0000 7faad9c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad4075420 0x7faad41a2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46268/0 (socket says 172.21.15.63:46268) 2024-05-27T08:01:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.152+0000 7faad9c00640 1 -- 172.21.15.63:0/2373736469 learned_addr learned my addr 172.21.15.63:0/2373736469 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.153+0000 7faad9200640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faad410dbc0 0x7faad41a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.153+0000 7faada600640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faad4110520 0x7faad419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.153+0000 7faad9c00640 1 -- 172.21.15.63:0/2373736469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faad4110520 msgr2=0x7faad419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.153+0000 7faad9c00640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faad4110520 0x7faad419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.153+0000 7faad9c00640 1 -- 172.21.15.63:0/2373736469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faad410dbc0 msgr2=0x7faad41a2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.153+0000 7faad9c00640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faad410dbc0 0x7faad41a2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.153+0000 7faad9c00640 1 -- 172.21.15.63:0/2373736469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad419cee0 con 0x7faad4075420 2024-05-27T08:01:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.153+0000 7faad9200640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faad410dbc0 0x7faad41a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.153+0000 7faad9c00640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad4075420 0x7faad41a2640 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7faac8009230 tx=0x7faac8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.154+0000 7faad2000640 1 -- 172.21.15.63:0/2373736469 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac803f030 con 0x7faad4075420 2024-05-27T08:01:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.154+0000 7faadc600640 1 -- 172.21.15.63:0/2373736469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad419d170 con 0x7faad4075420 2024-05-27T08:01:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.154+0000 7faad2000640 1 -- 172.21.15.63:0/2373736469 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faac8003a80 con 0x7faad4075420 2024-05-27T08:01:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.154+0000 7faad2000640 1 -- 172.21.15.63:0/2373736469 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8035070 con 0x7faad4075420 2024-05-27T08:01:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.154+0000 7faadc600640 1 -- 172.21.15.63:0/2373736469 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad419d6b0 con 0x7faad4075420 2024-05-27T08:01:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.158+0000 7faae3604740 1 Processor -- start 2024-05-27T08:01:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.158+0000 7faae3604740 1 -- start start 2024-05-27T08:01:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.159+0000 7faae3604740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.159+0000 7faae3604740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.159+0000 7faae3604740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822ec2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.159+0000 7faae3604740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56398225cb90 con 0x563981f491b0 2024-05-27T08:01:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.159+0000 7faae3604740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56398224c730 con 0x563981f284d0 2024-05-27T08:01:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.159+0000 7faae3604740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563982250770 con 0x563981f202f0 2024-05-27T08:01:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.159+0000 7faada600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.159+0000 7faada600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822eb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46274/0 (socket says 172.21.15.63:46274) 2024-05-27T08:01:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.159+0000 7faada600640 1 -- 172.21.15.63:0/716705199 learned_addr learned my addr 172.21.15.63:0/716705199 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.159+0000 7faad9200640 1 --2- 172.21.15.63:0/716705199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822ec2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.160+0000 7faada600640 1 -- 172.21.15.63:0/716705199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 msgr2=0x5639822ebda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:01:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.160+0000 7faada600640 1 --2- 172.21.15.63:0/716705199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822ebda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.160+0000 7faada600640 1 -- 172.21.15.63:0/716705199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 msgr2=0x5639822ec2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.160+0000 7faada600640 1 --2- 172.21.15.63:0/716705199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822ec2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.160+0000 7faada600640 1 -- 172.21.15.63:0/716705199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639821e58b0 con 0x563981f491b0 2024-05-27T08:01:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.160+0000 7faad2000640 1 -- 172.21.15.63:0/2373736469 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faac8042030 con 0x7faad4075420 2024-05-27T08:01:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.160+0000 7faada600640 1 --2- 172.21.15.63:0/716705199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822eb860 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7faacc007dd0 tx=0x7faacc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ba8997ba580d26 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.161+0000 7faac6a00640 1 -- 172.21.15.63:0/716705199 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc014070 con 0x563981f491b0 2024-05-27T08:01:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.161+0000 7faac6a00640 1 -- 172.21.15.63:0/716705199 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faacc004d20 con 0x563981f491b0 2024-05-27T08:01:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.161+0000 7faac6a00640 1 -- 172.21.15.63:0/716705199 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc010740 con 0x563981f491b0 2024-05-27T08:01:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.162+0000 7faad2000640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faabc0b29d0 0x7faabc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.162+0000 7faae3604740 1 -- 172.21.15.63:0/716705199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 msgr2=0x5639822eb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.162+0000 7faae3604740 1 --2- 172.21.15.63:0/716705199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822eb860 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7faacc007dd0 tx=0x7faacc00e720 comp rx=0 tx=0).stop 2024-05-27T08:01:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.163+0000 7faad2000640 1 -- 172.21.15.63:0/2373736469 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7faac81031d0 con 0x7faad4075420 2024-05-27T08:01:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.163+0000 7faad9c00640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faabc0b29d0 0x7faabc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.163+0000 7faae3604740 1 -- 172.21.15.63:0/716705199 shutdown_connections 2024-05-27T08:01:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.163+0000 7faae3604740 1 --2- 172.21.15.63:0/716705199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822ec2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.163+0000 7faae3604740 1 --2- 172.21.15.63:0/716705199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.163+0000 7faae3604740 1 --2- 172.21.15.63:0/716705199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822eb860 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.163+0000 7faae3604740 1 -- 172.21.15.63:0/716705199 >> 172.21.15.63:0/716705199 conn(0x563981f3d300 msgr2=0x5639822e8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.163+0000 7faae3604740 1 -- 172.21.15.63:0/716705199 shutdown_connections 2024-05-27T08:01:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.163+0000 7faae3604740 1 -- 172.21.15.63:0/716705199 wait complete. 2024-05-27T08:01:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.164+0000 7faad9c00640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faabc0b29d0 0x7faabc0b4ee0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x5639822e9350 tx=0x7faac802a480 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.164+0000 7faae3604740 1 -- start start 2024-05-27T08:01:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.164+0000 7faae3604740 1 Processor -- start 2024-05-27T08:01:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.165+0000 7faae3604740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.165+0000 7faae3604740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f284d0 0x5639822f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.165+0000 7faae3604740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f491b0 0x563982301cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.165+0000 7faae3604740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563981f32790 con 0x563981f284d0 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.165+0000 7faae3604740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5639821f4bc0 con 0x563981f491b0 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.165+0000 7faae3604740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5639822f8990 con 0x563981f202f0 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.165+0000 7faae3604740 10 client.0 _ll_register_callbacks cb 0x5639821d2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faad9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f284d0 0x5639822f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faad9200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f284d0 0x5639822f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46282/0 (socket says 172.21.15.63:46282) 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faad9200640 1 -- 172.21.15.63:0/2691220461 learned_addr learned my addr 172.21.15.63:0/2691220461 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faada600640 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faada600640 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f491b0 0x563982301cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faad9200640 1 -- 172.21.15.63:0/2691220461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 msgr2=0x5639822f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faad9200640 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faad9200640 1 -- 172.21.15.63:0/2691220461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f491b0 msgr2=0x563982301cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faad9200640 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f491b0 0x563982301cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faad9200640 1 -- 172.21.15.63:0/2691220461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563981f61210 con 0x563981f284d0 2024-05-27T08:01:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.166+0000 7faada600640 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.167+0000 7faad9200640 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f284d0 0x5639822f5f10 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7faac000be90 tx=0x7faac00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c584f9b0a438d99d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.167+0000 7faabb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.167+0000 7faabb400640 1 -- 172.21.15.63:0/2691220461 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0005220 con 0x563981f284d0 2024-05-27T08:01:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.167+0000 7faae3604740 1 -- 172.21.15.63:0/2691220461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faacc004d20 con 0x563981f284d0 2024-05-27T08:01:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.167+0000 7faabb400640 1 -- 172.21.15.63:0/2691220461 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7faac00053c0 con 0x563981f284d0 2024-05-27T08:01:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.168+0000 7faabb400640 1 -- 172.21.15.63:0/2691220461 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faac00055c0 con 0x563981f284d0 2024-05-27T08:01:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.168+0000 7faae3604740 10 client.20799 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.168+0000 7faae3604740 1 -- 172.21.15.63:0/2691220461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56398202d620 con 0x563981f284d0 2024-05-27T08:01:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.168+0000 7faabb400640 1 -- 172.21.15.63:0/2691220461 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faacc004d20 con 0x563981f284d0 2024-05-27T08:01:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.168+0000 7faabb400640 1 -- 172.21.15.63:0/2691220461 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7faac001b850 con 0x563981f284d0 2024-05-27T08:01:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.168+0000 7faae3604740 10 client.20799 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.168+0000 7faae3604740 10 client.20799 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.169+0000 7faae3604740 20 client.20799 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.169+0000 7faae3604740 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563981f07d80 0x563982307cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.169+0000 7faae3604740 4 client.20799 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.169+0000 7faae3604740 1 -- 172.21.15.63:0/2691220461 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56398225eae0 con 0x563981f07d80 2024-05-27T08:01:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.169+0000 7faada600640 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563981f07d80 0x563982307cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.170+0000 7faada600640 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563981f07d80 0x563982307cc0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a8fe3fc4f54f02 server_cookie=ab3bc4d415021e1d in_seq=0 out_seq=0 2024-05-27T08:01:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.170+0000 7faabb400640 10 client.20799 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.172+0000 7faabb400640 1 -- 172.21.15.63:0/2691220461 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56398225eae0 con 0x563981f07d80 2024-05-27T08:01:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.172+0000 7faabb400640 10 client.20799 handle_command_reply: tid=0 2024-05-27T08:01:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.172+0000 7faae3604740 1 client.20799 shutdown 2024-05-27T08:01:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.172+0000 7faae3604740 1 -- 172.21.15.63:0/2691220461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f284d0 msgr2=0x5639822f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.172+0000 7faae3604740 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f284d0 0x5639822f5f10 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7faac000be90 tx=0x7faac00117b0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.173+0000 7faae3604740 1 -- 172.21.15.63:0/2691220461 shutdown_connections 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.173+0000 7faae3604740 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563981f07d80 0x563982307cc0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.173+0000 7faae3604740 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f491b0 0x563982301cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.173+0000 7faae3604740 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f284d0 0x5639822f5f10 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.173+0000 7faae3604740 1 --2- 172.21.15.63:0/2691220461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.173+0000 7faae3604740 1 -- 172.21.15.63:0/2691220461 >> 172.21.15.63:0/2691220461 conn(0x563981f3d300 msgr2=0x5639822eb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.173+0000 7faabb400640 0 client.20799 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.173+0000 7faae3604740 1 -- 172.21.15.63:0/2691220461 shutdown_connections 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.173+0000 7faae3604740 1 -- 172.21.15.63:0/2691220461 wait complete. 2024-05-27T08:01:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.173+0000 7faae3604740 20 client.20799 trim_cache size 0 max 16384 2024-05-27T08:01:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.193+0000 7faae3604740 1 Processor -- start 2024-05-27T08:01:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.194+0000 7faae3604740 1 -- start start 2024-05-27T08:01:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.194+0000 7faae3604740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f202f0 0x5639822f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.194+0000 7faae3604740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.194+0000 7faae3604740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f491b0 0x5639822e7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.194+0000 7faae3604740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faad41b5c30 con 0x563981f202f0 2024-05-27T08:01:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.194+0000 7faae3604740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faad41b3500 con 0x563981f284d0 2024-05-27T08:01:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.194+0000 7faae3604740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faad41b1370 con 0x563981f491b0 2024-05-27T08:01:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.194+0000 7faada600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f202f0 0x5639822f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.194+0000 7faada600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f202f0 0x5639822f8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46294/0 (socket says 172.21.15.63:46294) 2024-05-27T08:01:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.194+0000 7faada600640 1 -- 172.21.15.63:0/2191636370 learned_addr learned my addr 172.21.15.63:0/2191636370 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.195+0000 7faad9200640 1 --2- 172.21.15.63:0/2191636370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.195+0000 7faada600640 1 --2- 172.21.15.63:0/2191636370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f491b0 0x5639822e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.195+0000 7faada600640 1 -- 172.21.15.63:0/2191636370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f491b0 msgr2=0x5639822e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:33.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.195+0000 7faada600640 1 --2- 172.21.15.63:0/2191636370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f491b0 0x5639822e7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.195+0000 7faada600640 1 -- 172.21.15.63:0/2191636370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 msgr2=0x5639822e80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.195+0000 7faada600640 1 --2- 172.21.15.63:0/2191636370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822e80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.195+0000 7faada600640 1 -- 172.21.15.63:0/2191636370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639822efee0 con 0x563981f202f0 2024-05-27T08:01:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.196+0000 7faada600640 1 --2- 172.21.15.63:0/2191636370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f202f0 0x5639822f8100 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7faacc007dd0 tx=0x7faacc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43453963de9a793 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.196+0000 7faac6000640 1 -- 172.21.15.63:0/2191636370 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc014040 con 0x563981f202f0 2024-05-27T08:01:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.196+0000 7faac6000640 1 -- 172.21.15.63:0/2191636370 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faacc004ce0 con 0x563981f202f0 2024-05-27T08:01:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.196+0000 7faac6000640 1 -- 172.21.15.63:0/2191636370 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc005550 con 0x563981f202f0 2024-05-27T08:01:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.197+0000 7faae3604740 1 -- 172.21.15.63:0/2191636370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f202f0 msgr2=0x5639822f8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.197+0000 7faae3604740 1 --2- 172.21.15.63:0/2191636370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f202f0 0x5639822f8100 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7faacc007dd0 tx=0x7faacc00e720 comp rx=0 tx=0).stop 2024-05-27T08:01:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.197+0000 7faae3604740 1 -- 172.21.15.63:0/2191636370 shutdown_connections 2024-05-27T08:01:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.197+0000 7faae3604740 1 --2- 172.21.15.63:0/2191636370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f491b0 0x5639822e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.197+0000 7faae3604740 1 --2- 172.21.15.63:0/2191636370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.197+0000 7faae3604740 1 --2- 172.21.15.63:0/2191636370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f202f0 0x5639822f8100 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.197+0000 7faae3604740 1 -- 172.21.15.63:0/2191636370 >> 172.21.15.63:0/2191636370 conn(0x563981f3d300 msgr2=0x5639822e79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.198+0000 7faae3604740 1 -- 172.21.15.63:0/2191636370 shutdown_connections 2024-05-27T08:01:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.198+0000 7faae3604740 1 -- 172.21.15.63:0/2191636370 wait complete. 2024-05-27T08:01:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.198+0000 7faae3604740 1 -- start start 2024-05-27T08:01:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.199+0000 7faae3604740 1 Processor -- start 2024-05-27T08:01:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.199+0000 7faae3604740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.199+0000 7faae3604740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822f9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.199+0000 7faae3604740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.199+0000 7faae3604740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faad41b19c0 con 0x563981f491b0 2024-05-27T08:01:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.199+0000 7faae3604740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faad41b2e30 con 0x563981f284d0 2024-05-27T08:01:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.199+0000 7faae3604740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faad41b2090 con 0x563981f202f0 2024-05-27T08:01:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.200+0000 7faae3604740 10 client.0 _ll_register_callbacks cb 0x7faac000ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.200+0000 7faada600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.200+0000 7faada600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822f96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46308/0 (socket says 172.21.15.63:46308) 2024-05-27T08:01:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.200+0000 7faada600640 1 -- 172.21.15.63:0/2001776528 learned_addr learned my addr 172.21.15.63:0/2001776528 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.200+0000 7faad9200640 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.200+0000 7faada600640 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.201+0000 7faada600640 1 -- 172.21.15.63:0/2001776528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 msgr2=0x5639822f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.201+0000 7faada600640 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822f8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.201+0000 7faada600640 1 -- 172.21.15.63:0/2001776528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 msgr2=0x5639822f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.201+0000 7faada600640 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.201+0000 7faada600640 1 -- 172.21.15.63:0/2001776528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56398236e600 con 0x563981f491b0 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.201+0000 7faada600640 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822f96e0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x56398230db70 tx=0x7faacc0076e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89aaa5f51e42b093 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.201+0000 7faabb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.201+0000 7faabb400640 1 -- 172.21.15.63:0/2001776528 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc014030 con 0x563981f491b0 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.201+0000 7faabb400640 1 -- 172.21.15.63:0/2001776528 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7faacc03c070 con 0x563981f491b0 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.201+0000 7faae3604740 1 -- 172.21.15.63:0/2001776528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faacc004ce0 con 0x563981f491b0 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.202+0000 7faabb400640 1 -- 172.21.15.63:0/2001776528 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faacc005470 con 0x563981f491b0 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.202+0000 7faae3604740 10 client.20805 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.202+0000 7faae3604740 1 -- 172.21.15.63:0/2001776528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56398236f890 con 0x563981f491b0 2024-05-27T08:01:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.202+0000 7faabb400640 1 -- 172.21.15.63:0/2001776528 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faacc004ce0 con 0x563981f491b0 2024-05-27T08:01:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.203+0000 7faabb400640 1 -- 172.21.15.63:0/2001776528 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7faacc028710 con 0x563981f491b0 2024-05-27T08:01:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.203+0000 7faae3604740 10 client.20805 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.203+0000 7faae3604740 10 client.20805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.203+0000 7faae3604740 20 client.20805 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.203+0000 7faae3604740 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563981edd1c0 0x563982371d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.203+0000 7faae3604740 4 client.20805 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:01:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.203+0000 7faae3604740 1 -- 172.21.15.63:0/2001776528 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639822582c0 con 0x563981edd1c0 2024-05-27T08:01:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.204+0000 7faad9200640 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563981edd1c0 0x563982371d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.205+0000 7faad9200640 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563981edd1c0 0x563982371d20 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c084f417af61de6 server_cookie=85b5856489bd513f in_seq=0 out_seq=0 2024-05-27T08:01:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.205+0000 7faabb400640 10 client.20805 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faabb400640 1 -- 172.21.15.63:0/2001776528 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5639822582c0 con 0x563981edd1c0 2024-05-27T08:01:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faabb400640 10 client.20805 handle_command_reply: tid=0 2024-05-27T08:01:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faae3604740 1 client.20805 shutdown 2024-05-27T08:01:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faae3604740 1 -- 172.21.15.63:0/2001776528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 msgr2=0x5639822f96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faae3604740 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822f96e0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x56398230db70 tx=0x7faacc0076e0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faae3604740 1 -- 172.21.15.63:0/2001776528 shutdown_connections 2024-05-27T08:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faae3604740 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563981edd1c0 0x563982371d20 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faae3604740 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563981f491b0 0x5639822f96e0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faae3604740 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563981f284d0 0x5639822f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faae3604740 1 --2- 172.21.15.63:0/2001776528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563981f202f0 0x5639822f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faabb400640 0 client.20805 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.243+0000 7faae3604740 1 -- 172.21.15.63:0/2001776528 >> 172.21.15.63:0/2001776528 conn(0x563981f3d300 msgr2=0x5639822e7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.244+0000 7faae3604740 1 -- 172.21.15.63:0/2001776528 shutdown_connections 2024-05-27T08:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.244+0000 7faae3604740 1 -- 172.21.15.63:0/2001776528 wait complete. 2024-05-27T08:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.244+0000 7faae3604740 20 client.20805 trim_cache size 0 max 16384 2024-05-27T08:01:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 -- 172.21.15.63:0/2373736469 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faabc0b29d0 msgr2=0x7faabc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faabc0b29d0 0x7faabc0b4ee0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x5639822e9350 tx=0x7faac802a480 comp rx=0 tx=0).stop 2024-05-27T08:01:33.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 -- 172.21.15.63:0/2373736469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad4075420 msgr2=0x7faad41a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad4075420 0x7faad41a2640 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7faac8009230 tx=0x7faac8004270 comp rx=0 tx=0).stop 2024-05-27T08:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 -- 172.21.15.63:0/2373736469 shutdown_connections 2024-05-27T08:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faabc0b29d0 0x7faabc0b4ee0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faad4110520 0x7faad419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faad410dbc0 0x7faad41a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 --2- 172.21.15.63:0/2373736469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad4075420 0x7faad41a2640 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 -- 172.21.15.63:0/2373736469 >> 172.21.15.63:0/2373736469 conn(0x7faad4079440 msgr2=0x7faad41023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:33.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 -- 172.21.15.63:0/2373736469 shutdown_connections 2024-05-27T08:01:33.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:33.245+0000 7faadc600640 1 -- 172.21.15.63:0/2373736469 wait complete. 2024-05-27T08:01:33.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:33 smithi086 ceph-mon[34687]: pgmap v887: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-27T08:01:33.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:33 smithi149 ceph-mon[35408]: pgmap v887: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-27T08:01:33.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:33 smithi063 ceph-mon[31613]: pgmap v887: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-27T08:01:35.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:35 smithi086 ceph-mon[34687]: pgmap v888: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:01:35.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:35 smithi149 ceph-mon[35408]: pgmap v888: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:01:35.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:35 smithi063 ceph-mon[31613]: pgmap v888: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:01:37.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:37 smithi086 ceph-mon[34687]: pgmap v889: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:01:37.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:37 smithi149 ceph-mon[35408]: pgmap v889: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:01:37.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:37 smithi063 ceph-mon[31613]: pgmap v889: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:01:39.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:39 smithi086 ceph-mon[34687]: pgmap v890: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:01:39.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:39 smithi149 ceph-mon[35408]: pgmap v890: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:01:39.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:39 smithi063 ceph-mon[31613]: pgmap v890: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:01:41.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:41 smithi086 ceph-mon[34687]: pgmap v891: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:01:41.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:41 smithi149 ceph-mon[35408]: pgmap v891: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:01:41.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:41 smithi063 ceph-mon[31613]: pgmap v891: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:01:43.679 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:43 smithi063 ceph-mon[31613]: pgmap v892: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:01:43.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:43 smithi086 ceph-mon[34687]: pgmap v892: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:01:43.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:43 smithi149 ceph-mon[35408]: pgmap v892: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:01:45.319 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:01:45.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:45 smithi086 ceph-mon[34687]: pgmap v893: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:01:45.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:45 smithi149 ceph-mon[35408]: pgmap v893: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:01:45.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:45 smithi063 ceph-mon[31613]: pgmap v893: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:01:45.850 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.507+0000 7f7aaae00640 1 -- 172.21.15.63:0/4128542781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aa4104bb0 msgr2=0x7f7aa4104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.507+0000 7f7aaae00640 1 --2- 172.21.15.63:0/4128542781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aa4104bb0 0x7f7aa4104fc0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009900 tx=0x7f7a9802a3f0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.507+0000 7f7aaae00640 1 -- 172.21.15.63:0/4128542781 shutdown_connections 2024-05-27T08:01:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.507+0000 7f7aaae00640 1 --2- 172.21.15.63:0/4128542781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa410e000 0x7f7aa41104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.507+0000 7f7aaae00640 1 --2- 172.21.15.63:0/4128542781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7aa410b610 0x7f7aa410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.507+0000 7f7aaae00640 1 --2- 172.21.15.63:0/4128542781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aa4104bb0 0x7f7aa4104fc0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.507+0000 7f7aaae00640 1 -- 172.21.15.63:0/4128542781 >> 172.21.15.63:0/4128542781 conn(0x7f7aa40fe210 msgr2=0x7f7aa4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.507+0000 7f7aaae00640 1 -- 172.21.15.63:0/4128542781 shutdown_connections 2024-05-27T08:01:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.507+0000 7f7aaae00640 1 -- 172.21.15.63:0/4128542781 wait complete. 2024-05-27T08:01:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.509+0000 7f7aaae00640 1 Processor -- start 2024-05-27T08:01:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.509+0000 7f7aaae00640 1 -- start start 2024-05-27T08:01:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.509+0000 7f7aaae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa4104bb0 0x7f7aa419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.510+0000 7f7aaae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7aa410b610 0x7f7aa419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.510+0000 7f7aaae00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aa410e000 0x7f7aa41a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.510+0000 7f7aaae00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4115d60 con 0x7f7aa410e000 2024-05-27T08:01:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.510+0000 7f7aaae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4115be0 con 0x7f7aa4104bb0 2024-05-27T08:01:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.510+0000 7f7aaae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4115ee0 con 0x7f7aa410b610 2024-05-27T08:01:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.510+0000 7f7aaa400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aa410e000 0x7f7aa41a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.510+0000 7f7aaa400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aa410e000 0x7f7aa41a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56680/0 (socket says 172.21.15.63:56680) 2024-05-27T08:01:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.510+0000 7f7aaa400640 1 -- 172.21.15.63:0/4127087685 learned_addr learned my addr 172.21.15.63:0/4127087685 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.510+0000 7f7aa9000640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7aa410b610 0x7f7aa419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.510+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa4104bb0 0x7f7aa419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.511+0000 7f7aaa400640 1 -- 172.21.15.63:0/4127087685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7aa410b610 msgr2=0x7f7aa419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.511+0000 7f7aaa400640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7aa410b610 0x7f7aa419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.511+0000 7f7aaa400640 1 -- 172.21.15.63:0/4127087685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa4104bb0 msgr2=0x7f7aa419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.511+0000 7f7aaa400640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa4104bb0 0x7f7aa419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.511+0000 7f7aaa400640 1 -- 172.21.15.63:0/4127087685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa41a1600 con 0x7f7aa410e000 2024-05-27T08:01:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.511+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa4104bb0 0x7f7aa419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.511+0000 7f7aaa400640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aa410e000 0x7f7aa41a0fc0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00eaa0 tx=0x7f7a9c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.512+0000 7f7aa2000640 1 -- 172.21.15.63:0/4127087685 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00cdb0 con 0x7f7aa410e000 2024-05-27T08:01:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.512+0000 7f7aa2000640 1 -- 172.21.15.63:0/4127087685 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a9c005020 con 0x7f7aa410e000 2024-05-27T08:01:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.512+0000 7f7aaae00640 1 -- 172.21.15.63:0/4127087685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa41a18f0 con 0x7f7aa410e000 2024-05-27T08:01:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.512+0000 7f7aa2000640 1 -- 172.21.15.63:0/4127087685 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0116d0 con 0x7f7aa410e000 2024-05-27T08:01:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.512+0000 7f7aaae00640 1 -- 172.21.15.63:0/4127087685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa41a9270 con 0x7f7aa410e000 2024-05-27T08:01:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.516+0000 7f7ab3421740 1 Processor -- start 2024-05-27T08:01:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.517+0000 7f7aa2000640 1 -- 172.21.15.63:0/4127087685 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7a9c0051d0 con 0x7f7aa410e000 2024-05-27T08:01:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.517+0000 7f7ab3421740 1 -- start start 2024-05-27T08:01:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93811b0 0x557ae9723a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae9723fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 0x557ae97244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7ab3421740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557ae9694b90 con 0x557ae93582f0 2024-05-27T08:01:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7ab3421740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ae9684730 con 0x557ae93604d0 2024-05-27T08:01:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7ab3421740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557ae9688770 con 0x557ae93811b0 2024-05-27T08:01:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7aa9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae9723fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7aa9000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93811b0 0x557ae9723a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7aa9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae9723fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56694/0 (socket says 172.21.15.63:56694) 2024-05-27T08:01:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7aa9a00640 1 -- 172.21.15.63:0/2740523672 learned_addr learned my addr 172.21.15.63:0/2740523672 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.518+0000 7f7aa9000640 1 --2- 172.21.15.63:0/2740523672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 0x557ae97244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.519+0000 7f7aa9a00640 1 -- 172.21.15.63:0/2740523672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93811b0 msgr2=0x557ae9723a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.519+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/2740523672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93811b0 0x557ae9723a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.519+0000 7f7aa9a00640 1 -- 172.21.15.63:0/2740523672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 msgr2=0x557ae97244e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.519+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/2740523672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 0x557ae97244e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.519+0000 7f7aa9a00640 1 -- 172.21.15.63:0/2740523672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae961d8b0 con 0x557ae93582f0 2024-05-27T08:01:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.519+0000 7f7aa9000640 1 --2- 172.21.15.63:0/2740523672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 0x557ae97244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.519+0000 7f7aa2000640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7a8c0b2a20 0x7f7a8c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.520+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/2740523672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae9723fa0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f7a98000c00 tx=0x7f7a98004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775bc4c4763930fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.520+0000 7f7aa2000640 1 -- 172.21.15.63:0/4127087685 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f7a9c0e2b40 con 0x7f7aa410e000 2024-05-27T08:01:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.520+0000 7f7aa9000640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7a8c0b2a20 0x7f7a8c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.520+0000 7f7a96a00640 1 -- 172.21.15.63:0/2740523672 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98035030 con 0x557ae93582f0 2024-05-27T08:01:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.520+0000 7f7a96a00640 1 -- 172.21.15.63:0/2740523672 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a980046c0 con 0x557ae93582f0 2024-05-27T08:01:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.520+0000 7f7a96a00640 1 -- 172.21.15.63:0/2740523672 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98003850 con 0x557ae93582f0 2024-05-27T08:01:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.522+0000 7f7ab3421740 1 -- 172.21.15.63:0/2740523672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 msgr2=0x557ae9723fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.522+0000 7f7aa9000640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7a8c0b2a20 0x7f7a8c0b4f30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x557ae9720bd0 tx=0x7f7a9000b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.522+0000 7f7ab3421740 1 --2- 172.21.15.63:0/2740523672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae9723fa0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f7a98000c00 tx=0x7f7a98004390 comp rx=0 tx=0).stop 2024-05-27T08:01:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.522+0000 7f7ab3421740 1 -- 172.21.15.63:0/2740523672 shutdown_connections 2024-05-27T08:01:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.522+0000 7f7ab3421740 1 --2- 172.21.15.63:0/2740523672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 0x557ae97244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.522+0000 7f7ab3421740 1 --2- 172.21.15.63:0/2740523672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae9723fa0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.522+0000 7f7ab3421740 1 --2- 172.21.15.63:0/2740523672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93811b0 0x557ae9723a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.522+0000 7f7ab3421740 1 -- 172.21.15.63:0/2740523672 >> 172.21.15.63:0/2740523672 conn(0x557ae9375300 msgr2=0x557ae9720210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.522+0000 7f7ab3421740 1 -- 172.21.15.63:0/2740523672 shutdown_connections 2024-05-27T08:01:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.522+0000 7f7ab3421740 1 -- 172.21.15.63:0/2740523672 wait complete. 2024-05-27T08:01:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.523+0000 7f7ab3421740 1 -- start start 2024-05-27T08:01:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.523+0000 7f7ab3421740 1 Processor -- start 2024-05-27T08:01:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.524+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93582f0 0x557ae972dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.524+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 0x557ae972e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.524+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93811b0 0x557ae9739e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.524+0000 7f7ab3421740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557ae936a790 con 0x557ae93811b0 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.524+0000 7f7ab3421740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ae962cbc0 con 0x557ae93582f0 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.524+0000 7f7ab3421740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557ae9730b30 con 0x557ae93604d0 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.524+0000 7f7ab3421740 10 client.0 _ll_register_callbacks cb 0x557ae960a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.524+0000 7f7aaa400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93811b0 0x557ae9739e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.524+0000 7f7aaa400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93811b0 0x557ae9739e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56698/0 (socket says 172.21.15.63:56698) 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.524+0000 7f7aaa400640 1 -- 172.21.15.63:0/161040677 learned_addr learned my addr 172.21.15.63:0/161040677 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.525+0000 7f7aa9000640 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 0x557ae972e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.525+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93582f0 0x557ae972dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.525+0000 7f7aaa400640 1 -- 172.21.15.63:0/161040677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 msgr2=0x557ae972e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.525+0000 7f7aaa400640 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 0x557ae972e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.525+0000 7f7aaa400640 1 -- 172.21.15.63:0/161040677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93582f0 msgr2=0x557ae972dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.525+0000 7f7aaa400640 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93582f0 0x557ae972dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.525+0000 7f7aaa400640 1 -- 172.21.15.63:0/161040677 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae9399210 con 0x557ae93811b0 2024-05-27T08:01:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.525+0000 7f7aa9000640 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 0x557ae972e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.525+0000 7f7aaa400640 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93811b0 0x557ae9739e60 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0a5a80 tx=0x7f7a9c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856c07eed94baad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.526+0000 7f7a8b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.526+0000 7f7a8b400640 1 -- 172.21.15.63:0/161040677 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c01f040 con 0x557ae93811b0 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.526+0000 7f7a8b400640 1 -- 172.21.15.63:0/161040677 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f7a9c0cc0d0 con 0x557ae93811b0 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.526+0000 7f7ab3421740 1 -- 172.21.15.63:0/161040677 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a980046c0 con 0x557ae93811b0 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.526+0000 7f7a8b400640 1 -- 172.21.15.63:0/161040677 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c0addb0 con 0x557ae93811b0 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.526+0000 7f7ab3421740 10 client.20817 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.526+0000 7f7ab3421740 1 -- 172.21.15.63:0/161040677 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557ae9465620 con 0x557ae93811b0 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.527+0000 7f7a8b400640 1 -- 172.21.15.63:0/161040677 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a980046c0 con 0x557ae93811b0 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.527+0000 7f7a8b400640 1 -- 172.21.15.63:0/161040677 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7a9c0c5030 con 0x557ae93811b0 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.527+0000 7f7ab3421740 10 client.20817 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.527+0000 7f7ab3421740 10 client.20817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.527+0000 7f7ab3421740 20 client.20817 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.527+0000 7f7ab3421740 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ae933fd80 0x557ae9748720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.527+0000 7f7ab3421740 4 client.20817 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.527+0000 7f7ab3421740 1 -- 172.21.15.63:0/161040677 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ae9696ae0 con 0x557ae933fd80 2024-05-27T08:01:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.528+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ae933fd80 0x557ae9748720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.529+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ae933fd80 0x557ae9748720 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9709aa039af9f4d server_cookie=4edff747684868ac in_seq=0 out_seq=0 2024-05-27T08:01:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.529+0000 7f7a8b400640 10 client.20817 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.531+0000 7f7a8b400640 1 -- 172.21.15.63:0/161040677 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557ae9696ae0 con 0x557ae933fd80 2024-05-27T08:01:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.531+0000 7f7a8b400640 10 client.20817 handle_command_reply: tid=0 2024-05-27T08:01:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.531+0000 7f7ab3421740 1 client.20817 shutdown 2024-05-27T08:01:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.531+0000 7f7ab3421740 1 -- 172.21.15.63:0/161040677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93811b0 msgr2=0x557ae9739e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.531+0000 7f7ab3421740 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93811b0 0x557ae9739e60 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0a5a80 tx=0x7f7a9c005720 comp rx=0 tx=0).stop 2024-05-27T08:01:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.532+0000 7f7ab3421740 1 -- 172.21.15.63:0/161040677 shutdown_connections 2024-05-27T08:01:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.532+0000 7f7ab3421740 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ae933fd80 0x557ae9748720 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.532+0000 7f7ab3421740 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93811b0 0x557ae9739e60 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.532+0000 7f7ab3421740 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 0x557ae972e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.532+0000 7f7ab3421740 1 --2- 172.21.15.63:0/161040677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93582f0 0x557ae972dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.532+0000 7f7ab3421740 1 -- 172.21.15.63:0/161040677 >> 172.21.15.63:0/161040677 conn(0x557ae9375300 msgr2=0x557ae9723890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.532+0000 7f7a8b400640 0 client.20817 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.532+0000 7f7ab3421740 1 -- 172.21.15.63:0/161040677 shutdown_connections 2024-05-27T08:01:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.532+0000 7f7ab3421740 1 -- 172.21.15.63:0/161040677 wait complete. 2024-05-27T08:01:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.532+0000 7f7ab3421740 20 client.20817 trim_cache size 0 max 16384 2024-05-27T08:01:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.557+0000 7f7ab3421740 1 Processor -- start 2024-05-27T08:01:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.557+0000 7f7ab3421740 1 -- start start 2024-05-27T08:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.557+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97294e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.557+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 0x557ae972fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.557+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93811b0 0x557ae9730020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.557+0000 7f7ab3421740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa41b1780 con 0x557ae93582f0 2024-05-27T08:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.557+0000 7f7ab3421740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa41af050 con 0x557ae93604d0 2024-05-27T08:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.557+0000 7f7ab3421740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa41acee0 con 0x557ae93811b0 2024-05-27T08:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.557+0000 7f7aa9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97294e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.558+0000 7f7aa9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97294e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56710/0 (socket says 172.21.15.63:56710) 2024-05-27T08:01:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.558+0000 7f7aa9a00640 1 -- 172.21.15.63:0/3051540943 learned_addr learned my addr 172.21.15.63:0/3051540943 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.558+0000 7f7aaa400640 1 --2- 172.21.15.63:0/3051540943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93811b0 0x557ae9730020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.558+0000 7f7aa9000640 1 --2- 172.21.15.63:0/3051540943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 0x557ae972fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.558+0000 7f7aa9a00640 1 -- 172.21.15.63:0/3051540943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93811b0 msgr2=0x557ae9730020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.558+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/3051540943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93811b0 0x557ae9730020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.558+0000 7f7aa9a00640 1 -- 172.21.15.63:0/3051540943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 msgr2=0x557ae972fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.558+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/3051540943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 0x557ae972fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.558+0000 7f7aa9a00640 1 -- 172.21.15.63:0/3051540943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae97211f0 con 0x557ae93582f0 2024-05-27T08:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.558+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/3051540943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97294e0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f7a9802a9e0 tx=0x7f7a980097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4933318adb278337 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.559+0000 7f7a96000640 1 -- 172.21.15.63:0/3051540943 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98054020 con 0x557ae93582f0 2024-05-27T08:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.559+0000 7f7a96000640 1 -- 172.21.15.63:0/3051540943 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a9803bac0 con 0x557ae93582f0 2024-05-27T08:01:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.559+0000 7f7a96000640 1 -- 172.21.15.63:0/3051540943 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98035040 con 0x557ae93582f0 2024-05-27T08:01:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.560+0000 7f7ab3421740 1 -- 172.21.15.63:0/3051540943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 msgr2=0x557ae97294e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.560+0000 7f7ab3421740 1 --2- 172.21.15.63:0/3051540943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97294e0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f7a9802a9e0 tx=0x7f7a980097c0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.560+0000 7f7ab3421740 1 -- 172.21.15.63:0/3051540943 shutdown_connections 2024-05-27T08:01:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.560+0000 7f7ab3421740 1 --2- 172.21.15.63:0/3051540943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93811b0 0x557ae9730020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.560+0000 7f7ab3421740 1 --2- 172.21.15.63:0/3051540943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93604d0 0x557ae972fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.560+0000 7f7ab3421740 1 --2- 172.21.15.63:0/3051540943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97294e0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.560+0000 7f7ab3421740 1 -- 172.21.15.63:0/3051540943 >> 172.21.15.63:0/3051540943 conn(0x557ae9375300 msgr2=0x557ae972f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.561+0000 7f7ab3421740 1 -- 172.21.15.63:0/3051540943 shutdown_connections 2024-05-27T08:01:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.561+0000 7f7ab3421740 1 -- 172.21.15.63:0/3051540943 wait complete. 2024-05-27T08:01:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.561+0000 7f7ab3421740 1 -- start start 2024-05-27T08:01:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.561+0000 7f7ab3421740 1 Processor -- start 2024-05-27T08:01:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.562+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.562+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 0x557ae97a8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.562+0000 7f7ab3421740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93811b0 0x557ae9733310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.562+0000 7f7ab3421740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa41ad530 con 0x557ae93582f0 2024-05-27T08:01:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.562+0000 7f7ab3421740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa41ae980 con 0x557ae93811b0 2024-05-27T08:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.562+0000 7f7ab3421740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa41adbe0 con 0x557ae93604d0 2024-05-27T08:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.562+0000 7f7ab3421740 10 client.0 _ll_register_callbacks cb 0x7f7a9c0a5940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.563+0000 7f7aa9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.563+0000 7f7aa9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97a8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56722/0 (socket says 172.21.15.63:56722) 2024-05-27T08:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.563+0000 7f7aa9a00640 1 -- 172.21.15.63:0/724057117 learned_addr learned my addr 172.21.15.63:0/724057117 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.563+0000 7f7aaa400640 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93811b0 0x557ae9733310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.563+0000 7f7aa9000640 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 0x557ae97a8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.563+0000 7f7aa9a00640 1 -- 172.21.15.63:0/724057117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 msgr2=0x557ae97a8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.563+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 0x557ae97a8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.563+0000 7f7aa9a00640 1 -- 172.21.15.63:0/724057117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93811b0 msgr2=0x557ae9733310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.563+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93811b0 0x557ae9733310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.563+0000 7f7aa9a00640 1 -- 172.21.15.63:0/724057117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae97339d0 con 0x557ae93582f0 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.564+0000 7f7aa9a00640 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97a8110 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f7a98004550 tx=0x7f7a98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e8fa7863878ba0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.564+0000 7f7a8b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.564+0000 7f7a8b400640 1 -- 172.21.15.63:0/724057117 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98054030 con 0x557ae93582f0 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.564+0000 7f7ab3421740 1 -- 172.21.15.63:0/724057117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a9803bac0 con 0x557ae93582f0 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.564+0000 7f7a8b400640 1 -- 172.21.15.63:0/724057117 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f7a98059420 con 0x557ae93582f0 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.565+0000 7f7a8b400640 1 -- 172.21.15.63:0/724057117 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7a98068050 con 0x557ae93582f0 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.565+0000 7f7ab3421740 10 client.20823 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.565+0000 7f7ab3421740 1 -- 172.21.15.63:0/724057117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557ae97b0770 con 0x557ae93582f0 2024-05-27T08:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.565+0000 7f7a8b400640 1 -- 172.21.15.63:0/724057117 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a9803bac0 con 0x557ae93582f0 2024-05-27T08:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.565+0000 7f7a8b400640 1 -- 172.21.15.63:0/724057117 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7a980038f0 con 0x557ae93582f0 2024-05-27T08:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.565+0000 7f7ab3421740 10 client.20823 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.565+0000 7f7ab3421740 10 client.20823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.566+0000 7f7ab3421740 20 client.20823 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.566+0000 7f7ab3421740 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ae93151c0 0x557ae97b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.566+0000 7f7ab3421740 4 client.20823 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:01:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.566+0000 7f7ab3421740 1 -- 172.21.15.63:0/724057117 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ae96902c0 con 0x557ae93151c0 2024-05-27T08:01:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.566+0000 7f7aa9000640 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ae93151c0 0x557ae97b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.567+0000 7f7aa9000640 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ae93151c0 0x557ae97b29b0 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d2a984aba3fccd8 server_cookie=6721a778086e82b8 in_seq=0 out_seq=0 2024-05-27T08:01:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.567+0000 7f7a8b400640 10 client.20823 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7a8b400640 1 -- 172.21.15.63:0/724057117 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557ae96902c0 con 0x557ae93151c0 2024-05-27T08:01:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7a8b400640 10 client.20823 handle_command_reply: tid=0 2024-05-27T08:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7ab3421740 1 client.20823 shutdown 2024-05-27T08:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7ab3421740 1 -- 172.21.15.63:0/724057117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 msgr2=0x557ae97a8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7ab3421740 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97a8110 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f7a98004550 tx=0x7f7a98004730 comp rx=0 tx=0).stop 2024-05-27T08:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7ab3421740 1 -- 172.21.15.63:0/724057117 shutdown_connections 2024-05-27T08:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7ab3421740 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ae93151c0 0x557ae97b29b0 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7ab3421740 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae93811b0 0x557ae9733310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7ab3421740 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ae93604d0 0x557ae97a8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7a8b400640 0 client.20823 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7ab3421740 1 --2- 172.21.15.63:0/724057117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ae93582f0 0x557ae97a8110 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.605+0000 7f7ab3421740 1 -- 172.21.15.63:0/724057117 >> 172.21.15.63:0/724057117 conn(0x557ae9375300 msgr2=0x557ae972fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.606+0000 7f7ab3421740 1 -- 172.21.15.63:0/724057117 shutdown_connections 2024-05-27T08:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.606+0000 7f7ab3421740 1 -- 172.21.15.63:0/724057117 wait complete. 2024-05-27T08:01:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.606+0000 7f7ab3421740 20 client.20823 trim_cache size 0 max 16384 2024-05-27T08:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:01:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 -- 172.21.15.63:0/4127087685 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7a8c0b2a20 msgr2=0x7f7a8c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7a8c0b2a20 0x7f7a8c0b4f30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x557ae9720bd0 tx=0x7f7a9000b390 comp rx=0 tx=0).stop 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 -- 172.21.15.63:0/4127087685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aa410e000 msgr2=0x7f7aa41a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aa410e000 0x7f7aa41a0fc0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00eaa0 tx=0x7f7a9c00ef60 comp rx=0 tx=0).stop 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 -- 172.21.15.63:0/4127087685 shutdown_connections 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7a8c0b2a20 0x7f7a8c0b4f30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7aa410e000 0x7f7aa41a0fc0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7aa410b610 0x7f7aa419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 --2- 172.21.15.63:0/4127087685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa4104bb0 0x7f7aa419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 -- 172.21.15.63:0/4127087685 >> 172.21.15.63:0/4127087685 conn(0x7f7aa40fe210 msgr2=0x7f7aa410e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 -- 172.21.15.63:0/4127087685 shutdown_connections 2024-05-27T08:01:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:46.607+0000 7f7aaae00640 1 -- 172.21.15.63:0/4127087685 wait complete. 2024-05-27T08:01:47.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:47 smithi086 ceph-mon[34687]: pgmap v894: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:01:47.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:47 smithi149 ceph-mon[35408]: pgmap v894: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:01:47.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:47 smithi063 ceph-mon[31613]: pgmap v894: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:01:48.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.433+0000 7f1859800700 1 -- 172.21.15.63:0/3991133182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18541045d0 msgr2=0x7f18541049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.433+0000 7f1859800700 1 --2- 172.21.15.63:0/3991133182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18541045d0 0x7f18541049e0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f18440099c0 tx=0x7f1844009cd0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.434+0000 7f1859800700 1 -- 172.21.15.63:0/3991133182 shutdown_connections 2024-05-27T08:01:48.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.434+0000 7f1859800700 1 --2- 172.21.15.63:0/3991133182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1854109850 0x7f18541111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.434+0000 7f1859800700 1 --2- 172.21.15.63:0/3991133182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18541045d0 0x7f18541049e0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.434+0000 7f1859800700 1 --2- 172.21.15.63:0/3991133182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1854108f00 0x7f1854109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.434+0000 7f1859800700 1 -- 172.21.15.63:0/3991133182 >> 172.21.15.63:0/3991133182 conn(0x7f18540ffe60 msgr2=0x7f18541022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:48.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.434+0000 7f1859800700 1 -- 172.21.15.63:0/3991133182 shutdown_connections 2024-05-27T08:01:48.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.434+0000 7f1859800700 1 -- 172.21.15.63:0/3991133182 wait complete. 2024-05-27T08:01:48.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.435+0000 7f1859800700 1 Processor -- start 2024-05-27T08:01:48.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.436+0000 7f1859800700 1 -- start start 2024-05-27T08:01:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.436+0000 7f1859800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18541045d0 0x7f1854075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.436+0000 7f1859800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1854108f00 0x7f1854075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.436+0000 7f1859800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1854109850 0x7f1854076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.436+0000 7f1859800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1854114380 con 0x7f18541045d0 2024-05-27T08:01:48.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.436+0000 7f1859800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1854114200 con 0x7f1854108f00 2024-05-27T08:01:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.436+0000 7f1859800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1854114500 con 0x7f1854109850 2024-05-27T08:01:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.437+0000 7f1852a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18541045d0 0x7f1854075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.437+0000 7f1852a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18541045d0 0x7f1854075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56740/0 (socket says 172.21.15.63:56740) 2024-05-27T08:01:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.437+0000 7f1852a00700 1 -- 172.21.15.63:0/3905656501 learned_addr learned my addr 172.21.15.63:0/3905656501 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:48.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.437+0000 7f1852000700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1854108f00 0x7f1854075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.437+0000 7f1852a00700 1 -- 172.21.15.63:0/3905656501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1854109850 msgr2=0x7f1854076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.437+0000 7f1852a00700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1854109850 0x7f1854076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.437+0000 7f1852a00700 1 -- 172.21.15.63:0/3905656501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1854108f00 msgr2=0x7f1854075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.437+0000 7f1852a00700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1854108f00 0x7f1854075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.438+0000 7f1852a00700 1 -- 172.21.15.63:0/3905656501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18541a90e0 con 0x7f18541045d0 2024-05-27T08:01:48.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.438+0000 7f1852a00700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18541045d0 0x7f1854075ab0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f184400b580 tx=0x7f18440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.439+0000 7f184b400700 1 -- 172.21.15.63:0/3905656501 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844027070 con 0x7f18541045d0 2024-05-27T08:01:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.439+0000 7f1859800700 1 -- 172.21.15.63:0/3905656501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18541a9370 con 0x7f18541045d0 2024-05-27T08:01:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.439+0000 7f184b400700 1 -- 172.21.15.63:0/3905656501 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f184400fd30 con 0x7f18541045d0 2024-05-27T08:01:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.439+0000 7f184b400700 1 -- 172.21.15.63:0/3905656501 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844022e50 con 0x7f18541045d0 2024-05-27T08:01:48.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.439+0000 7f1859800700 1 -- 172.21.15.63:0/3905656501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18541a9820 con 0x7f18541045d0 2024-05-27T08:01:48.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.445+0000 7f184b400700 1 -- 172.21.15.63:0/3905656501 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f184402a070 con 0x7f18541045d0 2024-05-27T08:01:48.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.447+0000 7f18683e3b80 1 Processor -- start 2024-05-27T08:01:48.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.447+0000 7f18683e3b80 1 -- start start 2024-05-27T08:01:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.448+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d58f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.448+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d410e40 0x55cb2d58f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.448+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.448+0000 7f18683e3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cb2d4bde40 con 0x55cb2d58fe70 2024-05-27T08:01:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.448+0000 7f18683e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb2d4ad9e0 con 0x55cb2d4fc9f0 2024-05-27T08:01:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.448+0000 7f18683e3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cb2d4b1a20 con 0x55cb2d410e40 2024-05-27T08:01:48.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.449+0000 7f1852000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.449+0000 7f1852000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56744/0 (socket says 172.21.15.63:56744) 2024-05-27T08:01:48.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.449+0000 7f1852000700 1 -- 172.21.15.63:0/3083413035 learned_addr learned my addr 172.21.15.63:0/3083413035 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:48.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.449+0000 7f1853400700 1 --2- 172.21.15.63:0/3083413035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d58f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.449+0000 7f184b400700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18380b2c10 0x7f18380b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.450+0000 7f1852000700 1 -- 172.21.15.63:0/3083413035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d410e40 msgr2=0x55cb2d58f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.450+0000 7f184b400700 1 -- 172.21.15.63:0/3905656501 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f18440e3660 con 0x7f18541045d0 2024-05-27T08:01:48.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.450+0000 7f1852000700 1 --2- 172.21.15.63:0/3083413035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d410e40 0x55cb2d58f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.450+0000 7f1852000700 1 -- 172.21.15.63:0/3083413035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d4fc9f0 msgr2=0x55cb2d58f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.450+0000 7f1852000700 1 --2- 172.21.15.63:0/3083413035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d58f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.450+0000 7f1852000700 1 -- 172.21.15.63:0/3083413035 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2d476420 con 0x55cb2d58fe70 2024-05-27T08:01:48.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.450+0000 7f1853400700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18380b2c10 0x7f18380b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.450+0000 7f1852000700 1 --2- 172.21.15.63:0/3083413035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59c110 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f184000bcf0 tx=0x7f18400103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6b4f9394bed22c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.451+0000 7f183fe00700 1 -- 172.21.15.63:0/3083413035 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840010d10 con 0x55cb2d58fe70 2024-05-27T08:01:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.451+0000 7f183fe00700 1 -- 172.21.15.63:0/3083413035 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18400044c0 con 0x55cb2d58fe70 2024-05-27T08:01:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.451+0000 7f183fe00700 1 -- 172.21.15.63:0/3083413035 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840005260 con 0x55cb2d58fe70 2024-05-27T08:01:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.452+0000 7f18683e3b80 1 -- 172.21.15.63:0/3083413035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 msgr2=0x55cb2d59c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.452+0000 7f18683e3b80 1 --2- 172.21.15.63:0/3083413035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59c110 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f184000bcf0 tx=0x7f18400103d0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.452+0000 7f1853400700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18380b2c10 0x7f18380b5120 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f184c0060b0 tx=0x7f184c009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.452+0000 7f18683e3b80 1 -- 172.21.15.63:0/3083413035 shutdown_connections 2024-05-27T08:01:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.452+0000 7f18683e3b80 1 --2- 172.21.15.63:0/3083413035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59c110 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.452+0000 7f18683e3b80 1 --2- 172.21.15.63:0/3083413035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d58f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.452+0000 7f18683e3b80 1 --2- 172.21.15.63:0/3083413035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d410e40 0x55cb2d58f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.452+0000 7f18683e3b80 1 -- 172.21.15.63:0/3083413035 >> 172.21.15.63:0/3083413035 conn(0x55cb2d4aa390 msgr2=0x55cb2d58b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:48.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.452+0000 7f18683e3b80 1 -- 172.21.15.63:0/3083413035 shutdown_connections 2024-05-27T08:01:48.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.453+0000 7f18683e3b80 1 -- 172.21.15.63:0/3083413035 wait complete. 2024-05-27T08:01:48.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.453+0000 7f18683e3b80 1 -- start start 2024-05-27T08:01:48.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.454+0000 7f18683e3b80 1 Processor -- start 2024-05-27T08:01:48.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.454+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d410e40 0x55cb2d599670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.454+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d599bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.454+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.454+0000 7f18683e3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cb2d3b3fa0 con 0x55cb2d58fe70 2024-05-27T08:01:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.454+0000 7f18683e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb2d44e760 con 0x55cb2d410e40 2024-05-27T08:01:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.454+0000 7f18683e3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cb2d59cbd0 con 0x55cb2d4fc9f0 2024-05-27T08:01:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.454+0000 7f18683e3b80 10 client.0 _ll_register_callbacks cb 0x55cb2d457c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:48.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.455+0000 7f1853400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.455+0000 7f1852000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d599bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.455+0000 7f1852000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d599bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:60360/0 (socket says 172.21.15.63:60360) 2024-05-27T08:01:48.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.455+0000 7f1852000700 1 -- 172.21.15.63:0/3086922782 learned_addr learned my addr 172.21.15.63:0/3086922782 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:48.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.455+0000 7f1852a00700 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d410e40 0x55cb2d599670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.456+0000 7f1853400700 1 -- 172.21.15.63:0/3086922782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 msgr2=0x55cb2d599bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.456+0000 7f1853400700 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d599bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.456+0000 7f1853400700 1 -- 172.21.15.63:0/3086922782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d410e40 msgr2=0x55cb2d599670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.456+0000 7f1853400700 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d410e40 0x55cb2d599670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.456+0000 7f1853400700 1 -- 172.21.15.63:0/3086922782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb2d3fd8d0 con 0x55cb2d58fe70 2024-05-27T08:01:48.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.456+0000 7f1853400700 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59a0f0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f184c009ba0 tx=0x7f184c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2293e7adae8689e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.456+0000 7f183cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.457+0000 7f183cc00700 1 -- 172.21.15.63:0/3086922782 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c025a10 con 0x55cb2d58fe70 2024-05-27T08:01:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.457+0000 7f183cc00700 1 -- 172.21.15.63:0/3086922782 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f184c025bb0 con 0x55cb2d58fe70 2024-05-27T08:01:48.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.457+0000 7f18683e3b80 1 -- 172.21.15.63:0/3086922782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18400044c0 con 0x55cb2d58fe70 2024-05-27T08:01:48.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.457+0000 7f183cc00700 1 -- 172.21.15.63:0/3086922782 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f184c036050 con 0x55cb2d58fe70 2024-05-27T08:01:48.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.457+0000 7f18683e3b80 10 client.20835 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:48.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.457+0000 7f18683e3b80 1 -- 172.21.15.63:0/3086922782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cb2d47e060 con 0x55cb2d58fe70 2024-05-27T08:01:48.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.458+0000 7f183cc00700 1 -- 172.21.15.63:0/3086922782 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18400044c0 con 0x55cb2d58fe70 2024-05-27T08:01:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.458+0000 7f183cc00700 1 -- 172.21.15.63:0/3086922782 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f184c02d3d0 con 0x55cb2d58fe70 2024-05-27T08:01:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.458+0000 7f18683e3b80 10 client.20835 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.458+0000 7f18683e3b80 10 client.20835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.458+0000 7f18683e3b80 20 client.20835 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.458+0000 7f18683e3b80 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb2d5a9bf0 0x55cb2d5ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.458+0000 7f18683e3b80 4 client.20835 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:48.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.458+0000 7f18683e3b80 1 -- 172.21.15.63:0/3086922782 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb2d4bfd90 con 0x55cb2d5a9bf0 2024-05-27T08:01:48.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.459+0000 7f1852a00700 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb2d5a9bf0 0x55cb2d5ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.460+0000 7f1852a00700 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb2d5a9bf0 0x55cb2d5ac010 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622fc76cd4ddd7d2 server_cookie=fe8bffccb861af7f in_seq=0 out_seq=0 2024-05-27T08:01:48.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.460+0000 7f183cc00700 10 client.20835 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:48.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f183cc00700 1 -- 172.21.15.63:0/3086922782 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cb2d4bfd90 con 0x55cb2d5a9bf0 2024-05-27T08:01:48.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f183cc00700 10 client.20835 handle_command_reply: tid=0 2024-05-27T08:01:48.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f18683e3b80 1 client.20835 shutdown 2024-05-27T08:01:48.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f18683e3b80 1 -- 172.21.15.63:0/3086922782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 msgr2=0x55cb2d59a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f18683e3b80 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59a0f0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f184c009ba0 tx=0x7f184c0091d0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f18683e3b80 1 -- 172.21.15.63:0/3086922782 shutdown_connections 2024-05-27T08:01:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f18683e3b80 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb2d5a9bf0 0x55cb2d5ac010 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f18683e3b80 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d59a0f0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f18683e3b80 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d410e40 0x55cb2d599670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f18683e3b80 1 --2- 172.21.15.63:0/3086922782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d599bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f183cc00700 0 client.20835 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.462+0000 7f18683e3b80 1 -- 172.21.15.63:0/3086922782 >> 172.21.15.63:0/3086922782 conn(0x55cb2d4aa390 msgr2=0x55cb2d58f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.463+0000 7f18683e3b80 1 -- 172.21.15.63:0/3086922782 shutdown_connections 2024-05-27T08:01:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.463+0000 7f18683e3b80 1 -- 172.21.15.63:0/3086922782 wait complete. 2024-05-27T08:01:48.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.463+0000 7f18683e3b80 20 client.20835 trim_cache size 0 max 16384 2024-05-27T08:01:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.490+0000 7f18683e3b80 1 Processor -- start 2024-05-27T08:01:48.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.490+0000 7f18683e3b80 1 -- start start 2024-05-27T08:01:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.490+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d410e40 0x55cb2d594e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.490+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d5a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.491+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d599d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.491+0000 7f18683e3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18541b0a40 con 0x55cb2d58fe70 2024-05-27T08:01:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.491+0000 7f18683e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18541ae2d0 con 0x55cb2d410e40 2024-05-27T08:01:48.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.491+0000 7f18683e3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18541ac160 con 0x55cb2d4fc9f0 2024-05-27T08:01:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.491+0000 7f1853400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d599d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.491+0000 7f1853400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d599d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56758/0 (socket says 172.21.15.63:56758) 2024-05-27T08:01:48.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.491+0000 7f1853400700 1 -- 172.21.15.63:0/1740295580 learned_addr learned my addr 172.21.15.63:0/1740295580 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.491+0000 7f1852a00700 1 --2- 172.21.15.63:0/1740295580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d410e40 0x55cb2d594e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.491+0000 7f1852000700 1 --2- 172.21.15.63:0/1740295580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d5a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.492+0000 7f1853400700 1 -- 172.21.15.63:0/1740295580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 msgr2=0x55cb2d5a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.492+0000 7f1853400700 1 --2- 172.21.15.63:0/1740295580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d5a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.492+0000 7f1853400700 1 -- 172.21.15.63:0/1740295580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d410e40 msgr2=0x55cb2d594e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.492+0000 7f1853400700 1 --2- 172.21.15.63:0/1740295580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d410e40 0x55cb2d594e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.492+0000 7f1853400700 1 -- 172.21.15.63:0/1740295580 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2d58c000 con 0x55cb2d58fe70 2024-05-27T08:01:48.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.492+0000 7f1853400700 1 --2- 172.21.15.63:0/1740295580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d599d40 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f184c000f80 tx=0x7f184c015cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4761acc99dd2ae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.493+0000 7f183f400700 1 -- 172.21.15.63:0/1740295580 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c03f070 con 0x55cb2d58fe70 2024-05-27T08:01:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.493+0000 7f183f400700 1 -- 172.21.15.63:0/1740295580 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f184c033070 con 0x55cb2d58fe70 2024-05-27T08:01:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.493+0000 7f183f400700 1 -- 172.21.15.63:0/1740295580 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c004e10 con 0x55cb2d58fe70 2024-05-27T08:01:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.494+0000 7f18683e3b80 1 -- 172.21.15.63:0/1740295580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 msgr2=0x55cb2d599d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.494+0000 7f18683e3b80 1 --2- 172.21.15.63:0/1740295580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d599d40 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f184c000f80 tx=0x7f184c015cc0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.494+0000 7f18683e3b80 1 -- 172.21.15.63:0/1740295580 shutdown_connections 2024-05-27T08:01:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.494+0000 7f18683e3b80 1 --2- 172.21.15.63:0/1740295580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d58fe70 0x55cb2d599d40 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.494+0000 7f18683e3b80 1 --2- 172.21.15.63:0/1740295580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d410e40 0x55cb2d594e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.494+0000 7f18683e3b80 1 --2- 172.21.15.63:0/1740295580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d5a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.494+0000 7f18683e3b80 1 -- 172.21.15.63:0/1740295580 >> 172.21.15.63:0/1740295580 conn(0x55cb2d4aa390 msgr2=0x55cb2d599630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.494+0000 7f18683e3b80 1 -- 172.21.15.63:0/1740295580 shutdown_connections 2024-05-27T08:01:48.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.494+0000 7f18683e3b80 1 -- 172.21.15.63:0/1740295580 wait complete. 2024-05-27T08:01:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.495+0000 7f18683e3b80 1 -- start start 2024-05-27T08:01:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.495+0000 7f18683e3b80 1 Processor -- start 2024-05-27T08:01:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d410e40 0x55cb2d5a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d5a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f18683e3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d58fe70 0x55cb2d59ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f18683e3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18541ac7b0 con 0x55cb2d410e40 2024-05-27T08:01:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f18683e3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18541adc00 con 0x55cb2d58fe70 2024-05-27T08:01:48.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f18683e3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18541ace60 con 0x55cb2d4fc9f0 2024-05-27T08:01:48.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f18683e3b80 10 client.0 _ll_register_callbacks cb 0x55cb2d457c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f1852a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d410e40 0x55cb2d5a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f1852a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d410e40 0x55cb2d5a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56774/0 (socket says 172.21.15.63:56774) 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f1852a00700 1 -- 172.21.15.63:0/4143100644 learned_addr learned my addr 172.21.15.63:0/4143100644 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.496+0000 7f1852000700 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d5a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.497+0000 7f1853400700 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d58fe70 0x55cb2d59ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.497+0000 7f1852a00700 1 -- 172.21.15.63:0/4143100644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 msgr2=0x55cb2d5a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.497+0000 7f1852a00700 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d5a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.497+0000 7f1852a00700 1 -- 172.21.15.63:0/4143100644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d58fe70 msgr2=0x55cb2d59ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.497+0000 7f1852a00700 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d58fe70 0x55cb2d59ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.497+0000 7f1852a00700 1 -- 172.21.15.63:0/4143100644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb2d59d360 con 0x55cb2d410e40 2024-05-27T08:01:48.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.497+0000 7f1852000700 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d5a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.497+0000 7f1853400700 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d58fe70 0x55cb2d59ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.497+0000 7f1852a00700 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d410e40 0x55cb2d5a1e70 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f18440a7820 tx=0x7f184400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6092b590fd87ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.497+0000 7f183cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.498+0000 7f183cc00700 1 -- 172.21.15.63:0/4143100644 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18440e0e20 con 0x55cb2d410e40 2024-05-27T08:01:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.498+0000 7f183cc00700 1 -- 172.21.15.63:0/4143100644 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f1844074b90 con 0x55cb2d410e40 2024-05-27T08:01:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.498+0000 7f18683e3b80 1 -- 172.21.15.63:0/4143100644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f184c033070 con 0x55cb2d410e40 2024-05-27T08:01:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.498+0000 7f183cc00700 1 -- 172.21.15.63:0/4143100644 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f18440ae960 con 0x55cb2d410e40 2024-05-27T08:01:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.498+0000 7f18683e3b80 10 client.20841 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.498+0000 7f18683e3b80 1 -- 172.21.15.63:0/4143100644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cb2d614df0 con 0x55cb2d410e40 2024-05-27T08:01:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.499+0000 7f183cc00700 1 -- 172.21.15.63:0/4143100644 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f184c033070 con 0x55cb2d410e40 2024-05-27T08:01:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.499+0000 7f183cc00700 1 -- 172.21.15.63:0/4143100644 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f18440b2ef0 con 0x55cb2d410e40 2024-05-27T08:01:48.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.499+0000 7f18683e3b80 10 client.20841 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.499+0000 7f18683e3b80 10 client.20841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.499+0000 7f18683e3b80 20 client.20841 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.499+0000 7f18683e3b80 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb2d615480 0x55cb2d6178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.499+0000 7f18683e3b80 4 client.20841 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:01:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.499+0000 7f18683e3b80 1 -- 172.21.15.63:0/4143100644 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb2d4b9570 con 0x55cb2d615480 2024-05-27T08:01:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.500+0000 7f1852000700 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb2d615480 0x55cb2d6178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.501+0000 7f1852000700 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb2d615480 0x55cb2d6178a0 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1963f128fb4c0c8c server_cookie=7fdf20d0a44c3ad3 in_seq=0 out_seq=0 2024-05-27T08:01:48.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.501+0000 7f183cc00700 10 client.20841 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f183cc00700 1 -- 172.21.15.63:0/4143100644 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cb2d4b9570 con 0x55cb2d615480 2024-05-27T08:01:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f183cc00700 10 client.20841 handle_command_reply: tid=0 2024-05-27T08:01:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f18683e3b80 1 client.20841 shutdown 2024-05-27T08:01:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f18683e3b80 1 -- 172.21.15.63:0/4143100644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d410e40 msgr2=0x55cb2d5a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f18683e3b80 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d410e40 0x55cb2d5a1e70 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f18440a7820 tx=0x7f184400f610 comp rx=0 tx=0).stop 2024-05-27T08:01:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f18683e3b80 1 -- 172.21.15.63:0/4143100644 shutdown_connections 2024-05-27T08:01:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f18683e3b80 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cb2d615480 0x55cb2d6178a0 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f18683e3b80 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb2d58fe70 0x55cb2d59ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f18683e3b80 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb2d410e40 0x55cb2d5a1e70 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f18683e3b80 1 --2- 172.21.15.63:0/4143100644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cb2d4fc9f0 0x55cb2d5a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f18683e3b80 1 -- 172.21.15.63:0/4143100644 >> 172.21.15.63:0/4143100644 conn(0x55cb2d4aa390 msgr2=0x55cb2d599450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.502+0000 7f183cc00700 0 client.20841 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.503+0000 7f18683e3b80 1 -- 172.21.15.63:0/4143100644 shutdown_connections 2024-05-27T08:01:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.503+0000 7f18683e3b80 1 -- 172.21.15.63:0/4143100644 wait complete. 2024-05-27T08:01:48.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.503+0000 7f18683e3b80 20 client.20841 trim_cache size 0 max 16384 2024-05-27T08:01:48.506 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:01:48.507 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:01:48.507 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:01:48.507 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:01:48.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 -- 172.21.15.63:0/3905656501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18380b2c10 msgr2=0x7f18380b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18380b2c10 0x7f18380b5120 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f184c0060b0 tx=0x7f184c009040 comp rx=0 tx=0).stop 2024-05-27T08:01:48.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 -- 172.21.15.63:0/3905656501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18541045d0 msgr2=0x7f1854075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18541045d0 0x7f1854075ab0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f184400b580 tx=0x7f18440047b0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 -- 172.21.15.63:0/3905656501 shutdown_connections 2024-05-27T08:01:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18380b2c10 0x7f18380b5120 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1854109850 0x7f1854076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18541045d0 0x7f1854075ab0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 --2- 172.21.15.63:0/3905656501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1854108f00 0x7f1854075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 -- 172.21.15.63:0/3905656501 >> 172.21.15.63:0/3905656501 conn(0x7f18540ffe60 msgr2=0x7f1854102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 -- 172.21.15.63:0/3905656501 shutdown_connections 2024-05-27T08:01:48.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:48.504+0000 7f1859800700 1 -- 172.21.15.63:0/3905656501 wait complete. 2024-05-27T08:01:49.510 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:01:49.510 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:01:49.624 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:49 smithi063 ceph-mon[31613]: pgmap v895: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 17 op/s 2024-05-27T08:01:49.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:49 smithi086 ceph-mon[34687]: pgmap v895: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 17 op/s 2024-05-27T08:01:49.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:49 smithi149 ceph-mon[35408]: pgmap v895: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 17 op/s 2024-05-27T08:01:50.036 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:51.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:51 smithi086 ceph-mon[34687]: pgmap v896: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T08:01:51.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:51 smithi149 ceph-mon[35408]: pgmap v896: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T08:01:51.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:51 smithi063 ceph-mon[31613]: pgmap v896: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 11 op/s 2024-05-27T08:01:52.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.503+0000 7fa8cc800700 1 -- 172.21.15.63:0/3230946745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c410e910 msgr2=0x7fa8c4110df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:52.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.503+0000 7fa8cc800700 1 --2- 172.21.15.63:0/3230946745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c410e910 0x7fa8c4110df0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000b0a0 tx=0x7fa8c000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:01:52.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.503+0000 7fa8cc800700 1 -- 172.21.15.63:0/3230946745 shutdown_connections 2024-05-27T08:01:52.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.503+0000 7fa8cc800700 1 --2- 172.21.15.63:0/3230946745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c410e910 0x7fa8c4110df0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:52.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.503+0000 7fa8cc800700 1 --2- 172.21.15.63:0/3230946745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8c41054e0 0x7fa8c41058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:52.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.503+0000 7fa8cc800700 1 --2- 172.21.15.63:0/3230946745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c4105e30 0x7fa8c410e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:52.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.503+0000 7fa8cc800700 1 -- 172.21.15.63:0/3230946745 >> 172.21.15.63:0/3230946745 conn(0x7fa8c4078df0 msgr2=0x7fa8c4079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:52.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.503+0000 7fa8cc800700 1 -- 172.21.15.63:0/3230946745 shutdown_connections 2024-05-27T08:01:52.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.503+0000 7fa8cc800700 1 -- 172.21.15.63:0/3230946745 wait complete. 2024-05-27T08:01:52.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.504+0000 7fa8cc800700 1 Processor -- start 2024-05-27T08:01:52.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.505+0000 7fa8cc800700 1 -- start start 2024-05-27T08:01:52.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.505+0000 7fa8cc800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8c41054e0 0x7fa8c419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:52.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.505+0000 7fa8cc800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c4105e30 0x7fa8c419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:52.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.505+0000 7fa8cc800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c410e910 0x7fa8c41a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:52.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.505+0000 7fa8cc800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c4114320 con 0x7fa8c4105e30 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.505+0000 7fa8cc800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41141a0 con 0x7fa8c410e910 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.505+0000 7fa8cc800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41144a0 con 0x7fa8c41054e0 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.506+0000 7fa8c9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c4105e30 0x7fa8c419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.506+0000 7fa8c9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c4105e30 0x7fa8c419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44128/0 (socket says 172.21.15.63:44128) 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.506+0000 7fa8c9400700 1 -- 172.21.15.63:0/1968467324 learned_addr learned my addr 172.21.15.63:0/1968467324 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.506+0000 7fa8c9e00700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8c41054e0 0x7fa8c419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.506+0000 7fa8ca800700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c410e910 0x7fa8c41a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.506+0000 7fa8c9400700 1 -- 172.21.15.63:0/1968467324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8c41054e0 msgr2=0x7fa8c419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.506+0000 7fa8c9400700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8c41054e0 0x7fa8c419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.506+0000 7fa8c9400700 1 -- 172.21.15.63:0/1968467324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c410e910 msgr2=0x7fa8c41a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.506+0000 7fa8c9400700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c410e910 0x7fa8c41a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:52.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.506+0000 7fa8c9400700 1 -- 172.21.15.63:0/1968467324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c41a1cd0 con 0x7fa8c4105e30 2024-05-27T08:01:52.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.507+0000 7fa8c9400700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c4105e30 0x7fa8c419d450 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400d950 tx=0x7fa8b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:52.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.507+0000 7fa8bea00700 1 -- 172.21.15.63:0/1968467324 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4016070 con 0x7fa8c4105e30 2024-05-27T08:01:52.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.507+0000 7fa8bea00700 1 -- 172.21.15.63:0/1968467324 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8b400fd20 con 0x7fa8c4105e30 2024-05-27T08:01:52.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.507+0000 7fa8cc800700 1 -- 172.21.15.63:0/1968467324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c41a1f60 con 0x7fa8c4105e30 2024-05-27T08:01:52.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.507+0000 7fa8bea00700 1 -- 172.21.15.63:0/1968467324 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b400e3b0 con 0x7fa8c4105e30 2024-05-27T08:01:52.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.507+0000 7fa8cc800700 1 -- 172.21.15.63:0/1968467324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c41a24a0 con 0x7fa8c4105e30 2024-05-27T08:01:52.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.509+0000 7fa8cc800700 1 -- 172.21.15.63:0/1968467324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c40512b0 con 0x7fa8c4105e30 2024-05-27T08:01:52.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.515+0000 7fa8bea00700 1 -- 172.21.15.63:0/1968467324 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa8b4019070 con 0x7fa8c4105e30 2024-05-27T08:01:52.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.516+0000 7fa8bea00700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa8ac0b29f0 0x7fa8ac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:52.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.517+0000 7fa8bea00700 1 -- 172.21.15.63:0/1968467324 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7fa8b40e2450 con 0x7fa8c4105e30 2024-05-27T08:01:52.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.517+0000 7fa8c9e00700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa8ac0b29f0 0x7fa8ac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:52.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.518+0000 7fa8c9e00700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa8ac0b29f0 0x7fa8ac0b4f00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800ac50 tx=0x7fa8b800a380 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:52.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.518+0000 7fa8bea00700 1 -- 172.21.15.63:0/1968467324 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8b40a9a10 con 0x7fa8c4105e30 2024-05-27T08:01:52.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.859+0000 7fa8cc800700 1 -- 172.21.15.63:0/1968467324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8c41a99f0 con 0x7fa8c4105e30 2024-05-27T08:01:52.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.866+0000 7fa8bea00700 1 -- 172.21.15.63:0/1968467324 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7fa8b40a9310 con 0x7fa8c4105e30 2024-05-27T08:01:52.870 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:01:52.870 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:01:52.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 -- 172.21.15.63:0/1968467324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa8ac0b29f0 msgr2=0x7fa8ac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:52.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa8ac0b29f0 0x7fa8ac0b4f00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800ac50 tx=0x7fa8b800a380 comp rx=0 tx=0).stop 2024-05-27T08:01:52.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 -- 172.21.15.63:0/1968467324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c4105e30 msgr2=0x7fa8c419d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:52.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c4105e30 0x7fa8c419d450 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400d950 tx=0x7fa8b400dc60 comp rx=0 tx=0).stop 2024-05-27T08:01:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 -- 172.21.15.63:0/1968467324 shutdown_connections 2024-05-27T08:01:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa8ac0b29f0 0x7fa8ac0b4f00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8c410e910 0x7fa8c41a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8c41054e0 0x7fa8c419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 --2- 172.21.15.63:0/1968467324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8c4105e30 0x7fa8c419d450 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 -- 172.21.15.63:0/1968467324 >> 172.21.15.63:0/1968467324 conn(0x7fa8c4078df0 msgr2=0x7fa8c4103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 -- 172.21.15.63:0/1968467324 shutdown_connections 2024-05-27T08:01:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:52.871+0000 7fa8cc800700 1 -- 172.21.15.63:0/1968467324 wait complete. 2024-05-27T08:01:52.877 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:01:53.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:53 smithi086 ceph-mon[34687]: pgmap v897: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T08:01:53.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:53 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1968467324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:01:53.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:53 smithi149 ceph-mon[35408]: pgmap v897: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T08:01:53.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:53 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1968467324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:01:53.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:53 smithi063 ceph-mon[31613]: pgmap v897: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T08:01:53.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:53 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1968467324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:01:53.879 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:01:54.386 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:55.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:55 smithi086 ceph-mon[34687]: pgmap v898: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:01:55.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:55 smithi149 ceph-mon[35408]: pgmap v898: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:01:55.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:55 smithi063 ceph-mon[31613]: pgmap v898: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:01:56.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.889+0000 7f5a40c00700 1 -- 172.21.15.63:0/2731138341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a3c10e710 msgr2=0x7f5a3c110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:56.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.889+0000 7f5a40c00700 1 --2- 172.21.15.63:0/2731138341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a3c10e710 0x7f5a3c110bd0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400b0a0 tx=0x7f5a3400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.889+0000 7f5a40c00700 1 -- 172.21.15.63:0/2731138341 shutdown_connections 2024-05-27T08:01:56.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.889+0000 7f5a40c00700 1 --2- 172.21.15.63:0/2731138341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a3c10e710 0x7f5a3c110bd0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.889+0000 7f5a40c00700 1 --2- 172.21.15.63:0/2731138341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a3c1052e0 0x7f5a3c1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.889+0000 7f5a40c00700 1 --2- 172.21.15.63:0/2731138341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a3c105c30 0x7f5a3c10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.889+0000 7f5a40c00700 1 -- 172.21.15.63:0/2731138341 >> 172.21.15.63:0/2731138341 conn(0x7f5a3c078ab0 msgr2=0x7f5a3c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:56.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.889+0000 7f5a40c00700 1 -- 172.21.15.63:0/2731138341 shutdown_connections 2024-05-27T08:01:56.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.889+0000 7f5a40c00700 1 -- 172.21.15.63:0/2731138341 wait complete. 2024-05-27T08:01:56.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.890+0000 7f5a40c00700 1 Processor -- start 2024-05-27T08:01:56.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.891+0000 7f5a40c00700 1 -- start start 2024-05-27T08:01:56.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.891+0000 7f5a40c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a3c1052e0 0x7f5a3c102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.891+0000 7f5a40c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a3c105c30 0x7f5a3c103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.891+0000 7f5a40c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a3c10e710 0x7f5a3c103a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.891+0000 7f5a40c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c114150 con 0x7f5a3c1052e0 2024-05-27T08:01:56.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.892+0000 7f5a40c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c113fd0 con 0x7f5a3c10e710 2024-05-27T08:01:56.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.892+0000 7f5a40c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1142d0 con 0x7f5a3c105c30 2024-05-27T08:01:56.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.892+0000 7f5a3b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a3c1052e0 0x7f5a3c102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.892+0000 7f5a3b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a3c1052e0 0x7f5a3c102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44146/0 (socket says 172.21.15.63:44146) 2024-05-27T08:01:56.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.892+0000 7f5a3b400700 1 -- 172.21.15.63:0/4165859489 learned_addr learned my addr 172.21.15.63:0/4165859489 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:56.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.892+0000 7f5a3be00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a3c10e710 0x7f5a3c103a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.892+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a3c105c30 0x7f5a3c103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.893+0000 7f5a3b400700 1 -- 172.21.15.63:0/4165859489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a3c105c30 msgr2=0x7f5a3c103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:56.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.893+0000 7f5a3b400700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a3c105c30 0x7f5a3c103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.893+0000 7f5a3b400700 1 -- 172.21.15.63:0/4165859489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a3c10e710 msgr2=0x7f5a3c103a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:56.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.893+0000 7f5a3b400700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a3c10e710 0x7f5a3c103a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.893+0000 7f5a3b400700 1 -- 172.21.15.63:0/4165859489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c1a8f00 con 0x7f5a3c1052e0 2024-05-27T08:01:56.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.893+0000 7f5a3b400700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a3c1052e0 0x7f5a3c102fd0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00d950 tx=0x7f5a2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:56.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.893+0000 7f5a33e00700 1 -- 172.21.15.63:0/4165859489 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c016070 con 0x7f5a3c1052e0 2024-05-27T08:01:56.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.894+0000 7f5a33e00700 1 -- 172.21.15.63:0/4165859489 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a2c00fd20 con 0x7f5a3c1052e0 2024-05-27T08:01:56.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.894+0000 7f5a40c00700 1 -- 172.21.15.63:0/4165859489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3c1a9190 con 0x7f5a3c1052e0 2024-05-27T08:01:56.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.894+0000 7f5a33e00700 1 -- 172.21.15.63:0/4165859489 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00e3b0 con 0x7f5a3c1052e0 2024-05-27T08:01:56.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.894+0000 7f5a40c00700 1 -- 172.21.15.63:0/4165859489 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3c1a96d0 con 0x7f5a3c1052e0 2024-05-27T08:01:56.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.899+0000 7f5a33e00700 1 -- 172.21.15.63:0/4165859489 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5a2c019070 con 0x7f5a3c1052e0 2024-05-27T08:01:56.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.899+0000 7f5a50d2db80 1 Processor -- start 2024-05-27T08:01:56.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.899+0000 7f5a50d2db80 1 -- start start 2024-05-27T08:01:56.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.900+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d8b59f0 0x55d10d949750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.900+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 0x55d10d949c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.900+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d9532c0 0x55d10d9556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.900+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d10d876e40 con 0x55d10d9532c0 2024-05-27T08:01:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.900+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d10d8669e0 con 0x55d10d7c9e40 2024-05-27T08:01:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.900+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d10d86aa20 con 0x55d10d8b59f0 2024-05-27T08:01:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.900+0000 7f5a3aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d9532c0 0x55d10d9556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.900+0000 7f5a3aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d9532c0 0x55d10d9556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44160/0 (socket says 172.21.15.63:44160) 2024-05-27T08:01:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.900+0000 7f5a3aa00700 1 -- 172.21.15.63:0/1760221394 learned_addr learned my addr 172.21.15.63:0/1760221394 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.900+0000 7f5a3be00700 1 --2- 172.21.15.63:0/1760221394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d8b59f0 0x55d10d949750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.901+0000 7f5a3b400700 1 --2- 172.21.15.63:0/1760221394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 0x55d10d949c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.901+0000 7f5a33e00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a200b29f0 0x7f5a200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.901+0000 7f5a3aa00700 1 -- 172.21.15.63:0/1760221394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d8b59f0 msgr2=0x55d10d949750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.901+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/1760221394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d8b59f0 0x55d10d949750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.901+0000 7f5a3aa00700 1 -- 172.21.15.63:0/1760221394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 msgr2=0x55d10d949c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.901+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/1760221394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 0x55d10d949c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.901+0000 7f5a3aa00700 1 -- 172.21.15.63:0/1760221394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d10d82f420 con 0x55d10d9532c0 2024-05-27T08:01:56.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.901+0000 7f5a33e00700 1 -- 172.21.15.63:0/4165859489 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f5a2c0e2570 con 0x7f5a3c1052e0 2024-05-27T08:01:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.902+0000 7f5a3be00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a200b29f0 0x7f5a200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.902+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/1760221394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d9532c0 0x55d10d9556e0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800bcb0 tx=0x7f5a280103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0102c198b4183d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.902+0000 7f5a30c00700 1 -- 172.21.15.63:0/1760221394 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28010c50 con 0x55d10d9532c0 2024-05-27T08:01:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.902+0000 7f5a30c00700 1 -- 172.21.15.63:0/1760221394 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a280044c0 con 0x55d10d9532c0 2024-05-27T08:01:56.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.902+0000 7f5a30c00700 1 -- 172.21.15.63:0/1760221394 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28011440 con 0x55d10d9532c0 2024-05-27T08:01:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.903+0000 7f5a50d2db80 1 -- 172.21.15.63:0/1760221394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d9532c0 msgr2=0x55d10d9556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.903+0000 7f5a3be00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a200b29f0 0x7f5a200b4f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009e60 tx=0x7f5a3400b3b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.903+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/1760221394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d9532c0 0x55d10d9556e0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800bcb0 tx=0x7f5a280103d0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.904+0000 7f5a50d2db80 1 -- 172.21.15.63:0/1760221394 shutdown_connections 2024-05-27T08:01:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.904+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/1760221394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d9532c0 0x55d10d9556e0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.904+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/1760221394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d8b59f0 0x55d10d949750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.904+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/1760221394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 0x55d10d949c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.904+0000 7f5a50d2db80 1 -- 172.21.15.63:0/1760221394 >> 172.21.15.63:0/1760221394 conn(0x55d10d863390 msgr2=0x55d10d944710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.904+0000 7f5a50d2db80 1 -- 172.21.15.63:0/1760221394 shutdown_connections 2024-05-27T08:01:56.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.904+0000 7f5a50d2db80 1 -- 172.21.15.63:0/1760221394 wait complete. 2024-05-27T08:01:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.904+0000 7f5a50d2db80 1 -- start start 2024-05-27T08:01:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.905+0000 7f5a50d2db80 1 Processor -- start 2024-05-27T08:01:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.905+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d950260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.905+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 0x55d10d9507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.905+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d950ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.905+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d10d76cfa0 con 0x55d10d7c9e40 2024-05-27T08:01:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.905+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d10d807760 con 0x55d10d8b59f0 2024-05-27T08:01:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.905+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d10d955e00 con 0x55d10d9532c0 2024-05-27T08:01:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a50d2db80 10 client.0 _ll_register_callbacks cb 0x55d10d810c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:56.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d950260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d950260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44166/0 (socket says 172.21.15.63:44166) 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3b400700 1 -- 172.21.15.63:0/477973155 learned_addr learned my addr 172.21.15.63:0/477973155 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 0x55d10d9507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3b400700 1 -- 172.21.15.63:0/477973155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 msgr2=0x55d10d950ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3be00700 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d950ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3b400700 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d950ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3b400700 1 -- 172.21.15.63:0/477973155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 msgr2=0x55d10d9507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3b400700 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 0x55d10d9507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3b400700 1 -- 172.21.15.63:0/477973155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d10d7b68d0 con 0x55d10d7c9e40 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3be00700 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d950ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:01:56.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.906+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 0x55d10d9507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:56.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.907+0000 7f5a3b400700 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d950260 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0a9130 tx=0x7f5a2c0a9330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb789c54e8df7d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.907+0000 7f5a25600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.907+0000 7f5a25600700 1 -- 172.21.15.63:0/477973155 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c020070 con 0x55d10d7c9e40 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.907+0000 7f5a25600700 1 -- 172.21.15.63:0/477973155 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f5a2c0e20f0 con 0x55d10d7c9e40 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.907+0000 7f5a50d2db80 1 -- 172.21.15.63:0/477973155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a280044c0 con 0x55d10d7c9e40 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.907+0000 7f5a25600700 1 -- 172.21.15.63:0/477973155 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5a2c00fed0 con 0x55d10d7c9e40 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.908+0000 7f5a50d2db80 10 client.20859 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.908+0000 7f5a50d2db80 1 -- 172.21.15.63:0/477973155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d10d837060 con 0x55d10d7c9e40 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.908+0000 7f5a25600700 1 -- 172.21.15.63:0/477973155 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a280044c0 con 0x55d10d7c9e40 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.908+0000 7f5a25600700 1 -- 172.21.15.63:0/477973155 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5a2c0a9690 con 0x55d10d7c9e40 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.908+0000 7f5a50d2db80 10 client.20859 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:56.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.908+0000 7f5a50d2db80 10 client.20859 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:01:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.908+0000 7f5a50d2db80 20 client.20859 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.908+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55d10d962be0 0x55d10d965000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.908+0000 7f5a50d2db80 4 client.20859 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.908+0000 7f5a50d2db80 1 -- 172.21.15.63:0/477973155 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d10d878d90 con 0x55d10d962be0 2024-05-27T08:01:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.909+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55d10d962be0 0x55d10d965000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.910+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55d10d962be0 0x55d10d965000 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ed0e10f4b8c5e3e server_cookie=11c916860aa768a1 in_seq=0 out_seq=0 2024-05-27T08:01:56.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.910+0000 7f5a25600700 10 client.20859 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:01:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.912+0000 7f5a25600700 1 -- 172.21.15.63:0/477973155 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d10d878d90 con 0x55d10d962be0 2024-05-27T08:01:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.912+0000 7f5a25600700 10 client.20859 handle_command_reply: tid=0 2024-05-27T08:01:56.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.912+0000 7f5a50d2db80 1 client.20859 shutdown 2024-05-27T08:01:56.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.912+0000 7f5a50d2db80 1 -- 172.21.15.63:0/477973155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 msgr2=0x55d10d950260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:56.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.912+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d950260 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0a9130 tx=0x7f5a2c0a9330 comp rx=0 tx=0).stop 2024-05-27T08:01:56.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.913+0000 7f5a50d2db80 1 -- 172.21.15.63:0/477973155 shutdown_connections 2024-05-27T08:01:56.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.913+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d950ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.913+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55d10d962be0 0x55d10d965000 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.913+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d950260 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.913+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/477973155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 0x55d10d9507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.913+0000 7f5a50d2db80 1 -- 172.21.15.63:0/477973155 >> 172.21.15.63:0/477973155 conn(0x55d10d863390 msgr2=0x55d10d9497f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.913+0000 7f5a25600700 0 client.20859 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:01:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.913+0000 7f5a50d2db80 1 -- 172.21.15.63:0/477973155 shutdown_connections 2024-05-27T08:01:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.913+0000 7f5a50d2db80 1 -- 172.21.15.63:0/477973155 wait complete. 2024-05-27T08:01:56.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.914+0000 7f5a50d2db80 20 client.20859 trim_cache size 0 max 16384 2024-05-27T08:01:56.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.934+0000 7f5a50d2db80 1 Processor -- start 2024-05-27T08:01:56.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.934+0000 7f5a50d2db80 1 -- start start 2024-05-27T08:01:56.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.935+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 0x55d10d944740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.935+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d8b59f0 0x55d10d95eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.935+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d96a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.935+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1b4d90 con 0x55d10d8b59f0 2024-05-27T08:01:56.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.935+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1b2620 con 0x55d10d7c9e40 2024-05-27T08:01:56.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.935+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1b0490 con 0x55d10d9532c0 2024-05-27T08:01:56.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.935+0000 7f5a3aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d8b59f0 0x55d10d95eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.935+0000 7f5a3aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d8b59f0 0x55d10d95eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44176/0 (socket says 172.21.15.63:44176) 2024-05-27T08:01:56.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.935+0000 7f5a3aa00700 1 -- 172.21.15.63:0/442117595 learned_addr learned my addr 172.21.15.63:0/442117595 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:56.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.936+0000 7f5a3be00700 1 --2- 172.21.15.63:0/442117595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d96a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.936+0000 7f5a3b400700 1 --2- 172.21.15.63:0/442117595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 0x55d10d944740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.936+0000 7f5a3aa00700 1 -- 172.21.15.63:0/442117595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 msgr2=0x55d10d96a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:56.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.936+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/442117595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d96a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.936+0000 7f5a3aa00700 1 -- 172.21.15.63:0/442117595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 msgr2=0x55d10d944740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:56.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.936+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/442117595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 0x55d10d944740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.936+0000 7f5a3aa00700 1 -- 172.21.15.63:0/442117595 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d10d945180 con 0x55d10d8b59f0 2024-05-27T08:01:56.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.936+0000 7f5a3b400700 1 --2- 172.21.15.63:0/442117595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 0x55d10d944740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:56.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.936+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/442117595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d8b59f0 0x55d10d95eea0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x55d10d94b210 tx=0x7f5a28003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b364592bb4a60fec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:56.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.937+0000 7f5a27e00700 1 -- 172.21.15.63:0/442117595 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28010d70 con 0x55d10d8b59f0 2024-05-27T08:01:56.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.937+0000 7f5a27e00700 1 -- 172.21.15.63:0/442117595 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a28003d40 con 0x55d10d8b59f0 2024-05-27T08:01:56.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.937+0000 7f5a27e00700 1 -- 172.21.15.63:0/442117595 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28028430 con 0x55d10d8b59f0 2024-05-27T08:01:56.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.938+0000 7f5a50d2db80 1 -- 172.21.15.63:0/442117595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d8b59f0 msgr2=0x55d10d95eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:56.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.938+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/442117595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d8b59f0 0x55d10d95eea0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x55d10d94b210 tx=0x7f5a28003840 comp rx=0 tx=0).stop 2024-05-27T08:01:56.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.938+0000 7f5a50d2db80 1 -- 172.21.15.63:0/442117595 shutdown_connections 2024-05-27T08:01:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.938+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/442117595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d96a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.938+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/442117595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d7c9e40 0x55d10d944740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.938+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/442117595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d8b59f0 0x55d10d95eea0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.938+0000 7f5a50d2db80 1 -- 172.21.15.63:0/442117595 >> 172.21.15.63:0/442117595 conn(0x55d10d863390 msgr2=0x55d10d94b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.938+0000 7f5a50d2db80 1 -- 172.21.15.63:0/442117595 shutdown_connections 2024-05-27T08:01:56.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.938+0000 7f5a50d2db80 1 -- 172.21.15.63:0/442117595 wait complete. 2024-05-27T08:01:56.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.939+0000 7f5a50d2db80 1 -- start start 2024-05-27T08:01:56.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.939+0000 7f5a50d2db80 1 Processor -- start 2024-05-27T08:01:56.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d956650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 0x55d10d956b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a50d2db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d9cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1b0ae0 con 0x55d10d7c9e40 2024-05-27T08:01:56.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1b1f50 con 0x55d10d8b59f0 2024-05-27T08:01:56.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a50d2db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c1b11b0 con 0x55d10d9532c0 2024-05-27T08:01:56.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a50d2db80 10 client.0 _ll_register_callbacks cb 0x55d10d810c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:56.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a3b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d956650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a3b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d956650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44178/0 (socket says 172.21.15.63:44178) 2024-05-27T08:01:56.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a3b400700 1 -- 172.21.15.63:0/85988692 learned_addr learned my addr 172.21.15.63:0/85988692 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:56.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.940+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 0x55d10d956b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.941+0000 7f5a3be00700 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d9cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.941+0000 7f5a3b400700 1 -- 172.21.15.63:0/85988692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 msgr2=0x55d10d9cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:56.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.941+0000 7f5a3b400700 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d9cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.941+0000 7f5a3b400700 1 -- 172.21.15.63:0/85988692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 msgr2=0x55d10d956b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:56.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.941+0000 7f5a3b400700 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 0x55d10d956b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.941+0000 7f5a3b400700 1 -- 172.21.15.63:0/85988692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d10d9cbbc0 con 0x55d10d7c9e40 2024-05-27T08:01:56.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.941+0000 7f5a3b400700 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d956650 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0a54b0 tx=0x7f5a2c0b2180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba805bcb9816b977 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:56.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.941+0000 7f5a25600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:56.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.941+0000 7f5a25600700 1 -- 172.21.15.63:0/85988692 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c020040 con 0x55d10d7c9e40 2024-05-27T08:01:56.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.942+0000 7f5a25600700 1 -- 172.21.15.63:0/85988692 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f5a2c0a5a20 con 0x55d10d7c9e40 2024-05-27T08:01:56.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.942+0000 7f5a50d2db80 1 -- 172.21.15.63:0/85988692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a28003d40 con 0x55d10d7c9e40 2024-05-27T08:01:56.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.942+0000 7f5a25600700 1 -- 172.21.15.63:0/85988692 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5a2c0a8f40 con 0x55d10d7c9e40 2024-05-27T08:01:56.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.942+0000 7f5a50d2db80 10 client.20865 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:56.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.942+0000 7f5a50d2db80 1 -- 172.21.15.63:0/85988692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d10d9ccd00 con 0x55d10d7c9e40 2024-05-27T08:01:56.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.942+0000 7f5a25600700 1 -- 172.21.15.63:0/85988692 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a28003d40 con 0x55d10d7c9e40 2024-05-27T08:01:56.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.942+0000 7f5a25600700 1 -- 172.21.15.63:0/85988692 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5a2c0a3b50 con 0x55d10d7c9e40 2024-05-27T08:01:56.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.942+0000 7f5a50d2db80 10 client.20865 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:56.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.942+0000 7f5a50d2db80 10 client.20865 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:01:56.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.943+0000 7f5a50d2db80 20 client.20865 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:56.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.943+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55d10d9cd3b0 0x55d10d9cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:56.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.943+0000 7f5a50d2db80 4 client.20865 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:01:56.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.943+0000 7f5a50d2db80 1 -- 172.21.15.63:0/85988692 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d10d872570 con 0x55d10d9cd3b0 2024-05-27T08:01:56.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.943+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55d10d9cd3b0 0x55d10d9cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:56.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.944+0000 7f5a3aa00700 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55d10d9cd3b0 0x55d10d9cf7d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18c01c5576084c41 server_cookie=62c4c6973e85afb in_seq=0 out_seq=0 2024-05-27T08:01:56.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.944+0000 7f5a25600700 10 client.20865 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:01:56.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a25600700 1 -- 172.21.15.63:0/85988692 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d10d872570 con 0x55d10d9cd3b0 2024-05-27T08:01:56.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a25600700 10 client.20865 handle_command_reply: tid=0 2024-05-27T08:01:56.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a50d2db80 1 client.20865 shutdown 2024-05-27T08:01:56.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a50d2db80 1 -- 172.21.15.63:0/85988692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 msgr2=0x55d10d956650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:56.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d956650 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0a54b0 tx=0x7f5a2c0b2180 comp rx=0 tx=0).stop 2024-05-27T08:01:56.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a50d2db80 1 -- 172.21.15.63:0/85988692 shutdown_connections 2024-05-27T08:01:56.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d10d9532c0 0x55d10d9cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55d10d9cd3b0 0x55d10d9cf7d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d10d7c9e40 0x55d10d956650 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a50d2db80 1 --2- 172.21.15.63:0/85988692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d10d8b59f0 0x55d10d956b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a25600700 0 client.20865 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:01:56.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.945+0000 7f5a50d2db80 1 -- 172.21.15.63:0/85988692 >> 172.21.15.63:0/85988692 conn(0x55d10d863390 msgr2=0x55d10d94b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:56.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.946+0000 7f5a50d2db80 1 -- 172.21.15.63:0/85988692 shutdown_connections 2024-05-27T08:01:56.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.946+0000 7f5a50d2db80 1 -- 172.21.15.63:0/85988692 wait complete. 2024-05-27T08:01:56.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.946+0000 7f5a50d2db80 20 client.20865 trim_cache size 0 max 16384 2024-05-27T08:01:56.950 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:01:56.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 -- 172.21.15.63:0/4165859489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a200b29f0 msgr2=0x7f5a200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:56.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a200b29f0 0x7f5a200b4f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009e60 tx=0x7f5a3400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 -- 172.21.15.63:0/4165859489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a3c1052e0 msgr2=0x7f5a3c102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:56.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a3c1052e0 0x7f5a3c102fd0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00d950 tx=0x7f5a2c00dc60 comp rx=0 tx=0).stop 2024-05-27T08:01:56.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 -- 172.21.15.63:0/4165859489 shutdown_connections 2024-05-27T08:01:56.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5a200b29f0 0x7f5a200b4f00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a3c10e710 0x7f5a3c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a3c1052e0 0x7f5a3c102fd0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 --2- 172.21.15.63:0/4165859489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a3c105c30 0x7f5a3c103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:56.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 -- 172.21.15.63:0/4165859489 >> 172.21.15.63:0/4165859489 conn(0x7f5a3c078ab0 msgr2=0x7f5a3c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:56.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 -- 172.21.15.63:0/4165859489 shutdown_connections 2024-05-27T08:01:56.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:01:56.947+0000 7f5a40c00700 1 -- 172.21.15.63:0/4165859489 wait complete. 2024-05-27T08:01:57.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:57 smithi086 ceph-mon[34687]: pgmap v899: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:01:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:57 smithi149 ceph-mon[35408]: pgmap v899: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:01:57.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:57 smithi063 ceph-mon[31613]: pgmap v899: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:01:58.095 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:01:58.609 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:59.692 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:01:59 smithi063 ceph-mon[31613]: pgmap v900: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:01:59.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:01:59 smithi086 ceph-mon[34687]: pgmap v900: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:01:59.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:01:59 smithi149 ceph-mon[35408]: pgmap v900: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:01:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.848+0000 7f116be00640 1 -- 172.21.15.63:0/774474878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c10e4e0 msgr2=0x7f116c110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.848+0000 7f116be00640 1 --2- 172.21.15.63:0/774474878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c10e4e0 0x7f116c110900 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f1164009ca0 tx=0x7f116402a8c0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.848+0000 7f116be00640 1 -- 172.21.15.63:0/774474878 shutdown_connections 2024-05-27T08:01:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.848+0000 7f116be00640 1 --2- 172.21.15.63:0/774474878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c10e4e0 0x7f116c110900 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.848+0000 7f116be00640 1 --2- 172.21.15.63:0/774474878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f116c101fe0 0x7f116c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.848+0000 7f116be00640 1 --2- 172.21.15.63:0/774474878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f116c109b30 0x7f116c101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.848+0000 7f116be00640 1 -- 172.21.15.63:0/774474878 >> 172.21.15.63:0/774474878 conn(0x7f116c0fd950 msgr2=0x7f116c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.848+0000 7f116be00640 1 -- 172.21.15.63:0/774474878 shutdown_connections 2024-05-27T08:01:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.848+0000 7f116be00640 1 -- 172.21.15.63:0/774474878 wait complete. 2024-05-27T08:01:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.849+0000 7f116be00640 1 Processor -- start 2024-05-27T08:01:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.849+0000 7f116be00640 1 -- start start 2024-05-27T08:01:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.850+0000 7f116be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c101fe0 0x7f116c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.850+0000 7f116be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f116c109b30 0x7f116c19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.850+0000 7f116be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f116c10e4e0 0x7f116c1a8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.850+0000 7f116be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f116c113ba0 con 0x7f116c101fe0 2024-05-27T08:01:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.850+0000 7f116be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f116c113a20 con 0x7f116c10e4e0 2024-05-27T08:01:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.850+0000 7f116be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f116c113d20 con 0x7f116c109b30 2024-05-27T08:01:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.850+0000 7f116aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c101fe0 0x7f116c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.850+0000 7f116aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c101fe0 0x7f116c19d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49206/0 (socket says 172.21.15.63:49206) 2024-05-27T08:01:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.850+0000 7f116aa00640 1 -- 172.21.15.63:0/2458169870 learned_addr learned my addr 172.21.15.63:0/2458169870 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.851+0000 7f116a000640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f116c109b30 0x7f116c19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.851+0000 7f116b400640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f116c10e4e0 0x7f116c1a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.851+0000 7f116aa00640 1 -- 172.21.15.63:0/2458169870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f116c109b30 msgr2=0x7f116c19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.851+0000 7f116aa00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f116c109b30 0x7f116c19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.851+0000 7f116aa00640 1 -- 172.21.15.63:0/2458169870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f116c10e4e0 msgr2=0x7f116c1a8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.851+0000 7f116aa00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f116c10e4e0 0x7f116c1a8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.851+0000 7f116aa00640 1 -- 172.21.15.63:0/2458169870 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c1a8a40 con 0x7f116c101fe0 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.851+0000 7f116a000640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f116c109b30 0x7f116c19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.851+0000 7f116b400640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f116c10e4e0 0x7f116c1a8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.851+0000 7f116aa00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c101fe0 0x7f116c19d480 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f115c005fd0 tx=0x7f115c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.852+0000 7f1163400640 1 -- 172.21.15.63:0/2458169870 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c017070 con 0x7f116c101fe0 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.852+0000 7f1163400640 1 -- 172.21.15.63:0/2458169870 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115c009c30 con 0x7f116c101fe0 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.852+0000 7f116be00640 1 -- 172.21.15.63:0/2458169870 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116c1a8c70 con 0x7f116c101fe0 2024-05-27T08:01:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.852+0000 7f1163400640 1 -- 172.21.15.63:0/2458169870 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c003c90 con 0x7f116c101fe0 2024-05-27T08:01:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.852+0000 7f116be00640 1 -- 172.21.15.63:0/2458169870 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116c1a91c0 con 0x7f116c101fe0 2024-05-27T08:01:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.856+0000 7f1163400640 1 -- 172.21.15.63:0/2458169870 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f115c003e30 con 0x7f116c101fe0 2024-05-27T08:01:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.857+0000 7f1178b91740 1 Processor -- start 2024-05-27T08:01:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.857+0000 7f1178b91740 1 -- start start 2024-05-27T08:01:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.857+0000 7f1178b91740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a65ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.857+0000 7f1178b91740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2912f0 0x55603a65cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.857+0000 7f1178b91740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2994d0 0x55603a65d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.857+0000 7f1178b91740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55603a5cdb90 con 0x55603a2ba1b0 2024-05-27T08:01:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.857+0000 7f1178b91740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55603a5bd730 con 0x55603a2994d0 2024-05-27T08:01:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.857+0000 7f1178b91740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55603a5c1770 con 0x55603a2912f0 2024-05-27T08:01:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.858+0000 7f116b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a65ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.858+0000 7f116b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a65ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49208/0 (socket says 172.21.15.63:49208) 2024-05-27T08:01:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.858+0000 7f116b400640 1 -- 172.21.15.63:0/3361226031 learned_addr learned my addr 172.21.15.63:0/3361226031 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.858+0000 7f116a000640 1 --2- 172.21.15.63:0/3361226031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2994d0 0x55603a65d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.858+0000 7f116aa00640 1 --2- 172.21.15.63:0/3361226031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2912f0 0x55603a65cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.858+0000 7f116b400640 1 -- 172.21.15.63:0/3361226031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2912f0 msgr2=0x55603a65cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.858+0000 7f116b400640 1 --2- 172.21.15.63:0/3361226031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2912f0 0x55603a65cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.858+0000 7f116b400640 1 -- 172.21.15.63:0/3361226031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2994d0 msgr2=0x55603a65d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.858+0000 7f116b400640 1 --2- 172.21.15.63:0/3361226031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2994d0 0x55603a65d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.858+0000 7f116b400640 1 -- 172.21.15.63:0/3361226031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55603a5568b0 con 0x55603a2ba1b0 2024-05-27T08:01:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.859+0000 7f1163400640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11500b29d0 0x7f11500b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.859+0000 7f116b400640 1 --2- 172.21.15.63:0/3361226031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a65ca60 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f1164035000 tx=0x7f1164009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d90de570dcd4f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.859+0000 7f1163400640 1 -- 172.21.15.63:0/2458169870 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f115c0e25f0 con 0x7f116c101fe0 2024-05-27T08:01:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.859+0000 7f116aa00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11500b29d0 0x7f11500b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.859+0000 7f1157e00640 1 -- 172.21.15.63:0/3361226031 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164031980 con 0x55603a2ba1b0 2024-05-27T08:01:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.859+0000 7f1157e00640 1 -- 172.21.15.63:0/3361226031 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1164036040 con 0x55603a2ba1b0 2024-05-27T08:01:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.859+0000 7f1157e00640 1 -- 172.21.15.63:0/3361226031 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164004460 con 0x55603a2ba1b0 2024-05-27T08:01:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.860+0000 7f1178b91740 1 -- 172.21.15.63:0/3361226031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 msgr2=0x55603a65ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.860+0000 7f1178b91740 1 --2- 172.21.15.63:0/3361226031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a65ca60 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f1164035000 tx=0x7f1164009a20 comp rx=0 tx=0).stop 2024-05-27T08:01:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.860+0000 7f116aa00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11500b29d0 0x7f11500b4ee0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f115c0dfa30 tx=0x7f115c00bf60 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.861+0000 7f1178b91740 1 -- 172.21.15.63:0/3361226031 shutdown_connections 2024-05-27T08:01:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.861+0000 7f1178b91740 1 --2- 172.21.15.63:0/3361226031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2994d0 0x55603a65d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.861+0000 7f1178b91740 1 --2- 172.21.15.63:0/3361226031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2912f0 0x55603a65cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.861+0000 7f1178b91740 1 --2- 172.21.15.63:0/3361226031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a65ca60 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.861+0000 7f1178b91740 1 -- 172.21.15.63:0/3361226031 >> 172.21.15.63:0/3361226031 conn(0x55603a2ae300 msgr2=0x55603a659210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.861+0000 7f1178b91740 1 -- 172.21.15.63:0/3361226031 shutdown_connections 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.861+0000 7f1178b91740 1 -- 172.21.15.63:0/3361226031 wait complete. 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.861+0000 7f1178b91740 1 -- start start 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.862+0000 7f1178b91740 1 Processor -- start 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.862+0000 7f1178b91740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2912f0 0x55603a666b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.862+0000 7f1178b91740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2994d0 0x55603a667090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.862+0000 7f1178b91740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a667610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.862+0000 7f1178b91740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55603a2a3790 con 0x55603a2ba1b0 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.862+0000 7f1178b91740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55603a565bc0 con 0x55603a2912f0 2024-05-27T08:01:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.862+0000 7f1178b91740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55603a669b30 con 0x55603a2994d0 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.862+0000 7f1178b91740 10 client.0 _ll_register_callbacks cb 0x55603a543400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.863+0000 7f116b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a667610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.863+0000 7f116b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a667610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49214/0 (socket says 172.21.15.63:49214) 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.863+0000 7f116b400640 1 -- 172.21.15.63:0/1840977133 learned_addr learned my addr 172.21.15.63:0/1840977133 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.863+0000 7f116b400640 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2912f0 0x55603a666b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.863+0000 7f116a000640 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2994d0 0x55603a667090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.863+0000 7f116b400640 1 -- 172.21.15.63:0/1840977133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2994d0 msgr2=0x55603a667090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.864+0000 7f116b400640 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2994d0 0x55603a667090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.864+0000 7f116b400640 1 -- 172.21.15.63:0/1840977133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2912f0 msgr2=0x55603a666b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.864+0000 7f116b400640 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2912f0 0x55603a666b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.864+0000 7f116b400640 1 -- 172.21.15.63:0/1840977133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55603a2d2210 con 0x55603a2ba1b0 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.864+0000 7f116b400640 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a667610 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f1164009620 tx=0x7f116403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deec71d96e9eaa86 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.864+0000 7f1154c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.864+0000 7f1154c00640 1 -- 172.21.15.63:0/1840977133 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164059060 con 0x55603a2ba1b0 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.864+0000 7f1154c00640 1 -- 172.21.15.63:0/1840977133 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f1164003970 con 0x55603a2ba1b0 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.865+0000 7f1178b91740 1 -- 172.21.15.63:0/1840977133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1164036040 con 0x55603a2ba1b0 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.865+0000 7f1154c00640 1 -- 172.21.15.63:0/1840977133 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1164003b70 con 0x55603a2ba1b0 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.865+0000 7f1178b91740 10 client.20877 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.865+0000 7f1178b91740 1 -- 172.21.15.63:0/1840977133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55603a39e620 con 0x55603a2ba1b0 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.865+0000 7f1154c00640 1 -- 172.21.15.63:0/1840977133 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1164036040 con 0x55603a2ba1b0 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.866+0000 7f1154c00640 1 -- 172.21.15.63:0/1840977133 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f11640305c0 con 0x55603a2ba1b0 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.866+0000 7f1178b91740 10 client.20877 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.866+0000 7f1178b91740 10 client.20877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.866+0000 7f1178b91740 20 client.20877 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.866+0000 7f1178b91740 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55603a278d80 0x55603a678bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.866+0000 7f1178b91740 4 client.20877 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.866+0000 7f1178b91740 1 -- 172.21.15.63:0/1840977133 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55603a5cfae0 con 0x55603a278d80 2024-05-27T08:01:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.867+0000 7f116a000640 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55603a278d80 0x55603a678bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.868+0000 7f116a000640 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55603a278d80 0x55603a678bb0 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c759f0fea6796a8 server_cookie=f600438cbea87c3b in_seq=0 out_seq=0 2024-05-27T08:01:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.868+0000 7f1154c00640 10 client.20877 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.869+0000 7f1154c00640 1 -- 172.21.15.63:0/1840977133 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55603a5cfae0 con 0x55603a278d80 2024-05-27T08:01:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.869+0000 7f1154c00640 10 client.20877 handle_command_reply: tid=0 2024-05-27T08:01:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 client.20877 shutdown 2024-05-27T08:01:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 -- 172.21.15.63:0/1840977133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 msgr2=0x55603a667610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a667610 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f1164009620 tx=0x7f116403e040 comp rx=0 tx=0).stop 2024-05-27T08:01:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 -- 172.21.15.63:0/1840977133 shutdown_connections 2024-05-27T08:01:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55603a278d80 0x55603a678bb0 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2ba1b0 0x55603a667610 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2994d0 0x55603a667090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 --2- 172.21.15.63:0/1840977133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2912f0 0x55603a666b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 -- 172.21.15.63:0/1840977133 >> 172.21.15.63:0/1840977133 conn(0x55603a2ae300 msgr2=0x55603a65c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1154c00640 0 client.20877 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 -- 172.21.15.63:0/1840977133 shutdown_connections 2024-05-27T08:01:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.870+0000 7f1178b91740 1 -- 172.21.15.63:0/1840977133 wait complete. 2024-05-27T08:01:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.871+0000 7f1178b91740 20 client.20877 trim_cache size 0 max 16384 2024-05-27T08:01:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.890+0000 7f1178b91740 1 Processor -- start 2024-05-27T08:01:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.890+0000 7f1178b91740 1 -- start start 2024-05-27T08:01:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.890+0000 7f1178b91740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2912f0 0x55603a6582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.890+0000 7f1178b91740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2994d0 0x55603a669280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.891+0000 7f1178b91740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2ba1b0 0x55603a6727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.891+0000 7f1178b91740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1b5c50 con 0x55603a2912f0 2024-05-27T08:01:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.891+0000 7f1178b91740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1b3520 con 0x55603a2994d0 2024-05-27T08:01:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.891+0000 7f1178b91740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1b1390 con 0x55603a2ba1b0 2024-05-27T08:01:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.891+0000 7f116b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2912f0 0x55603a6582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.891+0000 7f116b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2912f0 0x55603a6582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49216/0 (socket says 172.21.15.63:49216) 2024-05-27T08:01:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.891+0000 7f116b400640 1 -- 172.21.15.63:0/2481801821 learned_addr learned my addr 172.21.15.63:0/2481801821 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.891+0000 7f116a000640 1 --2- 172.21.15.63:0/2481801821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2994d0 0x55603a669280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.891+0000 7f116b400640 1 -- 172.21.15.63:0/2481801821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2ba1b0 msgr2=0x55603a6727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.892+0000 7f116b400640 1 --2- 172.21.15.63:0/2481801821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2ba1b0 0x55603a6727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.892+0000 7f116b400640 1 -- 172.21.15.63:0/2481801821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2994d0 msgr2=0x55603a669280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.892+0000 7f116b400640 1 --2- 172.21.15.63:0/2481801821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2994d0 0x55603a669280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.892+0000 7f116b400640 1 -- 172.21.15.63:0/2481801821 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55603a6650a0 con 0x55603a2912f0 2024-05-27T08:01:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.892+0000 7f116b400640 1 --2- 172.21.15.63:0/2481801821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2912f0 0x55603a6582e0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f1164030ac0 tx=0x7f1164005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b717d6242124cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.892+0000 7f1157400640 1 -- 172.21.15.63:0/2481801821 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164059070 con 0x55603a2912f0 2024-05-27T08:01:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.892+0000 7f1157400640 1 -- 172.21.15.63:0/2481801821 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1164036040 con 0x55603a2912f0 2024-05-27T08:01:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.892+0000 7f1157400640 1 -- 172.21.15.63:0/2481801821 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164004450 con 0x55603a2912f0 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.893+0000 7f1178b91740 1 -- 172.21.15.63:0/2481801821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2912f0 msgr2=0x55603a6582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.893+0000 7f1178b91740 1 --2- 172.21.15.63:0/2481801821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2912f0 0x55603a6582e0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f1164030ac0 tx=0x7f1164005fd0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.894+0000 7f1178b91740 1 -- 172.21.15.63:0/2481801821 shutdown_connections 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.894+0000 7f1178b91740 1 --2- 172.21.15.63:0/2481801821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2ba1b0 0x55603a6727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.894+0000 7f1178b91740 1 --2- 172.21.15.63:0/2481801821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2994d0 0x55603a669280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.894+0000 7f1178b91740 1 --2- 172.21.15.63:0/2481801821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2912f0 0x55603a6582e0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.894+0000 7f1178b91740 1 -- 172.21.15.63:0/2481801821 >> 172.21.15.63:0/2481801821 conn(0x55603a2ae300 msgr2=0x55603a661300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.894+0000 7f1178b91740 1 -- 172.21.15.63:0/2481801821 shutdown_connections 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.894+0000 7f1178b91740 1 -- 172.21.15.63:0/2481801821 wait complete. 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.894+0000 7f1178b91740 1 -- start start 2024-05-27T08:01:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.895+0000 7f1178b91740 1 Processor -- start 2024-05-27T08:01:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.895+0000 7f1178b91740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2912f0 0x55603a67bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.895+0000 7f1178b91740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2994d0 0x55603a67c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.895+0000 7f1178b91740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2ba1b0 0x55603a679060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.895+0000 7f1178b91740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1b19e0 con 0x55603a2994d0 2024-05-27T08:01:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.895+0000 7f1178b91740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1b2e50 con 0x55603a2ba1b0 2024-05-27T08:01:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.895+0000 7f1178b91740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1b20b0 con 0x55603a2912f0 2024-05-27T08:01:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.895+0000 7f1178b91740 10 client.0 _ll_register_callbacks cb 0x7f1164009430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.896+0000 7f116a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2994d0 0x55603a67c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.896+0000 7f116a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2994d0 0x55603a67c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49232/0 (socket says 172.21.15.63:49232) 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.896+0000 7f116a000640 1 -- 172.21.15.63:0/53412134 learned_addr learned my addr 172.21.15.63:0/53412134 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.896+0000 7f116b400640 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2912f0 0x55603a67bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.896+0000 7f116b400640 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2ba1b0 0x55603a679060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.896+0000 7f116a000640 1 -- 172.21.15.63:0/53412134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2912f0 msgr2=0x55603a67bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.896+0000 7f116a000640 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2912f0 0x55603a67bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.896+0000 7f116a000640 1 -- 172.21.15.63:0/53412134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2ba1b0 msgr2=0x55603a679060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.896+0000 7f116a000640 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2ba1b0 0x55603a679060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.897+0000 7f116a000640 1 -- 172.21.15.63:0/53412134 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55603a679750 con 0x55603a2994d0 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.897+0000 7f116a000640 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2994d0 0x55603a67c150 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f116c1049c0 tx=0x7f1158012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f4025cf7bff3a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.897+0000 7f1154c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.897+0000 7f1154c00640 1 -- 172.21.15.63:0/53412134 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158005220 con 0x55603a2994d0 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.897+0000 7f1154c00640 1 -- 172.21.15.63:0/53412134 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f11580053c0 con 0x55603a2994d0 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.897+0000 7f1178b91740 1 -- 172.21.15.63:0/53412134 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1164036040 con 0x55603a2994d0 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.898+0000 7f1154c00640 1 -- 172.21.15.63:0/53412134 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f115802e050 con 0x55603a2994d0 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.898+0000 7f1178b91740 10 client.20883 fetch_fsmap learned FSMap version 35 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.898+0000 7f1178b91740 1 -- 172.21.15.63:0/53412134 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55603a6e09e0 con 0x55603a2994d0 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.898+0000 7f1154c00640 1 -- 172.21.15.63:0/53412134 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1164036040 con 0x55603a2994d0 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.898+0000 7f1154c00640 1 -- 172.21.15.63:0/53412134 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1158025070 con 0x55603a2994d0 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.898+0000 7f1178b91740 10 client.20883 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.898+0000 7f1178b91740 10 client.20883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.898+0000 7f1178b91740 20 client.20883 populate_metadata read hostname 'smithi063' 2024-05-27T08:01:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.899+0000 7f1178b91740 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55603a24e1c0 0x55603a6e31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.899+0000 7f1178b91740 4 client.20883 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:01:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.899+0000 7f1178b91740 1 -- 172.21.15.63:0/53412134 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55603a5c92c0 con 0x55603a24e1c0 2024-05-27T08:01:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.899+0000 7f116b400640 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55603a24e1c0 0x55603a6e31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.900+0000 7f116b400640 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55603a24e1c0 0x55603a6e31d0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db8d7ed2ded534b server_cookie=eb3a686862536b21 in_seq=0 out_seq=0 2024-05-27T08:01:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.900+0000 7f1154c00640 10 client.20883 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1154c00640 1 -- 172.21.15.63:0/53412134 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55603a5c92c0 con 0x55603a24e1c0 2024-05-27T08:01:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1154c00640 10 client.20883 handle_command_reply: tid=0 2024-05-27T08:01:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 client.20883 shutdown 2024-05-27T08:01:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 -- 172.21.15.63:0/53412134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2994d0 msgr2=0x55603a67c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2994d0 0x55603a67c150 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f116c1049c0 tx=0x7f1158012820 comp rx=0 tx=0).stop 2024-05-27T08:01:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 -- 172.21.15.63:0/53412134 shutdown_connections 2024-05-27T08:01:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55603a24e1c0 0x55603a6e31d0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55603a2ba1b0 0x55603a679060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55603a2994d0 0x55603a67c150 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 --2- 172.21.15.63:0/53412134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55603a2912f0 0x55603a67bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1154c00640 0 client.20883 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:01:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 -- 172.21.15.63:0/53412134 >> 172.21.15.63:0/53412134 conn(0x55603a2ae300 msgr2=0x55603a672550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 -- 172.21.15.63:0/53412134 shutdown_connections 2024-05-27T08:01:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 1 -- 172.21.15.63:0/53412134 wait complete. 2024-05-27T08:01:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.941+0000 7f1178b91740 20 client.20883 trim_cache size 0 max 16384 2024-05-27T08:01:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:01:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:01:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.942+0000 7f116be00640 1 -- 172.21.15.63:0/2458169870 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11500b29d0 msgr2=0x7f11500b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.942+0000 7f116be00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11500b29d0 0x7f11500b4ee0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f115c0dfa30 tx=0x7f115c00bf60 comp rx=0 tx=0).stop 2024-05-27T08:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.942+0000 7f116be00640 1 -- 172.21.15.63:0/2458169870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c101fe0 msgr2=0x7f116c19d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.942+0000 7f116be00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c101fe0 0x7f116c19d480 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f115c005fd0 tx=0x7f115c00dde0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.942+0000 7f116be00640 1 -- 172.21.15.63:0/2458169870 shutdown_connections 2024-05-27T08:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.942+0000 7f116be00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f11500b29d0 0x7f11500b4ee0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.942+0000 7f116be00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f116c10e4e0 0x7f116c1a8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.942+0000 7f116be00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f116c109b30 0x7f116c19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.942+0000 7f116be00640 1 --2- 172.21.15.63:0/2458169870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c101fe0 0x7f116c19d480 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.942+0000 7f116be00640 1 -- 172.21.15.63:0/2458169870 >> 172.21.15.63:0/2458169870 conn(0x7f116c0fd950 msgr2=0x7f116c102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.943+0000 7f116be00640 1 -- 172.21.15.63:0/2458169870 shutdown_connections 2024-05-27T08:01:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:01:59.943+0000 7f116be00640 1 -- 172.21.15.63:0/2458169870 wait complete. 2024-05-27T08:02:01.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.220+0000 7f03fbe00700 1 -- 172.21.15.63:0/3383320611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03f4109860 msgr2=0x7f03f41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:01.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.220+0000 7f03fbe00700 1 --2- 172.21.15.63:0/3383320611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03f4109860 0x7f03f41111c0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00b0d0 tx=0x7f03ec00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.221+0000 7f03fbe00700 1 -- 172.21.15.63:0/3383320611 shutdown_connections 2024-05-27T08:02:01.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.221+0000 7f03fbe00700 1 --2- 172.21.15.63:0/3383320611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03f4109860 0x7f03f41111c0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.221+0000 7f03fbe00700 1 --2- 172.21.15.63:0/3383320611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03f41045e0 0x7f03f41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.221+0000 7f03fbe00700 1 --2- 172.21.15.63:0/3383320611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f4108f10 0x7f03f4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.221+0000 7f03fbe00700 1 -- 172.21.15.63:0/3383320611 >> 172.21.15.63:0/3383320611 conn(0x7f03f4100030 msgr2=0x7f03f4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:01.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.221+0000 7f03fbe00700 1 -- 172.21.15.63:0/3383320611 shutdown_connections 2024-05-27T08:02:01.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.221+0000 7f03fbe00700 1 -- 172.21.15.63:0/3383320611 wait complete. 2024-05-27T08:02:01.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.223+0000 7f03fbe00700 1 Processor -- start 2024-05-27T08:02:01.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.223+0000 7f03fbe00700 1 -- start start 2024-05-27T08:02:01.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.223+0000 7f03fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f41045e0 0x7f03f4075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.223+0000 7f03fbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03f4108f10 0x7f03f4075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.223+0000 7f03fbe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03f4109860 0x7f03f4076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.223+0000 7f03fbe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4114390 con 0x7f03f4108f10 2024-05-27T08:02:01.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.223+0000 7f03fbe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4114210 con 0x7f03f41045e0 2024-05-27T08:02:01.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.223+0000 7f03fbe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4114510 con 0x7f03f4109860 2024-05-27T08:02:01.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.224+0000 7f03f9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f41045e0 0x7f03f4075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.224+0000 7f03f8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03f4108f10 0x7f03f4075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.224+0000 7f03f8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03f4108f10 0x7f03f4075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49252/0 (socket says 172.21.15.63:49252) 2024-05-27T08:02:01.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.224+0000 7f03f8a00700 1 -- 172.21.15.63:0/733001434 learned_addr learned my addr 172.21.15.63:0/733001434 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:01.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.224+0000 7f03f9e00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03f4109860 0x7f03f4076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.225+0000 7f03f8a00700 1 -- 172.21.15.63:0/733001434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03f4109860 msgr2=0x7f03f4076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:01.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.225+0000 7f03f8a00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03f4109860 0x7f03f4076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.225+0000 7f03f8a00700 1 -- 172.21.15.63:0/733001434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f41045e0 msgr2=0x7f03f4075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:01.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.225+0000 7f03f8a00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f41045e0 0x7f03f4075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.225+0000 7f03f8a00700 1 -- 172.21.15.63:0/733001434 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f41a90e0 con 0x7f03f4108f10 2024-05-27T08:02:01.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.225+0000 7f03f8a00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03f4108f10 0x7f03f4075ff0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f03e000d980 tx=0x7f03e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:01.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.226+0000 7f03f2000700 1 -- 172.21.15.63:0/733001434 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0017070 con 0x7f03f4108f10 2024-05-27T08:02:01.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.226+0000 7f03f2000700 1 -- 172.21.15.63:0/733001434 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03e000fd20 con 0x7f03f4108f10 2024-05-27T08:02:01.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.226+0000 7f03f2000700 1 -- 172.21.15.63:0/733001434 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e000e3b0 con 0x7f03f4108f10 2024-05-27T08:02:01.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.226+0000 7f03fbe00700 1 -- 172.21.15.63:0/733001434 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f41a93d0 con 0x7f03f4108f10 2024-05-27T08:02:01.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.226+0000 7f03fbe00700 1 -- 172.21.15.63:0/733001434 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f41a9960 con 0x7f03f4108f10 2024-05-27T08:02:01.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.231+0000 7f03f2000700 1 -- 172.21.15.63:0/733001434 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f03e000e550 con 0x7f03f4108f10 2024-05-27T08:02:01.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.232+0000 7f040a9a2b80 1 Processor -- start 2024-05-27T08:02:01.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.232+0000 7f040a9a2b80 1 -- start start 2024-05-27T08:02:01.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 0x55fc47974330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc477f5e40 0x55fc47974870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc47981110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fc478a2e40 con 0x55fc47974e70 2024-05-27T08:02:01.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fc478929e0 con 0x55fc478e19f0 2024-05-27T08:02:01.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fc47896a20 con 0x55fc477f5e40 2024-05-27T08:02:01.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f03f8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc47981110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f03f9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 0x55fc47974330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f03f2000700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03dc0b2a70 0x7f03dc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f03f8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc47981110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49268/0 (socket says 172.21.15.63:49268) 2024-05-27T08:02:01.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f03f8a00700 1 -- 172.21.15.63:0/1710484106 learned_addr learned my addr 172.21.15.63:0/1710484106 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:01.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.233+0000 7f03f9400700 1 --2- 172.21.15.63:0/1710484106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc477f5e40 0x55fc47974870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.234+0000 7f03f9e00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03dc0b2a70 0x7f03dc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.234+0000 7f03f2000700 1 -- 172.21.15.63:0/733001434 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f03e00e23f0 con 0x7f03f4108f10 2024-05-27T08:02:01.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.234+0000 7f03f8a00700 1 -- 172.21.15.63:0/1710484106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc477f5e40 msgr2=0x55fc47974870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:01.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.234+0000 7f03f8a00700 1 --2- 172.21.15.63:0/1710484106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc477f5e40 0x55fc47974870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.234+0000 7f03f8a00700 1 -- 172.21.15.63:0/1710484106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 msgr2=0x55fc47974330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:01.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.234+0000 7f03f8a00700 1 --2- 172.21.15.63:0/1710484106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 0x55fc47974330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.234+0000 7f03f8a00700 1 -- 172.21.15.63:0/1710484106 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4785b420 con 0x55fc47974e70 2024-05-27T08:02:01.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.235+0000 7f03f8a00700 1 --2- 172.21.15.63:0/1710484106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc47981110 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f03e00df690 tx=0x7f03e0010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f17f77a80930385 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:01.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.235+0000 7f03e6a00700 1 -- 172.21.15.63:0/1710484106 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0020070 con 0x55fc47974e70 2024-05-27T08:02:01.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.235+0000 7f03e6a00700 1 -- 172.21.15.63:0/1710484106 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03e00a8dc0 con 0x55fc47974e70 2024-05-27T08:02:01.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.235+0000 7f03e6a00700 1 -- 172.21.15.63:0/1710484106 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e00a9770 con 0x55fc47974e70 2024-05-27T08:02:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.236+0000 7f040a9a2b80 1 -- 172.21.15.63:0/1710484106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 msgr2=0x55fc47981110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.236+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/1710484106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc47981110 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f03e00df690 tx=0x7f03e0010f50 comp rx=0 tx=0).stop 2024-05-27T08:02:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.236+0000 7f03f9e00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03dc0b2a70 0x7f03dc0b4f80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009e60 tx=0x7f03ec00b3e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.236+0000 7f040a9a2b80 1 -- 172.21.15.63:0/1710484106 shutdown_connections 2024-05-27T08:02:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.236+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/1710484106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc47981110 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.236+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/1710484106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 0x55fc47974330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.236+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/1710484106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc477f5e40 0x55fc47974870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.236+0000 7f040a9a2b80 1 -- 172.21.15.63:0/1710484106 >> 172.21.15.63:0/1710484106 conn(0x55fc4788f390 msgr2=0x55fc479704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.236+0000 7f040a9a2b80 1 -- 172.21.15.63:0/1710484106 shutdown_connections 2024-05-27T08:02:01.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.236+0000 7f040a9a2b80 1 -- 172.21.15.63:0/1710484106 wait complete. 2024-05-27T08:02:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.237+0000 7f040a9a2b80 1 -- start start 2024-05-27T08:02:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.237+0000 7f040a9a2b80 1 Processor -- start 2024-05-27T08:02:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.238+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc477f5e40 0x55fc4797e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.238+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc478e19f0 0x55fc4797ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.238+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc4797f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.238+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fc47798fa0 con 0x55fc47974e70 2024-05-27T08:02:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.238+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fc47833760 con 0x55fc477f5e40 2024-05-27T08:02:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.238+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fc47981ac0 con 0x55fc478e19f0 2024-05-27T08:02:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.238+0000 7f040a9a2b80 10 client.0 _ll_register_callbacks cb 0x55fc4783cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.238+0000 7f03f9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc4797f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.239+0000 7f03f9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc4797f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49280/0 (socket says 172.21.15.63:49280) 2024-05-27T08:02:01.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.239+0000 7f03f9e00700 1 -- 172.21.15.63:0/3103412064 learned_addr learned my addr 172.21.15.63:0/3103412064 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:01.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.239+0000 7f03f8a00700 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc478e19f0 0x55fc4797ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.239+0000 7f03f9400700 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc477f5e40 0x55fc4797e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.239+0000 7f03f9e00700 1 -- 172.21.15.63:0/3103412064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc478e19f0 msgr2=0x55fc4797ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.239+0000 7f03f9e00700 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc478e19f0 0x55fc4797ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.239+0000 7f03f9e00700 1 -- 172.21.15.63:0/3103412064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc477f5e40 msgr2=0x55fc4797e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.239+0000 7f03f9e00700 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc477f5e40 0x55fc4797e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.239+0000 7f03f9e00700 1 -- 172.21.15.63:0/3103412064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc477e28d0 con 0x55fc47974e70 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.240+0000 7f03f9e00700 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc4797f130 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f03ec015000 tx=0x7f03ec0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec24cfde9906b62d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.240+0000 7f03db400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.240+0000 7f03db400700 1 -- 172.21.15.63:0/3103412064 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec026040 con 0x55fc47974e70 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.240+0000 7f03db400700 1 -- 172.21.15.63:0/3103412064 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f03ec005410 con 0x55fc47974e70 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.240+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3103412064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e00a8dc0 con 0x55fc47974e70 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.240+0000 7f03db400700 1 -- 172.21.15.63:0/3103412064 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f03ec038050 con 0x55fc47974e70 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.240+0000 7f040a9a2b80 10 client.20895 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:01.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.240+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3103412064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fc47863060 con 0x55fc47974e70 2024-05-27T08:02:01.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.241+0000 7f03db400700 1 -- 172.21.15.63:0/3103412064 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03e00a8dc0 con 0x55fc47974e70 2024-05-27T08:02:01.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.241+0000 7f03db400700 1 -- 172.21.15.63:0/3103412064 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f03ec029030 con 0x55fc47974e70 2024-05-27T08:02:01.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.241+0000 7f040a9a2b80 10 client.20895 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:01.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.241+0000 7f040a9a2b80 10 client.20895 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:02:01.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.241+0000 7f040a9a2b80 20 client.20895 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.241+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55fc4798ead0 0x55fc47990f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.241+0000 7f040a9a2b80 4 client.20895 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.241+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3103412064 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc478a4d90 con 0x55fc4798ead0 2024-05-27T08:02:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.242+0000 7f03f9400700 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55fc4798ead0 0x55fc47990f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.243+0000 7f03f9400700 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55fc4798ead0 0x55fc47990f10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d6c615c1727f114 server_cookie=911c777a036057a8 in_seq=0 out_seq=0 2024-05-27T08:02:01.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.243+0000 7f03db400700 10 client.20895 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:02:01.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.245+0000 7f03db400700 1 -- 172.21.15.63:0/3103412064 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fc478a4d90 con 0x55fc4798ead0 2024-05-27T08:02:01.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.245+0000 7f03db400700 10 client.20895 handle_command_reply: tid=0 2024-05-27T08:02:01.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.245+0000 7f040a9a2b80 1 client.20895 shutdown 2024-05-27T08:02:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.245+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3103412064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 msgr2=0x55fc4797f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.245+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc4797f130 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f03ec015000 tx=0x7f03ec0099e0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.246+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3103412064 shutdown_connections 2024-05-27T08:02:01.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.246+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55fc4798ead0 0x55fc47990f10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.246+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc47974e70 0x55fc4797f130 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.246+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc477f5e40 0x55fc4797e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.246+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3103412064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc478e19f0 0x55fc4797ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.246+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3103412064 >> 172.21.15.63:0/3103412064 conn(0x55fc4788f390 msgr2=0x55fc47974350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.246+0000 7f03db400700 0 client.20895 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:02:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.246+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3103412064 shutdown_connections 2024-05-27T08:02:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.246+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3103412064 wait complete. 2024-05-27T08:02:01.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.246+0000 7f040a9a2b80 20 client.20895 trim_cache size 0 max 16384 2024-05-27T08:02:01.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.268+0000 7f040a9a2b80 1 Processor -- start 2024-05-27T08:02:01.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.268+0000 7f040a9a2b80 1 -- start start 2024-05-27T08:02:01.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.268+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc477f5e40 0x55fc47979e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.268+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 0x55fc47976900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.268+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc47974e70 0x55fc47974790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.268+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41b5140 con 0x55fc477f5e40 2024-05-27T08:02:01.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.268+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41b29d0 con 0x55fc478e19f0 2024-05-27T08:02:01.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.268+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41b0840 con 0x55fc47974e70 2024-05-27T08:02:01.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.269+0000 7f03f9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc477f5e40 0x55fc47979e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.269+0000 7f03f9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc477f5e40 0x55fc47979e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49296/0 (socket says 172.21.15.63:49296) 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.269+0000 7f03f9400700 1 -- 172.21.15.63:0/24615394 learned_addr learned my addr 172.21.15.63:0/24615394 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.269+0000 7f03f8a00700 1 --2- 172.21.15.63:0/24615394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 0x55fc47976900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.269+0000 7f03f9400700 1 -- 172.21.15.63:0/24615394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc47974e70 msgr2=0x55fc47974790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.269+0000 7f03f9400700 1 --2- 172.21.15.63:0/24615394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc47974e70 0x55fc47974790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.269+0000 7f03f9400700 1 -- 172.21.15.63:0/24615394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 msgr2=0x55fc47976900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.269+0000 7f03f9400700 1 --2- 172.21.15.63:0/24615394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 0x55fc47976900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.269+0000 7f03f9400700 1 -- 172.21.15.63:0/24615394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4796c240 con 0x55fc477f5e40 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.269+0000 7f03f8a00700 1 --2- 172.21.15.63:0/24615394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 0x55fc47976900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.270+0000 7f03f9400700 1 --2- 172.21.15.63:0/24615394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc477f5e40 0x55fc47979e90 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f03e8012770 tx=0x7f03e8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc31c3b31d95e5a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.270+0000 7f03e6000700 1 -- 172.21.15.63:0/24615394 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e80120d0 con 0x55fc477f5e40 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.270+0000 7f03e6000700 1 -- 172.21.15.63:0/24615394 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03e8012270 con 0x55fc477f5e40 2024-05-27T08:02:01.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.270+0000 7f03e6000700 1 -- 172.21.15.63:0/24615394 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8028680 con 0x55fc477f5e40 2024-05-27T08:02:01.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.271+0000 7f040a9a2b80 1 -- 172.21.15.63:0/24615394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc477f5e40 msgr2=0x55fc47979e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:01.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.271+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/24615394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc477f5e40 0x55fc47979e90 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f03e8012770 tx=0x7f03e8009260 comp rx=0 tx=0).stop 2024-05-27T08:02:01.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.272+0000 7f040a9a2b80 1 -- 172.21.15.63:0/24615394 shutdown_connections 2024-05-27T08:02:01.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.272+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/24615394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc47974e70 0x55fc47974790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.272+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/24615394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc477f5e40 0x55fc47979e90 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.272+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/24615394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc478e19f0 0x55fc47976900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.272+0000 7f040a9a2b80 1 -- 172.21.15.63:0/24615394 >> 172.21.15.63:0/24615394 conn(0x55fc4788f390 msgr2=0x55fc4797e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:01.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.272+0000 7f040a9a2b80 1 -- 172.21.15.63:0/24615394 shutdown_connections 2024-05-27T08:02:01.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.272+0000 7f040a9a2b80 1 -- 172.21.15.63:0/24615394 wait complete. 2024-05-27T08:02:01.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.272+0000 7f040a9a2b80 1 -- start start 2024-05-27T08:02:01.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.273+0000 7f040a9a2b80 1 Processor -- start 2024-05-27T08:02:01.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.273+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc477f5e40 0x55fc47978560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.273+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc478e19f0 0x55fc47978aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.273+0000 7f040a9a2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc47974e70 0x55fc47978fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.273+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41b0e90 con 0x55fc478e19f0 2024-05-27T08:02:01.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.273+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41b2300 con 0x55fc477f5e40 2024-05-27T08:02:01.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.273+0000 7f040a9a2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41b1560 con 0x55fc47974e70 2024-05-27T08:02:01.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.273+0000 7f040a9a2b80 10 client.0 _ll_register_callbacks cb 0x55fc4783cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:01.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc478e19f0 0x55fc47978aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc478e19f0 0x55fc47978aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49300/0 (socket says 172.21.15.63:49300) 2024-05-27T08:02:01.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f8a00700 1 -- 172.21.15.63:0/3855497436 learned_addr learned my addr 172.21.15.63:0/3855497436 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:01.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f9400700 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc477f5e40 0x55fc47978560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f9e00700 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc47974e70 0x55fc47978fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f8a00700 1 -- 172.21.15.63:0/3855497436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc47974e70 msgr2=0x55fc47978fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:01.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f8a00700 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc47974e70 0x55fc47978fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f8a00700 1 -- 172.21.15.63:0/3855497436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc477f5e40 msgr2=0x55fc47978560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:01.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f8a00700 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc477f5e40 0x55fc47978560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f8a00700 1 -- 172.21.15.63:0/3855497436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc47985ec0 con 0x55fc478e19f0 2024-05-27T08:02:01.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03f8a00700 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc478e19f0 0x55fc47978aa0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f03e00a8fb0 tx=0x7f03e00a9870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b24965c91be9b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:01.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.274+0000 7f03db400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:01.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.275+0000 7f03db400700 1 -- 172.21.15.63:0/3855497436 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e00e6020 con 0x55fc478e19f0 2024-05-27T08:02:01.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.275+0000 7f03db400700 1 -- 172.21.15.63:0/3855497436 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f03e00b1c80 con 0x55fc478e19f0 2024-05-27T08:02:01.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.275+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3855497436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e8012270 con 0x55fc478e19f0 2024-05-27T08:02:01.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.275+0000 7f03db400700 1 -- 172.21.15.63:0/3855497436 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f03e0020040 con 0x55fc478e19f0 2024-05-27T08:02:01.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.275+0000 7f040a9a2b80 10 client.20901 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:01.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.275+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3855497436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fc47987120 con 0x55fc478e19f0 2024-05-27T08:02:01.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.276+0000 7f03db400700 1 -- 172.21.15.63:0/3855497436 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03e8012270 con 0x55fc478e19f0 2024-05-27T08:02:01.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.276+0000 7f03db400700 1 -- 172.21.15.63:0/3855497436 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f03e000f890 con 0x55fc478e19f0 2024-05-27T08:02:01.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.276+0000 7f040a9a2b80 10 client.20901 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:01.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.276+0000 7f040a9a2b80 10 client.20901 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:02:01.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.276+0000 7f040a9a2b80 20 client.20901 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:01.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.276+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55fc47987460 0x55fc479fb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:01.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.276+0000 7f040a9a2b80 4 client.20901 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:02:01.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.276+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3855497436 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc4789e570 con 0x55fc47987460 2024-05-27T08:02:01.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.277+0000 7f03f9400700 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55fc47987460 0x55fc479fb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:01.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.278+0000 7f03f9400700 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55fc47987460 0x55fc479fb5b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f460acda1c74dc0 server_cookie=89c5afa2fc0786fb in_seq=0 out_seq=0 2024-05-27T08:02:01.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.278+0000 7f03db400700 10 client.20901 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:02:01.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.278+0000 7f03db400700 1 -- 172.21.15.63:0/3855497436 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc4789e570 con 0x55fc47987460 2024-05-27T08:02:01.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.278+0000 7f03db400700 10 client.20901 handle_command_reply: tid=0 2024-05-27T08:02:01.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.278+0000 7f040a9a2b80 1 client.20901 shutdown 2024-05-27T08:02:01.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.278+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3855497436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc478e19f0 msgr2=0x55fc47978aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:01.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.278+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc478e19f0 0x55fc47978aa0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f03e00a8fb0 tx=0x7f03e00a9870 comp rx=0 tx=0).stop 2024-05-27T08:02:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.279+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3855497436 shutdown_connections 2024-05-27T08:02:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.279+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc47974e70 0x55fc47978fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.279+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc477f5e40 0x55fc47978560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.279+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55fc47987460 0x55fc479fb5b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.279+0000 7f040a9a2b80 1 --2- 172.21.15.63:0/3855497436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc478e19f0 0x55fc47978aa0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.279+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3855497436 >> 172.21.15.63:0/3855497436 conn(0x55fc4788f390 msgr2=0x55fc4797eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.279+0000 7f03db400700 0 client.20901 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:02:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.279+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3855497436 shutdown_connections 2024-05-27T08:02:01.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.279+0000 7f040a9a2b80 1 -- 172.21.15.63:0/3855497436 wait complete. 2024-05-27T08:02:01.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.279+0000 7f040a9a2b80 20 client.20901 trim_cache size 0 max 16384 2024-05-27T08:02:01.283 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:02:01.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.280+0000 7f03fbe00700 1 -- 172.21.15.63:0/733001434 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03dc0b2a70 msgr2=0x7f03dc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:01.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.280+0000 7f03fbe00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03dc0b2a70 0x7f03dc0b4f80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009e60 tx=0x7f03ec00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.280+0000 7f03fbe00700 1 -- 172.21.15.63:0/733001434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03f4108f10 msgr2=0x7f03f4075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.280+0000 7f03fbe00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03f4108f10 0x7f03f4075ff0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f03e000d980 tx=0x7f03e000dc90 comp rx=0 tx=0).stop 2024-05-27T08:02:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.280+0000 7f03fbe00700 1 -- 172.21.15.63:0/733001434 shutdown_connections 2024-05-27T08:02:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.280+0000 7f03fbe00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f03dc0b2a70 0x7f03dc0b4f80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.280+0000 7f03fbe00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03f4109860 0x7f03f4076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.280+0000 7f03fbe00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03f41045e0 0x7f03f4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.280+0000 7f03fbe00700 1 --2- 172.21.15.63:0/733001434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03f4108f10 0x7f03f4075ff0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.280+0000 7f03fbe00700 1 -- 172.21.15.63:0/733001434 >> 172.21.15.63:0/733001434 conn(0x7f03f4100030 msgr2=0x7f03f4108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.281+0000 7f03fbe00700 1 -- 172.21.15.63:0/733001434 shutdown_connections 2024-05-27T08:02:01.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:01.281+0000 7f03fbe00700 1 -- 172.21.15.63:0/733001434 wait complete. 2024-05-27T08:02:01.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:01 smithi086 ceph-mon[34687]: pgmap v901: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T08:02:01.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:01 smithi149 ceph-mon[35408]: pgmap v901: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T08:02:01.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:01 smithi063 ceph-mon[31613]: pgmap v901: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 7 op/s 2024-05-27T08:02:02.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:02:02.804 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:03.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:03 smithi086 ceph-mon[34687]: pgmap v902: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 32 op/s 2024-05-27T08:02:03.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:03 smithi149 ceph-mon[35408]: pgmap v902: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 32 op/s 2024-05-27T08:02:03.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:03 smithi063 ceph-mon[31613]: pgmap v902: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 32 op/s 2024-05-27T08:02:05.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:05 smithi063 ceph-mon[31613]: pgmap v903: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 29 op/s 2024-05-27T08:02:05.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:05 smithi086 ceph-mon[34687]: pgmap v903: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 29 op/s 2024-05-27T08:02:05.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:05 smithi149 ceph-mon[35408]: pgmap v903: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 29 op/s 2024-05-27T08:02:06.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.242+0000 7f1d79a00700 1 -- 172.21.15.63:0/361446921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d741043f0 msgr2=0x7f1d74104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:06.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.242+0000 7f1d79a00700 1 --2- 172.21.15.63:0/361446921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d741043f0 0x7f1d74104800 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f1d640099c0 tx=0x7f1d64009cd0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.242+0000 7f1d79a00700 1 -- 172.21.15.63:0/361446921 shutdown_connections 2024-05-27T08:02:06.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.242+0000 7f1d79a00700 1 --2- 172.21.15.63:0/361446921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d74109670 0x7f1d74110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.242+0000 7f1d79a00700 1 --2- 172.21.15.63:0/361446921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d741043f0 0x7f1d74104800 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.242+0000 7f1d79a00700 1 --2- 172.21.15.63:0/361446921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d74108d20 0x7f1d74109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.242+0000 7f1d79a00700 1 -- 172.21.15.63:0/361446921 >> 172.21.15.63:0/361446921 conn(0x7f1d74079d00 msgr2=0x7f1d7407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:06.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.242+0000 7f1d79a00700 1 -- 172.21.15.63:0/361446921 shutdown_connections 2024-05-27T08:02:06.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.243+0000 7f1d79a00700 1 -- 172.21.15.63:0/361446921 wait complete. 2024-05-27T08:02:06.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.244+0000 7f1d79a00700 1 Processor -- start 2024-05-27T08:02:06.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.244+0000 7f1d79a00700 1 -- start start 2024-05-27T08:02:06.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.244+0000 7f1d79a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d741043f0 0x7f1d7419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d79a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d74108d20 0x7f1d7419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d79a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d74109670 0x7f1d741a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d79a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741141a0 con 0x7f1d741043f0 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d79a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74114020 con 0x7f1d74109670 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d79a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74114320 con 0x7f1d74108d20 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d73e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d741043f0 0x7f1d7419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d73e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d741043f0 0x7f1d7419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49326/0 (socket says 172.21.15.63:49326) 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d73e00700 1 -- 172.21.15.63:0/3575508285 learned_addr learned my addr 172.21.15.63:0/3575508285 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d73400700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d74108d20 0x7f1d7419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d79000700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d74109670 0x7f1d741a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d73e00700 1 -- 172.21.15.63:0/3575508285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d74108d20 msgr2=0x7f1d7419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.245+0000 7f1d73e00700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d74108d20 0x7f1d7419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.246+0000 7f1d73e00700 1 -- 172.21.15.63:0/3575508285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d74109670 msgr2=0x7f1d741a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:06.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.246+0000 7f1d73e00700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d74109670 0x7f1d741a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.246+0000 7f1d73e00700 1 -- 172.21.15.63:0/3575508285 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d741a1d20 con 0x7f1d741043f0 2024-05-27T08:02:06.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.246+0000 7f1d79000700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d74109670 0x7f1d741a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:06.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.246+0000 7f1d73e00700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d741043f0 0x7f1d7419ce50 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400b580 tx=0x7f1d6400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:06.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.246+0000 7f1d70c00700 1 -- 172.21.15.63:0/3575508285 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64027070 con 0x7f1d741043f0 2024-05-27T08:02:06.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.246+0000 7f1d70c00700 1 -- 172.21.15.63:0/3575508285 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d6400fdd0 con 0x7f1d741043f0 2024-05-27T08:02:06.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.246+0000 7f1d70c00700 1 -- 172.21.15.63:0/3575508285 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640173f0 con 0x7f1d741043f0 2024-05-27T08:02:06.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.246+0000 7f1d79a00700 1 -- 172.21.15.63:0/3575508285 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d741a1fb0 con 0x7f1d741043f0 2024-05-27T08:02:06.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.247+0000 7f1d79a00700 1 -- 172.21.15.63:0/3575508285 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d741a9930 con 0x7f1d741043f0 2024-05-27T08:02:06.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.251+0000 7f1d70c00700 1 -- 172.21.15.63:0/3575508285 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1d6402a070 con 0x7f1d741043f0 2024-05-27T08:02:06.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.252+0000 7f1d89bf5b80 1 Processor -- start 2024-05-27T08:02:06.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.253+0000 7f1d89bf5b80 1 -- start start 2024-05-27T08:02:06.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.253+0000 7f1d70c00700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d580b2a70 0x7f1d580b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.253+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d22ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.253+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.253+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d30020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.253+0000 7f1d70c00700 1 -- 172.21.15.63:0/3575508285 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f1d640e3ae0 con 0x7f1d741043f0 2024-05-27T08:02:06.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.253+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5635c0c51e40 con 0x5635c0ba4e40 2024-05-27T08:02:06.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.253+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635c0c419e0 con 0x5635c0d23410 2024-05-27T08:02:06.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.253+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5635c0c45a20 con 0x5635c0d23820 2024-05-27T08:02:06.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.254+0000 7f1d79000700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d580b2a70 0x7f1d580b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.254+0000 7f1d73400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.254+0000 7f1d73e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d22ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.254+0000 7f1d73400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44038/0 (socket says 172.21.15.63:44038) 2024-05-27T08:02:06.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.254+0000 7f1d73e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d22ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49328/0 (socket says 172.21.15.63:49328) 2024-05-27T08:02:06.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.254+0000 7f1d73400700 1 -- 172.21.15.63:0/1534064891 learned_addr learned my addr 172.21.15.63:0/1534064891 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:06.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.255+0000 7f1d79000700 1 --2- 172.21.15.63:0/1534064891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d30020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.255+0000 7f1d73e00700 1 -- 172.21.15.63:0/1534064891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 msgr2=0x5635c0d30020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:06.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.255+0000 7f1d73e00700 1 --2- 172.21.15.63:0/1534064891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d30020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.255+0000 7f1d73e00700 1 -- 172.21.15.63:0/1534064891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 msgr2=0x5635c0d2dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:06.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.255+0000 7f1d73e00700 1 --2- 172.21.15.63:0/1534064891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.255+0000 7f1d73e00700 1 -- 172.21.15.63:0/1534064891 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c0c0a420 con 0x5635c0ba4e40 2024-05-27T08:02:06.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.255+0000 7f1d79000700 1 --2- 172.21.15.63:0/1534064891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d30020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:06.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.255+0000 7f1d79000700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d580b2a70 0x7f1d580b4f80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0094f0 tx=0x7f1d6c009130 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:06.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.255+0000 7f1d73e00700 1 --2- 172.21.15.63:0/1534064891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d22ed0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f1d640e0e10 tx=0x7f1d640aa930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc484a010ac74952 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:06.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.256+0000 7f1d69600700 1 -- 172.21.15.63:0/1534064891 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640aaa30 con 0x5635c0ba4e40 2024-05-27T08:02:06.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.256+0000 7f1d69600700 1 -- 172.21.15.63:0/1534064891 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d640aede0 con 0x5635c0ba4e40 2024-05-27T08:02:06.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.256+0000 7f1d69600700 1 -- 172.21.15.63:0/1534064891 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640cd140 con 0x5635c0ba4e40 2024-05-27T08:02:06.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.257+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/1534064891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 msgr2=0x5635c0d22ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:06.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.257+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/1534064891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d22ed0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f1d640e0e10 tx=0x7f1d640aa930 comp rx=0 tx=0).stop 2024-05-27T08:02:06.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.257+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/1534064891 shutdown_connections 2024-05-27T08:02:06.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.257+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/1534064891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d30020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.257+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/1534064891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d22ed0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.257+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/1534064891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.257+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/1534064891 >> 172.21.15.63:0/1534064891 conn(0x5635c0c04fc0 msgr2=0x5635c0d1ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:06.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.257+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/1534064891 shutdown_connections 2024-05-27T08:02:06.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.257+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/1534064891 wait complete. 2024-05-27T08:02:06.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.258+0000 7f1d89bf5b80 1 -- start start 2024-05-27T08:02:06.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.258+0000 7f1d89bf5b80 1 Processor -- start 2024-05-27T08:02:06.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.258+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d2d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.258+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d2e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5635c0b47fa0 con 0x5635c0ba4e40 2024-05-27T08:02:06.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635c0be2760 con 0x5635c0d23410 2024-05-27T08:02:06.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5635c0d30750 con 0x5635c0d23820 2024-05-27T08:02:06.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d89bf5b80 10 client.0 _ll_register_callbacks cb 0x5635c0bebc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:06.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d73e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d2d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d73e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d2d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49344/0 (socket says 172.21.15.63:49344) 2024-05-27T08:02:06.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d73e00700 1 -- 172.21.15.63:0/3785895119 learned_addr learned my addr 172.21.15.63:0/3785895119 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:06.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d73400700 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d79000700 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d2e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.259+0000 7f1d73e00700 1 -- 172.21.15.63:0/3785895119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 msgr2=0x5635c0d2e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:06.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d73e00700 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d2e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d73e00700 1 -- 172.21.15.63:0/3785895119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 msgr2=0x5635c0d2db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d73e00700 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d73e00700 1 -- 172.21.15.63:0/3785895119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c0b918d0 con 0x5635c0ba4e40 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d73400700 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d79000700 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d2e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d73e00700 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d2d650 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f1d640aa730 tx=0x7f1d640b1430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b861b169ca0950f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d5e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d5e000700 1 -- 172.21.15.63:0/3785895119 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640c3020 con 0x5635c0ba4e40 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d5e000700 1 -- 172.21.15.63:0/3785895119 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f1d640a7020 con 0x5635c0ba4e40 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.260+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/3785895119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d640aede0 con 0x5635c0ba4e40 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.261+0000 7f1d5e000700 1 -- 172.21.15.63:0/3785895119 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1d640aaa30 con 0x5635c0ba4e40 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.261+0000 7f1d89bf5b80 10 client.20913 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.261+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/3785895119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5635c0c12060 con 0x5635c0ba4e40 2024-05-27T08:02:06.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.261+0000 7f1d5e000700 1 -- 172.21.15.63:0/3785895119 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d640aede0 con 0x5635c0ba4e40 2024-05-27T08:02:06.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.261+0000 7f1d5e000700 1 -- 172.21.15.63:0/3785895119 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1d640cd040 con 0x5635c0ba4e40 2024-05-27T08:02:06.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.262+0000 7f1d89bf5b80 10 client.20913 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:06.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.262+0000 7f1d89bf5b80 10 client.20913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:06.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.262+0000 7f1d89bf5b80 20 client.20913 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:06.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.262+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635c0d3d4a0 0x5635c0d3f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.262+0000 7f1d89bf5b80 4 client.20913 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:06.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.262+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/3785895119 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635c0c53d90 con 0x5635c0d3d4a0 2024-05-27T08:02:06.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.262+0000 7f1d73400700 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635c0d3d4a0 0x5635c0d3f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.263+0000 7f1d73400700 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635c0d3d4a0 0x5635c0d3f8c0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb487204554ca3cf server_cookie=26b00eeec689076c in_seq=0 out_seq=0 2024-05-27T08:02:06.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.263+0000 7f1d5e000700 10 client.20913 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.265+0000 7f1d5e000700 1 -- 172.21.15.63:0/3785895119 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5635c0c53d90 con 0x5635c0d3d4a0 2024-05-27T08:02:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.265+0000 7f1d5e000700 10 client.20913 handle_command_reply: tid=0 2024-05-27T08:02:06.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d89bf5b80 1 client.20913 shutdown 2024-05-27T08:02:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/3785895119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 msgr2=0x5635c0d2d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d2d650 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f1d640aa730 tx=0x7f1d640b1430 comp rx=0 tx=0).stop 2024-05-27T08:02:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/3785895119 shutdown_connections 2024-05-27T08:02:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635c0d3d4a0 0x5635c0d3f8c0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d2e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d2d650 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/3785895119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/3785895119 >> 172.21.15.63:0/3785895119 conn(0x5635c0c04fc0 msgr2=0x5635c0d1dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d5e000700 0 client.20913 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:06.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.266+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/3785895119 shutdown_connections 2024-05-27T08:02:06.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.267+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/3785895119 wait complete. 2024-05-27T08:02:06.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.267+0000 7f1d89bf5b80 20 client.20913 trim_cache size 0 max 16384 2024-05-27T08:02:06.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.287+0000 7f1d89bf5b80 1 Processor -- start 2024-05-27T08:02:06.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.287+0000 7f1d89bf5b80 1 -- start start 2024-05-27T08:02:06.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.287+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0ba4e40 0x5635c0d28ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.287+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23410 0x5635c0d277c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.287+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0d23820 0x5635c0d397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.287+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741b4d50 con 0x5635c0d23820 2024-05-27T08:02:06.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.287+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741b25e0 con 0x5635c0ba4e40 2024-05-27T08:02:06.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.287+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741b0450 con 0x5635c0d23410 2024-05-27T08:02:06.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d79000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0d23820 0x5635c0d397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d79000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0d23820 0x5635c0d397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49354/0 (socket says 172.21.15.63:49354) 2024-05-27T08:02:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d79000700 1 -- 172.21.15.63:0/4126996924 learned_addr learned my addr 172.21.15.63:0/4126996924 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d73400700 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23410 0x5635c0d277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d73e00700 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0ba4e40 0x5635c0d28ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d79000700 1 -- 172.21.15.63:0/4126996924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23410 msgr2=0x5635c0d277c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:06.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d79000700 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23410 0x5635c0d277c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d79000700 1 -- 172.21.15.63:0/4126996924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0ba4e40 msgr2=0x5635c0d28ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d79000700 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0ba4e40 0x5635c0d28ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d79000700 1 -- 172.21.15.63:0/4126996924 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c0d191d0 con 0x5635c0d23820 2024-05-27T08:02:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d73400700 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23410 0x5635c0d277c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.288+0000 7f1d73e00700 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0ba4e40 0x5635c0d28ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.289+0000 7f1d79000700 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0d23820 0x5635c0d397b0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0157b0 tx=0x7f1d6c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab03f2f674ccea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.289+0000 7f1d68c00700 1 -- 172.21.15.63:0/4126996924 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c009e60 con 0x5635c0d23820 2024-05-27T08:02:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.289+0000 7f1d68c00700 1 -- 172.21.15.63:0/4126996924 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d6c025070 con 0x5635c0d23820 2024-05-27T08:02:06.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.289+0000 7f1d68c00700 1 -- 172.21.15.63:0/4126996924 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c020960 con 0x5635c0d23820 2024-05-27T08:02:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.290+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/4126996924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0d23820 msgr2=0x5635c0d397b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.290+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0d23820 0x5635c0d397b0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0157b0 tx=0x7f1d6c004b40 comp rx=0 tx=0).stop 2024-05-27T08:02:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.290+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/4126996924 shutdown_connections 2024-05-27T08:02:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.290+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0d23820 0x5635c0d397b0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.290+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0ba4e40 0x5635c0d28ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.290+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/4126996924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23410 0x5635c0d277c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.290+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/4126996924 >> 172.21.15.63:0/4126996924 conn(0x5635c0c04fc0 msgr2=0x5635c0d2d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:06.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.291+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/4126996924 shutdown_connections 2024-05-27T08:02:06.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.291+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/4126996924 wait complete. 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.291+0000 7f1d89bf5b80 1 -- start start 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.291+0000 7f1d89bf5b80 1 Processor -- start 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.292+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d36000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.292+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.292+0000 7f1d89bf5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d2a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.292+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741b0aa0 con 0x5635c0ba4e40 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.292+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741b1f10 con 0x5635c0d23410 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.292+0000 7f1d89bf5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741b1170 con 0x5635c0d23820 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.292+0000 7f1d89bf5b80 10 client.0 _ll_register_callbacks cb 0x5635c0bebc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.293+0000 7f1d73e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d36000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.293+0000 7f1d73e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d36000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49366/0 (socket says 172.21.15.63:49366) 2024-05-27T08:02:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.293+0000 7f1d73e00700 1 -- 172.21.15.63:0/2182898957 learned_addr learned my addr 172.21.15.63:0/2182898957 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:06.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.293+0000 7f1d79000700 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d2a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.293+0000 7f1d73400700 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.293+0000 7f1d73e00700 1 -- 172.21.15.63:0/2182898957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 msgr2=0x5635c0d2a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.293+0000 7f1d73e00700 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d2a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.293+0000 7f1d73e00700 1 -- 172.21.15.63:0/2182898957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 msgr2=0x5635c0d2a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.293+0000 7f1d73e00700 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.293+0000 7f1d73e00700 1 -- 172.21.15.63:0/2182898957 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c0d2ace0 con 0x5635c0ba4e40 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.294+0000 7f1d73e00700 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d36000 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f1d640a9cb0 tx=0x7f1d640b12e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165d887a53e07b81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.294+0000 7f1d5e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.294+0000 7f1d5e000700 1 -- 172.21.15.63:0/2182898957 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640c3070 con 0x5635c0ba4e40 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.294+0000 7f1d5e000700 1 -- 172.21.15.63:0/2182898957 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9483+0+0 (secure 0 0 0) 0x7f1d64022470 con 0x5635c0ba4e40 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.294+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/2182898957 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d6c025070 con 0x5635c0ba4e40 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.294+0000 7f1d5e000700 1 -- 172.21.15.63:0/2182898957 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1d640a9ce0 con 0x5635c0ba4e40 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.294+0000 7f1d89bf5b80 10 client.20919 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:06.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.294+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/2182898957 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5635c0d2ace0 con 0x5635c0ba4e40 2024-05-27T08:02:06.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.295+0000 7f1d5e000700 1 -- 172.21.15.63:0/2182898957 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d6c025070 con 0x5635c0ba4e40 2024-05-27T08:02:06.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.295+0000 7f1d5e000700 1 -- 172.21.15.63:0/2182898957 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1d640a63a0 con 0x5635c0ba4e40 2024-05-27T08:02:06.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.295+0000 7f1d89bf5b80 10 client.20919 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:06.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.295+0000 7f1d89bf5b80 10 client.20919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:06.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.295+0000 7f1d89bf5b80 20 client.20919 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:06.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.295+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635c0da7ec0 0x5635c0daa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.295+0000 7f1d89bf5b80 4 client.20919 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:02:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.295+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/2182898957 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635c0c4d570 con 0x5635c0da7ec0 2024-05-27T08:02:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.296+0000 7f1d73400700 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635c0da7ec0 0x5635c0daa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:06.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.298+0000 7f1d73400700 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635c0da7ec0 0x5635c0daa2e0 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6b95a79aafaf59 server_cookie=2fb5a229bc7cd0bb in_seq=0 out_seq=0 2024-05-27T08:02:06.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.298+0000 7f1d5e000700 10 client.20919 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:06.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.305+0000 7f1d5e000700 1 -- 172.21.15.63:0/2182898957 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635c0c4d570 con 0x5635c0da7ec0 2024-05-27T08:02:06.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.305+0000 7f1d5e000700 10 client.20919 handle_command_reply: tid=0 2024-05-27T08:02:06.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.305+0000 7f1d89bf5b80 1 client.20919 shutdown 2024-05-27T08:02:06.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.305+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/2182898957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 msgr2=0x5635c0d36000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:06.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.305+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d36000 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f1d640a9cb0 tx=0x7f1d640b12e0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.306+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/2182898957 shutdown_connections 2024-05-27T08:02:06.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.306+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5635c0da7ec0 0x5635c0daa2e0 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.306+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5635c0d23820 0x5635c0d2a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.306+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5635c0ba4e40 0x5635c0d36000 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.306+0000 7f1d89bf5b80 1 --2- 172.21.15.63:0/2182898957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635c0d23410 0x5635c0d2a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.306+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/2182898957 >> 172.21.15.63:0/2182898957 conn(0x5635c0c04fc0 msgr2=0x5635c0d2de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:06.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.306+0000 7f1d5e000700 0 client.20919 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:06.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.306+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/2182898957 shutdown_connections 2024-05-27T08:02:06.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.306+0000 7f1d89bf5b80 1 -- 172.21.15.63:0/2182898957 wait complete. 2024-05-27T08:02:06.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.306+0000 7f1d89bf5b80 20 client.20919 trim_cache size 0 max 16384 2024-05-27T08:02:06.310 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:02:06.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.307+0000 7f1d79a00700 1 -- 172.21.15.63:0/3575508285 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d580b2a70 msgr2=0x7f1d580b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.307+0000 7f1d79a00700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d580b2a70 0x7f1d580b4f80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0094f0 tx=0x7f1d6c009130 comp rx=0 tx=0).stop 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.307+0000 7f1d79a00700 1 -- 172.21.15.63:0/3575508285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d741043f0 msgr2=0x7f1d7419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.307+0000 7f1d79a00700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d741043f0 0x7f1d7419ce50 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400b580 tx=0x7f1d6400f510 comp rx=0 tx=0).stop 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.308+0000 7f1d79a00700 1 -- 172.21.15.63:0/3575508285 shutdown_connections 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.308+0000 7f1d79a00700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1d580b2a70 0x7f1d580b4f80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.308+0000 7f1d79a00700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d74109670 0x7f1d741a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.308+0000 7f1d79a00700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d741043f0 0x7f1d7419ce50 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.308+0000 7f1d79a00700 1 --2- 172.21.15.63:0/3575508285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d74108d20 0x7f1d7419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.308+0000 7f1d79a00700 1 -- 172.21.15.63:0/3575508285 >> 172.21.15.63:0/3575508285 conn(0x7f1d74079d00 msgr2=0x7f1d74101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.308+0000 7f1d79a00700 1 -- 172.21.15.63:0/3575508285 shutdown_connections 2024-05-27T08:02:06.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:06.308+0000 7f1d79a00700 1 -- 172.21.15.63:0/3575508285 wait complete. 2024-05-27T08:02:07.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:07 smithi086 ceph-mon[34687]: pgmap v904: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 29 op/s 2024-05-27T08:02:07.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:07 smithi086 ceph-mon[34687]: osdmap e139: 12 total, 12 up, 12 in 2024-05-27T08:02:07.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:07 smithi149 ceph-mon[35408]: pgmap v904: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 29 op/s 2024-05-27T08:02:07.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:07 smithi149 ceph-mon[35408]: osdmap e139: 12 total, 12 up, 12 in 2024-05-27T08:02:07.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:07 smithi063 ceph-mon[31613]: pgmap v904: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 29 op/s 2024-05-27T08:02:07.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:07 smithi063 ceph-mon[31613]: osdmap e139: 12 total, 12 up, 12 in 2024-05-27T08:02:08.380 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:02:08.381 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:02:08.875 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:09.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:09 smithi086 ceph-mon[34687]: pgmap v906: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.1 MiB/s wr, 35 op/s 2024-05-27T08:02:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:09 smithi149 ceph-mon[35408]: pgmap v906: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.1 MiB/s wr, 35 op/s 2024-05-27T08:02:09.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:09 smithi063 ceph-mon[31613]: pgmap v906: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.1 MiB/s wr, 35 op/s 2024-05-27T08:02:11.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.365+0000 7f4adb200700 1 -- 172.21.15.63:0/4074015036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4adc1032d0 msgr2=0x7f4adc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:11.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.365+0000 7f4adb200700 1 --2- 172.21.15.63:0/4074015036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4adc1032d0 0x7f4adc1036e0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0099c0 tx=0x7f4acc009cd0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.365+0000 7f4adb200700 1 -- 172.21.15.63:0/4074015036 shutdown_connections 2024-05-27T08:02:11.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.365+0000 7f4adb200700 1 --2- 172.21.15.63:0/4074015036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4adc108550 0x7f4adc10feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.365+0000 7f4adb200700 1 --2- 172.21.15.63:0/4074015036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4adc1032d0 0x7f4adc1036e0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.365+0000 7f4adb200700 1 --2- 172.21.15.63:0/4074015036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4adc107c00 0x7f4adc108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.365+0000 7f4adb200700 1 -- 172.21.15.63:0/4074015036 >> 172.21.15.63:0/4074015036 conn(0x7f4adc078ab0 msgr2=0x7f4adc078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:11.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.365+0000 7f4adb200700 1 -- 172.21.15.63:0/4074015036 shutdown_connections 2024-05-27T08:02:11.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.365+0000 7f4adb200700 1 -- 172.21.15.63:0/4074015036 wait complete. 2024-05-27T08:02:11.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.366+0000 7f4adb200700 1 Processor -- start 2024-05-27T08:02:11.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.367+0000 7f4adb200700 1 -- start start 2024-05-27T08:02:11.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.367+0000 7f4adb200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4adc1032d0 0x7f4adc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.367+0000 7f4adb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4adc107c00 0x7f4adc19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.367+0000 7f4adb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4adc108550 0x7f4adc1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.367+0000 7f4adb200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc113080 con 0x7f4adc107c00 2024-05-27T08:02:11.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.367+0000 7f4adb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc112f00 con 0x7f4adc108550 2024-05-27T08:02:11.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.367+0000 7f4adb200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc113200 con 0x7f4adc1032d0 2024-05-27T08:02:11.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ad9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4adc107c00 0x7f4adc19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ad9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4adc107c00 0x7f4adc19d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41190/0 (socket says 172.21.15.63:41190) 2024-05-27T08:02:11.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ad9400700 1 -- 172.21.15.63:0/1224302032 learned_addr learned my addr 172.21.15.63:0/1224302032 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:11.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4adc1032d0 0x7f4adc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ada800700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4adc108550 0x7f4adc1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ad9400700 1 -- 172.21.15.63:0/1224302032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4adc1032d0 msgr2=0x7f4adc19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:11.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ad9400700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4adc1032d0 0x7f4adc19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ad9400700 1 -- 172.21.15.63:0/1224302032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4adc108550 msgr2=0x7f4adc1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:11.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ad9400700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4adc108550 0x7f4adc1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ad9400700 1 -- 172.21.15.63:0/1224302032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4adc1a1cb0 con 0x7f4adc107c00 2024-05-27T08:02:11.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.368+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4adc1032d0 0x7f4adc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:11.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.369+0000 7f4ad9400700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4adc107c00 0x7f4adc19d380 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800d980 tx=0x7f4ac800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:11.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.369+0000 7f4ad2a00700 1 -- 172.21.15.63:0/1224302032 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8017070 con 0x7f4adc107c00 2024-05-27T08:02:11.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.369+0000 7f4ad2a00700 1 -- 172.21.15.63:0/1224302032 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ac8009dc0 con 0x7f4adc107c00 2024-05-27T08:02:11.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.369+0000 7f4adb200700 1 -- 172.21.15.63:0/1224302032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4adc1a1fa0 con 0x7f4adc107c00 2024-05-27T08:02:11.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.369+0000 7f4ad2a00700 1 -- 172.21.15.63:0/1224302032 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac800f3c0 con 0x7f4adc107c00 2024-05-27T08:02:11.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.369+0000 7f4adb200700 1 -- 172.21.15.63:0/1224302032 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4adc1a98c0 con 0x7f4adc107c00 2024-05-27T08:02:11.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.375+0000 7f4aef712b80 1 Processor -- start 2024-05-27T08:02:11.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.375+0000 7f4aef712b80 1 -- start start 2024-05-27T08:02:11.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.375+0000 7f4ad2a00700 1 -- 172.21.15.63:0/1224302032 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4ac8009930 con 0x7f4adc107c00 2024-05-27T08:02:11.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.376+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 0x560d19396f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.376+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.376+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 0x560d193a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.376+0000 7f4aef712b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560d192c5ec0 con 0x560d19397490 2024-05-27T08:02:11.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.376+0000 7f4aef712b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d192b5a60 con 0x560d193978a0 2024-05-27T08:02:11.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.376+0000 7f4aef712b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560d192b9aa0 con 0x560d19218ec0 2024-05-27T08:02:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.376+0000 7f4ada800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 0x560d19396f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.377+0000 7f4ada800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 0x560d19396f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:59676/0 (socket says 172.21.15.63:59676) 2024-05-27T08:02:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.377+0000 7f4ada800700 1 -- 172.21.15.63:0/3446421720 learned_addr learned my addr 172.21.15.63:0/3446421720 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.377+0000 7f4ad9400700 1 --2- 172.21.15.63:0/3446421720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 0x560d193a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.377+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/3446421720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.377+0000 7f4ad9400700 1 -- 172.21.15.63:0/3446421720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 msgr2=0x560d19396f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.377+0000 7f4ad9400700 1 --2- 172.21.15.63:0/3446421720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 0x560d19396f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.377+0000 7f4ad9400700 1 -- 172.21.15.63:0/3446421720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 msgr2=0x560d193a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.377+0000 7f4ad9400700 1 --2- 172.21.15.63:0/3446421720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.378+0000 7f4ad9400700 1 -- 172.21.15.63:0/3446421720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1927e4a0 con 0x560d193978a0 2024-05-27T08:02:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.378+0000 7f4ad2a00700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ac00b29f0 0x7f4ac00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.378+0000 7f4ad9400700 1 --2- 172.21.15.63:0/3446421720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 0x560d193a40a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80df8a0 tx=0x7f4ac80e8990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb82ebdf328e5aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.378+0000 7f4ad2a00700 1 -- 172.21.15.63:0/1224302032 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f4ac80e2570 con 0x7f4adc107c00 2024-05-27T08:02:11.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.378+0000 7f4ada800700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ac00b29f0 0x7f4ac00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.379+0000 7f4ac7400700 1 -- 172.21.15.63:0/3446421720 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8020070 con 0x560d193978a0 2024-05-27T08:02:11.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.379+0000 7f4ac7400700 1 -- 172.21.15.63:0/3446421720 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ac80a8f40 con 0x560d193978a0 2024-05-27T08:02:11.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.379+0000 7f4ac7400700 1 -- 172.21.15.63:0/3446421720 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80a8d80 con 0x560d193978a0 2024-05-27T08:02:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.380+0000 7f4aef712b80 1 -- 172.21.15.63:0/3446421720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 msgr2=0x560d193a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.380+0000 7f4ada800700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ac00b29f0 0x7f4ac00b4f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x560d19393710 tx=0x7f4ad4009100 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.380+0000 7f4aef712b80 1 --2- 172.21.15.63:0/3446421720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 0x560d193a40a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80df8a0 tx=0x7f4ac80e8990 comp rx=0 tx=0).stop 2024-05-27T08:02:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.380+0000 7f4aef712b80 1 -- 172.21.15.63:0/3446421720 shutdown_connections 2024-05-27T08:02:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.380+0000 7f4aef712b80 1 --2- 172.21.15.63:0/3446421720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 0x560d193a40a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.380+0000 7f4aef712b80 1 --2- 172.21.15.63:0/3446421720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 0x560d19396f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.380+0000 7f4aef712b80 1 --2- 172.21.15.63:0/3446421720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.380+0000 7f4aef712b80 1 -- 172.21.15.63:0/3446421720 >> 172.21.15.63:0/3446421720 conn(0x560d19279040 msgr2=0x560d19392db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:11.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.380+0000 7f4aef712b80 1 -- 172.21.15.63:0/3446421720 shutdown_connections 2024-05-27T08:02:11.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.380+0000 7f4aef712b80 1 -- 172.21.15.63:0/3446421720 wait complete. 2024-05-27T08:02:11.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.381+0000 7f4aef712b80 1 -- start start 2024-05-27T08:02:11.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.381+0000 7f4aef712b80 1 Processor -- start 2024-05-27T08:02:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.382+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 0x560d193a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.382+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.382+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d193978a0 0x560d193a2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.382+0000 7f4aef712b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560d191bc020 con 0x560d19397490 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.382+0000 7f4aef712b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d192567e0 con 0x560d19218ec0 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.382+0000 7f4aef712b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560d193a4a10 con 0x560d193978a0 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.382+0000 7f4aef712b80 10 client.0 _ll_register_callbacks cb 0x560d1925fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.382+0000 7f4ad9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.382+0000 7f4ad9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41206/0 (socket says 172.21.15.63:41206) 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.382+0000 7f4ad9400700 1 -- 172.21.15.63:0/3445653342 learned_addr learned my addr 172.21.15.63:0/3445653342 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.383+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 0x560d193a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.383+0000 7f4ada800700 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d193978a0 0x560d193a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.383+0000 7f4ad9400700 1 -- 172.21.15.63:0/3445653342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d193978a0 msgr2=0x560d193a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.383+0000 7f4ad9400700 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d193978a0 0x560d193a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.383+0000 7f4ad9400700 1 -- 172.21.15.63:0/3445653342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 msgr2=0x560d193a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.383+0000 7f4ad9400700 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 0x560d193a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.383+0000 7f4ad9400700 1 -- 172.21.15.63:0/3445653342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d19205950 con 0x560d19397490 2024-05-27T08:02:11.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.384+0000 7f4ad9400700 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193a1b20 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80a9130 tx=0x7f4ac8010950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bb2f866f734e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:11.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.384+0000 7f4abbe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:11.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.384+0000 7f4abbe00700 1 -- 172.21.15.63:0/3445653342 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8020070 con 0x560d19397490 2024-05-27T08:02:11.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.384+0000 7f4abbe00700 1 -- 172.21.15.63:0/3445653342 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f4ac8010460 con 0x560d19397490 2024-05-27T08:02:11.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.384+0000 7f4aef712b80 1 -- 172.21.15.63:0/3445653342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac80a8f40 con 0x560d19397490 2024-05-27T08:02:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.384+0000 7f4abbe00700 1 -- 172.21.15.63:0/3445653342 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4ac8009c20 con 0x560d19397490 2024-05-27T08:02:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.384+0000 7f4aef712b80 10 client.20931 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.384+0000 7f4aef712b80 1 -- 172.21.15.63:0/3445653342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560d192860e0 con 0x560d19397490 2024-05-27T08:02:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.385+0000 7f4abbe00700 1 -- 172.21.15.63:0/3445653342 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ac80a8f40 con 0x560d19397490 2024-05-27T08:02:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.385+0000 7f4abbe00700 1 -- 172.21.15.63:0/3445653342 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4ac80dfb20 con 0x560d19397490 2024-05-27T08:02:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.385+0000 7f4aef712b80 10 client.20931 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.385+0000 7f4aef712b80 10 client.20931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.385+0000 7f4aef712b80 20 client.20931 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.385+0000 7f4aef712b80 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d193b17c0 0x560d193b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.385+0000 7f4aef712b80 4 client.20931 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.385+0000 7f4aef712b80 1 -- 172.21.15.63:0/3445653342 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d192c7e10 con 0x560d193b17c0 2024-05-27T08:02:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.386+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d193b17c0 0x560d193b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.387+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d193b17c0 0x560d193b3be0 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386bb9f7ef4e66dc server_cookie=7a5933be0344f4aa in_seq=0 out_seq=0 2024-05-27T08:02:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.387+0000 7f4abbe00700 10 client.20931 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4abbe00700 1 -- 172.21.15.63:0/3445653342 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560d192c7e10 con 0x560d193b17c0 2024-05-27T08:02:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4abbe00700 10 client.20931 handle_command_reply: tid=0 2024-05-27T08:02:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4aef712b80 1 client.20931 shutdown 2024-05-27T08:02:11.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4aef712b80 1 -- 172.21.15.63:0/3445653342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 msgr2=0x560d193a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:11.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4aef712b80 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193a1b20 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80a9130 tx=0x7f4ac8010950 comp rx=0 tx=0).stop 2024-05-27T08:02:11.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4aef712b80 1 -- 172.21.15.63:0/3445653342 shutdown_connections 2024-05-27T08:02:11.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4aef712b80 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d193b17c0 0x560d193b3be0 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4aef712b80 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d193978a0 0x560d193a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4aef712b80 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 0x560d193a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4aef712b80 1 --2- 172.21.15.63:0/3445653342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193a1b20 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4abbe00700 0 client.20931 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:11.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.389+0000 7f4aef712b80 1 -- 172.21.15.63:0/3445653342 >> 172.21.15.63:0/3445653342 conn(0x560d19279040 msgr2=0x560d19392070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:11.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.390+0000 7f4aef712b80 1 -- 172.21.15.63:0/3445653342 shutdown_connections 2024-05-27T08:02:11.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.390+0000 7f4aef712b80 1 -- 172.21.15.63:0/3445653342 wait complete. 2024-05-27T08:02:11.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.390+0000 7f4aef712b80 20 client.20931 trim_cache size 0 max 16384 2024-05-27T08:02:11.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.426+0000 7f4aef712b80 1 Processor -- start 2024-05-27T08:02:11.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.426+0000 7f4aef712b80 1 -- start start 2024-05-27T08:02:11.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.427+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 0x560d193a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.427+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19397490 0x560d193a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.427+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d193978a0 0x560d193adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.427+0000 7f4aef712b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc1b4f10 con 0x560d193978a0 2024-05-27T08:02:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.427+0000 7f4aef712b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc1b27a0 con 0x560d19218ec0 2024-05-27T08:02:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.427+0000 7f4aef712b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc1b0610 con 0x560d19397490 2024-05-27T08:02:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.427+0000 7f4ada800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d193978a0 0x560d193adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.427+0000 7f4ada800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d193978a0 0x560d193adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41218/0 (socket says 172.21.15.63:41218) 2024-05-27T08:02:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.427+0000 7f4ada800700 1 -- 172.21.15.63:0/2931456406 learned_addr learned my addr 172.21.15.63:0/2931456406 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.428+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/2931456406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 0x560d193a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.428+0000 7f4ad9400700 1 --2- 172.21.15.63:0/2931456406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19397490 0x560d193a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.428+0000 7f4ada800700 1 -- 172.21.15.63:0/2931456406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19397490 msgr2=0x560d193a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.428+0000 7f4ada800700 1 --2- 172.21.15.63:0/2931456406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19397490 0x560d193a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.428+0000 7f4ada800700 1 -- 172.21.15.63:0/2931456406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 msgr2=0x560d193a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.428+0000 7f4ada800700 1 --2- 172.21.15.63:0/2931456406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 0x560d193a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.428+0000 7f4ada800700 1 -- 172.21.15.63:0/2931456406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d193943d0 con 0x560d193978a0 2024-05-27T08:02:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.428+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/2931456406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 0x560d193a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.429+0000 7f4ada800700 1 --2- 172.21.15.63:0/2931456406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d193978a0 0x560d193adcb0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4004580 tx=0x7f4ad4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5816da48284f936a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:11.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.434+0000 7f4ac6a00700 1 -- 172.21.15.63:0/2931456406 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4007ba0 con 0x560d193978a0 2024-05-27T08:02:11.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.434+0000 7f4ac6a00700 1 -- 172.21.15.63:0/2931456406 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ad401c040 con 0x560d193978a0 2024-05-27T08:02:11.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.434+0000 7f4ac6a00700 1 -- 172.21.15.63:0/2931456406 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4024680 con 0x560d193978a0 2024-05-27T08:02:11.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.435+0000 7f4aef712b80 1 -- 172.21.15.63:0/2931456406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d193978a0 msgr2=0x560d193adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:11.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.435+0000 7f4aef712b80 1 --2- 172.21.15.63:0/2931456406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d193978a0 0x560d193adcb0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4004580 tx=0x7f4ad4016000 comp rx=0 tx=0).stop 2024-05-27T08:02:11.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.435+0000 7f4aef712b80 1 -- 172.21.15.63:0/2931456406 shutdown_connections 2024-05-27T08:02:11.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.435+0000 7f4aef712b80 1 --2- 172.21.15.63:0/2931456406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d193978a0 0x560d193adcb0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.435+0000 7f4aef712b80 1 --2- 172.21.15.63:0/2931456406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d19218ec0 0x560d193a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.435+0000 7f4aef712b80 1 --2- 172.21.15.63:0/2931456406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19397490 0x560d193a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.435+0000 7f4aef712b80 1 -- 172.21.15.63:0/2931456406 >> 172.21.15.63:0/2931456406 conn(0x560d19279040 msgr2=0x560d1939a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.436+0000 7f4aef712b80 1 -- 172.21.15.63:0/2931456406 shutdown_connections 2024-05-27T08:02:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.436+0000 7f4aef712b80 1 -- 172.21.15.63:0/2931456406 wait complete. 2024-05-27T08:02:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.436+0000 7f4aef712b80 1 -- start start 2024-05-27T08:02:11.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.437+0000 7f4aef712b80 1 Processor -- start 2024-05-27T08:02:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.437+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 0x560d193b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.437+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.437+0000 7f4aef712b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 0x560d193b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.437+0000 7f4aef712b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc1b0c60 con 0x560d19397490 2024-05-27T08:02:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.437+0000 7f4aef712b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc1b20d0 con 0x560d193978a0 2024-05-27T08:02:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.437+0000 7f4aef712b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc1b1330 con 0x560d19218ec0 2024-05-27T08:02:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.437+0000 7f4aef712b80 10 client.0 _ll_register_callbacks cb 0x560d1925fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:11.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.438+0000 7f4ad9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.438+0000 7f4ad9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41230/0 (socket says 172.21.15.63:41230) 2024-05-27T08:02:11.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.438+0000 7f4ad9400700 1 -- 172.21.15.63:0/1289071975 learned_addr learned my addr 172.21.15.63:0/1289071975 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:11.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.438+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 0x560d193b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.438+0000 7f4ada800700 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 0x560d193b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.438+0000 7f4ad9400700 1 -- 172.21.15.63:0/1289071975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 msgr2=0x560d193b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:11.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.438+0000 7f4ad9400700 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 0x560d193b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.439+0000 7f4ad9400700 1 -- 172.21.15.63:0/1289071975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 msgr2=0x560d193b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:11.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.439+0000 7f4ad9400700 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 0x560d193b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.439+0000 7f4ad9400700 1 -- 172.21.15.63:0/1289071975 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1941b100 con 0x560d19397490 2024-05-27T08:02:11.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.439+0000 7f4ad9400700 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193b4ff0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80a93c0 tx=0x7f4ac80b20b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fa269c5e134efc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:11.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.439+0000 7f4abbe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:11.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.439+0000 7f4abbe00700 1 -- 172.21.15.63:0/1289071975 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8020070 con 0x560d19397490 2024-05-27T08:02:11.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.439+0000 7f4abbe00700 1 -- 172.21.15.63:0/1289071975 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f4ac80a5a20 con 0x560d19397490 2024-05-27T08:02:11.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.439+0000 7f4aef712b80 1 -- 172.21.15.63:0/1289071975 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad401c040 con 0x560d19397490 2024-05-27T08:02:11.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.440+0000 7f4abbe00700 1 -- 172.21.15.63:0/1289071975 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4ac80a8ba0 con 0x560d19397490 2024-05-27T08:02:11.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.440+0000 7f4aef712b80 10 client.20937 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:11.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.440+0000 7f4aef712b80 1 -- 172.21.15.63:0/1289071975 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560d1941b100 con 0x560d19397490 2024-05-27T08:02:11.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.440+0000 7f4abbe00700 1 -- 172.21.15.63:0/1289071975 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ac8068e50 con 0x560d19397490 2024-05-27T08:02:11.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.440+0000 7f4abbe00700 1 -- 172.21.15.63:0/1289071975 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4ac80a8690 con 0x560d19397490 2024-05-27T08:02:11.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.441+0000 7f4aef712b80 10 client.20937 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:11.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.441+0000 7f4aef712b80 10 client.20937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:11.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.441+0000 7f4aef712b80 20 client.20937 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:11.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.441+0000 7f4aef712b80 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d1941c630 0x560d1941ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:11.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.441+0000 7f4aef712b80 4 client.20937 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:02:11.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.441+0000 7f4aef712b80 1 -- 172.21.15.63:0/1289071975 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560d192c15f0 con 0x560d1941c630 2024-05-27T08:02:11.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.441+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d1941c630 0x560d1941ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:11.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.442+0000 7f4ad9e00700 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d1941c630 0x560d1941ea50 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b7eab15a43652f server_cookie=f0524c4614d5a6e1 in_seq=0 out_seq=0 2024-05-27T08:02:11.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.443+0000 7f4abbe00700 10 client.20937 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:11.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.446+0000 7f4abbe00700 1 -- 172.21.15.63:0/1289071975 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560d192c15f0 con 0x560d1941c630 2024-05-27T08:02:11.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.446+0000 7f4abbe00700 10 client.20937 handle_command_reply: tid=0 2024-05-27T08:02:11.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.446+0000 7f4aef712b80 1 client.20937 shutdown 2024-05-27T08:02:11.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 1 -- 172.21.15.63:0/1289071975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 msgr2=0x560d193b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:11.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193b4ff0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80a93c0 tx=0x7f4ac80b20b0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 1 -- 172.21.15.63:0/1289071975 shutdown_connections 2024-05-27T08:02:11.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560d1941c630 0x560d1941ea50 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d193978a0 0x560d193b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d19218ec0 0x560d193b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 1 --2- 172.21.15.63:0/1289071975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d19397490 0x560d193b4ff0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 1 -- 172.21.15.63:0/1289071975 >> 172.21.15.63:0/1289071975 conn(0x560d19279040 msgr2=0x560d193a1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:11.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4abbe00700 0 client.20937 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:11.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 1 -- 172.21.15.63:0/1289071975 shutdown_connections 2024-05-27T08:02:11.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 1 -- 172.21.15.63:0/1289071975 wait complete. 2024-05-27T08:02:11.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.447+0000 7f4aef712b80 20 client.20937 trim_cache size 0 max 16384 2024-05-27T08:02:11.451 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:02:11.452 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:02:11.452 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "7ab5466e-c8b1-42f3-b874-bffed94ccac8", 2024-05-27T08:02:11.452 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:02:11.452 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:02:11.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.448+0000 7f4adb200700 1 -- 172.21.15.63:0/1224302032 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ac00b29f0 msgr2=0x7f4ac00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.448+0000 7f4adb200700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ac00b29f0 0x7f4ac00b4f00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x560d19393710 tx=0x7f4ad4009100 comp rx=0 tx=0).stop 2024-05-27T08:02:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.448+0000 7f4adb200700 1 -- 172.21.15.63:0/1224302032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4adc107c00 msgr2=0x7f4adc19d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.448+0000 7f4adb200700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4adc107c00 0x7f4adc19d380 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800d980 tx=0x7f4ac800dc90 comp rx=0 tx=0).stop 2024-05-27T08:02:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.449+0000 7f4adb200700 1 -- 172.21.15.63:0/1224302032 shutdown_connections 2024-05-27T08:02:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.449+0000 7f4adb200700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4ac00b29f0 0x7f4ac00b4f00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.449+0000 7f4adb200700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4adc108550 0x7f4adc1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.449+0000 7f4adb200700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4adc1032d0 0x7f4adc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.449+0000 7f4adb200700 1 --2- 172.21.15.63:0/1224302032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4adc107c00 0x7f4adc19d380 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:11.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.449+0000 7f4adb200700 1 -- 172.21.15.63:0/1224302032 >> 172.21.15.63:0/1224302032 conn(0x7f4adc078ab0 msgr2=0x7f4adc10ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:11.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.449+0000 7f4adb200700 1 -- 172.21.15.63:0/1224302032 shutdown_connections 2024-05-27T08:02:11.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:11.449+0000 7f4adb200700 1 -- 172.21.15.63:0/1224302032 wait complete. 2024-05-27T08:02:11.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:11 smithi086 ceph-mon[34687]: pgmap v907: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.1 MiB/s wr, 35 op/s 2024-05-27T08:02:11.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:11 smithi086 ceph-mon[34687]: osdmap e140: 12 total, 12 up, 12 in 2024-05-27T08:02:11.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:11 smithi149 ceph-mon[35408]: pgmap v907: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.1 MiB/s wr, 35 op/s 2024-05-27T08:02:11.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:11 smithi149 ceph-mon[35408]: osdmap e140: 12 total, 12 up, 12 in 2024-05-27T08:02:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:11 smithi063 ceph-mon[31613]: pgmap v907: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.1 MiB/s wr, 35 op/s 2024-05-27T08:02:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:11 smithi063 ceph-mon[31613]: osdmap e140: 12 total, 12 up, 12 in 2024-05-27T08:02:12.459 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:02:12.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:02:12.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:02:12.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:02:12.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:02:12.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:02:12.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:02:12.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:02:12.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:02:12.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:12 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:02:12.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:02:12.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:12 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:02:12.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:02:12.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:02:12.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:02:12.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:02:12.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:02:12.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:02:12.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:02:12.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:12 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:12.964 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.194+0000 7f0fee800640 1 -- 172.21.15.63:0/1193652914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fe810e200 msgr2=0x7f0fe81106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.194+0000 7f0fee800640 1 --2- 172.21.15.63:0/1193652914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fe810e200 0x7f0fe81106f0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009ca0 tx=0x7f0fe002a9e0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.195+0000 7f0fee800640 1 -- 172.21.15.63:0/1193652914 shutdown_connections 2024-05-27T08:02:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.195+0000 7f0fee800640 1 --2- 172.21.15.63:0/1193652914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fe810e200 0x7f0fe81106f0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.195+0000 7f0fee800640 1 --2- 172.21.15.63:0/1193652914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe810b850 0x7f0fe810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.195+0000 7f0fee800640 1 --2- 172.21.15.63:0/1193652914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe8104df0 0x7f0fe8105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.195+0000 7f0fee800640 1 -- 172.21.15.63:0/1193652914 >> 172.21.15.63:0/1193652914 conn(0x7f0fe80782d0 msgr2=0x7f0fe8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.195+0000 7f0fee800640 1 -- 172.21.15.63:0/1193652914 shutdown_connections 2024-05-27T08:02:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.195+0000 7f0fee800640 1 -- 172.21.15.63:0/1193652914 wait complete. 2024-05-27T08:02:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.196+0000 7f0fee800640 1 Processor -- start 2024-05-27T08:02:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.196+0000 7f0fee800640 1 -- start start 2024-05-27T08:02:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fee800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fe8104df0 0x7f0fe81a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fee800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe810b850 0x7f0fe81a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe810e200 0x7f0fe819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fee800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe8115f30 con 0x7f0fe8104df0 2024-05-27T08:02:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fee800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe8115db0 con 0x7f0fe810e200 2024-05-27T08:02:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fee800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe81160b0 con 0x7f0fe810b850 2024-05-27T08:02:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fe8104df0 0x7f0fe81a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fe8104df0 0x7f0fe81a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41248/0 (socket says 172.21.15.63:41248) 2024-05-27T08:02:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fe6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe810b850 0x7f0fe81a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fe7400640 1 -- 172.21.15.63:0/762853062 learned_addr learned my addr 172.21.15.63:0/762853062 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.197+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe810e200 0x7f0fe819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.198+0000 7f0fe6a00640 1 -- 172.21.15.63:0/762853062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe810e200 msgr2=0x7f0fe819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.198+0000 7f0fe6a00640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe810e200 0x7f0fe819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.198+0000 7f0fe6a00640 1 -- 172.21.15.63:0/762853062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fe8104df0 msgr2=0x7f0fe81a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.198+0000 7f0fe6a00640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fe8104df0 0x7f0fe81a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.198+0000 7f0fe6a00640 1 -- 172.21.15.63:0/762853062 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe819cec0 con 0x7f0fe810b850 2024-05-27T08:02:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.198+0000 7f0fe7400640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fe8104df0 0x7f0fe81a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.198+0000 7f0fe6a00640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe810b850 0x7f0fe81a2b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400e9f0 tx=0x7f0fd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.199+0000 7f0fdfe00640 1 -- 172.21.15.63:0/762853062 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd400ccc0 con 0x7f0fe810b850 2024-05-27T08:02:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.199+0000 7f0fdfe00640 1 -- 172.21.15.63:0/762853062 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fd40108c0 con 0x7f0fe810b850 2024-05-27T08:02:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.199+0000 7f0fdfe00640 1 -- 172.21.15.63:0/762853062 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd40187e0 con 0x7f0fe810b850 2024-05-27T08:02:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.199+0000 7f0fee800640 1 -- 172.21.15.63:0/762853062 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe819d150 con 0x7f0fe810b850 2024-05-27T08:02:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.199+0000 7f0fee800640 1 -- 172.21.15.63:0/762853062 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe81a93b0 con 0x7f0fe810b850 2024-05-27T08:02:13.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.203+0000 7f0ff5809740 1 Processor -- start 2024-05-27T08:02:13.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.203+0000 7f0ff5809740 1 -- start start 2024-05-27T08:02:13.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.204+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 0x55af7e457860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.204+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e08c2f0 0x55af7e457da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.204+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e4582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.204+0000 7f0ff5809740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55af7e3c8b90 con 0x55af7e0b51b0 2024-05-27T08:02:13.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.204+0000 7f0ff5809740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af7e3b8730 con 0x55af7e08c2f0 2024-05-27T08:02:13.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.204+0000 7f0ff5809740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55af7e3bc770 con 0x55af7e0944d0 2024-05-27T08:02:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.204+0000 7f0fe7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 0x55af7e457860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.204+0000 7f0fe7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 0x55af7e457860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41254/0 (socket says 172.21.15.63:41254) 2024-05-27T08:02:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.204+0000 7f0fe7e00640 1 -- 172.21.15.63:0/3468510777 learned_addr learned my addr 172.21.15.63:0/3468510777 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.204+0000 7f0fe7400640 1 --2- 172.21.15.63:0/3468510777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e08c2f0 0x55af7e457da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.205+0000 7f0fe7e00640 1 -- 172.21.15.63:0/3468510777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 msgr2=0x55af7e4582e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:02:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.205+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/3468510777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e4582e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.205+0000 7f0fe7e00640 1 -- 172.21.15.63:0/3468510777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e08c2f0 msgr2=0x55af7e457da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.205+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/3468510777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e08c2f0 0x55af7e457da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.205+0000 7f0fe7e00640 1 -- 172.21.15.63:0/3468510777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7e3518b0 con 0x55af7e0b51b0 2024-05-27T08:02:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.205+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/3468510777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 0x55af7e457860 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0035000 tx=0x7f0fe0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62e43ef76268d81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.205+0000 7f0fdfe00640 1 -- 172.21.15.63:0/762853062 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0fd4010430 con 0x7f0fe810b850 2024-05-27T08:02:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.206+0000 7f0fdcc00640 1 -- 172.21.15.63:0/3468510777 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0033050 con 0x55af7e0b51b0 2024-05-27T08:02:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.206+0000 7f0fdcc00640 1 -- 172.21.15.63:0/3468510777 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fe0047070 con 0x55af7e0b51b0 2024-05-27T08:02:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.206+0000 7f0fdcc00640 1 -- 172.21.15.63:0/3468510777 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe002eea0 con 0x55af7e0b51b0 2024-05-27T08:02:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.207+0000 7f0ff5809740 1 -- 172.21.15.63:0/3468510777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 msgr2=0x55af7e457860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.207+0000 7f0ff5809740 1 --2- 172.21.15.63:0/3468510777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 0x55af7e457860 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0035000 tx=0x7f0fe0005c50 comp rx=0 tx=0).stop 2024-05-27T08:02:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.208+0000 7f0ff5809740 1 -- 172.21.15.63:0/3468510777 shutdown_connections 2024-05-27T08:02:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.208+0000 7f0ff5809740 1 --2- 172.21.15.63:0/3468510777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e4582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.208+0000 7f0ff5809740 1 --2- 172.21.15.63:0/3468510777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e08c2f0 0x55af7e457da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.208+0000 7f0ff5809740 1 --2- 172.21.15.63:0/3468510777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 0x55af7e457860 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.208+0000 7f0ff5809740 1 -- 172.21.15.63:0/3468510777 >> 172.21.15.63:0/3468510777 conn(0x55af7e0a9300 msgr2=0x55af7e454010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.208+0000 7f0ff5809740 1 -- 172.21.15.63:0/3468510777 shutdown_connections 2024-05-27T08:02:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.208+0000 7f0ff5809740 1 -- 172.21.15.63:0/3468510777 wait complete. 2024-05-27T08:02:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.209+0000 7f0ff5809740 1 -- start start 2024-05-27T08:02:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.209+0000 7f0fdfe00640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fcc0b2960 0x7f0fcc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.209+0000 7f0ff5809740 1 Processor -- start 2024-05-27T08:02:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.209+0000 7f0fdfe00640 1 -- 172.21.15.63:0/762853062 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f0fd40e2c00 con 0x7f0fe810b850 2024-05-27T08:02:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.209+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fcc0b2960 0x7f0fcc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.210+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e461a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.210+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e461f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.210+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 0x55af7e46dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.210+0000 7f0ff5809740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55af7e09e790 con 0x55af7e08c2f0 2024-05-27T08:02:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.210+0000 7f0ff5809740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af7e360bc0 con 0x55af7e0b51b0 2024-05-27T08:02:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.210+0000 7f0ff5809740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55af7e464930 con 0x55af7e0944d0 2024-05-27T08:02:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.210+0000 7f0ff5809740 10 client.0 _ll_register_callbacks cb 0x55af7e33e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.210+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fcc0b2960 0x7f0fcc0b4e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00094e0 tx=0x7f0fe0005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.210+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e461a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.211+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e461a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41256/0 (socket says 172.21.15.63:41256) 2024-05-27T08:02:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.211+0000 7f0fe7400640 1 -- 172.21.15.63:0/1862417802 learned_addr learned my addr 172.21.15.63:0/1862417802 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.211+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 0x55af7e46dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.211+0000 7f0fe6a00640 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e461f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.211+0000 7f0fe7e00640 1 -- 172.21.15.63:0/1862417802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 msgr2=0x55af7e461f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.211+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e461f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.211+0000 7f0fe7e00640 1 -- 172.21.15.63:0/1862417802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 msgr2=0x55af7e461a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.211+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e461a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.211+0000 7f0fe7e00640 1 -- 172.21.15.63:0/1862417802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7e0cd210 con 0x55af7e0b51b0 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.211+0000 7f0fe7400640 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e461a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.212+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 0x55af7e46dc90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009ca0 tx=0x7f0fe002eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e88278b5f9a2e92 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.212+0000 7f0fd1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.212+0000 7f0fd1600640 1 -- 172.21.15.63:0/1862417802 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe005c020 con 0x55af7e0b51b0 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.212+0000 7f0ff5809740 1 -- 172.21.15.63:0/1862417802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe0047070 con 0x55af7e0b51b0 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.212+0000 7f0fd1600640 1 -- 172.21.15.63:0/1862417802 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f0fe0004790 con 0x55af7e0b51b0 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.213+0000 7f0fd1600640 1 -- 172.21.15.63:0/1862417802 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0fe0067050 con 0x55af7e0b51b0 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.213+0000 7f0ff5809740 10 client.40079 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.213+0000 7f0ff5809740 1 -- 172.21.15.63:0/1862417802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55af7e199620 con 0x55af7e0b51b0 2024-05-27T08:02:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.213+0000 7f0fd1600640 1 -- 172.21.15.63:0/1862417802 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fe0047070 con 0x55af7e0b51b0 2024-05-27T08:02:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.213+0000 7f0fd1600640 1 -- 172.21.15.63:0/1862417802 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0fe0058480 con 0x55af7e0b51b0 2024-05-27T08:02:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.214+0000 7f0ff5809740 10 client.40079 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.214+0000 7f0ff5809740 10 client.40079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.214+0000 7f0ff5809740 20 client.40079 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:13.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.214+0000 7f0ff5809740 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55af7e073d80 0x55af7e473a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.214+0000 7f0ff5809740 4 client.40079 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.214+0000 7f0ff5809740 1 -- 172.21.15.63:0/1862417802 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af7e3caae0 con 0x55af7e073d80 2024-05-27T08:02:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.214+0000 7f0fe7400640 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55af7e073d80 0x55af7e473a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.215+0000 7f0fe7400640 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55af7e073d80 0x55af7e473a50 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9084fe79d4d49fe server_cookie=10947967c50ba2b9 in_seq=0 out_seq=0 2024-05-27T08:02:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.215+0000 7f0fd1600640 10 client.40079 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0fd1600640 1 -- 172.21.15.63:0/1862417802 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55af7e3caae0 con 0x55af7e073d80 2024-05-27T08:02:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0fd1600640 10 client.40079 handle_command_reply: tid=0 2024-05-27T08:02:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0ff5809740 1 client.40079 shutdown 2024-05-27T08:02:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0ff5809740 1 -- 172.21.15.63:0/1862417802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 msgr2=0x55af7e46dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0ff5809740 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 0x55af7e46dc90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009ca0 tx=0x7f0fe002eac0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0ff5809740 1 -- 172.21.15.63:0/1862417802 shutdown_connections 2024-05-27T08:02:13.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0ff5809740 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55af7e073d80 0x55af7e473a50 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0ff5809740 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 0x55af7e46dc90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0ff5809740 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e461f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0ff5809740 1 --2- 172.21.15.63:0/1862417802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e461a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0fd1600640 0 client.40079 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.216+0000 7f0ff5809740 1 -- 172.21.15.63:0/1862417802 >> 172.21.15.63:0/1862417802 conn(0x55af7e0a9300 msgr2=0x55af7e457690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.217+0000 7f0ff5809740 1 -- 172.21.15.63:0/1862417802 shutdown_connections 2024-05-27T08:02:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.217+0000 7f0ff5809740 1 -- 172.21.15.63:0/1862417802 wait complete. 2024-05-27T08:02:13.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.217+0000 7f0ff5809740 20 client.40079 trim_cache size 0 max 16384 2024-05-27T08:02:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.242+0000 7f0ff5809740 1 Processor -- start 2024-05-27T08:02:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.242+0000 7f0ff5809740 1 -- start start 2024-05-27T08:02:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.242+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e453330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.242+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e46dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.242+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 0x55af7e463830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.242+0000 7f0ff5809740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe81b5ca0 con 0x55af7e08c2f0 2024-05-27T08:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.242+0000 7f0ff5809740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe81b3570 con 0x55af7e0b51b0 2024-05-27T08:02:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.242+0000 7f0ff5809740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe81b13e0 con 0x55af7e0944d0 2024-05-27T08:02:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e453330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e453330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41272/0 (socket says 172.21.15.63:41272) 2024-05-27T08:02:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe7400640 1 -- 172.21.15.63:0/2605434311 learned_addr learned my addr 172.21.15.63:0/2605434311 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/2605434311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 0x55af7e463830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe6a00640 1 --2- 172.21.15.63:0/2605434311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e46dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe7400640 1 -- 172.21.15.63:0/2605434311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 msgr2=0x55af7e46dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe7400640 1 --2- 172.21.15.63:0/2605434311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e46dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe7400640 1 -- 172.21.15.63:0/2605434311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 msgr2=0x55af7e463830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe7400640 1 --2- 172.21.15.63:0/2605434311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 0x55af7e463830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe7400640 1 -- 172.21.15.63:0/2605434311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7e479bb0 con 0x55af7e08c2f0 2024-05-27T08:02:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.243+0000 7f0fe6a00640 1 --2- 172.21.15.63:0/2605434311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e46dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.244+0000 7f0fe7400640 1 --2- 172.21.15.63:0/2605434311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e453330 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8012ab0 tx=0x7f0fd8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e781eeaa68c465f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.244+0000 7f0fd3e00640 1 -- 172.21.15.63:0/2605434311 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8012c80 con 0x55af7e08c2f0 2024-05-27T08:02:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.244+0000 7f0fd3e00640 1 -- 172.21.15.63:0/2605434311 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fd8012e20 con 0x55af7e08c2f0 2024-05-27T08:02:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.244+0000 7f0fd3e00640 1 -- 172.21.15.63:0/2605434311 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8004c90 con 0x55af7e08c2f0 2024-05-27T08:02:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.246+0000 7f0ff5809740 1 -- 172.21.15.63:0/2605434311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 msgr2=0x55af7e453330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.246+0000 7f0ff5809740 1 --2- 172.21.15.63:0/2605434311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e453330 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8012ab0 tx=0x7f0fd8005f90 comp rx=0 tx=0).stop 2024-05-27T08:02:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.246+0000 7f0ff5809740 1 -- 172.21.15.63:0/2605434311 shutdown_connections 2024-05-27T08:02:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.246+0000 7f0ff5809740 1 --2- 172.21.15.63:0/2605434311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0b51b0 0x55af7e463830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.246+0000 7f0ff5809740 1 --2- 172.21.15.63:0/2605434311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e0944d0 0x55af7e46dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.246+0000 7f0ff5809740 1 --2- 172.21.15.63:0/2605434311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e08c2f0 0x55af7e453330 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.246+0000 7f0ff5809740 1 -- 172.21.15.63:0/2605434311 >> 172.21.15.63:0/2605434311 conn(0x55af7e0a9300 msgr2=0x55af7e463120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.246+0000 7f0ff5809740 1 -- 172.21.15.63:0/2605434311 shutdown_connections 2024-05-27T08:02:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.246+0000 7f0ff5809740 1 -- 172.21.15.63:0/2605434311 wait complete. 2024-05-27T08:02:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.247+0000 7f0ff5809740 1 -- start start 2024-05-27T08:02:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.247+0000 7f0ff5809740 1 Processor -- start 2024-05-27T08:02:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e08c2f0 0x55af7e473ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0944d0 0x55af7e474430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0ff5809740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 0x55af7e4d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0ff5809740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe81b1a30 con 0x55af7e0b51b0 2024-05-27T08:02:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0ff5809740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe81b2ea0 con 0x55af7e0944d0 2024-05-27T08:02:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0ff5809740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe81b2100 con 0x55af7e08c2f0 2024-05-27T08:02:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0ff5809740 10 client.0 _ll_register_callbacks cb 0x7f0fe004f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e08c2f0 0x55af7e473ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0fe6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0944d0 0x55af7e474430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e08c2f0 0x55af7e473ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:59734/0 (socket says 172.21.15.63:59734) 2024-05-27T08:02:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0fe7400640 1 -- 172.21.15.63:0/4152686020 learned_addr learned my addr 172.21.15.63:0/4152686020 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.248+0000 7f0fe7e00640 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 0x55af7e4d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.249+0000 7f0fe7400640 1 -- 172.21.15.63:0/4152686020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0944d0 msgr2=0x55af7e474430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.249+0000 7f0fe7400640 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0944d0 0x55af7e474430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.249+0000 7f0fe7400640 1 -- 172.21.15.63:0/4152686020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 msgr2=0x55af7e4d9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.249+0000 7f0fe7400640 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 0x55af7e4d9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.249+0000 7f0fe7400640 1 -- 172.21.15.63:0/4152686020 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7e4e2be0 con 0x55af7e08c2f0 2024-05-27T08:02:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.249+0000 7f0fe7400640 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e08c2f0 0x55af7e473ef0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800a010 tx=0x7f0fd8009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80e708447c057f6b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.250+0000 7f0fd1600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:02:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.250+0000 7f0fd1600640 1 -- 172.21.15.63:0/4152686020 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8038060 con 0x55af7e08c2f0 2024-05-27T08:02:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.250+0000 7f0ff5809740 1 -- 172.21.15.63:0/4152686020 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fd8012e20 con 0x55af7e08c2f0 2024-05-27T08:02:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.250+0000 7f0fd1600640 1 -- 172.21.15.63:0/4152686020 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f0fd8005030 con 0x55af7e08c2f0 2024-05-27T08:02:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.250+0000 7f0fd1600640 1 -- 172.21.15.63:0/4152686020 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0fd803f050 con 0x55af7e08c2f0 2024-05-27T08:02:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.251+0000 7f0ff5809740 10 client.29902 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.251+0000 7f0ff5809740 1 -- 172.21.15.63:0/4152686020 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55af7e4e3d90 con 0x55af7e08c2f0 2024-05-27T08:02:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.251+0000 7f0fd1600640 1 -- 172.21.15.63:0/4152686020 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fd8012e20 con 0x55af7e08c2f0 2024-05-27T08:02:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.251+0000 7f0fd1600640 1 -- 172.21.15.63:0/4152686020 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0fd802a430 con 0x55af7e08c2f0 2024-05-27T08:02:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.251+0000 7f0ff5809740 10 client.29902 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.251+0000 7f0ff5809740 10 client.29902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.251+0000 7f0ff5809740 20 client.29902 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.251+0000 7f0ff5809740 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55af7e0491c0 0x55af7e4e6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.252+0000 7f0ff5809740 4 client.29902 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:02:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.252+0000 7f0ff5809740 1 -- 172.21.15.63:0/4152686020 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af7e3c42c0 con 0x55af7e0491c0 2024-05-27T08:02:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.252+0000 7f0fe6a00640 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55af7e0491c0 0x55af7e4e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.253+0000 7f0fe6a00640 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55af7e0491c0 0x55af7e4e6490 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6d409d33a7b414 server_cookie=15a7c54b9c750230 in_seq=0 out_seq=0 2024-05-27T08:02:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.253+0000 7f0fd1600640 10 client.29902 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.274+0000 7f0fd1600640 1 -- 172.21.15.63:0/4152686020 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55af7e3c42c0 con 0x55af7e0491c0 2024-05-27T08:02:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.274+0000 7f0fd1600640 10 client.29902 handle_command_reply: tid=0 2024-05-27T08:02:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.274+0000 7f0ff5809740 1 client.29902 shutdown 2024-05-27T08:02:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0ff5809740 1 -- 172.21.15.63:0/4152686020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e08c2f0 msgr2=0x55af7e473ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0ff5809740 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e08c2f0 0x55af7e473ef0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800a010 tx=0x7f0fd8009ec0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0ff5809740 1 -- 172.21.15.63:0/4152686020 shutdown_connections 2024-05-27T08:02:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0ff5809740 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55af7e0491c0 0x55af7e4e6490 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0ff5809740 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af7e0b51b0 0x55af7e4d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0ff5809740 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af7e0944d0 0x55af7e474430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0ff5809740 1 --2- 172.21.15.63:0/4152686020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af7e08c2f0 0x55af7e473ef0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0fd1600640 0 client.29902 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0ff5809740 1 -- 172.21.15.63:0/4152686020 >> 172.21.15.63:0/4152686020 conn(0x55af7e0a9300 msgr2=0x55af7e463ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0ff5809740 1 -- 172.21.15.63:0/4152686020 shutdown_connections 2024-05-27T08:02:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.275+0000 7f0ff5809740 1 -- 172.21.15.63:0/4152686020 wait complete. 2024-05-27T08:02:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.276+0000 7f0ff5809740 20 client.29902 trim_cache size 0 max 16384 2024-05-27T08:02:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:02:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:02:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:02:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.280+0000 7f0fee800640 1 -- 172.21.15.63:0/762853062 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fcc0b2960 msgr2=0x7f0fcc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.280+0000 7f0fee800640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fcc0b2960 0x7f0fcc0b4e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00094e0 tx=0x7f0fe0005fd0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.280+0000 7f0fee800640 1 -- 172.21.15.63:0/762853062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe810b850 msgr2=0x7f0fe81a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.280+0000 7f0fee800640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe810b850 0x7f0fe81a2b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400e9f0 tx=0x7f0fd400eeb0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.280+0000 7f0fee800640 1 -- 172.21.15.63:0/762853062 shutdown_connections 2024-05-27T08:02:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.280+0000 7f0fee800640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fcc0b2960 0x7f0fcc0b4e70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.280+0000 7f0fee800640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fe810e200 0x7f0fe819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.280+0000 7f0fee800640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe810b850 0x7f0fe81a2b50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.280+0000 7f0fee800640 1 --2- 172.21.15.63:0/762853062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0fe8104df0 0x7f0fe81a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.280+0000 7f0fee800640 1 -- 172.21.15.63:0/762853062 >> 172.21.15.63:0/762853062 conn(0x7f0fe80782d0 msgr2=0x7f0fe81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.281+0000 7f0fee800640 1 -- 172.21.15.63:0/762853062 shutdown_connections 2024-05-27T08:02:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:13.281+0000 7f0fee800640 1 -- 172.21.15.63:0/762853062 wait complete. 2024-05-27T08:02:13.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:13 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:13.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:13 smithi086 ceph-mon[34687]: pgmap v909: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.8 MiB/s wr, 57 op/s 2024-05-27T08:02:13.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:13 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:02:13.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:13 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:13.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:13 smithi149 ceph-mon[35408]: pgmap v909: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.8 MiB/s wr, 57 op/s 2024-05-27T08:02:13.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:13 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:02:13.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:13 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:13.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:13 smithi063 ceph-mon[31613]: pgmap v909: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.8 MiB/s wr, 57 op/s 2024-05-27T08:02:13.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:13 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:02:14.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:02:14.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:02:14.701 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:02:14.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:02:14.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:02:14.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:02:14.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:02:14.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:02:14.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:02:15.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.368+0000 7f0495c00700 1 -- 172.21.15.63:0/2609779631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0490104600 msgr2=0x7f0490104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:15.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.368+0000 7f0495c00700 1 --2- 172.21.15.63:0/2609779631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0490104600 0x7f0490104a10 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f04800099c0 tx=0x7f0480009cd0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.368+0000 7f0495c00700 1 -- 172.21.15.63:0/2609779631 shutdown_connections 2024-05-27T08:02:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.368+0000 7f0495c00700 1 --2- 172.21.15.63:0/2609779631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0490109880 0x7f04901111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.368+0000 7f0495c00700 1 --2- 172.21.15.63:0/2609779631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0490104600 0x7f0490104a10 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.368+0000 7f0495c00700 1 --2- 172.21.15.63:0/2609779631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0490108f30 0x7f0490109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.368+0000 7f0495c00700 1 -- 172.21.15.63:0/2609779631 >> 172.21.15.63:0/2609779631 conn(0x7f049007af60 msgr2=0x7f049007b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.368+0000 7f0495c00700 1 -- 172.21.15.63:0/2609779631 shutdown_connections 2024-05-27T08:02:15.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.368+0000 7f0495c00700 1 -- 172.21.15.63:0/2609779631 wait complete. 2024-05-27T08:02:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.369+0000 7f0495c00700 1 Processor -- start 2024-05-27T08:02:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.370+0000 7f0495c00700 1 -- start start 2024-05-27T08:02:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.370+0000 7f0495c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0490104600 0x7f0490075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.370+0000 7f0495c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0490108f30 0x7f0490075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.370+0000 7f0495c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0490109880 0x7f0490076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.370+0000 7f0495c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04901143b0 con 0x7f0490104600 2024-05-27T08:02:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.370+0000 7f0495c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0490114230 con 0x7f0490108f30 2024-05-27T08:02:15.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.370+0000 7f0495c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0490114530 con 0x7f0490109880 2024-05-27T08:02:15.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.371+0000 7f048ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0490104600 0x7f0490075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.371+0000 7f048ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0490104600 0x7f0490075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41298/0 (socket says 172.21.15.63:41298) 2024-05-27T08:02:15.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.371+0000 7f048ea00700 1 -- 172.21.15.63:0/1300752521 learned_addr learned my addr 172.21.15.63:0/1300752521 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.371+0000 7f048e000700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0490108f30 0x7f0490075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.371+0000 7f048f400700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0490109880 0x7f0490076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.371+0000 7f048ea00700 1 -- 172.21.15.63:0/1300752521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0490109880 msgr2=0x7f0490076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.371+0000 7f048ea00700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0490109880 0x7f0490076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.371+0000 7f048ea00700 1 -- 172.21.15.63:0/1300752521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0490108f30 msgr2=0x7f0490075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.371+0000 7f048ea00700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0490108f30 0x7f0490075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.371+0000 7f048ea00700 1 -- 172.21.15.63:0/1300752521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04901a9130 con 0x7f0490104600 2024-05-27T08:02:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.372+0000 7f048ea00700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0490104600 0x7f0490075ab0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f0480000c00 tx=0x7f048000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.372+0000 7f0487400700 1 -- 172.21.15.63:0/1300752521 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480027070 con 0x7f0490104600 2024-05-27T08:02:15.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.372+0000 7f0487400700 1 -- 172.21.15.63:0/1300752521 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f048000fe50 con 0x7f0490104600 2024-05-27T08:02:15.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.372+0000 7f0487400700 1 -- 172.21.15.63:0/1300752521 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800173f0 con 0x7f0490104600 2024-05-27T08:02:15.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.372+0000 7f0495c00700 1 -- 172.21.15.63:0/1300752521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04901a93c0 con 0x7f0490104600 2024-05-27T08:02:15.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.373+0000 7f0495c00700 1 -- 172.21.15.63:0/1300752521 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04901a97c0 con 0x7f0490104600 2024-05-27T08:02:15.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.376+0000 7f0487400700 1 -- 172.21.15.63:0/1300752521 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f048002a070 con 0x7f0490104600 2024-05-27T08:02:15.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.378+0000 7f04a4806b80 1 Processor -- start 2024-05-27T08:02:15.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.378+0000 7f04a4806b80 1 -- start start 2024-05-27T08:02:15.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.378+0000 7f0487400700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04740b29f0 0x7f04740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 0x555c5f1f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 0x555c5f1f2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 0x555c5f1f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f04a4806b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555c5f120e40 con 0x555c5f1f2db0 2024-05-27T08:02:15.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f04a4806b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555c5f1109e0 con 0x555c5f073e40 2024-05-27T08:02:15.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f04a4806b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555c5f114a20 con 0x555c5f1f8d30 2024-05-27T08:02:15.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f0487400700 1 -- 172.21.15.63:0/1300752521 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f04800e3950 con 0x7f0490104600 2024-05-27T08:02:15.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f048ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 0x555c5f1f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f048f400700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04740b29f0 0x7f04740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f048e000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 0x555c5f1f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f048ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 0x555c5f1f87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:52116/0 (socket says 172.21.15.63:52116) 2024-05-27T08:02:15.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f048ea00700 1 -- 172.21.15.63:0/1650978067 learned_addr learned my addr 172.21.15.63:0/1650978067 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:15.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.379+0000 7f048f400700 1 --2- 172.21.15.63:0/1650978067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 0x555c5f1f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.380+0000 7f048ea00700 1 -- 172.21.15.63:0/1650978067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 msgr2=0x555c5f1f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.380+0000 7f048ea00700 1 --2- 172.21.15.63:0/1650978067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 0x555c5f1f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.380+0000 7f048ea00700 1 -- 172.21.15.63:0/1650978067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 msgr2=0x555c5f1f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.380+0000 7f048ea00700 1 --2- 172.21.15.63:0/1650978067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 0x555c5f1f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.380+0000 7f048ea00700 1 -- 172.21.15.63:0/1650978067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c5f0d9420 con 0x555c5f073e40 2024-05-27T08:02:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.381+0000 7f048ea00700 1 --2- 172.21.15.63:0/1650978067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 0x555c5f1f87f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f04800a6920 tx=0x7f04800aa430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a81670ba124b360 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.381+0000 7f048f400700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04740b29f0 0x7f04740b4f00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0488009710 tx=0x7f04880090b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:15.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.381+0000 7f047be00700 1 -- 172.21.15.63:0/1650978067 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800aa8a0 con 0x555c5f073e40 2024-05-27T08:02:15.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.381+0000 7f047be00700 1 -- 172.21.15.63:0/1650978067 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04800aec50 con 0x555c5f073e40 2024-05-27T08:02:15.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.381+0000 7f047be00700 1 -- 172.21.15.63:0/1650978067 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800d90e0 con 0x555c5f073e40 2024-05-27T08:02:15.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.382+0000 7f04a4806b80 1 -- 172.21.15.63:0/1650978067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 msgr2=0x555c5f1f87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:15.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.382+0000 7f04a4806b80 1 --2- 172.21.15.63:0/1650978067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 0x555c5f1f87f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f04800a6920 tx=0x7f04800aa430 comp rx=0 tx=0).stop 2024-05-27T08:02:15.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.382+0000 7f04a4806b80 1 -- 172.21.15.63:0/1650978067 shutdown_connections 2024-05-27T08:02:15.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.382+0000 7f04a4806b80 1 --2- 172.21.15.63:0/1650978067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 0x555c5f1f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.382+0000 7f04a4806b80 1 --2- 172.21.15.63:0/1650978067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 0x555c5f1f87f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.382+0000 7f04a4806b80 1 --2- 172.21.15.63:0/1650978067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 0x555c5f1f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.382+0000 7f04a4806b80 1 -- 172.21.15.63:0/1650978067 >> 172.21.15.63:0/1650978067 conn(0x555c5f0d3fc0 msgr2=0x555c5f1ed8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:15.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.383+0000 7f04a4806b80 1 -- 172.21.15.63:0/1650978067 shutdown_connections 2024-05-27T08:02:15.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.383+0000 7f04a4806b80 1 -- 172.21.15.63:0/1650978067 wait complete. 2024-05-27T08:02:15.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.383+0000 7f04a4806b80 1 -- start start 2024-05-27T08:02:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.384+0000 7f04a4806b80 1 Processor -- start 2024-05-27T08:02:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.384+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 0x555c5f1fc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.384+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 0x555c5f1fca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.384+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 0x555c5f1fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.384+0000 7f04a4806b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555c5f016fa0 con 0x555c5f1f2db0 2024-05-27T08:02:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.384+0000 7f04a4806b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555c5f0b1760 con 0x555c5f073e40 2024-05-27T08:02:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.384+0000 7f04a4806b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555c5f1ff5c0 con 0x555c5f1f8d30 2024-05-27T08:02:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.385+0000 7f04a4806b80 10 client.0 _ll_register_callbacks cb 0x555c5f0bac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.385+0000 7f048e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 0x555c5f1fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.385+0000 7f048e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 0x555c5f1fca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41324/0 (socket says 172.21.15.63:41324) 2024-05-27T08:02:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.385+0000 7f048e000700 1 -- 172.21.15.63:0/4119707565 learned_addr learned my addr 172.21.15.63:0/4119707565 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.385+0000 7f048ea00700 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 0x555c5f1fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.385+0000 7f048f400700 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 0x555c5f1fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f048e000700 1 -- 172.21.15.63:0/4119707565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 msgr2=0x555c5f1fcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:15.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f048e000700 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 0x555c5f1fcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f048e000700 1 -- 172.21.15.63:0/4119707565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 msgr2=0x555c5f1fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:15.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f048e000700 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 0x555c5f1fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f048e000700 1 -- 172.21.15.63:0/4119707565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c5f0608d0 con 0x555c5f1f2db0 2024-05-27T08:02:15.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f048f400700 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 0x555c5f1fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:15.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f048e000700 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 0x555c5f1fca70 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f047c00bfd0 tx=0x7f047c012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfa1029968ff295 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:15.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f0478c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:15.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f0478c00700 1 -- 172.21.15.63:0/4119707565 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c012e60 con 0x555c5f1f2db0 2024-05-27T08:02:15.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f04a4806b80 1 -- 172.21.15.63:0/4119707565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04800aec50 con 0x555c5f1f2db0 2024-05-27T08:02:15.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.386+0000 7f0478c00700 1 -- 172.21.15.63:0/4119707565 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f047c004440 con 0x555c5f1f2db0 2024-05-27T08:02:15.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.387+0000 7f0478c00700 1 -- 172.21.15.63:0/4119707565 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f047c024050 con 0x555c5f1f2db0 2024-05-27T08:02:15.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.387+0000 7f04a4806b80 10 client.20967 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:15.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.387+0000 7f04a4806b80 1 -- 172.21.15.63:0/4119707565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555c5f0e1060 con 0x555c5f1f2db0 2024-05-27T08:02:15.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.387+0000 7f0478c00700 1 -- 172.21.15.63:0/4119707565 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04800aec50 con 0x555c5f1f2db0 2024-05-27T08:02:15.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.387+0000 7f0478c00700 1 -- 172.21.15.63:0/4119707565 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f047c019020 con 0x555c5f1f2db0 2024-05-27T08:02:15.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.388+0000 7f04a4806b80 10 client.20967 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:15.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.388+0000 7f04a4806b80 10 client.20967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:15.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.388+0000 7f04a4806b80 20 client.20967 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:15.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.388+0000 7f04a4806b80 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555c5f20c430 0x555c5f20e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.388+0000 7f04a4806b80 4 client.20967 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:15.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.388+0000 7f04a4806b80 1 -- 172.21.15.63:0/4119707565 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c5f122d90 con 0x555c5f20c430 2024-05-27T08:02:15.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.389+0000 7f048ea00700 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555c5f20c430 0x555c5f20e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.390+0000 7f048ea00700 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555c5f20c430 0x555c5f20e850 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204ba5380b6da5c server_cookie=6d76347c2108585e in_seq=0 out_seq=0 2024-05-27T08:02:15.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.390+0000 7f0478c00700 10 client.20967 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.391+0000 7f0478c00700 1 -- 172.21.15.63:0/4119707565 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555c5f122d90 con 0x555c5f20c430 2024-05-27T08:02:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.391+0000 7f0478c00700 10 client.20967 handle_command_reply: tid=0 2024-05-27T08:02:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.391+0000 7f04a4806b80 1 client.20967 shutdown 2024-05-27T08:02:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 1 -- 172.21.15.63:0/4119707565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 msgr2=0x555c5f1fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 0x555c5f1fca70 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f047c00bfd0 tx=0x7f047c012730 comp rx=0 tx=0).stop 2024-05-27T08:02:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 1 -- 172.21.15.63:0/4119707565 shutdown_connections 2024-05-27T08:02:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555c5f20c430 0x555c5f20e850 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1f8d30 0x555c5f1fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f073e40 0x555c5f1fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 1 --2- 172.21.15.63:0/4119707565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f2db0 0x555c5f1fca70 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f0478c00700 0 client.20967 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:15.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 1 -- 172.21.15.63:0/4119707565 >> 172.21.15.63:0/4119707565 conn(0x555c5f0d3fc0 msgr2=0x555c5f1ed580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:15.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 1 -- 172.21.15.63:0/4119707565 shutdown_connections 2024-05-27T08:02:15.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 1 -- 172.21.15.63:0/4119707565 wait complete. 2024-05-27T08:02:15.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.392+0000 7f04a4806b80 20 client.20967 trim_cache size 0 max 16384 2024-05-27T08:02:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.417+0000 7f04a4806b80 1 Processor -- start 2024-05-27T08:02:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.417+0000 7f04a4806b80 1 -- start start 2024-05-27T08:02:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.418+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f073e40 0x555c5f1f2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.418+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f8d30 0x555c5f1f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.418+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1edb60 0x555c5f1f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.418+0000 7f04a4806b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04901b0b40 con 0x555c5f1f8d30 2024-05-27T08:02:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.418+0000 7f04a4806b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04901ae3d0 con 0x555c5f1edb60 2024-05-27T08:02:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.418+0000 7f04a4806b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f04901ac240 con 0x555c5f073e40 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.418+0000 7f048e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f8d30 0x555c5f1f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.418+0000 7f048e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f8d30 0x555c5f1f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41336/0 (socket says 172.21.15.63:41336) 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.418+0000 7f048e000700 1 -- 172.21.15.63:0/4104819736 learned_addr learned my addr 172.21.15.63:0/4104819736 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.418+0000 7f048ea00700 1 --2- 172.21.15.63:0/4104819736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f073e40 0x555c5f1f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.419+0000 7f048f400700 1 --2- 172.21.15.63:0/4104819736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1edb60 0x555c5f1f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.419+0000 7f048e000700 1 -- 172.21.15.63:0/4104819736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f073e40 msgr2=0x555c5f1f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.419+0000 7f048e000700 1 --2- 172.21.15.63:0/4104819736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f073e40 0x555c5f1f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.419+0000 7f048e000700 1 -- 172.21.15.63:0/4104819736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1edb60 msgr2=0x555c5f1f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.419+0000 7f048e000700 1 --2- 172.21.15.63:0/4104819736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1edb60 0x555c5f1f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.419+0000 7f048e000700 1 -- 172.21.15.63:0/4104819736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c5f1e9980 con 0x555c5f1f8d30 2024-05-27T08:02:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.419+0000 7f048e000700 1 --2- 172.21.15.63:0/4104819736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f8d30 0x555c5f1f6b70 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f047c000c00 tx=0x7f047c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8aec9e55404292 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.420+0000 7f047b400700 1 -- 172.21.15.63:0/4104819736 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c02c070 con 0x555c5f1f8d30 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.420+0000 7f047b400700 1 -- 172.21.15.63:0/4104819736 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f047c01be50 con 0x555c5f1f8d30 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.420+0000 7f047b400700 1 -- 172.21.15.63:0/4104819736 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c012e60 con 0x555c5f1f8d30 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.421+0000 7f04a4806b80 1 -- 172.21.15.63:0/4104819736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f8d30 msgr2=0x555c5f1f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.421+0000 7f04a4806b80 1 --2- 172.21.15.63:0/4104819736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f8d30 0x555c5f1f6b70 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f047c000c00 tx=0x7f047c005720 comp rx=0 tx=0).stop 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.421+0000 7f04a4806b80 1 -- 172.21.15.63:0/4104819736 shutdown_connections 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.421+0000 7f04a4806b80 1 --2- 172.21.15.63:0/4104819736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1edb60 0x555c5f1f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.421+0000 7f04a4806b80 1 --2- 172.21.15.63:0/4104819736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f073e40 0x555c5f1f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.421+0000 7f04a4806b80 1 --2- 172.21.15.63:0/4104819736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f1f8d30 0x555c5f1f6b70 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.421+0000 7f04a4806b80 1 -- 172.21.15.63:0/4104819736 >> 172.21.15.63:0/4104819736 conn(0x555c5f0d3fc0 msgr2=0x555c5f1fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.421+0000 7f04a4806b80 1 -- 172.21.15.63:0/4104819736 shutdown_connections 2024-05-27T08:02:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.421+0000 7f04a4806b80 1 -- 172.21.15.63:0/4104819736 wait complete. 2024-05-27T08:02:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.422+0000 7f04a4806b80 1 -- start start 2024-05-27T08:02:15.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.422+0000 7f04a4806b80 1 Processor -- start 2024-05-27T08:02:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f073e40 0x555c5f204650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1edb60 0x555c5f204b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f04a4806b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1f8d30 0x555c5f201620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f04a4806b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04901ac890 con 0x555c5f073e40 2024-05-27T08:02:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f04a4806b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04901add00 con 0x555c5f1f8d30 2024-05-27T08:02:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f04a4806b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f04901acf60 con 0x555c5f1edb60 2024-05-27T08:02:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f04a4806b80 10 client.0 _ll_register_callbacks cb 0x555c5f0bac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f048ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f073e40 0x555c5f204650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f048ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f073e40 0x555c5f204650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41350/0 (socket says 172.21.15.63:41350) 2024-05-27T08:02:15.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f048ea00700 1 -- 172.21.15.63:0/1801161967 learned_addr learned my addr 172.21.15.63:0/1801161967 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.423+0000 7f048e000700 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1edb60 0x555c5f204b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.424+0000 7f048f400700 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1f8d30 0x555c5f201620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.424+0000 7f048ea00700 1 -- 172.21.15.63:0/1801161967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1edb60 msgr2=0x555c5f204b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.424+0000 7f048ea00700 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1edb60 0x555c5f204b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.424+0000 7f048ea00700 1 -- 172.21.15.63:0/1801161967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1f8d30 msgr2=0x555c5f201620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:15.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.424+0000 7f048ea00700 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1f8d30 0x555c5f201620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.424+0000 7f048ea00700 1 -- 172.21.15.63:0/1801161967 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c5f201b60 con 0x555c5f073e40 2024-05-27T08:02:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.424+0000 7f048f400700 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1f8d30 0x555c5f201620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.424+0000 7f048ea00700 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f073e40 0x555c5f204650 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f04800aa7e0 tx=0x7f04800b3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996eac677c393558 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.424+0000 7f0478c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.425+0000 7f0478c00700 1 -- 172.21.15.63:0/1801161967 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800aa8a0 con 0x555c5f073e40 2024-05-27T08:02:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.425+0000 7f0478c00700 1 -- 172.21.15.63:0/1801161967 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f04800a6e90 con 0x555c5f073e40 2024-05-27T08:02:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.425+0000 7f04a4806b80 1 -- 172.21.15.63:0/1801161967 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f047c01be50 con 0x555c5f073e40 2024-05-27T08:02:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.425+0000 7f0478c00700 1 -- 172.21.15.63:0/1801161967 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f04800cd040 con 0x555c5f073e40 2024-05-27T08:02:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.425+0000 7f04a4806b80 10 client.20973 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.425+0000 7f04a4806b80 1 -- 172.21.15.63:0/1801161967 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555c5f277540 con 0x555c5f073e40 2024-05-27T08:02:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.426+0000 7f0478c00700 1 -- 172.21.15.63:0/1801161967 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f047c01be50 con 0x555c5f073e40 2024-05-27T08:02:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.426+0000 7f0478c00700 1 -- 172.21.15.63:0/1801161967 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f04800cbce0 con 0x555c5f073e40 2024-05-27T08:02:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.426+0000 7f04a4806b80 10 client.20973 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.426+0000 7f04a4806b80 10 client.20973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.426+0000 7f04a4806b80 20 client.20973 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.426+0000 7f04a4806b80 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555c5f277b00 0x555c5f279f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.426+0000 7f04a4806b80 4 client.20973 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:02:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.426+0000 7f04a4806b80 1 -- 172.21.15.63:0/1801161967 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c5f11c570 con 0x555c5f277b00 2024-05-27T08:02:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.427+0000 7f048e000700 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555c5f277b00 0x555c5f279f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.428+0000 7f048e000700 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555c5f277b00 0x555c5f279f20 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5ae37e5ef27b server_cookie=fd9ab761fa93fe99 in_seq=0 out_seq=0 2024-05-27T08:02:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.428+0000 7f0478c00700 10 client.20973 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.428+0000 7f0478c00700 1 -- 172.21.15.63:0/1801161967 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555c5f11c570 con 0x555c5f277b00 2024-05-27T08:02:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.428+0000 7f0478c00700 10 client.20973 handle_command_reply: tid=0 2024-05-27T08:02:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.428+0000 7f04a4806b80 1 client.20973 shutdown 2024-05-27T08:02:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 1 -- 172.21.15.63:0/1801161967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f073e40 msgr2=0x555c5f204650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f073e40 0x555c5f204650 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f04800aa7e0 tx=0x7f04800b3040 comp rx=0 tx=0).stop 2024-05-27T08:02:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 1 -- 172.21.15.63:0/1801161967 shutdown_connections 2024-05-27T08:02:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555c5f277b00 0x555c5f279f20 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c5f1f8d30 0x555c5f201620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555c5f073e40 0x555c5f204650 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 1 --2- 172.21.15.63:0/1801161967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555c5f1edb60 0x555c5f204b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f0478c00700 0 client.20973 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 1 -- 172.21.15.63:0/1801161967 >> 172.21.15.63:0/1801161967 conn(0x555c5f0d3fc0 msgr2=0x555c5f1f68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 1 -- 172.21.15.63:0/1801161967 shutdown_connections 2024-05-27T08:02:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 1 -- 172.21.15.63:0/1801161967 wait complete. 2024-05-27T08:02:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.429+0000 7f04a4806b80 20 client.20973 trim_cache size 0 max 16384 2024-05-27T08:02:15.433 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:02:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:02:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:02:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: "7ab5466e-c8b1-42f3-b874-bffed94ccac8": { 2024-05-27T08:02:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:02:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "7ab5466e-c8b1-42f3-b874-bffed94ccac8", 2024-05-27T08:02:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:02:15.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:02:15.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:02:15.434 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:02:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.430+0000 7f0495c00700 1 -- 172.21.15.63:0/1300752521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04740b29f0 msgr2=0x7f04740b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.430+0000 7f0495c00700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04740b29f0 0x7f04740b4f00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0488009710 tx=0x7f04880090b0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.430+0000 7f0495c00700 1 -- 172.21.15.63:0/1300752521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0490104600 msgr2=0x7f0490075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.430+0000 7f0495c00700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0490104600 0x7f0490075ab0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f0480000c00 tx=0x7f048000f510 comp rx=0 tx=0).stop 2024-05-27T08:02:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.431+0000 7f0495c00700 1 -- 172.21.15.63:0/1300752521 shutdown_connections 2024-05-27T08:02:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.431+0000 7f0495c00700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f04740b29f0 0x7f04740b4f00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.431+0000 7f0495c00700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0490109880 0x7f0490076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.431+0000 7f0495c00700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0490104600 0x7f0490075ab0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.431+0000 7f0495c00700 1 --2- 172.21.15.63:0/1300752521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0490108f30 0x7f0490075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.431+0000 7f0495c00700 1 -- 172.21.15.63:0/1300752521 >> 172.21.15.63:0/1300752521 conn(0x7f049007af60 msgr2=0x7f0490101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.431+0000 7f0495c00700 1 -- 172.21.15.63:0/1300752521 shutdown_connections 2024-05-27T08:02:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:15.431+0000 7f0495c00700 1 -- 172.21.15.63:0/1300752521 wait complete. 2024-05-27T08:02:15.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:15 smithi063 ceph-mon[31613]: pgmap v910: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.8 MiB/s wr, 57 op/s 2024-05-27T08:02:15.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:15 smithi086 ceph-mon[34687]: pgmap v910: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.8 MiB/s wr, 57 op/s 2024-05-27T08:02:15.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:15 smithi149 ceph-mon[35408]: pgmap v910: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.8 MiB/s wr, 57 op/s 2024-05-27T08:02:16.347 INFO:tasks.cephfs.filesystem:scrub status for tag:7ab5466e-c8b1-42f3-b874-bffed94ccac8 - {'path': '/', 'tag': '7ab5466e-c8b1-42f3-b874-bffed94ccac8', 'options': 'recursive,force'} 2024-05-27T08:02:17.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:17 smithi086 ceph-mon[34687]: pgmap v911: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.4 MiB/s wr, 47 op/s 2024-05-27T08:02:17.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:17 smithi149 ceph-mon[35408]: pgmap v911: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.4 MiB/s wr, 47 op/s 2024-05-27T08:02:17.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:17 smithi063 ceph-mon[31613]: pgmap v911: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.4 MiB/s wr, 47 op/s 2024-05-27T08:02:19.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:19 smithi086 ceph-mon[34687]: pgmap v912: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:02:19.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:19 smithi149 ceph-mon[35408]: pgmap v912: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:02:19.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:19 smithi063 ceph-mon[31613]: pgmap v912: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:02:21.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:21 smithi086 ceph-mon[34687]: pgmap v913: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:02:21.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:21 smithi149 ceph-mon[35408]: pgmap v913: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:02:21.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:21 smithi063 ceph-mon[31613]: pgmap v913: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 51 op/s 2024-05-27T08:02:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:02:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:02:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:02:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:02:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:02:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:02:22.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:02:22.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:02:22.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:02:23.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:23 smithi086 ceph-mon[34687]: pgmap v914: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.9 MiB/s wr, 49 op/s 2024-05-27T08:02:23.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:23 smithi149 ceph-mon[35408]: pgmap v914: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.9 MiB/s wr, 49 op/s 2024-05-27T08:02:23.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:23 smithi063 ceph-mon[31613]: pgmap v914: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.9 MiB/s wr, 49 op/s 2024-05-27T08:02:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:25 smithi086 ceph-mon[34687]: pgmap v915: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:02:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:25 smithi149 ceph-mon[35408]: pgmap v915: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:02:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:25 smithi063 ceph-mon[31613]: pgmap v915: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.547+0000 7f6826600640 1 -- 172.21.15.63:0/3259479584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f682010a890 msgr2=0x7f6820116b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.547+0000 7f6826600640 1 --2- 172.21.15.63:0/3259479584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f682010a890 0x7f6820116b50 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f6818009ca0 tx=0x7f681802a950 comp rx=0 tx=0).stop 2024-05-27T08:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.547+0000 7f6826600640 1 -- 172.21.15.63:0/3259479584 shutdown_connections 2024-05-27T08:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.547+0000 7f6826600640 1 --2- 172.21.15.63:0/3259479584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f682010a890 0x7f6820116b50 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.547+0000 7f6826600640 1 --2- 172.21.15.63:0/3259479584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6820110520 0x7f682010a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.547+0000 7f6826600640 1 --2- 172.21.15.63:0/3259479584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820110110 0x7f6820109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.547+0000 7f6826600640 1 -- 172.21.15.63:0/3259479584 >> 172.21.15.63:0/3259479584 conn(0x7f6820076070 msgr2=0x7f6820078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.547+0000 7f6826600640 1 -- 172.21.15.63:0/3259479584 shutdown_connections 2024-05-27T08:02:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.547+0000 7f6826600640 1 -- 172.21.15.63:0/3259479584 wait complete. 2024-05-27T08:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.548+0000 7f6826600640 1 Processor -- start 2024-05-27T08:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.549+0000 7f6826600640 1 -- start start 2024-05-27T08:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.549+0000 7f6826600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820110110 0x7f68201a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.549+0000 7f6826600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6820110520 0x7f68201a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.549+0000 7f6826600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68201a4f40 0x7f68201a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.549+0000 7f6826600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f682011a180 con 0x7f6820110110 2024-05-27T08:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.549+0000 7f6826600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f682011a000 con 0x7f68201a4f40 2024-05-27T08:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.549+0000 7f6826600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f682011a300 con 0x7f6820110520 2024-05-27T08:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.549+0000 7f6825200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820110110 0x7f68201a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.549+0000 7f6825200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820110110 0x7f68201a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41378/0 (socket says 172.21.15.63:41378) 2024-05-27T08:02:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.549+0000 7f6825200640 1 -- 172.21.15.63:0/4176414016 learned_addr learned my addr 172.21.15.63:0/4176414016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.550+0000 7f6825c00640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68201a4f40 0x7f68201a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.550+0000 7f681fe00640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6820110520 0x7f68201a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.550+0000 7f6825200640 1 -- 172.21.15.63:0/4176414016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6820110520 msgr2=0x7f68201a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.550+0000 7f6825200640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6820110520 0x7f68201a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.550+0000 7f6825200640 1 -- 172.21.15.63:0/4176414016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68201a4f40 msgr2=0x7f68201a5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.550+0000 7f6825200640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68201a4f40 0x7f68201a5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.550+0000 7f6825200640 1 -- 172.21.15.63:0/4176414016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68201a5990 con 0x7f6820110110 2024-05-27T08:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.550+0000 7f681fe00640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6820110520 0x7f68201a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.551+0000 7f6825200640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820110110 0x7f68201a0720 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f681400d770 tx=0x7f681400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.551+0000 7f681d600640 1 -- 172.21.15.63:0/4176414016 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814017070 con 0x7f6820110110 2024-05-27T08:02:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.551+0000 7f681d600640 1 -- 172.21.15.63:0/4176414016 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6814009c30 con 0x7f6820110110 2024-05-27T08:02:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.551+0000 7f6826600640 1 -- 172.21.15.63:0/4176414016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68201a5c80 con 0x7f6820110110 2024-05-27T08:02:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.551+0000 7f681d600640 1 -- 172.21.15.63:0/4176414016 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814003c50 con 0x7f6820110110 2024-05-27T08:02:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.551+0000 7f6826600640 1 -- 172.21.15.63:0/4176414016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68201ad480 con 0x7f6820110110 2024-05-27T08:02:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.556+0000 7f682eb84740 1 Processor -- start 2024-05-27T08:02:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.556+0000 7f682eb84740 1 -- start start 2024-05-27T08:02:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.556+0000 7f682eb84740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823f21b0 0x558d82794a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.556+0000 7f682eb84740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d82794fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.556+0000 7f682eb84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 0x558d827954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.556+0000 7f682eb84740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558d82705b90 con 0x558d823c92f0 2024-05-27T08:02:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.556+0000 7f682eb84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558d826f5730 con 0x558d823d14d0 2024-05-27T08:02:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.556+0000 7f682eb84740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558d826f9770 con 0x558d823f21b0 2024-05-27T08:02:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.557+0000 7f681d600640 1 -- 172.21.15.63:0/4176414016 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6814003df0 con 0x7f6820110110 2024-05-27T08:02:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.557+0000 7f681fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 0x558d827954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.557+0000 7f6825c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823f21b0 0x558d82794a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.557+0000 7f681fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 0x558d827954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56952/0 (socket says 172.21.15.63:56952) 2024-05-27T08:02:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.557+0000 7f681fe00640 1 -- 172.21.15.63:0/1239831472 learned_addr learned my addr 172.21.15.63:0/1239831472 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.557+0000 7f6825200640 1 --2- 172.21.15.63:0/1239831472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d82794fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.558+0000 7f681fe00640 1 -- 172.21.15.63:0/1239831472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823f21b0 msgr2=0x558d82794a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.558+0000 7f681fe00640 1 --2- 172.21.15.63:0/1239831472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823f21b0 0x558d82794a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.558+0000 7f681fe00640 1 -- 172.21.15.63:0/1239831472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 msgr2=0x558d82794fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.558+0000 7f681fe00640 1 --2- 172.21.15.63:0/1239831472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d82794fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.558+0000 7f681fe00640 1 -- 172.21.15.63:0/1239831472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d8268e8b0 con 0x558d823d14d0 2024-05-27T08:02:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.558+0000 7f681fe00640 1 --2- 172.21.15.63:0/1239831472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 0x558d827954e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f680c00bce0 tx=0x7f680c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0bee7eb47df4f10 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.559+0000 7f6812000640 1 -- 172.21.15.63:0/1239831472 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c015070 con 0x558d823d14d0 2024-05-27T08:02:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.559+0000 7f6812000640 1 -- 172.21.15.63:0/1239831472 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f680c01a8f0 con 0x558d823d14d0 2024-05-27T08:02:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.559+0000 7f6812000640 1 -- 172.21.15.63:0/1239831472 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c011860 con 0x558d823d14d0 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.559+0000 7f681d600640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68080b2950 0x7f68080b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.559+0000 7f681d600640 1 -- 172.21.15.63:0/4176414016 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f68140e2440 con 0x7f6820110110 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.559+0000 7f6825c00640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68080b2950 0x7f68080b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.560+0000 7f682eb84740 1 -- 172.21.15.63:0/1239831472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 msgr2=0x558d827954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.560+0000 7f682eb84740 1 --2- 172.21.15.63:0/1239831472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 0x558d827954e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f680c00bce0 tx=0x7f680c00f730 comp rx=0 tx=0).stop 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.560+0000 7f682eb84740 1 -- 172.21.15.63:0/1239831472 shutdown_connections 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.560+0000 7f682eb84740 1 --2- 172.21.15.63:0/1239831472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 0x558d827954e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.560+0000 7f682eb84740 1 --2- 172.21.15.63:0/1239831472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d82794fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.560+0000 7f682eb84740 1 --2- 172.21.15.63:0/1239831472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823f21b0 0x558d82794a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.560+0000 7f682eb84740 1 -- 172.21.15.63:0/1239831472 >> 172.21.15.63:0/1239831472 conn(0x558d823e6300 msgr2=0x558d82791210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.560+0000 7f682eb84740 1 -- 172.21.15.63:0/1239831472 shutdown_connections 2024-05-27T08:02:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.560+0000 7f682eb84740 1 -- 172.21.15.63:0/1239831472 wait complete. 2024-05-27T08:02:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.561+0000 7f6825c00640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68080b2950 0x7f68080b4e60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f68180094e0 tx=0x7f6818005c50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.561+0000 7f682eb84740 1 -- start start 2024-05-27T08:02:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.561+0000 7f682eb84740 1 Processor -- start 2024-05-27T08:02:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.562+0000 7f682eb84740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d8279ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.562+0000 7f682eb84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 0x558d8279f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.562+0000 7f682eb84740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823f21b0 0x558d827aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.562+0000 7f682eb84740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558d823db790 con 0x558d823c92f0 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.562+0000 7f682eb84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558d8269dbc0 con 0x558d823d14d0 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.562+0000 7f682eb84740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558d827a1b90 con 0x558d823f21b0 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.562+0000 7f682eb84740 10 client.0 _ll_register_callbacks cb 0x558d8267b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.562+0000 7f6825200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d8279ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.562+0000 7f6825200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d8279ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41392/0 (socket says 172.21.15.63:41392) 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.562+0000 7f6825200640 1 -- 172.21.15.63:0/2769454641 learned_addr learned my addr 172.21.15.63:0/2769454641 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.563+0000 7f681fe00640 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 0x558d8279f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.563+0000 7f6825c00640 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823f21b0 0x558d827aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.563+0000 7f6825200640 1 -- 172.21.15.63:0/2769454641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823f21b0 msgr2=0x558d827aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.563+0000 7f6825200640 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823f21b0 0x558d827aaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.563+0000 7f6825200640 1 -- 172.21.15.63:0/2769454641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 msgr2=0x558d8279f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.563+0000 7f6825200640 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 0x558d8279f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.563+0000 7f6825200640 1 -- 172.21.15.63:0/2769454641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d8240a210 con 0x558d823c92f0 2024-05-27T08:02:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.563+0000 7f6825200640 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d8279ebc0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f681400d740 tx=0x7f68140dfd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8827eebccab7cf31 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.563+0000 7f6806a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.564+0000 7f6806a00640 1 -- 172.21.15.63:0/2769454641 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814020070 con 0x558d823c92f0 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.564+0000 7f682eb84740 1 -- 172.21.15.63:0/2769454641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f680c01a8f0 con 0x558d823c92f0 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.564+0000 7f6806a00640 1 -- 172.21.15.63:0/2769454641 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f68140e2150 con 0x558d823c92f0 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.564+0000 7f6806a00640 1 -- 172.21.15.63:0/2769454641 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f68140a8c50 con 0x558d823c92f0 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.564+0000 7f682eb84740 10 client.20985 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.564+0000 7f682eb84740 1 -- 172.21.15.63:0/2769454641 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558d824d6620 con 0x558d823c92f0 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.564+0000 7f6806a00640 1 -- 172.21.15.63:0/2769454641 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f680c01a8f0 con 0x558d823c92f0 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.565+0000 7f6806a00640 1 -- 172.21.15.63:0/2769454641 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f68140ad120 con 0x558d823c92f0 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.565+0000 7f682eb84740 10 client.20985 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.565+0000 7f682eb84740 10 client.20985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.565+0000 7f682eb84740 20 client.20985 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.565+0000 7f682eb84740 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d823b0d80 0x558d827b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.565+0000 7f682eb84740 4 client.20985 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.565+0000 7f682eb84740 1 -- 172.21.15.63:0/2769454641 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d82707ae0 con 0x558d823b0d80 2024-05-27T08:02:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.565+0000 7f681fe00640 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d823b0d80 0x558d827b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.566+0000 7f681fe00640 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d823b0d80 0x558d827b0eb0 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7768d190278b4800 server_cookie=c1985147e7a26c3c in_seq=0 out_seq=0 2024-05-27T08:02:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.566+0000 7f6806a00640 10 client.20985 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.568+0000 7f6806a00640 1 -- 172.21.15.63:0/2769454641 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558d82707ae0 con 0x558d823b0d80 2024-05-27T08:02:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.568+0000 7f6806a00640 10 client.20985 handle_command_reply: tid=0 2024-05-27T08:02:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.568+0000 7f682eb84740 1 client.20985 shutdown 2024-05-27T08:02:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 1 -- 172.21.15.63:0/2769454641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 msgr2=0x558d8279ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d8279ebc0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f681400d740 tx=0x7f68140dfd10 comp rx=0 tx=0).stop 2024-05-27T08:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 1 -- 172.21.15.63:0/2769454641 shutdown_connections 2024-05-27T08:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d823b0d80 0x558d827b0eb0 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823f21b0 0x558d827aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823d14d0 0x558d8279f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 1 --2- 172.21.15.63:0/2769454641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d8279ebc0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 1 -- 172.21.15.63:0/2769454641 >> 172.21.15.63:0/2769454641 conn(0x558d823e6300 msgr2=0x558d82796780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f6806a00640 0 client.20985 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 1 -- 172.21.15.63:0/2769454641 shutdown_connections 2024-05-27T08:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 1 -- 172.21.15.63:0/2769454641 wait complete. 2024-05-27T08:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.569+0000 7f682eb84740 20 client.20985 trim_cache size 0 max 16384 2024-05-27T08:02:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.593+0000 7f682eb84740 1 Processor -- start 2024-05-27T08:02:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.593+0000 7f682eb84740 1 -- start start 2024-05-27T08:02:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.594+0000 7f682eb84740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d827a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.594+0000 7f682eb84740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823d14d0 0x558d827959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.594+0000 7f682eb84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 0x558d8279eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.594+0000 7f682eb84740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b9e90 con 0x558d823c92f0 2024-05-27T08:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.594+0000 7f682eb84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b7760 con 0x558d823f21b0 2024-05-27T08:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.594+0000 7f682eb84740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b55d0 con 0x558d823d14d0 2024-05-27T08:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.594+0000 7f6825200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d827a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.594+0000 7f6825200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d827a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41408/0 (socket says 172.21.15.63:41408) 2024-05-27T08:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.594+0000 7f6825200640 1 -- 172.21.15.63:0/488874492 learned_addr learned my addr 172.21.15.63:0/488874492 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.595+0000 7f681fe00640 1 --2- 172.21.15.63:0/488874492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823d14d0 0x558d827959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.595+0000 7f6825c00640 1 --2- 172.21.15.63:0/488874492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 0x558d8279eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.595+0000 7f6825200640 1 -- 172.21.15.63:0/488874492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823d14d0 msgr2=0x558d827959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.595+0000 7f6825200640 1 --2- 172.21.15.63:0/488874492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823d14d0 0x558d827959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.595+0000 7f6825200640 1 -- 172.21.15.63:0/488874492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 msgr2=0x558d8279eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.595+0000 7f6825200640 1 --2- 172.21.15.63:0/488874492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 0x558d8279eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.595+0000 7f6825200640 1 -- 172.21.15.63:0/488874492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d827947f0 con 0x558d823c92f0 2024-05-27T08:02:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.595+0000 7f6825c00640 1 --2- 172.21.15.63:0/488874492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 0x558d8279eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.595+0000 7f6825200640 1 --2- 172.21.15.63:0/488874492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d827a0d10 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f681400df50 tx=0x7f68140ad6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d20ccb9547422a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.595+0000 7f6811600640 1 -- 172.21.15.63:0/488874492 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814020070 con 0x558d823c92f0 2024-05-27T08:02:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.596+0000 7f6811600640 1 -- 172.21.15.63:0/488874492 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68140a8e10 con 0x558d823c92f0 2024-05-27T08:02:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.596+0000 7f6811600640 1 -- 172.21.15.63:0/488874492 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68140a8a70 con 0x558d823c92f0 2024-05-27T08:02:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.596+0000 7f682eb84740 1 -- 172.21.15.63:0/488874492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 msgr2=0x558d827a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.596+0000 7f682eb84740 1 --2- 172.21.15.63:0/488874492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d827a0d10 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f681400df50 tx=0x7f68140ad6b0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.597+0000 7f682eb84740 1 -- 172.21.15.63:0/488874492 shutdown_connections 2024-05-27T08:02:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.597+0000 7f682eb84740 1 --2- 172.21.15.63:0/488874492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 0x558d8279eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.597+0000 7f682eb84740 1 --2- 172.21.15.63:0/488874492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823d14d0 0x558d827959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.597+0000 7f682eb84740 1 --2- 172.21.15.63:0/488874492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823c92f0 0x558d827a0d10 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.597+0000 7f682eb84740 1 -- 172.21.15.63:0/488874492 >> 172.21.15.63:0/488874492 conn(0x558d823e6300 msgr2=0x558d8279e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.597+0000 7f682eb84740 1 -- 172.21.15.63:0/488874492 shutdown_connections 2024-05-27T08:02:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.597+0000 7f682eb84740 1 -- 172.21.15.63:0/488874492 wait complete. 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.597+0000 7f682eb84740 1 -- start start 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f682eb84740 1 Processor -- start 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f682eb84740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823c92f0 0x558d82798b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f682eb84740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823d14d0 0x558d82799060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f682eb84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 0x558d827995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f682eb84740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b5c20 con 0x558d823d14d0 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f682eb84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b7090 con 0x558d823f21b0 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f682eb84740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b62f0 con 0x558d823c92f0 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f682eb84740 10 client.0 _ll_register_callbacks cb 0x7f68140e5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f681fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823d14d0 0x558d82799060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f681fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823d14d0 0x558d82799060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41410/0 (socket says 172.21.15.63:41410) 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.598+0000 7f681fe00640 1 -- 172.21.15.63:0/563857446 learned_addr learned my addr 172.21.15.63:0/563857446 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.599+0000 7f6825c00640 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 0x558d827995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.599+0000 7f6825200640 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823c92f0 0x558d82798b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.599+0000 7f681fe00640 1 -- 172.21.15.63:0/563857446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823c92f0 msgr2=0x558d82798b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.599+0000 7f681fe00640 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823c92f0 0x558d82798b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.599+0000 7f681fe00640 1 -- 172.21.15.63:0/563857446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 msgr2=0x558d827995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.599+0000 7f681fe00640 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 0x558d827995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.599+0000 7f681fe00640 1 -- 172.21.15.63:0/563857446 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d827a5fd0 con 0x558d823d14d0 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.599+0000 7f6825c00640 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 0x558d827995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.599+0000 7f681fe00640 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823d14d0 0x558d82799060 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f680c011480 tx=0x7f680c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9495ca5999daae53 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.599+0000 7f6806a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.600+0000 7f6806a00640 1 -- 172.21.15.63:0/563857446 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c02a070 con 0x558d823d14d0 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.600+0000 7f682eb84740 1 -- 172.21.15.63:0/563857446 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68140a8e10 con 0x558d823d14d0 2024-05-27T08:02:26.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.600+0000 7f6806a00640 1 -- 172.21.15.63:0/563857446 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f680c005240 con 0x558d823d14d0 2024-05-27T08:02:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.600+0000 7f6806a00640 1 -- 172.21.15.63:0/563857446 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f680c036050 con 0x558d823d14d0 2024-05-27T08:02:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.600+0000 7f682eb84740 10 client.20991 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.600+0000 7f682eb84740 1 -- 172.21.15.63:0/563857446 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558d827a7220 con 0x558d823d14d0 2024-05-27T08:02:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.600+0000 7f6806a00640 1 -- 172.21.15.63:0/563857446 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68140a8e10 con 0x558d823d14d0 2024-05-27T08:02:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.601+0000 7f6806a00640 1 -- 172.21.15.63:0/563857446 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f680c0263c0 con 0x558d823d14d0 2024-05-27T08:02:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.601+0000 7f682eb84740 10 client.20991 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.601+0000 7f682eb84740 10 client.20991 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.601+0000 7f682eb84740 20 client.20991 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.601+0000 7f682eb84740 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d823861c0 0x558d8281b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.601+0000 7f682eb84740 4 client.20991 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:02:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.601+0000 7f682eb84740 1 -- 172.21.15.63:0/563857446 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d827012c0 con 0x558d823861c0 2024-05-27T08:02:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.601+0000 7f6825200640 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d823861c0 0x558d8281b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.603+0000 7f6825200640 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d823861c0 0x558d8281b590 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4063eb73be9d3fd server_cookie=90b3d9ff5c8d2254 in_seq=0 out_seq=0 2024-05-27T08:02:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.603+0000 7f6806a00640 10 client.20991 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.644+0000 7f6806a00640 1 -- 172.21.15.63:0/563857446 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558d827012c0 con 0x558d823861c0 2024-05-27T08:02:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.644+0000 7f6806a00640 10 client.20991 handle_command_reply: tid=0 2024-05-27T08:02:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.644+0000 7f682eb84740 1 client.20991 shutdown 2024-05-27T08:02:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 1 -- 172.21.15.63:0/563857446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823d14d0 msgr2=0x558d82799060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823d14d0 0x558d82799060 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f680c011480 tx=0x7f680c00bdd0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 1 -- 172.21.15.63:0/563857446 shutdown_connections 2024-05-27T08:02:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d823861c0 0x558d8281b590 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d823f21b0 0x558d827995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d823d14d0 0x558d82799060 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 1 --2- 172.21.15.63:0/563857446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d823c92f0 0x558d82798b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f6806a00640 0 client.20991 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 1 -- 172.21.15.63:0/563857446 >> 172.21.15.63:0/563857446 conn(0x558d823e6300 msgr2=0x558d8279ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 1 -- 172.21.15.63:0/563857446 shutdown_connections 2024-05-27T08:02:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 1 -- 172.21.15.63:0/563857446 wait complete. 2024-05-27T08:02:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.645+0000 7f682eb84740 20 client.20991 trim_cache size 0 max 16384 2024-05-27T08:02:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:02:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:02:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:02:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.646+0000 7f6826600640 1 -- 172.21.15.63:0/4176414016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68080b2950 msgr2=0x7f68080b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.646+0000 7f6826600640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68080b2950 0x7f68080b4e60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f68180094e0 tx=0x7f6818005c50 comp rx=0 tx=0).stop 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.646+0000 7f6826600640 1 -- 172.21.15.63:0/4176414016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820110110 msgr2=0x7f68201a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.646+0000 7f6826600640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820110110 0x7f68201a0720 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f681400d770 tx=0x7f681400dc30 comp rx=0 tx=0).stop 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.646+0000 7f6826600640 1 -- 172.21.15.63:0/4176414016 shutdown_connections 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.646+0000 7f6826600640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f68080b2950 0x7f68080b4e60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.646+0000 7f6826600640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68201a4f40 0x7f68201a5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.646+0000 7f6826600640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6820110520 0x7f68201a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.646+0000 7f6826600640 1 --2- 172.21.15.63:0/4176414016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6820110110 0x7f68201a0720 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.646+0000 7f6826600640 1 -- 172.21.15.63:0/4176414016 >> 172.21.15.63:0/4176414016 conn(0x7f6820076070 msgr2=0x7f6820077120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.647+0000 7f6826600640 1 -- 172.21.15.63:0/4176414016 shutdown_connections 2024-05-27T08:02:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:26.647+0000 7f6826600640 1 -- 172.21.15.63:0/4176414016 wait complete. 2024-05-27T08:02:27.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:27 smithi086 ceph-mon[34687]: pgmap v916: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:02:27.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:27 smithi149 ceph-mon[35408]: pgmap v916: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:02:27.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:27 smithi063 ceph-mon[31613]: pgmap v916: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:02:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:29 smithi086 ceph-mon[34687]: pgmap v917: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 17 op/s 2024-05-27T08:02:29.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:29 smithi149 ceph-mon[35408]: pgmap v917: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 17 op/s 2024-05-27T08:02:29.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:29 smithi063 ceph-mon[31613]: pgmap v917: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 17 op/s 2024-05-27T08:02:31.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:31 smithi086 ceph-mon[34687]: pgmap v918: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.3 MiB/s wr, 9 op/s 2024-05-27T08:02:31.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:31 smithi149 ceph-mon[35408]: pgmap v918: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.3 MiB/s wr, 9 op/s 2024-05-27T08:02:31.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:31 smithi063 ceph-mon[31613]: pgmap v918: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.3 MiB/s wr, 9 op/s 2024-05-27T08:02:33.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:33 smithi086 ceph-mon[34687]: pgmap v919: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 12 op/s 2024-05-27T08:02:33.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:33 smithi149 ceph-mon[35408]: pgmap v919: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 12 op/s 2024-05-27T08:02:33.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:33 smithi063 ceph-mon[31613]: pgmap v919: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 12 op/s 2024-05-27T08:02:35.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:35 smithi086 ceph-mon[34687]: pgmap v920: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:02:35.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:35 smithi149 ceph-mon[35408]: pgmap v920: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:02:35.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:35 smithi063 ceph-mon[31613]: pgmap v920: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:02:37.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:37 smithi086 ceph-mon[34687]: pgmap v921: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:02:37.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:37 smithi149 ceph-mon[35408]: pgmap v921: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:02:37.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:37 smithi063 ceph-mon[31613]: pgmap v921: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:02:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.899+0000 7fc8ef000640 1 -- 172.21.15.63:0/1909839816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e81104d0 msgr2=0x7fc8e81128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.899+0000 7fc8ef000640 1 --2- 172.21.15.63:0/1909839816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e81104d0 0x7fc8e81128f0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009ca0 tx=0x7fc8e002a950 comp rx=0 tx=0).stop 2024-05-27T08:02:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.899+0000 7fc8ef000640 1 -- 172.21.15.63:0/1909839816 shutdown_connections 2024-05-27T08:02:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.899+0000 7fc8ef000640 1 --2- 172.21.15.63:0/1909839816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e81104d0 0x7fc8e81128f0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.899+0000 7fc8ef000640 1 --2- 172.21.15.63:0/1909839816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e810db70 0x7fc8e810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.899+0000 7fc8ef000640 1 --2- 172.21.15.63:0/1909839816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e8076530 0x7fc8e8076940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.899+0000 7fc8ef000640 1 -- 172.21.15.63:0/1909839816 >> 172.21.15.63:0/1909839816 conn(0x7fc8e80ff340 msgr2=0x7fc8e8101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.899+0000 7fc8ef000640 1 -- 172.21.15.63:0/1909839816 shutdown_connections 2024-05-27T08:02:39.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.899+0000 7fc8ef000640 1 -- 172.21.15.63:0/1909839816 wait complete. 2024-05-27T08:02:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.901+0000 7fc8ef000640 1 Processor -- start 2024-05-27T08:02:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.901+0000 7fc8ef000640 1 -- start start 2024-05-27T08:02:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.901+0000 7fc8ef000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e8076530 0x7fc8e81a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.901+0000 7fc8ef000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e810db70 0x7fc8e81a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.901+0000 7fc8ef000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e81104d0 0x7fc8e819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.901+0000 7fc8ef000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8115f00 con 0x7fc8e81104d0 2024-05-27T08:02:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.901+0000 7fc8ef000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8115d80 con 0x7fc8e8076530 2024-05-27T08:02:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.901+0000 7fc8ef000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8116080 con 0x7fc8e810db70 2024-05-27T08:02:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.902+0000 7fc8ed000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e81104d0 0x7fc8e819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.902+0000 7fc8ed000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e81104d0 0x7fc8e819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50580/0 (socket says 172.21.15.63:50580) 2024-05-27T08:02:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.902+0000 7fc8ed000640 1 -- 172.21.15.63:0/376890133 learned_addr learned my addr 172.21.15.63:0/376890133 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.902+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e8076530 0x7fc8e81a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.902+0000 7fc8e7400640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e810db70 0x7fc8e81a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.902+0000 7fc8ed000640 1 -- 172.21.15.63:0/376890133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e810db70 msgr2=0x7fc8e81a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.902+0000 7fc8ed000640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e810db70 0x7fc8e81a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.902+0000 7fc8ed000640 1 -- 172.21.15.63:0/376890133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e8076530 msgr2=0x7fc8e81a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.902+0000 7fc8ed000640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e8076530 0x7fc8e81a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.902+0000 7fc8ed000640 1 -- 172.21.15.63:0/376890133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e819ce90 con 0x7fc8e81104d0 2024-05-27T08:02:39.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.903+0000 7fc8ed000640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e81104d0 0x7fc8e819c760 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fc8e002ae70 tx=0x7fc8e0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.903+0000 7fc8e4c00640 1 -- 172.21.15.63:0/376890133 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0034040 con 0x7fc8e81104d0 2024-05-27T08:02:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.903+0000 7fc8ef000640 1 -- 172.21.15.63:0/376890133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e819d0c0 con 0x7fc8e81104d0 2024-05-27T08:02:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.903+0000 7fc8e4c00640 1 -- 172.21.15.63:0/376890133 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8e0030850 con 0x7fc8e81104d0 2024-05-27T08:02:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.903+0000 7fc8e4c00640 1 -- 172.21.15.63:0/376890133 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0004670 con 0x7fc8e81104d0 2024-05-27T08:02:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.903+0000 7fc8ef000640 1 -- 172.21.15.63:0/376890133 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e81a9380 con 0x7fc8e81104d0 2024-05-27T08:02:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.907+0000 7fc8f5fce740 1 Processor -- start 2024-05-27T08:02:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.908+0000 7fc8e4c00640 1 -- 172.21.15.63:0/376890133 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc8e0037020 con 0x7fc8e81104d0 2024-05-27T08:02:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.908+0000 7fc8f5fce740 1 -- start start 2024-05-27T08:02:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.909+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x55582330e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.909+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582330eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.909+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f4b4d0 0x55582330f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.909+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55582327fb90 con 0x555822f432f0 2024-05-27T08:02:39.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.909+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55582326f730 con 0x555822f4b4d0 2024-05-27T08:02:39.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.909+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555823273770 con 0x555822f6c1b0 2024-05-27T08:02:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.909+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582330eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.909+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582330eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50590/0 (socket says 172.21.15.63:50590) 2024-05-27T08:02:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.909+0000 7fc8e7e00640 1 -- 172.21.15.63:0/613168990 learned_addr learned my addr 172.21.15.63:0/613168990 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.909+0000 7fc8e7400640 1 --2- 172.21.15.63:0/613168990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x55582330e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.910+0000 7fc8e7400640 1 --2- 172.21.15.63:0/613168990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f4b4d0 0x55582330f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.910+0000 7fc8e7e00640 1 -- 172.21.15.63:0/613168990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 msgr2=0x55582330e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.910+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/613168990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x55582330e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.910+0000 7fc8e7e00640 1 -- 172.21.15.63:0/613168990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f4b4d0 msgr2=0x55582330f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.910+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/613168990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f4b4d0 0x55582330f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.910+0000 7fc8e7e00640 1 -- 172.21.15.63:0/613168990 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558232088b0 con 0x555822f432f0 2024-05-27T08:02:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.910+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/613168990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582330eda0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800b760 tx=0x7fc8d800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741bd65acebe46a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.910+0000 7fc8e4c00640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8cc0b2950 0x7fc8cc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.911+0000 7fc8e4c00640 1 -- 172.21.15.63:0/376890133 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fc8e0103900 con 0x7fc8e81104d0 2024-05-27T08:02:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.911+0000 7fc8e7400640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8cc0b2950 0x7fc8cc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.911+0000 7fc8dd600640 1 -- 172.21.15.63:0/613168990 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8003dd0 con 0x555822f432f0 2024-05-27T08:02:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.911+0000 7fc8dd600640 1 -- 172.21.15.63:0/613168990 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8d8015070 con 0x555822f432f0 2024-05-27T08:02:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.911+0000 7fc8dd600640 1 -- 172.21.15.63:0/613168990 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8004c80 con 0x555822f432f0 2024-05-27T08:02:39.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.912+0000 7fc8f5fce740 1 -- 172.21.15.63:0/613168990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 msgr2=0x55582330eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.912+0000 7fc8e7400640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8cc0b2950 0x7fc8cc0b4e60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55582330b9d0 tx=0x7fc8d40046c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.912+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/613168990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582330eda0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800b760 tx=0x7fc8d800bc20 comp rx=0 tx=0).stop 2024-05-27T08:02:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.912+0000 7fc8f5fce740 1 -- 172.21.15.63:0/613168990 shutdown_connections 2024-05-27T08:02:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.912+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/613168990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f4b4d0 0x55582330f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.912+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/613168990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582330eda0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.912+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/613168990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x55582330e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.912+0000 7fc8f5fce740 1 -- 172.21.15.63:0/613168990 >> 172.21.15.63:0/613168990 conn(0x555822f60300 msgr2=0x55582330b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.912+0000 7fc8f5fce740 1 -- 172.21.15.63:0/613168990 shutdown_connections 2024-05-27T08:02:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.912+0000 7fc8f5fce740 1 -- 172.21.15.63:0/613168990 wait complete. 2024-05-27T08:02:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.913+0000 7fc8f5fce740 1 -- start start 2024-05-27T08:02:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.913+0000 7fc8f5fce740 1 Processor -- start 2024-05-27T08:02:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.914+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f432f0 0x555823318a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.914+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x555823318f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.914+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x555823324cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.914+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555822f55790 con 0x555822f4b4d0 2024-05-27T08:02:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.914+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555823217bc0 con 0x555822f432f0 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.914+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55582331b990 con 0x555822f6c1b0 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.914+0000 7fc8f5fce740 10 client.0 _ll_register_callbacks cb 0x5558231f5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.914+0000 7fc8e7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x555823318f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.914+0000 7fc8e7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x555823318f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50598/0 (socket says 172.21.15.63:50598) 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.914+0000 7fc8e7400640 1 -- 172.21.15.63:0/3587325475 learned_addr learned my addr 172.21.15.63:0/3587325475 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.915+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f432f0 0x555823318a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.915+0000 7fc8ed000640 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x555823324cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.915+0000 7fc8e7400640 1 -- 172.21.15.63:0/3587325475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 msgr2=0x555823324cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.915+0000 7fc8e7400640 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x555823324cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.915+0000 7fc8e7400640 1 -- 172.21.15.63:0/3587325475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f432f0 msgr2=0x555823318a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.915+0000 7fc8e7400640 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f432f0 0x555823318a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.915+0000 7fc8e7400640 1 -- 172.21.15.63:0/3587325475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555822f84210 con 0x555822f4b4d0 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.915+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f432f0 0x555823318a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.915+0000 7fc8e7400640 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x555823318f60 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fc8d401dbe0 tx=0x7fc8d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ff2ea424a420eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.915+0000 7fc8d2000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.916+0000 7fc8d2000640 1 -- 172.21.15.63:0/3587325475 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4011d50 con 0x555822f4b4d0 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.916+0000 7fc8d2000640 1 -- 172.21.15.63:0/3587325475 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fc8d4004020 con 0x555822f4b4d0 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.916+0000 7fc8f5fce740 1 -- 172.21.15.63:0/3587325475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d8015070 con 0x555822f4b4d0 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.916+0000 7fc8d2000640 1 -- 172.21.15.63:0/3587325475 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc8d4030050 con 0x555822f4b4d0 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.916+0000 7fc8f5fce740 10 client.21003 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.916+0000 7fc8f5fce740 1 -- 172.21.15.63:0/3587325475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555823050620 con 0x555822f4b4d0 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.916+0000 7fc8d2000640 1 -- 172.21.15.63:0/3587325475 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8d8015070 con 0x555822f4b4d0 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.917+0000 7fc8d2000640 1 -- 172.21.15.63:0/3587325475 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc8d4019040 con 0x555822f4b4d0 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.917+0000 7fc8f5fce740 10 client.21003 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:39.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.917+0000 7fc8f5fce740 10 client.21003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.917+0000 7fc8f5fce740 20 client.21003 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.917+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555822f2ad80 0x55582332ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.917+0000 7fc8f5fce740 4 client.21003 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.917+0000 7fc8f5fce740 1 -- 172.21.15.63:0/3587325475 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555823281ae0 con 0x555822f2ad80 2024-05-27T08:02:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.918+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555822f2ad80 0x55582332ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.919+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555822f2ad80 0x55582332ad20 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782114f7274c7983 server_cookie=1fd79aa488758503 in_seq=0 out_seq=0 2024-05-27T08:02:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.919+0000 7fc8d2000640 10 client.21003 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.920+0000 7fc8d2000640 1 -- 172.21.15.63:0/3587325475 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555823281ae0 con 0x555822f2ad80 2024-05-27T08:02:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.920+0000 7fc8d2000640 10 client.21003 handle_command_reply: tid=0 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 client.21003 shutdown 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 -- 172.21.15.63:0/3587325475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 msgr2=0x555823318f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x555823318f60 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fc8d401dbe0 tx=0x7fc8d4004730 comp rx=0 tx=0).stop 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 -- 172.21.15.63:0/3587325475 shutdown_connections 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555822f2ad80 0x55582332ad20 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x555823324cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8d2000640 0 client.21003 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x555823318f60 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/3587325475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f432f0 0x555823318a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 -- 172.21.15.63:0/3587325475 >> 172.21.15.63:0/3587325475 conn(0x555822f60300 msgr2=0x55582330e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 -- 172.21.15.63:0/3587325475 shutdown_connections 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 1 -- 172.21.15.63:0/3587325475 wait complete. 2024-05-27T08:02:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.921+0000 7fc8f5fce740 20 client.21003 trim_cache size 0 max 16384 2024-05-27T08:02:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.941+0000 7fc8f5fce740 1 Processor -- start 2024-05-27T08:02:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.941+0000 7fc8f5fce740 1 -- start start 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.941+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582331b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.941+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f4b4d0 0x555823324a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.941+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x555823324fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.942+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e81b5cb0 con 0x555822f432f0 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.942+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e81b3580 con 0x555822f4b4d0 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.942+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e81b13f0 con 0x555822f6c1b0 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.942+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582331b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.942+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582331b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50608/0 (socket says 172.21.15.63:50608) 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.942+0000 7fc8e7e00640 1 -- 172.21.15.63:0/2290481065 learned_addr learned my addr 172.21.15.63:0/2290481065 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.942+0000 7fc8e7400640 1 --2- 172.21.15.63:0/2290481065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f4b4d0 0x555823324a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.942+0000 7fc8ed000640 1 --2- 172.21.15.63:0/2290481065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x555823324fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.942+0000 7fc8e7e00640 1 -- 172.21.15.63:0/2290481065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 msgr2=0x555823324fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.943+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/2290481065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x555823324fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.943+0000 7fc8e7e00640 1 -- 172.21.15.63:0/2290481065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f4b4d0 msgr2=0x555823324a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.943+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/2290481065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f4b4d0 0x555823324a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.943+0000 7fc8e7e00640 1 -- 172.21.15.63:0/2290481065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555823305050 con 0x555822f432f0 2024-05-27T08:02:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.943+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/2290481065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582331b150 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c010 tx=0x7fc8d8005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75def2c428ae7b74 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.943+0000 7fc8dcc00640 1 -- 172.21.15.63:0/2290481065 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80042d0 con 0x555822f432f0 2024-05-27T08:02:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.943+0000 7fc8dcc00640 1 -- 172.21.15.63:0/2290481065 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8d8015040 con 0x555822f432f0 2024-05-27T08:02:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.943+0000 7fc8dcc00640 1 -- 172.21.15.63:0/2290481065 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8026d60 con 0x555822f432f0 2024-05-27T08:02:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.944+0000 7fc8f5fce740 1 -- 172.21.15.63:0/2290481065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 msgr2=0x55582331b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.944+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/2290481065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582331b150 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c010 tx=0x7fc8d8005f30 comp rx=0 tx=0).stop 2024-05-27T08:02:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.945+0000 7fc8f5fce740 1 -- 172.21.15.63:0/2290481065 shutdown_connections 2024-05-27T08:02:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.945+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/2290481065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f6c1b0 0x555823324fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.945+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/2290481065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f4b4d0 0x555823324a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.945+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/2290481065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f432f0 0x55582331b150 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.945+0000 7fc8f5fce740 1 -- 172.21.15.63:0/2290481065 >> 172.21.15.63:0/2290481065 conn(0x555822f60300 msgr2=0x555823318ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.945+0000 7fc8f5fce740 1 -- 172.21.15.63:0/2290481065 shutdown_connections 2024-05-27T08:02:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.945+0000 7fc8f5fce740 1 -- 172.21.15.63:0/2290481065 wait complete. 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.945+0000 7fc8f5fce740 1 -- start start 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.945+0000 7fc8f5fce740 1 Processor -- start 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.946+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f432f0 0x55582331e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.946+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x55582331e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.946+0000 7fc8f5fce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f6c1b0 0x5558233913f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.946+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e81b1a40 con 0x555822f4b4d0 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.946+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e81b2eb0 con 0x555822f6c1b0 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.946+0000 7fc8f5fce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e81b2110 con 0x555822f432f0 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.946+0000 7fc8f5fce740 10 client.0 _ll_register_callbacks cb 0x7fc8d400f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.946+0000 7fc8e7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x55582331e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.946+0000 7fc8e7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x55582331e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50624/0 (socket says 172.21.15.63:50624) 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.946+0000 7fc8e7400640 1 -- 172.21.15.63:0/1218301651 learned_addr learned my addr 172.21.15.63:0/1218301651 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.947+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f432f0 0x55582331e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.947+0000 7fc8ed000640 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f6c1b0 0x5558233913f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.947+0000 7fc8e7400640 1 -- 172.21.15.63:0/1218301651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f432f0 msgr2=0x55582331e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.947+0000 7fc8e7400640 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f432f0 0x55582331e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.947+0000 7fc8e7400640 1 -- 172.21.15.63:0/1218301651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f6c1b0 msgr2=0x5558233913f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.947+0000 7fc8e7400640 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f6c1b0 0x5558233913f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.947+0000 7fc8e7400640 1 -- 172.21.15.63:0/1218301651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555823391930 con 0x555822f4b4d0 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.947+0000 7fc8ed000640 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f6c1b0 0x5558233913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.947+0000 7fc8e7400640 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x55582331e590 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4006010 tx=0x7fc8d4011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d0548b317e4c8e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.947+0000 7fc8d2000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.948+0000 7fc8d2000640 1 -- 172.21.15.63:0/1218301651 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4038070 con 0x555822f4b4d0 2024-05-27T08:02:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.948+0000 7fc8d2000640 1 -- 172.21.15.63:0/1218301651 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fc8d40037f0 con 0x555822f4b4d0 2024-05-27T08:02:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.948+0000 7fc8f5fce740 1 -- 172.21.15.63:0/1218301651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d8015040 con 0x555822f4b4d0 2024-05-27T08:02:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.948+0000 7fc8d2000640 1 -- 172.21.15.63:0/1218301651 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc8d4044050 con 0x555822f4b4d0 2024-05-27T08:02:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.948+0000 7fc8f5fce740 10 client.21009 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.948+0000 7fc8f5fce740 1 -- 172.21.15.63:0/1218301651 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555823392a70 con 0x555822f4b4d0 2024-05-27T08:02:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.949+0000 7fc8d2000640 1 -- 172.21.15.63:0/1218301651 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8d8015040 con 0x555822f4b4d0 2024-05-27T08:02:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.949+0000 7fc8d2000640 1 -- 172.21.15.63:0/1218301651 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc8d4019040 con 0x555822f4b4d0 2024-05-27T08:02:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.949+0000 7fc8f5fce740 10 client.21009 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.949+0000 7fc8f5fce740 10 client.21009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.949+0000 7fc8f5fce740 20 client.21009 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.949+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555822f001c0 0x555823395170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.949+0000 7fc8f5fce740 4 client.21009 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:02:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.949+0000 7fc8f5fce740 1 -- 172.21.15.63:0/1218301651 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55582327b2c0 con 0x555822f001c0 2024-05-27T08:02:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.950+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555822f001c0 0x555823395170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.951+0000 7fc8e7e00640 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555822f001c0 0x555823395170 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa6ad8a80fc46b90 server_cookie=e48bb6c20eebd6a5 in_seq=0 out_seq=0 2024-05-27T08:02:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:39.951+0000 7fc8d2000640 10 client.21009 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:39.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:39 smithi086 ceph-mon[34687]: pgmap v922: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:02:39.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:39 smithi149 ceph-mon[35408]: pgmap v922: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:02:39.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:39 smithi063 ceph-mon[31613]: pgmap v922: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:02:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.000+0000 7fc8d2000640 1 -- 172.21.15.63:0/1218301651 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55582327b2c0 con 0x555822f001c0 2024-05-27T08:02:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.000+0000 7fc8d2000640 10 client.21009 handle_command_reply: tid=0 2024-05-27T08:02:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.000+0000 7fc8f5fce740 1 client.21009 shutdown 2024-05-27T08:02:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.000+0000 7fc8f5fce740 1 -- 172.21.15.63:0/1218301651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 msgr2=0x55582331e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.000+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x55582331e590 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4006010 tx=0x7fc8d4011950 comp rx=0 tx=0).stop 2024-05-27T08:02:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.000+0000 7fc8f5fce740 1 -- 172.21.15.63:0/1218301651 shutdown_connections 2024-05-27T08:02:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.001+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555822f001c0 0x555823395170 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.001+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555822f6c1b0 0x5558233913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.001+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555822f4b4d0 0x55582331e590 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.001+0000 7fc8f5fce740 1 --2- 172.21.15.63:0/1218301651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555822f432f0 0x55582331e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.001+0000 7fc8f5fce740 1 -- 172.21.15.63:0/1218301651 >> 172.21.15.63:0/1218301651 conn(0x555822f60300 msgr2=0x555823313190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.001+0000 7fc8d2000640 0 client.21009 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.001+0000 7fc8f5fce740 1 -- 172.21.15.63:0/1218301651 shutdown_connections 2024-05-27T08:02:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.001+0000 7fc8f5fce740 1 -- 172.21.15.63:0/1218301651 wait complete. 2024-05-27T08:02:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.001+0000 7fc8f5fce740 20 client.21009 trim_cache size 0 max 16384 2024-05-27T08:02:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:02:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:02:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:02:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.002+0000 7fc8ef000640 1 -- 172.21.15.63:0/376890133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8cc0b2950 msgr2=0x7fc8cc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.002+0000 7fc8ef000640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8cc0b2950 0x7fc8cc0b4e60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55582330b9d0 tx=0x7fc8d40046c0 comp rx=0 tx=0).stop 2024-05-27T08:02:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.002+0000 7fc8ef000640 1 -- 172.21.15.63:0/376890133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e81104d0 msgr2=0x7fc8e819c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.002+0000 7fc8ef000640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e81104d0 0x7fc8e819c760 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fc8e002ae70 tx=0x7fc8e0003f10 comp rx=0 tx=0).stop 2024-05-27T08:02:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.002+0000 7fc8ef000640 1 -- 172.21.15.63:0/376890133 shutdown_connections 2024-05-27T08:02:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.002+0000 7fc8ef000640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8cc0b2950 0x7fc8cc0b4e60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.002+0000 7fc8ef000640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8e81104d0 0x7fc8e819c760 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.002+0000 7fc8ef000640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e810db70 0x7fc8e81a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.002+0000 7fc8ef000640 1 --2- 172.21.15.63:0/376890133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8e8076530 0x7fc8e81a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.002+0000 7fc8ef000640 1 -- 172.21.15.63:0/376890133 >> 172.21.15.63:0/376890133 conn(0x7fc8e80ff340 msgr2=0x7fc8e81108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.003+0000 7fc8ef000640 1 -- 172.21.15.63:0/376890133 shutdown_connections 2024-05-27T08:02:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:40.003+0000 7fc8ef000640 1 -- 172.21.15.63:0/376890133 wait complete. 2024-05-27T08:02:41.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:41 smithi086 ceph-mon[34687]: pgmap v923: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:02:41.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:41 smithi149 ceph-mon[35408]: pgmap v923: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:02:41.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:41 smithi063 ceph-mon[31613]: pgmap v923: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:02:43.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:43 smithi086 ceph-mon[34687]: pgmap v924: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:02:43.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:43 smithi149 ceph-mon[35408]: pgmap v924: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:02:43.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:43 smithi063 ceph-mon[31613]: pgmap v924: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:02:45.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:45 smithi086 ceph-mon[34687]: pgmap v925: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T08:02:45.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:45 smithi149 ceph-mon[35408]: pgmap v925: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T08:02:45.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:45 smithi063 ceph-mon[31613]: pgmap v925: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T08:02:46.349 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:02:46.864 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:47.958 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:47 smithi063 ceph-mon[31613]: pgmap v926: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T08:02:47.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:47 smithi086 ceph-mon[34687]: pgmap v926: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T08:02:47.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:47 smithi149 ceph-mon[35408]: pgmap v926: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 11 op/s 2024-05-27T08:02:49.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.394+0000 7fb147200700 1 -- 172.21.15.63:0/994599035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb148105300 msgr2=0x7fb148105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.394+0000 7fb147200700 1 --2- 172.21.15.63:0/994599035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb148105300 0x7fb148105710 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fb138009990 tx=0x7fb138009ca0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.395+0000 7fb147200700 1 -- 172.21.15.63:0/994599035 shutdown_connections 2024-05-27T08:02:49.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.395+0000 7fb147200700 1 --2- 172.21.15.63:0/994599035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14810e730 0x7fb148110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.395+0000 7fb147200700 1 --2- 172.21.15.63:0/994599035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb148105300 0x7fb148105710 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.395+0000 7fb147200700 1 --2- 172.21.15.63:0/994599035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb148105c50 0x7fb14810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.395+0000 7fb147200700 1 -- 172.21.15.63:0/994599035 >> 172.21.15.63:0/994599035 conn(0x7fb148078ab0 msgr2=0x7fb148078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:49.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.395+0000 7fb147200700 1 -- 172.21.15.63:0/994599035 shutdown_connections 2024-05-27T08:02:49.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.395+0000 7fb147200700 1 -- 172.21.15.63:0/994599035 wait complete. 2024-05-27T08:02:49.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.396+0000 7fb147200700 1 Processor -- start 2024-05-27T08:02:49.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.396+0000 7fb147200700 1 -- start start 2024-05-27T08:02:49.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.397+0000 7fb147200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb148105300 0x7fb14819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.397+0000 7fb147200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148105c50 0x7fb14819d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.397+0000 7fb147200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14810e730 0x7fb1481a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.397+0000 7fb147200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb148114140 con 0x7fb14810e730 2024-05-27T08:02:49.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.397+0000 7fb147200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb148113fc0 con 0x7fb148105300 2024-05-27T08:02:49.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.397+0000 7fb147200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb1481142c0 con 0x7fb148105c50 2024-05-27T08:02:49.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb146800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14810e730 0x7fb1481a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb146800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14810e730 0x7fb1481a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43746/0 (socket says 172.21.15.63:43746) 2024-05-27T08:02:49.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb146800700 1 -- 172.21.15.63:0/1589118374 learned_addr learned my addr 172.21.15.63:0/1589118374 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:49.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb145e00700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb148105300 0x7fb14819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb145400700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148105c50 0x7fb14819d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb146800700 1 -- 172.21.15.63:0/1589118374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148105c50 msgr2=0x7fb14819d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb146800700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148105c50 0x7fb14819d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb146800700 1 -- 172.21.15.63:0/1589118374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb148105300 msgr2=0x7fb14819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb146800700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb148105300 0x7fb14819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb146800700 1 -- 172.21.15.63:0/1589118374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1481a1c00 con 0x7fb14810e730 2024-05-27T08:02:49.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb145400700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148105c50 0x7fb14819d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:49.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.398+0000 7fb145e00700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb148105300 0x7fb14819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:49.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.399+0000 7fb146800700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14810e730 0x7fb1481a1560 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fb14000ef30 tx=0x7fb14000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.399+0000 7fb13ea00700 1 -- 172.21.15.63:0/1589118374 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1400092e0 con 0x7fb14810e730 2024-05-27T08:02:49.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.399+0000 7fb147200700 1 -- 172.21.15.63:0/1589118374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1481a1e90 con 0x7fb14810e730 2024-05-27T08:02:49.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.399+0000 7fb13ea00700 1 -- 172.21.15.63:0/1589118374 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb140004bb0 con 0x7fb14810e730 2024-05-27T08:02:49.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.399+0000 7fb13ea00700 1 -- 172.21.15.63:0/1589118374 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140004080 con 0x7fb14810e730 2024-05-27T08:02:49.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.400+0000 7fb147200700 1 -- 172.21.15.63:0/1589118374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1481a9810 con 0x7fb14810e730 2024-05-27T08:02:49.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.406+0000 7fb13ea00700 1 -- 172.21.15.63:0/1589118374 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb14001b030 con 0x7fb14810e730 2024-05-27T08:02:49.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.406+0000 7fb15ba08b80 1 Processor -- start 2024-05-27T08:02:49.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.407+0000 7fb15ba08b80 1 -- start start 2024-05-27T08:02:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.407+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ee4f9f0 0x56408eee3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.407+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408eee3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.407+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 0x56408eeef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.407+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56408ee10e40 con 0x56408ed63e40 2024-05-27T08:02:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.407+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56408ee009e0 con 0x56408eeed2c0 2024-05-27T08:02:49.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.407+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56408ee04a20 con 0x56408ee4f9f0 2024-05-27T08:02:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.408+0000 7fb145e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408eee3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.408+0000 7fb145400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ee4f9f0 0x56408eee3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.408+0000 7fb145e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408eee3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43752/0 (socket says 172.21.15.63:43752) 2024-05-27T08:02:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.408+0000 7fb145e00700 1 -- 172.21.15.63:0/1718075045 learned_addr learned my addr 172.21.15.63:0/1718075045 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:49.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.408+0000 7fb145400700 1 --2- 172.21.15.63:0/1718075045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 0x56408eeef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.409+0000 7fb13ea00700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb12c0b2a70 0x7fb12c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.409+0000 7fb145400700 1 -- 172.21.15.63:0/1718075045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 msgr2=0x56408eeef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.409+0000 7fb145400700 1 --2- 172.21.15.63:0/1718075045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 0x56408eeef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.409+0000 7fb145400700 1 -- 172.21.15.63:0/1718075045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 msgr2=0x56408eee3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.409+0000 7fb145400700 1 --2- 172.21.15.63:0/1718075045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408eee3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.409+0000 7fb145400700 1 -- 172.21.15.63:0/1718075045 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408edc9420 con 0x56408ee4f9f0 2024-05-27T08:02:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.409+0000 7fb145e00700 1 --2- 172.21.15.63:0/1718075045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408eee3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.409+0000 7fb13ea00700 1 -- 172.21.15.63:0/1589118374 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fb1400e2490 con 0x7fb14810e730 2024-05-27T08:02:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.409+0000 7fb146800700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb12c0b2a70 0x7fb12c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.409+0000 7fb145400700 1 --2- 172.21.15.63:0/1718075045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ee4f9f0 0x56408eee3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56408eee5210 tx=0x7fb134009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4f730094d2de30 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.410+0000 7fb133400700 1 -- 172.21.15.63:0/1718075045 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1340120d0 con 0x56408ee4f9f0 2024-05-27T08:02:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.410+0000 7fb146800700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb12c0b2a70 0x7fb12c0b4f80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb1400df970 tx=0x7fb140009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.410+0000 7fb133400700 1 -- 172.21.15.63:0/1718075045 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb134012270 con 0x56408ee4f9f0 2024-05-27T08:02:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.410+0000 7fb133400700 1 -- 172.21.15.63:0/1718075045 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13400f680 con 0x56408ee4f9f0 2024-05-27T08:02:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.411+0000 7fb15ba08b80 1 -- 172.21.15.63:0/1718075045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ee4f9f0 msgr2=0x56408eee3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.411+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/1718075045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ee4f9f0 0x56408eee3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56408eee5210 tx=0x7fb134009670 comp rx=0 tx=0).stop 2024-05-27T08:02:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.412+0000 7fb15ba08b80 1 -- 172.21.15.63:0/1718075045 shutdown_connections 2024-05-27T08:02:49.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.412+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/1718075045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 0x56408eeef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.412+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/1718075045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ee4f9f0 0x56408eee3750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.412+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/1718075045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408eee3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.412+0000 7fb15ba08b80 1 -- 172.21.15.63:0/1718075045 >> 172.21.15.63:0/1718075045 conn(0x56408edfd390 msgr2=0x56408eede710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.412+0000 7fb15ba08b80 1 -- 172.21.15.63:0/1718075045 shutdown_connections 2024-05-27T08:02:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.412+0000 7fb15ba08b80 1 -- 172.21.15.63:0/1718075045 wait complete. 2024-05-27T08:02:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.413+0000 7fb15ba08b80 1 -- start start 2024-05-27T08:02:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.413+0000 7fb15ba08b80 1 Processor -- start 2024-05-27T08:02:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.414+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 0x56408eeea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.414+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408eeea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.414+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408eeed2c0 0x56408eeead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.414+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56408ed06fa0 con 0x56408ee4f9f0 2024-05-27T08:02:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.414+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56408eda1760 con 0x56408ed63e40 2024-05-27T08:02:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.414+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56408eeefda0 con 0x56408eeed2c0 2024-05-27T08:02:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.414+0000 7fb15ba08b80 10 client.0 _ll_register_callbacks cb 0x56408edaac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.415+0000 7fb145e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408eeea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.415+0000 7fb145400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 0x56408eeea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.415+0000 7fb145400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 0x56408eeea290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:39126/0 (socket says 172.21.15.63:39126) 2024-05-27T08:02:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.415+0000 7fb145400700 1 -- 172.21.15.63:0/856500836 learned_addr learned my addr 172.21.15.63:0/856500836 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.416+0000 7fb145400700 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408eeed2c0 0x56408eeead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.416+0000 7fb145400700 1 -- 172.21.15.63:0/856500836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408eeed2c0 msgr2=0x56408eeead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.416+0000 7fb145400700 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408eeed2c0 0x56408eeead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.416+0000 7fb145400700 1 -- 172.21.15.63:0/856500836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 msgr2=0x56408eeea7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.416+0000 7fb145400700 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408eeea7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.416+0000 7fb145400700 1 -- 172.21.15.63:0/856500836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408ed508d0 con 0x56408ed63e40 2024-05-27T08:02:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.416+0000 7fb145e00700 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408eeea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.417+0000 7fb145400700 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 0x56408eeea290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb13400a010 tx=0x7fb134010d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0e7d0e5a305b3d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.417+0000 7fb127e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:02:49.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.418+0000 7fb127e00700 1 -- 172.21.15.63:0/856500836 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13402c060 con 0x56408ed63e40 2024-05-27T08:02:49.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.419+0000 7fb127e00700 1 -- 172.21.15.63:0/856500836 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fb134004310 con 0x56408ed63e40 2024-05-27T08:02:49.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.419+0000 7fb15ba08b80 1 -- 172.21.15.63:0/856500836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb134012270 con 0x56408ed63e40 2024-05-27T08:02:49.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.420+0000 7fb127e00700 1 -- 172.21.15.63:0/856500836 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb134004510 con 0x56408ed63e40 2024-05-27T08:02:49.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.420+0000 7fb15ba08b80 10 client.40139 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:49.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.420+0000 7fb15ba08b80 1 -- 172.21.15.63:0/856500836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56408edd1060 con 0x56408ed63e40 2024-05-27T08:02:49.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.420+0000 7fb127e00700 1 -- 172.21.15.63:0/856500836 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb134012270 con 0x56408ed63e40 2024-05-27T08:02:49.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.421+0000 7fb127e00700 1 -- 172.21.15.63:0/856500836 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb134020740 con 0x56408ed63e40 2024-05-27T08:02:49.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.421+0000 7fb15ba08b80 10 client.40139 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:49.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.421+0000 7fb15ba08b80 10 client.40139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:49.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.421+0000 7fb15ba08b80 20 client.40139 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:49.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.421+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408ef0dc10 0x56408ef10030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.421+0000 7fb15ba08b80 4 client.40139 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:49.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.421+0000 7fb15ba08b80 1 -- 172.21.15.63:0/856500836 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56408ee12d90 con 0x56408ef0dc10 2024-05-27T08:02:49.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.422+0000 7fb145e00700 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408ef0dc10 0x56408ef10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.423+0000 7fb145e00700 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408ef0dc10 0x56408ef10030 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329db6d55627967 server_cookie=fe8936894a95f779 in_seq=0 out_seq=0 2024-05-27T08:02:49.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.423+0000 7fb127e00700 10 client.40139 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:49.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.425+0000 7fb127e00700 1 -- 172.21.15.63:0/856500836 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56408ee12d90 con 0x56408ef0dc10 2024-05-27T08:02:49.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.425+0000 7fb127e00700 10 client.40139 handle_command_reply: tid=0 2024-05-27T08:02:49.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.425+0000 7fb15ba08b80 1 client.40139 shutdown 2024-05-27T08:02:49.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.425+0000 7fb15ba08b80 1 -- 172.21.15.63:0/856500836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 msgr2=0x56408eeea290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.425+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 0x56408eeea290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb13400a010 tx=0x7fb134010d90 comp rx=0 tx=0).stop 2024-05-27T08:02:49.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.426+0000 7fb15ba08b80 1 -- 172.21.15.63:0/856500836 shutdown_connections 2024-05-27T08:02:49.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.426+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408ef0dc10 0x56408ef10030 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.426+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408eeed2c0 0x56408eeead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.426+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 0x56408eeea290 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.426+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/856500836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408eeea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.426+0000 7fb15ba08b80 1 -- 172.21.15.63:0/856500836 >> 172.21.15.63:0/856500836 conn(0x56408edfd390 msgr2=0x56408eede300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:49.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.426+0000 7fb127e00700 0 client.40139 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:49.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.426+0000 7fb15ba08b80 1 -- 172.21.15.63:0/856500836 shutdown_connections 2024-05-27T08:02:49.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.426+0000 7fb15ba08b80 1 -- 172.21.15.63:0/856500836 wait complete. 2024-05-27T08:02:49.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.427+0000 7fb15ba08b80 20 client.40139 trim_cache size 0 max 16384 2024-05-27T08:02:49.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.454+0000 7fb15ba08b80 1 Processor -- start 2024-05-27T08:02:49.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.454+0000 7fb15ba08b80 1 -- start start 2024-05-27T08:02:49.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.454+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 0x56408eede740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.454+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408ef0a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.454+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408eeed2c0 0x56408eee2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.454+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1481ac710 con 0x56408ee4f9f0 2024-05-27T08:02:49.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.454+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1481a9fa0 con 0x56408ed63e40 2024-05-27T08:02:49.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.454+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb148073e10 con 0x56408eeed2c0 2024-05-27T08:02:49.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.455+0000 7fb145e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408ef0a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.455+0000 7fb145e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408ef0a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43770/0 (socket says 172.21.15.63:43770) 2024-05-27T08:02:49.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.455+0000 7fb145e00700 1 -- 172.21.15.63:0/1267105753 learned_addr learned my addr 172.21.15.63:0/1267105753 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.455+0000 7fb145400700 1 --2- 172.21.15.63:0/1267105753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 0x56408eede740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.455+0000 7fb145400700 1 --2- 172.21.15.63:0/1267105753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408eeed2c0 0x56408eee2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.455+0000 7fb145e00700 1 -- 172.21.15.63:0/1267105753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408eeed2c0 msgr2=0x56408eee2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.455+0000 7fb145e00700 1 --2- 172.21.15.63:0/1267105753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408eeed2c0 0x56408eee2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.455+0000 7fb145e00700 1 -- 172.21.15.63:0/1267105753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 msgr2=0x56408eede740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.455+0000 7fb145e00700 1 --2- 172.21.15.63:0/1267105753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 0x56408eede740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.456+0000 7fb145e00700 1 -- 172.21.15.63:0/1267105753 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408eee4270 con 0x56408ee4f9f0 2024-05-27T08:02:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.456+0000 7fb145e00700 1 --2- 172.21.15.63:0/1267105753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408ef0a130 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fb13800b2e0 tx=0x7fb13801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569fada8c0070d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.456+0000 7fb132a00700 1 -- 172.21.15.63:0/1267105753 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138004a30 con 0x56408ee4f9f0 2024-05-27T08:02:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.456+0000 7fb132a00700 1 -- 172.21.15.63:0/1267105753 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1380054b0 con 0x56408ee4f9f0 2024-05-27T08:02:49.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.457+0000 7fb132a00700 1 -- 172.21.15.63:0/1267105753 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13800f750 con 0x56408ee4f9f0 2024-05-27T08:02:49.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.457+0000 7fb15ba08b80 1 -- 172.21.15.63:0/1267105753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 msgr2=0x56408ef0a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.457+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/1267105753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408ef0a130 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fb13800b2e0 tx=0x7fb13801a040 comp rx=0 tx=0).stop 2024-05-27T08:02:49.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.458+0000 7fb15ba08b80 1 -- 172.21.15.63:0/1267105753 shutdown_connections 2024-05-27T08:02:49.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.458+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/1267105753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408eeed2c0 0x56408eee2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.458+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/1267105753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408ed63e40 0x56408eede740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.458+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/1267105753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ee4f9f0 0x56408ef0a130 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.458+0000 7fb15ba08b80 1 -- 172.21.15.63:0/1267105753 >> 172.21.15.63:0/1267105753 conn(0x56408edfd390 msgr2=0x56408eee8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:49.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.458+0000 7fb15ba08b80 1 -- 172.21.15.63:0/1267105753 shutdown_connections 2024-05-27T08:02:49.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.458+0000 7fb15ba08b80 1 -- 172.21.15.63:0/1267105753 wait complete. 2024-05-27T08:02:49.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.458+0000 7fb15ba08b80 1 -- start start 2024-05-27T08:02:49.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.459+0000 7fb15ba08b80 1 Processor -- start 2024-05-27T08:02:49.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.459+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408ef132f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.459+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 0x56408ef0f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.459+0000 7fb15ba08b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ef0fba0 0x56408ef0ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.459+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb148074460 con 0x56408ed63e40 2024-05-27T08:02:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.459+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1481a99e0 con 0x56408eeed2c0 2024-05-27T08:02:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.459+0000 7fb15ba08b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb148074b30 con 0x56408ef0fba0 2024-05-27T08:02:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.459+0000 7fb15ba08b80 10 client.0 _ll_register_callbacks cb 0x56408edaac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.460+0000 7fb145400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408ef132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.460+0000 7fb145400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408ef132f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43778/0 (socket says 172.21.15.63:43778) 2024-05-27T08:02:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.460+0000 7fb145400700 1 -- 172.21.15.63:0/4188231545 learned_addr learned my addr 172.21.15.63:0/4188231545 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.460+0000 7fb145e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 0x56408ef0f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.460+0000 7fb145400700 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ef0fba0 0x56408ef0ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.461+0000 7fb145e00700 1 -- 172.21.15.63:0/4188231545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ef0fba0 msgr2=0x56408ef0ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.461+0000 7fb145e00700 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ef0fba0 0x56408ef0ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.461+0000 7fb145e00700 1 -- 172.21.15.63:0/4188231545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 msgr2=0x56408ef132f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.461+0000 7fb145e00700 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408ef132f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.461+0000 7fb145e00700 1 -- 172.21.15.63:0/4188231545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408ef764f0 con 0x56408eeed2c0 2024-05-27T08:02:49.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.461+0000 7fb145400700 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408ef132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:49.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.461+0000 7fb145e00700 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 0x56408ef0f660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x56408ef0ac20 tx=0x7fb138009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564e9bc62c70fa02 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.461+0000 7fb127e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:02:49.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.462+0000 7fb127e00700 1 -- 172.21.15.63:0/4188231545 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138004830 con 0x56408eeed2c0 2024-05-27T08:02:49.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.462+0000 7fb127e00700 1 -- 172.21.15.63:0/4188231545 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fb1380049d0 con 0x56408eeed2c0 2024-05-27T08:02:49.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.462+0000 7fb15ba08b80 1 -- 172.21.15.63:0/4188231545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1380054b0 con 0x56408eeed2c0 2024-05-27T08:02:49.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.463+0000 7fb127e00700 1 -- 172.21.15.63:0/4188231545 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb138044050 con 0x56408eeed2c0 2024-05-27T08:02:49.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.463+0000 7fb15ba08b80 10 client.40145 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:49.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.463+0000 7fb15ba08b80 1 -- 172.21.15.63:0/4188231545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56408ef77740 con 0x56408eeed2c0 2024-05-27T08:02:49.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.463+0000 7fb127e00700 1 -- 172.21.15.63:0/4188231545 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1380054b0 con 0x56408eeed2c0 2024-05-27T08:02:49.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.464+0000 7fb127e00700 1 -- 172.21.15.63:0/4188231545 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb13800fc60 con 0x56408eeed2c0 2024-05-27T08:02:49.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.464+0000 7fb15ba08b80 10 client.40145 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:49.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.464+0000 7fb15ba08b80 10 client.40145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:49.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.464+0000 7fb15ba08b80 20 client.40145 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:49.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.464+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408ef77e60 0x56408ef7a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.464+0000 7fb15ba08b80 4 client.40145 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:02:49.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.464+0000 7fb15ba08b80 1 -- 172.21.15.63:0/4188231545 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56408eedd370 con 0x56408ef77e60 2024-05-27T08:02:49.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.465+0000 7fb145400700 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408ef77e60 0x56408ef7a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb127e00700 10 client.40145 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:49.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb145400700 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408ef77e60 0x56408ef7a280 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2985148699f97044 server_cookie=93847c02efc3c4ba in_seq=0 out_seq=0 2024-05-27T08:02:49.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb127e00700 1 -- 172.21.15.63:0/4188231545 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56408eedd370 con 0x56408ef77e60 2024-05-27T08:02:49.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb127e00700 10 client.40145 handle_command_reply: tid=0 2024-05-27T08:02:49.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb15ba08b80 1 client.40145 shutdown 2024-05-27T08:02:49.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb15ba08b80 1 -- 172.21.15.63:0/4188231545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 msgr2=0x56408ef0f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 0x56408ef0f660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x56408ef0ac20 tx=0x7fb138009670 comp rx=0 tx=0).stop 2024-05-27T08:02:49.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb15ba08b80 1 -- 172.21.15.63:0/4188231545 shutdown_connections 2024-05-27T08:02:49.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56408ef77e60 0x56408ef7a280 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56408ef0fba0 0x56408ef0ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56408ed63e40 0x56408ef132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb15ba08b80 1 --2- 172.21.15.63:0/4188231545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56408eeed2c0 0x56408ef0f660 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb15ba08b80 1 -- 172.21.15.63:0/4188231545 >> 172.21.15.63:0/4188231545 conn(0x56408edfd390 msgr2=0x56408eee28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:49.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.466+0000 7fb127e00700 0 client.40145 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:49.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.467+0000 7fb15ba08b80 1 -- 172.21.15.63:0/4188231545 shutdown_connections 2024-05-27T08:02:49.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.467+0000 7fb15ba08b80 1 -- 172.21.15.63:0/4188231545 wait complete. 2024-05-27T08:02:49.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.467+0000 7fb15ba08b80 20 client.40145 trim_cache size 0 max 16384 2024-05-27T08:02:49.469 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:02:49.469 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:02:49.470 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:02:49.470 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:02:49.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 -- 172.21.15.63:0/1589118374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb12c0b2a70 msgr2=0x7fb12c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb12c0b2a70 0x7fb12c0b4f80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb1400df970 tx=0x7fb140009040 comp rx=0 tx=0).stop 2024-05-27T08:02:49.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 -- 172.21.15.63:0/1589118374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14810e730 msgr2=0x7fb1481a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14810e730 0x7fb1481a1560 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fb14000ef30 tx=0x7fb14000c450 comp rx=0 tx=0).stop 2024-05-27T08:02:49.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 -- 172.21.15.63:0/1589118374 shutdown_connections 2024-05-27T08:02:49.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb12c0b2a70 0x7fb12c0b4f80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14810e730 0x7fb1481a1560 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb148105300 0x7fb14819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 --2- 172.21.15.63:0/1589118374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb148105c50 0x7fb14819d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 -- 172.21.15.63:0/1589118374 >> 172.21.15.63:0/1589118374 conn(0x7fb148078ab0 msgr2=0x7fb148104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:49.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 -- 172.21.15.63:0/1589118374 shutdown_connections 2024-05-27T08:02:49.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:49.468+0000 7fb147200700 1 -- 172.21.15.63:0/1589118374 wait complete. 2024-05-27T08:02:49.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:49 smithi063 ceph-mon[31613]: pgmap v927: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:02:49.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:49 smithi086 ceph-mon[34687]: pgmap v927: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:02:49.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:49 smithi149 ceph-mon[35408]: pgmap v927: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:02:50.406 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:02:50.407 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:02:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:51.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:51 smithi086 ceph-mon[34687]: pgmap v928: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:02:51.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:51 smithi149 ceph-mon[35408]: pgmap v928: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:02:51.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:51 smithi063 ceph-mon[31613]: pgmap v928: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:02:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.263+0000 7fda3d600640 1 -- 172.21.15.63:0/500787093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda38104bb0 msgr2=0x7fda38104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.263+0000 7fda3d600640 1 --2- 172.21.15.63:0/500787093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda38104bb0 0x7fda38104fc0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7fda28009900 tx=0x7fda2802a410 comp rx=0 tx=0).stop 2024-05-27T08:02:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.263+0000 7fda3d600640 1 -- 172.21.15.63:0/500787093 shutdown_connections 2024-05-27T08:02:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.263+0000 7fda3d600640 1 --2- 172.21.15.63:0/500787093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda3810e000 0x7fda381104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.263+0000 7fda3d600640 1 --2- 172.21.15.63:0/500787093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3810b610 0x7fda3810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.263+0000 7fda3d600640 1 --2- 172.21.15.63:0/500787093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda38104bb0 0x7fda38104fc0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.263+0000 7fda3d600640 1 -- 172.21.15.63:0/500787093 >> 172.21.15.63:0/500787093 conn(0x7fda380fe210 msgr2=0x7fda38100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.263+0000 7fda3d600640 1 -- 172.21.15.63:0/500787093 shutdown_connections 2024-05-27T08:02:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.263+0000 7fda3d600640 1 -- 172.21.15.63:0/500787093 wait complete. 2024-05-27T08:02:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.264+0000 7fda3d600640 1 Processor -- start 2024-05-27T08:02:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.264+0000 7fda3d600640 1 -- start start 2024-05-27T08:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.265+0000 7fda3d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda38104bb0 0x7fda3819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.265+0000 7fda3d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda3810b610 0x7fda3819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.265+0000 7fda3d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3810e000 0x7fda381a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.265+0000 7fda3d600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda38113b10 con 0x7fda3810b610 2024-05-27T08:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.265+0000 7fda3d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda38113990 con 0x7fda3810e000 2024-05-27T08:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.265+0000 7fda3d600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fda38113c90 con 0x7fda38104bb0 2024-05-27T08:02:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.265+0000 7fda37400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda3810b610 0x7fda3819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.265+0000 7fda37400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda3810b610 0x7fda3819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43804/0 (socket says 172.21.15.63:43804) 2024-05-27T08:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.265+0000 7fda37400640 1 -- 172.21.15.63:0/2579827619 learned_addr learned my addr 172.21.15.63:0/2579827619 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.266+0000 7fda37e00640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda38104bb0 0x7fda3819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.266+0000 7fda3cc00640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3810e000 0x7fda381a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.266+0000 7fda37400640 1 -- 172.21.15.63:0/2579827619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda38104bb0 msgr2=0x7fda3819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.266+0000 7fda37400640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda38104bb0 0x7fda3819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.266+0000 7fda37400640 1 -- 172.21.15.63:0/2579827619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3810e000 msgr2=0x7fda381a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.266+0000 7fda37400640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3810e000 0x7fda381a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.266+0000 7fda37400640 1 -- 172.21.15.63:0/2579827619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda381a1610 con 0x7fda3810b610 2024-05-27T08:02:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.266+0000 7fda37400640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda3810b610 0x7fda3819cd10 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e9f0 tx=0x7fda2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.267+0000 7fda34c00640 1 -- 172.21.15.63:0/2579827619 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2400cd80 con 0x7fda3810b610 2024-05-27T08:02:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.267+0000 7fda3d600640 1 -- 172.21.15.63:0/2579827619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda381a1900 con 0x7fda3810b610 2024-05-27T08:02:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.267+0000 7fda34c00640 1 -- 172.21.15.63:0/2579827619 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda240108c0 con 0x7fda3810b610 2024-05-27T08:02:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.267+0000 7fda34c00640 1 -- 172.21.15.63:0/2579827619 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240187c0 con 0x7fda3810b610 2024-05-27T08:02:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.267+0000 7fda3d600640 1 -- 172.21.15.63:0/2579827619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda381a9280 con 0x7fda3810b610 2024-05-27T08:02:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.271+0000 7fda45d6a740 1 Processor -- start 2024-05-27T08:02:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.271+0000 7fda45d6a740 1 -- start start 2024-05-27T08:02:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.271+0000 7fda34c00640 1 -- 172.21.15.63:0/2579827619 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fda24010430 con 0x7fda3810b610 2024-05-27T08:02:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.272+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92dc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.272+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dc9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.272+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a064d0 0x55ea92dca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.272+0000 7fda45d6a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ea92d3ab90 con 0x55ea929fe2f0 2024-05-27T08:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.272+0000 7fda45d6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea92d2a730 con 0x55ea92a271b0 2024-05-27T08:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.272+0000 7fda45d6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea92d2e770 con 0x55ea92a064d0 2024-05-27T08:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.272+0000 7fda37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dc9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.272+0000 7fda37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dc9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43806/0 (socket says 172.21.15.63:43806) 2024-05-27T08:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.272+0000 7fda37e00640 1 -- 172.21.15.63:0/3451446014 learned_addr learned my addr 172.21.15.63:0/3451446014 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.272+0000 7fda3cc00640 1 --2- 172.21.15.63:0/3451446014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92dc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.273+0000 7fda37400640 1 --2- 172.21.15.63:0/3451446014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a064d0 0x55ea92dca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.273+0000 7fda3cc00640 1 -- 172.21.15.63:0/3451446014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a064d0 msgr2=0x55ea92dca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.273+0000 7fda3cc00640 1 --2- 172.21.15.63:0/3451446014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a064d0 0x55ea92dca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.273+0000 7fda3cc00640 1 -- 172.21.15.63:0/3451446014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 msgr2=0x55ea92dc9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.273+0000 7fda3cc00640 1 --2- 172.21.15.63:0/3451446014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dc9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.273+0000 7fda3cc00640 1 -- 172.21.15.63:0/3451446014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea92cc38b0 con 0x55ea92a271b0 2024-05-27T08:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.273+0000 7fda37e00640 1 --2- 172.21.15.63:0/3451446014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dc9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.274+0000 7fda3cc00640 1 --2- 172.21.15.63:0/3451446014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92dc9a60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fda30009da0 tx=0x7fda30010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152ad31f086a3b53 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.274+0000 7fda34c00640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fda1c0b29d0 0x7fda1c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.274+0000 7fda34c00640 1 -- 172.21.15.63:0/2579827619 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fda240e2f60 con 0x7fda3810b610 2024-05-27T08:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.274+0000 7fda37e00640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fda1c0b29d0 0x7fda1c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.274+0000 7fda2d200640 1 -- 172.21.15.63:0/3451446014 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30016070 con 0x55ea92a271b0 2024-05-27T08:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.274+0000 7fda2d200640 1 -- 172.21.15.63:0/3451446014 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda30010e50 con 0x55ea92a271b0 2024-05-27T08:02:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.274+0000 7fda2d200640 1 -- 172.21.15.63:0/3451446014 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda300126b0 con 0x55ea92a271b0 2024-05-27T08:02:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.275+0000 7fda45d6a740 1 -- 172.21.15.63:0/3451446014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 msgr2=0x55ea92dc9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.275+0000 7fda45d6a740 1 --2- 172.21.15.63:0/3451446014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92dc9a60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fda30009da0 tx=0x7fda30010740 comp rx=0 tx=0).stop 2024-05-27T08:02:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.275+0000 7fda37e00640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fda1c0b29d0 0x7fda1c0b4ee0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55ea92dcae10 tx=0x7fda28032ba0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.275+0000 7fda45d6a740 1 -- 172.21.15.63:0/3451446014 shutdown_connections 2024-05-27T08:02:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.275+0000 7fda45d6a740 1 --2- 172.21.15.63:0/3451446014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a064d0 0x55ea92dca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.275+0000 7fda45d6a740 1 --2- 172.21.15.63:0/3451446014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dc9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.275+0000 7fda45d6a740 1 --2- 172.21.15.63:0/3451446014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92dc9a60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.276+0000 7fda45d6a740 1 -- 172.21.15.63:0/3451446014 >> 172.21.15.63:0/3451446014 conn(0x55ea92a1b300 msgr2=0x55ea92dc6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.276+0000 7fda45d6a740 1 -- 172.21.15.63:0/3451446014 shutdown_connections 2024-05-27T08:02:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.276+0000 7fda45d6a740 1 -- 172.21.15.63:0/3451446014 wait complete. 2024-05-27T08:02:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.276+0000 7fda45d6a740 1 -- start start 2024-05-27T08:02:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.277+0000 7fda45d6a740 1 Processor -- start 2024-05-27T08:02:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.277+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea929fe2f0 0x55ea92dd3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.277+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea92a064d0 0x55ea92dd41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.278+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92ddfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.278+0000 7fda45d6a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ea92a10790 con 0x55ea92a064d0 2024-05-27T08:02:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.278+0000 7fda45d6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea92cd2bc0 con 0x55ea92a271b0 2024-05-27T08:02:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.278+0000 7fda45d6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea92dd6b90 con 0x55ea929fe2f0 2024-05-27T08:02:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.278+0000 7fda45d6a740 10 client.0 _ll_register_callbacks cb 0x55ea92cb0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.278+0000 7fda37400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea92a064d0 0x55ea92dd41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.278+0000 7fda37400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea92a064d0 0x55ea92dd41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43818/0 (socket says 172.21.15.63:43818) 2024-05-27T08:02:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.278+0000 7fda37400640 1 -- 172.21.15.63:0/767828504 learned_addr learned my addr 172.21.15.63:0/767828504 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.278+0000 7fda37e00640 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea929fe2f0 0x55ea92dd3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.278+0000 7fda3cc00640 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92ddfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda37400640 1 -- 172.21.15.63:0/767828504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea929fe2f0 msgr2=0x55ea92dd3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda37400640 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea929fe2f0 0x55ea92dd3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda37400640 1 -- 172.21.15.63:0/767828504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 msgr2=0x55ea92ddfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda37400640 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92ddfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda37400640 1 -- 172.21.15.63:0/767828504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea92a3f210 con 0x55ea92a064d0 2024-05-27T08:02:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda37e00640 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea929fe2f0 0x55ea92dd3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda3cc00640 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92ddfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda37400640 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea92a064d0 0x55ea92dd41b0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fda240a9db0 tx=0x7fda240a9c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d58fd3f657faf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda21600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda21600640 1 -- 172.21.15.63:0/767828504 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240a9930 con 0x55ea92a064d0 2024-05-27T08:02:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda21600640 1 -- 172.21.15.63:0/767828504 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fda240a60b0 con 0x55ea92a064d0 2024-05-27T08:02:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.279+0000 7fda45d6a740 1 -- 172.21.15.63:0/767828504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda30010e50 con 0x55ea92a064d0 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.280+0000 7fda21600640 1 -- 172.21.15.63:0/767828504 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fda240aa520 con 0x55ea92a064d0 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.280+0000 7fda45d6a740 10 client.21039 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.280+0000 7fda45d6a740 1 -- 172.21.15.63:0/767828504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ea92b0b620 con 0x55ea92a064d0 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.280+0000 7fda21600640 1 -- 172.21.15.63:0/767828504 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda30010e50 con 0x55ea92a064d0 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.280+0000 7fda21600640 1 -- 172.21.15.63:0/767828504 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fda240a9080 con 0x55ea92a064d0 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.280+0000 7fda45d6a740 10 client.21039 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.280+0000 7fda45d6a740 10 client.21039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.280+0000 7fda45d6a740 20 client.21039 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.280+0000 7fda45d6a740 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea929e5d80 0x55ea92de5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.280+0000 7fda45d6a740 4 client.21039 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.281+0000 7fda45d6a740 1 -- 172.21.15.63:0/767828504 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea92d3cae0 con 0x55ea929e5d80 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.281+0000 7fda37e00640 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea929e5d80 0x55ea92de5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.282+0000 7fda37e00640 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea929e5d80 0x55ea92de5ef0 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e950cd87cc3ff7 server_cookie=16be0b24d2a380ad in_seq=0 out_seq=0 2024-05-27T08:02:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.282+0000 7fda21600640 10 client.21039 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.284+0000 7fda21600640 1 -- 172.21.15.63:0/767828504 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ea92d3cae0 con 0x55ea929e5d80 2024-05-27T08:02:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.284+0000 7fda21600640 10 client.21039 handle_command_reply: tid=0 2024-05-27T08:02:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.284+0000 7fda45d6a740 1 client.21039 shutdown 2024-05-27T08:02:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 1 -- 172.21.15.63:0/767828504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea92a064d0 msgr2=0x55ea92dd41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea92a064d0 0x55ea92dd41b0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fda240a9db0 tx=0x7fda240a9c70 comp rx=0 tx=0).stop 2024-05-27T08:02:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 1 -- 172.21.15.63:0/767828504 shutdown_connections 2024-05-27T08:02:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea929e5d80 0x55ea92de5ef0 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92ddfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda21600640 0 client.21039 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea92a064d0 0x55ea92dd41b0 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 1 --2- 172.21.15.63:0/767828504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea929fe2f0 0x55ea92dd3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 1 -- 172.21.15.63:0/767828504 >> 172.21.15.63:0/767828504 conn(0x55ea92a1b300 msgr2=0x55ea92dc9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 1 -- 172.21.15.63:0/767828504 shutdown_connections 2024-05-27T08:02:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 1 -- 172.21.15.63:0/767828504 wait complete. 2024-05-27T08:02:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.285+0000 7fda45d6a740 20 client.21039 trim_cache size 0 max 16384 2024-05-27T08:02:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.309+0000 7fda45d6a740 1 Processor -- start 2024-05-27T08:02:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.310+0000 7fda45d6a740 1 -- start start 2024-05-27T08:02:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.310+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dcf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.310+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a064d0 0x55ea92dd3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.310+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a271b0 0x55ea92dd4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.310+0000 7fda45d6a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda381b5bc0 con 0x55ea929fe2f0 2024-05-27T08:02:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.310+0000 7fda45d6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda381b3490 con 0x55ea92a064d0 2024-05-27T08:02:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.310+0000 7fda45d6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fda381b1300 con 0x55ea92a271b0 2024-05-27T08:02:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dcf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dcf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43828/0 (socket says 172.21.15.63:43828) 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda37e00640 1 -- 172.21.15.63:0/2616843132 learned_addr learned my addr 172.21.15.63:0/2616843132 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda37400640 1 --2- 172.21.15.63:0/2616843132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a064d0 0x55ea92dd3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda3cc00640 1 --2- 172.21.15.63:0/2616843132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a271b0 0x55ea92dd4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda37e00640 1 -- 172.21.15.63:0/2616843132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a271b0 msgr2=0x55ea92dd4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda37e00640 1 --2- 172.21.15.63:0/2616843132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a271b0 0x55ea92dd4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda37e00640 1 -- 172.21.15.63:0/2616843132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a064d0 msgr2=0x55ea92dd3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda37e00640 1 --2- 172.21.15.63:0/2616843132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a064d0 0x55ea92dd3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda37e00640 1 -- 172.21.15.63:0/2616843132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea92dd21c0 con 0x55ea929fe2f0 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.311+0000 7fda37400640 1 --2- 172.21.15.63:0/2616843132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a064d0 0x55ea92dd3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.312+0000 7fda37e00640 1 --2- 172.21.15.63:0/2616843132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dcf3e0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fda28006010 tx=0x7fda2803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b1581847596c6a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.312+0000 7fda23e00640 1 -- 172.21.15.63:0/2616843132 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28035040 con 0x55ea929fe2f0 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.312+0000 7fda23e00640 1 -- 172.21.15.63:0/2616843132 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda28005050 con 0x55ea929fe2f0 2024-05-27T08:02:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.312+0000 7fda23e00640 1 -- 172.21.15.63:0/2616843132 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28004b20 con 0x55ea929fe2f0 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.313+0000 7fda45d6a740 1 -- 172.21.15.63:0/2616843132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 msgr2=0x55ea92dcf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.313+0000 7fda45d6a740 1 --2- 172.21.15.63:0/2616843132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dcf3e0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fda28006010 tx=0x7fda2803e000 comp rx=0 tx=0).stop 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.313+0000 7fda45d6a740 1 -- 172.21.15.63:0/2616843132 shutdown_connections 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.313+0000 7fda45d6a740 1 --2- 172.21.15.63:0/2616843132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a271b0 0x55ea92dd4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.313+0000 7fda45d6a740 1 --2- 172.21.15.63:0/2616843132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a064d0 0x55ea92dd3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.313+0000 7fda45d6a740 1 --2- 172.21.15.63:0/2616843132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92dcf3e0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.313+0000 7fda45d6a740 1 -- 172.21.15.63:0/2616843132 >> 172.21.15.63:0/2616843132 conn(0x55ea92a1b300 msgr2=0x55ea92dd3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.313+0000 7fda45d6a740 1 -- 172.21.15.63:0/2616843132 shutdown_connections 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.313+0000 7fda45d6a740 1 -- 172.21.15.63:0/2616843132 wait complete. 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.314+0000 7fda45d6a740 1 -- start start 2024-05-27T08:02:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.314+0000 7fda45d6a740 1 Processor -- start 2024-05-27T08:02:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.315+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92ddc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.315+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a064d0 0x55ea92dd75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.315+0000 7fda45d6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92dd7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.315+0000 7fda45d6a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fda381b1950 con 0x55ea929fe2f0 2024-05-27T08:02:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.315+0000 7fda45d6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda381b2dc0 con 0x55ea92a271b0 2024-05-27T08:02:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.315+0000 7fda45d6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fda381b2020 con 0x55ea92a064d0 2024-05-27T08:02:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.315+0000 7fda45d6a740 10 client.0 _ll_register_callbacks cb 0x7fda2400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.315+0000 7fda37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92ddc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.315+0000 7fda37e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92ddc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43830/0 (socket says 172.21.15.63:43830) 2024-05-27T08:02:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.316+0000 7fda37e00640 1 -- 172.21.15.63:0/3386713407 learned_addr learned my addr 172.21.15.63:0/3386713407 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.316+0000 7fda37400640 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a064d0 0x55ea92dd75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.316+0000 7fda3cc00640 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92dd7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.316+0000 7fda37e00640 1 -- 172.21.15.63:0/3386713407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a064d0 msgr2=0x55ea92dd75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.316+0000 7fda37e00640 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a064d0 0x55ea92dd75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.316+0000 7fda37e00640 1 -- 172.21.15.63:0/3386713407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 msgr2=0x55ea92dd7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.316+0000 7fda37e00640 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92dd7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.316+0000 7fda37e00640 1 -- 172.21.15.63:0/3386713407 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea92dd8040 con 0x55ea929fe2f0 2024-05-27T08:02:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.316+0000 7fda3cc00640 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92dd7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.317+0000 7fda37e00640 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92ddc240 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fda280057c0 tx=0x7fda28005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c298f85687c59d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.317+0000 7fda21600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.317+0000 7fda21600640 1 -- 172.21.15.63:0/3386713407 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2805a020 con 0x55ea929fe2f0 2024-05-27T08:02:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.317+0000 7fda21600640 1 -- 172.21.15.63:0/3386713407 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fda280043d0 con 0x55ea929fe2f0 2024-05-27T08:02:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.317+0000 7fda45d6a740 1 -- 172.21.15.63:0/3386713407 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda28005050 con 0x55ea929fe2f0 2024-05-27T08:02:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.317+0000 7fda21600640 1 -- 172.21.15.63:0/3386713407 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fda28065050 con 0x55ea929fe2f0 2024-05-27T08:02:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.317+0000 7fda45d6a740 10 client.21045 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.317+0000 7fda45d6a740 1 -- 172.21.15.63:0/3386713407 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ea92e4e200 con 0x55ea929fe2f0 2024-05-27T08:02:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.318+0000 7fda21600640 1 -- 172.21.15.63:0/3386713407 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda28005050 con 0x55ea929fe2f0 2024-05-27T08:02:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.318+0000 7fda21600640 1 -- 172.21.15.63:0/3386713407 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fda280037f0 con 0x55ea929fe2f0 2024-05-27T08:02:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.318+0000 7fda45d6a740 10 client.21045 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.318+0000 7fda45d6a740 10 client.21045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.318+0000 7fda45d6a740 20 client.21045 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.318+0000 7fda45d6a740 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea929bb1c0 0x55ea92e50a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.318+0000 7fda45d6a740 4 client.21045 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:02:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.318+0000 7fda45d6a740 1 -- 172.21.15.63:0/3386713407 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea92d362c0 con 0x55ea929bb1c0 2024-05-27T08:02:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.319+0000 7fda37400640 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea929bb1c0 0x55ea92e50a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.320+0000 7fda37400640 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea929bb1c0 0x55ea92e50a40 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c0eb600c3638e22 server_cookie=8f924f8adf017dfe in_seq=0 out_seq=0 2024-05-27T08:02:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.320+0000 7fda21600640 10 client.21045 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.367+0000 7fda21600640 1 -- 172.21.15.63:0/3386713407 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ea92d362c0 con 0x55ea929bb1c0 2024-05-27T08:02:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.367+0000 7fda21600640 10 client.21045 handle_command_reply: tid=0 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.367+0000 7fda45d6a740 1 client.21045 shutdown 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.367+0000 7fda45d6a740 1 -- 172.21.15.63:0/3386713407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 msgr2=0x55ea92ddc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.367+0000 7fda45d6a740 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92ddc240 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fda280057c0 tx=0x7fda28005570 comp rx=0 tx=0).stop 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.368+0000 7fda45d6a740 1 -- 172.21.15.63:0/3386713407 shutdown_connections 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.368+0000 7fda45d6a740 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea929bb1c0 0x55ea92e50a40 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.368+0000 7fda45d6a740 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea92a271b0 0x55ea92dd7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.368+0000 7fda45d6a740 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea92a064d0 0x55ea92dd75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.368+0000 7fda45d6a740 1 --2- 172.21.15.63:0/3386713407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea929fe2f0 0x55ea92ddc240 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.368+0000 7fda45d6a740 1 -- 172.21.15.63:0/3386713407 >> 172.21.15.63:0/3386713407 conn(0x55ea92a1b300 msgr2=0x55ea92dd34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.368+0000 7fda21600640 0 client.21045 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.368+0000 7fda45d6a740 1 -- 172.21.15.63:0/3386713407 shutdown_connections 2024-05-27T08:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.368+0000 7fda45d6a740 1 -- 172.21.15.63:0/3386713407 wait complete. 2024-05-27T08:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.368+0000 7fda45d6a740 20 client.21045 trim_cache size 0 max 16384 2024-05-27T08:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:02:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 -- 172.21.15.63:0/2579827619 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fda1c0b29d0 msgr2=0x7fda1c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fda1c0b29d0 0x7fda1c0b4ee0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55ea92dcae10 tx=0x7fda28032ba0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 -- 172.21.15.63:0/2579827619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda3810b610 msgr2=0x7fda3819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda3810b610 0x7fda3819cd10 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fda2400e9f0 tx=0x7fda2400eeb0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 -- 172.21.15.63:0/2579827619 shutdown_connections 2024-05-27T08:02:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fda1c0b29d0 0x7fda1c0b4ee0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3810e000 0x7fda381a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fda3810b610 0x7fda3819cd10 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 --2- 172.21.15.63:0/2579827619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda38104bb0 0x7fda3819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 -- 172.21.15.63:0/2579827619 >> 172.21.15.63:0/2579827619 conn(0x7fda380fe210 msgr2=0x7fda38109c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.369+0000 7fda3d600640 1 -- 172.21.15.63:0/2579827619 shutdown_connections 2024-05-27T08:02:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:02:53.370+0000 7fda3d600640 1 -- 172.21.15.63:0/2579827619 wait complete. 2024-05-27T08:02:53.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.467+0000 7f277f400700 1 -- 172.21.15.63:0/679326799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27781054d0 msgr2=0x7f27781058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.467+0000 7f277f400700 1 --2- 172.21.15.63:0/679326799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27781054d0 0x7f27781058e0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f276c009990 tx=0x7f276c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.467+0000 7f277f400700 1 -- 172.21.15.63:0/679326799 shutdown_connections 2024-05-27T08:02:53.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.467+0000 7f277f400700 1 --2- 172.21.15.63:0/679326799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f277810e900 0x7f2778110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.467+0000 7f277f400700 1 --2- 172.21.15.63:0/679326799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27781054d0 0x7f27781058e0 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.467+0000 7f277f400700 1 --2- 172.21.15.63:0/679326799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2778105e20 0x7f277810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.467+0000 7f277f400700 1 -- 172.21.15.63:0/679326799 >> 172.21.15.63:0/679326799 conn(0x7f27780fedb0 msgr2=0x7f27781011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:53.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.467+0000 7f277f400700 1 -- 172.21.15.63:0/679326799 shutdown_connections 2024-05-27T08:02:53.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.467+0000 7f277f400700 1 -- 172.21.15.63:0/679326799 wait complete. 2024-05-27T08:02:53.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.469+0000 7f277f400700 1 Processor -- start 2024-05-27T08:02:53.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.469+0000 7f277f400700 1 -- start start 2024-05-27T08:02:53.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.469+0000 7f277f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27781054d0 0x7f277819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.469+0000 7f277f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778105e20 0x7f277819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.469+0000 7f277f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277810e900 0x7f27781a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.469+0000 7f277f400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2778114320 con 0x7f277810e900 2024-05-27T08:02:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.469+0000 7f277f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27781141a0 con 0x7f27781054d0 2024-05-27T08:02:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.469+0000 7f277f400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f27781144a0 con 0x7f2778105e20 2024-05-27T08:02:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.470+0000 7f277d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277810e900 0x7f27781a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.470+0000 7f277ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27781054d0 0x7f277819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.470+0000 7f277d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277810e900 0x7f27781a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43846/0 (socket says 172.21.15.63:43846) 2024-05-27T08:02:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.470+0000 7f277d400700 1 -- 172.21.15.63:0/2840916717 learned_addr learned my addr 172.21.15.63:0/2840916717 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:53.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.470+0000 7f2777e00700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778105e20 0x7f277819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.470+0000 7f277d400700 1 -- 172.21.15.63:0/2840916717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778105e20 msgr2=0x7f277819d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.471+0000 7f277d400700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778105e20 0x7f277819d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.471+0000 7f277d400700 1 -- 172.21.15.63:0/2840916717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27781054d0 msgr2=0x7f277819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.471+0000 7f277d400700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27781054d0 0x7f277819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.471+0000 7f277d400700 1 -- 172.21.15.63:0/2840916717 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781a1e20 con 0x7f277810e900 2024-05-27T08:02:53.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.471+0000 7f277d400700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277810e900 0x7f27781a1780 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f2770007ee0 tx=0x7f277000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:53.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.471+0000 7f2775600700 1 -- 172.21.15.63:0/2840916717 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770004bb0 con 0x7f277810e900 2024-05-27T08:02:53.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.471+0000 7f2775600700 1 -- 172.21.15.63:0/2840916717 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2770004d50 con 0x7f277810e900 2024-05-27T08:02:53.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.471+0000 7f277f400700 1 -- 172.21.15.63:0/2840916717 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27781a2110 con 0x7f277810e900 2024-05-27T08:02:53.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.471+0000 7f2775600700 1 -- 172.21.15.63:0/2840916717 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27700058b0 con 0x7f277810e900 2024-05-27T08:02:53.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.472+0000 7f277f400700 1 -- 172.21.15.63:0/2840916717 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781a9a30 con 0x7f277810e900 2024-05-27T08:02:53.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.473+0000 7f277f400700 1 -- 172.21.15.63:0/2840916717 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27780512b0 con 0x7f277810e900 2024-05-27T08:02:53.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.476+0000 7f2775600700 1 -- 172.21.15.63:0/2840916717 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f277001a020 con 0x7f277810e900 2024-05-27T08:02:53.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.479+0000 7f2775600700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f27600b2a70 0x7f27600b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:53.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.479+0000 7f2775600700 1 -- 172.21.15.63:0/2840916717 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7f27700e2660 con 0x7f277810e900 2024-05-27T08:02:53.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.479+0000 7f277ca00700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f27600b2a70 0x7f27600b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:53.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.480+0000 7f277ca00700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f27600b2a70 0x7f27600b4f80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f276c006010 tx=0x7f276c00b540 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:53.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.482+0000 7f2775600700 1 -- 172.21.15.63:0/2840916717 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2770010070 con 0x7f277810e900 2024-05-27T08:02:53.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.802+0000 7f277f400700 1 -- 172.21.15.63:0/2840916717 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27781a9c60 con 0x7f277810e900 2024-05-27T08:02:53.804 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:53 smithi063 ceph-mon[31613]: pgmap v929: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 14 op/s 2024-05-27T08:02:53.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.804+0000 7f2775600700 1 -- 172.21.15.63:0/2840916717 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7f27700a9c20 con 0x7f277810e900 2024-05-27T08:02:53.807 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:02:53.807 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:02:53.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 -- 172.21.15.63:0/2840916717 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f27600b2a70 msgr2=0x7f27600b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f27600b2a70 0x7f27600b4f80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f276c006010 tx=0x7f276c00b540 comp rx=0 tx=0).stop 2024-05-27T08:02:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 -- 172.21.15.63:0/2840916717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277810e900 msgr2=0x7f27781a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277810e900 0x7f27781a1780 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f2770007ee0 tx=0x7f277000d3b0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 -- 172.21.15.63:0/2840916717 shutdown_connections 2024-05-27T08:02:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f27600b2a70 0x7f27600b4f80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277810e900 0x7f27781a1780 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27781054d0 0x7f277819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 --2- 172.21.15.63:0/2840916717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778105e20 0x7f277819d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 -- 172.21.15.63:0/2840916717 >> 172.21.15.63:0/2840916717 conn(0x7f27780fedb0 msgr2=0x7f2778103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 -- 172.21.15.63:0/2840916717 shutdown_connections 2024-05-27T08:02:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:53.808+0000 7f277f400700 1 -- 172.21.15.63:0/2840916717 wait complete. 2024-05-27T08:02:53.811 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:02:53.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:53 smithi086 ceph-mon[34687]: pgmap v929: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 14 op/s 2024-05-27T08:02:53.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:53 smithi149 ceph-mon[35408]: pgmap v929: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 14 op/s 2024-05-27T08:02:54.773 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:02:54.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:54 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2840916717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:02:54.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:54 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2840916717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:02:54.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:54 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2840916717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:02:55.267 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:55.690 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:55 smithi063 ceph-mon[31613]: pgmap v930: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-05-27T08:02:55.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:55 smithi086 ceph-mon[34687]: pgmap v930: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-05-27T08:02:55.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:55 smithi149 ceph-mon[35408]: pgmap v930: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-05-27T08:02:57.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.919+0000 7fee16200700 1 -- 172.21.15.63:0/4154506396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10108d40 msgr2=0x7fee10109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.919+0000 7fee16200700 1 --2- 172.21.15.63:0/4154506396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10108d40 0x7fee10109150 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0099c0 tx=0x7fedfc009cd0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.920+0000 7fee16200700 1 -- 172.21.15.63:0/4154506396 shutdown_connections 2024-05-27T08:02:57.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.920+0000 7fee16200700 1 --2- 172.21.15.63:0/4154506396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10109690 0x7fee10110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.920+0000 7fee16200700 1 --2- 172.21.15.63:0/4154506396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee10102920 0x7fee10102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.920+0000 7fee16200700 1 --2- 172.21.15.63:0/4154506396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10108d40 0x7fee10109150 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.920+0000 7fee16200700 1 -- 172.21.15.63:0/4154506396 >> 172.21.15.63:0/4154506396 conn(0x7fee100783a0 msgr2=0x7fee100787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:57.922 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:57 smithi063 ceph-mon[31613]: pgmap v931: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-05-27T08:02:57.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.920+0000 7fee16200700 1 -- 172.21.15.63:0/4154506396 shutdown_connections 2024-05-27T08:02:57.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.920+0000 7fee16200700 1 -- 172.21.15.63:0/4154506396 wait complete. 2024-05-27T08:02:57.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.922+0000 7fee16200700 1 Processor -- start 2024-05-27T08:02:57.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.922+0000 7fee16200700 1 -- start start 2024-05-27T08:02:57.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.922+0000 7fee16200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10102920 0x7fee1019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.922+0000 7fee16200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee10108d40 0x7fee1019d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.922+0000 7fee16200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10109690 0x7fee101a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.922+0000 7fee16200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee10114190 con 0x7fee10109690 2024-05-27T08:02:57.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.922+0000 7fee16200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee10114010 con 0x7fee10102920 2024-05-27T08:02:57.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.922+0000 7fee16200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee10114310 con 0x7fee10108d40 2024-05-27T08:02:57.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.923+0000 7fee15800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10109690 0x7fee101a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.923+0000 7fee15800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10109690 0x7fee101a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43870/0 (socket says 172.21.15.63:43870) 2024-05-27T08:02:57.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.923+0000 7fee15800700 1 -- 172.21.15.63:0/4055490695 learned_addr learned my addr 172.21.15.63:0/4055490695 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:57.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.923+0000 7fee14e00700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10102920 0x7fee1019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.923+0000 7fee0fe00700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee10108d40 0x7fee1019d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.924+0000 7fee15800700 1 -- 172.21.15.63:0/4055490695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee10108d40 msgr2=0x7fee1019d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.924+0000 7fee15800700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee10108d40 0x7fee1019d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.924+0000 7fee15800700 1 -- 172.21.15.63:0/4055490695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10102920 msgr2=0x7fee1019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.924+0000 7fee15800700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10102920 0x7fee1019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.924+0000 7fee15800700 1 -- 172.21.15.63:0/4055490695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee101a1c10 con 0x7fee10109690 2024-05-27T08:02:57.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.924+0000 7fee0fe00700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee10108d40 0x7fee1019d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:57.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.924+0000 7fee15800700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10109690 0x7fee101a1570 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fee0800ef30 tx=0x7fee0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.925+0000 7fee0d600700 1 -- 172.21.15.63:0/4055490695 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee080092e0 con 0x7fee10109690 2024-05-27T08:02:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.925+0000 7fee16200700 1 -- 172.21.15.63:0/4055490695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee101a1ea0 con 0x7fee10109690 2024-05-27T08:02:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.925+0000 7fee0d600700 1 -- 172.21.15.63:0/4055490695 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee08004bb0 con 0x7fee10109690 2024-05-27T08:02:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.925+0000 7fee0d600700 1 -- 172.21.15.63:0/4055490695 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08004020 con 0x7fee10109690 2024-05-27T08:02:57.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.925+0000 7fee16200700 1 -- 172.21.15.63:0/4055490695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee101a23b0 con 0x7fee10109690 2024-05-27T08:02:57.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.930+0000 7fee0d600700 1 -- 172.21.15.63:0/4055490695 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fee0801b030 con 0x7fee10109690 2024-05-27T08:02:57.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.930+0000 7fee263d8b80 1 Processor -- start 2024-05-27T08:02:57.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.931+0000 7fee263d8b80 1 -- start start 2024-05-27T08:02:57.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.931+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4b30e40 0x5648b4caeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.931+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4caf410 0x5648b4cb9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.931+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cbc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.931+0000 7fee263d8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5648b4bdde40 con 0x5648b4caf820 2024-05-27T08:02:57.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.931+0000 7fee263d8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648b4bcd9e0 con 0x5648b4caf410 2024-05-27T08:02:57.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.931+0000 7fee263d8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648b4bd1a20 con 0x5648b4b30e40 2024-05-27T08:02:57.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.932+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cbc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.932+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cbc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43874/0 (socket says 172.21.15.63:43874) 2024-05-27T08:02:57.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.932+0000 7fee0fe00700 1 -- 172.21.15.63:0/829751936 learned_addr learned my addr 172.21.15.63:0/829751936 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:57.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.932+0000 7fee14e00700 1 --2- 172.21.15.63:0/829751936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4caf410 0x5648b4cb9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.932+0000 7fee0fe00700 1 --2- 172.21.15.63:0/829751936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4b30e40 0x5648b4caeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.932+0000 7fee0fe00700 1 -- 172.21.15.63:0/829751936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4b30e40 msgr2=0x5648b4caeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.932+0000 7fee0fe00700 1 --2- 172.21.15.63:0/829751936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4b30e40 0x5648b4caeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.932+0000 7fee0fe00700 1 -- 172.21.15.63:0/829751936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4caf410 msgr2=0x5648b4cb9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.932+0000 7fee0fe00700 1 --2- 172.21.15.63:0/829751936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4caf410 0x5648b4cb9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.932+0000 7fee0fe00700 1 -- 172.21.15.63:0/829751936 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b4b96420 con 0x5648b4caf820 2024-05-27T08:02:57.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.933+0000 7fee0d600700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fedf80b29f0 0x7fedf80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.933+0000 7fee0fe00700 1 --2- 172.21.15.63:0/829751936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cbc020 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fedfc005f50 tx=0x7fedfc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e9e09acf8b486b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.933+0000 7fee0d600700 1 -- 172.21.15.63:0/4055490695 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fee080e2560 con 0x7fee10109690 2024-05-27T08:02:57.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.933+0000 7fee14e00700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fedf80b29f0 0x7fedf80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.934+0000 7fee02000700 1 -- 172.21.15.63:0/829751936 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0054a0 con 0x5648b4caf820 2024-05-27T08:02:57.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.934+0000 7fee02000700 1 -- 172.21.15.63:0/829751936 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedfc005640 con 0x5648b4caf820 2024-05-27T08:02:57.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.934+0000 7fee02000700 1 -- 172.21.15.63:0/829751936 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc00bc70 con 0x5648b4caf820 2024-05-27T08:02:57.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.935+0000 7fee263d8b80 1 -- 172.21.15.63:0/829751936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 msgr2=0x5648b4cbc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.935+0000 7fee14e00700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fedf80b29f0 0x7fedf80b4f00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5648b4cb0430 tx=0x7fee0400b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.935+0000 7fee263d8b80 1 --2- 172.21.15.63:0/829751936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cbc020 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fedfc005f50 tx=0x7fedfc005e80 comp rx=0 tx=0).stop 2024-05-27T08:02:57.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.935+0000 7fee263d8b80 1 -- 172.21.15.63:0/829751936 shutdown_connections 2024-05-27T08:02:57.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.935+0000 7fee263d8b80 1 --2- 172.21.15.63:0/829751936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cbc020 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.935+0000 7fee263d8b80 1 --2- 172.21.15.63:0/829751936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4b30e40 0x5648b4caeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.935+0000 7fee263d8b80 1 --2- 172.21.15.63:0/829751936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4caf410 0x5648b4cb9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.935+0000 7fee263d8b80 1 -- 172.21.15.63:0/829751936 >> 172.21.15.63:0/829751936 conn(0x5648b4b90fc0 msgr2=0x5648b4caad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:57.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.935+0000 7fee263d8b80 1 -- 172.21.15.63:0/829751936 shutdown_connections 2024-05-27T08:02:57.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.935+0000 7fee263d8b80 1 -- 172.21.15.63:0/829751936 wait complete. 2024-05-27T08:02:57.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.936+0000 7fee263d8b80 1 -- start start 2024-05-27T08:02:57.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.936+0000 7fee263d8b80 1 Processor -- start 2024-05-27T08:02:57.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.937+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cb9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.937+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf410 0x5648b4cb9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.937+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.937+0000 7fee263d8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5648b4ad3fa0 con 0x5648b4caf820 2024-05-27T08:02:57.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.937+0000 7fee263d8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648b4b6e760 con 0x5648b4b30e40 2024-05-27T08:02:57.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.937+0000 7fee263d8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648b4cbc930 con 0x5648b4caf410 2024-05-27T08:02:57.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.937+0000 7fee263d8b80 10 client.0 _ll_register_callbacks cb 0x5648b4b77c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:57.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.937+0000 7fee15800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.938+0000 7fee15800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cba000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43878/0 (socket says 172.21.15.63:43878) 2024-05-27T08:02:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.938+0000 7fee14e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cb9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.938+0000 7fee15800700 1 -- 172.21.15.63:0/831461372 learned_addr learned my addr 172.21.15.63:0/831461372 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.938+0000 7fee0fe00700 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf410 0x5648b4cb9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.938+0000 7fee15800700 1 -- 172.21.15.63:0/831461372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf410 msgr2=0x5648b4cb9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.938+0000 7fee15800700 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf410 0x5648b4cb9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.938+0000 7fee15800700 1 -- 172.21.15.63:0/831461372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 msgr2=0x5648b4cb9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.938+0000 7fee15800700 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cb9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.938+0000 7fee15800700 1 -- 172.21.15.63:0/831461372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b4b1d8d0 con 0x5648b4caf820 2024-05-27T08:02:57.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.938+0000 7fee14e00700 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cb9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:57.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.939+0000 7fee15800700 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cba000 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fee080a9440 tx=0x7fee08004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f1c50eb1bcca8e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.939+0000 7fedf6a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:57.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.939+0000 7fedf6a00700 1 -- 172.21.15.63:0/831461372 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee080a9bb0 con 0x5648b4caf820 2024-05-27T08:02:57.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.939+0000 7fedf6a00700 1 -- 172.21.15.63:0/831461372 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fee080a3bd0 con 0x5648b4caf820 2024-05-27T08:02:57.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.939+0000 7fee263d8b80 1 -- 172.21.15.63:0/831461372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedfc005640 con 0x5648b4caf820 2024-05-27T08:02:57.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.940+0000 7fedf6a00700 1 -- 172.21.15.63:0/831461372 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fee080a94b0 con 0x5648b4caf820 2024-05-27T08:02:57.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.940+0000 7fee263d8b80 10 client.21063 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:57.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.940+0000 7fee263d8b80 1 -- 172.21.15.63:0/831461372 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5648b4b9e060 con 0x5648b4caf820 2024-05-27T08:02:57.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.940+0000 7fedf6a00700 1 -- 172.21.15.63:0/831461372 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedfc005640 con 0x5648b4caf820 2024-05-27T08:02:57.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.940+0000 7fedf6a00700 1 -- 172.21.15.63:0/831461372 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fee080b1e80 con 0x5648b4caf820 2024-05-27T08:02:57.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.941+0000 7fee263d8b80 10 client.21063 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:57.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.941+0000 7fee263d8b80 10 client.21063 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:02:57.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.941+0000 7fee263d8b80 20 client.21063 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:57.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.941+0000 7fee263d8b80 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648b4cc98f0 0x5648b4ccbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.941+0000 7fee263d8b80 4 client.21063 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:57.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.941+0000 7fee263d8b80 1 -- 172.21.15.63:0/831461372 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648b4bdfd90 con 0x5648b4cc98f0 2024-05-27T08:02:57.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.941+0000 7fee14e00700 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648b4cc98f0 0x5648b4ccbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.942+0000 7fee14e00700 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648b4cc98f0 0x5648b4ccbd10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7009ad8c569ccb48 server_cookie=a35412cfbbf05334 in_seq=0 out_seq=0 2024-05-27T08:02:57.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.942+0000 7fedf6a00700 10 client.21063 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:02:57.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.944+0000 7fedf6a00700 1 -- 172.21.15.63:0/831461372 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5648b4bdfd90 con 0x5648b4cc98f0 2024-05-27T08:02:57.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.944+0000 7fedf6a00700 10 client.21063 handle_command_reply: tid=0 2024-05-27T08:02:57.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.944+0000 7fee263d8b80 1 client.21063 shutdown 2024-05-27T08:02:57.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 1 -- 172.21.15.63:0/831461372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 msgr2=0x5648b4cba000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cba000 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fee080a9440 tx=0x7fee08004f90 comp rx=0 tx=0).stop 2024-05-27T08:02:57.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 1 -- 172.21.15.63:0/831461372 shutdown_connections 2024-05-27T08:02:57.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648b4cc98f0 0x5648b4ccbd10 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf820 0x5648b4cba000 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cb9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 1 --2- 172.21.15.63:0/831461372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf410 0x5648b4cb9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 1 -- 172.21.15.63:0/831461372 >> 172.21.15.63:0/831461372 conn(0x5648b4b90fc0 msgr2=0x5648b4ca9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:57.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fedf6a00700 0 client.21063 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:02:57.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 1 -- 172.21.15.63:0/831461372 shutdown_connections 2024-05-27T08:02:57.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 1 -- 172.21.15.63:0/831461372 wait complete. 2024-05-27T08:02:57.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.945+0000 7fee263d8b80 20 client.21063 trim_cache size 0 max 16384 2024-05-27T08:02:57.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:57 smithi086 ceph-mon[34687]: pgmap v931: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-05-27T08:02:57.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:57 smithi149 ceph-mon[35408]: pgmap v931: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 MiB/s wr, 6 op/s 2024-05-27T08:02:57.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.966+0000 7fee263d8b80 1 Processor -- start 2024-05-27T08:02:57.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.966+0000 7fee263d8b80 1 -- start start 2024-05-27T08:02:57.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.966+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cbba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.967+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cb37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.967+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf820 0x5648b4cd14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.967+0000 7fee263d8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee101b4d30 con 0x5648b4caf410 2024-05-27T08:02:57.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.967+0000 7fee263d8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee101b25c0 con 0x5648b4b30e40 2024-05-27T08:02:57.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.967+0000 7fee263d8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee101b0430 con 0x5648b4caf820 2024-05-27T08:02:57.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.967+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cb37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.967+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cb37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43892/0 (socket says 172.21.15.63:43892) 2024-05-27T08:02:57.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.967+0000 7fee0fe00700 1 -- 172.21.15.63:0/3373967053 learned_addr learned my addr 172.21.15.63:0/3373967053 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:57.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.967+0000 7fee14e00700 1 --2- 172.21.15.63:0/3373967053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cbba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.967+0000 7fee15800700 1 --2- 172.21.15.63:0/3373967053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf820 0x5648b4cd14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.968+0000 7fee0fe00700 1 -- 172.21.15.63:0/3373967053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf820 msgr2=0x5648b4cd14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.968+0000 7fee0fe00700 1 --2- 172.21.15.63:0/3373967053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf820 0x5648b4cd14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.968+0000 7fee0fe00700 1 -- 172.21.15.63:0/3373967053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 msgr2=0x5648b4cbba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.968+0000 7fee0fe00700 1 --2- 172.21.15.63:0/3373967053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cbba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.968+0000 7fee0fe00700 1 -- 172.21.15.63:0/3373967053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648b4cac350 con 0x5648b4caf410 2024-05-27T08:02:57.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.969+0000 7fee0fe00700 1 --2- 172.21.15.63:0/3373967053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cb37a0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x5648b4cafcb0 tx=0x7fedfc005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fb66572c0cdff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.969+0000 7fee01600700 1 -- 172.21.15.63:0/3373967053 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc034070 con 0x5648b4caf410 2024-05-27T08:02:57.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.969+0000 7fee01600700 1 -- 172.21.15.63:0/3373967053 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedfc003c50 con 0x5648b4caf410 2024-05-27T08:02:57.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.970+0000 7fee01600700 1 -- 172.21.15.63:0/3373967053 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc02b040 con 0x5648b4caf410 2024-05-27T08:02:57.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.971+0000 7fee263d8b80 1 -- 172.21.15.63:0/3373967053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 msgr2=0x5648b4cb37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.971+0000 7fee263d8b80 1 --2- 172.21.15.63:0/3373967053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cb37a0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x5648b4cafcb0 tx=0x7fedfc005fb0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.971+0000 7fee263d8b80 1 -- 172.21.15.63:0/3373967053 shutdown_connections 2024-05-27T08:02:57.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.971+0000 7fee263d8b80 1 --2- 172.21.15.63:0/3373967053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf820 0x5648b4cd14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.971+0000 7fee263d8b80 1 --2- 172.21.15.63:0/3373967053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cbba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.971+0000 7fee263d8b80 1 --2- 172.21.15.63:0/3373967053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cb37a0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.971+0000 7fee263d8b80 1 -- 172.21.15.63:0/3373967053 >> 172.21.15.63:0/3373967053 conn(0x5648b4b90fc0 msgr2=0x5648b4cb9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:57.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.972+0000 7fee263d8b80 1 -- 172.21.15.63:0/3373967053 shutdown_connections 2024-05-27T08:02:57.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.972+0000 7fee263d8b80 1 -- 172.21.15.63:0/3373967053 wait complete. 2024-05-27T08:02:57.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.972+0000 7fee263d8b80 1 -- start start 2024-05-27T08:02:57.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.973+0000 7fee263d8b80 1 Processor -- start 2024-05-27T08:02:57.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.974+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cc1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.974+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cbd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.974+0000 7fee263d8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf820 0x5648b4cbd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.974+0000 7fee263d8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee101b0a80 con 0x5648b4caf410 2024-05-27T08:02:57.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.974+0000 7fee263d8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee101b1ef0 con 0x5648b4b30e40 2024-05-27T08:02:57.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.974+0000 7fee263d8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee101b1150 con 0x5648b4caf820 2024-05-27T08:02:57.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.974+0000 7fee263d8b80 10 client.0 _ll_register_callbacks cb 0x5648b4b77c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:57.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cbd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cbd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43898/0 (socket says 172.21.15.63:43898) 2024-05-27T08:02:57.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee0fe00700 1 -- 172.21.15.63:0/2293097213 learned_addr learned my addr 172.21.15.63:0/2293097213 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:02:57.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee14e00700 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cc1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee15800700 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf820 0x5648b4cbd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee0fe00700 1 -- 172.21.15.63:0/2293097213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf820 msgr2=0x5648b4cbd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee0fe00700 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf820 0x5648b4cbd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee0fe00700 1 -- 172.21.15.63:0/2293097213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 msgr2=0x5648b4cc1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee0fe00700 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cc1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee0fe00700 1 -- 172.21.15.63:0/2293097213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648b4cbddc0 con 0x5648b4caf410 2024-05-27T08:02:57.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.975+0000 7fee14e00700 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cc1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:57.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.976+0000 7fee0fe00700 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cbd340 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fedfc005d10 tx=0x7fedfc009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9606a15a427e5918 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.976+0000 7fedf6a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:02:57.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.976+0000 7fedf6a00700 1 -- 172.21.15.63:0/2293097213 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc034070 con 0x5648b4caf410 2024-05-27T08:02:57.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.976+0000 7fedf6a00700 1 -- 172.21.15.63:0/2293097213 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9483+0+0 (secure 0 0 0) 0x7fedfc005220 con 0x5648b4caf410 2024-05-27T08:02:57.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.976+0000 7fee263d8b80 1 -- 172.21.15.63:0/2293097213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedfc003c50 con 0x5648b4caf410 2024-05-27T08:02:57.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.977+0000 7fedf6a00700 1 -- 172.21.15.63:0/2293097213 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fedfc00b850 con 0x5648b4caf410 2024-05-27T08:02:57.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.977+0000 7fee263d8b80 10 client.21069 fetch_fsmap learned FSMap version 35 2024-05-27T08:02:57.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.977+0000 7fee263d8b80 1 -- 172.21.15.63:0/2293097213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5648b4d3c770 con 0x5648b4caf410 2024-05-27T08:02:57.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.977+0000 7fedf6a00700 1 -- 172.21.15.63:0/2293097213 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedfc003c50 con 0x5648b4caf410 2024-05-27T08:02:57.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.978+0000 7fedf6a00700 1 -- 172.21.15.63:0/2293097213 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fedfc041030 con 0x5648b4caf410 2024-05-27T08:02:57.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.978+0000 7fee263d8b80 10 client.21069 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:02:57.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.978+0000 7fee263d8b80 10 client.21069 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:02:57.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.978+0000 7fee263d8b80 20 client.21069 populate_metadata read hostname 'smithi063' 2024-05-27T08:02:57.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.978+0000 7fee263d8b80 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648b4d3c940 0x5648b4d3ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.978+0000 7fee263d8b80 4 client.21069 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:02:57.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.978+0000 7fee263d8b80 1 -- 172.21.15.63:0/2293097213 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648b4bd9570 con 0x5648b4d3c940 2024-05-27T08:02:57.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.979+0000 7fee14e00700 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648b4d3c940 0x5648b4d3ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fee14e00700 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648b4d3c940 0x5648b4d3ed60 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=800b0422264ee6b6 server_cookie=6f83fc24462f4aaf in_seq=0 out_seq=0 2024-05-27T08:02:57.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fedf6a00700 10 client.21069 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:02:57.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fedf6a00700 1 -- 172.21.15.63:0/2293097213 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648b4bd9570 con 0x5648b4d3c940 2024-05-27T08:02:57.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fedf6a00700 10 client.21069 handle_command_reply: tid=0 2024-05-27T08:02:57.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fee263d8b80 1 client.21069 shutdown 2024-05-27T08:02:57.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fee263d8b80 1 -- 172.21.15.63:0/2293097213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 msgr2=0x5648b4cbd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fee263d8b80 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cbd340 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fedfc005d10 tx=0x7fedfc009690 comp rx=0 tx=0).stop 2024-05-27T08:02:57.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fee263d8b80 1 -- 172.21.15.63:0/2293097213 shutdown_connections 2024-05-27T08:02:57.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fee263d8b80 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648b4caf820 0x5648b4cbd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fee263d8b80 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648b4b30e40 0x5648b4cc1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fee263d8b80 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5648b4d3c940 0x5648b4d3ed60 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fee263d8b80 1 --2- 172.21.15.63:0/2293097213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b4caf410 0x5648b4cbd340 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.980+0000 7fee263d8b80 1 -- 172.21.15.63:0/2293097213 >> 172.21.15.63:0/2293097213 conn(0x5648b4b90fc0 msgr2=0x5648b4cc5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:57.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.981+0000 7fedf6a00700 0 client.21069 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:02:57.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.981+0000 7fee263d8b80 1 -- 172.21.15.63:0/2293097213 shutdown_connections 2024-05-27T08:02:57.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.981+0000 7fee263d8b80 1 -- 172.21.15.63:0/2293097213 wait complete. 2024-05-27T08:02:57.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.981+0000 7fee263d8b80 20 client.21069 trim_cache size 0 max 16384 2024-05-27T08:02:57.984 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:02:57.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 -- 172.21.15.63:0/4055490695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fedf80b29f0 msgr2=0x7fedf80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fedf80b29f0 0x7fedf80b4f00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5648b4cb0430 tx=0x7fee0400b410 comp rx=0 tx=0).stop 2024-05-27T08:02:57.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 -- 172.21.15.63:0/4055490695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10109690 msgr2=0x7fee101a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10109690 0x7fee101a1570 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fee0800ef30 tx=0x7fee0800c3a0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 -- 172.21.15.63:0/4055490695 shutdown_connections 2024-05-27T08:02:57.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fedf80b29f0 0x7fedf80b4f00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee10109690 0x7fee101a1570 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee10102920 0x7fee1019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 --2- 172.21.15.63:0/4055490695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee10108d40 0x7fee1019d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 -- 172.21.15.63:0/4055490695 >> 172.21.15.63:0/4055490695 conn(0x7fee100783a0 msgr2=0x7fee100ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:57.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 -- 172.21.15.63:0/4055490695 shutdown_connections 2024-05-27T08:02:57.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:02:57.982+0000 7fee16200700 1 -- 172.21.15.63:0/4055490695 wait complete. 2024-05-27T08:02:58.937 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:02:59.443 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:59.950 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:02:59 smithi063 ceph-mon[31613]: pgmap v932: 129 pgs: 129 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T08:02:59.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:02:59 smithi086 ceph-mon[34687]: pgmap v932: 129 pgs: 129 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T08:02:59.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:02:59 smithi149 ceph-mon[35408]: pgmap v932: 129 pgs: 129 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T08:03:01.758 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:01 smithi063 ceph-mon[31613]: pgmap v933: 129 pgs: 129 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:03:01.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:01 smithi086 ceph-mon[34687]: pgmap v933: 129 pgs: 129 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:03:01.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:01 smithi149 ceph-mon[35408]: pgmap v933: 129 pgs: 129 active+clean; 8.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:03:01.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.978+0000 7f1020a00700 1 -- 172.21.15.63:0/4127691174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c075b50 msgr2=0x7f101c075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:01.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.978+0000 7f1020a00700 1 --2- 172.21.15.63:0/4127691174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c075b50 0x7f101c075f60 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f100c009990 tx=0x7f100c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.978+0000 7f1020a00700 1 -- 172.21.15.63:0/4127691174 shutdown_connections 2024-05-27T08:03:01.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.978+0000 7f1020a00700 1 --2- 172.21.15.63:0/4127691174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f101c115170 0x7f101c117590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.978+0000 7f1020a00700 1 --2- 172.21.15.63:0/4127691174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c075b50 0x7f101c075f60 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.978+0000 7f1020a00700 1 --2- 172.21.15.63:0/4127691174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c0764a0 0x7f101c114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.978+0000 7f1020a00700 1 -- 172.21.15.63:0/4127691174 >> 172.21.15.63:0/4127691174 conn(0x7f101c079d00 msgr2=0x7f101c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:01.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.978+0000 7f1020a00700 1 -- 172.21.15.63:0/4127691174 shutdown_connections 2024-05-27T08:03:01.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.978+0000 7f1020a00700 1 -- 172.21.15.63:0/4127691174 wait complete. 2024-05-27T08:03:01.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.980+0000 7f1020a00700 1 Processor -- start 2024-05-27T08:03:01.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.980+0000 7f1020a00700 1 -- start start 2024-05-27T08:03:01.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.981+0000 7f1020a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c075b50 0x7f101c1a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:01.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.981+0000 7f1020a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c0764a0 0x7f101c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:01.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.981+0000 7f1020a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f101c115170 0x7f101c1a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:01.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.981+0000 7f1020a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f101c11a780 con 0x7f101c075b50 2024-05-27T08:03:01.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.981+0000 7f1020a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f101c11a600 con 0x7f101c0764a0 2024-05-27T08:03:01.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.981+0000 7f1020a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f101c11a900 con 0x7f101c115170 2024-05-27T08:03:01.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.982+0000 7f101b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c075b50 0x7f101c1a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:01.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.982+0000 7f101b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c075b50 0x7f101c1a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37872/0 (socket says 172.21.15.63:37872) 2024-05-27T08:03:01.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.982+0000 7f101b400700 1 -- 172.21.15.63:0/2544873337 learned_addr learned my addr 172.21.15.63:0/2544873337 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:01.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.982+0000 7f101aa00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c0764a0 0x7f101c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:01.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.982+0000 7f101be00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f101c115170 0x7f101c1a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:01.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.982+0000 7f101b400700 1 -- 172.21.15.63:0/2544873337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f101c115170 msgr2=0x7f101c1a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:01.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.982+0000 7f101b400700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f101c115170 0x7f101c1a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.982+0000 7f101b400700 1 -- 172.21.15.63:0/2544873337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c0764a0 msgr2=0x7f101c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:01.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.982+0000 7f101b400700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c0764a0 0x7f101c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.983+0000 7f101b400700 1 -- 172.21.15.63:0/2544873337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c1a6080 con 0x7f101c075b50 2024-05-27T08:03:01.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.983+0000 7f101b400700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c075b50 0x7f101c1a11b0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f100c0047b0 tx=0x7f100c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:01.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.983+0000 7f1013e00700 1 -- 172.21.15.63:0/2544873337 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c004830 con 0x7f101c075b50 2024-05-27T08:03:01.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.984+0000 7f1013e00700 1 -- 172.21.15.63:0/2544873337 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f100c005020 con 0x7f101c075b50 2024-05-27T08:03:01.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.984+0000 7f1020a00700 1 -- 172.21.15.63:0/2544873337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101c1a6310 con 0x7f101c075b50 2024-05-27T08:03:01.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.984+0000 7f1013e00700 1 -- 172.21.15.63:0/2544873337 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c003cc0 con 0x7f101c075b50 2024-05-27T08:03:01.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.984+0000 7f1020a00700 1 -- 172.21.15.63:0/2544873337 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101c1a6790 con 0x7f101c075b50 2024-05-27T08:03:01.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.989+0000 7f1013e00700 1 -- 172.21.15.63:0/2544873337 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f100c0054d0 con 0x7f101c075b50 2024-05-27T08:03:01.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.991+0000 7f1013e00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10000b29f0 0x7f10000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:01.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.991+0000 7f1013e00700 1 -- 172.21.15.63:0/2544873337 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9571+0+0 (secure 0 0 0) 0x7f100c0e3790 con 0x7f101c075b50 2024-05-27T08:03:01.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.991+0000 7f1030bd8b80 1 Processor -- start 2024-05-27T08:03:01.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.991+0000 7f101aa00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10000b29f0 0x7f10000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:01.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.991+0000 7f1030bd8b80 1 -- start start 2024-05-27T08:03:01.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.992+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b40a89f0 0x5560b413c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:01.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.992+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b3fbce40 0x5560b413cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:01.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.992+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 0x5560b41486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:01.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.992+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5560b4069e40 con 0x5560b40a89f0 2024-05-27T08:03:01.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.992+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560b40599e0 con 0x5560b3fbce40 2024-05-27T08:03:01.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.992+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5560b405da20 con 0x5560b41462c0 2024-05-27T08:03:01.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.992+0000 7f101b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b40a89f0 0x5560b413c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:01.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.992+0000 7f101b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b40a89f0 0x5560b413c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37878/0 (socket says 172.21.15.63:37878) 2024-05-27T08:03:01.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.992+0000 7f101b400700 1 -- 172.21.15.63:0/3828050480 learned_addr learned my addr 172.21.15.63:0/3828050480 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:01.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.993+0000 7f101aa00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10000b29f0 0x7f10000b4f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f101c116e30 tx=0x7f100800b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:01.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.993+0000 7f101be00700 1 --2- 172.21.15.63:0/3828050480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 0x5560b41486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:01.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.993+0000 7f101aa00700 1 --2- 172.21.15.63:0/3828050480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b3fbce40 0x5560b413cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:01.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.993+0000 7f101b400700 1 -- 172.21.15.63:0/3828050480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 msgr2=0x5560b41486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:01.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.993+0000 7f101b400700 1 --2- 172.21.15.63:0/3828050480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 0x5560b41486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.993+0000 7f101b400700 1 -- 172.21.15.63:0/3828050480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b3fbce40 msgr2=0x5560b413cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:01.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.993+0000 7f101b400700 1 --2- 172.21.15.63:0/3828050480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b3fbce40 0x5560b413cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.993+0000 7f101b400700 1 -- 172.21.15.63:0/3828050480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b4022420 con 0x5560b40a89f0 2024-05-27T08:03:01.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.994+0000 7f101b400700 1 --2- 172.21.15.63:0/3828050480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b40a89f0 0x5560b413c750 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f100c0aa380 tx=0x7f100c0a9e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb63ac92ac22ef7e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:01.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.994+0000 7f1010c00700 1 -- 172.21.15.63:0/3828050480 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c0aad80 con 0x5560b40a89f0 2024-05-27T08:03:01.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.994+0000 7f1010c00700 1 -- 172.21.15.63:0/3828050480 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f100c0aa680 con 0x5560b40a89f0 2024-05-27T08:03:01.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.994+0000 7f1010c00700 1 -- 172.21.15.63:0/3828050480 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c0b3220 con 0x5560b40a89f0 2024-05-27T08:03:01.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.995+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3828050480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b40a89f0 msgr2=0x5560b413c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:01.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.995+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/3828050480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b40a89f0 0x5560b413c750 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f100c0aa380 tx=0x7f100c0a9e70 comp rx=0 tx=0).stop 2024-05-27T08:03:01.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.996+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3828050480 shutdown_connections 2024-05-27T08:03:01.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.996+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/3828050480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 0x5560b41486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.996+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/3828050480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b40a89f0 0x5560b413c750 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.996+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/3828050480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b3fbce40 0x5560b413cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:01.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.996+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3828050480 >> 172.21.15.63:0/3828050480 conn(0x5560b4056390 msgr2=0x5560b4137710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:01.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.996+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3828050480 shutdown_connections 2024-05-27T08:03:01.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.996+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3828050480 wait complete. 2024-05-27T08:03:01.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.997+0000 7f1030bd8b80 1 -- start start 2024-05-27T08:03:01.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.997+0000 7f1030bd8b80 1 Processor -- start 2024-05-27T08:03:01.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.998+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b3fbce40 0x5560b4143320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:01.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.998+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 0x5560b4143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:02.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.998+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b4143da0 0x5560b4151e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:02.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.998+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5560b3f5ffa0 con 0x5560b3fbce40 2024-05-27T08:03:02.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.998+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560b3ffa760 con 0x5560b4143da0 2024-05-27T08:03:02.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.998+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5560b4148c20 con 0x5560b41462c0 2024-05-27T08:03:02.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.998+0000 7f1030bd8b80 10 client.0 _ll_register_callbacks cb 0x5560b4003c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:02.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.998+0000 7f101b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b3fbce40 0x5560b4143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.998+0000 7f101b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b3fbce40 0x5560b4143320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37884/0 (socket says 172.21.15.63:37884) 2024-05-27T08:03:02.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.998+0000 7f101b400700 1 -- 172.21.15.63:0/3807186557 learned_addr learned my addr 172.21.15.63:0/3807186557 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:02.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.999+0000 7f101aa00700 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 0x5560b4143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.999+0000 7f101be00700 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b4143da0 0x5560b4151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.999+0000 7f101b400700 1 -- 172.21.15.63:0/3807186557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 msgr2=0x5560b4143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:02.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.999+0000 7f101b400700 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 0x5560b4143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.999+0000 7f101b400700 1 -- 172.21.15.63:0/3807186557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b4143da0 msgr2=0x5560b4151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:02.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.999+0000 7f101b400700 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b4143da0 0x5560b4151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.999+0000 7f101b400700 1 -- 172.21.15.63:0/3807186557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b3fa98d0 con 0x5560b3fbce40 2024-05-27T08:03:02.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.999+0000 7f101aa00700 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 0x5560b4143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:02.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.999+0000 7f101b400700 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b3fbce40 0x5560b4143320 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f100c0a78f0 tx=0x7f100c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b809e7b3c470e000 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:02.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:01.999+0000 7f1005600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:02.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.000+0000 7f1005600700 1 -- 172.21.15.63:0/3807186557 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c0c4060 con 0x5560b3fbce40 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.000+0000 7f1005600700 1 -- 172.21.15.63:0/3807186557 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9571+0+0 (secure 0 0 0) 0x7f100c0afdd0 con 0x5560b3fbce40 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.000+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3807186557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100c0aa680 con 0x5560b3fbce40 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.000+0000 7f1005600700 1 -- 172.21.15.63:0/3807186557 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f100c0aad80 con 0x5560b3fbce40 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.000+0000 7f1030bd8b80 10 client.21081 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.000+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3807186557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5560b402a060 con 0x5560b3fbce40 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.001+0000 7f1005600700 1 -- 172.21.15.63:0/3807186557 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f100c0aa680 con 0x5560b3fbce40 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.001+0000 7f1005600700 1 -- 172.21.15.63:0/3807186557 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f100c0e32b0 con 0x5560b3fbce40 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.001+0000 7f1030bd8b80 10 client.21081 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.001+0000 7f1030bd8b80 10 client.21081 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.001+0000 7f1030bd8b80 20 client.21081 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.001+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5560b4155e70 0x5560b4158290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:02.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.001+0000 7f1030bd8b80 4 client.21081 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:02.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.001+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3807186557 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560b406bd90 con 0x5560b4155e70 2024-05-27T08:03:02.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.002+0000 7f101aa00700 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5560b4155e70 0x5560b4158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.003+0000 7f101aa00700 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5560b4155e70 0x5560b4158290 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa68659fce32133e server_cookie=2b5388c045240e6 in_seq=0 out_seq=0 2024-05-27T08:03:02.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.003+0000 7f1005600700 10 client.21081 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:03:02.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.005+0000 7f1005600700 1 -- 172.21.15.63:0/3807186557 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5560b406bd90 con 0x5560b4155e70 2024-05-27T08:03:02.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.005+0000 7f1005600700 10 client.21081 handle_command_reply: tid=0 2024-05-27T08:03:02.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.005+0000 7f1030bd8b80 1 client.21081 shutdown 2024-05-27T08:03:02.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.005+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3807186557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b3fbce40 msgr2=0x5560b4143320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:02.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.005+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b3fbce40 0x5560b4143320 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f100c0a78f0 tx=0x7f100c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.006+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3807186557 shutdown_connections 2024-05-27T08:03:02.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.006+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b4143da0 0x5560b4151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.006+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5560b4155e70 0x5560b4158290 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.006+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b3fbce40 0x5560b4143320 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.006+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/3807186557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b41462c0 0x5560b4143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.006+0000 7f1005600700 0 client.21081 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:03:02.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.006+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3807186557 >> 172.21.15.63:0/3807186557 conn(0x5560b4056390 msgr2=0x5560b4137300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:02.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.006+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3807186557 shutdown_connections 2024-05-27T08:03:02.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.006+0000 7f1030bd8b80 1 -- 172.21.15.63:0/3807186557 wait complete. 2024-05-27T08:03:02.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.007+0000 7f1030bd8b80 20 client.21081 trim_cache size 0 max 16384 2024-05-27T08:03:02.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.031+0000 7f1030bd8b80 1 Processor -- start 2024-05-27T08:03:02.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.031+0000 7f1030bd8b80 1 -- start start 2024-05-27T08:03:02.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.032+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b3fbce40 0x5560b4137740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:02.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.032+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b4143da0 0x5560b4142280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:02.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.032+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b41462c0 0x5560b413e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:02.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.032+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1b9380 con 0x5560b41462c0 2024-05-27T08:03:02.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.032+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1b6c10 con 0x5560b3fbce40 2024-05-27T08:03:02.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.032+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1b4a80 con 0x5560b4143da0 2024-05-27T08:03:02.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.032+0000 7f101be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b41462c0 0x5560b413e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.033+0000 7f101be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b41462c0 0x5560b413e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37886/0 (socket says 172.21.15.63:37886) 2024-05-27T08:03:02.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.033+0000 7f101be00700 1 -- 172.21.15.63:0/1175475844 learned_addr learned my addr 172.21.15.63:0/1175475844 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:02.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.033+0000 7f101aa00700 1 --2- 172.21.15.63:0/1175475844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b4143da0 0x5560b4142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.033+0000 7f101b400700 1 --2- 172.21.15.63:0/1175475844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b3fbce40 0x5560b4137740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.033+0000 7f101be00700 1 -- 172.21.15.63:0/1175475844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b4143da0 msgr2=0x5560b4142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:02.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.033+0000 7f101be00700 1 --2- 172.21.15.63:0/1175475844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b4143da0 0x5560b4142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.033+0000 7f101be00700 1 -- 172.21.15.63:0/1175475844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b3fbce40 msgr2=0x5560b4137740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:02.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.033+0000 7f101be00700 1 --2- 172.21.15.63:0/1175475844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b3fbce40 0x5560b4137740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.033+0000 7f101be00700 1 -- 172.21.15.63:0/1175475844 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b4138bf0 con 0x5560b41462c0 2024-05-27T08:03:02.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.034+0000 7f101be00700 1 --2- 172.21.15.63:0/1175475844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b41462c0 0x5560b413e950 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f1014009ec0 tx=0x7f1014009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bbc86a47da4096 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:02.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.034+0000 7f1007e00700 1 -- 172.21.15.63:0/1175475844 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014004db0 con 0x5560b41462c0 2024-05-27T08:03:02.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.034+0000 7f1007e00700 1 -- 172.21.15.63:0/1175475844 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1014007b80 con 0x5560b41462c0 2024-05-27T08:03:02.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.034+0000 7f1007e00700 1 -- 172.21.15.63:0/1175475844 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014017410 con 0x5560b41462c0 2024-05-27T08:03:02.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.036+0000 7f1030bd8b80 1 -- 172.21.15.63:0/1175475844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b41462c0 msgr2=0x5560b413e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:02.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.036+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/1175475844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b41462c0 0x5560b413e950 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f1014009ec0 tx=0x7f1014009ef0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.036+0000 7f1030bd8b80 1 -- 172.21.15.63:0/1175475844 shutdown_connections 2024-05-27T08:03:02.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.036+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/1175475844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b41462c0 0x5560b413e950 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.036+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/1175475844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b3fbce40 0x5560b4137740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.036+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/1175475844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b4143da0 0x5560b4142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.036+0000 7f1030bd8b80 1 -- 172.21.15.63:0/1175475844 >> 172.21.15.63:0/1175475844 conn(0x5560b4056390 msgr2=0x5560b413e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:02.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.036+0000 7f1030bd8b80 1 -- 172.21.15.63:0/1175475844 shutdown_connections 2024-05-27T08:03:02.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.036+0000 7f1030bd8b80 1 -- 172.21.15.63:0/1175475844 wait complete. 2024-05-27T08:03:02.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.037+0000 7f1030bd8b80 1 -- start start 2024-05-27T08:03:02.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.037+0000 7f1030bd8b80 1 Processor -- start 2024-05-27T08:03:02.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b3fbce40 0x5560b414e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:02.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b4143da0 0x5560b4149660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:02.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f1030bd8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b41462c0 0x5560b4149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:02.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1b50d0 con 0x5560b4143da0 2024-05-27T08:03:02.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1b6540 con 0x5560b41462c0 2024-05-27T08:03:02.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f1030bd8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f101c1b57a0 con 0x5560b3fbce40 2024-05-27T08:03:02.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f1030bd8b80 10 client.0 _ll_register_callbacks cb 0x5560b4003c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f101aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b4143da0 0x5560b4149660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f101aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b4143da0 0x5560b4149660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37898/0 (socket says 172.21.15.63:37898) 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f101aa00700 1 -- 172.21.15.63:0/2753171520 learned_addr learned my addr 172.21.15.63:0/2753171520 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.038+0000 7f101b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b3fbce40 0x5560b414e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.039+0000 7f101be00700 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b41462c0 0x5560b4149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.039+0000 7f101aa00700 1 -- 172.21.15.63:0/2753171520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b3fbce40 msgr2=0x5560b414e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.039+0000 7f101aa00700 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b3fbce40 0x5560b414e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.039+0000 7f101aa00700 1 -- 172.21.15.63:0/2753171520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b41462c0 msgr2=0x5560b4149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.039+0000 7f101aa00700 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b41462c0 0x5560b4149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.039+0000 7f101aa00700 1 -- 172.21.15.63:0/2753171520 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b414a0e0 con 0x5560b4143da0 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.039+0000 7f101b400700 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b3fbce40 0x5560b414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:02.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.039+0000 7f101be00700 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b41462c0 0x5560b4149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:02.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.039+0000 7f101aa00700 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b4143da0 0x5560b4149660 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x5560b4138970 tx=0x7f10080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f417c99d3b0fac8b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:02.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.039+0000 7f1005600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:02.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.040+0000 7f1005600700 1 -- 172.21.15.63:0/2753171520 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100802a7c0 con 0x5560b4143da0 2024-05-27T08:03:02.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.040+0000 7f1005600700 1 -- 172.21.15.63:0/2753171520 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9571+0+0 (secure 0 0 0) 0x7f100802a960 con 0x5560b4143da0 2024-05-27T08:03:02.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.040+0000 7f1030bd8b80 1 -- 172.21.15.63:0/2753171520 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1014007b80 con 0x5560b4143da0 2024-05-27T08:03:02.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.040+0000 7f1005600700 1 -- 172.21.15.63:0/2753171520 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f100803b050 con 0x5560b4143da0 2024-05-27T08:03:02.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.040+0000 7f1030bd8b80 10 client.21087 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:02.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.040+0000 7f1030bd8b80 1 -- 172.21.15.63:0/2753171520 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5560b41c0420 con 0x5560b4143da0 2024-05-27T08:03:02.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.041+0000 7f1005600700 1 -- 172.21.15.63:0/2753171520 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1014007b80 con 0x5560b4143da0 2024-05-27T08:03:02.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.041+0000 7f1005600700 1 -- 172.21.15.63:0/2753171520 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1008033420 con 0x5560b4143da0 2024-05-27T08:03:02.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.041+0000 7f1030bd8b80 10 client.21087 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:02.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.041+0000 7f1030bd8b80 10 client.21087 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:03:02.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.041+0000 7f1030bd8b80 20 client.21087 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:02.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.041+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5560b41c0b90 0x5560b41c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:02.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.041+0000 7f1030bd8b80 4 client.21087 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:03:02.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.041+0000 7f1030bd8b80 1 -- 172.21.15.63:0/2753171520 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560b4065570 con 0x5560b41c0b90 2024-05-27T08:03:02.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.042+0000 7f101b400700 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5560b41c0b90 0x5560b41c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:02.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.043+0000 7f101b400700 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5560b41c0b90 0x5560b41c2fb0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adabbd72959919ee server_cookie=ec16a2395392156d in_seq=0 out_seq=0 2024-05-27T08:03:02.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.043+0000 7f1005600700 10 client.21087 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:03:02.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.043+0000 7f1005600700 1 -- 172.21.15.63:0/2753171520 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560b4065570 con 0x5560b41c0b90 2024-05-27T08:03:02.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1005600700 10 client.21087 handle_command_reply: tid=0 2024-05-27T08:03:02.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 client.21087 shutdown 2024-05-27T08:03:02.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 -- 172.21.15.63:0/2753171520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b4143da0 msgr2=0x5560b4149660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:02.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b4143da0 0x5560b4149660 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x5560b4138970 tx=0x7f10080047c0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 -- 172.21.15.63:0/2753171520 shutdown_connections 2024-05-27T08:03:02.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560b41462c0 0x5560b4149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5560b3fbce40 0x5560b414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5560b41c0b90 0x5560b41c2fb0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 --2- 172.21.15.63:0/2753171520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5560b4143da0 0x5560b4149660 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1005600700 0 client.21087 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:03:02.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 -- 172.21.15.63:0/2753171520 >> 172.21.15.63:0/2753171520 conn(0x5560b4056390 msgr2=0x5560b413e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:02.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 -- 172.21.15.63:0/2753171520 shutdown_connections 2024-05-27T08:03:02.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.044+0000 7f1030bd8b80 1 -- 172.21.15.63:0/2753171520 wait complete. 2024-05-27T08:03:02.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.045+0000 7f1030bd8b80 20 client.21087 trim_cache size 0 max 16384 2024-05-27T08:03:02.048 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:03:02.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.045+0000 7f1020a00700 1 -- 172.21.15.63:0/2544873337 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10000b29f0 msgr2=0x7f10000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:02.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.045+0000 7f1020a00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10000b29f0 0x7f10000b4f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f101c116e30 tx=0x7f100800b410 comp rx=0 tx=0).stop 2024-05-27T08:03:02.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.046+0000 7f1020a00700 1 -- 172.21.15.63:0/2544873337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c075b50 msgr2=0x7f101c1a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:02.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.046+0000 7f1020a00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c075b50 0x7f101c1a11b0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f100c0047b0 tx=0x7f100c005e80 comp rx=0 tx=0).stop 2024-05-27T08:03:02.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.046+0000 7f1020a00700 1 -- 172.21.15.63:0/2544873337 shutdown_connections 2024-05-27T08:03:02.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.046+0000 7f1020a00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f10000b29f0 0x7f10000b4f00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.046+0000 7f1020a00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f101c115170 0x7f101c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.046+0000 7f1020a00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c075b50 0x7f101c1a11b0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.046+0000 7f1020a00700 1 --2- 172.21.15.63:0/2544873337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f101c0764a0 0x7f101c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:02.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.046+0000 7f1020a00700 1 -- 172.21.15.63:0/2544873337 >> 172.21.15.63:0/2544873337 conn(0x7f101c079d00 msgr2=0x7f101c101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:02.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.046+0000 7f1020a00700 1 -- 172.21.15.63:0/2544873337 shutdown_connections 2024-05-27T08:03:02.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:02.046+0000 7f1020a00700 1 -- 172.21.15.63:0/2544873337 wait complete. 2024-05-27T08:03:02.931 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:02 smithi063 ceph-mon[31613]: osdmap e141: 12 total, 12 up, 12 in 2024-05-27T08:03:02.934 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:03:02.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:02 smithi086 ceph-mon[34687]: osdmap e141: 12 total, 12 up, 12 in 2024-05-27T08:03:02.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:02 smithi149 ceph-mon[35408]: osdmap e141: 12 total, 12 up, 12 in 2024-05-27T08:03:03.456 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:03.939 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:03 smithi063 ceph-mon[31613]: pgmap v935: 129 pgs: 6 active+clean+snaptrim_wait, 123 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 34 op/s 2024-05-27T08:03:03.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:03 smithi086 ceph-mon[34687]: pgmap v935: 129 pgs: 6 active+clean+snaptrim_wait, 123 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 34 op/s 2024-05-27T08:03:03.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:03 smithi149 ceph-mon[35408]: pgmap v935: 129 pgs: 6 active+clean+snaptrim_wait, 123 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 34 op/s 2024-05-27T08:03:05.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.879+0000 7f5917200700 1 -- 172.21.15.63:0/43534084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5918106600 msgr2=0x7f5918106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:05.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.879+0000 7f5917200700 1 --2- 172.21.15.63:0/43534084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5918106600 0x7f5918106a10 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f59080099c0 tx=0x7f5908009cd0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.879+0000 7f5917200700 1 -- 172.21.15.63:0/43534084 shutdown_connections 2024-05-27T08:03:05.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.879+0000 7f5917200700 1 --2- 172.21.15.63:0/43534084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f591810b880 0x7f59181131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.879+0000 7f5917200700 1 --2- 172.21.15.63:0/43534084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5918106600 0x7f5918106a10 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.879+0000 7f5917200700 1 --2- 172.21.15.63:0/43534084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f591810af30 0x7f591810b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.879+0000 7f5917200700 1 -- 172.21.15.63:0/43534084 >> 172.21.15.63:0/43534084 conn(0x7f591807e070 msgr2=0x7f591807e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:05.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.879+0000 7f5917200700 1 -- 172.21.15.63:0/43534084 shutdown_connections 2024-05-27T08:03:05.882 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:05 smithi063 ceph-mon[31613]: pgmap v936: 129 pgs: 6 active+clean+snaptrim_wait, 123 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 34 op/s 2024-05-27T08:03:05.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.879+0000 7f5917200700 1 -- 172.21.15.63:0/43534084 wait complete. 2024-05-27T08:03:05.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.881+0000 7f5917200700 1 Processor -- start 2024-05-27T08:03:05.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.881+0000 7f5917200700 1 -- start start 2024-05-27T08:03:05.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.882+0000 7f5917200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5918106600 0x7f591819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.882+0000 7f5917200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f591810af30 0x7f591819d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.882+0000 7f5917200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f591810b880 0x7f59181a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.882+0000 7f5917200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59181163b0 con 0x7f591810b880 2024-05-27T08:03:05.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.882+0000 7f5917200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5918116230 con 0x7f5918106600 2024-05-27T08:03:05.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.882+0000 7f5917200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5918116530 con 0x7f591810af30 2024-05-27T08:03:05.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.882+0000 7f5916800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f591810b880 0x7f59181a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.882+0000 7f5915e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5918106600 0x7f591819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.882+0000 7f5916800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f591810b880 0x7f59181a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37910/0 (socket says 172.21.15.63:37910) 2024-05-27T08:03:05.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.882+0000 7f5916800700 1 -- 172.21.15.63:0/2433489993 learned_addr learned my addr 172.21.15.63:0/2433489993 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:05.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.883+0000 7f5915400700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f591810af30 0x7f591819d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.883+0000 7f5916800700 1 -- 172.21.15.63:0/2433489993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f591810af30 msgr2=0x7f591819d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:05.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.883+0000 7f5916800700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f591810af30 0x7f591819d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.883+0000 7f5916800700 1 -- 172.21.15.63:0/2433489993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5918106600 msgr2=0x7f591819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:05.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.883+0000 7f5916800700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5918106600 0x7f591819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.883+0000 7f5916800700 1 -- 172.21.15.63:0/2433489993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59181a1c60 con 0x7f591810b880 2024-05-27T08:03:05.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.884+0000 7f5916800700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f591810b880 0x7f59181a15c0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f5910007ee0 tx=0x7f591000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:05.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.884+0000 7f590ea00700 1 -- 172.21.15.63:0/2433489993 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910004bb0 con 0x7f591810b880 2024-05-27T08:03:05.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.884+0000 7f5917200700 1 -- 172.21.15.63:0/2433489993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59181a1f50 con 0x7f591810b880 2024-05-27T08:03:05.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.884+0000 7f590ea00700 1 -- 172.21.15.63:0/2433489993 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5910004d50 con 0x7f591810b880 2024-05-27T08:03:05.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.884+0000 7f590ea00700 1 -- 172.21.15.63:0/2433489993 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100058b0 con 0x7f591810b880 2024-05-27T08:03:05.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.884+0000 7f5917200700 1 -- 172.21.15.63:0/2433489993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59181a9870 con 0x7f591810b880 2024-05-27T08:03:05.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.890+0000 7f592b68eb80 1 Processor -- start 2024-05-27T08:03:05.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.890+0000 7f590ea00700 1 -- 172.21.15.63:0/2433489993 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f591001a020 con 0x7f591810b880 2024-05-27T08:03:05.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.890+0000 7f592b68eb80 1 -- start start 2024-05-27T08:03:05.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.890+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.890+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.890+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.890+0000 7f592b68eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a4acfeee40 con 0x55a4ad0c0410 2024-05-27T08:03:05.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.890+0000 7f592b68eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a4acfde9e0 con 0x55a4ad0c0820 2024-05-27T08:03:05.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.890+0000 7f592b68eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a4acfe2a20 con 0x55a4acf41e40 2024-05-27T08:03:05.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.891+0000 7f5915e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.891+0000 7f5915e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37926/0 (socket says 172.21.15.63:37926) 2024-05-27T08:03:05.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.891+0000 7f5915e00700 1 -- 172.21.15.63:0/1780736689 learned_addr learned my addr 172.21.15.63:0/1780736689 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.891+0000 7f5915400700 1 --2- 172.21.15.63:0/1780736689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.891+0000 7f5915400700 1 --2- 172.21.15.63:0/1780736689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.891+0000 7f5915e00700 1 -- 172.21.15.63:0/1780736689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 msgr2=0x55a4ad0bfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.891+0000 7f5915e00700 1 --2- 172.21.15.63:0/1780736689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0bfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.891+0000 7f5915e00700 1 -- 172.21.15.63:0/1780736689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4ad0c0820 msgr2=0x55a4ad0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.891+0000 7f5915e00700 1 --2- 172.21.15.63:0/1780736689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.891+0000 7f5915e00700 1 -- 172.21.15.63:0/1780736689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4acfa7420 con 0x55a4ad0c0410 2024-05-27T08:03:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.892+0000 7f5915400700 1 --2- 172.21.15.63:0/1780736689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.892+0000 7f5915e00700 1 --2- 172.21.15.63:0/1780736689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caad0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f59080058e0 tx=0x7f5908005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342c737ad3f20b5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.892+0000 7f590ea00700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f58fc0b29f0 0x7f58fc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.892+0000 7f590ea00700 1 -- 172.21.15.63:0/2433489993 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9571+0+0 (secure 0 0 0) 0x7f59100e20a0 con 0x7f591810b880 2024-05-27T08:03:05.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.892+0000 7f5903400700 1 -- 172.21.15.63:0/1780736689 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908027070 con 0x55a4ad0c0410 2024-05-27T08:03:05.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.892+0000 7f5915400700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f58fc0b29f0 0x7f58fc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.893+0000 7f5903400700 1 -- 172.21.15.63:0/1780736689 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59080054f0 con 0x55a4ad0c0410 2024-05-27T08:03:05.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.893+0000 7f5903400700 1 -- 172.21.15.63:0/1780736689 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908003990 con 0x55a4ad0c0410 2024-05-27T08:03:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.894+0000 7f592b68eb80 1 -- 172.21.15.63:0/1780736689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 msgr2=0x55a4ad0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.894+0000 7f5915400700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f58fc0b29f0 0x7f58fc0b4f00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55a4ad0c0cb0 tx=0x7f59040073d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.894+0000 7f592b68eb80 1 --2- 172.21.15.63:0/1780736689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caad0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f59080058e0 tx=0x7f5908005050 comp rx=0 tx=0).stop 2024-05-27T08:03:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.894+0000 7f592b68eb80 1 -- 172.21.15.63:0/1780736689 shutdown_connections 2024-05-27T08:03:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.894+0000 7f592b68eb80 1 --2- 172.21.15.63:0/1780736689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.894+0000 7f592b68eb80 1 --2- 172.21.15.63:0/1780736689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.894+0000 7f592b68eb80 1 --2- 172.21.15.63:0/1780736689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caad0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.894+0000 7f592b68eb80 1 -- 172.21.15.63:0/1780736689 >> 172.21.15.63:0/1780736689 conn(0x55a4acfa1fc0 msgr2=0x55a4ad0bbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:05.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.894+0000 7f592b68eb80 1 -- 172.21.15.63:0/1780736689 shutdown_connections 2024-05-27T08:03:05.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.894+0000 7f592b68eb80 1 -- 172.21.15.63:0/1780736689 wait complete. 2024-05-27T08:03:05.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.895+0000 7f592b68eb80 1 -- start start 2024-05-27T08:03:05.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.895+0000 7f592b68eb80 1 Processor -- start 2024-05-27T08:03:05.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.896+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acf41e40 0x55a4ad0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.896+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.896+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.896+0000 7f592b68eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a4acee4fa0 con 0x55a4ad0c0820 2024-05-27T08:03:05.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.896+0000 7f592b68eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a4acf7f760 con 0x55a4acf41e40 2024-05-27T08:03:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.896+0000 7f592b68eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a4ad0cd930 con 0x55a4ad0c0410 2024-05-27T08:03:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.896+0000 7f592b68eb80 10 client.0 _ll_register_callbacks cb 0x55a4acf88c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.896+0000 7f5916800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.897+0000 7f5916800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37936/0 (socket says 172.21.15.63:37936) 2024-05-27T08:03:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.897+0000 7f5916800700 1 -- 172.21.15.63:0/697294683 learned_addr learned my addr 172.21.15.63:0/697294683 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.897+0000 7f5915400700 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.897+0000 7f5915e00700 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acf41e40 0x55a4ad0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.897+0000 7f5916800700 1 -- 172.21.15.63:0/697294683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4ad0c0410 msgr2=0x55a4ad0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:05.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.897+0000 7f5916800700 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.897+0000 7f5916800700 1 -- 172.21.15.63:0/697294683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acf41e40 msgr2=0x55a4ad0ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.897+0000 7f5916800700 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acf41e40 0x55a4ad0ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.897+0000 7f5916800700 1 -- 172.21.15.63:0/697294683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4acf2e8d0 con 0x55a4ad0c0820 2024-05-27T08:03:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.897+0000 7f5915400700 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.898+0000 7f5916800700 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cafe0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f5910019ff0 tx=0x7f5910005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d0246228c8fe40 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.898+0000 7f58f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.898+0000 7f58f7e00700 1 -- 172.21.15.63:0/697294683 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100a8f60 con 0x55a4ad0c0820 2024-05-27T08:03:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.898+0000 7f58f7e00700 1 -- 172.21.15.63:0/697294683 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9571+0+0 (secure 0 0 0) 0x7f59100a91c0 con 0x55a4ad0c0820 2024-05-27T08:03:05.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.898+0000 7f592b68eb80 1 -- 172.21.15.63:0/697294683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59080054f0 con 0x55a4ad0c0820 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.898+0000 7f58f7e00700 1 -- 172.21.15.63:0/697294683 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f59100ad310 con 0x55a4ad0c0820 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.899+0000 7f592b68eb80 10 client.21099 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.899+0000 7f592b68eb80 1 -- 172.21.15.63:0/697294683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a4acfaf060 con 0x55a4ad0c0820 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.899+0000 7f58f7e00700 1 -- 172.21.15.63:0/697294683 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59080054f0 con 0x55a4ad0c0820 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.899+0000 7f58f7e00700 1 -- 172.21.15.63:0/697294683 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f59100ac070 con 0x55a4ad0c0820 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.899+0000 7f592b68eb80 10 client.21099 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.899+0000 7f592b68eb80 10 client.21099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.900+0000 7f592b68eb80 20 client.21099 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.900+0000 7f592b68eb80 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4ad0da820 0x55a4ad0dcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.900+0000 7f592b68eb80 4 client.21099 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:05.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.900+0000 7f592b68eb80 1 -- 172.21.15.63:0/697294683 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4acff0d90 con 0x55a4ad0da820 2024-05-27T08:03:05.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.900+0000 7f5915e00700 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4ad0da820 0x55a4ad0dcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.901+0000 7f5915e00700 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4ad0da820 0x55a4ad0dcc60 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed66555e7d47ce77 server_cookie=3bd26e82796179e2 in_seq=0 out_seq=0 2024-05-27T08:03:05.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.901+0000 7f58f7e00700 10 client.21099 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:05.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.903+0000 7f58f7e00700 1 -- 172.21.15.63:0/697294683 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a4acff0d90 con 0x55a4ad0da820 2024-05-27T08:03:05.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.903+0000 7f58f7e00700 10 client.21099 handle_command_reply: tid=0 2024-05-27T08:03:05.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.903+0000 7f592b68eb80 1 client.21099 shutdown 2024-05-27T08:03:05.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 1 -- 172.21.15.63:0/697294683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0820 msgr2=0x55a4ad0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:05.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cafe0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f5910019ff0 tx=0x7f5910005250 comp rx=0 tx=0).stop 2024-05-27T08:03:05.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 1 -- 172.21.15.63:0/697294683 shutdown_connections 2024-05-27T08:03:05.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4ad0da820 0x55a4ad0dcc60 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0820 0x55a4ad0cafe0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acf41e40 0x55a4ad0ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 1 --2- 172.21.15.63:0/697294683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 1 -- 172.21.15.63:0/697294683 >> 172.21.15.63:0/697294683 conn(0x55a4acfa1fc0 msgr2=0x55a4ad0baff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:05.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f58f7e00700 0 client.21099 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:05.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 1 -- 172.21.15.63:0/697294683 shutdown_connections 2024-05-27T08:03:05.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 1 -- 172.21.15.63:0/697294683 wait complete. 2024-05-27T08:03:05.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.904+0000 7f592b68eb80 20 client.21099 trim_cache size 0 max 16384 2024-05-27T08:03:05.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.925+0000 7f592b68eb80 1 Processor -- start 2024-05-27T08:03:05.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.925+0000 7f592b68eb80 1 -- start start 2024-05-27T08:03:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.925+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acfa1fc0 0x55a4ad0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.925+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.925+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.925+0000 7f592b68eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b4fc0 con 0x55a4ad0c0410 2024-05-27T08:03:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.925+0000 7f592b68eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b2850 con 0x55a4acfa1fc0 2024-05-27T08:03:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.925+0000 7f592b68eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b06c0 con 0x55a4acf41e40 2024-05-27T08:03:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.926+0000 7f5916800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.926+0000 7f5916800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37950/0 (socket says 172.21.15.63:37950) 2024-05-27T08:03:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.926+0000 7f5916800700 1 -- 172.21.15.63:0/170653283 learned_addr learned my addr 172.21.15.63:0/170653283 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.926+0000 7f5915400700 1 --2- 172.21.15.63:0/170653283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.926+0000 7f5915e00700 1 --2- 172.21.15.63:0/170653283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acfa1fc0 0x55a4ad0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.926+0000 7f5916800700 1 -- 172.21.15.63:0/170653283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 msgr2=0x55a4ad0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.926+0000 7f5916800700 1 --2- 172.21.15.63:0/170653283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.926+0000 7f5916800700 1 -- 172.21.15.63:0/170653283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acfa1fc0 msgr2=0x55a4ad0cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.926+0000 7f5916800700 1 --2- 172.21.15.63:0/170653283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acfa1fc0 0x55a4ad0cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.926+0000 7f5916800700 1 -- 172.21.15.63:0/170653283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4ad0cb620 con 0x55a4ad0c0410 2024-05-27T08:03:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.927+0000 7f5916800700 1 --2- 172.21.15.63:0/170653283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caa50 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f59100a8ef0 tx=0x7f59100a86d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdca1ccbe99332 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.927+0000 7f5902a00700 1 -- 172.21.15.63:0/170653283 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100a8f60 con 0x55a4ad0c0410 2024-05-27T08:03:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.927+0000 7f5902a00700 1 -- 172.21.15.63:0/170653283 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59100b1dc0 con 0x55a4ad0c0410 2024-05-27T08:03:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.927+0000 7f5902a00700 1 -- 172.21.15.63:0/170653283 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100a9420 con 0x55a4ad0c0410 2024-05-27T08:03:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.928+0000 7f592b68eb80 1 -- 172.21.15.63:0/170653283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 msgr2=0x55a4ad0caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.928+0000 7f592b68eb80 1 --2- 172.21.15.63:0/170653283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caa50 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f59100a8ef0 tx=0x7f59100a86d0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.928+0000 7f592b68eb80 1 -- 172.21.15.63:0/170653283 shutdown_connections 2024-05-27T08:03:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.928+0000 7f592b68eb80 1 --2- 172.21.15.63:0/170653283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0caa50 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.928+0000 7f592b68eb80 1 --2- 172.21.15.63:0/170653283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acfa1fc0 0x55a4ad0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.928+0000 7f592b68eb80 1 --2- 172.21.15.63:0/170653283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.928+0000 7f592b68eb80 1 -- 172.21.15.63:0/170653283 >> 172.21.15.63:0/170653283 conn(0x55a4ad0b4f60 msgr2=0x55a4ad0ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.929+0000 7f592b68eb80 1 -- 172.21.15.63:0/170653283 shutdown_connections 2024-05-27T08:03:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.929+0000 7f592b68eb80 1 -- 172.21.15.63:0/170653283 wait complete. 2024-05-27T08:03:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.929+0000 7f592b68eb80 1 -- start start 2024-05-27T08:03:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.929+0000 7f592b68eb80 1 Processor -- start 2024-05-27T08:03:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.930+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0d3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.930+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acfa1fc0 0x55a4ad0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.930+0000 7f592b68eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.930+0000 7f592b68eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b0d10 con 0x55a4ad0c0410 2024-05-27T08:03:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.930+0000 7f592b68eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b2180 con 0x55a4acfa1fc0 2024-05-27T08:03:05.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.930+0000 7f592b68eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b13e0 con 0x55a4acf41e40 2024-05-27T08:03:05.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.930+0000 7f592b68eb80 10 client.0 _ll_register_callbacks cb 0x55a4acf88c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:05.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.931+0000 7f5916800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.931+0000 7f5916800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37956/0 (socket says 172.21.15.63:37956) 2024-05-27T08:03:05.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.931+0000 7f5916800700 1 -- 172.21.15.63:0/2376896280 learned_addr learned my addr 172.21.15.63:0/2376896280 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:05.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.931+0000 7f5915400700 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acfa1fc0 0x55a4ad0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.931+0000 7f5915e00700 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0d3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.931+0000 7f5916800700 1 -- 172.21.15.63:0/2376896280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 msgr2=0x55a4ad0d3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:05.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.931+0000 7f5916800700 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0d3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.931+0000 7f5916800700 1 -- 172.21.15.63:0/2376896280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acfa1fc0 msgr2=0x55a4ad0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.931+0000 7f5916800700 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acfa1fc0 0x55a4ad0ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.931+0000 7f5916800700 1 -- 172.21.15.63:0/2376896280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ad0cf1d0 con 0x55a4ad0c0410 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.932+0000 7f5916800700 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0ceaf0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f5910005290 tx=0x7f59100a83f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c099c779dd8a2bab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.932+0000 7f58f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.932+0000 7f58f7e00700 1 -- 172.21.15.63:0/2376896280 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100c7030 con 0x55a4ad0c0410 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.932+0000 7f58f7e00700 1 -- 172.21.15.63:0/2376896280 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9571+0+0 (secure 0 0 0) 0x7f59100a5550 con 0x55a4ad0c0410 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.932+0000 7f592b68eb80 1 -- 172.21.15.63:0/2376896280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59100b1dc0 con 0x55a4ad0c0410 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.932+0000 7f58f7e00700 1 -- 172.21.15.63:0/2376896280 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f59100a8f60 con 0x55a4ad0c0410 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.932+0000 7f592b68eb80 10 client.21105 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.932+0000 7f592b68eb80 1 -- 172.21.15.63:0/2376896280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a4ad145160 con 0x55a4ad0c0410 2024-05-27T08:03:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.933+0000 7f58f7e00700 1 -- 172.21.15.63:0/2376896280 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59100b1dc0 con 0x55a4ad0c0410 2024-05-27T08:03:05.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.933+0000 7f58f7e00700 1 -- 172.21.15.63:0/2376896280 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f59100ac070 con 0x55a4ad0c0410 2024-05-27T08:03:05.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.933+0000 7f592b68eb80 10 client.21105 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:05.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.933+0000 7f592b68eb80 10 client.21105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:05.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.933+0000 7f592b68eb80 20 client.21105 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:05.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.933+0000 7f592b68eb80 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4ad0c0820 0x55a4ad1475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:05.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.933+0000 7f592b68eb80 4 client.21105 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:03:05.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.933+0000 7f592b68eb80 1 -- 172.21.15.63:0/2376896280 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4acfea570 con 0x55a4ad0c0820 2024-05-27T08:03:05.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.934+0000 7f5915e00700 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4ad0c0820 0x55a4ad1475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:05.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.935+0000 7f5915e00700 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4ad0c0820 0x55a4ad1475d0 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fafafaaafad5ca02 server_cookie=94d431f81aeabde4 in_seq=0 out_seq=0 2024-05-27T08:03:05.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.935+0000 7f58f7e00700 10 client.21105 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:05.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.935+0000 7f58f7e00700 1 -- 172.21.15.63:0/2376896280 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4acfea570 con 0x55a4ad0c0820 2024-05-27T08:03:05.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.935+0000 7f58f7e00700 10 client.21105 handle_command_reply: tid=0 2024-05-27T08:03:05.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.935+0000 7f592b68eb80 1 client.21105 shutdown 2024-05-27T08:03:05.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.935+0000 7f592b68eb80 1 -- 172.21.15.63:0/2376896280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 msgr2=0x55a4ad0ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:05.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.935+0000 7f592b68eb80 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0ceaf0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f5910005290 tx=0x7f59100a83f0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.936+0000 7f592b68eb80 1 -- 172.21.15.63:0/2376896280 shutdown_connections 2024-05-27T08:03:05.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.936+0000 7f592b68eb80 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a4ad0c0820 0x55a4ad1475d0 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.936+0000 7f592b68eb80 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a4ad0c0410 0x55a4ad0ceaf0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.936+0000 7f592b68eb80 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a4acf41e40 0x55a4ad0d3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.936+0000 7f592b68eb80 1 --2- 172.21.15.63:0/2376896280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a4acfa1fc0 0x55a4ad0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.936+0000 7f592b68eb80 1 -- 172.21.15.63:0/2376896280 >> 172.21.15.63:0/2376896280 conn(0x55a4ad0b4f60 msgr2=0x55a4ad0cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:05.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.936+0000 7f58f7e00700 0 client.21105 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:05.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.936+0000 7f592b68eb80 1 -- 172.21.15.63:0/2376896280 shutdown_connections 2024-05-27T08:03:05.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.936+0000 7f592b68eb80 1 -- 172.21.15.63:0/2376896280 wait complete. 2024-05-27T08:03:05.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.936+0000 7f592b68eb80 20 client.21105 trim_cache size 0 max 16384 2024-05-27T08:03:05.939 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 -- 172.21.15.63:0/2433489993 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f58fc0b29f0 msgr2=0x7f58fc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f58fc0b29f0 0x7f58fc0b4f00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55a4ad0c0cb0 tx=0x7f59040073d0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 -- 172.21.15.63:0/2433489993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f591810b880 msgr2=0x7f59181a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f591810b880 0x7f59181a15c0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f5910007ee0 tx=0x7f591000d3b0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 -- 172.21.15.63:0/2433489993 shutdown_connections 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f58fc0b29f0 0x7f58fc0b4f00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f591810b880 0x7f59181a15c0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5918106600 0x7f591819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 --2- 172.21.15.63:0/2433489993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f591810af30 0x7f591819d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 -- 172.21.15.63:0/2433489993 >> 172.21.15.63:0/2433489993 conn(0x7f591807e070 msgr2=0x7f5918103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 -- 172.21.15.63:0/2433489993 shutdown_connections 2024-05-27T08:03:05.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:05.937+0000 7f5917200700 1 -- 172.21.15.63:0/2433489993 wait complete. 2024-05-27T08:03:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:05 smithi086 ceph-mon[34687]: pgmap v936: 129 pgs: 6 active+clean+snaptrim_wait, 123 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 34 op/s 2024-05-27T08:03:05.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:05 smithi149 ceph-mon[35408]: pgmap v936: 129 pgs: 6 active+clean+snaptrim_wait, 123 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 34 op/s 2024-05-27T08:03:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.633+0000 7f424a800640 1 -- 172.21.15.63:0/3726380995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f424410b610 msgr2=0x7f424410da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.633+0000 7f424a800640 1 --2- 172.21.15.63:0/3726380995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f424410b610 0x7f424410da30 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f4230009900 tx=0x7f423002a3f0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.633+0000 7f424a800640 1 -- 172.21.15.63:0/3726380995 shutdown_connections 2024-05-27T08:03:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.633+0000 7f424a800640 1 --2- 172.21.15.63:0/3726380995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f424410e000 0x7f42441104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.633+0000 7f424a800640 1 --2- 172.21.15.63:0/3726380995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f424410b610 0x7f424410da30 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.633+0000 7f424a800640 1 --2- 172.21.15.63:0/3726380995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4244104bb0 0x7f4244104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.633+0000 7f424a800640 1 -- 172.21.15.63:0/3726380995 >> 172.21.15.63:0/3726380995 conn(0x7f42440fe210 msgr2=0x7f4244100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.633+0000 7f424a800640 1 -- 172.21.15.63:0/3726380995 shutdown_connections 2024-05-27T08:03:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.633+0000 7f424a800640 1 -- 172.21.15.63:0/3726380995 wait complete. 2024-05-27T08:03:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.635+0000 7f424a800640 1 Processor -- start 2024-05-27T08:03:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.635+0000 7f424a800640 1 -- start start 2024-05-27T08:03:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.635+0000 7f424a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4244104bb0 0x7f424419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.635+0000 7f424a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f424410b610 0x7f424419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.635+0000 7f424a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f424410e000 0x7f424419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.635+0000 7f424a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4244113b10 con 0x7f4244104bb0 2024-05-27T08:03:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.635+0000 7f424a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4244113990 con 0x7f424410e000 2024-05-27T08:03:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.635+0000 7f424a800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4244113c90 con 0x7f424410b610 2024-05-27T08:03:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.636+0000 7f4249400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4244104bb0 0x7f424419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.636+0000 7f4249400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4244104bb0 0x7f424419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37976/0 (socket says 172.21.15.63:37976) 2024-05-27T08:03:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.636+0000 7f4249400640 1 -- 172.21.15.63:0/2178286443 learned_addr learned my addr 172.21.15.63:0/2178286443 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.636+0000 7f4249e00640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f424410e000 0x7f424419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.636+0000 7f4248a00640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f424410b610 0x7f424419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.636+0000 7f4249400640 1 -- 172.21.15.63:0/2178286443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f424410b610 msgr2=0x7f424419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.636+0000 7f4249400640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f424410b610 0x7f424419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.636+0000 7f4249400640 1 -- 172.21.15.63:0/2178286443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f424410e000 msgr2=0x7f424419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.636+0000 7f4249400640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f424410e000 0x7f424419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.636+0000 7f4249400640 1 -- 172.21.15.63:0/2178286443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424419f3d0 con 0x7f4244104bb0 2024-05-27T08:03:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.637+0000 7f4249400640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4244104bb0 0x7f424419a570 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f423800d950 tx=0x7f423800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.637+0000 7f4242000640 1 -- 172.21.15.63:0/2178286443 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238017030 con 0x7f4244104bb0 2024-05-27T08:03:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.637+0000 7f4242000640 1 -- 172.21.15.63:0/2178286443 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4238004020 con 0x7f4244104bb0 2024-05-27T08:03:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.637+0000 7f424a800640 1 -- 172.21.15.63:0/2178286443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424419f6c0 con 0x7f4244104bb0 2024-05-27T08:03:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.637+0000 7f4242000640 1 -- 172.21.15.63:0/2178286443 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380055d0 con 0x7f4244104bb0 2024-05-27T08:03:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.637+0000 7f424a800640 1 -- 172.21.15.63:0/2178286443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42441a7040 con 0x7f4244104bb0 2024-05-27T08:03:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.641+0000 7f4252de4740 1 Processor -- start 2024-05-27T08:03:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.641+0000 7f4252de4740 1 -- start start 2024-05-27T08:03:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.641+0000 7f4252de4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d6429930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.641+0000 7f4252de4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6429e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.641+0000 7f4252de4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d642a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.641+0000 7f4252de4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5599d639ab90 con 0x5599d605e2f0 2024-05-27T08:03:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.641+0000 7f4252de4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599d638a730 con 0x5599d60871b0 2024-05-27T08:03:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.642+0000 7f4252de4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5599d638e770 con 0x5599d60664d0 2024-05-27T08:03:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.642+0000 7f4249e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d6429930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.642+0000 7f4249e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d6429930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:49466/0 (socket says 172.21.15.63:49466) 2024-05-27T08:03:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.642+0000 7f4249e00640 1 -- 172.21.15.63:0/2388691807 learned_addr learned my addr 172.21.15.63:0/2388691807 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.642+0000 7f4248a00640 1 --2- 172.21.15.63:0/2388691807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d642a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.643+0000 7f4249e00640 1 -- 172.21.15.63:0/2388691807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 msgr2=0x5599d642a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.643+0000 7f4249e00640 1 --2- 172.21.15.63:0/2388691807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d642a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.643+0000 7f4249e00640 1 -- 172.21.15.63:0/2388691807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 msgr2=0x5599d6429e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:03:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.643+0000 7f4249e00640 1 --2- 172.21.15.63:0/2388691807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6429e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.643+0000 7f4249e00640 1 -- 172.21.15.63:0/2388691807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d63238b0 con 0x5599d60871b0 2024-05-27T08:03:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.643+0000 7f4242000640 1 -- 172.21.15.63:0/2178286443 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4238009a30 con 0x7f4244104bb0 2024-05-27T08:03:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.643+0000 7f4249e00640 1 --2- 172.21.15.63:0/2388691807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d6429930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f423c009e90 tx=0x7f423c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d05335eecccd2a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.644+0000 7f4236600640 1 -- 172.21.15.63:0/2388691807 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c016070 con 0x5599d60871b0 2024-05-27T08:03:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.644+0000 7f4236600640 1 -- 172.21.15.63:0/2388691807 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f423c004d20 con 0x5599d60871b0 2024-05-27T08:03:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.644+0000 7f4236600640 1 -- 172.21.15.63:0/2388691807 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c0126b0 con 0x5599d60871b0 2024-05-27T08:03:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.645+0000 7f4242000640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f422c0b29d0 0x7f422c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.645+0000 7f4252de4740 1 -- 172.21.15.63:0/2388691807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 msgr2=0x5599d6429930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.645+0000 7f4252de4740 1 --2- 172.21.15.63:0/2388691807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d6429930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f423c009e90 tx=0x7f423c010740 comp rx=0 tx=0).stop 2024-05-27T08:03:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.646+0000 7f4252de4740 1 -- 172.21.15.63:0/2388691807 shutdown_connections 2024-05-27T08:03:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.646+0000 7f4252de4740 1 --2- 172.21.15.63:0/2388691807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d642a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.646+0000 7f4252de4740 1 --2- 172.21.15.63:0/2388691807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6429e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.646+0000 7f4252de4740 1 --2- 172.21.15.63:0/2388691807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d6429930 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.646+0000 7f4252de4740 1 -- 172.21.15.63:0/2388691807 >> 172.21.15.63:0/2388691807 conn(0x5599d607b300 msgr2=0x5599d6426210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.646+0000 7f4242000640 1 -- 172.21.15.63:0/2178286443 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9571+0+0 (secure 0 0 0) 0x7f42380e2490 con 0x7f4244104bb0 2024-05-27T08:03:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.646+0000 7f4249400640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f422c0b29d0 0x7f422c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.646+0000 7f4252de4740 1 -- 172.21.15.63:0/2388691807 shutdown_connections 2024-05-27T08:03:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.646+0000 7f4252de4740 1 -- 172.21.15.63:0/2388691807 wait complete. 2024-05-27T08:03:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.647+0000 7f4252de4740 1 -- start start 2024-05-27T08:03:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.647+0000 7f4249400640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f422c0b29d0 0x7f422c0b4ee0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x5599d64272c0 tx=0x7f4238091050 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.647+0000 7f4252de4740 1 Processor -- start 2024-05-27T08:03:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4252de4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6437fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4252de4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d6438520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4252de4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d6444380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4252de4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5599d6070790 con 0x5599d605e2f0 2024-05-27T08:03:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4252de4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599d6332bc0 con 0x5599d60871b0 2024-05-27T08:03:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4252de4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5599d643b050 con 0x5599d60664d0 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4252de4740 10 client.0 _ll_register_callbacks cb 0x5599d6310400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4249e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6437fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4249e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6437fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37988/0 (socket says 172.21.15.63:37988) 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4249e00640 1 -- 172.21.15.63:0/2450735364 learned_addr learned my addr 172.21.15.63:0/2450735364 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.648+0000 7f4248a00640 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d6438520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.649+0000 7f4249e00640 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d6444380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.649+0000 7f4249e00640 1 -- 172.21.15.63:0/2450735364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 msgr2=0x5599d6438520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.649+0000 7f4249e00640 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d6438520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.649+0000 7f4249e00640 1 -- 172.21.15.63:0/2450735364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 msgr2=0x5599d6444380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.649+0000 7f4249e00640 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d6444380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.649+0000 7f4249e00640 1 -- 172.21.15.63:0/2450735364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d609f210 con 0x5599d605e2f0 2024-05-27T08:03:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.649+0000 7f4249e00640 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6437fe0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x5599d64218f0 tx=0x7f423c010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b301b6741edede56 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.649+0000 7f422aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.650+0000 7f422aa00640 1 -- 172.21.15.63:0/2450735364 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c016070 con 0x5599d605e2f0 2024-05-27T08:03:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.650+0000 7f4252de4740 1 -- 172.21.15.63:0/2450735364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f423c004d20 con 0x5599d605e2f0 2024-05-27T08:03:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.650+0000 7f422aa00640 1 -- 172.21.15.63:0/2450735364 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9571+0+0 (secure 0 0 0) 0x7f423c005680 con 0x5599d605e2f0 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.650+0000 7f422aa00640 1 -- 172.21.15.63:0/2450735364 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f423c012450 con 0x5599d605e2f0 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.650+0000 7f4252de4740 10 client.21114 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.650+0000 7f4252de4740 1 -- 172.21.15.63:0/2450735364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599d616b620 con 0x5599d605e2f0 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.651+0000 7f422aa00640 1 -- 172.21.15.63:0/2450735364 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f423c004d20 con 0x5599d605e2f0 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.651+0000 7f422aa00640 1 -- 172.21.15.63:0/2450735364 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f423c00f040 con 0x5599d605e2f0 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.651+0000 7f4252de4740 10 client.21114 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.651+0000 7f4252de4740 10 client.21114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.651+0000 7f4252de4740 20 client.21114 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.651+0000 7f4252de4740 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599d6045d80 0x5599d644a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.651+0000 7f4252de4740 4 client.21114 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.651+0000 7f4252de4740 1 -- 172.21.15.63:0/2450735364 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599d639cae0 con 0x5599d6045d80 2024-05-27T08:03:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.652+0000 7f4248a00640 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599d6045d80 0x5599d644a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.653+0000 7f4248a00640 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599d6045d80 0x5599d644a1a0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9d4d2df7f89301 server_cookie=34b08e0493b61425 in_seq=0 out_seq=0 2024-05-27T08:03:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.653+0000 7f422aa00640 10 client.21114 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f422aa00640 1 -- 172.21.15.63:0/2450735364 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5599d639cae0 con 0x5599d6045d80 2024-05-27T08:03:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f422aa00640 10 client.21114 handle_command_reply: tid=0 2024-05-27T08:03:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 client.21114 shutdown 2024-05-27T08:03:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 -- 172.21.15.63:0/2450735364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 msgr2=0x5599d6437fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6437fe0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x5599d64218f0 tx=0x7f423c010ef0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 -- 172.21.15.63:0/2450735364 shutdown_connections 2024-05-27T08:03:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599d6045d80 0x5599d644a1a0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d6444380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d6438520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 --2- 172.21.15.63:0/2450735364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6437fe0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f422aa00640 0 client.21114 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 -- 172.21.15.63:0/2450735364 >> 172.21.15.63:0/2450735364 conn(0x5599d607b300 msgr2=0x5599d642c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 -- 172.21.15.63:0/2450735364 shutdown_connections 2024-05-27T08:03:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.655+0000 7f4252de4740 1 -- 172.21.15.63:0/2450735364 wait complete. 2024-05-27T08:03:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.656+0000 7f4252de4740 20 client.21114 trim_cache size 0 max 16384 2024-05-27T08:03:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.678+0000 7f4252de4740 1 Processor -- start 2024-05-27T08:03:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.678+0000 7f4252de4740 1 -- start start 2024-05-27T08:03:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.679+0000 7f4252de4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6433800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.679+0000 7f4252de4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60664d0 0x5599d643a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.679+0000 7f4252de4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60871b0 0x5599d64297f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.679+0000 7f4252de4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42441b1870 con 0x5599d605e2f0 2024-05-27T08:03:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.679+0000 7f4252de4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42441af140 con 0x5599d60664d0 2024-05-27T08:03:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.679+0000 7f4252de4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42441acfb0 con 0x5599d60871b0 2024-05-27T08:03:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.679+0000 7f4249e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6433800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.679+0000 7f4249e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6433800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37998/0 (socket says 172.21.15.63:37998) 2024-05-27T08:03:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.679+0000 7f4249e00640 1 -- 172.21.15.63:0/2234982398 learned_addr learned my addr 172.21.15.63:0/2234982398 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.680+0000 7f4248a00640 1 --2- 172.21.15.63:0/2234982398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60664d0 0x5599d643a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.680+0000 7f4249e00640 1 --2- 172.21.15.63:0/2234982398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60871b0 0x5599d64297f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.680+0000 7f4249e00640 1 -- 172.21.15.63:0/2234982398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60871b0 msgr2=0x5599d64297f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.680+0000 7f4249e00640 1 --2- 172.21.15.63:0/2234982398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60871b0 0x5599d64297f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.680+0000 7f4249e00640 1 -- 172.21.15.63:0/2234982398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60664d0 msgr2=0x5599d643a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.680+0000 7f4249e00640 1 --2- 172.21.15.63:0/2234982398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60664d0 0x5599d643a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.680+0000 7f4249e00640 1 -- 172.21.15.63:0/2234982398 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599d6436530 con 0x5599d605e2f0 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.680+0000 7f4248a00640 1 --2- 172.21.15.63:0/2234982398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60664d0 0x5599d643a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.680+0000 7f4249e00640 1 --2- 172.21.15.63:0/2234982398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6433800 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f423c006010 tx=0x7f423c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbab3b4d7edf071 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.681+0000 7f4235c00640 1 -- 172.21.15.63:0/2234982398 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c03a070 con 0x5599d605e2f0 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.681+0000 7f4235c00640 1 -- 172.21.15.63:0/2234982398 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f423c024040 con 0x5599d605e2f0 2024-05-27T08:03:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.681+0000 7f4235c00640 1 -- 172.21.15.63:0/2234982398 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c016070 con 0x5599d605e2f0 2024-05-27T08:03:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.682+0000 7f4252de4740 1 -- 172.21.15.63:0/2234982398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 msgr2=0x5599d6433800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.682+0000 7f4252de4740 1 --2- 172.21.15.63:0/2234982398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6433800 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f423c006010 tx=0x7f423c009f80 comp rx=0 tx=0).stop 2024-05-27T08:03:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.682+0000 7f4252de4740 1 -- 172.21.15.63:0/2234982398 shutdown_connections 2024-05-27T08:03:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.682+0000 7f4252de4740 1 --2- 172.21.15.63:0/2234982398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60871b0 0x5599d64297f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.682+0000 7f4252de4740 1 --2- 172.21.15.63:0/2234982398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60664d0 0x5599d643a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.682+0000 7f4252de4740 1 --2- 172.21.15.63:0/2234982398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6433800 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.682+0000 7f4252de4740 1 -- 172.21.15.63:0/2234982398 >> 172.21.15.63:0/2234982398 conn(0x5599d607b300 msgr2=0x5599d642c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.682+0000 7f4252de4740 1 -- 172.21.15.63:0/2234982398 shutdown_connections 2024-05-27T08:03:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.682+0000 7f4252de4740 1 -- 172.21.15.63:0/2234982398 wait complete. 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.683+0000 7f4252de4740 1 -- start start 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.683+0000 7f4252de4740 1 Processor -- start 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.683+0000 7f4252de4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6448270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.683+0000 7f4252de4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d64487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.683+0000 7f4252de4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d64b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.683+0000 7f4252de4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42441ad600 con 0x5599d605e2f0 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.683+0000 7f4252de4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42441aea70 con 0x5599d60871b0 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.683+0000 7f4252de4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42441adcd0 con 0x5599d60664d0 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.683+0000 7f4252de4740 10 client.0 _ll_register_callbacks cb 0x7f4230009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.684+0000 7f4249e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6448270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.684+0000 7f4249e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6448270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38000/0 (socket says 172.21.15.63:38000) 2024-05-27T08:03:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.684+0000 7f4249e00640 1 -- 172.21.15.63:0/3595321460 learned_addr learned my addr 172.21.15.63:0/3595321460 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.684+0000 7f4248a00640 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d64487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.684+0000 7f4249e00640 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d64b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.684+0000 7f4249e00640 1 -- 172.21.15.63:0/3595321460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 msgr2=0x5599d64487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.684+0000 7f4249e00640 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d64487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.684+0000 7f4249e00640 1 -- 172.21.15.63:0/3595321460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 msgr2=0x5599d64b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.684+0000 7f4249e00640 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d64b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.684+0000 7f4249e00640 1 -- 172.21.15.63:0/3595321460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599d6449050 con 0x5599d605e2f0 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.685+0000 7f4249e00640 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6448270 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f423c002230 tx=0x7f423c010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6c0893dd9f9f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.685+0000 7f422aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.685+0000 7f422aa00640 1 -- 172.21.15.63:0/3595321460 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c016070 con 0x5599d605e2f0 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.685+0000 7f422aa00640 1 -- 172.21.15.63:0/3595321460 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9571+0+0 (secure 0 0 0) 0x7f423c005480 con 0x5599d605e2f0 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.685+0000 7f4252de4740 1 -- 172.21.15.63:0/3595321460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f423c024040 con 0x5599d605e2f0 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f422aa00640 1 -- 172.21.15.63:0/3595321460 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f423c045050 con 0x5599d605e2f0 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f4252de4740 10 client.21120 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f4252de4740 1 -- 172.21.15.63:0/3595321460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599d64b1b70 con 0x5599d605e2f0 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f422aa00640 1 -- 172.21.15.63:0/3595321460 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f423c024040 con 0x5599d605e2f0 2024-05-27T08:03:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f422aa00640 1 -- 172.21.15.63:0/3595321460 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f423c012c20 con 0x5599d605e2f0 2024-05-27T08:03:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f4252de4740 10 client.21120 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f4252de4740 10 client.21120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f4252de4740 20 client.21120 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f4252de4740 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599d601b1c0 0x5599d64b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f4252de4740 4 client.21120 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:03:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.686+0000 7f4252de4740 1 -- 172.21.15.63:0/3595321460 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599d63962c0 con 0x5599d601b1c0 2024-05-27T08:03:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.687+0000 7f4248a00640 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599d601b1c0 0x5599d64b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.688+0000 7f4248a00640 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599d601b1c0 0x5599d64b4190 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a473383f7f1976bb server_cookie=c6d29ab93e10c0e8 in_seq=0 out_seq=0 2024-05-27T08:03:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.688+0000 7f422aa00640 10 client.21120 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f422aa00640 1 -- 172.21.15.63:0/3595321460 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5599d63962c0 con 0x5599d601b1c0 2024-05-27T08:03:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f422aa00640 10 client.21120 handle_command_reply: tid=0 2024-05-27T08:03:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f4252de4740 1 client.21120 shutdown 2024-05-27T08:03:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f4252de4740 1 -- 172.21.15.63:0/3595321460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 msgr2=0x5599d6448270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f4252de4740 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6448270 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f423c002230 tx=0x7f423c010d60 comp rx=0 tx=0).stop 2024-05-27T08:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f4252de4740 1 -- 172.21.15.63:0/3595321460 shutdown_connections 2024-05-27T08:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f4252de4740 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599d601b1c0 0x5599d64b4190 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f4252de4740 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599d60871b0 0x5599d64b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f4252de4740 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599d60664d0 0x5599d64487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f4252de4740 1 --2- 172.21.15.63:0/3595321460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599d605e2f0 0x5599d6448270 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f4252de4740 1 -- 172.21.15.63:0/3595321460 >> 172.21.15.63:0/3595321460 conn(0x5599d607b300 msgr2=0x5599d6438270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.731+0000 7f422aa00640 0 client.21120 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.732+0000 7f4252de4740 1 -- 172.21.15.63:0/3595321460 shutdown_connections 2024-05-27T08:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.732+0000 7f4252de4740 1 -- 172.21.15.63:0/3595321460 wait complete. 2024-05-27T08:03:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.732+0000 7f4252de4740 20 client.21120 trim_cache size 0 max 16384 2024-05-27T08:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:03:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 -- 172.21.15.63:0/2178286443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f422c0b29d0 msgr2=0x7f422c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f422c0b29d0 0x7f422c0b4ee0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x5599d64272c0 tx=0x7f4238091050 comp rx=0 tx=0).stop 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 -- 172.21.15.63:0/2178286443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4244104bb0 msgr2=0x7f424419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4244104bb0 0x7f424419a570 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f423800d950 tx=0x7f423800de10 comp rx=0 tx=0).stop 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 -- 172.21.15.63:0/2178286443 shutdown_connections 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f422c0b29d0 0x7f422c0b4ee0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f424410e000 0x7f424419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f424410b610 0x7f424419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 --2- 172.21.15.63:0/2178286443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4244104bb0 0x7f424419a570 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 -- 172.21.15.63:0/2178286443 >> 172.21.15.63:0/2178286443 conn(0x7f42440fe210 msgr2=0x7f424410e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 -- 172.21.15.63:0/2178286443 shutdown_connections 2024-05-27T08:03:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:06.733+0000 7f424a800640 1 -- 172.21.15.63:0/2178286443 wait complete. 2024-05-27T08:03:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:07 smithi086 ceph-mon[34687]: pgmap v937: 129 pgs: 6 active+clean+snaptrim_wait, 123 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 34 op/s 2024-05-27T08:03:07.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:07 smithi149 ceph-mon[35408]: pgmap v937: 129 pgs: 6 active+clean+snaptrim_wait, 123 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 34 op/s 2024-05-27T08:03:07.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:07 smithi063 ceph-mon[31613]: pgmap v937: 129 pgs: 6 active+clean+snaptrim_wait, 123 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.9 MiB/s wr, 34 op/s 2024-05-27T08:03:09.532 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:03:09.532 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:03:09.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:09 smithi086 ceph-mon[34687]: pgmap v938: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 29 op/s 2024-05-27T08:03:09.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:09 smithi149 ceph-mon[35408]: pgmap v938: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 29 op/s 2024-05-27T08:03:09.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:09 smithi063 ceph-mon[31613]: pgmap v938: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 29 op/s 2024-05-27T08:03:10.065 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:11.824 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:11 smithi063 ceph-mon[31613]: pgmap v939: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 29 op/s 2024-05-27T08:03:11.825 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:11 smithi063 ceph-mon[31613]: osdmap e142: 12 total, 12 up, 12 in 2024-05-27T08:03:11.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:11 smithi086 ceph-mon[34687]: pgmap v939: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 29 op/s 2024-05-27T08:03:11.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:11 smithi086 ceph-mon[34687]: osdmap e142: 12 total, 12 up, 12 in 2024-05-27T08:03:11.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:11 smithi149 ceph-mon[35408]: pgmap v939: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 29 op/s 2024-05-27T08:03:11.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:11 smithi149 ceph-mon[35408]: osdmap e142: 12 total, 12 up, 12 in 2024-05-27T08:03:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.554+0000 7fe80d000700 1 -- 172.21.15.63:0/2635828778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe80810af40 msgr2=0x7fe80810b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.554+0000 7fe80d000700 1 --2- 172.21.15.63:0/2635828778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe80810af40 0x7fe80810b350 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40099c0 tx=0x7fe7f4009cd0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.555+0000 7fe80d000700 1 -- 172.21.15.63:0/2635828778 shutdown_connections 2024-05-27T08:03:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.555+0000 7fe80d000700 1 --2- 172.21.15.63:0/2635828778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe80810b890 0x7fe8081131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.555+0000 7fe80d000700 1 --2- 172.21.15.63:0/2635828778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808106610 0x7fe808106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.555+0000 7fe80d000700 1 --2- 172.21.15.63:0/2635828778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe80810af40 0x7fe80810b350 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.555+0000 7fe80d000700 1 -- 172.21.15.63:0/2635828778 >> 172.21.15.63:0/2635828778 conn(0x7fe808102020 msgr2=0x7fe808104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.555+0000 7fe80d000700 1 -- 172.21.15.63:0/2635828778 shutdown_connections 2024-05-27T08:03:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.555+0000 7fe80d000700 1 -- 172.21.15.63:0/2635828778 wait complete. 2024-05-27T08:03:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.556+0000 7fe80d000700 1 Processor -- start 2024-05-27T08:03:12.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.557+0000 7fe80d000700 1 -- start start 2024-05-27T08:03:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.557+0000 7fe80d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808106610 0x7fe80819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.557+0000 7fe80d000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe80810af40 0x7fe80819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.557+0000 7fe80d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe80810b890 0x7fe8081a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.558+0000 7fe80d000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081163a0 con 0x7fe80810b890 2024-05-27T08:03:12.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.558+0000 7fe80d000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe808116220 con 0x7fe808106610 2024-05-27T08:03:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.558+0000 7fe80d000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe808116520 con 0x7fe80810af40 2024-05-27T08:03:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.558+0000 7fe807e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe80810b890 0x7fe8081a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.558+0000 7fe807e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe80810b890 0x7fe8081a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46002/0 (socket says 172.21.15.63:46002) 2024-05-27T08:03:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.558+0000 7fe807e00700 1 -- 172.21.15.63:0/14577410 learned_addr learned my addr 172.21.15.63:0/14577410 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.558+0000 7fe807400700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808106610 0x7fe80819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.558+0000 7fe806a00700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe80810af40 0x7fe80819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.559+0000 7fe807e00700 1 -- 172.21.15.63:0/14577410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe80810af40 msgr2=0x7fe80819d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.559+0000 7fe807e00700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe80810af40 0x7fe80819d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.559+0000 7fe807e00700 1 -- 172.21.15.63:0/14577410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808106610 msgr2=0x7fe80819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.559+0000 7fe807e00700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808106610 0x7fe80819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.559+0000 7fe807e00700 1 -- 172.21.15.63:0/14577410 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8081a1c80 con 0x7fe80810b890 2024-05-27T08:03:12.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.559+0000 7fe807e00700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe80810b890 0x7fe8081a15e0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fe800007ee0 tx=0x7fe80000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.560+0000 7fe7ffe00700 1 -- 172.21.15.63:0/14577410 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80000dbd0 con 0x7fe80810b890 2024-05-27T08:03:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.560+0000 7fe7ffe00700 1 -- 172.21.15.63:0/14577410 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe80000dd70 con 0x7fe80810b890 2024-05-27T08:03:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.560+0000 7fe7ffe00700 1 -- 172.21.15.63:0/14577410 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8000058d0 con 0x7fe80810b890 2024-05-27T08:03:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.560+0000 7fe80d000700 1 -- 172.21.15.63:0/14577410 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8081a1f70 con 0x7fe80810b890 2024-05-27T08:03:12.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.560+0000 7fe80d000700 1 -- 172.21.15.63:0/14577410 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8081a9890 con 0x7fe80810b890 2024-05-27T08:03:12.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.565+0000 7fe7ffe00700 1 -- 172.21.15.63:0/14577410 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe80001a020 con 0x7fe80810b890 2024-05-27T08:03:12.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.566+0000 7fe81d222b80 1 Processor -- start 2024-05-27T08:03:12.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.567+0000 7fe81d222b80 1 -- start start 2024-05-27T08:03:12.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.567+0000 7fe7ffe00700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7ec0b2a70 0x7fe7ec0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.567+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 0x55a536b2ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.567+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b30490 0x55a536b3ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.567+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.567+0000 7fe7ffe00700 1 -- 172.21.15.63:0/14577410 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7fe8000e2110 con 0x7fe80810b890 2024-05-27T08:03:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.567+0000 7fe81d222b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a536a5eec0 con 0x55a536b308a0 2024-05-27T08:03:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.567+0000 7fe81d222b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a536a4ea60 con 0x55a536b30490 2024-05-27T08:03:12.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.567+0000 7fe81d222b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a536a52aa0 con 0x55a5369b1ec0 2024-05-27T08:03:12.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.567+0000 7fe806a00700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7ec0b2a70 0x7fe7ec0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.568+0000 7fe807e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.568+0000 7fe807e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46004/0 (socket says 172.21.15.63:46004) 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.568+0000 7fe807e00700 1 -- 172.21.15.63:0/3800041162 learned_addr learned my addr 172.21.15.63:0/3800041162 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.568+0000 7fe807400700 1 --2- 172.21.15.63:0/3800041162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 0x55a536b2ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.568+0000 7fe806a00700 1 --2- 172.21.15.63:0/3800041162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b30490 0x55a536b3ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.569+0000 7fe807e00700 1 -- 172.21.15.63:0/3800041162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 msgr2=0x55a536b2ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.569+0000 7fe807e00700 1 --2- 172.21.15.63:0/3800041162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 0x55a536b2ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.569+0000 7fe807e00700 1 -- 172.21.15.63:0/3800041162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b30490 msgr2=0x55a536b3ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.569+0000 7fe807e00700 1 --2- 172.21.15.63:0/3800041162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b30490 0x55a536b3ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.569+0000 7fe807e00700 1 -- 172.21.15.63:0/3800041162 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a536a174a0 con 0x55a536b308a0 2024-05-27T08:03:12.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.569+0000 7fe807400700 1 --2- 172.21.15.63:0/3800041162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 0x55a536b2ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:12.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.569+0000 7fe806a00700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7ec0b2a70 0x7fe7ec0b4f80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009990 tx=0x7fe7f4005380 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:12.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.569+0000 7fe807e00700 1 --2- 172.21.15.63:0/3800041162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3d0a0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fe8000095b0 tx=0x7fe8000a87c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d5a702f2398542 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:12.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.569+0000 7fe7fcc00700 1 -- 172.21.15.63:0/3800041162 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8000a8fd0 con 0x55a536b308a0 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.570+0000 7fe7fcc00700 1 -- 172.21.15.63:0/3800041162 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8000ad380 con 0x55a536b308a0 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.570+0000 7fe7fcc00700 1 -- 172.21.15.63:0/3800041162 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8000a9230 con 0x55a536b308a0 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.571+0000 7fe81d222b80 1 -- 172.21.15.63:0/3800041162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 msgr2=0x55a536b3d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.571+0000 7fe81d222b80 1 --2- 172.21.15.63:0/3800041162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3d0a0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fe8000095b0 tx=0x7fe8000a87c0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.571+0000 7fe81d222b80 1 -- 172.21.15.63:0/3800041162 shutdown_connections 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.571+0000 7fe81d222b80 1 --2- 172.21.15.63:0/3800041162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3d0a0 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.571+0000 7fe81d222b80 1 --2- 172.21.15.63:0/3800041162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 0x55a536b2ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.571+0000 7fe81d222b80 1 --2- 172.21.15.63:0/3800041162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b30490 0x55a536b3ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.571+0000 7fe81d222b80 1 -- 172.21.15.63:0/3800041162 >> 172.21.15.63:0/3800041162 conn(0x55a536a12040 msgr2=0x55a536b2bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.571+0000 7fe81d222b80 1 -- 172.21.15.63:0/3800041162 shutdown_connections 2024-05-27T08:03:12.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.571+0000 7fe81d222b80 1 -- 172.21.15.63:0/3800041162 wait complete. 2024-05-27T08:03:12.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.572+0000 7fe81d222b80 1 -- start start 2024-05-27T08:03:12.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.572+0000 7fe81d222b80 1 Processor -- start 2024-05-27T08:03:12.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.573+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 0x55a536b3a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.573+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b30490 0x55a536b3ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.573+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.573+0000 7fe81d222b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a536955020 con 0x55a536b308a0 2024-05-27T08:03:12.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.573+0000 7fe81d222b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5369ef7e0 con 0x55a536b30490 2024-05-27T08:03:12.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.573+0000 7fe81d222b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a536b3da10 con 0x55a5369b1ec0 2024-05-27T08:03:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe81d222b80 10 client.0 _ll_register_callbacks cb 0x55a5369f8cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe807e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe807e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46008/0 (socket says 172.21.15.63:46008) 2024-05-27T08:03:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe807e00700 1 -- 172.21.15.63:0/2018121039 learned_addr learned my addr 172.21.15.63:0/2018121039 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe807400700 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 0x55a536b3a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe806a00700 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b30490 0x55a536b3ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe807e00700 1 -- 172.21.15.63:0/2018121039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 msgr2=0x55a536b3a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe807e00700 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 0x55a536b3a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe807e00700 1 -- 172.21.15.63:0/2018121039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b30490 msgr2=0x55a536b3ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:12.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe807e00700 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b30490 0x55a536b3ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.574+0000 7fe807e00700 1 -- 172.21.15.63:0/2018121039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53699e950 con 0x55a536b308a0 2024-05-27T08:03:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.575+0000 7fe807e00700 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3b080 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fe8000a8f60 tx=0x7fe8000a9510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbed3d1d38dd9794 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.575+0000 7fe7f1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.575+0000 7fe7f1600700 1 -- 172.21.15.63:0/2018121039 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8000c7020 con 0x55a536b308a0 2024-05-27T08:03:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.575+0000 7fe7f1600700 1 -- 172.21.15.63:0/2018121039 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7fe8000b1bb0 con 0x55a536b308a0 2024-05-27T08:03:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.575+0000 7fe81d222b80 1 -- 172.21.15.63:0/2018121039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8000ad380 con 0x55a536b308a0 2024-05-27T08:03:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.576+0000 7fe7f1600700 1 -- 172.21.15.63:0/2018121039 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe80008c050 con 0x55a536b308a0 2024-05-27T08:03:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.576+0000 7fe81d222b80 10 client.21132 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.576+0000 7fe81d222b80 1 -- 172.21.15.63:0/2018121039 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a536a1f0e0 con 0x55a536b308a0 2024-05-27T08:03:12.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.576+0000 7fe7f1600700 1 -- 172.21.15.63:0/2018121039 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8000ad380 con 0x55a536b308a0 2024-05-27T08:03:12.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.576+0000 7fe7f1600700 1 -- 172.21.15.63:0/2018121039 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe800005020 con 0x55a536b308a0 2024-05-27T08:03:12.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.577+0000 7fe81d222b80 10 client.21132 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:12.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.577+0000 7fe81d222b80 10 client.21132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:12.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.577+0000 7fe81d222b80 20 client.21132 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:12.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.577+0000 7fe81d222b80 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a536b4a810 0x55a536b4cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.577+0000 7fe81d222b80 4 client.21132 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:12.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.577+0000 7fe81d222b80 1 -- 172.21.15.63:0/2018121039 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a536a60e10 con 0x55a536b4a810 2024-05-27T08:03:12.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.578+0000 7fe807400700 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a536b4a810 0x55a536b4cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.579+0000 7fe807400700 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a536b4a810 0x55a536b4cc30 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74efbe567bb508b2 server_cookie=43db82597128c693 in_seq=0 out_seq=0 2024-05-27T08:03:12.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.579+0000 7fe7f1600700 10 client.21132 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:12.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe7f1600700 1 -- 172.21.15.63:0/2018121039 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a536a60e10 con 0x55a536b4a810 2024-05-27T08:03:12.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe7f1600700 10 client.21132 handle_command_reply: tid=0 2024-05-27T08:03:12.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe81d222b80 1 client.21132 shutdown 2024-05-27T08:03:12.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe81d222b80 1 -- 172.21.15.63:0/2018121039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 msgr2=0x55a536b3b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe81d222b80 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3b080 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fe8000a8f60 tx=0x7fe8000a9510 comp rx=0 tx=0).stop 2024-05-27T08:03:12.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe81d222b80 1 -- 172.21.15.63:0/2018121039 shutdown_connections 2024-05-27T08:03:12.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe81d222b80 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a536b4a810 0x55a536b4cc30 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe7f1600700 0 client.21132 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:12.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe81d222b80 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b308a0 0x55a536b3b080 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe81d222b80 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5369b1ec0 0x55a536b3a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe81d222b80 1 --2- 172.21.15.63:0/2018121039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b30490 0x55a536b3ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.581+0000 7fe81d222b80 1 -- 172.21.15.63:0/2018121039 >> 172.21.15.63:0/2018121039 conn(0x55a536a12040 msgr2=0x55a536b2b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:12.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.582+0000 7fe81d222b80 1 -- 172.21.15.63:0/2018121039 shutdown_connections 2024-05-27T08:03:12.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.582+0000 7fe81d222b80 1 -- 172.21.15.63:0/2018121039 wait complete. 2024-05-27T08:03:12.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.582+0000 7fe81d222b80 20 client.21132 trim_cache size 0 max 16384 2024-05-27T08:03:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.611+0000 7fe81d222b80 1 Processor -- start 2024-05-27T08:03:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.611+0000 7fe81d222b80 1 -- start start 2024-05-27T08:03:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.611+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5369b1ec0 0x55a536b34840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.611+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a536b30490 0x55a536b3caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.611+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b308a0 0x55a536b466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.611+0000 7fe81d222b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081b4dd0 con 0x55a5369b1ec0 2024-05-27T08:03:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.611+0000 7fe81d222b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081b2660 con 0x55a536b308a0 2024-05-27T08:03:12.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.611+0000 7fe81d222b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081b04d0 con 0x55a536b30490 2024-05-27T08:03:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe807400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5369b1ec0 0x55a536b34840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe807400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5369b1ec0 0x55a536b34840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46018/0 (socket says 172.21.15.63:46018) 2024-05-27T08:03:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe807400700 1 -- 172.21.15.63:0/2890764244 learned_addr learned my addr 172.21.15.63:0/2890764244 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe806a00700 1 --2- 172.21.15.63:0/2890764244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a536b30490 0x55a536b3caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe807e00700 1 --2- 172.21.15.63:0/2890764244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b308a0 0x55a536b466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe807400700 1 -- 172.21.15.63:0/2890764244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a536b30490 msgr2=0x55a536b3caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:12.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe807400700 1 --2- 172.21.15.63:0/2890764244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a536b30490 0x55a536b3caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe807400700 1 -- 172.21.15.63:0/2890764244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b308a0 msgr2=0x55a536b466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe807400700 1 --2- 172.21.15.63:0/2890764244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b308a0 0x55a536b466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe807400700 1 -- 172.21.15.63:0/2890764244 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a536b26250 con 0x55a5369b1ec0 2024-05-27T08:03:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.612+0000 7fe807e00700 1 --2- 172.21.15.63:0/2890764244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b308a0 0x55a536b466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.613+0000 7fe807400700 1 --2- 172.21.15.63:0/2890764244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5369b1ec0 0x55a536b34840 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8000c00 tx=0x7fe7f8013740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f15344535e1c2a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.613+0000 7fe7f3e00700 1 -- 172.21.15.63:0/2890764244 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8004050 con 0x55a5369b1ec0 2024-05-27T08:03:12.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.613+0000 7fe7f3e00700 1 -- 172.21.15.63:0/2890764244 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7f8003680 con 0x55a5369b1ec0 2024-05-27T08:03:12.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.613+0000 7fe7f3e00700 1 -- 172.21.15.63:0/2890764244 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f801d690 con 0x55a5369b1ec0 2024-05-27T08:03:12.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.614+0000 7fe81d222b80 1 -- 172.21.15.63:0/2890764244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5369b1ec0 msgr2=0x55a536b34840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.614+0000 7fe81d222b80 1 --2- 172.21.15.63:0/2890764244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5369b1ec0 0x55a536b34840 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8000c00 tx=0x7fe7f8013740 comp rx=0 tx=0).stop 2024-05-27T08:03:12.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.615+0000 7fe81d222b80 1 -- 172.21.15.63:0/2890764244 shutdown_connections 2024-05-27T08:03:12.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.615+0000 7fe81d222b80 1 --2- 172.21.15.63:0/2890764244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a536b308a0 0x55a536b466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.615+0000 7fe81d222b80 1 --2- 172.21.15.63:0/2890764244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5369b1ec0 0x55a536b34840 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.615+0000 7fe81d222b80 1 --2- 172.21.15.63:0/2890764244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a536b30490 0x55a536b3caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.615+0000 7fe81d222b80 1 -- 172.21.15.63:0/2890764244 >> 172.21.15.63:0/2890764244 conn(0x55a536a12040 msgr2=0x55a536b3ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:12.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.615+0000 7fe81d222b80 1 -- 172.21.15.63:0/2890764244 shutdown_connections 2024-05-27T08:03:12.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.615+0000 7fe81d222b80 1 -- 172.21.15.63:0/2890764244 wait complete. 2024-05-27T08:03:12.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.615+0000 7fe81d222b80 1 -- start start 2024-05-27T08:03:12.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.616+0000 7fe81d222b80 1 Processor -- start 2024-05-27T08:03:12.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.616+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5369b1ec0 0x55a536b4e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.616+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b30490 0x55a536b4e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.616+0000 7fe81d222b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a536b308a0 0x55a536bb4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.616+0000 7fe81d222b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081b0b20 con 0x55a536b30490 2024-05-27T08:03:12.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.616+0000 7fe81d222b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081b1f90 con 0x55a5369b1ec0 2024-05-27T08:03:12.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.616+0000 7fe81d222b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081b11f0 con 0x55a536b308a0 2024-05-27T08:03:12.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.616+0000 7fe81d222b80 10 client.0 _ll_register_callbacks cb 0x55a5369f8cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:12.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe806a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b30490 0x55a536b4e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe806a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b30490 0x55a536b4e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46026/0 (socket says 172.21.15.63:46026) 2024-05-27T08:03:12.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe806a00700 1 -- 172.21.15.63:0/3052537252 learned_addr learned my addr 172.21.15.63:0/3052537252 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:12.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe807400700 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5369b1ec0 0x55a536b4e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe807e00700 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a536b308a0 0x55a536bb4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe806a00700 1 -- 172.21.15.63:0/3052537252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a536b308a0 msgr2=0x55a536bb4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:12.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe806a00700 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a536b308a0 0x55a536bb4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe806a00700 1 -- 172.21.15.63:0/3052537252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5369b1ec0 msgr2=0x55a536b4e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:12.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe806a00700 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5369b1ec0 0x55a536b4e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe806a00700 1 -- 172.21.15.63:0/3052537252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a536bb45c0 con 0x55a536b30490 2024-05-27T08:03:12.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.617+0000 7fe807400700 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5369b1ec0 0x55a536b4e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:12.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.618+0000 7fe806a00700 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b30490 0x55a536b4e6b0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fe80806c3a0 tx=0x7fe7f4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892b5bead693e987 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:12.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.618+0000 7fe7f1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:12.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.618+0000 7fe7f1600700 1 -- 172.21.15.63:0/3052537252 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4003dc0 con 0x55a536b30490 2024-05-27T08:03:12.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.618+0000 7fe7f1600700 1 -- 172.21.15.63:0/3052537252 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7fe7f4021040 con 0x55a536b30490 2024-05-27T08:03:12.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.618+0000 7fe81d222b80 1 -- 172.21.15.63:0/3052537252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f8003680 con 0x55a536b30490 2024-05-27T08:03:12.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.618+0000 7fe7f1600700 1 -- 172.21.15.63:0/3052537252 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe7f403f050 con 0x55a536b30490 2024-05-27T08:03:12.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.619+0000 7fe81d222b80 10 client.21138 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:12.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.619+0000 7fe81d222b80 1 -- 172.21.15.63:0/3052537252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a536bb5870 con 0x55a536b30490 2024-05-27T08:03:12.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.619+0000 7fe7f1600700 1 -- 172.21.15.63:0/3052537252 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7f8003680 con 0x55a536b30490 2024-05-27T08:03:12.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.619+0000 7fe7f1600700 1 -- 172.21.15.63:0/3052537252 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe7f40284e0 con 0x55a536b30490 2024-05-27T08:03:12.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.619+0000 7fe81d222b80 10 client.21138 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:12.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.619+0000 7fe81d222b80 10 client.21138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:12.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.619+0000 7fe81d222b80 20 client.21138 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:12.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.619+0000 7fe81d222b80 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a536b4a810 0x55a536bb8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:12.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.619+0000 7fe81d222b80 4 client.21138 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:03:12.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.619+0000 7fe81d222b80 1 -- 172.21.15.63:0/3052537252 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a536a5a5f0 con 0x55a536b4a810 2024-05-27T08:03:12.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.620+0000 7fe807400700 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a536b4a810 0x55a536bb8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:12.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.621+0000 7fe7f1600700 10 client.21138 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:12.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.621+0000 7fe807400700 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a536b4a810 0x55a536bb8040 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29b6679c65f23ba server_cookie=73557ac7948b52b7 in_seq=0 out_seq=0 2024-05-27T08:03:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe7f1600700 1 -- 172.21.15.63:0/3052537252 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a536a5a5f0 con 0x55a536b4a810 2024-05-27T08:03:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe7f1600700 10 client.21138 handle_command_reply: tid=0 2024-05-27T08:03:12.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe81d222b80 1 client.21138 shutdown 2024-05-27T08:03:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe81d222b80 1 -- 172.21.15.63:0/3052537252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b30490 msgr2=0x55a536b4e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe81d222b80 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b30490 0x55a536b4e6b0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fe80806c3a0 tx=0x7fe7f4009cd0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe81d222b80 1 -- 172.21.15.63:0/3052537252 shutdown_connections 2024-05-27T08:03:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe81d222b80 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a536b4a810 0x55a536bb8040 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe81d222b80 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a536b308a0 0x55a536bb4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe81d222b80 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5369b1ec0 0x55a536b4e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe81d222b80 1 --2- 172.21.15.63:0/3052537252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a536b30490 0x55a536b4e6b0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe81d222b80 1 -- 172.21.15.63:0/3052537252 >> 172.21.15.63:0/3052537252 conn(0x55a536a12040 msgr2=0x55a536b3ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:12.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.625+0000 7fe7f1600700 0 client.21138 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.626+0000 7fe81d222b80 1 -- 172.21.15.63:0/3052537252 shutdown_connections 2024-05-27T08:03:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.626+0000 7fe81d222b80 1 -- 172.21.15.63:0/3052537252 wait complete. 2024-05-27T08:03:12.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.626+0000 7fe81d222b80 20 client.21138 trim_cache size 0 max 16384 2024-05-27T08:03:12.628 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:03:12.629 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:03:12.629 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "658a6c01-2f42-4c19-bbbe-59debc5aad93", 2024-05-27T08:03:12.629 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:03:12.629 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:03:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 -- 172.21.15.63:0/14577410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7ec0b2a70 msgr2=0x7fe7ec0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7ec0b2a70 0x7fe7ec0b4f80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009990 tx=0x7fe7f4005380 comp rx=0 tx=0).stop 2024-05-27T08:03:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 -- 172.21.15.63:0/14577410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe80810b890 msgr2=0x7fe8081a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe80810b890 0x7fe8081a15e0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fe800007ee0 tx=0x7fe80000d3b0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 -- 172.21.15.63:0/14577410 shutdown_connections 2024-05-27T08:03:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe7ec0b2a70 0x7fe7ec0b4f80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe80810b890 0x7fe8081a15e0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808106610 0x7fe80819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 --2- 172.21.15.63:0/14577410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe80810af40 0x7fe80819d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 -- 172.21.15.63:0/14577410 >> 172.21.15.63:0/14577410 conn(0x7fe808102020 msgr2=0x7fe80810a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 -- 172.21.15.63:0/14577410 shutdown_connections 2024-05-27T08:03:12.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:12.627+0000 7fe80d000700 1 -- 172.21.15.63:0/14577410 wait complete. 2024-05-27T08:03:12.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:12 smithi063 ceph-mon[31613]: pgmap v941: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:03:13.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:12 smithi086 ceph-mon[34687]: pgmap v941: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:03:13.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:12 smithi149 ceph-mon[35408]: pgmap v941: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:03:13.750 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:03:13.840 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:03:13.840 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:13.840 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:03:13.840 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:03:13.840 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:03:13.840 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:13.840 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:13.840 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:03:13.840 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:13.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:03:13.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:13.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:03:13.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:13.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:03:13.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:13.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:13 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:03:14.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:03:14.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:03:14.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:03:14.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:03:14.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.212 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:03:14.212 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.212 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:03:14.212 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.212 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:03:14.212 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.212 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:13 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:03:14.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:03:14.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:03:14.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:03:14.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:03:14.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:03:14.216 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.216 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:03:14.216 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.216 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:03:14.216 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.216 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:13 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:14.269 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:15.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:14 smithi086 ceph-mon[34687]: pgmap v942: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:03:15.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:14 smithi149 ceph-mon[35408]: pgmap v942: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:03:15.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:14 smithi063 ceph-mon[31613]: pgmap v942: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:03:15.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:15 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:03:16.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:15 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:03:16.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:15 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:03:16.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.631+0000 7f466f200700 1 -- 172.21.15.63:0/624582167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46701043f0 msgr2=0x7f4670104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:16.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.631+0000 7f466f200700 1 --2- 172.21.15.63:0/624582167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46701043f0 0x7f4670104800 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f46600099c0 tx=0x7f4660009cd0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.632+0000 7f466f200700 1 -- 172.21.15.63:0/624582167 shutdown_connections 2024-05-27T08:03:16.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.632+0000 7f466f200700 1 --2- 172.21.15.63:0/624582167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670109670 0x7f4670110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.632+0000 7f466f200700 1 --2- 172.21.15.63:0/624582167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46701043f0 0x7f4670104800 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.632+0000 7f466f200700 1 --2- 172.21.15.63:0/624582167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4670108d20 0x7f4670109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.632+0000 7f466f200700 1 -- 172.21.15.63:0/624582167 >> 172.21.15.63:0/624582167 conn(0x7f4670079d00 msgr2=0x7f467007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:16.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.632+0000 7f466f200700 1 -- 172.21.15.63:0/624582167 shutdown_connections 2024-05-27T08:03:16.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.632+0000 7f466f200700 1 -- 172.21.15.63:0/624582167 wait complete. 2024-05-27T08:03:16.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.634+0000 7f466f200700 1 Processor -- start 2024-05-27T08:03:16.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.634+0000 7f466f200700 1 -- start start 2024-05-27T08:03:16.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.634+0000 7f466f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46701043f0 0x7f467019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.634+0000 7f466f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670108d20 0x7f467019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.635+0000 7f466f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670109670 0x7f46701a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.635+0000 7f466f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46701141c0 con 0x7f4670108d20 2024-05-27T08:03:16.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.635+0000 7f466f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4670114040 con 0x7f46701043f0 2024-05-27T08:03:16.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.635+0000 7f466f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4670114340 con 0x7f4670109670 2024-05-27T08:03:16.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.635+0000 7f466d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670108d20 0x7f467019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.635+0000 7f466d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670108d20 0x7f467019d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46058/0 (socket says 172.21.15.63:46058) 2024-05-27T08:03:16.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.635+0000 7f466d400700 1 -- 172.21.15.63:0/3769436049 learned_addr learned my addr 172.21.15.63:0/3769436049 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:16.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.635+0000 7f466de00700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46701043f0 0x7f467019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.635+0000 7f466e800700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670109670 0x7f46701a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.636+0000 7f466d400700 1 -- 172.21.15.63:0/3769436049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670109670 msgr2=0x7f46701a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:16.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.636+0000 7f466d400700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670109670 0x7f46701a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.636+0000 7f466d400700 1 -- 172.21.15.63:0/3769436049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46701043f0 msgr2=0x7f467019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:16.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.636+0000 7f466d400700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46701043f0 0x7f467019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.636+0000 7f466d400700 1 -- 172.21.15.63:0/3769436049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46701a1d40 con 0x7f4670108d20 2024-05-27T08:03:16.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.636+0000 7f466d400700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670108d20 0x7f467019d410 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f465c00d980 tx=0x7f465c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:16.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.637+0000 7f4666a00700 1 -- 172.21.15.63:0/3769436049 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c017070 con 0x7f4670108d20 2024-05-27T08:03:16.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.637+0000 7f4666a00700 1 -- 172.21.15.63:0/3769436049 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f465c00fd20 con 0x7f4670108d20 2024-05-27T08:03:16.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.637+0000 7f4666a00700 1 -- 172.21.15.63:0/3769436049 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c00e3b0 con 0x7f4670108d20 2024-05-27T08:03:16.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.637+0000 7f466f200700 1 -- 172.21.15.63:0/3769436049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46701a2030 con 0x7f4670108d20 2024-05-27T08:03:16.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.637+0000 7f466f200700 1 -- 172.21.15.63:0/3769436049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46701a9950 con 0x7f4670108d20 2024-05-27T08:03:16.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.643+0000 7f4666a00700 1 -- 172.21.15.63:0/3769436049 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f465c00f890 con 0x7f4670108d20 2024-05-27T08:03:16.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.644+0000 7f468390bb80 1 Processor -- start 2024-05-27T08:03:16.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.644+0000 7f468390bb80 1 -- start start 2024-05-27T08:03:16.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.645+0000 7f468390bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf7328750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.645+0000 7f468390bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf71a8e40 0x564bf7328c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.645+0000 7f468390bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf73346e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.645+0000 7f468390bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564bf7255e40 con 0x564bf72949f0 2024-05-27T08:03:16.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.645+0000 7f468390bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564bf72459e0 con 0x564bf73322c0 2024-05-27T08:03:16.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.645+0000 7f468390bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564bf7249a20 con 0x564bf71a8e40 2024-05-27T08:03:16.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.645+0000 7f466e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf7328750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.645+0000 7f466e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf7328750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46064/0 (socket says 172.21.15.63:46064) 2024-05-27T08:03:16.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.645+0000 7f466e800700 1 -- 172.21.15.63:0/2233943184 learned_addr learned my addr 172.21.15.63:0/2233943184 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:16.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.645+0000 7f466d400700 1 --2- 172.21.15.63:0/2233943184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf73346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.646+0000 7f466de00700 1 --2- 172.21.15.63:0/2233943184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf71a8e40 0x564bf7328c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.646+0000 7f466e800700 1 -- 172.21.15.63:0/2233943184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf71a8e40 msgr2=0x564bf7328c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:16.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.646+0000 7f466e800700 1 --2- 172.21.15.63:0/2233943184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf71a8e40 0x564bf7328c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.646+0000 7f466e800700 1 -- 172.21.15.63:0/2233943184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 msgr2=0x564bf73346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:16.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.646+0000 7f466e800700 1 --2- 172.21.15.63:0/2233943184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf73346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.646+0000 7f466e800700 1 -- 172.21.15.63:0/2233943184 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf720e420 con 0x564bf72949f0 2024-05-27T08:03:16.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.646+0000 7f4666a00700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46540b29f0 0x7f46540b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.646+0000 7f4666a00700 1 -- 172.21.15.63:0/3769436049 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f465c0e24c0 con 0x7f4670108d20 2024-05-27T08:03:16.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.647+0000 7f466e800700 1 --2- 172.21.15.63:0/2233943184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf7328750 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f4668003fa0 tx=0x7f4668009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17305a29f21de794 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:16.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.647+0000 7f466de00700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46540b29f0 0x7f46540b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.647+0000 7f465b400700 1 -- 172.21.15.63:0/2233943184 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668004bb0 con 0x564bf72949f0 2024-05-27T08:03:16.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.647+0000 7f465b400700 1 -- 172.21.15.63:0/2233943184 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46680048e0 con 0x564bf72949f0 2024-05-27T08:03:16.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.647+0000 7f465b400700 1 -- 172.21.15.63:0/2233943184 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668007c80 con 0x564bf72949f0 2024-05-27T08:03:16.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.648+0000 7f468390bb80 1 -- 172.21.15.63:0/2233943184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 msgr2=0x564bf7328750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:16.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.648+0000 7f466de00700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46540b29f0 0x7f46540b4f00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4660000c00 tx=0x7f4660005040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:16.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.648+0000 7f468390bb80 1 --2- 172.21.15.63:0/2233943184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf7328750 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f4668003fa0 tx=0x7f4668009ba0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.648+0000 7f468390bb80 1 -- 172.21.15.63:0/2233943184 shutdown_connections 2024-05-27T08:03:16.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.648+0000 7f468390bb80 1 --2- 172.21.15.63:0/2233943184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf73346e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.648+0000 7f468390bb80 1 --2- 172.21.15.63:0/2233943184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf7328750 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.648+0000 7f468390bb80 1 --2- 172.21.15.63:0/2233943184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf71a8e40 0x564bf7328c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.648+0000 7f468390bb80 1 -- 172.21.15.63:0/2233943184 >> 172.21.15.63:0/2233943184 conn(0x564bf7242390 msgr2=0x564bf7323710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:16.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.649+0000 7f468390bb80 1 -- 172.21.15.63:0/2233943184 shutdown_connections 2024-05-27T08:03:16.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.649+0000 7f468390bb80 1 -- 172.21.15.63:0/2233943184 wait complete. 2024-05-27T08:03:16.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.649+0000 7f468390bb80 1 -- start start 2024-05-27T08:03:16.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.650+0000 7f468390bb80 1 Processor -- start 2024-05-27T08:03:16.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.650+0000 7f468390bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf71a8e40 0x564bf732f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.650+0000 7f468390bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf732f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.650+0000 7f468390bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf732fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.650+0000 7f468390bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564bf714bfa0 con 0x564bf72949f0 2024-05-27T08:03:16.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.650+0000 7f468390bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564bf71e6760 con 0x564bf73322c0 2024-05-27T08:03:16.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.650+0000 7f468390bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564bf7334e00 con 0x564bf71a8e40 2024-05-27T08:03:16.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.650+0000 7f468390bb80 10 client.0 _ll_register_callbacks cb 0x564bf71efc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:16.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.651+0000 7f466d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf732f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.651+0000 7f466d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf732f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46080/0 (socket says 172.21.15.63:46080) 2024-05-27T08:03:16.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.651+0000 7f466d400700 1 -- 172.21.15.63:0/2195761540 learned_addr learned my addr 172.21.15.63:0/2195761540 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:16.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.651+0000 7f466e800700 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf732fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.651+0000 7f466de00700 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf71a8e40 0x564bf732f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.651+0000 7f466d400700 1 -- 172.21.15.63:0/2195761540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf71a8e40 msgr2=0x564bf732f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:16.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.651+0000 7f466d400700 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf71a8e40 0x564bf732f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.651+0000 7f466d400700 1 -- 172.21.15.63:0/2195761540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 msgr2=0x564bf732fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.651+0000 7f466d400700 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf732fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.651+0000 7f466d400700 1 -- 172.21.15.63:0/2195761540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf71958d0 con 0x564bf72949f0 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.652+0000 7f466d400700 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf732f760 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f465c00bfd0 tx=0x7f465c0104e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db59f844721042ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.652+0000 7f464fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.652+0000 7f464fe00700 1 -- 172.21.15.63:0/2195761540 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c020030 con 0x564bf72949f0 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.652+0000 7f464fe00700 1 -- 172.21.15.63:0/2195761540 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f465c0a97f0 con 0x564bf72949f0 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.652+0000 7f468390bb80 1 -- 172.21.15.63:0/2195761540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46680048e0 con 0x564bf72949f0 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.653+0000 7f464fe00700 1 -- 172.21.15.63:0/2195761540 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f465c0a8cd0 con 0x564bf72949f0 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.653+0000 7f468390bb80 10 client.21150 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:16.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.653+0000 7f468390bb80 1 -- 172.21.15.63:0/2195761540 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564bf7216060 con 0x564bf72949f0 2024-05-27T08:03:16.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.653+0000 7f464fe00700 1 -- 172.21.15.63:0/2195761540 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46680048e0 con 0x564bf72949f0 2024-05-27T08:03:16.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.654+0000 7f464fe00700 1 -- 172.21.15.63:0/2195761540 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f465c0e2110 con 0x564bf72949f0 2024-05-27T08:03:16.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.654+0000 7f468390bb80 10 client.21150 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:16.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.654+0000 7f468390bb80 10 client.21150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:16.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.654+0000 7f468390bb80 20 client.21150 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:16.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.654+0000 7f468390bb80 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564bf7341c70 0x564bf7344090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.654+0000 7f468390bb80 4 client.21150 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:16.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.654+0000 7f468390bb80 1 -- 172.21.15.63:0/2195761540 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bf7257d90 con 0x564bf7341c70 2024-05-27T08:03:16.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.655+0000 7f466de00700 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564bf7341c70 0x564bf7344090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.656+0000 7f464fe00700 10 client.21150 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:16.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.656+0000 7f466de00700 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564bf7341c70 0x564bf7344090 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dece90461b8d3cea server_cookie=f806945b0fd5024 in_seq=0 out_seq=0 2024-05-27T08:03:16.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.657+0000 7f464fe00700 1 -- 172.21.15.63:0/2195761540 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564bf7257d90 con 0x564bf7341c70 2024-05-27T08:03:16.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.657+0000 7f464fe00700 10 client.21150 handle_command_reply: tid=0 2024-05-27T08:03:16.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.658+0000 7f468390bb80 1 client.21150 shutdown 2024-05-27T08:03:16.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.658+0000 7f468390bb80 1 -- 172.21.15.63:0/2195761540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 msgr2=0x564bf732f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:16.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.658+0000 7f468390bb80 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf732f760 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f465c00bfd0 tx=0x7f465c0104e0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.658+0000 7f468390bb80 1 -- 172.21.15.63:0/2195761540 shutdown_connections 2024-05-27T08:03:16.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.658+0000 7f468390bb80 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564bf7341c70 0x564bf7344090 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.658+0000 7f468390bb80 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf732fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.658+0000 7f468390bb80 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf71a8e40 0x564bf732f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.658+0000 7f468390bb80 1 --2- 172.21.15.63:0/2195761540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf72949f0 0x564bf732f760 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.658+0000 7f464fe00700 0 client.21150 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:16.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.658+0000 7f468390bb80 1 -- 172.21.15.63:0/2195761540 >> 172.21.15.63:0/2195761540 conn(0x564bf7242390 msgr2=0x564bf73287f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:16.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.659+0000 7f468390bb80 1 -- 172.21.15.63:0/2195761540 shutdown_connections 2024-05-27T08:03:16.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.659+0000 7f468390bb80 1 -- 172.21.15.63:0/2195761540 wait complete. 2024-05-27T08:03:16.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.659+0000 7f468390bb80 20 client.21150 trim_cache size 0 max 16384 2024-05-27T08:03:16.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.685+0000 7f468390bb80 1 Processor -- start 2024-05-27T08:03:16.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.685+0000 7f468390bb80 1 -- start start 2024-05-27T08:03:16.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.685+0000 7f468390bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf732e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.685+0000 7f468390bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf72949f0 0x564bf7331300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.685+0000 7f468390bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf733dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.685+0000 7f468390bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b4f30 con 0x564bf71a8e40 2024-05-27T08:03:16.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.685+0000 7f468390bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b27c0 con 0x564bf73322c0 2024-05-27T08:03:16.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.685+0000 7f468390bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b0630 con 0x564bf72949f0 2024-05-27T08:03:16.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.686+0000 7f466de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf732e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.686+0000 7f466de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf732e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46090/0 (socket says 172.21.15.63:46090) 2024-05-27T08:03:16.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.686+0000 7f466de00700 1 -- 172.21.15.63:0/728231285 learned_addr learned my addr 172.21.15.63:0/728231285 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:16.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.686+0000 7f466e800700 1 --2- 172.21.15.63:0/728231285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf733dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.686+0000 7f466d400700 1 --2- 172.21.15.63:0/728231285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf72949f0 0x564bf7331300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.686+0000 7f466de00700 1 -- 172.21.15.63:0/728231285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf72949f0 msgr2=0x564bf7331300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:16.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.686+0000 7f466de00700 1 --2- 172.21.15.63:0/728231285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf72949f0 0x564bf7331300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.686+0000 7f466de00700 1 -- 172.21.15.63:0/728231285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 msgr2=0x564bf733dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:16.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.686+0000 7f466de00700 1 --2- 172.21.15.63:0/728231285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf733dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.686+0000 7f466de00700 1 -- 172.21.15.63:0/728231285 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf733e3e0 con 0x564bf71a8e40 2024-05-27T08:03:16.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.687+0000 7f466de00700 1 --2- 172.21.15.63:0/728231285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf732e7a0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f466000b580 tx=0x7f466001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccf24093354682c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:16.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.687+0000 7f465aa00700 1 -- 172.21.15.63:0/728231285 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466000bba0 con 0x564bf71a8e40 2024-05-27T08:03:16.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.687+0000 7f465aa00700 1 -- 172.21.15.63:0/728231285 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f466000bd40 con 0x564bf71a8e40 2024-05-27T08:03:16.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.687+0000 7f465aa00700 1 -- 172.21.15.63:0/728231285 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46600258b0 con 0x564bf71a8e40 2024-05-27T08:03:16.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.688+0000 7f468390bb80 1 -- 172.21.15.63:0/728231285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 msgr2=0x564bf732e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:16.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.688+0000 7f468390bb80 1 --2- 172.21.15.63:0/728231285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf732e7a0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f466000b580 tx=0x7f466001a050 comp rx=0 tx=0).stop 2024-05-27T08:03:16.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.688+0000 7f468390bb80 1 -- 172.21.15.63:0/728231285 shutdown_connections 2024-05-27T08:03:16.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.688+0000 7f468390bb80 1 --2- 172.21.15.63:0/728231285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf733dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.688+0000 7f468390bb80 1 --2- 172.21.15.63:0/728231285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf732e7a0 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.689+0000 7f468390bb80 1 --2- 172.21.15.63:0/728231285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf72949f0 0x564bf7331300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.689+0000 7f468390bb80 1 -- 172.21.15.63:0/728231285 >> 172.21.15.63:0/728231285 conn(0x564bf7242390 msgr2=0x564bf732f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:16.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.689+0000 7f468390bb80 1 -- 172.21.15.63:0/728231285 shutdown_connections 2024-05-27T08:03:16.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.689+0000 7f468390bb80 1 -- 172.21.15.63:0/728231285 wait complete. 2024-05-27T08:03:16.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.689+0000 7f468390bb80 1 -- start start 2024-05-27T08:03:16.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.690+0000 7f468390bb80 1 Processor -- start 2024-05-27T08:03:16.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.690+0000 7f468390bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf733a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.690+0000 7f468390bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf72949f0 0x564bf7335a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.690+0000 7f468390bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf7335f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.690+0000 7f468390bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b0c80 con 0x564bf71a8e40 2024-05-27T08:03:16.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.690+0000 7f468390bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b20f0 con 0x564bf73322c0 2024-05-27T08:03:16.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.690+0000 7f468390bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46701b1350 con 0x564bf72949f0 2024-05-27T08:03:16.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.690+0000 7f468390bb80 10 client.0 _ll_register_callbacks cb 0x564bf71efc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:16.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf733a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf733a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46092/0 (socket says 172.21.15.63:46092) 2024-05-27T08:03:16.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466de00700 1 -- 172.21.15.63:0/2103079596 learned_addr learned my addr 172.21.15.63:0/2103079596 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:16.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466d400700 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf72949f0 0x564bf7335a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466e800700 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf7335f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466de00700 1 -- 172.21.15.63:0/2103079596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf72949f0 msgr2=0x564bf7335a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:16.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466de00700 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf72949f0 0x564bf7335a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466de00700 1 -- 172.21.15.63:0/2103079596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 msgr2=0x564bf7335f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:16.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466de00700 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf7335f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466de00700 1 -- 172.21.15.63:0/2103079596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf7336670 con 0x564bf71a8e40 2024-05-27T08:03:16.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f466de00700 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf733a6f0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f4660005b40 tx=0x7f4660003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b45e943e8c19f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:16.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.691+0000 7f464fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.692+0000 7f464fe00700 1 -- 172.21.15.63:0/2103079596 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660025480 con 0x564bf71a8e40 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.692+0000 7f464fe00700 1 -- 172.21.15.63:0/2103079596 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f4660025620 con 0x564bf71a8e40 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.692+0000 7f468390bb80 1 -- 172.21.15.63:0/2103079596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f466000bd40 con 0x564bf71a8e40 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.692+0000 7f464fe00700 1 -- 172.21.15.63:0/2103079596 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f466004c050 con 0x564bf71a8e40 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.692+0000 7f468390bb80 10 client.21156 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.692+0000 7f468390bb80 1 -- 172.21.15.63:0/2103079596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564bf73ac5a0 con 0x564bf71a8e40 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.693+0000 7f464fe00700 1 -- 172.21.15.63:0/2103079596 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f466000bd40 con 0x564bf71a8e40 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.693+0000 7f464fe00700 1 -- 172.21.15.63:0/2103079596 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f466003b020 con 0x564bf71a8e40 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.693+0000 7f468390bb80 10 client.21156 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.693+0000 7f468390bb80 10 client.21156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.693+0000 7f468390bb80 20 client.21156 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:16.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.693+0000 7f468390bb80 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564bf73acbe0 0x564bf73af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:16.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.693+0000 7f468390bb80 4 client.21156 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:03:16.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.693+0000 7f468390bb80 1 -- 172.21.15.63:0/2103079596 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564bf7251570 con 0x564bf73acbe0 2024-05-27T08:03:16.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.694+0000 7f466d400700 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564bf73acbe0 0x564bf73af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:16.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.695+0000 7f464fe00700 10 client.21156 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.695+0000 7f466d400700 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564bf73acbe0 0x564bf73af000 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=945d9556c67b13e1 server_cookie=7754d1cc65e20d43 in_seq=0 out_seq=0 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.695+0000 7f464fe00700 1 -- 172.21.15.63:0/2103079596 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564bf7251570 con 0x564bf73acbe0 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f464fe00700 10 client.21156 handle_command_reply: tid=0 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f468390bb80 1 client.21156 shutdown 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f468390bb80 1 -- 172.21.15.63:0/2103079596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 msgr2=0x564bf733a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f468390bb80 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf733a6f0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f4660005b40 tx=0x7f4660003960 comp rx=0 tx=0).stop 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f468390bb80 1 -- 172.21.15.63:0/2103079596 shutdown_connections 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f468390bb80 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564bf73acbe0 0x564bf73af000 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f468390bb80 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564bf73322c0 0x564bf7335f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f468390bb80 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bf71a8e40 0x564bf733a6f0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f468390bb80 1 --2- 172.21.15.63:0/2103079596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564bf72949f0 0x564bf7335a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f468390bb80 1 -- 172.21.15.63:0/2103079596 >> 172.21.15.63:0/2103079596 conn(0x564bf7242390 msgr2=0x564bf732fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:16.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f464fe00700 0 client.21156 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:16.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.696+0000 7f468390bb80 1 -- 172.21.15.63:0/2103079596 shutdown_connections 2024-05-27T08:03:16.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.697+0000 7f468390bb80 1 -- 172.21.15.63:0/2103079596 wait complete. 2024-05-27T08:03:16.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.697+0000 7f468390bb80 20 client.21156 trim_cache size 0 max 16384 2024-05-27T08:03:16.700 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:03:16.700 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:03:16.700 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:03:16.700 INFO:teuthology.orchestra.run.smithi063.stdout: "658a6c01-2f42-4c19-bbbe-59debc5aad93": { 2024-05-27T08:03:16.700 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:03:16.700 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "658a6c01-2f42-4c19-bbbe-59debc5aad93", 2024-05-27T08:03:16.700 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:03:16.700 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:03:16.700 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:03:16.700 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:03:16.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 -- 172.21.15.63:0/3769436049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46540b29f0 msgr2=0x7f46540b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:16.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46540b29f0 0x7f46540b4f00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4660000c00 tx=0x7f4660005040 comp rx=0 tx=0).stop 2024-05-27T08:03:16.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 -- 172.21.15.63:0/3769436049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670108d20 msgr2=0x7f467019d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:16.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670108d20 0x7f467019d410 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f465c00d980 tx=0x7f465c00dc90 comp rx=0 tx=0).stop 2024-05-27T08:03:16.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 -- 172.21.15.63:0/3769436049 shutdown_connections 2024-05-27T08:03:16.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46540b29f0 0x7f46540b4f00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4670109670 0x7f46701a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46701043f0 0x7f467019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 --2- 172.21.15.63:0/3769436049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670108d20 0x7f467019d410 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:16.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 -- 172.21.15.63:0/3769436049 >> 172.21.15.63:0/3769436049 conn(0x7f4670079d00 msgr2=0x7f4670101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:16.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 -- 172.21.15.63:0/3769436049 shutdown_connections 2024-05-27T08:03:16.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:16.698+0000 7f466f200700 1 -- 172.21.15.63:0/3769436049 wait complete. 2024-05-27T08:03:16.756 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:16 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:03:16.756 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:16 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:03:16.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:16 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:03:16.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:16 smithi063 ceph-mon[31613]: pgmap v943: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:03:17.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:16 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:03:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:16 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:03:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:16 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:03:17.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:16 smithi086 ceph-mon[34687]: pgmap v943: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:03:17.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:16 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:03:17.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:16 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:03:17.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:16 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:03:17.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:16 smithi149 ceph-mon[35408]: pgmap v943: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:03:17.659 INFO:tasks.cephfs.filesystem:scrub status for tag:658a6c01-2f42-4c19-bbbe-59debc5aad93 - {'path': '/', 'tag': '658a6c01-2f42-4c19-bbbe-59debc5aad93', 'options': 'recursive,force'} 2024-05-27T08:03:19.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:19 smithi063 ceph-mon[31613]: pgmap v944: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.4 MiB/s wr, 52 op/s 2024-05-27T08:03:19.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:19 smithi086 ceph-mon[34687]: pgmap v944: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.4 MiB/s wr, 52 op/s 2024-05-27T08:03:19.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:19 smithi149 ceph-mon[35408]: pgmap v944: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.4 MiB/s wr, 52 op/s 2024-05-27T08:03:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.003+0000 7efe09e00640 1 -- 172.21.15.63:0/904277503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0410a120 msgr2=0x7efe04103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.003+0000 7efe09e00640 1 --2- 172.21.15.63:0/904277503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0410a120 0x7efe04103d60 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7efdec009900 tx=0x7efdec02a410 comp rx=0 tx=0).stop 2024-05-27T08:03:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.003+0000 7efe09e00640 1 -- 172.21.15.63:0/904277503 shutdown_connections 2024-05-27T08:03:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.003+0000 7efe09e00640 1 --2- 172.21.15.63:0/904277503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe041043d0 0x7efe041107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.003+0000 7efe09e00640 1 --2- 172.21.15.63:0/904277503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe0410a120 0x7efe04103d60 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.003+0000 7efe09e00640 1 --2- 172.21.15.63:0/904277503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe04109c80 0x7efe04103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.003+0000 7efe09e00640 1 -- 172.21.15.63:0/904277503 >> 172.21.15.63:0/904277503 conn(0x7efe04079290 msgr2=0x7efe040796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.003+0000 7efe09e00640 1 -- 172.21.15.63:0/904277503 shutdown_connections 2024-05-27T08:03:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.003+0000 7efe09e00640 1 -- 172.21.15.63:0/904277503 wait complete. 2024-05-27T08:03:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.005+0000 7efe09e00640 1 Processor -- start 2024-05-27T08:03:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.005+0000 7efe09e00640 1 -- start start 2024-05-27T08:03:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.005+0000 7efe09e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe041043d0 0x7efe0419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.005+0000 7efe09e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe04109c80 0x7efe0419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.005+0000 7efe09e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe0410a120 0x7efe041a8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.005+0000 7efe09e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe04113d20 con 0x7efe04109c80 2024-05-27T08:03:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.005+0000 7efe09e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe04113ba0 con 0x7efe041043d0 2024-05-27T08:03:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.005+0000 7efe09e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe04113ea0 con 0x7efe0410a120 2024-05-27T08:03:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe04109c80 0x7efe0419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe04109c80 0x7efe0419dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54372/0 (socket says 172.21.15.63:54372) 2024-05-27T08:03:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efdfbe00640 1 -- 172.21.15.63:0/3166256749 learned_addr learned my addr 172.21.15.63:0/3166256749 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efe02a00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe041043d0 0x7efe0419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efe03400640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe0410a120 0x7efe041a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efdfbe00640 1 -- 172.21.15.63:0/3166256749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe0410a120 msgr2=0x7efe041a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efdfbe00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe0410a120 0x7efe041a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efdfbe00640 1 -- 172.21.15.63:0/3166256749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe041043d0 msgr2=0x7efe0419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efdfbe00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe041043d0 0x7efe0419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efdfbe00640 1 -- 172.21.15.63:0/3166256749 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe041a8ba0 con 0x7efe04109c80 2024-05-27T08:03:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.006+0000 7efe03400640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe0410a120 0x7efe041a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.007+0000 7efdfbe00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe04109c80 0x7efe0419dd00 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7efdec005940 tx=0x7efdec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.007+0000 7efe08a00640 1 -- 172.21.15.63:0/3166256749 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec03f030 con 0x7efe04109c80 2024-05-27T08:03:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.007+0000 7efe08a00640 1 -- 172.21.15.63:0/3166256749 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdec003c90 con 0x7efe04109c80 2024-05-27T08:03:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.007+0000 7efe08a00640 1 -- 172.21.15.63:0/3166256749 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec035070 con 0x7efe04109c80 2024-05-27T08:03:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.008+0000 7efe09e00640 1 -- 172.21.15.63:0/3166256749 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe041a8dd0 con 0x7efe04109c80 2024-05-27T08:03:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.008+0000 7efe09e00640 1 -- 172.21.15.63:0/3166256749 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe041a91d0 con 0x7efe04109c80 2024-05-27T08:03:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.012+0000 7efe10e7d740 1 Processor -- start 2024-05-27T08:03:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.012+0000 7efe10e7d740 1 -- start start 2024-05-27T08:03:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.012+0000 7efe08a00640 1 -- 172.21.15.63:0/3166256749 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efdec042030 con 0x7efe04109c80 2024-05-27T08:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.013+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943cf1b0 0x56039477a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.013+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x56039477a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.013+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943ae4d0 0x56039477aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.013+0000 7efe10e7d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5603946e2b90 con 0x5603943cf1b0 2024-05-27T08:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.013+0000 7efe10e7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5603946d2730 con 0x5603943a62f0 2024-05-27T08:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.013+0000 7efe10e7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5603946d6770 con 0x5603943ae4d0 2024-05-27T08:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.013+0000 7efe03400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943cf1b0 0x56039477a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.013+0000 7efe03400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943cf1b0 0x56039477a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54382/0 (socket says 172.21.15.63:54382) 2024-05-27T08:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.013+0000 7efe03400640 1 -- 172.21.15.63:0/4233032124 learned_addr learned my addr 172.21.15.63:0/4233032124 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.014+0000 7efdfbe00640 1 --2- 172.21.15.63:0/4233032124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943ae4d0 0x56039477aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.014+0000 7efe03400640 1 -- 172.21.15.63:0/4233032124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943ae4d0 msgr2=0x56039477aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.014+0000 7efe03400640 1 --2- 172.21.15.63:0/4233032124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943ae4d0 0x56039477aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.014+0000 7efe02a00640 1 --2- 172.21.15.63:0/4233032124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x56039477a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.014+0000 7efe03400640 1 -- 172.21.15.63:0/4233032124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 msgr2=0x56039477a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.014+0000 7efe03400640 1 --2- 172.21.15.63:0/4233032124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x56039477a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.014+0000 7efe03400640 1 -- 172.21.15.63:0/4233032124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039466b8b0 con 0x5603943cf1b0 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.014+0000 7efe02a00640 1 --2- 172.21.15.63:0/4233032124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x56039477a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.014+0000 7efe03400640 1 --2- 172.21.15.63:0/4233032124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943cf1b0 0x56039477a070 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7efdfc007f40 tx=0x7efdfc00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bc57411f1b48f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.015+0000 7efe08a00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde80b2950 0x7efde80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.015+0000 7efdf3e00640 1 -- 172.21.15.63:0/4233032124 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc014070 con 0x5603943cf1b0 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.015+0000 7efdf3e00640 1 -- 172.21.15.63:0/4233032124 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdfc004d20 con 0x5603943cf1b0 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.015+0000 7efe08a00640 1 -- 172.21.15.63:0/3166256749 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7efdec1032d0 con 0x7efe04109c80 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.015+0000 7efdf3e00640 1 -- 172.21.15.63:0/4233032124 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc005450 con 0x5603943cf1b0 2024-05-27T08:03:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.015+0000 7efe02a00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde80b2950 0x7efde80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.016+0000 7efe10e7d740 1 -- 172.21.15.63:0/4233032124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943cf1b0 msgr2=0x56039477a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.016+0000 7efe10e7d740 1 --2- 172.21.15.63:0/4233032124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943cf1b0 0x56039477a070 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7efdfc007f40 tx=0x7efdfc00e7e0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.016+0000 7efe02a00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde80b2950 0x7efde80b4e60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56039477b420 tx=0x7efdf400a2d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.016+0000 7efe10e7d740 1 -- 172.21.15.63:0/4233032124 shutdown_connections 2024-05-27T08:03:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.017+0000 7efe10e7d740 1 --2- 172.21.15.63:0/4233032124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943ae4d0 0x56039477aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.017+0000 7efe10e7d740 1 --2- 172.21.15.63:0/4233032124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x56039477a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.017+0000 7efe10e7d740 1 --2- 172.21.15.63:0/4233032124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943cf1b0 0x56039477a070 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.017+0000 7efe10e7d740 1 -- 172.21.15.63:0/4233032124 >> 172.21.15.63:0/4233032124 conn(0x5603943c3300 msgr2=0x560394776820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.017+0000 7efe10e7d740 1 -- 172.21.15.63:0/4233032124 shutdown_connections 2024-05-27T08:03:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.017+0000 7efe10e7d740 1 -- 172.21.15.63:0/4233032124 wait complete. 2024-05-27T08:03:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.017+0000 7efe10e7d740 1 -- start start 2024-05-27T08:03:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.018+0000 7efe10e7d740 1 Processor -- start 2024-05-27T08:03:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.018+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943a62f0 0x5603947844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.018+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943ae4d0 0x5603947849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.018+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x560394790500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.018+0000 7efe10e7d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5603943b8790 con 0x5603943a62f0 2024-05-27T08:03:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.018+0000 7efe10e7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56039467abc0 con 0x5603943ae4d0 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.018+0000 7efe10e7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5603947871a0 con 0x5603943cf1b0 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.019+0000 7efe10e7d740 10 client.0 _ll_register_callbacks cb 0x560394658400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.019+0000 7efe02a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943a62f0 0x5603947844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.019+0000 7efe02a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943a62f0 0x5603947844b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54390/0 (socket says 172.21.15.63:54390) 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.019+0000 7efe02a00640 1 -- 172.21.15.63:0/4020311165 learned_addr learned my addr 172.21.15.63:0/4020311165 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.019+0000 7efe03400640 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x560394790500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.019+0000 7efe02a00640 1 -- 172.21.15.63:0/4020311165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 msgr2=0x560394790500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.019+0000 7efe02a00640 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x560394790500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.019+0000 7efe02a00640 1 -- 172.21.15.63:0/4020311165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943ae4d0 msgr2=0x5603947849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.019+0000 7efe02a00640 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943ae4d0 0x5603947849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.019+0000 7efe02a00640 1 -- 172.21.15.63:0/4020311165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603943e7210 con 0x5603943a62f0 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.020+0000 7efe02a00640 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943a62f0 0x5603947844b0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7efdf40129b0 tx=0x7efdf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb612d70445b35f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.020+0000 7efdf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.020+0000 7efdf0c00640 1 -- 172.21.15.63:0/4020311165 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf400fce0 con 0x5603943a62f0 2024-05-27T08:03:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.020+0000 7efdf0c00640 1 -- 172.21.15.63:0/4020311165 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7efdf4005020 con 0x5603943a62f0 2024-05-27T08:03:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.020+0000 7efe10e7d740 1 -- 172.21.15.63:0/4020311165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc004d20 con 0x5603943a62f0 2024-05-27T08:03:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.021+0000 7efdf0c00640 1 -- 172.21.15.63:0/4020311165 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efdf400fe80 con 0x5603943a62f0 2024-05-27T08:03:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.021+0000 7efe10e7d740 10 client.21168 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.021+0000 7efe10e7d740 1 -- 172.21.15.63:0/4020311165 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5603944b3620 con 0x5603943a62f0 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.021+0000 7efdf0c00640 1 -- 172.21.15.63:0/4020311165 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdfc004d20 con 0x5603943a62f0 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.021+0000 7efdf0c00640 1 -- 172.21.15.63:0/4020311165 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7efdf4031420 con 0x5603943a62f0 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.022+0000 7efe10e7d740 10 client.21168 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.022+0000 7efe10e7d740 10 client.21168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.022+0000 7efe10e7d740 20 client.21168 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.022+0000 7efe10e7d740 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56039438dd80 0x560394796630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.022+0000 7efe10e7d740 4 client.21168 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.022+0000 7efe10e7d740 1 -- 172.21.15.63:0/4020311165 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603946e4ae0 con 0x56039438dd80 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.022+0000 7efdfbe00640 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56039438dd80 0x560394796630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.023+0000 7efdfbe00640 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56039438dd80 0x560394796630 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e04a3016cebc4a server_cookie=d900c6341fa997c2 in_seq=0 out_seq=0 2024-05-27T08:03:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.023+0000 7efdf0c00640 10 client.21168 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.025+0000 7efdf0c00640 1 -- 172.21.15.63:0/4020311165 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5603946e4ae0 con 0x56039438dd80 2024-05-27T08:03:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.025+0000 7efdf0c00640 10 client.21168 handle_command_reply: tid=0 2024-05-27T08:03:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.025+0000 7efe10e7d740 1 client.21168 shutdown 2024-05-27T08:03:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 1 -- 172.21.15.63:0/4020311165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943a62f0 msgr2=0x5603947844b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943a62f0 0x5603947844b0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7efdf40129b0 tx=0x7efdf40046c0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 1 -- 172.21.15.63:0/4020311165 shutdown_connections 2024-05-27T08:03:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56039438dd80 0x560394796630 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x560394790500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943ae4d0 0x5603947849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 1 --2- 172.21.15.63:0/4020311165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943a62f0 0x5603947844b0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efdf0c00640 0 client.21168 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 1 -- 172.21.15.63:0/4020311165 >> 172.21.15.63:0/4020311165 conn(0x5603943c3300 msgr2=0x560394779ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 1 -- 172.21.15.63:0/4020311165 shutdown_connections 2024-05-27T08:03:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 1 -- 172.21.15.63:0/4020311165 wait complete. 2024-05-27T08:03:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.026+0000 7efe10e7d740 20 client.21168 trim_cache size 0 max 16384 2024-05-27T08:03:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.050+0000 7efe10e7d740 1 Processor -- start 2024-05-27T08:03:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.050+0000 7efe10e7d740 1 -- start start 2024-05-27T08:03:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.050+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x56039476d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.050+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943ae4d0 0x560394790270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.050+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x5603947907b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.051+0000 7efe10e7d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b1890 con 0x5603943ae4d0 2024-05-27T08:03:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.051+0000 7efe10e7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe041af160 con 0x5603943a62f0 2024-05-27T08:03:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.051+0000 7efe10e7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe041ad030 con 0x5603943cf1b0 2024-05-27T08:03:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.051+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943ae4d0 0x560394790270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.051+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943ae4d0 0x560394790270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54400/0 (socket says 172.21.15.63:54400) 2024-05-27T08:03:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.051+0000 7efdfbe00640 1 -- 172.21.15.63:0/2661059300 learned_addr learned my addr 172.21.15.63:0/2661059300 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.051+0000 7efe02a00640 1 --2- 172.21.15.63:0/2661059300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x56039476d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.051+0000 7efe03400640 1 --2- 172.21.15.63:0/2661059300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x5603947907b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.052+0000 7efdfbe00640 1 -- 172.21.15.63:0/2661059300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 msgr2=0x5603947907b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.052+0000 7efdfbe00640 1 --2- 172.21.15.63:0/2661059300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x5603947907b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.052+0000 7efdfbe00640 1 -- 172.21.15.63:0/2661059300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 msgr2=0x56039476d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.052+0000 7efdfbe00640 1 --2- 172.21.15.63:0/2661059300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x56039476d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.052+0000 7efdfbe00640 1 -- 172.21.15.63:0/2661059300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039477a8d0 con 0x5603943ae4d0 2024-05-27T08:03:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.052+0000 7efdfbe00640 1 --2- 172.21.15.63:0/2661059300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943ae4d0 0x560394790270 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7efdec0ca180 tx=0x7efdec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4e53e40393b3a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.052+0000 7efdf3400640 1 -- 172.21.15.63:0/2661059300 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec049040 con 0x5603943ae4d0 2024-05-27T08:03:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.052+0000 7efdf3400640 1 -- 172.21.15.63:0/2661059300 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdec003e40 con 0x5603943ae4d0 2024-05-27T08:03:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.052+0000 7efdf3400640 1 -- 172.21.15.63:0/2661059300 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0d3050 con 0x5603943ae4d0 2024-05-27T08:03:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.053+0000 7efe10e7d740 1 -- 172.21.15.63:0/2661059300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943ae4d0 msgr2=0x560394790270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.053+0000 7efe10e7d740 1 --2- 172.21.15.63:0/2661059300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943ae4d0 0x560394790270 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7efdec0ca180 tx=0x7efdec000bc0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.054+0000 7efe10e7d740 1 -- 172.21.15.63:0/2661059300 shutdown_connections 2024-05-27T08:03:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.054+0000 7efe10e7d740 1 --2- 172.21.15.63:0/2661059300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x5603947907b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.054+0000 7efe10e7d740 1 --2- 172.21.15.63:0/2661059300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943ae4d0 0x560394790270 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.054+0000 7efe10e7d740 1 --2- 172.21.15.63:0/2661059300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x56039476d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.054+0000 7efe10e7d740 1 -- 172.21.15.63:0/2661059300 >> 172.21.15.63:0/2661059300 conn(0x5603943c3300 msgr2=0x5603947859c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.054+0000 7efe10e7d740 1 -- 172.21.15.63:0/2661059300 shutdown_connections 2024-05-27T08:03:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.054+0000 7efe10e7d740 1 -- 172.21.15.63:0/2661059300 wait complete. 2024-05-27T08:03:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.054+0000 7efe10e7d740 1 -- start start 2024-05-27T08:03:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.055+0000 7efe10e7d740 1 Processor -- start 2024-05-27T08:03:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.055+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x560394787830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.055+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x560394787d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.055+0000 7efe10e7d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943631c0 0x5603947fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.055+0000 7efe10e7d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe041ad680 con 0x5603943631c0 2024-05-27T08:03:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.055+0000 7efe10e7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe041aea90 con 0x5603943a62f0 2024-05-27T08:03:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.055+0000 7efe10e7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe041adcf0 con 0x5603943cf1b0 2024-05-27T08:03:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.055+0000 7efe10e7d740 10 client.0 _ll_register_callbacks cb 0x7efdf400a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.056+0000 7efe03400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943631c0 0x5603947fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.056+0000 7efe02a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x560394787830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.056+0000 7efe03400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943631c0 0x5603947fccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54414/0 (socket says 172.21.15.63:54414) 2024-05-27T08:03:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.056+0000 7efe03400640 1 -- 172.21.15.63:0/907332786 learned_addr learned my addr 172.21.15.63:0/907332786 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.056+0000 7efdfbe00640 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x560394787d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.057+0000 7efe02a00640 1 -- 172.21.15.63:0/907332786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 msgr2=0x560394787d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.057+0000 7efe02a00640 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x560394787d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.057+0000 7efe02a00640 1 -- 172.21.15.63:0/907332786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943631c0 msgr2=0x5603947fccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.057+0000 7efe02a00640 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943631c0 0x5603947fccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.057+0000 7efe02a00640 1 -- 172.21.15.63:0/907332786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603947fd230 con 0x5603943a62f0 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.057+0000 7efe03400640 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943631c0 0x5603947fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.057+0000 7efe02a00640 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x560394787830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efdf4031a80 tx=0x7efdf4031d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48e82731df65b3b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.057+0000 7efdf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.058+0000 7efdf0c00640 1 -- 172.21.15.63:0/907332786 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4040060 con 0x5603943a62f0 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.058+0000 7efe10e7d740 1 -- 172.21.15.63:0/907332786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603946de2c0 con 0x5603943a62f0 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.058+0000 7efdf0c00640 1 -- 172.21.15.63:0/907332786 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7efdf4005040 con 0x5603943a62f0 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.058+0000 7efdf0c00640 1 -- 172.21.15.63:0/907332786 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efdf4047050 con 0x5603943a62f0 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.058+0000 7efe10e7d740 10 client.40277 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.058+0000 7efe10e7d740 1 -- 172.21.15.63:0/907332786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5603947fe340 con 0x5603943a62f0 2024-05-27T08:03:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.059+0000 7efdf0c00640 1 -- 172.21.15.63:0/907332786 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5603946de2c0 con 0x5603943a62f0 2024-05-27T08:03:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.059+0000 7efdf0c00640 1 -- 172.21.15.63:0/907332786 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7efdf4004c20 con 0x5603943a62f0 2024-05-27T08:03:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.059+0000 7efe10e7d740 10 client.40277 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.059+0000 7efe10e7d740 10 client.40277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.059+0000 7efe10e7d740 20 client.40277 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.059+0000 7efe10e7d740 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5603947fea30 0x560394800e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.059+0000 7efe10e7d740 4 client.40277 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:03:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.059+0000 7efe10e7d740 1 -- 172.21.15.63:0/907332786 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603946584b0 con 0x5603947fea30 2024-05-27T08:03:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.060+0000 7efdfbe00640 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5603947fea30 0x560394800e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.061+0000 7efdf0c00640 10 client.40277 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.061+0000 7efdfbe00640 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5603947fea30 0x560394800e50 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9429f8b64e9f02b server_cookie=9de92189718f7bc4 in_seq=0 out_seq=0 2024-05-27T08:03:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.100+0000 7efdf0c00640 1 -- 172.21.15.63:0/907332786 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5603946584b0 con 0x5603947fea30 2024-05-27T08:03:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.100+0000 7efdf0c00640 10 client.40277 handle_command_reply: tid=0 2024-05-27T08:03:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.100+0000 7efe10e7d740 1 client.40277 shutdown 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 1 -- 172.21.15.63:0/907332786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 msgr2=0x560394787830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x560394787830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efdf4031a80 tx=0x7efdf4031d30 comp rx=0 tx=0).stop 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 1 -- 172.21.15.63:0/907332786 shutdown_connections 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5603947fea30 0x560394800e50 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603943631c0 0x5603947fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5603943cf1b0 0x560394787d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efdf0c00640 0 client.40277 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 1 --2- 172.21.15.63:0/907332786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603943a62f0 0x560394787830 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 1 -- 172.21.15.63:0/907332786 >> 172.21.15.63:0/907332786 conn(0x5603943c3300 msgr2=0x560394784560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 1 -- 172.21.15.63:0/907332786 shutdown_connections 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 1 -- 172.21.15.63:0/907332786 wait complete. 2024-05-27T08:03:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.101+0000 7efe10e7d740 20 client.40277 trim_cache size 0 max 16384 2024-05-27T08:03:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:03:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:03:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:03:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 -- 172.21.15.63:0/3166256749 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde80b2950 msgr2=0x7efde80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde80b2950 0x7efde80b4e60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56039477b420 tx=0x7efdf400a2d0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 -- 172.21.15.63:0/3166256749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe04109c80 msgr2=0x7efe0419dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe04109c80 0x7efe0419dd00 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7efdec005940 tx=0x7efdec0042f0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 -- 172.21.15.63:0/3166256749 shutdown_connections 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7efde80b2950 0x7efde80b4e60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe0410a120 0x7efe041a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe04109c80 0x7efe0419dd00 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 --2- 172.21.15.63:0/3166256749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe041043d0 0x7efe0419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 -- 172.21.15.63:0/3166256749 >> 172.21.15.63:0/3166256749 conn(0x7efe04079290 msgr2=0x7efe041003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 -- 172.21.15.63:0/3166256749 shutdown_connections 2024-05-27T08:03:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:20.102+0000 7efe09e00640 1 -- 172.21.15.63:0/3166256749 wait complete. 2024-05-27T08:03:21.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:21 smithi086 ceph-mon[34687]: pgmap v945: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.4 MiB/s wr, 52 op/s 2024-05-27T08:03:21.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:21 smithi149 ceph-mon[35408]: pgmap v945: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.4 MiB/s wr, 52 op/s 2024-05-27T08:03:21.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:21 smithi063 ceph-mon[31613]: pgmap v945: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.4 MiB/s wr, 52 op/s 2024-05-27T08:03:22.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:03:22.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:03:22.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:03:22.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:03:22.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:03:22.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:03:22.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:03:22.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:03:22.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:03:23.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:23 smithi086 ceph-mon[34687]: pgmap v946: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.2 MiB/s wr, 53 op/s 2024-05-27T08:03:23.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:23 smithi149 ceph-mon[35408]: pgmap v946: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.2 MiB/s wr, 53 op/s 2024-05-27T08:03:23.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:23 smithi063 ceph-mon[31613]: pgmap v946: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.2 MiB/s wr, 53 op/s 2024-05-27T08:03:25.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:25 smithi086 ceph-mon[34687]: pgmap v947: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 42 op/s 2024-05-27T08:03:25.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:25 smithi149 ceph-mon[35408]: pgmap v947: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 42 op/s 2024-05-27T08:03:25.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:25 smithi063 ceph-mon[31613]: pgmap v947: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 42 op/s 2024-05-27T08:03:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:27 smithi086 ceph-mon[34687]: pgmap v948: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 42 op/s 2024-05-27T08:03:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:27 smithi149 ceph-mon[35408]: pgmap v948: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 42 op/s 2024-05-27T08:03:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:27 smithi063 ceph-mon[31613]: pgmap v948: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 42 op/s 2024-05-27T08:03:29.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:29 smithi086 ceph-mon[34687]: pgmap v949: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.0 MiB/s wr, 46 op/s 2024-05-27T08:03:29.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:29 smithi149 ceph-mon[35408]: pgmap v949: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.0 MiB/s wr, 46 op/s 2024-05-27T08:03:29.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:29 smithi063 ceph-mon[31613]: pgmap v949: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.0 MiB/s wr, 46 op/s 2024-05-27T08:03:31.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:31 smithi086 ceph-mon[34687]: pgmap v950: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:03:31.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:31 smithi149 ceph-mon[35408]: pgmap v950: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:03:31.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:31 smithi063 ceph-mon[31613]: pgmap v950: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:03:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.418+0000 7f26a8a00640 1 -- 172.21.15.63:0/1907333743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a4075420 msgr2=0x7f26a4075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.418+0000 7f26a8a00640 1 --2- 172.21.15.63:0/1907333743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a4075420 0x7f26a4075830 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f2694009900 tx=0x7f269402a4a0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.418+0000 7f26a8a00640 1 -- 172.21.15.63:0/1907333743 shutdown_connections 2024-05-27T08:03:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.418+0000 7f26a8a00640 1 --2- 172.21.15.63:0/1907333743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a4110300 0x7f26a4112720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.418+0000 7f26a8a00640 1 --2- 172.21.15.63:0/1907333743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a410d9a0 0x7f26a410fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.418+0000 7f26a8a00640 1 --2- 172.21.15.63:0/1907333743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a4075420 0x7f26a4075830 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.418+0000 7f26a8a00640 1 -- 172.21.15.63:0/1907333743 >> 172.21.15.63:0/1907333743 conn(0x7f26a4079140 msgr2=0x7f26a4079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.418+0000 7f26a8a00640 1 -- 172.21.15.63:0/1907333743 shutdown_connections 2024-05-27T08:03:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.418+0000 7f26a8a00640 1 -- 172.21.15.63:0/1907333743 wait complete. 2024-05-27T08:03:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.419+0000 7f26a8a00640 1 Processor -- start 2024-05-27T08:03:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.420+0000 7f26a8a00640 1 -- start start 2024-05-27T08:03:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.420+0000 7f26a8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a4075420 0x7f26a41a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.420+0000 7f26a8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a410d9a0 0x7f26a41a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.420+0000 7f26a8a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a4110300 0x7f26a419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.420+0000 7f26a8a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4115df0 con 0x7f26a4110300 2024-05-27T08:03:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.420+0000 7f26a8a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4115c70 con 0x7f26a410d9a0 2024-05-27T08:03:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.420+0000 7f26a8a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4115f70 con 0x7f26a4075420 2024-05-27T08:03:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.421+0000 7f26a3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a4110300 0x7f26a419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.421+0000 7f26a3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a4110300 0x7f26a419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46046/0 (socket says 172.21.15.63:46046) 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.421+0000 7f26a3e00640 1 -- 172.21.15.63:0/2413205655 learned_addr learned my addr 172.21.15.63:0/2413205655 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.421+0000 7f26a2a00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a410d9a0 0x7f26a41a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.421+0000 7f26a3e00640 1 -- 172.21.15.63:0/2413205655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a4075420 msgr2=0x7f26a41a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.421+0000 7f26a3e00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a4075420 0x7f26a41a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.421+0000 7f26a3e00640 1 -- 172.21.15.63:0/2413205655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a410d9a0 msgr2=0x7f26a41a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.421+0000 7f26a3e00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a410d9a0 0x7f26a41a2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.421+0000 7f26a3e00640 1 -- 172.21.15.63:0/2413205655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a419ce00 con 0x7f26a4110300 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.422+0000 7f26a3e00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a4110300 0x7f26a419c6d0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f269c00b990 tx=0x7f269c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.422+0000 7f269be00640 1 -- 172.21.15.63:0/2413205655 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c019070 con 0x7f26a4110300 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.422+0000 7f26a8a00640 1 -- 172.21.15.63:0/2413205655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a419d090 con 0x7f26a4110300 2024-05-27T08:03:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.422+0000 7f269be00640 1 -- 172.21.15.63:0/2413205655 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f269c00cba0 con 0x7f26a4110300 2024-05-27T08:03:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.422+0000 7f269be00640 1 -- 172.21.15.63:0/2413205655 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c0075d0 con 0x7f26a4110300 2024-05-27T08:03:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.422+0000 7f26a8a00640 1 -- 172.21.15.63:0/2413205655 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a41a92f0 con 0x7f26a4110300 2024-05-27T08:03:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.427+0000 7f26b0ff8740 1 Processor -- start 2024-05-27T08:03:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.427+0000 7f269be00640 1 -- 172.21.15.63:0/2413205655 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f269c01c070 con 0x7f26a4110300 2024-05-27T08:03:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.427+0000 7f26b0ff8740 1 -- start start 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c820a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c4552f0 0x55b66c820fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c45d4d0 0x55b66c8214e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b66c791b90 con 0x55b66c45d4d0 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b66c781730 con 0x55b66c4552f0 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b66c785770 con 0x55b66c47e1b0 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26a2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c45d4d0 0x55b66c8214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26a2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c45d4d0 0x55b66c8214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46062/0 (socket says 172.21.15.63:46062) 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26a2a00640 1 -- 172.21.15.63:0/2538476709 learned_addr learned my addr 172.21.15.63:0/2538476709 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26a2a00640 1 --2- 172.21.15.63:0/2538476709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c820a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.428+0000 7f26a3400640 1 --2- 172.21.15.63:0/2538476709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c4552f0 0x55b66c820fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.429+0000 7f26a2a00640 1 -- 172.21.15.63:0/2538476709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 msgr2=0x55b66c820a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.429+0000 7f26a2a00640 1 --2- 172.21.15.63:0/2538476709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c820a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.429+0000 7f26a2a00640 1 -- 172.21.15.63:0/2538476709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c4552f0 msgr2=0x55b66c820fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.429+0000 7f26a2a00640 1 --2- 172.21.15.63:0/2538476709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c4552f0 0x55b66c820fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.429+0000 7f26a2a00640 1 -- 172.21.15.63:0/2538476709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b66c71a8b0 con 0x55b66c45d4d0 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.429+0000 7f26a3400640 1 --2- 172.21.15.63:0/2538476709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c4552f0 0x55b66c820fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.430+0000 7f269be00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26880b29d0 0x7f26880b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.430+0000 7f26a2a00640 1 --2- 172.21.15.63:0/2538476709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c45d4d0 0x55b66c8214e0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x55b66c81dbd0 tx=0x7f2690005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5de5f28bcfdbe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.430+0000 7f269be00640 1 -- 172.21.15.63:0/2413205655 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f269c0e1fe0 con 0x7f26a4110300 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.430+0000 7f26a3400640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26880b29d0 0x7f26880b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.430+0000 7f268fe00640 1 -- 172.21.15.63:0/2538476709 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690012ca0 con 0x55b66c45d4d0 2024-05-27T08:03:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.430+0000 7f268fe00640 1 -- 172.21.15.63:0/2538476709 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2690012e40 con 0x55b66c45d4d0 2024-05-27T08:03:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.430+0000 7f268fe00640 1 -- 172.21.15.63:0/2538476709 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26900047a0 con 0x55b66c45d4d0 2024-05-27T08:03:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.432+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2538476709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c45d4d0 msgr2=0x55b66c8214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.432+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/2538476709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c45d4d0 0x55b66c8214e0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x55b66c81dbd0 tx=0x7f2690005c90 comp rx=0 tx=0).stop 2024-05-27T08:03:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.432+0000 7f26a3400640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26880b29d0 0x7f26880b4ee0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55b66c81e550 tx=0x7f2694000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.432+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2538476709 shutdown_connections 2024-05-27T08:03:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.432+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/2538476709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c45d4d0 0x55b66c8214e0 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.432+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/2538476709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c4552f0 0x55b66c820fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.432+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/2538476709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c820a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.432+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2538476709 >> 172.21.15.63:0/2538476709 conn(0x55b66c472300 msgr2=0x55b66c81d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.432+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2538476709 shutdown_connections 2024-05-27T08:03:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.432+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2538476709 wait complete. 2024-05-27T08:03:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.433+0000 7f26b0ff8740 1 -- start start 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.433+0000 7f26b0ff8740 1 Processor -- start 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.434+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.434+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c45d4d0 0x55b66c82b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.434+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c836ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.434+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b66c467790 con 0x55b66c4552f0 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.434+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b66c729bc0 con 0x55b66c45d4d0 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.434+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b66c82db90 con 0x55b66c47e1b0 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.434+0000 7f26b0ff8740 10 client.0 _ll_register_callbacks cb 0x55b66c707400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.434+0000 7f26a3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.434+0000 7f26a3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46076/0 (socket says 172.21.15.63:46076) 2024-05-27T08:03:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.434+0000 7f26a3400640 1 -- 172.21.15.63:0/2265381943 learned_addr learned my addr 172.21.15.63:0/2265381943 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.435+0000 7f26a3e00640 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c836ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.435+0000 7f26a2a00640 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c45d4d0 0x55b66c82b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.435+0000 7f26a3400640 1 -- 172.21.15.63:0/2265381943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 msgr2=0x55b66c836ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.435+0000 7f26a3400640 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c836ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.435+0000 7f26a3400640 1 -- 172.21.15.63:0/2265381943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c45d4d0 msgr2=0x55b66c82b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.435+0000 7f26a3400640 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c45d4d0 0x55b66c82b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.435+0000 7f26a3400640 1 -- 172.21.15.63:0/2265381943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b66c496210 con 0x55b66c4552f0 2024-05-27T08:03:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.435+0000 7f26a3e00640 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c836ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.436+0000 7f26a3400640 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82ac80 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f2694033000 tx=0x7f2694005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1a4822e8868669 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.436+0000 7f268cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.436+0000 7f268cc00640 1 -- 172.21.15.63:0/2265381943 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269404d020 con 0x55b66c4552f0 2024-05-27T08:03:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.436+0000 7f268cc00640 1 -- 172.21.15.63:0/2265381943 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f2694004ad0 con 0x55b66c4552f0 2024-05-27T08:03:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.436+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2265381943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b66c793ae0 con 0x55b66c4552f0 2024-05-27T08:03:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.436+0000 7f268cc00640 1 -- 172.21.15.63:0/2265381943 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2694058050 con 0x55b66c4552f0 2024-05-27T08:03:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.436+0000 7f26b0ff8740 10 client.21186 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.436+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2265381943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b66c562620 con 0x55b66c4552f0 2024-05-27T08:03:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.437+0000 7f268cc00640 1 -- 172.21.15.63:0/2265381943 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b66c793ae0 con 0x55b66c4552f0 2024-05-27T08:03:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.437+0000 7f268cc00640 1 -- 172.21.15.63:0/2265381943 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2694049480 con 0x55b66c4552f0 2024-05-27T08:03:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.437+0000 7f26b0ff8740 10 client.21186 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.437+0000 7f26b0ff8740 10 client.21186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.437+0000 7f26b0ff8740 20 client.21186 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.437+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b66c43cd80 0x55b66c83cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.437+0000 7f26b0ff8740 4 client.21186 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.437+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2265381943 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b66c78d2c0 con 0x55b66c43cd80 2024-05-27T08:03:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.438+0000 7f26a2a00640 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b66c43cd80 0x55b66c83cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.439+0000 7f26a2a00640 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b66c43cd80 0x55b66c83cf40 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a855647e5bd2c9 server_cookie=d0449e33d0cdbd27 in_seq=0 out_seq=0 2024-05-27T08:03:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.439+0000 7f268cc00640 10 client.21186 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.441+0000 7f268cc00640 1 -- 172.21.15.63:0/2265381943 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b66c78d2c0 con 0x55b66c43cd80 2024-05-27T08:03:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.441+0000 7f268cc00640 10 client.21186 handle_command_reply: tid=0 2024-05-27T08:03:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.441+0000 7f26b0ff8740 1 client.21186 shutdown 2024-05-27T08:03:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.441+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2265381943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 msgr2=0x55b66c82ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.441+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82ac80 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f2694033000 tx=0x7f2694005fd0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.441+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2265381943 shutdown_connections 2024-05-27T08:03:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.441+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b66c43cd80 0x55b66c83cf40 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.442+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c836ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.442+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c45d4d0 0x55b66c82b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.442+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/2265381943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82ac80 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.442+0000 7f268cc00640 0 client.21186 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.442+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2265381943 >> 172.21.15.63:0/2265381943 conn(0x55b66c472300 msgr2=0x55b66c822780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.442+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2265381943 shutdown_connections 2024-05-27T08:03:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.442+0000 7f26b0ff8740 1 -- 172.21.15.63:0/2265381943 wait complete. 2024-05-27T08:03:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.442+0000 7f26b0ff8740 20 client.21186 trim_cache size 0 max 16384 2024-05-27T08:03:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.464+0000 7f26b0ff8740 1 Processor -- start 2024-05-27T08:03:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.464+0000 7f26b0ff8740 1 -- start start 2024-05-27T08:03:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.465+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c826480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.465+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c45d4d0 0x55b66c836c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.465+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c837170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.465+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b5b70 con 0x55b66c4552f0 2024-05-27T08:03:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.465+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b3440 con 0x55b66c45d4d0 2024-05-27T08:03:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.465+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b12b0 con 0x55b66c47e1b0 2024-05-27T08:03:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.465+0000 7f26a3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c826480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.465+0000 7f26a3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c826480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46086/0 (socket says 172.21.15.63:46086) 2024-05-27T08:03:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.465+0000 7f26a3400640 1 -- 172.21.15.63:0/3010113807 learned_addr learned my addr 172.21.15.63:0/3010113807 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.466+0000 7f26a3e00640 1 --2- 172.21.15.63:0/3010113807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c837170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.466+0000 7f26a2a00640 1 --2- 172.21.15.63:0/3010113807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c45d4d0 0x55b66c836c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.466+0000 7f26a3400640 1 -- 172.21.15.63:0/3010113807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 msgr2=0x55b66c837170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.466+0000 7f26a3400640 1 --2- 172.21.15.63:0/3010113807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c837170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.466+0000 7f26a3400640 1 -- 172.21.15.63:0/3010113807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c45d4d0 msgr2=0x55b66c836c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.466+0000 7f26a3400640 1 --2- 172.21.15.63:0/3010113807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c45d4d0 0x55b66c836c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.466+0000 7f26a3400640 1 -- 172.21.15.63:0/3010113807 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b66c81b610 con 0x55b66c4552f0 2024-05-27T08:03:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.466+0000 7f26a3400640 1 --2- 172.21.15.63:0/3010113807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c826480 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f2694004650 tx=0x7f26940042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7771dc874ac4d86f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.467+0000 7f268f400640 1 -- 172.21.15.63:0/3010113807 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269404d070 con 0x55b66c4552f0 2024-05-27T08:03:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.467+0000 7f268f400640 1 -- 172.21.15.63:0/3010113807 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2694060030 con 0x55b66c4552f0 2024-05-27T08:03:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.467+0000 7f268f400640 1 -- 172.21.15.63:0/3010113807 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694035040 con 0x55b66c4552f0 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.468+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3010113807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 msgr2=0x55b66c826480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.468+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/3010113807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c826480 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f2694004650 tx=0x7f26940042d0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.468+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3010113807 shutdown_connections 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.468+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/3010113807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c47e1b0 0x55b66c837170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.468+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/3010113807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c45d4d0 0x55b66c836c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.468+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/3010113807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c826480 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.468+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3010113807 >> 172.21.15.63:0/3010113807 conn(0x55b66c472300 msgr2=0x55b66c82cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.468+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3010113807 shutdown_connections 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.468+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3010113807 wait complete. 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.468+0000 7f26b0ff8740 1 -- start start 2024-05-27T08:03:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.469+0000 7f26b0ff8740 1 Processor -- start 2024-05-27T08:03:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.469+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.469+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c45d4d0 0x55b66c82eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.469+0000 7f26b0ff8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c47e1b0 0x55b66c89d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.469+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b1900 con 0x55b66c4552f0 2024-05-27T08:03:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.469+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b2d70 con 0x55b66c47e1b0 2024-05-27T08:03:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.469+0000 7f26b0ff8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b1fd0 con 0x55b66c45d4d0 2024-05-27T08:03:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.469+0000 7f26b0ff8740 10 client.0 _ll_register_callbacks cb 0x7f2694040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.469+0000 7f26a3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46096/0 (socket says 172.21.15.63:46096) 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a3400640 1 -- 172.21.15.63:0/3538780827 learned_addr learned my addr 172.21.15.63:0/3538780827 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a2a00640 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c45d4d0 0x55b66c82eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a3e00640 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c47e1b0 0x55b66c89d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a3400640 1 -- 172.21.15.63:0/3538780827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c45d4d0 msgr2=0x55b66c82eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a3400640 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c45d4d0 0x55b66c82eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a3400640 1 -- 172.21.15.63:0/3538780827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c47e1b0 msgr2=0x55b66c89d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a3400640 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c47e1b0 0x55b66c89d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a3400640 1 -- 172.21.15.63:0/3538780827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b66c8207f0 con 0x55b66c4552f0 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a2a00640 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c45d4d0 0x55b66c82eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.470+0000 7f26a3e00640 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c47e1b0 0x55b66c89d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.471+0000 7f26a3400640 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82e650 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f2694005df0 tx=0x7f2694049480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4251d85f0d9d6e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.471+0000 7f268cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.471+0000 7f268cc00640 1 -- 172.21.15.63:0/3538780827 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694069070 con 0x55b66c4552f0 2024-05-27T08:03:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.471+0000 7f268cc00640 1 -- 172.21.15.63:0/3538780827 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f26940048d0 con 0x55b66c4552f0 2024-05-27T08:03:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.471+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3538780827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2694060030 con 0x55b66c4552f0 2024-05-27T08:03:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.471+0000 7f268cc00640 1 -- 172.21.15.63:0/3538780827 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2694075050 con 0x55b66c4552f0 2024-05-27T08:03:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.471+0000 7f26b0ff8740 10 client.21192 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.471+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3538780827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b66c8a4c70 con 0x55b66c4552f0 2024-05-27T08:03:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.472+0000 7f268cc00640 1 -- 172.21.15.63:0/3538780827 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2694060030 con 0x55b66c4552f0 2024-05-27T08:03:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.472+0000 7f268cc00640 1 -- 172.21.15.63:0/3538780827 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2694003c20 con 0x55b66c4552f0 2024-05-27T08:03:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.472+0000 7f26b0ff8740 10 client.21192 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.472+0000 7f26b0ff8740 10 client.21192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.472+0000 7f26b0ff8740 20 client.21192 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.472+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b66c4121c0 0x55b66c8a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.472+0000 7f26b0ff8740 4 client.21192 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:03:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.472+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3538780827 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b66c7074b0 con 0x55b66c4121c0 2024-05-27T08:03:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.473+0000 7f26a2a00640 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b66c4121c0 0x55b66c8a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.474+0000 7f26a2a00640 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b66c4121c0 0x55b66c8a72b0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b22496c8b7bf3506 server_cookie=3c95bc52a2a7aa70 in_seq=0 out_seq=0 2024-05-27T08:03:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.474+0000 7f268cc00640 10 client.21192 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.509+0000 7f268cc00640 1 -- 172.21.15.63:0/3538780827 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b66c7074b0 con 0x55b66c4121c0 2024-05-27T08:03:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.509+0000 7f268cc00640 10 client.21192 handle_command_reply: tid=0 2024-05-27T08:03:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.509+0000 7f26b0ff8740 1 client.21192 shutdown 2024-05-27T08:03:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.509+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3538780827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 msgr2=0x55b66c82e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.509+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82e650 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f2694005df0 tx=0x7f2694049480 comp rx=0 tx=0).stop 2024-05-27T08:03:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.510+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3538780827 shutdown_connections 2024-05-27T08:03:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.510+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b66c4121c0 0x55b66c8a72b0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.510+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b66c47e1b0 0x55b66c89d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.510+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b66c45d4d0 0x55b66c82eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.510+0000 7f26b0ff8740 1 --2- 172.21.15.63:0/3538780827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b66c4552f0 0x55b66c82e650 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.510+0000 7f268cc00640 0 client.21192 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.510+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3538780827 >> 172.21.15.63:0/3538780827 conn(0x55b66c472300 msgr2=0x55b66c8209c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.510+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3538780827 shutdown_connections 2024-05-27T08:03:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.510+0000 7f26b0ff8740 1 -- 172.21.15.63:0/3538780827 wait complete. 2024-05-27T08:03:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.510+0000 7f26b0ff8740 20 client.21192 trim_cache size 0 max 16384 2024-05-27T08:03:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:03:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:03:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 -- 172.21.15.63:0/2413205655 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26880b29d0 msgr2=0x7f26880b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26880b29d0 0x7f26880b4ee0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55b66c81e550 tx=0x7f2694000bc0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 -- 172.21.15.63:0/2413205655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a4110300 msgr2=0x7f26a419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a4110300 0x7f26a419c6d0 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f269c00b990 tx=0x7f269c00be50 comp rx=0 tx=0).stop 2024-05-27T08:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 -- 172.21.15.63:0/2413205655 shutdown_connections 2024-05-27T08:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f26880b29d0 0x7f26880b4ee0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26a4110300 0x7f26a419c6d0 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26a410d9a0 0x7f26a41a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 --2- 172.21.15.63:0/2413205655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26a4075420 0x7f26a41a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 -- 172.21.15.63:0/2413205655 >> 172.21.15.63:0/2413205655 conn(0x7f26a4079140 msgr2=0x7f26a4102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 -- 172.21.15.63:0/2413205655 shutdown_connections 2024-05-27T08:03:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:33.511+0000 7f26a8a00640 1 -- 172.21.15.63:0/2413205655 wait complete. 2024-05-27T08:03:33.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:33 smithi086 ceph-mon[34687]: pgmap v951: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:03:33.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:33 smithi149 ceph-mon[35408]: pgmap v951: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:03:33.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:33 smithi063 ceph-mon[31613]: pgmap v951: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:03:35.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:35 smithi086 ceph-mon[34687]: pgmap v952: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T08:03:35.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:35 smithi149 ceph-mon[35408]: pgmap v952: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T08:03:35.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:35 smithi063 ceph-mon[31613]: pgmap v952: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T08:03:37.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:37 smithi086 ceph-mon[34687]: pgmap v953: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 8 op/s 2024-05-27T08:03:37.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:37 smithi149 ceph-mon[35408]: pgmap v953: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 8 op/s 2024-05-27T08:03:37.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:37 smithi063 ceph-mon[31613]: pgmap v953: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 8 op/s 2024-05-27T08:03:39.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:39 smithi086 ceph-mon[34687]: pgmap v954: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.4 MiB/s wr, 14 op/s 2024-05-27T08:03:39.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:39 smithi149 ceph-mon[35408]: pgmap v954: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.4 MiB/s wr, 14 op/s 2024-05-27T08:03:39.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:39 smithi063 ceph-mon[31613]: pgmap v954: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.4 MiB/s wr, 14 op/s 2024-05-27T08:03:41.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:41 smithi086 ceph-mon[34687]: pgmap v955: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:03:41.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:41 smithi149 ceph-mon[35408]: pgmap v955: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:03:41.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:41 smithi063 ceph-mon[31613]: pgmap v955: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:03:43.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:43 smithi086 ceph-mon[34687]: pgmap v956: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:03:43.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:43 smithi149 ceph-mon[35408]: pgmap v956: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:03:43.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:43 smithi063 ceph-mon[31613]: pgmap v956: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:03:45.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:45 smithi086 ceph-mon[34687]: pgmap v957: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:03:45.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:45 smithi149 ceph-mon[35408]: pgmap v957: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:03:45.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:45 smithi063 ceph-mon[31613]: pgmap v957: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.782+0000 7f7205200640 1 -- 172.21.15.63:0/2681304965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f720010b6b0 msgr2=0x7f720010dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.782+0000 7f7205200640 1 --2- 172.21.15.63:0/2681304965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f720010b6b0 0x7f720010dad0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009900 tx=0x7f71ec02a480 comp rx=0 tx=0).stop 2024-05-27T08:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.782+0000 7f7205200640 1 -- 172.21.15.63:0/2681304965 shutdown_connections 2024-05-27T08:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.782+0000 7f7205200640 1 --2- 172.21.15.63:0/2681304965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720010e010 0x7f7200110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.782+0000 7f7205200640 1 --2- 172.21.15.63:0/2681304965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f720010b6b0 0x7f720010dad0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.782+0000 7f7205200640 1 --2- 172.21.15.63:0/2681304965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7200104c50 0x7f7200105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.782+0000 7f7205200640 1 -- 172.21.15.63:0/2681304965 >> 172.21.15.63:0/2681304965 conn(0x7f72000fe210 msgr2=0x7f7200100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.782+0000 7f7205200640 1 -- 172.21.15.63:0/2681304965 shutdown_connections 2024-05-27T08:03:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.782+0000 7f7205200640 1 -- 172.21.15.63:0/2681304965 wait complete. 2024-05-27T08:03:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.784+0000 7f7205200640 1 Processor -- start 2024-05-27T08:03:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.784+0000 7f7205200640 1 -- start start 2024-05-27T08:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.784+0000 7f7205200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200104c50 0x7f7200102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.784+0000 7f7205200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f720010b6b0 0x7f7200102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.784+0000 7f7205200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f720010e010 0x7f7200102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.785+0000 7f7205200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7200113b20 con 0x7f720010e010 2024-05-27T08:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.785+0000 7f7205200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72001139a0 con 0x7f7200104c50 2024-05-27T08:03:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.785+0000 7f7205200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7200113ca0 con 0x7f720010b6b0 2024-05-27T08:03:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.785+0000 7f71ffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f720010e010 0x7f7200102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.785+0000 7f71ffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f720010e010 0x7f7200102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49692/0 (socket says 172.21.15.63:49692) 2024-05-27T08:03:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.785+0000 7f71ffe00640 1 -- 172.21.15.63:0/1010942213 learned_addr learned my addr 172.21.15.63:0/1010942213 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.785+0000 7f71ff400640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200104c50 0x7f7200102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.785+0000 7f71fea00640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f720010b6b0 0x7f7200102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.785+0000 7f71ffe00640 1 -- 172.21.15.63:0/1010942213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f720010b6b0 msgr2=0x7f7200102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.786+0000 7f71ffe00640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f720010b6b0 0x7f7200102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.786+0000 7f71ffe00640 1 -- 172.21.15.63:0/1010942213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200104c50 msgr2=0x7f7200102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.786+0000 7f71ffe00640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200104c50 0x7f7200102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.786+0000 7f71ffe00640 1 -- 172.21.15.63:0/1010942213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72001a8960 con 0x7f720010e010 2024-05-27T08:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.786+0000 7f71fea00640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f720010b6b0 0x7f7200102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.786+0000 7f71ffe00640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f720010e010 0x7f7200102f90 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f71f800eaa0 tx=0x7f71f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.786+0000 7f71f7e00640 1 -- 172.21.15.63:0/1010942213 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f800cdb0 con 0x7f720010e010 2024-05-27T08:03:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.786+0000 7f71f7e00640 1 -- 172.21.15.63:0/1010942213 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71f8005020 con 0x7f720010e010 2024-05-27T08:03:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.786+0000 7f7205200640 1 -- 172.21.15.63:0/1010942213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72001a8c50 con 0x7f720010e010 2024-05-27T08:03:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.786+0000 7f71f7e00640 1 -- 172.21.15.63:0/1010942213 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f80116d0 con 0x7f720010e010 2024-05-27T08:03:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.787+0000 7f7205200640 1 -- 172.21.15.63:0/1010942213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72001a9170 con 0x7f720010e010 2024-05-27T08:03:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.790+0000 7f720d822740 1 Processor -- start 2024-05-27T08:03:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.792+0000 7f71f7e00640 1 -- 172.21.15.63:0/1010942213 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f71f8004a40 con 0x7f720010e010 2024-05-27T08:03:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.792+0000 7f720d822740 1 -- start start 2024-05-27T08:03:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.792+0000 7f720d822740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e171b0 0x55ef541b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.792+0000 7f720d822740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53dee2f0 0x55ef541b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.792+0000 7f720d822740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53df64d0 0x55ef541ba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.792+0000 7f720d822740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ef5412ab90 con 0x55ef53df64d0 2024-05-27T08:03:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.792+0000 7f720d822740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef5411a730 con 0x55ef53dee2f0 2024-05-27T08:03:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.792+0000 7f720d822740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ef5411e770 con 0x55ef53e171b0 2024-05-27T08:03:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.793+0000 7f71fea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53df64d0 0x55ef541ba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.793+0000 7f71fea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53df64d0 0x55ef541ba4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49706/0 (socket says 172.21.15.63:49706) 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.793+0000 7f71fea00640 1 -- 172.21.15.63:0/2028489800 learned_addr learned my addr 172.21.15.63:0/2028489800 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.793+0000 7f71ff400640 1 --2- 172.21.15.63:0/2028489800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53dee2f0 0x55ef541b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.793+0000 7f71fea00640 1 --2- 172.21.15.63:0/2028489800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e171b0 0x55ef541b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.793+0000 7f71fea00640 1 -- 172.21.15.63:0/2028489800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e171b0 msgr2=0x55ef541b9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.793+0000 7f71fea00640 1 --2- 172.21.15.63:0/2028489800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e171b0 0x55ef541b9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.793+0000 7f71fea00640 1 -- 172.21.15.63:0/2028489800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53dee2f0 msgr2=0x55ef541b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.793+0000 7f71fea00640 1 --2- 172.21.15.63:0/2028489800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53dee2f0 0x55ef541b9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.793+0000 7f71fea00640 1 -- 172.21.15.63:0/2028489800 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef540b38b0 con 0x55ef53df64d0 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.794+0000 7f71fea00640 1 --2- 172.21.15.63:0/2028489800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53df64d0 0x55ef541ba4e0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005760 tx=0x7f71ec0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fef46678c33af9a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.794+0000 7f71f7e00640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f71e40b29d0 0x7f71e40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.794+0000 7f71f4c00640 1 -- 172.21.15.63:0/2028489800 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec043030 con 0x55ef53df64d0 2024-05-27T08:03:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.794+0000 7f71f4c00640 1 -- 172.21.15.63:0/2028489800 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71ec031400 con 0x55ef53df64d0 2024-05-27T08:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.794+0000 7f71f4c00640 1 -- 172.21.15.63:0/2028489800 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec035040 con 0x55ef53df64d0 2024-05-27T08:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.794+0000 7f71f7e00640 1 -- 172.21.15.63:0/1010942213 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f71f80e2750 con 0x7f720010e010 2024-05-27T08:03:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.795+0000 7f71ff400640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f71e40b29d0 0x7f71e40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.796+0000 7f720d822740 1 -- 172.21.15.63:0/2028489800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53df64d0 msgr2=0x55ef541ba4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.796+0000 7f71ff400640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f71e40b29d0 0x7f71e40b4ee0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55ef541bae10 tx=0x7f71f0005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.796+0000 7f720d822740 1 --2- 172.21.15.63:0/2028489800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53df64d0 0x55ef541ba4e0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005760 tx=0x7f71ec0056d0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.796+0000 7f720d822740 1 -- 172.21.15.63:0/2028489800 shutdown_connections 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.796+0000 7f720d822740 1 --2- 172.21.15.63:0/2028489800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53df64d0 0x55ef541ba4e0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.796+0000 7f720d822740 1 --2- 172.21.15.63:0/2028489800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53dee2f0 0x55ef541b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.796+0000 7f720d822740 1 --2- 172.21.15.63:0/2028489800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e171b0 0x55ef541b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.796+0000 7f720d822740 1 -- 172.21.15.63:0/2028489800 >> 172.21.15.63:0/2028489800 conn(0x55ef53e0b300 msgr2=0x55ef541b6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.796+0000 7f720d822740 1 -- 172.21.15.63:0/2028489800 shutdown_connections 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.796+0000 7f720d822740 1 -- 172.21.15.63:0/2028489800 wait complete. 2024-05-27T08:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.797+0000 7f720d822740 1 -- start start 2024-05-27T08:03:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.797+0000 7f720d822740 1 Processor -- start 2024-05-27T08:03:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f720d822740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53dee2f0 0x55ef541c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f720d822740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53df64d0 0x55ef541c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f720d822740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53e171b0 0x55ef541cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f720d822740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ef53e00790 con 0x55ef53e171b0 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f720d822740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef540c2bc0 con 0x55ef53df64d0 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f720d822740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ef541c6b30 con 0x55ef53dee2f0 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f720d822740 10 client.0 _ll_register_callbacks cb 0x55ef540a0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f71ffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53e171b0 0x55ef541cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f71ffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53e171b0 0x55ef541cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49714/0 (socket says 172.21.15.63:49714) 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f71ffe00640 1 -- 172.21.15.63:0/2156769480 learned_addr learned my addr 172.21.15.63:0/2156769480 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f71ff400640 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53dee2f0 0x55ef541c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.798+0000 7f71fea00640 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53df64d0 0x55ef541c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71ffe00640 1 -- 172.21.15.63:0/2156769480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53dee2f0 msgr2=0x55ef541c3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71ffe00640 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53dee2f0 0x55ef541c3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71ffe00640 1 -- 172.21.15.63:0/2156769480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53df64d0 msgr2=0x55ef541c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71ffe00640 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53df64d0 0x55ef541c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71ffe00640 1 -- 172.21.15.63:0/2156769480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef53e2f210 con 0x55ef53e171b0 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71ff400640 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53dee2f0 0x55ef541c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71fea00640 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53df64d0 0x55ef541c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71ffe00640 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53e171b0 0x55ef541cfe90 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009780 tx=0x7f71f80a8ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3e09b41ee89676 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71e9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71e9600640 1 -- 172.21.15.63:0/2156769480 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f801f040 con 0x55ef53e171b0 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f71e9600640 1 -- 172.21.15.63:0/2156769480 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f71f80a5900 con 0x55ef53e171b0 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.799+0000 7f720d822740 1 -- 172.21.15.63:0/2156769480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ec031400 con 0x55ef53e171b0 2024-05-27T08:03:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f71e9600640 1 -- 172.21.15.63:0/2156769480 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f71f80a9670 con 0x55ef53e171b0 2024-05-27T08:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f720d822740 10 client.21204 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f720d822740 1 -- 172.21.15.63:0/2156769480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ef53efb620 con 0x55ef53e171b0 2024-05-27T08:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f71e9600640 1 -- 172.21.15.63:0/2156769480 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71ec031400 con 0x55ef53e171b0 2024-05-27T08:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f71e9600640 1 -- 172.21.15.63:0/2156769480 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f71f800f040 con 0x55ef53e171b0 2024-05-27T08:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f720d822740 10 client.21204 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f720d822740 10 client.21204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f720d822740 20 client.21204 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f720d822740 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef53dd5d80 0x55ef541d5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f720d822740 4 client.21204 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.800+0000 7f720d822740 1 -- 172.21.15.63:0/2156769480 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef5412cae0 con 0x55ef53dd5d80 2024-05-27T08:03:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.801+0000 7f71ff400640 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef53dd5d80 0x55ef541d5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.802+0000 7f71ff400640 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef53dd5d80 0x55ef541d5ef0 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4531a61e6b5cf7 server_cookie=c348504ff4037fed in_seq=0 out_seq=0 2024-05-27T08:03:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.802+0000 7f71e9600640 10 client.21204 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.804+0000 7f71e9600640 1 -- 172.21.15.63:0/2156769480 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ef5412cae0 con 0x55ef53dd5d80 2024-05-27T08:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.804+0000 7f71e9600640 10 client.21204 handle_command_reply: tid=0 2024-05-27T08:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.804+0000 7f720d822740 1 client.21204 shutdown 2024-05-27T08:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.804+0000 7f720d822740 1 -- 172.21.15.63:0/2156769480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53e171b0 msgr2=0x55ef541cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.804+0000 7f720d822740 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53e171b0 0x55ef541cfe90 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009780 tx=0x7f71f80a8ed0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.805+0000 7f720d822740 1 -- 172.21.15.63:0/2156769480 shutdown_connections 2024-05-27T08:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.805+0000 7f720d822740 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef53dd5d80 0x55ef541d5ef0 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.805+0000 7f720d822740 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53e171b0 0x55ef541cfe90 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.805+0000 7f720d822740 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53df64d0 0x55ef541c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.805+0000 7f720d822740 1 --2- 172.21.15.63:0/2156769480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53dee2f0 0x55ef541c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.805+0000 7f720d822740 1 -- 172.21.15.63:0/2156769480 >> 172.21.15.63:0/2156769480 conn(0x55ef53e0b300 msgr2=0x55ef541bb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.805+0000 7f71e9600640 0 client.21204 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.805+0000 7f720d822740 1 -- 172.21.15.63:0/2156769480 shutdown_connections 2024-05-27T08:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.805+0000 7f720d822740 1 -- 172.21.15.63:0/2156769480 wait complete. 2024-05-27T08:03:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.805+0000 7f720d822740 20 client.21204 trim_cache size 0 max 16384 2024-05-27T08:03:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.826+0000 7f720d822740 1 Processor -- start 2024-05-27T08:03:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.827+0000 7f720d822740 1 -- start start 2024-05-27T08:03:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.827+0000 7f720d822740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef541bc0b0 0x55ef541c6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.827+0000 7f720d822740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e0b300 0x55ef541bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.827+0000 7f720d822740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53dee2f0 0x55ef541bdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.827+0000 7f720d822740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b5a20 con 0x55ef53dee2f0 2024-05-27T08:03:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.827+0000 7f720d822740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b32f0 con 0x55ef541bc0b0 2024-05-27T08:03:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.827+0000 7f720d822740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b1160 con 0x55ef53e0b300 2024-05-27T08:03:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.827+0000 7f71ffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53dee2f0 0x55ef541bdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.827+0000 7f71ffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53dee2f0 0x55ef541bdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49718/0 (socket says 172.21.15.63:49718) 2024-05-27T08:03:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.827+0000 7f71ffe00640 1 -- 172.21.15.63:0/2748265933 learned_addr learned my addr 172.21.15.63:0/2748265933 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.828+0000 7f71fea00640 1 --2- 172.21.15.63:0/2748265933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e0b300 0x55ef541bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.828+0000 7f71ff400640 1 --2- 172.21.15.63:0/2748265933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef541bc0b0 0x55ef541c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.828+0000 7f71ffe00640 1 -- 172.21.15.63:0/2748265933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e0b300 msgr2=0x55ef541bd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.828+0000 7f71ffe00640 1 --2- 172.21.15.63:0/2748265933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e0b300 0x55ef541bd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.828+0000 7f71ffe00640 1 -- 172.21.15.63:0/2748265933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef541bc0b0 msgr2=0x55ef541c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.828+0000 7f71ffe00640 1 --2- 172.21.15.63:0/2748265933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef541bc0b0 0x55ef541c6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.828+0000 7f71ffe00640 1 -- 172.21.15.63:0/2748265933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef541b97f0 con 0x55ef53dee2f0 2024-05-27T08:03:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.828+0000 7f71ff400640 1 --2- 172.21.15.63:0/2748265933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef541bc0b0 0x55ef541c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.828+0000 7f71ffe00640 1 --2- 172.21.15.63:0/2748265933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53dee2f0 0x55ef541bdd00 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f71f80dfb90 tx=0x7f71f80a5660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e0942d6cc194cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.829+0000 7f71ebe00640 1 -- 172.21.15.63:0/2748265933 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f80b7030 con 0x55ef53dee2f0 2024-05-27T08:03:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.829+0000 7f71ebe00640 1 -- 172.21.15.63:0/2748265933 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71f80a5320 con 0x55ef53dee2f0 2024-05-27T08:03:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.829+0000 7f71ebe00640 1 -- 172.21.15.63:0/2748265933 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f801f070 con 0x55ef53dee2f0 2024-05-27T08:03:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.830+0000 7f720d822740 1 -- 172.21.15.63:0/2748265933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53dee2f0 msgr2=0x55ef541bdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.830+0000 7f720d822740 1 --2- 172.21.15.63:0/2748265933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53dee2f0 0x55ef541bdd00 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f71f80dfb90 tx=0x7f71f80a5660 comp rx=0 tx=0).stop 2024-05-27T08:03:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.830+0000 7f720d822740 1 -- 172.21.15.63:0/2748265933 shutdown_connections 2024-05-27T08:03:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.830+0000 7f720d822740 1 --2- 172.21.15.63:0/2748265933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef53dee2f0 0x55ef541bdd00 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.830+0000 7f720d822740 1 --2- 172.21.15.63:0/2748265933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e0b300 0x55ef541bd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.830+0000 7f720d822740 1 --2- 172.21.15.63:0/2748265933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef541bc0b0 0x55ef541c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.830+0000 7f720d822740 1 -- 172.21.15.63:0/2748265933 >> 172.21.15.63:0/2748265933 conn(0x55ef541ba610 msgr2=0x55ef541c3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.830+0000 7f720d822740 1 -- 172.21.15.63:0/2748265933 shutdown_connections 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.830+0000 7f720d822740 1 -- 172.21.15.63:0/2748265933 wait complete. 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.831+0000 7f720d822740 1 -- start start 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.831+0000 7f720d822740 1 Processor -- start 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.831+0000 7f720d822740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53dee2f0 0x55ef541cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.831+0000 7f720d822740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e0b300 0x55ef541c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.831+0000 7f720d822740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef541bc0b0 0x55ef541c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.831+0000 7f720d822740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b17b0 con 0x55ef541bc0b0 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.831+0000 7f720d822740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b2c20 con 0x55ef53dee2f0 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.831+0000 7f720d822740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72001b1e80 con 0x55ef53e0b300 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.831+0000 7f720d822740 10 client.0 _ll_register_callbacks cb 0x7f71f80053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.832+0000 7f71ffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef541bc0b0 0x55ef541c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.832+0000 7f71ffe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef541bc0b0 0x55ef541c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49734/0 (socket says 172.21.15.63:49734) 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.832+0000 7f71ffe00640 1 -- 172.21.15.63:0/2681061759 learned_addr learned my addr 172.21.15.63:0/2681061759 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.832+0000 7f71ff400640 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53dee2f0 0x55ef541cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.832+0000 7f71fea00640 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e0b300 0x55ef541c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.832+0000 7f71ffe00640 1 -- 172.21.15.63:0/2681061759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e0b300 msgr2=0x55ef541c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.832+0000 7f71ffe00640 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e0b300 0x55ef541c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.832+0000 7f71ffe00640 1 -- 172.21.15.63:0/2681061759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53dee2f0 msgr2=0x55ef541cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.832+0000 7f71ffe00640 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53dee2f0 0x55ef541cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.832+0000 7f71ffe00640 1 -- 172.21.15.63:0/2681061759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef541c8080 con 0x55ef541bc0b0 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.833+0000 7f71ffe00640 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef541bc0b0 0x55ef541c7b40 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f71f80cc070 tx=0x7f71f80a50a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4de977e0fe10152 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.833+0000 7f71e9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.833+0000 7f71e9600640 1 -- 172.21.15.63:0/2681061759 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f80b7070 con 0x55ef541bc0b0 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.833+0000 7f71e9600640 1 -- 172.21.15.63:0/2681061759 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f71f80a8ae0 con 0x55ef541bc0b0 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.833+0000 7f720d822740 1 -- 172.21.15.63:0/2681061759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f80a5320 con 0x55ef541bc0b0 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.833+0000 7f71e9600640 1 -- 172.21.15.63:0/2681061759 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f71f801f040 con 0x55ef541bc0b0 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.833+0000 7f720d822740 10 client.21210 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.834+0000 7f720d822740 1 -- 172.21.15.63:0/2681061759 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ef5423e260 con 0x55ef541bc0b0 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.834+0000 7f71e9600640 1 -- 172.21.15.63:0/2681061759 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71f80a5320 con 0x55ef541bc0b0 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.834+0000 7f71e9600640 1 -- 172.21.15.63:0/2681061759 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f71f80aebb0 con 0x55ef541bc0b0 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.834+0000 7f720d822740 10 client.21210 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.834+0000 7f720d822740 10 client.21210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.834+0000 7f720d822740 20 client.21210 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.834+0000 7f720d822740 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef53df64d0 0x55ef54240650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.834+0000 7f720d822740 4 client.21210 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:03:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.834+0000 7f720d822740 1 -- 172.21.15.63:0/2681061759 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef541262c0 con 0x55ef53df64d0 2024-05-27T08:03:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.835+0000 7f71ff400640 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef53df64d0 0x55ef54240650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.836+0000 7f71ff400640 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef53df64d0 0x55ef54240650 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca43d1192bd2f4 server_cookie=3e96a963303ba4c3 in_seq=0 out_seq=0 2024-05-27T08:03:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.836+0000 7f71e9600640 10 client.21210 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.879+0000 7f71e9600640 1 -- 172.21.15.63:0/2681061759 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef541262c0 con 0x55ef53df64d0 2024-05-27T08:03:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.879+0000 7f71e9600640 10 client.21210 handle_command_reply: tid=0 2024-05-27T08:03:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.879+0000 7f720d822740 1 client.21210 shutdown 2024-05-27T08:03:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.879+0000 7f720d822740 1 -- 172.21.15.63:0/2681061759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef541bc0b0 msgr2=0x55ef541c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.879+0000 7f720d822740 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef541bc0b0 0x55ef541c7b40 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f71f80cc070 tx=0x7f71f80a50a0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.879+0000 7f720d822740 1 -- 172.21.15.63:0/2681061759 shutdown_connections 2024-05-27T08:03:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.879+0000 7f720d822740 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef53df64d0 0x55ef54240650 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.879+0000 7f720d822740 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef541bc0b0 0x55ef541c7b40 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.879+0000 7f720d822740 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef53e0b300 0x55ef541c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.880+0000 7f71e9600640 0 client.21210 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.880+0000 7f720d822740 1 --2- 172.21.15.63:0/2681061759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef53dee2f0 0x55ef541cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.880+0000 7f720d822740 1 -- 172.21.15.63:0/2681061759 >> 172.21.15.63:0/2681061759 conn(0x55ef541ba610 msgr2=0x55ef541c3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.880+0000 7f720d822740 1 -- 172.21.15.63:0/2681061759 shutdown_connections 2024-05-27T08:03:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.880+0000 7f720d822740 1 -- 172.21.15.63:0/2681061759 wait complete. 2024-05-27T08:03:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.880+0000 7f720d822740 20 client.21210 trim_cache size 0 max 16384 2024-05-27T08:03:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:03:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:03:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:03:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 -- 172.21.15.63:0/1010942213 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f71e40b29d0 msgr2=0x7f71e40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f71e40b29d0 0x7f71e40b4ee0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55ef541bae10 tx=0x7f71f0005f90 comp rx=0 tx=0).stop 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 -- 172.21.15.63:0/1010942213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f720010e010 msgr2=0x7f7200102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f720010e010 0x7f7200102f90 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f71f800eaa0 tx=0x7f71f800ef60 comp rx=0 tx=0).stop 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 -- 172.21.15.63:0/1010942213 shutdown_connections 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f71e40b29d0 0x7f71e40b4ee0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f720010e010 0x7f7200102f90 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f720010b6b0 0x7f7200102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 --2- 172.21.15.63:0/1010942213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7200104c50 0x7f7200102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 -- 172.21.15.63:0/1010942213 >> 172.21.15.63:0/1010942213 conn(0x7f72000fe210 msgr2=0x7f720010e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.881+0000 7f7205200640 1 -- 172.21.15.63:0/1010942213 shutdown_connections 2024-05-27T08:03:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:03:46.882+0000 7f7205200640 1 -- 172.21.15.63:0/1010942213 wait complete. 2024-05-27T08:03:47.660 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:03:47.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:47 smithi086 ceph-mon[34687]: pgmap v958: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:03:47.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:47 smithi149 ceph-mon[35408]: pgmap v958: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:03:47.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:47 smithi063 ceph-mon[31613]: pgmap v958: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:03:48.161 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:49.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:49 smithi086 ceph-mon[34687]: pgmap v959: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 12 op/s 2024-05-27T08:03:49.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:49 smithi149 ceph-mon[35408]: pgmap v959: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 12 op/s 2024-05-27T08:03:49.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:49 smithi063 ceph-mon[31613]: pgmap v959: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.0 MiB/s wr, 12 op/s 2024-05-27T08:03:50.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.821+0000 7fc6cbe00700 1 -- 172.21.15.63:0/2876834074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c4105dc0 msgr2=0x7fc6c410e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:50.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.821+0000 7fc6cbe00700 1 --2- 172.21.15.63:0/2876834074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c4105dc0 0x7fc6c410e310 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009990 tx=0x7fc6b0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.821+0000 7fc6cbe00700 1 -- 172.21.15.63:0/2876834074 shutdown_connections 2024-05-27T08:03:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.821+0000 7fc6cbe00700 1 --2- 172.21.15.63:0/2876834074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6c410e8e0 0x7fc6c4110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.821+0000 7fc6cbe00700 1 --2- 172.21.15.63:0/2876834074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c4105470 0x7fc6c4105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.821+0000 7fc6cbe00700 1 --2- 172.21.15.63:0/2876834074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c4105dc0 0x7fc6c410e310 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.821+0000 7fc6cbe00700 1 -- 172.21.15.63:0/2876834074 >> 172.21.15.63:0/2876834074 conn(0x7fc6c40fef20 msgr2=0x7fc6c4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.821+0000 7fc6cbe00700 1 -- 172.21.15.63:0/2876834074 shutdown_connections 2024-05-27T08:03:50.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.821+0000 7fc6cbe00700 1 -- 172.21.15.63:0/2876834074 wait complete. 2024-05-27T08:03:50.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.822+0000 7fc6cbe00700 1 Processor -- start 2024-05-27T08:03:50.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.823+0000 7fc6cbe00700 1 -- start start 2024-05-27T08:03:50.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.823+0000 7fc6cbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c4105470 0x7fc6c4103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.823+0000 7fc6cbe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6c4105dc0 0x7fc6c41036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.823+0000 7fc6cbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c410e8e0 0x7fc6c4103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.823+0000 7fc6cbe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41142f0 con 0x7fc6c4105470 2024-05-27T08:03:50.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.823+0000 7fc6cbe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4114170 con 0x7fc6c410e8e0 2024-05-27T08:03:50.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.823+0000 7fc6cbe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4114470 con 0x7fc6c4105dc0 2024-05-27T08:03:50.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c4105470 0x7fc6c4103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c4105470 0x7fc6c4103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55612/0 (socket says 172.21.15.63:55612) 2024-05-27T08:03:50.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c9400700 1 -- 172.21.15.63:0/112399795 learned_addr learned my addr 172.21.15.63:0/112399795 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:50.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c9e00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c410e8e0 0x7fc6c4103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6c4105dc0 0x7fc6c41036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c9400700 1 -- 172.21.15.63:0/112399795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6c4105dc0 msgr2=0x7fc6c41036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c9400700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6c4105dc0 0x7fc6c41036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c9400700 1 -- 172.21.15.63:0/112399795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c410e8e0 msgr2=0x7fc6c4103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c9400700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c410e8e0 0x7fc6c4103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c9400700 1 -- 172.21.15.63:0/112399795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c41a9010 con 0x7fc6c4105470 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.824+0000 7fc6c9e00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c410e8e0 0x7fc6c4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.825+0000 7fc6c9400700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c4105470 0x7fc6c4103160 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80060b0 tx=0x7fc6b8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.825+0000 7fc6c2000700 1 -- 172.21.15.63:0/112399795 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b801a070 con 0x7fc6c4105470 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.825+0000 7fc6c2000700 1 -- 172.21.15.63:0/112399795 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6b800de20 con 0x7fc6c4105470 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.825+0000 7fc6cbe00700 1 -- 172.21.15.63:0/112399795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c41a9300 con 0x7fc6c4105470 2024-05-27T08:03:50.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.825+0000 7fc6c2000700 1 -- 172.21.15.63:0/112399795 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8014690 con 0x7fc6c4105470 2024-05-27T08:03:50.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.825+0000 7fc6cbe00700 1 -- 172.21.15.63:0/112399795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c41a9840 con 0x7fc6c4105470 2024-05-27T08:03:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.830+0000 7fc6c2000700 1 -- 172.21.15.63:0/112399795 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc6b801d070 con 0x7fc6c4105470 2024-05-27T08:03:50.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.831+0000 7fc6da9f8b80 1 Processor -- start 2024-05-27T08:03:50.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.831+0000 7fc6da9f8b80 1 -- start start 2024-05-27T08:03:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.831+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7ac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.831+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.831+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.831+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55599a6dae40 con 0x55599a7199f0 2024-05-27T08:03:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.831+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55599a6ca9e0 con 0x55599a7ace70 2024-05-27T08:03:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.831+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55599a6cea20 con 0x55599a62de40 2024-05-27T08:03:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.832+0000 7fc6c9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.832+0000 7fc6c9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7ac330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55616/0 (socket says 172.21.15.63:55616) 2024-05-27T08:03:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.832+0000 7fc6c9e00700 1 -- 172.21.15.63:0/2322408419 learned_addr learned my addr 172.21.15.63:0/2322408419 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.832+0000 7fc6c2000700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6ac0b29f0 0x7fc6ac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.832+0000 7fc6c9400700 1 --2- 172.21.15.63:0/2322408419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.832+0000 7fc6c2000700 1 -- 172.21.15.63:0/112399795 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7fc6b80e2360 con 0x7fc6c4105470 2024-05-27T08:03:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.833+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/2322408419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.833+0000 7fc6c9e00700 1 -- 172.21.15.63:0/2322408419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 msgr2=0x55599a7ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.833+0000 7fc6c9e00700 1 --2- 172.21.15.63:0/2322408419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.833+0000 7fc6c9e00700 1 -- 172.21.15.63:0/2322408419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 msgr2=0x55599a7b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.833+0000 7fc6c9e00700 1 --2- 172.21.15.63:0/2322408419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.833+0000 7fc6c9e00700 1 -- 172.21.15.63:0/2322408419 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55599a693420 con 0x55599a7199f0 2024-05-27T08:03:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.833+0000 7fc6c9e00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6ac0b29f0 0x7fc6ac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.833+0000 7fc6c9e00700 1 --2- 172.21.15.63:0/2322408419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7ac330 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00cdf0 tx=0x7fc6bc00e630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e214a433ea1692a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.834+0000 7fc6b6a00700 1 -- 172.21.15.63:0/2322408419 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc022070 con 0x55599a7199f0 2024-05-27T08:03:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.834+0000 7fc6c9e00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6ac0b29f0 0x7fc6ac0b4f00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc01f000 tx=0x7fc6bc020000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.834+0000 7fc6b6a00700 1 -- 172.21.15.63:0/2322408419 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6bc01cb00 con 0x55599a7199f0 2024-05-27T08:03:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.834+0000 7fc6b6a00700 1 -- 172.21.15.63:0/2322408419 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc01a920 con 0x55599a7199f0 2024-05-27T08:03:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.835+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/2322408419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 msgr2=0x55599a7ac330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.835+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/2322408419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7ac330 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00cdf0 tx=0x7fc6bc00e630 comp rx=0 tx=0).stop 2024-05-27T08:03:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.835+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/2322408419 shutdown_connections 2024-05-27T08:03:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.835+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/2322408419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.835+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/2322408419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7ac330 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.835+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/2322408419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.835+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/2322408419 >> 172.21.15.63:0/2322408419 conn(0x55599a6c7390 msgr2=0x55599a7a84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.835+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/2322408419 shutdown_connections 2024-05-27T08:03:50.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.835+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/2322408419 wait complete. 2024-05-27T08:03:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.836+0000 7fc6da9f8b80 1 -- start start 2024-05-27T08:03:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.836+0000 7fc6da9f8b80 1 Processor -- start 2024-05-27T08:03:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.837+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.837+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.837+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.837+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55599a5d0fa0 con 0x55599a7199f0 2024-05-27T08:03:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.837+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55599a66b760 con 0x55599a7ace70 2024-05-27T08:03:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.837+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55599a7b9b80 con 0x55599a62de40 2024-05-27T08:03:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.837+0000 7fc6da9f8b80 10 client.0 _ll_register_callbacks cb 0x55599a674c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:50.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.838+0000 7fc6c9400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.838+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.838+0000 7fc6c9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.838+0000 7fc6c9400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:48552/0 (socket says 172.21.15.63:48552) 2024-05-27T08:03:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.838+0000 7fc6c9400700 1 -- 172.21.15.63:0/3874109700 learned_addr learned my addr 172.21.15.63:0/3874109700 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.838+0000 7fc6c8a00700 1 -- 172.21.15.63:0/3874109700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 msgr2=0x55599a7b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.838+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.838+0000 7fc6c8a00700 1 -- 172.21.15.63:0/3874109700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 msgr2=0x55599a7b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.838+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.839+0000 7fc6c8a00700 1 -- 172.21.15.63:0/3874109700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55599a61a8d0 con 0x55599a7199f0 2024-05-27T08:03:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.839+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b6d80 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fc6c41025d0 tx=0x7fc6b0003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5057f7e240b6ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.839+0000 7fc6ab400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.839+0000 7fc6ab400700 1 -- 172.21.15.63:0/3874109700 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0027060 con 0x55599a7199f0 2024-05-27T08:03:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.839+0000 7fc6ab400700 1 -- 172.21.15.63:0/3874109700 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7fc6b0005230 con 0x55599a7199f0 2024-05-27T08:03:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.839+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3874109700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6bc01cb00 con 0x55599a7199f0 2024-05-27T08:03:50.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.840+0000 7fc6ab400700 1 -- 172.21.15.63:0/3874109700 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc6b0005430 con 0x55599a7199f0 2024-05-27T08:03:50.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.840+0000 7fc6da9f8b80 10 client.21222 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:50.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.840+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3874109700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55599a69b060 con 0x55599a7199f0 2024-05-27T08:03:50.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.840+0000 7fc6ab400700 1 -- 172.21.15.63:0/3874109700 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6bc01cb00 con 0x55599a7199f0 2024-05-27T08:03:50.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.841+0000 7fc6ab400700 1 -- 172.21.15.63:0/3874109700 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc6b0021070 con 0x55599a7199f0 2024-05-27T08:03:50.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.841+0000 7fc6da9f8b80 10 client.21222 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:50.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.841+0000 7fc6da9f8b80 10 client.21222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:50.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.841+0000 7fc6da9f8b80 20 client.21222 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.841+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55599a7c6b90 0x55599a7c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.841+0000 7fc6da9f8b80 4 client.21222 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.841+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3874109700 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55599a6dcd90 con 0x55599a7c6b90 2024-05-27T08:03:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.841+0000 7fc6c9400700 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55599a7c6b90 0x55599a7c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.842+0000 7fc6c9400700 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55599a7c6b90 0x55599a7c8fb0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767322a72808908 server_cookie=82716bbabeebe4c1 in_seq=0 out_seq=0 2024-05-27T08:03:50.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.842+0000 7fc6ab400700 10 client.21222 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.844+0000 7fc6ab400700 1 -- 172.21.15.63:0/3874109700 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55599a6dcd90 con 0x55599a7c6b90 2024-05-27T08:03:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.844+0000 7fc6ab400700 10 client.21222 handle_command_reply: tid=0 2024-05-27T08:03:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.844+0000 7fc6da9f8b80 1 client.21222 shutdown 2024-05-27T08:03:50.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3874109700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 msgr2=0x55599a7b6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:50.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b6d80 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fc6c41025d0 tx=0x7fc6b0003ef0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3874109700 shutdown_connections 2024-05-27T08:03:50.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55599a7c6b90 0x55599a7c8fb0 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3874109700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b6d80 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6ab400700 0 client.21222 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:50.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3874109700 >> 172.21.15.63:0/3874109700 conn(0x55599a6c7390 msgr2=0x55599a7ac450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:50.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3874109700 shutdown_connections 2024-05-27T08:03:50.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3874109700 wait complete. 2024-05-27T08:03:50.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.845+0000 7fc6da9f8b80 20 client.21222 trim_cache size 0 max 16384 2024-05-27T08:03:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.875+0000 7fc6da9f8b80 1 Processor -- start 2024-05-27T08:03:50.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.876+0000 7fc6da9f8b80 1 -- start start 2024-05-27T08:03:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.876+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.876+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.876+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.876+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b4e40 con 0x55599a7199f0 2024-05-27T08:03:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.876+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b26d0 con 0x55599a7ace70 2024-05-27T08:03:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.876+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b0540 con 0x55599a62de40 2024-05-27T08:03:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.876+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55626/0 (socket says 172.21.15.63:55626) 2024-05-27T08:03:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c8a00700 1 -- 172.21.15.63:0/139548282 learned_addr learned my addr 172.21.15.63:0/139548282 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c9400700 1 --2- 172.21.15.63:0/139548282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c9e00700 1 --2- 172.21.15.63:0/139548282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c8a00700 1 -- 172.21.15.63:0/139548282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 msgr2=0x55599a7a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/139548282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c8a00700 1 -- 172.21.15.63:0/139548282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 msgr2=0x55599a7ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:50.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/139548282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c8a00700 1 -- 172.21.15.63:0/139548282 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55599a7b4db0 con 0x55599a7199f0 2024-05-27T08:03:50.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c9400700 1 --2- 172.21.15.63:0/139548282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:50.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.877+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/139548282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b2030 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000f8b0 tx=0x7fc6b0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa24a628efa1be91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:50.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.878+0000 7fc6b6000700 1 -- 172.21.15.63:0/139548282 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0027040 con 0x55599a7199f0 2024-05-27T08:03:50.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.878+0000 7fc6b6000700 1 -- 172.21.15.63:0/139548282 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6b000f460 con 0x55599a7199f0 2024-05-27T08:03:50.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.878+0000 7fc6b6000700 1 -- 172.21.15.63:0/139548282 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0022690 con 0x55599a7199f0 2024-05-27T08:03:50.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.879+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/139548282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 msgr2=0x55599a7b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:50.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.879+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/139548282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b2030 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000f8b0 tx=0x7fc6b0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.879+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/139548282 shutdown_connections 2024-05-27T08:03:50.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.879+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/139548282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.879+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/139548282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.879+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/139548282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b2030 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.879+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/139548282 >> 172.21.15.63:0/139548282 conn(0x55599a6c7390 msgr2=0x55599a7a7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:50.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.879+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/139548282 shutdown_connections 2024-05-27T08:03:50.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.879+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/139548282 wait complete. 2024-05-27T08:03:50.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.880+0000 7fc6da9f8b80 1 -- start start 2024-05-27T08:03:50.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.880+0000 7fc6da9f8b80 1 Processor -- start 2024-05-27T08:03:50.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.881+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.881+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.881+0000 7fc6da9f8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.881+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b0b90 con 0x55599a7199f0 2024-05-27T08:03:50.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.881+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b2000 con 0x55599a7ace70 2024-05-27T08:03:50.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.881+0000 7fc6da9f8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b1260 con 0x55599a62de40 2024-05-27T08:03:50.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.881+0000 7fc6da9f8b80 10 client.0 _ll_register_callbacks cb 0x55599a674c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.881+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.881+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55636/0 (socket says 172.21.15.63:55636) 2024-05-27T08:03:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.881+0000 7fc6c8a00700 1 -- 172.21.15.63:0/3826655785 learned_addr learned my addr 172.21.15.63:0/3826655785 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.882+0000 7fc6c9400700 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.882+0000 7fc6c9e00700 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.882+0000 7fc6c8a00700 1 -- 172.21.15.63:0/3826655785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 msgr2=0x55599a7b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.882+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.882+0000 7fc6c8a00700 1 -- 172.21.15.63:0/3826655785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 msgr2=0x55599a7b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.882+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.882+0000 7fc6c8a00700 1 -- 172.21.15.63:0/3826655785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55599a7bd3a0 con 0x55599a7199f0 2024-05-27T08:03:50.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.882+0000 7fc6c8a00700 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b18e0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009350 tx=0x7fc6b0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7ac6b0e32bcc25 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:50.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.882+0000 7fc6ab400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:50.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.883+0000 7fc6ab400700 1 -- 172.21.15.63:0/3826655785 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b003f060 con 0x55599a7199f0 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.883+0000 7fc6ab400700 1 -- 172.21.15.63:0/3826655785 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7fc6b000ba40 con 0x55599a7199f0 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.883+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3826655785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6b000f460 con 0x55599a7199f0 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.883+0000 7fc6ab400700 1 -- 172.21.15.63:0/3826655785 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc6b000bc40 con 0x55599a7199f0 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.883+0000 7fc6da9f8b80 10 client.21228 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.883+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3826655785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55599a831850 con 0x55599a7199f0 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.884+0000 7fc6ab400700 1 -- 172.21.15.63:0/3826655785 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6b000f460 con 0x55599a7199f0 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.884+0000 7fc6ab400700 1 -- 172.21.15.63:0/3826655785 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc6b0021070 con 0x55599a7199f0 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.884+0000 7fc6da9f8b80 10 client.21228 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.884+0000 7fc6da9f8b80 10 client.21228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.884+0000 7fc6da9f8b80 20 client.21228 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.884+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55599a831ce0 0x55599a834100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.884+0000 7fc6da9f8b80 4 client.21228 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:03:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.884+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3826655785 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55599a6d6570 con 0x55599a831ce0 2024-05-27T08:03:50.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.885+0000 7fc6c9400700 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55599a831ce0 0x55599a834100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:50.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.886+0000 7fc6c9400700 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55599a831ce0 0x55599a834100 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1170e99a522de9b4 server_cookie=2dea3b29a1fc08e4 in_seq=0 out_seq=0 2024-05-27T08:03:50.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.886+0000 7fc6ab400700 10 client.21228 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.886+0000 7fc6ab400700 1 -- 172.21.15.63:0/3826655785 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55599a6d6570 con 0x55599a831ce0 2024-05-27T08:03:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.886+0000 7fc6ab400700 10 client.21228 handle_command_reply: tid=0 2024-05-27T08:03:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.886+0000 7fc6da9f8b80 1 client.21228 shutdown 2024-05-27T08:03:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3826655785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 msgr2=0x55599a7b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:50.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b18e0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009350 tx=0x7fc6b0005e80 comp rx=0 tx=0).stop 2024-05-27T08:03:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3826655785 shutdown_connections 2024-05-27T08:03:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55599a831ce0 0x55599a834100 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55599a7ace70 0x55599a7b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55599a62de40 0x55599a7b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 1 --2- 172.21.15.63:0/3826655785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55599a7199f0 0x55599a7b18e0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6ab400700 0 client.21228 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:03:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3826655785 >> 172.21.15.63:0/3826655785 conn(0x55599a6c7390 msgr2=0x55599a7aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3826655785 shutdown_connections 2024-05-27T08:03:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 1 -- 172.21.15.63:0/3826655785 wait complete. 2024-05-27T08:03:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.887+0000 7fc6da9f8b80 20 client.21228 trim_cache size 0 max 16384 2024-05-27T08:03:50.891 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:03:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:03:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:03:50.891 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:03:50.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.888+0000 7fc6cbe00700 1 -- 172.21.15.63:0/112399795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6ac0b29f0 msgr2=0x7fc6ac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.888+0000 7fc6cbe00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6ac0b29f0 0x7fc6ac0b4f00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc01f000 tx=0x7fc6bc020000 comp rx=0 tx=0).stop 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.888+0000 7fc6cbe00700 1 -- 172.21.15.63:0/112399795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c4105470 msgr2=0x7fc6c4103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.888+0000 7fc6cbe00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c4105470 0x7fc6c4103160 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80060b0 tx=0x7fc6b8007f90 comp rx=0 tx=0).stop 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.888+0000 7fc6cbe00700 1 -- 172.21.15.63:0/112399795 shutdown_connections 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.888+0000 7fc6cbe00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6ac0b29f0 0x7fc6ac0b4f00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.888+0000 7fc6cbe00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c410e8e0 0x7fc6c4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.888+0000 7fc6cbe00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6c4105470 0x7fc6c4103160 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.888+0000 7fc6cbe00700 1 --2- 172.21.15.63:0/112399795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6c4105dc0 0x7fc6c41036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.888+0000 7fc6cbe00700 1 -- 172.21.15.63:0/112399795 >> 172.21.15.63:0/112399795 conn(0x7fc6c40fef20 msgr2=0x7fc6c40ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.889+0000 7fc6cbe00700 1 -- 172.21.15.63:0/112399795 shutdown_connections 2024-05-27T08:03:50.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:50.889+0000 7fc6cbe00700 1 -- 172.21.15.63:0/112399795 wait complete. 2024-05-27T08:03:51.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:51 smithi086 ceph-mon[34687]: pgmap v960: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:03:51.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:51 smithi149 ceph-mon[35408]: pgmap v960: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:03:51.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:51 smithi063 ceph-mon[31613]: pgmap v960: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:03:51.892 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:03:51.892 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:03:52.423 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:53.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:53 smithi086 ceph-mon[34687]: pgmap v961: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:03:53.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:53 smithi149 ceph-mon[35408]: pgmap v961: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:03:53.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:53 smithi063 ceph-mon[31613]: pgmap v961: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:03:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.884+0000 7fa80e200700 1 -- 172.21.15.63:0/742022167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa808105280 msgr2=0x7fa808105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:54.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.884+0000 7fa80e200700 1 --2- 172.21.15.63:0/742022167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa808105280 0x7fa808105690 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009990 tx=0x7fa7fc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:03:54.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.885+0000 7fa80e200700 1 -- 172.21.15.63:0/742022167 shutdown_connections 2024-05-27T08:03:54.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.885+0000 7fa80e200700 1 --2- 172.21.15.63:0/742022167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80810e6f0 0x7fa808110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:54.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.885+0000 7fa80e200700 1 --2- 172.21.15.63:0/742022167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa808105280 0x7fa808105690 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:54.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.885+0000 7fa80e200700 1 --2- 172.21.15.63:0/742022167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa808105bd0 0x7fa80810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:54.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.885+0000 7fa80e200700 1 -- 172.21.15.63:0/742022167 >> 172.21.15.63:0/742022167 conn(0x7fa808078ab0 msgr2=0x7fa808078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:54.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.885+0000 7fa80e200700 1 -- 172.21.15.63:0/742022167 shutdown_connections 2024-05-27T08:03:54.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.885+0000 7fa80e200700 1 -- 172.21.15.63:0/742022167 wait complete. 2024-05-27T08:03:54.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.886+0000 7fa80e200700 1 Processor -- start 2024-05-27T08:03:54.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.887+0000 7fa80e200700 1 -- start start 2024-05-27T08:03:54.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.887+0000 7fa80e200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa808105280 0x7fa80819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:54.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.887+0000 7fa80e200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa808105bd0 0x7fa80819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:54.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.887+0000 7fa80e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80810e6f0 0x7fa8081a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.887+0000 7fa80e200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa808114140 con 0x7fa808105280 2024-05-27T08:03:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.887+0000 7fa80e200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa808113fc0 con 0x7fa80810e6f0 2024-05-27T08:03:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.887+0000 7fa80e200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa8081142c0 con 0x7fa808105bd0 2024-05-27T08:03:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.887+0000 7fa80ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa808105280 0x7fa80819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.888+0000 7fa80ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa808105280 0x7fa80819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55642/0 (socket says 172.21.15.63:55642) 2024-05-27T08:03:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.888+0000 7fa80ce00700 1 -- 172.21.15.63:0/558846565 learned_addr learned my addr 172.21.15.63:0/558846565 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:54.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.888+0000 7fa807e00700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa808105bd0 0x7fa80819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.888+0000 7fa80d800700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80810e6f0 0x7fa8081a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.888+0000 7fa80ce00700 1 -- 172.21.15.63:0/558846565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa808105bd0 msgr2=0x7fa80819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.888+0000 7fa80ce00700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa808105bd0 0x7fa80819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:54.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.888+0000 7fa80ce00700 1 -- 172.21.15.63:0/558846565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80810e6f0 msgr2=0x7fa8081a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:54.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.888+0000 7fa80ce00700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80810e6f0 0x7fa8081a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:54.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.888+0000 7fa80ce00700 1 -- 172.21.15.63:0/558846565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8081a1cd0 con 0x7fa808105280 2024-05-27T08:03:54.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.889+0000 7fa80ce00700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa808105280 0x7fa80819ceb0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc000c00 tx=0x7fa7fc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:54.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.889+0000 7fa805600700 1 -- 172.21.15.63:0/558846565 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc027070 con 0x7fa808105280 2024-05-27T08:03:54.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.889+0000 7fa805600700 1 -- 172.21.15.63:0/558846565 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7fc00fb80 con 0x7fa808105280 2024-05-27T08:03:54.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.889+0000 7fa80e200700 1 -- 172.21.15.63:0/558846565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8081a1f60 con 0x7fa808105280 2024-05-27T08:03:54.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.889+0000 7fa80e200700 1 -- 172.21.15.63:0/558846565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8081a2440 con 0x7fa808105280 2024-05-27T08:03:54.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.890+0000 7fa805600700 1 -- 172.21.15.63:0/558846565 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc022c30 con 0x7fa808105280 2024-05-27T08:03:54.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.891+0000 7fa80e200700 1 -- 172.21.15.63:0/558846565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8080512b0 con 0x7fa808105280 2024-05-27T08:03:54.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.894+0000 7fa805600700 1 -- 172.21.15.63:0/558846565 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa7fc02a070 con 0x7fa808105280 2024-05-27T08:03:54.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.897+0000 7fa805600700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7f00b29f0 0x7fa7f00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:54.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.897+0000 7fa805600700 1 -- 172.21.15.63:0/558846565 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7fa7fc0e3c60 con 0x7fa808105280 2024-05-27T08:03:54.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.898+0000 7fa807e00700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7f00b29f0 0x7fa7f00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:54.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.898+0000 7fa807e00700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7f00b29f0 0x7fa7f00b4f00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4005950 tx=0x7fa7f40058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:54.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:54.900+0000 7fa805600700 1 -- 172.21.15.63:0/558846565 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7fc013070 con 0x7fa808105280 2024-05-27T08:03:55.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.214+0000 7fa80e200700 1 -- 172.21.15.63:0/558846565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa80802e5e0 con 0x7fa808105280 2024-05-27T08:03:55.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.216+0000 7fa805600700 1 -- 172.21.15.63:0/558846565 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7fa7fc0ab340 con 0x7fa808105280 2024-05-27T08:03:55.219 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:03:55.219 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:03:55.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.219+0000 7fa80e200700 1 -- 172.21.15.63:0/558846565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7f00b29f0 msgr2=0x7fa7f00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:55.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.219+0000 7fa80e200700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7f00b29f0 0x7fa7f00b4f00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4005950 tx=0x7fa7f40058e0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.219+0000 7fa80e200700 1 -- 172.21.15.63:0/558846565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa808105280 msgr2=0x7fa80819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:55.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.219+0000 7fa80e200700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa808105280 0x7fa80819ceb0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc000c00 tx=0x7fa7fc005ef0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.219+0000 7fa80e200700 1 -- 172.21.15.63:0/558846565 shutdown_connections 2024-05-27T08:03:55.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.219+0000 7fa80e200700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa7f00b29f0 0x7fa7f00b4f00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.219+0000 7fa80e200700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa80810e6f0 0x7fa8081a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.219+0000 7fa80e200700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa808105280 0x7fa80819ceb0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.219+0000 7fa80e200700 1 --2- 172.21.15.63:0/558846565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa808105bd0 0x7fa80819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.219+0000 7fa80e200700 1 -- 172.21.15.63:0/558846565 >> 172.21.15.63:0/558846565 conn(0x7fa808078ab0 msgr2=0x7fa8081030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:55.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.220+0000 7fa80e200700 1 -- 172.21.15.63:0/558846565 shutdown_connections 2024-05-27T08:03:55.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:55.220+0000 7fa80e200700 1 -- 172.21.15.63:0/558846565 wait complete. 2024-05-27T08:03:55.223 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:03:55.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:55 smithi063 ceph-mon[31613]: pgmap v962: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:03:55.687 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:55 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/558846565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:03:55.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:55 smithi086 ceph-mon[34687]: pgmap v962: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:03:55.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:55 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/558846565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:03:55.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:55 smithi149 ceph-mon[35408]: pgmap v962: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 9 op/s 2024-05-27T08:03:55.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:55 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/558846565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:03:56.255 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:03:56.773 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:57.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:57 smithi063 ceph-mon[31613]: pgmap v963: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:03:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:57 smithi149 ceph-mon[35408]: pgmap v963: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:03:57.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:57 smithi086 ceph-mon[34687]: pgmap v963: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:03:59.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.433+0000 7f01c5400700 1 -- 172.21.15.63:0/499494472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01c010b890 msgr2=0x7f01c01131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:59.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.433+0000 7f01c5400700 1 --2- 172.21.15.63:0/499494472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01c010b890 0x7f01c01131f0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f01b800b0d0 tx=0x7f01b800b3e0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.433+0000 7f01c5400700 1 -- 172.21.15.63:0/499494472 shutdown_connections 2024-05-27T08:03:59.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.433+0000 7f01c5400700 1 --2- 172.21.15.63:0/499494472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01c010b890 0x7f01c01131f0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.433+0000 7f01c5400700 1 --2- 172.21.15.63:0/499494472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c0106610 0x7f01c0106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.433+0000 7f01c5400700 1 --2- 172.21.15.63:0/499494472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c010af40 0x7f01c010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.433+0000 7f01c5400700 1 -- 172.21.15.63:0/499494472 >> 172.21.15.63:0/499494472 conn(0x7f01c0102020 msgr2=0x7f01c0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:59.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.433+0000 7f01c5400700 1 -- 172.21.15.63:0/499494472 shutdown_connections 2024-05-27T08:03:59.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.433+0000 7f01c5400700 1 -- 172.21.15.63:0/499494472 wait complete. 2024-05-27T08:03:59.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.435+0000 7f01c5400700 1 Processor -- start 2024-05-27T08:03:59.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.435+0000 7f01c5400700 1 -- start start 2024-05-27T08:03:59.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.436+0000 7f01c5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01c0106610 0x7f01c01a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.436+0000 7f01c5400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c010af40 0x7f01c01a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.436+0000 7f01c5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c010b890 0x7f01c019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.436+0000 7f01c5400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0116370 con 0x7f01c0106610 2024-05-27T08:03:59.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.436+0000 7f01c5400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01161f0 con 0x7f01c010b890 2024-05-27T08:03:59.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.436+0000 7f01c5400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01164f0 con 0x7f01c010af40 2024-05-27T08:03:59.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.436+0000 7f01bfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01c0106610 0x7f01c01a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.436+0000 7f01bfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01c0106610 0x7f01c01a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35948/0 (socket says 172.21.15.63:35948) 2024-05-27T08:03:59.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.436+0000 7f01bfe00700 1 -- 172.21.15.63:0/1055163500 learned_addr learned my addr 172.21.15.63:0/1055163500 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:59.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.437+0000 7f01bf400700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c010af40 0x7f01c01a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.437+0000 7f01c4a00700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c010b890 0x7f01c019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.437+0000 7f01bfe00700 1 -- 172.21.15.63:0/1055163500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c010af40 msgr2=0x7f01c01a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:59.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.437+0000 7f01bfe00700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c010af40 0x7f01c01a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.437+0000 7f01bfe00700 1 -- 172.21.15.63:0/1055163500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c010b890 msgr2=0x7f01c019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:59.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.437+0000 7f01bfe00700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c010b890 0x7f01c019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.437+0000 7f01bfe00700 1 -- 172.21.15.63:0/1055163500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c019d1d0 con 0x7f01c0106610 2024-05-27T08:03:59.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.438+0000 7f01bfe00700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01c0106610 0x7f01c01a28b0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f01b000d950 tx=0x7f01b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:59.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.438+0000 7f01bcc00700 1 -- 172.21.15.63:0/1055163500 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0016070 con 0x7f01c0106610 2024-05-27T08:03:59.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.438+0000 7f01bcc00700 1 -- 172.21.15.63:0/1055163500 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01b000fd20 con 0x7f01c0106610 2024-05-27T08:03:59.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.438+0000 7f01c5400700 1 -- 172.21.15.63:0/1055163500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c019d4c0 con 0x7f01c0106610 2024-05-27T08:03:59.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.438+0000 7f01c5400700 1 -- 172.21.15.63:0/1055163500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c01a9760 con 0x7f01c0106610 2024-05-27T08:03:59.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.438+0000 7f01bcc00700 1 -- 172.21.15.63:0/1055163500 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b000e3b0 con 0x7f01c0106610 2024-05-27T08:03:59.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.443+0000 7f01bcc00700 1 -- 172.21.15.63:0/1055163500 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f01b0019070 con 0x7f01c0106610 2024-05-27T08:03:59.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.444+0000 7f01bcc00700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01a40b29f0 0x7f01a40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.445+0000 7f01bcc00700 1 -- 172.21.15.63:0/1055163500 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f01b00e28b0 con 0x7f01c0106610 2024-05-27T08:03:59.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.445+0000 7f01d55ecb80 1 Processor -- start 2024-05-27T08:03:59.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.445+0000 7f01bf400700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01a40b29f0 0x7f01a40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.445+0000 7f01d55ecb80 1 -- start start 2024-05-27T08:03:59.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.445+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d225e40 0x55ce8d3a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4410 0x55ce8d3aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d3a4820 0x55ce8d3b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ce8d2d2e40 con 0x55ce8d225e40 2024-05-27T08:03:59.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ce8d2c29e0 con 0x55ce8d3a4820 2024-05-27T08:03:59.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ce8d2c6a20 con 0x55ce8d3a4410 2024-05-27T08:03:59.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01bf400700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01a40b29f0 0x7f01a40b4f00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f01c01059a0 tx=0x7f01ac00b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:59.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01bfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d225e40 0x55ce8d3a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01bfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d225e40 0x55ce8d3a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35950/0 (socket says 172.21.15.63:35950) 2024-05-27T08:03:59.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01bfe00700 1 -- 172.21.15.63:0/3151125711 learned_addr learned my addr 172.21.15.63:0/3151125711 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:59.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01bf400700 1 --2- 172.21.15.63:0/3151125711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4410 0x55ce8d3aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.446+0000 7f01c4a00700 1 --2- 172.21.15.63:0/3151125711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d3a4820 0x55ce8d3b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.447+0000 7f01bfe00700 1 -- 172.21.15.63:0/3151125711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4410 msgr2=0x55ce8d3aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:59.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.447+0000 7f01bfe00700 1 --2- 172.21.15.63:0/3151125711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4410 0x55ce8d3aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.447+0000 7f01bfe00700 1 -- 172.21.15.63:0/3151125711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d3a4820 msgr2=0x55ce8d3b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:59.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.447+0000 7f01bfe00700 1 --2- 172.21.15.63:0/3151125711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d3a4820 0x55ce8d3b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.447+0000 7f01bfe00700 1 -- 172.21.15.63:0/3151125711 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce8d28b420 con 0x55ce8d225e40 2024-05-27T08:03:59.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.447+0000 7f01bfe00700 1 --2- 172.21.15.63:0/3151125711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d225e40 0x55ce8d3a3ed0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f01b00a57f0 tx=0x7f01b00a9350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53de1844e4d6906 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:59.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.447+0000 7f01b5600700 1 -- 172.21.15.63:0/3151125711 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0020030 con 0x55ce8d225e40 2024-05-27T08:03:59.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.447+0000 7f01b5600700 1 -- 172.21.15.63:0/3151125711 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01b00adb20 con 0x55ce8d225e40 2024-05-27T08:03:59.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.447+0000 7f01b5600700 1 -- 172.21.15.63:0/3151125711 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b00e2430 con 0x55ce8d225e40 2024-05-27T08:03:59.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.448+0000 7f01d55ecb80 1 -- 172.21.15.63:0/3151125711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d225e40 msgr2=0x55ce8d3a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:59.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.448+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/3151125711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d225e40 0x55ce8d3a3ed0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f01b00a57f0 tx=0x7f01b00a9350 comp rx=0 tx=0).stop 2024-05-27T08:03:59.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.449+0000 7f01d55ecb80 1 -- 172.21.15.63:0/3151125711 shutdown_connections 2024-05-27T08:03:59.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.449+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/3151125711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d3a4820 0x55ce8d3b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.449+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/3151125711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d225e40 0x55ce8d3a3ed0 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.449+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/3151125711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4410 0x55ce8d3aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.449+0000 7f01d55ecb80 1 -- 172.21.15.63:0/3151125711 >> 172.21.15.63:0/3151125711 conn(0x55ce8d285fc0 msgr2=0x55ce8d39fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:59.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.449+0000 7f01d55ecb80 1 -- 172.21.15.63:0/3151125711 shutdown_connections 2024-05-27T08:03:59.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.449+0000 7f01d55ecb80 1 -- 172.21.15.63:0/3151125711 wait complete. 2024-05-27T08:03:59.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.449+0000 7f01d55ecb80 1 -- start start 2024-05-27T08:03:59.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.450+0000 7f01d55ecb80 1 Processor -- start 2024-05-27T08:03:59.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.450+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d3ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.450+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d3aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.450+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4820 0x55ce8d3af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.450+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ce8d1c8fa0 con 0x55ce8d3a4410 2024-05-27T08:03:59.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.450+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ce8d263760 con 0x55ce8d225e40 2024-05-27T08:03:59.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.450+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ce8d3b18d0 con 0x55ce8d3a4820 2024-05-27T08:03:59.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.450+0000 7f01d55ecb80 10 client.0 _ll_register_callbacks cb 0x55ce8d26cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:59.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.451+0000 7f01bf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d3aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.451+0000 7f01bf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d3aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35952/0 (socket says 172.21.15.63:35952) 2024-05-27T08:03:59.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.451+0000 7f01bf400700 1 -- 172.21.15.63:0/1772854308 learned_addr learned my addr 172.21.15.63:0/1772854308 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:59.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.451+0000 7f01bfe00700 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d3ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.451+0000 7f01c4a00700 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4820 0x55ce8d3af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.451+0000 7f01bf400700 1 -- 172.21.15.63:0/1772854308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4820 msgr2=0x55ce8d3af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:59.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.451+0000 7f01bf400700 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4820 0x55ce8d3af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.451+0000 7f01bf400700 1 -- 172.21.15.63:0/1772854308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 msgr2=0x55ce8d3ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:59.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.451+0000 7f01bf400700 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d3ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.451+0000 7f01bf400700 1 -- 172.21.15.63:0/1772854308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce8d2128d0 con 0x55ce8d3a4410 2024-05-27T08:03:59.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.452+0000 7f01bf400700 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d3aebe0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f01ac011010 tx=0x7f01ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678027447f259d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:59.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.452+0000 7f01aa000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:59.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.452+0000 7f01aa000700 1 -- 172.21.15.63:0/1772854308 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac004a30 con 0x55ce8d3a4410 2024-05-27T08:03:59.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.452+0000 7f01d55ecb80 1 -- 172.21.15.63:0/1772854308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b00adb20 con 0x55ce8d3a4410 2024-05-27T08:03:59.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.453+0000 7f01aa000700 1 -- 172.21.15.63:0/1772854308 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f01ac004bd0 con 0x55ce8d3a4410 2024-05-27T08:03:59.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.453+0000 7f01aa000700 1 -- 172.21.15.63:0/1772854308 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f01ac030050 con 0x55ce8d3a4410 2024-05-27T08:03:59.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.453+0000 7f01d55ecb80 10 client.21246 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:59.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.453+0000 7f01d55ecb80 1 -- 172.21.15.63:0/1772854308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ce8d293060 con 0x55ce8d3a4410 2024-05-27T08:03:59.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.454+0000 7f01aa000700 1 -- 172.21.15.63:0/1772854308 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01b00adb20 con 0x55ce8d3a4410 2024-05-27T08:03:59.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.454+0000 7f01aa000700 1 -- 172.21.15.63:0/1772854308 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f01ac01f6b0 con 0x55ce8d3a4410 2024-05-27T08:03:59.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.454+0000 7f01d55ecb80 10 client.21246 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:59.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.454+0000 7f01d55ecb80 10 client.21246 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:03:59.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.454+0000 7f01d55ecb80 20 client.21246 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:59.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.454+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ce8d3be960 0x55ce8d3c0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.454+0000 7f01d55ecb80 4 client.21246 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:59.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.454+0000 7f01d55ecb80 1 -- 172.21.15.63:0/1772854308 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce8d2d4d90 con 0x55ce8d3be960 2024-05-27T08:03:59.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.455+0000 7f01bfe00700 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ce8d3be960 0x55ce8d3c0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.455+0000 7f01bfe00700 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ce8d3be960 0x55ce8d3c0d80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc513f1d22cb7773 server_cookie=e86ef36e76a7a9ff in_seq=0 out_seq=0 2024-05-27T08:03:59.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.455+0000 7f01aa000700 10 client.21246 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:03:59.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.457+0000 7f01aa000700 1 -- 172.21.15.63:0/1772854308 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ce8d2d4d90 con 0x55ce8d3be960 2024-05-27T08:03:59.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.457+0000 7f01aa000700 10 client.21246 handle_command_reply: tid=0 2024-05-27T08:03:59.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.457+0000 7f01d55ecb80 1 client.21246 shutdown 2024-05-27T08:03:59.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 1 -- 172.21.15.63:0/1772854308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 msgr2=0x55ce8d3aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:59.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d3aebe0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f01ac011010 tx=0x7f01ac0047c0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 1 -- 172.21.15.63:0/1772854308 shutdown_connections 2024-05-27T08:03:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4820 0x55ce8d3af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d3ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ce8d3be960 0x55ce8d3c0d80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/1772854308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d3aebe0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 1 -- 172.21.15.63:0/1772854308 >> 172.21.15.63:0/1772854308 conn(0x55ce8d285fc0 msgr2=0x55ce8d39eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01aa000700 0 client.21246 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:03:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 1 -- 172.21.15.63:0/1772854308 shutdown_connections 2024-05-27T08:03:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 1 -- 172.21.15.63:0/1772854308 wait complete. 2024-05-27T08:03:59.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.458+0000 7f01d55ecb80 20 client.21246 trim_cache size 0 max 16384 2024-05-27T08:03:59.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.476+0000 7f01d55ecb80 1 Processor -- start 2024-05-27T08:03:59.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.477+0000 7f01d55ecb80 1 -- start start 2024-05-27T08:03:59.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.477+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d3a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.477+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4410 0x55ce8d3ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.477+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4820 0x55ce8d3a8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.477+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b4ef0 con 0x55ce8d3a4820 2024-05-27T08:03:59.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.477+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b2780 con 0x55ce8d225e40 2024-05-27T08:03:59.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.477+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b05f0 con 0x55ce8d3a4410 2024-05-27T08:03:59.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01c4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4820 0x55ce8d3a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01c4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4820 0x55ce8d3a8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35968/0 (socket says 172.21.15.63:35968) 2024-05-27T08:03:59.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01c4a00700 1 -- 172.21.15.63:0/621769820 learned_addr learned my addr 172.21.15.63:0/621769820 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:59.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01bf400700 1 --2- 172.21.15.63:0/621769820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4410 0x55ce8d3ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01bfe00700 1 --2- 172.21.15.63:0/621769820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d3a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01c4a00700 1 -- 172.21.15.63:0/621769820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4410 msgr2=0x55ce8d3ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:59.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01c4a00700 1 --2- 172.21.15.63:0/621769820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4410 0x55ce8d3ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01c4a00700 1 -- 172.21.15.63:0/621769820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 msgr2=0x55ce8d3a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:59.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01c4a00700 1 --2- 172.21.15.63:0/621769820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d3a9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01c4a00700 1 -- 172.21.15.63:0/621769820 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce8d3a1350 con 0x55ce8d3a4820 2024-05-27T08:03:59.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.478+0000 7f01bfe00700 1 --2- 172.21.15.63:0/621769820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d3a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:59.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.479+0000 7f01c4a00700 1 --2- 172.21.15.63:0/621769820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4820 0x55ce8d3a8880 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009e60 tx=0x7f01b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7c7e46849d01fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:59.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.479+0000 7f01b4c00700 1 -- 172.21.15.63:0/621769820 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8009230 con 0x55ce8d3a4820 2024-05-27T08:03:59.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.479+0000 7f01b4c00700 1 -- 172.21.15.63:0/621769820 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01b80093d0 con 0x55ce8d3a4820 2024-05-27T08:03:59.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.479+0000 7f01b4c00700 1 -- 172.21.15.63:0/621769820 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80050b0 con 0x55ce8d3a4820 2024-05-27T08:03:59.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.480+0000 7f01d55ecb80 1 -- 172.21.15.63:0/621769820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4820 msgr2=0x55ce8d3a8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:59.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.480+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/621769820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4820 0x55ce8d3a8880 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009e60 tx=0x7f01b800b040 comp rx=0 tx=0).stop 2024-05-27T08:03:59.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.480+0000 7f01d55ecb80 1 -- 172.21.15.63:0/621769820 shutdown_connections 2024-05-27T08:03:59.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.480+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/621769820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4820 0x55ce8d3a8880 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.480+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/621769820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d3a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.480+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/621769820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4410 0x55ce8d3ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.480+0000 7f01d55ecb80 1 -- 172.21.15.63:0/621769820 >> 172.21.15.63:0/621769820 conn(0x55ce8d285fc0 msgr2=0x55ce8d3ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:59.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.481+0000 7f01d55ecb80 1 -- 172.21.15.63:0/621769820 shutdown_connections 2024-05-27T08:03:59.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.481+0000 7f01d55ecb80 1 -- 172.21.15.63:0/621769820 wait complete. 2024-05-27T08:03:59.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.481+0000 7f01d55ecb80 1 -- start start 2024-05-27T08:03:59.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.481+0000 7f01d55ecb80 1 Processor -- start 2024-05-27T08:03:59.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.482+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d4217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.482+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d421ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.482+0000 7f01d55ecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4820 0x55ce8d3b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.482+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b0c40 con 0x55ce8d3a4410 2024-05-27T08:03:59.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.482+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b20b0 con 0x55ce8d225e40 2024-05-27T08:03:59.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.482+0000 7f01d55ecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b1310 con 0x55ce8d3a4820 2024-05-27T08:03:59.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.482+0000 7f01d55ecb80 10 client.0 _ll_register_callbacks cb 0x55ce8d26cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:59.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.482+0000 7f01bf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d421ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.483+0000 7f01bf400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d421ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35976/0 (socket says 172.21.15.63:35976) 2024-05-27T08:03:59.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.483+0000 7f01bf400700 1 -- 172.21.15.63:0/2967151605 learned_addr learned my addr 172.21.15.63:0/2967151605 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:03:59.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.483+0000 7f01bfe00700 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d4217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.483+0000 7f01c4a00700 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4820 0x55ce8d3b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.483+0000 7f01bf400700 1 -- 172.21.15.63:0/2967151605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4820 msgr2=0x55ce8d3b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:59.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.483+0000 7f01bf400700 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4820 0x55ce8d3b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.483+0000 7f01bf400700 1 -- 172.21.15.63:0/2967151605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 msgr2=0x55ce8d4217a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:59.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.483+0000 7f01bf400700 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d4217a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.483+0000 7f01bf400700 1 -- 172.21.15.63:0/2967151605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce8d3b27e0 con 0x55ce8d3a4410 2024-05-27T08:03:59.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.483+0000 7f01bfe00700 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d4217a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:59.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.484+0000 7f01bf400700 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d421ce0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f01ac01e450 tx=0x7f01ac01e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfeb93b646b4db5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:59.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.484+0000 7f01aa000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:03:59.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.484+0000 7f01aa000700 1 -- 172.21.15.63:0/2967151605 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac03a070 con 0x55ce8d3a4410 2024-05-27T08:03:59.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.484+0000 7f01d55ecb80 1 -- 172.21.15.63:0/2967151605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b80093d0 con 0x55ce8d3a4410 2024-05-27T08:03:59.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.484+0000 7f01aa000700 1 -- 172.21.15.63:0/2967151605 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f01ac036450 con 0x55ce8d3a4410 2024-05-27T08:03:59.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.484+0000 7f01aa000700 1 -- 172.21.15.63:0/2967151605 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f01ac046050 con 0x55ce8d3a4410 2024-05-27T08:03:59.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.484+0000 7f01d55ecb80 10 client.21252 fetch_fsmap learned FSMap version 35 2024-05-27T08:03:59.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.484+0000 7f01d55ecb80 1 -- 172.21.15.63:0/2967151605 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ce8d428e20 con 0x55ce8d3a4410 2024-05-27T08:03:59.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.485+0000 7f01aa000700 1 -- 172.21.15.63:0/2967151605 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01b80093d0 con 0x55ce8d3a4410 2024-05-27T08:03:59.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.485+0000 7f01aa000700 1 -- 172.21.15.63:0/2967151605 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f01ac033040 con 0x55ce8d3a4410 2024-05-27T08:03:59.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.485+0000 7f01d55ecb80 10 client.21252 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:03:59.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.485+0000 7f01d55ecb80 10 client.21252 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:03:59.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.485+0000 7f01d55ecb80 20 client.21252 populate_metadata read hostname 'smithi063' 2024-05-27T08:03:59.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.485+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ce8d4294d0 0x55ce8d42b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:59.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.485+0000 7f01d55ecb80 4 client.21252 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:03:59.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.485+0000 7f01d55ecb80 1 -- 172.21.15.63:0/2967151605 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce8d2ce570 con 0x55ce8d4294d0 2024-05-27T08:03:59.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.486+0000 7f01bfe00700 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ce8d4294d0 0x55ce8d42b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:59.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.487+0000 7f01bfe00700 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ce8d4294d0 0x55ce8d42b8f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f97b3c062d980c server_cookie=492b0c692dcca300 in_seq=0 out_seq=0 2024-05-27T08:03:59.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.487+0000 7f01aa000700 10 client.21252 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:03:59.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.487+0000 7f01aa000700 1 -- 172.21.15.63:0/2967151605 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce8d2ce570 con 0x55ce8d4294d0 2024-05-27T08:03:59.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.487+0000 7f01aa000700 10 client.21252 handle_command_reply: tid=0 2024-05-27T08:03:59.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.487+0000 7f01d55ecb80 1 client.21252 shutdown 2024-05-27T08:03:59.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 1 -- 172.21.15.63:0/2967151605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 msgr2=0x55ce8d421ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:59.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d421ce0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f01ac01e450 tx=0x7f01ac01e800 comp rx=0 tx=0).stop 2024-05-27T08:03:59.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 1 -- 172.21.15.63:0/2967151605 shutdown_connections 2024-05-27T08:03:59.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d3a4820 0x55ce8d3b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce8d225e40 0x55ce8d4217a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ce8d4294d0 0x55ce8d42b8f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 1 --2- 172.21.15.63:0/2967151605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce8d3a4410 0x55ce8d421ce0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 1 -- 172.21.15.63:0/2967151605 >> 172.21.15.63:0/2967151605 conn(0x55ce8d285fc0 msgr2=0x55ce8d3aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:59.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01aa000700 0 client.21252 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:03:59.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 1 -- 172.21.15.63:0/2967151605 shutdown_connections 2024-05-27T08:03:59.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 1 -- 172.21.15.63:0/2967151605 wait complete. 2024-05-27T08:03:59.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.488+0000 7f01d55ecb80 20 client.21252 trim_cache size 0 max 16384 2024-05-27T08:03:59.492 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.489+0000 7f01c5400700 1 -- 172.21.15.63:0/1055163500 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01a40b29f0 msgr2=0x7f01a40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.489+0000 7f01c5400700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01a40b29f0 0x7f01a40b4f00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f01c01059a0 tx=0x7f01ac00b410 comp rx=0 tx=0).stop 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.489+0000 7f01c5400700 1 -- 172.21.15.63:0/1055163500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01c0106610 msgr2=0x7f01c01a28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.489+0000 7f01c5400700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01c0106610 0x7f01c01a28b0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f01b000d950 tx=0x7f01b000dc60 comp rx=0 tx=0).stop 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.489+0000 7f01c5400700 1 -- 172.21.15.63:0/1055163500 shutdown_connections 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.489+0000 7f01c5400700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f01a40b29f0 0x7f01a40b4f00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.489+0000 7f01c5400700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01c010b890 0x7f01c019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.489+0000 7f01c5400700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01c0106610 0x7f01c01a28b0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.489+0000 7f01c5400700 1 --2- 172.21.15.63:0/1055163500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c010af40 0x7f01c01a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.489+0000 7f01c5400700 1 -- 172.21.15.63:0/1055163500 >> 172.21.15.63:0/1055163500 conn(0x7f01c0102020 msgr2=0x7f01c010a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.490+0000 7f01c5400700 1 -- 172.21.15.63:0/1055163500 shutdown_connections 2024-05-27T08:03:59.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:03:59.490+0000 7f01c5400700 1 -- 172.21.15.63:0/1055163500 wait complete. 2024-05-27T08:03:59.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:03:59 smithi063 ceph-mon[31613]: pgmap v964: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:03:59.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:03:59 smithi086 ceph-mon[34687]: pgmap v964: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:03:59.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:03:59 smithi149 ceph-mon[35408]: pgmap v964: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:04:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.143+0000 7f7e11200640 1 -- 172.21.15.63:0/1529541274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e0c108f30 msgr2=0x7f7e0c1106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.143+0000 7f7e11200640 1 --2- 172.21.15.63:0/1529541274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e0c108f30 0x7f7e0c1106d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009be0 tx=0x7f7e0402a6d0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.143+0000 7f7e11200640 1 -- 172.21.15.63:0/1529541274 shutdown_connections 2024-05-27T08:04:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.143+0000 7f7e11200640 1 --2- 172.21.15.63:0/1529541274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e0c108f30 0x7f7e0c1106d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.143+0000 7f7e11200640 1 --2- 172.21.15.63:0/1529541274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c108470 0x7f7e0c1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.143+0000 7f7e11200640 1 --2- 172.21.15.63:0/1529541274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0c1023f0 0x7f7e0c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.143+0000 7f7e11200640 1 -- 172.21.15.63:0/1529541274 >> 172.21.15.63:0/1529541274 conn(0x7f7e0c0fd9c0 msgr2=0x7f7e0c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.143+0000 7f7e11200640 1 -- 172.21.15.63:0/1529541274 shutdown_connections 2024-05-27T08:04:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.143+0000 7f7e11200640 1 -- 172.21.15.63:0/1529541274 wait complete. 2024-05-27T08:04:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.145+0000 7f7e11200640 1 Processor -- start 2024-05-27T08:04:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.145+0000 7f7e11200640 1 -- start start 2024-05-27T08:04:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.146+0000 7f7e11200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0c1023f0 0x7f7e0c1a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.146+0000 7f7e11200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c108470 0x7f7e0c1a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.146+0000 7f7e11200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e0c108f30 0x7f7e0c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.146+0000 7f7e11200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c113cf0 con 0x7f7e0c108470 2024-05-27T08:04:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.146+0000 7f7e11200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c113b70 con 0x7f7e0c1023f0 2024-05-27T08:04:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.146+0000 7f7e11200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c113e70 con 0x7f7e0c108f30 2024-05-27T08:04:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.147+0000 7f7e09400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c108470 0x7f7e0c1a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.147+0000 7f7e09400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c108470 0x7f7e0c1a2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35992/0 (socket says 172.21.15.63:35992) 2024-05-27T08:04:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.147+0000 7f7e09e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0c1023f0 0x7f7e0c1a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.147+0000 7f7e09400640 1 -- 172.21.15.63:0/4047052406 learned_addr learned my addr 172.21.15.63:0/4047052406 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.147+0000 7f7e0a800640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e0c108f30 0x7f7e0c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.147+0000 7f7e09400640 1 -- 172.21.15.63:0/4047052406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e0c108f30 msgr2=0x7f7e0c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.147+0000 7f7e09400640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e0c108f30 0x7f7e0c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.147+0000 7f7e09400640 1 -- 172.21.15.63:0/4047052406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0c1023f0 msgr2=0x7f7e0c1a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.147+0000 7f7e09400640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0c1023f0 0x7f7e0c1a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.147+0000 7f7e09400640 1 -- 172.21.15.63:0/4047052406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c19cea0 con 0x7f7e0c108470 2024-05-27T08:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.148+0000 7f7e09400640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c108470 0x7f7e0c1a2b30 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f7df800e9a0 tx=0x7f7df800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.148+0000 7f7e02a00640 1 -- 172.21.15.63:0/4047052406 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df800cd80 con 0x7f7e0c108470 2024-05-27T08:04:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.148+0000 7f7e11200640 1 -- 172.21.15.63:0/4047052406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0c19d130 con 0x7f7e0c108470 2024-05-27T08:04:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.148+0000 7f7e02a00640 1 -- 172.21.15.63:0/4047052406 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7df80035f0 con 0x7f7e0c108470 2024-05-27T08:04:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.148+0000 7f7e02a00640 1 -- 172.21.15.63:0/4047052406 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8010690 con 0x7f7e0c108470 2024-05-27T08:04:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.149+0000 7f7e11200640 1 -- 172.21.15.63:0/4047052406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0c1a9390 con 0x7f7e0c108470 2024-05-27T08:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.153+0000 7f7e181ce740 1 Processor -- start 2024-05-27T08:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.153+0000 7f7e181ce740 1 -- start start 2024-05-27T08:04:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.153+0000 7f7e02a00640 1 -- 172.21.15.63:0/4047052406 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7df8004000 con 0x7f7e0c108470 2024-05-27T08:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562516011b0 0x5562519a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515e04d0 0x5562519a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e181ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556251914b90 con 0x5562515e04d0 2024-05-27T08:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e181ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556251904730 con 0x5562515d82f0 2024-05-27T08:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e181ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556251908770 con 0x5562516011b0 2024-05-27T08:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e09400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515e04d0 0x5562519a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e09400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515e04d0 0x5562519a42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36006/0 (socket says 172.21.15.63:36006) 2024-05-27T08:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e0a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562516011b0 0x5562519a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e09400640 1 -- 172.21.15.63:0/2315902095 learned_addr learned my addr 172.21.15.63:0/2315902095 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.154+0000 7f7e09e00640 1 --2- 172.21.15.63:0/2315902095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.155+0000 7f7e09400640 1 -- 172.21.15.63:0/2315902095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562516011b0 msgr2=0x5562519a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.155+0000 7f7e09400640 1 --2- 172.21.15.63:0/2315902095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562516011b0 0x5562519a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.155+0000 7f7e09400640 1 -- 172.21.15.63:0/2315902095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 msgr2=0x5562519a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.155+0000 7f7e09400640 1 --2- 172.21.15.63:0/2315902095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.155+0000 7f7e09400640 1 -- 172.21.15.63:0/2315902095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55625189d8b0 con 0x5562515e04d0 2024-05-27T08:04:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.155+0000 7f7e02a00640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df00b2950 0x7f7df00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.155+0000 7f7e09400640 1 --2- 172.21.15.63:0/2315902095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515e04d0 0x5562519a42e0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f7df80dfef0 tx=0x7f7df80eb740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eee6df97358071 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.156+0000 7f7e02a00640 1 -- 172.21.15.63:0/4047052406 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f7df80e2c80 con 0x7f7e0c108470 2024-05-27T08:04:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.156+0000 7f7e0a800640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df00b2950 0x7f7df00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.156+0000 7f7df7400640 1 -- 172.21.15.63:0/2315902095 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df80a5a00 con 0x5562515e04d0 2024-05-27T08:04:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.156+0000 7f7df7400640 1 -- 172.21.15.63:0/2315902095 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7df80a52c0 con 0x5562515e04d0 2024-05-27T08:04:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.156+0000 7f7df7400640 1 -- 172.21.15.63:0/2315902095 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df80a92b0 con 0x5562515e04d0 2024-05-27T08:04:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.157+0000 7f7e181ce740 1 -- 172.21.15.63:0/2315902095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515e04d0 msgr2=0x5562519a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.157+0000 7f7e0a800640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df00b2950 0x7f7df00b4e60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e04004af0 tx=0x7f7e04005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:00.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.157+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2315902095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515e04d0 0x5562519a42e0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f7df80dfef0 tx=0x7f7df80eb740 comp rx=0 tx=0).stop 2024-05-27T08:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.157+0000 7f7e181ce740 1 -- 172.21.15.63:0/2315902095 shutdown_connections 2024-05-27T08:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.157+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2315902095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515e04d0 0x5562519a42e0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.157+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2315902095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.157+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2315902095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562516011b0 0x5562519a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.157+0000 7f7e181ce740 1 -- 172.21.15.63:0/2315902095 >> 172.21.15.63:0/2315902095 conn(0x5562515f5300 msgr2=0x5562519a0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.157+0000 7f7e181ce740 1 -- 172.21.15.63:0/2315902095 shutdown_connections 2024-05-27T08:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.157+0000 7f7e181ce740 1 -- 172.21.15.63:0/2315902095 wait complete. 2024-05-27T08:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.158+0000 7f7e181ce740 1 -- start start 2024-05-27T08:04:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.158+0000 7f7e181ce740 1 Processor -- start 2024-05-27T08:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e181ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5562515ea790 con 0x5562516011b0 2024-05-27T08:04:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e181ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5562518acbc0 con 0x5562515d82f0 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e181ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5562519b0990 con 0x5562515e04d0 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e181ce740 10 client.0 _ll_register_callbacks cb 0x55625188a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e0a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e0a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36022/0 (socket says 172.21.15.63:36022) 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e0a800640 1 -- 172.21.15.63:0/2380645029 learned_addr learned my addr 172.21.15.63:0/2380645029 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e09400640 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.159+0000 7f7e09e00640 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.160+0000 7f7e0a800640 1 -- 172.21.15.63:0/2380645029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 msgr2=0x5562519ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.160+0000 7f7e0a800640 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.160+0000 7f7e0a800640 1 -- 172.21.15.63:0/2380645029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 msgr2=0x5562519adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.160+0000 7f7e0a800640 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.160+0000 7f7e0a800640 1 -- 172.21.15.63:0/2380645029 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556251619210 con 0x5562516011b0 2024-05-27T08:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.160+0000 7f7e09e00640 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.160+0000 7f7e0a800640 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b9cc0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f7e040099a0 tx=0x7f7e0402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9382e88ea10ed7e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.160+0000 7f7debe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.161+0000 7f7debe00640 1 -- 172.21.15.63:0/2380645029 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04050020 con 0x5562516011b0 2024-05-27T08:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.161+0000 7f7e181ce740 1 -- 172.21.15.63:0/2380645029 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df80a52c0 con 0x5562516011b0 2024-05-27T08:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.161+0000 7f7debe00640 1 -- 172.21.15.63:0/2380645029 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f7e04004450 con 0x5562516011b0 2024-05-27T08:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.161+0000 7f7debe00640 1 -- 172.21.15.63:0/2380645029 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7e0405b050 con 0x5562516011b0 2024-05-27T08:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.161+0000 7f7e181ce740 10 client.21264 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.161+0000 7f7e181ce740 1 -- 172.21.15.63:0/2380645029 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5562516e5620 con 0x5562516011b0 2024-05-27T08:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.161+0000 7f7debe00640 1 -- 172.21.15.63:0/2380645029 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7df80a52c0 con 0x5562516011b0 2024-05-27T08:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.162+0000 7f7debe00640 1 -- 172.21.15.63:0/2380645029 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7e0403c040 con 0x5562516011b0 2024-05-27T08:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.162+0000 7f7e181ce740 10 client.21264 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.162+0000 7f7e181ce740 10 client.21264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.162+0000 7f7e181ce740 20 client.21264 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.162+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562515bfd80 0x5562519bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.162+0000 7f7e181ce740 4 client.21264 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.162+0000 7f7e181ce740 1 -- 172.21.15.63:0/2380645029 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556251916ae0 con 0x5562515bfd80 2024-05-27T08:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.163+0000 7f7e09e00640 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562515bfd80 0x5562519bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.164+0000 7f7e09e00640 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562515bfd80 0x5562519bfdb0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c3030312eb1b92 server_cookie=e95fc440771f7daa in_seq=0 out_seq=0 2024-05-27T08:04:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.164+0000 7f7debe00640 10 client.21264 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7debe00640 1 -- 172.21.15.63:0/2380645029 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556251916ae0 con 0x5562515bfd80 2024-05-27T08:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7debe00640 10 client.21264 handle_command_reply: tid=0 2024-05-27T08:04:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7e181ce740 1 client.21264 shutdown 2024-05-27T08:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7e181ce740 1 -- 172.21.15.63:0/2380645029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 msgr2=0x5562519b9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b9cc0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f7e040099a0 tx=0x7f7e0402a6d0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7e181ce740 1 -- 172.21.15.63:0/2380645029 shutdown_connections 2024-05-27T08:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562515bfd80 0x5562519bfdb0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b9cc0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2380645029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7e181ce740 1 -- 172.21.15.63:0/2380645029 >> 172.21.15.63:0/2380645029 conn(0x5562515f5300 msgr2=0x5562519a5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:00.169 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7debe00640 0 client.21264 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.166+0000 7f7e181ce740 1 -- 172.21.15.63:0/2380645029 shutdown_connections 2024-05-27T08:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.167+0000 7f7e181ce740 1 -- 172.21.15.63:0/2380645029 wait complete. 2024-05-27T08:04:00.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.167+0000 7f7e181ce740 20 client.21264 trim_cache size 0 max 16384 2024-05-27T08:04:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.193+0000 7f7e181ce740 1 Processor -- start 2024-05-27T08:04:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.193+0000 7f7e181ce740 1 -- start start 2024-05-27T08:04:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.194+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515d82f0 0x5562519a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.194+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519b9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.194+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562516011b0 0x5562519b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.194+0000 7f7e181ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1b5ca0 con 0x5562515d82f0 2024-05-27T08:04:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.194+0000 7f7e181ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1b3570 con 0x5562516011b0 2024-05-27T08:04:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.194+0000 7f7e181ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1b13e0 con 0x5562515e04d0 2024-05-27T08:04:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.194+0000 7f7e09e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515d82f0 0x5562519a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.194+0000 7f7e09e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515d82f0 0x5562519a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36028/0 (socket says 172.21.15.63:36028) 2024-05-27T08:04:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.194+0000 7f7e09e00640 1 -- 172.21.15.63:0/2637699480 learned_addr learned my addr 172.21.15.63:0/2637699480 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.195+0000 7f7e09400640 1 --2- 172.21.15.63:0/2637699480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519b9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.195+0000 7f7e0a800640 1 --2- 172.21.15.63:0/2637699480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562516011b0 0x5562519b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.195+0000 7f7e09e00640 1 -- 172.21.15.63:0/2637699480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 msgr2=0x5562519b9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.195+0000 7f7e09e00640 1 --2- 172.21.15.63:0/2637699480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519b9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.195+0000 7f7e09e00640 1 -- 172.21.15.63:0/2637699480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562516011b0 msgr2=0x5562519b9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.195+0000 7f7e09e00640 1 --2- 172.21.15.63:0/2637699480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562516011b0 0x5562519b9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.195+0000 7f7e09e00640 1 -- 172.21.15.63:0/2637699480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562515bffb0 con 0x5562515d82f0 2024-05-27T08:04:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.195+0000 7f7e09e00640 1 --2- 172.21.15.63:0/2637699480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515d82f0 0x5562519a5b80 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x5562519a1350 tx=0x7f7dfc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a22c0d6b6c0787 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.196+0000 7f7df6a00640 1 -- 172.21.15.63:0/2637699480 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc004050 con 0x5562515d82f0 2024-05-27T08:04:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.196+0000 7f7df6a00640 1 -- 172.21.15.63:0/2637699480 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dfc0035f0 con 0x5562515d82f0 2024-05-27T08:04:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.196+0000 7f7df6a00640 1 -- 172.21.15.63:0/2637699480 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc01e680 con 0x5562515d82f0 2024-05-27T08:04:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.197+0000 7f7e181ce740 1 -- 172.21.15.63:0/2637699480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515d82f0 msgr2=0x5562519a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.197+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2637699480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515d82f0 0x5562519a5b80 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x5562519a1350 tx=0x7f7dfc005f90 comp rx=0 tx=0).stop 2024-05-27T08:04:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.197+0000 7f7e181ce740 1 -- 172.21.15.63:0/2637699480 shutdown_connections 2024-05-27T08:04:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.197+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2637699480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562516011b0 0x5562519b9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.197+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2637699480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519b9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.197+0000 7f7e181ce740 1 --2- 172.21.15.63:0/2637699480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562515d82f0 0x5562519a5b80 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.197+0000 7f7e181ce740 1 -- 172.21.15.63:0/2637699480 >> 172.21.15.63:0/2637699480 conn(0x5562515f5300 msgr2=0x5562519a4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.197+0000 7f7e181ce740 1 -- 172.21.15.63:0/2637699480 shutdown_connections 2024-05-27T08:04:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.197+0000 7f7e181ce740 1 -- 172.21.15.63:0/2637699480 wait complete. 2024-05-27T08:04:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.198+0000 7f7e181ce740 1 -- start start 2024-05-27T08:04:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.198+0000 7f7e181ce740 1 Processor -- start 2024-05-27T08:04:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.198+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.198+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.198+0000 7f7e181ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.198+0000 7f7e181ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1b1a30 con 0x5562516011b0 2024-05-27T08:04:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.198+0000 7f7e181ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1b2ea0 con 0x5562515d82f0 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.198+0000 7f7e181ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1b2100 con 0x5562515e04d0 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.198+0000 7f7e181ce740 10 client.0 _ll_register_callbacks cb 0x7f7e04043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e0a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e0a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36040/0 (socket says 172.21.15.63:36040) 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e0a800640 1 -- 172.21.15.63:0/1848118190 learned_addr learned my addr 172.21.15.63:0/1848118190 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e09e00640 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e09400640 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e0a800640 1 -- 172.21.15.63:0/1848118190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 msgr2=0x5562519b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e0a800640 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e0a800640 1 -- 172.21.15.63:0/1848118190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 msgr2=0x5562519b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e0a800640 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e0a800640 1 -- 172.21.15.63:0/1848118190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556251a26620 con 0x5562516011b0 2024-05-27T08:04:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.199+0000 7f7e09400640 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.200+0000 7f7e0a800640 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b1660 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f7e0403f4c0 tx=0x7f7e04030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ec5bab133d66c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.200+0000 7f7debe00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.200+0000 7f7debe00640 1 -- 172.21.15.63:0/1848118190 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04050070 con 0x5562516011b0 2024-05-27T08:04:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.200+0000 7f7debe00640 1 -- 172.21.15.63:0/1848118190 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f7e04004250 con 0x5562516011b0 2024-05-27T08:04:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.200+0000 7f7e181ce740 1 -- 172.21.15.63:0/1848118190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dfc0035f0 con 0x5562516011b0 2024-05-27T08:04:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.200+0000 7f7debe00640 1 -- 172.21.15.63:0/1848118190 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7e04036040 con 0x5562516011b0 2024-05-27T08:04:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.201+0000 7f7e181ce740 10 client.21270 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.201+0000 7f7e181ce740 1 -- 172.21.15.63:0/1848118190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556251a27890 con 0x5562516011b0 2024-05-27T08:04:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.201+0000 7f7debe00640 1 -- 172.21.15.63:0/1848118190 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7dfc0035f0 con 0x5562516011b0 2024-05-27T08:04:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.201+0000 7f7debe00640 1 -- 172.21.15.63:0/1848118190 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7e0403f960 con 0x5562516011b0 2024-05-27T08:04:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.201+0000 7f7e181ce740 10 client.21270 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.201+0000 7f7e181ce740 10 client.21270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.201+0000 7f7e181ce740 20 client.21270 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.201+0000 7f7e181ce740 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562515951c0 0x556251a29f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.201+0000 7f7e181ce740 4 client.21270 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:04:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.201+0000 7f7e181ce740 1 -- 172.21.15.63:0/1848118190 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562519102c0 con 0x5562515951c0 2024-05-27T08:04:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.202+0000 7f7e09e00640 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562515951c0 0x556251a29f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.203+0000 7f7e09e00640 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562515951c0 0x556251a29f80 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7218e263b73e5bb server_cookie=5dab3b0c32b7ddcb in_seq=0 out_seq=0 2024-05-27T08:04:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.203+0000 7f7debe00640 10 client.21270 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.235+0000 7f7debe00640 1 -- 172.21.15.63:0/1848118190 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5562519102c0 con 0x5562515951c0 2024-05-27T08:04:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.235+0000 7f7debe00640 10 client.21270 handle_command_reply: tid=0 2024-05-27T08:04:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.235+0000 7f7e181ce740 1 client.21270 shutdown 2024-05-27T08:04:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 1 -- 172.21.15.63:0/1848118190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 msgr2=0x5562519b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b1660 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f7e0403f4c0 tx=0x7f7e04030d90 comp rx=0 tx=0).stop 2024-05-27T08:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 1 -- 172.21.15.63:0/1848118190 shutdown_connections 2024-05-27T08:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5562515951c0 0x556251a29f80 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5562516011b0 0x5562519b1660 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5562515e04d0 0x5562519b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 1 --2- 172.21.15.63:0/1848118190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562515d82f0 0x5562519b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 1 -- 172.21.15.63:0/1848118190 >> 172.21.15.63:0/1848118190 conn(0x5562515f5300 msgr2=0x5562519a3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7debe00640 0 client.21270 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 1 -- 172.21.15.63:0/1848118190 shutdown_connections 2024-05-27T08:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 1 -- 172.21.15.63:0/1848118190 wait complete. 2024-05-27T08:04:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.236+0000 7f7e181ce740 20 client.21270 trim_cache size 0 max 16384 2024-05-27T08:04:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:04:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:04:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:04:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 -- 172.21.15.63:0/4047052406 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df00b2950 msgr2=0x7f7df00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df00b2950 0x7f7df00b4e60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e04004af0 tx=0x7f7e04005fd0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 -- 172.21.15.63:0/4047052406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c108470 msgr2=0x7f7e0c1a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c108470 0x7f7e0c1a2b30 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f7df800e9a0 tx=0x7f7df800ee60 comp rx=0 tx=0).stop 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 -- 172.21.15.63:0/4047052406 shutdown_connections 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7df00b2950 0x7f7df00b4e60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e0c108f30 0x7f7e0c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c108470 0x7f7e0c1a2b30 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 --2- 172.21.15.63:0/4047052406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0c1023f0 0x7f7e0c1a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 -- 172.21.15.63:0/4047052406 >> 172.21.15.63:0/4047052406 conn(0x7f7e0c0fd9c0 msgr2=0x7f7e0c1045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 -- 172.21.15.63:0/4047052406 shutdown_connections 2024-05-27T08:04:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:00.237+0000 7f7e11200640 1 -- 172.21.15.63:0/4047052406 wait complete. 2024-05-27T08:04:00.528 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:04:01.016 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:01.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:01 smithi086 ceph-mon[34687]: pgmap v965: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:04:01.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:01 smithi149 ceph-mon[35408]: pgmap v965: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:04:01.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:01 smithi063 ceph-mon[31613]: pgmap v965: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:04:03.560 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:03 smithi063 ceph-mon[31613]: pgmap v966: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.6 MiB/s wr, 34 op/s 2024-05-27T08:04:03.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.556+0000 7f561f000700 1 -- 172.21.15.63:0/177056703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5618106560 msgr2=0x7f5618106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.556+0000 7f561f000700 1 --2- 172.21.15.63:0/177056703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5618106560 0x7f5618106970 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f5604009990 tx=0x7f5604009ca0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.556+0000 7f561f000700 1 -- 172.21.15.63:0/177056703 shutdown_connections 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.556+0000 7f561f000700 1 --2- 172.21.15.63:0/177056703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618106eb0 0x7f5618113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.556+0000 7f561f000700 1 --2- 172.21.15.63:0/177056703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f561810c5a0 0x7f561810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.556+0000 7f561f000700 1 --2- 172.21.15.63:0/177056703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5618106560 0x7f5618106970 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.556+0000 7f561f000700 1 -- 172.21.15.63:0/177056703 >> 172.21.15.63:0/177056703 conn(0x7f5618102040 msgr2=0x7f5618104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.557+0000 7f561f000700 1 -- 172.21.15.63:0/177056703 shutdown_connections 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.557+0000 7f561f000700 1 -- 172.21.15.63:0/177056703 wait complete. 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.558+0000 7f561f000700 1 Processor -- start 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.558+0000 7f561f000700 1 -- start start 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.559+0000 7f561f000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5618106560 0x7f561819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.559+0000 7f561f000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618106eb0 0x7f561819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.559+0000 7f561f000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f561810c5a0 0x7f56181a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.559+0000 7f561f000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5618116580 con 0x7f5618106560 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.559+0000 7f561f000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5618116400 con 0x7f5618106eb0 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.559+0000 7f561f000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5618116700 con 0x7f561810c5a0 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.559+0000 7f5617e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5618106560 0x7f561819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.559+0000 7f5617e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5618106560 0x7f561819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36060/0 (socket says 172.21.15.63:36060) 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.559+0000 7f5617e00700 1 -- 172.21.15.63:0/2827500255 learned_addr learned my addr 172.21.15.63:0/2827500255 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.560+0000 7f5617400700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618106eb0 0x7f561819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.560+0000 7f561d000700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f561810c5a0 0x7f56181a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.560+0000 7f5617e00700 1 -- 172.21.15.63:0/2827500255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f561810c5a0 msgr2=0x7f56181a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.560+0000 7f5617e00700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f561810c5a0 0x7f56181a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.560+0000 7f5617e00700 1 -- 172.21.15.63:0/2827500255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618106eb0 msgr2=0x7f561819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.560+0000 7f5617e00700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618106eb0 0x7f561819d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.560+0000 7f5617e00700 1 -- 172.21.15.63:0/2827500255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56181a1de0 con 0x7f5618106560 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.560+0000 7f5617400700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618106eb0 0x7f561819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.560+0000 7f5617e00700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5618106560 0x7f561819cf10 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f560800d980 tx=0x7f560800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.561+0000 7f5614c00700 1 -- 172.21.15.63:0/2827500255 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608017070 con 0x7f5618106560 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.561+0000 7f5614c00700 1 -- 172.21.15.63:0/2827500255 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5608010460 con 0x7f5618106560 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.561+0000 7f5614c00700 1 -- 172.21.15.63:0/2827500255 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560800de60 con 0x7f5618106560 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.561+0000 7f561f000700 1 -- 172.21.15.63:0/2827500255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56181a20d0 con 0x7f5618106560 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.561+0000 7f561f000700 1 -- 172.21.15.63:0/2827500255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56181a99f0 con 0x7f5618106560 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.566+0000 7f5614c00700 1 -- 172.21.15.63:0/2827500255 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5608010610 con 0x7f5618106560 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.567+0000 7f562da96b80 1 Processor -- start 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.567+0000 7f562da96b80 1 -- start start 2024-05-27T08:04:03.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.567+0000 7f562da96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fed37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.567+0000 7f562da96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed3d30 0x556b4fecd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.567+0000 7f562da96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fecddb0 0x556b4fece1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.567+0000 7f562da96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556b4fdfbe40 con 0x556b4fd4ee40 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.567+0000 7f562da96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b4fdeb9e0 con 0x556b4fecddb0 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.568+0000 7f562da96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556b4fdefa20 con 0x556b4fed3d30 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.568+0000 7f561d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fed37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.568+0000 7f561d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fed37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36062/0 (socket says 172.21.15.63:36062) 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.568+0000 7f561d000700 1 -- 172.21.15.63:0/1414700796 learned_addr learned my addr 172.21.15.63:0/1414700796 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.568+0000 7f5614c00700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55fc0b2a70 0x7f55fc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.568+0000 7f5617400700 1 --2- 172.21.15.63:0/1414700796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fecddb0 0x556b4fece1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.568+0000 7f5617e00700 1 --2- 172.21.15.63:0/1414700796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed3d30 0x556b4fecd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.568+0000 7f5614c00700 1 -- 172.21.15.63:0/2827500255 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f56080e2550 con 0x7f5618106560 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.569+0000 7f561d000700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55fc0b2a70 0x7f55fc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.569+0000 7f561d000700 1 -- 172.21.15.63:0/1414700796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed3d30 msgr2=0x556b4fecd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.569+0000 7f561d000700 1 --2- 172.21.15.63:0/1414700796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed3d30 0x556b4fecd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.569+0000 7f561d000700 1 -- 172.21.15.63:0/1414700796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fecddb0 msgr2=0x556b4fece1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.569+0000 7f561d000700 1 --2- 172.21.15.63:0/1414700796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fecddb0 0x556b4fece1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.569+0000 7f561d000700 1 -- 172.21.15.63:0/1414700796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4fdb4420 con 0x556b4fd4ee40 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.570+0000 7f561d000700 1 --2- 172.21.15.63:0/1414700796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fed37f0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f5610007ef0 tx=0x7f56100185e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50100bba5aa52c22 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.570+0000 7f561d000700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55fc0b2a70 0x7f55fc0b4f80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5610018930 tx=0x7f561000d000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.570+0000 7f560d600700 1 -- 172.21.15.63:0/1414700796 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561001f070 con 0x556b4fd4ee40 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.570+0000 7f560d600700 1 -- 172.21.15.63:0/1414700796 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f561001ab80 con 0x556b4fd4ee40 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.570+0000 7f560d600700 1 -- 172.21.15.63:0/1414700796 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610012040 con 0x556b4fd4ee40 2024-05-27T08:04:03.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.571+0000 7f562da96b80 1 -- 172.21.15.63:0/1414700796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 msgr2=0x556b4fed37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.571+0000 7f562da96b80 1 --2- 172.21.15.63:0/1414700796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fed37f0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f5610007ef0 tx=0x7f56100185e0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.571+0000 7f562da96b80 1 -- 172.21.15.63:0/1414700796 shutdown_connections 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.571+0000 7f562da96b80 1 --2- 172.21.15.63:0/1414700796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fecddb0 0x556b4fece1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.571+0000 7f562da96b80 1 --2- 172.21.15.63:0/1414700796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fed37f0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.571+0000 7f562da96b80 1 --2- 172.21.15.63:0/1414700796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed3d30 0x556b4fecd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.571+0000 7f562da96b80 1 -- 172.21.15.63:0/1414700796 >> 172.21.15.63:0/1414700796 conn(0x556b4fdaefc0 msgr2=0x556b4fec88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.571+0000 7f562da96b80 1 -- 172.21.15.63:0/1414700796 shutdown_connections 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.571+0000 7f562da96b80 1 -- 172.21.15.63:0/1414700796 wait complete. 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.572+0000 7f562da96b80 1 -- start start 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.572+0000 7f562da96b80 1 Processor -- start 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.573+0000 7f562da96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fd4ee40 0x556b4fed75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.573+0000 7f562da96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fecddb0 0x556b4fed7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.573+0000 7f562da96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4fed8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.573+0000 7f562da96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556b4fcf1fa0 con 0x556b4fecddb0 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.573+0000 7f562da96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b4fd8c760 con 0x556b4fed3d30 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.573+0000 7f562da96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556b4feda740 con 0x556b4fd4ee40 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.573+0000 7f562da96b80 10 client.0 _ll_register_callbacks cb 0x556b4fd95c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.573+0000 7f5617400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fecddb0 0x556b4fed7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.573+0000 7f5617e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fd4ee40 0x556b4fed75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.574+0000 7f5617400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fecddb0 0x556b4fed7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36064/0 (socket says 172.21.15.63:36064) 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.574+0000 7f5617400700 1 -- 172.21.15.63:0/1034899685 learned_addr learned my addr 172.21.15.63:0/1034899685 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.574+0000 7f561d000700 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4fed8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.574+0000 7f5617400700 1 -- 172.21.15.63:0/1034899685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fd4ee40 msgr2=0x556b4fed75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.574+0000 7f5617400700 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fd4ee40 0x556b4fed75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.574+0000 7f5617400700 1 -- 172.21.15.63:0/1034899685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 msgr2=0x556b4fed8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.574+0000 7f5617400700 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4fed8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.574+0000 7f5617400700 1 -- 172.21.15.63:0/1034899685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4fd3b8d0 con 0x556b4fecddb0 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.575+0000 7f5617400700 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fecddb0 0x556b4fed7b00 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f5618075b70 tx=0x7f5604003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb1a7a572257eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.575+0000 7f5602000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.575+0000 7f5602000700 1 -- 172.21.15.63:0/1034899685 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604029060 con 0x556b4fecddb0 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.575+0000 7f5602000700 1 -- 172.21.15.63:0/1034899685 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f5604005450 con 0x556b4fecddb0 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.575+0000 7f562da96b80 1 -- 172.21.15.63:0/1034899685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f561001ab80 con 0x556b4fecddb0 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.575+0000 7f5602000700 1 -- 172.21.15.63:0/1034899685 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5604030050 con 0x556b4fecddb0 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.575+0000 7f562da96b80 10 client.21282 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.576+0000 7f562da96b80 1 -- 172.21.15.63:0/1034899685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556b4fdbc060 con 0x556b4fecddb0 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.576+0000 7f5602000700 1 -- 172.21.15.63:0/1034899685 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f561001ab80 con 0x556b4fecddb0 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.576+0000 7f5602000700 1 -- 172.21.15.63:0/1034899685 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f560400f460 con 0x556b4fecddb0 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.576+0000 7f562da96b80 10 client.21282 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.576+0000 7f562da96b80 10 client.21282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:04:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.576+0000 7f562da96b80 20 client.21282 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.576+0000 7f562da96b80 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556b4fee7750 0x556b4fee9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.576+0000 7f562da96b80 4 client.21282 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.576+0000 7f562da96b80 1 -- 172.21.15.63:0/1034899685 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b4fdfdd90 con 0x556b4fee7750 2024-05-27T08:04:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.577+0000 7f5617e00700 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556b4fee7750 0x556b4fee9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.578+0000 7f5617e00700 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556b4fee7750 0x556b4fee9b90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57642f90874e1182 server_cookie=619e9c3f61cf1396 in_seq=0 out_seq=0 2024-05-27T08:04:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.578+0000 7f5602000700 10 client.21282 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:04:03.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f5602000700 1 -- 172.21.15.63:0/1034899685 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556b4fdfdd90 con 0x556b4fee7750 2024-05-27T08:04:03.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f5602000700 10 client.21282 handle_command_reply: tid=0 2024-05-27T08:04:03.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f562da96b80 1 client.21282 shutdown 2024-05-27T08:04:03.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f562da96b80 1 -- 172.21.15.63:0/1034899685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fecddb0 msgr2=0x556b4fed7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:03.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f562da96b80 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fecddb0 0x556b4fed7b00 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f5618075b70 tx=0x7f5604003e60 comp rx=0 tx=0).stop 2024-05-27T08:04:03.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f562da96b80 1 -- 172.21.15.63:0/1034899685 shutdown_connections 2024-05-27T08:04:03.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f562da96b80 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4fed8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f562da96b80 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fd4ee40 0x556b4fed75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f562da96b80 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556b4fee7750 0x556b4fee9b90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f562da96b80 1 --2- 172.21.15.63:0/1034899685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fecddb0 0x556b4fed7b00 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.580+0000 7f562da96b80 1 -- 172.21.15.63:0/1034899685 >> 172.21.15.63:0/1034899685 conn(0x556b4fdaefc0 msgr2=0x556b4fec8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.581+0000 7f5602000700 0 client.21282 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:04:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.581+0000 7f562da96b80 1 -- 172.21.15.63:0/1034899685 shutdown_connections 2024-05-27T08:04:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.581+0000 7f562da96b80 1 -- 172.21.15.63:0/1034899685 wait complete. 2024-05-27T08:04:03.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.581+0000 7f562da96b80 20 client.21282 trim_cache size 0 max 16384 2024-05-27T08:04:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.600+0000 7f562da96b80 1 Processor -- start 2024-05-27T08:04:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.600+0000 7f562da96b80 1 -- start start 2024-05-27T08:04:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.601+0000 7f562da96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fecd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.601+0000 7f562da96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4fed67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.601+0000 7f562da96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed5330 0x556b4fee37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.601+0000 7f562da96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56181b4f50 con 0x556b4fd4ee40 2024-05-27T08:04:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.601+0000 7f562da96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56181b27e0 con 0x556b4fed3d30 2024-05-27T08:04:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.601+0000 7f562da96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56181b0650 con 0x556b4fed5330 2024-05-27T08:04:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.601+0000 7f5617e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fecd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.601+0000 7f5617e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fecd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36076/0 (socket says 172.21.15.63:36076) 2024-05-27T08:04:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.601+0000 7f5617e00700 1 -- 172.21.15.63:0/1411213364 learned_addr learned my addr 172.21.15.63:0/1411213364 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.602+0000 7f5617400700 1 --2- 172.21.15.63:0/1411213364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4fed67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.602+0000 7f561d000700 1 --2- 172.21.15.63:0/1411213364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed5330 0x556b4fee37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.602+0000 7f5617e00700 1 -- 172.21.15.63:0/1411213364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed5330 msgr2=0x556b4fee37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.602+0000 7f5617e00700 1 --2- 172.21.15.63:0/1411213364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed5330 0x556b4fee37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.602+0000 7f5617e00700 1 -- 172.21.15.63:0/1411213364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 msgr2=0x556b4fed67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.602+0000 7f5617e00700 1 --2- 172.21.15.63:0/1411213364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4fed67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.602+0000 7f5617e00700 1 -- 172.21.15.63:0/1411213364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4fec2130 con 0x556b4fd4ee40 2024-05-27T08:04:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.602+0000 7f561d000700 1 --2- 172.21.15.63:0/1411213364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed5330 0x556b4fee37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.602+0000 7f5617e00700 1 --2- 172.21.15.63:0/1411213364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fecd870 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f56080a9110 tx=0x7f560801ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5983e72a62b8e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.602+0000 7f560cc00700 1 -- 172.21.15.63:0/1411213364 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56080dfd00 con 0x556b4fd4ee40 2024-05-27T08:04:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.603+0000 7f560cc00700 1 -- 172.21.15.63:0/1411213364 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56080a8f20 con 0x556b4fd4ee40 2024-05-27T08:04:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.603+0000 7f560cc00700 1 -- 172.21.15.63:0/1411213364 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608018270 con 0x556b4fd4ee40 2024-05-27T08:04:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.603+0000 7f562da96b80 1 -- 172.21.15.63:0/1411213364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 msgr2=0x556b4fecd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.603+0000 7f562da96b80 1 --2- 172.21.15.63:0/1411213364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fecd870 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f56080a9110 tx=0x7f560801ff90 comp rx=0 tx=0).stop 2024-05-27T08:04:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.604+0000 7f562da96b80 1 -- 172.21.15.63:0/1411213364 shutdown_connections 2024-05-27T08:04:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.604+0000 7f562da96b80 1 --2- 172.21.15.63:0/1411213364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fed5330 0x556b4fee37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.604+0000 7f562da96b80 1 --2- 172.21.15.63:0/1411213364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fd4ee40 0x556b4fecd870 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.604+0000 7f562da96b80 1 --2- 172.21.15.63:0/1411213364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4fed67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.604+0000 7f562da96b80 1 -- 172.21.15.63:0/1411213364 >> 172.21.15.63:0/1411213364 conn(0x556b4fdaefc0 msgr2=0x556b4fed73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.604+0000 7f562da96b80 1 -- 172.21.15.63:0/1411213364 shutdown_connections 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.604+0000 7f562da96b80 1 -- 172.21.15.63:0/1411213364 wait complete. 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.604+0000 7f562da96b80 1 -- start start 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.605+0000 7f562da96b80 1 Processor -- start 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.605+0000 7f562da96b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fd4ee40 0x556b4fedffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.605+0000 7f562da96b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4feeeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.605+0000 7f562da96b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fed5330 0x556b4feef020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.605+0000 7f562da96b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56181b0ca0 con 0x556b4fed5330 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.605+0000 7f562da96b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56181b2110 con 0x556b4fed3d30 2024-05-27T08:04:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.605+0000 7f562da96b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56181b1370 con 0x556b4fd4ee40 2024-05-27T08:04:03.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.605+0000 7f562da96b80 10 client.0 _ll_register_callbacks cb 0x556b4fd95c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f561d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fed5330 0x556b4feef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f561d000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fed5330 0x556b4feef020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36086/0 (socket says 172.21.15.63:36086) 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f561d000700 1 -- 172.21.15.63:0/648367095 learned_addr learned my addr 172.21.15.63:0/648367095 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f5617e00700 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fd4ee40 0x556b4fedffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f5617400700 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4feeeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f561d000700 1 -- 172.21.15.63:0/648367095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fd4ee40 msgr2=0x556b4fedffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f561d000700 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fd4ee40 0x556b4fedffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f561d000700 1 -- 172.21.15.63:0/648367095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 msgr2=0x556b4feeeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f561d000700 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4feeeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f561d000700 1 -- 172.21.15.63:0/648367095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4feef6c0 con 0x556b4fed5330 2024-05-27T08:04:03.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.606+0000 7f5617400700 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4feeeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.607+0000 7f561d000700 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fed5330 0x556b4feef020 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f5610009690 tx=0x7f5610006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ab11bdb987b0c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.607+0000 7f5602000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.607+0000 7f5602000700 1 -- 172.21.15.63:0/648367095 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610030020 con 0x556b4fed5330 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.607+0000 7f562da96b80 1 -- 172.21.15.63:0/648367095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56080a8f20 con 0x556b4fed5330 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.607+0000 7f5602000700 1 -- 172.21.15.63:0/648367095 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9483+0+0 (secure 0 0 0) 0x7f561001b7d0 con 0x556b4fed5330 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.607+0000 7f5602000700 1 -- 172.21.15.63:0/648367095 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f561003b050 con 0x556b4fed5330 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.607+0000 7f562da96b80 10 client.21288 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.607+0000 7f562da96b80 1 -- 172.21.15.63:0/648367095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556b4ff51fc0 con 0x556b4fed5330 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.608+0000 7f5602000700 1 -- 172.21.15.63:0/648367095 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56080a8f20 con 0x556b4fed5330 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.608+0000 7f5602000700 1 -- 172.21.15.63:0/648367095 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f561001aaa0 con 0x556b4fed5330 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.608+0000 7f562da96b80 10 client.21288 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.608+0000 7f562da96b80 10 client.21288 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:04:03.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.608+0000 7f562da96b80 20 client.21288 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:03.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.608+0000 7f562da96b80 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556b4ff523e0 0x556b4ff54800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:03.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.608+0000 7f562da96b80 4 client.21288 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:04:03.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.608+0000 7f562da96b80 1 -- 172.21.15.63:0/648367095 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b4fdf7570 con 0x556b4ff523e0 2024-05-27T08:04:03.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.609+0000 7f5617e00700 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556b4ff523e0 0x556b4ff54800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:03.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.610+0000 7f5617e00700 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556b4ff523e0 0x556b4ff54800 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a482f8cc932f468d server_cookie=5d03ff4a6ecc510b in_seq=0 out_seq=0 2024-05-27T08:04:03.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.610+0000 7f5602000700 10 client.21288 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:04:03.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.610+0000 7f5602000700 1 -- 172.21.15.63:0/648367095 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b4fdf7570 con 0x556b4ff523e0 2024-05-27T08:04:03.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.610+0000 7f5602000700 10 client.21288 handle_command_reply: tid=0 2024-05-27T08:04:03.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.610+0000 7f562da96b80 1 client.21288 shutdown 2024-05-27T08:04:03.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 1 -- 172.21.15.63:0/648367095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fed5330 msgr2=0x556b4feef020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:03.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fed5330 0x556b4feef020 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f5610009690 tx=0x7f5610006000 comp rx=0 tx=0).stop 2024-05-27T08:04:03.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 1 -- 172.21.15.63:0/648367095 shutdown_connections 2024-05-27T08:04:03.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x556b4ff523e0 0x556b4ff54800 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556b4fed5330 0x556b4feef020 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556b4fd4ee40 0x556b4fedffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 1 --2- 172.21.15.63:0/648367095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b4fed3d30 0x556b4feeeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 1 -- 172.21.15.63:0/648367095 >> 172.21.15.63:0/648367095 conn(0x556b4fdaefc0 msgr2=0x556b4fed6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:03.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f5602000700 0 client.21288 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:04:03.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 1 -- 172.21.15.63:0/648367095 shutdown_connections 2024-05-27T08:04:03.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 1 -- 172.21.15.63:0/648367095 wait complete. 2024-05-27T08:04:03.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.611+0000 7f562da96b80 20 client.21288 trim_cache size 0 max 16384 2024-05-27T08:04:03.617 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:04:03.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.612+0000 7f561f000700 1 -- 172.21.15.63:0/2827500255 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55fc0b2a70 msgr2=0x7f55fc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:03.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.612+0000 7f561f000700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55fc0b2a70 0x7f55fc0b4f80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5610018930 tx=0x7f561000d000 comp rx=0 tx=0).stop 2024-05-27T08:04:03.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.612+0000 7f561f000700 1 -- 172.21.15.63:0/2827500255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5618106560 msgr2=0x7f561819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:03.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.612+0000 7f561f000700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5618106560 0x7f561819cf10 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f560800d980 tx=0x7f560800dd40 comp rx=0 tx=0).stop 2024-05-27T08:04:03.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.612+0000 7f561f000700 1 -- 172.21.15.63:0/2827500255 shutdown_connections 2024-05-27T08:04:03.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.612+0000 7f561f000700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55fc0b2a70 0x7f55fc0b4f80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.612+0000 7f561f000700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f561810c5a0 0x7f56181a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.612+0000 7f561f000700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5618106560 0x7f561819cf10 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.612+0000 7f561f000700 1 --2- 172.21.15.63:0/2827500255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5618106eb0 0x7f561819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:03.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.612+0000 7f561f000700 1 -- 172.21.15.63:0/2827500255 >> 172.21.15.63:0/2827500255 conn(0x7f5618102040 msgr2=0x7f56181038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:03.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.613+0000 7f561f000700 1 -- 172.21.15.63:0/2827500255 shutdown_connections 2024-05-27T08:04:03.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:03.613+0000 7f561f000700 1 -- 172.21.15.63:0/2827500255 wait complete. 2024-05-27T08:04:03.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:03 smithi086 ceph-mon[34687]: pgmap v966: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.6 MiB/s wr, 34 op/s 2024-05-27T08:04:03.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:03 smithi149 ceph-mon[35408]: pgmap v966: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.6 MiB/s wr, 34 op/s 2024-05-27T08:04:04.613 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:04:05.187 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:05.597 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:05 smithi063 ceph-mon[31613]: pgmap v967: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.2 MiB/s wr, 27 op/s 2024-05-27T08:04:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:05 smithi086 ceph-mon[34687]: pgmap v967: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.2 MiB/s wr, 27 op/s 2024-05-27T08:04:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:05 smithi149 ceph-mon[35408]: pgmap v967: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.2 MiB/s wr, 27 op/s 2024-05-27T08:04:07.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.664+0000 7f5611e00700 1 -- 172.21.15.63:0/2875876732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f560c1054d0 msgr2=0x7f560c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:07.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.664+0000 7f5611e00700 1 --2- 172.21.15.63:0/2875876732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f560c1054d0 0x7f560c1058e0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009990 tx=0x7f55fc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.665+0000 7f5611e00700 1 -- 172.21.15.63:0/2875876732 shutdown_connections 2024-05-27T08:04:07.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.665+0000 7f5611e00700 1 --2- 172.21.15.63:0/2875876732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f560c10e900 0x7f560c110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.665+0000 7f5611e00700 1 --2- 172.21.15.63:0/2875876732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f560c1054d0 0x7f560c1058e0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.665+0000 7f5611e00700 1 --2- 172.21.15.63:0/2875876732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c105e20 0x7f560c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.665+0000 7f5611e00700 1 -- 172.21.15.63:0/2875876732 >> 172.21.15.63:0/2875876732 conn(0x7f560c0fedb0 msgr2=0x7f560c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:07.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.665+0000 7f5611e00700 1 -- 172.21.15.63:0/2875876732 shutdown_connections 2024-05-27T08:04:07.668 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:07 smithi063 ceph-mon[31613]: pgmap v968: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 27 op/s 2024-05-27T08:04:07.668 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:07 smithi063 ceph-mon[31613]: osdmap e143: 12 total, 12 up, 12 in 2024-05-27T08:04:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.665+0000 7f5611e00700 1 -- 172.21.15.63:0/2875876732 wait complete. 2024-05-27T08:04:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.667+0000 7f5611e00700 1 Processor -- start 2024-05-27T08:04:07.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.667+0000 7f5611e00700 1 -- start start 2024-05-27T08:04:07.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.667+0000 7f5611e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f560c1054d0 0x7f560c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.667+0000 7f5611e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c105e20 0x7f560c19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.667+0000 7f5611e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f560c10e900 0x7f560c1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.667+0000 7f5611e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f560c114320 con 0x7f560c10e900 2024-05-27T08:04:07.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.667+0000 7f5611e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1141a0 con 0x7f560c105e20 2024-05-27T08:04:07.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.667+0000 7f5611e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1144a0 con 0x7f560c1054d0 2024-05-27T08:04:07.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.668+0000 7f560b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f560c10e900 0x7f560c1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.668+0000 7f560b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f560c10e900 0x7f560c1a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36116/0 (socket says 172.21.15.63:36116) 2024-05-27T08:04:07.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.668+0000 7f560b400700 1 -- 172.21.15.63:0/782544507 learned_addr learned my addr 172.21.15.63:0/782544507 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:07.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.668+0000 7f560a000700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c105e20 0x7f560c19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.668+0000 7f560aa00700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f560c1054d0 0x7f560c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.668+0000 7f560b400700 1 -- 172.21.15.63:0/782544507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f560c1054d0 msgr2=0x7f560c19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:07.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.668+0000 7f560b400700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f560c1054d0 0x7f560c19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.668+0000 7f560b400700 1 -- 172.21.15.63:0/782544507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c105e20 msgr2=0x7f560c19d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.668+0000 7f560b400700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c105e20 0x7f560c19d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.668+0000 7f560b400700 1 -- 172.21.15.63:0/782544507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c1a1e60 con 0x7f560c10e900 2024-05-27T08:04:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.669+0000 7f560b400700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f560c10e900 0x7f560c1a17c0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f5604007ee0 tx=0x7f560400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.669+0000 7f5610a00700 1 -- 172.21.15.63:0/782544507 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560400dbd0 con 0x7f560c10e900 2024-05-27T08:04:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.669+0000 7f5611e00700 1 -- 172.21.15.63:0/782544507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c1a2150 con 0x7f560c10e900 2024-05-27T08:04:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.669+0000 7f5610a00700 1 -- 172.21.15.63:0/782544507 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f560400dd70 con 0x7f560c10e900 2024-05-27T08:04:07.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.669+0000 7f5610a00700 1 -- 172.21.15.63:0/782544507 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56040058d0 con 0x7f560c10e900 2024-05-27T08:04:07.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.670+0000 7f5611e00700 1 -- 172.21.15.63:0/782544507 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c1a9a70 con 0x7f560c10e900 2024-05-27T08:04:07.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.674+0000 7f5610a00700 1 -- 172.21.15.63:0/782544507 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f560401a020 con 0x7f560c10e900 2024-05-27T08:04:07.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.675+0000 7f5620868b80 1 Processor -- start 2024-05-27T08:04:07.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.676+0000 7f5620868b80 1 -- start start 2024-05-27T08:04:07.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.676+0000 7f5620868b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec09d29f0 0x55eec0a65330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.676+0000 7f5610a00700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55f00b2a70 0x7f55f00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.676+0000 7f5620868b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 0x55eec0a65870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.676+0000 7f5620868b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec0a65e70 0x55eec0a72110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.676+0000 7f5620868b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eec0993e40 con 0x55eec0a65e70 2024-05-27T08:04:07.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.676+0000 7f5620868b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eec09839e0 con 0x55eec09d29f0 2024-05-27T08:04:07.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.676+0000 7f5620868b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55eec0987a20 con 0x55eec08e6e40 2024-05-27T08:04:07.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.676+0000 7f5610a00700 1 -- 172.21.15.63:0/782544507 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 9571+0+0 (secure 0 0 0) 0x7f56040e2180 con 0x7f560c10e900 2024-05-27T08:04:07.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec0a65e70 0x55eec0a72110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec0a65e70 0x55eec0a72110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36128/0 (socket says 172.21.15.63:36128) 2024-05-27T08:04:07.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560b400700 1 -- 172.21.15.63:0/432921671 learned_addr learned my addr 172.21.15.63:0/432921671 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:07.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560a000700 1 --2- 172.21.15.63:0/432921671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec09d29f0 0x55eec0a65330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560a000700 1 --2- 172.21.15.63:0/432921671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 0x55eec0a65870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560aa00700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55f00b2a70 0x7f55f00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560b400700 1 -- 172.21.15.63:0/432921671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 msgr2=0x55eec0a65870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560b400700 1 --2- 172.21.15.63:0/432921671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 0x55eec0a65870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560b400700 1 -- 172.21.15.63:0/432921671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec09d29f0 msgr2=0x55eec0a65330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560b400700 1 --2- 172.21.15.63:0/432921671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec09d29f0 0x55eec0a65330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560b400700 1 -- 172.21.15.63:0/432921671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec094c420 con 0x55eec0a65e70 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.677+0000 7f560a000700 1 --2- 172.21.15.63:0/432921671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 0x55eec0a65870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.678+0000 7f560b400700 1 --2- 172.21.15.63:0/432921671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec0a65e70 0x55eec0a72110 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f56040a5060 tx=0x7f56040a4f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fc4946526eaa59 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.678+0000 7f5600c00700 1 -- 172.21.15.63:0/432921671 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56040a8fe0 con 0x55eec0a65e70 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.678+0000 7f5600c00700 1 -- 172.21.15.63:0/432921671 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56040ad390 con 0x55eec0a65e70 2024-05-27T08:04:07.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.678+0000 7f5600c00700 1 -- 172.21.15.63:0/432921671 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56040a9240 con 0x55eec0a65e70 2024-05-27T08:04:07.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.679+0000 7f5620868b80 1 -- 172.21.15.63:0/432921671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec0a65e70 msgr2=0x55eec0a72110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:07.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.679+0000 7f5620868b80 1 --2- 172.21.15.63:0/432921671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec0a65e70 0x55eec0a72110 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f56040a5060 tx=0x7f56040a4f20 comp rx=0 tx=0).stop 2024-05-27T08:04:07.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.679+0000 7f560aa00700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55f00b2a70 0x7f55f00b4f80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00b2e0 tx=0x7f55fc005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:07.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.680+0000 7f5620868b80 1 -- 172.21.15.63:0/432921671 shutdown_connections 2024-05-27T08:04:07.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.680+0000 7f5620868b80 1 --2- 172.21.15.63:0/432921671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec0a65e70 0x55eec0a72110 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.680+0000 7f5620868b80 1 --2- 172.21.15.63:0/432921671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec09d29f0 0x55eec0a65330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.680+0000 7f5620868b80 1 --2- 172.21.15.63:0/432921671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 0x55eec0a65870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.680+0000 7f5620868b80 1 -- 172.21.15.63:0/432921671 >> 172.21.15.63:0/432921671 conn(0x55eec0980390 msgr2=0x55eec0a614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:07.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.680+0000 7f5620868b80 1 -- 172.21.15.63:0/432921671 shutdown_connections 2024-05-27T08:04:07.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.680+0000 7f5620868b80 1 -- 172.21.15.63:0/432921671 wait complete. 2024-05-27T08:04:07.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.680+0000 7f5620868b80 1 -- start start 2024-05-27T08:04:07.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.681+0000 7f5620868b80 1 Processor -- start 2024-05-27T08:04:07.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.681+0000 7f5620868b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 0x55eec0a6f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.681+0000 7f5620868b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.681+0000 7f5620868b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec0a65e70 0x55eec0a701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.681+0000 7f5620868b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eec0889fa0 con 0x55eec09d29f0 2024-05-27T08:04:07.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.681+0000 7f5620868b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eec0924760 con 0x55eec0a65e70 2024-05-27T08:04:07.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.681+0000 7f5620868b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55eec0a72b80 con 0x55eec08e6e40 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f5620868b80 10 client.0 _ll_register_callbacks cb 0x55eec092dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36142/0 (socket says 172.21.15.63:36142) 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560a000700 1 -- 172.21.15.63:0/4178062265 learned_addr learned my addr 172.21.15.63:0/4178062265 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560aa00700 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 0x55eec0a6f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560b400700 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec0a65e70 0x55eec0a701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560a000700 1 -- 172.21.15.63:0/4178062265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 msgr2=0x55eec0a6f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560a000700 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 0x55eec0a6f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560a000700 1 -- 172.21.15.63:0/4178062265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec0a65e70 msgr2=0x55eec0a701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560a000700 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec0a65e70 0x55eec0a701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560a000700 1 -- 172.21.15.63:0/4178062265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec08d38d0 con 0x55eec09d29f0 2024-05-27T08:04:07.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.682+0000 7f560b400700 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec0a65e70 0x55eec0a701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:07.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.683+0000 7f560a000700 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6fc60 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f55f800faa0 tx=0x7f55f800fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75636fa850e340dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:07.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.683+0000 7f55f5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:07.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.683+0000 7f55f5600700 1 -- 172.21.15.63:0/4178062265 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8004260 con 0x55eec09d29f0 2024-05-27T08:04:07.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.683+0000 7f5620868b80 1 -- 172.21.15.63:0/4178062265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56040ad390 con 0x55eec09d29f0 2024-05-27T08:04:07.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.683+0000 7f55f5600700 1 -- 172.21.15.63:0/4178062265 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 9571+0+0 (secure 0 0 0) 0x7f55f8004400 con 0x55eec09d29f0 2024-05-27T08:04:07.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.684+0000 7f55f5600700 1 -- 172.21.15.63:0/4178062265 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f55f800e840 con 0x55eec09d29f0 2024-05-27T08:04:07.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.684+0000 7f5620868b80 10 client.21300 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:07.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.684+0000 7f5620868b80 1 -- 172.21.15.63:0/4178062265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55eec0954060 con 0x55eec09d29f0 2024-05-27T08:04:07.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.684+0000 7f55f5600700 1 -- 172.21.15.63:0/4178062265 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56040ad390 con 0x55eec09d29f0 2024-05-27T08:04:07.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.684+0000 7f55f5600700 1 -- 172.21.15.63:0/4178062265 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f55f8003b60 con 0x55eec09d29f0 2024-05-27T08:04:07.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.685+0000 7f5620868b80 10 client.21300 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:07.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.685+0000 7f5620868b80 10 client.21300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:07.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.685+0000 7f5620868b80 20 client.21300 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:07.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.685+0000 7f5620868b80 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eec0a88400 0x55eec0a8a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.685+0000 7f5620868b80 4 client.21300 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:07.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.685+0000 7f5620868b80 1 -- 172.21.15.63:0/4178062265 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eec0995d90 con 0x55eec0a88400 2024-05-27T08:04:07.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.685+0000 7f560aa00700 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eec0a88400 0x55eec0a8a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.686+0000 7f560aa00700 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eec0a88400 0x55eec0a8a860 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c1c9bb443f3c4c2 server_cookie=1ff722e0344958d6 in_seq=0 out_seq=0 2024-05-27T08:04:07.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.686+0000 7f55f5600700 10 client.21300 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:07.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.688+0000 7f55f5600700 1 -- 172.21.15.63:0/4178062265 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eec0995d90 con 0x55eec0a88400 2024-05-27T08:04:07.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.688+0000 7f55f5600700 10 client.21300 handle_command_reply: tid=0 2024-05-27T08:04:07.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 client.21300 shutdown 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 -- 172.21.15.63:0/4178062265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 msgr2=0x55eec0a6fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6fc60 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f55f800faa0 tx=0x7f55f800fe60 comp rx=0 tx=0).stop 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 -- 172.21.15.63:0/4178062265 shutdown_connections 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eec0a88400 0x55eec0a8a860 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec0a65e70 0x55eec0a701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec08e6e40 0x55eec0a6f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 --2- 172.21.15.63:0/4178062265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6fc60 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 -- 172.21.15.63:0/4178062265 >> 172.21.15.63:0/4178062265 conn(0x55eec0980390 msgr2=0x55eec0a653b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f55f5600700 0 client.21300 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 -- 172.21.15.63:0/4178062265 shutdown_connections 2024-05-27T08:04:07.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.689+0000 7f5620868b80 1 -- 172.21.15.63:0/4178062265 wait complete. 2024-05-27T08:04:07.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.690+0000 7f5620868b80 20 client.21300 trim_cache size 0 max 16384 2024-05-27T08:04:07.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.714+0000 7f5620868b80 1 Processor -- start 2024-05-27T08:04:07.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.714+0000 7f5620868b80 1 -- start start 2024-05-27T08:04:07.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.715+0000 7f5620868b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec08e6e40 0x55eec0a8aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.715+0000 7f5620868b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.715+0000 7f5620868b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec0a65e70 0x55eec0a65830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.715+0000 7f5620868b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1b0da0 con 0x55eec09d29f0 2024-05-27T08:04:07.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.715+0000 7f5620868b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1ae630 con 0x55eec08e6e40 2024-05-27T08:04:07.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.715+0000 7f5620868b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1ac520 con 0x55eec0a65e70 2024-05-27T08:04:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.716+0000 7f560a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.716+0000 7f560a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36146/0 (socket says 172.21.15.63:36146) 2024-05-27T08:04:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.716+0000 7f560a000700 1 -- 172.21.15.63:0/1303518370 learned_addr learned my addr 172.21.15.63:0/1303518370 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.717+0000 7f560aa00700 1 --2- 172.21.15.63:0/1303518370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec08e6e40 0x55eec0a8aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.717+0000 7f560a000700 1 -- 172.21.15.63:0/1303518370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec0a65e70 msgr2=0x55eec0a65830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.717+0000 7f560a000700 1 --2- 172.21.15.63:0/1303518370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec0a65e70 0x55eec0a65830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.717+0000 7f560a000700 1 -- 172.21.15.63:0/1303518370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec08e6e40 msgr2=0x55eec0a8aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:07.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.717+0000 7f560a000700 1 --2- 172.21.15.63:0/1303518370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec08e6e40 0x55eec0a8aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.717+0000 7f560a000700 1 -- 172.21.15.63:0/1303518370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec0a62000 con 0x55eec09d29f0 2024-05-27T08:04:07.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.718+0000 7f560aa00700 1 --2- 172.21.15.63:0/1303518370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec08e6e40 0x55eec0a8aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:07.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.718+0000 7f560a000700 1 --2- 172.21.15.63:0/1303518370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6aef0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f55f8010570 tx=0x7f55f8009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3daaba82343cd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:07.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.718+0000 7f55f7e00700 1 -- 172.21.15.63:0/1303518370 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8031070 con 0x55eec09d29f0 2024-05-27T08:04:07.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.718+0000 7f55f7e00700 1 -- 172.21.15.63:0/1303518370 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55f800e9e0 con 0x55eec09d29f0 2024-05-27T08:04:07.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.718+0000 7f55f7e00700 1 -- 172.21.15.63:0/1303518370 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8003ae0 con 0x55eec09d29f0 2024-05-27T08:04:07.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.720+0000 7f5620868b80 1 -- 172.21.15.63:0/1303518370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 msgr2=0x55eec0a6aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:07.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.720+0000 7f5620868b80 1 --2- 172.21.15.63:0/1303518370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6aef0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f55f8010570 tx=0x7f55f8009300 comp rx=0 tx=0).stop 2024-05-27T08:04:07.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.720+0000 7f5620868b80 1 -- 172.21.15.63:0/1303518370 shutdown_connections 2024-05-27T08:04:07.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.720+0000 7f5620868b80 1 --2- 172.21.15.63:0/1303518370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec0a65e70 0x55eec0a65830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.720+0000 7f5620868b80 1 --2- 172.21.15.63:0/1303518370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec08e6e40 0x55eec0a8aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.720+0000 7f5620868b80 1 --2- 172.21.15.63:0/1303518370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec09d29f0 0x55eec0a6aef0 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.720+0000 7f5620868b80 1 -- 172.21.15.63:0/1303518370 >> 172.21.15.63:0/1303518370 conn(0x55eec0980390 msgr2=0x55eec0a65120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:07.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.720+0000 7f5620868b80 1 -- 172.21.15.63:0/1303518370 shutdown_connections 2024-05-27T08:04:07.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.721+0000 7f5620868b80 1 -- 172.21.15.63:0/1303518370 wait complete. 2024-05-27T08:04:07.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.721+0000 7f5620868b80 1 -- start start 2024-05-27T08:04:07.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.722+0000 7f5620868b80 1 Processor -- start 2024-05-27T08:04:07.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.722+0000 7f5620868b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec08e6e40 0x55eec0a7bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.722+0000 7f5620868b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec09d29f0 0x55eec0a7c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.723+0000 7f5620868b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec0a65e70 0x55eec0af0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.723+0000 7f5620868b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1acb70 con 0x55eec08e6e40 2024-05-27T08:04:07.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.723+0000 7f5620868b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1adf60 con 0x55eec09d29f0 2024-05-27T08:04:07.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.723+0000 7f5620868b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1ad1c0 con 0x55eec0a65e70 2024-05-27T08:04:07.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.723+0000 7f560aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec08e6e40 0x55eec0a7bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.723+0000 7f5620868b80 10 client.0 _ll_register_callbacks cb 0x55eec092dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:07.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.723+0000 7f560aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec08e6e40 0x55eec0a7bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36150/0 (socket says 172.21.15.63:36150) 2024-05-27T08:04:07.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.723+0000 7f560aa00700 1 -- 172.21.15.63:0/1793963530 learned_addr learned my addr 172.21.15.63:0/1793963530 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:07.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.723+0000 7f560a000700 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec09d29f0 0x55eec0a7c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.724+0000 7f560b400700 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec0a65e70 0x55eec0af0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.724+0000 7f560aa00700 1 -- 172.21.15.63:0/1793963530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec0a65e70 msgr2=0x55eec0af0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.724+0000 7f560aa00700 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec0a65e70 0x55eec0af0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.724+0000 7f560aa00700 1 -- 172.21.15.63:0/1793963530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec09d29f0 msgr2=0x55eec0a7c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.724+0000 7f560aa00700 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec09d29f0 0x55eec0a7c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.724+0000 7f560aa00700 1 -- 172.21.15.63:0/1793963530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec0af1460 con 0x55eec08e6e40 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.724+0000 7f560b400700 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec0a65e70 0x55eec0af0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.724+0000 7f560aa00700 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec08e6e40 0x55eec0a7bc60 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009990 tx=0x7f55fc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3654065cb3be6c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.724+0000 7f55f5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.725+0000 7f55f5600700 1 -- 172.21.15.63:0/1793963530 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00bea0 con 0x55eec08e6e40 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.725+0000 7f55f5600700 1 -- 172.21.15.63:0/1793963530 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 9571+0+0 (secure 0 0 0) 0x7f55fc003680 con 0x55eec08e6e40 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.725+0000 7f5620868b80 1 -- 172.21.15.63:0/1793963530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f800e9e0 con 0x55eec08e6e40 2024-05-27T08:04:07.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.725+0000 7f55f5600700 1 -- 172.21.15.63:0/1793963530 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f55fc0038c0 con 0x55eec08e6e40 2024-05-27T08:04:07.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.725+0000 7f5620868b80 10 client.21306 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:07.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.725+0000 7f5620868b80 1 -- 172.21.15.63:0/1793963530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55eec0af25f0 con 0x55eec08e6e40 2024-05-27T08:04:07.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.726+0000 7f55f5600700 1 -- 172.21.15.63:0/1793963530 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55f800e9e0 con 0x55eec08e6e40 2024-05-27T08:04:07.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.726+0000 7f55f5600700 1 -- 172.21.15.63:0/1793963530 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f55fc022040 con 0x55eec08e6e40 2024-05-27T08:04:07.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.726+0000 7f5620868b80 10 client.21306 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:07.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.726+0000 7f5620868b80 10 client.21306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:07.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.726+0000 7f5620868b80 20 client.21306 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:07.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.726+0000 7f5620868b80 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eec0af2d60 0x55eec0af5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:07.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.726+0000 7f5620868b80 4 client.21306 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:04:07.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.726+0000 7f5620868b80 1 -- 172.21.15.63:0/1793963530 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eec098f570 con 0x55eec0af2d60 2024-05-27T08:04:07.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.727+0000 7f560a000700 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eec0af2d60 0x55eec0af5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:07.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.728+0000 7f560a000700 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eec0af2d60 0x55eec0af5180 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe82cef981bd57d2 server_cookie=4ae9f55d603118f8 in_seq=0 out_seq=0 2024-05-27T08:04:07.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.728+0000 7f55f5600700 10 client.21306 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:07.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.729+0000 7f55f5600700 1 -- 172.21.15.63:0/1793963530 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eec098f570 con 0x55eec0af2d60 2024-05-27T08:04:07.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.729+0000 7f55f5600700 10 client.21306 handle_command_reply: tid=0 2024-05-27T08:04:07.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.729+0000 7f5620868b80 1 client.21306 shutdown 2024-05-27T08:04:07.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.729+0000 7f5620868b80 1 -- 172.21.15.63:0/1793963530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec08e6e40 msgr2=0x55eec0a7bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:07.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.729+0000 7f5620868b80 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec08e6e40 0x55eec0a7bc60 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009990 tx=0x7f55fc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.729+0000 7f5620868b80 1 -- 172.21.15.63:0/1793963530 shutdown_connections 2024-05-27T08:04:07.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.729+0000 7f5620868b80 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55eec0af2d60 0x55eec0af5180 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.729+0000 7f5620868b80 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eec0a65e70 0x55eec0af0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.730+0000 7f5620868b80 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eec08e6e40 0x55eec0a7bc60 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.730+0000 7f55f5600700 0 client.21306 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:07.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.730+0000 7f5620868b80 1 --2- 172.21.15.63:0/1793963530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eec09d29f0 0x55eec0a7c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.730+0000 7f5620868b80 1 -- 172.21.15.63:0/1793963530 >> 172.21.15.63:0/1793963530 conn(0x55eec0980390 msgr2=0x55eec0a6f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:07.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.730+0000 7f5620868b80 1 -- 172.21.15.63:0/1793963530 shutdown_connections 2024-05-27T08:04:07.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.730+0000 7f5620868b80 1 -- 172.21.15.63:0/1793963530 wait complete. 2024-05-27T08:04:07.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.730+0000 7f5620868b80 20 client.21306 trim_cache size 0 max 16384 2024-05-27T08:04:07.733 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:04:07.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.731+0000 7f5611e00700 1 -- 172.21.15.63:0/782544507 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55f00b2a70 msgr2=0x7f55f00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:07.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.731+0000 7f5611e00700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55f00b2a70 0x7f55f00b4f80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00b2e0 tx=0x7f55fc005fb0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.731+0000 7f5611e00700 1 -- 172.21.15.63:0/782544507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f560c10e900 msgr2=0x7f560c1a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:07.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.731+0000 7f5611e00700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f560c10e900 0x7f560c1a17c0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f5604007ee0 tx=0x7f560400d3b0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.731+0000 7f5611e00700 1 -- 172.21.15.63:0/782544507 shutdown_connections 2024-05-27T08:04:07.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.731+0000 7f5611e00700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f55f00b2a70 0x7f55f00b4f80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.731+0000 7f5611e00700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f560c10e900 0x7f560c1a17c0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.731+0000 7f5611e00700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f560c1054d0 0x7f560c19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.731+0000 7f5611e00700 1 --2- 172.21.15.63:0/782544507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f560c105e20 0x7f560c19d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.731+0000 7f5611e00700 1 -- 172.21.15.63:0/782544507 >> 172.21.15.63:0/782544507 conn(0x7f560c0fedb0 msgr2=0x7f560c103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.732+0000 7f5611e00700 1 -- 172.21.15.63:0/782544507 shutdown_connections 2024-05-27T08:04:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:07.732+0000 7f5611e00700 1 -- 172.21.15.63:0/782544507 wait complete. 2024-05-27T08:04:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:07 smithi086 ceph-mon[34687]: pgmap v968: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 27 op/s 2024-05-27T08:04:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:07 smithi086 ceph-mon[34687]: osdmap e143: 12 total, 12 up, 12 in 2024-05-27T08:04:07.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:07 smithi149 ceph-mon[35408]: pgmap v968: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 27 op/s 2024-05-27T08:04:07.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:07 smithi149 ceph-mon[35408]: osdmap e143: 12 total, 12 up, 12 in 2024-05-27T08:04:09.752 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:04:09.752 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:04:09.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:09 smithi086 ceph-mon[34687]: pgmap v970: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T08:04:09.851 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:09 smithi063 ceph-mon[31613]: pgmap v970: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T08:04:09.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:09 smithi149 ceph-mon[35408]: pgmap v970: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T08:04:10.283 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:11 smithi063 ceph-mon[31613]: pgmap v971: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T08:04:11.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:11 smithi086 ceph-mon[34687]: pgmap v971: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T08:04:11.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:11 smithi149 ceph-mon[35408]: pgmap v971: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.9 MiB/s wr, 33 op/s 2024-05-27T08:04:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.461+0000 7f6c08400700 1 -- 172.21.15.63:0/2143902983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c001034b0 msgr2=0x7f6c001038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.461+0000 7f6c08400700 1 --2- 172.21.15.63:0/2143902983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c001034b0 0x7f6c001038c0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400b0d0 tx=0x7f6bf400b3e0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.461+0000 7f6c08400700 1 -- 172.21.15.63:0/2143902983 shutdown_connections 2024-05-27T08:04:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.461+0000 7f6c08400700 1 --2- 172.21.15.63:0/2143902983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c00108730 0x7f6c00110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.461+0000 7f6c08400700 1 --2- 172.21.15.63:0/2143902983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c001034b0 0x7f6c001038c0 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.461+0000 7f6c08400700 1 --2- 172.21.15.63:0/2143902983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c00107de0 0x7f6c001081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.461+0000 7f6c08400700 1 -- 172.21.15.63:0/2143902983 >> 172.21.15.63:0/2143902983 conn(0x7f6c000fedb0 msgr2=0x7f6c001011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.461+0000 7f6c08400700 1 -- 172.21.15.63:0/2143902983 shutdown_connections 2024-05-27T08:04:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.461+0000 7f6c08400700 1 -- 172.21.15.63:0/2143902983 wait complete. 2024-05-27T08:04:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.463+0000 7f6c08400700 1 Processor -- start 2024-05-27T08:04:13.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.463+0000 7f6c08400700 1 -- start start 2024-05-27T08:04:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.464+0000 7f6c08400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c001034b0 0x7f6c0019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.464+0000 7f6c08400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c00107de0 0x7f6c0019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.464+0000 7f6c08400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c00108730 0x7f6c001a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.464+0000 7f6c08400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c00113260 con 0x7f6c00107de0 2024-05-27T08:04:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.464+0000 7f6c08400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001130e0 con 0x7f6c00108730 2024-05-27T08:04:13.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.464+0000 7f6c08400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001133e0 con 0x7f6c001034b0 2024-05-27T08:04:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.464+0000 7f6c05a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c001034b0 0x7f6c0019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.464+0000 7f6c05000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c00107de0 0x7f6c0019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.464+0000 7f6c05a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c001034b0 0x7f6c0019cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:42524/0 (socket says 172.21.15.63:42524) 2024-05-27T08:04:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.464+0000 7f6c05a00700 1 -- 172.21.15.63:0/1162025384 learned_addr learned my addr 172.21.15.63:0/1162025384 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:13.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.465+0000 7f6c06400700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c00108730 0x7f6c001a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.465+0000 7f6c05000700 1 -- 172.21.15.63:0/1162025384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c001034b0 msgr2=0x7f6c0019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.465+0000 7f6c05000700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c001034b0 0x7f6c0019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.465+0000 7f6c05000700 1 -- 172.21.15.63:0/1162025384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c00108730 msgr2=0x7f6c001a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.465+0000 7f6c05000700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c00108730 0x7f6c001a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.465+0000 7f6c05000700 1 -- 172.21.15.63:0/1162025384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c001a1ea0 con 0x7f6c00107de0 2024-05-27T08:04:13.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.465+0000 7f6c05a00700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c001034b0 0x7f6c0019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:13.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.466+0000 7f6c05000700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c00107de0 0x7f6c0019d570 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f6c00076aa0 tx=0x7f6bec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.466+0000 7f6bfe000700 1 -- 172.21.15.63:0/1162025384 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec016070 con 0x7f6c00107de0 2024-05-27T08:04:13.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.466+0000 7f6c08400700 1 -- 172.21.15.63:0/1162025384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c001a2190 con 0x7f6c00107de0 2024-05-27T08:04:13.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.466+0000 7f6bfe000700 1 -- 172.21.15.63:0/1162025384 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bec00fd20 con 0x7f6c00107de0 2024-05-27T08:04:13.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.466+0000 7f6bfe000700 1 -- 172.21.15.63:0/1162025384 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec00e3b0 con 0x7f6c00107de0 2024-05-27T08:04:13.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.466+0000 7f6c08400700 1 -- 172.21.15.63:0/1162025384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c001a9ab0 con 0x7f6c00107de0 2024-05-27T08:04:13.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.472+0000 7f6c16e48b80 1 Processor -- start 2024-05-27T08:04:13.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.472+0000 7f6bfe000700 1 -- 172.21.15.63:0/1162025384 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6bec019070 con 0x7f6c00107de0 2024-05-27T08:04:13.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.472+0000 7f6c16e48b80 1 -- start start 2024-05-27T08:04:13.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.473+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec2724ec0 0x563ec28a9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.473+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a9db0 0x563ec28a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.473+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a3e30 0x563ec28a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.473+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563ec27d1ec0 con 0x563ec2724ec0 2024-05-27T08:04:13.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.473+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563ec27c1a60 con 0x563ec28a9db0 2024-05-27T08:04:13.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.473+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ec27c5aa0 con 0x563ec28a3e30 2024-05-27T08:04:13.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.473+0000 7f6c06400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec2724ec0 0x563ec28a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.473+0000 7f6c06400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec2724ec0 0x563ec28a9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46620/0 (socket says 172.21.15.63:46620) 2024-05-27T08:04:13.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.473+0000 7f6c06400700 1 -- 172.21.15.63:0/1763128393 learned_addr learned my addr 172.21.15.63:0/1763128393 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:13.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.473+0000 7f6c05000700 1 --2- 172.21.15.63:0/1763128393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a3e30 0x563ec28a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.474+0000 7f6c05a00700 1 --2- 172.21.15.63:0/1763128393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a9db0 0x563ec28a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.474+0000 7f6c05a00700 1 -- 172.21.15.63:0/1763128393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a3e30 msgr2=0x563ec28a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.474+0000 7f6c05a00700 1 --2- 172.21.15.63:0/1763128393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a3e30 0x563ec28a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.474+0000 7f6c05a00700 1 -- 172.21.15.63:0/1763128393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec2724ec0 msgr2=0x563ec28a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.474+0000 7f6c05a00700 1 --2- 172.21.15.63:0/1763128393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec2724ec0 0x563ec28a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.474+0000 7f6c05a00700 1 -- 172.21.15.63:0/1763128393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec278a4a0 con 0x563ec28a9db0 2024-05-27T08:04:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.474+0000 7f6bfe000700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6be80b2a70 0x7f6be80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.475+0000 7f6c06400700 1 --2- 172.21.15.63:0/1763128393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec2724ec0 0x563ec28a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.475+0000 7f6bfe000700 1 -- 172.21.15.63:0/1162025384 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 9571+0+0 (secure 0 0 0) 0x7f6bec0e2740 con 0x7f6c00107de0 2024-05-27T08:04:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.475+0000 7f6c05a00700 1 --2- 172.21.15.63:0/1763128393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a9db0 0x563ec28a38f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009610 tx=0x7f6bf40098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89c611a0ae4892c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.476+0000 7f6bf2a00700 1 -- 172.21.15.63:0/1763128393 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4003c00 con 0x563ec28a9db0 2024-05-27T08:04:13.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.476+0000 7f6bf2a00700 1 -- 172.21.15.63:0/1763128393 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bf4003da0 con 0x563ec28a9db0 2024-05-27T08:04:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.476+0000 7f6c06400700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6be80b2a70 0x7f6be80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.476+0000 7f6bf2a00700 1 -- 172.21.15.63:0/1763128393 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4005490 con 0x563ec28a9db0 2024-05-27T08:04:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.477+0000 7f6c16e48b80 1 -- 172.21.15.63:0/1763128393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a9db0 msgr2=0x563ec28a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.477+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/1763128393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a9db0 0x563ec28a38f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009610 tx=0x7f6bf40098c0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.477+0000 7f6c06400700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6be80b2a70 0x7f6be80b4f80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80046e0 tx=0x7f6bf800b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.477+0000 7f6c16e48b80 1 -- 172.21.15.63:0/1763128393 shutdown_connections 2024-05-27T08:04:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.477+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/1763128393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a3e30 0x563ec28a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.477+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/1763128393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec2724ec0 0x563ec28a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.477+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/1763128393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a9db0 0x563ec28a38f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.477+0000 7f6c16e48b80 1 -- 172.21.15.63:0/1763128393 >> 172.21.15.63:0/1763128393 conn(0x563ec2785040 msgr2=0x563ec289e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.477+0000 7f6c16e48b80 1 -- 172.21.15.63:0/1763128393 shutdown_connections 2024-05-27T08:04:13.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.478+0000 7f6c16e48b80 1 -- 172.21.15.63:0/1763128393 wait complete. 2024-05-27T08:04:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.478+0000 7f6c16e48b80 1 -- start start 2024-05-27T08:04:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.479+0000 7f6c16e48b80 1 Processor -- start 2024-05-27T08:04:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.479+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec2724ec0 0x563ec28ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.479+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a3e30 0x563ec28ada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.479+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a9db0 0x563ec28adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.479+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563ec26c8020 con 0x563ec28a9db0 2024-05-27T08:04:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.479+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563ec27627e0 con 0x563ec28a3e30 2024-05-27T08:04:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.479+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563ec28b0820 con 0x563ec2724ec0 2024-05-27T08:04:13.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.479+0000 7f6c16e48b80 10 client.0 _ll_register_callbacks cb 0x563ec276bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.480+0000 7f6c06400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a9db0 0x563ec28adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.480+0000 7f6c06400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a9db0 0x563ec28adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46636/0 (socket says 172.21.15.63:46636) 2024-05-27T08:04:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.480+0000 7f6c06400700 1 -- 172.21.15.63:0/885185761 learned_addr learned my addr 172.21.15.63:0/885185761 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.480+0000 7f6c05a00700 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec2724ec0 0x563ec28ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.480+0000 7f6c05000700 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a3e30 0x563ec28ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.481+0000 7f6c06400700 1 -- 172.21.15.63:0/885185761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec2724ec0 msgr2=0x563ec28ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.481+0000 7f6c06400700 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec2724ec0 0x563ec28ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.481+0000 7f6c06400700 1 -- 172.21.15.63:0/885185761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a3e30 msgr2=0x563ec28ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.481+0000 7f6c06400700 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a3e30 0x563ec28ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.481+0000 7f6c06400700 1 -- 172.21.15.63:0/885185761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec2711950 con 0x563ec28a9db0 2024-05-27T08:04:13.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.481+0000 7f6c06400700 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a9db0 0x563ec28adf90 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8003ac0 tx=0x7f6bf80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696dc51aa5993d8c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.481+0000 7f6be7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.482+0000 7f6be7400700 1 -- 172.21.15.63:0/885185761 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8004830 con 0x563ec28a9db0 2024-05-27T08:04:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.482+0000 7f6c16e48b80 1 -- 172.21.15.63:0/885185761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf4003da0 con 0x563ec28a9db0 2024-05-27T08:04:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.482+0000 7f6be7400700 1 -- 172.21.15.63:0/885185761 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 9571+0+0 (secure 0 0 0) 0x7f6bf8017030 con 0x563ec28a9db0 2024-05-27T08:04:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.482+0000 7f6be7400700 1 -- 172.21.15.63:0/885185761 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6bf8032050 con 0x563ec28a9db0 2024-05-27T08:04:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.482+0000 7f6c16e48b80 10 client.21318 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.482+0000 7f6c16e48b80 1 -- 172.21.15.63:0/885185761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563ec27920e0 con 0x563ec28a9db0 2024-05-27T08:04:13.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.483+0000 7f6be7400700 1 -- 172.21.15.63:0/885185761 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bf4003da0 con 0x563ec28a9db0 2024-05-27T08:04:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.483+0000 7f6be7400700 1 -- 172.21.15.63:0/885185761 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6bf8005020 con 0x563ec28a9db0 2024-05-27T08:04:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.483+0000 7f6c16e48b80 10 client.21318 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.483+0000 7f6c16e48b80 10 client.21318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.483+0000 7f6c16e48b80 20 client.21318 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.483+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ec28bd6f0 0x563ec28bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.483+0000 7f6c16e48b80 4 client.21318 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:13.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.483+0000 7f6c16e48b80 1 -- 172.21.15.63:0/885185761 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ec27d3e10 con 0x563ec28bd6f0 2024-05-27T08:04:13.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.484+0000 7f6c05a00700 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ec28bd6f0 0x563ec28bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.486+0000 7f6c05a00700 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ec28bd6f0 0x563ec28bfb70 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c624c11015aa5d server_cookie=e10c9f9a8355eb24 in_seq=0 out_seq=0 2024-05-27T08:04:13.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.486+0000 7f6be7400700 10 client.21318 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.487+0000 7f6be7400700 1 -- 172.21.15.63:0/885185761 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563ec27d3e10 con 0x563ec28bd6f0 2024-05-27T08:04:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.487+0000 7f6be7400700 10 client.21318 handle_command_reply: tid=0 2024-05-27T08:04:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.488+0000 7f6c16e48b80 1 client.21318 shutdown 2024-05-27T08:04:13.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.488+0000 7f6c16e48b80 1 -- 172.21.15.63:0/885185761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a9db0 msgr2=0x563ec28adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.488+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a9db0 0x563ec28adf90 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8003ac0 tx=0x7f6bf80047c0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.488+0000 7f6c16e48b80 1 -- 172.21.15.63:0/885185761 shutdown_connections 2024-05-27T08:04:13.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.488+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ec28bd6f0 0x563ec28bfb70 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.488+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a9db0 0x563ec28adf90 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.488+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec2724ec0 0x563ec28ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.488+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/885185761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a3e30 0x563ec28ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.488+0000 7f6c16e48b80 1 -- 172.21.15.63:0/885185761 >> 172.21.15.63:0/885185761 conn(0x563ec2785040 msgr2=0x563ec289e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.489+0000 7f6be7400700 0 client.21318 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:13.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.489+0000 7f6c16e48b80 1 -- 172.21.15.63:0/885185761 shutdown_connections 2024-05-27T08:04:13.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.489+0000 7f6c16e48b80 1 -- 172.21.15.63:0/885185761 wait complete. 2024-05-27T08:04:13.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.489+0000 7f6c16e48b80 20 client.21318 trim_cache size 0 max 16384 2024-05-27T08:04:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.498+0000 7f1c19200640 1 -- 172.21.15.63:0/1378529886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1410da00 msgr2=0x7f1c1410fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.498+0000 7f1c19200640 1 --2- 172.21.15.63:0/1378529886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1410da00 0x7f1c1410fe20 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009900 tx=0x7f1c0002a3f0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.498+0000 7f1c19200640 1 -- 172.21.15.63:0/1378529886 shutdown_connections 2024-05-27T08:04:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.498+0000 7f1c19200640 1 --2- 172.21.15.63:0/1378529886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c14110360 0x7f1c14112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.498+0000 7f1c19200640 1 --2- 172.21.15.63:0/1378529886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c1410da00 0x7f1c1410fe20 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.498+0000 7f1c19200640 1 --2- 172.21.15.63:0/1378529886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c141027c0 0x7f1c14102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.498+0000 7f1c19200640 1 -- 172.21.15.63:0/1378529886 >> 172.21.15.63:0/1378529886 conn(0x7f1c140fe210 msgr2=0x7f1c14100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.498+0000 7f1c19200640 1 -- 172.21.15.63:0/1378529886 shutdown_connections 2024-05-27T08:04:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.498+0000 7f1c19200640 1 -- 172.21.15.63:0/1378529886 wait complete. 2024-05-27T08:04:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.500+0000 7f1c19200640 1 Processor -- start 2024-05-27T08:04:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.500+0000 7f1c19200640 1 -- start start 2024-05-27T08:04:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.500+0000 7f1c19200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c141027c0 0x7f1c1419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.500+0000 7f1c19200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c1410da00 0x7f1c1419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.501+0000 7f1c19200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c14110360 0x7f1c141a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.501+0000 7f1c19200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14115d20 con 0x7f1c141027c0 2024-05-27T08:04:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.501+0000 7f1c19200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14115ba0 con 0x7f1c1410da00 2024-05-27T08:04:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.501+0000 7f1c19200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14115ea0 con 0x7f1c14110360 2024-05-27T08:04:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.501+0000 7f1c13400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c141027c0 0x7f1c1419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.501+0000 7f1c13400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c141027c0 0x7f1c1419c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46642/0 (socket says 172.21.15.63:46642) 2024-05-27T08:04:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.501+0000 7f1c13400640 1 -- 172.21.15.63:0/715069283 learned_addr learned my addr 172.21.15.63:0/715069283 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.501+0000 7f1c12a00640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c1410da00 0x7f1c1419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.501+0000 7f1c13e00640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c14110360 0x7f1c141a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.502+0000 7f1c13400640 1 -- 172.21.15.63:0/715069283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c14110360 msgr2=0x7f1c141a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.502+0000 7f1c13400640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c14110360 0x7f1c141a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.502+0000 7f1c13400640 1 -- 172.21.15.63:0/715069283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c1410da00 msgr2=0x7f1c1419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.502+0000 7f1c13400640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c1410da00 0x7f1c1419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.502+0000 7f1c13400640 1 -- 172.21.15.63:0/715069283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c141a15e0 con 0x7f1c141027c0 2024-05-27T08:04:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.502+0000 7f1c13400640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c141027c0 0x7f1c1419c780 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400d950 tx=0x7f1c0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.502+0000 7f1c12a00640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c1410da00 0x7f1c1419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.502+0000 7f1c13e00640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c14110360 0x7f1c141a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.502+0000 7f1c0be00640 1 -- 172.21.15.63:0/715069283 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04017030 con 0x7f1c141027c0 2024-05-27T08:04:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.503+0000 7f1c0be00640 1 -- 172.21.15.63:0/715069283 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c04004020 con 0x7f1c141027c0 2024-05-27T08:04:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.503+0000 7f1c19200640 1 -- 172.21.15.63:0/715069283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c141a18d0 con 0x7f1c141027c0 2024-05-27T08:04:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.503+0000 7f1c0be00640 1 -- 172.21.15.63:0/715069283 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04003b80 con 0x7f1c141027c0 2024-05-27T08:04:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.503+0000 7f1c19200640 1 -- 172.21.15.63:0/715069283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c141a9250 con 0x7f1c141027c0 2024-05-27T08:04:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.507+0000 7f1c217c6740 1 Processor -- start 2024-05-27T08:04:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.507+0000 7f1c217c6740 1 -- start start 2024-05-27T08:04:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.508+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579197471b0 0x557919ae9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.508+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55791971e2f0 0x557919ae9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.508+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197264d0 0x557919aea4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.508+0000 7f1c217c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557919a5ab90 con 0x5579197264d0 2024-05-27T08:04:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.508+0000 7f1c217c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557919a4a730 con 0x55791971e2f0 2024-05-27T08:04:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.508+0000 7f1c217c6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557919a4e770 con 0x5579197471b0 2024-05-27T08:04:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.509+0000 7f1c12a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197264d0 0x557919aea4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.509+0000 7f1c13e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579197471b0 0x557919ae9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.509+0000 7f1c12a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197264d0 0x557919aea4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46658/0 (socket says 172.21.15.63:46658) 2024-05-27T08:04:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.509+0000 7f1c12a00640 1 -- 172.21.15.63:0/3939037319 learned_addr learned my addr 172.21.15.63:0/3939037319 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.509+0000 7f1c0be00640 1 -- 172.21.15.63:0/715069283 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1c0401a070 con 0x7f1c141027c0 2024-05-27T08:04:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.509+0000 7f1c12a00640 1 -- 172.21.15.63:0/3939037319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579197471b0 msgr2=0x557919ae9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.509+0000 7f1c12a00640 1 --2- 172.21.15.63:0/3939037319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579197471b0 0x557919ae9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.509+0000 7f1c12a00640 1 -- 172.21.15.63:0/3939037319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55791971e2f0 msgr2=0x557919ae9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.509+0000 7f1c12a00640 1 --2- 172.21.15.63:0/3939037319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55791971e2f0 0x557919ae9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.509+0000 7f1c12a00640 1 -- 172.21.15.63:0/3939037319 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579199e38b0 con 0x5579197264d0 2024-05-27T08:04:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.510+0000 7f1c12a00640 1 --2- 172.21.15.63:0/3939037319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197264d0 0x557919aea4e0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009900 tx=0x7f1c00032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24d78412ef1d077 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.510+0000 7f1c08c00640 1 -- 172.21.15.63:0/3939037319 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0003f070 con 0x5579197264d0 2024-05-27T08:04:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.511+0000 7f1c08c00640 1 -- 172.21.15.63:0/3939037319 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c00005de0 con 0x5579197264d0 2024-05-27T08:04:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.511+0000 7f1c08c00640 1 -- 172.21.15.63:0/3939037319 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00035070 con 0x5579197264d0 2024-05-27T08:04:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.511+0000 7f1c0be00640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1bf80b29d0 0x7f1bf80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.512+0000 7f1c217c6740 1 -- 172.21.15.63:0/3939037319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197264d0 msgr2=0x557919aea4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.512+0000 7f1c217c6740 1 --2- 172.21.15.63:0/3939037319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197264d0 0x557919aea4e0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009900 tx=0x7f1c00032dd0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.512+0000 7f1c0be00640 1 -- 172.21.15.63:0/715069283 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 9571+0+0 (secure 0 0 0) 0x7f1c040e25a0 con 0x7f1c141027c0 2024-05-27T08:04:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.513+0000 7f1c13e00640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1bf80b29d0 0x7f1bf80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.513+0000 7f1c217c6740 1 -- 172.21.15.63:0/3939037319 shutdown_connections 2024-05-27T08:04:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.513+0000 7f1c217c6740 1 --2- 172.21.15.63:0/3939037319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197264d0 0x557919aea4e0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.513+0000 7f1c217c6740 1 --2- 172.21.15.63:0/3939037319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55791971e2f0 0x557919ae9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.513+0000 7f1c217c6740 1 --2- 172.21.15.63:0/3939037319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579197471b0 0x557919ae9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.513+0000 7f1c217c6740 1 -- 172.21.15.63:0/3939037319 >> 172.21.15.63:0/3939037319 conn(0x55791973b300 msgr2=0x557919ae6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.515+0000 7f1c13e00640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1bf80b29d0 0x7f1bf80b4ee0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x557919ae6bd0 tx=0x7f1c0c009680 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.517+0000 7f1c217c6740 1 -- 172.21.15.63:0/3939037319 shutdown_connections 2024-05-27T08:04:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.517+0000 7f1c217c6740 1 -- 172.21.15.63:0/3939037319 wait complete. 2024-05-27T08:04:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.518+0000 7f1c217c6740 1 -- start start 2024-05-27T08:04:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.519+0000 7f1c217c6740 1 Processor -- start 2024-05-27T08:04:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.520+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 0x557919af3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.520+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579197264d0 0x557919af41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.520+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919affe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.520+0000 7f1c217c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557919730790 con 0x5579197471b0 2024-05-27T08:04:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.520+0000 7f1c217c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579199f2bc0 con 0x5579197264d0 2024-05-27T08:04:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.520+0000 7f1c217c6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557919af6b30 con 0x55791971e2f0 2024-05-27T08:04:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.520+0000 7f1c217c6740 10 client.0 _ll_register_callbacks cb 0x5579199d0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.520+0000 7f1c13400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 0x557919af3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.521+0000 7f1c13400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 0x557919af3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:42602/0 (socket says 172.21.15.63:42602) 2024-05-27T08:04:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.521+0000 7f1c13400640 1 -- 172.21.15.63:0/487854697 learned_addr learned my addr 172.21.15.63:0/487854697 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.521+0000 7f1c12a00640 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579197264d0 0x557919af41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.521+0000 7f1c13e00640 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919affe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.521+0000 7f1c13400640 1 -- 172.21.15.63:0/487854697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579197264d0 msgr2=0x557919af41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.521+0000 7f1c13400640 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579197264d0 0x557919af41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.521+0000 7f1c13400640 1 -- 172.21.15.63:0/487854697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 msgr2=0x557919affe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.521+0000 7f1c13400640 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919affe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.521+0000 7f1c13400640 1 -- 172.21.15.63:0/487854697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55791975f210 con 0x55791971e2f0 2024-05-27T08:04:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.522+0000 7f1c13e00640 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919affe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.522+0000 7f1c13400640 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 0x557919af3c60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1c040dfed0 tx=0x7f1c040a9b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ef6b8ea7404ef9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.522+0000 7f1bfd600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:04:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.524+0000 7f1bfd600640 1 -- 172.21.15.63:0/487854697 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c040ad810 con 0x55791971e2f0 2024-05-27T08:04:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.524+0000 7f1bfd600640 1 -- 172.21.15.63:0/487854697 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 9571+0+0 (secure 0 0 0) 0x7f1c040dfb50 con 0x55791971e2f0 2024-05-27T08:04:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.524+0000 7f1c217c6740 1 -- 172.21.15.63:0/487854697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557919a5cae0 con 0x55791971e2f0 2024-05-27T08:04:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.525+0000 7f1bfd600640 1 -- 172.21.15.63:0/487854697 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1c040b22c0 con 0x55791971e2f0 2024-05-27T08:04:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.525+0000 7f1c217c6740 10 client.30244 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.525+0000 7f1c217c6740 1 -- 172.21.15.63:0/487854697 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55791982b620 con 0x55791971e2f0 2024-05-27T08:04:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.525+0000 7f1bfd600640 1 -- 172.21.15.63:0/487854697 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x557919a5cae0 con 0x55791971e2f0 2024-05-27T08:04:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.526+0000 7f1bfd600640 1 -- 172.21.15.63:0/487854697 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1c04024060 con 0x55791971e2f0 2024-05-27T08:04:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.526+0000 7f1c217c6740 10 client.30244 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.526+0000 7f1c217c6740 10 client.30244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.526+0000 7f1c217c6740 20 client.30244 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.526+0000 7f1c217c6740 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557919705d80 0x557919b05e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.526+0000 7f1c217c6740 4 client.30244 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.526+0000 7f1c217c6740 1 -- 172.21.15.63:0/487854697 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557919a562c0 con 0x557919705d80 2024-05-27T08:04:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.527+0000 7f1c12a00640 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557919705d80 0x557919b05e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.528+0000 7f1bfd600640 10 client.30244 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.528+0000 7f1c12a00640 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557919705d80 0x557919b05e70 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c52c8696c7e467a server_cookie=45f364b7c4e9a76 in_seq=0 out_seq=0 2024-05-27T08:04:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.530+0000 7f1bfd600640 1 -- 172.21.15.63:0/487854697 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557919a562c0 con 0x557919705d80 2024-05-27T08:04:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.530+0000 7f1bfd600640 10 client.30244 handle_command_reply: tid=0 2024-05-27T08:04:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.530+0000 7f1c217c6740 1 client.30244 shutdown 2024-05-27T08:04:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.530+0000 7f1c217c6740 1 -- 172.21.15.63:0/487854697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 msgr2=0x557919af3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.531+0000 7f1c217c6740 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 0x557919af3c60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1c040dfed0 tx=0x7f1c040a9b60 comp rx=0 tx=0).stop 2024-05-27T08:04:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.531+0000 7f1c217c6740 1 -- 172.21.15.63:0/487854697 shutdown_connections 2024-05-27T08:04:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.531+0000 7f1c217c6740 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557919705d80 0x557919b05e70 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.531+0000 7f1c217c6740 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919affe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.531+0000 7f1c217c6740 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579197264d0 0x557919af41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.531+0000 7f1c217c6740 1 --2- 172.21.15.63:0/487854697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 0x557919af3c60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.531+0000 7f1c217c6740 1 -- 172.21.15.63:0/487854697 >> 172.21.15.63:0/487854697 conn(0x55791973b300 msgr2=0x557919aeb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.531+0000 7f1bfd600640 0 client.30244 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:13.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.531+0000 7f6c16e48b80 1 Processor -- start 2024-05-27T08:04:13.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.531+0000 7f6c16e48b80 1 -- start start 2024-05-27T08:04:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.532+0000 7f1c217c6740 1 -- 172.21.15.63:0/487854697 shutdown_connections 2024-05-27T08:04:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.532+0000 7f1c217c6740 1 -- 172.21.15.63:0/487854697 wait complete. 2024-05-27T08:04:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.532+0000 7f1c217c6740 20 client.30244 trim_cache size 0 max 16384 2024-05-27T08:04:13.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.532+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec2724ec0 0x563ec28a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.532+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.532+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a9db0 0x563ec28af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.532+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001b50e0 con 0x563ec28a3e30 2024-05-27T08:04:13.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.532+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001b2970 con 0x563ec2724ec0 2024-05-27T08:04:13.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.532+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001b07e0 con 0x563ec28a9db0 2024-05-27T08:04:13.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.533+0000 7f6c05a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec2724ec0 0x563ec28a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.533+0000 7f6c06400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a9db0 0x563ec28af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.533+0000 7f6c06400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a9db0 0x563ec28af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:42614/0 (socket says 172.21.15.63:42614) 2024-05-27T08:04:13.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.533+0000 7f6c06400700 1 -- 172.21.15.63:0/55442368 learned_addr learned my addr 172.21.15.63:0/55442368 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:13.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.533+0000 7f6c05a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec2724ec0 0x563ec28a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:54236/0 (socket says 172.21.15.63:54236) 2024-05-27T08:04:13.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.534+0000 7f6c05000700 1 --2- 172.21.15.63:0/55442368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.534+0000 7f6c06400700 1 -- 172.21.15.63:0/55442368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec2724ec0 msgr2=0x563ec28a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.534+0000 7f6c06400700 1 --2- 172.21.15.63:0/55442368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec2724ec0 0x563ec28a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.534+0000 7f6c06400700 1 -- 172.21.15.63:0/55442368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 msgr2=0x563ec28a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.534+0000 7f6c06400700 1 --2- 172.21.15.63:0/55442368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.534+0000 7f6c06400700 1 -- 172.21.15.63:0/55442368 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec28bb170 con 0x563ec28a9db0 2024-05-27T08:04:13.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.534+0000 7f6c05000700 1 --2- 172.21.15.63:0/55442368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:13.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.535+0000 7f6c06400700 1 --2- 172.21.15.63:0/55442368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a9db0 0x563ec28af9b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8005440 tx=0x7f6bf80059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9605917e374dadf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.536+0000 7f6bf2000700 1 -- 172.21.15.63:0/55442368 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8011440 con 0x563ec28a9db0 2024-05-27T08:04:13.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.536+0000 7f6bf2000700 1 -- 172.21.15.63:0/55442368 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bf801f7f0 con 0x563ec28a9db0 2024-05-27T08:04:13.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.536+0000 7f6bf2000700 1 -- 172.21.15.63:0/55442368 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80277b0 con 0x563ec28a9db0 2024-05-27T08:04:13.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.537+0000 7f6c16e48b80 1 -- 172.21.15.63:0/55442368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a9db0 msgr2=0x563ec28af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.538+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/55442368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a9db0 0x563ec28af9b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8005440 tx=0x7f6bf80059d0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.538+0000 7f6c16e48b80 1 -- 172.21.15.63:0/55442368 shutdown_connections 2024-05-27T08:04:13.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.538+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/55442368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec28a9db0 0x563ec28af9b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.538+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/55442368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec2724ec0 0x563ec28a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.538+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/55442368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.538+0000 7f6c16e48b80 1 -- 172.21.15.63:0/55442368 >> 172.21.15.63:0/55442368 conn(0x563ec2785040 msgr2=0x563ec28ad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.538+0000 7f6c16e48b80 1 -- 172.21.15.63:0/55442368 shutdown_connections 2024-05-27T08:04:13.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.538+0000 7f6c16e48b80 1 -- 172.21.15.63:0/55442368 wait complete. 2024-05-27T08:04:13.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.539+0000 7f6c16e48b80 1 -- start start 2024-05-27T08:04:13.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.540+0000 7f6c16e48b80 1 Processor -- start 2024-05-27T08:04:13.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.540+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec2724ec0 0x563ec28b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.540+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.540+0000 7f6c16e48b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a9db0 0x563ec28b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.541+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001b0e30 con 0x563ec28a3e30 2024-05-27T08:04:13.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.541+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001b22a0 con 0x563ec28a9db0 2024-05-27T08:04:13.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.541+0000 7f6c16e48b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c001b1500 con 0x563ec2724ec0 2024-05-27T08:04:13.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.541+0000 7f6c05000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.541+0000 7f6c16e48b80 10 client.0 _ll_register_callbacks cb 0x563ec276bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:13.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.541+0000 7f6c05000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46698/0 (socket says 172.21.15.63:46698) 2024-05-27T08:04:13.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.541+0000 7f6c05000700 1 -- 172.21.15.63:0/3249199492 learned_addr learned my addr 172.21.15.63:0/3249199492 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:13.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.541+0000 7f6c05a00700 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec2724ec0 0x563ec28b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.542+0000 7f6c05000700 1 -- 172.21.15.63:0/3249199492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec2724ec0 msgr2=0x563ec28b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.542+0000 7f6c05000700 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec2724ec0 0x563ec28b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.542+0000 7f6c05000700 1 -- 172.21.15.63:0/3249199492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a9db0 msgr2=0x563ec28b2a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:04:13.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.542+0000 7f6c05000700 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a9db0 0x563ec28b2a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.542+0000 7f6c05000700 1 -- 172.21.15.63:0/3249199492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec28b2fc0 con 0x563ec28a3e30 2024-05-27T08:04:13.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.542+0000 7f6c05000700 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28b5ff0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0a9590 tx=0x7f6bec010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74aaf75bb6015483 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.542+0000 7f6be7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:13.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.543+0000 7f6be7400700 1 -- 172.21.15.63:0/3249199492 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec020040 con 0x563ec28a3e30 2024-05-27T08:04:13.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.543+0000 7f6be7400700 1 -- 172.21.15.63:0/3249199492 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 9571+0+0 (secure 0 0 0) 0x7f6bec0dfcf0 con 0x563ec28a3e30 2024-05-27T08:04:13.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.543+0000 7f6c16e48b80 1 -- 172.21.15.63:0/3249199492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bf801f7f0 con 0x563ec28a3e30 2024-05-27T08:04:13.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.544+0000 7f6be7400700 1 -- 172.21.15.63:0/3249199492 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6bec0a8d70 con 0x563ec28a3e30 2024-05-27T08:04:13.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.544+0000 7f6c16e48b80 10 client.21330 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:13.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.544+0000 7f6c16e48b80 1 -- 172.21.15.63:0/3249199492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563ec2931110 con 0x563ec28a3e30 2024-05-27T08:04:13.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.545+0000 7f6be7400700 1 -- 172.21.15.63:0/3249199492 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bf801f7f0 con 0x563ec28a3e30 2024-05-27T08:04:13.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.545+0000 7f6be7400700 1 -- 172.21.15.63:0/3249199492 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6bec0a9860 con 0x563ec28a3e30 2024-05-27T08:04:13.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.545+0000 7f6c16e48b80 10 client.21330 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:13.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.545+0000 7f6c16e48b80 10 client.21330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:13.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.545+0000 7f6c16e48b80 20 client.21330 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:13.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.545+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ec29317a0 0x563ec2933bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.545+0000 7f6c16e48b80 4 client.21330 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:04:13.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.545+0000 7f6c16e48b80 1 -- 172.21.15.63:0/3249199492 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563ec27cd5f0 con 0x563ec29317a0 2024-05-27T08:04:13.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.546+0000 7f6c05a00700 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ec29317a0 0x563ec2933bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.548+0000 7f6c05a00700 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ec29317a0 0x563ec2933bc0 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6255df9a62ee0 server_cookie=40d1c52994888395 in_seq=0 out_seq=0 2024-05-27T08:04:13.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.548+0000 7f6be7400700 10 client.21330 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:13.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.550+0000 7f6be7400700 1 -- 172.21.15.63:0/3249199492 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ec27cd5f0 con 0x563ec29317a0 2024-05-27T08:04:13.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.550+0000 7f6be7400700 10 client.21330 handle_command_reply: tid=0 2024-05-27T08:04:13.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.550+0000 7f6c16e48b80 1 client.21330 shutdown 2024-05-27T08:04:13.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.551+0000 7f6c16e48b80 1 -- 172.21.15.63:0/3249199492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 msgr2=0x563ec28b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.551+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28b5ff0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0a9590 tx=0x7f6bec010800 comp rx=0 tx=0).stop 2024-05-27T08:04:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.552+0000 7f6c16e48b80 1 -- 172.21.15.63:0/3249199492 shutdown_connections 2024-05-27T08:04:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.552+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563ec29317a0 0x563ec2933bc0 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.552+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ec28a9db0 0x563ec28b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.552+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ec2724ec0 0x563ec28b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.552+0000 7f6c16e48b80 1 --2- 172.21.15.63:0/3249199492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ec28a3e30 0x563ec28b5ff0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.552+0000 7f6be7400700 0 client.21330 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.552+0000 7f6c16e48b80 1 -- 172.21.15.63:0/3249199492 >> 172.21.15.63:0/3249199492 conn(0x563ec2785040 msgr2=0x563ec28add20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.553+0000 7f6c16e48b80 1 -- 172.21.15.63:0/3249199492 shutdown_connections 2024-05-27T08:04:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.553+0000 7f6c16e48b80 1 -- 172.21.15.63:0/3249199492 wait complete. 2024-05-27T08:04:13.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.553+0000 7f6c16e48b80 20 client.21330 trim_cache size 0 max 16384 2024-05-27T08:04:13.556 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:04:13.556 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:04:13.556 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "caa5bef5-dc73-42ae-9d49-30593444e32a", 2024-05-27T08:04:13.556 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:04:13.556 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:04:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.554+0000 7f6c08400700 1 -- 172.21.15.63:0/1162025384 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6be80b2a70 msgr2=0x7f6be80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.554+0000 7f6c08400700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6be80b2a70 0x7f6be80b4f80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80046e0 tx=0x7f6bf800b390 comp rx=0 tx=0).stop 2024-05-27T08:04:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.555+0000 7f6c08400700 1 -- 172.21.15.63:0/1162025384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c00107de0 msgr2=0x7f6c0019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.555+0000 7f6c08400700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c00107de0 0x7f6c0019d570 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f6c00076aa0 tx=0x7f6bec00dc30 comp rx=0 tx=0).stop 2024-05-27T08:04:13.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.555+0000 7f6c08400700 1 -- 172.21.15.63:0/1162025384 shutdown_connections 2024-05-27T08:04:13.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.555+0000 7f6c08400700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6be80b2a70 0x7f6be80b4f80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.555+0000 7f6c08400700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c00108730 0x7f6c001a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.555+0000 7f6c08400700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c001034b0 0x7f6c0019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.555+0000 7f6c08400700 1 --2- 172.21.15.63:0/1162025384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c00107de0 0x7f6c0019d570 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.555+0000 7f6c08400700 1 -- 172.21.15.63:0/1162025384 >> 172.21.15.63:0/1162025384 conn(0x7f6c000fedb0 msgr2=0x7f6c00109730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.555+0000 7f6c08400700 1 -- 172.21.15.63:0/1162025384 shutdown_connections 2024-05-27T08:04:13.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:13.555+0000 7f6c08400700 1 -- 172.21.15.63:0/1162025384 wait complete. 2024-05-27T08:04:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.563+0000 7f1c217c6740 1 Processor -- start 2024-05-27T08:04:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.563+0000 7f1c217c6740 1 -- start start 2024-05-27T08:04:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.563+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 0x557919aef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.563+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579197264d0 0x557919af6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.563+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919af2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.563+0000 7f1c217c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141b5ca0 con 0x5579197471b0 2024-05-27T08:04:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.563+0000 7f1c217c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141b3570 con 0x5579197264d0 2024-05-27T08:04:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.563+0000 7f1c217c6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141b13e0 con 0x55791971e2f0 2024-05-27T08:04:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.564+0000 7f1c13e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919af2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.564+0000 7f1c13e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919af2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46702/0 (socket says 172.21.15.63:46702) 2024-05-27T08:04:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.564+0000 7f1c13e00640 1 -- 172.21.15.63:0/2147128608 learned_addr learned my addr 172.21.15.63:0/2147128608 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.564+0000 7f1c13400640 1 --2- 172.21.15.63:0/2147128608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 0x557919aef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.564+0000 7f1c12a00640 1 --2- 172.21.15.63:0/2147128608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579197264d0 0x557919af6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.565+0000 7f1c13e00640 1 -- 172.21.15.63:0/2147128608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 msgr2=0x557919aef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.565+0000 7f1c13e00640 1 --2- 172.21.15.63:0/2147128608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 0x557919aef440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.565+0000 7f1c13e00640 1 -- 172.21.15.63:0/2147128608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579197264d0 msgr2=0x557919af6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.565+0000 7f1c13e00640 1 --2- 172.21.15.63:0/2147128608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579197264d0 0x557919af6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.565+0000 7f1c13e00640 1 -- 172.21.15.63:0/2147128608 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557919af21b0 con 0x5579197471b0 2024-05-27T08:04:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.565+0000 7f1c13e00640 1 --2- 172.21.15.63:0/2147128608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919af2ba0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c012850 tx=0x7f1c0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf17accd5a47ed1b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.566+0000 7f1bffe00640 1 -- 172.21.15.63:0/2147128608 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c004d20 con 0x5579197471b0 2024-05-27T08:04:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.566+0000 7f1bffe00640 1 -- 172.21.15.63:0/2147128608 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c0c01edc0 con 0x5579197471b0 2024-05-27T08:04:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.566+0000 7f1bffe00640 1 -- 172.21.15.63:0/2147128608 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c005740 con 0x5579197471b0 2024-05-27T08:04:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.567+0000 7f1c217c6740 1 -- 172.21.15.63:0/2147128608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 msgr2=0x557919af2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.567+0000 7f1c217c6740 1 --2- 172.21.15.63:0/2147128608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919af2ba0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c012850 tx=0x7f1c0c014000 comp rx=0 tx=0).stop 2024-05-27T08:04:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.567+0000 7f1c217c6740 1 -- 172.21.15.63:0/2147128608 shutdown_connections 2024-05-27T08:04:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.567+0000 7f1c217c6740 1 --2- 172.21.15.63:0/2147128608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919af2ba0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.567+0000 7f1c217c6740 1 --2- 172.21.15.63:0/2147128608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579197264d0 0x557919af6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.567+0000 7f1c217c6740 1 --2- 172.21.15.63:0/2147128608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55791971e2f0 0x557919aef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.567+0000 7f1c217c6740 1 -- 172.21.15.63:0/2147128608 >> 172.21.15.63:0/2147128608 conn(0x55791973b300 msgr2=0x557919aea340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.567+0000 7f1c217c6740 1 -- 172.21.15.63:0/2147128608 shutdown_connections 2024-05-27T08:04:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.567+0000 7f1c217c6740 1 -- 172.21.15.63:0/2147128608 wait complete. 2024-05-27T08:04:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.568+0000 7f1c217c6740 1 -- start start 2024-05-27T08:04:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.568+0000 7f1c217c6740 1 Processor -- start 2024-05-27T08:04:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.569+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55791971e2f0 0x557919aed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.569+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579197264d0 0x557919aede70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.569+0000 7f1c217c6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919aee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.569+0000 7f1c217c6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141b1a30 con 0x5579197471b0 2024-05-27T08:04:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.569+0000 7f1c217c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141b2ea0 con 0x55791971e2f0 2024-05-27T08:04:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.569+0000 7f1c217c6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141b2100 con 0x5579197264d0 2024-05-27T08:04:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.569+0000 7f1c217c6740 10 client.0 _ll_register_callbacks cb 0x7f1c040035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.570+0000 7f1c13e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919aee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.570+0000 7f1c13e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919aee3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46712/0 (socket says 172.21.15.63:46712) 2024-05-27T08:04:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.570+0000 7f1c13e00640 1 -- 172.21.15.63:0/136448301 learned_addr learned my addr 172.21.15.63:0/136448301 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.570+0000 7f1c12a00640 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579197264d0 0x557919aede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.570+0000 7f1c13400640 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55791971e2f0 0x557919aed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.570+0000 7f1c13e00640 1 -- 172.21.15.63:0/136448301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579197264d0 msgr2=0x557919aede70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.570+0000 7f1c13e00640 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579197264d0 0x557919aede70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.570+0000 7f1c13e00640 1 -- 172.21.15.63:0/136448301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55791971e2f0 msgr2=0x557919aed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.570+0000 7f1c13e00640 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55791971e2f0 0x557919aed930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.570+0000 7f1c13e00640 1 -- 172.21.15.63:0/136448301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557919afadc0 con 0x5579197471b0 2024-05-27T08:04:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.571+0000 7f1c13e00640 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919aee3b0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c012590 tx=0x7f1c0c0125c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5469d4328e2e8b48 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.571+0000 7f1bfd600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.571+0000 7f1bfd600640 1 -- 172.21.15.63:0/136448301 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c035020 con 0x5579197471b0 2024-05-27T08:04:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.572+0000 7f1c217c6740 1 -- 172.21.15.63:0/136448301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c01edc0 con 0x5579197471b0 2024-05-27T08:04:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.572+0000 7f1bfd600640 1 -- 172.21.15.63:0/136448301 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 9571+0+0 (secure 0 0 0) 0x7f1c0c004a40 con 0x5579197471b0 2024-05-27T08:04:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.572+0000 7f1bfd600640 1 -- 172.21.15.63:0/136448301 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c040050 con 0x5579197471b0 2024-05-27T08:04:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.572+0000 7f1c217c6740 10 client.21336 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.572+0000 7f1c217c6740 1 -- 172.21.15.63:0/136448301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557919afc050 con 0x5579197471b0 2024-05-27T08:04:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.572+0000 7f1bfd600640 1 -- 172.21.15.63:0/136448301 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c0c01edc0 con 0x5579197471b0 2024-05-27T08:04:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.573+0000 7f1bfd600640 1 -- 172.21.15.63:0/136448301 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1c0c01f420 con 0x5579197471b0 2024-05-27T08:04:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.573+0000 7f1c217c6740 10 client.21336 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.573+0000 7f1c217c6740 10 client.21336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.573+0000 7f1c217c6740 20 client.21336 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.574+0000 7f1c217c6740 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579196db1c0 0x557919b704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.574+0000 7f1c217c6740 4 client.21336 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:04:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.574+0000 7f1c217c6740 1 -- 172.21.15.63:0/136448301 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579199d04b0 con 0x5579196db1c0 2024-05-27T08:04:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.574+0000 7f1c13400640 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579196db1c0 0x557919b704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.576+0000 7f1c13400640 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579196db1c0 0x557919b704a0 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e818c1e3c2c73a server_cookie=955c332b5ad871d3 in_seq=0 out_seq=0 2024-05-27T08:04:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.576+0000 7f1bfd600640 10 client.21336 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.618+0000 7f1bfd600640 1 -- 172.21.15.63:0/136448301 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5579199d04b0 con 0x5579196db1c0 2024-05-27T08:04:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.618+0000 7f1bfd600640 10 client.21336 handle_command_reply: tid=0 2024-05-27T08:04:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.619+0000 7f1c217c6740 1 client.21336 shutdown 2024-05-27T08:04:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.619+0000 7f1c217c6740 1 -- 172.21.15.63:0/136448301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 msgr2=0x557919aee3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.619+0000 7f1c217c6740 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919aee3b0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c012590 tx=0x7f1c0c0125c0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.619+0000 7f1c217c6740 1 -- 172.21.15.63:0/136448301 shutdown_connections 2024-05-27T08:04:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.619+0000 7f1c217c6740 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5579196db1c0 0x557919b704a0 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.620+0000 7f1c217c6740 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5579197471b0 0x557919aee3b0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.620+0000 7f1c217c6740 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579197264d0 0x557919aede70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.620+0000 7f1c217c6740 1 --2- 172.21.15.63:0/136448301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55791971e2f0 0x557919aed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.620+0000 7f1c217c6740 1 -- 172.21.15.63:0/136448301 >> 172.21.15.63:0/136448301 conn(0x55791973b300 msgr2=0x557919ae98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.620+0000 7f1bfd600640 0 client.21336 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.620+0000 7f1c217c6740 1 -- 172.21.15.63:0/136448301 shutdown_connections 2024-05-27T08:04:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.620+0000 7f1c217c6740 1 -- 172.21.15.63:0/136448301 wait complete. 2024-05-27T08:04:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.620+0000 7f1c217c6740 20 client.21336 trim_cache size 0 max 16384 2024-05-27T08:04:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:04:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:04:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:04:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.621+0000 7f1c19200640 1 -- 172.21.15.63:0/715069283 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1bf80b29d0 msgr2=0x7f1bf80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.621+0000 7f1c19200640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1bf80b29d0 0x7f1bf80b4ee0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x557919ae6bd0 tx=0x7f1c0c009680 comp rx=0 tx=0).stop 2024-05-27T08:04:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.621+0000 7f1c19200640 1 -- 172.21.15.63:0/715069283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c141027c0 msgr2=0x7f1c1419c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.621+0000 7f1c19200640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c141027c0 0x7f1c1419c780 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400d950 tx=0x7f1c0400de10 comp rx=0 tx=0).stop 2024-05-27T08:04:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.622+0000 7f1c19200640 1 -- 172.21.15.63:0/715069283 shutdown_connections 2024-05-27T08:04:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.622+0000 7f1c19200640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1bf80b29d0 0x7f1bf80b4ee0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.622+0000 7f1c19200640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c14110360 0x7f1c141a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.622+0000 7f1c19200640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c1410da00 0x7f1c1419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.622+0000 7f1c19200640 1 --2- 172.21.15.63:0/715069283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c141027c0 0x7f1c1419c780 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.622+0000 7f1c19200640 1 -- 172.21.15.63:0/715069283 >> 172.21.15.63:0/715069283 conn(0x7f1c140fe210 msgr2=0x7f1c141111a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.622+0000 7f1c19200640 1 -- 172.21.15.63:0/715069283 shutdown_connections 2024-05-27T08:04:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:13.622+0000 7f1c19200640 1 -- 172.21.15.63:0/715069283 wait complete. 2024-05-27T08:04:13.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:13 smithi063 ceph-mon[31613]: pgmap v972: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 9 op/s 2024-05-27T08:04:13.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:13 smithi086 ceph-mon[34687]: pgmap v972: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 9 op/s 2024-05-27T08:04:13.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:13 smithi149 ceph-mon[35408]: pgmap v972: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 9 op/s 2024-05-27T08:04:14.700 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:04:14.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:04:14.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:14 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:04:14.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:04:14.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:14 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:04:14.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:04:14.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:04:14.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:04:14.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:04:14.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:04:14.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:04:14.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:14.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:04:14.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:14 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:15.228 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:15.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:15 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:15.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:15 smithi063 ceph-mon[31613]: pgmap v973: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 9 op/s 2024-05-27T08:04:15.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:15 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:15.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:15 smithi086 ceph-mon[34687]: pgmap v973: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 9 op/s 2024-05-27T08:04:15.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:15 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:15.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:15 smithi149 ceph-mon[35408]: pgmap v973: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.2 MiB/s wr, 9 op/s 2024-05-27T08:04:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:17 smithi063 ceph-mon[31613]: osdmap e144: 12 total, 12 up, 12 in 2024-05-27T08:04:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:17 smithi063 ceph-mon[31613]: pgmap v975: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.5 MiB/s wr, 31 op/s 2024-05-27T08:04:17.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:17 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:04:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:17 smithi086 ceph-mon[34687]: osdmap e144: 12 total, 12 up, 12 in 2024-05-27T08:04:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:17 smithi086 ceph-mon[34687]: pgmap v975: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.5 MiB/s wr, 31 op/s 2024-05-27T08:04:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:17 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:04:17.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:17 smithi149 ceph-mon[35408]: osdmap e144: 12 total, 12 up, 12 in 2024-05-27T08:04:17.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:17 smithi149 ceph-mon[35408]: pgmap v975: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.5 MiB/s wr, 31 op/s 2024-05-27T08:04:17.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:17 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:04:17.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.620+0000 7f836ac00700 1 -- 172.21.15.63:0/2009944553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8364105510 msgr2=0x7f8364105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.620+0000 7f836ac00700 1 --2- 172.21.15.63:0/2009944553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8364105510 0x7f8364105920 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f8354009990 tx=0x7f8354009ca0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.620+0000 7f836ac00700 1 -- 172.21.15.63:0/2009944553 shutdown_connections 2024-05-27T08:04:17.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.620+0000 7f836ac00700 1 --2- 172.21.15.63:0/2009944553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f836410e8f0 0x7f8364110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.620+0000 7f836ac00700 1 --2- 172.21.15.63:0/2009944553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8364105510 0x7f8364105920 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.620+0000 7f836ac00700 1 --2- 172.21.15.63:0/2009944553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364105e60 0x7f836410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.620+0000 7f836ac00700 1 -- 172.21.15.63:0/2009944553 >> 172.21.15.63:0/2009944553 conn(0x7f83640fedb0 msgr2=0x7f83641011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.620+0000 7f836ac00700 1 -- 172.21.15.63:0/2009944553 shutdown_connections 2024-05-27T08:04:17.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.620+0000 7f836ac00700 1 -- 172.21.15.63:0/2009944553 wait complete. 2024-05-27T08:04:17.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.622+0000 7f836ac00700 1 Processor -- start 2024-05-27T08:04:17.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.622+0000 7f836ac00700 1 -- start start 2024-05-27T08:04:17.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.623+0000 7f836ac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8364105510 0x7f836419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.623+0000 7f836ac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364105e60 0x7f836419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.623+0000 7f836ac00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f836410e8f0 0x7f83641a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.623+0000 7f836ac00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83641142d0 con 0x7f836410e8f0 2024-05-27T08:04:17.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.623+0000 7f836ac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8364114150 con 0x7f8364105e60 2024-05-27T08:04:17.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.623+0000 7f836ac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8364114450 con 0x7f8364105510 2024-05-27T08:04:17.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.624+0000 7f8363400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364105e60 0x7f836419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.624+0000 7f8363e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8364105510 0x7f836419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.624+0000 7f8363400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364105e60 0x7f836419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:54266/0 (socket says 172.21.15.63:54266) 2024-05-27T08:04:17.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.624+0000 7f8363400700 1 -- 172.21.15.63:0/2688775145 learned_addr learned my addr 172.21.15.63:0/2688775145 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:17.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.624+0000 7f8368c00700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f836410e8f0 0x7f83641a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.624+0000 7f8363400700 1 -- 172.21.15.63:0/2688775145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8364105510 msgr2=0x7f836419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.624+0000 7f8363400700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8364105510 0x7f836419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.624+0000 7f8363400700 1 -- 172.21.15.63:0/2688775145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f836410e8f0 msgr2=0x7f83641a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.624+0000 7f8363400700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f836410e8f0 0x7f83641a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.624+0000 7f8363400700 1 -- 172.21.15.63:0/2688775145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83641a1d90 con 0x7f8364105e60 2024-05-27T08:04:17.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.625+0000 7f8363400700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364105e60 0x7f836419d460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f835000e9a0 tx=0x7f835000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.625+0000 7f8360c00700 1 -- 172.21.15.63:0/2688775145 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835000cbd0 con 0x7f8364105e60 2024-05-27T08:04:17.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.625+0000 7f836ac00700 1 -- 172.21.15.63:0/2688775145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83641a2080 con 0x7f8364105e60 2024-05-27T08:04:17.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.625+0000 7f8360c00700 1 -- 172.21.15.63:0/2688775145 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f835000cd70 con 0x7f8364105e60 2024-05-27T08:04:17.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.625+0000 7f8360c00700 1 -- 172.21.15.63:0/2688775145 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350018680 con 0x7f8364105e60 2024-05-27T08:04:17.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.626+0000 7f836ac00700 1 -- 172.21.15.63:0/2688775145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83641a99a0 con 0x7f8364105e60 2024-05-27T08:04:17.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.632+0000 7f8379739b80 1 Processor -- start 2024-05-27T08:04:17.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.632+0000 7f8379739b80 1 -- start start 2024-05-27T08:04:17.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.632+0000 7f8360c00700 1 -- 172.21.15.63:0/2688775145 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8350010430 con 0x7f8364105e60 2024-05-27T08:04:17.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.632+0000 7f8379739b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e61538f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8379739b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615389870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8379739b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e61538a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8379739b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e6152b7e40 con 0x55e61538fd30 2024-05-27T08:04:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8379739b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6152a79e0 con 0x55e61520ae40 2024-05-27T08:04:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8379739b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e6152aba20 con 0x55e615389db0 2024-05-27T08:04:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8363e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615389870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8368c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e61538f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8363e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615389870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46740/0 (socket says 172.21.15.63:46740) 2024-05-27T08:04:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8368c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e61538f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:54268/0 (socket says 172.21.15.63:54268) 2024-05-27T08:04:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8363e00700 1 -- 172.21.15.63:0/534772037 learned_addr learned my addr 172.21.15.63:0/534772037 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.633+0000 7f8363400700 1 --2- 172.21.15.63:0/534772037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e61538a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.634+0000 7f8368c00700 1 -- 172.21.15.63:0/534772037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 msgr2=0x55e61538a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.634+0000 7f8368c00700 1 --2- 172.21.15.63:0/534772037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e61538a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.634+0000 7f8368c00700 1 -- 172.21.15.63:0/534772037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 msgr2=0x55e615389870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.634+0000 7f8368c00700 1 --2- 172.21.15.63:0/534772037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615389870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.634+0000 7f8368c00700 1 -- 172.21.15.63:0/534772037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e615270420 con 0x55e61520ae40 2024-05-27T08:04:17.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.634+0000 7f8368c00700 1 --2- 172.21.15.63:0/534772037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e61538f7f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f835c009c90 tx=0x7f835c007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b834428b7d2b68f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.635+0000 7f8359600700 1 -- 172.21.15.63:0/534772037 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c007cd0 con 0x55e61520ae40 2024-05-27T08:04:17.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.635+0000 7f8359600700 1 -- 172.21.15.63:0/534772037 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f835c004bb0 con 0x55e61520ae40 2024-05-27T08:04:17.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.635+0000 7f8359600700 1 -- 172.21.15.63:0/534772037 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c014410 con 0x55e61520ae40 2024-05-27T08:04:17.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.636+0000 7f8360c00700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83480b29f0 0x7f83480b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.636+0000 7f8360c00700 1 -- 172.21.15.63:0/2688775145 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f83500e2c50 con 0x7f8364105e60 2024-05-27T08:04:17.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.636+0000 7f8363e00700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83480b29f0 0x7f83480b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.636+0000 7f8379739b80 1 -- 172.21.15.63:0/534772037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 msgr2=0x55e61538f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.636+0000 7f8379739b80 1 --2- 172.21.15.63:0/534772037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e61538f7f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f835c009c90 tx=0x7f835c007500 comp rx=0 tx=0).stop 2024-05-27T08:04:17.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.636+0000 7f8379739b80 1 -- 172.21.15.63:0/534772037 shutdown_connections 2024-05-27T08:04:17.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.636+0000 7f8379739b80 1 --2- 172.21.15.63:0/534772037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e61538a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.636+0000 7f8379739b80 1 --2- 172.21.15.63:0/534772037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e61538f7f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.636+0000 7f8379739b80 1 --2- 172.21.15.63:0/534772037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615389870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.636+0000 7f8379739b80 1 -- 172.21.15.63:0/534772037 >> 172.21.15.63:0/534772037 conn(0x55e61526afc0 msgr2=0x55e6153848d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.637+0000 7f8379739b80 1 -- 172.21.15.63:0/534772037 shutdown_connections 2024-05-27T08:04:17.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.637+0000 7f8379739b80 1 -- 172.21.15.63:0/534772037 wait complete. 2024-05-27T08:04:17.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.637+0000 7f8363e00700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83480b29f0 0x7f83480b4f00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8354009750 tx=0x7f8354005040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.638+0000 7f8379739b80 1 -- start start 2024-05-27T08:04:17.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.638+0000 7f8379739b80 1 Processor -- start 2024-05-27T08:04:17.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8379739b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61520ae40 0x55e6153935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8379739b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e615389db0 0x55e615393b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8379739b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e61538fd30 0x55e615394040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8379739b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e6151adfa0 con 0x55e61520ae40 2024-05-27T08:04:17.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8379739b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e615248760 con 0x55e615389db0 2024-05-27T08:04:17.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8379739b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e615396800 con 0x55e61538fd30 2024-05-27T08:04:17.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8379739b80 10 client.0 _ll_register_callbacks cb 0x55e615251c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:17.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8363e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61520ae40 0x55e6153935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8363e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61520ae40 0x55e6153935c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46744/0 (socket says 172.21.15.63:46744) 2024-05-27T08:04:17.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8363e00700 1 -- 172.21.15.63:0/4019647572 learned_addr learned my addr 172.21.15.63:0/4019647572 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:17.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.639+0000 7f8363400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e615389db0 0x55e615393b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.640+0000 7f8368c00700 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e61538fd30 0x55e615394040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.640+0000 7f8363e00700 1 -- 172.21.15.63:0/4019647572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e61538fd30 msgr2=0x55e615394040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.640+0000 7f8363e00700 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e61538fd30 0x55e615394040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.640+0000 7f8363e00700 1 -- 172.21.15.63:0/4019647572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e615389db0 msgr2=0x55e615393b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.640+0000 7f8363e00700 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e615389db0 0x55e615393b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.640+0000 7f8363e00700 1 -- 172.21.15.63:0/4019647572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6151f78d0 con 0x55e61520ae40 2024-05-27T08:04:17.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.640+0000 7f8368c00700 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e61538fd30 0x55e615394040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:17.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.641+0000 7f8363e00700 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61520ae40 0x55e6153935c0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f835400b2e0 tx=0x7f8354009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51056c42b1f8ac6d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.641+0000 7f834e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:17.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.641+0000 7f834e000700 1 -- 172.21.15.63:0/4019647572 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354035060 con 0x55e61520ae40 2024-05-27T08:04:17.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.641+0000 7f834e000700 1 -- 172.21.15.63:0/4019647572 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f8354025480 con 0x55e61520ae40 2024-05-27T08:04:17.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.641+0000 7f8379739b80 1 -- 172.21.15.63:0/4019647572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835c004bb0 con 0x55e61520ae40 2024-05-27T08:04:17.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.641+0000 7f834e000700 1 -- 172.21.15.63:0/4019647572 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8354025680 con 0x55e61520ae40 2024-05-27T08:04:17.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.642+0000 7f8379739b80 10 client.21348 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:17.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.642+0000 7f8379739b80 1 -- 172.21.15.63:0/4019647572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e615278060 con 0x55e61520ae40 2024-05-27T08:04:17.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.642+0000 7f834e000700 1 -- 172.21.15.63:0/4019647572 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f835c004bb0 con 0x55e61520ae40 2024-05-27T08:04:17.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.642+0000 7f834e000700 1 -- 172.21.15.63:0/4019647572 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f835400f8b0 con 0x55e61520ae40 2024-05-27T08:04:17.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.642+0000 7f8379739b80 10 client.21348 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:17.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.642+0000 7f8379739b80 10 client.21348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:17.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.642+0000 7f8379739b80 20 client.21348 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:17.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.642+0000 7f8379739b80 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e6153a3770 0x55e6153a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.642+0000 7f8379739b80 4 client.21348 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:17.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.642+0000 7f8379739b80 1 -- 172.21.15.63:0/4019647572 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6152b9d90 con 0x55e6153a3770 2024-05-27T08:04:17.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.643+0000 7f8363400700 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e6153a3770 0x55e6153a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.644+0000 7f8363400700 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e6153a3770 0x55e6153a5c20 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95bb988ac74b9e50 server_cookie=34ff70bc08878401 in_seq=0 out_seq=0 2024-05-27T08:04:17.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.644+0000 7f834e000700 10 client.21348 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:17.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f834e000700 1 -- 172.21.15.63:0/4019647572 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e6152b9d90 con 0x55e6153a3770 2024-05-27T08:04:17.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f834e000700 10 client.21348 handle_command_reply: tid=0 2024-05-27T08:04:17.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f8379739b80 1 client.21348 shutdown 2024-05-27T08:04:17.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f8379739b80 1 -- 172.21.15.63:0/4019647572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61520ae40 msgr2=0x55e6153935c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f8379739b80 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61520ae40 0x55e6153935c0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f835400b2e0 tx=0x7f8354009ca0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f8379739b80 1 -- 172.21.15.63:0/4019647572 shutdown_connections 2024-05-27T08:04:17.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f8379739b80 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e6153a3770 0x55e6153a5c20 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f834e000700 0 client.21348 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:17.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f8379739b80 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e61538fd30 0x55e615394040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f8379739b80 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61520ae40 0x55e6153935c0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f8379739b80 1 --2- 172.21.15.63:0/4019647572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e615389db0 0x55e615393b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.645+0000 7f8379739b80 1 -- 172.21.15.63:0/4019647572 >> 172.21.15.63:0/4019647572 conn(0x55e61526afc0 msgr2=0x55e615384580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.646+0000 7f8379739b80 1 -- 172.21.15.63:0/4019647572 shutdown_connections 2024-05-27T08:04:17.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.646+0000 7f8379739b80 1 -- 172.21.15.63:0/4019647572 wait complete. 2024-05-27T08:04:17.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.646+0000 7f8379739b80 20 client.21348 trim_cache size 0 max 16384 2024-05-27T08:04:17.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.673+0000 7f8379739b80 1 Processor -- start 2024-05-27T08:04:17.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.673+0000 7f8379739b80 1 -- start start 2024-05-27T08:04:17.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.673+0000 7f8379739b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e615389870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.673+0000 7f8379739b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e6153958d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.673+0000 7f8379739b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e61539f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.673+0000 7f8379739b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83641b4f30 con 0x55e61538fd30 2024-05-27T08:04:17.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.673+0000 7f8379739b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83641b27c0 con 0x55e61520ae40 2024-05-27T08:04:17.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.673+0000 7f8379739b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f83641b0630 con 0x55e615389db0 2024-05-27T08:04:17.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.674+0000 7f8368c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e61539f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.674+0000 7f8368c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e61539f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46760/0 (socket says 172.21.15.63:46760) 2024-05-27T08:04:17.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.674+0000 7f8368c00700 1 -- 172.21.15.63:0/3568812600 learned_addr learned my addr 172.21.15.63:0/3568812600 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:17.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.674+0000 7f8363e00700 1 --2- 172.21.15.63:0/3568812600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e615389870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.674+0000 7f8363400700 1 --2- 172.21.15.63:0/3568812600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e6153958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.674+0000 7f8368c00700 1 -- 172.21.15.63:0/3568812600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 msgr2=0x55e6153958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.674+0000 7f8368c00700 1 --2- 172.21.15.63:0/3568812600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e6153958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.674+0000 7f8368c00700 1 -- 172.21.15.63:0/3568812600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 msgr2=0x55e615389870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.674+0000 7f8368c00700 1 --2- 172.21.15.63:0/3568812600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e615389870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.674+0000 7f8368c00700 1 -- 172.21.15.63:0/3568812600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e615395e10 con 0x55e61538fd30 2024-05-27T08:04:17.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.675+0000 7f8368c00700 1 --2- 172.21.15.63:0/3568812600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e61539f8a0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f835c007850 tx=0x7f835c007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969b4e76020d7623 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.675+0000 7f8358c00700 1 -- 172.21.15.63:0/3568812600 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c0057c0 con 0x55e61538fd30 2024-05-27T08:04:17.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.675+0000 7f8358c00700 1 -- 172.21.15.63:0/3568812600 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f835c023070 con 0x55e61538fd30 2024-05-27T08:04:17.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.675+0000 7f8358c00700 1 -- 172.21.15.63:0/3568812600 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c01e450 con 0x55e61538fd30 2024-05-27T08:04:17.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.677+0000 7f8379739b80 1 -- 172.21.15.63:0/3568812600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 msgr2=0x55e61539f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.677+0000 7f8379739b80 1 --2- 172.21.15.63:0/3568812600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e61539f8a0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f835c007850 tx=0x7f835c007ee0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.677+0000 7f8379739b80 1 -- 172.21.15.63:0/3568812600 shutdown_connections 2024-05-27T08:04:17.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.677+0000 7f8379739b80 1 --2- 172.21.15.63:0/3568812600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e61539f8a0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.677+0000 7f8379739b80 1 --2- 172.21.15.63:0/3568812600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e615389870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.677+0000 7f8379739b80 1 --2- 172.21.15.63:0/3568812600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e6153958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.677+0000 7f8379739b80 1 -- 172.21.15.63:0/3568812600 >> 172.21.15.63:0/3568812600 conn(0x55e61526afc0 msgr2=0x55e615393dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.677+0000 7f8379739b80 1 -- 172.21.15.63:0/3568812600 shutdown_connections 2024-05-27T08:04:17.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.677+0000 7f8379739b80 1 -- 172.21.15.63:0/3568812600 wait complete. 2024-05-27T08:04:17.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.678+0000 7f8379739b80 1 -- start start 2024-05-27T08:04:17.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.678+0000 7f8379739b80 1 Processor -- start 2024-05-27T08:04:17.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8379739b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e61539ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8379739b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e61539bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8379739b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615398a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8379739b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83641b0c80 con 0x55e61538fd30 2024-05-27T08:04:17.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8379739b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83641b20f0 con 0x55e61520ae40 2024-05-27T08:04:17.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8379739b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f83641b1350 con 0x55e615389db0 2024-05-27T08:04:17.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8379739b80 10 client.0 _ll_register_callbacks cb 0x55e615251c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:17.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8368c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615398a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8368c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615398a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46768/0 (socket says 172.21.15.63:46768) 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8368c00700 1 -- 172.21.15.63:0/30859870 learned_addr learned my addr 172.21.15.63:0/30859870 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8363400700 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e61539bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.679+0000 7f8363e00700 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e61539ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.680+0000 7f8363400700 1 -- 172.21.15.63:0/30859870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 msgr2=0x55e61539ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.680+0000 7f8363400700 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e61539ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.680+0000 7f8363400700 1 -- 172.21.15.63:0/30859870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 msgr2=0x55e615398a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.680+0000 7f8363400700 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615398a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.680+0000 7f8363400700 1 -- 172.21.15.63:0/30859870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e615398fa0 con 0x55e615389db0 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.680+0000 7f8368c00700 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615398a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.680+0000 7f8363400700 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e61539bfd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f83500dffe0 tx=0x7f83500e5160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3072b406b39129 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.680+0000 7f834e000700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:04:17.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.681+0000 7f834e000700 1 -- 172.21.15.63:0/30859870 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83500a92e0 con 0x55e615389db0 2024-05-27T08:04:17.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.681+0000 7f8379739b80 1 -- 172.21.15.63:0/30859870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835c023070 con 0x55e615389db0 2024-05-27T08:04:17.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.681+0000 7f834e000700 1 -- 172.21.15.63:0/30859870 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f83500a9fe0 con 0x55e615389db0 2024-05-27T08:04:17.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.681+0000 7f834e000700 1 -- 172.21.15.63:0/30859870 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f835006e050 con 0x55e615389db0 2024-05-27T08:04:17.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.681+0000 7f8379739b80 10 client.30271 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:17.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.681+0000 7f8379739b80 1 -- 172.21.15.63:0/30859870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e6154171e0 con 0x55e615389db0 2024-05-27T08:04:17.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.682+0000 7f834e000700 1 -- 172.21.15.63:0/30859870 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f835c023070 con 0x55e615389db0 2024-05-27T08:04:17.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.682+0000 7f834e000700 1 -- 172.21.15.63:0/30859870 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f83500b2540 con 0x55e615389db0 2024-05-27T08:04:17.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.682+0000 7f8379739b80 10 client.30271 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:17.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.682+0000 7f8379739b80 10 client.30271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:17.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.682+0000 7f8379739b80 20 client.30271 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:17.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.682+0000 7f8379739b80 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e6153a3770 0x55e615419900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.682+0000 7f8379739b80 4 client.30271 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:04:17.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.682+0000 7f8379739b80 1 -- 172.21.15.63:0/30859870 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e6152b3570 con 0x55e6153a3770 2024-05-27T08:04:17.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.683+0000 7f8363e00700 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e6153a3770 0x55e615419900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.684+0000 7f834e000700 10 client.30271 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:17.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.684+0000 7f8363e00700 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e6153a3770 0x55e615419900 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f25afc5a9d730c64 server_cookie=6b6c2d83e8916a97 in_seq=0 out_seq=0 2024-05-27T08:04:17.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.684+0000 7f834e000700 1 -- 172.21.15.63:0/30859870 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e6152b3570 con 0x55e6153a3770 2024-05-27T08:04:17.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.684+0000 7f834e000700 10 client.30271 handle_command_reply: tid=0 2024-05-27T08:04:17.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.684+0000 7f8379739b80 1 client.30271 shutdown 2024-05-27T08:04:17.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.684+0000 7f8379739b80 1 -- 172.21.15.63:0/30859870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 msgr2=0x55e61539bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.684+0000 7f8379739b80 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e61539bfd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f83500dffe0 tx=0x7f83500e5160 comp rx=0 tx=0).stop 2024-05-27T08:04:17.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.685+0000 7f8379739b80 1 -- 172.21.15.63:0/30859870 shutdown_connections 2024-05-27T08:04:17.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.685+0000 7f8379739b80 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e6153a3770 0x55e615419900 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.685+0000 7f8379739b80 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e61538fd30 0x55e615398a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.685+0000 7f8379739b80 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61520ae40 0x55e61539ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.685+0000 7f8379739b80 1 --2- 172.21.15.63:0/30859870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e615389db0 0x55e61539bfd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.685+0000 7f8379739b80 1 -- 172.21.15.63:0/30859870 >> 172.21.15.63:0/30859870 conn(0x55e61526afc0 msgr2=0x55e615393dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.685+0000 7f834e000700 0 client.30271 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:17.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.685+0000 7f8379739b80 1 -- 172.21.15.63:0/30859870 shutdown_connections 2024-05-27T08:04:17.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.685+0000 7f8379739b80 1 -- 172.21.15.63:0/30859870 wait complete. 2024-05-27T08:04:17.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.685+0000 7f8379739b80 20 client.30271 trim_cache size 0 max 16384 2024-05-27T08:04:17.689 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:04:17.689 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:04:17.689 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:04:17.689 INFO:teuthology.orchestra.run.smithi063.stdout: "caa5bef5-dc73-42ae-9d49-30593444e32a": { 2024-05-27T08:04:17.689 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:04:17.689 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "caa5bef5-dc73-42ae-9d49-30593444e32a", 2024-05-27T08:04:17.689 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:04:17.689 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:04:17.689 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:04:17.689 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:04:17.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.686+0000 7f836ac00700 1 -- 172.21.15.63:0/2688775145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83480b29f0 msgr2=0x7f83480b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.686+0000 7f836ac00700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83480b29f0 0x7f83480b4f00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8354009750 tx=0x7f8354005040 comp rx=0 tx=0).stop 2024-05-27T08:04:17.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.686+0000 7f836ac00700 1 -- 172.21.15.63:0/2688775145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364105e60 msgr2=0x7f836419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.686+0000 7f836ac00700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364105e60 0x7f836419d460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f835000e9a0 tx=0x7f835000ecb0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.686+0000 7f836ac00700 1 -- 172.21.15.63:0/2688775145 shutdown_connections 2024-05-27T08:04:17.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.686+0000 7f836ac00700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f83480b29f0 0x7f83480b4f00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.686+0000 7f836ac00700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f836410e8f0 0x7f83641a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.686+0000 7f836ac00700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8364105510 0x7f836419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.686+0000 7f836ac00700 1 --2- 172.21.15.63:0/2688775145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8364105e60 0x7f836419d460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.686+0000 7f836ac00700 1 -- 172.21.15.63:0/2688775145 >> 172.21.15.63:0/2688775145 conn(0x7f83640fedb0 msgr2=0x7f83641031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.687+0000 7f836ac00700 1 -- 172.21.15.63:0/2688775145 shutdown_connections 2024-05-27T08:04:17.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:17.687+0000 7f836ac00700 1 -- 172.21.15.63:0/2688775145 wait complete. 2024-05-27T08:04:18.580 INFO:tasks.cephfs.filesystem:scrub status for tag:caa5bef5-dc73-42ae-9d49-30593444e32a - {'path': '/', 'tag': 'caa5bef5-dc73-42ae-9d49-30593444e32a', 'options': 'recursive,force'} 2024-05-27T08:04:19.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:19 smithi086 ceph-mon[34687]: pgmap v976: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.0 MiB/s wr, 50 op/s 2024-05-27T08:04:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:04:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:04:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:19 smithi149 ceph-mon[35408]: pgmap v976: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.0 MiB/s wr, 50 op/s 2024-05-27T08:04:19.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:04:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:04:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:19 smithi063 ceph-mon[31613]: pgmap v976: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.0 MiB/s wr, 50 op/s 2024-05-27T08:04:19.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:19.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:04:19.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:04:19.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:04:21.712 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:21 smithi086 ceph-mon[34687]: pgmap v977: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.0 MiB/s wr, 50 op/s 2024-05-27T08:04:21.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:21 smithi149 ceph-mon[35408]: pgmap v977: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.0 MiB/s wr, 50 op/s 2024-05-27T08:04:21.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:21 smithi063 ceph-mon[31613]: pgmap v977: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.0 MiB/s wr, 50 op/s 2024-05-27T08:04:22.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:04:22.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:04:22.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:04:22.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:04:22.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:04:22.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:04:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:04:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:04:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:04:23.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:23 smithi086 ceph-mon[34687]: pgmap v978: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.4 MiB/s wr, 51 op/s 2024-05-27T08:04:23.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:23 smithi149 ceph-mon[35408]: pgmap v978: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.4 MiB/s wr, 51 op/s 2024-05-27T08:04:23.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:23 smithi063 ceph-mon[31613]: pgmap v978: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.4 MiB/s wr, 51 op/s 2024-05-27T08:04:25.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:25 smithi086 ceph-mon[34687]: pgmap v979: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.4 MiB/s wr, 51 op/s 2024-05-27T08:04:25.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:25 smithi149 ceph-mon[35408]: pgmap v979: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.4 MiB/s wr, 51 op/s 2024-05-27T08:04:25.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:25 smithi063 ceph-mon[31613]: pgmap v979: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.4 MiB/s wr, 51 op/s 2024-05-27T08:04:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.874+0000 7f29c9a00640 1 -- 172.21.15.63:0/3848973174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c410c340 msgr2=0x7f29c4105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.874+0000 7f29c9a00640 1 --2- 172.21.15.63:0/3848973174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c410c340 0x7f29c4105f80 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009900 tx=0x7f29b002a3f0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.874+0000 7f29c9a00640 1 -- 172.21.15.63:0/3848973174 shutdown_connections 2024-05-27T08:04:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.874+0000 7f29c9a00640 1 --2- 172.21.15.63:0/3848973174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c41065f0 0x7f29c41129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.874+0000 7f29c9a00640 1 --2- 172.21.15.63:0/3848973174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c410c340 0x7f29c4105f80 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.874+0000 7f29c9a00640 1 --2- 172.21.15.63:0/3848973174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c410bea0 0x7f29c4105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.874+0000 7f29c9a00640 1 -- 172.21.15.63:0/3848973174 >> 172.21.15.63:0/3848973174 conn(0x7f29c4077140 msgr2=0x7f29c4077580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.874+0000 7f29c9a00640 1 -- 172.21.15.63:0/3848973174 shutdown_connections 2024-05-27T08:04:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.875+0000 7f29c9a00640 1 -- 172.21.15.63:0/3848973174 wait complete. 2024-05-27T08:04:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.876+0000 7f29c9a00640 1 Processor -- start 2024-05-27T08:04:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.876+0000 7f29c9a00640 1 -- start start 2024-05-27T08:04:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.876+0000 7f29c9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c41065f0 0x7f29c41a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c410bea0 0x7f29c41a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c9a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c410c340 0x7f29c419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c9a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29c4115f40 con 0x7f29c41065f0 2024-05-27T08:04:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29c4115dc0 con 0x7f29c410bea0 2024-05-27T08:04:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c9a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29c41160c0 con 0x7f29c410c340 2024-05-27T08:04:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c41065f0 0x7f29c41a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c41065f0 0x7f29c41a25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41438/0 (socket says 172.21.15.63:41438) 2024-05-27T08:04:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c2a00640 1 -- 172.21.15.63:0/2658345096 learned_addr learned my addr 172.21.15.63:0/2658345096 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c2000640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c410bea0 0x7f29c41a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c3400640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c410c340 0x7f29c419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c2a00640 1 -- 172.21.15.63:0/2658345096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c410c340 msgr2=0x7f29c419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.877+0000 7f29c2a00640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c410c340 0x7f29c419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.878+0000 7f29c2a00640 1 -- 172.21.15.63:0/2658345096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c410bea0 msgr2=0x7f29c41a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.878+0000 7f29c2a00640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c410bea0 0x7f29c41a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.878+0000 7f29c2a00640 1 -- 172.21.15.63:0/2658345096 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c419cee0 con 0x7f29c41065f0 2024-05-27T08:04:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.878+0000 7f29c2a00640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c41065f0 0x7f29c41a25d0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f29b400d770 tx=0x7f29b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.878+0000 7f29bb400640 1 -- 172.21.15.63:0/2658345096 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4017070 con 0x7f29c41065f0 2024-05-27T08:04:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.878+0000 7f29bb400640 1 -- 172.21.15.63:0/2658345096 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29b4009c30 con 0x7f29c41065f0 2024-05-27T08:04:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.878+0000 7f29c9a00640 1 -- 172.21.15.63:0/2658345096 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c419d1d0 con 0x7f29c41065f0 2024-05-27T08:04:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.878+0000 7f29bb400640 1 -- 172.21.15.63:0/2658345096 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b400f620 con 0x7f29c41065f0 2024-05-27T08:04:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.879+0000 7f29c9a00640 1 -- 172.21.15.63:0/2658345096 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c41a9420 con 0x7f29c41065f0 2024-05-27T08:04:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.882+0000 7f29d0a64740 1 Processor -- start 2024-05-27T08:04:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.882+0000 7f29d0a64740 1 -- start start 2024-05-27T08:04:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 0x555d1db40860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 0x555d1db40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29d0a64740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555d1dab1b90 con 0x555d1d7752f0 2024-05-27T08:04:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29d0a64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555d1daa1730 con 0x555d1d79e1b0 2024-05-27T08:04:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29d0a64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555d1daa5770 con 0x555d1d77d4d0 2024-05-27T08:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 0x555d1db40860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29c2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29c2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45588/0 (socket says 172.21.15.63:45588) 2024-05-27T08:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 0x555d1db40860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:47916/0 (socket says 172.21.15.63:47916) 2024-05-27T08:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.883+0000 7f29c2000640 1 -- 172.21.15.63:0/569439933 learned_addr learned my addr 172.21.15.63:0/569439933 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.884+0000 7f29c3400640 1 -- 172.21.15.63:0/569439933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 msgr2=0x555d1db412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.884+0000 7f29c3400640 1 --2- 172.21.15.63:0/569439933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.884+0000 7f29c3400640 1 -- 172.21.15.63:0/569439933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 msgr2=0x555d1db40da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:04:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.884+0000 7f29c3400640 1 --2- 172.21.15.63:0/569439933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 0x555d1db40da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.884+0000 7f29c3400640 1 -- 172.21.15.63:0/569439933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d1da3a8b0 con 0x555d1d79e1b0 2024-05-27T08:04:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.884+0000 7f29c2000640 1 --2- 172.21.15.63:0/569439933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.885+0000 7f29c3400640 1 --2- 172.21.15.63:0/569439933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 0x555d1db40860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009e20 tx=0x7f29bc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a625f729748e3f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.885+0000 7f29bb400640 1 -- 172.21.15.63:0/2658345096 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f29b401a070 con 0x7f29c41065f0 2024-05-27T08:04:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.885+0000 7f29afe00640 1 -- 172.21.15.63:0/569439933 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc016070 con 0x555d1d79e1b0 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.885+0000 7f29afe00640 1 -- 172.21.15.63:0/569439933 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29bc004d20 con 0x555d1d79e1b0 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.885+0000 7f29afe00640 1 -- 172.21.15.63:0/569439933 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0126b0 con 0x555d1d79e1b0 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.886+0000 7f29d0a64740 1 -- 172.21.15.63:0/569439933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 msgr2=0x555d1db40860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.886+0000 7f29d0a64740 1 --2- 172.21.15.63:0/569439933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 0x555d1db40860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009e20 tx=0x7f29bc010740 comp rx=0 tx=0).stop 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.886+0000 7f29d0a64740 1 -- 172.21.15.63:0/569439933 shutdown_connections 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.886+0000 7f29d0a64740 1 --2- 172.21.15.63:0/569439933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.886+0000 7f29d0a64740 1 --2- 172.21.15.63:0/569439933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 0x555d1db40da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.886+0000 7f29d0a64740 1 --2- 172.21.15.63:0/569439933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 0x555d1db40860 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.886+0000 7f29d0a64740 1 -- 172.21.15.63:0/569439933 >> 172.21.15.63:0/569439933 conn(0x555d1d792300 msgr2=0x555d1db3d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.886+0000 7f29d0a64740 1 -- 172.21.15.63:0/569439933 shutdown_connections 2024-05-27T08:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.886+0000 7f29d0a64740 1 -- 172.21.15.63:0/569439933 wait complete. 2024-05-27T08:04:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.887+0000 7f29d0a64740 1 -- start start 2024-05-27T08:04:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.887+0000 7f29bb400640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a80b2870 0x7f29a80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.887+0000 7f29d0a64740 1 Processor -- start 2024-05-27T08:04:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.887+0000 7f29bb400640 1 -- 172.21.15.63:0/2658345096 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f29b40e2540 con 0x7f29c41065f0 2024-05-27T08:04:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.888+0000 7f29c3400640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a80b2870 0x7f29a80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.888+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 0x555d1db4aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.888+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db4b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.888+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d79e1b0 0x555d1db56cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.888+0000 7f29d0a64740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555d1d787790 con 0x555d1d79e1b0 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.888+0000 7f29d0a64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555d1da49bc0 con 0x555d1d7752f0 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.888+0000 7f29d0a64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555d1db4d990 con 0x555d1d77d4d0 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.888+0000 7f29d0a64740 10 client.0 _ll_register_callbacks cb 0x555d1da27400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d79e1b0 0x555d1db56cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d79e1b0 0x555d1db56cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41454/0 (socket says 172.21.15.63:41454) 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c3400640 1 -- 172.21.15.63:0/2769347272 learned_addr learned my addr 172.21.15.63:0/2769347272 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c2a00640 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 0x555d1db4aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c3400640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a80b2870 0x7f29a80b4d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00a010 tx=0x7f29bc005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c2000640 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db4b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c3400640 1 -- 172.21.15.63:0/2769347272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 msgr2=0x555d1db4b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c3400640 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db4b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c3400640 1 -- 172.21.15.63:0/2769347272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 msgr2=0x555d1db4aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c3400640 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 0x555d1db4aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.889+0000 7f29c3400640 1 -- 172.21.15.63:0/2769347272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d1d7b6210 con 0x555d1d79e1b0 2024-05-27T08:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.890+0000 7f29c3400640 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d79e1b0 0x555d1db56cc0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f29bc019000 tx=0x7f29bc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22d6cb5dfc5a4a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.890+0000 7f29acc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.890+0000 7f29acc00640 1 -- 172.21.15.63:0/2769347272 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc016050 con 0x555d1d79e1b0 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.890+0000 7f29acc00640 1 -- 172.21.15.63:0/2769347272 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f29bc012660 con 0x555d1d79e1b0 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.890+0000 7f29d0a64740 1 -- 172.21.15.63:0/2769347272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29bc004d20 con 0x555d1d79e1b0 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.891+0000 7f29acc00640 1 -- 172.21.15.63:0/2769347272 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f29bc012860 con 0x555d1d79e1b0 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.891+0000 7f29d0a64740 10 client.21363 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.891+0000 7f29d0a64740 1 -- 172.21.15.63:0/2769347272 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555d1d882620 con 0x555d1d79e1b0 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.891+0000 7f29acc00640 1 -- 172.21.15.63:0/2769347272 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29bc004d20 con 0x555d1d79e1b0 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.891+0000 7f29acc00640 1 -- 172.21.15.63:0/2769347272 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f29bc02e070 con 0x555d1d79e1b0 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.891+0000 7f29d0a64740 10 client.21363 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.891+0000 7f29d0a64740 10 client.21363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.892+0000 7f29d0a64740 20 client.21363 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.892+0000 7f29d0a64740 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555d1d75cd80 0x555d1db5cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.892+0000 7f29d0a64740 4 client.21363 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.892+0000 7f29d0a64740 1 -- 172.21.15.63:0/2769347272 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d1dab3ae0 con 0x555d1d75cd80 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.892+0000 7f29c2a00640 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555d1d75cd80 0x555d1db5cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.893+0000 7f29c2a00640 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555d1d75cd80 0x555d1db5cbb0 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8d8120bdfb415da server_cookie=c999bcd9f217844c in_seq=0 out_seq=0 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.893+0000 7f29acc00640 10 client.21363 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.895+0000 7f29acc00640 1 -- 172.21.15.63:0/2769347272 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555d1dab3ae0 con 0x555d1d75cd80 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.895+0000 7f29acc00640 10 client.21363 handle_command_reply: tid=0 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.895+0000 7f29d0a64740 1 client.21363 shutdown 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 1 -- 172.21.15.63:0/2769347272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d79e1b0 msgr2=0x555d1db56cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d79e1b0 0x555d1db56cc0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f29bc019000 tx=0x7f29bc010740 comp rx=0 tx=0).stop 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 1 -- 172.21.15.63:0/2769347272 shutdown_connections 2024-05-27T08:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555d1d75cd80 0x555d1db5cbb0 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d79e1b0 0x555d1db56cc0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db4b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 1 --2- 172.21.15.63:0/2769347272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 0x555d1db4aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 1 -- 172.21.15.63:0/2769347272 >> 172.21.15.63:0/2769347272 conn(0x555d1d792300 msgr2=0x555d1db40630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29acc00640 0 client.21363 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 1 -- 172.21.15.63:0/2769347272 shutdown_connections 2024-05-27T08:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 1 -- 172.21.15.63:0/2769347272 wait complete. 2024-05-27T08:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.896+0000 7f29d0a64740 20 client.21363 trim_cache size 0 max 16384 2024-05-27T08:04:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.918+0000 7f29d0a64740 1 Processor -- start 2024-05-27T08:04:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.918+0000 7f29d0a64740 1 -- start start 2024-05-27T08:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.918+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 0x555d1db4aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.918+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db4cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.918+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 0x555d1db62290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.918+0000 7f29d0a64740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29c41b5d50 con 0x555d1d7752f0 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.918+0000 7f29d0a64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29c41b3620 con 0x555d1d79e1b0 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.918+0000 7f29d0a64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29c41b1490 con 0x555d1d77d4d0 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.919+0000 7f29c2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 0x555d1db4aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.919+0000 7f29c2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 0x555d1db4aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41460/0 (socket says 172.21.15.63:41460) 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.919+0000 7f29c2a00640 1 -- 172.21.15.63:0/372466223 learned_addr learned my addr 172.21.15.63:0/372466223 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.919+0000 7f29c3400640 1 --2- 172.21.15.63:0/372466223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 0x555d1db62290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.919+0000 7f29c2000640 1 --2- 172.21.15.63:0/372466223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db4cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.919+0000 7f29c2a00640 1 -- 172.21.15.63:0/372466223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 msgr2=0x555d1db4cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.919+0000 7f29c2a00640 1 --2- 172.21.15.63:0/372466223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db4cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.919+0000 7f29c2a00640 1 -- 172.21.15.63:0/372466223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 msgr2=0x555d1db62290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.919+0000 7f29c2a00640 1 --2- 172.21.15.63:0/372466223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 0x555d1db62290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.919+0000 7f29c2a00640 1 -- 172.21.15.63:0/372466223 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d1db41520 con 0x555d1d7752f0 2024-05-27T08:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.920+0000 7f29c2a00640 1 --2- 172.21.15.63:0/372466223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 0x555d1db4aff0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f29b40df980 tx=0x7f29b4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39880b2abf0af9be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.920+0000 7f29af400640 1 -- 172.21.15.63:0/372466223 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4014030 con 0x555d1d7752f0 2024-05-27T08:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.920+0000 7f29af400640 1 -- 172.21.15.63:0/372466223 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29b40a9460 con 0x555d1d7752f0 2024-05-27T08:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.920+0000 7f29af400640 1 -- 172.21.15.63:0/372466223 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4021040 con 0x555d1d7752f0 2024-05-27T08:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.921+0000 7f29d0a64740 1 -- 172.21.15.63:0/372466223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 msgr2=0x555d1db4aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.921+0000 7f29d0a64740 1 --2- 172.21.15.63:0/372466223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 0x555d1db4aff0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f29b40df980 tx=0x7f29b4005bb0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.921+0000 7f29d0a64740 1 -- 172.21.15.63:0/372466223 shutdown_connections 2024-05-27T08:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.921+0000 7f29d0a64740 1 --2- 172.21.15.63:0/372466223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d79e1b0 0x555d1db62290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.921+0000 7f29d0a64740 1 --2- 172.21.15.63:0/372466223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d77d4d0 0x555d1db4cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.921+0000 7f29d0a64740 1 --2- 172.21.15.63:0/372466223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d7752f0 0x555d1db4aff0 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.921+0000 7f29d0a64740 1 -- 172.21.15.63:0/372466223 >> 172.21.15.63:0/372466223 conn(0x555d1d792300 msgr2=0x555d1db45270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.921+0000 7f29d0a64740 1 -- 172.21.15.63:0/372466223 shutdown_connections 2024-05-27T08:04:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.921+0000 7f29d0a64740 1 -- 172.21.15.63:0/372466223 wait complete. 2024-05-27T08:04:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.922+0000 7f29d0a64740 1 -- start start 2024-05-27T08:04:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.922+0000 7f29d0a64740 1 Processor -- start 2024-05-27T08:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.922+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 0x555d1db532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d77d4d0 0x555d1db4f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29d0a64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d79e1b0 0x555d1db4fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29d0a64740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29c41b1ae0 con 0x555d1d77d4d0 2024-05-27T08:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29d0a64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29c41b2f50 con 0x555d1d7752f0 2024-05-27T08:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29d0a64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29c41b21b0 con 0x555d1d79e1b0 2024-05-27T08:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29d0a64740 10 client.0 _ll_register_callbacks cb 0x7f29b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29c2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d77d4d0 0x555d1db4f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29c2000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d77d4d0 0x555d1db4f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41474/0 (socket says 172.21.15.63:41474) 2024-05-27T08:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29c2000640 1 -- 172.21.15.63:0/316635389 learned_addr learned my addr 172.21.15.63:0/316635389 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29c2a00640 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 0x555d1db532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29c3400640 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d79e1b0 0x555d1db4fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29c2000640 1 -- 172.21.15.63:0/316635389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d79e1b0 msgr2=0x555d1db4fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29c2000640 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d79e1b0 0x555d1db4fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.923+0000 7f29c2000640 1 -- 172.21.15.63:0/316635389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 msgr2=0x555d1db532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.924+0000 7f29c2000640 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 0x555d1db532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.924+0000 7f29c2000640 1 -- 172.21.15.63:0/316635389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d1db50280 con 0x555d1d77d4d0 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.924+0000 7f29c2a00640 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 0x555d1db532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.924+0000 7f29c2000640 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d77d4d0 0x555d1db4f650 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f29b002a910 tx=0x7f29b0032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f9233b030d1f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.924+0000 7f29acc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.924+0000 7f29acc00640 1 -- 172.21.15.63:0/316635389 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0035040 con 0x555d1d77d4d0 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.924+0000 7f29acc00640 1 -- 172.21.15.63:0/316635389 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f29b0004310 con 0x555d1d77d4d0 2024-05-27T08:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.924+0000 7f29d0a64740 1 -- 172.21.15.63:0/316635389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b40a9460 con 0x555d1d77d4d0 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.925+0000 7f29acc00640 1 -- 172.21.15.63:0/316635389 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f29b0050050 con 0x555d1d77d4d0 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.925+0000 7f29d0a64740 10 client.21369 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.925+0000 7f29d0a64740 1 -- 172.21.15.63:0/316635389 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555d1dbc4010 con 0x555d1d77d4d0 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.925+0000 7f29acc00640 1 -- 172.21.15.63:0/316635389 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29b40a9460 con 0x555d1d77d4d0 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.925+0000 7f29acc00640 1 -- 172.21.15.63:0/316635389 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f29b0004a30 con 0x555d1d77d4d0 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.925+0000 7f29d0a64740 10 client.21369 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.925+0000 7f29d0a64740 10 client.21369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.925+0000 7f29d0a64740 20 client.21369 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.926+0000 7f29d0a64740 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555d1d7321c0 0x555d1dbc6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.926+0000 7f29d0a64740 4 client.21369 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.926+0000 7f29d0a64740 1 -- 172.21.15.63:0/316635389 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d1db45060 con 0x555d1d7321c0 2024-05-27T08:04:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.926+0000 7f29c2a00640 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555d1d7321c0 0x555d1dbc6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.927+0000 7f29c2a00640 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555d1d7321c0 0x555d1dbc6850 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93bf9ed8c3af58d3 server_cookie=29f484b6c0986e96 in_seq=0 out_seq=0 2024-05-27T08:04:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.927+0000 7f29acc00640 10 client.21369 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.985+0000 7f29acc00640 1 -- 172.21.15.63:0/316635389 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555d1db45060 con 0x555d1d7321c0 2024-05-27T08:04:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.985+0000 7f29acc00640 10 client.21369 handle_command_reply: tid=0 2024-05-27T08:04:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.985+0000 7f29d0a64740 1 client.21369 shutdown 2024-05-27T08:04:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 1 -- 172.21.15.63:0/316635389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d77d4d0 msgr2=0x555d1db4f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d77d4d0 0x555d1db4f650 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f29b002a910 tx=0x7f29b0032ec0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 1 -- 172.21.15.63:0/316635389 shutdown_connections 2024-05-27T08:04:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555d1d7321c0 0x555d1dbc6850 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555d1d79e1b0 0x555d1db4fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d1d77d4d0 0x555d1db4f650 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29acc00640 0 client.21369 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 1 --2- 172.21.15.63:0/316635389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d1d7752f0 0x555d1db532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 1 -- 172.21.15.63:0/316635389 >> 172.21.15.63:0/316635389 conn(0x555d1d792300 msgr2=0x555d1db4c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 1 -- 172.21.15.63:0/316635389 shutdown_connections 2024-05-27T08:04:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 1 -- 172.21.15.63:0/316635389 wait complete. 2024-05-27T08:04:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.986+0000 7f29d0a64740 20 client.21369 trim_cache size 0 max 16384 2024-05-27T08:04:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:04:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:04:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:04:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 -- 172.21.15.63:0/2658345096 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a80b2870 msgr2=0x7f29a80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a80b2870 0x7f29a80b4d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00a010 tx=0x7f29bc005fd0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 -- 172.21.15.63:0/2658345096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c41065f0 msgr2=0x7f29c41a25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c41065f0 0x7f29c41a25d0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f29b400d770 tx=0x7f29b400dc30 comp rx=0 tx=0).stop 2024-05-27T08:04:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 -- 172.21.15.63:0/2658345096 shutdown_connections 2024-05-27T08:04:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29a80b2870 0x7f29a80b4d80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29c410c340 0x7f29c419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29c410bea0 0x7f29c41a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 --2- 172.21.15.63:0/2658345096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29c41065f0 0x7f29c41a25d0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 -- 172.21.15.63:0/2658345096 >> 172.21.15.63:0/2658345096 conn(0x7f29c4077140 msgr2=0x7f29c4107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 -- 172.21.15.63:0/2658345096 shutdown_connections 2024-05-27T08:04:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:26.988+0000 7f29c9a00640 1 -- 172.21.15.63:0/2658345096 wait complete. 2024-05-27T08:04:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:27 smithi086 ceph-mon[34687]: pgmap v980: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.4 MiB/s wr, 51 op/s 2024-05-27T08:04:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:27 smithi149 ceph-mon[35408]: pgmap v980: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.4 MiB/s wr, 51 op/s 2024-05-27T08:04:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:27 smithi063 ceph-mon[31613]: pgmap v980: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.4 MiB/s wr, 51 op/s 2024-05-27T08:04:29.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:29 smithi086 ceph-mon[34687]: pgmap v981: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 24 op/s 2024-05-27T08:04:29.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:29 smithi149 ceph-mon[35408]: pgmap v981: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 24 op/s 2024-05-27T08:04:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:29 smithi063 ceph-mon[31613]: pgmap v981: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.7 MiB/s wr, 24 op/s 2024-05-27T08:04:31.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:31 smithi086 ceph-mon[34687]: pgmap v982: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T08:04:31.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:31 smithi149 ceph-mon[35408]: pgmap v982: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T08:04:31.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:31 smithi063 ceph-mon[31613]: pgmap v982: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.3 MiB/s wr, 7 op/s 2024-05-27T08:04:33.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:33 smithi086 ceph-mon[34687]: pgmap v983: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:04:33.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:33 smithi149 ceph-mon[35408]: pgmap v983: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:04:33.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:33 smithi063 ceph-mon[31613]: pgmap v983: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:04:35.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:35 smithi086 ceph-mon[34687]: pgmap v984: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:04:35.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:35 smithi149 ceph-mon[35408]: pgmap v984: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:04:35.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:35 smithi063 ceph-mon[31613]: pgmap v984: 129 pgs: 129 active+clean; 9.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:04:37.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:37 smithi086 ceph-mon[34687]: pgmap v985: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:04:37.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:37 smithi149 ceph-mon[35408]: pgmap v985: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:04:37.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:37 smithi063 ceph-mon[31613]: pgmap v985: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:04:39.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:39 smithi086 ceph-mon[34687]: pgmap v986: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:04:39.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:39 smithi149 ceph-mon[35408]: pgmap v986: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:04:39.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:39 smithi063 ceph-mon[31613]: pgmap v986: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:04:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.598+0000 7fc8dc800640 1 -- 172.21.15.63:0/1447017117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8d4109020 msgr2=0x7fc8d4102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.598+0000 7fc8dc800640 1 --2- 172.21.15.63:0/1447017117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8d4109020 0x7fc8d4102c60 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009900 tx=0x7fc8c402a410 comp rx=0 tx=0).stop 2024-05-27T08:04:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.598+0000 7fc8dc800640 1 -- 172.21.15.63:0/1447017117 shutdown_connections 2024-05-27T08:04:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.598+0000 7fc8dc800640 1 --2- 172.21.15.63:0/1447017117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8d41032d0 0x7fc8d410f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.598+0000 7fc8dc800640 1 --2- 172.21.15.63:0/1447017117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8d4109020 0x7fc8d4102c60 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.598+0000 7fc8dc800640 1 --2- 172.21.15.63:0/1447017117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d4108b80 0x7fc8d4102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.598+0000 7fc8dc800640 1 -- 172.21.15.63:0/1447017117 >> 172.21.15.63:0/1447017117 conn(0x7fc8d40fe380 msgr2=0x7fc8d41007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.598+0000 7fc8dc800640 1 -- 172.21.15.63:0/1447017117 shutdown_connections 2024-05-27T08:04:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.598+0000 7fc8dc800640 1 -- 172.21.15.63:0/1447017117 wait complete. 2024-05-27T08:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.600+0000 7fc8dc800640 1 Processor -- start 2024-05-27T08:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.600+0000 7fc8dc800640 1 -- start start 2024-05-27T08:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.600+0000 7fc8dc800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d41032d0 0x7fc8d41a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.600+0000 7fc8dc800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8d4108b80 0x7fc8d41a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.600+0000 7fc8dc800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8d4109020 0x7fc8d419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.600+0000 7fc8dc800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d4112c20 con 0x7fc8d4108b80 2024-05-27T08:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.600+0000 7fc8dc800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d4112aa0 con 0x7fc8d41032d0 2024-05-27T08:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.600+0000 7fc8dc800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d4112da0 con 0x7fc8d4109020 2024-05-27T08:04:40.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.601+0000 7fc8d9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8d4108b80 0x7fc8d41a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.601+0000 7fc8d9400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8d4108b80 0x7fc8d41a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50116/0 (socket says 172.21.15.63:50116) 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.601+0000 7fc8d9400640 1 -- 172.21.15.63:0/2864826040 learned_addr learned my addr 172.21.15.63:0/2864826040 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.601+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d41032d0 0x7fc8d41a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.601+0000 7fc8da800640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8d4109020 0x7fc8d419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.601+0000 7fc8d9400640 1 -- 172.21.15.63:0/2864826040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8d4109020 msgr2=0x7fc8d419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.601+0000 7fc8d9400640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8d4109020 0x7fc8d419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.601+0000 7fc8d9400640 1 -- 172.21.15.63:0/2864826040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d41032d0 msgr2=0x7fc8d41a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.601+0000 7fc8d9400640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d41032d0 0x7fc8d41a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.602+0000 7fc8d9400640 1 -- 172.21.15.63:0/2864826040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d419ad80 con 0x7fc8d4108b80 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.602+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d41032d0 0x7fc8d41a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.602+0000 7fc8da800640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8d4109020 0x7fc8d419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.602+0000 7fc8d9400640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8d4108b80 0x7fc8d41a0a10 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4005940 tx=0x7fc8c4005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.602+0000 7fc8cea00640 1 -- 172.21.15.63:0/2864826040 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c403f030 con 0x7fc8d4108b80 2024-05-27T08:04:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.602+0000 7fc8dc800640 1 -- 172.21.15.63:0/2864826040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d419afb0 con 0x7fc8d4108b80 2024-05-27T08:04:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.602+0000 7fc8cea00640 1 -- 172.21.15.63:0/2864826040 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8c40043d0 con 0x7fc8d4108b80 2024-05-27T08:04:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.603+0000 7fc8cea00640 1 -- 172.21.15.63:0/2864826040 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4035070 con 0x7fc8d4108b80 2024-05-27T08:04:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.603+0000 7fc8dc800640 1 -- 172.21.15.63:0/2864826040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d41a7270 con 0x7fc8d4108b80 2024-05-27T08:04:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.607+0000 7fc8cea00640 1 -- 172.21.15.63:0/2864826040 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc8c4042030 con 0x7fc8d4108b80 2024-05-27T08:04:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.608+0000 7fc8cea00640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8bc0b29d0 0x7fc8bc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.608+0000 7fc8cea00640 1 -- 172.21.15.63:0/2864826040 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fc8c4102e10 con 0x7fc8d4108b80 2024-05-27T08:04:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.608+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8bc0b29d0 0x7fc8bc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.609+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8bc0b29d0 0x7fc8bc0b4ee0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4103860 tx=0x7fc8c8005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.624+0000 7fc8e379b740 1 Processor -- start 2024-05-27T08:04:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.624+0000 7fc8e379b740 1 -- start start 2024-05-27T08:04:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.624+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef41829c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.624+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef4145a2f0 0x55ef4182a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.624+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 0x55ef4182a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.624+0000 7fc8e379b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ef41796b90 con 0x55ef414831b0 2024-05-27T08:04:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.624+0000 7fc8e379b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef41786730 con 0x55ef414624d0 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.624+0000 7fc8e379b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ef4178a770 con 0x55ef4145a2f0 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.625+0000 7fc8d9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef41829c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.625+0000 7fc8d9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef41829c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50128/0 (socket says 172.21.15.63:50128) 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.625+0000 7fc8d9e00640 1 -- 172.21.15.63:0/2728571229 learned_addr learned my addr 172.21.15.63:0/2728571229 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.625+0000 7fc8da800640 1 --2- 172.21.15.63:0/2728571229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 0x55ef4182a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.625+0000 7fc8d9400640 1 --2- 172.21.15.63:0/2728571229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef4145a2f0 0x55ef4182a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.625+0000 7fc8d9e00640 1 -- 172.21.15.63:0/2728571229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef4145a2f0 msgr2=0x55ef4182a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.625+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/2728571229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef4145a2f0 0x55ef4182a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.625+0000 7fc8d9e00640 1 -- 172.21.15.63:0/2728571229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 msgr2=0x55ef4182a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.625+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/2728571229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 0x55ef4182a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.625+0000 7fc8d9e00640 1 -- 172.21.15.63:0/2728571229 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef4171f8b0 con 0x55ef414831b0 2024-05-27T08:04:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.626+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/2728571229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef41829c70 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8004030 tx=0x7fc8c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafa25c92da827cb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.626+0000 7fc8c3000640 1 -- 172.21.15.63:0/2728571229 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8003d70 con 0x55ef414831b0 2024-05-27T08:04:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.626+0000 7fc8c3000640 1 -- 172.21.15.63:0/2728571229 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8c8025070 con 0x55ef414831b0 2024-05-27T08:04:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.626+0000 7fc8c3000640 1 -- 172.21.15.63:0/2728571229 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8020690 con 0x55ef414831b0 2024-05-27T08:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.627+0000 7fc8e379b740 1 -- 172.21.15.63:0/2728571229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 msgr2=0x55ef41829c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.627+0000 7fc8e379b740 1 --2- 172.21.15.63:0/2728571229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef41829c70 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8004030 tx=0x7fc8c8014000 comp rx=0 tx=0).stop 2024-05-27T08:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.628+0000 7fc8e379b740 1 -- 172.21.15.63:0/2728571229 shutdown_connections 2024-05-27T08:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.628+0000 7fc8e379b740 1 --2- 172.21.15.63:0/2728571229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 0x55ef4182a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.628+0000 7fc8e379b740 1 --2- 172.21.15.63:0/2728571229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef4145a2f0 0x55ef4182a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.628+0000 7fc8e379b740 1 --2- 172.21.15.63:0/2728571229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef41829c70 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.628+0000 7fc8e379b740 1 -- 172.21.15.63:0/2728571229 >> 172.21.15.63:0/2728571229 conn(0x55ef41477300 msgr2=0x55ef41826420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.628+0000 7fc8e379b740 1 -- 172.21.15.63:0/2728571229 shutdown_connections 2024-05-27T08:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.628+0000 7fc8e379b740 1 -- 172.21.15.63:0/2728571229 wait complete. 2024-05-27T08:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.629+0000 7fc8e379b740 1 -- start start 2024-05-27T08:04:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.629+0000 7fc8e379b740 1 Processor -- start 2024-05-27T08:04:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef4145a2f0 0x55ef41833d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef414624d0 0x55ef41834260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414831b0 0x55ef418347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8e379b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ef4146c790 con 0x55ef4145a2f0 2024-05-27T08:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8e379b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef4172ebc0 con 0x55ef414831b0 2024-05-27T08:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8e379b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ef41836da0 con 0x55ef414624d0 2024-05-27T08:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8e379b740 10 client.0 _ll_register_callbacks cb 0x55ef4170c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8d9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef4145a2f0 0x55ef41833d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8d9e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef4145a2f0 0x55ef41833d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50142/0 (socket says 172.21.15.63:50142) 2024-05-27T08:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8d9e00640 1 -- 172.21.15.63:0/711687974 learned_addr learned my addr 172.21.15.63:0/711687974 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8d9400640 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef414624d0 0x55ef41834260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.630+0000 7fc8da800640 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414831b0 0x55ef418347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.631+0000 7fc8d9e00640 1 -- 172.21.15.63:0/711687974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef414624d0 msgr2=0x55ef41834260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.631+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef414624d0 0x55ef41834260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.631+0000 7fc8d9e00640 1 -- 172.21.15.63:0/711687974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414831b0 msgr2=0x55ef418347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.631+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414831b0 0x55ef418347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.631+0000 7fc8d9e00640 1 -- 172.21.15.63:0/711687974 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef4149b210 con 0x55ef4145a2f0 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.631+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef4145a2f0 0x55ef41833d20 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800a010 tx=0x7fc8c80124d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee48bdd01a33848 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.631+0000 7fc8b7400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.632+0000 7fc8b7400640 1 -- 172.21.15.63:0/711687974 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8012ad0 con 0x55ef4145a2f0 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.632+0000 7fc8b7400640 1 -- 172.21.15.63:0/711687974 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fc8c802a480 con 0x55ef4145a2f0 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.632+0000 7fc8e379b740 1 -- 172.21.15.63:0/711687974 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8c8025070 con 0x55ef4145a2f0 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.632+0000 7fc8b7400640 1 -- 172.21.15.63:0/711687974 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc8c803e050 con 0x55ef4145a2f0 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.632+0000 7fc8e379b740 10 client.21381 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.632+0000 7fc8e379b740 1 -- 172.21.15.63:0/711687974 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ef41567620 con 0x55ef4145a2f0 2024-05-27T08:04:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.633+0000 7fc8b7400640 1 -- 172.21.15.63:0/711687974 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8c8025070 con 0x55ef4145a2f0 2024-05-27T08:04:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.633+0000 7fc8b7400640 1 -- 172.21.15.63:0/711687974 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc8c8033020 con 0x55ef4145a2f0 2024-05-27T08:04:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.633+0000 7fc8e379b740 10 client.21381 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.633+0000 7fc8e379b740 10 client.21381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.633+0000 7fc8e379b740 20 client.21381 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.633+0000 7fc8e379b740 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef41441d80 0x55ef41845f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.633+0000 7fc8e379b740 4 client.21381 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.633+0000 7fc8e379b740 1 -- 172.21.15.63:0/711687974 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef41798ae0 con 0x55ef41441d80 2024-05-27T08:04:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.634+0000 7fc8d9400640 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef41441d80 0x55ef41845f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.635+0000 7fc8d9400640 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef41441d80 0x55ef41845f40 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af4fd4ce87165a0 server_cookie=684fe14f876482b9 in_seq=0 out_seq=0 2024-05-27T08:04:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.635+0000 7fc8b7400640 10 client.21381 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8b7400640 1 -- 172.21.15.63:0/711687974 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ef41798ae0 con 0x55ef41441d80 2024-05-27T08:04:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8b7400640 10 client.21381 handle_command_reply: tid=0 2024-05-27T08:04:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8e379b740 1 client.21381 shutdown 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8e379b740 1 -- 172.21.15.63:0/711687974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef4145a2f0 msgr2=0x55ef41833d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8e379b740 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef4145a2f0 0x55ef41833d20 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800a010 tx=0x7fc8c80124d0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8e379b740 1 -- 172.21.15.63:0/711687974 shutdown_connections 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8e379b740 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef41441d80 0x55ef41845f40 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8e379b740 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414831b0 0x55ef418347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8e379b740 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef414624d0 0x55ef41834260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8e379b740 1 --2- 172.21.15.63:0/711687974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef4145a2f0 0x55ef41833d20 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8b7400640 0 client.21381 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.637+0000 7fc8e379b740 1 -- 172.21.15.63:0/711687974 >> 172.21.15.63:0/711687974 conn(0x55ef41477300 msgr2=0x55ef41829aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.638+0000 7fc8e379b740 1 -- 172.21.15.63:0/711687974 shutdown_connections 2024-05-27T08:04:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.638+0000 7fc8e379b740 1 -- 172.21.15.63:0/711687974 wait complete. 2024-05-27T08:04:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.638+0000 7fc8e379b740 20 client.21381 trim_cache size 0 max 16384 2024-05-27T08:04:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.658+0000 7fc8e379b740 1 Processor -- start 2024-05-27T08:04:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.658+0000 7fc8e379b740 1 -- start start 2024-05-27T08:04:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.659+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef4145a2f0 0x55ef41836450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.659+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 0x55ef418345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.659+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef4183f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.659+0000 7fc8e379b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41b1950 con 0x55ef414831b0 2024-05-27T08:04:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.659+0000 7fc8e379b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41af220 con 0x55ef414624d0 2024-05-27T08:04:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.659+0000 7fc8e379b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41ad090 con 0x55ef4145a2f0 2024-05-27T08:04:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.659+0000 7fc8da800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef4183f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.659+0000 7fc8da800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef4183f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50152/0 (socket says 172.21.15.63:50152) 2024-05-27T08:04:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.659+0000 7fc8da800640 1 -- 172.21.15.63:0/1304302488 learned_addr learned my addr 172.21.15.63:0/1304302488 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.660+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/1304302488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef4145a2f0 0x55ef41836450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.660+0000 7fc8d9400640 1 --2- 172.21.15.63:0/1304302488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 0x55ef418345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.660+0000 7fc8da800640 1 -- 172.21.15.63:0/1304302488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef4145a2f0 msgr2=0x55ef41836450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.660+0000 7fc8da800640 1 --2- 172.21.15.63:0/1304302488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef4145a2f0 0x55ef41836450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.660+0000 7fc8da800640 1 -- 172.21.15.63:0/1304302488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 msgr2=0x55ef418345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.660+0000 7fc8da800640 1 --2- 172.21.15.63:0/1304302488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 0x55ef418345c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.660+0000 7fc8da800640 1 -- 172.21.15.63:0/1304302488 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef41829600 con 0x55ef414831b0 2024-05-27T08:04:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.660+0000 7fc8d9400640 1 --2- 172.21.15.63:0/1304302488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 0x55ef418345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.661+0000 7fc8da800640 1 --2- 172.21.15.63:0/1304302488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef4183f540 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0012610 tx=0x7fc8d0012ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db134a2cb28be937 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.661+0000 7fc8c2600640 1 -- 172.21.15.63:0/1304302488 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0010550 con 0x55ef414831b0 2024-05-27T08:04:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.661+0000 7fc8c2600640 1 -- 172.21.15.63:0/1304302488 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8d0010b80 con 0x55ef414831b0 2024-05-27T08:04:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.661+0000 7fc8c2600640 1 -- 172.21.15.63:0/1304302488 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d001bd00 con 0x55ef414831b0 2024-05-27T08:04:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.662+0000 7fc8e379b740 1 -- 172.21.15.63:0/1304302488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 msgr2=0x55ef4183f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.662+0000 7fc8e379b740 1 --2- 172.21.15.63:0/1304302488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef4183f540 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0012610 tx=0x7fc8d0012ad0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.662+0000 7fc8e379b740 1 -- 172.21.15.63:0/1304302488 shutdown_connections 2024-05-27T08:04:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.662+0000 7fc8e379b740 1 --2- 172.21.15.63:0/1304302488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef4183f540 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.662+0000 7fc8e379b740 1 --2- 172.21.15.63:0/1304302488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef414624d0 0x55ef418345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.662+0000 7fc8e379b740 1 --2- 172.21.15.63:0/1304302488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef4145a2f0 0x55ef41836450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.662+0000 7fc8e379b740 1 -- 172.21.15.63:0/1304302488 >> 172.21.15.63:0/1304302488 conn(0x55ef41477300 msgr2=0x55ef4183ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.662+0000 7fc8e379b740 1 -- 172.21.15.63:0/1304302488 shutdown_connections 2024-05-27T08:04:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.662+0000 7fc8e379b740 1 -- 172.21.15.63:0/1304302488 wait complete. 2024-05-27T08:04:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.663+0000 7fc8e379b740 1 -- start start 2024-05-27T08:04:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.663+0000 7fc8e379b740 1 Processor -- start 2024-05-27T08:04:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef4145a2f0 0x55ef418375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef414624d0 0x55ef41837b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8e379b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef418a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8e379b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41ad6e0 con 0x55ef414831b0 2024-05-27T08:04:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8e379b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41aeb50 con 0x55ef4145a2f0 2024-05-27T08:04:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8e379b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41addb0 con 0x55ef414624d0 2024-05-27T08:04:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8e379b740 10 client.0 _ll_register_callbacks cb 0x7fc8c80056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8da800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef418a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8da800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef418a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50162/0 (socket says 172.21.15.63:50162) 2024-05-27T08:04:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8da800640 1 -- 172.21.15.63:0/2611435730 learned_addr learned my addr 172.21.15.63:0/2611435730 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.664+0000 7fc8d9400640 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef414624d0 0x55ef41837b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.665+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef4145a2f0 0x55ef418375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.665+0000 7fc8da800640 1 -- 172.21.15.63:0/2611435730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef414624d0 msgr2=0x55ef41837b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.665+0000 7fc8da800640 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef414624d0 0x55ef41837b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.665+0000 7fc8da800640 1 -- 172.21.15.63:0/2611435730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef4145a2f0 msgr2=0x55ef418375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.665+0000 7fc8da800640 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef4145a2f0 0x55ef418375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.665+0000 7fc8da800640 1 -- 172.21.15.63:0/2611435730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef418a6ab0 con 0x55ef414831b0 2024-05-27T08:04:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.665+0000 7fc8da800640 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef418a64f0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fc8d001b7a0 tx=0x7fc8d001b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c2c013089898cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.665+0000 7fc8b7400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.665+0000 7fc8b7400640 1 -- 172.21.15.63:0/2611435730 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0029030 con 0x55ef414831b0 2024-05-27T08:04:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.666+0000 7fc8e379b740 1 -- 172.21.15.63:0/2611435730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d0010b80 con 0x55ef414831b0 2024-05-27T08:04:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.666+0000 7fc8b7400640 1 -- 172.21.15.63:0/2611435730 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fc8d001ba00 con 0x55ef414831b0 2024-05-27T08:04:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.666+0000 7fc8b7400640 1 -- 172.21.15.63:0/2611435730 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc8d001bc00 con 0x55ef414831b0 2024-05-27T08:04:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.666+0000 7fc8e379b740 10 client.21387 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.666+0000 7fc8e379b740 1 -- 172.21.15.63:0/2611435730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ef418addf0 con 0x55ef414831b0 2024-05-27T08:04:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.666+0000 7fc8b7400640 1 -- 172.21.15.63:0/2611435730 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8d0010b80 con 0x55ef414831b0 2024-05-27T08:04:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.667+0000 7fc8b7400640 1 -- 172.21.15.63:0/2611435730 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc8d001a070 con 0x55ef414831b0 2024-05-27T08:04:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.667+0000 7fc8e379b740 10 client.21387 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.667+0000 7fc8e379b740 10 client.21387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.667+0000 7fc8e379b740 20 client.21387 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.667+0000 7fc8e379b740 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef414171c0 0x55ef418b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.667+0000 7fc8e379b740 4 client.21387 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:04:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.667+0000 7fc8e379b740 1 -- 172.21.15.63:0/2611435730 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef417922c0 con 0x55ef414171c0 2024-05-27T08:04:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.667+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef414171c0 0x55ef418b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.668+0000 7fc8d9e00640 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef414171c0 0x55ef418b0570 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21190ac7e1c1d96a server_cookie=7788ae8cfed65e9d in_seq=0 out_seq=0 2024-05-27T08:04:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.668+0000 7fc8b7400640 10 client.21387 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.701+0000 7fc8b7400640 1 -- 172.21.15.63:0/2611435730 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef417922c0 con 0x55ef414171c0 2024-05-27T08:04:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.701+0000 7fc8b7400640 10 client.21387 handle_command_reply: tid=0 2024-05-27T08:04:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.701+0000 7fc8e379b740 1 client.21387 shutdown 2024-05-27T08:04:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 1 -- 172.21.15.63:0/2611435730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 msgr2=0x55ef418a64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef418a64f0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fc8d001b7a0 tx=0x7fc8d001b3f0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 1 -- 172.21.15.63:0/2611435730 shutdown_connections 2024-05-27T08:04:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ef414171c0 0x55ef418b0570 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ef414831b0 0x55ef418a64f0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef414624d0 0x55ef41837b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 1 --2- 172.21.15.63:0/2611435730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef4145a2f0 0x55ef418375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 1 -- 172.21.15.63:0/2611435730 >> 172.21.15.63:0/2611435730 conn(0x55ef41477300 msgr2=0x55ef4183f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8b7400640 0 client.21387 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 1 -- 172.21.15.63:0/2611435730 shutdown_connections 2024-05-27T08:04:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 1 -- 172.21.15.63:0/2611435730 wait complete. 2024-05-27T08:04:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.702+0000 7fc8e379b740 20 client.21387 trim_cache size 0 max 16384 2024-05-27T08:04:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:04:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:04:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:04:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.703+0000 7fc8dc800640 1 -- 172.21.15.63:0/2864826040 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8bc0b29d0 msgr2=0x7fc8bc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.703+0000 7fc8dc800640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8bc0b29d0 0x7fc8bc0b4ee0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4103860 tx=0x7fc8c8005f90 comp rx=0 tx=0).stop 2024-05-27T08:04:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.703+0000 7fc8dc800640 1 -- 172.21.15.63:0/2864826040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8d4108b80 msgr2=0x7fc8d41a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.703+0000 7fc8dc800640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8d4108b80 0x7fc8d41a0a10 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4005940 tx=0x7fc8c4005e10 comp rx=0 tx=0).stop 2024-05-27T08:04:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.703+0000 7fc8dc800640 1 -- 172.21.15.63:0/2864826040 shutdown_connections 2024-05-27T08:04:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.703+0000 7fc8dc800640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8bc0b29d0 0x7fc8bc0b4ee0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.703+0000 7fc8dc800640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8d4109020 0x7fc8d419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.703+0000 7fc8dc800640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8d4108b80 0x7fc8d41a0a10 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.703+0000 7fc8dc800640 1 --2- 172.21.15.63:0/2864826040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d41032d0 0x7fc8d41a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.703+0000 7fc8dc800640 1 -- 172.21.15.63:0/2864826040 >> 172.21.15.63:0/2864826040 conn(0x7fc8d40fe380 msgr2=0x7fc8d40753d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.704+0000 7fc8dc800640 1 -- 172.21.15.63:0/2864826040 shutdown_connections 2024-05-27T08:04:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:40.704+0000 7fc8dc800640 1 -- 172.21.15.63:0/2864826040 wait complete. 2024-05-27T08:04:41.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:41 smithi086 ceph-mon[34687]: pgmap v987: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:04:41.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:41 smithi149 ceph-mon[35408]: pgmap v987: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:04:41.727 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:41 smithi063 ceph-mon[31613]: pgmap v987: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:04:43.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:43 smithi149 ceph-mon[35408]: pgmap v988: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.0 MiB/s wr, 17 op/s 2024-05-27T08:04:43.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:43 smithi063 ceph-mon[31613]: pgmap v988: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.0 MiB/s wr, 17 op/s 2024-05-27T08:04:43.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:43 smithi086 ceph-mon[34687]: pgmap v988: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.0 MiB/s wr, 17 op/s 2024-05-27T08:04:45.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:45 smithi086 ceph-mon[34687]: pgmap v989: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:04:45.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:45 smithi149 ceph-mon[35408]: pgmap v989: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:04:45.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:45 smithi063 ceph-mon[31613]: pgmap v989: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:04:47.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:47 smithi086 ceph-mon[34687]: pgmap v990: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 12 op/s 2024-05-27T08:04:47.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:47 smithi149 ceph-mon[35408]: pgmap v990: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 12 op/s 2024-05-27T08:04:47.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:47 smithi063 ceph-mon[31613]: pgmap v990: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 12 op/s 2024-05-27T08:04:48.581 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:04:49.479 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:49.837 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:49 smithi063 ceph-mon[31613]: pgmap v991: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:04:49.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:49 smithi086 ceph-mon[34687]: pgmap v991: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:04:49.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:49 smithi149 ceph-mon[35408]: pgmap v991: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-05-27T08:04:51.820 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:51 smithi063 ceph-mon[31613]: pgmap v992: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:04:51.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:51 smithi086 ceph-mon[34687]: pgmap v992: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:04:51.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:51 smithi149 ceph-mon[35408]: pgmap v992: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:04:52.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.400+0000 7fed0b000700 1 -- 172.21.15.63:0/571949203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed04105bd0 msgr2=0x7fed0410e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:52.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.400+0000 7fed0b000700 1 --2- 172.21.15.63:0/571949203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed04105bd0 0x7fed0410e120 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009990 tx=0x7fecf0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.400+0000 7fed0b000700 1 -- 172.21.15.63:0/571949203 shutdown_connections 2024-05-27T08:04:52.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.400+0000 7fed0b000700 1 --2- 172.21.15.63:0/571949203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed0410e6f0 0x7fed04110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.400+0000 7fed0b000700 1 --2- 172.21.15.63:0/571949203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed04105280 0x7fed04105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.400+0000 7fed0b000700 1 --2- 172.21.15.63:0/571949203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed04105bd0 0x7fed0410e120 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.400+0000 7fed0b000700 1 -- 172.21.15.63:0/571949203 >> 172.21.15.63:0/571949203 conn(0x7fed04078ab0 msgr2=0x7fed04078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:52.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.400+0000 7fed0b000700 1 -- 172.21.15.63:0/571949203 shutdown_connections 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.401+0000 7fed0b000700 1 -- 172.21.15.63:0/571949203 wait complete. 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.402+0000 7fed0b000700 1 Processor -- start 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.402+0000 7fed0b000700 1 -- start start 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.403+0000 7fed0b000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed04105280 0x7fed0419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.403+0000 7fed0b000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed04105bd0 0x7fed0419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.403+0000 7fed0b000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0410e6f0 0x7fed041a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.403+0000 7fed0b000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed04114120 con 0x7fed04105bd0 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.403+0000 7fed0b000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed04113fa0 con 0x7fed04105280 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.403+0000 7fed0b000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed041142a0 con 0x7fed0410e6f0 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.403+0000 7fed09200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed04105bd0 0x7fed0419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.403+0000 7fed09200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed04105bd0 0x7fed0419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57574/0 (socket says 172.21.15.63:57574) 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.403+0000 7fed09200700 1 -- 172.21.15.63:0/2697141209 learned_addr learned my addr 172.21.15.63:0/2697141209 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:52.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.404+0000 7fed0a600700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0410e6f0 0x7fed041a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.404+0000 7fed09c00700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed04105280 0x7fed0419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.404+0000 7fed09200700 1 -- 172.21.15.63:0/2697141209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0410e6f0 msgr2=0x7fed041a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.404+0000 7fed09200700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0410e6f0 0x7fed041a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.404+0000 7fed09200700 1 -- 172.21.15.63:0/2697141209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed04105280 msgr2=0x7fed0419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.404+0000 7fed09200700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed04105280 0x7fed0419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.404+0000 7fed09200700 1 -- 172.21.15.63:0/2697141209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed041a1d60 con 0x7fed04105bd0 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.404+0000 7fed09c00700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed04105280 0x7fed0419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.404+0000 7fed09200700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed04105bd0 0x7fed0419d430 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fecf0006010 tx=0x7fecf0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.405+0000 7fed02000700 1 -- 172.21.15.63:0/2697141209 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0027070 con 0x7fed04105bd0 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.405+0000 7fed02000700 1 -- 172.21.15.63:0/2697141209 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecf000fb90 con 0x7fed04105bd0 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.405+0000 7fed0b000700 1 -- 172.21.15.63:0/2697141209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed041a1ff0 con 0x7fed04105bd0 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.405+0000 7fed02000700 1 -- 172.21.15.63:0/2697141209 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf00173f0 con 0x7fed04105bd0 2024-05-27T08:04:52.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.405+0000 7fed0b000700 1 -- 172.21.15.63:0/2697141209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041a24d0 con 0x7fed04105bd0 2024-05-27T08:04:52.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.410+0000 7fed02000700 1 -- 172.21.15.63:0/2697141209 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fecf002a070 con 0x7fed04105bd0 2024-05-27T08:04:52.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.412+0000 7fed1b06cb80 1 Processor -- start 2024-05-27T08:04:52.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.412+0000 7fed1b06cb80 1 -- start start 2024-05-27T08:04:52.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.412+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e950142e40 0x55e9502c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.412+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 0x55e9502cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.412+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9502c1820 0x55e9502ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.412+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e9501efe40 con 0x55e950142e40 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.412+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e9501df9e0 con 0x55e9502c1820 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.412+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e9501e3a20 con 0x55e9502c1410 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.413+0000 7fed02000700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fecec0b29f0 0x7fecec0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.413+0000 7fed09200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9502c1820 0x55e9502ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.413+0000 7fed09c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 0x55e9502cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.413+0000 7fed0a600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e950142e40 0x55e9502c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.413+0000 7fed09c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 0x55e9502cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:42692/0 (socket says 172.21.15.63:42692) 2024-05-27T08:04:52.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.413+0000 7fed09c00700 1 -- 172.21.15.63:0/3221910317 learned_addr learned my addr 172.21.15.63:0/3221910317 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.413+0000 7fed02000700 1 -- 172.21.15.63:0/2697141209 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fecf00e3b10 con 0x7fed04105bd0 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.414+0000 7fed0a600700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fecec0b29f0 0x7fecec0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.414+0000 7fed09c00700 1 -- 172.21.15.63:0/3221910317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9502c1820 msgr2=0x55e9502ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.414+0000 7fed09c00700 1 --2- 172.21.15.63:0/3221910317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9502c1820 0x55e9502ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.414+0000 7fed09c00700 1 -- 172.21.15.63:0/3221910317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e950142e40 msgr2=0x55e9502c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.414+0000 7fed09c00700 1 --2- 172.21.15.63:0/3221910317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e950142e40 0x55e9502c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.414+0000 7fed09c00700 1 -- 172.21.15.63:0/3221910317 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9501a8420 con 0x55e9502c1410 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.414+0000 7fed0a600700 1 --2- 172.21.15.63:0/3221910317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e950142e40 0x55e9502c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.415+0000 7fed09c00700 1 --2- 172.21.15.63:0/3221910317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 0x55e9502cbad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009dd0 tx=0x7fecf80116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd93a79a9bb3c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.415+0000 7fed0a600700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fecec0b29f0 0x7fecec0b4f00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55e9502bd690 tx=0x7fecfc00b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.415+0000 7fecf6a00700 1 -- 172.21.15.63:0/3221910317 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8011e20 con 0x55e9502c1410 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.415+0000 7fecf6a00700 1 -- 172.21.15.63:0/3221910317 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecf80128e0 con 0x55e9502c1410 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.415+0000 7fecf6a00700 1 -- 172.21.15.63:0/3221910317 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf800f730 con 0x55e9502c1410 2024-05-27T08:04:52.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.416+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3221910317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 msgr2=0x55e9502cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.417+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/3221910317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 0x55e9502cbad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009dd0 tx=0x7fecf80116c0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.417+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3221910317 shutdown_connections 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.417+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/3221910317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9502c1820 0x55e9502ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.417+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/3221910317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e950142e40 0x55e9502c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.417+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/3221910317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 0x55e9502cbad0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.417+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3221910317 >> 172.21.15.63:0/3221910317 conn(0x55e9501a2fc0 msgr2=0x55e9502bcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.417+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3221910317 shutdown_connections 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.417+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3221910317 wait complete. 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.418+0000 7fed1b06cb80 1 -- start start 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.418+0000 7fed1b06cb80 1 Processor -- start 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.419+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 0x55e9502cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.419+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.419+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1820 0x55e9502cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.419+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e9500e5fa0 con 0x55e9502c1410 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.419+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e950180760 con 0x55e950142e40 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.419+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e9502ce990 con 0x55e9502c1820 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.419+0000 7fed1b06cb80 10 client.0 _ll_register_callbacks cb 0x55e950189c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.420+0000 7fed09c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 0x55e9502cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.420+0000 7fed0a600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1820 0x55e9502cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.420+0000 7fed09c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 0x55e9502cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51704/0 (socket says 172.21.15.63:51704) 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.420+0000 7fed09c00700 1 -- 172.21.15.63:0/3541363741 learned_addr learned my addr 172.21.15.63:0/3541363741 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.420+0000 7fed0a600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1820 0x55e9502cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:42698/0 (socket says 172.21.15.63:42698) 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.420+0000 7fed09200700 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.420+0000 7fed09200700 1 -- 172.21.15.63:0/3541363741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1820 msgr2=0x55e9502cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:52.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.420+0000 7fed09200700 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1820 0x55e9502cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.421+0000 7fed09200700 1 -- 172.21.15.63:0/3541363741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 msgr2=0x55e9502cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.421+0000 7fed09200700 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 0x55e9502cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.421+0000 7fed09200700 1 -- 172.21.15.63:0/3541363741 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e95012f8d0 con 0x55e9502c1410 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.421+0000 7fed09c00700 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 0x55e9502cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.421+0000 7fed09200700 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502cbac0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fecf00e0ea0 tx=0x7fecf000c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac6732f8a526d99 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.421+0000 7feceb400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.421+0000 7feceb400700 1 -- 172.21.15.63:0/3541363741 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf00a6980 con 0x55e9502c1410 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.421+0000 7feceb400700 1 -- 172.21.15.63:0/3541363741 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fecf00aaf30 con 0x55e9502c1410 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.422+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3541363741 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf80128e0 con 0x55e9502c1410 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.422+0000 7feceb400700 1 -- 172.21.15.63:0/3541363741 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fecf00a6250 con 0x55e9502c1410 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.422+0000 7fed1b06cb80 10 client.21399 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.422+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3541363741 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e9501b0060 con 0x55e9502c1410 2024-05-27T08:04:52.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.422+0000 7feceb400700 1 -- 172.21.15.63:0/3541363741 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecf80128e0 con 0x55e9502c1410 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.423+0000 7feceb400700 1 -- 172.21.15.63:0/3541363741 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fecf00cbce0 con 0x55e9502c1410 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.423+0000 7fed1b06cb80 10 client.21399 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.423+0000 7fed1b06cb80 10 client.21399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.423+0000 7fed1b06cb80 20 client.21399 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.423+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e9502db8d0 0x55e9502ddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.423+0000 7fed1b06cb80 4 client.21399 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.423+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3541363741 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9501f1d90 con 0x55e9502db8d0 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.424+0000 7fed09c00700 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e9502db8d0 0x55e9502ddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.426+0000 7fed09c00700 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e9502db8d0 0x55e9502ddd50 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559993ea55171fb0 server_cookie=db0aeb5ce79c110e in_seq=0 out_seq=0 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.426+0000 7feceb400700 10 client.21399 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.428+0000 7feceb400700 1 -- 172.21.15.63:0/3541363741 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e9501f1d90 con 0x55e9502db8d0 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.428+0000 7feceb400700 10 client.21399 handle_command_reply: tid=0 2024-05-27T08:04:52.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.428+0000 7fed1b06cb80 1 client.21399 shutdown 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.428+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3541363741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 msgr2=0x55e9502cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.428+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502cbac0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fecf00e0ea0 tx=0x7fecf000c670 comp rx=0 tx=0).stop 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.429+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3541363741 shutdown_connections 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.429+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e9502db8d0 0x55e9502ddd50 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.429+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1820 0x55e9502cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.429+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 0x55e9502cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.429+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/3541363741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502cbac0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.429+0000 7feceb400700 0 client.21399 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.429+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3541363741 >> 172.21.15.63:0/3541363741 conn(0x55e9501a2fc0 msgr2=0x55e9502bbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.429+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3541363741 shutdown_connections 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.429+0000 7fed1b06cb80 1 -- 172.21.15.63:0/3541363741 wait complete. 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.429+0000 7fed1b06cb80 20 client.21399 trim_cache size 0 max 16384 2024-05-27T08:04:52.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.455+0000 7fed1b06cb80 1 Processor -- start 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.455+0000 7fed1b06cb80 1 -- start start 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.456+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9501a2fc0 0x55e9502c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.456+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 0x55e9502cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.456+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 0x55e9502d7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.456+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed041b4df0 con 0x55e9501a2fc0 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.456+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed041b2680 con 0x55e950142e40 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.456+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed041b04f0 con 0x55e9502c1410 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.456+0000 7fed09c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9501a2fc0 0x55e9502c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.456+0000 7fed09c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9501a2fc0 0x55e9502c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57604/0 (socket says 172.21.15.63:57604) 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.456+0000 7fed09c00700 1 -- 172.21.15.63:0/4283115919 learned_addr learned my addr 172.21.15.63:0/4283115919 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.456+0000 7fed09200700 1 --2- 172.21.15.63:0/4283115919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 0x55e9502cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.457+0000 7fed0a600700 1 --2- 172.21.15.63:0/4283115919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 0x55e9502d7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.457+0000 7fed09c00700 1 -- 172.21.15.63:0/4283115919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 msgr2=0x55e9502d7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:52.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.457+0000 7fed09c00700 1 --2- 172.21.15.63:0/4283115919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 0x55e9502d7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.457+0000 7fed09c00700 1 -- 172.21.15.63:0/4283115919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 msgr2=0x55e9502cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.457+0000 7fed09c00700 1 --2- 172.21.15.63:0/4283115919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 0x55e9502cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.457+0000 7fed09c00700 1 -- 172.21.15.63:0/4283115919 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9502b71d0 con 0x55e9501a2fc0 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.457+0000 7fed09c00700 1 --2- 172.21.15.63:0/4283115919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9501a2fc0 0x55e9502c57a0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009dd0 tx=0x7fecf8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503c239f779e4381 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.458+0000 7fecf6000700 1 -- 172.21.15.63:0/4283115919 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8012890 con 0x55e9501a2fc0 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.458+0000 7fecf6000700 1 -- 172.21.15.63:0/4283115919 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecf8005040 con 0x55e9501a2fc0 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.458+0000 7fecf6000700 1 -- 172.21.15.63:0/4283115919 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8004950 con 0x55e9501a2fc0 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.459+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4283115919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9501a2fc0 msgr2=0x55e9502c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.459+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/4283115919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9501a2fc0 0x55e9502c57a0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009dd0 tx=0x7fecf8009260 comp rx=0 tx=0).stop 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.459+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4283115919 shutdown_connections 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.459+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/4283115919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9502c1410 0x55e9502d7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.459+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/4283115919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9501a2fc0 0x55e9502c57a0 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.459+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/4283115919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e950142e40 0x55e9502cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.459+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4283115919 >> 172.21.15.63:0/4283115919 conn(0x55e9502b5f60 msgr2=0x55e9502c5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.459+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4283115919 shutdown_connections 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.459+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4283115919 wait complete. 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.460+0000 7fed1b06cb80 1 -- start start 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.460+0000 7fed1b06cb80 1 Processor -- start 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e950142e40 0x55e9502cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9501a2fc0 0x55e9502d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed1b06cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed041b0b40 con 0x55e9502c1410 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed041b1fb0 con 0x55e9501a2fc0 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed1b06cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed041b1210 con 0x55e950142e40 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed1b06cb80 10 client.0 _ll_register_callbacks cb 0x55e950189c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed0a600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed0a600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57608/0 (socket says 172.21.15.63:57608) 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed0a600700 1 -- 172.21.15.63:0/4096125292 learned_addr learned my addr 172.21.15.63:0/4096125292 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed09c00700 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e950142e40 0x55e9502cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.461+0000 7fed09200700 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9501a2fc0 0x55e9502d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.462+0000 7fed0a600700 1 -- 172.21.15.63:0/4096125292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e950142e40 msgr2=0x55e9502cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.462+0000 7fed0a600700 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e950142e40 0x55e9502cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.462+0000 7fed0a600700 1 -- 172.21.15.63:0/4096125292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9501a2fc0 msgr2=0x55e9502d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.462+0000 7fed0a600700 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9501a2fc0 0x55e9502d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.462+0000 7fed0a600700 1 -- 172.21.15.63:0/4096125292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9502be350 con 0x55e9502c1410 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.462+0000 7fed0a600700 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502d0610 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fecfc017a10 tx=0x7fecfc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7a615b8f32a9ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.462+0000 7feceb400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.462+0000 7feceb400700 1 -- 172.21.15.63:0/4096125292 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc004db0 con 0x55e9502c1410 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.463+0000 7feceb400700 1 -- 172.21.15.63:0/4096125292 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fecfc007990 con 0x55e9502c1410 2024-05-27T08:04:52.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.463+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4096125292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf8005040 con 0x55e9502c1410 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.463+0000 7feceb400700 1 -- 172.21.15.63:0/4096125292 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fecfc007b90 con 0x55e9502c1410 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.463+0000 7fed1b06cb80 10 client.21405 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.463+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4096125292 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e95034ee10 con 0x55e9502c1410 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.463+0000 7feceb400700 1 -- 172.21.15.63:0/4096125292 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecf8005040 con 0x55e9502c1410 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.463+0000 7feceb400700 1 -- 172.21.15.63:0/4096125292 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fecfc019c60 con 0x55e9502c1410 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.464+0000 7fed1b06cb80 10 client.21405 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.464+0000 7fed1b06cb80 10 client.21405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.464+0000 7fed1b06cb80 20 client.21405 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.464+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e9502c1820 0x55e9503515b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.464+0000 7fed1b06cb80 4 client.21405 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.464+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4096125292 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9501eb570 con 0x55e9502c1820 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.464+0000 7fed09c00700 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e9502c1820 0x55e9503515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.465+0000 7fed09c00700 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e9502c1820 0x55e9503515b0 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6add0e74027a1fe server_cookie=eaf7dffc434875ff in_seq=0 out_seq=0 2024-05-27T08:04:52.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.465+0000 7feceb400700 10 client.21405 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7feceb400700 1 -- 172.21.15.63:0/4096125292 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e9501eb570 con 0x55e9502c1820 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7feceb400700 10 client.21405 handle_command_reply: tid=0 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7fed1b06cb80 1 client.21405 shutdown 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4096125292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 msgr2=0x55e9502d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502d0610 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fecfc017a10 tx=0x7fecfc004ad0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4096125292 shutdown_connections 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e9502c1820 0x55e9503515b0 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e9502c1410 0x55e9502d0610 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e950142e40 0x55e9502cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7fed1b06cb80 1 --2- 172.21.15.63:0/4096125292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e9501a2fc0 0x55e9502d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4096125292 >> 172.21.15.63:0/4096125292 conn(0x55e9502b5f60 msgr2=0x55e9502cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7feceb400700 0 client.21405 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.466+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4096125292 shutdown_connections 2024-05-27T08:04:52.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.467+0000 7fed1b06cb80 1 -- 172.21.15.63:0/4096125292 wait complete. 2024-05-27T08:04:52.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.467+0000 7fed1b06cb80 20 client.21405 trim_cache size 0 max 16384 2024-05-27T08:04:52.601 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:04:52.601 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:04:52.601 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:04:52.601 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.467+0000 7fed0b000700 1 -- 172.21.15.63:0/2697141209 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fecec0b29f0 msgr2=0x7fecec0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fecec0b29f0 0x7fecec0b4f00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55e9502bd690 tx=0x7fecfc00b040 comp rx=0 tx=0).stop 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 -- 172.21.15.63:0/2697141209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed04105bd0 msgr2=0x7fed0419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed04105bd0 0x7fed0419d430 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fecf0006010 tx=0x7fecf0005e80 comp rx=0 tx=0).stop 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 -- 172.21.15.63:0/2697141209 shutdown_connections 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fecec0b29f0 0x7fecec0b4f00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed0410e6f0 0x7fed041a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed04105280 0x7fed0419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 --2- 172.21.15.63:0/2697141209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed04105bd0 0x7fed0419d430 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 -- 172.21.15.63:0/2697141209 >> 172.21.15.63:0/2697141209 conn(0x7fed04078ab0 msgr2=0x7fed04104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:52.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 -- 172.21.15.63:0/2697141209 shutdown_connections 2024-05-27T08:04:52.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:52.468+0000 7fed0b000700 1 -- 172.21.15.63:0/2697141209 wait complete. 2024-05-27T08:04:53.627 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:04:53.628 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:04:53.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:53 smithi063 ceph-mon[31613]: pgmap v993: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:04:53.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:53 smithi086 ceph-mon[34687]: pgmap v993: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:04:53.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:53 smithi149 ceph-mon[35408]: pgmap v993: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.976+0000 7f6316800640 1 -- 172.21.15.63:0/3244305730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631010f540 msgr2=0x7f6310116ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.976+0000 7f6316800640 1 --2- 172.21.15.63:0/3244305730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631010f540 0x7f6310116ce0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f6308009ca0 tx=0x7f630802a8c0 comp rx=0 tx=0).stop 2024-05-27T08:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.977+0000 7f6316800640 1 -- 172.21.15.63:0/3244305730 shutdown_connections 2024-05-27T08:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.977+0000 7f6316800640 1 --2- 172.21.15.63:0/3244305730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631010f540 0x7f6310116ce0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.977+0000 7f6316800640 1 --2- 172.21.15.63:0/3244305730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631010ea80 0x7f631010eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.977+0000 7f6316800640 1 --2- 172.21.15.63:0/3244305730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f631010a500 0x7f631010a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.977+0000 7f6316800640 1 -- 172.21.15.63:0/3244305730 >> 172.21.15.63:0/3244305730 conn(0x7f6310077190 msgr2=0x7f63100775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.977+0000 7f6316800640 1 -- 172.21.15.63:0/3244305730 shutdown_connections 2024-05-27T08:04:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.977+0000 7f6316800640 1 -- 172.21.15.63:0/3244305730 wait complete. 2024-05-27T08:04:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.979+0000 7f6316800640 1 Processor -- start 2024-05-27T08:04:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.979+0000 7f6316800640 1 -- start start 2024-05-27T08:04:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.980+0000 7f6316800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631010a500 0x7f63101a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.980+0000 7f6316800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f631010ea80 0x7f63101a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.980+0000 7f6316800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631010f540 0x7f63101a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.980+0000 7f6316800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f631011a330 con 0x7f631010f540 2024-05-27T08:04:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.980+0000 7f6316800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f631011a1b0 con 0x7f631010a500 2024-05-27T08:04:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.980+0000 7f6316800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f631011a4b0 con 0x7f631010ea80 2024-05-27T08:04:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.980+0000 7f630f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631010a500 0x7f63101a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.981+0000 7f630f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631010a500 0x7f63101a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51746/0 (socket says 172.21.15.63:51746) 2024-05-27T08:04:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.981+0000 7f630f400640 1 -- 172.21.15.63:0/2419366291 learned_addr learned my addr 172.21.15.63:0/2419366291 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.981+0000 7f630ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f631010ea80 0x7f63101a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.981+0000 7f630fe00640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631010f540 0x7f63101a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.982+0000 7f630fe00640 1 -- 172.21.15.63:0/2419366291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f631010ea80 msgr2=0x7f63101a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.982+0000 7f630fe00640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f631010ea80 0x7f63101a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.982+0000 7f630fe00640 1 -- 172.21.15.63:0/2419366291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631010a500 msgr2=0x7f63101a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.982+0000 7f630fe00640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631010a500 0x7f63101a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.982+0000 7f630fe00640 1 -- 172.21.15.63:0/2419366291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63101a5bd0 con 0x7f631010f540 2024-05-27T08:04:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.982+0000 7f630f400640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631010a500 0x7f63101a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.982+0000 7f630ea00640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f631010ea80 0x7f63101a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.982+0000 7f630fe00640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631010f540 0x7f63101a5590 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f6308031040 tx=0x7f6308003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.983+0000 7f6307e00640 1 -- 172.21.15.63:0/2419366291 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308032c90 con 0x7f631010f540 2024-05-27T08:04:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.983+0000 7f6307e00640 1 -- 172.21.15.63:0/2419366291 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6308032e30 con 0x7f631010f540 2024-05-27T08:04:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.983+0000 7f6307e00640 1 -- 172.21.15.63:0/2419366291 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308030a90 con 0x7f631010f540 2024-05-27T08:04:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.983+0000 7f6316800640 1 -- 172.21.15.63:0/2419366291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63101a5e60 con 0x7f631010f540 2024-05-27T08:04:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.984+0000 7f6316800640 1 -- 172.21.15.63:0/2419366291 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63101a6360 con 0x7f631010f540 2024-05-27T08:04:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.990+0000 7f631d7bf740 1 Processor -- start 2024-05-27T08:04:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.992+0000 7f6307e00640 1 -- 172.21.15.63:0/2419366291 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f63080046d0 con 0x7f631010f540 2024-05-27T08:04:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.993+0000 7f631d7bf740 1 -- start start 2024-05-27T08:04:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.994+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cf71b0 0x562233099860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.994+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x562233099da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.995+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cd64d0 0x56223309a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.995+0000 7f631d7bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56223300ab90 con 0x562232cce2f0 2024-05-27T08:04:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.995+0000 7f631d7bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562232ffa730 con 0x562232cf71b0 2024-05-27T08:04:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.995+0000 7f631d7bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562232ffe770 con 0x562232cd64d0 2024-05-27T08:04:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.995+0000 7f630f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x562233099da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.995+0000 7f6307e00640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62f40b2aa0 0x7f62f40b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.996+0000 7f630f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x562233099da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57628/0 (socket says 172.21.15.63:57628) 2024-05-27T08:04:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.996+0000 7f630f400640 1 -- 172.21.15.63:0/3797825704 learned_addr learned my addr 172.21.15.63:0/3797825704 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.996+0000 7f630ea00640 1 --2- 172.21.15.63:0/3797825704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cf71b0 0x562233099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.996+0000 7f630ea00640 1 --2- 172.21.15.63:0/3797825704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cd64d0 0x56223309a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.996+0000 7f6307e00640 1 -- 172.21.15.63:0/2419366291 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f6308103bf0 con 0x7f631010f540 2024-05-27T08:04:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.997+0000 7f630f400640 1 -- 172.21.15.63:0/3797825704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cd64d0 msgr2=0x56223309a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.997+0000 7f630f400640 1 --2- 172.21.15.63:0/3797825704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cd64d0 0x56223309a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.997+0000 7f630f400640 1 -- 172.21.15.63:0/3797825704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cf71b0 msgr2=0x562233099860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.997+0000 7f630f400640 1 --2- 172.21.15.63:0/3797825704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cf71b0 0x562233099860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.997+0000 7f630f400640 1 -- 172.21.15.63:0/3797825704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562232f938b0 con 0x562232cce2f0 2024-05-27T08:04:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.997+0000 7f630fe00640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62f40b2aa0 0x7f62f40b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:53.999+0000 7f630f400640 1 --2- 172.21.15.63:0/3797825704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x562233099da0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f6300010580 tx=0x7f6300010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba94a023fcf1aa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.000+0000 7f630fe00640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62f40b2aa0 0x7f62f40b4fb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f63080c6ca0 tx=0x7f63080095c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.000+0000 7f6304c00640 1 -- 172.21.15.63:0/3797825704 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630000e890 con 0x562232cce2f0 2024-05-27T08:04:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.001+0000 7f6304c00640 1 -- 172.21.15.63:0/3797825704 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6300011440 con 0x562232cce2f0 2024-05-27T08:04:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.001+0000 7f6304c00640 1 -- 172.21.15.63:0/3797825704 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630001a5e0 con 0x562232cce2f0 2024-05-27T08:04:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.002+0000 7f631d7bf740 1 -- 172.21.15.63:0/3797825704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 msgr2=0x562233099da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.002+0000 7f631d7bf740 1 --2- 172.21.15.63:0/3797825704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x562233099da0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f6300010580 tx=0x7f6300010a40 comp rx=0 tx=0).stop 2024-05-27T08:04:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.002+0000 7f631d7bf740 1 -- 172.21.15.63:0/3797825704 shutdown_connections 2024-05-27T08:04:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.003+0000 7f631d7bf740 1 --2- 172.21.15.63:0/3797825704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cd64d0 0x56223309a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.003+0000 7f631d7bf740 1 --2- 172.21.15.63:0/3797825704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x562233099da0 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.003+0000 7f631d7bf740 1 --2- 172.21.15.63:0/3797825704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cf71b0 0x562233099860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.003+0000 7f631d7bf740 1 -- 172.21.15.63:0/3797825704 >> 172.21.15.63:0/3797825704 conn(0x562232ceb300 msgr2=0x562233096010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.004+0000 7f631d7bf740 1 -- 172.21.15.63:0/3797825704 shutdown_connections 2024-05-27T08:04:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.004+0000 7f631d7bf740 1 -- 172.21.15.63:0/3797825704 wait complete. 2024-05-27T08:04:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.005+0000 7f631d7bf740 1 -- start start 2024-05-27T08:04:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.005+0000 7f631d7bf740 1 Processor -- start 2024-05-27T08:04:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.006+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cce2f0 0x5622330a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.006+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cd64d0 0x5622330a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.007+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cf71b0 0x5622330afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.007+0000 7f631d7bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562232ce0790 con 0x562232cd64d0 2024-05-27T08:04:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.007+0000 7f631d7bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562232fa2bc0 con 0x562232cf71b0 2024-05-27T08:04:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.007+0000 7f631d7bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5622330a6990 con 0x562232cce2f0 2024-05-27T08:04:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.007+0000 7f631d7bf740 10 client.0 _ll_register_callbacks cb 0x562232f80400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.008+0000 7f630ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cce2f0 0x5622330a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.008+0000 7f630ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cce2f0 0x5622330a3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:42746/0 (socket says 172.21.15.63:42746) 2024-05-27T08:04:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.008+0000 7f630ea00640 1 -- 172.21.15.63:0/2481300683 learned_addr learned my addr 172.21.15.63:0/2481300683 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.008+0000 7f630f400640 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cd64d0 0x5622330a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.008+0000 7f630ea00640 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cf71b0 0x5622330afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.009+0000 7f630ea00640 1 -- 172.21.15.63:0/2481300683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cf71b0 msgr2=0x5622330afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.009+0000 7f630ea00640 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cf71b0 0x5622330afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.009+0000 7f630ea00640 1 -- 172.21.15.63:0/2481300683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cd64d0 msgr2=0x5622330a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.009+0000 7f630ea00640 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cd64d0 0x5622330a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.009+0000 7f630ea00640 1 -- 172.21.15.63:0/2481300683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562232d0f210 con 0x562232cce2f0 2024-05-27T08:04:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.009+0000 7f630f400640 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cd64d0 0x5622330a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.010+0000 7f630ea00640 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cce2f0 0x5622330a3a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00f600 tx=0x7f62fc012040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25e9c6dd16573d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.010+0000 7f62f9600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:04:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.011+0000 7f62f9600640 1 -- 172.21.15.63:0/2481300683 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc019040 con 0x562232cce2f0 2024-05-27T08:04:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.011+0000 7f62f9600640 1 -- 172.21.15.63:0/2481300683 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f62fc0207e0 con 0x562232cce2f0 2024-05-27T08:04:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.012+0000 7f631d7bf740 1 -- 172.21.15.63:0/2481300683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6300011440 con 0x562232cce2f0 2024-05-27T08:04:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.012+0000 7f62f9600640 1 -- 172.21.15.63:0/2481300683 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f62fc0209e0 con 0x562232cce2f0 2024-05-27T08:04:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.012+0000 7f631d7bf740 10 client.30337 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.012+0000 7f631d7bf740 1 -- 172.21.15.63:0/2481300683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562232ddb620 con 0x562232cce2f0 2024-05-27T08:04:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.013+0000 7f62f9600640 1 -- 172.21.15.63:0/2481300683 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62fc02c070 con 0x562232cce2f0 2024-05-27T08:04:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.013+0000 7f62f9600640 1 -- 172.21.15.63:0/2481300683 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f62fc01f040 con 0x562232cce2f0 2024-05-27T08:04:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.014+0000 7f631d7bf740 10 client.30337 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.014+0000 7f631d7bf740 10 client.30337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.014+0000 7f631d7bf740 20 client.30337 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.014+0000 7f631d7bf740 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562232cb5d80 0x5622330b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.014+0000 7f631d7bf740 4 client.30337 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.014+0000 7f631d7bf740 1 -- 172.21.15.63:0/2481300683 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56223300cae0 con 0x562232cb5d80 2024-05-27T08:04:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.015+0000 7f630f400640 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562232cb5d80 0x5622330b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.016+0000 7f630f400640 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562232cb5d80 0x5622330b5d30 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19e467fa5d3b9a1f server_cookie=593b76f2b9101d5f in_seq=0 out_seq=0 2024-05-27T08:04:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.016+0000 7f62f9600640 10 client.30337 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.019+0000 7f62f9600640 1 -- 172.21.15.63:0/2481300683 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56223300cae0 con 0x562232cb5d80 2024-05-27T08:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.019+0000 7f62f9600640 10 client.30337 handle_command_reply: tid=0 2024-05-27T08:04:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.019+0000 7f631d7bf740 1 client.30337 shutdown 2024-05-27T08:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.020+0000 7f631d7bf740 1 -- 172.21.15.63:0/2481300683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cce2f0 msgr2=0x5622330a3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.020+0000 7f631d7bf740 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cce2f0 0x5622330a3a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00f600 tx=0x7f62fc012040 comp rx=0 tx=0).stop 2024-05-27T08:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.020+0000 7f631d7bf740 1 -- 172.21.15.63:0/2481300683 shutdown_connections 2024-05-27T08:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.020+0000 7f631d7bf740 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562232cb5d80 0x5622330b5d30 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.020+0000 7f631d7bf740 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cf71b0 0x5622330afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.020+0000 7f631d7bf740 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cd64d0 0x5622330a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.020+0000 7f631d7bf740 1 --2- 172.21.15.63:0/2481300683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cce2f0 0x5622330a3a70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.020+0000 7f62f9600640 0 client.30337 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.020+0000 7f631d7bf740 1 -- 172.21.15.63:0/2481300683 >> 172.21.15.63:0/2481300683 conn(0x562232ceb300 msgr2=0x56223309b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.021+0000 7f631d7bf740 1 -- 172.21.15.63:0/2481300683 shutdown_connections 2024-05-27T08:04:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.021+0000 7f631d7bf740 1 -- 172.21.15.63:0/2481300683 wait complete. 2024-05-27T08:04:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.021+0000 7f631d7bf740 20 client.30337 trim_cache size 0 max 16384 2024-05-27T08:04:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.045+0000 7f631d7bf740 1 Processor -- start 2024-05-27T08:04:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.045+0000 7f631d7bf740 1 -- start start 2024-05-27T08:04:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.045+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x5622330afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.045+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cd64d0 0x5622330aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.045+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cf71b0 0x56223309d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.045+0000 7f631d7bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63101ba090 con 0x562232cce2f0 2024-05-27T08:04:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.045+0000 7f631d7bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63101b7960 con 0x562232cd64d0 2024-05-27T08:04:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.045+0000 7f631d7bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f63101b57d0 con 0x562232cf71b0 2024-05-27T08:04:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x5622330afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x5622330afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57646/0 (socket says 172.21.15.63:57646) 2024-05-27T08:04:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630ea00640 1 -- 172.21.15.63:0/1662485301 learned_addr learned my addr 172.21.15.63:0/1662485301 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630f400640 1 --2- 172.21.15.63:0/1662485301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cd64d0 0x5622330aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630ea00640 1 --2- 172.21.15.63:0/1662485301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cf71b0 0x56223309d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630ea00640 1 -- 172.21.15.63:0/1662485301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cf71b0 msgr2=0x56223309d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630ea00640 1 --2- 172.21.15.63:0/1662485301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cf71b0 0x56223309d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630ea00640 1 -- 172.21.15.63:0/1662485301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cd64d0 msgr2=0x5622330aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630ea00640 1 --2- 172.21.15.63:0/1662485301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cd64d0 0x5622330aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630ea00640 1 -- 172.21.15.63:0/1662485301 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562233090050 con 0x562232cce2f0 2024-05-27T08:04:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.046+0000 7f630f400640 1 --2- 172.21.15.63:0/1662485301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cd64d0 0x5622330aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.047+0000 7f630ea00640 1 --2- 172.21.15.63:0/1662485301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x5622330afa30 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f62fc010af0 tx=0x7f62fc005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284407137349b2bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.047+0000 7f62fbe00640 1 -- 172.21.15.63:0/1662485301 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0207e0 con 0x562232cce2f0 2024-05-27T08:04:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.047+0000 7f62fbe00640 1 -- 172.21.15.63:0/1662485301 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62fc02c040 con 0x562232cce2f0 2024-05-27T08:04:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.047+0000 7f62fbe00640 1 -- 172.21.15.63:0/1662485301 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc005370 con 0x562232cce2f0 2024-05-27T08:04:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.048+0000 7f631d7bf740 1 -- 172.21.15.63:0/1662485301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 msgr2=0x5622330afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.048+0000 7f631d7bf740 1 --2- 172.21.15.63:0/1662485301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x5622330afa30 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f62fc010af0 tx=0x7f62fc005b90 comp rx=0 tx=0).stop 2024-05-27T08:04:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.049+0000 7f631d7bf740 1 -- 172.21.15.63:0/1662485301 shutdown_connections 2024-05-27T08:04:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.049+0000 7f631d7bf740 1 --2- 172.21.15.63:0/1662485301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cf71b0 0x56223309d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.049+0000 7f631d7bf740 1 --2- 172.21.15.63:0/1662485301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cd64d0 0x5622330aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.049+0000 7f631d7bf740 1 --2- 172.21.15.63:0/1662485301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cce2f0 0x5622330afa30 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.049+0000 7f631d7bf740 1 -- 172.21.15.63:0/1662485301 >> 172.21.15.63:0/1662485301 conn(0x562232ceb300 msgr2=0x5622330a3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.049+0000 7f631d7bf740 1 -- 172.21.15.63:0/1662485301 shutdown_connections 2024-05-27T08:04:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.049+0000 7f631d7bf740 1 -- 172.21.15.63:0/1662485301 wait complete. 2024-05-27T08:04:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.049+0000 7f631d7bf740 1 -- start start 2024-05-27T08:04:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.050+0000 7f631d7bf740 1 Processor -- start 2024-05-27T08:04:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.050+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cce2f0 0x5622330abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.050+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cd64d0 0x5622330a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.050+0000 7f631d7bf740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cf71b0 0x5622330a7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.050+0000 7f631d7bf740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63101b5e20 con 0x562232cf71b0 2024-05-27T08:04:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.050+0000 7f631d7bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63101b7290 con 0x562232cce2f0 2024-05-27T08:04:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.050+0000 7f631d7bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f63101b64f0 con 0x562232cd64d0 2024-05-27T08:04:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.050+0000 7f631d7bf740 10 client.0 _ll_register_callbacks cb 0x7f62fc010610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.051+0000 7f630ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cf71b0 0x5622330a7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.051+0000 7f630ea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cf71b0 0x5622330a7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57650/0 (socket says 172.21.15.63:57650) 2024-05-27T08:04:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.051+0000 7f630ea00640 1 -- 172.21.15.63:0/1993159643 learned_addr learned my addr 172.21.15.63:0/1993159643 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.051+0000 7f630f400640 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cd64d0 0x5622330a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.051+0000 7f630ea00640 1 -- 172.21.15.63:0/1993159643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cd64d0 msgr2=0x5622330a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.051+0000 7f630ea00640 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cd64d0 0x5622330a7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.051+0000 7f630ea00640 1 -- 172.21.15.63:0/1993159643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cce2f0 msgr2=0x5622330abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.051+0000 7f630ea00640 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cce2f0 0x5622330abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.051+0000 7f630ea00640 1 -- 172.21.15.63:0/1993159643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622330a7da0 con 0x562232cf71b0 2024-05-27T08:04:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.052+0000 7f630ea00640 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cf71b0 0x5622330a7860 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f62fc013040 tx=0x7f62fc020810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6b006f4e28a383 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.052+0000 7f62f9600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:04:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.052+0000 7f62f9600640 1 -- 172.21.15.63:0/1993159643 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc020b80 con 0x562232cf71b0 2024-05-27T08:04:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.052+0000 7f62f9600640 1 -- 172.21.15.63:0/1993159643 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f62fc020d20 con 0x562232cf71b0 2024-05-27T08:04:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.052+0000 7f631d7bf740 1 -- 172.21.15.63:0/1993159643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62fc02c040 con 0x562232cf71b0 2024-05-27T08:04:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.053+0000 7f62f9600640 1 -- 172.21.15.63:0/1993159643 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f62fc022440 con 0x562232cf71b0 2024-05-27T08:04:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.053+0000 7f631d7bf740 10 client.21423 fetch_fsmap learned FSMap version 35 2024-05-27T08:04:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.053+0000 7f631d7bf740 1 -- 172.21.15.63:0/1993159643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56223311df50 con 0x562232cf71b0 2024-05-27T08:04:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.053+0000 7f62f9600640 1 -- 172.21.15.63:0/1993159643 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62fc02c040 con 0x562232cf71b0 2024-05-27T08:04:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.054+0000 7f62f9600640 1 -- 172.21.15.63:0/1993159643 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f62fc003ce0 con 0x562232cf71b0 2024-05-27T08:04:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.054+0000 7f631d7bf740 10 client.21423 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:04:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.054+0000 7f631d7bf740 10 client.21423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.054+0000 7f631d7bf740 20 client.21423 populate_metadata read hostname 'smithi063' 2024-05-27T08:04:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.054+0000 7f631d7bf740 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562232c8b1c0 0x5622331205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.054+0000 7f631d7bf740 4 client.21423 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:04:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.054+0000 7f631d7bf740 1 -- 172.21.15.63:0/1993159643 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622330062c0 con 0x562232c8b1c0 2024-05-27T08:04:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.055+0000 7f630f400640 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562232c8b1c0 0x5622331205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.056+0000 7f630f400640 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562232c8b1c0 0x5622331205f0 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d52b8ac8e6aab36 server_cookie=3e416685765f7b4 in_seq=0 out_seq=0 2024-05-27T08:04:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.056+0000 7f62f9600640 10 client.21423 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.088+0000 7f62f9600640 1 -- 172.21.15.63:0/1993159643 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5622330062c0 con 0x562232c8b1c0 2024-05-27T08:04:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.088+0000 7f62f9600640 10 client.21423 handle_command_reply: tid=0 2024-05-27T08:04:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.088+0000 7f631d7bf740 1 client.21423 shutdown 2024-05-27T08:04:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f631d7bf740 1 -- 172.21.15.63:0/1993159643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cf71b0 msgr2=0x5622330a7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f631d7bf740 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cf71b0 0x5622330a7860 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f62fc013040 tx=0x7f62fc020810 comp rx=0 tx=0).stop 2024-05-27T08:04:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f631d7bf740 1 -- 172.21.15.63:0/1993159643 shutdown_connections 2024-05-27T08:04:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f631d7bf740 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562232c8b1c0 0x5622331205f0 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f631d7bf740 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562232cf71b0 0x5622330a7860 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f631d7bf740 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562232cd64d0 0x5622330a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f631d7bf740 1 --2- 172.21.15.63:0/1993159643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562232cce2f0 0x5622330abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f62f9600640 0 client.21423 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:04:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f631d7bf740 1 -- 172.21.15.63:0/1993159643 >> 172.21.15.63:0/1993159643 conn(0x562232ceb300 msgr2=0x5622330995f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f631d7bf740 1 -- 172.21.15.63:0/1993159643 shutdown_connections 2024-05-27T08:04:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.089+0000 7f631d7bf740 1 -- 172.21.15.63:0/1993159643 wait complete. 2024-05-27T08:04:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.090+0000 7f631d7bf740 20 client.21423 trim_cache size 0 max 16384 2024-05-27T08:04:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:04:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:04:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.090+0000 7f6316800640 1 -- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62f40b2aa0 msgr2=0x7f62f40b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.090+0000 7f6316800640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62f40b2aa0 0x7f62f40b4fb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f63080c6ca0 tx=0x7f63080095c0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.090+0000 7f6316800640 1 -- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631010f540 msgr2=0x7f63101a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.090+0000 7f6316800640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631010f540 0x7f63101a5590 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f6308031040 tx=0x7f6308003f10 comp rx=0 tx=0).stop 2024-05-27T08:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.091+0000 7f6316800640 1 -- 172.21.15.63:0/2419366291 shutdown_connections 2024-05-27T08:04:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.091+0000 7f6316800640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62f40b2aa0 0x7f62f40b4fb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.091+0000 7f6316800640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631010f540 0x7f63101a5590 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.091+0000 7f6316800640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f631010ea80 0x7f63101a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.091+0000 7f6316800640 1 --2- 172.21.15.63:0/2419366291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f631010a500 0x7f63101a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.091+0000 7f6316800640 1 -- 172.21.15.63:0/2419366291 >> 172.21.15.63:0/2419366291 conn(0x7f6310077190 msgr2=0x7f631007a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.091+0000 7f6316800640 1 -- 172.21.15.63:0/2419366291 shutdown_connections 2024-05-27T08:04:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:04:54.091+0000 7f6316800640 1 -- 172.21.15.63:0/2419366291 wait complete. 2024-05-27T08:04:54.204 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:55.751 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:55 smithi063 ceph-mon[31613]: pgmap v994: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:04:55.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:55 smithi086 ceph-mon[34687]: pgmap v994: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:04:55.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:55 smithi149 ceph-mon[35408]: pgmap v994: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:04:57.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.002+0000 7ff263e00700 1 -- 172.21.15.63:0/1347352635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2641043f0 msgr2=0x7ff264104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:57.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.002+0000 7ff263e00700 1 --2- 172.21.15.63:0/1347352635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2641043f0 0x7ff264104800 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7ff2540099c0 tx=0x7ff254009cd0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.002+0000 7ff263e00700 1 -- 172.21.15.63:0/1347352635 shutdown_connections 2024-05-27T08:04:57.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.002+0000 7ff263e00700 1 --2- 172.21.15.63:0/1347352635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff264109670 0x7ff264110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.002+0000 7ff263e00700 1 --2- 172.21.15.63:0/1347352635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2641043f0 0x7ff264104800 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.002+0000 7ff263e00700 1 --2- 172.21.15.63:0/1347352635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff264108d20 0x7ff264109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.002+0000 7ff263e00700 1 -- 172.21.15.63:0/1347352635 >> 172.21.15.63:0/1347352635 conn(0x7ff264079d00 msgr2=0x7ff26407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:57.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.002+0000 7ff263e00700 1 -- 172.21.15.63:0/1347352635 shutdown_connections 2024-05-27T08:04:57.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.002+0000 7ff263e00700 1 -- 172.21.15.63:0/1347352635 wait complete. 2024-05-27T08:04:57.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.004+0000 7ff263e00700 1 Processor -- start 2024-05-27T08:04:57.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.004+0000 7ff263e00700 1 -- start start 2024-05-27T08:04:57.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.004+0000 7ff263e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2641043f0 0x7ff264075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:57.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.004+0000 7ff263e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff264108d20 0x7ff264075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:57.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.004+0000 7ff263e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff264109670 0x7ff264076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:57.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.004+0000 7ff263e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff264114150 con 0x7ff264108d20 2024-05-27T08:04:57.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.004+0000 7ff263e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff264113fd0 con 0x7ff2641043f0 2024-05-27T08:04:57.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.004+0000 7ff263e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641142d0 con 0x7ff264109670 2024-05-27T08:04:57.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.005+0000 7ff262000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff264108d20 0x7ff264075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:57.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.005+0000 7ff262000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff264108d20 0x7ff264075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57680/0 (socket says 172.21.15.63:57680) 2024-05-27T08:04:57.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.005+0000 7ff262000700 1 -- 172.21.15.63:0/1219434543 learned_addr learned my addr 172.21.15.63:0/1219434543 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:04:57.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.005+0000 7ff262a00700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2641043f0 0x7ff264075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:57.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.005+0000 7ff263400700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff264109670 0x7ff264076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:57.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.005+0000 7ff262000700 1 -- 172.21.15.63:0/1219434543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff264109670 msgr2=0x7ff264076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.006+0000 7ff262000700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff264109670 0x7ff264076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.006+0000 7ff262000700 1 -- 172.21.15.63:0/1219434543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2641043f0 msgr2=0x7ff264075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.006+0000 7ff262000700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2641043f0 0x7ff264075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.006+0000 7ff262000700 1 -- 172.21.15.63:0/1219434543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2641a8f00 con 0x7ff264108d20 2024-05-27T08:04:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.006+0000 7ff262000700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff264108d20 0x7ff264075ff0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7ff25000d980 tx=0x7ff25000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.006+0000 7ff268a00700 1 -- 172.21.15.63:0/1219434543 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250017070 con 0x7ff264108d20 2024-05-27T08:04:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.006+0000 7ff268a00700 1 -- 172.21.15.63:0/1219434543 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff25000fd20 con 0x7ff264108d20 2024-05-27T08:04:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.006+0000 7ff263e00700 1 -- 172.21.15.63:0/1219434543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2641a91f0 con 0x7ff264108d20 2024-05-27T08:04:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.006+0000 7ff268a00700 1 -- 172.21.15.63:0/1219434543 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25000e3b0 con 0x7ff264108d20 2024-05-27T08:04:57.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.007+0000 7ff263e00700 1 -- 172.21.15.63:0/1219434543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641a9730 con 0x7ff264108d20 2024-05-27T08:04:57.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.008+0000 7ff263e00700 1 -- 172.21.15.63:0/1219434543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2641058a0 con 0x7ff264108d20 2024-05-27T08:04:57.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.011+0000 7ff268a00700 1 -- 172.21.15.63:0/1219434543 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff25000f890 con 0x7ff264108d20 2024-05-27T08:04:57.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.013+0000 7ff268a00700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff2480b29f0 0x7ff2480b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.014+0000 7ff268a00700 1 -- 172.21.15.63:0/1219434543 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7ff2500e24a0 con 0x7ff264108d20 2024-05-27T08:04:57.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.014+0000 7ff262a00700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff2480b29f0 0x7ff2480b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:57.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.015+0000 7ff262a00700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff2480b29f0 0x7ff2480b4f00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff2540058e0 tx=0x7ff254005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:57.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.016+0000 7ff268a00700 1 -- 172.21.15.63:0/1219434543 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2500a9a60 con 0x7ff264108d20 2024-05-27T08:04:57.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.374+0000 7ff263e00700 1 -- 172.21.15.63:0/1219434543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff26402e5e0 con 0x7ff264108d20 2024-05-27T08:04:57.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.375+0000 7ff268a00700 1 -- 172.21.15.63:0/1219434543 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7ff2500a9360 con 0x7ff264108d20 2024-05-27T08:04:57.377 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:04:57.377 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:04:57.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 -- 172.21.15.63:0/1219434543 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff2480b29f0 msgr2=0x7ff2480b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:57.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff2480b29f0 0x7ff2480b4f00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff2540058e0 tx=0x7ff254005fb0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 -- 172.21.15.63:0/1219434543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff264108d20 msgr2=0x7ff264075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:57.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff264108d20 0x7ff264075ff0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7ff25000d980 tx=0x7ff25000dc90 comp rx=0 tx=0).stop 2024-05-27T08:04:57.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 -- 172.21.15.63:0/1219434543 shutdown_connections 2024-05-27T08:04:57.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff2480b29f0 0x7ff2480b4f00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff264109670 0x7ff264076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2641043f0 0x7ff264075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 --2- 172.21.15.63:0/1219434543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff264108d20 0x7ff264075ff0 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:57.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 -- 172.21.15.63:0/1219434543 >> 172.21.15.63:0/1219434543 conn(0x7ff264079d00 msgr2=0x7ff2641029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:57.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.379+0000 7ff263e00700 1 -- 172.21.15.63:0/1219434543 shutdown_connections 2024-05-27T08:04:57.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:04:57.380+0000 7ff263e00700 1 -- 172.21.15.63:0/1219434543 wait complete. 2024-05-27T08:04:57.383 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:04:57.390 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:57 smithi063 ceph-mon[31613]: pgmap v995: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:04:57.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:57 smithi086 ceph-mon[34687]: pgmap v995: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:04:57.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:57 smithi149 ceph-mon[35408]: pgmap v995: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:04:58.280 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:04:58.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:58 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1219434543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:04:58.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:58 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1219434543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:04:58.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:58 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1219434543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:04:58.960 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:59.332 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:04:59 smithi063 ceph-mon[31613]: pgmap v996: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:04:59.366 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:04:59 smithi149 ceph-mon[35408]: pgmap v996: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:04:59.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:04:59 smithi086 ceph-mon[34687]: pgmap v996: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:05:01.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.371+0000 7f6f1be00700 1 -- 172.21.15.63:0/810666583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1c109670 msgr2=0x7f6f1c110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:01.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.371+0000 7f6f1be00700 1 --2- 172.21.15.63:0/810666583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1c109670 0x7f6f1c110fd0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400b0d0 tx=0x7f6f1400b3e0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.371+0000 7f6f1be00700 1 -- 172.21.15.63:0/810666583 shutdown_connections 2024-05-27T08:05:01.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.371+0000 7f6f1be00700 1 --2- 172.21.15.63:0/810666583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1c109670 0x7f6f1c110fd0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.371+0000 7f6f1be00700 1 --2- 172.21.15.63:0/810666583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f1c1043f0 0x7f6f1c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.371+0000 7f6f1be00700 1 --2- 172.21.15.63:0/810666583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f1c108d20 0x7f6f1c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.371+0000 7f6f1be00700 1 -- 172.21.15.63:0/810666583 >> 172.21.15.63:0/810666583 conn(0x7f6f1c079d00 msgr2=0x7f6f1c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:01.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.371+0000 7f6f1be00700 1 -- 172.21.15.63:0/810666583 shutdown_connections 2024-05-27T08:05:01.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.371+0000 7f6f1be00700 1 -- 172.21.15.63:0/810666583 wait complete. 2024-05-27T08:05:01.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.373+0000 7f6f1be00700 1 Processor -- start 2024-05-27T08:05:01.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.373+0000 7f6f1be00700 1 -- start start 2024-05-27T08:05:01.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f1c1043f0 0x7f6f1c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1c108d20 0x7f6f1c19d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f1c109670 0x7f6f1c1a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1be00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c114150 con 0x7f6f1c108d20 2024-05-27T08:05:01.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1be00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c113fd0 con 0x7f6f1c1043f0 2024-05-27T08:05:01.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1142d0 con 0x7f6f1c109670 2024-05-27T08:05:01.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1c108d20 0x7f6f1c19d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f1c1043f0 0x7f6f1c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1c108d20 0x7f6f1c19d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43344/0 (socket says 172.21.15.63:43344) 2024-05-27T08:05:01.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1a000700 1 -- 172.21.15.63:0/1852065145 learned_addr learned my addr 172.21.15.63:0/1852065145 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:01.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.374+0000 7f6f1b400700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f1c109670 0x7f6f1c1a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.375+0000 7f6f1a000700 1 -- 172.21.15.63:0/1852065145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f1c109670 msgr2=0x7f6f1c1a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:01.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.375+0000 7f6f1a000700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f1c109670 0x7f6f1c1a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.375+0000 7f6f1a000700 1 -- 172.21.15.63:0/1852065145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f1c1043f0 msgr2=0x7f6f1c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:01.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.375+0000 7f6f1a000700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f1c1043f0 0x7f6f1c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.375+0000 7f6f1a000700 1 -- 172.21.15.63:0/1852065145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c1a1c10 con 0x7f6f1c108d20 2024-05-27T08:05:01.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.376+0000 7f6f1a000700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1c108d20 0x7f6f1c19d2e0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800d980 tx=0x7f6f0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:01.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.376+0000 7f6f13400700 1 -- 172.21.15.63:0/1852065145 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08017070 con 0x7f6f1c108d20 2024-05-27T08:05:01.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.376+0000 7f6f13400700 1 -- 172.21.15.63:0/1852065145 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f08009dc0 con 0x7f6f1c108d20 2024-05-27T08:05:01.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.376+0000 7f6f1be00700 1 -- 172.21.15.63:0/1852065145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c1a1f00 con 0x7f6f1c108d20 2024-05-27T08:05:01.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.376+0000 7f6f1be00700 1 -- 172.21.15.63:0/1852065145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c1a9820 con 0x7f6f1c108d20 2024-05-27T08:05:01.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.376+0000 7f6f13400700 1 -- 172.21.15.63:0/1852065145 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0800f3c0 con 0x7f6f1c108d20 2024-05-27T08:05:01.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.382+0000 7f6f13400700 1 -- 172.21.15.63:0/1852065145 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6f08009930 con 0x7f6f1c108d20 2024-05-27T08:05:01.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.382+0000 7f6f30106b80 1 Processor -- start 2024-05-27T08:05:01.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.382+0000 7f6f30106b80 1 -- start start 2024-05-27T08:05:01.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.383+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3e9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.383+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.383+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 0x560d7e3f7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.383+0000 7f6f30106b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560d7e318e40 con 0x560d7e3ea410 2024-05-27T08:05:01.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.383+0000 7f6f30106b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d7e3089e0 con 0x560d7e3ea820 2024-05-27T08:05:01.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.383+0000 7f6f30106b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560d7e30ca20 con 0x560d7e26be40 2024-05-27T08:05:01.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.383+0000 7f6f1aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.384+0000 7f6f1aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43348/0 (socket says 172.21.15.63:43348) 2024-05-27T08:05:01.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.384+0000 7f6f1aa00700 1 -- 172.21.15.63:0/2550906994 learned_addr learned my addr 172.21.15.63:0/2550906994 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:01.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.384+0000 7f6f1b400700 1 --2- 172.21.15.63:0/2550906994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3e9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.384+0000 7f6f1a000700 1 --2- 172.21.15.63:0/2550906994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 0x560d7e3f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.384+0000 7f6f1aa00700 1 -- 172.21.15.63:0/2550906994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 msgr2=0x560d7e3e9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:01.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.384+0000 7f6f1aa00700 1 --2- 172.21.15.63:0/2550906994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3e9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.384+0000 7f6f1aa00700 1 -- 172.21.15.63:0/2550906994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 msgr2=0x560d7e3f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:01.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.384+0000 7f6f1aa00700 1 --2- 172.21.15.63:0/2550906994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 0x560d7e3f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.384+0000 7f6f1aa00700 1 -- 172.21.15.63:0/2550906994 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7e2d1420 con 0x560d7e3ea410 2024-05-27T08:05:01.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.385+0000 7f6f13400700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f000b29f0 0x7f6f000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.385+0000 7f6f13400700 1 -- 172.21.15.63:0/1852065145 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f6f080e24e0 con 0x7f6f1c108d20 2024-05-27T08:05:01.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.385+0000 7f6f1b400700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f000b29f0 0x7f6f000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.385+0000 7f6f1aa00700 1 --2- 172.21.15.63:0/2550906994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4ad0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00bd70 tx=0x7f6f0c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604c355c55572b9d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:01.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.386+0000 7f6f07e00700 1 -- 172.21.15.63:0/2550906994 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c010e10 con 0x560d7e3ea410 2024-05-27T08:05:01.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.386+0000 7f6f07e00700 1 -- 172.21.15.63:0/2550906994 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f0c01c910 con 0x560d7e3ea410 2024-05-27T08:05:01.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.386+0000 7f6f07e00700 1 -- 172.21.15.63:0/2550906994 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c011870 con 0x560d7e3ea410 2024-05-27T08:05:01.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.387+0000 7f6f30106b80 1 -- 172.21.15.63:0/2550906994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 msgr2=0x560d7e3f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:01.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.387+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2550906994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4ad0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00bd70 tx=0x7f6f0c0106b0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.387+0000 7f6f1b400700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f000b29f0 0x7f6f000b4f00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009900 tx=0x7f6f1400b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:01.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.387+0000 7f6f30106b80 1 -- 172.21.15.63:0/2550906994 shutdown_connections 2024-05-27T08:05:01.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.387+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2550906994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 0x560d7e3f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.387+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2550906994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3e9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.387+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2550906994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4ad0 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.387+0000 7f6f30106b80 1 -- 172.21.15.63:0/2550906994 >> 172.21.15.63:0/2550906994 conn(0x560d7e2cbfc0 msgr2=0x560d7e3e5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:01.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.387+0000 7f6f30106b80 1 -- 172.21.15.63:0/2550906994 shutdown_connections 2024-05-27T08:05:01.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.387+0000 7f6f30106b80 1 -- 172.21.15.63:0/2550906994 wait complete. 2024-05-27T08:05:01.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.388+0000 7f6f30106b80 1 -- start start 2024-05-27T08:05:01.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.388+0000 7f6f30106b80 1 Processor -- start 2024-05-27T08:05:01.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.389+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.389+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.389+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 0x560d7e3f5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.389+0000 7f6f30106b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560d7e20efa0 con 0x560d7e3ea410 2024-05-27T08:05:01.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.389+0000 7f6f30106b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d7e2a9760 con 0x560d7e3ea820 2024-05-27T08:05:01.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.389+0000 7f6f30106b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x560d7e3f7990 con 0x560d7e26be40 2024-05-27T08:05:01.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.389+0000 7f6f30106b80 10 client.0 _ll_register_callbacks cb 0x560d7e2b2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:01.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.389+0000 7f6f1a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.390+0000 7f6f1aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.390+0000 7f6f1a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43358/0 (socket says 172.21.15.63:43358) 2024-05-27T08:05:01.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.390+0000 7f6f1a000700 1 -- 172.21.15.63:0/2681221 learned_addr learned my addr 172.21.15.63:0/2681221 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:01.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.390+0000 7f6f1b400700 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 0x560d7e3f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.390+0000 7f6f1a000700 1 -- 172.21.15.63:0/2681221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 msgr2=0x560d7e3f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:01.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.390+0000 7f6f1a000700 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.390+0000 7f6f1a000700 1 -- 172.21.15.63:0/2681221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 msgr2=0x560d7e3f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:01.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.390+0000 7f6f1a000700 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 0x560d7e3f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.390+0000 7f6f1a000700 1 -- 172.21.15.63:0/2681221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7e2588d0 con 0x560d7e3ea410 2024-05-27T08:05:01.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.390+0000 7f6f1b400700 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 0x560d7e3f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:01.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.391+0000 7f6f1a000700 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4b30 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f6f080a90a0 tx=0x7f6f080108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328bad9f963985c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:01.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.391+0000 7f6f04c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:01.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.393+0000 7f6f04c00700 1 -- 172.21.15.63:0/2681221 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08020070 con 0x560d7e3ea410 2024-05-27T08:05:01.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.393+0000 7f6f04c00700 1 -- 172.21.15.63:0/2681221 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f6f080a5990 con 0x560d7e3ea410 2024-05-27T08:05:01.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.393+0000 7f6f30106b80 1 -- 172.21.15.63:0/2681221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f0c01c910 con 0x560d7e3ea410 2024-05-27T08:05:01.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.393+0000 7f6f04c00700 1 -- 172.21.15.63:0/2681221 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6f080a8cf0 con 0x560d7e3ea410 2024-05-27T08:05:01.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.393+0000 7f6f30106b80 10 client.21441 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:01.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.393+0000 7f6f30106b80 1 -- 172.21.15.63:0/2681221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560d7e2d9060 con 0x560d7e3ea410 2024-05-27T08:05:01.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.394+0000 7f6f04c00700 1 -- 172.21.15.63:0/2681221 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f0c01c910 con 0x560d7e3ea410 2024-05-27T08:05:01.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.394+0000 7f6f04c00700 1 -- 172.21.15.63:0/2681221 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6f080e2100 con 0x560d7e3ea410 2024-05-27T08:05:01.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.394+0000 7f6f30106b80 10 client.21441 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:01.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.394+0000 7f6f30106b80 10 client.21441 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:05:01.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.394+0000 7f6f30106b80 20 client.21441 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:01.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.394+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x560d7e404930 0x560d7e406d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.394+0000 7f6f30106b80 4 client.21441 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:01.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.394+0000 7f6f30106b80 1 -- 172.21.15.63:0/2681221 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d7e31ad90 con 0x560d7e404930 2024-05-27T08:05:01.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.395+0000 7f6f1aa00700 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x560d7e404930 0x560d7e406d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.396+0000 7f6f04c00700 10 client.21441 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:05:01.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.396+0000 7f6f1aa00700 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x560d7e404930 0x560d7e406d80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=376549f95822cabe server_cookie=e1331548bcabf997 in_seq=0 out_seq=0 2024-05-27T08:05:01.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.398+0000 7f6f04c00700 1 -- 172.21.15.63:0/2681221 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560d7e31ad90 con 0x560d7e404930 2024-05-27T08:05:01.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.398+0000 7f6f04c00700 10 client.21441 handle_command_reply: tid=0 2024-05-27T08:05:01.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.398+0000 7f6f30106b80 1 client.21441 shutdown 2024-05-27T08:05:01.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.398+0000 7f6f30106b80 1 -- 172.21.15.63:0/2681221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 msgr2=0x560d7e3f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:01.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.398+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4b30 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f6f080a90a0 tx=0x7f6f080108b0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.398+0000 7f6f30106b80 1 -- 172.21.15.63:0/2681221 shutdown_connections 2024-05-27T08:05:01.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.398+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea820 0x560d7e3f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.398+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.398+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x560d7e404930 0x560d7e406d80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.398+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2681221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f4b30 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.399+0000 7f6f30106b80 1 -- 172.21.15.63:0/2681221 >> 172.21.15.63:0/2681221 conn(0x560d7e2cbfc0 msgr2=0x560d7e3e4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:01.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.399+0000 7f6f04c00700 0 client.21441 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:05:01.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.399+0000 7f6f30106b80 1 -- 172.21.15.63:0/2681221 shutdown_connections 2024-05-27T08:05:01.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.399+0000 7f6f30106b80 1 -- 172.21.15.63:0/2681221 wait complete. 2024-05-27T08:05:01.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.399+0000 7f6f30106b80 20 client.21441 trim_cache size 0 max 16384 2024-05-27T08:05:01.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.419+0000 7f6f30106b80 1 Processor -- start 2024-05-27T08:05:01.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.419+0000 7f6f30106b80 1 -- start start 2024-05-27T08:05:01.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.420+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e26be40 0x560d7e3efe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.420+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.420+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e3ea820 0x560d7e3f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.420+0000 7f6f30106b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1b4df0 con 0x560d7e3ea410 2024-05-27T08:05:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.420+0000 7f6f30106b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1b2680 con 0x560d7e26be40 2024-05-27T08:05:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.420+0000 7f6f30106b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1b04f0 con 0x560d7e3ea820 2024-05-27T08:05:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.420+0000 7f6f1a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.420+0000 7f6f1a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43374/0 (socket says 172.21.15.63:43374) 2024-05-27T08:05:01.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.420+0000 7f6f1a000700 1 -- 172.21.15.63:0/2795009817 learned_addr learned my addr 172.21.15.63:0/2795009817 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.421+0000 7f6f1aa00700 1 --2- 172.21.15.63:0/2795009817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e26be40 0x560d7e3efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.421+0000 7f6f1b400700 1 --2- 172.21.15.63:0/2795009817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e3ea820 0x560d7e3f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.421+0000 7f6f1a000700 1 -- 172.21.15.63:0/2795009817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e3ea820 msgr2=0x560d7e3f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.421+0000 7f6f1a000700 1 --2- 172.21.15.63:0/2795009817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e3ea820 0x560d7e3f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.421+0000 7f6f1a000700 1 -- 172.21.15.63:0/2795009817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e26be40 msgr2=0x560d7e3efe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.421+0000 7f6f1a000700 1 --2- 172.21.15.63:0/2795009817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e26be40 0x560d7e3efe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.421+0000 7f6f1a000700 1 -- 172.21.15.63:0/2795009817 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7e400a90 con 0x560d7e3ea410 2024-05-27T08:05:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.421+0000 7f6f1a000700 1 --2- 172.21.15.63:0/2795009817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f6a90 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f6f080a5420 tx=0x7f6f080a98e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d93b43cc1edc109 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.422+0000 7f6f07400700 1 -- 172.21.15.63:0/2795009817 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08014020 con 0x560d7e3ea410 2024-05-27T08:05:01.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.422+0000 7f6f07400700 1 -- 172.21.15.63:0/2795009817 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f080a9aa0 con 0x560d7e3ea410 2024-05-27T08:05:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.422+0000 7f6f07400700 1 -- 172.21.15.63:0/2795009817 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08020030 con 0x560d7e3ea410 2024-05-27T08:05:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.423+0000 7f6f30106b80 1 -- 172.21.15.63:0/2795009817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 msgr2=0x560d7e3f6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.423+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2795009817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f6a90 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f6f080a5420 tx=0x7f6f080a98e0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.423+0000 7f6f30106b80 1 -- 172.21.15.63:0/2795009817 shutdown_connections 2024-05-27T08:05:01.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.423+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2795009817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e3ea820 0x560d7e3f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.423+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2795009817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e26be40 0x560d7e3efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.423+0000 7f6f30106b80 1 --2- 172.21.15.63:0/2795009817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea410 0x560d7e3f6a90 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.423+0000 7f6f30106b80 1 -- 172.21.15.63:0/2795009817 >> 172.21.15.63:0/2795009817 conn(0x560d7e2cbfc0 msgr2=0x560d7e3f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:01.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.423+0000 7f6f30106b80 1 -- 172.21.15.63:0/2795009817 shutdown_connections 2024-05-27T08:05:01.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.423+0000 7f6f30106b80 1 -- 172.21.15.63:0/2795009817 wait complete. 2024-05-27T08:05:01.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.424+0000 7f6f30106b80 1 -- start start 2024-05-27T08:05:01.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.424+0000 7f6f30106b80 1 Processor -- start 2024-05-27T08:05:01.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea410 0x560d7e3f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f30106b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea820 0x560d7e467420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f30106b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1b0b40 con 0x560d7e3ea820 2024-05-27T08:05:01.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f30106b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1b1fb0 con 0x560d7e3ea410 2024-05-27T08:05:01.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f30106b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1b1210 con 0x560d7e26be40 2024-05-27T08:05:01.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f30106b80 10 client.0 _ll_register_callbacks cb 0x560d7e4012f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:01.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f1b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea820 0x560d7e467420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f1a000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea410 0x560d7e3f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f1b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea820 0x560d7e467420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43386/0 (socket says 172.21.15.63:43386) 2024-05-27T08:05:01.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f1b400700 1 -- 172.21.15.63:0/1672747877 learned_addr learned my addr 172.21.15.63:0/1672747877 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:01.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.425+0000 7f6f1aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.426+0000 7f6f1b400700 1 -- 172.21.15.63:0/1672747877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 msgr2=0x560d7e3f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:01.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.426+0000 7f6f1b400700 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.426+0000 7f6f1b400700 1 -- 172.21.15.63:0/1672747877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea410 msgr2=0x560d7e3f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:01.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.426+0000 7f6f1b400700 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea410 0x560d7e3f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.426+0000 7f6f1b400700 1 -- 172.21.15.63:0/1672747877 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7e467990 con 0x560d7e3ea820 2024-05-27T08:05:01.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.426+0000 7f6f1b400700 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea820 0x560d7e467420 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009cb0 tx=0x7f6f140099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279f5ff9822f1690 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:01.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.426+0000 7f6f04c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:01.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.427+0000 7f6f04c00700 1 -- 172.21.15.63:0/1672747877 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14023070 con 0x560d7e3ea820 2024-05-27T08:05:01.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.427+0000 7f6f30106b80 1 -- 172.21.15.63:0/1672747877 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f080a9aa0 con 0x560d7e3ea820 2024-05-27T08:05:01.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.427+0000 7f6f04c00700 1 -- 172.21.15.63:0/1672747877 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f6f14009300 con 0x560d7e3ea820 2024-05-27T08:05:01.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.427+0000 7f6f04c00700 1 -- 172.21.15.63:0/1672747877 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6f14005030 con 0x560d7e3ea820 2024-05-27T08:05:01.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.427+0000 7f6f30106b80 10 client.21447 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:01.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.427+0000 7f6f30106b80 1 -- 172.21.15.63:0/1672747877 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560d7e46ecc0 con 0x560d7e3ea820 2024-05-27T08:05:01.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.428+0000 7f6f04c00700 1 -- 172.21.15.63:0/1672747877 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f080a9aa0 con 0x560d7e3ea820 2024-05-27T08:05:01.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.428+0000 7f6f04c00700 1 -- 172.21.15.63:0/1672747877 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6f14026070 con 0x560d7e3ea820 2024-05-27T08:05:01.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.428+0000 7f6f30106b80 10 client.21447 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:01.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.428+0000 7f6f30106b80 10 client.21447 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:05:01.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.428+0000 7f6f30106b80 20 client.21447 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:01.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.428+0000 7f6f30106b80 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x560d7e46f430 0x560d7e471850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:01.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.428+0000 7f6f30106b80 4 client.21447 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:05:01.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.428+0000 7f6f30106b80 1 -- 172.21.15.63:0/1672747877 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d7e314570 con 0x560d7e46f430 2024-05-27T08:05:01.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.429+0000 7f6f1aa00700 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x560d7e46f430 0x560d7e471850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:01.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.430+0000 7f6f04c00700 10 client.21447 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:05:01.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.430+0000 7f6f1aa00700 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x560d7e46f430 0x560d7e471850 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6213c4c277919fa server_cookie=9b88ee38077def45 in_seq=0 out_seq=0 2024-05-27T08:05:01.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.430+0000 7f6f04c00700 1 -- 172.21.15.63:0/1672747877 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d7e314570 con 0x560d7e46f430 2024-05-27T08:05:01.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.430+0000 7f6f04c00700 10 client.21447 handle_command_reply: tid=0 2024-05-27T08:05:01.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.430+0000 7f6f30106b80 1 client.21447 shutdown 2024-05-27T08:05:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.430+0000 7f6f30106b80 1 -- 172.21.15.63:0/1672747877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea820 msgr2=0x560d7e467420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f30106b80 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea820 0x560d7e467420 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009cb0 tx=0x7f6f140099e0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f30106b80 1 -- 172.21.15.63:0/1672747877 shutdown_connections 2024-05-27T08:05:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f30106b80 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x560d7e46f430 0x560d7e471850 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f30106b80 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560d7e3ea820 0x560d7e467420 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f30106b80 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560d7e26be40 0x560d7e3f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f30106b80 1 --2- 172.21.15.63:0/1672747877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d7e3ea410 0x560d7e3f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f04c00700 0 client.21447 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:05:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f30106b80 1 -- 172.21.15.63:0/1672747877 >> 172.21.15.63:0/1672747877 conn(0x560d7e2cbfc0 msgr2=0x560d7e3f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f30106b80 1 -- 172.21.15.63:0/1672747877 shutdown_connections 2024-05-27T08:05:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f30106b80 1 -- 172.21.15.63:0/1672747877 wait complete. 2024-05-27T08:05:01.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.431+0000 7f6f30106b80 20 client.21447 trim_cache size 0 max 16384 2024-05-27T08:05:01.433 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.432+0000 7f6f1be00700 1 -- 172.21.15.63:0/1852065145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f000b29f0 msgr2=0x7f6f000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.432+0000 7f6f1be00700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f000b29f0 0x7f6f000b4f00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009900 tx=0x7f6f1400b040 comp rx=0 tx=0).stop 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.432+0000 7f6f1be00700 1 -- 172.21.15.63:0/1852065145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1c108d20 msgr2=0x7f6f1c19d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.432+0000 7f6f1be00700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1c108d20 0x7f6f1c19d2e0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800d980 tx=0x7f6f0800dc90 comp rx=0 tx=0).stop 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.432+0000 7f6f1be00700 1 -- 172.21.15.63:0/1852065145 shutdown_connections 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.432+0000 7f6f1be00700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f6f000b29f0 0x7f6f000b4f00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.432+0000 7f6f1be00700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f1c109670 0x7f6f1c1a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.432+0000 7f6f1be00700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f1c1043f0 0x7f6f1c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.432+0000 7f6f1be00700 1 --2- 172.21.15.63:0/1852065145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f1c108d20 0x7f6f1c19d2e0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.432+0000 7f6f1be00700 1 -- 172.21.15.63:0/1852065145 >> 172.21.15.63:0/1852065145 conn(0x7f6f1c079d00 msgr2=0x7f6f1c110d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:01.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.433+0000 7f6f1be00700 1 -- 172.21.15.63:0/1852065145 shutdown_connections 2024-05-27T08:05:01.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:01.433+0000 7f6f1be00700 1 -- 172.21.15.63:0/1852065145 wait complete. 2024-05-27T08:05:01.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:01 smithi086 ceph-mon[34687]: pgmap v997: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:05:01.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:01 smithi149 ceph-mon[35408]: pgmap v997: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:05:01.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:01 smithi063 ceph-mon[31613]: pgmap v997: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:05:02.746 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:05:03.259 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:03.664 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:03 smithi063 ceph-mon[31613]: pgmap v998: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 28 op/s 2024-05-27T08:05:03.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:03 smithi086 ceph-mon[34687]: pgmap v998: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 28 op/s 2024-05-27T08:05:03.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:03 smithi149 ceph-mon[35408]: pgmap v998: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 28 op/s 2024-05-27T08:05:05.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:05 smithi149 ceph-mon[35408]: pgmap v999: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 25 op/s 2024-05-27T08:05:05.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:05 smithi063 ceph-mon[31613]: pgmap v999: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 25 op/s 2024-05-27T08:05:05.718 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:05 smithi086 ceph-mon[34687]: pgmap v999: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.6 MiB/s wr, 25 op/s 2024-05-27T08:05:05.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.790+0000 7fcac5e00700 1 -- 172.21.15.63:0/2821115792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac0115170 msgr2=0x7fcac0117590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.790+0000 7fcac5e00700 1 --2- 172.21.15.63:0/2821115792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac0115170 0x7fcac0117590 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fcab800b0a0 tx=0x7fcab800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.791+0000 7fcac5e00700 1 -- 172.21.15.63:0/2821115792 shutdown_connections 2024-05-27T08:05:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.791+0000 7fcac5e00700 1 --2- 172.21.15.63:0/2821115792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac0115170 0x7fcac0117590 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.791+0000 7fcac5e00700 1 --2- 172.21.15.63:0/2821115792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac0075b50 0x7fcac0075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.791+0000 7fcac5e00700 1 --2- 172.21.15.63:0/2821115792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac00764a0 0x7fcac0114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.791+0000 7fcac5e00700 1 -- 172.21.15.63:0/2821115792 >> 172.21.15.63:0/2821115792 conn(0x7fcac0079d00 msgr2=0x7fcac007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.791+0000 7fcac5e00700 1 -- 172.21.15.63:0/2821115792 shutdown_connections 2024-05-27T08:05:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.791+0000 7fcac5e00700 1 -- 172.21.15.63:0/2821115792 wait complete. 2024-05-27T08:05:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.793+0000 7fcac5e00700 1 Processor -- start 2024-05-27T08:05:05.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.793+0000 7fcac5e00700 1 -- start start 2024-05-27T08:05:05.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.793+0000 7fcac5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0075b50 0x7fcac01a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.793+0000 7fcac5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac00764a0 0x7fcac01a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.793+0000 7fcac5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac0115170 0x7fcac01a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.793+0000 7fcac5e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcac011a720 con 0x7fcac0115170 2024-05-27T08:05:05.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.793+0000 7fcac5e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcac011a5a0 con 0x7fcac0075b50 2024-05-27T08:05:05.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.793+0000 7fcac5e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcac011a8a0 con 0x7fcac00764a0 2024-05-27T08:05:05.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.794+0000 7fcac5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac0115170 0x7fcac01a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.794+0000 7fcac5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac0115170 0x7fcac01a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43398/0 (socket says 172.21.15.63:43398) 2024-05-27T08:05:05.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.794+0000 7fcac5400700 1 -- 172.21.15.63:0/3547638257 learned_addr learned my addr 172.21.15.63:0/3547638257 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:05.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.794+0000 7fcac4a00700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0075b50 0x7fcac01a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.794+0000 7fcabfe00700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac00764a0 0x7fcac01a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.794+0000 7fcac5400700 1 -- 172.21.15.63:0/3547638257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac00764a0 msgr2=0x7fcac01a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:05.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.794+0000 7fcac5400700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac00764a0 0x7fcac01a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.794+0000 7fcac5400700 1 -- 172.21.15.63:0/3547638257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0075b50 msgr2=0x7fcac01a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:05.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.794+0000 7fcac5400700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0075b50 0x7fcac01a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.794+0000 7fcac5400700 1 -- 172.21.15.63:0/3547638257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac01a5fc0 con 0x7fcac0115170 2024-05-27T08:05:05.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.795+0000 7fcac5400700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac0115170 0x7fcac01a5920 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fcab800b830 tx=0x7fcab8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:05.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.795+0000 7fcabd600700 1 -- 172.21.15.63:0/3547638257 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8021070 con 0x7fcac0115170 2024-05-27T08:05:05.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.795+0000 7fcac5e00700 1 -- 172.21.15.63:0/3547638257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac01a6250 con 0x7fcac0115170 2024-05-27T08:05:05.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.795+0000 7fcabd600700 1 -- 172.21.15.63:0/3547638257 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcab8007b80 con 0x7fcac0115170 2024-05-27T08:05:05.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.795+0000 7fcabd600700 1 -- 172.21.15.63:0/3547638257 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80052a0 con 0x7fcac0115170 2024-05-27T08:05:05.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.795+0000 7fcac5e00700 1 -- 172.21.15.63:0/3547638257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac01adbd0 con 0x7fcac0115170 2024-05-27T08:05:05.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.800+0000 7fcabd600700 1 -- 172.21.15.63:0/3547638257 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcab8024070 con 0x7fcac0115170 2024-05-27T08:05:05.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.801+0000 7fcad5f01b80 1 Processor -- start 2024-05-27T08:05:05.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.801+0000 7fcad5f01b80 1 -- start start 2024-05-27T08:05:05.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.801+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af0ff750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.801+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af0ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.801+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af1092c0 0x55b8af10b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.801+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b8af02ce40 con 0x55b8af06b9f0 2024-05-27T08:05:05.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.801+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8af01c9e0 con 0x55b8aef7fe40 2024-05-27T08:05:05.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.801+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b8af020a20 con 0x55b8af1092c0 2024-05-27T08:05:05.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.802+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af0ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.802+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af0ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43402/0 (socket says 172.21.15.63:43402) 2024-05-27T08:05:05.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.802+0000 7fcabfe00700 1 -- 172.21.15.63:0/1213132228 learned_addr learned my addr 172.21.15.63:0/1213132228 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:05.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.802+0000 7fcac4a00700 1 --2- 172.21.15.63:0/1213132228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af0ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.802+0000 7fcabfe00700 1 --2- 172.21.15.63:0/1213132228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af1092c0 0x55b8af10b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.802+0000 7fcabfe00700 1 -- 172.21.15.63:0/1213132228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af1092c0 msgr2=0x55b8af10b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:05.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.802+0000 7fcabfe00700 1 --2- 172.21.15.63:0/1213132228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af1092c0 0x55b8af10b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.803+0000 7fcabfe00700 1 -- 172.21.15.63:0/1213132228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 msgr2=0x55b8af0ffc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:05.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.803+0000 7fcabfe00700 1 --2- 172.21.15.63:0/1213132228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af0ffc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.803+0000 7fcabfe00700 1 -- 172.21.15.63:0/1213132228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8aefe5420 con 0x55b8af06b9f0 2024-05-27T08:05:05.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.803+0000 7fcabd600700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaa80b29f0 0x7fcaa80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.803+0000 7fcabd600700 1 -- 172.21.15.63:0/3547638257 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fcab800f020 con 0x7fcac0115170 2024-05-27T08:05:05.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.803+0000 7fcabfe00700 1 --2- 172.21.15.63:0/1213132228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af0ff750 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcaac013500 tx=0x7fcaac00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84503ce34e8e176 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:05.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.803+0000 7fcac4a00700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaa80b29f0 0x7fcaa80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.803+0000 7fcab2000700 1 -- 172.21.15.63:0/1213132228 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac019020 con 0x55b8af06b9f0 2024-05-27T08:05:05.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.804+0000 7fcab2000700 1 -- 172.21.15.63:0/1213132228 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcaac013e10 con 0x55b8af06b9f0 2024-05-27T08:05:05.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.804+0000 7fcab2000700 1 -- 172.21.15.63:0/1213132228 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00f740 con 0x55b8af06b9f0 2024-05-27T08:05:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.805+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1213132228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 msgr2=0x55b8af0ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.805+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1213132228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af0ff750 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcaac013500 tx=0x7fcaac00b690 comp rx=0 tx=0).stop 2024-05-27T08:05:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.805+0000 7fcac4a00700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaa80b29f0 0x7fcaa80b4f00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x55b8af0fb970 tx=0x7fcab400b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.805+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1213132228 shutdown_connections 2024-05-27T08:05:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.805+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1213132228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af1092c0 0x55b8af10b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.805+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1213132228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af0ff750 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.805+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1213132228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af0ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.805+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1213132228 >> 172.21.15.63:0/1213132228 conn(0x55b8af019390 msgr2=0x55b8af0fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:05.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.805+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1213132228 shutdown_connections 2024-05-27T08:05:05.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.805+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1213132228 wait complete. 2024-05-27T08:05:05.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.806+0000 7fcad5f01b80 1 -- start start 2024-05-27T08:05:05.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.806+0000 7fcad5f01b80 1 Processor -- start 2024-05-27T08:05:05.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af106200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af106740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af1092c0 0x55b8af106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b8aef22fa0 con 0x55b8af06b9f0 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8aefbd760 con 0x55b8aef7fe40 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b8af10be00 con 0x55b8af1092c0 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcad5f01b80 10 client.0 _ll_register_callbacks cb 0x55b8aefc6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af106740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af106740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43412/0 (socket says 172.21.15.63:43412) 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcabfe00700 1 -- 172.21.15.63:0/1139511476 learned_addr learned my addr 172.21.15.63:0/1139511476 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.807+0000 7fcac5400700 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af1092c0 0x55b8af106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.808+0000 7fcac4a00700 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af106200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.808+0000 7fcabfe00700 1 -- 172.21.15.63:0/1139511476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af1092c0 msgr2=0x55b8af106c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.808+0000 7fcabfe00700 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af1092c0 0x55b8af106c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.808+0000 7fcabfe00700 1 -- 172.21.15.63:0/1139511476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 msgr2=0x55b8af106200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.808+0000 7fcabfe00700 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af106200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.808+0000 7fcabfe00700 1 -- 172.21.15.63:0/1139511476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8aef6c8d0 con 0x55b8af06b9f0 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.809+0000 7fcabfe00700 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af106740 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcaac010f00 tx=0x7fcaac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774ff00464ef716e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.809+0000 7fcaa6a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.809+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1139511476 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac005250 con 0x55b8af06b9f0 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.809+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1139511476 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fcaac0053f0 con 0x55b8af06b9f0 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.809+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1139511476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaac013e10 con 0x55b8af06b9f0 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.810+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1139511476 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcaac031050 con 0x55b8af06b9f0 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.810+0000 7fcad5f01b80 10 client.21459 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:05.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.810+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1139511476 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b8aefed060 con 0x55b8af06b9f0 2024-05-27T08:05:05.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.810+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1139511476 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcaac013e10 con 0x55b8af06b9f0 2024-05-27T08:05:05.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.810+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1139511476 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcaac026020 con 0x55b8af06b9f0 2024-05-27T08:05:05.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.810+0000 7fcad5f01b80 10 client.21459 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:05.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.810+0000 7fcad5f01b80 10 client.21459 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:05:05.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.810+0000 7fcad5f01b80 20 client.21459 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:05.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.811+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b8af118c00 0x55b8af11b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.811+0000 7fcad5f01b80 4 client.21459 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:05.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.811+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1139511476 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8af02ed90 con 0x55b8af118c00 2024-05-27T08:05:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.811+0000 7fcac4a00700 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b8af118c00 0x55b8af11b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.812+0000 7fcac4a00700 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b8af118c00 0x55b8af11b020 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8ed4138dcf8d6f3 server_cookie=3eda9177839ef6d0 in_seq=0 out_seq=0 2024-05-27T08:05:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.812+0000 7fcaa6a00700 10 client.21459 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:05:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.814+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1139511476 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b8af02ed90 con 0x55b8af118c00 2024-05-27T08:05:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.814+0000 7fcaa6a00700 10 client.21459 handle_command_reply: tid=0 2024-05-27T08:05:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.814+0000 7fcad5f01b80 1 client.21459 shutdown 2024-05-27T08:05:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1139511476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 msgr2=0x55b8af106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af106740 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcaac010f00 tx=0x7fcaac004040 comp rx=0 tx=0).stop 2024-05-27T08:05:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1139511476 shutdown_connections 2024-05-27T08:05:05.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af1092c0 0x55b8af106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b8af118c00 0x55b8af11b020 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1139511476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af06b9f0 0x55b8af106740 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcaa6a00700 0 client.21459 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:05:05.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1139511476 >> 172.21.15.63:0/1139511476 conn(0x55b8af019390 msgr2=0x55b8af0fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:05.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1139511476 shutdown_connections 2024-05-27T08:05:05.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.815+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1139511476 wait complete. 2024-05-27T08:05:05.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.816+0000 7fcad5f01b80 20 client.21459 trim_cache size 0 max 16384 2024-05-27T08:05:05.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.838+0000 7fcad5f01b80 1 Processor -- start 2024-05-27T08:05:05.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.838+0000 7fcad5f01b80 1 -- start start 2024-05-27T08:05:05.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.839+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af0ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.839+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af06b9f0 0x55b8af114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.839+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.839+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01b91f0 con 0x55b8af1092c0 2024-05-27T08:05:05.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.839+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01b6a80 con 0x55b8aef7fe40 2024-05-27T08:05:05.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.839+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01b48f0 con 0x55b8af06b9f0 2024-05-27T08:05:05.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.839+0000 7fcac5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.839+0000 7fcabfe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af06b9f0 0x55b8af114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.839+0000 7fcac5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af107fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43422/0 (socket says 172.21.15.63:43422) 2024-05-27T08:05:05.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.839+0000 7fcac5400700 1 -- 172.21.15.63:0/1589805770 learned_addr learned my addr 172.21.15.63:0/1589805770 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:05.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.840+0000 7fcac4a00700 1 --2- 172.21.15.63:0/1589805770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af0ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.840+0000 7fcac5400700 1 -- 172.21.15.63:0/1589805770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af06b9f0 msgr2=0x55b8af114ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:05.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.840+0000 7fcac5400700 1 --2- 172.21.15.63:0/1589805770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af06b9f0 0x55b8af114ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.840+0000 7fcac5400700 1 -- 172.21.15.63:0/1589805770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 msgr2=0x55b8af0ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:05.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.840+0000 7fcac5400700 1 --2- 172.21.15.63:0/1589805770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af0ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.840+0000 7fcac5400700 1 -- 172.21.15.63:0/1589805770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8af100d60 con 0x55b8af1092c0 2024-05-27T08:05:05.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.840+0000 7fcac5400700 1 --2- 172.21.15.63:0/1589805770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af107fc0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fcab80a7060 tx=0x7fcab80aa870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1a61701f7d1a80 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:05.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.841+0000 7fcab1600700 1 -- 172.21.15.63:0/1589805770 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80aafe0 con 0x55b8af1092c0 2024-05-27T08:05:05.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.841+0000 7fcab1600700 1 -- 172.21.15.63:0/1589805770 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcab80af390 con 0x55b8af1092c0 2024-05-27T08:05:05.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.841+0000 7fcab1600700 1 -- 172.21.15.63:0/1589805770 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80b3920 con 0x55b8af1092c0 2024-05-27T08:05:05.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.842+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1589805770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 msgr2=0x55b8af107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:05.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.842+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1589805770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af107fc0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fcab80a7060 tx=0x7fcab80aa870 comp rx=0 tx=0).stop 2024-05-27T08:05:05.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.842+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1589805770 shutdown_connections 2024-05-27T08:05:05.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.842+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1589805770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af107fc0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.842+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1589805770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af0ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.842+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1589805770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af06b9f0 0x55b8af114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.842+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1589805770 >> 172.21.15.63:0/1589805770 conn(0x55b8af019390 msgr2=0x55b8af1078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:05.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.842+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1589805770 shutdown_connections 2024-05-27T08:05:05.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.842+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1589805770 wait complete. 2024-05-27T08:05:05.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.843+0000 7fcad5f01b80 1 -- start start 2024-05-27T08:05:05.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.843+0000 7fcad5f01b80 1 Processor -- start 2024-05-27T08:05:05.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af10c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af06b9f0 0x55b8af10cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcad5f01b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af1814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01b4f40 con 0x55b8af1092c0 2024-05-27T08:05:05.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01b63b0 con 0x55b8aef7fe40 2024-05-27T08:05:05.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcad5f01b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01b5610 con 0x55b8af06b9f0 2024-05-27T08:05:05.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcad5f01b80 10 client.0 _ll_register_callbacks cb 0x55b8aefc6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:05.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcac5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af1814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcac5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af1814b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43434/0 (socket says 172.21.15.63:43434) 2024-05-27T08:05:05.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcac5400700 1 -- 172.21.15.63:0/1576882342 learned_addr learned my addr 172.21.15.63:0/1576882342 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:05.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcabfe00700 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af06b9f0 0x55b8af10cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.844+0000 7fcac4a00700 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af10c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.845+0000 7fcac5400700 1 -- 172.21.15.63:0/1576882342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af06b9f0 msgr2=0x55b8af10cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.845+0000 7fcac5400700 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af06b9f0 0x55b8af10cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.845+0000 7fcac5400700 1 -- 172.21.15.63:0/1576882342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 msgr2=0x55b8af10c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.845+0000 7fcac5400700 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af10c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.845+0000 7fcac5400700 1 -- 172.21.15.63:0/1576882342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8af1819f0 con 0x55b8af1092c0 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.845+0000 7fcac5400700 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af1814b0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fcab80a7010 tx=0x7fcab80b3b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c2a9f3e9401326 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.845+0000 7fcaa6a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.845+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1576882342 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab806e020 con 0x55b8af1092c0 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.845+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1576882342 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fcab80a75d0 con 0x55b8af1092c0 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.845+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1576882342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab80af390 con 0x55b8af1092c0 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.846+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1576882342 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcab80b3db0 con 0x55b8af1092c0 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.846+0000 7fcad5f01b80 10 client.21465 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:05.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.846+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1576882342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b8af182b40 con 0x55b8af1092c0 2024-05-27T08:05:05.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.846+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1576882342 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcab80af390 con 0x55b8af1092c0 2024-05-27T08:05:05.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.846+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1576882342 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcab8011070 con 0x55b8af1092c0 2024-05-27T08:05:05.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.847+0000 7fcad5f01b80 10 client.21465 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:05.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.847+0000 7fcad5f01b80 10 client.21465 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:05:05.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.847+0000 7fcad5f01b80 20 client.21465 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:05.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.847+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b8af1831f0 0x55b8af185610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:05.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.847+0000 7fcad5f01b80 4 client.21465 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:05:05.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.847+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1576882342 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8af028570 con 0x55b8af1831f0 2024-05-27T08:05:05.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.847+0000 7fcac4a00700 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b8af1831f0 0x55b8af185610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:05.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.848+0000 7fcaa6a00700 10 client.21465 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:05:05.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.848+0000 7fcac4a00700 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b8af1831f0 0x55b8af185610 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b7e63037f34bd8c server_cookie=bda3e395aed85b13 in_seq=0 out_seq=0 2024-05-27T08:05:05.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcaa6a00700 1 -- 172.21.15.63:0/1576882342 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8af028570 con 0x55b8af1831f0 2024-05-27T08:05:05.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcaa6a00700 10 client.21465 handle_command_reply: tid=0 2024-05-27T08:05:05.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcad5f01b80 1 client.21465 shutdown 2024-05-27T08:05:05.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1576882342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 msgr2=0x55b8af1814b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:05.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af1814b0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fcab80a7010 tx=0x7fcab80b3b50 comp rx=0 tx=0).stop 2024-05-27T08:05:05.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1576882342 shutdown_connections 2024-05-27T08:05:05.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b8af1831f0 0x55b8af185610 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8af1092c0 0x55b8af1814b0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8aef7fe40 0x55b8af10c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcad5f01b80 1 --2- 172.21.15.63:0/1576882342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8af06b9f0 0x55b8af10cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1576882342 >> 172.21.15.63:0/1576882342 conn(0x55b8af019390 msgr2=0x55b8af101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:05.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.849+0000 7fcaa6a00700 0 client.21465 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:05:05.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.850+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1576882342 shutdown_connections 2024-05-27T08:05:05.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.850+0000 7fcad5f01b80 1 -- 172.21.15.63:0/1576882342 wait complete. 2024-05-27T08:05:05.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.850+0000 7fcad5f01b80 20 client.21465 trim_cache size 0 max 16384 2024-05-27T08:05:05.853 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:05:05.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 -- 172.21.15.63:0/3547638257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaa80b29f0 msgr2=0x7fcaa80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:05.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaa80b29f0 0x7fcaa80b4f00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x55b8af0fb970 tx=0x7fcab400b410 comp rx=0 tx=0).stop 2024-05-27T08:05:05.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 -- 172.21.15.63:0/3547638257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac0115170 msgr2=0x7fcac01a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:05.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac0115170 0x7fcac01a5920 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fcab800b830 tx=0x7fcab8004d00 comp rx=0 tx=0).stop 2024-05-27T08:05:05.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 -- 172.21.15.63:0/3547638257 shutdown_connections 2024-05-27T08:05:05.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcaa80b29f0 0x7fcaa80b4f00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcac0115170 0x7fcac01a5920 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcac0075b50 0x7fcac01a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 --2- 172.21.15.63:0/3547638257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac00764a0 0x7fcac01a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:05.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 -- 172.21.15.63:0/3547638257 >> 172.21.15.63:0/3547638257 conn(0x7fcac0079d00 msgr2=0x7fcac0100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:05.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 -- 172.21.15.63:0/3547638257 shutdown_connections 2024-05-27T08:05:05.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:05.851+0000 7fcac5e00700 1 -- 172.21.15.63:0/3547638257 wait complete. 2024-05-27T08:05:06.946 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:05:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.398+0000 7f8ee5a00640 1 -- 172.21.15.63:0/3699964397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ee010a150 msgr2=0x7f8ee0103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.398+0000 7f8ee5a00640 1 --2- 172.21.15.63:0/3699964397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ee010a150 0x7f8ee0103d90 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009900 tx=0x7f8ecc02a400 comp rx=0 tx=0).stop 2024-05-27T08:05:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.398+0000 7f8ee5a00640 1 -- 172.21.15.63:0/3699964397 shutdown_connections 2024-05-27T08:05:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.398+0000 7f8ee5a00640 1 --2- 172.21.15.63:0/3699964397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0104400 0x7f8ee01107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.398+0000 7f8ee5a00640 1 --2- 172.21.15.63:0/3699964397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ee010a150 0x7f8ee0103d90 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.398+0000 7f8ee5a00640 1 --2- 172.21.15.63:0/3699964397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee0109cb0 0x7f8ee0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.398+0000 7f8ee5a00640 1 -- 172.21.15.63:0/3699964397 >> 172.21.15.63:0/3699964397 conn(0x7f8ee00ff480 msgr2=0x7f8ee01018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.398+0000 7f8ee5a00640 1 -- 172.21.15.63:0/3699964397 shutdown_connections 2024-05-27T08:05:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.398+0000 7f8ee5a00640 1 -- 172.21.15.63:0/3699964397 wait complete. 2024-05-27T08:05:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.400+0000 7f8ee5a00640 1 Processor -- start 2024-05-27T08:05:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.400+0000 7f8ee5a00640 1 -- start start 2024-05-27T08:05:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.400+0000 7f8ee5a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee0104400 0x7f8ee019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.400+0000 7f8ee5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0109cb0 0x7f8ee019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.400+0000 7f8ee5a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ee010a150 0x7f8ee01a8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.400+0000 7f8ee5a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0113da0 con 0x7f8ee010a150 2024-05-27T08:05:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.400+0000 7f8ee5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0113c20 con 0x7f8ee0109cb0 2024-05-27T08:05:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.400+0000 7f8ee5a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0113f20 con 0x7f8ee0104400 2024-05-27T08:05:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.401+0000 7f8edf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ee010a150 0x7f8ee01a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.401+0000 7f8edf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ee010a150 0x7f8ee01a8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43448/0 (socket says 172.21.15.63:43448) 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.401+0000 7f8edf400640 1 -- 172.21.15.63:0/511257800 learned_addr learned my addr 172.21.15.63:0/511257800 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.401+0000 7f8ede000640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0109cb0 0x7f8ee019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.401+0000 7f8edea00640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee0104400 0x7f8ee019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.401+0000 7f8edf400640 1 -- 172.21.15.63:0/511257800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee0104400 msgr2=0x7f8ee019d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.401+0000 7f8edf400640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee0104400 0x7f8ee019d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.401+0000 7f8edf400640 1 -- 172.21.15.63:0/511257800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0109cb0 msgr2=0x7f8ee019dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.401+0000 7f8edf400640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0109cb0 0x7f8ee019dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.401+0000 7f8edf400640 1 -- 172.21.15.63:0/511257800 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee01a8c40 con 0x7f8ee010a150 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.402+0000 7f8edf400640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ee010a150 0x7f8ee01a8700 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009fd0 tx=0x7f8ed800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.402+0000 7f8ed7400640 1 -- 172.21.15.63:0/511257800 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8017070 con 0x7f8ee010a150 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.402+0000 7f8ee5a00640 1 -- 172.21.15.63:0/511257800 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee01a8ed0 con 0x7f8ee010a150 2024-05-27T08:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.402+0000 7f8ed7400640 1 -- 172.21.15.63:0/511257800 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ed8003e00 con 0x7f8ee010a150 2024-05-27T08:05:07.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.402+0000 7f8ed7400640 1 -- 172.21.15.63:0/511257800 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8010460 con 0x7f8ee010a150 2024-05-27T08:05:07.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.403+0000 7f8ee5a00640 1 -- 172.21.15.63:0/511257800 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee01a92e0 con 0x7f8ee010a150 2024-05-27T08:05:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.406+0000 7f8eec9a4740 1 Processor -- start 2024-05-27T08:05:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.407+0000 7f8ed7400640 1 -- 172.21.15.63:0/511257800 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8ed8003970 con 0x7f8ee010a150 2024-05-27T08:05:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.408+0000 7f8eec9a4740 1 -- start start 2024-05-27T08:05:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.408+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fadb31b0 0x5642fb155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.408+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb155c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.408+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.408+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5642fb0c6b90 con 0x5642fad924d0 2024-05-27T08:05:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.408+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642fb0b6730 con 0x5642fad8a2f0 2024-05-27T08:05:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.408+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5642fb0ba770 con 0x5642fadb31b0 2024-05-27T08:05:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.409+0000 7f8ede000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.409+0000 7f8ede000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb1561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43450/0 (socket says 172.21.15.63:43450) 2024-05-27T08:05:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.409+0000 7f8ede000640 1 -- 172.21.15.63:0/1938474797 learned_addr learned my addr 172.21.15.63:0/1938474797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.409+0000 7f8edea00640 1 --2- 172.21.15.63:0/1938474797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.409+0000 7f8ede000640 1 --2- 172.21.15.63:0/1938474797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fadb31b0 0x5642fb155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.410+0000 7f8edea00640 1 -- 172.21.15.63:0/1938474797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fadb31b0 msgr2=0x5642fb155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.410+0000 7f8edea00640 1 --2- 172.21.15.63:0/1938474797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fadb31b0 0x5642fb155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.410+0000 7f8edea00640 1 -- 172.21.15.63:0/1938474797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 msgr2=0x5642fb1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.410+0000 7f8edea00640 1 --2- 172.21.15.63:0/1938474797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.410+0000 7f8edea00640 1 -- 172.21.15.63:0/1938474797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642fb04f8b0 con 0x5642fad8a2f0 2024-05-27T08:05:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.410+0000 7f8ed7400640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ec40b2950 0x7f8ec40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.410+0000 7f8ede000640 1 --2- 172.21.15.63:0/1938474797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:05:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.410+0000 7f8ed7400640 1 -- 172.21.15.63:0/511257800 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f8ed80e2620 con 0x7f8ee010a150 2024-05-27T08:05:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.410+0000 7f8ede000640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ec40b2950 0x7f8ec40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.410+0000 7f8edea00640 1 --2- 172.21.15.63:0/1938474797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb155c90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0104990 tx=0x7f8ed000ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a087a74951d367f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.411+0000 7f8ecbe00640 1 -- 172.21.15.63:0/1938474797 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0003a90 con 0x5642fad8a2f0 2024-05-27T08:05:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.411+0000 7f8ecbe00640 1 -- 172.21.15.63:0/1938474797 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ed0015070 con 0x5642fad8a2f0 2024-05-27T08:05:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.411+0000 7f8ecbe00640 1 -- 172.21.15.63:0/1938474797 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0004b20 con 0x5642fad8a2f0 2024-05-27T08:05:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.412+0000 7f8eec9a4740 1 -- 172.21.15.63:0/1938474797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 msgr2=0x5642fb155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.412+0000 7f8ede000640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ec40b2950 0x7f8ec40b4e60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0057c0 tx=0x7f8ecc009560 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.412+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/1938474797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb155c90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0104990 tx=0x7f8ed000ba60 comp rx=0 tx=0).stop 2024-05-27T08:05:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.412+0000 7f8eec9a4740 1 -- 172.21.15.63:0/1938474797 shutdown_connections 2024-05-27T08:05:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.412+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/1938474797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.412+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/1938474797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb155c90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.412+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/1938474797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fadb31b0 0x5642fb155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.412+0000 7f8eec9a4740 1 -- 172.21.15.63:0/1938474797 >> 172.21.15.63:0/1938474797 conn(0x5642fada7300 msgr2=0x5642fb152010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.413+0000 7f8eec9a4740 1 -- 172.21.15.63:0/1938474797 shutdown_connections 2024-05-27T08:05:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.413+0000 7f8eec9a4740 1 -- 172.21.15.63:0/1938474797 wait complete. 2024-05-27T08:05:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.413+0000 7f8eec9a4740 1 -- start start 2024-05-27T08:05:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.414+0000 7f8eec9a4740 1 Processor -- start 2024-05-27T08:05:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.414+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb168200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.414+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb168740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.414+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fadb31b0 0x5642fb174500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.414+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5642fad9c790 con 0x5642fad924d0 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.414+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642fb05ebc0 con 0x5642fad8a2f0 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.415+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5642fb16b1a0 con 0x5642fadb31b0 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.415+0000 7f8eec9a4740 10 client.0 _ll_register_callbacks cb 0x5642fb03c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.415+0000 7f8ede000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb168740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.415+0000 7f8ede000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb168740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43452/0 (socket says 172.21.15.63:43452) 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.415+0000 7f8ede000640 1 -- 172.21.15.63:0/2191517157 learned_addr learned my addr 172.21.15.63:0/2191517157 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.415+0000 7f8edea00640 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb168200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.415+0000 7f8edf400640 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fadb31b0 0x5642fb174500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.416+0000 7f8ede000640 1 -- 172.21.15.63:0/2191517157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fadb31b0 msgr2=0x5642fb174500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.416+0000 7f8ede000640 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fadb31b0 0x5642fb174500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.416+0000 7f8ede000640 1 -- 172.21.15.63:0/2191517157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 msgr2=0x5642fb168200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.416+0000 7f8ede000640 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb168200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.416+0000 7f8ede000640 1 -- 172.21.15.63:0/2191517157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642fadcb210 con 0x5642fad924d0 2024-05-27T08:05:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.416+0000 7f8ede000640 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb168740 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc02a9f0 tx=0x7f8ecc02a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48aeeccb85bde67a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.416+0000 7f8ec8c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.417+0000 7f8ec8c00640 1 -- 172.21.15.63:0/2191517157 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc035040 con 0x5642fad924d0 2024-05-27T08:05:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.417+0000 7f8ec8c00640 1 -- 172.21.15.63:0/2191517157 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f8ecc04b9e0 con 0x5642fad924d0 2024-05-27T08:05:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.417+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2191517157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed0015070 con 0x5642fad924d0 2024-05-27T08:05:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.417+0000 7f8ec8c00640 1 -- 172.21.15.63:0/2191517157 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8ecc04bbe0 con 0x5642fad924d0 2024-05-27T08:05:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.418+0000 7f8eec9a4740 10 client.21477 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.418+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2191517157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5642fae97620 con 0x5642fad924d0 2024-05-27T08:05:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.422+0000 7f8ec8c00640 1 -- 172.21.15.63:0/2191517157 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ed0015070 con 0x5642fad924d0 2024-05-27T08:05:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.423+0000 7f8ec8c00640 1 -- 172.21.15.63:0/2191517157 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8ecc052070 con 0x5642fad924d0 2024-05-27T08:05:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.423+0000 7f8eec9a4740 10 client.21477 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.423+0000 7f8eec9a4740 10 client.21477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.423+0000 7f8eec9a4740 20 client.21477 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.423+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642fad71d80 0x5642fb17a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.423+0000 7f8eec9a4740 4 client.21477 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.423+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2191517157 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642fb0c8ae0 con 0x5642fad71d80 2024-05-27T08:05:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.424+0000 7f8edea00640 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642fad71d80 0x5642fb17a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.425+0000 7f8edea00640 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642fad71d80 0x5642fb17a4c0 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29668b7d7106388b server_cookie=83095ab7aa22717f in_seq=0 out_seq=0 2024-05-27T08:05:07.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.425+0000 7f8ec8c00640 10 client.21477 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.427+0000 7f8ec8c00640 1 -- 172.21.15.63:0/2191517157 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5642fb0c8ae0 con 0x5642fad71d80 2024-05-27T08:05:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.427+0000 7f8ec8c00640 10 client.21477 handle_command_reply: tid=0 2024-05-27T08:05:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 client.21477 shutdown 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2191517157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 msgr2=0x5642fb168740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb168740 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc02a9f0 tx=0x7f8ecc02a400 comp rx=0 tx=0).stop 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2191517157 shutdown_connections 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642fad71d80 0x5642fb17a4c0 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fadb31b0 0x5642fb174500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad924d0 0x5642fb168740 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/2191517157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb168200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2191517157 >> 172.21.15.63:0/2191517157 conn(0x5642fada7300 msgr2=0x5642fb157510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8ec8c00640 0 client.21477 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2191517157 shutdown_connections 2024-05-27T08:05:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.428+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2191517157 wait complete. 2024-05-27T08:05:07.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.429+0000 7f8eec9a4740 20 client.21477 trim_cache size 0 max 16384 2024-05-27T08:05:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.456+0000 7f8eec9a4740 1 Processor -- start 2024-05-27T08:05:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.456+0000 7f8eec9a4740 1 -- start start 2024-05-27T08:05:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.456+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad8a2f0 0x5642fb163a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.456+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fad924d0 0x5642fb174270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.456+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fadb31b0 0x5642fb1747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.456+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b1840 con 0x5642fad8a2f0 2024-05-27T08:05:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01af110 con 0x5642fadb31b0 2024-05-27T08:05:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01ad000 con 0x5642fad924d0 2024-05-27T08:05:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8edea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad8a2f0 0x5642fb163a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8edea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad8a2f0 0x5642fb163a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43468/0 (socket says 172.21.15.63:43468) 2024-05-27T08:05:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8edea00640 1 -- 172.21.15.63:0/2050578452 learned_addr learned my addr 172.21.15.63:0/2050578452 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8edf400640 1 --2- 172.21.15.63:0/2050578452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fadb31b0 0x5642fb1747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8edea00640 1 -- 172.21.15.63:0/2050578452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fad924d0 msgr2=0x5642fb174270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8edea00640 1 --2- 172.21.15.63:0/2050578452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fad924d0 0x5642fb174270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8edea00640 1 -- 172.21.15.63:0/2050578452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fadb31b0 msgr2=0x5642fb1747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8edea00640 1 --2- 172.21.15.63:0/2050578452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fadb31b0 0x5642fb1747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.457+0000 7f8edea00640 1 -- 172.21.15.63:0/2050578452 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642fad71fb0 con 0x5642fad8a2f0 2024-05-27T08:05:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.458+0000 7f8edea00640 1 --2- 172.21.15.63:0/2050578452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad8a2f0 0x5642fb163a40 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0004500 tx=0x7f8ed0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df42517c78e1b8fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.458+0000 7f8ecb400640 1 -- 172.21.15.63:0/2050578452 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0003af0 con 0x5642fad8a2f0 2024-05-27T08:05:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.458+0000 7f8ecb400640 1 -- 172.21.15.63:0/2050578452 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ed0015070 con 0x5642fad8a2f0 2024-05-27T08:05:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.458+0000 7f8ecb400640 1 -- 172.21.15.63:0/2050578452 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed00048e0 con 0x5642fad8a2f0 2024-05-27T08:05:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.459+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2050578452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad8a2f0 msgr2=0x5642fb163a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.459+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/2050578452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad8a2f0 0x5642fb163a40 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0004500 tx=0x7f8ed0013040 comp rx=0 tx=0).stop 2024-05-27T08:05:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.460+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2050578452 shutdown_connections 2024-05-27T08:05:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.460+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/2050578452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fadb31b0 0x5642fb1747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.460+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/2050578452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fad924d0 0x5642fb174270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.460+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/2050578452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fad8a2f0 0x5642fb163a40 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.460+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2050578452 >> 172.21.15.63:0/2050578452 conn(0x5642fada7300 msgr2=0x5642fb16a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.460+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2050578452 shutdown_connections 2024-05-27T08:05:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.460+0000 7f8eec9a4740 1 -- 172.21.15.63:0/2050578452 wait complete. 2024-05-27T08:05:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.460+0000 7f8eec9a4740 1 -- start start 2024-05-27T08:05:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.461+0000 7f8eec9a4740 1 Processor -- start 2024-05-27T08:05:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.461+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb16d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.461+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fad924d0 0x5642fb16d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.462+0000 7f8eec9a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fadb31b0 0x5642fb16dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.462+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01ad650 con 0x5642fadb31b0 2024-05-27T08:05:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.462+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01aea40 con 0x5642fad8a2f0 2024-05-27T08:05:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.462+0000 7f8eec9a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01adca0 con 0x5642fad924d0 2024-05-27T08:05:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.462+0000 7f8eec9a4740 10 client.0 _ll_register_callbacks cb 0x7f8ecc003900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.462+0000 7f8edf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fadb31b0 0x5642fb16dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.462+0000 7f8edf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fadb31b0 0x5642fb16dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43478/0 (socket says 172.21.15.63:43478) 2024-05-27T08:05:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.462+0000 7f8edf400640 1 -- 172.21.15.63:0/711199273 learned_addr learned my addr 172.21.15.63:0/711199273 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.462+0000 7f8ede000640 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fad924d0 0x5642fb16d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.462+0000 7f8edea00640 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb16d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.463+0000 7f8edf400640 1 -- 172.21.15.63:0/711199273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fad924d0 msgr2=0x5642fb16d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.463+0000 7f8edf400640 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fad924d0 0x5642fb16d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.463+0000 7f8edf400640 1 -- 172.21.15.63:0/711199273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 msgr2=0x5642fb16d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.463+0000 7f8edf400640 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb16d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.463+0000 7f8edf400640 1 -- 172.21.15.63:0/711199273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642fb1e0c20 con 0x5642fadb31b0 2024-05-27T08:05:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.464+0000 7f8edf400640 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fadb31b0 0x5642fb16dd90 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80a6860 tx=0x7f8ed80a8e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18687d298640c52f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.464+0000 7f8ec8c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.464+0000 7f8ec8c00640 1 -- 172.21.15.63:0/711199273 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80e2070 con 0x5642fadb31b0 2024-05-27T08:05:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.464+0000 7f8ec8c00640 1 -- 172.21.15.63:0/711199273 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f8ed80a5880 con 0x5642fadb31b0 2024-05-27T08:05:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.464+0000 7f8eec9a4740 1 -- 172.21.15.63:0/711199273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed0015070 con 0x5642fadb31b0 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.464+0000 7f8ec8c00640 1 -- 172.21.15.63:0/711199273 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8ed8020040 con 0x5642fadb31b0 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.465+0000 7f8eec9a4740 10 client.21483 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.465+0000 7f8eec9a4740 1 -- 172.21.15.63:0/711199273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5642fb1e1e90 con 0x5642fadb31b0 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.465+0000 7f8ec8c00640 1 -- 172.21.15.63:0/711199273 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ed0015070 con 0x5642fadb31b0 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.465+0000 7f8ec8c00640 1 -- 172.21.15.63:0/711199273 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8ed808fb10 con 0x5642fadb31b0 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.465+0000 7f8eec9a4740 10 client.21483 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.465+0000 7f8eec9a4740 10 client.21483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.465+0000 7f8eec9a4740 20 client.21483 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.465+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642fad471c0 0x5642fb1e4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.465+0000 7f8eec9a4740 4 client.21483 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:05:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.465+0000 7f8eec9a4740 1 -- 172.21.15.63:0/711199273 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642fb0c22c0 con 0x5642fad471c0 2024-05-27T08:05:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.466+0000 7f8edea00640 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642fad471c0 0x5642fb1e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.467+0000 7f8edea00640 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642fad471c0 0x5642fb1e4670 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a11f8d1bb86c5b1 server_cookie=7a9398def5e61370 in_seq=0 out_seq=0 2024-05-27T08:05:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.467+0000 7f8ec8c00640 10 client.21483 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:07.479 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8ec8c00640 1 -- 172.21.15.63:0/711199273 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5642fb0c22c0 con 0x5642fad471c0 2024-05-27T08:05:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8ec8c00640 10 client.21483 handle_command_reply: tid=0 2024-05-27T08:05:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 client.21483 shutdown 2024-05-27T08:05:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 -- 172.21.15.63:0/711199273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fadb31b0 msgr2=0x5642fb16dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fadb31b0 0x5642fb16dd90 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80a6860 tx=0x7f8ed80a8e20 comp rx=0 tx=0).stop 2024-05-27T08:05:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 -- 172.21.15.63:0/711199273 shutdown_connections 2024-05-27T08:05:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5642fad471c0 0x5642fb1e4670 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5642fadb31b0 0x5642fb16dd90 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fad924d0 0x5642fb16d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 --2- 172.21.15.63:0/711199273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642fad8a2f0 0x5642fb16d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 -- 172.21.15.63:0/711199273 >> 172.21.15.63:0/711199273 conn(0x5642fada7300 msgr2=0x5642fb173260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8ec8c00640 0 client.21483 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 -- 172.21.15.63:0/711199273 shutdown_connections 2024-05-27T08:05:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.497+0000 7f8eec9a4740 1 -- 172.21.15.63:0/711199273 wait complete. 2024-05-27T08:05:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.498+0000 7f8eec9a4740 20 client.21483 trim_cache size 0 max 16384 2024-05-27T08:05:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:05:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:05:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:05:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.498+0000 7f8ee5a00640 1 -- 172.21.15.63:0/511257800 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ec40b2950 msgr2=0x7f8ec40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.498+0000 7f8ee5a00640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ec40b2950 0x7f8ec40b4e60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0057c0 tx=0x7f8ecc009560 comp rx=0 tx=0).stop 2024-05-27T08:05:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.498+0000 7f8ee5a00640 1 -- 172.21.15.63:0/511257800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ee010a150 msgr2=0x7f8ee01a8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.498+0000 7f8ee5a00640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ee010a150 0x7f8ee01a8700 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009fd0 tx=0x7f8ed800b260 comp rx=0 tx=0).stop 2024-05-27T08:05:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.499+0000 7f8ee5a00640 1 -- 172.21.15.63:0/511257800 shutdown_connections 2024-05-27T08:05:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.499+0000 7f8ee5a00640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ec40b2950 0x7f8ec40b4e60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.499+0000 7f8ee5a00640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ee010a150 0x7f8ee01a8700 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.499+0000 7f8ee5a00640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ee0109cb0 0x7f8ee019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.499+0000 7f8ee5a00640 1 --2- 172.21.15.63:0/511257800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee0104400 0x7f8ee019d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.499+0000 7f8ee5a00640 1 -- 172.21.15.63:0/511257800 >> 172.21.15.63:0/511257800 conn(0x7f8ee00ff480 msgr2=0x7f8ee0074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.499+0000 7f8ee5a00640 1 -- 172.21.15.63:0/511257800 shutdown_connections 2024-05-27T08:05:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:07.499+0000 7f8ee5a00640 1 -- 172.21.15.63:0/511257800 wait complete. 2024-05-27T08:05:07.675 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:07 smithi063 ceph-mon[31613]: pgmap v1000: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 29 op/s 2024-05-27T08:05:07.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:07 smithi086 ceph-mon[34687]: pgmap v1000: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 29 op/s 2024-05-27T08:05:07.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:07 smithi149 ceph-mon[35408]: pgmap v1000: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 29 op/s 2024-05-27T08:05:09.621 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:09 smithi063 ceph-mon[31613]: pgmap v1001: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 22 op/s 2024-05-27T08:05:09.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:09 smithi086 ceph-mon[34687]: pgmap v1001: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 22 op/s 2024-05-27T08:05:09.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:09 smithi149 ceph-mon[35408]: pgmap v1001: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 22 op/s 2024-05-27T08:05:09.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.887+0000 7fb52ec00700 1 -- 172.21.15.63:0/623727197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528106630 msgr2=0x7fb528106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:09.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.887+0000 7fb52ec00700 1 --2- 172.21.15.63:0/623727197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528106630 0x7fb528106a40 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fb5180099c0 tx=0x7fb518009cd0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.887+0000 7fb52ec00700 1 -- 172.21.15.63:0/623727197 shutdown_connections 2024-05-27T08:05:09.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.887+0000 7fb52ec00700 1 --2- 172.21.15.63:0/623727197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52810b8b0 0x7fb528113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.887+0000 7fb52ec00700 1 --2- 172.21.15.63:0/623727197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528106630 0x7fb528106a40 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.887+0000 7fb52ec00700 1 --2- 172.21.15.63:0/623727197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb52810af60 0x7fb52810b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.887+0000 7fb52ec00700 1 -- 172.21.15.63:0/623727197 >> 172.21.15.63:0/623727197 conn(0x7fb528101ec0 msgr2=0x7fb528104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:09.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.887+0000 7fb52ec00700 1 -- 172.21.15.63:0/623727197 shutdown_connections 2024-05-27T08:05:09.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.887+0000 7fb52ec00700 1 -- 172.21.15.63:0/623727197 wait complete. 2024-05-27T08:05:09.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.889+0000 7fb52ec00700 1 Processor -- start 2024-05-27T08:05:09.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.889+0000 7fb52ec00700 1 -- start start 2024-05-27T08:05:09.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.890+0000 7fb52ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb528106630 0x7fb52819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.890+0000 7fb52ec00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb52810af60 0x7fb52819d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.890+0000 7fb52ec00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb52810b8b0 0x7fb5281a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.890+0000 7fb52ec00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281163e0 con 0x7fb52810b8b0 2024-05-27T08:05:09.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.890+0000 7fb52ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb528116260 con 0x7fb528106630 2024-05-27T08:05:09.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.890+0000 7fb52ec00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb528116560 con 0x7fb52810af60 2024-05-27T08:05:09.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.890+0000 7fb52e200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb52810b8b0 0x7fb5281a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.890+0000 7fb52e200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb52810b8b0 0x7fb5281a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57266/0 (socket says 172.21.15.63:57266) 2024-05-27T08:05:09.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.890+0000 7fb52e200700 1 -- 172.21.15.63:0/3925859474 learned_addr learned my addr 172.21.15.63:0/3925859474 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:09.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.891+0000 7fb52d800700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb528106630 0x7fb52819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.891+0000 7fb52e200700 1 -- 172.21.15.63:0/3925859474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb52810af60 msgr2=0x7fb52819d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:09.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.891+0000 7fb52e200700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb52810af60 0x7fb52819d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.891+0000 7fb52e200700 1 -- 172.21.15.63:0/3925859474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb528106630 msgr2=0x7fb52819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:09.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.891+0000 7fb52e200700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb528106630 0x7fb52819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.891+0000 7fb52e200700 1 -- 172.21.15.63:0/3925859474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5281a1c40 con 0x7fb52810b8b0 2024-05-27T08:05:09.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.891+0000 7fb52e200700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb52810b8b0 0x7fb5281a15a0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fb520007ee0 tx=0x7fb52000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:09.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.892+0000 7fb526000700 1 -- 172.21.15.63:0/3925859474 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52000dbd0 con 0x7fb52810b8b0 2024-05-27T08:05:09.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.892+0000 7fb526000700 1 -- 172.21.15.63:0/3925859474 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb52000dd70 con 0x7fb52810b8b0 2024-05-27T08:05:09.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.892+0000 7fb52ec00700 1 -- 172.21.15.63:0/3925859474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5281a1f30 con 0x7fb52810b8b0 2024-05-27T08:05:09.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.892+0000 7fb526000700 1 -- 172.21.15.63:0/3925859474 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200058d0 con 0x7fb52810b8b0 2024-05-27T08:05:09.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.892+0000 7fb52ec00700 1 -- 172.21.15.63:0/3925859474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5281a9850 con 0x7fb52810b8b0 2024-05-27T08:05:09.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.896+0000 7fb526000700 1 -- 172.21.15.63:0/3925859474 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb52001a020 con 0x7fb52810b8b0 2024-05-27T08:05:09.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.897+0000 7fb53ecffb80 1 Processor -- start 2024-05-27T08:05:09.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.897+0000 7fb53ecffb80 1 -- start start 2024-05-27T08:05:09.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3d7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8410 0x564a9f3e2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8820 0x564a9f3e5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564a9f306e40 con 0x564a9f259e40 2024-05-27T08:05:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a9f2f69e0 con 0x564a9f3d8820 2024-05-27T08:05:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564a9f2faa20 con 0x564a9f3d8410 2024-05-27T08:05:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb526000700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b29f0 0x7fb50c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb52ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3d7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb52ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3d7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57268/0 (socket says 172.21.15.63:57268) 2024-05-27T08:05:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb52ce00700 1 -- 172.21.15.63:0/1697043122 learned_addr learned my addr 172.21.15.63:0/1697043122 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.898+0000 7fb526000700 1 -- 172.21.15.63:0/3925859474 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fb5200e2190 con 0x7fb52810b8b0 2024-05-27T08:05:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.899+0000 7fb52d800700 1 --2- 172.21.15.63:0/1697043122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8410 0x564a9f3e2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.899+0000 7fb52ce00700 1 --2- 172.21.15.63:0/1697043122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8820 0x564a9f3e5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.899+0000 7fb52ce00700 1 -- 172.21.15.63:0/1697043122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8410 msgr2=0x564a9f3e2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:09.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.899+0000 7fb52ce00700 1 --2- 172.21.15.63:0/1697043122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8410 0x564a9f3e2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.899+0000 7fb52e200700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b29f0 0x7fb50c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.899+0000 7fb52ce00700 1 -- 172.21.15.63:0/1697043122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8820 msgr2=0x564a9f3e5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:09.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.899+0000 7fb52ce00700 1 --2- 172.21.15.63:0/1697043122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8820 0x564a9f3e5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.899+0000 7fb52ce00700 1 -- 172.21.15.63:0/1697043122 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a9f2bf420 con 0x564a9f259e40 2024-05-27T08:05:09.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.900+0000 7fb52ce00700 1 --2- 172.21.15.63:0/1697043122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3d7ed0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fb5140144e0 tx=0x7fb51400b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ac4e3e79da5b8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:09.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.900+0000 7fb51ea00700 1 -- 172.21.15.63:0/1697043122 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514014cc0 con 0x564a9f259e40 2024-05-27T08:05:09.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.900+0000 7fb51ea00700 1 -- 172.21.15.63:0/1697043122 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb514004cc0 con 0x564a9f259e40 2024-05-27T08:05:09.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.900+0000 7fb51ea00700 1 -- 172.21.15.63:0/1697043122 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514005540 con 0x564a9f259e40 2024-05-27T08:05:09.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.901+0000 7fb53ecffb80 1 -- 172.21.15.63:0/1697043122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 msgr2=0x564a9f3d7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:09.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.901+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/1697043122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3d7ed0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fb5140144e0 tx=0x7fb51400b550 comp rx=0 tx=0).stop 2024-05-27T08:05:09.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.901+0000 7fb52e200700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b29f0 0x7fb50c0b4f00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb520004fd0 tx=0x7fb520004e10 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:09.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.901+0000 7fb53ecffb80 1 -- 172.21.15.63:0/1697043122 shutdown_connections 2024-05-27T08:05:09.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.901+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/1697043122 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8820 0x564a9f3e5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.901+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/1697043122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3d7ed0 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.901+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/1697043122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8410 0x564a9f3e2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.901+0000 7fb53ecffb80 1 -- 172.21.15.63:0/1697043122 >> 172.21.15.63:0/1697043122 conn(0x564a9f2b9fc0 msgr2=0x564a9f3d3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:09.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.902+0000 7fb53ecffb80 1 -- 172.21.15.63:0/1697043122 shutdown_connections 2024-05-27T08:05:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.902+0000 7fb53ecffb80 1 -- 172.21.15.63:0/1697043122 wait complete. 2024-05-27T08:05:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.902+0000 7fb53ecffb80 1 -- start start 2024-05-27T08:05:09.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.903+0000 7fb53ecffb80 1 Processor -- start 2024-05-27T08:05:09.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.903+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.903+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.903+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8820 0x564a9f3e3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.903+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564a9f1fcfa0 con 0x564a9f259e40 2024-05-27T08:05:09.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.903+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a9f297760 con 0x564a9f3d8410 2024-05-27T08:05:09.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.903+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x564a9f3e5990 con 0x564a9f3d8820 2024-05-27T08:05:09.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.903+0000 7fb53ecffb80 10 client.0 _ll_register_callbacks cb 0x564a9f2a0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:09.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.904+0000 7fb52ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.904+0000 7fb52ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3e25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57282/0 (socket says 172.21.15.63:57282) 2024-05-27T08:05:09.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.904+0000 7fb52ce00700 1 -- 172.21.15.63:0/905426054 learned_addr learned my addr 172.21.15.63:0/905426054 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:09.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.904+0000 7fb52d800700 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.904+0000 7fb52ce00700 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8820 0x564a9f3e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.904+0000 7fb52ce00700 1 -- 172.21.15.63:0/905426054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8820 msgr2=0x564a9f3e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.904+0000 7fb52ce00700 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8820 0x564a9f3e3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.905+0000 7fb52ce00700 1 -- 172.21.15.63:0/905426054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 msgr2=0x564a9f3e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.905+0000 7fb52ce00700 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3e2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.905+0000 7fb52ce00700 1 -- 172.21.15.63:0/905426054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a9f2468d0 con 0x564a9f259e40 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.905+0000 7fb52ce00700 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3e25b0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fb51400b520 tx=0x7fb51400b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc288c91c3e7c27 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.905+0000 7fb513400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.905+0000 7fb513400700 1 -- 172.21.15.63:0/905426054 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514029060 con 0x564a9f259e40 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.905+0000 7fb513400700 1 -- 172.21.15.63:0/905426054 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fb514005230 con 0x564a9f259e40 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.905+0000 7fb53ecffb80 1 -- 172.21.15.63:0/905426054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb514004cc0 con 0x564a9f259e40 2024-05-27T08:05:09.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb513400700 1 -- 172.21.15.63:0/905426054 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb514005430 con 0x564a9f259e40 2024-05-27T08:05:09.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb53ecffb80 10 client.21495 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:09.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb53ecffb80 1 -- 172.21.15.63:0/905426054 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564a9f2c7060 con 0x564a9f259e40 2024-05-27T08:05:09.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb513400700 1 -- 172.21.15.63:0/905426054 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb514004cc0 con 0x564a9f259e40 2024-05-27T08:05:09.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb513400700 1 -- 172.21.15.63:0/905426054 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb514021040 con 0x564a9f259e40 2024-05-27T08:05:09.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb53ecffb80 10 client.21495 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:09.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb53ecffb80 10 client.21495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:09.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb53ecffb80 20 client.21495 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:09.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9f3f2900 0x564a9f3f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb53ecffb80 4 client.21495 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:09.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.906+0000 7fb53ecffb80 1 -- 172.21.15.63:0/905426054 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a9f308d90 con 0x564a9f3f2900 2024-05-27T08:05:09.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.908+0000 7fb52d800700 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9f3f2900 0x564a9f3f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.909+0000 7fb52d800700 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9f3f2900 0x564a9f3f4d20 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85e902aef1f1bdc server_cookie=7a53059bd7e0fdd in_seq=0 out_seq=0 2024-05-27T08:05:09.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.909+0000 7fb513400700 10 client.21495 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:09.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.911+0000 7fb513400700 1 -- 172.21.15.63:0/905426054 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564a9f308d90 con 0x564a9f3f2900 2024-05-27T08:05:09.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.911+0000 7fb513400700 10 client.21495 handle_command_reply: tid=0 2024-05-27T08:05:09.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.911+0000 7fb53ecffb80 1 client.21495 shutdown 2024-05-27T08:05:09.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.911+0000 7fb53ecffb80 1 -- 172.21.15.63:0/905426054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 msgr2=0x564a9f3e25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:09.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.911+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3e25b0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fb51400b520 tx=0x7fb51400b2d0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.912+0000 7fb53ecffb80 1 -- 172.21.15.63:0/905426054 shutdown_connections 2024-05-27T08:05:09.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.912+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9f3f2900 0x564a9f3f4d20 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.912+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8820 0x564a9f3e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.912+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3e25b0 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.912+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/905426054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.912+0000 7fb513400700 0 client.21495 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.912+0000 7fb53ecffb80 1 -- 172.21.15.63:0/905426054 >> 172.21.15.63:0/905426054 conn(0x564a9f2b9fc0 msgr2=0x564a9f3d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.912+0000 7fb53ecffb80 1 -- 172.21.15.63:0/905426054 shutdown_connections 2024-05-27T08:05:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.912+0000 7fb53ecffb80 1 -- 172.21.15.63:0/905426054 wait complete. 2024-05-27T08:05:09.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.912+0000 7fb53ecffb80 20 client.21495 trim_cache size 0 max 16384 2024-05-27T08:05:09.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.935+0000 7fb53ecffb80 1 Processor -- start 2024-05-27T08:05:09.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.935+0000 7fb53ecffb80 1 -- start start 2024-05-27T08:05:09.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.936+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f259e40 0x564a9f3dddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.936+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.936+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f3d8820 0x564a9f3e4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.936+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b4e60 con 0x564a9f3d8820 2024-05-27T08:05:09.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.936+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b26f0 con 0x564a9f3d8410 2024-05-27T08:05:09.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.936+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b0560 con 0x564a9f259e40 2024-05-27T08:05:09.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.936+0000 7fb52ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f3d8820 0x564a9f3e4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.936+0000 7fb52ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f3d8820 0x564a9f3e4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57294/0 (socket says 172.21.15.63:57294) 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.936+0000 7fb52ce00700 1 -- 172.21.15.63:0/3739221918 learned_addr learned my addr 172.21.15.63:0/3739221918 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.936+0000 7fb52ce00700 1 --2- 172.21.15.63:0/3739221918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f259e40 0x564a9f3dddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.937+0000 7fb52d800700 1 --2- 172.21.15.63:0/3739221918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.937+0000 7fb52ce00700 1 -- 172.21.15.63:0/3739221918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f259e40 msgr2=0x564a9f3dddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.937+0000 7fb52ce00700 1 --2- 172.21.15.63:0/3739221918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f259e40 0x564a9f3dddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.937+0000 7fb52ce00700 1 -- 172.21.15.63:0/3739221918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 msgr2=0x564a9f3e2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.937+0000 7fb52ce00700 1 --2- 172.21.15.63:0/3739221918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3e2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.937+0000 7fb52ce00700 1 -- 172.21.15.63:0/3739221918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a9f3ce1d0 con 0x564a9f3d8820 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.937+0000 7fb52d800700 1 --2- 172.21.15.63:0/3739221918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3e2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.937+0000 7fb52ce00700 1 --2- 172.21.15.63:0/3739221918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f3d8820 0x564a9f3e4a50 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x564a9f3f04f0 tx=0x7fb51400b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da831f2bc2ddf759 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.937+0000 7fb51e000700 1 -- 172.21.15.63:0/3739221918 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514029070 con 0x564a9f3d8820 2024-05-27T08:05:09.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.938+0000 7fb51e000700 1 -- 172.21.15.63:0/3739221918 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb51400e410 con 0x564a9f3d8820 2024-05-27T08:05:09.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.938+0000 7fb51e000700 1 -- 172.21.15.63:0/3739221918 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51400e690 con 0x564a9f3d8820 2024-05-27T08:05:09.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.939+0000 7fb53ecffb80 1 -- 172.21.15.63:0/3739221918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f3d8820 msgr2=0x564a9f3e4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:09.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.939+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/3739221918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f3d8820 0x564a9f3e4a50 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x564a9f3f04f0 tx=0x7fb51400b4b0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.939+0000 7fb53ecffb80 1 -- 172.21.15.63:0/3739221918 shutdown_connections 2024-05-27T08:05:09.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.939+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/3739221918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f3d8820 0x564a9f3e4a50 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.939+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/3739221918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f259e40 0x564a9f3dddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.939+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/3739221918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3e2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.939+0000 7fb53ecffb80 1 -- 172.21.15.63:0/3739221918 >> 172.21.15.63:0/3739221918 conn(0x564a9f2b9fc0 msgr2=0x564a9f3e0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.939+0000 7fb53ecffb80 1 -- 172.21.15.63:0/3739221918 shutdown_connections 2024-05-27T08:05:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.939+0000 7fb53ecffb80 1 -- 172.21.15.63:0/3739221918 wait complete. 2024-05-27T08:05:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.940+0000 7fb53ecffb80 1 -- start start 2024-05-27T08:05:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.940+0000 7fb53ecffb80 1 Processor -- start 2024-05-27T08:05:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb53ecffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8820 0x564a9f45b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b0bb0 con 0x564a9f259e40 2024-05-27T08:05:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b2020 con 0x564a9f3d8410 2024-05-27T08:05:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb53ecffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b1280 con 0x564a9f3d8820 2024-05-27T08:05:09.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb53ecffb80 10 client.0 _ll_register_callbacks cb 0x564a9f2a0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:09.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb52ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb52ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57310/0 (socket says 172.21.15.63:57310) 2024-05-27T08:05:09.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb52ce00700 1 -- 172.21.15.63:0/2237369261 learned_addr learned my addr 172.21.15.63:0/2237369261 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:09.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb52d800700 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.941+0000 7fb52ce00700 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8820 0x564a9f45b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.942+0000 7fb52ce00700 1 -- 172.21.15.63:0/2237369261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8820 msgr2=0x564a9f45b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.942+0000 7fb52ce00700 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8820 0x564a9f45b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.942+0000 7fb52ce00700 1 -- 172.21.15.63:0/2237369261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 msgr2=0x564a9f3f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.942+0000 7fb52ce00700 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.942+0000 7fb52ce00700 1 -- 172.21.15.63:0/2237369261 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a9f3d5350 con 0x564a9f259e40 2024-05-27T08:05:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.942+0000 7fb52ce00700 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3f51b0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fb51400b700 tx=0x7fb51400b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fd675f85287aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.942+0000 7fb513400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.943+0000 7fb513400700 1 -- 172.21.15.63:0/2237369261 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514029070 con 0x564a9f259e40 2024-05-27T08:05:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.943+0000 7fb513400700 1 -- 172.21.15.63:0/2237369261 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fb5140056e0 con 0x564a9f259e40 2024-05-27T08:05:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.943+0000 7fb53ecffb80 1 -- 172.21.15.63:0/2237369261 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb51400e410 con 0x564a9f259e40 2024-05-27T08:05:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.943+0000 7fb513400700 1 -- 172.21.15.63:0/2237369261 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb514023670 con 0x564a9f259e40 2024-05-27T08:05:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.943+0000 7fb53ecffb80 10 client.21501 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.943+0000 7fb53ecffb80 1 -- 172.21.15.63:0/2237369261 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564a9f45c5f0 con 0x564a9f259e40 2024-05-27T08:05:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.944+0000 7fb513400700 1 -- 172.21.15.63:0/2237369261 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb51400e410 con 0x564a9f259e40 2024-05-27T08:05:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.944+0000 7fb513400700 1 -- 172.21.15.63:0/2237369261 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb514034420 con 0x564a9f259e40 2024-05-27T08:05:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.944+0000 7fb53ecffb80 10 client.21501 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.944+0000 7fb53ecffb80 10 client.21501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.944+0000 7fb53ecffb80 20 client.21501 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.944+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9f45c940 0x564a9f45ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.944+0000 7fb53ecffb80 4 client.21501 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:05:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.944+0000 7fb53ecffb80 1 -- 172.21.15.63:0/2237369261 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a9f302570 con 0x564a9f45c940 2024-05-27T08:05:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.945+0000 7fb52d800700 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9f45c940 0x564a9f45ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.946+0000 7fb52d800700 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9f45c940 0x564a9f45ed60 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2129cb4bf91d0ba3 server_cookie=cd9f5104f6865dda in_seq=0 out_seq=0 2024-05-27T08:05:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.946+0000 7fb513400700 10 client.21501 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.946+0000 7fb513400700 1 -- 172.21.15.63:0/2237369261 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a9f302570 con 0x564a9f45c940 2024-05-27T08:05:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.946+0000 7fb513400700 10 client.21501 handle_command_reply: tid=0 2024-05-27T08:05:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.946+0000 7fb53ecffb80 1 client.21501 shutdown 2024-05-27T08:05:09.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 1 -- 172.21.15.63:0/2237369261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 msgr2=0x564a9f3f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:09.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3f51b0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fb51400b700 tx=0x7fb51400b2d0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 1 -- 172.21.15.63:0/2237369261 shutdown_connections 2024-05-27T08:05:09.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564a9f45c940 0x564a9f45ed60 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564a9f3d8820 0x564a9f45b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a9f259e40 0x564a9f3f51b0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 1 --2- 172.21.15.63:0/2237369261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a9f3d8410 0x564a9f3f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 1 -- 172.21.15.63:0/2237369261 >> 172.21.15.63:0/2237369261 conn(0x564a9f2b9fc0 msgr2=0x564a9f3e2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:09.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb513400700 0 client.21501 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:09.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 1 -- 172.21.15.63:0/2237369261 shutdown_connections 2024-05-27T08:05:09.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 1 -- 172.21.15.63:0/2237369261 wait complete. 2024-05-27T08:05:09.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.947+0000 7fb53ecffb80 20 client.21501 trim_cache size 0 max 16384 2024-05-27T08:05:09.950 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:05:09.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.948+0000 7fb52ec00700 1 -- 172.21.15.63:0/3925859474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b29f0 msgr2=0x7fb50c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:09.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.948+0000 7fb52ec00700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b29f0 0x7fb50c0b4f00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb520004fd0 tx=0x7fb520004e10 comp rx=0 tx=0).stop 2024-05-27T08:05:09.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.948+0000 7fb52ec00700 1 -- 172.21.15.63:0/3925859474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb52810b8b0 msgr2=0x7fb5281a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:09.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.948+0000 7fb52ec00700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb52810b8b0 0x7fb5281a15a0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fb520007ee0 tx=0x7fb52000d3b0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.948+0000 7fb52ec00700 1 -- 172.21.15.63:0/3925859474 shutdown_connections 2024-05-27T08:05:09.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.948+0000 7fb52ec00700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b29f0 0x7fb50c0b4f00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.948+0000 7fb52ec00700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb52810b8b0 0x7fb5281a15a0 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.948+0000 7fb52ec00700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb528106630 0x7fb52819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.948+0000 7fb52ec00700 1 --2- 172.21.15.63:0/3925859474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb52810af60 0x7fb52819d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:09.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.948+0000 7fb52ec00700 1 -- 172.21.15.63:0/3925859474 >> 172.21.15.63:0/3925859474 conn(0x7fb528101ec0 msgr2=0x7fb52810c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:09.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.949+0000 7fb52ec00700 1 -- 172.21.15.63:0/3925859474 shutdown_connections 2024-05-27T08:05:09.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:09.949+0000 7fb52ec00700 1 -- 172.21.15.63:0/3925859474 wait complete. 2024-05-27T08:05:11.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:11 smithi086 ceph-mon[34687]: pgmap v1002: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 22 op/s 2024-05-27T08:05:11.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:11 smithi149 ceph-mon[35408]: pgmap v1002: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 22 op/s 2024-05-27T08:05:11.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:11 smithi063 ceph-mon[31613]: pgmap v1002: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 22 op/s 2024-05-27T08:05:11.960 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:05:11.961 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:05:12.466 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:13.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:13 smithi086 ceph-mon[34687]: pgmap v1003: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.7 MiB/s wr, 29 op/s 2024-05-27T08:05:13.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:13 smithi149 ceph-mon[35408]: pgmap v1003: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.7 MiB/s wr, 29 op/s 2024-05-27T08:05:13.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:13 smithi063 ceph-mon[31613]: pgmap v1003: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.7 MiB/s wr, 29 op/s 2024-05-27T08:05:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.918+0000 7f32a8600700 1 -- 172.21.15.63:0/2143864184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0108730 msgr2=0x7f32a0110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.918+0000 7f32a8600700 1 --2- 172.21.15.63:0/2143864184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0108730 0x7f32a0110090 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f329800b0d0 tx=0x7f329800b3e0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.918+0000 7f32a8600700 1 -- 172.21.15.63:0/2143864184 shutdown_connections 2024-05-27T08:05:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.918+0000 7f32a8600700 1 --2- 172.21.15.63:0/2143864184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0108730 0x7f32a0110090 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.918+0000 7f32a8600700 1 --2- 172.21.15.63:0/2143864184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a01034b0 0x7f32a01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.918+0000 7f32a8600700 1 --2- 172.21.15.63:0/2143864184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a0107de0 0x7f32a01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.918+0000 7f32a8600700 1 -- 172.21.15.63:0/2143864184 >> 172.21.15.63:0/2143864184 conn(0x7f32a00fedb0 msgr2=0x7f32a01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.918+0000 7f32a8600700 1 -- 172.21.15.63:0/2143864184 shutdown_connections 2024-05-27T08:05:14.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.918+0000 7f32a8600700 1 -- 172.21.15.63:0/2143864184 wait complete. 2024-05-27T08:05:14.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.920+0000 7f32a8600700 1 Processor -- start 2024-05-27T08:05:14.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.920+0000 7f32a8600700 1 -- start start 2024-05-27T08:05:14.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.920+0000 7f32a8600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a01034b0 0x7f32a019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.921+0000 7f32a8600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0107de0 0x7f32a019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.921+0000 7f32a8600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a0108730 0x7f32a019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.921+0000 7f32a8600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0113260 con 0x7f32a0107de0 2024-05-27T08:05:14.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.921+0000 7f32a8600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01130e0 con 0x7f32a01034b0 2024-05-27T08:05:14.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.921+0000 7f32a8600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01133e0 con 0x7f32a0108730 2024-05-27T08:05:14.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.921+0000 7f32a5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0107de0 0x7f32a019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.921+0000 7f32a5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0107de0 0x7f32a019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57342/0 (socket says 172.21.15.63:57342) 2024-05-27T08:05:14.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.921+0000 7f32a5200700 1 -- 172.21.15.63:0/4257389249 learned_addr learned my addr 172.21.15.63:0/4257389249 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:14.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.921+0000 7f32a5c00700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a01034b0 0x7f32a019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.921+0000 7f32a6600700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a0108730 0x7f32a019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.922+0000 7f32a5200700 1 -- 172.21.15.63:0/4257389249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a0108730 msgr2=0x7f32a019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.922+0000 7f32a5200700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a0108730 0x7f32a019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.922+0000 7f32a5200700 1 -- 172.21.15.63:0/4257389249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a01034b0 msgr2=0x7f32a019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.922+0000 7f32a5200700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a01034b0 0x7f32a019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.922+0000 7f32a5200700 1 -- 172.21.15.63:0/4257389249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a019fc30 con 0x7f32a0107de0 2024-05-27T08:05:14.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.922+0000 7f32a5200700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0107de0 0x7f32a019b300 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f328c00d980 tx=0x7f328c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.923+0000 7f329e000700 1 -- 172.21.15.63:0/4257389249 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c017070 con 0x7f32a0107de0 2024-05-27T08:05:14.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.923+0000 7f32a8600700 1 -- 172.21.15.63:0/4257389249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a019ff20 con 0x7f32a0107de0 2024-05-27T08:05:14.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.923+0000 7f329e000700 1 -- 172.21.15.63:0/4257389249 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f328c00fd20 con 0x7f32a0107de0 2024-05-27T08:05:14.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.923+0000 7f329e000700 1 -- 172.21.15.63:0/4257389249 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c00e3b0 con 0x7f32a0107de0 2024-05-27T08:05:14.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.923+0000 7f32a8600700 1 -- 172.21.15.63:0/4257389249 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a00ff1c0 con 0x7f32a0107de0 2024-05-27T08:05:14.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.927+0000 7f329e000700 1 -- 172.21.15.63:0/4257389249 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f328c00f890 con 0x7f32a0107de0 2024-05-27T08:05:14.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.928+0000 7f329e000700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32880b29f0 0x7f32880b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.929+0000 7f329e000700 1 -- 172.21.15.63:0/4257389249 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f328c0e23e0 con 0x7f32a0107de0 2024-05-27T08:05:14.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.929+0000 7f32a5c00700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32880b29f0 0x7f32880b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.930+0000 7f32b70cdb80 1 Processor -- start 2024-05-27T08:05:14.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.930+0000 7f32b70cdb80 1 -- start start 2024-05-27T08:05:14.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.930+0000 7f32a5c00700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32880b29f0 0x7f32880b4f00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3294005950 tx=0x7f329400b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.930+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612884ffa70 0x5612885923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.930+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288413ec0 0x5612885928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.930+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 0x56128859f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.930+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5612884c0ec0 con 0x561288413ec0 2024-05-27T08:05:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.930+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612884b0a60 con 0x5612884ffa70 2024-05-27T08:05:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.930+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5612884b4aa0 con 0x561288592ef0 2024-05-27T08:05:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.931+0000 7f32a5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288413ec0 0x5612885928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.931+0000 7f32a5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288413ec0 0x5612885928f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57346/0 (socket says 172.21.15.63:57346) 2024-05-27T08:05:14.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.931+0000 7f32a5200700 1 -- 172.21.15.63:0/2151026145 learned_addr learned my addr 172.21.15.63:0/2151026145 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:14.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.931+0000 7f32a5c00700 1 --2- 172.21.15.63:0/2151026145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612884ffa70 0x5612885923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.931+0000 7f32a6600700 1 --2- 172.21.15.63:0/2151026145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 0x56128859f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.931+0000 7f32a5200700 1 -- 172.21.15.63:0/2151026145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 msgr2=0x56128859f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.931+0000 7f32a5200700 1 --2- 172.21.15.63:0/2151026145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 0x56128859f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.931+0000 7f32a5200700 1 -- 172.21.15.63:0/2151026145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612884ffa70 msgr2=0x5612885923b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.931+0000 7f32a5200700 1 --2- 172.21.15.63:0/2151026145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612884ffa70 0x5612885923b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.931+0000 7f32a5200700 1 -- 172.21.15.63:0/2151026145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612884794a0 con 0x561288413ec0 2024-05-27T08:05:14.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.932+0000 7f32a5200700 1 --2- 172.21.15.63:0/2151026145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288413ec0 0x5612885928f0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f328c0a5380 tx=0x7f328c00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453a158ebbd3744c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.932+0000 7f3292a00700 1 -- 172.21.15.63:0/2151026145 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c020040 con 0x561288413ec0 2024-05-27T08:05:14.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.932+0000 7f3292a00700 1 -- 172.21.15.63:0/2151026145 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f328c0a9a00 con 0x561288413ec0 2024-05-27T08:05:14.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.932+0000 7f3292a00700 1 -- 172.21.15.63:0/2151026145 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c0a97c0 con 0x561288413ec0 2024-05-27T08:05:14.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.933+0000 7f32b70cdb80 1 -- 172.21.15.63:0/2151026145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288413ec0 msgr2=0x5612885928f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.933+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/2151026145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288413ec0 0x5612885928f0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f328c0a5380 tx=0x7f328c00fc00 comp rx=0 tx=0).stop 2024-05-27T08:05:14.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.933+0000 7f32b70cdb80 1 -- 172.21.15.63:0/2151026145 shutdown_connections 2024-05-27T08:05:14.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.933+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/2151026145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 0x56128859f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.933+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/2151026145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612884ffa70 0x5612885923b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.933+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/2151026145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288413ec0 0x5612885928f0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.933+0000 7f32b70cdb80 1 -- 172.21.15.63:0/2151026145 >> 172.21.15.63:0/2151026145 conn(0x5612884ad410 msgr2=0x56128858e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.933+0000 7f32b70cdb80 1 -- 172.21.15.63:0/2151026145 shutdown_connections 2024-05-27T08:05:14.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.934+0000 7f32b70cdb80 1 -- 172.21.15.63:0/2151026145 wait complete. 2024-05-27T08:05:14.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.934+0000 7f32b70cdb80 1 -- start start 2024-05-27T08:05:14.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.934+0000 7f32b70cdb80 1 Processor -- start 2024-05-27T08:05:14.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.935+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x56128859c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.935+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612884ffa70 0x56128859ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.935+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.935+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5612883b7020 con 0x561288592ef0 2024-05-27T08:05:14.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.935+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612884517e0 con 0x561288413ec0 2024-05-27T08:05:14.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.935+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56128859fc00 con 0x5612884ffa70 2024-05-27T08:05:14.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.935+0000 7f32b70cdb80 10 client.0 _ll_register_callbacks cb 0x56128845acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:14.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.936+0000 7f32a6600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.936+0000 7f32a6600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57356/0 (socket says 172.21.15.63:57356) 2024-05-27T08:05:14.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.936+0000 7f32a6600700 1 -- 172.21.15.63:0/66586427 learned_addr learned my addr 172.21.15.63:0/66586427 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:14.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.936+0000 7f32a5200700 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612884ffa70 0x56128859ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.936+0000 7f32a5c00700 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x56128859c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.936+0000 7f32a6600700 1 -- 172.21.15.63:0/66586427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612884ffa70 msgr2=0x56128859ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.936+0000 7f32a6600700 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612884ffa70 0x56128859ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.936+0000 7f32a6600700 1 -- 172.21.15.63:0/66586427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 msgr2=0x56128859c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.936+0000 7f32a6600700 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x56128859c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.936+0000 7f32a6600700 1 -- 172.21.15.63:0/66586427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561288400950 con 0x561288592ef0 2024-05-27T08:05:14.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.937+0000 7f32a6600700 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859d210 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f3298009e20 tx=0x7f3298003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7141c45767f386 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.937+0000 7f3287400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:14.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.937+0000 7f3287400700 1 -- 172.21.15.63:0/66586427 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329802b060 con 0x561288592ef0 2024-05-27T08:05:14.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.937+0000 7f32b70cdb80 1 -- 172.21.15.63:0/66586427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f328c0a9a00 con 0x561288592ef0 2024-05-27T08:05:14.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.937+0000 7f3287400700 1 -- 172.21.15.63:0/66586427 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f329801c6c0 con 0x561288592ef0 2024-05-27T08:05:14.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.938+0000 7f3287400700 1 -- 172.21.15.63:0/66586427 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3298032050 con 0x561288592ef0 2024-05-27T08:05:14.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.938+0000 7f32b70cdb80 10 client.21513 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:14.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.938+0000 7f32b70cdb80 1 -- 172.21.15.63:0/66586427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5612884810e0 con 0x561288592ef0 2024-05-27T08:05:14.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.938+0000 7f3287400700 1 -- 172.21.15.63:0/66586427 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f328c0a9a00 con 0x561288592ef0 2024-05-27T08:05:14.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.938+0000 7f3287400700 1 -- 172.21.15.63:0/66586427 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3298011070 con 0x561288592ef0 2024-05-27T08:05:14.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.938+0000 7f32b70cdb80 10 client.21513 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:14.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.938+0000 7f32b70cdb80 10 client.21513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:14.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.938+0000 7f32b70cdb80 20 client.21513 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:14.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.939+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612885acb10 0x5612885aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.939+0000 7f32b70cdb80 4 client.21513 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:14.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.939+0000 7f32b70cdb80 1 -- 172.21.15.63:0/66586427 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612884c2e10 con 0x5612885acb10 2024-05-27T08:05:14.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.939+0000 7f32a5c00700 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612885acb10 0x5612885aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.940+0000 7f32a5c00700 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612885acb10 0x5612885aef30 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec432275a60ddaf2 server_cookie=a002f5ae4495bf9e in_seq=0 out_seq=0 2024-05-27T08:05:14.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.940+0000 7f3287400700 10 client.21513 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:14.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.942+0000 7f3287400700 1 -- 172.21.15.63:0/66586427 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5612884c2e10 con 0x5612885acb10 2024-05-27T08:05:14.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.942+0000 7f3287400700 10 client.21513 handle_command_reply: tid=0 2024-05-27T08:05:14.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.942+0000 7f32b70cdb80 1 client.21513 shutdown 2024-05-27T08:05:14.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 1 -- 172.21.15.63:0/66586427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 msgr2=0x56128859d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859d210 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f3298009e20 tx=0x7f3298003a00 comp rx=0 tx=0).stop 2024-05-27T08:05:14.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 1 -- 172.21.15.63:0/66586427 shutdown_connections 2024-05-27T08:05:14.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612885acb10 0x5612885aef30 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859d210 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x56128859c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/66586427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612884ffa70 0x56128859ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 1 -- 172.21.15.63:0/66586427 >> 172.21.15.63:0/66586427 conn(0x5612884ad410 msgr2=0x561288592430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f3287400700 0 client.21513 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 1 -- 172.21.15.63:0/66586427 shutdown_connections 2024-05-27T08:05:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 1 -- 172.21.15.63:0/66586427 wait complete. 2024-05-27T08:05:14.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.943+0000 7f32b70cdb80 20 client.21513 trim_cache size 0 max 16384 2024-05-27T08:05:14.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.967+0000 7f32b70cdb80 1 Processor -- start 2024-05-27T08:05:14.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.967+0000 7f32b70cdb80 1 -- start start 2024-05-27T08:05:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x561288594980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612884ffa70 0x5612885a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01b0bf0 con 0x561288592ef0 2024-05-27T08:05:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01ae480 con 0x561288413ec0 2024-05-27T08:05:14.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01ac2f0 con 0x5612884ffa70 2024-05-27T08:05:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32a6600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32a6600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57360/0 (socket says 172.21.15.63:57360) 2024-05-27T08:05:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32a6600700 1 -- 172.21.15.63:0/152681445 learned_addr learned my addr 172.21.15.63:0/152681445 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32a5c00700 1 --2- 172.21.15.63:0/152681445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x561288594980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.968+0000 7f32a5200700 1 --2- 172.21.15.63:0/152681445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612884ffa70 0x5612885a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.969+0000 7f32a6600700 1 -- 172.21.15.63:0/152681445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612884ffa70 msgr2=0x5612885a8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.969+0000 7f32a6600700 1 --2- 172.21.15.63:0/152681445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612884ffa70 0x5612885a8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.969+0000 7f32a6600700 1 -- 172.21.15.63:0/152681445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 msgr2=0x561288594980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.969+0000 7f32a6600700 1 --2- 172.21.15.63:0/152681445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x561288594980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.969+0000 7f32a6600700 1 -- 172.21.15.63:0/152681445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128858fa40 con 0x561288592ef0 2024-05-27T08:05:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.969+0000 7f32a6600700 1 --2- 172.21.15.63:0/152681445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859e550 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f329800b080 tx=0x7f3298007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7435c3421a76f686 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.970+0000 7f3292000700 1 -- 172.21.15.63:0/152681445 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329802b040 con 0x561288592ef0 2024-05-27T08:05:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.970+0000 7f3292000700 1 -- 172.21.15.63:0/152681445 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3298005c60 con 0x561288592ef0 2024-05-27T08:05:14.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.970+0000 7f3292000700 1 -- 172.21.15.63:0/152681445 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298005570 con 0x561288592ef0 2024-05-27T08:05:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.971+0000 7f32b70cdb80 1 -- 172.21.15.63:0/152681445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 msgr2=0x56128859e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.971+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/152681445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859e550 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f329800b080 tx=0x7f3298007da0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.971+0000 7f32b70cdb80 1 -- 172.21.15.63:0/152681445 shutdown_connections 2024-05-27T08:05:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.971+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/152681445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561288592ef0 0x56128859e550 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.971+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/152681445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x561288594980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.971+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/152681445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612884ffa70 0x5612885a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.971+0000 7f32b70cdb80 1 -- 172.21.15.63:0/152681445 >> 172.21.15.63:0/152681445 conn(0x5612884ad410 msgr2=0x56128859de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.971+0000 7f32b70cdb80 1 -- 172.21.15.63:0/152681445 shutdown_connections 2024-05-27T08:05:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.971+0000 7f32b70cdb80 1 -- 172.21.15.63:0/152681445 wait complete. 2024-05-27T08:05:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.972+0000 7f32b70cdb80 1 -- start start 2024-05-27T08:05:14.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.972+0000 7f32b70cdb80 1 Processor -- start 2024-05-27T08:05:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x56128859fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612884ffa70 0x5612885a0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32b70cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 0x5612885a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01ac940 con 0x5612884ffa70 2024-05-27T08:05:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01addb0 con 0x561288413ec0 2024-05-27T08:05:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32b70cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01ad010 con 0x561288592ef0 2024-05-27T08:05:14.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32b70cdb80 10 client.0 _ll_register_callbacks cb 0x56128845acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32a5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612884ffa70 0x5612885a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32a5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612884ffa70 0x5612885a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57364/0 (socket says 172.21.15.63:57364) 2024-05-27T08:05:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32a5200700 1 -- 172.21.15.63:0/1474941005 learned_addr learned my addr 172.21.15.63:0/1474941005 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32a6600700 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 0x5612885a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.973+0000 7f32a5c00700 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x56128859fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.974+0000 7f32a5200700 1 -- 172.21.15.63:0/1474941005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 msgr2=0x5612885a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.974+0000 7f32a5200700 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 0x5612885a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.974+0000 7f32a5200700 1 -- 172.21.15.63:0/1474941005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 msgr2=0x56128859fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.974+0000 7f32a5200700 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x56128859fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.974+0000 7f32a5200700 1 -- 172.21.15.63:0/1474941005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128858a2c0 con 0x5612884ffa70 2024-05-27T08:05:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.974+0000 7f32a6600700 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 0x5612885a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.974+0000 7f32a5200700 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612884ffa70 0x5612885a0390 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f328c00bfd0 tx=0x7f328c0a9ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289feac8135ee2c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.974+0000 7f3287400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:14.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.975+0000 7f3287400700 1 -- 172.21.15.63:0/1474941005 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c020040 con 0x5612884ffa70 2024-05-27T08:05:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.975+0000 7f3287400700 1 -- 172.21.15.63:0/1474941005 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f328c0a9770 con 0x5612884ffa70 2024-05-27T08:05:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.975+0000 7f32b70cdb80 1 -- 172.21.15.63:0/1474941005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3298005c60 con 0x5612884ffa70 2024-05-27T08:05:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.975+0000 7f3287400700 1 -- 172.21.15.63:0/1474941005 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f328c0b2160 con 0x5612884ffa70 2024-05-27T08:05:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.975+0000 7f32b70cdb80 10 client.21519 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:14.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.975+0000 7f32b70cdb80 1 -- 172.21.15.63:0/1474941005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561288618670 con 0x5612884ffa70 2024-05-27T08:05:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.975+0000 7f3287400700 1 -- 172.21.15.63:0/1474941005 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3298005c60 con 0x5612884ffa70 2024-05-27T08:05:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.976+0000 7f3287400700 1 -- 172.21.15.63:0/1474941005 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f328c0ad1d0 con 0x5612884ffa70 2024-05-27T08:05:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.976+0000 7f32b70cdb80 10 client.21519 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.976+0000 7f32b70cdb80 10 client.21519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.976+0000 7f32b70cdb80 20 client.21519 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.976+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612886188a0 0x56128861acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.976+0000 7f32b70cdb80 4 client.21519 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:05:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.976+0000 7f32b70cdb80 1 -- 172.21.15.63:0/1474941005 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5612884bc5f0 con 0x5612886188a0 2024-05-27T08:05:14.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.977+0000 7f32a5c00700 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612886188a0 0x56128861acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.978+0000 7f32a5c00700 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612886188a0 0x56128861acc0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13d66bc67558f1b server_cookie=babbb071858de20a in_seq=0 out_seq=0 2024-05-27T08:05:14.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.978+0000 7f3287400700 10 client.21519 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:14.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.980+0000 7f3287400700 1 -- 172.21.15.63:0/1474941005 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5612884bc5f0 con 0x5612886188a0 2024-05-27T08:05:14.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.980+0000 7f3287400700 10 client.21519 handle_command_reply: tid=0 2024-05-27T08:05:14.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.980+0000 7f32b70cdb80 1 client.21519 shutdown 2024-05-27T08:05:14.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 1 -- 172.21.15.63:0/1474941005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612884ffa70 msgr2=0x5612885a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612884ffa70 0x5612885a0390 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f328c00bfd0 tx=0x7f328c0a9ab0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 1 -- 172.21.15.63:0/1474941005 shutdown_connections 2024-05-27T08:05:14.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5612886188a0 0x56128861acc0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561288592ef0 0x5612885a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561288413ec0 0x56128859fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 1 --2- 172.21.15.63:0/1474941005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612884ffa70 0x5612885a0390 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 1 -- 172.21.15.63:0/1474941005 >> 172.21.15.63:0/1474941005 conn(0x5612884ad410 msgr2=0x5612885946e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f3287400700 0 client.21519 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:14.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 1 -- 172.21.15.63:0/1474941005 shutdown_connections 2024-05-27T08:05:14.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 1 -- 172.21.15.63:0/1474941005 wait complete. 2024-05-27T08:05:14.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.981+0000 7f32b70cdb80 20 client.21519 trim_cache size 0 max 16384 2024-05-27T08:05:14.984 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:05:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:05:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "8e8973c1-dc01-4220-9646-3eeebc118c40", 2024-05-27T08:05:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:05:14.985 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:05:14.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.982+0000 7f32a8600700 1 -- 172.21.15.63:0/4257389249 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32880b29f0 msgr2=0x7f32880b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.982+0000 7f32a8600700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32880b29f0 0x7f32880b4f00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3294005950 tx=0x7f329400b410 comp rx=0 tx=0).stop 2024-05-27T08:05:14.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.982+0000 7f32a8600700 1 -- 172.21.15.63:0/4257389249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0107de0 msgr2=0x7f32a019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.982+0000 7f32a8600700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0107de0 0x7f32a019b300 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f328c00d980 tx=0x7f328c00dc90 comp rx=0 tx=0).stop 2024-05-27T08:05:14.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.983+0000 7f32a8600700 1 -- 172.21.15.63:0/4257389249 shutdown_connections 2024-05-27T08:05:14.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.983+0000 7f32a8600700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f32880b29f0 0x7f32880b4f00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.983+0000 7f32a8600700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32a0108730 0x7f32a019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.983+0000 7f32a8600700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32a01034b0 0x7f32a019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.983+0000 7f32a8600700 1 --2- 172.21.15.63:0/4257389249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32a0107de0 0x7f32a019b300 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.983+0000 7f32a8600700 1 -- 172.21.15.63:0/4257389249 >> 172.21.15.63:0/4257389249 conn(0x7f32a00fedb0 msgr2=0x7f32a006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.983+0000 7f32a8600700 1 -- 172.21.15.63:0/4257389249 shutdown_connections 2024-05-27T08:05:14.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:14.983+0000 7f32a8600700 1 -- 172.21.15.63:0/4257389249 wait complete. 2024-05-27T08:05:15.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:15 smithi086 ceph-mon[34687]: pgmap v1004: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-05-27T08:05:15.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:15 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:05:15.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:15 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:15.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:15 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:05:15.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:15 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:15.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:15 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:05:15.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:15 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:15.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:15 smithi149 ceph-mon[35408]: pgmap v1004: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-05-27T08:05:15.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:15 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:05:15.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:15 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:15.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:15 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:05:15.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:15 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:15.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:15 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:05:15.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:15 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:15.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:15 smithi063 ceph-mon[31613]: pgmap v1004: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-05-27T08:05:15.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:15 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:05:15.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:15 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:15.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:15 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:05:15.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:15 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:15.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:15 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:05:15.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:15 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:15.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:05:16.417 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:16.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:16 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:05:16.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:16 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:16 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:05:16.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:16 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:05:16.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:16 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:16 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:05:16.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:16 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:16 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:05:16.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:16 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:16 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:16 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:05:16.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:16 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:16 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:05:16.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:16 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:05:16.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:16 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:16 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:05:16.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:16 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:16 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:05:16.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:16 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:16 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:16 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:05:16.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:16 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:16 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:05:16.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:16 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:05:16.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:16 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:16 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:05:16.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:16 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:16 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:05:16.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:16 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:16.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:16 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:17 smithi063 ceph-mon[31613]: pgmap v1005: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.5 MiB/s wr, 46 op/s 2024-05-27T08:05:17.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:17 smithi086 ceph-mon[34687]: pgmap v1005: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.5 MiB/s wr, 46 op/s 2024-05-27T08:05:17.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:17 smithi149 ceph-mon[35408]: pgmap v1005: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.5 MiB/s wr, 46 op/s 2024-05-27T08:05:18.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.867+0000 7f4320a00700 1 -- 172.21.15.63:0/1514006499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c115170 msgr2=0x7f431c117590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.867+0000 7f4320a00700 1 --2- 172.21.15.63:0/1514006499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c115170 0x7f431c117590 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f431400b0a0 tx=0x7f431400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.868+0000 7f4320a00700 1 -- 172.21.15.63:0/1514006499 shutdown_connections 2024-05-27T08:05:18.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.868+0000 7f4320a00700 1 --2- 172.21.15.63:0/1514006499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c115170 0x7f431c117590 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.868+0000 7f4320a00700 1 --2- 172.21.15.63:0/1514006499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c075b50 0x7f431c075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.868+0000 7f4320a00700 1 --2- 172.21.15.63:0/1514006499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c0764a0 0x7f431c114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.868+0000 7f4320a00700 1 -- 172.21.15.63:0/1514006499 >> 172.21.15.63:0/1514006499 conn(0x7f431c079d00 msgr2=0x7f431c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:18.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.868+0000 7f4320a00700 1 -- 172.21.15.63:0/1514006499 shutdown_connections 2024-05-27T08:05:18.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.868+0000 7f4320a00700 1 -- 172.21.15.63:0/1514006499 wait complete. 2024-05-27T08:05:18.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.869+0000 7f4320a00700 1 Processor -- start 2024-05-27T08:05:18.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.870+0000 7f4320a00700 1 -- start start 2024-05-27T08:05:18.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.870+0000 7f4320a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c075b50 0x7f431c1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.870+0000 7f4320a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c0764a0 0x7f431c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.870+0000 7f4320a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c115170 0x7f431c1a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.870+0000 7f4320a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f431c11a780 con 0x7f431c115170 2024-05-27T08:05:18.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.870+0000 7f4320a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f431c11a600 con 0x7f431c075b50 2024-05-27T08:05:18.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.870+0000 7f4320a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f431c11a900 con 0x7f431c0764a0 2024-05-27T08:05:18.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.871+0000 7f431b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c115170 0x7f431c1a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.871+0000 7f431b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c115170 0x7f431c1a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57384/0 (socket says 172.21.15.63:57384) 2024-05-27T08:05:18.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.871+0000 7f431b200700 1 -- 172.21.15.63:0/4136517439 learned_addr learned my addr 172.21.15.63:0/4136517439 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:18.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.871+0000 7f431a800700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c075b50 0x7f431c1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.871+0000 7f4319e00700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c0764a0 0x7f431c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.871+0000 7f431b200700 1 -- 172.21.15.63:0/4136517439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c0764a0 msgr2=0x7f431c1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.871+0000 7f431b200700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c0764a0 0x7f431c1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.871+0000 7f431b200700 1 -- 172.21.15.63:0/4136517439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c075b50 msgr2=0x7f431c1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.871+0000 7f431b200700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c075b50 0x7f431c1a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.871+0000 7f431b200700 1 -- 172.21.15.63:0/4136517439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c1a6020 con 0x7f431c115170 2024-05-27T08:05:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.872+0000 7f431b200700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c115170 0x7f431c1a5980 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f431400b070 tx=0x7f4314004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.872+0000 7f4313400700 1 -- 172.21.15.63:0/4136517439 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314021070 con 0x7f431c115170 2024-05-27T08:05:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.872+0000 7f4320a00700 1 -- 172.21.15.63:0/4136517439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c1a62b0 con 0x7f431c115170 2024-05-27T08:05:18.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.872+0000 7f4313400700 1 -- 172.21.15.63:0/4136517439 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4314007b80 con 0x7f431c115170 2024-05-27T08:05:18.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.872+0000 7f4313400700 1 -- 172.21.15.63:0/4136517439 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43140046f0 con 0x7f431c115170 2024-05-27T08:05:18.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.872+0000 7f4320a00700 1 -- 172.21.15.63:0/4136517439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1a6790 con 0x7f431c115170 2024-05-27T08:05:18.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.878+0000 7f4313400700 1 -- 172.21.15.63:0/4136517439 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4314003a00 con 0x7f431c115170 2024-05-27T08:05:18.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.879+0000 7f432ffe8b80 1 Processor -- start 2024-05-27T08:05:18.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.879+0000 7f432ffe8b80 1 -- start start 2024-05-27T08:05:18.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452723ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527247ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452723d820 0x56452724a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56452716be40 con 0x5645270bee40 2024-05-27T08:05:18.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56452715b9e0 con 0x56452723d820 2024-05-27T08:05:18.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56452715fa20 con 0x56452723d410 2024-05-27T08:05:18.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f4313400700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b29f0 0x7f43000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f4319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452723ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f4313400700 1 -- 172.21.15.63:0/4136517439 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f431400f020 con 0x7f431c115170 2024-05-27T08:05:18.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f4319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452723ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57386/0 (socket says 172.21.15.63:57386) 2024-05-27T08:05:18.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f4319e00700 1 -- 172.21.15.63:0/1542958728 learned_addr learned my addr 172.21.15.63:0/1542958728 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:18.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.880+0000 7f431a800700 1 --2- 172.21.15.63:0/1542958728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527247ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.881+0000 7f431b200700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b29f0 0x7f43000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.881+0000 7f4319e00700 1 --2- 172.21.15.63:0/1542958728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452723d820 0x56452724a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.881+0000 7f4319e00700 1 -- 172.21.15.63:0/1542958728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 msgr2=0x564527247ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.881+0000 7f4319e00700 1 --2- 172.21.15.63:0/1542958728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527247ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.881+0000 7f4319e00700 1 -- 172.21.15.63:0/1542958728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452723d820 msgr2=0x56452724a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.881+0000 7f4319e00700 1 --2- 172.21.15.63:0/1542958728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452723d820 0x56452724a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.881+0000 7f4319e00700 1 -- 172.21.15.63:0/1542958728 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564527124420 con 0x5645270bee40 2024-05-27T08:05:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.881+0000 7f431b200700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b29f0 0x7f43000b4f00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f431400d010 tx=0x7f4314004f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.882+0000 7f4319e00700 1 --2- 172.21.15.63:0/1542958728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452723ced0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f43080149d0 tx=0x7f430800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2c211603b23f1f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.882+0000 7f4307e00700 1 -- 172.21.15.63:0/1542958728 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43080142d0 con 0x5645270bee40 2024-05-27T08:05:18.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.882+0000 7f4307e00700 1 -- 172.21.15.63:0/1542958728 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43080044c0 con 0x5645270bee40 2024-05-27T08:05:18.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.882+0000 7f4307e00700 1 -- 172.21.15.63:0/1542958728 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308012450 con 0x5645270bee40 2024-05-27T08:05:18.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.883+0000 7f432ffe8b80 1 -- 172.21.15.63:0/1542958728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 msgr2=0x56452723ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.883+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/1542958728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452723ced0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f43080149d0 tx=0x7f430800c4d0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.883+0000 7f432ffe8b80 1 -- 172.21.15.63:0/1542958728 shutdown_connections 2024-05-27T08:05:18.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.883+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/1542958728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452723d820 0x56452724a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.883+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/1542958728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452723ced0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.883+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/1542958728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527247ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.883+0000 7f432ffe8b80 1 -- 172.21.15.63:0/1542958728 >> 172.21.15.63:0/1542958728 conn(0x56452711efc0 msgr2=0x564527238d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:18.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.883+0000 7f432ffe8b80 1 -- 172.21.15.63:0/1542958728 shutdown_connections 2024-05-27T08:05:18.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.884+0000 7f432ffe8b80 1 -- 172.21.15.63:0/1542958728 wait complete. 2024-05-27T08:05:18.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.884+0000 7f432ffe8b80 1 -- start start 2024-05-27T08:05:18.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.884+0000 7f432ffe8b80 1 Processor -- start 2024-05-27T08:05:18.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.885+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645270bee40 0x5645272475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.885+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527247b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.885+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x564527248060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.885+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564527061fa0 con 0x56452723d820 2024-05-27T08:05:18.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.885+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5645270fc760 con 0x5645270bee40 2024-05-27T08:05:18.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.885+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56452724a930 con 0x56452723d410 2024-05-27T08:05:18.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.885+0000 7f432ffe8b80 10 client.0 _ll_register_callbacks cb 0x564527105c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:18.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.885+0000 7f4319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x564527248060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.886+0000 7f4319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x564527248060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57400/0 (socket says 172.21.15.63:57400) 2024-05-27T08:05:18.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.886+0000 7f4319e00700 1 -- 172.21.15.63:0/484481172 learned_addr learned my addr 172.21.15.63:0/484481172 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:18.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.886+0000 7f431a800700 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527247b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.886+0000 7f4319e00700 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645270bee40 0x5645272475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.886+0000 7f4319e00700 1 -- 172.21.15.63:0/484481172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 msgr2=0x564527247b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.886+0000 7f4319e00700 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527247b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.886+0000 7f4319e00700 1 -- 172.21.15.63:0/484481172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645270bee40 msgr2=0x5645272475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.886+0000 7f4319e00700 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645270bee40 0x5645272475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.886+0000 7f4319e00700 1 -- 172.21.15.63:0/484481172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645270ab8d0 con 0x56452723d820 2024-05-27T08:05:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.887+0000 7f4319e00700 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x564527248060 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f4308005950 tx=0x7f43080058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11aca4df74465ae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.887+0000 7f4304c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.887+0000 7f4304c00700 1 -- 172.21.15.63:0/484481172 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308028020 con 0x56452723d820 2024-05-27T08:05:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.887+0000 7f4304c00700 1 -- 172.21.15.63:0/484481172 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f4308003e70 con 0x56452723d820 2024-05-27T08:05:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.887+0000 7f432ffe8b80 1 -- 172.21.15.63:0/484481172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43080044c0 con 0x56452723d820 2024-05-27T08:05:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.887+0000 7f4304c00700 1 -- 172.21.15.63:0/484481172 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4308033050 con 0x56452723d820 2024-05-27T08:05:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.887+0000 7f432ffe8b80 10 client.21531 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:18.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.887+0000 7f432ffe8b80 1 -- 172.21.15.63:0/484481172 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56452712c060 con 0x56452723d820 2024-05-27T08:05:18.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.888+0000 7f4304c00700 1 -- 172.21.15.63:0/484481172 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43080044c0 con 0x56452723d820 2024-05-27T08:05:18.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.888+0000 7f4304c00700 1 -- 172.21.15.63:0/484481172 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4308021440 con 0x56452723d820 2024-05-27T08:05:18.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.888+0000 7f432ffe8b80 10 client.21531 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:18.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.888+0000 7f432ffe8b80 10 client.21531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:18.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.888+0000 7f432ffe8b80 20 client.21531 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:18.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.888+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564527257840 0x564527259c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.888+0000 7f432ffe8b80 4 client.21531 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:18.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.888+0000 7f432ffe8b80 1 -- 172.21.15.63:0/484481172 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56452716dd90 con 0x564527257840 2024-05-27T08:05:18.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.889+0000 7f431a800700 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564527257840 0x564527259c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.890+0000 7f431a800700 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564527257840 0x564527259c60 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb5bd51e869df9df server_cookie=24bebe70ba974702 in_seq=0 out_seq=0 2024-05-27T08:05:18.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.890+0000 7f4304c00700 10 client.21531 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:18.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.893+0000 7f4304c00700 1 -- 172.21.15.63:0/484481172 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56452716dd90 con 0x564527257840 2024-05-27T08:05:18.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.893+0000 7f4304c00700 10 client.21531 handle_command_reply: tid=0 2024-05-27T08:05:18.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.893+0000 7f432ffe8b80 1 client.21531 shutdown 2024-05-27T08:05:18.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 1 -- 172.21.15.63:0/484481172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 msgr2=0x564527248060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x564527248060 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f4308005950 tx=0x7f43080058e0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 1 -- 172.21.15.63:0/484481172 shutdown_connections 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x564527257840 0x564527259c60 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x564527248060 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645270bee40 0x5645272475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/484481172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527247b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 1 -- 172.21.15.63:0/484481172 >> 172.21.15.63:0/484481172 conn(0x56452711efc0 msgr2=0x564527237ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f4304c00700 0 client.21531 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 1 -- 172.21.15.63:0/484481172 shutdown_connections 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 1 -- 172.21.15.63:0/484481172 wait complete. 2024-05-27T08:05:18.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.894+0000 7f432ffe8b80 20 client.21531 trim_cache size 0 max 16384 2024-05-27T08:05:18.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.921+0000 7f432ffe8b80 1 Processor -- start 2024-05-27T08:05:18.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.921+0000 7f432ffe8b80 1 -- start start 2024-05-27T08:05:18.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.922+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645270bee40 0x564527249a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.922+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527242d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.922+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x5645272535f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.922+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b9140 con 0x56452723d820 2024-05-27T08:05:18.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.922+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b69d0 con 0x5645270bee40 2024-05-27T08:05:18.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.922+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b4840 con 0x56452723d410 2024-05-27T08:05:18.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.922+0000 7f4319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x5645272535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.922+0000 7f4319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x5645272535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59318/0 (socket says 172.21.15.63:59318) 2024-05-27T08:05:18.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.922+0000 7f4319e00700 1 -- 172.21.15.63:0/3339931210 learned_addr learned my addr 172.21.15.63:0/3339931210 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:18.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.923+0000 7f431a800700 1 --2- 172.21.15.63:0/3339931210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527242d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.923+0000 7f4319e00700 1 --2- 172.21.15.63:0/3339931210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645270bee40 0x564527249a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.923+0000 7f4319e00700 1 -- 172.21.15.63:0/3339931210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 msgr2=0x564527242d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.923+0000 7f4319e00700 1 --2- 172.21.15.63:0/3339931210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527242d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.923+0000 7f4319e00700 1 -- 172.21.15.63:0/3339931210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645270bee40 msgr2=0x564527249a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.923+0000 7f4319e00700 1 --2- 172.21.15.63:0/3339931210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645270bee40 0x564527249a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.923+0000 7f4319e00700 1 -- 172.21.15.63:0/3339931210 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56452723a350 con 0x56452723d820 2024-05-27T08:05:18.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.923+0000 7f431a800700 1 --2- 172.21.15.63:0/3339931210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527242d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:18.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.923+0000 7f4319e00700 1 --2- 172.21.15.63:0/3339931210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x5645272535f0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x56452716be90 tx=0x7f4308016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3771541b04f5e4c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:18.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.924+0000 7f4307400700 1 -- 172.21.15.63:0/3339931210 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430803d070 con 0x56452723d820 2024-05-27T08:05:18.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.924+0000 7f4307400700 1 -- 172.21.15.63:0/3339931210 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4308021e50 con 0x56452723d820 2024-05-27T08:05:18.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.924+0000 7f4307400700 1 -- 172.21.15.63:0/3339931210 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308028070 con 0x56452723d820 2024-05-27T08:05:18.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.925+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3339931210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 msgr2=0x5645272535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.925+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/3339931210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x5645272535f0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x56452716be90 tx=0x7f4308016040 comp rx=0 tx=0).stop 2024-05-27T08:05:18.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.925+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3339931210 shutdown_connections 2024-05-27T08:05:18.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.925+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/3339931210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56452723d820 0x5645272535f0 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.925+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/3339931210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645270bee40 0x564527249a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.925+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/3339931210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x564527242d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.925+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3339931210 >> 172.21.15.63:0/3339931210 conn(0x56452711efc0 msgr2=0x5645272475a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:18.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.925+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3339931210 shutdown_connections 2024-05-27T08:05:18.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.925+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3339931210 wait complete. 2024-05-27T08:05:18.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.926+0000 7f432ffe8b80 1 -- start start 2024-05-27T08:05:18.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.926+0000 7f432ffe8b80 1 Processor -- start 2024-05-27T08:05:18.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.927+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452725a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.927+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x56452725aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.927+0000 7f432ffe8b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452723d820 0x56452725aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.927+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b4e90 con 0x5645270bee40 2024-05-27T08:05:18.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.927+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b6300 con 0x56452723d820 2024-05-27T08:05:18.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.927+0000 7f432ffe8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b5560 con 0x56452723d410 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.927+0000 7f432ffe8b80 10 client.0 _ll_register_callbacks cb 0x564527105c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.927+0000 7f4319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452725a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.927+0000 7f4319e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452725a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59320/0 (socket says 172.21.15.63:59320) 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.927+0000 7f4319e00700 1 -- 172.21.15.63:0/3810669397 learned_addr learned my addr 172.21.15.63:0/3810669397 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.928+0000 7f431a800700 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x56452725aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.928+0000 7f4319e00700 1 -- 172.21.15.63:0/3810669397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 msgr2=0x56452725aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.928+0000 7f4319e00700 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x56452725aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.928+0000 7f4319e00700 1 -- 172.21.15.63:0/3810669397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452723d820 msgr2=0x56452725aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.928+0000 7f4319e00700 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452723d820 0x56452725aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.928+0000 7f4319e00700 1 -- 172.21.15.63:0/3810669397 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645272c10b0 con 0x5645270bee40 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.928+0000 7f431a800700 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x56452725aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.928+0000 7f4319e00700 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452725a570 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f430802f040 tx=0x7f4308005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a919aedb05eefb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:18.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.928+0000 7f4304c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:18.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.929+0000 7f4304c00700 1 -- 172.21.15.63:0/3810669397 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308028070 con 0x5645270bee40 2024-05-27T08:05:18.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.929+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3810669397 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4308021e50 con 0x5645270bee40 2024-05-27T08:05:18.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.929+0000 7f4304c00700 1 -- 172.21.15.63:0/3810669397 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f4308003bd0 con 0x5645270bee40 2024-05-27T08:05:18.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.929+0000 7f4304c00700 1 -- 172.21.15.63:0/3810669397 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f430804b050 con 0x5645270bee40 2024-05-27T08:05:18.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.929+0000 7f432ffe8b80 10 client.21537 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:18.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.929+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3810669397 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5645272c2240 con 0x5645270bee40 2024-05-27T08:05:18.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.929+0000 7f4304c00700 1 -- 172.21.15.63:0/3810669397 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4308021e50 con 0x5645270bee40 2024-05-27T08:05:18.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.930+0000 7f4304c00700 1 -- 172.21.15.63:0/3810669397 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4308003e60 con 0x5645270bee40 2024-05-27T08:05:18.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.930+0000 7f432ffe8b80 10 client.21537 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:18.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.930+0000 7f432ffe8b80 10 client.21537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:18.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.930+0000 7f432ffe8b80 20 client.21537 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:18.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.930+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5645272c2870 0x5645272c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.930+0000 7f432ffe8b80 4 client.21537 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:05:18.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.930+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3810669397 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564527167570 con 0x5645272c2870 2024-05-27T08:05:18.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.931+0000 7f431a800700 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5645272c2870 0x5645272c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.932+0000 7f431a800700 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5645272c2870 0x5645272c4c90 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29e767758707b84 server_cookie=9c89f958da7d0ad7 in_seq=0 out_seq=0 2024-05-27T08:05:18.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.932+0000 7f4304c00700 10 client.21537 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:18.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.932+0000 7f4304c00700 1 -- 172.21.15.63:0/3810669397 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564527167570 con 0x5645272c2870 2024-05-27T08:05:18.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.932+0000 7f4304c00700 10 client.21537 handle_command_reply: tid=0 2024-05-27T08:05:18.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.932+0000 7f432ffe8b80 1 client.21537 shutdown 2024-05-27T08:05:18.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.932+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3810669397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 msgr2=0x56452725a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.932+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452725a570 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f430802f040 tx=0x7f4308005d70 comp rx=0 tx=0).stop 2024-05-27T08:05:18.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.933+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3810669397 shutdown_connections 2024-05-27T08:05:18.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.933+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5645272c2870 0x5645272c4c90 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.933+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56452723d820 0x56452725aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.933+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5645270bee40 0x56452725a570 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.933+0000 7f432ffe8b80 1 --2- 172.21.15.63:0/3810669397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56452723d410 0x56452725aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.933+0000 7f4304c00700 0 client.21537 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:18.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.933+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3810669397 >> 172.21.15.63:0/3810669397 conn(0x56452711efc0 msgr2=0x564527245b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:18.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.933+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3810669397 shutdown_connections 2024-05-27T08:05:18.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.933+0000 7f432ffe8b80 1 -- 172.21.15.63:0/3810669397 wait complete. 2024-05-27T08:05:18.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.933+0000 7f432ffe8b80 20 client.21537 trim_cache size 0 max 16384 2024-05-27T08:05:18.937 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:05:18.937 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:05:18.937 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:05:18.937 INFO:teuthology.orchestra.run.smithi063.stdout: "8e8973c1-dc01-4220-9646-3eeebc118c40": { 2024-05-27T08:05:18.938 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:05:18.938 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "8e8973c1-dc01-4220-9646-3eeebc118c40", 2024-05-27T08:05:18.938 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:05:18.938 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:05:18.938 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:05:18.938 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.934+0000 7f4320a00700 1 -- 172.21.15.63:0/4136517439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b29f0 msgr2=0x7f43000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.934+0000 7f4320a00700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b29f0 0x7f43000b4f00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f431400d010 tx=0x7f4314004f90 comp rx=0 tx=0).stop 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.934+0000 7f4320a00700 1 -- 172.21.15.63:0/4136517439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c115170 msgr2=0x7f431c1a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.934+0000 7f4320a00700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c115170 0x7f431c1a5980 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f431400b070 tx=0x7f4314004c80 comp rx=0 tx=0).stop 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.934+0000 7f4320a00700 1 -- 172.21.15.63:0/4136517439 shutdown_connections 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.934+0000 7f4320a00700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b29f0 0x7f43000b4f00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.934+0000 7f4320a00700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c115170 0x7f431c1a5980 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.934+0000 7f4320a00700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c075b50 0x7f431c1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.934+0000 7f4320a00700 1 --2- 172.21.15.63:0/4136517439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c0764a0 0x7f431c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.934+0000 7f4320a00700 1 -- 172.21.15.63:0/4136517439 >> 172.21.15.63:0/4136517439 conn(0x7f431c079d00 msgr2=0x7f431c100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:18.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.935+0000 7f4320a00700 1 -- 172.21.15.63:0/4136517439 shutdown_connections 2024-05-27T08:05:18.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:18.935+0000 7f4320a00700 1 -- 172.21.15.63:0/4136517439 wait complete. 2024-05-27T08:05:19.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:19 smithi086 ceph-mon[34687]: pgmap v1006: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:05:19.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:19 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:05:19.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:19 smithi149 ceph-mon[35408]: pgmap v1006: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:05:19.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:19 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:05:19.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:19 smithi063 ceph-mon[31613]: pgmap v1006: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:05:19.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:19 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:05:20.062 INFO:tasks.cephfs.filesystem:scrub status for tag:8e8973c1-dc01-4220-9646-3eeebc118c40 - {'path': '/', 'tag': '8e8973c1-dc01-4220-9646-3eeebc118c40', 'options': 'recursive,force'} 2024-05-27T08:05:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.797+0000 7f7317600640 1 -- 172.21.15.63:0/1247063440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731010a150 msgr2=0x7f7310103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.797+0000 7f7317600640 1 --2- 172.21.15.63:0/1247063440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731010a150 0x7f7310103d90 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009900 tx=0x7f72fc02a410 comp rx=0 tx=0).stop 2024-05-27T08:05:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.797+0000 7f7317600640 1 -- 172.21.15.63:0/1247063440 shutdown_connections 2024-05-27T08:05:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.797+0000 7f7317600640 1 --2- 172.21.15.63:0/1247063440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7310104400 0x7f73101107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.797+0000 7f7317600640 1 --2- 172.21.15.63:0/1247063440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f731010a150 0x7f7310103d90 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.797+0000 7f7317600640 1 --2- 172.21.15.63:0/1247063440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7310109cb0 0x7f7310103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.797+0000 7f7317600640 1 -- 172.21.15.63:0/1247063440 >> 172.21.15.63:0/1247063440 conn(0x7f73100ff480 msgr2=0x7f73101018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:20.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.797+0000 7f7317600640 1 -- 172.21.15.63:0/1247063440 shutdown_connections 2024-05-27T08:05:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.798+0000 7f7317600640 1 -- 172.21.15.63:0/1247063440 wait complete. 2024-05-27T08:05:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.799+0000 7f7317600640 1 Processor -- start 2024-05-27T08:05:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.799+0000 7f7317600640 1 -- start start 2024-05-27T08:05:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.800+0000 7f7317600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7310104400 0x7f73101a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.800+0000 7f7317600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7310109cb0 0x7f73101a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.800+0000 7f7317600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731010a150 0x7f731019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.800+0000 7f7317600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7310113d80 con 0x7f7310109cb0 2024-05-27T08:05:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.800+0000 7f7317600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7310113c00 con 0x7f731010a150 2024-05-27T08:05:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.800+0000 7f7317600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7310113f00 con 0x7f7310104400 2024-05-27T08:05:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.800+0000 7f730fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7310109cb0 0x7f73101a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.801+0000 7f730fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7310109cb0 0x7f73101a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59334/0 (socket says 172.21.15.63:59334) 2024-05-27T08:05:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.801+0000 7f730fe00640 1 -- 172.21.15.63:0/2291768671 learned_addr learned my addr 172.21.15.63:0/2291768671 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.801+0000 7f7314c00640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7310104400 0x7f73101a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.801+0000 7f7315600640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731010a150 0x7f731019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.801+0000 7f730fe00640 1 -- 172.21.15.63:0/2291768671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7310104400 msgr2=0x7f73101a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.801+0000 7f730fe00640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7310104400 0x7f73101a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.801+0000 7f730fe00640 1 -- 172.21.15.63:0/2291768671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731010a150 msgr2=0x7f731019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.801+0000 7f730fe00640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731010a150 0x7f731019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.801+0000 7f730fe00640 1 -- 172.21.15.63:0/2291768671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731019add0 con 0x7f7310109cb0 2024-05-27T08:05:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.802+0000 7f730fe00640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7310109cb0 0x7f73101a0a60 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f72fc02a930 tx=0x7f72fc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.802+0000 7f730d600640 1 -- 172.21.15.63:0/2291768671 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc03f070 con 0x7f7310109cb0 2024-05-27T08:05:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.802+0000 7f7317600640 1 -- 172.21.15.63:0/2291768671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731019b000 con 0x7f7310109cb0 2024-05-27T08:05:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.802+0000 7f730d600640 1 -- 172.21.15.63:0/2291768671 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72fc03baf0 con 0x7f7310109cb0 2024-05-27T08:05:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.802+0000 7f7317600640 1 -- 172.21.15.63:0/2291768671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73101a72c0 con 0x7f7310109cb0 2024-05-27T08:05:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.802+0000 7f730d600640 1 -- 172.21.15.63:0/2291768671 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc035030 con 0x7f7310109cb0 2024-05-27T08:05:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.807+0000 7f731e5ef740 1 Processor -- start 2024-05-27T08:05:20.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.807+0000 7f731e5ef740 1 -- start start 2024-05-27T08:05:20.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.807+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a7ef860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.807+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a4242f0 0x559d9a7efda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a42c4d0 0x559d9a7f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f731e5ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559d9a760b90 con 0x559d9a44d1b0 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f731e5ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d9a750730 con 0x559d9a42c4d0 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f731e5ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559d9a754770 con 0x559d9a4242f0 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f7315600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a7ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f7315600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a7ef860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59338/0 (socket says 172.21.15.63:59338) 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f7315600640 1 -- 172.21.15.63:0/1886612813 learned_addr learned my addr 172.21.15.63:0/1886612813 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f7314c00640 1 --2- 172.21.15.63:0/1886612813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a4242f0 0x559d9a7efda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f7315600640 1 -- 172.21.15.63:0/1886612813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a4242f0 msgr2=0x559d9a7efda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f7315600640 1 --2- 172.21.15.63:0/1886612813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a4242f0 0x559d9a7efda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f7315600640 1 -- 172.21.15.63:0/1886612813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a42c4d0 msgr2=0x559d9a7f02e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f7315600640 1 --2- 172.21.15.63:0/1886612813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a42c4d0 0x559d9a7f02e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.808+0000 7f7315600640 1 -- 172.21.15.63:0/1886612813 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d9a6e98b0 con 0x559d9a44d1b0 2024-05-27T08:05:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.809+0000 7f7315600640 1 --2- 172.21.15.63:0/1886612813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a7ef860 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f730800ae90 tx=0x7f730800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a8aabb7c1ff7a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.809+0000 7f7301c00640 1 -- 172.21.15.63:0/1886612813 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308003b70 con 0x559d9a44d1b0 2024-05-27T08:05:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.810+0000 7f7301c00640 1 -- 172.21.15.63:0/1886612813 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7308003d10 con 0x559d9a44d1b0 2024-05-27T08:05:20.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.810+0000 7f7301c00640 1 -- 172.21.15.63:0/1886612813 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308010b00 con 0x559d9a44d1b0 2024-05-27T08:05:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.810+0000 7f730d600640 1 -- 172.21.15.63:0/2291768671 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f72fc042070 con 0x7f7310109cb0 2024-05-27T08:05:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.811+0000 7f731e5ef740 1 -- 172.21.15.63:0/1886612813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 msgr2=0x559d9a7ef860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.811+0000 7f731e5ef740 1 --2- 172.21.15.63:0/1886612813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a7ef860 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f730800ae90 tx=0x7f730800f790 comp rx=0 tx=0).stop 2024-05-27T08:05:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.811+0000 7f731e5ef740 1 -- 172.21.15.63:0/1886612813 shutdown_connections 2024-05-27T08:05:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.811+0000 7f731e5ef740 1 --2- 172.21.15.63:0/1886612813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a42c4d0 0x559d9a7f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.811+0000 7f731e5ef740 1 --2- 172.21.15.63:0/1886612813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a4242f0 0x559d9a7efda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.811+0000 7f731e5ef740 1 --2- 172.21.15.63:0/1886612813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a7ef860 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.811+0000 7f731e5ef740 1 -- 172.21.15.63:0/1886612813 >> 172.21.15.63:0/1886612813 conn(0x559d9a441300 msgr2=0x559d9a7ec010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.811+0000 7f731e5ef740 1 -- 172.21.15.63:0/1886612813 shutdown_connections 2024-05-27T08:05:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.811+0000 7f731e5ef740 1 -- 172.21.15.63:0/1886612813 wait complete. 2024-05-27T08:05:20.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.812+0000 7f731e5ef740 1 -- start start 2024-05-27T08:05:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.813+0000 7f731e5ef740 1 Processor -- start 2024-05-27T08:05:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.813+0000 7f730d600640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b2ae0 0x7f72f80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.813+0000 7f730d600640 1 -- 172.21.15.63:0/2291768671 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f72fc0318c0 con 0x7f7310109cb0 2024-05-27T08:05:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.814+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a4242f0 0x559d9a7f9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.814+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a42c4d0 0x559d9a7f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.814+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a44d1b0 0x559d9a7fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.814+0000 7f731e5ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559d9a436790 con 0x559d9a4242f0 2024-05-27T08:05:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.814+0000 7f731e5ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d9a6f8bc0 con 0x559d9a42c4d0 2024-05-27T08:05:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.814+0000 7f731e5ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559d9a800da0 con 0x559d9a44d1b0 2024-05-27T08:05:20.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.814+0000 7f731e5ef740 10 client.0 _ll_register_callbacks cb 0x559d9a6d6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.814+0000 7f7314c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a4242f0 0x559d9a7f9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.814+0000 7f7314c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a4242f0 0x559d9a7f9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59352/0 (socket says 172.21.15.63:59352) 2024-05-27T08:05:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.814+0000 7f7314c00640 1 -- 172.21.15.63:0/1912357906 learned_addr learned my addr 172.21.15.63:0/1912357906 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.815+0000 7f730fe00640 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a42c4d0 0x559d9a7f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.815+0000 7f7315600640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b2ae0 0x7f72f80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.815+0000 7f730fe00640 1 -- 172.21.15.63:0/1912357906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a44d1b0 msgr2=0x559d9a7fa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.815+0000 7f730fe00640 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a44d1b0 0x559d9a7fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.815+0000 7f730fe00640 1 -- 172.21.15.63:0/1912357906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a4242f0 msgr2=0x559d9a7f9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.815+0000 7f730fe00640 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a4242f0 0x559d9a7f9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.815+0000 7f730fe00640 1 -- 172.21.15.63:0/1912357906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d9a465210 con 0x559d9a42c4d0 2024-05-27T08:05:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.816+0000 7f7315600640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b2ae0 0x7f72f80b4ff0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f73080107e0 tx=0x7f73080106e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.816+0000 7f730fe00640 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a42c4d0 0x559d9a7f9fa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f72fc1001c0 tx=0x7f72fc0c9810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9633dd49753cf95c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.816+0000 7f72f6000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:05:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.817+0000 7f72f6000640 1 -- 172.21.15.63:0/1912357906 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc049070 con 0x559d9a42c4d0 2024-05-27T08:05:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.817+0000 7f72f6000640 1 -- 172.21.15.63:0/1912357906 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f72fc0c6090 con 0x559d9a42c4d0 2024-05-27T08:05:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.817+0000 7f731e5ef740 1 -- 172.21.15.63:0/1912357906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7308003d10 con 0x559d9a42c4d0 2024-05-27T08:05:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.817+0000 7f72f6000640 1 -- 172.21.15.63:0/1912357906 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f72fc0ce100 con 0x559d9a42c4d0 2024-05-27T08:05:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.817+0000 7f731e5ef740 10 client.40628 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.817+0000 7f731e5ef740 1 -- 172.21.15.63:0/1912357906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559d9a531620 con 0x559d9a42c4d0 2024-05-27T08:05:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.818+0000 7f72f6000640 1 -- 172.21.15.63:0/1912357906 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7308003d10 con 0x559d9a42c4d0 2024-05-27T08:05:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.818+0000 7f72f6000640 1 -- 172.21.15.63:0/1912357906 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f72fc0ca060 con 0x559d9a42c4d0 2024-05-27T08:05:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.818+0000 7f731e5ef740 10 client.40628 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.818+0000 7f731e5ef740 10 client.40628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.818+0000 7f731e5ef740 20 client.40628 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.818+0000 7f731e5ef740 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559d9a40bd80 0x559d9a814470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.818+0000 7f731e5ef740 4 client.40628 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.818+0000 7f731e5ef740 1 -- 172.21.15.63:0/1912357906 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d9a762ae0 con 0x559d9a40bd80 2024-05-27T08:05:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.819+0000 7f7314c00640 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559d9a40bd80 0x559d9a814470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.819+0000 7f7314c00640 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559d9a40bd80 0x559d9a814470 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd1ee8a8bc17c04 server_cookie=cbe6210db372d80a in_seq=0 out_seq=0 2024-05-27T08:05:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.819+0000 7f72f6000640 10 client.40628 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f72f6000640 1 -- 172.21.15.63:0/1912357906 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559d9a762ae0 con 0x559d9a40bd80 2024-05-27T08:05:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f72f6000640 10 client.40628 handle_command_reply: tid=0 2024-05-27T08:05:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f731e5ef740 1 client.40628 shutdown 2024-05-27T08:05:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f731e5ef740 1 -- 172.21.15.63:0/1912357906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a42c4d0 msgr2=0x559d9a7f9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f731e5ef740 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a42c4d0 0x559d9a7f9fa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f72fc1001c0 tx=0x7f72fc0c9810 comp rx=0 tx=0).stop 2024-05-27T08:05:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f731e5ef740 1 -- 172.21.15.63:0/1912357906 shutdown_connections 2024-05-27T08:05:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f731e5ef740 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559d9a40bd80 0x559d9a814470 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f731e5ef740 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a44d1b0 0x559d9a7fa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f731e5ef740 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a42c4d0 0x559d9a7f9fa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f731e5ef740 1 --2- 172.21.15.63:0/1912357906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a4242f0 0x559d9a7f9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f72f6000640 0 client.40628 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.821+0000 7f731e5ef740 1 -- 172.21.15.63:0/1912357906 >> 172.21.15.63:0/1912357906 conn(0x559d9a441300 msgr2=0x559d9a7ef690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.822+0000 7f731e5ef740 1 -- 172.21.15.63:0/1912357906 shutdown_connections 2024-05-27T08:05:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.822+0000 7f731e5ef740 1 -- 172.21.15.63:0/1912357906 wait complete. 2024-05-27T08:05:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.822+0000 7f731e5ef740 20 client.40628 trim_cache size 0 max 16384 2024-05-27T08:05:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.846+0000 7f731e5ef740 1 Processor -- start 2024-05-27T08:05:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.846+0000 7f731e5ef740 1 -- start start 2024-05-27T08:05:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 0x559d9a7f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a42c4d0 0x559d9a7fa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a44d1b0 0x559d9a819b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f731e5ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73101b19a0 con 0x559d9a42c4d0 2024-05-27T08:05:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f731e5ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73101af270 con 0x559d9a4242f0 2024-05-27T08:05:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f731e5ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f73101ad0e0 con 0x559d9a44d1b0 2024-05-27T08:05:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f730fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a42c4d0 0x559d9a7fa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f730fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a42c4d0 0x559d9a7fa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59364/0 (socket says 172.21.15.63:59364) 2024-05-27T08:05:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f730fe00640 1 -- 172.21.15.63:0/293935883 learned_addr learned my addr 172.21.15.63:0/293935883 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f7314c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 0x559d9a7f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.847+0000 7f7315600640 1 --2- 172.21.15.63:0/293935883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a44d1b0 0x559d9a819b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.848+0000 7f730fe00640 1 -- 172.21.15.63:0/293935883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a44d1b0 msgr2=0x559d9a819b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.848+0000 7f730fe00640 1 --2- 172.21.15.63:0/293935883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a44d1b0 0x559d9a819b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.848+0000 7f730fe00640 1 -- 172.21.15.63:0/293935883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 msgr2=0x559d9a7f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.848+0000 7f730fe00640 1 --2- 172.21.15.63:0/293935883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 0x559d9a7f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.848+0000 7f730fe00640 1 -- 172.21.15.63:0/293935883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d9a7fa840 con 0x559d9a42c4d0 2024-05-27T08:05:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.848+0000 7f730fe00640 1 --2- 172.21.15.63:0/293935883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a42c4d0 0x559d9a7fa300 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0ce2c0 tx=0x7f72fc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d415e493824ef72c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.849+0000 7f7301200640 1 -- 172.21.15.63:0/293935883 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc049070 con 0x559d9a42c4d0 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.849+0000 7f7301200640 1 -- 172.21.15.63:0/293935883 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72fc0c6090 con 0x559d9a42c4d0 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.849+0000 7f7301200640 1 -- 172.21.15.63:0/293935883 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc0c9100 con 0x559d9a42c4d0 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.850+0000 7f731e5ef740 1 -- 172.21.15.63:0/293935883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a42c4d0 msgr2=0x559d9a7fa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.850+0000 7f731e5ef740 1 --2- 172.21.15.63:0/293935883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a42c4d0 0x559d9a7fa300 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0ce2c0 tx=0x7f72fc02a410 comp rx=0 tx=0).stop 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.850+0000 7f731e5ef740 1 -- 172.21.15.63:0/293935883 shutdown_connections 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.850+0000 7f731e5ef740 1 --2- 172.21.15.63:0/293935883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a44d1b0 0x559d9a819b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.850+0000 7f731e5ef740 1 --2- 172.21.15.63:0/293935883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a42c4d0 0x559d9a7fa300 secure :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0ce2c0 tx=0x7f72fc02a410 comp rx=0 tx=0).stop 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.850+0000 7f731e5ef740 1 --2- 172.21.15.63:0/293935883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 0x559d9a7f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.850+0000 7f731e5ef740 1 -- 172.21.15.63:0/293935883 >> 172.21.15.63:0/293935883 conn(0x559d9a441300 msgr2=0x559d9a7ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.850+0000 7f731e5ef740 1 -- 172.21.15.63:0/293935883 shutdown_connections 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.850+0000 7f731e5ef740 1 -- 172.21.15.63:0/293935883 wait complete. 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.851+0000 7f731e5ef740 1 -- start start 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.851+0000 7f731e5ef740 1 Processor -- start 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 0x559d9a80c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a80c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f731e5ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a3e11c0 0x559d9a874a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f731e5ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73101ad730 con 0x559d9a44d1b0 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f731e5ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73101aeba0 con 0x559d9a4242f0 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f731e5ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f73101ade00 con 0x559d9a3e11c0 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f731e5ef740 10 client.0 _ll_register_callbacks cb 0x7f72fc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f730fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a80c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f730fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a80c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59368/0 (socket says 172.21.15.63:59368) 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f730fe00640 1 -- 172.21.15.63:0/2148390065 learned_addr learned my addr 172.21.15.63:0/2148390065 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f7314c00640 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 0x559d9a80c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.852+0000 7f7315600640 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a3e11c0 0x559d9a874a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f730fe00640 1 -- 172.21.15.63:0/2148390065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a3e11c0 msgr2=0x559d9a874a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f730fe00640 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a3e11c0 0x559d9a874a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f730fe00640 1 -- 172.21.15.63:0/2148390065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 msgr2=0x559d9a80c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f730fe00640 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 0x559d9a80c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f730fe00640 1 -- 172.21.15.63:0/2148390065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d9a874fd0 con 0x559d9a44d1b0 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f7314c00640 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 0x559d9a80c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f7315600640 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a3e11c0 0x559d9a874a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f730fe00640 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a80c680 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0c9c30 tx=0x7f72fc0c9f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc799aa9f56012a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f72f6000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f72f6000640 1 -- 172.21.15.63:0/2148390065 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc049070 con 0x559d9a44d1b0 2024-05-27T08:05:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f72f6000640 1 -- 172.21.15.63:0/2148390065 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f72fc0e43e0 con 0x559d9a44d1b0 2024-05-27T08:05:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.853+0000 7f731e5ef740 1 -- 172.21.15.63:0/2148390065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72fc0c6090 con 0x559d9a44d1b0 2024-05-27T08:05:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.854+0000 7f72f6000640 1 -- 172.21.15.63:0/2148390065 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f72fc0c5cc0 con 0x559d9a44d1b0 2024-05-27T08:05:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.854+0000 7f731e5ef740 10 client.21555 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.854+0000 7f731e5ef740 1 -- 172.21.15.63:0/2148390065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559d9a874fd0 con 0x559d9a44d1b0 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.854+0000 7f72f6000640 1 -- 172.21.15.63:0/2148390065 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72fc0c6090 con 0x559d9a44d1b0 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.854+0000 7f72f6000640 1 -- 172.21.15.63:0/2148390065 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f72fc0c5770 con 0x559d9a44d1b0 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.854+0000 7f731e5ef740 10 client.21555 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.854+0000 7f731e5ef740 10 client.21555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.854+0000 7f731e5ef740 20 client.21555 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.854+0000 7f731e5ef740 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559d9a87c5e0 0x559d9a87ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.855+0000 7f731e5ef740 4 client.21555 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.855+0000 7f731e5ef740 1 -- 172.21.15.63:0/2148390065 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d9a75c2c0 con 0x559d9a87c5e0 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.855+0000 7f7314c00640 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559d9a87c5e0 0x559d9a87ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.856+0000 7f7314c00640 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559d9a87c5e0 0x559d9a87ea00 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ff6289093f43f4 server_cookie=f7acf8db1b5bd07e in_seq=0 out_seq=0 2024-05-27T08:05:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.856+0000 7f72f6000640 10 client.21555 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.885+0000 7f72f6000640 1 -- 172.21.15.63:0/2148390065 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559d9a75c2c0 con 0x559d9a87c5e0 2024-05-27T08:05:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.885+0000 7f72f6000640 10 client.21555 handle_command_reply: tid=0 2024-05-27T08:05:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.885+0000 7f731e5ef740 1 client.21555 shutdown 2024-05-27T08:05:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 1 -- 172.21.15.63:0/2148390065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 msgr2=0x559d9a80c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a80c680 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0c9c30 tx=0x7f72fc0c9f80 comp rx=0 tx=0).stop 2024-05-27T08:05:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 1 -- 172.21.15.63:0/2148390065 shutdown_connections 2024-05-27T08:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559d9a87c5e0 0x559d9a87ea00 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d9a3e11c0 0x559d9a874a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d9a44d1b0 0x559d9a80c680 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 1 --2- 172.21.15.63:0/2148390065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d9a4242f0 0x559d9a80c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 1 -- 172.21.15.63:0/2148390065 >> 172.21.15.63:0/2148390065 conn(0x559d9a441300 msgr2=0x559d9a7f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f72f6000640 0 client.21555 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 1 -- 172.21.15.63:0/2148390065 shutdown_connections 2024-05-27T08:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 1 -- 172.21.15.63:0/2148390065 wait complete. 2024-05-27T08:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.886+0000 7f731e5ef740 20 client.21555 trim_cache size 0 max 16384 2024-05-27T08:05:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:05:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:05:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:05:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.887+0000 7f7317600640 1 -- 172.21.15.63:0/2291768671 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b2ae0 msgr2=0x7f72f80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.887+0000 7f7317600640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b2ae0 0x7f72f80b4ff0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f73080107e0 tx=0x7f73080106e0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.887+0000 7f7317600640 1 -- 172.21.15.63:0/2291768671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7310109cb0 msgr2=0x7f73101a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.887+0000 7f7317600640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7310109cb0 0x7f73101a0a60 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f72fc02a930 tx=0x7f72fc003b90 comp rx=0 tx=0).stop 2024-05-27T08:05:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.888+0000 7f7317600640 1 -- 172.21.15.63:0/2291768671 shutdown_connections 2024-05-27T08:05:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.888+0000 7f7317600640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f72f80b2ae0 0x7f72f80b4ff0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.888+0000 7f7317600640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f731010a150 0x7f731019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.888+0000 7f7317600640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7310109cb0 0x7f73101a0a60 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.888+0000 7f7317600640 1 --2- 172.21.15.63:0/2291768671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7310104400 0x7f73101a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.888+0000 7f7317600640 1 -- 172.21.15.63:0/2291768671 >> 172.21.15.63:0/2291768671 conn(0x7f73100ff480 msgr2=0x7f73100ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.888+0000 7f7317600640 1 -- 172.21.15.63:0/2291768671 shutdown_connections 2024-05-27T08:05:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:20.888+0000 7f7317600640 1 -- 172.21.15.63:0/2291768671 wait complete. 2024-05-27T08:05:21.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:21 smithi063 ceph-mon[31613]: pgmap v1007: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:05:21.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:21 smithi086 ceph-mon[34687]: pgmap v1007: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:05:21.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:21 smithi149 ceph-mon[35408]: pgmap v1007: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:05:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:05:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:05:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:22 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:05:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:22 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:22 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:22 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:22 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:05:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:05:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:22 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:05:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:22 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:22 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:22 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:22 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:05:22.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:05:22.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:22 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:05:22.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:22 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:22 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:22 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:22.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:22 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:23.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:23 smithi086 ceph-mon[34687]: pgmap v1008: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.0 MiB/s wr, 50 op/s 2024-05-27T08:05:23.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:23 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:23.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:23 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:23.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:23 smithi149 ceph-mon[35408]: pgmap v1008: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.0 MiB/s wr, 50 op/s 2024-05-27T08:05:23.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:23 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:23.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:23 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:23.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:23 smithi063 ceph-mon[31613]: pgmap v1008: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 8.0 MiB/s wr, 50 op/s 2024-05-27T08:05:23.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:23 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:23.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:23 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:24.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:05:24.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:24 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:05:24.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:05:24.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:24 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:05:24.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:05:24.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:05:24.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:24 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:05:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:25 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T08:05:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:25 smithi086 ceph-mon[34687]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T08:05:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:25 smithi086 ceph-mon[34687]: pgmap v1009: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.4 MiB/s wr, 43 op/s 2024-05-27T08:05:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:25 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T08:05:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:25 smithi086 ceph-mon[34687]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T08:05:25.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:25 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:25 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T08:05:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:25 smithi149 ceph-mon[35408]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T08:05:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:25 smithi149 ceph-mon[35408]: pgmap v1009: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.4 MiB/s wr, 43 op/s 2024-05-27T08:05:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:25 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T08:05:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:25 smithi149 ceph-mon[35408]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T08:05:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:25 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:25 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T08:05:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:25 smithi063 ceph-mon[31613]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T08:05:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:25 smithi063 ceph-mon[31613]: pgmap v1009: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.4 MiB/s wr, 43 op/s 2024-05-27T08:05:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:25 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T08:05:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:25 smithi063 ceph-mon[31613]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T08:05:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:25 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:05:27.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:27 smithi086 ceph-mon[34687]: pgmap v1010: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.0 MiB/s wr, 46 op/s 2024-05-27T08:05:27.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:27 smithi149 ceph-mon[35408]: pgmap v1010: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.0 MiB/s wr, 46 op/s 2024-05-27T08:05:27.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:27 smithi063 ceph-mon[31613]: pgmap v1010: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.0 MiB/s wr, 46 op/s 2024-05-27T08:05:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:29 smithi086 ceph-mon[34687]: pgmap v1011: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-05-27T08:05:29.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:29 smithi149 ceph-mon[35408]: pgmap v1011: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-05-27T08:05:29.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:29 smithi063 ceph-mon[31613]: pgmap v1011: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-05-27T08:05:31.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:31 smithi086 ceph-mon[34687]: pgmap v1012: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-05-27T08:05:31.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:31 smithi149 ceph-mon[35408]: pgmap v1012: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-05-27T08:05:31.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:31 smithi063 ceph-mon[31613]: pgmap v1012: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 11 op/s 2024-05-27T08:05:33.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:33 smithi086 ceph-mon[34687]: pgmap v1013: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T08:05:33.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:33 smithi149 ceph-mon[35408]: pgmap v1013: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T08:05:33.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:33 smithi063 ceph-mon[31613]: pgmap v1013: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 14 op/s 2024-05-27T08:05:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.161+0000 7f9687200640 1 -- 172.21.15.63:0/1665395412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968810bcd0 msgr2=0x7f9688105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.161+0000 7f9687200640 1 --2- 172.21.15.63:0/1665395412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968810bcd0 0x7f9688105870 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f9678009900 tx=0x7f967802a410 comp rx=0 tx=0).stop 2024-05-27T08:05:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.161+0000 7f9687200640 1 -- 172.21.15.63:0/1665395412 shutdown_connections 2024-05-27T08:05:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.161+0000 7f9687200640 1 --2- 172.21.15.63:0/1665395412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9688106420 0x7f96881127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.161+0000 7f9687200640 1 --2- 172.21.15.63:0/1665395412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f968810c170 0x7f9688105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.161+0000 7f9687200640 1 --2- 172.21.15.63:0/1665395412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968810bcd0 0x7f9688105870 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.161+0000 7f9687200640 1 -- 172.21.15.63:0/1665395412 >> 172.21.15.63:0/1665395412 conn(0x7f96881013d0 msgr2=0x7f9688103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.161+0000 7f9687200640 1 -- 172.21.15.63:0/1665395412 shutdown_connections 2024-05-27T08:05:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.161+0000 7f9687200640 1 -- 172.21.15.63:0/1665395412 wait complete. 2024-05-27T08:05:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.162+0000 7f9687200640 1 Processor -- start 2024-05-27T08:05:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.162+0000 7f9687200640 1 -- start start 2024-05-27T08:05:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9687200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9688106420 0x7f96881a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9687200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968810bcd0 0x7f96881a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9687200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968810c170 0x7f968819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9687200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9688115d70 con 0x7f968810c170 2024-05-27T08:05:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9687200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9688115bf0 con 0x7f968810bcd0 2024-05-27T08:05:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9687200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9688115ef0 con 0x7f9688106420 2024-05-27T08:05:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9686800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968810c170 0x7f968819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9686800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968810c170 0x7f968819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57374/0 (socket says 172.21.15.63:57374) 2024-05-27T08:05:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9686800640 1 -- 172.21.15.63:0/3338760615 learned_addr learned my addr 172.21.15.63:0/3338760615 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9685e00640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9688106420 0x7f96881a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.163+0000 7f9685400640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968810bcd0 0x7f96881a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.164+0000 7f9686800640 1 -- 172.21.15.63:0/3338760615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9688106420 msgr2=0x7f96881a24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.164+0000 7f9686800640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9688106420 0x7f96881a24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.164+0000 7f9686800640 1 -- 172.21.15.63:0/3338760615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968810bcd0 msgr2=0x7f96881a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.164+0000 7f9686800640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968810bcd0 0x7f96881a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.164+0000 7f9686800640 1 -- 172.21.15.63:0/3338760615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968819cd80 con 0x7f968810c170 2024-05-27T08:05:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.164+0000 7f9686800640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968810c170 0x7f968819c650 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f968000eaa0 tx=0x7f968000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.165+0000 7f967ea00640 1 -- 172.21.15.63:0/3338760615 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968000cdb0 con 0x7f968810c170 2024-05-27T08:05:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.165+0000 7f967ea00640 1 -- 172.21.15.63:0/3338760615 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9680005020 con 0x7f968810c170 2024-05-27T08:05:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.165+0000 7f9687200640 1 -- 172.21.15.63:0/3338760615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968819d010 con 0x7f968810c170 2024-05-27T08:05:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.165+0000 7f967ea00640 1 -- 172.21.15.63:0/3338760615 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96800116d0 con 0x7f968810c170 2024-05-27T08:05:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.165+0000 7f9687200640 1 -- 172.21.15.63:0/3338760615 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96881a9270 con 0x7f968810c170 2024-05-27T08:05:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.169+0000 7f9693df3740 1 Processor -- start 2024-05-27T08:05:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.169+0000 7f967ea00640 1 -- 172.21.15.63:0/3338760615 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f96800051d0 con 0x7f968810c170 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.170+0000 7f9693df3740 1 -- start start 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.170+0000 7f9693df3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c30aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.170+0000 7f9693df3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c30afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.170+0000 7f9693df3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf474d0 0x55b93c30b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.170+0000 7f9693df3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b93c27bb90 con 0x55b93bf681b0 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.170+0000 7f9693df3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b93c26b730 con 0x55b93bf3f2f0 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.170+0000 7f9693df3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b93c26f770 con 0x55b93bf474d0 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.171+0000 7f9685400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c30aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.171+0000 7f9685e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c30afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.171+0000 7f9685400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c30aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57380/0 (socket says 172.21.15.63:57380) 2024-05-27T08:05:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.171+0000 7f9685400640 1 -- 172.21.15.63:0/4026209280 learned_addr learned my addr 172.21.15.63:0/4026209280 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.171+0000 7f9685400640 1 --2- 172.21.15.63:0/4026209280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf474d0 0x55b93c30b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.172+0000 7f9685400640 1 -- 172.21.15.63:0/4026209280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf474d0 msgr2=0x55b93c30b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.172+0000 7f9685400640 1 --2- 172.21.15.63:0/4026209280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf474d0 0x55b93c30b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.172+0000 7f9685400640 1 -- 172.21.15.63:0/4026209280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 msgr2=0x55b93c30afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.172+0000 7f9685400640 1 --2- 172.21.15.63:0/4026209280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c30afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.172+0000 7f9685400640 1 -- 172.21.15.63:0/4026209280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93c2048b0 con 0x55b93bf681b0 2024-05-27T08:05:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.172+0000 7f9685e00640 1 --2- 172.21.15.63:0/4026209280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c30afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:05:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.172+0000 7f9685400640 1 --2- 172.21.15.63:0/4026209280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c30aa60 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f968806c3a0 tx=0x7f967400b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863da1f057b0f49d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.173+0000 7f967ea00640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f966c0b2950 0x7f966c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.173+0000 7f9673400640 1 -- 172.21.15.63:0/4026209280 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674005020 con 0x55b93bf681b0 2024-05-27T08:05:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.173+0000 7f9673400640 1 -- 172.21.15.63:0/4026209280 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96740047a0 con 0x55b93bf681b0 2024-05-27T08:05:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.173+0000 7f9673400640 1 -- 172.21.15.63:0/4026209280 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674005650 con 0x55b93bf681b0 2024-05-27T08:05:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.173+0000 7f967ea00640 1 -- 172.21.15.63:0/3338760615 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f96800e27a0 con 0x7f968810c170 2024-05-27T08:05:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.173+0000 7f9685e00640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f966c0b2950 0x7f966c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.174+0000 7f9693df3740 1 -- 172.21.15.63:0/4026209280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 msgr2=0x55b93c30aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.174+0000 7f9693df3740 1 --2- 172.21.15.63:0/4026209280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c30aa60 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f968806c3a0 tx=0x7f967400b740 comp rx=0 tx=0).stop 2024-05-27T08:05:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.174+0000 7f9685e00640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f966c0b2950 0x7f966c0b4e60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9678004770 tx=0x7f96780058d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.174+0000 7f9693df3740 1 -- 172.21.15.63:0/4026209280 shutdown_connections 2024-05-27T08:05:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.174+0000 7f9693df3740 1 --2- 172.21.15.63:0/4026209280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf474d0 0x55b93c30b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.174+0000 7f9693df3740 1 --2- 172.21.15.63:0/4026209280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c30afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.174+0000 7f9693df3740 1 --2- 172.21.15.63:0/4026209280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c30aa60 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.174+0000 7f9693df3740 1 -- 172.21.15.63:0/4026209280 >> 172.21.15.63:0/4026209280 conn(0x55b93bf5c300 msgr2=0x55b93c307210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.174+0000 7f9693df3740 1 -- 172.21.15.63:0/4026209280 shutdown_connections 2024-05-27T08:05:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.174+0000 7f9693df3740 1 -- 172.21.15.63:0/4026209280 wait complete. 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.175+0000 7f9693df3740 1 -- start start 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.175+0000 7f9693df3740 1 Processor -- start 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9693df3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c314c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9693df3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf474d0 0x55b93c315170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9693df3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c320ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9693df3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b93bf51790 con 0x55b93bf681b0 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9693df3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b93c213bc0 con 0x55b93bf3f2f0 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9693df3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b93c317b90 con 0x55b93bf474d0 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9693df3740 10 client.0 _ll_register_callbacks cb 0x55b93c1f1400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9686800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c320ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9686800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c320ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57392/0 (socket says 172.21.15.63:57392) 2024-05-27T08:05:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9686800640 1 -- 172.21.15.63:0/1943371604 learned_addr learned my addr 172.21.15.63:0/1943371604 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9685400640 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf474d0 0x55b93c315170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.176+0000 7f9685e00640 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c314c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.177+0000 7f9686800640 1 -- 172.21.15.63:0/1943371604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf474d0 msgr2=0x55b93c315170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.177+0000 7f9686800640 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf474d0 0x55b93c315170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.177+0000 7f9686800640 1 -- 172.21.15.63:0/1943371604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 msgr2=0x55b93c314c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.177+0000 7f9686800640 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c314c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.177+0000 7f9686800640 1 -- 172.21.15.63:0/1943371604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93bf80210 con 0x55b93bf681b0 2024-05-27T08:05:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.177+0000 7f9685e00640 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c314c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.177+0000 7f9686800640 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c320ec0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f96800a57f0 tx=0x7f96800a9770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f83e5297a7991a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.177+0000 7f9667e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.178+0000 7f9667e00640 1 -- 172.21.15.63:0/1943371604 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968001f070 con 0x55b93bf681b0 2024-05-27T08:05:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.178+0000 7f9667e00640 1 -- 172.21.15.63:0/1943371604 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f96800e2380 con 0x55b93bf681b0 2024-05-27T08:05:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.178+0000 7f9693df3740 1 -- 172.21.15.63:0/1943371604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96740047a0 con 0x55b93bf681b0 2024-05-27T08:05:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.178+0000 7f9667e00640 1 -- 172.21.15.63:0/1943371604 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f96800b2520 con 0x55b93bf681b0 2024-05-27T08:05:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.178+0000 7f9693df3740 10 client.21567 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.178+0000 7f9693df3740 1 -- 172.21.15.63:0/1943371604 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b93c04c620 con 0x55b93bf681b0 2024-05-27T08:05:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.179+0000 7f9667e00640 1 -- 172.21.15.63:0/1943371604 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96740047a0 con 0x55b93bf681b0 2024-05-27T08:05:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.179+0000 7f9667e00640 1 -- 172.21.15.63:0/1943371604 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f968000f040 con 0x55b93bf681b0 2024-05-27T08:05:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.180+0000 7f9693df3740 10 client.21567 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.180+0000 7f9693df3740 10 client.21567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.180+0000 7f9693df3740 20 client.21567 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.180+0000 7f9693df3740 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93bf26d80 0x55b93c326e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.180+0000 7f9693df3740 4 client.21567 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.180+0000 7f9693df3740 1 -- 172.21.15.63:0/1943371604 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b93c27dae0 con 0x55b93bf26d80 2024-05-27T08:05:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.181+0000 7f9685e00640 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93bf26d80 0x55b93c326e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.182+0000 7f9685e00640 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93bf26d80 0x55b93c326e60 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5edfd12bf543c3 server_cookie=7c766eb294e360a8 in_seq=0 out_seq=0 2024-05-27T08:05:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.182+0000 7f9667e00640 10 client.21567 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9667e00640 1 -- 172.21.15.63:0/1943371604 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b93c27dae0 con 0x55b93bf26d80 2024-05-27T08:05:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9667e00640 10 client.21567 handle_command_reply: tid=0 2024-05-27T08:05:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9693df3740 1 client.21567 shutdown 2024-05-27T08:05:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9693df3740 1 -- 172.21.15.63:0/1943371604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 msgr2=0x55b93c320ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9693df3740 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c320ec0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f96800a57f0 tx=0x7f96800a9770 comp rx=0 tx=0).stop 2024-05-27T08:05:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9693df3740 1 -- 172.21.15.63:0/1943371604 shutdown_connections 2024-05-27T08:05:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9693df3740 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93bf26d80 0x55b93c326e60 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9693df3740 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf681b0 0x55b93c320ec0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9693df3740 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf474d0 0x55b93c315170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9667e00640 0 client.21567 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9693df3740 1 --2- 172.21.15.63:0/1943371604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c314c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9693df3740 1 -- 172.21.15.63:0/1943371604 >> 172.21.15.63:0/1943371604 conn(0x55b93bf5c300 msgr2=0x55b93c30a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.184+0000 7f9693df3740 1 -- 172.21.15.63:0/1943371604 shutdown_connections 2024-05-27T08:05:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.185+0000 7f9693df3740 1 -- 172.21.15.63:0/1943371604 wait complete. 2024-05-27T08:05:34.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.185+0000 7f9693df3740 20 client.21567 trim_cache size 0 max 16384 2024-05-27T08:05:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.207+0000 7f9693df3740 1 Processor -- start 2024-05-27T08:05:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.207+0000 7f9693df3740 1 -- start start 2024-05-27T08:05:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.207+0000 7f9693df3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c317360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.207+0000 7f9693df3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf474d0 0x55b93c32c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.207+0000 7f9693df3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 0x55b93c32cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.207+0000 7f9693df3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96881b5bc0 con 0x55b93bf474d0 2024-05-27T08:05:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.207+0000 7f9693df3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96881b3490 con 0x55b93bf3f2f0 2024-05-27T08:05:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.207+0000 7f9693df3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f96881b1300 con 0x55b93bf681b0 2024-05-27T08:05:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.208+0000 7f9685400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf474d0 0x55b93c32c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.208+0000 7f9685400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf474d0 0x55b93c32c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57402/0 (socket says 172.21.15.63:57402) 2024-05-27T08:05:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.208+0000 7f9685400640 1 -- 172.21.15.63:0/1398051999 learned_addr learned my addr 172.21.15.63:0/1398051999 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.208+0000 7f9685e00640 1 --2- 172.21.15.63:0/1398051999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c317360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.208+0000 7f9686800640 1 --2- 172.21.15.63:0/1398051999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 0x55b93c32cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.208+0000 7f9685400640 1 -- 172.21.15.63:0/1398051999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 msgr2=0x55b93c32cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.208+0000 7f9685400640 1 --2- 172.21.15.63:0/1398051999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 0x55b93c32cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.208+0000 7f9685400640 1 -- 172.21.15.63:0/1398051999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 msgr2=0x55b93c317360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.208+0000 7f9685400640 1 --2- 172.21.15.63:0/1398051999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c317360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.208+0000 7f9685400640 1 -- 172.21.15.63:0/1398051999 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93c313180 con 0x55b93bf474d0 2024-05-27T08:05:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.209+0000 7f9686800640 1 --2- 172.21.15.63:0/1398051999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 0x55b93c32cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.209+0000 7f9685400640 1 --2- 172.21.15.63:0/1398051999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf474d0 0x55b93c32c640 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f968806c3a0 tx=0x7f9674005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7298cf89529a895 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.209+0000 7f9672a00640 1 -- 172.21.15.63:0/1398051999 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674005040 con 0x55b93bf474d0 2024-05-27T08:05:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.210+0000 7f9672a00640 1 -- 172.21.15.63:0/1398051999 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9674004660 con 0x55b93bf474d0 2024-05-27T08:05:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.210+0000 7f9672a00640 1 -- 172.21.15.63:0/1398051999 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96740106a0 con 0x55b93bf474d0 2024-05-27T08:05:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.211+0000 7f9693df3740 1 -- 172.21.15.63:0/1398051999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf474d0 msgr2=0x55b93c32c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.211+0000 7f9693df3740 1 --2- 172.21.15.63:0/1398051999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf474d0 0x55b93c32c640 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f968806c3a0 tx=0x7f9674005f30 comp rx=0 tx=0).stop 2024-05-27T08:05:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.211+0000 7f9693df3740 1 -- 172.21.15.63:0/1398051999 shutdown_connections 2024-05-27T08:05:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.211+0000 7f9693df3740 1 --2- 172.21.15.63:0/1398051999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 0x55b93c32cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.211+0000 7f9693df3740 1 --2- 172.21.15.63:0/1398051999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf474d0 0x55b93c32c640 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.211+0000 7f9693df3740 1 --2- 172.21.15.63:0/1398051999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf3f2f0 0x55b93c317360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.211+0000 7f9693df3740 1 -- 172.21.15.63:0/1398051999 >> 172.21.15.63:0/1398051999 conn(0x55b93bf5c300 msgr2=0x55b93c314ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.211+0000 7f9693df3740 1 -- 172.21.15.63:0/1398051999 shutdown_connections 2024-05-27T08:05:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.211+0000 7f9693df3740 1 -- 172.21.15.63:0/1398051999 wait complete. 2024-05-27T08:05:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.212+0000 7f9693df3740 1 -- start start 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.212+0000 7f9693df3740 1 Processor -- start 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.212+0000 7f9693df3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf3f2f0 0x55b93c31bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.212+0000 7f9693df3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf474d0 0x55b93c31c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.212+0000 7f9693df3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 0x55b93c31c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.212+0000 7f9693df3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96881b1950 con 0x55b93bf3f2f0 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.212+0000 7f9693df3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96881b2dc0 con 0x55b93bf474d0 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.212+0000 7f9693df3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f96881b2020 con 0x55b93bf681b0 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.212+0000 7f9693df3740 10 client.0 _ll_register_callbacks cb 0x7f96800a5630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.213+0000 7f9685e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf3f2f0 0x55b93c31bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.213+0000 7f9685e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf3f2f0 0x55b93c31bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57406/0 (socket says 172.21.15.63:57406) 2024-05-27T08:05:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.213+0000 7f9685e00640 1 -- 172.21.15.63:0/883770757 learned_addr learned my addr 172.21.15.63:0/883770757 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.213+0000 7f9685400640 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf474d0 0x55b93c31c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.213+0000 7f9686800640 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 0x55b93c31c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.213+0000 7f9685e00640 1 -- 172.21.15.63:0/883770757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 msgr2=0x55b93c31c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.213+0000 7f9685e00640 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 0x55b93c31c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.213+0000 7f9685e00640 1 -- 172.21.15.63:0/883770757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf474d0 msgr2=0x55b93c31c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.213+0000 7f9685e00640 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf474d0 0x55b93c31c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.213+0000 7f9685e00640 1 -- 172.21.15.63:0/883770757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93c38d650 con 0x55b93bf3f2f0 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.214+0000 7f9685e00640 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf3f2f0 0x55b93c31bd50 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f9678006010 tx=0x7f9678009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642bfa2d75bd48a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.214+0000 7f9667e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.214+0000 7f9667e00640 1 -- 172.21.15.63:0/883770757 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678035040 con 0x55b93bf3f2f0 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.214+0000 7f9667e00640 1 -- 172.21.15.63:0/883770757 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f96780037f0 con 0x55b93bf3f2f0 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.214+0000 7f9693df3740 1 -- 172.21.15.63:0/883770757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9674004660 con 0x55b93bf3f2f0 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.215+0000 7f9667e00640 1 -- 172.21.15.63:0/883770757 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f967805f050 con 0x55b93bf3f2f0 2024-05-27T08:05:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.215+0000 7f9693df3740 10 client.21573 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.215+0000 7f9693df3740 1 -- 172.21.15.63:0/883770757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b93c38e900 con 0x55b93bf3f2f0 2024-05-27T08:05:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.215+0000 7f9667e00640 1 -- 172.21.15.63:0/883770757 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9674004660 con 0x55b93bf3f2f0 2024-05-27T08:05:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.215+0000 7f9667e00640 1 -- 172.21.15.63:0/883770757 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9678044040 con 0x55b93bf3f2f0 2024-05-27T08:05:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.215+0000 7f9693df3740 10 client.21573 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.216+0000 7f9693df3740 10 client.21573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.216+0000 7f9693df3740 20 client.21573 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.216+0000 7f9693df3740 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93befc1c0 0x55b93c3910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.216+0000 7f9693df3740 4 client.21573 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:05:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.216+0000 7f9693df3740 1 -- 172.21.15.63:0/883770757 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b93c2772c0 con 0x55b93befc1c0 2024-05-27T08:05:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.216+0000 7f9685400640 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93befc1c0 0x55b93c3910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.217+0000 7f9685400640 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93befc1c0 0x55b93c3910c0 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95300a9835517e28 server_cookie=4654b93faf4a5998 in_seq=0 out_seq=0 2024-05-27T08:05:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.217+0000 7f9667e00640 10 client.21573 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9667e00640 1 -- 172.21.15.63:0/883770757 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b93c2772c0 con 0x55b93befc1c0 2024-05-27T08:05:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9667e00640 10 client.21573 handle_command_reply: tid=0 2024-05-27T08:05:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9693df3740 1 client.21573 shutdown 2024-05-27T08:05:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9693df3740 1 -- 172.21.15.63:0/883770757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf3f2f0 msgr2=0x55b93c31bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9693df3740 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf3f2f0 0x55b93c31bd50 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f9678006010 tx=0x7f9678009260 comp rx=0 tx=0).stop 2024-05-27T08:05:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9693df3740 1 -- 172.21.15.63:0/883770757 shutdown_connections 2024-05-27T08:05:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9693df3740 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93befc1c0 0x55b93c3910c0 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9693df3740 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93bf681b0 0x55b93c31c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9693df3740 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93bf474d0 0x55b93c31c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9693df3740 1 --2- 172.21.15.63:0/883770757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93bf3f2f0 0x55b93c31bd50 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9693df3740 1 -- 172.21.15.63:0/883770757 >> 172.21.15.63:0/883770757 conn(0x55b93bf5c300 msgr2=0x55b93c316000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.247+0000 7f9667e00640 0 client.21573 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.248+0000 7f9693df3740 1 -- 172.21.15.63:0/883770757 shutdown_connections 2024-05-27T08:05:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.248+0000 7f9693df3740 1 -- 172.21.15.63:0/883770757 wait complete. 2024-05-27T08:05:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.248+0000 7f9693df3740 20 client.21573 trim_cache size 0 max 16384 2024-05-27T08:05:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:05:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:05:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 -- 172.21.15.63:0/3338760615 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f966c0b2950 msgr2=0x7f966c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f966c0b2950 0x7f966c0b4e60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9678004770 tx=0x7f96780058d0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 -- 172.21.15.63:0/3338760615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968810c170 msgr2=0x7f968819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968810c170 0x7f968819c650 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f968000eaa0 tx=0x7f968000ef60 comp rx=0 tx=0).stop 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 -- 172.21.15.63:0/3338760615 shutdown_connections 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f966c0b2950 0x7f966c0b4e60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968810c170 0x7f968819c650 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968810bcd0 0x7f96881a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 --2- 172.21.15.63:0/3338760615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9688106420 0x7f96881a24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 -- 172.21.15.63:0/3338760615 >> 172.21.15.63:0/3338760615 conn(0x7f96881013d0 msgr2=0x7f9688103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 -- 172.21.15.63:0/3338760615 shutdown_connections 2024-05-27T08:05:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:34.249+0000 7f9687200640 1 -- 172.21.15.63:0/3338760615 wait complete. 2024-05-27T08:05:35.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:35 smithi086 ceph-mon[34687]: pgmap v1014: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T08:05:35.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:35 smithi149 ceph-mon[35408]: pgmap v1014: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T08:05:35.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:35 smithi063 ceph-mon[31613]: pgmap v1014: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T08:05:37.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:37 smithi086 ceph-mon[34687]: pgmap v1015: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:05:37.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:37 smithi149 ceph-mon[35408]: pgmap v1015: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:05:37.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:37 smithi063 ceph-mon[31613]: pgmap v1015: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:05:39.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:39 smithi086 ceph-mon[34687]: pgmap v1016: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:05:39.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:39 smithi149 ceph-mon[35408]: pgmap v1016: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:05:39.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:39 smithi063 ceph-mon[31613]: pgmap v1016: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:05:41.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:41 smithi086 ceph-mon[34687]: pgmap v1017: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:05:41.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:41 smithi149 ceph-mon[35408]: pgmap v1017: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:05:41.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:41 smithi063 ceph-mon[31613]: pgmap v1017: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:05:43.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:43 smithi086 ceph-mon[34687]: pgmap v1018: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:05:43.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:43 smithi149 ceph-mon[35408]: pgmap v1018: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:05:43.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:43 smithi063 ceph-mon[31613]: pgmap v1018: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:05:45.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:45 smithi086 ceph-mon[34687]: pgmap v1019: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:05:45.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:45 smithi149 ceph-mon[35408]: pgmap v1019: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:05:45.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:45 smithi063 ceph-mon[31613]: pgmap v1019: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:05:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.546+0000 7fba98a00640 1 -- 172.21.15.63:0/3023169521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9410b670 msgr2=0x7fba9410da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.546+0000 7fba98a00640 1 --2- 172.21.15.63:0/3023169521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9410b670 0x7fba9410da90 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fba80009900 tx=0x7fba8002a480 comp rx=0 tx=0).stop 2024-05-27T08:05:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.546+0000 7fba98a00640 1 -- 172.21.15.63:0/3023169521 shutdown_connections 2024-05-27T08:05:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.546+0000 7fba98a00640 1 --2- 172.21.15.63:0/3023169521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba9410e020 0x7fba94110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.546+0000 7fba98a00640 1 --2- 172.21.15.63:0/3023169521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba9410b670 0x7fba9410da90 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.546+0000 7fba98a00640 1 --2- 172.21.15.63:0/3023169521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba94104c10 0x7fba94105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.546+0000 7fba98a00640 1 -- 172.21.15.63:0/3023169521 >> 172.21.15.63:0/3023169521 conn(0x7fba940fe210 msgr2=0x7fba94100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.546+0000 7fba98a00640 1 -- 172.21.15.63:0/3023169521 shutdown_connections 2024-05-27T08:05:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.546+0000 7fba98a00640 1 -- 172.21.15.63:0/3023169521 wait complete. 2024-05-27T08:05:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.547+0000 7fba98a00640 1 Processor -- start 2024-05-27T08:05:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.548+0000 7fba98a00640 1 -- start start 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.548+0000 7fba98a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba94104c10 0x7fba9419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.548+0000 7fba98a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba9410b670 0x7fba9419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.548+0000 7fba98a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba9410e020 0x7fba9419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.548+0000 7fba98a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba94113bc0 con 0x7fba94104c10 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.548+0000 7fba98a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba94113a40 con 0x7fba9410b670 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.548+0000 7fba98a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba94113d40 con 0x7fba9410e020 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.548+0000 7fba93400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba94104c10 0x7fba9419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.548+0000 7fba93400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba94104c10 0x7fba9419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50700/0 (socket says 172.21.15.63:50700) 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.549+0000 7fba93400640 1 -- 172.21.15.63:0/2582285185 learned_addr learned my addr 172.21.15.63:0/2582285185 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.549+0000 7fba92a00640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba9410b670 0x7fba9419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.549+0000 7fba93e00640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba9410e020 0x7fba9419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.549+0000 7fba93400640 1 -- 172.21.15.63:0/2582285185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba9410e020 msgr2=0x7fba9419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.549+0000 7fba93400640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba9410e020 0x7fba9419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.549+0000 7fba93400640 1 -- 172.21.15.63:0/2582285185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba9410b670 msgr2=0x7fba9419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.549+0000 7fba93400640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba9410b670 0x7fba9419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.549+0000 7fba93400640 1 -- 172.21.15.63:0/2582285185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9419f480 con 0x7fba94104c10 2024-05-27T08:05:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.549+0000 7fba93e00640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba9410e020 0x7fba9419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.549+0000 7fba93400640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba94104c10 0x7fba9419a620 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fba8400d950 tx=0x7fba8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.550+0000 7fba8be00640 1 -- 172.21.15.63:0/2582285185 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84017070 con 0x7fba94104c10 2024-05-27T08:05:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.550+0000 7fba8be00640 1 -- 172.21.15.63:0/2582285185 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba84009c30 con 0x7fba94104c10 2024-05-27T08:05:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.550+0000 7fba8be00640 1 -- 172.21.15.63:0/2582285185 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8400f620 con 0x7fba94104c10 2024-05-27T08:05:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.550+0000 7fba98a00640 1 -- 172.21.15.63:0/2582285185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9419f770 con 0x7fba94104c10 2024-05-27T08:05:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.550+0000 7fba98a00640 1 -- 172.21.15.63:0/2582285185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba941a70f0 con 0x7fba94104c10 2024-05-27T08:05:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.554+0000 7fbaa0f9f740 1 Processor -- start 2024-05-27T08:05:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.554+0000 7fbaa0f9f740 1 -- start start 2024-05-27T08:05:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.554+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.554+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 0x55f3b00cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.555+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 0x55f3b00cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.555+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f3b003cb90 con 0x55f3afd291b0 2024-05-27T08:05:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.555+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3b002c730 con 0x55f3afd084d0 2024-05-27T08:05:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.555+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f3b0030770 con 0x55f3afd002f0 2024-05-27T08:05:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.555+0000 7fba93e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.555+0000 7fba8be00640 1 -- 172.21.15.63:0/2582285185 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fba84010460 con 0x7fba94104c10 2024-05-27T08:05:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.555+0000 7fba93e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50702/0 (socket says 172.21.15.63:50702) 2024-05-27T08:05:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.555+0000 7fba93e00640 1 -- 172.21.15.63:0/3025130749 learned_addr learned my addr 172.21.15.63:0/3025130749 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.555+0000 7fba92a00640 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 0x55f3b00cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.555+0000 7fba93400640 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 0x55f3b00cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.556+0000 7fba93e00640 1 -- 172.21.15.63:0/3025130749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 msgr2=0x55f3b00cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.556+0000 7fba93e00640 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 0x55f3b00cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.556+0000 7fba93e00640 1 -- 172.21.15.63:0/3025130749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 msgr2=0x55f3b00cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.556+0000 7fba93e00640 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 0x55f3b00cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.556+0000 7fba93e00640 1 -- 172.21.15.63:0/3025130749 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3affc58b0 con 0x55f3afd291b0 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.556+0000 7fba92a00640 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 0x55f3b00cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.556+0000 7fba93400640 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 0x55f3b00cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.556+0000 7fba93e00640 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00cba60 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fba8c007f30 tx=0x7fba8c00c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d97d86c61c723e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.556+0000 7fba7fe00640 1 -- 172.21.15.63:0/3025130749 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c014070 con 0x55f3afd291b0 2024-05-27T08:05:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.557+0000 7fba7fe00640 1 -- 172.21.15.63:0/3025130749 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba8c00fce0 con 0x55f3afd291b0 2024-05-27T08:05:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.557+0000 7fba7fe00640 1 -- 172.21.15.63:0/3025130749 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00ebc0 con 0x55f3afd291b0 2024-05-27T08:05:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.557+0000 7fba8be00640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba780b29d0 0x7fba780b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.557+0000 7fba93400640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba780b29d0 0x7fba780b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.557+0000 7fba8be00640 1 -- 172.21.15.63:0/2582285185 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fba840e26a0 con 0x7fba94104c10 2024-05-27T08:05:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.558+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3025130749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 msgr2=0x55f3b00cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.558+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00cba60 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fba8c007f30 tx=0x7fba8c00c750 comp rx=0 tx=0).stop 2024-05-27T08:05:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.558+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3025130749 shutdown_connections 2024-05-27T08:05:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.558+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 0x55f3b00cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.558+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 0x55f3b00cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.558+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/3025130749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00cba60 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.558+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3025130749 >> 172.21.15.63:0/3025130749 conn(0x55f3afd1d300 msgr2=0x55f3b00c8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.558+0000 7fba93400640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba780b29d0 0x7fba780b4ee0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55f3b00c9550 tx=0x7fba8400df40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.558+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3025130749 shutdown_connections 2024-05-27T08:05:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.558+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3025130749 wait complete. 2024-05-27T08:05:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.559+0000 7fbaa0f9f740 1 -- start start 2024-05-27T08:05:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.559+0000 7fbaa0f9f740 1 Processor -- start 2024-05-27T08:05:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.560+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 0x55f3b00d5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.560+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd084d0 0x55f3b00d6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.560+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd291b0 0x55f3b00e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.560+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f3afd12790 con 0x55f3afd084d0 2024-05-27T08:05:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.560+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3affd4bc0 con 0x55f3afd291b0 2024-05-27T08:05:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.560+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f3b00d8b90 con 0x55f3afd002f0 2024-05-27T08:05:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.560+0000 7fbaa0f9f740 10 client.0 _ll_register_callbacks cb 0x55f3affb2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.560+0000 7fba92a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd084d0 0x55f3b00d6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.560+0000 7fba92a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd084d0 0x55f3b00d6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50706/0 (socket says 172.21.15.63:50706) 2024-05-27T08:05:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.560+0000 7fba92a00640 1 -- 172.21.15.63:0/1507045368 learned_addr learned my addr 172.21.15.63:0/1507045368 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.561+0000 7fba93e00640 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 0x55f3b00d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.561+0000 7fba93e00640 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd291b0 0x55f3b00e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.561+0000 7fba92a00640 1 -- 172.21.15.63:0/1507045368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 msgr2=0x55f3b00d5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.561+0000 7fba92a00640 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 0x55f3b00d5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.561+0000 7fba92a00640 1 -- 172.21.15.63:0/1507045368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd291b0 msgr2=0x55f3b00e62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.561+0000 7fba92a00640 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd291b0 0x55f3b00e62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.561+0000 7fba92a00640 1 -- 172.21.15.63:0/1507045368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3afd41210 con 0x55f3afd084d0 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.561+0000 7fba92a00640 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd084d0 0x55f3b00d6150 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fba8002aa70 tx=0x7fba800052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a3cd287358626d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.561+0000 7fba7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.562+0000 7fba7cc00640 1 -- 172.21.15.63:0/1507045368 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80035030 con 0x55f3afd084d0 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.562+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/1507045368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba8c00fce0 con 0x55f3afd084d0 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.562+0000 7fba7cc00640 1 -- 172.21.15.63:0/1507045368 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fba800048c0 con 0x55f3afd084d0 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.562+0000 7fba7cc00640 1 -- 172.21.15.63:0/1507045368 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fba8004d050 con 0x55f3afd084d0 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.562+0000 7fbaa0f9f740 10 client.21585 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.562+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/1507045368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f3afe0d620 con 0x55f3afd084d0 2024-05-27T08:05:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.562+0000 7fba7cc00640 1 -- 172.21.15.63:0/1507045368 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba8c00fce0 con 0x55f3afd084d0 2024-05-27T08:05:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.563+0000 7fba7cc00640 1 -- 172.21.15.63:0/1507045368 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fba80003cb0 con 0x55f3afd084d0 2024-05-27T08:05:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.563+0000 7fbaa0f9f740 10 client.21585 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.563+0000 7fbaa0f9f740 10 client.21585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.563+0000 7fbaa0f9f740 20 client.21585 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.563+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3afce7d80 0x55f3b00ec210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.563+0000 7fbaa0f9f740 4 client.21585 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.563+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/1507045368 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3b003eae0 con 0x55f3afce7d80 2024-05-27T08:05:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.564+0000 7fba93e00640 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3afce7d80 0x55f3b00ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.564+0000 7fba93e00640 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3afce7d80 0x55f3b00ec210 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923ec127e2fdaf1e server_cookie=bd40018b754da423 in_seq=0 out_seq=0 2024-05-27T08:05:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.564+0000 7fba7cc00640 10 client.21585 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.566+0000 7fba7cc00640 1 -- 172.21.15.63:0/1507045368 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f3b003eae0 con 0x55f3afce7d80 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.566+0000 7fba7cc00640 10 client.21585 handle_command_reply: tid=0 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.566+0000 7fbaa0f9f740 1 client.21585 shutdown 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/1507045368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd084d0 msgr2=0x55f3b00d6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd084d0 0x55f3b00d6150 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fba8002aa70 tx=0x7fba800052f0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/1507045368 shutdown_connections 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3afce7d80 0x55f3b00ec210 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd291b0 0x55f3b00e62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd084d0 0x55f3b00d6150 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/1507045368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd002f0 0x55f3b00d5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/1507045368 >> 172.21.15.63:0/1507045368 conn(0x55f3afd1d300 msgr2=0x55f3b00cb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fba7cc00640 0 client.21585 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/1507045368 shutdown_connections 2024-05-27T08:05:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/1507045368 wait complete. 2024-05-27T08:05:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.567+0000 7fbaa0f9f740 20 client.21585 trim_cache size 0 max 16384 2024-05-27T08:05:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.591+0000 7fbaa0f9f740 1 Processor -- start 2024-05-27T08:05:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.591+0000 7fbaa0f9f740 1 -- start start 2024-05-27T08:05:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.591+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd002f0 0x55f3b00d1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.591+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 0x55f3b00d5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.591+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd291b0 0x55f3b00d60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba941b1770 con 0x55f3afd002f0 2024-05-27T08:05:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba941af040 con 0x55f3afd084d0 2024-05-27T08:05:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba941aceb0 con 0x55f3afd291b0 2024-05-27T08:05:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fba93e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd002f0 0x55f3b00d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fba93e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd002f0 0x55f3b00d1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50710/0 (socket says 172.21.15.63:50710) 2024-05-27T08:05:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fba93e00640 1 -- 172.21.15.63:0/2730950473 learned_addr learned my addr 172.21.15.63:0/2730950473 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fba92a00640 1 --2- 172.21.15.63:0/2730950473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 0x55f3b00d5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fba93e00640 1 -- 172.21.15.63:0/2730950473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd291b0 msgr2=0x55f3b00d60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fba93e00640 1 --2- 172.21.15.63:0/2730950473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd291b0 0x55f3b00d60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fba93e00640 1 -- 172.21.15.63:0/2730950473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 msgr2=0x55f3b00d5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fba93e00640 1 --2- 172.21.15.63:0/2730950473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 0x55f3b00d5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.592+0000 7fba93e00640 1 -- 172.21.15.63:0/2730950473 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3b00e7040 con 0x55f3afd002f0 2024-05-27T08:05:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.593+0000 7fba93e00640 1 --2- 172.21.15.63:0/2730950473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd002f0 0x55f3b00d1440 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00e810 tx=0x7fba8c00c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3f8a22640df313 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.593+0000 7fba7f400640 1 -- 172.21.15.63:0/2730950473 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c024040 con 0x55f3afd002f0 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.593+0000 7fba7f400640 1 -- 172.21.15.63:0/2730950473 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba8c02c070 con 0x55f3afd002f0 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.593+0000 7fba7f400640 1 -- 172.21.15.63:0/2730950473 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c014040 con 0x55f3afd002f0 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.594+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/2730950473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd002f0 msgr2=0x55f3b00d1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.594+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/2730950473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd002f0 0x55f3b00d1440 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00e810 tx=0x7fba8c00c750 comp rx=0 tx=0).stop 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.594+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/2730950473 shutdown_connections 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.594+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/2730950473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd291b0 0x55f3b00d60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.594+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/2730950473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd084d0 0x55f3b00d5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.594+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/2730950473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd002f0 0x55f3b00d1440 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.594+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/2730950473 >> 172.21.15.63:0/2730950473 conn(0x55f3afd1d300 msgr2=0x55f3b00d54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.594+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/2730950473 shutdown_connections 2024-05-27T08:05:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.594+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/2730950473 wait complete. 2024-05-27T08:05:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.595+0000 7fbaa0f9f740 1 -- start start 2024-05-27T08:05:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.595+0000 7fbaa0f9f740 1 Processor -- start 2024-05-27T08:05:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd002f0 0x55f3b00e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd084d0 0x55f3b00dd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fbaa0f9f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00ddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fba941ad500 con 0x55f3afd291b0 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba941ae970 con 0x55f3afd002f0 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fbaa0f9f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba941adbd0 con 0x55f3afd084d0 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fbaa0f9f740 10 client.0 _ll_register_callbacks cb 0x7fba80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fba93e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00ddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fba93e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00ddeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50718/0 (socket says 172.21.15.63:50718) 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fba93e00640 1 -- 172.21.15.63:0/3159740609 learned_addr learned my addr 172.21.15.63:0/3159740609 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fba92a00640 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd084d0 0x55f3b00dd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fba93e00640 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd002f0 0x55f3b00e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.596+0000 7fba93e00640 1 -- 172.21.15.63:0/3159740609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd084d0 msgr2=0x55f3b00dd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.597+0000 7fba93e00640 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd084d0 0x55f3b00dd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.597+0000 7fba93e00640 1 -- 172.21.15.63:0/3159740609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd002f0 msgr2=0x55f3b00e2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.597+0000 7fba93e00640 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd002f0 0x55f3b00e2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.597+0000 7fba93e00640 1 -- 172.21.15.63:0/3159740609 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3b00de3f0 con 0x55f3afd291b0 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.597+0000 7fba93e00640 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00ddeb0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fba8c000f80 tx=0x7fba8c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab4bc56549f21f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.597+0000 7fba7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.597+0000 7fba7cc00640 1 -- 172.21.15.63:0/3159740609 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c014070 con 0x55f3afd291b0 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.597+0000 7fba7cc00640 1 -- 172.21.15.63:0/3159740609 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fba8c023040 con 0x55f3afd291b0 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.597+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3159740609 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba8c02c070 con 0x55f3afd291b0 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fba7cc00640 1 -- 172.21.15.63:0/3159740609 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fba8c00f830 con 0x55f3afd291b0 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fbaa0f9f740 10 client.21591 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3159740609 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f3b0154410 con 0x55f3afd291b0 2024-05-27T08:05:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fba7cc00640 1 -- 172.21.15.63:0/3159740609 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fba8c02c070 con 0x55f3afd291b0 2024-05-27T08:05:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fba7cc00640 1 -- 172.21.15.63:0/3159740609 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fba8c0273c0 con 0x55f3afd291b0 2024-05-27T08:05:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fbaa0f9f740 10 client.21591 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fbaa0f9f740 10 client.21591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fbaa0f9f740 20 client.21591 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3afcbd1c0 0x55f3b01569a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fbaa0f9f740 4 client.21591 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:05:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.598+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3159740609 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3b00382c0 con 0x55f3afcbd1c0 2024-05-27T08:05:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.599+0000 7fba92a00640 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3afcbd1c0 0x55f3b01569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.600+0000 7fba92a00640 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3afcbd1c0 0x55f3b01569a0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6907282105c579f server_cookie=28e1cedfde2319fc in_seq=0 out_seq=0 2024-05-27T08:05:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.600+0000 7fba7cc00640 10 client.21591 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.629+0000 7fba7cc00640 1 -- 172.21.15.63:0/3159740609 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f3b00382c0 con 0x55f3afcbd1c0 2024-05-27T08:05:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.629+0000 7fba7cc00640 10 client.21591 handle_command_reply: tid=0 2024-05-27T08:05:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.629+0000 7fbaa0f9f740 1 client.21591 shutdown 2024-05-27T08:05:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3159740609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 msgr2=0x55f3b00ddeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00ddeb0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fba8c000f80 tx=0x7fba8c01c040 comp rx=0 tx=0).stop 2024-05-27T08:05:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3159740609 shutdown_connections 2024-05-27T08:05:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f3afcbd1c0 0x55f3b01569a0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3afd291b0 0x55f3b00ddeb0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3afd084d0 0x55f3b00dd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 1 --2- 172.21.15.63:0/3159740609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3afd002f0 0x55f3b00e2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fba7cc00640 0 client.21591 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3159740609 >> 172.21.15.63:0/3159740609 conn(0x55f3afd1d300 msgr2=0x55f3b00d5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3159740609 shutdown_connections 2024-05-27T08:05:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 1 -- 172.21.15.63:0/3159740609 wait complete. 2024-05-27T08:05:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.630+0000 7fbaa0f9f740 20 client.21591 trim_cache size 0 max 16384 2024-05-27T08:05:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:05:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:05:47.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:05:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.631+0000 7fba98a00640 1 -- 172.21.15.63:0/2582285185 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba780b29d0 msgr2=0x7fba780b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.631+0000 7fba98a00640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba780b29d0 0x7fba780b4ee0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55f3b00c9550 tx=0x7fba8400df40 comp rx=0 tx=0).stop 2024-05-27T08:05:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.631+0000 7fba98a00640 1 -- 172.21.15.63:0/2582285185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba94104c10 msgr2=0x7fba9419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.631+0000 7fba98a00640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba94104c10 0x7fba9419a620 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fba8400d950 tx=0x7fba8400de10 comp rx=0 tx=0).stop 2024-05-27T08:05:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.631+0000 7fba98a00640 1 -- 172.21.15.63:0/2582285185 shutdown_connections 2024-05-27T08:05:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.631+0000 7fba98a00640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fba780b29d0 0x7fba780b4ee0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.631+0000 7fba98a00640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba9410e020 0x7fba9419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.631+0000 7fba98a00640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba9410b670 0x7fba9419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.631+0000 7fba98a00640 1 --2- 172.21.15.63:0/2582285185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba94104c10 0x7fba9419a620 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.631+0000 7fba98a00640 1 -- 172.21.15.63:0/2582285185 >> 172.21.15.63:0/2582285185 conn(0x7fba940fe210 msgr2=0x7fba9410e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.632+0000 7fba98a00640 1 -- 172.21.15.63:0/2582285185 shutdown_connections 2024-05-27T08:05:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:05:47.632+0000 7fba98a00640 1 -- 172.21.15.63:0/2582285185 wait complete. 2024-05-27T08:05:47.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:47 smithi086 ceph-mon[34687]: pgmap v1020: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:05:47.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:47 smithi149 ceph-mon[35408]: pgmap v1020: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:05:47.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:47 smithi063 ceph-mon[31613]: pgmap v1020: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:05:49.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:49 smithi086 ceph-mon[34687]: pgmap v1021: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T08:05:49.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:49 smithi149 ceph-mon[35408]: pgmap v1021: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T08:05:49.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:49 smithi063 ceph-mon[31613]: pgmap v1021: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T08:05:50.063 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:05:50.580 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:51.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:51 smithi086 ceph-mon[34687]: pgmap v1022: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T08:05:51.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:51 smithi149 ceph-mon[35408]: pgmap v1022: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T08:05:51.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:51 smithi063 ceph-mon[31613]: pgmap v1022: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.0 MiB/s wr, 6 op/s 2024-05-27T08:05:53.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.528+0000 7f295cc00700 1 -- 172.21.15.63:0/2711800131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29581034b0 msgr2=0x7f29581038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:53.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.528+0000 7f295cc00700 1 --2- 172.21.15.63:0/2711800131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29581034b0 0x7f29581038c0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f29400099c0 tx=0x7f2940009cd0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.528+0000 7f295cc00700 1 -- 172.21.15.63:0/2711800131 shutdown_connections 2024-05-27T08:05:53.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.528+0000 7f295cc00700 1 --2- 172.21.15.63:0/2711800131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2958108730 0x7f2958110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.528+0000 7f295cc00700 1 --2- 172.21.15.63:0/2711800131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29581034b0 0x7f29581038c0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.528+0000 7f295cc00700 1 --2- 172.21.15.63:0/2711800131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2958107de0 0x7f29581081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.528+0000 7f295cc00700 1 -- 172.21.15.63:0/2711800131 >> 172.21.15.63:0/2711800131 conn(0x7f29580fedb0 msgr2=0x7f29581011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:53.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.528+0000 7f295cc00700 1 -- 172.21.15.63:0/2711800131 shutdown_connections 2024-05-27T08:05:53.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.529+0000 7f295cc00700 1 -- 172.21.15.63:0/2711800131 wait complete. 2024-05-27T08:05:53.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.530+0000 7f295cc00700 1 Processor -- start 2024-05-27T08:05:53.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.530+0000 7f295cc00700 1 -- start start 2024-05-27T08:05:53.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f295cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29581034b0 0x7f295810a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f295cc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2958107de0 0x7f295810a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f295cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2958108730 0x7f295810ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f295cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f295806c2b0 con 0x7f2958108730 2024-05-27T08:05:53.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f295cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f295806c130 con 0x7f29581034b0 2024-05-27T08:05:53.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f295cc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f295806c430 con 0x7f2958107de0 2024-05-27T08:05:53.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f2956800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2958108730 0x7f295810ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f2956800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2958108730 0x7f295810ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39798/0 (socket says 172.21.15.63:39798) 2024-05-27T08:05:53.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f2956800700 1 -- 172.21.15.63:0/957718322 learned_addr learned my addr 172.21.15.63:0/957718322 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:53.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f2955e00700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29581034b0 0x7f295810a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.531+0000 7f2955400700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2958107de0 0x7f295810a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.532+0000 7f2956800700 1 -- 172.21.15.63:0/957718322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2958107de0 msgr2=0x7f295810a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:53.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.532+0000 7f2956800700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2958107de0 0x7f295810a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.532+0000 7f2956800700 1 -- 172.21.15.63:0/957718322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29581034b0 msgr2=0x7f295810a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:53.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.532+0000 7f2956800700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29581034b0 0x7f295810a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.532+0000 7f2956800700 1 -- 172.21.15.63:0/957718322 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29581ad590 con 0x7f2958108730 2024-05-27T08:05:53.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.532+0000 7f2955400700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2958107de0 0x7f295810a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:53.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.532+0000 7f2956800700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2958108730 0x7f295810ab30 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f2950007ee0 tx=0x7f295000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:53.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.533+0000 7f294ea00700 1 -- 172.21.15.63:0/957718322 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295000dbd0 con 0x7f2958108730 2024-05-27T08:05:53.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.533+0000 7f294ea00700 1 -- 172.21.15.63:0/957718322 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f295000dd70 con 0x7f2958108730 2024-05-27T08:05:53.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.533+0000 7f294ea00700 1 -- 172.21.15.63:0/957718322 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29500058d0 con 0x7f2958108730 2024-05-27T08:05:53.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.533+0000 7f295cc00700 1 -- 172.21.15.63:0/957718322 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29581ad820 con 0x7f2958108730 2024-05-27T08:05:53.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.533+0000 7f295cc00700 1 -- 172.21.15.63:0/957718322 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29581add60 con 0x7f2958108730 2024-05-27T08:05:53.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.538+0000 7f294ea00700 1 -- 172.21.15.63:0/957718322 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f295001a020 con 0x7f2958108730 2024-05-27T08:05:53.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.538+0000 7f296b69db80 1 Processor -- start 2024-05-27T08:05:53.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.539+0000 7f296b69db80 1 -- start start 2024-05-27T08:05:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.539+0000 7f296b69db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc78f49e40 0x55dc790ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.539+0000 7f296b69db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790ced30 0x55dc790c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.539+0000 7f296b69db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.539+0000 7f296b69db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dc78ff6e40 con 0x55dc790ced30 2024-05-27T08:05:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.539+0000 7f296b69db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc78fe69e0 con 0x55dc78f49e40 2024-05-27T08:05:53.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.539+0000 7f296b69db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dc78feaa20 con 0x55dc790c8db0 2024-05-27T08:05:53.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790ced30 0x55dc790c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790ced30 0x55dc790c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39806/0 (socket says 172.21.15.63:39806) 2024-05-27T08:05:53.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955e00700 1 -- 172.21.15.63:0/1951187364 learned_addr learned my addr 172.21.15.63:0/1951187364 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955400700 1 --2- 172.21.15.63:0/1951187364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc78f49e40 0x55dc790ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955400700 1 --2- 172.21.15.63:0/1951187364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955e00700 1 -- 172.21.15.63:0/1951187364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 msgr2=0x55dc790c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955e00700 1 --2- 172.21.15.63:0/1951187364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955e00700 1 -- 172.21.15.63:0/1951187364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc78f49e40 msgr2=0x55dc790ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955e00700 1 --2- 172.21.15.63:0/1951187364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc78f49e40 0x55dc790ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955e00700 1 -- 172.21.15.63:0/1951187364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc78faf420 con 0x55dc790ced30 2024-05-27T08:05:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f2955400700 1 --2- 172.21.15.63:0/1951187364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc78f49e40 0x55dc790ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:53.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.540+0000 7f294ea00700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f293c0b2a70 0x7f293c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.541+0000 7f294ea00700 1 -- 172.21.15.63:0/957718322 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f29500e2220 con 0x7f2958108730 2024-05-27T08:05:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.541+0000 7f2955400700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f293c0b2a70 0x7f293c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.541+0000 7f2955e00700 1 --2- 172.21.15.63:0/1951187364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790ced30 0x55dc790c8870 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f29400099c0 tx=0x7f2940009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7662dad9ce9292b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.542+0000 7f2955400700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f293c0b2a70 0x7f293c0b4f80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55dc790c9b30 tx=0x7f294800b510 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.542+0000 7f2947400700 1 -- 172.21.15.63:0/1951187364 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940026070 con 0x55dc790ced30 2024-05-27T08:05:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.542+0000 7f2947400700 1 -- 172.21.15.63:0/1951187364 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2940005600 con 0x55dc790ced30 2024-05-27T08:05:53.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.542+0000 7f2947400700 1 -- 172.21.15.63:0/1951187364 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400038e0 con 0x55dc790ced30 2024-05-27T08:05:53.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.543+0000 7f296b69db80 1 -- 172.21.15.63:0/1951187364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790ced30 msgr2=0x55dc790c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:53.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.543+0000 7f296b69db80 1 --2- 172.21.15.63:0/1951187364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790ced30 0x55dc790c8870 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f29400099c0 tx=0x7f2940009710 comp rx=0 tx=0).stop 2024-05-27T08:05:53.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.543+0000 7f296b69db80 1 -- 172.21.15.63:0/1951187364 shutdown_connections 2024-05-27T08:05:53.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.543+0000 7f296b69db80 1 --2- 172.21.15.63:0/1951187364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.543+0000 7f296b69db80 1 --2- 172.21.15.63:0/1951187364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc78f49e40 0x55dc790ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.543+0000 7f296b69db80 1 --2- 172.21.15.63:0/1951187364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790ced30 0x55dc790c8870 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.543+0000 7f296b69db80 1 -- 172.21.15.63:0/1951187364 >> 172.21.15.63:0/1951187364 conn(0x55dc78fa9fc0 msgr2=0x55dc790c38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:53.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.543+0000 7f296b69db80 1 -- 172.21.15.63:0/1951187364 shutdown_connections 2024-05-27T08:05:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.543+0000 7f296b69db80 1 -- 172.21.15.63:0/1951187364 wait complete. 2024-05-27T08:05:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.544+0000 7f296b69db80 1 -- start start 2024-05-27T08:05:53.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.544+0000 7f296b69db80 1 Processor -- start 2024-05-27T08:05:53.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.545+0000 7f296b69db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.545+0000 7f296b69db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.545+0000 7f296b69db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc790d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.545+0000 7f296b69db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dc78eecfa0 con 0x55dc78f49e40 2024-05-27T08:05:53.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.545+0000 7f296b69db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc78f87760 con 0x55dc790ced30 2024-05-27T08:05:53.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.545+0000 7f296b69db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dc790d57a0 con 0x55dc790c8db0 2024-05-27T08:05:53.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.545+0000 7f296b69db80 10 client.0 _ll_register_callbacks cb 0x55dc78f90c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:53.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.545+0000 7f2955e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.545+0000 7f2955e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39818/0 (socket says 172.21.15.63:39818) 2024-05-27T08:05:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.545+0000 7f2955e00700 1 -- 172.21.15.63:0/2753169300 learned_addr learned my addr 172.21.15.63:0/2753169300 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.546+0000 7f2955400700 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.546+0000 7f2956800700 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc790d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.546+0000 7f2955e00700 1 -- 172.21.15.63:0/2753169300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 msgr2=0x55dc790d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.546+0000 7f2955e00700 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.546+0000 7f2955e00700 1 -- 172.21.15.63:0/2753169300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 msgr2=0x55dc790d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.546+0000 7f2955e00700 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc790d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.546+0000 7f2955e00700 1 -- 172.21.15.63:0/2753169300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc78f368d0 con 0x55dc78f49e40 2024-05-27T08:05:53.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.547+0000 7f2956800700 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc790d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:53.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.547+0000 7f2955e00700 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790d24e0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f2940005850 tx=0x7f294000b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da63a8661ffe91a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:53.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.547+0000 7f293be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:53.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.547+0000 7f293be00700 1 -- 172.21.15.63:0/2753169300 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940026040 con 0x55dc78f49e40 2024-05-27T08:05:53.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.547+0000 7f293be00700 1 -- 172.21.15.63:0/2753169300 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f294000bd10 con 0x55dc78f49e40 2024-05-27T08:05:53.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.547+0000 7f296b69db80 1 -- 172.21.15.63:0/2753169300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2940005600 con 0x55dc78f49e40 2024-05-27T08:05:53.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.548+0000 7f293be00700 1 -- 172.21.15.63:0/2753169300 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2940038050 con 0x55dc78f49e40 2024-05-27T08:05:53.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.548+0000 7f296b69db80 10 client.21603 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:53.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.548+0000 7f296b69db80 1 -- 172.21.15.63:0/2753169300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dc78fb7060 con 0x55dc78f49e40 2024-05-27T08:05:53.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.548+0000 7f293be00700 1 -- 172.21.15.63:0/2753169300 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2940005600 con 0x55dc78f49e40 2024-05-27T08:05:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.549+0000 7f293be00700 1 -- 172.21.15.63:0/2753169300 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2940029070 con 0x55dc78f49e40 2024-05-27T08:05:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.549+0000 7f296b69db80 10 client.21603 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.549+0000 7f296b69db80 10 client.21603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.549+0000 7f296b69db80 20 client.21603 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.549+0000 7f296b69db80 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc790e2530 0x55dc790e4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.549+0000 7f296b69db80 4 client.21603 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.549+0000 7f296b69db80 1 -- 172.21.15.63:0/2753169300 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc78ff8d90 con 0x55dc790e2530 2024-05-27T08:05:53.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.550+0000 7f2955400700 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc790e2530 0x55dc790e4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.551+0000 7f293be00700 10 client.21603 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:53.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.551+0000 7f2955400700 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc790e2530 0x55dc790e4950 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39744ad49de95ece server_cookie=c0228de6389d0c0e in_seq=0 out_seq=0 2024-05-27T08:05:53.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.553+0000 7f293be00700 1 -- 172.21.15.63:0/2753169300 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dc78ff8d90 con 0x55dc790e2530 2024-05-27T08:05:53.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.553+0000 7f293be00700 10 client.21603 handle_command_reply: tid=0 2024-05-27T08:05:53.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.553+0000 7f296b69db80 1 client.21603 shutdown 2024-05-27T08:05:53.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.553+0000 7f296b69db80 1 -- 172.21.15.63:0/2753169300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 msgr2=0x55dc790d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:53.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.553+0000 7f296b69db80 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790d24e0 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f2940005850 tx=0x7f294000b9c0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.554+0000 7f296b69db80 1 -- 172.21.15.63:0/2753169300 shutdown_connections 2024-05-27T08:05:53.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.554+0000 7f296b69db80 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc790e2530 0x55dc790e4950 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.554+0000 7f296b69db80 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc790d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.554+0000 7f296b69db80 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790d24e0 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.554+0000 7f296b69db80 1 --2- 172.21.15.63:0/2753169300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.554+0000 7f296b69db80 1 -- 172.21.15.63:0/2753169300 >> 172.21.15.63:0/2753169300 conn(0x55dc78fa9fc0 msgr2=0x55dc790c3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:53.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.554+0000 7f293be00700 0 client.21603 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:53.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.554+0000 7f296b69db80 1 -- 172.21.15.63:0/2753169300 shutdown_connections 2024-05-27T08:05:53.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.554+0000 7f296b69db80 1 -- 172.21.15.63:0/2753169300 wait complete. 2024-05-27T08:05:53.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.554+0000 7f296b69db80 20 client.21603 trim_cache size 0 max 16384 2024-05-27T08:05:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.580+0000 7f296b69db80 1 Processor -- start 2024-05-27T08:05:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.580+0000 7f296b69db80 1 -- start start 2024-05-27T08:05:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.580+0000 7f296b69db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.580+0000 7f296b69db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.580+0000 7f296b69db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc790dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.580+0000 7f296b69db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29581b9350 con 0x55dc78f49e40 2024-05-27T08:05:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.580+0000 7f296b69db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29581b6be0 con 0x55dc790ced30 2024-05-27T08:05:53.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.580+0000 7f296b69db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29581b4a50 con 0x55dc790c8db0 2024-05-27T08:05:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2955e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2955e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39832/0 (socket says 172.21.15.63:39832) 2024-05-27T08:05:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2955e00700 1 -- 172.21.15.63:0/870803725 learned_addr learned my addr 172.21.15.63:0/870803725 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2955400700 1 --2- 172.21.15.63:0/870803725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2956800700 1 --2- 172.21.15.63:0/870803725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc790dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2955e00700 1 -- 172.21.15.63:0/870803725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 msgr2=0x55dc790d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2955e00700 1 --2- 172.21.15.63:0/870803725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790d4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2955e00700 1 -- 172.21.15.63:0/870803725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 msgr2=0x55dc790dea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:53.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2955e00700 1 --2- 172.21.15.63:0/870803725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc790dea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2955e00700 1 -- 172.21.15.63:0/870803725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc790d1b90 con 0x55dc78f49e40 2024-05-27T08:05:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.581+0000 7f2955400700 1 --2- 172.21.15.63:0/870803725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.582+0000 7f2955e00700 1 --2- 172.21.15.63:0/870803725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790c8870 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f294000f6b0 tx=0x7f2940000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9d9dc1cb4c4f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.582+0000 7f2946a00700 1 -- 172.21.15.63:0/870803725 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940026070 con 0x55dc78f49e40 2024-05-27T08:05:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.582+0000 7f2946a00700 1 -- 172.21.15.63:0/870803725 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f294002f070 con 0x55dc78f49e40 2024-05-27T08:05:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.582+0000 7f2946a00700 1 -- 172.21.15.63:0/870803725 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400039e0 con 0x55dc78f49e40 2024-05-27T08:05:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.583+0000 7f296b69db80 1 -- 172.21.15.63:0/870803725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 msgr2=0x55dc790c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:53.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.583+0000 7f296b69db80 1 --2- 172.21.15.63:0/870803725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790c8870 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f294000f6b0 tx=0x7f2940000bc0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.583+0000 7f296b69db80 1 -- 172.21.15.63:0/870803725 shutdown_connections 2024-05-27T08:05:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.583+0000 7f296b69db80 1 --2- 172.21.15.63:0/870803725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc790dea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.583+0000 7f296b69db80 1 --2- 172.21.15.63:0/870803725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc78f49e40 0x55dc790c8870 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.583+0000 7f296b69db80 1 --2- 172.21.15.63:0/870803725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc790c8db0 0x55dc790d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.583+0000 7f296b69db80 1 -- 172.21.15.63:0/870803725 >> 172.21.15.63:0/870803725 conn(0x55dc78fa9fc0 msgr2=0x55dc790d22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.584+0000 7f296b69db80 1 -- 172.21.15.63:0/870803725 shutdown_connections 2024-05-27T08:05:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.584+0000 7f296b69db80 1 -- 172.21.15.63:0/870803725 wait complete. 2024-05-27T08:05:53.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.584+0000 7f296b69db80 1 -- start start 2024-05-27T08:05:53.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.585+0000 7f296b69db80 1 Processor -- start 2024-05-27T08:05:53.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.585+0000 7f296b69db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc78f49e40 0x55dc790d3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.585+0000 7f296b69db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790c8db0 0x55dc790d3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.585+0000 7f296b69db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc7914ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.585+0000 7f296b69db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29581b50a0 con 0x55dc790c8db0 2024-05-27T08:05:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f296b69db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29581b6510 con 0x55dc790ced30 2024-05-27T08:05:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f296b69db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29581b5770 con 0x55dc78f49e40 2024-05-27T08:05:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f296b69db80 10 client.0 _ll_register_callbacks cb 0x55dc78f90c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f2955400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790c8db0 0x55dc790d3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f2955400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790c8db0 0x55dc790d3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39846/0 (socket says 172.21.15.63:39846) 2024-05-27T08:05:53.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f2955400700 1 -- 172.21.15.63:0/1473280021 learned_addr learned my addr 172.21.15.63:0/1473280021 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f2955e00700 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc78f49e40 0x55dc790d3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f2956800700 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc7914ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f2955400700 1 -- 172.21.15.63:0/1473280021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc78f49e40 msgr2=0x55dc790d3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f2955400700 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc78f49e40 0x55dc790d3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.586+0000 7f2955400700 1 -- 172.21.15.63:0/1473280021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 msgr2=0x55dc7914ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.587+0000 7f2955400700 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc7914ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.587+0000 7f2955400700 1 -- 172.21.15.63:0/1473280021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc790d4860 con 0x55dc790c8db0 2024-05-27T08:05:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.587+0000 7f2956800700 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc7914ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.587+0000 7f2955400700 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790c8db0 0x55dc790d3f80 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f29581146a0 tx=0x7f294800b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602d37337052b8a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.587+0000 7f293be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:05:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.588+0000 7f293be00700 1 -- 172.21.15.63:0/1473280021 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948021ad0 con 0x55dc790c8db0 2024-05-27T08:05:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.588+0000 7f293be00700 1 -- 172.21.15.63:0/1473280021 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f2948021c70 con 0x55dc790c8db0 2024-05-27T08:05:53.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.588+0000 7f296b69db80 1 -- 172.21.15.63:0/1473280021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f294002f070 con 0x55dc790c8db0 2024-05-27T08:05:53.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.588+0000 7f293be00700 1 -- 172.21.15.63:0/1473280021 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2948039050 con 0x55dc790c8db0 2024-05-27T08:05:53.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.588+0000 7f296b69db80 10 client.21609 fetch_fsmap learned FSMap version 35 2024-05-27T08:05:53.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.588+0000 7f296b69db80 1 -- 172.21.15.63:0/1473280021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dc7914c3d0 con 0x55dc790c8db0 2024-05-27T08:05:53.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.589+0000 7f293be00700 1 -- 172.21.15.63:0/1473280021 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f294002f070 con 0x55dc790c8db0 2024-05-27T08:05:53.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.589+0000 7f293be00700 1 -- 172.21.15.63:0/1473280021 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2948012450 con 0x55dc790c8db0 2024-05-27T08:05:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.589+0000 7f296b69db80 10 client.21609 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:05:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.589+0000 7f296b69db80 10 client.21609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.589+0000 7f296b69db80 20 client.21609 populate_metadata read hostname 'smithi063' 2024-05-27T08:05:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.589+0000 7f296b69db80 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc7914cb40 0x55dc7914ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.589+0000 7f296b69db80 4 client.21609 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:05:53.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.589+0000 7f296b69db80 1 -- 172.21.15.63:0/1473280021 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc78ff2570 con 0x55dc7914cb40 2024-05-27T08:05:53.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.590+0000 7f2955e00700 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc7914cb40 0x55dc7914ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:53.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.591+0000 7f2955e00700 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc7914cb40 0x55dc7914ef60 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3652fa0429ee962 server_cookie=b8cecf79bff63206 in_seq=0 out_seq=0 2024-05-27T08:05:53.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.591+0000 7f293be00700 10 client.21609 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:53.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.591+0000 7f293be00700 1 -- 172.21.15.63:0/1473280021 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc78ff2570 con 0x55dc7914cb40 2024-05-27T08:05:53.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.591+0000 7f293be00700 10 client.21609 handle_command_reply: tid=0 2024-05-27T08:05:53.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.591+0000 7f296b69db80 1 client.21609 shutdown 2024-05-27T08:05:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 1 -- 172.21.15.63:0/1473280021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790c8db0 msgr2=0x55dc790d3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790c8db0 0x55dc790d3f80 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f29581146a0 tx=0x7f294800b2e0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 1 -- 172.21.15.63:0/1473280021 shutdown_connections 2024-05-27T08:05:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc7914cb40 0x55dc7914ef60 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc790ced30 0x55dc7914ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc78f49e40 0x55dc790d3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 1 --2- 172.21.15.63:0/1473280021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc790c8db0 0x55dc790d3f80 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 1 -- 172.21.15.63:0/1473280021 >> 172.21.15.63:0/1473280021 conn(0x55dc78fa9fc0 msgr2=0x55dc790d2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:53.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f293be00700 0 client.21609 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:05:53.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 1 -- 172.21.15.63:0/1473280021 shutdown_connections 2024-05-27T08:05:53.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 1 -- 172.21.15.63:0/1473280021 wait complete. 2024-05-27T08:05:53.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.592+0000 7f296b69db80 20 client.21609 trim_cache size 0 max 16384 2024-05-27T08:05:53.595 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:05:53.596 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:05:53.596 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:05:53.596 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:05:53.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.593+0000 7f295cc00700 1 -- 172.21.15.63:0/957718322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f293c0b2a70 msgr2=0x7f293c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:53.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.593+0000 7f295cc00700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f293c0b2a70 0x7f293c0b4f80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55dc790c9b30 tx=0x7f294800b510 comp rx=0 tx=0).stop 2024-05-27T08:05:53.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.593+0000 7f295cc00700 1 -- 172.21.15.63:0/957718322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2958108730 msgr2=0x7f295810ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:53.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.593+0000 7f295cc00700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2958108730 0x7f295810ab30 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f2950007ee0 tx=0x7f295000d3b0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.593+0000 7f295cc00700 1 -- 172.21.15.63:0/957718322 shutdown_connections 2024-05-27T08:05:53.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.594+0000 7f295cc00700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f293c0b2a70 0x7f293c0b4f80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.594+0000 7f295cc00700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2958108730 0x7f295810ab30 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.594+0000 7f295cc00700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29581034b0 0x7f295810a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.594+0000 7f295cc00700 1 --2- 172.21.15.63:0/957718322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2958107de0 0x7f295810a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:53.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.594+0000 7f295cc00700 1 -- 172.21.15.63:0/957718322 >> 172.21.15.63:0/957718322 conn(0x7f29580fedb0 msgr2=0x7f29581096f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:53.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.594+0000 7f295cc00700 1 -- 172.21.15.63:0/957718322 shutdown_connections 2024-05-27T08:05:53.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:53.594+0000 7f295cc00700 1 -- 172.21.15.63:0/957718322 wait complete. 2024-05-27T08:05:53.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:53 smithi086 ceph-mon[34687]: pgmap v1023: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:05:53.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:53 smithi149 ceph-mon[35408]: pgmap v1023: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:05:53.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:53 smithi063 ceph-mon[31613]: pgmap v1023: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:05:54.926 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:05:54.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:05:55.453 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:55.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:55 smithi086 ceph-mon[34687]: pgmap v1024: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 9 op/s 2024-05-27T08:05:55.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:55 smithi149 ceph-mon[35408]: pgmap v1024: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 9 op/s 2024-05-27T08:05:55.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:55 smithi063 ceph-mon[31613]: pgmap v1024: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.3 MiB/s wr, 9 op/s 2024-05-27T08:05:57.876 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:57 smithi063 ceph-mon[31613]: pgmap v1025: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:05:57.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:57 smithi086 ceph-mon[34687]: pgmap v1025: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:05:57.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:57 smithi149 ceph-mon[35408]: pgmap v1025: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 13 op/s 2024-05-27T08:05:58.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.359+0000 7f5af5a00700 1 -- 172.21.15.63:0/2509053655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5af0108d20 msgr2=0x7f5af0109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.359+0000 7f5af5a00700 1 --2- 172.21.15.63:0/2509053655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5af0108d20 0x7f5af0109130 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0099c0 tx=0x7f5adc009cd0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.359+0000 7f5af5a00700 1 -- 172.21.15.63:0/2509053655 shutdown_connections 2024-05-27T08:05:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.359+0000 7f5af5a00700 1 --2- 172.21.15.63:0/2509053655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af0109670 0x7f5af0110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.359+0000 7f5af5a00700 1 --2- 172.21.15.63:0/2509053655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af01043f0 0x7f5af0104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.359+0000 7f5af5a00700 1 --2- 172.21.15.63:0/2509053655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5af0108d20 0x7f5af0109130 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.359+0000 7f5af5a00700 1 -- 172.21.15.63:0/2509053655 >> 172.21.15.63:0/2509053655 conn(0x7f5af0079d00 msgr2=0x7f5af007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.359+0000 7f5af5a00700 1 -- 172.21.15.63:0/2509053655 shutdown_connections 2024-05-27T08:05:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.359+0000 7f5af5a00700 1 -- 172.21.15.63:0/2509053655 wait complete. 2024-05-27T08:05:58.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.361+0000 7f5af5a00700 1 Processor -- start 2024-05-27T08:05:58.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.361+0000 7f5af5a00700 1 -- start start 2024-05-27T08:05:58.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.361+0000 7f5af5a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af01043f0 0x7f5af0075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.361+0000 7f5af5a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0108d20 0x7f5af0075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.361+0000 7f5af5a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5af0109670 0x7f5af0076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.361+0000 7f5af5a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0114180 con 0x7f5af0109670 2024-05-27T08:05:58.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.361+0000 7f5af5a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0114000 con 0x7f5af01043f0 2024-05-27T08:05:58.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.361+0000 7f5af5a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0114300 con 0x7f5af0108d20 2024-05-27T08:05:58.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5af5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5af0109670 0x7f5af0076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5af5000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5af0109670 0x7f5af0076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39874/0 (socket says 172.21.15.63:39874) 2024-05-27T08:05:58.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5af5000700 1 -- 172.21.15.63:0/1889120112 learned_addr learned my addr 172.21.15.63:0/1889120112 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:05:58.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5aefe00700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af01043f0 0x7f5af0075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5aef400700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0108d20 0x7f5af0075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5af5000700 1 -- 172.21.15.63:0/1889120112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0108d20 msgr2=0x7f5af0075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5af5000700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0108d20 0x7f5af0075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5af5000700 1 -- 172.21.15.63:0/1889120112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af01043f0 msgr2=0x7f5af0075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5af5000700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af01043f0 0x7f5af0075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5af5000700 1 -- 172.21.15.63:0/1889120112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af01a8f40 con 0x7f5af0109670 2024-05-27T08:05:58.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.362+0000 7f5aef400700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0108d20 0x7f5af0075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:58.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.363+0000 7f5af5000700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5af0109670 0x7f5af0076530 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8007d70 tx=0x7f5ae800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:58.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.363+0000 7f5aecc00700 1 -- 172.21.15.63:0/1889120112 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8016020 con 0x7f5af0109670 2024-05-27T08:05:58.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.363+0000 7f5aecc00700 1 -- 172.21.15.63:0/1889120112 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ae800ad40 con 0x7f5af0109670 2024-05-27T08:05:58.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.363+0000 7f5af5a00700 1 -- 172.21.15.63:0/1889120112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af01a9230 con 0x7f5af0109670 2024-05-27T08:05:58.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.363+0000 7f5aecc00700 1 -- 172.21.15.63:0/1889120112 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80115e0 con 0x7f5af0109670 2024-05-27T08:05:58.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.364+0000 7f5af5a00700 1 -- 172.21.15.63:0/1889120112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af01a9720 con 0x7f5af0109670 2024-05-27T08:05:58.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.365+0000 7f5af5a00700 1 -- 172.21.15.63:0/1889120112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af00512b0 con 0x7f5af0109670 2024-05-27T08:05:58.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.368+0000 7f5aecc00700 1 -- 172.21.15.63:0/1889120112 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5ae8019020 con 0x7f5af0109670 2024-05-27T08:05:58.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.370+0000 7f5aecc00700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ad40b29f0 0x7f5ad40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.370+0000 7f5aecc00700 1 -- 172.21.15.63:0/1889120112 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f5ae80e2850 con 0x7f5af0109670 2024-05-27T08:05:58.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.370+0000 7f5aefe00700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ad40b29f0 0x7f5ad40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.371+0000 7f5aefe00700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ad40b29f0 0x7f5ad40b4f00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0005950 tx=0x7f5ae000b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:58.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.376+0000 7f5aecc00700 1 -- 172.21.15.63:0/1889120112 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ae8020020 con 0x7f5af0109670 2024-05-27T08:05:58.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.727+0000 7f5af5a00700 1 -- 172.21.15.63:0/1889120112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5af002e5e0 con 0x7f5af0109670 2024-05-27T08:05:58.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.730+0000 7f5aecc00700 1 -- 172.21.15.63:0/1889120112 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7f5ae80a9f20 con 0x7f5af0109670 2024-05-27T08:05:58.732 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:05:58.732 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:05:58.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.733+0000 7f5af5a00700 1 -- 172.21.15.63:0/1889120112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ad40b29f0 msgr2=0x7f5ad40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.733+0000 7f5af5a00700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ad40b29f0 0x7f5ad40b4f00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0005950 tx=0x7f5ae000b410 comp rx=0 tx=0).stop 2024-05-27T08:05:58.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.733+0000 7f5af5a00700 1 -- 172.21.15.63:0/1889120112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5af0109670 msgr2=0x7f5af0076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.733+0000 7f5af5a00700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5af0109670 0x7f5af0076530 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8007d70 tx=0x7f5ae800a340 comp rx=0 tx=0).stop 2024-05-27T08:05:58.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.733+0000 7f5af5a00700 1 -- 172.21.15.63:0/1889120112 shutdown_connections 2024-05-27T08:05:58.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.733+0000 7f5af5a00700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ad40b29f0 0x7f5ad40b4f00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.733+0000 7f5af5a00700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5af0109670 0x7f5af0076530 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.733+0000 7f5af5a00700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5af01043f0 0x7f5af0075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.733+0000 7f5af5a00700 1 --2- 172.21.15.63:0/1889120112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5af0108d20 0x7f5af0075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.733+0000 7f5af5a00700 1 -- 172.21.15.63:0/1889120112 >> 172.21.15.63:0/1889120112 conn(0x7f5af0079d00 msgr2=0x7f5af0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:58.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.734+0000 7f5af5a00700 1 -- 172.21.15.63:0/1889120112 shutdown_connections 2024-05-27T08:05:58.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:05:58.734+0000 7f5af5a00700 1 -- 172.21.15.63:0/1889120112 wait complete. 2024-05-27T08:05:58.737 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:05:59.577 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:05:59.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:59 smithi086 ceph-mon[34687]: pgmap v1026: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:05:59.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:05:59 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1889120112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:05:59.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:59 smithi149 ceph-mon[35408]: pgmap v1026: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:05:59.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:05:59 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1889120112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:05:59.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:59 smithi063 ceph-mon[31613]: pgmap v1026: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:05:59.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:05:59 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1889120112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.887+0000 7fa43ac00640 1 -- 172.21.15.63:0/3719725633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa43410a100 msgr2=0x7fa434103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.887+0000 7fa43ac00640 1 --2- 172.21.15.63:0/3719725633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa43410a100 0x7fa434103d40 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009900 tx=0x7fa41c02a410 comp rx=0 tx=0).stop 2024-05-27T08:06:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.887+0000 7fa43ac00640 1 -- 172.21.15.63:0/3719725633 shutdown_connections 2024-05-27T08:06:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.887+0000 7fa43ac00640 1 --2- 172.21.15.63:0/3719725633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4341043b0 0x7fa434110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.887+0000 7fa43ac00640 1 --2- 172.21.15.63:0/3719725633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa43410a100 0x7fa434103d40 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.887+0000 7fa43ac00640 1 --2- 172.21.15.63:0/3719725633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa434109c60 0x7fa434103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.887+0000 7fa43ac00640 1 -- 172.21.15.63:0/3719725633 >> 172.21.15.63:0/3719725633 conn(0x7fa4340ff320 msgr2=0x7fa434101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.888+0000 7fa43ac00640 1 -- 172.21.15.63:0/3719725633 shutdown_connections 2024-05-27T08:06:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.888+0000 7fa43ac00640 1 -- 172.21.15.63:0/3719725633 wait complete. 2024-05-27T08:06:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.889+0000 7fa43ac00640 1 Processor -- start 2024-05-27T08:06:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.890+0000 7fa43ac00640 1 -- start start 2024-05-27T08:06:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.890+0000 7fa43ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4341043b0 0x7fa4341993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.890+0000 7fa43ac00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa434109c60 0x7fa4341998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.890+0000 7fa43ac00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa43410a100 0x7fa4341a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.890+0000 7fa43ac00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa434113d00 con 0x7fa434109c60 2024-05-27T08:06:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.890+0000 7fa43ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa434113b80 con 0x7fa4341043b0 2024-05-27T08:06:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.890+0000 7fa43ac00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa434113e80 con 0x7fa43410a100 2024-05-27T08:06:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.890+0000 7fa42b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa434109c60 0x7fa4341998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.890+0000 7fa42b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa434109c60 0x7fa4341998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36922/0 (socket says 172.21.15.63:36922) 2024-05-27T08:06:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.890+0000 7fa42b400640 1 -- 172.21.15.63:0/714736374 learned_addr learned my addr 172.21.15.63:0/714736374 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.891+0000 7fa433e00640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4341043b0 0x7fa4341993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.891+0000 7fa42b400640 1 -- 172.21.15.63:0/714736374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa43410a100 msgr2=0x7fa4341a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.891+0000 7fa42b400640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa43410a100 0x7fa4341a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.891+0000 7fa42b400640 1 -- 172.21.15.63:0/714736374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4341043b0 msgr2=0x7fa4341993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.891+0000 7fa42b400640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4341043b0 0x7fa4341993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.891+0000 7fa42b400640 1 -- 172.21.15.63:0/714736374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4341a4790 con 0x7fa434109c60 2024-05-27T08:06:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.892+0000 7fa42b400640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa434109c60 0x7fa4341998f0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fa41c004770 tx=0x7fa41c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.892+0000 7fa431600640 1 -- 172.21.15.63:0/714736374 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c03f030 con 0x7fa434109c60 2024-05-27T08:06:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.892+0000 7fa43ac00640 1 -- 172.21.15.63:0/714736374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4341a4a20 con 0x7fa434109c60 2024-05-27T08:06:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.892+0000 7fa431600640 1 -- 172.21.15.63:0/714736374 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa41c004440 con 0x7fa434109c60 2024-05-27T08:06:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.892+0000 7fa431600640 1 -- 172.21.15.63:0/714736374 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c035070 con 0x7fa434109c60 2024-05-27T08:06:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.892+0000 7fa43ac00640 1 -- 172.21.15.63:0/714736374 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4341a4f10 con 0x7fa434109c60 2024-05-27T08:06:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.896+0000 7fa441bf4740 1 Processor -- start 2024-05-27T08:06:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.896+0000 7fa441bf4740 1 -- start start 2024-05-27T08:06:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.897+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f31121b0 0x5558f34b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.897+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 0x5558f34b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.897+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30f14d0 0x5558f34b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.897+0000 7fa441bf4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5558f3425b90 con 0x5558f30f14d0 2024-05-27T08:06:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.897+0000 7fa441bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5558f3415730 con 0x5558f30e92f0 2024-05-27T08:06:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.897+0000 7fa441bf4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5558f3419770 con 0x5558f31121b0 2024-05-27T08:06:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.898+0000 7fa431600640 1 -- 172.21.15.63:0/714736374 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa41c042030 con 0x7fa434109c60 2024-05-27T08:06:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.898+0000 7fa433e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 0x5558f34b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.898+0000 7fa438c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f31121b0 0x5558f34b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.898+0000 7fa433e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 0x5558f34b4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:53550/0 (socket says 172.21.15.63:53550) 2024-05-27T08:06:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.898+0000 7fa42b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30f14d0 0x5558f34b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.898+0000 7fa433e00640 1 -- 172.21.15.63:0/3620208159 learned_addr learned my addr 172.21.15.63:0/3620208159 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.899+0000 7fa42b400640 1 -- 172.21.15.63:0/3620208159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f31121b0 msgr2=0x5558f34b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.899+0000 7fa42b400640 1 --2- 172.21.15.63:0/3620208159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f31121b0 0x5558f34b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.899+0000 7fa42b400640 1 -- 172.21.15.63:0/3620208159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 msgr2=0x5558f34b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.899+0000 7fa42b400640 1 --2- 172.21.15.63:0/3620208159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 0x5558f34b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.899+0000 7fa42b400640 1 -- 172.21.15.63:0/3620208159 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558f33ae8b0 con 0x5558f30f14d0 2024-05-27T08:06:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.899+0000 7fa42b400640 1 --2- 172.21.15.63:0/3620208159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30f14d0 0x5558f34b52e0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fa41c100180 tx=0x7fa41c10b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2509e602c7f824 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.900+0000 7fa431600640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4180b29d0 0x7fa4180b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.900+0000 7fa429200640 1 -- 172.21.15.63:0/3620208159 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c0c6360 con 0x5558f30f14d0 2024-05-27T08:06:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.900+0000 7fa429200640 1 -- 172.21.15.63:0/3620208159 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa41c110070 con 0x5558f30f14d0 2024-05-27T08:06:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.900+0000 7fa429200640 1 -- 172.21.15.63:0/3620208159 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c0c9660 con 0x5558f30f14d0 2024-05-27T08:06:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.900+0000 7fa438c00640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4180b29d0 0x7fa4180b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.900+0000 7fa431600640 1 -- 172.21.15.63:0/714736374 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fa41c102e50 con 0x7fa434109c60 2024-05-27T08:06:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.901+0000 7fa441bf4740 1 -- 172.21.15.63:0/3620208159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30f14d0 msgr2=0x5558f34b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.901+0000 7fa441bf4740 1 --2- 172.21.15.63:0/3620208159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30f14d0 0x5558f34b52e0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fa41c100180 tx=0x7fa41c10b0e0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.902+0000 7fa438c00640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4180b29d0 0x7fa4180b4ee0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x5558f34b19d0 tx=0x7fa42c009620 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.902+0000 7fa441bf4740 1 -- 172.21.15.63:0/3620208159 shutdown_connections 2024-05-27T08:06:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.902+0000 7fa441bf4740 1 --2- 172.21.15.63:0/3620208159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30f14d0 0x5558f34b52e0 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.902+0000 7fa441bf4740 1 --2- 172.21.15.63:0/3620208159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 0x5558f34b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.902+0000 7fa441bf4740 1 --2- 172.21.15.63:0/3620208159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f31121b0 0x5558f34b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.902+0000 7fa441bf4740 1 -- 172.21.15.63:0/3620208159 >> 172.21.15.63:0/3620208159 conn(0x5558f3106300 msgr2=0x5558f34b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.902+0000 7fa441bf4740 1 -- 172.21.15.63:0/3620208159 shutdown_connections 2024-05-27T08:06:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.903+0000 7fa441bf4740 1 -- 172.21.15.63:0/3620208159 wait complete. 2024-05-27T08:06:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.903+0000 7fa441bf4740 1 -- start start 2024-05-27T08:06:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.904+0000 7fa441bf4740 1 Processor -- start 2024-05-27T08:06:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 0x5558f34becb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f34bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 0x5558f34caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa441bf4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5558f30fb790 con 0x5558f30e92f0 2024-05-27T08:06:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa441bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5558f33bdbc0 con 0x5558f31121b0 2024-05-27T08:06:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa441bf4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5558f34c1750 con 0x5558f30f14d0 2024-05-27T08:06:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa441bf4740 10 client.0 _ll_register_callbacks cb 0x5558f339b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa42b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f34bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa433e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 0x5558f34becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa42b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f34bf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:52010/0 (socket says 172.21.15.63:52010) 2024-05-27T08:06:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.905+0000 7fa42b400640 1 -- 172.21.15.63:0/2373067841 learned_addr learned my addr 172.21.15.63:0/2373067841 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.906+0000 7fa42b400640 1 -- 172.21.15.63:0/2373067841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 msgr2=0x5558f34caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.906+0000 7fa42b400640 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 0x5558f34caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.906+0000 7fa42b400640 1 -- 172.21.15.63:0/2373067841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 msgr2=0x5558f34becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.906+0000 7fa42b400640 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 0x5558f34becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.906+0000 7fa42b400640 1 -- 172.21.15.63:0/2373067841 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558f312a210 con 0x5558f30f14d0 2024-05-27T08:06:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.907+0000 7fa42b400640 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f34bf1f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa41c006010 tx=0x7fa41c003bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e575950eff4780d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.907+0000 7fa421600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:06:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.907+0000 7fa421600640 1 -- 172.21.15.63:0/2373067841 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c049070 con 0x5558f30f14d0 2024-05-27T08:06:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.907+0000 7fa441bf4740 1 -- 172.21.15.63:0/2373067841 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa41c110070 con 0x5558f30f14d0 2024-05-27T08:06:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.907+0000 7fa421600640 1 -- 172.21.15.63:0/2373067841 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fa41c003c80 con 0x5558f30f14d0 2024-05-27T08:06:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.908+0000 7fa421600640 1 -- 172.21.15.63:0/2373067841 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa41c0c9850 con 0x5558f30f14d0 2024-05-27T08:06:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.908+0000 7fa441bf4740 10 client.30511 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.908+0000 7fa441bf4740 1 -- 172.21.15.63:0/2373067841 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5558f31f6620 con 0x5558f30f14d0 2024-05-27T08:06:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.908+0000 7fa421600640 1 -- 172.21.15.63:0/2373067841 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa41c110070 con 0x5558f30f14d0 2024-05-27T08:06:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.909+0000 7fa421600640 1 -- 172.21.15.63:0/2373067841 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa41c100400 con 0x5558f30f14d0 2024-05-27T08:06:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.909+0000 7fa441bf4740 10 client.30511 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.909+0000 7fa441bf4740 10 client.30511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.909+0000 7fa441bf4740 20 client.30511 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.909+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558f30d0d80 0x5558f34e1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.909+0000 7fa441bf4740 4 client.30511 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.909+0000 7fa441bf4740 1 -- 172.21.15.63:0/2373067841 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558f3427ae0 con 0x5558f30d0d80 2024-05-27T08:06:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.910+0000 7fa433e00640 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558f30d0d80 0x5558f34e1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.911+0000 7fa433e00640 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558f30d0d80 0x5558f34e1890 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75fb3e79ab9c0a2 server_cookie=396cd6a7cb4ec2c4 in_seq=0 out_seq=0 2024-05-27T08:06:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.911+0000 7fa421600640 10 client.30511 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa421600640 1 -- 172.21.15.63:0/2373067841 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5558f3427ae0 con 0x5558f30d0d80 2024-05-27T08:06:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa421600640 10 client.30511 handle_command_reply: tid=0 2024-05-27T08:06:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa441bf4740 1 client.30511 shutdown 2024-05-27T08:06:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa441bf4740 1 -- 172.21.15.63:0/2373067841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 msgr2=0x5558f34bf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f34bf1f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa41c006010 tx=0x7fa41c003bf0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa441bf4740 1 -- 172.21.15.63:0/2373067841 shutdown_connections 2024-05-27T08:06:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558f30d0d80 0x5558f34e1890 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 0x5558f34caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f34bf1f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2373067841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 0x5558f34becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa441bf4740 1 -- 172.21.15.63:0/2373067841 >> 172.21.15.63:0/2373067841 conn(0x5558f3106300 msgr2=0x5558f34b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.913+0000 7fa421600640 0 client.30511 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.914+0000 7fa441bf4740 1 -- 172.21.15.63:0/2373067841 shutdown_connections 2024-05-27T08:06:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.914+0000 7fa441bf4740 1 -- 172.21.15.63:0/2373067841 wait complete. 2024-05-27T08:06:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.914+0000 7fa441bf4740 20 client.30511 trim_cache size 0 max 16384 2024-05-27T08:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.940+0000 7fa441bf4740 1 Processor -- start 2024-05-27T08:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.940+0000 7fa441bf4740 1 -- start start 2024-05-27T08:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 0x5558f34ba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f34b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f31121b0 0x5558f34be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa441bf4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4341ba120 con 0x5558f31121b0 2024-05-27T08:06:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa441bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4341b79f0 con 0x5558f30e92f0 2024-05-27T08:06:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa441bf4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa434073940 con 0x5558f30f14d0 2024-05-27T08:06:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa438c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f31121b0 0x5558f34be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa433e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 0x5558f34ba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa438c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f31121b0 0x5558f34be380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36950/0 (socket says 172.21.15.63:36950) 2024-05-27T08:06:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa438c00640 1 -- 172.21.15.63:0/2792243336 learned_addr learned my addr 172.21.15.63:0/2792243336 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.941+0000 7fa42b400640 1 --2- 172.21.15.63:0/2792243336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f34b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.942+0000 7fa438c00640 1 -- 172.21.15.63:0/2792243336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 msgr2=0x5558f34b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.942+0000 7fa438c00640 1 --2- 172.21.15.63:0/2792243336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f34b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.942+0000 7fa438c00640 1 -- 172.21.15.63:0/2792243336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 msgr2=0x5558f34ba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.942+0000 7fa438c00640 1 --2- 172.21.15.63:0/2792243336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 0x5558f34ba230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.942+0000 7fa438c00640 1 -- 172.21.15.63:0/2792243336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558f34ab050 con 0x5558f31121b0 2024-05-27T08:06:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.943+0000 7fa438c00640 1 --2- 172.21.15.63:0/2792243336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f31121b0 0x5558f34be380 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009e00 tx=0x7fa42c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2945c2e620e6b61e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.943+0000 7fa423e00640 1 -- 172.21.15.63:0/2792243336 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c003cf0 con 0x5558f31121b0 2024-05-27T08:06:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.944+0000 7fa423e00640 1 -- 172.21.15.63:0/2792243336 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa42c009e30 con 0x5558f31121b0 2024-05-27T08:06:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.944+0000 7fa423e00640 1 -- 172.21.15.63:0/2792243336 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c01e650 con 0x5558f31121b0 2024-05-27T08:06:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.945+0000 7fa441bf4740 1 -- 172.21.15.63:0/2792243336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f31121b0 msgr2=0x5558f34be380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.945+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2792243336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f31121b0 0x5558f34be380 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009e00 tx=0x7fa42c013040 comp rx=0 tx=0).stop 2024-05-27T08:06:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.945+0000 7fa441bf4740 1 -- 172.21.15.63:0/2792243336 shutdown_connections 2024-05-27T08:06:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.945+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2792243336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f31121b0 0x5558f34be380 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.945+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2792243336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f34b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.945+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2792243336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f30e92f0 0x5558f34ba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.945+0000 7fa441bf4740 1 -- 172.21.15.63:0/2792243336 >> 172.21.15.63:0/2792243336 conn(0x5558f3106300 msgr2=0x5558f34c0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.945+0000 7fa441bf4740 1 -- 172.21.15.63:0/2792243336 shutdown_connections 2024-05-27T08:06:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.945+0000 7fa441bf4740 1 -- 172.21.15.63:0/2792243336 wait complete. 2024-05-27T08:06:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.946+0000 7fa441bf4740 1 -- start start 2024-05-27T08:06:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.946+0000 7fa441bf4740 1 Processor -- start 2024-05-27T08:06:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.947+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 0x5558f3542210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.947+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f3542750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.947+0000 7fa441bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 0x5558f34c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.947+0000 7fa441bf4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa434073f90 con 0x5558f30e92f0 2024-05-27T08:06:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.947+0000 7fa441bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4341b7320 con 0x5558f31121b0 2024-05-27T08:06:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.947+0000 7fa441bf4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa434074660 con 0x5558f30f14d0 2024-05-27T08:06:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.947+0000 7fa441bf4740 10 client.0 _ll_register_callbacks cb 0x7fa424009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.948+0000 7fa42b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f3542750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.948+0000 7fa433e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 0x5558f3542210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.948+0000 7fa438c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 0x5558f34c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.948+0000 7fa438c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 0x5558f34c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:53568/0 (socket says 172.21.15.63:53568) 2024-05-27T08:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.948+0000 7fa433e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 0x5558f3542210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36966/0 (socket says 172.21.15.63:36966) 2024-05-27T08:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.948+0000 7fa438c00640 1 -- 172.21.15.63:0/2202217002 learned_addr learned my addr 172.21.15.63:0/2202217002 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.949+0000 7fa433e00640 1 -- 172.21.15.63:0/2202217002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 msgr2=0x5558f3542750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.949+0000 7fa433e00640 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f3542750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.949+0000 7fa433e00640 1 -- 172.21.15.63:0/2202217002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 msgr2=0x5558f34c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.949+0000 7fa433e00640 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 0x5558f34c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.949+0000 7fa433e00640 1 -- 172.21.15.63:0/2202217002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558f34c2560 con 0x5558f30e92f0 2024-05-27T08:06:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.949+0000 7fa438c00640 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 0x5558f34c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:06:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.950+0000 7fa433e00640 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 0x5558f3542210 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x5558f34e1ee0 tx=0x7fa424013a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546881d960a9328d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.950+0000 7fa421600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.950+0000 7fa421600640 1 -- 172.21.15.63:0/2202217002 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424003b70 con 0x5558f30e92f0 2024-05-27T08:06:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.950+0000 7fa421600640 1 -- 172.21.15.63:0/2202217002 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7fa424003d10 con 0x5558f30e92f0 2024-05-27T08:06:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.950+0000 7fa441bf4740 1 -- 172.21.15.63:0/2202217002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa42c009e30 con 0x5558f30e92f0 2024-05-27T08:06:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.951+0000 7fa421600640 1 -- 172.21.15.63:0/2202217002 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa42401d4a0 con 0x5558f30e92f0 2024-05-27T08:06:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.951+0000 7fa441bf4740 10 client.21633 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.951+0000 7fa441bf4740 1 -- 172.21.15.63:0/2202217002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5558f35498e0 con 0x5558f30e92f0 2024-05-27T08:06:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.951+0000 7fa421600640 1 -- 172.21.15.63:0/2202217002 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa42c009e30 con 0x5558f30e92f0 2024-05-27T08:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.952+0000 7fa421600640 1 -- 172.21.15.63:0/2202217002 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa4240044d0 con 0x5558f30e92f0 2024-05-27T08:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.952+0000 7fa441bf4740 10 client.21633 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.952+0000 7fa441bf4740 10 client.21633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.952+0000 7fa441bf4740 20 client.21633 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.952+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558f30a61c0 0x5558f354bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.952+0000 7fa441bf4740 4 client.21633 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.952+0000 7fa441bf4740 1 -- 172.21.15.63:0/2202217002 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558f34212c0 con 0x5558f30a61c0 2024-05-27T08:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.953+0000 7fa42b400640 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558f30a61c0 0x5558f354bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.954+0000 7fa42b400640 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558f30a61c0 0x5558f354bfc0 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57939f3ddb61867 server_cookie=bae896630c3371e in_seq=0 out_seq=0 2024-05-27T08:06:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.954+0000 7fa421600640 10 client.21633 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa421600640 1 -- 172.21.15.63:0/2202217002 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5558f34212c0 con 0x5558f30a61c0 2024-05-27T08:06:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa421600640 10 client.21633 handle_command_reply: tid=0 2024-05-27T08:06:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa441bf4740 1 client.21633 shutdown 2024-05-27T08:06:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa441bf4740 1 -- 172.21.15.63:0/2202217002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 msgr2=0x5558f3542210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 0x5558f3542210 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x5558f34e1ee0 tx=0x7fa424013a30 comp rx=0 tx=0).stop 2024-05-27T08:06:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa441bf4740 1 -- 172.21.15.63:0/2202217002 shutdown_connections 2024-05-27T08:06:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558f30a61c0 0x5558f354bfc0 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558f31121b0 0x5558f34c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558f30f14d0 0x5558f3542750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa441bf4740 1 --2- 172.21.15.63:0/2202217002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558f30e92f0 0x5558f3542210 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.986+0000 7fa441bf4740 1 -- 172.21.15.63:0/2202217002 >> 172.21.15.63:0/2202217002 conn(0x5558f3106300 msgr2=0x5558f34c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.987+0000 7fa421600640 0 client.21633 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.987+0000 7fa441bf4740 1 -- 172.21.15.63:0/2202217002 shutdown_connections 2024-05-27T08:06:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.987+0000 7fa441bf4740 1 -- 172.21.15.63:0/2202217002 wait complete. 2024-05-27T08:06:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.987+0000 7fa441bf4740 20 client.21633 trim_cache size 0 max 16384 2024-05-27T08:06:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:06:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:06:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:06:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.988+0000 7fa43ac00640 1 -- 172.21.15.63:0/714736374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4180b29d0 msgr2=0x7fa4180b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.988+0000 7fa43ac00640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4180b29d0 0x7fa4180b4ee0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x5558f34b19d0 tx=0x7fa42c009620 comp rx=0 tx=0).stop 2024-05-27T08:06:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.988+0000 7fa43ac00640 1 -- 172.21.15.63:0/714736374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa434109c60 msgr2=0x7fa4341998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.988+0000 7fa43ac00640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa434109c60 0x7fa4341998f0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fa41c004770 tx=0x7fa41c004040 comp rx=0 tx=0).stop 2024-05-27T08:06:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.988+0000 7fa43ac00640 1 -- 172.21.15.63:0/714736374 shutdown_connections 2024-05-27T08:06:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.988+0000 7fa43ac00640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4180b29d0 0x7fa4180b4ee0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.988+0000 7fa43ac00640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa43410a100 0x7fa4341a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.988+0000 7fa43ac00640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa434109c60 0x7fa4341998f0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.988+0000 7fa43ac00640 1 --2- 172.21.15.63:0/714736374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4341043b0 0x7fa4341993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.988+0000 7fa43ac00640 1 -- 172.21.15.63:0/714736374 >> 172.21.15.63:0/714736374 conn(0x7fa4340ff320 msgr2=0x7fa4340769a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.989+0000 7fa43ac00640 1 -- 172.21.15.63:0/714736374 shutdown_connections 2024-05-27T08:06:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:00.989+0000 7fa43ac00640 1 -- 172.21.15.63:0/714736374 wait complete. 2024-05-27T08:06:01.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:01 smithi086 ceph-mon[34687]: pgmap v1027: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:06:01.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:01 smithi149 ceph-mon[35408]: pgmap v1027: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:06:01.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:01 smithi063 ceph-mon[31613]: pgmap v1027: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:06:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.542+0000 7f90a4c00700 1 -- 172.21.15.63:0/254127546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90a01045e0 msgr2=0x7f90a01049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.542+0000 7f90a4c00700 1 --2- 172.21.15.63:0/254127546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90a01045e0 0x7f90a01049f0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f90900099c0 tx=0x7f9090009cd0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.542+0000 7f90a4c00700 1 -- 172.21.15.63:0/254127546 shutdown_connections 2024-05-27T08:06:03.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.542+0000 7f90a4c00700 1 --2- 172.21.15.63:0/254127546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a0109860 0x7f90a01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.542+0000 7f90a4c00700 1 --2- 172.21.15.63:0/254127546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90a01045e0 0x7f90a01049f0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.542+0000 7f90a4c00700 1 --2- 172.21.15.63:0/254127546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a0108f10 0x7f90a0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.542+0000 7f90a4c00700 1 -- 172.21.15.63:0/254127546 >> 172.21.15.63:0/254127546 conn(0x7f90a0100030 msgr2=0x7f90a0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:03.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.542+0000 7f90a4c00700 1 -- 172.21.15.63:0/254127546 shutdown_connections 2024-05-27T08:06:03.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.542+0000 7f90a4c00700 1 -- 172.21.15.63:0/254127546 wait complete. 2024-05-27T08:06:03.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.544+0000 7f90a4c00700 1 Processor -- start 2024-05-27T08:06:03.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.544+0000 7f90a4c00700 1 -- start start 2024-05-27T08:06:03.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.544+0000 7f90a4c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01045e0 0x7f90a019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.544+0000 7f90a4c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a0108f10 0x7f90a019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.544+0000 7f90a4c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90a0109860 0x7f90a01a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.545+0000 7f90a4c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0114390 con 0x7f90a0109860 2024-05-27T08:06:03.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.545+0000 7f90a4c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0114210 con 0x7f90a01045e0 2024-05-27T08:06:03.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.545+0000 7f90a4c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0114510 con 0x7f90a0108f10 2024-05-27T08:06:03.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.545+0000 7f909e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90a0109860 0x7f90a01a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.545+0000 7f909de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01045e0 0x7f90a019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.545+0000 7f909e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90a0109860 0x7f90a01a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36976/0 (socket says 172.21.15.63:36976) 2024-05-27T08:06:03.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.545+0000 7f909e800700 1 -- 172.21.15.63:0/4165867656 learned_addr learned my addr 172.21.15.63:0/4165867656 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:03.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.545+0000 7f909d400700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a0108f10 0x7f90a019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.546+0000 7f909e800700 1 -- 172.21.15.63:0/4165867656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a0108f10 msgr2=0x7f90a019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:03.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.546+0000 7f909e800700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a0108f10 0x7f90a019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.546+0000 7f909e800700 1 -- 172.21.15.63:0/4165867656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01045e0 msgr2=0x7f90a019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:03.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.546+0000 7f909e800700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01045e0 0x7f90a019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.546+0000 7f909e800700 1 -- 172.21.15.63:0/4165867656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a01a1ef0 con 0x7f90a0109860 2024-05-27T08:06:03.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.546+0000 7f909e800700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90a0109860 0x7f90a01a1850 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f9098007ee0 tx=0x7f909800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:03.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.547+0000 7f9096a00700 1 -- 172.21.15.63:0/4165867656 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098004bb0 con 0x7f90a0109860 2024-05-27T08:06:03.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.547+0000 7f90a4c00700 1 -- 172.21.15.63:0/4165867656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a01a21e0 con 0x7f90a0109860 2024-05-27T08:06:03.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.547+0000 7f9096a00700 1 -- 172.21.15.63:0/4165867656 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9098004d50 con 0x7f90a0109860 2024-05-27T08:06:03.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.547+0000 7f9096a00700 1 -- 172.21.15.63:0/4165867656 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980058b0 con 0x7f90a0109860 2024-05-27T08:06:03.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.547+0000 7f90a4c00700 1 -- 172.21.15.63:0/4165867656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a01a9b00 con 0x7f90a0109860 2024-05-27T08:06:03.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.553+0000 7f9096a00700 1 -- 172.21.15.63:0/4165867656 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f909801a020 con 0x7f90a0109860 2024-05-27T08:06:03.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.555+0000 7f90b37a4b80 1 Processor -- start 2024-05-27T08:06:03.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.555+0000 7f90b37a4b80 1 -- start start 2024-05-27T08:06:03.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.555+0000 7f9096a00700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90840b29f0 0x7f90840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.556+0000 7f9096a00700 1 -- 172.21.15.63:0/4165867656 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f90980e1fa0 con 0x7f90a0109860 2024-05-27T08:06:03.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.556+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b046330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.556+0000 7f909d400700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90840b29f0 0x7f90840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.556+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b046870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.556+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b053110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.556+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ed7af74e40 con 0x55ed7b046e70 2024-05-27T08:06:03.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.556+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed7af649e0 con 0x55ed7aec7e40 2024-05-27T08:06:03.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.556+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ed7af68a20 con 0x55ed7afb39f0 2024-05-27T08:06:03.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.557+0000 7f909e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b053110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.557+0000 7f909e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b053110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36982/0 (socket says 172.21.15.63:36982) 2024-05-27T08:06:03.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.557+0000 7f909e800700 1 -- 172.21.15.63:0/3886433740 learned_addr learned my addr 172.21.15.63:0/3886433740 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:03.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.557+0000 7f909de00700 1 --2- 172.21.15.63:0/3886433740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b046330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.557+0000 7f909d400700 1 --2- 172.21.15.63:0/3886433740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b046870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.557+0000 7f909d400700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90840b29f0 0x7f90840b4f00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f90a0107e20 tx=0x7f908c0106b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.557+0000 7f909e800700 1 -- 172.21.15.63:0/3886433740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 msgr2=0x55ed7b046330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.557+0000 7f909e800700 1 --2- 172.21.15.63:0/3886433740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b046330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.557+0000 7f909e800700 1 -- 172.21.15.63:0/3886433740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 msgr2=0x55ed7b046870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.558+0000 7f909e800700 1 --2- 172.21.15.63:0/3886433740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b046870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.558+0000 7f909e800700 1 -- 172.21.15.63:0/3886433740 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7af2d420 con 0x55ed7b046e70 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.558+0000 7f909e800700 1 --2- 172.21.15.63:0/3886433740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b053110 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f90980a8e50 tx=0x7f90980a8700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0b37045868a700 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.558+0000 7f908b400700 1 -- 172.21.15.63:0/3886433740 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980cb070 con 0x55ed7b046e70 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.558+0000 7f908b400700 1 -- 172.21.15.63:0/3886433740 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90980a8ec0 con 0x55ed7b046e70 2024-05-27T08:06:03.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.558+0000 7f908b400700 1 -- 172.21.15.63:0/3886433740 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980a9120 con 0x55ed7b046e70 2024-05-27T08:06:03.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.560+0000 7f90b37a4b80 1 -- 172.21.15.63:0/3886433740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 msgr2=0x55ed7b053110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:03.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.560+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/3886433740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b053110 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f90980a8e50 tx=0x7f90980a8700 comp rx=0 tx=0).stop 2024-05-27T08:06:03.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.560+0000 7f90b37a4b80 1 -- 172.21.15.63:0/3886433740 shutdown_connections 2024-05-27T08:06:03.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.560+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/3886433740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b053110 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.560+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/3886433740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b046330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.560+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/3886433740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b046870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.560+0000 7f90b37a4b80 1 -- 172.21.15.63:0/3886433740 >> 172.21.15.63:0/3886433740 conn(0x55ed7af61390 msgr2=0x55ed7b0424c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:03.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.560+0000 7f90b37a4b80 1 -- 172.21.15.63:0/3886433740 shutdown_connections 2024-05-27T08:06:03.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.560+0000 7f90b37a4b80 1 -- 172.21.15.63:0/3886433740 wait complete. 2024-05-27T08:06:03.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.561+0000 7f90b37a4b80 1 -- start start 2024-05-27T08:06:03.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.561+0000 7f90b37a4b80 1 Processor -- start 2024-05-27T08:06:03.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.562+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7aec7e40 0x55ed7b0506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.562+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b050c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.562+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7b046e70 0x55ed7b051140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.562+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ed7ae6afa0 con 0x55ed7aec7e40 2024-05-27T08:06:03.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.562+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed7af05760 con 0x55ed7b046e70 2024-05-27T08:06:03.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.562+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ed7b053b80 con 0x55ed7afb39f0 2024-05-27T08:06:03.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.562+0000 7f90b37a4b80 10 client.0 _ll_register_callbacks cb 0x55ed7af0ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:03.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7aec7e40 0x55ed7b0506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7aec7e40 0x55ed7b0506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36992/0 (socket says 172.21.15.63:36992) 2024-05-27T08:06:03.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909de00700 1 -- 172.21.15.63:0/1554506478 learned_addr learned my addr 172.21.15.63:0/1554506478 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:03.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909d400700 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b050c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909e800700 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7b046e70 0x55ed7b051140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909de00700 1 -- 172.21.15.63:0/1554506478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 msgr2=0x55ed7b050c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909de00700 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b050c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909de00700 1 -- 172.21.15.63:0/1554506478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7b046e70 msgr2=0x55ed7b051140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909de00700 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7b046e70 0x55ed7b051140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909de00700 1 -- 172.21.15.63:0/1554506478 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7aeb48d0 con 0x55ed7aec7e40 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909e800700 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7b046e70 0x55ed7b051140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.563+0000 7f909d400700 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b050c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.564+0000 7f909de00700 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7aec7e40 0x55ed7b0506c0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f90900099c0 tx=0x7f9090009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76744fdbc2b6946 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.564+0000 7f907fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.564+0000 7f907fe00700 1 -- 172.21.15.63:0/1554506478 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909000bce0 con 0x55ed7aec7e40 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.564+0000 7f907fe00700 1 -- 172.21.15.63:0/1554506478 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f9090003680 con 0x55ed7aec7e40 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.564+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1554506478 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90980a8ec0 con 0x55ed7aec7e40 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.565+0000 7f907fe00700 1 -- 172.21.15.63:0/1554506478 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9090031050 con 0x55ed7aec7e40 2024-05-27T08:06:03.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.565+0000 7f90b37a4b80 10 client.21645 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.565+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1554506478 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ed7af35060 con 0x55ed7aec7e40 2024-05-27T08:06:03.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.565+0000 7f907fe00700 1 -- 172.21.15.63:0/1554506478 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90980a8ec0 con 0x55ed7aec7e40 2024-05-27T08:06:03.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.565+0000 7f907fe00700 1 -- 172.21.15.63:0/1554506478 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f90900237e0 con 0x55ed7aec7e40 2024-05-27T08:06:03.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.566+0000 7f90b37a4b80 10 client.21645 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:03.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.566+0000 7f90b37a4b80 10 client.21645 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:06:03.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.566+0000 7f90b37a4b80 20 client.21645 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:03.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.566+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed7b060c10 0x55ed7b063050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.566+0000 7f90b37a4b80 4 client.21645 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:03.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.566+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1554506478 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed7af76d90 con 0x55ed7b060c10 2024-05-27T08:06:03.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.567+0000 7f909d400700 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed7b060c10 0x55ed7b063050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.568+0000 7f909d400700 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed7b060c10 0x55ed7b063050 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4071bf53d5ca5156 server_cookie=a9304ce41f51a0a4 in_seq=0 out_seq=0 2024-05-27T08:06:03.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.568+0000 7f907fe00700 10 client.21645 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:06:03.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.570+0000 7f907fe00700 1 -- 172.21.15.63:0/1554506478 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ed7af76d90 con 0x55ed7b060c10 2024-05-27T08:06:03.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.570+0000 7f907fe00700 10 client.21645 handle_command_reply: tid=0 2024-05-27T08:06:03.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.570+0000 7f90b37a4b80 1 client.21645 shutdown 2024-05-27T08:06:03.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.570+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1554506478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7aec7e40 msgr2=0x55ed7b0506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:03.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.570+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7aec7e40 0x55ed7b0506c0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f90900099c0 tx=0x7f9090009640 comp rx=0 tx=0).stop 2024-05-27T08:06:03.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.571+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1554506478 shutdown_connections 2024-05-27T08:06:03.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.571+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7b046e70 0x55ed7b051140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.571+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed7b060c10 0x55ed7b063050 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.571+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7aec7e40 0x55ed7b0506c0 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.571+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1554506478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b050c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.571+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1554506478 >> 172.21.15.63:0/1554506478 conn(0x55ed7af61390 msgr2=0x55ed7b046310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:03.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.571+0000 7f907fe00700 0 client.21645 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:06:03.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.571+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1554506478 shutdown_connections 2024-05-27T08:06:03.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.571+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1554506478 wait complete. 2024-05-27T08:06:03.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.571+0000 7f90b37a4b80 20 client.21645 trim_cache size 0 max 16384 2024-05-27T08:06:03.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.593+0000 7f90b37a4b80 1 Processor -- start 2024-05-27T08:06:03.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.593+0000 7f90b37a4b80 1 -- start start 2024-05-27T08:06:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.594+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b04f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.594+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7afb39f0 0x55ed7b04be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.594+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7b046e70 0x55ed7b0467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.594+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9080011610 con 0x55ed7afb39f0 2024-05-27T08:06:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.594+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f908000eea0 con 0x55ed7aec7e40 2024-05-27T08:06:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.594+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f908000cd10 con 0x55ed7b046e70 2024-05-27T08:06:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.594+0000 7f909d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7afb39f0 0x55ed7b04be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.594+0000 7f909d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7afb39f0 0x55ed7b04be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36998/0 (socket says 172.21.15.63:36998) 2024-05-27T08:06:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.594+0000 7f909d400700 1 -- 172.21.15.63:0/2927038333 learned_addr learned my addr 172.21.15.63:0/2927038333 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.594+0000 7f909de00700 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b04f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.595+0000 7f909e800700 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7b046e70 0x55ed7b0467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.595+0000 7f909d400700 1 -- 172.21.15.63:0/2927038333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7b046e70 msgr2=0x55ed7b0467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.595+0000 7f909d400700 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7b046e70 0x55ed7b0467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.595+0000 7f909d400700 1 -- 172.21.15.63:0/2927038333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 msgr2=0x55ed7b04f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.595+0000 7f909d400700 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b04f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.595+0000 7f909d400700 1 -- 172.21.15.63:0/2927038333 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7b047a50 con 0x55ed7afb39f0 2024-05-27T08:06:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.595+0000 7f909d400700 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7afb39f0 0x55ed7b04be90 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x55ed7b043a20 tx=0x7f908c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e99f2189f6806a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.595+0000 7f909e800700 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7b046e70 0x55ed7b0467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:06:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.595+0000 7f909de00700 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b04f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.596+0000 7f908aa00700 1 -- 172.21.15.63:0/2927038333 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c004a30 con 0x55ed7afb39f0 2024-05-27T08:06:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.596+0000 7f908aa00700 1 -- 172.21.15.63:0/2927038333 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f908c0054b0 con 0x55ed7afb39f0 2024-05-27T08:06:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.596+0000 7f908aa00700 1 -- 172.21.15.63:0/2927038333 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c02a730 con 0x55ed7afb39f0 2024-05-27T08:06:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.597+0000 7f90b37a4b80 1 -- 172.21.15.63:0/2927038333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7afb39f0 msgr2=0x55ed7b04be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.597+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7afb39f0 0x55ed7b04be90 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x55ed7b043a20 tx=0x7f908c015000 comp rx=0 tx=0).stop 2024-05-27T08:06:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.597+0000 7f90b37a4b80 1 -- 172.21.15.63:0/2927038333 shutdown_connections 2024-05-27T08:06:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.597+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7b046e70 0x55ed7b0467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.597+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b04f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.597+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/2927038333 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7afb39f0 0x55ed7b04be90 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.597+0000 7f90b37a4b80 1 -- 172.21.15.63:0/2927038333 >> 172.21.15.63:0/2927038333 conn(0x55ed7af61390 msgr2=0x55ed7b0460b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.597+0000 7f90b37a4b80 1 -- 172.21.15.63:0/2927038333 shutdown_connections 2024-05-27T08:06:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.597+0000 7f90b37a4b80 1 -- 172.21.15.63:0/2927038333 wait complete. 2024-05-27T08:06:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.598+0000 7f90b37a4b80 1 -- start start 2024-05-27T08:06:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.598+0000 7f90b37a4b80 1 Processor -- start 2024-05-27T08:06:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b0543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b054910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f90b37a4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f908000d360 con 0x55ed7b046e70 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f908000e7d0 con 0x55ed7aec7e40 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f90b37a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f908000da30 con 0x55ed7afb39f0 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f90b37a4b80 10 client.0 _ll_register_callbacks cb 0x55ed7af0ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f909e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f909e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b0c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37010/0 (socket says 172.21.15.63:37010) 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f909e800700 1 -- 172.21.15.63:0/1275589845 learned_addr learned my addr 172.21.15.63:0/1275589845 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f909d400700 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b054910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.599+0000 7f909de00700 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b0543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.600+0000 7f909e800700 1 -- 172.21.15.63:0/1275589845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 msgr2=0x55ed7b054910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:03.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.600+0000 7f909e800700 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b054910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.600+0000 7f909e800700 1 -- 172.21.15.63:0/1275589845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 msgr2=0x55ed7b0543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.600+0000 7f909e800700 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b0543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.600+0000 7f909e800700 1 -- 172.21.15.63:0/1275589845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7b0c3c50 con 0x55ed7b046e70 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.600+0000 7f909e800700 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b0c36e0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f90980df930 tx=0x7f90980194f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a08531688ed46d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.600+0000 7f907fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.601+0000 7f907fe00700 1 -- 172.21.15.63:0/1275589845 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909806c020 con 0x55ed7b046e70 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.601+0000 7f907fe00700 1 -- 172.21.15.63:0/1275589845 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9483+0+0 (secure 0 0 0) 0x7f90980a9330 con 0x55ed7b046e70 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.601+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1275589845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908c0054b0 con 0x55ed7b046e70 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.601+0000 7f907fe00700 1 -- 172.21.15.63:0/1275589845 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f90980cb040 con 0x55ed7b046e70 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.601+0000 7f90b37a4b80 10 client.21651 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.601+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1275589845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ed7b0caea0 con 0x55ed7b046e70 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.601+0000 7f907fe00700 1 -- 172.21.15.63:0/1275589845 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f908c0054b0 con 0x55ed7b046e70 2024-05-27T08:06:03.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.602+0000 7f907fe00700 1 -- 172.21.15.63:0/1275589845 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f90980b1890 con 0x55ed7b046e70 2024-05-27T08:06:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.602+0000 7f90b37a4b80 10 client.21651 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.602+0000 7f90b37a4b80 10 client.21651 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:06:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.602+0000 7f90b37a4b80 20 client.21651 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.602+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed7b0cb180 0x55ed7b0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.602+0000 7f90b37a4b80 4 client.21651 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:06:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.602+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1275589845 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed7af70570 con 0x55ed7b0cb180 2024-05-27T08:06:03.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.603+0000 7f909de00700 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed7b0cb180 0x55ed7b0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.604+0000 7f909de00700 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed7b0cb180 0x55ed7b0cd5a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea5a5420f72cf710 server_cookie=574c11d9a9211729 in_seq=0 out_seq=0 2024-05-27T08:06:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.604+0000 7f907fe00700 10 client.21651 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:06:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.604+0000 7f907fe00700 1 -- 172.21.15.63:0/1275589845 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed7af70570 con 0x55ed7b0cb180 2024-05-27T08:06:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.604+0000 7f907fe00700 10 client.21651 handle_command_reply: tid=0 2024-05-27T08:06:03.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.604+0000 7f90b37a4b80 1 client.21651 shutdown 2024-05-27T08:06:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.604+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1275589845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 msgr2=0x55ed7b0c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.604+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b0c36e0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f90980df930 tx=0x7f90980194f0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.605+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1275589845 shutdown_connections 2024-05-27T08:06:03.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.605+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed7b0cb180 0x55ed7b0cd5a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.605+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed7b046e70 0x55ed7b0c36e0 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.605+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed7aec7e40 0x55ed7b0543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.605+0000 7f90b37a4b80 1 --2- 172.21.15.63:0/1275589845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed7afb39f0 0x55ed7b054910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.605+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1275589845 >> 172.21.15.63:0/1275589845 conn(0x55ed7af61390 msgr2=0x55ed7b0504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.605+0000 7f907fe00700 0 client.21651 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:06:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.605+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1275589845 shutdown_connections 2024-05-27T08:06:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.605+0000 7f90b37a4b80 1 -- 172.21.15.63:0/1275589845 wait complete. 2024-05-27T08:06:03.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.605+0000 7f90b37a4b80 20 client.21651 trim_cache size 0 max 16384 2024-05-27T08:06:03.607 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:06:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 -- 172.21.15.63:0/4165867656 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90840b29f0 msgr2=0x7f90840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90840b29f0 0x7f90840b4f00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f90a0107e20 tx=0x7f908c0106b0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 -- 172.21.15.63:0/4165867656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90a0109860 msgr2=0x7f90a01a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90a0109860 0x7f90a01a1850 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f9098007ee0 tx=0x7f909800d3b0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 -- 172.21.15.63:0/4165867656 shutdown_connections 2024-05-27T08:06:03.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90840b29f0 0x7f90840b4f00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90a0109860 0x7f90a01a1850 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a01045e0 0x7f90a019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 --2- 172.21.15.63:0/4165867656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90a0108f10 0x7f90a019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:03.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 -- 172.21.15.63:0/4165867656 >> 172.21.15.63:0/4165867656 conn(0x7f90a0100030 msgr2=0x7f90a010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:03.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.606+0000 7f90a4c00700 1 -- 172.21.15.63:0/4165867656 shutdown_connections 2024-05-27T08:06:03.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:03.607+0000 7f90a4c00700 1 -- 172.21.15.63:0/4165867656 wait complete. 2024-05-27T08:06:03.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:03 smithi086 ceph-mon[34687]: pgmap v1028: 129 pgs: 129 active+clean; 9.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.5 MiB/s wr, 34 op/s 2024-05-27T08:06:03.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:03 smithi149 ceph-mon[35408]: pgmap v1028: 129 pgs: 129 active+clean; 9.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.5 MiB/s wr, 34 op/s 2024-05-27T08:06:03.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:03 smithi063 ceph-mon[31613]: pgmap v1028: 129 pgs: 129 active+clean; 9.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.5 MiB/s wr, 34 op/s 2024-05-27T08:06:04.644 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:06:05.202 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:05 smithi086 ceph-mon[34687]: pgmap v1029: 129 pgs: 129 active+clean; 9.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.5 MiB/s wr, 28 op/s 2024-05-27T08:06:05.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:05 smithi149 ceph-mon[35408]: pgmap v1029: 129 pgs: 129 active+clean; 9.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.5 MiB/s wr, 28 op/s 2024-05-27T08:06:05.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:05 smithi063 ceph-mon[31613]: pgmap v1029: 129 pgs: 129 active+clean; 9.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.5 MiB/s wr, 28 op/s 2024-05-27T08:06:07.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.693+0000 7f9bc2400700 1 -- 172.21.15.63:0/701101057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc10e930 msgr2=0x7f9bbc110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:07.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.693+0000 7f9bc2400700 1 --2- 172.21.15.63:0/701101057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc10e930 0x7f9bbc110e00 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400b0a0 tx=0x7f9bb400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.693+0000 7f9bc2400700 1 -- 172.21.15.63:0/701101057 shutdown_connections 2024-05-27T08:06:07.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.693+0000 7f9bc2400700 1 --2- 172.21.15.63:0/701101057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc10e930 0x7f9bbc110e00 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.693+0000 7f9bc2400700 1 --2- 172.21.15.63:0/701101057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bbc105470 0x7f9bbc105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.693+0000 7f9bc2400700 1 --2- 172.21.15.63:0/701101057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bbc105e50 0x7f9bbc10e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.693+0000 7f9bc2400700 1 -- 172.21.15.63:0/701101057 >> 172.21.15.63:0/701101057 conn(0x7f9bbc0fef40 msgr2=0x7f9bbc101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:07.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.693+0000 7f9bc2400700 1 -- 172.21.15.63:0/701101057 shutdown_connections 2024-05-27T08:06:07.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.693+0000 7f9bc2400700 1 -- 172.21.15.63:0/701101057 wait complete. 2024-05-27T08:06:07.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.694+0000 7f9bc2400700 1 Processor -- start 2024-05-27T08:06:07.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.694+0000 7f9bc2400700 1 -- start start 2024-05-27T08:06:07.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.695+0000 7f9bc2400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bbc105470 0x7f9bbc103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.695+0000 7f9bc2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bbc105e50 0x7f9bbc1036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.695+0000 7f9bc2400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc10e930 0x7f9bbc103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.695+0000 7f9bc2400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1143a0 con 0x7f9bbc10e930 2024-05-27T08:06:07.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.695+0000 7f9bc2400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc114220 con 0x7f9bbc105e50 2024-05-27T08:06:07.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.695+0000 7f9bc2400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc114520 con 0x7f9bbc105470 2024-05-27T08:06:07.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.695+0000 7f9bbbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc10e930 0x7f9bbc103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc10e930 0x7f9bbc103be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37026/0 (socket says 172.21.15.63:37026) 2024-05-27T08:06:07.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbbe00700 1 -- 172.21.15.63:0/3193928479 learned_addr learned my addr 172.21.15.63:0/3193928479 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:07.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbb400700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bbc105470 0x7f9bbc103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bbc105e50 0x7f9bbc1036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbbe00700 1 -- 172.21.15.63:0/3193928479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bbc105470 msgr2=0x7f9bbc103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:07.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbbe00700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bbc105470 0x7f9bbc103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbbe00700 1 -- 172.21.15.63:0/3193928479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bbc105e50 msgr2=0x7f9bbc1036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:07.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbbe00700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bbc105e50 0x7f9bbc1036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbbe00700 1 -- 172.21.15.63:0/3193928479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bbc1a9110 con 0x7f9bbc10e930 2024-05-27T08:06:07.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbb400700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bbc105470 0x7f9bbc103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:07.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.696+0000 7f9bbbe00700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc10e930 0x7f9bbc103be0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400b070 tx=0x7f9bb4004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:07.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.697+0000 7f9bb3e00700 1 -- 172.21.15.63:0/3193928479 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4021070 con 0x7f9bbc10e930 2024-05-27T08:06:07.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.697+0000 7f9bc2400700 1 -- 172.21.15.63:0/3193928479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bbc1a93a0 con 0x7f9bbc10e930 2024-05-27T08:06:07.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.697+0000 7f9bb3e00700 1 -- 172.21.15.63:0/3193928479 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bb4007b80 con 0x7f9bbc10e930 2024-05-27T08:06:07.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.697+0000 7f9bb3e00700 1 -- 172.21.15.63:0/3193928479 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40046e0 con 0x7f9bbc10e930 2024-05-27T08:06:07.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.697+0000 7f9bc2400700 1 -- 172.21.15.63:0/3193928479 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bbc1a9850 con 0x7f9bbc10e930 2024-05-27T08:06:07.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.702+0000 7f9bb3e00700 1 -- 172.21.15.63:0/3193928479 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9bb4003a00 con 0x7f9bbc10e930 2024-05-27T08:06:07.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.703+0000 7f9bb3e00700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ba00b2a70 0x7f9ba00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.703+0000 7f9bd0fd1b80 1 Processor -- start 2024-05-27T08:06:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.704+0000 7f9bb3e00700 1 -- 172.21.15.63:0/3193928479 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 9571+0+0 (secure 0 0 0) 0x7f9bb400f020 con 0x7f9bbc10e930 2024-05-27T08:06:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.704+0000 7f9bd0fd1b80 1 -- start start 2024-05-27T08:06:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.704+0000 7f9bbb400700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ba00b2a70 0x7f9ba00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.704+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b83330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.704+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6a04e40 0x55cae6b83870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.704+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b90110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.704+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cae6ab1e40 con 0x55cae6af09f0 2024-05-27T08:06:07.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.704+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cae6aa19e0 con 0x55cae6a04e40 2024-05-27T08:06:07.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.704+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cae6aa5a20 con 0x55cae6b83e70 2024-05-27T08:06:07.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.705+0000 7f9bbb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b83330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.705+0000 7f9bbb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b83330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37030/0 (socket says 172.21.15.63:37030) 2024-05-27T08:06:07.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.705+0000 7f9bbb400700 1 -- 172.21.15.63:0/4085462508 learned_addr learned my addr 172.21.15.63:0/4085462508 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:07.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.705+0000 7f9bbbe00700 1 --2- 172.21.15.63:0/4085462508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b90110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.705+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/4085462508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6a04e40 0x55cae6b83870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.706+0000 7f9bbb400700 1 -- 172.21.15.63:0/4085462508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 msgr2=0x55cae6b90110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:07.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.706+0000 7f9bbb400700 1 --2- 172.21.15.63:0/4085462508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b90110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.706+0000 7f9bbb400700 1 -- 172.21.15.63:0/4085462508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6a04e40 msgr2=0x55cae6b83870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:07.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.706+0000 7f9bbb400700 1 --2- 172.21.15.63:0/4085462508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6a04e40 0x55cae6b83870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.706+0000 7f9bbb400700 1 -- 172.21.15.63:0/4085462508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae6a6a420 con 0x55cae6af09f0 2024-05-27T08:06:07.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.706+0000 7f9bbb400700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ba00b2a70 0x7f9ba00b4f80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9bac013670 tx=0x7f9bac00bf90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:07.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.706+0000 7f9bbb400700 1 --2- 172.21.15.63:0/4085462508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b83330 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f9bac013dc0 tx=0x7f9bac01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb602de2418352a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:07.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.706+0000 7f9bb0c00700 1 -- 172.21.15.63:0/4085462508 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac023030 con 0x55cae6af09f0 2024-05-27T08:06:07.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.707+0000 7f9bb0c00700 1 -- 172.21.15.63:0/4085462508 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bac0044b0 con 0x55cae6af09f0 2024-05-27T08:06:07.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.707+0000 7f9bb0c00700 1 -- 172.21.15.63:0/4085462508 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac003c20 con 0x55cae6af09f0 2024-05-27T08:06:07.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.708+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/4085462508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 msgr2=0x55cae6b83330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:07.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.708+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/4085462508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b83330 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f9bac013dc0 tx=0x7f9bac01d000 comp rx=0 tx=0).stop 2024-05-27T08:06:07.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.708+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/4085462508 shutdown_connections 2024-05-27T08:06:07.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.708+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/4085462508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b90110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.708+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/4085462508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b83330 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.708+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/4085462508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6a04e40 0x55cae6b83870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.708+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/4085462508 >> 172.21.15.63:0/4085462508 conn(0x55cae6a9e390 msgr2=0x55cae6b7f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:07.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.708+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/4085462508 shutdown_connections 2024-05-27T08:06:07.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.708+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/4085462508 wait complete. 2024-05-27T08:06:07.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.709+0000 7f9bd0fd1b80 1 -- start start 2024-05-27T08:06:07.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.709+0000 7f9bd0fd1b80 1 Processor -- start 2024-05-27T08:06:07.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.710+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6a04e40 0x55cae6b8d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.710+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 0x55cae6b8dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.710+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6b83e70 0x55cae6b8e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.710+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cae69a7fa0 con 0x55cae6b83e70 2024-05-27T08:06:07.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.710+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cae6a42760 con 0x55cae6af09f0 2024-05-27T08:06:07.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.710+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cae6b90b20 con 0x55cae6a04e40 2024-05-27T08:06:07.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.710+0000 7f9bd0fd1b80 10 client.0 _ll_register_callbacks cb 0x55cae6a4bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:07.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.710+0000 7f9bbbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6b83e70 0x55cae6b8e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.710+0000 7f9bbbe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6b83e70 0x55cae6b8e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37044/0 (socket says 172.21.15.63:37044) 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.710+0000 7f9bbbe00700 1 -- 172.21.15.63:0/1300955575 learned_addr learned my addr 172.21.15.63:0/1300955575 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.711+0000 7f9bbb400700 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6a04e40 0x55cae6b8d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.711+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 0x55cae6b8dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.711+0000 7f9bbbe00700 1 -- 172.21.15.63:0/1300955575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6a04e40 msgr2=0x55cae6b8d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.711+0000 7f9bbbe00700 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6a04e40 0x55cae6b8d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.711+0000 7f9bbbe00700 1 -- 172.21.15.63:0/1300955575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 msgr2=0x55cae6b8dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.711+0000 7f9bbbe00700 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 0x55cae6b8dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.711+0000 7f9bbbe00700 1 -- 172.21.15.63:0/1300955575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae69f18d0 con 0x55cae6b83e70 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.711+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 0x55cae6b8dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.711+0000 7f9bbbe00700 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6b83e70 0x55cae6b8e130 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40a6e60 tx=0x7f9bb40075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128458e813c0bfb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.711+0000 7f9ba5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:07.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.712+0000 7f9ba5600700 1 -- 172.21.15.63:0/1300955575 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4080a80 con 0x55cae6b83e70 2024-05-27T08:06:07.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.712+0000 7f9ba5600700 1 -- 172.21.15.63:0/1300955575 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 9571+0+0 (secure 0 0 0) 0x7f9bb40a7420 con 0x55cae6b83e70 2024-05-27T08:06:07.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.712+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1300955575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bac0044b0 con 0x55cae6b83e70 2024-05-27T08:06:07.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.712+0000 7f9ba5600700 1 -- 172.21.15.63:0/1300955575 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9bb40ab530 con 0x55cae6b83e70 2024-05-27T08:06:07.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.712+0000 7f9bd0fd1b80 10 client.21663 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:07.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.712+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1300955575 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cae6a72060 con 0x55cae6b83e70 2024-05-27T08:06:07.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.713+0000 7f9ba5600700 1 -- 172.21.15.63:0/1300955575 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bac0044b0 con 0x55cae6b83e70 2024-05-27T08:06:07.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.713+0000 7f9ba5600700 1 -- 172.21.15.63:0/1300955575 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9bb4011070 con 0x55cae6b83e70 2024-05-27T08:06:07.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.713+0000 7f9bd0fd1b80 10 client.21663 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:07.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.713+0000 7f9bd0fd1b80 10 client.21663 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:06:07.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.713+0000 7f9bd0fd1b80 20 client.21663 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:07.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.713+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55cae6b9db00 0x55cae6b9ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.713+0000 7f9bd0fd1b80 4 client.21663 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:07.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.713+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1300955575 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cae6ab3d90 con 0x55cae6b9db00 2024-05-27T08:06:07.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.714+0000 7f9bbb400700 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55cae6b9db00 0x55cae6b9ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.715+0000 7f9bbb400700 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55cae6b9db00 0x55cae6b9ff50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8dd906930f1b7c89 server_cookie=9f38709ee7f459d3 in_seq=0 out_seq=0 2024-05-27T08:06:07.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.715+0000 7f9ba5600700 10 client.21663 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:06:07.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.717+0000 7f9ba5600700 1 -- 172.21.15.63:0/1300955575 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cae6ab3d90 con 0x55cae6b9db00 2024-05-27T08:06:07.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.717+0000 7f9ba5600700 10 client.21663 handle_command_reply: tid=0 2024-05-27T08:06:07.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.717+0000 7f9bd0fd1b80 1 client.21663 shutdown 2024-05-27T08:06:07.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.717+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1300955575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6b83e70 msgr2=0x55cae6b8e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:07.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.717+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6b83e70 0x55cae6b8e130 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40a6e60 tx=0x7f9bb40075c0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.718+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1300955575 shutdown_connections 2024-05-27T08:06:07.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.718+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55cae6b9db00 0x55cae6b9ff50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.718+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6b83e70 0x55cae6b8e130 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.718+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6a04e40 0x55cae6b8d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.718+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/1300955575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 0x55cae6b8dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.718+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1300955575 >> 172.21.15.63:0/1300955575 conn(0x55cae6a9e390 msgr2=0x55cae6b833b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.718+0000 7f9ba5600700 0 client.21663 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:06:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.718+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1300955575 shutdown_connections 2024-05-27T08:06:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.718+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1300955575 wait complete. 2024-05-27T08:06:07.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.718+0000 7f9bd0fd1b80 20 client.21663 trim_cache size 0 max 16384 2024-05-27T08:06:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.737+0000 7f9bd0fd1b80 1 Processor -- start 2024-05-27T08:06:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.737+0000 7f9bd0fd1b80 1 -- start start 2024-05-27T08:06:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.737+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6a04e40 0x55cae6b88e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.737+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b7f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b8c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1b5030 con 0x55cae6af09f0 2024-05-27T08:06:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1b28c0 con 0x55cae6a04e40 2024-05-27T08:06:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1b0730 con 0x55cae6b83e70 2024-05-27T08:06:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bbaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b7f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bbaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b7f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37056/0 (socket says 172.21.15.63:37056) 2024-05-27T08:06:07.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bbaa00700 1 -- 172.21.15.63:0/1382725840 learned_addr learned my addr 172.21.15.63:0/1382725840 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:07.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bbb400700 1 --2- 172.21.15.63:0/1382725840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6a04e40 0x55cae6b88e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bbbe00700 1 --2- 172.21.15.63:0/1382725840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b8c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bbaa00700 1 -- 172.21.15.63:0/1382725840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 msgr2=0x55cae6b8c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:07.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/1382725840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b8c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bbaa00700 1 -- 172.21.15.63:0/1382725840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6a04e40 msgr2=0x55cae6b88e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:07.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/1382725840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6a04e40 0x55cae6b88e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.738+0000 7f9bbaa00700 1 -- 172.21.15.63:0/1382725840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae6ba0520 con 0x55cae6af09f0 2024-05-27T08:06:07.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.739+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/1382725840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b7f5c0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8005950 tx=0x7f9ba8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c056d3fc07a676e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:07.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.739+0000 7f9ba7e00700 1 -- 172.21.15.63:0/1382725840 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8004050 con 0x55cae6af09f0 2024-05-27T08:06:07.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.739+0000 7f9ba7e00700 1 -- 172.21.15.63:0/1382725840 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ba8004830 con 0x55cae6af09f0 2024-05-27T08:06:07.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.739+0000 7f9ba7e00700 1 -- 172.21.15.63:0/1382725840 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8005280 con 0x55cae6af09f0 2024-05-27T08:06:07.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.740+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1382725840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 msgr2=0x55cae6b7f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:07.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.740+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/1382725840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b7f5c0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8005950 tx=0x7f9ba8011720 comp rx=0 tx=0).stop 2024-05-27T08:06:07.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.741+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1382725840 shutdown_connections 2024-05-27T08:06:07.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.741+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/1382725840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b8c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.741+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/1382725840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6a04e40 0x55cae6b88e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.741+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/1382725840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6af09f0 0x55cae6b7f5c0 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.741+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1382725840 >> 172.21.15.63:0/1382725840 conn(0x55cae6a9e390 msgr2=0x55cae6b8bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:07.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.741+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1382725840 shutdown_connections 2024-05-27T08:06:07.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.741+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/1382725840 wait complete. 2024-05-27T08:06:07.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.741+0000 7f9bd0fd1b80 1 -- start start 2024-05-27T08:06:07.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.742+0000 7f9bd0fd1b80 1 Processor -- start 2024-05-27T08:06:07.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.742+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6a04e40 0x55cae6b88b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.742+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 0x55cae6b86ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.742+0000 7f9bd0fd1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b87430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.742+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1b0d80 con 0x55cae6a04e40 2024-05-27T08:06:07.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.742+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1b21f0 con 0x55cae6af09f0 2024-05-27T08:06:07.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.742+0000 7f9bd0fd1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1b1450 con 0x55cae6b83e70 2024-05-27T08:06:07.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.742+0000 7f9bd0fd1b80 10 client.0 _ll_register_callbacks cb 0x55cae6a4bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:07.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6a04e40 0x55cae6b88b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6a04e40 0x55cae6b88b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37070/0 (socket says 172.21.15.63:37070) 2024-05-27T08:06:07.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbb400700 1 -- 172.21.15.63:0/38072266 learned_addr learned my addr 172.21.15.63:0/38072266 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:07.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 0x55cae6b86ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbbe00700 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b87430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbb400700 1 -- 172.21.15.63:0/38072266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 msgr2=0x55cae6b87430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:07.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbb400700 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b87430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbb400700 1 -- 172.21.15.63:0/38072266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 msgr2=0x55cae6b86ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:07.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbb400700 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 0x55cae6b86ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbb400700 1 -- 172.21.15.63:0/38072266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae6b879c0 con 0x55cae6a04e40 2024-05-27T08:06:07.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 0x55cae6b86ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:07.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.743+0000 7f9bbbe00700 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b87430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:06:07.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.744+0000 7f9bbb400700 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6a04e40 0x55cae6b88b60 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f9bac016040 tx=0x7f9bac00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e0dc33d1b157b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:07.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.744+0000 7f9ba5600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:07.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.744+0000 7f9ba5600700 1 -- 172.21.15.63:0/38072266 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac013250 con 0x55cae6a04e40 2024-05-27T08:06:07.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.744+0000 7f9ba5600700 1 -- 172.21.15.63:0/38072266 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 9571+0+0 (secure 0 0 0) 0x7f9bac0037d0 con 0x55cae6a04e40 2024-05-27T08:06:07.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.744+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/38072266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ba8004830 con 0x55cae6a04e40 2024-05-27T08:06:07.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.745+0000 7f9ba5600700 1 -- 172.21.15.63:0/38072266 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9bac040050 con 0x55cae6a04e40 2024-05-27T08:06:07.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.745+0000 7f9bd0fd1b80 10 client.21669 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:07.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.745+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/38072266 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cae6b95b20 con 0x55cae6a04e40 2024-05-27T08:06:07.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.745+0000 7f9ba5600700 1 -- 172.21.15.63:0/38072266 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ba8004830 con 0x55cae6a04e40 2024-05-27T08:06:07.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.745+0000 7f9ba5600700 1 -- 172.21.15.63:0/38072266 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9bac005040 con 0x55cae6a04e40 2024-05-27T08:06:07.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.745+0000 7f9bd0fd1b80 10 client.21669 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:07.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.745+0000 7f9bd0fd1b80 10 client.21669 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:06:07.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.745+0000 7f9bd0fd1b80 20 client.21669 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:07.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.746+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55cae6b95f50 0x55cae6b96360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:07.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.746+0000 7f9bd0fd1b80 4 client.21669 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:06:07.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.746+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/38072266 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cae6aad570 con 0x55cae6b95f50 2024-05-27T08:06:07.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.746+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55cae6b95f50 0x55cae6b96360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:07.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.747+0000 7f9ba5600700 10 client.21669 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:06:07.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.747+0000 7f9bbaa00700 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55cae6b95f50 0x55cae6b96360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=394546e0014bdacb server_cookie=4efcf5e946f6e99e in_seq=0 out_seq=0 2024-05-27T08:06:07.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9ba5600700 1 -- 172.21.15.63:0/38072266 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cae6aad570 con 0x55cae6b95f50 2024-05-27T08:06:07.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9ba5600700 10 client.21669 handle_command_reply: tid=0 2024-05-27T08:06:07.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9bd0fd1b80 1 client.21669 shutdown 2024-05-27T08:06:07.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/38072266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6a04e40 msgr2=0x55cae6b88b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6a04e40 0x55cae6b88b60 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f9bac016040 tx=0x7f9bac00bab0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/38072266 shutdown_connections 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cae6b83e70 0x55cae6b87430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55cae6b95f50 0x55cae6b96360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cae6a04e40 0x55cae6b88b60 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9bd0fd1b80 1 --2- 172.21.15.63:0/38072266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cae6af09f0 0x55cae6b86ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9ba5600700 0 client.21669 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.748+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/38072266 >> 172.21.15.63:0/38072266 conn(0x55cae6a9e390 msgr2=0x55cae6b84ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.749+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/38072266 shutdown_connections 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.749+0000 7f9bd0fd1b80 1 -- 172.21.15.63:0/38072266 wait complete. 2024-05-27T08:06:07.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.749+0000 7f9bd0fd1b80 20 client.21669 trim_cache size 0 max 16384 2024-05-27T08:06:07.751 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:06:07.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 -- 172.21.15.63:0/3193928479 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ba00b2a70 msgr2=0x7f9ba00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:07.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ba00b2a70 0x7f9ba00b4f80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9bac013670 tx=0x7f9bac00bf90 comp rx=0 tx=0).stop 2024-05-27T08:06:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 -- 172.21.15.63:0/3193928479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc10e930 msgr2=0x7f9bbc103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc10e930 0x7f9bbc103be0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400b070 tx=0x7f9bb4004d50 comp rx=0 tx=0).stop 2024-05-27T08:06:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 -- 172.21.15.63:0/3193928479 shutdown_connections 2024-05-27T08:06:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9ba00b2a70 0x7f9ba00b4f80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc10e930 0x7f9bbc103be0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9bbc105470 0x7f9bbc103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 --2- 172.21.15.63:0/3193928479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bbc105e50 0x7f9bbc1036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 -- 172.21.15.63:0/3193928479 >> 172.21.15.63:0/3193928479 conn(0x7f9bbc0fef40 msgr2=0x7f9bbc0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 -- 172.21.15.63:0/3193928479 shutdown_connections 2024-05-27T08:06:07.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:07.750+0000 7f9bc2400700 1 -- 172.21.15.63:0/3193928479 wait complete. 2024-05-27T08:06:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:07 smithi086 ceph-mon[34687]: pgmap v1030: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T08:06:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:07 smithi086 ceph-mon[34687]: osdmap e145: 12 total, 12 up, 12 in 2024-05-27T08:06:07.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:07 smithi149 ceph-mon[35408]: pgmap v1030: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T08:06:07.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:07 smithi149 ceph-mon[35408]: osdmap e145: 12 total, 12 up, 12 in 2024-05-27T08:06:07.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:07 smithi063 ceph-mon[31613]: pgmap v1030: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T08:06:07.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:07 smithi063 ceph-mon[31613]: osdmap e145: 12 total, 12 up, 12 in 2024-05-27T08:06:09.003 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:06:09.486 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:09.913 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:09 smithi063 ceph-mon[31613]: pgmap v1032: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.1 MiB/s wr, 33 op/s 2024-05-27T08:06:09.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:09 smithi086 ceph-mon[34687]: pgmap v1032: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.1 MiB/s wr, 33 op/s 2024-05-27T08:06:09.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:09 smithi149 ceph-mon[35408]: pgmap v1032: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.1 MiB/s wr, 33 op/s 2024-05-27T08:06:11.740 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:11 smithi063 ceph-mon[31613]: pgmap v1033: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.1 MiB/s wr, 33 op/s 2024-05-27T08:06:11.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:11 smithi086 ceph-mon[34687]: pgmap v1033: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.1 MiB/s wr, 33 op/s 2024-05-27T08:06:11.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:11 smithi149 ceph-mon[35408]: pgmap v1033: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.1 MiB/s wr, 33 op/s 2024-05-27T08:06:12.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.427+0000 7fdf14e00700 1 -- 172.21.15.63:0/14958511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf10106560 msgr2=0x7fdf10106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.427+0000 7fdf14e00700 1 --2- 172.21.15.63:0/14958511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf10106560 0x7fdf10106970 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009990 tx=0x7fdefc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.427+0000 7fdf14e00700 1 -- 172.21.15.63:0/14958511 shutdown_connections 2024-05-27T08:06:12.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.427+0000 7fdf14e00700 1 --2- 172.21.15.63:0/14958511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf10106eb0 0x7fdf10113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.427+0000 7fdf14e00700 1 --2- 172.21.15.63:0/14958511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf1010c5a0 0x7fdf1010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.427+0000 7fdf14e00700 1 --2- 172.21.15.63:0/14958511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf10106560 0x7fdf10106970 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.427+0000 7fdf14e00700 1 -- 172.21.15.63:0/14958511 >> 172.21.15.63:0/14958511 conn(0x7fdf10102040 msgr2=0x7fdf10104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:12.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.427+0000 7fdf14e00700 1 -- 172.21.15.63:0/14958511 shutdown_connections 2024-05-27T08:06:12.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.427+0000 7fdf14e00700 1 -- 172.21.15.63:0/14958511 wait complete. 2024-05-27T08:06:12.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.428+0000 7fdf14e00700 1 Processor -- start 2024-05-27T08:06:12.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.428+0000 7fdf14e00700 1 -- start start 2024-05-27T08:06:12.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.429+0000 7fdf14e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf10106560 0x7fdf1019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.429+0000 7fdf14e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf10106eb0 0x7fdf1019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.429+0000 7fdf14e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf1010c5a0 0x7fdf101a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.429+0000 7fdf14e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf10116530 con 0x7fdf1010c5a0 2024-05-27T08:06:12.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.429+0000 7fdf14e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101163b0 con 0x7fdf10106560 2024-05-27T08:06:12.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.429+0000 7fdf14e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101166b0 con 0x7fdf10106eb0 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.429+0000 7fdf0e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf1010c5a0 0x7fdf101a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.429+0000 7fdf0e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf1010c5a0 0x7fdf101a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50364/0 (socket says 172.21.15.63:50364) 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.429+0000 7fdf0e800700 1 -- 172.21.15.63:0/2126182796 learned_addr learned my addr 172.21.15.63:0/2126182796 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.429+0000 7fdf0d400700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf10106eb0 0x7fdf1019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.430+0000 7fdf0de00700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf10106560 0x7fdf1019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.430+0000 7fdf0e800700 1 -- 172.21.15.63:0/2126182796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf10106eb0 msgr2=0x7fdf1019d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.430+0000 7fdf0e800700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf10106eb0 0x7fdf1019d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.430+0000 7fdf0e800700 1 -- 172.21.15.63:0/2126182796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf10106560 msgr2=0x7fdf1019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.430+0000 7fdf0e800700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf10106560 0x7fdf1019cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.430+0000 7fdf0e800700 1 -- 172.21.15.63:0/2126182796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf101a1dd0 con 0x7fdf1010c5a0 2024-05-27T08:06:12.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.430+0000 7fdf0d400700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf10106eb0 0x7fdf1019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:12.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.430+0000 7fdf0de00700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf10106560 0x7fdf1019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:12.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.430+0000 7fdf0e800700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf1010c5a0 0x7fdf101a1730 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fdf08007ee0 tx=0x7fdf0800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:12.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.431+0000 7fdf06a00700 1 -- 172.21.15.63:0/2126182796 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800dbd0 con 0x7fdf1010c5a0 2024-05-27T08:06:12.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.431+0000 7fdf06a00700 1 -- 172.21.15.63:0/2126182796 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf0800dd70 con 0x7fdf1010c5a0 2024-05-27T08:06:12.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.431+0000 7fdf14e00700 1 -- 172.21.15.63:0/2126182796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf101a20c0 con 0x7fdf1010c5a0 2024-05-27T08:06:12.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.431+0000 7fdf06a00700 1 -- 172.21.15.63:0/2126182796 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf080058d0 con 0x7fdf1010c5a0 2024-05-27T08:06:12.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.431+0000 7fdf14e00700 1 -- 172.21.15.63:0/2126182796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf101a99e0 con 0x7fdf1010c5a0 2024-05-27T08:06:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.436+0000 7fdf06a00700 1 -- 172.21.15.63:0/2126182796 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdf0801a020 con 0x7fdf1010c5a0 2024-05-27T08:06:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.436+0000 7fdf239a4b80 1 Processor -- start 2024-05-27T08:06:12.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.437+0000 7fdf239a4b80 1 -- start start 2024-05-27T08:06:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.437+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d9b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.437+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d9bd30 0x558d74d95870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.437+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d95db0 0x558d74d961c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.437+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558d74cc3e40 con 0x558d74d9bd30 2024-05-27T08:06:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.437+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558d74cb39e0 con 0x558d74d95db0 2024-05-27T08:06:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.437+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558d74cb7a20 con 0x558d74c16e40 2024-05-27T08:06:12.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.437+0000 7fdf0de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d9bd30 0x558d74d95870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.438+0000 7fdf0de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d9bd30 0x558d74d95870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50374/0 (socket says 172.21.15.63:50374) 2024-05-27T08:06:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.438+0000 7fdf0de00700 1 -- 172.21.15.63:0/2269833356 learned_addr learned my addr 172.21.15.63:0/2269833356 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.438+0000 7fdf0d400700 1 --2- 172.21.15.63:0/2269833356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d9b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.438+0000 7fdf0d400700 1 --2- 172.21.15.63:0/2269833356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d95db0 0x558d74d961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.438+0000 7fdf0de00700 1 -- 172.21.15.63:0/2269833356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 msgr2=0x558d74d9b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.438+0000 7fdf0de00700 1 --2- 172.21.15.63:0/2269833356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d9b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.438+0000 7fdf0de00700 1 -- 172.21.15.63:0/2269833356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d95db0 msgr2=0x558d74d961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.438+0000 7fdf0de00700 1 --2- 172.21.15.63:0/2269833356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d95db0 0x558d74d961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.438+0000 7fdf0de00700 1 -- 172.21.15.63:0/2269833356 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d74c7c420 con 0x558d74d9bd30 2024-05-27T08:06:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.438+0000 7fdf0d400700 1 --2- 172.21.15.63:0/2269833356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d95db0 0x558d74d961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.439+0000 7fdf06a00700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdef40b2a70 0x7fdef40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.439+0000 7fdf0de00700 1 --2- 172.21.15.63:0/2269833356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d9bd30 0x558d74d95870 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000bd60 tx=0x7fdf0000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a84b46bf2d7fa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.439+0000 7fdf06a00700 1 -- 172.21.15.63:0/2126182796 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 9571+0+0 (secure 0 0 0) 0x7fdf080e2280 con 0x7fdf1010c5a0 2024-05-27T08:06:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.439+0000 7fdf0d400700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdef40b2a70 0x7fdef40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.439+0000 7fdefb400700 1 -- 172.21.15.63:0/2269833356 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000fe50 con 0x558d74d9bd30 2024-05-27T08:06:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.440+0000 7fdefb400700 1 -- 172.21.15.63:0/2269833356 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf000118d0 con 0x558d74d9bd30 2024-05-27T08:06:12.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.440+0000 7fdefb400700 1 -- 172.21.15.63:0/2269833356 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00010750 con 0x558d74d9bd30 2024-05-27T08:06:12.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.441+0000 7fdf0d400700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdef40b2a70 0x7fdef40b4f80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdf101055f0 tx=0x7fdefc009310 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:12.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.441+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2269833356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d9bd30 msgr2=0x558d74d95870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.441+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/2269833356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d9bd30 0x558d74d95870 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000bd60 tx=0x7fdf0000f640 comp rx=0 tx=0).stop 2024-05-27T08:06:12.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.441+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2269833356 shutdown_connections 2024-05-27T08:06:12.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.441+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/2269833356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d95db0 0x558d74d961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.441+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/2269833356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d9b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.441+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/2269833356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d9bd30 0x558d74d95870 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.441+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2269833356 >> 172.21.15.63:0/2269833356 conn(0x558d74c76fc0 msgr2=0x558d74d908d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:12.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.441+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2269833356 shutdown_connections 2024-05-27T08:06:12.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.442+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2269833356 wait complete. 2024-05-27T08:06:12.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.442+0000 7fdf239a4b80 1 -- start start 2024-05-27T08:06:12.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.442+0000 7fdf239a4b80 1 Processor -- start 2024-05-27T08:06:12.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.443+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d9f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.443+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74d9fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.443+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74d9ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.443+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558d74bb9fa0 con 0x558d74d95db0 2024-05-27T08:06:12.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.443+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558d74c54760 con 0x558d74d9bd30 2024-05-27T08:06:12.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.443+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558d74da2800 con 0x558d74c16e40 2024-05-27T08:06:12.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.443+0000 7fdf239a4b80 10 client.0 _ll_register_callbacks cb 0x558d74c5dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:12.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.444+0000 7fdf0d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74d9fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.444+0000 7fdf0d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74d9fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50382/0 (socket says 172.21.15.63:50382) 2024-05-27T08:06:12.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.444+0000 7fdf0d400700 1 -- 172.21.15.63:0/4247196551 learned_addr learned my addr 172.21.15.63:0/4247196551 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.444+0000 7fdf0e800700 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74d9ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.444+0000 7fdf0de00700 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d9f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.444+0000 7fdf0d400700 1 -- 172.21.15.63:0/4247196551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 msgr2=0x558d74d9f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:12.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.444+0000 7fdf0d400700 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d9f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.444+0000 7fdf0d400700 1 -- 172.21.15.63:0/4247196551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 msgr2=0x558d74d9ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:12.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.444+0000 7fdf0d400700 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74d9ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.444+0000 7fdf0d400700 1 -- 172.21.15.63:0/4247196551 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d74c038d0 con 0x558d74d95db0 2024-05-27T08:06:12.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.445+0000 7fdf0d400700 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74d9fa80 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009990 tx=0x7fdefc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57eca76a84154512 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:12.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.445+0000 7fdeefe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.445+0000 7fdeefe00700 1 -- 172.21.15.63:0/4247196551 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0054f0 con 0x558d74d95db0 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.445+0000 7fdeefe00700 1 -- 172.21.15.63:0/4247196551 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 9571+0+0 (secure 0 0 0) 0x7fdefc024420 con 0x558d74d95db0 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.445+0000 7fdf239a4b80 1 -- 172.21.15.63:0/4247196551 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf000118d0 con 0x558d74d95db0 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.446+0000 7fdeefe00700 1 -- 172.21.15.63:0/4247196551 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdefc038050 con 0x558d74d95db0 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.446+0000 7fdf239a4b80 10 client.21681 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.446+0000 7fdf239a4b80 1 -- 172.21.15.63:0/4247196551 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558d74c84060 con 0x558d74d95db0 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.446+0000 7fdeefe00700 1 -- 172.21.15.63:0/4247196551 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf000118d0 con 0x558d74d95db0 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.446+0000 7fdeefe00700 1 -- 172.21.15.63:0/4247196551 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fdefc010590 con 0x558d74d95db0 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.446+0000 7fdf239a4b80 10 client.21681 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.446+0000 7fdf239a4b80 10 client.21681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:12.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.447+0000 7fdf239a4b80 20 client.21681 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.447+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d74db7f00 0x558d74dba3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.447+0000 7fdf239a4b80 4 client.21681 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.447+0000 7fdf239a4b80 1 -- 172.21.15.63:0/4247196551 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d74cc5d90 con 0x558d74db7f00 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.447+0000 7fdf0de00700 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d74db7f00 0x558d74dba3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.448+0000 7fdf0de00700 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d74db7f00 0x558d74dba3b0 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec4407a51babab7 server_cookie=5c5c803b885ce909 in_seq=0 out_seq=0 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.448+0000 7fdeefe00700 10 client.21681 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.450+0000 7fdeefe00700 1 -- 172.21.15.63:0/4247196551 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558d74cc5d90 con 0x558d74db7f00 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.450+0000 7fdeefe00700 10 client.21681 handle_command_reply: tid=0 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.450+0000 7fdf239a4b80 1 client.21681 shutdown 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 1 -- 172.21.15.63:0/4247196551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 msgr2=0x558d74d9fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74d9fa80 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009990 tx=0x7fdefc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 1 -- 172.21.15.63:0/4247196551 shutdown_connections 2024-05-27T08:06:12.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d74db7f00 0x558d74dba3b0 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74d9ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d9f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/4247196551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74d9fa80 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 1 -- 172.21.15.63:0/4247196551 >> 172.21.15.63:0/4247196551 conn(0x558d74c76fc0 msgr2=0x558d74d90580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:12.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdeefe00700 0 client.21681 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:12.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 1 -- 172.21.15.63:0/4247196551 shutdown_connections 2024-05-27T08:06:12.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 1 -- 172.21.15.63:0/4247196551 wait complete. 2024-05-27T08:06:12.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.451+0000 7fdf239a4b80 20 client.21681 trim_cache size 0 max 16384 2024-05-27T08:06:12.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.474+0000 7fdf239a4b80 1 Processor -- start 2024-05-27T08:06:12.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.474+0000 7fdf239a4b80 1 -- start start 2024-05-27T08:06:12.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.474+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d95870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.474+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74da19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.474+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74d99c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.474+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101b0bf0 con 0x558d74d95db0 2024-05-27T08:06:12.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.474+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101ae480 con 0x558d74d9bd30 2024-05-27T08:06:12.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.474+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101ac310 con 0x558d74c16e40 2024-05-27T08:06:12.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.475+0000 7fdf0d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74da19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.475+0000 7fdf0d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74da19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50398/0 (socket says 172.21.15.63:50398) 2024-05-27T08:06:12.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.475+0000 7fdf0d400700 1 -- 172.21.15.63:0/690561128 learned_addr learned my addr 172.21.15.63:0/690561128 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:12.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.475+0000 7fdf0de00700 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d95870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.475+0000 7fdf0e800700 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74d99c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.475+0000 7fdf0d400700 1 -- 172.21.15.63:0/690561128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 msgr2=0x558d74d95870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:12.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.475+0000 7fdf0d400700 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d95870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.475+0000 7fdf0d400700 1 -- 172.21.15.63:0/690561128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 msgr2=0x558d74d99c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:12.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.476+0000 7fdf0d400700 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74d99c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.476+0000 7fdf0d400700 1 -- 172.21.15.63:0/690561128 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d74d9c140 con 0x558d74d95db0 2024-05-27T08:06:12.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.476+0000 7fdf0e800700 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74d99c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:12.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.476+0000 7fdf0de00700 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d95870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:12.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.476+0000 7fdf0d400700 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74da19e0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fdefc026010 tx=0x7fdefc003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9540abbf3c97087f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:12.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.476+0000 7fdefaa00700 1 -- 172.21.15.63:0/690561128 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc034020 con 0x558d74d95db0 2024-05-27T08:06:12.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.476+0000 7fdefaa00700 1 -- 172.21.15.63:0/690561128 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdefc010bd0 con 0x558d74d95db0 2024-05-27T08:06:12.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.476+0000 7fdefaa00700 1 -- 172.21.15.63:0/690561128 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc024c90 con 0x558d74d95db0 2024-05-27T08:06:12.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.477+0000 7fdf239a4b80 1 -- 172.21.15.63:0/690561128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 msgr2=0x558d74da19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.477+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74da19e0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fdefc026010 tx=0x7fdefc003e80 comp rx=0 tx=0).stop 2024-05-27T08:06:12.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.478+0000 7fdf239a4b80 1 -- 172.21.15.63:0/690561128 shutdown_connections 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.478+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74d99c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.478+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74d95870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.478+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/690561128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74d95db0 0x558d74da19e0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.478+0000 7fdf239a4b80 1 -- 172.21.15.63:0/690561128 >> 172.21.15.63:0/690561128 conn(0x558d74c76fc0 msgr2=0x558d74d9f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.478+0000 7fdf239a4b80 1 -- 172.21.15.63:0/690561128 shutdown_connections 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.478+0000 7fdf239a4b80 1 -- 172.21.15.63:0/690561128 wait complete. 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.478+0000 7fdf239a4b80 1 -- start start 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.479+0000 7fdf239a4b80 1 Processor -- start 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.479+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74db0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.479+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74db0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.479+0000 7fdf239a4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74dbf220 0x558d74dbf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.479+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101ac960 con 0x558d74dbf220 2024-05-27T08:06:12.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.479+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101addb0 con 0x558d74d9bd30 2024-05-27T08:06:12.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.479+0000 7fdf239a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101ad010 con 0x558d74c16e40 2024-05-27T08:06:12.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.480+0000 7fdf239a4b80 10 client.0 _ll_register_callbacks cb 0x558d74c5dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:12.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.480+0000 7fdf0e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74dbf220 0x558d74dbf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.480+0000 7fdf0e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74dbf220 0x558d74dbf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50400/0 (socket says 172.21.15.63:50400) 2024-05-27T08:06:12.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.480+0000 7fdf0d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74db0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.480+0000 7fdf0e800700 1 -- 172.21.15.63:0/2459541805 learned_addr learned my addr 172.21.15.63:0/2459541805 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:12.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.480+0000 7fdf0de00700 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74db0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.480+0000 7fdf0e800700 1 -- 172.21.15.63:0/2459541805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 msgr2=0x558d74db0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:12.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.481+0000 7fdf0e800700 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74db0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.481+0000 7fdf0e800700 1 -- 172.21.15.63:0/2459541805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 msgr2=0x558d74db0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:12.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.481+0000 7fdf0e800700 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74db0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.481+0000 7fdf0e800700 1 -- 172.21.15.63:0/2459541805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d74dbfe50 con 0x558d74dbf220 2024-05-27T08:06:12.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.481+0000 7fdf0de00700 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74db0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:06:12.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.481+0000 7fdf0e800700 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74dbf220 0x558d74dbf6f0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fdf080a8de0 tx=0x7fdf08004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7f9db5b69a851a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:12.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.481+0000 7fdeefe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:12.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.481+0000 7fdeefe00700 1 -- 172.21.15.63:0/2459541805 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf080a97e0 con 0x558d74dbf220 2024-05-27T08:06:12.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.481+0000 7fdeefe00700 1 -- 172.21.15.63:0/2459541805 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 9571+0+0 (secure 0 0 0) 0x7fdf080c36e0 con 0x558d74dbf220 2024-05-27T08:06:12.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.481+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2459541805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdefc010bd0 con 0x558d74dbf220 2024-05-27T08:06:12.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.482+0000 7fdeefe00700 1 -- 172.21.15.63:0/2459541805 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdf080a90e0 con 0x558d74dbf220 2024-05-27T08:06:12.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.482+0000 7fdf239a4b80 10 client.21687 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:12.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.482+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2459541805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558d74e229a0 con 0x558d74dbf220 2024-05-27T08:06:12.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.482+0000 7fdeefe00700 1 -- 172.21.15.63:0/2459541805 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdefc010bd0 con 0x558d74dbf220 2024-05-27T08:06:12.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.482+0000 7fdeefe00700 1 -- 172.21.15.63:0/2459541805 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fdf080c3260 con 0x558d74dbf220 2024-05-27T08:06:12.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.483+0000 7fdf239a4b80 10 client.21687 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.483+0000 7fdf239a4b80 10 client.21687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.483+0000 7fdf239a4b80 20 client.21687 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.483+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d74e23000 0x558d74e25420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.483+0000 7fdf239a4b80 4 client.21687 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.483+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2459541805 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d74cbf570 con 0x558d74e23000 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.483+0000 7fdf0de00700 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d74e23000 0x558d74e25420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdf0de00700 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d74e23000 0x558d74e25420 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c048fc6cee47d296 server_cookie=199f99ae93c713e0 in_seq=0 out_seq=0 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdeefe00700 10 client.21687 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdeefe00700 1 -- 172.21.15.63:0/2459541805 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d74cbf570 con 0x558d74e23000 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdeefe00700 10 client.21687 handle_command_reply: tid=0 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdf239a4b80 1 client.21687 shutdown 2024-05-27T08:06:12.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2459541805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74dbf220 msgr2=0x558d74dbf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74dbf220 0x558d74dbf6f0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fdf080a8de0 tx=0x7fdf08004d80 comp rx=0 tx=0).stop 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2459541805 shutdown_connections 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d74e23000 0x558d74e25420 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d74dbf220 0x558d74dbf6f0 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d74c16e40 0x558d74db0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdf239a4b80 1 --2- 172.21.15.63:0/2459541805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d74d9bd30 0x558d74db0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.485+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2459541805 >> 172.21.15.63:0/2459541805 conn(0x558d74c76fc0 msgr2=0x558d74d9da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.486+0000 7fdeefe00700 0 client.21687 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.486+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2459541805 shutdown_connections 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.486+0000 7fdf239a4b80 1 -- 172.21.15.63:0/2459541805 wait complete. 2024-05-27T08:06:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.486+0000 7fdf239a4b80 20 client.21687 trim_cache size 0 max 16384 2024-05-27T08:06:12.490 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:06:12.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 -- 172.21.15.63:0/2126182796 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdef40b2a70 msgr2=0x7fdef40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdef40b2a70 0x7fdef40b4f80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdf101055f0 tx=0x7fdefc009310 comp rx=0 tx=0).stop 2024-05-27T08:06:12.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 -- 172.21.15.63:0/2126182796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf1010c5a0 msgr2=0x7fdf101a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf1010c5a0 0x7fdf101a1730 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fdf08007ee0 tx=0x7fdf0800d3b0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 -- 172.21.15.63:0/2126182796 shutdown_connections 2024-05-27T08:06:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fdef40b2a70 0x7fdef40b4f80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf1010c5a0 0x7fdf101a1730 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf10106560 0x7fdf1019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 --2- 172.21.15.63:0/2126182796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf10106eb0 0x7fdf1019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 -- 172.21.15.63:0/2126182796 >> 172.21.15.63:0/2126182796 conn(0x7fdf10102040 msgr2=0x7fdf101038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 -- 172.21.15.63:0/2126182796 shutdown_connections 2024-05-27T08:06:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:12.487+0000 7fdf14e00700 1 -- 172.21.15.63:0/2126182796 wait complete. 2024-05-27T08:06:13.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:13 smithi086 ceph-mon[34687]: pgmap v1034: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-05-27T08:06:13.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:13 smithi149 ceph-mon[35408]: pgmap v1034: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-05-27T08:06:13.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:13 smithi063 ceph-mon[31613]: pgmap v1034: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-05-27T08:06:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.235+0000 7fd3ae000640 1 -- 172.21.15.63:0/1258289177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a8107e20 msgr2=0x7fd3a81082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.235+0000 7fd3ae000640 1 --2- 172.21.15.63:0/1258289177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a8107e20 0x7fd3a81082b0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009ca0 tx=0x7fd3a002a8c0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.235+0000 7fd3ae000640 1 -- 172.21.15.63:0/1258289177 shutdown_connections 2024-05-27T08:06:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.235+0000 7fd3ae000640 1 --2- 172.21.15.63:0/1258289177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a8107e20 0x7fd3a81082b0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.235+0000 7fd3ae000640 1 --2- 172.21.15.63:0/1258289177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a81073f0 0x7fd3a8107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.235+0000 7fd3ae000640 1 --2- 172.21.15.63:0/1258289177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a81029a0 0x7fd3a8102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.235+0000 7fd3ae000640 1 -- 172.21.15.63:0/1258289177 >> 172.21.15.63:0/1258289177 conn(0x7fd3a80fe210 msgr2=0x7fd3a8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.235+0000 7fd3ae000640 1 -- 172.21.15.63:0/1258289177 shutdown_connections 2024-05-27T08:06:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.236+0000 7fd3ae000640 1 -- 172.21.15.63:0/1258289177 wait complete. 2024-05-27T08:06:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.237+0000 7fd3ae000640 1 Processor -- start 2024-05-27T08:06:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.237+0000 7fd3ae000640 1 -- start start 2024-05-27T08:06:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3ae000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a81029a0 0x7fd3a81a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3ae000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a81073f0 0x7fd3a81a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3ae000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a8107e20 0x7fd3a81a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3ae000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a811a150 con 0x7fd3a81073f0 2024-05-27T08:06:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3ae000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8119fd0 con 0x7fd3a8107e20 2024-05-27T08:06:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3ae000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a811a2d0 con 0x7fd3a81029a0 2024-05-27T08:06:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a81073f0 0x7fd3a81a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a81073f0 0x7fd3a81a6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50428/0 (socket says 172.21.15.63:50428) 2024-05-27T08:06:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3660200042 learned_addr learned my addr 172.21.15.63:0/3660200042 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3ad600640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a8107e20 0x7fd3a81a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.238+0000 7fd3acc00640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a81029a0 0x7fd3a81a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.239+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3660200042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a81029a0 msgr2=0x7fd3a81a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.239+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a81029a0 0x7fd3a81a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.239+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3660200042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a8107e20 msgr2=0x7fd3a81a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.239+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a8107e20 0x7fd3a81a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.239+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3660200042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a81a0fa0 con 0x7fd3a81073f0 2024-05-27T08:06:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.239+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a81073f0 0x7fd3a81a6ce0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7fd39400c920 tx=0x7fd39400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.239+0000 7fd3a5600640 1 -- 172.21.15.63:0/3660200042 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394007c70 con 0x7fd3a81073f0 2024-05-27T08:06:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.239+0000 7fd3ae000640 1 -- 172.21.15.63:0/3660200042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a81a1290 con 0x7fd3a81073f0 2024-05-27T08:06:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.239+0000 7fd3a5600640 1 -- 172.21.15.63:0/3660200042 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd394007e10 con 0x7fd3a81073f0 2024-05-27T08:06:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.240+0000 7fd3ae000640 1 -- 172.21.15.63:0/3660200042 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a81ad620 con 0x7fd3a81073f0 2024-05-27T08:06:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.240+0000 7fd3a5600640 1 -- 172.21.15.63:0/3660200042 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394005260 con 0x7fd3a81073f0 2024-05-27T08:06:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.243+0000 7fd3b65e0740 1 Processor -- start 2024-05-27T08:06:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.243+0000 7fd3b65e0740 1 -- start start 2024-05-27T08:06:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.244+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd951b0 0x56151e137a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.244+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd6c2f0 0x56151e137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.244+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.244+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56151e0a8b90 con 0x56151dd744d0 2024-05-27T08:06:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.244+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56151e098730 con 0x56151dd6c2f0 2024-05-27T08:06:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.244+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56151e09c770 con 0x56151dd951b0 2024-05-27T08:06:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.245+0000 7fd3acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd6c2f0 0x56151e137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.245+0000 7fd3ad600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd951b0 0x56151e137a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.245+0000 7fd3a5600640 1 -- 172.21.15.63:0/3660200042 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd394005400 con 0x7fd3a81073f0 2024-05-27T08:06:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.245+0000 7fd3acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd6c2f0 0x56151e137fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:54838/0 (socket says 172.21.15.63:54838) 2024-05-27T08:06:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.245+0000 7fd3ad600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd951b0 0x56151e137a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:40616/0 (socket says 172.21.15.63:40616) 2024-05-27T08:06:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.245+0000 7fd3acc00640 1 -- 172.21.15.63:0/457171983 learned_addr learned my addr 172.21.15.63:0/457171983 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.245+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/457171983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.245+0000 7fd3acc00640 1 -- 172.21.15.63:0/457171983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd951b0 msgr2=0x56151e137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.246+0000 7fd3acc00640 1 --2- 172.21.15.63:0/457171983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd951b0 0x56151e137a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.246+0000 7fd3acc00640 1 -- 172.21.15.63:0/457171983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 msgr2=0x56151e1384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.246+0000 7fd3acc00640 1 --2- 172.21.15.63:0/457171983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.246+0000 7fd3acc00640 1 -- 172.21.15.63:0/457171983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56151e0318b0 con 0x56151dd6c2f0 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.246+0000 7fd3ad600640 1 --2- 172.21.15.63:0/457171983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd951b0 0x56151e137a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.246+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/457171983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.246+0000 7fd3acc00640 1 --2- 172.21.15.63:0/457171983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd6c2f0 0x56151e137fa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00cbb0 tx=0x7fd39c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b169c68fe931d8cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.247+0000 7fd3a5600640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3900b2950 0x7fd3900b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.247+0000 7fd399c00640 1 -- 172.21.15.63:0/457171983 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c010630 con 0x56151dd6c2f0 2024-05-27T08:06:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.247+0000 7fd3a5600640 1 -- 172.21.15.63:0/3660200042 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 9571+0+0 (secure 0 0 0) 0x7fd3940e2c60 con 0x7fd3a81073f0 2024-05-27T08:06:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.247+0000 7fd399c00640 1 -- 172.21.15.63:0/457171983 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd39c010c60 con 0x56151dd6c2f0 2024-05-27T08:06:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.247+0000 7fd3ad600640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3900b2950 0x7fd3900b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.247+0000 7fd399c00640 1 -- 172.21.15.63:0/457171983 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c01aa30 con 0x56151dd6c2f0 2024-05-27T08:06:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.248+0000 7fd3b65e0740 1 -- 172.21.15.63:0/457171983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd6c2f0 msgr2=0x56151e137fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.248+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/457171983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd6c2f0 0x56151e137fa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00cbb0 tx=0x7fd39c00e720 comp rx=0 tx=0).stop 2024-05-27T08:06:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.248+0000 7fd3ad600640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3900b2950 0x7fd3900b4e60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x56151e134bd0 tx=0x7fd3a0009380 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.248+0000 7fd3b65e0740 1 -- 172.21.15.63:0/457171983 shutdown_connections 2024-05-27T08:06:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.248+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/457171983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.248+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/457171983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd6c2f0 0x56151e137fa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.248+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/457171983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd951b0 0x56151e137a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.248+0000 7fd3b65e0740 1 -- 172.21.15.63:0/457171983 >> 172.21.15.63:0/457171983 conn(0x56151dd89300 msgr2=0x56151e134210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.249+0000 7fd3b65e0740 1 -- 172.21.15.63:0/457171983 shutdown_connections 2024-05-27T08:06:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.249+0000 7fd3b65e0740 1 -- 172.21.15.63:0/457171983 wait complete. 2024-05-27T08:06:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.249+0000 7fd3b65e0740 1 -- start start 2024-05-27T08:06:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.250+0000 7fd3b65e0740 1 Processor -- start 2024-05-27T08:06:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.250+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e141c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.250+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1421b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.250+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd951b0 0x56151e14dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.250+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56151dd7e790 con 0x56151dd744d0 2024-05-27T08:06:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.250+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56151e040bc0 con 0x56151dd951b0 2024-05-27T08:06:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.250+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56151e144b90 con 0x56151dd6c2f0 2024-05-27T08:06:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.250+0000 7fd3b65e0740 10 client.0 _ll_register_callbacks cb 0x56151e01e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.251+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.251+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50446/0 (socket says 172.21.15.63:50446) 2024-05-27T08:06:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.251+0000 7fd3a7e00640 1 -- 172.21.15.63:0/2876809302 learned_addr learned my addr 172.21.15.63:0/2876809302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.251+0000 7fd3acc00640 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.251+0000 7fd3ad600640 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd951b0 0x56151e14dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.251+0000 7fd3a7e00640 1 -- 172.21.15.63:0/2876809302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 msgr2=0x56151e141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.251+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.251+0000 7fd3a7e00640 1 -- 172.21.15.63:0/2876809302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd951b0 msgr2=0x56151e14dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.251+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd951b0 0x56151e14dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.251+0000 7fd3a7e00640 1 -- 172.21.15.63:0/2876809302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151ddad210 con 0x56151dd744d0 2024-05-27T08:06:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.252+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1421b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fd3940a6d90 tx=0x7fd39400fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b18fd0b5eb6c7be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.252+0000 7fd38e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.252+0000 7fd38e000640 1 -- 172.21.15.63:0/2876809302 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3940a52b0 con 0x56151dd744d0 2024-05-27T08:06:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.252+0000 7fd38e000640 1 -- 172.21.15.63:0/2876809302 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 9571+0+0 (secure 0 0 0) 0x7fd3940b24f0 con 0x56151dd744d0 2024-05-27T08:06:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.252+0000 7fd3b65e0740 1 -- 172.21.15.63:0/2876809302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd39c010c60 con 0x56151dd744d0 2024-05-27T08:06:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.252+0000 7fd38e000640 1 -- 172.21.15.63:0/2876809302 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd3940e0420 con 0x56151dd744d0 2024-05-27T08:06:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.252+0000 7fd3b65e0740 10 client.21699 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.252+0000 7fd3b65e0740 1 -- 172.21.15.63:0/2876809302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56151de79620 con 0x56151dd744d0 2024-05-27T08:06:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.253+0000 7fd38e000640 1 -- 172.21.15.63:0/2876809302 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd39c010c60 con 0x56151dd744d0 2024-05-27T08:06:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.253+0000 7fd38e000640 1 -- 172.21.15.63:0/2876809302 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd3940a5490 con 0x56151dd744d0 2024-05-27T08:06:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.253+0000 7fd3b65e0740 10 client.21699 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.253+0000 7fd3b65e0740 10 client.21699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.253+0000 7fd3b65e0740 20 client.21699 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.253+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56151dd53d80 0x56151e153f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.253+0000 7fd3b65e0740 4 client.21699 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.253+0000 7fd3b65e0740 1 -- 172.21.15.63:0/2876809302 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56151e0aaae0 con 0x56151dd53d80 2024-05-27T08:06:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.254+0000 7fd3acc00640 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56151dd53d80 0x56151e153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.255+0000 7fd3acc00640 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56151dd53d80 0x56151e153f20 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6deead2d699e10a9 server_cookie=e0de157aa5339dc in_seq=0 out_seq=0 2024-05-27T08:06:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.255+0000 7fd38e000640 10 client.21699 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd38e000640 1 -- 172.21.15.63:0/2876809302 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56151e0aaae0 con 0x56151dd53d80 2024-05-27T08:06:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd38e000640 10 client.21699 handle_command_reply: tid=0 2024-05-27T08:06:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 client.21699 shutdown 2024-05-27T08:06:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 -- 172.21.15.63:0/2876809302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 msgr2=0x56151e1421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1421b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fd3940a6d90 tx=0x7fd39400fca0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 -- 172.21.15.63:0/2876809302 shutdown_connections 2024-05-27T08:06:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56151dd53d80 0x56151e153f20 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd951b0 0x56151e14dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1421b0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd38e000640 0 client.21699 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/2876809302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 -- 172.21.15.63:0/2876809302 >> 172.21.15.63:0/2876809302 conn(0x56151dd89300 msgr2=0x56151e137890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 -- 172.21.15.63:0/2876809302 shutdown_connections 2024-05-27T08:06:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.257+0000 7fd3b65e0740 1 -- 172.21.15.63:0/2876809302 wait complete. 2024-05-27T08:06:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.258+0000 7fd3b65e0740 20 client.21699 trim_cache size 0 max 16384 2024-05-27T08:06:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.279+0000 7fd3b65e0740 1 Processor -- start 2024-05-27T08:06:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.279+0000 7fd3b65e0740 1 -- start start 2024-05-27T08:06:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.280+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.280+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.280+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd951b0 0x56151e159c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.280+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b9f70 con 0x56151dd744d0 2024-05-27T08:06:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.280+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b7840 con 0x56151dd951b0 2024-05-27T08:06:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.280+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b56b0 con 0x56151dd6c2f0 2024-05-27T08:06:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.280+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.280+0000 7fd3a7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50456/0 (socket says 172.21.15.63:50456) 2024-05-27T08:06:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.281+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3096013204 learned_addr learned my addr 172.21.15.63:0/3096013204 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.281+0000 7fd3acc00640 1 --2- 172.21.15.63:0/3096013204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.281+0000 7fd3ad600640 1 --2- 172.21.15.63:0/3096013204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd951b0 0x56151e159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.281+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3096013204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 msgr2=0x56151e143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.281+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3096013204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.281+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3096013204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd951b0 msgr2=0x56151e159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.281+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3096013204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd951b0 0x56151e159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.281+0000 7fd3a7e00640 1 -- 172.21.15.63:0/3096013204 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56151e1401c0 con 0x56151dd744d0 2024-05-27T08:06:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.282+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/3096013204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1596d0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7fd3940a60c0 tx=0x7fd3940e6390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3df84d3ff97b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.282+0000 7fd399200640 1 -- 172.21.15.63:0/3096013204 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3940aded0 con 0x56151dd744d0 2024-05-27T08:06:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.283+0000 7fd399200640 1 -- 172.21.15.63:0/3096013204 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3940e5050 con 0x56151dd744d0 2024-05-27T08:06:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.283+0000 7fd399200640 1 -- 172.21.15.63:0/3096013204 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3940a9470 con 0x56151dd744d0 2024-05-27T08:06:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.284+0000 7fd3b65e0740 1 -- 172.21.15.63:0/3096013204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 msgr2=0x56151e1596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.284+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/3096013204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1596d0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7fd3940a60c0 tx=0x7fd3940e6390 comp rx=0 tx=0).stop 2024-05-27T08:06:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.284+0000 7fd3b65e0740 1 -- 172.21.15.63:0/3096013204 shutdown_connections 2024-05-27T08:06:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.284+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/3096013204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd951b0 0x56151e159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.284+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/3096013204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd744d0 0x56151e1596d0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.284+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/3096013204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.284+0000 7fd3b65e0740 1 -- 172.21.15.63:0/3096013204 >> 172.21.15.63:0/3096013204 conn(0x56151dd89300 msgr2=0x56151e141fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.284+0000 7fd3b65e0740 1 -- 172.21.15.63:0/3096013204 shutdown_connections 2024-05-27T08:06:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.284+0000 7fd3b65e0740 1 -- 172.21.15.63:0/3096013204 wait complete. 2024-05-27T08:06:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.284+0000 7fd3b65e0740 1 -- start start 2024-05-27T08:06:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.285+0000 7fd3b65e0740 1 Processor -- start 2024-05-27T08:06:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.285+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e147b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.285+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd744d0 0x56151e148080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.285+0000 7fd3b65e0740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd951b0 0x56151e1485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.285+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b5d00 con 0x56151dd951b0 2024-05-27T08:06:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.285+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b7170 con 0x56151dd744d0 2024-05-27T08:06:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.285+0000 7fd3b65e0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a81b63d0 con 0x56151dd6c2f0 2024-05-27T08:06:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.285+0000 7fd3b65e0740 10 client.0 _ll_register_callbacks cb 0x7fd3940e6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.286+0000 7fd3ad600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd951b0 0x56151e1485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.286+0000 7fd3ad600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd951b0 0x56151e1485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50464/0 (socket says 172.21.15.63:50464) 2024-05-27T08:06:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.286+0000 7fd3ad600640 1 -- 172.21.15.63:0/1038640265 learned_addr learned my addr 172.21.15.63:0/1038640265 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.286+0000 7fd3a7e00640 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd744d0 0x56151e148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.286+0000 7fd3acc00640 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.286+0000 7fd3ad600640 1 -- 172.21.15.63:0/1038640265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 msgr2=0x56151e147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.286+0000 7fd3ad600640 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e147b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.286+0000 7fd3ad600640 1 -- 172.21.15.63:0/1038640265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd744d0 msgr2=0x56151e148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.286+0000 7fd3ad600640 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd744d0 0x56151e148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.286+0000 7fd3ad600640 1 -- 172.21.15.63:0/1038640265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151e1bb5b0 con 0x56151dd951b0 2024-05-27T08:06:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.287+0000 7fd3ad600640 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd951b0 0x56151e1485c0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009ca0 tx=0x7fd3a00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb58308828a180 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.287+0000 7fd38e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.287+0000 7fd38e000640 1 -- 172.21.15.63:0/1038640265 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0059060 con 0x56151dd951b0 2024-05-27T08:06:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.287+0000 7fd38e000640 1 -- 172.21.15.63:0/1038640265 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 9571+0+0 (secure 0 0 0) 0x7fd3a0004250 con 0x56151dd951b0 2024-05-27T08:06:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.287+0000 7fd3b65e0740 1 -- 172.21.15.63:0/1038640265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3940e5050 con 0x56151dd951b0 2024-05-27T08:06:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.288+0000 7fd38e000640 1 -- 172.21.15.63:0/1038640265 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd3a0060050 con 0x56151dd951b0 2024-05-27T08:06:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.288+0000 7fd3b65e0740 10 client.21705 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.288+0000 7fd3b65e0740 1 -- 172.21.15.63:0/1038640265 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56151e1bc740 con 0x56151dd951b0 2024-05-27T08:06:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.288+0000 7fd38e000640 1 -- 172.21.15.63:0/1038640265 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3940e5050 con 0x56151dd951b0 2024-05-27T08:06:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.289+0000 7fd38e000640 1 -- 172.21.15.63:0/1038640265 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd3a0030c60 con 0x56151dd951b0 2024-05-27T08:06:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.289+0000 7fd3b65e0740 10 client.21705 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.289+0000 7fd3b65e0740 10 client.21705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.289+0000 7fd3b65e0740 20 client.21705 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.289+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56151dd291c0 0x56151e1bef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.289+0000 7fd3b65e0740 4 client.21705 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:06:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.289+0000 7fd3b65e0740 1 -- 172.21.15.63:0/1038640265 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56151e0a42c0 con 0x56151dd291c0 2024-05-27T08:06:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.289+0000 7fd3acc00640 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56151dd291c0 0x56151e1bef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.291+0000 7fd38e000640 10 client.21705 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.290+0000 7fd3acc00640 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56151dd291c0 0x56151e1bef30 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49244676b48c5360 server_cookie=1db1b76c063ba954 in_seq=0 out_seq=0 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.331+0000 7fd38e000640 1 -- 172.21.15.63:0/1038640265 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56151e0a42c0 con 0x56151dd291c0 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.331+0000 7fd38e000640 10 client.21705 handle_command_reply: tid=0 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.331+0000 7fd3b65e0740 1 client.21705 shutdown 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.331+0000 7fd3b65e0740 1 -- 172.21.15.63:0/1038640265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd951b0 msgr2=0x56151e1485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.331+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd951b0 0x56151e1485c0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0009ca0 tx=0x7fd3a00095c0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.332+0000 7fd3b65e0740 1 -- 172.21.15.63:0/1038640265 shutdown_connections 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.332+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56151dd291c0 0x56151e1bef30 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.332+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56151dd951b0 0x56151e1485c0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.332+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56151dd744d0 0x56151e148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.332+0000 7fd38e000640 0 client.21705 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.332+0000 7fd3b65e0740 1 --2- 172.21.15.63:0/1038640265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56151dd6c2f0 0x56151e147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.332+0000 7fd3b65e0740 1 -- 172.21.15.63:0/1038640265 >> 172.21.15.63:0/1038640265 conn(0x56151dd89300 msgr2=0x56151e143d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.332+0000 7fd3b65e0740 1 -- 172.21.15.63:0/1038640265 shutdown_connections 2024-05-27T08:06:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.332+0000 7fd3b65e0740 1 -- 172.21.15.63:0/1038640265 wait complete. 2024-05-27T08:06:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.332+0000 7fd3b65e0740 20 client.21705 trim_cache size 0 max 16384 2024-05-27T08:06:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:06:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:06:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:06:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 -- 172.21.15.63:0/3660200042 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3900b2950 msgr2=0x7fd3900b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3900b2950 0x7fd3900b4e60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x56151e134bd0 tx=0x7fd3a0009380 comp rx=0 tx=0).stop 2024-05-27T08:06:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 -- 172.21.15.63:0/3660200042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a81073f0 msgr2=0x7fd3a81a6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a81073f0 0x7fd3a81a6ce0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7fd39400c920 tx=0x7fd39400cde0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 -- 172.21.15.63:0/3660200042 shutdown_connections 2024-05-27T08:06:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3900b2950 0x7fd3900b4e60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3a8107e20 0x7fd3a81a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3a81073f0 0x7fd3a81a6ce0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 --2- 172.21.15.63:0/3660200042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3a81029a0 0x7fd3a81a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 -- 172.21.15.63:0/3660200042 >> 172.21.15.63:0/3660200042 conn(0x7fd3a80fe210 msgr2=0x7fd3a8104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 -- 172.21.15.63:0/3660200042 shutdown_connections 2024-05-27T08:06:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:14.333+0000 7fd3ae000640 1 -- 172.21.15.63:0/3660200042 wait complete. 2024-05-27T08:06:14.641 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:06:14.641 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:06:15.186 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:15.963 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:15 smithi086 ceph-mon[34687]: pgmap v1035: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-05-27T08:06:15.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:15 smithi149 ceph-mon[35408]: pgmap v1035: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-05-27T08:06:15.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:15 smithi063 ceph-mon[31613]: pgmap v1035: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 8 op/s 2024-05-27T08:06:17.312 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:17 smithi063 ceph-mon[31613]: osdmap e146: 12 total, 12 up, 12 in 2024-05-27T08:06:17.313 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:17 smithi063 ceph-mon[31613]: pgmap v1037: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.2 MiB/s wr, 13 op/s 2024-05-27T08:06:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:17 smithi086 ceph-mon[34687]: osdmap e146: 12 total, 12 up, 12 in 2024-05-27T08:06:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:17 smithi086 ceph-mon[34687]: pgmap v1037: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.2 MiB/s wr, 13 op/s 2024-05-27T08:06:17.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:17 smithi149 ceph-mon[35408]: osdmap e146: 12 total, 12 up, 12 in 2024-05-27T08:06:17.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:17 smithi149 ceph-mon[35408]: pgmap v1037: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.2 MiB/s wr, 13 op/s 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.942+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3255475402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb4109860 msgr2=0x7fbbb41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.942+0000 7fbbb8a00700 1 --2- 172.21.15.63:0/3255475402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb4109860 0x7fbbb41111c0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00b0d0 tx=0x7fbbac00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.943+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3255475402 shutdown_connections 2024-05-27T08:06:17.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.943+0000 7fbbb8a00700 1 --2- 172.21.15.63:0/3255475402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb4109860 0x7fbbb41111c0 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.943+0000 7fbbb8a00700 1 --2- 172.21.15.63:0/3255475402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb41045e0 0x7fbbb41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.943+0000 7fbbb8a00700 1 --2- 172.21.15.63:0/3255475402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4108f10 0x7fbbb4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.943+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3255475402 >> 172.21.15.63:0/3255475402 conn(0x7fbbb4100030 msgr2=0x7fbbb4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:17.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.943+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3255475402 shutdown_connections 2024-05-27T08:06:17.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.943+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3255475402 wait complete. 2024-05-27T08:06:17.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.945+0000 7fbbb8a00700 1 Processor -- start 2024-05-27T08:06:17.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.945+0000 7fbbb8a00700 1 -- start start 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.945+0000 7fbbb8a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb41045e0 0x7fbbb419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.945+0000 7fbbb8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4108f10 0x7fbbb419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.945+0000 7fbbb8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb4109860 0x7fbbb41a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.945+0000 7fbbb8a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4114340 con 0x7fbbb4109860 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.945+0000 7fbbb8a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41141c0 con 0x7fbbb4108f10 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.945+0000 7fbbb8a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41144c0 con 0x7fbbb41045e0 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb4109860 0x7fbbb41a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb4109860 0x7fbbb41a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50476/0 (socket says 172.21.15.63:50476) 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb2800700 1 -- 172.21.15.63:0/3681982636 learned_addr learned my addr 172.21.15.63:0/3681982636 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb1400700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4108f10 0x7fbbb419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb41045e0 0x7fbbb419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb2800700 1 -- 172.21.15.63:0/3681982636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb41045e0 msgr2=0x7fbbb419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb2800700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb41045e0 0x7fbbb419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb2800700 1 -- 172.21.15.63:0/3681982636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4108f10 msgr2=0x7fbbb419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb2800700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4108f10 0x7fbbb419d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb2800700 1 -- 172.21.15.63:0/3681982636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb41a1d10 con 0x7fbbb4109860 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.946+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb41045e0 0x7fbbb419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.947+0000 7fbbb2800700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb4109860 0x7fbbb41a1720 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009850 tx=0x7fbbac007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.947+0000 7fbbaaa00700 1 -- 172.21.15.63:0/3681982636 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac01f070 con 0x7fbbb4109860 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.947+0000 7fbbaaa00700 1 -- 172.21.15.63:0/3681982636 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbbac005500 con 0x7fbbb4109860 2024-05-27T08:06:17.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.947+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3681982636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb41a1fa0 con 0x7fbbb4109860 2024-05-27T08:06:17.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.947+0000 7fbbaaa00700 1 -- 172.21.15.63:0/3681982636 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac01b9a0 con 0x7fbbb4109860 2024-05-27T08:06:17.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.948+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3681982636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb41a24d0 con 0x7fbbb4109860 2024-05-27T08:06:17.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.952+0000 7fbbaaa00700 1 -- 172.21.15.63:0/3681982636 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbbac022070 con 0x7fbbb4109860 2024-05-27T08:06:17.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.953+0000 7fbbc759fb80 1 Processor -- start 2024-05-27T08:06:17.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.953+0000 7fbbc759fb80 1 -- start start 2024-05-27T08:06:17.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.953+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8960a6ec0 0x55d89622b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.953+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 0x55d8962258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.953+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d896225e30 0x55d896226240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.953+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d896153ec0 con 0x55d8960a6ec0 2024-05-27T08:06:17.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.953+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d896143a60 con 0x55d89622bdb0 2024-05-27T08:06:17.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.954+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d896147aa0 con 0x55d896225e30 2024-05-27T08:06:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.954+0000 7fbbb1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8960a6ec0 0x55d89622b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.954+0000 7fbbb1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8960a6ec0 0x55d89622b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50488/0 (socket says 172.21.15.63:50488) 2024-05-27T08:06:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.954+0000 7fbbb1400700 1 -- 172.21.15.63:0/3508851940 learned_addr learned my addr 172.21.15.63:0/3508851940 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.954+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/3508851940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 0x55d8962258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.955+0000 7fbbb1400700 1 -- 172.21.15.63:0/3508851940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d896225e30 msgr2=0x55d896226240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.955+0000 7fbbb1400700 1 --2- 172.21.15.63:0/3508851940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d896225e30 0x55d896226240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.955+0000 7fbbb1400700 1 -- 172.21.15.63:0/3508851940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 msgr2=0x55d8962258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:17.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.955+0000 7fbbb1400700 1 --2- 172.21.15.63:0/3508851940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 0x55d8962258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.955+0000 7fbbb1400700 1 -- 172.21.15.63:0/3508851940 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89610c4a0 con 0x55d8960a6ec0 2024-05-27T08:06:17.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.955+0000 7fbbaaa00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbb980b2a70 0x7fbb980b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.955+0000 7fbbb1400700 1 --2- 172.21.15.63:0/3508851940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8960a6ec0 0x55d89622b870 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fbba000ad60 tx=0x7fbba000f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cdc252b5ed6fbf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:17.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.955+0000 7fbbaaa00700 1 -- 172.21.15.63:0/3681982636 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7fbbac00f020 con 0x7fbbb4109860 2024-05-27T08:06:17.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.955+0000 7fbb9f400700 1 -- 172.21.15.63:0/3508851940 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0004230 con 0x55d8960a6ec0 2024-05-27T08:06:17.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.956+0000 7fbb9f400700 1 -- 172.21.15.63:0/3508851940 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbba000fe10 con 0x55d8960a6ec0 2024-05-27T08:06:17.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.956+0000 7fbb9f400700 1 -- 172.21.15.63:0/3508851940 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0010630 con 0x55d8960a6ec0 2024-05-27T08:06:17.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.956+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbb980b2a70 0x7fbb980b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.957+0000 7fbbc759fb80 1 -- 172.21.15.63:0/3508851940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8960a6ec0 msgr2=0x55d89622b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:17.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.957+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbb980b2a70 0x7fbb980b4f80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbba400be10 tx=0x7fbba400b5c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:17.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.957+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/3508851940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8960a6ec0 0x55d89622b870 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fbba000ad60 tx=0x7fbba000f6a0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.957+0000 7fbbc759fb80 1 -- 172.21.15.63:0/3508851940 shutdown_connections 2024-05-27T08:06:17.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.957+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/3508851940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d896225e30 0x55d896226240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.957+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/3508851940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8960a6ec0 0x55d89622b870 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.957+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/3508851940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 0x55d8962258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.957+0000 7fbbc759fb80 1 -- 172.21.15.63:0/3508851940 >> 172.21.15.63:0/3508851940 conn(0x55d896107040 msgr2=0x55d896220950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:17.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.957+0000 7fbbc759fb80 1 -- 172.21.15.63:0/3508851940 shutdown_connections 2024-05-27T08:06:17.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.957+0000 7fbbc759fb80 1 -- 172.21.15.63:0/3508851940 wait complete. 2024-05-27T08:06:17.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.958+0000 7fbbc759fb80 1 -- start start 2024-05-27T08:06:17.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.958+0000 7fbbc759fb80 1 Processor -- start 2024-05-27T08:06:17.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8960a6ec0 0x55d89622f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89622fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 0x55d89622ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d89604a020 con 0x55d896225e30 2024-05-27T08:06:17.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8960e47e0 con 0x55d89622bdb0 2024-05-27T08:06:17.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d896232880 con 0x55d8960a6ec0 2024-05-27T08:06:17.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbc759fb80 10 client.0 _ll_register_callbacks cb 0x55d8960edcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:17.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbb1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89622fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbb1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89622fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50504/0 (socket says 172.21.15.63:50504) 2024-05-27T08:06:17.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbb1400700 1 -- 172.21.15.63:0/267867375 learned_addr learned my addr 172.21.15.63:0/267867375 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:17.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.959+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8960a6ec0 0x55d89622f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.960+0000 7fbbb2800700 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 0x55d89622ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.960+0000 7fbbb1400700 1 -- 172.21.15.63:0/267867375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8960a6ec0 msgr2=0x55d89622f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:17.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.960+0000 7fbbb1400700 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8960a6ec0 0x55d89622f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.960+0000 7fbbb1400700 1 -- 172.21.15.63:0/267867375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 msgr2=0x55d89622ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.960+0000 7fbbb1400700 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 0x55d89622ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.960+0000 7fbbb1400700 1 -- 172.21.15.63:0/267867375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d896093950 con 0x55d896225e30 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.960+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8960a6ec0 0x55d89622f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.960+0000 7fbbb2800700 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 0x55d89622ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.960+0000 7fbbb1400700 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89622fa90 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fbba000a6c0 tx=0x7fbba0003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bed1fb109e859f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.961+0000 7fbb93e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.961+0000 7fbb93e00700 1 -- 172.21.15.63:0/267867375 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0026020 con 0x55d896225e30 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.961+0000 7fbb93e00700 1 -- 172.21.15.63:0/267867375 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7fbba00048a0 con 0x55d896225e30 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.961+0000 7fbbc759fb80 1 -- 172.21.15.63:0/267867375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbba000fe10 con 0x55d896225e30 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.961+0000 7fbb93e00700 1 -- 172.21.15.63:0/267867375 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbba0031050 con 0x55d896225e30 2024-05-27T08:06:17.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.961+0000 7fbbc759fb80 10 client.21717 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:17.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.962+0000 7fbbc759fb80 1 -- 172.21.15.63:0/267867375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d8961140e0 con 0x55d896225e30 2024-05-27T08:06:17.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.962+0000 7fbb93e00700 1 -- 172.21.15.63:0/267867375 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbba000fe10 con 0x55d896225e30 2024-05-27T08:06:17.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.962+0000 7fbb93e00700 1 -- 172.21.15.63:0/267867375 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbba0004190 con 0x55d896225e30 2024-05-27T08:06:17.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.962+0000 7fbbc759fb80 10 client.21717 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:17.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.962+0000 7fbbc759fb80 10 client.21717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:17.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.962+0000 7fbbc759fb80 20 client.21717 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:17.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.963+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d89623f680 0x55d896241aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.963+0000 7fbbc759fb80 4 client.21717 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:17.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.963+0000 7fbbc759fb80 1 -- 172.21.15.63:0/267867375 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d896155e10 con 0x55d89623f680 2024-05-27T08:06:17.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.963+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d89623f680 0x55d896241aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.964+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d89623f680 0x55d896241aa0 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436e93d1939c3ad3 server_cookie=77e0c6b3aca94f3e in_seq=0 out_seq=0 2024-05-27T08:06:17.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.964+0000 7fbb93e00700 10 client.21717 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.966+0000 7fbb93e00700 1 -- 172.21.15.63:0/267867375 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d896155e10 con 0x55d89623f680 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.966+0000 7fbb93e00700 10 client.21717 handle_command_reply: tid=0 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 client.21717 shutdown 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 -- 172.21.15.63:0/267867375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 msgr2=0x55d89622fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89622fa90 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fbba000a6c0 tx=0x7fbba0003b80 comp rx=0 tx=0).stop 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 -- 172.21.15.63:0/267867375 shutdown_connections 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d89623f680 0x55d896241aa0 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d89622bdb0 0x55d89622ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8960a6ec0 0x55d89622f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/267867375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89622fa90 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 -- 172.21.15.63:0/267867375 >> 172.21.15.63:0/267867375 conn(0x55d896107040 msgr2=0x55d896220600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:17.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbb93e00700 0 client.21717 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:17.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 -- 172.21.15.63:0/267867375 shutdown_connections 2024-05-27T08:06:17.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.967+0000 7fbbc759fb80 1 -- 172.21.15.63:0/267867375 wait complete. 2024-05-27T08:06:17.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.968+0000 7fbbc759fb80 20 client.21717 trim_cache size 0 max 16384 2024-05-27T08:06:17.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.991+0000 7fbbc759fb80 1 Processor -- start 2024-05-27T08:06:17.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.991+0000 7fbbc759fb80 1 -- start start 2024-05-27T08:06:17.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.992+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8960a6ec0 0x55d8962258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.992+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89623bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.992+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89622bdb0 0x55d896229900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.992+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b4e30 con 0x55d896225e30 2024-05-27T08:06:17.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.992+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b26c0 con 0x55d8960a6ec0 2024-05-27T08:06:17.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.992+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b0530 con 0x55d89622bdb0 2024-05-27T08:06:17.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.992+0000 7fbbb1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89623bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.992+0000 7fbbb1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89623bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50512/0 (socket says 172.21.15.63:50512) 2024-05-27T08:06:17.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.992+0000 7fbbb1400700 1 -- 172.21.15.63:0/3037371203 learned_addr learned my addr 172.21.15.63:0/3037371203 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.993+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/3037371203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8960a6ec0 0x55d8962258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.993+0000 7fbbb2800700 1 --2- 172.21.15.63:0/3037371203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89622bdb0 0x55d896229900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.993+0000 7fbbb1400700 1 -- 172.21.15.63:0/3037371203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89622bdb0 msgr2=0x55d896229900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.993+0000 7fbbb1400700 1 --2- 172.21.15.63:0/3037371203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89622bdb0 0x55d896229900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.993+0000 7fbbb1400700 1 -- 172.21.15.63:0/3037371203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8960a6ec0 msgr2=0x55d8962258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.993+0000 7fbbb1400700 1 --2- 172.21.15.63:0/3037371203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8960a6ec0 0x55d8962258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.993+0000 7fbbb1400700 1 -- 172.21.15.63:0/3037371203 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d896226b90 con 0x55d896225e30 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.993+0000 7fbbb1400700 1 --2- 172.21.15.63:0/3037371203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89623bb20 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fbba00045e0 tx=0x7fbba000a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff61876abacf79 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.993+0000 7fbb9ea00700 1 -- 172.21.15.63:0/3037371203 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba003a070 con 0x55d896225e30 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.994+0000 7fbb9ea00700 1 -- 172.21.15.63:0/3037371203 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbba00225d0 con 0x55d896225e30 2024-05-27T08:06:17.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.994+0000 7fbb9ea00700 1 -- 172.21.15.63:0/3037371203 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0026070 con 0x55d896225e30 2024-05-27T08:06:17.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 -- 172.21.15.63:0/3037371203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 msgr2=0x55d89623bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:17.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/3037371203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89623bb20 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fbba00045e0 tx=0x7fbba000a4b0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 -- 172.21.15.63:0/3037371203 shutdown_connections 2024-05-27T08:06:17.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/3037371203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89622bdb0 0x55d896229900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/3037371203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8960a6ec0 0x55d8962258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/3037371203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d896225e30 0x55d89623bb20 secure :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fbba00045e0 tx=0x7fbba000a4b0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 -- 172.21.15.63:0/3037371203 >> 172.21.15.63:0/3037371203 conn(0x55d896107040 msgr2=0x55d89622fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:17.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 -- 172.21.15.63:0/3037371203 shutdown_connections 2024-05-27T08:06:17.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 -- 172.21.15.63:0/3037371203 wait complete. 2024-05-27T08:06:17.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 -- start start 2024-05-27T08:06:17.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.995+0000 7fbbc759fb80 1 Processor -- start 2024-05-27T08:06:17.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.996+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8960a6ec0 0x55d896242b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.996+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89622bdb0 0x55d896243040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.996+0000 7fbbc759fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8962a6ee0 0x55d8962a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.996+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b0b80 con 0x55d8962a6ee0 2024-05-27T08:06:17.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.996+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b1ff0 con 0x55d8960a6ec0 2024-05-27T08:06:17.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.996+0000 7fbbc759fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b1250 con 0x55d89622bdb0 2024-05-27T08:06:17.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.996+0000 7fbbc759fb80 10 client.0 _ll_register_callbacks cb 0x55d8960edcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:17.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.997+0000 7fbbb2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8962a6ee0 0x55d8962a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.997+0000 7fbbb2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8962a6ee0 0x55d8962a9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50514/0 (socket says 172.21.15.63:50514) 2024-05-27T08:06:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.997+0000 7fbbb2800700 1 -- 172.21.15.63:0/2961477704 learned_addr learned my addr 172.21.15.63:0/2961477704 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.997+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8960a6ec0 0x55d896242b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.997+0000 7fbbb1400700 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89622bdb0 0x55d896243040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.997+0000 7fbbb2800700 1 -- 172.21.15.63:0/2961477704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89622bdb0 msgr2=0x55d896243040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.997+0000 7fbbb2800700 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89622bdb0 0x55d896243040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.997+0000 7fbbb2800700 1 -- 172.21.15.63:0/2961477704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8960a6ec0 msgr2=0x55d896242b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.997+0000 7fbbb2800700 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8960a6ec0 0x55d896242b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.997+0000 7fbbb2800700 1 -- 172.21.15.63:0/2961477704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8962a9840 con 0x55d8962a6ee0 2024-05-27T08:06:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.998+0000 7fbbb2800700 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8962a6ee0 0x55d8962a9300 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0a80a0 tx=0x7fbbac0aa5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abacee4b1e00ecd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.998+0000 7fbb93e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.998+0000 7fbb93e00700 1 -- 172.21.15.63:0/2961477704 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0aae30 con 0x55d8962a6ee0 2024-05-27T08:06:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.998+0000 7fbb93e00700 1 -- 172.21.15.63:0/2961477704 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7fbbac0ab2a0 con 0x55d8962a6ee0 2024-05-27T08:06:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.998+0000 7fbbc759fb80 1 -- 172.21.15.63:0/2961477704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbba00225d0 con 0x55d8962a6ee0 2024-05-27T08:06:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.998+0000 7fbb93e00700 1 -- 172.21.15.63:0/2961477704 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbbac0af1e0 con 0x55d8962a6ee0 2024-05-27T08:06:18.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.998+0000 7fbbc759fb80 10 client.21723 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:18.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.998+0000 7fbbc759fb80 1 -- 172.21.15.63:0/2961477704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d8962aaa10 con 0x55d8962a6ee0 2024-05-27T08:06:18.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.999+0000 7fbb93e00700 1 -- 172.21.15.63:0/2961477704 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbba00225d0 con 0x55d8962a6ee0 2024-05-27T08:06:18.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.999+0000 7fbb93e00700 1 -- 172.21.15.63:0/2961477704 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbbac0c3020 con 0x55d8962a6ee0 2024-05-27T08:06:18.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.999+0000 7fbbc759fb80 10 client.21723 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:18.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.999+0000 7fbbc759fb80 10 client.21723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:18.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.999+0000 7fbbc759fb80 20 client.21723 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:18.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.999+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d8962ab0c0 0x55d8962ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:18.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.999+0000 7fbbc759fb80 4 client.21723 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:06:18.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:17.999+0000 7fbbc759fb80 1 -- 172.21.15.63:0/2961477704 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d89614f5f0 con 0x55d8962ab0c0 2024-05-27T08:06:18.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.000+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d8962ab0c0 0x55d8962ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:18.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.001+0000 7fbbb1e00700 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d8962ab0c0 0x55d8962ad4e0 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fffe64fb2d39bbec server_cookie=ba1528f39e157e99 in_seq=0 out_seq=0 2024-05-27T08:06:18.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.001+0000 7fbb93e00700 10 client.21723 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:18.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.004+0000 7fbb93e00700 1 -- 172.21.15.63:0/2961477704 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d89614f5f0 con 0x55d8962ab0c0 2024-05-27T08:06:18.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.004+0000 7fbb93e00700 10 client.21723 handle_command_reply: tid=0 2024-05-27T08:06:18.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.004+0000 7fbbc759fb80 1 client.21723 shutdown 2024-05-27T08:06:18.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 1 -- 172.21.15.63:0/2961477704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8962a6ee0 msgr2=0x55d8962a9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:18.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8962a6ee0 0x55d8962a9300 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0a80a0 tx=0x7fbbac0aa5a0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 1 -- 172.21.15.63:0/2961477704 shutdown_connections 2024-05-27T08:06:18.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d8962ab0c0 0x55d8962ad4e0 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d8962a6ee0 0x55d8962a9300 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8960a6ec0 0x55d896242b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 1 --2- 172.21.15.63:0/2961477704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d89622bdb0 0x55d896243040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 1 -- 172.21.15.63:0/2961477704 >> 172.21.15.63:0/2961477704 conn(0x55d896107040 msgr2=0x55d89622fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:18.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbb93e00700 0 client.21723 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:18.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 1 -- 172.21.15.63:0/2961477704 shutdown_connections 2024-05-27T08:06:18.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 1 -- 172.21.15.63:0/2961477704 wait complete. 2024-05-27T08:06:18.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.005+0000 7fbbc759fb80 20 client.21723 trim_cache size 0 max 16384 2024-05-27T08:06:18.007 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:06:18.007 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:06:18.008 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "ad647c7e-abff-4715-a7fd-7bb57e808ff9", 2024-05-27T08:06:18.008 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:06:18.008 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:06:18.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.006+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3681982636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbb980b2a70 msgr2=0x7fbb980b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:18.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.006+0000 7fbbb8a00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbb980b2a70 0x7fbb980b4f80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbba400be10 tx=0x7fbba400b5c0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.006+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3681982636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb4109860 msgr2=0x7fbbb41a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:18.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.006+0000 7fbbb8a00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb4109860 0x7fbbb41a1720 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009850 tx=0x7fbbac007d90 comp rx=0 tx=0).stop 2024-05-27T08:06:18.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.007+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3681982636 shutdown_connections 2024-05-27T08:06:18.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.007+0000 7fbbb8a00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbb980b2a70 0x7fbb980b4f80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.007+0000 7fbbb8a00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbb4109860 0x7fbbb41a1720 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.007+0000 7fbbb8a00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb41045e0 0x7fbbb419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.007+0000 7fbbb8a00700 1 --2- 172.21.15.63:0/3681982636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4108f10 0x7fbbb419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.007+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3681982636 >> 172.21.15.63:0/3681982636 conn(0x7fbbb4100030 msgr2=0x7fbbb4110e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:18.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.007+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3681982636 shutdown_connections 2024-05-27T08:06:18.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:18.007+0000 7fbbb8a00700 1 -- 172.21.15.63:0/3681982636 wait complete. 2024-05-27T08:06:18.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:18 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:06:18.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:18 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:18.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:18 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:06:18.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:18 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:18.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:18 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:06:18.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:18 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:18.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:18 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:06:18.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:18 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:18.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:18 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:06:18.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:18 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:18.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:18 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:06:18.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:18 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:18.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:18 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:06:18.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:18 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:18.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:18 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:06:18.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:18 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:18.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:18 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:06:18.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:18 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:18.818 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:06:19.348 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:19.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:06:19.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.698 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:06:19.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:06:19.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:06:19.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:06:19.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.699 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:19 smithi063 ceph-mon[31613]: pgmap v1038: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:06:19.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:06:19.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:06:19.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:06:19.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:06:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:06:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.711 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:19 smithi086 ceph-mon[34687]: pgmap v1038: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:06:19.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:06:19.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:06:19.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:06:19.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:06:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:06:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:06:19.715 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:19 smithi149 ceph-mon[35408]: pgmap v1038: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:06:21.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:21 smithi086 ceph-mon[34687]: pgmap v1039: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:06:21.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:21 smithi149 ceph-mon[35408]: pgmap v1039: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:06:21.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:21 smithi063 ceph-mon[31613]: pgmap v1039: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:06:21.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.747+0000 7f2980000700 1 -- 172.21.15.63:0/3914765347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29781045c0 msgr2=0x7f29781049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:21.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.747+0000 7f2980000700 1 --2- 172.21.15.63:0/3914765347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29781045c0 0x7f29781049d0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f296c0099c0 tx=0x7f296c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.748+0000 7f2980000700 1 -- 172.21.15.63:0/3914765347 shutdown_connections 2024-05-27T08:06:21.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.748+0000 7f2980000700 1 --2- 172.21.15.63:0/3914765347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2978109840 0x7f29781111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.748+0000 7f2980000700 1 --2- 172.21.15.63:0/3914765347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29781045c0 0x7f29781049d0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.748+0000 7f2980000700 1 --2- 172.21.15.63:0/3914765347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2978108ef0 0x7f2978109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.748+0000 7f2980000700 1 -- 172.21.15.63:0/3914765347 >> 172.21.15.63:0/3914765347 conn(0x7f29780ffe90 msgr2=0x7f29781022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:21.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.748+0000 7f2980000700 1 -- 172.21.15.63:0/3914765347 shutdown_connections 2024-05-27T08:06:21.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.748+0000 7f2980000700 1 -- 172.21.15.63:0/3914765347 wait complete. 2024-05-27T08:06:21.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.749+0000 7f2980000700 1 Processor -- start 2024-05-27T08:06:21.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.750+0000 7f2980000700 1 -- start start 2024-05-27T08:06:21.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.750+0000 7f2980000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29781045c0 0x7f2978076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.750+0000 7f2980000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2978108ef0 0x7f2978077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.750+0000 7f2980000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2978109840 0x7f2978077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.750+0000 7f2980000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2978114370 con 0x7f2978108ef0 2024-05-27T08:06:21.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.750+0000 7f2980000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29781141f0 con 0x7f2978109840 2024-05-27T08:06:21.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.750+0000 7f2980000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29781144f0 con 0x7f29781045c0 2024-05-27T08:06:21.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2978108ef0 0x7f2978077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2978108ef0 0x7f2978077100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49506/0 (socket says 172.21.15.63:49506) 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297cc00700 1 -- 172.21.15.63:0/2203806797 learned_addr learned my addr 172.21.15.63:0/2203806797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297d600700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29781045c0 0x7f2978076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297e000700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2978109840 0x7f2978077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297cc00700 1 -- 172.21.15.63:0/2203806797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29781045c0 msgr2=0x7f2978076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297cc00700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29781045c0 0x7f2978076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297cc00700 1 -- 172.21.15.63:0/2203806797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2978109840 msgr2=0x7f2978077640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297cc00700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2978109840 0x7f2978077640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297cc00700 1 -- 172.21.15.63:0/2203806797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29781b1890 con 0x7f2978108ef0 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.751+0000 7f297d600700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29781045c0 0x7f2978076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:21.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.752+0000 7f297cc00700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2978108ef0 0x7f2978077100 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f296400d980 tx=0x7f296400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:21.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.752+0000 7f2976000700 1 -- 172.21.15.63:0/2203806797 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964017070 con 0x7f2978108ef0 2024-05-27T08:06:21.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.752+0000 7f2976000700 1 -- 172.21.15.63:0/2203806797 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2964009dc0 con 0x7f2978108ef0 2024-05-27T08:06:21.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.752+0000 7f2980000700 1 -- 172.21.15.63:0/2203806797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29781b1b20 con 0x7f2978108ef0 2024-05-27T08:06:21.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.752+0000 7f2976000700 1 -- 172.21.15.63:0/2203806797 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296400f3c0 con 0x7f2978108ef0 2024-05-27T08:06:21.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.752+0000 7f2980000700 1 -- 172.21.15.63:0/2203806797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29781b20b0 con 0x7f2978108ef0 2024-05-27T08:06:21.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.757+0000 7f2976000700 1 -- 172.21.15.63:0/2203806797 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f296400f560 con 0x7f2978108ef0 2024-05-27T08:06:21.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.758+0000 7f298ec2cb80 1 Processor -- start 2024-05-27T08:06:21.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.758+0000 7f298ec2cb80 1 -- start start 2024-05-27T08:06:21.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.758+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b812889f0 0x555b8131b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.758+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b8131b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.758+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b8131be70 0x555b81328110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.758+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555b81249e40 con 0x555b8119ce40 2024-05-27T08:06:21.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.758+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555b812399e0 con 0x555b8131be70 2024-05-27T08:06:21.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.758+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555b8123da20 con 0x555b812889f0 2024-05-27T08:06:21.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f297d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b8131b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f297d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b8131b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49514/0 (socket says 172.21.15.63:49514) 2024-05-27T08:06:21.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f297d600700 1 -- 172.21.15.63:0/2457970797 learned_addr learned my addr 172.21.15.63:0/2457970797 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:21.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f297cc00700 1 --2- 172.21.15.63:0/2457970797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b8131be70 0x555b81328110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f2976000700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29600b2a70 0x7f29600b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f297e000700 1 --2- 172.21.15.63:0/2457970797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b812889f0 0x555b8131b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f297d600700 1 -- 172.21.15.63:0/2457970797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b812889f0 msgr2=0x555b8131b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:21.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f297d600700 1 --2- 172.21.15.63:0/2457970797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b812889f0 0x555b8131b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f297d600700 1 -- 172.21.15.63:0/2457970797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b8131be70 msgr2=0x555b81328110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:21.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f297d600700 1 --2- 172.21.15.63:0/2457970797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b8131be70 0x555b81328110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.759+0000 7f297d600700 1 -- 172.21.15.63:0/2457970797 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b81202420 con 0x555b8119ce40 2024-05-27T08:06:21.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.760+0000 7f297e000700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29600b2a70 0x7f29600b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.760+0000 7f297d600700 1 --2- 172.21.15.63:0/2457970797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b8131b870 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f296c0099c0 tx=0x7f296c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da4c69a6342d838 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.760+0000 7f2976000700 1 -- 172.21.15.63:0/2203806797 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f29640e24b0 con 0x7f2978108ef0 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.760+0000 7f296aa00700 1 -- 172.21.15.63:0/2457970797 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c00b9d0 con 0x555b8119ce40 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.760+0000 7f296aa00700 1 -- 172.21.15.63:0/2457970797 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f296c004050 con 0x555b8119ce40 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.760+0000 7f296aa00700 1 -- 172.21.15.63:0/2457970797 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c004d50 con 0x555b8119ce40 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.762+0000 7f298ec2cb80 1 -- 172.21.15.63:0/2457970797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 msgr2=0x555b8131b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.762+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/2457970797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b8131b870 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f296c0099c0 tx=0x7f296c0096a0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.762+0000 7f297e000700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29600b2a70 0x7f29600b4f80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2970006130 tx=0x7f2970009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.762+0000 7f298ec2cb80 1 -- 172.21.15.63:0/2457970797 shutdown_connections 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.762+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/2457970797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b8131be70 0x555b81328110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.762+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/2457970797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b812889f0 0x555b8131b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.762+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/2457970797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b8131b870 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.762+0000 7f298ec2cb80 1 -- 172.21.15.63:0/2457970797 >> 172.21.15.63:0/2457970797 conn(0x555b81236390 msgr2=0x555b813174c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:21.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.762+0000 7f298ec2cb80 1 -- 172.21.15.63:0/2457970797 shutdown_connections 2024-05-27T08:06:21.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.762+0000 7f298ec2cb80 1 -- 172.21.15.63:0/2457970797 wait complete. 2024-05-27T08:06:21.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.763+0000 7f298ec2cb80 1 -- start start 2024-05-27T08:06:21.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.763+0000 7f298ec2cb80 1 Processor -- start 2024-05-27T08:06:21.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.764+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b8119ce40 0x555b813256e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.764+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b812889f0 0x555b81325c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.764+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8131be70 0x555b81326160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.764+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555b8113ffa0 con 0x555b812889f0 2024-05-27T08:06:21.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.764+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555b811da760 con 0x555b8119ce40 2024-05-27T08:06:21.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.764+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555b81328ac0 con 0x555b8131be70 2024-05-27T08:06:21.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.764+0000 7f298ec2cb80 10 client.0 _ll_register_callbacks cb 0x555b811e3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:21.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.764+0000 7f297cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b812889f0 0x555b81325c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.764+0000 7f297d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b8119ce40 0x555b813256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.765+0000 7f297cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b812889f0 0x555b81325c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49530/0 (socket says 172.21.15.63:49530) 2024-05-27T08:06:21.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.765+0000 7f297cc00700 1 -- 172.21.15.63:0/1602777961 learned_addr learned my addr 172.21.15.63:0/1602777961 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:21.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.765+0000 7f297e000700 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8131be70 0x555b81326160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.765+0000 7f297cc00700 1 -- 172.21.15.63:0/1602777961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8131be70 msgr2=0x555b81326160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:21.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.765+0000 7f297cc00700 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8131be70 0x555b81326160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.765+0000 7f297cc00700 1 -- 172.21.15.63:0/1602777961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b8119ce40 msgr2=0x555b813256e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:21.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.765+0000 7f297cc00700 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b8119ce40 0x555b813256e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.765+0000 7f297cc00700 1 -- 172.21.15.63:0/1602777961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b811898d0 con 0x555b812889f0 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.766+0000 7f297cc00700 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b812889f0 0x555b81325c20 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f29640a6640 tx=0x7f29640108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c9c34777405f1e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.766+0000 7f295f400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.766+0000 7f295f400700 1 -- 172.21.15.63:0/1602777961 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964020070 con 0x555b812889f0 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.766+0000 7f295f400700 1 -- 172.21.15.63:0/1602777961 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f2964010460 con 0x555b812889f0 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.766+0000 7f298ec2cb80 1 -- 172.21.15.63:0/1602777961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f296c004050 con 0x555b812889f0 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.766+0000 7f295f400700 1 -- 172.21.15.63:0/1602777961 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f29640e5e50 con 0x555b812889f0 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.766+0000 7f298ec2cb80 10 client.21735 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.766+0000 7f298ec2cb80 1 -- 172.21.15.63:0/1602777961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555b8120a060 con 0x555b812889f0 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.767+0000 7f295f400700 1 -- 172.21.15.63:0/1602777961 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f296c004050 con 0x555b812889f0 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.767+0000 7f295f400700 1 -- 172.21.15.63:0/1602777961 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f29640e2130 con 0x555b812889f0 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.767+0000 7f298ec2cb80 10 client.21735 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.767+0000 7f298ec2cb80 10 client.21735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:21.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.767+0000 7f298ec2cb80 20 client.21735 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:21.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.767+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555b81335a80 0x555b81337f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.767+0000 7f298ec2cb80 4 client.21735 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:21.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.767+0000 7f298ec2cb80 1 -- 172.21.15.63:0/1602777961 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b8124bd90 con 0x555b81335a80 2024-05-27T08:06:21.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.768+0000 7f297d600700 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555b81335a80 0x555b81337f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.769+0000 7f297d600700 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555b81335a80 0x555b81337f30 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c33f414fceb7e83e server_cookie=49593e78dec54bd0 in_seq=0 out_seq=0 2024-05-27T08:06:21.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.769+0000 7f295f400700 10 client.21735 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:21.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.771+0000 7f295f400700 1 -- 172.21.15.63:0/1602777961 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555b8124bd90 con 0x555b81335a80 2024-05-27T08:06:21.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.771+0000 7f295f400700 10 client.21735 handle_command_reply: tid=0 2024-05-27T08:06:21.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.771+0000 7f298ec2cb80 1 client.21735 shutdown 2024-05-27T08:06:21.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.771+0000 7f298ec2cb80 1 -- 172.21.15.63:0/1602777961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b812889f0 msgr2=0x555b81325c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.771+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b812889f0 0x555b81325c20 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f29640a6640 tx=0x7f29640108b0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.772+0000 7f298ec2cb80 1 -- 172.21.15.63:0/1602777961 shutdown_connections 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.772+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555b81335a80 0x555b81337f30 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.772+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8131be70 0x555b81326160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.772+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b8119ce40 0x555b813256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.772+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/1602777961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b812889f0 0x555b81325c20 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.772+0000 7f295f400700 0 client.21735 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.772+0000 7f298ec2cb80 1 -- 172.21.15.63:0/1602777961 >> 172.21.15.63:0/1602777961 conn(0x555b81236390 msgr2=0x555b8131b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.772+0000 7f298ec2cb80 1 -- 172.21.15.63:0/1602777961 shutdown_connections 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.772+0000 7f298ec2cb80 1 -- 172.21.15.63:0/1602777961 wait complete. 2024-05-27T08:06:21.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.772+0000 7f298ec2cb80 20 client.21735 trim_cache size 0 max 16384 2024-05-27T08:06:21.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.806+0000 7f298ec2cb80 1 Processor -- start 2024-05-27T08:06:21.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.806+0000 7f298ec2cb80 1 -- start start 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.806+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b81324a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.806+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b812889f0 0x555b81320f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.806+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8131be70 0x555b81325db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.806+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29781bd6c0 con 0x555b8119ce40 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.806+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29781baf50 con 0x555b812889f0 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.806+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b8dc0 con 0x555b8131be70 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.807+0000 7f297d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b81324a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.807+0000 7f297d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b81324a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49536/0 (socket says 172.21.15.63:49536) 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.807+0000 7f297d600700 1 -- 172.21.15.63:0/2446858181 learned_addr learned my addr 172.21.15.63:0/2446858181 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.807+0000 7f297cc00700 1 --2- 172.21.15.63:0/2446858181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b812889f0 0x555b81320f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.807+0000 7f297e000700 1 --2- 172.21.15.63:0/2446858181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8131be70 0x555b81325db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.807+0000 7f297d600700 1 -- 172.21.15.63:0/2446858181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8131be70 msgr2=0x555b81325db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:21.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.807+0000 7f297d600700 1 --2- 172.21.15.63:0/2446858181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8131be70 0x555b81325db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.807+0000 7f297d600700 1 -- 172.21.15.63:0/2446858181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b812889f0 msgr2=0x555b81320f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:21.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.807+0000 7f297d600700 1 --2- 172.21.15.63:0/2446858181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b812889f0 0x555b81320f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.807+0000 7f297d600700 1 -- 172.21.15.63:0/2446858181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b813284e0 con 0x555b8119ce40 2024-05-27T08:06:21.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.808+0000 7f297d600700 1 --2- 172.21.15.63:0/2446858181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b81324a00 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f296c006010 tx=0x7f296c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3128a11a027ad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:21.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.808+0000 7f296a000700 1 -- 172.21.15.63:0/2446858181 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c00b9d0 con 0x555b8119ce40 2024-05-27T08:06:21.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.808+0000 7f296a000700 1 -- 172.21.15.63:0/2446858181 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f296c030030 con 0x555b8119ce40 2024-05-27T08:06:21.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.808+0000 7f296a000700 1 -- 172.21.15.63:0/2446858181 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c0052a0 con 0x555b8119ce40 2024-05-27T08:06:21.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.809+0000 7f298ec2cb80 1 -- 172.21.15.63:0/2446858181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 msgr2=0x555b81324a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:21.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.809+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/2446858181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b81324a00 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f296c006010 tx=0x7f296c009f70 comp rx=0 tx=0).stop 2024-05-27T08:06:21.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.809+0000 7f298ec2cb80 1 -- 172.21.15.63:0/2446858181 shutdown_connections 2024-05-27T08:06:21.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.809+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/2446858181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8131be70 0x555b81325db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.809+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/2446858181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8119ce40 0x555b81324a00 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.809+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/2446858181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b812889f0 0x555b81320f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.810+0000 7f298ec2cb80 1 -- 172.21.15.63:0/2446858181 >> 172.21.15.63:0/2446858181 conn(0x555b81236390 msgr2=0x555b813256a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:21.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.810+0000 7f298ec2cb80 1 -- 172.21.15.63:0/2446858181 shutdown_connections 2024-05-27T08:06:21.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.810+0000 7f298ec2cb80 1 -- 172.21.15.63:0/2446858181 wait complete. 2024-05-27T08:06:21.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.810+0000 7f298ec2cb80 1 -- start start 2024-05-27T08:06:21.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.811+0000 7f298ec2cb80 1 Processor -- start 2024-05-27T08:06:21.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.811+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8119ce40 0x555b8132e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.811+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b812889f0 0x555b8131fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.811+0000 7f298ec2cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8131be70 0x555b813202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.811+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b9410 con 0x555b8131be70 2024-05-27T08:06:21.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.811+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29781ba880 con 0x555b812889f0 2024-05-27T08:06:21.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.811+0000 7f298ec2cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b9ae0 con 0x555b8119ce40 2024-05-27T08:06:21.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.811+0000 7f298ec2cb80 10 client.0 _ll_register_callbacks cb 0x555b811e3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:21.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.812+0000 7f297e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8131be70 0x555b813202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.812+0000 7f297e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8131be70 0x555b813202f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49552/0 (socket says 172.21.15.63:49552) 2024-05-27T08:06:21.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.812+0000 7f297e000700 1 -- 172.21.15.63:0/4098256484 learned_addr learned my addr 172.21.15.63:0/4098256484 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:21.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.812+0000 7f297cc00700 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b812889f0 0x555b8131fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.812+0000 7f297e000700 1 -- 172.21.15.63:0/4098256484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8119ce40 msgr2=0x555b8132e3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:06:21.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.812+0000 7f297d600700 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8119ce40 0x555b8132e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.812+0000 7f297e000700 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8119ce40 0x555b8132e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.812+0000 7f297e000700 1 -- 172.21.15.63:0/4098256484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b812889f0 msgr2=0x555b8131fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:21.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.813+0000 7f297e000700 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b812889f0 0x555b8131fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.813+0000 7f297e000700 1 -- 172.21.15.63:0/4098256484 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b813208e0 con 0x555b8131be70 2024-05-27T08:06:21.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.813+0000 7f297e000700 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8131be70 0x555b813202f0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f2970015dc0 tx=0x7f2970007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6bc64a5da1dd07 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:21.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.813+0000 7f295f400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:21.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.813+0000 7f295f400700 1 -- 172.21.15.63:0/4098256484 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970021de0 con 0x555b8131be70 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.813+0000 7f295f400700 1 -- 172.21.15.63:0/4098256484 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f297002b470 con 0x555b8131be70 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.813+0000 7f298ec2cb80 1 -- 172.21.15.63:0/4098256484 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f296c030030 con 0x555b8131be70 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.814+0000 7f295f400700 1 -- 172.21.15.63:0/4098256484 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2970034050 con 0x555b8131be70 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.814+0000 7f298ec2cb80 10 client.21741 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.814+0000 7f298ec2cb80 1 -- 172.21.15.63:0/4098256484 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555b813a05c0 con 0x555b8131be70 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.814+0000 7f295f400700 1 -- 172.21.15.63:0/4098256484 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f296c030030 con 0x555b8131be70 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.814+0000 7f295f400700 1 -- 172.21.15.63:0/4098256484 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f297001f3d0 con 0x555b8131be70 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.815+0000 7f298ec2cb80 10 client.21741 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.815+0000 7f298ec2cb80 10 client.21741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.815+0000 7f298ec2cb80 20 client.21741 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.815+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555b813a0b40 0x555b813a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:21.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.815+0000 7f298ec2cb80 4 client.21741 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:06:21.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.815+0000 7f298ec2cb80 1 -- 172.21.15.63:0/4098256484 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555b81245570 con 0x555b813a0b40 2024-05-27T08:06:21.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.815+0000 7f297d600700 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555b813a0b40 0x555b813a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:21.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.816+0000 7f297d600700 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555b813a0b40 0x555b813a2f60 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50fca4ceb15ea8dc server_cookie=939a8df4c153cac5 in_seq=0 out_seq=0 2024-05-27T08:06:21.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.816+0000 7f295f400700 10 client.21741 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:21.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f295f400700 1 -- 172.21.15.63:0/4098256484 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555b81245570 con 0x555b813a0b40 2024-05-27T08:06:21.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f295f400700 10 client.21741 handle_command_reply: tid=0 2024-05-27T08:06:21.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f298ec2cb80 1 client.21741 shutdown 2024-05-27T08:06:21.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f298ec2cb80 1 -- 172.21.15.63:0/4098256484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8131be70 msgr2=0x555b813202f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8131be70 0x555b813202f0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f2970015dc0 tx=0x7f2970007950 comp rx=0 tx=0).stop 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f298ec2cb80 1 -- 172.21.15.63:0/4098256484 shutdown_connections 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555b813a0b40 0x555b813a2f60 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555b8131be70 0x555b813202f0 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8119ce40 0x555b8132e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f298ec2cb80 1 --2- 172.21.15.63:0/4098256484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555b812889f0 0x555b8131fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f298ec2cb80 1 -- 172.21.15.63:0/4098256484 >> 172.21.15.63:0/4098256484 conn(0x555b81236390 msgr2=0x555b81324760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.817+0000 7f295f400700 0 client.21741 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.818+0000 7f298ec2cb80 1 -- 172.21.15.63:0/4098256484 shutdown_connections 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.818+0000 7f298ec2cb80 1 -- 172.21.15.63:0/4098256484 wait complete. 2024-05-27T08:06:21.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.818+0000 7f298ec2cb80 20 client.21741 trim_cache size 0 max 16384 2024-05-27T08:06:21.820 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:06:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:06:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:06:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "ad647c7e-abff-4715-a7fd-7bb57e808ff9": { 2024-05-27T08:06:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:06:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "ad647c7e-abff-4715-a7fd-7bb57e808ff9", 2024-05-27T08:06:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:06:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:06:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:06:21.821 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:06:21.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 -- 172.21.15.63:0/2203806797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29600b2a70 msgr2=0x7f29600b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:21.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29600b2a70 0x7f29600b4f80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2970006130 tx=0x7f2970009040 comp rx=0 tx=0).stop 2024-05-27T08:06:21.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 -- 172.21.15.63:0/2203806797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2978108ef0 msgr2=0x7f2978077100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:21.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2978108ef0 0x7f2978077100 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f296400d980 tx=0x7f296400dc90 comp rx=0 tx=0).stop 2024-05-27T08:06:21.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 -- 172.21.15.63:0/2203806797 shutdown_connections 2024-05-27T08:06:21.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f29600b2a70 0x7f29600b4f80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2978109840 0x7f2978077640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29781045c0 0x7f2978076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 --2- 172.21.15.63:0/2203806797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2978108ef0 0x7f2978077100 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:21.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 -- 172.21.15.63:0/2203806797 >> 172.21.15.63:0/2203806797 conn(0x7f29780ffe90 msgr2=0x7f29781022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:21.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 -- 172.21.15.63:0/2203806797 shutdown_connections 2024-05-27T08:06:21.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:21.819+0000 7f2980000700 1 -- 172.21.15.63:0/2203806797 wait complete. 2024-05-27T08:06:22.834 INFO:tasks.cephfs.filesystem:scrub status for tag:ad647c7e-abff-4715-a7fd-7bb57e808ff9 - {'path': '/', 'tag': 'ad647c7e-abff-4715-a7fd-7bb57e808ff9', 'options': 'recursive,force'} 2024-05-27T08:06:23.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:23 smithi086 ceph-mon[34687]: pgmap v1040: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.0 MiB/s wr, 51 op/s 2024-05-27T08:06:23.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:23 smithi149 ceph-mon[35408]: pgmap v1040: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.0 MiB/s wr, 51 op/s 2024-05-27T08:06:23.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:23 smithi063 ceph-mon[31613]: pgmap v1040: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.0 MiB/s wr, 51 op/s 2024-05-27T08:06:25.448 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:25 smithi063 ceph-mon[31613]: pgmap v1041: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.0 MiB/s wr, 51 op/s 2024-05-27T08:06:25.448 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:25 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:06:25.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:25 smithi086 ceph-mon[34687]: pgmap v1041: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.0 MiB/s wr, 51 op/s 2024-05-27T08:06:25.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:25 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:06:25.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:25 smithi149 ceph-mon[35408]: pgmap v1041: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.0 MiB/s wr, 51 op/s 2024-05-27T08:06:25.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:25 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:06:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:27 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:06:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:27 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:06:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:27 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:06:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:27 smithi086 ceph-mon[34687]: pgmap v1042: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.0 MiB/s wr, 54 op/s 2024-05-27T08:06:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:27 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:06:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:27 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:06:27.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:27 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:06:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:27 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:06:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:27 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:06:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:27 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:06:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:27 smithi149 ceph-mon[35408]: pgmap v1042: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.0 MiB/s wr, 54 op/s 2024-05-27T08:06:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:27 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:06:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:27 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:06:27.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:27 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:06:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:27 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:06:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:27 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:06:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:27 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:06:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:27 smithi063 ceph-mon[31613]: pgmap v1042: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.0 MiB/s wr, 54 op/s 2024-05-27T08:06:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:27 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:06:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:27 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:06:27.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:27 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:06:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.752+0000 7f5ae3e00640 1 -- 172.21.15.63:0/419559641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae410b6b0 msgr2=0x7f5ae410dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.752+0000 7f5ae3e00640 1 --2- 172.21.15.63:0/419559641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae410b6b0 0x7f5ae410dad0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009900 tx=0x7f5ad002a480 comp rx=0 tx=0).stop 2024-05-27T08:06:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.752+0000 7f5ae3e00640 1 -- 172.21.15.63:0/419559641 shutdown_connections 2024-05-27T08:06:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.752+0000 7f5ae3e00640 1 --2- 172.21.15.63:0/419559641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e010 0x7f5ae4110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.752+0000 7f5ae3e00640 1 --2- 172.21.15.63:0/419559641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae410b6b0 0x7f5ae410dad0 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.752+0000 7f5ae3e00640 1 --2- 172.21.15.63:0/419559641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae4104c50 0x7f5ae4105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.752+0000 7f5ae3e00640 1 -- 172.21.15.63:0/419559641 >> 172.21.15.63:0/419559641 conn(0x7f5ae40fe210 msgr2=0x7f5ae4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.752+0000 7f5ae3e00640 1 -- 172.21.15.63:0/419559641 shutdown_connections 2024-05-27T08:06:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.752+0000 7f5ae3e00640 1 -- 172.21.15.63:0/419559641 wait complete. 2024-05-27T08:06:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.754+0000 7f5ae3e00640 1 Processor -- start 2024-05-27T08:06:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.754+0000 7f5ae3e00640 1 -- start start 2024-05-27T08:06:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae4104c50 0x7f5ae419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae410b6b0 0x7f5ae419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e010 0x7f5ae419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae3e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4113b20 con 0x7f5ae4104c50 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41139a0 con 0x7f5ae410b6b0 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae3e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4113ca0 con 0x7f5ae410e010 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae4104c50 0x7f5ae419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae2a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae4104c50 0x7f5ae419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49574/0 (socket says 172.21.15.63:49574) 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae2a00640 1 -- 172.21.15.63:0/2066377818 learned_addr learned my addr 172.21.15.63:0/2066377818 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae3400640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e010 0x7f5ae419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.755+0000 7f5ae2000640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae410b6b0 0x7f5ae419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.756+0000 7f5ae2a00640 1 -- 172.21.15.63:0/2066377818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e010 msgr2=0x7f5ae419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.756+0000 7f5ae2a00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e010 0x7f5ae419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.756+0000 7f5ae2a00640 1 -- 172.21.15.63:0/2066377818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae410b6b0 msgr2=0x7f5ae419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.756+0000 7f5ae2a00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae410b6b0 0x7f5ae419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.756+0000 7f5ae2a00640 1 -- 172.21.15.63:0/2066377818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae419f440 con 0x7f5ae4104c50 2024-05-27T08:06:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.756+0000 7f5ae3400640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e010 0x7f5ae419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.756+0000 7f5ae2a00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae4104c50 0x7f5ae419a5e0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400d950 tx=0x7f5ad400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.756+0000 7f5adb400640 1 -- 172.21.15.63:0/2066377818 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4017070 con 0x7f5ae4104c50 2024-05-27T08:06:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.756+0000 7f5adb400640 1 -- 172.21.15.63:0/2066377818 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad4009c30 con 0x7f5ae4104c50 2024-05-27T08:06:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.756+0000 7f5ae3e00640 1 -- 172.21.15.63:0/2066377818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae419f730 con 0x7f5ae4104c50 2024-05-27T08:06:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.757+0000 7f5adb400640 1 -- 172.21.15.63:0/2066377818 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad400f620 con 0x7f5ae4104c50 2024-05-27T08:06:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.757+0000 7f5ae3e00640 1 -- 172.21.15.63:0/2066377818 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae41a70b0 con 0x7f5ae4104c50 2024-05-27T08:06:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.761+0000 7f5af083d740 1 Processor -- start 2024-05-27T08:06:27.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.761+0000 7f5af083d740 1 -- start start 2024-05-27T08:06:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.761+0000 7f5af083d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c46a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.761+0000 7f5af083d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de387b2f0 0x556de3c46fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.761+0000 7f5af083d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.761+0000 7f5af083d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556de3bb7b90 con 0x556de38a41b0 2024-05-27T08:06:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.761+0000 7f5af083d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556de3ba7730 con 0x556de387b2f0 2024-05-27T08:06:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.761+0000 7f5af083d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556de3bab770 con 0x556de38834d0 2024-05-27T08:06:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.761+0000 7f5adb400640 1 -- 172.21.15.63:0/2066377818 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5ad4010460 con 0x7f5ae4104c50 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c46a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c46a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49590/0 (socket says 172.21.15.63:49590) 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae3400640 1 -- 172.21.15.63:0/1160916414 learned_addr learned my addr 172.21.15.63:0/1160916414 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae2000640 1 --2- 172.21.15.63:0/1160916414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae2a00640 1 --2- 172.21.15.63:0/1160916414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de387b2f0 0x556de3c46fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae3400640 1 -- 172.21.15.63:0/1160916414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 msgr2=0x556de3c474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae3400640 1 --2- 172.21.15.63:0/1160916414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae3400640 1 -- 172.21.15.63:0/1160916414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de387b2f0 msgr2=0x556de3c46fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae3400640 1 --2- 172.21.15.63:0/1160916414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de387b2f0 0x556de3c46fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae3400640 1 -- 172.21.15.63:0/1160916414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556de3b408b0 con 0x556de38a41b0 2024-05-27T08:06:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.762+0000 7f5ae2000640 1 --2- 172.21.15.63:0/1160916414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.763+0000 7f5ae3400640 1 --2- 172.21.15.63:0/1160916414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c46a60 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009f30 tx=0x7f5adc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6d444a618b708 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.763+0000 7f5acfe00640 1 -- 172.21.15.63:0/1160916414 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc016070 con 0x556de38a41b0 2024-05-27T08:06:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.763+0000 7f5acfe00640 1 -- 172.21.15.63:0/1160916414 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5adc01baf0 con 0x556de38a41b0 2024-05-27T08:06:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.763+0000 7f5acfe00640 1 -- 172.21.15.63:0/1160916414 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc012b50 con 0x556de38a41b0 2024-05-27T08:06:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.764+0000 7f5adb400640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ac80b2950 0x7f5ac80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.764+0000 7f5adb400640 1 -- 172.21.15.63:0/2066377818 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f5ad40e26c0 con 0x7f5ae4104c50 2024-05-27T08:06:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.764+0000 7f5ae2a00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ac80b2950 0x7f5ac80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.764+0000 7f5af083d740 1 -- 172.21.15.63:0/1160916414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 msgr2=0x556de3c46a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.764+0000 7f5af083d740 1 --2- 172.21.15.63:0/1160916414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c46a60 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009f30 tx=0x7f5adc010800 comp rx=0 tx=0).stop 2024-05-27T08:06:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.765+0000 7f5af083d740 1 -- 172.21.15.63:0/1160916414 shutdown_connections 2024-05-27T08:06:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.765+0000 7f5af083d740 1 --2- 172.21.15.63:0/1160916414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.765+0000 7f5af083d740 1 --2- 172.21.15.63:0/1160916414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de387b2f0 0x556de3c46fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.765+0000 7f5af083d740 1 --2- 172.21.15.63:0/1160916414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c46a60 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.765+0000 7f5af083d740 1 -- 172.21.15.63:0/1160916414 >> 172.21.15.63:0/1160916414 conn(0x556de3898300 msgr2=0x556de3c43210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.765+0000 7f5af083d740 1 -- 172.21.15.63:0/1160916414 shutdown_connections 2024-05-27T08:06:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.765+0000 7f5af083d740 1 -- 172.21.15.63:0/1160916414 wait complete. 2024-05-27T08:06:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.765+0000 7f5ae2a00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ac80b2950 0x7f5ac80b4e60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40dfb00 tx=0x7f5ad4000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.766+0000 7f5af083d740 1 -- start start 2024-05-27T08:06:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.766+0000 7f5af083d740 1 Processor -- start 2024-05-27T08:06:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.766+0000 7f5af083d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c5b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5af083d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c5b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5af083d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de38a41b0 0x556de3c555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5af083d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556de388d790 con 0x556de387b2f0 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5af083d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556de3b4fbc0 con 0x556de38a41b0 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5af083d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556de3c53b90 con 0x556de38834d0 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5af083d740 10 client.0 _ll_register_callbacks cb 0x556de3b2d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5ae3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c5b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5ae3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c5b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49606/0 (socket says 172.21.15.63:49606) 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5ae3400640 1 -- 172.21.15.63:0/3342813220 learned_addr learned my addr 172.21.15.63:0/3342813220 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5ae2000640 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c5b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5ae3400640 1 -- 172.21.15.63:0/3342813220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 msgr2=0x556de3c5b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5ae3400640 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c5b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5ae3400640 1 -- 172.21.15.63:0/3342813220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de38a41b0 msgr2=0x556de3c555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5ae3400640 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de38a41b0 0x556de3c555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.767+0000 7f5ae3400640 1 -- 172.21.15.63:0/3342813220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556de38bc210 con 0x556de387b2f0 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.768+0000 7f5ae3400640 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c5b200 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009f00 tx=0x7f5adc012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4ab7c7c9a58b05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.768+0000 7f5accc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.768+0000 7f5accc00640 1 -- 172.21.15.63:0/3342813220 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc02b060 con 0x556de387b2f0 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.768+0000 7f5accc00640 1 -- 172.21.15.63:0/3342813220 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f5adc01b460 con 0x556de387b2f0 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.768+0000 7f5af083d740 1 -- 172.21.15.63:0/3342813220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5adc01baf0 con 0x556de387b2f0 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.769+0000 7f5accc00640 1 -- 172.21.15.63:0/3342813220 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5adc016040 con 0x556de387b2f0 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.769+0000 7f5af083d740 10 client.21753 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.769+0000 7f5af083d740 1 -- 172.21.15.63:0/3342813220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556de3988620 con 0x556de387b2f0 2024-05-27T08:06:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.769+0000 7f5accc00640 1 -- 172.21.15.63:0/3342813220 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5adc01baf0 con 0x556de387b2f0 2024-05-27T08:06:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.770+0000 7f5accc00640 1 -- 172.21.15.63:0/3342813220 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5adc00f040 con 0x556de387b2f0 2024-05-27T08:06:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.770+0000 7f5af083d740 10 client.21753 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.770+0000 7f5af083d740 10 client.21753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.770+0000 7f5af083d740 20 client.21753 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.770+0000 7f5af083d740 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556de3862d80 0x556de3c67190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.770+0000 7f5af083d740 4 client.21753 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.770+0000 7f5af083d740 1 -- 172.21.15.63:0/3342813220 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556de3bb9ae0 con 0x556de3862d80 2024-05-27T08:06:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.771+0000 7f5ae2000640 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556de3862d80 0x556de3c67190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.772+0000 7f5ae2000640 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556de3862d80 0x556de3c67190 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf2dd3c9b6bf205 server_cookie=11908668088e65bb in_seq=0 out_seq=0 2024-05-27T08:06:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.772+0000 7f5accc00640 10 client.21753 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.773+0000 7f5accc00640 1 -- 172.21.15.63:0/3342813220 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556de3bb9ae0 con 0x556de3862d80 2024-05-27T08:06:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.773+0000 7f5accc00640 10 client.21753 handle_command_reply: tid=0 2024-05-27T08:06:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 client.21753 shutdown 2024-05-27T08:06:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 -- 172.21.15.63:0/3342813220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 msgr2=0x556de3c5b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c5b200 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009f00 tx=0x7f5adc012520 comp rx=0 tx=0).stop 2024-05-27T08:06:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 -- 172.21.15.63:0/3342813220 shutdown_connections 2024-05-27T08:06:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556de3862d80 0x556de3c67190 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de38a41b0 0x556de3c555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c5b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 --2- 172.21.15.63:0/3342813220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c5b200 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5accc00640 0 client.21753 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 -- 172.21.15.63:0/3342813220 >> 172.21.15.63:0/3342813220 conn(0x556de3898300 msgr2=0x556de3c46890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 -- 172.21.15.63:0/3342813220 shutdown_connections 2024-05-27T08:06:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 1 -- 172.21.15.63:0/3342813220 wait complete. 2024-05-27T08:06:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.774+0000 7f5af083d740 20 client.21753 trim_cache size 0 max 16384 2024-05-27T08:06:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.797+0000 7f5af083d740 1 Processor -- start 2024-05-27T08:06:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.797+0000 7f5af083d740 1 -- start start 2024-05-27T08:06:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.797+0000 7f5af083d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c50850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.797+0000 7f5af083d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de38834d0 0x556de3c5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.797+0000 7f5af083d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38a41b0 0x556de3c48e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.797+0000 7f5af083d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41b17c0 con 0x556de387b2f0 2024-05-27T08:06:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.797+0000 7f5af083d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41af090 con 0x556de38834d0 2024-05-27T08:06:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.797+0000 7f5af083d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41acf00 con 0x556de38a41b0 2024-05-27T08:06:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.798+0000 7f5ae3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c50850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.798+0000 7f5ae3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c50850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49616/0 (socket says 172.21.15.63:49616) 2024-05-27T08:06:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.798+0000 7f5ae3400640 1 -- 172.21.15.63:0/3489029242 learned_addr learned my addr 172.21.15.63:0/3489029242 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.798+0000 7f5ae2000640 1 --2- 172.21.15.63:0/3489029242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de38834d0 0x556de3c5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.798+0000 7f5ae3400640 1 --2- 172.21.15.63:0/3489029242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38a41b0 0x556de3c48e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.798+0000 7f5ae3400640 1 -- 172.21.15.63:0/3489029242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38a41b0 msgr2=0x556de3c48e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.798+0000 7f5ae3400640 1 --2- 172.21.15.63:0/3489029242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38a41b0 0x556de3c48e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.798+0000 7f5ae3400640 1 -- 172.21.15.63:0/3489029242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de38834d0 msgr2=0x556de3c5b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.798+0000 7f5ae3400640 1 --2- 172.21.15.63:0/3489029242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de38834d0 0x556de3c5b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.798+0000 7f5ae3400640 1 -- 172.21.15.63:0/3489029242 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556de3c535c0 con 0x556de387b2f0 2024-05-27T08:06:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.799+0000 7f5ae3400640 1 --2- 172.21.15.63:0/3489029242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c50850 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009ae0 tx=0x7f5adc000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244e9a8637cbe546 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.799+0000 7f5acf400640 1 -- 172.21.15.63:0/3489029242 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc03e070 con 0x556de387b2f0 2024-05-27T08:06:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.799+0000 7f5acf400640 1 -- 172.21.15.63:0/3489029242 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5adc004c50 con 0x556de387b2f0 2024-05-27T08:06:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.799+0000 7f5acf400640 1 -- 172.21.15.63:0/3489029242 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc02b040 con 0x556de387b2f0 2024-05-27T08:06:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.801+0000 7f5af083d740 1 -- 172.21.15.63:0/3489029242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 msgr2=0x556de3c50850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.801+0000 7f5af083d740 1 --2- 172.21.15.63:0/3489029242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c50850 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009ae0 tx=0x7f5adc000f40 comp rx=0 tx=0).stop 2024-05-27T08:06:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.801+0000 7f5af083d740 1 -- 172.21.15.63:0/3489029242 shutdown_connections 2024-05-27T08:06:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.801+0000 7f5af083d740 1 --2- 172.21.15.63:0/3489029242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38a41b0 0x556de3c48e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.801+0000 7f5af083d740 1 --2- 172.21.15.63:0/3489029242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de38834d0 0x556de3c5b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.801+0000 7f5af083d740 1 --2- 172.21.15.63:0/3489029242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de387b2f0 0x556de3c50850 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.801+0000 7f5af083d740 1 -- 172.21.15.63:0/3489029242 >> 172.21.15.63:0/3489029242 conn(0x556de3898300 msgr2=0x556de3c48710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.801+0000 7f5af083d740 1 -- 172.21.15.63:0/3489029242 shutdown_connections 2024-05-27T08:06:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.801+0000 7f5af083d740 1 -- 172.21.15.63:0/3489029242 wait complete. 2024-05-27T08:06:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.802+0000 7f5af083d740 1 -- start start 2024-05-27T08:06:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.802+0000 7f5af083d740 1 Processor -- start 2024-05-27T08:06:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.802+0000 7f5af083d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de387b2f0 0x556de3c60440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.802+0000 7f5af083d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c60980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.802+0000 7f5af083d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c60ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.802+0000 7f5af083d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41ad550 con 0x556de38a41b0 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.802+0000 7f5af083d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41ae9c0 con 0x556de387b2f0 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.802+0000 7f5af083d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41adc20 con 0x556de38834d0 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5af083d740 10 client.0 _ll_register_callbacks cb 0x7f5adc009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5ae3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c60ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5ae3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c60ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49622/0 (socket says 172.21.15.63:49622) 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5ae3400640 1 -- 172.21.15.63:0/2705969636 learned_addr learned my addr 172.21.15.63:0/2705969636 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5ae3400640 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de387b2f0 0x556de3c60440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5ae2000640 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c60980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5ae3400640 1 -- 172.21.15.63:0/2705969636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 msgr2=0x556de3c60980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5ae3400640 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c60980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5ae3400640 1 -- 172.21.15.63:0/2705969636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de387b2f0 msgr2=0x556de3c60440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5ae3400640 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de387b2f0 0x556de3c60440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.803+0000 7f5ae3400640 1 -- 172.21.15.63:0/2705969636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556de3ccd760 con 0x556de38a41b0 2024-05-27T08:06:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.804+0000 7f5ae3400640 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c60ec0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f5adc012e90 tx=0x7f5adc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81cd2fdf84fc177 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.804+0000 7f5accc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.804+0000 7f5accc00640 1 -- 172.21.15.63:0/2705969636 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc03e070 con 0x556de38a41b0 2024-05-27T08:06:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.804+0000 7f5accc00640 1 -- 172.21.15.63:0/2705969636 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f5adc0128f0 con 0x556de38a41b0 2024-05-27T08:06:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.804+0000 7f5af083d740 1 -- 172.21.15.63:0/2705969636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5adc004c50 con 0x556de38a41b0 2024-05-27T08:06:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.804+0000 7f5accc00640 1 -- 172.21.15.63:0/2705969636 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5adc02b040 con 0x556de38a41b0 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.804+0000 7f5af083d740 10 client.21759 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.804+0000 7f5af083d740 1 -- 172.21.15.63:0/2705969636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556de3cce9b0 con 0x556de38a41b0 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.805+0000 7f5accc00640 1 -- 172.21.15.63:0/2705969636 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5adc004c50 con 0x556de38a41b0 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.805+0000 7f5accc00640 1 -- 172.21.15.63:0/2705969636 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5adc01b9c0 con 0x556de38a41b0 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.805+0000 7f5af083d740 10 client.21759 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.805+0000 7f5af083d740 10 client.21759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.805+0000 7f5af083d740 20 client.21759 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.805+0000 7f5af083d740 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556de38381c0 0x556de3cd0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.805+0000 7f5af083d740 4 client.21759 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.805+0000 7f5af083d740 1 -- 172.21.15.63:0/2705969636 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556de3bb32c0 con 0x556de38381c0 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.806+0000 7f5ae2000640 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556de38381c0 0x556de3cd0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.807+0000 7f5ae2000640 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556de38381c0 0x556de3cd0df0 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c5b03eb6559488 server_cookie=cfc93db48ac2994d in_seq=0 out_seq=0 2024-05-27T08:06:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.807+0000 7f5accc00640 10 client.21759 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.845+0000 7f5accc00640 1 -- 172.21.15.63:0/2705969636 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556de3bb32c0 con 0x556de38381c0 2024-05-27T08:06:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.845+0000 7f5accc00640 10 client.21759 handle_command_reply: tid=0 2024-05-27T08:06:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.845+0000 7f5af083d740 1 client.21759 shutdown 2024-05-27T08:06:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.845+0000 7f5af083d740 1 -- 172.21.15.63:0/2705969636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 msgr2=0x556de3c60ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.845+0000 7f5af083d740 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c60ec0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f5adc012e90 tx=0x7f5adc010800 comp rx=0 tx=0).stop 2024-05-27T08:06:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.846+0000 7f5af083d740 1 -- 172.21.15.63:0/2705969636 shutdown_connections 2024-05-27T08:06:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.846+0000 7f5af083d740 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556de38381c0 0x556de3cd0df0 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.846+0000 7f5af083d740 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556de38a41b0 0x556de3c60ec0 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.846+0000 7f5af083d740 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556de38834d0 0x556de3c60980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.846+0000 7f5accc00640 0 client.21759 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.846+0000 7f5af083d740 1 --2- 172.21.15.63:0/2705969636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556de387b2f0 0x556de3c60440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.846+0000 7f5af083d740 1 -- 172.21.15.63:0/2705969636 >> 172.21.15.63:0/2705969636 conn(0x556de3898300 msgr2=0x556de3c5b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.846+0000 7f5af083d740 1 -- 172.21.15.63:0/2705969636 shutdown_connections 2024-05-27T08:06:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.846+0000 7f5af083d740 1 -- 172.21.15.63:0/2705969636 wait complete. 2024-05-27T08:06:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.846+0000 7f5af083d740 20 client.21759 trim_cache size 0 max 16384 2024-05-27T08:06:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:06:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:06:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:06:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 -- 172.21.15.63:0/2066377818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ac80b2950 msgr2=0x7f5ac80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ac80b2950 0x7f5ac80b4e60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40dfb00 tx=0x7f5ad4000bc0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 -- 172.21.15.63:0/2066377818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae4104c50 msgr2=0x7f5ae419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae4104c50 0x7f5ae419a5e0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400d950 tx=0x7f5ad400de10 comp rx=0 tx=0).stop 2024-05-27T08:06:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 -- 172.21.15.63:0/2066377818 shutdown_connections 2024-05-27T08:06:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ac80b2950 0x7f5ac80b4e60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae410e010 0x7f5ae419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ae410b6b0 0x7f5ae419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 --2- 172.21.15.63:0/2066377818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae4104c50 0x7f5ae419a5e0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 -- 172.21.15.63:0/2066377818 >> 172.21.15.63:0/2066377818 conn(0x7f5ae40fe210 msgr2=0x7f5ae410e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.847+0000 7f5ae3e00640 1 -- 172.21.15.63:0/2066377818 shutdown_connections 2024-05-27T08:06:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:27.848+0000 7f5ae3e00640 1 -- 172.21.15.63:0/2066377818 wait complete. 2024-05-27T08:06:29.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:29 smithi149 ceph-mon[35408]: pgmap v1043: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.0 MiB/s wr, 39 op/s 2024-05-27T08:06:29.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:29 smithi063 ceph-mon[31613]: pgmap v1043: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.0 MiB/s wr, 39 op/s 2024-05-27T08:06:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:29 smithi086 ceph-mon[34687]: pgmap v1043: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.0 MiB/s wr, 39 op/s 2024-05-27T08:06:31.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:31 smithi086 ceph-mon[34687]: pgmap v1044: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.0 MiB/s wr, 39 op/s 2024-05-27T08:06:31.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:31 smithi149 ceph-mon[35408]: pgmap v1044: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.0 MiB/s wr, 39 op/s 2024-05-27T08:06:31.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:31 smithi063 ceph-mon[31613]: pgmap v1044: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.0 MiB/s wr, 39 op/s 2024-05-27T08:06:33.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:33 smithi086 ceph-mon[34687]: pgmap v1045: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.0 MiB/s wr, 46 op/s 2024-05-27T08:06:33.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:33 smithi149 ceph-mon[35408]: pgmap v1045: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.0 MiB/s wr, 46 op/s 2024-05-27T08:06:33.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:33 smithi063 ceph-mon[31613]: pgmap v1045: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.0 MiB/s wr, 46 op/s 2024-05-27T08:06:35.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:35 smithi063 ceph-mon[31613]: pgmap v1046: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:06:35.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:35 smithi086 ceph-mon[34687]: pgmap v1046: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:06:35.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:35 smithi149 ceph-mon[35408]: pgmap v1046: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:06:37.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:37 smithi086 ceph-mon[34687]: pgmap v1047: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T08:06:37.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:37 smithi149 ceph-mon[35408]: pgmap v1047: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T08:06:37.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:37 smithi063 ceph-mon[31613]: pgmap v1047: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T08:06:39.763 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:39 smithi086 ceph-mon[34687]: pgmap v1048: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:06:39.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:39 smithi149 ceph-mon[35408]: pgmap v1048: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:06:39.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:39 smithi063 ceph-mon[31613]: pgmap v1048: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:06:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.213+0000 7f1fb5000640 1 -- 172.21.15.63:0/1198340766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb010f390 msgr2=0x7f1fb0116b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.213+0000 7f1fb5000640 1 --2- 172.21.15.63:0/1198340766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb010f390 0x7f1fb0116b30 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009900 tx=0x7f1fa802a410 comp rx=0 tx=0).stop 2024-05-27T08:06:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.213+0000 7f1fb5000640 1 -- 172.21.15.63:0/1198340766 shutdown_connections 2024-05-27T08:06:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.213+0000 7f1fb5000640 1 --2- 172.21.15.63:0/1198340766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb010f390 0x7f1fb0116b30 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.213+0000 7f1fb5000640 1 --2- 172.21.15.63:0/1198340766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb010e8d0 0x7f1fb010ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.213+0000 7f1fb5000640 1 --2- 172.21.15.63:0/1198340766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb010a340 0x7f1fb010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.213+0000 7f1fb5000640 1 -- 172.21.15.63:0/1198340766 >> 172.21.15.63:0/1198340766 conn(0x7f1fb0076cd0 msgr2=0x7f1fb0077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.214+0000 7f1fb5000640 1 -- 172.21.15.63:0/1198340766 shutdown_connections 2024-05-27T08:06:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.214+0000 7f1fb5000640 1 -- 172.21.15.63:0/1198340766 wait complete. 2024-05-27T08:06:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.215+0000 7f1fb5000640 1 Processor -- start 2024-05-27T08:06:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.215+0000 7f1fb5000640 1 -- start start 2024-05-27T08:06:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1fb5000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb010a340 0x7f1fb01a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1fb5000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb010e8d0 0x7f1fb01a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1fb5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb010f390 0x7f1fb01a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1fb5000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb011a150 con 0x7f1fb010e8d0 2024-05-27T08:06:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1fb5000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb0119fd0 con 0x7f1fb010f390 2024-05-27T08:06:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1fb5000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb011a2d0 con 0x7f1fb010a340 2024-05-27T08:06:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1faea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb010e8d0 0x7f1fb01a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1faea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb010e8d0 0x7f1fb01a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60434/0 (socket says 172.21.15.63:60434) 2024-05-27T08:06:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1faea00640 1 -- 172.21.15.63:0/79069706 learned_addr learned my addr 172.21.15.63:0/79069706 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1faf400640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb010a340 0x7f1fb01a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.216+0000 7f1fafe00640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb010f390 0x7f1fb01a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.217+0000 7f1faea00640 1 -- 172.21.15.63:0/79069706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb010a340 msgr2=0x7f1fb01a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.217+0000 7f1faea00640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb010a340 0x7f1fb01a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.217+0000 7f1faea00640 1 -- 172.21.15.63:0/79069706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb010f390 msgr2=0x7f1fb01a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.217+0000 7f1faea00640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb010f390 0x7f1fb01a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.217+0000 7f1faea00640 1 -- 172.21.15.63:0/79069706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb01a1130 con 0x7f1fb010e8d0 2024-05-27T08:06:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.217+0000 7f1fafe00640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb010f390 0x7f1fb01a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.217+0000 7f1faea00640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb010e8d0 0x7f1fb01a6dc0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800d740 tx=0x7f1f9800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.218+0000 7f1fa7e00640 1 -- 172.21.15.63:0/79069706 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98017070 con 0x7f1fb010e8d0 2024-05-27T08:06:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.218+0000 7f1fa7e00640 1 -- 172.21.15.63:0/79069706 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f9800de50 con 0x7f1fb010e8d0 2024-05-27T08:06:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.218+0000 7f1fb5000640 1 -- 172.21.15.63:0/79069706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb01a1420 con 0x7f1fb010e8d0 2024-05-27T08:06:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.218+0000 7f1fa7e00640 1 -- 172.21.15.63:0/79069706 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98004c60 con 0x7f1fb010e8d0 2024-05-27T08:06:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.218+0000 7f1fb5000640 1 -- 172.21.15.63:0/79069706 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb01ad700 con 0x7f1fb010e8d0 2024-05-27T08:06:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.223+0000 7f1fbd5c8740 1 Processor -- start 2024-05-27T08:06:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.223+0000 7f1fbd5c8740 1 -- start start 2024-05-27T08:06:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.223+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fe41b0 0x55cd94386a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.223+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fbb2f0 0x55cd94386fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd943874e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cd942f7b90 con 0x55cd93fe41b0 2024-05-27T08:06:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd942e7730 con 0x55cd93fc34d0 2024-05-27T08:06:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cd942eb770 con 0x55cd93fbb2f0 2024-05-27T08:06:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fa7e00640 1 -- 172.21.15.63:0/79069706 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1f98004e00 con 0x7f1fb010e8d0 2024-05-27T08:06:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fafe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fe41b0 0x55cd94386a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fafe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fe41b0 0x55cd94386a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60436/0 (socket says 172.21.15.63:60436) 2024-05-27T08:06:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fafe00640 1 -- 172.21.15.63:0/920970221 learned_addr learned my addr 172.21.15.63:0/920970221 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1faf400640 1 --2- 172.21.15.63:0/920970221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fbb2f0 0x55cd94386fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fafe00640 1 -- 172.21.15.63:0/920970221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fbb2f0 msgr2=0x55cd94386fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fafe00640 1 --2- 172.21.15.63:0/920970221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fbb2f0 0x55cd94386fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fafe00640 1 -- 172.21.15.63:0/920970221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 msgr2=0x55cd943874e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.224+0000 7f1fafe00640 1 --2- 172.21.15.63:0/920970221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd943874e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.225+0000 7f1fafe00640 1 -- 172.21.15.63:0/920970221 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd942808b0 con 0x55cd93fe41b0 2024-05-27T08:06:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.225+0000 7f1fafe00640 1 --2- 172.21.15.63:0/920970221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fe41b0 0x55cd94386a60 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009900 tx=0x7f1fa8032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837dc5225e3db43f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.225+0000 7f1fa4c00640 1 -- 172.21.15.63:0/920970221 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa803f070 con 0x55cd93fe41b0 2024-05-27T08:06:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.225+0000 7f1fa4c00640 1 -- 172.21.15.63:0/920970221 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fa8004030 con 0x55cd93fe41b0 2024-05-27T08:06:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.226+0000 7f1fa4c00640 1 -- 172.21.15.63:0/920970221 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8035070 con 0x55cd93fe41b0 2024-05-27T08:06:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.226+0000 7f1fa7e00640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f940b2950 0x7f1f940b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.226+0000 7f1fa7e00640 1 -- 172.21.15.63:0/79069706 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f1f980e2390 con 0x7f1fb010e8d0 2024-05-27T08:06:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.226+0000 7f1faf400640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f940b2950 0x7f1f940b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.227+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/920970221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fe41b0 msgr2=0x55cd94386a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.227+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/920970221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fe41b0 0x55cd94386a60 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009900 tx=0x7f1fa8032b90 comp rx=0 tx=0).stop 2024-05-27T08:06:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.227+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/920970221 shutdown_connections 2024-05-27T08:06:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.227+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/920970221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd943874e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.227+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/920970221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fbb2f0 0x55cd94386fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.227+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/920970221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fe41b0 0x55cd94386a60 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.227+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/920970221 >> 172.21.15.63:0/920970221 conn(0x55cd93fd8300 msgr2=0x55cd94383210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.227+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/920970221 shutdown_connections 2024-05-27T08:06:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.227+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/920970221 wait complete. 2024-05-27T08:06:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.227+0000 7f1faf400640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f940b2950 0x7f1f940b4e60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x55cd94387e10 tx=0x7f1fa0009500 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.228+0000 7f1fbd5c8740 1 -- start start 2024-05-27T08:06:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.228+0000 7f1fbd5c8740 1 Processor -- start 2024-05-27T08:06:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.229+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94390c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.229+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fc34d0 0x55cd943911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.229+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fe41b0 0x55cd9439cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.229+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cd93fcd790 con 0x55cd93fbb2f0 2024-05-27T08:06:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.229+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cd9428fbc0 con 0x55cd93fe41b0 2024-05-27T08:06:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.229+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cd94393b30 con 0x55cd93fc34d0 2024-05-27T08:06:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.229+0000 7f1fbd5c8740 10 client.0 _ll_register_callbacks cb 0x55cd9426d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.229+0000 7f1faf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94390c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.230+0000 7f1faf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94390c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60450/0 (socket says 172.21.15.63:60450) 2024-05-27T08:06:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.230+0000 7f1faf400640 1 -- 172.21.15.63:0/1702330363 learned_addr learned my addr 172.21.15.63:0/1702330363 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.230+0000 7f1faea00640 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fc34d0 0x55cd943911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.230+0000 7f1fafe00640 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fe41b0 0x55cd9439cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.230+0000 7f1faf400640 1 -- 172.21.15.63:0/1702330363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fc34d0 msgr2=0x55cd943911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.230+0000 7f1faf400640 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fc34d0 0x55cd943911b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.230+0000 7f1faf400640 1 -- 172.21.15.63:0/1702330363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fe41b0 msgr2=0x55cd9439cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.230+0000 7f1faf400640 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fe41b0 0x55cd9439cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.230+0000 7f1faf400640 1 -- 172.21.15.63:0/1702330363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd93ffc210 con 0x55cd93fbb2f0 2024-05-27T08:06:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.230+0000 7f1fafe00640 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fe41b0 0x55cd9439cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.231+0000 7f1faf400640 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94390c70 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0004080 tx=0x7f1fa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d591246e019c3d5a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.231+0000 7f1f9d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.231+0000 7f1f9d600640 1 -- 172.21.15.63:0/1702330363 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa002d060 con 0x55cd93fbb2f0 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.231+0000 7f1f9d600640 1 -- 172.21.15.63:0/1702330363 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f1fa0018070 con 0x55cd93fbb2f0 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.231+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/1702330363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa8004030 con 0x55cd93fbb2f0 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.231+0000 7f1f9d600640 1 -- 172.21.15.63:0/1702330363 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1fa0020840 con 0x55cd93fbb2f0 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.231+0000 7f1fbd5c8740 10 client.21771 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.231+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/1702330363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cd940c8620 con 0x55cd93fbb2f0 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.232+0000 7f1f9d600640 1 -- 172.21.15.63:0/1702330363 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1fa8004030 con 0x55cd93fbb2f0 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.232+0000 7f1f9d600640 1 -- 172.21.15.63:0/1702330363 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1fa001eb30 con 0x55cd93fbb2f0 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.232+0000 7f1fbd5c8740 10 client.21771 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.232+0000 7f1fbd5c8740 10 client.21771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.232+0000 7f1fbd5c8740 20 client.21771 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.232+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cd93fa2d80 0x55cd943a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.232+0000 7f1fbd5c8740 4 client.21771 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.232+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/1702330363 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd942f9ae0 con 0x55cd93fa2d80 2024-05-27T08:06:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.233+0000 7f1faea00640 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cd93fa2d80 0x55cd943a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.234+0000 7f1faea00640 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cd93fa2d80 0x55cd943a2f30 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad8fe594ce2b628 server_cookie=23a98cd592149e18 in_seq=0 out_seq=0 2024-05-27T08:06:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.234+0000 7f1f9d600640 10 client.21771 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1f9d600640 1 -- 172.21.15.63:0/1702330363 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cd942f9ae0 con 0x55cd93fa2d80 2024-05-27T08:06:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1f9d600640 10 client.21771 handle_command_reply: tid=0 2024-05-27T08:06:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1fbd5c8740 1 client.21771 shutdown 2024-05-27T08:06:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/1702330363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 msgr2=0x55cd94390c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94390c70 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0004080 tx=0x7f1fa0004a40 comp rx=0 tx=0).stop 2024-05-27T08:06:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/1702330363 shutdown_connections 2024-05-27T08:06:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cd93fa2d80 0x55cd943a2f30 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fe41b0 0x55cd9439cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fc34d0 0x55cd943911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/1702330363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94390c70 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1f9d600640 0 client.21771 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/1702330363 >> 172.21.15.63:0/1702330363 conn(0x55cd93fd8300 msgr2=0x55cd94388780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.236+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/1702330363 shutdown_connections 2024-05-27T08:06:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.237+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/1702330363 wait complete. 2024-05-27T08:06:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.237+0000 7f1fbd5c8740 20 client.21771 trim_cache size 0 max 16384 2024-05-27T08:06:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.254+0000 7f1fbd5c8740 1 Processor -- start 2024-05-27T08:06:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.254+0000 7f1fbd5c8740 1 -- start start 2024-05-27T08:06:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.254+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fd8300 0x55cd94392dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.254+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd9438c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.254+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd9439cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.254+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b9fb0 con 0x55cd93fbb2f0 2024-05-27T08:06:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.254+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b7880 con 0x55cd93fc34d0 2024-05-27T08:06:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.254+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b56f0 con 0x55cd93fd8300 2024-05-27T08:06:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1faea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd9438c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1faea00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd9438c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60462/0 (socket says 172.21.15.63:60462) 2024-05-27T08:06:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1faea00640 1 -- 172.21.15.63:0/2723631051 learned_addr learned my addr 172.21.15.63:0/2723631051 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1faf400640 1 --2- 172.21.15.63:0/2723631051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fd8300 0x55cd94392dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1fafe00640 1 --2- 172.21.15.63:0/2723631051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd9439cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1faea00640 1 -- 172.21.15.63:0/2723631051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fd8300 msgr2=0x55cd94392dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1faea00640 1 --2- 172.21.15.63:0/2723631051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fd8300 0x55cd94392dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1faea00640 1 -- 172.21.15.63:0/2723631051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 msgr2=0x55cd9439cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1faea00640 1 --2- 172.21.15.63:0/2723631051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd9439cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1faea00640 1 -- 172.21.15.63:0/2723631051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd943863f0 con 0x55cd93fbb2f0 2024-05-27T08:06:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.255+0000 7f1fafe00640 1 --2- 172.21.15.63:0/2723631051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd9439cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:06:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.256+0000 7f1faea00640 1 --2- 172.21.15.63:0/2723631051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd9438c470 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f1f980a52d0 tx=0x7f1f98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce29f78d11d219b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.256+0000 7f1f9fe00640 1 -- 172.21.15.63:0/2723631051 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98020040 con 0x55cd93fbb2f0 2024-05-27T08:06:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.256+0000 7f1f9fe00640 1 -- 172.21.15.63:0/2723631051 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f980e2170 con 0x55cd93fbb2f0 2024-05-27T08:06:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.256+0000 7f1f9fe00640 1 -- 172.21.15.63:0/2723631051 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f980ad120 con 0x55cd93fbb2f0 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.257+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/2723631051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 msgr2=0x55cd9438c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.257+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/2723631051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd9438c470 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f1f980a52d0 tx=0x7f1f98004730 comp rx=0 tx=0).stop 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.257+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/2723631051 shutdown_connections 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.257+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/2723631051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd9439cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.257+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/2723631051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd9438c470 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.257+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/2723631051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fd8300 0x55cd94392dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.257+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/2723631051 >> 172.21.15.63:0/2723631051 conn(0x55cd943890b0 msgr2=0x55cd943868a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.257+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/2723631051 shutdown_connections 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.257+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/2723631051 wait complete. 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.258+0000 7f1fbd5c8740 1 -- start start 2024-05-27T08:06:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.258+0000 7f1fbd5c8740 1 Processor -- start 2024-05-27T08:06:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94399210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd94394560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1fbd5c8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fd8300 0x55cd94394aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b5d40 con 0x55cd93fbb2f0 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b71b0 con 0x55cd93fc34d0 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1fbd5c8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b6410 con 0x55cd93fd8300 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1fbd5c8740 10 client.0 _ll_register_callbacks cb 0x7f1fa0009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1faf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94399210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1faf400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94399210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60476/0 (socket says 172.21.15.63:60476) 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1faf400640 1 -- 172.21.15.63:0/3381577116 learned_addr learned my addr 172.21.15.63:0/3381577116 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1faea00640 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd94394560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.259+0000 7f1fafe00640 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fd8300 0x55cd94394aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.260+0000 7f1faf400640 1 -- 172.21.15.63:0/3381577116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fd8300 msgr2=0x55cd94394aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.260+0000 7f1faf400640 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fd8300 0x55cd94394aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.260+0000 7f1faf400640 1 -- 172.21.15.63:0/3381577116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 msgr2=0x55cd94394560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.260+0000 7f1faf400640 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd94394560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.260+0000 7f1faf400640 1 -- 172.21.15.63:0/3381577116 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd94394fe0 con 0x55cd93fbb2f0 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.260+0000 7f1faf400640 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94399210 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0005b00 tx=0x7f1fa00202d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce24dfe8f540584b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.260+0000 7f1f9d600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.260+0000 7f1f9d600640 1 -- 172.21.15.63:0/3381577116 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0028c50 con 0x55cd93fbb2f0 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.260+0000 7f1f9d600640 1 -- 172.21.15.63:0/3381577116 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f1fa0028df0 con 0x55cd93fbb2f0 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.260+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/3381577116 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cd942f32c0 con 0x55cd93fbb2f0 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.261+0000 7f1f9d600640 1 -- 172.21.15.63:0/3381577116 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1fa00209e0 con 0x55cd93fbb2f0 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.261+0000 7f1fbd5c8740 10 client.21777 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.261+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/3381577116 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cd9440b140 con 0x55cd93fbb2f0 2024-05-27T08:06:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.261+0000 7f1f9d600640 1 -- 172.21.15.63:0/3381577116 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55cd942f32c0 con 0x55cd93fbb2f0 2024-05-27T08:06:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.261+0000 7f1f9d600640 1 -- 172.21.15.63:0/3381577116 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1fa001eb90 con 0x55cd93fbb2f0 2024-05-27T08:06:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.262+0000 7f1fbd5c8740 10 client.21777 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.262+0000 7f1fbd5c8740 10 client.21777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.262+0000 7f1fbd5c8740 20 client.21777 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.262+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cd93fe41b0 0x55cd9440d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.262+0000 7f1fbd5c8740 4 client.21777 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:06:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.262+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/3381577116 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd9426d4b0 con 0x55cd93fe41b0 2024-05-27T08:06:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.262+0000 7f1faea00640 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cd93fe41b0 0x55cd9440d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.263+0000 7f1faea00640 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cd93fe41b0 0x55cd9440d920 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90e42d5ea531ecd server_cookie=9da055fe1c5c844c in_seq=0 out_seq=0 2024-05-27T08:06:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.263+0000 7f1f9d600640 10 client.21777 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1f9d600640 1 -- 172.21.15.63:0/3381577116 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cd9426d4b0 con 0x55cd93fe41b0 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1f9d600640 10 client.21777 handle_command_reply: tid=0 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1fbd5c8740 1 client.21777 shutdown 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/3381577116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 msgr2=0x55cd94399210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94399210 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0005b00 tx=0x7f1fa00202d0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/3381577116 shutdown_connections 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cd93fe41b0 0x55cd9440d920 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cd93fd8300 0x55cd94394aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cd93fc34d0 0x55cd94394560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1fbd5c8740 1 --2- 172.21.15.63:0/3381577116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd93fbb2f0 0x55cd94399210 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.298+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/3381577116 >> 172.21.15.63:0/3381577116 conn(0x55cd943890b0 msgr2=0x55cd94392ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.299+0000 7f1f9d600640 0 client.21777 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.299+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/3381577116 shutdown_connections 2024-05-27T08:06:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.299+0000 7f1fbd5c8740 1 -- 172.21.15.63:0/3381577116 wait complete. 2024-05-27T08:06:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.299+0000 7f1fbd5c8740 20 client.21777 trim_cache size 0 max 16384 2024-05-27T08:06:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:06:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:06:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:06:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 -- 172.21.15.63:0/79069706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f940b2950 msgr2=0x7f1f940b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f940b2950 0x7f1f940b4e60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x55cd94387e10 tx=0x7f1fa0009500 comp rx=0 tx=0).stop 2024-05-27T08:06:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 -- 172.21.15.63:0/79069706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb010e8d0 msgr2=0x7f1fb01a6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb010e8d0 0x7f1fb01a6dc0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800d740 tx=0x7f1f9800dc00 comp rx=0 tx=0).stop 2024-05-27T08:06:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 -- 172.21.15.63:0/79069706 shutdown_connections 2024-05-27T08:06:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f940b2950 0x7f1f940b4e60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fb010f390 0x7f1fb01a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fb010e8d0 0x7f1fb01a6dc0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 --2- 172.21.15.63:0/79069706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fb010a340 0x7f1fb01a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 -- 172.21.15.63:0/79069706 >> 172.21.15.63:0/79069706 conn(0x7f1fb0076cd0 msgr2=0x7f1fb0107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 -- 172.21.15.63:0/79069706 shutdown_connections 2024-05-27T08:06:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:41.300+0000 7f1fb5000640 1 -- 172.21.15.63:0/79069706 wait complete. 2024-05-27T08:06:41.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:41 smithi086 ceph-mon[34687]: pgmap v1049: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:06:41.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:41 smithi149 ceph-mon[35408]: pgmap v1049: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:06:41.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:41 smithi063 ceph-mon[31613]: pgmap v1049: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:06:43.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:43 smithi086 ceph-mon[34687]: pgmap v1050: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T08:06:43.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:43 smithi149 ceph-mon[35408]: pgmap v1050: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T08:06:43.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:43 smithi063 ceph-mon[31613]: pgmap v1050: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 16 op/s 2024-05-27T08:06:45.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:45 smithi086 ceph-mon[34687]: pgmap v1051: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:06:45.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:45 smithi149 ceph-mon[35408]: pgmap v1051: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:06:45.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:45 smithi063 ceph-mon[31613]: pgmap v1051: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:06:47.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:47 smithi086 ceph-mon[34687]: pgmap v1052: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T08:06:47.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:47 smithi149 ceph-mon[35408]: pgmap v1052: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T08:06:47.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:47 smithi063 ceph-mon[31613]: pgmap v1052: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T08:06:49.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:49 smithi086 ceph-mon[34687]: pgmap v1053: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:06:49.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:49 smithi149 ceph-mon[35408]: pgmap v1053: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:06:49.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:49 smithi063 ceph-mon[31613]: pgmap v1053: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:06:51.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:51 smithi086 ceph-mon[34687]: pgmap v1054: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:06:51.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:51 smithi149 ceph-mon[35408]: pgmap v1054: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:06:51.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:51 smithi063 ceph-mon[31613]: pgmap v1054: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.7 MiB/s wr, 9 op/s 2024-05-27T08:06:52.835 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:06:53.387 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:53.805 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:53 smithi063 ceph-mon[31613]: pgmap v1055: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T08:06:53.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:53 smithi086 ceph-mon[34687]: pgmap v1055: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T08:06:53.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:53 smithi149 ceph-mon[35408]: pgmap v1055: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 13 op/s 2024-05-27T08:06:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.541+0000 7f8e84c00640 1 -- 172.21.15.63:0/1453334242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8010beb0 msgr2=0x7f8e80105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.541+0000 7f8e84c00640 1 --2- 172.21.15.63:0/1453334242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8010beb0 0x7f8e80105a50 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009900 tx=0x7f8e7002a410 comp rx=0 tx=0).stop 2024-05-27T08:06:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.541+0000 7f8e84c00640 1 -- 172.21.15.63:0/1453334242 shutdown_connections 2024-05-27T08:06:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.541+0000 7f8e84c00640 1 --2- 172.21.15.63:0/1453334242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e80106600 0x7f8e801129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.541+0000 7f8e84c00640 1 --2- 172.21.15.63:0/1453334242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e8010c350 0x7f8e80105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.541+0000 7f8e84c00640 1 --2- 172.21.15.63:0/1453334242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8010beb0 0x7f8e80105a50 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.541+0000 7f8e84c00640 1 -- 172.21.15.63:0/1453334242 >> 172.21.15.63:0/1453334242 conn(0x7f8e801016b0 msgr2=0x7f8e80103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.541+0000 7f8e84c00640 1 -- 172.21.15.63:0/1453334242 shutdown_connections 2024-05-27T08:06:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.541+0000 7f8e84c00640 1 -- 172.21.15.63:0/1453334242 wait complete. 2024-05-27T08:06:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.543+0000 7f8e84c00640 1 Processor -- start 2024-05-27T08:06:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.543+0000 7f8e84c00640 1 -- start start 2024-05-27T08:06:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.543+0000 7f8e84c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e80106600 0x7f8e801a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.543+0000 7f8e84c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e8010beb0 0x7f8e801a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.543+0000 7f8e84c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8010c350 0x7f8e8019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.543+0000 7f8e84c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80115f60 con 0x7f8e8010c350 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.543+0000 7f8e84c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80115de0 con 0x7f8e80106600 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.543+0000 7f8e84c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8e801160e0 con 0x7f8e8010beb0 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8010c350 0x7f8e8019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8010c350 0x7f8e8019c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34706/0 (socket says 172.21.15.63:34706) 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7e800640 1 -- 172.21.15.63:0/3494881831 learned_addr learned my addr 172.21.15.63:0/3494881831 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7de00640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e80106600 0x7f8e801a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7d400640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e8010beb0 0x7f8e801a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7e800640 1 -- 172.21.15.63:0/3494881831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e8010beb0 msgr2=0x7f8e801a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7e800640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e8010beb0 0x7f8e801a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7e800640 1 -- 172.21.15.63:0/3494881831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e80106600 msgr2=0x7f8e801a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7e800640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e80106600 0x7f8e801a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7e800640 1 -- 172.21.15.63:0/3494881831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8019cf00 con 0x7f8e8010c350 2024-05-27T08:06:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.544+0000 7f8e7de00640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e80106600 0x7f8e801a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:06:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.545+0000 7f8e7e800640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8010c350 0x7f8e8019c860 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800eaa0 tx=0x7f8e7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.545+0000 7f8e76a00640 1 -- 172.21.15.63:0/3494881831 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800cdb0 con 0x7f8e8010c350 2024-05-27T08:06:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.545+0000 7f8e76a00640 1 -- 172.21.15.63:0/3494881831 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e78005020 con 0x7f8e8010c350 2024-05-27T08:06:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.545+0000 7f8e76a00640 1 -- 172.21.15.63:0/3494881831 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e780116d0 con 0x7f8e8010c350 2024-05-27T08:06:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.545+0000 7f8e84c00640 1 -- 172.21.15.63:0/3494881831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8019d1f0 con 0x7f8e8010c350 2024-05-27T08:06:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.545+0000 7f8e84c00640 1 -- 172.21.15.63:0/3494881831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e801a94d0 con 0x7f8e8010c350 2024-05-27T08:06:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.549+0000 7f8e8bd7b740 1 Processor -- start 2024-05-27T08:06:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.550+0000 7f8e76a00640 1 -- 172.21.15.63:0/3494881831 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8e780051d0 con 0x7f8e8010c350 2024-05-27T08:06:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.550+0000 7f8e8bd7b740 1 -- start start 2024-05-27T08:06:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.551+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd4216860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.551+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd4216da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.551+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e534d0 0x561dd42172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.551+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561dd4187b90 con 0x561dd3e4b2f0 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.551+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561dd4177730 con 0x561dd3e741b0 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.551+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561dd417b770 con 0x561dd3e534d0 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.551+0000 7f8e7de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd4216da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.551+0000 7f8e7de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd4216da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34708/0 (socket says 172.21.15.63:34708) 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.551+0000 7f8e7de00640 1 -- 172.21.15.63:0/4060422134 learned_addr learned my addr 172.21.15.63:0/4060422134 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.552+0000 7f8e7d400640 1 --2- 172.21.15.63:0/4060422134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd4216860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.552+0000 7f8e7d400640 1 --2- 172.21.15.63:0/4060422134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e534d0 0x561dd42172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.552+0000 7f8e7de00640 1 -- 172.21.15.63:0/4060422134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e534d0 msgr2=0x561dd42172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.552+0000 7f8e7de00640 1 --2- 172.21.15.63:0/4060422134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e534d0 0x561dd42172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.552+0000 7f8e7de00640 1 -- 172.21.15.63:0/4060422134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 msgr2=0x561dd4216860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.552+0000 7f8e7de00640 1 --2- 172.21.15.63:0/4060422134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd4216860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.552+0000 7f8e7de00640 1 -- 172.21.15.63:0/4060422134 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd41108b0 con 0x561dd3e4b2f0 2024-05-27T08:06:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.552+0000 7f8e76a00640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e640b29d0 0x7f8e640b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.553+0000 7f8e76a00640 1 -- 172.21.15.63:0/3494881831 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f8e780e2c10 con 0x7f8e8010c350 2024-05-27T08:06:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.553+0000 7f8e7de00640 1 --2- 172.21.15.63:0/4060422134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd4216da0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f8e80106b90 tx=0x7f8e70032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3abd1c4dc35f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.553+0000 7f8e7d400640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e640b29d0 0x7f8e640b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.553+0000 7f8e6b000640 1 -- 172.21.15.63:0/4060422134 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7003f070 con 0x561dd3e4b2f0 2024-05-27T08:06:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.553+0000 7f8e6b000640 1 -- 172.21.15.63:0/4060422134 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e700044c0 con 0x561dd3e4b2f0 2024-05-27T08:06:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.553+0000 7f8e6b000640 1 -- 172.21.15.63:0/4060422134 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70035070 con 0x561dd3e4b2f0 2024-05-27T08:06:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.554+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/4060422134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 msgr2=0x561dd4216da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.554+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/4060422134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd4216da0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f8e80106b90 tx=0x7f8e70032c40 comp rx=0 tx=0).stop 2024-05-27T08:06:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.554+0000 7f8e7d400640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e640b29d0 0x7f8e640b4ee0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x561dd42139d0 tx=0x7f8e6c004730 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.555+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/4060422134 shutdown_connections 2024-05-27T08:06:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.555+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/4060422134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e534d0 0x561dd42172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.555+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/4060422134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd4216da0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.555+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/4060422134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd4216860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.555+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/4060422134 >> 172.21.15.63:0/4060422134 conn(0x561dd3e68300 msgr2=0x561dd4213010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.555+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/4060422134 shutdown_connections 2024-05-27T08:06:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.555+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/4060422134 wait complete. 2024-05-27T08:06:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.556+0000 7f8e8bd7b740 1 -- start start 2024-05-27T08:06:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.556+0000 7f8e8bd7b740 1 Processor -- start 2024-05-27T08:06:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.556+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e4b2f0 0x561dd4220a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e534d0 0x561dd4220f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd422cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561dd3e5d790 con 0x561dd3e534d0 2024-05-27T08:06:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561dd411fbc0 con 0x561dd3e741b0 2024-05-27T08:06:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561dd4223930 con 0x561dd3e4b2f0 2024-05-27T08:06:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e8bd7b740 10 client.0 _ll_register_callbacks cb 0x561dd40fd400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e7d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e534d0 0x561dd4220f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e7d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e534d0 0x561dd4220f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34714/0 (socket says 172.21.15.63:34714) 2024-05-27T08:06:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e7d400640 1 -- 172.21.15.63:0/1115998130 learned_addr learned my addr 172.21.15.63:0/1115998130 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e7de00640 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e4b2f0 0x561dd4220a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.557+0000 7f8e7e800640 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd422cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.558+0000 7f8e7d400640 1 -- 172.21.15.63:0/1115998130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e4b2f0 msgr2=0x561dd4220a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.558+0000 7f8e7d400640 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e4b2f0 0x561dd4220a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.558+0000 7f8e7d400640 1 -- 172.21.15.63:0/1115998130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 msgr2=0x561dd422cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.558+0000 7f8e7d400640 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd422cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.558+0000 7f8e7d400640 1 -- 172.21.15.63:0/1115998130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd3e8c210 con 0x561dd3e534d0 2024-05-27T08:06:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.558+0000 7f8e7d400640 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e534d0 0x561dd4220f80 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c011910 tx=0x7f8e6c00c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1ed778092e67d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.558+0000 7f8e5f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.559+0000 7f8e5f400640 1 -- 172.21.15.63:0/1115998130 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c00fdb0 con 0x561dd3e534d0 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.559+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/1115998130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561dd4189ae0 con 0x561dd3e534d0 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.559+0000 7f8e5f400640 1 -- 172.21.15.63:0/1115998130 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f8e6c004030 con 0x561dd3e534d0 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.559+0000 7f8e5f400640 1 -- 172.21.15.63:0/1115998130 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e6c031050 con 0x561dd3e534d0 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.559+0000 7f8e8bd7b740 10 client.21789 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.559+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/1115998130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561dd3f58620 con 0x561dd3e534d0 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.559+0000 7f8e5f400640 1 -- 172.21.15.63:0/1115998130 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561dd4189ae0 con 0x561dd3e534d0 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.560+0000 7f8e5f400640 1 -- 172.21.15.63:0/1115998130 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8e6c003c90 con 0x561dd3e534d0 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.560+0000 7f8e8bd7b740 10 client.21789 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.560+0000 7f8e8bd7b740 10 client.21789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.560+0000 7f8e8bd7b740 20 client.21789 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.560+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561dd3e32d80 0x561dd4232d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.560+0000 7f8e8bd7b740 4 client.21789 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.560+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/1115998130 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dd41832c0 con 0x561dd3e32d80 2024-05-27T08:06:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.561+0000 7f8e7de00640 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561dd3e32d80 0x561dd4232d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.562+0000 7f8e5f400640 10 client.21789 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.562+0000 7f8e7de00640 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561dd3e32d80 0x561dd4232d10 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42379743967a8e88 server_cookie=493ab047b32cdce1 in_seq=0 out_seq=0 2024-05-27T08:06:54.566 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e5f400640 1 -- 172.21.15.63:0/1115998130 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561dd41832c0 con 0x561dd3e32d80 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e5f400640 10 client.21789 handle_command_reply: tid=0 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e8bd7b740 1 client.21789 shutdown 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/1115998130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e534d0 msgr2=0x561dd4220f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e534d0 0x561dd4220f80 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c011910 tx=0x7f8e6c00c3b0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/1115998130 shutdown_connections 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561dd3e32d80 0x561dd4232d10 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd422cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e534d0 0x561dd4220f80 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/1115998130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e4b2f0 0x561dd4220a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/1115998130 >> 172.21.15.63:0/1115998130 conn(0x561dd3e68300 msgr2=0x561dd4216800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.564+0000 7f8e5f400640 0 client.21789 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.565+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/1115998130 shutdown_connections 2024-05-27T08:06:54.567 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.565+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/1115998130 wait complete. 2024-05-27T08:06:54.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.565+0000 7f8e8bd7b740 20 client.21789 trim_cache size 0 max 16384 2024-05-27T08:06:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.588+0000 7f8e8bd7b740 1 Processor -- start 2024-05-27T08:06:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.588+0000 7f8e8bd7b740 1 -- start start 2024-05-27T08:06:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.588+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd421c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.588+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e534d0 0x561dd422ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.588+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e741b0 0x561dd422cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.588+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e801b5d70 con 0x561dd3e4b2f0 2024-05-27T08:06:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.588+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e801b3640 con 0x561dd3e534d0 2024-05-27T08:06:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.588+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8e801b14b0 con 0x561dd3e741b0 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.589+0000 7f8e7de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd421c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.589+0000 7f8e7de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd421c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34716/0 (socket says 172.21.15.63:34716) 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.589+0000 7f8e7de00640 1 -- 172.21.15.63:0/2311753126 learned_addr learned my addr 172.21.15.63:0/2311753126 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.589+0000 7f8e7d400640 1 --2- 172.21.15.63:0/2311753126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e534d0 0x561dd422ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.589+0000 7f8e7e800640 1 --2- 172.21.15.63:0/2311753126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e741b0 0x561dd422cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.589+0000 7f8e7de00640 1 -- 172.21.15.63:0/2311753126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e741b0 msgr2=0x561dd422cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.589+0000 7f8e7de00640 1 --2- 172.21.15.63:0/2311753126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e741b0 0x561dd422cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.589+0000 7f8e7de00640 1 -- 172.21.15.63:0/2311753126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e534d0 msgr2=0x561dd422ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.589+0000 7f8e7de00640 1 --2- 172.21.15.63:0/2311753126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e534d0 0x561dd422ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.589+0000 7f8e7de00640 1 -- 172.21.15.63:0/2311753126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd420d050 con 0x561dd3e4b2f0 2024-05-27T08:06:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.590+0000 7f8e7de00640 1 --2- 172.21.15.63:0/2311753126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd421c280 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f8e70033000 tx=0x7f8e7002aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c660122601120cf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.590+0000 7f8e6a600640 1 -- 172.21.15.63:0/2311753126 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7003f040 con 0x561dd3e4b2f0 2024-05-27T08:06:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.590+0000 7f8e6a600640 1 -- 172.21.15.63:0/2311753126 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e700046c0 con 0x561dd3e4b2f0 2024-05-27T08:06:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.590+0000 7f8e6a600640 1 -- 172.21.15.63:0/2311753126 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70035040 con 0x561dd3e4b2f0 2024-05-27T08:06:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.591+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/2311753126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 msgr2=0x561dd421c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.591+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/2311753126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd421c280 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f8e70033000 tx=0x7f8e7002aa90 comp rx=0 tx=0).stop 2024-05-27T08:06:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.591+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/2311753126 shutdown_connections 2024-05-27T08:06:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.591+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/2311753126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e741b0 0x561dd422cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.591+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/2311753126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e534d0 0x561dd422ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.591+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/2311753126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd421c280 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.591+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/2311753126 >> 172.21.15.63:0/2311753126 conn(0x561dd3e68300 msgr2=0x561dd4220d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.591+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/2311753126 shutdown_connections 2024-05-27T08:06:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.591+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/2311753126 wait complete. 2024-05-27T08:06:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.592+0000 7f8e8bd7b740 1 -- start start 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.592+0000 7f8e8bd7b740 1 Processor -- start 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.593+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd42260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.593+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e534d0 0x561dd4226630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.593+0000 7f8e8bd7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd4299430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.593+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e801b1b00 con 0x561dd3e4b2f0 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.593+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e801b2f70 con 0x561dd3e741b0 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.593+0000 7f8e8bd7b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8e801b21d0 con 0x561dd3e534d0 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.593+0000 7f8e8bd7b740 10 client.0 _ll_register_callbacks cb 0x7f8e6c0115c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.593+0000 7f8e7de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd42260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.593+0000 7f8e7de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd42260f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34718/0 (socket says 172.21.15.63:34718) 2024-05-27T08:06:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.593+0000 7f8e7de00640 1 -- 172.21.15.63:0/108559656 learned_addr learned my addr 172.21.15.63:0/108559656 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.594+0000 7f8e7d400640 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e534d0 0x561dd4226630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.594+0000 7f8e7e800640 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd4299430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.594+0000 7f8e7de00640 1 -- 172.21.15.63:0/108559656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e534d0 msgr2=0x561dd4226630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.594+0000 7f8e7de00640 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e534d0 0x561dd4226630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.594+0000 7f8e7de00640 1 -- 172.21.15.63:0/108559656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 msgr2=0x561dd4299430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.594+0000 7f8e7de00640 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd4299430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.594+0000 7f8e7de00640 1 -- 172.21.15.63:0/108559656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd4299970 con 0x561dd3e4b2f0 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.594+0000 7f8e7de00640 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd42260f0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f8e700057c0 tx=0x7f8e70032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525a595c09da4a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.594+0000 7f8e5f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.595+0000 7f8e5f400640 1 -- 172.21.15.63:0/108559656 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7003f030 con 0x561dd3e4b2f0 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.595+0000 7f8e5f400640 1 -- 172.21.15.63:0/108559656 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f8e70031810 con 0x561dd3e4b2f0 2024-05-27T08:06:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.595+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/108559656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e700046c0 con 0x561dd3e4b2f0 2024-05-27T08:06:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.595+0000 7f8e5f400640 1 -- 172.21.15.63:0/108559656 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e70068050 con 0x561dd3e4b2f0 2024-05-27T08:06:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.595+0000 7f8e8bd7b740 10 client.21795 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.595+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/108559656 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561dd429ab90 con 0x561dd3e4b2f0 2024-05-27T08:06:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.595+0000 7f8e5f400640 1 -- 172.21.15.63:0/108559656 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e700046c0 con 0x561dd3e4b2f0 2024-05-27T08:06:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.596+0000 7f8e5f400640 1 -- 172.21.15.63:0/108559656 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8e7003b7d0 con 0x561dd3e4b2f0 2024-05-27T08:06:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.596+0000 7f8e8bd7b740 10 client.21795 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.596+0000 7f8e8bd7b740 10 client.21795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.596+0000 7f8e8bd7b740 20 client.21795 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.596+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561dd3e081c0 0x561dd429ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.596+0000 7f8e8bd7b740 4 client.21795 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:06:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.596+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/108559656 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561dd40fd4b0 con 0x561dd3e081c0 2024-05-27T08:06:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.596+0000 7f8e7d400640 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561dd3e081c0 0x561dd429ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.597+0000 7f8e7d400640 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561dd3e081c0 0x561dd429ce20 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92ebc86d1fe0d1b server_cookie=cee72ee79bd76a23 in_seq=0 out_seq=0 2024-05-27T08:06:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.597+0000 7f8e5f400640 10 client.21795 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e5f400640 1 -- 172.21.15.63:0/108559656 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561dd40fd4b0 con 0x561dd3e081c0 2024-05-27T08:06:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e5f400640 10 client.21795 handle_command_reply: tid=0 2024-05-27T08:06:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e8bd7b740 1 client.21795 shutdown 2024-05-27T08:06:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/108559656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 msgr2=0x561dd42260f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd42260f0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f8e700057c0 tx=0x7f8e70032dc0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/108559656 shutdown_connections 2024-05-27T08:06:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561dd3e081c0 0x561dd429ce20 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561dd3e741b0 0x561dd4299430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561dd3e534d0 0x561dd4226630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e8bd7b740 1 --2- 172.21.15.63:0/108559656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561dd3e4b2f0 0x561dd42260f0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/108559656 >> 172.21.15.63:0/108559656 conn(0x561dd3e68300 msgr2=0x561dd421b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e5f400640 0 client.21795 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.630+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/108559656 shutdown_connections 2024-05-27T08:06:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.631+0000 7f8e8bd7b740 1 -- 172.21.15.63:0/108559656 wait complete. 2024-05-27T08:06:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.631+0000 7f8e8bd7b740 20 client.21795 trim_cache size 0 max 16384 2024-05-27T08:06:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:06:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:06:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.631+0000 7f8e84c00640 1 -- 172.21.15.63:0/3494881831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e640b29d0 msgr2=0x7f8e640b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e640b29d0 0x7f8e640b4ee0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x561dd42139d0 tx=0x7f8e6c004730 comp rx=0 tx=0).stop 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 -- 172.21.15.63:0/3494881831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8010c350 msgr2=0x7f8e8019c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8010c350 0x7f8e8019c860 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800eaa0 tx=0x7f8e7800ef60 comp rx=0 tx=0).stop 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 -- 172.21.15.63:0/3494881831 shutdown_connections 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e640b29d0 0x7f8e640b4ee0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8010c350 0x7f8e8019c860 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e8010beb0 0x7f8e801a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 --2- 172.21.15.63:0/3494881831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e80106600 0x7f8e801a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 -- 172.21.15.63:0/3494881831 >> 172.21.15.63:0/3494881831 conn(0x7f8e801016b0 msgr2=0x7f8e80101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 -- 172.21.15.63:0/3494881831 shutdown_connections 2024-05-27T08:06:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:06:54.632+0000 7f8e84c00640 1 -- 172.21.15.63:0/3494881831 wait complete. 2024-05-27T08:06:55.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:55 smithi063 ceph-mon[31613]: pgmap v1056: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T08:06:55.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:55 smithi086 ceph-mon[34687]: pgmap v1056: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T08:06:55.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:55 smithi149 ceph-mon[35408]: pgmap v1056: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.7 MiB/s wr, 7 op/s 2024-05-27T08:06:57.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.065+0000 7f4e8c400700 1 -- 172.21.15.63:0/992508736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e841045e0 msgr2=0x7f4e841049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.065+0000 7f4e8c400700 1 --2- 172.21.15.63:0/992508736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e841045e0 0x7f4e841049f0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f4e780099c0 tx=0x7f4e78009cd0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.065+0000 7f4e8c400700 1 -- 172.21.15.63:0/992508736 shutdown_connections 2024-05-27T08:06:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.065+0000 7f4e8c400700 1 --2- 172.21.15.63:0/992508736 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e84109860 0x7f4e841111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.065+0000 7f4e8c400700 1 --2- 172.21.15.63:0/992508736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e841045e0 0x7f4e841049f0 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.065+0000 7f4e8c400700 1 --2- 172.21.15.63:0/992508736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84108f10 0x7f4e84109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.065+0000 7f4e8c400700 1 -- 172.21.15.63:0/992508736 >> 172.21.15.63:0/992508736 conn(0x7f4e84100030 msgr2=0x7f4e84102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.065+0000 7f4e8c400700 1 -- 172.21.15.63:0/992508736 shutdown_connections 2024-05-27T08:06:57.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.066+0000 7f4e8c400700 1 -- 172.21.15.63:0/992508736 wait complete. 2024-05-27T08:06:57.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.067+0000 7f4e8c400700 1 Processor -- start 2024-05-27T08:06:57.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.067+0000 7f4e8c400700 1 -- start start 2024-05-27T08:06:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.068+0000 7f4e8c400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e841045e0 0x7f4e84075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.068+0000 7f4e8c400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e84108f10 0x7f4e84075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.068+0000 7f4e8c400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84109860 0x7f4e84076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.068+0000 7f4e8c400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84114390 con 0x7f4e841045e0 2024-05-27T08:06:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.068+0000 7f4e8c400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84114210 con 0x7f4e84109860 2024-05-27T08:06:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.068+0000 7f4e8c400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84114510 con 0x7f4e84108f10 2024-05-27T08:06:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.069+0000 7f4e89a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e841045e0 0x7f4e84075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.069+0000 7f4e89a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e841045e0 0x7f4e84075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34742/0 (socket says 172.21.15.63:34742) 2024-05-27T08:06:57.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.069+0000 7f4e89a00700 1 -- 172.21.15.63:0/3726860237 learned_addr learned my addr 172.21.15.63:0/3726860237 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.069+0000 7f4e89000700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e84108f10 0x7f4e84075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.069+0000 7f4e8a400700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84109860 0x7f4e84076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.070+0000 7f4e89a00700 1 -- 172.21.15.63:0/3726860237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e84108f10 msgr2=0x7f4e84075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.070+0000 7f4e89a00700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e84108f10 0x7f4e84075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.070+0000 7f4e89a00700 1 -- 172.21.15.63:0/3726860237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84109860 msgr2=0x7f4e84076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.070+0000 7f4e89a00700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84109860 0x7f4e84076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.070+0000 7f4e89a00700 1 -- 172.21.15.63:0/3726860237 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e841a9140 con 0x7f4e841045e0 2024-05-27T08:06:57.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.070+0000 7f4e89a00700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e841045e0 0x7f4e84075ab0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b580 tx=0x7f4e780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.071+0000 7f4e82000700 1 -- 172.21.15.63:0/3726860237 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78027070 con 0x7f4e841045e0 2024-05-27T08:06:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.071+0000 7f4e8c400700 1 -- 172.21.15.63:0/3726860237 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e841a93d0 con 0x7f4e841045e0 2024-05-27T08:06:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.071+0000 7f4e82000700 1 -- 172.21.15.63:0/3726860237 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e7800fc80 con 0x7f4e841045e0 2024-05-27T08:06:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.071+0000 7f4e82000700 1 -- 172.21.15.63:0/3726860237 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78022d30 con 0x7f4e841045e0 2024-05-27T08:06:57.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.071+0000 7f4e8c400700 1 -- 172.21.15.63:0/3726860237 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e841a9880 con 0x7f4e841045e0 2024-05-27T08:06:57.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.077+0000 7f4e82000700 1 -- 172.21.15.63:0/3726860237 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4e7802a070 con 0x7f4e841045e0 2024-05-27T08:06:57.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.077+0000 7f4e9afc9b80 1 Processor -- start 2024-05-27T08:06:57.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.078+0000 7f4e9afc9b80 1 -- start start 2024-05-27T08:06:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.078+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.078+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6bc4e0 0x5625cf6b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.078+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 0x5625cf6b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.078+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625cf5dbe40 con 0x5625cf6b65c0 2024-05-27T08:06:57.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.078+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5625cf5cb9e0 con 0x5625cf6bc4e0 2024-05-27T08:06:57.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.078+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5625cf5cfa20 con 0x5625cf52ee40 2024-05-27T08:06:57.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.078+0000 7f4e82000700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e6c0b2a70 0x7f4e6c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.079+0000 7f4e8a400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.079+0000 7f4e82000700 1 -- 172.21.15.63:0/3726860237 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f4e780e3750 con 0x7f4e841045e0 2024-05-27T08:06:57.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.079+0000 7f4e8a400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41806/0 (socket says 172.21.15.63:41806) 2024-05-27T08:06:57.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.079+0000 7f4e8a400700 1 -- 172.21.15.63:0/3970299852 learned_addr learned my addr 172.21.15.63:0/3970299852 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:57.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.079+0000 7f4e8a400700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e6c0b2a70 0x7f4e6c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.079+0000 7f4e89000700 1 --2- 172.21.15.63:0/3970299852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 0x5625cf6b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.079+0000 7f4e89a00700 1 --2- 172.21.15.63:0/3970299852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6bc4e0 0x5625cf6b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.080+0000 7f4e8a400700 1 -- 172.21.15.63:0/3970299852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6bc4e0 msgr2=0x5625cf6b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:57.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.080+0000 7f4e8a400700 1 --2- 172.21.15.63:0/3970299852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6bc4e0 0x5625cf6b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.080+0000 7f4e8a400700 1 -- 172.21.15.63:0/3970299852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 msgr2=0x5625cf6b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:57.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.080+0000 7f4e8a400700 1 --2- 172.21.15.63:0/3970299852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 0x5625cf6b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.080+0000 7f4e8a400700 1 -- 172.21.15.63:0/3970299852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625cf594420 con 0x5625cf52ee40 2024-05-27T08:06:57.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.080+0000 7f4e8a400700 1 --2- 172.21.15.63:0/3970299852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bbfa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c007cd0 tx=0x7f4e7c019370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe08c5436ba9bd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:57.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.081+0000 7f4e8a400700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e6c0b2a70 0x7f4e6c0b4f80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009fd0 tx=0x7f4e7c00d000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:57.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.081+0000 7f4e76a00700 1 -- 172.21.15.63:0/3970299852 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c01f070 con 0x5625cf52ee40 2024-05-27T08:06:57.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.081+0000 7f4e76a00700 1 -- 172.21.15.63:0/3970299852 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e7c009af0 con 0x5625cf52ee40 2024-05-27T08:06:57.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.081+0000 7f4e76a00700 1 -- 172.21.15.63:0/3970299852 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c004e10 con 0x5625cf52ee40 2024-05-27T08:06:57.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.082+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/3970299852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 msgr2=0x5625cf6bbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:57.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.082+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/3970299852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bbfa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c007cd0 tx=0x7f4e7c019370 comp rx=0 tx=0).stop 2024-05-27T08:06:57.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.082+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/3970299852 shutdown_connections 2024-05-27T08:06:57.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.082+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/3970299852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 0x5625cf6b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.082+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/3970299852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bbfa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.082+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/3970299852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6bc4e0 0x5625cf6b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.082+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/3970299852 >> 172.21.15.63:0/3970299852 conn(0x5625cf58efc0 msgr2=0x5625cf6a88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:57.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.083+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/3970299852 shutdown_connections 2024-05-27T08:06:57.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.083+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/3970299852 wait complete. 2024-05-27T08:06:57.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.083+0000 7f4e9afc9b80 1 -- start start 2024-05-27T08:06:57.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.084+0000 7f4e9afc9b80 1 Processor -- start 2024-05-27T08:06:57.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.084+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.084+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6b65c0 0x5625cf6c0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625cf4d1fa0 con 0x5625cf6bc4e0 2024-05-27T08:06:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5625cf56c760 con 0x5625cf6b65c0 2024-05-27T08:06:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5625cf6c2f60 con 0x5625cf52ee40 2024-05-27T08:06:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e9afc9b80 10 client.0 _ll_register_callbacks cb 0x5625cf575c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e89a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e89000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6b65c0 0x5625cf6c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e8a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e89a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41816/0 (socket says 172.21.15.63:41816) 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e89000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6b65c0 0x5625cf6c0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:37432/0 (socket says 172.21.15.63:37432) 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.085+0000 7f4e89a00700 1 -- 172.21.15.63:0/4268219773 learned_addr learned my addr 172.21.15.63:0/4268219773 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.086+0000 7f4e89a00700 1 -- 172.21.15.63:0/4268219773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6b65c0 msgr2=0x5625cf6c0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.086+0000 7f4e89a00700 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6b65c0 0x5625cf6c0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.086+0000 7f4e89a00700 1 -- 172.21.15.63:0/4268219773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 msgr2=0x5625cf6c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.086+0000 7f4e89a00700 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.086+0000 7f4e89a00700 1 -- 172.21.15.63:0/4268219773 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625cf51b8d0 con 0x5625cf52ee40 2024-05-27T08:06:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.086+0000 7f4e8a400700 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:06:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.087+0000 7f4e89a00700 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bfce0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e780a6720 tx=0x7f4e7801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a24ad8ff36c90bf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.087+0000 7f4e6b400700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:06:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.087+0000 7f4e6b400700 1 -- 172.21.15.63:0/4268219773 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e780a9ff0 con 0x5625cf52ee40 2024-05-27T08:06:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.087+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/4268219773 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e7c009af0 con 0x5625cf52ee40 2024-05-27T08:06:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.087+0000 7f4e6b400700 1 -- 172.21.15.63:0/4268219773 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f4e780a6c90 con 0x5625cf52ee40 2024-05-27T08:06:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.087+0000 7f4e6b400700 1 -- 172.21.15.63:0/4268219773 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4e780a9e10 con 0x5625cf52ee40 2024-05-27T08:06:57.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.088+0000 7f4e9afc9b80 10 client.30664 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:57.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.088+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/4268219773 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5625cf59c060 con 0x5625cf52ee40 2024-05-27T08:06:57.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.088+0000 7f4e6b400700 1 -- 172.21.15.63:0/4268219773 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e7c009af0 con 0x5625cf52ee40 2024-05-27T08:06:57.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.088+0000 7f4e6b400700 1 -- 172.21.15.63:0/4268219773 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4e780e3310 con 0x5625cf52ee40 2024-05-27T08:06:57.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.089+0000 7f4e9afc9b80 10 client.30664 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:57.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.089+0000 7f4e9afc9b80 10 client.30664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:57.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.089+0000 7f4e9afc9b80 20 client.30664 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:57.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.089+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625cf6cfda0 0x5625cf6d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.089+0000 7f4e9afc9b80 4 client.30664 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:06:57.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.089+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/4268219773 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625cf5ddd90 con 0x5625cf6cfda0 2024-05-27T08:06:57.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.089+0000 7f4e89000700 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625cf6cfda0 0x5625cf6d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.090+0000 7f4e6b400700 10 client.30664 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:57.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.090+0000 7f4e89000700 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625cf6cfda0 0x5625cf6d2210 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61924fc6131f0d90 server_cookie=ce375951eeb8af1 in_seq=0 out_seq=0 2024-05-27T08:06:57.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.092+0000 7f4e6b400700 1 -- 172.21.15.63:0/4268219773 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5625cf5ddd90 con 0x5625cf6cfda0 2024-05-27T08:06:57.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.092+0000 7f4e6b400700 10 client.30664 handle_command_reply: tid=0 2024-05-27T08:06:57.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.092+0000 7f4e9afc9b80 1 client.30664 shutdown 2024-05-27T08:06:57.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/4268219773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 msgr2=0x5625cf6bfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:57.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bfce0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e780a6720 tx=0x7f4e7801f000 comp rx=0 tx=0).stop 2024-05-27T08:06:57.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/4268219773 shutdown_connections 2024-05-27T08:06:57.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625cf6cfda0 0x5625cf6d2210 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf52ee40 0x5625cf6bfce0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/4268219773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf6b65c0 0x5625cf6c0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e6b400700 0 client.30664 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:57.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/4268219773 >> 172.21.15.63:0/4268219773 conn(0x5625cf58efc0 msgr2=0x5625cf6a8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:57.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/4268219773 shutdown_connections 2024-05-27T08:06:57.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.093+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/4268219773 wait complete. 2024-05-27T08:06:57.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.094+0000 7f4e9afc9b80 20 client.30664 trim_cache size 0 max 16384 2024-05-27T08:06:57.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.127+0000 7f4e9afc9b80 1 Processor -- start 2024-05-27T08:06:57.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.127+0000 7f4e9afc9b80 1 -- start start 2024-05-27T08:06:57.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.127+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf52ee40 0x5625cf6ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.127+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf6b65c0 0x5625cf6cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.127+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.127+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b0af0 con 0x5625cf6bc4e0 2024-05-27T08:06:57.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.127+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841ae380 con 0x5625cf52ee40 2024-05-27T08:06:57.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.127+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841ac210 con 0x5625cf6b65c0 2024-05-27T08:06:57.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.128+0000 7f4e8a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.128+0000 7f4e8a400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34752/0 (socket says 172.21.15.63:34752) 2024-05-27T08:06:57.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.128+0000 7f4e8a400700 1 -- 172.21.15.63:0/3098382441 learned_addr learned my addr 172.21.15.63:0/3098382441 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:57.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.128+0000 7f4e89a00700 1 --2- 172.21.15.63:0/3098382441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf52ee40 0x5625cf6ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.128+0000 7f4e89000700 1 --2- 172.21.15.63:0/3098382441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf6b65c0 0x5625cf6cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.129+0000 7f4e8a400700 1 -- 172.21.15.63:0/3098382441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf6b65c0 msgr2=0x5625cf6cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:57.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.129+0000 7f4e8a400700 1 --2- 172.21.15.63:0/3098382441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf6b65c0 0x5625cf6cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.129+0000 7f4e8a400700 1 -- 172.21.15.63:0/3098382441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf52ee40 msgr2=0x5625cf6ba430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:57.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.129+0000 7f4e8a400700 1 --2- 172.21.15.63:0/3098382441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf52ee40 0x5625cf6ba430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.129+0000 7f4e8a400700 1 -- 172.21.15.63:0/3098382441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625cf6c0ad0 con 0x5625cf6bc4e0 2024-05-27T08:06:57.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.129+0000 7f4e8a400700 1 --2- 172.21.15.63:0/3098382441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c1390 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0093f0 tx=0x7f4e7c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582a4a9c206907c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:57.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.130+0000 7f4e76000700 1 -- 172.21.15.63:0/3098382441 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c031020 con 0x5625cf6bc4e0 2024-05-27T08:06:57.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.130+0000 7f4e76000700 1 -- 172.21.15.63:0/3098382441 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e7c009af0 con 0x5625cf6bc4e0 2024-05-27T08:06:57.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.130+0000 7f4e76000700 1 -- 172.21.15.63:0/3098382441 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c01f040 con 0x5625cf6bc4e0 2024-05-27T08:06:57.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.131+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/3098382441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 msgr2=0x5625cf6c1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:57.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.131+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/3098382441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c1390 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0093f0 tx=0x7f4e7c0077b0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.131+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/3098382441 shutdown_connections 2024-05-27T08:06:57.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.131+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/3098382441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c1390 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.131+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/3098382441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf52ee40 0x5625cf6ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.131+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/3098382441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf6b65c0 0x5625cf6cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.131+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/3098382441 >> 172.21.15.63:0/3098382441 conn(0x5625cf58efc0 msgr2=0x5625cf6aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:57.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.131+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/3098382441 shutdown_connections 2024-05-27T08:06:57.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.132+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/3098382441 wait complete. 2024-05-27T08:06:57.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.132+0000 7f4e9afc9b80 1 -- start start 2024-05-27T08:06:57.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.132+0000 7f4e9afc9b80 1 Processor -- start 2024-05-27T08:06:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.133+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf52ee40 0x5625cf6c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.133+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 0x5625cf6c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.133+0000 7f4e9afc9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.133+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841ac860 con 0x5625cf6b65c0 2024-05-27T08:06:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.133+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841adcb0 con 0x5625cf52ee40 2024-05-27T08:06:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.133+0000 7f4e9afc9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841acf10 con 0x5625cf6bc4e0 2024-05-27T08:06:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.133+0000 7f4e9afc9b80 10 client.0 _ll_register_callbacks cb 0x5625cf575c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:06:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.133+0000 7f4e89000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 0x5625cf6c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.134+0000 7f4e89000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 0x5625cf6c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34768/0 (socket says 172.21.15.63:34768) 2024-05-27T08:06:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.134+0000 7f4e89000700 1 -- 172.21.15.63:0/603783702 learned_addr learned my addr 172.21.15.63:0/603783702 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:06:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.134+0000 7f4e89a00700 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf52ee40 0x5625cf6c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.134+0000 7f4e8a400700 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.134+0000 7f4e89000700 1 -- 172.21.15.63:0/603783702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf6bc4e0 msgr2=0x5625cf6c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.134+0000 7f4e89000700 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.134+0000 7f4e89000700 1 -- 172.21.15.63:0/603783702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf52ee40 msgr2=0x5625cf6c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:06:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.134+0000 7f4e89000700 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf52ee40 0x5625cf6c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.134+0000 7f4e89000700 1 -- 172.21.15.63:0/603783702 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625cf6c5700 con 0x5625cf6b65c0 2024-05-27T08:06:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.135+0000 7f4e89000700 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 0x5625cf6c8730 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f4e84107b50 tx=0x7f4e70003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe89de5720a4e306 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.135+0000 7f4e6b400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.135+0000 7f4e6b400700 1 -- 172.21.15.63:0/603783702 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70004a30 con 0x5625cf6b65c0 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.135+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/603783702 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e7c009af0 con 0x5625cf6b65c0 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.135+0000 7f4e6b400700 1 -- 172.21.15.63:0/603783702 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f4e70004bd0 con 0x5625cf6b65c0 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.135+0000 7f4e6b400700 1 -- 172.21.15.63:0/603783702 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4e70030050 con 0x5625cf6b65c0 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.136+0000 7f4e9afc9b80 10 client.21813 fetch_fsmap learned FSMap version 35 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.136+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/603783702 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5625cf73afa0 con 0x5625cf6b65c0 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.136+0000 7f4e6b400700 1 -- 172.21.15.63:0/603783702 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e7c009af0 con 0x5625cf6b65c0 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.136+0000 7f4e6b400700 1 -- 172.21.15.63:0/603783702 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4e70004ef0 con 0x5625cf6b65c0 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.136+0000 7f4e9afc9b80 10 client.21813 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.136+0000 7f4e9afc9b80 10 client.21813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:06:57.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.136+0000 7f4e9afc9b80 20 client.21813 populate_metadata read hostname 'smithi063' 2024-05-27T08:06:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.136+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625cf73b690 0x5625cf73dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.136+0000 7f4e9afc9b80 4 client.21813 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:06:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.136+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/603783702 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5625cf5d7570 con 0x5625cf73b690 2024-05-27T08:06:57.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.137+0000 7f4e89a00700 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625cf73b690 0x5625cf73dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.138+0000 7f4e89a00700 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625cf73b690 0x5625cf73dab0 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4e9afd9c093893 server_cookie=676ab47fff375f53 in_seq=0 out_seq=0 2024-05-27T08:06:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.138+0000 7f4e6b400700 10 client.21813 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.138+0000 7f4e6b400700 1 -- 172.21.15.63:0/603783702 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5625cf5d7570 con 0x5625cf73b690 2024-05-27T08:06:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.138+0000 7f4e6b400700 10 client.21813 handle_command_reply: tid=0 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 client.21813 shutdown 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/603783702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 msgr2=0x5625cf6c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 0x5625cf6c8730 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f4e84107b50 tx=0x7f4e70003960 comp rx=0 tx=0).stop 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/603783702 shutdown_connections 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5625cf73b690 0x5625cf73dab0 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5625cf6bc4e0 0x5625cf6c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625cf52ee40 0x5625cf6c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 --2- 172.21.15.63:0/603783702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625cf6b65c0 0x5625cf6c8730 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e6b400700 0 client.21813 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/603783702 >> 172.21.15.63:0/603783702 conn(0x5625cf58efc0 msgr2=0x5625cf6bfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/603783702 shutdown_connections 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.139+0000 7f4e9afc9b80 1 -- 172.21.15.63:0/603783702 wait complete. 2024-05-27T08:06:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.140+0000 7f4e9afc9b80 20 client.21813 trim_cache size 0 max 16384 2024-05-27T08:06:57.143 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:06:57.143 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:06:57.143 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:06:57.143 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:06:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 -- 172.21.15.63:0/3726860237 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e6c0b2a70 msgr2=0x7f4e6c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e6c0b2a70 0x7f4e6c0b4f80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009fd0 tx=0x7f4e7c00d000 comp rx=0 tx=0).stop 2024-05-27T08:06:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 -- 172.21.15.63:0/3726860237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e841045e0 msgr2=0x7f4e84075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e841045e0 0x7f4e84075ab0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800b580 tx=0x7f4e780047b0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 -- 172.21.15.63:0/3726860237 shutdown_connections 2024-05-27T08:06:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4e6c0b2a70 0x7f4e6c0b4f80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e84109860 0x7f4e84076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e841045e0 0x7f4e84075ab0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 --2- 172.21.15.63:0/3726860237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e84108f10 0x7f4e84075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:57.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 -- 172.21.15.63:0/3726860237 >> 172.21.15.63:0/3726860237 conn(0x7f4e84100030 msgr2=0x7f4e84102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:57.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 -- 172.21.15.63:0/3726860237 shutdown_connections 2024-05-27T08:06:57.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:06:57.141+0000 7f4e8c400700 1 -- 172.21.15.63:0/3726860237 wait complete. 2024-05-27T08:06:57.818 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:57 smithi063 ceph-mon[31613]: pgmap v1057: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:06:57.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:57 smithi086 ceph-mon[34687]: pgmap v1057: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:06:57.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:57 smithi149 ceph-mon[35408]: pgmap v1057: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:06:58.388 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:06:58.389 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:06:58.956 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:59.651 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:06:59 smithi063 ceph-mon[31613]: pgmap v1058: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T08:06:59.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:06:59 smithi086 ceph-mon[34687]: pgmap v1058: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T08:06:59.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:06:59 smithi149 ceph-mon[35408]: pgmap v1058: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T08:07:01.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.947+0000 7f96bc600700 1 -- 172.21.15.63:0/1932136326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96b4109860 msgr2=0x7f96b41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:01.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.947+0000 7f96bc600700 1 --2- 172.21.15.63:0/1932136326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96b4109860 0x7f96b41111c0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00b0d0 tx=0x7f96ac00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:07:01.950 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:01 smithi063 ceph-mon[31613]: pgmap v1059: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T08:07:01.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.948+0000 7f96bc600700 1 -- 172.21.15.63:0/1932136326 shutdown_connections 2024-05-27T08:07:01.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.948+0000 7f96bc600700 1 --2- 172.21.15.63:0/1932136326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96b4109860 0x7f96b41111c0 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:01.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.948+0000 7f96bc600700 1 --2- 172.21.15.63:0/1932136326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b41045e0 0x7f96b41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:01.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.948+0000 7f96bc600700 1 --2- 172.21.15.63:0/1932136326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96b4108f10 0x7f96b4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:01.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.948+0000 7f96bc600700 1 -- 172.21.15.63:0/1932136326 >> 172.21.15.63:0/1932136326 conn(0x7f96b4100030 msgr2=0x7f96b4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:01.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.948+0000 7f96bc600700 1 -- 172.21.15.63:0/1932136326 shutdown_connections 2024-05-27T08:07:01.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.948+0000 7f96bc600700 1 -- 172.21.15.63:0/1932136326 wait complete. 2024-05-27T08:07:01.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.950+0000 7f96bc600700 1 Processor -- start 2024-05-27T08:07:01.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.950+0000 7f96bc600700 1 -- start start 2024-05-27T08:07:01.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.951+0000 7f96bc600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96b41045e0 0x7f96b4075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:01.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.951+0000 7f96bc600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96b4108f10 0x7f96b4075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:01.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.951+0000 7f96bc600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4109860 0x7f96b4076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:01.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.951+0000 7f96bc600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4114390 con 0x7f96b41045e0 2024-05-27T08:07:01.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.951+0000 7f96bc600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4114210 con 0x7f96b4109860 2024-05-27T08:07:01.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.951+0000 7f96bc600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4114510 con 0x7f96b4108f10 2024-05-27T08:07:01.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.952+0000 7f96b9200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96b4108f10 0x7f96b4075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:01.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.952+0000 7f96b9200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96b4108f10 0x7f96b4075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:40546/0 (socket says 172.21.15.63:40546) 2024-05-27T08:07:01.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.952+0000 7f96b9200700 1 -- 172.21.15.63:0/3384229230 learned_addr learned my addr 172.21.15.63:0/3384229230 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:01.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.952+0000 7f96ba600700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4109860 0x7f96b4076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:01.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.952+0000 7f96b9c00700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96b41045e0 0x7f96b4075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:01.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.953+0000 7f96b9200700 1 -- 172.21.15.63:0/3384229230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4109860 msgr2=0x7f96b4076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:01.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.953+0000 7f96b9200700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4109860 0x7f96b4076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:01.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.953+0000 7f96b9200700 1 -- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96b41045e0 msgr2=0x7f96b4075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:01.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.953+0000 7f96b9200700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96b41045e0 0x7f96b4075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:01.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.953+0000 7f96b9200700 1 -- 172.21.15.63:0/3384229230 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b41a9100 con 0x7f96b4108f10 2024-05-27T08:07:01.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.953+0000 7f96b9c00700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96b41045e0 0x7f96b4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:07:01.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.953+0000 7f96b9200700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96b4108f10 0x7f96b4075ff0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f96a000d980 tx=0x7f96a000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:01.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.954+0000 7f96b2000700 1 -- 172.21.15.63:0/3384229230 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0017070 con 0x7f96b4108f10 2024-05-27T08:07:01.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.954+0000 7f96b2000700 1 -- 172.21.15.63:0/3384229230 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96a0009dc0 con 0x7f96b4108f10 2024-05-27T08:07:01.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.954+0000 7f96bc600700 1 -- 172.21.15.63:0/3384229230 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b41a93f0 con 0x7f96b4108f10 2024-05-27T08:07:01.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.955+0000 7f96bc600700 1 -- 172.21.15.63:0/3384229230 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b41a9930 con 0x7f96b4108f10 2024-05-27T08:07:01.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.956+0000 7f96b2000700 1 -- 172.21.15.63:0/3384229230 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0009930 con 0x7f96b4108f10 2024-05-27T08:07:01.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.956+0000 7f96bc600700 1 -- 172.21.15.63:0/3384229230 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b40512b0 con 0x7f96b4108f10 2024-05-27T08:07:01.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:01 smithi086 ceph-mon[34687]: pgmap v1059: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T08:07:01.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:01 smithi149 ceph-mon[35408]: pgmap v1059: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T08:07:01.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.962+0000 7f96b2000700 1 -- 172.21.15.63:0/3384229230 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f96a0009ad0 con 0x7f96b4108f10 2024-05-27T08:07:01.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.964+0000 7f96b2000700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f969c0b29f0 0x7f969c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:01.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.964+0000 7f96b2000700 1 -- 172.21.15.63:0/3384229230 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7f96a00e2900 con 0x7f96b4108f10 2024-05-27T08:07:01.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.964+0000 7f96b9c00700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f969c0b29f0 0x7f969c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:01.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.965+0000 7f96b9c00700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f969c0b29f0 0x7f969c0b4f00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f96a8005950 tx=0x7f96a80058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:01.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:01.977+0000 7f96b2000700 1 -- 172.21.15.63:0/3384229230 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96a0020070 con 0x7f96b4108f10 2024-05-27T08:07:02.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.297+0000 7f96bc600700 1 -- 172.21.15.63:0/3384229230 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96b4076fe0 con 0x7f96b4108f10 2024-05-27T08:07:02.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.299+0000 7f96b2000700 1 -- 172.21.15.63:0/3384229230 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7f96a00a9f20 con 0x7f96b4108f10 2024-05-27T08:07:02.302 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:07:02.302 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:07:02.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.302+0000 7f96bc600700 1 -- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f969c0b29f0 msgr2=0x7f969c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:02.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.302+0000 7f96bc600700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f969c0b29f0 0x7f969c0b4f00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f96a8005950 tx=0x7f96a80058e0 comp rx=0 tx=0).stop 2024-05-27T08:07:02.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.302+0000 7f96bc600700 1 -- 172.21.15.63:0/3384229230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96b4108f10 msgr2=0x7f96b4075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:02.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.302+0000 7f96bc600700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96b4108f10 0x7f96b4075ff0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f96a000d980 tx=0x7f96a000dc90 comp rx=0 tx=0).stop 2024-05-27T08:07:02.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.303+0000 7f96bc600700 1 -- 172.21.15.63:0/3384229230 shutdown_connections 2024-05-27T08:07:02.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.303+0000 7f96bc600700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f969c0b29f0 0x7f969c0b4f00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:02.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.303+0000 7f96bc600700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4109860 0x7f96b4076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:02.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.303+0000 7f96bc600700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96b41045e0 0x7f96b4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:02.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.303+0000 7f96bc600700 1 --2- 172.21.15.63:0/3384229230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96b4108f10 0x7f96b4075ff0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:02.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.303+0000 7f96bc600700 1 -- 172.21.15.63:0/3384229230 >> 172.21.15.63:0/3384229230 conn(0x7f96b4100030 msgr2=0x7f96b410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:02.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.303+0000 7f96bc600700 1 -- 172.21.15.63:0/3384229230 shutdown_connections 2024-05-27T08:07:02.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:02.303+0000 7f96bc600700 1 -- 172.21.15.63:0/3384229230 wait complete. 2024-05-27T08:07:02.306 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:07:02.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:02 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3384229230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:07:02.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:02 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3384229230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:07:02.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:02 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3384229230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:07:03.242 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:07:03.790 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:07:03.943 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:03 smithi063 ceph-mon[31613]: pgmap v1060: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.9 MiB/s wr, 31 op/s 2024-05-27T08:07:03.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:03 smithi086 ceph-mon[34687]: pgmap v1060: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.9 MiB/s wr, 31 op/s 2024-05-27T08:07:03.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:03 smithi149 ceph-mon[35408]: pgmap v1060: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.9 MiB/s wr, 31 op/s 2024-05-27T08:07:05.865 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:05 smithi063 ceph-mon[31613]: pgmap v1061: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.3 MiB/s wr, 27 op/s 2024-05-27T08:07:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:05 smithi086 ceph-mon[34687]: pgmap v1061: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.3 MiB/s wr, 27 op/s 2024-05-27T08:07:05.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:05 smithi149 ceph-mon[35408]: pgmap v1061: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.3 MiB/s wr, 27 op/s 2024-05-27T08:07:06.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.467+0000 7ff378a00700 1 -- 172.21.15.63:0/1444457009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff374102f20 msgr2=0x7ff374103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:06.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.467+0000 7ff378a00700 1 --2- 172.21.15.63:0/1444457009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff374102f20 0x7ff374103330 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7ff364009990 tx=0x7ff364009ca0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.467+0000 7ff378a00700 1 -- 172.21.15.63:0/1444457009 shutdown_connections 2024-05-27T08:07:06.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.467+0000 7ff378a00700 1 --2- 172.21.15.63:0/1444457009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff374110ad0 0x7ff374112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.467+0000 7ff378a00700 1 --2- 172.21.15.63:0/1444457009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff374102f20 0x7ff374103330 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.467+0000 7ff378a00700 1 --2- 172.21.15.63:0/1444457009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374103870 0x7ff374110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.467+0000 7ff378a00700 1 -- 172.21.15.63:0/1444457009 >> 172.21.15.63:0/1444457009 conn(0x7ff374078ab0 msgr2=0x7ff374078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:06.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.467+0000 7ff378a00700 1 -- 172.21.15.63:0/1444457009 shutdown_connections 2024-05-27T08:07:06.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.467+0000 7ff378a00700 1 -- 172.21.15.63:0/1444457009 wait complete. 2024-05-27T08:07:06.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.469+0000 7ff378a00700 1 Processor -- start 2024-05-27T08:07:06.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.469+0000 7ff378a00700 1 -- start start 2024-05-27T08:07:06.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff378a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374102f20 0x7ff37410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff378a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff374103870 0x7ff37410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff378a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff374110ad0 0x7ff37410e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff378a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741163a0 con 0x7ff374103870 2024-05-27T08:07:06.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff378a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff374116220 con 0x7ff374102f20 2024-05-27T08:07:06.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff378a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff374116520 con 0x7ff374110ad0 2024-05-27T08:07:06.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff372a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff374103870 0x7ff37410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff372a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff374103870 0x7ff37410df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54140/0 (socket says 172.21.15.63:54140) 2024-05-27T08:07:06.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff372a00700 1 -- 172.21.15.63:0/319303072 learned_addr learned my addr 172.21.15.63:0/319303072 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:06.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff373400700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374102f20 0x7ff37410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.470+0000 7ff373e00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff374110ad0 0x7ff37410e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.471+0000 7ff372a00700 1 -- 172.21.15.63:0/319303072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff374110ad0 msgr2=0x7ff37410e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:06.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.471+0000 7ff372a00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff374110ad0 0x7ff37410e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.471+0000 7ff372a00700 1 -- 172.21.15.63:0/319303072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374102f20 msgr2=0x7ff37410da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:06.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.471+0000 7ff372a00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374102f20 0x7ff37410da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.471+0000 7ff372a00700 1 -- 172.21.15.63:0/319303072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3741ad2e0 con 0x7ff374103870 2024-05-27T08:07:06.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.471+0000 7ff372a00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff374103870 0x7ff37410df50 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7ff36000d950 tx=0x7ff36000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:06.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.472+0000 7ff36be00700 1 -- 172.21.15.63:0/319303072 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360017070 con 0x7ff374103870 2024-05-27T08:07:06.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.472+0000 7ff36be00700 1 -- 172.21.15.63:0/319303072 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff36000de30 con 0x7ff374103870 2024-05-27T08:07:06.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.472+0000 7ff378a00700 1 -- 172.21.15.63:0/319303072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3741ad5d0 con 0x7ff374103870 2024-05-27T08:07:06.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.472+0000 7ff36be00700 1 -- 172.21.15.63:0/319303072 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360003aa0 con 0x7ff374103870 2024-05-27T08:07:06.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.472+0000 7ff378a00700 1 -- 172.21.15.63:0/319303072 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3741adb60 con 0x7ff374103870 2024-05-27T08:07:06.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.477+0000 7ff36be00700 1 -- 172.21.15.63:0/319303072 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff360003c40 con 0x7ff374103870 2024-05-27T08:07:06.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.479+0000 7ff36be00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff3580b2a70 0x7ff3580b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.479+0000 7ff36be00700 1 -- 172.21.15.63:0/319303072 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9483+0+0 (secure 0 0 0) 0x7ff3600e24f0 con 0x7ff374103870 2024-05-27T08:07:06.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.479+0000 7ff373400700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff3580b2a70 0x7ff3580b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.480+0000 7ff373400700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff3580b2a70 0x7ff3580b4f80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff36400b5c0 tx=0x7ff364005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:06.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.600+0000 7ff388a75b80 1 Processor -- start 2024-05-27T08:07:06.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.600+0000 7ff388a75b80 1 -- start start 2024-05-27T08:07:06.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.601+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.601+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 0x55a30e9abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.601+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e9a1820 0x55a30e9ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.601+0000 7ff388a75b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a30e8cfe40 con 0x55a30e822e40 2024-05-27T08:07:06.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.601+0000 7ff388a75b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a30e8bf9e0 con 0x55a30e9a1410 2024-05-27T08:07:06.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.601+0000 7ff388a75b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a30e8c3a20 con 0x55a30e9a1820 2024-05-27T08:07:06.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.602+0000 7ff373400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.602+0000 7ff372a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 0x55a30e9abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.602+0000 7ff373400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54156/0 (socket says 172.21.15.63:54156) 2024-05-27T08:07:06.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.602+0000 7ff373400700 1 -- 172.21.15.63:0/862395646 learned_addr learned my addr 172.21.15.63:0/862395646 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:06.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.602+0000 7ff373e00700 1 --2- 172.21.15.63:0/862395646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e9a1820 0x55a30e9ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.603+0000 7ff373400700 1 -- 172.21.15.63:0/862395646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e9a1820 msgr2=0x55a30e9ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:06.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.603+0000 7ff373400700 1 --2- 172.21.15.63:0/862395646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e9a1820 0x55a30e9ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.603+0000 7ff373400700 1 -- 172.21.15.63:0/862395646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 msgr2=0x55a30e9abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:06.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.603+0000 7ff373400700 1 --2- 172.21.15.63:0/862395646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 0x55a30e9abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.603+0000 7ff373400700 1 -- 172.21.15.63:0/862395646 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a30e888420 con 0x55a30e822e40 2024-05-27T08:07:06.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.603+0000 7ff373400700 1 --2- 172.21.15.63:0/862395646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9a0ed0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7ff364015000 tx=0x7ff36401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5171d286caf441 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:06.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.607+0000 7ff368c00700 1 -- 172.21.15.63:0/862395646 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364005630 con 0x55a30e822e40 2024-05-27T08:07:06.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.607+0000 7ff368c00700 1 -- 172.21.15.63:0/862395646 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff36402a070 con 0x55a30e822e40 2024-05-27T08:07:06.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.607+0000 7ff368c00700 1 -- 172.21.15.63:0/862395646 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36402f6d0 con 0x55a30e822e40 2024-05-27T08:07:06.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.608+0000 7ff388a75b80 1 -- 172.21.15.63:0/862395646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 msgr2=0x55a30e9a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:06.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.608+0000 7ff388a75b80 1 --2- 172.21.15.63:0/862395646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9a0ed0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7ff364015000 tx=0x7ff36401d000 comp rx=0 tx=0).stop 2024-05-27T08:07:06.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.608+0000 7ff388a75b80 1 -- 172.21.15.63:0/862395646 shutdown_connections 2024-05-27T08:07:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.608+0000 7ff388a75b80 1 --2- 172.21.15.63:0/862395646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e9a1820 0x55a30e9ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.608+0000 7ff388a75b80 1 --2- 172.21.15.63:0/862395646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9a0ed0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.608+0000 7ff388a75b80 1 --2- 172.21.15.63:0/862395646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 0x55a30e9abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.608+0000 7ff388a75b80 1 -- 172.21.15.63:0/862395646 >> 172.21.15.63:0/862395646 conn(0x55a30e882fc0 msgr2=0x55a30e99cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.609+0000 7ff388a75b80 1 -- 172.21.15.63:0/862395646 shutdown_connections 2024-05-27T08:07:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.609+0000 7ff388a75b80 1 -- 172.21.15.63:0/862395646 wait complete. 2024-05-27T08:07:06.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.610+0000 7ff388a75b80 1 -- start start 2024-05-27T08:07:06.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.610+0000 7ff388a75b80 1 Processor -- start 2024-05-27T08:07:06.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.611+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 0x55a30e9ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.611+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9abad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.611+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1820 0x55a30e9ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.611+0000 7ff388a75b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a30e7c5fa0 con 0x55a30e9a1410 2024-05-27T08:07:06.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.611+0000 7ff388a75b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a30e860760 con 0x55a30e9a1820 2024-05-27T08:07:06.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.611+0000 7ff388a75b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a30e9ae930 con 0x55a30e822e40 2024-05-27T08:07:06.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.611+0000 7ff388a75b80 10 client.0 _ll_register_callbacks cb 0x55a30e869c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:06.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.612+0000 7ff373400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 0x55a30e9ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.612+0000 7ff373400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 0x55a30e9ab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:40596/0 (socket says 172.21.15.63:40596) 2024-05-27T08:07:06.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.612+0000 7ff373400700 1 -- 172.21.15.63:0/332808260 learned_addr learned my addr 172.21.15.63:0/332808260 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:06.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.612+0000 7ff372a00700 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.612+0000 7ff373e00700 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1820 0x55a30e9ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.613+0000 7ff372a00700 1 -- 172.21.15.63:0/332808260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 msgr2=0x55a30e9ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:06.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.613+0000 7ff372a00700 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 0x55a30e9ab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.613+0000 7ff372a00700 1 -- 172.21.15.63:0/332808260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1820 msgr2=0x55a30e9ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:06.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.613+0000 7ff372a00700 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1820 0x55a30e9ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.613+0000 7ff372a00700 1 -- 172.21.15.63:0/332808260 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a30e80f8d0 con 0x55a30e9a1410 2024-05-27T08:07:06.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.613+0000 7ff372a00700 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9abad0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7ff3600df880 tx=0x7ff3600a9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ca2accc2584fb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:06.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.614+0000 7ff35d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.617+0000 7ff35d600700 1 -- 172.21.15.63:0/332808260 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3600b2270 con 0x55a30e9a1410 2024-05-27T08:07:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.617+0000 7ff35d600700 1 -- 172.21.15.63:0/332808260 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7ff3600a56a0 con 0x55a30e9a1410 2024-05-27T08:07:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.617+0000 7ff388a75b80 1 -- 172.21.15.63:0/332808260 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff36402a070 con 0x55a30e9a1410 2024-05-27T08:07:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.618+0000 7ff35d600700 1 -- 172.21.15.63:0/332808260 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff3600e2160 con 0x55a30e9a1410 2024-05-27T08:07:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.618+0000 7ff388a75b80 10 client.21831 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:06.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.618+0000 7ff388a75b80 1 -- 172.21.15.63:0/332808260 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a30e890060 con 0x55a30e9a1410 2024-05-27T08:07:06.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.619+0000 7ff35d600700 1 -- 172.21.15.63:0/332808260 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff36402a070 con 0x55a30e9a1410 2024-05-27T08:07:06.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.619+0000 7ff35d600700 1 -- 172.21.15.63:0/332808260 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff3600b1de0 con 0x55a30e9a1410 2024-05-27T08:07:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.622+0000 7ff388a75b80 10 client.21831 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:06.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.622+0000 7ff388a75b80 10 client.21831 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:07:06.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.622+0000 7ff388a75b80 20 client.21831 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:06.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.622+0000 7ff388a75b80 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55a30e9bb800 0x55a30e9bdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.622+0000 7ff388a75b80 4 client.21831 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:07:06.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.622+0000 7ff388a75b80 1 -- 172.21.15.63:0/332808260 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a30e8d1d90 con 0x55a30e9bb800 2024-05-27T08:07:06.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.623+0000 7ff373400700 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55a30e9bb800 0x55a30e9bdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.627+0000 7ff35d600700 10 client.21831 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:07:06.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.627+0000 7ff373400700 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55a30e9bb800 0x55a30e9bdc20 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be62fb8b2f57e94e server_cookie=a1b5458000f94a10 in_seq=0 out_seq=0 2024-05-27T08:07:06.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.631+0000 7ff35d600700 1 -- 172.21.15.63:0/332808260 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a30e8d1d90 con 0x55a30e9bb800 2024-05-27T08:07:06.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.631+0000 7ff35d600700 10 client.21831 handle_command_reply: tid=0 2024-05-27T08:07:06.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.631+0000 7ff388a75b80 1 client.21831 shutdown 2024-05-27T08:07:06.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.632+0000 7ff388a75b80 1 -- 172.21.15.63:0/332808260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 msgr2=0x55a30e9abad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:06.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.632+0000 7ff388a75b80 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9abad0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7ff3600df880 tx=0x7ff3600a9b60 comp rx=0 tx=0).stop 2024-05-27T08:07:06.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.632+0000 7ff388a75b80 1 -- 172.21.15.63:0/332808260 shutdown_connections 2024-05-27T08:07:06.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.632+0000 7ff388a75b80 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1820 0x55a30e9ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.632+0000 7ff388a75b80 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 0x55a30e9ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.632+0000 7ff388a75b80 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55a30e9bb800 0x55a30e9bdc20 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.633+0000 7ff388a75b80 1 --2- 172.21.15.63:0/332808260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9abad0 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.633+0000 7ff388a75b80 1 -- 172.21.15.63:0/332808260 >> 172.21.15.63:0/332808260 conn(0x55a30e882fc0 msgr2=0x55a30e99bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:06.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.633+0000 7ff35d600700 0 client.21831 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:07:06.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.633+0000 7ff388a75b80 1 -- 172.21.15.63:0/332808260 shutdown_connections 2024-05-27T08:07:06.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.633+0000 7ff388a75b80 1 -- 172.21.15.63:0/332808260 wait complete. 2024-05-27T08:07:06.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.633+0000 7ff388a75b80 20 client.21831 trim_cache size 0 max 16384 2024-05-27T08:07:06.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.671+0000 7ff388a75b80 1 Processor -- start 2024-05-27T08:07:06.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.671+0000 7ff388a75b80 1 -- start start 2024-05-27T08:07:06.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.671+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 0x55a30e99cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.671+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.671+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1820 0x55a30e9a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.671+0000 7ff388a75b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b9570 con 0x55a30e9a1410 2024-05-27T08:07:06.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.671+0000 7ff388a75b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b6e00 con 0x55a30e9a1820 2024-05-27T08:07:06.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.671+0000 7ff388a75b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b4c70 con 0x55a30e822e40 2024-05-27T08:07:06.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.672+0000 7ff372a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.672+0000 7ff372a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54170/0 (socket says 172.21.15.63:54170) 2024-05-27T08:07:06.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.672+0000 7ff372a00700 1 -- 172.21.15.63:0/4231434352 learned_addr learned my addr 172.21.15.63:0/4231434352 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:06.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.672+0000 7ff373400700 1 --2- 172.21.15.63:0/4231434352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 0x55a30e99cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.673+0000 7ff372a00700 1 -- 172.21.15.63:0/4231434352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 msgr2=0x55a30e99cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:06.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.673+0000 7ff372a00700 1 --2- 172.21.15.63:0/4231434352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 0x55a30e99cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.673+0000 7ff372a00700 1 -- 172.21.15.63:0/4231434352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1820 msgr2=0x55a30e9a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:06.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.673+0000 7ff372a00700 1 --2- 172.21.15.63:0/4231434352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1820 0x55a30e9a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.673+0000 7ff372a00700 1 -- 172.21.15.63:0/4231434352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a30e9ac650 con 0x55a30e9a1410 2024-05-27T08:07:06.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.673+0000 7ff372a00700 1 --2- 172.21.15.63:0/4231434352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9aa8b0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7ff3600a93a0 tx=0x7ff360003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdc174e6d8eb614 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:06.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.674+0000 7ff35fe00700 1 -- 172.21.15.63:0/4231434352 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3600ca070 con 0x55a30e9a1410 2024-05-27T08:07:06.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.674+0000 7ff35fe00700 1 -- 172.21.15.63:0/4231434352 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff360022b90 con 0x55a30e9a1410 2024-05-27T08:07:06.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.674+0000 7ff35fe00700 1 -- 172.21.15.63:0/4231434352 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3600a56a0 con 0x55a30e9a1410 2024-05-27T08:07:06.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.675+0000 7ff388a75b80 1 -- 172.21.15.63:0/4231434352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 msgr2=0x55a30e9aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:06.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.675+0000 7ff388a75b80 1 --2- 172.21.15.63:0/4231434352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9aa8b0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7ff3600a93a0 tx=0x7ff360003fb0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.675+0000 7ff388a75b80 1 -- 172.21.15.63:0/4231434352 shutdown_connections 2024-05-27T08:07:06.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.675+0000 7ff388a75b80 1 --2- 172.21.15.63:0/4231434352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1820 0x55a30e9a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.675+0000 7ff388a75b80 1 --2- 172.21.15.63:0/4231434352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e822e40 0x55a30e99cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.675+0000 7ff388a75b80 1 --2- 172.21.15.63:0/4231434352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e9a1410 0x55a30e9aa8b0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.675+0000 7ff388a75b80 1 -- 172.21.15.63:0/4231434352 >> 172.21.15.63:0/4231434352 conn(0x55a30e882fc0 msgr2=0x55a30e9ab370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:06.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.676+0000 7ff388a75b80 1 -- 172.21.15.63:0/4231434352 shutdown_connections 2024-05-27T08:07:06.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.676+0000 7ff388a75b80 1 -- 172.21.15.63:0/4231434352 wait complete. 2024-05-27T08:07:06.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.676+0000 7ff388a75b80 1 -- start start 2024-05-27T08:07:06.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.677+0000 7ff388a75b80 1 Processor -- start 2024-05-27T08:07:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.677+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.677+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 0x55a30e9b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.677+0000 7ff388a75b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e9a1820 0x55a30ea1ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.677+0000 7ff388a75b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b52c0 con 0x55a30e822e40 2024-05-27T08:07:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.677+0000 7ff388a75b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b6730 con 0x55a30e9a1410 2024-05-27T08:07:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.677+0000 7ff388a75b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b5990 con 0x55a30e9a1820 2024-05-27T08:07:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.677+0000 7ff388a75b80 10 client.0 _ll_register_callbacks cb 0x55a30e869c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff373400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff372a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 0x55a30e9b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff372a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 0x55a30e9b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:60526/0 (socket says 172.21.15.63:60526) 2024-05-27T08:07:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff372a00700 1 -- 172.21.15.63:0/1338474321 learned_addr learned my addr 172.21.15.63:0/1338474321 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff373e00700 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e9a1820 0x55a30ea1ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff373400700 1 -- 172.21.15.63:0/1338474321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e9a1820 msgr2=0x55a30ea1ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff373400700 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e9a1820 0x55a30ea1ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff373400700 1 -- 172.21.15.63:0/1338474321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 msgr2=0x55a30e9b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff373400700 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 0x55a30e9b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff373400700 1 -- 172.21.15.63:0/1338474321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a30ea1f490 con 0x55a30e822e40 2024-05-27T08:07:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.678+0000 7ff372a00700 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 0x55a30e9b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:07:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.679+0000 7ff373400700 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9b3e90 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7ff364000c00 tx=0x7ff364003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa1e643bda42d6a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.679+0000 7ff35d600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.679+0000 7ff35d600700 1 -- 172.21.15.63:0/1338474321 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36403b020 con 0x55a30e822e40 2024-05-27T08:07:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.679+0000 7ff35d600700 1 -- 172.21.15.63:0/1338474321 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7ff36402f670 con 0x55a30e822e40 2024-05-27T08:07:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.679+0000 7ff388a75b80 1 -- 172.21.15.63:0/1338474321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff360022b90 con 0x55a30e822e40 2024-05-27T08:07:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.679+0000 7ff35d600700 1 -- 172.21.15.63:0/1338474321 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff364046050 con 0x55a30e822e40 2024-05-27T08:07:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.679+0000 7ff388a75b80 10 client.21837 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.680+0000 7ff388a75b80 1 -- 172.21.15.63:0/1338474321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a30e9b0870 con 0x55a30e822e40 2024-05-27T08:07:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.680+0000 7ff35d600700 1 -- 172.21.15.63:0/1338474321 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff360022b90 con 0x55a30e822e40 2024-05-27T08:07:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.680+0000 7ff35d600700 1 -- 172.21.15.63:0/1338474321 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff36400bc10 con 0x55a30e822e40 2024-05-27T08:07:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.680+0000 7ff388a75b80 10 client.21837 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.681+0000 7ff388a75b80 10 client.21837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:07:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.681+0000 7ff388a75b80 20 client.21837 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.681+0000 7ff388a75b80 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55a30ea37280 0x55a30ea396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.681+0000 7ff388a75b80 4 client.21837 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:07:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.681+0000 7ff388a75b80 1 -- 172.21.15.63:0/1338474321 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a30e8cb570 con 0x55a30ea37280 2024-05-27T08:07:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.681+0000 7ff372a00700 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55a30ea37280 0x55a30ea396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.682+0000 7ff35d600700 10 client.21837 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:07:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.682+0000 7ff372a00700 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55a30ea37280 0x55a30ea396a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=348590720786d020 server_cookie=a729c52bb1faf9bb in_seq=0 out_seq=0 2024-05-27T08:07:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.682+0000 7ff35d600700 1 -- 172.21.15.63:0/1338474321 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a30e8cb570 con 0x55a30ea37280 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.682+0000 7ff35d600700 10 client.21837 handle_command_reply: tid=0 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.682+0000 7ff388a75b80 1 client.21837 shutdown 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.682+0000 7ff388a75b80 1 -- 172.21.15.63:0/1338474321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 msgr2=0x55a30e9b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.682+0000 7ff388a75b80 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9b3e90 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7ff364000c00 tx=0x7ff364003e30 comp rx=0 tx=0).stop 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.682+0000 7ff388a75b80 1 -- 172.21.15.63:0/1338474321 shutdown_connections 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.682+0000 7ff388a75b80 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a30e9a1820 0x55a30ea1ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.683+0000 7ff388a75b80 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55a30ea37280 0x55a30ea396a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.683+0000 7ff388a75b80 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a30e822e40 0x55a30e9b3e90 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.683+0000 7ff35d600700 0 client.21837 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.683+0000 7ff388a75b80 1 --2- 172.21.15.63:0/1338474321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a30e9a1410 0x55a30e9b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.683+0000 7ff388a75b80 1 -- 172.21.15.63:0/1338474321 >> 172.21.15.63:0/1338474321 conn(0x55a30e882fc0 msgr2=0x55a30e9a31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.683+0000 7ff388a75b80 1 -- 172.21.15.63:0/1338474321 shutdown_connections 2024-05-27T08:07:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.683+0000 7ff388a75b80 1 -- 172.21.15.63:0/1338474321 wait complete. 2024-05-27T08:07:06.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.683+0000 7ff388a75b80 20 client.21837 trim_cache size 0 max 16384 2024-05-27T08:07:06.686 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:07:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 -- 172.21.15.63:0/319303072 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff3580b2a70 msgr2=0x7ff3580b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff3580b2a70 0x7ff3580b4f80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff36400b5c0 tx=0x7ff364005fb0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 -- 172.21.15.63:0/319303072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff374103870 msgr2=0x7ff37410df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff374103870 0x7ff37410df50 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7ff36000d950 tx=0x7ff36000dd10 comp rx=0 tx=0).stop 2024-05-27T08:07:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 -- 172.21.15.63:0/319303072 shutdown_connections 2024-05-27T08:07:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff3580b2a70 0x7ff3580b4f80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff374110ad0 0x7ff37410e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374102f20 0x7ff37410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 --2- 172.21.15.63:0/319303072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff374103870 0x7ff37410df50 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 -- 172.21.15.63:0/319303072 >> 172.21.15.63:0/319303072 conn(0x7ff374078ab0 msgr2=0x7ff374111f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.684+0000 7ff378a00700 1 -- 172.21.15.63:0/319303072 shutdown_connections 2024-05-27T08:07:06.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:06.685+0000 7ff378a00700 1 -- 172.21.15.63:0/319303072 wait complete. 2024-05-27T08:07:07.915 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:07 smithi063 ceph-mon[31613]: pgmap v1062: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T08:07:07.915 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:07 smithi063 ceph-mon[31613]: osdmap e147: 12 total, 12 up, 12 in 2024-05-27T08:07:07.919 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:07:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.925+0000 7f9119c00640 1 -- 172.21.15.63:0/4018334725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9114103b90 msgr2=0x7f9114103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.925+0000 7f9119c00640 1 --2- 172.21.15.63:0/4018334725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9114103b90 0x7f9114103fa0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f9100009900 tx=0x7f910002a3f0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.925+0000 7f9119c00640 1 -- 172.21.15.63:0/4018334725 shutdown_connections 2024-05-27T08:07:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.925+0000 7f9119c00640 1 --2- 172.21.15.63:0/4018334725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f911410e510 0x7f9114110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.925+0000 7f9119c00640 1 --2- 172.21.15.63:0/4018334725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9114103b90 0x7f9114103fa0 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.925+0000 7f9119c00640 1 --2- 172.21.15.63:0/4018334725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9114109b40 0x7f9114103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.925+0000 7f9119c00640 1 -- 172.21.15.63:0/4018334725 >> 172.21.15.63:0/4018334725 conn(0x7f91140ff170 msgr2=0x7f91141015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.926+0000 7f9119c00640 1 -- 172.21.15.63:0/4018334725 shutdown_connections 2024-05-27T08:07:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.926+0000 7f9119c00640 1 -- 172.21.15.63:0/4018334725 wait complete. 2024-05-27T08:07:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.927+0000 7f9119c00640 1 Processor -- start 2024-05-27T08:07:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.927+0000 7f9119c00640 1 -- start start 2024-05-27T08:07:07.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9119c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9114103b90 0x7f911419d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9119c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9114109b40 0x7f911419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9119c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f911410e510 0x7f91141a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9119c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9114113b50 con 0x7f9114109b40 2024-05-27T08:07:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9119c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91141139d0 con 0x7f911410e510 2024-05-27T08:07:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9119c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9114113cd0 con 0x7f9114103b90 2024-05-27T08:07:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9113400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9114109b40 0x7f911419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9113400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9114109b40 0x7f911419db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54214/0 (socket says 172.21.15.63:54214) 2024-05-27T08:07:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9113400640 1 -- 172.21.15.63:0/509029672 learned_addr learned my addr 172.21.15.63:0/509029672 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9119200640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f911410e510 0x7f91141a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.928+0000 7f9113e00640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9114103b90 0x7f911419d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.929+0000 7f9119200640 1 -- 172.21.15.63:0/509029672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9114103b90 msgr2=0x7f911419d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.929+0000 7f9119200640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9114103b90 0x7f911419d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.929+0000 7f9119200640 1 -- 172.21.15.63:0/509029672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9114109b40 msgr2=0x7f911419db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.929+0000 7f9119200640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9114109b40 0x7f911419db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.929+0000 7f9119200640 1 -- 172.21.15.63:0/509029672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91141a8a00 con 0x7f911410e510 2024-05-27T08:07:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.930+0000 7f9119200640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f911410e510 0x7f91141a8490 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f910c005fd0 tx=0x7f910c00edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.930+0000 7f9110c00640 1 -- 172.21.15.63:0/509029672 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c004d20 con 0x7f911410e510 2024-05-27T08:07:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.930+0000 7f9119c00640 1 -- 172.21.15.63:0/509029672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91141a8cf0 con 0x7f911410e510 2024-05-27T08:07:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.930+0000 7f9110c00640 1 -- 172.21.15.63:0/509029672 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f910c005020 con 0x7f911410e510 2024-05-27T08:07:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.930+0000 7f9110c00640 1 -- 172.21.15.63:0/509029672 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c0116d0 con 0x7f911410e510 2024-05-27T08:07:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.930+0000 7f9119c00640 1 -- 172.21.15.63:0/509029672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91141a91c0 con 0x7f911410e510 2024-05-27T08:07:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.935+0000 7f91221f7740 1 Processor -- start 2024-05-27T08:07:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.936+0000 7f9110c00640 1 -- 172.21.15.63:0/509029672 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f910c00cac0 con 0x7f911410e510 2024-05-27T08:07:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.936+0000 7f91221f7740 1 -- start start 2024-05-27T08:07:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f91221f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d12ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f91221f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd632f0 0x56416d12efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f91221f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd6b4d0 0x56416d12f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f91221f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56416d09fb90 con 0x56416cd632f0 2024-05-27T08:07:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f91221f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56416d08f730 con 0x56416cd8c1b0 2024-05-27T08:07:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f91221f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56416d093770 con 0x56416cd6b4d0 2024-05-27T08:07:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f9113e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd632f0 0x56416d12efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f9113e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd632f0 0x56416d12efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54224/0 (socket says 172.21.15.63:54224) 2024-05-27T08:07:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f9113e00640 1 -- 172.21.15.63:0/2638123421 learned_addr learned my addr 172.21.15.63:0/2638123421 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f9113400640 1 --2- 172.21.15.63:0/2638123421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d12ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.937+0000 7f9113400640 1 --2- 172.21.15.63:0/2638123421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd6b4d0 0x56416d12f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.938+0000 7f9113e00640 1 -- 172.21.15.63:0/2638123421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd6b4d0 msgr2=0x56416d12f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.938+0000 7f9113e00640 1 --2- 172.21.15.63:0/2638123421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd6b4d0 0x56416d12f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.938+0000 7f9113e00640 1 -- 172.21.15.63:0/2638123421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 msgr2=0x56416d12ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.938+0000 7f9113e00640 1 --2- 172.21.15.63:0/2638123421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d12ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.938+0000 7f9113e00640 1 -- 172.21.15.63:0/2638123421 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416d0288b0 con 0x56416cd632f0 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.938+0000 7f9113e00640 1 --2- 172.21.15.63:0/2638123421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd632f0 0x56416d12efa0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f910400b760 tx=0x7f910400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbb1963cc175848 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.939+0000 7f9110c00640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90f80b2950 0x7f90f80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.939+0000 7f9109600640 1 -- 172.21.15.63:0/2638123421 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104003dd0 con 0x56416cd632f0 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.939+0000 7f9109600640 1 -- 172.21.15.63:0/2638123421 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9104015070 con 0x56416cd632f0 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.939+0000 7f9109600640 1 -- 172.21.15.63:0/2638123421 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104004cc0 con 0x56416cd632f0 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.939+0000 7f9110c00640 1 -- 172.21.15.63:0/509029672 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7f910c0e2aa0 con 0x7f911410e510 2024-05-27T08:07:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.939+0000 7f9113400640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90f80b2950 0x7f90f80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.940+0000 7f91221f7740 1 -- 172.21.15.63:0/2638123421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd632f0 msgr2=0x56416d12efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.940+0000 7f91221f7740 1 --2- 172.21.15.63:0/2638123421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd632f0 0x56416d12efa0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f910400b760 tx=0x7f910400bc20 comp rx=0 tx=0).stop 2024-05-27T08:07:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.940+0000 7f9113400640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90f80b2950 0x7f90f80b4e60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9100000c00 tx=0x7f910002a3f0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.940+0000 7f91221f7740 1 -- 172.21.15.63:0/2638123421 shutdown_connections 2024-05-27T08:07:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.940+0000 7f91221f7740 1 --2- 172.21.15.63:0/2638123421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd6b4d0 0x56416d12f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.940+0000 7f91221f7740 1 --2- 172.21.15.63:0/2638123421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd632f0 0x56416d12efa0 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.940+0000 7f91221f7740 1 --2- 172.21.15.63:0/2638123421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d12ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.940+0000 7f91221f7740 1 -- 172.21.15.63:0/2638123421 >> 172.21.15.63:0/2638123421 conn(0x56416cd80300 msgr2=0x56416d12b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.940+0000 7f91221f7740 1 -- 172.21.15.63:0/2638123421 shutdown_connections 2024-05-27T08:07:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.940+0000 7f91221f7740 1 -- 172.21.15.63:0/2638123421 wait complete. 2024-05-27T08:07:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.941+0000 7f91221f7740 1 -- start start 2024-05-27T08:07:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.941+0000 7f91221f7740 1 Processor -- start 2024-05-27T08:07:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.942+0000 7f91221f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd632f0 0x56416d138c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.942+0000 7f91221f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d139190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.942+0000 7f91221f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd8c1b0 0x56416d144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.942+0000 7f91221f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56416cd75790 con 0x56416cd6b4d0 2024-05-27T08:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.942+0000 7f91221f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56416d037bc0 con 0x56416cd632f0 2024-05-27T08:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.942+0000 7f91221f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56416d13bb90 con 0x56416cd8c1b0 2024-05-27T08:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.942+0000 7f91221f7740 10 client.0 _ll_register_callbacks cb 0x56416d015400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.942+0000 7f9113400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d139190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9113400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d139190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54228/0 (socket says 172.21.15.63:54228) 2024-05-27T08:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9113400640 1 -- 172.21.15.63:0/2584087996 learned_addr learned my addr 172.21.15.63:0/2584087996 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9113e00640 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd632f0 0x56416d138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9119200640 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd8c1b0 0x56416d144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9113400640 1 -- 172.21.15.63:0/2584087996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd8c1b0 msgr2=0x56416d144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9113400640 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd8c1b0 0x56416d144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9113400640 1 -- 172.21.15.63:0/2584087996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd632f0 msgr2=0x56416d138c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9113400640 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd632f0 0x56416d138c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9113400640 1 -- 172.21.15.63:0/2584087996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416cda4210 con 0x56416cd6b4d0 2024-05-27T08:07:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9113e00640 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd632f0 0x56416d138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:07:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.943+0000 7f9113400640 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d139190 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f910002a910 tx=0x7f91000058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33670858950e6b1b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.944+0000 7f90fe000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.944+0000 7f90fe000640 1 -- 172.21.15.63:0/2584087996 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100035040 con 0x56416cd6b4d0 2024-05-27T08:07:07.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.944+0000 7f90fe000640 1 -- 172.21.15.63:0/2584087996 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7f9100003a00 con 0x56416cd6b4d0 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.944+0000 7f91221f7740 1 -- 172.21.15.63:0/2584087996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9104015070 con 0x56416cd6b4d0 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.944+0000 7f90fe000640 1 -- 172.21.15.63:0/2584087996 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9100003c00 con 0x56416cd6b4d0 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.944+0000 7f91221f7740 10 client.21849 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.944+0000 7f91221f7740 1 -- 172.21.15.63:0/2584087996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56416ce70620 con 0x56416cd6b4d0 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.945+0000 7f90fe000640 1 -- 172.21.15.63:0/2584087996 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9104015070 con 0x56416cd6b4d0 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.945+0000 7f90fe000640 1 -- 172.21.15.63:0/2584087996 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9100047030 con 0x56416cd6b4d0 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.945+0000 7f91221f7740 10 client.21849 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.945+0000 7f91221f7740 10 client.21849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.945+0000 7f91221f7740 20 client.21849 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.945+0000 7f91221f7740 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56416cd4ad80 0x56416d14ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.945+0000 7f91221f7740 4 client.21849 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.945+0000 7f91221f7740 1 -- 172.21.15.63:0/2584087996 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56416d0a1ae0 con 0x56416cd4ad80 2024-05-27T08:07:07.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.946+0000 7f9113e00640 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56416cd4ad80 0x56416d14ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.947+0000 7f9113e00640 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56416cd4ad80 0x56416d14ae70 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290ce842f89619c6 server_cookie=a2289e30a8b33f0b in_seq=0 out_seq=0 2024-05-27T08:07:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.947+0000 7f90fe000640 10 client.21849 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.949+0000 7f90fe000640 1 -- 172.21.15.63:0/2584087996 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56416d0a1ae0 con 0x56416cd4ad80 2024-05-27T08:07:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.949+0000 7f90fe000640 10 client.21849 handle_command_reply: tid=0 2024-05-27T08:07:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.949+0000 7f91221f7740 1 client.21849 shutdown 2024-05-27T08:07:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 1 -- 172.21.15.63:0/2584087996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 msgr2=0x56416d139190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d139190 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f910002a910 tx=0x7f91000058d0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 1 -- 172.21.15.63:0/2584087996 shutdown_connections 2024-05-27T08:07:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56416cd4ad80 0x56416d14ae70 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd8c1b0 0x56416d144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d139190 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 1 --2- 172.21.15.63:0/2584087996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd632f0 0x56416d138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 1 -- 172.21.15.63:0/2584087996 >> 172.21.15.63:0/2584087996 conn(0x56416cd80300 msgr2=0x56416d130780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f90fe000640 0 client.21849 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 1 -- 172.21.15.63:0/2584087996 shutdown_connections 2024-05-27T08:07:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 1 -- 172.21.15.63:0/2584087996 wait complete. 2024-05-27T08:07:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.950+0000 7f91221f7740 20 client.21849 trim_cache size 0 max 16384 2024-05-27T08:07:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:07 smithi086 ceph-mon[34687]: pgmap v1062: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T08:07:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:07 smithi086 ceph-mon[34687]: osdmap e147: 12 total, 12 up, 12 in 2024-05-27T08:07:07.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:07 smithi149 ceph-mon[35408]: pgmap v1062: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.7 MiB/s wr, 31 op/s 2024-05-27T08:07:07.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:07 smithi149 ceph-mon[35408]: osdmap e147: 12 total, 12 up, 12 in 2024-05-27T08:07:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.971+0000 7f91221f7740 1 Processor -- start 2024-05-27T08:07:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.972+0000 7f91221f7740 1 -- start start 2024-05-27T08:07:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.972+0000 7f91221f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 0x56416d13ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.972+0000 7f91221f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d138bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.972+0000 7f91221f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d139130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.972+0000 7f91221f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b5c80 con 0x56416cd6b4d0 2024-05-27T08:07:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.972+0000 7f91221f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b3550 con 0x56416cd8c1b0 2024-05-27T08:07:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.972+0000 7f91221f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b13c0 con 0x56416cd632f0 2024-05-27T08:07:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9113400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d138bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9113400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d138bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54236/0 (socket says 172.21.15.63:54236) 2024-05-27T08:07:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9113400640 1 -- 172.21.15.63:0/1703672043 learned_addr learned my addr 172.21.15.63:0/1703672043 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9113e00640 1 --2- 172.21.15.63:0/1703672043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 0x56416d13ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9119200640 1 --2- 172.21.15.63:0/1703672043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d139130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9113400640 1 -- 172.21.15.63:0/1703672043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 msgr2=0x56416d13ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9113400640 1 --2- 172.21.15.63:0/1703672043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 0x56416d13ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9113400640 1 -- 172.21.15.63:0/1703672043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 msgr2=0x56416d139130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9113400640 1 --2- 172.21.15.63:0/1703672043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d139130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9113400640 1 -- 172.21.15.63:0/1703672043 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416d12e7f0 con 0x56416cd6b4d0 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.973+0000 7f9119200640 1 --2- 172.21.15.63:0/1703672043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d139130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.974+0000 7f9113400640 1 --2- 172.21.15.63:0/1703672043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d138bf0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f910005d560 tx=0x7f91000319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a224150d57736 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.974+0000 7f9108c00640 1 -- 172.21.15.63:0/1703672043 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100061030 con 0x56416cd6b4d0 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.974+0000 7f9108c00640 1 -- 172.21.15.63:0/1703672043 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f910004f070 con 0x56416cd6b4d0 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.974+0000 7f9108c00640 1 -- 172.21.15.63:0/1703672043 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100035040 con 0x56416cd6b4d0 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.975+0000 7f91221f7740 1 -- 172.21.15.63:0/1703672043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 msgr2=0x56416d138bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.975+0000 7f91221f7740 1 --2- 172.21.15.63:0/1703672043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d138bf0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f910005d560 tx=0x7f91000319b0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.975+0000 7f91221f7740 1 -- 172.21.15.63:0/1703672043 shutdown_connections 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.975+0000 7f91221f7740 1 --2- 172.21.15.63:0/1703672043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d139130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.975+0000 7f91221f7740 1 --2- 172.21.15.63:0/1703672043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d138bf0 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.975+0000 7f91221f7740 1 --2- 172.21.15.63:0/1703672043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 0x56416d13ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.982 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.975+0000 7f91221f7740 1 -- 172.21.15.63:0/1703672043 >> 172.21.15.63:0/1703672043 conn(0x56416cd80300 msgr2=0x56416d1384e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.975+0000 7f91221f7740 1 -- 172.21.15.63:0/1703672043 shutdown_connections 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.975+0000 7f91221f7740 1 -- 172.21.15.63:0/1703672043 wait complete. 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.976+0000 7f91221f7740 1 -- start start 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.976+0000 7f91221f7740 1 Processor -- start 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f91221f7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 0x56416d132990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f91221f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d132ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f91221f7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d133410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f91221f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b1a10 con 0x56416cd6b4d0 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f91221f7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b2e80 con 0x56416cd8c1b0 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f91221f7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b20e0 con 0x56416cd632f0 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f91221f7740 10 client.0 _ll_register_callbacks cb 0x7f9100032c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f9113400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d132ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f9113400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d132ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54240/0 (socket says 172.21.15.63:54240) 2024-05-27T08:07:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f9113400640 1 -- 172.21.15.63:0/879678630 learned_addr learned my addr 172.21.15.63:0/879678630 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.977+0000 7f9113e00640 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 0x56416d132990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.978+0000 7f9119200640 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d133410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.978+0000 7f9113400640 1 -- 172.21.15.63:0/879678630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 msgr2=0x56416d132990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.978+0000 7f9113400640 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 0x56416d132990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.978+0000 7f9113400640 1 -- 172.21.15.63:0/879678630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 msgr2=0x56416d133410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.978+0000 7f9113400640 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d133410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.978+0000 7f9113400640 1 -- 172.21.15.63:0/879678630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416d13fe20 con 0x56416cd6b4d0 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.978+0000 7f9113e00640 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 0x56416d132990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.978+0000 7f9113400640 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d132ed0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f910005d490 tx=0x7f910005d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc462e3056129f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.978+0000 7f90fe000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.979+0000 7f90fe000640 1 -- 172.21.15.63:0/879678630 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910006a060 con 0x56416cd6b4d0 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.979+0000 7f90fe000640 1 -- 172.21.15.63:0/879678630 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7f910005da30 con 0x56416cd6b4d0 2024-05-27T08:07:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.979+0000 7f91221f7740 1 -- 172.21.15.63:0/879678630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910004f070 con 0x56416cd6b4d0 2024-05-27T08:07:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.979+0000 7f90fe000640 1 -- 172.21.15.63:0/879678630 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9100071050 con 0x56416cd6b4d0 2024-05-27T08:07:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.979+0000 7f91221f7740 10 client.21855 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.979+0000 7f91221f7740 1 -- 172.21.15.63:0/879678630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56416d141080 con 0x56416cd6b4d0 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.979+0000 7f90fe000640 1 -- 172.21.15.63:0/879678630 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f910004f070 con 0x56416cd6b4d0 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.980+0000 7f90fe000640 1 -- 172.21.15.63:0/879678630 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9100047070 con 0x56416cd6b4d0 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.980+0000 7f91221f7740 10 client.21855 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.980+0000 7f91221f7740 10 client.21855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.980+0000 7f91221f7740 20 client.21855 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.980+0000 7f91221f7740 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56416cd201c0 0x56416d1b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.980+0000 7f91221f7740 4 client.21855 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.980+0000 7f91221f7740 1 -- 172.21.15.63:0/879678630 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56416d09b2c0 con 0x56416cd201c0 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.981+0000 7f9113e00640 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56416cd201c0 0x56416d1b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.982+0000 7f9113e00640 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56416cd201c0 0x56416d1b5500 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa7462ac74269e67 server_cookie=3228ee3b71be0673 in_seq=0 out_seq=0 2024-05-27T08:07:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:07.982+0000 7f90fe000640 10 client.21855 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.023+0000 7f90fe000640 1 -- 172.21.15.63:0/879678630 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56416d09b2c0 con 0x56416cd201c0 2024-05-27T08:07:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.023+0000 7f90fe000640 10 client.21855 handle_command_reply: tid=0 2024-05-27T08:07:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.023+0000 7f91221f7740 1 client.21855 shutdown 2024-05-27T08:07:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 1 -- 172.21.15.63:0/879678630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 msgr2=0x56416d132ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d132ed0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f910005d490 tx=0x7f910005d840 comp rx=0 tx=0).stop 2024-05-27T08:07:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 1 -- 172.21.15.63:0/879678630 shutdown_connections 2024-05-27T08:07:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56416cd201c0 0x56416d1b5500 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56416cd8c1b0 0x56416d133410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56416cd6b4d0 0x56416d132ed0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 1 --2- 172.21.15.63:0/879678630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56416cd632f0 0x56416d132990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f90fe000640 0 client.21855 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 1 -- 172.21.15.63:0/879678630 >> 172.21.15.63:0/879678630 conn(0x56416cd80300 msgr2=0x56416d138ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 1 -- 172.21.15.63:0/879678630 shutdown_connections 2024-05-27T08:07:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 1 -- 172.21.15.63:0/879678630 wait complete. 2024-05-27T08:07:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.024+0000 7f91221f7740 20 client.21855 trim_cache size 0 max 16384 2024-05-27T08:07:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:07:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:07:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:07:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.025+0000 7f9119c00640 1 -- 172.21.15.63:0/509029672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90f80b2950 msgr2=0x7f90f80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.025+0000 7f9119c00640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90f80b2950 0x7f90f80b4e60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9100000c00 tx=0x7f910002a3f0 comp rx=0 tx=0).stop 2024-05-27T08:07:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.025+0000 7f9119c00640 1 -- 172.21.15.63:0/509029672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f911410e510 msgr2=0x7f91141a8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.025+0000 7f9119c00640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f911410e510 0x7f91141a8490 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f910c005fd0 tx=0x7f910c00edd0 comp rx=0 tx=0).stop 2024-05-27T08:07:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.026+0000 7f9119c00640 1 -- 172.21.15.63:0/509029672 shutdown_connections 2024-05-27T08:07:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.026+0000 7f9119c00640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f90f80b2950 0x7f90f80b4e60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.026+0000 7f9119c00640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f911410e510 0x7f91141a8490 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.026+0000 7f9119c00640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9114109b40 0x7f911419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.026+0000 7f9119c00640 1 --2- 172.21.15.63:0/509029672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9114103b90 0x7f911419d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.026+0000 7f9119c00640 1 -- 172.21.15.63:0/509029672 >> 172.21.15.63:0/509029672 conn(0x7f91140ff170 msgr2=0x7f91140768f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.026+0000 7f9119c00640 1 -- 172.21.15.63:0/509029672 shutdown_connections 2024-05-27T08:07:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:08.026+0000 7f9119c00640 1 -- 172.21.15.63:0/509029672 wait complete. 2024-05-27T08:07:08.418 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:07:09.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:09 smithi086 ceph-mon[34687]: pgmap v1064: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.0 MiB/s wr, 29 op/s 2024-05-27T08:07:09.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:09 smithi149 ceph-mon[35408]: pgmap v1064: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.0 MiB/s wr, 29 op/s 2024-05-27T08:07:09.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:09 smithi063 ceph-mon[31613]: pgmap v1064: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.0 MiB/s wr, 29 op/s 2024-05-27T08:07:11.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.323+0000 7f4322a00700 1 -- 172.21.15.63:0/498993777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109850 msgr2=0x7f431c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:11.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.323+0000 7f4322a00700 1 --2- 172.21.15.63:0/498993777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109850 0x7f431c1111b0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f431400b0d0 tx=0x7f431400b3e0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.323+0000 7f4322a00700 1 -- 172.21.15.63:0/498993777 shutdown_connections 2024-05-27T08:07:11.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.323+0000 7f4322a00700 1 --2- 172.21.15.63:0/498993777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109850 0x7f431c1111b0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.323+0000 7f4322a00700 1 --2- 172.21.15.63:0/498993777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c1045d0 0x7f431c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.323+0000 7f4322a00700 1 --2- 172.21.15.63:0/498993777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c108f00 0x7f431c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.323+0000 7f4322a00700 1 -- 172.21.15.63:0/498993777 >> 172.21.15.63:0/498993777 conn(0x7f431c0ffe60 msgr2=0x7f431c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:11.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.323+0000 7f4322a00700 1 -- 172.21.15.63:0/498993777 shutdown_connections 2024-05-27T08:07:11.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.323+0000 7f4322a00700 1 -- 172.21.15.63:0/498993777 wait complete. 2024-05-27T08:07:11.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.325+0000 7f4322a00700 1 Processor -- start 2024-05-27T08:07:11.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.325+0000 7f4322a00700 1 -- start start 2024-05-27T08:07:11.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.326+0000 7f4322a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c1045d0 0x7f431c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.326+0000 7f4322a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c108f00 0x7f431c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.326+0000 7f4322a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109850 0x7f431c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.326+0000 7f4322a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f431c114330 con 0x7f431c109850 2024-05-27T08:07:11.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.326+0000 7f4322a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1141b0 con 0x7f431c1045d0 2024-05-27T08:07:11.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.326+0000 7f4322a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1144b0 con 0x7f431c108f00 2024-05-27T08:07:11.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.327+0000 7f431b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c108f00 0x7f431c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.327+0000 7f4320a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109850 0x7f431c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.327+0000 7f4320a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109850 0x7f431c1a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49534/0 (socket says 172.21.15.63:49534) 2024-05-27T08:07:11.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.327+0000 7f4320a00700 1 -- 172.21.15.63:0/2286913436 learned_addr learned my addr 172.21.15.63:0/2286913436 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:11.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.327+0000 7f431be00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c1045d0 0x7f431c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.328+0000 7f4320a00700 1 -- 172.21.15.63:0/2286913436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c108f00 msgr2=0x7f431c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:11.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.328+0000 7f4320a00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c108f00 0x7f431c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.328+0000 7f4320a00700 1 -- 172.21.15.63:0/2286913436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c1045d0 msgr2=0x7f431c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:11.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.328+0000 7f4320a00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c1045d0 0x7f431c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.328+0000 7f4320a00700 1 -- 172.21.15.63:0/2286913436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c1a1d70 con 0x7f431c109850 2024-05-27T08:07:11.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.328+0000 7f431be00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c1045d0 0x7f431c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:07:11.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.329+0000 7f4320a00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109850 0x7f431c1a16d0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f4314007a00 tx=0x7f4314007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:11.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.329+0000 7f4318c00700 1 -- 172.21.15.63:0/2286913436 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431401f070 con 0x7f431c109850 2024-05-27T08:07:11.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.329+0000 7f4318c00700 1 -- 172.21.15.63:0/2286913436 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4314004c50 con 0x7f431c109850 2024-05-27T08:07:11.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.329+0000 7f4318c00700 1 -- 172.21.15.63:0/2286913436 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431401b690 con 0x7f431c109850 2024-05-27T08:07:11.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.329+0000 7f4322a00700 1 -- 172.21.15.63:0/2286913436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c1a2000 con 0x7f431c109850 2024-05-27T08:07:11.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.329+0000 7f4322a00700 1 -- 172.21.15.63:0/2286913436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1a9980 con 0x7f431c109850 2024-05-27T08:07:11.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.334+0000 7f4318c00700 1 -- 172.21.15.63:0/2286913436 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4314022070 con 0x7f431c109850 2024-05-27T08:07:11.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.336+0000 7f43314fab80 1 Processor -- start 2024-05-27T08:07:11.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.337+0000 7f43314fab80 1 -- start start 2024-05-27T08:07:11.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.337+0000 7f4318c00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b2a70 0x7f43000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.337+0000 7f43314fab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.337+0000 7f43314fab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fcf5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.337+0000 7f43314fab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fd02110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.338+0000 7f43314fab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d33fc23e40 con 0x55d33fc629f0 2024-05-27T08:07:11.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.338+0000 7f43314fab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d33fc139e0 con 0x55d33fb76e40 2024-05-27T08:07:11.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.338+0000 7f43314fab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d33fc17a20 con 0x55d33fcf5e70 2024-05-27T08:07:11.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.338+0000 7f4318c00700 1 -- 172.21.15.63:0/2286913436 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7f431400f020 con 0x7f431c109850 2024-05-27T08:07:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.338+0000 7f431b400700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b2a70 0x7f43000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.338+0000 7f431b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fcf5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.338+0000 7f431be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.338+0000 7f4320a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fd02110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.338+0000 7f431b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fcf5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46674/0 (socket says 172.21.15.63:46674) 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.338+0000 7f431b400700 1 -- 172.21.15.63:0/8789730 learned_addr learned my addr 172.21.15.63:0/8789730 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.339+0000 7f431b400700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b2a70 0x7f43000b4f80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4308013570 tx=0x7f43080098c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.339+0000 7f431b400700 1 -- 172.21.15.63:0/8789730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 msgr2=0x55d33fd02110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.339+0000 7f431b400700 1 --2- 172.21.15.63:0/8789730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fd02110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.339+0000 7f431b400700 1 -- 172.21.15.63:0/8789730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 msgr2=0x55d33fcf5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.339+0000 7f431b400700 1 --2- 172.21.15.63:0/8789730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.339+0000 7f431b400700 1 -- 172.21.15.63:0/8789730 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d33fbdc420 con 0x55d33fb76e40 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.339+0000 7f431be00700 1 --2- 172.21.15.63:0/8789730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.340+0000 7f431b400700 1 --2- 172.21.15.63:0/8789730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fcf5870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4308009d80 tx=0x7f4308020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2f00fbd11800f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.340+0000 7f4311600700 1 -- 172.21.15.63:0/8789730 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43080130d0 con 0x55d33fb76e40 2024-05-27T08:07:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.341+0000 7f4311600700 1 -- 172.21.15.63:0/8789730 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4308024070 con 0x55d33fb76e40 2024-05-27T08:07:11.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.341+0000 7f4311600700 1 -- 172.21.15.63:0/8789730 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308003c10 con 0x55d33fb76e40 2024-05-27T08:07:11.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.342+0000 7f43314fab80 1 -- 172.21.15.63:0/8789730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 msgr2=0x55d33fcf5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:11.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.342+0000 7f43314fab80 1 --2- 172.21.15.63:0/8789730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fcf5870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4308009d80 tx=0x7f4308020000 comp rx=0 tx=0).stop 2024-05-27T08:07:11.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.342+0000 7f43314fab80 1 -- 172.21.15.63:0/8789730 shutdown_connections 2024-05-27T08:07:11.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.342+0000 7f43314fab80 1 --2- 172.21.15.63:0/8789730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fd02110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.342+0000 7f43314fab80 1 --2- 172.21.15.63:0/8789730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.342+0000 7f43314fab80 1 --2- 172.21.15.63:0/8789730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fcf5870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.342+0000 7f43314fab80 1 -- 172.21.15.63:0/8789730 >> 172.21.15.63:0/8789730 conn(0x55d33fc10390 msgr2=0x55d33fcf14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:11.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.342+0000 7f43314fab80 1 -- 172.21.15.63:0/8789730 shutdown_connections 2024-05-27T08:07:11.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.342+0000 7f43314fab80 1 -- 172.21.15.63:0/8789730 wait complete. 2024-05-27T08:07:11.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.343+0000 7f43314fab80 1 -- start start 2024-05-27T08:07:11.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.343+0000 7f43314fab80 1 Processor -- start 2024-05-27T08:07:11.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.344+0000 7f43314fab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fb76e40 0x55d33fcff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.344+0000 7f43314fab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fc629f0 0x55d33fcffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.344+0000 7f43314fab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fcf5e70 0x55d33fd00120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.344+0000 7f43314fab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d33fb19fa0 con 0x55d33fb76e40 2024-05-27T08:07:11.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.344+0000 7f43314fab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d33fbb4760 con 0x55d33fcf5e70 2024-05-27T08:07:11.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.344+0000 7f43314fab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d33fd02b20 con 0x55d33fc629f0 2024-05-27T08:07:11.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.344+0000 7f43314fab80 10 client.0 _ll_register_callbacks cb 0x55d33fbbdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:11.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.345+0000 7f431b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fc629f0 0x55d33fcffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.345+0000 7f431be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fb76e40 0x55d33fcff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.345+0000 7f431be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fb76e40 0x55d33fcff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49554/0 (socket says 172.21.15.63:49554) 2024-05-27T08:07:11.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.345+0000 7f431be00700 1 -- 172.21.15.63:0/2652196899 learned_addr learned my addr 172.21.15.63:0/2652196899 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:11.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.345+0000 7f4320a00700 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fcf5e70 0x55d33fd00120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.345+0000 7f431be00700 1 -- 172.21.15.63:0/2652196899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fc629f0 msgr2=0x55d33fcffbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:11.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.345+0000 7f431be00700 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fc629f0 0x55d33fcffbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.345+0000 7f431be00700 1 -- 172.21.15.63:0/2652196899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fcf5e70 msgr2=0x55d33fd00120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:11.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.345+0000 7f431be00700 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fcf5e70 0x55d33fd00120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.345+0000 7f431be00700 1 -- 172.21.15.63:0/2652196899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d33fb638d0 con 0x55d33fb76e40 2024-05-27T08:07:11.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.346+0000 7f431be00700 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fb76e40 0x55d33fcff6a0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f430c012d20 tx=0x7f430c00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8656b28c326f95cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:11.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.346+0000 7f4306000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:11.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.346+0000 7f4306000700 1 -- 172.21.15.63:0/2652196899 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00fb80 con 0x55d33fb76e40 2024-05-27T08:07:11.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.346+0000 7f4306000700 1 -- 172.21.15.63:0/2652196899 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7f430c00fd20 con 0x55d33fb76e40 2024-05-27T08:07:11.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.346+0000 7f43314fab80 1 -- 172.21.15.63:0/2652196899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4308024070 con 0x55d33fb76e40 2024-05-27T08:07:11.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.347+0000 7f4306000700 1 -- 172.21.15.63:0/2652196899 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f430c025050 con 0x55d33fb76e40 2024-05-27T08:07:11.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.347+0000 7f43314fab80 10 client.21867 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:11.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.347+0000 7f43314fab80 1 -- 172.21.15.63:0/2652196899 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d33fbe4060 con 0x55d33fb76e40 2024-05-27T08:07:11.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.348+0000 7f4306000700 1 -- 172.21.15.63:0/2652196899 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4308024070 con 0x55d33fb76e40 2024-05-27T08:07:11.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.348+0000 7f4306000700 1 -- 172.21.15.63:0/2652196899 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f430c010430 con 0x55d33fb76e40 2024-05-27T08:07:11.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.348+0000 7f43314fab80 10 client.21867 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:11.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.348+0000 7f43314fab80 10 client.21867 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:07:11.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.348+0000 7f43314fab80 20 client.21867 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:11.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.348+0000 7f43314fab80 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d33fd0fb50 0x55d33fd11f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.348+0000 7f43314fab80 4 client.21867 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:07:11.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.348+0000 7f43314fab80 1 -- 172.21.15.63:0/2652196899 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d33fc25d90 con 0x55d33fd0fb50 2024-05-27T08:07:11.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.349+0000 7f431b400700 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d33fd0fb50 0x55d33fd11f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.350+0000 7f431b400700 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d33fd0fb50 0x55d33fd11f90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62e183b37b969a0f server_cookie=ae3f61dccd7908 in_seq=0 out_seq=0 2024-05-27T08:07:11.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.350+0000 7f4306000700 10 client.21867 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:07:11.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.352+0000 7f4306000700 1 -- 172.21.15.63:0/2652196899 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d33fc25d90 con 0x55d33fd0fb50 2024-05-27T08:07:11.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.352+0000 7f4306000700 10 client.21867 handle_command_reply: tid=0 2024-05-27T08:07:11.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.352+0000 7f43314fab80 1 client.21867 shutdown 2024-05-27T08:07:11.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f43314fab80 1 -- 172.21.15.63:0/2652196899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fb76e40 msgr2=0x55d33fcff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:11.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f43314fab80 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fb76e40 0x55d33fcff6a0 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f430c012d20 tx=0x7f430c00f6a0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f43314fab80 1 -- 172.21.15.63:0/2652196899 shutdown_connections 2024-05-27T08:07:11.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f43314fab80 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fcf5e70 0x55d33fd00120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f43314fab80 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d33fd0fb50 0x55d33fd11f90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f43314fab80 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fb76e40 0x55d33fcff6a0 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f43314fab80 1 --2- 172.21.15.63:0/2652196899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fc629f0 0x55d33fcffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f43314fab80 1 -- 172.21.15.63:0/2652196899 >> 172.21.15.63:0/2652196899 conn(0x55d33fc10390 msgr2=0x55d33fcf53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f4306000700 0 client.21867 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:07:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f43314fab80 1 -- 172.21.15.63:0/2652196899 shutdown_connections 2024-05-27T08:07:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.353+0000 7f43314fab80 1 -- 172.21.15.63:0/2652196899 wait complete. 2024-05-27T08:07:11.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.354+0000 7f43314fab80 20 client.21867 trim_cache size 0 max 16384 2024-05-27T08:07:11.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.374+0000 7f43314fab80 1 Processor -- start 2024-05-27T08:07:11.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.374+0000 7f43314fab80 1 -- start start 2024-05-27T08:07:11.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.375+0000 7f43314fab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fcfae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.375+0000 7f43314fab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.375+0000 7f43314fab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fcfe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.375+0000 7f43314fab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b4fe0 con 0x55d33fc629f0 2024-05-27T08:07:11.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.375+0000 7f43314fab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b2870 con 0x55d33fb76e40 2024-05-27T08:07:11.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.375+0000 7f43314fab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b06e0 con 0x55d33fcf5e70 2024-05-27T08:07:11.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.376+0000 7f431b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.376+0000 7f431b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49558/0 (socket says 172.21.15.63:49558) 2024-05-27T08:07:11.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.376+0000 7f431be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fcfae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.376+0000 7f431b400700 1 -- 172.21.15.63:0/1750389209 learned_addr learned my addr 172.21.15.63:0/1750389209 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:11.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.376+0000 7f4320a00700 1 --2- 172.21.15.63:0/1750389209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fcfe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.376+0000 7f431b400700 1 -- 172.21.15.63:0/1750389209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 msgr2=0x55d33fcfe6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.376+0000 7f431b400700 1 --2- 172.21.15.63:0/1750389209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fcfe6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.376+0000 7f431b400700 1 -- 172.21.15.63:0/1750389209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 msgr2=0x55d33fcfae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.376+0000 7f431b400700 1 --2- 172.21.15.63:0/1750389209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fcfae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.376+0000 7f431b400700 1 -- 172.21.15.63:0/1750389209 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d33fd00490 con 0x55d33fc629f0 2024-05-27T08:07:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.377+0000 7f431b400700 1 --2- 172.21.15.63:0/1750389209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5720 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f4308020c80 tx=0x7f43080058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8250883c6100668e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.377+0000 7f4310c00700 1 -- 172.21.15.63:0/1750389209 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308005110 con 0x55d33fc629f0 2024-05-27T08:07:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.377+0000 7f4310c00700 1 -- 172.21.15.63:0/1750389209 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4308024070 con 0x55d33fc629f0 2024-05-27T08:07:11.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.377+0000 7f4310c00700 1 -- 172.21.15.63:0/1750389209 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308010430 con 0x55d33fc629f0 2024-05-27T08:07:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.378+0000 7f43314fab80 1 -- 172.21.15.63:0/1750389209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 msgr2=0x55d33fcf5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.378+0000 7f43314fab80 1 --2- 172.21.15.63:0/1750389209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5720 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f4308020c80 tx=0x7f43080058e0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.378+0000 7f43314fab80 1 -- 172.21.15.63:0/1750389209 shutdown_connections 2024-05-27T08:07:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.378+0000 7f43314fab80 1 --2- 172.21.15.63:0/1750389209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fcfe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.378+0000 7f43314fab80 1 --2- 172.21.15.63:0/1750389209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fcfae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.378+0000 7f43314fab80 1 --2- 172.21.15.63:0/1750389209 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fcf5720 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.378+0000 7f43314fab80 1 -- 172.21.15.63:0/1750389209 >> 172.21.15.63:0/1750389209 conn(0x55d33fc10390 msgr2=0x55d33fcfdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:11.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.379+0000 7f43314fab80 1 -- 172.21.15.63:0/1750389209 shutdown_connections 2024-05-27T08:07:11.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.379+0000 7f43314fab80 1 -- 172.21.15.63:0/1750389209 wait complete. 2024-05-27T08:07:11.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.380+0000 7f43314fab80 1 -- start start 2024-05-27T08:07:11.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.380+0000 7f43314fab80 1 Processor -- start 2024-05-27T08:07:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f43314fab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fd729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f43314fab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fd72f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f43314fab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fd03340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f43314fab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b0d30 con 0x55d33fc629f0 2024-05-27T08:07:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f43314fab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b21a0 con 0x55d33fb76e40 2024-05-27T08:07:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f43314fab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b1400 con 0x55d33fcf5e70 2024-05-27T08:07:11.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f43314fab80 10 client.0 _ll_register_callbacks cb 0x55d33fbbdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f431b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fd72f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f431b400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fd72f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49568/0 (socket says 172.21.15.63:49568) 2024-05-27T08:07:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f431b400700 1 -- 172.21.15.63:0/1978897196 learned_addr learned my addr 172.21.15.63:0/1978897196 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f431be00700 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fd729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.381+0000 7f4320a00700 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fd03340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.382+0000 7f431b400700 1 -- 172.21.15.63:0/1978897196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 msgr2=0x55d33fd03340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.382+0000 7f431b400700 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fd03340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.382+0000 7f431b400700 1 -- 172.21.15.63:0/1978897196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 msgr2=0x55d33fd729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:11.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.382+0000 7f431b400700 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fd729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.382+0000 7f431b400700 1 -- 172.21.15.63:0/1978897196 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d33fd03980 con 0x55d33fc629f0 2024-05-27T08:07:11.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.382+0000 7f431b400700 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fd72f00 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f4308013cc0 tx=0x7f4308003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef6344b0fee3226 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:11.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.382+0000 7f4306000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.383+0000 7f4306000700 1 -- 172.21.15.63:0/1978897196 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308005210 con 0x55d33fc629f0 2024-05-27T08:07:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.383+0000 7f4306000700 1 -- 172.21.15.63:0/1978897196 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7f43080053b0 con 0x55d33fc629f0 2024-05-27T08:07:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.383+0000 7f43314fab80 1 -- 172.21.15.63:0/1978897196 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4308024070 con 0x55d33fc629f0 2024-05-27T08:07:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f4306000700 1 -- 172.21.15.63:0/1978897196 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f43080055b0 con 0x55d33fc629f0 2024-05-27T08:07:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f43314fab80 10 client.21873 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f43314fab80 1 -- 172.21.15.63:0/1978897196 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d33fd7a030 con 0x55d33fc629f0 2024-05-27T08:07:11.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f4306000700 1 -- 172.21.15.63:0/1978897196 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4308024070 con 0x55d33fc629f0 2024-05-27T08:07:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f4306000700 1 -- 172.21.15.63:0/1978897196 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f430801f070 con 0x55d33fc629f0 2024-05-27T08:07:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f43314fab80 10 client.21873 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f43314fab80 10 client.21873 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:07:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f43314fab80 20 client.21873 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f43314fab80 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d33fd7a3a0 0x55d33fd7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f43314fab80 4 client.21873 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:07:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.384+0000 7f43314fab80 1 -- 172.21.15.63:0/1978897196 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d33fc1f570 con 0x55d33fd7a3a0 2024-05-27T08:07:11.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.385+0000 7f431be00700 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d33fd7a3a0 0x55d33fd7c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:11.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.386+0000 7f431be00700 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d33fd7a3a0 0x55d33fd7c7c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1f10e33a9f528b0 server_cookie=11ea131d7571f30e in_seq=0 out_seq=0 2024-05-27T08:07:11.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.386+0000 7f4306000700 10 client.21873 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:07:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.387+0000 7f4306000700 1 -- 172.21.15.63:0/1978897196 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d33fc1f570 con 0x55d33fd7a3a0 2024-05-27T08:07:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.387+0000 7f4306000700 10 client.21873 handle_command_reply: tid=0 2024-05-27T08:07:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.387+0000 7f43314fab80 1 client.21873 shutdown 2024-05-27T08:07:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.387+0000 7f43314fab80 1 -- 172.21.15.63:0/1978897196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 msgr2=0x55d33fd72f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:11.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.387+0000 7f43314fab80 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fd72f00 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f4308013cc0 tx=0x7f4308003a10 comp rx=0 tx=0).stop 2024-05-27T08:07:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.388+0000 7f43314fab80 1 -- 172.21.15.63:0/1978897196 shutdown_connections 2024-05-27T08:07:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.388+0000 7f43314fab80 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d33fcf5e70 0x55d33fd03340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.388+0000 7f43314fab80 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d33fb76e40 0x55d33fd729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.388+0000 7f43314fab80 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d33fd7a3a0 0x55d33fd7c7c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.388+0000 7f43314fab80 1 --2- 172.21.15.63:0/1978897196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d33fc629f0 0x55d33fd72f00 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.388+0000 7f4306000700 0 client.21873 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:07:11.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.388+0000 7f43314fab80 1 -- 172.21.15.63:0/1978897196 >> 172.21.15.63:0/1978897196 conn(0x55d33fc10390 msgr2=0x55d33fcfe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.389+0000 7f43314fab80 1 -- 172.21.15.63:0/1978897196 shutdown_connections 2024-05-27T08:07:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.389+0000 7f43314fab80 1 -- 172.21.15.63:0/1978897196 wait complete. 2024-05-27T08:07:11.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.389+0000 7f43314fab80 20 client.21873 trim_cache size 0 max 16384 2024-05-27T08:07:11.391 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:07:11.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.390+0000 7f4322a00700 1 -- 172.21.15.63:0/2286913436 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b2a70 msgr2=0x7f43000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:11.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.390+0000 7f4322a00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b2a70 0x7f43000b4f80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4308013570 tx=0x7f43080098c0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.390+0000 7f4322a00700 1 -- 172.21.15.63:0/2286913436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109850 msgr2=0x7f431c1a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:11.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.390+0000 7f4322a00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109850 0x7f431c1a16d0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f4314007a00 tx=0x7f4314007a30 comp rx=0 tx=0).stop 2024-05-27T08:07:11.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.391+0000 7f4322a00700 1 -- 172.21.15.63:0/2286913436 shutdown_connections 2024-05-27T08:07:11.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.391+0000 7f4322a00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43000b2a70 0x7f43000b4f80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.391+0000 7f4322a00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c109850 0x7f431c1a16d0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.391+0000 7f4322a00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f431c1045d0 0x7f431c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.391+0000 7f4322a00700 1 --2- 172.21.15.63:0/2286913436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c108f00 0x7f431c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:11.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.391+0000 7f4322a00700 1 -- 172.21.15.63:0/2286913436 >> 172.21.15.63:0/2286913436 conn(0x7f431c0ffe60 msgr2=0x7f431c110e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:11.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.391+0000 7f4322a00700 1 -- 172.21.15.63:0/2286913436 shutdown_connections 2024-05-27T08:07:11.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:11.391+0000 7f4322a00700 1 -- 172.21.15.63:0/2286913436 wait complete. 2024-05-27T08:07:11.889 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:11 smithi063 ceph-mon[31613]: pgmap v1065: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 29 op/s 2024-05-27T08:07:11.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:11 smithi086 ceph-mon[34687]: pgmap v1065: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 29 op/s 2024-05-27T08:07:11.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:11 smithi149 ceph-mon[35408]: pgmap v1065: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 29 op/s 2024-05-27T08:07:12.427 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:07:13.020 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:07:13.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:13 smithi086 ceph-mon[34687]: pgmap v1066: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-27T08:07:13.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:13 smithi149 ceph-mon[35408]: pgmap v1066: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-27T08:07:13.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:13 smithi063 ceph-mon[31613]: pgmap v1066: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-27T08:07:15.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.771+0000 7f4b9f200700 1 -- 172.21.15.63:0/2785287182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ba01043f0 msgr2=0x7f4ba0104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.771+0000 7f4b9f200700 1 --2- 172.21.15.63:0/2785287182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ba01043f0 0x7f4ba0104800 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f4b900099c0 tx=0x7f4b90009cd0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.771+0000 7f4b9f200700 1 -- 172.21.15.63:0/2785287182 shutdown_connections 2024-05-27T08:07:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.771+0000 7f4b9f200700 1 --2- 172.21.15.63:0/2785287182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba0109670 0x7f4ba0110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.771+0000 7f4b9f200700 1 --2- 172.21.15.63:0/2785287182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ba01043f0 0x7f4ba0104800 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.771+0000 7f4b9f200700 1 --2- 172.21.15.63:0/2785287182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0108d20 0x7f4ba0109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.771+0000 7f4b9f200700 1 -- 172.21.15.63:0/2785287182 >> 172.21.15.63:0/2785287182 conn(0x7f4ba0079d00 msgr2=0x7f4ba007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.772+0000 7f4b9f200700 1 -- 172.21.15.63:0/2785287182 shutdown_connections 2024-05-27T08:07:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.772+0000 7f4b9f200700 1 -- 172.21.15.63:0/2785287182 wait complete. 2024-05-27T08:07:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.773+0000 7f4b9f200700 1 Processor -- start 2024-05-27T08:07:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.774+0000 7f4b9f200700 1 -- start start 2024-05-27T08:07:15.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.774+0000 7f4b9f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba01043f0 0x7f4ba019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.774+0000 7f4b9f200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ba0108d20 0x7f4ba019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.774+0000 7f4b9f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0109670 0x7f4ba01a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.774+0000 7f4b9f200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01141a0 con 0x7f4ba0108d20 2024-05-27T08:07:15.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.774+0000 7f4b9f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0114020 con 0x7f4ba01043f0 2024-05-27T08:07:15.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.774+0000 7f4b9f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0114320 con 0x7f4ba0109670 2024-05-27T08:07:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.775+0000 7f4b9d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ba0108d20 0x7f4ba019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.775+0000 7f4b9d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ba0108d20 0x7f4ba019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49588/0 (socket says 172.21.15.63:49588) 2024-05-27T08:07:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.775+0000 7f4b9d400700 1 -- 172.21.15.63:0/631339121 learned_addr learned my addr 172.21.15.63:0/631339121 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:15.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.775+0000 7f4b9de00700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba01043f0 0x7f4ba019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.775+0000 7f4b9e800700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0109670 0x7f4ba01a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.775+0000 7f4b9d400700 1 -- 172.21.15.63:0/631339121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0109670 msgr2=0x7f4ba01a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:15.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.775+0000 7f4b9d400700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0109670 0x7f4ba01a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.775+0000 7f4b9d400700 1 -- 172.21.15.63:0/631339121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba01043f0 msgr2=0x7f4ba019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:15.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.775+0000 7f4b9d400700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba01043f0 0x7f4ba019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.775+0000 7f4b9d400700 1 -- 172.21.15.63:0/631339121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba01a1d20 con 0x7f4ba0108d20 2024-05-27T08:07:15.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.776+0000 7f4b9d400700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ba0108d20 0x7f4ba019d3f0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00d980 tx=0x7f4b8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:15.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.776+0000 7f4b96a00700 1 -- 172.21.15.63:0/631339121 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c017070 con 0x7f4ba0108d20 2024-05-27T08:07:15.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.776+0000 7f4b96a00700 1 -- 172.21.15.63:0/631339121 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b8c009dc0 con 0x7f4ba0108d20 2024-05-27T08:07:15.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.776+0000 7f4b9f200700 1 -- 172.21.15.63:0/631339121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba01a2010 con 0x7f4ba0108d20 2024-05-27T08:07:15.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.776+0000 7f4b96a00700 1 -- 172.21.15.63:0/631339121 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c00f3c0 con 0x7f4ba0108d20 2024-05-27T08:07:15.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.776+0000 7f4b9f200700 1 -- 172.21.15.63:0/631339121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba01a9930 con 0x7f4ba0108d20 2024-05-27T08:07:15.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.781+0000 7f4b96a00700 1 -- 172.21.15.63:0/631339121 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4b8c00f560 con 0x7f4ba0108d20 2024-05-27T08:07:15.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.782+0000 7f4bb3840b80 1 Processor -- start 2024-05-27T08:07:15.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.782+0000 7f4bb3840b80 1 -- start start 2024-05-27T08:07:15.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.783+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 0x55abf2d88ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.783+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d93ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.783+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89820 0x55abf2d96020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.783+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55abf2cb7e40 con 0x55abf2d89410 2024-05-27T08:07:15.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.783+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55abf2ca79e0 con 0x55abf2d89820 2024-05-27T08:07:15.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.783+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55abf2caba20 con 0x55abf2c0ae40 2024-05-27T08:07:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.783+0000 7f4b9de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d93ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.783+0000 7f4b9de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d93ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49594/0 (socket says 172.21.15.63:49594) 2024-05-27T08:07:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.783+0000 7f4b9de00700 1 -- 172.21.15.63:0/1318863313 learned_addr learned my addr 172.21.15.63:0/1318863313 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.783+0000 7f4b9e800700 1 --2- 172.21.15.63:0/1318863313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 0x55abf2d88ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.784+0000 7f4b9de00700 1 -- 172.21.15.63:0/1318863313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 msgr2=0x55abf2d88ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.784+0000 7f4b9de00700 1 --2- 172.21.15.63:0/1318863313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 0x55abf2d88ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.784+0000 7f4b9de00700 1 -- 172.21.15.63:0/1318863313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89820 msgr2=0x55abf2d96020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:07:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.784+0000 7f4b9de00700 1 --2- 172.21.15.63:0/1318863313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89820 0x55abf2d96020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.784+0000 7f4b9de00700 1 -- 172.21.15.63:0/1318863313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf2c70420 con 0x55abf2d89410 2024-05-27T08:07:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.784+0000 7f4b96a00700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b840b29f0 0x7f4b840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.784+0000 7f4b96a00700 1 -- 172.21.15.63:0/631339121 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7f4b8c0e2590 con 0x7f4ba0108d20 2024-05-27T08:07:15.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.784+0000 7f4b9de00700 1 --2- 172.21.15.63:0/1318863313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d93ad0 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000b580 tx=0x7f4b900096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac036a6df875cb6e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:15.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.784+0000 7f4b9e800700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b840b29f0 0x7f4b840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.785+0000 7f4b8b400700 1 -- 172.21.15.63:0/1318863313 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90027070 con 0x55abf2d89410 2024-05-27T08:07:15.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.785+0000 7f4b8b400700 1 -- 172.21.15.63:0/1318863313 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b9000bc20 con 0x55abf2d89410 2024-05-27T08:07:15.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.785+0000 7f4b8b400700 1 -- 172.21.15.63:0/1318863313 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90003b10 con 0x55abf2d89410 2024-05-27T08:07:15.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.786+0000 7f4bb3840b80 1 -- 172.21.15.63:0/1318863313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 msgr2=0x55abf2d93ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:15.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.786+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/1318863313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d93ad0 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000b580 tx=0x7f4b900096e0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.786+0000 7f4b9e800700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b840b29f0 0x7f4b840b4f00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55abf2d85690 tx=0x7f4b98009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:15.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.786+0000 7f4bb3840b80 1 -- 172.21.15.63:0/1318863313 shutdown_connections 2024-05-27T08:07:15.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.786+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/1318863313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89820 0x55abf2d96020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.786+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/1318863313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 0x55abf2d88ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.786+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/1318863313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d93ad0 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.786+0000 7f4bb3840b80 1 -- 172.21.15.63:0/1318863313 >> 172.21.15.63:0/1318863313 conn(0x55abf2c6afc0 msgr2=0x55abf2d84d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:15.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.786+0000 7f4bb3840b80 1 -- 172.21.15.63:0/1318863313 shutdown_connections 2024-05-27T08:07:15.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.787+0000 7f4bb3840b80 1 -- 172.21.15.63:0/1318863313 wait complete. 2024-05-27T08:07:15.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.787+0000 7f4bb3840b80 1 -- start start 2024-05-27T08:07:15.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.787+0000 7f4bb3840b80 1 Processor -- start 2024-05-27T08:07:15.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.788+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c0ae40 0x55abf2d93680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.788+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2d89410 0x55abf2d93bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.788+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89820 0x55abf2d94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.788+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55abf2badfa0 con 0x55abf2c0ae40 2024-05-27T08:07:15.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.788+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55abf2c48760 con 0x55abf2d89820 2024-05-27T08:07:15.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.788+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55abf2d96930 con 0x55abf2d89410 2024-05-27T08:07:15.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.788+0000 7f4bb3840b80 10 client.0 _ll_register_callbacks cb 0x55abf2c51c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:15.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.788+0000 7f4b9de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c0ae40 0x55abf2d93680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.788+0000 7f4b9de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c0ae40 0x55abf2d93680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49598/0 (socket says 172.21.15.63:49598) 2024-05-27T08:07:15.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.789+0000 7f4b9de00700 1 -- 172.21.15.63:0/2135717020 learned_addr learned my addr 172.21.15.63:0/2135717020 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:15.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.789+0000 7f4b9d400700 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2d89410 0x55abf2d93bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.789+0000 7f4b9de00700 1 -- 172.21.15.63:0/2135717020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2d89410 msgr2=0x55abf2d93bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:15.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.789+0000 7f4b9de00700 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2d89410 0x55abf2d93bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.789+0000 7f4b9de00700 1 -- 172.21.15.63:0/2135717020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89820 msgr2=0x55abf2d94100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:15.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.789+0000 7f4b9de00700 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89820 0x55abf2d94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.789+0000 7f4b9de00700 1 -- 172.21.15.63:0/2135717020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abf2bf78d0 con 0x55abf2c0ae40 2024-05-27T08:07:15.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.789+0000 7f4b9d400700 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2d89410 0x55abf2d93bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:07:15.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.789+0000 7f4b9de00700 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c0ae40 0x55abf2d93680 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f4b900058e0 tx=0x7f4b9000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a6328181e4907f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:15.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.790+0000 7f4b7fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:15.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.790+0000 7f4b7fe00700 1 -- 172.21.15.63:0/2135717020 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90030020 con 0x55abf2c0ae40 2024-05-27T08:07:15.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.790+0000 7f4bb3840b80 1 -- 172.21.15.63:0/2135717020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b9000bc20 con 0x55abf2c0ae40 2024-05-27T08:07:15.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.790+0000 7f4b7fe00700 1 -- 172.21.15.63:0/2135717020 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7f4b90005280 con 0x55abf2c0ae40 2024-05-27T08:07:15.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.790+0000 7f4b7fe00700 1 -- 172.21.15.63:0/2135717020 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4b90027040 con 0x55abf2c0ae40 2024-05-27T08:07:15.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.790+0000 7f4bb3840b80 10 client.21885 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.790+0000 7f4bb3840b80 1 -- 172.21.15.63:0/2135717020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55abf2c78060 con 0x55abf2c0ae40 2024-05-27T08:07:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.791+0000 7f4b7fe00700 1 -- 172.21.15.63:0/2135717020 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b9000bc20 con 0x55abf2c0ae40 2024-05-27T08:07:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.791+0000 7f4b7fe00700 1 -- 172.21.15.63:0/2135717020 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4b90022580 con 0x55abf2c0ae40 2024-05-27T08:07:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.791+0000 7f4bb3840b80 10 client.21885 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.791+0000 7f4bb3840b80 10 client.21885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.791+0000 7f4bb3840b80 20 client.21885 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.791+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55abf2da3860 0x55abf2da5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.791+0000 7f4bb3840b80 4 client.21885 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:07:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.791+0000 7f4bb3840b80 1 -- 172.21.15.63:0/2135717020 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abf2cb9d90 con 0x55abf2da3860 2024-05-27T08:07:15.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.792+0000 7f4b9d400700 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55abf2da3860 0x55abf2da5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.793+0000 7f4b9d400700 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55abf2da3860 0x55abf2da5c80 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d1000e0d727a35 server_cookie=8163f4598cf0bea3 in_seq=0 out_seq=0 2024-05-27T08:07:15.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.793+0000 7f4b7fe00700 10 client.21885 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:15.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.795+0000 7f4b7fe00700 1 -- 172.21.15.63:0/2135717020 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55abf2cb9d90 con 0x55abf2da3860 2024-05-27T08:07:15.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.795+0000 7f4b7fe00700 10 client.21885 handle_command_reply: tid=0 2024-05-27T08:07:15.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.795+0000 7f4bb3840b80 1 client.21885 shutdown 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.795+0000 7f4bb3840b80 1 -- 172.21.15.63:0/2135717020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c0ae40 msgr2=0x55abf2d93680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.795+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c0ae40 0x55abf2d93680 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f4b900058e0 tx=0x7f4b9000b840 comp rx=0 tx=0).stop 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.796+0000 7f4bb3840b80 1 -- 172.21.15.63:0/2135717020 shutdown_connections 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.796+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55abf2da3860 0x55abf2da5c80 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.796+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89820 0x55abf2d94100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.796+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c0ae40 0x55abf2d93680 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.796+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/2135717020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2d89410 0x55abf2d93bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.796+0000 7f4b7fe00700 0 client.21885 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.796+0000 7f4bb3840b80 1 -- 172.21.15.63:0/2135717020 >> 172.21.15.63:0/2135717020 conn(0x55abf2c6afc0 msgr2=0x55abf2d83ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.796+0000 7f4bb3840b80 1 -- 172.21.15.63:0/2135717020 shutdown_connections 2024-05-27T08:07:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.796+0000 7f4bb3840b80 1 -- 172.21.15.63:0/2135717020 wait complete. 2024-05-27T08:07:15.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.796+0000 7f4bb3840b80 20 client.21885 trim_cache size 0 max 16384 2024-05-27T08:07:15.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.818+0000 7f4bb3840b80 1 Processor -- start 2024-05-27T08:07:15.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.818+0000 7f4bb3840b80 1 -- start start 2024-05-27T08:07:15.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.819+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c6afc0 0x55abf2d95b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.819+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2c0ae40 0x55abf2d8d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.819+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d9fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.819+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01b4f70 con 0x55abf2d89410 2024-05-27T08:07:15.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.819+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01b2800 con 0x55abf2c0ae40 2024-05-27T08:07:15.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.819+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01b0670 con 0x55abf2c6afc0 2024-05-27T08:07:15.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.819+0000 7f4b9e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d9fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.819+0000 7f4b9e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d9fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49600/0 (socket says 172.21.15.63:49600) 2024-05-27T08:07:15.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.819+0000 7f4b9e800700 1 -- 172.21.15.63:0/1509375386 learned_addr learned my addr 172.21.15.63:0/1509375386 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:15.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.820+0000 7f4b9de00700 1 --2- 172.21.15.63:0/1509375386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c6afc0 0x55abf2d95b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.820+0000 7f4b9d400700 1 --2- 172.21.15.63:0/1509375386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2c0ae40 0x55abf2d8d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.820+0000 7f4b9e800700 1 -- 172.21.15.63:0/1509375386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c6afc0 msgr2=0x55abf2d95b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:15.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.820+0000 7f4b9e800700 1 --2- 172.21.15.63:0/1509375386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c6afc0 0x55abf2d95b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.820+0000 7f4b9e800700 1 -- 172.21.15.63:0/1509375386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2c0ae40 msgr2=0x55abf2d8d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:15.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.820+0000 7f4b9e800700 1 --2- 172.21.15.63:0/1509375386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2c0ae40 0x55abf2d8d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.820+0000 7f4b9e800700 1 -- 172.21.15.63:0/1509375386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf2d8a420 con 0x55abf2d89410 2024-05-27T08:07:15.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.821+0000 7f4b9e800700 1 --2- 172.21.15.63:0/1509375386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d9fc00 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f4b98016580 tx=0x7f4b98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf3af212710dda3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:15.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.821+0000 7f4b8aa00700 1 -- 172.21.15.63:0/1509375386 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9801d050 con 0x55abf2d89410 2024-05-27T08:07:15.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.821+0000 7f4b8aa00700 1 -- 172.21.15.63:0/1509375386 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b98027070 con 0x55abf2d89410 2024-05-27T08:07:15.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.821+0000 7f4b8aa00700 1 -- 172.21.15.63:0/1509375386 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98017400 con 0x55abf2d89410 2024-05-27T08:07:15.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.822+0000 7f4bb3840b80 1 -- 172.21.15.63:0/1509375386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 msgr2=0x55abf2d9fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:15.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.822+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/1509375386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d9fc00 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f4b98016580 tx=0x7f4b98004ad0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.822+0000 7f4bb3840b80 1 -- 172.21.15.63:0/1509375386 shutdown_connections 2024-05-27T08:07:15.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.822+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/1509375386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2d89410 0x55abf2d9fc00 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.822+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/1509375386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c6afc0 0x55abf2d95b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.822+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/1509375386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2c0ae40 0x55abf2d8d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.822+0000 7f4bb3840b80 1 -- 172.21.15.63:0/1509375386 >> 172.21.15.63:0/1509375386 conn(0x55abf2d7df60 msgr2=0x55abf2d93460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:15.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.823+0000 7f4bb3840b80 1 -- 172.21.15.63:0/1509375386 shutdown_connections 2024-05-27T08:07:15.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.823+0000 7f4bb3840b80 1 -- 172.21.15.63:0/1509375386 wait complete. 2024-05-27T08:07:15.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.823+0000 7f4bb3840b80 1 -- start start 2024-05-27T08:07:15.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.823+0000 7f4bb3840b80 1 Processor -- start 2024-05-27T08:07:15.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.824+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 0x55abf2da9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.824+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c6afc0 0x55abf2da6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.824+0000 7f4bb3840b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89410 0x55abf2da6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.824+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01b0cc0 con 0x55abf2c6afc0 2024-05-27T08:07:15.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.824+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01b2130 con 0x55abf2d89410 2024-05-27T08:07:15.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.824+0000 7f4bb3840b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01b1390 con 0x55abf2c0ae40 2024-05-27T08:07:15.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.824+0000 7f4bb3840b80 10 client.0 _ll_register_callbacks cb 0x55abf2c51c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.824+0000 7f4b9d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c6afc0 0x55abf2da6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.824+0000 7f4b9d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c6afc0 0x55abf2da6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49618/0 (socket says 172.21.15.63:49618) 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.824+0000 7f4b9d400700 1 -- 172.21.15.63:0/3259241983 learned_addr learned my addr 172.21.15.63:0/3259241983 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.825+0000 7f4b9de00700 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 0x55abf2da9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.825+0000 7f4b9e800700 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89410 0x55abf2da6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.825+0000 7f4b9d400700 1 -- 172.21.15.63:0/3259241983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 msgr2=0x55abf2da9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.825+0000 7f4b9d400700 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 0x55abf2da9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.825+0000 7f4b9d400700 1 -- 172.21.15.63:0/3259241983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89410 msgr2=0x55abf2da6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.825+0000 7f4b9d400700 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89410 0x55abf2da6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.825+0000 7f4b9d400700 1 -- 172.21.15.63:0/3259241983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abf2da6ea0 con 0x55abf2c6afc0 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.825+0000 7f4b9de00700 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 0x55abf2da9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:07:15.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.826+0000 7f4b9d400700 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c6afc0 0x55abf2da6420 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0df920 tx=0x7f4b8c0108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb8651efd0cdb98 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:15.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.826+0000 7f4b7fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:15.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.826+0000 7f4b7fe00700 1 -- 172.21.15.63:0/3259241983 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c020040 con 0x55abf2c6afc0 2024-05-27T08:07:15.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.826+0000 7f4b7fe00700 1 -- 172.21.15.63:0/3259241983 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 9571+0+0 (secure 0 0 0) 0x7f4b8c010460 con 0x55abf2c6afc0 2024-05-27T08:07:15.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.826+0000 7f4bb3840b80 1 -- 172.21.15.63:0/3259241983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b98027070 con 0x55abf2c6afc0 2024-05-27T08:07:15.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.826+0000 7f4b7fe00700 1 -- 172.21.15.63:0/3259241983 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4b8c0ad800 con 0x55abf2c6afc0 2024-05-27T08:07:15.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.826+0000 7f4bb3840b80 10 client.21891 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:15.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.826+0000 7f4bb3840b80 1 -- 172.21.15.63:0/3259241983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55abf2e0dc70 con 0x55abf2c6afc0 2024-05-27T08:07:15.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.827+0000 7f4b7fe00700 1 -- 172.21.15.63:0/3259241983 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b98027070 con 0x55abf2c6afc0 2024-05-27T08:07:15.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.827+0000 7f4b7fe00700 1 -- 172.21.15.63:0/3259241983 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4b8c0b0b80 con 0x55abf2c6afc0 2024-05-27T08:07:15.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.827+0000 7f4bb3840b80 10 client.21891 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:15.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.827+0000 7f4bb3840b80 10 client.21891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:15.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.827+0000 7f4bb3840b80 20 client.21891 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:15.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.827+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55abf2d89820 0x55abf2e10370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:15.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.827+0000 7f4bb3840b80 4 client.21891 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:07:15.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.827+0000 7f4bb3840b80 1 -- 172.21.15.63:0/3259241983 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abf2cb3570 con 0x55abf2d89820 2024-05-27T08:07:15.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.828+0000 7f4b9de00700 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55abf2d89820 0x55abf2e10370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:15.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.829+0000 7f4b9de00700 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55abf2d89820 0x55abf2e10370 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a5c71b20d12758 server_cookie=862841b861467c57 in_seq=0 out_seq=0 2024-05-27T08:07:15.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.829+0000 7f4b7fe00700 10 client.21891 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:15.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.829+0000 7f4b7fe00700 1 -- 172.21.15.63:0/3259241983 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abf2cb3570 con 0x55abf2d89820 2024-05-27T08:07:15.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.829+0000 7f4b7fe00700 10 client.21891 handle_command_reply: tid=0 2024-05-27T08:07:15.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.829+0000 7f4bb3840b80 1 client.21891 shutdown 2024-05-27T08:07:15.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 1 -- 172.21.15.63:0/3259241983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c6afc0 msgr2=0x55abf2da6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:15.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c6afc0 0x55abf2da6420 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0df920 tx=0x7f4b8c0108b0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 1 -- 172.21.15.63:0/3259241983 shutdown_connections 2024-05-27T08:07:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55abf2d89820 0x55abf2e10370 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55abf2d89410 0x55abf2da6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55abf2c0ae40 0x55abf2da9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 1 --2- 172.21.15.63:0/3259241983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55abf2c6afc0 0x55abf2da6420 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 1 -- 172.21.15.63:0/3259241983 >> 172.21.15.63:0/3259241983 conn(0x55abf2d7df60 msgr2=0x55abf2d93e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4b7fe00700 0 client.21891 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 1 -- 172.21.15.63:0/3259241983 shutdown_connections 2024-05-27T08:07:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 1 -- 172.21.15.63:0/3259241983 wait complete. 2024-05-27T08:07:15.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.830+0000 7f4bb3840b80 20 client.21891 trim_cache size 0 max 16384 2024-05-27T08:07:15.834 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:07:15.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.831+0000 7f4b9f200700 1 -- 172.21.15.63:0/631339121 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b840b29f0 msgr2=0x7f4b840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:15.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.831+0000 7f4b9f200700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b840b29f0 0x7f4b840b4f00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55abf2d85690 tx=0x7f4b98009040 comp rx=0 tx=0).stop 2024-05-27T08:07:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.831+0000 7f4b9f200700 1 -- 172.21.15.63:0/631339121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ba0108d20 msgr2=0x7f4ba019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.831+0000 7f4b9f200700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ba0108d20 0x7f4ba019d3f0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00d980 tx=0x7f4b8c00dc90 comp rx=0 tx=0).stop 2024-05-27T08:07:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.832+0000 7f4b9f200700 1 -- 172.21.15.63:0/631339121 shutdown_connections 2024-05-27T08:07:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.832+0000 7f4b9f200700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f4b840b29f0 0x7f4b840b4f00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.832+0000 7f4b9f200700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0109670 0x7f4ba01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.832+0000 7f4b9f200700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba01043f0 0x7f4ba019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.832+0000 7f4b9f200700 1 --2- 172.21.15.63:0/631339121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ba0108d20 0x7f4ba019d3f0 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.832+0000 7f4b9f200700 1 -- 172.21.15.63:0/631339121 >> 172.21.15.63:0/631339121 conn(0x7f4ba0079d00 msgr2=0x7f4ba0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.832+0000 7f4b9f200700 1 -- 172.21.15.63:0/631339121 shutdown_connections 2024-05-27T08:07:15.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:15.832+0000 7f4b9f200700 1 -- 172.21.15.63:0/631339121 wait complete. 2024-05-27T08:07:15.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:15 smithi086 ceph-mon[34687]: pgmap v1067: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-27T08:07:15.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:15 smithi149 ceph-mon[35408]: pgmap v1067: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-27T08:07:15.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:15 smithi063 ceph-mon[31613]: pgmap v1067: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-27T08:07:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:17 smithi086 ceph-mon[34687]: osdmap e148: 12 total, 12 up, 12 in 2024-05-27T08:07:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:17 smithi086 ceph-mon[34687]: pgmap v1069: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.5 MiB/s wr, 13 op/s 2024-05-27T08:07:17.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:17 smithi149 ceph-mon[35408]: osdmap e148: 12 total, 12 up, 12 in 2024-05-27T08:07:17.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:17 smithi149 ceph-mon[35408]: pgmap v1069: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.5 MiB/s wr, 13 op/s 2024-05-27T08:07:17.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:17 smithi063 ceph-mon[31613]: osdmap e148: 12 total, 12 up, 12 in 2024-05-27T08:07:17.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:17 smithi063 ceph-mon[31613]: pgmap v1069: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.5 MiB/s wr, 13 op/s 2024-05-27T08:07:18.273 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:07:18.273 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:07:18.776 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:07:19.849 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:19 smithi063 ceph-mon[31613]: pgmap v1070: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:07:19.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:19 smithi086 ceph-mon[34687]: pgmap v1070: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:07:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:19 smithi149 ceph-mon[35408]: pgmap v1070: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:07:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.291+0000 7f72ba400640 1 -- 172.21.15.63:0/470094753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410e020 msgr2=0x7f72b4110510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.291+0000 7f72ba400640 1 --2- 172.21.15.63:0/470094753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410e020 0x7f72b4110510 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009ca0 tx=0x7f72ac02a9e0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.291+0000 7f72ba400640 1 -- 172.21.15.63:0/470094753 shutdown_connections 2024-05-27T08:07:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.291+0000 7f72ba400640 1 --2- 172.21.15.63:0/470094753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410e020 0x7f72b4110510 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.291+0000 7f72ba400640 1 --2- 172.21.15.63:0/470094753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410b670 0x7f72b410da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.291+0000 7f72ba400640 1 --2- 172.21.15.63:0/470094753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4104c10 0x7f72b4105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.291+0000 7f72ba400640 1 -- 172.21.15.63:0/470094753 >> 172.21.15.63:0/470094753 conn(0x7f72b40fe210 msgr2=0x7f72b4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.291+0000 7f72ba400640 1 -- 172.21.15.63:0/470094753 shutdown_connections 2024-05-27T08:07:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.291+0000 7f72ba400640 1 -- 172.21.15.63:0/470094753 wait complete. 2024-05-27T08:07:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.292+0000 7f72ba400640 1 Processor -- start 2024-05-27T08:07:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.293+0000 7f72ba400640 1 -- start start 2024-05-27T08:07:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.293+0000 7f72ba400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4104c10 0x7f72b419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.293+0000 7f72ba400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410b670 0x7f72b419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.293+0000 7f72ba400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e020 0x7f72b41a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.293+0000 7f72ba400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4113b80 con 0x7f72b410b670 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.293+0000 7f72ba400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4113a00 con 0x7f72b4104c10 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.293+0000 7f72ba400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4113d00 con 0x7f72b410e020 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410b670 0x7f72b419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410b670 0x7f72b419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33114/0 (socket says 172.21.15.63:33114) 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b3e00640 1 -- 172.21.15.63:0/3178040386 learned_addr learned my addr 172.21.15.63:0/3178040386 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b9000640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4104c10 0x7f72b419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b9a00640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e020 0x7f72b41a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b3e00640 1 -- 172.21.15.63:0/3178040386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e020 msgr2=0x7f72b41a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b3e00640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e020 0x7f72b41a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b3e00640 1 -- 172.21.15.63:0/3178040386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4104c10 msgr2=0x7f72b419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b3e00640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4104c10 0x7f72b419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b3e00640 1 -- 172.21.15.63:0/3178040386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b41a1660 con 0x7f72b410b670 2024-05-27T08:07:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.294+0000 7f72b9a00640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e020 0x7f72b41a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:07:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.295+0000 7f72b3e00640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410b670 0x7f72b419cd60 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f72a000d950 tx=0x7f72a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.295+0000 7f72b1600640 1 -- 172.21.15.63:0/3178040386 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0017070 con 0x7f72b410b670 2024-05-27T08:07:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.295+0000 7f72ba400640 1 -- 172.21.15.63:0/3178040386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b41a1950 con 0x7f72b410b670 2024-05-27T08:07:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.295+0000 7f72b1600640 1 -- 172.21.15.63:0/3178040386 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a0009c30 con 0x7f72b410b670 2024-05-27T08:07:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.295+0000 7f72b1600640 1 -- 172.21.15.63:0/3178040386 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a000f620 con 0x7f72b410b670 2024-05-27T08:07:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.295+0000 7f72ba400640 1 -- 172.21.15.63:0/3178040386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b41a92d0 con 0x7f72b410b670 2024-05-27T08:07:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.300+0000 7f72c2a2f740 1 Processor -- start 2024-05-27T08:07:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.300+0000 7f72c2a2f740 1 -- start start 2024-05-27T08:07:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.300+0000 7f72b1600640 1 -- 172.21.15.63:0/3178040386 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f72a000f7c0 con 0x7f72b410b670 2024-05-27T08:07:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.300+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5951b0 0x5599fe937a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.300+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe937fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.301+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5744d0 0x5599fe9384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.301+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5599fe8a8b90 con 0x5599fe5951b0 2024-05-27T08:07:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.301+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599fe898730 con 0x5599fe5744d0 2024-05-27T08:07:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.301+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5599fe89c770 con 0x5599fe56c2f0 2024-05-27T08:07:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.301+0000 7f72b9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5951b0 0x5599fe937a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.301+0000 7f72b9a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5951b0 0x5599fe937a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33118/0 (socket says 172.21.15.63:33118) 2024-05-27T08:07:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.301+0000 7f72b9a00640 1 -- 172.21.15.63:0/3835080555 learned_addr learned my addr 172.21.15.63:0/3835080555 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.301+0000 7f72b9000640 1 --2- 172.21.15.63:0/3835080555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe937fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.301+0000 7f72b3e00640 1 --2- 172.21.15.63:0/3835080555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5744d0 0x5599fe9384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.302+0000 7f72b3e00640 1 -- 172.21.15.63:0/3835080555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 msgr2=0x5599fe937fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.302+0000 7f72b3e00640 1 --2- 172.21.15.63:0/3835080555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe937fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.302+0000 7f72b1600640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b29d0 0x7f729c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.302+0000 7f72b3e00640 1 -- 172.21.15.63:0/3835080555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5951b0 msgr2=0x5599fe937a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.302+0000 7f72b3e00640 1 --2- 172.21.15.63:0/3835080555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5951b0 0x5599fe937a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.303+0000 7f72b3e00640 1 -- 172.21.15.63:0/3835080555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599fe8318b0 con 0x5599fe5744d0 2024-05-27T08:07:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.303+0000 7f72b9a00640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b29d0 0x7f729c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.303+0000 7f72b3e00640 1 --2- 172.21.15.63:0/3835080555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5744d0 0x5599fe9384e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f72a00df800 tx=0x7f72a00e8a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35aeb094e8dc046f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.303+0000 7f72b1600640 1 -- 172.21.15.63:0/3178040386 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7f72a00e2470 con 0x7f72b410b670 2024-05-27T08:07:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.304+0000 7f72a6000640 1 -- 172.21.15.63:0/3835080555 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0020020 con 0x5599fe5744d0 2024-05-27T08:07:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.304+0000 7f72a6000640 1 -- 172.21.15.63:0/3835080555 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a00a8ea0 con 0x5599fe5744d0 2024-05-27T08:07:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.304+0000 7f72a6000640 1 -- 172.21.15.63:0/3835080555 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a00a8ce0 con 0x5599fe5744d0 2024-05-27T08:07:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.305+0000 7f72c2a2f740 1 -- 172.21.15.63:0/3835080555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5744d0 msgr2=0x5599fe9384e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.305+0000 7f72b9a00640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b29d0 0x7f729c0b4ee0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x5599fe934bd0 tx=0x7f72ac0095c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.305+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/3835080555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5744d0 0x5599fe9384e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f72a00df800 tx=0x7f72a00e8a20 comp rx=0 tx=0).stop 2024-05-27T08:07:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.305+0000 7f72c2a2f740 1 -- 172.21.15.63:0/3835080555 shutdown_connections 2024-05-27T08:07:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.305+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/3835080555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5744d0 0x5599fe9384e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.306+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/3835080555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe937fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.306+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/3835080555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5951b0 0x5599fe937a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.306+0000 7f72c2a2f740 1 -- 172.21.15.63:0/3835080555 >> 172.21.15.63:0/3835080555 conn(0x5599fe589300 msgr2=0x5599fe934210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.306+0000 7f72c2a2f740 1 -- 172.21.15.63:0/3835080555 shutdown_connections 2024-05-27T08:07:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.306+0000 7f72c2a2f740 1 -- 172.21.15.63:0/3835080555 wait complete. 2024-05-27T08:07:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.307+0000 7f72c2a2f740 1 -- start start 2024-05-27T08:07:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.307+0000 7f72c2a2f740 1 Processor -- start 2024-05-27T08:07:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.308+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe56c2f0 0x5599fe941bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.308+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe942130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.308+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe5951b0 0x5599fe956730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.308+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5599fe57e790 con 0x5599fe5744d0 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.308+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5599fe840bc0 con 0x5599fe56c2f0 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.308+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5599fe944b90 con 0x5599fe5951b0 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.308+0000 7f72c2a2f740 10 client.0 _ll_register_callbacks cb 0x5599fe81e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.309+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe942130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.309+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe942130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33132/0 (socket says 172.21.15.63:33132) 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.309+0000 7f72b3e00640 1 -- 172.21.15.63:0/2182279112 learned_addr learned my addr 172.21.15.63:0/2182279112 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.309+0000 7f72b9a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe5951b0 0x5599fe956730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.309+0000 7f72b9000640 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe56c2f0 0x5599fe941bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.310+0000 7f72b3e00640 1 -- 172.21.15.63:0/2182279112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe5951b0 msgr2=0x5599fe956730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.310+0000 7f72b3e00640 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe5951b0 0x5599fe956730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.310+0000 7f72b3e00640 1 -- 172.21.15.63:0/2182279112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe56c2f0 msgr2=0x5599fe941bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.310+0000 7f72b3e00640 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe56c2f0 0x5599fe941bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.310+0000 7f72b3e00640 1 -- 172.21.15.63:0/2182279112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599fe5ad210 con 0x5599fe5744d0 2024-05-27T08:07:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.310+0000 7f72b3e00640 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe942130 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f72a00dfe00 tx=0x7f72a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1ec7be2d1e5024 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.310+0000 7f729aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.311+0000 7f729aa00640 1 -- 172.21.15.63:0/2182279112 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0020030 con 0x5599fe5744d0 2024-05-27T08:07:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.311+0000 7f72c2a2f740 1 -- 172.21.15.63:0/2182279112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a00a8ea0 con 0x5599fe5744d0 2024-05-27T08:07:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.311+0000 7f729aa00640 1 -- 172.21.15.63:0/2182279112 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7f72a00e2120 con 0x5599fe5744d0 2024-05-27T08:07:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.311+0000 7f729aa00640 1 -- 172.21.15.63:0/2182279112 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f72a00a8b00 con 0x5599fe5744d0 2024-05-27T08:07:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.311+0000 7f72c2a2f740 10 client.21903 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.311+0000 7f72c2a2f740 1 -- 172.21.15.63:0/2182279112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599fe679620 con 0x5599fe5744d0 2024-05-27T08:07:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.312+0000 7f729aa00640 1 -- 172.21.15.63:0/2182279112 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a00a8ea0 con 0x5599fe5744d0 2024-05-27T08:07:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.312+0000 7f729aa00640 1 -- 172.21.15.63:0/2182279112 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f72a00b1d60 con 0x5599fe5744d0 2024-05-27T08:07:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.312+0000 7f72c2a2f740 10 client.21903 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.312+0000 7f72c2a2f740 10 client.21903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.312+0000 7f72c2a2f740 20 client.21903 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.313+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599fe553d80 0x5599fe95c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.313+0000 7f72c2a2f740 4 client.21903 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:07:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.313+0000 7f72c2a2f740 1 -- 172.21.15.63:0/2182279112 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599fe8aaae0 con 0x5599fe553d80 2024-05-27T08:07:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.314+0000 7f72b9000640 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599fe553d80 0x5599fe95c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.315+0000 7f72b9000640 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599fe553d80 0x5599fe95c500 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c6d780d5c4bb16 server_cookie=f6b0e5ecd4ce8cb0 in_seq=0 out_seq=0 2024-05-27T08:07:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.315+0000 7f729aa00640 10 client.21903 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f729aa00640 1 -- 172.21.15.63:0/2182279112 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5599fe8aaae0 con 0x5599fe553d80 2024-05-27T08:07:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f729aa00640 10 client.21903 handle_command_reply: tid=0 2024-05-27T08:07:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f72c2a2f740 1 client.21903 shutdown 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f72c2a2f740 1 -- 172.21.15.63:0/2182279112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 msgr2=0x5599fe942130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe942130 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f72a00dfe00 tx=0x7f72a0005880 comp rx=0 tx=0).stop 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f72c2a2f740 1 -- 172.21.15.63:0/2182279112 shutdown_connections 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599fe553d80 0x5599fe95c500 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe5951b0 0x5599fe956730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe942130 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/2182279112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe56c2f0 0x5599fe941bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.317+0000 7f72c2a2f740 1 -- 172.21.15.63:0/2182279112 >> 172.21.15.63:0/2182279112 conn(0x5599fe589300 msgr2=0x5599fe939780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.318+0000 7f729aa00640 0 client.21903 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.318+0000 7f72c2a2f740 1 -- 172.21.15.63:0/2182279112 shutdown_connections 2024-05-27T08:07:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.318+0000 7f72c2a2f740 1 -- 172.21.15.63:0/2182279112 wait complete. 2024-05-27T08:07:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.318+0000 7f72c2a2f740 20 client.21903 trim_cache size 0 max 16384 2024-05-27T08:07:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.339+0000 7f72c2a2f740 1 Processor -- start 2024-05-27T08:07:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.339+0000 7f72c2a2f740 1 -- start start 2024-05-27T08:07:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.339+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe93d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.339+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe956720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.339+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5951b0 0x5599fe93b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72940127e0 con 0x5599fe5744d0 2024-05-27T08:07:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72940100b0 con 0x5599fe5951b0 2024-05-27T08:07:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f729400df20 con 0x5599fe56c2f0 2024-05-27T08:07:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe956720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe956720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33144/0 (socket says 172.21.15.63:33144) 2024-05-27T08:07:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72b3e00640 1 -- 172.21.15.63:0/591492240 learned_addr learned my addr 172.21.15.63:0/591492240 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72b9a00640 1 --2- 172.21.15.63:0/591492240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5951b0 0x5599fe93b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72b9000640 1 --2- 172.21.15.63:0/591492240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe93d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72b3e00640 1 -- 172.21.15.63:0/591492240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 msgr2=0x5599fe93d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72b3e00640 1 --2- 172.21.15.63:0/591492240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe93d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72b3e00640 1 -- 172.21.15.63:0/591492240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5951b0 msgr2=0x5599fe93b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72b3e00640 1 --2- 172.21.15.63:0/591492240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5951b0 0x5599fe93b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.340+0000 7f72b3e00640 1 -- 172.21.15.63:0/591492240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599fe932610 con 0x5599fe5744d0 2024-05-27T08:07:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.341+0000 7f72b3e00640 1 --2- 172.21.15.63:0/591492240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe956720 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f72a00a6600 tx=0x7f72a000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54edf7615261c466 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.341+0000 7f72a5600640 1 -- 172.21.15.63:0/591492240 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0020070 con 0x5599fe5744d0 2024-05-27T08:07:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.341+0000 7f72a5600640 1 -- 172.21.15.63:0/591492240 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a00e8220 con 0x5599fe5744d0 2024-05-27T08:07:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.341+0000 7f72a5600640 1 -- 172.21.15.63:0/591492240 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a00ad740 con 0x5599fe5744d0 2024-05-27T08:07:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.342+0000 7f72c2a2f740 1 -- 172.21.15.63:0/591492240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 msgr2=0x5599fe956720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.342+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/591492240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe956720 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f72a00a6600 tx=0x7f72a000bf60 comp rx=0 tx=0).stop 2024-05-27T08:07:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.342+0000 7f72c2a2f740 1 -- 172.21.15.63:0/591492240 shutdown_connections 2024-05-27T08:07:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.342+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/591492240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5951b0 0x5599fe93b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.342+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/591492240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe956720 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.342+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/591492240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe93d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.342+0000 7f72c2a2f740 1 -- 172.21.15.63:0/591492240 >> 172.21.15.63:0/591492240 conn(0x5599fe589300 msgr2=0x5599fe943d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.342+0000 7f72c2a2f740 1 -- 172.21.15.63:0/591492240 shutdown_connections 2024-05-27T08:07:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.342+0000 7f72c2a2f740 1 -- 172.21.15.63:0/591492240 wait complete. 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.343+0000 7f72c2a2f740 1 -- start start 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.343+0000 7f72c2a2f740 1 Processor -- start 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe95cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe95d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72c2a2f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5951b0 0x5599fe9bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f729400e570 con 0x5599fe5744d0 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f729400f9e0 con 0x5599fe5951b0 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72c2a2f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f729400ec40 con 0x5599fe56c2f0 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72c2a2f740 10 client.0 _ll_register_callbacks cb 0x7f72a80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe95d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe95d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33156/0 (socket says 172.21.15.63:33156) 2024-05-27T08:07:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72b3e00640 1 -- 172.21.15.63:0/101855124 learned_addr learned my addr 172.21.15.63:0/101855124 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.344+0000 7f72b9a00640 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5951b0 0x5599fe9bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.345+0000 7f72b3e00640 1 -- 172.21.15.63:0/101855124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 msgr2=0x5599fe95cb10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.345+0000 7f72b3e00640 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe95cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.345+0000 7f72b3e00640 1 -- 172.21.15.63:0/101855124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5951b0 msgr2=0x5599fe9bc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.345+0000 7f72b3e00640 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5951b0 0x5599fe9bc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.345+0000 7f72b3e00640 1 -- 172.21.15.63:0/101855124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599fe9bce50 con 0x5599fe5744d0 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.345+0000 7f72b3e00640 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe95d050 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f72a00dfe00 tx=0x7f72a00a9a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff819b23adde1ca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.345+0000 7f729aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.345+0000 7f729aa00640 1 -- 172.21.15.63:0/101855124 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0020070 con 0x5599fe5744d0 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.345+0000 7f729aa00640 1 -- 172.21.15.63:0/101855124 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7f72a00aead0 con 0x5599fe5744d0 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.345+0000 7f72c2a2f740 1 -- 172.21.15.63:0/101855124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a00e8220 con 0x5599fe5744d0 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f729aa00640 1 -- 172.21.15.63:0/101855124 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f72a00a8ea0 con 0x5599fe5744d0 2024-05-27T08:07:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f72c2a2f740 10 client.21909 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f72c2a2f740 1 -- 172.21.15.63:0/101855124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5599fe9bce50 con 0x5599fe5744d0 2024-05-27T08:07:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f729aa00640 1 -- 172.21.15.63:0/101855124 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a00e8220 con 0x5599fe5744d0 2024-05-27T08:07:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f729aa00640 1 -- 172.21.15.63:0/101855124 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f72a00ca0d0 con 0x5599fe5744d0 2024-05-27T08:07:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f72c2a2f740 10 client.21909 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f72c2a2f740 10 client.21909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f72c2a2f740 20 client.21909 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599fe5291c0 0x5599fe9c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f72c2a2f740 4 client.21909 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:07:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.346+0000 7f72c2a2f740 1 -- 172.21.15.63:0/101855124 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599fe8a42c0 con 0x5599fe5291c0 2024-05-27T08:07:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.347+0000 7f72b9000640 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599fe5291c0 0x5599fe9c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.348+0000 7f72b9000640 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599fe5291c0 0x5599fe9c6470 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a6e0c86743adc54 server_cookie=4132030c70054ef0 in_seq=0 out_seq=0 2024-05-27T08:07:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.348+0000 7f729aa00640 10 client.21909 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.385+0000 7f729aa00640 1 -- 172.21.15.63:0/101855124 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5599fe8a42c0 con 0x5599fe5291c0 2024-05-27T08:07:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.385+0000 7f729aa00640 10 client.21909 handle_command_reply: tid=0 2024-05-27T08:07:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.385+0000 7f72c2a2f740 1 client.21909 shutdown 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.385+0000 7f72c2a2f740 1 -- 172.21.15.63:0/101855124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 msgr2=0x5599fe95d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.385+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe95d050 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f72a00dfe00 tx=0x7f72a00a9a90 comp rx=0 tx=0).stop 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.386+0000 7f72c2a2f740 1 -- 172.21.15.63:0/101855124 shutdown_connections 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.386+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5599fe5291c0 0x5599fe9c6470 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.386+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599fe5951b0 0x5599fe9bc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.386+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5599fe5744d0 0x5599fe95d050 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.386+0000 7f72c2a2f740 1 --2- 172.21.15.63:0/101855124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599fe56c2f0 0x5599fe95cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.386+0000 7f72c2a2f740 1 -- 172.21.15.63:0/101855124 >> 172.21.15.63:0/101855124 conn(0x5599fe589300 msgr2=0x5599fe941130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.386+0000 7f729aa00640 0 client.21909 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.386+0000 7f72c2a2f740 1 -- 172.21.15.63:0/101855124 shutdown_connections 2024-05-27T08:07:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.386+0000 7f72c2a2f740 1 -- 172.21.15.63:0/101855124 wait complete. 2024-05-27T08:07:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.386+0000 7f72c2a2f740 20 client.21909 trim_cache size 0 max 16384 2024-05-27T08:07:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:07:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:07:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:07:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.387+0000 7f72ba400640 1 -- 172.21.15.63:0/3178040386 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b29d0 msgr2=0x7f729c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.387+0000 7f72ba400640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b29d0 0x7f729c0b4ee0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x5599fe934bd0 tx=0x7f72ac0095c0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.387+0000 7f72ba400640 1 -- 172.21.15.63:0/3178040386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410b670 msgr2=0x7f72b419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.387+0000 7f72ba400640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410b670 0x7f72b419cd60 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f72a000d950 tx=0x7f72a000de10 comp rx=0 tx=0).stop 2024-05-27T08:07:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.388+0000 7f72ba400640 1 -- 172.21.15.63:0/3178040386 shutdown_connections 2024-05-27T08:07:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.388+0000 7f72ba400640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b29d0 0x7f729c0b4ee0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.388+0000 7f72ba400640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b410e020 0x7f72b41a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.388+0000 7f72ba400640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b410b670 0x7f72b419cd60 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.388+0000 7f72ba400640 1 --2- 172.21.15.63:0/3178040386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b4104c10 0x7f72b419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.388+0000 7f72ba400640 1 -- 172.21.15.63:0/3178040386 >> 172.21.15.63:0/3178040386 conn(0x7f72b40fe210 msgr2=0x7f72b4109c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.388+0000 7f72ba400640 1 -- 172.21.15.63:0/3178040386 shutdown_connections 2024-05-27T08:07:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:21.388+0000 7f72ba400640 1 -- 172.21.15.63:0/3178040386 wait complete. 2024-05-27T08:07:21.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.389+0000 7f6503a00700 1 -- 172.21.15.63:0/2132725801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64fc075ad0 msgr2=0x7f64fc075ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.389+0000 7f6503a00700 1 --2- 172.21.15.63:0/2132725801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64fc075ad0 0x7f64fc075ee0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009990 tx=0x7f64f0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.389+0000 7f6503a00700 1 -- 172.21.15.63:0/2132725801 shutdown_connections 2024-05-27T08:07:21.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.389+0000 7f6503a00700 1 --2- 172.21.15.63:0/2132725801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64fc110c10 0x7f64fc113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.389+0000 7f6503a00700 1 --2- 172.21.15.63:0/2132725801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64fc075ad0 0x7f64fc075ee0 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.389+0000 7f6503a00700 1 --2- 172.21.15.63:0/2132725801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc076420 0x7f64fc1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.389+0000 7f6503a00700 1 -- 172.21.15.63:0/2132725801 >> 172.21.15.63:0/2132725801 conn(0x7f64fc100030 msgr2=0x7f64fc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.389+0000 7f6503a00700 1 -- 172.21.15.63:0/2132725801 shutdown_connections 2024-05-27T08:07:21.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.389+0000 7f6503a00700 1 -- 172.21.15.63:0/2132725801 wait complete. 2024-05-27T08:07:21.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.391+0000 7f6503a00700 1 Processor -- start 2024-05-27T08:07:21.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.392+0000 7f6503a00700 1 -- start start 2024-05-27T08:07:21.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.392+0000 7f6503a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc075ad0 0x7f64fc104330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.392+0000 7f6503a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64fc076420 0x7f64fc104870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.392+0000 7f6503a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64fc110c10 0x7f64fc104db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.393+0000 7f6503a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc116540 con 0x7f64fc110c10 2024-05-27T08:07:21.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.393+0000 7f6503a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1163c0 con 0x7f64fc075ad0 2024-05-27T08:07:21.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.393+0000 7f6503a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1166c0 con 0x7f64fc076420 2024-05-27T08:07:21.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.393+0000 7f6501000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc075ad0 0x7f64fc104330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.393+0000 7f6501a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64fc110c10 0x7f64fc104db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.393+0000 7f6501000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc075ad0 0x7f64fc104330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:47128/0 (socket says 172.21.15.63:47128) 2024-05-27T08:07:21.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.393+0000 7f6501000700 1 -- 172.21.15.63:0/1040742282 learned_addr learned my addr 172.21.15.63:0/1040742282 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:21.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.393+0000 7f64fbe00700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64fc076420 0x7f64fc104870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.394+0000 7f6501a00700 1 -- 172.21.15.63:0/1040742282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64fc076420 msgr2=0x7f64fc104870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.394+0000 7f6501a00700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64fc076420 0x7f64fc104870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.394+0000 7f6501a00700 1 -- 172.21.15.63:0/1040742282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc075ad0 msgr2=0x7f64fc104330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.394+0000 7f6501a00700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc075ad0 0x7f64fc104330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.394+0000 7f6501a00700 1 -- 172.21.15.63:0/1040742282 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc1ad5d0 con 0x7f64fc110c10 2024-05-27T08:07:21.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.394+0000 7f6501000700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc075ad0 0x7f64fc104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:07:21.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.394+0000 7f6501a00700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64fc110c10 0x7f64fc104db0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f64f400efc0 tx=0x7f64f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.395+0000 7f64f9600700 1 -- 172.21.15.63:0/1040742282 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40092e0 con 0x7f64fc110c10 2024-05-27T08:07:21.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.395+0000 7f64f9600700 1 -- 172.21.15.63:0/1040742282 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64f4004bb0 con 0x7f64fc110c10 2024-05-27T08:07:21.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.395+0000 7f64f9600700 1 -- 172.21.15.63:0/1040742282 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4004020 con 0x7f64fc110c10 2024-05-27T08:07:21.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.395+0000 7f6503a00700 1 -- 172.21.15.63:0/1040742282 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc1ad800 con 0x7f64fc110c10 2024-05-27T08:07:21.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.395+0000 7f6503a00700 1 -- 172.21.15.63:0/1040742282 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc1adcf0 con 0x7f64fc110c10 2024-05-27T08:07:21.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.400+0000 7f64f9600700 1 -- 172.21.15.63:0/1040742282 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f64f401b030 con 0x7f64fc110c10 2024-05-27T08:07:21.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.402+0000 7f64f9600700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f64e40b2a70 0x7f64e40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.402+0000 7f64f9600700 1 -- 172.21.15.63:0/1040742282 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7f64f40e23d0 con 0x7f64fc110c10 2024-05-27T08:07:21.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.402+0000 7f6501000700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f64e40b2a70 0x7f64e40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.403+0000 7f6501000700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f64e40b2a70 0x7f64e40b4f80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009750 tx=0x7f64f0005960 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.414+0000 7f651254ab80 1 Processor -- start 2024-05-27T08:07:21.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.414+0000 7f651254ab80 1 -- start start 2024-05-27T08:07:21.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.414+0000 7f651254ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743ce2a70 0x559743d753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.414+0000 7f651254ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743bf6ec0 0x559743d758f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.414+0000 7f651254ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743d75ef0 0x559743d82190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.414+0000 7f651254ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559743ca3ec0 con 0x559743ce2a70 2024-05-27T08:07:21.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.414+0000 7f651254ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559743c93a60 con 0x559743bf6ec0 2024-05-27T08:07:21.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.414+0000 7f651254ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559743c97aa0 con 0x559743d75ef0 2024-05-27T08:07:21.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.415+0000 7f6501000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743ce2a70 0x559743d753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.415+0000 7f6501000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743ce2a70 0x559743d753b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33182/0 (socket says 172.21.15.63:33182) 2024-05-27T08:07:21.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.415+0000 7f6501000700 1 -- 172.21.15.63:0/2577664354 learned_addr learned my addr 172.21.15.63:0/2577664354 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:21.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.415+0000 7f64fbe00700 1 --2- 172.21.15.63:0/2577664354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743bf6ec0 0x559743d758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.415+0000 7f6501a00700 1 --2- 172.21.15.63:0/2577664354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743d75ef0 0x559743d82190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.416+0000 7f6501000700 1 -- 172.21.15.63:0/2577664354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743d75ef0 msgr2=0x559743d82190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.416+0000 7f6501000700 1 --2- 172.21.15.63:0/2577664354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743d75ef0 0x559743d82190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.416+0000 7f6501000700 1 -- 172.21.15.63:0/2577664354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743bf6ec0 msgr2=0x559743d758f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.416+0000 7f6501000700 1 --2- 172.21.15.63:0/2577664354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743bf6ec0 0x559743d758f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.416+0000 7f6501000700 1 -- 172.21.15.63:0/2577664354 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559743c5c4a0 con 0x559743ce2a70 2024-05-27T08:07:21.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.416+0000 7f6501000700 1 --2- 172.21.15.63:0/2577664354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743ce2a70 0x559743d753b0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f64f0005320 tx=0x7f64f001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6904f4f064c792fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.416+0000 7f64ee000700 1 -- 172.21.15.63:0/2577664354 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0003a90 con 0x559743ce2a70 2024-05-27T08:07:21.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.417+0000 7f64ee000700 1 -- 172.21.15.63:0/2577664354 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64f0003c30 con 0x559743ce2a70 2024-05-27T08:07:21.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.417+0000 7f64ee000700 1 -- 172.21.15.63:0/2577664354 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0023400 con 0x559743ce2a70 2024-05-27T08:07:21.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.418+0000 7f651254ab80 1 -- 172.21.15.63:0/2577664354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743ce2a70 msgr2=0x559743d753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.418+0000 7f651254ab80 1 --2- 172.21.15.63:0/2577664354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743ce2a70 0x559743d753b0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f64f0005320 tx=0x7f64f001e000 comp rx=0 tx=0).stop 2024-05-27T08:07:21.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.418+0000 7f651254ab80 1 -- 172.21.15.63:0/2577664354 shutdown_connections 2024-05-27T08:07:21.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.418+0000 7f651254ab80 1 --2- 172.21.15.63:0/2577664354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743d75ef0 0x559743d82190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.418+0000 7f651254ab80 1 --2- 172.21.15.63:0/2577664354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743ce2a70 0x559743d753b0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.418+0000 7f651254ab80 1 --2- 172.21.15.63:0/2577664354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743bf6ec0 0x559743d758f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.418+0000 7f651254ab80 1 -- 172.21.15.63:0/2577664354 >> 172.21.15.63:0/2577664354 conn(0x559743c90410 msgr2=0x559743d71540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.418+0000 7f651254ab80 1 -- 172.21.15.63:0/2577664354 shutdown_connections 2024-05-27T08:07:21.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.418+0000 7f651254ab80 1 -- 172.21.15.63:0/2577664354 wait complete. 2024-05-27T08:07:21.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.419+0000 7f651254ab80 1 -- start start 2024-05-27T08:07:21.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.419+0000 7f651254ab80 1 Processor -- start 2024-05-27T08:07:21.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.419+0000 7f651254ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d7f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f651254ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743ce2a70 0x559743d7fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f651254ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743d75ef0 0x559743d801d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f651254ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559743b9a020 con 0x559743bf6ec0 2024-05-27T08:07:21.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f651254ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559743c347e0 con 0x559743ce2a70 2024-05-27T08:07:21.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f651254ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559743d82b40 con 0x559743d75ef0 2024-05-27T08:07:21.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f651254ab80 10 client.0 _ll_register_callbacks cb 0x559743c3dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f6501000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d7f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f6501000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d7f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33196/0 (socket says 172.21.15.63:33196) 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f6501000700 1 -- 172.21.15.63:0/2138816830 learned_addr learned my addr 172.21.15.63:0/2138816830 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f64fbe00700 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743ce2a70 0x559743d7fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f6501000700 1 -- 172.21.15.63:0/2138816830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743d75ef0 msgr2=0x559743d801d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f6501000700 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743d75ef0 0x559743d801d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f6501000700 1 -- 172.21.15.63:0/2138816830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743ce2a70 msgr2=0x559743d7fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f6501000700 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743ce2a70 0x559743d7fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.420+0000 7f6501000700 1 -- 172.21.15.63:0/2138816830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559743be3950 con 0x559743bf6ec0 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.421+0000 7f6501000700 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d7f750 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f64f0030040 tx=0x7f64f000fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a631d3b989c1d7c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.421+0000 7f64e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.421+0000 7f64e2a00700 1 -- 172.21.15.63:0/2138816830 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f003c060 con 0x559743bf6ec0 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.421+0000 7f64e2a00700 1 -- 172.21.15.63:0/2138816830 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7f64f0037470 con 0x559743bf6ec0 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.421+0000 7f651254ab80 1 -- 172.21.15.63:0/2138816830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f0003c30 con 0x559743bf6ec0 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.422+0000 7f64e2a00700 1 -- 172.21.15.63:0/2138816830 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f64f0003a90 con 0x559743bf6ec0 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.422+0000 7f651254ab80 10 client.21921 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.422+0000 7f651254ab80 1 -- 172.21.15.63:0/2138816830 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559743c640e0 con 0x559743bf6ec0 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.422+0000 7f64e2a00700 1 -- 172.21.15.63:0/2138816830 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64f0003c30 con 0x559743bf6ec0 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.422+0000 7f64e2a00700 1 -- 172.21.15.63:0/2138816830 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f64f0022070 con 0x559743bf6ec0 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.422+0000 7f651254ab80 10 client.21921 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.422+0000 7f651254ab80 10 client.21921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:21.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.422+0000 7f651254ab80 20 client.21921 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:21.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.423+0000 7f651254ab80 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559743d8f9a0 0x559743d91dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.423+0000 7f651254ab80 4 client.21921 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:07:21.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.423+0000 7f651254ab80 1 -- 172.21.15.63:0/2138816830 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559743ca5e10 con 0x559743d8f9a0 2024-05-27T08:07:21.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.423+0000 7f64fbe00700 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559743d8f9a0 0x559743d91dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.424+0000 7f64fbe00700 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559743d8f9a0 0x559743d91dc0 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ddc7b3bd4ee41d server_cookie=fdb70eab836fd675 in_seq=0 out_seq=0 2024-05-27T08:07:21.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.424+0000 7f64e2a00700 10 client.21921 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:21.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.426+0000 7f64e2a00700 1 -- 172.21.15.63:0/2138816830 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559743ca5e10 con 0x559743d8f9a0 2024-05-27T08:07:21.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.426+0000 7f64e2a00700 10 client.21921 handle_command_reply: tid=0 2024-05-27T08:07:21.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.426+0000 7f651254ab80 1 client.21921 shutdown 2024-05-27T08:07:21.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.426+0000 7f651254ab80 1 -- 172.21.15.63:0/2138816830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 msgr2=0x559743d7f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.426+0000 7f651254ab80 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d7f750 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f64f0030040 tx=0x7f64f000fb70 comp rx=0 tx=0).stop 2024-05-27T08:07:21.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.427+0000 7f651254ab80 1 -- 172.21.15.63:0/2138816830 shutdown_connections 2024-05-27T08:07:21.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.427+0000 7f651254ab80 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559743d8f9a0 0x559743d91dc0 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.427+0000 7f651254ab80 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743d75ef0 0x559743d801d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.427+0000 7f651254ab80 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d7f750 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.427+0000 7f651254ab80 1 --2- 172.21.15.63:0/2138816830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743ce2a70 0x559743d7fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.427+0000 7f651254ab80 1 -- 172.21.15.63:0/2138816830 >> 172.21.15.63:0/2138816830 conn(0x559743c90410 msgr2=0x559743d75430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.427+0000 7f64e2a00700 0 client.21921 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:21.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.427+0000 7f651254ab80 1 -- 172.21.15.63:0/2138816830 shutdown_connections 2024-05-27T08:07:21.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.427+0000 7f651254ab80 1 -- 172.21.15.63:0/2138816830 wait complete. 2024-05-27T08:07:21.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.427+0000 7f651254ab80 20 client.21921 trim_cache size 0 max 16384 2024-05-27T08:07:21.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.450+0000 7f651254ab80 1 Processor -- start 2024-05-27T08:07:21.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.450+0000 7f651254ab80 1 -- start start 2024-05-27T08:07:21.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.450+0000 7f651254ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d79670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.451+0000 7f651254ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 0x559743d94de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.451+0000 7f651254ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743d75ef0 0x559743d95320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.451+0000 7f651254ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1b9450 con 0x559743bf6ec0 2024-05-27T08:07:21.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.451+0000 7f651254ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1b6ce0 con 0x559743d75ef0 2024-05-27T08:07:21.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.451+0000 7f651254ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1b4b50 con 0x559743ce2a70 2024-05-27T08:07:21.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.451+0000 7f6501000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d79670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.451+0000 7f6501000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d79670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33212/0 (socket says 172.21.15.63:33212) 2024-05-27T08:07:21.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.451+0000 7f6501000700 1 -- 172.21.15.63:0/842364186 learned_addr learned my addr 172.21.15.63:0/842364186 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:21.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.451+0000 7f64fbe00700 1 --2- 172.21.15.63:0/842364186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 0x559743d94de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.451+0000 7f6501a00700 1 --2- 172.21.15.63:0/842364186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743d75ef0 0x559743d95320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.452+0000 7f6501000700 1 -- 172.21.15.63:0/842364186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 msgr2=0x559743d94de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.452+0000 7f6501000700 1 --2- 172.21.15.63:0/842364186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 0x559743d94de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.452+0000 7f6501000700 1 -- 172.21.15.63:0/842364186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743d75ef0 msgr2=0x559743d95320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.452+0000 7f6501000700 1 --2- 172.21.15.63:0/842364186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743d75ef0 0x559743d95320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.452+0000 7f6501000700 1 -- 172.21.15.63:0/842364186 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559743d80540 con 0x559743bf6ec0 2024-05-27T08:07:21.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.452+0000 7f64fbe00700 1 --2- 172.21.15.63:0/842364186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 0x559743d94de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:07:21.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.452+0000 7f6501000700 1 --2- 172.21.15.63:0/842364186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d79670 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f64f0004010 tx=0x7f64f00098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d0efeeed2e5ad2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.452+0000 7f64ed600700 1 -- 172.21.15.63:0/842364186 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f003c040 con 0x559743bf6ec0 2024-05-27T08:07:21.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.452+0000 7f64ed600700 1 -- 172.21.15.63:0/842364186 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64f000fe50 con 0x559743bf6ec0 2024-05-27T08:07:21.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.452+0000 7f64ed600700 1 -- 172.21.15.63:0/842364186 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f00236e0 con 0x559743bf6ec0 2024-05-27T08:07:21.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.453+0000 7f651254ab80 1 -- 172.21.15.63:0/842364186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 msgr2=0x559743d79670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.453+0000 7f651254ab80 1 --2- 172.21.15.63:0/842364186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d79670 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f64f0004010 tx=0x7f64f00098e0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.453+0000 7f651254ab80 1 -- 172.21.15.63:0/842364186 shutdown_connections 2024-05-27T08:07:21.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.453+0000 7f651254ab80 1 --2- 172.21.15.63:0/842364186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743d75ef0 0x559743d95320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.453+0000 7f651254ab80 1 --2- 172.21.15.63:0/842364186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d79670 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.453+0000 7f651254ab80 1 --2- 172.21.15.63:0/842364186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 0x559743d94de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.453+0000 7f651254ab80 1 -- 172.21.15.63:0/842364186 >> 172.21.15.63:0/842364186 conn(0x559743c90410 msgr2=0x559743d78f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.454+0000 7f651254ab80 1 -- 172.21.15.63:0/842364186 shutdown_connections 2024-05-27T08:07:21.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.454+0000 7f651254ab80 1 -- 172.21.15.63:0/842364186 wait complete. 2024-05-27T08:07:21.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.454+0000 7f651254ab80 1 -- start start 2024-05-27T08:07:21.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.455+0000 7f651254ab80 1 Processor -- start 2024-05-27T08:07:21.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.455+0000 7f651254ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d92d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.455+0000 7f651254ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 0x559743d93280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.455+0000 7f651254ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743d75ef0 0x559743d937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.455+0000 7f651254ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1b51a0 con 0x559743bf6ec0 2024-05-27T08:07:21.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.455+0000 7f651254ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1b6610 con 0x559743d75ef0 2024-05-27T08:07:21.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.455+0000 7f651254ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1b5870 con 0x559743ce2a70 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.455+0000 7f651254ab80 10 client.0 _ll_register_callbacks cb 0x559743c3dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.456+0000 7f6501000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d92d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.456+0000 7f6501000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d92d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33218/0 (socket says 172.21.15.63:33218) 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.456+0000 7f6501000700 1 -- 172.21.15.63:0/4171483336 learned_addr learned my addr 172.21.15.63:0/4171483336 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.456+0000 7f6501a00700 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743d75ef0 0x559743d937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.456+0000 7f64fbe00700 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 0x559743d93280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.456+0000 7f6501000700 1 -- 172.21.15.63:0/4171483336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 msgr2=0x559743d93280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.456+0000 7f6501000700 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 0x559743d93280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.456+0000 7f6501000700 1 -- 172.21.15.63:0/4171483336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743d75ef0 msgr2=0x559743d937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.456+0000 7f6501000700 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743d75ef0 0x559743d937c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.456+0000 7f6501000700 1 -- 172.21.15.63:0/4171483336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559743df9380 con 0x559743bf6ec0 2024-05-27T08:07:21.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.457+0000 7f6501000700 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d92d40 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f64f000b650 tx=0x7f64f002eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f35f61162aee81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:21.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.457+0000 7f64e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:21.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.457+0000 7f64e2a00700 1 -- 172.21.15.63:0/4171483336 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f003c070 con 0x559743bf6ec0 2024-05-27T08:07:21.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.457+0000 7f651254ab80 1 -- 172.21.15.63:0/4171483336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f000fe50 con 0x559743bf6ec0 2024-05-27T08:07:21.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.457+0000 7f64e2a00700 1 -- 172.21.15.63:0/4171483336 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7f64f00237f0 con 0x559743bf6ec0 2024-05-27T08:07:21.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f64e2a00700 1 -- 172.21.15.63:0/4171483336 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f64f0056050 con 0x559743bf6ec0 2024-05-27T08:07:21.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f651254ab80 10 client.21927 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:21.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f651254ab80 1 -- 172.21.15.63:0/4171483336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559743df9380 con 0x559743bf6ec0 2024-05-27T08:07:21.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f64e2a00700 1 -- 172.21.15.63:0/4171483336 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64f000fe50 con 0x559743bf6ec0 2024-05-27T08:07:21.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f64e2a00700 1 -- 172.21.15.63:0/4171483336 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f64f0022050 con 0x559743bf6ec0 2024-05-27T08:07:21.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f651254ab80 10 client.21927 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:21.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f651254ab80 10 client.21927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:21.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f651254ab80 20 client.21927 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:21.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f651254ab80 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559743dfa900 0x559743dfcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:21.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f651254ab80 4 client.21927 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:07:21.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.458+0000 7f651254ab80 1 -- 172.21.15.63:0/4171483336 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559743c9f5f0 con 0x559743dfa900 2024-05-27T08:07:21.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.459+0000 7f64fbe00700 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559743dfa900 0x559743dfcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:21.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.460+0000 7f64fbe00700 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559743dfa900 0x559743dfcd20 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d7474e0db8e73f server_cookie=2e46bd61135c13ce in_seq=0 out_seq=0 2024-05-27T08:07:21.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.460+0000 7f64e2a00700 10 client.21927 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:21.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.463+0000 7f64e2a00700 1 -- 172.21.15.63:0/4171483336 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559743c9f5f0 con 0x559743dfa900 2024-05-27T08:07:21.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.463+0000 7f64e2a00700 10 client.21927 handle_command_reply: tid=0 2024-05-27T08:07:21.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.463+0000 7f651254ab80 1 client.21927 shutdown 2024-05-27T08:07:21.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.464+0000 7f651254ab80 1 -- 172.21.15.63:0/4171483336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 msgr2=0x559743d92d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.464+0000 7f651254ab80 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d92d40 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f64f000b650 tx=0x7f64f002eb90 comp rx=0 tx=0).stop 2024-05-27T08:07:21.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.464+0000 7f651254ab80 1 -- 172.21.15.63:0/4171483336 shutdown_connections 2024-05-27T08:07:21.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.464+0000 7f651254ab80 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559743dfa900 0x559743dfcd20 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.464+0000 7f651254ab80 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559743d75ef0 0x559743d937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.464+0000 7f651254ab80 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559743bf6ec0 0x559743d92d40 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.464+0000 7f651254ab80 1 --2- 172.21.15.63:0/4171483336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559743ce2a70 0x559743d93280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.464+0000 7f651254ab80 1 -- 172.21.15.63:0/4171483336 >> 172.21.15.63:0/4171483336 conn(0x559743c90410 msgr2=0x559743d80d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.465+0000 7f64e2a00700 0 client.21927 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:21.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.465+0000 7f651254ab80 1 -- 172.21.15.63:0/4171483336 shutdown_connections 2024-05-27T08:07:21.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.465+0000 7f651254ab80 1 -- 172.21.15.63:0/4171483336 wait complete. 2024-05-27T08:07:21.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.465+0000 7f651254ab80 20 client.21927 trim_cache size 0 max 16384 2024-05-27T08:07:21.468 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:07:21.468 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:07:21.468 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "76c1fd8d-f78a-4581-811c-da6203a944c3", 2024-05-27T08:07:21.468 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:07:21.468 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:07:21.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 -- 172.21.15.63:0/1040742282 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f64e40b2a70 msgr2=0x7f64e40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f64e40b2a70 0x7f64e40b4f80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009750 tx=0x7f64f0005960 comp rx=0 tx=0).stop 2024-05-27T08:07:21.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 -- 172.21.15.63:0/1040742282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64fc110c10 msgr2=0x7f64fc104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:21.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64fc110c10 0x7f64fc104db0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f64f400efc0 tx=0x7f64f400c3a0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 -- 172.21.15.63:0/1040742282 shutdown_connections 2024-05-27T08:07:21.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f64e40b2a70 0x7f64e40b4f80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64fc110c10 0x7f64fc104db0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64fc075ad0 0x7f64fc104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 --2- 172.21.15.63:0/1040742282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64fc076420 0x7f64fc104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:21.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 -- 172.21.15.63:0/1040742282 >> 172.21.15.63:0/1040742282 conn(0x7f64fc100030 msgr2=0x7f64fc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:21.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.466+0000 7f6503a00700 1 -- 172.21.15.63:0/1040742282 shutdown_connections 2024-05-27T08:07:21.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:21.467+0000 7f6503a00700 1 -- 172.21.15.63:0/1040742282 wait complete. 2024-05-27T08:07:21.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:21 smithi063 ceph-mon[31613]: pgmap v1071: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:07:21.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:21 smithi086 ceph-mon[34687]: pgmap v1071: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:07:21.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:21 smithi149 ceph-mon[35408]: pgmap v1071: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:07:22.487 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:07:22.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:07:22.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:07:22.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:07:22.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:22 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:07:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:07:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:22 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:07:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:07:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:07:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:07:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:07:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:07:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:07:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:07:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:22.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:22 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:07:23.047 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:07:23.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:23 smithi086 ceph-mon[34687]: pgmap v1072: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:07:23.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:23 smithi149 ceph-mon[35408]: pgmap v1072: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:07:23.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:23 smithi063 ceph-mon[31613]: pgmap v1072: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:07:25.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.791+0000 7f62b3200700 1 -- 172.21.15.63:0/4187806548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62b410edf0 msgr2=0x7f62b410f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:25.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.791+0000 7f62b3200700 1 --2- 172.21.15.63:0/4187806548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62b410edf0 0x7f62b410f200 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f62a40099c0 tx=0x7f62a4009cd0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.791+0000 7f62b3200700 1 -- 172.21.15.63:0/4187806548 shutdown_connections 2024-05-27T08:07:25.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.791+0000 7f62b3200700 1 --2- 172.21.15.63:0/4187806548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4114070 0x7f62b411b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.791+0000 7f62b3200700 1 --2- 172.21.15.63:0/4187806548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62b410edf0 0x7f62b410f200 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.791+0000 7f62b3200700 1 --2- 172.21.15.63:0/4187806548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62b4113720 0x7f62b4113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.791+0000 7f62b3200700 1 -- 172.21.15.63:0/4187806548 >> 172.21.15.63:0/4187806548 conn(0x7f62b410a980 msgr2=0x7f62b410cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:25.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.791+0000 7f62b3200700 1 -- 172.21.15.63:0/4187806548 shutdown_connections 2024-05-27T08:07:25.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.791+0000 7f62b3200700 1 -- 172.21.15.63:0/4187806548 wait complete. 2024-05-27T08:07:25.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.792+0000 7f62b3200700 1 Processor -- start 2024-05-27T08:07:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.793+0000 7f62b3200700 1 -- start start 2024-05-27T08:07:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.793+0000 7f62b3200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62b410edf0 0x7f62b41a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.793+0000 7f62b3200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62b4113720 0x7f62b41a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.793+0000 7f62b3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4114070 0x7f62b41a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.793+0000 7f62b3200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62b411eb90 con 0x7f62b410edf0 2024-05-27T08:07:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.793+0000 7f62b3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62b411ea10 con 0x7f62b4114070 2024-05-27T08:07:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.793+0000 7f62b3200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62b411ed10 con 0x7f62b4113720 2024-05-27T08:07:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.794+0000 7f62b1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62b410edf0 0x7f62b41a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.794+0000 7f62b1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62b410edf0 0x7f62b41a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33242/0 (socket says 172.21.15.63:33242) 2024-05-27T08:07:25.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.794+0000 7f62b1e00700 1 -- 172.21.15.63:0/2890496 learned_addr learned my addr 172.21.15.63:0/2890496 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:25.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.794+0000 7f62b1400700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62b4113720 0x7f62b41a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.794+0000 7f62b2800700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4114070 0x7f62b41a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.794+0000 7f62b1e00700 1 -- 172.21.15.63:0/2890496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62b4113720 msgr2=0x7f62b41a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.794+0000 7f62b1e00700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62b4113720 0x7f62b41a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.794+0000 7f62b1e00700 1 -- 172.21.15.63:0/2890496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4114070 msgr2=0x7f62b41a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.794+0000 7f62b1e00700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4114070 0x7f62b41a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.794+0000 7f62b1e00700 1 -- 172.21.15.63:0/2890496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b41aa3e0 con 0x7f62b410edf0 2024-05-27T08:07:25.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.795+0000 7f62b1e00700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62b410edf0 0x7f62b41a55c0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f62a4000c00 tx=0x7f62a400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:25.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.795+0000 7f62aaa00700 1 -- 172.21.15.63:0/2890496 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4027070 con 0x7f62b410edf0 2024-05-27T08:07:25.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.795+0000 7f62aaa00700 1 -- 172.21.15.63:0/2890496 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62a400fe50 con 0x7f62b410edf0 2024-05-27T08:07:25.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.795+0000 7f62b3200700 1 -- 172.21.15.63:0/2890496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b41aa670 con 0x7f62b410edf0 2024-05-27T08:07:25.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.795+0000 7f62b3200700 1 -- 172.21.15.63:0/2890496 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b41aab50 con 0x7f62b410edf0 2024-05-27T08:07:25.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.795+0000 7f62aaa00700 1 -- 172.21.15.63:0/2890496 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40173f0 con 0x7f62b410edf0 2024-05-27T08:07:25.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.800+0000 7f62aaa00700 1 -- 172.21.15.63:0/2890496 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f62a402a070 con 0x7f62b410edf0 2024-05-27T08:07:25.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.801+0000 7f62c7795b80 1 Processor -- start 2024-05-27T08:07:25.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.802+0000 7f62c7795b80 1 -- start start 2024-05-27T08:07:25.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.802+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035da750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.802+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 0x5604035dac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.802+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 0x5604035e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.802+0000 7f62c7795b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560403507e40 con 0x5604035469f0 2024-05-27T08:07:25.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.802+0000 7f62c7795b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5604034f79e0 con 0x56040345ae40 2024-05-27T08:07:25.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.802+0000 7f62c7795b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5604034fba20 con 0x5604035e42c0 2024-05-27T08:07:25.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.803+0000 7f62aaa00700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62980b29f0 0x7f62980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.803+0000 7f62b2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035da750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.803+0000 7f62b2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035da750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33246/0 (socket says 172.21.15.63:33246) 2024-05-27T08:07:25.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.803+0000 7f62b2800700 1 -- 172.21.15.63:0/1824779526 learned_addr learned my addr 172.21.15.63:0/1824779526 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:25.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.803+0000 7f62b1400700 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 0x5604035e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.803+0000 7f62aaa00700 1 -- 172.21.15.63:0/2890496 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7f62a40e3770 con 0x7f62b410edf0 2024-05-27T08:07:25.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.803+0000 7f62b1e00700 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 0x5604035dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.803+0000 7f62b2800700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62980b29f0 0x7f62980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.804+0000 7f62b2800700 1 -- 172.21.15.63:0/1824779526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 msgr2=0x5604035e66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.804+0000 7f62b2800700 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 0x5604035e66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.804+0000 7f62b2800700 1 -- 172.21.15.63:0/1824779526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 msgr2=0x5604035dac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.804+0000 7f62b2800700 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 0x5604035dac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.804+0000 7f62b2800700 1 -- 172.21.15.63:0/1824779526 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604034c0420 con 0x5604035469f0 2024-05-27T08:07:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.804+0000 7f62b1400700 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 0x5604035e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:07:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.804+0000 7f62b1e00700 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 0x5604035dac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:07:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.804+0000 7f62b2800700 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035da750 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f62ac012a60 tx=0x7f62ac012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123bfa78f682a655 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:25.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.805+0000 7f629f400700 1 -- 172.21.15.63:0/1824779526 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac010050 con 0x5604035469f0 2024-05-27T08:07:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.805+0000 7f629f400700 1 -- 172.21.15.63:0/1824779526 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62ac011040 con 0x5604035469f0 2024-05-27T08:07:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.805+0000 7f629f400700 1 -- 172.21.15.63:0/1824779526 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac018c90 con 0x5604035469f0 2024-05-27T08:07:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.806+0000 7f62c7795b80 1 -- 172.21.15.63:0/1824779526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 msgr2=0x5604035da750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:25.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.806+0000 7f62b2800700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62980b29f0 0x7f62980b4f00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f62ac01f000 tx=0x7f62ac01d000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.806+0000 7f62c7795b80 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035da750 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f62ac012a60 tx=0x7f62ac012d70 comp rx=0 tx=0).stop 2024-05-27T08:07:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.806+0000 7f62c7795b80 1 -- 172.21.15.63:0/1824779526 shutdown_connections 2024-05-27T08:07:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.806+0000 7f62c7795b80 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 0x5604035e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.806+0000 7f62c7795b80 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035da750 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.806+0000 7f62c7795b80 1 --2- 172.21.15.63:0/1824779526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 0x5604035dac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.806+0000 7f62c7795b80 1 -- 172.21.15.63:0/1824779526 >> 172.21.15.63:0/1824779526 conn(0x5604034f4390 msgr2=0x5604035d5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.807+0000 7f62c7795b80 1 -- 172.21.15.63:0/1824779526 shutdown_connections 2024-05-27T08:07:25.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.807+0000 7f62c7795b80 1 -- 172.21.15.63:0/1824779526 wait complete. 2024-05-27T08:07:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.807+0000 7f62c7795b80 1 -- start start 2024-05-27T08:07:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.808+0000 7f62c7795b80 1 Processor -- start 2024-05-27T08:07:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.808+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 0x5604035e1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.808+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035469f0 0x5604035e1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.808+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035e42c0 0x5604035e1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.808+0000 7f62c7795b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5604033fdfa0 con 0x5604035e42c0 2024-05-27T08:07:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.809+0000 7f62c7795b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560403498760 con 0x56040345ae40 2024-05-27T08:07:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.809+0000 7f62c7795b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5604035e6e00 con 0x5604035469f0 2024-05-27T08:07:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.809+0000 7f62c7795b80 10 client.0 _ll_register_callbacks cb 0x5604034a1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.809+0000 7f62b2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035e42c0 0x5604035e1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.809+0000 7f62b1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 0x5604035e1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.809+0000 7f62b2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035e42c0 0x5604035e1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33250/0 (socket says 172.21.15.63:33250) 2024-05-27T08:07:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.809+0000 7f62b2800700 1 -- 172.21.15.63:0/2409641560 learned_addr learned my addr 172.21.15.63:0/2409641560 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.809+0000 7f62b1400700 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035469f0 0x5604035e1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.810+0000 7f62b2800700 1 -- 172.21.15.63:0/2409641560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035469f0 msgr2=0x5604035e1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.810+0000 7f62b2800700 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035469f0 0x5604035e1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.810+0000 7f62b2800700 1 -- 172.21.15.63:0/2409641560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 msgr2=0x5604035e1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.810+0000 7f62b2800700 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 0x5604035e1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.810+0000 7f62b2800700 1 -- 172.21.15.63:0/2409641560 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604034478d0 con 0x5604035e42c0 2024-05-27T08:07:25.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.810+0000 7f62b2800700 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035e42c0 0x5604035e1d80 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f62ac012a30 tx=0x7f62ac004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1c71835a4fd96b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:25.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.810+0000 7f6293e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:25.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.811+0000 7f6293e00700 1 -- 172.21.15.63:0/2409641560 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac030020 con 0x5604035e42c0 2024-05-27T08:07:25.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.811+0000 7f62c7795b80 1 -- 172.21.15.63:0/2409641560 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ac011040 con 0x5604035e42c0 2024-05-27T08:07:25.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.811+0000 7f6293e00700 1 -- 172.21.15.63:0/2409641560 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7f62ac018620 con 0x5604035e42c0 2024-05-27T08:07:25.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.811+0000 7f6293e00700 1 -- 172.21.15.63:0/2409641560 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f62ac03b050 con 0x5604035e42c0 2024-05-27T08:07:25.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.811+0000 7f62c7795b80 10 client.21939 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:25.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.811+0000 7f62c7795b80 1 -- 172.21.15.63:0/2409641560 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5604034c8060 con 0x5604035e42c0 2024-05-27T08:07:25.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.811+0000 7f6293e00700 1 -- 172.21.15.63:0/2409641560 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62ac011040 con 0x5604035e42c0 2024-05-27T08:07:25.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.812+0000 7f6293e00700 1 -- 172.21.15.63:0/2409641560 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f62ac02ca30 con 0x5604035e42c0 2024-05-27T08:07:25.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.812+0000 7f62c7795b80 10 client.21939 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:25.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.812+0000 7f62c7795b80 10 client.21939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.812+0000 7f62c7795b80 20 client.21939 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.812+0000 7f62c7795b80 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5604035f3cb0 0x5604035f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.812+0000 7f62c7795b80 4 client.21939 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:07:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.812+0000 7f62c7795b80 1 -- 172.21.15.63:0/2409641560 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560403509d90 con 0x5604035f3cb0 2024-05-27T08:07:25.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.813+0000 7f62b1e00700 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5604035f3cb0 0x5604035f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.814+0000 7f62b1e00700 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5604035f3cb0 0x5604035f60d0 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c66245e5a44c53 server_cookie=4539d5b99b1701ba in_seq=0 out_seq=0 2024-05-27T08:07:25.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.814+0000 7f6293e00700 10 client.21939 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:25.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.816+0000 7f6293e00700 1 -- 172.21.15.63:0/2409641560 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560403509d90 con 0x5604035f3cb0 2024-05-27T08:07:25.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.816+0000 7f6293e00700 10 client.21939 handle_command_reply: tid=0 2024-05-27T08:07:25.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.816+0000 7f62c7795b80 1 client.21939 shutdown 2024-05-27T08:07:25.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 1 -- 172.21.15.63:0/2409641560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035e42c0 msgr2=0x5604035e1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035e42c0 0x5604035e1d80 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f62ac012a30 tx=0x7f62ac004cc0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 1 -- 172.21.15.63:0/2409641560 shutdown_connections 2024-05-27T08:07:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5604035f3cb0 0x5604035f60d0 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035e42c0 0x5604035e1d80 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56040345ae40 0x5604035e1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 1 --2- 172.21.15.63:0/2409641560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035469f0 0x5604035e1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 1 -- 172.21.15.63:0/2409641560 >> 172.21.15.63:0/2409641560 conn(0x5604034f4390 msgr2=0x5604035d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f6293e00700 0 client.21939 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 1 -- 172.21.15.63:0/2409641560 shutdown_connections 2024-05-27T08:07:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 1 -- 172.21.15.63:0/2409641560 wait complete. 2024-05-27T08:07:25.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.817+0000 7f62c7795b80 20 client.21939 trim_cache size 0 max 16384 2024-05-27T08:07:25.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.846+0000 7f62c7795b80 1 Processor -- start 2024-05-27T08:07:25.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.846+0000 7f62c7795b80 1 -- start start 2024-05-27T08:07:25.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.846+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040345ae40 0x5604035d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.846+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604035469f0 0x5604035fb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.846+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 0x5604035fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.846+0000 7f62c7795b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6294011610 con 0x56040345ae40 2024-05-27T08:07:25.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.846+0000 7f62c7795b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f629400eea0 con 0x5604035469f0 2024-05-27T08:07:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.846+0000 7f62c7795b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f629400cd10 con 0x5604035e42c0 2024-05-27T08:07:25.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.847+0000 7f62b1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040345ae40 0x5604035d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.847+0000 7f62b1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040345ae40 0x5604035d5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33254/0 (socket says 172.21.15.63:33254) 2024-05-27T08:07:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.847+0000 7f62b1e00700 1 -- 172.21.15.63:0/2258136016 learned_addr learned my addr 172.21.15.63:0/2258136016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.847+0000 7f62b2800700 1 --2- 172.21.15.63:0/2258136016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 0x5604035fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.847+0000 7f62b1400700 1 --2- 172.21.15.63:0/2258136016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604035469f0 0x5604035fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.847+0000 7f62b1e00700 1 -- 172.21.15.63:0/2258136016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 msgr2=0x5604035fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.847+0000 7f62b1e00700 1 --2- 172.21.15.63:0/2258136016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 0x5604035fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.847+0000 7f62b1e00700 1 -- 172.21.15.63:0/2258136016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604035469f0 msgr2=0x5604035fb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.847+0000 7f62b1e00700 1 --2- 172.21.15.63:0/2258136016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604035469f0 0x5604035fb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.847+0000 7f62b1e00700 1 -- 172.21.15.63:0/2258136016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604035e20f0 con 0x56040345ae40 2024-05-27T08:07:25.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.848+0000 7f62b1e00700 1 --2- 172.21.15.63:0/2258136016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040345ae40 0x5604035d5740 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f62a40a77b0 tx=0x7f62a40e5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1345f6d06112d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.848+0000 7f629ea00700 1 -- 172.21.15.63:0/2258136016 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40afae0 con 0x56040345ae40 2024-05-27T08:07:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.848+0000 7f629ea00700 1 -- 172.21.15.63:0/2258136016 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62a40a5e50 con 0x56040345ae40 2024-05-27T08:07:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.848+0000 7f629ea00700 1 -- 172.21.15.63:0/2258136016 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40ab990 con 0x56040345ae40 2024-05-27T08:07:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.849+0000 7f62c7795b80 1 -- 172.21.15.63:0/2258136016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040345ae40 msgr2=0x5604035d5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:25.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.849+0000 7f62c7795b80 1 --2- 172.21.15.63:0/2258136016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040345ae40 0x5604035d5740 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f62a40a77b0 tx=0x7f62a40e5040 comp rx=0 tx=0).stop 2024-05-27T08:07:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.849+0000 7f62c7795b80 1 -- 172.21.15.63:0/2258136016 shutdown_connections 2024-05-27T08:07:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.849+0000 7f62c7795b80 1 --2- 172.21.15.63:0/2258136016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5604035e42c0 0x5604035fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.849+0000 7f62c7795b80 1 --2- 172.21.15.63:0/2258136016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56040345ae40 0x5604035d5740 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.849+0000 7f62c7795b80 1 --2- 172.21.15.63:0/2258136016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604035469f0 0x5604035fb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.849+0000 7f62c7795b80 1 -- 172.21.15.63:0/2258136016 >> 172.21.15.63:0/2258136016 conn(0x5604034f4390 msgr2=0x5604035da750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.850+0000 7f62c7795b80 1 -- 172.21.15.63:0/2258136016 shutdown_connections 2024-05-27T08:07:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.850+0000 7f62c7795b80 1 -- 172.21.15.63:0/2258136016 wait complete. 2024-05-27T08:07:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.850+0000 7f62c7795b80 1 -- start start 2024-05-27T08:07:25.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.850+0000 7f62c7795b80 1 Processor -- start 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.851+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56040345ae40 0x5604035ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.851+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035e7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.851+0000 7f62c7795b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604035e42c0 0x5604035e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.851+0000 7f62c7795b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f629400d360 con 0x5604035469f0 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.851+0000 7f62c7795b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f629400e7d0 con 0x5604035e42c0 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.851+0000 7f62c7795b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f629400da30 con 0x56040345ae40 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.851+0000 7f62c7795b80 10 client.0 _ll_register_callbacks cb 0x5604034a1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.851+0000 7f62b1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.852+0000 7f62b1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035e7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33258/0 (socket says 172.21.15.63:33258) 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.852+0000 7f62b1400700 1 -- 172.21.15.63:0/745048500 learned_addr learned my addr 172.21.15.63:0/745048500 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:25.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.852+0000 7f62b1e00700 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56040345ae40 0x5604035ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.852+0000 7f62b1400700 1 -- 172.21.15.63:0/745048500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56040345ae40 msgr2=0x5604035ec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.852+0000 7f62b1400700 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56040345ae40 0x5604035ec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.852+0000 7f62b1400700 1 -- 172.21.15.63:0/745048500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604035e42c0 msgr2=0x5604035e7e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:07:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.852+0000 7f62b1400700 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604035e42c0 0x5604035e7e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.852+0000 7f62b1400700 1 -- 172.21.15.63:0/745048500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604035e8390 con 0x5604035469f0 2024-05-27T08:07:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.852+0000 7f62b1400700 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035e7910 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f62a0013850 tx=0x7f62a0013c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7067d925a255ed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.853+0000 7f6293e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.853+0000 7f6293e00700 1 -- 172.21.15.63:0/745048500 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0021060 con 0x5604035469f0 2024-05-27T08:07:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.853+0000 7f6293e00700 1 -- 172.21.15.63:0/745048500 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7f62a0011960 con 0x5604035469f0 2024-05-27T08:07:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.853+0000 7f62c7795b80 1 -- 172.21.15.63:0/745048500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a40a5e50 con 0x5604035469f0 2024-05-27T08:07:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.853+0000 7f6293e00700 1 -- 172.21.15.63:0/745048500 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f62a0028050 con 0x5604035469f0 2024-05-27T08:07:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.853+0000 7f62c7795b80 10 client.21945 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.853+0000 7f62c7795b80 1 -- 172.21.15.63:0/745048500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560403666ba0 con 0x5604035469f0 2024-05-27T08:07:25.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.854+0000 7f6293e00700 1 -- 172.21.15.63:0/745048500 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62a40a5e50 con 0x5604035469f0 2024-05-27T08:07:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.854+0000 7f6293e00700 1 -- 172.21.15.63:0/745048500 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f62a000f040 con 0x5604035469f0 2024-05-27T08:07:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.854+0000 7f62c7795b80 10 client.21945 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.854+0000 7f62c7795b80 10 client.21945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.854+0000 7f62c7795b80 20 client.21945 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.854+0000 7f62c7795b80 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560403667310 0x560403669730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.854+0000 7f62c7795b80 4 client.21945 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:07:25.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.854+0000 7f62c7795b80 1 -- 172.21.15.63:0/745048500 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560403503570 con 0x560403667310 2024-05-27T08:07:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.855+0000 7f62b1e00700 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560403667310 0x560403669730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.856+0000 7f62b1e00700 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560403667310 0x560403669730 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c40b6b06831d136 server_cookie=d6f75c6037c98263 in_seq=0 out_seq=0 2024-05-27T08:07:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.856+0000 7f6293e00700 10 client.21945 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.856+0000 7f6293e00700 1 -- 172.21.15.63:0/745048500 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560403503570 con 0x560403667310 2024-05-27T08:07:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.856+0000 7f6293e00700 10 client.21945 handle_command_reply: tid=0 2024-05-27T08:07:25.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.856+0000 7f62c7795b80 1 client.21945 shutdown 2024-05-27T08:07:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.856+0000 7f62c7795b80 1 -- 172.21.15.63:0/745048500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 msgr2=0x5604035e7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.856+0000 7f62c7795b80 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035e7910 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f62a0013850 tx=0x7f62a0013c10 comp rx=0 tx=0).stop 2024-05-27T08:07:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.857+0000 7f62c7795b80 1 -- 172.21.15.63:0/745048500 shutdown_connections 2024-05-27T08:07:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.857+0000 7f62c7795b80 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x560403667310 0x560403669730 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.857+0000 7f62c7795b80 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604035e42c0 0x5604035e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.857+0000 7f62c7795b80 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56040345ae40 0x5604035ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.857+0000 7f62c7795b80 1 --2- 172.21.15.63:0/745048500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5604035469f0 0x5604035e7910 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.857+0000 7f62c7795b80 1 -- 172.21.15.63:0/745048500 >> 172.21.15.63:0/745048500 conn(0x5604034f4390 msgr2=0x5604035da750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:25.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.857+0000 7f6293e00700 0 client.21945 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:25.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.857+0000 7f62c7795b80 1 -- 172.21.15.63:0/745048500 shutdown_connections 2024-05-27T08:07:25.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.857+0000 7f62c7795b80 1 -- 172.21.15.63:0/745048500 wait complete. 2024-05-27T08:07:25.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.857+0000 7f62c7795b80 20 client.21945 trim_cache size 0 max 16384 2024-05-27T08:07:25.860 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:07:25.861 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:07:25.861 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:07:25.861 INFO:teuthology.orchestra.run.smithi063.stdout: "76c1fd8d-f78a-4581-811c-da6203a944c3": { 2024-05-27T08:07:25.861 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:07:25.861 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "76c1fd8d-f78a-4581-811c-da6203a944c3", 2024-05-27T08:07:25.861 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:07:25.861 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:07:25.861 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:07:25.861 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:07:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.858+0000 7f62b3200700 1 -- 172.21.15.63:0/2890496 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62980b29f0 msgr2=0x7f62980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.858+0000 7f62b3200700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62980b29f0 0x7f62980b4f00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f62ac01f000 tx=0x7f62ac01d000 comp rx=0 tx=0).stop 2024-05-27T08:07:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.858+0000 7f62b3200700 1 -- 172.21.15.63:0/2890496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62b410edf0 msgr2=0x7f62b41a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.858+0000 7f62b3200700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62b410edf0 0x7f62b41a55c0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f62a4000c00 tx=0x7f62a400f510 comp rx=0 tx=0).stop 2024-05-27T08:07:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.858+0000 7f62b3200700 1 -- 172.21.15.63:0/2890496 shutdown_connections 2024-05-27T08:07:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.858+0000 7f62b3200700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62980b29f0 0x7f62980b4f00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.858+0000 7f62b3200700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62b4114070 0x7f62b41a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.858+0000 7f62b3200700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62b410edf0 0x7f62b41a55c0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.858+0000 7f62b3200700 1 --2- 172.21.15.63:0/2890496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62b4113720 0x7f62b41a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.858+0000 7f62b3200700 1 -- 172.21.15.63:0/2890496 >> 172.21.15.63:0/2890496 conn(0x7f62b410a980 msgr2=0x7f62b4111950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.859+0000 7f62b3200700 1 -- 172.21.15.63:0/2890496 shutdown_connections 2024-05-27T08:07:25.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:07:25.859+0000 7f62b3200700 1 -- 172.21.15.63:0/2890496 wait complete. 2024-05-27T08:07:25.891 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:25 smithi063 ceph-mon[31613]: pgmap v1073: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:07:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:25 smithi086 ceph-mon[34687]: pgmap v1073: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:07:25.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:25 smithi149 ceph-mon[35408]: pgmap v1073: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:07:26.864 INFO:tasks.cephfs.filesystem:scrub status for tag:76c1fd8d-f78a-4581-811c-da6203a944c3 - {'path': '/', 'tag': '76c1fd8d-f78a-4581-811c-da6203a944c3', 'options': 'recursive,force'} 2024-05-27T08:07:27.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:27 smithi086 ceph-mon[34687]: pgmap v1074: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.9 MiB/s wr, 54 op/s 2024-05-27T08:07:27.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:27 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:07:27.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:27 smithi149 ceph-mon[35408]: pgmap v1074: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.9 MiB/s wr, 54 op/s 2024-05-27T08:07:27.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:27 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:07:27.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:27 smithi063 ceph-mon[31613]: pgmap v1074: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.9 MiB/s wr, 54 op/s 2024-05-27T08:07:27.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:27 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:07:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:28 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:07:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:28 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:07:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:28 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:07:28.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:28 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:07:28.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:28 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:07:28.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:28 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:07:28.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:28 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:07:28.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:28 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:07:28.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:28 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:07:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:29 smithi086 ceph-mon[34687]: pgmap v1075: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 43 op/s 2024-05-27T08:07:29.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:29 smithi149 ceph-mon[35408]: pgmap v1075: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 43 op/s 2024-05-27T08:07:29.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:29 smithi063 ceph-mon[31613]: pgmap v1075: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 43 op/s 2024-05-27T08:07:31.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:31 smithi086 ceph-mon[34687]: pgmap v1076: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 43 op/s 2024-05-27T08:07:31.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:31 smithi149 ceph-mon[35408]: pgmap v1076: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 43 op/s 2024-05-27T08:07:31.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:31 smithi063 ceph-mon[31613]: pgmap v1076: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.0 MiB/s wr, 43 op/s 2024-05-27T08:07:32.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:32 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:07:32.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:32 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:07:32.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:32 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:07:32.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:32 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:07:32.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:32 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:07:32.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:32 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:07:32.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:32 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:07:32.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:32 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:07:32.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:32 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:07:33.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:33 smithi086 ceph-mon[34687]: pgmap v1077: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.4 MiB/s wr, 47 op/s 2024-05-27T08:07:33.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:33 smithi149 ceph-mon[35408]: pgmap v1077: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.4 MiB/s wr, 47 op/s 2024-05-27T08:07:33.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:33 smithi063 ceph-mon[31613]: pgmap v1077: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.4 MiB/s wr, 47 op/s 2024-05-27T08:07:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.635+0000 7fc513e00640 1 -- 172.21.15.63:0/4039741528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc51410e000 msgr2=0x7fc5141104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.635+0000 7fc513e00640 1 --2- 172.21.15.63:0/4039741528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc51410e000 0x7fc5141104f0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009ca0 tx=0x7fc50c02a8c0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.635+0000 7fc513e00640 1 -- 172.21.15.63:0/4039741528 shutdown_connections 2024-05-27T08:07:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.635+0000 7fc513e00640 1 --2- 172.21.15.63:0/4039741528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc51410e000 0x7fc5141104f0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.635+0000 7fc513e00640 1 --2- 172.21.15.63:0/4039741528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410b610 0x7fc51410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.635+0000 7fc513e00640 1 --2- 172.21.15.63:0/4039741528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc514104bb0 0x7fc514104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.635+0000 7fc513e00640 1 -- 172.21.15.63:0/4039741528 >> 172.21.15.63:0/4039741528 conn(0x7fc5140fe210 msgr2=0x7fc514100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.635+0000 7fc513e00640 1 -- 172.21.15.63:0/4039741528 shutdown_connections 2024-05-27T08:07:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.635+0000 7fc513e00640 1 -- 172.21.15.63:0/4039741528 wait complete. 2024-05-27T08:07:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.637+0000 7fc513e00640 1 Processor -- start 2024-05-27T08:07:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.637+0000 7fc513e00640 1 -- start start 2024-05-27T08:07:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.637+0000 7fc513e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc514104bb0 0x7fc51419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.637+0000 7fc513e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410b610 0x7fc51419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.637+0000 7fc513e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc51410e000 0x7fc51419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.637+0000 7fc513e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc514113b60 con 0x7fc51410e000 2024-05-27T08:07:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.637+0000 7fc513e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141139e0 con 0x7fc514104bb0 2024-05-27T08:07:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.637+0000 7fc513e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc514113ce0 con 0x7fc51410b610 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.638+0000 7fc513400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc51410e000 0x7fc51419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.638+0000 7fc513400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc51410e000 0x7fc51419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50082/0 (socket says 172.21.15.63:50082) 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.638+0000 7fc513400640 1 -- 172.21.15.63:0/3826455034 learned_addr learned my addr 172.21.15.63:0/3826455034 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.638+0000 7fc512a00640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc514104bb0 0x7fc51419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.638+0000 7fc512000640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410b610 0x7fc51419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.639+0000 7fc513400640 1 -- 172.21.15.63:0/3826455034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410b610 msgr2=0x7fc51419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.639+0000 7fc513400640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410b610 0x7fc51419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.639+0000 7fc513400640 1 -- 172.21.15.63:0/3826455034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc514104bb0 msgr2=0x7fc51419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.639+0000 7fc513400640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc514104bb0 0x7fc51419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.639+0000 7fc513400640 1 -- 172.21.15.63:0/3826455034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51419f3e0 con 0x7fc51410e000 2024-05-27T08:07:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.639+0000 7fc513400640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc51410e000 0x7fc51419eda0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005cc0 tx=0x7fc50c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.639+0000 7fc518a00640 1 -- 172.21.15.63:0/3826455034 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c032e00 con 0x7fc51410e000 2024-05-27T08:07:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.639+0000 7fc518a00640 1 -- 172.21.15.63:0/3826455034 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc50c0044d0 con 0x7fc51410e000 2024-05-27T08:07:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.639+0000 7fc518a00640 1 -- 172.21.15.63:0/3826455034 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c030b70 con 0x7fc51410e000 2024-05-27T08:07:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.640+0000 7fc513e00640 1 -- 172.21.15.63:0/3826455034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51419f670 con 0x7fc51410e000 2024-05-27T08:07:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.640+0000 7fc513e00640 1 -- 172.21.15.63:0/3826455034 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51419fb70 con 0x7fc51410e000 2024-05-27T08:07:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.643+0000 7fc520db4740 1 Processor -- start 2024-05-27T08:07:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.644+0000 7fc518a00640 1 -- 172.21.15.63:0/3826455034 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc50c004680 con 0x7fc51410e000 2024-05-27T08:07:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.644+0000 7fc520db4740 1 -- start start 2024-05-27T08:07:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.645+0000 7fc520db4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e2671b0 0x55629e60de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.645+0000 7fc520db4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e23e2f0 0x55629e60e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.645+0000 7fc520db4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e60e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.645+0000 7fc520db4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55629e57ab90 con 0x55629e23e2f0 2024-05-27T08:07:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.645+0000 7fc520db4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55629e56a730 con 0x55629e2464d0 2024-05-27T08:07:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.645+0000 7fc520db4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55629e56e770 con 0x55629e2671b0 2024-05-27T08:07:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.645+0000 7fc512a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e23e2f0 0x55629e60e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.645+0000 7fc512a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e23e2f0 0x55629e60e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50088/0 (socket says 172.21.15.63:50088) 2024-05-27T08:07:34.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.645+0000 7fc512a00640 1 -- 172.21.15.63:0/711223667 learned_addr learned my addr 172.21.15.63:0/711223667 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.646+0000 7fc512000640 1 --2- 172.21.15.63:0/711223667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e2671b0 0x55629e60de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.646+0000 7fc512000640 1 --2- 172.21.15.63:0/711223667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e60e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.646+0000 7fc512a00640 1 -- 172.21.15.63:0/711223667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e2671b0 msgr2=0x55629e60de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.646+0000 7fc512a00640 1 --2- 172.21.15.63:0/711223667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e2671b0 0x55629e60de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.646+0000 7fc512a00640 1 -- 172.21.15.63:0/711223667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 msgr2=0x55629e60e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.646+0000 7fc512a00640 1 --2- 172.21.15.63:0/711223667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e60e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.646+0000 7fc512a00640 1 -- 172.21.15.63:0/711223667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629e5038b0 con 0x55629e23e2f0 2024-05-27T08:07:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.646+0000 7fc518a00640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4f80b2950 0x7fc4f80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.646+0000 7fc518a00640 1 -- 172.21.15.63:0/3826455034 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fc50c103da0 con 0x7fc51410e000 2024-05-27T08:07:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.647+0000 7fc512000640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4f80b2950 0x7fc4f80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.647+0000 7fc512a00640 1 --2- 172.21.15.63:0/711223667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e23e2f0 0x55629e60e3b0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fc50400bcd0 tx=0x7fc5040107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cd07952e2771c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:34.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.647+0000 7fc4ffe00640 1 -- 172.21.15.63:0/711223667 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504005220 con 0x55629e23e2f0 2024-05-27T08:07:34.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.647+0000 7fc4ffe00640 1 -- 172.21.15.63:0/711223667 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc504004c30 con 0x55629e23e2f0 2024-05-27T08:07:34.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.647+0000 7fc4ffe00640 1 -- 172.21.15.63:0/711223667 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5040116a0 con 0x55629e23e2f0 2024-05-27T08:07:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.648+0000 7fc520db4740 1 -- 172.21.15.63:0/711223667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e23e2f0 msgr2=0x55629e60e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.648+0000 7fc520db4740 1 --2- 172.21.15.63:0/711223667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e23e2f0 0x55629e60e3b0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fc50400bcd0 tx=0x7fc5040107a0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.648+0000 7fc512000640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4f80b2950 0x7fc4f80b4e60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55629e60afe0 tx=0x7fc50000b390 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.648+0000 7fc520db4740 1 -- 172.21.15.63:0/711223667 shutdown_connections 2024-05-27T08:07:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.648+0000 7fc520db4740 1 --2- 172.21.15.63:0/711223667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e60e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.648+0000 7fc520db4740 1 --2- 172.21.15.63:0/711223667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e23e2f0 0x55629e60e3b0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.648+0000 7fc520db4740 1 --2- 172.21.15.63:0/711223667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e2671b0 0x55629e60de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.648+0000 7fc520db4740 1 -- 172.21.15.63:0/711223667 >> 172.21.15.63:0/711223667 conn(0x55629e25b300 msgr2=0x55629e60a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:34.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.648+0000 7fc520db4740 1 -- 172.21.15.63:0/711223667 shutdown_connections 2024-05-27T08:07:34.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.649+0000 7fc520db4740 1 -- 172.21.15.63:0/711223667 wait complete. 2024-05-27T08:07:34.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.649+0000 7fc520db4740 1 -- start start 2024-05-27T08:07:34.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.649+0000 7fc520db4740 1 Processor -- start 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.650+0000 7fc520db4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 0x55629e618070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.650+0000 7fc520db4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e6185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.650+0000 7fc520db4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e6242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.650+0000 7fc520db4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55629e250790 con 0x55629e2671b0 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.650+0000 7fc520db4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55629e512bc0 con 0x55629e2464d0 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.650+0000 7fc520db4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55629e61afa0 con 0x55629e23e2f0 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.650+0000 7fc520db4740 10 client.0 _ll_register_callbacks cb 0x55629e4f0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.650+0000 7fc513400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e6242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc513400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e6242d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50090/0 (socket says 172.21.15.63:50090) 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc513400640 1 -- 172.21.15.63:0/1074203114 learned_addr learned my addr 172.21.15.63:0/1074203114 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc512a00640 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 0x55629e618070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc512000640 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e6185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc513400640 1 -- 172.21.15.63:0/1074203114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 msgr2=0x55629e618070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc513400640 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 0x55629e618070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc513400640 1 -- 172.21.15.63:0/1074203114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 msgr2=0x55629e6185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc513400640 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e6185b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc513400640 1 -- 172.21.15.63:0/1074203114 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629e27f210 con 0x55629e2671b0 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc513400640 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e6242d0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005e30 tx=0x7fc50c0ca7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11da4750db976461 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.651+0000 7fc4fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.652+0000 7fc4fcc00640 1 -- 172.21.15.63:0/1074203114 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0cad70 con 0x55629e2671b0 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.652+0000 7fc4fcc00640 1 -- 172.21.15.63:0/1074203114 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fc50c0e53f0 con 0x55629e2671b0 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.652+0000 7fc520db4740 1 -- 172.21.15.63:0/1074203114 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc504004c30 con 0x55629e2671b0 2024-05-27T08:07:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.652+0000 7fc4fcc00640 1 -- 172.21.15.63:0/1074203114 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc50c0cf120 con 0x55629e2671b0 2024-05-27T08:07:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.652+0000 7fc520db4740 10 client.21957 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.652+0000 7fc520db4740 1 -- 172.21.15.63:0/1074203114 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55629e34b620 con 0x55629e2671b0 2024-05-27T08:07:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.656+0000 7fc4fcc00640 1 -- 172.21.15.63:0/1074203114 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc504004c30 con 0x55629e2671b0 2024-05-27T08:07:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.656+0000 7fc4fcc00640 1 -- 172.21.15.63:0/1074203114 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc50c0d3740 con 0x55629e2671b0 2024-05-27T08:07:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.656+0000 7fc520db4740 10 client.21957 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.656+0000 7fc520db4740 10 client.21957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.656+0000 7fc520db4740 20 client.21957 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.656+0000 7fc520db4740 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55629e225d80 0x55629e62a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.656+0000 7fc520db4740 4 client.21957 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:07:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.656+0000 7fc520db4740 1 -- 172.21.15.63:0/1074203114 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55629e57cae0 con 0x55629e225d80 2024-05-27T08:07:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.657+0000 7fc512a00640 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55629e225d80 0x55629e62a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.659 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.658+0000 7fc512a00640 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55629e225d80 0x55629e62a320 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc879a18634e756 server_cookie=a8be01408193186c in_seq=0 out_seq=0 2024-05-27T08:07:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.658+0000 7fc4fcc00640 10 client.21957 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.660+0000 7fc4fcc00640 1 -- 172.21.15.63:0/1074203114 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55629e57cae0 con 0x55629e225d80 2024-05-27T08:07:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.660+0000 7fc4fcc00640 10 client.21957 handle_command_reply: tid=0 2024-05-27T08:07:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.660+0000 7fc520db4740 1 client.21957 shutdown 2024-05-27T08:07:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.660+0000 7fc520db4740 1 -- 172.21.15.63:0/1074203114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 msgr2=0x55629e6242d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.660+0000 7fc520db4740 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e6242d0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005e30 tx=0x7fc50c0ca7f0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.661+0000 7fc520db4740 1 -- 172.21.15.63:0/1074203114 shutdown_connections 2024-05-27T08:07:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.661+0000 7fc520db4740 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55629e225d80 0x55629e62a320 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.661+0000 7fc520db4740 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e6242d0 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.661+0000 7fc520db4740 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e6185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.661+0000 7fc520db4740 1 --2- 172.21.15.63:0/1074203114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 0x55629e618070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.661+0000 7fc520db4740 1 -- 172.21.15.63:0/1074203114 >> 172.21.15.63:0/1074203114 conn(0x55629e25b300 msgr2=0x55629e60dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.661+0000 7fc4fcc00640 0 client.21957 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.661+0000 7fc520db4740 1 -- 172.21.15.63:0/1074203114 shutdown_connections 2024-05-27T08:07:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.661+0000 7fc520db4740 1 -- 172.21.15.63:0/1074203114 wait complete. 2024-05-27T08:07:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.661+0000 7fc520db4740 20 client.21957 trim_cache size 0 max 16384 2024-05-27T08:07:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.683+0000 7fc520db4740 1 Processor -- start 2024-05-27T08:07:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.683+0000 7fc520db4740 1 -- start start 2024-05-27T08:07:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.683+0000 7fc520db4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 0x55629e61a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.683+0000 7fc520db4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e613850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.683+0000 7fc520db4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e624040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.683+0000 7fc520db4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141b1600 con 0x55629e2671b0 2024-05-27T08:07:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.683+0000 7fc520db4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141aeed0 con 0x55629e2464d0 2024-05-27T08:07:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.683+0000 7fc520db4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141acd40 con 0x55629e23e2f0 2024-05-27T08:07:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc513400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e624040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc513400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e624040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50098/0 (socket says 172.21.15.63:50098) 2024-05-27T08:07:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc513400640 1 -- 172.21.15.63:0/2650013767 learned_addr learned my addr 172.21.15.63:0/2650013767 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc512000640 1 --2- 172.21.15.63:0/2650013767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e613850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc512a00640 1 --2- 172.21.15.63:0/2650013767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 0x55629e61a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc513400640 1 -- 172.21.15.63:0/2650013767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 msgr2=0x55629e61a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc513400640 1 --2- 172.21.15.63:0/2650013767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 0x55629e61a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc513400640 1 -- 172.21.15.63:0/2650013767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 msgr2=0x55629e613850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc513400640 1 --2- 172.21.15.63:0/2650013767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e613850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc513400640 1 -- 172.21.15.63:0/2650013767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629e60b210 con 0x55629e2671b0 2024-05-27T08:07:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.684+0000 7fc512a00640 1 --2- 172.21.15.63:0/2650013767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 0x55629e61a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:07:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.685+0000 7fc513400640 1 --2- 172.21.15.63:0/2650013767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e624040 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fc50c030670 tx=0x7fc50c0ca9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1aba294d007ec2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.685+0000 7fc4ff400640 1 -- 172.21.15.63:0/2650013767 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0cad70 con 0x55629e2671b0 2024-05-27T08:07:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.685+0000 7fc4ff400640 1 -- 172.21.15.63:0/2650013767 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc50c036040 con 0x55629e2671b0 2024-05-27T08:07:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.685+0000 7fc4ff400640 1 -- 172.21.15.63:0/2650013767 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0d3740 con 0x55629e2671b0 2024-05-27T08:07:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.686+0000 7fc520db4740 1 -- 172.21.15.63:0/2650013767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 msgr2=0x55629e624040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.686+0000 7fc520db4740 1 --2- 172.21.15.63:0/2650013767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e624040 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fc50c030670 tx=0x7fc50c0ca9e0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.686+0000 7fc520db4740 1 -- 172.21.15.63:0/2650013767 shutdown_connections 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.686+0000 7fc520db4740 1 --2- 172.21.15.63:0/2650013767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e624040 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.686+0000 7fc520db4740 1 --2- 172.21.15.63:0/2650013767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e2464d0 0x55629e613850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.686+0000 7fc520db4740 1 --2- 172.21.15.63:0/2650013767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e23e2f0 0x55629e61a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.686+0000 7fc520db4740 1 -- 172.21.15.63:0/2650013767 >> 172.21.15.63:0/2650013767 conn(0x55629e25b300 msgr2=0x55629e60e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.686+0000 7fc520db4740 1 -- 172.21.15.63:0/2650013767 shutdown_connections 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.686+0000 7fc520db4740 1 -- 172.21.15.63:0/2650013767 wait complete. 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.687+0000 7fc520db4740 1 -- start start 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.687+0000 7fc520db4740 1 Processor -- start 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc520db4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e23e2f0 0x55629e620660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc520db4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e2464d0 0x55629e61b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc520db4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e61bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc520db4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141ad390 con 0x55629e2671b0 2024-05-27T08:07:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc520db4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141ae800 con 0x55629e23e2f0 2024-05-27T08:07:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc520db4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141ada60 con 0x55629e2464d0 2024-05-27T08:07:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc520db4740 10 client.0 _ll_register_callbacks cb 0x7fc50c0305c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc513400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e61bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc513400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e61bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50110/0 (socket says 172.21.15.63:50110) 2024-05-27T08:07:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc513400640 1 -- 172.21.15.63:0/4179156634 learned_addr learned my addr 172.21.15.63:0/4179156634 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc512000640 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e2464d0 0x55629e61b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc512a00640 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e23e2f0 0x55629e620660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.688+0000 7fc513400640 1 -- 172.21.15.63:0/4179156634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e2464d0 msgr2=0x55629e61b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.689+0000 7fc513400640 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e2464d0 0x55629e61b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.689+0000 7fc513400640 1 -- 172.21.15.63:0/4179156634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e23e2f0 msgr2=0x55629e620660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.689+0000 7fc513400640 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e23e2f0 0x55629e620660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.689+0000 7fc513400640 1 -- 172.21.15.63:0/4179156634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629e61c430 con 0x55629e2671b0 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.689+0000 7fc512a00640 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e23e2f0 0x55629e620660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.689+0000 7fc513400640 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e61bef0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fc50c02ae80 tx=0x7fc50c0ca7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab5af81b155a539 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.689+0000 7fc4fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.689+0000 7fc4fcc00640 1 -- 172.21.15.63:0/4179156634 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0cb230 con 0x55629e2671b0 2024-05-27T08:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.689+0000 7fc4fcc00640 1 -- 172.21.15.63:0/4179156634 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fc50c0c7000 con 0x55629e2671b0 2024-05-27T08:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.689+0000 7fc520db4740 1 -- 172.21.15.63:0/4179156634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc50c036040 con 0x55629e2671b0 2024-05-27T08:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc4fcc00640 1 -- 172.21.15.63:0/4179156634 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc50c0cad70 con 0x55629e2671b0 2024-05-27T08:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc520db4740 10 client.21963 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc520db4740 1 -- 172.21.15.63:0/4179156634 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55629e692570 con 0x55629e2671b0 2024-05-27T08:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc4fcc00640 1 -- 172.21.15.63:0/4179156634 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc50c036040 con 0x55629e2671b0 2024-05-27T08:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc4fcc00640 1 -- 172.21.15.63:0/4179156634 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc50c0c66e0 con 0x55629e2671b0 2024-05-27T08:07:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc520db4740 10 client.21963 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc520db4740 10 client.21963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc520db4740 20 client.21963 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc520db4740 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55629e1fb1c0 0x55629e694a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc520db4740 4 client.21963 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:07:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.690+0000 7fc520db4740 1 -- 172.21.15.63:0/4179156634 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55629e5762c0 con 0x55629e1fb1c0 2024-05-27T08:07:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.691+0000 7fc512a00640 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55629e1fb1c0 0x55629e694a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.692+0000 7fc512a00640 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55629e1fb1c0 0x55629e694a50 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad8e3ee5f5a7d1 server_cookie=49613d8578986af5 in_seq=0 out_seq=0 2024-05-27T08:07:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.692+0000 7fc4fcc00640 10 client.21963 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.744+0000 7fc4fcc00640 1 -- 172.21.15.63:0/4179156634 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55629e5762c0 con 0x55629e1fb1c0 2024-05-27T08:07:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.744+0000 7fc4fcc00640 10 client.21963 handle_command_reply: tid=0 2024-05-27T08:07:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.744+0000 7fc520db4740 1 client.21963 shutdown 2024-05-27T08:07:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.744+0000 7fc520db4740 1 -- 172.21.15.63:0/4179156634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 msgr2=0x55629e61bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.744+0000 7fc520db4740 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e61bef0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fc50c02ae80 tx=0x7fc50c0ca7f0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.744+0000 7fc520db4740 1 -- 172.21.15.63:0/4179156634 shutdown_connections 2024-05-27T08:07:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.744+0000 7fc520db4740 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55629e1fb1c0 0x55629e694a50 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.745+0000 7fc520db4740 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55629e2671b0 0x55629e61bef0 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.745+0000 7fc520db4740 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55629e2464d0 0x55629e61b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.745+0000 7fc520db4740 1 --2- 172.21.15.63:0/4179156634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55629e23e2f0 0x55629e620660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.745+0000 7fc520db4740 1 -- 172.21.15.63:0/4179156634 >> 172.21.15.63:0/4179156634 conn(0x55629e25b300 msgr2=0x55629e60e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.745+0000 7fc4fcc00640 0 client.21963 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.745+0000 7fc520db4740 1 -- 172.21.15.63:0/4179156634 shutdown_connections 2024-05-27T08:07:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.745+0000 7fc520db4740 1 -- 172.21.15.63:0/4179156634 wait complete. 2024-05-27T08:07:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.745+0000 7fc520db4740 20 client.21963 trim_cache size 0 max 16384 2024-05-27T08:07:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:07:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:07:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:07:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 -- 172.21.15.63:0/3826455034 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4f80b2950 msgr2=0x7fc4f80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4f80b2950 0x7fc4f80b4e60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55629e60afe0 tx=0x7fc50000b390 comp rx=0 tx=0).stop 2024-05-27T08:07:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 -- 172.21.15.63:0/3826455034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc51410e000 msgr2=0x7fc51419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc51410e000 0x7fc51419eda0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005cc0 tx=0x7fc50c032a10 comp rx=0 tx=0).stop 2024-05-27T08:07:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 -- 172.21.15.63:0/3826455034 shutdown_connections 2024-05-27T08:07:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4f80b2950 0x7fc4f80b4e60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc51410e000 0x7fc51419eda0 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc51410b610 0x7fc51419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 --2- 172.21.15.63:0/3826455034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc514104bb0 0x7fc51419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 -- 172.21.15.63:0/3826455034 >> 172.21.15.63:0/3826455034 conn(0x7fc5140fe210 msgr2=0x7fc51410e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 -- 172.21.15.63:0/3826455034 shutdown_connections 2024-05-27T08:07:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:34.746+0000 7fc513e00640 1 -- 172.21.15.63:0/3826455034 wait complete. 2024-05-27T08:07:35.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:35 smithi086 ceph-mon[34687]: pgmap v1078: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.0 MiB/s wr, 44 op/s 2024-05-27T08:07:35.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:35 smithi149 ceph-mon[35408]: pgmap v1078: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.0 MiB/s wr, 44 op/s 2024-05-27T08:07:35.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:35 smithi063 ceph-mon[31613]: pgmap v1078: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.0 MiB/s wr, 44 op/s 2024-05-27T08:07:37.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:37 smithi086 ceph-mon[34687]: pgmap v1079: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.0 MiB/s wr, 50 op/s 2024-05-27T08:07:37.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:37 smithi149 ceph-mon[35408]: pgmap v1079: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.0 MiB/s wr, 50 op/s 2024-05-27T08:07:37.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:37 smithi063 ceph-mon[31613]: pgmap v1079: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.0 MiB/s wr, 50 op/s 2024-05-27T08:07:39.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:39 smithi086 ceph-mon[34687]: pgmap v1080: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:07:39.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:39 smithi149 ceph-mon[35408]: pgmap v1080: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:07:39.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:39 smithi063 ceph-mon[31613]: pgmap v1080: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:07:41.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:41 smithi086 ceph-mon[34687]: pgmap v1081: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:07:41.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:41 smithi149 ceph-mon[35408]: pgmap v1081: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:07:41.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:41 smithi063 ceph-mon[31613]: pgmap v1081: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:07:43.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:43 smithi086 ceph-mon[34687]: pgmap v1082: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:07:43.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:43 smithi149 ceph-mon[35408]: pgmap v1082: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:07:43.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:43 smithi063 ceph-mon[31613]: pgmap v1082: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:07:45.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:45 smithi086 ceph-mon[34687]: pgmap v1083: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:07:45.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:45 smithi149 ceph-mon[35408]: pgmap v1083: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:07:45.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:45 smithi063 ceph-mon[31613]: pgmap v1083: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:07:47.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:47 smithi086 ceph-mon[34687]: pgmap v1084: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:07:47.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:47 smithi149 ceph-mon[35408]: pgmap v1084: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:07:47.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:47 smithi063 ceph-mon[31613]: pgmap v1084: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:07:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.003+0000 7fbd4f200640 1 -- 172.21.15.63:0/2313237267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd481073f0 msgr2=0x7fbd48107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.003+0000 7fbd4f200640 1 --2- 172.21.15.63:0/2313237267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd481073f0 0x7fbd48107800 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009900 tx=0x7fbd3802a400 comp rx=0 tx=0).stop 2024-05-27T08:07:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.003+0000 7fbd4f200640 1 -- 172.21.15.63:0/2313237267 shutdown_connections 2024-05-27T08:07:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.003+0000 7fbd4f200640 1 --2- 172.21.15.63:0/2313237267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48107e20 0x7fbd481082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.003+0000 7fbd4f200640 1 --2- 172.21.15.63:0/2313237267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd481073f0 0x7fbd48107800 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.003+0000 7fbd4f200640 1 --2- 172.21.15.63:0/2313237267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd481029a0 0x7fbd48102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.003+0000 7fbd4f200640 1 -- 172.21.15.63:0/2313237267 >> 172.21.15.63:0/2313237267 conn(0x7fbd480fe210 msgr2=0x7fbd48100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.004+0000 7fbd4f200640 1 -- 172.21.15.63:0/2313237267 shutdown_connections 2024-05-27T08:07:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.004+0000 7fbd4f200640 1 -- 172.21.15.63:0/2313237267 wait complete. 2024-05-27T08:07:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.005+0000 7fbd4f200640 1 Processor -- start 2024-05-27T08:07:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.005+0000 7fbd4f200640 1 -- start start 2024-05-27T08:07:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.005+0000 7fbd4f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd481029a0 0x7fbd481a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.006+0000 7fbd4f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd481073f0 0x7fbd481a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.006+0000 7fbd4f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48107e20 0x7fbd481a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.006+0000 7fbd4f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4811a0e0 con 0x7fbd481029a0 2024-05-27T08:07:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.006+0000 7fbd4f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48119f60 con 0x7fbd481073f0 2024-05-27T08:07:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.006+0000 7fbd4f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4811a260 con 0x7fbd48107e20 2024-05-27T08:07:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.006+0000 7fbd4de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd481029a0 0x7fbd481a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.006+0000 7fbd4de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd481029a0 0x7fbd481a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46656/0 (socket says 172.21.15.63:46656) 2024-05-27T08:07:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.006+0000 7fbd4de00640 1 -- 172.21.15.63:0/46950556 learned_addr learned my addr 172.21.15.63:0/46950556 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.006+0000 7fbd4d400640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd481073f0 0x7fbd481a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.006+0000 7fbd4e800640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48107e20 0x7fbd481a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd4de00640 1 -- 172.21.15.63:0/46950556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48107e20 msgr2=0x7fbd481a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd4de00640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48107e20 0x7fbd481a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd4de00640 1 -- 172.21.15.63:0/46950556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd481073f0 msgr2=0x7fbd481a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd4de00640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd481073f0 0x7fbd481a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd4de00640 1 -- 172.21.15.63:0/46950556 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd481a0f70 con 0x7fbd481029a0 2024-05-27T08:07:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd4de00640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd481029a0 0x7fbd481a6770 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00c920 tx=0x7fbd3c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd42a00640 1 -- 172.21.15.63:0/46950556 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c007c70 con 0x7fbd481029a0 2024-05-27T08:07:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd4f200640 1 -- 172.21.15.63:0/46950556 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd481a1260 con 0x7fbd481029a0 2024-05-27T08:07:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd42a00640 1 -- 172.21.15.63:0/46950556 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd3c007e10 con 0x7fbd481029a0 2024-05-27T08:07:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd42a00640 1 -- 172.21.15.63:0/46950556 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0186f0 con 0x7fbd481029a0 2024-05-27T08:07:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.007+0000 7fbd4f200640 1 -- 172.21.15.63:0/46950556 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd481a1780 con 0x7fbd481029a0 2024-05-27T08:07:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.011+0000 7fbd57796740 1 Processor -- start 2024-05-27T08:07:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.011+0000 7fbd57796740 1 -- start start 2024-05-27T08:07:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd42a00640 1 -- 172.21.15.63:0/46950556 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbd3c00f450 con 0x7fbd481029a0 2024-05-27T08:07:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd57796740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 0x55c28e627a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd57796740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e627fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd57796740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e6284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd57796740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c28e598b90 con 0x55c28e2644d0 2024-05-27T08:07:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd57796740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c28e588730 con 0x55c28e25c2f0 2024-05-27T08:07:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd57796740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c28e58c770 con 0x55c28e2851b0 2024-05-27T08:07:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e6284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e6284e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46662/0 (socket says 172.21.15.63:46662) 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd4d400640 1 -- 172.21.15.63:0/509496750 learned_addr learned my addr 172.21.15.63:0/509496750 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.012+0000 7fbd4e800640 1 --2- 172.21.15.63:0/509496750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 0x55c28e627a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.013+0000 7fbd4de00640 1 --2- 172.21.15.63:0/509496750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e627fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.013+0000 7fbd4d400640 1 -- 172.21.15.63:0/509496750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 msgr2=0x55c28e627a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.013+0000 7fbd4d400640 1 --2- 172.21.15.63:0/509496750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 0x55c28e627a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.013+0000 7fbd4d400640 1 -- 172.21.15.63:0/509496750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 msgr2=0x55c28e627fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.013+0000 7fbd4d400640 1 --2- 172.21.15.63:0/509496750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e627fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.013+0000 7fbd4d400640 1 -- 172.21.15.63:0/509496750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28e5218b0 con 0x55c28e2644d0 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.014+0000 7fbd42a00640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd300b2950 0x7fbd300b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.014+0000 7fbd4d400640 1 --2- 172.21.15.63:0/509496750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e6284e0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fbd3802a920 tx=0x7fbd38031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efc990b17573c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.014+0000 7fbd42a00640 1 -- 172.21.15.63:0/46950556 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fbd3c0e2cd0 con 0x7fbd481029a0 2024-05-27T08:07:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.014+0000 7fbd4e800640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd300b2950 0x7fbd300b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.014+0000 7fbd37000640 1 -- 172.21.15.63:0/509496750 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38035030 con 0x55c28e2644d0 2024-05-27T08:07:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.014+0000 7fbd37000640 1 -- 172.21.15.63:0/509496750 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd380044c0 con 0x55c28e2644d0 2024-05-27T08:07:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.014+0000 7fbd37000640 1 -- 172.21.15.63:0/509496750 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38003b40 con 0x55c28e2644d0 2024-05-27T08:07:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.015+0000 7fbd57796740 1 -- 172.21.15.63:0/509496750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 msgr2=0x55c28e6284e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.015+0000 7fbd4e800640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd300b2950 0x7fbd300b4e60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55c28e624bd0 tx=0x7fbd44007600 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.015+0000 7fbd57796740 1 --2- 172.21.15.63:0/509496750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e6284e0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fbd3802a920 tx=0x7fbd38031dc0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.016+0000 7fbd57796740 1 -- 172.21.15.63:0/509496750 shutdown_connections 2024-05-27T08:07:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.016+0000 7fbd57796740 1 --2- 172.21.15.63:0/509496750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e6284e0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.016+0000 7fbd57796740 1 --2- 172.21.15.63:0/509496750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e627fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.016+0000 7fbd57796740 1 --2- 172.21.15.63:0/509496750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 0x55c28e627a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.016+0000 7fbd57796740 1 -- 172.21.15.63:0/509496750 >> 172.21.15.63:0/509496750 conn(0x55c28e279300 msgr2=0x55c28e624210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.016+0000 7fbd57796740 1 -- 172.21.15.63:0/509496750 shutdown_connections 2024-05-27T08:07:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.016+0000 7fbd57796740 1 -- 172.21.15.63:0/509496750 wait complete. 2024-05-27T08:07:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.016+0000 7fbd57796740 1 -- start start 2024-05-27T08:07:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.017+0000 7fbd57796740 1 Processor -- start 2024-05-27T08:07:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.017+0000 7fbd57796740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e25c2f0 0x55c28e631bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.017+0000 7fbd57796740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e632110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.017+0000 7fbd57796740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e2851b0 0x55c28e63de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.017+0000 7fbd57796740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c28e26e790 con 0x55c28e2644d0 2024-05-27T08:07:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.017+0000 7fbd57796740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c28e530bc0 con 0x55c28e2851b0 2024-05-27T08:07:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.017+0000 7fbd57796740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c28e634b30 con 0x55c28e25c2f0 2024-05-27T08:07:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.017+0000 7fbd57796740 10 client.0 _ll_register_callbacks cb 0x55c28e50e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.018+0000 7fbd4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e632110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.018+0000 7fbd4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e632110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46666/0 (socket says 172.21.15.63:46666) 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.018+0000 7fbd4d400640 1 -- 172.21.15.63:0/371672500 learned_addr learned my addr 172.21.15.63:0/371672500 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.018+0000 7fbd4de00640 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e25c2f0 0x55c28e631bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.018+0000 7fbd4e800640 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e2851b0 0x55c28e63de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.018+0000 7fbd4d400640 1 -- 172.21.15.63:0/371672500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e25c2f0 msgr2=0x55c28e631bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.018+0000 7fbd4d400640 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e25c2f0 0x55c28e631bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.018+0000 7fbd4d400640 1 -- 172.21.15.63:0/371672500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e2851b0 msgr2=0x55c28e63de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.018+0000 7fbd4d400640 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e2851b0 0x55c28e63de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.018+0000 7fbd4d400640 1 -- 172.21.15.63:0/371672500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28e29d210 con 0x55c28e2644d0 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.019+0000 7fbd4d400640 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e632110 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fbd38000c00 tx=0x7fbd38031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b53d6074cc41f8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.019+0000 7fbd2b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.019+0000 7fbd2b400640 1 -- 172.21.15.63:0/371672500 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38035070 con 0x55c28e2644d0 2024-05-27T08:07:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.019+0000 7fbd2b400640 1 -- 172.21.15.63:0/371672500 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fbd380317a0 con 0x55c28e2644d0 2024-05-27T08:07:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.019+0000 7fbd57796740 1 -- 172.21.15.63:0/371672500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c28e59aae0 con 0x55c28e2644d0 2024-05-27T08:07:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.019+0000 7fbd2b400640 1 -- 172.21.15.63:0/371672500 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbd3803c650 con 0x55c28e2644d0 2024-05-27T08:07:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.019+0000 7fbd57796740 10 client.21975 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.019+0000 7fbd57796740 1 -- 172.21.15.63:0/371672500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c28e369620 con 0x55c28e2644d0 2024-05-27T08:07:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.020+0000 7fbd2b400640 1 -- 172.21.15.63:0/371672500 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c28e59aae0 con 0x55c28e2644d0 2024-05-27T08:07:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.020+0000 7fbd2b400640 1 -- 172.21.15.63:0/371672500 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbd3804e690 con 0x55c28e2644d0 2024-05-27T08:07:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.020+0000 7fbd57796740 10 client.21975 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.020+0000 7fbd57796740 10 client.21975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.020+0000 7fbd57796740 20 client.21975 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.020+0000 7fbd57796740 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c28e243d80 0x55c28e643c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.020+0000 7fbd57796740 4 client.21975 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:07:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.020+0000 7fbd57796740 1 -- 172.21.15.63:0/371672500 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c28e5942c0 con 0x55c28e243d80 2024-05-27T08:07:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.021+0000 7fbd4de00640 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c28e243d80 0x55c28e643c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.022+0000 7fbd4de00640 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c28e243d80 0x55c28e643c20 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1dea9025adcec75 server_cookie=4c12ec1c2c4be8b9 in_seq=0 out_seq=0 2024-05-27T08:07:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.022+0000 7fbd2b400640 10 client.21975 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd2b400640 1 -- 172.21.15.63:0/371672500 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c28e5942c0 con 0x55c28e243d80 2024-05-27T08:07:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd2b400640 10 client.21975 handle_command_reply: tid=0 2024-05-27T08:07:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd57796740 1 client.21975 shutdown 2024-05-27T08:07:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd57796740 1 -- 172.21.15.63:0/371672500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 msgr2=0x55c28e632110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd57796740 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e632110 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fbd38000c00 tx=0x7fbd38031630 comp rx=0 tx=0).stop 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd57796740 1 -- 172.21.15.63:0/371672500 shutdown_connections 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd57796740 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c28e243d80 0x55c28e643c20 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd57796740 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e2851b0 0x55c28e63de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd57796740 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e632110 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd57796740 1 --2- 172.21.15.63:0/371672500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e25c2f0 0x55c28e631bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd2b400640 0 client.21975 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.024+0000 7fbd57796740 1 -- 172.21.15.63:0/371672500 >> 172.21.15.63:0/371672500 conn(0x55c28e279300 msgr2=0x55c28e629780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.025+0000 7fbd57796740 1 -- 172.21.15.63:0/371672500 shutdown_connections 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.025+0000 7fbd57796740 1 -- 172.21.15.63:0/371672500 wait complete. 2024-05-27T08:07:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.025+0000 7fbd57796740 20 client.21975 trim_cache size 0 max 16384 2024-05-27T08:07:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.046+0000 7fbd57796740 1 Processor -- start 2024-05-27T08:07:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.046+0000 7fbd57796740 1 -- start start 2024-05-27T08:07:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd57796740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e633dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd57796740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2644d0 0x55c28e63ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd57796740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2851b0 0x55c28e62b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd57796740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b9e10 con 0x55c28e2851b0 2024-05-27T08:07:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd57796740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b76e0 con 0x55c28e25c2f0 2024-05-27T08:07:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd57796740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b5550 con 0x55c28e2644d0 2024-05-27T08:07:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd4e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2851b0 0x55c28e62b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd4e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2851b0 0x55c28e62b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46680/0 (socket says 172.21.15.63:46680) 2024-05-27T08:07:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd4e800640 1 -- 172.21.15.63:0/682263734 learned_addr learned my addr 172.21.15.63:0/682263734 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd4d400640 1 --2- 172.21.15.63:0/682263734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2644d0 0x55c28e63ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.047+0000 7fbd4de00640 1 --2- 172.21.15.63:0/682263734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e633dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.048+0000 7fbd4e800640 1 -- 172.21.15.63:0/682263734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2644d0 msgr2=0x55c28e63ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.048+0000 7fbd4e800640 1 --2- 172.21.15.63:0/682263734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2644d0 0x55c28e63ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.048+0000 7fbd4e800640 1 -- 172.21.15.63:0/682263734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 msgr2=0x55c28e633dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.048+0000 7fbd4e800640 1 --2- 172.21.15.63:0/682263734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e633dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.048+0000 7fbd4e800640 1 -- 172.21.15.63:0/682263734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28e63eba0 con 0x55c28e2851b0 2024-05-27T08:07:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.048+0000 7fbd4de00640 1 --2- 172.21.15.63:0/682263734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e633dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:07:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.048+0000 7fbd4e800640 1 --2- 172.21.15.63:0/682263734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2851b0 0x55c28e62b710 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fbd4401e470 tx=0x7fbd44014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e4732066f6b534 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.049+0000 7fbd36600640 1 -- 172.21.15.63:0/682263734 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44003970 con 0x55c28e2851b0 2024-05-27T08:07:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.049+0000 7fbd36600640 1 -- 172.21.15.63:0/682263734 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd44004020 con 0x55c28e2851b0 2024-05-27T08:07:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.049+0000 7fbd36600640 1 -- 172.21.15.63:0/682263734 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44026790 con 0x55c28e2851b0 2024-05-27T08:07:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.050+0000 7fbd57796740 1 -- 172.21.15.63:0/682263734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2851b0 msgr2=0x55c28e62b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.050+0000 7fbd57796740 1 --2- 172.21.15.63:0/682263734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2851b0 0x55c28e62b710 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fbd4401e470 tx=0x7fbd44014000 comp rx=0 tx=0).stop 2024-05-27T08:07:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.050+0000 7fbd57796740 1 -- 172.21.15.63:0/682263734 shutdown_connections 2024-05-27T08:07:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.050+0000 7fbd57796740 1 --2- 172.21.15.63:0/682263734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2851b0 0x55c28e62b710 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.050+0000 7fbd57796740 1 --2- 172.21.15.63:0/682263734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2644d0 0x55c28e63ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.050+0000 7fbd57796740 1 --2- 172.21.15.63:0/682263734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e633dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.050+0000 7fbd57796740 1 -- 172.21.15.63:0/682263734 >> 172.21.15.63:0/682263734 conn(0x55c28e279300 msgr2=0x55c28e628310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.050+0000 7fbd57796740 1 -- 172.21.15.63:0/682263734 shutdown_connections 2024-05-27T08:07:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.050+0000 7fbd57796740 1 -- 172.21.15.63:0/682263734 wait complete. 2024-05-27T08:07:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.050+0000 7fbd57796740 1 -- start start 2024-05-27T08:07:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.051+0000 7fbd57796740 1 Processor -- start 2024-05-27T08:07:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.051+0000 7fbd57796740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e643a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.051+0000 7fbd57796740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e643fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.051+0000 7fbd57796740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 0x55c28e644510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.051+0000 7fbd57796740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b5ba0 con 0x55c28e2644d0 2024-05-27T08:07:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.051+0000 7fbd57796740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b7010 con 0x55c28e25c2f0 2024-05-27T08:07:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.051+0000 7fbd57796740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481b6270 con 0x55c28e2851b0 2024-05-27T08:07:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.051+0000 7fbd57796740 10 client.0 _ll_register_callbacks cb 0x7fbd3c0e6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:07:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e643fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e643fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46688/0 (socket says 172.21.15.63:46688) 2024-05-27T08:07:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4d400640 1 -- 172.21.15.63:0/1768918757 learned_addr learned my addr 172.21.15.63:0/1768918757 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4de00640 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e643a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4e800640 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 0x55c28e644510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4d400640 1 -- 172.21.15.63:0/1768918757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 msgr2=0x55c28e644510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4d400640 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 0x55c28e644510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4d400640 1 -- 172.21.15.63:0/1768918757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 msgr2=0x55c28e643a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4d400640 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e643a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4d400640 1 -- 172.21.15.63:0/1768918757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28e6aa0b0 con 0x55c28e2644d0 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.052+0000 7fbd4e800640 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 0x55c28e644510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.053+0000 7fbd4d400640 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e643fd0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fbd3802a920 tx=0x7fbd3804e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18093ed52dd3c486 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.053+0000 7fbd2b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.053+0000 7fbd2b400640 1 -- 172.21.15.63:0/1768918757 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38035040 con 0x55c28e2644d0 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.053+0000 7fbd2b400640 1 -- 172.21.15.63:0/1768918757 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fbd3804e840 con 0x55c28e2644d0 2024-05-27T08:07:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.053+0000 7fbd57796740 1 -- 172.21.15.63:0/1768918757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd44004020 con 0x55c28e2644d0 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.053+0000 7fbd2b400640 1 -- 172.21.15.63:0/1768918757 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbd38069050 con 0x55c28e2644d0 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.053+0000 7fbd57796740 10 client.21981 fetch_fsmap learned FSMap version 35 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.053+0000 7fbd57796740 1 -- 172.21.15.63:0/1768918757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c28e6ab320 con 0x55c28e2644d0 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.054+0000 7fbd2b400640 1 -- 172.21.15.63:0/1768918757 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd44004020 con 0x55c28e2644d0 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.054+0000 7fbd2b400640 1 -- 172.21.15.63:0/1768918757 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbd3805b680 con 0x55c28e2644d0 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.054+0000 7fbd57796740 10 client.21981 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.054+0000 7fbd57796740 10 client.21981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.054+0000 7fbd57796740 20 client.21981 populate_metadata read hostname 'smithi063' 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.054+0000 7fbd57796740 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c28e2191c0 0x55c28e6ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.054+0000 7fbd57796740 4 client.21981 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:07:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.054+0000 7fbd57796740 1 -- 172.21.15.63:0/1768918757 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c28e50e4b0 con 0x55c28e2191c0 2024-05-27T08:07:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.055+0000 7fbd4de00640 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c28e2191c0 0x55c28e6ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:07:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.056+0000 7fbd4de00640 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c28e2191c0 0x55c28e6ada40 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac8b97f14f3517d server_cookie=f59ba845a61e02a8 in_seq=0 out_seq=0 2024-05-27T08:07:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.056+0000 7fbd2b400640 10 client.21981 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.093+0000 7fbd2b400640 1 -- 172.21.15.63:0/1768918757 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c28e50e4b0 con 0x55c28e2191c0 2024-05-27T08:07:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.093+0000 7fbd2b400640 10 client.21981 handle_command_reply: tid=0 2024-05-27T08:07:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.093+0000 7fbd57796740 1 client.21981 shutdown 2024-05-27T08:07:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.093+0000 7fbd57796740 1 -- 172.21.15.63:0/1768918757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 msgr2=0x55c28e643fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.093+0000 7fbd57796740 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e643fd0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fbd3802a920 tx=0x7fbd3804e650 comp rx=0 tx=0).stop 2024-05-27T08:07:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.094+0000 7fbd57796740 1 -- 172.21.15.63:0/1768918757 shutdown_connections 2024-05-27T08:07:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.094+0000 7fbd57796740 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c28e2191c0 0x55c28e6ada40 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.094+0000 7fbd57796740 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c28e2851b0 0x55c28e644510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.094+0000 7fbd57796740 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c28e2644d0 0x55c28e643fd0 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.094+0000 7fbd57796740 1 --2- 172.21.15.63:0/1768918757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c28e25c2f0 0x55c28e643a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.094+0000 7fbd57796740 1 -- 172.21.15.63:0/1768918757 >> 172.21.15.63:0/1768918757 conn(0x55c28e279300 msgr2=0x55c28e631e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.094+0000 7fbd2b400640 0 client.21981 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:07:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.094+0000 7fbd57796740 1 -- 172.21.15.63:0/1768918757 shutdown_connections 2024-05-27T08:07:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.094+0000 7fbd57796740 1 -- 172.21.15.63:0/1768918757 wait complete. 2024-05-27T08:07:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.094+0000 7fbd57796740 20 client.21981 trim_cache size 0 max 16384 2024-05-27T08:07:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:07:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:07:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:07:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 -- 172.21.15.63:0/46950556 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd300b2950 msgr2=0x7fbd300b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd300b2950 0x7fbd300b4e60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55c28e624bd0 tx=0x7fbd44007600 comp rx=0 tx=0).stop 2024-05-27T08:07:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 -- 172.21.15.63:0/46950556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd481029a0 msgr2=0x7fbd481a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:07:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd481029a0 0x7fbd481a6770 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00c920 tx=0x7fbd3c00cde0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 -- 172.21.15.63:0/46950556 shutdown_connections 2024-05-27T08:07:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd300b2950 0x7fbd300b4e60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48107e20 0x7fbd481a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd481073f0 0x7fbd481a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 --2- 172.21.15.63:0/46950556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd481029a0 0x7fbd481a6770 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:07:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 -- 172.21.15.63:0/46950556 >> 172.21.15.63:0/46950556 conn(0x7fbd480fe210 msgr2=0x7fbd48104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:07:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 -- 172.21.15.63:0/46950556 shutdown_connections 2024-05-27T08:07:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:07:48.095+0000 7fbd4f200640 1 -- 172.21.15.63:0/46950556 wait complete. 2024-05-27T08:07:49.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:49 smithi086 ceph-mon[34687]: pgmap v1085: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:07:49.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:49 smithi149 ceph-mon[35408]: pgmap v1085: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:07:49.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:49 smithi063 ceph-mon[31613]: pgmap v1085: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:07:51.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:51 smithi063 ceph-mon[31613]: pgmap v1086: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:07:52.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:51 smithi086 ceph-mon[34687]: pgmap v1086: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:07:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:51 smithi149 ceph-mon[35408]: pgmap v1086: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 7 op/s 2024-05-27T08:07:54.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:53 smithi086 ceph-mon[34687]: pgmap v1087: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:07:54.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:53 smithi149 ceph-mon[35408]: pgmap v1087: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:07:54.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:53 smithi063 ceph-mon[31613]: pgmap v1087: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:07:56.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:55 smithi086 ceph-mon[34687]: pgmap v1088: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:07:56.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:55 smithi149 ceph-mon[35408]: pgmap v1088: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:07:56.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:55 smithi063 ceph-mon[31613]: pgmap v1088: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:07:56.865 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:07:57.389 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:07:57.816 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:57 smithi063 ceph-mon[31613]: pgmap v1089: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:07:58.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:57 smithi086 ceph-mon[34687]: pgmap v1089: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:07:58.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:57 smithi149 ceph-mon[35408]: pgmap v1089: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:07:59.875 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:07:59 smithi063 ceph-mon[31613]: pgmap v1090: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:00.017 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:07:59 smithi149 ceph-mon[35408]: pgmap v1090: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:00.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:07:59 smithi086 ceph-mon[34687]: pgmap v1090: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:00.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.792+0000 7fea64200700 1 -- 172.21.15.63:0/1542158259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c10b2a0 msgr2=0x7fea5c117820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:00.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.792+0000 7fea64200700 1 --2- 172.21.15.63:0/1542158259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c10b2a0 0x7fea5c117820 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fea5400b0a0 tx=0x7fea5400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.792+0000 7fea64200700 1 -- 172.21.15.63:0/1542158259 shutdown_connections 2024-05-27T08:08:00.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.792+0000 7fea64200700 1 --2- 172.21.15.63:0/1542158259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c10b2a0 0x7fea5c117820 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.792+0000 7fea64200700 1 --2- 172.21.15.63:0/1542158259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c110980 0x7fea5c110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.792+0000 7fea64200700 1 --2- 172.21.15.63:0/1542158259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c10a950 0x7fea5c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.792+0000 7fea64200700 1 -- 172.21.15.63:0/1542158259 >> 172.21.15.63:0/1542158259 conn(0x7fea5c076d20 msgr2=0x7fea5c077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:00.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.792+0000 7fea64200700 1 -- 172.21.15.63:0/1542158259 shutdown_connections 2024-05-27T08:08:00.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.792+0000 7fea64200700 1 -- 172.21.15.63:0/1542158259 wait complete. 2024-05-27T08:08:00.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.794+0000 7fea64200700 1 Processor -- start 2024-05-27T08:08:00.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.794+0000 7fea64200700 1 -- start start 2024-05-27T08:08:00.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.794+0000 7fea64200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c10a950 0x7fea5c1a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.794+0000 7fea64200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c10b2a0 0x7fea5c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.794+0000 7fea64200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c110980 0x7fea5c1a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.794+0000 7fea64200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c11a9b0 con 0x7fea5c110980 2024-05-27T08:08:00.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.794+0000 7fea64200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c11a830 con 0x7fea5c10a950 2024-05-27T08:08:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.794+0000 7fea64200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c11ab30 con 0x7fea5c10b2a0 2024-05-27T08:08:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.795+0000 7fea62200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c110980 0x7fea5c1a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.795+0000 7fea61800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c10a950 0x7fea5c1a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.795+0000 7fea62200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c110980 0x7fea5c1a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37566/0 (socket says 172.21.15.63:37566) 2024-05-27T08:08:00.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.795+0000 7fea62200700 1 -- 172.21.15.63:0/4033380695 learned_addr learned my addr 172.21.15.63:0/4033380695 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:00.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.795+0000 7fea60e00700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c10b2a0 0x7fea5c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.796+0000 7fea62200700 1 -- 172.21.15.63:0/4033380695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c10b2a0 msgr2=0x7fea5c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:00.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.796+0000 7fea62200700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c10b2a0 0x7fea5c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.796+0000 7fea62200700 1 -- 172.21.15.63:0/4033380695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c10a950 msgr2=0x7fea5c1a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:00.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.796+0000 7fea62200700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c10a950 0x7fea5c1a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.796+0000 7fea62200700 1 -- 172.21.15.63:0/4033380695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c1a6180 con 0x7fea5c110980 2024-05-27T08:08:00.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.796+0000 7fea62200700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c110980 0x7fea5c1a5b90 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fea5400b070 tx=0x7fea54004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:00.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.796+0000 7fea5a000700 1 -- 172.21.15.63:0/4033380695 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54021070 con 0x7fea5c110980 2024-05-27T08:08:00.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.797+0000 7fea64200700 1 -- 172.21.15.63:0/4033380695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c1a6410 con 0x7fea5c110980 2024-05-27T08:08:00.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.797+0000 7fea5a000700 1 -- 172.21.15.63:0/4033380695 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea54007b80 con 0x7fea5c110980 2024-05-27T08:08:00.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.797+0000 7fea5a000700 1 -- 172.21.15.63:0/4033380695 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54004640 con 0x7fea5c110980 2024-05-27T08:08:00.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.797+0000 7fea64200700 1 -- 172.21.15.63:0/4033380695 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c1a6940 con 0x7fea5c110980 2024-05-27T08:08:00.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.802+0000 7fea5a000700 1 -- 172.21.15.63:0/4033380695 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fea54003a00 con 0x7fea5c110980 2024-05-27T08:08:00.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.804+0000 7fea5a000700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea440b2a70 0x7fea440b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.804+0000 7fea5a000700 1 -- 172.21.15.63:0/4033380695 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fea5400f020 con 0x7fea5c110980 2024-05-27T08:08:00.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.804+0000 7fea61800700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea440b2a70 0x7fea440b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.805+0000 7fea61800700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea440b2a70 0x7fea440b4f80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fea50005950 tx=0x7fea5000b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:00.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.818+0000 7fea72d2eb80 1 Processor -- start 2024-05-27T08:08:00.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.818+0000 7fea72d2eb80 1 -- start start 2024-05-27T08:08:00.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.818+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 0x562b5521c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.818+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b5521cd30 0x562b55216870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.818+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b55216db0 0x562b552171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.818+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562b55144e40 con 0x562b5521cd30 2024-05-27T08:08:00.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.818+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b551349e0 con 0x562b55097e40 2024-05-27T08:08:00.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.818+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562b55138a20 con 0x562b55216db0 2024-05-27T08:08:00.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.819+0000 7fea60e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b5521cd30 0x562b55216870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.819+0000 7fea60e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b5521cd30 0x562b55216870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37568/0 (socket says 172.21.15.63:37568) 2024-05-27T08:08:00.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.819+0000 7fea60e00700 1 -- 172.21.15.63:0/3939923813 learned_addr learned my addr 172.21.15.63:0/3939923813 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:00.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.819+0000 7fea61800700 1 --2- 172.21.15.63:0/3939923813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 0x562b5521c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.819+0000 7fea62200700 1 --2- 172.21.15.63:0/3939923813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b55216db0 0x562b552171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.820+0000 7fea60e00700 1 -- 172.21.15.63:0/3939923813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b55216db0 msgr2=0x562b552171c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:00.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.820+0000 7fea60e00700 1 --2- 172.21.15.63:0/3939923813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b55216db0 0x562b552171c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.820+0000 7fea60e00700 1 -- 172.21.15.63:0/3939923813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 msgr2=0x562b5521c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:00.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.820+0000 7fea60e00700 1 --2- 172.21.15.63:0/3939923813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 0x562b5521c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.820+0000 7fea60e00700 1 -- 172.21.15.63:0/3939923813 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b550fd420 con 0x562b5521cd30 2024-05-27T08:08:00.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.820+0000 7fea61800700 1 --2- 172.21.15.63:0/3939923813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 0x562b5521c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.820+0000 7fea60e00700 1 --2- 172.21.15.63:0/3939923813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b5521cd30 0x562b55216870 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fea4800ace0 tx=0x7fea4800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad31fcaca4c6ed9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.820+0000 7fea4ea00700 1 -- 172.21.15.63:0/3939923813 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48004020 con 0x562b5521cd30 2024-05-27T08:08:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.821+0000 7fea4ea00700 1 -- 172.21.15.63:0/3939923813 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea48003680 con 0x562b5521cd30 2024-05-27T08:08:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.821+0000 7fea4ea00700 1 -- 172.21.15.63:0/3939923813 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48018680 con 0x562b5521cd30 2024-05-27T08:08:00.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.822+0000 7fea72d2eb80 1 -- 172.21.15.63:0/3939923813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b5521cd30 msgr2=0x562b55216870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:00.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.822+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/3939923813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b5521cd30 0x562b55216870 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fea4800ace0 tx=0x7fea4800f3c0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.822+0000 7fea72d2eb80 1 -- 172.21.15.63:0/3939923813 shutdown_connections 2024-05-27T08:08:00.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.822+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/3939923813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b55216db0 0x562b552171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.822+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/3939923813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 0x562b5521c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.822+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/3939923813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b5521cd30 0x562b55216870 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.822+0000 7fea72d2eb80 1 -- 172.21.15.63:0/3939923813 >> 172.21.15.63:0/3939923813 conn(0x562b550f7fc0 msgr2=0x562b552118d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:00.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.822+0000 7fea72d2eb80 1 -- 172.21.15.63:0/3939923813 shutdown_connections 2024-05-27T08:08:00.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.822+0000 7fea72d2eb80 1 -- 172.21.15.63:0/3939923813 wait complete. 2024-05-27T08:08:00.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.823+0000 7fea72d2eb80 1 -- start start 2024-05-27T08:08:00.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.823+0000 7fea72d2eb80 1 Processor -- start 2024-05-27T08:08:00.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.824+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 0x562b55220530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.824+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55216db0 0x562b55220a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.824+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b5521cd30 0x562b55220fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.824+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562b5503afa0 con 0x562b55216db0 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.824+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b550d5760 con 0x562b55097e40 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.824+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562b55223800 con 0x562b5521cd30 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.824+0000 7fea72d2eb80 10 client.0 _ll_register_callbacks cb 0x562b550dec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.824+0000 7fea60e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55216db0 0x562b55220a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.824+0000 7fea60e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55216db0 0x562b55220a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37584/0 (socket says 172.21.15.63:37584) 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.824+0000 7fea60e00700 1 -- 172.21.15.63:0/4011774969 learned_addr learned my addr 172.21.15.63:0/4011774969 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.825+0000 7fea61800700 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 0x562b55220530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.825+0000 7fea62200700 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b5521cd30 0x562b55220fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.825+0000 7fea60e00700 1 -- 172.21.15.63:0/4011774969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b5521cd30 msgr2=0x562b55220fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.825+0000 7fea60e00700 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b5521cd30 0x562b55220fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.825+0000 7fea60e00700 1 -- 172.21.15.63:0/4011774969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 msgr2=0x562b55220530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.825+0000 7fea60e00700 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 0x562b55220530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.825+0000 7fea60e00700 1 -- 172.21.15.63:0/4011774969 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b550848d0 con 0x562b55216db0 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.825+0000 7fea60e00700 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55216db0 0x562b55220a70 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fea4800a8e0 tx=0x7fea48003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d70c8fa399cb62 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.826+0000 7fea43400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.826+0000 7fea43400700 1 -- 172.21.15.63:0/4011774969 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48026020 con 0x562b55216db0 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.826+0000 7fea43400700 1 -- 172.21.15.63:0/4011774969 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fea48004b60 con 0x562b55216db0 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.826+0000 7fea72d2eb80 1 -- 172.21.15.63:0/4011774969 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea48003680 con 0x562b55216db0 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.827+0000 7fea43400700 1 -- 172.21.15.63:0/4011774969 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fea48031050 con 0x562b55216db0 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.827+0000 7fea72d2eb80 10 client.21993 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.827+0000 7fea72d2eb80 1 -- 172.21.15.63:0/4011774969 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562b55105060 con 0x562b55216db0 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.827+0000 7fea43400700 1 -- 172.21.15.63:0/4011774969 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea48003680 con 0x562b55216db0 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.827+0000 7fea43400700 1 -- 172.21.15.63:0/4011774969 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fea48022580 con 0x562b55216db0 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.828+0000 7fea72d2eb80 10 client.21993 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:00.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.828+0000 7fea72d2eb80 10 client.21993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:00.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.828+0000 7fea72d2eb80 20 client.21993 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:00.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.828+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562b552305b0 0x562b552329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.828+0000 7fea72d2eb80 4 client.21993 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:00.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.828+0000 7fea72d2eb80 1 -- 172.21.15.63:0/4011774969 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b55146d90 con 0x562b552305b0 2024-05-27T08:08:00.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.828+0000 7fea61800700 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562b552305b0 0x562b552329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.829+0000 7fea61800700 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562b552305b0 0x562b552329d0 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34367a23f8407343 server_cookie=238e2a3206b07039 in_seq=0 out_seq=0 2024-05-27T08:08:00.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.829+0000 7fea43400700 10 client.21993 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:00.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.831+0000 7fea43400700 1 -- 172.21.15.63:0/4011774969 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562b55146d90 con 0x562b552305b0 2024-05-27T08:08:00.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.831+0000 7fea43400700 10 client.21993 handle_command_reply: tid=0 2024-05-27T08:08:00.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.831+0000 7fea72d2eb80 1 client.21993 shutdown 2024-05-27T08:08:00.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 1 -- 172.21.15.63:0/4011774969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55216db0 msgr2=0x562b55220a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:00.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55216db0 0x562b55220a70 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fea4800a8e0 tx=0x7fea48003cd0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 1 -- 172.21.15.63:0/4011774969 shutdown_connections 2024-05-27T08:08:00.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562b552305b0 0x562b552329d0 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b5521cd30 0x562b55220fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55097e40 0x562b55220530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/4011774969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55216db0 0x562b55220a70 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea43400700 0 client.21993 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:00.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 1 -- 172.21.15.63:0/4011774969 >> 172.21.15.63:0/4011774969 conn(0x562b550f7fc0 msgr2=0x562b55211580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:00.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 1 -- 172.21.15.63:0/4011774969 shutdown_connections 2024-05-27T08:08:00.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 1 -- 172.21.15.63:0/4011774969 wait complete. 2024-05-27T08:08:00.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.832+0000 7fea72d2eb80 20 client.21993 trim_cache size 0 max 16384 2024-05-27T08:08:00.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.856+0000 7fea72d2eb80 1 Processor -- start 2024-05-27T08:08:00.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.856+0000 7fea72d2eb80 1 -- start start 2024-05-27T08:08:00.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.856+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b5521e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.856+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b55216db0 0x562b55220f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.856+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5521cd30 0x562b552229d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.856+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1b92e0 con 0x562b55097e40 2024-05-27T08:08:00.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.856+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1b6b70 con 0x562b5521cd30 2024-05-27T08:08:00.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.856+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1b49e0 con 0x562b55216db0 2024-05-27T08:08:00.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.857+0000 7fea61800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b5521e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.857+0000 7fea61800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b5521e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37586/0 (socket says 172.21.15.63:37586) 2024-05-27T08:08:00.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.857+0000 7fea61800700 1 -- 172.21.15.63:0/1628648319 learned_addr learned my addr 172.21.15.63:0/1628648319 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:00.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.857+0000 7fea62200700 1 --2- 172.21.15.63:0/1628648319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5521cd30 0x562b552229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.857+0000 7fea60e00700 1 --2- 172.21.15.63:0/1628648319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b55216db0 0x562b55220f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.857+0000 7fea61800700 1 -- 172.21.15.63:0/1628648319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b55216db0 msgr2=0x562b55220f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:00.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.857+0000 7fea61800700 1 --2- 172.21.15.63:0/1628648319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b55216db0 0x562b55220f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.857+0000 7fea61800700 1 -- 172.21.15.63:0/1628648319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5521cd30 msgr2=0x562b552229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:00.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.857+0000 7fea61800700 1 --2- 172.21.15.63:0/1628648319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5521cd30 0x562b552229d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.857+0000 7fea61800700 1 -- 172.21.15.63:0/1628648319 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b5521eaa0 con 0x562b55097e40 2024-05-27T08:08:00.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.858+0000 7fea61800700 1 --2- 172.21.15.63:0/1628648319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b5521e150 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x562b5521fdb0 tx=0x7fea50013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22dfa56b064c3900 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:00.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.858+0000 7fea4e000700 1 -- 172.21.15.63:0/1628648319 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50011bf0 con 0x562b55097e40 2024-05-27T08:08:00.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.858+0000 7fea4e000700 1 -- 172.21.15.63:0/1628648319 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea50011d90 con 0x562b55097e40 2024-05-27T08:08:00.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.858+0000 7fea4e000700 1 -- 172.21.15.63:0/1628648319 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50027280 con 0x562b55097e40 2024-05-27T08:08:00.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.859+0000 7fea72d2eb80 1 -- 172.21.15.63:0/1628648319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 msgr2=0x562b5521e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:00.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.859+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/1628648319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b5521e150 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x562b5521fdb0 tx=0x7fea50013000 comp rx=0 tx=0).stop 2024-05-27T08:08:00.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.860+0000 7fea72d2eb80 1 -- 172.21.15.63:0/1628648319 shutdown_connections 2024-05-27T08:08:00.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.860+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/1628648319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b5521cd30 0x562b552229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.860+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/1628648319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b5521e150 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.860+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/1628648319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b55216db0 0x562b55220f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.860+0000 7fea72d2eb80 1 -- 172.21.15.63:0/1628648319 >> 172.21.15.63:0/1628648319 conn(0x562b550f7fc0 msgr2=0x562b5521ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:00.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.860+0000 7fea72d2eb80 1 -- 172.21.15.63:0/1628648319 shutdown_connections 2024-05-27T08:08:00.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.860+0000 7fea72d2eb80 1 -- 172.21.15.63:0/1628648319 wait complete. 2024-05-27T08:08:00.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.860+0000 7fea72d2eb80 1 -- start start 2024-05-27T08:08:00.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.860+0000 7fea72d2eb80 1 Processor -- start 2024-05-27T08:08:00.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.861+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b55235d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.861+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55216db0 0x562b55236280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.861+0000 7fea72d2eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b5521cd30 0x562b55233250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.861+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1b5030 con 0x562b55097e40 2024-05-27T08:08:00.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.861+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1b64a0 con 0x562b55216db0 2024-05-27T08:08:00.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.861+0000 7fea72d2eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1b5700 con 0x562b5521cd30 2024-05-27T08:08:00.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.861+0000 7fea72d2eb80 10 client.0 _ll_register_callbacks cb 0x562b550dec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:00.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea61800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b55235d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea61800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b55235d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37588/0 (socket says 172.21.15.63:37588) 2024-05-27T08:08:00.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea61800700 1 -- 172.21.15.63:0/2763301047 learned_addr learned my addr 172.21.15.63:0/2763301047 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:00.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea60e00700 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55216db0 0x562b55236280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea62200700 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b5521cd30 0x562b55233250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea61800700 1 -- 172.21.15.63:0/2763301047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b5521cd30 msgr2=0x562b55233250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:00.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea61800700 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b5521cd30 0x562b55233250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea61800700 1 -- 172.21.15.63:0/2763301047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55216db0 msgr2=0x562b55236280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:00.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea61800700 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55216db0 0x562b55236280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea61800700 1 -- 172.21.15.63:0/2763301047 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b552339f0 con 0x562b55097e40 2024-05-27T08:08:00.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.862+0000 7fea60e00700 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55216db0 0x562b55236280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:00.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.863+0000 7fea61800700 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b55235d40 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fea50005350 tx=0x7fea50011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c1fea5d762fa78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:00.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.863+0000 7fea43400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:00.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.863+0000 7fea43400700 1 -- 172.21.15.63:0/2763301047 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50034020 con 0x562b55097e40 2024-05-27T08:08:00.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.863+0000 7fea43400700 1 -- 172.21.15.63:0/2763301047 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fea50025040 con 0x562b55097e40 2024-05-27T08:08:00.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.863+0000 7fea72d2eb80 1 -- 172.21.15.63:0/2763301047 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea50011d90 con 0x562b55097e40 2024-05-27T08:08:00.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.863+0000 7fea43400700 1 -- 172.21.15.63:0/2763301047 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fea500056c0 con 0x562b55097e40 2024-05-27T08:08:00.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.863+0000 7fea72d2eb80 10 client.21999 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:00.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.863+0000 7fea72d2eb80 1 -- 172.21.15.63:0/2763301047 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562b5529a990 con 0x562b55097e40 2024-05-27T08:08:00.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.864+0000 7fea43400700 1 -- 172.21.15.63:0/2763301047 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea50011d90 con 0x562b55097e40 2024-05-27T08:08:00.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.864+0000 7fea43400700 1 -- 172.21.15.63:0/2763301047 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fea5001e5d0 con 0x562b55097e40 2024-05-27T08:08:00.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.864+0000 7fea72d2eb80 10 client.21999 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:00.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.864+0000 7fea72d2eb80 10 client.21999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:00.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.864+0000 7fea72d2eb80 20 client.21999 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:00.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.864+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562b5529b000 0x562b5529d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:00.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.864+0000 7fea72d2eb80 4 client.21999 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:08:00.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.864+0000 7fea72d2eb80 1 -- 172.21.15.63:0/2763301047 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b55140570 con 0x562b5529b000 2024-05-27T08:08:00.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.865+0000 7fea60e00700 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562b5529b000 0x562b5529d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:00.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.866+0000 7fea60e00700 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562b5529b000 0x562b5529d420 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c182433d98133f42 server_cookie=cd2eae1eb29b5345 in_seq=0 out_seq=0 2024-05-27T08:08:00.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.866+0000 7fea43400700 10 client.21999 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:00.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea43400700 1 -- 172.21.15.63:0/2763301047 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562b55140570 con 0x562b5529b000 2024-05-27T08:08:00.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea43400700 10 client.21999 handle_command_reply: tid=0 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 client.21999 shutdown 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 -- 172.21.15.63:0/2763301047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 msgr2=0x562b55235d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b55235d40 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fea50005350 tx=0x7fea50011d20 comp rx=0 tx=0).stop 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 -- 172.21.15.63:0/2763301047 shutdown_connections 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562b5529b000 0x562b5529d420 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b5521cd30 0x562b55233250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562b55097e40 0x562b55235d40 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 --2- 172.21.15.63:0/2763301047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b55216db0 0x562b55236280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 -- 172.21.15.63:0/2763301047 >> 172.21.15.63:0/2763301047 conn(0x562b550f7fc0 msgr2=0x562b55220c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea43400700 0 client.21999 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 -- 172.21.15.63:0/2763301047 shutdown_connections 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 1 -- 172.21.15.63:0/2763301047 wait complete. 2024-05-27T08:08:00.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.867+0000 7fea72d2eb80 20 client.21999 trim_cache size 0 max 16384 2024-05-27T08:08:00.870 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:08:00.870 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:08:00.870 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:08:00.870 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:08:00.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.868+0000 7fea64200700 1 -- 172.21.15.63:0/4033380695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea440b2a70 msgr2=0x7fea440b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:00.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.868+0000 7fea64200700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea440b2a70 0x7fea440b4f80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fea50005950 tx=0x7fea5000b410 comp rx=0 tx=0).stop 2024-05-27T08:08:00.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.868+0000 7fea64200700 1 -- 172.21.15.63:0/4033380695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c110980 msgr2=0x7fea5c1a5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:00.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.868+0000 7fea64200700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c110980 0x7fea5c1a5b90 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7fea5400b070 tx=0x7fea54004d50 comp rx=0 tx=0).stop 2024-05-27T08:08:00.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.869+0000 7fea64200700 1 -- 172.21.15.63:0/4033380695 shutdown_connections 2024-05-27T08:08:00.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.869+0000 7fea64200700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea440b2a70 0x7fea440b4f80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.869+0000 7fea64200700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea5c110980 0x7fea5c1a5b90 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.869+0000 7fea64200700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea5c10a950 0x7fea5c1a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.869+0000 7fea64200700 1 --2- 172.21.15.63:0/4033380695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c10b2a0 0x7fea5c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:00.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.869+0000 7fea64200700 1 -- 172.21.15.63:0/4033380695 >> 172.21.15.63:0/4033380695 conn(0x7fea5c076d20 msgr2=0x7fea5c1088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:00.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.869+0000 7fea64200700 1 -- 172.21.15.63:0/4033380695 shutdown_connections 2024-05-27T08:08:00.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:00.869+0000 7fea64200700 1 -- 172.21.15.63:0/4033380695 wait complete. 2024-05-27T08:08:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.378+0000 7fd94be00640 1 -- 172.21.15.63:0/804655227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd94c10d940 msgr2=0x7fd94c10fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.378+0000 7fd94be00640 1 --2- 172.21.15.63:0/804655227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd94c10d940 0x7fd94c10fd60 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fd938009900 tx=0x7fd93802a410 comp rx=0 tx=0).stop 2024-05-27T08:08:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.378+0000 7fd94be00640 1 -- 172.21.15.63:0/804655227 shutdown_connections 2024-05-27T08:08:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.378+0000 7fd94be00640 1 --2- 172.21.15.63:0/804655227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c1102a0 0x7fd94c112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.378+0000 7fd94be00640 1 --2- 172.21.15.63:0/804655227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd94c10d940 0x7fd94c10fd60 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.378+0000 7fd94be00640 1 --2- 172.21.15.63:0/804655227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd94c076770 0x7fd94c076b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.378+0000 7fd94be00640 1 -- 172.21.15.63:0/804655227 >> 172.21.15.63:0/804655227 conn(0x7fd94c0ff170 msgr2=0x7fd94c1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.378+0000 7fd94be00640 1 -- 172.21.15.63:0/804655227 shutdown_connections 2024-05-27T08:08:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.378+0000 7fd94be00640 1 -- 172.21.15.63:0/804655227 wait complete. 2024-05-27T08:08:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.379+0000 7fd94be00640 1 Processor -- start 2024-05-27T08:08:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.380+0000 7fd94be00640 1 -- start start 2024-05-27T08:08:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.380+0000 7fd94be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd94c076770 0x7fd94c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.380+0000 7fd94be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c10d940 0x7fd94c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.380+0000 7fd94be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd94c1102a0 0x7fd94c1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.380+0000 7fd94be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c115d90 con 0x7fd94c1102a0 2024-05-27T08:08:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.380+0000 7fd94be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c115c10 con 0x7fd94c076770 2024-05-27T08:08:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.380+0000 7fd94be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c115f10 con 0x7fd94c10d940 2024-05-27T08:08:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.380+0000 7fd94b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd94c1102a0 0x7fd94c1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.381+0000 7fd94b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd94c1102a0 0x7fd94c1a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37608/0 (socket says 172.21.15.63:37608) 2024-05-27T08:08:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.381+0000 7fd94b400640 1 -- 172.21.15.63:0/639192666 learned_addr learned my addr 172.21.15.63:0/639192666 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.381+0000 7fd94aa00640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd94c076770 0x7fd94c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.381+0000 7fd94a000640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c10d940 0x7fd94c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.381+0000 7fd94b400640 1 -- 172.21.15.63:0/639192666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c10d940 msgr2=0x7fd94c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.381+0000 7fd94b400640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c10d940 0x7fd94c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.381+0000 7fd94b400640 1 -- 172.21.15.63:0/639192666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd94c076770 msgr2=0x7fd94c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.381+0000 7fd94b400640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd94c076770 0x7fd94c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.381+0000 7fd94b400640 1 -- 172.21.15.63:0/639192666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c1a15d0 con 0x7fd94c1102a0 2024-05-27T08:08:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.382+0000 7fd94b400640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd94c1102a0 0x7fd94c1a0f90 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fd94400eaa0 tx=0x7fd94400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.382+0000 7fd943400640 1 -- 172.21.15.63:0/639192666 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94400cdb0 con 0x7fd94c1102a0 2024-05-27T08:08:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.382+0000 7fd943400640 1 -- 172.21.15.63:0/639192666 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd944005020 con 0x7fd94c1102a0 2024-05-27T08:08:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.382+0000 7fd94be00640 1 -- 172.21.15.63:0/639192666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c1a18c0 con 0x7fd94c1102a0 2024-05-27T08:08:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.382+0000 7fd943400640 1 -- 172.21.15.63:0/639192666 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440116d0 con 0x7fd94c1102a0 2024-05-27T08:08:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.382+0000 7fd94be00640 1 -- 172.21.15.63:0/639192666 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c1a9240 con 0x7fd94c1102a0 2024-05-27T08:08:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.386+0000 7fd958b77740 1 Processor -- start 2024-05-27T08:08:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.387+0000 7fd943400640 1 -- 172.21.15.63:0/639192666 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd9440051d0 con 0x7fd94c1102a0 2024-05-27T08:08:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.387+0000 7fd958b77740 1 -- start start 2024-05-27T08:08:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.387+0000 7fd958b77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343631b0 0x556d34705a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.387+0000 7fd958b77740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d34705fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.387+0000 7fd958b77740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d343424d0 0x556d347064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.387+0000 7fd958b77740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556d34676b90 con 0x556d3433a2f0 2024-05-27T08:08:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.387+0000 7fd958b77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d34666730 con 0x556d343631b0 2024-05-27T08:08:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.387+0000 7fd958b77740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556d3466a770 con 0x556d343424d0 2024-05-27T08:08:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.388+0000 7fd94aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d34705fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.388+0000 7fd94aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d34705fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37616/0 (socket says 172.21.15.63:37616) 2024-05-27T08:08:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.388+0000 7fd94aa00640 1 -- 172.21.15.63:0/3813338158 learned_addr learned my addr 172.21.15.63:0/3813338158 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.388+0000 7fd94a000640 1 --2- 172.21.15.63:0/3813338158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343631b0 0x556d34705a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.388+0000 7fd94a000640 1 --2- 172.21.15.63:0/3813338158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d343424d0 0x556d347064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.388+0000 7fd94aa00640 1 -- 172.21.15.63:0/3813338158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d343424d0 msgr2=0x556d347064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.388+0000 7fd94aa00640 1 --2- 172.21.15.63:0/3813338158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d343424d0 0x556d347064e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.388+0000 7fd94aa00640 1 -- 172.21.15.63:0/3813338158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343631b0 msgr2=0x556d34705a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.388+0000 7fd94aa00640 1 --2- 172.21.15.63:0/3813338158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343631b0 0x556d34705a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.388+0000 7fd94aa00640 1 -- 172.21.15.63:0/3813338158 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d345ff8b0 con 0x556d3433a2f0 2024-05-27T08:08:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.389+0000 7fd943400640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9300b2950 0x7fd9300b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.389+0000 7fd94aa00640 1 --2- 172.21.15.63:0/3813338158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d34705fa0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00bd50 tx=0x7fd93c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aeafd7ba7a52e1d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.389+0000 7fd943400640 1 -- 172.21.15.63:0/639192666 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fd9440e2b80 con 0x7fd94c1102a0 2024-05-27T08:08:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.389+0000 7fd94a000640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9300b2950 0x7fd9300b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.389+0000 7fd937e00640 1 -- 172.21.15.63:0/3813338158 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c005220 con 0x556d3433a2f0 2024-05-27T08:08:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.389+0000 7fd937e00640 1 -- 172.21.15.63:0/3813338158 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd93c00be40 con 0x556d3433a2f0 2024-05-27T08:08:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.389+0000 7fd937e00640 1 -- 172.21.15.63:0/3813338158 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0116a0 con 0x556d3433a2f0 2024-05-27T08:08:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.391+0000 7fd958b77740 1 -- 172.21.15.63:0/3813338158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 msgr2=0x556d34705fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.391+0000 7fd958b77740 1 --2- 172.21.15.63:0/3813338158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d34705fa0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00bd50 tx=0x7fd93c0107a0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.391+0000 7fd94a000640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9300b2950 0x7fd9300b4e60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd938006010 tx=0x7fd9380058d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.391+0000 7fd958b77740 1 -- 172.21.15.63:0/3813338158 shutdown_connections 2024-05-27T08:08:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.391+0000 7fd958b77740 1 --2- 172.21.15.63:0/3813338158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d343424d0 0x556d347064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.391+0000 7fd958b77740 1 --2- 172.21.15.63:0/3813338158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d34705fa0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.391+0000 7fd958b77740 1 --2- 172.21.15.63:0/3813338158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343631b0 0x556d34705a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.391+0000 7fd958b77740 1 -- 172.21.15.63:0/3813338158 >> 172.21.15.63:0/3813338158 conn(0x556d34357300 msgr2=0x556d34702210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.391+0000 7fd958b77740 1 -- 172.21.15.63:0/3813338158 shutdown_connections 2024-05-27T08:08:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.391+0000 7fd958b77740 1 -- 172.21.15.63:0/3813338158 wait complete. 2024-05-27T08:08:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.392+0000 7fd958b77740 1 -- start start 2024-05-27T08:08:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.392+0000 7fd958b77740 1 Processor -- start 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd958b77740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d3470fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd958b77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 0x556d34710120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd958b77740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d343631b0 0x556d3471bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd958b77740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556d3434c790 con 0x556d3433a2f0 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd958b77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556d3460ebc0 con 0x556d343424d0 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd958b77740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556d34712b90 con 0x556d343631b0 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd958b77740 10 client.0 _ll_register_callbacks cb 0x556d345ec400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd94aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d3470fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd94aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d3470fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37618/0 (socket says 172.21.15.63:37618) 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd94aa00640 1 -- 172.21.15.63:0/2009464911 learned_addr learned my addr 172.21.15.63:0/2009464911 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd94b400640 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d343631b0 0x556d3471bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.393+0000 7fd94a000640 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 0x556d34710120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd94aa00640 1 -- 172.21.15.63:0/2009464911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d343631b0 msgr2=0x556d3471bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd94aa00640 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d343631b0 0x556d3471bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd94aa00640 1 -- 172.21.15.63:0/2009464911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 msgr2=0x556d34710120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd94aa00640 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 0x556d34710120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd94aa00640 1 -- 172.21.15.63:0/2009464911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d3437b210 con 0x556d3433a2f0 2024-05-27T08:08:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd94a000640 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 0x556d34710120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:08:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd94aa00640 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d3470fbe0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00bd20 tx=0x7fd93c005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4baf38e68d927a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd934c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd934c00640 1 -- 172.21.15.63:0/2009464911 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c004020 con 0x556d3433a2f0 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd958b77740 1 -- 172.21.15.63:0/2009464911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d34678ae0 con 0x556d3433a2f0 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.394+0000 7fd934c00640 1 -- 172.21.15.63:0/2009464911 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fd93c0041c0 con 0x556d3433a2f0 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.395+0000 7fd934c00640 1 -- 172.21.15.63:0/2009464911 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd93c0043c0 con 0x556d3433a2f0 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.395+0000 7fd958b77740 10 client.22011 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.395+0000 7fd958b77740 1 -- 172.21.15.63:0/2009464911 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556d34447620 con 0x556d3433a2f0 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.395+0000 7fd934c00640 1 -- 172.21.15.63:0/2009464911 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556d34678ae0 con 0x556d3433a2f0 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.395+0000 7fd934c00640 1 -- 172.21.15.63:0/2009464911 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd93c003a20 con 0x556d3433a2f0 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.396+0000 7fd958b77740 10 client.22011 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.396+0000 7fd958b77740 10 client.22011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.396+0000 7fd958b77740 20 client.22011 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.396+0000 7fd958b77740 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d34321d80 0x556d34721ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.396+0000 7fd958b77740 4 client.22011 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.396+0000 7fd958b77740 1 -- 172.21.15.63:0/2009464911 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d346722c0 con 0x556d34321d80 2024-05-27T08:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.396+0000 7fd94a000640 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d34321d80 0x556d34721ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.397+0000 7fd94a000640 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d34321d80 0x556d34721ce0 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6ddb2818eb7d1 server_cookie=affb667d84209a7b in_seq=0 out_seq=0 2024-05-27T08:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.397+0000 7fd934c00640 10 client.22011 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.399+0000 7fd934c00640 1 -- 172.21.15.63:0/2009464911 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556d346722c0 con 0x556d34321d80 2024-05-27T08:08:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.399+0000 7fd934c00640 10 client.22011 handle_command_reply: tid=0 2024-05-27T08:08:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.399+0000 7fd958b77740 1 client.22011 shutdown 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 1 -- 172.21.15.63:0/2009464911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 msgr2=0x556d3470fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d3470fbe0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00bd20 tx=0x7fd93c005230 comp rx=0 tx=0).stop 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 1 -- 172.21.15.63:0/2009464911 shutdown_connections 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d34321d80 0x556d34721ce0 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d343631b0 0x556d3471bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 0x556d34710120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 1 --2- 172.21.15.63:0/2009464911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d3433a2f0 0x556d3470fbe0 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 1 -- 172.21.15.63:0/2009464911 >> 172.21.15.63:0/2009464911 conn(0x556d34357300 msgr2=0x556d34707780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd934c00640 0 client.22011 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 1 -- 172.21.15.63:0/2009464911 shutdown_connections 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 1 -- 172.21.15.63:0/2009464911 wait complete. 2024-05-27T08:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.400+0000 7fd958b77740 20 client.22011 trim_cache size 0 max 16384 2024-05-27T08:08:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.424+0000 7fd958b77740 1 Processor -- start 2024-05-27T08:08:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.424+0000 7fd958b77740 1 -- start start 2024-05-27T08:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd958b77740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d3433a2f0 0x556d3470b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd958b77740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343424d0 0x556d3470f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd958b77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343631b0 0x556d3471bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd958b77740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b5b30 con 0x556d343424d0 2024-05-27T08:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd958b77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b3400 con 0x556d343631b0 2024-05-27T08:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd958b77740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b1270 con 0x556d3433a2f0 2024-05-27T08:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd94a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343424d0 0x556d3470f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd94a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343424d0 0x556d3470f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37634/0 (socket says 172.21.15.63:37634) 2024-05-27T08:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd94a000640 1 -- 172.21.15.63:0/2585807031 learned_addr learned my addr 172.21.15.63:0/2585807031 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd94aa00640 1 --2- 172.21.15.63:0/2585807031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d3433a2f0 0x556d3470b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.425+0000 7fd94b400640 1 --2- 172.21.15.63:0/2585807031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343631b0 0x556d3471bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.426+0000 7fd94a000640 1 -- 172.21.15.63:0/2585807031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d3433a2f0 msgr2=0x556d3470b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.426+0000 7fd94a000640 1 --2- 172.21.15.63:0/2585807031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d3433a2f0 0x556d3470b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.426+0000 7fd94a000640 1 -- 172.21.15.63:0/2585807031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343631b0 msgr2=0x556d3471bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.426+0000 7fd94a000640 1 --2- 172.21.15.63:0/2585807031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343631b0 0x556d3471bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.426+0000 7fd94a000640 1 -- 172.21.15.63:0/2585807031 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d34321fb0 con 0x556d343424d0 2024-05-27T08:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.426+0000 7fd94a000640 1 --2- 172.21.15.63:0/2585807031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343424d0 0x556d3470f470 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fd94c050b30 tx=0x7fd938009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c91b8233372d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.426+0000 7fd937400640 1 -- 172.21.15.63:0/2585807031 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938045070 con 0x556d343424d0 2024-05-27T08:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.427+0000 7fd937400640 1 -- 172.21.15.63:0/2585807031 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9380044d0 con 0x556d343424d0 2024-05-27T08:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.427+0000 7fd937400640 1 -- 172.21.15.63:0/2585807031 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938035040 con 0x556d343424d0 2024-05-27T08:08:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.428+0000 7fd958b77740 1 -- 172.21.15.63:0/2585807031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343424d0 msgr2=0x556d3470f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.428+0000 7fd958b77740 1 --2- 172.21.15.63:0/2585807031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343424d0 0x556d3470f470 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fd94c050b30 tx=0x7fd938009260 comp rx=0 tx=0).stop 2024-05-27T08:08:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.429+0000 7fd958b77740 1 -- 172.21.15.63:0/2585807031 shutdown_connections 2024-05-27T08:08:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.429+0000 7fd958b77740 1 --2- 172.21.15.63:0/2585807031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343631b0 0x556d3471bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.429+0000 7fd958b77740 1 --2- 172.21.15.63:0/2585807031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343424d0 0x556d3470f470 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.429+0000 7fd958b77740 1 --2- 172.21.15.63:0/2585807031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d3433a2f0 0x556d3470b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.429+0000 7fd958b77740 1 -- 172.21.15.63:0/2585807031 >> 172.21.15.63:0/2585807031 conn(0x556d34357300 msgr2=0x556d347063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.429+0000 7fd958b77740 1 -- 172.21.15.63:0/2585807031 shutdown_connections 2024-05-27T08:08:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.429+0000 7fd958b77740 1 -- 172.21.15.63:0/2585807031 wait complete. 2024-05-27T08:08:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.430+0000 7fd958b77740 1 -- start start 2024-05-27T08:08:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.430+0000 7fd958b77740 1 Processor -- start 2024-05-27T08:08:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd958b77740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d3433a2f0 0x556d34712de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd958b77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 0x556d34713320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd958b77740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343631b0 0x556d34713860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd958b77740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b18c0 con 0x556d343631b0 2024-05-27T08:08:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd958b77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b2d30 con 0x556d343424d0 2024-05-27T08:08:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd958b77740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1b1f90 con 0x556d3433a2f0 2024-05-27T08:08:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd958b77740 10 client.0 _ll_register_callbacks cb 0x7fd938009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd94b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343631b0 0x556d34713860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd94b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343631b0 0x556d34713860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37644/0 (socket says 172.21.15.63:37644) 2024-05-27T08:08:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd94b400640 1 -- 172.21.15.63:0/985388854 learned_addr learned my addr 172.21.15.63:0/985388854 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.431+0000 7fd94a000640 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 0x556d34713320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.432+0000 7fd94aa00640 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d3433a2f0 0x556d34712de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.432+0000 7fd94b400640 1 -- 172.21.15.63:0/985388854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d3433a2f0 msgr2=0x556d34712de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.432+0000 7fd94b400640 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d3433a2f0 0x556d34712de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.432+0000 7fd94b400640 1 -- 172.21.15.63:0/985388854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 msgr2=0x556d34713320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.432+0000 7fd94b400640 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 0x556d34713320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.432+0000 7fd94b400640 1 -- 172.21.15.63:0/985388854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d34788380 con 0x556d343631b0 2024-05-27T08:08:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.432+0000 7fd94b400640 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343631b0 0x556d34713860 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fd944005bc0 tx=0x7fd9440a51d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08303a690c10a78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.432+0000 7fd934c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.433+0000 7fd934c00640 1 -- 172.21.15.63:0/985388854 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440c3060 con 0x556d343631b0 2024-05-27T08:08:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.433+0000 7fd934c00640 1 -- 172.21.15.63:0/985388854 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 9483+0+0 (secure 0 0 0) 0x7fd9440cc280 con 0x556d343631b0 2024-05-27T08:08:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.433+0000 7fd958b77740 1 -- 172.21.15.63:0/985388854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9380044d0 con 0x556d343631b0 2024-05-27T08:08:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.433+0000 7fd934c00640 1 -- 172.21.15.63:0/985388854 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd94401f040 con 0x556d343631b0 2024-05-27T08:08:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.433+0000 7fd958b77740 10 client.22017 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.433+0000 7fd958b77740 1 -- 172.21.15.63:0/985388854 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556d34789670 con 0x556d343631b0 2024-05-27T08:08:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.433+0000 7fd934c00640 1 -- 172.21.15.63:0/985388854 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9380044d0 con 0x556d343631b0 2024-05-27T08:08:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.434+0000 7fd934c00640 1 -- 172.21.15.63:0/985388854 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd94400f040 con 0x556d343631b0 2024-05-27T08:08:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.434+0000 7fd958b77740 10 client.22017 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.434+0000 7fd958b77740 10 client.22017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.434+0000 7fd958b77740 20 client.22017 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.434+0000 7fd958b77740 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d342f71c0 0x556d3478bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.434+0000 7fd958b77740 4 client.22017 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:08:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.434+0000 7fd958b77740 1 -- 172.21.15.63:0/985388854 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d345ec4b0 con 0x556d342f71c0 2024-05-27T08:08:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.434+0000 7fd94aa00640 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d342f71c0 0x556d3478bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.435+0000 7fd94aa00640 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d342f71c0 0x556d3478bdd0 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f3d50c46825dfb server_cookie=308479b6d09488f7 in_seq=0 out_seq=0 2024-05-27T08:08:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.435+0000 7fd934c00640 10 client.22017 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.480+0000 7fd934c00640 1 -- 172.21.15.63:0/985388854 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556d345ec4b0 con 0x556d342f71c0 2024-05-27T08:08:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.480+0000 7fd934c00640 10 client.22017 handle_command_reply: tid=0 2024-05-27T08:08:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.480+0000 7fd958b77740 1 client.22017 shutdown 2024-05-27T08:08:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.480+0000 7fd958b77740 1 -- 172.21.15.63:0/985388854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343631b0 msgr2=0x556d34713860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.480+0000 7fd958b77740 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343631b0 0x556d34713860 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fd944005bc0 tx=0x7fd9440a51d0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.481+0000 7fd958b77740 1 -- 172.21.15.63:0/985388854 shutdown_connections 2024-05-27T08:08:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.481+0000 7fd958b77740 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556d342f71c0 0x556d3478bdd0 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.481+0000 7fd958b77740 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d343631b0 0x556d34713860 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.481+0000 7fd958b77740 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556d343424d0 0x556d34713320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.481+0000 7fd958b77740 1 --2- 172.21.15.63:0/985388854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556d3433a2f0 0x556d34712de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.481+0000 7fd934c00640 0 client.22017 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.481+0000 7fd958b77740 1 -- 172.21.15.63:0/985388854 >> 172.21.15.63:0/985388854 conn(0x556d34357300 msgr2=0x556d34711290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.481+0000 7fd958b77740 1 -- 172.21.15.63:0/985388854 shutdown_connections 2024-05-27T08:08:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.481+0000 7fd958b77740 1 -- 172.21.15.63:0/985388854 wait complete. 2024-05-27T08:08:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.481+0000 7fd958b77740 20 client.22017 trim_cache size 0 max 16384 2024-05-27T08:08:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:08:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:08:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:08:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.501+0000 7fd94be00640 1 -- 172.21.15.63:0/639192666 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9300b2950 msgr2=0x7fd9300b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.501+0000 7fd94be00640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9300b2950 0x7fd9300b4e60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd938006010 tx=0x7fd9380058d0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.501+0000 7fd94be00640 1 -- 172.21.15.63:0/639192666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd94c1102a0 msgr2=0x7fd94c1a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.501+0000 7fd94be00640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd94c1102a0 0x7fd94c1a0f90 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fd94400eaa0 tx=0x7fd94400ef60 comp rx=0 tx=0).stop 2024-05-27T08:08:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.502+0000 7fd94be00640 1 -- 172.21.15.63:0/639192666 shutdown_connections 2024-05-27T08:08:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.502+0000 7fd94be00640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd9300b2950 0x7fd9300b4e60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.502+0000 7fd94be00640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd94c1102a0 0x7fd94c1a0f90 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.502+0000 7fd94be00640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd94c10d940 0x7fd94c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.502+0000 7fd94be00640 1 --2- 172.21.15.63:0/639192666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd94c076770 0x7fd94c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.502+0000 7fd94be00640 1 -- 172.21.15.63:0/639192666 >> 172.21.15.63:0/639192666 conn(0x7fd94c0ff170 msgr2=0x7fd94c110770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.502+0000 7fd94be00640 1 -- 172.21.15.63:0/639192666 shutdown_connections 2024-05-27T08:08:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:01.502+0000 7fd94be00640 1 -- 172.21.15.63:0/639192666 wait complete. 2024-05-27T08:08:02.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:01 smithi086 ceph-mon[34687]: pgmap v1091: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:02.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:01 smithi149 ceph-mon[35408]: pgmap v1091: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:02.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:01 smithi063 ceph-mon[31613]: pgmap v1091: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:02.967 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:08:02.967 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:08:03.089 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:02 smithi063 ceph-mon[31613]: osdmap e149: 12 total, 12 up, 12 in 2024-05-27T08:08:03.090 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:02 smithi063 ceph-mon[31613]: pgmap v1093: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 28 op/s 2024-05-27T08:08:03.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:02 smithi086 ceph-mon[34687]: osdmap e149: 12 total, 12 up, 12 in 2024-05-27T08:08:03.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:02 smithi086 ceph-mon[34687]: pgmap v1093: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 28 op/s 2024-05-27T08:08:03.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:02 smithi149 ceph-mon[35408]: osdmap e149: 12 total, 12 up, 12 in 2024-05-27T08:08:03.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:02 smithi149 ceph-mon[35408]: pgmap v1093: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 28 op/s 2024-05-27T08:08:03.538 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:08:05.915 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:05 smithi063 ceph-mon[31613]: pgmap v1094: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 28 op/s 2024-05-27T08:08:05.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:05 smithi086 ceph-mon[34687]: pgmap v1094: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 28 op/s 2024-05-27T08:08:05.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:05 smithi149 ceph-mon[35408]: pgmap v1094: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 28 op/s 2024-05-27T08:08:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.029+0000 7f56faa00700 1 -- 172.21.15.63:0/3602451141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56f4105de0 msgr2=0x7f56f410e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.029+0000 7f56faa00700 1 --2- 172.21.15.63:0/3602451141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56f4105de0 0x7f56f410e330 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009990 tx=0x7f56e0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.029+0000 7f56faa00700 1 -- 172.21.15.63:0/3602451141 shutdown_connections 2024-05-27T08:08:07.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.029+0000 7f56faa00700 1 --2- 172.21.15.63:0/3602451141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f410e900 0x7f56f4110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.029+0000 7f56faa00700 1 --2- 172.21.15.63:0/3602451141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f4105490 0x7f56f41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.029+0000 7f56faa00700 1 --2- 172.21.15.63:0/3602451141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56f4105de0 0x7f56f410e330 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.029+0000 7f56faa00700 1 -- 172.21.15.63:0/3602451141 >> 172.21.15.63:0/3602451141 conn(0x7f56f40fedb0 msgr2=0x7f56f41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:07.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.029+0000 7f56faa00700 1 -- 172.21.15.63:0/3602451141 shutdown_connections 2024-05-27T08:08:07.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.029+0000 7f56faa00700 1 -- 172.21.15.63:0/3602451141 wait complete. 2024-05-27T08:08:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.030+0000 7f56faa00700 1 Processor -- start 2024-05-27T08:08:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.030+0000 7f56faa00700 1 -- start start 2024-05-27T08:08:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.031+0000 7f56faa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f4105490 0x7f56f419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.031+0000 7f56faa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f4105de0 0x7f56f419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.031+0000 7f56faa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56f410e900 0x7f56f41a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.031+0000 7f56faa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56f41142f0 con 0x7f56f410e900 2024-05-27T08:08:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.031+0000 7f56faa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56f4114170 con 0x7f56f4105490 2024-05-27T08:08:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.031+0000 7f56faa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56f4114470 con 0x7f56f4105de0 2024-05-27T08:08:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.031+0000 7f56f8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56f410e900 0x7f56f41a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.031+0000 7f56f8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56f410e900 0x7f56f41a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37652/0 (socket says 172.21.15.63:37652) 2024-05-27T08:08:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.031+0000 7f56f8a00700 1 -- 172.21.15.63:0/1953523028 learned_addr learned my addr 172.21.15.63:0/1953523028 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.032+0000 7f56f3e00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f4105490 0x7f56f419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.032+0000 7f56f3400700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f4105de0 0x7f56f419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.032+0000 7f56f8a00700 1 -- 172.21.15.63:0/1953523028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f4105de0 msgr2=0x7f56f419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.032+0000 7f56f8a00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f4105de0 0x7f56f419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.032+0000 7f56f8a00700 1 -- 172.21.15.63:0/1953523028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f4105490 msgr2=0x7f56f419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.032+0000 7f56f8a00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f4105490 0x7f56f419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.032+0000 7f56f8a00700 1 -- 172.21.15.63:0/1953523028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f41a1dd0 con 0x7f56f410e900 2024-05-27T08:08:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.033+0000 7f56f8a00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56f410e900 0x7f56f41a1730 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f56ec007ee0 tx=0x7f56ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:07.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.033+0000 7f56f0c00700 1 -- 172.21.15.63:0/1953523028 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec004bb0 con 0x7f56f410e900 2024-05-27T08:08:07.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.033+0000 7f56faa00700 1 -- 172.21.15.63:0/1953523028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f41a20c0 con 0x7f56f410e900 2024-05-27T08:08:07.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.033+0000 7f56f0c00700 1 -- 172.21.15.63:0/1953523028 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56ec004d50 con 0x7f56f410e900 2024-05-27T08:08:07.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.033+0000 7f56f0c00700 1 -- 172.21.15.63:0/1953523028 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0058b0 con 0x7f56f410e900 2024-05-27T08:08:07.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.033+0000 7f56faa00700 1 -- 172.21.15.63:0/1953523028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f41a99e0 con 0x7f56f410e900 2024-05-27T08:08:07.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.035+0000 7f56faa00700 1 -- 172.21.15.63:0/1953523028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f406bea0 con 0x7f56f410e900 2024-05-27T08:08:07.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.037+0000 7f56f0c00700 1 -- 172.21.15.63:0/1953523028 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f56ec01a020 con 0x7f56f410e900 2024-05-27T08:08:07.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.040+0000 7f56f0c00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56d80b29f0 0x7f56d80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:07.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.040+0000 7f56f0c00700 1 -- 172.21.15.63:0/1953523028 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 9571+0+0 (secure 0 0 0) 0x7f56ec0e20a0 con 0x7f56f410e900 2024-05-27T08:08:07.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.040+0000 7f56f3e00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56d80b29f0 0x7f56d80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:07.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.041+0000 7f56f3e00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56d80b29f0 0x7f56d80b4f00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005950 tx=0x7f56e400b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:07.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.045+0000 7f56f0c00700 1 -- 172.21.15.63:0/1953523028 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56ec010070 con 0x7f56f410e900 2024-05-27T08:08:07.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.358+0000 7f56faa00700 1 -- 172.21.15.63:0/1953523028 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56f402e5e0 con 0x7f56f410e900 2024-05-27T08:08:07.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.360+0000 7f56f0c00700 1 -- 172.21.15.63:0/1953523028 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7f56ec0a9660 con 0x7f56f410e900 2024-05-27T08:08:07.362 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:08:07.362 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:08:07.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.363+0000 7f56faa00700 1 -- 172.21.15.63:0/1953523028 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56d80b29f0 msgr2=0x7f56d80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:07.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.363+0000 7f56faa00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56d80b29f0 0x7f56d80b4f00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005950 tx=0x7f56e400b410 comp rx=0 tx=0).stop 2024-05-27T08:08:07.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.363+0000 7f56faa00700 1 -- 172.21.15.63:0/1953523028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56f410e900 msgr2=0x7f56f41a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:07.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.363+0000 7f56faa00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56f410e900 0x7f56f41a1730 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f56ec007ee0 tx=0x7f56ec00d3b0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.364+0000 7f56faa00700 1 -- 172.21.15.63:0/1953523028 shutdown_connections 2024-05-27T08:08:07.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.364+0000 7f56faa00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56d80b29f0 0x7f56d80b4f00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.364+0000 7f56faa00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56f410e900 0x7f56f41a1730 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.364+0000 7f56faa00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f4105490 0x7f56f419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.364+0000 7f56faa00700 1 --2- 172.21.15.63:0/1953523028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56f4105de0 0x7f56f419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:07.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.364+0000 7f56faa00700 1 -- 172.21.15.63:0/1953523028 >> 172.21.15.63:0/1953523028 conn(0x7f56f40fedb0 msgr2=0x7f56f41031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:07.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.364+0000 7f56faa00700 1 -- 172.21.15.63:0/1953523028 shutdown_connections 2024-05-27T08:08:07.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:07.364+0000 7f56faa00700 1 -- 172.21.15.63:0/1953523028 wait complete. 2024-05-27T08:08:07.366 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:08:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:07 smithi086 ceph-mon[34687]: pgmap v1095: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:08:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:07 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1953523028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:08:07.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:07 smithi149 ceph-mon[35408]: pgmap v1095: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:08:07.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:07 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1953523028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:08:07.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:07 smithi063 ceph-mon[31613]: pgmap v1095: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:08:07.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:07 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1953523028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:08:08.235 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:08:08.795 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:08:09.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:09 smithi063 ceph-mon[31613]: pgmap v1096: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:08:09.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:09 smithi086 ceph-mon[34687]: pgmap v1096: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:08:09.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:09 smithi149 ceph-mon[35408]: pgmap v1096: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:08:11.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.502+0000 7fc74e800700 1 -- 172.21.15.63:0/57819162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc748075af0 msgr2=0x7fc748075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:11.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.502+0000 7fc74e800700 1 --2- 172.21.15.63:0/57819162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc748075af0 0x7fc748075f00 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fc738009990 tx=0x7fc738009ca0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.502+0000 7fc74e800700 1 -- 172.21.15.63:0/57819162 shutdown_connections 2024-05-27T08:08:11.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.502+0000 7fc74e800700 1 --2- 172.21.15.63:0/57819162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc748110bf0 0x7fc748113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.502+0000 7fc74e800700 1 --2- 172.21.15.63:0/57819162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc748075af0 0x7fc748075f00 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.502+0000 7fc74e800700 1 --2- 172.21.15.63:0/57819162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc748076440 0x7fc7481106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.502+0000 7fc74e800700 1 -- 172.21.15.63:0/57819162 >> 172.21.15.63:0/57819162 conn(0x7fc748100030 msgr2=0x7fc748102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:11.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.502+0000 7fc74e800700 1 -- 172.21.15.63:0/57819162 shutdown_connections 2024-05-27T08:08:11.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.502+0000 7fc74e800700 1 -- 172.21.15.63:0/57819162 wait complete. 2024-05-27T08:08:11.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.503+0000 7fc74e800700 1 Processor -- start 2024-05-27T08:08:11.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.503+0000 7fc74e800700 1 -- start start 2024-05-27T08:08:11.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.504+0000 7fc74e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc748075af0 0x7fc74819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.504+0000 7fc74e800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc748076440 0x7fc74819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.504+0000 7fc74e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc748110bf0 0x7fc7481a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.504+0000 7fc74e800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc748116580 con 0x7fc748075af0 2024-05-27T08:08:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.504+0000 7fc74e800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc748116400 con 0x7fc748110bf0 2024-05-27T08:08:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.504+0000 7fc74e800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc748116700 con 0x7fc748076440 2024-05-27T08:08:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.504+0000 7fc747400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc748075af0 0x7fc74819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.504+0000 7fc747400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc748075af0 0x7fc74819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45054/0 (socket says 172.21.15.63:45054) 2024-05-27T08:08:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.504+0000 7fc747400700 1 -- 172.21.15.63:0/2095951780 learned_addr learned my addr 172.21.15.63:0/2095951780 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.505+0000 7fc746a00700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc748076440 0x7fc74819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.505+0000 7fc747e00700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc748110bf0 0x7fc7481a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.505+0000 7fc747400700 1 -- 172.21.15.63:0/2095951780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc748076440 msgr2=0x7fc74819d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:11.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.505+0000 7fc747400700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc748076440 0x7fc74819d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.505+0000 7fc747400700 1 -- 172.21.15.63:0/2095951780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc748110bf0 msgr2=0x7fc7481a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:11.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.505+0000 7fc747400700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc748110bf0 0x7fc7481a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.505+0000 7fc747400700 1 -- 172.21.15.63:0/2095951780 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7481a1e60 con 0x7fc748075af0 2024-05-27T08:08:11.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.505+0000 7fc747e00700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc748110bf0 0x7fc7481a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:11.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.505+0000 7fc746a00700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc748076440 0x7fc74819d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:11.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.506+0000 7fc747400700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc748075af0 0x7fc74819cf90 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fc7380057c0 tx=0x7fc738009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:11.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.506+0000 7fc73fe00700 1 -- 172.21.15.63:0/2095951780 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738027070 con 0x7fc748075af0 2024-05-27T08:08:11.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.506+0000 7fc74e800700 1 -- 172.21.15.63:0/2095951780 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7481a20f0 con 0x7fc748075af0 2024-05-27T08:08:11.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.506+0000 7fc74e800700 1 -- 172.21.15.63:0/2095951780 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7481a9a70 con 0x7fc748075af0 2024-05-27T08:08:11.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.506+0000 7fc73fe00700 1 -- 172.21.15.63:0/2095951780 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc73800fda0 con 0x7fc748075af0 2024-05-27T08:08:11.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.506+0000 7fc73fe00700 1 -- 172.21.15.63:0/2095951780 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738022c70 con 0x7fc748075af0 2024-05-27T08:08:11.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.511+0000 7fc73fe00700 1 -- 172.21.15.63:0/2095951780 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc73802a070 con 0x7fc748075af0 2024-05-27T08:08:11.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.513+0000 7fc73fe00700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc72c0b2a70 0x7fc72c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.513+0000 7fc73fe00700 1 -- 172.21.15.63:0/2095951780 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7fc7380e4880 con 0x7fc748075af0 2024-05-27T08:08:11.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.513+0000 7fc746a00700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc72c0b2a70 0x7fc72c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.525+0000 7fc746a00700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc72c0b2a70 0x7fc72c0b4f80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc734005950 tx=0x7fc7340058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:11.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.528+0000 7fc75d3c3b80 1 Processor -- start 2024-05-27T08:08:11.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.529+0000 7fc75d3c3b80 1 -- start start 2024-05-27T08:08:11.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.529+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb1e7e40 0x556aeb36c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.529+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb366870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.529+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb366db0 0x556aeb3671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.529+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556aeb294e40 con 0x556aeb36cd30 2024-05-27T08:08:11.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.529+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556aeb2849e0 con 0x556aeb366db0 2024-05-27T08:08:11.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.529+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556aeb288a20 con 0x556aeb1e7e40 2024-05-27T08:08:11.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.530+0000 7fc746a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb366870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.530+0000 7fc746a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb366870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45064/0 (socket says 172.21.15.63:45064) 2024-05-27T08:08:11.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.530+0000 7fc746a00700 1 -- 172.21.15.63:0/3583325 learned_addr learned my addr 172.21.15.63:0/3583325 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:11.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.530+0000 7fc747400700 1 --2- 172.21.15.63:0/3583325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb1e7e40 0x556aeb36c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.530+0000 7fc747e00700 1 --2- 172.21.15.63:0/3583325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb366db0 0x556aeb3671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.531+0000 7fc746a00700 1 -- 172.21.15.63:0/3583325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb1e7e40 msgr2=0x556aeb36c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:11.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.531+0000 7fc746a00700 1 --2- 172.21.15.63:0/3583325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb1e7e40 0x556aeb36c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.531+0000 7fc746a00700 1 -- 172.21.15.63:0/3583325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb366db0 msgr2=0x556aeb3671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:11.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.531+0000 7fc746a00700 1 --2- 172.21.15.63:0/3583325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb366db0 0x556aeb3671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.531+0000 7fc746a00700 1 -- 172.21.15.63:0/3583325 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aeb24d420 con 0x556aeb36cd30 2024-05-27T08:08:11.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.531+0000 7fc747400700 1 --2- 172.21.15.63:0/3583325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb1e7e40 0x556aeb36c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:11.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.531+0000 7fc746a00700 1 --2- 172.21.15.63:0/3583325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb366870 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fc73400b5d0 tx=0x7fc734012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d272a2a07e5e8c40 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:11.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.531+0000 7fc73cc00700 1 -- 172.21.15.63:0/3583325 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734003880 con 0x556aeb36cd30 2024-05-27T08:08:11.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.532+0000 7fc73cc00700 1 -- 172.21.15.63:0/3583325 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc734024070 con 0x556aeb36cd30 2024-05-27T08:08:11.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.532+0000 7fc73cc00700 1 -- 172.21.15.63:0/3583325 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73401fc90 con 0x556aeb36cd30 2024-05-27T08:08:11.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.533+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3583325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 msgr2=0x556aeb366870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:11.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.533+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/3583325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb366870 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fc73400b5d0 tx=0x7fc734012000 comp rx=0 tx=0).stop 2024-05-27T08:08:11.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.534+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3583325 shutdown_connections 2024-05-27T08:08:11.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.534+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/3583325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb366db0 0x556aeb3671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.534+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/3583325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb1e7e40 0x556aeb36c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.534+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/3583325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb366870 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.534+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3583325 >> 172.21.15.63:0/3583325 conn(0x556aeb247fc0 msgr2=0x556aeb3618d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:11.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.534+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3583325 shutdown_connections 2024-05-27T08:08:11.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.534+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3583325 wait complete. 2024-05-27T08:08:11.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.535+0000 7fc75d3c3b80 1 -- start start 2024-05-27T08:08:11.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.535+0000 7fc75d3c3b80 1 Processor -- start 2024-05-27T08:08:11.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb1e7e40 0x556aeb370480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb366db0 0x556aeb3709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb36cd30 0x556aeb370f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556aeb18afa0 con 0x556aeb1e7e40 2024-05-27T08:08:11.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556aeb225760 con 0x556aeb36cd30 2024-05-27T08:08:11.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556aeb373800 con 0x556aeb366db0 2024-05-27T08:08:11.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc75d3c3b80 10 client.0 _ll_register_callbacks cb 0x556aeb22ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:11.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc747400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb1e7e40 0x556aeb370480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc747400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb1e7e40 0x556aeb370480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45066/0 (socket says 172.21.15.63:45066) 2024-05-27T08:08:11.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc747400700 1 -- 172.21.15.63:0/3475241283 learned_addr learned my addr 172.21.15.63:0/3475241283 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:11.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.536+0000 7fc746a00700 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb366db0 0x556aeb3709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.537+0000 7fc747e00700 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb36cd30 0x556aeb370f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.537+0000 7fc747400700 1 -- 172.21.15.63:0/3475241283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb366db0 msgr2=0x556aeb3709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:11.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.537+0000 7fc747400700 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb366db0 0x556aeb3709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.537+0000 7fc747400700 1 -- 172.21.15.63:0/3475241283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb36cd30 msgr2=0x556aeb370f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:11.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.537+0000 7fc747400700 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb36cd30 0x556aeb370f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.537+0000 7fc747400700 1 -- 172.21.15.63:0/3475241283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aeb1d48d0 con 0x556aeb1e7e40 2024-05-27T08:08:11.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.537+0000 7fc747400700 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb1e7e40 0x556aeb370480 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fc7380098e0 tx=0x7fc738009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efebdbe90bc5a21 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:11.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.537+0000 7fc731600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:11.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.538+0000 7fc731600700 1 -- 172.21.15.63:0/3475241283 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7380aaf40 con 0x556aeb1e7e40 2024-05-27T08:08:11.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.538+0000 7fc731600700 1 -- 172.21.15.63:0/3475241283 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7fc7380b4320 con 0x556aeb1e7e40 2024-05-27T08:08:11.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.538+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3475241283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc734024070 con 0x556aeb1e7e40 2024-05-27T08:08:11.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.538+0000 7fc731600700 1 -- 172.21.15.63:0/3475241283 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc7380ab2e0 con 0x556aeb1e7e40 2024-05-27T08:08:11.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.538+0000 7fc75d3c3b80 10 client.22035 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:11.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.538+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3475241283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556aeb255060 con 0x556aeb1e7e40 2024-05-27T08:08:11.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.539+0000 7fc731600700 1 -- 172.21.15.63:0/3475241283 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc734024070 con 0x556aeb1e7e40 2024-05-27T08:08:11.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.539+0000 7fc731600700 1 -- 172.21.15.63:0/3475241283 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc7380e4400 con 0x556aeb1e7e40 2024-05-27T08:08:11.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.539+0000 7fc75d3c3b80 10 client.22035 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:11.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.539+0000 7fc75d3c3b80 10 client.22035 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:08:11.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.539+0000 7fc75d3c3b80 20 client.22035 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:11.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.539+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556aeb380670 0x556aeb382af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.539+0000 7fc75d3c3b80 4 client.22035 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:11.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.539+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3475241283 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aeb296d90 con 0x556aeb380670 2024-05-27T08:08:11.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.540+0000 7fc746a00700 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556aeb380670 0x556aeb382af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.541+0000 7fc746a00700 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556aeb380670 0x556aeb382af0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=600558f17e10b366 server_cookie=b451b5006f4059cc in_seq=0 out_seq=0 2024-05-27T08:08:11.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.541+0000 7fc731600700 10 client.22035 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:08:11.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.543+0000 7fc731600700 1 -- 172.21.15.63:0/3475241283 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556aeb296d90 con 0x556aeb380670 2024-05-27T08:08:11.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.543+0000 7fc731600700 10 client.22035 handle_command_reply: tid=0 2024-05-27T08:08:11.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.543+0000 7fc75d3c3b80 1 client.22035 shutdown 2024-05-27T08:08:11.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3475241283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb1e7e40 msgr2=0x556aeb370480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:11.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb1e7e40 0x556aeb370480 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fc7380098e0 tx=0x7fc738009ca0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3475241283 shutdown_connections 2024-05-27T08:08:11.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb36cd30 0x556aeb370f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556aeb380670 0x556aeb382af0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb1e7e40 0x556aeb370480 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/3475241283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb366db0 0x556aeb3709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3475241283 >> 172.21.15.63:0/3475241283 conn(0x556aeb247fc0 msgr2=0x556aeb360b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:11.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc731600700 0 client.22035 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:08:11.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3475241283 shutdown_connections 2024-05-27T08:08:11.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/3475241283 wait complete. 2024-05-27T08:08:11.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.544+0000 7fc75d3c3b80 20 client.22035 trim_cache size 0 max 16384 2024-05-27T08:08:11.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.566+0000 7fc75d3c3b80 1 Processor -- start 2024-05-27T08:08:11.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.566+0000 7fc75d3c3b80 1 -- start start 2024-05-27T08:08:11.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.566+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb366db0 0x556aeb366870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.566+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb36cd30 0x556aeb36f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.567+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb372920 0x556aeb37c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.567+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7280115c0 con 0x556aeb372920 2024-05-27T08:08:11.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.567+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc72800ee50 con 0x556aeb36cd30 2024-05-27T08:08:11.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.567+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc72800ccc0 con 0x556aeb366db0 2024-05-27T08:08:11.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.567+0000 7fc747e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb372920 0x556aeb37c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.567+0000 7fc747e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb372920 0x556aeb37c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45076/0 (socket says 172.21.15.63:45076) 2024-05-27T08:08:11.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.567+0000 7fc747e00700 1 -- 172.21.15.63:0/171540325 learned_addr learned my addr 172.21.15.63:0/171540325 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:11.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.567+0000 7fc746a00700 1 --2- 172.21.15.63:0/171540325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb36cd30 0x556aeb36f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.567+0000 7fc747400700 1 --2- 172.21.15.63:0/171540325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb366db0 0x556aeb366870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.568+0000 7fc747e00700 1 -- 172.21.15.63:0/171540325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb366db0 msgr2=0x556aeb366870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:11.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.568+0000 7fc747e00700 1 --2- 172.21.15.63:0/171540325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb366db0 0x556aeb366870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.568+0000 7fc747e00700 1 -- 172.21.15.63:0/171540325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb36cd30 msgr2=0x556aeb36f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:11.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.568+0000 7fc747e00700 1 --2- 172.21.15.63:0/171540325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb36cd30 0x556aeb36f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.568+0000 7fc747e00700 1 -- 172.21.15.63:0/171540325 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aeb372d30 con 0x556aeb372920 2024-05-27T08:08:11.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.568+0000 7fc747e00700 1 --2- 172.21.15.63:0/171540325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb372920 0x556aeb37c8d0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fc740009dd0 tx=0x7fc7400077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beee7555371041d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:11.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.569+0000 7fc733e00700 1 -- 172.21.15.63:0/171540325 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74001b030 con 0x556aeb372920 2024-05-27T08:08:11.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.569+0000 7fc733e00700 1 -- 172.21.15.63:0/171540325 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc740004bb0 con 0x556aeb372920 2024-05-27T08:08:11.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.569+0000 7fc733e00700 1 -- 172.21.15.63:0/171540325 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740017e20 con 0x556aeb372920 2024-05-27T08:08:11.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.570+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/171540325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb372920 msgr2=0x556aeb37c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:11.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.570+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/171540325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb372920 0x556aeb37c8d0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7fc740009dd0 tx=0x7fc7400077e0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.570+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/171540325 shutdown_connections 2024-05-27T08:08:11.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.570+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/171540325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb372920 0x556aeb37c8d0 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.570+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/171540325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb366db0 0x556aeb366870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.570+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/171540325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb36cd30 0x556aeb36f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.570+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/171540325 >> 172.21.15.63:0/171540325 conn(0x556aeb247fc0 msgr2=0x556aeb36bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:11.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.570+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/171540325 shutdown_connections 2024-05-27T08:08:11.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.570+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/171540325 wait complete. 2024-05-27T08:08:11.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.571+0000 7fc75d3c3b80 1 -- start start 2024-05-27T08:08:11.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.571+0000 7fc75d3c3b80 1 Processor -- start 2024-05-27T08:08:11.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb366db0 0x556aeb378d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb3792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc75d3c3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb372920 0x556aeb3879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc72800d310 con 0x556aeb36cd30 2024-05-27T08:08:11.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc72800e780 con 0x556aeb366db0 2024-05-27T08:08:11.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc75d3c3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc72800d9e0 con 0x556aeb372920 2024-05-27T08:08:11.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc75d3c3b80 10 client.0 _ll_register_callbacks cb 0x556aeb22ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:11.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc746a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb3792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc746a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb3792d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45080/0 (socket says 172.21.15.63:45080) 2024-05-27T08:08:11.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc746a00700 1 -- 172.21.15.63:0/206821129 learned_addr learned my addr 172.21.15.63:0/206821129 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:11.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.572+0000 7fc747400700 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb366db0 0x556aeb378d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.573+0000 7fc746a00700 1 -- 172.21.15.63:0/206821129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb372920 msgr2=0x556aeb3879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:11.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.573+0000 7fc746a00700 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb372920 0x556aeb3879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.573+0000 7fc746a00700 1 -- 172.21.15.63:0/206821129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb366db0 msgr2=0x556aeb378d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:11.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.573+0000 7fc746a00700 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb366db0 0x556aeb378d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.573+0000 7fc746a00700 1 -- 172.21.15.63:0/206821129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aeb388110 con 0x556aeb36cd30 2024-05-27T08:08:11.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.574+0000 7fc746a00700 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb3792d0 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fc73400baa0 tx=0x7fc73401f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66d839a060344f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:11.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.574+0000 7fc731600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:11.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.574+0000 7fc731600700 1 -- 172.21.15.63:0/206821129 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73401fcd0 con 0x556aeb36cd30 2024-05-27T08:08:11.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.574+0000 7fc731600700 1 -- 172.21.15.63:0/206821129 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7fc734003680 con 0x556aeb36cd30 2024-05-27T08:08:11.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.574+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/206821129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc740004bb0 con 0x556aeb36cd30 2024-05-27T08:08:11.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.574+0000 7fc731600700 1 -- 172.21.15.63:0/206821129 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc734044050 con 0x556aeb36cd30 2024-05-27T08:08:11.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.574+0000 7fc75d3c3b80 10 client.22041 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:11.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.574+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/206821129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556aeb3eac40 con 0x556aeb36cd30 2024-05-27T08:08:11.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.575+0000 7fc731600700 1 -- 172.21.15.63:0/206821129 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc740004bb0 con 0x556aeb36cd30 2024-05-27T08:08:11.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.575+0000 7fc731600700 1 -- 172.21.15.63:0/206821129 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc734036750 con 0x556aeb36cd30 2024-05-27T08:08:11.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.575+0000 7fc75d3c3b80 10 client.22041 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:11.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.575+0000 7fc75d3c3b80 10 client.22041 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:08:11.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.575+0000 7fc75d3c3b80 20 client.22041 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:11.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.575+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556aeb3eb120 0x556aeb3ed540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:11.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.575+0000 7fc75d3c3b80 4 client.22041 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:08:11.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.575+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/206821129 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556aeb290570 con 0x556aeb3eb120 2024-05-27T08:08:11.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.576+0000 7fc747400700 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556aeb3eb120 0x556aeb3ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:11.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.577+0000 7fc731600700 10 client.22041 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:08:11.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.577+0000 7fc747400700 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556aeb3eb120 0x556aeb3ed540 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24e4d71e443b31d0 server_cookie=63eb63b1561012e6 in_seq=0 out_seq=0 2024-05-27T08:08:11.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.578+0000 7fc731600700 1 -- 172.21.15.63:0/206821129 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556aeb290570 con 0x556aeb3eb120 2024-05-27T08:08:11.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.578+0000 7fc731600700 10 client.22041 handle_command_reply: tid=0 2024-05-27T08:08:11.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.579+0000 7fc75d3c3b80 1 client.22041 shutdown 2024-05-27T08:08:11.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.580+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/206821129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 msgr2=0x556aeb3792d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:11.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.580+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb3792d0 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fc73400baa0 tx=0x7fc73401f460 comp rx=0 tx=0).stop 2024-05-27T08:08:11.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.580+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/206821129 shutdown_connections 2024-05-27T08:08:11.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.580+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aeb372920 0x556aeb3879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.580+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aeb366db0 0x556aeb378d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.580+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x556aeb3eb120 0x556aeb3ed540 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.581+0000 7fc75d3c3b80 1 --2- 172.21.15.63:0/206821129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aeb36cd30 0x556aeb3792d0 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.581+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/206821129 >> 172.21.15.63:0/206821129 conn(0x556aeb247fc0 msgr2=0x556aeb36bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:11.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.581+0000 7fc731600700 0 client.22041 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:08:11.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.581+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/206821129 shutdown_connections 2024-05-27T08:08:11.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.582+0000 7fc75d3c3b80 1 -- 172.21.15.63:0/206821129 wait complete. 2024-05-27T08:08:11.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.582+0000 7fc75d3c3b80 20 client.22041 trim_cache size 0 max 16384 2024-05-27T08:08:11.583 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:08:11.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.583+0000 7fc74e800700 1 -- 172.21.15.63:0/2095951780 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc72c0b2a70 msgr2=0x7fc72c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:11.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.583+0000 7fc74e800700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc72c0b2a70 0x7fc72c0b4f80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc734005950 tx=0x7fc7340058e0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.583+0000 7fc74e800700 1 -- 172.21.15.63:0/2095951780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc748075af0 msgr2=0x7fc74819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:11.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.583+0000 7fc74e800700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc748075af0 0x7fc74819cf90 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fc7380057c0 tx=0x7fc738009f90 comp rx=0 tx=0).stop 2024-05-27T08:08:11.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.583+0000 7fc74e800700 1 -- 172.21.15.63:0/2095951780 shutdown_connections 2024-05-27T08:08:11.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.583+0000 7fc74e800700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc72c0b2a70 0x7fc72c0b4f80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.583+0000 7fc74e800700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc748110bf0 0x7fc7481a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.583+0000 7fc74e800700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc748075af0 0x7fc74819cf90 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.583+0000 7fc74e800700 1 --2- 172.21.15.63:0/2095951780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc748076440 0x7fc74819d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:11.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.583+0000 7fc74e800700 1 -- 172.21.15.63:0/2095951780 >> 172.21.15.63:0/2095951780 conn(0x7fc748100030 msgr2=0x7fc748100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:11.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.584+0000 7fc74e800700 1 -- 172.21.15.63:0/2095951780 shutdown_connections 2024-05-27T08:08:11.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:11.584+0000 7fc74e800700 1 -- 172.21.15.63:0/2095951780 wait complete. 2024-05-27T08:08:11.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:11 smithi086 ceph-mon[34687]: pgmap v1097: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:08:11.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:11 smithi086 ceph-mon[34687]: osdmap e150: 12 total, 12 up, 12 in 2024-05-27T08:08:11.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:11 smithi149 ceph-mon[35408]: pgmap v1097: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:08:11.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:11 smithi149 ceph-mon[35408]: osdmap e150: 12 total, 12 up, 12 in 2024-05-27T08:08:11.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:11 smithi063 ceph-mon[31613]: pgmap v1097: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.5 MiB/s wr, 34 op/s 2024-05-27T08:08:11.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:11 smithi063 ceph-mon[31613]: osdmap e150: 12 total, 12 up, 12 in 2024-05-27T08:08:12.827 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:08:13.343 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:08:13.850 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:13 smithi063 ceph-mon[31613]: pgmap v1099: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:13.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:13 smithi086 ceph-mon[34687]: pgmap v1099: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:13.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:13 smithi149 ceph-mon[35408]: pgmap v1099: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.766+0000 7f7750200640 1 -- 172.21.15.63:0/1672527288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774810b7f0 msgr2=0x7f774810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.766+0000 7f7750200640 1 --2- 172.21.15.63:0/1672527288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774810b7f0 0x7f774810dc10 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f7734009900 tx=0x7f773402a3f0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.766+0000 7f7750200640 1 -- 172.21.15.63:0/1672527288 shutdown_connections 2024-05-27T08:08:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.766+0000 7f7750200640 1 --2- 172.21.15.63:0/1672527288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f774810e1e0 0x7f77481106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.766+0000 7f7750200640 1 --2- 172.21.15.63:0/1672527288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774810b7f0 0x7f774810dc10 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.766+0000 7f7750200640 1 --2- 172.21.15.63:0/1672527288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7748104d90 0x7f77481051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.766+0000 7f7750200640 1 -- 172.21.15.63:0/1672527288 >> 172.21.15.63:0/1672527288 conn(0x7f77480782d0 msgr2=0x7f7748078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.766+0000 7f7750200640 1 -- 172.21.15.63:0/1672527288 shutdown_connections 2024-05-27T08:08:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.766+0000 7f7750200640 1 -- 172.21.15.63:0/1672527288 wait complete. 2024-05-27T08:08:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.768+0000 7f7750200640 1 Processor -- start 2024-05-27T08:08:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.768+0000 7f7750200640 1 -- start start 2024-05-27T08:08:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.768+0000 7f7750200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7748104d90 0x7f77481a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.768+0000 7f7750200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f774810b7f0 0x7f77481a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.768+0000 7f7750200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774810e1e0 0x7f774819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.769+0000 7f7750200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7748113cf0 con 0x7f774810e1e0 2024-05-27T08:08:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.769+0000 7f7750200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7748113b70 con 0x7f7748104d90 2024-05-27T08:08:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.769+0000 7f7750200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7748113e70 con 0x7f774810b7f0 2024-05-27T08:08:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.769+0000 7f774e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774810e1e0 0x7f774819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.769+0000 7f774d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7748104d90 0x7f77481a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.769+0000 7f774e200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774810e1e0 0x7f774819c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45110/0 (socket says 172.21.15.63:45110) 2024-05-27T08:08:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.769+0000 7f774e200640 1 -- 172.21.15.63:0/134022661 learned_addr learned my addr 172.21.15.63:0/134022661 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.769+0000 7f774ce00640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f774810b7f0 0x7f77481a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.770+0000 7f774e200640 1 -- 172.21.15.63:0/134022661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f774810b7f0 msgr2=0x7f77481a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.770+0000 7f774e200640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f774810b7f0 0x7f77481a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.770+0000 7f774e200640 1 -- 172.21.15.63:0/134022661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7748104d90 msgr2=0x7f77481a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.770+0000 7f774e200640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7748104d90 0x7f77481a25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.770+0000 7f774e200640 1 -- 172.21.15.63:0/134022661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774819ceb0 con 0x7f774810e1e0 2024-05-27T08:08:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.770+0000 7f774e200640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774810e1e0 0x7f774819c7d0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f774000eaa0 tx=0x7f774000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.770+0000 7f7746000640 1 -- 172.21.15.63:0/134022661 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774000cdb0 con 0x7f774810e1e0 2024-05-27T08:08:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.770+0000 7f7746000640 1 -- 172.21.15.63:0/134022661 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7740005020 con 0x7f774810e1e0 2024-05-27T08:08:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.770+0000 7f7746000640 1 -- 172.21.15.63:0/134022661 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77400116d0 con 0x7f774810e1e0 2024-05-27T08:08:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.770+0000 7f7750200640 1 -- 172.21.15.63:0/134022661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774819d1a0 con 0x7f774810e1e0 2024-05-27T08:08:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.771+0000 7f7750200640 1 -- 172.21.15.63:0/134022661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77481a93f0 con 0x7f774810e1e0 2024-05-27T08:08:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.774+0000 7f7746000640 1 -- 172.21.15.63:0/134022661 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f77400051d0 con 0x7f774810e1e0 2024-05-27T08:08:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.776+0000 7f77571e7740 1 Processor -- start 2024-05-27T08:08:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.776+0000 7f77571e7740 1 -- start start 2024-05-27T08:08:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.776+0000 7f7746000640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77300b29d0 0x7f77300b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.776+0000 7f77571e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 0x56507a798860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f77571e7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a798da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f77571e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f77571e7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56507a709b90 con 0x56507a3cd2f0 2024-05-27T08:08:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f77571e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56507a6f9730 con 0x56507a3d54d0 2024-05-27T08:08:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f77571e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56507a6fd770 con 0x56507a3f61b0 2024-05-27T08:08:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f7746000640 1 -- 172.21.15.63:0/134022661 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f77400e2a90 con 0x7f774810e1e0 2024-05-27T08:08:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f774ce00640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77300b29d0 0x7f77300b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f774ce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a798da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f774d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 0x56507a798860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f774d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 0x56507a798860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41846/0 (socket says 172.21.15.63:41846) 2024-05-27T08:08:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f774e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.777+0000 7f774d800640 1 -- 172.21.15.63:0/2270814773 learned_addr learned my addr 172.21.15.63:0/2270814773 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.778+0000 7f774d800640 1 -- 172.21.15.63:0/2270814773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 msgr2=0x56507a7992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.778+0000 7f774d800640 1 --2- 172.21.15.63:0/2270814773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.778+0000 7f774d800640 1 -- 172.21.15.63:0/2270814773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 msgr2=0x56507a798da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.778+0000 7f774d800640 1 --2- 172.21.15.63:0/2270814773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a798da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.778+0000 7f774d800640 1 -- 172.21.15.63:0/2270814773 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507a6928b0 con 0x56507a3f61b0 2024-05-27T08:08:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.778+0000 7f774ce00640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77300b29d0 0x7f77300b4ee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f77340057c0 tx=0x7f7734005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.779+0000 7f774d800640 1 --2- 172.21.15.63:0/2270814773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 0x56507a798860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f773c00bcb0 tx=0x7f773c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51c4251301d54f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.779+0000 7f773aa00640 1 -- 172.21.15.63:0/2270814773 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c01c480 con 0x56507a3f61b0 2024-05-27T08:08:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.779+0000 7f773aa00640 1 -- 172.21.15.63:0/2270814773 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f773c01cab0 con 0x56507a3f61b0 2024-05-27T08:08:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.779+0000 7f773aa00640 1 -- 172.21.15.63:0/2270814773 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c011af0 con 0x56507a3f61b0 2024-05-27T08:08:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.781+0000 7f77571e7740 1 -- 172.21.15.63:0/2270814773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 msgr2=0x56507a798860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.781+0000 7f77571e7740 1 --2- 172.21.15.63:0/2270814773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 0x56507a798860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f773c00bcb0 tx=0x7f773c0107a0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.781+0000 7f77571e7740 1 -- 172.21.15.63:0/2270814773 shutdown_connections 2024-05-27T08:08:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.781+0000 7f77571e7740 1 --2- 172.21.15.63:0/2270814773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.781+0000 7f77571e7740 1 --2- 172.21.15.63:0/2270814773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a798da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.781+0000 7f77571e7740 1 --2- 172.21.15.63:0/2270814773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 0x56507a798860 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.781+0000 7f77571e7740 1 -- 172.21.15.63:0/2270814773 >> 172.21.15.63:0/2270814773 conn(0x56507a3ea300 msgr2=0x56507a795010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.781+0000 7f77571e7740 1 -- 172.21.15.63:0/2270814773 shutdown_connections 2024-05-27T08:08:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.781+0000 7f77571e7740 1 -- 172.21.15.63:0/2270814773 wait complete. 2024-05-27T08:08:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.782+0000 7f77571e7740 1 -- start start 2024-05-27T08:08:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.782+0000 7f77571e7740 1 Processor -- start 2024-05-27T08:08:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.783+0000 7f77571e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3cd2f0 0x56507a7a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.783+0000 7f77571e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.783+0000 7f77571e7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3f61b0 0x56507a7aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.783+0000 7f77571e7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56507a3df790 con 0x56507a3f61b0 2024-05-27T08:08:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.783+0000 7f77571e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56507a6a1bc0 con 0x56507a3d54d0 2024-05-27T08:08:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.783+0000 7f77571e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56507a7a5990 con 0x56507a3cd2f0 2024-05-27T08:08:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.783+0000 7f77571e7740 10 client.0 _ll_register_callbacks cb 0x56507a67f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.783+0000 7f774ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.783+0000 7f774d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3cd2f0 0x56507a7a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.784+0000 7f774ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7a2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:33436/0 (socket says 172.21.15.63:33436) 2024-05-27T08:08:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.784+0000 7f774ce00640 1 -- 172.21.15.63:0/1951351787 learned_addr learned my addr 172.21.15.63:0/1951351787 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.784+0000 7f774e200640 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3f61b0 0x56507a7aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.784+0000 7f774ce00640 1 -- 172.21.15.63:0/1951351787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3cd2f0 msgr2=0x56507a7a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.784+0000 7f774ce00640 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3cd2f0 0x56507a7a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.784+0000 7f774ce00640 1 -- 172.21.15.63:0/1951351787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3f61b0 msgr2=0x56507a7aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.784+0000 7f774ce00640 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3f61b0 0x56507a7aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.784+0000 7f774ce00640 1 -- 172.21.15.63:0/1951351787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56507a40e210 con 0x56507a3d54d0 2024-05-27T08:08:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.785+0000 7f774ce00640 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7a2f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f773402a910 tx=0x7f77340058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4952905d71bee556 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.785+0000 7f772f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:08:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.785+0000 7f772f400640 1 -- 172.21.15.63:0/1951351787 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773404e030 con 0x56507a3d54d0 2024-05-27T08:08:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.786+0000 7f772f400640 1 -- 172.21.15.63:0/1951351787 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f7734046040 con 0x56507a3d54d0 2024-05-27T08:08:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.786+0000 7f77571e7740 1 -- 172.21.15.63:0/1951351787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f773c01cab0 con 0x56507a3d54d0 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.786+0000 7f772f400640 1 -- 172.21.15.63:0/1951351787 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f773405b050 con 0x56507a3d54d0 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.786+0000 7f77571e7740 10 client.41072 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.786+0000 7f77571e7740 1 -- 172.21.15.63:0/1951351787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56507a4da620 con 0x56507a3d54d0 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.787+0000 7f772f400640 1 -- 172.21.15.63:0/1951351787 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f773c01cab0 con 0x56507a3d54d0 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.787+0000 7f772f400640 1 -- 172.21.15.63:0/1951351787 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f773403d910 con 0x56507a3d54d0 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.787+0000 7f77571e7740 10 client.41072 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.787+0000 7f77571e7740 10 client.41072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.787+0000 7f77571e7740 20 client.41072 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.787+0000 7f77571e7740 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507a3b4d80 0x56507a7b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.787+0000 7f77571e7740 4 client.41072 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.787+0000 7f77571e7740 1 -- 172.21.15.63:0/1951351787 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56507a70bae0 con 0x56507a3b4d80 2024-05-27T08:08:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.788+0000 7f774d800640 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507a3b4d80 0x56507a7b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.789+0000 7f774d800640 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507a3b4d80 0x56507a7b4c70 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f90a7522d6fa96b5 server_cookie=1c59905e86c4758c in_seq=0 out_seq=0 2024-05-27T08:08:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.789+0000 7f772f400640 10 client.41072 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f772f400640 1 -- 172.21.15.63:0/1951351787 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56507a70bae0 con 0x56507a3b4d80 2024-05-27T08:08:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f772f400640 10 client.41072 handle_command_reply: tid=0 2024-05-27T08:08:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f77571e7740 1 client.41072 shutdown 2024-05-27T08:08:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f77571e7740 1 -- 172.21.15.63:0/1951351787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 msgr2=0x56507a7a2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f77571e7740 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7a2f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f773402a910 tx=0x7f77340058d0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f77571e7740 1 -- 172.21.15.63:0/1951351787 shutdown_connections 2024-05-27T08:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f77571e7740 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507a3b4d80 0x56507a7b4c70 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f77571e7740 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3f61b0 0x56507a7aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f77571e7740 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7a2f70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f77571e7740 1 --2- 172.21.15.63:0/1951351787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3cd2f0 0x56507a7a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f772f400640 0 client.41072 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.791+0000 7f77571e7740 1 -- 172.21.15.63:0/1951351787 >> 172.21.15.63:0/1951351787 conn(0x56507a3ea300 msgr2=0x56507a798690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.792+0000 7f77571e7740 1 -- 172.21.15.63:0/1951351787 shutdown_connections 2024-05-27T08:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.792+0000 7f77571e7740 1 -- 172.21.15.63:0/1951351787 wait complete. 2024-05-27T08:08:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.792+0000 7f77571e7740 20 client.41072 trim_cache size 0 max 16384 2024-05-27T08:08:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.811+0000 7f77571e7740 1 Processor -- start 2024-05-27T08:08:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.811+0000 7f77571e7740 1 -- start start 2024-05-27T08:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.811+0000 7f77571e7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.811+0000 7f77571e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.811+0000 7f77571e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 0x56507a7aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.811+0000 7f77571e7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77481b5c70 con 0x56507a3cd2f0 2024-05-27T08:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.811+0000 7f77571e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77481b3540 con 0x56507a3d54d0 2024-05-27T08:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.811+0000 7f77571e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f77481b13b0 con 0x56507a3f61b0 2024-05-27T08:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7a5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45122/0 (socket says 172.21.15.63:45122) 2024-05-27T08:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774d800640 1 -- 172.21.15.63:0/998951164 learned_addr learned my addr 172.21.15.63:0/998951164 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774ce00640 1 --2- 172.21.15.63:0/998951164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774e200640 1 --2- 172.21.15.63:0/998951164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 0x56507a7aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774d800640 1 -- 172.21.15.63:0/998951164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 msgr2=0x56507a7aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774d800640 1 --2- 172.21.15.63:0/998951164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 0x56507a7aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774d800640 1 -- 172.21.15.63:0/998951164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 msgr2=0x56507a7aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774d800640 1 --2- 172.21.15.63:0/998951164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774d800640 1 -- 172.21.15.63:0/998951164 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507a78f050 con 0x56507a3cd2f0 2024-05-27T08:08:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.812+0000 7f774ce00640 1 --2- 172.21.15.63:0/998951164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.813+0000 7f774d800640 1 --2- 172.21.15.63:0/998951164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7a5160 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f773c010f90 tx=0x7f773c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6aa468fef7ac50 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.813+0000 7f773a000640 1 -- 172.21.15.63:0/998951164 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c005040 con 0x56507a3cd2f0 2024-05-27T08:08:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.813+0000 7f773a000640 1 -- 172.21.15.63:0/998951164 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f773c01c480 con 0x56507a3cd2f0 2024-05-27T08:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.813+0000 7f773a000640 1 -- 172.21.15.63:0/998951164 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c004880 con 0x56507a3cd2f0 2024-05-27T08:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.814+0000 7f77571e7740 1 -- 172.21.15.63:0/998951164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 msgr2=0x56507a7a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.814+0000 7f77571e7740 1 --2- 172.21.15.63:0/998951164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7a5160 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f773c010f90 tx=0x7f773c013040 comp rx=0 tx=0).stop 2024-05-27T08:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.815+0000 7f77571e7740 1 -- 172.21.15.63:0/998951164 shutdown_connections 2024-05-27T08:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.815+0000 7f77571e7740 1 --2- 172.21.15.63:0/998951164 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3f61b0 0x56507a7aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.815+0000 7f77571e7740 1 --2- 172.21.15.63:0/998951164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3d54d0 0x56507a7aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.815+0000 7f77571e7740 1 --2- 172.21.15.63:0/998951164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7a5160 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.815+0000 7f77571e7740 1 -- 172.21.15.63:0/998951164 >> 172.21.15.63:0/998951164 conn(0x56507a3ea300 msgr2=0x56507a7a2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.815+0000 7f77571e7740 1 -- 172.21.15.63:0/998951164 shutdown_connections 2024-05-27T08:08:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.815+0000 7f77571e7740 1 -- 172.21.15.63:0/998951164 wait complete. 2024-05-27T08:08:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.815+0000 7f77571e7740 1 -- start start 2024-05-27T08:08:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.816+0000 7f77571e7740 1 Processor -- start 2024-05-27T08:08:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.816+0000 7f77571e7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.816+0000 7f77571e7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3d54d0 0x56507a7ab100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.816+0000 7f77571e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3f61b0 0x56507a7a8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.816+0000 7f77571e7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77481b1a00 con 0x56507a3cd2f0 2024-05-27T08:08:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.816+0000 7f77571e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77481b2e70 con 0x56507a3f61b0 2024-05-27T08:08:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.816+0000 7f77571e7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f77481b20d0 con 0x56507a3d54d0 2024-05-27T08:08:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.816+0000 7f77571e7740 10 client.0 _ll_register_callbacks cb 0x7f77340095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.817+0000 7f774d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.817+0000 7f774d800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7aabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45136/0 (socket says 172.21.15.63:45136) 2024-05-27T08:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.817+0000 7f774d800640 1 -- 172.21.15.63:0/1856515756 learned_addr learned my addr 172.21.15.63:0/1856515756 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.817+0000 7f774ce00640 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3d54d0 0x56507a7ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.817+0000 7f774e200640 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3f61b0 0x56507a7a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.817+0000 7f774d800640 1 -- 172.21.15.63:0/1856515756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3d54d0 msgr2=0x56507a7ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.817+0000 7f774d800640 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3d54d0 0x56507a7ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.817+0000 7f774d800640 1 -- 172.21.15.63:0/1856515756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3f61b0 msgr2=0x56507a7a8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.817+0000 7f774d800640 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3f61b0 0x56507a7a8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.817+0000 7f774d800640 1 -- 172.21.15.63:0/1856515756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56507a7a86c0 con 0x56507a3cd2f0 2024-05-27T08:08:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.818+0000 7f774d800640 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7aabc0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f773c00bbd0 tx=0x7f773c010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e7805f4b9c523a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.818+0000 7f772f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.818+0000 7f772f400640 1 -- 172.21.15.63:0/1856515756 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c01ca40 con 0x56507a3cd2f0 2024-05-27T08:08:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.818+0000 7f772f400640 1 -- 172.21.15.63:0/1856515756 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f773c005040 con 0x56507a3cd2f0 2024-05-27T08:08:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.818+0000 7f77571e7740 1 -- 172.21.15.63:0/1856515756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f773c01c480 con 0x56507a3cd2f0 2024-05-27T08:08:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.819+0000 7f772f400640 1 -- 172.21.15.63:0/1856515756 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f773c005240 con 0x56507a3cd2f0 2024-05-27T08:08:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.819+0000 7f77571e7740 10 client.22059 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.819+0000 7f77571e7740 1 -- 172.21.15.63:0/1856515756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56507a81cc50 con 0x56507a3cd2f0 2024-05-27T08:08:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.819+0000 7f772f400640 1 -- 172.21.15.63:0/1856515756 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f773c01c480 con 0x56507a3cd2f0 2024-05-27T08:08:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.819+0000 7f772f400640 1 -- 172.21.15.63:0/1856515756 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f773c0035f0 con 0x56507a3cd2f0 2024-05-27T08:08:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.819+0000 7f77571e7740 10 client.22059 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.819+0000 7f77571e7740 10 client.22059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.819+0000 7f77571e7740 20 client.22059 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.819+0000 7f77571e7740 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507a38a1c0 0x56507a81f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.820+0000 7f77571e7740 4 client.22059 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:08:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.820+0000 7f77571e7740 1 -- 172.21.15.63:0/1856515756 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56507a7052c0 con 0x56507a38a1c0 2024-05-27T08:08:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.820+0000 7f774ce00640 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507a38a1c0 0x56507a81f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.821+0000 7f772f400640 10 client.22059 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.821+0000 7f774ce00640 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507a38a1c0 0x56507a81f170 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba82e428e4cd886 server_cookie=9e400f1e8643682f in_seq=0 out_seq=0 2024-05-27T08:08:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.863+0000 7f772f400640 1 -- 172.21.15.63:0/1856515756 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56507a7052c0 con 0x56507a38a1c0 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.863+0000 7f772f400640 10 client.22059 handle_command_reply: tid=0 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.863+0000 7f77571e7740 1 client.22059 shutdown 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.863+0000 7f77571e7740 1 -- 172.21.15.63:0/1856515756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 msgr2=0x56507a7aabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.863+0000 7f77571e7740 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7aabc0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f773c00bbd0 tx=0x7f773c010db0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.864+0000 7f77571e7740 1 -- 172.21.15.63:0/1856515756 shutdown_connections 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.864+0000 7f77571e7740 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507a38a1c0 0x56507a81f170 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.864+0000 7f77571e7740 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507a3f61b0 0x56507a7a8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.864+0000 7f77571e7740 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507a3d54d0 0x56507a7ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.864+0000 7f772f400640 0 client.22059 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.864+0000 7f77571e7740 1 --2- 172.21.15.63:0/1856515756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507a3cd2f0 0x56507a7aabc0 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.864+0000 7f77571e7740 1 -- 172.21.15.63:0/1856515756 >> 172.21.15.63:0/1856515756 conn(0x56507a3ea300 msgr2=0x56507a7a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.864+0000 7f77571e7740 1 -- 172.21.15.63:0/1856515756 shutdown_connections 2024-05-27T08:08:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.864+0000 7f77571e7740 1 -- 172.21.15.63:0/1856515756 wait complete. 2024-05-27T08:08:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.864+0000 7f77571e7740 20 client.22059 trim_cache size 0 max 16384 2024-05-27T08:08:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:08:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:08:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:08:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.865+0000 7f7750200640 1 -- 172.21.15.63:0/134022661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77300b29d0 msgr2=0x7f77300b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.865+0000 7f7750200640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77300b29d0 0x7f77300b4ee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f77340057c0 tx=0x7f7734005fd0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.865+0000 7f7750200640 1 -- 172.21.15.63:0/134022661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774810e1e0 msgr2=0x7f774819c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.865+0000 7f7750200640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774810e1e0 0x7f774819c7d0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f774000eaa0 tx=0x7f774000ef60 comp rx=0 tx=0).stop 2024-05-27T08:08:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.865+0000 7f7750200640 1 -- 172.21.15.63:0/134022661 shutdown_connections 2024-05-27T08:08:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.865+0000 7f7750200640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77300b29d0 0x7f77300b4ee0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.865+0000 7f7750200640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774810e1e0 0x7f774819c7d0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.865+0000 7f7750200640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f774810b7f0 0x7f77481a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.865+0000 7f7750200640 1 --2- 172.21.15.63:0/134022661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7748104d90 0x7f77481a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.865+0000 7f7750200640 1 -- 172.21.15.63:0/134022661 >> 172.21.15.63:0/134022661 conn(0x7f77480782d0 msgr2=0x7f774810a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.866+0000 7f7750200640 1 -- 172.21.15.63:0/134022661 shutdown_connections 2024-05-27T08:08:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:14.866+0000 7f7750200640 1 -- 172.21.15.63:0/134022661 wait complete. 2024-05-27T08:08:15.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:15 smithi063 ceph-mon[31613]: pgmap v1100: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:15.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.880+0000 7fadbb200700 1 -- 172.21.15.63:0/3393381336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadbc10e6f0 msgr2=0x7fadbc110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:15.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.880+0000 7fadbb200700 1 --2- 172.21.15.63:0/3393381336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadbc10e6f0 0x7fadbc110bd0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b0a0 tx=0x7fadb400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.880+0000 7fadbb200700 1 -- 172.21.15.63:0/3393381336 shutdown_connections 2024-05-27T08:08:15.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.880+0000 7fadbb200700 1 --2- 172.21.15.63:0/3393381336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadbc10e6f0 0x7fadbc110bd0 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.880+0000 7fadbb200700 1 --2- 172.21.15.63:0/3393381336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadbc105280 0x7fadbc105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.880+0000 7fadbb200700 1 --2- 172.21.15.63:0/3393381336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc105bd0 0x7fadbc10e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.880+0000 7fadbb200700 1 -- 172.21.15.63:0/3393381336 >> 172.21.15.63:0/3393381336 conn(0x7fadbc078ab0 msgr2=0x7fadbc078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:15.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.880+0000 7fadbb200700 1 -- 172.21.15.63:0/3393381336 shutdown_connections 2024-05-27T08:08:15.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.880+0000 7fadbb200700 1 -- 172.21.15.63:0/3393381336 wait complete. 2024-05-27T08:08:15.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.882+0000 7fadbb200700 1 Processor -- start 2024-05-27T08:08:15.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.882+0000 7fadbb200700 1 -- start start 2024-05-27T08:08:15.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.882+0000 7fadbb200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadbc105280 0x7fadbc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadbb200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadbc105bd0 0x7fadbc1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadbb200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc10e6f0 0x7fadbc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadbb200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc114150 con 0x7fadbc105280 2024-05-27T08:08:15.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadbb200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc113fd0 con 0x7fadbc10e6f0 2024-05-27T08:08:15.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadbb200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1142d0 con 0x7fadbc105bd0 2024-05-27T08:08:15.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadb9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadbc105280 0x7fadbc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadb9e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadbc105280 0x7fadbc1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45162/0 (socket says 172.21.15.63:45162) 2024-05-27T08:08:15.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadb9e00700 1 -- 172.21.15.63:0/2973862146 learned_addr learned my addr 172.21.15.63:0/2973862146 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:15.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadb9400700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadbc105bd0 0x7fadbc1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadba800700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc10e6f0 0x7fadbc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadb9e00700 1 -- 172.21.15.63:0/2973862146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadbc105bd0 msgr2=0x7fadbc1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:15.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadb9e00700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadbc105bd0 0x7fadbc1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadb9e00700 1 -- 172.21.15.63:0/2973862146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc10e6f0 msgr2=0x7fadbc19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:15.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadb9e00700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc10e6f0 0x7fadbc19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.883+0000 7fadb9e00700 1 -- 172.21.15.63:0/2973862146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc19afc0 con 0x7fadbc105280 2024-05-27T08:08:15.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.884+0000 7fadba800700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc10e6f0 0x7fadbc19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:08:15.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.884+0000 7fadb9e00700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadbc105280 0x7fadbc1a06f0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fadac00d980 tx=0x7fadac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:15.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.884+0000 7fadb2a00700 1 -- 172.21.15.63:0/2973862146 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac016070 con 0x7fadbc105280 2024-05-27T08:08:15.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.884+0000 7fadb2a00700 1 -- 172.21.15.63:0/2973862146 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadac004020 con 0x7fadbc105280 2024-05-27T08:08:15.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.884+0000 7fadbb200700 1 -- 172.21.15.63:0/2973862146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc19b250 con 0x7fadbc105280 2024-05-27T08:08:15.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.884+0000 7fadb2a00700 1 -- 172.21.15.63:0/2973862146 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac004bb0 con 0x7fadbc105280 2024-05-27T08:08:15.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.885+0000 7fadbb200700 1 -- 172.21.15.63:0/2973862146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc1a7580 con 0x7fadbc105280 2024-05-27T08:08:15.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.889+0000 7fadb2a00700 1 -- 172.21.15.63:0/2973862146 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fadac00bb60 con 0x7fadbc105280 2024-05-27T08:08:15.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.890+0000 7fadb2a00700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fada00b29f0 0x7fada00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.891+0000 7fadb2a00700 1 -- 172.21.15.63:0/2973862146 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7fadac0e28f0 con 0x7fadbc105280 2024-05-27T08:08:15.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.891+0000 7fadcf57ab80 1 Processor -- start 2024-05-27T08:08:15.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.891+0000 7fadb9400700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fada00b29f0 0x7fada00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.891+0000 7fadcf57ab80 1 -- start start 2024-05-27T08:08:15.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 0x55d86e4baee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d86e3dae40 con 0x55d86e4b0c30 2024-05-27T08:08:15.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d86e3ca9e0 con 0x55d86e32de40 2024-05-27T08:08:15.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d86e3cea20 con 0x55d86e4b0820 2024-05-27T08:08:15.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadb9400700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fada00b29f0 0x7fada00b4f00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fadbc102600 tx=0x7fada8009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:15.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4bd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45164/0 (socket says 172.21.15.63:45164) 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadba800700 1 -- 172.21.15.63:0/2431272803 learned_addr learned my addr 172.21.15.63:0/2431272803 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadb9e00700 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.892+0000 7fadb9400700 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 0x55d86e4baee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.893+0000 7fadba800700 1 -- 172.21.15.63:0/2431272803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 msgr2=0x55d86e4baee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.893+0000 7fadba800700 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 0x55d86e4baee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.893+0000 7fadba800700 1 -- 172.21.15.63:0/2431272803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 msgr2=0x55d86e4b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.893+0000 7fadba800700 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.893+0000 7fadba800700 1 -- 172.21.15.63:0/2431272803 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86e393420 con 0x55d86e4b0c30 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.893+0000 7fadb9e00700 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.893+0000 7fadb9400700 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 0x55d86e4baee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:15.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.893+0000 7fadba800700 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4bd430 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009d30 tx=0x7fadb4009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10838302120e372b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:15.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.894+0000 7fada7400700 1 -- 172.21.15.63:0/2431272803 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb400bcb0 con 0x55d86e4b0c30 2024-05-27T08:08:15.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.894+0000 7fada7400700 1 -- 172.21.15.63:0/2431272803 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadb4009490 con 0x55d86e4b0c30 2024-05-27T08:08:15.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.894+0000 7fada7400700 1 -- 172.21.15.63:0/2431272803 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4005040 con 0x55d86e4b0c30 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.895+0000 7fadcf57ab80 1 -- 172.21.15.63:0/2431272803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 msgr2=0x55d86e4bd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.895+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4bd430 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009d30 tx=0x7fadb4009d60 comp rx=0 tx=0).stop 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.895+0000 7fadcf57ab80 1 -- 172.21.15.63:0/2431272803 shutdown_connections 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.895+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4bd430 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.895+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.895+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/2431272803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 0x55d86e4baee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.895+0000 7fadcf57ab80 1 -- 172.21.15.63:0/2431272803 >> 172.21.15.63:0/2431272803 conn(0x55d86e38dfc0 msgr2=0x55d86e4a7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.895+0000 7fadcf57ab80 1 -- 172.21.15.63:0/2431272803 shutdown_connections 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.896+0000 7fadcf57ab80 1 -- 172.21.15.63:0/2431272803 wait complete. 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.896+0000 7fadcf57ab80 1 -- start start 2024-05-27T08:08:15.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.896+0000 7fadcf57ab80 1 Processor -- start 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.897+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4ba9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.897+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0820 0x55d86e4baf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.897+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0c30 0x55d86e4bb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.897+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d86e2d0fa0 con 0x55d86e4b0820 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.897+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d86e36b760 con 0x55d86e32de40 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.897+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d86e4bdd40 con 0x55d86e4b0c30 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.897+0000 7fadcf57ab80 10 client.0 _ll_register_callbacks cb 0x55d86e374c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.897+0000 7fadb9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0820 0x55d86e4baf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.898+0000 7fadb9400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0820 0x55d86e4baf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45170/0 (socket says 172.21.15.63:45170) 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.898+0000 7fadb9400700 1 -- 172.21.15.63:0/785118513 learned_addr learned my addr 172.21.15.63:0/785118513 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:15.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.898+0000 7fadb9e00700 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4ba9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.898+0000 7fadba800700 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0c30 0x55d86e4bb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.898+0000 7fadb9400700 1 -- 172.21.15.63:0/785118513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0c30 msgr2=0x55d86e4bb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:15.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.898+0000 7fadb9400700 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0c30 0x55d86e4bb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.898+0000 7fadb9400700 1 -- 172.21.15.63:0/785118513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 msgr2=0x55d86e4ba9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:15.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.898+0000 7fadb9400700 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4ba9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.898+0000 7fadb9400700 1 -- 172.21.15.63:0/785118513 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d86e31a8d0 con 0x55d86e4b0820 2024-05-27T08:08:15.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.899+0000 7fadb9400700 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0820 0x55d86e4baf30 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fada8011840 tx=0x7fada80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bacc6d749b3d9f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:15.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.899+0000 7fad9be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:15.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.899+0000 7fad9be00700 1 -- 172.21.15.63:0/785118513 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8028630 con 0x55d86e4b0820 2024-05-27T08:08:15.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.899+0000 7fad9be00700 1 -- 172.21.15.63:0/785118513 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7fada801f430 con 0x55d86e4b0820 2024-05-27T08:08:15.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.899+0000 7fadcf57ab80 1 -- 172.21.15.63:0/785118513 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb4009490 con 0x55d86e4b0820 2024-05-27T08:08:15.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.899+0000 7fad9be00700 1 -- 172.21.15.63:0/785118513 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fada80287d0 con 0x55d86e4b0820 2024-05-27T08:08:15.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.900+0000 7fadcf57ab80 10 client.22071 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:15.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.900+0000 7fadcf57ab80 1 -- 172.21.15.63:0/785118513 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d86e39b060 con 0x55d86e4b0820 2024-05-27T08:08:15.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.900+0000 7fad9be00700 1 -- 172.21.15.63:0/785118513 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadb4009490 con 0x55d86e4b0820 2024-05-27T08:08:15.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.900+0000 7fad9be00700 1 -- 172.21.15.63:0/785118513 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fada801fc10 con 0x55d86e4b0820 2024-05-27T08:08:15.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.900+0000 7fadcf57ab80 10 client.22071 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:15.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.900+0000 7fadcf57ab80 10 client.22071 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:08:15.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.900+0000 7fadcf57ab80 20 client.22071 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:15.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.901+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d86e4cad90 0x55d86e4cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.901+0000 7fadcf57ab80 4 client.22071 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:15.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.901+0000 7fadcf57ab80 1 -- 172.21.15.63:0/785118513 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d86e3dcd90 con 0x55d86e4cad90 2024-05-27T08:08:15.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.901+0000 7fadb9e00700 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d86e4cad90 0x55d86e4cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.902+0000 7fad9be00700 10 client.22071 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:08:15.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.902+0000 7fadb9e00700 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d86e4cad90 0x55d86e4cd1f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfc23d34d37aa235 server_cookie=2c3d9f9b527d6621 in_seq=0 out_seq=0 2024-05-27T08:08:15.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.904+0000 7fad9be00700 1 -- 172.21.15.63:0/785118513 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d86e3dcd90 con 0x55d86e4cad90 2024-05-27T08:08:15.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.904+0000 7fad9be00700 10 client.22071 handle_command_reply: tid=0 2024-05-27T08:08:15.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.904+0000 7fadcf57ab80 1 client.22071 shutdown 2024-05-27T08:08:15.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fadcf57ab80 1 -- 172.21.15.63:0/785118513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0820 msgr2=0x55d86e4baf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:15.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0820 0x55d86e4baf30 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fada8011840 tx=0x7fada80047c0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fadcf57ab80 1 -- 172.21.15.63:0/785118513 shutdown_connections 2024-05-27T08:08:15.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0c30 0x55d86e4bb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4ba9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d86e4cad90 0x55d86e4cd1f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/785118513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0820 0x55d86e4baf30 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fadcf57ab80 1 -- 172.21.15.63:0/785118513 >> 172.21.15.63:0/785118513 conn(0x55d86e38dfc0 msgr2=0x55d86e4a6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:15.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fad9be00700 0 client.22071 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:08:15.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fadcf57ab80 1 -- 172.21.15.63:0/785118513 shutdown_connections 2024-05-27T08:08:15.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.905+0000 7fadcf57ab80 1 -- 172.21.15.63:0/785118513 wait complete. 2024-05-27T08:08:15.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.906+0000 7fadcf57ab80 20 client.22071 trim_cache size 0 max 16384 2024-05-27T08:08:15.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.928+0000 7fadcf57ab80 1 Processor -- start 2024-05-27T08:08:15.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.928+0000 7fadcf57ab80 1 -- start start 2024-05-27T08:08:15.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.928+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4a7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.929+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 0x55d86e4b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.929+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.929+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1b0950 con 0x55d86e4b0c30 2024-05-27T08:08:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.929+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1ae1e0 con 0x55d86e32de40 2024-05-27T08:08:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.929+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1ac050 con 0x55d86e4b0820 2024-05-27T08:08:15.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.929+0000 7fadba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.929+0000 7fadb9e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4a7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.929+0000 7fadba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45174/0 (socket says 172.21.15.63:45174) 2024-05-27T08:08:15.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.929+0000 7fadba800700 1 -- 172.21.15.63:0/2477726258 learned_addr learned my addr 172.21.15.63:0/2477726258 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:15.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.929+0000 7fadb9400700 1 --2- 172.21.15.63:0/2477726258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 0x55d86e4b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.930+0000 7fadba800700 1 -- 172.21.15.63:0/2477726258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 msgr2=0x55d86e4b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:15.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.930+0000 7fadba800700 1 --2- 172.21.15.63:0/2477726258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 0x55d86e4b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.930+0000 7fadba800700 1 -- 172.21.15.63:0/2477726258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 msgr2=0x55d86e4a7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:15.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.930+0000 7fadba800700 1 --2- 172.21.15.63:0/2477726258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4a7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.930+0000 7fadba800700 1 -- 172.21.15.63:0/2477726258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d86e4b8f60 con 0x55d86e4b0c30 2024-05-27T08:08:15.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.930+0000 7fadba800700 1 --2- 172.21.15.63:0/2477726258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4b4c40 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7fadb4000f80 tx=0x7fadb40098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffb21f880fe15b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:15.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.931+0000 7fada6a00700 1 -- 172.21.15.63:0/2477726258 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4004a50 con 0x55d86e4b0c30 2024-05-27T08:08:15.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.931+0000 7fada6a00700 1 -- 172.21.15.63:0/2477726258 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadb400be50 con 0x55d86e4b0c30 2024-05-27T08:08:15.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.931+0000 7fada6a00700 1 -- 172.21.15.63:0/2477726258 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4005040 con 0x55d86e4b0c30 2024-05-27T08:08:15.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.932+0000 7fadcf57ab80 1 -- 172.21.15.63:0/2477726258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 msgr2=0x55d86e4b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:15.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.932+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/2477726258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4b4c40 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7fadb4000f80 tx=0x7fadb40098a0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.932+0000 7fadcf57ab80 1 -- 172.21.15.63:0/2477726258 shutdown_connections 2024-05-27T08:08:15.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.932+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/2477726258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4b4c40 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.932+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/2477726258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e32de40 0x55d86e4a7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.932+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/2477726258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e4b0820 0x55d86e4b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.932+0000 7fadcf57ab80 1 -- 172.21.15.63:0/2477726258 >> 172.21.15.63:0/2477726258 conn(0x55d86e38dfc0 msgr2=0x55d86e4a77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:15.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.932+0000 7fadcf57ab80 1 -- 172.21.15.63:0/2477726258 shutdown_connections 2024-05-27T08:08:15.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.932+0000 7fadcf57ab80 1 -- 172.21.15.63:0/2477726258 wait complete. 2024-05-27T08:08:15.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.933+0000 7fadcf57ab80 1 -- start start 2024-05-27T08:08:15.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.933+0000 7fadcf57ab80 1 Processor -- start 2024-05-27T08:08:15.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e32de40 0x55d86e4c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e4b0820 0x55d86e4c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadcf57ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4be7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1ac6a0 con 0x55d86e4b0c30 2024-05-27T08:08:15.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1adb10 con 0x55d86e4b0820 2024-05-27T08:08:15.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadcf57ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1acd70 con 0x55d86e32de40 2024-05-27T08:08:15.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadcf57ab80 10 client.0 _ll_register_callbacks cb 0x55d86e4cd6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:15.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4be7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadba800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4be7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45186/0 (socket says 172.21.15.63:45186) 2024-05-27T08:08:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadba800700 1 -- 172.21.15.63:0/1999588173 learned_addr learned my addr 172.21.15.63:0/1999588173 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadb9e00700 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e32de40 0x55d86e4c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.934+0000 7fadb9400700 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e4b0820 0x55d86e4c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.935+0000 7fadba800700 1 -- 172.21.15.63:0/1999588173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e32de40 msgr2=0x55d86e4c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.935+0000 7fadba800700 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e32de40 0x55d86e4c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.935+0000 7fadba800700 1 -- 172.21.15.63:0/1999588173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e4b0820 msgr2=0x55d86e4c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.935+0000 7fadba800700 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e4b0820 0x55d86e4c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.935+0000 7fadba800700 1 -- 172.21.15.63:0/1999588173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d86e4bef40 con 0x55d86e4b0c30 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.935+0000 7fadba800700 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4be7a0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b8e0 tx=0x7fadb400bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f128ec8a26e2c765 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.935+0000 7fad9be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.935+0000 7fad9be00700 1 -- 172.21.15.63:0/1999588173 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40077c0 con 0x55d86e4b0c30 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.936+0000 7fadcf57ab80 1 -- 172.21.15.63:0/1999588173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb400be50 con 0x55d86e4b0c30 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.936+0000 7fad9be00700 1 -- 172.21.15.63:0/1999588173 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7fadb4004a50 con 0x55d86e4b0c30 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.936+0000 7fad9be00700 1 -- 172.21.15.63:0/1999588173 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fadb4007d80 con 0x55d86e4b0c30 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.936+0000 7fadcf57ab80 10 client.22077 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.936+0000 7fadcf57ab80 1 -- 172.21.15.63:0/1999588173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d86e5353a0 con 0x55d86e4b0c30 2024-05-27T08:08:15.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.936+0000 7fad9be00700 1 -- 172.21.15.63:0/1999588173 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadb400be50 con 0x55d86e4b0c30 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.936+0000 7fad9be00700 1 -- 172.21.15.63:0/1999588173 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fadb4011070 con 0x55d86e4b0c30 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.937+0000 7fadcf57ab80 10 client.22077 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.937+0000 7fadcf57ab80 10 client.22077 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.937+0000 7fadcf57ab80 20 client.22077 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.937+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d86e535620 0x55d86e537a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.937+0000 7fadcf57ab80 4 client.22077 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.937+0000 7fadcf57ab80 1 -- 172.21.15.63:0/1999588173 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d86e3d6570 con 0x55d86e535620 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.937+0000 7fadb9e00700 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d86e535620 0x55d86e537a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.938+0000 7fadb9e00700 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d86e535620 0x55d86e537a40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=830a6126b0b91a96 server_cookie=51c6a53349f675de in_seq=0 out_seq=0 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.938+0000 7fad9be00700 10 client.22077 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fad9be00700 1 -- 172.21.15.63:0/1999588173 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d86e3d6570 con 0x55d86e535620 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fad9be00700 10 client.22077 handle_command_reply: tid=0 2024-05-27T08:08:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fadcf57ab80 1 client.22077 shutdown 2024-05-27T08:08:15.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fadcf57ab80 1 -- 172.21.15.63:0/1999588173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 msgr2=0x55d86e4be7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:15.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4be7a0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fadb400b8e0 tx=0x7fadb400bbe0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fadcf57ab80 1 -- 172.21.15.63:0/1999588173 shutdown_connections 2024-05-27T08:08:15.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55d86e535620 0x55d86e537a40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d86e4b0c30 0x55d86e4be7a0 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d86e32de40 0x55d86e4c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fad9be00700 0 client.22077 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:08:15.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fadcf57ab80 1 --2- 172.21.15.63:0/1999588173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d86e4b0820 0x55d86e4c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.939+0000 7fadcf57ab80 1 -- 172.21.15.63:0/1999588173 >> 172.21.15.63:0/1999588173 conn(0x55d86e38dfc0 msgr2=0x55d86e4ba7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:15.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.940+0000 7fadcf57ab80 1 -- 172.21.15.63:0/1999588173 shutdown_connections 2024-05-27T08:08:15.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.940+0000 7fadcf57ab80 1 -- 172.21.15.63:0/1999588173 wait complete. 2024-05-27T08:08:15.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.940+0000 7fadcf57ab80 20 client.22077 trim_cache size 0 max 16384 2024-05-27T08:08:15.942 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:08:15.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 -- 172.21.15.63:0/2973862146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fada00b29f0 msgr2=0x7fada00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:15.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fada00b29f0 0x7fada00b4f00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fadbc102600 tx=0x7fada8009260 comp rx=0 tx=0).stop 2024-05-27T08:08:15.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 -- 172.21.15.63:0/2973862146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadbc105280 msgr2=0x7fadbc1a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:15.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadbc105280 0x7fadbc1a06f0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fadac00d980 tx=0x7fadac00dd40 comp rx=0 tx=0).stop 2024-05-27T08:08:15.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 -- 172.21.15.63:0/2973862146 shutdown_connections 2024-05-27T08:08:15.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fada00b29f0 0x7fada00b4f00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc10e6f0 0x7fadbc19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadbc105280 0x7fadbc1a06f0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 --2- 172.21.15.63:0/2973862146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fadbc105bd0 0x7fadbc1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:15.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 -- 172.21.15.63:0/2973862146 >> 172.21.15.63:0/2973862146 conn(0x7fadbc078ab0 msgr2=0x7fadbc0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:15.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 -- 172.21.15.63:0/2973862146 shutdown_connections 2024-05-27T08:08:15.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:15.941+0000 7fadbb200700 1 -- 172.21.15.63:0/2973862146 wait complete. 2024-05-27T08:08:15.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:15 smithi086 ceph-mon[34687]: pgmap v1100: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:15.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:15 smithi149 ceph-mon[35408]: pgmap v1100: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:17.386 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:08:17.897 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:08:17.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:17 smithi086 ceph-mon[34687]: pgmap v1101: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:17.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:17 smithi149 ceph-mon[35408]: pgmap v1101: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:17.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:17 smithi063 ceph-mon[31613]: pgmap v1101: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:19.903 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:19 smithi063 ceph-mon[31613]: pgmap v1102: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:19.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:19 smithi086 ceph-mon[34687]: pgmap v1102: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:19 smithi149 ceph-mon[35408]: pgmap v1102: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.277+0000 7f6a03000700 1 -- 172.21.15.63:0/1789445708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69fc105bd0 msgr2=0x7f69fc10e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.277+0000 7f6a03000700 1 --2- 172.21.15.63:0/1789445708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69fc105bd0 0x7f69fc10e120 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009990 tx=0x7f69e8009ca0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.278+0000 7f6a03000700 1 -- 172.21.15.63:0/1789445708 shutdown_connections 2024-05-27T08:08:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.278+0000 7f6a03000700 1 --2- 172.21.15.63:0/1789445708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69fc10e6f0 0x7f69fc110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.278+0000 7f6a03000700 1 --2- 172.21.15.63:0/1789445708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc105280 0x7f69fc105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.278+0000 7f6a03000700 1 --2- 172.21.15.63:0/1789445708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69fc105bd0 0x7f69fc10e120 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.278+0000 7f6a03000700 1 -- 172.21.15.63:0/1789445708 >> 172.21.15.63:0/1789445708 conn(0x7f69fc078ab0 msgr2=0x7f69fc078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:20.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.278+0000 7f6a03000700 1 -- 172.21.15.63:0/1789445708 shutdown_connections 2024-05-27T08:08:20.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.279+0000 7f6a03000700 1 -- 172.21.15.63:0/1789445708 wait complete. 2024-05-27T08:08:20.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.280+0000 7f6a03000700 1 Processor -- start 2024-05-27T08:08:20.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.280+0000 7f6a03000700 1 -- start start 2024-05-27T08:08:20.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.280+0000 7f6a03000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69fc105280 0x7f69fc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.280+0000 7f6a03000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69fc105bd0 0x7f69fc19d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.280+0000 7f6a03000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc10e6f0 0x7f69fc1a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.280+0000 7f6a03000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc114150 con 0x7f69fc105280 2024-05-27T08:08:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a03000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc113fd0 con 0x7f69fc10e6f0 2024-05-27T08:08:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a03000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc1142d0 con 0x7f69fc105bd0 2024-05-27T08:08:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a01c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69fc105280 0x7f69fc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a01c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69fc105280 0x7f69fc19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47980/0 (socket says 172.21.15.63:47980) 2024-05-27T08:08:20.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a01c00700 1 -- 172.21.15.63:0/403310584 learned_addr learned my addr 172.21.15.63:0/403310584 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:20.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a01200700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69fc105bd0 0x7f69fc19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a02600700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc10e6f0 0x7f69fc1a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a01c00700 1 -- 172.21.15.63:0/403310584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69fc105bd0 msgr2=0x7f69fc19d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:20.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a01c00700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69fc105bd0 0x7f69fc19d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a01c00700 1 -- 172.21.15.63:0/403310584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc10e6f0 msgr2=0x7f69fc1a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:20.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.281+0000 7f6a01c00700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc10e6f0 0x7f69fc1a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.282+0000 7f6a01c00700 1 -- 172.21.15.63:0/403310584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc1a1c30 con 0x7f69fc105280 2024-05-27T08:08:20.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.282+0000 7f6a01c00700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69fc105280 0x7f69fc19cd60 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f69f000d950 tx=0x7f69f000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:20.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.282+0000 7f69fa000700 1 -- 172.21.15.63:0/403310584 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0016070 con 0x7f69fc105280 2024-05-27T08:08:20.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.282+0000 7f69fa000700 1 -- 172.21.15.63:0/403310584 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69f0010460 con 0x7f69fc105280 2024-05-27T08:08:20.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.282+0000 7f69fa000700 1 -- 172.21.15.63:0/403310584 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f000f620 con 0x7f69fc105280 2024-05-27T08:08:20.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.283+0000 7f6a03000700 1 -- 172.21.15.63:0/403310584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc1a1f20 con 0x7f69fc105280 2024-05-27T08:08:20.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.283+0000 7f6a03000700 1 -- 172.21.15.63:0/403310584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc1a9840 con 0x7f69fc105280 2024-05-27T08:08:20.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.287+0000 7f69fa000700 1 -- 172.21.15.63:0/403310584 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f69f0019070 con 0x7f69fc105280 2024-05-27T08:08:20.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.289+0000 7f6a13183b80 1 Processor -- start 2024-05-27T08:08:20.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.289+0000 7f6a13183b80 1 -- start start 2024-05-27T08:08:20.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.289+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 0x563b17687690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.289+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17507e40 0x563b17687bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.289+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17699ad0 0x563b1769bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.289+0000 7f6a13183b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563b175b4e40 con 0x563b17507e40 2024-05-27T08:08:20.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.289+0000 7f6a13183b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b175a49e0 con 0x563b175f39f0 2024-05-27T08:08:20.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.289+0000 7f6a13183b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563b175a8a20 con 0x563b17699ad0 2024-05-27T08:08:20.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.290+0000 7f69fa000700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69e40b2a70 0x7f69e40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.290+0000 7f6a01c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17507e40 0x563b17687bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.290+0000 7f6a02600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 0x563b17687690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.290+0000 7f6a01c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17507e40 0x563b17687bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47996/0 (socket says 172.21.15.63:47996) 2024-05-27T08:08:20.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.290+0000 7f6a01c00700 1 -- 172.21.15.63:0/2515266121 learned_addr learned my addr 172.21.15.63:0/2515266121 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:20.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.290+0000 7f6a01200700 1 --2- 172.21.15.63:0/2515266121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17699ad0 0x563b1769bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.291+0000 7f69fa000700 1 -- 172.21.15.63:0/403310584 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f69f00e24d0 con 0x7f69fc105280 2024-05-27T08:08:20.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.291+0000 7f6a02600700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69e40b2a70 0x7f69e40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.291+0000 7f6a01c00700 1 -- 172.21.15.63:0/2515266121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17699ad0 msgr2=0x563b1769bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:20.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.291+0000 7f6a01c00700 1 --2- 172.21.15.63:0/2515266121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17699ad0 0x563b1769bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.291+0000 7f6a01c00700 1 -- 172.21.15.63:0/2515266121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 msgr2=0x563b17687690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:20.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.291+0000 7f6a01c00700 1 --2- 172.21.15.63:0/2515266121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 0x563b17687690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.291+0000 7f6a01c00700 1 -- 172.21.15.63:0/2515266121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1756d420 con 0x563b17507e40 2024-05-27T08:08:20.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.291+0000 7f6a01c00700 1 --2- 172.21.15.63:0/2515266121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17507e40 0x563b17687bd0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f69f00a6700 tx=0x7f69f0077060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168036a79d07dba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:20.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.292+0000 7f6a02600700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69e40b2a70 0x7f69e40b4f80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f69f40060b0 tx=0x7f69f4004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:20.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.292+0000 7f69eea00700 1 -- 172.21.15.63:0/2515266121 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0020070 con 0x563b17507e40 2024-05-27T08:08:20.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.292+0000 7f69eea00700 1 -- 172.21.15.63:0/2515266121 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69f006ee90 con 0x563b17507e40 2024-05-27T08:08:20.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.292+0000 7f69eea00700 1 -- 172.21.15.63:0/2515266121 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f006e9f0 con 0x563b17507e40 2024-05-27T08:08:20.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.293+0000 7f6a13183b80 1 -- 172.21.15.63:0/2515266121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17507e40 msgr2=0x563b17687bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:20.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.293+0000 7f6a13183b80 1 --2- 172.21.15.63:0/2515266121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17507e40 0x563b17687bd0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f69f00a6700 tx=0x7f69f0077060 comp rx=0 tx=0).stop 2024-05-27T08:08:20.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.293+0000 7f6a13183b80 1 -- 172.21.15.63:0/2515266121 shutdown_connections 2024-05-27T08:08:20.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.293+0000 7f6a13183b80 1 --2- 172.21.15.63:0/2515266121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17699ad0 0x563b1769bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.293+0000 7f6a13183b80 1 --2- 172.21.15.63:0/2515266121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 0x563b17687690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.293+0000 7f6a13183b80 1 --2- 172.21.15.63:0/2515266121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17507e40 0x563b17687bd0 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.293+0000 7f6a13183b80 1 -- 172.21.15.63:0/2515266121 >> 172.21.15.63:0/2515266121 conn(0x563b175a1390 msgr2=0x563b17682710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:20.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.294+0000 7f6a13183b80 1 -- 172.21.15.63:0/2515266121 shutdown_connections 2024-05-27T08:08:20.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.294+0000 7f6a13183b80 1 -- 172.21.15.63:0/2515266121 wait complete. 2024-05-27T08:08:20.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.294+0000 7f6a13183b80 1 -- start start 2024-05-27T08:08:20.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.295+0000 7f6a13183b80 1 Processor -- start 2024-05-27T08:08:20.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.295+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b17507e40 0x563b17696ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b175f39f0 0x563b17697010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17699ad0 0x563b17697550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a13183b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563b174aafa0 con 0x563b175f39f0 2024-05-27T08:08:20.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a13183b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b17545760 con 0x563b17507e40 2024-05-27T08:08:20.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a13183b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563b1769c520 con 0x563b17699ad0 2024-05-27T08:08:20.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a13183b80 10 client.0 _ll_register_callbacks cb 0x563b1754ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a01200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b175f39f0 0x563b17697010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a01200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b175f39f0 0x563b17697010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48002/0 (socket says 172.21.15.63:48002) 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a01200700 1 -- 172.21.15.63:0/3216158412 learned_addr learned my addr 172.21.15.63:0/3216158412 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a01c00700 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b17507e40 0x563b17696ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.296+0000 7f6a02600700 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17699ad0 0x563b17697550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.297+0000 7f6a01200700 1 -- 172.21.15.63:0/3216158412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17699ad0 msgr2=0x563b17697550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.297+0000 7f6a01200700 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17699ad0 0x563b17697550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.297+0000 7f6a01200700 1 -- 172.21.15.63:0/3216158412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b17507e40 msgr2=0x563b17696ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.297+0000 7f6a01200700 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b17507e40 0x563b17696ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.297+0000 7f6a01200700 1 -- 172.21.15.63:0/3216158412 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b174f48d0 con 0x563b175f39f0 2024-05-27T08:08:20.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.297+0000 7f6a01200700 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b175f39f0 0x563b17697010 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009990 tx=0x7f69e8003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91e8f33a620d2b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:20.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.297+0000 7f69e3400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:20.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.298+0000 7f69e3400700 1 -- 172.21.15.63:0/3216158412 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8027060 con 0x563b175f39f0 2024-05-27T08:08:20.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.298+0000 7f69e3400700 1 -- 172.21.15.63:0/3216158412 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f69e8005230 con 0x563b175f39f0 2024-05-27T08:08:20.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.298+0000 7f6a13183b80 1 -- 172.21.15.63:0/3216158412 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f006ee90 con 0x563b175f39f0 2024-05-27T08:08:20.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.298+0000 7f69e3400700 1 -- 172.21.15.63:0/3216158412 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f69e802e050 con 0x563b175f39f0 2024-05-27T08:08:20.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.298+0000 7f6a13183b80 10 client.22089 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:20.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.298+0000 7f6a13183b80 1 -- 172.21.15.63:0/3216158412 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563b17575060 con 0x563b175f39f0 2024-05-27T08:08:20.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.298+0000 7f69e3400700 1 -- 172.21.15.63:0/3216158412 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69f006ee90 con 0x563b175f39f0 2024-05-27T08:08:20.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.299+0000 7f69e3400700 1 -- 172.21.15.63:0/3216158412 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f69e8021070 con 0x563b175f39f0 2024-05-27T08:08:20.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.299+0000 7f6a13183b80 10 client.22089 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:20.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.299+0000 7f6a13183b80 10 client.22089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:20.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.299+0000 7f6a13183b80 20 client.22089 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:20.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.299+0000 7f6a13183b80 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b176a94a0 0x563b176ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.299+0000 7f6a13183b80 4 client.22089 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:20.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.299+0000 7f6a13183b80 1 -- 172.21.15.63:0/3216158412 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b175b6d90 con 0x563b176a94a0 2024-05-27T08:08:20.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.300+0000 7f6a01c00700 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b176a94a0 0x563b176ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.301+0000 7f6a01c00700 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b176a94a0 0x563b176ab8c0 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8b08a86b9edeaa9 server_cookie=7758cc70f04aae54 in_seq=0 out_seq=0 2024-05-27T08:08:20.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.301+0000 7f69e3400700 10 client.22089 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:20.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f69e3400700 1 -- 172.21.15.63:0/3216158412 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563b175b6d90 con 0x563b176a94a0 2024-05-27T08:08:20.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f69e3400700 10 client.22089 handle_command_reply: tid=0 2024-05-27T08:08:20.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f6a13183b80 1 client.22089 shutdown 2024-05-27T08:08:20.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f6a13183b80 1 -- 172.21.15.63:0/3216158412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b175f39f0 msgr2=0x563b17697010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:20.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f6a13183b80 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b175f39f0 0x563b17697010 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009990 tx=0x7f69e8003ef0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f6a13183b80 1 -- 172.21.15.63:0/3216158412 shutdown_connections 2024-05-27T08:08:20.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f6a13183b80 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b176a94a0 0x563b176ab8c0 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f6a13183b80 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17699ad0 0x563b17697550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f6a13183b80 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b17507e40 0x563b17696ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f6a13183b80 1 --2- 172.21.15.63:0/3216158412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b175f39f0 0x563b17697010 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f6a13183b80 1 -- 172.21.15.63:0/3216158412 >> 172.21.15.63:0/3216158412 conn(0x563b175a1390 msgr2=0x563b17687710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:20.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.303+0000 7f69e3400700 0 client.22089 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:20.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.304+0000 7f6a13183b80 1 -- 172.21.15.63:0/3216158412 shutdown_connections 2024-05-27T08:08:20.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.304+0000 7f6a13183b80 1 -- 172.21.15.63:0/3216158412 wait complete. 2024-05-27T08:08:20.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.304+0000 7f6a13183b80 20 client.22089 trim_cache size 0 max 16384 2024-05-27T08:08:20.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.325+0000 7f6a13183b80 1 Processor -- start 2024-05-27T08:08:20.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.325+0000 7f6a13183b80 1 -- start start 2024-05-27T08:08:20.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.325+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17507e40 0x563b17682740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.326+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 0x563b17686830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.326+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b176a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.326+0000 7f6a13183b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc1b0a80 con 0x563b17699ad0 2024-05-27T08:08:20.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.326+0000 7f6a13183b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc1ae310 con 0x563b175f39f0 2024-05-27T08:08:20.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.326+0000 7f6a13183b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc1ac1e0 con 0x563b17507e40 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.326+0000 7f6a02600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b176a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.326+0000 7f6a02600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b176a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48012/0 (socket says 172.21.15.63:48012) 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.326+0000 7f6a02600700 1 -- 172.21.15.63:0/1279673676 learned_addr learned my addr 172.21.15.63:0/1279673676 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.326+0000 7f6a01c00700 1 --2- 172.21.15.63:0/1279673676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17507e40 0x563b17682740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.326+0000 7f6a01200700 1 --2- 172.21.15.63:0/1279673676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 0x563b17686830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.327+0000 7f6a02600700 1 -- 172.21.15.63:0/1279673676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17507e40 msgr2=0x563b17682740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.327+0000 7f6a02600700 1 --2- 172.21.15.63:0/1279673676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17507e40 0x563b17682740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.327+0000 7f6a02600700 1 -- 172.21.15.63:0/1279673676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 msgr2=0x563b17686830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.327+0000 7f6a02600700 1 --2- 172.21.15.63:0/1279673676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 0x563b17686830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.327+0000 7f6a02600700 1 -- 172.21.15.63:0/1279673676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1767c910 con 0x563b17699ad0 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.327+0000 7f6a01200700 1 --2- 172.21.15.63:0/1279673676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 0x563b17686830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:20.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.327+0000 7f6a02600700 1 --2- 172.21.15.63:0/1279673676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b176a55f0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f69f4007fc0 tx=0x7f69f400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76f58a1a08da03e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:20.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.327+0000 7f69ee000700 1 -- 172.21.15.63:0/1279673676 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4018b10 con 0x563b17699ad0 2024-05-27T08:08:20.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.328+0000 7f69ee000700 1 -- 172.21.15.63:0/1279673676 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69f4029070 con 0x563b17699ad0 2024-05-27T08:08:20.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.328+0000 7f69ee000700 1 -- 172.21.15.63:0/1279673676 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4023430 con 0x563b17699ad0 2024-05-27T08:08:20.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.329+0000 7f6a13183b80 1 -- 172.21.15.63:0/1279673676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 msgr2=0x563b176a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:20.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.329+0000 7f6a13183b80 1 --2- 172.21.15.63:0/1279673676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b176a55f0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f69f4007fc0 tx=0x7f69f400d000 comp rx=0 tx=0).stop 2024-05-27T08:08:20.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.329+0000 7f6a13183b80 1 -- 172.21.15.63:0/1279673676 shutdown_connections 2024-05-27T08:08:20.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.329+0000 7f6a13183b80 1 --2- 172.21.15.63:0/1279673676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b176a55f0 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.329+0000 7f6a13183b80 1 --2- 172.21.15.63:0/1279673676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b17507e40 0x563b17682740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.329+0000 7f6a13183b80 1 --2- 172.21.15.63:0/1279673676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b175f39f0 0x563b17686830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.329+0000 7f6a13183b80 1 -- 172.21.15.63:0/1279673676 >> 172.21.15.63:0/1279673676 conn(0x563b175a1390 msgr2=0x563b17686120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:20.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.329+0000 7f6a13183b80 1 -- 172.21.15.63:0/1279673676 shutdown_connections 2024-05-27T08:08:20.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.329+0000 7f6a13183b80 1 -- 172.21.15.63:0/1279673676 wait complete. 2024-05-27T08:08:20.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.330+0000 7f6a13183b80 1 -- start start 2024-05-27T08:08:20.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.330+0000 7f6a13183b80 1 Processor -- start 2024-05-27T08:08:20.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.331+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b17507e40 0x563b1769c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.331+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b175f39f0 0x563b1769cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.331+0000 7f6a13183b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b17711f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.331+0000 7f6a13183b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc1ac830 con 0x563b17699ad0 2024-05-27T08:08:20.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.331+0000 7f6a13183b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc1adc40 con 0x563b17507e40 2024-05-27T08:08:20.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.331+0000 7f6a13183b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc1acea0 con 0x563b175f39f0 2024-05-27T08:08:20.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.331+0000 7f6a13183b80 10 client.0 _ll_register_callbacks cb 0x563b1754ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:20.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.332+0000 7f6a02600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b17711f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.332+0000 7f6a02600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b17711f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48024/0 (socket says 172.21.15.63:48024) 2024-05-27T08:08:20.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.332+0000 7f6a02600700 1 -- 172.21.15.63:0/1487892616 learned_addr learned my addr 172.21.15.63:0/1487892616 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:20.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.332+0000 7f6a01c00700 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b17507e40 0x563b1769c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.332+0000 7f6a01200700 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b175f39f0 0x563b1769cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.332+0000 7f6a02600700 1 -- 172.21.15.63:0/1487892616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b175f39f0 msgr2=0x563b1769cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:20.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.332+0000 7f6a02600700 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b175f39f0 0x563b1769cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.332+0000 7f6a02600700 1 -- 172.21.15.63:0/1487892616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b17507e40 msgr2=0x563b1769c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.332+0000 7f6a02600700 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b17507e40 0x563b1769c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.332+0000 7f6a02600700 1 -- 172.21.15.63:0/1487892616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1769d580 con 0x563b17699ad0 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.333+0000 7f6a02600700 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b17711f10 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f69f4024f80 tx=0x7f69f4009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fc70db08ba0f1a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.333+0000 7f69e3400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.333+0000 7f69e3400700 1 -- 172.21.15.63:0/1487892616 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4018da0 con 0x563b17699ad0 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.333+0000 7f6a13183b80 1 -- 172.21.15.63:0/1487892616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f4029070 con 0x563b17699ad0 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.333+0000 7f69e3400700 1 -- 172.21.15.63:0/1487892616 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f69f4012050 con 0x563b17699ad0 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.334+0000 7f69e3400700 1 -- 172.21.15.63:0/1487892616 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f69f403c050 con 0x563b17699ad0 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.334+0000 7f6a13183b80 10 client.22095 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:20.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.334+0000 7f6a13183b80 1 -- 172.21.15.63:0/1487892616 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563b177133e0 con 0x563b17699ad0 2024-05-27T08:08:20.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.334+0000 7f69e3400700 1 -- 172.21.15.63:0/1487892616 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69f4029070 con 0x563b17699ad0 2024-05-27T08:08:20.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.335+0000 7f69e3400700 1 -- 172.21.15.63:0/1487892616 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f69f4004140 con 0x563b17699ad0 2024-05-27T08:08:20.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.335+0000 7f6a13183b80 10 client.22095 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:20.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.335+0000 7f6a13183b80 10 client.22095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:20.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.335+0000 7f6a13183b80 20 client.22095 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:20.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.335+0000 7f6a13183b80 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b17713a50 0x563b17715e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:20.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.335+0000 7f6a13183b80 4 client.22095 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:08:20.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.335+0000 7f6a13183b80 1 -- 172.21.15.63:0/1487892616 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b17687ff0 con 0x563b17713a50 2024-05-27T08:08:20.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.336+0000 7f6a01c00700 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b17713a50 0x563b17715e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.337+0000 7f6a01c00700 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b17713a50 0x563b17715e70 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84dafa11f545d306 server_cookie=e4614355c995f39c in_seq=0 out_seq=0 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.337+0000 7f69e3400700 10 client.22095 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.337+0000 7f69e3400700 1 -- 172.21.15.63:0/1487892616 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b17687ff0 con 0x563b17713a50 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.337+0000 7f69e3400700 10 client.22095 handle_command_reply: tid=0 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.337+0000 7f6a13183b80 1 client.22095 shutdown 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.337+0000 7f6a13183b80 1 -- 172.21.15.63:0/1487892616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 msgr2=0x563b17711f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.337+0000 7f6a13183b80 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b17711f10 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f69f4024f80 tx=0x7f69f4009ab0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.337+0000 7f6a13183b80 1 -- 172.21.15.63:0/1487892616 shutdown_connections 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.337+0000 7f6a13183b80 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b17713a50 0x563b17715e70 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.338+0000 7f6a13183b80 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b17699ad0 0x563b17711f10 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.338+0000 7f6a13183b80 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b17507e40 0x563b1769c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.338+0000 7f6a13183b80 1 --2- 172.21.15.63:0/1487892616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b175f39f0 0x563b1769cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.338+0000 7f6a13183b80 1 -- 172.21.15.63:0/1487892616 >> 172.21.15.63:0/1487892616 conn(0x563b175a1390 msgr2=0x563b176968b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:20.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.338+0000 7f69e3400700 0 client.22095 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:20.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.338+0000 7f6a13183b80 1 -- 172.21.15.63:0/1487892616 shutdown_connections 2024-05-27T08:08:20.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.338+0000 7f6a13183b80 1 -- 172.21.15.63:0/1487892616 wait complete. 2024-05-27T08:08:20.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.338+0000 7f6a13183b80 20 client.22095 trim_cache size 0 max 16384 2024-05-27T08:08:20.340 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:08:20.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 -- 172.21.15.63:0/403310584 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69e40b2a70 msgr2=0x7f69e40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:20.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69e40b2a70 0x7f69e40b4f80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f69f40060b0 tx=0x7f69f4004ad0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 -- 172.21.15.63:0/403310584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69fc105280 msgr2=0x7f69fc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:20.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69fc105280 0x7f69fc19cd60 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f69f000d950 tx=0x7f69f000dd10 comp rx=0 tx=0).stop 2024-05-27T08:08:20.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 -- 172.21.15.63:0/403310584 shutdown_connections 2024-05-27T08:08:20.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f69e40b2a70 0x7f69e40b4f80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69fc10e6f0 0x7f69fc1a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69fc105280 0x7f69fc19cd60 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 --2- 172.21.15.63:0/403310584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69fc105bd0 0x7f69fc19d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:20.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 -- 172.21.15.63:0/403310584 >> 172.21.15.63:0/403310584 conn(0x7f69fc078ab0 msgr2=0x7f69fc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:20.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.339+0000 7f6a03000700 1 -- 172.21.15.63:0/403310584 shutdown_connections 2024-05-27T08:08:20.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:20.340+0000 7f6a03000700 1 -- 172.21.15.63:0/403310584 wait complete. 2024-05-27T08:08:21.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:21 smithi086 ceph-mon[34687]: pgmap v1103: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:21.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:21 smithi149 ceph-mon[35408]: pgmap v1103: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:21.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:21 smithi063 ceph-mon[31613]: pgmap v1103: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:08:22.169 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:08:22.169 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:08:22.676 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:08:23.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:23 smithi086 ceph-mon[34687]: pgmap v1104: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:08:23.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:23 smithi149 ceph-mon[35408]: pgmap v1104: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:08:23.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:23 smithi063 ceph-mon[31613]: pgmap v1104: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:08:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.068+0000 7f264b200700 1 -- 172.21.15.63:0/1825423381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264410af40 msgr2=0x7f264410b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.068+0000 7f264b200700 1 --2- 172.21.15.63:0/1825423381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264410af40 0x7f264410b350 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f26340099c0 tx=0x7f2634009cd0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.069+0000 7f264b200700 1 -- 172.21.15.63:0/1825423381 shutdown_connections 2024-05-27T08:08:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.069+0000 7f264b200700 1 --2- 172.21.15.63:0/1825423381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264410b890 0x7f26441131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.069+0000 7f264b200700 1 --2- 172.21.15.63:0/1825423381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2644106610 0x7f2644106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.069+0000 7f264b200700 1 --2- 172.21.15.63:0/1825423381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264410af40 0x7f264410b350 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.069+0000 7f264b200700 1 -- 172.21.15.63:0/1825423381 >> 172.21.15.63:0/1825423381 conn(0x7f2644102020 msgr2=0x7f2644104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.069+0000 7f264b200700 1 -- 172.21.15.63:0/1825423381 shutdown_connections 2024-05-27T08:08:25.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.069+0000 7f264b200700 1 -- 172.21.15.63:0/1825423381 wait complete. 2024-05-27T08:08:25.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.071+0000 7f264b200700 1 Processor -- start 2024-05-27T08:08:25.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.071+0000 7f264b200700 1 -- start start 2024-05-27T08:08:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.071+0000 7f264b200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2644106610 0x7f264419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.071+0000 7f264b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264410af40 0x7f264419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.071+0000 7f264b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264410b890 0x7f26441a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.071+0000 7f264b200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26441163a0 con 0x7f264410b890 2024-05-27T08:08:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.071+0000 7f264b200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2644116220 con 0x7f264410af40 2024-05-27T08:08:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.071+0000 7f264b200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2644116520 con 0x7f2644106610 2024-05-27T08:08:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.072+0000 7f264a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264410b890 0x7f26441a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.072+0000 7f2649400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264410af40 0x7f264419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.072+0000 7f264a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264410b890 0x7f26441a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48036/0 (socket says 172.21.15.63:48036) 2024-05-27T08:08:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.072+0000 7f264a800700 1 -- 172.21.15.63:0/3731074943 learned_addr learned my addr 172.21.15.63:0/3731074943 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.072+0000 7f2649e00700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2644106610 0x7f264419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.073+0000 7f264a800700 1 -- 172.21.15.63:0/3731074943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2644106610 msgr2=0x7f264419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.073+0000 7f264a800700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2644106610 0x7f264419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.073+0000 7f264a800700 1 -- 172.21.15.63:0/3731074943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264410af40 msgr2=0x7f264419d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.073+0000 7f264a800700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264410af40 0x7f264419d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.073+0000 7f264a800700 1 -- 172.21.15.63:0/3731074943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26441a1ca0 con 0x7f264410b890 2024-05-27T08:08:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.073+0000 7f2649400700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264410af40 0x7f264419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:08:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.073+0000 7f264a800700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264410b890 0x7f26441a1600 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f264000ada0 tx=0x7f264000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.074+0000 7f263ea00700 1 -- 172.21.15.63:0/3731074943 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640017020 con 0x7f264410b890 2024-05-27T08:08:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.074+0000 7f263ea00700 1 -- 172.21.15.63:0/3731074943 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f264000cd60 con 0x7f264410b890 2024-05-27T08:08:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.074+0000 7f264b200700 1 -- 172.21.15.63:0/3731074943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26441a1f90 con 0x7f264410b890 2024-05-27T08:08:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.074+0000 7f263ea00700 1 -- 172.21.15.63:0/3731074943 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640005280 con 0x7f264410b890 2024-05-27T08:08:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.074+0000 7f264b200700 1 -- 172.21.15.63:0/3731074943 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26441a98b0 con 0x7f264410b890 2024-05-27T08:08:25.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.080+0000 7f263ea00700 1 -- 172.21.15.63:0/3731074943 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2640005420 con 0x7f264410b890 2024-05-27T08:08:25.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.081+0000 7f265b408b80 1 Processor -- start 2024-05-27T08:08:25.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.081+0000 7f265b408b80 1 -- start start 2024-05-27T08:08:25.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.081+0000 7f265b408b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200bf7ec0 0x55c200d75f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.081+0000 7f265b408b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d80b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.081+0000 7f265b408b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.081+0000 7f265b408b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c200ca4ec0 con 0x55c200d768a0 2024-05-27T08:08:25.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.081+0000 7f265b408b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c200c94a60 con 0x55c200d76490 2024-05-27T08:08:25.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.081+0000 7f265b408b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c200c98aa0 con 0x55c200bf7ec0 2024-05-27T08:08:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.082+0000 7f2649400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.082+0000 7f2649400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d830a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48044/0 (socket says 172.21.15.63:48044) 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.082+0000 7f2649400700 1 -- 172.21.15.63:0/3922284140 learned_addr learned my addr 172.21.15.63:0/3922284140 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.082+0000 7f2649400700 1 --2- 172.21.15.63:0/3922284140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200bf7ec0 0x55c200d75f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.082+0000 7f2649e00700 1 --2- 172.21.15.63:0/3922284140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d80b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.082+0000 7f263ea00700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f262c0b2a70 0x7f262c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.082+0000 7f263ea00700 1 -- 172.21.15.63:0/3731074943 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f26400e20e0 con 0x7f264410b890 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.083+0000 7f2649400700 1 -- 172.21.15.63:0/3922284140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200bf7ec0 msgr2=0x55c200d75f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.083+0000 7f2649400700 1 --2- 172.21.15.63:0/3922284140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200bf7ec0 0x55c200d75f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.083+0000 7f2649400700 1 -- 172.21.15.63:0/3922284140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 msgr2=0x55c200d80b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.083+0000 7f2649400700 1 --2- 172.21.15.63:0/3922284140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d80b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.083+0000 7f2649400700 1 -- 172.21.15.63:0/3922284140 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c200c5d4a0 con 0x55c200d768a0 2024-05-27T08:08:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.083+0000 7f2649400700 1 --2- 172.21.15.63:0/3922284140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d830a0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f2634005f50 tx=0x7f2634005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db05b7bb3caf7fda server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.083+0000 7f264a800700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f262c0b2a70 0x7f262c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.084+0000 7f2633400700 1 -- 172.21.15.63:0/3922284140 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634005230 con 0x55c200d768a0 2024-05-27T08:08:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.084+0000 7f2633400700 1 -- 172.21.15.63:0/3922284140 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2634003680 con 0x55c200d768a0 2024-05-27T08:08:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.084+0000 7f2633400700 1 -- 172.21.15.63:0/3922284140 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263400f460 con 0x55c200d768a0 2024-05-27T08:08:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.085+0000 7f265b408b80 1 -- 172.21.15.63:0/3922284140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 msgr2=0x55c200d830a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.085+0000 7f265b408b80 1 --2- 172.21.15.63:0/3922284140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d830a0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f2634005f50 tx=0x7f2634005e80 comp rx=0 tx=0).stop 2024-05-27T08:08:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.085+0000 7f264a800700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f262c0b2a70 0x7f262c0b4f80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f26400a50b0 tx=0x7f2640007b30 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.085+0000 7f265b408b80 1 -- 172.21.15.63:0/3922284140 shutdown_connections 2024-05-27T08:08:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.085+0000 7f265b408b80 1 --2- 172.21.15.63:0/3922284140 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d830a0 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.085+0000 7f265b408b80 1 --2- 172.21.15.63:0/3922284140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200bf7ec0 0x55c200d75f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.085+0000 7f265b408b80 1 --2- 172.21.15.63:0/3922284140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d80b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.085+0000 7f265b408b80 1 -- 172.21.15.63:0/3922284140 >> 172.21.15.63:0/3922284140 conn(0x55c200c58040 msgr2=0x55c200d71db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.085+0000 7f265b408b80 1 -- 172.21.15.63:0/3922284140 shutdown_connections 2024-05-27T08:08:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.085+0000 7f265b408b80 1 -- 172.21.15.63:0/3922284140 wait complete. 2024-05-27T08:08:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.086+0000 7f265b408b80 1 -- start start 2024-05-27T08:08:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.086+0000 7f265b408b80 1 Processor -- start 2024-05-27T08:08:25.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.087+0000 7f265b408b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.087+0000 7f265b408b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d80c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.087+0000 7f265b408b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200d768a0 0x55c200d81140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.087+0000 7f265b408b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c200b9b020 con 0x55c200bf7ec0 2024-05-27T08:08:25.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.087+0000 7f265b408b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c200c357e0 con 0x55c200d76490 2024-05-27T08:08:25.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.087+0000 7f265b408b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c200d839b0 con 0x55c200d768a0 2024-05-27T08:08:25.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.087+0000 7f265b408b80 10 client.0 _ll_register_callbacks cb 0x55c200c3ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:25.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.087+0000 7f2649400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.087+0000 7f2649400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d806c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48056/0 (socket says 172.21.15.63:48056) 2024-05-27T08:08:25.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.087+0000 7f2649400700 1 -- 172.21.15.63:0/71962769 learned_addr learned my addr 172.21.15.63:0/71962769 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:25.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.088+0000 7f2649e00700 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d80c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.088+0000 7f2649400700 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200d768a0 0x55c200d81140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.088+0000 7f2649400700 1 -- 172.21.15.63:0/71962769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200d768a0 msgr2=0x55c200d81140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.088+0000 7f2649400700 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200d768a0 0x55c200d81140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.088+0000 7f2649400700 1 -- 172.21.15.63:0/71962769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 msgr2=0x55c200d80c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.088+0000 7f2649400700 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d80c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.088+0000 7f2649400700 1 -- 172.21.15.63:0/71962769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c200be4950 con 0x55c200bf7ec0 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.088+0000 7f2649e00700 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d80c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.088+0000 7f2649400700 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d806c0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f2634006010 tx=0x7f263401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20456440d686f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.088+0000 7f2627e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.089+0000 7f2627e00700 1 -- 172.21.15.63:0/71962769 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634003c70 con 0x55c200bf7ec0 2024-05-27T08:08:25.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.089+0000 7f2627e00700 1 -- 172.21.15.63:0/71962769 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f2634027480 con 0x55c200bf7ec0 2024-05-27T08:08:25.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.089+0000 7f265b408b80 1 -- 172.21.15.63:0/71962769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2634003680 con 0x55c200bf7ec0 2024-05-27T08:08:25.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.089+0000 7f2627e00700 1 -- 172.21.15.63:0/71962769 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2634027680 con 0x55c200bf7ec0 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.089+0000 7f265b408b80 10 client.22107 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.089+0000 7f265b408b80 1 -- 172.21.15.63:0/71962769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c200c650e0 con 0x55c200bf7ec0 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.090+0000 7f2627e00700 1 -- 172.21.15.63:0/71962769 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2634003680 con 0x55c200bf7ec0 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.090+0000 7f2627e00700 1 -- 172.21.15.63:0/71962769 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f263400f800 con 0x55c200bf7ec0 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.090+0000 7f265b408b80 10 client.22107 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.090+0000 7f265b408b80 10 client.22107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.090+0000 7f265b408b80 20 client.22107 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.090+0000 7f265b408b80 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c200d908e0 0x55c200d92d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.090+0000 7f265b408b80 4 client.22107 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.090+0000 7f265b408b80 1 -- 172.21.15.63:0/71962769 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c200ca6e10 con 0x55c200d908e0 2024-05-27T08:08:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.091+0000 7f2649e00700 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c200d908e0 0x55c200d92d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.092+0000 7f2627e00700 10 client.22107 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:25.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.092+0000 7f2649e00700 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c200d908e0 0x55c200d92d00 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20604fa3a4985445 server_cookie=a250e617d1233cfa in_seq=0 out_seq=0 2024-05-27T08:08:25.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.094+0000 7f2627e00700 1 -- 172.21.15.63:0/71962769 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c200ca6e10 con 0x55c200d908e0 2024-05-27T08:08:25.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.094+0000 7f2627e00700 10 client.22107 handle_command_reply: tid=0 2024-05-27T08:08:25.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.094+0000 7f265b408b80 1 client.22107 shutdown 2024-05-27T08:08:25.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.094+0000 7f265b408b80 1 -- 172.21.15.63:0/71962769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 msgr2=0x55c200d806c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:25.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.094+0000 7f265b408b80 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d806c0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f2634006010 tx=0x7f263401a040 comp rx=0 tx=0).stop 2024-05-27T08:08:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.094+0000 7f265b408b80 1 -- 172.21.15.63:0/71962769 shutdown_connections 2024-05-27T08:08:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.095+0000 7f265b408b80 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c200d908e0 0x55c200d92d00 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.095+0000 7f265b408b80 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200d768a0 0x55c200d81140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.095+0000 7f265b408b80 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d806c0 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.095+0000 7f265b408b80 1 --2- 172.21.15.63:0/71962769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d80c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.095+0000 7f265b408b80 1 -- 172.21.15.63:0/71962769 >> 172.21.15.63:0/71962769 conn(0x55c200c58040 msgr2=0x55c200d71070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.095+0000 7f2627e00700 0 client.22107 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.095+0000 7f265b408b80 1 -- 172.21.15.63:0/71962769 shutdown_connections 2024-05-27T08:08:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.095+0000 7f265b408b80 1 -- 172.21.15.63:0/71962769 wait complete. 2024-05-27T08:08:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.095+0000 7f265b408b80 20 client.22107 trim_cache size 0 max 16384 2024-05-27T08:08:25.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.120+0000 7f265b408b80 1 Processor -- start 2024-05-27T08:08:25.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.120+0000 7f265b408b80 1 -- start start 2024-05-27T08:08:25.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.120+0000 7f265b408b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d82b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.120+0000 7f265b408b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200d76490 0x55c200d7be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.120+0000 7f265b408b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d768a0 0x55c200d95d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.120+0000 7f265b408b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2628011610 con 0x55c200bf7ec0 2024-05-27T08:08:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.120+0000 7f265b408b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f262800eea0 con 0x55c200d768a0 2024-05-27T08:08:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.120+0000 7f265b408b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f262800cd10 con 0x55c200d76490 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.121+0000 7f2649400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d82b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.121+0000 7f2649400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d82b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48072/0 (socket says 172.21.15.63:48072) 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.121+0000 7f2649400700 1 -- 172.21.15.63:0/930767874 learned_addr learned my addr 172.21.15.63:0/930767874 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.121+0000 7f2649e00700 1 --2- 172.21.15.63:0/930767874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200d76490 0x55c200d7be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.121+0000 7f2649400700 1 -- 172.21.15.63:0/930767874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200d76490 msgr2=0x55c200d7be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.121+0000 7f2649400700 1 --2- 172.21.15.63:0/930767874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200d76490 0x55c200d7be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.121+0000 7f2649400700 1 -- 172.21.15.63:0/930767874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d768a0 msgr2=0x55c200d95d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.121+0000 7f2649400700 1 --2- 172.21.15.63:0/930767874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d768a0 0x55c200d95d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.121+0000 7f2649400700 1 -- 172.21.15.63:0/930767874 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c200d834c0 con 0x55c200bf7ec0 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.122+0000 7f2649400700 1 --2- 172.21.15.63:0/930767874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d82b60 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f2634005f50 tx=0x7f263400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33baf5d01fd5367c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:25.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.122+0000 7f2632a00700 1 -- 172.21.15.63:0/930767874 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263402c040 con 0x55c200bf7ec0 2024-05-27T08:08:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.122+0000 7f2632a00700 1 -- 172.21.15.63:0/930767874 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2634042070 con 0x55c200bf7ec0 2024-05-27T08:08:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.122+0000 7f2632a00700 1 -- 172.21.15.63:0/930767874 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634003c70 con 0x55c200bf7ec0 2024-05-27T08:08:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.123+0000 7f265b408b80 1 -- 172.21.15.63:0/930767874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 msgr2=0x55c200d82b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.123+0000 7f265b408b80 1 --2- 172.21.15.63:0/930767874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d82b60 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f2634005f50 tx=0x7f263400bee0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.123+0000 7f265b408b80 1 -- 172.21.15.63:0/930767874 shutdown_connections 2024-05-27T08:08:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.123+0000 7f265b408b80 1 --2- 172.21.15.63:0/930767874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d768a0 0x55c200d95d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.123+0000 7f265b408b80 1 --2- 172.21.15.63:0/930767874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200bf7ec0 0x55c200d82b60 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.123+0000 7f265b408b80 1 --2- 172.21.15.63:0/930767874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200d76490 0x55c200d7be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.123+0000 7f265b408b80 1 -- 172.21.15.63:0/930767874 >> 172.21.15.63:0/930767874 conn(0x55c200c58040 msgr2=0x55c200d80ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.124+0000 7f265b408b80 1 -- 172.21.15.63:0/930767874 shutdown_connections 2024-05-27T08:08:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.124+0000 7f265b408b80 1 -- 172.21.15.63:0/930767874 wait complete. 2024-05-27T08:08:25.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.124+0000 7f265b408b80 1 -- start start 2024-05-27T08:08:25.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.125+0000 7f265b408b80 1 Processor -- start 2024-05-27T08:08:25.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.125+0000 7f265b408b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200bf7ec0 0x55c200d960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.125+0000 7f265b408b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d94460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.125+0000 7f265b408b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d949a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.125+0000 7f265b408b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f262800d360 con 0x55c200d768a0 2024-05-27T08:08:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.125+0000 7f265b408b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f262800e7d0 con 0x55c200d76490 2024-05-27T08:08:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.125+0000 7f265b408b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f262800da30 con 0x55c200bf7ec0 2024-05-27T08:08:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.125+0000 7f265b408b80 10 client.0 _ll_register_callbacks cb 0x55c200c3ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.126+0000 7f2649400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.126+0000 7f2649400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d949a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48088/0 (socket says 172.21.15.63:48088) 2024-05-27T08:08:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.126+0000 7f2649400700 1 -- 172.21.15.63:0/3220553108 learned_addr learned my addr 172.21.15.63:0/3220553108 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.126+0000 7f2649e00700 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d94460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.126+0000 7f2649400700 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200bf7ec0 0x55c200d960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.126+0000 7f2649400700 1 -- 172.21.15.63:0/3220553108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200bf7ec0 msgr2=0x55c200d960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:25.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.126+0000 7f2649400700 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200bf7ec0 0x55c200d960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.126+0000 7f2649400700 1 -- 172.21.15.63:0/3220553108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 msgr2=0x55c200d94460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:25.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.126+0000 7f2649400700 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d94460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.126+0000 7f2649400700 1 -- 172.21.15.63:0/3220553108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c200d815e0 con 0x55c200d768a0 2024-05-27T08:08:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.127+0000 7f2649400700 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d949a0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f26340093f0 tx=0x7f2634005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499db039f8c3f0e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.127+0000 7f2627e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.127+0000 7f2627e00700 1 -- 172.21.15.63:0/3220553108 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340173f0 con 0x55c200d768a0 2024-05-27T08:08:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.127+0000 7f2627e00700 1 -- 172.21.15.63:0/3220553108 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f263400f460 con 0x55c200d768a0 2024-05-27T08:08:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.127+0000 7f265b408b80 1 -- 172.21.15.63:0/3220553108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2634042070 con 0x55c200d768a0 2024-05-27T08:08:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.127+0000 7f2627e00700 1 -- 172.21.15.63:0/3220553108 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2634055050 con 0x55c200d768a0 2024-05-27T08:08:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.127+0000 7f265b408b80 10 client.22113 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.127+0000 7f265b408b80 1 -- 172.21.15.63:0/3220553108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c200d733d0 con 0x55c200d768a0 2024-05-27T08:08:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.128+0000 7f2627e00700 1 -- 172.21.15.63:0/3220553108 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2634042070 con 0x55c200d768a0 2024-05-27T08:08:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.128+0000 7f2627e00700 1 -- 172.21.15.63:0/3220553108 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2634017750 con 0x55c200d768a0 2024-05-27T08:08:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.128+0000 7f265b408b80 10 client.22113 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.128+0000 7f265b408b80 10 client.22113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.128+0000 7f265b408b80 20 client.22113 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.128+0000 7f265b408b80 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c200dfbfe0 0x55c200dfe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.128+0000 7f265b408b80 4 client.22113 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:08:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.128+0000 7f265b408b80 1 -- 172.21.15.63:0/3220553108 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c200ca05f0 con 0x55c200dfbfe0 2024-05-27T08:08:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.129+0000 7f2649e00700 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c200dfbfe0 0x55c200dfe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.130+0000 7f2649e00700 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c200dfbfe0 0x55c200dfe400 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32021ac908514fb2 server_cookie=397b9ebc5ff71640 in_seq=0 out_seq=0 2024-05-27T08:08:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.130+0000 7f2627e00700 10 client.22113 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f2627e00700 1 -- 172.21.15.63:0/3220553108 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c200ca05f0 con 0x55c200dfbfe0 2024-05-27T08:08:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f2627e00700 10 client.22113 handle_command_reply: tid=0 2024-05-27T08:08:25.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f265b408b80 1 client.22113 shutdown 2024-05-27T08:08:25.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f265b408b80 1 -- 172.21.15.63:0/3220553108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 msgr2=0x55c200d949a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:25.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f265b408b80 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d949a0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f26340093f0 tx=0x7f2634005e80 comp rx=0 tx=0).stop 2024-05-27T08:08:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f265b408b80 1 -- 172.21.15.63:0/3220553108 shutdown_connections 2024-05-27T08:08:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f265b408b80 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c200dfbfe0 0x55c200dfe400 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f265b408b80 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c200d768a0 0x55c200d949a0 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f265b408b80 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c200bf7ec0 0x55c200d960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f265b408b80 1 --2- 172.21.15.63:0/3220553108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c200d76490 0x55c200d94460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.133+0000 7f265b408b80 1 -- 172.21.15.63:0/3220553108 >> 172.21.15.63:0/3220553108 conn(0x55c200c58040 msgr2=0x55c200d80c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.134+0000 7f2627e00700 0 client.22113 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.134+0000 7f265b408b80 1 -- 172.21.15.63:0/3220553108 shutdown_connections 2024-05-27T08:08:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.134+0000 7f265b408b80 1 -- 172.21.15.63:0/3220553108 wait complete. 2024-05-27T08:08:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.134+0000 7f265b408b80 20 client.22113 trim_cache size 0 max 16384 2024-05-27T08:08:25.136 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:08:25.136 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:08:25.136 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "ce112e9a-439a-4b30-9226-c832498909f8", 2024-05-27T08:08:25.136 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:08:25.136 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 -- 172.21.15.63:0/3731074943 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f262c0b2a70 msgr2=0x7f262c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f262c0b2a70 0x7f262c0b4f80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f26400a50b0 tx=0x7f2640007b30 comp rx=0 tx=0).stop 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 -- 172.21.15.63:0/3731074943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264410b890 msgr2=0x7f26441a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264410b890 0x7f26441a1600 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f264000ada0 tx=0x7f264000c330 comp rx=0 tx=0).stop 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 -- 172.21.15.63:0/3731074943 shutdown_connections 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f262c0b2a70 0x7f262c0b4f80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f264410b890 0x7f26441a1600 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2644106610 0x7f264419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 --2- 172.21.15.63:0/3731074943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f264410af40 0x7f264419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 -- 172.21.15.63:0/3731074943 >> 172.21.15.63:0/3731074943 conn(0x7f2644102020 msgr2=0x7f264410a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 -- 172.21.15.63:0/3731074943 shutdown_connections 2024-05-27T08:08:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:25.135+0000 7f264b200700 1 -- 172.21.15.63:0/3731074943 wait complete. 2024-05-27T08:08:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:25 smithi086 ceph-mon[34687]: pgmap v1105: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:08:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:25 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:08:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:25 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:25 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:08:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:25 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:25 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:08:25.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:25 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:25 smithi149 ceph-mon[35408]: pgmap v1105: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:08:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:25 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:08:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:25 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:25 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:08:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:25 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:25 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:08:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:25 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:25 smithi063 ceph-mon[31613]: pgmap v1105: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:08:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:25 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:08:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:25 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:25 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:08:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:25 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:25 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:08:25.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:25 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.093 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:08:26.622 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:08:26.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:26 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:08:26.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:26 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:26 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:08:26.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:26 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:08:26.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:26 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:26 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:08:26.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:26 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:26 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:08:26.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:26 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:26 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:26 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:08:26.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:26 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:26 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:08:26.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:26 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:08:26.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:26 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:26 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:08:26.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:26 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:26 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:08:26.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:26 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:26 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:26 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:08:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:26 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:26 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:08:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:26 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:08:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:26 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:26 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:08:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:26 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:26 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:08:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:26 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:26 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:08:27.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:27 smithi086 ceph-mon[34687]: pgmap v1106: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.7 MiB/s wr, 47 op/s 2024-05-27T08:08:27.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:27 smithi149 ceph-mon[35408]: pgmap v1106: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.7 MiB/s wr, 47 op/s 2024-05-27T08:08:27.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:27 smithi063 ceph-mon[31613]: pgmap v1106: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.7 MiB/s wr, 47 op/s 2024-05-27T08:08:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.133+0000 7f50c7e00640 1 -- 172.21.15.63:0/3136757088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50c810b610 msgr2=0x7f50c810da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.133+0000 7f50c7e00640 1 --2- 172.21.15.63:0/3136757088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50c810b610 0x7f50c810da30 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009900 tx=0x7f50b402a410 comp rx=0 tx=0).stop 2024-05-27T08:08:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.133+0000 7f50c7e00640 1 -- 172.21.15.63:0/3136757088 shutdown_connections 2024-05-27T08:08:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.133+0000 7f50c7e00640 1 --2- 172.21.15.63:0/3136757088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c810e000 0x7f50c81104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.133+0000 7f50c7e00640 1 --2- 172.21.15.63:0/3136757088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50c810b610 0x7f50c810da30 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.133+0000 7f50c7e00640 1 --2- 172.21.15.63:0/3136757088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c8104bb0 0x7f50c8104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.133+0000 7f50c7e00640 1 -- 172.21.15.63:0/3136757088 >> 172.21.15.63:0/3136757088 conn(0x7f50c80fe210 msgr2=0x7f50c8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.133+0000 7f50c7e00640 1 -- 172.21.15.63:0/3136757088 shutdown_connections 2024-05-27T08:08:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.133+0000 7f50c7e00640 1 -- 172.21.15.63:0/3136757088 wait complete. 2024-05-27T08:08:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.135+0000 7f50c7e00640 1 Processor -- start 2024-05-27T08:08:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.135+0000 7f50c7e00640 1 -- start start 2024-05-27T08:08:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.136+0000 7f50c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c8104bb0 0x7f50c8102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.136+0000 7f50c7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c810b610 0x7f50c81029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.136+0000 7f50c7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50c810e000 0x7f50c8102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.136+0000 7f50c7e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8113b10 con 0x7f50c810e000 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.136+0000 7f50c7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8113990 con 0x7f50c8104bb0 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.136+0000 7f50c7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8113c90 con 0x7f50c810b610 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.136+0000 7f50c7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50c810e000 0x7f50c8102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.136+0000 7f50c7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50c810e000 0x7f50c8102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48116/0 (socket says 172.21.15.63:48116) 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.136+0000 7f50c7400640 1 -- 172.21.15.63:0/3419142425 learned_addr learned my addr 172.21.15.63:0/3419142425 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.136+0000 7f50c6000640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c810b610 0x7f50c81029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.137+0000 7f50c6a00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c8104bb0 0x7f50c8102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.137+0000 7f50c7400640 1 -- 172.21.15.63:0/3419142425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c810b610 msgr2=0x7f50c81029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.137+0000 7f50c7400640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c810b610 0x7f50c81029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.137+0000 7f50c7400640 1 -- 172.21.15.63:0/3419142425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c8104bb0 msgr2=0x7f50c8102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.137+0000 7f50c7400640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c8104bb0 0x7f50c8102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.137+0000 7f50c7400640 1 -- 172.21.15.63:0/3419142425 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c81a89d0 con 0x7f50c810e000 2024-05-27T08:08:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.137+0000 7f50c6a00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c8104bb0 0x7f50c8102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:08:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.137+0000 7f50c7400640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50c810e000 0x7f50c8102ef0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f50c000eaa0 tx=0x7f50c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.137+0000 7f50cca00640 1 -- 172.21.15.63:0/3419142425 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c000cdb0 con 0x7f50c810e000 2024-05-27T08:08:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.138+0000 7f50cca00640 1 -- 172.21.15.63:0/3419142425 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50c0005020 con 0x7f50c810e000 2024-05-27T08:08:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.138+0000 7f50cca00640 1 -- 172.21.15.63:0/3419142425 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c00116d0 con 0x7f50c810e000 2024-05-27T08:08:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.138+0000 7f50c7e00640 1 -- 172.21.15.63:0/3419142425 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c81a8cc0 con 0x7f50c810e000 2024-05-27T08:08:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.138+0000 7f50c7e00640 1 -- 172.21.15.63:0/3419142425 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c81a9190 con 0x7f50c810e000 2024-05-27T08:08:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.142+0000 7f50d4e6d740 1 Processor -- start 2024-05-27T08:08:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.142+0000 7f50cca00640 1 -- 172.21.15.63:0/3419142425 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f50c0004a40 con 0x7f50c810e000 2024-05-27T08:08:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.142+0000 7f50d4e6d740 1 -- start start 2024-05-27T08:08:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.143+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.143+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fc9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.143+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.143+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f929f3ab90 con 0x55f929c271b0 2024-05-27T08:08:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.143+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f929f2a730 con 0x55f929c064d0 2024-05-27T08:08:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.143+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f929f2e770 con 0x55f929bfe2f0 2024-05-27T08:08:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.143+0000 7f50c6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50c6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fc9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48132/0 (socket says 172.21.15.63:48132) 2024-05-27T08:08:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50c6000640 1 -- 172.21.15.63:0/1357512731 learned_addr learned my addr 172.21.15.63:0/1357512731 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50c6000640 1 --2- 172.21.15.63:0/1357512731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50c6a00640 1 --2- 172.21.15.63:0/1357512731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fc9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50c6000640 1 -- 172.21.15.63:0/1357512731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 msgr2=0x55f929fc9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50c6000640 1 --2- 172.21.15.63:0/1357512731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fc9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50c6000640 1 -- 172.21.15.63:0/1357512731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 msgr2=0x55f929fca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50c6000640 1 --2- 172.21.15.63:0/1357512731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50c6000640 1 -- 172.21.15.63:0/1357512731 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f929ec38b0 con 0x55f929c271b0 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50c6a00640 1 --2- 172.21.15.63:0/1357512731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fc9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.144+0000 7f50cca00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50ac0b2950 0x7f50ac0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.145+0000 7f50c6000640 1 --2- 172.21.15.63:0/1357512731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fc9a60 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f50b402a930 tx=0x7f50b4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56777913217f17c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.145+0000 7f50cca00640 1 -- 172.21.15.63:0/3419142425 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f50c00e2880 con 0x7f50c810e000 2024-05-27T08:08:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.145+0000 7f50c6a00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50ac0b2950 0x7f50ac0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.145+0000 7f50b3e00640 1 -- 172.21.15.63:0/1357512731 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4043020 con 0x55f929c271b0 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.145+0000 7f50b3e00640 1 -- 172.21.15.63:0/1357512731 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50b40044c0 con 0x55f929c271b0 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.145+0000 7f50b3e00640 1 -- 172.21.15.63:0/1357512731 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4035050 con 0x55f929c271b0 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.146+0000 7f50d4e6d740 1 -- 172.21.15.63:0/1357512731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 msgr2=0x55f929fc9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.146+0000 7f50c6a00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50ac0b2950 0x7f50ac0b4e60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f50b8005fd0 tx=0x7f50b8005f60 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.146+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/1357512731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fc9a60 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f50b402a930 tx=0x7f50b4005e50 comp rx=0 tx=0).stop 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.147+0000 7f50d4e6d740 1 -- 172.21.15.63:0/1357512731 shutdown_connections 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.147+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/1357512731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.147+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/1357512731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fc9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.147+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/1357512731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fc9a60 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.147+0000 7f50d4e6d740 1 -- 172.21.15.63:0/1357512731 >> 172.21.15.63:0/1357512731 conn(0x55f929c1b300 msgr2=0x55f929fc6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.147+0000 7f50d4e6d740 1 -- 172.21.15.63:0/1357512731 shutdown_connections 2024-05-27T08:08:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.147+0000 7f50d4e6d740 1 -- 172.21.15.63:0/1357512731 wait complete. 2024-05-27T08:08:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.147+0000 7f50d4e6d740 1 -- start start 2024-05-27T08:08:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.148+0000 7f50d4e6d740 1 Processor -- start 2024-05-27T08:08:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.148+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929bfe2f0 0x55f929fd3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.148+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929c064d0 0x55f929fd44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.148+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c271b0 0x55f929fdfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.148+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f929c10790 con 0x55f929bfe2f0 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.148+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f929ed2bc0 con 0x55f929c271b0 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.148+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f929fd6b30 con 0x55f929c064d0 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.148+0000 7f50d4e6d740 10 client.0 _ll_register_callbacks cb 0x55f929eb0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929bfe2f0 0x55f929fd3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929bfe2f0 0x55f929fd3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48142/0 (socket says 172.21.15.63:48142) 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c6a00640 1 -- 172.21.15.63:0/4203661747 learned_addr learned my addr 172.21.15.63:0/4203661747 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c7400640 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c271b0 0x55f929fdfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c6000640 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929c064d0 0x55f929fd44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c6a00640 1 -- 172.21.15.63:0/4203661747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929c064d0 msgr2=0x55f929fd44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c6a00640 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929c064d0 0x55f929fd44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c6a00640 1 -- 172.21.15.63:0/4203661747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c271b0 msgr2=0x55f929fdfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c6a00640 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c271b0 0x55f929fdfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c6a00640 1 -- 172.21.15.63:0/4203661747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f929c3f210 con 0x55f929bfe2f0 2024-05-27T08:08:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.149+0000 7f50c6000640 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929c064d0 0x55f929fd44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.150+0000 7f50c6a00640 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929bfe2f0 0x55f929fd3f60 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009fa0 tx=0x7f50b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3cbf0635515209 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.150+0000 7f50b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.150+0000 7f50b0c00640 1 -- 172.21.15.63:0/4203661747 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8022070 con 0x55f929bfe2f0 2024-05-27T08:08:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.150+0000 7f50d4e6d740 1 -- 172.21.15.63:0/4203661747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f929f3cae0 con 0x55f929bfe2f0 2024-05-27T08:08:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.150+0000 7f50b0c00640 1 -- 172.21.15.63:0/4203661747 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f50b801e860 con 0x55f929bfe2f0 2024-05-27T08:08:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.151+0000 7f50b0c00640 1 -- 172.21.15.63:0/4203661747 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f50b802e050 con 0x55f929bfe2f0 2024-05-27T08:08:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.151+0000 7f50d4e6d740 10 client.22125 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.151+0000 7f50d4e6d740 1 -- 172.21.15.63:0/4203661747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f929d0b620 con 0x55f929bfe2f0 2024-05-27T08:08:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.151+0000 7f50b0c00640 1 -- 172.21.15.63:0/4203661747 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f929f3cae0 con 0x55f929bfe2f0 2024-05-27T08:08:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.151+0000 7f50b0c00640 1 -- 172.21.15.63:0/4203661747 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f50b80109e0 con 0x55f929bfe2f0 2024-05-27T08:08:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.152+0000 7f50d4e6d740 10 client.22125 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.152+0000 7f50d4e6d740 10 client.22125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.152+0000 7f50d4e6d740 20 client.22125 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.152+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f929be5d80 0x55f929fe5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.152+0000 7f50d4e6d740 4 client.22125 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.152+0000 7f50d4e6d740 1 -- 172.21.15.63:0/4203661747 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f929f362c0 con 0x55f929be5d80 2024-05-27T08:08:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.152+0000 7f50c6000640 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f929be5d80 0x55f929fe5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.153+0000 7f50c6000640 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f929be5d80 0x55f929fe5fc0 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff86d2d9daede37 server_cookie=fd56940508c92e71 in_seq=0 out_seq=0 2024-05-27T08:08:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.153+0000 7f50b0c00640 10 client.22125 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.155+0000 7f50b0c00640 1 -- 172.21.15.63:0/4203661747 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f929f362c0 con 0x55f929be5d80 2024-05-27T08:08:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.155+0000 7f50b0c00640 10 client.22125 handle_command_reply: tid=0 2024-05-27T08:08:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.156+0000 7f50d4e6d740 1 client.22125 shutdown 2024-05-27T08:08:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.156+0000 7f50d4e6d740 1 -- 172.21.15.63:0/4203661747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929bfe2f0 msgr2=0x55f929fd3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.156+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929bfe2f0 0x55f929fd3f60 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009fa0 tx=0x7f50b8004730 comp rx=0 tx=0).stop 2024-05-27T08:08:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.156+0000 7f50d4e6d740 1 -- 172.21.15.63:0/4203661747 shutdown_connections 2024-05-27T08:08:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.156+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f929be5d80 0x55f929fe5fc0 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.156+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c271b0 0x55f929fdfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.156+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929c064d0 0x55f929fd44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.156+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/4203661747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929bfe2f0 0x55f929fd3f60 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.156+0000 7f50d4e6d740 1 -- 172.21.15.63:0/4203661747 >> 172.21.15.63:0/4203661747 conn(0x55f929c1b300 msgr2=0x55f929fc9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.156+0000 7f50b0c00640 0 client.22125 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.157+0000 7f50d4e6d740 1 -- 172.21.15.63:0/4203661747 shutdown_connections 2024-05-27T08:08:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.157+0000 7f50d4e6d740 1 -- 172.21.15.63:0/4203661747 wait complete. 2024-05-27T08:08:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.157+0000 7f50d4e6d740 20 client.22125 trim_cache size 0 max 16384 2024-05-27T08:08:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.180+0000 7f50d4e6d740 1 Processor -- start 2024-05-27T08:08:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.181+0000 7f50d4e6d740 1 -- start start 2024-05-27T08:08:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.181+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fdfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.181+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fe0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.181+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929feb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.181+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81b5ad0 con 0x55f929c271b0 2024-05-27T08:08:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.181+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81b33a0 con 0x55f929c064d0 2024-05-27T08:08:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.181+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81b1210 con 0x55f929bfe2f0 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.181+0000 7f50c7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929feb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.182+0000 7f50c7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929feb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48148/0 (socket says 172.21.15.63:48148) 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.182+0000 7f50c7400640 1 -- 172.21.15.63:0/3642193848 learned_addr learned my addr 172.21.15.63:0/3642193848 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.182+0000 7f50c6a00640 1 --2- 172.21.15.63:0/3642193848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fdfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.182+0000 7f50c6000640 1 --2- 172.21.15.63:0/3642193848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fe0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.182+0000 7f50c7400640 1 -- 172.21.15.63:0/3642193848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 msgr2=0x55f929fdfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.182+0000 7f50c7400640 1 --2- 172.21.15.63:0/3642193848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fdfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.182+0000 7f50c7400640 1 -- 172.21.15.63:0/3642193848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 msgr2=0x55f929fe0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.182+0000 7f50c7400640 1 --2- 172.21.15.63:0/3642193848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fe0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.182+0000 7f50c7400640 1 -- 172.21.15.63:0/3642193848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f929fcae10 con 0x55f929c271b0 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.183+0000 7f50c7400640 1 --2- 172.21.15.63:0/3642193848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929feb7c0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f50c00a9730 tx=0x7f50c00a5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbc63a233726daf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.183+0000 7f50b3400640 1 -- 172.21.15.63:0/3642193848 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c001f040 con 0x55f929c271b0 2024-05-27T08:08:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.183+0000 7f50b3400640 1 -- 172.21.15.63:0/3642193848 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50c00a97a0 con 0x55f929c271b0 2024-05-27T08:08:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.183+0000 7f50b3400640 1 -- 172.21.15.63:0/3642193848 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c00dfe90 con 0x55f929c271b0 2024-05-27T08:08:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.184+0000 7f50d4e6d740 1 -- 172.21.15.63:0/3642193848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 msgr2=0x55f929feb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.184+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/3642193848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929feb7c0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f50c00a9730 tx=0x7f50c00a5040 comp rx=0 tx=0).stop 2024-05-27T08:08:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.184+0000 7f50d4e6d740 1 -- 172.21.15.63:0/3642193848 shutdown_connections 2024-05-27T08:08:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.184+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/3642193848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929feb7c0 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.184+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/3642193848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fe0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.184+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/3642193848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fdfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.184+0000 7f50d4e6d740 1 -- 172.21.15.63:0/3642193848 >> 172.21.15.63:0/3642193848 conn(0x55f929c1b300 msgr2=0x55f929fd4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.184+0000 7f50d4e6d740 1 -- 172.21.15.63:0/3642193848 shutdown_connections 2024-05-27T08:08:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.184+0000 7f50d4e6d740 1 -- 172.21.15.63:0/3642193848 wait complete. 2024-05-27T08:08:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.185+0000 7f50d4e6d740 1 -- start start 2024-05-27T08:08:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.185+0000 7f50d4e6d740 1 Processor -- start 2024-05-27T08:08:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fd8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fd8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50d4e6d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fd94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81b1860 con 0x55f929c271b0 2024-05-27T08:08:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81b2cd0 con 0x55f929c064d0 2024-05-27T08:08:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50d4e6d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81b1f30 con 0x55f929bfe2f0 2024-05-27T08:08:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50d4e6d740 10 client.0 _ll_register_callbacks cb 0x7f50b8015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50c7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fd94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50c7400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fd94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48164/0 (socket says 172.21.15.63:48164) 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50c7400640 1 -- 172.21.15.63:0/2082984623 learned_addr learned my addr 172.21.15.63:0/2082984623 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.186+0000 7f50c6a00640 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fd8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.187+0000 7f50c7400640 1 -- 172.21.15.63:0/2082984623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 msgr2=0x55f929fd8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.187+0000 7f50c7400640 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fd8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.187+0000 7f50c7400640 1 -- 172.21.15.63:0/2082984623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 msgr2=0x55f929fd8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.187+0000 7f50c7400640 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fd8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.187+0000 7f50c7400640 1 -- 172.21.15.63:0/2082984623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92a04c690 con 0x55f929c271b0 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.187+0000 7f50c7400640 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fd94c0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f50c00dfcc0 tx=0x7f50c0004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45a1f191ffa8498 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.187+0000 7f50b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.187+0000 7f50b0c00640 1 -- 172.21.15.63:0/2082984623 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0083e60 con 0x55f929c271b0 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.188+0000 7f50b0c00640 1 -- 172.21.15.63:0/2082984623 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f50c00dfe90 con 0x55f929c271b0 2024-05-27T08:08:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.188+0000 7f50d4e6d740 1 -- 172.21.15.63:0/2082984623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c00a97a0 con 0x55f929c271b0 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.188+0000 7f50b0c00640 1 -- 172.21.15.63:0/2082984623 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f50c008c050 con 0x55f929c271b0 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.188+0000 7f50d4e6d740 10 client.22131 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.188+0000 7f50d4e6d740 1 -- 172.21.15.63:0/2082984623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f92a04d740 con 0x55f929c271b0 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.188+0000 7f50b0c00640 1 -- 172.21.15.63:0/2082984623 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50c00a97a0 con 0x55f929c271b0 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.189+0000 7f50b0c00640 1 -- 172.21.15.63:0/2082984623 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f50c000f040 con 0x55f929c271b0 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.189+0000 7f50d4e6d740 10 client.22131 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.189+0000 7f50d4e6d740 10 client.22131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.189+0000 7f50d4e6d740 20 client.22131 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.189+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f929bbb1c0 0x55f92a04fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.189+0000 7f50d4e6d740 4 client.22131 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.189+0000 7f50d4e6d740 1 -- 172.21.15.63:0/2082984623 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f929eb04b0 con 0x55f929bbb1c0 2024-05-27T08:08:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.190+0000 7f50c6a00640 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f929bbb1c0 0x55f92a04fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.191+0000 7f50c6a00640 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f929bbb1c0 0x55f92a04fd90 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b3309017a870c8 server_cookie=acc881cc0c63a764 in_seq=0 out_seq=0 2024-05-27T08:08:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.191+0000 7f50b0c00640 10 client.22131 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50b0c00640 1 -- 172.21.15.63:0/2082984623 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f929eb04b0 con 0x55f929bbb1c0 2024-05-27T08:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50b0c00640 10 client.22131 handle_command_reply: tid=0 2024-05-27T08:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 client.22131 shutdown 2024-05-27T08:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 -- 172.21.15.63:0/2082984623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 msgr2=0x55f929fd94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fd94c0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f50c00dfcc0 tx=0x7f50c0004f60 comp rx=0 tx=0).stop 2024-05-27T08:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 -- 172.21.15.63:0/2082984623 shutdown_connections 2024-05-27T08:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f929bbb1c0 0x55f92a04fd90 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f929c271b0 0x55f929fd94c0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f929c064d0 0x55f929fd8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 --2- 172.21.15.63:0/2082984623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f929bfe2f0 0x55f929fd8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 -- 172.21.15.63:0/2082984623 >> 172.21.15.63:0/2082984623 conn(0x55f929c1b300 msgr2=0x55f929fd5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50b0c00640 0 client.22131 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 -- 172.21.15.63:0/2082984623 shutdown_connections 2024-05-27T08:08:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.228+0000 7f50d4e6d740 1 -- 172.21.15.63:0/2082984623 wait complete. 2024-05-27T08:08:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.229+0000 7f50d4e6d740 20 client.22131 trim_cache size 0 max 16384 2024-05-27T08:08:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:08:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:08:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:08:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 -- 172.21.15.63:0/3419142425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50ac0b2950 msgr2=0x7f50ac0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50ac0b2950 0x7f50ac0b4e60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f50b8005fd0 tx=0x7f50b8005f60 comp rx=0 tx=0).stop 2024-05-27T08:08:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 -- 172.21.15.63:0/3419142425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50c810e000 msgr2=0x7f50c8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50c810e000 0x7f50c8102ef0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f50c000eaa0 tx=0x7f50c000ef60 comp rx=0 tx=0).stop 2024-05-27T08:08:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 -- 172.21.15.63:0/3419142425 shutdown_connections 2024-05-27T08:08:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f50ac0b2950 0x7f50ac0b4e60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50c810e000 0x7f50c8102ef0 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50c810b610 0x7f50c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 --2- 172.21.15.63:0/3419142425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50c8104bb0 0x7f50c8102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 -- 172.21.15.63:0/3419142425 >> 172.21.15.63:0/3419142425 conn(0x7f50c80fe210 msgr2=0x7f50c810e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 -- 172.21.15.63:0/3419142425 shutdown_connections 2024-05-27T08:08:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:28.230+0000 7f50c7e00640 1 -- 172.21.15.63:0/3419142425 wait complete. 2024-05-27T08:08:29.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.938+0000 7ff496600700 1 -- 172.21.15.63:0/3410055543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff490107de0 msgr2=0x7ff4901081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:29.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.938+0000 7ff496600700 1 --2- 172.21.15.63:0/3410055543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff490107de0 0x7ff4901081f0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0099c0 tx=0x7ff47c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.938+0000 7ff496600700 1 -- 172.21.15.63:0/3410055543 shutdown_connections 2024-05-27T08:08:29.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.938+0000 7ff496600700 1 --2- 172.21.15.63:0/3410055543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff490108730 0x7ff490110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.938+0000 7ff496600700 1 --2- 172.21.15.63:0/3410055543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4901034b0 0x7ff4901038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.938+0000 7ff496600700 1 --2- 172.21.15.63:0/3410055543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff490107de0 0x7ff4901081f0 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.938+0000 7ff496600700 1 -- 172.21.15.63:0/3410055543 >> 172.21.15.63:0/3410055543 conn(0x7ff4900fedb0 msgr2=0x7ff4901011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:29.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.938+0000 7ff496600700 1 -- 172.21.15.63:0/3410055543 shutdown_connections 2024-05-27T08:08:29.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.939+0000 7ff496600700 1 -- 172.21.15.63:0/3410055543 wait complete. 2024-05-27T08:08:29.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.940+0000 7ff496600700 1 Processor -- start 2024-05-27T08:08:29.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.940+0000 7ff496600700 1 -- start start 2024-05-27T08:08:29.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.941+0000 7ff496600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4901034b0 0x7ff4900ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.941+0000 7ff496600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff490107de0 0x7ff4900ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.941+0000 7ff496600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff490108730 0x7ff4900ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.941+0000 7ff496600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901131f0 con 0x7ff4901034b0 2024-05-27T08:08:29.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.941+0000 7ff496600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff490113070 con 0x7ff490108730 2024-05-27T08:08:29.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.941+0000 7ff496600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff490113370 con 0x7ff490107de0 2024-05-27T08:08:29.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff490107de0 0x7ff4900ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4901034b0 0x7ff4900ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff490107de0 0x7ff4900ff7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:37112/0 (socket says 172.21.15.63:37112) 2024-05-27T08:08:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48ea00700 1 -- 172.21.15.63:0/2061187962 learned_addr learned my addr 172.21.15.63:0/2061187962 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48fe00700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff490108730 0x7ff4900ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48f400700 1 -- 172.21.15.63:0/2061187962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff490107de0 msgr2=0x7ff4900ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48f400700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff490107de0 0x7ff4900ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48f400700 1 -- 172.21.15.63:0/2061187962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff490108730 msgr2=0x7ff4900ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48f400700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff490108730 0x7ff4900ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48f400700 1 -- 172.21.15.63:0/2061187962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4901a90d0 con 0x7ff4901034b0 2024-05-27T08:08:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.942+0000 7ff48fe00700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff490108730 0x7ff4900ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:08:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.943+0000 7ff48f400700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4901034b0 0x7ff4900ff290 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7ff48000d950 tx=0x7ff48000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:29.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.943+0000 7ff487e00700 1 -- 172.21.15.63:0/2061187962 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480017070 con 0x7ff4901034b0 2024-05-27T08:08:29.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.943+0000 7ff496600700 1 -- 172.21.15.63:0/2061187962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4901a93c0 con 0x7ff4901034b0 2024-05-27T08:08:29.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.943+0000 7ff487e00700 1 -- 172.21.15.63:0/2061187962 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff480009dc0 con 0x7ff4901034b0 2024-05-27T08:08:29.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.943+0000 7ff487e00700 1 -- 172.21.15.63:0/2061187962 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480010cf0 con 0x7ff4901034b0 2024-05-27T08:08:29.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.943+0000 7ff496600700 1 -- 172.21.15.63:0/2061187962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4901a9900 con 0x7ff4901034b0 2024-05-27T08:08:29.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.950+0000 7ff487e00700 1 -- 172.21.15.63:0/2061187962 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff48001a070 con 0x7ff4901034b0 2024-05-27T08:08:29.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:29 smithi063 ceph-mon[31613]: pgmap v1107: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.7 MiB/s wr, 40 op/s 2024-05-27T08:08:29.952 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:29 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:08:29.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.952+0000 7ff487e00700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4740b29f0 0x7ff4740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.952+0000 7ff487e00700 1 -- 172.21.15.63:0/2061187962 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7ff4800e31f0 con 0x7ff4901034b0 2024-05-27T08:08:29.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.953+0000 7ff48ea00700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4740b29f0 0x7ff4740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:29 smithi086 ceph-mon[34687]: pgmap v1107: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.7 MiB/s wr, 40 op/s 2024-05-27T08:08:29.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:29 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:08:29.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:29 smithi149 ceph-mon[35408]: pgmap v1107: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.7 MiB/s wr, 40 op/s 2024-05-27T08:08:29.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:29 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:08:29.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.969+0000 7ff48ea00700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4740b29f0 0x7ff4740b4f00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009750 tx=0x7ff47c005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:29.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.972+0000 7ff4a5228b80 1 Processor -- start 2024-05-27T08:08:29.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.972+0000 7ff4a5228b80 1 -- start start 2024-05-27T08:08:29.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.973+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b05afe40 0x5557b07347f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.973+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b072e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.973+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b072f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.973+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5557b065ce40 con 0x5557b072edb0 2024-05-27T08:08:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.973+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557b064c9e0 con 0x5557b05afe40 2024-05-27T08:08:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.973+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5557b0650a20 con 0x5557b0734d30 2024-05-27T08:08:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.973+0000 7ff48fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b072f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.973+0000 7ff48fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b072f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55738/0 (socket says 172.21.15.63:55738) 2024-05-27T08:08:29.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.973+0000 7ff48ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b072e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.973+0000 7ff48fe00700 1 -- 172.21.15.63:0/632370550 learned_addr learned my addr 172.21.15.63:0/632370550 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:29.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.974+0000 7ff48f400700 1 --2- 172.21.15.63:0/632370550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b05afe40 0x5557b07347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.974+0000 7ff48ea00700 1 -- 172.21.15.63:0/632370550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b05afe40 msgr2=0x5557b07347f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:29.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.974+0000 7ff48ea00700 1 --2- 172.21.15.63:0/632370550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b05afe40 0x5557b07347f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.974+0000 7ff48ea00700 1 -- 172.21.15.63:0/632370550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 msgr2=0x5557b072f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:29.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.974+0000 7ff48ea00700 1 --2- 172.21.15.63:0/632370550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b072f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.974+0000 7ff48ea00700 1 -- 172.21.15.63:0/632370550 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557b0615420 con 0x5557b0734d30 2024-05-27T08:08:29.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.974+0000 7ff48fe00700 1 --2- 172.21.15.63:0/632370550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b072f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:08:29.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.975+0000 7ff48ea00700 1 --2- 172.21.15.63:0/632370550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b072e870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009f50 tx=0x7ff47c01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a9ec404703353c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:29.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.976+0000 7ff484c00700 1 -- 172.21.15.63:0/632370550 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00be50 con 0x5557b0734d30 2024-05-27T08:08:29.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.976+0000 7ff484c00700 1 -- 172.21.15.63:0/632370550 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff47c0055f0 con 0x5557b0734d30 2024-05-27T08:08:29.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.976+0000 7ff484c00700 1 -- 172.21.15.63:0/632370550 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00f900 con 0x5557b0734d30 2024-05-27T08:08:29.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.977+0000 7ff4a5228b80 1 -- 172.21.15.63:0/632370550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 msgr2=0x5557b072e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:29.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.977+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/632370550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b072e870 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009f50 tx=0x7ff47c01e000 comp rx=0 tx=0).stop 2024-05-27T08:08:29.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.977+0000 7ff4a5228b80 1 -- 172.21.15.63:0/632370550 shutdown_connections 2024-05-27T08:08:29.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.977+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/632370550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b072f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.977+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/632370550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b05afe40 0x5557b07347f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.977+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/632370550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b072e870 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.977+0000 7ff4a5228b80 1 -- 172.21.15.63:0/632370550 >> 172.21.15.63:0/632370550 conn(0x5557b060ffc0 msgr2=0x5557b07298d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:29.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.977+0000 7ff4a5228b80 1 -- 172.21.15.63:0/632370550 shutdown_connections 2024-05-27T08:08:29.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.978+0000 7ff4a5228b80 1 -- 172.21.15.63:0/632370550 wait complete. 2024-05-27T08:08:29.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.978+0000 7ff4a5228b80 1 -- start start 2024-05-27T08:08:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.979+0000 7ff4a5228b80 1 Processor -- start 2024-05-27T08:08:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.979+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b05afe40 0x5557b0738540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.979+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b0738a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.980+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b0734d30 0x5557b0738fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.980+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5557b0552fa0 con 0x5557b072edb0 2024-05-27T08:08:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.980+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557b05ed760 con 0x5557b0734d30 2024-05-27T08:08:29.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.980+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5557b073b740 con 0x5557b05afe40 2024-05-27T08:08:29.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.980+0000 7ff4a5228b80 10 client.0 _ll_register_callbacks cb 0x5557b05f6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.980+0000 7ff48f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b05afe40 0x5557b0738540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.980+0000 7ff48f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b05afe40 0x5557b0738540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:37128/0 (socket says 172.21.15.63:37128) 2024-05-27T08:08:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.980+0000 7ff48f400700 1 -- 172.21.15.63:0/1290478457 learned_addr learned my addr 172.21.15.63:0/1290478457 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.981+0000 7ff48fe00700 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b0734d30 0x5557b0738fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.981+0000 7ff48f400700 1 -- 172.21.15.63:0/1290478457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b0734d30 msgr2=0x5557b0738fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.981+0000 7ff48f400700 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b0734d30 0x5557b0738fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.981+0000 7ff48f400700 1 -- 172.21.15.63:0/1290478457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 msgr2=0x5557b0738a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:08:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.981+0000 7ff48ea00700 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b0738a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.981+0000 7ff48f400700 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b0738a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.981+0000 7ff48f400700 1 -- 172.21.15.63:0/1290478457 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557b059c8d0 con 0x5557b05afe40 2024-05-27T08:08:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.981+0000 7ff48ea00700 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b0738a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:08:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.982+0000 7ff48f400700 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b05afe40 0x5557b0738540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff4800e0b80 tx=0x7ff4800a9cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f17f9b801b921cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.982+0000 7ff479600700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:08:29.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.982+0000 7ff479600700 1 -- 172.21.15.63:0/1290478457 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480021030 con 0x5557b05afe40 2024-05-27T08:08:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.982+0000 7ff4a5228b80 1 -- 172.21.15.63:0/1290478457 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff47c0055f0 con 0x5557b05afe40 2024-05-27T08:08:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.982+0000 7ff479600700 1 -- 172.21.15.63:0/1290478457 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7ff4800b2cf0 con 0x5557b05afe40 2024-05-27T08:08:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.982+0000 7ff479600700 1 -- 172.21.15.63:0/1290478457 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff4800ae4c0 con 0x5557b05afe40 2024-05-27T08:08:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.983+0000 7ff4a5228b80 10 client.30967 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.983+0000 7ff4a5228b80 1 -- 172.21.15.63:0/1290478457 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5557b061d060 con 0x5557b05afe40 2024-05-27T08:08:29.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.983+0000 7ff479600700 1 -- 172.21.15.63:0/1290478457 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff47c0055f0 con 0x5557b05afe40 2024-05-27T08:08:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.983+0000 7ff479600700 1 -- 172.21.15.63:0/1290478457 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff4800a9370 con 0x5557b05afe40 2024-05-27T08:08:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.984+0000 7ff4a5228b80 10 client.30967 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.984+0000 7ff4a5228b80 10 client.30967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.984+0000 7ff4a5228b80 20 client.30967 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.984+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557b0750e30 0x5557b0753250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.984+0000 7ff4a5228b80 4 client.30967 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.984+0000 7ff4a5228b80 1 -- 172.21.15.63:0/1290478457 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557b065ed90 con 0x5557b0750e30 2024-05-27T08:08:29.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.984+0000 7ff48ea00700 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557b0750e30 0x5557b0753250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:29.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.986+0000 7ff479600700 10 client.30967 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:29.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.986+0000 7ff48ea00700 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557b0750e30 0x5557b0753250 crc :-1 s=READY pgs=628 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69e2cc9aaa43e904 server_cookie=fb6c1aa17ee42b37 in_seq=0 out_seq=0 2024-05-27T08:08:29.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.987+0000 7ff479600700 1 -- 172.21.15.63:0/1290478457 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5557b065ed90 con 0x5557b0750e30 2024-05-27T08:08:29.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.987+0000 7ff479600700 10 client.30967 handle_command_reply: tid=0 2024-05-27T08:08:29.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.987+0000 7ff4a5228b80 1 client.30967 shutdown 2024-05-27T08:08:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff4a5228b80 1 -- 172.21.15.63:0/1290478457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b05afe40 msgr2=0x5557b0738540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b05afe40 0x5557b0738540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff4800e0b80 tx=0x7ff4800a9cf0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff4a5228b80 1 -- 172.21.15.63:0/1290478457 shutdown_connections 2024-05-27T08:08:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557b0750e30 0x5557b0753250 crc :-1 s=READY pgs=628 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b0734d30 0x5557b0738fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b05afe40 0x5557b0738540 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/1290478457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b0738a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff4a5228b80 1 -- 172.21.15.63:0/1290478457 >> 172.21.15.63:0/1290478457 conn(0x5557b060ffc0 msgr2=0x5557b0729580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff479600700 0 client.30967 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff4a5228b80 1 -- 172.21.15.63:0/1290478457 shutdown_connections 2024-05-27T08:08:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.988+0000 7ff4a5228b80 1 -- 172.21.15.63:0/1290478457 wait complete. 2024-05-27T08:08:29.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:29.989+0000 7ff4a5228b80 20 client.30967 trim_cache size 0 max 16384 2024-05-27T08:08:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.011+0000 7ff4a5228b80 1 Processor -- start 2024-05-27T08:08:30.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.011+0000 7ff4a5228b80 1 -- start start 2024-05-27T08:08:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.012+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b05afe40 0x5557b072e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.012+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b073a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.012+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b074d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.012+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b0b50 con 0x5557b072edb0 2024-05-27T08:08:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.012+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901ae3e0 con 0x5557b05afe40 2024-05-27T08:08:30.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.012+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901ac270 con 0x5557b0734d30 2024-05-27T08:08:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.012+0000 7ff48ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b073a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.013+0000 7ff48ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b073a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55752/0 (socket says 172.21.15.63:55752) 2024-05-27T08:08:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.013+0000 7ff48ea00700 1 -- 172.21.15.63:0/43848120 learned_addr learned my addr 172.21.15.63:0/43848120 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.013+0000 7ff48f400700 1 --2- 172.21.15.63:0/43848120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b05afe40 0x5557b072e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.013+0000 7ff48fe00700 1 --2- 172.21.15.63:0/43848120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b074d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.013+0000 7ff48ea00700 1 -- 172.21.15.63:0/43848120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 msgr2=0x5557b074d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.013+0000 7ff48ea00700 1 --2- 172.21.15.63:0/43848120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b074d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.013+0000 7ff48ea00700 1 -- 172.21.15.63:0/43848120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b05afe40 msgr2=0x5557b072e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.013+0000 7ff48ea00700 1 --2- 172.21.15.63:0/43848120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b05afe40 0x5557b072e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.013+0000 7ff48ea00700 1 -- 172.21.15.63:0/43848120 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557b0736ad0 con 0x5557b072edb0 2024-05-27T08:08:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.014+0000 7ff48ea00700 1 --2- 172.21.15.63:0/43848120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b073a9e0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0058e0 tx=0x7ff47c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568b624357945162 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.014+0000 7ff47be00700 1 -- 172.21.15.63:0/43848120 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00fd00 con 0x5557b072edb0 2024-05-27T08:08:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.014+0000 7ff47be00700 1 -- 172.21.15.63:0/43848120 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff47c00f960 con 0x5557b072edb0 2024-05-27T08:08:30.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.014+0000 7ff47be00700 1 -- 172.21.15.63:0/43848120 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c003d30 con 0x5557b072edb0 2024-05-27T08:08:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.015+0000 7ff4a5228b80 1 -- 172.21.15.63:0/43848120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 msgr2=0x5557b073a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.015+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/43848120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b073a9e0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0058e0 tx=0x7ff47c005730 comp rx=0 tx=0).stop 2024-05-27T08:08:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.015+0000 7ff4a5228b80 1 -- 172.21.15.63:0/43848120 shutdown_connections 2024-05-27T08:08:30.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.015+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/43848120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b074d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.015+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/43848120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b05afe40 0x5557b072e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.015+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/43848120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b072edb0 0x5557b073a9e0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.015+0000 7ff4a5228b80 1 -- 172.21.15.63:0/43848120 >> 172.21.15.63:0/43848120 conn(0x5557b060ffc0 msgr2=0x5557b0738d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.016+0000 7ff4a5228b80 1 -- 172.21.15.63:0/43848120 shutdown_connections 2024-05-27T08:08:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.016+0000 7ff4a5228b80 1 -- 172.21.15.63:0/43848120 wait complete. 2024-05-27T08:08:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.016+0000 7ff4a5228b80 1 -- start start 2024-05-27T08:08:30.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.017+0000 7ff4a5228b80 1 Processor -- start 2024-05-27T08:08:30.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.017+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b05afe40 0x5557b0749460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.017+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b072edb0 0x5557b07499a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.018+0000 7ff4a5228b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b0746230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.018+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901ac8c0 con 0x5557b05afe40 2024-05-27T08:08:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.018+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901add10 con 0x5557b072edb0 2024-05-27T08:08:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.018+0000 7ff4a5228b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901acf70 con 0x5557b0734d30 2024-05-27T08:08:30.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.018+0000 7ff4a5228b80 10 client.0 _ll_register_callbacks cb 0x5557b05f6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:30.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.018+0000 7ff48f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b05afe40 0x5557b0749460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.018+0000 7ff48fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b0746230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.018+0000 7ff48f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b05afe40 0x5557b0749460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55760/0 (socket says 172.21.15.63:55760) 2024-05-27T08:08:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.018+0000 7ff48f400700 1 -- 172.21.15.63:0/2709634685 learned_addr learned my addr 172.21.15.63:0/2709634685 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.018+0000 7ff48ea00700 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b072edb0 0x5557b07499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.019+0000 7ff48f400700 1 -- 172.21.15.63:0/2709634685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 msgr2=0x5557b0746230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.019+0000 7ff48f400700 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b0746230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.019+0000 7ff48f400700 1 -- 172.21.15.63:0/2709634685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b072edb0 msgr2=0x5557b07499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.019+0000 7ff48f400700 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b072edb0 0x5557b07499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.019+0000 7ff48f400700 1 -- 172.21.15.63:0/2709634685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557b07468a0 con 0x5557b05afe40 2024-05-27T08:08:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.019+0000 7ff48f400700 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b05afe40 0x5557b0749460 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7ff480005950 tx=0x7ff4800a97b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737cab87add1aa5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.019+0000 7ff479600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.020+0000 7ff479600700 1 -- 172.21.15.63:0/2709634685 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480021070 con 0x5557b05afe40 2024-05-27T08:08:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.020+0000 7ff479600700 1 -- 172.21.15.63:0/2709634685 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7ff4800a9370 con 0x5557b05afe40 2024-05-27T08:08:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.020+0000 7ff4a5228b80 1 -- 172.21.15.63:0/2709634685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff47c00f960 con 0x5557b05afe40 2024-05-27T08:08:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.020+0000 7ff479600700 1 -- 172.21.15.63:0/2709634685 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff4800cb0d0 con 0x5557b05afe40 2024-05-27T08:08:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.020+0000 7ff4a5228b80 10 client.22146 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:30.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.020+0000 7ff4a5228b80 1 -- 172.21.15.63:0/2709634685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5557b07bc1b0 con 0x5557b05afe40 2024-05-27T08:08:30.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.021+0000 7ff479600700 1 -- 172.21.15.63:0/2709634685 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff47c00f960 con 0x5557b05afe40 2024-05-27T08:08:30.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.021+0000 7ff479600700 1 -- 172.21.15.63:0/2709634685 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff4800a5a80 con 0x5557b05afe40 2024-05-27T08:08:30.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.021+0000 7ff4a5228b80 10 client.22146 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:30.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.021+0000 7ff4a5228b80 10 client.22146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:30.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.021+0000 7ff4a5228b80 20 client.22146 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:30.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.021+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557b07bc6d0 0x5557b07beaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:30.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.021+0000 7ff4a5228b80 4 client.22146 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:08:30.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.021+0000 7ff4a5228b80 1 -- 172.21.15.63:0/2709634685 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557b0658570 con 0x5557b07bc6d0 2024-05-27T08:08:30.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.022+0000 7ff48ea00700 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557b07bc6d0 0x5557b07beaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:30.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.023+0000 7ff48ea00700 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557b07bc6d0 0x5557b07beaf0 crc :-1 s=READY pgs=629 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=928154028762f8b server_cookie=2be5a79e3528fdff in_seq=0 out_seq=0 2024-05-27T08:08:30.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.023+0000 7ff479600700 10 client.22146 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:30.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff479600700 1 -- 172.21.15.63:0/2709634685 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5557b0658570 con 0x5557b07bc6d0 2024-05-27T08:08:30.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff479600700 10 client.22146 handle_command_reply: tid=0 2024-05-27T08:08:30.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff4a5228b80 1 client.22146 shutdown 2024-05-27T08:08:30.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff4a5228b80 1 -- 172.21.15.63:0/2709634685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b05afe40 msgr2=0x5557b0749460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:30.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b05afe40 0x5557b0749460 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7ff480005950 tx=0x7ff4800a97b0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff4a5228b80 1 -- 172.21.15.63:0/2709634685 shutdown_connections 2024-05-27T08:08:30.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5557b07bc6d0 0x5557b07beaf0 crc :-1 s=READY pgs=629 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5557b0734d30 0x5557b0746230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5557b05afe40 0x5557b0749460 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff479600700 0 client.22146 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:30.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff4a5228b80 1 --2- 172.21.15.63:0/2709634685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557b072edb0 0x5557b07499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.024+0000 7ff4a5228b80 1 -- 172.21.15.63:0/2709634685 >> 172.21.15.63:0/2709634685 conn(0x5557b060ffc0 msgr2=0x5557b0729580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:30.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.025+0000 7ff4a5228b80 1 -- 172.21.15.63:0/2709634685 shutdown_connections 2024-05-27T08:08:30.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.025+0000 7ff4a5228b80 1 -- 172.21.15.63:0/2709634685 wait complete. 2024-05-27T08:08:30.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.025+0000 7ff4a5228b80 20 client.22146 trim_cache size 0 max 16384 2024-05-27T08:08:30.028 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:08:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:08:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:08:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "ce112e9a-439a-4b30-9226-c832498909f8": { 2024-05-27T08:08:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:08:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "ce112e9a-439a-4b30-9226-c832498909f8", 2024-05-27T08:08:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:08:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:08:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:08:30.029 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:08:30.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.026+0000 7ff496600700 1 -- 172.21.15.63:0/2061187962 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4740b29f0 msgr2=0x7ff4740b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:30.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.026+0000 7ff496600700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4740b29f0 0x7ff4740b4f00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009750 tx=0x7ff47c005fb0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.026+0000 7ff496600700 1 -- 172.21.15.63:0/2061187962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4901034b0 msgr2=0x7ff4900ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:30.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.026+0000 7ff496600700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4901034b0 0x7ff4900ff290 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7ff48000d950 tx=0x7ff48000dc60 comp rx=0 tx=0).stop 2024-05-27T08:08:30.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.027+0000 7ff496600700 1 -- 172.21.15.63:0/2061187962 shutdown_connections 2024-05-27T08:08:30.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.027+0000 7ff496600700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4740b29f0 0x7ff4740b4f00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.027+0000 7ff496600700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff490108730 0x7ff4900ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.027+0000 7ff496600700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4901034b0 0x7ff4900ff290 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.027+0000 7ff496600700 1 --2- 172.21.15.63:0/2061187962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff490107de0 0x7ff4900ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:30.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.027+0000 7ff496600700 1 -- 172.21.15.63:0/2061187962 >> 172.21.15.63:0/2061187962 conn(0x7ff4900fedb0 msgr2=0x7ff49006ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:30.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.027+0000 7ff496600700 1 -- 172.21.15.63:0/2061187962 shutdown_connections 2024-05-27T08:08:30.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:08:30.027+0000 7ff496600700 1 -- 172.21.15.63:0/2061187962 wait complete. 2024-05-27T08:08:30.939 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:30 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:08:30.940 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:30 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:08:30.940 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:30 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:08:30.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:30 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:08:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:30 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:08:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:30 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:08:30.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:30 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:08:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:30 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:08:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:30 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:08:31.000 INFO:tasks.cephfs.filesystem:scrub status for tag:ce112e9a-439a-4b30-9226-c832498909f8 - {'path': '/', 'tag': 'ce112e9a-439a-4b30-9226-c832498909f8', 'options': 'recursive,force'} 2024-05-27T08:08:31.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:31 smithi086 ceph-mon[34687]: pgmap v1108: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.7 MiB/s wr, 40 op/s 2024-05-27T08:08:31.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:31 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:08:31.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:31 smithi149 ceph-mon[35408]: pgmap v1108: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.7 MiB/s wr, 40 op/s 2024-05-27T08:08:31.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:31 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:08:31.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:31 smithi063 ceph-mon[31613]: pgmap v1108: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.7 MiB/s wr, 40 op/s 2024-05-27T08:08:31.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:31 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:08:33.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:32 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:08:33.110 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:32 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:08:33.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:32 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:08:33.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:32 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:08:33.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:32 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:08:33.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:32 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:08:33.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:33 smithi063 ceph-mon[31613]: pgmap v1109: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.4 MiB/s wr, 47 op/s 2024-05-27T08:08:34.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:33 smithi086 ceph-mon[34687]: pgmap v1109: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.4 MiB/s wr, 47 op/s 2024-05-27T08:08:34.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:33 smithi149 ceph-mon[35408]: pgmap v1109: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.4 MiB/s wr, 47 op/s 2024-05-27T08:08:36.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:35 smithi086 ceph-mon[34687]: pgmap v1110: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.3 MiB/s wr, 43 op/s 2024-05-27T08:08:36.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:35 smithi149 ceph-mon[35408]: pgmap v1110: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.3 MiB/s wr, 43 op/s 2024-05-27T08:08:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:35 smithi063 ceph-mon[31613]: pgmap v1110: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.3 MiB/s wr, 43 op/s 2024-05-27T08:08:38.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:37 smithi086 ceph-mon[34687]: pgmap v1111: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.4 MiB/s wr, 48 op/s 2024-05-27T08:08:38.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:37 smithi149 ceph-mon[35408]: pgmap v1111: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.4 MiB/s wr, 48 op/s 2024-05-27T08:08:38.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:37 smithi063 ceph-mon[31613]: pgmap v1111: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.4 MiB/s wr, 48 op/s 2024-05-27T08:08:40.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:39 smithi086 ceph-mon[34687]: pgmap v1112: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:08:40.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:39 smithi149 ceph-mon[35408]: pgmap v1112: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:08:40.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:39 smithi063 ceph-mon[31613]: pgmap v1112: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:08:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.625+0000 7f92cda00640 1 -- 172.21.15.63:0/571919351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92c810e000 msgr2=0x7f92c81104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.625+0000 7f92cda00640 1 --2- 172.21.15.63:0/571919351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92c810e000 0x7f92c81104f0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009ca0 tx=0x7f92c002a8c0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.625+0000 7f92cda00640 1 -- 172.21.15.63:0/571919351 shutdown_connections 2024-05-27T08:08:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.625+0000 7f92cda00640 1 --2- 172.21.15.63:0/571919351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92c810e000 0x7f92c81104f0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.625+0000 7f92cda00640 1 --2- 172.21.15.63:0/571919351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92c810b610 0x7f92c810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.625+0000 7f92cda00640 1 --2- 172.21.15.63:0/571919351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92c8104bb0 0x7f92c8104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.625+0000 7f92cda00640 1 -- 172.21.15.63:0/571919351 >> 172.21.15.63:0/571919351 conn(0x7f92c80fe210 msgr2=0x7f92c8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.626+0000 7f92cda00640 1 -- 172.21.15.63:0/571919351 shutdown_connections 2024-05-27T08:08:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.626+0000 7f92cda00640 1 -- 172.21.15.63:0/571919351 wait complete. 2024-05-27T08:08:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.627+0000 7f92cda00640 1 Processor -- start 2024-05-27T08:08:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.627+0000 7f92cda00640 1 -- start start 2024-05-27T08:08:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.627+0000 7f92cda00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92c8104bb0 0x7f92c8102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92cda00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92c810b610 0x7f92c81029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92cda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92c810e000 0x7f92c8102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92cda00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8113b60 con 0x7f92c8104bb0 2024-05-27T08:08:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92cda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92c81139e0 con 0x7f92c810e000 2024-05-27T08:08:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92cda00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8113ce0 con 0x7f92c810b610 2024-05-27T08:08:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92c7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92c8104bb0 0x7f92c8102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92c7e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92c8104bb0 0x7f92c8102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45132/0 (socket says 172.21.15.63:45132) 2024-05-27T08:08:41.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92c7e00640 1 -- 172.21.15.63:0/2036453100 learned_addr learned my addr 172.21.15.63:0/2036453100 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92c7400640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92c810b610 0x7f92c81029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92cd000640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92c810e000 0x7f92c8102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92c7e00640 1 -- 172.21.15.63:0/2036453100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92c810b610 msgr2=0x7f92c81029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92c7e00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92c810b610 0x7f92c81029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92c7e00640 1 -- 172.21.15.63:0/2036453100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92c810e000 msgr2=0x7f92c8102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92c7e00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92c810e000 0x7f92c8102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.628+0000 7f92c7e00640 1 -- 172.21.15.63:0/2036453100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c81a88e0 con 0x7f92c8104bb0 2024-05-27T08:08:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.629+0000 7f92cd000640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92c810e000 0x7f92c8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:08:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.629+0000 7f92c7e00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92c8104bb0 0x7f92c8102470 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f92b800d950 tx=0x7f92b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.629+0000 7f92c4c00640 1 -- 172.21.15.63:0/2036453100 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8017070 con 0x7f92c8104bb0 2024-05-27T08:08:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.629+0000 7f92c4c00640 1 -- 172.21.15.63:0/2036453100 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92b8009c30 con 0x7f92c8104bb0 2024-05-27T08:08:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.629+0000 7f92cda00640 1 -- 172.21.15.63:0/2036453100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c81a8bd0 con 0x7f92c8104bb0 2024-05-27T08:08:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.629+0000 7f92c4c00640 1 -- 172.21.15.63:0/2036453100 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b800f620 con 0x7f92c8104bb0 2024-05-27T08:08:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.629+0000 7f92cda00640 1 -- 172.21.15.63:0/2036453100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c81a9120 con 0x7f92c8104bb0 2024-05-27T08:08:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.634+0000 7f92d5ff7740 1 Processor -- start 2024-05-27T08:08:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.634+0000 7f92c4c00640 1 -- 172.21.15.63:0/2036453100 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f92b8010460 con 0x7f92c8104bb0 2024-05-27T08:08:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.634+0000 7f92d5ff7740 1 -- start start 2024-05-27T08:08:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.634+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f4331b0 0x55a19f7d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.634+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f40a2f0 0x55a19f7d5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.634+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.634+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a19f746b90 con 0x55a19f4331b0 2024-05-27T08:08:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.634+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a19f736730 con 0x55a19f4124d0 2024-05-27T08:08:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.634+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a19f73a770 con 0x55a19f40a2f0 2024-05-27T08:08:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.635+0000 7f92cd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f4331b0 0x55a19f7d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.635+0000 7f92cd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f4331b0 0x55a19f7d5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45134/0 (socket says 172.21.15.63:45134) 2024-05-27T08:08:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.635+0000 7f92cd000640 1 -- 172.21.15.63:0/750337178 learned_addr learned my addr 172.21.15.63:0/750337178 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.635+0000 7f92c7e00640 1 --2- 172.21.15.63:0/750337178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f40a2f0 0x55a19f7d5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.635+0000 7f92c7400640 1 --2- 172.21.15.63:0/750337178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7d64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.635+0000 7f92cd000640 1 -- 172.21.15.63:0/750337178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f40a2f0 msgr2=0x55a19f7d5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.635+0000 7f92cd000640 1 --2- 172.21.15.63:0/750337178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f40a2f0 0x55a19f7d5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.635+0000 7f92cd000640 1 -- 172.21.15.63:0/750337178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 msgr2=0x55a19f7d64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.635+0000 7f92cd000640 1 --2- 172.21.15.63:0/750337178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7d64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.635+0000 7f92cd000640 1 -- 172.21.15.63:0/750337178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19f6cf8b0 con 0x55a19f4331b0 2024-05-27T08:08:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.636+0000 7f92cd000640 1 --2- 172.21.15.63:0/750337178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f4331b0 0x55a19f7d5a60 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f92c0006010 tx=0x7f92c0031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cf26522e806518 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.636+0000 7f92c4c00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92ac0b2950 0x7f92ac0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.636+0000 7f92bd600640 1 -- 172.21.15.63:0/750337178 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0031980 con 0x55a19f4331b0 2024-05-27T08:08:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.636+0000 7f92bd600640 1 -- 172.21.15.63:0/750337178 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92c0036070 con 0x55a19f4331b0 2024-05-27T08:08:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.636+0000 7f92bd600640 1 -- 172.21.15.63:0/750337178 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0004480 con 0x55a19f4331b0 2024-05-27T08:08:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.637+0000 7f92c4c00640 1 -- 172.21.15.63:0/2036453100 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f92b80e2700 con 0x7f92c8104bb0 2024-05-27T08:08:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.637+0000 7f92c7e00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92ac0b2950 0x7f92ac0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.638+0000 7f92d5ff7740 1 -- 172.21.15.63:0/750337178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f4331b0 msgr2=0x55a19f7d5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.638+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/750337178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f4331b0 0x55a19f7d5a60 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f92c0006010 tx=0x7f92c0031500 comp rx=0 tx=0).stop 2024-05-27T08:08:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.638+0000 7f92c7e00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92ac0b2950 0x7f92ac0b4e60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55a19f7d3550 tx=0x7f92b80e6040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.638+0000 7f92d5ff7740 1 -- 172.21.15.63:0/750337178 shutdown_connections 2024-05-27T08:08:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.638+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/750337178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7d64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.638+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/750337178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f40a2f0 0x55a19f7d5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.651 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.638+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/750337178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f4331b0 0x55a19f7d5a60 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.638+0000 7f92d5ff7740 1 -- 172.21.15.63:0/750337178 >> 172.21.15.63:0/750337178 conn(0x55a19f427300 msgr2=0x55a19f7d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.638+0000 7f92d5ff7740 1 -- 172.21.15.63:0/750337178 shutdown_connections 2024-05-27T08:08:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.638+0000 7f92d5ff7740 1 -- 172.21.15.63:0/750337178 wait complete. 2024-05-27T08:08:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.639+0000 7f92d5ff7740 1 -- start start 2024-05-27T08:08:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.639+0000 7f92d5ff7740 1 Processor -- start 2024-05-27T08:08:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.639+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7dfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.639+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7e01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.639+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4331b0 0x55a19f7ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.639+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a19f41c790 con 0x55a19f40a2f0 2024-05-27T08:08:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.639+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a19f6debc0 con 0x55a19f4124d0 2024-05-27T08:08:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.639+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a19f7e2b30 con 0x55a19f4331b0 2024-05-27T08:08:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.639+0000 7f92d5ff7740 10 client.0 _ll_register_callbacks cb 0x55a19f6bc400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.640+0000 7f92cd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7dfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.640+0000 7f92cd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7dfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45138/0 (socket says 172.21.15.63:45138) 2024-05-27T08:08:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.640+0000 7f92cd000640 1 -- 172.21.15.63:0/2919665916 learned_addr learned my addr 172.21.15.63:0/2919665916 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.640+0000 7f92c7400640 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7e01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.640+0000 7f92cd000640 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4331b0 0x55a19f7ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.640+0000 7f92cd000640 1 -- 172.21.15.63:0/2919665916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4331b0 msgr2=0x55a19f7ebe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.640+0000 7f92cd000640 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4331b0 0x55a19f7ebe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.640+0000 7f92cd000640 1 -- 172.21.15.63:0/2919665916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 msgr2=0x55a19f7e01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.640+0000 7f92cd000640 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7e01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.641+0000 7f92cd000640 1 -- 172.21.15.63:0/2919665916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19f44b210 con 0x55a19f40a2f0 2024-05-27T08:08:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.641+0000 7f92cd000640 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7dfc70 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f92c0004af0 tx=0x7f92c0004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fd3b1c8475132f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.641+0000 7f92b2000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.641+0000 7f92b2000640 1 -- 172.21.15.63:0/2919665916 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0031b80 con 0x55a19f40a2f0 2024-05-27T08:08:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.641+0000 7f92d5ff7740 1 -- 172.21.15.63:0/2919665916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c0036070 con 0x55a19f40a2f0 2024-05-27T08:08:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.641+0000 7f92b2000640 1 -- 172.21.15.63:0/2919665916 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f92c0031d20 con 0x55a19f40a2f0 2024-05-27T08:08:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92b2000640 1 -- 172.21.15.63:0/2919665916 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f92c005d050 con 0x55a19f40a2f0 2024-05-27T08:08:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92d5ff7740 10 client.22158 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92d5ff7740 1 -- 172.21.15.63:0/2919665916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a19f517620 con 0x55a19f40a2f0 2024-05-27T08:08:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92b2000640 1 -- 172.21.15.63:0/2919665916 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92c0036070 con 0x55a19f40a2f0 2024-05-27T08:08:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92b2000640 1 -- 172.21.15.63:0/2919665916 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f92c0030710 con 0x55a19f40a2f0 2024-05-27T08:08:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92d5ff7740 10 client.22158 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92d5ff7740 10 client.22158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92d5ff7740 20 client.22158 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a19f3f1d80 0x55a19f7f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92d5ff7740 4 client.22158 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.642+0000 7f92d5ff7740 1 -- 172.21.15.63:0/2919665916 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a19f748ae0 con 0x55a19f3f1d80 2024-05-27T08:08:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.643+0000 7f92c7400640 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a19f3f1d80 0x55a19f7f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.644+0000 7f92c7400640 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a19f3f1d80 0x55a19f7f1cb0 crc :-1 s=READY pgs=630 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=888d254eb2e64bcd server_cookie=d61e251451b3a18c in_seq=0 out_seq=0 2024-05-27T08:08:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.644+0000 7f92b2000640 10 client.22158 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92b2000640 1 -- 172.21.15.63:0/2919665916 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a19f748ae0 con 0x55a19f3f1d80 2024-05-27T08:08:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92b2000640 10 client.22158 handle_command_reply: tid=0 2024-05-27T08:08:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92d5ff7740 1 client.22158 shutdown 2024-05-27T08:08:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92d5ff7740 1 -- 172.21.15.63:0/2919665916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 msgr2=0x55a19f7dfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7dfc70 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f92c0004af0 tx=0x7f92c0004000 comp rx=0 tx=0).stop 2024-05-27T08:08:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92d5ff7740 1 -- 172.21.15.63:0/2919665916 shutdown_connections 2024-05-27T08:08:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a19f3f1d80 0x55a19f7f1cb0 crc :-1 s=READY pgs=630 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4331b0 0x55a19f7ebe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7e01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/2919665916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7dfc70 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92d5ff7740 1 -- 172.21.15.63:0/2919665916 >> 172.21.15.63:0/2919665916 conn(0x55a19f427300 msgr2=0x55a19f7d5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92b2000640 0 client.22158 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.646+0000 7f92d5ff7740 1 -- 172.21.15.63:0/2919665916 shutdown_connections 2024-05-27T08:08:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.647+0000 7f92d5ff7740 1 -- 172.21.15.63:0/2919665916 wait complete. 2024-05-27T08:08:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.647+0000 7f92d5ff7740 20 client.22158 trim_cache size 0 max 16384 2024-05-27T08:08:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.664+0000 7f92d5ff7740 1 Processor -- start 2024-05-27T08:08:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.664+0000 7f92d5ff7740 1 -- start start 2024-05-27T08:08:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.664+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7ebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.664+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.664+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4331b0 0x55a19f7f5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.664+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92c81b5a70 con 0x55a19f40a2f0 2024-05-27T08:08:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.664+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92c81b3340 con 0x55a19f4124d0 2024-05-27T08:08:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.664+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92c81b11b0 con 0x55a19f4331b0 2024-05-27T08:08:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.664+0000 7f92cd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7ebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.665+0000 7f92cd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7ebdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45142/0 (socket says 172.21.15.63:45142) 2024-05-27T08:08:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.665+0000 7f92cd000640 1 -- 172.21.15.63:0/1872175305 learned_addr learned my addr 172.21.15.63:0/1872175305 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.665+0000 7f92c7400640 1 --2- 172.21.15.63:0/1872175305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.665+0000 7f92cd000640 1 --2- 172.21.15.63:0/1872175305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4331b0 0x55a19f7f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.665+0000 7f92cd000640 1 -- 172.21.15.63:0/1872175305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4331b0 msgr2=0x55a19f7f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.665+0000 7f92cd000640 1 --2- 172.21.15.63:0/1872175305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4331b0 0x55a19f7f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.665+0000 7f92cd000640 1 -- 172.21.15.63:0/1872175305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 msgr2=0x55a19f7f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.665+0000 7f92cd000640 1 --2- 172.21.15.63:0/1872175305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.665+0000 7f92cd000640 1 -- 172.21.15.63:0/1872175305 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19f3f1fb0 con 0x55a19f40a2f0 2024-05-27T08:08:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.665+0000 7f92cd000640 1 --2- 172.21.15.63:0/1872175305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7ebdd0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f92c0003f50 tx=0x7f92c0033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf22e45d9af4ee6c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.666+0000 7f92bcc00640 1 -- 172.21.15.63:0/1872175305 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0030d00 con 0x55a19f40a2f0 2024-05-27T08:08:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.666+0000 7f92bcc00640 1 -- 172.21.15.63:0/1872175305 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92c0036070 con 0x55a19f40a2f0 2024-05-27T08:08:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.666+0000 7f92bcc00640 1 -- 172.21.15.63:0/1872175305 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0004070 con 0x55a19f40a2f0 2024-05-27T08:08:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.667+0000 7f92d5ff7740 1 -- 172.21.15.63:0/1872175305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 msgr2=0x55a19f7ebdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.667+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/1872175305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7ebdd0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f92c0003f50 tx=0x7f92c0033040 comp rx=0 tx=0).stop 2024-05-27T08:08:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.667+0000 7f92d5ff7740 1 -- 172.21.15.63:0/1872175305 shutdown_connections 2024-05-27T08:08:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.667+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/1872175305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4331b0 0x55a19f7f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.667+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/1872175305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4124d0 0x55a19f7f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.667+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/1872175305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7ebdd0 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.667+0000 7f92d5ff7740 1 -- 172.21.15.63:0/1872175305 >> 172.21.15.63:0/1872175305 conn(0x55a19f427300 msgr2=0x55a19f7d6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.667+0000 7f92d5ff7740 1 -- 172.21.15.63:0/1872175305 shutdown_connections 2024-05-27T08:08:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.667+0000 7f92d5ff7740 1 -- 172.21.15.63:0/1872175305 wait complete. 2024-05-27T08:08:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.668+0000 7f92d5ff7740 1 -- start start 2024-05-27T08:08:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.668+0000 7f92d5ff7740 1 Processor -- start 2024-05-27T08:08:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4124d0 0x55a19f7f2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92d5ff7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4331b0 0x55a19f7f2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92c81b1800 con 0x55a19f40a2f0 2024-05-27T08:08:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92c81b2c70 con 0x55a19f4331b0 2024-05-27T08:08:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92d5ff7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92c81b1ed0 con 0x55a19f4124d0 2024-05-27T08:08:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92d5ff7740 10 client.0 _ll_register_callbacks cb 0x7f92c0031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92cd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92cd000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7f1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45146/0 (socket says 172.21.15.63:45146) 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92cd000640 1 -- 172.21.15.63:0/3832193762 learned_addr learned my addr 172.21.15.63:0/3832193762 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92c7400640 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4124d0 0x55a19f7f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92cd000640 1 -- 172.21.15.63:0/3832193762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4124d0 msgr2=0x55a19f7f2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92cd000640 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4124d0 0x55a19f7f2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92cd000640 1 -- 172.21.15.63:0/3832193762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4331b0 msgr2=0x55a19f7f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92cd000640 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4331b0 0x55a19f7f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.669+0000 7f92cd000640 1 -- 172.21.15.63:0/3832193762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19f8582f0 con 0x55a19f40a2f0 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.670+0000 7f92cd000640 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7f1cc0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f92c00317f0 tx=0x7f92c0009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81180006992afc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.670+0000 7f92b2000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.670+0000 7f92b2000640 1 -- 172.21.15.63:0/3832193762 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0004c00 con 0x55a19f40a2f0 2024-05-27T08:08:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.670+0000 7f92b2000640 1 -- 172.21.15.63:0/3832193762 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f92c0004da0 con 0x55a19f40a2f0 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.670+0000 7f92d5ff7740 1 -- 172.21.15.63:0/3832193762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c0036070 con 0x55a19f40a2f0 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92b2000640 1 -- 172.21.15.63:0/3832193762 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f92c00305c0 con 0x55a19f40a2f0 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92d5ff7740 10 client.22164 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92d5ff7740 1 -- 172.21.15.63:0/3832193762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a19f859560 con 0x55a19f40a2f0 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92b2000640 1 -- 172.21.15.63:0/3832193762 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f92c0036070 con 0x55a19f40a2f0 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92b2000640 1 -- 172.21.15.63:0/3832193762 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f92c0073030 con 0x55a19f40a2f0 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92d5ff7740 10 client.22164 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92d5ff7740 10 client.22164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92d5ff7740 20 client.22164 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a19f3c71c0 0x55a19f85bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92d5ff7740 4 client.22164 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.671+0000 7f92d5ff7740 1 -- 172.21.15.63:0/3832193762 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a19f7422c0 con 0x55a19f3c71c0 2024-05-27T08:08:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.672+0000 7f92c7400640 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a19f3c71c0 0x55a19f85bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.673+0000 7f92b2000640 10 client.22164 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.673+0000 7f92c7400640 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a19f3c71c0 0x55a19f85bc20 crc :-1 s=READY pgs=631 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5df2c37062658c4 server_cookie=b77ef0a8ffdad543 in_seq=0 out_seq=0 2024-05-27T08:08:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92b2000640 1 -- 172.21.15.63:0/3832193762 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a19f7422c0 con 0x55a19f3c71c0 2024-05-27T08:08:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92b2000640 10 client.22164 handle_command_reply: tid=0 2024-05-27T08:08:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92d5ff7740 1 client.22164 shutdown 2024-05-27T08:08:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92d5ff7740 1 -- 172.21.15.63:0/3832193762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 msgr2=0x55a19f7f1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7f1cc0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f92c00317f0 tx=0x7f92c0009b90 comp rx=0 tx=0).stop 2024-05-27T08:08:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92d5ff7740 1 -- 172.21.15.63:0/3832193762 shutdown_connections 2024-05-27T08:08:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a19f3c71c0 0x55a19f85bc20 crc :-1 s=READY pgs=631 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a19f4331b0 0x55a19f7f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a19f4124d0 0x55a19f7f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92d5ff7740 1 --2- 172.21.15.63:0/3832193762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a19f40a2f0 0x55a19f7f1cc0 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92d5ff7740 1 -- 172.21.15.63:0/3832193762 >> 172.21.15.63:0/3832193762 conn(0x55a19f427300 msgr2=0x55a19f7e1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92b2000640 0 client.22164 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.712+0000 7f92d5ff7740 1 -- 172.21.15.63:0/3832193762 shutdown_connections 2024-05-27T08:08:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.713+0000 7f92d5ff7740 1 -- 172.21.15.63:0/3832193762 wait complete. 2024-05-27T08:08:41.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.713+0000 7f92d5ff7740 20 client.22164 trim_cache size 0 max 16384 2024-05-27T08:08:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:08:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:08:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:08:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 -- 172.21.15.63:0/2036453100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92ac0b2950 msgr2=0x7f92ac0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92ac0b2950 0x7f92ac0b4e60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55a19f7d3550 tx=0x7f92b80e6040 comp rx=0 tx=0).stop 2024-05-27T08:08:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 -- 172.21.15.63:0/2036453100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92c8104bb0 msgr2=0x7f92c8102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92c8104bb0 0x7f92c8102470 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f92b800d950 tx=0x7f92b800de10 comp rx=0 tx=0).stop 2024-05-27T08:08:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 -- 172.21.15.63:0/2036453100 shutdown_connections 2024-05-27T08:08:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f92ac0b2950 0x7f92ac0b4e60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92c810e000 0x7f92c8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92c810b610 0x7f92c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 --2- 172.21.15.63:0/2036453100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92c8104bb0 0x7f92c8102470 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 -- 172.21.15.63:0/2036453100 >> 172.21.15.63:0/2036453100 conn(0x7f92c80fe210 msgr2=0x7f92c810e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.714+0000 7f92cda00640 1 -- 172.21.15.63:0/2036453100 shutdown_connections 2024-05-27T08:08:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:41.715+0000 7f92cda00640 1 -- 172.21.15.63:0/2036453100 wait complete. 2024-05-27T08:08:42.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:41 smithi086 ceph-mon[34687]: pgmap v1113: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:08:42.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:41 smithi149 ceph-mon[35408]: pgmap v1113: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:08:42.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:41 smithi063 ceph-mon[31613]: pgmap v1113: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:08:44.019 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:43 smithi063 ceph-mon[31613]: pgmap v1114: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.7 MiB/s wr, 15 op/s 2024-05-27T08:08:44.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:43 smithi086 ceph-mon[34687]: pgmap v1114: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.7 MiB/s wr, 15 op/s 2024-05-27T08:08:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:43 smithi149 ceph-mon[35408]: pgmap v1114: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.7 MiB/s wr, 15 op/s 2024-05-27T08:08:46.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:45 smithi086 ceph-mon[34687]: pgmap v1115: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 8 op/s 2024-05-27T08:08:46.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:45 smithi149 ceph-mon[35408]: pgmap v1115: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 8 op/s 2024-05-27T08:08:46.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:45 smithi063 ceph-mon[31613]: pgmap v1115: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.0 MiB/s wr, 8 op/s 2024-05-27T08:08:47.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:46 smithi086 ceph-mon[34687]: pgmap v1116: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-27T08:08:47.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:46 smithi149 ceph-mon[35408]: pgmap v1116: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-27T08:08:47.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:46 smithi063 ceph-mon[31613]: pgmap v1116: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.7 MiB/s wr, 15 op/s 2024-05-27T08:08:49.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:49 smithi086 ceph-mon[34687]: pgmap v1117: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:49.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:49 smithi149 ceph-mon[35408]: pgmap v1117: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:49.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:49 smithi063 ceph-mon[31613]: pgmap v1117: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:51.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:51 smithi086 ceph-mon[34687]: pgmap v1118: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:51.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:51 smithi149 ceph-mon[35408]: pgmap v1118: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:51.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:51 smithi063 ceph-mon[31613]: pgmap v1118: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:53.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:53 smithi086 ceph-mon[34687]: pgmap v1119: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:08:53.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:53 smithi149 ceph-mon[35408]: pgmap v1119: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:08:53.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:53 smithi063 ceph-mon[31613]: pgmap v1119: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-05-27T08:08:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.996+0000 7f8908e00640 1 -- 172.21.15.63:0/1678109754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f890410a130 msgr2=0x7f8904103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.996+0000 7f8908e00640 1 --2- 172.21.15.63:0/1678109754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f890410a130 0x7f8904103d70 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009900 tx=0x7f88f002a410 comp rx=0 tx=0).stop 2024-05-27T08:08:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.996+0000 7f8908e00640 1 -- 172.21.15.63:0/1678109754 shutdown_connections 2024-05-27T08:08:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.996+0000 7f8908e00640 1 --2- 172.21.15.63:0/1678109754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89041043e0 0x7f89041107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.996+0000 7f8908e00640 1 --2- 172.21.15.63:0/1678109754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f890410a130 0x7f8904103d70 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.996+0000 7f8908e00640 1 --2- 172.21.15.63:0/1678109754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8904109c90 0x7f8904103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.996+0000 7f8908e00640 1 -- 172.21.15.63:0/1678109754 >> 172.21.15.63:0/1678109754 conn(0x7f8904079390 msgr2=0x7f89040797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.997+0000 7f8908e00640 1 -- 172.21.15.63:0/1678109754 shutdown_connections 2024-05-27T08:08:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.997+0000 7f8908e00640 1 -- 172.21.15.63:0/1678109754 wait complete. 2024-05-27T08:08:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.998+0000 7f8908e00640 1 Processor -- start 2024-05-27T08:08:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.999+0000 7f8908e00640 1 -- start start 2024-05-27T08:08:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.999+0000 7f8908e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89041043e0 0x7f890419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.999+0000 7f8908e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8904109c90 0x7f890419dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.999+0000 7f8908e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890410a130 0x7f89041a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:54.999+0000 7f8908e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8904113d60 con 0x7f89041043e0 2024-05-27T08:08:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.000+0000 7f8908e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8904113be0 con 0x7f890410a130 2024-05-27T08:08:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.000+0000 7f8908e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8904113ee0 con 0x7f8904109c90 2024-05-27T08:08:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.000+0000 7f8901e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89041043e0 0x7f890419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.000+0000 7f8901e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89041043e0 0x7f890419d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42340/0 (socket says 172.21.15.63:42340) 2024-05-27T08:08:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.000+0000 7f8901e00640 1 -- 172.21.15.63:0/2693489020 learned_addr learned my addr 172.21.15.63:0/2693489020 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.000+0000 7f8901400640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8904109c90 0x7f890419dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.000+0000 7f8902800640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890410a130 0x7f89041a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.001+0000 7f8901e00640 1 -- 172.21.15.63:0/2693489020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8904109c90 msgr2=0x7f890419dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.001+0000 7f8901e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8904109c90 0x7f890419dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.001+0000 7f8901e00640 1 -- 172.21.15.63:0/2693489020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890410a130 msgr2=0x7f89041a86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.001+0000 7f8901e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890410a130 0x7f89041a86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.001+0000 7f8901e00640 1 -- 172.21.15.63:0/2693489020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89041a8c00 con 0x7f89041043e0 2024-05-27T08:08:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.001+0000 7f8901e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89041043e0 0x7f890419d820 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f88f400d770 tx=0x7f88f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.001+0000 7f88faa00640 1 -- 172.21.15.63:0/2693489020 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4017070 con 0x7f89041043e0 2024-05-27T08:08:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.002+0000 7f8908e00640 1 -- 172.21.15.63:0/2693489020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89041a8e90 con 0x7f89041043e0 2024-05-27T08:08:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.002+0000 7f88faa00640 1 -- 172.21.15.63:0/2693489020 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88f4009c30 con 0x7f89041043e0 2024-05-27T08:08:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.002+0000 7f88faa00640 1 -- 172.21.15.63:0/2693489020 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f400f620 con 0x7f89041043e0 2024-05-27T08:08:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.002+0000 7f8908e00640 1 -- 172.21.15.63:0/2693489020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89041a9350 con 0x7f89041043e0 2024-05-27T08:08:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.006+0000 7f890fe16740 1 Processor -- start 2024-05-27T08:08:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.006+0000 7f890fe16740 1 -- start start 2024-05-27T08:08:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.007+0000 7f890fe16740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 0x5630b1888750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.007+0000 7f890fe16740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14bd2f0 0x5630b1888c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.007+0000 7f890fe16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14c54d0 0x5630b18891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.007+0000 7f890fe16740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5630b17f9b90 con 0x5630b14bd2f0 2024-05-27T08:08:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.007+0000 7f890fe16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630b17e9730 con 0x5630b14c54d0 2024-05-27T08:08:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.007+0000 7f890fe16740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5630b17ed770 con 0x5630b14e61b0 2024-05-27T08:08:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.007+0000 7f8902800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 0x5630b1888750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.007+0000 7f8902800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 0x5630b1888750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:35600/0 (socket says 172.21.15.63:35600) 2024-05-27T08:08:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.007+0000 7f8902800640 1 -- 172.21.15.63:0/2130527671 learned_addr learned my addr 172.21.15.63:0/2130527671 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.007+0000 7f8901400640 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14c54d0 0x5630b18891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.008+0000 7f88faa00640 1 -- 172.21.15.63:0/2693489020 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f88f401a070 con 0x7f89041043e0 2024-05-27T08:08:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.008+0000 7f8901e00640 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14bd2f0 0x5630b1888c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.008+0000 7f8902800640 1 -- 172.21.15.63:0/2130527671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14c54d0 msgr2=0x5630b18891d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.008+0000 7f8902800640 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14c54d0 0x5630b18891d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.008+0000 7f8902800640 1 -- 172.21.15.63:0/2130527671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14bd2f0 msgr2=0x5630b1888c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.008+0000 7f8902800640 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14bd2f0 0x5630b1888c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.008+0000 7f8902800640 1 -- 172.21.15.63:0/2130527671 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b17828b0 con 0x5630b14e61b0 2024-05-27T08:08:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.008+0000 7f8901e00640 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14bd2f0 0x5630b1888c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:08:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.009+0000 7f8901400640 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14c54d0 0x5630b18891d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:08:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.009+0000 7f8902800640 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 0x5630b1888750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009eb0 tx=0x7f88fc010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abfc4b4cd178e8dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.009+0000 7f88ef400640 1 -- 172.21.15.63:0/2130527671 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc016070 con 0x5630b14e61b0 2024-05-27T08:08:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.010+0000 7f88ef400640 1 -- 172.21.15.63:0/2130527671 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88fc004d20 con 0x5630b14e61b0 2024-05-27T08:08:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.010+0000 7f88ef400640 1 -- 172.21.15.63:0/2130527671 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc0126b0 con 0x5630b14e61b0 2024-05-27T08:08:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.010+0000 7f88faa00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88e80b2950 0x7f88e80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.010+0000 7f88faa00640 1 -- 172.21.15.63:0/2693489020 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f88f40e2350 con 0x7f89041043e0 2024-05-27T08:08:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.010+0000 7f8901e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88e80b2950 0x7f88e80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.012+0000 7f890fe16740 1 -- 172.21.15.63:0/2130527671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 msgr2=0x5630b1888750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.012+0000 7f890fe16740 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 0x5630b1888750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009eb0 tx=0x7f88fc010740 comp rx=0 tx=0).stop 2024-05-27T08:08:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.012+0000 7f8901e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88e80b2950 0x7f88e80b4e60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f88f40df790 tx=0x7f88f40057f0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.012+0000 7f890fe16740 1 -- 172.21.15.63:0/2130527671 shutdown_connections 2024-05-27T08:08:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.012+0000 7f890fe16740 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14c54d0 0x5630b18891d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.012+0000 7f890fe16740 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14bd2f0 0x5630b1888c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.012+0000 7f890fe16740 1 --2- 172.21.15.63:0/2130527671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 0x5630b1888750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.012+0000 7f890fe16740 1 -- 172.21.15.63:0/2130527671 >> 172.21.15.63:0/2130527671 conn(0x5630b14da300 msgr2=0x5630b1885010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.012+0000 7f890fe16740 1 -- 172.21.15.63:0/2130527671 shutdown_connections 2024-05-27T08:08:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.012+0000 7f890fe16740 1 -- 172.21.15.63:0/2130527671 wait complete. 2024-05-27T08:08:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.013+0000 7f890fe16740 1 -- start start 2024-05-27T08:08:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.013+0000 7f890fe16740 1 Processor -- start 2024-05-27T08:08:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f890fe16740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14bd2f0 0x5630b189b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f890fe16740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f890fe16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14e61b0 0x5630b18a7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f890fe16740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5630b14cf790 con 0x5630b14c54d0 2024-05-27T08:08:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f890fe16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630b1791bc0 con 0x5630b14e61b0 2024-05-27T08:08:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f890fe16740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5630b189e1a0 con 0x5630b14bd2f0 2024-05-27T08:08:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f890fe16740 10 client.0 _ll_register_callbacks cb 0x5630b176f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f8901400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f8901400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42358/0 (socket says 172.21.15.63:42358) 2024-05-27T08:08:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f8901400640 1 -- 172.21.15.63:0/4620300 learned_addr learned my addr 172.21.15.63:0/4620300 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.014+0000 7f8902800640 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14bd2f0 0x5630b189b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.015+0000 7f8902800640 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14e61b0 0x5630b18a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.015+0000 7f8901400640 1 -- 172.21.15.63:0/4620300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14bd2f0 msgr2=0x5630b189b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.015+0000 7f8901400640 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14bd2f0 0x5630b189b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.015+0000 7f8901400640 1 -- 172.21.15.63:0/4620300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14e61b0 msgr2=0x5630b18a7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.015+0000 7f8901400640 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14e61b0 0x5630b18a7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.015+0000 7f8901400640 1 -- 172.21.15.63:0/4620300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b14fe210 con 0x5630b14c54d0 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.016+0000 7f8901400640 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189b730 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f88f00057c0 tx=0x7f88f00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cd8d5c7cc290d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.016+0000 7f88e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.016+0000 7f88e3e00640 1 -- 172.21.15.63:0/4620300 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0035040 con 0x5630b14c54d0 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.016+0000 7f88e3e00640 1 -- 172.21.15.63:0/4620300 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f88f0003c40 con 0x5630b14c54d0 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.016+0000 7f890fe16740 1 -- 172.21.15.63:0/4620300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88fc004d20 con 0x5630b14c54d0 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.016+0000 7f88e3e00640 1 -- 172.21.15.63:0/4620300 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f88f004e050 con 0x5630b14c54d0 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.016+0000 7f890fe16740 10 client.22173 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.016+0000 7f890fe16740 1 -- 172.21.15.63:0/4620300 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5630b15ca620 con 0x5630b14c54d0 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.017+0000 7f88e3e00640 1 -- 172.21.15.63:0/4620300 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88fc004d20 con 0x5630b14c54d0 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.017+0000 7f88e3e00640 1 -- 172.21.15.63:0/4620300 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f88f0031400 con 0x5630b14c54d0 2024-05-27T08:08:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.017+0000 7f890fe16740 10 client.22173 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.017+0000 7f890fe16740 10 client.22173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.017+0000 7f890fe16740 20 client.22173 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.017+0000 7f890fe16740 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630b14a4d80 0x5630b18ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.017+0000 7f890fe16740 4 client.22173 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:08:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.017+0000 7f890fe16740 1 -- 172.21.15.63:0/4620300 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630b17fbae0 con 0x5630b14a4d80 2024-05-27T08:08:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.018+0000 7f8902800640 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630b14a4d80 0x5630b18ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.019+0000 7f88e3e00640 10 client.22173 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.019+0000 7f8902800640 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630b14a4d80 0x5630b18ad460 crc :-1 s=READY pgs=632 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2fd022d3543b509 server_cookie=8a9fc26952ac4716 in_seq=0 out_seq=0 2024-05-27T08:08:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f88e3e00640 1 -- 172.21.15.63:0/4620300 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5630b17fbae0 con 0x5630b14a4d80 2024-05-27T08:08:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f88e3e00640 10 client.22173 handle_command_reply: tid=0 2024-05-27T08:08:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 client.22173 shutdown 2024-05-27T08:08:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 -- 172.21.15.63:0/4620300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 msgr2=0x5630b189b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189b730 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f88f00057c0 tx=0x7f88f00045e0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 -- 172.21.15.63:0/4620300 shutdown_connections 2024-05-27T08:08:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630b14a4d80 0x5630b18ad460 crc :-1 s=READY pgs=632 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14e61b0 0x5630b18a7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189b730 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 --2- 172.21.15.63:0/4620300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14bd2f0 0x5630b189b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f88e3e00640 0 client.22173 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 -- 172.21.15.63:0/4620300 >> 172.21.15.63:0/4620300 conn(0x5630b14da300 msgr2=0x5630b188a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 -- 172.21.15.63:0/4620300 shutdown_connections 2024-05-27T08:08:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.021+0000 7f890fe16740 1 -- 172.21.15.63:0/4620300 wait complete. 2024-05-27T08:08:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.022+0000 7f890fe16740 20 client.22173 trim_cache size 0 max 16384 2024-05-27T08:08:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.043+0000 7f890fe16740 1 Processor -- start 2024-05-27T08:08:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.043+0000 7f890fe16740 1 -- start start 2024-05-27T08:08:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.044+0000 7f890fe16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14bd2f0 0x5630b1884330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.044+0000 7f890fe16740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.044+0000 7f890fe16740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 0x5630b1888d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.044+0000 7f890fe16740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88e40127e0 con 0x5630b14c54d0 2024-05-27T08:08:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.044+0000 7f890fe16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88e40100b0 con 0x5630b14bd2f0 2024-05-27T08:08:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.044+0000 7f890fe16740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88e400df20 con 0x5630b14e61b0 2024-05-27T08:08:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.044+0000 7f8901400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.044+0000 7f8901400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42364/0 (socket says 172.21.15.63:42364) 2024-05-27T08:08:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.044+0000 7f8901400640 1 -- 172.21.15.63:0/3777267126 learned_addr learned my addr 172.21.15.63:0/3777267126 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.044+0000 7f8902800640 1 --2- 172.21.15.63:0/3777267126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 0x5630b1888d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.045+0000 7f8902800640 1 --2- 172.21.15.63:0/3777267126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14bd2f0 0x5630b1884330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.045+0000 7f8901400640 1 -- 172.21.15.63:0/3777267126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 msgr2=0x5630b1888d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.045+0000 7f8901400640 1 --2- 172.21.15.63:0/3777267126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 0x5630b1888d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.045+0000 7f8901400640 1 -- 172.21.15.63:0/3777267126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14bd2f0 msgr2=0x5630b1884330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.045+0000 7f8901400640 1 --2- 172.21.15.63:0/3777267126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14bd2f0 0x5630b1884330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.045+0000 7f8901400640 1 -- 172.21.15.63:0/3777267126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b189bd80 con 0x5630b14c54d0 2024-05-27T08:08:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.045+0000 7f8901400640 1 --2- 172.21.15.63:0/3777267126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189d340 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f88f00317f0 tx=0x7f88f002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93460b1569ed9753 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.045+0000 7f88eea00640 1 -- 172.21.15.63:0/3777267126 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0035070 con 0x5630b14c54d0 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.046+0000 7f88eea00640 1 -- 172.21.15.63:0/3777267126 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88f0005850 con 0x5630b14c54d0 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.046+0000 7f88eea00640 1 -- 172.21.15.63:0/3777267126 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0031e60 con 0x5630b14c54d0 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.047+0000 7f890fe16740 1 -- 172.21.15.63:0/3777267126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 msgr2=0x5630b189d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.047+0000 7f890fe16740 1 --2- 172.21.15.63:0/3777267126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189d340 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f88f00317f0 tx=0x7f88f002a410 comp rx=0 tx=0).stop 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.047+0000 7f890fe16740 1 -- 172.21.15.63:0/3777267126 shutdown_connections 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.047+0000 7f890fe16740 1 --2- 172.21.15.63:0/3777267126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14e61b0 0x5630b1888d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.047+0000 7f890fe16740 1 --2- 172.21.15.63:0/3777267126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14c54d0 0x5630b189d340 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.047+0000 7f890fe16740 1 --2- 172.21.15.63:0/3777267126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14bd2f0 0x5630b1884330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.047+0000 7f890fe16740 1 -- 172.21.15.63:0/3777267126 >> 172.21.15.63:0/3777267126 conn(0x5630b14da300 msgr2=0x5630b18885f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.047+0000 7f890fe16740 1 -- 172.21.15.63:0/3777267126 shutdown_connections 2024-05-27T08:08:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.047+0000 7f890fe16740 1 -- 172.21.15.63:0/3777267126 wait complete. 2024-05-27T08:08:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.047+0000 7f890fe16740 1 -- start start 2024-05-27T08:08:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.048+0000 7f890fe16740 1 Processor -- start 2024-05-27T08:08:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.048+0000 7f890fe16740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14bd2f0 0x5630b1895240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.048+0000 7f890fe16740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14c54d0 0x5630b1895780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.048+0000 7f890fe16740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14e61b0 0x5630b18a2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.048+0000 7f890fe16740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88e400e570 con 0x5630b14e61b0 2024-05-27T08:08:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.048+0000 7f890fe16740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88e400f9e0 con 0x5630b14bd2f0 2024-05-27T08:08:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.048+0000 7f890fe16740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88e400ec40 con 0x5630b14c54d0 2024-05-27T08:08:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.048+0000 7f890fe16740 10 client.0 _ll_register_callbacks cb 0x7f88f0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:08:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.049+0000 7f8902800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14e61b0 0x5630b18a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.049+0000 7f8902800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14e61b0 0x5630b18a2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42374/0 (socket says 172.21.15.63:42374) 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.049+0000 7f8902800640 1 -- 172.21.15.63:0/1483209627 learned_addr learned my addr 172.21.15.63:0/1483209627 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.049+0000 7f8902800640 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14bd2f0 0x5630b1895240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.049+0000 7f8901400640 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14c54d0 0x5630b1895780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.049+0000 7f8902800640 1 -- 172.21.15.63:0/1483209627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14c54d0 msgr2=0x5630b1895780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.049+0000 7f8902800640 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14c54d0 0x5630b1895780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.049+0000 7f8902800640 1 -- 172.21.15.63:0/1483209627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14bd2f0 msgr2=0x5630b1895240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.049+0000 7f8902800640 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14bd2f0 0x5630b1895240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.049+0000 7f8902800640 1 -- 172.21.15.63:0/1483209627 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b1895fa0 con 0x5630b14e61b0 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.050+0000 7f8902800640 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14e61b0 0x5630b18a2460 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f88fc010c40 tx=0x7f88fc01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baafc2ade7701cfb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.050+0000 7f88e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:08:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.050+0000 7f88e3e00640 1 -- 172.21.15.63:0/1483209627 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc030030 con 0x5630b14e61b0 2024-05-27T08:08:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.050+0000 7f88e3e00640 1 -- 172.21.15.63:0/1483209627 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7f88fc025040 con 0x5630b14e61b0 2024-05-27T08:08:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.050+0000 7f890fe16740 1 -- 172.21.15.63:0/1483209627 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f0005850 con 0x5630b14e61b0 2024-05-27T08:08:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.050+0000 7f88e3e00640 1 -- 172.21.15.63:0/1483209627 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f88fc016040 con 0x5630b14e61b0 2024-05-27T08:08:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.050+0000 7f890fe16740 10 client.22179 fetch_fsmap learned FSMap version 35 2024-05-27T08:08:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.050+0000 7f890fe16740 1 -- 172.21.15.63:0/1483209627 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5630b18a3980 con 0x5630b14e61b0 2024-05-27T08:08:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.051+0000 7f88e3e00640 1 -- 172.21.15.63:0/1483209627 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88f0005850 con 0x5630b14e61b0 2024-05-27T08:08:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.051+0000 7f88e3e00640 1 -- 172.21.15.63:0/1483209627 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f88fc00f040 con 0x5630b14e61b0 2024-05-27T08:08:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.051+0000 7f890fe16740 10 client.22179 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:08:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.051+0000 7f890fe16740 10 client.22179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:08:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.051+0000 7f890fe16740 20 client.22179 populate_metadata read hostname 'smithi063' 2024-05-27T08:08:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.051+0000 7f890fe16740 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630b147a1c0 0x5630b1917ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:08:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.051+0000 7f890fe16740 4 client.22179 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:08:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.051+0000 7f890fe16740 1 -- 172.21.15.63:0/1483209627 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630b17f52c0 con 0x5630b147a1c0 2024-05-27T08:08:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.052+0000 7f8901400640 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630b147a1c0 0x5630b1917ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:08:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.053+0000 7f8901400640 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630b147a1c0 0x5630b1917ae0 crc :-1 s=READY pgs=633 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fb1181608bce46 server_cookie=6c5af160dbd0206e in_seq=0 out_seq=0 2024-05-27T08:08:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.053+0000 7f88e3e00640 10 client.22179 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.088+0000 7f88e3e00640 1 -- 172.21.15.63:0/1483209627 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5630b17f52c0 con 0x5630b147a1c0 2024-05-27T08:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.088+0000 7f88e3e00640 10 client.22179 handle_command_reply: tid=0 2024-05-27T08:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.088+0000 7f890fe16740 1 client.22179 shutdown 2024-05-27T08:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 1 -- 172.21.15.63:0/1483209627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14e61b0 msgr2=0x5630b18a2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14e61b0 0x5630b18a2460 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f88fc010c40 tx=0x7f88fc01c040 comp rx=0 tx=0).stop 2024-05-27T08:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 1 -- 172.21.15.63:0/1483209627 shutdown_connections 2024-05-27T08:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5630b147a1c0 0x5630b1917ae0 crc :-1 s=READY pgs=633 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5630b14e61b0 0x5630b18a2460 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5630b14c54d0 0x5630b1895780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 1 --2- 172.21.15.63:0/1483209627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630b14bd2f0 0x5630b1895240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f88e3e00640 0 client.22179 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 1 -- 172.21.15.63:0/1483209627 >> 172.21.15.63:0/1483209627 conn(0x5630b14da300 msgr2=0x5630b1889020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 1 -- 172.21.15.63:0/1483209627 shutdown_connections 2024-05-27T08:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 1 -- 172.21.15.63:0/1483209627 wait complete. 2024-05-27T08:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.089+0000 7f890fe16740 20 client.22179 trim_cache size 0 max 16384 2024-05-27T08:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:08:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.090+0000 7f8908e00640 1 -- 172.21.15.63:0/2693489020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88e80b2950 msgr2=0x7f88e80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.090+0000 7f8908e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88e80b2950 0x7f88e80b4e60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f88f40df790 tx=0x7f88f40057f0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.090+0000 7f8908e00640 1 -- 172.21.15.63:0/2693489020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89041043e0 msgr2=0x7f890419d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.090+0000 7f8908e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89041043e0 0x7f890419d820 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f88f400d770 tx=0x7f88f400dc30 comp rx=0 tx=0).stop 2024-05-27T08:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.090+0000 7f8908e00640 1 -- 172.21.15.63:0/2693489020 shutdown_connections 2024-05-27T08:08:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.090+0000 7f8908e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88e80b2950 0x7f88e80b4e60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.090+0000 7f8908e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890410a130 0x7f89041a86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.090+0000 7f8908e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8904109c90 0x7f890419dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.090+0000 7f8908e00640 1 --2- 172.21.15.63:0/2693489020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89041043e0 0x7f890419d820 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:08:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.090+0000 7f8908e00640 1 -- 172.21.15.63:0/2693489020 >> 172.21.15.63:0/2693489020 conn(0x7f8904079390 msgr2=0x7f89041023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:08:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.091+0000 7f8908e00640 1 -- 172.21.15.63:0/2693489020 shutdown_connections 2024-05-27T08:08:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:08:55.091+0000 7f8908e00640 1 -- 172.21.15.63:0/2693489020 wait complete. 2024-05-27T08:08:55.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:55 smithi086 ceph-mon[34687]: pgmap v1120: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:55.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:55 smithi149 ceph-mon[35408]: pgmap v1120: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:55.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:55 smithi063 ceph-mon[31613]: pgmap v1120: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:57.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:57 smithi086 ceph-mon[34687]: pgmap v1121: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 16 op/s 2024-05-27T08:08:57.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:57 smithi149 ceph-mon[35408]: pgmap v1121: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 16 op/s 2024-05-27T08:08:57.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:57 smithi063 ceph-mon[31613]: pgmap v1121: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 16 op/s 2024-05-27T08:08:59.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:08:59 smithi086 ceph-mon[34687]: pgmap v1122: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:59.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:08:59 smithi149 ceph-mon[35408]: pgmap v1122: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:08:59.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:08:59 smithi063 ceph-mon[31613]: pgmap v1122: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:09:01.002 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:09:01.575 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:09:01.741 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:01 smithi063 ceph-mon[31613]: pgmap v1123: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:09:01.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:01 smithi086 ceph-mon[34687]: pgmap v1123: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:09:01.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:01 smithi149 ceph-mon[35408]: pgmap v1123: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:09:03.841 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:03 smithi063 ceph-mon[31613]: pgmap v1124: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.0 MiB/s wr, 31 op/s 2024-05-27T08:09:03.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:03 smithi086 ceph-mon[34687]: pgmap v1124: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.0 MiB/s wr, 31 op/s 2024-05-27T08:09:03.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:03 smithi149 ceph-mon[35408]: pgmap v1124: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.0 MiB/s wr, 31 op/s 2024-05-27T08:09:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.117+0000 7fbda1400700 1 -- 172.21.15.63:0/3946936160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd9c076420 msgr2=0x7fbd9c1106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.117+0000 7fbda1400700 1 --2- 172.21.15.63:0/3946936160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd9c076420 0x7fbd9c1106d0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009990 tx=0x7fbd88009ca0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.117+0000 7fbda1400700 1 -- 172.21.15.63:0/3946936160 shutdown_connections 2024-05-27T08:09:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.117+0000 7fbda1400700 1 --2- 172.21.15.63:0/3946936160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c110c10 0x7fbd9c113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.117+0000 7fbda1400700 1 --2- 172.21.15.63:0/3946936160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c075ad0 0x7fbd9c075ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.117+0000 7fbda1400700 1 --2- 172.21.15.63:0/3946936160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd9c076420 0x7fbd9c1106d0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.117+0000 7fbda1400700 1 -- 172.21.15.63:0/3946936160 >> 172.21.15.63:0/3946936160 conn(0x7fbd9c100030 msgr2=0x7fbd9c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:05.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.117+0000 7fbda1400700 1 -- 172.21.15.63:0/3946936160 shutdown_connections 2024-05-27T08:09:05.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.117+0000 7fbda1400700 1 -- 172.21.15.63:0/3946936160 wait complete. 2024-05-27T08:09:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.119+0000 7fbda1400700 1 Processor -- start 2024-05-27T08:09:05.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.120+0000 7fbda1400700 1 -- start start 2024-05-27T08:09:05.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.120+0000 7fbda1400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c075ad0 0x7fbd9c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.120+0000 7fbda1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd9c076420 0x7fbd9c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.120+0000 7fbda1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c110c10 0x7fbd9c1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.120+0000 7fbda1400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c116570 con 0x7fbd9c076420 2024-05-27T08:09:05.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.120+0000 7fbda1400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1163f0 con 0x7fbd9c110c10 2024-05-27T08:09:05.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.120+0000 7fbda1400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1166f0 con 0x7fbd9c075ad0 2024-05-27T08:09:05.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.121+0000 7fbd99e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd9c076420 0x7fbd9c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.121+0000 7fbd99e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd9c076420 0x7fbd9c19d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57372/0 (socket says 172.21.15.63:57372) 2024-05-27T08:09:05.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.121+0000 7fbd99e00700 1 -- 172.21.15.63:0/3898079335 learned_addr learned my addr 172.21.15.63:0/3898079335 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:05.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.121+0000 7fbd9a800700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c075ad0 0x7fbd9c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.121+0000 7fbd9b200700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c110c10 0x7fbd9c1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.122+0000 7fbd99e00700 1 -- 172.21.15.63:0/3898079335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c075ad0 msgr2=0x7fbd9c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:05.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.122+0000 7fbd99e00700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c075ad0 0x7fbd9c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.122+0000 7fbd99e00700 1 -- 172.21.15.63:0/3898079335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c110c10 msgr2=0x7fbd9c1a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:05.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.122+0000 7fbd99e00700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c110c10 0x7fbd9c1a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.122+0000 7fbd99e00700 1 -- 172.21.15.63:0/3898079335 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c1a1eb0 con 0x7fbd9c076420 2024-05-27T08:09:05.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.122+0000 7fbd99e00700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd9c076420 0x7fbd9c19d580 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fbd880098e0 tx=0x7fbd880057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:05.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.123+0000 7fbd93400700 1 -- 172.21.15.63:0/3898079335 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88027070 con 0x7fbd9c076420 2024-05-27T08:09:05.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.123+0000 7fbd93400700 1 -- 172.21.15.63:0/3898079335 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd88022470 con 0x7fbd9c076420 2024-05-27T08:09:05.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.123+0000 7fbd93400700 1 -- 172.21.15.63:0/3898079335 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88017650 con 0x7fbd9c076420 2024-05-27T08:09:05.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.123+0000 7fbda1400700 1 -- 172.21.15.63:0/3898079335 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c1a2140 con 0x7fbd9c076420 2024-05-27T08:09:05.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.123+0000 7fbda1400700 1 -- 172.21.15.63:0/3898079335 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c1a9ac0 con 0x7fbd9c076420 2024-05-27T08:09:05.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.127+0000 7fbd93400700 1 -- 172.21.15.63:0/3898079335 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbd8802a070 con 0x7fbd9c076420 2024-05-27T08:09:05.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.129+0000 7fbd93400700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd800b2a70 0x7fbd800b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.130+0000 7fbd93400700 1 -- 172.21.15.63:0/3898079335 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7fbd880e3830 con 0x7fbd9c076420 2024-05-27T08:09:05.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.130+0000 7fbd9a800700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd800b2a70 0x7fbd800b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.131+0000 7fbd9a800700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd800b2a70 0x7fbd800b4f80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009c60 tx=0x7fbd8c009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:05.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.151+0000 7fbdaff2eb80 1 Processor -- start 2024-05-27T08:09:05.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.151+0000 7fbdaff2eb80 1 -- start start 2024-05-27T08:09:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.151+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad210fa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.151+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad210fa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.151+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210fae70 0x55ad21107110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.151+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ad21028e40 con 0x55ad210679f0 2024-05-27T08:09:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.151+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad210189e0 con 0x55ad210fae70 2024-05-27T08:09:05.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.151+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ad2101ca20 con 0x55ad20f7be40 2024-05-27T08:09:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.152+0000 7fbd9a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad210fa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.152+0000 7fbd9a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad210fa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57376/0 (socket says 172.21.15.63:57376) 2024-05-27T08:09:05.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.152+0000 7fbd9a800700 1 -- 172.21.15.63:0/3407397569 learned_addr learned my addr 172.21.15.63:0/3407397569 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.152+0000 7fbd99e00700 1 --2- 172.21.15.63:0/3407397569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad210fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.152+0000 7fbd9b200700 1 --2- 172.21.15.63:0/3407397569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210fae70 0x55ad21107110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.153+0000 7fbd9a800700 1 -- 172.21.15.63:0/3407397569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 msgr2=0x55ad210fa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.153+0000 7fbd9a800700 1 --2- 172.21.15.63:0/3407397569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad210fa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.153+0000 7fbd9a800700 1 -- 172.21.15.63:0/3407397569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210fae70 msgr2=0x55ad21107110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.153+0000 7fbd9a800700 1 --2- 172.21.15.63:0/3407397569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210fae70 0x55ad21107110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.153+0000 7fbd9a800700 1 -- 172.21.15.63:0/3407397569 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad20fe1420 con 0x55ad210679f0 2024-05-27T08:09:05.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.153+0000 7fbd9a800700 1 --2- 172.21.15.63:0/3407397569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad210fa330 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00e4e0 tx=0x7fbd8c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c6d236350c6642 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:05.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.154+0000 7fbd87e00700 1 -- 172.21.15.63:0/3407397569 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00ee00 con 0x55ad210679f0 2024-05-27T08:09:05.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.154+0000 7fbd87e00700 1 -- 172.21.15.63:0/3407397569 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd8c01c8a0 con 0x55ad210679f0 2024-05-27T08:09:05.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.154+0000 7fbd87e00700 1 -- 172.21.15.63:0/3407397569 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c01d950 con 0x55ad210679f0 2024-05-27T08:09:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.155+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/3407397569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 msgr2=0x55ad210fa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.155+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/3407397569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad210fa330 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00e4e0 tx=0x7fbd8c014000 comp rx=0 tx=0).stop 2024-05-27T08:09:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.155+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/3407397569 shutdown_connections 2024-05-27T08:09:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.155+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/3407397569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210fae70 0x55ad21107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.155+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/3407397569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad210fa330 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.155+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/3407397569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad210fa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.155+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/3407397569 >> 172.21.15.63:0/3407397569 conn(0x55ad21015390 msgr2=0x55ad210f64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.155+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/3407397569 shutdown_connections 2024-05-27T08:09:05.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.155+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/3407397569 wait complete. 2024-05-27T08:09:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.156+0000 7fbdaff2eb80 1 -- start start 2024-05-27T08:09:05.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.156+0000 7fbdaff2eb80 1 Processor -- start 2024-05-27T08:09:05.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.157+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad21113200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.157+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210679f0 0x55ad21113740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.157+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210fae70 0x55ad2110d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.157+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ad20f1efa0 con 0x55ad210fae70 2024-05-27T08:09:05.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.157+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad20fb9760 con 0x55ad210679f0 2024-05-27T08:09:05.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.157+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ad21107b80 con 0x55ad20f7be40 2024-05-27T08:09:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.157+0000 7fbdaff2eb80 10 client.0 _ll_register_callbacks cb 0x55ad20fc2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.157+0000 7fbd9b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210fae70 0x55ad2110d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.158+0000 7fbd9b200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210fae70 0x55ad2110d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57386/0 (socket says 172.21.15.63:57386) 2024-05-27T08:09:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.158+0000 7fbd9b200700 1 -- 172.21.15.63:0/1367384730 learned_addr learned my addr 172.21.15.63:0/1367384730 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.158+0000 7fbd99e00700 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210679f0 0x55ad21113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.158+0000 7fbd9a800700 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad21113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.158+0000 7fbd9b200700 1 -- 172.21.15.63:0/1367384730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 msgr2=0x55ad21113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:05.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.158+0000 7fbd9b200700 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad21113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.158+0000 7fbd9b200700 1 -- 172.21.15.63:0/1367384730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210679f0 msgr2=0x55ad21113740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:05.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.158+0000 7fbd9b200700 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210679f0 0x55ad21113740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.158+0000 7fbd9b200700 1 -- 172.21.15.63:0/1367384730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad20f688d0 con 0x55ad210fae70 2024-05-27T08:09:05.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.159+0000 7fbd9b200700 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210fae70 0x55ad2110d600 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fbd940042f0 tx=0x7fbd94004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60040d7ed4154ccc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:05.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.159+0000 7fbd84c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:05.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.159+0000 7fbd84c00700 1 -- 172.21.15.63:0/1367384730 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9401a070 con 0x55ad210fae70 2024-05-27T08:09:05.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.159+0000 7fbd84c00700 1 -- 172.21.15.63:0/1367384730 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7fbd94007700 con 0x55ad210fae70 2024-05-27T08:09:05.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.159+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1367384730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd8c01c8a0 con 0x55ad210fae70 2024-05-27T08:09:05.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.160+0000 7fbd84c00700 1 -- 172.21.15.63:0/1367384730 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbd94026050 con 0x55ad210fae70 2024-05-27T08:09:05.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.160+0000 7fbdaff2eb80 10 client.22191 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:05.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.160+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1367384730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ad20fe9060 con 0x55ad210fae70 2024-05-27T08:09:05.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.160+0000 7fbd84c00700 1 -- 172.21.15.63:0/1367384730 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd8c01c8a0 con 0x55ad210fae70 2024-05-27T08:09:05.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.160+0000 7fbd84c00700 1 -- 172.21.15.63:0/1367384730 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbd94016910 con 0x55ad210fae70 2024-05-27T08:09:05.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.160+0000 7fbdaff2eb80 10 client.22191 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:05.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.160+0000 7fbdaff2eb80 10 client.22191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:05.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.161+0000 7fbdaff2eb80 20 client.22191 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:05.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.161+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ad2111d290 0x55ad2111f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.161+0000 7fbdaff2eb80 4 client.22191 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:09:05.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.161+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1367384730 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad2102ad90 con 0x55ad2111d290 2024-05-27T08:09:05.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.161+0000 7fbd9a800700 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ad2111d290 0x55ad2111f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.163+0000 7fbd9a800700 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ad2111d290 0x55ad2111f6d0 crc :-1 s=READY pgs=634 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50877b9c9b6ceef server_cookie=e345a8b2d779d65b in_seq=0 out_seq=0 2024-05-27T08:09:05.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.163+0000 7fbd84c00700 10 client.22191 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:05.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.165+0000 7fbd84c00700 1 -- 172.21.15.63:0/1367384730 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ad2102ad90 con 0x55ad2111d290 2024-05-27T08:09:05.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.165+0000 7fbd84c00700 10 client.22191 handle_command_reply: tid=0 2024-05-27T08:09:05.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.165+0000 7fbdaff2eb80 1 client.22191 shutdown 2024-05-27T08:09:05.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1367384730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210fae70 msgr2=0x55ad2110d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:05.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210fae70 0x55ad2110d600 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fbd940042f0 tx=0x7fbd94004320 comp rx=0 tx=0).stop 2024-05-27T08:09:05.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1367384730 shutdown_connections 2024-05-27T08:09:05.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ad2111d290 0x55ad2111f6d0 crc :-1 s=READY pgs=634 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210fae70 0x55ad2110d600 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad21113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1367384730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210679f0 0x55ad21113740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1367384730 >> 172.21.15.63:0/1367384730 conn(0x55ad21015390 msgr2=0x55ad210fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:05.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbd84c00700 0 client.22191 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:05.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1367384730 shutdown_connections 2024-05-27T08:09:05.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1367384730 wait complete. 2024-05-27T08:09:05.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.166+0000 7fbdaff2eb80 20 client.22191 trim_cache size 0 max 16384 2024-05-27T08:09:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.197+0000 7fbdaff2eb80 1 Processor -- start 2024-05-27T08:09:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.197+0000 7fbdaff2eb80 1 -- start start 2024-05-27T08:09:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.197+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad20f7be40 0x55ad210fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.197+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21124e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.197+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad210fae70 0x55ad211253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.197+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1b0c50 con 0x55ad210679f0 2024-05-27T08:09:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.197+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1ae4e0 con 0x55ad20f7be40 2024-05-27T08:09:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.197+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1ac350 con 0x55ad210fae70 2024-05-27T08:09:05.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.198+0000 7fbd99e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21124e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.198+0000 7fbd99e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21124e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57390/0 (socket says 172.21.15.63:57390) 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.198+0000 7fbd99e00700 1 -- 172.21.15.63:0/1976039936 learned_addr learned my addr 172.21.15.63:0/1976039936 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.198+0000 7fbd9a800700 1 --2- 172.21.15.63:0/1976039936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad20f7be40 0x55ad210fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.198+0000 7fbd9b200700 1 --2- 172.21.15.63:0/1976039936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad210fae70 0x55ad211253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.198+0000 7fbd99e00700 1 -- 172.21.15.63:0/1976039936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad210fae70 msgr2=0x55ad211253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.198+0000 7fbd99e00700 1 --2- 172.21.15.63:0/1976039936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad210fae70 0x55ad211253d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.198+0000 7fbd99e00700 1 -- 172.21.15.63:0/1976039936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad20f7be40 msgr2=0x55ad210fc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.198+0000 7fbd99e00700 1 --2- 172.21.15.63:0/1976039936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad20f7be40 0x55ad210fc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.198+0000 7fbd99e00700 1 -- 172.21.15.63:0/1976039936 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad2110da20 con 0x55ad210679f0 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.199+0000 7fbd99e00700 1 --2- 172.21.15.63:0/1976039936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21124e90 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fbd880058b0 tx=0x7fbd880a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe819586e0fd2538 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.199+0000 7fbd87400700 1 -- 172.21.15.63:0/1976039936 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd880aa890 con 0x55ad210679f0 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.199+0000 7fbd87400700 1 -- 172.21.15.63:0/1976039936 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd880aec40 con 0x55ad210679f0 2024-05-27T08:09:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.199+0000 7fbd87400700 1 -- 172.21.15.63:0/1976039936 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88022920 con 0x55ad210679f0 2024-05-27T08:09:05.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.200+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1976039936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 msgr2=0x55ad21124e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:05.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.200+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1976039936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21124e90 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fbd880058b0 tx=0x7fbd880a6040 comp rx=0 tx=0).stop 2024-05-27T08:09:05.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.200+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1976039936 shutdown_connections 2024-05-27T08:09:05.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.200+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1976039936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad210fae70 0x55ad211253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.200+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1976039936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad20f7be40 0x55ad210fc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.200+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1976039936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21124e90 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.200+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1976039936 >> 172.21.15.63:0/1976039936 conn(0x55ad21015390 msgr2=0x55ad210fbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:05.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.201+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1976039936 shutdown_connections 2024-05-27T08:09:05.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.201+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1976039936 wait complete. 2024-05-27T08:09:05.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.202+0000 7fbdaff2eb80 1 -- start start 2024-05-27T08:09:05.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.202+0000 7fbdaff2eb80 1 Processor -- start 2024-05-27T08:09:05.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad211150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21115630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbdaff2eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210fae70 0x55ad21115b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1ac9a0 con 0x55ad210679f0 2024-05-27T08:09:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1ade10 con 0x55ad210fae70 2024-05-27T08:09:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbdaff2eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c1ad070 con 0x55ad20f7be40 2024-05-27T08:09:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbdaff2eb80 10 client.0 _ll_register_callbacks cb 0x55ad20fc2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbd99e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21115630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbd9a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad211150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbd99e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21115630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57394/0 (socket says 172.21.15.63:57394) 2024-05-27T08:09:05.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.203+0000 7fbd99e00700 1 -- 172.21.15.63:0/1478893928 learned_addr learned my addr 172.21.15.63:0/1478893928 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.204+0000 7fbd9b200700 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210fae70 0x55ad21115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.204+0000 7fbd99e00700 1 -- 172.21.15.63:0/1478893928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 msgr2=0x55ad211150f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.204+0000 7fbd99e00700 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad211150f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.204+0000 7fbd99e00700 1 -- 172.21.15.63:0/1478893928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210fae70 msgr2=0x55ad21115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.204+0000 7fbd99e00700 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210fae70 0x55ad21115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.204+0000 7fbd99e00700 1 -- 172.21.15.63:0/1478893928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad21186d60 con 0x55ad210679f0 2024-05-27T08:09:05.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.205+0000 7fbd99e00700 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21115630 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fbd880aa820 tx=0x7fbd88005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459b748e2b95a90b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:05.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.205+0000 7fbd84c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:05.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.205+0000 7fbd84c00700 1 -- 172.21.15.63:0/1478893928 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd880b3660 con 0x55ad210679f0 2024-05-27T08:09:05.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.205+0000 7fbd84c00700 1 -- 172.21.15.63:0/1478893928 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 9483+0+0 (secure 0 0 0) 0x7fbd88022920 con 0x55ad210679f0 2024-05-27T08:09:05.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.205+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1478893928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd880aec40 con 0x55ad210679f0 2024-05-27T08:09:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.206+0000 7fbd84c00700 1 -- 172.21.15.63:0/1478893928 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbd880e33b0 con 0x55ad210679f0 2024-05-27T08:09:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.206+0000 7fbdaff2eb80 10 client.22197 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.206+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1478893928 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ad21187fa0 con 0x55ad210679f0 2024-05-27T08:09:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.207+0000 7fbd84c00700 1 -- 172.21.15.63:0/1478893928 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd880aec40 con 0x55ad210679f0 2024-05-27T08:09:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.207+0000 7fbd84c00700 1 -- 172.21.15.63:0/1478893928 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbd880b3260 con 0x55ad210679f0 2024-05-27T08:09:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.207+0000 7fbdaff2eb80 10 client.22197 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.207+0000 7fbdaff2eb80 10 client.22197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.207+0000 7fbdaff2eb80 20 client.22197 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.207+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ad211885f0 0x55ad2118aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.207+0000 7fbdaff2eb80 4 client.22197 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:09:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.207+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1478893928 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad21024570 con 0x55ad211885f0 2024-05-27T08:09:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.208+0000 7fbd9a800700 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ad211885f0 0x55ad2118aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.209+0000 7fbd9a800700 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ad211885f0 0x55ad2118aa10 crc :-1 s=READY pgs=635 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=842bbc0995df8a5d server_cookie=baaa02fbf4ae9d81 in_seq=0 out_seq=0 2024-05-27T08:09:05.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.209+0000 7fbd84c00700 10 client.22197 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:05.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbd84c00700 1 -- 172.21.15.63:0/1478893928 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ad21024570 con 0x55ad211885f0 2024-05-27T08:09:05.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbd84c00700 10 client.22197 handle_command_reply: tid=0 2024-05-27T08:09:05.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 client.22197 shutdown 2024-05-27T08:09:05.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1478893928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 msgr2=0x55ad21115630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:05.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21115630 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fbd880aa820 tx=0x7fbd88005fb0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1478893928 shutdown_connections 2024-05-27T08:09:05.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ad211885f0 0x55ad2118aa10 crc :-1 s=READY pgs=635 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad210fae70 0x55ad21115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ad20f7be40 0x55ad211150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 --2- 172.21.15.63:0/1478893928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ad210679f0 0x55ad21115630 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1478893928 >> 172.21.15.63:0/1478893928 conn(0x55ad21015390 msgr2=0x55ad2110bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:05.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbd84c00700 0 client.22197 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:05.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1478893928 shutdown_connections 2024-05-27T08:09:05.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.210+0000 7fbdaff2eb80 1 -- 172.21.15.63:0/1478893928 wait complete. 2024-05-27T08:09:05.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.211+0000 7fbdaff2eb80 20 client.22197 trim_cache size 0 max 16384 2024-05-27T08:09:05.213 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:09:05.213 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:09:05.213 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:09:05.214 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:09:05.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 -- 172.21.15.63:0/3898079335 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd800b2a70 msgr2=0x7fbd800b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:05.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd800b2a70 0x7fbd800b4f80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009c60 tx=0x7fbd8c009260 comp rx=0 tx=0).stop 2024-05-27T08:09:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 -- 172.21.15.63:0/3898079335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd9c076420 msgr2=0x7fbd9c19d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd9c076420 0x7fbd9c19d580 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fbd880098e0 tx=0x7fbd880057c0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 -- 172.21.15.63:0/3898079335 shutdown_connections 2024-05-27T08:09:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbd800b2a70 0x7fbd800b4f80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd9c110c10 0x7fbd9c1a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c075ad0 0x7fbd9c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 --2- 172.21.15.63:0/3898079335 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd9c076420 0x7fbd9c19d580 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 -- 172.21.15.63:0/3898079335 >> 172.21.15.63:0/3898079335 conn(0x7fbd9c100030 msgr2=0x7fbd9c1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 -- 172.21.15.63:0/3898079335 shutdown_connections 2024-05-27T08:09:05.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:05.212+0000 7fbda1400700 1 -- 172.21.15.63:0/3898079335 wait complete. 2024-05-27T08:09:05.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:05 smithi063 ceph-mon[31613]: pgmap v1125: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.0 MiB/s wr, 27 op/s 2024-05-27T08:09:05.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:05 smithi086 ceph-mon[34687]: pgmap v1125: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.0 MiB/s wr, 27 op/s 2024-05-27T08:09:05.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:05 smithi149 ceph-mon[35408]: pgmap v1125: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.0 MiB/s wr, 27 op/s 2024-05-27T08:09:06.557 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:09:06.557 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:09:07.117 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:09:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:07 smithi086 ceph-mon[34687]: pgmap v1126: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-05-27T08:09:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:07 smithi086 ceph-mon[34687]: osdmap e151: 12 total, 12 up, 12 in 2024-05-27T08:09:07.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:07 smithi149 ceph-mon[35408]: pgmap v1126: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-05-27T08:09:07.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:07 smithi149 ceph-mon[35408]: osdmap e151: 12 total, 12 up, 12 in 2024-05-27T08:09:07.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:07 smithi063 ceph-mon[31613]: pgmap v1126: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-05-27T08:09:07.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:07 smithi063 ceph-mon[31613]: osdmap e151: 12 total, 12 up, 12 in 2024-05-27T08:09:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.351+0000 7f1b55e00640 1 -- 172.21.15.63:0/1070836399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b50102ab0 msgr2=0x7f1b50102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.351+0000 7f1b55e00640 1 --2- 172.21.15.63:0/1070836399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b50102ab0 0x7f1b50102ec0 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009900 tx=0x7f1b3c02a410 comp rx=0 tx=0).stop 2024-05-27T08:09:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.351+0000 7f1b55e00640 1 -- 172.21.15.63:0/1070836399 shutdown_connections 2024-05-27T08:09:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.351+0000 7f1b55e00640 1 --2- 172.21.15.63:0/1070836399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5010d340 0x7f1b5010f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.351+0000 7f1b55e00640 1 --2- 172.21.15.63:0/1070836399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b50102ab0 0x7f1b50102ec0 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.351+0000 7f1b55e00640 1 --2- 172.21.15.63:0/1070836399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b501089e0 0x7f1b50102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.351+0000 7f1b55e00640 1 -- 172.21.15.63:0/1070836399 >> 172.21.15.63:0/1070836399 conn(0x7f1b500fe210 msgr2=0x7f1b50100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.351+0000 7f1b55e00640 1 -- 172.21.15.63:0/1070836399 shutdown_connections 2024-05-27T08:09:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.351+0000 7f1b55e00640 1 -- 172.21.15.63:0/1070836399 wait complete. 2024-05-27T08:09:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.352+0000 7f1b55e00640 1 Processor -- start 2024-05-27T08:09:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.353+0000 7f1b55e00640 1 -- start start 2024-05-27T08:09:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.353+0000 7f1b55e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b50102ab0 0x7f1b5019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.353+0000 7f1b55e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b501089e0 0x7f1b5019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.353+0000 7f1b55e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5010d340 0x7f1b5019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.353+0000 7f1b55e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50112a50 con 0x7f1b50102ab0 2024-05-27T08:09:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.353+0000 7f1b55e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501128d0 con 0x7f1b5010d340 2024-05-27T08:09:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.353+0000 7f1b55e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50112bd0 con 0x7f1b501089e0 2024-05-27T08:09:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.353+0000 7f1b54a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b50102ab0 0x7f1b5019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.353+0000 7f1b54a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b50102ab0 0x7f1b5019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57412/0 (socket says 172.21.15.63:57412) 2024-05-27T08:09:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.353+0000 7f1b54a00640 1 -- 172.21.15.63:0/991482404 learned_addr learned my addr 172.21.15.63:0/991482404 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.354+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b501089e0 0x7f1b5019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.354+0000 7f1b55400640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5010d340 0x7f1b5019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.354+0000 7f1b54a00640 1 -- 172.21.15.63:0/991482404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b501089e0 msgr2=0x7f1b5019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.354+0000 7f1b54a00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b501089e0 0x7f1b5019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.354+0000 7f1b54a00640 1 -- 172.21.15.63:0/991482404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5010d340 msgr2=0x7f1b5019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.354+0000 7f1b54a00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5010d340 0x7f1b5019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.354+0000 7f1b54a00640 1 -- 172.21.15.63:0/991482404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5019f400 con 0x7f1b50102ab0 2024-05-27T08:09:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.354+0000 7f1b54a00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b50102ab0 0x7f1b5019a5a0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400d770 tx=0x7f1b4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.355+0000 7f1b4d600640 1 -- 172.21.15.63:0/991482404 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44017070 con 0x7f1b50102ab0 2024-05-27T08:09:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.355+0000 7f1b4d600640 1 -- 172.21.15.63:0/991482404 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b44009c30 con 0x7f1b50102ab0 2024-05-27T08:09:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.355+0000 7f1b55e00640 1 -- 172.21.15.63:0/991482404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5019f6f0 con 0x7f1b50102ab0 2024-05-27T08:09:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.355+0000 7f1b4d600640 1 -- 172.21.15.63:0/991482404 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400f620 con 0x7f1b50102ab0 2024-05-27T08:09:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.355+0000 7f1b55e00640 1 -- 172.21.15.63:0/991482404 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b501a7070 con 0x7f1b50102ab0 2024-05-27T08:09:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.358+0000 7f1b5e587740 1 Processor -- start 2024-05-27T08:09:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.358+0000 7f1b5e587740 1 -- start start 2024-05-27T08:09:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.359+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f5b1b0 0x5611872fd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.359+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f322f0 0x5611872fde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.359+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f3a4d0 0x5611872fe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.359+0000 7f1b5e587740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56118726eb90 con 0x561186f3a4d0 2024-05-27T08:09:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.359+0000 7f1b5e587740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56118725e730 con 0x561186f5b1b0 2024-05-27T08:09:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.359+0000 7f1b5e587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561187262770 con 0x561186f322f0 2024-05-27T08:09:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.359+0000 7f1b4fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f3a4d0 0x5611872fe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.359+0000 7f1b4fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f3a4d0 0x5611872fe390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57416/0 (socket says 172.21.15.63:57416) 2024-05-27T08:09:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.359+0000 7f1b4fe00640 1 -- 172.21.15.63:0/1641972418 learned_addr learned my addr 172.21.15.63:0/1641972418 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.359+0000 7f1b4d600640 1 -- 172.21.15.63:0/991482404 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1b4401a070 con 0x7f1b50102ab0 2024-05-27T08:09:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.360+0000 7f1b55400640 1 --2- 172.21.15.63:0/1641972418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f5b1b0 0x5611872fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.360+0000 7f1b4fe00640 1 -- 172.21.15.63:0/1641972418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f322f0 msgr2=0x5611872fde50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:09:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.360+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/1641972418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f322f0 0x5611872fde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.360+0000 7f1b4fe00640 1 -- 172.21.15.63:0/1641972418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f5b1b0 msgr2=0x5611872fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.360+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/1641972418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f5b1b0 0x5611872fd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.360+0000 7f1b4fe00640 1 -- 172.21.15.63:0/1641972418 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611871f78b0 con 0x561186f3a4d0 2024-05-27T08:09:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.360+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/1641972418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f3a4d0 0x5611872fe390 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c000c00 tx=0x7f1b3c032a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efbaae43d34c22c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.361+0000 7f1b41c00640 1 -- 172.21.15.63:0/1641972418 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c03f070 con 0x561186f3a4d0 2024-05-27T08:09:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.361+0000 7f1b41c00640 1 -- 172.21.15.63:0/1641972418 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b3c004030 con 0x561186f3a4d0 2024-05-27T08:09:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.361+0000 7f1b41c00640 1 -- 172.21.15.63:0/1641972418 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c035070 con 0x561186f3a4d0 2024-05-27T08:09:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.362+0000 7f1b5e587740 1 -- 172.21.15.63:0/1641972418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f3a4d0 msgr2=0x5611872fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.362+0000 7f1b5e587740 1 --2- 172.21.15.63:0/1641972418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f3a4d0 0x5611872fe390 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c000c00 tx=0x7f1b3c032a80 comp rx=0 tx=0).stop 2024-05-27T08:09:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.362+0000 7f1b5e587740 1 -- 172.21.15.63:0/1641972418 shutdown_connections 2024-05-27T08:09:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.362+0000 7f1b5e587740 1 --2- 172.21.15.63:0/1641972418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f3a4d0 0x5611872fe390 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.362+0000 7f1b5e587740 1 --2- 172.21.15.63:0/1641972418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f322f0 0x5611872fde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.362+0000 7f1b5e587740 1 --2- 172.21.15.63:0/1641972418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f5b1b0 0x5611872fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.362+0000 7f1b5e587740 1 -- 172.21.15.63:0/1641972418 >> 172.21.15.63:0/1641972418 conn(0x561186f4f300 msgr2=0x5611872fa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.362+0000 7f1b5e587740 1 -- 172.21.15.63:0/1641972418 shutdown_connections 2024-05-27T08:09:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.362+0000 7f1b5e587740 1 -- 172.21.15.63:0/1641972418 wait complete. 2024-05-27T08:09:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.362+0000 7f1b4d600640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b380b2860 0x7f1b380b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.363+0000 7f1b4d600640 1 -- 172.21.15.63:0/991482404 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 9571+0+0 (secure 0 0 0) 0x7f1b440e2650 con 0x7f1b50102ab0 2024-05-27T08:09:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.363+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b380b2860 0x7f1b380b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.363+0000 7f1b5e587740 1 -- start start 2024-05-27T08:09:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.363+0000 7f1b5e587740 1 Processor -- start 2024-05-27T08:09:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.363+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b380b2860 0x7f1b380b4d70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c005f70 tx=0x7f1b3c02a410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f3a4d0 0x56118730c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f5b1b0 0x561187318270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b5e587740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561186f44790 con 0x561186f322f0 2024-05-27T08:09:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b5e587740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561187206bc0 con 0x561186f3a4d0 2024-05-27T08:09:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b5e587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56118730ef40 con 0x561186f5b1b0 2024-05-27T08:09:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b5e587740 10 client.0 _ll_register_callbacks cb 0x5611871e4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b54a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b54a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57422/0 (socket says 172.21.15.63:57422) 2024-05-27T08:09:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b54a00640 1 -- 172.21.15.63:0/1441888845 learned_addr learned my addr 172.21.15.63:0/1441888845 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b55400640 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f5b1b0 0x561187318270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.364+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f3a4d0 0x56118730c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b54a00640 1 -- 172.21.15.63:0/1441888845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f5b1b0 msgr2=0x561187318270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:08.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b54a00640 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f5b1b0 0x561187318270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b54a00640 1 -- 172.21.15.63:0/1441888845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f3a4d0 msgr2=0x56118730c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b54a00640 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f3a4d0 0x56118730c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b54a00640 1 -- 172.21.15.63:0/1441888845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561186f73210 con 0x561186f322f0 2024-05-27T08:09:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f3a4d0 0x56118730c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:08.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b54a00640 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730c040 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f1b440a91b0 tx=0x7f1b44010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c6e9a01d0fc7b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b36000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b36000640 1 -- 172.21.15.63:0/1441888845 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44021050 con 0x561186f322f0 2024-05-27T08:09:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b36000640 1 -- 172.21.15.63:0/1441888845 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(151..151 src has 1..151) ==== 9571+0+0 (secure 0 0 0) 0x7f1b440b1e80 con 0x561186f322f0 2024-05-27T08:09:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.365+0000 7f1b5e587740 1 -- 172.21.15.63:0/1441888845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b3c004030 con 0x561186f322f0 2024-05-27T08:09:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b36000640 1 -- 172.21.15.63:0/1441888845 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1b440dfdb0 con 0x561186f322f0 2024-05-27T08:09:08.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b5e587740 10 client.22209 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b5e587740 1 -- 172.21.15.63:0/1441888845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56118703f620 con 0x561186f322f0 2024-05-27T08:09:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b36000640 1 -- 172.21.15.63:0/1441888845 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b3c004030 con 0x561186f322f0 2024-05-27T08:09:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b36000640 1 -- 172.21.15.63:0/1441888845 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1b440e2170 con 0x561186f322f0 2024-05-27T08:09:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b5e587740 10 client.22209 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b5e587740 10 client.22209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b5e587740 20 client.22209 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b5e587740 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561186f19d80 0x56118731e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b5e587740 4 client.22209 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:09:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.366+0000 7f1b5e587740 1 -- 172.21.15.63:0/1441888845 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561187270ae0 con 0x561186f19d80 2024-05-27T08:09:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.367+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561186f19d80 0x56118731e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.368+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561186f19d80 0x56118731e250 crc :-1 s=READY pgs=636 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be20a14f27fd1fb2 server_cookie=662be85f009b7982 in_seq=0 out_seq=0 2024-05-27T08:09:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.368+0000 7f1b36000640 10 client.22209 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b36000640 1 -- 172.21.15.63:0/1441888845 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561187270ae0 con 0x561186f19d80 2024-05-27T08:09:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b36000640 10 client.22209 handle_command_reply: tid=0 2024-05-27T08:09:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b5e587740 1 client.22209 shutdown 2024-05-27T08:09:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b5e587740 1 -- 172.21.15.63:0/1441888845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 msgr2=0x56118730c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b5e587740 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730c040 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f1b440a91b0 tx=0x7f1b44010f00 comp rx=0 tx=0).stop 2024-05-27T08:09:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b5e587740 1 -- 172.21.15.63:0/1441888845 shutdown_connections 2024-05-27T08:09:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b5e587740 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561186f19d80 0x56118731e250 crc :-1 s=READY pgs=636 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b5e587740 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f5b1b0 0x561187318270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b5e587740 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f3a4d0 0x56118730c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b5e587740 1 --2- 172.21.15.63:0/1441888845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730c040 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b5e587740 1 -- 172.21.15.63:0/1441888845 >> 172.21.15.63:0/1441888845 conn(0x561186f4f300 msgr2=0x5611872ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.370+0000 7f1b36000640 0 client.22209 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.371+0000 7f1b5e587740 1 -- 172.21.15.63:0/1441888845 shutdown_connections 2024-05-27T08:09:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.371+0000 7f1b5e587740 1 -- 172.21.15.63:0/1441888845 wait complete. 2024-05-27T08:09:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.371+0000 7f1b5e587740 20 client.22209 trim_cache size 0 max 16384 2024-05-27T08:09:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.392+0000 7f1b5e587740 1 Processor -- start 2024-05-27T08:09:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.392+0000 7f1b5e587740 1 -- start start 2024-05-27T08:09:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.392+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f322f0 0x5611872fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.392+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f3a4d0 0x561187305ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.392+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f5b1b0 0x5611873061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.392+0000 7f1b5e587740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501b1650 con 0x561186f5b1b0 2024-05-27T08:09:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.392+0000 7f1b5e587740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501aef20 con 0x561186f322f0 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.392+0000 7f1b5e587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501acd90 con 0x561186f3a4d0 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.393+0000 7f1b55400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f5b1b0 0x5611873061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.393+0000 7f1b55400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f5b1b0 0x5611873061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57436/0 (socket says 172.21.15.63:57436) 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.393+0000 7f1b55400640 1 -- 172.21.15.63:0/4212438422 learned_addr learned my addr 172.21.15.63:0/4212438422 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.393+0000 7f1b54a00640 1 --2- 172.21.15.63:0/4212438422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f322f0 0x5611872fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.393+0000 7f1b55400640 1 -- 172.21.15.63:0/4212438422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f3a4d0 msgr2=0x561187305ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.393+0000 7f1b55400640 1 --2- 172.21.15.63:0/4212438422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f3a4d0 0x561187305ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.393+0000 7f1b55400640 1 -- 172.21.15.63:0/4212438422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f322f0 msgr2=0x5611872fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.393+0000 7f1b55400640 1 --2- 172.21.15.63:0/4212438422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f322f0 0x5611872fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.393+0000 7f1b55400640 1 -- 172.21.15.63:0/4212438422 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611872f5230 con 0x561186f5b1b0 2024-05-27T08:09:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.393+0000 7f1b55400640 1 --2- 172.21.15.63:0/4212438422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f5b1b0 0x5611873061e0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f1b48013460 tx=0x7f1b48013920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadc80a99997a021 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.394+0000 7f1b41200640 1 -- 172.21.15.63:0/4212438422 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48011700 con 0x561186f5b1b0 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.394+0000 7f1b41200640 1 -- 172.21.15.63:0/4212438422 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b48011d30 con 0x561186f5b1b0 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.394+0000 7f1b41200640 1 -- 172.21.15.63:0/4212438422 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4801cce0 con 0x561186f5b1b0 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.395+0000 7f1b5e587740 1 -- 172.21.15.63:0/4212438422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f5b1b0 msgr2=0x5611873061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.395+0000 7f1b5e587740 1 --2- 172.21.15.63:0/4212438422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f5b1b0 0x5611873061e0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f1b48013460 tx=0x7f1b48013920 comp rx=0 tx=0).stop 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.395+0000 7f1b5e587740 1 -- 172.21.15.63:0/4212438422 shutdown_connections 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.395+0000 7f1b5e587740 1 --2- 172.21.15.63:0/4212438422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f5b1b0 0x5611873061e0 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.395+0000 7f1b5e587740 1 --2- 172.21.15.63:0/4212438422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f3a4d0 0x561187305ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.395+0000 7f1b5e587740 1 --2- 172.21.15.63:0/4212438422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f322f0 0x5611872fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.395+0000 7f1b5e587740 1 -- 172.21.15.63:0/4212438422 >> 172.21.15.63:0/4212438422 conn(0x561186f4f300 msgr2=0x56118730c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.395+0000 7f1b5e587740 1 -- 172.21.15.63:0/4212438422 shutdown_connections 2024-05-27T08:09:08.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.395+0000 7f1b5e587740 1 -- 172.21.15.63:0/4212438422 wait complete. 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.396+0000 7f1b5e587740 1 -- start start 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.396+0000 7f1b5e587740 1 Processor -- start 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.396+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f3a4d0 0x56118730fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b5e587740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f5b1b0 0x56118737e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b5e587740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501ad3e0 con 0x561186f322f0 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b5e587740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501ae850 con 0x561186f5b1b0 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b5e587740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501adab0 con 0x561186f3a4d0 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b5e587740 10 client.0 _ll_register_callbacks cb 0x7f1b44010ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b54a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b54a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57448/0 (socket says 172.21.15.63:57448) 2024-05-27T08:09:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b54a00640 1 -- 172.21.15.63:0/8391705 learned_addr learned my addr 172.21.15.63:0/8391705 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b54a00640 1 -- 172.21.15.63:0/8391705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f3a4d0 msgr2=0x56118730fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b54a00640 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f3a4d0 0x56118730fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b54a00640 1 -- 172.21.15.63:0/8391705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f5b1b0 msgr2=0x56118737e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b54a00640 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f5b1b0 0x56118737e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.397+0000 7f1b54a00640 1 -- 172.21.15.63:0/8391705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118737ee50 con 0x561186f322f0 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.398+0000 7f1b54a00640 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730f790 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f1b440e28f0 tx=0x7f1b440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6926514d00af9b5c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.398+0000 7f1b36000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.398+0000 7f1b36000640 1 -- 172.21.15.63:0/8391705 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44021070 con 0x561186f322f0 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.398+0000 7f1b36000640 1 -- 172.21.15.63:0/8391705 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(151..151 src has 1..151) ==== 9571+0+0 (secure 0 0 0) 0x7f1b440a50d0 con 0x561186f322f0 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.398+0000 7f1b5e587740 1 -- 172.21.15.63:0/8391705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b48011d30 con 0x561186f322f0 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.398+0000 7f1b36000640 1 -- 172.21.15.63:0/8391705 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1b440a8e00 con 0x561186f322f0 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.398+0000 7f1b5e587740 10 client.22215 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.398+0000 7f1b5e587740 1 -- 172.21.15.63:0/8391705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561187386190 con 0x561186f322f0 2024-05-27T08:09:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.398+0000 7f1b36000640 1 -- 172.21.15.63:0/8391705 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b48011d30 con 0x561186f322f0 2024-05-27T08:09:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.399+0000 7f1b36000640 1 -- 172.21.15.63:0/8391705 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1b440a8180 con 0x561186f322f0 2024-05-27T08:09:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.399+0000 7f1b5e587740 10 client.22215 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.399+0000 7f1b5e587740 10 client.22215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.399+0000 7f1b5e587740 20 client.22215 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.399+0000 7f1b5e587740 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561186eef1c0 0x5611873889a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.399+0000 7f1b5e587740 4 client.22215 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:09:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.399+0000 7f1b5e587740 1 -- 172.21.15.63:0/8391705 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56118726a2c0 con 0x561186eef1c0 2024-05-27T08:09:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.399+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561186eef1c0 0x5611873889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.400+0000 7f1b36000640 10 client.22215 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.400+0000 7f1b4fe00640 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561186eef1c0 0x5611873889a0 crc :-1 s=READY pgs=637 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4866a62974b4955c server_cookie=163f87acf0cb177b in_seq=0 out_seq=0 2024-05-27T08:09:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.437+0000 7f1b36000640 1 -- 172.21.15.63:0/8391705 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56118726a2c0 con 0x561186eef1c0 2024-05-27T08:09:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.437+0000 7f1b36000640 10 client.22215 handle_command_reply: tid=0 2024-05-27T08:09:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.437+0000 7f1b5e587740 1 client.22215 shutdown 2024-05-27T08:09:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 1 -- 172.21.15.63:0/8391705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 msgr2=0x56118730f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730f790 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f1b440e28f0 tx=0x7f1b440057f0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 1 -- 172.21.15.63:0/8391705 shutdown_connections 2024-05-27T08:09:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561186eef1c0 0x5611873889a0 crc :-1 s=READY pgs=637 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561186f5b1b0 0x56118737e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561186f3a4d0 0x56118730fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 1 --2- 172.21.15.63:0/8391705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561186f322f0 0x56118730f790 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b36000640 0 client.22215 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 1 -- 172.21.15.63:0/8391705 >> 172.21.15.63:0/8391705 conn(0x561186f4f300 msgr2=0x56118730c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 1 -- 172.21.15.63:0/8391705 shutdown_connections 2024-05-27T08:09:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 1 -- 172.21.15.63:0/8391705 wait complete. 2024-05-27T08:09:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.438+0000 7f1b5e587740 20 client.22215 trim_cache size 0 max 16384 2024-05-27T08:09:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:09:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:09:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:09:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 -- 172.21.15.63:0/991482404 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b380b2860 msgr2=0x7f1b380b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b380b2860 0x7f1b380b4d70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c005f70 tx=0x7f1b3c02a410 comp rx=0 tx=0).stop 2024-05-27T08:09:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 -- 172.21.15.63:0/991482404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b50102ab0 msgr2=0x7f1b5019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b50102ab0 0x7f1b5019a5a0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400d770 tx=0x7f1b4400dc30 comp rx=0 tx=0).stop 2024-05-27T08:09:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 -- 172.21.15.63:0/991482404 shutdown_connections 2024-05-27T08:09:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b380b2860 0x7f1b380b4d70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5010d340 0x7f1b5019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b501089e0 0x7f1b5019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 --2- 172.21.15.63:0/991482404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b50102ab0 0x7f1b5019a5a0 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 -- 172.21.15.63:0/991482404 >> 172.21.15.63:0/991482404 conn(0x7f1b500fe210 msgr2=0x7f1b5006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 -- 172.21.15.63:0/991482404 shutdown_connections 2024-05-27T08:09:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:08.439+0000 7f1b55e00640 1 -- 172.21.15.63:0/991482404 wait complete. 2024-05-27T08:09:09.956 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:09 smithi063 ceph-mon[31613]: pgmap v1128: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.3 MiB/s wr, 29 op/s 2024-05-27T08:09:09.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:09 smithi086 ceph-mon[34687]: pgmap v1128: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.3 MiB/s wr, 29 op/s 2024-05-27T08:09:09.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:09 smithi149 ceph-mon[35408]: pgmap v1128: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.3 MiB/s wr, 29 op/s 2024-05-27T08:09:10.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.684+0000 7f24e1e00700 1 -- 172.21.15.63:0/1822870997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24dc1032d0 msgr2=0x7f24dc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:10.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.684+0000 7f24e1e00700 1 --2- 172.21.15.63:0/1822870997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24dc1032d0 0x7f24dc1036e0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f24d00099c0 tx=0x7f24d0009cd0 comp rx=0 tx=0).stop 2024-05-27T08:09:10.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.685+0000 7f24e1e00700 1 -- 172.21.15.63:0/1822870997 shutdown_connections 2024-05-27T08:09:10.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.685+0000 7f24e1e00700 1 --2- 172.21.15.63:0/1822870997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24dc108550 0x7f24dc10feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:10.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.685+0000 7f24e1e00700 1 --2- 172.21.15.63:0/1822870997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24dc1032d0 0x7f24dc1036e0 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:10.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.685+0000 7f24e1e00700 1 --2- 172.21.15.63:0/1822870997 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24dc107c00 0x7f24dc108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:10.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.685+0000 7f24e1e00700 1 -- 172.21.15.63:0/1822870997 >> 172.21.15.63:0/1822870997 conn(0x7f24dc078ab0 msgr2=0x7f24dc078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:10.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.685+0000 7f24e1e00700 1 -- 172.21.15.63:0/1822870997 shutdown_connections 2024-05-27T08:09:10.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.685+0000 7f24e1e00700 1 -- 172.21.15.63:0/1822870997 wait complete. 2024-05-27T08:09:10.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.686+0000 7f24e1e00700 1 Processor -- start 2024-05-27T08:09:10.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.687+0000 7f24e1e00700 1 -- start start 2024-05-27T08:09:10.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.687+0000 7f24e1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24dc1032d0 0x7f24dc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:10.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.687+0000 7f24e1e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24dc107c00 0x7f24dc19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:10.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.687+0000 7f24e1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24dc108550 0x7f24dc1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:10.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.687+0000 7f24e1e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc113030 con 0x7f24dc108550 2024-05-27T08:09:10.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.687+0000 7f24e1e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc112eb0 con 0x7f24dc1032d0 2024-05-27T08:09:10.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.687+0000 7f24e1e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1131b0 con 0x7f24dc107c00 2024-05-27T08:09:10.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.688+0000 7f24e0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24dc1032d0 0x7f24dc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:10.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.688+0000 7f24e0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24dc1032d0 0x7f24dc19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:39078/0 (socket says 172.21.15.63:39078) 2024-05-27T08:09:10.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.688+0000 7f24e0a00700 1 -- 172.21.15.63:0/3691372345 learned_addr learned my addr 172.21.15.63:0/3691372345 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:10.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.688+0000 7f24e1400700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24dc108550 0x7f24dc1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:10.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.688+0000 7f24dbe00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24dc107c00 0x7f24dc19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:10.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.689+0000 7f24e0a00700 1 -- 172.21.15.63:0/3691372345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24dc107c00 msgr2=0x7f24dc19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:10.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.689+0000 7f24e0a00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24dc107c00 0x7f24dc19d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:10.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.689+0000 7f24e0a00700 1 -- 172.21.15.63:0/3691372345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24dc108550 msgr2=0x7f24dc1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:10.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.689+0000 7f24e0a00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24dc108550 0x7f24dc1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:10.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.689+0000 7f24e0a00700 1 -- 172.21.15.63:0/3691372345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc1a1c00 con 0x7f24dc1032d0 2024-05-27T08:09:10.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.689+0000 7f24e0a00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24dc1032d0 0x7f24dc19cd30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f24d0005f50 tx=0x7f24d00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:10.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.697+0000 7f24d9600700 1 -- 172.21.15.63:0/3691372345 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0027070 con 0x7f24dc1032d0 2024-05-27T08:09:10.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.697+0000 7f24d9600700 1 -- 172.21.15.63:0/3691372345 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24d000fd60 con 0x7f24dc1032d0 2024-05-27T08:09:10.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.697+0000 7f24d9600700 1 -- 172.21.15.63:0/3691372345 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0022e30 con 0x7f24dc1032d0 2024-05-27T08:09:10.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.697+0000 7f24e1e00700 1 -- 172.21.15.63:0/3691372345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc1a1e90 con 0x7f24dc1032d0 2024-05-27T08:09:10.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.697+0000 7f24e1e00700 1 -- 172.21.15.63:0/3691372345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc1a2320 con 0x7f24dc1032d0 2024-05-27T08:09:10.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.699+0000 7f24e1e00700 1 -- 172.21.15.63:0/3691372345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc0512b0 con 0x7f24dc1032d0 2024-05-27T08:09:10.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.703+0000 7f24d9600700 1 -- 172.21.15.63:0/3691372345 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f24d0022600 con 0x7f24dc1032d0 2024-05-27T08:09:10.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.705+0000 7f24d9600700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24c40b2a70 0x7f24c40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:10.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.705+0000 7f24d9600700 1 -- 172.21.15.63:0/3691372345 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 9571+0+0 (secure 0 0 0) 0x7f24d00e3f10 con 0x7f24dc1032d0 2024-05-27T08:09:10.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.705+0000 7f24dbe00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24c40b2a70 0x7f24c40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:10.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.706+0000 7f24dbe00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24c40b2a70 0x7f24c40b4f80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005950 tx=0x7f24c80058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:10.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:10.719+0000 7f24d9600700 1 -- 172.21.15.63:0/3691372345 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f24d0013070 con 0x7f24dc1032d0 2024-05-27T08:09:11.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.052+0000 7f24e1e00700 1 -- 172.21.15.63:0/3691372345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24dc1a98d0 con 0x7f24dc1032d0 2024-05-27T08:09:11.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.054+0000 7f24d9600700 1 -- 172.21.15.63:0/3691372345 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7f24d00ab500 con 0x7f24dc1032d0 2024-05-27T08:09:11.056 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:09:11.057 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:09:11.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 -- 172.21.15.63:0/3691372345 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24c40b2a70 msgr2=0x7f24c40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:11.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24c40b2a70 0x7f24c40b4f80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005950 tx=0x7f24c80058e0 comp rx=0 tx=0).stop 2024-05-27T08:09:11.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 -- 172.21.15.63:0/3691372345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24dc1032d0 msgr2=0x7f24dc19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:11.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24dc1032d0 0x7f24dc19cd30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f24d0005f50 tx=0x7f24d00057c0 comp rx=0 tx=0).stop 2024-05-27T08:09:11.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 -- 172.21.15.63:0/3691372345 shutdown_connections 2024-05-27T08:09:11.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24c40b2a70 0x7f24c40b4f80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:11.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24dc108550 0x7f24dc1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:11.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24dc1032d0 0x7f24dc19cd30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:11.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 --2- 172.21.15.63:0/3691372345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24dc107c00 0x7f24dc19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:11.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 -- 172.21.15.63:0/3691372345 >> 172.21.15.63:0/3691372345 conn(0x7f24dc078ab0 msgr2=0x7f24dc06b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:11.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 -- 172.21.15.63:0/3691372345 shutdown_connections 2024-05-27T08:09:11.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:11.058+0000 7f24e1e00700 1 -- 172.21.15.63:0/3691372345 wait complete. 2024-05-27T08:09:11.061 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:09:11.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:11 smithi086 ceph-mon[34687]: pgmap v1129: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.3 MiB/s wr, 29 op/s 2024-05-27T08:09:11.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:11 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3691372345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:09:11.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:11 smithi149 ceph-mon[35408]: pgmap v1129: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.3 MiB/s wr, 29 op/s 2024-05-27T08:09:11.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:11 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3691372345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:09:11.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:11 smithi063 ceph-mon[31613]: pgmap v1129: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.3 MiB/s wr, 29 op/s 2024-05-27T08:09:11.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:11 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3691372345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:09:12.086 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:09:12.629 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:09:13.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:13 smithi086 ceph-mon[34687]: pgmap v1130: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T08:09:13.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:13 smithi149 ceph-mon[35408]: pgmap v1130: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T08:09:13.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:13 smithi063 ceph-mon[31613]: pgmap v1130: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T08:09:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.415+0000 7fc040800700 1 -- 172.21.15.63:0/825004237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc038105de0 msgr2=0x7fc03810e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.415+0000 7fc040800700 1 --2- 172.21.15.63:0/825004237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc038105de0 0x7fc03810e330 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fc028009990 tx=0x7fc028009ca0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.416+0000 7fc040800700 1 -- 172.21.15.63:0/825004237 shutdown_connections 2024-05-27T08:09:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.416+0000 7fc040800700 1 --2- 172.21.15.63:0/825004237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc03810e900 0x7fc038110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.416+0000 7fc040800700 1 --2- 172.21.15.63:0/825004237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc038105490 0x7fc0381058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.416+0000 7fc040800700 1 --2- 172.21.15.63:0/825004237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc038105de0 0x7fc03810e330 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.416+0000 7fc040800700 1 -- 172.21.15.63:0/825004237 >> 172.21.15.63:0/825004237 conn(0x7fc0380fedb0 msgr2=0x7fc0381011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.416+0000 7fc040800700 1 -- 172.21.15.63:0/825004237 shutdown_connections 2024-05-27T08:09:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.416+0000 7fc040800700 1 -- 172.21.15.63:0/825004237 wait complete. 2024-05-27T08:09:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.418+0000 7fc040800700 1 Processor -- start 2024-05-27T08:09:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.418+0000 7fc040800700 1 -- start start 2024-05-27T08:09:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.419+0000 7fc040800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc038105490 0x7fc03819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.419+0000 7fc040800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc038105de0 0x7fc03819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.419+0000 7fc040800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc03810e900 0x7fc0381a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.419+0000 7fc040800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc038114340 con 0x7fc038105490 2024-05-27T08:09:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.419+0000 7fc040800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0381141c0 con 0x7fc03810e900 2024-05-27T08:09:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.419+0000 7fc040800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc0381144c0 con 0x7fc038105de0 2024-05-27T08:09:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.419+0000 7fc03de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc038105490 0x7fc03819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.419+0000 7fc03de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc038105490 0x7fc03819cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38968/0 (socket says 172.21.15.63:38968) 2024-05-27T08:09:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.419+0000 7fc03de00700 1 -- 172.21.15.63:0/1716788432 learned_addr learned my addr 172.21.15.63:0/1716788432 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:15.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.419+0000 7fc03e800700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc03810e900 0x7fc0381a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.420+0000 7fc03d400700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc038105de0 0x7fc03819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.420+0000 7fc03de00700 1 -- 172.21.15.63:0/1716788432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc038105de0 msgr2=0x7fc03819d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:15.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.420+0000 7fc03de00700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc038105de0 0x7fc03819d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.420+0000 7fc03de00700 1 -- 172.21.15.63:0/1716788432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc03810e900 msgr2=0x7fc0381a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.420+0000 7fc03de00700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc03810e900 0x7fc0381a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.420+0000 7fc03de00700 1 -- 172.21.15.63:0/1716788432 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0381a1e80 con 0x7fc038105490 2024-05-27T08:09:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.421+0000 7fc03de00700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc038105490 0x7fc03819cfb0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00e9a0 tx=0x7fc02c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.421+0000 7fc032a00700 1 -- 172.21.15.63:0/1716788432 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c00ca10 con 0x7fc038105490 2024-05-27T08:09:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.421+0000 7fc032a00700 1 -- 172.21.15.63:0/1716788432 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc02c00cbb0 con 0x7fc038105490 2024-05-27T08:09:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.421+0000 7fc040800700 1 -- 172.21.15.63:0/1716788432 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0381a2170 con 0x7fc038105490 2024-05-27T08:09:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.421+0000 7fc032a00700 1 -- 172.21.15.63:0/1716788432 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c018680 con 0x7fc038105490 2024-05-27T08:09:15.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.421+0000 7fc040800700 1 -- 172.21.15.63:0/1716788432 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0381a9a90 con 0x7fc038105490 2024-05-27T08:09:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.426+0000 7fc032a00700 1 -- 172.21.15.63:0/1716788432 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc02c010430 con 0x7fc038105490 2024-05-27T08:09:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.427+0000 7fc04f37ab80 1 Processor -- start 2024-05-27T08:09:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.427+0000 7fc04f37ab80 1 -- start start 2024-05-27T08:09:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.427+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfc379f0 0x5568dfcca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.427+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.427+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfcd7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.427+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5568dfbf8e40 con 0x5568dfb4be40 2024-05-27T08:09:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.427+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568dfbe89e0 con 0x5568dfc379f0 2024-05-27T08:09:15.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.427+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5568dfbeca20 con 0x5568dfccae70 2024-05-27T08:09:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.428+0000 7fc03de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.428+0000 7fc03de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38984/0 (socket says 172.21.15.63:38984) 2024-05-27T08:09:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.428+0000 7fc03de00700 1 -- 172.21.15.63:0/581297 learned_addr learned my addr 172.21.15.63:0/581297 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.428+0000 7fc03e800700 1 --2- 172.21.15.63:0/581297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfc379f0 0x5568dfcca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.428+0000 7fc03d400700 1 --2- 172.21.15.63:0/581297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfcd7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.429+0000 7fc032a00700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0200b29f0 0x7fc0200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.429+0000 7fc03de00700 1 -- 172.21.15.63:0/581297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 msgr2=0x5568dfcd7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.429+0000 7fc03de00700 1 --2- 172.21.15.63:0/581297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfcd7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.429+0000 7fc03de00700 1 -- 172.21.15.63:0/581297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfc379f0 msgr2=0x5568dfcca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.429+0000 7fc03de00700 1 --2- 172.21.15.63:0/581297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfc379f0 0x5568dfcca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.429+0000 7fc03de00700 1 -- 172.21.15.63:0/581297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568dfbb1420 con 0x5568dfb4be40 2024-05-27T08:09:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.429+0000 7fc032a00700 1 -- 172.21.15.63:0/1716788432 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 9571+0+0 (secure 0 0 0) 0x7fc02c0e2c40 con 0x7fc038105490 2024-05-27T08:09:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.429+0000 7fc03e800700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0200b29f0 0x7fc0200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.429+0000 7fc03de00700 1 --2- 172.21.15.63:0/581297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcca870 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0dfe50 tx=0x7fc02c024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfe0d0b6c1f77e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.430+0000 7fc027400700 1 -- 172.21.15.63:0/581297 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0a9870 con 0x5568dfb4be40 2024-05-27T08:09:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.430+0000 7fc027400700 1 -- 172.21.15.63:0/581297 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc02c0a95b0 con 0x5568dfb4be40 2024-05-27T08:09:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.430+0000 7fc027400700 1 -- 172.21.15.63:0/581297 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0213f0 con 0x5568dfb4be40 2024-05-27T08:09:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.431+0000 7fc04f37ab80 1 -- 172.21.15.63:0/581297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 msgr2=0x5568dfcca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.431+0000 7fc03e800700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0200b29f0 0x7fc0200b4f00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc034009790 tx=0x7fc034004ad0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.431+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/581297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcca870 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0dfe50 tx=0x7fc02c024000 comp rx=0 tx=0).stop 2024-05-27T08:09:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.431+0000 7fc04f37ab80 1 -- 172.21.15.63:0/581297 shutdown_connections 2024-05-27T08:09:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.431+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/581297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfcd7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.431+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/581297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfc379f0 0x5568dfcca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.431+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/581297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcca870 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.431+0000 7fc04f37ab80 1 -- 172.21.15.63:0/581297 >> 172.21.15.63:0/581297 conn(0x5568dfbe5390 msgr2=0x5568dfcc64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.431+0000 7fc04f37ab80 1 -- 172.21.15.63:0/581297 shutdown_connections 2024-05-27T08:09:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.431+0000 7fc04f37ab80 1 -- 172.21.15.63:0/581297 wait complete. 2024-05-27T08:09:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.432+0000 7fc04f37ab80 1 -- start start 2024-05-27T08:09:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.432+0000 7fc04f37ab80 1 Processor -- start 2024-05-27T08:09:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.433+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfb4be40 0x5568dfcd4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.433+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfcd4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.433+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfcd5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.433+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5568dfaeefa0 con 0x5568dfc379f0 2024-05-27T08:09:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.433+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568dfb89760 con 0x5568dfb4be40 2024-05-27T08:09:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.433+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5568dfcd7b20 con 0x5568dfccae70 2024-05-27T08:09:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.433+0000 7fc04f37ab80 10 client.0 _ll_register_callbacks cb 0x5568dfb92c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.434+0000 7fc03d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfcd4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.434+0000 7fc03d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfcd4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38988/0 (socket says 172.21.15.63:38988) 2024-05-27T08:09:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.434+0000 7fc03d400700 1 -- 172.21.15.63:0/610114746 learned_addr learned my addr 172.21.15.63:0/610114746 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:15.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.434+0000 7fc03de00700 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfb4be40 0x5568dfcd4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.434+0000 7fc03e800700 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfcd5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.434+0000 7fc03d400700 1 -- 172.21.15.63:0/610114746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 msgr2=0x5568dfcd5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:15.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.434+0000 7fc03d400700 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfcd5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.434+0000 7fc03d400700 1 -- 172.21.15.63:0/610114746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfb4be40 msgr2=0x5568dfcd4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:15.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.434+0000 7fc03d400700 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfb4be40 0x5568dfcd4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.434+0000 7fc03d400700 1 -- 172.21.15.63:0/610114746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568dfb388d0 con 0x5568dfc379f0 2024-05-27T08:09:15.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.435+0000 7fc03d400700 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfcd4c40 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fc028009990 tx=0x7fc0280057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e3c6af7685a493 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:15.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.435+0000 7fc01be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:15.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.435+0000 7fc01be00700 1 -- 172.21.15.63:0/610114746 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028028060 con 0x5568dfc379f0 2024-05-27T08:09:15.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.435+0000 7fc04f37ab80 1 -- 172.21.15.63:0/610114746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc02c0a95b0 con 0x5568dfc379f0 2024-05-27T08:09:15.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.435+0000 7fc01be00700 1 -- 172.21.15.63:0/610114746 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(151..151 src has 1..151) ==== 9571+0+0 (secure 0 0 0) 0x7fc02800b6b0 con 0x5568dfc379f0 2024-05-27T08:09:15.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.436+0000 7fc01be00700 1 -- 172.21.15.63:0/610114746 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc02802f050 con 0x5568dfc379f0 2024-05-27T08:09:15.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.436+0000 7fc04f37ab80 10 client.22233 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:15.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.436+0000 7fc04f37ab80 1 -- 172.21.15.63:0/610114746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5568dfbb9060 con 0x5568dfc379f0 2024-05-27T08:09:15.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.436+0000 7fc01be00700 1 -- 172.21.15.63:0/610114746 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc02c0a95b0 con 0x5568dfc379f0 2024-05-27T08:09:15.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.437+0000 7fc01be00700 1 -- 172.21.15.63:0/610114746 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc02800f7d0 con 0x5568dfc379f0 2024-05-27T08:09:15.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.437+0000 7fc04f37ab80 10 client.22233 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:15.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.437+0000 7fc04f37ab80 10 client.22233 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:09:15.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.437+0000 7fc04f37ab80 20 client.22233 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:15.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.437+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5568dfce4990 0x5568dfce6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.437+0000 7fc04f37ab80 4 client.22233 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:09:15.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.437+0000 7fc04f37ab80 1 -- 172.21.15.63:0/610114746 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568dfbfad90 con 0x5568dfce4990 2024-05-27T08:09:15.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.438+0000 7fc03de00700 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5568dfce4990 0x5568dfce6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.439+0000 7fc03de00700 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5568dfce4990 0x5568dfce6db0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f90269a836df2d7 server_cookie=3a5489631ebe6065 in_seq=0 out_seq=0 2024-05-27T08:09:15.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.439+0000 7fc01be00700 10 client.22233 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:09:15.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc01be00700 1 -- 172.21.15.63:0/610114746 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5568dfbfad90 con 0x5568dfce4990 2024-05-27T08:09:15.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc01be00700 10 client.22233 handle_command_reply: tid=0 2024-05-27T08:09:15.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc04f37ab80 1 client.22233 shutdown 2024-05-27T08:09:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc04f37ab80 1 -- 172.21.15.63:0/610114746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 msgr2=0x5568dfcd4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfcd4c40 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fc028009990 tx=0x7fc0280057e0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc04f37ab80 1 -- 172.21.15.63:0/610114746 shutdown_connections 2024-05-27T08:09:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfcd5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfb4be40 0x5568dfcd4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5568dfce4990 0x5568dfce6db0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/610114746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfcd4c40 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc04f37ab80 1 -- 172.21.15.63:0/610114746 >> 172.21.15.63:0/610114746 conn(0x5568dfbe5390 msgr2=0x5568dfcca310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.441+0000 7fc01be00700 0 client.22233 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:09:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.442+0000 7fc04f37ab80 1 -- 172.21.15.63:0/610114746 shutdown_connections 2024-05-27T08:09:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.442+0000 7fc04f37ab80 1 -- 172.21.15.63:0/610114746 wait complete. 2024-05-27T08:09:15.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.442+0000 7fc04f37ab80 20 client.22233 trim_cache size 0 max 16384 2024-05-27T08:09:15.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.459+0000 7fc04f37ab80 1 Processor -- start 2024-05-27T08:09:15.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.459+0000 7fc04f37ab80 1 -- start start 2024-05-27T08:09:15.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.459+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfb4be40 0x5568dfcd3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.460+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfccfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.460+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccae70 0x5568dfce0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.460+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0381b51d0 con 0x5568dfc379f0 2024-05-27T08:09:15.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.460+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0381b2a60 con 0x5568dfccae70 2024-05-27T08:09:15.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.460+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc0381b08d0 con 0x5568dfb4be40 2024-05-27T08:09:15.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.460+0000 7fc03d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfccfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.460+0000 7fc03d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfccfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39000/0 (socket says 172.21.15.63:39000) 2024-05-27T08:09:15.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.460+0000 7fc03d400700 1 -- 172.21.15.63:0/425373747 learned_addr learned my addr 172.21.15.63:0/425373747 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:15.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.460+0000 7fc03de00700 1 --2- 172.21.15.63:0/425373747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfb4be40 0x5568dfcd3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.460+0000 7fc03e800700 1 --2- 172.21.15.63:0/425373747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccae70 0x5568dfce0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.461+0000 7fc03d400700 1 -- 172.21.15.63:0/425373747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfb4be40 msgr2=0x5568dfcd3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:15.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.461+0000 7fc03d400700 1 --2- 172.21.15.63:0/425373747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfb4be40 0x5568dfcd3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.461+0000 7fc03d400700 1 -- 172.21.15.63:0/425373747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccae70 msgr2=0x5568dfce0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:15.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.461+0000 7fc03d400700 1 --2- 172.21.15.63:0/425373747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccae70 0x5568dfce0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.461+0000 7fc03d400700 1 -- 172.21.15.63:0/425373747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568dfccb800 con 0x5568dfc379f0 2024-05-27T08:09:15.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.461+0000 7fc03d400700 1 --2- 172.21.15.63:0/425373747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfccfef0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b650 tx=0x7fc02800fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd374372797fe5e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:15.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.462+0000 7fc026a00700 1 -- 172.21.15.63:0/425373747 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028028070 con 0x5568dfc379f0 2024-05-27T08:09:15.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.462+0000 7fc026a00700 1 -- 172.21.15.63:0/425373747 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc028037020 con 0x5568dfc379f0 2024-05-27T08:09:15.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.462+0000 7fc026a00700 1 -- 172.21.15.63:0/425373747 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02800bc80 con 0x5568dfc379f0 2024-05-27T08:09:15.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.463+0000 7fc04f37ab80 1 -- 172.21.15.63:0/425373747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 msgr2=0x5568dfccfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:15.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.463+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/425373747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfccfef0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b650 tx=0x7fc02800fef0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.463+0000 7fc04f37ab80 1 -- 172.21.15.63:0/425373747 shutdown_connections 2024-05-27T08:09:15.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.463+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/425373747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccae70 0x5568dfce0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.463+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/425373747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfb4be40 0x5568dfcd3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.463+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/425373747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfc379f0 0x5568dfccfef0 secure :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b650 tx=0x7fc02800fef0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.463+0000 7fc04f37ab80 1 -- 172.21.15.63:0/425373747 >> 172.21.15.63:0/425373747 conn(0x5568dfbe5390 msgr2=0x5568dfcd46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:15.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.463+0000 7fc04f37ab80 1 -- 172.21.15.63:0/425373747 shutdown_connections 2024-05-27T08:09:15.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.464+0000 7fc04f37ab80 1 -- 172.21.15.63:0/425373747 wait complete. 2024-05-27T08:09:15.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.464+0000 7fc04f37ab80 1 -- start start 2024-05-27T08:09:15.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.464+0000 7fc04f37ab80 1 Processor -- start 2024-05-27T08:09:15.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.465+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcdd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.465+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfccebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.465+0000 7fc04f37ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccf110 0x5568dfccf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.465+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0381b0f20 con 0x5568dfb4be40 2024-05-27T08:09:15.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.465+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0381b2390 con 0x5568dfccf110 2024-05-27T08:09:15.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.465+0000 7fc04f37ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc0381b15f0 con 0x5568dfccae70 2024-05-27T08:09:15.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.465+0000 7fc04f37ab80 10 client.0 _ll_register_callbacks cb 0x5568dfb92c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:15.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.465+0000 7fc03de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcdd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.465+0000 7fc03de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcdd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39016/0 (socket says 172.21.15.63:39016) 2024-05-27T08:09:15.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.465+0000 7fc03de00700 1 -- 172.21.15.63:0/1334900788 learned_addr learned my addr 172.21.15.63:0/1334900788 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:15.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.466+0000 7fc03d400700 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfccebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.466+0000 7fc03e800700 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccf110 0x5568dfccf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.466+0000 7fc03de00700 1 -- 172.21.15.63:0/1334900788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 msgr2=0x5568dfccebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:15.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.466+0000 7fc03de00700 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfccebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.466+0000 7fc03de00700 1 -- 172.21.15.63:0/1334900788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccf110 msgr2=0x5568dfccf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:15.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.466+0000 7fc03de00700 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccf110 0x5568dfccf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.466+0000 7fc03de00700 1 -- 172.21.15.63:0/1334900788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568dfcdd9a0 con 0x5568dfb4be40 2024-05-27T08:09:15.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.466+0000 7fc03e800700 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccf110 0x5568dfccf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:09:15.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.467+0000 7fc03de00700 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcdd180 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7fc02c000c00 tx=0x7fc02c024fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa0bce0f31f1f99 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:15.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.467+0000 7fc01be00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:15.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.467+0000 7fc01be00700 1 -- 172.21.15.63:0/1334900788 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c08e070 con 0x5568dfb4be40 2024-05-27T08:09:15.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.467+0000 7fc01be00700 1 -- 172.21.15.63:0/1334900788 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(151..151 src has 1..151) ==== 9571+0+0 (secure 0 0 0) 0x7fc02c00cd60 con 0x5568dfb4be40 2024-05-27T08:09:15.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.467+0000 7fc04f37ab80 1 -- 172.21.15.63:0/1334900788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc028037020 con 0x5568dfb4be40 2024-05-27T08:09:15.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.467+0000 7fc01be00700 1 -- 172.21.15.63:0/1334900788 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc02c011480 con 0x5568dfb4be40 2024-05-27T08:09:15.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.467+0000 7fc04f37ab80 10 client.22239 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:15.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.467+0000 7fc04f37ab80 1 -- 172.21.15.63:0/1334900788 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5568dfd4f490 con 0x5568dfb4be40 2024-05-27T08:09:15.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.468+0000 7fc01be00700 1 -- 172.21.15.63:0/1334900788 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc028037020 con 0x5568dfb4be40 2024-05-27T08:09:15.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.468+0000 7fc01be00700 1 -- 172.21.15.63:0/1334900788 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc02c01ca00 con 0x5568dfb4be40 2024-05-27T08:09:15.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.468+0000 7fc04f37ab80 10 client.22239 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:15.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.468+0000 7fc04f37ab80 10 client.22239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:09:15.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.468+0000 7fc04f37ab80 20 client.22239 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:15.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.468+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5568dfd4f720 0x5568dfd51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:15.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.468+0000 7fc04f37ab80 4 client.22239 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:09:15.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.468+0000 7fc04f37ab80 1 -- 172.21.15.63:0/1334900788 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568dfbf4570 con 0x5568dfd4f720 2024-05-27T08:09:15.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.469+0000 7fc03d400700 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5568dfd4f720 0x5568dfd51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:15.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.470+0000 7fc01be00700 10 client.22239 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:09:15.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.470+0000 7fc03d400700 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5568dfd4f720 0x5568dfd51b40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96d4d6528334919b server_cookie=2a74ad81fa845c6d in_seq=0 out_seq=0 2024-05-27T08:09:15.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.471+0000 7fc01be00700 1 -- 172.21.15.63:0/1334900788 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568dfbf4570 con 0x5568dfd4f720 2024-05-27T08:09:15.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.471+0000 7fc01be00700 10 client.22239 handle_command_reply: tid=0 2024-05-27T08:09:15.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.471+0000 7fc04f37ab80 1 client.22239 shutdown 2024-05-27T08:09:15.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.471+0000 7fc04f37ab80 1 -- 172.21.15.63:0/1334900788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 msgr2=0x5568dfcdd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:15.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.471+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcdd180 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7fc02c000c00 tx=0x7fc02c024fb0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.472+0000 7fc04f37ab80 1 -- 172.21.15.63:0/1334900788 shutdown_connections 2024-05-27T08:09:15.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.472+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568dfccf110 0x5568dfccf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.472+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5568dfd4f720 0x5568dfd51b40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.472+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5568dfb4be40 0x5568dfcdd180 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.472+0000 7fc04f37ab80 1 --2- 172.21.15.63:0/1334900788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568dfccae70 0x5568dfccebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.472+0000 7fc04f37ab80 1 -- 172.21.15.63:0/1334900788 >> 172.21.15.63:0/1334900788 conn(0x5568dfbe5390 msgr2=0x5568dfccbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:15.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.472+0000 7fc01be00700 0 client.22239 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:09:15.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.472+0000 7fc04f37ab80 1 -- 172.21.15.63:0/1334900788 shutdown_connections 2024-05-27T08:09:15.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.472+0000 7fc04f37ab80 1 -- 172.21.15.63:0/1334900788 wait complete. 2024-05-27T08:09:15.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.472+0000 7fc04f37ab80 20 client.22239 trim_cache size 0 max 16384 2024-05-27T08:09:15.475 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:09:15.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.473+0000 7fc040800700 1 -- 172.21.15.63:0/1716788432 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0200b29f0 msgr2=0x7fc0200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:15.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.473+0000 7fc040800700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0200b29f0 0x7fc0200b4f00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc034009790 tx=0x7fc034004ad0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.473+0000 7fc040800700 1 -- 172.21.15.63:0/1716788432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc038105490 msgr2=0x7fc03819cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:15.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.473+0000 7fc040800700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc038105490 0x7fc03819cfb0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00e9a0 tx=0x7fc02c00ecb0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.473+0000 7fc040800700 1 -- 172.21.15.63:0/1716788432 shutdown_connections 2024-05-27T08:09:15.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.473+0000 7fc040800700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0200b29f0 0x7fc0200b4f00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.473+0000 7fc040800700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc03810e900 0x7fc0381a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.473+0000 7fc040800700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc038105490 0x7fc03819cfb0 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.473+0000 7fc040800700 1 --2- 172.21.15.63:0/1716788432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc038105de0 0x7fc03819d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:15.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.473+0000 7fc040800700 1 -- 172.21.15.63:0/1716788432 >> 172.21.15.63:0/1716788432 conn(0x7fc0380fedb0 msgr2=0x7fc038103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:15.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.474+0000 7fc040800700 1 -- 172.21.15.63:0/1716788432 shutdown_connections 2024-05-27T08:09:15.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:15.474+0000 7fc040800700 1 -- 172.21.15.63:0/1716788432 wait complete. 2024-05-27T08:09:15.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:15 smithi086 ceph-mon[34687]: pgmap v1131: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T08:09:15.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:15 smithi149 ceph-mon[35408]: pgmap v1131: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T08:09:15.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:15 smithi063 ceph-mon[31613]: pgmap v1131: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 12 op/s 2024-05-27T08:09:16.663 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:09:17.193 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:09:17.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:17 smithi086 ceph-mon[34687]: osdmap e152: 12 total, 12 up, 12 in 2024-05-27T08:09:17.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:17 smithi086 ceph-mon[34687]: pgmap v1133: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.5 MiB/s wr, 13 op/s 2024-05-27T08:09:17.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:17 smithi149 ceph-mon[35408]: osdmap e152: 12 total, 12 up, 12 in 2024-05-27T08:09:17.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:17 smithi149 ceph-mon[35408]: pgmap v1133: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.5 MiB/s wr, 13 op/s 2024-05-27T08:09:17.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:17 smithi063 ceph-mon[31613]: osdmap e152: 12 total, 12 up, 12 in 2024-05-27T08:09:17.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:17 smithi063 ceph-mon[31613]: pgmap v1133: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.5 MiB/s wr, 13 op/s 2024-05-27T08:09:19.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:19 smithi063 ceph-mon[31613]: pgmap v1134: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:09:19.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.873+0000 7f2e16800700 1 -- 172.21.15.63:0/3721589228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e1010c5a0 msgr2=0x7f2e1010c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:19.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.873+0000 7f2e16800700 1 --2- 172.21.15.63:0/3721589228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e1010c5a0 0x7f2e1010c9b0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009990 tx=0x7f2e00009ca0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.873+0000 7f2e16800700 1 -- 172.21.15.63:0/3721589228 shutdown_connections 2024-05-27T08:09:19.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.873+0000 7f2e16800700 1 --2- 172.21.15.63:0/3721589228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e10106eb0 0x7f2e10113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.873+0000 7f2e16800700 1 --2- 172.21.15.63:0/3721589228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e1010c5a0 0x7f2e1010c9b0 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.873+0000 7f2e16800700 1 --2- 172.21.15.63:0/3721589228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e10106560 0x7f2e10106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.873+0000 7f2e16800700 1 -- 172.21.15.63:0/3721589228 >> 172.21.15.63:0/3721589228 conn(0x7f2e101020c0 msgr2=0x7f2e10104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:19.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.873+0000 7f2e16800700 1 -- 172.21.15.63:0/3721589228 shutdown_connections 2024-05-27T08:09:19.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.873+0000 7f2e16800700 1 -- 172.21.15.63:0/3721589228 wait complete. 2024-05-27T08:09:19.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.874+0000 7f2e16800700 1 Processor -- start 2024-05-27T08:09:19.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.875+0000 7f2e16800700 1 -- start start 2024-05-27T08:09:19.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.875+0000 7f2e16800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e10106560 0x7f2e1019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.875+0000 7f2e16800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e10106eb0 0x7f2e1019d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.875+0000 7f2e16800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1010c5a0 0x7f2e101a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.875+0000 7f2e16800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101165b0 con 0x7f2e10106560 2024-05-27T08:09:19.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.875+0000 7f2e16800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10116430 con 0x7f2e1010c5a0 2024-05-27T08:09:19.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.875+0000 7f2e16800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10116730 con 0x7f2e10106eb0 2024-05-27T08:09:19.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.876+0000 7f2e0f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e10106560 0x7f2e1019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.876+0000 7f2e0f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e10106560 0x7f2e1019cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44778/0 (socket says 172.21.15.63:44778) 2024-05-27T08:09:19.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.876+0000 7f2e0f400700 1 -- 172.21.15.63:0/3030674352 learned_addr learned my addr 172.21.15.63:0/3030674352 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:19.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.876+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1010c5a0 0x7f2e101a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.876+0000 7f2e0f400700 1 -- 172.21.15.63:0/3030674352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e10106eb0 msgr2=0x7f2e1019d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:19.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.876+0000 7f2e0f400700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e10106eb0 0x7f2e1019d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.876+0000 7f2e0f400700 1 -- 172.21.15.63:0/3030674352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1010c5a0 msgr2=0x7f2e101a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:19.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.876+0000 7f2e0f400700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1010c5a0 0x7f2e101a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.876+0000 7f2e0f400700 1 -- 172.21.15.63:0/3030674352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e101a1e10 con 0x7f2e10106560 2024-05-27T08:09:19.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.877+0000 7f2e0f400700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e10106560 0x7f2e1019cf40 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f2e00000c00 tx=0x7f2e00005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:19.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.877+0000 7f2e07e00700 1 -- 172.21.15.63:0/3030674352 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00027070 con 0x7f2e10106560 2024-05-27T08:09:19.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.877+0000 7f2e07e00700 1 -- 172.21.15.63:0/3030674352 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e0000fb80 con 0x7f2e10106560 2024-05-27T08:09:19.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.877+0000 7f2e16800700 1 -- 172.21.15.63:0/3030674352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e101a20a0 con 0x7f2e10106560 2024-05-27T08:09:19.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.877+0000 7f2e07e00700 1 -- 172.21.15.63:0/3030674352 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00022c00 con 0x7f2e10106560 2024-05-27T08:09:19.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.877+0000 7f2e16800700 1 -- 172.21.15.63:0/3030674352 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e101a2580 con 0x7f2e10106560 2024-05-27T08:09:19.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.881+0000 7f2e07e00700 1 -- 172.21.15.63:0/3030674352 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2e0002a070 con 0x7f2e10106560 2024-05-27T08:09:19.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.883+0000 7f2e2531eb80 1 Processor -- start 2024-05-27T08:09:19.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.883+0000 7f2e07e00700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2df40b29f0 0x7f2df40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.884+0000 7f2e2531eb80 1 -- start start 2024-05-27T08:09:19.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.884+0000 7f2e07e00700 1 -- 172.21.15.63:0/3030674352 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f2e000e3ae0 con 0x7f2e10106560 2024-05-27T08:09:19.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.884+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2df40b29f0 0x7f2df40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.884+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 0x56526768e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.884+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x565267688870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.884+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267688db0 0x5652676891c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.884+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5652675b6e40 con 0x56526768ed30 2024-05-27T08:09:19.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.884+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5652675a69e0 con 0x565267688db0 2024-05-27T08:09:19.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.884+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5652675aaa20 con 0x565267509e40 2024-05-27T08:09:19.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x565267688870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x565267688870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44788/0 (socket says 172.21.15.63:44788) 2024-05-27T08:09:19.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0ea00700 1 -- 172.21.15.63:0/362792579 learned_addr learned my addr 172.21.15.63:0/362792579 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0f400700 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 0x56526768e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267688db0 0x5652676891c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0ea00700 1 -- 172.21.15.63:0/362792579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 msgr2=0x56526768e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0ea00700 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 0x56526768e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0ea00700 1 -- 172.21.15.63:0/362792579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267688db0 msgr2=0x5652676891c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0ea00700 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267688db0 0x5652676891c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0ea00700 1 -- 172.21.15.63:0/362792579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56526756f420 con 0x56526768ed30 2024-05-27T08:09:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0f400700 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 0x56526768e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:09:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267688db0 0x5652676891c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.885+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2df40b29f0 0x7f2df40b4f00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009ce0 tx=0x7f2e08009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:19.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.886+0000 7f2e0ea00700 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x565267688870 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00bb00 tx=0x7f2dfc00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5177683daa7344 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:19.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.887+0000 7f2e04c00700 1 -- 172.21.15.63:0/362792579 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc016070 con 0x56526768ed30 2024-05-27T08:09:19.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.887+0000 7f2e04c00700 1 -- 172.21.15.63:0/362792579 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2dfc004830 con 0x56526768ed30 2024-05-27T08:09:19.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.887+0000 7f2e04c00700 1 -- 172.21.15.63:0/362792579 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0053e0 con 0x56526768ed30 2024-05-27T08:09:19.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.888+0000 7f2e2531eb80 1 -- 172.21.15.63:0/362792579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 msgr2=0x565267688870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:19.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.888+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x565267688870 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00bb00 tx=0x7f2dfc00be10 comp rx=0 tx=0).stop 2024-05-27T08:09:19.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.888+0000 7f2e2531eb80 1 -- 172.21.15.63:0/362792579 shutdown_connections 2024-05-27T08:09:19.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.888+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267688db0 0x5652676891c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.888+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 0x56526768e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.888+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/362792579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x565267688870 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.888+0000 7f2e2531eb80 1 -- 172.21.15.63:0/362792579 >> 172.21.15.63:0/362792579 conn(0x565267569fc0 msgr2=0x5652676838d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:19.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.888+0000 7f2e2531eb80 1 -- 172.21.15.63:0/362792579 shutdown_connections 2024-05-27T08:09:19.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.888+0000 7f2e2531eb80 1 -- 172.21.15.63:0/362792579 wait complete. 2024-05-27T08:09:19.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.889+0000 7f2e2531eb80 1 -- start start 2024-05-27T08:09:19.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.889+0000 7f2e2531eb80 1 Processor -- start 2024-05-27T08:09:19.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 0x565267692440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267688db0 0x565267692980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 0x565267692ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5652674acfa0 con 0x565267688db0 2024-05-27T08:09:19.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565267547760 con 0x56526768ed30 2024-05-27T08:09:19.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x565267695800 con 0x565267509e40 2024-05-27T08:09:19.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e2531eb80 10 client.0 _ll_register_callbacks cb 0x565267550c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e0ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267688db0 0x565267692980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e0ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267688db0 0x565267692980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44800/0 (socket says 172.21.15.63:44800) 2024-05-27T08:09:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e0ea00700 1 -- 172.21.15.63:0/73124840 learned_addr learned my addr 172.21.15.63:0/73124840 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.890+0000 7f2e0f400700 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 0x565267692440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.891+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 0x565267692ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.891+0000 7f2e0ea00700 1 -- 172.21.15.63:0/73124840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 msgr2=0x565267692440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.891+0000 7f2e0ea00700 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 0x565267692440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.891+0000 7f2e0ea00700 1 -- 172.21.15.63:0/73124840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 msgr2=0x565267692ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.891+0000 7f2e0ea00700 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 0x565267692ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.891+0000 7f2e0ea00700 1 -- 172.21.15.63:0/73124840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652674f68d0 con 0x565267688db0 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.891+0000 7f2e0f400700 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 0x565267692440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.891+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 0x565267692ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.891+0000 7f2e0ea00700 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267688db0 0x565267692980 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc000c00 tx=0x7f2dfc004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65952d8f337108c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.891+0000 7f2df9600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.892+0000 7f2df9600700 1 -- 172.21.15.63:0/73124840 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc016070 con 0x565267688db0 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.892+0000 7f2df9600700 1 -- 172.21.15.63:0/73124840 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f2dfc004030 con 0x565267688db0 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.892+0000 7f2e2531eb80 1 -- 172.21.15.63:0/73124840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dfc004830 con 0x565267688db0 2024-05-27T08:09:19.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.892+0000 7f2df9600700 1 -- 172.21.15.63:0/73124840 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2dfc02d050 con 0x565267688db0 2024-05-27T08:09:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.892+0000 7f2e2531eb80 10 client.22251 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.892+0000 7f2e2531eb80 1 -- 172.21.15.63:0/73124840 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x565267577060 con 0x565267688db0 2024-05-27T08:09:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.893+0000 7f2df9600700 1 -- 172.21.15.63:0/73124840 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2dfc004830 con 0x565267688db0 2024-05-27T08:09:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.893+0000 7f2df9600700 1 -- 172.21.15.63:0/73124840 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2dfc003680 con 0x565267688db0 2024-05-27T08:09:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.893+0000 7f2e2531eb80 10 client.22251 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.893+0000 7f2e2531eb80 10 client.22251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:09:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.893+0000 7f2e2531eb80 20 client.22251 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.893+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5652676aaca0 0x5652676ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.893+0000 7f2e2531eb80 4 client.22251 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:09:19.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.893+0000 7f2e2531eb80 1 -- 172.21.15.63:0/73124840 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652675b8d90 con 0x5652676aaca0 2024-05-27T08:09:19.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.894+0000 7f2e0f400700 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5652676aaca0 0x5652676ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.895+0000 7f2e0f400700 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5652676aaca0 0x5652676ad0c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59fc8a2bbdb5d79f server_cookie=47c7eeb58ef33131 in_seq=0 out_seq=0 2024-05-27T08:09:19.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.895+0000 7f2df9600700 10 client.22251 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:09:19.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2df9600700 1 -- 172.21.15.63:0/73124840 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5652675b8d90 con 0x5652676aaca0 2024-05-27T08:09:19.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2df9600700 10 client.22251 handle_command_reply: tid=0 2024-05-27T08:09:19.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2e2531eb80 1 client.22251 shutdown 2024-05-27T08:09:19.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2e2531eb80 1 -- 172.21.15.63:0/73124840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267688db0 msgr2=0x565267692980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:19.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267688db0 0x565267692980 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc000c00 tx=0x7f2dfc004e90 comp rx=0 tx=0).stop 2024-05-27T08:09:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2e2531eb80 1 -- 172.21.15.63:0/73124840 shutdown_connections 2024-05-27T08:09:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 0x565267692ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267509e40 0x565267692440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5652676aaca0 0x5652676ad0c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/73124840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267688db0 0x565267692980 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2e2531eb80 1 -- 172.21.15.63:0/73124840 >> 172.21.15.63:0/73124840 conn(0x565267569fc0 msgr2=0x565267683580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.897+0000 7f2df9600700 0 client.22251 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:09:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.898+0000 7f2e2531eb80 1 -- 172.21.15.63:0/73124840 shutdown_connections 2024-05-27T08:09:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.898+0000 7f2e2531eb80 1 -- 172.21.15.63:0/73124840 wait complete. 2024-05-27T08:09:19.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.898+0000 7f2e2531eb80 20 client.22251 trim_cache size 0 max 16384 2024-05-27T08:09:19.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.916+0000 7f2e2531eb80 1 Processor -- start 2024-05-27T08:09:19.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.916+0000 7f2e2531eb80 1 -- start start 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.916+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267509e40 0x565267688870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.916+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267688db0 0x5652676948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 0x565267691310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101b0ac0 con 0x565267509e40 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101ae350 con 0x56526768ed30 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101ac1c0 con 0x565267688db0 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e0f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267509e40 0x565267688870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e0f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267509e40 0x565267688870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44816/0 (socket says 172.21.15.63:44816) 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e0f400700 1 -- 172.21.15.63:0/3053648083 learned_addr learned my addr 172.21.15.63:0/3053648083 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e0ea00700 1 --2- 172.21.15.63:0/3053648083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267688db0 0x5652676948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/3053648083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 0x565267691310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e0f400700 1 -- 172.21.15.63:0/3053648083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267688db0 msgr2=0x5652676948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:19.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e0f400700 1 --2- 172.21.15.63:0/3053648083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267688db0 0x5652676948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e0f400700 1 -- 172.21.15.63:0/3053648083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 msgr2=0x565267691310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:19.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e0f400700 1 --2- 172.21.15.63:0/3053648083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 0x565267691310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.917+0000 7f2e0f400700 1 -- 172.21.15.63:0/3053648083 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565267689510 con 0x565267509e40 2024-05-27T08:09:19.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.918+0000 7f2e0f400700 1 --2- 172.21.15.63:0/3053648083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267509e40 0x565267688870 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f2e000aa730 tx=0x7f2e00009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bf2fcbcabaf3f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:19.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.918+0000 7f2dfbe00700 1 -- 172.21.15.63:0/3053648083 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000aaa30 con 0x565267509e40 2024-05-27T08:09:19.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.918+0000 7f2dfbe00700 1 -- 172.21.15.63:0/3053648083 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e000aede0 con 0x565267509e40 2024-05-27T08:09:19.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.918+0000 7f2dfbe00700 1 -- 172.21.15.63:0/3053648083 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000aac90 con 0x565267509e40 2024-05-27T08:09:19.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.919+0000 7f2e2531eb80 1 -- 172.21.15.63:0/3053648083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267509e40 msgr2=0x565267688870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:19.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.919+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/3053648083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267509e40 0x565267688870 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f2e000aa730 tx=0x7f2e00009ca0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.919+0000 7f2e2531eb80 1 -- 172.21.15.63:0/3053648083 shutdown_connections 2024-05-27T08:09:19.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.919+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/3053648083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56526768ed30 0x565267691310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.919+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/3053648083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x565267509e40 0x565267688870 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.919+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/3053648083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267688db0 0x5652676948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.919+0000 7f2e2531eb80 1 -- 172.21.15.63:0/3053648083 >> 172.21.15.63:0/3053648083 conn(0x565267569fc0 msgr2=0x565267690c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:19.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.919+0000 7f2e2531eb80 1 -- 172.21.15.63:0/3053648083 shutdown_connections 2024-05-27T08:09:19.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.920+0000 7f2e2531eb80 1 -- 172.21.15.63:0/3053648083 wait complete. 2024-05-27T08:09:19.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.920+0000 7f2e2531eb80 1 -- start start 2024-05-27T08:09:19.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.920+0000 7f2e2531eb80 1 Processor -- start 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.921+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267509e40 0x5652676b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.921+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267688db0 0x5652676ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.921+0000 7f2e2531eb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x5652676adf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.921+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101ac810 con 0x56526768ed30 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.921+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101adc80 con 0x565267509e40 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.921+0000 7f2e2531eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101acee0 con 0x565267688db0 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.921+0000 7f2e2531eb80 10 client.0 _ll_register_callbacks cb 0x565267550c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.922+0000 7f2e0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x5652676adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.922+0000 7f2e0fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x5652676adf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44820/0 (socket says 172.21.15.63:44820) 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.922+0000 7f2e0fe00700 1 -- 172.21.15.63:0/2821368368 learned_addr learned my addr 172.21.15.63:0/2821368368 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.922+0000 7f2e0f400700 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267509e40 0x5652676b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.922+0000 7f2e0ea00700 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267688db0 0x5652676ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.922+0000 7f2e0fe00700 1 -- 172.21.15.63:0/2821368368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267688db0 msgr2=0x5652676ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:19.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.922+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267688db0 0x5652676ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.922+0000 7f2e0fe00700 1 -- 172.21.15.63:0/2821368368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267509e40 msgr2=0x5652676b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:19.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.922+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267509e40 0x5652676b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.922+0000 7f2e0fe00700 1 -- 172.21.15.63:0/2821368368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652676ae520 con 0x56526768ed30 2024-05-27T08:09:19.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.923+0000 7f2e0fe00700 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x5652676adf30 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f2e08017ce0 tx=0x7f2e08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcef9bd1ba2158 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:19.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.923+0000 7f2df9600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:19.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.923+0000 7f2df9600700 1 -- 172.21.15.63:0/2821368368 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08027020 con 0x56526768ed30 2024-05-27T08:09:19.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.923+0000 7f2df9600700 1 -- 172.21.15.63:0/2821368368 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f2e08016d30 con 0x56526768ed30 2024-05-27T08:09:19.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.923+0000 7f2e2531eb80 1 -- 172.21.15.63:0/2821368368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e000aede0 con 0x56526768ed30 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.923+0000 7f2df9600700 1 -- 172.21.15.63:0/2821368368 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2e08032050 con 0x56526768ed30 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.923+0000 7f2e2531eb80 10 client.22257 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.923+0000 7f2e2531eb80 1 -- 172.21.15.63:0/2821368368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5652677152f0 con 0x56526768ed30 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.924+0000 7f2df9600700 1 -- 172.21.15.63:0/2821368368 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e000aede0 con 0x56526768ed30 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.924+0000 7f2df9600700 1 -- 172.21.15.63:0/2821368368 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2e08014440 con 0x56526768ed30 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.924+0000 7f2e2531eb80 10 client.22257 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.924+0000 7f2e2531eb80 10 client.22257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.924+0000 7f2e2531eb80 20 client.22257 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.924+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x565267715a50 0x565267717e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.924+0000 7f2e2531eb80 4 client.22257 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.924+0000 7f2e2531eb80 1 -- 172.21.15.63:0/2821368368 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652675b2570 con 0x565267715a50 2024-05-27T08:09:19.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.925+0000 7f2e0f400700 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x565267715a50 0x565267717e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:19.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.926+0000 7f2e0f400700 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x565267715a50 0x565267717e70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ed431fb7eea8679 server_cookie=d846175462366a42 in_seq=0 out_seq=0 2024-05-27T08:09:19.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.926+0000 7f2df9600700 10 client.22257 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:09:19.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.926+0000 7f2df9600700 1 -- 172.21.15.63:0/2821368368 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652675b2570 con 0x565267715a50 2024-05-27T08:09:19.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.926+0000 7f2df9600700 10 client.22257 handle_command_reply: tid=0 2024-05-27T08:09:19.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.926+0000 7f2e2531eb80 1 client.22257 shutdown 2024-05-27T08:09:19.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.926+0000 7f2e2531eb80 1 -- 172.21.15.63:0/2821368368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 msgr2=0x5652676adf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:19.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.926+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x5652676adf30 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f2e08017ce0 tx=0x7f2e08004ad0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.927+0000 7f2e2531eb80 1 -- 172.21.15.63:0/2821368368 shutdown_connections 2024-05-27T08:09:19.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.927+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x565267715a50 0x565267717e70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.927+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56526768ed30 0x5652676adf30 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.927+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565267509e40 0x5652676b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.927+0000 7f2e2531eb80 1 --2- 172.21.15.63:0/2821368368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565267688db0 0x5652676ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.927+0000 7f2df9600700 0 client.22257 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:09:19.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.927+0000 7f2e2531eb80 1 -- 172.21.15.63:0/2821368368 >> 172.21.15.63:0/2821368368 conn(0x565267569fc0 msgr2=0x56526768f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:19.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.927+0000 7f2e2531eb80 1 -- 172.21.15.63:0/2821368368 shutdown_connections 2024-05-27T08:09:19.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.927+0000 7f2e2531eb80 1 -- 172.21.15.63:0/2821368368 wait complete. 2024-05-27T08:09:19.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.927+0000 7f2e2531eb80 20 client.22257 trim_cache size 0 max 16384 2024-05-27T08:09:19.931 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:09:19.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.928+0000 7f2e16800700 1 -- 172.21.15.63:0/3030674352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2df40b29f0 msgr2=0x7f2df40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:19.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.928+0000 7f2e16800700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2df40b29f0 0x7f2df40b4f00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009ce0 tx=0x7f2e08009040 comp rx=0 tx=0).stop 2024-05-27T08:09:19.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.928+0000 7f2e16800700 1 -- 172.21.15.63:0/3030674352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e10106560 msgr2=0x7f2e1019cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:19.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.928+0000 7f2e16800700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e10106560 0x7f2e1019cf40 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f2e00000c00 tx=0x7f2e00005ef0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.928+0000 7f2e16800700 1 -- 172.21.15.63:0/3030674352 shutdown_connections 2024-05-27T08:09:19.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.928+0000 7f2e16800700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2df40b29f0 0x7f2df40b4f00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.928+0000 7f2e16800700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1010c5a0 0x7f2e101a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.928+0000 7f2e16800700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e10106560 0x7f2e1019cf40 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.928+0000 7f2e16800700 1 --2- 172.21.15.63:0/3030674352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e10106eb0 0x7f2e1019d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:19.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.928+0000 7f2e16800700 1 -- 172.21.15.63:0/3030674352 >> 172.21.15.63:0/3030674352 conn(0x7f2e101020c0 msgr2=0x7f2e10104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:19.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.929+0000 7f2e16800700 1 -- 172.21.15.63:0/3030674352 shutdown_connections 2024-05-27T08:09:19.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:19.929+0000 7f2e16800700 1 -- 172.21.15.63:0/3030674352 wait complete. 2024-05-27T08:09:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:19 smithi086 ceph-mon[34687]: pgmap v1134: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:09:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:19 smithi149 ceph-mon[35408]: pgmap v1134: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:09:20.891 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:09:21.402 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:09:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.715+0000 7f2c93200640 1 -- 172.21.15.63:0/4122775623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c94075420 msgr2=0x7f2c94075830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.715+0000 7f2c93200640 1 --2- 172.21.15.63:0/4122775623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c94075420 0x7f2c94075830 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402a480 comp rx=0 tx=0).stop 2024-05-27T08:09:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.715+0000 7f2c93200640 1 -- 172.21.15.63:0/4122775623 shutdown_connections 2024-05-27T08:09:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.715+0000 7f2c93200640 1 --2- 172.21.15.63:0/4122775623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c94110320 0x7f2c94112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.715+0000 7f2c93200640 1 --2- 172.21.15.63:0/4122775623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9410d9c0 0x7f2c9410fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.715+0000 7f2c93200640 1 --2- 172.21.15.63:0/4122775623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c94075420 0x7f2c94075830 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.715+0000 7f2c93200640 1 -- 172.21.15.63:0/4122775623 >> 172.21.15.63:0/4122775623 conn(0x7f2c94079100 msgr2=0x7f2c94079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.715+0000 7f2c93200640 1 -- 172.21.15.63:0/4122775623 shutdown_connections 2024-05-27T08:09:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.715+0000 7f2c93200640 1 -- 172.21.15.63:0/4122775623 wait complete. 2024-05-27T08:09:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.717+0000 7f2c93200640 1 Processor -- start 2024-05-27T08:09:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.717+0000 7f2c93200640 1 -- start start 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.718+0000 7f2c93200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c94075420 0x7f2c941a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.718+0000 7f2c93200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9410d9c0 0x7f2c941a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.718+0000 7f2c93200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c94110320 0x7f2c9419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.718+0000 7f2c93200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94115d40 con 0x7f2c94110320 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.718+0000 7f2c93200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94115bc0 con 0x7f2c9410d9c0 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.718+0000 7f2c93200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94115ec0 con 0x7f2c94075420 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.718+0000 7f2c92800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c94110320 0x7f2c9419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.718+0000 7f2c92800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c94110320 0x7f2c9419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44832/0 (socket says 172.21.15.63:44832) 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.718+0000 7f2c92800640 1 -- 172.21.15.63:0/3528498812 learned_addr learned my addr 172.21.15.63:0/3528498812 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.718+0000 7f2c91e00640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c94075420 0x7f2c941a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.719+0000 7f2c91400640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9410d9c0 0x7f2c941a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.719+0000 7f2c92800640 1 -- 172.21.15.63:0/3528498812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c94075420 msgr2=0x7f2c941a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.719+0000 7f2c92800640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c94075420 0x7f2c941a2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.719+0000 7f2c92800640 1 -- 172.21.15.63:0/3528498812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9410d9c0 msgr2=0x7f2c941a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.719+0000 7f2c92800640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9410d9c0 0x7f2c941a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.719+0000 7f2c92800640 1 -- 172.21.15.63:0/3528498812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9419ce10 con 0x7f2c94110320 2024-05-27T08:09:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.720+0000 7f2c92800640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c94110320 0x7f2c9419c6e0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00eb10 tx=0x7f2c8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.720+0000 7f2c8aa00640 1 -- 172.21.15.63:0/3528498812 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c004b20 con 0x7f2c94110320 2024-05-27T08:09:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.720+0000 7f2c8aa00640 1 -- 172.21.15.63:0/3528498812 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c8c004cc0 con 0x7f2c94110320 2024-05-27T08:09:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.720+0000 7f2c93200640 1 -- 172.21.15.63:0/3528498812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9419d0a0 con 0x7f2c94110320 2024-05-27T08:09:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.720+0000 7f2c8aa00640 1 -- 172.21.15.63:0/3528498812 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c010690 con 0x7f2c94110320 2024-05-27T08:09:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.720+0000 7f2c93200640 1 -- 172.21.15.63:0/3528498812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c941a9300 con 0x7f2c94110320 2024-05-27T08:09:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.724+0000 7f2c9fe5a740 1 Processor -- start 2024-05-27T08:09:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.726+0000 7f2c8aa00640 1 -- 172.21.15.63:0/3528498812 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2c8c010830 con 0x7f2c94110320 2024-05-27T08:09:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.726+0000 7f2c9fe5a740 1 -- start start 2024-05-27T08:09:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.727+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bdddc1b0 0x55f6be17ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.727+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be17efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.727+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bddbb4d0 0x55f6be17f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.727+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f6be0efb90 con 0x55f6bddb32f0 2024-05-27T08:09:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.727+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f6be0df730 con 0x55f6bddbb4d0 2024-05-27T08:09:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.727+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f6be0e3770 con 0x55f6bdddc1b0 2024-05-27T08:09:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.728+0000 7f2c8aa00640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2c780b2950 0x7f2c780b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.729+0000 7f2c8aa00640 1 -- 172.21.15.63:0/3528498812 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f2c8c0e2f00 con 0x7f2c94110320 2024-05-27T08:09:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.729+0000 7f2c91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bddbb4d0 0x55f6be17f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.729+0000 7f2c91400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bdddc1b0 0x55f6be17ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.729+0000 7f2c91e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be17efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.729+0000 7f2c91400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bdddc1b0 0x55f6be17ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41720/0 (socket says 172.21.15.63:41720) 2024-05-27T08:09:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.729+0000 7f2c91400640 1 -- 172.21.15.63:0/372214003 learned_addr learned my addr 172.21.15.63:0/372214003 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.729+0000 7f2c92800640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2c780b2950 0x7f2c780b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.730+0000 7f2c91e00640 1 -- 172.21.15.63:0/372214003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bdddc1b0 msgr2=0x55f6be17ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.730+0000 7f2c91e00640 1 --2- 172.21.15.63:0/372214003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bdddc1b0 0x55f6be17ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.730+0000 7f2c91e00640 1 -- 172.21.15.63:0/372214003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bddbb4d0 msgr2=0x55f6be17f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.730+0000 7f2c91e00640 1 --2- 172.21.15.63:0/372214003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bddbb4d0 0x55f6be17f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.730+0000 7f2c91e00640 1 -- 172.21.15.63:0/372214003 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6be0788b0 con 0x55f6bddb32f0 2024-05-27T08:09:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.730+0000 7f2c91e00640 1 --2- 172.21.15.63:0/372214003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be17efa0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005940 tx=0x7f2c84031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5616e9689c4665e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.730+0000 7f2c92800640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2c780b2950 0x7f2c780b4e60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009270 tx=0x7f2c8c009200 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.731+0000 7f2c7f400640 1 -- 172.21.15.63:0/372214003 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84035030 con 0x55f6bddb32f0 2024-05-27T08:09:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.731+0000 7f2c7f400640 1 -- 172.21.15.63:0/372214003 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c84040070 con 0x55f6bddb32f0 2024-05-27T08:09:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.731+0000 7f2c7f400640 1 -- 172.21.15.63:0/372214003 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c840035f0 con 0x55f6bddb32f0 2024-05-27T08:09:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.732+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/372214003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 msgr2=0x55f6be17efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.732+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/372214003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be17efa0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005940 tx=0x7f2c84031a70 comp rx=0 tx=0).stop 2024-05-27T08:09:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.732+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/372214003 shutdown_connections 2024-05-27T08:09:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.732+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/372214003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bddbb4d0 0x55f6be17f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.732+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/372214003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be17efa0 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.732+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/372214003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bdddc1b0 0x55f6be17ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.732+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/372214003 >> 172.21.15.63:0/372214003 conn(0x55f6bddd0300 msgr2=0x55f6be17b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.732+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/372214003 shutdown_connections 2024-05-27T08:09:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.732+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/372214003 wait complete. 2024-05-27T08:09:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.733+0000 7f2c9fe5a740 1 -- start start 2024-05-27T08:09:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.733+0000 7f2c9fe5a740 1 Processor -- start 2024-05-27T08:09:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be188ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddbb4d0 0x55f6be189400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bdddc1b0 0x55f6be194e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f6bddc5790 con 0x55f6bddb32f0 2024-05-27T08:09:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f6be087bc0 con 0x55f6bdddc1b0 2024-05-27T08:09:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f6be18bb30 con 0x55f6bddbb4d0 2024-05-27T08:09:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c9fe5a740 10 client.0 _ll_register_callbacks cb 0x55f6be065400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c91400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be188ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c91400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be188ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44852/0 (socket says 172.21.15.63:44852) 2024-05-27T08:09:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c91400640 1 -- 172.21.15.63:0/2471984363 learned_addr learned my addr 172.21.15.63:0/2471984363 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.734+0000 7f2c91e00640 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddbb4d0 0x55f6be189400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.735+0000 7f2c91400640 1 -- 172.21.15.63:0/2471984363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddbb4d0 msgr2=0x55f6be189400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.735+0000 7f2c91400640 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddbb4d0 0x55f6be189400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.735+0000 7f2c91400640 1 -- 172.21.15.63:0/2471984363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bdddc1b0 msgr2=0x55f6be194e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.735+0000 7f2c91400640 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bdddc1b0 0x55f6be194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.735+0000 7f2c91400640 1 -- 172.21.15.63:0/2471984363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6bddf4210 con 0x55f6bddb32f0 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.735+0000 7f2c91400640 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be188ec0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009c10 tx=0x7f2c80009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5cff9073ab2dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.735+0000 7f2c73e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.736+0000 7f2c73e00640 1 -- 172.21.15.63:0/2471984363 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80019040 con 0x55f6bddb32f0 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.736+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/2471984363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c84040070 con 0x55f6bddb32f0 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.736+0000 7f2c73e00640 1 -- 172.21.15.63:0/2471984363 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f2c800047a0 con 0x55f6bddb32f0 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.736+0000 7f2c73e00640 1 -- 172.21.15.63:0/2471984363 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2c800049e0 con 0x55f6bddb32f0 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.736+0000 7f2c9fe5a740 10 client.22269 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.736+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/2471984363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f6bdec0620 con 0x55f6bddb32f0 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.737+0000 7f2c73e00640 1 -- 172.21.15.63:0/2471984363 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c84040070 con 0x55f6bddb32f0 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.737+0000 7f2c73e00640 1 -- 172.21.15.63:0/2471984363 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2c800273d0 con 0x55f6bddb32f0 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.737+0000 7f2c9fe5a740 10 client.22269 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.737+0000 7f2c9fe5a740 10 client.22269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.737+0000 7f2c9fe5a740 20 client.22269 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.737+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f6bdd9ad80 0x55f6be19af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.737+0000 7f2c9fe5a740 4 client.22269 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:09:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.737+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/2471984363 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6be0f1ae0 con 0x55f6bdd9ad80 2024-05-27T08:09:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.738+0000 7f2c91e00640 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f6bdd9ad80 0x55f6be19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.739+0000 7f2c91e00640 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f6bdd9ad80 0x55f6be19af20 crc :-1 s=READY pgs=638 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acfd776bb5634635 server_cookie=a8d8a6adabedb109 in_seq=0 out_seq=0 2024-05-27T08:09:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.739+0000 7f2c73e00640 10 client.22269 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.740+0000 7f2c73e00640 1 -- 172.21.15.63:0/2471984363 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f6be0f1ae0 con 0x55f6bdd9ad80 2024-05-27T08:09:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c73e00640 10 client.22269 handle_command_reply: tid=0 2024-05-27T08:09:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 client.22269 shutdown 2024-05-27T08:09:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/2471984363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 msgr2=0x55f6be188ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be188ec0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009c10 tx=0x7f2c80009c40 comp rx=0 tx=0).stop 2024-05-27T08:09:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/2471984363 shutdown_connections 2024-05-27T08:09:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f6bdd9ad80 0x55f6be19af20 crc :-1 s=READY pgs=638 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bdddc1b0 0x55f6be194e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddbb4d0 0x55f6be189400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/2471984363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddb32f0 0x55f6be188ec0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/2471984363 >> 172.21.15.63:0/2471984363 conn(0x55f6bddd0300 msgr2=0x55f6be17e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c73e00640 0 client.22269 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/2471984363 shutdown_connections 2024-05-27T08:09:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/2471984363 wait complete. 2024-05-27T08:09:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.741+0000 7f2c9fe5a740 20 client.22269 trim_cache size 0 max 16384 2024-05-27T08:09:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.763+0000 7f2c9fe5a740 1 Processor -- start 2024-05-27T08:09:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.763+0000 7f2c9fe5a740 1 -- start start 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.763+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddb32f0 0x55f6be189550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.763+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bddbb4d0 0x55f6be194bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.763+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bdddc1b0 0x55f6be195110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.763+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941b5b40 con 0x55f6bdddc1b0 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.763+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941b3410 con 0x55f6bddbb4d0 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.763+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941b1280 con 0x55f6bddb32f0 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.764+0000 7f2c91400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bdddc1b0 0x55f6be195110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.764+0000 7f2c91400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bdddc1b0 0x55f6be195110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44860/0 (socket says 172.21.15.63:44860) 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.764+0000 7f2c91400640 1 -- 172.21.15.63:0/1715588201 learned_addr learned my addr 172.21.15.63:0/1715588201 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.764+0000 7f2c91e00640 1 --2- 172.21.15.63:0/1715588201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bddbb4d0 0x55f6be194bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.764+0000 7f2c91400640 1 --2- 172.21.15.63:0/1715588201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddb32f0 0x55f6be189550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.764+0000 7f2c91400640 1 -- 172.21.15.63:0/1715588201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddb32f0 msgr2=0x55f6be189550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.764+0000 7f2c91400640 1 --2- 172.21.15.63:0/1715588201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddb32f0 0x55f6be189550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.764+0000 7f2c91400640 1 -- 172.21.15.63:0/1715588201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bddbb4d0 msgr2=0x55f6be194bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.764+0000 7f2c91400640 1 --2- 172.21.15.63:0/1715588201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bddbb4d0 0x55f6be194bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.764+0000 7f2c91400640 1 -- 172.21.15.63:0/1715588201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6be17bde0 con 0x55f6bdddc1b0 2024-05-27T08:09:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.765+0000 7f2c91400640 1 --2- 172.21.15.63:0/1715588201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bdddc1b0 0x55f6be195110 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f2c80027840 tx=0x7f2c80014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb79d8276ae0271 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.765+0000 7f2c7ea00640 1 -- 172.21.15.63:0/1715588201 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80027e80 con 0x55f6bdddc1b0 2024-05-27T08:09:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.765+0000 7f2c7ea00640 1 -- 172.21.15.63:0/1715588201 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c8001e890 con 0x55f6bdddc1b0 2024-05-27T08:09:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.765+0000 7f2c7ea00640 1 -- 172.21.15.63:0/1715588201 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80027a30 con 0x55f6bdddc1b0 2024-05-27T08:09:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.766+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1715588201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bdddc1b0 msgr2=0x55f6be195110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.766+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/1715588201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bdddc1b0 0x55f6be195110 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f2c80027840 tx=0x7f2c80014040 comp rx=0 tx=0).stop 2024-05-27T08:09:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.766+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1715588201 shutdown_connections 2024-05-27T08:09:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.766+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/1715588201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bdddc1b0 0x55f6be195110 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.766+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/1715588201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bddbb4d0 0x55f6be194bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.766+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/1715588201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddb32f0 0x55f6be189550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.766+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1715588201 >> 172.21.15.63:0/1715588201 conn(0x55f6bddd0300 msgr2=0x55f6be18ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.766+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1715588201 shutdown_connections 2024-05-27T08:09:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.766+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1715588201 wait complete. 2024-05-27T08:09:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.767+0000 7f2c9fe5a740 1 -- start start 2024-05-27T08:09:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.767+0000 7f2c9fe5a740 1 Processor -- start 2024-05-27T08:09:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.768+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddb32f0 0x55f6be18de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.768+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddbb4d0 0x55f6be18e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.768+0000 7f2c9fe5a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bdddc1b0 0x55f6be18e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.768+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941b18d0 con 0x55f6bddbb4d0 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.768+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941b2d40 con 0x55f6bdddc1b0 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.768+0000 7f2c9fe5a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941b1fa0 con 0x55f6bddb32f0 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.768+0000 7f2c9fe5a740 10 client.0 _ll_register_callbacks cb 0x7f2c80010970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.768+0000 7f2c91e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddbb4d0 0x55f6be18e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.768+0000 7f2c91e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddbb4d0 0x55f6be18e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44862/0 (socket says 172.21.15.63:44862) 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.768+0000 7f2c91e00640 1 -- 172.21.15.63:0/1405125676 learned_addr learned my addr 172.21.15.63:0/1405125676 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.769+0000 7f2c91e00640 1 -- 172.21.15.63:0/1405125676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddb32f0 msgr2=0x55f6be18de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.769+0000 7f2c91e00640 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddb32f0 0x55f6be18de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.769+0000 7f2c91e00640 1 -- 172.21.15.63:0/1405125676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bdddc1b0 msgr2=0x55f6be18e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.769+0000 7f2c91e00640 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bdddc1b0 0x55f6be18e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.769+0000 7f2c91e00640 1 -- 172.21.15.63:0/1405125676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6be201840 con 0x55f6bddbb4d0 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.769+0000 7f2c91e00640 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddbb4d0 0x55f6be18e390 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a9a0 tx=0x7f2c84005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589d2ffc0cff3456 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.769+0000 7f2c73e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.769+0000 7f2c73e00640 1 -- 172.21.15.63:0/1405125676 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84035040 con 0x55f6bddbb4d0 2024-05-27T08:09:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.769+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1405125676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c8001e890 con 0x55f6bddbb4d0 2024-05-27T08:09:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.769+0000 7f2c73e00640 1 -- 172.21.15.63:0/1405125676 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f2c84004240 con 0x55f6bddbb4d0 2024-05-27T08:09:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c73e00640 1 -- 172.21.15.63:0/1405125676 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2c8405d050 con 0x55f6bddbb4d0 2024-05-27T08:09:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c9fe5a740 10 client.22275 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1405125676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f6be202ad0 con 0x55f6bddbb4d0 2024-05-27T08:09:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c73e00640 1 -- 172.21.15.63:0/1405125676 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c8001e890 con 0x55f6bddbb4d0 2024-05-27T08:09:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c73e00640 1 -- 172.21.15.63:0/1405125676 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2c84003b20 con 0x55f6bddbb4d0 2024-05-27T08:09:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c9fe5a740 10 client.22275 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c9fe5a740 10 client.22275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c9fe5a740 20 client.22275 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f6bdd701c0 0x55f6be204dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c9fe5a740 4 client.22275 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:09:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.770+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1405125676 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6be0eb2c0 con 0x55f6bdd701c0 2024-05-27T08:09:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.771+0000 7f2c91400640 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f6bdd701c0 0x55f6be204dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.773+0000 7f2c73e00640 10 client.22275 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.773+0000 7f2c91400640 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f6bdd701c0 0x55f6be204dc0 crc :-1 s=READY pgs=639 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e62e385230b069 server_cookie=681bd5385628fd38 in_seq=0 out_seq=0 2024-05-27T08:09:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c73e00640 1 -- 172.21.15.63:0/1405125676 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f6be0eb2c0 con 0x55f6bdd701c0 2024-05-27T08:09:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c73e00640 10 client.22275 handle_command_reply: tid=0 2024-05-27T08:09:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c9fe5a740 1 client.22275 shutdown 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1405125676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddbb4d0 msgr2=0x55f6be18e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddbb4d0 0x55f6be18e390 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f2c8402a9a0 tx=0x7f2c84005a20 comp rx=0 tx=0).stop 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1405125676 shutdown_connections 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f6bdd701c0 0x55f6be204dc0 crc :-1 s=READY pgs=639 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6bdddc1b0 0x55f6be18e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f6bddbb4d0 0x55f6be18e390 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c73e00640 0 client.22275 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c9fe5a740 1 --2- 172.21.15.63:0/1405125676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f6bddb32f0 0x55f6be18de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.813+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1405125676 >> 172.21.15.63:0/1405125676 conn(0x55f6bddd0300 msgr2=0x55f6be188510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.814+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1405125676 shutdown_connections 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.814+0000 7f2c9fe5a740 1 -- 172.21.15.63:0/1405125676 wait complete. 2024-05-27T08:09:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.814+0000 7f2c9fe5a740 20 client.22275 trim_cache size 0 max 16384 2024-05-27T08:09:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:09:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:09:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 -- 172.21.15.63:0/3528498812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2c780b2950 msgr2=0x7f2c780b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2c780b2950 0x7f2c780b4e60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009270 tx=0x7f2c8c009200 comp rx=0 tx=0).stop 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 -- 172.21.15.63:0/3528498812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c94110320 msgr2=0x7f2c9419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c94110320 0x7f2c9419c6e0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00eb10 tx=0x7f2c8c00c2d0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 -- 172.21.15.63:0/3528498812 shutdown_connections 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2c780b2950 0x7f2c780b4e60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c94110320 0x7f2c9419c6e0 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c9410d9c0 0x7f2c941a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 --2- 172.21.15.63:0/3528498812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c94075420 0x7f2c941a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 -- 172.21.15.63:0/3528498812 >> 172.21.15.63:0/3528498812 conn(0x7f2c94079100 msgr2=0x7f2c941022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 -- 172.21.15.63:0/3528498812 shutdown_connections 2024-05-27T08:09:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:21.815+0000 7f2c93200640 1 -- 172.21.15.63:0/3528498812 wait complete. 2024-05-27T08:09:21.894 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:21 smithi063 ceph-mon[31613]: pgmap v1135: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:09:21.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:21 smithi086 ceph-mon[34687]: pgmap v1135: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:09:21.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:21 smithi149 ceph-mon[35408]: pgmap v1135: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:09:23.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.711+0000 7f656ca00700 1 -- 172.21.15.63:0/920317373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f656810e710 msgr2=0x7f6568110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:23.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.711+0000 7f656ca00700 1 --2- 172.21.15.63:0/920317373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f656810e710 0x7f6568110bf0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f656000b0a0 tx=0x7f656000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:23 smithi063 ceph-mon[31613]: pgmap v1136: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 9 op/s 2024-05-27T08:09:23.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.711+0000 7f656ca00700 1 -- 172.21.15.63:0/920317373 shutdown_connections 2024-05-27T08:09:23.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.711+0000 7f656ca00700 1 --2- 172.21.15.63:0/920317373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f656810e710 0x7f6568110bf0 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.711+0000 7f656ca00700 1 --2- 172.21.15.63:0/920317373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65681052e0 0x7f65681056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.711+0000 7f656ca00700 1 --2- 172.21.15.63:0/920317373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6568105c30 0x7f656810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.711+0000 7f656ca00700 1 -- 172.21.15.63:0/920317373 >> 172.21.15.63:0/920317373 conn(0x7f6568078af0 msgr2=0x7f6568078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:23.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.711+0000 7f656ca00700 1 -- 172.21.15.63:0/920317373 shutdown_connections 2024-05-27T08:09:23.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.711+0000 7f656ca00700 1 -- 172.21.15.63:0/920317373 wait complete. 2024-05-27T08:09:23.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.712+0000 7f656ca00700 1 Processor -- start 2024-05-27T08:09:23.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.713+0000 7f656ca00700 1 -- start start 2024-05-27T08:09:23.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.713+0000 7f656ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65681052e0 0x7f6568102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.713+0000 7f656ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6568105c30 0x7f6568103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.713+0000 7f656ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656810e710 0x7f6568103a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.713+0000 7f656ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6568114120 con 0x7f65681052e0 2024-05-27T08:09:23.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.713+0000 7f656ca00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6568113fa0 con 0x7f6568105c30 2024-05-27T08:09:23.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.713+0000 7f656ca00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65681142a0 con 0x7f656810e710 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.713+0000 7f6566800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65681052e0 0x7f6568102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.714+0000 7f6566800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65681052e0 0x7f6568102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44888/0 (socket says 172.21.15.63:44888) 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.714+0000 7f6566800700 1 -- 172.21.15.63:0/2988752321 learned_addr learned my addr 172.21.15.63:0/2988752321 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.714+0000 7f6565e00700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6568105c30 0x7f6568103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.714+0000 7f6567200700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656810e710 0x7f6568103a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.714+0000 7f6566800700 1 -- 172.21.15.63:0/2988752321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656810e710 msgr2=0x7f6568103a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.714+0000 7f6566800700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656810e710 0x7f6568103a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.714+0000 7f6566800700 1 -- 172.21.15.63:0/2988752321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6568105c30 msgr2=0x7f6568103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.714+0000 7f6566800700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6568105c30 0x7f6568103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.714+0000 7f6566800700 1 -- 172.21.15.63:0/2988752321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65681a8e50 con 0x7f65681052e0 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.714+0000 7f6567200700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656810e710 0x7f6568103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:23.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.715+0000 7f6566800700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65681052e0 0x7f6568102fd0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f655800d950 tx=0x7f655800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:23.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.715+0000 7f655f400700 1 -- 172.21.15.63:0/2988752321 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558016070 con 0x7f65681052e0 2024-05-27T08:09:23.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.715+0000 7f655f400700 1 -- 172.21.15.63:0/2988752321 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6558010460 con 0x7f65681052e0 2024-05-27T08:09:23.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.715+0000 7f655f400700 1 -- 172.21.15.63:0/2988752321 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655800f620 con 0x7f65681052e0 2024-05-27T08:09:23.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.715+0000 7f656ca00700 1 -- 172.21.15.63:0/2988752321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65681a90e0 con 0x7f65681052e0 2024-05-27T08:09:23.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.715+0000 7f656ca00700 1 -- 172.21.15.63:0/2988752321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65681a95d0 con 0x7f65681052e0 2024-05-27T08:09:23.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.721+0000 7f655f400700 1 -- 172.21.15.63:0/2988752321 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6558019070 con 0x7f65681052e0 2024-05-27T08:09:23.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.721+0000 7f657c02cb80 1 Processor -- start 2024-05-27T08:09:23.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.721+0000 7f657c02cb80 1 -- start start 2024-05-27T08:09:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.722+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee33c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.722+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee1bce40 0x558fee33cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.722+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee3486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.722+0000 7f657c02cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558fee269e40 con 0x558fee3462c0 2024-05-27T08:09:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.722+0000 7f657c02cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fee2599e0 con 0x558fee1bce40 2024-05-27T08:09:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.722+0000 7f657c02cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558fee25da20 con 0x558fee2a89f0 2024-05-27T08:09:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.722+0000 7f6567200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee33c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.722+0000 7f6567200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee33c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41756/0 (socket says 172.21.15.63:41756) 2024-05-27T08:09:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.722+0000 7f6567200700 1 -- 172.21.15.63:0/794619290 learned_addr learned my addr 172.21.15.63:0/794619290 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.723+0000 7f6566800700 1 --2- 172.21.15.63:0/794619290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee1bce40 0x558fee33cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.723+0000 7f655f400700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f654c0b2b40 0x7f654c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.723+0000 7f6565e00700 1 --2- 172.21.15.63:0/794619290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee3486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.723+0000 7f655f400700 1 -- 172.21.15.63:0/2988752321 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f65580e2550 con 0x7f65681052e0 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.723+0000 7f6567200700 1 -- 172.21.15.63:0/794619290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee1bce40 msgr2=0x558fee33cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.723+0000 7f6567200700 1 --2- 172.21.15.63:0/794619290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee1bce40 0x558fee33cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.723+0000 7f6567200700 1 -- 172.21.15.63:0/794619290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 msgr2=0x558fee3486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.723+0000 7f6567200700 1 --2- 172.21.15.63:0/794619290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee3486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.723+0000 7f6567200700 1 -- 172.21.15.63:0/794619290 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fee222420 con 0x558fee2a89f0 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.724+0000 7f6567200700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f654c0b2b40 0x7f654c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.724+0000 7f6567200700 1 --2- 172.21.15.63:0/794619290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee33c750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6560006130 tx=0x7f65600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9827e2806473af8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:23.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.724+0000 7f6553e00700 1 -- 172.21.15.63:0/794619290 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560023070 con 0x558fee2a89f0 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.725+0000 7f6567200700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f654c0b2b40 0x7f654c0b5050 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6560029000 tx=0x7f6560026000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.725+0000 7f6553e00700 1 -- 172.21.15.63:0/794619290 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6560007c60 con 0x558fee2a89f0 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.725+0000 7f6553e00700 1 -- 172.21.15.63:0/794619290 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656001d450 con 0x558fee2a89f0 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.726+0000 7f657c02cb80 1 -- 172.21.15.63:0/794619290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 msgr2=0x558fee33c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.726+0000 7f657c02cb80 1 --2- 172.21.15.63:0/794619290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee33c750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6560006130 tx=0x7f65600091a0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.726+0000 7f657c02cb80 1 -- 172.21.15.63:0/794619290 shutdown_connections 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.726+0000 7f657c02cb80 1 --2- 172.21.15.63:0/794619290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee3486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.726+0000 7f657c02cb80 1 --2- 172.21.15.63:0/794619290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee33c750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.726+0000 7f657c02cb80 1 --2- 172.21.15.63:0/794619290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee1bce40 0x558fee33cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.726+0000 7f657c02cb80 1 -- 172.21.15.63:0/794619290 >> 172.21.15.63:0/794619290 conn(0x558fee256390 msgr2=0x558fee337710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:23.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.726+0000 7f657c02cb80 1 -- 172.21.15.63:0/794619290 shutdown_connections 2024-05-27T08:09:23.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.726+0000 7f657c02cb80 1 -- 172.21.15.63:0/794619290 wait complete. 2024-05-27T08:09:23.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.727+0000 7f657c02cb80 1 -- start start 2024-05-27T08:09:23.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.727+0000 7f657c02cb80 1 Processor -- start 2024-05-27T08:09:23.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee1bce40 0x558fee343320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee343860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee343da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f657c02cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558fee15ffa0 con 0x558fee3462c0 2024-05-27T08:09:23.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f657c02cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fee1fa760 con 0x558fee1bce40 2024-05-27T08:09:23.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f657c02cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558fee348e00 con 0x558fee2a89f0 2024-05-27T08:09:23.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f657c02cb80 10 client.0 _ll_register_callbacks cb 0x558fee203c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f6567200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee343da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f6567200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee343da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44902/0 (socket says 172.21.15.63:44902) 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f6567200700 1 -- 172.21.15.63:0/2285704359 learned_addr learned my addr 172.21.15.63:0/2285704359 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f6566800700 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee1bce40 0x558fee343320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.728+0000 7f6565e00700 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee343860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.729+0000 7f6567200700 1 -- 172.21.15.63:0/2285704359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 msgr2=0x558fee343860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.729+0000 7f6567200700 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee343860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.729+0000 7f6567200700 1 -- 172.21.15.63:0/2285704359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee1bce40 msgr2=0x558fee343320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.729+0000 7f6567200700 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee1bce40 0x558fee343320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.729+0000 7f6567200700 1 -- 172.21.15.63:0/2285704359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fee1a98d0 con 0x558fee3462c0 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.729+0000 7f6567200700 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee343da0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f6560006100 tx=0x7f65600099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32691a9a5aff772b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:23.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.729+0000 7f6550c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.730+0000 7f6550c00700 1 -- 172.21.15.63:0/2285704359 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560036020 con 0x558fee3462c0 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.730+0000 7f6550c00700 1 -- 172.21.15.63:0/2285704359 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f656001b670 con 0x558fee3462c0 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.730+0000 7f657c02cb80 1 -- 172.21.15.63:0/2285704359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6560007c60 con 0x558fee3462c0 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.730+0000 7f6550c00700 1 -- 172.21.15.63:0/2285704359 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6560023070 con 0x558fee3462c0 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.730+0000 7f657c02cb80 10 client.22287 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.730+0000 7f657c02cb80 1 -- 172.21.15.63:0/2285704359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558fee22a060 con 0x558fee3462c0 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.730+0000 7f6550c00700 1 -- 172.21.15.63:0/2285704359 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6560007c60 con 0x558fee3462c0 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.731+0000 7f6550c00700 1 -- 172.21.15.63:0/2285704359 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6560011070 con 0x558fee3462c0 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.731+0000 7f657c02cb80 10 client.22287 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.731+0000 7f657c02cb80 10 client.22287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.731+0000 7f657c02cb80 20 client.22287 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.731+0000 7f657c02cb80 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558fee355c50 0x558fee358070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.731+0000 7f657c02cb80 4 client.22287 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:09:23.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.731+0000 7f657c02cb80 1 -- 172.21.15.63:0/2285704359 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fee26bd90 con 0x558fee355c50 2024-05-27T08:09:23.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.732+0000 7f6566800700 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558fee355c50 0x558fee358070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.733+0000 7f6550c00700 10 client.22287 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:23.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.733+0000 7f6566800700 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558fee355c50 0x558fee358070 crc :-1 s=READY pgs=640 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e4a6e6cb8233ec server_cookie=1e56d0e2b1fceb64 in_seq=0 out_seq=0 2024-05-27T08:09:23.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f6550c00700 1 -- 172.21.15.63:0/2285704359 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558fee26bd90 con 0x558fee355c50 2024-05-27T08:09:23.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f6550c00700 10 client.22287 handle_command_reply: tid=0 2024-05-27T08:09:23.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f657c02cb80 1 client.22287 shutdown 2024-05-27T08:09:23.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f657c02cb80 1 -- 172.21.15.63:0/2285704359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 msgr2=0x558fee343da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:23.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f657c02cb80 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee343da0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f6560006100 tx=0x7f65600099e0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f657c02cb80 1 -- 172.21.15.63:0/2285704359 shutdown_connections 2024-05-27T08:09:23.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f657c02cb80 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558fee355c50 0x558fee358070 crc :-1 s=READY pgs=640 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f657c02cb80 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee343da0 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f657c02cb80 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee1bce40 0x558fee343320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f657c02cb80 1 --2- 172.21.15.63:0/2285704359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee343860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f6550c00700 0 client.22287 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:23.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.735+0000 7f657c02cb80 1 -- 172.21.15.63:0/2285704359 >> 172.21.15.63:0/2285704359 conn(0x558fee256390 msgr2=0x558fee33c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:23.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.736+0000 7f657c02cb80 1 -- 172.21.15.63:0/2285704359 shutdown_connections 2024-05-27T08:09:23.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.736+0000 7f657c02cb80 1 -- 172.21.15.63:0/2285704359 wait complete. 2024-05-27T08:09:23.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.736+0000 7f657c02cb80 20 client.22287 trim_cache size 0 max 16384 2024-05-27T08:09:23.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.755+0000 7f657c02cb80 1 Processor -- start 2024-05-27T08:09:23.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.755+0000 7f657c02cb80 1 -- start start 2024-05-27T08:09:23.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.755+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee1bce40 0x558fee337740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.755+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee340580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.755+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee3462c0 0x558fee340ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.755+0000 7f657c02cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65681b4d90 con 0x558fee1bce40 2024-05-27T08:09:23.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.755+0000 7f657c02cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65681b2620 con 0x558fee3462c0 2024-05-27T08:09:23.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.755+0000 7f657c02cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65681b0490 con 0x558fee2a89f0 2024-05-27T08:09:23.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.756+0000 7f6566800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee1bce40 0x558fee337740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.756+0000 7f6566800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee1bce40 0x558fee337740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44904/0 (socket says 172.21.15.63:44904) 2024-05-27T08:09:23.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.756+0000 7f6566800700 1 -- 172.21.15.63:0/2391386804 learned_addr learned my addr 172.21.15.63:0/2391386804 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.756+0000 7f6565e00700 1 --2- 172.21.15.63:0/2391386804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee340580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.756+0000 7f6567200700 1 --2- 172.21.15.63:0/2391386804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee3462c0 0x558fee340ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.756+0000 7f6566800700 1 -- 172.21.15.63:0/2391386804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 msgr2=0x558fee340580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.756+0000 7f6566800700 1 --2- 172.21.15.63:0/2391386804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee340580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.756+0000 7f6566800700 1 -- 172.21.15.63:0/2391386804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee3462c0 msgr2=0x558fee340ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.756+0000 7f6566800700 1 --2- 172.21.15.63:0/2391386804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee3462c0 0x558fee340ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.756+0000 7f6566800700 1 -- 172.21.15.63:0/2391386804 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fee344110 con 0x558fee1bce40 2024-05-27T08:09:23.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.757+0000 7f6566800700 1 --2- 172.21.15.63:0/2391386804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee1bce40 0x558fee337740 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f65580a9110 tx=0x7f65580e7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d6847295c938a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:23.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.757+0000 7f6553400700 1 -- 172.21.15.63:0/2391386804 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558020040 con 0x558fee1bce40 2024-05-27T08:09:23.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.757+0000 7f6553400700 1 -- 172.21.15.63:0/2391386804 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65580a8f20 con 0x558fee1bce40 2024-05-27T08:09:23.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.757+0000 7f6553400700 1 -- 172.21.15.63:0/2391386804 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580e2150 con 0x558fee1bce40 2024-05-27T08:09:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.758+0000 7f657c02cb80 1 -- 172.21.15.63:0/2391386804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee1bce40 msgr2=0x558fee337740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.758+0000 7f657c02cb80 1 --2- 172.21.15.63:0/2391386804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee1bce40 0x558fee337740 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f65580a9110 tx=0x7f65580e7040 comp rx=0 tx=0).stop 2024-05-27T08:09:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.758+0000 7f657c02cb80 1 -- 172.21.15.63:0/2391386804 shutdown_connections 2024-05-27T08:09:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.758+0000 7f657c02cb80 1 --2- 172.21.15.63:0/2391386804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee3462c0 0x558fee340ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.758+0000 7f657c02cb80 1 --2- 172.21.15.63:0/2391386804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee1bce40 0x558fee337740 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.758+0000 7f657c02cb80 1 --2- 172.21.15.63:0/2391386804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee2a89f0 0x558fee340580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.758+0000 7f657c02cb80 1 -- 172.21.15.63:0/2391386804 >> 172.21.15.63:0/2391386804 conn(0x558fee256390 msgr2=0x558fee33e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:23.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.759+0000 7f657c02cb80 1 -- 172.21.15.63:0/2391386804 shutdown_connections 2024-05-27T08:09:23.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.759+0000 7f657c02cb80 1 -- 172.21.15.63:0/2391386804 wait complete. 2024-05-27T08:09:23.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.759+0000 7f657c02cb80 1 -- start start 2024-05-27T08:09:23.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.759+0000 7f657c02cb80 1 Processor -- start 2024-05-27T08:09:23.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.760+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee1bce40 0x558fee358720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.760+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee2a89f0 0x558fee358c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.760+0000 7f657c02cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee3b8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.760+0000 7f657c02cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65681b0ae0 con 0x558fee3462c0 2024-05-27T08:09:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.760+0000 7f657c02cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65681b1f50 con 0x558fee2a89f0 2024-05-27T08:09:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.760+0000 7f657c02cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65681b11b0 con 0x558fee1bce40 2024-05-27T08:09:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.760+0000 7f657c02cb80 10 client.0 _ll_register_callbacks cb 0x558fee203c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.761+0000 7f6567200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee3b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.761+0000 7f6567200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee3b8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44916/0 (socket says 172.21.15.63:44916) 2024-05-27T08:09:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.761+0000 7f6567200700 1 -- 172.21.15.63:0/419384112 learned_addr learned my addr 172.21.15.63:0/419384112 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.761+0000 7f6565e00700 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee2a89f0 0x558fee358c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.761+0000 7f6566800700 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee1bce40 0x558fee358720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.761+0000 7f6567200700 1 -- 172.21.15.63:0/419384112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee1bce40 msgr2=0x558fee358720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:23.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.761+0000 7f6567200700 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee1bce40 0x558fee358720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.761+0000 7f6567200700 1 -- 172.21.15.63:0/419384112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee2a89f0 msgr2=0x558fee358c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:23.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.761+0000 7f6567200700 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee2a89f0 0x558fee358c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.761+0000 7f6567200700 1 -- 172.21.15.63:0/419384112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fee3b8af0 con 0x558fee3462c0 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.762+0000 7f6567200700 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee3b8560 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f6560032e80 tx=0x7f65600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228a2fa1b2520fe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.762+0000 7f6550c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.762+0000 7f6550c00700 1 -- 172.21.15.63:0/419384112 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656004a060 con 0x558fee3462c0 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.762+0000 7f6550c00700 1 -- 172.21.15.63:0/419384112 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f6560005c60 con 0x558fee3462c0 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.762+0000 7f657c02cb80 1 -- 172.21.15.63:0/419384112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65580a8f20 con 0x558fee3462c0 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f6550c00700 1 -- 172.21.15.63:0/419384112 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6560051050 con 0x558fee3462c0 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f657c02cb80 10 client.22293 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f657c02cb80 1 -- 172.21.15.63:0/419384112 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558fee3bfdf0 con 0x558fee3462c0 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f6550c00700 1 -- 172.21.15.63:0/419384112 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65580a8f20 con 0x558fee3462c0 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f6550c00700 1 -- 172.21.15.63:0/419384112 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6560011070 con 0x558fee3462c0 2024-05-27T08:09:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f657c02cb80 10 client.22293 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:23.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f657c02cb80 10 client.22293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:23.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f657c02cb80 20 client.22293 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:23.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f657c02cb80 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558fee3c0380 0x558fee3c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:23.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f657c02cb80 4 client.22293 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:09:23.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.763+0000 7f657c02cb80 1 -- 172.21.15.63:0/419384112 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558fee265570 con 0x558fee3c0380 2024-05-27T08:09:23.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.764+0000 7f6566800700 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558fee3c0380 0x558fee3c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:23.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.765+0000 7f6566800700 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558fee3c0380 0x558fee3c27a0 crc :-1 s=READY pgs=641 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afdb802da80d173a server_cookie=457d109695a51ce1 in_seq=0 out_seq=0 2024-05-27T08:09:23.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.765+0000 7f6550c00700 10 client.22293 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:23.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f6550c00700 1 -- 172.21.15.63:0/419384112 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558fee265570 con 0x558fee3c0380 2024-05-27T08:09:23.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f6550c00700 10 client.22293 handle_command_reply: tid=0 2024-05-27T08:09:23.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f657c02cb80 1 client.22293 shutdown 2024-05-27T08:09:23.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f657c02cb80 1 -- 172.21.15.63:0/419384112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 msgr2=0x558fee3b8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:23.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f657c02cb80 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee3b8560 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f6560032e80 tx=0x7f65600091a0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f657c02cb80 1 -- 172.21.15.63:0/419384112 shutdown_connections 2024-05-27T08:09:23.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f657c02cb80 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558fee3c0380 0x558fee3c27a0 crc :-1 s=READY pgs=641 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f657c02cb80 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558fee3462c0 0x558fee3b8560 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f657c02cb80 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558fee1bce40 0x558fee358720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f657c02cb80 1 --2- 172.21.15.63:0/419384112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fee2a89f0 0x558fee358c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f6550c00700 0 client.22293 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:23.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.766+0000 7f657c02cb80 1 -- 172.21.15.63:0/419384112 >> 172.21.15.63:0/419384112 conn(0x558fee256390 msgr2=0x558fee351090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:23.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.767+0000 7f657c02cb80 1 -- 172.21.15.63:0/419384112 shutdown_connections 2024-05-27T08:09:23.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.767+0000 7f657c02cb80 1 -- 172.21.15.63:0/419384112 wait complete. 2024-05-27T08:09:23.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.767+0000 7f657c02cb80 20 client.22293 trim_cache size 0 max 16384 2024-05-27T08:09:23.770 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:09:23.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 -- 172.21.15.63:0/2988752321 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f654c0b2b40 msgr2=0x7f654c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:23.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f654c0b2b40 0x7f654c0b5050 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6560029000 tx=0x7f6560026000 comp rx=0 tx=0).stop 2024-05-27T08:09:23.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 -- 172.21.15.63:0/2988752321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65681052e0 msgr2=0x7f6568102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:23.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65681052e0 0x7f6568102fd0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f655800d950 tx=0x7f655800dd10 comp rx=0 tx=0).stop 2024-05-27T08:09:23.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 -- 172.21.15.63:0/2988752321 shutdown_connections 2024-05-27T08:09:23.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f654c0b2b40 0x7f654c0b5050 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f656810e710 0x7f6568103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65681052e0 0x7f6568102fd0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 --2- 172.21.15.63:0/2988752321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6568105c30 0x7f6568103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:23.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 -- 172.21.15.63:0/2988752321 >> 172.21.15.63:0/2988752321 conn(0x7f6568078af0 msgr2=0x7f65680ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:23.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 -- 172.21.15.63:0/2988752321 shutdown_connections 2024-05-27T08:09:23.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:23.768+0000 7f656ca00700 1 -- 172.21.15.63:0/2988752321 wait complete. 2024-05-27T08:09:23.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:23 smithi086 ceph-mon[34687]: pgmap v1136: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 9 op/s 2024-05-27T08:09:23.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:23 smithi149 ceph-mon[35408]: pgmap v1136: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 9 op/s 2024-05-27T08:09:25.828 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:09:25.828 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:09:25.928 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:25 smithi063 ceph-mon[31613]: pgmap v1137: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 9 op/s 2024-05-27T08:09:25.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:25 smithi086 ceph-mon[34687]: pgmap v1137: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 9 op/s 2024-05-27T08:09:25.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:25 smithi149 ceph-mon[35408]: pgmap v1137: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 9 op/s 2024-05-27T08:09:26.351 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:09:27.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:27 smithi063 ceph-mon[31613]: pgmap v1138: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-27T08:09:27.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:27 smithi086 ceph-mon[34687]: pgmap v1138: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-27T08:09:27.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:27 smithi149 ceph-mon[35408]: pgmap v1138: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 16 op/s 2024-05-27T08:09:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.642+0000 7f07af400700 1 -- 172.21.15.63:0/3277218177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a810c5a0 msgr2=0x7f07a810c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.642+0000 7f07af400700 1 --2- 172.21.15.63:0/3277218177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a810c5a0 0x7f07a810c9b0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f079c009990 tx=0x7f079c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.642+0000 7f07af400700 1 -- 172.21.15.63:0/3277218177 shutdown_connections 2024-05-27T08:09:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.642+0000 7f07af400700 1 --2- 172.21.15.63:0/3277218177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8106eb0 0x7f07a8113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.642+0000 7f07af400700 1 --2- 172.21.15.63:0/3277218177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a810c5a0 0x7f07a810c9b0 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.642+0000 7f07af400700 1 --2- 172.21.15.63:0/3277218177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07a8106560 0x7f07a8106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.642+0000 7f07af400700 1 -- 172.21.15.63:0/3277218177 >> 172.21.15.63:0/3277218177 conn(0x7f07a8102040 msgr2=0x7f07a8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:29.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.642+0000 7f07af400700 1 -- 172.21.15.63:0/3277218177 shutdown_connections 2024-05-27T08:09:29.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.642+0000 7f07af400700 1 -- 172.21.15.63:0/3277218177 wait complete. 2024-05-27T08:09:29.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.644+0000 7f07af400700 1 Processor -- start 2024-05-27T08:09:29.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.644+0000 7f07af400700 1 -- start start 2024-05-27T08:09:29.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.645+0000 7f07af400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07a8106560 0x7f07a819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.645+0000 7f07af400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8106eb0 0x7f07a819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.645+0000 7f07af400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a810c5a0 0x7f07a81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.645+0000 7f07af400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07a8116560 con 0x7f07a810c5a0 2024-05-27T08:09:29.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.645+0000 7f07af400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81163e0 con 0x7f07a8106560 2024-05-27T08:09:29.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.645+0000 7f07af400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81166e0 con 0x7f07a8106eb0 2024-05-27T08:09:29.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.646+0000 7f07aca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07a8106560 0x7f07a819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.646+0000 7f07ad400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a810c5a0 0x7f07a81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.646+0000 7f07aca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07a8106560 0x7f07a819cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56298/0 (socket says 172.21.15.63:56298) 2024-05-27T08:09:29.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.646+0000 7f07ad400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a810c5a0 0x7f07a81a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60086/0 (socket says 172.21.15.63:60086) 2024-05-27T08:09:29.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.646+0000 7f07aca00700 1 -- 172.21.15.63:0/480705387 learned_addr learned my addr 172.21.15.63:0/480705387 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:29.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.646+0000 7f07a7e00700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8106eb0 0x7f07a819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.647+0000 7f07aca00700 1 -- 172.21.15.63:0/480705387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8106eb0 msgr2=0x7f07a819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:29.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.647+0000 7f07aca00700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8106eb0 0x7f07a819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.647+0000 7f07aca00700 1 -- 172.21.15.63:0/480705387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a810c5a0 msgr2=0x7f07a81a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:29.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.647+0000 7f07aca00700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a810c5a0 0x7f07a81a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.647+0000 7f07aca00700 1 -- 172.21.15.63:0/480705387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a81a1e00 con 0x7f07a8106560 2024-05-27T08:09:29.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.647+0000 7f07ad400700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a810c5a0 0x7f07a81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:09:29.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.647+0000 7f07aca00700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07a8106560 0x7f07a819cf30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f079c000c00 tx=0x7f079c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:29.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.648+0000 7f07a5600700 1 -- 172.21.15.63:0/480705387 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c027070 con 0x7f07a8106560 2024-05-27T08:09:29.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.649+0000 7f07a5600700 1 -- 172.21.15.63:0/480705387 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f079c00fc40 con 0x7f07a8106560 2024-05-27T08:09:29.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.649+0000 7f07a5600700 1 -- 172.21.15.63:0/480705387 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c022bc0 con 0x7f07a8106560 2024-05-27T08:09:29.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.651+0000 7f07af400700 1 -- 172.21.15.63:0/480705387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a81a2090 con 0x7f07a8106560 2024-05-27T08:09:29.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.651+0000 7f07af400700 1 -- 172.21.15.63:0/480705387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a81a24f0 con 0x7f07a8106560 2024-05-27T08:09:29.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.657+0000 7f07be03fb80 1 Processor -- start 2024-05-27T08:09:29.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.657+0000 7f07be03fb80 1 -- start start 2024-05-27T08:09:29.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.658+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.658+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5d5db0 0x562faf5cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.658+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 0x562faf5d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.658+0000 7f07be03fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562faf4fdec0 con 0x562faf450ec0 2024-05-27T08:09:29.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.658+0000 7f07be03fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562faf4eda60 con 0x562faf5cfe30 2024-05-27T08:09:29.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.658+0000 7f07be03fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562faf4f1aa0 con 0x562faf5d5db0 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.658+0000 7f07a5600700 1 -- 172.21.15.63:0/480705387 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f079c00f7b0 con 0x7f07a8106560 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.658+0000 7f07ad400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.658+0000 7f07ad400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60092/0 (socket says 172.21.15.63:60092) 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.658+0000 7f07ad400700 1 -- 172.21.15.63:0/2142856619 learned_addr learned my addr 172.21.15.63:0/2142856619 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.659+0000 7f07a7e00700 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 0x562faf5d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.659+0000 7f07aca00700 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5d5db0 0x562faf5cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.659+0000 7f07ad400700 1 -- 172.21.15.63:0/2142856619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5d5db0 msgr2=0x562faf5cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.659+0000 7f07ad400700 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5d5db0 0x562faf5cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.659+0000 7f07ad400700 1 -- 172.21.15.63:0/2142856619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 msgr2=0x562faf5d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.659+0000 7f07ad400700 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 0x562faf5d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.659+0000 7f07ad400700 1 -- 172.21.15.63:0/2142856619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562faf4b64a0 con 0x562faf450ec0 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.659+0000 7f07a7e00700 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 0x562faf5d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:29.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.659+0000 7f07aca00700 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5d5db0 0x562faf5cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:29.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.660+0000 7f07ad400700 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d5870 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f07a00078d0 tx=0x7f07a0007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bad65981b48d0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:29.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.660+0000 7f07a5600700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07900b2b40 0x7f07900b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.661+0000 7f07a5600700 1 -- 172.21.15.63:0/480705387 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f079c0e37b0 con 0x7f07a8106560 2024-05-27T08:09:29.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.661+0000 7f07aca00700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07900b2b40 0x7f07900b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.661+0000 7f07aca00700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07900b2b40 0x7f07900b5050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f079c0e0bf0 tx=0x7f079c00b540 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:29.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.663+0000 7f079a000700 1 -- 172.21.15.63:0/2142856619 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a001a030 con 0x562faf450ec0 2024-05-27T08:09:29.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.663+0000 7f079a000700 1 -- 172.21.15.63:0/2142856619 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a0004bb0 con 0x562faf450ec0 2024-05-27T08:09:29.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.663+0000 7f079a000700 1 -- 172.21.15.63:0/2142856619 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0005370 con 0x562faf450ec0 2024-05-27T08:09:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.664+0000 7f07be03fb80 1 -- 172.21.15.63:0/2142856619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 msgr2=0x562faf5d5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.664+0000 7f07be03fb80 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d5870 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f07a00078d0 tx=0x7f07a0007be0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.664+0000 7f07be03fb80 1 -- 172.21.15.63:0/2142856619 shutdown_connections 2024-05-27T08:09:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.664+0000 7f07be03fb80 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 0x562faf5d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.664+0000 7f07be03fb80 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d5870 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.664+0000 7f07be03fb80 1 --2- 172.21.15.63:0/2142856619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5d5db0 0x562faf5cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.664+0000 7f07be03fb80 1 -- 172.21.15.63:0/2142856619 >> 172.21.15.63:0/2142856619 conn(0x562faf4b1040 msgr2=0x562faf5ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:29.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.665+0000 7f07be03fb80 1 -- 172.21.15.63:0/2142856619 shutdown_connections 2024-05-27T08:09:29.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.665+0000 7f07be03fb80 1 -- 172.21.15.63:0/2142856619 wait complete. 2024-05-27T08:09:29.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.666+0000 7f07be03fb80 1 -- start start 2024-05-27T08:09:29.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.666+0000 7f07be03fb80 1 Processor -- start 2024-05-27T08:09:29.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.667+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.667+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 0x562faf5d9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.667+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5d5db0 0x562faf5da100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.667+0000 7f07be03fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562faf3f4020 con 0x562faf450ec0 2024-05-27T08:09:29.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.667+0000 7f07be03fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562faf48e7e0 con 0x562faf5cfe30 2024-05-27T08:09:29.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.667+0000 7f07be03fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562faf5dc880 con 0x562faf5d5db0 2024-05-27T08:09:29.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.667+0000 7f07be03fb80 10 client.0 _ll_register_callbacks cb 0x562faf497cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.668+0000 7f07ad400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.668+0000 7f07a7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 0x562faf5d9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.668+0000 7f07ad400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60106/0 (socket says 172.21.15.63:60106) 2024-05-27T08:09:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.668+0000 7f07ad400700 1 -- 172.21.15.63:0/1515255916 learned_addr learned my addr 172.21.15.63:0/1515255916 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.669+0000 7f07ad400700 1 -- 172.21.15.63:0/1515255916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5d5db0 msgr2=0x562faf5da100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.669+0000 7f07ad400700 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5d5db0 0x562faf5da100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.669+0000 7f07ad400700 1 -- 172.21.15.63:0/1515255916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 msgr2=0x562faf5d9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.669+0000 7f07ad400700 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 0x562faf5d9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.669+0000 7f07ad400700 1 -- 172.21.15.63:0/1515255916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562faf43d950 con 0x562faf450ec0 2024-05-27T08:09:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.670+0000 7f07ad400700 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d9680 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f07a00078a0 tx=0x7f07a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9526db6cff86990b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:29.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.670+0000 7f078ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:29.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.671+0000 7f078ea00700 1 -- 172.21.15.63:0/1515255916 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0023070 con 0x562faf450ec0 2024-05-27T08:09:29.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.671+0000 7f078ea00700 1 -- 172.21.15.63:0/1515255916 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f07a0004450 con 0x562faf450ec0 2024-05-27T08:09:29.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.671+0000 7f07be03fb80 1 -- 172.21.15.63:0/1515255916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a0004bb0 con 0x562faf450ec0 2024-05-27T08:09:29.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.671+0000 7f078ea00700 1 -- 172.21.15.63:0/1515255916 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f07a0004650 con 0x562faf450ec0 2024-05-27T08:09:29.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.671+0000 7f07be03fb80 10 client.22305 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:29.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.671+0000 7f07be03fb80 1 -- 172.21.15.63:0/1515255916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562faf4be0e0 con 0x562faf450ec0 2024-05-27T08:09:29.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.672+0000 7f078ea00700 1 -- 172.21.15.63:0/1515255916 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a0004bb0 con 0x562faf450ec0 2024-05-27T08:09:29.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.672+0000 7f078ea00700 1 -- 172.21.15.63:0/1515255916 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f07a0028bf0 con 0x562faf450ec0 2024-05-27T08:09:29.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.672+0000 7f07be03fb80 10 client.22305 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:29.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.672+0000 7f07be03fb80 10 client.22305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:29.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.672+0000 7f07be03fb80 20 client.22305 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:29.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.672+0000 7f07be03fb80 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562faf5e95e0 0x562faf5eba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.672+0000 7f07be03fb80 4 client.22305 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:09:29.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.672+0000 7f07be03fb80 1 -- 172.21.15.63:0/1515255916 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562faf4ffe10 con 0x562faf5e95e0 2024-05-27T08:09:29.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.673+0000 7f07a7e00700 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562faf5e95e0 0x562faf5eba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.674+0000 7f07a7e00700 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562faf5e95e0 0x562faf5eba00 crc :-1 s=READY pgs=642 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01ecc970b9c20ad server_cookie=efae8a5399407ea0 in_seq=0 out_seq=0 2024-05-27T08:09:29.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.674+0000 7f078ea00700 10 client.22305 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:29.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f078ea00700 1 -- 172.21.15.63:0/1515255916 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562faf4ffe10 con 0x562faf5e95e0 2024-05-27T08:09:29.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f078ea00700 10 client.22305 handle_command_reply: tid=0 2024-05-27T08:09:29.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f07be03fb80 1 client.22305 shutdown 2024-05-27T08:09:29.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f07be03fb80 1 -- 172.21.15.63:0/1515255916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 msgr2=0x562faf5d9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:29.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f07be03fb80 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d9680 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f07a00078a0 tx=0x7f07a0003c10 comp rx=0 tx=0).stop 2024-05-27T08:09:29.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f07be03fb80 1 -- 172.21.15.63:0/1515255916 shutdown_connections 2024-05-27T08:09:29.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f07be03fb80 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562faf5e95e0 0x562faf5eba00 crc :-1 s=READY pgs=642 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f07be03fb80 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5d5db0 0x562faf5da100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f07be03fb80 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf450ec0 0x562faf5d9680 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f07be03fb80 1 --2- 172.21.15.63:0/1515255916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5cfe30 0x562faf5d9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f07be03fb80 1 -- 172.21.15.63:0/1515255916 >> 172.21.15.63:0/1515255916 conn(0x562faf4b1040 msgr2=0x562faf5ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:29.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.676+0000 7f078ea00700 0 client.22305 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:29.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.677+0000 7f07be03fb80 1 -- 172.21.15.63:0/1515255916 shutdown_connections 2024-05-27T08:09:29.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.677+0000 7f07be03fb80 1 -- 172.21.15.63:0/1515255916 wait complete. 2024-05-27T08:09:29.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.677+0000 7f07be03fb80 20 client.22305 trim_cache size 0 max 16384 2024-05-27T08:09:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.707+0000 7f07be03fb80 1 Processor -- start 2024-05-27T08:09:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.707+0000 7f07be03fb80 1 -- start start 2024-05-27T08:09:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.707+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf450ec0 0x562faf5cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.707+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5cfe30 0x562faf5d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.707+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5d5db0 0x562faf5eea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.707+0000 7f07be03fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81b50c0 con 0x562faf5cfe30 2024-05-27T08:09:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.707+0000 7f07be03fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81b2950 con 0x562faf5d5db0 2024-05-27T08:09:29.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.707+0000 7f07be03fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81b07c0 con 0x562faf450ec0 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07a7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5cfe30 0x562faf5d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07a7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5cfe30 0x562faf5d38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60112/0 (socket says 172.21.15.63:60112) 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07a7e00700 1 -- 172.21.15.63:0/2803436093 learned_addr learned my addr 172.21.15.63:0/2803436093 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07ad400700 1 --2- 172.21.15.63:0/2803436093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf450ec0 0x562faf5cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07ad400700 1 --2- 172.21.15.63:0/2803436093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5d5db0 0x562faf5eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07a7e00700 1 -- 172.21.15.63:0/2803436093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf450ec0 msgr2=0x562faf5cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07a7e00700 1 --2- 172.21.15.63:0/2803436093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf450ec0 0x562faf5cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07a7e00700 1 -- 172.21.15.63:0/2803436093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5d5db0 msgr2=0x562faf5eea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07a7e00700 1 --2- 172.21.15.63:0/2803436093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5d5db0 0x562faf5eea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07a7e00700 1 -- 172.21.15.63:0/2803436093 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562faf5da470 con 0x562faf5cfe30 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.708+0000 7f07ad400700 1 --2- 172.21.15.63:0/2803436093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5d5db0 0x562faf5eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.709+0000 7f07a7e00700 1 --2- 172.21.15.63:0/2803436093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5cfe30 0x562faf5d38d0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f07a81056c0 tx=0x7f079400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0adc8993fecdbf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.709+0000 7f0799600700 1 -- 172.21.15.63:0/2803436093 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794014dc0 con 0x562faf5cfe30 2024-05-27T08:09:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.709+0000 7f0799600700 1 -- 172.21.15.63:0/2803436093 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07940044c0 con 0x562faf5cfe30 2024-05-27T08:09:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.709+0000 7f0799600700 1 -- 172.21.15.63:0/2803436093 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07940055f0 con 0x562faf5cfe30 2024-05-27T08:09:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.710+0000 7f07be03fb80 1 -- 172.21.15.63:0/2803436093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5cfe30 msgr2=0x562faf5d38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:29.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.710+0000 7f07be03fb80 1 --2- 172.21.15.63:0/2803436093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5cfe30 0x562faf5d38d0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f07a81056c0 tx=0x7f079400e4b0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.710+0000 7f07be03fb80 1 -- 172.21.15.63:0/2803436093 shutdown_connections 2024-05-27T08:09:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.710+0000 7f07be03fb80 1 --2- 172.21.15.63:0/2803436093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf5d5db0 0x562faf5eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.710+0000 7f07be03fb80 1 --2- 172.21.15.63:0/2803436093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf450ec0 0x562faf5cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.710+0000 7f07be03fb80 1 --2- 172.21.15.63:0/2803436093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5cfe30 0x562faf5d38d0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.710+0000 7f07be03fb80 1 -- 172.21.15.63:0/2803436093 >> 172.21.15.63:0/2803436093 conn(0x562faf4b1040 msgr2=0x562faf5d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.711+0000 7f07be03fb80 1 -- 172.21.15.63:0/2803436093 shutdown_connections 2024-05-27T08:09:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.711+0000 7f07be03fb80 1 -- 172.21.15.63:0/2803436093 wait complete. 2024-05-27T08:09:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.711+0000 7f07be03fb80 1 -- start start 2024-05-27T08:09:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.712+0000 7f07be03fb80 1 Processor -- start 2024-05-27T08:09:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.712+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf450ec0 0x562faf5eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.712+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5cfe30 0x562faf5ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.712+0000 7f07be03fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5d5db0 0x562faf5ed920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.712+0000 7f07be03fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81b0e10 con 0x562faf5d5db0 2024-05-27T08:09:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.712+0000 7f07be03fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81b2280 con 0x562faf450ec0 2024-05-27T08:09:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.712+0000 7f07be03fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07a81b14e0 con 0x562faf5cfe30 2024-05-27T08:09:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.712+0000 7f07be03fb80 10 client.0 _ll_register_callbacks cb 0x562faf497cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.713+0000 7f07ad400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5d5db0 0x562faf5ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.713+0000 7f07ad400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5d5db0 0x562faf5ed920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60122/0 (socket says 172.21.15.63:60122) 2024-05-27T08:09:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.713+0000 7f07ad400700 1 -- 172.21.15.63:0/80818762 learned_addr learned my addr 172.21.15.63:0/80818762 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.713+0000 7f07a7e00700 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5cfe30 0x562faf5ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.713+0000 7f07ad400700 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf450ec0 0x562faf5eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.713+0000 7f07ad400700 1 -- 172.21.15.63:0/80818762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5cfe30 msgr2=0x562faf5ed3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:29.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.713+0000 7f07ad400700 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5cfe30 0x562faf5ed3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.713+0000 7f07ad400700 1 -- 172.21.15.63:0/80818762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf450ec0 msgr2=0x562faf5eef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:29.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.713+0000 7f07ad400700 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf450ec0 0x562faf5eef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.714+0000 7f07ad400700 1 -- 172.21.15.63:0/80818762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562faf5ede90 con 0x562faf5d5db0 2024-05-27T08:09:29.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.714+0000 7f07ad400700 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5d5db0 0x562faf5ed920 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009cf0 tx=0x7f07a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebb4b44f9aae1c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:29.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.714+0000 7f078ea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:29.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.714+0000 7f078ea00700 1 -- 172.21.15.63:0/80818762 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0023070 con 0x562faf5d5db0 2024-05-27T08:09:29.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.714+0000 7f078ea00700 1 -- 172.21.15.63:0/80818762 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f07a0004450 con 0x562faf5d5db0 2024-05-27T08:09:29.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.714+0000 7f07be03fb80 1 -- 172.21.15.63:0/80818762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07940044c0 con 0x562faf5d5db0 2024-05-27T08:09:29.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f078ea00700 1 -- 172.21.15.63:0/80818762 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f07a001a040 con 0x562faf5d5db0 2024-05-27T08:09:29.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f07be03fb80 10 client.22311 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:29.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f07be03fb80 1 -- 172.21.15.63:0/80818762 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562faf654b20 con 0x562faf5d5db0 2024-05-27T08:09:29.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f078ea00700 1 -- 172.21.15.63:0/80818762 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07940044c0 con 0x562faf5d5db0 2024-05-27T08:09:29.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f078ea00700 1 -- 172.21.15.63:0/80818762 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f07a0028460 con 0x562faf5d5db0 2024-05-27T08:09:29.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f07be03fb80 10 client.22311 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:29.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f07be03fb80 10 client.22311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:29.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f07be03fb80 20 client.22311 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:29.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f07be03fb80 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562faf6551a0 0x562faf6575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:29.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f07be03fb80 4 client.22311 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:09:29.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.715+0000 7f07be03fb80 1 -- 172.21.15.63:0/80818762 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562faf4f95f0 con 0x562faf6551a0 2024-05-27T08:09:29.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.716+0000 7f07a7e00700 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562faf6551a0 0x562faf6575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:29.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.717+0000 7f07a7e00700 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562faf6551a0 0x562faf6575c0 crc :-1 s=READY pgs=643 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=400015088acab634 server_cookie=2aa0e866a9705027 in_seq=0 out_seq=0 2024-05-27T08:09:29.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.717+0000 7f078ea00700 10 client.22311 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:29.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.720+0000 7f078ea00700 1 -- 172.21.15.63:0/80818762 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562faf4f95f0 con 0x562faf6551a0 2024-05-27T08:09:29.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.720+0000 7f078ea00700 10 client.22311 handle_command_reply: tid=0 2024-05-27T08:09:29.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.720+0000 7f07be03fb80 1 client.22311 shutdown 2024-05-27T08:09:29.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.720+0000 7f07be03fb80 1 -- 172.21.15.63:0/80818762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5d5db0 msgr2=0x562faf5ed920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:29.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.720+0000 7f07be03fb80 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5d5db0 0x562faf5ed920 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009cf0 tx=0x7f07a0003c10 comp rx=0 tx=0).stop 2024-05-27T08:09:29.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.721+0000 7f07be03fb80 1 -- 172.21.15.63:0/80818762 shutdown_connections 2024-05-27T08:09:29.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.721+0000 7f07be03fb80 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562faf6551a0 0x562faf6575c0 crc :-1 s=READY pgs=643 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.721+0000 7f07be03fb80 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562faf5d5db0 0x562faf5ed920 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.721+0000 7f07be03fb80 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562faf450ec0 0x562faf5eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.721+0000 7f07be03fb80 1 --2- 172.21.15.63:0/80818762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562faf5cfe30 0x562faf5ed3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.721+0000 7f07be03fb80 1 -- 172.21.15.63:0/80818762 >> 172.21.15.63:0/80818762 conn(0x562faf4b1040 msgr2=0x562faf5d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:29.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.721+0000 7f078ea00700 0 client.22311 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:29.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.721+0000 7f07be03fb80 1 -- 172.21.15.63:0/80818762 shutdown_connections 2024-05-27T08:09:29.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.721+0000 7f07be03fb80 1 -- 172.21.15.63:0/80818762 wait complete. 2024-05-27T08:09:29.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.721+0000 7f07be03fb80 20 client.22311 trim_cache size 0 max 16384 2024-05-27T08:09:29.723 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:09:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:09:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "efa4012c-e897-4b29-8ee1-735fb1cd1a26", 2024-05-27T08:09:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:09:29.723 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:09:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.722+0000 7f07af400700 1 -- 172.21.15.63:0/480705387 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07900b2b40 msgr2=0x7f07900b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.722+0000 7f07af400700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07900b2b40 0x7f07900b5050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f079c0e0bf0 tx=0x7f079c00b540 comp rx=0 tx=0).stop 2024-05-27T08:09:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.722+0000 7f07af400700 1 -- 172.21.15.63:0/480705387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07a8106560 msgr2=0x7f07a819cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.722+0000 7f07af400700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07a8106560 0x7f07a819cf30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f079c000c00 tx=0x7f079c005ef0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.722+0000 7f07af400700 1 -- 172.21.15.63:0/480705387 shutdown_connections 2024-05-27T08:09:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.722+0000 7f07af400700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f07900b2b40 0x7f07900b5050 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.722+0000 7f07af400700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07a810c5a0 0x7f07a81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.722+0000 7f07af400700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07a8106560 0x7f07a819cf30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.722+0000 7f07af400700 1 --2- 172.21.15.63:0/480705387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8106eb0 0x7f07a819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.722+0000 7f07af400700 1 -- 172.21.15.63:0/480705387 >> 172.21.15.63:0/480705387 conn(0x7f07a8102040 msgr2=0x7f07a81039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.723+0000 7f07af400700 1 -- 172.21.15.63:0/480705387 shutdown_connections 2024-05-27T08:09:29.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:29.723+0000 7f07af400700 1 -- 172.21.15.63:0/480705387 wait complete. 2024-05-27T08:09:29.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:29 smithi086 ceph-mon[34687]: pgmap v1139: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:09:29.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:29 smithi149 ceph-mon[35408]: pgmap v1139: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:09:29.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:29 smithi063 ceph-mon[31613]: pgmap v1139: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:09:30.677 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:09:30.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:09:30.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:30 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:09:30.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:09:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:30 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:09:30.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:09:30.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:09:30.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:09:30.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:09:30.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:09:30.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:09:30.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:09:30.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:30.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:30 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:09:31.217 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:09:31.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:31 smithi086 ceph-mon[34687]: pgmap v1140: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:09:31.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:31 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:09:31.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:31 smithi149 ceph-mon[35408]: pgmap v1140: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:09:31.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:31 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:09:31.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:31 smithi063 ceph-mon[31613]: pgmap v1140: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:09:31.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:31 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:09:32.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:09:32.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:09:32.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:09:32.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:09:32.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:09:32.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:09:32.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:09:32.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:09:32.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:09:33.744 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:33 smithi063 ceph-mon[31613]: pgmap v1141: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 47 op/s 2024-05-27T08:09:33.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:33 smithi149 ceph-mon[35408]: pgmap v1141: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 47 op/s 2024-05-27T08:09:34.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:33 smithi086 ceph-mon[34687]: pgmap v1141: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 47 op/s 2024-05-27T08:09:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.100+0000 7f5245800640 1 -- 172.21.15.63:0/3183231594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f524010d340 msgr2=0x7f524010f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.100+0000 7f5245800640 1 --2- 172.21.15.63:0/3183231594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f524010d340 0x7f524010f760 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f5238009ca0 tx=0x7f523802a8c0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.100+0000 7f5245800640 1 -- 172.21.15.63:0/3183231594 shutdown_connections 2024-05-27T08:09:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.100+0000 7f5245800640 1 --2- 172.21.15.63:0/3183231594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f524010d340 0x7f524010f760 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.100+0000 7f5245800640 1 --2- 172.21.15.63:0/3183231594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5240102ab0 0x7f5240102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.100+0000 7f5245800640 1 --2- 172.21.15.63:0/3183231594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52401089e0 0x7f5240102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.100+0000 7f5245800640 1 -- 172.21.15.63:0/3183231594 >> 172.21.15.63:0/3183231594 conn(0x7f52400fe210 msgr2=0x7f5240100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.100+0000 7f5245800640 1 -- 172.21.15.63:0/3183231594 shutdown_connections 2024-05-27T08:09:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.100+0000 7f5245800640 1 -- 172.21.15.63:0/3183231594 wait complete. 2024-05-27T08:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.101+0000 7f5245800640 1 Processor -- start 2024-05-27T08:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.102+0000 7f5245800640 1 -- start start 2024-05-27T08:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.102+0000 7f5245800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240102ab0 0x7f524019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.102+0000 7f5245800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52401089e0 0x7f524019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.102+0000 7f5245800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f524010d340 0x7f524019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.102+0000 7f5245800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5240112ac0 con 0x7f5240102ab0 2024-05-27T08:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.102+0000 7f5245800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5240112940 con 0x7f524010d340 2024-05-27T08:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.102+0000 7f5245800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5240112c40 con 0x7f52401089e0 2024-05-27T08:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.102+0000 7f523fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240102ab0 0x7f524019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.103+0000 7f523fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240102ab0 0x7f524019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60130/0 (socket says 172.21.15.63:60130) 2024-05-27T08:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.103+0000 7f523fe00640 1 -- 172.21.15.63:0/831091862 learned_addr learned my addr 172.21.15.63:0/831091862 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.103+0000 7f5244e00640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f524010d340 0x7f524019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.103+0000 7f523f400640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52401089e0 0x7f524019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.103+0000 7f523fe00640 1 -- 172.21.15.63:0/831091862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52401089e0 msgr2=0x7f524019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.103+0000 7f523fe00640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52401089e0 0x7f524019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.103+0000 7f523fe00640 1 -- 172.21.15.63:0/831091862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f524010d340 msgr2=0x7f524019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.103+0000 7f523fe00640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f524010d340 0x7f524019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.103+0000 7f523fe00640 1 -- 172.21.15.63:0/831091862 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524019f3b0 con 0x7f5240102ab0 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.103+0000 7f523f400640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52401089e0 0x7f524019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.104+0000 7f523fe00640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240102ab0 0x7f524019a550 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f523000e9f0 tx=0x7f523000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.104+0000 7f523cc00640 1 -- 172.21.15.63:0/831091862 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523000ccc0 con 0x7f5240102ab0 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.104+0000 7f523cc00640 1 -- 172.21.15.63:0/831091862 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52300054b0 con 0x7f5240102ab0 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.104+0000 7f5245800640 1 -- 172.21.15.63:0/831091862 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524019f6a0 con 0x7f5240102ab0 2024-05-27T08:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.104+0000 7f523cc00640 1 -- 172.21.15.63:0/831091862 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230010430 con 0x7f5240102ab0 2024-05-27T08:09:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.104+0000 7f5245800640 1 -- 172.21.15.63:0/831091862 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524019fba0 con 0x7f5240102ab0 2024-05-27T08:09:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.108+0000 7f524dda9740 1 Processor -- start 2024-05-27T08:09:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.108+0000 7f524dda9740 1 -- start start 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f524dda9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558303601b0 0x555830702a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f524dda9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303372f0 0x555830702fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f524dda9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55583033f4d0 0x5558307034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f524dda9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555830673b90 con 0x55583033f4d0 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f524dda9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555830663730 con 0x5558303601b0 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f524dda9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555830667770 con 0x5558303372f0 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f523f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55583033f4d0 0x5558307034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f523f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55583033f4d0 0x5558307034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60144/0 (socket says 172.21.15.63:60144) 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f523f400640 1 -- 172.21.15.63:0/911536000 learned_addr learned my addr 172.21.15.63:0/911536000 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f5244e00640 1 --2- 172.21.15.63:0/911536000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558303601b0 0x555830702a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.109+0000 7f523cc00640 1 -- 172.21.15.63:0/831091862 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5230004f40 con 0x7f5240102ab0 2024-05-27T08:09:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.110+0000 7f523f400640 1 -- 172.21.15.63:0/911536000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303372f0 msgr2=0x555830702fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.110+0000 7f523f400640 1 --2- 172.21.15.63:0/911536000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303372f0 0x555830702fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.110+0000 7f523f400640 1 -- 172.21.15.63:0/911536000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558303601b0 msgr2=0x555830702a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.110+0000 7f523f400640 1 --2- 172.21.15.63:0/911536000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558303601b0 0x555830702a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.110+0000 7f523f400640 1 -- 172.21.15.63:0/911536000 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558305fc8b0 con 0x55583033f4d0 2024-05-27T08:09:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.110+0000 7f523f400640 1 --2- 172.21.15.63:0/911536000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55583033f4d0 0x5558307034e0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f522c009c90 tx=0x7f522c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f00ca223dab2b76 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.111+0000 7f5235600640 1 -- 172.21.15.63:0/911536000 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c005020 con 0x55583033f4d0 2024-05-27T08:09:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.111+0000 7f5235600640 1 -- 172.21.15.63:0/911536000 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f522c0047a0 con 0x55583033f4d0 2024-05-27T08:09:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.111+0000 7f5235600640 1 -- 172.21.15.63:0/911536000 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0126b0 con 0x55583033f4d0 2024-05-27T08:09:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.112+0000 7f524dda9740 1 -- 172.21.15.63:0/911536000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55583033f4d0 msgr2=0x5558307034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.112+0000 7f524dda9740 1 --2- 172.21.15.63:0/911536000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55583033f4d0 0x5558307034e0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f522c009c90 tx=0x7f522c010740 comp rx=0 tx=0).stop 2024-05-27T08:09:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.112+0000 7f523cc00640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f52240b2950 0x7f52240b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.112+0000 7f523cc00640 1 -- 172.21.15.63:0/831091862 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f52300e2e60 con 0x7f5240102ab0 2024-05-27T08:09:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.113+0000 7f524dda9740 1 -- 172.21.15.63:0/911536000 shutdown_connections 2024-05-27T08:09:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.113+0000 7f524dda9740 1 --2- 172.21.15.63:0/911536000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55583033f4d0 0x5558307034e0 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.113+0000 7f524dda9740 1 --2- 172.21.15.63:0/911536000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303372f0 0x555830702fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.113+0000 7f524dda9740 1 --2- 172.21.15.63:0/911536000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558303601b0 0x555830702a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.113+0000 7f524dda9740 1 -- 172.21.15.63:0/911536000 >> 172.21.15.63:0/911536000 conn(0x555830354300 msgr2=0x5558306ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.113+0000 7f5244e00640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f52240b2950 0x7f52240b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.113+0000 7f524dda9740 1 -- 172.21.15.63:0/911536000 shutdown_connections 2024-05-27T08:09:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.113+0000 7f524dda9740 1 -- 172.21.15.63:0/911536000 wait complete. 2024-05-27T08:09:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.114+0000 7f524dda9740 1 -- start start 2024-05-27T08:09:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.114+0000 7f5244e00640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f52240b2950 0x7f52240b4e60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5238005e30 tx=0x7f5238005c50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.114+0000 7f524dda9740 1 Processor -- start 2024-05-27T08:09:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.115+0000 7f524dda9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x55583070cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.115+0000 7f524dda9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x55583070d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.115+0000 7f524dda9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x555830718f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.115+0000 7f524dda9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555830349790 con 0x5558303372f0 2024-05-27T08:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.115+0000 7f524dda9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55583060bbc0 con 0x55583033f4d0 2024-05-27T08:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.115+0000 7f524dda9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55583070fbe0 con 0x5558303601b0 2024-05-27T08:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.115+0000 7f524dda9740 10 client.0 _ll_register_callbacks cb 0x5558305e9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.116+0000 7f523fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x55583070cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.116+0000 7f523fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x55583070cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60152/0 (socket says 172.21.15.63:60152) 2024-05-27T08:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.116+0000 7f523fe00640 1 -- 172.21.15.63:0/2612986132 learned_addr learned my addr 172.21.15.63:0/2612986132 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.116+0000 7f523f400640 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x55583070d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.116+0000 7f5244e00640 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x555830718f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.117+0000 7f523fe00640 1 -- 172.21.15.63:0/2612986132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 msgr2=0x555830718f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.117+0000 7f523fe00640 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x555830718f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.117+0000 7f523fe00640 1 -- 172.21.15.63:0/2612986132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 msgr2=0x55583070d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.117+0000 7f523fe00640 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x55583070d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.117+0000 7f523fe00640 1 -- 172.21.15.63:0/2612986132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555830378210 con 0x5558303372f0 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.117+0000 7f523fe00640 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x55583070cef0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f52300e07f0 tx=0x7f52300a9890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cacd370da29a962 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.117+0000 7f522a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.117+0000 7f522a000640 1 -- 172.21.15.63:0/2612986132 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300ae130 con 0x5558303372f0 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.118+0000 7f522a000640 1 -- 172.21.15.63:0/2612986132 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f52300a6370 con 0x5558303372f0 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.118+0000 7f524dda9740 1 -- 172.21.15.63:0/2612986132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522c0047a0 con 0x5558303372f0 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.118+0000 7f522a000640 1 -- 172.21.15.63:0/2612986132 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f52300b2be0 con 0x5558303372f0 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.118+0000 7f524dda9740 10 client.22323 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.118+0000 7f524dda9740 1 -- 172.21.15.63:0/2612986132 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555830444620 con 0x5558303372f0 2024-05-27T08:09:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.118+0000 7f522a000640 1 -- 172.21.15.63:0/2612986132 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f522c0047a0 con 0x5558303372f0 2024-05-27T08:09:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.119+0000 7f522a000640 1 -- 172.21.15.63:0/2612986132 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f52300b2750 con 0x5558303372f0 2024-05-27T08:09:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.120+0000 7f524dda9740 10 client.22323 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.120+0000 7f524dda9740 10 client.22323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.120+0000 7f524dda9740 20 client.22323 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.120+0000 7f524dda9740 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55583031ed80 0x5558307277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.120+0000 7f524dda9740 4 client.22323 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:09:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.120+0000 7f524dda9740 1 -- 172.21.15.63:0/2612986132 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555830675ae0 con 0x55583031ed80 2024-05-27T08:09:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.121+0000 7f523f400640 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55583031ed80 0x5558307277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.122+0000 7f523f400640 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55583031ed80 0x5558307277a0 crc :-1 s=READY pgs=644 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e4fb6542d80e79 server_cookie=90bce3a17f17d131 in_seq=0 out_seq=0 2024-05-27T08:09:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.122+0000 7f522a000640 10 client.22323 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.123+0000 7f522a000640 1 -- 172.21.15.63:0/2612986132 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555830675ae0 con 0x55583031ed80 2024-05-27T08:09:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f522a000640 10 client.22323 handle_command_reply: tid=0 2024-05-27T08:09:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f524dda9740 1 client.22323 shutdown 2024-05-27T08:09:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f524dda9740 1 -- 172.21.15.63:0/2612986132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 msgr2=0x55583070cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f524dda9740 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x55583070cef0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f52300e07f0 tx=0x7f52300a9890 comp rx=0 tx=0).stop 2024-05-27T08:09:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f524dda9740 1 -- 172.21.15.63:0/2612986132 shutdown_connections 2024-05-27T08:09:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f524dda9740 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55583031ed80 0x5558307277a0 crc :-1 s=READY pgs=644 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f524dda9740 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x555830718f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f524dda9740 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x55583070d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f524dda9740 1 --2- 172.21.15.63:0/2612986132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x55583070cef0 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f522a000640 0 client.22323 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.124+0000 7f524dda9740 1 -- 172.21.15.63:0/2612986132 >> 172.21.15.63:0/2612986132 conn(0x555830354300 msgr2=0x555830704780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.125+0000 7f524dda9740 1 -- 172.21.15.63:0/2612986132 shutdown_connections 2024-05-27T08:09:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.125+0000 7f524dda9740 1 -- 172.21.15.63:0/2612986132 wait complete. 2024-05-27T08:09:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.125+0000 7f524dda9740 20 client.22323 trim_cache size 0 max 16384 2024-05-27T08:09:35.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.139+0000 7fd885c00700 1 -- 172.21.15.63:0/1797199143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd880107de0 msgr2=0x7fd8801081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.139+0000 7fd885c00700 1 --2- 172.21.15.63:0/1797199143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd880107de0 0x7fd8801081f0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0099c0 tx=0x7fd86c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.139+0000 7fd885c00700 1 -- 172.21.15.63:0/1797199143 shutdown_connections 2024-05-27T08:09:35.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.139+0000 7fd885c00700 1 --2- 172.21.15.63:0/1797199143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880108730 0x7fd880110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.139+0000 7fd885c00700 1 --2- 172.21.15.63:0/1797199143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801034b0 0x7fd8801038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.139+0000 7fd885c00700 1 --2- 172.21.15.63:0/1797199143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd880107de0 0x7fd8801081f0 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.139+0000 7fd885c00700 1 -- 172.21.15.63:0/1797199143 >> 172.21.15.63:0/1797199143 conn(0x7fd8800fedb0 msgr2=0x7fd8801011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.139+0000 7fd885c00700 1 -- 172.21.15.63:0/1797199143 shutdown_connections 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.140+0000 7fd885c00700 1 -- 172.21.15.63:0/1797199143 wait complete. 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.141+0000 7fd885c00700 1 Processor -- start 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.141+0000 7fd885c00700 1 -- start start 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd885c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801034b0 0x7fd88019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd885c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd880107de0 0x7fd88019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd885c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880108730 0x7fd8801a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd885c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd880113240 con 0x7fd880107de0 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd885c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801130c0 con 0x7fd8801034b0 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd885c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801133c0 con 0x7fd880108730 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd87e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd880107de0 0x7fd88019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd87ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801034b0 0x7fd88019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd87e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd880107de0 0x7fd88019d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60162/0 (socket says 172.21.15.63:60162) 2024-05-27T08:09:35.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd87ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801034b0 0x7fd88019d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56396/0 (socket says 172.21.15.63:56396) 2024-05-27T08:09:35.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd87ea00700 1 -- 172.21.15.63:0/4164184205 learned_addr learned my addr 172.21.15.63:0/4164184205 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:35.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.142+0000 7fd87f400700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880108730 0x7fd8801a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.143+0000 7fd87e000700 1 -- 172.21.15.63:0/4164184205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880108730 msgr2=0x7fd8801a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.143+0000 7fd87e000700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880108730 0x7fd8801a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.143+0000 7fd87e000700 1 -- 172.21.15.63:0/4164184205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801034b0 msgr2=0x7fd88019d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.143+0000 7fd87e000700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801034b0 0x7fd88019d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.143+0000 7fd87e000700 1 -- 172.21.15.63:0/4164184205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8801a1e80 con 0x7fd880107de0 2024-05-27T08:09:35.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.144+0000 7fd87e000700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd880107de0 0x7fd88019d5f0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0098e0 tx=0x7fd86c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.144+0000 7fd877400700 1 -- 172.21.15.63:0/4164184205 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c027070 con 0x7fd880107de0 2024-05-27T08:09:35.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.144+0000 7fd877400700 1 -- 172.21.15.63:0/4164184205 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86c00fc80 con 0x7fd880107de0 2024-05-27T08:09:35.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.144+0000 7fd885c00700 1 -- 172.21.15.63:0/4164184205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8801a2110 con 0x7fd880107de0 2024-05-27T08:09:35.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.144+0000 7fd877400700 1 -- 172.21.15.63:0/4164184205 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c022d90 con 0x7fd880107de0 2024-05-27T08:09:35.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.145+0000 7fd885c00700 1 -- 172.21.15.63:0/4164184205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8801a9a90 con 0x7fd880107de0 2024-05-27T08:09:35.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.149+0000 7fd877400700 1 -- 172.21.15.63:0/4164184205 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd86c02a070 con 0x7fd880107de0 2024-05-27T08:09:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.149+0000 7f524dda9740 1 Processor -- start 2024-05-27T08:09:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.150+0000 7f524dda9740 1 -- start start 2024-05-27T08:09:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.150+0000 7f524dda9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830708480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.150+0000 7f524dda9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x555830718c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.150+0000 7f524dda9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x55583072cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.150+0000 7f524dda9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f52401b1560 con 0x5558303372f0 2024-05-27T08:09:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.150+0000 7f524dda9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52401aee30 con 0x55583033f4d0 2024-05-27T08:09:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.150+0000 7f524dda9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f52401acca0 con 0x5558303601b0 2024-05-27T08:09:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.151+0000 7f523fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830708480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.151+0000 7f523fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830708480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60168/0 (socket says 172.21.15.63:60168) 2024-05-27T08:09:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.151+0000 7f523fe00640 1 -- 172.21.15.63:0/1882087460 learned_addr learned my addr 172.21.15.63:0/1882087460 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.151+0000 7f523f400640 1 --2- 172.21.15.63:0/1882087460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x555830718c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.151+0000 7f5244e00640 1 --2- 172.21.15.63:0/1882087460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x55583072cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.151+0000 7fd877400700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd8640b2a70 0x7fd8640b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.151+0000 7f523fe00640 1 -- 172.21.15.63:0/1882087460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 msgr2=0x55583072cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.151+0000 7f523fe00640 1 --2- 172.21.15.63:0/1882087460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x55583072cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.152+0000 7f523fe00640 1 -- 172.21.15.63:0/1882087460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 msgr2=0x555830718c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.152+0000 7f523fe00640 1 --2- 172.21.15.63:0/1882087460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x555830718c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.152+0000 7f523fe00640 1 -- 172.21.15.63:0/1882087460 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55583070b230 con 0x5558303372f0 2024-05-27T08:09:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.152+0000 7f5244e00640 1 --2- 172.21.15.63:0/1882087460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x55583072cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:09:35.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.151+0000 7fd877400700 1 -- 172.21.15.63:0/4164184205 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7fd86c0e36c0 con 0x7fd880107de0 2024-05-27T08:09:35.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.152+0000 7fd87ea00700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd8640b2a70 0x7fd8640b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.152+0000 7f523fe00640 1 --2- 172.21.15.63:0/1882087460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830708480 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f52300a9d10 tx=0x7f52300a99c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e400efd2630184b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.153+0000 7f5234c00640 1 -- 172.21.15.63:0/1882087460 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300ae130 con 0x5558303372f0 2024-05-27T08:09:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.153+0000 7f5234c00640 1 -- 172.21.15.63:0/1882087460 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52300a6370 con 0x5558303372f0 2024-05-27T08:09:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.153+0000 7f5234c00640 1 -- 172.21.15.63:0/1882087460 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300a58e0 con 0x5558303372f0 2024-05-27T08:09:35.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.153+0000 7fd87ea00700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd8640b2a70 0x7fd8640b4f80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd870005950 tx=0x7fd87000b500 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.154+0000 7f524dda9740 1 -- 172.21.15.63:0/1882087460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 msgr2=0x555830708480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.154+0000 7f524dda9740 1 --2- 172.21.15.63:0/1882087460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830708480 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f52300a9d10 tx=0x7f52300a99c0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.154+0000 7f524dda9740 1 -- 172.21.15.63:0/1882087460 shutdown_connections 2024-05-27T08:09:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.154+0000 7f524dda9740 1 --2- 172.21.15.63:0/1882087460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x55583072cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.154+0000 7f524dda9740 1 --2- 172.21.15.63:0/1882087460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x555830718c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.154+0000 7f524dda9740 1 --2- 172.21.15.63:0/1882087460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830708480 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.154+0000 7f524dda9740 1 -- 172.21.15.63:0/1882087460 >> 172.21.15.63:0/1882087460 conn(0x555830354300 msgr2=0x5558306fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.154+0000 7f524dda9740 1 -- 172.21.15.63:0/1882087460 shutdown_connections 2024-05-27T08:09:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.154+0000 7f524dda9740 1 -- 172.21.15.63:0/1882087460 wait complete. 2024-05-27T08:09:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.155+0000 7f524dda9740 1 -- start start 2024-05-27T08:09:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.155+0000 7f524dda9740 1 Processor -- start 2024-05-27T08:09:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.156+0000 7f524dda9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830714fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.156+0000 7f524dda9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x5558307154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.156+0000 7f524dda9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x555830710430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.156+0000 7f524dda9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f52401ad2f0 con 0x5558303372f0 2024-05-27T08:09:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.156+0000 7f524dda9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52401ae760 con 0x55583033f4d0 2024-05-27T08:09:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.156+0000 7f524dda9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f52401ad9c0 con 0x5558303601b0 2024-05-27T08:09:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.156+0000 7f524dda9740 10 client.0 _ll_register_callbacks cb 0x7f523000cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.156+0000 7f523fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830714fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f523fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830714fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60176/0 (socket says 172.21.15.63:60176) 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f523fe00640 1 -- 172.21.15.63:0/2841753220 learned_addr learned my addr 172.21.15.63:0/2841753220 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f523f400640 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x5558307154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f5244e00640 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x555830710430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f523fe00640 1 -- 172.21.15.63:0/2841753220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 msgr2=0x555830710430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f523fe00640 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x555830710430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f523fe00640 1 -- 172.21.15.63:0/2841753220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 msgr2=0x5558307154e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f523fe00640 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x5558307154e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f523fe00640 1 -- 172.21.15.63:0/2841753220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555830710af0 con 0x5558303372f0 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f5244e00640 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x555830710430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f523fe00640 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830714fa0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f52300e07f0 tx=0x7f52300096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff24d9219bd7ace server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.157+0000 7f522a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.158+0000 7f522a000640 1 -- 172.21.15.63:0/2841753220 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230088020 con 0x5558303372f0 2024-05-27T08:09:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.158+0000 7f522a000640 1 -- 172.21.15.63:0/2841753220 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7f52300a8fe0 con 0x5558303372f0 2024-05-27T08:09:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.158+0000 7f524dda9740 1 -- 172.21.15.63:0/2841753220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52300a6370 con 0x5558303372f0 2024-05-27T08:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.159+0000 7f522a000640 1 -- 172.21.15.63:0/2841753220 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f52300a56f0 con 0x5558303372f0 2024-05-27T08:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.159+0000 7f524dda9740 10 client.22335 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.159+0000 7f524dda9740 1 -- 172.21.15.63:0/2841753220 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5558307a0790 con 0x5558303372f0 2024-05-27T08:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.159+0000 7f522a000640 1 -- 172.21.15.63:0/2841753220 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52300a6370 con 0x5558303372f0 2024-05-27T08:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.159+0000 7f522a000640 1 -- 172.21.15.63:0/2841753220 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f52300af2c0 con 0x5558303372f0 2024-05-27T08:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.159+0000 7f524dda9740 10 client.22335 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.159+0000 7f524dda9740 10 client.22335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.159+0000 7f524dda9740 20 client.22335 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.160+0000 7f524dda9740 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558302f41c0 0x5558307a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.160+0000 7f524dda9740 4 client.22335 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.160+0000 7f524dda9740 1 -- 172.21.15.63:0/2841753220 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555830707260 con 0x5558302f41c0 2024-05-27T08:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.160+0000 7f523f400640 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558302f41c0 0x5558307a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.161+0000 7f523f400640 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558302f41c0 0x5558307a2c70 crc :-1 s=READY pgs=645 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ca42f224bc97b server_cookie=fa581d82f49663cb in_seq=0 out_seq=0 2024-05-27T08:09:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.161+0000 7f522a000640 10 client.22335 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:35.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.165+0000 7fd8947d9b80 1 Processor -- start 2024-05-27T08:09:35.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.165+0000 7fd8947d9b80 1 -- start start 2024-05-27T08:09:35.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.166+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1afde40 0x562bd1c8b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.166+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 0x562bd1c85080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.166+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c855c0 0x562bd1c859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.166+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562bd1baae40 con 0x562bd1c855c0 2024-05-27T08:09:35.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.166+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562bd1b9a9e0 con 0x562bd1afde40 2024-05-27T08:09:35.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.166+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562bd1b9ea20 con 0x562bd1c8b540 2024-05-27T08:09:35.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.167+0000 7fd87f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c855c0 0x562bd1c859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.167+0000 7fd87f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c855c0 0x562bd1c859d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60188/0 (socket says 172.21.15.63:60188) 2024-05-27T08:09:35.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.167+0000 7fd87f400700 1 -- 172.21.15.63:0/4105519591 learned_addr learned my addr 172.21.15.63:0/4105519591 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:35.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.167+0000 7fd87e000700 1 --2- 172.21.15.63:0/4105519591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 0x562bd1c85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.167+0000 7fd87ea00700 1 --2- 172.21.15.63:0/4105519591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1afde40 0x562bd1c8b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.167+0000 7fd87f400700 1 -- 172.21.15.63:0/4105519591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 msgr2=0x562bd1c85080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.167+0000 7fd87f400700 1 --2- 172.21.15.63:0/4105519591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 0x562bd1c85080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.167+0000 7fd87f400700 1 -- 172.21.15.63:0/4105519591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1afde40 msgr2=0x562bd1c8b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.167+0000 7fd87f400700 1 --2- 172.21.15.63:0/4105519591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1afde40 0x562bd1c8b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.167+0000 7fd87f400700 1 -- 172.21.15.63:0/4105519591 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bd1b63420 con 0x562bd1c855c0 2024-05-27T08:09:35.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.168+0000 7fd87f400700 1 --2- 172.21.15.63:0/4105519591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c855c0 0x562bd1c859d0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7fd878003fa0 tx=0x7fd878009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f26a51d27abefc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.169+0000 7fd86be00700 1 -- 172.21.15.63:0/4105519591 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878004bb0 con 0x562bd1c855c0 2024-05-27T08:09:35.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.169+0000 7fd86be00700 1 -- 172.21.15.63:0/4105519591 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8780048e0 con 0x562bd1c855c0 2024-05-27T08:09:35.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.169+0000 7fd86be00700 1 -- 172.21.15.63:0/4105519591 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878007d20 con 0x562bd1c855c0 2024-05-27T08:09:35.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.171+0000 7fd8947d9b80 1 -- 172.21.15.63:0/4105519591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c855c0 msgr2=0x562bd1c859d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.171+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/4105519591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c855c0 0x562bd1c859d0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7fd878003fa0 tx=0x7fd878009ba0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.171+0000 7fd8947d9b80 1 -- 172.21.15.63:0/4105519591 shutdown_connections 2024-05-27T08:09:35.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.171+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/4105519591 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c855c0 0x562bd1c859d0 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.171+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/4105519591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1afde40 0x562bd1c8b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.171+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/4105519591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 0x562bd1c85080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.171+0000 7fd8947d9b80 1 -- 172.21.15.63:0/4105519591 >> 172.21.15.63:0/4105519591 conn(0x562bd1b5dfc0 msgr2=0x562bd1c778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.171+0000 7fd8947d9b80 1 -- 172.21.15.63:0/4105519591 shutdown_connections 2024-05-27T08:09:35.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.171+0000 7fd8947d9b80 1 -- 172.21.15.63:0/4105519591 wait complete. 2024-05-27T08:09:35.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.172+0000 7fd8947d9b80 1 -- start start 2024-05-27T08:09:35.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.173+0000 7fd8947d9b80 1 Processor -- start 2024-05-27T08:09:35.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.173+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1afde40 0x562bd1c8ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.173+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c8f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.173+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c8f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.173+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562bd1aa0fa0 con 0x562bd1c8b540 2024-05-27T08:09:35.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.173+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562bd1b3b760 con 0x562bd1c855c0 2024-05-27T08:09:35.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.173+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562bd1c92010 con 0x562bd1afde40 2024-05-27T08:09:35.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.173+0000 7fd8947d9b80 10 client.0 _ll_register_callbacks cb 0x562bd1b44c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:35.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.174+0000 7fd87ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1afde40 0x562bd1c8ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.174+0000 7fd87f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c8f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.174+0000 7fd87ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1afde40 0x562bd1c8ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:60498/0 (socket says 172.21.15.63:60498) 2024-05-27T08:09:35.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.174+0000 7fd87ea00700 1 -- 172.21.15.63:0/2295144780 learned_addr learned my addr 172.21.15.63:0/2295144780 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:35.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.174+0000 7fd87e000700 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c8f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.175+0000 7fd87ea00700 1 -- 172.21.15.63:0/2295144780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 msgr2=0x562bd1c8f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.175+0000 7fd87ea00700 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c8f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.175+0000 7fd87ea00700 1 -- 172.21.15.63:0/2295144780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 msgr2=0x562bd1c8f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.175+0000 7fd87ea00700 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c8f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.175+0000 7fd87ea00700 1 -- 172.21.15.63:0/2295144780 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bd1aea8d0 con 0x562bd1afde40 2024-05-27T08:09:35.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.175+0000 7fd87f400700 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c8f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:09:35.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.177+0000 7fd87ea00700 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1afde40 0x562bd1c8ed50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd8700116a0 tx=0x7fd8700047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f04369e5f2ba941 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.177+0000 7fd868c00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:09:35.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.179+0000 7fd868c00700 1 -- 172.21.15.63:0/2295144780 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870004230 con 0x562bd1afde40 2024-05-27T08:09:35.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.179+0000 7fd868c00700 1 -- 172.21.15.63:0/2295144780 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7fd8700043d0 con 0x562bd1afde40 2024-05-27T08:09:35.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.179+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2295144780 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8780048e0 con 0x562bd1afde40 2024-05-27T08:09:35.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.181+0000 7fd868c00700 1 -- 172.21.15.63:0/2295144780 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd870030050 con 0x562bd1afde40 2024-05-27T08:09:35.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.181+0000 7fd8947d9b80 10 client.31135 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:35.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.181+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2295144780 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562bd1b6b060 con 0x562bd1afde40 2024-05-27T08:09:35.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.182+0000 7fd868c00700 1 -- 172.21.15.63:0/2295144780 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8780048e0 con 0x562bd1afde40 2024-05-27T08:09:35.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.183+0000 7fd868c00700 1 -- 172.21.15.63:0/2295144780 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd87001f430 con 0x562bd1afde40 2024-05-27T08:09:35.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.183+0000 7fd8947d9b80 10 client.31135 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:35.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.183+0000 7fd8947d9b80 10 client.31135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:35.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.183+0000 7fd8947d9b80 20 client.31135 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:35.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.183+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562bd1c9efb0 0x562bd1ca1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.183+0000 7fd8947d9b80 4 client.31135 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:09:35.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.183+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2295144780 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bd1bacd90 con 0x562bd1c9efb0 2024-05-27T08:09:35.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.184+0000 7fd87e000700 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562bd1c9efb0 0x562bd1ca1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.185+0000 7fd87e000700 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562bd1c9efb0 0x562bd1ca1450 crc :-1 s=READY pgs=646 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41514d7d7c55459 server_cookie=2269dda21f049d6d in_seq=0 out_seq=0 2024-05-27T08:09:35.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.185+0000 7fd868c00700 10 client.31135 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.188+0000 7fd868c00700 1 -- 172.21.15.63:0/2295144780 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562bd1bacd90 con 0x562bd1c9efb0 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.188+0000 7fd868c00700 10 client.31135 handle_command_reply: tid=0 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.188+0000 7fd8947d9b80 1 client.31135 shutdown 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.189+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2295144780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1afde40 msgr2=0x562bd1c8ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.189+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1afde40 0x562bd1c8ed50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd8700116a0 tx=0x7fd8700047c0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.189+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2295144780 shutdown_connections 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.189+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562bd1c9efb0 0x562bd1ca1450 crc :-1 s=READY pgs=646 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.189+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c8f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.189+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1afde40 0x562bd1c8ed50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.189+0000 7fd868c00700 0 client.31135 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.189+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2295144780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c8f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.189+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2295144780 >> 172.21.15.63:0/2295144780 conn(0x562bd1b5dfc0 msgr2=0x562bd1c77580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.190+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2295144780 shutdown_connections 2024-05-27T08:09:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.190+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2295144780 wait complete. 2024-05-27T08:09:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.190+0000 7fd8947d9b80 20 client.31135 trim_cache size 0 max 16384 2024-05-27T08:09:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.198+0000 7f522a000640 1 -- 172.21.15.63:0/2841753220 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555830707260 con 0x5558302f41c0 2024-05-27T08:09:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.198+0000 7f522a000640 10 client.22335 handle_command_reply: tid=0 2024-05-27T08:09:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f524dda9740 1 client.22335 shutdown 2024-05-27T08:09:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f524dda9740 1 -- 172.21.15.63:0/2841753220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 msgr2=0x555830714fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f524dda9740 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830714fa0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f52300e07f0 tx=0x7f52300096c0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f524dda9740 1 -- 172.21.15.63:0/2841753220 shutdown_connections 2024-05-27T08:09:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f524dda9740 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5558302f41c0 0x5558307a2c70 crc :-1 s=READY pgs=645 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f524dda9740 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558303601b0 0x555830710430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f524dda9740 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55583033f4d0 0x5558307154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f524dda9740 1 --2- 172.21.15.63:0/2841753220 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558303372f0 0x555830714fa0 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f524dda9740 1 -- 172.21.15.63:0/2841753220 >> 172.21.15.63:0/2841753220 conn(0x555830354300 msgr2=0x5558306fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f522a000640 0 client.22335 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.199+0000 7f524dda9740 1 -- 172.21.15.63:0/2841753220 shutdown_connections 2024-05-27T08:09:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.200+0000 7f524dda9740 1 -- 172.21.15.63:0/2841753220 wait complete. 2024-05-27T08:09:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.200+0000 7f524dda9740 20 client.22335 trim_cache size 0 max 16384 2024-05-27T08:09:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:09:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:09:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:09:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.201+0000 7f5245800640 1 -- 172.21.15.63:0/831091862 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f52240b2950 msgr2=0x7f52240b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.201+0000 7f5245800640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f52240b2950 0x7f52240b4e60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5238005e30 tx=0x7f5238005c50 comp rx=0 tx=0).stop 2024-05-27T08:09:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.201+0000 7f5245800640 1 -- 172.21.15.63:0/831091862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240102ab0 msgr2=0x7f524019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.201+0000 7f5245800640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240102ab0 0x7f524019a550 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f523000e9f0 tx=0x7f523000eeb0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.201+0000 7f5245800640 1 -- 172.21.15.63:0/831091862 shutdown_connections 2024-05-27T08:09:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.201+0000 7f5245800640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f52240b2950 0x7f52240b4e60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.201+0000 7f5245800640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f524010d340 0x7f524019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.201+0000 7f5245800640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52401089e0 0x7f524019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.201+0000 7f5245800640 1 --2- 172.21.15.63:0/831091862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240102ab0 0x7f524019a550 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.201+0000 7f5245800640 1 -- 172.21.15.63:0/831091862 >> 172.21.15.63:0/831091862 conn(0x7f52400fe210 msgr2=0x7f524006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.202+0000 7f5245800640 1 -- 172.21.15.63:0/831091862 shutdown_connections 2024-05-27T08:09:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:35.202+0000 7f5245800640 1 -- 172.21.15.63:0/831091862 wait complete. 2024-05-27T08:09:35.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.227+0000 7fd8947d9b80 1 Processor -- start 2024-05-27T08:09:35.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.227+0000 7fd8947d9b80 1 -- start start 2024-05-27T08:09:35.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.227+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1afde40 0x562bd1c85080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.227+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c9b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.227+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 0x562bd1c8f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.227+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801b0a90 con 0x562bd1afde40 2024-05-27T08:09:35.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.227+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801ae320 con 0x562bd1c855c0 2024-05-27T08:09:35.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.227+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801ac190 con 0x562bd1c8b540 2024-05-27T08:09:35.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.228+0000 7fd87ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1afde40 0x562bd1c85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.228+0000 7fd87ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1afde40 0x562bd1c85080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60210/0 (socket says 172.21.15.63:60210) 2024-05-27T08:09:35.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.228+0000 7fd87ea00700 1 -- 172.21.15.63:0/3661368290 learned_addr learned my addr 172.21.15.63:0/3661368290 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.228+0000 7fd87e000700 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c9b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.229+0000 7fd87f400700 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 0x562bd1c8f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.229+0000 7fd87ea00700 1 -- 172.21.15.63:0/3661368290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 msgr2=0x562bd1c8f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.229+0000 7fd87ea00700 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 0x562bd1c8f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.229+0000 7fd87ea00700 1 -- 172.21.15.63:0/3661368290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 msgr2=0x562bd1c9b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.229+0000 7fd87ea00700 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c9b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.229+0000 7fd87ea00700 1 -- 172.21.15.63:0/3661368290 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bd1c91ef0 con 0x562bd1afde40 2024-05-27T08:09:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.229+0000 7fd87e000700 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c9b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.229+0000 7fd87f400700 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 0x562bd1c8f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.229+0000 7fd87ea00700 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1afde40 0x562bd1c85080 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fd87000b8d0 tx=0x7fd870000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488ee4e71871b0cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.230+0000 7fd86b400700 1 -- 172.21.15.63:0/3661368290 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870038070 con 0x562bd1afde40 2024-05-27T08:09:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.230+0000 7fd86b400700 1 -- 172.21.15.63:0/3661368290 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd870023030 con 0x562bd1afde40 2024-05-27T08:09:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.230+0000 7fd86b400700 1 -- 172.21.15.63:0/3661368290 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870003a00 con 0x562bd1afde40 2024-05-27T08:09:35.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.232+0000 7fd8947d9b80 1 -- 172.21.15.63:0/3661368290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1afde40 msgr2=0x562bd1c85080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.232+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1afde40 0x562bd1c85080 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fd87000b8d0 tx=0x7fd870000bc0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.232+0000 7fd8947d9b80 1 -- 172.21.15.63:0/3661368290 shutdown_connections 2024-05-27T08:09:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.232+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c8b540 0x562bd1c8f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.232+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1afde40 0x562bd1c85080 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.232+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/3661368290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c9b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.232+0000 7fd8947d9b80 1 -- 172.21.15.63:0/3661368290 >> 172.21.15.63:0/3661368290 conn(0x562bd1b5dfc0 msgr2=0x562bd1c8eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.232+0000 7fd8947d9b80 1 -- 172.21.15.63:0/3661368290 shutdown_connections 2024-05-27T08:09:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.233+0000 7fd8947d9b80 1 -- 172.21.15.63:0/3661368290 wait complete. 2024-05-27T08:09:35.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.233+0000 7fd8947d9b80 1 -- start start 2024-05-27T08:09:35.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.234+0000 7fd8947d9b80 1 Processor -- start 2024-05-27T08:09:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.234+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c974a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.234+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c979e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.234+0000 7fd8947d9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c94270 0x562bd1c94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.234+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801ac7e0 con 0x562bd1c8b540 2024-05-27T08:09:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.234+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801adc50 con 0x562bd1c855c0 2024-05-27T08:09:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.234+0000 7fd8947d9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801aceb0 con 0x562bd1c94270 2024-05-27T08:09:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.234+0000 7fd8947d9b80 10 client.0 _ll_register_callbacks cb 0x562bd1b44c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:35.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.235+0000 7fd87e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.235+0000 7fd87ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.235+0000 7fd87ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56470/0 (socket says 172.21.15.63:56470) 2024-05-27T08:09:35.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.235+0000 7fd87e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60226/0 (socket says 172.21.15.63:60226) 2024-05-27T08:09:35.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.235+0000 7fd87ea00700 1 -- 172.21.15.63:0/2519668093 learned_addr learned my addr 172.21.15.63:0/2519668093 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:35.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.235+0000 7fd87f400700 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c94270 0x562bd1c94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.236+0000 7fd87e000700 1 -- 172.21.15.63:0/2519668093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c94270 msgr2=0x562bd1c94680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.236+0000 7fd87e000700 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c94270 0x562bd1c94680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.236+0000 7fd87e000700 1 -- 172.21.15.63:0/2519668093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 msgr2=0x562bd1c974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.236+0000 7fd87e000700 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.236+0000 7fd87e000700 1 -- 172.21.15.63:0/2519668093 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bd1c94e20 con 0x562bd1c8b540 2024-05-27T08:09:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.236+0000 7fd87ea00700 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:09:35.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.237+0000 7fd87e000700 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c979e0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0e09f0 tx=0x7fd86c0922e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d1e53814f94be4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:35.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.237+0000 7fd868c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:35.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.237+0000 7fd868c00700 1 -- 172.21.15.63:0/2519668093 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0aad10 con 0x562bd1c8b540 2024-05-27T08:09:35.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.237+0000 7fd868c00700 1 -- 172.21.15.63:0/2519668093 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7fd86c022590 con 0x562bd1c8b540 2024-05-27T08:09:35.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.237+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2519668093 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd870023030 con 0x562bd1c8b540 2024-05-27T08:09:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.238+0000 7fd868c00700 1 -- 172.21.15.63:0/2519668093 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd86c0aa610 con 0x562bd1c8b540 2024-05-27T08:09:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.238+0000 7fd8947d9b80 10 client.22347 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.238+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2519668093 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562bd1d0a840 con 0x562bd1c8b540 2024-05-27T08:09:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.238+0000 7fd868c00700 1 -- 172.21.15.63:0/2519668093 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd870023030 con 0x562bd1c8b540 2024-05-27T08:09:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.239+0000 7fd868c00700 1 -- 172.21.15.63:0/2519668093 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd86c0b2f50 con 0x562bd1c8b540 2024-05-27T08:09:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.239+0000 7fd8947d9b80 10 client.22347 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.239+0000 7fd8947d9b80 10 client.22347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.239+0000 7fd8947d9b80 20 client.22347 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.239+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562bd1d0ae50 0x562bd1d0d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.239+0000 7fd8947d9b80 4 client.22347 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:09:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.239+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2519668093 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562bd1ba6570 con 0x562bd1d0ae50 2024-05-27T08:09:35.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.240+0000 7fd87ea00700 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562bd1d0ae50 0x562bd1d0d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:35.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.241+0000 7fd868c00700 10 client.22347 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:35.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.241+0000 7fd87ea00700 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562bd1d0ae50 0x562bd1d0d270 crc :-1 s=READY pgs=647 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ea64d03b58e47d server_cookie=c01c1dc3eb5efa8e in_seq=0 out_seq=0 2024-05-27T08:09:35.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd868c00700 1 -- 172.21.15.63:0/2519668093 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562bd1ba6570 con 0x562bd1d0ae50 2024-05-27T08:09:35.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd868c00700 10 client.22347 handle_command_reply: tid=0 2024-05-27T08:09:35.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd8947d9b80 1 client.22347 shutdown 2024-05-27T08:09:35.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2519668093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 msgr2=0x562bd1c979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c979e0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0e09f0 tx=0x7fd86c0922e0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2519668093 shutdown_connections 2024-05-27T08:09:35.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562bd1d0ae50 0x562bd1d0d270 crc :-1 s=READY pgs=647 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562bd1c94270 0x562bd1c94680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562bd1c855c0 0x562bd1c974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd8947d9b80 1 --2- 172.21.15.63:0/2519668093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562bd1c8b540 0x562bd1c979e0 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2519668093 >> 172.21.15.63:0/2519668093 conn(0x562bd1b5dfc0 msgr2=0x562bd1c8a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.242+0000 7fd868c00700 0 client.22347 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:35.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.243+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2519668093 shutdown_connections 2024-05-27T08:09:35.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.243+0000 7fd8947d9b80 1 -- 172.21.15.63:0/2519668093 wait complete. 2024-05-27T08:09:35.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.243+0000 7fd8947d9b80 20 client.22347 trim_cache size 0 max 16384 2024-05-27T08:09:35.245 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:09:35.245 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:09:35.245 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:09:35.245 INFO:teuthology.orchestra.run.smithi063.stdout: "efa4012c-e897-4b29-8ee1-735fb1cd1a26": { 2024-05-27T08:09:35.245 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:09:35.245 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "efa4012c-e897-4b29-8ee1-735fb1cd1a26", 2024-05-27T08:09:35.245 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:09:35.245 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:09:35.245 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:09:35.245 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:09:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 -- 172.21.15.63:0/4164184205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd8640b2a70 msgr2=0x7fd8640b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd8640b2a70 0x7fd8640b4f80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd870005950 tx=0x7fd87000b500 comp rx=0 tx=0).stop 2024-05-27T08:09:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 -- 172.21.15.63:0/4164184205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd880107de0 msgr2=0x7fd88019d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd880107de0 0x7fd88019d5f0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0098e0 tx=0x7fd86c0057c0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 -- 172.21.15.63:0/4164184205 shutdown_connections 2024-05-27T08:09:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd8640b2a70 0x7fd8640b4f80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880108730 0x7fd8801a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8801034b0 0x7fd88019d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 --2- 172.21.15.63:0/4164184205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd880107de0 0x7fd88019d5f0 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 -- 172.21.15.63:0/4164184205 >> 172.21.15.63:0/4164184205 conn(0x7fd8800fedb0 msgr2=0x7fd8801066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:35.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 -- 172.21.15.63:0/4164184205 shutdown_connections 2024-05-27T08:09:35.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:09:35.244+0000 7fd885c00700 1 -- 172.21.15.63:0/4164184205 wait complete. 2024-05-27T08:09:35.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:35 smithi063 ceph-mon[31613]: pgmap v1142: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 44 op/s 2024-05-27T08:09:36.125 INFO:tasks.cephfs.filesystem:scrub status for tag:efa4012c-e897-4b29-8ee1-735fb1cd1a26 - {'path': '/', 'tag': 'efa4012c-e897-4b29-8ee1-735fb1cd1a26', 'options': 'recursive,force'} 2024-05-27T08:09:36.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:35 smithi086 ceph-mon[34687]: pgmap v1142: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 44 op/s 2024-05-27T08:09:36.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:35 smithi149 ceph-mon[35408]: pgmap v1142: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 44 op/s 2024-05-27T08:09:37.152 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:36 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:09:37.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:36 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:09:37.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:36 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:09:38.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:37 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:09:38.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:37 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:09:38.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:37 smithi086 ceph-mon[34687]: pgmap v1143: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.4 MiB/s wr, 51 op/s 2024-05-27T08:09:38.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:37 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:09:38.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:37 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:09:38.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:37 smithi149 ceph-mon[35408]: pgmap v1143: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.4 MiB/s wr, 51 op/s 2024-05-27T08:09:38.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:37 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:09:38.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:37 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:09:38.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:37 smithi063 ceph-mon[31613]: pgmap v1143: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.4 MiB/s wr, 51 op/s 2024-05-27T08:09:40.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:39 smithi086 ceph-mon[34687]: pgmap v1144: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.7 MiB/s wr, 44 op/s 2024-05-27T08:09:40.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:39 smithi149 ceph-mon[35408]: pgmap v1144: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.7 MiB/s wr, 44 op/s 2024-05-27T08:09:40.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:39 smithi063 ceph-mon[31613]: pgmap v1144: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.7 MiB/s wr, 44 op/s 2024-05-27T08:09:42.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:41 smithi086 ceph-mon[34687]: pgmap v1145: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.7 MiB/s wr, 44 op/s 2024-05-27T08:09:42.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:41 smithi149 ceph-mon[35408]: pgmap v1145: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.7 MiB/s wr, 44 op/s 2024-05-27T08:09:42.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:41 smithi063 ceph-mon[31613]: pgmap v1145: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.7 MiB/s wr, 44 op/s 2024-05-27T08:09:44.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:43 smithi086 ceph-mon[34687]: pgmap v1146: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 48 op/s 2024-05-27T08:09:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:43 smithi149 ceph-mon[35408]: pgmap v1146: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 48 op/s 2024-05-27T08:09:44.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:43 smithi063 ceph-mon[31613]: pgmap v1146: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 48 op/s 2024-05-27T08:09:46.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:45 smithi086 ceph-mon[34687]: pgmap v1147: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:09:46.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:45 smithi149 ceph-mon[35408]: pgmap v1147: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:09:46.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:45 smithi063 ceph-mon[31613]: pgmap v1147: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:09:48.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:47 smithi086 ceph-mon[34687]: pgmap v1148: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T08:09:48.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:47 smithi149 ceph-mon[35408]: pgmap v1148: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T08:09:48.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:47 smithi063 ceph-mon[31613]: pgmap v1148: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.7 MiB/s wr, 14 op/s 2024-05-27T08:09:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.464+0000 7fa63a000640 1 -- 172.21.15.63:0/2126122266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa63410a120 msgr2=0x7fa634103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.464+0000 7fa63a000640 1 --2- 172.21.15.63:0/2126122266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa63410a120 0x7fa634103d60 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7fa620009900 tx=0x7fa62002a410 comp rx=0 tx=0).stop 2024-05-27T08:09:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.464+0000 7fa63a000640 1 -- 172.21.15.63:0/2126122266 shutdown_connections 2024-05-27T08:09:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.464+0000 7fa63a000640 1 --2- 172.21.15.63:0/2126122266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6341043d0 0x7fa6341107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.464+0000 7fa63a000640 1 --2- 172.21.15.63:0/2126122266 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa63410a120 0x7fa634103d60 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.464+0000 7fa63a000640 1 --2- 172.21.15.63:0/2126122266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa634109c80 0x7fa634103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.464+0000 7fa63a000640 1 -- 172.21.15.63:0/2126122266 >> 172.21.15.63:0/2126122266 conn(0x7fa634079290 msgr2=0x7fa6340796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.464+0000 7fa63a000640 1 -- 172.21.15.63:0/2126122266 shutdown_connections 2024-05-27T08:09:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.464+0000 7fa63a000640 1 -- 172.21.15.63:0/2126122266 wait complete. 2024-05-27T08:09:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.466+0000 7fa63a000640 1 Processor -- start 2024-05-27T08:09:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.466+0000 7fa63a000640 1 -- start start 2024-05-27T08:09:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa63a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6341043d0 0x7fa63419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa63a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa634109c80 0x7fa63419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa63a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa63410a120 0x7fa6341a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa63a000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa634113d20 con 0x7fa6341043d0 2024-05-27T08:09:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa63a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa634113ba0 con 0x7fa63410a120 2024-05-27T08:09:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa63a000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa634113ea0 con 0x7fa634109c80 2024-05-27T08:09:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa633400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6341043d0 0x7fa63419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa633400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6341043d0 0x7fa63419d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42510/0 (socket says 172.21.15.63:42510) 2024-05-27T08:09:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa633400640 1 -- 172.21.15.63:0/2681111071 learned_addr learned my addr 172.21.15.63:0/2681111071 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa632a00640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa634109c80 0x7fa63419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.467+0000 7fa633e00640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa63410a120 0x7fa6341a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa633400640 1 -- 172.21.15.63:0/2681111071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa634109c80 msgr2=0x7fa63419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa633400640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa634109c80 0x7fa63419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa633400640 1 -- 172.21.15.63:0/2681111071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa63410a120 msgr2=0x7fa6341a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa633400640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa63410a120 0x7fa6341a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa633400640 1 -- 172.21.15.63:0/2681111071 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341a8bc0 con 0x7fa6341043d0 2024-05-27T08:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa632a00640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa634109c80 0x7fa63419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa633e00640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa63410a120 0x7fa6341a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa633400640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6341043d0 0x7fa63419d7e0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fa62400d740 tx=0x7fa62400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa62be00640 1 -- 172.21.15.63:0/2681111071 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624017070 con 0x7fa6341043d0 2024-05-27T08:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa62be00640 1 -- 172.21.15.63:0/2681111071 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa62400de50 con 0x7fa6341043d0 2024-05-27T08:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa63a000640 1 -- 172.21.15.63:0/2681111071 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341a8eb0 con 0x7fa6341043d0 2024-05-27T08:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.468+0000 7fa62be00640 1 -- 172.21.15.63:0/2681111071 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624004c60 con 0x7fa6341043d0 2024-05-27T08:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.469+0000 7fa63a000640 1 -- 172.21.15.63:0/2681111071 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341a9320 con 0x7fa6341043d0 2024-05-27T08:09:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.473+0000 7fa64117e740 1 Processor -- start 2024-05-27T08:09:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.473+0000 7fa62be00640 1 -- 172.21.15.63:0/2681111071 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa624004e00 con 0x7fa6341043d0 2024-05-27T08:09:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.473+0000 7fa64117e740 1 -- start start 2024-05-27T08:09:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.473+0000 7fa64117e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 0x55a04cd87860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.473+0000 7fa64117e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9bc2f0 0x55a04cd87da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.473+0000 7fa64117e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 0x55a04cd882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.474+0000 7fa64117e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a04ccf8b90 con 0x55a04c9bc2f0 2024-05-27T08:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.474+0000 7fa64117e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a04cce8730 con 0x55a04c9e51b0 2024-05-27T08:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.474+0000 7fa64117e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a04ccec770 con 0x55a04c9c44d0 2024-05-27T08:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.474+0000 7fa633400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9bc2f0 0x55a04cd87da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.474+0000 7fa633400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9bc2f0 0x55a04cd87da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42516/0 (socket says 172.21.15.63:42516) 2024-05-27T08:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.474+0000 7fa633400640 1 -- 172.21.15.63:0/987155827 learned_addr learned my addr 172.21.15.63:0/987155827 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.474+0000 7fa633e00640 1 --2- 172.21.15.63:0/987155827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 0x55a04cd87860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.474+0000 7fa632a00640 1 --2- 172.21.15.63:0/987155827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 0x55a04cd882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.475+0000 7fa633400640 1 -- 172.21.15.63:0/987155827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 msgr2=0x55a04cd882e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.475+0000 7fa633400640 1 --2- 172.21.15.63:0/987155827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 0x55a04cd882e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.475+0000 7fa633400640 1 -- 172.21.15.63:0/987155827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 msgr2=0x55a04cd87860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.475+0000 7fa633400640 1 --2- 172.21.15.63:0/987155827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 0x55a04cd87860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.475+0000 7fa633400640 1 -- 172.21.15.63:0/987155827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04cc818b0 con 0x55a04c9bc2f0 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.475+0000 7fa632a00640 1 --2- 172.21.15.63:0/987155827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 0x55a04cd882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.475+0000 7fa62be00640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6180b29d0 0x7fa6180b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.475+0000 7fa633400640 1 --2- 172.21.15.63:0/987155827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9bc2f0 0x55a04cd87da0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7fa6240df660 tx=0x7fa6240e8ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2e2b46c57e3615 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.475+0000 7fa633e00640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6180b29d0 0x7fa6180b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.476+0000 7fa62be00640 1 -- 172.21.15.63:0/2681111071 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7fa6240e2330 con 0x7fa6341043d0 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.476+0000 7fa61fe00640 1 -- 172.21.15.63:0/987155827 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6240a5110 con 0x55a04c9bc2f0 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.476+0000 7fa61fe00640 1 -- 172.21.15.63:0/987155827 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6240a8d60 con 0x55a04c9bc2f0 2024-05-27T08:09:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.476+0000 7fa61fe00640 1 -- 172.21.15.63:0/987155827 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6240a89c0 con 0x55a04c9bc2f0 2024-05-27T08:09:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.477+0000 7fa64117e740 1 -- 172.21.15.63:0/987155827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9bc2f0 msgr2=0x55a04cd87da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.477+0000 7fa64117e740 1 --2- 172.21.15.63:0/987155827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9bc2f0 0x55a04cd87da0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7fa6240df660 tx=0x7fa6240e8ab0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.477+0000 7fa633e00640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6180b29d0 0x7fa6180b4ee0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55a04cd849d0 tx=0x7fa62c009700 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.477+0000 7fa64117e740 1 -- 172.21.15.63:0/987155827 shutdown_connections 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.477+0000 7fa64117e740 1 --2- 172.21.15.63:0/987155827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 0x55a04cd882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.477+0000 7fa64117e740 1 --2- 172.21.15.63:0/987155827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9bc2f0 0x55a04cd87da0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.477+0000 7fa64117e740 1 --2- 172.21.15.63:0/987155827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 0x55a04cd87860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.477+0000 7fa64117e740 1 -- 172.21.15.63:0/987155827 >> 172.21.15.63:0/987155827 conn(0x55a04c9d9300 msgr2=0x55a04cd84010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.477+0000 7fa64117e740 1 -- 172.21.15.63:0/987155827 shutdown_connections 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.477+0000 7fa64117e740 1 -- 172.21.15.63:0/987155827 wait complete. 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.478+0000 7fa64117e740 1 -- start start 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.478+0000 7fa64117e740 1 Processor -- start 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa64117e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9bc2f0 0x55a04cd91a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa64117e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd91f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa64117e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 0x55a04cd9dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa64117e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a04c9ce790 con 0x55a04c9c44d0 2024-05-27T08:09:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa64117e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a04cc90bc0 con 0x55a04c9e51b0 2024-05-27T08:09:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa64117e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a04cd94990 con 0x55a04c9bc2f0 2024-05-27T08:09:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa64117e740 10 client.0 _ll_register_callbacks cb 0x55a04cc6e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa632a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd91f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa632a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd91f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42528/0 (socket says 172.21.15.63:42528) 2024-05-27T08:09:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa632a00640 1 -- 172.21.15.63:0/3018032593 learned_addr learned my addr 172.21.15.63:0/3018032593 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa633400640 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9bc2f0 0x55a04cd91a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa633e00640 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 0x55a04cd9dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa632a00640 1 -- 172.21.15.63:0/3018032593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9bc2f0 msgr2=0x55a04cd91a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa632a00640 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9bc2f0 0x55a04cd91a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa632a00640 1 -- 172.21.15.63:0/3018032593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 msgr2=0x55a04cd9dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa632a00640 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 0x55a04cd9dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.479+0000 7fa632a00640 1 -- 172.21.15.63:0/3018032593 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04c9fd210 con 0x55a04c9c44d0 2024-05-27T08:09:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.480+0000 7fa633e00640 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 0x55a04cd9dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:09:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.480+0000 7fa632a00640 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd91f60 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7fa620005ab0 tx=0x7fa620031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59406b56438daca7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.480+0000 7fa61cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.480+0000 7fa61cc00640 1 -- 172.21.15.63:0/3018032593 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62003f070 con 0x55a04c9c44d0 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.480+0000 7fa64117e740 1 -- 172.21.15.63:0/3018032593 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6240a8d60 con 0x55a04c9c44d0 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.480+0000 7fa61cc00640 1 -- 172.21.15.63:0/3018032593 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7fa620004250 con 0x55a04c9c44d0 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.481+0000 7fa61cc00640 1 -- 172.21.15.63:0/3018032593 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa62004c050 con 0x55a04c9c44d0 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.481+0000 7fa64117e740 10 client.22359 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.481+0000 7fa64117e740 1 -- 172.21.15.63:0/3018032593 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a04cac9620 con 0x55a04c9c44d0 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.481+0000 7fa61cc00640 1 -- 172.21.15.63:0/3018032593 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6240a8d60 con 0x55a04c9c44d0 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.481+0000 7fa61cc00640 1 -- 172.21.15.63:0/3018032593 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa620004a40 con 0x55a04c9c44d0 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.481+0000 7fa64117e740 10 client.22359 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.481+0000 7fa64117e740 10 client.22359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.481+0000 7fa64117e740 20 client.22359 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.481+0000 7fa64117e740 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a04c9a3d80 0x55a04cda4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.481+0000 7fa64117e740 4 client.22359 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:09:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.482+0000 7fa64117e740 1 -- 172.21.15.63:0/3018032593 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a04ccfaae0 con 0x55a04c9a3d80 2024-05-27T08:09:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.482+0000 7fa633400640 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a04c9a3d80 0x55a04cda4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.483+0000 7fa633400640 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a04c9a3d80 0x55a04cda4070 crc :-1 s=READY pgs=648 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9e4a194fb04e28 server_cookie=75436e9b671a52a in_seq=0 out_seq=0 2024-05-27T08:09:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.483+0000 7fa61cc00640 10 client.22359 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa61cc00640 1 -- 172.21.15.63:0/3018032593 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a04ccfaae0 con 0x55a04c9a3d80 2024-05-27T08:09:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa61cc00640 10 client.22359 handle_command_reply: tid=0 2024-05-27T08:09:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa64117e740 1 client.22359 shutdown 2024-05-27T08:09:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa64117e740 1 -- 172.21.15.63:0/3018032593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 msgr2=0x55a04cd91f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa64117e740 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd91f60 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7fa620005ab0 tx=0x7fa620031630 comp rx=0 tx=0).stop 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa64117e740 1 -- 172.21.15.63:0/3018032593 shutdown_connections 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa64117e740 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a04c9a3d80 0x55a04cda4070 crc :-1 s=READY pgs=648 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa64117e740 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9e51b0 0x55a04cd9dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa64117e740 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd91f60 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa64117e740 1 --2- 172.21.15.63:0/3018032593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9bc2f0 0x55a04cd91a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa64117e740 1 -- 172.21.15.63:0/3018032593 >> 172.21.15.63:0/3018032593 conn(0x55a04c9d9300 msgr2=0x55a04cd87690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.485+0000 7fa61cc00640 0 client.22359 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.486+0000 7fa64117e740 1 -- 172.21.15.63:0/3018032593 shutdown_connections 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.486+0000 7fa64117e740 1 -- 172.21.15.63:0/3018032593 wait complete. 2024-05-27T08:09:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.486+0000 7fa64117e740 20 client.22359 trim_cache size 0 max 16384 2024-05-27T08:09:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.503+0000 7fa64117e740 1 Processor -- start 2024-05-27T08:09:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.503+0000 7fa64117e740 1 -- start start 2024-05-27T08:09:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.503+0000 7fa64117e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 0x55a04cd8d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.503+0000 7fa64117e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 0x55a04cd87d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.503+0000 7fa64117e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9e51b0 0x55a04cd83330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.503+0000 7fa64117e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341b5d90 con 0x55a04c9e51b0 2024-05-27T08:09:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.503+0000 7fa64117e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341b3660 con 0x55a04c9bc2f0 2024-05-27T08:09:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.503+0000 7fa64117e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341b14d0 con 0x55a04c9c44d0 2024-05-27T08:09:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.504+0000 7fa633e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9e51b0 0x55a04cd83330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.504+0000 7fa633e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9e51b0 0x55a04cd83330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42534/0 (socket says 172.21.15.63:42534) 2024-05-27T08:09:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.504+0000 7fa633e00640 1 -- 172.21.15.63:0/2685218008 learned_addr learned my addr 172.21.15.63:0/2685218008 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.504+0000 7fa633400640 1 --2- 172.21.15.63:0/2685218008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 0x55a04cd8d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.504+0000 7fa632a00640 1 --2- 172.21.15.63:0/2685218008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 0x55a04cd87d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.504+0000 7fa633e00640 1 -- 172.21.15.63:0/2685218008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 msgr2=0x55a04cd87d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.504+0000 7fa633e00640 1 --2- 172.21.15.63:0/2685218008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 0x55a04cd87d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.504+0000 7fa633e00640 1 -- 172.21.15.63:0/2685218008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 msgr2=0x55a04cd8d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.504+0000 7fa633e00640 1 --2- 172.21.15.63:0/2685218008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 0x55a04cd8d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.504+0000 7fa633e00640 1 -- 172.21.15.63:0/2685218008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04c9a3fb0 con 0x55a04c9e51b0 2024-05-27T08:09:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.505+0000 7fa633e00640 1 --2- 172.21.15.63:0/2685218008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9e51b0 0x55a04cd83330 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0106b0 tx=0x7fa62c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b3cd0109e3795f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.505+0000 7fa61f400640 1 -- 172.21.15.63:0/2685218008 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00f780 con 0x55a04c9e51b0 2024-05-27T08:09:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.505+0000 7fa61f400640 1 -- 172.21.15.63:0/2685218008 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa62c023070 con 0x55a04c9e51b0 2024-05-27T08:09:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.505+0000 7fa61f400640 1 -- 172.21.15.63:0/2685218008 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c028d60 con 0x55a04c9e51b0 2024-05-27T08:09:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.506+0000 7fa64117e740 1 -- 172.21.15.63:0/2685218008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9e51b0 msgr2=0x55a04cd83330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.506+0000 7fa64117e740 1 --2- 172.21.15.63:0/2685218008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9e51b0 0x55a04cd83330 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0106b0 tx=0x7fa62c015000 comp rx=0 tx=0).stop 2024-05-27T08:09:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.506+0000 7fa64117e740 1 -- 172.21.15.63:0/2685218008 shutdown_connections 2024-05-27T08:09:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.506+0000 7fa64117e740 1 --2- 172.21.15.63:0/2685218008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9e51b0 0x55a04cd83330 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.506+0000 7fa64117e740 1 --2- 172.21.15.63:0/2685218008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9c44d0 0x55a04cd87d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.506+0000 7fa64117e740 1 --2- 172.21.15.63:0/2685218008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 0x55a04cd8d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.506+0000 7fa64117e740 1 -- 172.21.15.63:0/2685218008 >> 172.21.15.63:0/2685218008 conn(0x55a04c9d9300 msgr2=0x55a04cd875f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.507+0000 7fa64117e740 1 -- 172.21.15.63:0/2685218008 shutdown_connections 2024-05-27T08:09:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.507+0000 7fa64117e740 1 -- 172.21.15.63:0/2685218008 wait complete. 2024-05-27T08:09:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.507+0000 7fa64117e740 1 -- start start 2024-05-27T08:09:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.507+0000 7fa64117e740 1 Processor -- start 2024-05-27T08:09:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.508+0000 7fa64117e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 0x55a04cd9a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.508+0000 7fa64117e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd94c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.508+0000 7fa64117e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9e51b0 0x55a04cd95150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.508+0000 7fa64117e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341b1b20 con 0x55a04c9c44d0 2024-05-27T08:09:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.508+0000 7fa64117e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341b2f90 con 0x55a04c9bc2f0 2024-05-27T08:09:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.508+0000 7fa64117e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341b21f0 con 0x55a04c9e51b0 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.508+0000 7fa64117e740 10 client.0 _ll_register_callbacks cb 0x7fa620009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.508+0000 7fa632a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd94c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.508+0000 7fa632a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd94c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42538/0 (socket says 172.21.15.63:42538) 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.508+0000 7fa632a00640 1 -- 172.21.15.63:0/1738405312 learned_addr learned my addr 172.21.15.63:0/1738405312 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.509+0000 7fa633400640 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 0x55a04cd9a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.509+0000 7fa633e00640 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9e51b0 0x55a04cd95150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.509+0000 7fa632a00640 1 -- 172.21.15.63:0/1738405312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9e51b0 msgr2=0x55a04cd95150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.509+0000 7fa632a00640 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9e51b0 0x55a04cd95150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.509+0000 7fa632a00640 1 -- 172.21.15.63:0/1738405312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 msgr2=0x55a04cd9a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.509+0000 7fa632a00640 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 0x55a04cd9a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.509+0000 7fa632a00640 1 -- 172.21.15.63:0/1738405312 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04cd95710 con 0x55a04c9c44d0 2024-05-27T08:09:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.509+0000 7fa633400640 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 0x55a04cd9a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.509+0000 7fa632a00640 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd94c10 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fa6200057c0 tx=0x7fa620031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5b192d0639b8e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.509+0000 7fa61cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.510+0000 7fa61cc00640 1 -- 172.21.15.63:0/1738405312 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62003f070 con 0x55a04c9c44d0 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.510+0000 7fa61cc00640 1 -- 172.21.15.63:0/1738405312 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7fa6200046c0 con 0x55a04c9c44d0 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.510+0000 7fa64117e740 1 -- 172.21.15.63:0/1738405312 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa62c023070 con 0x55a04c9c44d0 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.510+0000 7fa61cc00640 1 -- 172.21.15.63:0/1738405312 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa620061050 con 0x55a04c9c44d0 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.510+0000 7fa64117e740 10 client.22365 fetch_fsmap learned FSMap version 35 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.510+0000 7fa64117e740 1 -- 172.21.15.63:0/1738405312 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a04ce0b4b0 con 0x55a04c9c44d0 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.510+0000 7fa61cc00640 1 -- 172.21.15.63:0/1738405312 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa62c023070 con 0x55a04c9c44d0 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.511+0000 7fa61cc00640 1 -- 172.21.15.63:0/1738405312 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa6200035f0 con 0x55a04c9c44d0 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.511+0000 7fa64117e740 10 client.22365 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.511+0000 7fa64117e740 10 client.22365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.511+0000 7fa64117e740 20 client.22365 populate_metadata read hostname 'smithi063' 2024-05-27T08:09:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.511+0000 7fa64117e740 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a04c9791c0 0x55a04ce0dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:09:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.511+0000 7fa64117e740 4 client.22365 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:09:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.511+0000 7fa64117e740 1 -- 172.21.15.63:0/1738405312 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a04ccf42c0 con 0x55a04c9791c0 2024-05-27T08:09:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.511+0000 7fa633400640 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a04c9791c0 0x55a04ce0dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:09:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.513+0000 7fa633400640 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a04c9791c0 0x55a04ce0dbe0 crc :-1 s=READY pgs=649 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870a8353ea404858 server_cookie=121771d4ba073ac8 in_seq=0 out_seq=0 2024-05-27T08:09:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.513+0000 7fa61cc00640 10 client.22365 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.548+0000 7fa61cc00640 1 -- 172.21.15.63:0/1738405312 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a04ccf42c0 con 0x55a04c9791c0 2024-05-27T08:09:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.548+0000 7fa61cc00640 10 client.22365 handle_command_reply: tid=0 2024-05-27T08:09:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.548+0000 7fa64117e740 1 client.22365 shutdown 2024-05-27T08:09:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 1 -- 172.21.15.63:0/1738405312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 msgr2=0x55a04cd94c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd94c10 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fa6200057c0 tx=0x7fa620031930 comp rx=0 tx=0).stop 2024-05-27T08:09:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 1 -- 172.21.15.63:0/1738405312 shutdown_connections 2024-05-27T08:09:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a04c9791c0 0x55a04ce0dbe0 crc :-1 s=READY pgs=649 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a04c9e51b0 0x55a04cd95150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a04c9c44d0 0x55a04cd94c10 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 1 --2- 172.21.15.63:0/1738405312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a04c9bc2f0 0x55a04cd9a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 1 -- 172.21.15.63:0/1738405312 >> 172.21.15.63:0/1738405312 conn(0x55a04c9d9300 msgr2=0x55a04cd93020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa61cc00640 0 client.22365 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:09:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 1 -- 172.21.15.63:0/1738405312 shutdown_connections 2024-05-27T08:09:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 1 -- 172.21.15.63:0/1738405312 wait complete. 2024-05-27T08:09:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.549+0000 7fa64117e740 20 client.22365 trim_cache size 0 max 16384 2024-05-27T08:09:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:09:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:09:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:09:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.550+0000 7fa63a000640 1 -- 172.21.15.63:0/2681111071 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6180b29d0 msgr2=0x7fa6180b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.550+0000 7fa63a000640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6180b29d0 0x7fa6180b4ee0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55a04cd849d0 tx=0x7fa62c009700 comp rx=0 tx=0).stop 2024-05-27T08:09:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.550+0000 7fa63a000640 1 -- 172.21.15.63:0/2681111071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6341043d0 msgr2=0x7fa63419d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:09:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.550+0000 7fa63a000640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6341043d0 0x7fa63419d7e0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fa62400d740 tx=0x7fa62400dc00 comp rx=0 tx=0).stop 2024-05-27T08:09:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.550+0000 7fa63a000640 1 -- 172.21.15.63:0/2681111071 shutdown_connections 2024-05-27T08:09:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.550+0000 7fa63a000640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6180b29d0 0x7fa6180b4ee0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.550+0000 7fa63a000640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa63410a120 0x7fa6341a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.550+0000 7fa63a000640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa634109c80 0x7fa63419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.550+0000 7fa63a000640 1 --2- 172.21.15.63:0/2681111071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6341043d0 0x7fa63419d7e0 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:09:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.550+0000 7fa63a000640 1 -- 172.21.15.63:0/2681111071 >> 172.21.15.63:0/2681111071 conn(0x7fa634079290 msgr2=0x7fa6341003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:09:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.551+0000 7fa63a000640 1 -- 172.21.15.63:0/2681111071 shutdown_connections 2024-05-27T08:09:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:09:48.551+0000 7fa63a000640 1 -- 172.21.15.63:0/2681111071 wait complete. 2024-05-27T08:09:50.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:49 smithi086 ceph-mon[34687]: pgmap v1149: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:09:50.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:49 smithi149 ceph-mon[35408]: pgmap v1149: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:09:50.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:49 smithi063 ceph-mon[31613]: pgmap v1149: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:09:52.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:51 smithi086 ceph-mon[34687]: pgmap v1150: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:09:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:51 smithi149 ceph-mon[35408]: pgmap v1150: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:09:52.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:51 smithi063 ceph-mon[31613]: pgmap v1150: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 MiB/s wr, 6 op/s 2024-05-27T08:09:54.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:53 smithi086 ceph-mon[34687]: pgmap v1151: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:09:54.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:53 smithi149 ceph-mon[35408]: pgmap v1151: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:09:54.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:53 smithi063 ceph-mon[31613]: pgmap v1151: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 13 op/s 2024-05-27T08:09:55.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:54 smithi086 ceph-mon[34687]: pgmap v1152: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:09:55.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:54 smithi149 ceph-mon[35408]: pgmap v1152: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:09:55.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:54 smithi063 ceph-mon[31613]: pgmap v1152: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:09:57.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:57 smithi086 ceph-mon[34687]: pgmap v1153: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 14 op/s 2024-05-27T08:09:57.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:57 smithi149 ceph-mon[35408]: pgmap v1153: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 14 op/s 2024-05-27T08:09:57.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:57 smithi063 ceph-mon[31613]: pgmap v1153: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.4 MiB/s wr, 14 op/s 2024-05-27T08:09:59.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:09:59 smithi086 ceph-mon[34687]: pgmap v1154: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:09:59.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:09:59 smithi149 ceph-mon[35408]: pgmap v1154: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:09:59.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:09:59 smithi063 ceph-mon[31613]: pgmap v1154: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:10:00.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:00 smithi086 ceph-mon[34687]: overall HEALTH_OK 2024-05-27T08:10:00.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:00 smithi149 ceph-mon[35408]: overall HEALTH_OK 2024-05-27T08:10:00.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:00 smithi063 ceph-mon[31613]: overall HEALTH_OK 2024-05-27T08:10:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.882+0000 7fb52d000640 1 -- 172.21.15.63:0/2858714888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528075400 msgr2=0x7fb528075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.882+0000 7fb52d000640 1 --2- 172.21.15.63:0/2858714888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528075400 0x7fb528075810 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fb518009900 tx=0x7fb51802a480 comp rx=0 tx=0).stop 2024-05-27T08:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.882+0000 7fb52d000640 1 -- 172.21.15.63:0/2858714888 shutdown_connections 2024-05-27T08:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.882+0000 7fb52d000640 1 --2- 172.21.15.63:0/2858714888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb528110340 0x7fb528112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.882+0000 7fb52d000640 1 --2- 172.21.15.63:0/2858714888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb52810d9e0 0x7fb52810fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.882+0000 7fb52d000640 1 --2- 172.21.15.63:0/2858714888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528075400 0x7fb528075810 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.882+0000 7fb52d000640 1 -- 172.21.15.63:0/2858714888 >> 172.21.15.63:0/2858714888 conn(0x7fb528079100 msgr2=0x7fb528079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.882+0000 7fb52d000640 1 -- 172.21.15.63:0/2858714888 shutdown_connections 2024-05-27T08:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.883+0000 7fb52d000640 1 -- 172.21.15.63:0/2858714888 wait complete. 2024-05-27T08:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.884+0000 7fb52d000640 1 Processor -- start 2024-05-27T08:10:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.884+0000 7fb52d000640 1 -- start start 2024-05-27T08:10:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.885+0000 7fb52d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528075400 0x7fb5281a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.885+0000 7fb52d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52810d9e0 0x7fb5281a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.885+0000 7fb52d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528110340 0x7fb52819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.885+0000 7fb52d000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb528115db0 con 0x7fb528110340 2024-05-27T08:10:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.885+0000 7fb52d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb528115c30 con 0x7fb52810d9e0 2024-05-27T08:10:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.885+0000 7fb52d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb528115f30 con 0x7fb528075400 2024-05-27T08:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.885+0000 7fb527e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528110340 0x7fb52819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.885+0000 7fb527e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528110340 0x7fb52819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34068/0 (socket says 172.21.15.63:34068) 2024-05-27T08:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.886+0000 7fb527e00640 1 -- 172.21.15.63:0/3745560157 learned_addr learned my addr 172.21.15.63:0/3745560157 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.886+0000 7fb527400640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528075400 0x7fb5281a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.886+0000 7fb526a00640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52810d9e0 0x7fb5281a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.886+0000 7fb527e00640 1 -- 172.21.15.63:0/3745560157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528075400 msgr2=0x7fb5281a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.886+0000 7fb527e00640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528075400 0x7fb5281a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.886+0000 7fb527e00640 1 -- 172.21.15.63:0/3745560157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52810d9e0 msgr2=0x7fb5281a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.886+0000 7fb527e00640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52810d9e0 0x7fb5281a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.886+0000 7fb527e00640 1 -- 172.21.15.63:0/3745560157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52819cdc0 con 0x7fb528110340 2024-05-27T08:10:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.887+0000 7fb527e00640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528110340 0x7fb52819c690 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7fb520009fd0 tx=0x7fb52000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.887+0000 7fb51fe00640 1 -- 172.21.15.63:0/3745560157 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520017070 con 0x7fb528110340 2024-05-27T08:10:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.887+0000 7fb51fe00640 1 -- 172.21.15.63:0/3745560157 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb520003e00 con 0x7fb528110340 2024-05-27T08:10:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.887+0000 7fb52d000640 1 -- 172.21.15.63:0/3745560157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52819d050 con 0x7fb528110340 2024-05-27T08:10:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.887+0000 7fb51fe00640 1 -- 172.21.15.63:0/3745560157 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520010460 con 0x7fb528110340 2024-05-27T08:10:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.887+0000 7fb52d000640 1 -- 172.21.15.63:0/3745560157 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5281a92b0 con 0x7fb528110340 2024-05-27T08:10:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.892+0000 7fb535621740 1 Processor -- start 2024-05-27T08:10:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.892+0000 7fb51fe00640 1 -- 172.21.15.63:0/3745560157 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb520003970 con 0x7fb528110340 2024-05-27T08:10:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.892+0000 7fb535621740 1 -- start start 2024-05-27T08:10:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb535621740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f8c1b0 0x5590b132ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb535621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f632f0 0x5590b132efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb535621740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b132f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb535621740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590b129fb90 con 0x5590b0f6b4d0 2024-05-27T08:10:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb535621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5590b128f730 con 0x5590b0f632f0 2024-05-27T08:10:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb535621740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5590b1293770 con 0x5590b0f8c1b0 2024-05-27T08:10:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b132f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b132f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34074/0 (socket says 172.21.15.63:34074) 2024-05-27T08:10:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb526a00640 1 -- 172.21.15.63:0/2613930614 learned_addr learned my addr 172.21.15.63:0/2613930614 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb527400640 1 --2- 172.21.15.63:0/2613930614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f632f0 0x5590b132efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.893+0000 7fb526a00640 1 --2- 172.21.15.63:0/2613930614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f8c1b0 0x5590b132ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.894+0000 7fb526a00640 1 -- 172.21.15.63:0/2613930614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f8c1b0 msgr2=0x5590b132ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.894+0000 7fb526a00640 1 --2- 172.21.15.63:0/2613930614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f8c1b0 0x5590b132ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.894+0000 7fb526a00640 1 -- 172.21.15.63:0/2613930614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f632f0 msgr2=0x5590b132efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.894+0000 7fb526a00640 1 --2- 172.21.15.63:0/2613930614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f632f0 0x5590b132efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.894+0000 7fb526a00640 1 -- 172.21.15.63:0/2613930614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b12288b0 con 0x5590b0f6b4d0 2024-05-27T08:10:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.894+0000 7fb51fe00640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b2950 0x7fb50c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.894+0000 7fb51fe00640 1 -- 172.21.15.63:0/3745560157 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7fb5200e2a80 con 0x7fb528110340 2024-05-27T08:10:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.894+0000 7fb527400640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b2950 0x7fb50c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.895+0000 7fb526a00640 1 --2- 172.21.15.63:0/2613930614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b132f4e0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x5590b132bbd0 tx=0x7fb510005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da29fdea8d15d58e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.895+0000 7fb51cc00640 1 -- 172.21.15.63:0/2613930614 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510005020 con 0x5590b0f6b4d0 2024-05-27T08:10:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.895+0000 7fb51cc00640 1 -- 172.21.15.63:0/2613930614 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5100047a0 con 0x5590b0f6b4d0 2024-05-27T08:10:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.895+0000 7fb51cc00640 1 -- 172.21.15.63:0/2613930614 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51000f9f0 con 0x5590b0f6b4d0 2024-05-27T08:10:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.896+0000 7fb535621740 1 -- 172.21.15.63:0/2613930614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 msgr2=0x5590b132f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.896+0000 7fb527400640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b2950 0x7fb50c0b4e60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb518006010 tx=0x7fb518000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.896+0000 7fb535621740 1 --2- 172.21.15.63:0/2613930614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b132f4e0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x5590b132bbd0 tx=0x7fb510005850 comp rx=0 tx=0).stop 2024-05-27T08:10:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.896+0000 7fb535621740 1 -- 172.21.15.63:0/2613930614 shutdown_connections 2024-05-27T08:10:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.896+0000 7fb535621740 1 --2- 172.21.15.63:0/2613930614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b132f4e0 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.896+0000 7fb535621740 1 --2- 172.21.15.63:0/2613930614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f632f0 0x5590b132efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.896+0000 7fb535621740 1 --2- 172.21.15.63:0/2613930614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f8c1b0 0x5590b132ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.896+0000 7fb535621740 1 -- 172.21.15.63:0/2613930614 >> 172.21.15.63:0/2613930614 conn(0x5590b0f80300 msgr2=0x5590b132b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.897+0000 7fb535621740 1 -- 172.21.15.63:0/2613930614 shutdown_connections 2024-05-27T08:10:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.897+0000 7fb535621740 1 -- 172.21.15.63:0/2613930614 wait complete. 2024-05-27T08:10:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.897+0000 7fb535621740 1 -- start start 2024-05-27T08:10:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.898+0000 7fb535621740 1 Processor -- start 2024-05-27T08:10:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.898+0000 7fb535621740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f632f0 0x5590b1338ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.898+0000 7fb535621740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b1339400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.898+0000 7fb535621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b1344ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb535621740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590b0f75790 con 0x5590b0f6b4d0 2024-05-27T08:10:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb535621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5590b1237bc0 con 0x5590b0f8c1b0 2024-05-27T08:10:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb535621740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5590b133bb90 con 0x5590b0f632f0 2024-05-27T08:10:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb535621740 10 client.0 _ll_register_callbacks cb 0x5590b1215400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b1339400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b1339400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34078/0 (socket says 172.21.15.63:34078) 2024-05-27T08:10:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb526a00640 1 -- 172.21.15.63:0/1020639908 learned_addr learned my addr 172.21.15.63:0/1020639908 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb527e00640 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b1344ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb527400640 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f632f0 0x5590b1338ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb526a00640 1 -- 172.21.15.63:0/1020639908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f632f0 msgr2=0x5590b1338ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb526a00640 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f632f0 0x5590b1338ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb526a00640 1 -- 172.21.15.63:0/1020639908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 msgr2=0x5590b1344ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.899+0000 7fb526a00640 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b1344ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.900+0000 7fb526a00640 1 -- 172.21.15.63:0/1020639908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b0fa4210 con 0x5590b0f6b4d0 2024-05-27T08:10:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.900+0000 7fb526a00640 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b1339400 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fb510005bc0 tx=0x7fb510004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9471b63d19224871 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.900+0000 7fb515600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.900+0000 7fb515600640 1 -- 172.21.15.63:0/1020639908 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100053f0 con 0x5590b0f6b4d0 2024-05-27T08:10:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.900+0000 7fb515600640 1 -- 172.21.15.63:0/1020639908 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7fb510004020 con 0x5590b0f6b4d0 2024-05-27T08:10:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.900+0000 7fb535621740 1 -- 172.21.15.63:0/1020639908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5100047a0 con 0x5590b0f6b4d0 2024-05-27T08:10:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.901+0000 7fb515600640 1 -- 172.21.15.63:0/1020639908 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb510032050 con 0x5590b0f6b4d0 2024-05-27T08:10:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.901+0000 7fb535621740 10 client.22377 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.901+0000 7fb535621740 1 -- 172.21.15.63:0/1020639908 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5590b1070620 con 0x5590b0f6b4d0 2024-05-27T08:10:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.901+0000 7fb515600640 1 -- 172.21.15.63:0/1020639908 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5100047a0 con 0x5590b0f6b4d0 2024-05-27T08:10:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.901+0000 7fb515600640 1 -- 172.21.15.63:0/1020639908 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb51001d030 con 0x5590b0f6b4d0 2024-05-27T08:10:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.901+0000 7fb535621740 10 client.22377 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.901+0000 7fb535621740 10 client.22377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.902+0000 7fb535621740 20 client.22377 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.902+0000 7fb535621740 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590b0f4ad80 0x5590b134adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.902+0000 7fb535621740 4 client.22377 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.902+0000 7fb535621740 1 -- 172.21.15.63:0/1020639908 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590b12a1ae0 con 0x5590b0f4ad80 2024-05-27T08:10:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.903+0000 7fb527400640 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590b0f4ad80 0x5590b134adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.904+0000 7fb527400640 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590b0f4ad80 0x5590b134adb0 crc :-1 s=READY pgs=650 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b84c0afc2fbdbe2 server_cookie=93a4fb481a48ddcd in_seq=0 out_seq=0 2024-05-27T08:10:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.904+0000 7fb515600640 10 client.22377 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb515600640 1 -- 172.21.15.63:0/1020639908 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5590b12a1ae0 con 0x5590b0f4ad80 2024-05-27T08:10:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb515600640 10 client.22377 handle_command_reply: tid=0 2024-05-27T08:10:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb535621740 1 client.22377 shutdown 2024-05-27T08:10:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb535621740 1 -- 172.21.15.63:0/1020639908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 msgr2=0x5590b1339400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb535621740 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b1339400 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fb510005bc0 tx=0x7fb510004730 comp rx=0 tx=0).stop 2024-05-27T08:10:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb535621740 1 -- 172.21.15.63:0/1020639908 shutdown_connections 2024-05-27T08:10:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb535621740 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590b0f4ad80 0x5590b134adb0 crc :-1 s=READY pgs=650 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb535621740 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b1344ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb535621740 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b1339400 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb535621740 1 --2- 172.21.15.63:0/1020639908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f632f0 0x5590b1338ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb515600640 0 client.22377 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.906+0000 7fb535621740 1 -- 172.21.15.63:0/1020639908 >> 172.21.15.63:0/1020639908 conn(0x5590b0f80300 msgr2=0x5590b1330780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.907+0000 7fb535621740 1 -- 172.21.15.63:0/1020639908 shutdown_connections 2024-05-27T08:10:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.907+0000 7fb535621740 1 -- 172.21.15.63:0/1020639908 wait complete. 2024-05-27T08:10:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.907+0000 7fb535621740 20 client.22377 trim_cache size 0 max 16384 2024-05-27T08:10:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.926+0000 7fb535621740 1 Processor -- start 2024-05-27T08:10:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.926+0000 7fb535621740 1 -- start start 2024-05-27T08:10:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.927+0000 7fb535621740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f632f0 0x5590b132a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.927+0000 7fb535621740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f6b4d0 0x5590b1344c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.927+0000 7fb535621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b1345170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.927+0000 7fb535621740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b5be0 con 0x5590b0f632f0 2024-05-27T08:10:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.927+0000 7fb535621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b34b0 con 0x5590b0f8c1b0 2024-05-27T08:10:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.927+0000 7fb535621740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b1320 con 0x5590b0f6b4d0 2024-05-27T08:10:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.927+0000 7fb527400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f632f0 0x5590b132a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.927+0000 7fb527400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f632f0 0x5590b132a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34088/0 (socket says 172.21.15.63:34088) 2024-05-27T08:10:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.927+0000 7fb527400640 1 -- 172.21.15.63:0/1544103510 learned_addr learned my addr 172.21.15.63:0/1544103510 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.927+0000 7fb526a00640 1 --2- 172.21.15.63:0/1544103510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f6b4d0 0x5590b1344c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.928+0000 7fb527e00640 1 --2- 172.21.15.63:0/1544103510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b1345170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.928+0000 7fb527400640 1 -- 172.21.15.63:0/1544103510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f6b4d0 msgr2=0x5590b1344c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.928+0000 7fb527400640 1 --2- 172.21.15.63:0/1544103510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f6b4d0 0x5590b1344c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.928+0000 7fb527400640 1 -- 172.21.15.63:0/1544103510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 msgr2=0x5590b1345170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.928+0000 7fb527400640 1 --2- 172.21.15.63:0/1544103510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b1345170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.928+0000 7fb527400640 1 -- 172.21.15.63:0/1544103510 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b1337200 con 0x5590b0f632f0 2024-05-27T08:10:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.928+0000 7fb527400640 1 --2- 172.21.15.63:0/1544103510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f632f0 0x5590b132a240 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fb518009900 tx=0x7fb518042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79535474364fcbdf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.929+0000 7fb517e00640 1 -- 172.21.15.63:0/1544103510 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518035040 con 0x5590b0f632f0 2024-05-27T08:10:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.929+0000 7fb517e00640 1 -- 172.21.15.63:0/1544103510 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb518057070 con 0x5590b0f632f0 2024-05-27T08:10:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.929+0000 7fb517e00640 1 -- 172.21.15.63:0/1544103510 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51803d890 con 0x5590b0f632f0 2024-05-27T08:10:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.930+0000 7fb535621740 1 -- 172.21.15.63:0/1544103510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f632f0 msgr2=0x5590b132a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.930+0000 7fb535621740 1 --2- 172.21.15.63:0/1544103510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f632f0 0x5590b132a240 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fb518009900 tx=0x7fb518042000 comp rx=0 tx=0).stop 2024-05-27T08:10:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.930+0000 7fb535621740 1 -- 172.21.15.63:0/1544103510 shutdown_connections 2024-05-27T08:10:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.930+0000 7fb535621740 1 --2- 172.21.15.63:0/1544103510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b1345170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.930+0000 7fb535621740 1 --2- 172.21.15.63:0/1544103510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f6b4d0 0x5590b1344c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.930+0000 7fb535621740 1 --2- 172.21.15.63:0/1544103510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f632f0 0x5590b132a240 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.930+0000 7fb535621740 1 -- 172.21.15.63:0/1544103510 >> 172.21.15.63:0/1544103510 conn(0x5590b0f80300 msgr2=0x5590b132f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.930+0000 7fb535621740 1 -- 172.21.15.63:0/1544103510 shutdown_connections 2024-05-27T08:10:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.930+0000 7fb535621740 1 -- 172.21.15.63:0/1544103510 wait complete. 2024-05-27T08:10:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.930+0000 7fb535621740 1 -- start start 2024-05-27T08:10:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.931+0000 7fb535621740 1 Processor -- start 2024-05-27T08:10:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.931+0000 7fb535621740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f632f0 0x5590b133c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.931+0000 7fb535621740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b133c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.931+0000 7fb535621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b13b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.931+0000 7fb535621740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b1970 con 0x5590b0f6b4d0 2024-05-27T08:10:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.931+0000 7fb535621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b2de0 con 0x5590b0f8c1b0 2024-05-27T08:10:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.931+0000 7fb535621740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281b2040 con 0x5590b0f632f0 2024-05-27T08:10:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.931+0000 7fb535621740 10 client.0 _ll_register_callbacks cb 0x7fb518009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.932+0000 7fb526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b133c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.932+0000 7fb526a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b133c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34094/0 (socket says 172.21.15.63:34094) 2024-05-27T08:10:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.932+0000 7fb526a00640 1 -- 172.21.15.63:0/2653429373 learned_addr learned my addr 172.21.15.63:0/2653429373 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.932+0000 7fb527400640 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f632f0 0x5590b133c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.932+0000 7fb527e00640 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b13b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.932+0000 7fb526a00640 1 -- 172.21.15.63:0/2653429373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f632f0 msgr2=0x5590b133c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.932+0000 7fb526a00640 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f632f0 0x5590b133c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.932+0000 7fb526a00640 1 -- 172.21.15.63:0/2653429373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 msgr2=0x5590b13b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.932+0000 7fb526a00640 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b13b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.932+0000 7fb526a00640 1 -- 172.21.15.63:0/2653429373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b13b1810 con 0x5590b0f6b4d0 2024-05-27T08:10:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.933+0000 7fb526a00640 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b133c980 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x5590b132beb0 tx=0x7fb510005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb39df293c27732 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.933+0000 7fb515600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.933+0000 7fb515600640 1 -- 172.21.15.63:0/2653429373 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51003b060 con 0x5590b0f6b4d0 2024-05-27T08:10:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.933+0000 7fb515600640 1 -- 172.21.15.63:0/2653429373 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 9483+0+0 (secure 0 0 0) 0x7fb51000fca0 con 0x5590b0f6b4d0 2024-05-27T08:10:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.933+0000 7fb535621740 1 -- 172.21.15.63:0/2653429373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb518057070 con 0x5590b0f6b4d0 2024-05-27T08:10:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.933+0000 7fb515600640 1 -- 172.21.15.63:0/2653429373 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb510042050 con 0x5590b0f6b4d0 2024-05-27T08:10:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.933+0000 7fb535621740 10 client.22383 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.933+0000 7fb535621740 1 -- 172.21.15.63:0/2653429373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5590b13b29a0 con 0x5590b0f6b4d0 2024-05-27T08:10:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.934+0000 7fb515600640 1 -- 172.21.15.63:0/2653429373 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb518057070 con 0x5590b0f6b4d0 2024-05-27T08:10:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.934+0000 7fb515600640 1 -- 172.21.15.63:0/2653429373 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb51001d070 con 0x5590b0f6b4d0 2024-05-27T08:10:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.934+0000 7fb535621740 10 client.22383 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.934+0000 7fb535621740 10 client.22383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.934+0000 7fb535621740 20 client.22383 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.934+0000 7fb535621740 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590b0f201c0 0x5590b13b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.934+0000 7fb535621740 4 client.22383 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:10:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.934+0000 7fb535621740 1 -- 172.21.15.63:0/2653429373 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590b129b2c0 con 0x5590b0f201c0 2024-05-27T08:10:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.935+0000 7fb527400640 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590b0f201c0 0x5590b13b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.936+0000 7fb527400640 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590b0f201c0 0x5590b13b5130 crc :-1 s=READY pgs=651 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3173019e2658b6 server_cookie=8b038dad0088d525 in_seq=0 out_seq=0 2024-05-27T08:10:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.936+0000 7fb515600640 10 client.22383 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:01.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:01 smithi086 ceph-mon[34687]: pgmap v1155: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:10:01.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:01 smithi149 ceph-mon[35408]: pgmap v1155: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:10:01.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:01 smithi063 ceph-mon[31613]: pgmap v1155: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:10:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb515600640 1 -- 172.21.15.63:0/2653429373 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5590b129b2c0 con 0x5590b0f201c0 2024-05-27T08:10:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb515600640 10 client.22383 handle_command_reply: tid=0 2024-05-27T08:10:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb535621740 1 client.22383 shutdown 2024-05-27T08:10:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb535621740 1 -- 172.21.15.63:0/2653429373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 msgr2=0x5590b133c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb535621740 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b133c980 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x5590b132beb0 tx=0x7fb510005690 comp rx=0 tx=0).stop 2024-05-27T08:10:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb535621740 1 -- 172.21.15.63:0/2653429373 shutdown_connections 2024-05-27T08:10:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb535621740 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5590b0f201c0 0x5590b13b5130 crc :-1 s=READY pgs=651 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb535621740 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590b0f8c1b0 0x5590b13b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb535621740 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590b0f6b4d0 0x5590b133c980 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb535621740 1 --2- 172.21.15.63:0/2653429373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5590b0f632f0 0x5590b133c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb515600640 0 client.22383 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.972+0000 7fb535621740 1 -- 172.21.15.63:0/2653429373 >> 172.21.15.63:0/2653429373 conn(0x5590b0f80300 msgr2=0x5590b132a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.973+0000 7fb535621740 1 -- 172.21.15.63:0/2653429373 shutdown_connections 2024-05-27T08:10:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.973+0000 7fb535621740 1 -- 172.21.15.63:0/2653429373 wait complete. 2024-05-27T08:10:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.973+0000 7fb535621740 20 client.22383 trim_cache size 0 max 16384 2024-05-27T08:10:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:10:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:10:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 -- 172.21.15.63:0/3745560157 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b2950 msgr2=0x7fb50c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b2950 0x7fb50c0b4e60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb518006010 tx=0x7fb518000bc0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 -- 172.21.15.63:0/3745560157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528110340 msgr2=0x7fb52819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528110340 0x7fb52819c690 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7fb520009fd0 tx=0x7fb52000b260 comp rx=0 tx=0).stop 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 -- 172.21.15.63:0/3745560157 shutdown_connections 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb50c0b2950 0x7fb50c0b4e60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb528110340 0x7fb52819c690 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb52810d9e0 0x7fb5281a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 --2- 172.21.15.63:0/3745560157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb528075400 0x7fb5281a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 -- 172.21.15.63:0/3745560157 >> 172.21.15.63:0/3745560157 conn(0x7fb528079100 msgr2=0x7fb5281022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 -- 172.21.15.63:0/3745560157 shutdown_connections 2024-05-27T08:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:01.974+0000 7fb52d000640 1 -- 172.21.15.63:0/3745560157 wait complete. 2024-05-27T08:10:03.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:03 smithi063 ceph-mon[31613]: pgmap v1156: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 31 op/s 2024-05-27T08:10:04.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:03 smithi086 ceph-mon[34687]: pgmap v1156: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 31 op/s 2024-05-27T08:10:04.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:03 smithi149 ceph-mon[35408]: pgmap v1156: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 31 op/s 2024-05-27T08:10:06.126 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:10:06.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:05 smithi086 ceph-mon[34687]: pgmap v1157: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.8 MiB/s wr, 24 op/s 2024-05-27T08:10:06.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:05 smithi149 ceph-mon[35408]: pgmap v1157: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.8 MiB/s wr, 24 op/s 2024-05-27T08:10:06.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:05 smithi063 ceph-mon[31613]: pgmap v1157: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.8 MiB/s wr, 24 op/s 2024-05-27T08:10:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:10:08.047 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:07 smithi063 ceph-mon[31613]: pgmap v1158: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.4 MiB/s wr, 30 op/s 2024-05-27T08:10:08.047 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:07 smithi063 ceph-mon[31613]: osdmap e153: 12 total, 12 up, 12 in 2024-05-27T08:10:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:07 smithi086 ceph-mon[34687]: pgmap v1158: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.4 MiB/s wr, 30 op/s 2024-05-27T08:10:08.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:07 smithi086 ceph-mon[34687]: osdmap e153: 12 total, 12 up, 12 in 2024-05-27T08:10:08.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:07 smithi149 ceph-mon[35408]: pgmap v1158: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.4 MiB/s wr, 30 op/s 2024-05-27T08:10:08.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:07 smithi149 ceph-mon[35408]: osdmap e153: 12 total, 12 up, 12 in 2024-05-27T08:10:10.000 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:09 smithi063 ceph-mon[31613]: pgmap v1160: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:10:10.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:09 smithi086 ceph-mon[34687]: pgmap v1160: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:10:10.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:09 smithi149 ceph-mon[35408]: pgmap v1160: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:10:10.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.691+0000 7f61f1600700 1 -- 172.21.15.63:0/2058167486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61ec108ef0 msgr2=0x7f61ec109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:10.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.691+0000 7f61f1600700 1 --2- 172.21.15.63:0/2058167486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61ec108ef0 0x7f61ec109300 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f61d80099c0 tx=0x7f61d8009cd0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.691+0000 7f61f1600700 1 -- 172.21.15.63:0/2058167486 shutdown_connections 2024-05-27T08:10:10.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.691+0000 7f61f1600700 1 --2- 172.21.15.63:0/2058167486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61ec109840 0x7f61ec1111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.691+0000 7f61f1600700 1 --2- 172.21.15.63:0/2058167486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec1045c0 0x7f61ec1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.691+0000 7f61f1600700 1 --2- 172.21.15.63:0/2058167486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61ec108ef0 0x7f61ec109300 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.691+0000 7f61f1600700 1 -- 172.21.15.63:0/2058167486 >> 172.21.15.63:0/2058167486 conn(0x7f61ec0ffe90 msgr2=0x7f61ec1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:10.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.691+0000 7f61f1600700 1 -- 172.21.15.63:0/2058167486 shutdown_connections 2024-05-27T08:10:10.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.691+0000 7f61f1600700 1 -- 172.21.15.63:0/2058167486 wait complete. 2024-05-27T08:10:10.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.692+0000 7f61f1600700 1 Processor -- start 2024-05-27T08:10:10.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.693+0000 7f61f1600700 1 -- start start 2024-05-27T08:10:10.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.693+0000 7f61f1600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61ec1045c0 0x7f61ec19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.693+0000 7f61f1600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61ec108ef0 0x7f61ec19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.693+0000 7f61f1600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec109840 0x7f61ec1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.693+0000 7f61f1600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1143c0 con 0x7f61ec1045c0 2024-05-27T08:10:10.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.693+0000 7f61f1600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec114240 con 0x7f61ec109840 2024-05-27T08:10:10.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.693+0000 7f61f1600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec114540 con 0x7f61ec108ef0 2024-05-27T08:10:10.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61eaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61ec1045c0 0x7f61ec19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61eaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61ec1045c0 0x7f61ec19cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37316/0 (socket says 172.21.15.63:37316) 2024-05-27T08:10:10.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61eaa00700 1 -- 172.21.15.63:0/4035467295 learned_addr learned my addr 172.21.15.63:0/4035467295 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:10.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61ea000700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61ec108ef0 0x7f61ec19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61eb400700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec109840 0x7f61ec1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61eaa00700 1 -- 172.21.15.63:0/4035467295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61ec108ef0 msgr2=0x7f61ec19d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:10.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61eaa00700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61ec108ef0 0x7f61ec19d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61eaa00700 1 -- 172.21.15.63:0/4035467295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec109840 msgr2=0x7f61ec1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:10.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61eaa00700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec109840 0x7f61ec1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61eaa00700 1 -- 172.21.15.63:0/4035467295 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec1a1ec0 con 0x7f61ec1045c0 2024-05-27T08:10:10.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.694+0000 7f61eb400700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec109840 0x7f61ec1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:10.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.695+0000 7f61eaa00700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61ec1045c0 0x7f61ec19cff0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00c920 tx=0x7f61dc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:10.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.695+0000 7f61e3400700 1 -- 172.21.15.63:0/4035467295 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc007b00 con 0x7f61ec1045c0 2024-05-27T08:10:10.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.695+0000 7f61e3400700 1 -- 172.21.15.63:0/4035467295 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61dc004cc0 con 0x7f61ec1045c0 2024-05-27T08:10:10.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.695+0000 7f61e3400700 1 -- 172.21.15.63:0/4035467295 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc005440 con 0x7f61ec1045c0 2024-05-27T08:10:10.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.695+0000 7f61f1600700 1 -- 172.21.15.63:0/4035467295 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ec1a21b0 con 0x7f61ec1045c0 2024-05-27T08:10:10.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.695+0000 7f61f1600700 1 -- 172.21.15.63:0/4035467295 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ec1a9ad0 con 0x7f61ec1045c0 2024-05-27T08:10:10.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.700+0000 7f61e3400700 1 -- 172.21.15.63:0/4035467295 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f61dc004750 con 0x7f61ec1045c0 2024-05-27T08:10:10.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.701+0000 7f62001d0b80 1 Processor -- start 2024-05-27T08:10:10.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.702+0000 7f62001d0b80 1 -- start start 2024-05-27T08:10:10.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.702+0000 7f61e3400700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61d00b2a70 0x7f61d00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.702+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 0x55e470fbd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.702+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fb7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.702+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fb81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.702+0000 7f62001d0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e470ee5e40 con 0x55e470fbdd30 2024-05-27T08:10:10.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.702+0000 7f62001d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e470ed59e0 con 0x55e470fb7db0 2024-05-27T08:10:10.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.702+0000 7f62001d0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e470ed9a20 con 0x55e470e38e40 2024-05-27T08:10:10.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.702+0000 7f61e3400700 1 -- 172.21.15.63:0/4035467295 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 9571+0+0 (secure 0 0 0) 0x7f61dc0e2590 con 0x7f61ec1045c0 2024-05-27T08:10:10.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61eb400700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61d00b2a70 0x7f61d00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61ea000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fb7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61ea000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fb7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37330/0 (socket says 172.21.15.63:37330) 2024-05-27T08:10:10.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61ea000700 1 -- 172.21.15.63:0/2854118173 learned_addr learned my addr 172.21.15.63:0/2854118173 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:10.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61eb400700 1 --2- 172.21.15.63:0/2854118173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fb81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61eaa00700 1 --2- 172.21.15.63:0/2854118173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 0x55e470fbd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61ea000700 1 -- 172.21.15.63:0/2854118173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 msgr2=0x55e470fbd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:10.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61ea000700 1 --2- 172.21.15.63:0/2854118173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 0x55e470fbd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61ea000700 1 -- 172.21.15.63:0/2854118173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 msgr2=0x55e470fb81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:10.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61ea000700 1 --2- 172.21.15.63:0/2854118173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fb81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61ea000700 1 -- 172.21.15.63:0/2854118173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e470e9e420 con 0x55e470fbdd30 2024-05-27T08:10:10.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.703+0000 7f61eaa00700 1 --2- 172.21.15.63:0/2854118173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 0x55e470fbd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:10.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.704+0000 7f61eb400700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61d00b2a70 0x7f61d00b4f80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009440 tx=0x7f61e4009e30 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:10.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.704+0000 7f61ea000700 1 --2- 172.21.15.63:0/2854118173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fb7870 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f61d80099c0 tx=0x7f61d8005100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8539738ce8bea04f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:10.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.704+0000 7f61d7e00700 1 -- 172.21.15.63:0/2854118173 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8027070 con 0x55e470fbdd30 2024-05-27T08:10:10.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.704+0000 7f61d7e00700 1 -- 172.21.15.63:0/2854118173 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61d80054f0 con 0x55e470fbdd30 2024-05-27T08:10:10.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.705+0000 7f61d7e00700 1 -- 172.21.15.63:0/2854118173 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80039e0 con 0x55e470fbdd30 2024-05-27T08:10:10.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.706+0000 7f62001d0b80 1 -- 172.21.15.63:0/2854118173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 msgr2=0x55e470fb7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:10.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.706+0000 7f62001d0b80 1 --2- 172.21.15.63:0/2854118173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fb7870 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f61d80099c0 tx=0x7f61d8005100 comp rx=0 tx=0).stop 2024-05-27T08:10:10.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.706+0000 7f62001d0b80 1 -- 172.21.15.63:0/2854118173 shutdown_connections 2024-05-27T08:10:10.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.706+0000 7f62001d0b80 1 --2- 172.21.15.63:0/2854118173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fb81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.706+0000 7f62001d0b80 1 --2- 172.21.15.63:0/2854118173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 0x55e470fbd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.706+0000 7f62001d0b80 1 --2- 172.21.15.63:0/2854118173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fb7870 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.706+0000 7f62001d0b80 1 -- 172.21.15.63:0/2854118173 >> 172.21.15.63:0/2854118173 conn(0x55e470e98fc0 msgr2=0x55e470fb28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:10.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.706+0000 7f62001d0b80 1 -- 172.21.15.63:0/2854118173 shutdown_connections 2024-05-27T08:10:10.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.706+0000 7f62001d0b80 1 -- 172.21.15.63:0/2854118173 wait complete. 2024-05-27T08:10:10.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.707+0000 7f62001d0b80 1 -- start start 2024-05-27T08:10:10.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.707+0000 7f62001d0b80 1 Processor -- start 2024-05-27T08:10:10.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470e38e40 0x55e470fc15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fc1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470fbdd30 0x55e470fc2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f62001d0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e470ddbfa0 con 0x55e470e38e40 2024-05-27T08:10:10.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f62001d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e470e76760 con 0x55e470fb7db0 2024-05-27T08:10:10.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f62001d0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e470fccfb0 con 0x55e470fbdd30 2024-05-27T08:10:10.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f62001d0b80 10 client.0 _ll_register_callbacks cb 0x55e470e7fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:10.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f61eaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470e38e40 0x55e470fc15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f61eaa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470e38e40 0x55e470fc15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37336/0 (socket says 172.21.15.63:37336) 2024-05-27T08:10:10.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f61eaa00700 1 -- 172.21.15.63:0/3605143913 learned_addr learned my addr 172.21.15.63:0/3605143913 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:10.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f61ea000700 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fc1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.708+0000 7f61eb400700 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470fbdd30 0x55e470fc2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.709+0000 7f61eaa00700 1 -- 172.21.15.63:0/3605143913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470fbdd30 msgr2=0x55e470fc2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:10.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.709+0000 7f61eaa00700 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470fbdd30 0x55e470fc2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.709+0000 7f61eaa00700 1 -- 172.21.15.63:0/3605143913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 msgr2=0x55e470fc1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:10.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.709+0000 7f61eaa00700 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fc1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.709+0000 7f61eaa00700 1 -- 172.21.15.63:0/3605143913 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e470e258d0 con 0x55e470e38e40 2024-05-27T08:10:10.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.709+0000 7f61ea000700 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fc1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:10:10.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.709+0000 7f61eaa00700 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470e38e40 0x55e470fc15d0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0a9380 tx=0x7f61dc0a8fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cb967030d878c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:10.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.709+0000 7f61d4c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:10.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.709+0000 7f61d4c00700 1 -- 172.21.15.63:0/3605143913 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0c3060 con 0x55e470e38e40 2024-05-27T08:10:10.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.709+0000 7f61d4c00700 1 -- 172.21.15.63:0/3605143913 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(153..153 src has 1..153) ==== 9571+0+0 (secure 0 0 0) 0x7f61dc0e2150 con 0x55e470e38e40 2024-05-27T08:10:10.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.710+0000 7f62001d0b80 1 -- 172.21.15.63:0/3605143913 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d80054f0 con 0x55e470e38e40 2024-05-27T08:10:10.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.710+0000 7f61d4c00700 1 -- 172.21.15.63:0/3605143913 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f61dc01d040 con 0x55e470e38e40 2024-05-27T08:10:10.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.710+0000 7f62001d0b80 10 client.22395 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:10.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.710+0000 7f62001d0b80 1 -- 172.21.15.63:0/3605143913 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e470ea6060 con 0x55e470e38e40 2024-05-27T08:10:10.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.711+0000 7f61d4c00700 1 -- 172.21.15.63:0/3605143913 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61d80054f0 con 0x55e470e38e40 2024-05-27T08:10:10.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.711+0000 7f61d4c00700 1 -- 172.21.15.63:0/3605143913 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f61dc0c3ce0 con 0x55e470e38e40 2024-05-27T08:10:10.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.711+0000 7f62001d0b80 10 client.22395 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:10.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.711+0000 7f62001d0b80 10 client.22395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:10.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.711+0000 7f62001d0b80 20 client.22395 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:10.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.711+0000 7f62001d0b80 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e470fd9f20 0x55e470fdc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.711+0000 7f62001d0b80 4 client.22395 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:10.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.711+0000 7f62001d0b80 1 -- 172.21.15.63:0/3605143913 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e470ee7d90 con 0x55e470fd9f20 2024-05-27T08:10:10.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.712+0000 7f61ea000700 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e470fd9f20 0x55e470fdc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.713+0000 7f61ea000700 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e470fd9f20 0x55e470fdc3c0 crc :-1 s=READY pgs=652 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f1f7c29dbc7cb6 server_cookie=acef810a8c466b1d in_seq=0 out_seq=0 2024-05-27T08:10:10.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.713+0000 7f61d4c00700 10 client.22395 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:10.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f61d4c00700 1 -- 172.21.15.63:0/3605143913 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e470ee7d90 con 0x55e470fd9f20 2024-05-27T08:10:10.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f61d4c00700 10 client.22395 handle_command_reply: tid=0 2024-05-27T08:10:10.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 client.22395 shutdown 2024-05-27T08:10:10.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 -- 172.21.15.63:0/3605143913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470e38e40 msgr2=0x55e470fc15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:10.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470e38e40 0x55e470fc15d0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0a9380 tx=0x7f61dc0a8fb0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 -- 172.21.15.63:0/3605143913 shutdown_connections 2024-05-27T08:10:10.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e470fd9f20 0x55e470fdc3c0 crc :-1 s=READY pgs=652 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470fbdd30 0x55e470fc2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470e38e40 0x55e470fc15d0 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 --2- 172.21.15.63:0/3605143913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fc1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 -- 172.21.15.63:0/3605143913 >> 172.21.15.63:0/3605143913 conn(0x55e470e98fc0 msgr2=0x55e470fb2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:10.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f61d4c00700 0 client.22395 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:10.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 -- 172.21.15.63:0/3605143913 shutdown_connections 2024-05-27T08:10:10.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.715+0000 7f62001d0b80 1 -- 172.21.15.63:0/3605143913 wait complete. 2024-05-27T08:10:10.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.716+0000 7f62001d0b80 20 client.22395 trim_cache size 0 max 16384 2024-05-27T08:10:10.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.743+0000 7f62001d0b80 1 Processor -- start 2024-05-27T08:10:10.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.743+0000 7f62001d0b80 1 -- start start 2024-05-27T08:10:10.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.743+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 0x55e470fb7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.743+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fc08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.743+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fdc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.743+0000 7f62001d0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1b0d30 con 0x55e470fbdd30 2024-05-27T08:10:10.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.743+0000 7f62001d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1ae5c0 con 0x55e470fb7db0 2024-05-27T08:10:10.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.743+0000 7f62001d0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1ac430 con 0x55e470e38e40 2024-05-27T08:10:10.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.744+0000 7f61eb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fdc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.744+0000 7f61eb400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fdc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37344/0 (socket says 172.21.15.63:37344) 2024-05-27T08:10:10.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.744+0000 7f61eb400700 1 -- 172.21.15.63:0/3596252065 learned_addr learned my addr 172.21.15.63:0/3596252065 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:10.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.744+0000 7f61ea000700 1 --2- 172.21.15.63:0/3596252065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fc08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.744+0000 7f61eaa00700 1 --2- 172.21.15.63:0/3596252065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 0x55e470fb7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.745+0000 7f61eb400700 1 -- 172.21.15.63:0/3596252065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 msgr2=0x55e470fb7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:10.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.745+0000 7f61eb400700 1 --2- 172.21.15.63:0/3596252065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 0x55e470fb7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.745+0000 7f61eb400700 1 -- 172.21.15.63:0/3596252065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 msgr2=0x55e470fc08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:10.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.745+0000 7f61eb400700 1 --2- 172.21.15.63:0/3596252065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fc08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.745+0000 7f61eb400700 1 -- 172.21.15.63:0/3596252065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e470fc23c0 con 0x55e470fbdd30 2024-05-27T08:10:10.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.745+0000 7f61eb400700 1 --2- 172.21.15.63:0/3596252065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fdc5a0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f61e40097c0 tx=0x7f61e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ef79ee82a101d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:10.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.745+0000 7f61eaa00700 1 --2- 172.21.15.63:0/3596252065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 0x55e470fb7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:10:10.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.748+0000 7f61d7400700 1 -- 172.21.15.63:0/3596252065 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4029070 con 0x55e470fbdd30 2024-05-27T08:10:10.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.748+0000 7f61d7400700 1 -- 172.21.15.63:0/3596252065 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61e40098a0 con 0x55e470fbdd30 2024-05-27T08:10:10.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.748+0000 7f61d7400700 1 -- 172.21.15.63:0/3596252065 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4013040 con 0x55e470fbdd30 2024-05-27T08:10:10.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.749+0000 7f62001d0b80 1 -- 172.21.15.63:0/3596252065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 msgr2=0x55e470fdc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:10.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.749+0000 7f62001d0b80 1 --2- 172.21.15.63:0/3596252065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fdc5a0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f61e40097c0 tx=0x7f61e4004ad0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.749+0000 7f62001d0b80 1 -- 172.21.15.63:0/3596252065 shutdown_connections 2024-05-27T08:10:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.749+0000 7f62001d0b80 1 --2- 172.21.15.63:0/3596252065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fbdd30 0x55e470fdc5a0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.749+0000 7f62001d0b80 1 --2- 172.21.15.63:0/3596252065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470e38e40 0x55e470fb7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.749+0000 7f62001d0b80 1 --2- 172.21.15.63:0/3596252065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470fb7db0 0x55e470fc08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.749+0000 7f62001d0b80 1 -- 172.21.15.63:0/3596252065 >> 172.21.15.63:0/3596252065 conn(0x55e470e98fc0 msgr2=0x55e470fbb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:10.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.750+0000 7f62001d0b80 1 -- 172.21.15.63:0/3596252065 shutdown_connections 2024-05-27T08:10:10.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.750+0000 7f62001d0b80 1 -- 172.21.15.63:0/3596252065 wait complete. 2024-05-27T08:10:10.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.750+0000 7f62001d0b80 1 -- start start 2024-05-27T08:10:10.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.751+0000 7f62001d0b80 1 Processor -- start 2024-05-27T08:10:10.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.751+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470e38e40 0x55e470fd2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.751+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fb7db0 0x55e470fd2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.751+0000 7f62001d0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470fbdd30 0x55e470fe1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.751+0000 7f62001d0b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1aca80 con 0x55e470fb7db0 2024-05-27T08:10:10.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.751+0000 7f62001d0b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1adef0 con 0x55e470e38e40 2024-05-27T08:10:10.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.751+0000 7f62001d0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec1ad150 con 0x55e470fbdd30 2024-05-27T08:10:10.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.751+0000 7f62001d0b80 10 client.0 _ll_register_callbacks cb 0x55e470e7fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:10.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.752+0000 7f61ea000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fb7db0 0x55e470fd2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.752+0000 7f61ea000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fb7db0 0x55e470fd2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37356/0 (socket says 172.21.15.63:37356) 2024-05-27T08:10:10.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.752+0000 7f61ea000700 1 -- 172.21.15.63:0/4251247379 learned_addr learned my addr 172.21.15.63:0/4251247379 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:10.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.752+0000 7f61eaa00700 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470e38e40 0x55e470fd2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.752+0000 7f61eb400700 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470fbdd30 0x55e470fe1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.753+0000 7f61ea000700 1 -- 172.21.15.63:0/4251247379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470fbdd30 msgr2=0x55e470fe1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:10.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.753+0000 7f61ea000700 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470fbdd30 0x55e470fe1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.753+0000 7f61ea000700 1 -- 172.21.15.63:0/4251247379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470e38e40 msgr2=0x55e470fd2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:10.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.753+0000 7f61ea000700 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470e38e40 0x55e470fd2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.753+0000 7f61ea000700 1 -- 172.21.15.63:0/4251247379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e470fe1a30 con 0x55e470fb7db0 2024-05-27T08:10:10.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.753+0000 7f61ea000700 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fb7db0 0x55e470fd2a80 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f61d8000c00 tx=0x7f61d8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56706494fff6b155 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:10.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.753+0000 7f61d4c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:10.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.753+0000 7f61d4c00700 1 -- 172.21.15.63:0/4251247379 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d803c060 con 0x55e470fb7db0 2024-05-27T08:10:10.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.753+0000 7f61d4c00700 1 -- 172.21.15.63:0/4251247379 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(153..153 src has 1..153) ==== 9571+0+0 (secure 0 0 0) 0x7f61d800ba10 con 0x55e470fb7db0 2024-05-27T08:10:10.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.753+0000 7f62001d0b80 1 -- 172.21.15.63:0/4251247379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e470ee1570 con 0x55e470fb7db0 2024-05-27T08:10:10.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.754+0000 7f61d4c00700 1 -- 172.21.15.63:0/4251247379 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f61d8043050 con 0x55e470fb7db0 2024-05-27T08:10:10.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.754+0000 7f62001d0b80 10 client.22401 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:10.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.754+0000 7f62001d0b80 1 -- 172.21.15.63:0/4251247379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e471044510 con 0x55e470fb7db0 2024-05-27T08:10:10.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.754+0000 7f61d4c00700 1 -- 172.21.15.63:0/4251247379 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55e470ee1570 con 0x55e470fb7db0 2024-05-27T08:10:10.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.754+0000 7f61d4c00700 1 -- 172.21.15.63:0/4251247379 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f61d8022480 con 0x55e470fb7db0 2024-05-27T08:10:10.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.754+0000 7f62001d0b80 10 client.22401 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:10.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.754+0000 7f62001d0b80 10 client.22401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:10.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.754+0000 7f62001d0b80 20 client.22401 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:10.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.754+0000 7f62001d0b80 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4710446e0 0x55e471046b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:10.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.755+0000 7f62001d0b80 4 client.22401 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:10:10.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.755+0000 7f62001d0b80 1 -- 172.21.15.63:0/4251247379 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e470d58250 con 0x55e4710446e0 2024-05-27T08:10:10.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.755+0000 7f61eaa00700 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4710446e0 0x55e471046b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:10.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.756+0000 7f61d4c00700 10 client.22401 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:10.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.756+0000 7f61eaa00700 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4710446e0 0x55e471046b00 crc :-1 s=READY pgs=653 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87206d70a79bfb7 server_cookie=f5c36c008ac547df in_seq=0 out_seq=0 2024-05-27T08:10:10.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f61d4c00700 1 -- 172.21.15.63:0/4251247379 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e470d58250 con 0x55e4710446e0 2024-05-27T08:10:10.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f61d4c00700 10 client.22401 handle_command_reply: tid=0 2024-05-27T08:10:10.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f62001d0b80 1 client.22401 shutdown 2024-05-27T08:10:10.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f62001d0b80 1 -- 172.21.15.63:0/4251247379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fb7db0 msgr2=0x55e470fd2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:10.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f62001d0b80 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fb7db0 0x55e470fd2a80 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f61d8000c00 tx=0x7f61d8009810 comp rx=0 tx=0).stop 2024-05-27T08:10:10.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f62001d0b80 1 -- 172.21.15.63:0/4251247379 shutdown_connections 2024-05-27T08:10:10.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f62001d0b80 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e4710446e0 0x55e471046b00 crc :-1 s=READY pgs=653 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f62001d0b80 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e470fbdd30 0x55e470fe1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f62001d0b80 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e470e38e40 0x55e470fd2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f62001d0b80 1 --2- 172.21.15.63:0/4251247379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e470fb7db0 0x55e470fd2a80 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f62001d0b80 1 -- 172.21.15.63:0/4251247379 >> 172.21.15.63:0/4251247379 conn(0x55e470e98fc0 msgr2=0x55e470fc0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:10.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.757+0000 7f61d4c00700 0 client.22401 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:10.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.758+0000 7f62001d0b80 1 -- 172.21.15.63:0/4251247379 shutdown_connections 2024-05-27T08:10:10.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.758+0000 7f62001d0b80 1 -- 172.21.15.63:0/4251247379 wait complete. 2024-05-27T08:10:10.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.758+0000 7f62001d0b80 20 client.22401 trim_cache size 0 max 16384 2024-05-27T08:10:10.760 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:10:10.760 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:10:10.760 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:10:10.760 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 -- 172.21.15.63:0/4035467295 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61d00b2a70 msgr2=0x7f61d00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61d00b2a70 0x7f61d00b4f80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009440 tx=0x7f61e4009e30 comp rx=0 tx=0).stop 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 -- 172.21.15.63:0/4035467295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61ec1045c0 msgr2=0x7f61ec19cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61ec1045c0 0x7f61ec19cff0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00c920 tx=0x7f61dc00cc30 comp rx=0 tx=0).stop 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 -- 172.21.15.63:0/4035467295 shutdown_connections 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61d00b2a70 0x7f61d00b4f80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61ec109840 0x7f61ec1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61ec1045c0 0x7f61ec19cff0 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 --2- 172.21.15.63:0/4035467295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61ec108ef0 0x7f61ec19d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 -- 172.21.15.63:0/4035467295 >> 172.21.15.63:0/4035467295 conn(0x7f61ec0ffe90 msgr2=0x7f61ec10a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 -- 172.21.15.63:0/4035467295 shutdown_connections 2024-05-27T08:10:10.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:10.759+0000 7f61f1600700 1 -- 172.21.15.63:0/4035467295 wait complete. 2024-05-27T08:10:11.866 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:10:11.866 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:10:12.012 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:11 smithi086 ceph-mon[34687]: pgmap v1161: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:10:12.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:11 smithi149 ceph-mon[35408]: pgmap v1161: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:10:12.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:11 smithi063 ceph-mon[31613]: pgmap v1161: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:10:12.361 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:10:14.029 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:13 smithi063 ceph-mon[31613]: pgmap v1162: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:10:14.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:13 smithi086 ceph-mon[34687]: pgmap v1162: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:10:14.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:13 smithi149 ceph-mon[35408]: pgmap v1162: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:10:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.379+0000 7f43fca00640 1 -- 172.21.15.63:0/2561210022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f8109f50 msgr2=0x7f43f8103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.379+0000 7f43fca00640 1 --2- 172.21.15.63:0/2561210022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f8109f50 0x7f43f8103be0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009900 tx=0x7f43e402a410 comp rx=0 tx=0).stop 2024-05-27T08:10:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.379+0000 7f43fca00640 1 -- 172.21.15.63:0/2561210022 shutdown_connections 2024-05-27T08:10:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.379+0000 7f43fca00640 1 --2- 172.21.15.63:0/2561210022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f8104250 0x7f43f81105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.379+0000 7f43fca00640 1 --2- 172.21.15.63:0/2561210022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f8109f50 0x7f43f8103be0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.379+0000 7f43fca00640 1 --2- 172.21.15.63:0/2561210022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f8109ab0 0x7f43f81036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.379+0000 7f43fca00640 1 -- 172.21.15.63:0/2561210022 >> 172.21.15.63:0/2561210022 conn(0x7f43f8079240 msgr2=0x7f43f8079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.379+0000 7f43fca00640 1 -- 172.21.15.63:0/2561210022 shutdown_connections 2024-05-27T08:10:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.379+0000 7f43fca00640 1 -- 172.21.15.63:0/2561210022 wait complete. 2024-05-27T08:10:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.380+0000 7f43fca00640 1 Processor -- start 2024-05-27T08:10:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.380+0000 7f43fca00640 1 -- start start 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43fca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f8104250 0x7f43f8074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43fca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f8109ab0 0x7f43f8075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43fca00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f8109f50 0x7f43f8075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43fca00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43f8113b50 con 0x7f43f8109f50 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43fca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81139d0 con 0x7f43f8109ab0 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43fca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f43f8113cd0 con 0x7f43f8104250 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43f7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f8109f50 0x7f43f8075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43f7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f8109f50 0x7f43f8075a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37364/0 (socket says 172.21.15.63:37364) 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43f7200640 1 -- 172.21.15.63:0/3373266459 learned_addr learned my addr 172.21.15.63:0/3373266459 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43f6800640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f8104250 0x7f43f8074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.381+0000 7f43f5e00640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f8109ab0 0x7f43f8075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.382+0000 7f43f7200640 1 -- 172.21.15.63:0/3373266459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f8104250 msgr2=0x7f43f8074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.382+0000 7f43f7200640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f8104250 0x7f43f8074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.382+0000 7f43f7200640 1 -- 172.21.15.63:0/3373266459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f8109ab0 msgr2=0x7f43f8075500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.382+0000 7f43f7200640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f8109ab0 0x7f43f8075500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.382+0000 7f43f7200640 1 -- 172.21.15.63:0/3373266459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f81a89f0 con 0x7f43f8109f50 2024-05-27T08:10:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.382+0000 7f43f7200640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f8109f50 0x7f43f8075a40 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009fd0 tx=0x7f43f000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.382+0000 7f43ef400640 1 -- 172.21.15.63:0/3373266459 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0017070 con 0x7f43f8109f50 2024-05-27T08:10:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.383+0000 7f43fca00640 1 -- 172.21.15.63:0/3373266459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f81a8ce0 con 0x7f43f8109f50 2024-05-27T08:10:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.383+0000 7f43ef400640 1 -- 172.21.15.63:0/3373266459 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43f0003e00 con 0x7f43f8109f50 2024-05-27T08:10:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.383+0000 7f43ef400640 1 -- 172.21.15.63:0/3373266459 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0010460 con 0x7f43f8109f50 2024-05-27T08:10:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.383+0000 7f43fca00640 1 -- 172.21.15.63:0/3373266459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f81a91b0 con 0x7f43f8109f50 2024-05-27T08:10:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.387+0000 7f440439b740 1 Processor -- start 2024-05-27T08:10:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.387+0000 7f43ef400640 1 -- 172.21.15.63:0/3373266459 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f43f0003970 con 0x7f43f8109f50 2024-05-27T08:10:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.387+0000 7f440439b740 1 -- start start 2024-05-27T08:10:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.388+0000 7f440439b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8cb111b0 0x556e8ceb3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.388+0000 7f440439b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8ceb3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.388+0000 7f440439b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8ceb44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.388+0000 7f440439b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556e8ce24b90 con 0x556e8caf04d0 2024-05-27T08:10:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.388+0000 7f440439b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e8ce14730 con 0x556e8cae82f0 2024-05-27T08:10:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.388+0000 7f440439b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556e8ce18770 con 0x556e8cb111b0 2024-05-27T08:10:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.388+0000 7f43f5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8ceb44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.388+0000 7f43f5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8ceb44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37378/0 (socket says 172.21.15.63:37378) 2024-05-27T08:10:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.388+0000 7f43f5e00640 1 -- 172.21.15.63:0/33078714 learned_addr learned my addr 172.21.15.63:0/33078714 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.389+0000 7f43f6800640 1 --2- 172.21.15.63:0/33078714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8ceb3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.389+0000 7f43f5e00640 1 --2- 172.21.15.63:0/33078714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8cb111b0 0x556e8ceb3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.389+0000 7f43f5e00640 1 -- 172.21.15.63:0/33078714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8cb111b0 msgr2=0x556e8ceb3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.389+0000 7f43f5e00640 1 --2- 172.21.15.63:0/33078714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8cb111b0 0x556e8ceb3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.389+0000 7f43f5e00640 1 -- 172.21.15.63:0/33078714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 msgr2=0x556e8ceb3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.389+0000 7f43f5e00640 1 --2- 172.21.15.63:0/33078714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8ceb3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.389+0000 7f43f5e00640 1 -- 172.21.15.63:0/33078714 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8cdad8b0 con 0x556e8caf04d0 2024-05-27T08:10:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.389+0000 7f43ef400640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43dc0b2950 0x7f43dc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.389+0000 7f43f6800640 1 --2- 172.21.15.63:0/33078714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8ceb3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:10:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.390+0000 7f43ef400640 1 -- 172.21.15.63:0/3373266459 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 9571+0+0 (secure 0 0 0) 0x7f43f00e2c10 con 0x7f43f8109f50 2024-05-27T08:10:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.390+0000 7f43f7200640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43dc0b2950 0x7f43dc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.390+0000 7f43f5e00640 1 --2- 172.21.15.63:0/33078714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8ceb44e0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f43e4005940 tx=0x7f43e40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242d43969eac3569 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.390+0000 7f43e3400640 1 -- 172.21.15.63:0/33078714 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4035040 con 0x556e8caf04d0 2024-05-27T08:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.390+0000 7f43e3400640 1 -- 172.21.15.63:0/33078714 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43e40044c0 con 0x556e8caf04d0 2024-05-27T08:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.390+0000 7f43e3400640 1 -- 172.21.15.63:0/33078714 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4003b90 con 0x556e8caf04d0 2024-05-27T08:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.391+0000 7f440439b740 1 -- 172.21.15.63:0/33078714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 msgr2=0x556e8ceb44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.391+0000 7f43f7200640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43dc0b2950 0x7f43dc0b4e60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f43f000b9a0 tx=0x7f43f008ff90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.391+0000 7f440439b740 1 --2- 172.21.15.63:0/33078714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8ceb44e0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f43e4005940 tx=0x7f43e40058b0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.391+0000 7f440439b740 1 -- 172.21.15.63:0/33078714 shutdown_connections 2024-05-27T08:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.391+0000 7f440439b740 1 --2- 172.21.15.63:0/33078714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8ceb44e0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.391+0000 7f440439b740 1 --2- 172.21.15.63:0/33078714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8ceb3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.392+0000 7f440439b740 1 --2- 172.21.15.63:0/33078714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8cb111b0 0x556e8ceb3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.392+0000 7f440439b740 1 -- 172.21.15.63:0/33078714 >> 172.21.15.63:0/33078714 conn(0x556e8cb05300 msgr2=0x556e8ceb0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.392+0000 7f440439b740 1 -- 172.21.15.63:0/33078714 shutdown_connections 2024-05-27T08:10:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.392+0000 7f440439b740 1 -- 172.21.15.63:0/33078714 wait complete. 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.392+0000 7f440439b740 1 -- start start 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.393+0000 7f440439b740 1 Processor -- start 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.393+0000 7f440439b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8cebdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.393+0000 7f440439b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8caf04d0 0x556e8cebe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.393+0000 7f440439b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8cec9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.393+0000 7f440439b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556e8cafa790 con 0x556e8cb111b0 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.393+0000 7f440439b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e8cdbcbc0 con 0x556e8cae82f0 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.393+0000 7f440439b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556e8cec0b30 con 0x556e8caf04d0 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.393+0000 7f440439b740 10 client.0 _ll_register_callbacks cb 0x556e8cd9a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.394+0000 7f43f5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8cec9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.394+0000 7f43f5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8cec9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37394/0 (socket says 172.21.15.63:37394) 2024-05-27T08:10:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.394+0000 7f43f5e00640 1 -- 172.21.15.63:0/1023545553 learned_addr learned my addr 172.21.15.63:0/1023545553 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.394+0000 7f43f6800640 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8caf04d0 0x556e8cebe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.394+0000 7f43f5e00640 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8cebdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.394+0000 7f43f5e00640 1 -- 172.21.15.63:0/1023545553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8caf04d0 msgr2=0x556e8cebe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.394+0000 7f43f5e00640 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8caf04d0 0x556e8cebe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.394+0000 7f43f5e00640 1 -- 172.21.15.63:0/1023545553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 msgr2=0x556e8cebdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.394+0000 7f43f5e00640 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8cebdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.394+0000 7f43f5e00640 1 -- 172.21.15.63:0/1023545553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8cb29210 con 0x556e8cb111b0 2024-05-27T08:10:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.395+0000 7f43f5e00640 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8cec9e60 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f43e4032b60 tx=0x7f43e402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f766ccb51e7b9b3a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.395+0000 7f43d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.395+0000 7f43d7e00640 1 -- 172.21.15.63:0/1023545553 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4035040 con 0x556e8cb111b0 2024-05-27T08:10:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.395+0000 7f43d7e00640 1 -- 172.21.15.63:0/1023545553 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(153..153 src has 1..153) ==== 9571+0+0 (secure 0 0 0) 0x7f43e4031c80 con 0x556e8cb111b0 2024-05-27T08:10:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.395+0000 7f440439b740 1 -- 172.21.15.63:0/1023545553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e8ce26ae0 con 0x556e8cb111b0 2024-05-27T08:10:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.396+0000 7f43d7e00640 1 -- 172.21.15.63:0/1023545553 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f43e405d050 con 0x556e8cb111b0 2024-05-27T08:10:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.396+0000 7f440439b740 10 client.22413 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.396+0000 7f440439b740 1 -- 172.21.15.63:0/1023545553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556e8cbf5620 con 0x556e8cb111b0 2024-05-27T08:10:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.396+0000 7f43d7e00640 1 -- 172.21.15.63:0/1023545553 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556e8ce26ae0 con 0x556e8cb111b0 2024-05-27T08:10:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.396+0000 7f43d7e00640 1 -- 172.21.15.63:0/1023545553 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f43e403a040 con 0x556e8cb111b0 2024-05-27T08:10:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.396+0000 7f440439b740 10 client.22413 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.396+0000 7f440439b740 10 client.22413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.396+0000 7f440439b740 20 client.22413 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.397+0000 7f440439b740 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e8cacfd80 0x556e8cecfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.397+0000 7f440439b740 4 client.22413 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.397+0000 7f440439b740 1 -- 172.21.15.63:0/1023545553 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e8ce202c0 con 0x556e8cacfd80 2024-05-27T08:10:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.397+0000 7f43f6800640 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e8cacfd80 0x556e8cecfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.398+0000 7f43f6800640 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e8cacfd80 0x556e8cecfc80 crc :-1 s=READY pgs=654 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdcc8c4f08f00d5c server_cookie=d137d3590b4d09d2 in_seq=0 out_seq=0 2024-05-27T08:10:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.398+0000 7f43d7e00640 10 client.22413 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.400+0000 7f43d7e00640 1 -- 172.21.15.63:0/1023545553 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556e8ce202c0 con 0x556e8cacfd80 2024-05-27T08:10:15.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.400+0000 7f43d7e00640 10 client.22413 handle_command_reply: tid=0 2024-05-27T08:10:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.400+0000 7f440439b740 1 client.22413 shutdown 2024-05-27T08:10:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.400+0000 7f440439b740 1 -- 172.21.15.63:0/1023545553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 msgr2=0x556e8cec9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.400+0000 7f440439b740 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8cec9e60 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f43e4032b60 tx=0x7f43e402a410 comp rx=0 tx=0).stop 2024-05-27T08:10:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.401+0000 7f440439b740 1 -- 172.21.15.63:0/1023545553 shutdown_connections 2024-05-27T08:10:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.401+0000 7f440439b740 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e8cacfd80 0x556e8cecfc80 crc :-1 s=READY pgs=654 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.401+0000 7f440439b740 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8cec9e60 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.401+0000 7f440439b740 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8caf04d0 0x556e8cebe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.401+0000 7f440439b740 1 --2- 172.21.15.63:0/1023545553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8cebdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.401+0000 7f440439b740 1 -- 172.21.15.63:0/1023545553 >> 172.21.15.63:0/1023545553 conn(0x556e8cb05300 msgr2=0x556e8ceb5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.401+0000 7f43d7e00640 0 client.22413 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.401+0000 7f440439b740 1 -- 172.21.15.63:0/1023545553 shutdown_connections 2024-05-27T08:10:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.401+0000 7f440439b740 1 -- 172.21.15.63:0/1023545553 wait complete. 2024-05-27T08:10:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.401+0000 7f440439b740 20 client.22413 trim_cache size 0 max 16384 2024-05-27T08:10:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.425+0000 7f440439b740 1 Processor -- start 2024-05-27T08:10:15.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.425+0000 7f440439b740 1 -- start start 2024-05-27T08:10:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.425+0000 7f440439b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8ceb9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.425+0000 7f440439b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8caf04d0 0x556e8cec9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.425+0000 7f440439b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8ceb7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.425+0000 7f440439b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b5bb0 con 0x556e8cb111b0 2024-05-27T08:10:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.425+0000 7f440439b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b3480 con 0x556e8cae82f0 2024-05-27T08:10:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.425+0000 7f440439b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b12f0 con 0x556e8caf04d0 2024-05-27T08:10:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.426+0000 7f43f5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8ceb7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.426+0000 7f43f5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8ceb7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37404/0 (socket says 172.21.15.63:37404) 2024-05-27T08:10:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.426+0000 7f43f5e00640 1 -- 172.21.15.63:0/3816902978 learned_addr learned my addr 172.21.15.63:0/3816902978 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.426+0000 7f43f5e00640 1 --2- 172.21.15.63:0/3816902978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8ceb9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.426+0000 7f43f6800640 1 --2- 172.21.15.63:0/3816902978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8caf04d0 0x556e8cec9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.426+0000 7f43f5e00640 1 -- 172.21.15.63:0/3816902978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8caf04d0 msgr2=0x556e8cec9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.426+0000 7f43f5e00640 1 --2- 172.21.15.63:0/3816902978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8caf04d0 0x556e8cec9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.426+0000 7f43f5e00640 1 -- 172.21.15.63:0/3816902978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 msgr2=0x556e8ceb9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.426+0000 7f43f5e00640 1 --2- 172.21.15.63:0/3816902978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8ceb9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.426+0000 7f43f5e00640 1 -- 172.21.15.63:0/3816902978 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8ceb4e40 con 0x556e8cb111b0 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.427+0000 7f43f5e00640 1 --2- 172.21.15.63:0/3816902978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8ceb7710 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f43e4001eb0 tx=0x7f43e4040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb317f26f062837a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.427+0000 7f43e2a00640 1 -- 172.21.15.63:0/3816902978 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4035070 con 0x556e8cb111b0 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.427+0000 7f43e2a00640 1 -- 172.21.15.63:0/3816902978 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43e4032750 con 0x556e8cb111b0 2024-05-27T08:10:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.427+0000 7f43e2a00640 1 -- 172.21.15.63:0/3816902978 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4031ee0 con 0x556e8cb111b0 2024-05-27T08:10:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.428+0000 7f440439b740 1 -- 172.21.15.63:0/3816902978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 msgr2=0x556e8ceb7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.428+0000 7f440439b740 1 --2- 172.21.15.63:0/3816902978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8ceb7710 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f43e4001eb0 tx=0x7f43e4040040 comp rx=0 tx=0).stop 2024-05-27T08:10:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.428+0000 7f440439b740 1 -- 172.21.15.63:0/3816902978 shutdown_connections 2024-05-27T08:10:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.428+0000 7f440439b740 1 --2- 172.21.15.63:0/3816902978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8cb111b0 0x556e8ceb7710 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.428+0000 7f440439b740 1 --2- 172.21.15.63:0/3816902978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8caf04d0 0x556e8cec9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.428+0000 7f440439b740 1 --2- 172.21.15.63:0/3816902978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8ceb9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.428+0000 7f440439b740 1 -- 172.21.15.63:0/3816902978 >> 172.21.15.63:0/3816902978 conn(0x556e8cb05300 msgr2=0x556e8cebfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.428+0000 7f440439b740 1 -- 172.21.15.63:0/3816902978 shutdown_connections 2024-05-27T08:10:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.428+0000 7f440439b740 1 -- 172.21.15.63:0/3816902978 wait complete. 2024-05-27T08:10:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.429+0000 7f440439b740 1 -- start start 2024-05-27T08:10:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.429+0000 7f440439b740 1 Processor -- start 2024-05-27T08:10:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.429+0000 7f440439b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8cecfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.429+0000 7f440439b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8cecfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.429+0000 7f440439b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8cb111b0 0x556e8ced0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.429+0000 7f440439b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b1940 con 0x556e8caf04d0 2024-05-27T08:10:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f440439b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b2db0 con 0x556e8cae82f0 2024-05-27T08:10:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f440439b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b2010 con 0x556e8cb111b0 2024-05-27T08:10:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f440439b740 10 client.0 _ll_register_callbacks cb 0x7f43e800b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8cecfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8cecfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37420/0 (socket says 172.21.15.63:37420) 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f6800640 1 -- 172.21.15.63:0/835772012 learned_addr learned my addr 172.21.15.63:0/835772012 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f5e00640 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8cecfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f5e00640 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8cb111b0 0x556e8ced0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f6800640 1 -- 172.21.15.63:0/835772012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8cb111b0 msgr2=0x556e8ced0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f6800640 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8cb111b0 0x556e8ced0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f6800640 1 -- 172.21.15.63:0/835772012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 msgr2=0x556e8cecfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f6800640 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8cecfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f6800640 1 -- 172.21.15.63:0/835772012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8cf360f0 con 0x556e8caf04d0 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.430+0000 7f43f5e00640 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8cecfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.431+0000 7f43f6800640 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8cecfff0 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000c00 tx=0x7f43e800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0a4ed78b583f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.431+0000 7f43d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.431+0000 7f43d7e00640 1 -- 172.21.15.63:0/835772012 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8011440 con 0x556e8caf04d0 2024-05-27T08:10:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.431+0000 7f43d7e00640 1 -- 172.21.15.63:0/835772012 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(153..153 src has 1..153) ==== 9571+0+0 (secure 0 0 0) 0x7f43e80115e0 con 0x556e8caf04d0 2024-05-27T08:10:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.431+0000 7f440439b740 1 -- 172.21.15.63:0/835772012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43e4032750 con 0x556e8caf04d0 2024-05-27T08:10:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.431+0000 7f43d7e00640 1 -- 172.21.15.63:0/835772012 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f43e80117e0 con 0x556e8caf04d0 2024-05-27T08:10:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.431+0000 7f440439b740 10 client.22419 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.431+0000 7f440439b740 1 -- 172.21.15.63:0/835772012 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556e8cf37370 con 0x556e8caf04d0 2024-05-27T08:10:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.432+0000 7f43d7e00640 1 -- 172.21.15.63:0/835772012 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43e4032750 con 0x556e8caf04d0 2024-05-27T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.432+0000 7f43d7e00640 1 -- 172.21.15.63:0/835772012 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f43e8027a60 con 0x556e8caf04d0 2024-05-27T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.432+0000 7f440439b740 10 client.22419 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.432+0000 7f440439b740 10 client.22419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.432+0000 7f440439b740 20 client.22419 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.432+0000 7f440439b740 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e8caa51c0 0x556e8cf39b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.432+0000 7f440439b740 4 client.22419 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.432+0000 7f440439b740 1 -- 172.21.15.63:0/835772012 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e8cd9a4b0 con 0x556e8caa51c0 2024-05-27T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.433+0000 7f43f5e00640 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e8caa51c0 0x556e8cf39b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.434+0000 7f43f5e00640 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e8caa51c0 0x556e8cf39b60 crc :-1 s=READY pgs=655 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3946c5f7c5c776e server_cookie=109d698ada82f5e0 in_seq=0 out_seq=0 2024-05-27T08:10:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.434+0000 7f43d7e00640 10 client.22419 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.470+0000 7f43d7e00640 1 -- 172.21.15.63:0/835772012 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556e8cd9a4b0 con 0x556e8caa51c0 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.470+0000 7f43d7e00640 10 client.22419 handle_command_reply: tid=0 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.470+0000 7f440439b740 1 client.22419 shutdown 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 1 -- 172.21.15.63:0/835772012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 msgr2=0x556e8cecfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8cecfff0 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000c00 tx=0x7f43e800be80 comp rx=0 tx=0).stop 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 1 -- 172.21.15.63:0/835772012 shutdown_connections 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e8caa51c0 0x556e8cf39b60 crc :-1 s=READY pgs=655 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e8cb111b0 0x556e8ced0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e8caf04d0 0x556e8cecfff0 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 1 --2- 172.21.15.63:0/835772012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e8cae82f0 0x556e8cecfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 1 -- 172.21.15.63:0/835772012 >> 172.21.15.63:0/835772012 conn(0x556e8cb05300 msgr2=0x556e8cebd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f43d7e00640 0 client.22419 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 1 -- 172.21.15.63:0/835772012 shutdown_connections 2024-05-27T08:10:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 1 -- 172.21.15.63:0/835772012 wait complete. 2024-05-27T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.471+0000 7f440439b740 20 client.22419 trim_cache size 0 max 16384 2024-05-27T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:10:15.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.472+0000 7f43fca00640 1 -- 172.21.15.63:0/3373266459 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43dc0b2950 msgr2=0x7f43dc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.472+0000 7f43fca00640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43dc0b2950 0x7f43dc0b4e60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f43f000b9a0 tx=0x7f43f008ff90 comp rx=0 tx=0).stop 2024-05-27T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.472+0000 7f43fca00640 1 -- 172.21.15.63:0/3373266459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f8109f50 msgr2=0x7f43f8075a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.472+0000 7f43fca00640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f8109f50 0x7f43f8075a40 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009fd0 tx=0x7f43f000b260 comp rx=0 tx=0).stop 2024-05-27T08:10:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.473+0000 7f43fca00640 1 -- 172.21.15.63:0/3373266459 shutdown_connections 2024-05-27T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.473+0000 7f43fca00640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f43dc0b2950 0x7f43dc0b4e60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.473+0000 7f43fca00640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f8109f50 0x7f43f8075a40 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.473+0000 7f43fca00640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43f8109ab0 0x7f43f8075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.473+0000 7f43fca00640 1 --2- 172.21.15.63:0/3373266459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43f8104250 0x7f43f8074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.473+0000 7f43fca00640 1 -- 172.21.15.63:0/3373266459 >> 172.21.15.63:0/3373266459 conn(0x7f43f8079240 msgr2=0x7f43f8102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.473+0000 7f43fca00640 1 -- 172.21.15.63:0/3373266459 shutdown_connections 2024-05-27T08:10:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:15.473+0000 7f43fca00640 1 -- 172.21.15.63:0/3373266459 wait complete. 2024-05-27T08:10:16.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:15 smithi086 ceph-mon[34687]: pgmap v1163: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:10:16.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:15 smithi149 ceph-mon[35408]: pgmap v1163: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:10:16.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:15 smithi063 ceph-mon[31613]: pgmap v1163: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:10:16.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.425+0000 7fc0c2200700 1 -- 172.21.15.63:0/2512191144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106630 msgr2=0x7fc0bc106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:16.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.425+0000 7fc0c2200700 1 --2- 172.21.15.63:0/2512191144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106630 0x7fc0bc106a40 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00099c0 tx=0x7fc0b0009cd0 comp rx=0 tx=0).stop 2024-05-27T08:10:16.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.425+0000 7fc0c2200700 1 -- 172.21.15.63:0/2512191144 shutdown_connections 2024-05-27T08:10:16.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.425+0000 7fc0c2200700 1 --2- 172.21.15.63:0/2512191144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0bc10b860 0x7fc0bc1131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:16.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.425+0000 7fc0c2200700 1 --2- 172.21.15.63:0/2512191144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106630 0x7fc0bc106a40 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:16.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.425+0000 7fc0c2200700 1 --2- 172.21.15.63:0/2512191144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0bc10af10 0x7fc0bc10b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:16.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.425+0000 7fc0c2200700 1 -- 172.21.15.63:0/2512191144 >> 172.21.15.63:0/2512191144 conn(0x7fc0bc102020 msgr2=0x7fc0bc104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:16.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.425+0000 7fc0c2200700 1 -- 172.21.15.63:0/2512191144 shutdown_connections 2024-05-27T08:10:16.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.425+0000 7fc0c2200700 1 -- 172.21.15.63:0/2512191144 wait complete. 2024-05-27T08:10:16.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.426+0000 7fc0c2200700 1 Processor -- start 2024-05-27T08:10:16.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.426+0000 7fc0c2200700 1 -- start start 2024-05-27T08:10:16.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.427+0000 7fc0c2200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106630 0x7fc0bc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:16.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.427+0000 7fc0c2200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0bc10af10 0x7fc0bc19d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:16.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.427+0000 7fc0c2200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0bc10b860 0x7fc0bc1a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:16.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.427+0000 7fc0c2200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc116390 con 0x7fc0bc106630 2024-05-27T08:10:16.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.427+0000 7fc0c2200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc116210 con 0x7fc0bc10af10 2024-05-27T08:10:16.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.427+0000 7fc0c2200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc116510 con 0x7fc0bc10b860 2024-05-27T08:10:16.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0c0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106630 0x7fc0bc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:16.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0c0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106630 0x7fc0bc19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37444/0 (socket says 172.21.15.63:37444) 2024-05-27T08:10:16.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0c0e00700 1 -- 172.21.15.63:0/3624832060 learned_addr learned my addr 172.21.15.63:0/3624832060 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:16.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0bbe00700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0bc10af10 0x7fc0bc19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:16.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0c1800700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0bc10b860 0x7fc0bc1a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:16.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0c0e00700 1 -- 172.21.15.63:0/3624832060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0bc10b860 msgr2=0x7fc0bc1a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:16.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0c0e00700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0bc10b860 0x7fc0bc1a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:16.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0c0e00700 1 -- 172.21.15.63:0/3624832060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0bc10af10 msgr2=0x7fc0bc19d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:16.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0c0e00700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0bc10af10 0x7fc0bc19d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:16.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0c0e00700 1 -- 172.21.15.63:0/3624832060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc1a1c30 con 0x7fc0bc106630 2024-05-27T08:10:16.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.428+0000 7fc0c1800700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0bc10b860 0x7fc0bc1a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:16.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.429+0000 7fc0c0e00700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106630 0x7fc0bc19cd60 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0000c00 tx=0x7fc0b000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:16.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.429+0000 7fc0b9600700 1 -- 172.21.15.63:0/3624832060 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0027070 con 0x7fc0bc106630 2024-05-27T08:10:16.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.429+0000 7fc0b9600700 1 -- 172.21.15.63:0/3624832060 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0b000fda0 con 0x7fc0bc106630 2024-05-27T08:10:16.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.429+0000 7fc0c2200700 1 -- 172.21.15.63:0/3624832060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0bc1a1ec0 con 0x7fc0bc106630 2024-05-27T08:10:16.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.429+0000 7fc0b9600700 1 -- 172.21.15.63:0/3624832060 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0022e50 con 0x7fc0bc106630 2024-05-27T08:10:16.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.430+0000 7fc0c2200700 1 -- 172.21.15.63:0/3624832060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0bc1a9840 con 0x7fc0bc106630 2024-05-27T08:10:16.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.431+0000 7fc0c2200700 1 -- 172.21.15.63:0/3624832060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0bc0512b0 con 0x7fc0bc106630 2024-05-27T08:10:16.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.434+0000 7fc0b9600700 1 -- 172.21.15.63:0/3624832060 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc0b002a070 con 0x7fc0bc106630 2024-05-27T08:10:16.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.435+0000 7fc0b9600700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0a40b2c10 0x7fc0a40b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:16.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.435+0000 7fc0b9600700 1 -- 172.21.15.63:0/3624832060 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7fc0b00e3af0 con 0x7fc0bc106630 2024-05-27T08:10:16.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.436+0000 7fc0bbe00700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0a40b2c10 0x7fc0a40b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:16.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.436+0000 7fc0bbe00700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0a40b2c10 0x7fc0a40b5120 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc1098c0 tx=0x7fc0a800b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:16.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.441+0000 7fc0b9600700 1 -- 172.21.15.63:0/3624832060 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0b0013070 con 0x7fc0bc106630 2024-05-27T08:10:16.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.786+0000 7fc0c2200700 1 -- 172.21.15.63:0/3624832060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0bc19e0c0 con 0x7fc0bc106630 2024-05-27T08:10:16.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.787+0000 7fc0b9600700 1 -- 172.21.15.63:0/3624832060 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7fc0b00ab140 con 0x7fc0bc106630 2024-05-27T08:10:16.789 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:10:16.789 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:10:16.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 -- 172.21.15.63:0/3624832060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0a40b2c10 msgr2=0x7fc0a40b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:16.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0a40b2c10 0x7fc0a40b5120 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc1098c0 tx=0x7fc0a800b410 comp rx=0 tx=0).stop 2024-05-27T08:10:16.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 -- 172.21.15.63:0/3624832060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106630 msgr2=0x7fc0bc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:16.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106630 0x7fc0bc19cd60 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0000c00 tx=0x7fc0b000f460 comp rx=0 tx=0).stop 2024-05-27T08:10:16.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 -- 172.21.15.63:0/3624832060 shutdown_connections 2024-05-27T08:10:16.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc0a40b2c10 0x7fc0a40b5120 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:16.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0bc10b860 0x7fc0bc1a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:16.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0bc106630 0x7fc0bc19cd60 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:16.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 --2- 172.21.15.63:0/3624832060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0bc10af10 0x7fc0bc19d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:16.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 -- 172.21.15.63:0/3624832060 >> 172.21.15.63:0/3624832060 conn(0x7fc0bc102020 msgr2=0x7fc0bc10c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:16.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 -- 172.21.15.63:0/3624832060 shutdown_connections 2024-05-27T08:10:16.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:16.791+0000 7fc0c2200700 1 -- 172.21.15.63:0/3624832060 wait complete. 2024-05-27T08:10:16.794 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:10:17.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:17 smithi063 ceph-mon[31613]: osdmap e154: 12 total, 12 up, 12 in 2024-05-27T08:10:17.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:17 smithi063 ceph-mon[31613]: pgmap v1165: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-27T08:10:17.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:17 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3624832060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:10:17.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:17 smithi086 ceph-mon[34687]: osdmap e154: 12 total, 12 up, 12 in 2024-05-27T08:10:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:17 smithi086 ceph-mon[34687]: pgmap v1165: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-27T08:10:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:17 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3624832060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:10:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:17 smithi149 ceph-mon[35408]: osdmap e154: 12 total, 12 up, 12 in 2024-05-27T08:10:17.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:17 smithi149 ceph-mon[35408]: pgmap v1165: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 13 op/s 2024-05-27T08:10:17.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:17 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3624832060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:10:17.785 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:10:18.320 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:10:19.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:19 smithi086 ceph-mon[34687]: pgmap v1166: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:10:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:19 smithi149 ceph-mon[35408]: pgmap v1166: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:10:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:19 smithi063 ceph-mon[31613]: pgmap v1166: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:10:20.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.832+0000 7f9a00800700 1 -- 172.21.15.63:0/3590012688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f8107de0 msgr2=0x7f99f81081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:20.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.832+0000 7f9a00800700 1 --2- 172.21.15.63:0/3590012688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f8107de0 0x7f99f81081f0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f99e80099c0 tx=0x7f99e8009cd0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.832+0000 7f9a00800700 1 -- 172.21.15.63:0/3590012688 shutdown_connections 2024-05-27T08:10:20.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.832+0000 7f9a00800700 1 --2- 172.21.15.63:0/3590012688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f8108730 0x7f99f8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.832+0000 7f9a00800700 1 --2- 172.21.15.63:0/3590012688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f81034b0 0x7f99f81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.832+0000 7f9a00800700 1 --2- 172.21.15.63:0/3590012688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f8107de0 0x7f99f81081f0 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.832+0000 7f9a00800700 1 -- 172.21.15.63:0/3590012688 >> 172.21.15.63:0/3590012688 conn(0x7f99f80fede0 msgr2=0x7f99f8101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:20.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.832+0000 7f9a00800700 1 -- 172.21.15.63:0/3590012688 shutdown_connections 2024-05-27T08:10:20.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.833+0000 7f9a00800700 1 -- 172.21.15.63:0/3590012688 wait complete. 2024-05-27T08:10:20.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.834+0000 7f9a00800700 1 Processor -- start 2024-05-27T08:10:20.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.834+0000 7f9a00800700 1 -- start start 2024-05-27T08:10:20.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.835+0000 7f9a00800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f81034b0 0x7f99f819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.835+0000 7f9a00800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f8107de0 0x7f99f819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.835+0000 7f9a00800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f8108730 0x7f99f819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.835+0000 7f9a00800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99f8113240 con 0x7f99f8107de0 2024-05-27T08:10:20.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.835+0000 7f9a00800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99f81130c0 con 0x7f99f81034b0 2024-05-27T08:10:20.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.835+0000 7f9a00800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99f81133c0 con 0x7f99f8108730 2024-05-27T08:10:20.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.835+0000 7f99fd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f8107de0 0x7f99f819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.835+0000 7f99fd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f8107de0 0x7f99f819b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40408/0 (socket says 172.21.15.63:40408) 2024-05-27T08:10:20.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.835+0000 7f99fd400700 1 -- 172.21.15.63:0/3818389151 learned_addr learned my addr 172.21.15.63:0/3818389151 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.836+0000 7f99fde00700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f81034b0 0x7f99f819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.836+0000 7f99fe800700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f8108730 0x7f99f819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.836+0000 7f99fd400700 1 -- 172.21.15.63:0/3818389151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f8108730 msgr2=0x7f99f819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.836+0000 7f99fd400700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f8108730 0x7f99f819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.836+0000 7f99fd400700 1 -- 172.21.15.63:0/3818389151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f81034b0 msgr2=0x7f99f819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.836+0000 7f99fd400700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f81034b0 0x7f99f819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.836+0000 7f99fd400700 1 -- 172.21.15.63:0/3818389151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f819fc30 con 0x7f99f8107de0 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.836+0000 7f99fde00700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f81034b0 0x7f99f819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.836+0000 7f99fd400700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f8107de0 0x7f99f819b300 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f99e80098e0 tx=0x7f99e800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.837+0000 7f99f2a00700 1 -- 172.21.15.63:0/3818389151 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8027070 con 0x7f99f8107de0 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.837+0000 7f99f2a00700 1 -- 172.21.15.63:0/3818389151 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99e800fdd0 con 0x7f99f8107de0 2024-05-27T08:10:20.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.837+0000 7f9a00800700 1 -- 172.21.15.63:0/3818389151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f819fec0 con 0x7f99f8107de0 2024-05-27T08:10:20.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.837+0000 7f99f2a00700 1 -- 172.21.15.63:0/3818389151 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8022e30 con 0x7f99f8107de0 2024-05-27T08:10:20.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.837+0000 7f9a00800700 1 -- 172.21.15.63:0/3818389151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f81a03a0 con 0x7f99f8107de0 2024-05-27T08:10:20.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.841+0000 7f99f2a00700 1 -- 172.21.15.63:0/3818389151 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f99e802a070 con 0x7f99f8107de0 2024-05-27T08:10:20.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.842+0000 7f99f2a00700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99e00b29f0 0x7f99e00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.843+0000 7f99f2a00700 1 -- 172.21.15.63:0/3818389151 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f99e80e3750 con 0x7f99f8107de0 2024-05-27T08:10:20.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.843+0000 7f99fde00700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99e00b29f0 0x7f99e00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.844+0000 7f99fde00700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99e00b29f0 0x7f99e00b4f00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f99ec005950 tx=0x7f99ec0058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:20.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.883+0000 7f9a0f3deb80 1 Processor -- start 2024-05-27T08:10:20.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.883+0000 7f9a0f3deb80 1 -- start start 2024-05-27T08:10:20.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.883+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10539f0 0x5612b10e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.883+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10e6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.884+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10e6c50 0x5612b10fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.884+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5612b1014e40 con 0x5612b0f67e40 2024-05-27T08:10:20.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.884+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612b10049e0 con 0x5612b10e6c50 2024-05-27T08:10:20.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.884+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5612b1008a20 con 0x5612b10539f0 2024-05-27T08:10:20.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.884+0000 7f99fd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.884+0000 7f99fd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10e6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40414/0 (socket says 172.21.15.63:40414) 2024-05-27T08:10:20.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.884+0000 7f99fd400700 1 -- 172.21.15.63:0/448635319 learned_addr learned my addr 172.21.15.63:0/448635319 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:20.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.884+0000 7f99fde00700 1 --2- 172.21.15.63:0/448635319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10539f0 0x5612b10e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.884+0000 7f99fe800700 1 --2- 172.21.15.63:0/448635319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10e6c50 0x5612b10fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.885+0000 7f99fd400700 1 -- 172.21.15.63:0/448635319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10539f0 msgr2=0x5612b10e61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:20.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.885+0000 7f99fd400700 1 --2- 172.21.15.63:0/448635319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10539f0 0x5612b10e61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.885+0000 7f99fd400700 1 -- 172.21.15.63:0/448635319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10e6c50 msgr2=0x5612b10fbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:20.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.885+0000 7f99fd400700 1 --2- 172.21.15.63:0/448635319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10e6c50 0x5612b10fbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.885+0000 7f99fd400700 1 -- 172.21.15.63:0/448635319 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b0fcd420 con 0x5612b0f67e40 2024-05-27T08:10:20.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.885+0000 7f99fd400700 1 --2- 172.21.15.63:0/448635319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10e6710 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f99e8024000 tx=0x7f99e80a6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b12ed3702dd1aea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:20.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.886+0000 7f99e7400700 1 -- 172.21.15.63:0/448635319 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e80aa6a0 con 0x5612b0f67e40 2024-05-27T08:10:20.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.886+0000 7f99e7400700 1 -- 172.21.15.63:0/448635319 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99e80aea50 con 0x5612b0f67e40 2024-05-27T08:10:20.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.886+0000 7f99e7400700 1 -- 172.21.15.63:0/448635319 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e80afbe0 con 0x5612b0f67e40 2024-05-27T08:10:20.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.887+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/448635319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 msgr2=0x5612b10e6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:20.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.887+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/448635319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10e6710 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f99e8024000 tx=0x7f99e80a6690 comp rx=0 tx=0).stop 2024-05-27T08:10:20.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.887+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/448635319 shutdown_connections 2024-05-27T08:10:20.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.887+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/448635319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10e6c50 0x5612b10fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.887+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/448635319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10539f0 0x5612b10e61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.887+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/448635319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10e6710 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.887+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/448635319 >> 172.21.15.63:0/448635319 conn(0x5612b1001390 msgr2=0x5612b10e24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:20.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.887+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/448635319 shutdown_connections 2024-05-27T08:10:20.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.887+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/448635319 wait complete. 2024-05-27T08:10:20.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.888+0000 7f9a0f3deb80 1 -- start start 2024-05-27T08:10:20.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.888+0000 7f9a0f3deb80 1 Processor -- start 2024-05-27T08:10:20.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b0f67e40 0x5612b10f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b10539f0 0x5612b10f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10e6c50 0x5612b10f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5612b0f0afa0 con 0x5612b10539f0 2024-05-27T08:10:20.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612b0fa5760 con 0x5612b10e6c50 2024-05-27T08:10:20.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5612b10fc230 con 0x5612b0f67e40 2024-05-27T08:10:20.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f9a0f3deb80 10 client.0 _ll_register_callbacks cb 0x5612b0faec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:20.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f99fd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b10539f0 0x5612b10f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f99fd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b10539f0 0x5612b10f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40428/0 (socket says 172.21.15.63:40428) 2024-05-27T08:10:20.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f99fd400700 1 -- 172.21.15.63:0/2539596995 learned_addr learned my addr 172.21.15.63:0/2539596995 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:20.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.889+0000 7f99fde00700 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b0f67e40 0x5612b10f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.890+0000 7f99fd400700 1 -- 172.21.15.63:0/2539596995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b0f67e40 msgr2=0x5612b10f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:20.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.890+0000 7f99fd400700 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b0f67e40 0x5612b10f91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.890+0000 7f99fd400700 1 -- 172.21.15.63:0/2539596995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10e6c50 msgr2=0x5612b10f9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:20.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.890+0000 7f99fd400700 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10e6c50 0x5612b10f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.890+0000 7f99fd400700 1 -- 172.21.15.63:0/2539596995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b0f548d0 con 0x5612b10539f0 2024-05-27T08:10:20.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.890+0000 7f99fde00700 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b0f67e40 0x5612b10f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:20.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.891+0000 7f99fd400700 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b10539f0 0x5612b10f9730 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f99e80a7910 tx=0x7f99e8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2412e2f9179baf47 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:20.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.891+0000 7f99dbe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:20.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.891+0000 7f99dbe00700 1 -- 172.21.15.63:0/2539596995 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e80b3470 con 0x5612b10539f0 2024-05-27T08:10:20.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.891+0000 7f99dbe00700 1 -- 172.21.15.63:0/2539596995 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f99e80a6c90 con 0x5612b10539f0 2024-05-27T08:10:20.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.891+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2539596995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e80aea50 con 0x5612b10539f0 2024-05-27T08:10:20.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.892+0000 7f99dbe00700 1 -- 172.21.15.63:0/2539596995 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f99e80afdf0 con 0x5612b10539f0 2024-05-27T08:10:20.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.892+0000 7f9a0f3deb80 10 client.22437 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:20.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.892+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2539596995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5612b0fd5060 con 0x5612b10539f0 2024-05-27T08:10:20.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.892+0000 7f99dbe00700 1 -- 172.21.15.63:0/2539596995 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99e80aea50 con 0x5612b10539f0 2024-05-27T08:10:20.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.892+0000 7f99dbe00700 1 -- 172.21.15.63:0/2539596995 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f99e80b2fe0 con 0x5612b10539f0 2024-05-27T08:10:20.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.892+0000 7f9a0f3deb80 10 client.22437 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:20.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.892+0000 7f9a0f3deb80 10 client.22437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:10:20.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.892+0000 7f9a0f3deb80 20 client.22437 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:20.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.893+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5612b1108ec0 0x5612b110b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.893+0000 7f9a0f3deb80 4 client.22437 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:20.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.893+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2539596995 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612b1016d90 con 0x5612b1108ec0 2024-05-27T08:10:20.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.893+0000 7f99fde00700 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5612b1108ec0 0x5612b110b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.894+0000 7f99fde00700 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5612b1108ec0 0x5612b110b2e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52d860802beec093 server_cookie=ea34864b13eaf50d in_seq=0 out_seq=0 2024-05-27T08:10:20.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.894+0000 7f99dbe00700 10 client.22437 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:10:20.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.896+0000 7f99dbe00700 1 -- 172.21.15.63:0/2539596995 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5612b1016d90 con 0x5612b1108ec0 2024-05-27T08:10:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.896+0000 7f99dbe00700 10 client.22437 handle_command_reply: tid=0 2024-05-27T08:10:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.896+0000 7f9a0f3deb80 1 client.22437 shutdown 2024-05-27T08:10:20.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2539596995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b10539f0 msgr2=0x5612b10f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b10539f0 0x5612b10f9730 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f99e80a7910 tx=0x7f99e8009cd0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2539596995 shutdown_connections 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10e6c50 0x5612b10f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b0f67e40 0x5612b10f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5612b1108ec0 0x5612b110b2e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2539596995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b10539f0 0x5612b10f9730 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f99dbe00700 0 client.22437 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2539596995 >> 172.21.15.63:0/2539596995 conn(0x5612b1001390 msgr2=0x5612b10e61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2539596995 shutdown_connections 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2539596995 wait complete. 2024-05-27T08:10:20.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.897+0000 7f9a0f3deb80 20 client.22437 trim_cache size 0 max 16384 2024-05-27T08:10:20.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.914+0000 7f9a0f3deb80 1 Processor -- start 2024-05-27T08:10:20.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.915+0000 7f9a0f3deb80 1 -- start start 2024-05-27T08:10:20.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.915+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10ebd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.915+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10539f0 0x5612b10f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.915+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 0x5612b11051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.915+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99f81b0b30 con 0x5612b0f67e40 2024-05-27T08:10:20.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.915+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99f81ae3c0 con 0x5612b10539f0 2024-05-27T08:10:20.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.915+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99f81ac250 con 0x5612b10e6c50 2024-05-27T08:10:20.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.916+0000 7f99fde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10ebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.916+0000 7f99fd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10539f0 0x5612b10f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.916+0000 7f99fde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10ebd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40440/0 (socket says 172.21.15.63:40440) 2024-05-27T08:10:20.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.916+0000 7f99fde00700 1 -- 172.21.15.63:0/731979779 learned_addr learned my addr 172.21.15.63:0/731979779 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:20.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.916+0000 7f99fe800700 1 --2- 172.21.15.63:0/731979779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 0x5612b11051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.917+0000 7f99fde00700 1 -- 172.21.15.63:0/731979779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 msgr2=0x5612b11051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:20.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.917+0000 7f99fde00700 1 --2- 172.21.15.63:0/731979779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 0x5612b11051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.917+0000 7f99fde00700 1 -- 172.21.15.63:0/731979779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10539f0 msgr2=0x5612b10f6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:20.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.917+0000 7f99fde00700 1 --2- 172.21.15.63:0/731979779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10539f0 0x5612b10f6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.917+0000 7f99fde00700 1 -- 172.21.15.63:0/731979779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b10e3830 con 0x5612b0f67e40 2024-05-27T08:10:20.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.917+0000 7f99fde00700 1 --2- 172.21.15.63:0/731979779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10ebd80 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x5612b10e8fa0 tx=0x7f99ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c869de649b6b149d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:20.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.918+0000 7f99e6a00700 1 -- 172.21.15.63:0/731979779 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec02f070 con 0x5612b0f67e40 2024-05-27T08:10:20.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.918+0000 7f99e6a00700 1 -- 172.21.15.63:0/731979779 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99ec003b10 con 0x5612b0f67e40 2024-05-27T08:10:20.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.918+0000 7f99e6a00700 1 -- 172.21.15.63:0/731979779 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec029810 con 0x5612b0f67e40 2024-05-27T08:10:20.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.919+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/731979779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 msgr2=0x5612b10ebd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:20.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.919+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/731979779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10ebd80 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x5612b10e8fa0 tx=0x7f99ec015000 comp rx=0 tx=0).stop 2024-05-27T08:10:20.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.919+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/731979779 shutdown_connections 2024-05-27T08:10:20.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.919+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/731979779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 0x5612b11051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.919+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/731979779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b10ebd80 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.919+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/731979779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10539f0 0x5612b10f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.919+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/731979779 >> 172.21.15.63:0/731979779 conn(0x5612b1001390 msgr2=0x5612b10f7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:20.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.919+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/731979779 shutdown_connections 2024-05-27T08:10:20.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.919+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/731979779 wait complete. 2024-05-27T08:10:20.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.920+0000 7f9a0f3deb80 1 -- start start 2024-05-27T08:10:20.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.920+0000 7f9a0f3deb80 1 Processor -- start 2024-05-27T08:10:20.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.921+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b110fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.921+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10539f0 0x5612b1110210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.921+0000 7f9a0f3deb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 0x5612b1100380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.921+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99f81ac8a0 con 0x5612b0f67e40 2024-05-27T08:10:20.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.921+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f99f81adcf0 con 0x5612b10539f0 2024-05-27T08:10:20.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.921+0000 7f9a0f3deb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f99f81acf50 con 0x5612b10e6c50 2024-05-27T08:10:20.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.921+0000 7f9a0f3deb80 10 client.0 _ll_register_callbacks cb 0x5612b0faec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:20.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.921+0000 7f99fde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b110fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.921+0000 7f99fde00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b110fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40452/0 (socket says 172.21.15.63:40452) 2024-05-27T08:10:20.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.921+0000 7f99fde00700 1 -- 172.21.15.63:0/2615446722 learned_addr learned my addr 172.21.15.63:0/2615446722 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:20.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.922+0000 7f99fd400700 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10539f0 0x5612b1110210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.922+0000 7f99fe800700 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 0x5612b1100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.922+0000 7f99fde00700 1 -- 172.21.15.63:0/2615446722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 msgr2=0x5612b1100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:20.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.922+0000 7f99fde00700 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 0x5612b1100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.922+0000 7f99fde00700 1 -- 172.21.15.63:0/2615446722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10539f0 msgr2=0x5612b1110210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:20.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.922+0000 7f99fde00700 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10539f0 0x5612b1110210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.922+0000 7f99fde00700 1 -- 172.21.15.63:0/2615446722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b11008c0 con 0x5612b0f67e40 2024-05-27T08:10:20.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.922+0000 7f99fe800700 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 0x5612b1100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:10:20.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.922+0000 7f99fde00700 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b110fc70 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f99ec02d040 tx=0x7f99ec004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff49a0a2787f6e78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:20.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.922+0000 7f99dbe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:20.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.923+0000 7f99dbe00700 1 -- 172.21.15.63:0/2615446722 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec02f070 con 0x5612b0f67e40 2024-05-27T08:10:20.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.923+0000 7f99dbe00700 1 -- 172.21.15.63:0/2615446722 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f99ec003880 con 0x5612b0f67e40 2024-05-27T08:10:20.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.923+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2615446722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99ec003b10 con 0x5612b0f67e40 2024-05-27T08:10:20.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.923+0000 7f99dbe00700 1 -- 172.21.15.63:0/2615446722 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f99ec03e050 con 0x5612b0f67e40 2024-05-27T08:10:20.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.923+0000 7f9a0f3deb80 10 client.22443 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:20.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.923+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2615446722 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5612b1101b40 con 0x5612b0f67e40 2024-05-27T08:10:20.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.923+0000 7f99dbe00700 1 -- 172.21.15.63:0/2615446722 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99ec003b10 con 0x5612b0f67e40 2024-05-27T08:10:20.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.924+0000 7f99dbe00700 1 -- 172.21.15.63:0/2615446722 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f99ec0114d0 con 0x5612b0f67e40 2024-05-27T08:10:20.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.924+0000 7f9a0f3deb80 10 client.22443 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:20.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.924+0000 7f9a0f3deb80 10 client.22443 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:10:20.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.924+0000 7f9a0f3deb80 20 client.22443 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:20.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.924+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5612b1173970 0x5612b1175d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:20.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.924+0000 7f9a0f3deb80 4 client.22443 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:10:20.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.924+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2615446722 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612b1010570 con 0x5612b1173970 2024-05-27T08:10:20.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.925+0000 7f99fd400700 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5612b1173970 0x5612b1175d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:20.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.926+0000 7f99fd400700 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5612b1173970 0x5612b1175d90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e519f5988231c4b8 server_cookie=7546e8522f631fe4 in_seq=0 out_seq=0 2024-05-27T08:10:20.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.926+0000 7f99dbe00700 10 client.22443 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.926+0000 7f99dbe00700 1 -- 172.21.15.63:0/2615446722 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612b1010570 con 0x5612b1173970 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.926+0000 7f99dbe00700 10 client.22443 handle_command_reply: tid=0 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 client.22443 shutdown 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2615446722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 msgr2=0x5612b110fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b110fc70 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f99ec02d040 tx=0x7f99ec004660 comp rx=0 tx=0).stop 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2615446722 shutdown_connections 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5612b10e6c50 0x5612b1100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5612b1173970 0x5612b1175d90 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5612b0f67e40 0x5612b110fc70 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 --2- 172.21.15.63:0/2615446722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612b10539f0 0x5612b1110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2615446722 >> 172.21.15.63:0/2615446722 conn(0x5612b1001390 msgr2=0x5612b10e7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:20.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f99dbe00700 0 client.22443 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:10:20.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2615446722 shutdown_connections 2024-05-27T08:10:20.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 1 -- 172.21.15.63:0/2615446722 wait complete. 2024-05-27T08:10:20.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.927+0000 7f9a0f3deb80 20 client.22443 trim_cache size 0 max 16384 2024-05-27T08:10:20.930 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:10:20.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.928+0000 7f9a00800700 1 -- 172.21.15.63:0/3818389151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99e00b29f0 msgr2=0x7f99e00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:20.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.928+0000 7f9a00800700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99e00b29f0 0x7f99e00b4f00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f99ec005950 tx=0x7f99ec0058e0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.928+0000 7f9a00800700 1 -- 172.21.15.63:0/3818389151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f8107de0 msgr2=0x7f99f819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:20.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.928+0000 7f9a00800700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f8107de0 0x7f99f819b300 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f99e80098e0 tx=0x7f99e800f510 comp rx=0 tx=0).stop 2024-05-27T08:10:20.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.929+0000 7f9a00800700 1 -- 172.21.15.63:0/3818389151 shutdown_connections 2024-05-27T08:10:20.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.929+0000 7f9a00800700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99e00b29f0 0x7f99e00b4f00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.929+0000 7f9a00800700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99f8108730 0x7f99f819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.929+0000 7f9a00800700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f81034b0 0x7f99f819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.929+0000 7f9a00800700 1 --2- 172.21.15.63:0/3818389151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99f8107de0 0x7f99f819b300 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:20.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.929+0000 7f9a00800700 1 -- 172.21.15.63:0/3818389151 >> 172.21.15.63:0/3818389151 conn(0x7f99f80fede0 msgr2=0x7f99f8075dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:20.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.929+0000 7f9a00800700 1 -- 172.21.15.63:0/3818389151 shutdown_connections 2024-05-27T08:10:20.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:20.929+0000 7f9a00800700 1 -- 172.21.15.63:0/3818389151 wait complete. 2024-05-27T08:10:21.866 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:21 smithi063 ceph-mon[31613]: pgmap v1167: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:10:21.951 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:10:21.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:21 smithi086 ceph-mon[34687]: pgmap v1167: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:10:21.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:21 smithi149 ceph-mon[35408]: pgmap v1167: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-05-27T08:10:22.521 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:10:23.925 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:23 smithi063 ceph-mon[31613]: pgmap v1168: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:10:23.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:23 smithi086 ceph-mon[34687]: pgmap v1168: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:10:23.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:23 smithi149 ceph-mon[35408]: pgmap v1168: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:10:25.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.055+0000 7f53ede00700 1 -- 172.21.15.63:0/3364520119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8110ad0 msgr2=0x7f53e8112ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:25.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.055+0000 7f53ede00700 1 --2- 172.21.15.63:0/3364520119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8110ad0 0x7f53e8112ef0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f53e000b0a0 tx=0x7f53e000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.055+0000 7f53ede00700 1 -- 172.21.15.63:0/3364520119 shutdown_connections 2024-05-27T08:10:25.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.055+0000 7f53ede00700 1 --2- 172.21.15.63:0/3364520119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8110ad0 0x7f53e8112ef0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.055+0000 7f53ede00700 1 --2- 172.21.15.63:0/3364520119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8102f20 0x7f53e8103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.055+0000 7f53ede00700 1 --2- 172.21.15.63:0/3364520119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8103870 0x7f53e8110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.055+0000 7f53ede00700 1 -- 172.21.15.63:0/3364520119 >> 172.21.15.63:0/3364520119 conn(0x7f53e8078ab0 msgr2=0x7f53e8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:25.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.056+0000 7f53ede00700 1 -- 172.21.15.63:0/3364520119 shutdown_connections 2024-05-27T08:10:25.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.056+0000 7f53ede00700 1 -- 172.21.15.63:0/3364520119 wait complete. 2024-05-27T08:10:25.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.057+0000 7f53ede00700 1 Processor -- start 2024-05-27T08:10:25.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.057+0000 7f53ede00700 1 -- start start 2024-05-27T08:10:25.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.058+0000 7f53ede00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8102f20 0x7f53e819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.058+0000 7f53ede00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8103870 0x7f53e819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.058+0000 7f53ede00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8110ad0 0x7f53e81a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.058+0000 7f53ede00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81163a0 con 0x7f53e8110ad0 2024-05-27T08:10:25.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.058+0000 7f53ede00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8116220 con 0x7f53e8103870 2024-05-27T08:10:25.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.058+0000 7f53ede00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8116520 con 0x7f53e8102f20 2024-05-27T08:10:25.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.059+0000 7f53ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8110ad0 0x7f53e81a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.059+0000 7f53ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8110ad0 0x7f53e81a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40470/0 (socket says 172.21.15.63:40470) 2024-05-27T08:10:25.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.059+0000 7f53ed400700 1 -- 172.21.15.63:0/3346963905 learned_addr learned my addr 172.21.15.63:0/3346963905 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:25.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.059+0000 7f53eca00700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8102f20 0x7f53e819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.059+0000 7f53e7e00700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8103870 0x7f53e819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.059+0000 7f53ed400700 1 -- 172.21.15.63:0/3346963905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8102f20 msgr2=0x7f53e819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:25.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.059+0000 7f53ed400700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8102f20 0x7f53e819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.059+0000 7f53ed400700 1 -- 172.21.15.63:0/3346963905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8103870 msgr2=0x7f53e819d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:25.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.059+0000 7f53ed400700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8103870 0x7f53e819d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.059+0000 7f53ed400700 1 -- 172.21.15.63:0/3346963905 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e81a1be0 con 0x7f53e8110ad0 2024-05-27T08:10:25.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.060+0000 7f53ed400700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8110ad0 0x7f53e81a1540 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f53e000b070 tx=0x7f53e0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:25.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.060+0000 7f53e5600700 1 -- 172.21.15.63:0/3346963905 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0021070 con 0x7f53e8110ad0 2024-05-27T08:10:25.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.060+0000 7f53ede00700 1 -- 172.21.15.63:0/3346963905 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e81a1e70 con 0x7f53e8110ad0 2024-05-27T08:10:25.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.060+0000 7f53e5600700 1 -- 172.21.15.63:0/3346963905 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53e0007b80 con 0x7f53e8110ad0 2024-05-27T08:10:25.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.061+0000 7f53ede00700 1 -- 172.21.15.63:0/3346963905 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e81a97f0 con 0x7f53e8110ad0 2024-05-27T08:10:25.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.061+0000 7f53e5600700 1 -- 172.21.15.63:0/3346963905 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0004490 con 0x7f53e8110ad0 2024-05-27T08:10:25.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.066+0000 7f53e5600700 1 -- 172.21.15.63:0/3346963905 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f53e0004630 con 0x7f53e8110ad0 2024-05-27T08:10:25.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.068+0000 7f53e5600700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 0x7f53d00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.068+0000 7f53e5600700 1 -- 172.21.15.63:0/3346963905 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f53e00e3c30 con 0x7f53e8110ad0 2024-05-27T08:10:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.068+0000 7f53fdef4b80 1 Processor -- start 2024-05-27T08:10:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.068+0000 7f53eca00700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 0x7f53d00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.068+0000 7f53fdef4b80 1 -- start start 2024-05-27T08:10:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.069+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 0x5577e7f99ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.069+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.069+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7f9a820 0x5577e7fa7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.069+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5577e7ec8e40 con 0x5577e7f9a820 2024-05-27T08:10:25.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.069+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577e7eb89e0 con 0x5577e7e1be40 2024-05-27T08:10:25.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.069+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5577e7ebca20 con 0x5577e7f9a410 2024-05-27T08:10:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.069+0000 7f53eca00700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 0x7f53d00b4f00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009c60 tx=0x7f53dc009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.069+0000 7f53ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7f9a820 0x5577e7fa7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.069+0000 7f53ed400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7f9a820 0x5577e7fa7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40476/0 (socket says 172.21.15.63:40476) 2024-05-27T08:10:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.070+0000 7f53ed400700 1 -- 172.21.15.63:0/2224477779 learned_addr learned my addr 172.21.15.63:0/2224477779 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.070+0000 7f53eca00700 1 --2- 172.21.15.63:0/2224477779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 0x5577e7f99ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.070+0000 7f53e7e00700 1 --2- 172.21.15.63:0/2224477779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.070+0000 7f53ed400700 1 -- 172.21.15.63:0/2224477779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 msgr2=0x5577e7fa4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.070+0000 7f53ed400700 1 --2- 172.21.15.63:0/2224477779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.070+0000 7f53ed400700 1 -- 172.21.15.63:0/2224477779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 msgr2=0x5577e7f99ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.070+0000 7f53ed400700 1 --2- 172.21.15.63:0/2224477779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 0x5577e7f99ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.070+0000 7f53ed400700 1 -- 172.21.15.63:0/2224477779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e7e81420 con 0x5577e7f9a820 2024-05-27T08:10:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.070+0000 7f53eca00700 1 --2- 172.21.15.63:0/2224477779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 0x5577e7f99ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.070+0000 7f53ed400700 1 --2- 172.21.15.63:0/2224477779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7f9a820 0x5577e7fa7020 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f53e00a6c00 tx=0x7f53e0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e89daccbed13bc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.071+0000 7f53da000700 1 -- 172.21.15.63:0/2224477779 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00b32a0 con 0x5577e7f9a820 2024-05-27T08:10:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.071+0000 7f53da000700 1 -- 172.21.15.63:0/2224477779 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53e0011e50 con 0x5577e7f9a820 2024-05-27T08:10:25.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.071+0000 7f53da000700 1 -- 172.21.15.63:0/2224477779 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00a6e10 con 0x5577e7f9a820 2024-05-27T08:10:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.072+0000 7f53fdef4b80 1 -- 172.21.15.63:0/2224477779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7f9a820 msgr2=0x5577e7fa7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.072+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/2224477779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7f9a820 0x5577e7fa7020 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f53e00a6c00 tx=0x7f53e0004f90 comp rx=0 tx=0).stop 2024-05-27T08:10:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.072+0000 7f53fdef4b80 1 -- 172.21.15.63:0/2224477779 shutdown_connections 2024-05-27T08:10:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.072+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/2224477779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7f9a820 0x5577e7fa7020 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.072+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/2224477779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 0x5577e7f99ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.072+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/2224477779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.072+0000 7f53fdef4b80 1 -- 172.21.15.63:0/2224477779 >> 172.21.15.63:0/2224477779 conn(0x5577e7e7bfc0 msgr2=0x5577e7f95d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.072+0000 7f53fdef4b80 1 -- 172.21.15.63:0/2224477779 shutdown_connections 2024-05-27T08:10:25.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.072+0000 7f53fdef4b80 1 -- 172.21.15.63:0/2224477779 wait complete. 2024-05-27T08:10:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.073+0000 7f53fdef4b80 1 -- start start 2024-05-27T08:10:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.073+0000 7f53fdef4b80 1 Processor -- start 2024-05-27T08:10:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e1be40 0x5577e7fa45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7f9a820 0x5577e7fa5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5577e7dbefa0 con 0x5577e7e1be40 2024-05-27T08:10:25.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577e7e59760 con 0x5577e7f9a820 2024-05-27T08:10:25.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5577e7fa7750 con 0x5577e7f9a410 2024-05-27T08:10:25.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53fdef4b80 10 client.0 _ll_register_callbacks cb 0x5577e7e62c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:25.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53eca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e1be40 0x5577e7fa45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53ed400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7f9a820 0x5577e7fa5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53e7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53eca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e1be40 0x5577e7fa45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40486/0 (socket says 172.21.15.63:40486) 2024-05-27T08:10:25.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.074+0000 7f53eca00700 1 -- 172.21.15.63:0/1251783447 learned_addr learned my addr 172.21.15.63:0/1251783447 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:25.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.075+0000 7f53eca00700 1 -- 172.21.15.63:0/1251783447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 msgr2=0x5577e7fa4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:25.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.075+0000 7f53eca00700 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.075+0000 7f53eca00700 1 -- 172.21.15.63:0/1251783447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7f9a820 msgr2=0x5577e7fa5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:25.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.075+0000 7f53eca00700 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7f9a820 0x5577e7fa5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.075+0000 7f53eca00700 1 -- 172.21.15.63:0/1251783447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e7e088d0 con 0x5577e7e1be40 2024-05-27T08:10:25.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.075+0000 7f53eca00700 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e1be40 0x5577e7fa45a0 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f53dc01ec90 tx=0x7f53dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26c1dff669e3528 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:25.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.075+0000 7f53cea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:25.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.076+0000 7f53cea00700 1 -- 172.21.15.63:0/1251783447 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00ec50 con 0x5577e7e1be40 2024-05-27T08:10:25.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.076+0000 7f53cea00700 1 -- 172.21.15.63:0/1251783447 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f53dc00edf0 con 0x5577e7e1be40 2024-05-27T08:10:25.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.076+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1251783447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e0011e50 con 0x5577e7e1be40 2024-05-27T08:10:25.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.076+0000 7f53cea00700 1 -- 172.21.15.63:0/1251783447 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f53dc032050 con 0x5577e7e1be40 2024-05-27T08:10:25.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.076+0000 7f53fdef4b80 10 client.22455 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:25.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.076+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1251783447 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5577e7e89060 con 0x5577e7e1be40 2024-05-27T08:10:25.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.077+0000 7f53cea00700 1 -- 172.21.15.63:0/1251783447 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53e0011e50 con 0x5577e7e1be40 2024-05-27T08:10:25.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.077+0000 7f53cea00700 1 -- 172.21.15.63:0/1251783447 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f53dc01b390 con 0x5577e7e1be40 2024-05-27T08:10:25.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.077+0000 7f53fdef4b80 10 client.22455 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:25.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.077+0000 7f53fdef4b80 10 client.22455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:10:25.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.077+0000 7f53fdef4b80 20 client.22455 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:25.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.077+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5577e7fb4730 0x5577e7fb6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.077+0000 7f53fdef4b80 4 client.22455 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:25.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.077+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1251783447 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577e7ecad90 con 0x5577e7fb4730 2024-05-27T08:10:25.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.078+0000 7f53e7e00700 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5577e7fb4730 0x5577e7fb6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.079+0000 7f53e7e00700 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5577e7fb4730 0x5577e7fb6ba0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=207cfa6ed8479410 server_cookie=dfa3933959a27102 in_seq=0 out_seq=0 2024-05-27T08:10:25.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.079+0000 7f53cea00700 10 client.22455 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:10:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.081+0000 7f53cea00700 1 -- 172.21.15.63:0/1251783447 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5577e7ecad90 con 0x5577e7fb4730 2024-05-27T08:10:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.081+0000 7f53cea00700 10 client.22455 handle_command_reply: tid=0 2024-05-27T08:10:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.081+0000 7f53fdef4b80 1 client.22455 shutdown 2024-05-27T08:10:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1251783447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e1be40 msgr2=0x5577e7fa45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e1be40 0x5577e7fa45a0 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f53dc01ec90 tx=0x7f53dc0047c0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1251783447 shutdown_connections 2024-05-27T08:10:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7f9a820 0x5577e7fa5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5577e7fb4730 0x5577e7fb6ba0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e1be40 0x5577e7fa45a0 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1251783447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53cea00700 0 client.22455 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:10:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1251783447 >> 172.21.15.63:0/1251783447 conn(0x5577e7e7bfc0 msgr2=0x5577e7f94ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1251783447 shutdown_connections 2024-05-27T08:10:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1251783447 wait complete. 2024-05-27T08:10:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.082+0000 7f53fdef4b80 20 client.22455 trim_cache size 0 max 16384 2024-05-27T08:10:25.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.106+0000 7f53fdef4b80 1 Processor -- start 2024-05-27T08:10:25.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.106+0000 7f53fdef4b80 1 -- start start 2024-05-27T08:10:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.106+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.106+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7e1be40 0x5577e7fa6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.106+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7f9a410 0x5577e7fb07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.106+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81b4cb0 con 0x5577e7e7bfc0 2024-05-27T08:10:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.106+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81b2540 con 0x5577e7f9a410 2024-05-27T08:10:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.106+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81b03b0 con 0x5577e7e1be40 2024-05-27T08:10:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53eca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53eca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40498/0 (socket says 172.21.15.63:40498) 2024-05-27T08:10:25.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53eca00700 1 -- 172.21.15.63:0/3087504294 learned_addr learned my addr 172.21.15.63:0/3087504294 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:25.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53ed400700 1 --2- 172.21.15.63:0/3087504294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7f9a410 0x5577e7fb07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53e7e00700 1 --2- 172.21.15.63:0/3087504294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7e1be40 0x5577e7fa6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53eca00700 1 -- 172.21.15.63:0/3087504294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7e1be40 msgr2=0x5577e7fa6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:25.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53eca00700 1 --2- 172.21.15.63:0/3087504294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7e1be40 0x5577e7fa6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53eca00700 1 -- 172.21.15.63:0/3087504294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7f9a410 msgr2=0x5577e7fb07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:25.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53eca00700 1 --2- 172.21.15.63:0/3087504294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7f9a410 0x5577e7fb07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53eca00700 1 -- 172.21.15.63:0/3087504294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e7f971f0 con 0x5577e7e7bfc0 2024-05-27T08:10:25.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.107+0000 7f53e7e00700 1 --2- 172.21.15.63:0/3087504294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7e1be40 0x5577e7fa6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:10:25.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.108+0000 7f53eca00700 1 --2- 172.21.15.63:0/3087504294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa49f0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f53dc005950 tx=0x7f53dc00e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f627cb4f8708c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:25.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.108+0000 7f53d9600700 1 -- 172.21.15.63:0/3087504294 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc01b590 con 0x5577e7e7bfc0 2024-05-27T08:10:25.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.108+0000 7f53d9600700 1 -- 172.21.15.63:0/3087504294 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53dc01b9f0 con 0x5577e7e7bfc0 2024-05-27T08:10:25.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.108+0000 7f53d9600700 1 -- 172.21.15.63:0/3087504294 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc004830 con 0x5577e7e7bfc0 2024-05-27T08:10:25.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.109+0000 7f53fdef4b80 1 -- 172.21.15.63:0/3087504294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 msgr2=0x5577e7fa49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:25.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.109+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/3087504294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa49f0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f53dc005950 tx=0x7f53dc00e710 comp rx=0 tx=0).stop 2024-05-27T08:10:25.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.109+0000 7f53fdef4b80 1 -- 172.21.15.63:0/3087504294 shutdown_connections 2024-05-27T08:10:25.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.109+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/3087504294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7f9a410 0x5577e7fb07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.109+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/3087504294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa49f0 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.109+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/3087504294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7e1be40 0x5577e7fa6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.109+0000 7f53fdef4b80 1 -- 172.21.15.63:0/3087504294 >> 172.21.15.63:0/3087504294 conn(0x5577e7f8ef60 msgr2=0x5577e7f9e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:25.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.110+0000 7f53fdef4b80 1 -- 172.21.15.63:0/3087504294 shutdown_connections 2024-05-27T08:10:25.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.110+0000 7f53fdef4b80 1 -- 172.21.15.63:0/3087504294 wait complete. 2024-05-27T08:10:25.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.110+0000 7f53fdef4b80 1 -- start start 2024-05-27T08:10:25.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.110+0000 7f53fdef4b80 1 Processor -- start 2024-05-27T08:10:25.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.111+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 0x5577e7fad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.111+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.111+0000 7f53fdef4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.111+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81b0a00 con 0x5577e7e7bfc0 2024-05-27T08:10:25.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.111+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81b1e70 con 0x5577e7e1be40 2024-05-27T08:10:25.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.111+0000 7f53fdef4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f53e81b10d0 con 0x5577e7f9a410 2024-05-27T08:10:25.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.111+0000 7f53fdef4b80 10 client.0 _ll_register_callbacks cb 0x5577e7e62c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:25.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53e7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40512/0 (socket says 172.21.15.63:40512) 2024-05-27T08:10:25.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53e7e00700 1 -- 172.21.15.63:0/1769520746 learned_addr learned my addr 172.21.15.63:0/1769520746 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:25.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53eca00700 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 0x5577e7fad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53ed400700 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53e7e00700 1 -- 172.21.15.63:0/1769520746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 msgr2=0x5577e7fa8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:25.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53e7e00700 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53e7e00700 1 -- 172.21.15.63:0/1769520746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 msgr2=0x5577e7fad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53e7e00700 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 0x5577e7fad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53e7e00700 1 -- 172.21.15.63:0/1769520746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e7fa9250 con 0x5577e7e7bfc0 2024-05-27T08:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.112+0000 7f53ed400700 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:25.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.113+0000 7f53e7e00700 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa84f0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f53e8076a30 tx=0x7f53d4012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2134282549dcbab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:25.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.113+0000 7f53cea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:25.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.113+0000 7f53cea00700 1 -- 172.21.15.63:0/1769520746 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4004a30 con 0x5577e7e7bfc0 2024-05-27T08:10:25.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.114+0000 7f53cea00700 1 -- 172.21.15.63:0/1769520746 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f53d4004bd0 con 0x5577e7e7bfc0 2024-05-27T08:10:25.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.114+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1769520746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53dc01b9f0 con 0x5577e7e7bfc0 2024-05-27T08:10:25.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.114+0000 7f53cea00700 1 -- 172.21.15.63:0/1769520746 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f53d402e050 con 0x5577e7e7bfc0 2024-05-27T08:10:25.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.114+0000 7f53fdef4b80 10 client.22461 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:25.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.114+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1769520746 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5577e801f1e0 con 0x5577e7e7bfc0 2024-05-27T08:10:25.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.114+0000 7f53cea00700 1 -- 172.21.15.63:0/1769520746 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53dc01b9f0 con 0x5577e7e7bfc0 2024-05-27T08:10:25.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.115+0000 7f53cea00700 1 -- 172.21.15.63:0/1769520746 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f53d401d7c0 con 0x5577e7e7bfc0 2024-05-27T08:10:25.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.115+0000 7f53fdef4b80 10 client.22461 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:25.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.115+0000 7f53fdef4b80 10 client.22461 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:10:25.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.115+0000 7f53fdef4b80 20 client.22461 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:25.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.115+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5577e7f9a820 0x5577e8021960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:25.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.115+0000 7f53fdef4b80 4 client.22461 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:10:25.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.115+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1769520746 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577e7ec4570 con 0x5577e7f9a820 2024-05-27T08:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.116+0000 7f53eca00700 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5577e7f9a820 0x5577e8021960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.117+0000 7f53eca00700 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5577e7f9a820 0x5577e8021960 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c79e53fe38c29eab server_cookie=3f6eb6b2fd7f2d1b in_seq=0 out_seq=0 2024-05-27T08:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.117+0000 7f53cea00700 10 client.22461 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:10:25.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.117+0000 7f53cea00700 1 -- 172.21.15.63:0/1769520746 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577e7ec4570 con 0x5577e7f9a820 2024-05-27T08:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.117+0000 7f53cea00700 10 client.22461 handle_command_reply: tid=0 2024-05-27T08:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.117+0000 7f53fdef4b80 1 client.22461 shutdown 2024-05-27T08:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.117+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1769520746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 msgr2=0x5577e7fa84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:25.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.117+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa84f0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f53e8076a30 tx=0x7f53d4012730 comp rx=0 tx=0).stop 2024-05-27T08:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.118+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1769520746 shutdown_connections 2024-05-27T08:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.118+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5577e7f9a410 0x5577e7fa8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.118+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577e7e1be40 0x5577e7fad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.118+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5577e7f9a820 0x5577e8021960 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.118+0000 7f53fdef4b80 1 --2- 172.21.15.63:0/1769520746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5577e7e7bfc0 0x5577e7fa84f0 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.118+0000 7f53cea00700 0 client.22461 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.118+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1769520746 >> 172.21.15.63:0/1769520746 conn(0x5577e7f8ef60 msgr2=0x5577e7fa2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.118+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1769520746 shutdown_connections 2024-05-27T08:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.118+0000 7f53fdef4b80 1 -- 172.21.15.63:0/1769520746 wait complete. 2024-05-27T08:10:25.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.118+0000 7f53fdef4b80 20 client.22461 trim_cache size 0 max 16384 2024-05-27T08:10:25.121 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:10:25.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.119+0000 7f53ede00700 1 -- 172.21.15.63:0/3346963905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 msgr2=0x7f53d00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:25.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.119+0000 7f53ede00700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 0x7f53d00b4f00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009c60 tx=0x7f53dc009260 comp rx=0 tx=0).stop 2024-05-27T08:10:25.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.119+0000 7f53ede00700 1 -- 172.21.15.63:0/3346963905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8110ad0 msgr2=0x7f53e81a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:25.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.119+0000 7f53ede00700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8110ad0 0x7f53e81a1540 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f53e000b070 tx=0x7f53e0004c80 comp rx=0 tx=0).stop 2024-05-27T08:10:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.119+0000 7f53ede00700 1 -- 172.21.15.63:0/3346963905 shutdown_connections 2024-05-27T08:10:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.119+0000 7f53ede00700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f53d00b29f0 0x7f53d00b4f00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.119+0000 7f53ede00700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e8110ad0 0x7f53e81a1540 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.119+0000 7f53ede00700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53e8102f20 0x7f53e819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.119+0000 7f53ede00700 1 --2- 172.21.15.63:0/3346963905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e8103870 0x7f53e819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.119+0000 7f53ede00700 1 -- 172.21.15.63:0/3346963905 >> 172.21.15.63:0/3346963905 conn(0x7f53e8078ab0 msgr2=0x7f53e8111e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.120+0000 7f53ede00700 1 -- 172.21.15.63:0/3346963905 shutdown_connections 2024-05-27T08:10:25.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:25.120+0000 7f53ede00700 1 -- 172.21.15.63:0/3346963905 wait complete. 2024-05-27T08:10:25.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:25 smithi063 ceph-mon[31613]: pgmap v1169: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:10:26.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:25 smithi086 ceph-mon[34687]: pgmap v1169: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:10:26.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:25 smithi149 ceph-mon[35408]: pgmap v1169: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 13 op/s 2024-05-27T08:10:26.556 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:10:27.047 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:10:27.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:27 smithi063 ceph-mon[31613]: pgmap v1170: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.2 MiB/s wr, 16 op/s 2024-05-27T08:10:28.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:27 smithi086 ceph-mon[34687]: pgmap v1170: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.2 MiB/s wr, 16 op/s 2024-05-27T08:10:28.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:27 smithi149 ceph-mon[35408]: pgmap v1170: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.2 MiB/s wr, 16 op/s 2024-05-27T08:10:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.720+0000 7f5fbe000640 1 -- 172.21.15.63:0/813606120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb810c170 msgr2=0x7f5fb8105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.720+0000 7f5fbe000640 1 --2- 172.21.15.63:0/813606120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb810c170 0x7f5fb8105db0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009900 tx=0x7f5fa402a3f0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.721+0000 7f5fbe000640 1 -- 172.21.15.63:0/813606120 shutdown_connections 2024-05-27T08:10:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.721+0000 7f5fbe000640 1 --2- 172.21.15.63:0/813606120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb8106420 0x7f5fb81127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.721+0000 7f5fbe000640 1 --2- 172.21.15.63:0/813606120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb810c170 0x7f5fb8105db0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.721+0000 7f5fbe000640 1 --2- 172.21.15.63:0/813606120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb810bcd0 0x7f5fb8105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.721+0000 7f5fbe000640 1 -- 172.21.15.63:0/813606120 >> 172.21.15.63:0/813606120 conn(0x7f5fb81013d0 msgr2=0x7f5fb8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.721+0000 7f5fbe000640 1 -- 172.21.15.63:0/813606120 shutdown_connections 2024-05-27T08:10:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.721+0000 7f5fbe000640 1 -- 172.21.15.63:0/813606120 wait complete. 2024-05-27T08:10:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.722+0000 7f5fbe000640 1 Processor -- start 2024-05-27T08:10:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.723+0000 7f5fbe000640 1 -- start start 2024-05-27T08:10:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.723+0000 7f5fbe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb8106420 0x7f5fb81a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.723+0000 7f5fbe000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb810bcd0 0x7f5fb81a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.723+0000 7f5fbe000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb810c170 0x7f5fb819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.723+0000 7f5fbe000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8115da0 con 0x7f5fb810c170 2024-05-27T08:10:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.724+0000 7f5fbe000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8115c20 con 0x7f5fb8106420 2024-05-27T08:10:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.724+0000 7f5fbe000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8115f20 con 0x7f5fb810bcd0 2024-05-27T08:10:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.724+0000 7f5fbcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb8106420 0x7f5fb81a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.724+0000 7f5fbd600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb810c170 0x7f5fb819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.724+0000 7f5fbcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb8106420 0x7f5fb81a2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:38472/0 (socket says 172.21.15.63:38472) 2024-05-27T08:10:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.724+0000 7f5fbcc00640 1 -- 172.21.15.63:0/1855375100 learned_addr learned my addr 172.21.15.63:0/1855375100 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.724+0000 7f5fb7e00640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb810bcd0 0x7f5fb81a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.725+0000 7f5fbd600640 1 -- 172.21.15.63:0/1855375100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb810bcd0 msgr2=0x7f5fb81a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.725+0000 7f5fbd600640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb810bcd0 0x7f5fb81a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.725+0000 7f5fbd600640 1 -- 172.21.15.63:0/1855375100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb8106420 msgr2=0x7f5fb81a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.725+0000 7f5fbd600640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb8106420 0x7f5fb81a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.725+0000 7f5fbd600640 1 -- 172.21.15.63:0/1855375100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb819cce0 con 0x7f5fb810c170 2024-05-27T08:10:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.725+0000 7f5fbd600640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb810c170 0x7f5fb819c6a0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000cab0 tx=0x7f5fb000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.725+0000 7f5fb5600640 1 -- 172.21.15.63:0/1855375100 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000ae20 con 0x7f5fb810c170 2024-05-27T08:10:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.725+0000 7f5fb5600640 1 -- 172.21.15.63:0/1855375100 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fb000f8e0 con 0x7f5fb810c170 2024-05-27T08:10:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.726+0000 7f5fbe000640 1 -- 172.21.15.63:0/1855375100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb819cfd0 con 0x7f5fb810c170 2024-05-27T08:10:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.726+0000 7f5fb5600640 1 -- 172.21.15.63:0/1855375100 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00189a0 con 0x7f5fb810c170 2024-05-27T08:10:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.726+0000 7f5fbe000640 1 -- 172.21.15.63:0/1855375100 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb819d4a0 con 0x7f5fb810c170 2024-05-27T08:10:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.730+0000 7f5fc659b740 1 Processor -- start 2024-05-27T08:10:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.731+0000 7f5fb5600640 1 -- 172.21.15.63:0/1855375100 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5fb000f450 con 0x7f5fb810c170 2024-05-27T08:10:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.731+0000 7f5fc659b740 1 -- start start 2024-05-27T08:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.732+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 0x559fbdeada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.732+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeadfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.732+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 0x559fbdeae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.732+0000 7f5fc659b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559fbde1eb90 con 0x559fbdae22f0 2024-05-27T08:10:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.732+0000 7f5fc659b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559fbde0e730 con 0x559fbdaea4d0 2024-05-27T08:10:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.732+0000 7f5fc659b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559fbde12770 con 0x559fbdb0b1b0 2024-05-27T08:10:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.732+0000 7f5fbcc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeadfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.732+0000 7f5fbcc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeadfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40542/0 (socket says 172.21.15.63:40542) 2024-05-27T08:10:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.732+0000 7f5fbcc00640 1 -- 172.21.15.63:0/2974608202 learned_addr learned my addr 172.21.15.63:0/2974608202 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.732+0000 7f5fb7e00640 1 --2- 172.21.15.63:0/2974608202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 0x559fbdeada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.733+0000 7f5fb7e00640 1 --2- 172.21.15.63:0/2974608202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 0x559fbdeae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.733+0000 7f5fbcc00640 1 -- 172.21.15.63:0/2974608202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 msgr2=0x559fbdeada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.733+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/2974608202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 0x559fbdeada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.733+0000 7f5fbcc00640 1 -- 172.21.15.63:0/2974608202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 msgr2=0x559fbdeae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.733+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/2974608202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 0x559fbdeae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.733+0000 7f5fbcc00640 1 -- 172.21.15.63:0/2974608202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fbdda78b0 con 0x559fbdae22f0 2024-05-27T08:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.733+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/2974608202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeadfa0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0045c0 tx=0x7f5fac00bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d7508292706754 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.734+0000 7f5fa9c00640 1 -- 172.21.15.63:0/2974608202 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00be60 con 0x559fbdae22f0 2024-05-27T08:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.734+0000 7f5fa9c00640 1 -- 172.21.15.63:0/2974608202 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fac015070 con 0x559fbdae22f0 2024-05-27T08:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.734+0000 7f5fa9c00640 1 -- 172.21.15.63:0/2974608202 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0116a0 con 0x559fbdae22f0 2024-05-27T08:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.734+0000 7f5fb5600640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5fa00b2950 0x7f5fa00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.734+0000 7f5fb5600640 1 -- 172.21.15.63:0/1855375100 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f5fb00e2d20 con 0x7f5fb810c170 2024-05-27T08:10:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.734+0000 7f5fb7e00640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5fa00b2950 0x7f5fa00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.735+0000 7f5fc659b740 1 -- 172.21.15.63:0/2974608202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 msgr2=0x559fbdeadfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.735+0000 7f5fc659b740 1 --2- 172.21.15.63:0/2974608202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeadfa0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0045c0 tx=0x7f5fac00bc70 comp rx=0 tx=0).stop 2024-05-27T08:10:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.735+0000 7f5fc659b740 1 -- 172.21.15.63:0/2974608202 shutdown_connections 2024-05-27T08:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.735+0000 7f5fc659b740 1 --2- 172.21.15.63:0/2974608202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 0x559fbdeae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.735+0000 7f5fc659b740 1 --2- 172.21.15.63:0/2974608202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeadfa0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.735+0000 7f5fc659b740 1 --2- 172.21.15.63:0/2974608202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 0x559fbdeada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.735+0000 7f5fc659b740 1 -- 172.21.15.63:0/2974608202 >> 172.21.15.63:0/2974608202 conn(0x559fbdaff300 msgr2=0x559fbdeaa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.735+0000 7f5fc659b740 1 -- 172.21.15.63:0/2974608202 shutdown_connections 2024-05-27T08:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.735+0000 7f5fb7e00640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5fa00b2950 0x7f5fa00b4e60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402a910 tx=0x7f5fa4005a40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.735+0000 7f5fc659b740 1 -- 172.21.15.63:0/2974608202 wait complete. 2024-05-27T08:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.736+0000 7f5fc659b740 1 -- start start 2024-05-27T08:10:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.736+0000 7f5fc659b740 1 Processor -- start 2024-05-27T08:10:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.737+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeb7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.737+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 0x559fbdeb80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.737+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 0x559fbdec3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.737+0000 7f5fc659b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559fbdaf4790 con 0x559fbdae22f0 2024-05-27T08:10:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.737+0000 7f5fc659b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559fbddb6bc0 con 0x559fbdaea4d0 2024-05-27T08:10:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.737+0000 7f5fc659b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559fbdebab90 con 0x559fbdb0b1b0 2024-05-27T08:10:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.737+0000 7f5fc659b740 10 client.0 _ll_register_callbacks cb 0x559fbdd94400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fbcc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeb7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fbcc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeb7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40558/0 (socket says 172.21.15.63:40558) 2024-05-27T08:10:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fbcc00640 1 -- 172.21.15.63:0/1164190934 learned_addr learned my addr 172.21.15.63:0/1164190934 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fb7e00640 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 0x559fbdeb80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fbd600640 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 0x559fbdec3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fbcc00640 1 -- 172.21.15.63:0/1164190934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 msgr2=0x559fbdec3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 0x559fbdec3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fbcc00640 1 -- 172.21.15.63:0/1164190934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 msgr2=0x559fbdeb80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 0x559fbdeb80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fbcc00640 1 -- 172.21.15.63:0/1164190934 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fbdb23210 con 0x559fbdae22f0 2024-05-27T08:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fb7e00640 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 0x559fbdeb80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.738+0000 7f5fbd600640 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 0x559fbdec3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.739+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeb7bb0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f5fac000c00 tx=0x7f5fac00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d4d01dc406e95f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.739+0000 7f5f9e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.739+0000 7f5f9e000640 1 -- 172.21.15.63:0/1164190934 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac02a060 con 0x559fbdae22f0 2024-05-27T08:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.739+0000 7f5f9e000640 1 -- 172.21.15.63:0/1164190934 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f5fac01a470 con 0x559fbdae22f0 2024-05-27T08:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.739+0000 7f5fc659b740 1 -- 172.21.15.63:0/1164190934 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fac015070 con 0x559fbdae22f0 2024-05-27T08:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.740+0000 7f5f9e000640 1 -- 172.21.15.63:0/1164190934 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5fac031050 con 0x559fbdae22f0 2024-05-27T08:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.740+0000 7f5fc659b740 10 client.22473 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.740+0000 7f5fc659b740 1 -- 172.21.15.63:0/1164190934 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559fbdbef620 con 0x559fbdae22f0 2024-05-27T08:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.740+0000 7f5f9e000640 1 -- 172.21.15.63:0/1164190934 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fac015070 con 0x559fbdae22f0 2024-05-27T08:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.741+0000 7f5f9e000640 1 -- 172.21.15.63:0/1164190934 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5fac0119e0 con 0x559fbdae22f0 2024-05-27T08:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.741+0000 7f5fc659b740 10 client.22473 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.741+0000 7f5fc659b740 10 client.22473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.741+0000 7f5fc659b740 20 client.22473 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.741+0000 7f5fc659b740 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fbdac9d80 0x559fbdec9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.741+0000 7f5fc659b740 4 client.22473 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.741+0000 7f5fc659b740 1 -- 172.21.15.63:0/1164190934 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fbde20ae0 con 0x559fbdac9d80 2024-05-27T08:10:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.742+0000 7f5fb7e00640 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fbdac9d80 0x559fbdec9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.743+0000 7f5f9e000640 10 client.22473 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.743+0000 7f5fb7e00640 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fbdac9d80 0x559fbdec9d20 crc :-1 s=READY pgs=656 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b0d884b1ebacc1 server_cookie=279e97e39b91fe33 in_seq=0 out_seq=0 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.745+0000 7f5f9e000640 1 -- 172.21.15.63:0/1164190934 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559fbde20ae0 con 0x559fbdac9d80 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.745+0000 7f5f9e000640 10 client.22473 handle_command_reply: tid=0 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.745+0000 7f5fc659b740 1 client.22473 shutdown 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.745+0000 7f5fc659b740 1 -- 172.21.15.63:0/1164190934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 msgr2=0x559fbdeb7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.745+0000 7f5fc659b740 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeb7bb0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f5fac000c00 tx=0x7f5fac00be60 comp rx=0 tx=0).stop 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.746+0000 7f5fc659b740 1 -- 172.21.15.63:0/1164190934 shutdown_connections 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.746+0000 7f5fc659b740 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fbdac9d80 0x559fbdec9d20 crc :-1 s=READY pgs=656 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.746+0000 7f5fc659b740 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdb0b1b0 0x559fbdec3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.746+0000 7f5fc659b740 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdaea4d0 0x559fbdeb80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.746+0000 7f5fc659b740 1 --2- 172.21.15.63:0/1164190934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdeb7bb0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.746+0000 7f5f9e000640 0 client.22473 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.746+0000 7f5fc659b740 1 -- 172.21.15.63:0/1164190934 >> 172.21.15.63:0/1164190934 conn(0x559fbdaff300 msgr2=0x559fbdead890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.746+0000 7f5fc659b740 1 -- 172.21.15.63:0/1164190934 shutdown_connections 2024-05-27T08:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.746+0000 7f5fc659b740 1 -- 172.21.15.63:0/1164190934 wait complete. 2024-05-27T08:10:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.746+0000 7f5fc659b740 20 client.22473 trim_cache size 0 max 16384 2024-05-27T08:10:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.771+0000 7f5fc659b740 1 Processor -- start 2024-05-27T08:10:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.772+0000 7f5fc659b740 1 -- start start 2024-05-27T08:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.773+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdec3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.773+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 0x559fbdeb7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.773+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdb0b1b0 0x559fbdeb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.773+0000 7f5fc659b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b5ab0 con 0x559fbdae22f0 2024-05-27T08:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.773+0000 7f5fc659b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b3380 con 0x559fbdb0b1b0 2024-05-27T08:10:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.773+0000 7f5fc659b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b11f0 con 0x559fbdaea4d0 2024-05-27T08:10:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.774+0000 7f5fbcc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdec3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.774+0000 7f5fb7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 0x559fbdeb7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.774+0000 7f5fbd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdb0b1b0 0x559fbdeb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.774+0000 7f5fb7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 0x559fbdeb7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:51946/0 (socket says 172.21.15.63:51946) 2024-05-27T08:10:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.774+0000 7f5fb7e00640 1 -- 172.21.15.63:0/1280598017 learned_addr learned my addr 172.21.15.63:0/1280598017 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.775+0000 7f5fbcc00640 1 -- 172.21.15.63:0/1280598017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 msgr2=0x559fbdeb7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.775+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/1280598017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 0x559fbdeb7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.775+0000 7f5fbcc00640 1 -- 172.21.15.63:0/1280598017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdb0b1b0 msgr2=0x559fbdeb8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.775+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/1280598017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdb0b1b0 0x559fbdeb8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.775+0000 7f5fbcc00640 1 -- 172.21.15.63:0/1280598017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fbdeae220 con 0x559fbdae22f0 2024-05-27T08:10:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.775+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/1280598017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdec3e60 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00bcb0 tx=0x7f5fac005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8b6694ffcdabde server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.775+0000 7f5fa9200640 1 -- 172.21.15.63:0/1280598017 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac02a070 con 0x559fbdae22f0 2024-05-27T08:10:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.776+0000 7f5fa9200640 1 -- 172.21.15.63:0/1280598017 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fac015040 con 0x559fbdae22f0 2024-05-27T08:10:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.776+0000 7f5fa9200640 1 -- 172.21.15.63:0/1280598017 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0116a0 con 0x559fbdae22f0 2024-05-27T08:10:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.777+0000 7f5fc659b740 1 -- 172.21.15.63:0/1280598017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 msgr2=0x559fbdec3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.777+0000 7f5fc659b740 1 --2- 172.21.15.63:0/1280598017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdec3e60 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00bcb0 tx=0x7f5fac005660 comp rx=0 tx=0).stop 2024-05-27T08:10:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.777+0000 7f5fc659b740 1 -- 172.21.15.63:0/1280598017 shutdown_connections 2024-05-27T08:10:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.777+0000 7f5fc659b740 1 --2- 172.21.15.63:0/1280598017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdb0b1b0 0x559fbdeb8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.777+0000 7f5fc659b740 1 --2- 172.21.15.63:0/1280598017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 0x559fbdeb7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.777+0000 7f5fc659b740 1 --2- 172.21.15.63:0/1280598017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdae22f0 0x559fbdec3e60 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.777+0000 7f5fc659b740 1 -- 172.21.15.63:0/1280598017 >> 172.21.15.63:0/1280598017 conn(0x559fbdaff300 msgr2=0x559fbdeb7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.777+0000 7f5fc659b740 1 -- 172.21.15.63:0/1280598017 shutdown_connections 2024-05-27T08:10:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.777+0000 7f5fc659b740 1 -- 172.21.15.63:0/1280598017 wait complete. 2024-05-27T08:10:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.778+0000 7f5fc659b740 1 -- start start 2024-05-27T08:10:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.778+0000 7f5fc659b740 1 Processor -- start 2024-05-27T08:10:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.779+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdae22f0 0x559fbdecbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.779+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 0x559fbdecc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.779+0000 7f5fc659b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdb0b1b0 0x559fbdf38980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.779+0000 7f5fc659b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b1840 con 0x559fbdb0b1b0 2024-05-27T08:10:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.779+0000 7f5fc659b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b2cb0 con 0x559fbdae22f0 2024-05-27T08:10:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.779+0000 7f5fc659b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b1f10 con 0x559fbdaea4d0 2024-05-27T08:10:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.779+0000 7f5fc659b740 10 client.0 _ll_register_callbacks cb 0x7f5fac004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.780+0000 7f5fbd600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdb0b1b0 0x559fbdf38980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.780+0000 7f5fbcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdae22f0 0x559fbdecbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.780+0000 7f5fbd600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdb0b1b0 0x559fbdf38980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40588/0 (socket says 172.21.15.63:40588) 2024-05-27T08:10:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.780+0000 7f5fbd600640 1 -- 172.21.15.63:0/3562956966 learned_addr learned my addr 172.21.15.63:0/3562956966 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.780+0000 7f5fb7e00640 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 0x559fbdecc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.780+0000 7f5fbd600640 1 -- 172.21.15.63:0/3562956966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 msgr2=0x559fbdecc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.780+0000 7f5fbd600640 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 0x559fbdecc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.780+0000 7f5fbd600640 1 -- 172.21.15.63:0/3562956966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdae22f0 msgr2=0x559fbdecbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.780+0000 7f5fbd600640 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdae22f0 0x559fbdecbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.780+0000 7f5fbd600640 1 -- 172.21.15.63:0/3562956966 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fbdeccbc0 con 0x559fbdb0b1b0 2024-05-27T08:10:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.781+0000 7f5fbd600640 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdb0b1b0 0x559fbdf38980 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00a5cc0 tx=0x7f5fb00b24a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b227f25df3b819 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.781+0000 7f5f9e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.781+0000 7f5f9e000640 1 -- 172.21.15.63:0/3562956966 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00c4320 con 0x559fbdb0b1b0 2024-05-27T08:10:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.781+0000 7f5f9e000640 1 -- 172.21.15.63:0/3562956966 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f5fb00a6230 con 0x559fbdb0b1b0 2024-05-27T08:10:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.781+0000 7f5fc659b740 1 -- 172.21.15.63:0/3562956966 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fac015040 con 0x559fbdb0b1b0 2024-05-27T08:10:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.781+0000 7f5f9e000640 1 -- 172.21.15.63:0/3562956966 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5fb00a9c40 con 0x559fbdb0b1b0 2024-05-27T08:10:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.782+0000 7f5fc659b740 10 client.22479 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.782+0000 7f5fc659b740 1 -- 172.21.15.63:0/3562956966 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559fbdf39eb0 con 0x559fbdb0b1b0 2024-05-27T08:10:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.782+0000 7f5f9e000640 1 -- 172.21.15.63:0/3562956966 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fac015040 con 0x559fbdb0b1b0 2024-05-27T08:10:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.782+0000 7f5f9e000640 1 -- 172.21.15.63:0/3562956966 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5fb00b2710 con 0x559fbdb0b1b0 2024-05-27T08:10:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.782+0000 7f5fc659b740 10 client.22479 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.782+0000 7f5fc659b740 10 client.22479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.782+0000 7f5fc659b740 20 client.22479 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.782+0000 7f5fc659b740 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fbda9f1c0 0x559fbdf3c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.782+0000 7f5fc659b740 4 client.22479 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:10:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.782+0000 7f5fc659b740 1 -- 172.21.15.63:0/3562956966 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fbde1a2c0 con 0x559fbda9f1c0 2024-05-27T08:10:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.783+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fbda9f1c0 0x559fbdf3c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.784+0000 7f5fbcc00640 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fbda9f1c0 0x559fbdf3c5e0 crc :-1 s=READY pgs=657 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e76272a5c32393d1 server_cookie=5c392d93eadc54a in_seq=0 out_seq=0 2024-05-27T08:10:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.784+0000 7f5f9e000640 10 client.22479 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.819+0000 7f5f9e000640 1 -- 172.21.15.63:0/3562956966 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559fbde1a2c0 con 0x559fbda9f1c0 2024-05-27T08:10:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.819+0000 7f5f9e000640 10 client.22479 handle_command_reply: tid=0 2024-05-27T08:10:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.819+0000 7f5fc659b740 1 client.22479 shutdown 2024-05-27T08:10:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.819+0000 7f5fc659b740 1 -- 172.21.15.63:0/3562956966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdb0b1b0 msgr2=0x559fbdf38980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.819+0000 7f5fc659b740 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdb0b1b0 0x559fbdf38980 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00a5cc0 tx=0x7f5fb00b24a0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.819+0000 7f5fc659b740 1 -- 172.21.15.63:0/3562956966 shutdown_connections 2024-05-27T08:10:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.819+0000 7f5fc659b740 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559fbda9f1c0 0x559fbdf3c5e0 crc :-1 s=READY pgs=657 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.820+0000 7f5fc659b740 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559fbdb0b1b0 0x559fbdf38980 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.820+0000 7f5fc659b740 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559fbdaea4d0 0x559fbdecc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.820+0000 7f5fc659b740 1 --2- 172.21.15.63:0/3562956966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fbdae22f0 0x559fbdecbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.820+0000 7f5fc659b740 1 -- 172.21.15.63:0/3562956966 >> 172.21.15.63:0/3562956966 conn(0x559fbdaff300 msgr2=0x559fbdeb7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.820+0000 7f5f9e000640 0 client.22479 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.820+0000 7f5fc659b740 1 -- 172.21.15.63:0/3562956966 shutdown_connections 2024-05-27T08:10:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.820+0000 7f5fc659b740 1 -- 172.21.15.63:0/3562956966 wait complete. 2024-05-27T08:10:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.820+0000 7f5fc659b740 20 client.22479 trim_cache size 0 max 16384 2024-05-27T08:10:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:10:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:10:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 -- 172.21.15.63:0/1855375100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5fa00b2950 msgr2=0x7f5fa00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5fa00b2950 0x7f5fa00b4e60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402a910 tx=0x7f5fa4005a40 comp rx=0 tx=0).stop 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 -- 172.21.15.63:0/1855375100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb810c170 msgr2=0x7f5fb819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb810c170 0x7f5fb819c6a0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000cab0 tx=0x7f5fb000cf70 comp rx=0 tx=0).stop 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 -- 172.21.15.63:0/1855375100 shutdown_connections 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5fa00b2950 0x7f5fa00b4e60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb810c170 0x7f5fb819c6a0 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb810bcd0 0x7f5fb81a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 --2- 172.21.15.63:0/1855375100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb8106420 0x7f5fb81a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 -- 172.21.15.63:0/1855375100 >> 172.21.15.63:0/1855375100 conn(0x7f5fb81013d0 msgr2=0x7f5fb81017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 -- 172.21.15.63:0/1855375100 shutdown_connections 2024-05-27T08:10:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:28.821+0000 7f5fbe000640 1 -- 172.21.15.63:0/1855375100 wait complete. 2024-05-27T08:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.566+0000 7fc714800700 1 -- 172.21.15.63:0/2268376673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc70c1054d0 msgr2=0x7fc70c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:29.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.566+0000 7fc714800700 1 --2- 172.21.15.63:0/2268376673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc70c1054d0 0x7fc70c1058e0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fc700009990 tx=0x7fc700009ca0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.566+0000 7fc714800700 1 -- 172.21.15.63:0/2268376673 shutdown_connections 2024-05-27T08:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.566+0000 7fc714800700 1 --2- 172.21.15.63:0/2268376673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c10e900 0x7fc70c110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.566+0000 7fc714800700 1 --2- 172.21.15.63:0/2268376673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc70c1054d0 0x7fc70c1058e0 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.566+0000 7fc714800700 1 --2- 172.21.15.63:0/2268376673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c105e20 0x7fc70c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.566+0000 7fc714800700 1 -- 172.21.15.63:0/2268376673 >> 172.21.15.63:0/2268376673 conn(0x7fc70c0fedb0 msgr2=0x7fc70c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.567+0000 7fc714800700 1 -- 172.21.15.63:0/2268376673 shutdown_connections 2024-05-27T08:10:29.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.567+0000 7fc714800700 1 -- 172.21.15.63:0/2268376673 wait complete. 2024-05-27T08:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.568+0000 7fc714800700 1 Processor -- start 2024-05-27T08:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.568+0000 7fc714800700 1 -- start start 2024-05-27T08:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.569+0000 7fc714800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc70c1054d0 0x7fc70c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.569+0000 7fc714800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c105e20 0x7fc70c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.569+0000 7fc714800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c10e900 0x7fc70c1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.569+0000 7fc714800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c114340 con 0x7fc70c1054d0 2024-05-27T08:10:29.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.569+0000 7fc714800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1141c0 con 0x7fc70c10e900 2024-05-27T08:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.569+0000 7fc714800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1144c0 con 0x7fc70c105e20 2024-05-27T08:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.569+0000 7fc711e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc70c1054d0 0x7fc70c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.569+0000 7fc711e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc70c1054d0 0x7fc70c19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33420/0 (socket says 172.21.15.63:33420) 2024-05-27T08:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.569+0000 7fc711e00700 1 -- 172.21.15.63:0/3425336424 learned_addr learned my addr 172.21.15.63:0/3425336424 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.570+0000 7fc711400700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c105e20 0x7fc70c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.570+0000 7fc712800700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c10e900 0x7fc70c1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.570+0000 7fc711e00700 1 -- 172.21.15.63:0/3425336424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c105e20 msgr2=0x7fc70c19d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.570+0000 7fc711e00700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c105e20 0x7fc70c19d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.570+0000 7fc711e00700 1 -- 172.21.15.63:0/3425336424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c10e900 msgr2=0x7fc70c1a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.570+0000 7fc711e00700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c10e900 0x7fc70c1a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.570+0000 7fc711e00700 1 -- 172.21.15.63:0/3425336424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c1a1de0 con 0x7fc70c1054d0 2024-05-27T08:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.570+0000 7fc711e00700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc70c1054d0 0x7fc70c19cf10 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7fc700005910 tx=0x7fc7000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.571+0000 7fc706a00700 1 -- 172.21.15.63:0/3425336424 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700027070 con 0x7fc70c1054d0 2024-05-27T08:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.571+0000 7fc706a00700 1 -- 172.21.15.63:0/3425336424 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc70000fc00 con 0x7fc70c1054d0 2024-05-27T08:10:29.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.571+0000 7fc714800700 1 -- 172.21.15.63:0/3425336424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c1a2070 con 0x7fc70c1054d0 2024-05-27T08:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.571+0000 7fc706a00700 1 -- 172.21.15.63:0/3425336424 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc700022d80 con 0x7fc70c1054d0 2024-05-27T08:10:29.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.571+0000 7fc714800700 1 -- 172.21.15.63:0/3425336424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1a2550 con 0x7fc70c1054d0 2024-05-27T08:10:29.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.576+0000 7fc706a00700 1 -- 172.21.15.63:0/3425336424 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc70002a070 con 0x7fc70c1054d0 2024-05-27T08:10:29.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.577+0000 7fc72331bb80 1 Processor -- start 2024-05-27T08:10:29.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc72331bb80 1 -- start start 2024-05-27T08:10:29.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a795330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 0x55bf1a795870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc706a00700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6f40b29f0 0x7fc6f40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a795e70 0x55bf1a7a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc72331bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a6c3e40 con 0x55bf1a7029f0 2024-05-27T08:10:29.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc72331bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a6b39e0 con 0x55bf1a616e40 2024-05-27T08:10:29.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc72331bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a6b7a20 con 0x55bf1a795e70 2024-05-27T08:10:29.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc706a00700 1 -- 172.21.15.63:0/3425336424 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7fc7000e3990 con 0x7fc70c1054d0 2024-05-27T08:10:29.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc712800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a795330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc712800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a795330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33432/0 (socket says 172.21.15.63:33432) 2024-05-27T08:10:29.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.578+0000 7fc712800700 1 -- 172.21.15.63:0/3119908752 learned_addr learned my addr 172.21.15.63:0/3119908752 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:29.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.579+0000 7fc711400700 1 --2- 172.21.15.63:0/3119908752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 0x55bf1a795870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.579+0000 7fc712800700 1 -- 172.21.15.63:0/3119908752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a795e70 msgr2=0x55bf1a7a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.579+0000 7fc712800700 1 --2- 172.21.15.63:0/3119908752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a795e70 0x55bf1a7a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.579+0000 7fc712800700 1 -- 172.21.15.63:0/3119908752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 msgr2=0x55bf1a795870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.579+0000 7fc712800700 1 --2- 172.21.15.63:0/3119908752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 0x55bf1a795870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.579+0000 7fc712800700 1 -- 172.21.15.63:0/3119908752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf1a67c420 con 0x55bf1a7029f0 2024-05-27T08:10:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.579+0000 7fc711e00700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6f40b29f0 0x7fc6f40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.579+0000 7fc711400700 1 --2- 172.21.15.63:0/3119908752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 0x55bf1a795870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:29.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.580+0000 7fc712800700 1 --2- 172.21.15.63:0/3119908752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a795330 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fc708009db0 tx=0x7fc7080077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd30acec98ad418 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:29.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.580+0000 7fc6fb400700 1 -- 172.21.15.63:0/3119908752 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708012040 con 0x55bf1a7029f0 2024-05-27T08:10:29.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.580+0000 7fc711e00700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6f40b29f0 0x7fc6f40b4f00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc7000a6960 tx=0x7fc700005840 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:29.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.580+0000 7fc6fb400700 1 -- 172.21.15.63:0/3119908752 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc708016860 con 0x55bf1a7029f0 2024-05-27T08:10:29.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.580+0000 7fc6fb400700 1 -- 172.21.15.63:0/3119908752 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708015780 con 0x55bf1a7029f0 2024-05-27T08:10:29.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.581+0000 7fc72331bb80 1 -- 172.21.15.63:0/3119908752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 msgr2=0x55bf1a795330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.581+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3119908752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a795330 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fc708009db0 tx=0x7fc7080077e0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.581+0000 7fc72331bb80 1 -- 172.21.15.63:0/3119908752 shutdown_connections 2024-05-27T08:10:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.581+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3119908752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a795e70 0x55bf1a7a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.581+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3119908752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a795330 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.581+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3119908752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 0x55bf1a795870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.581+0000 7fc72331bb80 1 -- 172.21.15.63:0/3119908752 >> 172.21.15.63:0/3119908752 conn(0x55bf1a6b0390 msgr2=0x55bf1a7914c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.582+0000 7fc72331bb80 1 -- 172.21.15.63:0/3119908752 shutdown_connections 2024-05-27T08:10:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.582+0000 7fc72331bb80 1 -- 172.21.15.63:0/3119908752 wait complete. 2024-05-27T08:10:29.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.582+0000 7fc72331bb80 1 -- start start 2024-05-27T08:10:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.583+0000 7fc72331bb80 1 Processor -- start 2024-05-27T08:10:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.583+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a616e40 0x55bf1a79f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.583+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.583+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.583+0000 7fc72331bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a5b9fa0 con 0x55bf1a795e70 2024-05-27T08:10:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.583+0000 7fc72331bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a654760 con 0x55bf1a7029f0 2024-05-27T08:10:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.583+0000 7fc72331bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a7a2b80 con 0x55bf1a616e40 2024-05-27T08:10:29.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.584+0000 7fc72331bb80 10 client.0 _ll_register_callbacks cb 0x55bf1a65dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.584+0000 7fc712800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a616e40 0x55bf1a79f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.584+0000 7fc712800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.584+0000 7fc711400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.584+0000 7fc711400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56232/0 (socket says 172.21.15.63:56232) 2024-05-27T08:10:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.584+0000 7fc711400700 1 -- 172.21.15.63:0/3188641199 learned_addr learned my addr 172.21.15.63:0/3188641199 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:29.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.584+0000 7fc712800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33446/0 (socket says 172.21.15.63:33446) 2024-05-27T08:10:29.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.585+0000 7fc712800700 1 -- 172.21.15.63:0/3188641199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a616e40 msgr2=0x55bf1a79f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:29.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.585+0000 7fc712800700 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a616e40 0x55bf1a79f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.585+0000 7fc712800700 1 -- 172.21.15.63:0/3188641199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 msgr2=0x55bf1a79fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:29.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.585+0000 7fc712800700 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.585+0000 7fc712800700 1 -- 172.21.15.63:0/3188641199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf1a6038d0 con 0x55bf1a795e70 2024-05-27T08:10:29.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.585+0000 7fc712800700 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7a0240 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7fc708009db0 tx=0x7fc708009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab876697deb8a3d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:29.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.585+0000 7fc6efe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:29.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.586+0000 7fc6efe00700 1 -- 172.21.15.63:0/3188641199 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70802b060 con 0x55bf1a795e70 2024-05-27T08:10:29.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.586+0000 7fc6efe00700 1 -- 172.21.15.63:0/3188641199 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7fc708017630 con 0x55bf1a795e70 2024-05-27T08:10:29.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.586+0000 7fc72331bb80 1 -- 172.21.15.63:0/3188641199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc708016860 con 0x55bf1a795e70 2024-05-27T08:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.586+0000 7fc6efe00700 1 -- 172.21.15.63:0/3188641199 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc70801c040 con 0x55bf1a795e70 2024-05-27T08:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.586+0000 7fc72331bb80 10 client.22491 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.586+0000 7fc72331bb80 1 -- 172.21.15.63:0/3188641199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bf1a684060 con 0x55bf1a795e70 2024-05-27T08:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.587+0000 7fc6efe00700 1 -- 172.21.15.63:0/3188641199 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc708016860 con 0x55bf1a795e70 2024-05-27T08:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.587+0000 7fc6efe00700 1 -- 172.21.15.63:0/3188641199 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc7080255a0 con 0x55bf1a795e70 2024-05-27T08:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.587+0000 7fc72331bb80 10 client.22491 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.587+0000 7fc72331bb80 10 client.22491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.587+0000 7fc72331bb80 20 client.22491 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.587+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bf1a7afa30 0x55bf1a7b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.587+0000 7fc72331bb80 4 client.22491 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.587+0000 7fc72331bb80 1 -- 172.21.15.63:0/3188641199 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf1a6c5d90 con 0x55bf1a7afa30 2024-05-27T08:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.588+0000 7fc711400700 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bf1a7afa30 0x55bf1a7b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.589+0000 7fc6efe00700 10 client.22491 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:29.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.589+0000 7fc711400700 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bf1a7afa30 0x55bf1a7b1e50 crc :-1 s=READY pgs=658 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69a18a06328ac83 server_cookie=57470fe297acdf18 in_seq=0 out_seq=0 2024-05-27T08:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc6efe00700 1 -- 172.21.15.63:0/3188641199 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bf1a6c5d90 con 0x55bf1a7afa30 2024-05-27T08:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc6efe00700 10 client.22491 handle_command_reply: tid=0 2024-05-27T08:10:29.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc72331bb80 1 client.22491 shutdown 2024-05-27T08:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc72331bb80 1 -- 172.21.15.63:0/3188641199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 msgr2=0x55bf1a7a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7a0240 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7fc708009db0 tx=0x7fc708009040 comp rx=0 tx=0).stop 2024-05-27T08:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc72331bb80 1 -- 172.21.15.63:0/3188641199 shutdown_connections 2024-05-27T08:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bf1a7afa30 0x55bf1a7b1e50 crc :-1 s=READY pgs=658 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7a0240 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a616e40 0x55bf1a79f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3188641199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc72331bb80 1 -- 172.21.15.63:0/3188641199 >> 172.21.15.63:0/3188641199 conn(0x55bf1a6b0390 msgr2=0x55bf1a795450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.591+0000 7fc6efe00700 0 client.22491 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.592+0000 7fc72331bb80 1 -- 172.21.15.63:0/3188641199 shutdown_connections 2024-05-27T08:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.592+0000 7fc72331bb80 1 -- 172.21.15.63:0/3188641199 wait complete. 2024-05-27T08:10:29.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.592+0000 7fc72331bb80 20 client.22491 trim_cache size 0 max 16384 2024-05-27T08:10:29.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.612+0000 7fc72331bb80 1 Processor -- start 2024-05-27T08:10:29.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.612+0000 7fc72331bb80 1 -- start start 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 0x55bf1a79af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a795e70 0x55bf1a79ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc72331bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1b4f10 con 0x55bf1a7029f0 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc72331bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1b27a0 con 0x55bf1a616e40 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc72331bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1b0610 con 0x55bf1a795e70 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc711400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc711400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33462/0 (socket says 172.21.15.63:33462) 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc711400700 1 -- 172.21.15.63:0/3928643767 learned_addr learned my addr 172.21.15.63:0/3928643767 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc712800700 1 --2- 172.21.15.63:0/3928643767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 0x55bf1a79af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.613+0000 7fc712800700 1 --2- 172.21.15.63:0/3928643767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a795e70 0x55bf1a79ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.614+0000 7fc711400700 1 -- 172.21.15.63:0/3928643767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a795e70 msgr2=0x55bf1a79ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:29.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.614+0000 7fc711400700 1 --2- 172.21.15.63:0/3928643767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a795e70 0x55bf1a79ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.614+0000 7fc711400700 1 -- 172.21.15.63:0/3928643767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 msgr2=0x55bf1a79af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:29.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.614+0000 7fc711400700 1 --2- 172.21.15.63:0/3928643767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 0x55bf1a79af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.614+0000 7fc711400700 1 -- 172.21.15.63:0/3928643767 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf1a78d240 con 0x55bf1a7029f0 2024-05-27T08:10:29.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.614+0000 7fc711400700 1 --2- 172.21.15.63:0/3928643767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fe90 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00bfd0 tx=0x7fc6fc00b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2992b817189da997 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:29.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.614+0000 7fc6faa00700 1 -- 172.21.15.63:0/3928643767 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc004030 con 0x55bf1a7029f0 2024-05-27T08:10:29.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.615+0000 7fc6faa00700 1 -- 172.21.15.63:0/3928643767 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6fc003680 con 0x55bf1a7029f0 2024-05-27T08:10:29.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.615+0000 7fc6faa00700 1 -- 172.21.15.63:0/3928643767 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0126b0 con 0x55bf1a7029f0 2024-05-27T08:10:29.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.616+0000 7fc72331bb80 1 -- 172.21.15.63:0/3928643767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 msgr2=0x55bf1a79fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:29.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.616+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3928643767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fe90 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00bfd0 tx=0x7fc6fc00b550 comp rx=0 tx=0).stop 2024-05-27T08:10:29.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.616+0000 7fc72331bb80 1 -- 172.21.15.63:0/3928643767 shutdown_connections 2024-05-27T08:10:29.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.616+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3928643767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a795e70 0x55bf1a79ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.616+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3928643767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a616e40 0x55bf1a79af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.616+0000 7fc72331bb80 1 --2- 172.21.15.63:0/3928643767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a7029f0 0x55bf1a79fe90 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.616+0000 7fc72331bb80 1 -- 172.21.15.63:0/3928643767 >> 172.21.15.63:0/3928643767 conn(0x55bf1a6b0390 msgr2=0x55bf1a79f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:29.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.616+0000 7fc72331bb80 1 -- 172.21.15.63:0/3928643767 shutdown_connections 2024-05-27T08:10:29.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.616+0000 7fc72331bb80 1 -- 172.21.15.63:0/3928643767 wait complete. 2024-05-27T08:10:29.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.616+0000 7fc72331bb80 1 -- start start 2024-05-27T08:10:29.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.617+0000 7fc72331bb80 1 Processor -- start 2024-05-27T08:10:29.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.617+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a616e40 0x55bf1a7b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.617+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 0x55bf1a7b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.617+0000 7fc72331bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.617+0000 7fc72331bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1b0c60 con 0x55bf1a795e70 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.617+0000 7fc72331bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1b20d0 con 0x55bf1a7029f0 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.617+0000 7fc72331bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1b1330 con 0x55bf1a616e40 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.618+0000 7fc72331bb80 10 client.0 _ll_register_callbacks cb 0x55bf1a65dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.618+0000 7fc712800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.618+0000 7fc712800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33464/0 (socket says 172.21.15.63:33464) 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.618+0000 7fc712800700 1 -- 172.21.15.63:0/614782016 learned_addr learned my addr 172.21.15.63:0/614782016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.618+0000 7fc712800700 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a616e40 0x55bf1a7b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.618+0000 7fc711400700 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 0x55bf1a7b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.619+0000 7fc712800700 1 -- 172.21.15.63:0/614782016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a616e40 msgr2=0x55bf1a7b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:29.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.619+0000 7fc712800700 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a616e40 0x55bf1a7b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.619+0000 7fc712800700 1 -- 172.21.15.63:0/614782016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 msgr2=0x55bf1a7b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:29.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.619+0000 7fc712800700 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 0x55bf1a7b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.619+0000 7fc712800700 1 -- 172.21.15.63:0/614782016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf1a7b2510 con 0x55bf1a795e70 2024-05-27T08:10:29.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.619+0000 7fc712800700 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7b1d40 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fc7080259f0 tx=0x7fc7080095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d11ff5ddd092b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:29.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.619+0000 7fc6efe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:29.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.619+0000 7fc6efe00700 1 -- 172.21.15.63:0/614782016 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70802b070 con 0x55bf1a795e70 2024-05-27T08:10:29.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.619+0000 7fc72331bb80 1 -- 172.21.15.63:0/614782016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6fc003680 con 0x55bf1a795e70 2024-05-27T08:10:29.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.620+0000 7fc6efe00700 1 -- 172.21.15.63:0/614782016 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7fc708005c80 con 0x55bf1a795e70 2024-05-27T08:10:29.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.620+0000 7fc6efe00700 1 -- 172.21.15.63:0/614782016 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc708025420 con 0x55bf1a795e70 2024-05-27T08:10:29.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.620+0000 7fc72331bb80 10 client.22497 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:29.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.620+0000 7fc72331bb80 1 -- 172.21.15.63:0/614782016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bf1a819270 con 0x55bf1a795e70 2024-05-27T08:10:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.620+0000 7fc6efe00700 1 -- 172.21.15.63:0/614782016 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6fc003680 con 0x55bf1a795e70 2024-05-27T08:10:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.620+0000 7fc6efe00700 1 -- 172.21.15.63:0/614782016 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc708012040 con 0x55bf1a795e70 2024-05-27T08:10:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.621+0000 7fc72331bb80 10 client.22497 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.621+0000 7fc72331bb80 10 client.22497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.621+0000 7fc72331bb80 20 client.22497 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.621+0000 7fc72331bb80 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bf1a8198f0 0x55bf1a81bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.621+0000 7fc72331bb80 4 client.22497 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:10:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.621+0000 7fc72331bb80 1 -- 172.21.15.63:0/614782016 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf1a6bf570 con 0x55bf1a8198f0 2024-05-27T08:10:29.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.621+0000 7fc711400700 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bf1a8198f0 0x55bf1a81bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:29.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.622+0000 7fc711400700 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bf1a8198f0 0x55bf1a81bd10 crc :-1 s=READY pgs=659 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50e94edac70523e server_cookie=8834080bff2aceb9 in_seq=0 out_seq=0 2024-05-27T08:10:29.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.622+0000 7fc6efe00700 10 client.22497 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:29.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc6efe00700 1 -- 172.21.15.63:0/614782016 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf1a6bf570 con 0x55bf1a8198f0 2024-05-27T08:10:29.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc6efe00700 10 client.22497 handle_command_reply: tid=0 2024-05-27T08:10:29.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 client.22497 shutdown 2024-05-27T08:10:29.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 -- 172.21.15.63:0/614782016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 msgr2=0x55bf1a7b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:29.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7b1d40 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fc7080259f0 tx=0x7fc7080095e0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 -- 172.21.15.63:0/614782016 shutdown_connections 2024-05-27T08:10:29.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55bf1a8198f0 0x55bf1a81bd10 crc :-1 s=READY pgs=659 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bf1a795e70 0x55bf1a7b1d40 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bf1a616e40 0x55bf1a7b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 --2- 172.21.15.63:0/614782016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bf1a7029f0 0x55bf1a7b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 -- 172.21.15.63:0/614782016 >> 172.21.15.63:0/614782016 conn(0x55bf1a6b0390 msgr2=0x55bf1a79f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:29.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc6efe00700 0 client.22497 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:29.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 -- 172.21.15.63:0/614782016 shutdown_connections 2024-05-27T08:10:29.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.623+0000 7fc72331bb80 1 -- 172.21.15.63:0/614782016 wait complete. 2024-05-27T08:10:29.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.624+0000 7fc72331bb80 20 client.22497 trim_cache size 0 max 16384 2024-05-27T08:10:29.626 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:10:29.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.624+0000 7fc714800700 1 -- 172.21.15.63:0/3425336424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6f40b29f0 msgr2=0x7fc6f40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:29.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.624+0000 7fc714800700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6f40b29f0 0x7fc6f40b4f00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc7000a6960 tx=0x7fc700005840 comp rx=0 tx=0).stop 2024-05-27T08:10:29.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.624+0000 7fc714800700 1 -- 172.21.15.63:0/3425336424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc70c1054d0 msgr2=0x7fc70c19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:29.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.624+0000 7fc714800700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc70c1054d0 0x7fc70c19cf10 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7fc700005910 tx=0x7fc7000047b0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.625+0000 7fc714800700 1 -- 172.21.15.63:0/3425336424 shutdown_connections 2024-05-27T08:10:29.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.625+0000 7fc714800700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc6f40b29f0 0x7fc6f40b4f00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.625+0000 7fc714800700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc70c10e900 0x7fc70c1a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.625+0000 7fc714800700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc70c1054d0 0x7fc70c19cf10 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.625+0000 7fc714800700 1 --2- 172.21.15.63:0/3425336424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc70c105e20 0x7fc70c19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.625+0000 7fc714800700 1 -- 172.21.15.63:0/3425336424 >> 172.21.15.63:0/3425336424 conn(0x7fc70c0fedb0 msgr2=0x7fc70c1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.625+0000 7fc714800700 1 -- 172.21.15.63:0/3425336424 shutdown_connections 2024-05-27T08:10:29.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:29.625+0000 7fc714800700 1 -- 172.21.15.63:0/3425336424 wait complete. 2024-05-27T08:10:29.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:29 smithi063 ceph-mon[31613]: pgmap v1171: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 6 op/s 2024-05-27T08:10:30.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:29 smithi086 ceph-mon[34687]: pgmap v1171: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 6 op/s 2024-05-27T08:10:30.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:29 smithi149 ceph-mon[35408]: pgmap v1171: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 6 op/s 2024-05-27T08:10:31.581 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:10:31.581 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:10:31.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:31 smithi063 ceph-mon[31613]: pgmap v1172: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 6 op/s 2024-05-27T08:10:32.016 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:31 smithi149 ceph-mon[35408]: pgmap v1172: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 6 op/s 2024-05-27T08:10:32.099 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:10:32.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:31 smithi086 ceph-mon[34687]: pgmap v1172: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.3 MiB/s wr, 6 op/s 2024-05-27T08:10:32.822 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:10:33.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:10:33.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:10:34.113 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:33 smithi063 ceph-mon[31613]: pgmap v1173: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T08:10:34.113 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:33 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:10:34.113 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:33 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:10:34.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:33 smithi086 ceph-mon[34687]: pgmap v1173: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T08:10:34.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:33 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:10:34.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:33 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:10:34.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:33 smithi149 ceph-mon[35408]: pgmap v1173: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T08:10:34.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:33 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:10:34.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:33 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:10:35.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.745+0000 7f8ea5e00700 1 -- 172.21.15.63:0/837128692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ea010e8d0 msgr2=0x7f8ea0110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:35.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.745+0000 7f8ea5e00700 1 --2- 172.21.15.63:0/837128692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ea010e8d0 0x7f8ea0110db0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800b0a0 tx=0x7f8e9800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.745+0000 7f8ea5e00700 1 -- 172.21.15.63:0/837128692 shutdown_connections 2024-05-27T08:10:35.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.745+0000 7f8ea5e00700 1 --2- 172.21.15.63:0/837128692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ea010e8d0 0x7f8ea0110db0 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.745+0000 7f8ea5e00700 1 --2- 172.21.15.63:0/837128692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea01054f0 0x7f8ea0105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.745+0000 7f8ea5e00700 1 --2- 172.21.15.63:0/837128692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea0105e40 0x7f8ea010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.745+0000 7f8ea5e00700 1 -- 172.21.15.63:0/837128692 >> 172.21.15.63:0/837128692 conn(0x7f8ea00fedb0 msgr2=0x7f8ea01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:35.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.746+0000 7f8ea5e00700 1 -- 172.21.15.63:0/837128692 shutdown_connections 2024-05-27T08:10:35.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.746+0000 7f8ea5e00700 1 -- 172.21.15.63:0/837128692 wait complete. 2024-05-27T08:10:35.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.747+0000 7f8ea5e00700 1 Processor -- start 2024-05-27T08:10:35.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.747+0000 7f8ea5e00700 1 -- start start 2024-05-27T08:10:35.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.748+0000 7f8ea5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ea01054f0 0x7f8ea019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.748+0000 7f8ea5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea0105e40 0x7f8ea019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.748+0000 7f8ea5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea010e8d0 0x7f8ea01a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.748+0000 7f8ea5e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01142e0 con 0x7f8ea01054f0 2024-05-27T08:10:35.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.748+0000 7f8ea5e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0114160 con 0x7f8ea0105e40 2024-05-27T08:10:35.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.748+0000 7f8ea5e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0114460 con 0x7f8ea010e8d0 2024-05-27T08:10:35.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.748+0000 7f8e9ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ea01054f0 0x7f8ea019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.748+0000 7f8e9ea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ea01054f0 0x7f8ea019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33474/0 (socket says 172.21.15.63:33474) 2024-05-27T08:10:35.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.748+0000 7f8e9ea00700 1 -- 172.21.15.63:0/3132947200 learned_addr learned my addr 172.21.15.63:0/3132947200 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:35.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.749+0000 7f8e9f400700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea010e8d0 0x7f8ea01a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.749+0000 7f8e9ea00700 1 -- 172.21.15.63:0/3132947200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea010e8d0 msgr2=0x7f8ea01a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:35.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.749+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea010e8d0 0x7f8ea01a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.749+0000 7f8e9ea00700 1 -- 172.21.15.63:0/3132947200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea0105e40 msgr2=0x7f8ea019d430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:10:35.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.749+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea0105e40 0x7f8ea019d430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.749+0000 7f8e9ea00700 1 -- 172.21.15.63:0/3132947200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea01a1d60 con 0x7f8ea01054f0 2024-05-27T08:10:35.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.749+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ea01054f0 0x7f8ea019ce90 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000e9a0 tx=0x7f8e9000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:35.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.752+0000 7f8ea4a00700 1 -- 172.21.15.63:0/3132947200 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9000cbd0 con 0x7f8ea01054f0 2024-05-27T08:10:35.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.752+0000 7f8ea4a00700 1 -- 172.21.15.63:0/3132947200 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e9000cd70 con 0x7f8ea01054f0 2024-05-27T08:10:35.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.752+0000 7f8ea4a00700 1 -- 172.21.15.63:0/3132947200 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90018680 con 0x7f8ea01054f0 2024-05-27T08:10:35.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.753+0000 7f8ea5e00700 1 -- 172.21.15.63:0/3132947200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea01a2050 con 0x7f8ea01054f0 2024-05-27T08:10:35.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.753+0000 7f8ea5e00700 1 -- 172.21.15.63:0/3132947200 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea01a9970 con 0x7f8ea01054f0 2024-05-27T08:10:35.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.759+0000 7f8ea4a00700 1 -- 172.21.15.63:0/3132947200 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8e90010950 con 0x7f8ea01054f0 2024-05-27T08:10:35.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.761+0000 7f8ea4a00700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e840b2a70 0x7f8e840b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.762+0000 7f8ea4a00700 1 -- 172.21.15.63:0/3132947200 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f8e900e2e10 con 0x7f8ea01054f0 2024-05-27T08:10:35.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.762+0000 7f8e9e000700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e840b2a70 0x7f8e840b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.763+0000 7f8e9e000700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e840b2a70 0x7f8e840b4f80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0110840 tx=0x7f8e8c00b370 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:35.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.773+0000 7f8eb49a9b80 1 Processor -- start 2024-05-27T08:10:35.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.773+0000 7f8eb49a9b80 1 -- start start 2024-05-27T08:10:35.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.773+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc55ea70 0x559cdc5f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.773+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc472ec0 0x559cdc5f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.773+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.773+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559cdc51fec0 con 0x559cdc472ec0 2024-05-27T08:10:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.773+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559cdc50fa60 con 0x559cdc5f1ef0 2024-05-27T08:10:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.773+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559cdc513aa0 con 0x559cdc55ea70 2024-05-27T08:10:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.774+0000 7f8e9e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc472ec0 0x559cdc5f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.774+0000 7f8e9e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc472ec0 0x559cdc5f18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33486/0 (socket says 172.21.15.63:33486) 2024-05-27T08:10:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.774+0000 7f8e9e000700 1 -- 172.21.15.63:0/1521502317 learned_addr learned my addr 172.21.15.63:0/1521502317 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.774+0000 7f8e9f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.774+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/1521502317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc55ea70 0x559cdc5f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.774+0000 7f8e9f400700 1 -- 172.21.15.63:0/1521502317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc55ea70 msgr2=0x559cdc5f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:35.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.774+0000 7f8e9f400700 1 --2- 172.21.15.63:0/1521502317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc55ea70 0x559cdc5f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.774+0000 7f8e9f400700 1 -- 172.21.15.63:0/1521502317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc472ec0 msgr2=0x559cdc5f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:35.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.774+0000 7f8e9f400700 1 --2- 172.21.15.63:0/1521502317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc472ec0 0x559cdc5f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.775+0000 7f8e9f400700 1 -- 172.21.15.63:0/1521502317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cdc4d84a0 con 0x559cdc5f1ef0 2024-05-27T08:10:35.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.775+0000 7f8e9f400700 1 --2- 172.21.15.63:0/1521502317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fe190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800b780 tx=0x7f8e98009db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d0c1f5439b139b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:35.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.776+0000 7f8e94c00700 1 -- 172.21.15.63:0/1521502317 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800bab0 con 0x559cdc5f1ef0 2024-05-27T08:10:35.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.776+0000 7f8e94c00700 1 -- 172.21.15.63:0/1521502317 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e9800bc50 con 0x559cdc5f1ef0 2024-05-27T08:10:35.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.776+0000 7f8e94c00700 1 -- 172.21.15.63:0/1521502317 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98007c90 con 0x559cdc5f1ef0 2024-05-27T08:10:35.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.777+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/1521502317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc5f1ef0 msgr2=0x559cdc5fe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:35.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.777+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/1521502317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fe190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800b780 tx=0x7f8e98009db0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.777+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/1521502317 shutdown_connections 2024-05-27T08:10:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.777+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/1521502317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fe190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.777+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/1521502317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc55ea70 0x559cdc5f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.777+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/1521502317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc472ec0 0x559cdc5f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.777+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/1521502317 >> 172.21.15.63:0/1521502317 conn(0x559cdc50c410 msgr2=0x559cdc5ed540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.777+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/1521502317 shutdown_connections 2024-05-27T08:10:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.777+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/1521502317 wait complete. 2024-05-27T08:10:35.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.778+0000 7f8eb49a9b80 1 -- start start 2024-05-27T08:10:35.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.778+0000 7f8eb49a9b80 1 Processor -- start 2024-05-27T08:10:35.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.779+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc472ec0 0x559cdc5fb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.779+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc55ea70 0x559cdc5fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.779+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.779+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559cdc416020 con 0x559cdc5f1ef0 2024-05-27T08:10:35.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.779+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559cdc4b07e0 con 0x559cdc55ea70 2024-05-27T08:10:35.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.779+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559cdc5feba0 con 0x559cdc472ec0 2024-05-27T08:10:35.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.779+0000 7f8eb49a9b80 10 client.0 _ll_register_callbacks cb 0x559cdc4b9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.780+0000 7f8e9f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.780+0000 7f8e9f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33494/0 (socket says 172.21.15.63:33494) 2024-05-27T08:10:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.780+0000 7f8e9f400700 1 -- 172.21.15.63:0/312708064 learned_addr learned my addr 172.21.15.63:0/312708064 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.780+0000 7f8e9e000700 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc55ea70 0x559cdc5fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.780+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc472ec0 0x559cdc5fb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.780+0000 7f8e9f400700 1 -- 172.21.15.63:0/312708064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc472ec0 msgr2=0x559cdc5fb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.780+0000 7f8e9f400700 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc472ec0 0x559cdc5fb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.780+0000 7f8e9f400700 1 -- 172.21.15.63:0/312708064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc55ea70 msgr2=0x559cdc5fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.780+0000 7f8e9f400700 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc55ea70 0x559cdc5fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.780+0000 7f8e9f400700 1 -- 172.21.15.63:0/312708064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cdc45f950 con 0x559cdc5f1ef0 2024-05-27T08:10:35.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.781+0000 7f8e9f400700 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fc170 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009a20 tx=0x7f8e98007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0411baba3fd73a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:35.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.781+0000 7f8e89600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:35.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.781+0000 7f8e89600700 1 -- 172.21.15.63:0/312708064 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9802a030 con 0x559cdc5f1ef0 2024-05-27T08:10:35.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.781+0000 7f8e89600700 1 -- 172.21.15.63:0/312708064 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f8e9800f070 con 0x559cdc5f1ef0 2024-05-27T08:10:35.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.781+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/312708064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e9800bc50 con 0x559cdc5f1ef0 2024-05-27T08:10:35.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.782+0000 7f8e89600700 1 -- 172.21.15.63:0/312708064 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e98037050 con 0x559cdc5f1ef0 2024-05-27T08:10:35.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.782+0000 7f8eb49a9b80 10 client.22509 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:35.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.782+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/312708064 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559cdc4e00e0 con 0x559cdc5f1ef0 2024-05-27T08:10:35.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.782+0000 7f8e89600700 1 -- 172.21.15.63:0/312708064 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e9800bc50 con 0x559cdc5f1ef0 2024-05-27T08:10:35.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.784+0000 7f8e89600700 1 -- 172.21.15.63:0/312708064 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8e98011070 con 0x559cdc5f1ef0 2024-05-27T08:10:35.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.784+0000 7f8eb49a9b80 10 client.22509 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:35.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.784+0000 7f8eb49a9b80 10 client.22509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:35.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.784+0000 7f8eb49a9b80 20 client.22509 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:35.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.784+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559cdc6141b0 0x559cdc6165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.784+0000 7f8eb49a9b80 4 client.22509 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:35.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.784+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/312708064 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cdc521e10 con 0x559cdc6141b0 2024-05-27T08:10:35.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.784+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559cdc6141b0 0x559cdc6165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.785+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559cdc6141b0 0x559cdc6165d0 crc :-1 s=READY pgs=660 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17962379a58e091d server_cookie=2384377f6c9eaf70 in_seq=0 out_seq=0 2024-05-27T08:10:35.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.785+0000 7f8e89600700 10 client.22509 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.787+0000 7f8e89600700 1 -- 172.21.15.63:0/312708064 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559cdc521e10 con 0x559cdc6141b0 2024-05-27T08:10:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.787+0000 7f8e89600700 10 client.22509 handle_command_reply: tid=0 2024-05-27T08:10:35.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.787+0000 7f8eb49a9b80 1 client.22509 shutdown 2024-05-27T08:10:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.787+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/312708064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 msgr2=0x559cdc5fc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:35.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.787+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fc170 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009a20 tx=0x7f8e98007c20 comp rx=0 tx=0).stop 2024-05-27T08:10:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.788+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/312708064 shutdown_connections 2024-05-27T08:10:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.788+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559cdc6141b0 0x559cdc6165d0 crc :-1 s=READY pgs=660 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.788+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fc170 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.788+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc472ec0 0x559cdc5fb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.788+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/312708064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc55ea70 0x559cdc5fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.788+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/312708064 >> 172.21.15.63:0/312708064 conn(0x559cdc50c410 msgr2=0x559cdc5f1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.788+0000 7f8e89600700 0 client.22509 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.788+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/312708064 shutdown_connections 2024-05-27T08:10:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.788+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/312708064 wait complete. 2024-05-27T08:10:35.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.788+0000 7f8eb49a9b80 20 client.22509 trim_cache size 0 max 16384 2024-05-27T08:10:35.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.815+0000 7f8eb49a9b80 1 Processor -- start 2024-05-27T08:10:35.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.815+0000 7f8eb49a9b80 1 -- start start 2024-05-27T08:10:35.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.816+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc472ec0 0x559cdc6106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.816+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc55ea70 0x559cdc5f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.816+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.816+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01b0d40 con 0x559cdc55ea70 2024-05-27T08:10:35.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.816+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01ae5d0 con 0x559cdc472ec0 2024-05-27T08:10:35.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.816+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01ac4a0 con 0x559cdc5f1ef0 2024-05-27T08:10:35.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.816+0000 7f8e9e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc55ea70 0x559cdc5f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.816+0000 7f8e9e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc55ea70 0x559cdc5f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33496/0 (socket says 172.21.15.63:33496) 2024-05-27T08:10:35.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.816+0000 7f8e9e000700 1 -- 172.21.15.63:0/1971547055 learned_addr learned my addr 172.21.15.63:0/1971547055 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:35.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.817+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/1971547055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc472ec0 0x559cdc6106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.817+0000 7f8e9e000700 1 -- 172.21.15.63:0/1971547055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc5f1ef0 msgr2=0x559cdc5fd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:35.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.817+0000 7f8e9e000700 1 --2- 172.21.15.63:0/1971547055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.817+0000 7f8e9e000700 1 -- 172.21.15.63:0/1971547055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc472ec0 msgr2=0x559cdc6106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:35.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.817+0000 7f8e9e000700 1 --2- 172.21.15.63:0/1971547055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc472ec0 0x559cdc6106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.817+0000 7f8e9e000700 1 -- 172.21.15.63:0/1971547055 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cdc5e92c0 con 0x559cdc55ea70 2024-05-27T08:10:35.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.817+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/1971547055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc472ec0 0x559cdc6106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:35.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.817+0000 7f8e9e000700 1 --2- 172.21.15.63:0/1971547055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc55ea70 0x559cdc5f6ea0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00bfd0 tx=0x7f8e8c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61fa4455637d498 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:35.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.818+0000 7f8e8be00700 1 -- 172.21.15.63:0/1971547055 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c010c50 con 0x559cdc55ea70 2024-05-27T08:10:35.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.818+0000 7f8e8be00700 1 -- 172.21.15.63:0/1971547055 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e8c016050 con 0x559cdc55ea70 2024-05-27T08:10:35.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.818+0000 7f8e8be00700 1 -- 172.21.15.63:0/1971547055 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c01e650 con 0x559cdc55ea70 2024-05-27T08:10:35.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.819+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/1971547055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc55ea70 msgr2=0x559cdc5f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:35.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.819+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/1971547055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc55ea70 0x559cdc5f6ea0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00bfd0 tx=0x7f8e8c010000 comp rx=0 tx=0).stop 2024-05-27T08:10:35.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.819+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/1971547055 shutdown_connections 2024-05-27T08:10:35.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.819+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/1971547055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc5f1ef0 0x559cdc5fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.819+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/1971547055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc472ec0 0x559cdc6106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.819+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/1971547055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc55ea70 0x559cdc5f6ea0 secure :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00bfd0 tx=0x7f8e8c010000 comp rx=0 tx=0).stop 2024-05-27T08:10:35.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.819+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/1971547055 >> 172.21.15.63:0/1971547055 conn(0x559cdc50c410 msgr2=0x559cdc5f55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:35.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.819+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/1971547055 shutdown_connections 2024-05-27T08:10:35.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.820+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/1971547055 wait complete. 2024-05-27T08:10:35.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.820+0000 7f8eb49a9b80 1 -- start start 2024-05-27T08:10:35.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.820+0000 7f8eb49a9b80 1 Processor -- start 2024-05-27T08:10:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.821+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc472ec0 0x559cdc6170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.821+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc6175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.821+0000 7f8eb49a9b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc617b30 0x559cdc67da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.821+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01acaf0 con 0x559cdc5f1ef0 2024-05-27T08:10:35.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.821+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01adf00 con 0x559cdc617b30 2024-05-27T08:10:35.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.821+0000 7f8eb49a9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01ad160 con 0x559cdc472ec0 2024-05-27T08:10:35.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.821+0000 7f8eb49a9b80 10 client.0 _ll_register_callbacks cb 0x559cdc4b9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:35.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.822+0000 7f8e9e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc6175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.822+0000 7f8e9e000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc6175f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33502/0 (socket says 172.21.15.63:33502) 2024-05-27T08:10:35.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.822+0000 7f8e9e000700 1 -- 172.21.15.63:0/551069777 learned_addr learned my addr 172.21.15.63:0/551069777 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:35.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.822+0000 7f8e9f400700 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc617b30 0x559cdc67da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.822+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc472ec0 0x559cdc6170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.822+0000 7f8e9e000700 1 -- 172.21.15.63:0/551069777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc472ec0 msgr2=0x559cdc6170b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.822+0000 7f8e9e000700 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc472ec0 0x559cdc6170b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.823+0000 7f8e9e000700 1 -- 172.21.15.63:0/551069777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc617b30 msgr2=0x559cdc67da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.823+0000 7f8e9e000700 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc617b30 0x559cdc67da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.823+0000 7f8e9e000700 1 -- 172.21.15.63:0/551069777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cdc67df60 con 0x559cdc5f1ef0 2024-05-27T08:10:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.823+0000 7f8e9e000700 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc6175f0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c012040 tx=0x7f8e8c029fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28e3b9b65a19863 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:35.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.823+0000 7f8e89600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:35.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.824+0000 7f8e89600700 1 -- 172.21.15.63:0/551069777 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c005250 con 0x559cdc5f1ef0 2024-05-27T08:10:35.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.824+0000 7f8e89600700 1 -- 172.21.15.63:0/551069777 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f8e8c0053f0 con 0x559cdc5f1ef0 2024-05-27T08:10:35.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.824+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/551069777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e8c016050 con 0x559cdc5f1ef0 2024-05-27T08:10:35.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.824+0000 7f8e89600700 1 -- 172.21.15.63:0/551069777 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e8c040050 con 0x559cdc5f1ef0 2024-05-27T08:10:35.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.824+0000 7f8eb49a9b80 10 client.22515 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:35.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.824+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/551069777 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559cdc67f1b0 con 0x559cdc5f1ef0 2024-05-27T08:10:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.825+0000 7f8e89600700 1 -- 172.21.15.63:0/551069777 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e8c016050 con 0x559cdc5f1ef0 2024-05-27T08:10:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.825+0000 7f8e89600700 1 -- 172.21.15.63:0/551069777 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8e8c03b030 con 0x559cdc5f1ef0 2024-05-27T08:10:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.825+0000 7f8eb49a9b80 10 client.22515 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.825+0000 7f8eb49a9b80 10 client.22515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.825+0000 7f8eb49a9b80 20 client.22515 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.825+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559cdc67f7e0 0x559cdc681c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.825+0000 7f8eb49a9b80 4 client.22515 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:10:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.825+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/551069777 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559cdc51b5f0 con 0x559cdc67f7e0 2024-05-27T08:10:35.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.826+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559cdc67f7e0 0x559cdc681c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:35.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.827+0000 7f8e9ea00700 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559cdc67f7e0 0x559cdc681c00 crc :-1 s=READY pgs=661 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78b23cae18c5534 server_cookie=77b32c19c95dd08b in_seq=0 out_seq=0 2024-05-27T08:10:35.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.827+0000 7f8e89600700 10 client.22515 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:35.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.830+0000 7f8e89600700 1 -- 172.21.15.63:0/551069777 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559cdc51b5f0 con 0x559cdc67f7e0 2024-05-27T08:10:35.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.830+0000 7f8e89600700 10 client.22515 handle_command_reply: tid=0 2024-05-27T08:10:35.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.830+0000 7f8eb49a9b80 1 client.22515 shutdown 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/551069777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 msgr2=0x559cdc6175f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc6175f0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c012040 tx=0x7f8e8c029fc0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/551069777 shutdown_connections 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559cdc67f7e0 0x559cdc681c00 crc :-1 s=READY pgs=661 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559cdc617b30 0x559cdc67da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559cdc472ec0 0x559cdc6170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8eb49a9b80 1 --2- 172.21.15.63:0/551069777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559cdc5f1ef0 0x559cdc6175f0 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8e89600700 0 client.22515 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/551069777 >> 172.21.15.63:0/551069777 conn(0x559cdc50c410 msgr2=0x559cdc5f55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/551069777 shutdown_connections 2024-05-27T08:10:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.831+0000 7f8eb49a9b80 1 -- 172.21.15.63:0/551069777 wait complete. 2024-05-27T08:10:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.832+0000 7f8eb49a9b80 20 client.22515 trim_cache size 0 max 16384 2024-05-27T08:10:35.834 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:10:35.834 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:10:35.834 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "ed16eb7e-8a67-4a94-a809-ffd3a728ffe6", 2024-05-27T08:10:35.834 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:10:35.834 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:10:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 -- 172.21.15.63:0/3132947200 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e840b2a70 msgr2=0x7f8e840b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e840b2a70 0x7f8e840b4f80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0110840 tx=0x7f8e8c00b370 comp rx=0 tx=0).stop 2024-05-27T08:10:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 -- 172.21.15.63:0/3132947200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ea01054f0 msgr2=0x7f8ea019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ea01054f0 0x7f8ea019ce90 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000e9a0 tx=0x7f8e9000ed60 comp rx=0 tx=0).stop 2024-05-27T08:10:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 -- 172.21.15.63:0/3132947200 shutdown_connections 2024-05-27T08:10:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8e840b2a70 0x7f8e840b4f80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea010e8d0 0x7f8ea01a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ea01054f0 0x7f8ea019ce90 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 --2- 172.21.15.63:0/3132947200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea0105e40 0x7f8ea019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 -- 172.21.15.63:0/3132947200 >> 172.21.15.63:0/3132947200 conn(0x7f8ea00fedb0 msgr2=0x7f8ea0103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 -- 172.21.15.63:0/3132947200 shutdown_connections 2024-05-27T08:10:35.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:35.833+0000 7f8ea5e00700 1 -- 172.21.15.63:0/3132947200 wait complete. 2024-05-27T08:10:36.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:35 smithi063 ceph-mon[31613]: pgmap v1174: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:10:36.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:10:36.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:10:36.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:10:36.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:35 smithi086 ceph-mon[34687]: pgmap v1174: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:10:36.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:10:36.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:10:36.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:10:36.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:35 smithi149 ceph-mon[35408]: pgmap v1174: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:10:36.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:10:36.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:10:36.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:10:36.750 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:10:36.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:10:36.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:36.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:10:36.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:10:36.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:36.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:10:36.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:36.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:10:36.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:36.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:10:36.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:36.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:10:36.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:36.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:10:36.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:36.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:36 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:10:37.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:10:37.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:10:37.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:10:37.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:10:37.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:10:37.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:10:37.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:10:37.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.212 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:36 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:10:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:10:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:10:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:10:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:10:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:10:37.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:10:37.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:10:37.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:36 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:10:37.304 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:10:38.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:37 smithi086 ceph-mon[34687]: pgmap v1175: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.0 MiB/s wr, 48 op/s 2024-05-27T08:10:38.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:37 smithi149 ceph-mon[35408]: pgmap v1175: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.0 MiB/s wr, 48 op/s 2024-05-27T08:10:38.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:37 smithi063 ceph-mon[31613]: pgmap v1175: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.0 MiB/s wr, 48 op/s 2024-05-27T08:10:40.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:39 smithi086 ceph-mon[34687]: pgmap v1176: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.4 MiB/s wr, 44 op/s 2024-05-27T08:10:40.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:39 smithi149 ceph-mon[35408]: pgmap v1176: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.4 MiB/s wr, 44 op/s 2024-05-27T08:10:40.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:39 smithi063 ceph-mon[31613]: pgmap v1176: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.4 MiB/s wr, 44 op/s 2024-05-27T08:10:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.432+0000 7f6215600700 1 -- 172.21.15.63:0/3026680877 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204003b40 con 0x7f6218105da0 2024-05-27T08:10:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.433+0000 7f621d600700 1 -- 172.21.15.63:0/3026680877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6218105da0 msgr2=0x7f621810e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.433+0000 7f621d600700 1 --2- 172.21.15.63:0/3026680877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6218105da0 0x7f621810e2f0 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f6204009990 tx=0x7f6204009ca0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.433+0000 7f621d600700 1 -- 172.21.15.63:0/3026680877 shutdown_connections 2024-05-27T08:10:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.433+0000 7f621d600700 1 --2- 172.21.15.63:0/3026680877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f621810e8c0 0x7f6218110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.433+0000 7f621d600700 1 --2- 172.21.15.63:0/3026680877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6218105450 0x7f6218105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.433+0000 7f621d600700 1 --2- 172.21.15.63:0/3026680877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6218105da0 0x7f621810e2f0 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.433+0000 7f621d600700 1 -- 172.21.15.63:0/3026680877 >> 172.21.15.63:0/3026680877 conn(0x7f62180fedb0 msgr2=0x7f62181011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.434+0000 7f621d600700 1 -- 172.21.15.63:0/3026680877 shutdown_connections 2024-05-27T08:10:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.434+0000 7f621d600700 1 -- 172.21.15.63:0/3026680877 wait complete. 2024-05-27T08:10:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.435+0000 7f621d600700 1 Processor -- start 2024-05-27T08:10:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.435+0000 7f621d600700 1 -- start start 2024-05-27T08:10:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.436+0000 7f621d600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6218105450 0x7f621819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.436+0000 7f621d600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6218105da0 0x7f621819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.436+0000 7f621d600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f621810e8c0 0x7f62181a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.436+0000 7f621d600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6218104570 con 0x7f6218105da0 2024-05-27T08:10:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.436+0000 7f621d600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62181043f0 con 0x7f621810e8c0 2024-05-27T08:10:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.436+0000 7f621d600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62181046f0 con 0x7f6218105450 2024-05-27T08:10:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.436+0000 7f6216000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6218105da0 0x7f621819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.436+0000 7f6216000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6218105da0 0x7f621819d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40892/0 (socket says 172.21.15.63:40892) 2024-05-27T08:10:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.436+0000 7f6216000700 1 -- 172.21.15.63:0/2008911239 learned_addr learned my addr 172.21.15.63:0/2008911239 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.437+0000 7f6216a00700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6218105450 0x7f621819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.437+0000 7f6217400700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f621810e8c0 0x7f62181a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.437+0000 7f6216000700 1 -- 172.21.15.63:0/2008911239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6218105450 msgr2=0x7f621819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.437+0000 7f6216000700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6218105450 0x7f621819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.437+0000 7f6216000700 1 -- 172.21.15.63:0/2008911239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f621810e8c0 msgr2=0x7f62181a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.437+0000 7f6216000700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f621810e8c0 0x7f62181a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.437+0000 7f6216000700 1 -- 172.21.15.63:0/2008911239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62181a1e40 con 0x7f6218105da0 2024-05-27T08:10:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.438+0000 7f6216000700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6218105da0 0x7f621819d510 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f62040057c0 tx=0x7f62040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.438+0000 7f620f400700 1 -- 172.21.15.63:0/2008911239 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204027070 con 0x7f6218105da0 2024-05-27T08:10:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.438+0000 7f620f400700 1 -- 172.21.15.63:0/2008911239 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f620400fb00 con 0x7f6218105da0 2024-05-27T08:10:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.438+0000 7f620f400700 1 -- 172.21.15.63:0/2008911239 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204022a80 con 0x7f6218105da0 2024-05-27T08:10:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.439+0000 7f621d600700 1 -- 172.21.15.63:0/2008911239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62181a20d0 con 0x7f6218105da0 2024-05-27T08:10:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.439+0000 7f621d600700 1 -- 172.21.15.63:0/2008911239 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62181a2560 con 0x7f6218105da0 2024-05-27T08:10:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.443+0000 7f620f400700 1 -- 172.21.15.63:0/2008911239 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f620402a070 con 0x7f6218105da0 2024-05-27T08:10:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.446+0000 7f622c128b80 1 Processor -- start 2024-05-27T08:10:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.446+0000 7f622c128b80 1 -- start start 2024-05-27T08:10:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.446+0000 7f620f400700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61fc0b29f0 0x7f61fc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.446+0000 7f622c128b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 0x555cc4b22330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.446+0000 7f622c128b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc49a3e40 0x555cc4b22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f622c128b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4b22e70 0x555cc4b2f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f622c128b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555cc4a50e40 con 0x555cc4b22e70 2024-05-27T08:10:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f622c128b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555cc4a409e0 con 0x555cc49a3e40 2024-05-27T08:10:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f622c128b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555cc4a44a20 con 0x555cc4a8f9f0 2024-05-27T08:10:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f620f400700 1 -- 172.21.15.63:0/2008911239 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f62040e3a40 con 0x7f6218105da0 2024-05-27T08:10:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f6217400700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61fc0b29f0 0x7f61fc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f6217400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4b22e70 0x555cc4b2f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f6216a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 0x555cc4b22330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f6216a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 0x555cc4b22330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41834/0 (socket says 172.21.15.63:41834) 2024-05-27T08:10:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f6216a00700 1 -- 172.21.15.63:0/1932254334 learned_addr learned my addr 172.21.15.63:0/1932254334 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.447+0000 7f6217400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4b22e70 0x555cc4b2f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40908/0 (socket says 172.21.15.63:40908) 2024-05-27T08:10:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.448+0000 7f6216000700 1 --2- 172.21.15.63:0/1932254334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc49a3e40 0x555cc4b22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.448+0000 7f6217400700 1 -- 172.21.15.63:0/1932254334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 msgr2=0x555cc4b22330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.448+0000 7f6217400700 1 --2- 172.21.15.63:0/1932254334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 0x555cc4b22330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.448+0000 7f6217400700 1 -- 172.21.15.63:0/1932254334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc49a3e40 msgr2=0x555cc4b22870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.448+0000 7f6217400700 1 --2- 172.21.15.63:0/1932254334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc49a3e40 0x555cc4b22870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.448+0000 7f6217400700 1 -- 172.21.15.63:0/1932254334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc4a09420 con 0x555cc4b22e70 2024-05-27T08:10:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.449+0000 7f6217400700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61fc0b29f0 0x7f61fc0b4f00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6210007af0 tx=0x7f6210009bf0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.449+0000 7f6217400700 1 --2- 172.21.15.63:0/1932254334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4b22e70 0x555cc4b2f110 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f6210007fc0 tx=0x7f621001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decdfb634f320a4b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.449+0000 7f6203e00700 1 -- 172.21.15.63:0/1932254334 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210019e90 con 0x555cc4b22e70 2024-05-27T08:10:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.449+0000 7f6203e00700 1 -- 172.21.15.63:0/1932254334 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f621001f070 con 0x555cc4b22e70 2024-05-27T08:10:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.449+0000 7f6203e00700 1 -- 172.21.15.63:0/1932254334 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210023ad0 con 0x555cc4b22e70 2024-05-27T08:10:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.450+0000 7f622c128b80 1 -- 172.21.15.63:0/1932254334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4b22e70 msgr2=0x555cc4b2f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.450+0000 7f622c128b80 1 --2- 172.21.15.63:0/1932254334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4b22e70 0x555cc4b2f110 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f6210007fc0 tx=0x7f621001c000 comp rx=0 tx=0).stop 2024-05-27T08:10:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.450+0000 7f622c128b80 1 -- 172.21.15.63:0/1932254334 shutdown_connections 2024-05-27T08:10:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.450+0000 7f622c128b80 1 --2- 172.21.15.63:0/1932254334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4b22e70 0x555cc4b2f110 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.450+0000 7f622c128b80 1 --2- 172.21.15.63:0/1932254334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 0x555cc4b22330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.450+0000 7f622c128b80 1 --2- 172.21.15.63:0/1932254334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc49a3e40 0x555cc4b22870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.451+0000 7f622c128b80 1 -- 172.21.15.63:0/1932254334 >> 172.21.15.63:0/1932254334 conn(0x555cc4a3d390 msgr2=0x555cc4b1e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.451+0000 7f622c128b80 1 -- 172.21.15.63:0/1932254334 shutdown_connections 2024-05-27T08:10:40.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.451+0000 7f622c128b80 1 -- 172.21.15.63:0/1932254334 wait complete. 2024-05-27T08:10:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.452+0000 7f622c128b80 1 -- start start 2024-05-27T08:10:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.452+0000 7f622c128b80 1 Processor -- start 2024-05-27T08:10:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f622c128b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc49a3e40 0x555cc4b2c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f622c128b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b2cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f622c128b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4b22e70 0x555cc4b2d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f622c128b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555cc4946fa0 con 0x555cc4a8f9f0 2024-05-27T08:10:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f622c128b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555cc49e1760 con 0x555cc49a3e40 2024-05-27T08:10:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f622c128b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555cc4b2fb20 con 0x555cc4b22e70 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f622c128b80 10 client.0 _ll_register_callbacks cb 0x555cc49eac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f6216000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b2cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f6216000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b2cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40916/0 (socket says 172.21.15.63:40916) 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f6216000700 1 -- 172.21.15.63:0/104724464 learned_addr learned my addr 172.21.15.63:0/104724464 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.453+0000 7f6216a00700 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc49a3e40 0x555cc4b2c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.454+0000 7f6217400700 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4b22e70 0x555cc4b2d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.454+0000 7f6216000700 1 -- 172.21.15.63:0/104724464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4b22e70 msgr2=0x555cc4b2d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.454+0000 7f6216000700 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4b22e70 0x555cc4b2d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.454+0000 7f6216000700 1 -- 172.21.15.63:0/104724464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc49a3e40 msgr2=0x555cc4b2c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.454+0000 7f6216000700 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc49a3e40 0x555cc4b2c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.454+0000 7f6216000700 1 -- 172.21.15.63:0/104724464 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc49908d0 con 0x555cc4a8f9f0 2024-05-27T08:10:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.454+0000 7f6216000700 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b2cbe0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f62040e0dd0 tx=0x7f6204022520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2490cd8eab385cb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.454+0000 7f6200c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.455+0000 7f6200c00700 1 -- 172.21.15.63:0/104724464 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62040aa990 con 0x555cc4a8f9f0 2024-05-27T08:10:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.455+0000 7f6200c00700 1 -- 172.21.15.63:0/104724464 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f62040aabf0 con 0x555cc4a8f9f0 2024-05-27T08:10:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.455+0000 7f622c128b80 1 -- 172.21.15.63:0/104724464 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f621001f070 con 0x555cc4a8f9f0 2024-05-27T08:10:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.455+0000 7f6200c00700 1 -- 172.21.15.63:0/104724464 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f62040aed40 con 0x555cc4a8f9f0 2024-05-27T08:10:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.455+0000 7f622c128b80 10 client.22527 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.455+0000 7f622c128b80 1 -- 172.21.15.63:0/104724464 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555cc4a11060 con 0x555cc4a8f9f0 2024-05-27T08:10:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.455+0000 7f6200c00700 1 -- 172.21.15.63:0/104724464 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f621001f070 con 0x555cc4a8f9f0 2024-05-27T08:10:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.456+0000 7f6200c00700 1 -- 172.21.15.63:0/104724464 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f62040cbbb0 con 0x555cc4a8f9f0 2024-05-27T08:10:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.456+0000 7f622c128b80 10 client.22527 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.456+0000 7f622c128b80 10 client.22527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.456+0000 7f622c128b80 20 client.22527 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.456+0000 7f622c128b80 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555cc4b3c9a0 0x555cc4b3ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.456+0000 7f622c128b80 4 client.22527 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.456+0000 7f622c128b80 1 -- 172.21.15.63:0/104724464 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cc4a52d90 con 0x555cc4b3c9a0 2024-05-27T08:10:40.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.457+0000 7f6216a00700 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555cc4b3c9a0 0x555cc4b3ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.458+0000 7f6216a00700 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555cc4b3c9a0 0x555cc4b3ee00 crc :-1 s=READY pgs=662 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=901f7b99c55d303 server_cookie=a0bef3962ddadd6f in_seq=0 out_seq=0 2024-05-27T08:10:40.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.458+0000 7f6200c00700 10 client.22527 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:40.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f6200c00700 1 -- 172.21.15.63:0/104724464 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555cc4a52d90 con 0x555cc4b3c9a0 2024-05-27T08:10:40.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f6200c00700 10 client.22527 handle_command_reply: tid=0 2024-05-27T08:10:40.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f622c128b80 1 client.22527 shutdown 2024-05-27T08:10:40.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f622c128b80 1 -- 172.21.15.63:0/104724464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 msgr2=0x555cc4b2cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:40.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f622c128b80 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b2cbe0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f62040e0dd0 tx=0x7f6204022520 comp rx=0 tx=0).stop 2024-05-27T08:10:40.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f622c128b80 1 -- 172.21.15.63:0/104724464 shutdown_connections 2024-05-27T08:10:40.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f622c128b80 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555cc4b3c9a0 0x555cc4b3ee00 crc :-1 s=READY pgs=662 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f622c128b80 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4b22e70 0x555cc4b2d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f622c128b80 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc49a3e40 0x555cc4b2c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f622c128b80 1 --2- 172.21.15.63:0/104724464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b2cbe0 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.460+0000 7f622c128b80 1 -- 172.21.15.63:0/104724464 >> 172.21.15.63:0/104724464 conn(0x555cc4a3d390 msgr2=0x555cc4b22310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.461+0000 7f6200c00700 0 client.22527 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.461+0000 7f622c128b80 1 -- 172.21.15.63:0/104724464 shutdown_connections 2024-05-27T08:10:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.461+0000 7f622c128b80 1 -- 172.21.15.63:0/104724464 wait complete. 2024-05-27T08:10:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.461+0000 7f622c128b80 20 client.22527 trim_cache size 0 max 16384 2024-05-27T08:10:40.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.493+0000 7f622c128b80 1 Processor -- start 2024-05-27T08:10:40.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.493+0000 7f622c128b80 1 -- start start 2024-05-27T08:10:40.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.493+0000 7f622c128b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc49a3e40 0x555cc4b2d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.493+0000 7f622c128b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 0x555cc4b27e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.493+0000 7f622c128b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 0x555cc4b44600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.493+0000 7f622c128b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62181b4f20 con 0x555cc49a3e40 2024-05-27T08:10:40.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.493+0000 7f622c128b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62181b27b0 con 0x555cc4b22e70 2024-05-27T08:10:40.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.493+0000 7f622c128b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62181b0620 con 0x555cc4a8f9f0 2024-05-27T08:10:40.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.494+0000 7f6216a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc49a3e40 0x555cc4b2d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.494+0000 7f6216a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc49a3e40 0x555cc4b2d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40926/0 (socket says 172.21.15.63:40926) 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.494+0000 7f6216a00700 1 -- 172.21.15.63:0/3392691381 learned_addr learned my addr 172.21.15.63:0/3392691381 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.494+0000 7f6216000700 1 --2- 172.21.15.63:0/3392691381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 0x555cc4b27e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.494+0000 7f6217400700 1 --2- 172.21.15.63:0/3392691381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 0x555cc4b44600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.495+0000 7f6216a00700 1 -- 172.21.15.63:0/3392691381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 msgr2=0x555cc4b27e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.495+0000 7f6216a00700 1 --2- 172.21.15.63:0/3392691381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 0x555cc4b27e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.495+0000 7f6216a00700 1 -- 172.21.15.63:0/3392691381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 msgr2=0x555cc4b44600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.495+0000 7f6216a00700 1 --2- 172.21.15.63:0/3392691381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 0x555cc4b44600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.495+0000 7f6216a00700 1 -- 172.21.15.63:0/3392691381 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc4b1e9d0 con 0x555cc49a3e40 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.495+0000 7f6217400700 1 --2- 172.21.15.63:0/3392691381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 0x555cc4b44600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.495+0000 7f6216a00700 1 --2- 172.21.15.63:0/3392691381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc49a3e40 0x555cc4b2d090 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x555cc4b24ff0 tx=0x7f62080073d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed2d6faccf5fd6b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.495+0000 7f6203400700 1 -- 172.21.15.63:0/3392691381 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208004050 con 0x555cc49a3e40 2024-05-27T08:10:40.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.495+0000 7f6203400700 1 -- 172.21.15.63:0/3392691381 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6208003680 con 0x555cc49a3e40 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.496+0000 7f6203400700 1 -- 172.21.15.63:0/3392691381 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620801d6e0 con 0x555cc49a3e40 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.497+0000 7f622c128b80 1 -- 172.21.15.63:0/3392691381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc49a3e40 msgr2=0x555cc4b2d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.497+0000 7f622c128b80 1 --2- 172.21.15.63:0/3392691381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc49a3e40 0x555cc4b2d090 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x555cc4b24ff0 tx=0x7f62080073d0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.497+0000 7f622c128b80 1 -- 172.21.15.63:0/3392691381 shutdown_connections 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.497+0000 7f622c128b80 1 --2- 172.21.15.63:0/3392691381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 0x555cc4b44600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.497+0000 7f622c128b80 1 --2- 172.21.15.63:0/3392691381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc49a3e40 0x555cc4b2d090 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.497+0000 7f622c128b80 1 --2- 172.21.15.63:0/3392691381 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc4a8f9f0 0x555cc4b27e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.497+0000 7f622c128b80 1 -- 172.21.15.63:0/3392691381 >> 172.21.15.63:0/3392691381 conn(0x555cc4a3d390 msgr2=0x555cc4b1e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.497+0000 7f622c128b80 1 -- 172.21.15.63:0/3392691381 shutdown_connections 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.497+0000 7f622c128b80 1 -- 172.21.15.63:0/3392691381 wait complete. 2024-05-27T08:10:40.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.498+0000 7f622c128b80 1 -- start start 2024-05-27T08:10:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.498+0000 7f622c128b80 1 Processor -- start 2024-05-27T08:10:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f622c128b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc49a3e40 0x555cc4b34fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f622c128b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b35500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f622c128b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 0x555cc4b2fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f622c128b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62181b0c70 con 0x555cc4a8f9f0 2024-05-27T08:10:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f622c128b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62181b20e0 con 0x555cc4b22e70 2024-05-27T08:10:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f622c128b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62181b1340 con 0x555cc49a3e40 2024-05-27T08:10:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f622c128b80 10 client.0 _ll_register_callbacks cb 0x555cc49eac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:40.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f6216a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc49a3e40 0x555cc4b34fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f6216000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b35500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f6216a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc49a3e40 0x555cc4b34fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41868/0 (socket says 172.21.15.63:41868) 2024-05-27T08:10:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f6216a00700 1 -- 172.21.15.63:0/1058202661 learned_addr learned my addr 172.21.15.63:0/1058202661 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.499+0000 7f6217400700 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 0x555cc4b2fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.500+0000 7f6216000700 1 -- 172.21.15.63:0/1058202661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc49a3e40 msgr2=0x555cc4b34fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.500+0000 7f6216000700 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc49a3e40 0x555cc4b34fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.500+0000 7f6216000700 1 -- 172.21.15.63:0/1058202661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 msgr2=0x555cc4b2fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.500+0000 7f6216000700 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 0x555cc4b2fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.500+0000 7f6216000700 1 -- 172.21.15.63:0/1058202661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc4b304a0 con 0x555cc4a8f9f0 2024-05-27T08:10:40.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.500+0000 7f6216000700 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b35500 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f62040aa920 tx=0x7f62040cbde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5dbdf74952681f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:40.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.500+0000 7f6200c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:40.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.501+0000 7f6200c00700 1 -- 172.21.15.63:0/1058202661 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62040aa990 con 0x555cc4a8f9f0 2024-05-27T08:10:40.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.501+0000 7f6200c00700 1 -- 172.21.15.63:0/1058202661 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7f62040b32d0 con 0x555cc4a8f9f0 2024-05-27T08:10:40.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.501+0000 7f622c128b80 1 -- 172.21.15.63:0/1058202661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6208003680 con 0x555cc4a8f9f0 2024-05-27T08:10:40.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.501+0000 7f6200c00700 1 -- 172.21.15.63:0/1058202661 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f62040aae00 con 0x555cc4a8f9f0 2024-05-27T08:10:40.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.501+0000 7f622c128b80 10 client.22533 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:40.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.501+0000 7f622c128b80 1 -- 172.21.15.63:0/1058202661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555cc4b304a0 con 0x555cc4a8f9f0 2024-05-27T08:10:40.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.501+0000 7f6200c00700 1 -- 172.21.15.63:0/1058202661 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6208003680 con 0x555cc4a8f9f0 2024-05-27T08:10:40.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.502+0000 7f6200c00700 1 -- 172.21.15.63:0/1058202661 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f62040050d0 con 0x555cc4a8f9f0 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.502+0000 7f622c128b80 10 client.22533 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.502+0000 7f622c128b80 10 client.22533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.502+0000 7f622c128b80 20 client.22533 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.502+0000 7f622c128b80 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555cc4bb0a20 0x555cc4bb2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.502+0000 7f622c128b80 4 client.22533 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.502+0000 7f622c128b80 1 -- 172.21.15.63:0/1058202661 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555cc4a4c570 con 0x555cc4bb0a20 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.503+0000 7f6216a00700 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555cc4bb0a20 0x555cc4bb2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.504+0000 7f6216a00700 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555cc4bb0a20 0x555cc4bb2e40 crc :-1 s=READY pgs=663 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7c9a9d8b270cb0 server_cookie=ea13d648d066ccb2 in_seq=0 out_seq=0 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.504+0000 7f6200c00700 10 client.22533 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.504+0000 7f6200c00700 1 -- 172.21.15.63:0/1058202661 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555cc4a4c570 con 0x555cc4bb0a20 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.504+0000 7f6200c00700 10 client.22533 handle_command_reply: tid=0 2024-05-27T08:10:40.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.504+0000 7f622c128b80 1 client.22533 shutdown 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.504+0000 7f622c128b80 1 -- 172.21.15.63:0/1058202661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 msgr2=0x555cc4b35500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f622c128b80 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b35500 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f62040aa920 tx=0x7f62040cbde0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f622c128b80 1 -- 172.21.15.63:0/1058202661 shutdown_connections 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f622c128b80 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x555cc4bb0a20 0x555cc4bb2e40 crc :-1 s=READY pgs=663 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f622c128b80 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc4b22e70 0x555cc4b2fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f622c128b80 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555cc49a3e40 0x555cc4b34fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f622c128b80 1 --2- 172.21.15.63:0/1058202661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555cc4a8f9f0 0x555cc4b35500 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f622c128b80 1 -- 172.21.15.63:0/1058202661 >> 172.21.15.63:0/1058202661 conn(0x555cc4a3d390 msgr2=0x555cc4b1e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f6200c00700 0 client.22533 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f622c128b80 1 -- 172.21.15.63:0/1058202661 shutdown_connections 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f622c128b80 1 -- 172.21.15.63:0/1058202661 wait complete. 2024-05-27T08:10:40.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.505+0000 7f622c128b80 20 client.22533 trim_cache size 0 max 16384 2024-05-27T08:10:40.509 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:10:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:10:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:10:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "ed16eb7e-8a67-4a94-a809-ffd3a728ffe6": { 2024-05-27T08:10:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:10:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "ed16eb7e-8a67-4a94-a809-ffd3a728ffe6", 2024-05-27T08:10:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:10:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:10:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:10:40.510 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:10:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.506+0000 7f621d600700 1 -- 172.21.15.63:0/2008911239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61fc0b29f0 msgr2=0x7f61fc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:40.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.506+0000 7f621d600700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61fc0b29f0 0x7f61fc0b4f00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6210007af0 tx=0x7f6210009bf0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.506+0000 7f621d600700 1 -- 172.21.15.63:0/2008911239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6218105da0 msgr2=0x7f621819d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.506+0000 7f621d600700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6218105da0 0x7f621819d510 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f62040057c0 tx=0x7f62040057f0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.506+0000 7f621d600700 1 -- 172.21.15.63:0/2008911239 shutdown_connections 2024-05-27T08:10:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.506+0000 7f621d600700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61fc0b29f0 0x7f61fc0b4f00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.506+0000 7f621d600700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f621810e8c0 0x7f62181a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.506+0000 7f621d600700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6218105450 0x7f621819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.506+0000 7f621d600700 1 --2- 172.21.15.63:0/2008911239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6218105da0 0x7f621819d510 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.506+0000 7f621d600700 1 -- 172.21.15.63:0/2008911239 >> 172.21.15.63:0/2008911239 conn(0x7f62180fedb0 msgr2=0x7f62181061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.507+0000 7f621d600700 1 -- 172.21.15.63:0/2008911239 shutdown_connections 2024-05-27T08:10:40.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:10:40.507+0000 7f621d600700 1 -- 172.21.15.63:0/2008911239 wait complete. 2024-05-27T08:10:41.578 INFO:tasks.cephfs.filesystem:scrub status for tag:ed16eb7e-8a67-4a94-a809-ffd3a728ffe6 - {'path': '/', 'tag': 'ed16eb7e-8a67-4a94-a809-ffd3a728ffe6', 'options': 'recursive,force'} 2024-05-27T08:10:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.106+0000 7fca0e800640 1 -- 172.21.15.63:0/4201104179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca08102ab0 msgr2=0x7fca08102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.106+0000 7fca0e800640 1 --2- 172.21.15.63:0/4201104179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca08102ab0 0x7fca08102ec0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009900 tx=0x7fc9f402a410 comp rx=0 tx=0).stop 2024-05-27T08:10:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.106+0000 7fca0e800640 1 -- 172.21.15.63:0/4201104179 shutdown_connections 2024-05-27T08:10:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.106+0000 7fca0e800640 1 --2- 172.21.15.63:0/4201104179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca0810d340 0x7fca0810f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.106+0000 7fca0e800640 1 --2- 172.21.15.63:0/4201104179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca08102ab0 0x7fca08102ec0 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.106+0000 7fca0e800640 1 --2- 172.21.15.63:0/4201104179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca081089e0 0x7fca08102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.106+0000 7fca0e800640 1 -- 172.21.15.63:0/4201104179 >> 172.21.15.63:0/4201104179 conn(0x7fca080fe210 msgr2=0x7fca08100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.106+0000 7fca0e800640 1 -- 172.21.15.63:0/4201104179 shutdown_connections 2024-05-27T08:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.106+0000 7fca0e800640 1 -- 172.21.15.63:0/4201104179 wait complete. 2024-05-27T08:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.108+0000 7fca0e800640 1 Processor -- start 2024-05-27T08:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.108+0000 7fca0e800640 1 -- start start 2024-05-27T08:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.108+0000 7fca0e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca08102ab0 0x7fca0819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.108+0000 7fca0e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca081089e0 0x7fca0819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.108+0000 7fca0e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca0810d340 0x7fca081a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.108+0000 7fca0e800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fca08112a50 con 0x7fca08102ab0 2024-05-27T08:10:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.108+0000 7fca0e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca081128d0 con 0x7fca0810d340 2024-05-27T08:10:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.108+0000 7fca0e800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fca08112bd0 con 0x7fca081089e0 2024-05-27T08:10:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca08102ab0 0x7fca0819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca08102ab0 0x7fca0819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40956/0 (socket says 172.21.15.63:40956) 2024-05-27T08:10:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0d400640 1 -- 172.21.15.63:0/1414140293 learned_addr learned my addr 172.21.15.63:0/1414140293 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0de00640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca0810d340 0x7fca081a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0ca00640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca081089e0 0x7fca0819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0d400640 1 -- 172.21.15.63:0/1414140293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca081089e0 msgr2=0x7fca0819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0d400640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca081089e0 0x7fca0819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0d400640 1 -- 172.21.15.63:0/1414140293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca0810d340 msgr2=0x7fca081a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0d400640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca0810d340 0x7fca081a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0d400640 1 -- 172.21.15.63:0/1414140293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca081a15f0 con 0x7fca08102ab0 2024-05-27T08:10:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.109+0000 7fca0ca00640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca081089e0 0x7fca0819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:10:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.110+0000 7fca0d400640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca08102ab0 0x7fca0819c790 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00d740 tx=0x7fc9fc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.110+0000 7fca06000640 1 -- 172.21.15.63:0/1414140293 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc017070 con 0x7fca08102ab0 2024-05-27T08:10:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.110+0000 7fca06000640 1 -- 172.21.15.63:0/1414140293 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9fc00de50 con 0x7fca08102ab0 2024-05-27T08:10:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.110+0000 7fca0e800640 1 -- 172.21.15.63:0/1414140293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca081a18e0 con 0x7fca08102ab0 2024-05-27T08:10:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.110+0000 7fca06000640 1 -- 172.21.15.63:0/1414140293 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc004c60 con 0x7fca08102ab0 2024-05-27T08:10:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.110+0000 7fca0e800640 1 -- 172.21.15.63:0/1414140293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca081a1de0 con 0x7fca08102ab0 2024-05-27T08:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.114+0000 7fca16e42740 1 Processor -- start 2024-05-27T08:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.114+0000 7fca16e42740 1 -- start start 2024-05-27T08:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca16e42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca16e42740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bd992f0 0x56067c164fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca16e42740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c1654e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca16e42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56067c0d5b90 con 0x56067bdc21b0 2024-05-27T08:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca16e42740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56067c0c5730 con 0x56067bd992f0 2024-05-27T08:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca06000640 1 -- 172.21.15.63:0/1414140293 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc9fc004e00 con 0x7fca08102ab0 2024-05-27T08:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca16e42740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56067c0c9770 con 0x56067bda14d0 2024-05-27T08:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c164a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40958/0 (socket says 172.21.15.63:40958) 2024-05-27T08:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca0de00640 1 -- 172.21.15.63:0/1055898306 learned_addr learned my addr 172.21.15.63:0/1055898306 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca0ca00640 1 --2- 172.21.15.63:0/1055898306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c1654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.115+0000 7fca0d400640 1 --2- 172.21.15.63:0/1055898306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bd992f0 0x56067c164fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.116+0000 7fca0de00640 1 -- 172.21.15.63:0/1055898306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 msgr2=0x56067c1654e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.116+0000 7fca0de00640 1 --2- 172.21.15.63:0/1055898306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c1654e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.116+0000 7fca0de00640 1 -- 172.21.15.63:0/1055898306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bd992f0 msgr2=0x56067c164fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.116+0000 7fca0de00640 1 --2- 172.21.15.63:0/1055898306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bd992f0 0x56067c164fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.116+0000 7fca0de00640 1 -- 172.21.15.63:0/1055898306 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56067c05e8b0 con 0x56067bdc21b0 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.116+0000 7fca0de00640 1 --2- 172.21.15.63:0/1055898306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c164a60 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fca00009ee0 tx=0x7fca00010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d32d28eb2875d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.116+0000 7fc9fa600640 1 -- 172.21.15.63:0/1055898306 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00016070 con 0x56067bdc21b0 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.117+0000 7fc9fa600640 1 -- 172.21.15.63:0/1055898306 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca00004d20 con 0x56067bdc21b0 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.117+0000 7fca06000640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9f00b2950 0x7fc9f00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.117+0000 7fc9fa600640 1 -- 172.21.15.63:0/1055898306 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00012650 con 0x56067bdc21b0 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.117+0000 7fca06000640 1 -- 172.21.15.63:0/1414140293 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7fc9fc0e2100 con 0x7fca08102ab0 2024-05-27T08:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.117+0000 7fca0d400640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9f00b2950 0x7fc9f00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.118+0000 7fca16e42740 1 -- 172.21.15.63:0/1055898306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 msgr2=0x56067c164a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.118+0000 7fca16e42740 1 --2- 172.21.15.63:0/1055898306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c164a60 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fca00009ee0 tx=0x7fca00010740 comp rx=0 tx=0).stop 2024-05-27T08:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.118+0000 7fca0d400640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9f00b2950 0x7fc9f00b4e60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0df540 tx=0x7fc9fc009f00 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.118+0000 7fca16e42740 1 -- 172.21.15.63:0/1055898306 shutdown_connections 2024-05-27T08:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.118+0000 7fca16e42740 1 --2- 172.21.15.63:0/1055898306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c1654e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.118+0000 7fca16e42740 1 --2- 172.21.15.63:0/1055898306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bd992f0 0x56067c164fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.118+0000 7fca16e42740 1 --2- 172.21.15.63:0/1055898306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c164a60 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.118+0000 7fca16e42740 1 -- 172.21.15.63:0/1055898306 >> 172.21.15.63:0/1055898306 conn(0x56067bdb6300 msgr2=0x56067c161210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.118+0000 7fca16e42740 1 -- 172.21.15.63:0/1055898306 shutdown_connections 2024-05-27T08:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.118+0000 7fca16e42740 1 -- 172.21.15.63:0/1055898306 wait complete. 2024-05-27T08:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.119+0000 7fca16e42740 1 -- start start 2024-05-27T08:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.119+0000 7fca16e42740 1 Processor -- start 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.119+0000 7fca16e42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca16e42740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c16f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca16e42740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bdc21b0 0x56067c17aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca16e42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56067bdab790 con 0x56067bd992f0 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca16e42740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56067c06dbc0 con 0x56067bdc21b0 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca16e42740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56067c171b90 con 0x56067bda14d0 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca16e42740 10 client.0 _ll_register_callbacks cb 0x56067c04b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40970/0 (socket says 172.21.15.63:40970) 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca0de00640 1 -- 172.21.15.63:0/3003602155 learned_addr learned my addr 172.21.15.63:0/3003602155 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca0ca00640 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c16f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.120+0000 7fca0de00640 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bdc21b0 0x56067c17aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.121+0000 7fca0de00640 1 -- 172.21.15.63:0/3003602155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 msgr2=0x56067c16f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.121+0000 7fca0de00640 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c16f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.121+0000 7fca0de00640 1 -- 172.21.15.63:0/3003602155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bdc21b0 msgr2=0x56067c17aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.121+0000 7fca0de00640 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bdc21b0 0x56067c17aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.121+0000 7fca0de00640 1 -- 172.21.15.63:0/3003602155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56067bdda210 con 0x56067bd992f0 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.121+0000 7fca0de00640 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16ec80 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7fca00006010 tx=0x7fca00005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4f71f7e24c9c2b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.121+0000 7fc9eea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.121+0000 7fc9eea00640 1 -- 172.21.15.63:0/3003602155 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00016030 con 0x56067bd992f0 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.121+0000 7fc9eea00640 1 -- 172.21.15.63:0/3003602155 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7fca00005690 con 0x56067bd992f0 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.121+0000 7fca16e42740 1 -- 172.21.15.63:0/3003602155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca00004d20 con 0x56067bd992f0 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.122+0000 7fc9eea00640 1 -- 172.21.15.63:0/3003602155 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fca00012450 con 0x56067bd992f0 2024-05-27T08:10:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.122+0000 7fca16e42740 10 client.22545 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.122+0000 7fca16e42740 1 -- 172.21.15.63:0/3003602155 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56067bea6620 con 0x56067bd992f0 2024-05-27T08:10:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.122+0000 7fc9eea00640 1 -- 172.21.15.63:0/3003602155 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca00004d20 con 0x56067bd992f0 2024-05-27T08:10:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.123+0000 7fc9eea00640 1 -- 172.21.15.63:0/3003602155 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fca0000f040 con 0x56067bd992f0 2024-05-27T08:10:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.123+0000 7fca16e42740 10 client.22545 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.123+0000 7fca16e42740 10 client.22545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.123+0000 7fca16e42740 20 client.22545 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.123+0000 7fca16e42740 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56067bd80d80 0x56067c180d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.123+0000 7fca16e42740 4 client.22545 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.123+0000 7fca16e42740 1 -- 172.21.15.63:0/3003602155 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56067c0d7ae0 con 0x56067bd80d80 2024-05-27T08:10:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.124+0000 7fca0ca00640 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56067bd80d80 0x56067c180d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.124+0000 7fca0ca00640 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56067bd80d80 0x56067c180d70 crc :-1 s=READY pgs=664 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6983978356938957 server_cookie=c5e80a9df285472a in_seq=0 out_seq=0 2024-05-27T08:10:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.124+0000 7fc9eea00640 10 client.22545 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.126+0000 7fc9eea00640 1 -- 172.21.15.63:0/3003602155 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56067c0d7ae0 con 0x56067bd80d80 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.126+0000 7fc9eea00640 10 client.22545 handle_command_reply: tid=0 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 client.22545 shutdown 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 -- 172.21.15.63:0/3003602155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 msgr2=0x56067c16ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16ec80 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7fca00006010 tx=0x7fca00005ac0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 -- 172.21.15.63:0/3003602155 shutdown_connections 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56067bd80d80 0x56067c180d70 crc :-1 s=READY pgs=664 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bdc21b0 0x56067c17aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c16f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 --2- 172.21.15.63:0/3003602155 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16ec80 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 -- 172.21.15.63:0/3003602155 >> 172.21.15.63:0/3003602155 conn(0x56067bdb6300 msgr2=0x56067c164890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fc9eea00640 0 client.22545 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 -- 172.21.15.63:0/3003602155 shutdown_connections 2024-05-27T08:10:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 1 -- 172.21.15.63:0/3003602155 wait complete. 2024-05-27T08:10:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.127+0000 7fca16e42740 20 client.22545 trim_cache size 0 max 16384 2024-05-27T08:10:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.149+0000 7fca16e42740 1 Processor -- start 2024-05-27T08:10:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.149+0000 7fca16e42740 1 -- start start 2024-05-27T08:10:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.149+0000 7fca16e42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.149+0000 7fca16e42740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c170dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.149+0000 7fca16e42740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bdc21b0 0x56067c171310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.149+0000 7fca16e42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fca081b5b70 con 0x56067bd992f0 2024-05-27T08:10:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.149+0000 7fca16e42740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca081b3440 con 0x56067bdc21b0 2024-05-27T08:10:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.149+0000 7fca16e42740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fca081b12b0 con 0x56067bda14d0 2024-05-27T08:10:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.150+0000 7fca0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.150+0000 7fca0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40984/0 (socket says 172.21.15.63:40984) 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.150+0000 7fca0de00640 1 -- 172.21.15.63:0/1113293993 learned_addr learned my addr 172.21.15.63:0/1113293993 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.150+0000 7fca0ca00640 1 --2- 172.21.15.63:0/1113293993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c170dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.150+0000 7fca0de00640 1 --2- 172.21.15.63:0/1113293993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bdc21b0 0x56067c171310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.150+0000 7fca0de00640 1 -- 172.21.15.63:0/1113293993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 msgr2=0x56067c170dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.150+0000 7fca0de00640 1 --2- 172.21.15.63:0/1113293993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c170dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.150+0000 7fca0de00640 1 -- 172.21.15.63:0/1113293993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bdc21b0 msgr2=0x56067c171310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.150+0000 7fca0de00640 1 --2- 172.21.15.63:0/1113293993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bdc21b0 0x56067c171310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.150+0000 7fca0de00640 1 -- 172.21.15.63:0/1113293993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56067c16d1d0 con 0x56067bd992f0 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.151+0000 7fca0de00640 1 --2- 172.21.15.63:0/1113293993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16a440 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7fca00004ae0 tx=0x7fca00005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce61da2c3af0f8e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.151+0000 7fc9f9c00640 1 -- 172.21.15.63:0/1113293993 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0003c070 con 0x56067bd992f0 2024-05-27T08:10:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.151+0000 7fc9f9c00640 1 -- 172.21.15.63:0/1113293993 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca00024040 con 0x56067bd992f0 2024-05-27T08:10:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.151+0000 7fc9f9c00640 1 -- 172.21.15.63:0/1113293993 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00016030 con 0x56067bd992f0 2024-05-27T08:10:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.152+0000 7fca16e42740 1 -- 172.21.15.63:0/1113293993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 msgr2=0x56067c16a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.152+0000 7fca16e42740 1 --2- 172.21.15.63:0/1113293993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16a440 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7fca00004ae0 tx=0x7fca00005f10 comp rx=0 tx=0).stop 2024-05-27T08:10:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.152+0000 7fca16e42740 1 -- 172.21.15.63:0/1113293993 shutdown_connections 2024-05-27T08:10:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.152+0000 7fca16e42740 1 --2- 172.21.15.63:0/1113293993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bdc21b0 0x56067c171310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.152+0000 7fca16e42740 1 --2- 172.21.15.63:0/1113293993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bda14d0 0x56067c170dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.152+0000 7fca16e42740 1 --2- 172.21.15.63:0/1113293993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bd992f0 0x56067c16a440 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.152+0000 7fca16e42740 1 -- 172.21.15.63:0/1113293993 >> 172.21.15.63:0/1113293993 conn(0x56067bdb6300 msgr2=0x56067c16efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.152+0000 7fca16e42740 1 -- 172.21.15.63:0/1113293993 shutdown_connections 2024-05-27T08:10:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.152+0000 7fca16e42740 1 -- 172.21.15.63:0/1113293993 wait complete. 2024-05-27T08:10:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.153+0000 7fca16e42740 1 -- start start 2024-05-27T08:10:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.153+0000 7fca16e42740 1 Processor -- start 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca16e42740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bd992f0 0x56067c183450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca16e42740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bda14d0 0x56067c183990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca16e42740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c1e1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca16e42740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fca081b1900 con 0x56067bdc21b0 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca16e42740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca081b2d70 con 0x56067bda14d0 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca16e42740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fca081b1fd0 con 0x56067bd992f0 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca16e42740 10 client.0 _ll_register_callbacks cb 0x7fca00009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c1e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca0de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c1e1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40986/0 (socket says 172.21.15.63:40986) 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca0de00640 1 -- 172.21.15.63:0/1551571640 learned_addr learned my addr 172.21.15.63:0/1551571640 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.154+0000 7fca0de00640 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bd992f0 0x56067c183450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fca0ca00640 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bda14d0 0x56067c183990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fca0de00640 1 -- 172.21.15.63:0/1551571640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bd992f0 msgr2=0x56067c183450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fca0de00640 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bd992f0 0x56067c183450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fca0de00640 1 -- 172.21.15.63:0/1551571640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bda14d0 msgr2=0x56067c183990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fca0de00640 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bda14d0 0x56067c183990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fca0de00640 1 -- 172.21.15.63:0/1551571640 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56067c1e1790 con 0x56067bdc21b0 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fca0de00640 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c1e1200 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7fca0001a800 tx=0x7fca0001a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b23f5f9bf6e742 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fc9eea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fc9eea00640 1 -- 172.21.15.63:0/1551571640 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0003c070 con 0x56067bdc21b0 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fc9eea00640 1 -- 172.21.15.63:0/1551571640 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7fca000056a0 con 0x56067bdc21b0 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.155+0000 7fca16e42740 1 -- 172.21.15.63:0/1551571640 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca00024040 con 0x56067bdc21b0 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fc9eea00640 1 -- 172.21.15.63:0/1551571640 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fca00016030 con 0x56067bdc21b0 2024-05-27T08:10:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fca16e42740 10 client.22551 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fca16e42740 1 -- 172.21.15.63:0/1551571640 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56067c1e8ac0 con 0x56067bdc21b0 2024-05-27T08:10:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fc9eea00640 1 -- 172.21.15.63:0/1551571640 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca00024040 con 0x56067bdc21b0 2024-05-27T08:10:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fc9eea00640 1 -- 172.21.15.63:0/1551571640 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fca000464a0 con 0x56067bdc21b0 2024-05-27T08:10:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fca16e42740 10 client.22551 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fca16e42740 10 client.22551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fca16e42740 20 client.22551 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fca16e42740 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56067bd561c0 0x56067c1eb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fca16e42740 4 client.22551 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:10:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.156+0000 7fca16e42740 1 -- 172.21.15.63:0/1551571640 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56067c165420 con 0x56067bd561c0 2024-05-27T08:10:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.157+0000 7fca0ca00640 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56067bd561c0 0x56067c1eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.158+0000 7fca0ca00640 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56067bd561c0 0x56067c1eb110 crc :-1 s=READY pgs=665 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e7c46f2483ec2e server_cookie=78e58d216994b470 in_seq=0 out_seq=0 2024-05-27T08:10:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.158+0000 7fc9eea00640 10 client.22551 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fc9eea00640 1 -- 172.21.15.63:0/1551571640 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56067c165420 con 0x56067bd561c0 2024-05-27T08:10:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fc9eea00640 10 client.22551 handle_command_reply: tid=0 2024-05-27T08:10:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fca16e42740 1 client.22551 shutdown 2024-05-27T08:10:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fca16e42740 1 -- 172.21.15.63:0/1551571640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 msgr2=0x56067c1e1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fca16e42740 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c1e1200 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7fca0001a800 tx=0x7fca0001a970 comp rx=0 tx=0).stop 2024-05-27T08:10:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fca16e42740 1 -- 172.21.15.63:0/1551571640 shutdown_connections 2024-05-27T08:10:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fca16e42740 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56067bd561c0 0x56067c1eb110 crc :-1 s=READY pgs=665 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fca16e42740 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56067bdc21b0 0x56067c1e1200 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fca16e42740 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56067bda14d0 0x56067c183990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fca16e42740 1 --2- 172.21.15.63:0/1551571640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56067bd992f0 0x56067c183450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fca16e42740 1 -- 172.21.15.63:0/1551571640 >> 172.21.15.63:0/1551571640 conn(0x56067bdb6300 msgr2=0x56067c169e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.199+0000 7fc9eea00640 0 client.22551 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.200+0000 7fca16e42740 1 -- 172.21.15.63:0/1551571640 shutdown_connections 2024-05-27T08:10:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.200+0000 7fca16e42740 1 -- 172.21.15.63:0/1551571640 wait complete. 2024-05-27T08:10:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.200+0000 7fca16e42740 20 client.22551 trim_cache size 0 max 16384 2024-05-27T08:10:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:10:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:10:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:10:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 -- 172.21.15.63:0/1414140293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9f00b2950 msgr2=0x7fc9f00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9f00b2950 0x7fc9f00b4e60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0df540 tx=0x7fc9fc009f00 comp rx=0 tx=0).stop 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 -- 172.21.15.63:0/1414140293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca08102ab0 msgr2=0x7fca0819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca08102ab0 0x7fca0819c790 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00d740 tx=0x7fc9fc00dc00 comp rx=0 tx=0).stop 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 -- 172.21.15.63:0/1414140293 shutdown_connections 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9f00b2950 0x7fc9f00b4e60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca0810d340 0x7fca081a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca081089e0 0x7fca0819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 --2- 172.21.15.63:0/1414140293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca08102ab0 0x7fca0819c790 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 -- 172.21.15.63:0/1414140293 >> 172.21.15.63:0/1414140293 conn(0x7fca080fe210 msgr2=0x7fca0806a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 -- 172.21.15.63:0/1414140293 shutdown_connections 2024-05-27T08:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:42.201+0000 7fca0e800640 1 -- 172.21.15.63:0/1414140293 wait complete. 2024-05-27T08:10:42.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:41 smithi086 ceph-mon[34687]: pgmap v1177: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.4 MiB/s wr, 44 op/s 2024-05-27T08:10:42.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:41 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:10:42.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:41 smithi149 ceph-mon[35408]: pgmap v1177: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.4 MiB/s wr, 44 op/s 2024-05-27T08:10:42.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:41 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:10:42.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:41 smithi063 ceph-mon[31613]: pgmap v1177: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.4 MiB/s wr, 44 op/s 2024-05-27T08:10:42.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:41 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:10:42.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:42 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:10:42.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:42 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:10:43.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:42 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:10:43.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:42 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:10:43.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:42 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:10:43.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:42 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:10:44.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:43 smithi086 ceph-mon[34687]: pgmap v1178: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.0 MiB/s wr, 47 op/s 2024-05-27T08:10:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:43 smithi149 ceph-mon[35408]: pgmap v1178: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.0 MiB/s wr, 47 op/s 2024-05-27T08:10:44.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:43 smithi063 ceph-mon[31613]: pgmap v1178: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.0 MiB/s wr, 47 op/s 2024-05-27T08:10:45.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:44 smithi086 ceph-mon[34687]: pgmap v1179: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.0 MiB/s wr, 40 op/s 2024-05-27T08:10:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:44 smithi149 ceph-mon[35408]: pgmap v1179: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.0 MiB/s wr, 40 op/s 2024-05-27T08:10:45.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:44 smithi063 ceph-mon[31613]: pgmap v1179: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.0 MiB/s wr, 40 op/s 2024-05-27T08:10:47.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:47 smithi086 ceph-mon[34687]: pgmap v1180: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 47 op/s 2024-05-27T08:10:47.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:47 smithi149 ceph-mon[35408]: pgmap v1180: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 47 op/s 2024-05-27T08:10:47.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:47 smithi063 ceph-mon[31613]: pgmap v1180: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.7 MiB/s wr, 47 op/s 2024-05-27T08:10:49.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:49 smithi063 ceph-mon[31613]: pgmap v1181: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:10:50.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:49 smithi086 ceph-mon[34687]: pgmap v1181: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:10:50.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:49 smithi149 ceph-mon[35408]: pgmap v1181: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:10:52.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:51 smithi086 ceph-mon[34687]: pgmap v1182: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:10:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:51 smithi149 ceph-mon[35408]: pgmap v1182: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:10:52.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:51 smithi063 ceph-mon[31613]: pgmap v1182: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:10:54.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:53 smithi086 ceph-mon[34687]: pgmap v1183: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:10:54.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:53 smithi149 ceph-mon[35408]: pgmap v1183: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:10:54.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:53 smithi063 ceph-mon[31613]: pgmap v1183: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.0 MiB/s wr, 14 op/s 2024-05-27T08:10:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.446+0000 7fa3eae00640 1 -- 172.21.15.63:0/2485917664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e410e200 msgr2=0x7fa3e41106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.446+0000 7fa3eae00640 1 --2- 172.21.15.63:0/2485917664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e410e200 0x7fa3e41106f0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009ca0 tx=0x7fa3dc02a9e0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.446+0000 7fa3eae00640 1 -- 172.21.15.63:0/2485917664 shutdown_connections 2024-05-27T08:10:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.446+0000 7fa3eae00640 1 --2- 172.21.15.63:0/2485917664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e410e200 0x7fa3e41106f0 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.446+0000 7fa3eae00640 1 --2- 172.21.15.63:0/2485917664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e410b850 0x7fa3e410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.446+0000 7fa3eae00640 1 --2- 172.21.15.63:0/2485917664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e4104df0 0x7fa3e4105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.446+0000 7fa3eae00640 1 -- 172.21.15.63:0/2485917664 >> 172.21.15.63:0/2485917664 conn(0x7fa3e40782d0 msgr2=0x7fa3e4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.446+0000 7fa3eae00640 1 -- 172.21.15.63:0/2485917664 shutdown_connections 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.446+0000 7fa3eae00640 1 -- 172.21.15.63:0/2485917664 wait complete. 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.447+0000 7fa3eae00640 1 Processor -- start 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.447+0000 7fa3eae00640 1 -- start start 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.448+0000 7fa3eae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e4104df0 0x7fa3e419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.448+0000 7fa3eae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e410b850 0x7fa3e419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.448+0000 7fa3eae00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e410e200 0x7fa3e419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.448+0000 7fa3eae00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e4113d10 con 0x7fa3e410e200 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.448+0000 7fa3eae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e4113b90 con 0x7fa3e410b850 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.448+0000 7fa3eae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e4113e90 con 0x7fa3e4104df0 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.448+0000 7fa3e8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e410e200 0x7fa3e419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.448+0000 7fa3e8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e410e200 0x7fa3e419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46572/0 (socket says 172.21.15.63:46572) 2024-05-27T08:10:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.448+0000 7fa3e8e00640 1 -- 172.21.15.63:0/2555996001 learned_addr learned my addr 172.21.15.63:0/2555996001 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.449+0000 7fa3e3400640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e410b850 0x7fa3e419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.449+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e4104df0 0x7fa3e419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.449+0000 7fa3e8e00640 1 -- 172.21.15.63:0/2555996001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e4104df0 msgr2=0x7fa3e419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.449+0000 7fa3e8e00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e4104df0 0x7fa3e419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.449+0000 7fa3e8e00640 1 -- 172.21.15.63:0/2555996001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e410b850 msgr2=0x7fa3e419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.449+0000 7fa3e8e00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e410b850 0x7fa3e419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.449+0000 7fa3e8e00640 1 -- 172.21.15.63:0/2555996001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e419f5d0 con 0x7fa3e410e200 2024-05-27T08:10:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.449+0000 7fa3e3400640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e410b850 0x7fa3e419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.449+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e4104df0 0x7fa3e419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:10:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.449+0000 7fa3e8e00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e410e200 0x7fa3e419ef90 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc02af50 tx=0x7fa3dc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.450+0000 7fa3e0c00640 1 -- 172.21.15.63:0/2555996001 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc046070 con 0x7fa3e410e200 2024-05-27T08:10:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.450+0000 7fa3e0c00640 1 -- 172.21.15.63:0/2555996001 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3dc02ee40 con 0x7fa3e410e200 2024-05-27T08:10:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.450+0000 7fa3eae00640 1 -- 172.21.15.63:0/2555996001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e419f860 con 0x7fa3e410e200 2024-05-27T08:10:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.450+0000 7fa3e0c00640 1 -- 172.21.15.63:0/2555996001 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc033070 con 0x7fa3e410e200 2024-05-27T08:10:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.450+0000 7fa3eae00640 1 -- 172.21.15.63:0/2555996001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e419fd50 con 0x7fa3e410e200 2024-05-27T08:10:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.454+0000 7fa3f1e6f740 1 Processor -- start 2024-05-27T08:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.454+0000 7fa3e0c00640 1 -- 172.21.15.63:0/2555996001 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa3dc049070 con 0x7fa3e410e200 2024-05-27T08:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.455+0000 7fa3f1e6f740 1 -- start start 2024-05-27T08:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.455+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a47967750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.455+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a4759c2f0 0x562a47967c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.455+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 0x562a479681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.455+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562a478d8b90 con 0x562a475c51b0 2024-05-27T08:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.455+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562a478c8730 con 0x562a4759c2f0 2024-05-27T08:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.455+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562a478cc770 con 0x562a475a44d0 2024-05-27T08:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a47967750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a47967750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46584/0 (socket says 172.21.15.63:46584) 2024-05-27T08:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e3400640 1 -- 172.21.15.63:0/4171592334 learned_addr learned my addr 172.21.15.63:0/4171592334 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/4171592334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a4759c2f0 0x562a47967c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e3400640 1 --2- 172.21.15.63:0/4171592334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 0x562a479681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e3400640 1 -- 172.21.15.63:0/4171592334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 msgr2=0x562a479681d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e3400640 1 --2- 172.21.15.63:0/4171592334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 0x562a479681d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e3400640 1 -- 172.21.15.63:0/4171592334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a4759c2f0 msgr2=0x562a47967c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e3400640 1 --2- 172.21.15.63:0/4171592334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a4759c2f0 0x562a47967c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e3400640 1 -- 172.21.15.63:0/4171592334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a478618b0 con 0x562a475c51b0 2024-05-27T08:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.456+0000 7fa3e0c00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3c80b29a0 0x7fa3c80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.457+0000 7fa3e0c00640 1 -- 172.21.15.63:0/2555996001 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7fa3dc1035a0 con 0x7fa3e410e200 2024-05-27T08:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.457+0000 7fa3e3400640 1 --2- 172.21.15.63:0/4171592334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a47967750 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0013550 tx=0x7fa3d000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26f7f1982488446 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.457+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3c80b29a0 0x7fa3c80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.457+0000 7fa3d9600640 1 -- 172.21.15.63:0/4171592334 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0019030 con 0x562a475c51b0 2024-05-27T08:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.457+0000 7fa3d9600640 1 -- 172.21.15.63:0/4171592334 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3d00054b0 con 0x562a475c51b0 2024-05-27T08:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.457+0000 7fa3d9600640 1 -- 172.21.15.63:0/4171592334 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00114c0 con 0x562a475c51b0 2024-05-27T08:10:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.458+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/4171592334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 msgr2=0x562a47967750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.458+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3c80b29a0 0x7fa3c80b4eb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x562a47965350 tx=0x7fa3d4005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.458+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/4171592334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a47967750 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0013550 tx=0x7fa3d000b6b0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.458+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/4171592334 shutdown_connections 2024-05-27T08:10:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.458+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/4171592334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 0x562a479681d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.458+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/4171592334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a4759c2f0 0x562a47967c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.458+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/4171592334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a47967750 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.458+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/4171592334 >> 172.21.15.63:0/4171592334 conn(0x562a475b9300 msgr2=0x562a47964010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.459+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/4171592334 shutdown_connections 2024-05-27T08:10:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.459+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/4171592334 wait complete. 2024-05-27T08:10:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.459+0000 7fa3f1e6f740 1 -- start start 2024-05-27T08:10:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.459+0000 7fa3f1e6f740 1 Processor -- start 2024-05-27T08:10:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.460+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.460+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a475a44d0 0x562a4797a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.460+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475c51b0 0x562a479864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.460+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562a475ae790 con 0x562a4759c2f0 2024-05-27T08:10:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.460+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562a47870bc0 con 0x562a475a44d0 2024-05-27T08:10:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.460+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562a4797d1a0 con 0x562a475c51b0 2024-05-27T08:10:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.460+0000 7fa3f1e6f740 10 client.0 _ll_register_callbacks cb 0x562a4784e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.460+0000 7fa3e3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3e3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46592/0 (socket says 172.21.15.63:46592) 2024-05-27T08:10:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3e3e00640 1 -- 172.21.15.63:0/3336240600 learned_addr learned my addr 172.21.15.63:0/3336240600 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3e3400640 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a475a44d0 0x562a4797a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3e8e00640 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475c51b0 0x562a479864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3e3e00640 1 -- 172.21.15.63:0/3336240600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475c51b0 msgr2=0x562a479864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475c51b0 0x562a479864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3e3e00640 1 -- 172.21.15.63:0/3336240600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a475a44d0 msgr2=0x562a4797a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a475a44d0 0x562a4797a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3e3e00640 1 -- 172.21.15.63:0/3336240600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a475dd210 con 0x562a4759c2f0 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797a280 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40095a0 tx=0x7fa3d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7029666e9b63e2bc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.461+0000 7fa3ce000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.462+0000 7fa3ce000640 1 -- 172.21.15.63:0/3336240600 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d401d620 con 0x562a4759c2f0 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.462+0000 7fa3ce000640 1 -- 172.21.15.63:0/3336240600 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7fa3d401d7c0 con 0x562a4759c2f0 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.462+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3336240600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d00054b0 con 0x562a4759c2f0 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.462+0000 7fa3ce000640 1 -- 172.21.15.63:0/3336240600 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa3d4034050 con 0x562a4759c2f0 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.462+0000 7fa3f1e6f740 10 client.22563 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.462+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3336240600 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562a476a9620 con 0x562a4759c2f0 2024-05-27T08:10:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.463+0000 7fa3ce000640 1 -- 172.21.15.63:0/3336240600 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3d00054b0 con 0x562a4759c2f0 2024-05-27T08:10:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.463+0000 7fa3ce000640 1 -- 172.21.15.63:0/3336240600 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa3d4011a90 con 0x562a4759c2f0 2024-05-27T08:10:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.463+0000 7fa3f1e6f740 10 client.22563 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.463+0000 7fa3f1e6f740 10 client.22563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.463+0000 7fa3f1e6f740 20 client.22563 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.463+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a47583d80 0x562a4798c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.463+0000 7fa3f1e6f740 4 client.22563 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:10:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.463+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3336240600 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a478daae0 con 0x562a47583d80 2024-05-27T08:10:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.464+0000 7fa3e3400640 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a47583d80 0x562a4798c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.465+0000 7fa3e3400640 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a47583d80 0x562a4798c5f0 crc :-1 s=READY pgs=666 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa0fe58feb14de43 server_cookie=b57e78cb2e61d4af in_seq=0 out_seq=0 2024-05-27T08:10:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.465+0000 7fa3ce000640 10 client.22563 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3ce000640 1 -- 172.21.15.63:0/3336240600 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562a478daae0 con 0x562a47583d80 2024-05-27T08:10:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3ce000640 10 client.22563 handle_command_reply: tid=0 2024-05-27T08:10:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 client.22563 shutdown 2024-05-27T08:10:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3336240600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 msgr2=0x562a4797a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797a280 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40095a0 tx=0x7fa3d4004730 comp rx=0 tx=0).stop 2024-05-27T08:10:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3336240600 shutdown_connections 2024-05-27T08:10:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a47583d80 0x562a4798c5f0 crc :-1 s=READY pgs=666 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475c51b0 0x562a479864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a475a44d0 0x562a4797a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3336240600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797a280 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3336240600 >> 172.21.15.63:0/3336240600 conn(0x562a475b9300 msgr2=0x562a47969510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3ce000640 0 client.22563 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3336240600 shutdown_connections 2024-05-27T08:10:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.467+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3336240600 wait complete. 2024-05-27T08:10:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.468+0000 7fa3f1e6f740 20 client.22563 trim_cache size 0 max 16384 2024-05-27T08:10:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.490+0000 7fa3f1e6f740 1 Processor -- start 2024-05-27T08:10:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.490+0000 7fa3f1e6f740 1 -- start start 2024-05-27T08:10:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.490+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a4759c2f0 0x562a47973ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.490+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 0x562a479743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.490+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a4797a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.490+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41b18e0 con 0x562a475c51b0 2024-05-27T08:10:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.490+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41af1b0 con 0x562a4759c2f0 2024-05-27T08:10:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.490+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41ad020 con 0x562a475a44d0 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a4797a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e8e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a4797a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46602/0 (socket says 172.21.15.63:46602) 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e8e00640 1 -- 172.21.15.63:0/844961419 learned_addr learned my addr 172.21.15.63:0/844961419 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/844961419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a4759c2f0 0x562a47973ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e3400640 1 --2- 172.21.15.63:0/844961419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 0x562a479743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e8e00640 1 -- 172.21.15.63:0/844961419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 msgr2=0x562a479743e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e8e00640 1 --2- 172.21.15.63:0/844961419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 0x562a479743e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e8e00640 1 -- 172.21.15.63:0/844961419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a4759c2f0 msgr2=0x562a47973ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e8e00640 1 --2- 172.21.15.63:0/844961419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a4759c2f0 0x562a47973ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e8e00640 1 -- 172.21.15.63:0/844961419 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a47968c30 con 0x562a475c51b0 2024-05-27T08:10:55.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.491+0000 7fa3e3400640 1 --2- 172.21.15.63:0/844961419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 0x562a479743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:10:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.492+0000 7fa3e8e00640 1 --2- 172.21.15.63:0/844961419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a4797a220 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0094e0 tx=0x7fa3dc0c9ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617d1584b02cb20b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.492+0000 7fa3d8c00640 1 -- 172.21.15.63:0/844961419 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0e4070 con 0x562a475c51b0 2024-05-27T08:10:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.492+0000 7fa3d8c00640 1 -- 172.21.15.63:0/844961419 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3dc0ca4c0 con 0x562a475c51b0 2024-05-27T08:10:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.492+0000 7fa3d8c00640 1 -- 172.21.15.63:0/844961419 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0d30f0 con 0x562a475c51b0 2024-05-27T08:10:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.493+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/844961419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 msgr2=0x562a4797a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.493+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/844961419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a4797a220 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0094e0 tx=0x7fa3dc0c9ee0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.493+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/844961419 shutdown_connections 2024-05-27T08:10:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.493+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/844961419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a475c51b0 0x562a4797a220 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.493+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/844961419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475a44d0 0x562a479743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.493+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/844961419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a4759c2f0 0x562a47973ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.493+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/844961419 >> 172.21.15.63:0/844961419 conn(0x562a475b9300 msgr2=0x562a47979b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.493+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/844961419 shutdown_connections 2024-05-27T08:10:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.493+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/844961419 wait complete. 2024-05-27T08:10:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.494+0000 7fa3f1e6f740 1 -- start start 2024-05-27T08:10:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.494+0000 7fa3f1e6f740 1 Processor -- start 2024-05-27T08:10:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a475a44d0 0x562a4797df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3f1e6f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475c51b0 0x562a479f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41ad670 con 0x562a4759c2f0 2024-05-27T08:10:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41aeae0 con 0x562a475a44d0 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3f1e6f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e41add40 con 0x562a475c51b0 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3f1e6f740 10 client.0 _ll_register_callbacks cb 0x7fa3d4009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3e3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3e3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46606/0 (socket says 172.21.15.63:46606) 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3e3e00640 1 -- 172.21.15.63:0/3580699117 learned_addr learned my addr 172.21.15.63:0/3580699117 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3e8e00640 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475c51b0 0x562a479f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.495+0000 7fa3e3400640 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a475a44d0 0x562a4797df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.496+0000 7fa3e3e00640 1 -- 172.21.15.63:0/3580699117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475c51b0 msgr2=0x562a479f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.496+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475c51b0 0x562a479f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.496+0000 7fa3e3e00640 1 -- 172.21.15.63:0/3580699117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a475a44d0 msgr2=0x562a4797df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.496+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a475a44d0 0x562a4797df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.496+0000 7fa3e3e00640 1 -- 172.21.15.63:0/3580699117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a479f31e0 con 0x562a4759c2f0 2024-05-27T08:10:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.496+0000 7fa3e3e00640 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797da10 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4013010 tx=0x7fa3d4012e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e720c45c946bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:10:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.496+0000 7fa3ce000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:10:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.496+0000 7fa3ce000640 1 -- 172.21.15.63:0/3580699117 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d403e070 con 0x562a4759c2f0 2024-05-27T08:10:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.496+0000 7fa3ce000640 1 -- 172.21.15.63:0/3580699117 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 9483+0+0 (secure 0 0 0) 0x7fa3d4029d10 con 0x562a4759c2f0 2024-05-27T08:10:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.496+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3580699117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3dc0ca4c0 con 0x562a4759c2f0 2024-05-27T08:10:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.497+0000 7fa3ce000640 1 -- 172.21.15.63:0/3580699117 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa3d4049050 con 0x562a4759c2f0 2024-05-27T08:10:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.497+0000 7fa3f1e6f740 10 client.22569 fetch_fsmap learned FSMap version 35 2024-05-27T08:10:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.497+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3580699117 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562a479f43c0 con 0x562a4759c2f0 2024-05-27T08:10:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.497+0000 7fa3ce000640 1 -- 172.21.15.63:0/3580699117 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3dc0ca4c0 con 0x562a4759c2f0 2024-05-27T08:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.497+0000 7fa3ce000640 1 -- 172.21.15.63:0/3580699117 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa3d4011920 con 0x562a4759c2f0 2024-05-27T08:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.497+0000 7fa3f1e6f740 10 client.22569 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.497+0000 7fa3f1e6f740 10 client.22569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.497+0000 7fa3f1e6f740 20 client.22569 populate_metadata read hostname 'smithi063' 2024-05-27T08:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.498+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a475591c0 0x562a479f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.498+0000 7fa3f1e6f740 4 client.22569 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.498+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3580699117 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a478d42c0 con 0x562a475591c0 2024-05-27T08:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.498+0000 7fa3e3400640 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a475591c0 0x562a479f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.499+0000 7fa3e3400640 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a475591c0 0x562a479f6b50 crc :-1 s=READY pgs=667 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08b4e4c1158ec5f server_cookie=46d79c644f5bb02b in_seq=0 out_seq=0 2024-05-27T08:10:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.499+0000 7fa3ce000640 10 client.22569 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3ce000640 1 -- 172.21.15.63:0/3580699117 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562a478d42c0 con 0x562a475591c0 2024-05-27T08:10:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3ce000640 10 client.22569 handle_command_reply: tid=0 2024-05-27T08:10:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3f1e6f740 1 client.22569 shutdown 2024-05-27T08:10:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3580699117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 msgr2=0x562a4797da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797da10 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4013010 tx=0x7fa3d4012e20 comp rx=0 tx=0).stop 2024-05-27T08:10:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3580699117 shutdown_connections 2024-05-27T08:10:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562a475591c0 0x562a479f6b50 crc :-1 s=READY pgs=667 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562a475c51b0 0x562a479f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a475a44d0 0x562a4797df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3f1e6f740 1 --2- 172.21.15.63:0/3580699117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562a4759c2f0 0x562a4797da10 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3ce000640 0 client.22569 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:10:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.537+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3580699117 >> 172.21.15.63:0/3580699117 conn(0x562a475b9300 msgr2=0x562a4797a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.538+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3580699117 shutdown_connections 2024-05-27T08:10:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.538+0000 7fa3f1e6f740 1 -- 172.21.15.63:0/3580699117 wait complete. 2024-05-27T08:10:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.538+0000 7fa3f1e6f740 20 client.22569 trim_cache size 0 max 16384 2024-05-27T08:10:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:10:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:10:55.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:10:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 -- 172.21.15.63:0/2555996001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3c80b29a0 msgr2=0x7fa3c80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3c80b29a0 0x7fa3c80b4eb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x562a47965350 tx=0x7fa3d4005f90 comp rx=0 tx=0).stop 2024-05-27T08:10:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 -- 172.21.15.63:0/2555996001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e410e200 msgr2=0x7fa3e419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:10:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e410e200 0x7fa3e419ef90 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc02af50 tx=0x7fa3dc003f10 comp rx=0 tx=0).stop 2024-05-27T08:10:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 -- 172.21.15.63:0/2555996001 shutdown_connections 2024-05-27T08:10:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3c80b29a0 0x7fa3c80b4eb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3e410e200 0x7fa3e419ef90 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e410b850 0x7fa3e419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 --2- 172.21.15.63:0/2555996001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3e4104df0 0x7fa3e419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:10:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 -- 172.21.15.63:0/2555996001 >> 172.21.15.63:0/2555996001 conn(0x7fa3e40782d0 msgr2=0x7fa3e41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:10:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 -- 172.21.15.63:0/2555996001 shutdown_connections 2024-05-27T08:10:55.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:10:55.539+0000 7fa3eae00640 1 -- 172.21.15.63:0/2555996001 wait complete. 2024-05-27T08:10:56.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:55 smithi086 ceph-mon[34687]: pgmap v1184: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.4 MiB/s wr, 10 op/s 2024-05-27T08:10:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:55 smithi149 ceph-mon[35408]: pgmap v1184: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.4 MiB/s wr, 10 op/s 2024-05-27T08:10:56.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:55 smithi063 ceph-mon[31613]: pgmap v1184: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.4 MiB/s wr, 10 op/s 2024-05-27T08:10:58.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:57 smithi086 ceph-mon[34687]: pgmap v1185: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 17 op/s 2024-05-27T08:10:58.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:57 smithi149 ceph-mon[35408]: pgmap v1185: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 17 op/s 2024-05-27T08:10:58.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:57 smithi063 ceph-mon[31613]: pgmap v1185: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 17 op/s 2024-05-27T08:11:00.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:10:59 smithi086 ceph-mon[34687]: pgmap v1186: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:11:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:10:59 smithi149 ceph-mon[35408]: pgmap v1186: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:11:00.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:10:59 smithi063 ceph-mon[31613]: pgmap v1186: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:11:02.016 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:01 smithi149 ceph-mon[35408]: pgmap v1187: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:11:02.019 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:01 smithi063 ceph-mon[31613]: pgmap v1187: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:11:02.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:01 smithi086 ceph-mon[34687]: pgmap v1187: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-05-27T08:11:04.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:03 smithi086 ceph-mon[34687]: pgmap v1188: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 30 op/s 2024-05-27T08:11:04.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:03 smithi149 ceph-mon[35408]: pgmap v1188: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 30 op/s 2024-05-27T08:11:04.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:03 smithi063 ceph-mon[31613]: pgmap v1188: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.6 MiB/s wr, 30 op/s 2024-05-27T08:11:06.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:05 smithi086 ceph-mon[34687]: pgmap v1189: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-05-27T08:11:06.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:05 smithi149 ceph-mon[35408]: pgmap v1189: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-05-27T08:11:06.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:05 smithi063 ceph-mon[31613]: pgmap v1189: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.9 MiB/s wr, 26 op/s 2024-05-27T08:11:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:07 smithi086 ceph-mon[34687]: pgmap v1190: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.7 MiB/s wr, 30 op/s 2024-05-27T08:11:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:07 smithi086 ceph-mon[34687]: osdmap e155: 12 total, 12 up, 12 in 2024-05-27T08:11:08.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:07 smithi149 ceph-mon[35408]: pgmap v1190: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.7 MiB/s wr, 30 op/s 2024-05-27T08:11:08.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:07 smithi149 ceph-mon[35408]: osdmap e155: 12 total, 12 up, 12 in 2024-05-27T08:11:08.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:07 smithi063 ceph-mon[31613]: pgmap v1190: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.7 MiB/s wr, 30 op/s 2024-05-27T08:11:08.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:07 smithi063 ceph-mon[31613]: osdmap e155: 12 total, 12 up, 12 in 2024-05-27T08:11:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.850+0000 7fafabe00640 1 -- 172.21.15.63:0/710585016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafac109f50 msgr2=0x7fafac103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.850+0000 7fafabe00640 1 --2- 172.21.15.63:0/710585016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafac109f50 0x7fafac103b90 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7faf98009900 tx=0x7faf9802a3f0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.850+0000 7fafabe00640 1 -- 172.21.15.63:0/710585016 shutdown_connections 2024-05-27T08:11:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.850+0000 7fafabe00640 1 --2- 172.21.15.63:0/710585016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafac104200 0x7fafac1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.850+0000 7fafabe00640 1 --2- 172.21.15.63:0/710585016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafac109f50 0x7fafac103b90 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.850+0000 7fafabe00640 1 --2- 172.21.15.63:0/710585016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafac109ab0 0x7fafac103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.850+0000 7fafabe00640 1 -- 172.21.15.63:0/710585016 >> 172.21.15.63:0/710585016 conn(0x7fafac079140 msgr2=0x7fafac079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.850+0000 7fafabe00640 1 -- 172.21.15.63:0/710585016 shutdown_connections 2024-05-27T08:11:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.850+0000 7fafabe00640 1 -- 172.21.15.63:0/710585016 wait complete. 2024-05-27T08:11:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.851+0000 7fafabe00640 1 Processor -- start 2024-05-27T08:11:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.851+0000 7fafabe00640 1 -- start start 2024-05-27T08:11:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.852+0000 7fafabe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafac104200 0x7fafac19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.852+0000 7fafabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafac109ab0 0x7fafac19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.852+0000 7fafabe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafac109f50 0x7fafac19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.852+0000 7fafabe00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fafac113b50 con 0x7fafac104200 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.852+0000 7fafabe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1139d0 con 0x7fafac109ab0 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.852+0000 7fafabe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fafac113cd0 con 0x7fafac109f50 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.852+0000 7fafaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafac104200 0x7fafac19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.852+0000 7fafaaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafac104200 0x7fafac19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53908/0 (socket says 172.21.15.63:53908) 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.852+0000 7fafaaa00640 1 -- 172.21.15.63:0/3066630086 learned_addr learned my addr 172.21.15.63:0/3066630086 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.853+0000 7fafaa000640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafac109ab0 0x7fafac19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.853+0000 7fafab400640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafac109f50 0x7fafac19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.853+0000 7fafaaa00640 1 -- 172.21.15.63:0/3066630086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafac109f50 msgr2=0x7fafac19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.853+0000 7fafaaa00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafac109f50 0x7fafac19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.853+0000 7fafaaa00640 1 -- 172.21.15.63:0/3066630086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafac109ab0 msgr2=0x7fafac19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.853+0000 7fafaaa00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafac109ab0 0x7fafac19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.853+0000 7fafaaa00640 1 -- 172.21.15.63:0/3066630086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac19f450 con 0x7fafac104200 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.853+0000 7fafaa000640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafac109ab0 0x7fafac19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.853+0000 7fafaaa00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafac104200 0x7fafac19a5f0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00d770 tx=0x7faf9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.854+0000 7fafb0a00640 1 -- 172.21.15.63:0/3066630086 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c017070 con 0x7fafac104200 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.854+0000 7fafabe00640 1 -- 172.21.15.63:0/3066630086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafac19f740 con 0x7fafac104200 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.854+0000 7fafb0a00640 1 -- 172.21.15.63:0/3066630086 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf9c009c30 con 0x7fafac104200 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.854+0000 7fafb0a00640 1 -- 172.21.15.63:0/3066630086 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c00f620 con 0x7fafac104200 2024-05-27T08:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.854+0000 7fafabe00640 1 -- 172.21.15.63:0/3066630086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafac1a70c0 con 0x7fafac104200 2024-05-27T08:11:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.857+0000 7fafb8db6740 1 Processor -- start 2024-05-27T08:11:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafb8db6740 1 -- start start 2024-05-27T08:11:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e261b0 0x5640c31c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2dfd2f0 0x5640c31c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e054d0 0x5640c31c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafb8db6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5640c3139b90 con 0x5640c2e261b0 2024-05-27T08:11:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafb8db6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640c3129730 con 0x5640c2dfd2f0 2024-05-27T08:11:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafb8db6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5640c312d770 con 0x5640c2e054d0 2024-05-27T08:11:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafab400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e261b0 0x5640c31c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafab400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e261b0 0x5640c31c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53920/0 (socket says 172.21.15.63:53920) 2024-05-27T08:11:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafab400640 1 -- 172.21.15.63:0/782261212 learned_addr learned my addr 172.21.15.63:0/782261212 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.858+0000 7fafb0a00640 1 -- 172.21.15.63:0/3066630086 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faf9c01a070 con 0x7fafac104200 2024-05-27T08:11:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.859+0000 7fafaa000640 1 --2- 172.21.15.63:0/782261212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e054d0 0x5640c31c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.859+0000 7fafab400640 1 -- 172.21.15.63:0/782261212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e054d0 msgr2=0x5640c31c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.859+0000 7fafab400640 1 --2- 172.21.15.63:0/782261212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e054d0 0x5640c31c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.859+0000 7fafab400640 1 -- 172.21.15.63:0/782261212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2dfd2f0 msgr2=0x5640c31c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.859+0000 7fafab400640 1 --2- 172.21.15.63:0/782261212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2dfd2f0 0x5640c31c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.859+0000 7fafab400640 1 -- 172.21.15.63:0/782261212 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640c30c28b0 con 0x5640c2e261b0 2024-05-27T08:11:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.860+0000 7fafab400640 1 --2- 172.21.15.63:0/782261212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e261b0 0x5640c31c8a60 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009ed0 tx=0x7fafa4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc88d441042d6c5e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.860+0000 7faf97e00640 1 -- 172.21.15.63:0/782261212 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4016070 con 0x5640c2e261b0 2024-05-27T08:11:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.860+0000 7faf97e00640 1 -- 172.21.15.63:0/782261212 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafa401baf0 con 0x5640c2e261b0 2024-05-27T08:11:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.860+0000 7faf97e00640 1 -- 172.21.15.63:0/782261212 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4012b00 con 0x5640c2e261b0 2024-05-27T08:11:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.861+0000 7fafb0a00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf900b2950 0x7faf900b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.861+0000 7fafb0a00640 1 -- 172.21.15.63:0/3066630086 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 9571+0+0 (secure 0 0 0) 0x7faf9c0e2590 con 0x7fafac104200 2024-05-27T08:11:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.861+0000 7fafaaa00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf900b2950 0x7faf900b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.861+0000 7fafb8db6740 1 -- 172.21.15.63:0/782261212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e261b0 msgr2=0x5640c31c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.861+0000 7fafb8db6740 1 --2- 172.21.15.63:0/782261212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e261b0 0x5640c31c8a60 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009ed0 tx=0x7fafa4010740 comp rx=0 tx=0).stop 2024-05-27T08:11:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.861+0000 7fafb8db6740 1 -- 172.21.15.63:0/782261212 shutdown_connections 2024-05-27T08:11:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.861+0000 7fafb8db6740 1 --2- 172.21.15.63:0/782261212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e054d0 0x5640c31c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.861+0000 7fafb8db6740 1 --2- 172.21.15.63:0/782261212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2dfd2f0 0x5640c31c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.861+0000 7fafb8db6740 1 --2- 172.21.15.63:0/782261212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e261b0 0x5640c31c8a60 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.861+0000 7fafb8db6740 1 -- 172.21.15.63:0/782261212 >> 172.21.15.63:0/782261212 conn(0x5640c2e1a300 msgr2=0x5640c31c5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.862+0000 7fafb8db6740 1 -- 172.21.15.63:0/782261212 shutdown_connections 2024-05-27T08:11:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.862+0000 7fafb8db6740 1 -- 172.21.15.63:0/782261212 wait complete. 2024-05-27T08:11:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.862+0000 7fafaaa00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf900b2950 0x7faf900b4e60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x5640c31c6550 tx=0x7faf9c0a53a0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.863+0000 7fafb8db6740 1 -- start start 2024-05-27T08:11:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.863+0000 7fafb8db6740 1 Processor -- start 2024-05-27T08:11:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2e054d0 0x5640c31d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31deec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafb8db6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5640c2e0f790 con 0x5640c2dfd2f0 2024-05-27T08:11:08.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafb8db6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640c30d1bc0 con 0x5640c2e054d0 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafb8db6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5640c31d5b90 con 0x5640c2e261b0 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafb8db6740 10 client.0 _ll_register_callbacks cb 0x5640c30af400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafab400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafab400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31d2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53930/0 (socket says 172.21.15.63:53930) 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafab400640 1 -- 172.21.15.63:0/3014915733 learned_addr learned my addr 172.21.15.63:0/3014915733 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafaa000640 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2e054d0 0x5640c31d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.864+0000 7fafab400640 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31deec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.865+0000 7fafab400640 1 -- 172.21.15.63:0/3014915733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 msgr2=0x5640c31deec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.865+0000 7fafab400640 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31deec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.865+0000 7fafab400640 1 -- 172.21.15.63:0/3014915733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2e054d0 msgr2=0x5640c31d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.865+0000 7fafab400640 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2e054d0 0x5640c31d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.865+0000 7fafab400640 1 -- 172.21.15.63:0/3014915733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640c2e3e210 con 0x5640c2dfd2f0 2024-05-27T08:11:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.865+0000 7fafab400640 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31d2bc0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009ea0 tx=0x7fafa4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8e9ef9109777df server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.865+0000 7faf94c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.865+0000 7faf94c00640 1 -- 172.21.15.63:0/3014915733 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4016040 con 0x5640c2dfd2f0 2024-05-27T08:11:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.866+0000 7fafb8db6740 1 -- 172.21.15.63:0/3014915733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa401baf0 con 0x5640c2dfd2f0 2024-05-27T08:11:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.866+0000 7faf94c00640 1 -- 172.21.15.63:0/3014915733 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(155..155 src has 1..155) ==== 9571+0+0 (secure 0 0 0) 0x7fafa401b730 con 0x5640c2dfd2f0 2024-05-27T08:11:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.866+0000 7faf94c00640 1 -- 172.21.15.63:0/3014915733 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fafa401b930 con 0x5640c2dfd2f0 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.866+0000 7fafb8db6740 10 client.22581 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.866+0000 7fafb8db6740 1 -- 172.21.15.63:0/3014915733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5640c2f0a620 con 0x5640c2dfd2f0 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.866+0000 7faf94c00640 1 -- 172.21.15.63:0/3014915733 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafa401baf0 con 0x5640c2dfd2f0 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.867+0000 7faf94c00640 1 -- 172.21.15.63:0/3014915733 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fafa400f040 con 0x5640c2dfd2f0 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.867+0000 7fafb8db6740 10 client.22581 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.867+0000 7fafb8db6740 10 client.22581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.867+0000 7fafb8db6740 20 client.22581 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.867+0000 7fafb8db6740 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5640c2de4d80 0x5640c31e4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.867+0000 7fafb8db6740 4 client.22581 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.867+0000 7fafb8db6740 1 -- 172.21.15.63:0/3014915733 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640c313bae0 con 0x5640c2de4d80 2024-05-27T08:11:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.867+0000 7fafaa000640 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5640c2de4d80 0x5640c31e4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.869+0000 7fafaa000640 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5640c2de4d80 0x5640c31e4cb0 crc :-1 s=READY pgs=668 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8127b1bea5702c97 server_cookie=ec4977a6e57acfc8 in_seq=0 out_seq=0 2024-05-27T08:11:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.869+0000 7faf94c00640 10 client.22581 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.870+0000 7faf94c00640 1 -- 172.21.15.63:0/3014915733 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5640c313bae0 con 0x5640c2de4d80 2024-05-27T08:11:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.870+0000 7faf94c00640 10 client.22581 handle_command_reply: tid=0 2024-05-27T08:11:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 client.22581 shutdown 2024-05-27T08:11:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 -- 172.21.15.63:0/3014915733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 msgr2=0x5640c31d2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31d2bc0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009ea0 tx=0x7fafa4005bd0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 -- 172.21.15.63:0/3014915733 shutdown_connections 2024-05-27T08:11:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5640c2de4d80 0x5640c31e4cb0 crc :-1 s=READY pgs=668 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31deec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2e054d0 0x5640c31d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 --2- 172.21.15.63:0/3014915733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31d2bc0 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7faf94c00640 0 client.22581 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 -- 172.21.15.63:0/3014915733 >> 172.21.15.63:0/3014915733 conn(0x5640c2e1a300 msgr2=0x5640c31c8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 -- 172.21.15.63:0/3014915733 shutdown_connections 2024-05-27T08:11:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 1 -- 172.21.15.63:0/3014915733 wait complete. 2024-05-27T08:11:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.871+0000 7fafb8db6740 20 client.22581 trim_cache size 0 max 16384 2024-05-27T08:11:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.897+0000 7fafb8db6740 1 Processor -- start 2024-05-27T08:11:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.897+0000 7fafb8db6740 1 -- start start 2024-05-27T08:11:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.897+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.897+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2e054d0 0x5640c31d5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.897+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31dec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.897+0000 7fafb8db6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1b3af0 con 0x5640c2dfd2f0 2024-05-27T08:11:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.897+0000 7fafb8db6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1b13c0 con 0x5640c2e054d0 2024-05-27T08:11:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.897+0000 7fafb8db6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1af230 con 0x5640c2e261b0 2024-05-27T08:11:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.898+0000 7fafab400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.898+0000 7fafab400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31c4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41092/0 (socket says 172.21.15.63:41092) 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.898+0000 7fafab400640 1 -- 172.21.15.63:0/1625808130 learned_addr learned my addr 172.21.15.63:0/1625808130 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.898+0000 7fafaa000640 1 --2- 172.21.15.63:0/1625808130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2e054d0 0x5640c31d5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.898+0000 7fafab400640 1 --2- 172.21.15.63:0/1625808130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.898+0000 7fafab400640 1 -- 172.21.15.63:0/1625808130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 msgr2=0x5640c31dec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.898+0000 7fafab400640 1 --2- 172.21.15.63:0/1625808130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31dec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.898+0000 7fafab400640 1 -- 172.21.15.63:0/1625808130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2e054d0 msgr2=0x5640c31d5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.898+0000 7fafab400640 1 --2- 172.21.15.63:0/1625808130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2e054d0 0x5640c31d5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.898+0000 7fafab400640 1 -- 172.21.15.63:0/1625808130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640c31d1110 con 0x5640c2dfd2f0 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.899+0000 7fafab400640 1 --2- 172.21.15.63:0/1625808130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31c4240 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fafa4000f80 tx=0x7fafa40097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88831c0e8d7e572 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.899+0000 7faf97400640 1 -- 172.21.15.63:0/1625808130 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4010c40 con 0x5640c2dfd2f0 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.899+0000 7faf97400640 1 -- 172.21.15.63:0/1625808130 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafa401bad0 con 0x5640c2dfd2f0 2024-05-27T08:11:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.899+0000 7faf97400640 1 -- 172.21.15.63:0/1625808130 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4012910 con 0x5640c2dfd2f0 2024-05-27T08:11:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.900+0000 7fafb8db6740 1 -- 172.21.15.63:0/1625808130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 msgr2=0x5640c31c4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.900+0000 7fafb8db6740 1 --2- 172.21.15.63:0/1625808130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31c4240 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fafa4000f80 tx=0x7fafa40097a0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.900+0000 7fafb8db6740 1 -- 172.21.15.63:0/1625808130 shutdown_connections 2024-05-27T08:11:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.900+0000 7fafb8db6740 1 --2- 172.21.15.63:0/1625808130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31dec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.900+0000 7fafb8db6740 1 --2- 172.21.15.63:0/1625808130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2e054d0 0x5640c31d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.900+0000 7fafb8db6740 1 --2- 172.21.15.63:0/1625808130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2dfd2f0 0x5640c31c4240 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.900+0000 7fafb8db6740 1 -- 172.21.15.63:0/1625808130 >> 172.21.15.63:0/1625808130 conn(0x5640c2e1a300 msgr2=0x5640c31d2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.900+0000 7fafb8db6740 1 -- 172.21.15.63:0/1625808130 shutdown_connections 2024-05-27T08:11:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.901+0000 7fafb8db6740 1 -- 172.21.15.63:0/1625808130 wait complete. 2024-05-27T08:11:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.901+0000 7fafb8db6740 1 -- start start 2024-05-27T08:11:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.901+0000 7fafb8db6740 1 Processor -- start 2024-05-27T08:11:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.902+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2dfd2f0 0x5640c31d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.902+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e054d0 0x5640c31d6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.902+0000 7fafb8db6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31d6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.902+0000 7fafb8db6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1af880 con 0x5640c2e054d0 2024-05-27T08:11:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.902+0000 7fafb8db6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1b0cf0 con 0x5640c2dfd2f0 2024-05-27T08:11:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.902+0000 7fafb8db6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1aff50 con 0x5640c2e261b0 2024-05-27T08:11:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.902+0000 7fafb8db6740 10 client.0 _ll_register_callbacks cb 0x7faf98009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.902+0000 7fafaa000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e054d0 0x5640c31d6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.902+0000 7fafaa000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e054d0 0x5640c31d6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41094/0 (socket says 172.21.15.63:41094) 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.902+0000 7fafaa000640 1 -- 172.21.15.63:0/2400731791 learned_addr learned my addr 172.21.15.63:0/2400731791 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.903+0000 7fafab400640 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.903+0000 7fafab400640 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2dfd2f0 0x5640c31d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.903+0000 7fafaa000640 1 -- 172.21.15.63:0/2400731791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 msgr2=0x5640c31d6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.903+0000 7fafaa000640 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31d6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.903+0000 7fafaa000640 1 -- 172.21.15.63:0/2400731791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2dfd2f0 msgr2=0x5640c31d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.903+0000 7fafaa000640 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2dfd2f0 0x5640c31d5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.903+0000 7fafaa000640 1 -- 172.21.15.63:0/2400731791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640c324b2f0 con 0x5640c2e054d0 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.903+0000 7fafab400640 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2dfd2f0 0x5640c31d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.903+0000 7fafaa000640 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e054d0 0x5640c31d6340 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7faf98009850 tx=0x7faf980043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e8303e25cf0aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.903+0000 7faf94c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.904+0000 7faf94c00640 1 -- 172.21.15.63:0/2400731791 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98049070 con 0x5640c2e054d0 2024-05-27T08:11:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.904+0000 7faf94c00640 1 -- 172.21.15.63:0/2400731791 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(155..155 src has 1..155) ==== 9571+0+0 (secure 0 0 0) 0x7faf98005450 con 0x5640c2e054d0 2024-05-27T08:11:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.904+0000 7fafb8db6740 1 -- 172.21.15.63:0/2400731791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa401bad0 con 0x5640c2e054d0 2024-05-27T08:11:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.904+0000 7faf94c00640 1 -- 172.21.15.63:0/2400731791 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faf98055050 con 0x5640c2e054d0 2024-05-27T08:11:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.904+0000 7fafb8db6740 10 client.22587 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.904+0000 7fafb8db6740 1 -- 172.21.15.63:0/2400731791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5640c324c580 con 0x5640c2e054d0 2024-05-27T08:11:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.904+0000 7faf94c00640 1 -- 172.21.15.63:0/2400731791 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fafa401bad0 con 0x5640c2e054d0 2024-05-27T08:11:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.905+0000 7faf94c00640 1 -- 172.21.15.63:0/2400731791 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7faf98003680 con 0x5640c2e054d0 2024-05-27T08:11:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.905+0000 7fafb8db6740 10 client.22587 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.905+0000 7fafb8db6740 10 client.22587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.905+0000 7fafb8db6740 20 client.22587 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.905+0000 7fafb8db6740 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5640c2dba1c0 0x5640c324ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.905+0000 7fafb8db6740 4 client.22587 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:11:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.905+0000 7fafb8db6740 1 -- 172.21.15.63:0/2400731791 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5640c31352c0 con 0x5640c2dba1c0 2024-05-27T08:11:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.905+0000 7fafab400640 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5640c2dba1c0 0x5640c324ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.906+0000 7fafab400640 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5640c2dba1c0 0x5640c324ec90 crc :-1 s=READY pgs=669 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=917ce14c7565a664 server_cookie=15880116d1df6d63 in_seq=0 out_seq=0 2024-05-27T08:11:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.906+0000 7faf94c00640 10 client.22587 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.947+0000 7faf94c00640 1 -- 172.21.15.63:0/2400731791 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5640c31352c0 con 0x5640c2dba1c0 2024-05-27T08:11:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.947+0000 7faf94c00640 10 client.22587 handle_command_reply: tid=0 2024-05-27T08:11:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.947+0000 7fafb8db6740 1 client.22587 shutdown 2024-05-27T08:11:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.947+0000 7fafb8db6740 1 -- 172.21.15.63:0/2400731791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e054d0 msgr2=0x5640c31d6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.947+0000 7fafb8db6740 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e054d0 0x5640c31d6340 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7faf98009850 tx=0x7faf980043d0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.947+0000 7fafb8db6740 1 -- 172.21.15.63:0/2400731791 shutdown_connections 2024-05-27T08:11:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.948+0000 7fafb8db6740 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5640c2dba1c0 0x5640c324ec90 crc :-1 s=READY pgs=669 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.948+0000 7fafb8db6740 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5640c2e261b0 0x5640c31d6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.948+0000 7fafb8db6740 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5640c2e054d0 0x5640c31d6340 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.948+0000 7fafb8db6740 1 --2- 172.21.15.63:0/2400731791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640c2dfd2f0 0x5640c31d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.948+0000 7fafb8db6740 1 -- 172.21.15.63:0/2400731791 >> 172.21.15.63:0/2400731791 conn(0x5640c2e1a300 msgr2=0x5640c31d2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.948+0000 7faf94c00640 0 client.22587 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.948+0000 7fafb8db6740 1 -- 172.21.15.63:0/2400731791 shutdown_connections 2024-05-27T08:11:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.948+0000 7fafb8db6740 1 -- 172.21.15.63:0/2400731791 wait complete. 2024-05-27T08:11:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.948+0000 7fafb8db6740 20 client.22587 trim_cache size 0 max 16384 2024-05-27T08:11:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:11:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:11:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 -- 172.21.15.63:0/3066630086 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf900b2950 msgr2=0x7faf900b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf900b2950 0x7faf900b4e60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x5640c31c6550 tx=0x7faf9c0a53a0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 -- 172.21.15.63:0/3066630086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafac104200 msgr2=0x7fafac19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafac104200 0x7fafac19a5f0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00d770 tx=0x7faf9c00dc30 comp rx=0 tx=0).stop 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 -- 172.21.15.63:0/3066630086 shutdown_connections 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faf900b2950 0x7faf900b4e60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafac109f50 0x7fafac19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafac109ab0 0x7fafac19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 --2- 172.21.15.63:0/3066630086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafac104200 0x7fafac19a5f0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 -- 172.21.15.63:0/3066630086 >> 172.21.15.63:0/3066630086 conn(0x7fafac079140 msgr2=0x7fafac102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 -- 172.21.15.63:0/3066630086 shutdown_connections 2024-05-27T08:11:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:08.949+0000 7fafabe00640 1 -- 172.21.15.63:0/3066630086 wait complete. 2024-05-27T08:11:10.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:09 smithi086 ceph-mon[34687]: pgmap v1192: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.5 MiB/s wr, 28 op/s 2024-05-27T08:11:10.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:09 smithi149 ceph-mon[35408]: pgmap v1192: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.5 MiB/s wr, 28 op/s 2024-05-27T08:11:10.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:09 smithi063 ceph-mon[31613]: pgmap v1192: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.5 MiB/s wr, 28 op/s 2024-05-27T08:11:11.578 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:11:11.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:11 smithi063 ceph-mon[31613]: pgmap v1193: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.5 MiB/s wr, 28 op/s 2024-05-27T08:11:12.124 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:11:12.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:11 smithi086 ceph-mon[34687]: pgmap v1193: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.5 MiB/s wr, 28 op/s 2024-05-27T08:11:12.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:11 smithi149 ceph-mon[35408]: pgmap v1193: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.5 MiB/s wr, 28 op/s 2024-05-27T08:11:12.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:12 smithi063 ceph-mon[31613]: pgmap v1194: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.6 MiB/s wr, 13 op/s 2024-05-27T08:11:13.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:12 smithi086 ceph-mon[34687]: pgmap v1194: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.6 MiB/s wr, 13 op/s 2024-05-27T08:11:13.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:12 smithi149 ceph-mon[35408]: pgmap v1194: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.6 MiB/s wr, 13 op/s 2024-05-27T08:11:15.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.992+0000 7fc8c4e00700 1 -- 172.21.15.63:0/3567635915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c0106eb0 msgr2=0x7fc8c0113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:15.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.992+0000 7fc8c4e00700 1 --2- 172.21.15.63:0/3567635915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c0106eb0 0x7fc8c0113340 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800b0a0 tx=0x7fc8b800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:11:15.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.992+0000 7fc8c4e00700 1 -- 172.21.15.63:0/3567635915 shutdown_connections 2024-05-27T08:11:15.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.992+0000 7fc8c4e00700 1 --2- 172.21.15.63:0/3567635915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c0106eb0 0x7fc8c0113340 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:15.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.992+0000 7fc8c4e00700 1 --2- 172.21.15.63:0/3567635915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8c010c5a0 0x7fc8c010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:15.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.992+0000 7fc8c4e00700 1 --2- 172.21.15.63:0/3567635915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c0106560 0x7fc8c0106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:15.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.992+0000 7fc8c4e00700 1 -- 172.21.15.63:0/3567635915 >> 172.21.15.63:0/3567635915 conn(0x7fc8c0102040 msgr2=0x7fc8c0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:15.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.992+0000 7fc8c4e00700 1 -- 172.21.15.63:0/3567635915 shutdown_connections 2024-05-27T08:11:15.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.992+0000 7fc8c4e00700 1 -- 172.21.15.63:0/3567635915 wait complete. 2024-05-27T08:11:15.995 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:15 smithi063 ceph-mon[31613]: pgmap v1195: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.6 MiB/s wr, 13 op/s 2024-05-27T08:11:15.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.993+0000 7fc8c4e00700 1 Processor -- start 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.994+0000 7fc8c4e00700 1 -- start start 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.994+0000 7fc8c4e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8c0106560 0x7fc8c019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.994+0000 7fc8c4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c0106eb0 0x7fc8c019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.994+0000 7fc8c4e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c010c5a0 0x7fc8c01a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.994+0000 7fc8c4e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01165d0 con 0x7fc8c010c5a0 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.994+0000 7fc8c4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0116450 con 0x7fc8c0106eb0 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.994+0000 7fc8c4e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0116750 con 0x7fc8c0106560 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.995+0000 7fc8be800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c010c5a0 0x7fc8c01a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.995+0000 7fc8bd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c0106eb0 0x7fc8c019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.995+0000 7fc8bde00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8c0106560 0x7fc8c019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.995+0000 7fc8bd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c0106eb0 0x7fc8c019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51952/0 (socket says 172.21.15.63:51952) 2024-05-27T08:11:15.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.995+0000 7fc8bd400700 1 -- 172.21.15.63:0/4259114208 learned_addr learned my addr 172.21.15.63:0/4259114208 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:15.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.996+0000 7fc8be800700 1 -- 172.21.15.63:0/4259114208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8c0106560 msgr2=0x7fc8c019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:15.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.996+0000 7fc8be800700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8c0106560 0x7fc8c019cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:15.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.996+0000 7fc8be800700 1 -- 172.21.15.63:0/4259114208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c0106eb0 msgr2=0x7fc8c019d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:15.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.996+0000 7fc8be800700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c0106eb0 0x7fc8c019d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:15.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.996+0000 7fc8be800700 1 -- 172.21.15.63:0/4259114208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c01a1dd0 con 0x7fc8c010c5a0 2024-05-27T08:11:15.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.996+0000 7fc8be800700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c010c5a0 0x7fc8c01a1730 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800b070 tx=0x7fc8b8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:15.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.997+0000 7fc8b6a00700 1 -- 172.21.15.63:0/4259114208 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8021070 con 0x7fc8c010c5a0 2024-05-27T08:11:15.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.997+0000 7fc8b6a00700 1 -- 172.21.15.63:0/4259114208 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8b8007b80 con 0x7fc8c010c5a0 2024-05-27T08:11:15.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.997+0000 7fc8b6a00700 1 -- 172.21.15.63:0/4259114208 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8005040 con 0x7fc8c010c5a0 2024-05-27T08:11:15.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.997+0000 7fc8c4e00700 1 -- 172.21.15.63:0/4259114208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c01a2060 con 0x7fc8c010c5a0 2024-05-27T08:11:15.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:15.997+0000 7fc8c4e00700 1 -- 172.21.15.63:0/4259114208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c01a24f0 con 0x7fc8c010c5a0 2024-05-27T08:11:16.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.003+0000 7fc8b6a00700 1 -- 172.21.15.63:0/4259114208 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc8b8024070 con 0x7fc8c010c5a0 2024-05-27T08:11:16.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.005+0000 7fc8b6a00700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8a40b2a70 0x7fc8a40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.005+0000 7fc8b6a00700 1 -- 172.21.15.63:0/4259114208 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 9571+0+0 (secure 0 0 0) 0x7fc8b800f020 con 0x7fc8c010c5a0 2024-05-27T08:11:16.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.005+0000 7fc8bde00700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8a40b2a70 0x7fc8a40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.006+0000 7fc8bde00700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8a40b2a70 0x7fc8a40b4f80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc8c01079d0 tx=0x7fc8b0009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:16.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.027+0000 7fc8d3a1ab80 1 Processor -- start 2024-05-27T08:11:16.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.028+0000 7fc8d3a1ab80 1 -- start start 2024-05-27T08:11:16.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.028+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95fe99f0 0x557f9607c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.028+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95efde40 0x557f9607c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.028+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f9607ce70 0x557f96089110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.028+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557f95faae40 con 0x557f95efde40 2024-05-27T08:11:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.028+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f95f9a9e0 con 0x557f95fe99f0 2024-05-27T08:11:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.028+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557f95f9ea20 con 0x557f9607ce70 2024-05-27T08:11:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.029+0000 7fc8bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95efde40 0x557f9607c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.029+0000 7fc8bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95efde40 0x557f9607c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41114/0 (socket says 172.21.15.63:41114) 2024-05-27T08:11:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.029+0000 7fc8bd400700 1 -- 172.21.15.63:0/1642095885 learned_addr learned my addr 172.21.15.63:0/1642095885 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.029+0000 7fc8bde00700 1 --2- 172.21.15.63:0/1642095885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95fe99f0 0x557f9607c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.029+0000 7fc8be800700 1 --2- 172.21.15.63:0/1642095885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f9607ce70 0x557f96089110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.030+0000 7fc8bd400700 1 -- 172.21.15.63:0/1642095885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f9607ce70 msgr2=0x557f96089110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:16.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.030+0000 7fc8bd400700 1 --2- 172.21.15.63:0/1642095885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f9607ce70 0x557f96089110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.030+0000 7fc8bd400700 1 -- 172.21.15.63:0/1642095885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95fe99f0 msgr2=0x557f9607c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:16.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.030+0000 7fc8bd400700 1 --2- 172.21.15.63:0/1642095885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95fe99f0 0x557f9607c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.030+0000 7fc8bd400700 1 -- 172.21.15.63:0/1642095885 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f95f63420 con 0x557f95efde40 2024-05-27T08:11:16.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.030+0000 7fc8bd400700 1 --2- 172.21.15.63:0/1642095885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95efde40 0x557f9607c870 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00bda0 tx=0x7fc8ac0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cef350de55d5ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:16.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.031+0000 7fc8ab400700 1 -- 172.21.15.63:0/1642095885 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac010e10 con 0x557f95efde40 2024-05-27T08:11:16.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.031+0000 7fc8ab400700 1 -- 172.21.15.63:0/1642095885 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8ac01c910 con 0x557f95efde40 2024-05-27T08:11:16.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.031+0000 7fc8ab400700 1 -- 172.21.15.63:0/1642095885 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac011810 con 0x557f95efde40 2024-05-27T08:11:16.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.032+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/1642095885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95efde40 msgr2=0x557f9607c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:16.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.032+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/1642095885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95efde40 0x557f9607c870 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00bda0 tx=0x7fc8ac0106b0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.033+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/1642095885 shutdown_connections 2024-05-27T08:11:16.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.033+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/1642095885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f9607ce70 0x557f96089110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.033+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/1642095885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95fe99f0 0x557f9607c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.033+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/1642095885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95efde40 0x557f9607c870 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.033+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/1642095885 >> 172.21.15.63:0/1642095885 conn(0x557f95f97390 msgr2=0x557f960784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:16.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.033+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/1642095885 shutdown_connections 2024-05-27T08:11:16.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.033+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/1642095885 wait complete. 2024-05-27T08:11:16.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.034+0000 7fc8d3a1ab80 1 -- start start 2024-05-27T08:11:16.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.034+0000 7fc8d3a1ab80 1 Processor -- start 2024-05-27T08:11:16.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.035+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95efde40 0x557f960866c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.035+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95fe99f0 0x557f96086c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.035+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f9607ce70 0x557f96087140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.035+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557f95ea0fa0 con 0x557f9607ce70 2024-05-27T08:11:16.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.035+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f95f3b760 con 0x557f95efde40 2024-05-27T08:11:16.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.035+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557f96089b80 con 0x557f95fe99f0 2024-05-27T08:11:16.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.035+0000 7fc8d3a1ab80 10 client.0 _ll_register_callbacks cb 0x557f95f44c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:16.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.036+0000 7fc8be800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f9607ce70 0x557f96087140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.036+0000 7fc8be800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f9607ce70 0x557f96087140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41116/0 (socket says 172.21.15.63:41116) 2024-05-27T08:11:16.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.036+0000 7fc8be800700 1 -- 172.21.15.63:0/484971750 learned_addr learned my addr 172.21.15.63:0/484971750 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:16.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.036+0000 7fc8bde00700 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95efde40 0x557f960866c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.036+0000 7fc8bd400700 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95fe99f0 0x557f96086c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.036+0000 7fc8be800700 1 -- 172.21.15.63:0/484971750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95fe99f0 msgr2=0x557f96086c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:16.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.036+0000 7fc8be800700 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95fe99f0 0x557f96086c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.036+0000 7fc8be800700 1 -- 172.21.15.63:0/484971750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95efde40 msgr2=0x557f960866c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:16.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.036+0000 7fc8be800700 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95efde40 0x557f960866c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.036+0000 7fc8be800700 1 -- 172.21.15.63:0/484971750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f95eea8d0 con 0x557f9607ce70 2024-05-27T08:11:16.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.037+0000 7fc8be800700 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f9607ce70 0x557f96087140 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80aacb0 tx=0x7fc8b80ab280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71815b3e0116d698 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:16.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.037+0000 7fc89fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:16.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.037+0000 7fc89fe00700 1 -- 172.21.15.63:0/484971750 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b80aa830 con 0x557f9607ce70 2024-05-27T08:11:16.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.037+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/484971750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ac01c910 con 0x557f9607ce70 2024-05-27T08:11:16.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.037+0000 7fc89fe00700 1 -- 172.21.15.63:0/484971750 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(155..155 src has 1..155) ==== 9571+0+0 (secure 0 0 0) 0x7fc8b8007d30 con 0x557f9607ce70 2024-05-27T08:11:16.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.038+0000 7fc89fe00700 1 -- 172.21.15.63:0/484971750 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc8b80ab420 con 0x557f9607ce70 2024-05-27T08:11:16.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.038+0000 7fc8d3a1ab80 10 client.22599 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:16.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.038+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/484971750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557f95f6b060 con 0x557f9607ce70 2024-05-27T08:11:16.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.038+0000 7fc89fe00700 1 -- 172.21.15.63:0/484971750 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8ac01c910 con 0x557f9607ce70 2024-05-27T08:11:16.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.040+0000 7fc89fe00700 1 -- 172.21.15.63:0/484971750 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc8b8011070 con 0x557f9607ce70 2024-05-27T08:11:16.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.040+0000 7fc8d3a1ab80 10 client.22599 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:16.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.040+0000 7fc8d3a1ab80 10 client.22599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:16.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.040+0000 7fc8d3a1ab80 20 client.22599 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:16.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.040+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557f96096ad0 0x557f96098f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.040+0000 7fc8d3a1ab80 4 client.22599 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:11:16.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.040+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/484971750 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f95facd90 con 0x557f96096ad0 2024-05-27T08:11:16.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.041+0000 7fc8bde00700 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557f96096ad0 0x557f96098f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.042+0000 7fc8bde00700 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557f96096ad0 0x557f96098f40 crc :-1 s=READY pgs=670 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b2cb2fdbac15ef server_cookie=26d077f0e5a3a9c7 in_seq=0 out_seq=0 2024-05-27T08:11:16.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.042+0000 7fc89fe00700 10 client.22599 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:16.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.044+0000 7fc89fe00700 1 -- 172.21.15.63:0/484971750 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557f95facd90 con 0x557f96096ad0 2024-05-27T08:11:16.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.044+0000 7fc89fe00700 10 client.22599 handle_command_reply: tid=0 2024-05-27T08:11:16.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.044+0000 7fc8d3a1ab80 1 client.22599 shutdown 2024-05-27T08:11:16.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/484971750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f9607ce70 msgr2=0x557f96087140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:16.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f9607ce70 0x557f96087140 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80aacb0 tx=0x7fc8b80ab280 comp rx=0 tx=0).stop 2024-05-27T08:11:16.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/484971750 shutdown_connections 2024-05-27T08:11:16.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557f96096ad0 0x557f96098f40 crc :-1 s=READY pgs=670 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f9607ce70 0x557f96087140 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95efde40 0x557f960866c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/484971750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95fe99f0 0x557f96086c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/484971750 >> 172.21.15.63:0/484971750 conn(0x557f95f97390 msgr2=0x557f9607c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:16.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc89fe00700 0 client.22599 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:16.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/484971750 shutdown_connections 2024-05-27T08:11:16.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/484971750 wait complete. 2024-05-27T08:11:16.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.045+0000 7fc8d3a1ab80 20 client.22599 trim_cache size 0 max 16384 2024-05-27T08:11:16.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.072+0000 7fc8d3a1ab80 1 Processor -- start 2024-05-27T08:11:16.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.072+0000 7fc8d3a1ab80 1 -- start start 2024-05-27T08:11:16.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.072+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95efde40 0x557f96081e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.073+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f9609e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.073+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f9607ce70 0x557f9609ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.073+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b4f80 con 0x557f95fe99f0 2024-05-27T08:11:16.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.073+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b2810 con 0x557f9607ce70 2024-05-27T08:11:16.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.073+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b0680 con 0x557f95efde40 2024-05-27T08:11:16.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.073+0000 7fc8bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f9609e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.073+0000 7fc8bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f9609e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41120/0 (socket says 172.21.15.63:41120) 2024-05-27T08:11:16.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.073+0000 7fc8bd400700 1 -- 172.21.15.63:0/3990423806 learned_addr learned my addr 172.21.15.63:0/3990423806 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:16.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.073+0000 7fc8bde00700 1 --2- 172.21.15.63:0/3990423806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95efde40 0x557f96081e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.073+0000 7fc8be800700 1 --2- 172.21.15.63:0/3990423806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f9607ce70 0x557f9609ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.074+0000 7fc8bd400700 1 -- 172.21.15.63:0/3990423806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95efde40 msgr2=0x557f96081e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:16.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.074+0000 7fc8bd400700 1 --2- 172.21.15.63:0/3990423806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95efde40 0x557f96081e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.074+0000 7fc8bd400700 1 -- 172.21.15.63:0/3990423806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f9607ce70 msgr2=0x557f9609ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:16.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.074+0000 7fc8bd400700 1 --2- 172.21.15.63:0/3990423806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f9607ce70 0x557f9609ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.074+0000 7fc8bd400700 1 -- 172.21.15.63:0/3990423806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f960876d0 con 0x557f95fe99f0 2024-05-27T08:11:16.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.074+0000 7fc8bde00700 1 --2- 172.21.15.63:0/3990423806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95efde40 0x557f96081e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:16.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.074+0000 7fc8bd400700 1 --2- 172.21.15.63:0/3990423806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f9609e700 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00bfd0 tx=0x7fc8ac010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165269681fc3089c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:16.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.075+0000 7fc8aaa00700 1 -- 172.21.15.63:0/3990423806 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac005240 con 0x557f95fe99f0 2024-05-27T08:11:16.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.075+0000 7fc8aaa00700 1 -- 172.21.15.63:0/3990423806 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8ac0053e0 con 0x557f95fe99f0 2024-05-27T08:11:16.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.075+0000 7fc8aaa00700 1 -- 172.21.15.63:0/3990423806 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac01c6e0 con 0x557f95fe99f0 2024-05-27T08:11:16.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.076+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/3990423806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 msgr2=0x557f9609e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:16.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.076+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/3990423806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f9609e700 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00bfd0 tx=0x7fc8ac010da0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.076+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/3990423806 shutdown_connections 2024-05-27T08:11:16.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.076+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/3990423806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f9607ce70 0x557f9609ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.076+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/3990423806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f95efde40 0x557f96081e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.076+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/3990423806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f9609e700 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.076+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/3990423806 >> 172.21.15.63:0/3990423806 conn(0x557f95f97390 msgr2=0x557f96077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:16.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.076+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/3990423806 shutdown_connections 2024-05-27T08:11:16.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.077+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/3990423806 wait complete. 2024-05-27T08:11:16.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.077+0000 7fc8d3a1ab80 1 -- start start 2024-05-27T08:11:16.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.077+0000 7fc8d3a1ab80 1 Processor -- start 2024-05-27T08:11:16.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.078+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95efde40 0x557f9608f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.078+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f96080ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.078+0000 7fc8d3a1ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f9607ce70 0x557f96081400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.078+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b0cd0 con 0x557f95fe99f0 2024-05-27T08:11:16.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.078+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b2140 con 0x557f95efde40 2024-05-27T08:11:16.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.078+0000 7fc8d3a1ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01b13a0 con 0x557f9607ce70 2024-05-27T08:11:16.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.078+0000 7fc8d3a1ab80 10 client.0 _ll_register_callbacks cb 0x557f95f44c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:16.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.079+0000 7fc8bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f96080ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.079+0000 7fc8bd400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f96080ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41132/0 (socket says 172.21.15.63:41132) 2024-05-27T08:11:16.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.079+0000 7fc8bd400700 1 -- 172.21.15.63:0/4218164689 learned_addr learned my addr 172.21.15.63:0/4218164689 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:16.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.079+0000 7fc8bde00700 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95efde40 0x557f9608f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.079+0000 7fc8be800700 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f9607ce70 0x557f96081400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.079+0000 7fc8bd400700 1 -- 172.21.15.63:0/4218164689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f9607ce70 msgr2=0x557f96081400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:16.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.079+0000 7fc8bd400700 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f9607ce70 0x557f96081400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.079+0000 7fc8bd400700 1 -- 172.21.15.63:0/4218164689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95efde40 msgr2=0x557f9608f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:16.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.079+0000 7fc8bd400700 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95efde40 0x557f9608f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.079+0000 7fc8bd400700 1 -- 172.21.15.63:0/4218164689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f96081b30 con 0x557f95fe99f0 2024-05-27T08:11:16.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.080+0000 7fc8bd400700 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f96080ec0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0106f0 tx=0x7fc8ac010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401b122de221d201 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:16.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.080+0000 7fc89fe00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:16.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.080+0000 7fc89fe00700 1 -- 172.21.15.63:0/4218164689 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac034060 con 0x557f95fe99f0 2024-05-27T08:11:16.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.080+0000 7fc89fe00700 1 -- 172.21.15.63:0/4218164689 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(155..155 src has 1..155) ==== 9571+0+0 (secure 0 0 0) 0x7fc8ac003880 con 0x557f95fe99f0 2024-05-27T08:11:16.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.080+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/4218164689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ac0053e0 con 0x557f95fe99f0 2024-05-27T08:11:16.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.080+0000 7fc89fe00700 1 -- 172.21.15.63:0/4218164689 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc8ac03b050 con 0x557f95fe99f0 2024-05-27T08:11:16.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.080+0000 7fc8d3a1ab80 10 client.22605 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:16.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.080+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/4218164689 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557f961015d0 con 0x557f95fe99f0 2024-05-27T08:11:16.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.081+0000 7fc89fe00700 1 -- 172.21.15.63:0/4218164689 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8ac0053e0 con 0x557f95fe99f0 2024-05-27T08:11:16.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.081+0000 7fc89fe00700 1 -- 172.21.15.63:0/4218164689 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc8ac027450 con 0x557f95fe99f0 2024-05-27T08:11:16.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.081+0000 7fc8d3a1ab80 10 client.22605 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:16.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.081+0000 7fc8d3a1ab80 10 client.22605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:16.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.081+0000 7fc8d3a1ab80 20 client.22605 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:16.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.081+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557f96101a50 0x557f96103e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:16.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.081+0000 7fc8d3a1ab80 4 client.22605 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:11:16.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.081+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/4218164689 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f95fa6570 con 0x557f96101a50 2024-05-27T08:11:16.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.083+0000 7fc8bde00700 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557f96101a50 0x557f96103e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:16.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.084+0000 7fc8bde00700 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557f96101a50 0x557f96103e70 crc :-1 s=READY pgs=671 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af59ebedb3dfa4b server_cookie=d29e508664fb82e9 in_seq=0 out_seq=0 2024-05-27T08:11:16.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.084+0000 7fc89fe00700 10 client.22605 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:16.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.084+0000 7fc89fe00700 1 -- 172.21.15.63:0/4218164689 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557f95fa6570 con 0x557f96101a50 2024-05-27T08:11:16.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.084+0000 7fc89fe00700 10 client.22605 handle_command_reply: tid=0 2024-05-27T08:11:16.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.084+0000 7fc8d3a1ab80 1 client.22605 shutdown 2024-05-27T08:11:16.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.084+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/4218164689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 msgr2=0x557f96080ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:16.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.084+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f96080ec0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0106f0 tx=0x7fc8ac010f10 comp rx=0 tx=0).stop 2024-05-27T08:11:16.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.085+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/4218164689 shutdown_connections 2024-05-27T08:11:16.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.085+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557f96101a50 0x557f96103e70 crc :-1 s=READY pgs=671 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.085+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f9607ce70 0x557f96081400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.085+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f95efde40 0x557f9608f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.085+0000 7fc8d3a1ab80 1 --2- 172.21.15.63:0/4218164689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f95fe99f0 0x557f96080ec0 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.085+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/4218164689 >> 172.21.15.63:0/4218164689 conn(0x557f95f97390 msgr2=0x557f9607ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:16.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.085+0000 7fc89fe00700 0 client.22605 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:16.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.085+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/4218164689 shutdown_connections 2024-05-27T08:11:16.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.085+0000 7fc8d3a1ab80 1 -- 172.21.15.63:0/4218164689 wait complete. 2024-05-27T08:11:16.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.085+0000 7fc8d3a1ab80 20 client.22605 trim_cache size 0 max 16384 2024-05-27T08:11:16.087 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:11:16.087 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:11:16.087 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:11:16.087 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.086+0000 7fc8c4e00700 1 -- 172.21.15.63:0/4259114208 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8a40b2a70 msgr2=0x7fc8a40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.086+0000 7fc8c4e00700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8a40b2a70 0x7fc8a40b4f80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc8c01079d0 tx=0x7fc8b0009260 comp rx=0 tx=0).stop 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.086+0000 7fc8c4e00700 1 -- 172.21.15.63:0/4259114208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c010c5a0 msgr2=0x7fc8c01a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.086+0000 7fc8c4e00700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c010c5a0 0x7fc8c01a1730 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800b070 tx=0x7fc8b8004c80 comp rx=0 tx=0).stop 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.086+0000 7fc8c4e00700 1 -- 172.21.15.63:0/4259114208 shutdown_connections 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.086+0000 7fc8c4e00700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc8a40b2a70 0x7fc8a40b4f80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.086+0000 7fc8c4e00700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8c010c5a0 0x7fc8c01a1730 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.086+0000 7fc8c4e00700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8c0106560 0x7fc8c019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.086+0000 7fc8c4e00700 1 --2- 172.21.15.63:0/4259114208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c0106eb0 0x7fc8c019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.086+0000 7fc8c4e00700 1 -- 172.21.15.63:0/4259114208 >> 172.21.15.63:0/4259114208 conn(0x7fc8c0102040 msgr2=0x7fc8c01035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.087+0000 7fc8c4e00700 1 -- 172.21.15.63:0/4259114208 shutdown_connections 2024-05-27T08:11:16.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:16.087+0000 7fc8c4e00700 1 -- 172.21.15.63:0/4259114208 wait complete. 2024-05-27T08:11:16.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:15 smithi086 ceph-mon[34687]: pgmap v1195: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.6 MiB/s wr, 13 op/s 2024-05-27T08:11:16.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:15 smithi149 ceph-mon[35408]: pgmap v1195: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.6 MiB/s wr, 13 op/s 2024-05-27T08:11:17.076 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:11:17.076 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:11:17.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:17 smithi063 ceph-mon[31613]: osdmap e156: 12 total, 12 up, 12 in 2024-05-27T08:11:17.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:17 smithi063 ceph-mon[31613]: pgmap v1197: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 13 op/s 2024-05-27T08:11:17.619 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:11:17.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:17 smithi086 ceph-mon[34687]: osdmap e156: 12 total, 12 up, 12 in 2024-05-27T08:11:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:17 smithi086 ceph-mon[34687]: pgmap v1197: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 13 op/s 2024-05-27T08:11:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:17 smithi149 ceph-mon[35408]: osdmap e156: 12 total, 12 up, 12 in 2024-05-27T08:11:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:17 smithi149 ceph-mon[35408]: pgmap v1197: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 13 op/s 2024-05-27T08:11:19.754 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:19 smithi063 ceph-mon[31613]: pgmap v1198: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T08:11:20.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.145+0000 7ffb8fe00700 1 -- 172.21.15.63:0/338251461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9010af40 msgr2=0x7ffb9010b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:20.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.145+0000 7ffb8fe00700 1 --2- 172.21.15.63:0/338251461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9010af40 0x7ffb9010b350 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0099c0 tx=0x7ffb7c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.145+0000 7ffb8fe00700 1 -- 172.21.15.63:0/338251461 shutdown_connections 2024-05-27T08:11:20.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.145+0000 7ffb8fe00700 1 --2- 172.21.15.63:0/338251461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9010b890 0x7ffb901131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.145+0000 7ffb8fe00700 1 --2- 172.21.15.63:0/338251461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb90106610 0x7ffb90106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.145+0000 7ffb8fe00700 1 --2- 172.21.15.63:0/338251461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9010af40 0x7ffb9010b350 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.145+0000 7ffb8fe00700 1 -- 172.21.15.63:0/338251461 >> 172.21.15.63:0/338251461 conn(0x7ffb90102020 msgr2=0x7ffb90104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:20.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.145+0000 7ffb8fe00700 1 -- 172.21.15.63:0/338251461 shutdown_connections 2024-05-27T08:11:20.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.145+0000 7ffb8fe00700 1 -- 172.21.15.63:0/338251461 wait complete. 2024-05-27T08:11:20.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.146+0000 7ffb8fe00700 1 Processor -- start 2024-05-27T08:11:20.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.147+0000 7ffb8fe00700 1 -- start start 2024-05-27T08:11:20.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.147+0000 7ffb8fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb90106610 0x7ffb9019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:20.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.147+0000 7ffb8fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9010af40 0x7ffb9019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:20.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.147+0000 7ffb8fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9010b890 0x7ffb901a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:20.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.147+0000 7ffb8fe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901163a0 con 0x7ffb9010b890 2024-05-27T08:11:20.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.147+0000 7ffb8fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90116220 con 0x7ffb90106610 2024-05-27T08:11:20.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.147+0000 7ffb8fe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90116520 con 0x7ffb9010af40 2024-05-27T08:11:20.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.148+0000 7ffb8f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9010b890 0x7ffb901a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:20.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.148+0000 7ffb8f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9010b890 0x7ffb901a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47440/0 (socket says 172.21.15.63:47440) 2024-05-27T08:11:20.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.148+0000 7ffb8ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb90106610 0x7ffb9019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:20.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.148+0000 7ffb8f400700 1 -- 172.21.15.63:0/214593158 learned_addr learned my addr 172.21.15.63:0/214593158 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:20.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.148+0000 7ffb8f400700 1 -- 172.21.15.63:0/214593158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9010af40 msgr2=0x7ffb9019d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:20.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.148+0000 7ffb8e000700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9010af40 0x7ffb9019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:20.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.148+0000 7ffb8f400700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9010af40 0x7ffb9019d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.148+0000 7ffb8f400700 1 -- 172.21.15.63:0/214593158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb90106610 msgr2=0x7ffb9019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:20.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.148+0000 7ffb8f400700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb90106610 0x7ffb9019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.148+0000 7ffb8f400700 1 -- 172.21.15.63:0/214593158 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb901a1c80 con 0x7ffb9010b890 2024-05-27T08:11:20.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.149+0000 7ffb8e000700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9010af40 0x7ffb9019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:11:20.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.149+0000 7ffb8f400700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9010b890 0x7ffb901a15e0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800ef30 tx=0x7ffb8800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:20.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.149+0000 7ffb94a00700 1 -- 172.21.15.63:0/214593158 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb880092e0 con 0x7ffb9010b890 2024-05-27T08:11:20.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.149+0000 7ffb8fe00700 1 -- 172.21.15.63:0/214593158 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb901a1f10 con 0x7ffb9010b890 2024-05-27T08:11:20.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.149+0000 7ffb94a00700 1 -- 172.21.15.63:0/214593158 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb88004bb0 con 0x7ffb9010b890 2024-05-27T08:11:20.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.149+0000 7ffb94a00700 1 -- 172.21.15.63:0/214593158 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88004020 con 0x7ffb9010b890 2024-05-27T08:11:20.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.150+0000 7ffb8fe00700 1 -- 172.21.15.63:0/214593158 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb901a2400 con 0x7ffb9010b890 2024-05-27T08:11:20.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.151+0000 7ffb8fe00700 1 -- 172.21.15.63:0/214593158 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb900512b0 con 0x7ffb9010b890 2024-05-27T08:11:20.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.153+0000 7ffb94a00700 1 -- 172.21.15.63:0/214593158 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffb8801b030 con 0x7ffb9010b890 2024-05-27T08:11:20.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.156+0000 7ffb94a00700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb740b29f0 0x7ffb740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:20.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.156+0000 7ffb94a00700 1 -- 172.21.15.63:0/214593158 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7ffb880e2500 con 0x7ffb9010b890 2024-05-27T08:11:20.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.156+0000 7ffb8ea00700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb740b29f0 0x7ffb740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:20.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.157+0000 7ffb8ea00700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb740b29f0 0x7ffb740b4f00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009c60 tx=0x7ffb80009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:20.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.160+0000 7ffb94a00700 1 -- 172.21.15.63:0/214593158 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffb88022020 con 0x7ffb9010b890 2024-05-27T08:11:20.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:19 smithi086 ceph-mon[34687]: pgmap v1198: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T08:11:20.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:19 smithi149 ceph-mon[35408]: pgmap v1198: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T08:11:20.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.515+0000 7ffb8fe00700 1 -- 172.21.15.63:0/214593158 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb9002e5e0 con 0x7ffb9010b890 2024-05-27T08:11:20.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.516+0000 7ffb94a00700 1 -- 172.21.15.63:0/214593158 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7ffb88010040 con 0x7ffb9010b890 2024-05-27T08:11:20.517 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:11:20.518 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:11:20.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.520+0000 7ffb8fe00700 1 -- 172.21.15.63:0/214593158 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb740b29f0 msgr2=0x7ffb740b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:20.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.520+0000 7ffb8fe00700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb740b29f0 0x7ffb740b4f00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009c60 tx=0x7ffb80009260 comp rx=0 tx=0).stop 2024-05-27T08:11:20.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.520+0000 7ffb8fe00700 1 -- 172.21.15.63:0/214593158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9010b890 msgr2=0x7ffb901a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:20.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.520+0000 7ffb8fe00700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9010b890 0x7ffb901a15e0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800ef30 tx=0x7ffb8800c3a0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.520+0000 7ffb8fe00700 1 -- 172.21.15.63:0/214593158 shutdown_connections 2024-05-27T08:11:20.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.520+0000 7ffb8fe00700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffb740b29f0 0x7ffb740b4f00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.520+0000 7ffb8fe00700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb9010b890 0x7ffb901a15e0 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.520+0000 7ffb8fe00700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb90106610 0x7ffb9019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.520+0000 7ffb8fe00700 1 --2- 172.21.15.63:0/214593158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9010af40 0x7ffb9019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:20.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.520+0000 7ffb8fe00700 1 -- 172.21.15.63:0/214593158 >> 172.21.15.63:0/214593158 conn(0x7ffb90102020 msgr2=0x7ffb9010a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:20.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.521+0000 7ffb8fe00700 1 -- 172.21.15.63:0/214593158 shutdown_connections 2024-05-27T08:11:20.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:20.521+0000 7ffb8fe00700 1 -- 172.21.15.63:0/214593158 wait complete. 2024-05-27T08:11:20.523 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:11:20.943 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:20 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/214593158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:11:21.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:20 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/214593158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:11:21.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:20 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/214593158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:11:21.498 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:11:22.000 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:11:22.160 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:21 smithi063 ceph-mon[31613]: pgmap v1199: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T08:11:22.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:21 smithi086 ceph-mon[34687]: pgmap v1199: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T08:11:22.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:21 smithi149 ceph-mon[35408]: pgmap v1199: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 13 op/s 2024-05-27T08:11:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.256+0000 7f2bbc000640 1 -- 172.21.15.63:0/1580086451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bb4108bc0 msgr2=0x7f2bb4102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.256+0000 7f2bbc000640 1 --2- 172.21.15.63:0/1580086451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bb4108bc0 0x7f2bb4102760 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009900 tx=0x7f2ba802a410 comp rx=0 tx=0).stop 2024-05-27T08:11:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.257+0000 7f2bbc000640 1 -- 172.21.15.63:0/1580086451 shutdown_connections 2024-05-27T08:11:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.257+0000 7f2bbc000640 1 --2- 172.21.15.63:0/1580086451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb4103310 0x7f2bb410f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.257+0000 7f2bbc000640 1 --2- 172.21.15.63:0/1580086451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bb4109060 0x7f2bb4102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.257+0000 7f2bbc000640 1 --2- 172.21.15.63:0/1580086451 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bb4108bc0 0x7f2bb4102760 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.257+0000 7f2bbc000640 1 -- 172.21.15.63:0/1580086451 >> 172.21.15.63:0/1580086451 conn(0x7f2bb40782c0 msgr2=0x7f2bb4078700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.257+0000 7f2bbc000640 1 -- 172.21.15.63:0/1580086451 shutdown_connections 2024-05-27T08:11:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.257+0000 7f2bbc000640 1 -- 172.21.15.63:0/1580086451 wait complete. 2024-05-27T08:11:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.258+0000 7f2bbc000640 1 Processor -- start 2024-05-27T08:11:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.258+0000 7f2bbc000640 1 -- start start 2024-05-27T08:11:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bbc000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bb4103310 0x7f2bb40ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bbc000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb4108bc0 0x7f2bb40ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bbc000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bb4109060 0x7f2bb40ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bbc000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb4112c60 con 0x7f2bb4109060 2024-05-27T08:11:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bbc000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb4112ae0 con 0x7f2bb4103310 2024-05-27T08:11:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bbc000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb4112de0 con 0x7f2bb4108bc0 2024-05-27T08:11:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bba000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bb4109060 0x7f2bb40ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bba000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bb4109060 0x7f2bb40ffe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47472/0 (socket says 172.21.15.63:47472) 2024-05-27T08:11:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bba000640 1 -- 172.21.15.63:0/2894396423 learned_addr learned my addr 172.21.15.63:0/2894396423 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bb9600640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bb4103310 0x7f2bb40ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.259+0000 7f2bb8c00640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb4108bc0 0x7f2bb40ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bba000640 1 -- 172.21.15.63:0/2894396423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb4108bc0 msgr2=0x7f2bb40ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bba000640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb4108bc0 0x7f2bb40ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bba000640 1 -- 172.21.15.63:0/2894396423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bb4103310 msgr2=0x7f2bb40ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bba000640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bb4103310 0x7f2bb40ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bba000640 1 -- 172.21.15.63:0/2894396423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb41a8bb0 con 0x7f2bb4109060 2024-05-27T08:11:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bb9600640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bb4103310 0x7f2bb40ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bba000640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bb4109060 0x7f2bb40ffe20 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00eaa0 tx=0x7f2bac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bb2000640 1 -- 172.21.15.63:0/2894396423 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00cdb0 con 0x7f2bb4109060 2024-05-27T08:11:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bb2000640 1 -- 172.21.15.63:0/2894396423 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bac005020 con 0x7f2bb4109060 2024-05-27T08:11:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bbc000640 1 -- 172.21.15.63:0/2894396423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb41a8ea0 con 0x7f2bb4109060 2024-05-27T08:11:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.260+0000 7f2bb2000640 1 -- 172.21.15.63:0/2894396423 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0116d0 con 0x7f2bb4109060 2024-05-27T08:11:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.261+0000 7f2bbc000640 1 -- 172.21.15.63:0/2894396423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb41a9330 con 0x7f2bb4109060 2024-05-27T08:11:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.264+0000 7f2bc3082740 1 Processor -- start 2024-05-27T08:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.265+0000 7f2bb2000640 1 -- 172.21.15.63:0/2894396423 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2bac004b20 con 0x7f2bb4109060 2024-05-27T08:11:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.265+0000 7f2bc3082740 1 -- start start 2024-05-27T08:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807ed21b0 0x556808274860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x556808274da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807eb14d0 0x5568082752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bc3082740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5568081e5b90 con 0x556807eb14d0 2024-05-27T08:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bc3082740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568081d5730 con 0x556807ed21b0 2024-05-27T08:11:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bc3082740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5568081d9770 con 0x556807ea92f0 2024-05-27T08:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bb8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807eb14d0 0x5568082752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bb8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807eb14d0 0x5568082752e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47488/0 (socket says 172.21.15.63:47488) 2024-05-27T08:11:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bb8c00640 1 -- 172.21.15.63:0/800787731 learned_addr learned my addr 172.21.15.63:0/800787731 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bb8c00640 1 --2- 172.21.15.63:0/800787731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807ed21b0 0x556808274860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.266+0000 7f2bb9600640 1 --2- 172.21.15.63:0/800787731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x556808274da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.267+0000 7f2bb8c00640 1 -- 172.21.15.63:0/800787731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 msgr2=0x556808274da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.267+0000 7f2bb8c00640 1 --2- 172.21.15.63:0/800787731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x556808274da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.267+0000 7f2bb8c00640 1 -- 172.21.15.63:0/800787731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807ed21b0 msgr2=0x556808274860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.267+0000 7f2bb8c00640 1 --2- 172.21.15.63:0/800787731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807ed21b0 0x556808274860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.267+0000 7f2bb8c00640 1 -- 172.21.15.63:0/800787731 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55680816e8b0 con 0x556807eb14d0 2024-05-27T08:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.267+0000 7f2bb2000640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b9c0b2950 0x7f2b9c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.267+0000 7f2bb8c00640 1 --2- 172.21.15.63:0/800787731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807eb14d0 0x5568082752e0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0005e00 tx=0x7f2ba000f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39686f60092f0fef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.267+0000 7f2bb2000640 1 -- 172.21.15.63:0/2894396423 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f2bac0e2c90 con 0x7f2bb4109060 2024-05-27T08:11:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.268+0000 7f2bb9600640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b9c0b2950 0x7f2b9c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.268+0000 7f2ba6600640 1 -- 172.21.15.63:0/800787731 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0019020 con 0x556807eb14d0 2024-05-27T08:11:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.268+0000 7f2ba6600640 1 -- 172.21.15.63:0/800787731 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ba0011b90 con 0x556807eb14d0 2024-05-27T08:11:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.268+0000 7f2ba6600640 1 -- 172.21.15.63:0/800787731 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0010ad0 con 0x556807eb14d0 2024-05-27T08:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.269+0000 7f2bc3082740 1 -- 172.21.15.63:0/800787731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807eb14d0 msgr2=0x5568082752e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.269+0000 7f2bc3082740 1 --2- 172.21.15.63:0/800787731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807eb14d0 0x5568082752e0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0005e00 tx=0x7f2ba000f420 comp rx=0 tx=0).stop 2024-05-27T08:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.269+0000 7f2bb9600640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b9c0b2950 0x7f2b9c0b4e60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8006010 tx=0x7f2ba80058b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.269+0000 7f2bc3082740 1 -- 172.21.15.63:0/800787731 shutdown_connections 2024-05-27T08:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.269+0000 7f2bc3082740 1 --2- 172.21.15.63:0/800787731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807eb14d0 0x5568082752e0 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.269+0000 7f2bc3082740 1 --2- 172.21.15.63:0/800787731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x556808274da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.269+0000 7f2bc3082740 1 --2- 172.21.15.63:0/800787731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807ed21b0 0x556808274860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.269+0000 7f2bc3082740 1 -- 172.21.15.63:0/800787731 >> 172.21.15.63:0/800787731 conn(0x556807ec6300 msgr2=0x556808271010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.269+0000 7f2bc3082740 1 -- 172.21.15.63:0/800787731 shutdown_connections 2024-05-27T08:11:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.269+0000 7f2bc3082740 1 -- 172.21.15.63:0/800787731 wait complete. 2024-05-27T08:11:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.270+0000 7f2bc3082740 1 -- start start 2024-05-27T08:11:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.270+0000 7f2bc3082740 1 Processor -- start 2024-05-27T08:11:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.271+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x55680827ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.271+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x55680827efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.271+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x55680828acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.271+0000 7f2bc3082740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556807ebb790 con 0x556807ed21b0 2024-05-27T08:11:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.271+0000 7f2bc3082740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55680817dbc0 con 0x556807eb14d0 2024-05-27T08:11:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.271+0000 7f2bc3082740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556808281990 con 0x556807ea92f0 2024-05-27T08:11:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.271+0000 7f2bc3082740 10 client.0 _ll_register_callbacks cb 0x55680815b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.271+0000 7f2bba000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x55680828acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.271+0000 7f2bba000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x55680828acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47496/0 (socket says 172.21.15.63:47496) 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.272+0000 7f2bba000640 1 -- 172.21.15.63:0/1776096581 learned_addr learned my addr 172.21.15.63:0/1776096581 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.272+0000 7f2bb9600640 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x55680827ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.272+0000 7f2bb8c00640 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x55680827efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.272+0000 7f2bba000640 1 -- 172.21.15.63:0/1776096581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 msgr2=0x55680827ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.272+0000 7f2bba000640 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x55680827ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.272+0000 7f2bba000640 1 -- 172.21.15.63:0/1776096581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 msgr2=0x55680827efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.272+0000 7f2bba000640 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x55680827efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.272+0000 7f2bba000640 1 -- 172.21.15.63:0/1776096581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556807eea210 con 0x556807ed21b0 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.272+0000 7f2bba000640 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x55680828acc0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0a9ae0 tx=0x7f2bac004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bd4418ea8f2857 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.272+0000 7f2b9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.273+0000 7f2b9aa00640 1 -- 172.21.15.63:0/1776096581 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac01f030 con 0x556807ed21b0 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.273+0000 7f2b9aa00640 1 -- 172.21.15.63:0/1776096581 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f2bac0a6140 con 0x556807ed21b0 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.273+0000 7f2bc3082740 1 -- 172.21.15.63:0/1776096581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ba0011b90 con 0x556807ed21b0 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.273+0000 7f2b9aa00640 1 -- 172.21.15.63:0/1776096581 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2bac06e050 con 0x556807ed21b0 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.273+0000 7f2bc3082740 10 client.22623 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.273+0000 7f2bc3082740 1 -- 172.21.15.63:0/1776096581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556807fb6620 con 0x556807ed21b0 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.273+0000 7f2b9aa00640 1 -- 172.21.15.63:0/1776096581 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ba0011b90 con 0x556807ed21b0 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.274+0000 7f2b9aa00640 1 -- 172.21.15.63:0/1776096581 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2bac00f040 con 0x556807ed21b0 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.274+0000 7f2bc3082740 10 client.22623 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.274+0000 7f2bc3082740 10 client.22623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.274+0000 7f2bc3082740 20 client.22623 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.274+0000 7f2bc3082740 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556807e90d80 0x556808290d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.274+0000 7f2bc3082740 4 client.22623 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:11:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.274+0000 7f2bc3082740 1 -- 172.21.15.63:0/1776096581 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568081e7ae0 con 0x556807e90d80 2024-05-27T08:11:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.275+0000 7f2bb9600640 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556807e90d80 0x556808290d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.276+0000 7f2bb9600640 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556807e90d80 0x556808290d80 crc :-1 s=READY pgs=672 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44ac713c17cbbc62 server_cookie=692d77672b3fece7 in_seq=0 out_seq=0 2024-05-27T08:11:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.276+0000 7f2b9aa00640 10 client.22623 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.277+0000 7f2b9aa00640 1 -- 172.21.15.63:0/1776096581 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5568081e7ae0 con 0x556807e90d80 2024-05-27T08:11:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.277+0000 7f2b9aa00640 10 client.22623 handle_command_reply: tid=0 2024-05-27T08:11:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 client.22623 shutdown 2024-05-27T08:11:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 -- 172.21.15.63:0/1776096581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 msgr2=0x55680828acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x55680828acc0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0a9ae0 tx=0x7f2bac004ed0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 -- 172.21.15.63:0/1776096581 shutdown_connections 2024-05-27T08:11:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556807e90d80 0x556808290d80 crc :-1 s=READY pgs=672 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x55680828acc0 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x55680827efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2b9aa00640 0 client.22623 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 --2- 172.21.15.63:0/1776096581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x55680827ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 -- 172.21.15.63:0/1776096581 >> 172.21.15.63:0/1776096581 conn(0x556807ec6300 msgr2=0x556808276580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 -- 172.21.15.63:0/1776096581 shutdown_connections 2024-05-27T08:11:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 1 -- 172.21.15.63:0/1776096581 wait complete. 2024-05-27T08:11:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.278+0000 7f2bc3082740 20 client.22623 trim_cache size 0 max 16384 2024-05-27T08:11:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.303+0000 7f2bc3082740 1 Processor -- start 2024-05-27T08:11:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.303+0000 7f2bc3082740 1 -- start start 2024-05-27T08:11:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.303+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ea92f0 0x556808280bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.303+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x55680828aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.303+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ed21b0 0x55680828af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.303+0000 7f2bc3082740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41b5d10 con 0x556807ea92f0 2024-05-27T08:11:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.303+0000 7f2bc3082740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41b35e0 con 0x556807eb14d0 2024-05-27T08:11:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.303+0000 7f2bc3082740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41b1450 con 0x556807ed21b0 2024-05-27T08:11:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.304+0000 7f2bb9600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ea92f0 0x556808280bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.304+0000 7f2bb9600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ea92f0 0x556808280bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47498/0 (socket says 172.21.15.63:47498) 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.304+0000 7f2bb9600640 1 -- 172.21.15.63:0/1101944177 learned_addr learned my addr 172.21.15.63:0/1101944177 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.304+0000 7f2bb8c00640 1 --2- 172.21.15.63:0/1101944177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x55680828aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.304+0000 7f2bba000640 1 --2- 172.21.15.63:0/1101944177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ed21b0 0x55680828af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.304+0000 7f2bb9600640 1 -- 172.21.15.63:0/1101944177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ed21b0 msgr2=0x55680828af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.304+0000 7f2bb9600640 1 --2- 172.21.15.63:0/1101944177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ed21b0 0x55680828af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.304+0000 7f2bb9600640 1 -- 172.21.15.63:0/1101944177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 msgr2=0x55680828aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.304+0000 7f2bb9600640 1 --2- 172.21.15.63:0/1101944177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x55680828aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.304+0000 7f2bb9600640 1 -- 172.21.15.63:0/1101944177 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55680828b4b0 con 0x556807ea92f0 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.305+0000 7f2bb9600640 1 --2- 172.21.15.63:0/1101944177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ea92f0 0x556808280bc0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80315f0 tx=0x7f2ba803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda4aaa414ef575d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.305+0000 7f2ba5c00640 1 -- 172.21.15.63:0/1101944177 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8054070 con 0x556807ea92f0 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.305+0000 7f2ba5c00640 1 -- 172.21.15.63:0/1101944177 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ba80035f0 con 0x556807ea92f0 2024-05-27T08:11:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.305+0000 7f2ba5c00640 1 -- 172.21.15.63:0/1101944177 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8035050 con 0x556807ea92f0 2024-05-27T08:11:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.306+0000 7f2bc3082740 1 -- 172.21.15.63:0/1101944177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ea92f0 msgr2=0x556808280bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.306+0000 7f2bc3082740 1 --2- 172.21.15.63:0/1101944177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ea92f0 0x556808280bc0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80315f0 tx=0x7f2ba803d000 comp rx=0 tx=0).stop 2024-05-27T08:11:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.306+0000 7f2bc3082740 1 -- 172.21.15.63:0/1101944177 shutdown_connections 2024-05-27T08:11:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.306+0000 7f2bc3082740 1 --2- 172.21.15.63:0/1101944177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ed21b0 0x55680828af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.306+0000 7f2bc3082740 1 --2- 172.21.15.63:0/1101944177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x55680828aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.306+0000 7f2bc3082740 1 --2- 172.21.15.63:0/1101944177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ea92f0 0x556808280bc0 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.306+0000 7f2bc3082740 1 -- 172.21.15.63:0/1101944177 >> 172.21.15.63:0/1101944177 conn(0x556807ec6300 msgr2=0x55680827edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.306+0000 7f2bc3082740 1 -- 172.21.15.63:0/1101944177 shutdown_connections 2024-05-27T08:11:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.307+0000 7f2bc3082740 1 -- 172.21.15.63:0/1101944177 wait complete. 2024-05-27T08:11:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.307+0000 7f2bc3082740 1 -- start start 2024-05-27T08:11:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.307+0000 7f2bc3082740 1 Processor -- start 2024-05-27T08:11:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x556808287120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x5568082824c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bc3082740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x556808282a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bc3082740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41b1aa0 con 0x556807ed21b0 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bc3082740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41b2f10 con 0x556807eb14d0 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bc3082740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41b2170 con 0x556807ea92f0 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bc3082740 10 client.0 _ll_register_callbacks cb 0x7f2bac0057c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bba000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x556808282a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bba000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x556808282a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47514/0 (socket says 172.21.15.63:47514) 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bba000640 1 -- 172.21.15.63:0/2940750785 learned_addr learned my addr 172.21.15.63:0/2940750785 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.308+0000 7f2bb9600640 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x556808287120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2bb8c00640 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x5568082824c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2bba000640 1 -- 172.21.15.63:0/2940750785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 msgr2=0x556808287120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2bba000640 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x556808287120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2bba000640 1 -- 172.21.15.63:0/2940750785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 msgr2=0x5568082824c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2bba000640 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x5568082824c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2bba000640 1 -- 172.21.15.63:0/2940750785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556808282f40 con 0x556807ed21b0 2024-05-27T08:11:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2bba000640 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x556808282a00 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0a6dc0 tx=0x7f2bac0a9060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c65566ff0ac8ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2b9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2b9aa00640 1 -- 172.21.15.63:0/2940750785 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac01f070 con 0x556807ed21b0 2024-05-27T08:11:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2bc3082740 1 -- 172.21.15.63:0/2940750785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ba80035f0 con 0x556807ed21b0 2024-05-27T08:11:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.309+0000 7f2b9aa00640 1 -- 172.21.15.63:0/2940750785 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f2bac0cae00 con 0x556807ed21b0 2024-05-27T08:11:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2b9aa00640 1 -- 172.21.15.63:0/2940750785 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2bac08d050 con 0x556807ed21b0 2024-05-27T08:11:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2bc3082740 10 client.22629 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2bc3082740 1 -- 172.21.15.63:0/2940750785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5568082f9110 con 0x556807ed21b0 2024-05-27T08:11:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2b9aa00640 1 -- 172.21.15.63:0/2940750785 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ba80035f0 con 0x556807ed21b0 2024-05-27T08:11:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2b9aa00640 1 -- 172.21.15.63:0/2940750785 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2bac087020 con 0x556807ed21b0 2024-05-27T08:11:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2bc3082740 10 client.22629 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2bc3082740 10 client.22629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2bc3082740 20 client.22629 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2bc3082740 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556807e661c0 0x5568082fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2bc3082740 4 client.22629 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:11:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.310+0000 7f2bc3082740 1 -- 172.21.15.63:0/2940750785 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568081e12c0 con 0x556807e661c0 2024-05-27T08:11:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.311+0000 7f2bb9600640 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556807e661c0 0x5568082fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.312+0000 7f2bb9600640 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556807e661c0 0x5568082fb8c0 crc :-1 s=READY pgs=673 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=226c5a0965654835 server_cookie=2fa8a10411244b57 in_seq=0 out_seq=0 2024-05-27T08:11:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.312+0000 7f2b9aa00640 10 client.22629 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.352+0000 7f2b9aa00640 1 -- 172.21.15.63:0/2940750785 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5568081e12c0 con 0x556807e661c0 2024-05-27T08:11:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.352+0000 7f2b9aa00640 10 client.22629 handle_command_reply: tid=0 2024-05-27T08:11:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.353+0000 7f2bc3082740 1 client.22629 shutdown 2024-05-27T08:11:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.353+0000 7f2bc3082740 1 -- 172.21.15.63:0/2940750785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 msgr2=0x556808282a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.353+0000 7f2bc3082740 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x556808282a00 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0a6dc0 tx=0x7f2bac0a9060 comp rx=0 tx=0).stop 2024-05-27T08:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.353+0000 7f2bc3082740 1 -- 172.21.15.63:0/2940750785 shutdown_connections 2024-05-27T08:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.353+0000 7f2bc3082740 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556807e661c0 0x5568082fb8c0 crc :-1 s=READY pgs=673 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.353+0000 7f2bc3082740 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556807ed21b0 0x556808282a00 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.353+0000 7f2bc3082740 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556807eb14d0 0x5568082824c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.353+0000 7f2bc3082740 1 --2- 172.21.15.63:0/2940750785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556807ea92f0 0x556808287120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.353+0000 7f2bc3082740 1 -- 172.21.15.63:0/2940750785 >> 172.21.15.63:0/2940750785 conn(0x556807ec6300 msgr2=0x55680827edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.353+0000 7f2b9aa00640 0 client.22629 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.354+0000 7f2bc3082740 1 -- 172.21.15.63:0/2940750785 shutdown_connections 2024-05-27T08:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.354+0000 7f2bc3082740 1 -- 172.21.15.63:0/2940750785 wait complete. 2024-05-27T08:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.354+0000 7f2bc3082740 20 client.22629 trim_cache size 0 max 16384 2024-05-27T08:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:11:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:11:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 -- 172.21.15.63:0/2894396423 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b9c0b2950 msgr2=0x7f2b9c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b9c0b2950 0x7f2b9c0b4e60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8006010 tx=0x7f2ba80058b0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 -- 172.21.15.63:0/2894396423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bb4109060 msgr2=0x7f2bb40ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bb4109060 0x7f2bb40ffe20 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00eaa0 tx=0x7f2bac00ef60 comp rx=0 tx=0).stop 2024-05-27T08:11:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 -- 172.21.15.63:0/2894396423 shutdown_connections 2024-05-27T08:11:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b9c0b2950 0x7f2b9c0b4e60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bb4109060 0x7f2bb40ffe20 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb4108bc0 0x7f2bb40ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 --2- 172.21.15.63:0/2894396423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bb4103310 0x7f2bb40ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 -- 172.21.15.63:0/2894396423 >> 172.21.15.63:0/2894396423 conn(0x7f2bb40782c0 msgr2=0x7f2bb4074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.355+0000 7f2bbc000640 1 -- 172.21.15.63:0/2894396423 shutdown_connections 2024-05-27T08:11:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:22.356+0000 7f2bbc000640 1 -- 172.21.15.63:0/2894396423 wait complete. 2024-05-27T08:11:23.878 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:23 smithi063 ceph-mon[31613]: pgmap v1200: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:11:24.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:23 smithi086 ceph-mon[34687]: pgmap v1200: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:11:24.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:23 smithi149 ceph-mon[35408]: pgmap v1200: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:11:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.082+0000 7fcce7e00700 1 -- 172.21.15.63:0/246234251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce8106610 msgr2=0x7fcce8106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.082+0000 7fcce7e00700 1 --2- 172.21.15.63:0/246234251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce8106610 0x7fcce8106a20 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fccd80099c0 tx=0x7fccd8009cd0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.082+0000 7fcce7e00700 1 -- 172.21.15.63:0/246234251 shutdown_connections 2024-05-27T08:11:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.082+0000 7fcce7e00700 1 --2- 172.21.15.63:0/246234251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcce810b890 0x7fcce81131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.082+0000 7fcce7e00700 1 --2- 172.21.15.63:0/246234251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce8106610 0x7fcce8106a20 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.082+0000 7fcce7e00700 1 --2- 172.21.15.63:0/246234251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce810af40 0x7fcce810b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.082+0000 7fcce7e00700 1 -- 172.21.15.63:0/246234251 >> 172.21.15.63:0/246234251 conn(0x7fcce8102020 msgr2=0x7fcce8104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.082+0000 7fcce7e00700 1 -- 172.21.15.63:0/246234251 shutdown_connections 2024-05-27T08:11:25.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.082+0000 7fcce7e00700 1 -- 172.21.15.63:0/246234251 wait complete. 2024-05-27T08:11:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.083+0000 7fcce7e00700 1 Processor -- start 2024-05-27T08:11:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.084+0000 7fcce7e00700 1 -- start start 2024-05-27T08:11:25.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.084+0000 7fcce7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce8106610 0x7fcce819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.084+0000 7fcce7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcce810af40 0x7fcce819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.084+0000 7fcce7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce810b890 0x7fcce81a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.084+0000 7fcce7e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81163c0 con 0x7fcce8106610 2024-05-27T08:11:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.084+0000 7fcce7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8116240 con 0x7fcce810af40 2024-05-27T08:11:25.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.084+0000 7fcce7e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8116540 con 0x7fcce810b890 2024-05-27T08:11:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.085+0000 7fcce6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce8106610 0x7fcce819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.085+0000 7fcce6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce8106610 0x7fcce819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47532/0 (socket says 172.21.15.63:47532) 2024-05-27T08:11:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.085+0000 7fcce6a00700 1 -- 172.21.15.63:0/1684230785 learned_addr learned my addr 172.21.15.63:0/1684230785 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.085+0000 7fcce7400700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce810b890 0x7fcce81a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.085+0000 7fcce6a00700 1 -- 172.21.15.63:0/1684230785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce810b890 msgr2=0x7fcce81a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.085+0000 7fcce6a00700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce810b890 0x7fcce81a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.085+0000 7fcce6a00700 1 -- 172.21.15.63:0/1684230785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcce810af40 msgr2=0x7fcce819d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.085+0000 7fcce6a00700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcce810af40 0x7fcce819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.085+0000 7fcce6a00700 1 -- 172.21.15.63:0/1684230785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce81a1bb0 con 0x7fcce8106610 2024-05-27T08:11:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.086+0000 7fcce7400700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce810b890 0x7fcce81a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.086+0000 7fcce6a00700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce8106610 0x7fcce819cd90 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fccd800b580 tx=0x7fccd800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.086+0000 7fccdf400700 1 -- 172.21.15.63:0/1684230785 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8027070 con 0x7fcce8106610 2024-05-27T08:11:25.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.086+0000 7fccdf400700 1 -- 172.21.15.63:0/1684230785 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccd800fdd0 con 0x7fcce8106610 2024-05-27T08:11:25.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.086+0000 7fccdf400700 1 -- 172.21.15.63:0/1684230785 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80173f0 con 0x7fcce8106610 2024-05-27T08:11:25.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.087+0000 7fcce7e00700 1 -- 172.21.15.63:0/1684230785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce81a1e40 con 0x7fcce8106610 2024-05-27T08:11:25.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.087+0000 7fcce7e00700 1 -- 172.21.15.63:0/1684230785 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce81a22d0 con 0x7fcce8106610 2024-05-27T08:11:25.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.091+0000 7fccdf400700 1 -- 172.21.15.63:0/1684230785 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fccd8022690 con 0x7fcce8106610 2024-05-27T08:11:25.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.092+0000 7fccfc645b80 1 Processor -- start 2024-05-27T08:11:25.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.093+0000 7fccfc645b80 1 -- start start 2024-05-27T08:11:25.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.093+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72279f0 0x5558b72bb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.093+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 0x5558b72bbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.093+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72c52c0 0x5558b72c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.093+0000 7fccfc645b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5558b71e8e40 con 0x5558b72279f0 2024-05-27T08:11:25.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.093+0000 7fccfc645b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5558b71d89e0 con 0x5558b713be40 2024-05-27T08:11:25.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.093+0000 7fccfc645b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5558b71dca20 con 0x5558b72c52c0 2024-05-27T08:11:25.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.094+0000 7fccdf400700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcccc0b2a70 0x7fcccc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.094+0000 7fccdf400700 1 -- 172.21.15.63:0/1684230785 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7fccd80e3b10 con 0x7fcce8106610 2024-05-27T08:11:25.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.094+0000 7fcce6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 0x5558b72bbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.094+0000 7fcce7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72279f0 0x5558b72bb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.094+0000 7fcce6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 0x5558b72bbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:52200/0 (socket says 172.21.15.63:52200) 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.094+0000 7fcce6a00700 1 -- 172.21.15.63:0/710734331 learned_addr learned my addr 172.21.15.63:0/710734331 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.094+0000 7fcce6000700 1 --2- 172.21.15.63:0/710734331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72c52c0 0x5558b72c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.094+0000 7fcce7400700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcccc0b2a70 0x7fcccc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.095+0000 7fcce6a00700 1 -- 172.21.15.63:0/710734331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72c52c0 msgr2=0x5558b72c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.095+0000 7fcce6a00700 1 --2- 172.21.15.63:0/710734331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72c52c0 0x5558b72c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.095+0000 7fcce6a00700 1 -- 172.21.15.63:0/710734331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72279f0 msgr2=0x5558b72bb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.095+0000 7fcce6a00700 1 --2- 172.21.15.63:0/710734331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72279f0 0x5558b72bb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.095+0000 7fcce6a00700 1 -- 172.21.15.63:0/710734331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558b71a1420 con 0x5558b713be40 2024-05-27T08:11:25.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.095+0000 7fcce6a00700 1 --2- 172.21.15.63:0/710734331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 0x5558b72bbc90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fccd80a6ae0 tx=0x7fccd80a61f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4784f60ab657ce7e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:25.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.096+0000 7fcce7400700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcccc0b2a70 0x7fcccc0b4f80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x5558b72b7a90 tx=0x7fcce0004b40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:25.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.095+0000 7fccd3e00700 1 -- 172.21.15.63:0/710734331 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80ab160 con 0x5558b713be40 2024-05-27T08:11:25.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.096+0000 7fccd3e00700 1 -- 172.21.15.63:0/710734331 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccd80a6040 con 0x5558b713be40 2024-05-27T08:11:25.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.096+0000 7fccd3e00700 1 -- 172.21.15.63:0/710734331 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80b10e0 con 0x5558b713be40 2024-05-27T08:11:25.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.097+0000 7fccfc645b80 1 -- 172.21.15.63:0/710734331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 msgr2=0x5558b72bbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:25.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.097+0000 7fccfc645b80 1 --2- 172.21.15.63:0/710734331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 0x5558b72bbc90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fccd80a6ae0 tx=0x7fccd80a61f0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.097+0000 7fccfc645b80 1 -- 172.21.15.63:0/710734331 shutdown_connections 2024-05-27T08:11:25.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.097+0000 7fccfc645b80 1 --2- 172.21.15.63:0/710734331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72c52c0 0x5558b72c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.097+0000 7fccfc645b80 1 --2- 172.21.15.63:0/710734331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72279f0 0x5558b72bb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.097+0000 7fccfc645b80 1 --2- 172.21.15.63:0/710734331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 0x5558b72bbc90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.097+0000 7fccfc645b80 1 -- 172.21.15.63:0/710734331 >> 172.21.15.63:0/710734331 conn(0x5558b71d5390 msgr2=0x5558b72b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.097+0000 7fccfc645b80 1 -- 172.21.15.63:0/710734331 shutdown_connections 2024-05-27T08:11:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.097+0000 7fccfc645b80 1 -- 172.21.15.63:0/710734331 wait complete. 2024-05-27T08:11:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.098+0000 7fccfc645b80 1 -- start start 2024-05-27T08:11:25.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.098+0000 7fccfc645b80 1 Processor -- start 2024-05-27T08:11:25.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.099+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 0x5558b72c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.099+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72279f0 0x5558b72c2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.099+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.099+0000 7fccfc645b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5558b70defa0 con 0x5558b72c52c0 2024-05-27T08:11:25.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.099+0000 7fccfc645b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5558b7179760 con 0x5558b713be40 2024-05-27T08:11:25.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.099+0000 7fccfc645b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5558b72c7c20 con 0x5558b72279f0 2024-05-27T08:11:25.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.099+0000 7fccfc645b80 10 client.0 _ll_register_callbacks cb 0x5558b7182c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:25.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.099+0000 7fcce7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.099+0000 7fcce7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47556/0 (socket says 172.21.15.63:47556) 2024-05-27T08:11:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.100+0000 7fcce7400700 1 -- 172.21.15.63:0/2893642321 learned_addr learned my addr 172.21.15.63:0/2893642321 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.100+0000 7fcce6a00700 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 0x5558b72c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.100+0000 7fcce6000700 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72279f0 0x5558b72c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.100+0000 7fcce7400700 1 -- 172.21.15.63:0/2893642321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72279f0 msgr2=0x5558b72c2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.100+0000 7fcce7400700 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72279f0 0x5558b72c2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.100+0000 7fcce7400700 1 -- 172.21.15.63:0/2893642321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 msgr2=0x5558b72c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:25.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.100+0000 7fcce7400700 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 0x5558b72c22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.100+0000 7fcce7400700 1 -- 172.21.15.63:0/2893642321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558b71288d0 con 0x5558b72c52c0 2024-05-27T08:11:25.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.100+0000 7fcce6000700 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72279f0 0x5558b72c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:25.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.101+0000 7fcce7400700 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c2d60 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7fcce0018e30 tx=0x7fcce0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0e0168fac92b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:25.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.101+0000 7fccd0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:25.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.101+0000 7fccd0c00700 1 -- 172.21.15.63:0/2893642321 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0010040 con 0x5558b72c52c0 2024-05-27T08:11:25.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.101+0000 7fccfc645b80 1 -- 172.21.15.63:0/2893642321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd80a6040 con 0x5558b72c52c0 2024-05-27T08:11:25.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.101+0000 7fccd0c00700 1 -- 172.21.15.63:0/2893642321 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7fcce00223f0 con 0x5558b72c52c0 2024-05-27T08:11:25.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.101+0000 7fccd0c00700 1 -- 172.21.15.63:0/2893642321 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcce0030050 con 0x5558b72c52c0 2024-05-27T08:11:25.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.101+0000 7fccfc645b80 10 client.22641 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:25.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.101+0000 7fccfc645b80 1 -- 172.21.15.63:0/2893642321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5558b71a9060 con 0x5558b72c52c0 2024-05-27T08:11:25.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.102+0000 7fccd0c00700 1 -- 172.21.15.63:0/2893642321 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fccd80a6040 con 0x5558b72c52c0 2024-05-27T08:11:25.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.102+0000 7fccd0c00700 1 -- 172.21.15.63:0/2893642321 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcce002b440 con 0x5558b72c52c0 2024-05-27T08:11:25.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.102+0000 7fccfc645b80 10 client.22641 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:25.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.102+0000 7fccfc645b80 10 client.22641 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:11:25.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.102+0000 7fccfc645b80 20 client.22641 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:25.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.102+0000 7fccfc645b80 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5558b72d4ad0 0x5558b72d6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.103+0000 7fccfc645b80 4 client.22641 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:11:25.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.103+0000 7fccfc645b80 1 -- 172.21.15.63:0/2893642321 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558b71ead90 con 0x5558b72d4ad0 2024-05-27T08:11:25.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.103+0000 7fcce6a00700 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5558b72d4ad0 0x5558b72d6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.104+0000 7fccd0c00700 10 client.22641 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:11:25.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.104+0000 7fcce6a00700 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5558b72d4ad0 0x5558b72d6f10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49317542774e3e66 server_cookie=5a948408ae80291d in_seq=0 out_seq=0 2024-05-27T08:11:25.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.106+0000 7fccd0c00700 1 -- 172.21.15.63:0/2893642321 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5558b71ead90 con 0x5558b72d4ad0 2024-05-27T08:11:25.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.106+0000 7fccd0c00700 10 client.22641 handle_command_reply: tid=0 2024-05-27T08:11:25.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.106+0000 7fccfc645b80 1 client.22641 shutdown 2024-05-27T08:11:25.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 1 -- 172.21.15.63:0/2893642321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 msgr2=0x5558b72c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c2d60 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7fcce0018e30 tx=0x7fcce0004ad0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 1 -- 172.21.15.63:0/2893642321 shutdown_connections 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5558b72d4ad0 0x5558b72d6f10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c2d60 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b713be40 0x5558b72c22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 1 --2- 172.21.15.63:0/2893642321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b72279f0 0x5558b72c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 1 -- 172.21.15.63:0/2893642321 >> 172.21.15.63:0/2893642321 conn(0x5558b71d5390 msgr2=0x5558b72bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccd0c00700 0 client.22641 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 1 -- 172.21.15.63:0/2893642321 shutdown_connections 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 1 -- 172.21.15.63:0/2893642321 wait complete. 2024-05-27T08:11:25.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.107+0000 7fccfc645b80 20 client.22641 trim_cache size 0 max 16384 2024-05-27T08:11:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.126+0000 7fccfc645b80 1 Processor -- start 2024-05-27T08:11:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.127+0000 7fccfc645b80 1 -- start start 2024-05-27T08:11:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.127+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 0x5558b72bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.127+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 0x5558b72b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.127+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.127+0000 7fccfc645b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b4d50 con 0x5558b72c52c0 2024-05-27T08:11:25.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.127+0000 7fccfc645b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b25e0 con 0x5558b72279f0 2024-05-27T08:11:25.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.127+0000 7fccfc645b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b0450 con 0x5558b713be40 2024-05-27T08:11:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.127+0000 7fcce7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72d0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47566/0 (socket says 172.21.15.63:47566) 2024-05-27T08:11:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce7400700 1 -- 172.21.15.63:0/4219226030 learned_addr learned my addr 172.21.15.63:0/4219226030 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce6000700 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 0x5558b72b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce6a00700 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 0x5558b72bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce7400700 1 -- 172.21.15.63:0/4219226030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 msgr2=0x5558b72bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce7400700 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 0x5558b72bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce7400700 1 -- 172.21.15.63:0/4219226030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 msgr2=0x5558b72b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce7400700 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 0x5558b72b6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce7400700 1 -- 172.21.15.63:0/4219226030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558b72d2550 con 0x5558b72c52c0 2024-05-27T08:11:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce6a00700 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 0x5558b72bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.128+0000 7fcce6000700 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 0x5558b72b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:25.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.129+0000 7fcce7400700 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72d0bf0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fcce000f040 tx=0x7fcce0017ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248e374857667b9b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.129+0000 7fccd3400700 1 -- 172.21.15.63:0/4219226030 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce002bc40 con 0x5558b72c52c0 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.129+0000 7fccd3400700 1 -- 172.21.15.63:0/4219226030 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcce0027070 con 0x5558b72c52c0 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.129+0000 7fccd3400700 1 -- 172.21.15.63:0/4219226030 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0022880 con 0x5558b72c52c0 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.130+0000 7fccfc645b80 1 -- 172.21.15.63:0/4219226030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 msgr2=0x5558b72d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.130+0000 7fccfc645b80 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72d0bf0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fcce000f040 tx=0x7fcce0017ab0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.130+0000 7fccfc645b80 1 -- 172.21.15.63:0/4219226030 shutdown_connections 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.130+0000 7fccfc645b80 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72d0bf0 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.130+0000 7fccfc645b80 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 0x5558b72bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.130+0000 7fccfc645b80 1 --2- 172.21.15.63:0/4219226030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 0x5558b72b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.130+0000 7fccfc645b80 1 -- 172.21.15.63:0/4219226030 >> 172.21.15.63:0/4219226030 conn(0x5558b71d5390 msgr2=0x5558b72bd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:25.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.130+0000 7fccfc645b80 1 -- 172.21.15.63:0/4219226030 shutdown_connections 2024-05-27T08:11:25.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.130+0000 7fccfc645b80 1 -- 172.21.15.63:0/4219226030 wait complete. 2024-05-27T08:11:25.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.131+0000 7fccfc645b80 1 -- start start 2024-05-27T08:11:25.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.131+0000 7fccfc645b80 1 Processor -- start 2024-05-27T08:11:25.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.132+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 0x5558b72c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.132+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 0x5558b72c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.132+0000 7fccfc645b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.132+0000 7fccfc645b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b0aa0 con 0x5558b72c52c0 2024-05-27T08:11:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.132+0000 7fccfc645b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b1f10 con 0x5558b72279f0 2024-05-27T08:11:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.132+0000 7fccfc645b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b1170 con 0x5558b713be40 2024-05-27T08:11:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.132+0000 7fccfc645b80 10 client.0 _ll_register_callbacks cb 0x5558b7182c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.133+0000 7fcce6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 0x5558b72c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.133+0000 7fcce7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.133+0000 7fcce6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 0x5558b72c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:49360/0 (socket says 172.21.15.63:49360) 2024-05-27T08:11:25.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.133+0000 7fcce6a00700 1 -- 172.21.15.63:0/1065878640 learned_addr learned my addr 172.21.15.63:0/1065878640 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:25.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.133+0000 7fcce6000700 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 0x5558b72c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.133+0000 7fcce7400700 1 -- 172.21.15.63:0/1065878640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 msgr2=0x5558b72c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:25.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.133+0000 7fcce7400700 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 0x5558b72c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.133+0000 7fcce7400700 1 -- 172.21.15.63:0/1065878640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 msgr2=0x5558b72c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:25.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.133+0000 7fcce7400700 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 0x5558b72c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.133+0000 7fcce7400700 1 -- 172.21.15.63:0/1065878640 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558b72c30d0 con 0x5558b72c52c0 2024-05-27T08:11:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.134+0000 7fcce7400700 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c8600 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fcce0018490 tx=0x7fcce0007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e66c44260317fe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.134+0000 7fccd0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:25.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.134+0000 7fccd0c00700 1 -- 172.21.15.63:0/1065878640 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce002bbc0 con 0x5558b72c52c0 2024-05-27T08:11:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.134+0000 7fccd0c00700 1 -- 172.21.15.63:0/1065878640 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7fcce00343f0 con 0x5558b72c52c0 2024-05-27T08:11:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.134+0000 7fccfc645b80 1 -- 172.21.15.63:0/1065878640 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce0027070 con 0x5558b72c52c0 2024-05-27T08:11:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.135+0000 7fccd0c00700 1 -- 172.21.15.63:0/1065878640 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcce0044050 con 0x5558b72c52c0 2024-05-27T08:11:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.135+0000 7fccfc645b80 10 client.22647 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.135+0000 7fccfc645b80 1 -- 172.21.15.63:0/1065878640 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5558b7346d90 con 0x5558b72c52c0 2024-05-27T08:11:25.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.135+0000 7fccd0c00700 1 -- 172.21.15.63:0/1065878640 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcce0027070 con 0x5558b72c52c0 2024-05-27T08:11:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.136+0000 7fccd0c00700 1 -- 172.21.15.63:0/1065878640 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcce0022580 con 0x5558b72c52c0 2024-05-27T08:11:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.136+0000 7fccfc645b80 10 client.22647 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.136+0000 7fccfc645b80 10 client.22647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:11:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.136+0000 7fccfc645b80 20 client.22647 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.136+0000 7fccfc645b80 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5558b73472d0 0x5558b73496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.136+0000 7fccfc645b80 4 client.22647 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:11:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.136+0000 7fccfc645b80 1 -- 172.21.15.63:0/1065878640 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558b71e4570 con 0x5558b73472d0 2024-05-27T08:11:25.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.137+0000 7fcce6a00700 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5558b73472d0 0x5558b73496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.138+0000 7fcce6a00700 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5558b73472d0 0x5558b73496f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7728757025cc467e server_cookie=18e9af00c81a747f in_seq=0 out_seq=0 2024-05-27T08:11:25.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.138+0000 7fccd0c00700 10 client.22647 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:11:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccd0c00700 1 -- 172.21.15.63:0/1065878640 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558b71e4570 con 0x5558b73472d0 2024-05-27T08:11:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccd0c00700 10 client.22647 handle_command_reply: tid=0 2024-05-27T08:11:25.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 client.22647 shutdown 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 -- 172.21.15.63:0/1065878640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 msgr2=0x5558b72c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c8600 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fcce0018490 tx=0x7fcce0007b60 comp rx=0 tx=0).stop 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 -- 172.21.15.63:0/1065878640 shutdown_connections 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5558b73472d0 0x5558b73496f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5558b72c52c0 0x5558b72c8600 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558b713be40 0x5558b72c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 --2- 172.21.15.63:0/1065878640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5558b72279f0 0x5558b72c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccd0c00700 0 client.22647 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 -- 172.21.15.63:0/1065878640 >> 172.21.15.63:0/1065878640 conn(0x5558b71d5390 msgr2=0x5558b72bd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 -- 172.21.15.63:0/1065878640 shutdown_connections 2024-05-27T08:11:25.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.139+0000 7fccfc645b80 1 -- 172.21.15.63:0/1065878640 wait complete. 2024-05-27T08:11:25.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.140+0000 7fccfc645b80 20 client.22647 trim_cache size 0 max 16384 2024-05-27T08:11:25.141 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:11:25.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.140+0000 7fcce7e00700 1 -- 172.21.15.63:0/1684230785 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcccc0b2a70 msgr2=0x7fcccc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:25.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.140+0000 7fcce7e00700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcccc0b2a70 0x7fcccc0b4f80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x5558b72b7a90 tx=0x7fcce0004b40 comp rx=0 tx=0).stop 2024-05-27T08:11:25.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.140+0000 7fcce7e00700 1 -- 172.21.15.63:0/1684230785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce8106610 msgr2=0x7fcce819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:25.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.140+0000 7fcce7e00700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce8106610 0x7fcce819cd90 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fccd800b580 tx=0x7fccd800f510 comp rx=0 tx=0).stop 2024-05-27T08:11:25.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.141+0000 7fcce7e00700 1 -- 172.21.15.63:0/1684230785 shutdown_connections 2024-05-27T08:11:25.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.141+0000 7fcce7e00700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcccc0b2a70 0x7fcccc0b4f80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.141+0000 7fcce7e00700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce810b890 0x7fcce81a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.141+0000 7fcce7e00700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcce8106610 0x7fcce819cd90 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.141+0000 7fcce7e00700 1 --2- 172.21.15.63:0/1684230785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcce810af40 0x7fcce819d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:25.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.141+0000 7fcce7e00700 1 -- 172.21.15.63:0/1684230785 >> 172.21.15.63:0/1684230785 conn(0x7fcce8102020 msgr2=0x7fcce810a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:25.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.141+0000 7fcce7e00700 1 -- 172.21.15.63:0/1684230785 shutdown_connections 2024-05-27T08:11:25.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:25.141+0000 7fcce7e00700 1 -- 172.21.15.63:0/1684230785 wait complete. 2024-05-27T08:11:25.929 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:25 smithi063 ceph-mon[31613]: pgmap v1201: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:11:26.006 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:11:26.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:25 smithi086 ceph-mon[34687]: pgmap v1201: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:11:26.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:25 smithi149 ceph-mon[35408]: pgmap v1201: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 8 op/s 2024-05-27T08:11:26.571 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:11:27.938 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:27 smithi063 ceph-mon[31613]: pgmap v1202: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.9 MiB/s wr, 17 op/s 2024-05-27T08:11:28.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:27 smithi086 ceph-mon[34687]: pgmap v1202: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.9 MiB/s wr, 17 op/s 2024-05-27T08:11:28.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:27 smithi149 ceph-mon[35408]: pgmap v1202: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.9 MiB/s wr, 17 op/s 2024-05-27T08:11:29.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.090+0000 7f1bb6600700 1 -- 172.21.15.63:0/3474746957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb0102f40 msgr2=0x7f1bb0103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:29.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.090+0000 7f1bb6600700 1 --2- 172.21.15.63:0/3474746957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb0102f40 0x7f1bb0103350 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400b0a0 tx=0x7f1ba400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.090+0000 7f1bb6600700 1 -- 172.21.15.63:0/3474746957 shutdown_connections 2024-05-27T08:11:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.090+0000 7f1bb6600700 1 --2- 172.21.15.63:0/3474746957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb0110af0 0x7f1bb0112f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.090+0000 7f1bb6600700 1 --2- 172.21.15.63:0/3474746957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb0102f40 0x7f1bb0103350 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.090+0000 7f1bb6600700 1 --2- 172.21.15.63:0/3474746957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bb0103890 0x7f1bb01105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.090+0000 7f1bb6600700 1 -- 172.21.15.63:0/3474746957 >> 172.21.15.63:0/3474746957 conn(0x7f1bb0078ab0 msgr2=0x7f1bb0078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.091+0000 7f1bb6600700 1 -- 172.21.15.63:0/3474746957 shutdown_connections 2024-05-27T08:11:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.091+0000 7f1bb6600700 1 -- 172.21.15.63:0/3474746957 wait complete. 2024-05-27T08:11:29.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.092+0000 7f1bb6600700 1 Processor -- start 2024-05-27T08:11:29.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.093+0000 7f1bb6600700 1 -- start start 2024-05-27T08:11:29.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.093+0000 7f1bb6600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb0102f40 0x7f1bb019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.093+0000 7f1bb6600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bb0103890 0x7f1bb019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.093+0000 7f1bb6600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb0110af0 0x7f1bb01a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.093+0000 7f1bb6600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0116390 con 0x7f1bb0102f40 2024-05-27T08:11:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.093+0000 7f1bb6600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0116210 con 0x7f1bb0110af0 2024-05-27T08:11:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.093+0000 7f1bb6600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0116510 con 0x7f1bb0103890 2024-05-27T08:11:29.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.093+0000 7f1bb5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb0102f40 0x7f1bb019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bb5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb0102f40 0x7f1bb019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35662/0 (socket says 172.21.15.63:35662) 2024-05-27T08:11:29.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bb5200700 1 -- 172.21.15.63:0/3604632326 learned_addr learned my addr 172.21.15.63:0/3604632326 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:29.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bb5c00700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb0110af0 0x7f1bb01a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bafe00700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bb0103890 0x7f1bb019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bb5200700 1 -- 172.21.15.63:0/3604632326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bb0103890 msgr2=0x7f1bb019d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:29.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bb5200700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bb0103890 0x7f1bb019d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bb5200700 1 -- 172.21.15.63:0/3604632326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb0110af0 msgr2=0x7f1bb01a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:29.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bb5200700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb0110af0 0x7f1bb01a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bb5200700 1 -- 172.21.15.63:0/3604632326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb01a1c50 con 0x7f1bb0102f40 2024-05-27T08:11:29.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bafe00700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bb0103890 0x7f1bb019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:11:29.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.094+0000 7f1bb5c00700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb0110af0 0x7f1bb01a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:29.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.095+0000 7f1bb5200700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb0102f40 0x7f1bb019cd80 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400b070 tx=0x7f1ba4004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.095+0000 7f1bad600700 1 -- 172.21.15.63:0/3604632326 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4021070 con 0x7f1bb0102f40 2024-05-27T08:11:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.095+0000 7f1bb6600700 1 -- 172.21.15.63:0/3604632326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb01a1ee0 con 0x7f1bb0102f40 2024-05-27T08:11:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.095+0000 7f1bad600700 1 -- 172.21.15.63:0/3604632326 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ba4007c80 con 0x7f1bb0102f40 2024-05-27T08:11:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.095+0000 7f1bad600700 1 -- 172.21.15.63:0/3604632326 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40047d0 con 0x7f1bb0102f40 2024-05-27T08:11:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.096+0000 7f1bb6600700 1 -- 172.21.15.63:0/3604632326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb01a23c0 con 0x7f1bb0102f40 2024-05-27T08:11:29.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.101+0000 7f1bad600700 1 -- 172.21.15.63:0/3604632326 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1ba4003a00 con 0x7f1bb0102f40 2024-05-27T08:11:29.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.103+0000 7f1bc6821b80 1 Processor -- start 2024-05-27T08:11:29.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.103+0000 7f1bc6821b80 1 -- start start 2024-05-27T08:11:29.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.103+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 0x55e858756750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.103+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8585d6e40 0x55e858756c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8587602c0 0x55e8587626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e858683e40 con 0x55e8585d6e40 2024-05-27T08:11:29.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8586739e0 con 0x55e8586c29f0 2024-05-27T08:11:29.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e858677a20 con 0x55e8587602c0 2024-05-27T08:11:29.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bad600700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b980b29f0 0x7f1b980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bad600700 1 -- 172.21.15.63:0/3604632326 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f1ba400f020 con 0x7f1bb0102f40 2024-05-27T08:11:29.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bb5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8585d6e40 0x55e858756c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bb5c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 0x55e858756750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bb5200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8585d6e40 0x55e858756c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35676/0 (socket says 172.21.15.63:35676) 2024-05-27T08:11:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bb5200700 1 -- 172.21.15.63:0/2144921190 learned_addr learned my addr 172.21.15.63:0/2144921190 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bafe00700 1 --2- 172.21.15.63:0/2144921190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8587602c0 0x55e8587626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.104+0000 7f1bb5c00700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b980b29f0 0x7f1b980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.105+0000 7f1bb5200700 1 -- 172.21.15.63:0/2144921190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8587602c0 msgr2=0x55e8587626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.105+0000 7f1bb5200700 1 --2- 172.21.15.63:0/2144921190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8587602c0 0x55e8587626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.105+0000 7f1bb5200700 1 -- 172.21.15.63:0/2144921190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 msgr2=0x55e858756750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.105+0000 7f1bb5200700 1 --2- 172.21.15.63:0/2144921190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 0x55e858756750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.105+0000 7f1bb5200700 1 -- 172.21.15.63:0/2144921190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85863c420 con 0x55e8585d6e40 2024-05-27T08:11:29.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.105+0000 7f1bb5c00700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b980b29f0 0x7f1b980b4f00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8005e10 tx=0x7f1ba800c7d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:29.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.105+0000 7f1bb5200700 1 --2- 172.21.15.63:0/2144921190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8585d6e40 0x55e858756c90 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40e4170 tx=0x7f1ba40757f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4b8f192c122d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:29.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.106+0000 7f1ba2000700 1 -- 172.21.15.63:0/2144921190 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40aab30 con 0x55e8585d6e40 2024-05-27T08:11:29.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.106+0000 7f1ba2000700 1 -- 172.21.15.63:0/2144921190 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ba40ab720 con 0x55e8585d6e40 2024-05-27T08:11:29.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.106+0000 7f1ba2000700 1 -- 172.21.15.63:0/2144921190 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40b3df0 con 0x55e8585d6e40 2024-05-27T08:11:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.107+0000 7f1bc6821b80 1 -- 172.21.15.63:0/2144921190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8585d6e40 msgr2=0x55e858756c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.107+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/2144921190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8585d6e40 0x55e858756c90 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40e4170 tx=0x7f1ba40757f0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.107+0000 7f1bc6821b80 1 -- 172.21.15.63:0/2144921190 shutdown_connections 2024-05-27T08:11:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.107+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/2144921190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8587602c0 0x55e8587626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.107+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/2144921190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 0x55e858756750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.107+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/2144921190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8585d6e40 0x55e858756c90 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.107+0000 7f1bc6821b80 1 -- 172.21.15.63:0/2144921190 >> 172.21.15.63:0/2144921190 conn(0x55e858670390 msgr2=0x55e858751710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.108+0000 7f1bc6821b80 1 -- 172.21.15.63:0/2144921190 shutdown_connections 2024-05-27T08:11:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.108+0000 7f1bc6821b80 1 -- 172.21.15.63:0/2144921190 wait complete. 2024-05-27T08:11:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.108+0000 7f1bc6821b80 1 -- start start 2024-05-27T08:11:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.109+0000 7f1bc6821b80 1 Processor -- start 2024-05-27T08:11:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8586c29f0 0x55e85875d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85875d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e85875ddf0 0x55e8587746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e858579fa0 con 0x55e8587602c0 2024-05-27T08:11:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e858614760 con 0x55e85875ddf0 2024-05-27T08:11:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e858762c20 con 0x55e8586c29f0 2024-05-27T08:11:29.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bc6821b80 10 client.0 _ll_register_callbacks cb 0x55e85861dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bafe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85875d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bafe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85875d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35684/0 (socket says 172.21.15.63:35684) 2024-05-27T08:11:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bafe00700 1 -- 172.21.15.63:0/589333017 learned_addr learned my addr 172.21.15.63:0/589333017 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bb5200700 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8586c29f0 0x55e85875d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.110+0000 7f1bb5c00700 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e85875ddf0 0x55e8587746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.111+0000 7f1bafe00700 1 -- 172.21.15.63:0/589333017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8586c29f0 msgr2=0x55e85875d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.111+0000 7f1bafe00700 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8586c29f0 0x55e85875d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.111+0000 7f1bafe00700 1 -- 172.21.15.63:0/589333017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e85875ddf0 msgr2=0x55e8587746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.111+0000 7f1bafe00700 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e85875ddf0 0x55e8587746a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.111+0000 7f1bafe00700 1 -- 172.21.15.63:0/589333017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8585c38d0 con 0x55e8587602c0 2024-05-27T08:11:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.111+0000 7f1bafe00700 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85875d8b0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c010850 tx=0x7f1b9c010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c68705aa7e9a05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:29.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.111+0000 7f1b96a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.112+0000 7f1b96a00700 1 -- 172.21.15.63:0/589333017 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c004830 con 0x55e8587602c0 2024-05-27T08:11:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.112+0000 7f1b96a00700 1 -- 172.21.15.63:0/589333017 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f1b9c0049d0 con 0x55e8587602c0 2024-05-27T08:11:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.112+0000 7f1bc6821b80 1 -- 172.21.15.63:0/589333017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ba40ab720 con 0x55e8587602c0 2024-05-27T08:11:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.112+0000 7f1b96a00700 1 -- 172.21.15.63:0/589333017 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1b9c023050 con 0x55e8587602c0 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.112+0000 7f1bc6821b80 10 client.22659 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.112+0000 7f1bc6821b80 1 -- 172.21.15.63:0/589333017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e858644060 con 0x55e8587602c0 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.113+0000 7f1b96a00700 1 -- 172.21.15.63:0/589333017 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ba40ab720 con 0x55e8587602c0 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.113+0000 7f1b96a00700 1 -- 172.21.15.63:0/589333017 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1b9c018860 con 0x55e8587602c0 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.113+0000 7f1bc6821b80 10 client.22659 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.113+0000 7f1bc6821b80 10 client.22659 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.113+0000 7f1bc6821b80 20 client.22659 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.113+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e858778950 0x55e85877adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.113+0000 7f1bc6821b80 4 client.22659 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.113+0000 7f1bc6821b80 1 -- 172.21.15.63:0/589333017 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e858685d90 con 0x55e858778950 2024-05-27T08:11:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.114+0000 7f1bb5200700 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e858778950 0x55e85877adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.115+0000 7f1bb5200700 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e858778950 0x55e85877adc0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cafea6f57d0f442d server_cookie=7233dd898d5d23a2 in_seq=0 out_seq=0 2024-05-27T08:11:29.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.115+0000 7f1b96a00700 10 client.22659 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:11:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1b96a00700 1 -- 172.21.15.63:0/589333017 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e858685d90 con 0x55e858778950 2024-05-27T08:11:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1b96a00700 10 client.22659 handle_command_reply: tid=0 2024-05-27T08:11:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1bc6821b80 1 client.22659 shutdown 2024-05-27T08:11:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1bc6821b80 1 -- 172.21.15.63:0/589333017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 msgr2=0x55e85875d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85875d8b0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c010850 tx=0x7f1b9c010b60 comp rx=0 tx=0).stop 2024-05-27T08:11:29.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1bc6821b80 1 -- 172.21.15.63:0/589333017 shutdown_connections 2024-05-27T08:11:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e85875ddf0 0x55e8587746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8586c29f0 0x55e85875d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e858778950 0x55e85877adc0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/589333017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85875d8b0 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1bc6821b80 1 -- 172.21.15.63:0/589333017 >> 172.21.15.63:0/589333017 conn(0x55e858670390 msgr2=0x55e8587513a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.117+0000 7f1b96a00700 0 client.22659 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:11:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.118+0000 7f1bc6821b80 1 -- 172.21.15.63:0/589333017 shutdown_connections 2024-05-27T08:11:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.118+0000 7f1bc6821b80 1 -- 172.21.15.63:0/589333017 wait complete. 2024-05-27T08:11:29.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.118+0000 7f1bc6821b80 20 client.22659 trim_cache size 0 max 16384 2024-05-27T08:11:29.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.136+0000 7f1bc6821b80 1 Processor -- start 2024-05-27T08:11:29.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.136+0000 7f1bc6821b80 1 -- start start 2024-05-27T08:11:29.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.137+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8586c29f0 0x55e858751740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.137+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e85875ddf0 0x55e858774490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.137+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8587602c0 0x55e858755830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.137+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01b08e0 con 0x55e85875ddf0 2024-05-27T08:11:29.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.137+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01ae170 con 0x55e8587602c0 2024-05-27T08:11:29.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.137+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01abfe0 con 0x55e8586c29f0 2024-05-27T08:11:29.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.137+0000 7f1bafe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e85875ddf0 0x55e858774490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.137+0000 7f1bafe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e85875ddf0 0x55e858774490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35698/0 (socket says 172.21.15.63:35698) 2024-05-27T08:11:29.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.137+0000 7f1bafe00700 1 -- 172.21.15.63:0/3081318357 learned_addr learned my addr 172.21.15.63:0/3081318357 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:29.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.138+0000 7f1bb5200700 1 --2- 172.21.15.63:0/3081318357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8586c29f0 0x55e858751740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.138+0000 7f1bb5c00700 1 --2- 172.21.15.63:0/3081318357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8587602c0 0x55e858755830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.138+0000 7f1bafe00700 1 -- 172.21.15.63:0/3081318357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8586c29f0 msgr2=0x55e858751740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:29.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.138+0000 7f1bafe00700 1 --2- 172.21.15.63:0/3081318357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8586c29f0 0x55e858751740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.138+0000 7f1bafe00700 1 -- 172.21.15.63:0/3081318357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8587602c0 msgr2=0x55e858755830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:29.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.138+0000 7f1bafe00700 1 --2- 172.21.15.63:0/3081318357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8587602c0 0x55e858755830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.138+0000 7f1bafe00700 1 -- 172.21.15.63:0/3081318357 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e858776320 con 0x55e85875ddf0 2024-05-27T08:11:29.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.138+0000 7f1bafe00700 1 --2- 172.21.15.63:0/3081318357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e85875ddf0 0x55e858774490 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000c00 tx=0x7f1b9c0186a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838ad1000f15bc79 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:29.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.139+0000 7f1ba1600700 1 -- 172.21.15.63:0/3081318357 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c02b020 con 0x55e85875ddf0 2024-05-27T08:11:29.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.139+0000 7f1ba1600700 1 -- 172.21.15.63:0/3081318357 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b9c0054d0 con 0x55e85875ddf0 2024-05-27T08:11:29.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.139+0000 7f1ba1600700 1 -- 172.21.15.63:0/3081318357 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c004ad0 con 0x55e85875ddf0 2024-05-27T08:11:29.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.140+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3081318357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e85875ddf0 msgr2=0x55e858774490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:29.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.140+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/3081318357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e85875ddf0 0x55e858774490 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000c00 tx=0x7f1b9c0186a0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.140+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3081318357 shutdown_connections 2024-05-27T08:11:29.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.140+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/3081318357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8587602c0 0x55e858755830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.140+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/3081318357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e8586c29f0 0x55e858751740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.140+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/3081318357 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e85875ddf0 0x55e858774490 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.140+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3081318357 >> 172.21.15.63:0/3081318357 conn(0x55e858670390 msgr2=0x55e858755120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:29.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.140+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3081318357 shutdown_connections 2024-05-27T08:11:29.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.140+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3081318357 wait complete. 2024-05-27T08:11:29.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.141+0000 7f1bc6821b80 1 -- start start 2024-05-27T08:11:29.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.141+0000 7f1bc6821b80 1 Processor -- start 2024-05-27T08:11:29.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 0x55e85876b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e85875ddf0 0x55e85876bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bc6821b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85876c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01ac630 con 0x55e8587602c0 2024-05-27T08:11:29.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01adaa0 con 0x55e8586c29f0 2024-05-27T08:11:29.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bc6821b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb01acd00 con 0x55e85875ddf0 2024-05-27T08:11:29.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bc6821b80 10 client.0 _ll_register_callbacks cb 0x55e85861dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:29.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bb5c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85876c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bb5c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85876c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35702/0 (socket says 172.21.15.63:35702) 2024-05-27T08:11:29.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bb5c00700 1 -- 172.21.15.63:0/3564048587 learned_addr learned my addr 172.21.15.63:0/3564048587 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:29.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.142+0000 7f1bb5200700 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 0x55e85876b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.143+0000 7f1bafe00700 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e85875ddf0 0x55e85876bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.143+0000 7f1bb5c00700 1 -- 172.21.15.63:0/3564048587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e85875ddf0 msgr2=0x55e85876bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.143+0000 7f1bb5c00700 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e85875ddf0 0x55e85876bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.143+0000 7f1bb5c00700 1 -- 172.21.15.63:0/3564048587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 msgr2=0x55e85876b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.143+0000 7f1bb5c00700 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 0x55e85876b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.143+0000 7f1bb5c00700 1 -- 172.21.15.63:0/3564048587 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8587e1510 con 0x55e8587602c0 2024-05-27T08:11:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.143+0000 7f1bb5200700 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 0x55e85876b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.143+0000 7f1bb5c00700 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85876c0c0 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8011c20 tx=0x7f1ba801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615745a5cbdf7e6e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.143+0000 7f1b96a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.144+0000 7f1b96a00700 1 -- 172.21.15.63:0/3564048587 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8020ac0 con 0x55e8587602c0 2024-05-27T08:11:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.144+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3564048587 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b9c0054d0 con 0x55e8587602c0 2024-05-27T08:11:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.144+0000 7f1b96a00700 1 -- 172.21.15.63:0/3564048587 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f1ba8020c60 con 0x55e8587602c0 2024-05-27T08:11:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.144+0000 7f1b96a00700 1 -- 172.21.15.63:0/3564048587 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1ba8037050 con 0x55e8587602c0 2024-05-27T08:11:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.144+0000 7f1bc6821b80 10 client.22665 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.144+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3564048587 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e8587e27c0 con 0x55e8587602c0 2024-05-27T08:11:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.144+0000 7f1b96a00700 1 -- 172.21.15.63:0/3564048587 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b9c0054d0 con 0x55e8587602c0 2024-05-27T08:11:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.145+0000 7f1b96a00700 1 -- 172.21.15.63:0/3564048587 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1ba8005430 con 0x55e8587602c0 2024-05-27T08:11:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.145+0000 7f1bc6821b80 10 client.22665 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.145+0000 7f1bc6821b80 10 client.22665 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:11:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.145+0000 7f1bc6821b80 20 client.22665 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.145+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e8587e2ed0 0x55e8587e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.145+0000 7f1bc6821b80 4 client.22665 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:11:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.145+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3564048587 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e85867f570 con 0x55e8587e2ed0 2024-05-27T08:11:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.146+0000 7f1bb5200700 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e8587e2ed0 0x55e8587e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:29.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.147+0000 7f1bb5200700 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e8587e2ed0 0x55e8587e52f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=907f9fc013a5cd43 server_cookie=1c2fa23305c55a4e in_seq=0 out_seq=0 2024-05-27T08:11:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.147+0000 7f1b96a00700 10 client.22665 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:11:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.147+0000 7f1b96a00700 1 -- 172.21.15.63:0/3564048587 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e85867f570 con 0x55e8587e2ed0 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.147+0000 7f1b96a00700 10 client.22665 handle_command_reply: tid=0 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.147+0000 7f1bc6821b80 1 client.22665 shutdown 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3564048587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 msgr2=0x55e85876c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85876c0c0 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8011c20 tx=0x7f1ba801b000 comp rx=0 tx=0).stop 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3564048587 shutdown_connections 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55e8587e2ed0 0x55e8587e52f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8587602c0 0x55e85876c0c0 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8586c29f0 0x55e85876b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 1 --2- 172.21.15.63:0/3564048587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e85875ddf0 0x55e85876bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3564048587 >> 172.21.15.63:0/3564048587 conn(0x55e858670390 msgr2=0x55e85875bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1b96a00700 0 client.22665 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3564048587 shutdown_connections 2024-05-27T08:11:29.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 1 -- 172.21.15.63:0/3564048587 wait complete. 2024-05-27T08:11:29.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.148+0000 7f1bc6821b80 20 client.22665 trim_cache size 0 max 16384 2024-05-27T08:11:29.150 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:11:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.149+0000 7f1bb6600700 1 -- 172.21.15.63:0/3604632326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b980b29f0 msgr2=0x7f1b980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.149+0000 7f1bb6600700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b980b29f0 0x7f1b980b4f00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8005e10 tx=0x7f1ba800c7d0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.149+0000 7f1bb6600700 1 -- 172.21.15.63:0/3604632326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb0102f40 msgr2=0x7f1bb019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.149+0000 7f1bb6600700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb0102f40 0x7f1bb019cd80 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400b070 tx=0x7f1ba4004e00 comp rx=0 tx=0).stop 2024-05-27T08:11:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.149+0000 7f1bb6600700 1 -- 172.21.15.63:0/3604632326 shutdown_connections 2024-05-27T08:11:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.149+0000 7f1bb6600700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b980b29f0 0x7f1b980b4f00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.149+0000 7f1bb6600700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bb0110af0 0x7f1bb01a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.149+0000 7f1bb6600700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb0102f40 0x7f1bb019cd80 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.149+0000 7f1bb6600700 1 --2- 172.21.15.63:0/3604632326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1bb0103890 0x7f1bb019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.149+0000 7f1bb6600700 1 -- 172.21.15.63:0/3604632326 >> 172.21.15.63:0/3604632326 conn(0x7f1bb0078ab0 msgr2=0x7f1bb0111f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.150+0000 7f1bb6600700 1 -- 172.21.15.63:0/3604632326 shutdown_connections 2024-05-27T08:11:29.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:29.150+0000 7f1bb6600700 1 -- 172.21.15.63:0/3604632326 wait complete. 2024-05-27T08:11:30.204 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:29 smithi063 ceph-mon[31613]: pgmap v1203: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:11:30.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:29 smithi086 ceph-mon[34687]: pgmap v1203: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:11:30.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:29 smithi149 ceph-mon[35408]: pgmap v1203: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:11:30.268 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:11:30.794 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:11:32.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:31 smithi086 ceph-mon[34687]: pgmap v1204: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:11:32.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:31 smithi149 ceph-mon[35408]: pgmap v1204: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:11:32.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:31 smithi063 ceph-mon[31613]: pgmap v1204: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:11:33.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.650+0000 7eff73200700 1 -- 172.21.15.63:0/2654924584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff74075b50 msgr2=0x7eff74075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:33.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.650+0000 7eff73200700 1 --2- 172.21.15.63:0/2654924584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff74075b50 0x7eff74075f60 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7eff64009990 tx=0x7eff64009ca0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.650+0000 7eff73200700 1 -- 172.21.15.63:0/2654924584 shutdown_connections 2024-05-27T08:11:33.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.650+0000 7eff73200700 1 --2- 172.21.15.63:0/2654924584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff74115170 0x7eff74117590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.650+0000 7eff73200700 1 --2- 172.21.15.63:0/2654924584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff74075b50 0x7eff74075f60 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.650+0000 7eff73200700 1 --2- 172.21.15.63:0/2654924584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff740764a0 0x7eff74114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.650+0000 7eff73200700 1 -- 172.21.15.63:0/2654924584 >> 172.21.15.63:0/2654924584 conn(0x7eff74079d00 msgr2=0x7eff7407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:33.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.650+0000 7eff73200700 1 -- 172.21.15.63:0/2654924584 shutdown_connections 2024-05-27T08:11:33.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.650+0000 7eff73200700 1 -- 172.21.15.63:0/2654924584 wait complete. 2024-05-27T08:11:33.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.652+0000 7eff73200700 1 Processor -- start 2024-05-27T08:11:33.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.652+0000 7eff73200700 1 -- start start 2024-05-27T08:11:33.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.652+0000 7eff73200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff74075b50 0x7eff741a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.652+0000 7eff73200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff740764a0 0x7eff741a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.652+0000 7eff73200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff74115170 0x7eff741a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.652+0000 7eff73200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff7411a780 con 0x7eff740764a0 2024-05-27T08:11:33.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.652+0000 7eff73200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff7411a600 con 0x7eff74115170 2024-05-27T08:11:33.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.652+0000 7eff73200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7eff7411a900 con 0x7eff74075b50 2024-05-27T08:11:33.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.653+0000 7eff71400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff740764a0 0x7eff741a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.653+0000 7eff71400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff740764a0 0x7eff741a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35728/0 (socket says 172.21.15.63:35728) 2024-05-27T08:11:33.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.653+0000 7eff71400700 1 -- 172.21.15.63:0/3343698382 learned_addr learned my addr 172.21.15.63:0/3343698382 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.653+0000 7eff71e00700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff74075b50 0x7eff741a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.653+0000 7eff72800700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff74115170 0x7eff741a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff71400700 1 -- 172.21.15.63:0/3343698382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff74075b50 msgr2=0x7eff741a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff71400700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff74075b50 0x7eff741a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff71400700 1 -- 172.21.15.63:0/3343698382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff74115170 msgr2=0x7eff741a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff71400700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff74115170 0x7eff741a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff71400700 1 -- 172.21.15.63:0/3343698382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff741a6080 con 0x7eff740764a0 2024-05-27T08:11:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff72800700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff74115170 0x7eff741a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:33.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff71400700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff740764a0 0x7eff741a1750 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7eff6000d950 tx=0x7eff6000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff6aa00700 1 -- 172.21.15.63:0/3343698382 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60016070 con 0x7eff740764a0 2024-05-27T08:11:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff6aa00700 1 -- 172.21.15.63:0/3343698382 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff60009dd0 con 0x7eff740764a0 2024-05-27T08:11:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff73200700 1 -- 172.21.15.63:0/3343698382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff741a6370 con 0x7eff740764a0 2024-05-27T08:11:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.654+0000 7eff6aa00700 1 -- 172.21.15.63:0/3343698382 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6000f3c0 con 0x7eff740764a0 2024-05-27T08:11:33.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.655+0000 7eff73200700 1 -- 172.21.15.63:0/3343698382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff741adc90 con 0x7eff740764a0 2024-05-27T08:11:33.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.660+0000 7eff6aa00700 1 -- 172.21.15.63:0/3343698382 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7eff60019070 con 0x7eff740764a0 2024-05-27T08:11:33.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.660+0000 7eff87539b80 1 Processor -- start 2024-05-27T08:11:33.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.660+0000 7eff87539b80 1 -- start start 2024-05-27T08:11:33.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.661+0000 7eff87539b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67c2be40 0x559c67da9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.661+0000 7eff87539b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 0x559c67db4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.661+0000 7eff87539b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.661+0000 7eff87539b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559c67cd8e40 con 0x559c67daa820 2024-05-27T08:11:33.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.661+0000 7eff87539b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c67cc89e0 con 0x559c67daa410 2024-05-27T08:11:33.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.661+0000 7eff87539b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559c67ccca20 con 0x559c67c2be40 2024-05-27T08:11:33.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.661+0000 7eff71400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.661+0000 7eff71400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35738/0 (socket says 172.21.15.63:35738) 2024-05-27T08:11:33.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.662+0000 7eff71400700 1 -- 172.21.15.63:0/2113916320 learned_addr learned my addr 172.21.15.63:0/2113916320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:33.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.662+0000 7eff72800700 1 --2- 172.21.15.63:0/2113916320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67c2be40 0x559c67da9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.662+0000 7eff71e00700 1 --2- 172.21.15.63:0/2113916320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 0x559c67db4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.662+0000 7eff71400700 1 -- 172.21.15.63:0/2113916320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67c2be40 msgr2=0x559c67da9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:33.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.662+0000 7eff71400700 1 --2- 172.21.15.63:0/2113916320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67c2be40 0x559c67da9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.662+0000 7eff71400700 1 -- 172.21.15.63:0/2113916320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 msgr2=0x559c67db4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:33.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.662+0000 7eff71400700 1 --2- 172.21.15.63:0/2113916320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 0x559c67db4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.662+0000 7eff71400700 1 -- 172.21.15.63:0/2113916320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c67c91420 con 0x559c67daa820 2024-05-27T08:11:33.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.662+0000 7eff6aa00700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff580b29f0 0x7eff580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.662+0000 7eff71e00700 1 --2- 172.21.15.63:0/2113916320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 0x559c67db4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:33.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.663+0000 7eff72800700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff580b29f0 0x7eff580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.663+0000 7eff6aa00700 1 -- 172.21.15.63:0/3343698382 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7eff600e26b0 con 0x7eff740764a0 2024-05-27T08:11:33.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.663+0000 7eff71400700 1 --2- 172.21.15.63:0/2113916320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db7020 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7eff600df920 tx=0x7eff600e8ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c91099079d42169 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:33.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.663+0000 7eff5f400700 1 -- 172.21.15.63:0/2113916320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60020030 con 0x559c67daa820 2024-05-27T08:11:33.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.663+0000 7eff5f400700 1 -- 172.21.15.63:0/2113916320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff600a9080 con 0x559c67daa820 2024-05-27T08:11:33.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.663+0000 7eff5f400700 1 -- 172.21.15.63:0/2113916320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600a9c70 con 0x559c67daa820 2024-05-27T08:11:33.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.665+0000 7eff87539b80 1 -- 172.21.15.63:0/2113916320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 msgr2=0x559c67db7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:33.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.665+0000 7eff72800700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff580b29f0 0x7eff580b4f00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x559c67da6690 tx=0x7eff6c009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:33.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.665+0000 7eff87539b80 1 --2- 172.21.15.63:0/2113916320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db7020 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7eff600df920 tx=0x7eff600e8ed0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.665+0000 7eff87539b80 1 -- 172.21.15.63:0/2113916320 shutdown_connections 2024-05-27T08:11:33.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.665+0000 7eff87539b80 1 --2- 172.21.15.63:0/2113916320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db7020 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.665+0000 7eff87539b80 1 --2- 172.21.15.63:0/2113916320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67c2be40 0x559c67da9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.665+0000 7eff87539b80 1 --2- 172.21.15.63:0/2113916320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 0x559c67db4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.665+0000 7eff87539b80 1 -- 172.21.15.63:0/2113916320 >> 172.21.15.63:0/2113916320 conn(0x559c67c8bfc0 msgr2=0x559c67da5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:33.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.665+0000 7eff87539b80 1 -- 172.21.15.63:0/2113916320 shutdown_connections 2024-05-27T08:11:33.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.665+0000 7eff87539b80 1 -- 172.21.15.63:0/2113916320 wait complete. 2024-05-27T08:11:33.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.666+0000 7eff87539b80 1 -- start start 2024-05-27T08:11:33.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.666+0000 7eff87539b80 1 Processor -- start 2024-05-27T08:11:33.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.667+0000 7eff87539b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67c2be40 0x559c67db4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.667+0000 7eff87539b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 0x559c67db4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.667+0000 7eff87539b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa820 0x559c67db4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.667+0000 7eff87539b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559c67bcefa0 con 0x559c67c2be40 2024-05-27T08:11:33.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.667+0000 7eff87539b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c67c69760 con 0x559c67daa410 2024-05-27T08:11:33.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.667+0000 7eff87539b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559c67db7990 con 0x559c67daa820 2024-05-27T08:11:33.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.667+0000 7eff87539b80 10 client.0 _ll_register_callbacks cb 0x559c67c72c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:33.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.667+0000 7eff71e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67c2be40 0x559c67db4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.668+0000 7eff71400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 0x559c67db4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.668+0000 7eff71e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67c2be40 0x559c67db4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35740/0 (socket says 172.21.15.63:35740) 2024-05-27T08:11:33.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.668+0000 7eff71e00700 1 -- 172.21.15.63:0/3721286657 learned_addr learned my addr 172.21.15.63:0/3721286657 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:33.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.668+0000 7eff72800700 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa820 0x559c67db4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.668+0000 7eff71e00700 1 -- 172.21.15.63:0/3721286657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa820 msgr2=0x559c67db4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:33.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.668+0000 7eff71e00700 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa820 0x559c67db4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.668+0000 7eff71e00700 1 -- 172.21.15.63:0/3721286657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 msgr2=0x559c67db4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:33.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.668+0000 7eff71e00700 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 0x559c67db4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.668+0000 7eff71e00700 1 -- 172.21.15.63:0/3721286657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c67c188d0 con 0x559c67c2be40 2024-05-27T08:11:33.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.669+0000 7eff71e00700 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67c2be40 0x559c67db4510 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7eff64009990 tx=0x7eff64005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40acccbe1eb9b1dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:33.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.669+0000 7eff53e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:33.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.669+0000 7eff53e00700 1 -- 172.21.15.63:0/3721286657 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64003bd0 con 0x559c67c2be40 2024-05-27T08:11:33.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.669+0000 7eff53e00700 1 -- 172.21.15.63:0/3721286657 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7eff64003d70 con 0x559c67c2be40 2024-05-27T08:11:33.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.669+0000 7eff87539b80 1 -- 172.21.15.63:0/3721286657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff600a9080 con 0x559c67c2be40 2024-05-27T08:11:33.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.670+0000 7eff53e00700 1 -- 172.21.15.63:0/3721286657 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff64030050 con 0x559c67c2be40 2024-05-27T08:11:33.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.670+0000 7eff87539b80 10 client.22677 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:33.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.670+0000 7eff87539b80 1 -- 172.21.15.63:0/3721286657 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559c67c99060 con 0x559c67c2be40 2024-05-27T08:11:33.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.670+0000 7eff53e00700 1 -- 172.21.15.63:0/3721286657 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff600a9080 con 0x559c67c2be40 2024-05-27T08:11:33.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.671+0000 7eff53e00700 1 -- 172.21.15.63:0/3721286657 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7eff6400f790 con 0x559c67c2be40 2024-05-27T08:11:33.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.671+0000 7eff87539b80 10 client.22677 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:33.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.671+0000 7eff87539b80 10 client.22677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:33.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.671+0000 7eff87539b80 20 client.22677 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:33.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.671+0000 7eff87539b80 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c67dc4920 0x559c67dc6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.671+0000 7eff87539b80 4 client.22677 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:11:33.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.671+0000 7eff87539b80 1 -- 172.21.15.63:0/3721286657 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c67cdad90 con 0x559c67dc4920 2024-05-27T08:11:33.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.672+0000 7eff71400700 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c67dc4920 0x559c67dc6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.673+0000 7eff71400700 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c67dc4920 0x559c67dc6d80 crc :-1 s=READY pgs=674 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787a0068417b5b88 server_cookie=10b5b571d0601bbf in_seq=0 out_seq=0 2024-05-27T08:11:33.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.673+0000 7eff53e00700 10 client.22677 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:33.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff53e00700 1 -- 172.21.15.63:0/3721286657 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559c67cdad90 con 0x559c67dc4920 2024-05-27T08:11:33.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff53e00700 10 client.22677 handle_command_reply: tid=0 2024-05-27T08:11:33.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff87539b80 1 client.22677 shutdown 2024-05-27T08:11:33.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff87539b80 1 -- 172.21.15.63:0/3721286657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67c2be40 msgr2=0x559c67db4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:33.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff87539b80 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67c2be40 0x559c67db4510 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7eff64009990 tx=0x7eff64005800 comp rx=0 tx=0).stop 2024-05-27T08:11:33.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff87539b80 1 -- 172.21.15.63:0/3721286657 shutdown_connections 2024-05-27T08:11:33.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff87539b80 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c67dc4920 0x559c67dc6d80 crc :-1 s=READY pgs=674 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff87539b80 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa820 0x559c67db4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff87539b80 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67c2be40 0x559c67db4510 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff87539b80 1 --2- 172.21.15.63:0/3721286657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67daa410 0x559c67db4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff53e00700 0 client.22677 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:33.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.675+0000 7eff87539b80 1 -- 172.21.15.63:0/3721286657 >> 172.21.15.63:0/3721286657 conn(0x559c67c8bfc0 msgr2=0x559c67da4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:33.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.676+0000 7eff87539b80 1 -- 172.21.15.63:0/3721286657 shutdown_connections 2024-05-27T08:11:33.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.676+0000 7eff87539b80 1 -- 172.21.15.63:0/3721286657 wait complete. 2024-05-27T08:11:33.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.676+0000 7eff87539b80 20 client.22677 trim_cache size 0 max 16384 2024-05-27T08:11:33.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.695+0000 7eff87539b80 1 Processor -- start 2024-05-27T08:11:33.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.695+0000 7eff87539b80 1 -- start start 2024-05-27T08:11:33.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.696+0000 7eff87539b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 0x559c67db6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.696+0000 7eff87539b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa410 0x559c67dc0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.696+0000 7eff87539b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa820 0x559c67db3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.696+0000 7eff87539b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff741b9260 con 0x559c67daa410 2024-05-27T08:11:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.696+0000 7eff87539b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff741b6af0 con 0x559c67c2be40 2024-05-27T08:11:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.696+0000 7eff87539b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7eff741b4960 con 0x559c67daa820 2024-05-27T08:11:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.696+0000 7eff71400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa410 0x559c67dc0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.696+0000 7eff71400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa410 0x559c67dc0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35746/0 (socket says 172.21.15.63:35746) 2024-05-27T08:11:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.696+0000 7eff71400700 1 -- 172.21.15.63:0/232966048 learned_addr learned my addr 172.21.15.63:0/232966048 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:33.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.696+0000 7eff71e00700 1 --2- 172.21.15.63:0/232966048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 0x559c67db6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.697+0000 7eff72800700 1 --2- 172.21.15.63:0/232966048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa820 0x559c67db3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.697+0000 7eff71400700 1 -- 172.21.15.63:0/232966048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa820 msgr2=0x559c67db3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:33.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.697+0000 7eff71400700 1 --2- 172.21.15.63:0/232966048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa820 0x559c67db3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.697+0000 7eff71400700 1 -- 172.21.15.63:0/232966048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 msgr2=0x559c67db6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:33.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.697+0000 7eff71400700 1 --2- 172.21.15.63:0/232966048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 0x559c67db6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.697+0000 7eff71400700 1 -- 172.21.15.63:0/232966048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c67db6db0 con 0x559c67daa410 2024-05-27T08:11:33.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.697+0000 7eff71e00700 1 --2- 172.21.15.63:0/232966048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 0x559c67db6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:33.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.697+0000 7eff71400700 1 --2- 172.21.15.63:0/232966048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa410 0x559c67dc0a60 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b2f0 tx=0x7eff600e2930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948e954ee7a0f9c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:33.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.697+0000 7eff5ea00700 1 -- 172.21.15.63:0/232966048 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600c3070 con 0x559c67daa410 2024-05-27T08:11:33.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.698+0000 7eff5ea00700 1 -- 172.21.15.63:0/232966048 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff600a9080 con 0x559c67daa410 2024-05-27T08:11:33.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.698+0000 7eff5ea00700 1 -- 172.21.15.63:0/232966048 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600a9570 con 0x559c67daa410 2024-05-27T08:11:33.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.698+0000 7eff87539b80 1 -- 172.21.15.63:0/232966048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa410 msgr2=0x559c67dc0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:33.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.698+0000 7eff87539b80 1 --2- 172.21.15.63:0/232966048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa410 0x559c67dc0a60 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7eff6000b2f0 tx=0x7eff600e2930 comp rx=0 tx=0).stop 2024-05-27T08:11:33.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.699+0000 7eff87539b80 1 -- 172.21.15.63:0/232966048 shutdown_connections 2024-05-27T08:11:33.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.699+0000 7eff87539b80 1 --2- 172.21.15.63:0/232966048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa820 0x559c67db3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.699+0000 7eff87539b80 1 --2- 172.21.15.63:0/232966048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 0x559c67db6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.699+0000 7eff87539b80 1 --2- 172.21.15.63:0/232966048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa410 0x559c67dc0a60 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.699+0000 7eff87539b80 1 -- 172.21.15.63:0/232966048 >> 172.21.15.63:0/232966048 conn(0x559c67c8bfc0 msgr2=0x559c67db2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:33.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.699+0000 7eff87539b80 1 -- 172.21.15.63:0/232966048 shutdown_connections 2024-05-27T08:11:33.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.699+0000 7eff87539b80 1 -- 172.21.15.63:0/232966048 wait complete. 2024-05-27T08:11:33.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.699+0000 7eff87539b80 1 -- start start 2024-05-27T08:11:33.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.700+0000 7eff87539b80 1 Processor -- start 2024-05-27T08:11:33.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.700+0000 7eff87539b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 0x559c67db8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff87539b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa410 0x559c67db8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff87539b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff87539b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff741b4fb0 con 0x559c67daa820 2024-05-27T08:11:33.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff87539b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff741b6420 con 0x559c67c2be40 2024-05-27T08:11:33.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff87539b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7eff741b5680 con 0x559c67daa410 2024-05-27T08:11:33.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff87539b80 10 client.0 _ll_register_callbacks cb 0x559c67c72c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:33.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff72800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff72800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35762/0 (socket says 172.21.15.63:35762) 2024-05-27T08:11:33.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff72800700 1 -- 172.21.15.63:0/2632625218 learned_addr learned my addr 172.21.15.63:0/2632625218 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:33.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff71e00700 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 0x559c67db8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.701+0000 7eff71400700 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa410 0x559c67db8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.702+0000 7eff72800700 1 -- 172.21.15.63:0/2632625218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa410 msgr2=0x559c67db8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:33.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.702+0000 7eff72800700 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa410 0x559c67db8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.702+0000 7eff72800700 1 -- 172.21.15.63:0/2632625218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 msgr2=0x559c67db8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:33.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.702+0000 7eff72800700 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 0x559c67db8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.702+0000 7eff72800700 1 -- 172.21.15.63:0/2632625218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c67e2e460 con 0x559c67daa820 2024-05-27T08:11:33.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.702+0000 7eff72800700 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db94d0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7eff6c018ea0 tx=0x7eff6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f42271a032355e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.702+0000 7eff53e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.703+0000 7eff53e00700 1 -- 172.21.15.63:0/2632625218 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c02a070 con 0x559c67daa820 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.703+0000 7eff87539b80 1 -- 172.21.15.63:0/2632625218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff600a9080 con 0x559c67daa820 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.703+0000 7eff53e00700 1 -- 172.21.15.63:0/2632625218 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7eff6c016cc0 con 0x559c67daa820 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.703+0000 7eff53e00700 1 -- 172.21.15.63:0/2632625218 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff6c01e040 con 0x559c67daa820 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.703+0000 7eff87539b80 10 client.22683 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.703+0000 7eff87539b80 1 -- 172.21.15.63:0/2632625218 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559c67e2f720 con 0x559c67daa820 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.703+0000 7eff53e00700 1 -- 172.21.15.63:0/2632625218 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff600a9080 con 0x559c67daa820 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.704+0000 7eff53e00700 1 -- 172.21.15.63:0/2632625218 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7eff6c024040 con 0x559c67daa820 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.704+0000 7eff87539b80 10 client.22683 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.704+0000 7eff87539b80 10 client.22683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.704+0000 7eff87539b80 20 client.22683 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.704+0000 7eff87539b80 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c67e2fe00 0x559c67e32220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:33.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.704+0000 7eff87539b80 4 client.22683 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:11:33.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.704+0000 7eff87539b80 1 -- 172.21.15.63:0/2632625218 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c67cd4570 con 0x559c67e2fe00 2024-05-27T08:11:33.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.705+0000 7eff71e00700 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c67e2fe00 0x559c67e32220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:33.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.706+0000 7eff71e00700 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c67e2fe00 0x559c67e32220 crc :-1 s=READY pgs=675 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24dba3d90c8b67c server_cookie=ecc838d6e6fc03c in_seq=0 out_seq=0 2024-05-27T08:11:33.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.706+0000 7eff53e00700 10 client.22683 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:33.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.706+0000 7eff53e00700 1 -- 172.21.15.63:0/2632625218 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c67cd4570 con 0x559c67e2fe00 2024-05-27T08:11:33.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.706+0000 7eff53e00700 10 client.22683 handle_command_reply: tid=0 2024-05-27T08:11:33.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.706+0000 7eff87539b80 1 client.22683 shutdown 2024-05-27T08:11:33.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.706+0000 7eff87539b80 1 -- 172.21.15.63:0/2632625218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 msgr2=0x559c67db94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:33.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.706+0000 7eff87539b80 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db94d0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7eff6c018ea0 tx=0x7eff6c004ad0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.707+0000 7eff87539b80 1 -- 172.21.15.63:0/2632625218 shutdown_connections 2024-05-27T08:11:33.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.707+0000 7eff87539b80 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559c67e2fe00 0x559c67e32220 crc :-1 s=READY pgs=675 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.707+0000 7eff87539b80 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559c67daa820 0x559c67db94d0 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.707+0000 7eff87539b80 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c67c2be40 0x559c67db8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.707+0000 7eff87539b80 1 --2- 172.21.15.63:0/2632625218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559c67daa410 0x559c67db8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.707+0000 7eff87539b80 1 -- 172.21.15.63:0/2632625218 >> 172.21.15.63:0/2632625218 conn(0x559c67c8bfc0 msgr2=0x559c67db34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:33.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.707+0000 7eff53e00700 0 client.22683 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:33.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.707+0000 7eff87539b80 1 -- 172.21.15.63:0/2632625218 shutdown_connections 2024-05-27T08:11:33.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.707+0000 7eff87539b80 1 -- 172.21.15.63:0/2632625218 wait complete. 2024-05-27T08:11:33.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.707+0000 7eff87539b80 20 client.22683 trim_cache size 0 max 16384 2024-05-27T08:11:33.709 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:11:33.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 -- 172.21.15.63:0/3343698382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff580b29f0 msgr2=0x7eff580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:33.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff580b29f0 0x7eff580b4f00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x559c67da6690 tx=0x7eff6c009040 comp rx=0 tx=0).stop 2024-05-27T08:11:33.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 -- 172.21.15.63:0/3343698382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff740764a0 msgr2=0x7eff741a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:33.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff740764a0 0x7eff741a1750 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7eff6000d950 tx=0x7eff6000dc60 comp rx=0 tx=0).stop 2024-05-27T08:11:33.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 -- 172.21.15.63:0/3343698382 shutdown_connections 2024-05-27T08:11:33.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7eff580b29f0 0x7eff580b4f00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff74115170 0x7eff741a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff74075b50 0x7eff741a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 --2- 172.21.15.63:0/3343698382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff740764a0 0x7eff741a1750 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:33.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 -- 172.21.15.63:0/3343698382 >> 172.21.15.63:0/3343698382 conn(0x7eff74079d00 msgr2=0x7eff74101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:33.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 -- 172.21.15.63:0/3343698382 shutdown_connections 2024-05-27T08:11:33.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:33.708+0000 7eff73200700 1 -- 172.21.15.63:0/3343698382 wait complete. 2024-05-27T08:11:33.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:33 smithi063 ceph-mon[31613]: pgmap v1205: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:11:34.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:33 smithi086 ceph-mon[34687]: pgmap v1205: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:11:34.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:33 smithi149 ceph-mon[35408]: pgmap v1205: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:11:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.619+0000 7f12bce00640 1 -- 172.21.15.63:0/1941240853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b81043d0 msgr2=0x7f12b81107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.619+0000 7f12bce00640 1 --2- 172.21.15.63:0/1941240853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b81043d0 0x7f12b81107a0 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009ca0 tx=0x7f12b002a8c0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.619+0000 7f12bce00640 1 -- 172.21.15.63:0/1941240853 shutdown_connections 2024-05-27T08:11:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.619+0000 7f12bce00640 1 --2- 172.21.15.63:0/1941240853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b81043d0 0x7f12b81107a0 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.619+0000 7f12bce00640 1 --2- 172.21.15.63:0/1941240853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12b810a120 0x7f12b8103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.619+0000 7f12bce00640 1 --2- 172.21.15.63:0/1941240853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8109c80 0x7f12b8103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.619+0000 7f12bce00640 1 -- 172.21.15.63:0/1941240853 >> 172.21.15.63:0/1941240853 conn(0x7f12b80ff340 msgr2=0x7f12b8101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.619+0000 7f12bce00640 1 -- 172.21.15.63:0/1941240853 shutdown_connections 2024-05-27T08:11:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.619+0000 7f12bce00640 1 -- 172.21.15.63:0/1941240853 wait complete. 2024-05-27T08:11:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.620+0000 7f12bce00640 1 Processor -- start 2024-05-27T08:11:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.620+0000 7f12bce00640 1 -- start start 2024-05-27T08:11:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.621+0000 7f12bce00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b81043d0 0x7f12b81a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.621+0000 7f12bce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8109c80 0x7f12b81a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.621+0000 7f12bce00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12b810a120 0x7f12b819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.621+0000 7f12bce00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8113d90 con 0x7f12b81043d0 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.621+0000 7f12bce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8113c10 con 0x7f12b8109c80 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.621+0000 7f12bce00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8113f10 con 0x7f12b810a120 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.621+0000 7f12b5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b81043d0 0x7f12b81a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.621+0000 7f12b5e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b81043d0 0x7f12b81a2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35780/0 (socket says 172.21.15.63:35780) 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.621+0000 7f12b5e00640 1 -- 172.21.15.63:0/3603510831 learned_addr learned my addr 172.21.15.63:0/3603510831 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.622+0000 7f12b6800640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12b810a120 0x7f12b819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.622+0000 7f12b5400640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8109c80 0x7f12b81a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.622+0000 7f12b5e00640 1 -- 172.21.15.63:0/3603510831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12b810a120 msgr2=0x7f12b819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.622+0000 7f12b5e00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12b810a120 0x7f12b819c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.622+0000 7f12b5e00640 1 -- 172.21.15.63:0/3603510831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8109c80 msgr2=0x7f12b81a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.622+0000 7f12b5e00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8109c80 0x7f12b81a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.622+0000 7f12b5e00640 1 -- 172.21.15.63:0/3603510831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b819cf30 con 0x7f12b81043d0 2024-05-27T08:11:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.622+0000 7f12b5e00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b81043d0 0x7f12b81a2680 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f12a8005fd0 tx=0x7f12a800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.623+0000 7f12aea00640 1 -- 172.21.15.63:0/3603510831 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8017070 con 0x7f12b81043d0 2024-05-27T08:11:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.623+0000 7f12aea00640 1 -- 172.21.15.63:0/3603510831 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12a8009c30 con 0x7f12b81043d0 2024-05-27T08:11:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.623+0000 7f12bce00640 1 -- 172.21.15.63:0/3603510831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b819d220 con 0x7f12b81043d0 2024-05-27T08:11:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.623+0000 7f12aea00640 1 -- 172.21.15.63:0/3603510831 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8003c70 con 0x7f12b81043d0 2024-05-27T08:11:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.623+0000 7f12bce00640 1 -- 172.21.15.63:0/3603510831 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b81a9500 con 0x7f12b81043d0 2024-05-27T08:11:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.627+0000 7f12c3d94740 1 Processor -- start 2024-05-27T08:11:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.627+0000 7f12c3d94740 1 -- start start 2024-05-27T08:11:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.627+0000 7f12aea00640 1 -- 172.21.15.63:0/3603510831 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f12a8003e10 con 0x7f12b81043d0 2024-05-27T08:11:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edabda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 0x55e36edac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12c3d94740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e36ed1cb90 con 0x55e36ea091b0 2024-05-27T08:11:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12c3d94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e36ed0c730 con 0x55e36e9e02f0 2024-05-27T08:11:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12c3d94740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e36ed10770 con 0x55e36e9e84d0 2024-05-27T08:11:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12b6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12b6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35788/0 (socket says 172.21.15.63:35788) 2024-05-27T08:11:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12b6800640 1 -- 172.21.15.63:0/1259765809 learned_addr learned my addr 172.21.15.63:0/1259765809 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12b5400640 1 --2- 172.21.15.63:0/1259765809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 0x55e36edac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.628+0000 7f12b5e00640 1 --2- 172.21.15.63:0/1259765809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edabda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.629+0000 7f12b6800640 1 -- 172.21.15.63:0/1259765809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 msgr2=0x55e36edac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.629+0000 7f12b6800640 1 --2- 172.21.15.63:0/1259765809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 0x55e36edac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.629+0000 7f12b6800640 1 -- 172.21.15.63:0/1259765809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 msgr2=0x55e36edabda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.629+0000 7f12b6800640 1 --2- 172.21.15.63:0/1259765809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edabda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.629+0000 7f12b6800640 1 -- 172.21.15.63:0/1259765809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e36eca58b0 con 0x55e36ea091b0 2024-05-27T08:11:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.629+0000 7f12b6800640 1 --2- 172.21.15.63:0/1259765809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edab860 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f12b0005cc0 tx=0x7f12b0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318b9188bdd4f69 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.630+0000 7f12a3400640 1 -- 172.21.15.63:0/1259765809 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0031980 con 0x55e36ea091b0 2024-05-27T08:11:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.630+0000 7f12aea00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f129c0b29d0 0x7f129c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.630+0000 7f12a3400640 1 -- 172.21.15.63:0/1259765809 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12b0036040 con 0x55e36ea091b0 2024-05-27T08:11:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.630+0000 7f12a3400640 1 -- 172.21.15.63:0/1259765809 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0004330 con 0x55e36ea091b0 2024-05-27T08:11:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.630+0000 7f12aea00640 1 -- 172.21.15.63:0/3603510831 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f12a80e26f0 con 0x7f12b81043d0 2024-05-27T08:11:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.630+0000 7f12b5e00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f129c0b29d0 0x7f129c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.631+0000 7f12c3d94740 1 -- 172.21.15.63:0/1259765809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 msgr2=0x55e36edab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.631+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1259765809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edab860 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f12b0005cc0 tx=0x7f12b0009a20 comp rx=0 tx=0).stop 2024-05-27T08:11:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.631+0000 7f12b5e00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f129c0b29d0 0x7f129c0b4ee0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55e36eda9350 tx=0x7f12a800bf60 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.631+0000 7f12c3d94740 1 -- 172.21.15.63:0/1259765809 shutdown_connections 2024-05-27T08:11:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.631+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1259765809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 0x55e36edac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.631+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1259765809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edabda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.631+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1259765809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edab860 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.631+0000 7f12c3d94740 1 -- 172.21.15.63:0/1259765809 >> 172.21.15.63:0/1259765809 conn(0x55e36e9fd300 msgr2=0x55e36eda8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.631+0000 7f12c3d94740 1 -- 172.21.15.63:0/1259765809 shutdown_connections 2024-05-27T08:11:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.631+0000 7f12c3d94740 1 -- 172.21.15.63:0/1259765809 wait complete. 2024-05-27T08:11:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.632+0000 7f12c3d94740 1 -- start start 2024-05-27T08:11:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.632+0000 7f12c3d94740 1 Processor -- start 2024-05-27T08:11:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edb59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36e9e84d0 0x55e36edb5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36ea091b0 0x55e36edc1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12c3d94740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e36e9f2790 con 0x55e36e9e84d0 2024-05-27T08:11:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12c3d94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e36ecb4bc0 con 0x55e36e9e02f0 2024-05-27T08:11:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12c3d94740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e36edb8930 con 0x55e36ea091b0 2024-05-27T08:11:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12c3d94740 10 client.0 _ll_register_callbacks cb 0x55e36ec92400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12b5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36e9e84d0 0x55e36edb5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edb59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12b5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36e9e84d0 0x55e36edb5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35796/0 (socket says 172.21.15.63:35796) 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12b5400640 1 -- 172.21.15.63:0/1450033196 learned_addr learned my addr 172.21.15.63:0/1450033196 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.633+0000 7f12b6800640 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36ea091b0 0x55e36edc1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.634+0000 7f12b5400640 1 -- 172.21.15.63:0/1450033196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36ea091b0 msgr2=0x55e36edc1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.634+0000 7f12b5400640 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36ea091b0 0x55e36edc1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.634+0000 7f12b5400640 1 -- 172.21.15.63:0/1450033196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 msgr2=0x55e36edb59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.634+0000 7f12b5400640 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edb59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.634+0000 7f12b5400640 1 -- 172.21.15.63:0/1450033196 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36ea21210 con 0x55e36e9e84d0 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.634+0000 7f12b6800640 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36ea091b0 0x55e36edc1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.634+0000 7f12b5400640 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36e9e84d0 0x55e36edb5f20 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f12a400be40 tx=0x7f12a4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d924be721216d22d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.634+0000 7f1297e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.635+0000 7f1297e00640 1 -- 172.21.15.63:0/1450033196 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a40035f0 con 0x55e36e9e84d0 2024-05-27T08:11:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.635+0000 7f1297e00640 1 -- 172.21.15.63:0/1450033196 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f12a4003790 con 0x55e36e9e84d0 2024-05-27T08:11:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.635+0000 7f12c3d94740 1 -- 172.21.15.63:0/1450033196 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b0036040 con 0x55e36e9e84d0 2024-05-27T08:11:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.635+0000 7f1297e00640 1 -- 172.21.15.63:0/1450033196 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f12a4003990 con 0x55e36e9e84d0 2024-05-27T08:11:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.635+0000 7f12c3d94740 10 client.22695 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.635+0000 7f12c3d94740 1 -- 172.21.15.63:0/1450033196 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e36eaed620 con 0x55e36e9e84d0 2024-05-27T08:11:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.635+0000 7f1297e00640 1 -- 172.21.15.63:0/1450033196 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12b0036040 con 0x55e36e9e84d0 2024-05-27T08:11:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.636+0000 7f1297e00640 1 -- 172.21.15.63:0/1450033196 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f12a4019070 con 0x55e36e9e84d0 2024-05-27T08:11:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.636+0000 7f12c3d94740 10 client.22695 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.636+0000 7f12c3d94740 10 client.22695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.636+0000 7f12c3d94740 20 client.22695 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.636+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e36e9c7d80 0x55e36edc7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.636+0000 7f12c3d94740 4 client.22695 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:11:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.636+0000 7f12c3d94740 1 -- 172.21.15.63:0/1450033196 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e36ed1eae0 con 0x55e36e9c7d80 2024-05-27T08:11:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.637+0000 7f12b6800640 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e36e9c7d80 0x55e36edc7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.638+0000 7f1297e00640 10 client.22695 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.638+0000 7f12b6800640 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e36e9c7d80 0x55e36edc7c40 crc :-1 s=READY pgs=676 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f87ae87ab898450 server_cookie=896f8a644e11856c in_seq=0 out_seq=0 2024-05-27T08:11:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f1297e00640 1 -- 172.21.15.63:0/1450033196 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e36ed1eae0 con 0x55e36e9c7d80 2024-05-27T08:11:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f1297e00640 10 client.22695 handle_command_reply: tid=0 2024-05-27T08:11:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f12c3d94740 1 client.22695 shutdown 2024-05-27T08:11:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f12c3d94740 1 -- 172.21.15.63:0/1450033196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36e9e84d0 msgr2=0x55e36edb5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36e9e84d0 0x55e36edb5f20 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f12a400be40 tx=0x7f12a4012820 comp rx=0 tx=0).stop 2024-05-27T08:11:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f12c3d94740 1 -- 172.21.15.63:0/1450033196 shutdown_connections 2024-05-27T08:11:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e36e9c7d80 0x55e36edc7c40 crc :-1 s=READY pgs=676 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36ea091b0 0x55e36edc1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36e9e84d0 0x55e36edb5f20 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1450033196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edb59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f12c3d94740 1 -- 172.21.15.63:0/1450033196 >> 172.21.15.63:0/1450033196 conn(0x55e36e9fd300 msgr2=0x55e36edab690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f1297e00640 0 client.22695 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.640+0000 7f12c3d94740 1 -- 172.21.15.63:0/1450033196 shutdown_connections 2024-05-27T08:11:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.641+0000 7f12c3d94740 1 -- 172.21.15.63:0/1450033196 wait complete. 2024-05-27T08:11:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.641+0000 7f12c3d94740 20 client.22695 trim_cache size 0 max 16384 2024-05-27T08:11:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.665+0000 7f12c3d94740 1 Processor -- start 2024-05-27T08:11:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.665+0000 7f12c3d94740 1 -- start start 2024-05-27T08:11:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.665+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e02f0 0x55e36edb7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.665+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e84d0 0x55e36edc19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.665+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edc1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.665+0000 7f12c3d94740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81b5df0 con 0x55e36ea091b0 2024-05-27T08:11:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.665+0000 7f12c3d94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81b36c0 con 0x55e36e9e84d0 2024-05-27T08:11:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.665+0000 7f12c3d94740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81b1530 con 0x55e36e9e02f0 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.666+0000 7f12b6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edc1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.666+0000 7f12b6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edc1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35806/0 (socket says 172.21.15.63:35806) 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.666+0000 7f12b6800640 1 -- 172.21.15.63:0/1457280813 learned_addr learned my addr 172.21.15.63:0/1457280813 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.666+0000 7f12b5400640 1 --2- 172.21.15.63:0/1457280813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e84d0 0x55e36edc19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.666+0000 7f12b6800640 1 --2- 172.21.15.63:0/1457280813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e02f0 0x55e36edb7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.666+0000 7f12b6800640 1 -- 172.21.15.63:0/1457280813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e02f0 msgr2=0x55e36edb7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.666+0000 7f12b6800640 1 --2- 172.21.15.63:0/1457280813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e02f0 0x55e36edb7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.666+0000 7f12b6800640 1 -- 172.21.15.63:0/1457280813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e84d0 msgr2=0x55e36edc19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.666+0000 7f12b6800640 1 --2- 172.21.15.63:0/1457280813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e84d0 0x55e36edc19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.666+0000 7f12b6800640 1 -- 172.21.15.63:0/1457280813 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e36e9c7fb0 con 0x55e36ea091b0 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.667+0000 7f12b6800640 1 --2- 172.21.15.63:0/1457280813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edc1f10 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f12b0000f80 tx=0x7f12b0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44986f43d53ece4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.667+0000 7f12a2a00640 1 -- 172.21.15.63:0/1457280813 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b004a040 con 0x55e36ea091b0 2024-05-27T08:11:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.667+0000 7f12a2a00640 1 -- 172.21.15.63:0/1457280813 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12b0036070 con 0x55e36ea091b0 2024-05-27T08:11:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.667+0000 7f12a2a00640 1 -- 172.21.15.63:0/1457280813 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0004430 con 0x55e36ea091b0 2024-05-27T08:11:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.668+0000 7f12c3d94740 1 -- 172.21.15.63:0/1457280813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 msgr2=0x55e36edc1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.668+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1457280813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edc1f10 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f12b0000f80 tx=0x7f12b0005c50 comp rx=0 tx=0).stop 2024-05-27T08:11:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.669+0000 7f12c3d94740 1 -- 172.21.15.63:0/1457280813 shutdown_connections 2024-05-27T08:11:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.669+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1457280813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edc1f10 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.669+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1457280813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e84d0 0x55e36edc19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.669+0000 7f12c3d94740 1 --2- 172.21.15.63:0/1457280813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e02f0 0x55e36edb7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.669+0000 7f12c3d94740 1 -- 172.21.15.63:0/1457280813 >> 172.21.15.63:0/1457280813 conn(0x55e36e9fd300 msgr2=0x55e36edb7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.669+0000 7f12c3d94740 1 -- 172.21.15.63:0/1457280813 shutdown_connections 2024-05-27T08:11:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.669+0000 7f12c3d94740 1 -- 172.21.15.63:0/1457280813 wait complete. 2024-05-27T08:11:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.670+0000 7f12c3d94740 1 -- start start 2024-05-27T08:11:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.670+0000 7f12c3d94740 1 Processor -- start 2024-05-27T08:11:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.670+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edbaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 0x55e36edbb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12c3d94740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edbb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.671 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12c3d94740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81b1b80 con 0x55e36ea091b0 2024-05-27T08:11:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12c3d94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81b2ff0 con 0x55e36e9e02f0 2024-05-27T08:11:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12c3d94740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81b2250 con 0x55e36e9e84d0 2024-05-27T08:11:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12c3d94740 10 client.0 _ll_register_callbacks cb 0x7f12a400b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12b6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edbb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12b6800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edbb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35818/0 (socket says 172.21.15.63:35818) 2024-05-27T08:11:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12b6800640 1 -- 172.21.15.63:0/3606595160 learned_addr learned my addr 172.21.15.63:0/3606595160 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12b6800640 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edbaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.671+0000 7f12b5400640 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 0x55e36edbb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.672+0000 7f12b5400640 1 -- 172.21.15.63:0/3606595160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 msgr2=0x55e36edbaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.672+0000 7f12b5400640 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edbaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.672+0000 7f12b5400640 1 -- 172.21.15.63:0/3606595160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 msgr2=0x55e36edbb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.672+0000 7f12b5400640 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edbb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.672+0000 7f12b5400640 1 -- 172.21.15.63:0/3606595160 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36ee2e430 con 0x55e36e9e84d0 2024-05-27T08:11:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.672+0000 7f12b6800640 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edbb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:11:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.673+0000 7f12b5400640 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 0x55e36edbb030 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f12a401bcd0 tx=0x7f12a4003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec0bee055643625a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.673+0000 7f1297e00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:11:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.673+0000 7f1297e00640 1 -- 172.21.15.63:0/3606595160 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a400b9c0 con 0x55e36e9e84d0 2024-05-27T08:11:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.673+0000 7f12c3d94740 1 -- 172.21.15.63:0/3606595160 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b0036070 con 0x55e36e9e84d0 2024-05-27T08:11:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.674+0000 7f1297e00640 1 -- 172.21.15.63:0/3606595160 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f12a4005040 con 0x55e36e9e84d0 2024-05-27T08:11:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.674+0000 7f1297e00640 1 -- 172.21.15.63:0/3606595160 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f12a4005240 con 0x55e36e9e84d0 2024-05-27T08:11:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.674+0000 7f12c3d94740 10 client.31447 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.674+0000 7f12c3d94740 1 -- 172.21.15.63:0/3606595160 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e36ee2f680 con 0x55e36e9e84d0 2024-05-27T08:11:35.675 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.675+0000 7f1297e00640 1 -- 172.21.15.63:0/3606595160 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12b0036070 con 0x55e36e9e84d0 2024-05-27T08:11:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.675+0000 7f1297e00640 1 -- 172.21.15.63:0/3606595160 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f12a4019070 con 0x55e36e9e84d0 2024-05-27T08:11:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.675+0000 7f12c3d94740 10 client.31447 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.675+0000 7f12c3d94740 10 client.31447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.675+0000 7f12c3d94740 20 client.31447 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.675+0000 7f12c3d94740 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e36e99d1c0 0x55e36ee31e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.675+0000 7f12c3d94740 4 client.31447 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:11:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.675+0000 7f12c3d94740 1 -- 172.21.15.63:0/3606595160 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e36ed182c0 con 0x55e36e99d1c0 2024-05-27T08:11:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.676+0000 7f12b6800640 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e36e99d1c0 0x55e36ee31e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.677+0000 7f12b6800640 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e36e99d1c0 0x55e36ee31e70 crc :-1 s=READY pgs=677 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4f7d9f2c42f7220 server_cookie=985f3a93c683129f in_seq=0 out_seq=0 2024-05-27T08:11:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.677+0000 7f1297e00640 10 client.31447 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:35.712 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:11:35.713 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:11:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.722+0000 7f1297e00640 1 -- 172.21.15.63:0/3606595160 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e36ed182c0 con 0x55e36e99d1c0 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.722+0000 7f1297e00640 10 client.31447 handle_command_reply: tid=0 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.722+0000 7f12c3d94740 1 client.31447 shutdown 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.723+0000 7f12c3d94740 1 -- 172.21.15.63:0/3606595160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 msgr2=0x55e36edbb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.723+0000 7f12c3d94740 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 0x55e36edbb030 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f12a401bcd0 tx=0x7f12a4003820 comp rx=0 tx=0).stop 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.723+0000 7f12c3d94740 1 -- 172.21.15.63:0/3606595160 shutdown_connections 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.723+0000 7f12c3d94740 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e36e99d1c0 0x55e36ee31e70 crc :-1 s=READY pgs=677 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.723+0000 7f12c3d94740 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e36ea091b0 0x55e36edbb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.723+0000 7f12c3d94740 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e36e9e84d0 0x55e36edbb030 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.723+0000 7f12c3d94740 1 --2- 172.21.15.63:0/3606595160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e36e9e02f0 0x55e36edbaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.723+0000 7f1297e00640 0 client.31447 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.723+0000 7f12c3d94740 1 -- 172.21.15.63:0/3606595160 >> 172.21.15.63:0/3606595160 conn(0x55e36e9fd300 msgr2=0x55e36edb5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.723+0000 7f12c3d94740 1 -- 172.21.15.63:0/3606595160 shutdown_connections 2024-05-27T08:11:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.724+0000 7f12c3d94740 1 -- 172.21.15.63:0/3606595160 wait complete. 2024-05-27T08:11:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.725+0000 7f12c3d94740 20 client.31447 trim_cache size 0 max 16384 2024-05-27T08:11:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:11:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:11:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:11:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 -- 172.21.15.63:0/3603510831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f129c0b29d0 msgr2=0x7f129c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f129c0b29d0 0x7f129c0b4ee0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55e36eda9350 tx=0x7f12a800bf60 comp rx=0 tx=0).stop 2024-05-27T08:11:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 -- 172.21.15.63:0/3603510831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b81043d0 msgr2=0x7f12b81a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b81043d0 0x7f12b81a2680 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f12a8005fd0 tx=0x7f12a800dde0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 -- 172.21.15.63:0/3603510831 shutdown_connections 2024-05-27T08:11:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f129c0b29d0 0x7f129c0b4ee0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12b810a120 0x7f12b819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8109c80 0x7f12b81a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 --2- 172.21.15.63:0/3603510831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12b81043d0 0x7f12b81a2680 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 -- 172.21.15.63:0/3603510831 >> 172.21.15.63:0/3603510831 conn(0x7f12b80ff340 msgr2=0x7f12b8077970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.726+0000 7f12bce00640 1 -- 172.21.15.63:0/3603510831 shutdown_connections 2024-05-27T08:11:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:35.727+0000 7f12bce00640 1 -- 172.21.15.63:0/3603510831 wait complete. 2024-05-27T08:11:35.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:35 smithi063 ceph-mon[31613]: pgmap v1206: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:11:36.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:35 smithi086 ceph-mon[34687]: pgmap v1206: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:11:36.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:35 smithi149 ceph-mon[35408]: pgmap v1206: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:11:36.247 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:11:36.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:36 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:11:36.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:36 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:36.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:36 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:37.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:36 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:11:37.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:36 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:37.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:36 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:37.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:36 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:11:37.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:36 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:37.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:36 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:38.055 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:37 smithi063 ceph-mon[31613]: pgmap v1207: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.7 MiB/s wr, 18 op/s 2024-05-27T08:11:38.056 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:37 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:38.056 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:37 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:38.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:37 smithi086 ceph-mon[34687]: pgmap v1207: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.7 MiB/s wr, 18 op/s 2024-05-27T08:11:38.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:37 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:38.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:37 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:38.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:37 smithi149 ceph-mon[35408]: pgmap v1207: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.7 MiB/s wr, 18 op/s 2024-05-27T08:11:38.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:37 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:38.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:37 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:38.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:38 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:11:38.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:38 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:11:38.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:38 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:39.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:38 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:11:39.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:38 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:11:39.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:38 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:39.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:38 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:11:39.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:38 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:11:39.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:38 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:11:40.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.074+0000 7f56c9800700 1 -- 172.21.15.63:0/2685441181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56c4110b90 msgr2=0x7f56c4112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:40.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.074+0000 7f56c9800700 1 --2- 172.21.15.63:0/2685441181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56c4110b90 0x7f56c4112ff0 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009990 tx=0x7f56bc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.074+0000 7f56c9800700 1 -- 172.21.15.63:0/2685441181 shutdown_connections 2024-05-27T08:11:40.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.074+0000 7f56c9800700 1 --2- 172.21.15.63:0/2685441181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56c4110b90 0x7f56c4112ff0 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.074+0000 7f56c9800700 1 --2- 172.21.15.63:0/2685441181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4075d20 0x7f56c4076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.074+0000 7f56c9800700 1 --2- 172.21.15.63:0/2685441181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c4076670 0x7f56c4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.074+0000 7f56c9800700 1 -- 172.21.15.63:0/2685441181 >> 172.21.15.63:0/2685441181 conn(0x7f56c4100030 msgr2=0x7f56c4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:40.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.075+0000 7f56c9800700 1 -- 172.21.15.63:0/2685441181 shutdown_connections 2024-05-27T08:11:40.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.075+0000 7f56c9800700 1 -- 172.21.15.63:0/2685441181 wait complete. 2024-05-27T08:11:40.076 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:39 smithi063 ceph-mon[31613]: pgmap v1208: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:11:40.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.076+0000 7f56c9800700 1 Processor -- start 2024-05-27T08:11:40.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.076+0000 7f56c9800700 1 -- start start 2024-05-27T08:11:40.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.077+0000 7f56c9800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4075d20 0x7f56c419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.077+0000 7f56c9800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56c4076670 0x7f56c419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.077+0000 7f56c9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c4110b90 0x7f56c41a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.077+0000 7f56c9800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56c4116570 con 0x7f56c4076670 2024-05-27T08:11:40.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.077+0000 7f56c9800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41163f0 con 0x7f56c4110b90 2024-05-27T08:11:40.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.077+0000 7f56c9800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41166f0 con 0x7f56c4075d20 2024-05-27T08:11:40.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.077+0000 7f56ba000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56c4076670 0x7f56c419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.077+0000 7f56ba000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56c4076670 0x7f56c419d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34014/0 (socket says 172.21.15.63:34014) 2024-05-27T08:11:40.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.077+0000 7f56ba000700 1 -- 172.21.15.63:0/3513157379 learned_addr learned my addr 172.21.15.63:0/3513157379 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:40.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.078+0000 7f56c3400700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c4110b90 0x7f56c41a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.078+0000 7f56ba000700 1 -- 172.21.15.63:0/3513157379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4075d20 msgr2=0x7f56c419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.078+0000 7f56ba000700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4075d20 0x7f56c419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.078+0000 7f56ba000700 1 -- 172.21.15.63:0/3513157379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c4110b90 msgr2=0x7f56c41a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.078+0000 7f56ba000700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c4110b90 0x7f56c41a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.078+0000 7f56ba000700 1 -- 172.21.15.63:0/3513157379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c41a1df0 con 0x7f56c4076670 2024-05-27T08:11:40.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.078+0000 7f56ba000700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56c4076670 0x7f56c419d4c0 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00ef10 tx=0x7f56ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.079+0000 7f56bbe00700 1 -- 172.21.15.63:0/3513157379 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00cad0 con 0x7f56c4076670 2024-05-27T08:11:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.079+0000 7f56c9800700 1 -- 172.21.15.63:0/3513157379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c41a2080 con 0x7f56c4076670 2024-05-27T08:11:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.079+0000 7f56bbe00700 1 -- 172.21.15.63:0/3513157379 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56ac019070 con 0x7f56c4076670 2024-05-27T08:11:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.079+0000 7f56bbe00700 1 -- 172.21.15.63:0/3513157379 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac014480 con 0x7f56c4076670 2024-05-27T08:11:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.079+0000 7f56c9800700 1 -- 172.21.15.63:0/3513157379 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c41a9a00 con 0x7f56c4076670 2024-05-27T08:11:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.084+0000 7f56bbe00700 1 -- 172.21.15.63:0/3513157379 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f56ac012050 con 0x7f56c4076670 2024-05-27T08:11:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.085+0000 7f56bbe00700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56a80b29f0 0x7f56a80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.086+0000 7f56bbe00700 1 -- 172.21.15.63:0/3513157379 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f56ac0e2a70 con 0x7f56c4076670 2024-05-27T08:11:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.086+0000 7f56c2a00700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56a80b29f0 0x7f56a80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.087+0000 7f56c2a00700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56a80b29f0 0x7f56a80b4f00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f56b4005950 tx=0x7f56b40058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.103+0000 7f56d8306b80 1 Processor -- start 2024-05-27T08:11:40.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.104+0000 7f56d8306b80 1 -- start start 2024-05-27T08:11:40.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.105+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.105+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee328f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.105+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eee32ef0 0x5597eee3f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.105+0000 7f56d8306b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5597eed60ec0 con 0x5597eecb3ec0 2024-05-27T08:11:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.105+0000 7f56d8306b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597eed50a60 con 0x5597eed9fa70 2024-05-27T08:11:40.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.105+0000 7f56d8306b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5597eed54aa0 con 0x5597eee32ef0 2024-05-27T08:11:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.106+0000 7f56ba000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.106+0000 7f56c2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.106+0000 7f56ba000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee328f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34024/0 (socket says 172.21.15.63:34024) 2024-05-27T08:11:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.106+0000 7f56ba000700 1 -- 172.21.15.63:0/2394859044 learned_addr learned my addr 172.21.15.63:0/2394859044 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.106+0000 7f56c3400700 1 --2- 172.21.15.63:0/2394859044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eee32ef0 0x5597eee3f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.106+0000 7f56ba000700 1 -- 172.21.15.63:0/2394859044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eee32ef0 msgr2=0x5597eee3f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.106+0000 7f56ba000700 1 --2- 172.21.15.63:0/2394859044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eee32ef0 0x5597eee3f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.106+0000 7f56ba000700 1 -- 172.21.15.63:0/2394859044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 msgr2=0x5597eee323b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.106+0000 7f56ba000700 1 --2- 172.21.15.63:0/2394859044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee323b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.106+0000 7f56ba000700 1 -- 172.21.15.63:0/2394859044 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597eed194a0 con 0x5597eecb3ec0 2024-05-27T08:11:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.107+0000 7f56ba000700 1 --2- 172.21.15.63:0/2394859044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee328f0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0a98c0 tx=0x7f56ac0a90a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d450aa6aa850de server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.107+0000 7f56b3e00700 1 -- 172.21.15.63:0/2394859044 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0aa030 con 0x5597eecb3ec0 2024-05-27T08:11:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.107+0000 7f56b3e00700 1 -- 172.21.15.63:0/2394859044 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56ac08fe50 con 0x5597eecb3ec0 2024-05-27T08:11:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.107+0000 7f56b3e00700 1 -- 172.21.15.63:0/2394859044 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0cc290 con 0x5597eecb3ec0 2024-05-27T08:11:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.108+0000 7f56d8306b80 1 -- 172.21.15.63:0/2394859044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 msgr2=0x5597eee328f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.108+0000 7f56d8306b80 1 --2- 172.21.15.63:0/2394859044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee328f0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0a98c0 tx=0x7f56ac0a90a0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.109+0000 7f56d8306b80 1 -- 172.21.15.63:0/2394859044 shutdown_connections 2024-05-27T08:11:40.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.109+0000 7f56d8306b80 1 --2- 172.21.15.63:0/2394859044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eee32ef0 0x5597eee3f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.109+0000 7f56d8306b80 1 --2- 172.21.15.63:0/2394859044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.109+0000 7f56d8306b80 1 --2- 172.21.15.63:0/2394859044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee328f0 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.109+0000 7f56d8306b80 1 -- 172.21.15.63:0/2394859044 >> 172.21.15.63:0/2394859044 conn(0x5597eed4d410 msgr2=0x5597eee2e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.109+0000 7f56d8306b80 1 -- 172.21.15.63:0/2394859044 shutdown_connections 2024-05-27T08:11:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.109+0000 7f56d8306b80 1 -- 172.21.15.63:0/2394859044 wait complete. 2024-05-27T08:11:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.110+0000 7f56d8306b80 1 -- start start 2024-05-27T08:11:40.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.110+0000 7f56d8306b80 1 Processor -- start 2024-05-27T08:11:40.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eecb3ec0 0x5597eee3c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee3cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eee32ef0 0x5597eee3d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56d8306b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5597eec57020 con 0x5597eee32ef0 2024-05-27T08:11:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56d8306b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597eecf17e0 con 0x5597eed9fa70 2024-05-27T08:11:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56d8306b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5597eee3fc00 con 0x5597eecb3ec0 2024-05-27T08:11:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56d8306b80 10 client.0 _ll_register_callbacks cb 0x5597eecfacf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56c3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eee32ef0 0x5597eee3d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56c3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eee32ef0 0x5597eee3d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34026/0 (socket says 172.21.15.63:34026) 2024-05-27T08:11:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56c3400700 1 -- 172.21.15.63:0/470791606 learned_addr learned my addr 172.21.15.63:0/470791606 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56c2a00700 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eecb3ec0 0x5597eee3c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.111+0000 7f56ba000700 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee3cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.112+0000 7f56c3400700 1 -- 172.21.15.63:0/470791606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eecb3ec0 msgr2=0x5597eee3c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.112+0000 7f56c3400700 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eecb3ec0 0x5597eee3c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.112+0000 7f56c3400700 1 -- 172.21.15.63:0/470791606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 msgr2=0x5597eee3cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.112+0000 7f56c3400700 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee3cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.112+0000 7f56c3400700 1 -- 172.21.15.63:0/470791606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597eeca0950 con 0x5597eee32ef0 2024-05-27T08:11:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.112+0000 7f56c3400700 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eee32ef0 0x5597eee3d2e0 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f56bc005750 tx=0x7f56bc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f4d438e16044b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.112+0000 7f56b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.113+0000 7f56b0c00700 1 -- 172.21.15.63:0/470791606 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc027060 con 0x5597eee32ef0 2024-05-27T08:11:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.113+0000 7f56b0c00700 1 -- 172.21.15.63:0/470791606 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f56bc003c10 con 0x5597eee32ef0 2024-05-27T08:11:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.113+0000 7f56d8306b80 1 -- 172.21.15.63:0/470791606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56ac08fe50 con 0x5597eee32ef0 2024-05-27T08:11:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.114+0000 7f56b0c00700 1 -- 172.21.15.63:0/470791606 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f56bc003e10 con 0x5597eee32ef0 2024-05-27T08:11:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.114+0000 7f56d8306b80 10 client.22713 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.114+0000 7f56d8306b80 1 -- 172.21.15.63:0/470791606 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5597eed210e0 con 0x5597eee32ef0 2024-05-27T08:11:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.115+0000 7f56b0c00700 1 -- 172.21.15.63:0/470791606 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56ac08fe50 con 0x5597eee32ef0 2024-05-27T08:11:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.115+0000 7f56b0c00700 1 -- 172.21.15.63:0/470791606 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f56bc021030 con 0x5597eee32ef0 2024-05-27T08:11:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.115+0000 7f56d8306b80 10 client.22713 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.115+0000 7f56d8306b80 10 client.22713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:40.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.115+0000 7f56d8306b80 20 client.22713 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:40.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.115+0000 7f56d8306b80 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5597eee4cbc0 0x5597eee4f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.115+0000 7f56d8306b80 4 client.22713 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:11:40.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.115+0000 7f56d8306b80 1 -- 172.21.15.63:0/470791606 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597eed62e10 con 0x5597eee4cbc0 2024-05-27T08:11:40.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.116+0000 7f56c2a00700 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5597eee4cbc0 0x5597eee4f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.117+0000 7f56b0c00700 10 client.22713 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:40.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.117+0000 7f56c2a00700 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5597eee4cbc0 0x5597eee4f000 crc :-1 s=READY pgs=678 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a6d13af54b5b8b server_cookie=9850645d2a9ac01 in_seq=0 out_seq=0 2024-05-27T08:11:40.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56b0c00700 1 -- 172.21.15.63:0/470791606 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5597eed62e10 con 0x5597eee4cbc0 2024-05-27T08:11:40.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56b0c00700 10 client.22713 handle_command_reply: tid=0 2024-05-27T08:11:40.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56d8306b80 1 client.22713 shutdown 2024-05-27T08:11:40.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56d8306b80 1 -- 172.21.15.63:0/470791606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eee32ef0 msgr2=0x5597eee3d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:40.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56d8306b80 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eee32ef0 0x5597eee3d2e0 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f56bc005750 tx=0x7f56bc005780 comp rx=0 tx=0).stop 2024-05-27T08:11:40.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56d8306b80 1 -- 172.21.15.63:0/470791606 shutdown_connections 2024-05-27T08:11:40.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56d8306b80 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5597eee4cbc0 0x5597eee4f000 crc :-1 s=READY pgs=678 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56d8306b80 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eee32ef0 0x5597eee3d2e0 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56d8306b80 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eecb3ec0 0x5597eee3c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56d8306b80 1 --2- 172.21.15.63:0/470791606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee3cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56d8306b80 1 -- 172.21.15.63:0/470791606 >> 172.21.15.63:0/470791606 conn(0x5597eed4d410 msgr2=0x5597eee32430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:40.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.119+0000 7f56b0c00700 0 client.22713 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:40.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.120+0000 7f56d8306b80 1 -- 172.21.15.63:0/470791606 shutdown_connections 2024-05-27T08:11:40.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.120+0000 7f56d8306b80 1 -- 172.21.15.63:0/470791606 wait complete. 2024-05-27T08:11:40.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.120+0000 7f56d8306b80 20 client.22713 trim_cache size 0 max 16384 2024-05-27T08:11:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.148+0000 7f56d8306b80 1 Processor -- start 2024-05-27T08:11:40.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.148+0000 7f56d8306b80 1 -- start start 2024-05-27T08:11:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.148+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee38010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.148+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee48ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.148+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eee32ef0 0x5597eee34660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.148+0000 7f56d8306b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b4f60 con 0x5597eecb3ec0 2024-05-27T08:11:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.148+0000 7f56d8306b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b27f0 con 0x5597eed9fa70 2024-05-27T08:11:40.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.148+0000 7f56d8306b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b0660 con 0x5597eee32ef0 2024-05-27T08:11:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.149+0000 7f56c2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee38010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.149+0000 7f56c2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee38010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34032/0 (socket says 172.21.15.63:34032) 2024-05-27T08:11:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.149+0000 7f56c2a00700 1 -- 172.21.15.63:0/2921266098 learned_addr learned my addr 172.21.15.63:0/2921266098 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.149+0000 7f56ba000700 1 --2- 172.21.15.63:0/2921266098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee48ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.149+0000 7f56c3400700 1 --2- 172.21.15.63:0/2921266098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eee32ef0 0x5597eee34660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.149+0000 7f56c2a00700 1 -- 172.21.15.63:0/2921266098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eee32ef0 msgr2=0x5597eee34660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:40.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.149+0000 7f56c2a00700 1 --2- 172.21.15.63:0/2921266098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eee32ef0 0x5597eee34660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.149+0000 7f56c2a00700 1 -- 172.21.15.63:0/2921266098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 msgr2=0x5597eee48ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.149+0000 7f56c2a00700 1 --2- 172.21.15.63:0/2921266098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee48ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.149+0000 7f56c2a00700 1 -- 172.21.15.63:0/2921266098 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597eee2a2c0 con 0x5597eecb3ec0 2024-05-27T08:11:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.150+0000 7f56c2a00700 1 --2- 172.21.15.63:0/2921266098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee38010 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f56b4012040 tx=0x7f56b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14367367b4cbea73 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.150+0000 7f56b3400700 1 -- 172.21.15.63:0/2921266098 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4010da0 con 0x5597eecb3ec0 2024-05-27T08:11:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.150+0000 7f56b3400700 1 -- 172.21.15.63:0/2921266098 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56b402d070 con 0x5597eecb3ec0 2024-05-27T08:11:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.150+0000 7f56b3400700 1 -- 172.21.15.63:0/2921266098 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b401e650 con 0x5597eecb3ec0 2024-05-27T08:11:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.151+0000 7f56d8306b80 1 -- 172.21.15.63:0/2921266098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 msgr2=0x5597eee38010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.151+0000 7f56d8306b80 1 --2- 172.21.15.63:0/2921266098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee38010 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f56b4012040 tx=0x7f56b4013000 comp rx=0 tx=0).stop 2024-05-27T08:11:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.152+0000 7f56d8306b80 1 -- 172.21.15.63:0/2921266098 shutdown_connections 2024-05-27T08:11:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.152+0000 7f56d8306b80 1 --2- 172.21.15.63:0/2921266098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eee32ef0 0x5597eee34660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.152+0000 7f56d8306b80 1 --2- 172.21.15.63:0/2921266098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee38010 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.152+0000 7f56d8306b80 1 --2- 172.21.15.63:0/2921266098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eed9fa70 0x5597eee48ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.152+0000 7f56d8306b80 1 -- 172.21.15.63:0/2921266098 >> 172.21.15.63:0/2921266098 conn(0x5597eed4d410 msgr2=0x5597eee33f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.152+0000 7f56d8306b80 1 -- 172.21.15.63:0/2921266098 shutdown_connections 2024-05-27T08:11:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.152+0000 7f56d8306b80 1 -- 172.21.15.63:0/2921266098 wait complete. 2024-05-27T08:11:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.152+0000 7f56d8306b80 1 -- start start 2024-05-27T08:11:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.153+0000 7f56d8306b80 1 Processor -- start 2024-05-27T08:11:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.153+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee45520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.153+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eed9fa70 0x5597eee36f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.153+0000 7f56d8306b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eee32ef0 0x5597eee37450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.153+0000 7f56d8306b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b0cb0 con 0x5597eecb3ec0 2024-05-27T08:11:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.153+0000 7f56d8306b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b2120 con 0x5597eee32ef0 2024-05-27T08:11:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.153+0000 7f56d8306b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b1380 con 0x5597eed9fa70 2024-05-27T08:11:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.153+0000 7f56d8306b80 10 client.0 _ll_register_callbacks cb 0x5597eecfacf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.154+0000 7f56c2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee45520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.154+0000 7f56c2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee45520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34038/0 (socket says 172.21.15.63:34038) 2024-05-27T08:11:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.154+0000 7f56c2a00700 1 -- 172.21.15.63:0/1900649672 learned_addr learned my addr 172.21.15.63:0/1900649672 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.154+0000 7f56c3400700 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eee32ef0 0x5597eee37450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.154+0000 7f56ba000700 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eed9fa70 0x5597eee36f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.154+0000 7f56c2a00700 1 -- 172.21.15.63:0/1900649672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eed9fa70 msgr2=0x5597eee36f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:40.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.154+0000 7f56c2a00700 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eed9fa70 0x5597eee36f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.154+0000 7f56c2a00700 1 -- 172.21.15.63:0/1900649672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eee32ef0 msgr2=0x5597eee37450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:40.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.154+0000 7f56c2a00700 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eee32ef0 0x5597eee37450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.154+0000 7f56c2a00700 1 -- 172.21.15.63:0/1900649672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597eee37ac0 con 0x5597eecb3ec0 2024-05-27T08:11:40.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.155+0000 7f56c2a00700 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee45520 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f56b400dfa0 tx=0x7f56b400dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb124fa2f21f69b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:40.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.155+0000 7f56b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:40.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.155+0000 7f56b0c00700 1 -- 172.21.15.63:0/1900649672 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4036070 con 0x5597eecb3ec0 2024-05-27T08:11:40.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.155+0000 7f56d8306b80 1 -- 172.21.15.63:0/1900649672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b402d070 con 0x5597eecb3ec0 2024-05-27T08:11:40.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.155+0000 7f56b0c00700 1 -- 172.21.15.63:0/1900649672 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f56b4027750 con 0x5597eecb3ec0 2024-05-27T08:11:40.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.156+0000 7f56b0c00700 1 -- 172.21.15.63:0/1900649672 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f56b4010de0 con 0x5597eecb3ec0 2024-05-27T08:11:40.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.156+0000 7f56d8306b80 10 client.22719 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:40.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.156+0000 7f56d8306b80 1 -- 172.21.15.63:0/1900649672 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5597eeeb7690 con 0x5597eecb3ec0 2024-05-27T08:11:40.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.156+0000 7f56b0c00700 1 -- 172.21.15.63:0/1900649672 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56b402d070 con 0x5597eecb3ec0 2024-05-27T08:11:40.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.156+0000 7f56b0c00700 1 -- 172.21.15.63:0/1900649672 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f56b403bc10 con 0x5597eecb3ec0 2024-05-27T08:11:40.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.157+0000 7f56d8306b80 10 client.22719 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:40.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.157+0000 7f56d8306b80 10 client.22719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:40.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.157+0000 7f56d8306b80 20 client.22719 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:40.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.157+0000 7f56d8306b80 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5597eeeb7be0 0x5597eeeba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:40.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.157+0000 7f56d8306b80 4 client.22719 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:11:40.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.157+0000 7f56d8306b80 1 -- 172.21.15.63:0/1900649672 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5597eed5c5f0 con 0x5597eeeb7be0 2024-05-27T08:11:40.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.157+0000 7f56ba000700 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5597eeeb7be0 0x5597eeeba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:40.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.158+0000 7f56ba000700 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5597eeeb7be0 0x5597eeeba000 crc :-1 s=READY pgs=679 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18cd9f80e40f7962 server_cookie=bd81976d97f5df60 in_seq=0 out_seq=0 2024-05-27T08:11:40.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.158+0000 7f56b0c00700 10 client.22719 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:40.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.161+0000 7f56b0c00700 1 -- 172.21.15.63:0/1900649672 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5597eed5c5f0 con 0x5597eeeb7be0 2024-05-27T08:11:40.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.161+0000 7f56b0c00700 10 client.22719 handle_command_reply: tid=0 2024-05-27T08:11:40.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.161+0000 7f56d8306b80 1 client.22719 shutdown 2024-05-27T08:11:40.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.161+0000 7f56d8306b80 1 -- 172.21.15.63:0/1900649672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 msgr2=0x5597eee45520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:40.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.161+0000 7f56d8306b80 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee45520 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f56b400dfa0 tx=0x7f56b400dfd0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.162+0000 7f56d8306b80 1 -- 172.21.15.63:0/1900649672 shutdown_connections 2024-05-27T08:11:40.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.162+0000 7f56d8306b80 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5597eeeb7be0 0x5597eeeba000 crc :-1 s=READY pgs=679 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.162+0000 7f56d8306b80 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597eee32ef0 0x5597eee37450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.162+0000 7f56d8306b80 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5597eecb3ec0 0x5597eee45520 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.162+0000 7f56d8306b80 1 --2- 172.21.15.63:0/1900649672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5597eed9fa70 0x5597eee36f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.162+0000 7f56d8306b80 1 -- 172.21.15.63:0/1900649672 >> 172.21.15.63:0/1900649672 conn(0x5597eed4d410 msgr2=0x5597eee3d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:40.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.162+0000 7f56b0c00700 0 client.22719 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:40.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.162+0000 7f56d8306b80 1 -- 172.21.15.63:0/1900649672 shutdown_connections 2024-05-27T08:11:40.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.162+0000 7f56d8306b80 1 -- 172.21.15.63:0/1900649672 wait complete. 2024-05-27T08:11:40.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.162+0000 7f56d8306b80 20 client.22719 trim_cache size 0 max 16384 2024-05-27T08:11:40.164 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:11:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:11:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "4f4e8a12-f555-4873-8ef0-e3e7ab2c570a", 2024-05-27T08:11:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:11:40.164 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.163+0000 7f56c9800700 1 -- 172.21.15.63:0/3513157379 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56a80b29f0 msgr2=0x7f56a80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.163+0000 7f56c9800700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56a80b29f0 0x7f56a80b4f00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f56b4005950 tx=0x7f56b40058e0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.163+0000 7f56c9800700 1 -- 172.21.15.63:0/3513157379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56c4076670 msgr2=0x7f56c419d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.163+0000 7f56c9800700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56c4076670 0x7f56c419d4c0 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00ef10 tx=0x7f56ac00c3a0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.163+0000 7f56c9800700 1 -- 172.21.15.63:0/3513157379 shutdown_connections 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.163+0000 7f56c9800700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f56a80b29f0 0x7f56a80b4f00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.163+0000 7f56c9800700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c4110b90 0x7f56c41a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.163+0000 7f56c9800700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56c4075d20 0x7f56c419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.163+0000 7f56c9800700 1 --2- 172.21.15.63:0/3513157379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56c4076670 0x7f56c419d4c0 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.163+0000 7f56c9800700 1 -- 172.21.15.63:0/3513157379 >> 172.21.15.63:0/3513157379 conn(0x7f56c4100030 msgr2=0x7f56c41006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:40.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.164+0000 7f56c9800700 1 -- 172.21.15.63:0/3513157379 shutdown_connections 2024-05-27T08:11:40.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:40.164+0000 7f56c9800700 1 -- 172.21.15.63:0/3513157379 wait complete. 2024-05-27T08:11:40.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:39 smithi086 ceph-mon[34687]: pgmap v1208: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:11:40.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:39 smithi149 ceph-mon[35408]: pgmap v1208: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:11:40.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:11:40.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:40.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:11:40.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:11:40.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:40.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:11:40.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:40.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:11:40.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:40.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:40.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:40 smithi063 ceph-mon[31613]: pgmap v1209: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:11:41.101 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:11:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:11:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:11:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:11:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:11:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:11:41.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:41.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:41.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:40 smithi086 ceph-mon[34687]: pgmap v1209: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:40 smithi149 ceph-mon[35408]: pgmap v1209: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:11:41.668 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:11:42.111 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:41 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:11:42.112 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:41 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:42.112 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:41 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:11:42.112 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:41 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:42.112 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:41 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:11:42.112 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:41 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:42.177 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:41 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:11:42.177 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:41 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:42.177 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:41 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:11:42.177 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:41 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:42.177 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:41 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:11:42.178 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:41 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:42.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:41 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:11:42.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:41 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:42.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:41 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:11:42.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:41 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:42.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:41 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:11:42.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:41 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:11:43.105 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:42 smithi063 ceph-mon[31613]: pgmap v1210: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.0 MiB/s wr, 48 op/s 2024-05-27T08:11:43.143 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:42 smithi086 ceph-mon[34687]: pgmap v1210: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.0 MiB/s wr, 48 op/s 2024-05-27T08:11:43.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:42 smithi149 ceph-mon[35408]: pgmap v1210: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.0 MiB/s wr, 48 op/s 2024-05-27T08:11:44.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.072+0000 7f3bf5400700 1 -- 172.21.15.63:0/975572370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf0105c50 msgr2=0x7f3bf010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:44.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.072+0000 7f3bf5400700 1 --2- 172.21.15.63:0/975572370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf0105c50 0x7f3bf010e1a0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009990 tx=0x7f3bdc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.072+0000 7f3bf5400700 1 -- 172.21.15.63:0/975572370 shutdown_connections 2024-05-27T08:11:44.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.072+0000 7f3bf5400700 1 --2- 172.21.15.63:0/975572370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf010e730 0x7f3bf0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.072+0000 7f3bf5400700 1 --2- 172.21.15.63:0/975572370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bf0105300 0x7f3bf0105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.072+0000 7f3bf5400700 1 --2- 172.21.15.63:0/975572370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf0105c50 0x7f3bf010e1a0 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.072+0000 7f3bf5400700 1 -- 172.21.15.63:0/975572370 >> 172.21.15.63:0/975572370 conn(0x7f3bf0078c30 msgr2=0x7f3bf0079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.072+0000 7f3bf5400700 1 -- 172.21.15.63:0/975572370 shutdown_connections 2024-05-27T08:11:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.073+0000 7f3bf5400700 1 -- 172.21.15.63:0/975572370 wait complete. 2024-05-27T08:11:44.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.074+0000 7f3bf5400700 1 Processor -- start 2024-05-27T08:11:44.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.074+0000 7f3bf5400700 1 -- start start 2024-05-27T08:11:44.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.075+0000 7f3bf5400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bf0105300 0x7f3bf0106210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.075+0000 7f3bf5400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf0105c50 0x7f3bf0106750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.075+0000 7f3bf5400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf010e730 0x7f3bf0106c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.075+0000 7f3bf5400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0114190 con 0x7f3bf0105c50 2024-05-27T08:11:44.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.075+0000 7f3bf5400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0114010 con 0x7f3bf010e730 2024-05-27T08:11:44.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.075+0000 7f3bf5400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0114310 con 0x7f3bf0105300 2024-05-27T08:11:44.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.075+0000 7f3bef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf0105c50 0x7f3bf0106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.075+0000 7f3bef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf0105c50 0x7f3bf0106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34060/0 (socket says 172.21.15.63:34060) 2024-05-27T08:11:44.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.075+0000 7f3bef400700 1 -- 172.21.15.63:0/1437063669 learned_addr learned my addr 172.21.15.63:0/1437063669 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.075+0000 7f3befe00700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bf0105300 0x7f3bf0106210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.076+0000 7f3bf4a00700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf010e730 0x7f3bf0106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.076+0000 7f3bef400700 1 -- 172.21.15.63:0/1437063669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bf0105300 msgr2=0x7f3bf0106210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.076+0000 7f3bef400700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bf0105300 0x7f3bf0106210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.076+0000 7f3bef400700 1 -- 172.21.15.63:0/1437063669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf010e730 msgr2=0x7f3bf0106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.076+0000 7f3bef400700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf010e730 0x7f3bf0106c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.076+0000 7f3bef400700 1 -- 172.21.15.63:0/1437063669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf01ad3c0 con 0x7f3bf0105c50 2024-05-27T08:11:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.076+0000 7f3bf4a00700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf010e730 0x7f3bf0106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:44.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.077+0000 7f3bef400700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf0105c50 0x7f3bf0106750 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc005910 tx=0x7f3bdc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.077+0000 7f3becc00700 1 -- 172.21.15.63:0/1437063669 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc004830 con 0x7f3bf0105c50 2024-05-27T08:11:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.077+0000 7f3becc00700 1 -- 172.21.15.63:0/1437063669 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bdc005020 con 0x7f3bf0105c50 2024-05-27T08:11:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.077+0000 7f3bf5400700 1 -- 172.21.15.63:0/1437063669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf01ad5f0 con 0x7f3bf0105c50 2024-05-27T08:11:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.077+0000 7f3becc00700 1 -- 172.21.15.63:0/1437063669 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00f960 con 0x7f3bf0105c50 2024-05-27T08:11:44.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.077+0000 7f3bf5400700 1 -- 172.21.15.63:0/1437063669 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf01adac0 con 0x7f3bf0105c50 2024-05-27T08:11:44.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.082+0000 7f3c0557ab80 1 Processor -- start 2024-05-27T08:11:44.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.083+0000 7f3c0557ab80 1 -- start start 2024-05-27T08:11:44.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.083+0000 7f3becc00700 1 -- 172.21.15.63:0/1437063669 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3bdc0049d0 con 0x7f3bf0105c50 2024-05-27T08:11:44.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.083+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1d179f0 0x55ffa1dab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.083+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1dabc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.083+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1db76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.083+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ffa1cd8e40 con 0x55ffa1db52c0 2024-05-27T08:11:44.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.083+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ffa1cc89e0 con 0x55ffa1c2be40 2024-05-27T08:11:44.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.083+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ffa1ccca20 con 0x55ffa1d179f0 2024-05-27T08:11:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.083+0000 7f3bef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1db76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.084+0000 7f3bef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1db76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34068/0 (socket says 172.21.15.63:34068) 2024-05-27T08:11:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.084+0000 7f3bef400700 1 -- 172.21.15.63:0/3927204199 learned_addr learned my addr 172.21.15.63:0/3927204199 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.084+0000 7f3bf4a00700 1 --2- 172.21.15.63:0/3927204199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1d179f0 0x55ffa1dab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.084+0000 7f3befe00700 1 --2- 172.21.15.63:0/3927204199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1dabc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.084+0000 7f3bef400700 1 -- 172.21.15.63:0/3927204199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1d179f0 msgr2=0x55ffa1dab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.084+0000 7f3bef400700 1 --2- 172.21.15.63:0/3927204199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1d179f0 0x55ffa1dab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.084+0000 7f3bef400700 1 -- 172.21.15.63:0/3927204199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 msgr2=0x55ffa1dabc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.084+0000 7f3bef400700 1 --2- 172.21.15.63:0/3927204199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1dabc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.084+0000 7f3bef400700 1 -- 172.21.15.63:0/3927204199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa1c91420 con 0x55ffa1db52c0 2024-05-27T08:11:44.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.084+0000 7f3befe00700 1 --2- 172.21.15.63:0/3927204199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1dabc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:11:44.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.085+0000 7f3bef400700 1 --2- 172.21.15.63:0/3927204199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1db76e0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0e0e90 tx=0x7f3bdc0ec5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd8c9bf7c0947 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:44.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.085+0000 7f3be5600700 1 -- 172.21.15.63:0/3927204199 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0f6400 con 0x55ffa1db52c0 2024-05-27T08:11:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.085+0000 7f3becc00700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd40b2a70 0x7f3bd40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.085+0000 7f3be5600700 1 -- 172.21.15.63:0/3927204199 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bdc0f6a30 con 0x55ffa1db52c0 2024-05-27T08:11:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.085+0000 7f3be5600700 1 -- 172.21.15.63:0/3927204199 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0a71c0 con 0x55ffa1db52c0 2024-05-27T08:11:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.086+0000 7f3becc00700 1 -- 172.21.15.63:0/1437063669 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f3bdc0e3c20 con 0x7f3bf0105c50 2024-05-27T08:11:44.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.086+0000 7f3bf4a00700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd40b2a70 0x7f3bd40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.087+0000 7f3c0557ab80 1 -- 172.21.15.63:0/3927204199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 msgr2=0x55ffa1db76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.087+0000 7f3bf4a00700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd40b2a70 0x7f3bd40b4f80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55ffa1da7a90 tx=0x7f3be8009040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.087+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/3927204199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1db76e0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0e0e90 tx=0x7f3bdc0ec5f0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.087+0000 7f3c0557ab80 1 -- 172.21.15.63:0/3927204199 shutdown_connections 2024-05-27T08:11:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.087+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/3927204199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1db76e0 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.087+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/3927204199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1d179f0 0x55ffa1dab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.087+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/3927204199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1dabc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.087+0000 7f3c0557ab80 1 -- 172.21.15.63:0/3927204199 >> 172.21.15.63:0/3927204199 conn(0x55ffa1cc5390 msgr2=0x55ffa1da6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.087+0000 7f3c0557ab80 1 -- 172.21.15.63:0/3927204199 shutdown_connections 2024-05-27T08:11:44.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.087+0000 7f3c0557ab80 1 -- 172.21.15.63:0/3927204199 wait complete. 2024-05-27T08:11:44.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.088+0000 7f3c0557ab80 1 -- start start 2024-05-27T08:11:44.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.088+0000 7f3c0557ab80 1 Processor -- start 2024-05-27T08:11:44.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1db2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1db52c0 0x55ffa1db2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ffa1bcefa0 con 0x55ffa1d179f0 2024-05-27T08:11:44.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ffa1c69760 con 0x55ffa1c2be40 2024-05-27T08:11:44.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ffa1db7c20 con 0x55ffa1db52c0 2024-05-27T08:11:44.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3c0557ab80 10 client.0 _ll_register_callbacks cb 0x55ffa1c72c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3bef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3bef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34074/0 (socket says 172.21.15.63:34074) 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3bef400700 1 -- 172.21.15.63:0/2857066067 learned_addr learned my addr 172.21.15.63:0/2857066067 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.089+0000 7f3befe00700 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1db2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3bf4a00700 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1db52c0 0x55ffa1db2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3bef400700 1 -- 172.21.15.63:0/2857066067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1db52c0 msgr2=0x55ffa1db2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3bef400700 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1db52c0 0x55ffa1db2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3bef400700 1 -- 172.21.15.63:0/2857066067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 msgr2=0x55ffa1db2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3bef400700 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1db2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3bef400700 1 -- 172.21.15.63:0/2857066067 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa1c188d0 con 0x55ffa1d179f0 2024-05-27T08:11:44.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3befe00700 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1db2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3bef400700 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db2790 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0aab00 tx=0x7f3bdc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661884388ec0d2cd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3bda000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3bda000700 1 -- 172.21.15.63:0/2857066067 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0aa4c0 con 0x55ffa1d179f0 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.090+0000 7f3bda000700 1 -- 172.21.15.63:0/2857066067 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f3bdc0054d0 con 0x55ffa1d179f0 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.091+0000 7f3c0557ab80 1 -- 172.21.15.63:0/2857066067 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bdc0f6a30 con 0x55ffa1d179f0 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.091+0000 7f3bda000700 1 -- 172.21.15.63:0/2857066067 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3bdc0ae050 con 0x55ffa1d179f0 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.091+0000 7f3c0557ab80 10 client.22731 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.091+0000 7f3c0557ab80 1 -- 172.21.15.63:0/2857066067 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ffa1c99060 con 0x55ffa1d179f0 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.091+0000 7f3bda000700 1 -- 172.21.15.63:0/2857066067 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bdc0f6a30 con 0x55ffa1d179f0 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.092+0000 7f3bda000700 1 -- 172.21.15.63:0/2857066067 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3bdc0ea070 con 0x55ffa1d179f0 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.092+0000 7f3c0557ab80 10 client.22731 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:44.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.092+0000 7f3c0557ab80 10 client.22731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:44.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.092+0000 7f3c0557ab80 20 client.22731 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:44.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.092+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ffa1dc4850 0x55ffa1dc6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.092+0000 7f3c0557ab80 4 client.22731 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:11:44.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.092+0000 7f3c0557ab80 1 -- 172.21.15.63:0/2857066067 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffa1cdad90 con 0x55ffa1dc4850 2024-05-27T08:11:44.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.092+0000 7f3befe00700 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ffa1dc4850 0x55ffa1dc6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.093+0000 7f3befe00700 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ffa1dc4850 0x55ffa1dc6c70 crc :-1 s=READY pgs=680 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c86f97010aa7400 server_cookie=46491864389d8799 in_seq=0 out_seq=0 2024-05-27T08:11:44.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.093+0000 7f3bda000700 10 client.22731 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:44.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.095+0000 7f3bda000700 1 -- 172.21.15.63:0/2857066067 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ffa1cdad90 con 0x55ffa1dc4850 2024-05-27T08:11:44.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.095+0000 7f3bda000700 10 client.22731 handle_command_reply: tid=0 2024-05-27T08:11:44.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.095+0000 7f3c0557ab80 1 client.22731 shutdown 2024-05-27T08:11:44.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.095+0000 7f3c0557ab80 1 -- 172.21.15.63:0/2857066067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 msgr2=0x55ffa1db2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:44.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.095+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db2790 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0aab00 tx=0x7f3bdc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.096+0000 7f3c0557ab80 1 -- 172.21.15.63:0/2857066067 shutdown_connections 2024-05-27T08:11:44.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.096+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ffa1dc4850 0x55ffa1dc6c70 crc :-1 s=READY pgs=680 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.096+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1db52c0 0x55ffa1db2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.096+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1db2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.096+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/2857066067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db2790 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.096+0000 7f3bda000700 0 client.22731 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:44.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.096+0000 7f3c0557ab80 1 -- 172.21.15.63:0/2857066067 >> 172.21.15.63:0/2857066067 conn(0x55ffa1cc5390 msgr2=0x55ffa1dab820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:44.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.096+0000 7f3c0557ab80 1 -- 172.21.15.63:0/2857066067 shutdown_connections 2024-05-27T08:11:44.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.096+0000 7f3c0557ab80 1 -- 172.21.15.63:0/2857066067 wait complete. 2024-05-27T08:11:44.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.096+0000 7f3c0557ab80 20 client.22731 trim_cache size 0 max 16384 2024-05-27T08:11:44.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.123+0000 7f3c0557ab80 1 Processor -- start 2024-05-27T08:11:44.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.124+0000 7f3c0557ab80 1 -- start start 2024-05-27T08:11:44.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.124+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1dad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.124+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1d179f0 0x55ffa1dc0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.124+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1dcc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.124+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01b91c0 con 0x55ffa1db52c0 2024-05-27T08:11:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.124+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01b6a50 con 0x55ffa1c2be40 2024-05-27T08:11:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.124+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01b48c0 con 0x55ffa1d179f0 2024-05-27T08:11:44.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3bf4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1dcc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3befe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1dad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3bf4a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1dcc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34090/0 (socket says 172.21.15.63:34090) 2024-05-27T08:11:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3bf4a00700 1 -- 172.21.15.63:0/3814731652 learned_addr learned my addr 172.21.15.63:0/3814731652 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3bef400700 1 --2- 172.21.15.63:0/3814731652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1d179f0 0x55ffa1dc0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3bf4a00700 1 -- 172.21.15.63:0/3814731652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1d179f0 msgr2=0x55ffa1dc0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3bf4a00700 1 --2- 172.21.15.63:0/3814731652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1d179f0 0x55ffa1dc0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3bf4a00700 1 -- 172.21.15.63:0/3814731652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 msgr2=0x55ffa1dad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3bf4a00700 1 --2- 172.21.15.63:0/3814731652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1dad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3bf4a00700 1 -- 172.21.15.63:0/3814731652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa1dacd60 con 0x55ffa1db52c0 2024-05-27T08:11:44.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.125+0000 7f3befe00700 1 --2- 172.21.15.63:0/3814731652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1dad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:11:44.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.126+0000 7f3bf4a00700 1 --2- 172.21.15.63:0/3814731652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1dcc3a0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f3be8018730 tx=0x7f3be8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3743c733a8c0db0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:44.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.126+0000 7f3be4c00700 1 -- 172.21.15.63:0/3814731652 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be801e030 con 0x55ffa1db52c0 2024-05-27T08:11:44.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.126+0000 7f3be4c00700 1 -- 172.21.15.63:0/3814731652 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3be8029070 con 0x55ffa1db52c0 2024-05-27T08:11:44.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.126+0000 7f3be4c00700 1 -- 172.21.15.63:0/3814731652 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8016800 con 0x55ffa1db52c0 2024-05-27T08:11:44.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.127+0000 7f3c0557ab80 1 -- 172.21.15.63:0/3814731652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 msgr2=0x55ffa1dcc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:44.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.127+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/3814731652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1dcc3a0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f3be8018730 tx=0x7f3be8015000 comp rx=0 tx=0).stop 2024-05-27T08:11:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.127+0000 7f3c0557ab80 1 -- 172.21.15.63:0/3814731652 shutdown_connections 2024-05-27T08:11:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.127+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/3814731652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1db52c0 0x55ffa1dcc3a0 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.127+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/3814731652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1c2be40 0x55ffa1dad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.127+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/3814731652 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1d179f0 0x55ffa1dc0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.127+0000 7f3c0557ab80 1 -- 172.21.15.63:0/3814731652 >> 172.21.15.63:0/3814731652 conn(0x55ffa1cc5390 msgr2=0x55ffa1dad170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.127+0000 7f3c0557ab80 1 -- 172.21.15.63:0/3814731652 shutdown_connections 2024-05-27T08:11:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.127+0000 7f3c0557ab80 1 -- 172.21.15.63:0/3814731652 wait complete. 2024-05-27T08:11:44.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.128+0000 7f3c0557ab80 1 -- start start 2024-05-27T08:11:44.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.128+0000 7f3c0557ab80 1 Processor -- start 2024-05-27T08:11:44.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1c2be40 0x55ffa1db8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3c0557ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1db52c0 0x55ffa1db97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01b4f10 con 0x55ffa1d179f0 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01b6380 con 0x55ffa1db52c0 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3c0557ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01b55e0 con 0x55ffa1c2be40 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3c0557ab80 10 client.0 _ll_register_callbacks cb 0x55ffa1c72c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3bef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3bef400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34100/0 (socket says 172.21.15.63:34100) 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3bef400700 1 -- 172.21.15.63:0/4096037768 learned_addr learned my addr 172.21.15.63:0/4096037768 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3bf4a00700 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1db52c0 0x55ffa1db97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.129+0000 7f3befe00700 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1c2be40 0x55ffa1db8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.130+0000 7f3bef400700 1 -- 172.21.15.63:0/4096037768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1c2be40 msgr2=0x55ffa1db8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.130+0000 7f3bef400700 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1c2be40 0x55ffa1db8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.130+0000 7f3bef400700 1 -- 172.21.15.63:0/4096037768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1db52c0 msgr2=0x55ffa1db97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:44.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.130+0000 7f3bef400700 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1db52c0 0x55ffa1db97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.130+0000 7f3bef400700 1 -- 172.21.15.63:0/4096037768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa1e2e3b0 con 0x55ffa1d179f0 2024-05-27T08:11:44.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.130+0000 7f3befe00700 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1c2be40 0x55ffa1db8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:44.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.130+0000 7f3bef400700 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db92b0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x55ffa1da7180 tx=0x7f3bdc0ec5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140ce185e32118f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:44.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.130+0000 7f3bda000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:44.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.130+0000 7f3bda000700 1 -- 172.21.15.63:0/4096037768 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0aa6b0 con 0x55ffa1d179f0 2024-05-27T08:11:44.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.131+0000 7f3bda000700 1 -- 172.21.15.63:0/4096037768 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f3bdc0e11d0 con 0x55ffa1d179f0 2024-05-27T08:11:44.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.131+0000 7f3c0557ab80 1 -- 172.21.15.63:0/4096037768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be8029070 con 0x55ffa1d179f0 2024-05-27T08:11:44.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.131+0000 7f3bda000700 1 -- 172.21.15.63:0/4096037768 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3bdc0aa940 con 0x55ffa1d179f0 2024-05-27T08:11:44.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.131+0000 7f3c0557ab80 10 client.22737 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:44.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.131+0000 7f3c0557ab80 1 -- 172.21.15.63:0/4096037768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ffa1e2e3b0 con 0x55ffa1d179f0 2024-05-27T08:11:44.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.131+0000 7f3bda000700 1 -- 172.21.15.63:0/4096037768 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3be8029070 con 0x55ffa1d179f0 2024-05-27T08:11:44.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.132+0000 7f3bda000700 1 -- 172.21.15.63:0/4096037768 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3bdc0ea070 con 0x55ffa1d179f0 2024-05-27T08:11:44.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.132+0000 7f3c0557ab80 10 client.22737 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.132+0000 7f3c0557ab80 10 client.22737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.132+0000 7f3c0557ab80 20 client.22737 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.132+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ffa1e2f910 0x55ffa1e31d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.132+0000 7f3c0557ab80 4 client.22737 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:11:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.132+0000 7f3c0557ab80 1 -- 172.21.15.63:0/4096037768 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ffa1cd4570 con 0x55ffa1e2f910 2024-05-27T08:11:44.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.132+0000 7f3befe00700 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ffa1e2f910 0x55ffa1e31d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.133+0000 7f3befe00700 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ffa1e2f910 0x55ffa1e31d30 crc :-1 s=READY pgs=681 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d501950761f79a server_cookie=165fe42c897b12f7 in_seq=0 out_seq=0 2024-05-27T08:11:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.133+0000 7f3bda000700 10 client.22737 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3bda000700 1 -- 172.21.15.63:0/4096037768 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ffa1cd4570 con 0x55ffa1e2f910 2024-05-27T08:11:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3bda000700 10 client.22737 handle_command_reply: tid=0 2024-05-27T08:11:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 client.22737 shutdown 2024-05-27T08:11:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 -- 172.21.15.63:0/4096037768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 msgr2=0x55ffa1db92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db92b0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x55ffa1da7180 tx=0x7f3bdc0ec5f0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 -- 172.21.15.63:0/4096037768 shutdown_connections 2024-05-27T08:11:44.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ffa1e2f910 0x55ffa1e31d30 crc :-1 s=READY pgs=681 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ffa1db52c0 0x55ffa1db97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ffa1c2be40 0x55ffa1db8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 --2- 172.21.15.63:0/4096037768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ffa1d179f0 0x55ffa1db92b0 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 -- 172.21.15.63:0/4096037768 >> 172.21.15.63:0/4096037768 conn(0x55ffa1cc5390 msgr2=0x55ffa1dad900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:44.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3bda000700 0 client.22737 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:44.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 -- 172.21.15.63:0/4096037768 shutdown_connections 2024-05-27T08:11:44.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 1 -- 172.21.15.63:0/4096037768 wait complete. 2024-05-27T08:11:44.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.134+0000 7f3c0557ab80 20 client.22737 trim_cache size 0 max 16384 2024-05-27T08:11:44.136 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:11:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:11:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:11:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "4f4e8a12-f555-4873-8ef0-e3e7ab2c570a": { 2024-05-27T08:11:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:11:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "4f4e8a12-f555-4873-8ef0-e3e7ab2c570a", 2024-05-27T08:11:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:11:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:11:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:11:44.137 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:11:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.135+0000 7f3bf5400700 1 -- 172.21.15.63:0/1437063669 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd40b2a70 msgr2=0x7f3bd40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.135+0000 7f3bf5400700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd40b2a70 0x7f3bd40b4f80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55ffa1da7a90 tx=0x7f3be8009040 comp rx=0 tx=0).stop 2024-05-27T08:11:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.135+0000 7f3bf5400700 1 -- 172.21.15.63:0/1437063669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf0105c50 msgr2=0x7f3bf0106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.135+0000 7f3bf5400700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf0105c50 0x7f3bf0106750 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc005910 tx=0x7f3bdc0047b0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.136+0000 7f3bf5400700 1 -- 172.21.15.63:0/1437063669 shutdown_connections 2024-05-27T08:11:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.136+0000 7f3bf5400700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3bd40b2a70 0x7f3bd40b4f80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.136+0000 7f3bf5400700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf010e730 0x7f3bf0106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.136+0000 7f3bf5400700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3bf0105300 0x7f3bf0106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.136+0000 7f3bf5400700 1 --2- 172.21.15.63:0/1437063669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf0105c50 0x7f3bf0106750 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:44.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.136+0000 7f3bf5400700 1 -- 172.21.15.63:0/1437063669 >> 172.21.15.63:0/1437063669 conn(0x7f3bf0078c30 msgr2=0x7f3bf00ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:44.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.136+0000 7f3bf5400700 1 -- 172.21.15.63:0/1437063669 shutdown_connections 2024-05-27T08:11:44.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:11:44.136+0000 7f3bf5400700 1 -- 172.21.15.63:0/1437063669 wait complete. 2024-05-27T08:11:45.149 INFO:tasks.cephfs.filesystem:scrub status for tag:4f4e8a12-f555-4873-8ef0-e3e7ab2c570a - {'path': '/', 'tag': '4f4e8a12-f555-4873-8ef0-e3e7ab2c570a', 'options': 'recursive,force'} 2024-05-27T08:11:46.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:45 smithi086 ceph-mon[34687]: pgmap v1211: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.7 MiB/s wr, 44 op/s 2024-05-27T08:11:46.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:45 smithi149 ceph-mon[35408]: pgmap v1211: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.7 MiB/s wr, 44 op/s 2024-05-27T08:11:46.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:45 smithi063 ceph-mon[31613]: pgmap v1211: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.7 MiB/s wr, 44 op/s 2024-05-27T08:11:47.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:46 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:11:47.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:46 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:11:47.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:46 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:11:48.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:47 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:11:48.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:47 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:11:48.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:47 smithi086 ceph-mon[34687]: pgmap v1212: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:11:48.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:47 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:11:48.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:47 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:11:48.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:47 smithi149 ceph-mon[35408]: pgmap v1212: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:11:48.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:47 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:11:48.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:47 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:11:48.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:47 smithi063 ceph-mon[31613]: pgmap v1212: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:11:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.000+0000 7f22f6a00640 1 -- 172.21.15.63:0/2068510219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22f010c190 msgr2=0x7f22f0105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.000+0000 7f22f6a00640 1 --2- 172.21.15.63:0/2068510219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22f010c190 0x7f22f0105dd0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009900 tx=0x7f22dc02a3f0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.000+0000 7f22f6a00640 1 -- 172.21.15.63:0/2068510219 shutdown_connections 2024-05-27T08:11:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.000+0000 7f22f6a00640 1 --2- 172.21.15.63:0/2068510219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22f0106440 0x7f22f0112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.000+0000 7f22f6a00640 1 --2- 172.21.15.63:0/2068510219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22f010c190 0x7f22f0105dd0 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.000+0000 7f22f6a00640 1 --2- 172.21.15.63:0/2068510219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22f010bcf0 0x7f22f0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.000+0000 7f22f6a00640 1 -- 172.21.15.63:0/2068510219 >> 172.21.15.63:0/2068510219 conn(0x7f22f007d510 msgr2=0x7f22f007d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.000+0000 7f22f6a00640 1 -- 172.21.15.63:0/2068510219 shutdown_connections 2024-05-27T08:11:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.001+0000 7f22f6a00640 1 -- 172.21.15.63:0/2068510219 wait complete. 2024-05-27T08:11:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.002+0000 7f22f6a00640 1 Processor -- start 2024-05-27T08:11:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.002+0000 7f22f6a00640 1 -- start start 2024-05-27T08:11:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.002+0000 7f22f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22f0106440 0x7f22f01a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.002+0000 7f22f6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22f010bcf0 0x7f22f01a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.002+0000 7f22f6a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22f010c190 0x7f22f019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.002+0000 7f22f6a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0115d90 con 0x7f22f010c190 2024-05-27T08:11:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.002+0000 7f22f6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0115c10 con 0x7f22f0106440 2024-05-27T08:11:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.002+0000 7f22f6a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0115f10 con 0x7f22f010bcf0 2024-05-27T08:11:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.003+0000 7f22f6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22f010c190 0x7f22f019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.003+0000 7f22f6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22f010c190 0x7f22f019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51652/0 (socket says 172.21.15.63:51652) 2024-05-27T08:11:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.003+0000 7f22f6000640 1 -- 172.21.15.63:0/2484926153 learned_addr learned my addr 172.21.15.63:0/2484926153 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.003+0000 7f22f5600640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22f0106440 0x7f22f01a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.003+0000 7f22f4c00640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22f010bcf0 0x7f22f01a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.003+0000 7f22f6000640 1 -- 172.21.15.63:0/2484926153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22f010bcf0 msgr2=0x7f22f01a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.003+0000 7f22f6000640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22f010bcf0 0x7f22f01a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.003+0000 7f22f6000640 1 -- 172.21.15.63:0/2484926153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22f0106440 msgr2=0x7f22f01a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.003+0000 7f22f6000640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22f0106440 0x7f22f01a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.003+0000 7f22f6000640 1 -- 172.21.15.63:0/2484926153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f019cda0 con 0x7f22f010c190 2024-05-27T08:11:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.004+0000 7f22f6000640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22f010c190 0x7f22f019c670 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009fd0 tx=0x7f22e800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.004+0000 7f22ee000640 1 -- 172.21.15.63:0/2484926153 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8017070 con 0x7f22f010c190 2024-05-27T08:11:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.004+0000 7f22f6a00640 1 -- 172.21.15.63:0/2484926153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f019d030 con 0x7f22f010c190 2024-05-27T08:11:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.004+0000 7f22ee000640 1 -- 172.21.15.63:0/2484926153 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e8003e00 con 0x7f22f010c190 2024-05-27T08:11:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.004+0000 7f22ee000640 1 -- 172.21.15.63:0/2484926153 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8010460 con 0x7f22f010c190 2024-05-27T08:11:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.004+0000 7f22f6a00640 1 -- 172.21.15.63:0/2484926153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f01a9290 con 0x7f22f010c190 2024-05-27T08:11:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.009+0000 7f22ff02b740 1 Processor -- start 2024-05-27T08:11:49.009 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.009+0000 7f22ee000640 1 -- 172.21.15.63:0/2484926153 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f22e8003970 con 0x7f22f010c190 2024-05-27T08:11:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.009+0000 7f22ff02b740 1 -- start start 2024-05-27T08:11:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.009+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed691b0 0x56103f10ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.009+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f10bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.009+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f10c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.009+0000 7f22ff02b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56103f07cb90 con 0x56103ed484d0 2024-05-27T08:11:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.009+0000 7f22ff02b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56103f06c730 con 0x56103ed402f0 2024-05-27T08:11:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.009+0000 7f22ff02b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56103f070770 con 0x56103ed691b0 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.010+0000 7f22f4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f10c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.010+0000 7f22f4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f10c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51666/0 (socket says 172.21.15.63:51666) 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.010+0000 7f22f4c00640 1 -- 172.21.15.63:0/3952785912 learned_addr learned my addr 172.21.15.63:0/3952785912 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.010+0000 7f22f5600640 1 --2- 172.21.15.63:0/3952785912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f10bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.010+0000 7f22f4c00640 1 --2- 172.21.15.63:0/3952785912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed691b0 0x56103f10ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.011+0000 7f22f5600640 1 -- 172.21.15.63:0/3952785912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed691b0 msgr2=0x56103f10ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.011+0000 7f22f5600640 1 --2- 172.21.15.63:0/3952785912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed691b0 0x56103f10ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.011+0000 7f22f5600640 1 -- 172.21.15.63:0/3952785912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 msgr2=0x56103f10c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.011+0000 7f22f5600640 1 --2- 172.21.15.63:0/3952785912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f10c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.011+0000 7f22f5600640 1 -- 172.21.15.63:0/3952785912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56103f0058b0 con 0x56103ed402f0 2024-05-27T08:11:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.011+0000 7f22f4c00640 1 --2- 172.21.15.63:0/3952785912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:11:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.011+0000 7f22ee000640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22d80b2950 0x7f22d80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.011+0000 7f22ee000640 1 -- 172.21.15.63:0/2484926153 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f22e80e2c10 con 0x7f22f010c190 2024-05-27T08:11:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.011+0000 7f22f4c00640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22d80b2950 0x7f22d80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.011+0000 7f22f5600640 1 --2- 172.21.15.63:0/3952785912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f10bfa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009bd0 tx=0x7f22e4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a99d254674b2d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.012+0000 7f22e2600640 1 -- 172.21.15.63:0/3952785912 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4004050 con 0x56103ed402f0 2024-05-27T08:11:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.012+0000 7f22e2600640 1 -- 172.21.15.63:0/3952785912 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e4005020 con 0x56103ed402f0 2024-05-27T08:11:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.012+0000 7f22e2600640 1 -- 172.21.15.63:0/3952785912 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4004a90 con 0x56103ed402f0 2024-05-27T08:11:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.013+0000 7f22ff02b740 1 -- 172.21.15.63:0/3952785912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 msgr2=0x56103f10bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.013+0000 7f22f4c00640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22d80b2950 0x7f22d80b4e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x56103f108bd0 tx=0x7f22dc040040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.013+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3952785912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f10bfa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009bd0 tx=0x7f22e4011810 comp rx=0 tx=0).stop 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.013+0000 7f22ff02b740 1 -- 172.21.15.63:0/3952785912 shutdown_connections 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.013+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3952785912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.013+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3952785912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f10bfa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.013+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3952785912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed691b0 0x56103f10ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.013+0000 7f22ff02b740 1 -- 172.21.15.63:0/3952785912 >> 172.21.15.63:0/3952785912 conn(0x56103ed5d300 msgr2=0x56103f108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.013+0000 7f22ff02b740 1 -- 172.21.15.63:0/3952785912 shutdown_connections 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.013+0000 7f22ff02b740 1 -- 172.21.15.63:0/3952785912 wait complete. 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.014+0000 7f22ff02b740 1 -- start start 2024-05-27T08:11:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.014+0000 7f22ff02b740 1 Processor -- start 2024-05-27T08:11:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.015+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed402f0 0x56103f115be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.015+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f116120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.015+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed691b0 0x56103f121ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.015+0000 7f22ff02b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56103ed52790 con 0x56103ed484d0 2024-05-27T08:11:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.015+0000 7f22ff02b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56103f014bc0 con 0x56103ed691b0 2024-05-27T08:11:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.015+0000 7f22ff02b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56103f118b90 con 0x56103ed402f0 2024-05-27T08:11:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.015+0000 7f22ff02b740 10 client.0 _ll_register_callbacks cb 0x56103eff2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.015+0000 7f22f4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.015+0000 7f22f4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f116120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51680/0 (socket says 172.21.15.63:51680) 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.015+0000 7f22f4c00640 1 -- 172.21.15.63:0/3301722594 learned_addr learned my addr 172.21.15.63:0/3301722594 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.016+0000 7f22f5600640 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed402f0 0x56103f115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.016+0000 7f22f6000640 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed691b0 0x56103f121ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.016+0000 7f22f4c00640 1 -- 172.21.15.63:0/3301722594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed402f0 msgr2=0x56103f115be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.016+0000 7f22f4c00640 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed402f0 0x56103f115be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.016+0000 7f22f4c00640 1 -- 172.21.15.63:0/3301722594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed691b0 msgr2=0x56103f121ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.016+0000 7f22f4c00640 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed691b0 0x56103f121ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.016+0000 7f22f4c00640 1 -- 172.21.15.63:0/3301722594 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56103ed81210 con 0x56103ed484d0 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.016+0000 7f22f6000640 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed691b0 0x56103f121ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.016+0000 7f22f4c00640 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f116120 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f22dc032ca0 tx=0x7f22dc02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebde9bc7183640 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.016+0000 7f22d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.017+0000 7f22d6a00640 1 -- 172.21.15.63:0/3301722594 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc035040 con 0x56103ed484d0 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.017+0000 7f22ff02b740 1 -- 172.21.15.63:0/3301722594 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e4005020 con 0x56103ed484d0 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.017+0000 7f22d6a00640 1 -- 172.21.15.63:0/3301722594 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f22dc003b40 con 0x56103ed484d0 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.017+0000 7f22d6a00640 1 -- 172.21.15.63:0/3301722594 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f22dc003d40 con 0x56103ed484d0 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.017+0000 7f22ff02b740 10 client.22749 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.017+0000 7f22ff02b740 1 -- 172.21.15.63:0/3301722594 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56103ee4d620 con 0x56103ed484d0 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.018+0000 7f22d6a00640 1 -- 172.21.15.63:0/3301722594 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e4005020 con 0x56103ed484d0 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.018+0000 7f22d6a00640 1 -- 172.21.15.63:0/3301722594 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f22dc03a040 con 0x56103ed484d0 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.018+0000 7f22ff02b740 10 client.22749 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.018+0000 7f22ff02b740 10 client.22749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.018+0000 7f22ff02b740 20 client.22749 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.018+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56103ed27d80 0x56103f127e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.018+0000 7f22ff02b740 4 client.22749 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:11:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.018+0000 7f22ff02b740 1 -- 172.21.15.63:0/3301722594 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56103f07eae0 con 0x56103ed27d80 2024-05-27T08:11:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.019+0000 7f22f5600640 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56103ed27d80 0x56103f127e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.020+0000 7f22f5600640 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56103ed27d80 0x56103f127e60 crc :-1 s=READY pgs=682 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b98c7b4286dec6 server_cookie=b1a9ac90ae5c96b2 in_seq=0 out_seq=0 2024-05-27T08:11:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.020+0000 7f22d6a00640 10 client.22749 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22d6a00640 1 -- 172.21.15.63:0/3301722594 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56103f07eae0 con 0x56103ed27d80 2024-05-27T08:11:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22d6a00640 10 client.22749 handle_command_reply: tid=0 2024-05-27T08:11:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22ff02b740 1 client.22749 shutdown 2024-05-27T08:11:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22ff02b740 1 -- 172.21.15.63:0/3301722594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 msgr2=0x56103f116120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f116120 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f22dc032ca0 tx=0x7f22dc02a3f0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22ff02b740 1 -- 172.21.15.63:0/3301722594 shutdown_connections 2024-05-27T08:11:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56103ed27d80 0x56103f127e60 crc :-1 s=READY pgs=682 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed691b0 0x56103f121ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f116120 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3301722594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed402f0 0x56103f115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22ff02b740 1 -- 172.21.15.63:0/3301722594 >> 172.21.15.63:0/3301722594 conn(0x56103ed5d300 msgr2=0x56103f10b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.022+0000 7f22d6a00640 0 client.22749 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.023+0000 7f22ff02b740 1 -- 172.21.15.63:0/3301722594 shutdown_connections 2024-05-27T08:11:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.023+0000 7f22ff02b740 1 -- 172.21.15.63:0/3301722594 wait complete. 2024-05-27T08:11:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.023+0000 7f22ff02b740 20 client.22749 trim_cache size 0 max 16384 2024-05-27T08:11:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.047+0000 7f22ff02b740 1 Processor -- start 2024-05-27T08:11:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.047+0000 7f22ff02b740 1 -- start start 2024-05-27T08:11:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f116130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed484d0 0x56103f1113a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed691b0 0x56103f107240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22ff02b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01b5b00 con 0x56103ed691b0 2024-05-27T08:11:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22ff02b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01b33d0 con 0x56103ed402f0 2024-05-27T08:11:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22ff02b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01b1240 con 0x56103ed484d0 2024-05-27T08:11:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22f6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed691b0 0x56103f107240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22f6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed691b0 0x56103f107240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51694/0 (socket says 172.21.15.63:51694) 2024-05-27T08:11:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22f6000640 1 -- 172.21.15.63:0/2991763214 learned_addr learned my addr 172.21.15.63:0/2991763214 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22f5600640 1 --2- 172.21.15.63:0/2991763214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.048+0000 7f22f4c00640 1 --2- 172.21.15.63:0/2991763214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed484d0 0x56103f1113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.049+0000 7f22f6000640 1 -- 172.21.15.63:0/2991763214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed484d0 msgr2=0x56103f1113a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.049+0000 7f22f6000640 1 --2- 172.21.15.63:0/2991763214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed484d0 0x56103f1113a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.049+0000 7f22f6000640 1 -- 172.21.15.63:0/2991763214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 msgr2=0x56103f116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.049+0000 7f22f6000640 1 --2- 172.21.15.63:0/2991763214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f116130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.049+0000 7f22f6000640 1 -- 172.21.15.63:0/2991763214 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56103f114130 con 0x56103ed691b0 2024-05-27T08:11:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.049+0000 7f22f6000640 1 --2- 172.21.15.63:0/2991763214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed691b0 0x56103f107240 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f22e80a5b50 tx=0x7f22e80a5260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e0c8a42da7f043 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.049+0000 7f22e1c00640 1 -- 172.21.15.63:0/2991763214 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8020040 con 0x56103ed691b0 2024-05-27T08:11:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.049+0000 7f22e1c00640 1 -- 172.21.15.63:0/2991763214 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e80aa1d0 con 0x56103ed691b0 2024-05-27T08:11:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.049+0000 7f22e1c00640 1 -- 172.21.15.63:0/2991763214 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e80a5d60 con 0x56103ed691b0 2024-05-27T08:11:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.051+0000 7f22ff02b740 1 -- 172.21.15.63:0/2991763214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed691b0 msgr2=0x56103f107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.051+0000 7f22ff02b740 1 --2- 172.21.15.63:0/2991763214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed691b0 0x56103f107240 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f22e80a5b50 tx=0x7f22e80a5260 comp rx=0 tx=0).stop 2024-05-27T08:11:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.051+0000 7f22ff02b740 1 -- 172.21.15.63:0/2991763214 shutdown_connections 2024-05-27T08:11:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.051+0000 7f22ff02b740 1 --2- 172.21.15.63:0/2991763214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed691b0 0x56103f107240 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.051+0000 7f22ff02b740 1 --2- 172.21.15.63:0/2991763214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed484d0 0x56103f1113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.051+0000 7f22ff02b740 1 --2- 172.21.15.63:0/2991763214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.051+0000 7f22ff02b740 1 -- 172.21.15.63:0/2991763214 >> 172.21.15.63:0/2991763214 conn(0x56103ed5d300 msgr2=0x56103f1080d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.051+0000 7f22ff02b740 1 -- 172.21.15.63:0/2991763214 shutdown_connections 2024-05-27T08:11:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.051+0000 7f22ff02b740 1 -- 172.21.15.63:0/2991763214 wait complete. 2024-05-27T08:11:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.051+0000 7f22ff02b740 1 -- start start 2024-05-27T08:11:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.052+0000 7f22ff02b740 1 Processor -- start 2024-05-27T08:11:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.052+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f118de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.052+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f119320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.052+0000 7f22ff02b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed691b0 0x56103f119860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.052+0000 7f22ff02b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01b1890 con 0x56103ed484d0 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.052+0000 7f22ff02b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01b2d00 con 0x56103ed402f0 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.052+0000 7f22ff02b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01b1f60 con 0x56103ed691b0 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.052+0000 7f22ff02b740 10 client.0 _ll_register_callbacks cb 0x7f22dc032bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.052+0000 7f22f4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f119320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22f4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f119320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51702/0 (socket says 172.21.15.63:51702) 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22f4c00640 1 -- 172.21.15.63:0/3224614812 learned_addr learned my addr 172.21.15.63:0/3224614812 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22f5600640 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f118de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22f6000640 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed691b0 0x56103f119860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22f4c00640 1 -- 172.21.15.63:0/3224614812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed691b0 msgr2=0x56103f119860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22f4c00640 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed691b0 0x56103f119860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22f4c00640 1 -- 172.21.15.63:0/3224614812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 msgr2=0x56103f118de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22f4c00640 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f118de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22f4c00640 1 -- 172.21.15.63:0/3224614812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56103f18e650 con 0x56103ed484d0 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22f4c00640 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f119320 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0326e0 tx=0x7f22dc005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b474cb01eda77adf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.053+0000 7f22d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.054+0000 7f22d6a00640 1 -- 172.21.15.63:0/3224614812 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc035070 con 0x56103ed484d0 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.054+0000 7f22d6a00640 1 -- 172.21.15.63:0/3224614812 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f22dc03d7e0 con 0x56103ed484d0 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.054+0000 7f22ff02b740 1 -- 172.21.15.63:0/3224614812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e80aa1d0 con 0x56103ed484d0 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.054+0000 7f22d6a00640 1 -- 172.21.15.63:0/3224614812 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f22dc06e050 con 0x56103ed484d0 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.054+0000 7f22ff02b740 10 client.22755 fetch_fsmap learned FSMap version 35 2024-05-27T08:11:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.054+0000 7f22ff02b740 1 -- 172.21.15.63:0/3224614812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56103f18f8c0 con 0x56103ed484d0 2024-05-27T08:11:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.054+0000 7f22d6a00640 1 -- 172.21.15.63:0/3224614812 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e80aa1d0 con 0x56103ed484d0 2024-05-27T08:11:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.055+0000 7f22d6a00640 1 -- 172.21.15.63:0/3224614812 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f22dc031c70 con 0x56103ed484d0 2024-05-27T08:11:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.055+0000 7f22ff02b740 10 client.22755 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:11:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.055+0000 7f22ff02b740 10 client.22755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:11:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.055+0000 7f22ff02b740 20 client.22755 populate_metadata read hostname 'smithi063' 2024-05-27T08:11:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.055+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56103ecfd1c0 0x56103f192040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:11:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.055+0000 7f22ff02b740 4 client.22755 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:11:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.055+0000 7f22ff02b740 1 -- 172.21.15.63:0/3224614812 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56103f0782c0 con 0x56103ecfd1c0 2024-05-27T08:11:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.055+0000 7f22f5600640 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56103ecfd1c0 0x56103f192040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:11:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.057+0000 7f22d6a00640 10 client.22755 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.057+0000 7f22f5600640 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56103ecfd1c0 0x56103f192040 crc :-1 s=READY pgs=683 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52dae7d3f7e9dade server_cookie=6d2cc1cb761cdf8e in_seq=0 out_seq=0 2024-05-27T08:11:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.103+0000 7f22d6a00640 1 -- 172.21.15.63:0/3224614812 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56103f0782c0 con 0x56103ecfd1c0 2024-05-27T08:11:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.103+0000 7f22d6a00640 10 client.22755 handle_command_reply: tid=0 2024-05-27T08:11:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.103+0000 7f22ff02b740 1 client.22755 shutdown 2024-05-27T08:11:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 1 -- 172.21.15.63:0/3224614812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 msgr2=0x56103f119320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f119320 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0326e0 tx=0x7f22dc005f10 comp rx=0 tx=0).stop 2024-05-27T08:11:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 1 -- 172.21.15.63:0/3224614812 shutdown_connections 2024-05-27T08:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56103ecfd1c0 0x56103f192040 crc :-1 s=READY pgs=683 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56103ed691b0 0x56103f119860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56103ed484d0 0x56103f119320 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 1 --2- 172.21.15.63:0/3224614812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56103ed402f0 0x56103f118de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22d6a00640 0 client.22755 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:11:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 1 -- 172.21.15.63:0/3224614812 >> 172.21.15.63:0/3224614812 conn(0x56103ed5d300 msgr2=0x56103f1080d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 1 -- 172.21.15.63:0/3224614812 shutdown_connections 2024-05-27T08:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 1 -- 172.21.15.63:0/3224614812 wait complete. 2024-05-27T08:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.104+0000 7f22ff02b740 20 client.22755 trim_cache size 0 max 16384 2024-05-27T08:11:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:11:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:11:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.105+0000 7f22f6a00640 1 -- 172.21.15.63:0/2484926153 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22d80b2950 msgr2=0x7f22d80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.105+0000 7f22f6a00640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22d80b2950 0x7f22d80b4e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x56103f108bd0 tx=0x7f22dc040040 comp rx=0 tx=0).stop 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.105+0000 7f22f6a00640 1 -- 172.21.15.63:0/2484926153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22f010c190 msgr2=0x7f22f019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.105+0000 7f22f6a00640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22f010c190 0x7f22f019c670 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009fd0 tx=0x7f22e800b260 comp rx=0 tx=0).stop 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.106+0000 7f22f6a00640 1 -- 172.21.15.63:0/2484926153 shutdown_connections 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.106+0000 7f22f6a00640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f22d80b2950 0x7f22d80b4e60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.106+0000 7f22f6a00640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22f010c190 0x7f22f019c670 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.106+0000 7f22f6a00640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22f010bcf0 0x7f22f01a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.106+0000 7f22f6a00640 1 --2- 172.21.15.63:0/2484926153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22f0106440 0x7f22f01a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.106+0000 7f22f6a00640 1 -- 172.21.15.63:0/2484926153 >> 172.21.15.63:0/2484926153 conn(0x7f22f007d510 msgr2=0x7f22f0104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:11:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.106+0000 7f22f6a00640 1 -- 172.21.15.63:0/2484926153 shutdown_connections 2024-05-27T08:11:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:11:49.106+0000 7f22f6a00640 1 -- 172.21.15.63:0/2484926153 wait complete. 2024-05-27T08:11:50.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:49 smithi086 ceph-mon[34687]: pgmap v1213: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:11:50.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:49 smithi149 ceph-mon[35408]: pgmap v1213: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:11:50.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:49 smithi063 ceph-mon[31613]: pgmap v1213: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:11:52.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:51 smithi086 ceph-mon[34687]: pgmap v1214: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:11:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:51 smithi149 ceph-mon[35408]: pgmap v1214: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:11:52.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:51 smithi063 ceph-mon[31613]: pgmap v1214: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 42 op/s 2024-05-27T08:11:54.185 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:53 smithi086 ceph-mon[34687]: pgmap v1215: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.4 MiB/s wr, 49 op/s 2024-05-27T08:11:54.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:53 smithi149 ceph-mon[35408]: pgmap v1215: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.4 MiB/s wr, 49 op/s 2024-05-27T08:11:54.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:53 smithi063 ceph-mon[31613]: pgmap v1215: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.4 MiB/s wr, 49 op/s 2024-05-27T08:11:56.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:55 smithi086 ceph-mon[34687]: pgmap v1216: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T08:11:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:55 smithi149 ceph-mon[35408]: pgmap v1216: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T08:11:56.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:55 smithi063 ceph-mon[31613]: pgmap v1216: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-05-27T08:11:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:57 smithi086 ceph-mon[34687]: pgmap v1217: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:11:57.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:57 smithi149 ceph-mon[35408]: pgmap v1217: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:11:57.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:57 smithi063 ceph-mon[31613]: pgmap v1217: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 14 op/s 2024-05-27T08:12:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:11:59 smithi086 ceph-mon[34687]: pgmap v1218: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:12:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:11:59 smithi149 ceph-mon[35408]: pgmap v1218: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:12:00.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:11:59 smithi063 ceph-mon[31613]: pgmap v1218: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:12:02.016 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:01 smithi149 ceph-mon[35408]: pgmap v1219: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:12:02.019 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:01 smithi063 ceph-mon[31613]: pgmap v1219: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:12:02.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:01 smithi086 ceph-mon[34687]: pgmap v1219: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 10 op/s 2024-05-27T08:12:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.365+0000 7f3662800640 1 -- 172.21.15.63:0/2369024926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c075480 msgr2=0x7f365c075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.365+0000 7f3662800640 1 --2- 172.21.15.63:0/2369024926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c075480 0x7f365c075890 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f3654009ca0 tx=0x7f365402a950 comp rx=0 tx=0).stop 2024-05-27T08:12:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.365+0000 7f3662800640 1 -- 172.21.15.63:0/2369024926 shutdown_connections 2024-05-27T08:12:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.365+0000 7f3662800640 1 --2- 172.21.15.63:0/2369024926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f365c114ac0 0x7f365c116ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.365+0000 7f3662800640 1 --2- 172.21.15.63:0/2369024926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f365c112160 0x7f365c114580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.365+0000 7f3662800640 1 --2- 172.21.15.63:0/2369024926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c075480 0x7f365c075890 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.365+0000 7f3662800640 1 -- 172.21.15.63:0/2369024926 >> 172.21.15.63:0/2369024926 conn(0x7f365c079240 msgr2=0x7f365c079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.366+0000 7f3662800640 1 -- 172.21.15.63:0/2369024926 shutdown_connections 2024-05-27T08:12:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.366+0000 7f3662800640 1 -- 172.21.15.63:0/2369024926 wait complete. 2024-05-27T08:12:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.367+0000 7f3662800640 1 Processor -- start 2024-05-27T08:12:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.367+0000 7f3662800640 1 -- start start 2024-05-27T08:12:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.367+0000 7f3662800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f365c075480 0x7f365c1a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.367+0000 7f3662800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c112160 0x7f365c1a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.367+0000 7f3662800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f365c114ac0 0x7f365c1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.368+0000 7f3662800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f365c11a0f0 con 0x7f365c112160 2024-05-27T08:12:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.368+0000 7f3662800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f365c119f70 con 0x7f365c075480 2024-05-27T08:12:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.368+0000 7f3662800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f365c11a270 con 0x7f365c114ac0 2024-05-27T08:12:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.368+0000 7f3660a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c112160 0x7f365c1a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.368+0000 7f3660a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c112160 0x7f365c1a6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58950/0 (socket says 172.21.15.63:58950) 2024-05-27T08:12:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.368+0000 7f3660a00640 1 -- 172.21.15.63:0/400798206 learned_addr learned my addr 172.21.15.63:0/400798206 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.368+0000 7f3661400640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f365c075480 0x7f365c1a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.368+0000 7f3661e00640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f365c114ac0 0x7f365c1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.368+0000 7f3660a00640 1 -- 172.21.15.63:0/400798206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f365c114ac0 msgr2=0x7f365c1a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.368+0000 7f3660a00640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f365c114ac0 0x7f365c1a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.369+0000 7f3660a00640 1 -- 172.21.15.63:0/400798206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f365c075480 msgr2=0x7f365c1a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.369+0000 7f3660a00640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f365c075480 0x7f365c1a6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.369+0000 7f3660a00640 1 -- 172.21.15.63:0/400798206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c1a1020 con 0x7f365c112160 2024-05-27T08:12:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.369+0000 7f3661400640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f365c075480 0x7f365c1a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.369+0000 7f3661e00640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f365c114ac0 0x7f365c1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.369+0000 7f3660a00640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c112160 0x7f365c1a6d60 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f364c00c920 tx=0x7f364c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.369+0000 7f365a000640 1 -- 172.21.15.63:0/400798206 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c007c70 con 0x7f365c112160 2024-05-27T08:12:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.369+0000 7f3662800640 1 -- 172.21.15.63:0/400798206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c1a1310 con 0x7f365c112160 2024-05-27T08:12:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.369+0000 7f365a000640 1 -- 172.21.15.63:0/400798206 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f364c007e10 con 0x7f365c112160 2024-05-27T08:12:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.369+0000 7f365a000640 1 -- 172.21.15.63:0/400798206 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c0186f0 con 0x7f365c112160 2024-05-27T08:12:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.370+0000 7f3662800640 1 -- 172.21.15.63:0/400798206 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c1a1860 con 0x7f365c112160 2024-05-27T08:12:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.373+0000 7f366af7d740 1 Processor -- start 2024-05-27T08:12:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.373+0000 7f366af7d740 1 -- start start 2024-05-27T08:12:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.374+0000 7f366af7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a61fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.374+0000 7f366af7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a61ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.374+0000 7f366af7d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a6204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.374+0000 7f366af7d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55db9a590b90 con 0x55db9a25c4d0 2024-05-27T08:12:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.374+0000 7f366af7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db9a580730 con 0x55db9a2542f0 2024-05-27T08:12:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.374+0000 7f366af7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55db9a584770 con 0x55db9a27d1b0 2024-05-27T08:12:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.374+0000 7f365a000640 1 -- 172.21.15.63:0/400798206 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f364c00f450 con 0x7f365c112160 2024-05-27T08:12:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.374+0000 7f3661e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a61fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.374+0000 7f3661400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a61ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.374+0000 7f3661e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a61fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:48044/0 (socket says 172.21.15.63:48044) 2024-05-27T08:12:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.375+0000 7f3660a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a6204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.375+0000 7f3661e00640 1 -- 172.21.15.63:0/1826772424 learned_addr learned my addr 172.21.15.63:0/1826772424 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.375+0000 7f3660a00640 1 -- 172.21.15.63:0/1826772424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 msgr2=0x55db9a61fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.375+0000 7f3660a00640 1 --2- 172.21.15.63:0/1826772424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a61fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.375+0000 7f3660a00640 1 -- 172.21.15.63:0/1826772424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 msgr2=0x55db9a61ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.375+0000 7f3660a00640 1 --2- 172.21.15.63:0/1826772424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a61ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.375+0000 7f3660a00640 1 -- 172.21.15.63:0/1826772424 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9a5198b0 con 0x55db9a25c4d0 2024-05-27T08:12:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.376+0000 7f3660a00640 1 --2- 172.21.15.63:0/1826772424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a6204e0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f364c0e0000 tx=0x7f364c0e6f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a784b51b5511882 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.376+0000 7f364a600640 1 -- 172.21.15.63:0/1826772424 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c0f5470 con 0x55db9a25c4d0 2024-05-27T08:12:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.376+0000 7f364a600640 1 -- 172.21.15.63:0/1826772424 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f364c0f0070 con 0x55db9a25c4d0 2024-05-27T08:12:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.376+0000 7f364a600640 1 -- 172.21.15.63:0/1826772424 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c0ec6a0 con 0x55db9a25c4d0 2024-05-27T08:12:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.376+0000 7f365a000640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f36440b2950 0x7f36440b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.377+0000 7f365a000640 1 -- 172.21.15.63:0/400798206 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f364c0e2c70 con 0x7f365c112160 2024-05-27T08:12:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.377+0000 7f3661e00640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f36440b2950 0x7f36440b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.377+0000 7f366af7d740 1 -- 172.21.15.63:0/1826772424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 msgr2=0x55db9a6204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.377+0000 7f366af7d740 1 --2- 172.21.15.63:0/1826772424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a6204e0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f364c0e0000 tx=0x7f364c0e6f30 comp rx=0 tx=0).stop 2024-05-27T08:12:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.378+0000 7f366af7d740 1 -- 172.21.15.63:0/1826772424 shutdown_connections 2024-05-27T08:12:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.378+0000 7f366af7d740 1 --2- 172.21.15.63:0/1826772424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a6204e0 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.378+0000 7f366af7d740 1 --2- 172.21.15.63:0/1826772424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a61ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.378+0000 7f366af7d740 1 --2- 172.21.15.63:0/1826772424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a61fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.378+0000 7f366af7d740 1 -- 172.21.15.63:0/1826772424 >> 172.21.15.63:0/1826772424 conn(0x55db9a271300 msgr2=0x55db9a61c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.378+0000 7f3661e00640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f36440b2950 0x7f36440b4e60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3650005fd0 tx=0x7f365000b5c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.378+0000 7f366af7d740 1 -- 172.21.15.63:0/1826772424 shutdown_connections 2024-05-27T08:12:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.378+0000 7f366af7d740 1 -- 172.21.15.63:0/1826772424 wait complete. 2024-05-27T08:12:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.378+0000 7f366af7d740 1 -- start start 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.379+0000 7f366af7d740 1 Processor -- start 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.379+0000 7f366af7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a2542f0 0x55db9a629c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.379+0000 7f366af7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a25c4d0 0x55db9a62a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.379+0000 7f366af7d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a27d1b0 0x55db9a635ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.379+0000 7f366af7d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55db9a266790 con 0x55db9a27d1b0 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.379+0000 7f366af7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db9a528bc0 con 0x55db9a25c4d0 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.379+0000 7f366af7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55db9a62cb90 con 0x55db9a2542f0 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.379+0000 7f366af7d740 10 client.0 _ll_register_callbacks cb 0x55db9a506400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.380+0000 7f3661e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a27d1b0 0x55db9a635ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.380+0000 7f3661e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a27d1b0 0x55db9a635ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58958/0 (socket says 172.21.15.63:58958) 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.380+0000 7f3661e00640 1 -- 172.21.15.63:0/3654723629 learned_addr learned my addr 172.21.15.63:0/3654723629 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.380+0000 7f3660a00640 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a25c4d0 0x55db9a62a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.380+0000 7f3661400640 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a2542f0 0x55db9a629c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.380+0000 7f3661e00640 1 -- 172.21.15.63:0/3654723629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a2542f0 msgr2=0x55db9a629c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.380+0000 7f3661e00640 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a2542f0 0x55db9a629c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.380+0000 7f3661e00640 1 -- 172.21.15.63:0/3654723629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a25c4d0 msgr2=0x55db9a62a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.380+0000 7f3661e00640 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a25c4d0 0x55db9a62a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.380+0000 7f3661e00640 1 -- 172.21.15.63:0/3654723629 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9a295210 con 0x55db9a27d1b0 2024-05-27T08:12:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.381+0000 7f3661e00640 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a27d1b0 0x55db9a635ec0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f3650010730 tx=0x7f36500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea7dd1aa2abd070 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.381+0000 7f363ea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.381+0000 7f363ea00640 1 -- 172.21.15.63:0/3654723629 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365000fa10 con 0x55db9a27d1b0 2024-05-27T08:12:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.381+0000 7f363ea00640 1 -- 172.21.15.63:0/3654723629 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f365000fbb0 con 0x55db9a27d1b0 2024-05-27T08:12:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.381+0000 7f366af7d740 1 -- 172.21.15.63:0/3654723629 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364c0f0070 con 0x55db9a27d1b0 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.381+0000 7f363ea00640 1 -- 172.21.15.63:0/3654723629 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3650030050 con 0x55db9a27d1b0 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.381+0000 7f366af7d740 10 client.22767 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.381+0000 7f366af7d740 1 -- 172.21.15.63:0/3654723629 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55db9a361620 con 0x55db9a27d1b0 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.382+0000 7f363ea00640 1 -- 172.21.15.63:0/3654723629 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f364c0f0070 con 0x55db9a27d1b0 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.382+0000 7f363ea00640 1 -- 172.21.15.63:0/3654723629 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f365001d380 con 0x55db9a27d1b0 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.382+0000 7f366af7d740 10 client.22767 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.382+0000 7f366af7d740 10 client.22767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.382+0000 7f366af7d740 20 client.22767 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.382+0000 7f366af7d740 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55db9a23bd80 0x55db9a63bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.382+0000 7f366af7d740 4 client.22767 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.382+0000 7f366af7d740 1 -- 172.21.15.63:0/3654723629 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db9a592ae0 con 0x55db9a23bd80 2024-05-27T08:12:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.383+0000 7f3661400640 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55db9a23bd80 0x55db9a63bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.384+0000 7f3661400640 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55db9a23bd80 0x55db9a63bed0 crc :-1 s=READY pgs=684 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1411a9a2d71d53c server_cookie=b9e6ea5de6b0a50d in_seq=0 out_seq=0 2024-05-27T08:12:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.384+0000 7f363ea00640 10 client.22767 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f363ea00640 1 -- 172.21.15.63:0/3654723629 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55db9a592ae0 con 0x55db9a23bd80 2024-05-27T08:12:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f363ea00640 10 client.22767 handle_command_reply: tid=0 2024-05-27T08:12:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 client.22767 shutdown 2024-05-27T08:12:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 -- 172.21.15.63:0/3654723629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a27d1b0 msgr2=0x55db9a635ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a27d1b0 0x55db9a635ec0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f3650010730 tx=0x7f36500046c0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 -- 172.21.15.63:0/3654723629 shutdown_connections 2024-05-27T08:12:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55db9a23bd80 0x55db9a63bed0 crc :-1 s=READY pgs=684 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a27d1b0 0x55db9a635ec0 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a25c4d0 0x55db9a62a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 --2- 172.21.15.63:0/3654723629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a2542f0 0x55db9a629c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f363ea00640 0 client.22767 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 -- 172.21.15.63:0/3654723629 >> 172.21.15.63:0/3654723629 conn(0x55db9a271300 msgr2=0x55db9a621780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 -- 172.21.15.63:0/3654723629 shutdown_connections 2024-05-27T08:12:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.386+0000 7f366af7d740 1 -- 172.21.15.63:0/3654723629 wait complete. 2024-05-27T08:12:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.387+0000 7f366af7d740 20 client.22767 trim_cache size 0 max 16384 2024-05-27T08:12:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.406+0000 7f366af7d740 1 Processor -- start 2024-05-27T08:12:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.406+0000 7f366af7d740 1 -- start start 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.406+0000 7f366af7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a625440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f366af7d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a61f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f366af7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a635c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f366af7d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1b9f70 con 0x55db9a25c4d0 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f366af7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1b7840 con 0x55db9a2542f0 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f366af7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1b56b0 con 0x55db9a27d1b0 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f3660a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a61f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f3660a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a61f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58966/0 (socket says 172.21.15.63:58966) 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f3660a00640 1 -- 172.21.15.63:0/2484815148 learned_addr learned my addr 172.21.15.63:0/2484815148 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f3661400640 1 --2- 172.21.15.63:0/2484815148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a625440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f3661e00640 1 --2- 172.21.15.63:0/2484815148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a635c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f3660a00640 1 -- 172.21.15.63:0/2484815148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 msgr2=0x55db9a635c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f3660a00640 1 --2- 172.21.15.63:0/2484815148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a635c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f3660a00640 1 -- 172.21.15.63:0/2484815148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 msgr2=0x55db9a625440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f3660a00640 1 --2- 172.21.15.63:0/2484815148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a625440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.407+0000 7f3660a00640 1 -- 172.21.15.63:0/2484815148 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9a6281a0 con 0x55db9a25c4d0 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.408+0000 7f3660a00640 1 --2- 172.21.15.63:0/2484815148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a61f970 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f364c0a9890 tx=0x7f364c00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9a78d1667d8265 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.408+0000 7f3649c00640 1 -- 172.21.15.63:0/2484815148 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c0a9960 con 0x55db9a25c4d0 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.408+0000 7f3649c00640 1 -- 172.21.15.63:0/2484815148 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f364c0f0040 con 0x55db9a25c4d0 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.408+0000 7f3649c00640 1 -- 172.21.15.63:0/2484815148 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c0e9ea0 con 0x55db9a25c4d0 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.409+0000 7f366af7d740 1 -- 172.21.15.63:0/2484815148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 msgr2=0x55db9a61f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.409+0000 7f366af7d740 1 --2- 172.21.15.63:0/2484815148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a61f970 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f364c0a9890 tx=0x7f364c00fd40 comp rx=0 tx=0).stop 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.409+0000 7f366af7d740 1 -- 172.21.15.63:0/2484815148 shutdown_connections 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.409+0000 7f366af7d740 1 --2- 172.21.15.63:0/2484815148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a635c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.409+0000 7f366af7d740 1 --2- 172.21.15.63:0/2484815148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a61f970 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.409+0000 7f366af7d740 1 --2- 172.21.15.63:0/2484815148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a625440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.409+0000 7f366af7d740 1 -- 172.21.15.63:0/2484815148 >> 172.21.15.63:0/2484815148 conn(0x55db9a271300 msgr2=0x55db9a62bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.409+0000 7f366af7d740 1 -- 172.21.15.63:0/2484815148 shutdown_connections 2024-05-27T08:12:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.409+0000 7f366af7d740 1 -- 172.21.15.63:0/2484815148 wait complete. 2024-05-27T08:12:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.410+0000 7f366af7d740 1 -- start start 2024-05-27T08:12:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.410+0000 7f366af7d740 1 Processor -- start 2024-05-27T08:12:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f366af7d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a62d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f366af7d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a62d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f366af7d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a6a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f366af7d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1b5d00 con 0x55db9a25c4d0 2024-05-27T08:12:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f366af7d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1b7170 con 0x55db9a2542f0 2024-05-27T08:12:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f366af7d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1b63d0 con 0x55db9a27d1b0 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f366af7d740 10 client.0 _ll_register_callbacks cb 0x7f365000bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f3660a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a62d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f3660a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a62d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58980/0 (socket says 172.21.15.63:58980) 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f3660a00640 1 -- 172.21.15.63:0/3076002555 learned_addr learned my addr 172.21.15.63:0/3076002555 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f3661e00640 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a6a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f3661400640 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a62d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f3660a00640 1 -- 172.21.15.63:0/3076002555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 msgr2=0x55db9a6a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f3660a00640 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a6a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f3660a00640 1 -- 172.21.15.63:0/3076002555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 msgr2=0x55db9a62d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.411+0000 7f3660a00640 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a62d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.412+0000 7f3660a00640 1 -- 172.21.15.63:0/3076002555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9a6a2a80 con 0x55db9a25c4d0 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.412+0000 7f3661e00640 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a6a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.412+0000 7f3661400640 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a62d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:12:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.412+0000 7f3660a00640 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a62d710 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f364c0a5c10 tx=0x7f364c0e67b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5330c23f7f8000c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.412+0000 7f363ea00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.412+0000 7f363ea00640 1 -- 172.21.15.63:0/3076002555 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c06e060 con 0x55db9a25c4d0 2024-05-27T08:12:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.412+0000 7f363ea00640 1 -- 172.21.15.63:0/3076002555 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 9483+0+0 (secure 0 0 0) 0x7f364c0c4170 con 0x55db9a25c4d0 2024-05-27T08:12:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.412+0000 7f366af7d740 1 -- 172.21.15.63:0/3076002555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364c0f0040 con 0x55db9a25c4d0 2024-05-27T08:12:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f363ea00640 1 -- 172.21.15.63:0/3076002555 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f364c0a5320 con 0x55db9a25c4d0 2024-05-27T08:12:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f366af7d740 10 client.22773 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f366af7d740 1 -- 172.21.15.63:0/3076002555 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55db9a6a3be0 con 0x55db9a25c4d0 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f363ea00640 1 -- 172.21.15.63:0/3076002555 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f364c0f0040 con 0x55db9a25c4d0 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f363ea00640 1 -- 172.21.15.63:0/3076002555 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f364c0a5e20 con 0x55db9a25c4d0 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f366af7d740 10 client.22773 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f366af7d740 10 client.22773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f366af7d740 20 client.22773 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f366af7d740 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55db9a2111c0 0x55db9a6a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f366af7d740 4 client.22773 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.413+0000 7f366af7d740 1 -- 172.21.15.63:0/3076002555 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db9a58c2c0 con 0x55db9a2111c0 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.414+0000 7f3661400640 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55db9a2111c0 0x55db9a6a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.415+0000 7f3661400640 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55db9a2111c0 0x55db9a6a5e10 crc :-1 s=READY pgs=685 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87bd3d791364366 server_cookie=409bf5317b8e3c08 in_seq=0 out_seq=0 2024-05-27T08:12:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.415+0000 7f363ea00640 10 client.22773 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f363ea00640 1 -- 172.21.15.63:0/3076002555 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55db9a58c2c0 con 0x55db9a2111c0 2024-05-27T08:12:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f363ea00640 10 client.22773 handle_command_reply: tid=0 2024-05-27T08:12:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f366af7d740 1 client.22773 shutdown 2024-05-27T08:12:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f366af7d740 1 -- 172.21.15.63:0/3076002555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 msgr2=0x55db9a62d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f366af7d740 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a62d710 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f364c0a5c10 tx=0x7f364c0e67b0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f366af7d740 1 -- 172.21.15.63:0/3076002555 shutdown_connections 2024-05-27T08:12:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f366af7d740 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55db9a2111c0 0x55db9a6a5e10 crc :-1 s=READY pgs=685 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f366af7d740 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55db9a27d1b0 0x55db9a6a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f366af7d740 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db9a25c4d0 0x55db9a62d710 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f366af7d740 1 --2- 172.21.15.63:0/3076002555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db9a2542f0 0x55db9a62d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f366af7d740 1 -- 172.21.15.63:0/3076002555 >> 172.21.15.63:0/3076002555 conn(0x55db9a271300 msgr2=0x55db9a629190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.459+0000 7f363ea00640 0 client.22773 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.460+0000 7f366af7d740 1 -- 172.21.15.63:0/3076002555 shutdown_connections 2024-05-27T08:12:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.460+0000 7f366af7d740 1 -- 172.21.15.63:0/3076002555 wait complete. 2024-05-27T08:12:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.460+0000 7f366af7d740 20 client.22773 trim_cache size 0 max 16384 2024-05-27T08:12:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:12:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:12:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:12:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 -- 172.21.15.63:0/400798206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f36440b2950 msgr2=0x7f36440b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f36440b2950 0x7f36440b4e60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3650005fd0 tx=0x7f365000b5c0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 -- 172.21.15.63:0/400798206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c112160 msgr2=0x7f365c1a6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c112160 0x7f365c1a6d60 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f364c00c920 tx=0x7f364c00cde0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 -- 172.21.15.63:0/400798206 shutdown_connections 2024-05-27T08:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f36440b2950 0x7f36440b4e60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f365c114ac0 0x7f365c1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f365c112160 0x7f365c1a6d60 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 --2- 172.21.15.63:0/400798206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f365c075480 0x7f365c1a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 -- 172.21.15.63:0/400798206 >> 172.21.15.63:0/400798206 conn(0x7f365c079240 msgr2=0x7f365c1125d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 -- 172.21.15.63:0/400798206 shutdown_connections 2024-05-27T08:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:02.461+0000 7f3662800640 1 -- 172.21.15.63:0/400798206 wait complete. 2024-05-27T08:12:04.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:03 smithi086 ceph-mon[34687]: pgmap v1220: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 31 op/s 2024-05-27T08:12:04.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:03 smithi149 ceph-mon[35408]: pgmap v1220: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 31 op/s 2024-05-27T08:12:04.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:03 smithi063 ceph-mon[31613]: pgmap v1220: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 31 op/s 2024-05-27T08:12:06.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:05 smithi086 ceph-mon[34687]: pgmap v1221: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.0 MiB/s wr, 24 op/s 2024-05-27T08:12:06.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:05 smithi149 ceph-mon[35408]: pgmap v1221: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.0 MiB/s wr, 24 op/s 2024-05-27T08:12:06.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:05 smithi063 ceph-mon[31613]: pgmap v1221: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.0 MiB/s wr, 24 op/s 2024-05-27T08:12:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:07 smithi086 ceph-mon[34687]: pgmap v1222: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.7 MiB/s wr, 31 op/s 2024-05-27T08:12:08.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:07 smithi086 ceph-mon[34687]: osdmap e157: 12 total, 12 up, 12 in 2024-05-27T08:12:08.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:07 smithi149 ceph-mon[35408]: pgmap v1222: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.7 MiB/s wr, 31 op/s 2024-05-27T08:12:08.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:07 smithi149 ceph-mon[35408]: osdmap e157: 12 total, 12 up, 12 in 2024-05-27T08:12:08.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:07 smithi063 ceph-mon[31613]: pgmap v1222: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.7 MiB/s wr, 31 op/s 2024-05-27T08:12:08.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:07 smithi063 ceph-mon[31613]: osdmap e157: 12 total, 12 up, 12 in 2024-05-27T08:12:10.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:09 smithi086 ceph-mon[34687]: pgmap v1224: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.5 MiB/s wr, 33 op/s 2024-05-27T08:12:10.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:09 smithi149 ceph-mon[35408]: pgmap v1224: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.5 MiB/s wr, 33 op/s 2024-05-27T08:12:10.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:09 smithi063 ceph-mon[31613]: pgmap v1224: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.5 MiB/s wr, 33 op/s 2024-05-27T08:12:12.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:11 smithi086 ceph-mon[34687]: pgmap v1225: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.5 MiB/s wr, 33 op/s 2024-05-27T08:12:12.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:11 smithi149 ceph-mon[35408]: pgmap v1225: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.5 MiB/s wr, 33 op/s 2024-05-27T08:12:12.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:11 smithi063 ceph-mon[31613]: pgmap v1225: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.5 MiB/s wr, 33 op/s 2024-05-27T08:12:14.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:13 smithi086 ceph-mon[34687]: pgmap v1226: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.5 MiB/s wr, 12 op/s 2024-05-27T08:12:14.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:13 smithi149 ceph-mon[35408]: pgmap v1226: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.5 MiB/s wr, 12 op/s 2024-05-27T08:12:14.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:13 smithi063 ceph-mon[31613]: pgmap v1226: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.5 MiB/s wr, 12 op/s 2024-05-27T08:12:15.150 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:12:15.703 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:12:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.762+0000 7fd0f1800640 1 -- 172.21.15.63:0/4095831904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec10be90 msgr2=0x7fd0ec105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.762+0000 7fd0f1800640 1 --2- 172.21.15.63:0/4095831904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec10be90 0x7fd0ec105a30 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009900 tx=0x7fd0dc02a3f0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.762+0000 7fd0f1800640 1 -- 172.21.15.63:0/4095831904 shutdown_connections 2024-05-27T08:12:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.762+0000 7fd0f1800640 1 --2- 172.21.15.63:0/4095831904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0ec1065e0 0x7fd0ec1129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.762+0000 7fd0f1800640 1 --2- 172.21.15.63:0/4095831904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0ec10c330 0x7fd0ec105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.762+0000 7fd0f1800640 1 --2- 172.21.15.63:0/4095831904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec10be90 0x7fd0ec105a30 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.762+0000 7fd0f1800640 1 -- 172.21.15.63:0/4095831904 >> 172.21.15.63:0/4095831904 conn(0x7fd0ec101530 msgr2=0x7fd0ec103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.762+0000 7fd0f1800640 1 -- 172.21.15.63:0/4095831904 shutdown_connections 2024-05-27T08:12:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.762+0000 7fd0f1800640 1 -- 172.21.15.63:0/4095831904 wait complete. 2024-05-27T08:12:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.763+0000 7fd0f1800640 1 Processor -- start 2024-05-27T08:12:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.763+0000 7fd0f1800640 1 -- start start 2024-05-27T08:12:15.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0f1800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0ec1065e0 0x7fd0ec1a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0f1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0ec10be90 0x7fd0ec1a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0f1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec10c330 0x7fd0ec19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0f1800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec115f90 con 0x7fd0ec10c330 2024-05-27T08:12:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0f1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec115e10 con 0x7fd0ec10be90 2024-05-27T08:12:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0f1800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec116110 con 0x7fd0ec1065e0 2024-05-27T08:12:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0eb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec10c330 0x7fd0ec19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0eb400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec10c330 0x7fd0ec19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52790/0 (socket says 172.21.15.63:52790) 2024-05-27T08:12:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0eb400640 1 -- 172.21.15.63:0/883372857 learned_addr learned my addr 172.21.15.63:0/883372857 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0ec1065e0 0x7fd0ec1a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.764+0000 7fd0ea000640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0ec10be90 0x7fd0ec1a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.765+0000 7fd0eb400640 1 -- 172.21.15.63:0/883372857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0ec1065e0 msgr2=0x7fd0ec1a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.765+0000 7fd0eb400640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0ec1065e0 0x7fd0ec1a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.765+0000 7fd0eb400640 1 -- 172.21.15.63:0/883372857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0ec10be90 msgr2=0x7fd0ec1a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.765+0000 7fd0eb400640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0ec10be90 0x7fd0ec1a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.765+0000 7fd0eb400640 1 -- 172.21.15.63:0/883372857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec19cf30 con 0x7fd0ec10c330 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.765+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0ec1065e0 0x7fd0ec1a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.765+0000 7fd0eb400640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec10c330 0x7fd0ec19c850 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400eaa0 tx=0x7fd0e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.766+0000 7fd0e3400640 1 -- 172.21.15.63:0/883372857 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e400cdb0 con 0x7fd0ec10c330 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.766+0000 7fd0e3400640 1 -- 172.21.15.63:0/883372857 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0e4005020 con 0x7fd0ec10c330 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.766+0000 7fd0f1800640 1 -- 172.21.15.63:0/883372857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ec19d220 con 0x7fd0ec10c330 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.766+0000 7fd0e3400640 1 -- 172.21.15.63:0/883372857 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e40116d0 con 0x7fd0ec10c330 2024-05-27T08:12:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.766+0000 7fd0f1800640 1 -- 172.21.15.63:0/883372857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ec1a9470 con 0x7fd0ec10c330 2024-05-27T08:12:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.770+0000 7fd0f8777740 1 Processor -- start 2024-05-27T08:12:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.771+0000 7fd0e3400640 1 -- 172.21.15.63:0/883372857 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd0e40051d0 con 0x7fd0ec10c330 2024-05-27T08:12:15.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.772+0000 7fd0f8777740 1 -- start start 2024-05-27T08:12:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.772+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291ea1b0 0x55fb2958c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.772+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 0x55fb2958cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.772+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c94d0 0x55fb2958d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.772+0000 7fd0f8777740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fb294fdb90 con 0x55fb291ea1b0 2024-05-27T08:12:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.772+0000 7fd0f8777740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fb294ed730 con 0x55fb291c94d0 2024-05-27T08:12:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.772+0000 7fd0f8777740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb294f1770 con 0x55fb291c12f0 2024-05-27T08:12:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.773+0000 7fd0ea000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291ea1b0 0x55fb2958c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.773+0000 7fd0ea000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291ea1b0 0x55fb2958c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52792/0 (socket says 172.21.15.63:52792) 2024-05-27T08:12:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.773+0000 7fd0ea000640 1 -- 172.21.15.63:0/964487250 learned_addr learned my addr 172.21.15.63:0/964487250 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.773+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/964487250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 0x55fb2958cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.773+0000 7fd0ea000640 1 --2- 172.21.15.63:0/964487250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c94d0 0x55fb2958d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.773+0000 7fd0e3400640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd0d00b2950 0x7fd0d00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.774+0000 7fd0e3400640 1 -- 172.21.15.63:0/883372857 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 9571+0+0 (secure 0 0 0) 0x7fd0e40e3060 con 0x7fd0ec10c330 2024-05-27T08:12:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.774+0000 7fd0ea000640 1 -- 172.21.15.63:0/964487250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 msgr2=0x55fb2958cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.774+0000 7fd0ea000640 1 --2- 172.21.15.63:0/964487250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 0x55fb2958cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.774+0000 7fd0ea000640 1 -- 172.21.15.63:0/964487250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c94d0 msgr2=0x55fb2958d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.774+0000 7fd0ea000640 1 --2- 172.21.15.63:0/964487250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c94d0 0x55fb2958d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.774+0000 7fd0ea000640 1 -- 172.21.15.63:0/964487250 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb294868b0 con 0x55fb291ea1b0 2024-05-27T08:12:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.774+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/964487250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 0x55fb2958cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:12:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.774+0000 7fd0eb400640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd0d00b2950 0x7fd0d00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.774+0000 7fd0ea000640 1 --2- 172.21.15.63:0/964487250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291ea1b0 0x55fb2958c860 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8014830 tx=0x7fd0d800b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc15fd343f26091 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.775+0000 7fd0eb400640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd0d00b2950 0x7fd0d00b4e60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005bc0 tx=0x7fd0e400cf50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.775+0000 7fd0d7e00640 1 -- 172.21.15.63:0/964487250 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8004030 con 0x55fb291ea1b0 2024-05-27T08:12:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.775+0000 7fd0d7e00640 1 -- 172.21.15.63:0/964487250 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0d800f700 con 0x55fb291ea1b0 2024-05-27T08:12:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.775+0000 7fd0d7e00640 1 -- 172.21.15.63:0/964487250 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8003850 con 0x55fb291ea1b0 2024-05-27T08:12:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.777+0000 7fd0f8777740 1 -- 172.21.15.63:0/964487250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291ea1b0 msgr2=0x55fb2958c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.777+0000 7fd0f8777740 1 --2- 172.21.15.63:0/964487250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291ea1b0 0x55fb2958c860 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8014830 tx=0x7fd0d800b3c0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.777+0000 7fd0f8777740 1 -- 172.21.15.63:0/964487250 shutdown_connections 2024-05-27T08:12:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.777+0000 7fd0f8777740 1 --2- 172.21.15.63:0/964487250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c94d0 0x55fb2958d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.777+0000 7fd0f8777740 1 --2- 172.21.15.63:0/964487250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 0x55fb2958cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.777+0000 7fd0f8777740 1 --2- 172.21.15.63:0/964487250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291ea1b0 0x55fb2958c860 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.777+0000 7fd0f8777740 1 -- 172.21.15.63:0/964487250 >> 172.21.15.63:0/964487250 conn(0x55fb291de300 msgr2=0x55fb29589010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.777+0000 7fd0f8777740 1 -- 172.21.15.63:0/964487250 shutdown_connections 2024-05-27T08:12:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.777+0000 7fd0f8777740 1 -- 172.21.15.63:0/964487250 wait complete. 2024-05-27T08:12:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.778+0000 7fd0f8777740 1 -- start start 2024-05-27T08:12:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.778+0000 7fd0f8777740 1 Processor -- start 2024-05-27T08:12:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.778+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 0x55fb29596a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb29596f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291ea1b0 0x55fb295a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0f8777740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fb291d3790 con 0x55fb291c94d0 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0f8777740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fb29495bc0 con 0x55fb291ea1b0 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0f8777740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb29599930 con 0x55fb291c12f0 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0f8777740 10 client.0 _ll_register_callbacks cb 0x55fb29473400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0eaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb29596f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0eaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb29596f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52796/0 (socket says 172.21.15.63:52796) 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0eaa00640 1 -- 172.21.15.63:0/3149032176 learned_addr learned my addr 172.21.15.63:0/3149032176 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0ea000640 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 0x55fb29596a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.779+0000 7fd0ea000640 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291ea1b0 0x55fb295a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.780+0000 7fd0eaa00640 1 -- 172.21.15.63:0/3149032176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 msgr2=0x55fb29596a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.780+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 0x55fb29596a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.780+0000 7fd0eaa00640 1 -- 172.21.15.63:0/3149032176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291ea1b0 msgr2=0x55fb295a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.780+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291ea1b0 0x55fb295a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.780+0000 7fd0eaa00640 1 -- 172.21.15.63:0/3149032176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb29202210 con 0x55fb291c94d0 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.780+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb29596f90 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec106b70 tx=0x7fd0dc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43688b2f7b72d5f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.780+0000 7fd0d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.780+0000 7fd0d4c00640 1 -- 172.21.15.63:0/3149032176 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc035040 con 0x55fb291c94d0 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.781+0000 7fd0d4c00640 1 -- 172.21.15.63:0/3149032176 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(157..157 src has 1..157) ==== 9571+0+0 (secure 0 0 0) 0x7fd0dc0048c0 con 0x55fb291c94d0 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.781+0000 7fd0f8777740 1 -- 172.21.15.63:0/3149032176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0d800f700 con 0x55fb291c94d0 2024-05-27T08:12:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.781+0000 7fd0d4c00640 1 -- 172.21.15.63:0/3149032176 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd0dc04e050 con 0x55fb291c94d0 2024-05-27T08:12:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.781+0000 7fd0f8777740 10 client.22785 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.781+0000 7fd0f8777740 1 -- 172.21.15.63:0/3149032176 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fb292ce620 con 0x55fb291c94d0 2024-05-27T08:12:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.781+0000 7fd0d4c00640 1 -- 172.21.15.63:0/3149032176 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0d800f700 con 0x55fb291c94d0 2024-05-27T08:12:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.782+0000 7fd0d4c00640 1 -- 172.21.15.63:0/3149032176 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd0dc003ca0 con 0x55fb291c94d0 2024-05-27T08:12:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.782+0000 7fd0f8777740 10 client.22785 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.782+0000 7fd0f8777740 10 client.22785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.782+0000 7fd0f8777740 20 client.22785 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.782+0000 7fd0f8777740 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb291a8d80 0x55fb295a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.782+0000 7fd0f8777740 4 client.22785 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.782+0000 7fd0f8777740 1 -- 172.21.15.63:0/3149032176 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb294ffae0 con 0x55fb291a8d80 2024-05-27T08:12:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.783+0000 7fd0ea000640 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb291a8d80 0x55fb295a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.784+0000 7fd0ea000640 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb291a8d80 0x55fb295a8c50 crc :-1 s=READY pgs=686 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6e102f461906ee5 server_cookie=6537ca564eac2eed in_seq=0 out_seq=0 2024-05-27T08:12:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.784+0000 7fd0d4c00640 10 client.22785 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0d4c00640 1 -- 172.21.15.63:0/3149032176 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fb294ffae0 con 0x55fb291a8d80 2024-05-27T08:12:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0d4c00640 10 client.22785 handle_command_reply: tid=0 2024-05-27T08:12:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 client.22785 shutdown 2024-05-27T08:12:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 -- 172.21.15.63:0/3149032176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 msgr2=0x55fb29596f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:15.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb29596f90 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec106b70 tx=0x7fd0dc005270 comp rx=0 tx=0).stop 2024-05-27T08:12:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 -- 172.21.15.63:0/3149032176 shutdown_connections 2024-05-27T08:12:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb291a8d80 0x55fb295a8c50 crc :-1 s=READY pgs=686 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291ea1b0 0x55fb295a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb29596f90 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 --2- 172.21.15.63:0/3149032176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291c12f0 0x55fb29596a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 -- 172.21.15.63:0/3149032176 >> 172.21.15.63:0/3149032176 conn(0x55fb291de300 msgr2=0x55fb2958c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0d4c00640 0 client.22785 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 -- 172.21.15.63:0/3149032176 shutdown_connections 2024-05-27T08:12:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.786+0000 7fd0f8777740 1 -- 172.21.15.63:0/3149032176 wait complete. 2024-05-27T08:12:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.787+0000 7fd0f8777740 20 client.22785 trim_cache size 0 max 16384 2024-05-27T08:12:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.804+0000 7fd0f8777740 1 Processor -- start 2024-05-27T08:12:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.804+0000 7fd0f8777740 1 -- start start 2024-05-27T08:12:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.804+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c12f0 0x55fb29592280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.804+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c94d0 0x55fb295a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.804+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291ea1b0 0x55fb295a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.804+0000 7fd0f8777740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b5e00 con 0x55fb291c12f0 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0f8777740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b36d0 con 0x55fb291c94d0 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0f8777740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b1540 con 0x55fb291ea1b0 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0ea000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c12f0 0x55fb29592280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0ea000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c12f0 0x55fb29592280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52812/0 (socket says 172.21.15.63:52812) 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0ea000640 1 -- 172.21.15.63:0/3516178135 learned_addr learned my addr 172.21.15.63:0/3516178135 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/3516178135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c94d0 0x55fb295a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0ea000640 1 -- 172.21.15.63:0/3516178135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291ea1b0 msgr2=0x55fb295a2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0ea000640 1 --2- 172.21.15.63:0/3516178135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291ea1b0 0x55fb295a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0ea000640 1 -- 172.21.15.63:0/3516178135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c94d0 msgr2=0x55fb295a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0ea000640 1 --2- 172.21.15.63:0/3516178135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c94d0 0x55fb295a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.805+0000 7fd0ea000640 1 -- 172.21.15.63:0/3516178135 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb2958d520 con 0x55fb291c12f0 2024-05-27T08:12:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.806+0000 7fd0ea000640 1 --2- 172.21.15.63:0/3516178135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c12f0 0x55fb29592280 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8014830 tx=0x7fd0d800b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92abaa29bdb51022 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.806+0000 7fd0d7400640 1 -- 172.21.15.63:0/3516178135 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8032070 con 0x55fb291c12f0 2024-05-27T08:12:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.806+0000 7fd0d7400640 1 -- 172.21.15.63:0/3516178135 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0d80144f0 con 0x55fb291c12f0 2024-05-27T08:12:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.806+0000 7fd0d7400640 1 -- 172.21.15.63:0/3516178135 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d802d410 con 0x55fb291c12f0 2024-05-27T08:12:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.807+0000 7fd0f8777740 1 -- 172.21.15.63:0/3516178135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c12f0 msgr2=0x55fb29592280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.807+0000 7fd0f8777740 1 --2- 172.21.15.63:0/3516178135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c12f0 0x55fb29592280 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8014830 tx=0x7fd0d800b3c0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.807+0000 7fd0f8777740 1 -- 172.21.15.63:0/3516178135 shutdown_connections 2024-05-27T08:12:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.807+0000 7fd0f8777740 1 --2- 172.21.15.63:0/3516178135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291ea1b0 0x55fb295a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.807+0000 7fd0f8777740 1 --2- 172.21.15.63:0/3516178135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c94d0 0x55fb295a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.807+0000 7fd0f8777740 1 --2- 172.21.15.63:0/3516178135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c12f0 0x55fb29592280 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.807+0000 7fd0f8777740 1 -- 172.21.15.63:0/3516178135 >> 172.21.15.63:0/3516178135 conn(0x55fb291de300 msgr2=0x55fb29598ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.808+0000 7fd0f8777740 1 -- 172.21.15.63:0/3516178135 shutdown_connections 2024-05-27T08:12:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.808+0000 7fd0f8777740 1 -- 172.21.15.63:0/3516178135 wait complete. 2024-05-27T08:12:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.808+0000 7fd0f8777740 1 -- start start 2024-05-27T08:12:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.808+0000 7fd0f8777740 1 Processor -- start 2024-05-27T08:12:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.809+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c12f0 0x55fb2959c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.809+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb2959c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.809+0000 7fd0f8777740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291ea1b0 0x55fb2960f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.809+0000 7fd0f8777740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b1b90 con 0x55fb291c94d0 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.809+0000 7fd0f8777740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b3000 con 0x55fb291c12f0 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.809+0000 7fd0f8777740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b2260 con 0x55fb291ea1b0 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.809+0000 7fd0f8777740 10 client.0 _ll_register_callbacks cb 0x7fd0dc004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.809+0000 7fd0eaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb2959c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.809+0000 7fd0eaa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb2959c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52820/0 (socket says 172.21.15.63:52820) 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.809+0000 7fd0eaa00640 1 -- 172.21.15.63:0/1222744454 learned_addr learned my addr 172.21.15.63:0/1222744454 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.810+0000 7fd0ea000640 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c12f0 0x55fb2959c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.810+0000 7fd0ea000640 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291ea1b0 0x55fb2960f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.810+0000 7fd0eaa00640 1 -- 172.21.15.63:0/1222744454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291ea1b0 msgr2=0x55fb2960f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.810+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291ea1b0 0x55fb2960f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.810+0000 7fd0eaa00640 1 -- 172.21.15.63:0/1222744454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c12f0 msgr2=0x55fb2959c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.810+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c12f0 0x55fb2959c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.810+0000 7fd0eaa00640 1 -- 172.21.15.63:0/1222744454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2960f870 con 0x55fb291c94d0 2024-05-27T08:12:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.810+0000 7fd0ea000640 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291ea1b0 0x55fb2960f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:12:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.810+0000 7fd0eaa00640 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb2959c600 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009850 tx=0x7fd0dc0459b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac206037529af44b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.810+0000 7fd0d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.815+0000 7fd0d4c00640 1 -- 172.21.15.63:0/1222744454 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc035070 con 0x55fb291c94d0 2024-05-27T08:12:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.815+0000 7fd0d4c00640 1 -- 172.21.15.63:0/1222744454 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(157..157 src has 1..157) ==== 9571+0+0 (secure 0 0 0) 0x7fd0dc045d30 con 0x55fb291c94d0 2024-05-27T08:12:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.815+0000 7fd0f8777740 1 -- 172.21.15.63:0/1222744454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0d80144f0 con 0x55fb291c94d0 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.816+0000 7fd0d4c00640 1 -- 172.21.15.63:0/1222744454 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd0dc0046c0 con 0x55fb291c94d0 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.816+0000 7fd0f8777740 10 client.22791 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.816+0000 7fd0f8777740 1 -- 172.21.15.63:0/1222744454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fb29610a90 con 0x55fb291c94d0 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.816+0000 7fd0d4c00640 1 -- 172.21.15.63:0/1222744454 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0d80144f0 con 0x55fb291c94d0 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.816+0000 7fd0d4c00640 1 -- 172.21.15.63:0/1222744454 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd0dc003680 con 0x55fb291c94d0 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.816+0000 7fd0f8777740 10 client.22791 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.816+0000 7fd0f8777740 10 client.22791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.816+0000 7fd0f8777740 20 client.22791 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.816+0000 7fd0f8777740 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb2917e1c0 0x55fb29613240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.817+0000 7fd0f8777740 4 client.22791 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:12:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.817+0000 7fd0f8777740 1 -- 172.21.15.63:0/1222744454 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb2958d130 con 0x55fb2917e1c0 2024-05-27T08:12:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.817+0000 7fd0ea000640 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb2917e1c0 0x55fb29613240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.818+0000 7fd0ea000640 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb2917e1c0 0x55fb29613240 crc :-1 s=READY pgs=687 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3389ac445b6fcc server_cookie=e9fc9131e152509a in_seq=0 out_seq=0 2024-05-27T08:12:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.818+0000 7fd0d4c00640 10 client.22791 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.858+0000 7fd0d4c00640 1 -- 172.21.15.63:0/1222744454 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fb2958d130 con 0x55fb2917e1c0 2024-05-27T08:12:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.858+0000 7fd0d4c00640 10 client.22791 handle_command_reply: tid=0 2024-05-27T08:12:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.858+0000 7fd0f8777740 1 client.22791 shutdown 2024-05-27T08:12:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 1 -- 172.21.15.63:0/1222744454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 msgr2=0x55fb2959c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb2959c600 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009850 tx=0x7fd0dc0459b0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 1 -- 172.21.15.63:0/1222744454 shutdown_connections 2024-05-27T08:12:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb2917e1c0 0x55fb29613240 crc :-1 s=READY pgs=687 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb291ea1b0 0x55fb2960f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb291c94d0 0x55fb2959c600 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 1 --2- 172.21.15.63:0/1222744454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb291c12f0 0x55fb2959c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 1 -- 172.21.15.63:0/1222744454 >> 172.21.15.63:0/1222744454 conn(0x55fb291de300 msgr2=0x55fb29595f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0d4c00640 0 client.22791 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 1 -- 172.21.15.63:0/1222744454 shutdown_connections 2024-05-27T08:12:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 1 -- 172.21.15.63:0/1222744454 wait complete. 2024-05-27T08:12:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.859+0000 7fd0f8777740 20 client.22791 trim_cache size 0 max 16384 2024-05-27T08:12:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:12:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:12:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:12:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.860+0000 7fd0f1800640 1 -- 172.21.15.63:0/883372857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd0d00b2950 msgr2=0x7fd0d00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.860+0000 7fd0f1800640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd0d00b2950 0x7fd0d00b4e60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4005bc0 tx=0x7fd0e400cf50 comp rx=0 tx=0).stop 2024-05-27T08:12:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.860+0000 7fd0f1800640 1 -- 172.21.15.63:0/883372857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec10c330 msgr2=0x7fd0ec19c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.860+0000 7fd0f1800640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec10c330 0x7fd0ec19c850 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400eaa0 tx=0x7fd0e400ef60 comp rx=0 tx=0).stop 2024-05-27T08:12:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.860+0000 7fd0f1800640 1 -- 172.21.15.63:0/883372857 shutdown_connections 2024-05-27T08:12:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.860+0000 7fd0f1800640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd0d00b2950 0x7fd0d00b4e60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.860+0000 7fd0f1800640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec10c330 0x7fd0ec19c850 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.860+0000 7fd0f1800640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0ec10be90 0x7fd0ec1a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.860+0000 7fd0f1800640 1 --2- 172.21.15.63:0/883372857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0ec1065e0 0x7fd0ec1a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.860+0000 7fd0f1800640 1 -- 172.21.15.63:0/883372857 >> 172.21.15.63:0/883372857 conn(0x7fd0ec101530 msgr2=0x7fd0ec101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.861+0000 7fd0f1800640 1 -- 172.21.15.63:0/883372857 shutdown_connections 2024-05-27T08:12:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:15.861+0000 7fd0f1800640 1 -- 172.21.15.63:0/883372857 wait complete. 2024-05-27T08:12:16.133 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:15 smithi063 ceph-mon[31613]: pgmap v1227: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.5 MiB/s wr, 12 op/s 2024-05-27T08:12:16.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:15 smithi086 ceph-mon[34687]: pgmap v1227: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.5 MiB/s wr, 12 op/s 2024-05-27T08:12:16.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:15 smithi149 ceph-mon[35408]: pgmap v1227: 129 pgs: 129 active+clean; 11 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.5 MiB/s wr, 12 op/s 2024-05-27T08:12:17.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:17 smithi086 ceph-mon[34687]: osdmap e158: 12 total, 12 up, 12 in 2024-05-27T08:12:17.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:17 smithi086 ceph-mon[34687]: pgmap v1229: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 12 op/s 2024-05-27T08:12:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:17 smithi149 ceph-mon[35408]: osdmap e158: 12 total, 12 up, 12 in 2024-05-27T08:12:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:17 smithi149 ceph-mon[35408]: pgmap v1229: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 12 op/s 2024-05-27T08:12:17.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:17 smithi063 ceph-mon[31613]: osdmap e158: 12 total, 12 up, 12 in 2024-05-27T08:12:17.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:17 smithi063 ceph-mon[31613]: pgmap v1229: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 12 op/s 2024-05-27T08:12:18.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.600+0000 7fcfe3800700 1 -- 172.21.15.63:0/2300519475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfdc1054d0 msgr2=0x7fcfdc1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:18.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.600+0000 7fcfe3800700 1 --2- 172.21.15.63:0/2300519475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfdc1054d0 0x7fcfdc1058e0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009990 tx=0x7fcfd0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.600+0000 7fcfe3800700 1 -- 172.21.15.63:0/2300519475 shutdown_connections 2024-05-27T08:12:18.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.600+0000 7fcfe3800700 1 --2- 172.21.15.63:0/2300519475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfdc10e900 0x7fcfdc110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.600+0000 7fcfe3800700 1 --2- 172.21.15.63:0/2300519475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfdc1054d0 0x7fcfdc1058e0 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.600+0000 7fcfe3800700 1 --2- 172.21.15.63:0/2300519475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfdc105e20 0x7fcfdc10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.600+0000 7fcfe3800700 1 -- 172.21.15.63:0/2300519475 >> 172.21.15.63:0/2300519475 conn(0x7fcfdc0fedb0 msgr2=0x7fcfdc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:18.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.600+0000 7fcfe3800700 1 -- 172.21.15.63:0/2300519475 shutdown_connections 2024-05-27T08:12:18.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.600+0000 7fcfe3800700 1 -- 172.21.15.63:0/2300519475 wait complete. 2024-05-27T08:12:18.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.601+0000 7fcfe3800700 1 Processor -- start 2024-05-27T08:12:18.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.602+0000 7fcfe3800700 1 -- start start 2024-05-27T08:12:18.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.602+0000 7fcfe3800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfdc1054d0 0x7fcfdc19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.602+0000 7fcfe3800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfdc105e20 0x7fcfdc19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.602+0000 7fcfe3800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfdc10e900 0x7fcfdc1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.602+0000 7fcfe3800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1142d0 con 0x7fcfdc1054d0 2024-05-27T08:12:18.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.602+0000 7fcfe3800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc114150 con 0x7fcfdc105e20 2024-05-27T08:12:18.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.602+0000 7fcfe3800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc114450 con 0x7fcfdc10e900 2024-05-27T08:12:18.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.603+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfdc1054d0 0x7fcfdc19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.603+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfdc1054d0 0x7fcfdc19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52836/0 (socket says 172.21.15.63:52836) 2024-05-27T08:12:18.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.603+0000 7fcfe0e00700 1 -- 172.21.15.63:0/3365687705 learned_addr learned my addr 172.21.15.63:0/3365687705 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:18.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.603+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfdc105e20 0x7fcfdc19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.603+0000 7fcfe1800700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfdc10e900 0x7fcfdc1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.603+0000 7fcfe0e00700 1 -- 172.21.15.63:0/3365687705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfdc10e900 msgr2=0x7fcfdc1a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:18.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.603+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfdc10e900 0x7fcfdc1a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.603+0000 7fcfe0e00700 1 -- 172.21.15.63:0/3365687705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfdc105e20 msgr2=0x7fcfdc19d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:18.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.603+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfdc105e20 0x7fcfdc19d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.603+0000 7fcfe0e00700 1 -- 172.21.15.63:0/3365687705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc1a1e50 con 0x7fcfdc1054d0 2024-05-27T08:12:18.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.604+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfdc1054d0 0x7fcfdc19cf80 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0006010 tx=0x7fcfd00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:18.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.604+0000 7fcfd9600700 1 -- 172.21.15.63:0/3365687705 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0027070 con 0x7fcfdc1054d0 2024-05-27T08:12:18.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.604+0000 7fcfd9600700 1 -- 172.21.15.63:0/3365687705 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfd000fb90 con 0x7fcfdc1054d0 2024-05-27T08:12:18.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.604+0000 7fcfd9600700 1 -- 172.21.15.63:0/3365687705 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00173f0 con 0x7fcfdc1054d0 2024-05-27T08:12:18.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.604+0000 7fcfe3800700 1 -- 172.21.15.63:0/3365687705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc1a20e0 con 0x7fcfdc1054d0 2024-05-27T08:12:18.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.605+0000 7fcfe3800700 1 -- 172.21.15.63:0/3365687705 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1a2570 con 0x7fcfdc1054d0 2024-05-27T08:12:18.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.609+0000 7fcfd9600700 1 -- 172.21.15.63:0/3365687705 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcfd002a070 con 0x7fcfdc1054d0 2024-05-27T08:12:18.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.610+0000 7fcff2292b80 1 Processor -- start 2024-05-27T08:12:18.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.610+0000 7fcff2292b80 1 -- start start 2024-05-27T08:12:18.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.611+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dcab9f0 0x55b49dd3e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.611+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd3e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.611+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dd3ee70 0x55b49dd4b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.611+0000 7fcff2292b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b49dc6ce40 con 0x55b49dbbfe40 2024-05-27T08:12:18.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.611+0000 7fcff2292b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b49dc5c9e0 con 0x55b49dcab9f0 2024-05-27T08:12:18.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.611+0000 7fcff2292b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b49dc60a20 con 0x55b49dd3ee70 2024-05-27T08:12:18.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.611+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd3e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.611+0000 7fcfd9600700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcfc40b2a70 0x7fcfc40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.611+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd3e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52848/0 (socket says 172.21.15.63:52848) 2024-05-27T08:12:18.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.611+0000 7fcfe0e00700 1 -- 172.21.15.63:0/3019821088 learned_addr learned my addr 172.21.15.63:0/3019821088 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:18.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.612+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/3019821088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dd3ee70 0x55b49dd4b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.612+0000 7fcfe1800700 1 --2- 172.21.15.63:0/3019821088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dcab9f0 0x55b49dd3e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.612+0000 7fcfd9600700 1 -- 172.21.15.63:0/3365687705 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7fcfd00e3920 con 0x7fcfdc1054d0 2024-05-27T08:12:18.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.612+0000 7fcfe1800700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcfc40b2a70 0x7fcfc40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.612+0000 7fcfdbe00700 1 -- 172.21.15.63:0/3019821088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dcab9f0 msgr2=0x55b49dd3e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:18.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.612+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/3019821088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dcab9f0 0x55b49dd3e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.612+0000 7fcfdbe00700 1 -- 172.21.15.63:0/3019821088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 msgr2=0x55b49dd3e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:18.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.612+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/3019821088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd3e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.612+0000 7fcfdbe00700 1 -- 172.21.15.63:0/3019821088 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49dc25420 con 0x55b49dd3ee70 2024-05-27T08:12:18.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.612+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/3019821088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd3e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:12:18.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.613+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/3019821088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dd3ee70 0x55b49dd4b110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005950 tx=0x7fcfc800bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e1d836f2ecb21cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:18.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.613+0000 7fcfe1800700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcfc40b2a70 0x7fcfc40b4f80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55b49dd40ff0 tx=0x7fcfd400b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:18.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.613+0000 7fcfce000700 1 -- 172.21.15.63:0/3019821088 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8010c70 con 0x55b49dd3ee70 2024-05-27T08:12:18.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.614+0000 7fcfce000700 1 -- 172.21.15.63:0/3019821088 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfc8010e10 con 0x55b49dd3ee70 2024-05-27T08:12:18.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.614+0000 7fcfce000700 1 -- 172.21.15.63:0/3019821088 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8005020 con 0x55b49dd3ee70 2024-05-27T08:12:18.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.615+0000 7fcff2292b80 1 -- 172.21.15.63:0/3019821088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dd3ee70 msgr2=0x55b49dd4b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:18.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.615+0000 7fcff2292b80 1 --2- 172.21.15.63:0/3019821088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dd3ee70 0x55b49dd4b110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005950 tx=0x7fcfc800bf60 comp rx=0 tx=0).stop 2024-05-27T08:12:18.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.615+0000 7fcff2292b80 1 -- 172.21.15.63:0/3019821088 shutdown_connections 2024-05-27T08:12:18.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.615+0000 7fcff2292b80 1 --2- 172.21.15.63:0/3019821088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dd3ee70 0x55b49dd4b110 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.615+0000 7fcff2292b80 1 --2- 172.21.15.63:0/3019821088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dcab9f0 0x55b49dd3e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.615+0000 7fcff2292b80 1 --2- 172.21.15.63:0/3019821088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd3e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.615+0000 7fcff2292b80 1 -- 172.21.15.63:0/3019821088 >> 172.21.15.63:0/3019821088 conn(0x55b49dc59390 msgr2=0x55b49dd3a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:18.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.615+0000 7fcff2292b80 1 -- 172.21.15.63:0/3019821088 shutdown_connections 2024-05-27T08:12:18.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.615+0000 7fcff2292b80 1 -- 172.21.15.63:0/3019821088 wait complete. 2024-05-27T08:12:18.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.616+0000 7fcff2292b80 1 -- start start 2024-05-27T08:12:18.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.616+0000 7fcff2292b80 1 Processor -- start 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd48610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 0x55b49dd48b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 0x55b49dd49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcff2292b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b49db62fa0 con 0x55b49dbbfe40 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcff2292b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b49dbfd760 con 0x55b49dd3ee70 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcff2292b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b49dd4bb80 con 0x55b49dcab9f0 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcff2292b80 10 client.0 _ll_register_callbacks cb 0x55b49dc06c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd48610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd48610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52854/0 (socket says 172.21.15.63:52854) 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcfe0e00700 1 -- 172.21.15.63:0/4086188554 learned_addr learned my addr 172.21.15.63:0/4086188554 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:18.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 0x55b49dd48b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.617+0000 7fcfe1800700 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 0x55b49dd49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.618+0000 7fcfe0e00700 1 -- 172.21.15.63:0/4086188554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 msgr2=0x55b49dd48b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.618+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 0x55b49dd48b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.618+0000 7fcfe0e00700 1 -- 172.21.15.63:0/4086188554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 msgr2=0x55b49dd49090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.618+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 0x55b49dd49090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.618+0000 7fcfe0e00700 1 -- 172.21.15.63:0/4086188554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49dbac8d0 con 0x55b49dbbfe40 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.618+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 0x55b49dd48b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.618+0000 7fcfe1800700 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 0x55b49dd49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.618+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd48610 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0005f50 tx=0x7fcfd002a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4039d9162afc666 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.618+0000 7fcfc2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.619+0000 7fcfc2a00700 1 -- 172.21.15.63:0/4086188554 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0031040 con 0x55b49dbbfe40 2024-05-27T08:12:18.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.619+0000 7fcfc2a00700 1 -- 172.21.15.63:0/4086188554 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7fcfd00b31b0 con 0x55b49dbbfe40 2024-05-27T08:12:18.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.619+0000 7fcff2292b80 1 -- 172.21.15.63:0/4086188554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc8010e10 con 0x55b49dbbfe40 2024-05-27T08:12:18.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.619+0000 7fcfc2a00700 1 -- 172.21.15.63:0/4086188554 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcfd00a9fe0 con 0x55b49dbbfe40 2024-05-27T08:12:18.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.619+0000 7fcff2292b80 10 client.22803 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:18.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.619+0000 7fcff2292b80 1 -- 172.21.15.63:0/4086188554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b49dc2d060 con 0x55b49dbbfe40 2024-05-27T08:12:18.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.619+0000 7fcfc2a00700 1 -- 172.21.15.63:0/4086188554 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfc8010e10 con 0x55b49dbbfe40 2024-05-27T08:12:18.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.620+0000 7fcfc2a00700 1 -- 172.21.15.63:0/4086188554 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcfd00abba0 con 0x55b49dbbfe40 2024-05-27T08:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.620+0000 7fcff2292b80 10 client.22803 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.620+0000 7fcff2292b80 10 client.22803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.620+0000 7fcff2292b80 20 client.22803 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.620+0000 7fcff2292b80 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b49dd58920 0x55b49dd5ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.620+0000 7fcff2292b80 4 client.22803 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.620+0000 7fcff2292b80 1 -- 172.21.15.63:0/4086188554 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b49dc6ed90 con 0x55b49dd58920 2024-05-27T08:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.621+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b49dd58920 0x55b49dd5ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.622+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b49dd58920 0x55b49dd5ad60 crc :-1 s=READY pgs=688 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a318b7b6cb4ede8 server_cookie=a458f9f489a313f2 in_seq=0 out_seq=0 2024-05-27T08:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.622+0000 7fcfc2a00700 10 client.22803 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:18.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.623+0000 7fcfc2a00700 1 -- 172.21.15.63:0/4086188554 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b49dc6ed90 con 0x55b49dd58920 2024-05-27T08:12:18.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.623+0000 7fcfc2a00700 10 client.22803 handle_command_reply: tid=0 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 client.22803 shutdown 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 -- 172.21.15.63:0/4086188554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 msgr2=0x55b49dd48610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd48610 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0005f50 tx=0x7fcfd002a000 comp rx=0 tx=0).stop 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 -- 172.21.15.63:0/4086188554 shutdown_connections 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b49dd58920 0x55b49dd5ad60 crc :-1 s=READY pgs=688 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 0x55b49dd49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd48610 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 --2- 172.21.15.63:0/4086188554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 0x55b49dd48b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 -- 172.21.15.63:0/4086188554 >> 172.21.15.63:0/4086188554 conn(0x55b49dc59390 msgr2=0x55b49dd3e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcfc2a00700 0 client.22803 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 -- 172.21.15.63:0/4086188554 shutdown_connections 2024-05-27T08:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.624+0000 7fcff2292b80 1 -- 172.21.15.63:0/4086188554 wait complete. 2024-05-27T08:12:18.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.625+0000 7fcff2292b80 20 client.22803 trim_cache size 0 max 16384 2024-05-27T08:12:18.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.649+0000 7fcff2292b80 1 Processor -- start 2024-05-27T08:12:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.649+0000 7fcff2292b80 1 -- start start 2024-05-27T08:12:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dbbfe40 0x55b49dd47930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dcab9f0 0x55b49dd54c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dd3ee70 0x55b49dd405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcff2292b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1b4fc0 con 0x55b49dd3ee70 2024-05-27T08:12:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcff2292b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1b2850 con 0x55b49dcab9f0 2024-05-27T08:12:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcff2292b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1b06c0 con 0x55b49dbbfe40 2024-05-27T08:12:18.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcfe1800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dd3ee70 0x55b49dd405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcfe1800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dd3ee70 0x55b49dd405e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52862/0 (socket says 172.21.15.63:52862) 2024-05-27T08:12:18.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcfe1800700 1 -- 172.21.15.63:0/1348643285 learned_addr learned my addr 172.21.15.63:0/1348643285 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:18.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/1348643285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dcab9f0 0x55b49dd54c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.650+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/1348643285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dbbfe40 0x55b49dd47930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.651+0000 7fcfe1800700 1 -- 172.21.15.63:0/1348643285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dbbfe40 msgr2=0x55b49dd47930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:18.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.651+0000 7fcfe1800700 1 --2- 172.21.15.63:0/1348643285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dbbfe40 0x55b49dd47930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.651+0000 7fcfe1800700 1 -- 172.21.15.63:0/1348643285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dcab9f0 msgr2=0x55b49dd54c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:18.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.651+0000 7fcfe1800700 1 --2- 172.21.15.63:0/1348643285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dcab9f0 0x55b49dd54c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.651+0000 7fcfe1800700 1 -- 172.21.15.63:0/1348643285 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49dd36240 con 0x55b49dd3ee70 2024-05-27T08:12:18.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.651+0000 7fcfe1800700 1 --2- 172.21.15.63:0/1348643285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dd3ee70 0x55b49dd405e0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009280 tx=0x7fcfd400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d60d9512d34fcda server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:18.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.652+0000 7fcfcd600700 1 -- 172.21.15.63:0/1348643285 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd40093e0 con 0x55b49dd3ee70 2024-05-27T08:12:18.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.652+0000 7fcfcd600700 1 -- 172.21.15.63:0/1348643285 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfd402a070 con 0x55b49dd3ee70 2024-05-27T08:12:18.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.652+0000 7fcfcd600700 1 -- 172.21.15.63:0/1348643285 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd401ac40 con 0x55b49dd3ee70 2024-05-27T08:12:18.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.653+0000 7fcff2292b80 1 -- 172.21.15.63:0/1348643285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dd3ee70 msgr2=0x55b49dd405e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:18.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.653+0000 7fcff2292b80 1 --2- 172.21.15.63:0/1348643285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dd3ee70 0x55b49dd405e0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009280 tx=0x7fcfd400f000 comp rx=0 tx=0).stop 2024-05-27T08:12:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.653+0000 7fcff2292b80 1 -- 172.21.15.63:0/1348643285 shutdown_connections 2024-05-27T08:12:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.653+0000 7fcff2292b80 1 --2- 172.21.15.63:0/1348643285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dd3ee70 0x55b49dd405e0 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.653+0000 7fcff2292b80 1 --2- 172.21.15.63:0/1348643285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dbbfe40 0x55b49dd47930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.653+0000 7fcff2292b80 1 --2- 172.21.15.63:0/1348643285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dcab9f0 0x55b49dd54c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.653+0000 7fcff2292b80 1 -- 172.21.15.63:0/1348643285 >> 172.21.15.63:0/1348643285 conn(0x55b49dc59390 msgr2=0x55b49dd3fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.653+0000 7fcff2292b80 1 -- 172.21.15.63:0/1348643285 shutdown_connections 2024-05-27T08:12:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.653+0000 7fcff2292b80 1 -- 172.21.15.63:0/1348643285 wait complete. 2024-05-27T08:12:18.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.654+0000 7fcff2292b80 1 -- start start 2024-05-27T08:12:18.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.654+0000 7fcff2292b80 1 Processor -- start 2024-05-27T08:12:18.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 0x55b49dd42d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcff2292b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 0x55b49dd432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcff2292b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1b0d10 con 0x55b49dbbfe40 2024-05-27T08:12:18.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcff2292b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1b2180 con 0x55b49dd3ee70 2024-05-27T08:12:18.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcff2292b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1b13e0 con 0x55b49dcab9f0 2024-05-27T08:12:18.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcff2292b80 10 client.0 _ll_register_callbacks cb 0x55b49dc06c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:18.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52866/0 (socket says 172.21.15.63:52866) 2024-05-27T08:12:18.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcfe0e00700 1 -- 172.21.15.63:0/665974438 learned_addr learned my addr 172.21.15.63:0/665974438 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:18.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.655+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 0x55b49dd42d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.656+0000 7fcfe1800700 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 0x55b49dd432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.656+0000 7fcfe0e00700 1 -- 172.21.15.63:0/665974438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 msgr2=0x55b49dd42d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:18.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.656+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 0x55b49dd42d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.656+0000 7fcfe0e00700 1 -- 172.21.15.63:0/665974438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 msgr2=0x55b49dd432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:18.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.656+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 0x55b49dd432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.656+0000 7fcfe0e00700 1 -- 172.21.15.63:0/665974438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49dd439a0 con 0x55b49dbbfe40 2024-05-27T08:12:18.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.656+0000 7fcfe1800700 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 0x55b49dd432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:12:18.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.656+0000 7fcfe0e00700 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd514b0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00a6e10 tx=0x7fcfd00a6130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75187e4d61b7eb42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:18.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.656+0000 7fcfc2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:18.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.656+0000 7fcfc2a00700 1 -- 172.21.15.63:0/665974438 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00a6730 con 0x55b49dbbfe40 2024-05-27T08:12:18.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.657+0000 7fcfc2a00700 1 -- 172.21.15.63:0/665974438 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7fcfd00a6e60 con 0x55b49dbbfe40 2024-05-27T08:12:18.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.657+0000 7fcff2292b80 1 -- 172.21.15.63:0/665974438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd402a070 con 0x55b49dbbfe40 2024-05-27T08:12:18.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.657+0000 7fcfc2a00700 1 -- 172.21.15.63:0/665974438 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcfd00aa380 con 0x55b49dbbfe40 2024-05-27T08:12:18.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.657+0000 7fcff2292b80 10 client.22809 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:18.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.657+0000 7fcff2292b80 1 -- 172.21.15.63:0/665974438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b49ddc33f0 con 0x55b49dbbfe40 2024-05-27T08:12:18.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.657+0000 7fcfc2a00700 1 -- 172.21.15.63:0/665974438 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfd402a070 con 0x55b49dbbfe40 2024-05-27T08:12:18.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.657+0000 7fcfc2a00700 1 -- 172.21.15.63:0/665974438 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcfd0005110 con 0x55b49dbbfe40 2024-05-27T08:12:18.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.657+0000 7fcff2292b80 10 client.22809 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:18.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.657+0000 7fcff2292b80 10 client.22809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:18.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.657+0000 7fcff2292b80 20 client.22809 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:18.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.658+0000 7fcff2292b80 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b49ddc3790 0x55b49ddc5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:18.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.658+0000 7fcff2292b80 4 client.22809 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:12:18.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.658+0000 7fcff2292b80 1 -- 172.21.15.63:0/665974438 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b49dc68570 con 0x55b49ddc3790 2024-05-27T08:12:18.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.658+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b49ddc3790 0x55b49ddc5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:18.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.659+0000 7fcfdbe00700 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b49ddc3790 0x55b49ddc5bb0 crc :-1 s=READY pgs=689 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56581166865dda4b server_cookie=42508f436499fcfc in_seq=0 out_seq=0 2024-05-27T08:12:18.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.659+0000 7fcfc2a00700 10 client.22809 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:18.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcfc2a00700 1 -- 172.21.15.63:0/665974438 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b49dc68570 con 0x55b49ddc3790 2024-05-27T08:12:18.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcfc2a00700 10 client.22809 handle_command_reply: tid=0 2024-05-27T08:12:18.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 client.22809 shutdown 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 -- 172.21.15.63:0/665974438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 msgr2=0x55b49dd514b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd514b0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00a6e10 tx=0x7fcfd00a6130 comp rx=0 tx=0).stop 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 -- 172.21.15.63:0/665974438 shutdown_connections 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b49ddc3790 0x55b49ddc5bb0 crc :-1 s=READY pgs=689 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49dd3ee70 0x55b49dd432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b49dbbfe40 0x55b49dd514b0 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 --2- 172.21.15.63:0/665974438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b49dcab9f0 0x55b49dd42d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 -- 172.21.15.63:0/665974438 >> 172.21.15.63:0/665974438 conn(0x55b49dc59390 msgr2=0x55b49dd47690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcfc2a00700 0 client.22809 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 -- 172.21.15.63:0/665974438 shutdown_connections 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.660+0000 7fcff2292b80 1 -- 172.21.15.63:0/665974438 wait complete. 2024-05-27T08:12:18.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.661+0000 7fcff2292b80 20 client.22809 trim_cache size 0 max 16384 2024-05-27T08:12:18.664 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:12:18.664 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:12:18.664 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:12:18.664 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:12:18.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 -- 172.21.15.63:0/3365687705 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcfc40b2a70 msgr2=0x7fcfc40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:18.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcfc40b2a70 0x7fcfc40b4f80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55b49dd40ff0 tx=0x7fcfd400b040 comp rx=0 tx=0).stop 2024-05-27T08:12:18.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 -- 172.21.15.63:0/3365687705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfdc1054d0 msgr2=0x7fcfdc19cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:18.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfdc1054d0 0x7fcfdc19cf80 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0006010 tx=0x7fcfd00057c0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 -- 172.21.15.63:0/3365687705 shutdown_connections 2024-05-27T08:12:18.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcfc40b2a70 0x7fcfc40b4f80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcfdc10e900 0x7fcfdc1a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfdc1054d0 0x7fcfdc19cf80 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 --2- 172.21.15.63:0/3365687705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfdc105e20 0x7fcfdc19d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:18.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 -- 172.21.15.63:0/3365687705 >> 172.21.15.63:0/3365687705 conn(0x7fcfdc0fedb0 msgr2=0x7fcfdc103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:18.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 -- 172.21.15.63:0/3365687705 shutdown_connections 2024-05-27T08:12:18.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:18.662+0000 7fcfe3800700 1 -- 172.21.15.63:0/3365687705 wait complete. 2024-05-27T08:12:19.655 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:12:19.656 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:12:19.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:19 smithi063 ceph-mon[31613]: pgmap v1230: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 12 op/s 2024-05-27T08:12:20.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:19 smithi086 ceph-mon[34687]: pgmap v1230: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 12 op/s 2024-05-27T08:12:20.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:19 smithi149 ceph-mon[35408]: pgmap v1230: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 12 op/s 2024-05-27T08:12:20.251 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:12:22.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:21 smithi086 ceph-mon[34687]: pgmap v1231: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 12 op/s 2024-05-27T08:12:22.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:21 smithi149 ceph-mon[35408]: pgmap v1231: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 12 op/s 2024-05-27T08:12:22.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:21 smithi063 ceph-mon[31613]: pgmap v1231: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 12 op/s 2024-05-27T08:12:23.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.366+0000 7f02be400700 1 -- 172.21.15.63:0/4020826859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b8109680 msgr2=0x7f02b8110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:23.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.366+0000 7f02be400700 1 --2- 172.21.15.63:0/4020826859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b8109680 0x7f02b8110fe0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f02b000b0d0 tx=0x7f02b000b3e0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.366+0000 7f02be400700 1 -- 172.21.15.63:0/4020826859 shutdown_connections 2024-05-27T08:12:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.366+0000 7f02be400700 1 --2- 172.21.15.63:0/4020826859 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b8109680 0x7f02b8110fe0 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.366+0000 7f02be400700 1 --2- 172.21.15.63:0/4020826859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b8104400 0x7f02b8104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.366+0000 7f02be400700 1 --2- 172.21.15.63:0/4020826859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b8108d30 0x7f02b8109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.366+0000 7f02be400700 1 -- 172.21.15.63:0/4020826859 >> 172.21.15.63:0/4020826859 conn(0x7f02b8079d40 msgr2=0x7f02b807a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.366+0000 7f02be400700 1 -- 172.21.15.63:0/4020826859 shutdown_connections 2024-05-27T08:12:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.366+0000 7f02be400700 1 -- 172.21.15.63:0/4020826859 wait complete. 2024-05-27T08:12:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.367+0000 7f02be400700 1 Processor -- start 2024-05-27T08:12:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.368+0000 7f02be400700 1 -- start start 2024-05-27T08:12:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.368+0000 7f02be400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b8104400 0x7f02b81a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.368+0000 7f02be400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b8108d30 0x7f02b81a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.368+0000 7f02be400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b8109680 0x7f02b819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.368+0000 7f02be400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02b81141b0 con 0x7f02b8104400 2024-05-27T08:12:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.368+0000 7f02be400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02b8114030 con 0x7f02b8108d30 2024-05-27T08:12:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.368+0000 7f02be400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02b8114330 con 0x7f02b8109680 2024-05-27T08:12:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.369+0000 7f02bd000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b8104400 0x7f02b81a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.369+0000 7f02bd000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b8104400 0x7f02b81a29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39266/0 (socket says 172.21.15.63:39266) 2024-05-27T08:12:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.369+0000 7f02bd000700 1 -- 172.21.15.63:0/2872805145 learned_addr learned my addr 172.21.15.63:0/2872805145 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.369+0000 7f02b7e00700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b8108d30 0x7f02b81a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.369+0000 7f02bda00700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b8109680 0x7f02b819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.369+0000 7f02bd000700 1 -- 172.21.15.63:0/2872805145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b8109680 msgr2=0x7f02b819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.369+0000 7f02bd000700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b8109680 0x7f02b819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.369+0000 7f02bd000700 1 -- 172.21.15.63:0/2872805145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b8108d30 msgr2=0x7f02b81a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.369+0000 7f02bd000700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b8108d30 0x7f02b81a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.369+0000 7f02bd000700 1 -- 172.21.15.63:0/2872805145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b819d280 con 0x7f02b8104400 2024-05-27T08:12:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.370+0000 7f02bd000700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b8104400 0x7f02b81a29c0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00d950 tx=0x7f02ac00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.370+0000 7f02b5600700 1 -- 172.21.15.63:0/2872805145 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac016070 con 0x7f02b8104400 2024-05-27T08:12:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.370+0000 7f02be400700 1 -- 172.21.15.63:0/2872805145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b819d570 con 0x7f02b8104400 2024-05-27T08:12:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.370+0000 7f02b5600700 1 -- 172.21.15.63:0/2872805145 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02ac010460 con 0x7f02b8104400 2024-05-27T08:12:23.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.370+0000 7f02b5600700 1 -- 172.21.15.63:0/2872805145 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00f620 con 0x7f02b8104400 2024-05-27T08:12:23.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.370+0000 7f02be400700 1 -- 172.21.15.63:0/2872805145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b81a9770 con 0x7f02b8104400 2024-05-27T08:12:23.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.372+0000 7f02be400700 1 -- 172.21.15.63:0/2872805145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b81058b0 con 0x7f02b8104400 2024-05-27T08:12:23.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.375+0000 7f02b5600700 1 -- 172.21.15.63:0/2872805145 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f02ac019070 con 0x7f02b8104400 2024-05-27T08:12:23.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.377+0000 7f02b5600700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02a00b2a70 0x7f02a00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:23.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.377+0000 7f02b5600700 1 -- 172.21.15.63:0/2872805145 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f02ac0e2960 con 0x7f02b8104400 2024-05-27T08:12:23.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.378+0000 7f02b7e00700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02a00b2a70 0x7f02a00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:23.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.379+0000 7f02b7e00700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02a00b2a70 0x7f02a00b4f80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f02b819e280 tx=0x7f02a400b5b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:23.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.382+0000 7f02b5600700 1 -- 172.21.15.63:0/2872805145 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02ac0a9f20 con 0x7f02b8104400 2024-05-27T08:12:23.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.719+0000 7f02be400700 1 -- 172.21.15.63:0/2872805145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02b80512b0 con 0x7f02b8104400 2024-05-27T08:12:23.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.721+0000 7f02b5600700 1 -- 172.21.15.63:0/2872805145 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7f02ac0a9820 con 0x7f02b8104400 2024-05-27T08:12:23.723 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:12:23.723 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:12:23.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.724+0000 7f02be400700 1 -- 172.21.15.63:0/2872805145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02a00b2a70 msgr2=0x7f02a00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:23.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.724+0000 7f02be400700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02a00b2a70 0x7f02a00b4f80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f02b819e280 tx=0x7f02a400b5b0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.724+0000 7f02be400700 1 -- 172.21.15.63:0/2872805145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b8104400 msgr2=0x7f02b81a29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:23.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.724+0000 7f02be400700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b8104400 0x7f02b81a29c0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00d950 tx=0x7f02ac00dd10 comp rx=0 tx=0).stop 2024-05-27T08:12:23.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.725+0000 7f02be400700 1 -- 172.21.15.63:0/2872805145 shutdown_connections 2024-05-27T08:12:23.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.725+0000 7f02be400700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02a00b2a70 0x7f02a00b4f80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.725+0000 7f02be400700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b8109680 0x7f02b819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.725+0000 7f02be400700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b8104400 0x7f02b81a29c0 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.725+0000 7f02be400700 1 --2- 172.21.15.63:0/2872805145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b8108d30 0x7f02b81a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:23.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.725+0000 7f02be400700 1 -- 172.21.15.63:0/2872805145 >> 172.21.15.63:0/2872805145 conn(0x7f02b8079d40 msgr2=0x7f02b8101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.725+0000 7f02be400700 1 -- 172.21.15.63:0/2872805145 shutdown_connections 2024-05-27T08:12:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:23.725+0000 7f02be400700 1 -- 172.21.15.63:0/2872805145 wait complete. 2024-05-27T08:12:23.728 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:12:24.078 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:23 smithi063 ceph-mon[31613]: pgmap v1232: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T08:12:24.078 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:23 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2872805145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:12:24.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:23 smithi086 ceph-mon[34687]: pgmap v1232: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T08:12:24.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:23 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2872805145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:12:24.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:23 smithi149 ceph-mon[35408]: pgmap v1232: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T08:12:24.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:23 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2872805145' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:12:24.644 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:12:25.167 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:12:25.868 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:25 smithi063 ceph-mon[31613]: pgmap v1233: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T08:12:26.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:25 smithi086 ceph-mon[34687]: pgmap v1233: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T08:12:26.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:25 smithi149 ceph-mon[35408]: pgmap v1233: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-05-27T08:12:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.034+0000 7f9a19000700 1 -- 172.21.15.63:0/2799778828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a14076c00 msgr2=0x7f9a14077010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.034+0000 7f9a19000700 1 --2- 172.21.15.63:0/2799778828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a14076c00 0x7f9a14077010 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009990 tx=0x7f9a04009ca0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.034+0000 7f9a19000700 1 -- 172.21.15.63:0/2799778828 shutdown_connections 2024-05-27T08:12:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.034+0000 7f9a19000700 1 --2- 172.21.15.63:0/2799778828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a14110bc0 0x7f9a14112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.034+0000 7f9a19000700 1 --2- 172.21.15.63:0/2799778828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a14076c00 0x7f9a14077010 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.034+0000 7f9a19000700 1 --2- 172.21.15.63:0/2799778828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a14077550 0x7f9a14110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.034+0000 7f9a19000700 1 -- 172.21.15.63:0/2799778828 >> 172.21.15.63:0/2799778828 conn(0x7f9a140ffea0 msgr2=0x7f9a141022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.034+0000 7f9a19000700 1 -- 172.21.15.63:0/2799778828 shutdown_connections 2024-05-27T08:12:28.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.034+0000 7f9a19000700 1 -- 172.21.15.63:0/2799778828 wait complete. 2024-05-27T08:12:28.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.035+0000 7f9a19000700 1 Processor -- start 2024-05-27T08:12:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.035+0000 7f9a19000700 1 -- start start 2024-05-27T08:12:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.036+0000 7f9a19000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a14076c00 0x7f9a1419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.036+0000 7f9a19000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a14077550 0x7f9a1419d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.036+0000 7f9a19000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a14110bc0 0x7f9a141a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.036+0000 7f9a19000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a14116550 con 0x7f9a14110bc0 2024-05-27T08:12:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.036+0000 7f9a19000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141163d0 con 0x7f9a14076c00 2024-05-27T08:12:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.036+0000 7f9a19000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141166d0 con 0x7f9a14077550 2024-05-27T08:12:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.036+0000 7f9a12800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a14110bc0 0x7f9a141a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.036+0000 7f9a12800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a14110bc0 0x7f9a141a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39294/0 (socket says 172.21.15.63:39294) 2024-05-27T08:12:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.036+0000 7f9a12800700 1 -- 172.21.15.63:0/823240498 learned_addr learned my addr 172.21.15.63:0/823240498 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.036+0000 7f9a11400700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a14077550 0x7f9a1419d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.037+0000 7f9a11e00700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a14076c00 0x7f9a1419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.037+0000 7f9a12800700 1 -- 172.21.15.63:0/823240498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a14077550 msgr2=0x7f9a1419d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.037+0000 7f9a12800700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a14077550 0x7f9a1419d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.037+0000 7f9a12800700 1 -- 172.21.15.63:0/823240498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a14076c00 msgr2=0x7f9a1419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:28.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.037+0000 7f9a12800700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a14076c00 0x7f9a1419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.037+0000 7f9a12800700 1 -- 172.21.15.63:0/823240498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a141a1e10 con 0x7f9a14110bc0 2024-05-27T08:12:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.037+0000 7f9a12800700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a14110bc0 0x7f9a141a1770 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0094f0 tx=0x7f9a0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.038+0000 7f9a0aa00700 1 -- 172.21.15.63:0/823240498 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c004db0 con 0x7f9a14110bc0 2024-05-27T08:12:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.038+0000 7f9a0aa00700 1 -- 172.21.15.63:0/823240498 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a0c009e20 con 0x7f9a14110bc0 2024-05-27T08:12:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.038+0000 7f9a19000700 1 -- 172.21.15.63:0/823240498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a141a2100 con 0x7f9a14110bc0 2024-05-27T08:12:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.038+0000 7f9a0aa00700 1 -- 172.21.15.63:0/823240498 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c015400 con 0x7f9a14110bc0 2024-05-27T08:12:28.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.038+0000 7f9a19000700 1 -- 172.21.15.63:0/823240498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a141a9a20 con 0x7f9a14110bc0 2024-05-27T08:12:28.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.043+0000 7f9a0aa00700 1 -- 172.21.15.63:0/823240498 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9a0c005260 con 0x7f9a14110bc0 2024-05-27T08:12:28.045 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:27 smithi063 ceph-mon[31613]: pgmap v1234: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.1 MiB/s wr, 16 op/s 2024-05-27T08:12:28.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.044+0000 7f9a27b04b80 1 Processor -- start 2024-05-27T08:12:28.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.044+0000 7f9a27b04b80 1 -- start start 2024-05-27T08:12:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.045+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29f903e40 0x55b29fa887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.045+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa82870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.045+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa82db0 0x55b29fa831c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.045+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b29f9b0e40 con 0x55b29fa88d30 2024-05-27T08:12:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.045+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b29f9a09e0 con 0x55b29fa82db0 2024-05-27T08:12:28.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.045+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b29f9a4a20 con 0x55b29f903e40 2024-05-27T08:12:28.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.045+0000 7f9a11e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa82870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.045+0000 7f9a11e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa82870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39310/0 (socket says 172.21.15.63:39310) 2024-05-27T08:12:28.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.045+0000 7f9a11e00700 1 -- 172.21.15.63:0/2611982014 learned_addr learned my addr 172.21.15.63:0/2611982014 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:28.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.046+0000 7f9a11400700 1 --2- 172.21.15.63:0/2611982014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29f903e40 0x55b29fa887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.046+0000 7f9a0aa00700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99f80b2a70 0x7f99f80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.046+0000 7f9a11400700 1 --2- 172.21.15.63:0/2611982014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa82db0 0x55b29fa831c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.046+0000 7f9a0aa00700 1 -- 172.21.15.63:0/823240498 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f9a0c0e2320 con 0x7f9a14110bc0 2024-05-27T08:12:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.046+0000 7f9a11e00700 1 -- 172.21.15.63:0/2611982014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29f903e40 msgr2=0x55b29fa887f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.046+0000 7f9a11e00700 1 --2- 172.21.15.63:0/2611982014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29f903e40 0x55b29fa887f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.046+0000 7f9a11e00700 1 -- 172.21.15.63:0/2611982014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa82db0 msgr2=0x55b29fa831c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.046+0000 7f9a11e00700 1 --2- 172.21.15.63:0/2611982014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa82db0 0x55b29fa831c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.046+0000 7f9a11e00700 1 -- 172.21.15.63:0/2611982014 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b29f969420 con 0x55b29fa88d30 2024-05-27T08:12:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.046+0000 7f9a11400700 1 --2- 172.21.15.63:0/2611982014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa82db0 0x55b29fa831c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:28.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.047+0000 7f9a11e00700 1 --2- 172.21.15.63:0/2611982014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa82870 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009990 tx=0x7f9a04005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373200cc8add9c87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:28.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.047+0000 7f9a12800700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99f80b2a70 0x7f99f80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.047+0000 7f99ff400700 1 -- 172.21.15.63:0/2611982014 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04027070 con 0x55b29fa88d30 2024-05-27T08:12:28.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.047+0000 7f99ff400700 1 -- 172.21.15.63:0/2611982014 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a040037c0 con 0x55b29fa88d30 2024-05-27T08:12:28.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.047+0000 7f99ff400700 1 -- 172.21.15.63:0/2611982014 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400bb00 con 0x55b29fa88d30 2024-05-27T08:12:28.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.048+0000 7f9a27b04b80 1 -- 172.21.15.63:0/2611982014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 msgr2=0x55b29fa82870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:28.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.048+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/2611982014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa82870 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009990 tx=0x7f9a04005f00 comp rx=0 tx=0).stop 2024-05-27T08:12:28.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.048+0000 7f9a12800700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99f80b2a70 0x7f99f80b4f80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0a6450 tx=0x7f9a0c0096d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:28.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.049+0000 7f9a27b04b80 1 -- 172.21.15.63:0/2611982014 shutdown_connections 2024-05-27T08:12:28.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.049+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/2611982014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa82db0 0x55b29fa831c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.049+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/2611982014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29f903e40 0x55b29fa887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.049+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/2611982014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa82870 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.049+0000 7f9a27b04b80 1 -- 172.21.15.63:0/2611982014 >> 172.21.15.63:0/2611982014 conn(0x55b29f963fc0 msgr2=0x55b29fa7d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:28.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.049+0000 7f9a27b04b80 1 -- 172.21.15.63:0/2611982014 shutdown_connections 2024-05-27T08:12:28.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.049+0000 7f9a27b04b80 1 -- 172.21.15.63:0/2611982014 wait complete. 2024-05-27T08:12:28.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.050+0000 7f9a27b04b80 1 -- start start 2024-05-27T08:12:28.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.050+0000 7f9a27b04b80 1 Processor -- start 2024-05-27T08:12:28.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29f903e40 0x55b29fa8c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29fa82db0 0x55b29fa8caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa8cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b29f8a6fa0 con 0x55b29fa88d30 2024-05-27T08:12:28.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b29f941760 con 0x55b29f903e40 2024-05-27T08:12:28.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b29fa97fb0 con 0x55b29fa82db0 2024-05-27T08:12:28.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a27b04b80 10 client.0 _ll_register_callbacks cb 0x55b29f94ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:28.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a11400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa8cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a11400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa8cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39316/0 (socket says 172.21.15.63:39316) 2024-05-27T08:12:28.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a11400700 1 -- 172.21.15.63:0/1229328059 learned_addr learned my addr 172.21.15.63:0/1229328059 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:28.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a11e00700 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29fa82db0 0x55b29fa8caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.051+0000 7f9a11400700 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29f903e40 0x55b29fa8c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.052+0000 7f9a11400700 1 -- 172.21.15.63:0/1229328059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29fa82db0 msgr2=0x55b29fa8caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:28.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.052+0000 7f9a11400700 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29fa82db0 0x55b29fa8caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.052+0000 7f9a11400700 1 -- 172.21.15.63:0/1229328059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29f903e40 msgr2=0x55b29fa8c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:28.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.052+0000 7f9a11400700 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29f903e40 0x55b29fa8c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.052+0000 7f9a11400700 1 -- 172.21.15.63:0/1229328059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b29f8f08d0 con 0x55b29fa88d30 2024-05-27T08:12:28.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.052+0000 7f9a11400700 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa8cfe0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f9a1406c430 tx=0x7f9a00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da8a21f5db66246 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:28.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.052+0000 7f99f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:28.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.053+0000 7f99f3e00700 1 -- 172.21.15.63:0/1229328059 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00012660 con 0x55b29fa88d30 2024-05-27T08:12:28.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.053+0000 7f99f3e00700 1 -- 172.21.15.63:0/1229328059 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f9a00012800 con 0x55b29fa88d30 2024-05-27T08:12:28.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.053+0000 7f9a27b04b80 1 -- 172.21.15.63:0/1229328059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a040037c0 con 0x55b29fa88d30 2024-05-27T08:12:28.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.053+0000 7f99f3e00700 1 -- 172.21.15.63:0/1229328059 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9a0002a050 con 0x55b29fa88d30 2024-05-27T08:12:28.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.053+0000 7f9a27b04b80 10 client.22827 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:28.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.053+0000 7f9a27b04b80 1 -- 172.21.15.63:0/1229328059 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b29f971060 con 0x55b29fa88d30 2024-05-27T08:12:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.054+0000 7f99f3e00700 1 -- 172.21.15.63:0/1229328059 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a040037c0 con 0x55b29fa88d30 2024-05-27T08:12:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.054+0000 7f99f3e00700 1 -- 172.21.15.63:0/1229328059 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9a00010820 con 0x55b29fa88d30 2024-05-27T08:12:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.054+0000 7f9a27b04b80 10 client.22827 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.054+0000 7f9a27b04b80 10 client.22827 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:12:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.054+0000 7f9a27b04b80 20 client.22827 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.054+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b29faa5000 0x55b29faa7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.054+0000 7f9a27b04b80 4 client.22827 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.054+0000 7f9a27b04b80 1 -- 172.21.15.63:0/1229328059 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b29f9b2d90 con 0x55b29faa5000 2024-05-27T08:12:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.055+0000 7f9a11e00700 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b29faa5000 0x55b29faa7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.056+0000 7f9a11e00700 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b29faa5000 0x55b29faa7420 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bea68462d1e3be5e server_cookie=1c8fdd425fcf5bac in_seq=0 out_seq=0 2024-05-27T08:12:28.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.056+0000 7f99f3e00700 10 client.22827 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:12:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.058+0000 7f99f3e00700 1 -- 172.21.15.63:0/1229328059 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b29f9b2d90 con 0x55b29faa5000 2024-05-27T08:12:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.058+0000 7f99f3e00700 10 client.22827 handle_command_reply: tid=0 2024-05-27T08:12:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.058+0000 7f9a27b04b80 1 client.22827 shutdown 2024-05-27T08:12:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f9a27b04b80 1 -- 172.21.15.63:0/1229328059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 msgr2=0x55b29fa8cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:28.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa8cfe0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f9a1406c430 tx=0x7f9a00004750 comp rx=0 tx=0).stop 2024-05-27T08:12:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f9a27b04b80 1 -- 172.21.15.63:0/1229328059 shutdown_connections 2024-05-27T08:12:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b29faa5000 0x55b29faa7420 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa88d30 0x55b29fa8cfe0 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29f903e40 0x55b29fa8c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/1229328059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29fa82db0 0x55b29fa8caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f9a27b04b80 1 -- 172.21.15.63:0/1229328059 >> 172.21.15.63:0/1229328059 conn(0x55b29f963fc0 msgr2=0x55b29fa7d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f99f3e00700 0 client.22827 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:12:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f9a27b04b80 1 -- 172.21.15.63:0/1229328059 shutdown_connections 2024-05-27T08:12:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.059+0000 7f9a27b04b80 1 -- 172.21.15.63:0/1229328059 wait complete. 2024-05-27T08:12:28.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.060+0000 7f9a27b04b80 20 client.22827 trim_cache size 0 max 16384 2024-05-27T08:12:28.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.076+0000 7f9a27b04b80 1 Processor -- start 2024-05-27T08:12:28.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.077+0000 7f9a27b04b80 1 -- start start 2024-05-27T08:12:28.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.077+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29f903e40 0x55b29fa82870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.077+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa86c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.077+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29fa88d30 0x55b29fa8ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.077+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141b0b30 con 0x55b29fa82db0 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.077+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141ae3c0 con 0x55b29f903e40 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.077+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141ac230 con 0x55b29fa88d30 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.077+0000 7f9a11e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa86c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.077+0000 7f9a11e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa86c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39326/0 (socket says 172.21.15.63:39326) 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.077+0000 7f9a11e00700 1 -- 172.21.15.63:0/700215413 learned_addr learned my addr 172.21.15.63:0/700215413 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.078+0000 7f9a11400700 1 --2- 172.21.15.63:0/700215413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29f903e40 0x55b29fa82870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.078+0000 7f9a11400700 1 --2- 172.21.15.63:0/700215413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29fa88d30 0x55b29fa8ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.078+0000 7f9a11e00700 1 -- 172.21.15.63:0/700215413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29fa88d30 msgr2=0x55b29fa8ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.078+0000 7f9a11e00700 1 --2- 172.21.15.63:0/700215413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29fa88d30 0x55b29fa8ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.078+0000 7f9a11e00700 1 -- 172.21.15.63:0/700215413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29f903e40 msgr2=0x55b29fa82870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.078+0000 7f9a11e00700 1 --2- 172.21.15.63:0/700215413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29f903e40 0x55b29fa82870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.078+0000 7f9a11e00700 1 -- 172.21.15.63:0/700215413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b29faa29d0 con 0x55b29fa82db0 2024-05-27T08:12:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.078+0000 7f9a11e00700 1 --2- 172.21.15.63:0/700215413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa86c40 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f9a040098e0 tx=0x7f9a0401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ffc913e9d8b865 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.079+0000 7f99fea00700 1 -- 172.21.15.63:0/700215413 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04027040 con 0x55b29fa82db0 2024-05-27T08:12:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.079+0000 7f99fea00700 1 -- 172.21.15.63:0/700215413 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a04032070 con 0x55b29fa82db0 2024-05-27T08:12:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.079+0000 7f99fea00700 1 -- 172.21.15.63:0/700215413 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400bda0 con 0x55b29fa82db0 2024-05-27T08:12:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.080+0000 7f9a27b04b80 1 -- 172.21.15.63:0/700215413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 msgr2=0x55b29fa86c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.080+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/700215413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa86c40 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f9a040098e0 tx=0x7f9a0401a040 comp rx=0 tx=0).stop 2024-05-27T08:12:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.080+0000 7f9a27b04b80 1 -- 172.21.15.63:0/700215413 shutdown_connections 2024-05-27T08:12:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.080+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/700215413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29fa88d30 0x55b29fa8ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.080+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/700215413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29f903e40 0x55b29fa82870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.080+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/700215413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa86c40 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.080+0000 7f9a27b04b80 1 -- 172.21.15.63:0/700215413 >> 172.21.15.63:0/700215413 conn(0x55b29f963fc0 msgr2=0x55b29fa8c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.080+0000 7f9a27b04b80 1 -- 172.21.15.63:0/700215413 shutdown_connections 2024-05-27T08:12:28.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.080+0000 7f9a27b04b80 1 -- 172.21.15.63:0/700215413 wait complete. 2024-05-27T08:12:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.081+0000 7f9a27b04b80 1 -- start start 2024-05-27T08:12:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.081+0000 7f9a27b04b80 1 Processor -- start 2024-05-27T08:12:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.082+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29f903e40 0x55b29faad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.082+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa9d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.082+0000 7f9a27b04b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa88d30 0x55b29fa9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.082+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141ac880 con 0x55b29fa82db0 2024-05-27T08:12:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.082+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141adcf0 con 0x55b29fa88d30 2024-05-27T08:12:28.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.082+0000 7f9a27b04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141acf50 con 0x55b29f903e40 2024-05-27T08:12:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.082+0000 7f9a27b04b80 10 client.0 _ll_register_callbacks cb 0x55b29f94ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.082+0000 7f9a11e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa9d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.083+0000 7f9a11e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa9d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39336/0 (socket says 172.21.15.63:39336) 2024-05-27T08:12:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.083+0000 7f9a11400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29f903e40 0x55b29faad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.083+0000 7f9a11e00700 1 -- 172.21.15.63:0/3079780855 learned_addr learned my addr 172.21.15.63:0/3079780855 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:28.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.083+0000 7f9a11400700 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa88d30 0x55b29fa9d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.083+0000 7f9a11e00700 1 -- 172.21.15.63:0/3079780855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29f903e40 msgr2=0x55b29faad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.083+0000 7f9a11e00700 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29f903e40 0x55b29faad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.083+0000 7f9a11e00700 1 -- 172.21.15.63:0/3079780855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa88d30 msgr2=0x55b29fa9d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.083+0000 7f9a11e00700 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa88d30 0x55b29fa9d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.083+0000 7f9a11e00700 1 -- 172.21.15.63:0/3079780855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b29faabb80 con 0x55b29fa82db0 2024-05-27T08:12:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.084+0000 7f9a11e00700 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa9d200 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f9a04025040 tx=0x7f9a04003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73d019e8f7a52a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.084+0000 7f99f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:28.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.084+0000 7f99f3e00700 1 -- 172.21.15.63:0/3079780855 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0403f070 con 0x55b29fa82db0 2024-05-27T08:12:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.084+0000 7f99f3e00700 1 -- 172.21.15.63:0/3079780855 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f9a04005390 con 0x55b29fa82db0 2024-05-27T08:12:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.084+0000 7f9a27b04b80 1 -- 172.21.15.63:0/3079780855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a04032070 con 0x55b29fa82db0 2024-05-27T08:12:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.084+0000 7f99f3e00700 1 -- 172.21.15.63:0/3079780855 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9a04027040 con 0x55b29fa82db0 2024-05-27T08:12:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.084+0000 7f9a27b04b80 10 client.22833 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:28.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.084+0000 7f9a27b04b80 1 -- 172.21.15.63:0/3079780855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b29fb0eac0 con 0x55b29fa82db0 2024-05-27T08:12:28.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.085+0000 7f99f3e00700 1 -- 172.21.15.63:0/3079780855 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a04032070 con 0x55b29fa82db0 2024-05-27T08:12:28.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.085+0000 7f99f3e00700 1 -- 172.21.15.63:0/3079780855 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9a04022590 con 0x55b29fa82db0 2024-05-27T08:12:28.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.085+0000 7f9a27b04b80 10 client.22833 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:28.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.085+0000 7f9a27b04b80 10 client.22833 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:12:28.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.085+0000 7f9a27b04b80 20 client.22833 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:28.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.085+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b29fb0ed40 0x55b29fb11160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:28.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.085+0000 7f9a27b04b80 4 client.22833 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:12:28.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.085+0000 7f9a27b04b80 1 -- 172.21.15.63:0/3079780855 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b29fa881e0 con 0x55b29fb0ed40 2024-05-27T08:12:28.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.086+0000 7f9a11400700 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b29fb0ed40 0x55b29fb11160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:28.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.087+0000 7f9a11400700 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b29fb0ed40 0x55b29fb11160 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23906eacb062eb24 server_cookie=bf1573f7b68e3ee2 in_seq=0 out_seq=0 2024-05-27T08:12:28.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.087+0000 7f99f3e00700 10 client.22833 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:12:28.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f99f3e00700 1 -- 172.21.15.63:0/3079780855 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b29fa881e0 con 0x55b29fb0ed40 2024-05-27T08:12:28.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f99f3e00700 10 client.22833 handle_command_reply: tid=0 2024-05-27T08:12:28.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 client.22833 shutdown 2024-05-27T08:12:28.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 -- 172.21.15.63:0/3079780855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 msgr2=0x55b29fa9d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa9d200 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f9a04025040 tx=0x7f9a04003e20 comp rx=0 tx=0).stop 2024-05-27T08:12:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 -- 172.21.15.63:0/3079780855 shutdown_connections 2024-05-27T08:12:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b29fa88d30 0x55b29fa9d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b29f903e40 0x55b29faad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b29fb0ed40 0x55b29fb11160 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 --2- 172.21.15.63:0/3079780855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b29fa82db0 0x55b29fa9d200 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 -- 172.21.15.63:0/3079780855 >> 172.21.15.63:0/3079780855 conn(0x55b29f963fc0 msgr2=0x55b29fa8cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f99f3e00700 0 client.22833 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:12:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 -- 172.21.15.63:0/3079780855 shutdown_connections 2024-05-27T08:12:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.088+0000 7f9a27b04b80 1 -- 172.21.15.63:0/3079780855 wait complete. 2024-05-27T08:12:28.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.089+0000 7f9a27b04b80 20 client.22833 trim_cache size 0 max 16384 2024-05-27T08:12:28.091 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:12:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.089+0000 7f9a19000700 1 -- 172.21.15.63:0/823240498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99f80b2a70 msgr2=0x7f99f80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.089+0000 7f9a19000700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99f80b2a70 0x7f99f80b4f80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0a6450 tx=0x7f9a0c0096d0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.089+0000 7f9a19000700 1 -- 172.21.15.63:0/823240498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a14110bc0 msgr2=0x7f9a141a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.089+0000 7f9a19000700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a14110bc0 0x7f9a141a1770 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0094f0 tx=0x7f9a0c00d3b0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.090+0000 7f9a19000700 1 -- 172.21.15.63:0/823240498 shutdown_connections 2024-05-27T08:12:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.090+0000 7f9a19000700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f99f80b2a70 0x7f99f80b4f80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.090+0000 7f9a19000700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a14110bc0 0x7f9a141a1770 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.090+0000 7f9a19000700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a14076c00 0x7f9a1419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.090+0000 7f9a19000700 1 --2- 172.21.15.63:0/823240498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a14077550 0x7f9a1419d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:28.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.090+0000 7f9a19000700 1 -- 172.21.15.63:0/823240498 >> 172.21.15.63:0/823240498 conn(0x7f9a140ffea0 msgr2=0x7f9a14100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:28.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.090+0000 7f9a19000700 1 -- 172.21.15.63:0/823240498 shutdown_connections 2024-05-27T08:12:28.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:28.090+0000 7f9a19000700 1 -- 172.21.15.63:0/823240498 wait complete. 2024-05-27T08:12:28.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:27 smithi086 ceph-mon[34687]: pgmap v1234: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.1 MiB/s wr, 16 op/s 2024-05-27T08:12:28.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:27 smithi149 ceph-mon[35408]: pgmap v1234: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.1 MiB/s wr, 16 op/s 2024-05-27T08:12:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.133+0000 7f7da9000640 1 -- 172.21.15.63:0/2113455712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7da4102820 msgr2=0x7f7da4102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.133+0000 7f7da9000640 1 --2- 172.21.15.63:0/2113455712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7da4102820 0x7f7da4102c30 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009ca0 tx=0x7f7d9402a8c0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.133+0000 7f7da9000640 1 -- 172.21.15.63:0/2113455712 shutdown_connections 2024-05-27T08:12:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.133+0000 7f7da9000640 1 --2- 172.21.15.63:0/2113455712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7da41103c0 0x7f7da41127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.133+0000 7f7da9000640 1 --2- 172.21.15.63:0/2113455712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da410da60 0x7f7da410fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.133+0000 7f7da9000640 1 --2- 172.21.15.63:0/2113455712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7da4102820 0x7f7da4102c30 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.133+0000 7f7da9000640 1 -- 172.21.15.63:0/2113455712 >> 172.21.15.63:0/2113455712 conn(0x7f7da40fe210 msgr2=0x7f7da4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.133+0000 7f7da9000640 1 -- 172.21.15.63:0/2113455712 shutdown_connections 2024-05-27T08:12:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.133+0000 7f7da9000640 1 -- 172.21.15.63:0/2113455712 wait complete. 2024-05-27T08:12:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.134+0000 7f7da9000640 1 Processor -- start 2024-05-27T08:12:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.134+0000 7f7da9000640 1 -- start start 2024-05-27T08:12:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.135+0000 7f7da9000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7da4102820 0x7f7da419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.135+0000 7f7da9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da410da60 0x7f7da419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.135+0000 7f7da9000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7da41103c0 0x7f7da41a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.135+0000 7f7da9000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4115ce0 con 0x7f7da4102820 2024-05-27T08:12:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.135+0000 7f7da9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4115b60 con 0x7f7da410da60 2024-05-27T08:12:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.135+0000 7f7da9000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4115e60 con 0x7f7da41103c0 2024-05-27T08:12:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.135+0000 7f7da3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7da4102820 0x7f7da419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.135+0000 7f7da3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7da4102820 0x7f7da419c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44214/0 (socket says 172.21.15.63:44214) 2024-05-27T08:12:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.135+0000 7f7da3400640 1 -- 172.21.15.63:0/3601461979 learned_addr learned my addr 172.21.15.63:0/3601461979 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.136+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da410da60 0x7f7da419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.136+0000 7f7da3e00640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7da41103c0 0x7f7da41a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.136+0000 7f7da3400640 1 -- 172.21.15.63:0/3601461979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7da41103c0 msgr2=0x7f7da41a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.136+0000 7f7da3400640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7da41103c0 0x7f7da41a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.136+0000 7f7da3400640 1 -- 172.21.15.63:0/3601461979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da410da60 msgr2=0x7f7da419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.136+0000 7f7da3400640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da410da60 0x7f7da419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.136+0000 7f7da3400640 1 -- 172.21.15.63:0/3601461979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da41a1600 con 0x7f7da4102820 2024-05-27T08:12:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.136+0000 7f7da3400640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7da4102820 0x7f7da419c7a0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f7d94006010 tx=0x7f7d94003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.137+0000 7f7da0c00640 1 -- 172.21.15.63:0/3601461979 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94003d60 con 0x7f7da4102820 2024-05-27T08:12:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.137+0000 7f7da9000640 1 -- 172.21.15.63:0/3601461979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da41a1890 con 0x7f7da4102820 2024-05-27T08:12:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.137+0000 7f7da0c00640 1 -- 172.21.15.63:0/3601461979 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d940044d0 con 0x7f7da4102820 2024-05-27T08:12:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.137+0000 7f7da0c00640 1 -- 172.21.15.63:0/3601461979 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94030c80 con 0x7f7da4102820 2024-05-27T08:12:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.137+0000 7f7da9000640 1 -- 172.21.15.63:0/3601461979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41a1de0 con 0x7f7da4102820 2024-05-27T08:12:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.141+0000 7f7db15b6740 1 Processor -- start 2024-05-27T08:12:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.141+0000 7f7db15b6740 1 -- start start 2024-05-27T08:12:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.142+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x5561705fea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.142+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561702332f0 0x5561705fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.142+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 0x5561705ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.142+0000 7f7db15b6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55617056fb90 con 0x55617025c1b0 2024-05-27T08:12:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.142+0000 7f7db15b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55617055f730 con 0x55617023b4d0 2024-05-27T08:12:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.142+0000 7f7db15b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556170563770 con 0x5561702332f0 2024-05-27T08:12:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.142+0000 7f7da0c00640 1 -- 172.21.15.63:0/3601461979 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7d94004040 con 0x7f7da4102820 2024-05-27T08:12:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.142+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x5561705fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.142+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x5561705fea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44220/0 (socket says 172.21.15.63:44220) 2024-05-27T08:12:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.142+0000 7f7da3e00640 1 -- 172.21.15.63:0/1449074902 learned_addr learned my addr 172.21.15.63:0/1449074902 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.143+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/1449074902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 0x5561705ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.143+0000 7f7da3400640 1 --2- 172.21.15.63:0/1449074902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561702332f0 0x5561705fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.143+0000 7f7da3e00640 1 -- 172.21.15.63:0/1449074902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561702332f0 msgr2=0x5561705fefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.143+0000 7f7da3e00640 1 --2- 172.21.15.63:0/1449074902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561702332f0 0x5561705fefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.143+0000 7f7da3e00640 1 -- 172.21.15.63:0/1449074902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 msgr2=0x5561705ff4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.143+0000 7f7da3e00640 1 --2- 172.21.15.63:0/1449074902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 0x5561705ff4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.143+0000 7f7da3e00640 1 -- 172.21.15.63:0/1449074902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561704f88b0 con 0x55617025c1b0 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.143+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/1449074902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 0x5561705ff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.143+0000 7f7da3e00640 1 --2- 172.21.15.63:0/1449074902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x5561705fea60 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00bba0 tx=0x7f7d9c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f08d513fd983bdc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.144+0000 7f7d98c00640 1 -- 172.21.15.63:0/1449074902 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c015070 con 0x55617025c1b0 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.144+0000 7f7d98c00640 1 -- 172.21.15.63:0/1449074902 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d9c0054b0 con 0x55617025c1b0 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.144+0000 7f7d98c00640 1 -- 172.21.15.63:0/1449074902 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c011440 con 0x55617025c1b0 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.144+0000 7f7da0c00640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7d880b2950 0x7f7d880b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7da0c00640 1 -- 172.21.15.63:0/3601461979 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f7d94103e90 con 0x7f7da4102820 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7db15b6740 1 -- 172.21.15.63:0/1449074902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 msgr2=0x5561705fea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7db15b6740 1 --2- 172.21.15.63:0/1449074902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x5561705fea60 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00bba0 tx=0x7f7d9c00f730 comp rx=0 tx=0).stop 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7da3400640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7d880b2950 0x7f7d880b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7db15b6740 1 -- 172.21.15.63:0/1449074902 shutdown_connections 2024-05-27T08:12:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7db15b6740 1 --2- 172.21.15.63:0/1449074902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 0x5561705ff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7db15b6740 1 --2- 172.21.15.63:0/1449074902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561702332f0 0x5561705fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7db15b6740 1 --2- 172.21.15.63:0/1449074902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x5561705fea60 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7db15b6740 1 -- 172.21.15.63:0/1449074902 >> 172.21.15.63:0/1449074902 conn(0x556170250300 msgr2=0x5561705fb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7db15b6740 1 -- 172.21.15.63:0/1449074902 shutdown_connections 2024-05-27T08:12:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.145+0000 7f7db15b6740 1 -- 172.21.15.63:0/1449074902 wait complete. 2024-05-27T08:12:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.146+0000 7f7da3400640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7d880b2950 0x7f7d880b4e60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7d941013e0 tx=0x7f7d940095c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.146+0000 7f7db15b6740 1 -- start start 2024-05-27T08:12:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.146+0000 7f7db15b6740 1 Processor -- start 2024-05-27T08:12:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561702332f0 0x556170608ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55617023b4d0 0x5561706093e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x556170614ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7db15b6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556170245790 con 0x55617025c1b0 2024-05-27T08:12:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7db15b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556170507bc0 con 0x5561702332f0 2024-05-27T08:12:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7db15b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55617060bb90 con 0x55617023b4d0 2024-05-27T08:12:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7db15b6740 10 client.0 _ll_register_callbacks cb 0x5561704e5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x556170614ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x556170614ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44224/0 (socket says 172.21.15.63:44224) 2024-05-27T08:12:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7da3e00640 1 -- 172.21.15.63:0/1978281144 learned_addr learned my addr 172.21.15.63:0/1978281144 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55617023b4d0 0x5561706093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.147+0000 7f7da3e00640 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561702332f0 0x556170608ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7da3e00640 1 -- 172.21.15.63:0/1978281144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55617023b4d0 msgr2=0x5561706093e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7da3e00640 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55617023b4d0 0x5561706093e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7da3e00640 1 -- 172.21.15.63:0/1978281144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561702332f0 msgr2=0x556170608ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7da3e00640 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561702332f0 0x556170608ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7da3e00640 1 -- 172.21.15.63:0/1978281144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556170274210 con 0x55617025c1b0 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55617023b4d0 0x5561706093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7da3e00640 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x556170614ec0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c005780 tx=0x7f7d9c004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffae3a9e9060275d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7d91600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7d91600640 1 -- 172.21.15.63:0/1978281144 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c01d020 con 0x55617025c1b0 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7db15b6740 1 -- 172.21.15.63:0/1978281144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d9c0054b0 con 0x55617025c1b0 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.148+0000 7f7d91600640 1 -- 172.21.15.63:0/1978281144 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f7d9c004c90 con 0x55617025c1b0 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.149+0000 7f7d91600640 1 -- 172.21.15.63:0/1978281144 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c015070 con 0x55617025c1b0 2024-05-27T08:12:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.149+0000 7f7db15b6740 10 client.22845 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.149+0000 7f7db15b6740 1 -- 172.21.15.63:0/1978281144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556170340620 con 0x55617025c1b0 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.149+0000 7f7d91600640 1 -- 172.21.15.63:0/1978281144 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d9c0054b0 con 0x55617025c1b0 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.149+0000 7f7d91600640 1 -- 172.21.15.63:0/1978281144 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7d9c011a30 con 0x55617025c1b0 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.150+0000 7f7db15b6740 10 client.22845 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.150+0000 7f7db15b6740 10 client.22845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.150+0000 7f7db15b6740 20 client.22845 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.150+0000 7f7db15b6740 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55617021ad80 0x55617061ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.150+0000 7f7db15b6740 4 client.22845 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.150+0000 7f7db15b6740 1 -- 172.21.15.63:0/1978281144 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556170571ae0 con 0x55617021ad80 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.150+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55617021ad80 0x55617061ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.152+0000 7f7d91600640 10 client.22845 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.152+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55617021ad80 0x55617061ae50 crc :-1 s=READY pgs=690 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb05cb3186a9ddd6 server_cookie=12fa61aa1c5a8e42 in_seq=0 out_seq=0 2024-05-27T08:12:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.153+0000 7f7d91600640 1 -- 172.21.15.63:0/1978281144 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556170571ae0 con 0x55617021ad80 2024-05-27T08:12:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.153+0000 7f7d91600640 10 client.22845 handle_command_reply: tid=0 2024-05-27T08:12:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.153+0000 7f7db15b6740 1 client.22845 shutdown 2024-05-27T08:12:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 1 -- 172.21.15.63:0/1978281144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 msgr2=0x556170614ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x556170614ec0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c005780 tx=0x7f7d9c004bb0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 1 -- 172.21.15.63:0/1978281144 shutdown_connections 2024-05-27T08:12:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55617021ad80 0x55617061ae50 crc :-1 s=READY pgs=690 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617025c1b0 0x556170614ec0 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55617023b4d0 0x5561706093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 1 --2- 172.21.15.63:0/1978281144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561702332f0 0x556170608ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 1 -- 172.21.15.63:0/1978281144 >> 172.21.15.63:0/1978281144 conn(0x556170250300 msgr2=0x5561705fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7d91600640 0 client.22845 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 1 -- 172.21.15.63:0/1978281144 shutdown_connections 2024-05-27T08:12:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 1 -- 172.21.15.63:0/1978281144 wait complete. 2024-05-27T08:12:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.154+0000 7f7db15b6740 20 client.22845 trim_cache size 0 max 16384 2024-05-27T08:12:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.173+0000 7f7db15b6740 1 Processor -- start 2024-05-27T08:12:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.173+0000 7f7db15b6740 1 -- start start 2024-05-27T08:12:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.174+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561702332f0 0x5561706044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.174+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617023b4d0 0x5561705fef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.174+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617025c1b0 0x556170614c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.174+0000 7f7db15b6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41b5ae0 con 0x55617023b4d0 2024-05-27T08:12:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.174+0000 7f7db15b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41b33b0 con 0x55617025c1b0 2024-05-27T08:12:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.174+0000 7f7db15b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41b1220 con 0x5561702332f0 2024-05-27T08:12:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.174+0000 7f7d9aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617023b4d0 0x5561705fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.174+0000 7f7d9aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617023b4d0 0x5561705fef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44234/0 (socket says 172.21.15.63:44234) 2024-05-27T08:12:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.174+0000 7f7d9aa00640 1 -- 172.21.15.63:0/4161099518 learned_addr learned my addr 172.21.15.63:0/4161099518 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.175+0000 7f7da3e00640 1 --2- 172.21.15.63:0/4161099518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561702332f0 0x5561706044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.175+0000 7f7d9aa00640 1 -- 172.21.15.63:0/4161099518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561702332f0 msgr2=0x5561706044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.175+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/4161099518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561702332f0 0x5561706044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.175+0000 7f7d9aa00640 1 -- 172.21.15.63:0/4161099518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617025c1b0 msgr2=0x556170614c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.175+0000 7f7da3e00640 1 --2- 172.21.15.63:0/4161099518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617025c1b0 0x556170614c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.175+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/4161099518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617025c1b0 0x556170614c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.175+0000 7f7d9aa00640 1 -- 172.21.15.63:0/4161099518 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617060b810 con 0x55617023b4d0 2024-05-27T08:12:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.175+0000 7f7da3e00640 1 --2- 172.21.15.63:0/4161099518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617025c1b0 0x556170614c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:12:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.175+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/4161099518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617023b4d0 0x5561705fef00 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00bfd0 tx=0x7f7d8c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c6992ac3cba881 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.176+0000 7f7d93e00640 1 -- 172.21.15.63:0/4161099518 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c01f020 con 0x55617023b4d0 2024-05-27T08:12:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.176+0000 7f7d93e00640 1 -- 172.21.15.63:0/4161099518 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d8c0042f0 con 0x55617023b4d0 2024-05-27T08:12:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.176+0000 7f7d93e00640 1 -- 172.21.15.63:0/4161099518 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c01b5b0 con 0x55617023b4d0 2024-05-27T08:12:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.177+0000 7f7db15b6740 1 -- 172.21.15.63:0/4161099518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617023b4d0 msgr2=0x5561705fef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.177+0000 7f7db15b6740 1 --2- 172.21.15.63:0/4161099518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617023b4d0 0x5561705fef00 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00bfd0 tx=0x7f7d8c005c70 comp rx=0 tx=0).stop 2024-05-27T08:12:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.177+0000 7f7db15b6740 1 -- 172.21.15.63:0/4161099518 shutdown_connections 2024-05-27T08:12:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.177+0000 7f7db15b6740 1 --2- 172.21.15.63:0/4161099518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617025c1b0 0x556170614c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.177+0000 7f7db15b6740 1 --2- 172.21.15.63:0/4161099518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55617023b4d0 0x5561705fef00 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.177+0000 7f7db15b6740 1 --2- 172.21.15.63:0/4161099518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5561702332f0 0x5561706044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.177+0000 7f7db15b6740 1 -- 172.21.15.63:0/4161099518 >> 172.21.15.63:0/4161099518 conn(0x556170250300 msgr2=0x5561705fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.177+0000 7f7db15b6740 1 -- 172.21.15.63:0/4161099518 shutdown_connections 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.177+0000 7f7db15b6740 1 -- 172.21.15.63:0/4161099518 wait complete. 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.177+0000 7f7db15b6740 1 -- start start 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7db15b6740 1 Processor -- start 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561702332f0 0x556170611450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 0x55617060bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7db15b6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55617025c1b0 0x55617060c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7db15b6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41b1870 con 0x5561702332f0 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7db15b6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41b2ce0 con 0x55617023b4d0 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7db15b6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41b1f40 con 0x55617025c1b0 2024-05-27T08:12:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7db15b6740 10 client.0 _ll_register_callbacks cb 0x7f7d9c00b9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561702332f0 0x556170611450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561702332f0 0x556170611450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44236/0 (socket says 172.21.15.63:44236) 2024-05-27T08:12:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.178+0000 7f7da3e00640 1 -- 172.21.15.63:0/4212431294 learned_addr learned my addr 172.21.15.63:0/4212431294 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.179+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 0x55617060bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.179+0000 7f7da3e00640 1 -- 172.21.15.63:0/4212431294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55617025c1b0 msgr2=0x55617060c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.179+0000 7f7da3e00640 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55617025c1b0 0x55617060c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.179+0000 7f7da3e00640 1 -- 172.21.15.63:0/4212431294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 msgr2=0x55617060bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.179+0000 7f7da3e00640 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 0x55617060bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.179+0000 7f7da3e00640 1 -- 172.21.15.63:0/4212431294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561705f9610 con 0x5561702332f0 2024-05-27T08:12:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.179+0000 7f7da3e00640 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561702332f0 0x556170611450 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00bcd0 tx=0x7f7d9c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e4b38407fa2f47 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.179+0000 7f7d91600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.180+0000 7f7d91600640 1 -- 172.21.15.63:0/4212431294 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c038020 con 0x5561702332f0 2024-05-27T08:12:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.180+0000 7f7d91600640 1 -- 172.21.15.63:0/4212431294 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f7d9c011530 con 0x5561702332f0 2024-05-27T08:12:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.180+0000 7f7db15b6740 1 -- 172.21.15.63:0/4212431294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d8c0042f0 con 0x5561702332f0 2024-05-27T08:12:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.180+0000 7f7d91600640 1 -- 172.21.15.63:0/4212431294 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c01d040 con 0x5561702332f0 2024-05-27T08:12:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.180+0000 7f7db15b6740 10 client.22851 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.180+0000 7f7db15b6740 1 -- 172.21.15.63:0/4212431294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556170681f90 con 0x5561702332f0 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.180+0000 7f7d91600640 1 -- 172.21.15.63:0/4212431294 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d8c0042f0 con 0x5561702332f0 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.181+0000 7f7d91600640 1 -- 172.21.15.63:0/4212431294 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7d9c004c20 con 0x5561702332f0 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.181+0000 7f7db15b6740 10 client.22851 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.181+0000 7f7db15b6740 10 client.22851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.181+0000 7f7db15b6740 20 client.22851 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.181+0000 7f7db15b6740 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561701f01c0 0x556170684780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.181+0000 7f7db15b6740 4 client.22851 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.181+0000 7f7db15b6740 1 -- 172.21.15.63:0/4212431294 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55617056b2c0 con 0x5561701f01c0 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.182+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561701f01c0 0x556170684780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.183+0000 7f7d91600640 10 client.22851 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.183+0000 7f7d9aa00640 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561701f01c0 0x556170684780 crc :-1 s=READY pgs=691 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea6b8162c651b19 server_cookie=df13dc57a5822ec0 in_seq=0 out_seq=0 2024-05-27T08:12:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7d91600640 1 -- 172.21.15.63:0/4212431294 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55617056b2c0 con 0x5561701f01c0 2024-05-27T08:12:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7d91600640 10 client.22851 handle_command_reply: tid=0 2024-05-27T08:12:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7db15b6740 1 client.22851 shutdown 2024-05-27T08:12:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7db15b6740 1 -- 172.21.15.63:0/4212431294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561702332f0 msgr2=0x556170611450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7db15b6740 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561702332f0 0x556170611450 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00bcd0 tx=0x7f7d9c00bf10 comp rx=0 tx=0).stop 2024-05-27T08:12:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7db15b6740 1 -- 172.21.15.63:0/4212431294 shutdown_connections 2024-05-27T08:12:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7db15b6740 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5561701f01c0 0x556170684780 crc :-1 s=READY pgs=691 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7db15b6740 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55617025c1b0 0x55617060c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7db15b6740 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55617023b4d0 0x55617060bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7db15b6740 1 --2- 172.21.15.63:0/4212431294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5561702332f0 0x556170611450 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7db15b6740 1 -- 172.21.15.63:0/4212431294 >> 172.21.15.63:0/4212431294 conn(0x556170250300 msgr2=0x55617060ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.224+0000 7f7d91600640 0 client.22851 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.225+0000 7f7db15b6740 1 -- 172.21.15.63:0/4212431294 shutdown_connections 2024-05-27T08:12:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.225+0000 7f7db15b6740 1 -- 172.21.15.63:0/4212431294 wait complete. 2024-05-27T08:12:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.225+0000 7f7db15b6740 20 client.22851 trim_cache size 0 max 16384 2024-05-27T08:12:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:12:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:12:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:12:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.226+0000 7f7da9000640 1 -- 172.21.15.63:0/3601461979 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7d880b2950 msgr2=0x7f7d880b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.226+0000 7f7da9000640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7d880b2950 0x7f7d880b4e60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7d941013e0 tx=0x7f7d940095c0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.226+0000 7f7da9000640 1 -- 172.21.15.63:0/3601461979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7da4102820 msgr2=0x7f7da419c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.226+0000 7f7da9000640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7da4102820 0x7f7da419c7a0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f7d94006010 tx=0x7f7d94003970 comp rx=0 tx=0).stop 2024-05-27T08:12:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.227+0000 7f7da9000640 1 -- 172.21.15.63:0/3601461979 shutdown_connections 2024-05-27T08:12:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.227+0000 7f7da9000640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f7d880b2950 0x7f7d880b4e60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.227+0000 7f7da9000640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7da41103c0 0x7f7da41a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.227+0000 7f7da9000640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da410da60 0x7f7da419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.227+0000 7f7da9000640 1 --2- 172.21.15.63:0/3601461979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7da4102820 0x7f7da419c7a0 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.227+0000 7f7da9000640 1 -- 172.21.15.63:0/3601461979 >> 172.21.15.63:0/3601461979 conn(0x7f7da40fe210 msgr2=0x7f7da4111200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.227+0000 7f7da9000640 1 -- 172.21.15.63:0/3601461979 shutdown_connections 2024-05-27T08:12:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:29.227+0000 7f7da9000640 1 -- 172.21.15.63:0/3601461979 wait complete. 2024-05-27T08:12:29.276 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:12:29.762 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:12:30.094 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:29 smithi063 ceph-mon[31613]: pgmap v1235: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 7 op/s 2024-05-27T08:12:30.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:29 smithi086 ceph-mon[34687]: pgmap v1235: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 7 op/s 2024-05-27T08:12:30.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:29 smithi149 ceph-mon[35408]: pgmap v1235: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 7 op/s 2024-05-27T08:12:32.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:31 smithi086 ceph-mon[34687]: pgmap v1236: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 7 op/s 2024-05-27T08:12:32.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:31 smithi149 ceph-mon[35408]: pgmap v1236: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 7 op/s 2024-05-27T08:12:32.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:31 smithi063 ceph-mon[31613]: pgmap v1236: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.4 MiB/s wr, 7 op/s 2024-05-27T08:12:32.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.541+0000 7f60b5e00700 1 -- 172.21.15.63:0/2645970916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b010c5b0 msgr2=0x7f60b010c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:32.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.541+0000 7f60b5e00700 1 --2- 172.21.15.63:0/2645970916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b010c5b0 0x7f60b010c9c0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009990 tx=0x7f60a0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.541+0000 7f60b5e00700 1 -- 172.21.15.63:0/2645970916 shutdown_connections 2024-05-27T08:12:32.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.541+0000 7f60b5e00700 1 --2- 172.21.15.63:0/2645970916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60b0106ec0 0x7f60b0113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.541+0000 7f60b5e00700 1 --2- 172.21.15.63:0/2645970916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b010c5b0 0x7f60b010c9c0 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.541+0000 7f60b5e00700 1 --2- 172.21.15.63:0/2645970916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b0106570 0x7f60b0106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.541+0000 7f60b5e00700 1 -- 172.21.15.63:0/2645970916 >> 172.21.15.63:0/2645970916 conn(0x7f60b0102070 msgr2=0x7f60b01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:32.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.541+0000 7f60b5e00700 1 -- 172.21.15.63:0/2645970916 shutdown_connections 2024-05-27T08:12:32.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.541+0000 7f60b5e00700 1 -- 172.21.15.63:0/2645970916 wait complete. 2024-05-27T08:12:32.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.542+0000 7f60b5e00700 1 Processor -- start 2024-05-27T08:12:32.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.543+0000 7f60b5e00700 1 -- start start 2024-05-27T08:12:32.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.543+0000 7f60b5e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60b0106570 0x7f60b019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.543+0000 7f60b5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b0106ec0 0x7f60b019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.543+0000 7f60b5e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b010c5b0 0x7f60b01a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.543+0000 7f60b5e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0116570 con 0x7f60b0106ec0 2024-05-27T08:12:32.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.543+0000 7f60b5e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01163f0 con 0x7f60b010c5b0 2024-05-27T08:12:32.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.543+0000 7f60b5e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01166f0 con 0x7f60b0106570 2024-05-27T08:12:32.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.544+0000 7f60ae000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b0106ec0 0x7f60b019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.544+0000 7f60ae000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b0106ec0 0x7f60b019d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44254/0 (socket says 172.21.15.63:44254) 2024-05-27T08:12:32.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.544+0000 7f60ae000700 1 -- 172.21.15.63:0/2558101113 learned_addr learned my addr 172.21.15.63:0/2558101113 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:32.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.544+0000 7f60aea00700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60b0106570 0x7f60b019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.544+0000 7f60af400700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b010c5b0 0x7f60b01a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.545+0000 7f60ae000700 1 -- 172.21.15.63:0/2558101113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60b0106570 msgr2=0x7f60b019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:32.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.545+0000 7f60ae000700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60b0106570 0x7f60b019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.545+0000 7f60ae000700 1 -- 172.21.15.63:0/2558101113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b010c5b0 msgr2=0x7f60b01a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:32.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.545+0000 7f60ae000700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b010c5b0 0x7f60b01a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.545+0000 7f60ae000700 1 -- 172.21.15.63:0/2558101113 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b01a1e00 con 0x7f60b0106ec0 2024-05-27T08:12:32.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.545+0000 7f60af400700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b010c5b0 0x7f60b01a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:32.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.545+0000 7f60ae000700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b0106ec0 0x7f60b019d520 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f609c00d950 tx=0x7f609c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:32.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.546+0000 7f60b4a00700 1 -- 172.21.15.63:0/2558101113 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c017070 con 0x7f60b0106ec0 2024-05-27T08:12:32.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.546+0000 7f60b5e00700 1 -- 172.21.15.63:0/2558101113 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b01a20f0 con 0x7f60b0106ec0 2024-05-27T08:12:32.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.546+0000 7f60b4a00700 1 -- 172.21.15.63:0/2558101113 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f609c009dc0 con 0x7f60b0106ec0 2024-05-27T08:12:32.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.546+0000 7f60b4a00700 1 -- 172.21.15.63:0/2558101113 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c00f3c0 con 0x7f60b0106ec0 2024-05-27T08:12:32.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.546+0000 7f60b5e00700 1 -- 172.21.15.63:0/2558101113 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b01a9a10 con 0x7f60b0106ec0 2024-05-27T08:12:32.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.551+0000 7f60b4a00700 1 -- 172.21.15.63:0/2558101113 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f609c01a070 con 0x7f60b0106ec0 2024-05-27T08:12:32.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.552+0000 7f60c48e2b80 1 Processor -- start 2024-05-27T08:12:32.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.552+0000 7f60c48e2b80 1 -- start start 2024-05-27T08:12:32.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.553+0000 7f60b4a00700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60940b29f0 0x7f60940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.553+0000 7f60b4a00700 1 -- 172.21.15.63:0/2558101113 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f609c0e2320 con 0x7f60b0106ec0 2024-05-27T08:12:32.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.553+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7c92e40 0x5566c7e177f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.553+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e17d30 0x5566c7e11870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.553+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7e11db0 0x5566c7e121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.553+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5566c7d3fe40 con 0x5566c7c92e40 2024-05-27T08:12:32.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.553+0000 7f60af400700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60940b29f0 0x7f60940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.553+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5566c7d2f9e0 con 0x5566c7e11db0 2024-05-27T08:12:32.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.553+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5566c7d33a20 con 0x5566c7e17d30 2024-05-27T08:12:32.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.553+0000 7f60aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7c92e40 0x5566c7e177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.554+0000 7f60aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7c92e40 0x5566c7e177f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44258/0 (socket says 172.21.15.63:44258) 2024-05-27T08:12:32.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.554+0000 7f60aea00700 1 -- 172.21.15.63:0/1117235679 learned_addr learned my addr 172.21.15.63:0/1117235679 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:32.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.554+0000 7f60ae000700 1 --2- 172.21.15.63:0/1117235679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e17d30 0x5566c7e11870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.554+0000 7f60af400700 1 --2- 172.21.15.63:0/1117235679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7e11db0 0x5566c7e121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.554+0000 7f60aea00700 1 -- 172.21.15.63:0/1117235679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e17d30 msgr2=0x5566c7e11870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:32.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.554+0000 7f60aea00700 1 --2- 172.21.15.63:0/1117235679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e17d30 0x5566c7e11870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.554+0000 7f60aea00700 1 -- 172.21.15.63:0/1117235679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7e11db0 msgr2=0x5566c7e121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:32.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.554+0000 7f60aea00700 1 --2- 172.21.15.63:0/1117235679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7e11db0 0x5566c7e121c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.554+0000 7f60aea00700 1 -- 172.21.15.63:0/1117235679 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c7cf8420 con 0x5566c7c92e40 2024-05-27T08:12:32.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.555+0000 7f60af400700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60940b29f0 0x7f60940b4f00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f60a80060b0 tx=0x7f60a80093b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:32.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.555+0000 7f60aea00700 1 --2- 172.21.15.63:0/1117235679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7c92e40 0x5566c7e177f0 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f60b01079e0 tx=0x7f60a0009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e9ee8634b0e142 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:32.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.555+0000 7f60a4c00700 1 -- 172.21.15.63:0/1117235679 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0027070 con 0x5566c7c92e40 2024-05-27T08:12:32.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.555+0000 7f60a4c00700 1 -- 172.21.15.63:0/1117235679 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60a00037c0 con 0x5566c7c92e40 2024-05-27T08:12:32.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.555+0000 7f60a4c00700 1 -- 172.21.15.63:0/1117235679 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a000ba10 con 0x5566c7c92e40 2024-05-27T08:12:32.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.556+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1117235679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7c92e40 msgr2=0x5566c7e177f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:32.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.556+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/1117235679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7c92e40 0x5566c7e177f0 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f60b01079e0 tx=0x7f60a0009700 comp rx=0 tx=0).stop 2024-05-27T08:12:32.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.557+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1117235679 shutdown_connections 2024-05-27T08:12:32.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.557+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/1117235679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7e11db0 0x5566c7e121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.557+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/1117235679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7c92e40 0x5566c7e177f0 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.557+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/1117235679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e17d30 0x5566c7e11870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.557+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1117235679 >> 172.21.15.63:0/1117235679 conn(0x5566c7cf2fc0 msgr2=0x5566c7e0c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:32.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.557+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1117235679 shutdown_connections 2024-05-27T08:12:32.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.557+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1117235679 wait complete. 2024-05-27T08:12:32.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.558+0000 7f60c48e2b80 1 -- start start 2024-05-27T08:12:32.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.558+0000 7f60c48e2b80 1 Processor -- start 2024-05-27T08:12:32.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.558+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 0x5566c7e1b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.558+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 0x5566c7e1b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.558+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.558+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5566c7c35fa0 con 0x5566c7e17d30 2024-05-27T08:12:32.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.558+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5566c7cd0760 con 0x5566c7c92e40 2024-05-27T08:12:32.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.558+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5566c7e1e7a0 con 0x5566c7e11db0 2024-05-27T08:12:32.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.559+0000 7f60c48e2b80 10 client.0 _ll_register_callbacks cb 0x5566c7cd9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:32.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.559+0000 7f60af400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.559+0000 7f60af400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44262/0 (socket says 172.21.15.63:44262) 2024-05-27T08:12:32.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.559+0000 7f60af400700 1 -- 172.21.15.63:0/1904653027 learned_addr learned my addr 172.21.15.63:0/1904653027 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:32.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.559+0000 7f60aea00700 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 0x5566c7e1b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.559+0000 7f60ae000700 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 0x5566c7e1b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.560+0000 7f60af400700 1 -- 172.21.15.63:0/1904653027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 msgr2=0x5566c7e1b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:32.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.560+0000 7f60af400700 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 0x5566c7e1b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.560+0000 7f60af400700 1 -- 172.21.15.63:0/1904653027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 msgr2=0x5566c7e1b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:32.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.560+0000 7f60af400700 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 0x5566c7e1b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.560+0000 7f60af400700 1 -- 172.21.15.63:0/1904653027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c7c7f8d0 con 0x5566c7e17d30 2024-05-27T08:12:32.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.560+0000 7f60af400700 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bf00 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f60a8016cf0 tx=0x7f60a8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13797494c7c447b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:32.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.560+0000 7f6099600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:32.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.560+0000 7f6099600700 1 -- 172.21.15.63:0/1904653027 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8029060 con 0x5566c7e17d30 2024-05-27T08:12:32.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.560+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1904653027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a00037c0 con 0x5566c7e17d30 2024-05-27T08:12:32.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.560+0000 7f6099600700 1 -- 172.21.15.63:0/1904653027 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f60a801cad0 con 0x5566c7e17d30 2024-05-27T08:12:32.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.561+0000 7f6099600700 1 -- 172.21.15.63:0/1904653027 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f60a8030050 con 0x5566c7e17d30 2024-05-27T08:12:32.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.561+0000 7f60c48e2b80 10 client.22863 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:32.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.561+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1904653027 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5566c7d00060 con 0x5566c7e17d30 2024-05-27T08:12:32.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.561+0000 7f6099600700 1 -- 172.21.15.63:0/1904653027 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60a00037c0 con 0x5566c7e17d30 2024-05-27T08:12:32.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.562+0000 7f6099600700 1 -- 172.21.15.63:0/1904653027 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f60a8018070 con 0x5566c7e17d30 2024-05-27T08:12:32.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.562+0000 7f60c48e2b80 10 client.22863 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:32.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.562+0000 7f60c48e2b80 10 client.22863 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:12:32.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.562+0000 7f60c48e2b80 20 client.22863 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:32.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.562+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5566c7e2b610 0x5566c7e2da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.562+0000 7f60c48e2b80 4 client.22863 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:32.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.562+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1904653027 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566c7d41d90 con 0x5566c7e2b610 2024-05-27T08:12:32.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.563+0000 7f60aea00700 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5566c7e2b610 0x5566c7e2da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.564+0000 7f6099600700 10 client.22863 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:12:32.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.564+0000 7f60aea00700 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5566c7e2b610 0x5566c7e2da30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30ddab3b0eb3657a server_cookie=bd77c3720da4e623 in_seq=0 out_seq=0 2024-05-27T08:12:32.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f6099600700 1 -- 172.21.15.63:0/1904653027 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5566c7d41d90 con 0x5566c7e2b610 2024-05-27T08:12:32.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f6099600700 10 client.22863 handle_command_reply: tid=0 2024-05-27T08:12:32.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f60c48e2b80 1 client.22863 shutdown 2024-05-27T08:12:32.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1904653027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 msgr2=0x5566c7e1bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:32.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bf00 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f60a8016cf0 tx=0x7f60a8009040 comp rx=0 tx=0).stop 2024-05-27T08:12:32.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1904653027 shutdown_connections 2024-05-27T08:12:32.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5566c7e2b610 0x5566c7e2da30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bf00 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 0x5566c7e1b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/1904653027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 0x5566c7e1b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f6099600700 0 client.22863 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:12:32.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1904653027 >> 172.21.15.63:0/1904653027 conn(0x5566c7cf2fc0 msgr2=0x5566c7e0c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:32.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.566+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1904653027 shutdown_connections 2024-05-27T08:12:32.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.567+0000 7f60c48e2b80 1 -- 172.21.15.63:0/1904653027 wait complete. 2024-05-27T08:12:32.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.567+0000 7f60c48e2b80 20 client.22863 trim_cache size 0 max 16384 2024-05-27T08:12:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.588+0000 7f60c48e2b80 1 Processor -- start 2024-05-27T08:12:32.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.588+0000 7f60c48e2b80 1 -- start start 2024-05-27T08:12:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.588+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 0x5566c7e11870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.588+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 0x5566c7e27800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.588+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.588+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b4ff0 con 0x5566c7e17d30 2024-05-27T08:12:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.588+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b2880 con 0x5566c7c92e40 2024-05-27T08:12:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.588+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b06f0 con 0x5566c7e11db0 2024-05-27T08:12:32.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60af400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60af400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44268/0 (socket says 172.21.15.63:44268) 2024-05-27T08:12:32.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60af400700 1 -- 172.21.15.63:0/3869766583 learned_addr learned my addr 172.21.15.63:0/3869766583 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:32.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60aea00700 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 0x5566c7e11870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60ae000700 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 0x5566c7e27800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60af400700 1 -- 172.21.15.63:0/3869766583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 msgr2=0x5566c7e27800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60af400700 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 0x5566c7e27800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60af400700 1 -- 172.21.15.63:0/3869766583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 msgr2=0x5566c7e11870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60af400700 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 0x5566c7e11870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60af400700 1 -- 172.21.15.63:0/3869766583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c7e12850 con 0x5566c7e17d30 2024-05-27T08:12:32.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60ae000700 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 0x5566c7e27800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.589+0000 7f60aea00700 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 0x5566c7e11870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.590+0000 7f60af400700 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bb50 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009010 tx=0x7f60a8009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3def0045b43aa910 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.590+0000 7f609be00700 1 -- 172.21.15.63:0/3869766583 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8012040 con 0x5566c7e17d30 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.590+0000 7f609be00700 1 -- 172.21.15.63:0/3869766583 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60a8019910 con 0x5566c7e17d30 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.590+0000 7f609be00700 1 -- 172.21.15.63:0/3869766583 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8024870 con 0x5566c7e17d30 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.591+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3869766583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 msgr2=0x5566c7e1bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.591+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bb50 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009010 tx=0x7f60a8009d10 comp rx=0 tx=0).stop 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.591+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3869766583 shutdown_connections 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.591+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e1bb50 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.591+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7c92e40 0x5566c7e11870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.591+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/3869766583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7e11db0 0x5566c7e27800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.591+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3869766583 >> 172.21.15.63:0/3869766583 conn(0x5566c7cf2fc0 msgr2=0x5566c7e1b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.591+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3869766583 shutdown_connections 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.591+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3869766583 wait complete. 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.592+0000 7f60c48e2b80 1 -- start start 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.592+0000 7f60c48e2b80 1 Processor -- start 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7c92e40 0x5566c7e23bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7e11db0 0x5566c7e24130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60c48e2b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e32620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b0d40 con 0x5566c7e17d30 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b21b0 con 0x5566c7e11db0 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60c48e2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b1410 con 0x5566c7c92e40 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60c48e2b80 10 client.0 _ll_register_callbacks cb 0x5566c7cd9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60af400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e32620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60af400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e32620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44276/0 (socket says 172.21.15.63:44276) 2024-05-27T08:12:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60af400700 1 -- 172.21.15.63:0/3747072286 learned_addr learned my addr 172.21.15.63:0/3747072286 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60aea00700 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7c92e40 0x5566c7e23bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.593+0000 7f60ae000700 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7e11db0 0x5566c7e24130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.594+0000 7f60af400700 1 -- 172.21.15.63:0/3747072286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7c92e40 msgr2=0x5566c7e23bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.594+0000 7f60af400700 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7c92e40 0x5566c7e23bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.594+0000 7f60af400700 1 -- 172.21.15.63:0/3747072286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7e11db0 msgr2=0x5566c7e24130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.594+0000 7f60af400700 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7e11db0 0x5566c7e24130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.594+0000 7f60af400700 1 -- 172.21.15.63:0/3747072286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c7e32d70 con 0x5566c7e17d30 2024-05-27T08:12:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.594+0000 7f60af400700 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e32620 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f60a80197e0 tx=0x7f60a801c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2e4693937fea68 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.594+0000 7f6099600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.594+0000 7f6099600700 1 -- 172.21.15.63:0/3747072286 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a801ccd0 con 0x5566c7e17d30 2024-05-27T08:12:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.594+0000 7f6099600700 1 -- 172.21.15.63:0/3747072286 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f60a8024410 con 0x5566c7e17d30 2024-05-27T08:12:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.594+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3747072286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a8019910 con 0x5566c7e17d30 2024-05-27T08:12:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.595+0000 7f6099600700 1 -- 172.21.15.63:0/3747072286 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f60a8029020 con 0x5566c7e17d30 2024-05-27T08:12:32.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.595+0000 7f60c48e2b80 10 client.22869 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.595+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3747072286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5566c7e95a10 con 0x5566c7e17d30 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.595+0000 7f6099600700 1 -- 172.21.15.63:0/3747072286 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60a8019910 con 0x5566c7e17d30 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.596+0000 7f6099600700 1 -- 172.21.15.63:0/3747072286 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f60a8018070 con 0x5566c7e17d30 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.596+0000 7f60c48e2b80 10 client.22869 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.596+0000 7f60c48e2b80 10 client.22869 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.596+0000 7f60c48e2b80 20 client.22869 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.596+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5566c7e95fd0 0x5566c7e983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.596+0000 7f60c48e2b80 4 client.22869 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.596+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3747072286 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566c7d3b570 con 0x5566c7e95fd0 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.596+0000 7f60aea00700 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5566c7e95fd0 0x5566c7e983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.597+0000 7f6099600700 10 client.22869 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:12:32.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.597+0000 7f60aea00700 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5566c7e95fd0 0x5566c7e983f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7c2d52bb1699a58 server_cookie=8ab7811b801ebe48 in_seq=0 out_seq=0 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f6099600700 1 -- 172.21.15.63:0/3747072286 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566c7d3b570 con 0x5566c7e95fd0 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f6099600700 10 client.22869 handle_command_reply: tid=0 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f60c48e2b80 1 client.22869 shutdown 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3747072286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 msgr2=0x5566c7e32620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e32620 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f60a80197e0 tx=0x7f60a801c710 comp rx=0 tx=0).stop 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3747072286 shutdown_connections 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5566c7e95fd0 0x5566c7e983f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5566c7e17d30 0x5566c7e32620 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5566c7c92e40 0x5566c7e23bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f60c48e2b80 1 --2- 172.21.15.63:0/3747072286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566c7e11db0 0x5566c7e24130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3747072286 >> 172.21.15.63:0/3747072286 conn(0x5566c7cf2fc0 msgr2=0x5566c7e16d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:32.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.598+0000 7f6099600700 0 client.22869 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:12:32.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.599+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3747072286 shutdown_connections 2024-05-27T08:12:32.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.599+0000 7f60c48e2b80 1 -- 172.21.15.63:0/3747072286 wait complete. 2024-05-27T08:12:32.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.599+0000 7f60c48e2b80 20 client.22869 trim_cache size 0 max 16384 2024-05-27T08:12:32.603 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:12:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 -- 172.21.15.63:0/2558101113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60940b29f0 msgr2=0x7f60940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60940b29f0 0x7f60940b4f00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f60a80060b0 tx=0x7f60a80093b0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 -- 172.21.15.63:0/2558101113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b0106ec0 msgr2=0x7f60b019d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b0106ec0 0x7f60b019d520 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f609c00d950 tx=0x7f609c00dc60 comp rx=0 tx=0).stop 2024-05-27T08:12:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 -- 172.21.15.63:0/2558101113 shutdown_connections 2024-05-27T08:12:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f60940b29f0 0x7f60940b4f00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60b010c5b0 0x7f60b01a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60b0106570 0x7f60b019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 --2- 172.21.15.63:0/2558101113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b0106ec0 0x7f60b019d520 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:32.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 -- 172.21.15.63:0/2558101113 >> 172.21.15.63:0/2558101113 conn(0x7f60b0102070 msgr2=0x7f60b01049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:32.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 -- 172.21.15.63:0/2558101113 shutdown_connections 2024-05-27T08:12:32.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:32.600+0000 7f60b5e00700 1 -- 172.21.15.63:0/2558101113 wait complete. 2024-05-27T08:12:33.552 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:12:34.080 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:12:34.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:33 smithi086 ceph-mon[34687]: pgmap v1237: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.1 MiB/s wr, 15 op/s 2024-05-27T08:12:34.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:33 smithi149 ceph-mon[35408]: pgmap v1237: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.1 MiB/s wr, 15 op/s 2024-05-27T08:12:34.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:33 smithi063 ceph-mon[31613]: pgmap v1237: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.1 MiB/s wr, 15 op/s 2024-05-27T08:12:36.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:35 smithi086 ceph-mon[34687]: pgmap v1238: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:12:36.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:35 smithi149 ceph-mon[35408]: pgmap v1238: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:12:36.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:35 smithi063 ceph-mon[31613]: pgmap v1238: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:12:36.879 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:36 smithi063 ceph-mon[31613]: pgmap v1239: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:12:37.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:36 smithi086 ceph-mon[34687]: pgmap v1239: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:12:37.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:36 smithi149 ceph-mon[35408]: pgmap v1239: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 15 op/s 2024-05-27T08:12:37.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.829+0000 7f5fbe200700 1 -- 172.21.15.63:0/1473389799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb8106560 msgr2=0x7f5fb8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:37.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.829+0000 7f5fbe200700 1 --2- 172.21.15.63:0/1473389799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb8106560 0x7f5fb8106970 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009990 tx=0x7f5fa4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.829+0000 7f5fbe200700 1 -- 172.21.15.63:0/1473389799 shutdown_connections 2024-05-27T08:12:37.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.829+0000 7f5fbe200700 1 --2- 172.21.15.63:0/1473389799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb8106eb0 0x7f5fb8113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.829+0000 7f5fbe200700 1 --2- 172.21.15.63:0/1473389799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb810c5a0 0x7f5fb810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.829+0000 7f5fbe200700 1 --2- 172.21.15.63:0/1473389799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb8106560 0x7f5fb8106970 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.829+0000 7f5fbe200700 1 -- 172.21.15.63:0/1473389799 >> 172.21.15.63:0/1473389799 conn(0x7f5fb8102040 msgr2=0x7f5fb8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:37.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.829+0000 7f5fbe200700 1 -- 172.21.15.63:0/1473389799 shutdown_connections 2024-05-27T08:12:37.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.829+0000 7f5fbe200700 1 -- 172.21.15.63:0/1473389799 wait complete. 2024-05-27T08:12:37.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.831+0000 7f5fbe200700 1 Processor -- start 2024-05-27T08:12:37.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.831+0000 7f5fbe200700 1 -- start start 2024-05-27T08:12:37.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.832+0000 7f5fbe200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb8106560 0x7f5fb819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.832+0000 7f5fbe200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb8106eb0 0x7f5fb819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.832+0000 7f5fbe200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb810c5a0 0x7f5fb81a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.832+0000 7f5fbe200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8116580 con 0x7f5fb8106560 2024-05-27T08:12:37.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.832+0000 7f5fbe200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8116400 con 0x7f5fb810c5a0 2024-05-27T08:12:37.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.832+0000 7f5fbe200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8116700 con 0x7f5fb8106eb0 2024-05-27T08:12:37.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.832+0000 7f5fb7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb8106560 0x7f5fb819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.832+0000 7f5fb7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb8106560 0x7f5fb819cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44292/0 (socket says 172.21.15.63:44292) 2024-05-27T08:12:37.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.832+0000 7f5fb7400700 1 -- 172.21.15.63:0/3820302143 learned_addr learned my addr 172.21.15.63:0/3820302143 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:37.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.832+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb8106eb0 0x7f5fb819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.833+0000 7f5fb7e00700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb810c5a0 0x7f5fb81a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.833+0000 7f5fb7400700 1 -- 172.21.15.63:0/3820302143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb8106eb0 msgr2=0x7f5fb819d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:37.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.833+0000 7f5fb7400700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb8106eb0 0x7f5fb819d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.833+0000 7f5fb7400700 1 -- 172.21.15.63:0/3820302143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb810c5a0 msgr2=0x7f5fb81a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:37.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.833+0000 7f5fb7400700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb810c5a0 0x7f5fb81a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.833+0000 7f5fb7400700 1 -- 172.21.15.63:0/3820302143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb81a1e20 con 0x7f5fb8106560 2024-05-27T08:12:37.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.834+0000 7f5fb7400700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb8106560 0x7f5fb819cf50 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800d950 tx=0x7f5fa800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:37.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.834+0000 7f5fafe00700 1 -- 172.21.15.63:0/3820302143 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8016070 con 0x7f5fb8106560 2024-05-27T08:12:37.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.834+0000 7f5fafe00700 1 -- 172.21.15.63:0/3820302143 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fa8004830 con 0x7f5fb8106560 2024-05-27T08:12:37.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.834+0000 7f5fbe200700 1 -- 172.21.15.63:0/3820302143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb81a2110 con 0x7f5fb8106560 2024-05-27T08:12:37.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.834+0000 7f5fafe00700 1 -- 172.21.15.63:0/3820302143 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8005410 con 0x7f5fb8106560 2024-05-27T08:12:37.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.834+0000 7f5fbe200700 1 -- 172.21.15.63:0/3820302143 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb81a9a30 con 0x7f5fb8106560 2024-05-27T08:12:37.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.839+0000 7f5fafe00700 1 -- 172.21.15.63:0/3820302143 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5fa80055f0 con 0x7f5fb8106560 2024-05-27T08:12:37.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.841+0000 7f5fafe00700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5f9c0b29f0 0x7f5f9c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.841+0000 7f5fafe00700 1 -- 172.21.15.63:0/3820302143 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f5fa80e2710 con 0x7f5fb8106560 2024-05-27T08:12:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.841+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5f9c0b29f0 0x7f5f9c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.842+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5f9c0b29f0 0x7f5f9c0b4f00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4006010 tx=0x7f5fa400b540 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:37.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.870+0000 7f5fccddeb80 1 Processor -- start 2024-05-27T08:12:37.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.870+0000 7f5fccddeb80 1 -- start start 2024-05-27T08:12:37.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.871+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce310e40 0x5622ce4957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce495d30 0x5622ce48f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce48fdb0 0x5622ce4901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5622ce3bde40 con 0x5622ce310e40 2024-05-27T08:12:37.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622ce3ad9e0 con 0x5622ce495d30 2024-05-27T08:12:37.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5622ce3b1a20 con 0x5622ce48fdb0 2024-05-27T08:12:37.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fb7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce310e40 0x5622ce4957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fb7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce310e40 0x5622ce4957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44300/0 (socket says 172.21.15.63:44300) 2024-05-27T08:12:37.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fb7400700 1 -- 172.21.15.63:0/1569337461 learned_addr learned my addr 172.21.15.63:0/1569337461 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:37.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/1569337461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce495d30 0x5622ce48f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fb7400700 1 -- 172.21.15.63:0/1569337461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce48fdb0 msgr2=0x5622ce4901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:37.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.872+0000 7f5fb7400700 1 --2- 172.21.15.63:0/1569337461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce48fdb0 0x5622ce4901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.873+0000 7f5fb7400700 1 -- 172.21.15.63:0/1569337461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce495d30 msgr2=0x5622ce48f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:37.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.873+0000 7f5fb7400700 1 --2- 172.21.15.63:0/1569337461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce495d30 0x5622ce48f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.873+0000 7f5fb7400700 1 -- 172.21.15.63:0/1569337461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ce376420 con 0x5622ce310e40 2024-05-27T08:12:37.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.873+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/1569337461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce495d30 0x5622ce48f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:12:37.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.873+0000 7f5fb7400700 1 --2- 172.21.15.63:0/1569337461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce310e40 0x5622ce4957f0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80a5650 tx=0x7f5fa80a9e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bbeecdbb89f1ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:37.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.874+0000 7f5facc00700 1 -- 172.21.15.63:0/1569337461 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80b2430 con 0x5622ce310e40 2024-05-27T08:12:37.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.874+0000 7f5facc00700 1 -- 172.21.15.63:0/1569337461 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fa80049e0 con 0x5622ce310e40 2024-05-27T08:12:37.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.874+0000 7f5facc00700 1 -- 172.21.15.63:0/1569337461 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa80b2200 con 0x5622ce310e40 2024-05-27T08:12:37.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.875+0000 7f5fccddeb80 1 -- 172.21.15.63:0/1569337461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce310e40 msgr2=0x5622ce4957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:37.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.875+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/1569337461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce310e40 0x5622ce4957f0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80a5650 tx=0x7f5fa80a9e00 comp rx=0 tx=0).stop 2024-05-27T08:12:37.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.876+0000 7f5fccddeb80 1 -- 172.21.15.63:0/1569337461 shutdown_connections 2024-05-27T08:12:37.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.876+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/1569337461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce48fdb0 0x5622ce4901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.876+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/1569337461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce310e40 0x5622ce4957f0 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.876+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/1569337461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce495d30 0x5622ce48f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.876+0000 7f5fccddeb80 1 -- 172.21.15.63:0/1569337461 >> 172.21.15.63:0/1569337461 conn(0x5622ce370fc0 msgr2=0x5622ce48a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:37.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.876+0000 7f5fccddeb80 1 -- 172.21.15.63:0/1569337461 shutdown_connections 2024-05-27T08:12:37.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.876+0000 7f5fccddeb80 1 -- 172.21.15.63:0/1569337461 wait complete. 2024-05-27T08:12:37.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.877+0000 7f5fccddeb80 1 -- start start 2024-05-27T08:12:37.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.877+0000 7f5fccddeb80 1 Processor -- start 2024-05-27T08:12:37.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.878+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce310e40 0x5622ce499560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.878+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce499aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.878+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce495d30 0x5622ce499fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.878+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5622ce2b3fa0 con 0x5622ce48fdb0 2024-05-27T08:12:37.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.878+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5622ce34e760 con 0x5622ce310e40 2024-05-27T08:12:37.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.878+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5622ce49c800 con 0x5622ce495d30 2024-05-27T08:12:37.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.878+0000 7f5fccddeb80 10 client.0 _ll_register_callbacks cb 0x5622ce357c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:37.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.878+0000 7f5fb6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce499aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.878+0000 7f5fb6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce499aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44304/0 (socket says 172.21.15.63:44304) 2024-05-27T08:12:37.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.879+0000 7f5fb6a00700 1 -- 172.21.15.63:0/682917979 learned_addr learned my addr 172.21.15.63:0/682917979 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:37.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.879+0000 7f5fb7400700 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce310e40 0x5622ce499560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.879+0000 7f5fb7e00700 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce495d30 0x5622ce499fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.879+0000 7f5fb6a00700 1 -- 172.21.15.63:0/682917979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce495d30 msgr2=0x5622ce499fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.879+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce495d30 0x5622ce499fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.879+0000 7f5fb6a00700 1 -- 172.21.15.63:0/682917979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce310e40 msgr2=0x5622ce499560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.879+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce310e40 0x5622ce499560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.879+0000 7f5fb6a00700 1 -- 172.21.15.63:0/682917979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ce2fd8d0 con 0x5622ce48fdb0 2024-05-27T08:12:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.880+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce499aa0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4015000 tx=0x7f5fa4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d39125f26e79454 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.880+0000 7f5fa1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.880+0000 7f5fa1600700 1 -- 172.21.15.63:0/682917979 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4033060 con 0x5622ce48fdb0 2024-05-27T08:12:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.880+0000 7f5fa1600700 1 -- 172.21.15.63:0/682917979 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f5fa4003bd0 con 0x5622ce48fdb0 2024-05-27T08:12:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.880+0000 7f5fccddeb80 1 -- 172.21.15.63:0/682917979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa80049e0 con 0x5622ce48fdb0 2024-05-27T08:12:37.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.881+0000 7f5fa1600700 1 -- 172.21.15.63:0/682917979 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5fa403a050 con 0x5622ce48fdb0 2024-05-27T08:12:37.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.881+0000 7f5fccddeb80 10 client.22881 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:37.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.881+0000 7f5fccddeb80 1 -- 172.21.15.63:0/682917979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5622ce37e060 con 0x5622ce48fdb0 2024-05-27T08:12:37.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.881+0000 7f5fa1600700 1 -- 172.21.15.63:0/682917979 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fa80049e0 con 0x5622ce48fdb0 2024-05-27T08:12:37.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.881+0000 7f5fa1600700 1 -- 172.21.15.63:0/682917979 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5fa4022070 con 0x5622ce48fdb0 2024-05-27T08:12:37.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.882+0000 7f5fccddeb80 10 client.22881 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:37.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.882+0000 7f5fccddeb80 10 client.22881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:37.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.882+0000 7f5fccddeb80 20 client.22881 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:37.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.882+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622ce4a9710 0x5622ce4abb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.882+0000 7f5fccddeb80 4 client.22881 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:37.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.882+0000 7f5fccddeb80 1 -- 172.21.15.63:0/682917979 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622ce3bfd90 con 0x5622ce4a9710 2024-05-27T08:12:37.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.883+0000 7f5fb7400700 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622ce4a9710 0x5622ce4abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.883+0000 7f5fb7400700 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622ce4a9710 0x5622ce4abb30 crc :-1 s=READY pgs=692 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451e366a7097536 server_cookie=beee47b81f96f5df in_seq=0 out_seq=0 2024-05-27T08:12:37.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.883+0000 7f5fa1600700 10 client.22881 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:37.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.885+0000 7f5fa1600700 1 -- 172.21.15.63:0/682917979 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5622ce3bfd90 con 0x5622ce4a9710 2024-05-27T08:12:37.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.885+0000 7f5fa1600700 10 client.22881 handle_command_reply: tid=0 2024-05-27T08:12:37.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fccddeb80 1 client.22881 shutdown 2024-05-27T08:12:37.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fccddeb80 1 -- 172.21.15.63:0/682917979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 msgr2=0x5622ce499aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:37.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce499aa0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4015000 tx=0x7f5fa4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fccddeb80 1 -- 172.21.15.63:0/682917979 shutdown_connections 2024-05-27T08:12:37.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622ce4a9710 0x5622ce4abb30 crc :-1 s=READY pgs=692 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce495d30 0x5622ce499fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce310e40 0x5622ce499560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/682917979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce499aa0 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fccddeb80 1 -- 172.21.15.63:0/682917979 >> 172.21.15.63:0/682917979 conn(0x5622ce370fc0 msgr2=0x5622ce48a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:37.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fa1600700 0 client.22881 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:37.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.886+0000 7f5fccddeb80 1 -- 172.21.15.63:0/682917979 shutdown_connections 2024-05-27T08:12:37.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.887+0000 7f5fccddeb80 1 -- 172.21.15.63:0/682917979 wait complete. 2024-05-27T08:12:37.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.887+0000 7f5fccddeb80 20 client.22881 trim_cache size 0 max 16384 2024-05-27T08:12:37.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.909+0000 7f5fccddeb80 1 Processor -- start 2024-05-27T08:12:37.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.909+0000 7f5fccddeb80 1 -- start start 2024-05-27T08:12:37.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.909+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce310e40 0x5622ce48f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.909+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce48fdb0 0x5622ce49ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.909+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce495d30 0x5622ce4a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.909+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b4f80 con 0x5622ce495d30 2024-05-27T08:12:37.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.909+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b2810 con 0x5622ce48fdb0 2024-05-27T08:12:37.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.909+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b0680 con 0x5622ce310e40 2024-05-27T08:12:37.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.910+0000 7f5fb7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce495d30 0x5622ce4a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.910+0000 7f5fb7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce495d30 0x5622ce4a5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44308/0 (socket says 172.21.15.63:44308) 2024-05-27T08:12:37.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.910+0000 7f5fb7e00700 1 -- 172.21.15.63:0/474585518 learned_addr learned my addr 172.21.15.63:0/474585518 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:37.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.910+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/474585518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce48fdb0 0x5622ce49ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.910+0000 7f5fb7400700 1 --2- 172.21.15.63:0/474585518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce310e40 0x5622ce48f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.910+0000 7f5fb7e00700 1 -- 172.21.15.63:0/474585518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce310e40 msgr2=0x5622ce48f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:37.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.910+0000 7f5fb7e00700 1 --2- 172.21.15.63:0/474585518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce310e40 0x5622ce48f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.910+0000 7f5fb7e00700 1 -- 172.21.15.63:0/474585518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce48fdb0 msgr2=0x5622ce49ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:37.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.910+0000 7f5fb7e00700 1 --2- 172.21.15.63:0/474585518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce48fdb0 0x5622ce49ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.910+0000 7f5fb7e00700 1 -- 172.21.15.63:0/474585518 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ce497ad0 con 0x5622ce495d30 2024-05-27T08:12:37.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.911+0000 7f5fb7e00700 1 --2- 172.21.15.63:0/474585518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce495d30 0x5622ce4a5860 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00098f0 tx=0x7f5fb0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a63906a93b0a831 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:37.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.911+0000 7f5fa3e00700 1 -- 172.21.15.63:0/474585518 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb001a020 con 0x5622ce495d30 2024-05-27T08:12:37.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.911+0000 7f5fa3e00700 1 -- 172.21.15.63:0/474585518 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fb0004650 con 0x5622ce495d30 2024-05-27T08:12:37.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.911+0000 7f5fa3e00700 1 -- 172.21.15.63:0/474585518 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00166c0 con 0x5622ce495d30 2024-05-27T08:12:37.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.912+0000 7f5fccddeb80 1 -- 172.21.15.63:0/474585518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce495d30 msgr2=0x5622ce4a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:37.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.912+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/474585518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce495d30 0x5622ce4a5860 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00098f0 tx=0x7f5fb0009c00 comp rx=0 tx=0).stop 2024-05-27T08:12:37.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.912+0000 7f5fccddeb80 1 -- 172.21.15.63:0/474585518 shutdown_connections 2024-05-27T08:12:37.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.912+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/474585518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce495d30 0x5622ce4a5860 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.912+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/474585518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce310e40 0x5622ce48f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.912+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/474585518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce48fdb0 0x5622ce49ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.912+0000 7f5fccddeb80 1 -- 172.21.15.63:0/474585518 >> 172.21.15.63:0/474585518 conn(0x5622ce370fc0 msgr2=0x5622ce499520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:37.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.913+0000 7f5fccddeb80 1 -- 172.21.15.63:0/474585518 shutdown_connections 2024-05-27T08:12:37.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.913+0000 7f5fccddeb80 1 -- 172.21.15.63:0/474585518 wait complete. 2024-05-27T08:12:37.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.913+0000 7f5fccddeb80 1 -- start start 2024-05-27T08:12:37.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.914+0000 7f5fccddeb80 1 Processor -- start 2024-05-27T08:12:37.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.914+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce310e40 0x5622ce4a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.914+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce4a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.914+0000 7f5fccddeb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce495d30 0x5622ce4b0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.914+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b0cd0 con 0x5622ce48fdb0 2024-05-27T08:12:37.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.914+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b2140 con 0x5622ce310e40 2024-05-27T08:12:37.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.914+0000 7f5fccddeb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb81b13a0 con 0x5622ce495d30 2024-05-27T08:12:37.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fccddeb80 10 client.0 _ll_register_callbacks cb 0x5622ce357c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:37.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fb6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce4a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fb6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce4a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44320/0 (socket says 172.21.15.63:44320) 2024-05-27T08:12:37.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fb6a00700 1 -- 172.21.15.63:0/3707149691 learned_addr learned my addr 172.21.15.63:0/3707149691 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:37.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fb7e00700 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce495d30 0x5622ce4b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fb7400700 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce310e40 0x5622ce4a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fb6a00700 1 -- 172.21.15.63:0/3707149691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce495d30 msgr2=0x5622ce4b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:37.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce495d30 0x5622ce4b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fb6a00700 1 -- 172.21.15.63:0/3707149691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce310e40 msgr2=0x5622ce4a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:37.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce310e40 0x5622ce4a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.915+0000 7f5fb6a00700 1 -- 172.21.15.63:0/3707149691 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ce4b1d10 con 0x5622ce48fdb0 2024-05-27T08:12:37.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.916+0000 7f5fb6a00700 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce4a1f90 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4000c00 tx=0x7f5fa4003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58637481f15f94d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:37.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.916+0000 7f5fa1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:37.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.916+0000 7f5fa1600700 1 -- 172.21.15.63:0/3707149691 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4033040 con 0x5622ce48fdb0 2024-05-27T08:12:37.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.916+0000 7f5fa1600700 1 -- 172.21.15.63:0/3707149691 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f5fa402eb10 con 0x5622ce48fdb0 2024-05-27T08:12:37.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.916+0000 7f5fccddeb80 1 -- 172.21.15.63:0/3707149691 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fb0004650 con 0x5622ce48fdb0 2024-05-27T08:12:37.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.917+0000 7f5fa1600700 1 -- 172.21.15.63:0/3707149691 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5fa400be90 con 0x5622ce48fdb0 2024-05-27T08:12:37.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.917+0000 7f5fccddeb80 10 client.22887 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:37.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.917+0000 7f5fccddeb80 1 -- 172.21.15.63:0/3707149691 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5622ce5135b0 con 0x5622ce48fdb0 2024-05-27T08:12:37.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.917+0000 7f5fa1600700 1 -- 172.21.15.63:0/3707149691 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fb0004650 con 0x5622ce48fdb0 2024-05-27T08:12:37.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.917+0000 7f5fa1600700 1 -- 172.21.15.63:0/3707149691 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5fa4022070 con 0x5622ce48fdb0 2024-05-27T08:12:37.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.917+0000 7f5fccddeb80 10 client.22887 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:37.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.917+0000 7f5fccddeb80 10 client.22887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:37.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.917+0000 7f5fccddeb80 20 client.22887 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:37.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.918+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622ce513b80 0x5622ce515fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:37.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.918+0000 7f5fccddeb80 4 client.22887 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:12:37.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.918+0000 7f5fccddeb80 1 -- 172.21.15.63:0/3707149691 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622ce3b9570 con 0x5622ce513b80 2024-05-27T08:12:37.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.918+0000 7f5fb7400700 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622ce513b80 0x5622ce515fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:37.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.919+0000 7f5fb7400700 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622ce513b80 0x5622ce515fa0 crc :-1 s=READY pgs=693 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01793bdd76e6546 server_cookie=7831a511a5d3b0bd in_seq=0 out_seq=0 2024-05-27T08:12:37.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.919+0000 7f5fa1600700 10 client.22887 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:37.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fa1600700 1 -- 172.21.15.63:0/3707149691 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622ce3b9570 con 0x5622ce513b80 2024-05-27T08:12:37.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fa1600700 10 client.22887 handle_command_reply: tid=0 2024-05-27T08:12:37.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fccddeb80 1 client.22887 shutdown 2024-05-27T08:12:37.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fccddeb80 1 -- 172.21.15.63:0/3707149691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 msgr2=0x5622ce4a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:37.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce4a1f90 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4000c00 tx=0x7f5fa4003d30 comp rx=0 tx=0).stop 2024-05-27T08:12:37.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fccddeb80 1 -- 172.21.15.63:0/3707149691 shutdown_connections 2024-05-27T08:12:37.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5622ce513b80 0x5622ce515fa0 crc :-1 s=READY pgs=693 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5622ce495d30 0x5622ce4b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5622ce310e40 0x5622ce4a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fccddeb80 1 --2- 172.21.15.63:0/3707149691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5622ce48fdb0 0x5622ce4a1f90 unknown :-1 s=CLOSED pgs=2792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fccddeb80 1 -- 172.21.15.63:0/3707149691 >> 172.21.15.63:0/3707149691 conn(0x5622ce370fc0 msgr2=0x5622ce493140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:37.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fa1600700 0 client.22887 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:37.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.920+0000 7f5fccddeb80 1 -- 172.21.15.63:0/3707149691 shutdown_connections 2024-05-27T08:12:37.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.921+0000 7f5fccddeb80 1 -- 172.21.15.63:0/3707149691 wait complete. 2024-05-27T08:12:37.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.921+0000 7f5fccddeb80 20 client.22887 trim_cache size 0 max 16384 2024-05-27T08:12:37.923 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:12:37.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 -- 172.21.15.63:0/3820302143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5f9c0b29f0 msgr2=0x7f5f9c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:37.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5f9c0b29f0 0x7f5f9c0b4f00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4006010 tx=0x7f5fa400b540 comp rx=0 tx=0).stop 2024-05-27T08:12:37.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 -- 172.21.15.63:0/3820302143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb8106560 msgr2=0x7f5fb819cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:37.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb8106560 0x7f5fb819cf50 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800d950 tx=0x7f5fa800dc60 comp rx=0 tx=0).stop 2024-05-27T08:12:37.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 -- 172.21.15.63:0/3820302143 shutdown_connections 2024-05-27T08:12:37.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5f9c0b29f0 0x7f5f9c0b4f00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fb810c5a0 0x7f5fb81a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5fb8106560 0x7f5fb819cf50 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 --2- 172.21.15.63:0/3820302143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fb8106eb0 0x7f5fb819d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:37.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 -- 172.21.15.63:0/3820302143 >> 172.21.15.63:0/3820302143 conn(0x7f5fb8102040 msgr2=0x7f5fb8103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:37.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 -- 172.21.15.63:0/3820302143 shutdown_connections 2024-05-27T08:12:37.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:37.922+0000 7f5fbe200700 1 -- 172.21.15.63:0/3820302143 wait complete. 2024-05-27T08:12:38.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:38 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:12:38.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:38 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:12:38.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:38 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:12:39.859 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:12:39.860 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:12:39.937 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:39 smithi063 ceph-mon[31613]: pgmap v1240: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:12:39.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:39 smithi086 ceph-mon[34687]: pgmap v1240: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:12:39.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:39 smithi149 ceph-mon[35408]: pgmap v1240: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:12:40.381 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:12:40.789 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:40 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:12:40.789 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:40 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:12:40.789 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:40 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:12:40.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:40 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:12:40.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:40 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:12:40.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:40 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:12:40.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:40 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:12:40.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:40 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:12:40.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:40 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:12:41.833 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:41 smithi063 ceph-mon[31613]: pgmap v1241: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:12:41.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:41 smithi086 ceph-mon[34687]: pgmap v1241: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:12:41.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:41 smithi149 ceph-mon[35408]: pgmap v1241: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-05-27T08:12:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.498+0000 7f586d600640 1 -- 172.21.15.63:0/912970741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5868109ab0 msgr2=0x7f58681036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.498+0000 7f586d600640 1 --2- 172.21.15.63:0/912970741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5868109ab0 0x7f58681036a0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f5858009900 tx=0x7f585802a410 comp rx=0 tx=0).stop 2024-05-27T08:12:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.499+0000 7f586d600640 1 -- 172.21.15.63:0/912970741 shutdown_connections 2024-05-27T08:12:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.499+0000 7f586d600640 1 --2- 172.21.15.63:0/912970741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5868104250 0x7f58681105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.499+0000 7f586d600640 1 --2- 172.21.15.63:0/912970741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5868109f50 0x7f5868103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.499+0000 7f586d600640 1 --2- 172.21.15.63:0/912970741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5868109ab0 0x7f58681036a0 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.499+0000 7f586d600640 1 -- 172.21.15.63:0/912970741 >> 172.21.15.63:0/912970741 conn(0x7f5868079240 msgr2=0x7f5868079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.499+0000 7f586d600640 1 -- 172.21.15.63:0/912970741 shutdown_connections 2024-05-27T08:12:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.499+0000 7f586d600640 1 -- 172.21.15.63:0/912970741 wait complete. 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.500+0000 7f586d600640 1 Processor -- start 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.500+0000 7f586d600640 1 -- start start 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f586d600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5868104250 0x7f586819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f586d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5868109ab0 0x7f586819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f586d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5868109f50 0x7f58681a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f586d600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5868113b50 con 0x7f5868104250 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f586d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58681139d0 con 0x7f5868109f50 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f586d600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5868113cd0 con 0x7f5868109ab0 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f5867e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5868104250 0x7f586819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f5867e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5868104250 0x7f586819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38166/0 (socket says 172.21.15.63:38166) 2024-05-27T08:12:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f5867e00640 1 -- 172.21.15.63:0/327143180 learned_addr learned my addr 172.21.15.63:0/327143180 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f5867400640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5868109ab0 0x7f586819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.501+0000 7f586cc00640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5868109f50 0x7f58681a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.502+0000 7f5867e00640 1 -- 172.21.15.63:0/327143180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5868109ab0 msgr2=0x7f586819db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.502+0000 7f5867e00640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5868109ab0 0x7f586819db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.502+0000 7f5867e00640 1 -- 172.21.15.63:0/327143180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5868109f50 msgr2=0x7f58681a8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.502+0000 7f5867e00640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5868109f50 0x7f58681a8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.502+0000 7f5867e00640 1 -- 172.21.15.63:0/327143180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58681a89d0 con 0x7f5868104250 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.502+0000 7f586cc00640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5868109f50 0x7f58681a8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.502+0000 7f5867e00640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5868104250 0x7f586819d5f0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f5858005940 tx=0x7f58580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.503+0000 7f5864c00640 1 -- 172.21.15.63:0/327143180 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585803f030 con 0x7f5868104250 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.503+0000 7f5864c00640 1 -- 172.21.15.63:0/327143180 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5858003c90 con 0x7f5868104250 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.503+0000 7f586d600640 1 -- 172.21.15.63:0/327143180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58681a8c00 con 0x7f5868104250 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.503+0000 7f5864c00640 1 -- 172.21.15.63:0/327143180 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858035070 con 0x7f5868104250 2024-05-27T08:12:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.503+0000 7f586d600640 1 -- 172.21.15.63:0/327143180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58681a91a0 con 0x7f5868104250 2024-05-27T08:12:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.506+0000 7f5875ba8740 1 Processor -- start 2024-05-27T08:12:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.507+0000 7f5875ba8740 1 -- start start 2024-05-27T08:12:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.507+0000 7f5864c00640 1 -- 172.21.15.63:0/327143180 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5858042030 con 0x7f5868104250 2024-05-27T08:12:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.507+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb8021b0 0x5618ebba4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.507+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebba4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.507+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebba54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.507+0000 7f5875ba8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5618ebb15b90 con 0x5618eb7d92f0 2024-05-27T08:12:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.507+0000 7f5875ba8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5618ebb05730 con 0x5618eb8021b0 2024-05-27T08:12:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.507+0000 7f5875ba8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5618ebb09770 con 0x5618eb7e14d0 2024-05-27T08:12:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.508+0000 7f5867e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebba4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.508+0000 7f5867e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebba4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38178/0 (socket says 172.21.15.63:38178) 2024-05-27T08:12:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.508+0000 7f5867e00640 1 -- 172.21.15.63:0/1305837850 learned_addr learned my addr 172.21.15.63:0/1305837850 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.508+0000 7f586cc00640 1 --2- 172.21.15.63:0/1305837850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb8021b0 0x5618ebba4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.508+0000 7f5867400640 1 --2- 172.21.15.63:0/1305837850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebba54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.508+0000 7f5867e00640 1 -- 172.21.15.63:0/1305837850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 msgr2=0x5618ebba54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.508+0000 7f5867e00640 1 --2- 172.21.15.63:0/1305837850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebba54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.508+0000 7f5867e00640 1 -- 172.21.15.63:0/1305837850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb8021b0 msgr2=0x5618ebba4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.508+0000 7f5867e00640 1 --2- 172.21.15.63:0/1305837850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb8021b0 0x5618ebba4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.509+0000 7f5867e00640 1 -- 172.21.15.63:0/1305837850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618eba9e8b0 con 0x5618eb7d92f0 2024-05-27T08:12:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.509+0000 7f5867400640 1 --2- 172.21.15.63:0/1305837850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebba54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.509+0000 7f5867e00640 1 --2- 172.21.15.63:0/1305837850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebba4fa0 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f5858100160 tx=0x7f585810b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83395fad4a8d8127 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.509+0000 7f585d200640 1 -- 172.21.15.63:0/1305837850 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585810b9e0 con 0x5618eb7d92f0 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.510+0000 7f585d200640 1 -- 172.21.15.63:0/1305837850 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5858110070 con 0x5618eb7d92f0 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.510+0000 7f585d200640 1 -- 172.21.15.63:0/1305837850 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858113370 con 0x5618eb7d92f0 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.510+0000 7f5864c00640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f584c0b29d0 0x7f584c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.510+0000 7f5864c00640 1 -- 172.21.15.63:0/327143180 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f5858102ef0 con 0x7f5868104250 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.510+0000 7f586cc00640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f584c0b29d0 0x7f584c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.511+0000 7f5875ba8740 1 -- 172.21.15.63:0/1305837850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 msgr2=0x5618ebba4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.511+0000 7f5875ba8740 1 --2- 172.21.15.63:0/1305837850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebba4fa0 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f5858100160 tx=0x7f585810b1a0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.511+0000 7f586cc00640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f584c0b29d0 0x7f584c0b4ee0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x5618ebba1bd0 tx=0x7f5860009800 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.511+0000 7f5875ba8740 1 -- 172.21.15.63:0/1305837850 shutdown_connections 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.511+0000 7f5875ba8740 1 --2- 172.21.15.63:0/1305837850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebba54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.511+0000 7f5875ba8740 1 --2- 172.21.15.63:0/1305837850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebba4fa0 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.511+0000 7f5875ba8740 1 --2- 172.21.15.63:0/1305837850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb8021b0 0x5618ebba4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.511+0000 7f5875ba8740 1 -- 172.21.15.63:0/1305837850 >> 172.21.15.63:0/1305837850 conn(0x5618eb7f6300 msgr2=0x5618ebba1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.511+0000 7f5875ba8740 1 -- 172.21.15.63:0/1305837850 shutdown_connections 2024-05-27T08:12:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.511+0000 7f5875ba8740 1 -- 172.21.15.63:0/1305837850 wait complete. 2024-05-27T08:12:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.512+0000 7f5875ba8740 1 -- start start 2024-05-27T08:12:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.512+0000 7f5875ba8740 1 Processor -- start 2024-05-27T08:12:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7d92f0 0x5618ebbaec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7e14d0 0x5618ebbaf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbbac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f5875ba8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5618eb7eb790 con 0x5618eb8021b0 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f5875ba8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5618ebaadbc0 con 0x5618eb7e14d0 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f5875ba8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5618ebbb1950 con 0x5618eb7d92f0 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f5875ba8740 10 client.0 _ll_register_callbacks cb 0x5618eba8b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f586cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbbac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f586cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbbac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38190/0 (socket says 172.21.15.63:38190) 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f586cc00640 1 -- 172.21.15.63:0/2254083959 learned_addr learned my addr 172.21.15.63:0/2254083959 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.513+0000 7f5867e00640 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7d92f0 0x5618ebbaec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.514+0000 7f5867400640 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7e14d0 0x5618ebbaf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.514+0000 7f586cc00640 1 -- 172.21.15.63:0/2254083959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7d92f0 msgr2=0x5618ebbaec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.514+0000 7f586cc00640 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7d92f0 0x5618ebbaec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.514+0000 7f586cc00640 1 -- 172.21.15.63:0/2254083959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7e14d0 msgr2=0x5618ebbaf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.514+0000 7f586cc00640 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7e14d0 0x5618ebbaf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.514+0000 7f586cc00640 1 -- 172.21.15.63:0/2254083959 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618eb81a210 con 0x5618eb8021b0 2024-05-27T08:12:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.514+0000 7f5867400640 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7e14d0 0x5618ebbaf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:12:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.514+0000 7f586cc00640 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbbac00 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f5860011ce0 tx=0x7f5860004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516d451ef3f63215 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.514+0000 7f5851600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.514+0000 7f5851600640 1 -- 172.21.15.63:0/2254083959 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860020620 con 0x5618eb8021b0 2024-05-27T08:12:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.515+0000 7f5875ba8740 1 -- 172.21.15.63:0/2254083959 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5858110070 con 0x5618eb8021b0 2024-05-27T08:12:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.515+0000 7f5851600640 1 -- 172.21.15.63:0/2254083959 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f58600207c0 con 0x5618eb8021b0 2024-05-27T08:12:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.515+0000 7f5851600640 1 -- 172.21.15.63:0/2254083959 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5860033050 con 0x5618eb8021b0 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.515+0000 7f5875ba8740 10 client.22899 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.515+0000 7f5875ba8740 1 -- 172.21.15.63:0/2254083959 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5618eb8e6620 con 0x5618eb8021b0 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.515+0000 7f5851600640 1 -- 172.21.15.63:0/2254083959 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5858110070 con 0x5618eb8021b0 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.516+0000 7f5851600640 1 -- 172.21.15.63:0/2254083959 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f586000f040 con 0x5618eb8021b0 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.516+0000 7f5875ba8740 10 client.22899 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.516+0000 7f5875ba8740 10 client.22899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.516+0000 7f5875ba8740 20 client.22899 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.516+0000 7f5875ba8740 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5618eb7c0d80 0x5618ebbc0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.516+0000 7f5875ba8740 4 client.22899 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.516+0000 7f5875ba8740 1 -- 172.21.15.63:0/2254083959 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618ebb17ae0 con 0x5618eb7c0d80 2024-05-27T08:12:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.516+0000 7f5867e00640 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5618eb7c0d80 0x5618ebbc0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.517+0000 7f5867e00640 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5618eb7c0d80 0x5618ebbc0cd0 crc :-1 s=READY pgs=694 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=823f8797a8c20970 server_cookie=912d0aea039d4619 in_seq=0 out_seq=0 2024-05-27T08:12:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.517+0000 7f5851600640 10 client.22899 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.519+0000 7f5851600640 1 -- 172.21.15.63:0/2254083959 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5618ebb17ae0 con 0x5618eb7c0d80 2024-05-27T08:12:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.519+0000 7f5851600640 10 client.22899 handle_command_reply: tid=0 2024-05-27T08:12:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.519+0000 7f5875ba8740 1 client.22899 shutdown 2024-05-27T08:12:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 1 -- 172.21.15.63:0/2254083959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 msgr2=0x5618ebbbac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbbac00 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f5860011ce0 tx=0x7f5860004a40 comp rx=0 tx=0).stop 2024-05-27T08:12:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 1 -- 172.21.15.63:0/2254083959 shutdown_connections 2024-05-27T08:12:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5618eb7c0d80 0x5618ebbc0cd0 crc :-1 s=READY pgs=694 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbbac00 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7e14d0 0x5618ebbaf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 1 --2- 172.21.15.63:0/2254083959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7d92f0 0x5618ebbaec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 1 -- 172.21.15.63:0/2254083959 >> 172.21.15.63:0/2254083959 conn(0x5618eb7f6300 msgr2=0x5618ebba4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5851600640 0 client.22899 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 1 -- 172.21.15.63:0/2254083959 shutdown_connections 2024-05-27T08:12:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 1 -- 172.21.15.63:0/2254083959 wait complete. 2024-05-27T08:12:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.520+0000 7f5875ba8740 20 client.22899 trim_cache size 0 max 16384 2024-05-27T08:12:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.537+0000 7f5875ba8740 1 Processor -- start 2024-05-27T08:12:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.537+0000 7f5875ba8740 1 -- start start 2024-05-27T08:12:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebbb1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebbba9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb8021b0 0x5618ebbbaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5875ba8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b5b70 con 0x5618eb7d92f0 2024-05-27T08:12:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5875ba8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b3440 con 0x5618eb8021b0 2024-05-27T08:12:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5875ba8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b12b0 con 0x5618eb7e14d0 2024-05-27T08:12:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5867e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebbb1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5867e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebbb1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38194/0 (socket says 172.21.15.63:38194) 2024-05-27T08:12:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5867e00640 1 -- 172.21.15.63:0/3490199160 learned_addr learned my addr 172.21.15.63:0/3490199160 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5867400640 1 --2- 172.21.15.63:0/3490199160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebbba9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f586cc00640 1 --2- 172.21.15.63:0/3490199160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb8021b0 0x5618ebbbaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5867e00640 1 -- 172.21.15.63:0/3490199160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 msgr2=0x5618ebbba9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5867e00640 1 --2- 172.21.15.63:0/3490199160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebbba9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5867e00640 1 -- 172.21.15.63:0/3490199160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb8021b0 msgr2=0x5618ebbbaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5867e00640 1 --2- 172.21.15.63:0/3490199160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb8021b0 0x5618ebbbaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5867e00640 1 -- 172.21.15.63:0/3490199160 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618ebb9c230 con 0x5618eb7d92f0 2024-05-27T08:12:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.538+0000 7f5867400640 1 --2- 172.21.15.63:0/3490199160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebbba9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:12:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.539+0000 7f5867e00640 1 --2- 172.21.15.63:0/3490199160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebbb1390 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f58580c9ab0 tx=0x7f585802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedfbdb30ac335e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.539+0000 7f5853e00640 1 -- 172.21.15.63:0/3490199160 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858049040 con 0x5618eb7d92f0 2024-05-27T08:12:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.539+0000 7f5853e00640 1 -- 172.21.15.63:0/3490199160 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5858110040 con 0x5618eb7d92f0 2024-05-27T08:12:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.539+0000 7f5853e00640 1 -- 172.21.15.63:0/3490199160 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858003e40 con 0x5618eb7d92f0 2024-05-27T08:12:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.540+0000 7f5875ba8740 1 -- 172.21.15.63:0/3490199160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 msgr2=0x5618ebbb1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.540+0000 7f5875ba8740 1 --2- 172.21.15.63:0/3490199160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebbb1390 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f58580c9ab0 tx=0x7f585802a410 comp rx=0 tx=0).stop 2024-05-27T08:12:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.540+0000 7f5875ba8740 1 -- 172.21.15.63:0/3490199160 shutdown_connections 2024-05-27T08:12:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.540+0000 7f5875ba8740 1 --2- 172.21.15.63:0/3490199160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb8021b0 0x5618ebbbaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.540+0000 7f5875ba8740 1 --2- 172.21.15.63:0/3490199160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebbba9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.540+0000 7f5875ba8740 1 --2- 172.21.15.63:0/3490199160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb7d92f0 0x5618ebbb1390 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.540+0000 7f5875ba8740 1 -- 172.21.15.63:0/3490199160 >> 172.21.15.63:0/3490199160 conn(0x5618eb7f6300 msgr2=0x5618ebbaef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.540+0000 7f5875ba8740 1 -- 172.21.15.63:0/3490199160 shutdown_connections 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.540+0000 7f5875ba8740 1 -- 172.21.15.63:0/3490199160 wait complete. 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.541+0000 7f5875ba8740 1 -- start start 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.541+0000 7f5875ba8740 1 Processor -- start 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.542+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7d92f0 0x5618ebbb6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.542+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebbb71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.542+0000 7f5875ba8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbb40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.542+0000 7f5875ba8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b1900 con 0x5618eb8021b0 2024-05-27T08:12:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.542+0000 7f5875ba8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b2d70 con 0x5618eb7d92f0 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.542+0000 7f5875ba8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f58681b1fd0 con 0x5618eb7e14d0 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.542+0000 7f5875ba8740 10 client.0 _ll_register_callbacks cb 0x7f5860009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.542+0000 7f586cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbb40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.542+0000 7f586cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbb40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38204/0 (socket says 172.21.15.63:38204) 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.542+0000 7f586cc00640 1 -- 172.21.15.63:0/1636603579 learned_addr learned my addr 172.21.15.63:0/1636603579 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.543+0000 7f5867400640 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebbb71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.543+0000 7f5867e00640 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7d92f0 0x5618ebbb6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.543+0000 7f586cc00640 1 -- 172.21.15.63:0/1636603579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 msgr2=0x5618ebbb71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.543+0000 7f586cc00640 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebbb71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.543+0000 7f586cc00640 1 -- 172.21.15.63:0/1636603579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7d92f0 msgr2=0x5618ebbb6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.543+0000 7f586cc00640 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7d92f0 0x5618ebbb6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.543+0000 7f586cc00640 1 -- 172.21.15.63:0/1636603579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618ebbb47e0 con 0x5618eb8021b0 2024-05-27T08:12:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.543+0000 7f586cc00640 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbb40f0 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f5860018040 tx=0x7f586001fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d65f2e8caf17ad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.543+0000 7f5851600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.543+0000 7f5851600640 1 -- 172.21.15.63:0/1636603579 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586003c060 con 0x5618eb8021b0 2024-05-27T08:12:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.544+0000 7f5875ba8740 1 -- 172.21.15.63:0/1636603579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5858110040 con 0x5618eb8021b0 2024-05-27T08:12:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.544+0000 7f5851600640 1 -- 172.21.15.63:0/1636603579 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f5860004cb0 con 0x5618eb8021b0 2024-05-27T08:12:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.544+0000 7f5851600640 1 -- 172.21.15.63:0/1636603579 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5860043050 con 0x5618eb8021b0 2024-05-27T08:12:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.544+0000 7f5875ba8740 10 client.22905 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.544+0000 7f5875ba8740 1 -- 172.21.15.63:0/1636603579 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5618ebc28c90 con 0x5618eb8021b0 2024-05-27T08:12:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.544+0000 7f5851600640 1 -- 172.21.15.63:0/1636603579 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5858110040 con 0x5618eb8021b0 2024-05-27T08:12:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.545+0000 7f5851600640 1 -- 172.21.15.63:0/1636603579 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f58600053a0 con 0x5618eb8021b0 2024-05-27T08:12:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.545+0000 7f5875ba8740 10 client.22905 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.545+0000 7f5875ba8740 10 client.22905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.545+0000 7f5875ba8740 20 client.22905 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.545+0000 7f5875ba8740 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5618eb7961c0 0x5618ebc2b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.545+0000 7f5875ba8740 4 client.22905 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:12:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.545+0000 7f5875ba8740 1 -- 172.21.15.63:0/1636603579 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618ebba9220 con 0x5618eb7961c0 2024-05-27T08:12:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.545+0000 7f5867e00640 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5618eb7961c0 0x5618ebc2b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.546+0000 7f5851600640 10 client.22905 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.546+0000 7f5867e00640 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5618eb7961c0 0x5618ebc2b3f0 crc :-1 s=READY pgs=695 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5281dc204d2f9e20 server_cookie=dffe6fedf2d69b0c in_seq=0 out_seq=0 2024-05-27T08:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5851600640 1 -- 172.21.15.63:0/1636603579 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5618ebba9220 con 0x5618eb7961c0 2024-05-27T08:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5851600640 10 client.22905 handle_command_reply: tid=0 2024-05-27T08:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 client.22905 shutdown 2024-05-27T08:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 -- 172.21.15.63:0/1636603579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 msgr2=0x5618ebbb40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbb40f0 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f5860018040 tx=0x7f586001fee0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 -- 172.21.15.63:0/1636603579 shutdown_connections 2024-05-27T08:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5618eb7961c0 0x5618ebc2b3f0 crc :-1 s=READY pgs=695 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618eb8021b0 0x5618ebbb40f0 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618eb7e14d0 0x5618ebbb71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 --2- 172.21.15.63:0/1636603579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618eb7d92f0 0x5618ebbb6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5851600640 0 client.22905 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 -- 172.21.15.63:0/1636603579 >> 172.21.15.63:0/1636603579 conn(0x5618eb7f6300 msgr2=0x5618ebba93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 -- 172.21.15.63:0/1636603579 shutdown_connections 2024-05-27T08:12:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.583+0000 7f5875ba8740 1 -- 172.21.15.63:0/1636603579 wait complete. 2024-05-27T08:12:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.584+0000 7f5875ba8740 20 client.22905 trim_cache size 0 max 16384 2024-05-27T08:12:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:12:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:12:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:12:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.584+0000 7f586d600640 1 -- 172.21.15.63:0/327143180 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f584c0b29d0 msgr2=0x7f584c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.584+0000 7f586d600640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f584c0b29d0 0x7f584c0b4ee0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x5618ebba1bd0 tx=0x7f5860009800 comp rx=0 tx=0).stop 2024-05-27T08:12:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.584+0000 7f586d600640 1 -- 172.21.15.63:0/327143180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5868104250 msgr2=0x7f586819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.584+0000 7f586d600640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5868104250 0x7f586819d5f0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f5858005940 tx=0x7f58580042f0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.585+0000 7f586d600640 1 -- 172.21.15.63:0/327143180 shutdown_connections 2024-05-27T08:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.585+0000 7f586d600640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f584c0b29d0 0x7f584c0b4ee0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.585+0000 7f586d600640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5868109f50 0x7f58681a8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.585+0000 7f586d600640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5868109ab0 0x7f586819db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.585+0000 7f586d600640 1 --2- 172.21.15.63:0/327143180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5868104250 0x7f586819d5f0 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.585+0000 7f586d600640 1 -- 172.21.15.63:0/327143180 >> 172.21.15.63:0/327143180 conn(0x7f5868079240 msgr2=0x7f5868102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.585+0000 7f586d600640 1 -- 172.21.15.63:0/327143180 shutdown_connections 2024-05-27T08:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:42.585+0000 7f586d600640 1 -- 172.21.15.63:0/327143180 wait complete. 2024-05-27T08:12:43.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.468+0000 7f02afe00700 1 -- 172.21.15.63:0/735431156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b0110a20 msgr2=0x7f02b0112e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:43.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.468+0000 7f02afe00700 1 --2- 172.21.15.63:0/735431156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b0110a20 0x7f02b0112e40 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b0a0 tx=0x7f02a800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.469+0000 7f02afe00700 1 -- 172.21.15.63:0/735431156 shutdown_connections 2024-05-27T08:12:43.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.469+0000 7f02afe00700 1 --2- 172.21.15.63:0/735431156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b0110a20 0x7f02b0112e40 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.469+0000 7f02afe00700 1 --2- 172.21.15.63:0/735431156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0075ad0 0x7f02b0075ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.469+0000 7f02afe00700 1 --2- 172.21.15.63:0/735431156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b0076420 0x7f02b01104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.469+0000 7f02afe00700 1 -- 172.21.15.63:0/735431156 >> 172.21.15.63:0/735431156 conn(0x7f02b0079d00 msgr2=0x7f02b007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:43.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.469+0000 7f02afe00700 1 -- 172.21.15.63:0/735431156 shutdown_connections 2024-05-27T08:12:43.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.469+0000 7f02afe00700 1 -- 172.21.15.63:0/735431156 wait complete. 2024-05-27T08:12:43.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.470+0000 7f02afe00700 1 Processor -- start 2024-05-27T08:12:43.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.471+0000 7f02afe00700 1 -- start start 2024-05-27T08:12:43.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.471+0000 7f02afe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b0075ad0 0x7f02b019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.471+0000 7f02afe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b0076420 0x7f02b019d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.471+0000 7f02afe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0110a20 0x7f02b01a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.471+0000 7f02afe00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01163b0 con 0x7f02b0075ad0 2024-05-27T08:12:43.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.471+0000 7f02afe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02b0116230 con 0x7f02b0110a20 2024-05-27T08:12:43.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.471+0000 7f02afe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02b0116530 con 0x7f02b0076420 2024-05-27T08:12:43.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.472+0000 7f02aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b0075ad0 0x7f02b019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.472+0000 7f02aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b0075ad0 0x7f02b019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38216/0 (socket says 172.21.15.63:38216) 2024-05-27T08:12:43.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.472+0000 7f02aea00700 1 -- 172.21.15.63:0/54959531 learned_addr learned my addr 172.21.15.63:0/54959531 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:43.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.472+0000 7f02ae000700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b0076420 0x7f02b019d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.472+0000 7f02af400700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0110a20 0x7f02b01a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.472+0000 7f02aea00700 1 -- 172.21.15.63:0/54959531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b0076420 msgr2=0x7f02b019d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:43.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.472+0000 7f02aea00700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b0076420 0x7f02b019d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.472+0000 7f02aea00700 1 -- 172.21.15.63:0/54959531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0110a20 msgr2=0x7f02b01a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:43.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.472+0000 7f02aea00700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0110a20 0x7f02b01a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.472+0000 7f02aea00700 1 -- 172.21.15.63:0/54959531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b01a1cb0 con 0x7f02b0075ad0 2024-05-27T08:12:43.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.474+0000 7f02aea00700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b0075ad0 0x7f02b019cde0 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f02a000d950 tx=0x7f02a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:43.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.474+0000 7f02a7400700 1 -- 172.21.15.63:0/54959531 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0016070 con 0x7f02b0075ad0 2024-05-27T08:12:43.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.474+0000 7f02a7400700 1 -- 172.21.15.63:0/54959531 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02a0009dd0 con 0x7f02b0075ad0 2024-05-27T08:12:43.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.474+0000 7f02afe00700 1 -- 172.21.15.63:0/54959531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b01a1fa0 con 0x7f02b0075ad0 2024-05-27T08:12:43.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.474+0000 7f02a7400700 1 -- 172.21.15.63:0/54959531 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a000f3c0 con 0x7f02b0075ad0 2024-05-27T08:12:43.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.474+0000 7f02afe00700 1 -- 172.21.15.63:0/54959531 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b01a98c0 con 0x7f02b0075ad0 2024-05-27T08:12:43.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.479+0000 7f02a7400700 1 -- 172.21.15.63:0/54959531 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f02a0019070 con 0x7f02b0075ad0 2024-05-27T08:12:43.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.479+0000 7f02c44bab80 1 Processor -- start 2024-05-27T08:12:43.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.480+0000 7f02c44bab80 1 -- start start 2024-05-27T08:12:43.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.480+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06dbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.480+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.480+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.480+0000 7f02c44bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b4c060aec0 con 0x55b4c06dc490 2024-05-27T08:12:43.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.480+0000 7f02c44bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b4c05faa60 con 0x55b4c06dc8a0 2024-05-27T08:12:43.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.480+0000 7f02c44bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b4c05feaa0 con 0x55b4c055dec0 2024-05-27T08:12:43.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.481+0000 7f02aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.481+0000 7f02aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38222/0 (socket says 172.21.15.63:38222) 2024-05-27T08:12:43.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.481+0000 7f02aea00700 1 -- 172.21.15.63:0/1343465060 learned_addr learned my addr 172.21.15.63:0/1343465060 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:43.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.481+0000 7f02af400700 1 --2- 172.21.15.63:0/1343465060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06dbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.481+0000 7f02ae000700 1 --2- 172.21.15.63:0/1343465060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.481+0000 7f02a7400700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02940b2a70 0x7f02940b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.482+0000 7f02a7400700 1 -- 172.21.15.63:0/54959531 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f02a00e2a00 con 0x7f02b0075ad0 2024-05-27T08:12:43.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.482+0000 7f02af400700 1 -- 172.21.15.63:0/1343465060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 msgr2=0x55b4c06e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:43.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.482+0000 7f02af400700 1 --2- 172.21.15.63:0/1343465060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.482+0000 7f02af400700 1 -- 172.21.15.63:0/1343465060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 msgr2=0x55b4c06e6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:43.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.482+0000 7f02af400700 1 --2- 172.21.15.63:0/1343465060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.482+0000 7f02af400700 1 -- 172.21.15.63:0/1343465060 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4c05c34a0 con 0x55b4c055dec0 2024-05-27T08:12:43.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.482+0000 7f02aea00700 1 --2- 172.21.15.63:0/1343465060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:12:43.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.482+0000 7f02af400700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02940b2a70 0x7f02940b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.482+0000 7f02af400700 1 --2- 172.21.15.63:0/1343465060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06dbf50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009fd0 tx=0x7f02a8009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f0c165e68f29d61 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:43.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.483+0000 7f029be00700 1 -- 172.21.15.63:0/1343465060 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8024070 con 0x55b4c055dec0 2024-05-27T08:12:43.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.483+0000 7f02af400700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02940b2a70 0x7f02940b4f80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f02a802a000 tx=0x7f02a8027000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:43.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.483+0000 7f029be00700 1 -- 172.21.15.63:0/1343465060 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02a8004e30 con 0x55b4c055dec0 2024-05-27T08:12:43.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.483+0000 7f029be00700 1 -- 172.21.15.63:0/1343465060 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a801e7c0 con 0x55b4c055dec0 2024-05-27T08:12:43.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.484+0000 7f02c44bab80 1 -- 172.21.15.63:0/1343465060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 msgr2=0x55b4c06dbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:43.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.484+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1343465060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06dbf50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009fd0 tx=0x7f02a8009e00 comp rx=0 tx=0).stop 2024-05-27T08:12:43.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.484+0000 7f02c44bab80 1 -- 172.21.15.63:0/1343465060 shutdown_connections 2024-05-27T08:12:43.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.484+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1343465060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.484+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1343465060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06dbf50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.484+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1343465060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.484+0000 7f02c44bab80 1 -- 172.21.15.63:0/1343465060 >> 172.21.15.63:0/1343465060 conn(0x55b4c05be040 msgr2=0x55b4c06d7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:43.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.485+0000 7f02c44bab80 1 -- 172.21.15.63:0/1343465060 shutdown_connections 2024-05-27T08:12:43.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.485+0000 7f02c44bab80 1 -- 172.21.15.63:0/1343465060 wait complete. 2024-05-27T08:12:43.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.485+0000 7f02c44bab80 1 -- start start 2024-05-27T08:12:43.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.486+0000 7f02c44bab80 1 Processor -- start 2024-05-27T08:12:43.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.486+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06e6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.486+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.486+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.486+0000 7f02c44bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b4c0501020 con 0x55b4c06dc490 2024-05-27T08:12:43.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.486+0000 7f02c44bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b4c059b7e0 con 0x55b4c06dc8a0 2024-05-27T08:12:43.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.486+0000 7f02c44bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b4c06e9a10 con 0x55b4c055dec0 2024-05-27T08:12:43.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.486+0000 7f02c44bab80 10 client.0 _ll_register_callbacks cb 0x55b4c05a4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:43.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.487+0000 7f02ae000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.487+0000 7f02ae000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38230/0 (socket says 172.21.15.63:38230) 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.487+0000 7f02ae000700 1 -- 172.21.15.63:0/1395864550 learned_addr learned my addr 172.21.15.63:0/1395864550 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.487+0000 7f02aea00700 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.487+0000 7f02ae000700 1 -- 172.21.15.63:0/1395864550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 msgr2=0x55b4c06e6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.487+0000 7f02ae000700 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06e6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.487+0000 7f02ae000700 1 -- 172.21.15.63:0/1395864550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 msgr2=0x55b4c06e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.487+0000 7f02ae000700 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.487+0000 7f02ae000700 1 -- 172.21.15.63:0/1395864550 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4c054a950 con 0x55b4c06dc490 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.488+0000 7f02ae000700 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6bd0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f029c012700 tx=0x7f029c012ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735875f51bf03661 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.488+0000 7f0298c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:43.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.488+0000 7f0298c00700 1 -- 172.21.15.63:0/1395864550 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c004830 con 0x55b4c06dc490 2024-05-27T08:12:43.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.488+0000 7f02c44bab80 1 -- 172.21.15.63:0/1395864550 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02a8004e30 con 0x55b4c06dc490 2024-05-27T08:12:43.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.488+0000 7f0298c00700 1 -- 172.21.15.63:0/1395864550 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f029c0049d0 con 0x55b4c06dc490 2024-05-27T08:12:43.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.488+0000 7f0298c00700 1 -- 172.21.15.63:0/1395864550 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f029c025050 con 0x55b4c06dc490 2024-05-27T08:12:43.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.488+0000 7f02c44bab80 10 client.22917 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:43.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.489+0000 7f02c44bab80 1 -- 172.21.15.63:0/1395864550 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b4c05cb0e0 con 0x55b4c06dc490 2024-05-27T08:12:43.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.489+0000 7f0298c00700 1 -- 172.21.15.63:0/1395864550 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02a8004e30 con 0x55b4c06dc490 2024-05-27T08:12:43.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.489+0000 7f0298c00700 1 -- 172.21.15.63:0/1395864550 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f029c01c730 con 0x55b4c06dc490 2024-05-27T08:12:43.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.489+0000 7f02c44bab80 10 client.22917 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:43.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.489+0000 7f02c44bab80 10 client.22917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:43.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.489+0000 7f02c44bab80 20 client.22917 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:43.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.489+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b4c06f6ae0 0x55b4c06f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.489+0000 7f02c44bab80 4 client.22917 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:43.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.489+0000 7f02c44bab80 1 -- 172.21.15.63:0/1395864550 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4c060ce10 con 0x55b4c06f6ae0 2024-05-27T08:12:43.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.490+0000 7f02aea00700 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b4c06f6ae0 0x55b4c06f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.491+0000 7f02aea00700 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b4c06f6ae0 0x55b4c06f8f90 crc :-1 s=READY pgs=696 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1bed03205baaf9 server_cookie=e1d9692909ac1aa7 in_seq=0 out_seq=0 2024-05-27T08:12:43.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.491+0000 7f0298c00700 10 client.22917 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:43.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f0298c00700 1 -- 172.21.15.63:0/1395864550 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b4c060ce10 con 0x55b4c06f6ae0 2024-05-27T08:12:43.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f0298c00700 10 client.22917 handle_command_reply: tid=0 2024-05-27T08:12:43.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f02c44bab80 1 client.22917 shutdown 2024-05-27T08:12:43.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f02c44bab80 1 -- 172.21.15.63:0/1395864550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 msgr2=0x55b4c06e6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:43.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6bd0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f029c012700 tx=0x7f029c012ac0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f02c44bab80 1 -- 172.21.15.63:0/1395864550 shutdown_connections 2024-05-27T08:12:43.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b4c06f6ae0 0x55b4c06f8f90 crc :-1 s=READY pgs=696 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1395864550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06e6bd0 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.498+0000 7f02c44bab80 1 -- 172.21.15.63:0/1395864550 >> 172.21.15.63:0/1395864550 conn(0x55b4c05be040 msgr2=0x55b4c06d7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:43.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.499+0000 7f0298c00700 0 client.22917 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:43.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.499+0000 7f02c44bab80 1 -- 172.21.15.63:0/1395864550 shutdown_connections 2024-05-27T08:12:43.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.499+0000 7f02c44bab80 1 -- 172.21.15.63:0/1395864550 wait complete. 2024-05-27T08:12:43.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.499+0000 7f02c44bab80 20 client.22917 trim_cache size 0 max 16384 2024-05-27T08:12:43.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.524+0000 7f02c44bab80 1 Processor -- start 2024-05-27T08:12:43.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.524+0000 7f02c44bab80 1 -- start start 2024-05-27T08:12:43.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.524+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06e7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.524+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06d7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.524+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.524+0000 7f02c44bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01b4e90 con 0x55b4c06dc490 2024-05-27T08:12:43.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.524+0000 7f02c44bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01b2720 con 0x55b4c06dc8a0 2024-05-27T08:12:43.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.524+0000 7f02c44bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01b0590 con 0x55b4c055dec0 2024-05-27T08:12:43.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.525+0000 7f02ae000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06d7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.525+0000 7f02ae000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06d7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38240/0 (socket says 172.21.15.63:38240) 2024-05-27T08:12:43.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.525+0000 7f02ae000700 1 -- 172.21.15.63:0/1619498627 learned_addr learned my addr 172.21.15.63:0/1619498627 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:43.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.525+0000 7f02aea00700 1 --2- 172.21.15.63:0/1619498627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.525+0000 7f02af400700 1 --2- 172.21.15.63:0/1619498627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.525+0000 7f02ae000700 1 -- 172.21.15.63:0/1619498627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 msgr2=0x55b4c06e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:43.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.525+0000 7f02ae000700 1 --2- 172.21.15.63:0/1619498627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.525+0000 7f02ae000700 1 -- 172.21.15.63:0/1619498627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 msgr2=0x55b4c06f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:43.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.525+0000 7f02ae000700 1 --2- 172.21.15.63:0/1619498627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.525+0000 7f02ae000700 1 -- 172.21.15.63:0/1619498627 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4c06f44b0 con 0x55b4c06dc490 2024-05-27T08:12:43.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.526+0000 7f02ae000700 1 --2- 172.21.15.63:0/1619498627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06d7eb0 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f029c012ec0 tx=0x7f029c01bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09f7371862d2542 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:43.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.526+0000 7f029b400700 1 -- 172.21.15.63:0/1619498627 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c01bea0 con 0x55b4c06dc490 2024-05-27T08:12:43.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.526+0000 7f029b400700 1 -- 172.21.15.63:0/1619498627 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f029c01b410 con 0x55b4c06dc490 2024-05-27T08:12:43.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.526+0000 7f029b400700 1 -- 172.21.15.63:0/1619498627 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c005330 con 0x55b4c06dc490 2024-05-27T08:12:43.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.527+0000 7f02c44bab80 1 -- 172.21.15.63:0/1619498627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 msgr2=0x55b4c06d7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:43.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.527+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1619498627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06d7eb0 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f029c012ec0 tx=0x7f029c01bcb0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.528+0000 7f02c44bab80 1 -- 172.21.15.63:0/1619498627 shutdown_connections 2024-05-27T08:12:43.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.528+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1619498627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.528+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1619498627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c055dec0 0x55b4c06e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.528+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1619498627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c06dc490 0x55b4c06d7eb0 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.528+0000 7f02c44bab80 1 -- 172.21.15.63:0/1619498627 >> 172.21.15.63:0/1619498627 conn(0x55b4c05be040 msgr2=0x55b4c06dd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:43.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.528+0000 7f02c44bab80 1 -- 172.21.15.63:0/1619498627 shutdown_connections 2024-05-27T08:12:43.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.528+0000 7f02c44bab80 1 -- 172.21.15.63:0/1619498627 wait complete. 2024-05-27T08:12:43.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.528+0000 7f02c44bab80 1 -- start start 2024-05-27T08:12:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.529+0000 7f02c44bab80 1 Processor -- start 2024-05-27T08:12:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.529+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c055dec0 0x55b4c06eeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.529+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c06dc490 0x55b4c06ef200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.529+0000 7f02c44bab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06eb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.529+0000 7f02c44bab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01b0be0 con 0x55b4c055dec0 2024-05-27T08:12:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.529+0000 7f02c44bab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01b2050 con 0x55b4c06dc8a0 2024-05-27T08:12:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.529+0000 7f02c44bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01b12b0 con 0x55b4c06dc490 2024-05-27T08:12:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.529+0000 7f02c44bab80 10 client.0 _ll_register_callbacks cb 0x55b4c06f9680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.530+0000 7f02aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c055dec0 0x55b4c06eeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.530+0000 7f02aea00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c055dec0 0x55b4c06eeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38250/0 (socket says 172.21.15.63:38250) 2024-05-27T08:12:43.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.530+0000 7f02aea00700 1 -- 172.21.15.63:0/1201542190 learned_addr learned my addr 172.21.15.63:0/1201542190 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:43.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.530+0000 7f02ae000700 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c06dc490 0x55b4c06ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.530+0000 7f02af400700 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.530+0000 7f02aea00700 1 -- 172.21.15.63:0/1201542190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c06dc490 msgr2=0x55b4c06ef200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:43.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.530+0000 7f02aea00700 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c06dc490 0x55b4c06ef200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.530+0000 7f02aea00700 1 -- 172.21.15.63:0/1201542190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 msgr2=0x55b4c06eb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:43.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.530+0000 7f02aea00700 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06eb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.530+0000 7f02aea00700 1 -- 172.21.15.63:0/1201542190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4c06d93d0 con 0x55b4c055dec0 2024-05-27T08:12:43.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.531+0000 7f02aea00700 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c055dec0 0x55b4c06eeca0 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f02a0005950 tx=0x7f02a00a5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50059347fbfbc37e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.531+0000 7f0298c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.531+0000 7f0298c00700 1 -- 172.21.15.63:0/1201542190 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0020040 con 0x55b4c055dec0 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.531+0000 7f0298c00700 1 -- 172.21.15.63:0/1201542190 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f02a00a5b50 con 0x55b4c055dec0 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.531+0000 7f02c44bab80 1 -- 172.21.15.63:0/1201542190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f029c01b410 con 0x55b4c055dec0 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.532+0000 7f0298c00700 1 -- 172.21.15.63:0/1201542190 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f02a00b2060 con 0x55b4c055dec0 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.532+0000 7f02c44bab80 10 client.22923 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.532+0000 7f02c44bab80 1 -- 172.21.15.63:0/1201542190 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b4c0761eb0 con 0x55b4c055dec0 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.532+0000 7f0298c00700 1 -- 172.21.15.63:0/1201542190 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f029c01b410 con 0x55b4c055dec0 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.532+0000 7f0298c00700 1 -- 172.21.15.63:0/1201542190 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f02a00a9b20 con 0x55b4c055dec0 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.532+0000 7f02c44bab80 10 client.22923 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:43.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.532+0000 7f02c44bab80 10 client.22923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:43.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.532+0000 7f02c44bab80 20 client.22923 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:43.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.532+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b4c0762570 0x55b4c0764990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:43.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.533+0000 7f02c44bab80 4 client.22923 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:12:43.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.533+0000 7f02c44bab80 1 -- 172.21.15.63:0/1201542190 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b4c06065f0 con 0x55b4c0762570 2024-05-27T08:12:43.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.533+0000 7f02ae000700 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b4c0762570 0x55b4c0764990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:43.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.534+0000 7f02ae000700 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b4c0762570 0x55b4c0764990 crc :-1 s=READY pgs=697 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b18640f5041386d server_cookie=4eda0b1b5eb79fee in_seq=0 out_seq=0 2024-05-27T08:12:43.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.534+0000 7f0298c00700 10 client.22923 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:43.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f0298c00700 1 -- 172.21.15.63:0/1201542190 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b4c06065f0 con 0x55b4c0762570 2024-05-27T08:12:43.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f0298c00700 10 client.22923 handle_command_reply: tid=0 2024-05-27T08:12:43.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f02c44bab80 1 client.22923 shutdown 2024-05-27T08:12:43.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f02c44bab80 1 -- 172.21.15.63:0/1201542190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c055dec0 msgr2=0x55b4c06eeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:43.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c055dec0 0x55b4c06eeca0 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f02a0005950 tx=0x7f02a00a5040 comp rx=0 tx=0).stop 2024-05-27T08:12:43.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f02c44bab80 1 -- 172.21.15.63:0/1201542190 shutdown_connections 2024-05-27T08:12:43.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b4c0762570 0x55b4c0764990 crc :-1 s=READY pgs=697 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b4c06dc8a0 0x55b4c06eb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b4c055dec0 0x55b4c06eeca0 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f02c44bab80 1 --2- 172.21.15.63:0/1201542190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4c06dc490 0x55b4c06ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f02c44bab80 1 -- 172.21.15.63:0/1201542190 >> 172.21.15.63:0/1201542190 conn(0x55b4c05be040 msgr2=0x55b4c06de230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:43.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.537+0000 7f0298c00700 0 client.22923 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:43.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.538+0000 7f02c44bab80 1 -- 172.21.15.63:0/1201542190 shutdown_connections 2024-05-27T08:12:43.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.538+0000 7f02c44bab80 1 -- 172.21.15.63:0/1201542190 wait complete. 2024-05-27T08:12:43.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.538+0000 7f02c44bab80 20 client.22923 trim_cache size 0 max 16384 2024-05-27T08:12:43.540 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:12:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:12:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "2d0e8131-7f69-4cf1-89fe-0bb8693e5ad0", 2024-05-27T08:12:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:12:43.541 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:12:43.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 -- 172.21.15.63:0/54959531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02940b2a70 msgr2=0x7f02940b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:43.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02940b2a70 0x7f02940b4f80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f02a802a000 tx=0x7f02a8027000 comp rx=0 tx=0).stop 2024-05-27T08:12:43.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 -- 172.21.15.63:0/54959531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b0075ad0 msgr2=0x7f02b019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:43.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b0075ad0 0x7f02b019cde0 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f02a000d950 tx=0x7f02a000dc60 comp rx=0 tx=0).stop 2024-05-27T08:12:43.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 -- 172.21.15.63:0/54959531 shutdown_connections 2024-05-27T08:12:43.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f02940b2a70 0x7f02940b4f80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0110a20 0x7f02b01a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02b0075ad0 0x7f02b019cde0 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 --2- 172.21.15.63:0/54959531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02b0076420 0x7f02b019d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:43.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 -- 172.21.15.63:0/54959531 >> 172.21.15.63:0/54959531 conn(0x7f02b0079d00 msgr2=0x7f02b0101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:43.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 -- 172.21.15.63:0/54959531 shutdown_connections 2024-05-27T08:12:43.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:43.539+0000 7f02afe00700 1 -- 172.21.15.63:0/54959531 wait complete. 2024-05-27T08:12:44.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:43 smithi086 ceph-mon[34687]: pgmap v1242: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.4 MiB/s wr, 15 op/s 2024-05-27T08:12:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:43 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:12:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:43 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:43 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:12:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:43 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:43 smithi149 ceph-mon[35408]: pgmap v1242: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.4 MiB/s wr, 15 op/s 2024-05-27T08:12:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:43 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:12:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:43 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:43 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:12:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:43 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:44.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:43 smithi063 ceph-mon[31613]: pgmap v1242: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.4 MiB/s wr, 15 op/s 2024-05-27T08:12:44.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:43 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:12:44.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:43 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:44.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:43 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:12:44.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:43 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:44.445 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:12:44.953 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:12:45.102 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:12:45.103 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:12:45.103 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.103 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.103 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:12:45.103 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.103 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:12:45.103 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.103 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:12:45.103 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.103 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:12:45.104 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:44 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:12:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:12:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:12:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:12:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:12:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:12:45.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:44 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:12:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:44 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:12:45.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:45 smithi063 ceph-mon[31613]: pgmap v1243: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 8 op/s 2024-05-27T08:12:46.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:45 smithi086 ceph-mon[34687]: pgmap v1243: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 8 op/s 2024-05-27T08:12:46.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:45 smithi149 ceph-mon[35408]: pgmap v1243: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.7 MiB/s wr, 8 op/s 2024-05-27T08:12:47.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.481+0000 7f73b1000700 1 -- 172.21.15.63:0/2861014060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac10af40 msgr2=0x7f73ac10b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:47.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.481+0000 7f73b1000700 1 --2- 172.21.15.63:0/2861014060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac10af40 0x7f73ac10b350 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f73940099c0 tx=0x7f7394009cd0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.481+0000 7f73b1000700 1 -- 172.21.15.63:0/2861014060 shutdown_connections 2024-05-27T08:12:47.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.481+0000 7f73b1000700 1 --2- 172.21.15.63:0/2861014060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73ac10b890 0x7f73ac1131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.481+0000 7f73b1000700 1 --2- 172.21.15.63:0/2861014060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ac106610 0x7f73ac106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.481+0000 7f73b1000700 1 --2- 172.21.15.63:0/2861014060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac10af40 0x7f73ac10b350 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.481+0000 7f73b1000700 1 -- 172.21.15.63:0/2861014060 >> 172.21.15.63:0/2861014060 conn(0x7f73ac102020 msgr2=0x7f73ac104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:47.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.482+0000 7f73b1000700 1 -- 172.21.15.63:0/2861014060 shutdown_connections 2024-05-27T08:12:47.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.482+0000 7f73b1000700 1 -- 172.21.15.63:0/2861014060 wait complete. 2024-05-27T08:12:47.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.483+0000 7f73b1000700 1 Processor -- start 2024-05-27T08:12:47.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.484+0000 7f73b1000700 1 -- start start 2024-05-27T08:12:47.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.484+0000 7f73b1000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73ac106610 0x7f73ac19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.484+0000 7f73b1000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac10af40 0x7f73ac19d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.484+0000 7f73b1000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ac10b890 0x7f73ac1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.484+0000 7f73b1000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1163a0 con 0x7f73ac10af40 2024-05-27T08:12:47.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.484+0000 7f73b1000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac116220 con 0x7f73ac10b890 2024-05-27T08:12:47.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.484+0000 7f73b1000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac116520 con 0x7f73ac106610 2024-05-27T08:12:47.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.485+0000 7f73a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac10af40 0x7f73ac19d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.485+0000 7f73a2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac10af40 0x7f73ac19d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38262/0 (socket says 172.21.15.63:38262) 2024-05-27T08:12:47.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.485+0000 7f73a2a00700 1 -- 172.21.15.63:0/2789928257 learned_addr learned my addr 172.21.15.63:0/2789928257 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:47.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.485+0000 7f73ab400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73ac106610 0x7f73ac19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.485+0000 7f73abe00700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ac10b890 0x7f73ac1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.486+0000 7f73a2a00700 1 -- 172.21.15.63:0/2789928257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73ac106610 msgr2=0x7f73ac19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:47.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.486+0000 7f73a2a00700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73ac106610 0x7f73ac19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.486+0000 7f73a2a00700 1 -- 172.21.15.63:0/2789928257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ac10b890 msgr2=0x7f73ac1a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:47.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.486+0000 7f73a2a00700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ac10b890 0x7f73ac1a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.486+0000 7f73a2a00700 1 -- 172.21.15.63:0/2789928257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac1a1c80 con 0x7f73ac10af40 2024-05-27T08:12:47.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.486+0000 7f73a2a00700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac10af40 0x7f73ac19d350 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f73940098e0 tx=0x7f73940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:47.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.486+0000 7f73a8c00700 1 -- 172.21.15.63:0/2789928257 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394027070 con 0x7f73ac10af40 2024-05-27T08:12:47.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.487+0000 7f73b1000700 1 -- 172.21.15.63:0/2789928257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac1a1f10 con 0x7f73ac10af40 2024-05-27T08:12:47.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.487+0000 7f73a8c00700 1 -- 172.21.15.63:0/2789928257 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f739400fc00 con 0x7f73ac10af40 2024-05-27T08:12:47.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.487+0000 7f73a8c00700 1 -- 172.21.15.63:0/2789928257 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394022ca0 con 0x7f73ac10af40 2024-05-27T08:12:47.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.487+0000 7f73b1000700 1 -- 172.21.15.63:0/2789928257 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac1a9890 con 0x7f73ac10af40 2024-05-27T08:12:47.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.492+0000 7f73a8c00700 1 -- 172.21.15.63:0/2789928257 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f739402a070 con 0x7f73ac10af40 2024-05-27T08:12:47.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.493+0000 7f73c10d6b80 1 Processor -- start 2024-05-27T08:12:47.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.493+0000 7f73c10d6b80 1 -- start start 2024-05-27T08:12:47.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.493+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 0x55834777c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.493+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583475fce40 0x55834777cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.493+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x5583477886e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.494+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5583476a9e40 con 0x5583477862c0 2024-05-27T08:12:47.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.494+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5583476999e0 con 0x5583475fce40 2024-05-27T08:12:47.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.494+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55834769da20 con 0x5583476e89f0 2024-05-27T08:12:47.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.494+0000 7f73ab400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583475fce40 0x55834777cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.494+0000 7f73abe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 0x55834777c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.494+0000 7f73ab400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583475fce40 0x55834777cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:58300/0 (socket says 172.21.15.63:58300) 2024-05-27T08:12:47.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.494+0000 7f73ab400700 1 -- 172.21.15.63:0/2581629863 learned_addr learned my addr 172.21.15.63:0/2581629863 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:47.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.494+0000 7f73a2a00700 1 --2- 172.21.15.63:0/2581629863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x5583477886e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.494+0000 7f73a8c00700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73900b2a70 0x7f73900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.495+0000 7f73a8c00700 1 -- 172.21.15.63:0/2789928257 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f73940e3860 con 0x7f73ac10af40 2024-05-27T08:12:47.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.495+0000 7f73abe00700 1 -- 172.21.15.63:0/2581629863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583475fce40 msgr2=0x55834777cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:47.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.495+0000 7f73abe00700 1 --2- 172.21.15.63:0/2581629863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583475fce40 0x55834777cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.495+0000 7f73abe00700 1 -- 172.21.15.63:0/2581629863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 msgr2=0x5583477886e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:47.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.495+0000 7f73abe00700 1 --2- 172.21.15.63:0/2581629863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x5583477886e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.495+0000 7f73abe00700 1 -- 172.21.15.63:0/2581629863 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558347662420 con 0x5583476e89f0 2024-05-27T08:12:47.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.495+0000 7f73abe00700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73900b2a70 0x7f73900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.496+0000 7f73abe00700 1 --2- 172.21.15.63:0/2581629863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 0x55834777c750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f73a4000b60 tx=0x7f73a400b970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf0983ab175a426 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:47.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.497+0000 7f73a0c00700 1 -- 172.21.15.63:0/2581629863 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4022070 con 0x5583476e89f0 2024-05-27T08:12:47.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.497+0000 7f73a0c00700 1 -- 172.21.15.63:0/2581629863 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73a4010440 con 0x5583476e89f0 2024-05-27T08:12:47.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.497+0000 7f73a0c00700 1 -- 172.21.15.63:0/2581629863 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4003a60 con 0x5583476e89f0 2024-05-27T08:12:47.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.498+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2581629863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 msgr2=0x55834777c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:47.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.498+0000 7f73abe00700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73900b2a70 0x7f73900b4f80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f73a401f000 tx=0x7f73a401c000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:47.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.498+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2581629863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 0x55834777c750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f73a4000b60 tx=0x7f73a400b970 comp rx=0 tx=0).stop 2024-05-27T08:12:47.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.498+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2581629863 shutdown_connections 2024-05-27T08:12:47.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.498+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2581629863 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x5583477886e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.498+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2581629863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 0x55834777c750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.498+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2581629863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583475fce40 0x55834777cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.498+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2581629863 >> 172.21.15.63:0/2581629863 conn(0x558347696390 msgr2=0x558347777710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:47.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.498+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2581629863 shutdown_connections 2024-05-27T08:12:47.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.499+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2581629863 wait complete. 2024-05-27T08:12:47.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.499+0000 7f73c10d6b80 1 -- start start 2024-05-27T08:12:47.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.499+0000 7f73c10d6b80 1 Processor -- start 2024-05-27T08:12:47.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.500+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583475fce40 0x5583477831f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.500+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 0x558347783730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.500+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583477862c0 0x558347783c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.500+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55834759ffa0 con 0x5583475fce40 2024-05-27T08:12:47.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.500+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55834763a760 con 0x5583477862c0 2024-05-27T08:12:47.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.500+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558347788e60 con 0x5583476e89f0 2024-05-27T08:12:47.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.500+0000 7f73c10d6b80 10 client.0 _ll_register_callbacks cb 0x558347643c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:47.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73ab400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583475fce40 0x5583477831f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73ab400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583475fce40 0x5583477831f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38280/0 (socket says 172.21.15.63:38280) 2024-05-27T08:12:47.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73ab400700 1 -- 172.21.15.63:0/2477376916 learned_addr learned my addr 172.21.15.63:0/2477376916 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:47.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73a2a00700 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 0x558347783730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73abe00700 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583477862c0 0x558347783c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73ab400700 1 -- 172.21.15.63:0/2477376916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 msgr2=0x558347783730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:47.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73ab400700 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 0x558347783730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73ab400700 1 -- 172.21.15.63:0/2477376916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583477862c0 msgr2=0x558347783c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:47.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73ab400700 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583477862c0 0x558347783c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73ab400700 1 -- 172.21.15.63:0/2477376916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583475e98d0 con 0x5583475fce40 2024-05-27T08:12:47.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.501+0000 7f73abe00700 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583477862c0 0x558347783c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:12:47.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.502+0000 7f73ab400700 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583475fce40 0x5583477831f0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f739c0042a0 tx=0x7f739c004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abfb6ffb611fe38 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:47.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.502+0000 7f7399600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:47.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.502+0000 7f7399600700 1 -- 172.21.15.63:0/2477376916 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c01a070 con 0x5583475fce40 2024-05-27T08:12:47.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.502+0000 7f7399600700 1 -- 172.21.15.63:0/2477376916 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f739c004db0 con 0x5583475fce40 2024-05-27T08:12:47.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.502+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2477376916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a4010440 con 0x5583475fce40 2024-05-27T08:12:47.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.503+0000 7f7399600700 1 -- 172.21.15.63:0/2477376916 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f739c025050 con 0x5583475fce40 2024-05-27T08:12:47.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.503+0000 7f73c10d6b80 10 client.22935 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:47.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.503+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2477376916 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55834766a060 con 0x5583475fce40 2024-05-27T08:12:47.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.503+0000 7f7399600700 1 -- 172.21.15.63:0/2477376916 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73a4010440 con 0x5583475fce40 2024-05-27T08:12:47.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.504+0000 7f7399600700 1 -- 172.21.15.63:0/2477376916 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f739c0167a0 con 0x5583475fce40 2024-05-27T08:12:47.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.504+0000 7f73c10d6b80 10 client.22935 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:47.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.504+0000 7f73c10d6b80 10 client.22935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:47.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.504+0000 7f73c10d6b80 20 client.22935 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:47.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.504+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558347795d60 0x558347798230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.504+0000 7f73c10d6b80 4 client.22935 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:47.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.504+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2477376916 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583476abd90 con 0x558347795d60 2024-05-27T08:12:47.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.505+0000 7f73a2a00700 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558347795d60 0x558347798230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.506+0000 7f73a2a00700 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558347795d60 0x558347798230 crc :-1 s=READY pgs=698 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef845286f27eb7d8 server_cookie=53f66c8bdec86131 in_seq=0 out_seq=0 2024-05-27T08:12:47.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.506+0000 7f7399600700 10 client.22935 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:47.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f7399600700 1 -- 172.21.15.63:0/2477376916 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5583476abd90 con 0x558347795d60 2024-05-27T08:12:47.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f7399600700 10 client.22935 handle_command_reply: tid=0 2024-05-27T08:12:47.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f73c10d6b80 1 client.22935 shutdown 2024-05-27T08:12:47.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2477376916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583475fce40 msgr2=0x5583477831f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:47.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583475fce40 0x5583477831f0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f739c0042a0 tx=0x7f739c004380 comp rx=0 tx=0).stop 2024-05-27T08:12:47.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2477376916 shutdown_connections 2024-05-27T08:12:47.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558347795d60 0x558347798230 crc :-1 s=READY pgs=698 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583477862c0 0x558347783c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583475fce40 0x5583477831f0 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2477376916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583476e89f0 0x558347783730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2477376916 >> 172.21.15.63:0/2477376916 conn(0x558347696390 msgr2=0x55834777c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:47.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.508+0000 7f7399600700 0 client.22935 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:47.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.509+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2477376916 shutdown_connections 2024-05-27T08:12:47.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.509+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2477376916 wait complete. 2024-05-27T08:12:47.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.509+0000 7f73c10d6b80 20 client.22935 trim_cache size 0 max 16384 2024-05-27T08:12:47.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.557+0000 7f73c10d6b80 1 Processor -- start 2024-05-27T08:12:47.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.557+0000 7f73c10d6b80 1 -- start start 2024-05-27T08:12:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.558+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 0x558347777740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.558+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583476e89f0 0x558347791f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.558+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x55834779d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.558+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1b4ca0 con 0x5583477862c0 2024-05-27T08:12:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.558+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1b2530 con 0x5583476e89f0 2024-05-27T08:12:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.558+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1b03a0 con 0x5583475fce40 2024-05-27T08:12:47.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.559+0000 7f73abe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x55834779d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.559+0000 7f73abe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x55834779d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38294/0 (socket says 172.21.15.63:38294) 2024-05-27T08:12:47.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.559+0000 7f73abe00700 1 -- 172.21.15.63:0/2078115401 learned_addr learned my addr 172.21.15.63:0/2078115401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:47.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.559+0000 7f73ab400700 1 --2- 172.21.15.63:0/2078115401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 0x558347777740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.560+0000 7f73abe00700 1 -- 172.21.15.63:0/2078115401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 msgr2=0x558347777740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:47.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.560+0000 7f73abe00700 1 --2- 172.21.15.63:0/2078115401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 0x558347777740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.560+0000 7f73abe00700 1 -- 172.21.15.63:0/2078115401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583476e89f0 msgr2=0x558347791f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:47.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.560+0000 7f73abe00700 1 --2- 172.21.15.63:0/2078115401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583476e89f0 0x558347791f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.560+0000 7f73abe00700 1 -- 172.21.15.63:0/2078115401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558347781760 con 0x5583477862c0 2024-05-27T08:12:47.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.560+0000 7f73abe00700 1 --2- 172.21.15.63:0/2078115401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x55834779d9f0 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f73a4004640 tx=0x7f73a4025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3745da81390229ee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:47.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.561+0000 7f739be00700 1 -- 172.21.15.63:0/2078115401 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4022040 con 0x5583477862c0 2024-05-27T08:12:47.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.561+0000 7f739be00700 1 -- 172.21.15.63:0/2078115401 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73a4013c50 con 0x5583477862c0 2024-05-27T08:12:47.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.561+0000 7f739be00700 1 -- 172.21.15.63:0/2078115401 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a40116a0 con 0x5583477862c0 2024-05-27T08:12:47.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.562+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2078115401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 msgr2=0x55834779d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:47.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.562+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2078115401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x55834779d9f0 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f73a4004640 tx=0x7f73a4025000 comp rx=0 tx=0).stop 2024-05-27T08:12:47.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.562+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2078115401 shutdown_connections 2024-05-27T08:12:47.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.562+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2078115401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x55834779d9f0 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.562+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2078115401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 0x558347777740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.562+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/2078115401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583476e89f0 0x558347791f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.562+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2078115401 >> 172.21.15.63:0/2078115401 conn(0x558347696390 msgr2=0x55834777e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:47.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.563+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2078115401 shutdown_connections 2024-05-27T08:12:47.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.563+0000 7f73c10d6b80 1 -- 172.21.15.63:0/2078115401 wait complete. 2024-05-27T08:12:47.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.563+0000 7f73c10d6b80 1 -- start start 2024-05-27T08:12:47.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.564+0000 7f73c10d6b80 1 Processor -- start 2024-05-27T08:12:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.564+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 0x5583477896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.564+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583476e89f0 0x558347789c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.564+0000 7f73c10d6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x5583478070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.564+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1b09f0 con 0x5583477862c0 2024-05-27T08:12:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.564+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1b1e60 con 0x5583476e89f0 2024-05-27T08:12:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.564+0000 7f73c10d6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1b10c0 con 0x5583475fce40 2024-05-27T08:12:47.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.564+0000 7f73c10d6b80 10 client.0 _ll_register_callbacks cb 0x558347643c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:47.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.565+0000 7f73abe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x5583478070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.565+0000 7f73ab400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 0x5583477896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.565+0000 7f73a2a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583476e89f0 0x558347789c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.565+0000 7f73ab400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 0x5583477896e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:57966/0 (socket says 172.21.15.63:57966) 2024-05-27T08:12:47.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.565+0000 7f73ab400700 1 -- 172.21.15.63:0/319798476 learned_addr learned my addr 172.21.15.63:0/319798476 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:47.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.565+0000 7f73abe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x5583478070c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38296/0 (socket says 172.21.15.63:38296) 2024-05-27T08:12:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.566+0000 7f73ab400700 1 -- 172.21.15.63:0/319798476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583476e89f0 msgr2=0x558347789c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.566+0000 7f73ab400700 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583476e89f0 0x558347789c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.566+0000 7f73ab400700 1 -- 172.21.15.63:0/319798476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 msgr2=0x5583478070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.566+0000 7f73ab400700 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x5583478070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.566+0000 7f73ab400700 1 -- 172.21.15.63:0/319798476 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558347807600 con 0x5583475fce40 2024-05-27T08:12:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.566+0000 7f73ab400700 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 0x5583477896e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f739c0042a0 tx=0x7f739c005e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8f6542e044aaa36 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:47.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.566+0000 7f7399600700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:12:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.567+0000 7f7399600700 1 -- 172.21.15.63:0/319798476 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c01a040 con 0x5583475fce40 2024-05-27T08:12:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.567+0000 7f73c10d6b80 1 -- 172.21.15.63:0/319798476 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a4013c50 con 0x5583475fce40 2024-05-27T08:12:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.567+0000 7f7399600700 1 -- 172.21.15.63:0/319798476 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f739c016840 con 0x5583475fce40 2024-05-27T08:12:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.567+0000 7f7399600700 1 -- 172.21.15.63:0/319798476 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f739c03a050 con 0x5583475fce40 2024-05-27T08:12:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.568+0000 7f73c10d6b80 10 client.31642 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:47.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.568+0000 7f73c10d6b80 1 -- 172.21.15.63:0/319798476 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558347808840 con 0x5583475fce40 2024-05-27T08:12:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.568+0000 7f7399600700 1 -- 172.21.15.63:0/319798476 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73a4013c50 con 0x5583475fce40 2024-05-27T08:12:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.569+0000 7f7399600700 1 -- 172.21.15.63:0/319798476 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f739c0316d0 con 0x5583475fce40 2024-05-27T08:12:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.569+0000 7f73c10d6b80 10 client.31642 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.569+0000 7f73c10d6b80 10 client.31642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.569+0000 7f73c10d6b80 20 client.31642 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.569+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558347808f20 0x55834780b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:47.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.569+0000 7f73c10d6b80 4 client.31642 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:12:47.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.569+0000 7f73c10d6b80 1 -- 172.21.15.63:0/319798476 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583476a5570 con 0x558347808f20 2024-05-27T08:12:47.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.570+0000 7f73a2a00700 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558347808f20 0x55834780b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:47.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.570+0000 7f73a2a00700 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558347808f20 0x55834780b340 crc :-1 s=READY pgs=699 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f68e8682341be1d server_cookie=15c4b978f5c6dff5 in_seq=0 out_seq=0 2024-05-27T08:12:47.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.570+0000 7f7399600700 10 client.31642 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:47.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.571+0000 7f7399600700 1 -- 172.21.15.63:0/319798476 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5583476a5570 con 0x558347808f20 2024-05-27T08:12:47.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.571+0000 7f7399600700 10 client.31642 handle_command_reply: tid=0 2024-05-27T08:12:47.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.571+0000 7f73c10d6b80 1 client.31642 shutdown 2024-05-27T08:12:47.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.572+0000 7f73c10d6b80 1 -- 172.21.15.63:0/319798476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 msgr2=0x5583477896e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:47.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.572+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 0x5583477896e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f739c0042a0 tx=0x7f739c005e90 comp rx=0 tx=0).stop 2024-05-27T08:12:47.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.572+0000 7f73c10d6b80 1 -- 172.21.15.63:0/319798476 shutdown_connections 2024-05-27T08:12:47.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.572+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558347808f20 0x55834780b340 crc :-1 s=READY pgs=699 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.572+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583477862c0 0x5583478070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.572+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583475fce40 0x5583477896e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.572+0000 7f7399600700 0 client.31642 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:47.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.572+0000 7f73c10d6b80 1 --2- 172.21.15.63:0/319798476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583476e89f0 0x558347789c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.572+0000 7f73c10d6b80 1 -- 172.21.15.63:0/319798476 >> 172.21.15.63:0/319798476 conn(0x558347696390 msgr2=0x55834777e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:47.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.572+0000 7f73c10d6b80 1 -- 172.21.15.63:0/319798476 shutdown_connections 2024-05-27T08:12:47.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.573+0000 7f73c10d6b80 1 -- 172.21.15.63:0/319798476 wait complete. 2024-05-27T08:12:47.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.573+0000 7f73c10d6b80 20 client.31642 trim_cache size 0 max 16384 2024-05-27T08:12:47.575 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:12:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:12:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:12:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "2d0e8131-7f69-4cf1-89fe-0bb8693e5ad0": { 2024-05-27T08:12:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:12:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "2d0e8131-7f69-4cf1-89fe-0bb8693e5ad0", 2024-05-27T08:12:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:12:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:12:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:12:47.576 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:12:47.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.574+0000 7f73b1000700 1 -- 172.21.15.63:0/2789928257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73900b2a70 msgr2=0x7f73900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.574+0000 7f73b1000700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73900b2a70 0x7f73900b4f80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f73a401f000 tx=0x7f73a401c000 comp rx=0 tx=0).stop 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.574+0000 7f73b1000700 1 -- 172.21.15.63:0/2789928257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac10af40 msgr2=0x7f73ac19d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.574+0000 7f73b1000700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac10af40 0x7f73ac19d350 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f73940098e0 tx=0x7f73940047b0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.575+0000 7f73b1000700 1 -- 172.21.15.63:0/2789928257 shutdown_connections 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.575+0000 7f73b1000700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f73900b2a70 0x7f73900b4f80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.575+0000 7f73b1000700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73ac10b890 0x7f73ac1a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.575+0000 7f73b1000700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73ac106610 0x7f73ac19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.575+0000 7f73b1000700 1 --2- 172.21.15.63:0/2789928257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73ac10af40 0x7f73ac19d350 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.575+0000 7f73b1000700 1 -- 172.21.15.63:0/2789928257 >> 172.21.15.63:0/2789928257 conn(0x7f73ac102020 msgr2=0x7f73ac10a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.575+0000 7f73b1000700 1 -- 172.21.15.63:0/2789928257 shutdown_connections 2024-05-27T08:12:47.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:12:47.575+0000 7f73b1000700 1 -- 172.21.15.63:0/2789928257 wait complete. 2024-05-27T08:12:47.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:47 smithi063 ceph-mon[31613]: pgmap v1244: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.4 MiB/s wr, 49 op/s 2024-05-27T08:12:48.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:47 smithi086 ceph-mon[34687]: pgmap v1244: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.4 MiB/s wr, 49 op/s 2024-05-27T08:12:48.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:47 smithi149 ceph-mon[35408]: pgmap v1244: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.4 MiB/s wr, 49 op/s 2024-05-27T08:12:48.440 INFO:tasks.cephfs.filesystem:scrub status for tag:2d0e8131-7f69-4cf1-89fe-0bb8693e5ad0 - {'path': '/', 'tag': '2d0e8131-7f69-4cf1-89fe-0bb8693e5ad0', 'options': 'recursive,force'} 2024-05-27T08:12:50.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:49 smithi086 ceph-mon[34687]: pgmap v1245: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 45 op/s 2024-05-27T08:12:50.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:49 smithi149 ceph-mon[35408]: pgmap v1245: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 45 op/s 2024-05-27T08:12:50.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:49 smithi063 ceph-mon[31613]: pgmap v1245: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 45 op/s 2024-05-27T08:12:52.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:51 smithi086 ceph-mon[34687]: pgmap v1246: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 45 op/s 2024-05-27T08:12:52.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:51 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:12:52.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:51 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:12:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:51 smithi149 ceph-mon[35408]: pgmap v1246: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 45 op/s 2024-05-27T08:12:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:51 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:12:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:51 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:12:52.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:51 smithi063 ceph-mon[31613]: pgmap v1246: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 45 op/s 2024-05-27T08:12:52.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:51 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:12:52.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:51 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:12:53.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:52 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:12:53.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:52 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:12:53.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:52 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:12:54.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:53 smithi086 ceph-mon[34687]: pgmap v1247: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:12:54.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:53 smithi149 ceph-mon[35408]: pgmap v1247: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:12:54.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:53 smithi063 ceph-mon[31613]: pgmap v1247: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:12:55.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:54 smithi086 ceph-mon[34687]: pgmap v1248: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.0 MiB/s wr, 45 op/s 2024-05-27T08:12:55.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:54 smithi149 ceph-mon[35408]: pgmap v1248: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.0 MiB/s wr, 45 op/s 2024-05-27T08:12:55.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:54 smithi063 ceph-mon[31613]: pgmap v1248: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.0 MiB/s wr, 45 op/s 2024-05-27T08:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.830+0000 7f24a2200640 1 -- 172.21.15.63:0/695866135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f249c109ab0 msgr2=0x7f249c1036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.830+0000 7f24a2200640 1 --2- 172.21.15.63:0/695866135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f249c109ab0 0x7f249c1036a0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f2490009900 tx=0x7f249002a410 comp rx=0 tx=0).stop 2024-05-27T08:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.830+0000 7f24a2200640 1 -- 172.21.15.63:0/695866135 shutdown_connections 2024-05-27T08:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.830+0000 7f24a2200640 1 --2- 172.21.15.63:0/695866135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f249c104250 0x7f249c1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.830+0000 7f24a2200640 1 --2- 172.21.15.63:0/695866135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f249c109f50 0x7f249c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.830+0000 7f24a2200640 1 --2- 172.21.15.63:0/695866135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f249c109ab0 0x7f249c1036a0 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.830+0000 7f24a2200640 1 -- 172.21.15.63:0/695866135 >> 172.21.15.63:0/695866135 conn(0x7f249c079240 msgr2=0x7f249c079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.830+0000 7f24a2200640 1 -- 172.21.15.63:0/695866135 shutdown_connections 2024-05-27T08:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.830+0000 7f24a2200640 1 -- 172.21.15.63:0/695866135 wait complete. 2024-05-27T08:12:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.832+0000 7f24a2200640 1 Processor -- start 2024-05-27T08:12:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.832+0000 7f24a2200640 1 -- start start 2024-05-27T08:12:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.832+0000 7f24a2200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f249c104250 0x7f249c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.832+0000 7f24a2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f249c109ab0 0x7f249c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.832+0000 7f24a2200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f249c109f50 0x7f249c1a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.832+0000 7f24a2200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f249c113b50 con 0x7f249c109f50 2024-05-27T08:12:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.832+0000 7f24a2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f249c1139d0 con 0x7f249c109ab0 2024-05-27T08:12:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.832+0000 7f24a2200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f249c113cd0 con 0x7f249c104250 2024-05-27T08:12:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f24a1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f249c109f50 0x7f249c1a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f24a1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f249c109f50 0x7f249c1a84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39386/0 (socket says 172.21.15.63:39386) 2024-05-27T08:12:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f24a1800640 1 -- 172.21.15.63:0/3500563368 learned_addr learned my addr 172.21.15.63:0/3500563368 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f249be00640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f249c109ab0 0x7f249c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f24a0e00640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f249c104250 0x7f249c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f24a1800640 1 -- 172.21.15.63:0/3500563368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f249c104250 msgr2=0x7f249c19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f24a1800640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f249c104250 0x7f249c19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f24a1800640 1 -- 172.21.15.63:0/3500563368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f249c109ab0 msgr2=0x7f249c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f24a1800640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f249c109ab0 0x7f249c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f24a1800640 1 -- 172.21.15.63:0/3500563368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249c1a89f0 con 0x7f249c109f50 2024-05-27T08:12:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f249be00640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f249c109ab0 0x7f249c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.833+0000 7f24a0e00640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f249c104250 0x7f249c19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.834+0000 7f24a1800640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f249c109f50 0x7f249c1a84b0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f249400eaa0 tx=0x7f249400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.834+0000 7f2499600640 1 -- 172.21.15.63:0/3500563368 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249400cdb0 con 0x7f249c109f50 2024-05-27T08:12:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.834+0000 7f2499600640 1 -- 172.21.15.63:0/3500563368 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2494005020 con 0x7f249c109f50 2024-05-27T08:12:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.834+0000 7f24a2200640 1 -- 172.21.15.63:0/3500563368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249c1a8c80 con 0x7f249c109f50 2024-05-27T08:12:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.834+0000 7f2499600640 1 -- 172.21.15.63:0/3500563368 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24940116d0 con 0x7f249c109f50 2024-05-27T08:12:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.834+0000 7f24a2200640 1 -- 172.21.15.63:0/3500563368 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f249c1a91d0 con 0x7f249c109f50 2024-05-27T08:12:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.838+0000 7f24aa7af740 1 Processor -- start 2024-05-27T08:12:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.838+0000 7f2499600640 1 -- 172.21.15.63:0/3500563368 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f24940051d0 con 0x7f249c109f50 2024-05-27T08:12:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.838+0000 7f24aa7af740 1 -- start start 2024-05-27T08:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.839+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.839+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.839+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.839+0000 7f24aa7af740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dc3c942b90 con 0x55dc3c60e4d0 2024-05-27T08:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.839+0000 7f24aa7af740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc3c932730 con 0x55dc3c62f1b0 2024-05-27T08:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.839+0000 7f24aa7af740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dc3c936770 con 0x55dc3c6062f0 2024-05-27T08:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.839+0000 7f249be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.839+0000 7f249be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39402/0 (socket says 172.21.15.63:39402) 2024-05-27T08:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.839+0000 7f249be00640 1 -- 172.21.15.63:0/1837246497 learned_addr learned my addr 172.21.15.63:0/1837246497 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.839+0000 7f24a0e00640 1 --2- 172.21.15.63:0/1837246497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.840+0000 7f249be00640 1 --2- 172.21.15.63:0/1837246497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.840+0000 7f249be00640 1 -- 172.21.15.63:0/1837246497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 msgr2=0x55dc3c9d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.840+0000 7f249be00640 1 --2- 172.21.15.63:0/1837246497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.840+0000 7f249be00640 1 -- 172.21.15.63:0/1837246497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 msgr2=0x55dc3c9d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.840+0000 7f249be00640 1 --2- 172.21.15.63:0/1837246497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.840+0000 7f249be00640 1 -- 172.21.15.63:0/1837246497 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc3c8cb8b0 con 0x55dc3c60e4d0 2024-05-27T08:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.840+0000 7f24a0e00640 1 --2- 172.21.15.63:0/1837246497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.840+0000 7f249be00640 1 --2- 172.21.15.63:0/1837246497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9d24e0 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f24880058c0 tx=0x7f248800c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdce426a099ed43c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.840+0000 7f2499600640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24840b29d0 0x7f24840b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.841+0000 7f2499600640 1 -- 172.21.15.63:0/3500563368 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f24940e2c90 con 0x7f249c109f50 2024-05-27T08:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.841+0000 7f248e000640 1 -- 172.21.15.63:0/1837246497 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488019030 con 0x55dc3c60e4d0 2024-05-27T08:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.841+0000 7f24a0e00640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24840b29d0 0x7f24840b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.841+0000 7f248e000640 1 -- 172.21.15.63:0/1837246497 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24880119b0 con 0x55dc3c60e4d0 2024-05-27T08:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.841+0000 7f248e000640 1 -- 172.21.15.63:0/1837246497 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248800f950 con 0x55dc3c60e4d0 2024-05-27T08:12:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.842+0000 7f24aa7af740 1 -- 172.21.15.63:0/1837246497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 msgr2=0x55dc3c9d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.842+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1837246497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9d24e0 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f24880058c0 tx=0x7f248800c690 comp rx=0 tx=0).stop 2024-05-27T08:12:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.842+0000 7f24a0e00640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24840b29d0 0x7f24840b4ee0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2490006010 tx=0x7f24900058d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.842+0000 7f24aa7af740 1 -- 172.21.15.63:0/1837246497 shutdown_connections 2024-05-27T08:12:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.842+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1837246497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9d24e0 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.842+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1837246497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.842+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1837246497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.842+0000 7f24aa7af740 1 -- 172.21.15.63:0/1837246497 >> 172.21.15.63:0/1837246497 conn(0x55dc3c623300 msgr2=0x55dc3c9ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.842+0000 7f24aa7af740 1 -- 172.21.15.63:0/1837246497 shutdown_connections 2024-05-27T08:12:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.842+0000 7f24aa7af740 1 -- 172.21.15.63:0/1837246497 wait complete. 2024-05-27T08:12:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.843+0000 7f24aa7af740 1 -- start start 2024-05-27T08:12:55.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.843+0000 7f24aa7af740 1 Processor -- start 2024-05-27T08:12:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.843+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.843+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.843+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.843+0000 7f24aa7af740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dc3c618790 con 0x55dc3c60e4d0 2024-05-27T08:12:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.843+0000 7f24aa7af740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dc3c8dabc0 con 0x55dc3c62f1b0 2024-05-27T08:12:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f24aa7af740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dc3c9deb90 con 0x55dc3c6062f0 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f24aa7af740 10 client.0 _ll_register_callbacks cb 0x55dc3c8b8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f249be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f249be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9dc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39414/0 (socket says 172.21.15.63:39414) 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f249be00640 1 -- 172.21.15.63:0/12289198 learned_addr learned my addr 172.21.15.63:0/12289198 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f24a1800640 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f24a0e00640 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f249be00640 1 -- 172.21.15.63:0/12289198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 msgr2=0x55dc3c9dbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f249be00640 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9dbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f249be00640 1 -- 172.21.15.63:0/12289198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 msgr2=0x55dc3c9e7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f249be00640 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9e7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.844+0000 7f249be00640 1 -- 172.21.15.63:0/12289198 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc3c647210 con 0x55dc3c60e4d0 2024-05-27T08:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.845+0000 7f249be00640 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9dc420 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f248800d010 tx=0x7f248800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6090ae47f43243 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.845+0000 7f2482a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.845+0000 7f2482a00640 1 -- 172.21.15.63:0/12289198 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488019040 con 0x55dc3c60e4d0 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.845+0000 7f2482a00640 1 -- 172.21.15.63:0/12289198 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f248800f6d0 con 0x55dc3c60e4d0 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.845+0000 7f24aa7af740 1 -- 172.21.15.63:0/12289198 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24880119b0 con 0x55dc3c60e4d0 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.845+0000 7f2482a00640 1 -- 172.21.15.63:0/12289198 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f248800f8d0 con 0x55dc3c60e4d0 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.845+0000 7f24aa7af740 10 client.22953 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.845+0000 7f24aa7af740 1 -- 172.21.15.63:0/12289198 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dc3c713620 con 0x55dc3c60e4d0 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.846+0000 7f2482a00640 1 -- 172.21.15.63:0/12289198 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24880119b0 con 0x55dc3c60e4d0 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.846+0000 7f2482a00640 1 -- 172.21.15.63:0/12289198 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2488027b10 con 0x55dc3c60e4d0 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.846+0000 7f24aa7af740 10 client.22953 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.846+0000 7f24aa7af740 10 client.22953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.846+0000 7f24aa7af740 20 client.22953 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.846+0000 7f24aa7af740 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc3c5edd80 0x55dc3c9edde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.846+0000 7f24aa7af740 4 client.22953 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.846+0000 7f24aa7af740 1 -- 172.21.15.63:0/12289198 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc3c944ae0 con 0x55dc3c5edd80 2024-05-27T08:12:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.847+0000 7f24a0e00640 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc3c5edd80 0x55dc3c9edde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.848+0000 7f24a0e00640 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc3c5edd80 0x55dc3c9edde0 crc :-1 s=READY pgs=700 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acfeb1ebd6adf0dc server_cookie=f6438e2841499ff3 in_seq=0 out_seq=0 2024-05-27T08:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.848+0000 7f2482a00640 10 client.22953 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f2482a00640 1 -- 172.21.15.63:0/12289198 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dc3c944ae0 con 0x55dc3c5edd80 2024-05-27T08:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f2482a00640 10 client.22953 handle_command_reply: tid=0 2024-05-27T08:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f24aa7af740 1 client.22953 shutdown 2024-05-27T08:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f24aa7af740 1 -- 172.21.15.63:0/12289198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 msgr2=0x55dc3c9dc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f24aa7af740 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9dc420 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f248800d010 tx=0x7f248800cda0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f24aa7af740 1 -- 172.21.15.63:0/12289198 shutdown_connections 2024-05-27T08:12:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f24aa7af740 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc3c5edd80 0x55dc3c9edde0 crc :-1 s=READY pgs=700 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f24aa7af740 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9e7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f24aa7af740 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9dc420 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f24aa7af740 1 --2- 172.21.15.63:0/12289198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f2482a00640 0 client.22953 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f24aa7af740 1 -- 172.21.15.63:0/12289198 >> 172.21.15.63:0/12289198 conn(0x55dc3c623300 msgr2=0x55dc3c9d3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.850+0000 7f24aa7af740 1 -- 172.21.15.63:0/12289198 shutdown_connections 2024-05-27T08:12:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.851+0000 7f24aa7af740 1 -- 172.21.15.63:0/12289198 wait complete. 2024-05-27T08:12:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.851+0000 7f24aa7af740 20 client.22953 trim_cache size 0 max 16384 2024-05-27T08:12:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.873+0000 7f24aa7af740 1 Processor -- start 2024-05-27T08:12:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.873+0000 7f24aa7af740 1 -- start start 2024-05-27T08:12:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.873+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c6062f0 0x55dc3c9d74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.873+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.873+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.873+0000 7f24aa7af740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f249c1b5b30 con 0x55dc3c62f1b0 2024-05-27T08:12:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.873+0000 7f24aa7af740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f249c1b3400 con 0x55dc3c6062f0 2024-05-27T08:12:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.873+0000 7f24aa7af740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f249c1b1270 con 0x55dc3c60e4d0 2024-05-27T08:12:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a1800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9e7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39416/0 (socket says 172.21.15.63:39416) 2024-05-27T08:12:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a1800640 1 -- 172.21.15.63:0/1414554438 learned_addr learned my addr 172.21.15.63:0/1414554438 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f249be00640 1 --2- 172.21.15.63:0/1414554438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a0e00640 1 --2- 172.21.15.63:0/1414554438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c6062f0 0x55dc3c9d74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a1800640 1 -- 172.21.15.63:0/1414554438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c60e4d0 msgr2=0x55dc3c9d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a1800640 1 --2- 172.21.15.63:0/1414554438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a1800640 1 -- 172.21.15.63:0/1414554438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c6062f0 msgr2=0x55dc3c9d74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a1800640 1 --2- 172.21.15.63:0/1414554438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c6062f0 0x55dc3c9d74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a1800640 1 -- 172.21.15.63:0/1414554438 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc3c9cc610 con 0x55dc3c62f1b0 2024-05-27T08:12:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a0e00640 1 --2- 172.21.15.63:0/1414554438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c6062f0 0x55dc3c9d74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:12:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.874+0000 7f24a1800640 1 --2- 172.21.15.63:0/1414554438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9e7e30 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f24940e0020 tx=0x7f24940a95b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa97b95073b27ef2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.875+0000 7f248d600640 1 -- 172.21.15.63:0/1414554438 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249401f030 con 0x55dc3c62f1b0 2024-05-27T08:12:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.875+0000 7f248d600640 1 -- 172.21.15.63:0/1414554438 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24940a9b50 con 0x55dc3c62f1b0 2024-05-27T08:12:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.875+0000 7f248d600640 1 -- 172.21.15.63:0/1414554438 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24940a5de0 con 0x55dc3c62f1b0 2024-05-27T08:12:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.876+0000 7f24aa7af740 1 -- 172.21.15.63:0/1414554438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c62f1b0 msgr2=0x55dc3c9e7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.876+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1414554438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9e7e30 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f24940e0020 tx=0x7f24940a95b0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.876+0000 7f24aa7af740 1 -- 172.21.15.63:0/1414554438 shutdown_connections 2024-05-27T08:12:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.876+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1414554438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c62f1b0 0x55dc3c9e7e30 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.876+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1414554438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.876+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1414554438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c6062f0 0x55dc3c9d74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.876+0000 7f24aa7af740 1 -- 172.21.15.63:0/1414554438 >> 172.21.15.63:0/1414554438 conn(0x55dc3c623300 msgr2=0x55dc3c9dd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.876+0000 7f24aa7af740 1 -- 172.21.15.63:0/1414554438 shutdown_connections 2024-05-27T08:12:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.876+0000 7f24aa7af740 1 -- 172.21.15.63:0/1414554438 wait complete. 2024-05-27T08:12:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.877+0000 7f24aa7af740 1 -- start start 2024-05-27T08:12:55.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.877+0000 7f24aa7af740 1 Processor -- start 2024-05-27T08:12:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.877+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9ec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.877+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9ec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.877+0000 7f24aa7af740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3ca54210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.877+0000 7f24aa7af740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f249c1b18c0 con 0x55dc3c60e4d0 2024-05-27T08:12:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.877+0000 7f24aa7af740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f249c1b2d30 con 0x55dc3c62f1b0 2024-05-27T08:12:55.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.877+0000 7f24aa7af740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f249c1b1f90 con 0x55dc3c6062f0 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.877+0000 7f24aa7af740 10 client.0 _ll_register_callbacks cb 0x7f2488013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.878+0000 7f249be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.878+0000 7f249be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9ec910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39430/0 (socket says 172.21.15.63:39430) 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.878+0000 7f249be00640 1 -- 172.21.15.63:0/1236404293 learned_addr learned my addr 172.21.15.63:0/1236404293 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.878+0000 7f24a0e00640 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9ec3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.878+0000 7f24a1800640 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3ca54210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.878+0000 7f249be00640 1 -- 172.21.15.63:0/1236404293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 msgr2=0x55dc3c9ec3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.878+0000 7f249be00640 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9ec3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.878+0000 7f249be00640 1 -- 172.21.15.63:0/1236404293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 msgr2=0x55dc3ca54210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.878+0000 7f249be00640 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3ca54210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.878+0000 7f249be00640 1 -- 172.21.15.63:0/1236404293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc3ca54750 con 0x55dc3c60e4d0 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.879+0000 7f249be00640 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9ec910 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f248801e4b0 tx=0x7f248800c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdd14a09977cfce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.879+0000 7f2482a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.879+0000 7f2482a00640 1 -- 172.21.15.63:0/1236404293 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488019040 con 0x55dc3c60e4d0 2024-05-27T08:12:55.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.879+0000 7f2482a00640 1 -- 172.21.15.63:0/1236404293 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 9483+0+0 (secure 0 0 0) 0x7f248800fa70 con 0x55dc3c60e4d0 2024-05-27T08:12:55.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.879+0000 7f24aa7af740 1 -- 172.21.15.63:0/1236404293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24940a9b50 con 0x55dc3c60e4d0 2024-05-27T08:12:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.879+0000 7f2482a00640 1 -- 172.21.15.63:0/1236404293 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f248803f050 con 0x55dc3c60e4d0 2024-05-27T08:12:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.879+0000 7f24aa7af740 10 client.22959 fetch_fsmap learned FSMap version 35 2024-05-27T08:12:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.879+0000 7f24aa7af740 1 -- 172.21.15.63:0/1236404293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dc3ca559b0 con 0x55dc3c60e4d0 2024-05-27T08:12:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.880+0000 7f2482a00640 1 -- 172.21.15.63:0/1236404293 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24940a9b50 con 0x55dc3c60e4d0 2024-05-27T08:12:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.880+0000 7f2482a00640 1 -- 172.21.15.63:0/1236404293 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2488005150 con 0x55dc3c60e4d0 2024-05-27T08:12:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.880+0000 7f24aa7af740 10 client.22959 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:12:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.880+0000 7f24aa7af740 10 client.22959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:12:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.880+0000 7f24aa7af740 20 client.22959 populate_metadata read hostname 'smithi063' 2024-05-27T08:12:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.880+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc3c5c31c0 0x55dc3ca58160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:12:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.880+0000 7f24aa7af740 4 client.22959 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:12:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.880+0000 7f24aa7af740 1 -- 172.21.15.63:0/1236404293 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc3c93e2c0 con 0x55dc3c5c31c0 2024-05-27T08:12:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.881+0000 7f24a0e00640 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc3c5c31c0 0x55dc3ca58160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:12:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.882+0000 7f24a0e00640 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc3c5c31c0 0x55dc3ca58160 crc :-1 s=READY pgs=701 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8b9579d3d6bb6b3 server_cookie=2bbbeabecd7206b9 in_seq=0 out_seq=0 2024-05-27T08:12:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.882+0000 7f2482a00640 10 client.22959 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.937+0000 7f2482a00640 1 -- 172.21.15.63:0/1236404293 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dc3c93e2c0 con 0x55dc3c5c31c0 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.937+0000 7f2482a00640 10 client.22959 handle_command_reply: tid=0 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.937+0000 7f24aa7af740 1 client.22959 shutdown 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 1 -- 172.21.15.63:0/1236404293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 msgr2=0x55dc3c9ec910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9ec910 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f248801e4b0 tx=0x7f248800c690 comp rx=0 tx=0).stop 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 1 -- 172.21.15.63:0/1236404293 shutdown_connections 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55dc3c5c31c0 0x55dc3ca58160 crc :-1 s=READY pgs=701 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dc3c62f1b0 0x55dc3ca54210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dc3c60e4d0 0x55dc3c9ec910 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 1 --2- 172.21.15.63:0/1236404293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc3c6062f0 0x55dc3c9ec3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 1 -- 172.21.15.63:0/1236404293 >> 172.21.15.63:0/1236404293 conn(0x55dc3c623300 msgr2=0x55dc3c9dde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f2482a00640 0 client.22959 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 1 -- 172.21.15.63:0/1236404293 shutdown_connections 2024-05-27T08:12:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 1 -- 172.21.15.63:0/1236404293 wait complete. 2024-05-27T08:12:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.938+0000 7f24aa7af740 20 client.22959 trim_cache size 0 max 16384 2024-05-27T08:12:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:12:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:12:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.939+0000 7f24a2200640 1 -- 172.21.15.63:0/3500563368 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24840b29d0 msgr2=0x7f24840b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.939+0000 7f24a2200640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24840b29d0 0x7f24840b4ee0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2490006010 tx=0x7f24900058d0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.939+0000 7f24a2200640 1 -- 172.21.15.63:0/3500563368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f249c109f50 msgr2=0x7f249c1a84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.939+0000 7f24a2200640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f249c109f50 0x7f249c1a84b0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f249400eaa0 tx=0x7f249400ef60 comp rx=0 tx=0).stop 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.939+0000 7f24a2200640 1 -- 172.21.15.63:0/3500563368 shutdown_connections 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.939+0000 7f24a2200640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f24840b29d0 0x7f24840b4ee0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.939+0000 7f24a2200640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f249c109f50 0x7f249c1a84b0 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.939+0000 7f24a2200640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f249c109ab0 0x7f249c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.939+0000 7f24a2200640 1 --2- 172.21.15.63:0/3500563368 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f249c104250 0x7f249c19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.939+0000 7f24a2200640 1 -- 172.21.15.63:0/3500563368 >> 172.21.15.63:0/3500563368 conn(0x7f249c079240 msgr2=0x7f249c102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.940+0000 7f24a2200640 1 -- 172.21.15.63:0/3500563368 shutdown_connections 2024-05-27T08:12:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:12:55.940+0000 7f24a2200640 1 -- 172.21.15.63:0/3500563368 wait complete. 2024-05-27T08:12:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:57 smithi086 ceph-mon[34687]: pgmap v1249: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.0 MiB/s wr, 52 op/s 2024-05-27T08:12:57.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:57 smithi149 ceph-mon[35408]: pgmap v1249: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.0 MiB/s wr, 52 op/s 2024-05-27T08:12:57.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:57 smithi063 ceph-mon[31613]: pgmap v1249: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.0 MiB/s wr, 52 op/s 2024-05-27T08:13:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:12:59 smithi086 ceph-mon[34687]: pgmap v1250: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:13:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:12:59 smithi149 ceph-mon[35408]: pgmap v1250: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:13:00.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:12:59 smithi063 ceph-mon[31613]: pgmap v1250: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:13:02.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:01 smithi086 ceph-mon[34687]: pgmap v1251: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:13:02.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:01 smithi149 ceph-mon[35408]: pgmap v1251: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:13:02.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:01 smithi063 ceph-mon[31613]: pgmap v1251: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.4 MiB/s wr, 10 op/s 2024-05-27T08:13:04.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:03 smithi086 ceph-mon[34687]: pgmap v1252: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 31 op/s 2024-05-27T08:13:04.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:03 smithi149 ceph-mon[35408]: pgmap v1252: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 31 op/s 2024-05-27T08:13:04.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:03 smithi063 ceph-mon[31613]: pgmap v1252: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 31 op/s 2024-05-27T08:13:06.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:05 smithi086 ceph-mon[34687]: pgmap v1253: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.2 MiB/s wr, 27 op/s 2024-05-27T08:13:06.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:05 smithi149 ceph-mon[35408]: pgmap v1253: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.2 MiB/s wr, 27 op/s 2024-05-27T08:13:06.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:05 smithi063 ceph-mon[31613]: pgmap v1253: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.2 MiB/s wr, 27 op/s 2024-05-27T08:13:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:07 smithi086 ceph-mon[34687]: pgmap v1254: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-05-27T08:13:08.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:07 smithi086 ceph-mon[34687]: osdmap e159: 12 total, 12 up, 12 in 2024-05-27T08:13:08.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:07 smithi149 ceph-mon[35408]: pgmap v1254: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-05-27T08:13:08.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:07 smithi149 ceph-mon[35408]: osdmap e159: 12 total, 12 up, 12 in 2024-05-27T08:13:08.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:07 smithi063 ceph-mon[31613]: pgmap v1254: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-05-27T08:13:08.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:07 smithi063 ceph-mon[31613]: osdmap e159: 12 total, 12 up, 12 in 2024-05-27T08:13:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.187+0000 7fe15aa00640 1 -- 172.21.15.63:0/3363044223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe154109ab0 msgr2=0x7fe1541036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.187+0000 7fe15aa00640 1 --2- 172.21.15.63:0/3363044223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe154109ab0 0x7fe1541036a0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7fe148009900 tx=0x7fe14802a410 comp rx=0 tx=0).stop 2024-05-27T08:13:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.188+0000 7fe15aa00640 1 -- 172.21.15.63:0/3363044223 shutdown_connections 2024-05-27T08:13:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.188+0000 7fe15aa00640 1 --2- 172.21.15.63:0/3363044223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe154104250 0x7fe1541105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.188+0000 7fe15aa00640 1 --2- 172.21.15.63:0/3363044223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154109f50 0x7fe154103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.188+0000 7fe15aa00640 1 --2- 172.21.15.63:0/3363044223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe154109ab0 0x7fe1541036a0 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.188+0000 7fe15aa00640 1 -- 172.21.15.63:0/3363044223 >> 172.21.15.63:0/3363044223 conn(0x7fe154079240 msgr2=0x7fe154079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.188+0000 7fe15aa00640 1 -- 172.21.15.63:0/3363044223 shutdown_connections 2024-05-27T08:13:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.188+0000 7fe15aa00640 1 -- 172.21.15.63:0/3363044223 wait complete. 2024-05-27T08:13:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.189+0000 7fe15aa00640 1 Processor -- start 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.189+0000 7fe15aa00640 1 -- start start 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.189+0000 7fe15aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe154104250 0x7fe154074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.189+0000 7fe15aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154109ab0 0x7fe154075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.189+0000 7fe15aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe154109f50 0x7fe154075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.189+0000 7fe15aa00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe154113ba0 con 0x7fe154104250 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe15aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe154113a20 con 0x7fe154109f50 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe15aa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe154113d20 con 0x7fe154109ab0 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe159600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe154104250 0x7fe154074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe159600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe154104250 0x7fe154074fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47156/0 (socket says 172.21.15.63:47156) 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe159600640 1 -- 172.21.15.63:0/2453345969 learned_addr learned my addr 172.21.15.63:0/2453345969 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe158c00640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154109ab0 0x7fe154075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe15a000640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe154109f50 0x7fe154075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe159600640 1 -- 172.21.15.63:0/2453345969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154109ab0 msgr2=0x7fe154075500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe159600640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154109ab0 0x7fe154075500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe159600640 1 -- 172.21.15.63:0/2453345969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe154109f50 msgr2=0x7fe154075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe159600640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe154109f50 0x7fe154075a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.190+0000 7fe159600640 1 -- 172.21.15.63:0/2453345969 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1541a89a0 con 0x7fe154104250 2024-05-27T08:13:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.191+0000 7fe159600640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe154104250 0x7fe154074fc0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7fe14802a930 tx=0x7fe148003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.191+0000 7fe152000640 1 -- 172.21.15.63:0/2453345969 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14803f070 con 0x7fe154104250 2024-05-27T08:13:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.191+0000 7fe152000640 1 -- 172.21.15.63:0/2453345969 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe148003e10 con 0x7fe154104250 2024-05-27T08:13:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.191+0000 7fe152000640 1 -- 172.21.15.63:0/2453345969 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148035030 con 0x7fe154104250 2024-05-27T08:13:09.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.191+0000 7fe15aa00640 1 -- 172.21.15.63:0/2453345969 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1541a8c30 con 0x7fe154104250 2024-05-27T08:13:09.195 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.191+0000 7fe15aa00640 1 -- 172.21.15.63:0/2453345969 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541a90d0 con 0x7fe154104250 2024-05-27T08:13:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.195+0000 7fe162fef740 1 Processor -- start 2024-05-27T08:13:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.195+0000 7fe162fef740 1 -- start start 2024-05-27T08:13:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.195+0000 7fe162fef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2feba1b0 0x563b3025ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.195+0000 7fe162fef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2fe912f0 0x563b3025cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe162fef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe994d0 0x563b3025d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe162fef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563b301cdb90 con 0x563b2fe994d0 2024-05-27T08:13:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe162fef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b301bd730 con 0x563b2feba1b0 2024-05-27T08:13:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe162fef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563b301c1770 con 0x563b2fe912f0 2024-05-27T08:13:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe158c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe994d0 0x563b3025d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe158c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe994d0 0x563b3025d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47158/0 (socket says 172.21.15.63:47158) 2024-05-27T08:13:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe158c00640 1 -- 172.21.15.63:0/1602253742 learned_addr learned my addr 172.21.15.63:0/1602253742 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe15a000640 1 --2- 172.21.15.63:0/1602253742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2feba1b0 0x563b3025ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe158c00640 1 -- 172.21.15.63:0/1602253742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2fe912f0 msgr2=0x563b3025cfa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:13:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe158c00640 1 --2- 172.21.15.63:0/1602253742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2fe912f0 0x563b3025cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe158c00640 1 -- 172.21.15.63:0/1602253742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2feba1b0 msgr2=0x563b3025ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe158c00640 1 --2- 172.21.15.63:0/1602253742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2feba1b0 0x563b3025ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe152000640 1 -- 172.21.15.63:0/2453345969 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe148042070 con 0x7fe154104250 2024-05-27T08:13:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.196+0000 7fe158c00640 1 -- 172.21.15.63:0/1602253742 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b301568b0 con 0x563b2fe994d0 2024-05-27T08:13:09.199 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.197+0000 7fe158c00640 1 --2- 172.21.15.63:0/1602253742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe994d0 0x563b3025d4e0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7fe14000bba0 tx=0x7fe14000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579575774c517b7b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.197+0000 7fe146600640 1 -- 172.21.15.63:0/1602253742 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140015070 con 0x563b2fe994d0 2024-05-27T08:13:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.197+0000 7fe146600640 1 -- 172.21.15.63:0/1602253742 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe14001a8f0 con 0x563b2fe994d0 2024-05-27T08:13:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.197+0000 7fe146600640 1 -- 172.21.15.63:0/1602253742 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400117a0 con 0x563b2fe994d0 2024-05-27T08:13:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.198+0000 7fe162fef740 1 -- 172.21.15.63:0/1602253742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe994d0 msgr2=0x563b3025d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.198+0000 7fe162fef740 1 --2- 172.21.15.63:0/1602253742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe994d0 0x563b3025d4e0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7fe14000bba0 tx=0x7fe14000f730 comp rx=0 tx=0).stop 2024-05-27T08:13:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.198+0000 7fe152000640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe13c0b2950 0x7fe13c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.199+0000 7fe162fef740 1 -- 172.21.15.63:0/1602253742 shutdown_connections 2024-05-27T08:13:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.199+0000 7fe162fef740 1 --2- 172.21.15.63:0/1602253742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe994d0 0x563b3025d4e0 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.199+0000 7fe162fef740 1 --2- 172.21.15.63:0/1602253742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2fe912f0 0x563b3025cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.199+0000 7fe162fef740 1 --2- 172.21.15.63:0/1602253742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2feba1b0 0x563b3025ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.199+0000 7fe162fef740 1 -- 172.21.15.63:0/1602253742 >> 172.21.15.63:0/1602253742 conn(0x563b2feae300 msgr2=0x563b30259210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.199+0000 7fe152000640 1 -- 172.21.15.63:0/2453345969 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 9571+0+0 (secure 0 0 0) 0x7fe1481032f0 con 0x7fe154104250 2024-05-27T08:13:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.199+0000 7fe15a000640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe13c0b2950 0x7fe13c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.199+0000 7fe162fef740 1 -- 172.21.15.63:0/1602253742 shutdown_connections 2024-05-27T08:13:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.199+0000 7fe162fef740 1 -- 172.21.15.63:0/1602253742 wait complete. 2024-05-27T08:13:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.200+0000 7fe162fef740 1 -- start start 2024-05-27T08:13:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.200+0000 7fe15a000640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe13c0b2950 0x7fe13c0b4e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x563b30259bd0 tx=0x7fe14c009580 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.200+0000 7fe162fef740 1 Processor -- start 2024-05-27T08:13:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe162fef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30275570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe162fef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 0x563b30275ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe162fef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2feba1b0 0x563b3026f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe162fef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563b2fea3790 con 0x563b2fe912f0 2024-05-27T08:13:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe162fef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b30165bc0 con 0x563b2fe994d0 2024-05-27T08:13:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe162fef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563b30269b90 con 0x563b2feba1b0 2024-05-27T08:13:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe162fef740 10 client.0 _ll_register_callbacks cb 0x563b30143400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe159600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30275570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe159600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30275570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47170/0 (socket says 172.21.15.63:47170) 2024-05-27T08:13:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe159600640 1 -- 172.21.15.63:0/3019478889 learned_addr learned my addr 172.21.15.63:0/3019478889 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe15a000640 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2feba1b0 0x563b3026f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.201+0000 7fe158c00640 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 0x563b30275ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe159600640 1 -- 172.21.15.63:0/3019478889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2feba1b0 msgr2=0x563b3026f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe159600640 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2feba1b0 0x563b3026f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe159600640 1 -- 172.21.15.63:0/3019478889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 msgr2=0x563b30275ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe159600640 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 0x563b30275ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe159600640 1 -- 172.21.15.63:0/3019478889 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b2fed2210 con 0x563b2fe912f0 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe158c00640 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 0x563b30275ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe159600640 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30275570 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7fe148100c20 tx=0x7fe1480c9bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890435f4c2d60e43 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe13aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe13aa00640 1 -- 172.21.15.63:0/3019478889 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148049040 con 0x563b2fe912f0 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe13aa00640 1 -- 172.21.15.63:0/3019478889 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(159..159 src has 1..159) ==== 9571+0+0 (secure 0 0 0) 0x7fe1480ce120 con 0x563b2fe912f0 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.202+0000 7fe162fef740 1 -- 172.21.15.63:0/3019478889 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe14001a8f0 con 0x563b2fe912f0 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe13aa00640 1 -- 172.21.15.63:0/3019478889 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe1480ce340 con 0x563b2fe912f0 2024-05-27T08:13:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe162fef740 10 client.22971 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe162fef740 1 -- 172.21.15.63:0/3019478889 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563b2ff9e620 con 0x563b2fe912f0 2024-05-27T08:13:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe13aa00640 1 -- 172.21.15.63:0/3019478889 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe14001a8f0 con 0x563b2fe912f0 2024-05-27T08:13:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe13aa00640 1 -- 172.21.15.63:0/3019478889 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe148089bd0 con 0x563b2fe912f0 2024-05-27T08:13:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe162fef740 10 client.22971 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe162fef740 10 client.22971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe162fef740 20 client.22971 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe162fef740 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b2fe78d80 0x563b302815e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe162fef740 4 client.22971 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:13:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.203+0000 7fe162fef740 1 -- 172.21.15.63:0/3019478889 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b301cfae0 con 0x563b2fe78d80 2024-05-27T08:13:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.204+0000 7fe158c00640 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b2fe78d80 0x563b302815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.206+0000 7fe13aa00640 10 client.22971 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.206+0000 7fe158c00640 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b2fe78d80 0x563b302815e0 crc :-1 s=READY pgs=702 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a67c04e99a3588 server_cookie=ee306b10a0cc2aba in_seq=0 out_seq=0 2024-05-27T08:13:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.208+0000 7fe13aa00640 1 -- 172.21.15.63:0/3019478889 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563b301cfae0 con 0x563b2fe78d80 2024-05-27T08:13:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.208+0000 7fe13aa00640 10 client.22971 handle_command_reply: tid=0 2024-05-27T08:13:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.208+0000 7fe162fef740 1 client.22971 shutdown 2024-05-27T08:13:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.208+0000 7fe162fef740 1 -- 172.21.15.63:0/3019478889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 msgr2=0x563b30275570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.208+0000 7fe162fef740 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30275570 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7fe148100c20 tx=0x7fe1480c9bc0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.208+0000 7fe162fef740 1 -- 172.21.15.63:0/3019478889 shutdown_connections 2024-05-27T08:13:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.208+0000 7fe162fef740 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b2fe78d80 0x563b302815e0 crc :-1 s=READY pgs=702 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.209+0000 7fe162fef740 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2feba1b0 0x563b3026f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.209+0000 7fe162fef740 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 0x563b30275ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.209+0000 7fe162fef740 1 --2- 172.21.15.63:0/3019478889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30275570 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.209+0000 7fe162fef740 1 -- 172.21.15.63:0/3019478889 >> 172.21.15.63:0/3019478889 conn(0x563b2feae300 msgr2=0x563b3025e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.209+0000 7fe13aa00640 0 client.22971 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.209+0000 7fe162fef740 1 -- 172.21.15.63:0/3019478889 shutdown_connections 2024-05-27T08:13:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.209+0000 7fe162fef740 1 -- 172.21.15.63:0/3019478889 wait complete. 2024-05-27T08:13:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.209+0000 7fe162fef740 20 client.22971 trim_cache size 0 max 16384 2024-05-27T08:13:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.231+0000 7fe162fef740 1 Processor -- start 2024-05-27T08:13:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.231+0000 7fe162fef740 1 -- start start 2024-05-27T08:13:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.231+0000 7fe162fef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30269060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.231+0000 7fe162fef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 0x563b30259350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.231+0000 7fe162fef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2feba1b0 0x563b302750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.231+0000 7fe162fef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541b17a0 con 0x563b2fe912f0 2024-05-27T08:13:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.231+0000 7fe162fef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541af070 con 0x563b2fe994d0 2024-05-27T08:13:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.231+0000 7fe162fef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541acf20 con 0x563b2feba1b0 2024-05-27T08:13:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.231+0000 7fe159600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30269060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.232+0000 7fe159600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30269060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47180/0 (socket says 172.21.15.63:47180) 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.232+0000 7fe159600640 1 -- 172.21.15.63:0/966080178 learned_addr learned my addr 172.21.15.63:0/966080178 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.232+0000 7fe158c00640 1 --2- 172.21.15.63:0/966080178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 0x563b30259350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.232+0000 7fe15a000640 1 --2- 172.21.15.63:0/966080178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2feba1b0 0x563b302750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.232+0000 7fe159600640 1 -- 172.21.15.63:0/966080178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2feba1b0 msgr2=0x563b302750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.232+0000 7fe159600640 1 --2- 172.21.15.63:0/966080178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2feba1b0 0x563b302750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.232+0000 7fe159600640 1 -- 172.21.15.63:0/966080178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 msgr2=0x563b30259350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.232+0000 7fe159600640 1 --2- 172.21.15.63:0/966080178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 0x563b30259350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.232+0000 7fe159600640 1 -- 172.21.15.63:0/966080178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b30257610 con 0x563b2fe912f0 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.232+0000 7fe159600640 1 --2- 172.21.15.63:0/966080178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30269060 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fe148103610 tx=0x7fe148005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630f9bef1f500df1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.233+0000 7fe145c00640 1 -- 172.21.15.63:0/966080178 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14808e070 con 0x563b2fe912f0 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.233+0000 7fe145c00640 1 -- 172.21.15.63:0/966080178 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1480ca1b0 con 0x563b2fe912f0 2024-05-27T08:13:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.233+0000 7fe145c00640 1 -- 172.21.15.63:0/966080178 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148049070 con 0x563b2fe912f0 2024-05-27T08:13:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.234+0000 7fe162fef740 1 -- 172.21.15.63:0/966080178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 msgr2=0x563b30269060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.234+0000 7fe162fef740 1 --2- 172.21.15.63:0/966080178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30269060 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fe148103610 tx=0x7fe148005de0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.234+0000 7fe162fef740 1 -- 172.21.15.63:0/966080178 shutdown_connections 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.234+0000 7fe162fef740 1 --2- 172.21.15.63:0/966080178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2feba1b0 0x563b302750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.234+0000 7fe162fef740 1 --2- 172.21.15.63:0/966080178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe994d0 0x563b30259350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.234+0000 7fe162fef740 1 --2- 172.21.15.63:0/966080178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2fe912f0 0x563b30269060 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.234+0000 7fe162fef740 1 -- 172.21.15.63:0/966080178 >> 172.21.15.63:0/966080178 conn(0x563b2feae300 msgr2=0x563b30257810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.234+0000 7fe162fef740 1 -- 172.21.15.63:0/966080178 shutdown_connections 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.234+0000 7fe162fef740 1 -- 172.21.15.63:0/966080178 wait complete. 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.235+0000 7fe162fef740 1 -- start start 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.235+0000 7fe162fef740 1 Processor -- start 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.235+0000 7fe162fef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe912f0 0x563b302764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.235+0000 7fe162fef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2fe994d0 0x563b30276a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.235+0000 7fe162fef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2feba1b0 0x563b30276f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.235+0000 7fe162fef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541ad570 con 0x563b2feba1b0 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.235+0000 7fe162fef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541ae9a0 con 0x563b2fe912f0 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.235+0000 7fe162fef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541adc00 con 0x563b2fe994d0 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe162fef740 10 client.0 _ll_register_callbacks cb 0x7fe148005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe15a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2feba1b0 0x563b30276f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe15a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2feba1b0 0x563b30276f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47182/0 (socket says 172.21.15.63:47182) 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe15a000640 1 -- 172.21.15.63:0/1906708883 learned_addr learned my addr 172.21.15.63:0/1906708883 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe159600640 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe912f0 0x563b302764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe158c00640 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2fe994d0 0x563b30276a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe15a000640 1 -- 172.21.15.63:0/1906708883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2fe994d0 msgr2=0x563b30276a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe15a000640 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2fe994d0 0x563b30276a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe15a000640 1 -- 172.21.15.63:0/1906708883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe912f0 msgr2=0x563b302764e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe15a000640 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe912f0 0x563b302764e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe15a000640 1 -- 172.21.15.63:0/1906708883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b302e7d10 con 0x563b2feba1b0 2024-05-27T08:13:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.236+0000 7fe159600640 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe912f0 0x563b302764e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.237+0000 7fe15a000640 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2feba1b0 0x563b30276f60 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0129d0 tx=0x7fe14c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2508ef72c71d77e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.237+0000 7fe13aa00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.237+0000 7fe13aa00640 1 -- 172.21.15.63:0/1906708883 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c02a440 con 0x563b2feba1b0 2024-05-27T08:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.237+0000 7fe13aa00640 1 -- 172.21.15.63:0/1906708883 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(159..159 src has 1..159) ==== 9571+0+0 (secure 0 0 0) 0x7fe14c01f450 con 0x563b2feba1b0 2024-05-27T08:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.237+0000 7fe162fef740 1 -- 172.21.15.63:0/1906708883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1480ca1b0 con 0x563b2feba1b0 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.237+0000 7fe13aa00640 1 -- 172.21.15.63:0/1906708883 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe14c01f650 con 0x563b2feba1b0 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.237+0000 7fe162fef740 10 client.22977 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.237+0000 7fe162fef740 1 -- 172.21.15.63:0/1906708883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563b302e9000 con 0x563b2feba1b0 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.238+0000 7fe13aa00640 1 -- 172.21.15.63:0/1906708883 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe1480ca1b0 con 0x563b2feba1b0 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.238+0000 7fe13aa00640 1 -- 172.21.15.63:0/1906708883 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe14c028070 con 0x563b2feba1b0 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.238+0000 7fe162fef740 10 client.22977 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.238+0000 7fe162fef740 10 client.22977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.238+0000 7fe162fef740 20 client.22977 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.238+0000 7fe162fef740 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b2fe4e1c0 0x563b302eb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.238+0000 7fe162fef740 4 client.22977 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.238+0000 7fe162fef740 1 -- 172.21.15.63:0/1906708883 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b301c92c0 con 0x563b2fe4e1c0 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.239+0000 7fe159600640 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b2fe4e1c0 0x563b302eb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.240+0000 7fe159600640 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b2fe4e1c0 0x563b302eb870 crc :-1 s=READY pgs=703 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97dccc09e23fc88 server_cookie=fae32b4144d0d8eb in_seq=0 out_seq=0 2024-05-27T08:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.240+0000 7fe13aa00640 10 client.22977 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.275+0000 7fe13aa00640 1 -- 172.21.15.63:0/1906708883 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563b301c92c0 con 0x563b2fe4e1c0 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.275+0000 7fe13aa00640 10 client.22977 handle_command_reply: tid=0 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.275+0000 7fe162fef740 1 client.22977 shutdown 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 1 -- 172.21.15.63:0/1906708883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2feba1b0 msgr2=0x563b30276f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2feba1b0 0x563b30276f60 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0129d0 tx=0x7fe14c004a40 comp rx=0 tx=0).stop 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 1 -- 172.21.15.63:0/1906708883 shutdown_connections 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563b2fe4e1c0 0x563b302eb870 crc :-1 s=READY pgs=703 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563b2feba1b0 0x563b30276f60 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b2fe994d0 0x563b30276a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 1 --2- 172.21.15.63:0/1906708883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b2fe912f0 0x563b302764e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe13aa00640 0 client.22977 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 1 -- 172.21.15.63:0/1906708883 >> 172.21.15.63:0/1906708883 conn(0x563b2feae300 msgr2=0x563b30257fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 1 -- 172.21.15.63:0/1906708883 shutdown_connections 2024-05-27T08:13:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 1 -- 172.21.15.63:0/1906708883 wait complete. 2024-05-27T08:13:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.276+0000 7fe162fef740 20 client.22977 trim_cache size 0 max 16384 2024-05-27T08:13:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:13:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:13:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:13:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.277+0000 7fe15aa00640 1 -- 172.21.15.63:0/2453345969 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe13c0b2950 msgr2=0x7fe13c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.277+0000 7fe15aa00640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe13c0b2950 0x7fe13c0b4e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x563b30259bd0 tx=0x7fe14c009580 comp rx=0 tx=0).stop 2024-05-27T08:13:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.277+0000 7fe15aa00640 1 -- 172.21.15.63:0/2453345969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe154104250 msgr2=0x7fe154074fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.277+0000 7fe15aa00640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe154104250 0x7fe154074fc0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7fe14802a930 tx=0x7fe148003c20 comp rx=0 tx=0).stop 2024-05-27T08:13:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.278+0000 7fe15aa00640 1 -- 172.21.15.63:0/2453345969 shutdown_connections 2024-05-27T08:13:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.278+0000 7fe15aa00640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe13c0b2950 0x7fe13c0b4e60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.278+0000 7fe15aa00640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe154109f50 0x7fe154075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.278+0000 7fe15aa00640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe154109ab0 0x7fe154075500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.278+0000 7fe15aa00640 1 --2- 172.21.15.63:0/2453345969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe154104250 0x7fe154074fc0 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.278+0000 7fe15aa00640 1 -- 172.21.15.63:0/2453345969 >> 172.21.15.63:0/2453345969 conn(0x7fe154079240 msgr2=0x7fe1541017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.278+0000 7fe15aa00640 1 -- 172.21.15.63:0/2453345969 shutdown_connections 2024-05-27T08:13:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:09.278+0000 7fe15aa00640 1 -- 172.21.15.63:0/2453345969 wait complete. 2024-05-27T08:13:10.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:09 smithi086 ceph-mon[34687]: pgmap v1256: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-05-27T08:13:10.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:09 smithi149 ceph-mon[35408]: pgmap v1256: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-05-27T08:13:10.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:09 smithi063 ceph-mon[31613]: pgmap v1256: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-05-27T08:13:11.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:10 smithi086 ceph-mon[34687]: pgmap v1257: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-05-27T08:13:11.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:10 smithi149 ceph-mon[35408]: pgmap v1257: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-05-27T08:13:11.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:10 smithi063 ceph-mon[31613]: pgmap v1257: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 28 op/s 2024-05-27T08:13:14.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:13 smithi086 ceph-mon[34687]: pgmap v1258: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:13:14.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:13 smithi149 ceph-mon[35408]: pgmap v1258: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:13:14.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:13 smithi063 ceph-mon[31613]: pgmap v1258: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:13:16.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:15 smithi086 ceph-mon[34687]: pgmap v1259: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:13:16.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:15 smithi149 ceph-mon[35408]: pgmap v1259: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:13:16.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:15 smithi063 ceph-mon[31613]: pgmap v1259: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:13:17.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:17 smithi086 ceph-mon[34687]: osdmap e160: 12 total, 12 up, 12 in 2024-05-27T08:13:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:17 smithi086 ceph-mon[34687]: pgmap v1261: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.5 MiB/s wr, 12 op/s 2024-05-27T08:13:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:17 smithi149 ceph-mon[35408]: osdmap e160: 12 total, 12 up, 12 in 2024-05-27T08:13:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:17 smithi149 ceph-mon[35408]: pgmap v1261: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.5 MiB/s wr, 12 op/s 2024-05-27T08:13:17.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:17 smithi063 ceph-mon[31613]: osdmap e160: 12 total, 12 up, 12 in 2024-05-27T08:13:17.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:17 smithi063 ceph-mon[31613]: pgmap v1261: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.5 MiB/s wr, 12 op/s 2024-05-27T08:13:18.441 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:13:19.037 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:13:20.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:19 smithi086 ceph-mon[34687]: pgmap v1262: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:13:20.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:19 smithi149 ceph-mon[35408]: pgmap v1262: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:13:20.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:19 smithi063 ceph-mon[31613]: pgmap v1262: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:13:21.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:21 smithi063 ceph-mon[31613]: pgmap v1263: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:13:22.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.176+0000 7f57e5800700 1 -- 172.21.15.63:0/3453887964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57e0106610 msgr2=0x7f57e0106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.176+0000 7f57e5800700 1 --2- 172.21.15.63:0/3453887964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57e0106610 0x7f57e0106a20 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f57d00099c0 tx=0x7f57d0009cd0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.176+0000 7f57e5800700 1 -- 172.21.15.63:0/3453887964 shutdown_connections 2024-05-27T08:13:22.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.176+0000 7f57e5800700 1 --2- 172.21.15.63:0/3453887964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e010b890 0x7f57e01131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.176+0000 7f57e5800700 1 --2- 172.21.15.63:0/3453887964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57e0106610 0x7f57e0106a20 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.176+0000 7f57e5800700 1 --2- 172.21.15.63:0/3453887964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e010af40 0x7f57e010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.176+0000 7f57e5800700 1 -- 172.21.15.63:0/3453887964 >> 172.21.15.63:0/3453887964 conn(0x7f57e0102020 msgr2=0x7f57e0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.176+0000 7f57e5800700 1 -- 172.21.15.63:0/3453887964 shutdown_connections 2024-05-27T08:13:22.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.177+0000 7f57e5800700 1 -- 172.21.15.63:0/3453887964 wait complete. 2024-05-27T08:13:22.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.178+0000 7f57e5800700 1 Processor -- start 2024-05-27T08:13:22.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.178+0000 7f57e5800700 1 -- start start 2024-05-27T08:13:22.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57e5800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0106610 0x7f57e019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57e5800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57e010af40 0x7f57e019d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57e5800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e010b890 0x7f57e01a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57e5800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01163c0 con 0x7f57e010af40 2024-05-27T08:13:22.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57e5800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57e0116240 con 0x7f57e0106610 2024-05-27T08:13:22.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57e5800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57e0116540 con 0x7f57e010b890 2024-05-27T08:13:22.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57e010af40 0x7f57e019d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57e010af40 0x7f57e019d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48600/0 (socket says 172.21.15.63:48600) 2024-05-27T08:13:22.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57df400700 1 -- 172.21.15.63:0/4122982253 learned_addr learned my addr 172.21.15.63:0/4122982253 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:22.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57e4e00700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e010b890 0x7f57e01a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.179+0000 7f57dfe00700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0106610 0x7f57e019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.180+0000 7f57df400700 1 -- 172.21.15.63:0/4122982253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e010b890 msgr2=0x7f57e01a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.180+0000 7f57df400700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e010b890 0x7f57e01a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.180+0000 7f57df400700 1 -- 172.21.15.63:0/4122982253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0106610 msgr2=0x7f57e019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.180+0000 7f57df400700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0106610 0x7f57e019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.180+0000 7f57df400700 1 -- 172.21.15.63:0/4122982253 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e01a1c00 con 0x7f57e010af40 2024-05-27T08:13:22.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.180+0000 7f57df400700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57e010af40 0x7f57e019d2d0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00d980 tx=0x7f57cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.181+0000 7f57dcc00700 1 -- 172.21.15.63:0/4122982253 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc017070 con 0x7f57e010af40 2024-05-27T08:13:22.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.181+0000 7f57e5800700 1 -- 172.21.15.63:0/4122982253 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e01a1ef0 con 0x7f57e010af40 2024-05-27T08:13:22.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.181+0000 7f57dcc00700 1 -- 172.21.15.63:0/4122982253 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57cc00fd20 con 0x7f57e010af40 2024-05-27T08:13:22.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.181+0000 7f57dcc00700 1 -- 172.21.15.63:0/4122982253 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc00e3b0 con 0x7f57e010af40 2024-05-27T08:13:22.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.181+0000 7f57e5800700 1 -- 172.21.15.63:0/4122982253 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e01a9810 con 0x7f57e010af40 2024-05-27T08:13:22.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.186+0000 7f57dcc00700 1 -- 172.21.15.63:0/4122982253 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f57cc00f890 con 0x7f57e010af40 2024-05-27T08:13:22.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.186+0000 7f57f59a3b80 1 Processor -- start 2024-05-27T08:13:22.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.187+0000 7f57f59a3b80 1 -- start start 2024-05-27T08:13:22.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.187+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e79ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.187+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e84ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.187+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a820 0x561b60e87020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.187+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561b60da8e40 con 0x561b60e7a410 2024-05-27T08:13:22.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.187+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b60d989e0 con 0x561b60cfbe40 2024-05-27T08:13:22.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.187+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561b60d9ca20 con 0x561b60e7a820 2024-05-27T08:13:22.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.188+0000 7f57dfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e84ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.188+0000 7f57dfe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e84ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48606/0 (socket says 172.21.15.63:48606) 2024-05-27T08:13:22.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.188+0000 7f57dfe00700 1 -- 172.21.15.63:0/4213679338 learned_addr learned my addr 172.21.15.63:0/4213679338 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:22.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.188+0000 7f57e4e00700 1 --2- 172.21.15.63:0/4213679338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e79ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.188+0000 7f57df400700 1 --2- 172.21.15.63:0/4213679338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a820 0x561b60e87020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.188+0000 7f57dfe00700 1 -- 172.21.15.63:0/4213679338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a820 msgr2=0x561b60e87020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.188+0000 7f57dfe00700 1 --2- 172.21.15.63:0/4213679338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a820 0x561b60e87020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.188+0000 7f57dfe00700 1 -- 172.21.15.63:0/4213679338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 msgr2=0x561b60e79ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.188+0000 7f57dfe00700 1 --2- 172.21.15.63:0/4213679338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e79ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.188+0000 7f57dfe00700 1 -- 172.21.15.63:0/4213679338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b60d61420 con 0x561b60e7a410 2024-05-27T08:13:22.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.189+0000 7f57dcc00700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57c40b29f0 0x7f57c40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.189+0000 7f57dfe00700 1 --2- 172.21.15.63:0/4213679338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e84ad0 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f57d000b580 tx=0x7f57d00096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24576f15bf1ec54a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.189+0000 7f57dcc00700 1 -- 172.21.15.63:0/4122982253 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f57cc0e2460 con 0x7f57e010af40 2024-05-27T08:13:22.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.189+0000 7f57d5600700 1 -- 172.21.15.63:0/4213679338 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0027070 con 0x561b60e7a410 2024-05-27T08:13:22.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.189+0000 7f57e4e00700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57c40b29f0 0x7f57c40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.189+0000 7f57d5600700 1 -- 172.21.15.63:0/4213679338 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57d000b790 con 0x561b60e7a410 2024-05-27T08:13:22.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.189+0000 7f57d5600700 1 -- 172.21.15.63:0/4213679338 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0003970 con 0x561b60e7a410 2024-05-27T08:13:22.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.191+0000 7f57f59a3b80 1 -- 172.21.15.63:0/4213679338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 msgr2=0x561b60e84ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.191+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/4213679338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e84ad0 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f57d000b580 tx=0x7f57d00096e0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.191+0000 7f57e4e00700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57c40b29f0 0x7f57c40b4f00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x561b60e76690 tx=0x7f57d8009100 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.191+0000 7f57f59a3b80 1 -- 172.21.15.63:0/4213679338 shutdown_connections 2024-05-27T08:13:22.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.191+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/4213679338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a820 0x561b60e87020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.191+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/4213679338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e79ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.191+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/4213679338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e84ad0 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.191+0000 7f57f59a3b80 1 -- 172.21.15.63:0/4213679338 >> 172.21.15.63:0/4213679338 conn(0x561b60d5bfc0 msgr2=0x561b60e75d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.191+0000 7f57f59a3b80 1 -- 172.21.15.63:0/4213679338 shutdown_connections 2024-05-27T08:13:22.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.191+0000 7f57f59a3b80 1 -- 172.21.15.63:0/4213679338 wait complete. 2024-05-27T08:13:22.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.192+0000 7f57f59a3b80 1 -- start start 2024-05-27T08:13:22.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.192+0000 7f57f59a3b80 1 Processor -- start 2024-05-27T08:13:22.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.193+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60cfbe40 0x561b60e84540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.193+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60e7a410 0x561b60e84a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.193+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e84fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.193+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561b60c9efa0 con 0x561b60e7a820 2024-05-27T08:13:22.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.193+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b60d39760 con 0x561b60e7a410 2024-05-27T08:13:22.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.193+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561b60e87930 con 0x561b60cfbe40 2024-05-27T08:13:22.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.193+0000 7f57f59a3b80 10 client.0 _ll_register_callbacks cb 0x561b60d42c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:22.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.193+0000 7f57e4e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e84fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.193+0000 7f57e4e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e84fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48608/0 (socket says 172.21.15.63:48608) 2024-05-27T08:13:22.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.193+0000 7f57e4e00700 1 -- 172.21.15.63:0/1704292478 learned_addr learned my addr 172.21.15.63:0/1704292478 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:22.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.194+0000 7f57dfe00700 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60cfbe40 0x561b60e84540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.194+0000 7f57df400700 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60e7a410 0x561b60e84a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.194+0000 7f57e4e00700 1 -- 172.21.15.63:0/1704292478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60cfbe40 msgr2=0x561b60e84540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.194+0000 7f57e4e00700 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60cfbe40 0x561b60e84540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.194+0000 7f57e4e00700 1 -- 172.21.15.63:0/1704292478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60e7a410 msgr2=0x561b60e84a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.194+0000 7f57e4e00700 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60e7a410 0x561b60e84a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.194+0000 7f57e4e00700 1 -- 172.21.15.63:0/1704292478 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b60ce88d0 con 0x561b60e7a820 2024-05-27T08:13:22.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.195+0000 7f57e4e00700 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e84fc0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f57d8007570 tx=0x7f57d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dce3ac765b2dd75 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.195+0000 7f57ca000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:22.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.195+0000 7f57ca000700 1 -- 172.21.15.63:0/1704292478 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8027020 con 0x561b60e7a820 2024-05-27T08:13:22.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.195+0000 7f57ca000700 1 -- 172.21.15.63:0/1704292478 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f57d8023440 con 0x561b60e7a820 2024-05-27T08:13:22.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.195+0000 7f57f59a3b80 1 -- 172.21.15.63:0/1704292478 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d000b790 con 0x561b60e7a820 2024-05-27T08:13:22.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.196+0000 7f57ca000700 1 -- 172.21.15.63:0/1704292478 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f57d8032050 con 0x561b60e7a820 2024-05-27T08:13:22.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.196+0000 7f57f59a3b80 10 client.22989 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:22.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.196+0000 7f57f59a3b80 1 -- 172.21.15.63:0/1704292478 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561b60d69060 con 0x561b60e7a820 2024-05-27T08:13:22.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.196+0000 7f57ca000700 1 -- 172.21.15.63:0/1704292478 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57d000b790 con 0x561b60e7a820 2024-05-27T08:13:22.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.197+0000 7f57ca000700 1 -- 172.21.15.63:0/1704292478 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f57d80166d0 con 0x561b60e7a820 2024-05-27T08:13:22.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.197+0000 7f57f59a3b80 10 client.22989 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:22.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.197+0000 7f57f59a3b80 10 client.22989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:22.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.197+0000 7f57f59a3b80 20 client.22989 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:22.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.197+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561b60e94900 0x561b60e96db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.197+0000 7f57f59a3b80 4 client.22989 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:13:22.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.197+0000 7f57f59a3b80 1 -- 172.21.15.63:0/1704292478 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b60daad90 con 0x561b60e94900 2024-05-27T08:13:22.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.198+0000 7f57dfe00700 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561b60e94900 0x561b60e96db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.199+0000 7f57dfe00700 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561b60e94900 0x561b60e96db0 crc :-1 s=READY pgs=704 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316405167584b1f7 server_cookie=98f6c7337f4757d5 in_seq=0 out_seq=0 2024-05-27T08:13:22.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.199+0000 7f57ca000700 10 client.22989 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:22.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.200+0000 7f57ca000700 1 -- 172.21.15.63:0/1704292478 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561b60daad90 con 0x561b60e94900 2024-05-27T08:13:22.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.200+0000 7f57ca000700 10 client.22989 handle_command_reply: tid=0 2024-05-27T08:13:22.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 client.22989 shutdown 2024-05-27T08:13:22.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 -- 172.21.15.63:0/1704292478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 msgr2=0x561b60e84fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e84fc0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f57d8007570 tx=0x7f57d8004ad0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 -- 172.21.15.63:0/1704292478 shutdown_connections 2024-05-27T08:13:22.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561b60e94900 0x561b60e96db0 crc :-1 s=READY pgs=704 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e84fc0 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60cfbe40 0x561b60e84540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/1704292478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60e7a410 0x561b60e84a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 -- 172.21.15.63:0/1704292478 >> 172.21.15.63:0/1704292478 conn(0x561b60d5bfc0 msgr2=0x561b60e74ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57ca000700 0 client.22989 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:22.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 -- 172.21.15.63:0/1704292478 shutdown_connections 2024-05-27T08:13:22.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.201+0000 7f57f59a3b80 1 -- 172.21.15.63:0/1704292478 wait complete. 2024-05-27T08:13:22.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.202+0000 7f57f59a3b80 20 client.22989 trim_cache size 0 max 16384 2024-05-27T08:13:22.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:21 smithi086 ceph-mon[34687]: pgmap v1263: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:13:22.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:21 smithi149 ceph-mon[35408]: pgmap v1263: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:13:22.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.235+0000 7f57f59a3b80 1 Processor -- start 2024-05-27T08:13:22.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.235+0000 7f57f59a3b80 1 -- start start 2024-05-27T08:13:22.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.235+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e83860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.235+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e869e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.235+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a820 0x561b60e90a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.235+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b4e50 con 0x561b60e7a410 2024-05-27T08:13:22.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.236+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b26e0 con 0x561b60cfbe40 2024-05-27T08:13:22.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.236+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b0550 con 0x561b60e7a820 2024-05-27T08:13:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.236+0000 7f57df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e869e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.236+0000 7f57df400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e869e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48622/0 (socket says 172.21.15.63:48622) 2024-05-27T08:13:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.236+0000 7f57df400700 1 -- 172.21.15.63:0/2254421393 learned_addr learned my addr 172.21.15.63:0/2254421393 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.236+0000 7f57e4e00700 1 --2- 172.21.15.63:0/2254421393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a820 0x561b60e90a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.237+0000 7f57dfe00700 1 --2- 172.21.15.63:0/2254421393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e83860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.237+0000 7f57df400700 1 -- 172.21.15.63:0/2254421393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a820 msgr2=0x561b60e90a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.237+0000 7f57df400700 1 --2- 172.21.15.63:0/2254421393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a820 0x561b60e90a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.237+0000 7f57df400700 1 -- 172.21.15.63:0/2254421393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 msgr2=0x561b60e83860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.237+0000 7f57df400700 1 --2- 172.21.15.63:0/2254421393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e83860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.237+0000 7f57df400700 1 -- 172.21.15.63:0/2254421393 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b60e77350 con 0x561b60e7a410 2024-05-27T08:13:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.237+0000 7f57df400700 1 --2- 172.21.15.63:0/2254421393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e869e0 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0a5400 tx=0x7f57cc0a5340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9652d28dd95fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.237+0000 7f57d4c00700 1 -- 172.21.15.63:0/2254421393 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc020040 con 0x561b60e7a410 2024-05-27T08:13:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.238+0000 7f57d4c00700 1 -- 172.21.15.63:0/2254421393 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57cc0a8e90 con 0x561b60e7a410 2024-05-27T08:13:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.238+0000 7f57d4c00700 1 -- 172.21.15.63:0/2254421393 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0ad730 con 0x561b60e7a410 2024-05-27T08:13:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.239+0000 7f57f59a3b80 1 -- 172.21.15.63:0/2254421393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 msgr2=0x561b60e869e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.239+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/2254421393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e869e0 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0a5400 tx=0x7f57cc0a5340 comp rx=0 tx=0).stop 2024-05-27T08:13:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.239+0000 7f57f59a3b80 1 -- 172.21.15.63:0/2254421393 shutdown_connections 2024-05-27T08:13:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.239+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/2254421393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a820 0x561b60e90a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.239+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/2254421393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e83860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.239+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/2254421393 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a410 0x561b60e869e0 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.239+0000 7f57f59a3b80 1 -- 172.21.15.63:0/2254421393 >> 172.21.15.63:0/2254421393 conn(0x561b60d5bfc0 msgr2=0x561b60e84320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.239+0000 7f57f59a3b80 1 -- 172.21.15.63:0/2254421393 shutdown_connections 2024-05-27T08:13:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.239+0000 7f57f59a3b80 1 -- 172.21.15.63:0/2254421393 wait complete. 2024-05-27T08:13:22.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.240+0000 7f57f59a3b80 1 -- start start 2024-05-27T08:13:22.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.240+0000 7f57f59a3b80 1 Processor -- start 2024-05-27T08:13:22.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e8d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a410 0x561b60e885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57f59a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e88af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b0ba0 con 0x561b60e7a820 2024-05-27T08:13:22.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b2010 con 0x561b60cfbe40 2024-05-27T08:13:22.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57f59a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b1270 con 0x561b60e7a410 2024-05-27T08:13:22.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57f59a3b80 10 client.0 _ll_register_callbacks cb 0x561b60d42c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:22.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57e4e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e88af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57e4e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e88af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48628/0 (socket says 172.21.15.63:48628) 2024-05-27T08:13:22.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57e4e00700 1 -- 172.21.15.63:0/3146251244 learned_addr learned my addr 172.21.15.63:0/3146251244 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:22.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57df400700 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a410 0x561b60e885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.241+0000 7f57dfe00700 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e8d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.242+0000 7f57e4e00700 1 -- 172.21.15.63:0/3146251244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a410 msgr2=0x561b60e885b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.242+0000 7f57e4e00700 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a410 0x561b60e885b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.242+0000 7f57e4e00700 1 -- 172.21.15.63:0/3146251244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 msgr2=0x561b60e8d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.242+0000 7f57e4e00700 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e8d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.242+0000 7f57e4e00700 1 -- 172.21.15.63:0/3146251244 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b60e891d0 con 0x561b60e7a820 2024-05-27T08:13:22.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.242+0000 7f57e4e00700 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e88af0 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f57d8016af0 tx=0x7f57d8007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf5ffdba10a9d7d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.242+0000 7f57ca000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:22.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.243+0000 7f57ca000700 1 -- 172.21.15.63:0/3146251244 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8027040 con 0x561b60e7a820 2024-05-27T08:13:22.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.243+0000 7f57f59a3b80 1 -- 172.21.15.63:0/3146251244 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57cc0a8e90 con 0x561b60e7a820 2024-05-27T08:13:22.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.243+0000 7f57ca000700 1 -- 172.21.15.63:0/3146251244 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f57d8005de0 con 0x561b60e7a820 2024-05-27T08:13:22.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.243+0000 7f57ca000700 1 -- 172.21.15.63:0/3146251244 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f57d8047050 con 0x561b60e7a820 2024-05-27T08:13:22.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.243+0000 7f57f59a3b80 10 client.22995 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:22.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.243+0000 7f57f59a3b80 1 -- 172.21.15.63:0/3146251244 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561b60f07b20 con 0x561b60e7a820 2024-05-27T08:13:22.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.244+0000 7f57ca000700 1 -- 172.21.15.63:0/3146251244 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57cc0a8e90 con 0x561b60e7a820 2024-05-27T08:13:22.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.244+0000 7f57ca000700 1 -- 172.21.15.63:0/3146251244 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f57d8023840 con 0x561b60e7a820 2024-05-27T08:13:22.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.244+0000 7f57f59a3b80 10 client.22995 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:22.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.244+0000 7f57f59a3b80 10 client.22995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:22.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.244+0000 7f57f59a3b80 20 client.22995 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:22.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.244+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561b60f08200 0x561b60f0a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.244+0000 7f57f59a3b80 4 client.22995 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:13:22.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.244+0000 7f57f59a3b80 1 -- 172.21.15.63:0/3146251244 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b60da4570 con 0x561b60f08200 2024-05-27T08:13:22.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.245+0000 7f57dfe00700 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561b60f08200 0x561b60f0a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.246+0000 7f57dfe00700 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561b60f08200 0x561b60f0a620 crc :-1 s=READY pgs=705 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd7838f28a145e7 server_cookie=1246a736effdcf4c in_seq=0 out_seq=0 2024-05-27T08:13:22.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.246+0000 7f57ca000700 10 client.22995 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:22.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57ca000700 1 -- 172.21.15.63:0/3146251244 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561b60da4570 con 0x561b60f08200 2024-05-27T08:13:22.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57ca000700 10 client.22995 handle_command_reply: tid=0 2024-05-27T08:13:22.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 client.22995 shutdown 2024-05-27T08:13:22.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 -- 172.21.15.63:0/3146251244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 msgr2=0x561b60e88af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e88af0 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f57d8016af0 tx=0x7f57d8007ec0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 -- 172.21.15.63:0/3146251244 shutdown_connections 2024-05-27T08:13:22.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561b60f08200 0x561b60f0a620 crc :-1 s=READY pgs=705 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561b60e7a820 0x561b60e88af0 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b60cfbe40 0x561b60e8d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 --2- 172.21.15.63:0/3146251244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b60e7a410 0x561b60e885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57ca000700 0 client.22995 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:22.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 -- 172.21.15.63:0/3146251244 >> 172.21.15.63:0/3146251244 conn(0x561b60d5bfc0 msgr2=0x561b60e835c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 -- 172.21.15.63:0/3146251244 shutdown_connections 2024-05-27T08:13:22.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.247+0000 7f57f59a3b80 1 -- 172.21.15.63:0/3146251244 wait complete. 2024-05-27T08:13:22.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.248+0000 7f57f59a3b80 20 client.22995 trim_cache size 0 max 16384 2024-05-27T08:13:22.249 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:13:22.249 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:13:22.249 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:13:22.249 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:13:22.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 -- 172.21.15.63:0/4122982253 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57c40b29f0 msgr2=0x7f57c40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57c40b29f0 0x7f57c40b4f00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x561b60e76690 tx=0x7f57d8009100 comp rx=0 tx=0).stop 2024-05-27T08:13:22.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 -- 172.21.15.63:0/4122982253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57e010af40 msgr2=0x7f57e019d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57e010af40 0x7f57e019d2d0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00d980 tx=0x7f57cc00dc90 comp rx=0 tx=0).stop 2024-05-27T08:13:22.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 -- 172.21.15.63:0/4122982253 shutdown_connections 2024-05-27T08:13:22.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f57c40b29f0 0x7f57c40b4f00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e010b890 0x7f57e01a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0106610 0x7f57e019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 --2- 172.21.15.63:0/4122982253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57e010af40 0x7f57e019d2d0 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 -- 172.21.15.63:0/4122982253 >> 172.21.15.63:0/4122982253 conn(0x7f57e0102020 msgr2=0x7f57e010a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 -- 172.21.15.63:0/4122982253 shutdown_connections 2024-05-27T08:13:22.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:22.249+0000 7f57e5800700 1 -- 172.21.15.63:0/4122982253 wait complete. 2024-05-27T08:13:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.530+0000 7f5e3be00640 1 -- 172.21.15.63:0/2992222298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c10b670 msgr2=0x7f5e3c10da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.530+0000 7f5e3be00640 1 --2- 172.21.15.63:0/2992222298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c10b670 0x7f5e3c10da90 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009900 tx=0x7f5e2802a510 comp rx=0 tx=0).stop 2024-05-27T08:13:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.530+0000 7f5e3be00640 1 -- 172.21.15.63:0/2992222298 shutdown_connections 2024-05-27T08:13:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.530+0000 7f5e3be00640 1 --2- 172.21.15.63:0/2992222298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e3c10e020 0x7f5e3c110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.530+0000 7f5e3be00640 1 --2- 172.21.15.63:0/2992222298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c10b670 0x7f5e3c10da90 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.530+0000 7f5e3be00640 1 --2- 172.21.15.63:0/2992222298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e3c104c10 0x7f5e3c105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.530+0000 7f5e3be00640 1 -- 172.21.15.63:0/2992222298 >> 172.21.15.63:0/2992222298 conn(0x7f5e3c0fe210 msgr2=0x7f5e3c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.530+0000 7f5e3be00640 1 -- 172.21.15.63:0/2992222298 shutdown_connections 2024-05-27T08:13:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.530+0000 7f5e3be00640 1 -- 172.21.15.63:0/2992222298 wait complete. 2024-05-27T08:13:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.532+0000 7f5e3be00640 1 Processor -- start 2024-05-27T08:13:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.532+0000 7f5e3be00640 1 -- start start 2024-05-27T08:13:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e3c104c10 0x7f5e3c1a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c10b670 0x7f5e3c1a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e3c10e020 0x7f5e3c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c113b50 con 0x7f5e3c10b670 2024-05-27T08:13:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1139d0 con 0x7f5e3c104c10 2024-05-27T08:13:22.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c113cd0 con 0x7f5e3c10e020 2024-05-27T08:13:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c10b670 0x7f5e3c1a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c10b670 0x7f5e3c1a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48648/0 (socket says 172.21.15.63:48648) 2024-05-27T08:13:22.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3a000640 1 -- 172.21.15.63:0/3204451187 learned_addr learned my addr 172.21.15.63:0/3204451187 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3b400640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e3c10e020 0x7f5e3c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.533+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e3c104c10 0x7f5e3c1a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.534+0000 7f5e3a000640 1 -- 172.21.15.63:0/3204451187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e3c10e020 msgr2=0x7f5e3c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.534+0000 7f5e3a000640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e3c10e020 0x7f5e3c19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.534+0000 7f5e3a000640 1 -- 172.21.15.63:0/3204451187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e3c104c10 msgr2=0x7f5e3c1a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.534+0000 7f5e3a000640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e3c104c10 0x7f5e3c1a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.534+0000 7f5e3a000640 1 -- 172.21.15.63:0/3204451187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c19ccd0 con 0x7f5e3c10b670 2024-05-27T08:13:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.534+0000 7f5e3a000640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c10b670 0x7f5e3c1a2a50 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009230 tx=0x7f5e28004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.534+0000 7f5e33400640 1 -- 172.21.15.63:0/3204451187 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2803f030 con 0x7f5e3c10b670 2024-05-27T08:13:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.534+0000 7f5e33400640 1 -- 172.21.15.63:0/3204451187 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e28003a80 con 0x7f5e3c10b670 2024-05-27T08:13:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.534+0000 7f5e3be00640 1 -- 172.21.15.63:0/3204451187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c19cf60 con 0x7f5e3c10b670 2024-05-27T08:13:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.535+0000 7f5e33400640 1 -- 172.21.15.63:0/3204451187 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28035070 con 0x7f5e3c10b670 2024-05-27T08:13:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.535+0000 7f5e3be00640 1 -- 172.21.15.63:0/3204451187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c19d450 con 0x7f5e3c10b670 2024-05-27T08:13:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.539+0000 7f5e489a8740 1 Processor -- start 2024-05-27T08:13:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.539+0000 7f5e489a8740 1 -- start start 2024-05-27T08:13:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.540+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6bd1b0 0x563a5ba5fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.540+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba5ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.540+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b69c4d0 0x563a5ba604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.540+0000 7f5e489a8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563a5b9d0b90 con 0x563a5b6942f0 2024-05-27T08:13:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.540+0000 7f5e489a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a5b9c0730 con 0x563a5b69c4d0 2024-05-27T08:13:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.540+0000 7f5e489a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563a5b9c4770 con 0x563a5b6bd1b0 2024-05-27T08:13:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.540+0000 7f5e3aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba5ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.540+0000 7f5e3aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba5ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48650/0 (socket says 172.21.15.63:48650) 2024-05-27T08:13:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.540+0000 7f5e3aa00640 1 -- 172.21.15.63:0/3863392550 learned_addr learned my addr 172.21.15.63:0/3863392550 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.540+0000 7f5e3b400640 1 --2- 172.21.15.63:0/3863392550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6bd1b0 0x563a5ba5fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.541 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.541+0000 7f5e33400640 1 -- 172.21.15.63:0/3204451187 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5e28042030 con 0x7f5e3c10b670 2024-05-27T08:13:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.541+0000 7f5e3aa00640 1 -- 172.21.15.63:0/3863392550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6bd1b0 msgr2=0x563a5ba5fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.541+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/3863392550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6bd1b0 0x563a5ba5fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.541+0000 7f5e3aa00640 1 -- 172.21.15.63:0/3863392550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b69c4d0 msgr2=0x563a5ba604e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:13:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.541+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/3863392550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b69c4d0 0x563a5ba604e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.541+0000 7f5e3aa00640 1 -- 172.21.15.63:0/3863392550 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5b9598b0 con 0x563a5b6942f0 2024-05-27T08:13:22.542 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.541+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/3863392550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba5ffa0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00bcb0 tx=0x7f5e2c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab097f454cec9ad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.541+0000 7f5e27e00640 1 -- 172.21.15.63:0/3863392550 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c015070 con 0x563a5b6942f0 2024-05-27T08:13:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.542+0000 7f5e27e00640 1 -- 172.21.15.63:0/3863392550 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e2c01a8f0 con 0x563a5b6942f0 2024-05-27T08:13:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e489a8740 1 -- 172.21.15.63:0/3863392550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 msgr2=0x563a5ba5ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e27e00640 1 -- 172.21.15.63:0/3863392550 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c011860 con 0x563a5b6942f0 2024-05-27T08:13:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e489a8740 1 --2- 172.21.15.63:0/3863392550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba5ffa0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00bcb0 tx=0x7f5e2c00f730 comp rx=0 tx=0).stop 2024-05-27T08:13:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e33400640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5e200b2950 0x7f5e200b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e33400640 1 -- 172.21.15.63:0/3204451187 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f5e281031f0 con 0x7f5e3c10b670 2024-05-27T08:13:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e489a8740 1 -- 172.21.15.63:0/3863392550 shutdown_connections 2024-05-27T08:13:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e489a8740 1 --2- 172.21.15.63:0/3863392550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b69c4d0 0x563a5ba604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e489a8740 1 --2- 172.21.15.63:0/3863392550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba5ffa0 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e489a8740 1 --2- 172.21.15.63:0/3863392550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6bd1b0 0x563a5ba5fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e489a8740 1 -- 172.21.15.63:0/3863392550 >> 172.21.15.63:0/3863392550 conn(0x563a5b6b1300 msgr2=0x563a5ba5c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e3b400640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5e200b2950 0x7f5e200b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.543+0000 7f5e489a8740 1 -- 172.21.15.63:0/3863392550 shutdown_connections 2024-05-27T08:13:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.544+0000 7f5e489a8740 1 -- 172.21.15.63:0/3863392550 wait complete. 2024-05-27T08:13:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.544+0000 7f5e3b400640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5e200b2950 0x7f5e200b4e60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x563a5ba5cbd0 tx=0x7f5e34007380 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.544+0000 7f5e489a8740 1 -- start start 2024-05-27T08:13:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.545+0000 7f5e489a8740 1 Processor -- start 2024-05-27T08:13:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.545+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba69bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.545+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b69c4d0 0x563a5ba6a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.545+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6bd1b0 0x563a5ba75f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.545+0000 7f5e489a8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563a5b6a6790 con 0x563a5b6942f0 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.545+0000 7f5e489a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a5b968bc0 con 0x563a5b69c4d0 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.545+0000 7f5e489a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563a5ba6cbe0 con 0x563a5b6bd1b0 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.545+0000 7f5e489a8740 10 client.0 _ll_register_callbacks cb 0x563a5b946400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.546+0000 7f5e3aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba69bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.546+0000 7f5e3aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba69bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48662/0 (socket says 172.21.15.63:48662) 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.546+0000 7f5e3aa00640 1 -- 172.21.15.63:0/899700867 learned_addr learned my addr 172.21.15.63:0/899700867 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.546+0000 7f5e3a000640 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b69c4d0 0x563a5ba6a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.546+0000 7f5e3b400640 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6bd1b0 0x563a5ba75f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.546+0000 7f5e3aa00640 1 -- 172.21.15.63:0/899700867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6bd1b0 msgr2=0x563a5ba75f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.546+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6bd1b0 0x563a5ba75f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.546+0000 7f5e3aa00640 1 -- 172.21.15.63:0/899700867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b69c4d0 msgr2=0x563a5ba6a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.546+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b69c4d0 0x563a5ba6a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.546+0000 7f5e3aa00640 1 -- 172.21.15.63:0/899700867 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5b6d5210 con 0x563a5b6942f0 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.547+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba69bb0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c019000 tx=0x7f5e2c01a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81eec52cbe2372b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.547+0000 7f5e24c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.547+0000 7f5e24c00640 1 -- 172.21.15.63:0/899700867 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c015040 con 0x563a5b6942f0 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.547+0000 7f5e489a8740 1 -- 172.21.15.63:0/899700867 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e2c01a8f0 con 0x563a5b6942f0 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.547+0000 7f5e24c00640 1 -- 172.21.15.63:0/899700867 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f5e2c01aca0 con 0x563a5b6942f0 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.547+0000 7f5e24c00640 1 -- 172.21.15.63:0/899700867 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5e2c030050 con 0x563a5b6942f0 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.548+0000 7f5e489a8740 10 client.23007 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.548+0000 7f5e489a8740 1 -- 172.21.15.63:0/899700867 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563a5b7a1620 con 0x563a5b6942f0 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.548+0000 7f5e24c00640 1 -- 172.21.15.63:0/899700867 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e2c01a8f0 con 0x563a5b6942f0 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.548+0000 7f5e24c00640 1 -- 172.21.15.63:0/899700867 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5e2c02b030 con 0x563a5b6942f0 2024-05-27T08:13:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.548+0000 7f5e489a8740 10 client.23007 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.548+0000 7f5e489a8740 10 client.23007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.548+0000 7f5e489a8740 20 client.23007 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.548+0000 7f5e489a8740 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563a5b67bd80 0x563a5ba7bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.548+0000 7f5e489a8740 4 client.23007 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:13:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.548+0000 7f5e489a8740 1 -- 172.21.15.63:0/899700867 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a5b9d2ae0 con 0x563a5b67bd80 2024-05-27T08:13:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.549+0000 7f5e3a000640 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563a5b67bd80 0x563a5ba7bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.550+0000 7f5e3a000640 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563a5b67bd80 0x563a5ba7bd90 crc :-1 s=READY pgs=706 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89cf47b0e79ca511 server_cookie=92c5b8bf9ab3c7f3 in_seq=0 out_seq=0 2024-05-27T08:13:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.550+0000 7f5e24c00640 10 client.23007 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e24c00640 1 -- 172.21.15.63:0/899700867 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563a5b9d2ae0 con 0x563a5b67bd80 2024-05-27T08:13:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e24c00640 10 client.23007 handle_command_reply: tid=0 2024-05-27T08:13:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e489a8740 1 client.23007 shutdown 2024-05-27T08:13:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e489a8740 1 -- 172.21.15.63:0/899700867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 msgr2=0x563a5ba69bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e489a8740 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba69bb0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c019000 tx=0x7f5e2c01a4e0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e489a8740 1 -- 172.21.15.63:0/899700867 shutdown_connections 2024-05-27T08:13:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e489a8740 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563a5b67bd80 0x563a5ba7bd90 crc :-1 s=READY pgs=706 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e489a8740 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6bd1b0 0x563a5ba75f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e489a8740 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b69c4d0 0x563a5ba6a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e489a8740 1 --2- 172.21.15.63:0/899700867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6942f0 0x563a5ba69bb0 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e24c00640 0 client.23007 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.552+0000 7f5e489a8740 1 -- 172.21.15.63:0/899700867 >> 172.21.15.63:0/899700867 conn(0x563a5b6b1300 msgr2=0x563a5ba61780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.553+0000 7f5e489a8740 1 -- 172.21.15.63:0/899700867 shutdown_connections 2024-05-27T08:13:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.553+0000 7f5e489a8740 1 -- 172.21.15.63:0/899700867 wait complete. 2024-05-27T08:13:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.553+0000 7f5e489a8740 20 client.23007 trim_cache size 0 max 16384 2024-05-27T08:13:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.575+0000 7f5e489a8740 1 Processor -- start 2024-05-27T08:13:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.575+0000 7f5e489a8740 1 -- start start 2024-05-27T08:13:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6942f0 0x563a5ba6c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b69c4d0 0x563a5ba60410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b6bd1b0 0x563a5ba75ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e489a8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1b5b00 con 0x563a5b69c4d0 2024-05-27T08:13:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e489a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1b33d0 con 0x563a5b6bd1b0 2024-05-27T08:13:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e489a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1b1240 con 0x563a5b6942f0 2024-05-27T08:13:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e3a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b69c4d0 0x563a5ba60410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e3a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b69c4d0 0x563a5ba60410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48674/0 (socket says 172.21.15.63:48674) 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e3a000640 1 -- 172.21.15.63:0/3673612506 learned_addr learned my addr 172.21.15.63:0/3673612506 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/3673612506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6942f0 0x563a5ba6c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.576+0000 7f5e3b400640 1 --2- 172.21.15.63:0/3673612506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b6bd1b0 0x563a5ba75ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.577+0000 7f5e3a000640 1 -- 172.21.15.63:0/3673612506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6942f0 msgr2=0x563a5ba6c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.577+0000 7f5e3a000640 1 --2- 172.21.15.63:0/3673612506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6942f0 0x563a5ba6c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.577+0000 7f5e3a000640 1 -- 172.21.15.63:0/3673612506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b6bd1b0 msgr2=0x563a5ba75ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.577+0000 7f5e3a000640 1 --2- 172.21.15.63:0/3673612506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b6bd1b0 0x563a5ba75ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.577+0000 7f5e3a000640 1 -- 172.21.15.63:0/3673612506 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5ba5d360 con 0x563a5b69c4d0 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.577+0000 7f5e3a000640 1 --2- 172.21.15.63:0/3673612506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b69c4d0 0x563a5ba60410 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f5e280c7380 tx=0x7f5e2804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea32d95637ed9739 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.577+0000 7f5e27400640 1 -- 172.21.15.63:0/3673612506 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28049040 con 0x563a5b69c4d0 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.577+0000 7f5e27400640 1 -- 172.21.15.63:0/3673612506 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e280e4070 con 0x563a5b69c4d0 2024-05-27T08:13:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.577+0000 7f5e27400640 1 -- 172.21.15.63:0/3673612506 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e280c63a0 con 0x563a5b69c4d0 2024-05-27T08:13:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.579+0000 7f5e489a8740 1 -- 172.21.15.63:0/3673612506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b69c4d0 msgr2=0x563a5ba60410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.579+0000 7f5e489a8740 1 --2- 172.21.15.63:0/3673612506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b69c4d0 0x563a5ba60410 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f5e280c7380 tx=0x7f5e2804b000 comp rx=0 tx=0).stop 2024-05-27T08:13:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.579+0000 7f5e489a8740 1 -- 172.21.15.63:0/3673612506 shutdown_connections 2024-05-27T08:13:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.579+0000 7f5e489a8740 1 --2- 172.21.15.63:0/3673612506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b6bd1b0 0x563a5ba75ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.579+0000 7f5e489a8740 1 --2- 172.21.15.63:0/3673612506 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b69c4d0 0x563a5ba60410 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.579+0000 7f5e489a8740 1 --2- 172.21.15.63:0/3673612506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b6942f0 0x563a5ba6c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.579+0000 7f5e489a8740 1 -- 172.21.15.63:0/3673612506 >> 172.21.15.63:0/3673612506 conn(0x563a5b6b1300 msgr2=0x563a5ba69440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.579+0000 7f5e489a8740 1 -- 172.21.15.63:0/3673612506 shutdown_connections 2024-05-27T08:13:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.579+0000 7f5e489a8740 1 -- 172.21.15.63:0/3673612506 wait complete. 2024-05-27T08:13:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.579+0000 7f5e489a8740 1 -- start start 2024-05-27T08:13:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.580+0000 7f5e489a8740 1 Processor -- start 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.580+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b6942f0 0x563a5ba7c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.580+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b69c4d0 0x563a5ba7c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e489a8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6bd1b0 0x563a5badc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e489a8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1b1890 con 0x563a5b6bd1b0 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e489a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1b2d00 con 0x563a5b6942f0 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e489a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1b1f60 con 0x563a5b69c4d0 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e489a8740 10 client.0 _ll_register_callbacks cb 0x7f5e2c00bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6bd1b0 0x563a5badc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6bd1b0 0x563a5badc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48680/0 (socket says 172.21.15.63:48680) 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e3b400640 1 -- 172.21.15.63:0/1465016582 learned_addr learned my addr 172.21.15.63:0/1465016582 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b6942f0 0x563a5ba7c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e3a000640 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b69c4d0 0x563a5ba7c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e3b400640 1 -- 172.21.15.63:0/1465016582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b69c4d0 msgr2=0x563a5ba7c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e3b400640 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b69c4d0 0x563a5ba7c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e3b400640 1 -- 172.21.15.63:0/1465016582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b6942f0 msgr2=0x563a5ba7c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e3b400640 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b6942f0 0x563a5ba7c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.581+0000 7f5e3b400640 1 -- 172.21.15.63:0/1465016582 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5ba5f7f0 con 0x563a5b6bd1b0 2024-05-27T08:13:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.582+0000 7f5e3b400640 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6bd1b0 0x563a5badc1a0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f5e34020e90 tx=0x7f5e34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0744a5303cf702 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.582+0000 7f5e24c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.582+0000 7f5e24c00640 1 -- 172.21.15.63:0/1465016582 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34003970 con 0x563a5b6bd1b0 2024-05-27T08:13:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.582+0000 7f5e489a8740 1 -- 172.21.15.63:0/1465016582 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e280e4070 con 0x563a5b6bd1b0 2024-05-27T08:13:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.582+0000 7f5e24c00640 1 -- 172.21.15.63:0/1465016582 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f5e34003b10 con 0x563a5b6bd1b0 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.582+0000 7f5e24c00640 1 -- 172.21.15.63:0/1465016582 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5e34031050 con 0x563a5b6bd1b0 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.583+0000 7f5e489a8740 10 client.23013 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.583+0000 7f5e489a8740 1 -- 172.21.15.63:0/1465016582 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563a5bae3880 con 0x563a5b6bd1b0 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.583+0000 7f5e24c00640 1 -- 172.21.15.63:0/1465016582 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e280e4070 con 0x563a5b6bd1b0 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.583+0000 7f5e24c00640 1 -- 172.21.15.63:0/1465016582 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5e34029430 con 0x563a5b6bd1b0 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.583+0000 7f5e489a8740 10 client.23013 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.583+0000 7f5e489a8740 10 client.23013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.583+0000 7f5e489a8740 20 client.23013 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.583+0000 7f5e489a8740 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563a5b6511c0 0x563a5bae6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.583+0000 7f5e489a8740 4 client.23013 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.583+0000 7f5e489a8740 1 -- 172.21.15.63:0/1465016582 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a5b9cc2c0 con 0x563a5b6511c0 2024-05-27T08:13:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.584+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563a5b6511c0 0x563a5bae6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.585+0000 7f5e3aa00640 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563a5b6511c0 0x563a5bae6060 crc :-1 s=READY pgs=707 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f36c7e46e2213ff server_cookie=e40888b7c8712795 in_seq=0 out_seq=0 2024-05-27T08:13:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.585+0000 7f5e24c00640 10 client.23013 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e24c00640 1 -- 172.21.15.63:0/1465016582 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563a5b9cc2c0 con 0x563a5b6511c0 2024-05-27T08:13:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e24c00640 10 client.23013 handle_command_reply: tid=0 2024-05-27T08:13:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e489a8740 1 client.23013 shutdown 2024-05-27T08:13:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e489a8740 1 -- 172.21.15.63:0/1465016582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6bd1b0 msgr2=0x563a5badc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e489a8740 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6bd1b0 0x563a5badc1a0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f5e34020e90 tx=0x7f5e34004a40 comp rx=0 tx=0).stop 2024-05-27T08:13:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e489a8740 1 -- 172.21.15.63:0/1465016582 shutdown_connections 2024-05-27T08:13:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e489a8740 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x563a5b6511c0 0x563a5bae6060 crc :-1 s=READY pgs=707 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e489a8740 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a5b6bd1b0 0x563a5badc1a0 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e489a8740 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a5b69c4d0 0x563a5ba7c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e489a8740 1 --2- 172.21.15.63:0/1465016582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5b6942f0 0x563a5ba7c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e489a8740 1 -- 172.21.15.63:0/1465016582 >> 172.21.15.63:0/1465016582 conn(0x563a5b6b1300 msgr2=0x563a5ba65140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.621+0000 7f5e24c00640 0 client.23013 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.622+0000 7f5e489a8740 1 -- 172.21.15.63:0/1465016582 shutdown_connections 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.622+0000 7f5e489a8740 1 -- 172.21.15.63:0/1465016582 wait complete. 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.622+0000 7f5e489a8740 20 client.23013 trim_cache size 0 max 16384 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:13:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:13:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 -- 172.21.15.63:0/3204451187 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5e200b2950 msgr2=0x7f5e200b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5e200b2950 0x7f5e200b4e60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x563a5ba5cbd0 tx=0x7f5e34007380 comp rx=0 tx=0).stop 2024-05-27T08:13:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 -- 172.21.15.63:0/3204451187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c10b670 msgr2=0x7f5e3c1a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c10b670 0x7f5e3c1a2a50 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009230 tx=0x7f5e28004270 comp rx=0 tx=0).stop 2024-05-27T08:13:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 -- 172.21.15.63:0/3204451187 shutdown_connections 2024-05-27T08:13:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5e200b2950 0x7f5e200b4e60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e3c10e020 0x7f5e3c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e3c10b670 0x7f5e3c1a2a50 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 --2- 172.21.15.63:0/3204451187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e3c104c10 0x7f5e3c1a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 -- 172.21.15.63:0/3204451187 >> 172.21.15.63:0/3204451187 conn(0x7f5e3c0fe210 msgr2=0x7f5e3c109e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 -- 172.21.15.63:0/3204451187 shutdown_connections 2024-05-27T08:13:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:22.623+0000 7f5e3be00640 1 -- 172.21.15.63:0/3204451187 wait complete. 2024-05-27T08:13:23.246 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:13:23.247 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:13:23.802 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:13:24.137 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:23 smithi063 ceph-mon[31613]: pgmap v1264: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T08:13:24.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:23 smithi086 ceph-mon[34687]: pgmap v1264: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T08:13:24.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:23 smithi149 ceph-mon[35408]: pgmap v1264: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T08:13:25.948 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:25 smithi063 ceph-mon[31613]: pgmap v1265: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T08:13:26.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:25 smithi086 ceph-mon[34687]: pgmap v1265: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T08:13:26.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:25 smithi149 ceph-mon[35408]: pgmap v1265: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.4 MiB/s wr, 8 op/s 2024-05-27T08:13:26.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.924+0000 7fae9e800700 1 -- 172.21.15.63:0/917993892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae98106560 msgr2=0x7fae98106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:26.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.924+0000 7fae9e800700 1 --2- 172.21.15.63:0/917993892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae98106560 0x7fae98106970 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7fae84009990 tx=0x7fae84009ca0 comp rx=0 tx=0).stop 2024-05-27T08:13:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.924+0000 7fae9e800700 1 -- 172.21.15.63:0/917993892 shutdown_connections 2024-05-27T08:13:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.924+0000 7fae9e800700 1 --2- 172.21.15.63:0/917993892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae98106eb0 0x7fae98113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.924+0000 7fae9e800700 1 --2- 172.21.15.63:0/917993892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9810c5a0 0x7fae9810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.924+0000 7fae9e800700 1 --2- 172.21.15.63:0/917993892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae98106560 0x7fae98106970 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.924+0000 7fae9e800700 1 -- 172.21.15.63:0/917993892 >> 172.21.15.63:0/917993892 conn(0x7fae98102040 msgr2=0x7fae98104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.925+0000 7fae9e800700 1 -- 172.21.15.63:0/917993892 shutdown_connections 2024-05-27T08:13:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.925+0000 7fae9e800700 1 -- 172.21.15.63:0/917993892 wait complete. 2024-05-27T08:13:26.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.926+0000 7fae9e800700 1 Processor -- start 2024-05-27T08:13:26.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.926+0000 7fae9e800700 1 -- start start 2024-05-27T08:13:26.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.927+0000 7fae9e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae98106560 0x7fae9819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:26.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.927+0000 7fae9e800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae98106eb0 0x7fae9819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:26.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.927+0000 7fae9e800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae9810c5a0 0x7fae981a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:26.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.927+0000 7fae9e800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae98116530 con 0x7fae9810c5a0 2024-05-27T08:13:26.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.927+0000 7fae9e800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae981163b0 con 0x7fae98106560 2024-05-27T08:13:26.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.927+0000 7fae9e800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fae981166b0 con 0x7fae98106eb0 2024-05-27T08:13:26.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.927+0000 7fae97e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae9810c5a0 0x7fae981a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:26.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.928+0000 7fae97e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae9810c5a0 0x7fae981a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48704/0 (socket says 172.21.15.63:48704) 2024-05-27T08:13:26.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.928+0000 7fae97e00700 1 -- 172.21.15.63:0/4207474572 learned_addr learned my addr 172.21.15.63:0/4207474572 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:26.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.928+0000 7fae97400700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae98106560 0x7fae9819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:26.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.928+0000 7fae96a00700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae98106eb0 0x7fae9819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:26.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.928+0000 7fae97e00700 1 -- 172.21.15.63:0/4207474572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae98106eb0 msgr2=0x7fae9819d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:26.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.928+0000 7fae97e00700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae98106eb0 0x7fae9819d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:26.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.928+0000 7fae97e00700 1 -- 172.21.15.63:0/4207474572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae98106560 msgr2=0x7fae9819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:26.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.928+0000 7fae97e00700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae98106560 0x7fae9819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:26.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.928+0000 7fae97e00700 1 -- 172.21.15.63:0/4207474572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae981a1df0 con 0x7fae9810c5a0 2024-05-27T08:13:26.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.929+0000 7fae97e00700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae9810c5a0 0x7fae981a1750 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7fae90007ee0 tx=0x7fae9000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:26.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.929+0000 7fae8fe00700 1 -- 172.21.15.63:0/4207474572 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9000dd20 con 0x7fae9810c5a0 2024-05-27T08:13:26.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.929+0000 7fae9e800700 1 -- 172.21.15.63:0/4207474572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae981a20e0 con 0x7fae9810c5a0 2024-05-27T08:13:26.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.930+0000 7fae8fe00700 1 -- 172.21.15.63:0/4207474572 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae90004020 con 0x7fae9810c5a0 2024-05-27T08:13:26.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.930+0000 7fae8fe00700 1 -- 172.21.15.63:0/4207474572 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90015660 con 0x7fae9810c5a0 2024-05-27T08:13:26.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.930+0000 7fae9e800700 1 -- 172.21.15.63:0/4207474572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981a9a00 con 0x7fae9810c5a0 2024-05-27T08:13:26.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.931+0000 7fae9e800700 1 -- 172.21.15.63:0/4207474572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae78005330 con 0x7fae9810c5a0 2024-05-27T08:13:26.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.935+0000 7fae8fe00700 1 -- 172.21.15.63:0/4207474572 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fae90003a00 con 0x7fae9810c5a0 2024-05-27T08:13:26.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.938+0000 7fae8fe00700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fae7c0b2a70 0x7fae7c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:26.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.938+0000 7fae8fe00700 1 -- 172.21.15.63:0/4207474572 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7fae900e2ac0 con 0x7fae9810c5a0 2024-05-27T08:13:26.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.938+0000 7fae97400700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fae7c0b2a70 0x7fae7c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:26.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.939+0000 7fae97400700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fae7c0b2a70 0x7fae7c0b4f80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fae88005950 tx=0x7fae8800b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:26.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:26.940+0000 7fae8fe00700 1 -- 172.21.15.63:0/4207474572 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae90010070 con 0x7fae9810c5a0 2024-05-27T08:13:27.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.271+0000 7fae9e800700 1 -- 172.21.15.63:0/4207474572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae78005620 con 0x7fae9810c5a0 2024-05-27T08:13:27.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.273+0000 7fae8fe00700 1 -- 172.21.15.63:0/4207474572 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7fae900aa080 con 0x7fae9810c5a0 2024-05-27T08:13:27.274 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:13:27.274 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:13:27.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.276+0000 7fae9e800700 1 -- 172.21.15.63:0/4207474572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fae7c0b2a70 msgr2=0x7fae7c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:27.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.276+0000 7fae9e800700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fae7c0b2a70 0x7fae7c0b4f80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fae88005950 tx=0x7fae8800b410 comp rx=0 tx=0).stop 2024-05-27T08:13:27.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.276+0000 7fae9e800700 1 -- 172.21.15.63:0/4207474572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae9810c5a0 msgr2=0x7fae981a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:27.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.277+0000 7fae9e800700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae9810c5a0 0x7fae981a1750 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7fae90007ee0 tx=0x7fae9000d3b0 comp rx=0 tx=0).stop 2024-05-27T08:13:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.277+0000 7fae9e800700 1 -- 172.21.15.63:0/4207474572 shutdown_connections 2024-05-27T08:13:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.277+0000 7fae9e800700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fae7c0b2a70 0x7fae7c0b4f80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.277+0000 7fae9e800700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae9810c5a0 0x7fae981a1750 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.277+0000 7fae9e800700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae98106560 0x7fae9819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.277+0000 7fae9e800700 1 --2- 172.21.15.63:0/4207474572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae98106eb0 0x7fae9819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.277+0000 7fae9e800700 1 -- 172.21.15.63:0/4207474572 >> 172.21.15.63:0/4207474572 conn(0x7fae98102040 msgr2=0x7fae98103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.277+0000 7fae9e800700 1 -- 172.21.15.63:0/4207474572 shutdown_connections 2024-05-27T08:13:27.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:27.277+0000 7fae9e800700 1 -- 172.21.15.63:0/4207474572 wait complete. 2024-05-27T08:13:27.279 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:13:27.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:27 smithi063 ceph-mon[31613]: pgmap v1266: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.3 MiB/s wr, 16 op/s 2024-05-27T08:13:27.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:27 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4207474572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:13:28.168 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:13:28.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:27 smithi086 ceph-mon[34687]: pgmap v1266: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.3 MiB/s wr, 16 op/s 2024-05-27T08:13:28.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:27 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4207474572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:13:28.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:27 smithi149 ceph-mon[35408]: pgmap v1266: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.3 MiB/s wr, 16 op/s 2024-05-27T08:13:28.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:27 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4207474572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:13:28.685 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:13:30.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:29 smithi063 ceph-mon[31613]: pgmap v1267: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:13:30.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:29 smithi086 ceph-mon[34687]: pgmap v1267: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:13:30.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:29 smithi149 ceph-mon[35408]: pgmap v1267: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:13:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.197+0000 7f3f1b600700 1 -- 172.21.15.63:0/3233859013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f14108f10 msgr2=0x7f3f14109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.197+0000 7f3f1b600700 1 --2- 172.21.15.63:0/3233859013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f14108f10 0x7f3f14109320 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f3f000099c0 tx=0x7f3f00009cd0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.197+0000 7f3f1b600700 1 -- 172.21.15.63:0/3233859013 shutdown_connections 2024-05-27T08:13:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.197+0000 7f3f1b600700 1 --2- 172.21.15.63:0/3233859013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14109860 0x7f3f141111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.197+0000 7f3f1b600700 1 --2- 172.21.15.63:0/3233859013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f141045e0 0x7f3f141049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.197+0000 7f3f1b600700 1 --2- 172.21.15.63:0/3233859013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f14108f10 0x7f3f14109320 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.197+0000 7f3f1b600700 1 -- 172.21.15.63:0/3233859013 >> 172.21.15.63:0/3233859013 conn(0x7f3f14100030 msgr2=0x7f3f14102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.197+0000 7f3f1b600700 1 -- 172.21.15.63:0/3233859013 shutdown_connections 2024-05-27T08:13:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.197+0000 7f3f1b600700 1 -- 172.21.15.63:0/3233859013 wait complete. 2024-05-27T08:13:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.199+0000 7f3f1b600700 1 Processor -- start 2024-05-27T08:13:31.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.199+0000 7f3f1b600700 1 -- start start 2024-05-27T08:13:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.199+0000 7f3f1b600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f141045e0 0x7f3f1419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.199+0000 7f3f1b600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14108f10 0x7f3f1419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.199+0000 7f3f1b600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14109860 0x7f3f141a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.199+0000 7f3f1b600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f14114370 con 0x7f3f141045e0 2024-05-27T08:13:31.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.199+0000 7f3f1b600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f141141f0 con 0x7f3f14108f10 2024-05-27T08:13:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.199+0000 7f3f1b600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3f141144f0 con 0x7f3f14109860 2024-05-27T08:13:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.200+0000 7f3f18c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f141045e0 0x7f3f1419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.200+0000 7f3f18c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f141045e0 0x7f3f1419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53134/0 (socket says 172.21.15.63:53134) 2024-05-27T08:13:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.200+0000 7f3f18c00700 1 -- 172.21.15.63:0/2971931146 learned_addr learned my addr 172.21.15.63:0/2971931146 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:31.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.200+0000 7f3f13e00700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14108f10 0x7f3f1419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.200+0000 7f3f19600700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14109860 0x7f3f141a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.201+0000 7f3f18c00700 1 -- 172.21.15.63:0/2971931146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14109860 msgr2=0x7f3f141a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.201+0000 7f3f18c00700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14109860 0x7f3f141a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.201+0000 7f3f18c00700 1 -- 172.21.15.63:0/2971931146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14108f10 msgr2=0x7f3f1419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.201+0000 7f3f18c00700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14108f10 0x7f3f1419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.201+0000 7f3f18c00700 1 -- 172.21.15.63:0/2971931146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f141a1f10 con 0x7f3f141045e0 2024-05-27T08:13:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.201+0000 7f3f18c00700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f141045e0 0x7f3f1419d040 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800d950 tx=0x7f3f0800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.201+0000 7f3f11600700 1 -- 172.21.15.63:0/2971931146 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08016070 con 0x7f3f141045e0 2024-05-27T08:13:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.202+0000 7f3f11600700 1 -- 172.21.15.63:0/2971931146 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f0800fd20 con 0x7f3f141045e0 2024-05-27T08:13:31.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.202+0000 7f3f1b600700 1 -- 172.21.15.63:0/2971931146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f141a2200 con 0x7f3f141045e0 2024-05-27T08:13:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.202+0000 7f3f11600700 1 -- 172.21.15.63:0/2971931146 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800e3b0 con 0x7f3f141045e0 2024-05-27T08:13:31.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.202+0000 7f3f1b600700 1 -- 172.21.15.63:0/2971931146 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f141a9b20 con 0x7f3f141045e0 2024-05-27T08:13:31.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.207+0000 7f3f11600700 1 -- 172.21.15.63:0/2971931146 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3f08019070 con 0x7f3f141045e0 2024-05-27T08:13:31.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.209+0000 7f3f11600700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3efc0b2a70 0x7f3efc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.209+0000 7f3f11600700 1 -- 172.21.15.63:0/2971931146 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f3f080e2790 con 0x7f3f141045e0 2024-05-27T08:13:31.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.209+0000 7f3f13e00700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3efc0b2a70 0x7f3efc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.210+0000 7f3f13e00700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3efc0b2a70 0x7f3efc0b4f80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009990 tx=0x7f3f00005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:31.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.236+0000 7f3f2a1d4b80 1 Processor -- start 2024-05-27T08:13:31.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.237+0000 7f3f2a1d4b80 1 -- start start 2024-05-27T08:13:31.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.237+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd690e40 0x5614dd8157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.237+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 0x5614dd80f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.237+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd80fdb0 0x5614dd8101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.237+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5614dd73de40 con 0x5614dd80fdb0 2024-05-27T08:13:31.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.237+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5614dd72d9e0 con 0x5614dd690e40 2024-05-27T08:13:31.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.237+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5614dd731a20 con 0x5614dd815d30 2024-05-27T08:13:31.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.238+0000 7f3f19600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd80fdb0 0x5614dd8101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.238+0000 7f3f19600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd80fdb0 0x5614dd8101c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53142/0 (socket says 172.21.15.63:53142) 2024-05-27T08:13:31.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.238+0000 7f3f19600700 1 -- 172.21.15.63:0/512002380 learned_addr learned my addr 172.21.15.63:0/512002380 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:31.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.238+0000 7f3f18c00700 1 --2- 172.21.15.63:0/512002380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd690e40 0x5614dd8157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.238+0000 7f3f13e00700 1 --2- 172.21.15.63:0/512002380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 0x5614dd80f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.238+0000 7f3f19600700 1 -- 172.21.15.63:0/512002380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 msgr2=0x5614dd80f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:31.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.238+0000 7f3f19600700 1 --2- 172.21.15.63:0/512002380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 0x5614dd80f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.238+0000 7f3f19600700 1 -- 172.21.15.63:0/512002380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd690e40 msgr2=0x5614dd8157f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:31.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.238+0000 7f3f19600700 1 --2- 172.21.15.63:0/512002380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd690e40 0x5614dd8157f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.238+0000 7f3f19600700 1 -- 172.21.15.63:0/512002380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614dd6f6420 con 0x5614dd80fdb0 2024-05-27T08:13:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.239+0000 7f3f13e00700 1 --2- 172.21.15.63:0/512002380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 0x5614dd80f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.239+0000 7f3f19600700 1 --2- 172.21.15.63:0/512002380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd80fdb0 0x5614dd8101c0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c003fa0 tx=0x7f3f0c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc6cd38967e4d55 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.239+0000 7f3f06000700 1 -- 172.21.15.63:0/512002380 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c004bb0 con 0x5614dd80fdb0 2024-05-27T08:13:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.239+0000 7f3f06000700 1 -- 172.21.15.63:0/512002380 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f0c0048e0 con 0x5614dd80fdb0 2024-05-27T08:13:31.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.239+0000 7f3f06000700 1 -- 172.21.15.63:0/512002380 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c007c90 con 0x5614dd80fdb0 2024-05-27T08:13:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.241+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/512002380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd80fdb0 msgr2=0x5614dd8101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.241+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/512002380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd80fdb0 0x5614dd8101c0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c003fa0 tx=0x7f3f0c009ba0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.241+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/512002380 shutdown_connections 2024-05-27T08:13:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.241+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/512002380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd80fdb0 0x5614dd8101c0 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.241+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/512002380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd690e40 0x5614dd8157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.241+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/512002380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 0x5614dd80f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.241+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/512002380 >> 172.21.15.63:0/512002380 conn(0x5614dd6f0fc0 msgr2=0x5614dd80a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.241+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/512002380 shutdown_connections 2024-05-27T08:13:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.241+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/512002380 wait complete. 2024-05-27T08:13:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.242+0000 7f3f2a1d4b80 1 -- start start 2024-05-27T08:13:31.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.242+0000 7f3f2a1d4b80 1 Processor -- start 2024-05-27T08:13:31.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd819480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd80fdb0 0x5614dd8199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 0x5614dd819f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5614dd633fa0 con 0x5614dd690e40 2024-05-27T08:13:31.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5614dd6ce760 con 0x5614dd80fdb0 2024-05-27T08:13:31.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5614dd81c800 con 0x5614dd815d30 2024-05-27T08:13:31.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f2a1d4b80 10 client.0 _ll_register_callbacks cb 0x5614dd6d7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:31.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f18c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd819480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f19600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 0x5614dd819f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f18c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd819480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53158/0 (socket says 172.21.15.63:53158) 2024-05-27T08:13:31.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f18c00700 1 -- 172.21.15.63:0/1034345486 learned_addr learned my addr 172.21.15.63:0/1034345486 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:31.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.243+0000 7f3f13e00700 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd80fdb0 0x5614dd8199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.244+0000 7f3f18c00700 1 -- 172.21.15.63:0/1034345486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 msgr2=0x5614dd819f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:31.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.244+0000 7f3f18c00700 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 0x5614dd819f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.244+0000 7f3f18c00700 1 -- 172.21.15.63:0/1034345486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd80fdb0 msgr2=0x5614dd8199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:31.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.244+0000 7f3f18c00700 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd80fdb0 0x5614dd8199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.244+0000 7f3f18c00700 1 -- 172.21.15.63:0/1034345486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614dd67d8d0 con 0x5614dd690e40 2024-05-27T08:13:31.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.244+0000 7f3f18c00700 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd819480 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f3f080a9640 tx=0x7f3f080108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc7d32cc8d476c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:31.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.245+0000 7f3efaa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:31.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.245+0000 7f3efaa00700 1 -- 172.21.15.63:0/1034345486 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08020040 con 0x5614dd690e40 2024-05-27T08:13:31.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.245+0000 7f3efaa00700 1 -- 172.21.15.63:0/1034345486 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f3f080910e0 con 0x5614dd690e40 2024-05-27T08:13:31.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.245+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1034345486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0c0048e0 con 0x5614dd690e40 2024-05-27T08:13:31.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.245+0000 7f3efaa00700 1 -- 172.21.15.63:0/1034345486 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3f080ada60 con 0x5614dd690e40 2024-05-27T08:13:31.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.245+0000 7f3f2a1d4b80 10 client.23031 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:31.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.245+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1034345486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5614dd6fe060 con 0x5614dd690e40 2024-05-27T08:13:31.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.246+0000 7f3efaa00700 1 -- 172.21.15.63:0/1034345486 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f0c0048e0 con 0x5614dd690e40 2024-05-27T08:13:31.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.246+0000 7f3efaa00700 1 -- 172.21.15.63:0/1034345486 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3f080a9910 con 0x5614dd690e40 2024-05-27T08:13:31.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.246+0000 7f3f2a1d4b80 10 client.23031 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:31.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.246+0000 7f3f2a1d4b80 10 client.23031 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:13:31.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.246+0000 7f3f2a1d4b80 20 client.23031 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:31.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.246+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5614dd8295f0 0x5614dd82ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.246+0000 7f3f2a1d4b80 4 client.23031 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:13:31.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.246+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1034345486 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614dd73fd90 con 0x5614dd8295f0 2024-05-27T08:13:31.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.247+0000 7f3f13e00700 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5614dd8295f0 0x5614dd82ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.248+0000 7f3f13e00700 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5614dd8295f0 0x5614dd82ba10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad483e358eba1ae6 server_cookie=5ca2ea4ba4fe0397 in_seq=0 out_seq=0 2024-05-27T08:13:31.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.248+0000 7f3efaa00700 10 client.23031 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:13:31.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3efaa00700 1 -- 172.21.15.63:0/1034345486 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5614dd73fd90 con 0x5614dd8295f0 2024-05-27T08:13:31.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3efaa00700 10 client.23031 handle_command_reply: tid=0 2024-05-27T08:13:31.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3f2a1d4b80 1 client.23031 shutdown 2024-05-27T08:13:31.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1034345486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 msgr2=0x5614dd819480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:31.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd819480 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f3f080a9640 tx=0x7f3f080108d0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1034345486 shutdown_connections 2024-05-27T08:13:31.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd815d30 0x5614dd819f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5614dd8295f0 0x5614dd82ba10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd819480 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1034345486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd80fdb0 0x5614dd8199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3efaa00700 0 client.23031 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:13:31.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.250+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1034345486 >> 172.21.15.63:0/1034345486 conn(0x5614dd6f0fc0 msgr2=0x5614dd80a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:31.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.251+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1034345486 shutdown_connections 2024-05-27T08:13:31.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.251+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1034345486 wait complete. 2024-05-27T08:13:31.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.251+0000 7f3f2a1d4b80 20 client.23031 trim_cache size 0 max 16384 2024-05-27T08:13:31.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.272+0000 7f3f2a1d4b80 1 Processor -- start 2024-05-27T08:13:31.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.272+0000 7f3f2a1d4b80 1 -- start start 2024-05-27T08:13:31.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.272+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd80f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.272+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd80fdb0 0x5614dd8183e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.272+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd815d30 0x5614dd813b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.272+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f141b5070 con 0x5614dd690e40 2024-05-27T08:13:31.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.272+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f141b2900 con 0x5614dd815d30 2024-05-27T08:13:31.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.272+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3f141b0770 con 0x5614dd80fdb0 2024-05-27T08:13:31.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.273+0000 7f3f18c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd80f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.273+0000 7f3f18c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd80f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53160/0 (socket says 172.21.15.63:53160) 2024-05-27T08:13:31.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.273+0000 7f3f18c00700 1 -- 172.21.15.63:0/1452507048 learned_addr learned my addr 172.21.15.63:0/1452507048 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:31.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.273+0000 7f3f13e00700 1 --2- 172.21.15.63:0/1452507048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd80fdb0 0x5614dd8183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.273+0000 7f3f19600700 1 --2- 172.21.15.63:0/1452507048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd815d30 0x5614dd813b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.273+0000 7f3f18c00700 1 -- 172.21.15.63:0/1452507048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd80fdb0 msgr2=0x5614dd8183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:31.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.273+0000 7f3f18c00700 1 --2- 172.21.15.63:0/1452507048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd80fdb0 0x5614dd8183e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.273+0000 7f3f18c00700 1 -- 172.21.15.63:0/1452507048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd815d30 msgr2=0x5614dd813b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:31.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.273+0000 7f3f18c00700 1 --2- 172.21.15.63:0/1452507048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd815d30 0x5614dd813b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.273+0000 7f3f18c00700 1 -- 172.21.15.63:0/1452507048 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614dd817440 con 0x5614dd690e40 2024-05-27T08:13:31.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.274+0000 7f3f18c00700 1 --2- 172.21.15.63:0/1452507048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd80f870 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f3f080e2a80 tx=0x7f3f08005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925a327675eaaf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:31.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.274+0000 7f3f05600700 1 -- 172.21.15.63:0/1452507048 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08020070 con 0x5614dd690e40 2024-05-27T08:13:31.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.274+0000 7f3f05600700 1 -- 172.21.15.63:0/1452507048 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f080e2380 con 0x5614dd690e40 2024-05-27T08:13:31.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.274+0000 7f3f05600700 1 -- 172.21.15.63:0/1452507048 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f080910b0 con 0x5614dd690e40 2024-05-27T08:13:31.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.275+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1452507048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 msgr2=0x5614dd80f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:31.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.275+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1452507048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd80f870 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f3f080e2a80 tx=0x7f3f08005910 comp rx=0 tx=0).stop 2024-05-27T08:13:31.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.275+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1452507048 shutdown_connections 2024-05-27T08:13:31.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.275+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1452507048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd815d30 0x5614dd813b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.275+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1452507048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd690e40 0x5614dd80f870 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.275+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1452507048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd80fdb0 0x5614dd8183e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.275+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1452507048 >> 172.21.15.63:0/1452507048 conn(0x5614dd6f0fc0 msgr2=0x5614dd819260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:31.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.275+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1452507048 shutdown_connections 2024-05-27T08:13:31.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.275+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1452507048 wait complete. 2024-05-27T08:13:31.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.276+0000 7f3f2a1d4b80 1 -- start start 2024-05-27T08:13:31.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.276+0000 7f3f2a1d4b80 1 Processor -- start 2024-05-27T08:13:31.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd690e40 0x5614dd821ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd80fdb0 0x5614dd830860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f2a1d4b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd815d30 0x5614dd830da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f141b0dc0 con 0x5614dd815d30 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f141b2230 con 0x5614dd690e40 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f2a1d4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3f141b1490 con 0x5614dd80fdb0 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f2a1d4b80 10 client.0 _ll_register_callbacks cb 0x5614dd6d7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f19600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd815d30 0x5614dd830da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f19600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd815d30 0x5614dd830da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53172/0 (socket says 172.21.15.63:53172) 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f19600700 1 -- 172.21.15.63:0/1466598298 learned_addr learned my addr 172.21.15.63:0/1466598298 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f18c00700 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd690e40 0x5614dd821ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.277+0000 7f3f13e00700 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd80fdb0 0x5614dd830860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.278+0000 7f3f19600700 1 -- 172.21.15.63:0/1466598298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd80fdb0 msgr2=0x5614dd830860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.278+0000 7f3f19600700 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd80fdb0 0x5614dd830860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.278+0000 7f3f19600700 1 -- 172.21.15.63:0/1466598298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd690e40 msgr2=0x5614dd821ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:31.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.278+0000 7f3f19600700 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd690e40 0x5614dd821ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.278+0000 7f3f19600700 1 -- 172.21.15.63:0/1466598298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614dd831310 con 0x5614dd815d30 2024-05-27T08:13:31.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.278+0000 7f3f19600700 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd815d30 0x5614dd830da0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0078b0 tx=0x7f3f0c008cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba87095384d13928 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:31.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.278+0000 7f3efaa00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:31.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.279+0000 7f3efaa00700 1 -- 172.21.15.63:0/1466598298 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c008e60 con 0x5614dd815d30 2024-05-27T08:13:31.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.279+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1466598298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f080e2380 con 0x5614dd815d30 2024-05-27T08:13:31.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.279+0000 7f3efaa00700 1 -- 172.21.15.63:0/1466598298 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f3f0c01a040 con 0x5614dd815d30 2024-05-27T08:13:31.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.279+0000 7f3efaa00700 1 -- 172.21.15.63:0/1466598298 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3f0c031050 con 0x5614dd815d30 2024-05-27T08:13:31.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.279+0000 7f3f2a1d4b80 10 client.23037 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:31.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.279+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1466598298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5614dd893de0 con 0x5614dd815d30 2024-05-27T08:13:31.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.279+0000 7f3efaa00700 1 -- 172.21.15.63:0/1466598298 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f080e2380 con 0x5614dd815d30 2024-05-27T08:13:31.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.280+0000 7f3efaa00700 1 -- 172.21.15.63:0/1466598298 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3f0c015710 con 0x5614dd815d30 2024-05-27T08:13:31.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.280+0000 7f3f2a1d4b80 10 client.23037 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:31.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.280+0000 7f3f2a1d4b80 10 client.23037 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:13:31.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.280+0000 7f3f2a1d4b80 20 client.23037 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:31.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.280+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5614dd894160 0x5614dd896580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:31.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.280+0000 7f3f2a1d4b80 4 client.23037 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:13:31.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.280+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1466598298 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614dd739570 con 0x5614dd894160 2024-05-27T08:13:31.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.281+0000 7f3f18c00700 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5614dd894160 0x5614dd896580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:31.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.281+0000 7f3f18c00700 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5614dd894160 0x5614dd896580 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e748404fd3cf04da server_cookie=ff4621d42d995ce9 in_seq=0 out_seq=0 2024-05-27T08:13:31.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.281+0000 7f3efaa00700 10 client.23037 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3efaa00700 1 -- 172.21.15.63:0/1466598298 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614dd739570 con 0x5614dd894160 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3efaa00700 10 client.23037 handle_command_reply: tid=0 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3f2a1d4b80 1 client.23037 shutdown 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1466598298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd815d30 msgr2=0x5614dd830da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd815d30 0x5614dd830da0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0078b0 tx=0x7f3f0c008cc0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1466598298 shutdown_connections 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5614dd894160 0x5614dd896580 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5614dd815d30 0x5614dd830da0 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614dd690e40 0x5614dd821ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3f2a1d4b80 1 --2- 172.21.15.63:0/1466598298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5614dd80fdb0 0x5614dd830860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1466598298 >> 172.21.15.63:0/1466598298 conn(0x5614dd6f0fc0 msgr2=0x5614dd8138d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:31.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.282+0000 7f3efaa00700 0 client.23037 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:13:31.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.283+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1466598298 shutdown_connections 2024-05-27T08:13:31.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.283+0000 7f3f2a1d4b80 1 -- 172.21.15.63:0/1466598298 wait complete. 2024-05-27T08:13:31.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.283+0000 7f3f2a1d4b80 20 client.23037 trim_cache size 0 max 16384 2024-05-27T08:13:31.285 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:13:31.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 -- 172.21.15.63:0/2971931146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3efc0b2a70 msgr2=0x7f3efc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:31.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3efc0b2a70 0x7f3efc0b4f80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009990 tx=0x7f3f00005fb0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 -- 172.21.15.63:0/2971931146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f141045e0 msgr2=0x7f3f1419d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:31.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f141045e0 0x7f3f1419d040 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800d950 tx=0x7f3f0800dc60 comp rx=0 tx=0).stop 2024-05-27T08:13:31.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 -- 172.21.15.63:0/2971931146 shutdown_connections 2024-05-27T08:13:31.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3efc0b2a70 0x7f3efc0b4f80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14109860 0x7f3f141a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f141045e0 0x7f3f1419d040 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 --2- 172.21.15.63:0/2971931146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14108f10 0x7f3f1419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:31.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 -- 172.21.15.63:0/2971931146 >> 172.21.15.63:0/2971931146 conn(0x7f3f14100030 msgr2=0x7f3f1410a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:31.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 -- 172.21.15.63:0/2971931146 shutdown_connections 2024-05-27T08:13:31.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:31.284+0000 7f3f1b600700 1 -- 172.21.15.63:0/2971931146 wait complete. 2024-05-27T08:13:32.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:31 smithi086 ceph-mon[34687]: pgmap v1268: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:13:32.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:31 smithi149 ceph-mon[35408]: pgmap v1268: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:13:32.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:31 smithi063 ceph-mon[31613]: pgmap v1268: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.7 MiB/s wr, 10 op/s 2024-05-27T08:13:32.479 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:13:32.979 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:13:33.160 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:32 smithi063 ceph-mon[31613]: pgmap v1269: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T08:13:33.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:32 smithi086 ceph-mon[34687]: pgmap v1269: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T08:13:33.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:32 smithi149 ceph-mon[35408]: pgmap v1269: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 14 op/s 2024-05-27T08:13:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.876+0000 7fb823e00640 1 -- 172.21.15.63:0/1817171558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8241027c0 msgr2=0x7fb824102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.876+0000 7fb823e00640 1 --2- 172.21.15.63:0/1817171558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8241027c0 0x7fb824102bd0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7fb814009900 tx=0x7fb81402a410 comp rx=0 tx=0).stop 2024-05-27T08:13:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.877+0000 7fb823e00640 1 -- 172.21.15.63:0/1817171558 shutdown_connections 2024-05-27T08:13:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.877+0000 7fb823e00640 1 --2- 172.21.15.63:0/1817171558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb824110360 0x7fb824112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.877+0000 7fb823e00640 1 --2- 172.21.15.63:0/1817171558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb82410da00 0x7fb82410fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.877+0000 7fb823e00640 1 --2- 172.21.15.63:0/1817171558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8241027c0 0x7fb824102bd0 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.877+0000 7fb823e00640 1 -- 172.21.15.63:0/1817171558 >> 172.21.15.63:0/1817171558 conn(0x7fb8240fe210 msgr2=0x7fb824100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.877+0000 7fb823e00640 1 -- 172.21.15.63:0/1817171558 shutdown_connections 2024-05-27T08:13:35.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.877+0000 7fb823e00640 1 -- 172.21.15.63:0/1817171558 wait complete. 2024-05-27T08:13:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.878+0000 7fb823e00640 1 Processor -- start 2024-05-27T08:13:35.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.878+0000 7fb823e00640 1 -- start start 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.878+0000 7fb823e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8241027c0 0x7fb8241a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.878+0000 7fb823e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb82410da00 0x7fb8241a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb823e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb824110360 0x7fb82419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb823e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb824115d70 con 0x7fb82410da00 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb823e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb824115bf0 con 0x7fb8241027c0 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb823e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb824115ef0 con 0x7fb824110360 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb822000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb82410da00 0x7fb8241a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb822000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb82410da00 0x7fb8241a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53196/0 (socket says 172.21.15.63:53196) 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb822000640 1 -- 172.21.15.63:0/557749004 learned_addr learned my addr 172.21.15.63:0/557749004 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb822a00640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8241027c0 0x7fb8241a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb823400640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb824110360 0x7fb82419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb822000640 1 -- 172.21.15.63:0/557749004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb824110360 msgr2=0x7fb82419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb822000640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb824110360 0x7fb82419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb822000640 1 -- 172.21.15.63:0/557749004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8241027c0 msgr2=0x7fb8241a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb822000640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8241027c0 0x7fb8241a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.879+0000 7fb822000640 1 -- 172.21.15.63:0/557749004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82419cdc0 con 0x7fb82410da00 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.880+0000 7fb822a00640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8241027c0 0x7fb8241a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.880+0000 7fb822000640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb82410da00 0x7fb8241a2a50 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7fb81000d770 tx=0x7fb81000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.880+0000 7fb81b400640 1 -- 172.21.15.63:0/557749004 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810017070 con 0x7fb82410da00 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.880+0000 7fb81b400640 1 -- 172.21.15.63:0/557749004 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8100054b0 con 0x7fb82410da00 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.880+0000 7fb823e00640 1 -- 172.21.15.63:0/557749004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82419d050 con 0x7fb82410da00 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.880+0000 7fb81b400640 1 -- 172.21.15.63:0/557749004 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810004c60 con 0x7fb82410da00 2024-05-27T08:13:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.880+0000 7fb823e00640 1 -- 172.21.15.63:0/557749004 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8241a92b0 con 0x7fb82410da00 2024-05-27T08:13:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.884+0000 7fb830637740 1 Processor -- start 2024-05-27T08:13:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.884+0000 7fb830637740 1 -- start start 2024-05-27T08:13:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.885+0000 7fb830637740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240ea1b0 0x55d12448ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.885+0000 7fb830637740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12448cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.885+0000 7fb830637740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d12448d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.885+0000 7fb830637740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d1243fdb90 con 0x55d1240c12f0 2024-05-27T08:13:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.885+0000 7fb830637740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1243ed730 con 0x55d1240c94d0 2024-05-27T08:13:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.885+0000 7fb830637740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d1243f1770 con 0x55d1240ea1b0 2024-05-27T08:13:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.885+0000 7fb822a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12448cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.885+0000 7fb822a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12448cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53204/0 (socket says 172.21.15.63:53204) 2024-05-27T08:13:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.885+0000 7fb822a00640 1 -- 172.21.15.63:0/1137733881 learned_addr learned my addr 172.21.15.63:0/1137733881 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.886+0000 7fb81b400640 1 -- 172.21.15.63:0/557749004 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb810004e00 con 0x7fb82410da00 2024-05-27T08:13:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.886+0000 7fb823400640 1 --2- 172.21.15.63:0/1137733881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240ea1b0 0x55d12448ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.886+0000 7fb822a00640 1 -- 172.21.15.63:0/1137733881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240ea1b0 msgr2=0x55d12448ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.886+0000 7fb822a00640 1 --2- 172.21.15.63:0/1137733881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240ea1b0 0x55d12448ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.886+0000 7fb822a00640 1 -- 172.21.15.63:0/1137733881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 msgr2=0x55d12448d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.886+0000 7fb822a00640 1 --2- 172.21.15.63:0/1137733881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d12448d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.886+0000 7fb822a00640 1 -- 172.21.15.63:0/1137733881 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1243868b0 con 0x55d1240c12f0 2024-05-27T08:13:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.887+0000 7fb822a00640 1 --2- 172.21.15.63:0/1137733881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12448cfa0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fb81402aa00 tx=0x7fb814004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6f22d04af174b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.887+0000 7fb80f400640 1 -- 172.21.15.63:0/1137733881 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814035030 con 0x55d1240c12f0 2024-05-27T08:13:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.887+0000 7fb80f400640 1 -- 172.21.15.63:0/1137733881 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb814005de0 con 0x55d1240c12f0 2024-05-27T08:13:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.887+0000 7fb80f400640 1 -- 172.21.15.63:0/1137733881 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8140035f0 con 0x55d1240c12f0 2024-05-27T08:13:35.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.888+0000 7fb81b400640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8080b2950 0x7fb8080b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.888+0000 7fb81b400640 1 -- 172.21.15.63:0/557749004 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7fb8100e24f0 con 0x7fb82410da00 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.888+0000 7fb823400640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8080b2950 0x7fb8080b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.888+0000 7fb830637740 1 -- 172.21.15.63:0/1137733881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 msgr2=0x55d12448cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.888+0000 7fb830637740 1 --2- 172.21.15.63:0/1137733881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12448cfa0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fb81402aa00 tx=0x7fb814004390 comp rx=0 tx=0).stop 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.889+0000 7fb830637740 1 -- 172.21.15.63:0/1137733881 shutdown_connections 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.889+0000 7fb830637740 1 --2- 172.21.15.63:0/1137733881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d12448d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.889+0000 7fb830637740 1 --2- 172.21.15.63:0/1137733881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12448cfa0 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.889+0000 7fb830637740 1 --2- 172.21.15.63:0/1137733881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240ea1b0 0x55d12448ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.889+0000 7fb830637740 1 -- 172.21.15.63:0/1137733881 >> 172.21.15.63:0/1137733881 conn(0x55d1240de300 msgr2=0x55d124489210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.889+0000 7fb830637740 1 -- 172.21.15.63:0/1137733881 shutdown_connections 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.889+0000 7fb830637740 1 -- 172.21.15.63:0/1137733881 wait complete. 2024-05-27T08:13:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.889+0000 7fb823400640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8080b2950 0x7fb8080b4e60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0092f0 tx=0x7fb81c009280 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.890+0000 7fb830637740 1 -- start start 2024-05-27T08:13:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.890+0000 7fb830637740 1 Processor -- start 2024-05-27T08:13:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb830637740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 0x55d124496f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb830637740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d124497460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb830637740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240ea1b0 0x55d1244a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb830637740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d1240d3790 con 0x55d1240ea1b0 2024-05-27T08:13:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb830637740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d124395bc0 con 0x55d1240c94d0 2024-05-27T08:13:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb830637740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d124499b30 con 0x55d1240c12f0 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb830637740 10 client.0 _ll_register_callbacks cb 0x55d124373400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb823400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240ea1b0 0x55d1244a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb823400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240ea1b0 0x55d1244a2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53216/0 (socket says 172.21.15.63:53216) 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb823400640 1 -- 172.21.15.63:0/3131878622 learned_addr learned my addr 172.21.15.63:0/3131878622 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb822000640 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d124497460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.891+0000 7fb822a00640 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 0x55d124496f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb823400640 1 -- 172.21.15.63:0/3131878622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 msgr2=0x55d124496f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb823400640 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 0x55d124496f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb823400640 1 -- 172.21.15.63:0/3131878622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 msgr2=0x55d124497460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb823400640 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d124497460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb823400640 1 -- 172.21.15.63:0/3131878622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d124102210 con 0x55d1240ea1b0 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb822a00640 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 0x55d124496f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb822000640 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d124497460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb823400640 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240ea1b0 0x55d1244a2e90 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0127e0 tx=0x7fb81c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500ee09b413dd82b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb803e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb803e00640 1 -- 172.21.15.63:0/3131878622 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c00ece0 con 0x55d1240ea1b0 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.892+0000 7fb803e00640 1 -- 172.21.15.63:0/3131878622 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7fb81c01f420 con 0x55d1240ea1b0 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.893+0000 7fb830637740 1 -- 172.21.15.63:0/3131878622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1243ffae0 con 0x55d1240ea1b0 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.893+0000 7fb803e00640 1 -- 172.21.15.63:0/3131878622 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb81c012e20 con 0x55d1240ea1b0 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.893+0000 7fb830637740 10 client.23049 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.893+0000 7fb830637740 1 -- 172.21.15.63:0/3131878622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d1241ce620 con 0x55d1240ea1b0 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.893+0000 7fb803e00640 1 -- 172.21.15.63:0/3131878622 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d1243ffae0 con 0x55d1240ea1b0 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.893+0000 7fb803e00640 1 -- 172.21.15.63:0/3131878622 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb81c00f040 con 0x55d1240ea1b0 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.894+0000 7fb830637740 10 client.23049 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.894+0000 7fb830637740 10 client.23049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.894+0000 7fb830637740 20 client.23049 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.894+0000 7fb830637740 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d1240a8d80 0x55d1244a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.894+0000 7fb830637740 4 client.23049 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:13:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.894+0000 7fb830637740 1 -- 172.21.15.63:0/3131878622 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1243f92c0 con 0x55d1240a8d80 2024-05-27T08:13:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.894+0000 7fb822a00640 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d1240a8d80 0x55d1244a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.896+0000 7fb822a00640 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d1240a8d80 0x55d1244a8fa0 crc :-1 s=READY pgs=708 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e71c14602cb1e4d server_cookie=528382e325ff59da in_seq=0 out_seq=0 2024-05-27T08:13:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.896+0000 7fb803e00640 10 client.23049 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.897+0000 7fb803e00640 1 -- 172.21.15.63:0/3131878622 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d1243f92c0 con 0x55d1240a8d80 2024-05-27T08:13:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.897+0000 7fb803e00640 10 client.23049 handle_command_reply: tid=0 2024-05-27T08:13:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 client.23049 shutdown 2024-05-27T08:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 -- 172.21.15.63:0/3131878622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240ea1b0 msgr2=0x55d1244a2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240ea1b0 0x55d1244a2e90 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0127e0 tx=0x7fb81c004a40 comp rx=0 tx=0).stop 2024-05-27T08:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 -- 172.21.15.63:0/3131878622 shutdown_connections 2024-05-27T08:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d1240a8d80 0x55d1244a8fa0 crc :-1 s=READY pgs=708 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240ea1b0 0x55d1244a2e90 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d124497460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 --2- 172.21.15.63:0/3131878622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 0x55d124496f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 -- 172.21.15.63:0/3131878622 >> 172.21.15.63:0/3131878622 conn(0x55d1240de300 msgr2=0x55d12448e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb803e00640 0 client.23049 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 -- 172.21.15.63:0/3131878622 shutdown_connections 2024-05-27T08:13:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 1 -- 172.21.15.63:0/3131878622 wait complete. 2024-05-27T08:13:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.898+0000 7fb830637740 20 client.23049 trim_cache size 0 max 16384 2024-05-27T08:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.916+0000 7fb830637740 1 Processor -- start 2024-05-27T08:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.916+0000 7fb830637740 1 -- start start 2024-05-27T08:13:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.916+0000 7fb830637740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 0x55d1244a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.916+0000 7fb830637740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c94d0 0x55d1244a3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.916+0000 7fb830637740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240ea1b0 0x55d1244ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.916+0000 7fb830637740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241b5a70 con 0x55d1240c94d0 2024-05-27T08:13:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.916+0000 7fb830637740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241b3340 con 0x55d1240ea1b0 2024-05-27T08:13:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.916+0000 7fb830637740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241b11b0 con 0x55d1240c12f0 2024-05-27T08:13:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.917+0000 7fb822000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c94d0 0x55d1244a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.917+0000 7fb822000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c94d0 0x55d1244a3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53222/0 (socket says 172.21.15.63:53222) 2024-05-27T08:13:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.917+0000 7fb822000640 1 -- 172.21.15.63:0/2544301725 learned_addr learned my addr 172.21.15.63:0/2544301725 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.917+0000 7fb822a00640 1 --2- 172.21.15.63:0/2544301725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 0x55d1244a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.917+0000 7fb823400640 1 --2- 172.21.15.63:0/2544301725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240ea1b0 0x55d1244ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.917+0000 7fb822000640 1 -- 172.21.15.63:0/2544301725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 msgr2=0x55d1244a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.917+0000 7fb822000640 1 --2- 172.21.15.63:0/2544301725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 0x55d1244a2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.917+0000 7fb822000640 1 -- 172.21.15.63:0/2544301725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240ea1b0 msgr2=0x55d1244ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.917+0000 7fb822000640 1 --2- 172.21.15.63:0/2544301725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240ea1b0 0x55d1244ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.917+0000 7fb822000640 1 -- 172.21.15.63:0/2544301725 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1244a9500 con 0x55d1240c94d0 2024-05-27T08:13:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.918+0000 7fb822000640 1 --2- 172.21.15.63:0/2544301725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c94d0 0x55d1244a3140 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7fb8100a6680 tx=0x7fb8100a8e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b701e0f9dcadbf2d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.918+0000 7fb80ea00640 1 -- 172.21.15.63:0/2544301725 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810020040 con 0x55d1240c94d0 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.918+0000 7fb80ea00640 1 -- 172.21.15.63:0/2544301725 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8100a9410 con 0x55d1240c94d0 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.918+0000 7fb80ea00640 1 -- 172.21.15.63:0/2544301725 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8100050f0 con 0x55d1240c94d0 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.919+0000 7fb830637740 1 -- 172.21.15.63:0/2544301725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c94d0 msgr2=0x55d1244a3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.919+0000 7fb830637740 1 --2- 172.21.15.63:0/2544301725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c94d0 0x55d1244a3140 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7fb8100a6680 tx=0x7fb8100a8e20 comp rx=0 tx=0).stop 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.919+0000 7fb830637740 1 -- 172.21.15.63:0/2544301725 shutdown_connections 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.919+0000 7fb830637740 1 --2- 172.21.15.63:0/2544301725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240ea1b0 0x55d1244ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.919+0000 7fb830637740 1 --2- 172.21.15.63:0/2544301725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c94d0 0x55d1244a3140 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.919+0000 7fb830637740 1 --2- 172.21.15.63:0/2544301725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240c12f0 0x55d1244a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.919+0000 7fb830637740 1 -- 172.21.15.63:0/2544301725 >> 172.21.15.63:0/2544301725 conn(0x55d1240de300 msgr2=0x55d12448d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.919+0000 7fb830637740 1 -- 172.21.15.63:0/2544301725 shutdown_connections 2024-05-27T08:13:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.919+0000 7fb830637740 1 -- 172.21.15.63:0/2544301725 wait complete. 2024-05-27T08:13:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.920+0000 7fb830637740 1 -- start start 2024-05-27T08:13:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.920+0000 7fb830637740 1 Processor -- start 2024-05-27T08:13:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb830637740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12449a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb830637740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d12449a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb830637740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240ea1b0 0x55d12450f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb830637740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241b1800 con 0x55d1240c12f0 2024-05-27T08:13:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb830637740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241b2c70 con 0x55d1240c94d0 2024-05-27T08:13:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb830637740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241b1ed0 con 0x55d1240ea1b0 2024-05-27T08:13:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb830637740 10 client.0 _ll_register_callbacks cb 0x7fb81c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb822a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12449a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb822a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12449a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53232/0 (socket says 172.21.15.63:53232) 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb822a00640 1 -- 172.21.15.63:0/3959088648 learned_addr learned my addr 172.21.15.63:0/3959088648 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb822000640 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d12449a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb823400640 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240ea1b0 0x55d12450f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb822a00640 1 -- 172.21.15.63:0/3959088648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240ea1b0 msgr2=0x55d12450f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb822a00640 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240ea1b0 0x55d12450f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb822a00640 1 -- 172.21.15.63:0/3959088648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 msgr2=0x55d12449a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb822a00640 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d12449a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb822a00640 1 -- 172.21.15.63:0/3959088648 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d12450fb30 con 0x55d1240c12f0 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.921+0000 7fb822000640 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d12449a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.922+0000 7fb822a00640 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12449a3b0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fb8140057c0 tx=0x7fb8140095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be1f6d5b0a3474d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.922+0000 7fb803e00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.922+0000 7fb803e00640 1 -- 172.21.15.63:0/3959088648 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814035040 con 0x55d1240c12f0 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.922+0000 7fb803e00640 1 -- 172.21.15.63:0/3959088648 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7fb814005000 con 0x55d1240c12f0 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.922+0000 7fb830637740 1 -- 172.21.15.63:0/3959088648 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8100a9410 con 0x55d1240c12f0 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.922+0000 7fb803e00640 1 -- 172.21.15.63:0/3959088648 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb81405c050 con 0x55d1240c12f0 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.922+0000 7fb830637740 10 client.23055 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.922+0000 7fb830637740 1 -- 172.21.15.63:0/3959088648 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d124510db0 con 0x55d1240c12f0 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.923+0000 7fb803e00640 1 -- 172.21.15.63:0/3959088648 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8100a9410 con 0x55d1240c12f0 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.923+0000 7fb803e00640 1 -- 172.21.15.63:0/3959088648 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb81404ea70 con 0x55d1240c12f0 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.923+0000 7fb830637740 10 client.23055 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.923+0000 7fb830637740 10 client.23055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.923+0000 7fb830637740 20 client.23055 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.923+0000 7fb830637740 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d12407e1c0 0x55d124513100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.923+0000 7fb830637740 4 client.23055 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.923+0000 7fb830637740 1 -- 172.21.15.63:0/3959088648 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1243734b0 con 0x55d12407e1c0 2024-05-27T08:13:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.924+0000 7fb822000640 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d12407e1c0 0x55d124513100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.925+0000 7fb822000640 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d12407e1c0 0x55d124513100 crc :-1 s=READY pgs=709 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39c63dd924f7be2 server_cookie=372515766ab4939e in_seq=0 out_seq=0 2024-05-27T08:13:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.925+0000 7fb803e00640 10 client.23055 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb803e00640 1 -- 172.21.15.63:0/3959088648 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d1243734b0 con 0x55d12407e1c0 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb803e00640 10 client.23055 handle_command_reply: tid=0 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 client.23055 shutdown 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 -- 172.21.15.63:0/3959088648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 msgr2=0x55d12449a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12449a3b0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fb8140057c0 tx=0x7fb8140095d0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 -- 172.21.15.63:0/3959088648 shutdown_connections 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d12407e1c0 0x55d124513100 crc :-1 s=READY pgs=709 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d1240ea1b0 0x55d12450f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1240c94d0 0x55d12449a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 --2- 172.21.15.63:0/3959088648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d1240c12f0 0x55d12449a3b0 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb803e00640 0 client.23055 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 -- 172.21.15.63:0/3959088648 >> 172.21.15.63:0/3959088648 conn(0x55d1240de300 msgr2=0x55d1244983c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 -- 172.21.15.63:0/3959088648 shutdown_connections 2024-05-27T08:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.961+0000 7fb830637740 1 -- 172.21.15.63:0/3959088648 wait complete. 2024-05-27T08:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.962+0000 7fb830637740 20 client.23055 trim_cache size 0 max 16384 2024-05-27T08:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.962+0000 7fb823e00640 1 -- 172.21.15.63:0/557749004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8080b2950 msgr2=0x7fb8080b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.962+0000 7fb823e00640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8080b2950 0x7fb8080b4e60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0092f0 tx=0x7fb81c009280 comp rx=0 tx=0).stop 2024-05-27T08:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.962+0000 7fb823e00640 1 -- 172.21.15.63:0/557749004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb82410da00 msgr2=0x7fb8241a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.962+0000 7fb823e00640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb82410da00 0x7fb8241a2a50 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7fb81000d770 tx=0x7fb81000dc30 comp rx=0 tx=0).stop 2024-05-27T08:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.963+0000 7fb823e00640 1 -- 172.21.15.63:0/557749004 shutdown_connections 2024-05-27T08:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.963+0000 7fb823e00640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb8080b2950 0x7fb8080b4e60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.963+0000 7fb823e00640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb824110360 0x7fb82419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.963+0000 7fb823e00640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb82410da00 0x7fb8241a2a50 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.963+0000 7fb823e00640 1 --2- 172.21.15.63:0/557749004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8241027c0 0x7fb8241a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.963+0000 7fb823e00640 1 -- 172.21.15.63:0/557749004 >> 172.21.15.63:0/557749004 conn(0x7fb8240fe210 msgr2=0x7fb824100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.963+0000 7fb823e00640 1 -- 172.21.15.63:0/557749004 shutdown_connections 2024-05-27T08:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:35.963+0000 7fb823e00640 1 -- 172.21.15.63:0/557749004 wait complete. 2024-05-27T08:13:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.122+0000 7f88a8a00700 1 -- 172.21.15.63:0/479809312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a4075af0 msgr2=0x7f88a4075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.122+0000 7f88a8a00700 1 --2- 172.21.15.63:0/479809312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a4075af0 0x7f88a4075f00 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f889c009990 tx=0x7f889c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.122+0000 7f88a8a00700 1 -- 172.21.15.63:0/479809312 shutdown_connections 2024-05-27T08:13:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.122+0000 7f88a8a00700 1 --2- 172.21.15.63:0/479809312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88a4110bf0 0x7f88a4113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.122+0000 7f88a8a00700 1 --2- 172.21.15.63:0/479809312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a4075af0 0x7f88a4075f00 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.122+0000 7f88a8a00700 1 --2- 172.21.15.63:0/479809312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4076440 0x7f88a41106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.122+0000 7f88a8a00700 1 -- 172.21.15.63:0/479809312 >> 172.21.15.63:0/479809312 conn(0x7f88a4100030 msgr2=0x7f88a4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:36.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.123+0000 7f88a8a00700 1 -- 172.21.15.63:0/479809312 shutdown_connections 2024-05-27T08:13:36.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.123+0000 7f88a8a00700 1 -- 172.21.15.63:0/479809312 wait complete. 2024-05-27T08:13:36.126 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:35 smithi063 ceph-mon[31613]: pgmap v1270: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T08:13:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.124+0000 7f88a8a00700 1 Processor -- start 2024-05-27T08:13:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.124+0000 7f88a8a00700 1 -- start start 2024-05-27T08:13:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.125+0000 7f88a8a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4075af0 0x7f88a419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.125+0000 7f88a8a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88a4076440 0x7f88a419d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.125+0000 7f88a8a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a4110bf0 0x7f88a41a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.125+0000 7f88a8a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88a4116580 con 0x7f88a4110bf0 2024-05-27T08:13:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.125+0000 7f88a8a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88a4116400 con 0x7f88a4075af0 2024-05-27T08:13:36.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.125+0000 7f88a8a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88a4116700 con 0x7f88a4076440 2024-05-27T08:13:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a4110bf0 0x7f88a41a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a4110bf0 0x7f88a41a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53242/0 (socket says 172.21.15.63:53242) 2024-05-27T08:13:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a2800700 1 -- 172.21.15.63:0/493672141 learned_addr learned my addr 172.21.15.63:0/493672141 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:36.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a1e00700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4075af0 0x7f88a419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a1400700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88a4076440 0x7f88a419d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a2800700 1 -- 172.21.15.63:0/493672141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88a4076440 msgr2=0x7f88a419d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:36.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a2800700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88a4076440 0x7f88a419d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a2800700 1 -- 172.21.15.63:0/493672141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4075af0 msgr2=0x7f88a419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:36.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a2800700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4075af0 0x7f88a419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a2800700 1 -- 172.21.15.63:0/493672141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a41a1e60 con 0x7f88a4110bf0 2024-05-27T08:13:36.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.126+0000 7f88a1400700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88a4076440 0x7f88a419d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:36.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.127+0000 7f88a2800700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a4110bf0 0x7f88a41a17c0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f888c00d740 tx=0x7f888c00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:36.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.127+0000 7f889aa00700 1 -- 172.21.15.63:0/493672141 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c016070 con 0x7f88a4110bf0 2024-05-27T08:13:36.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.127+0000 7f889aa00700 1 -- 172.21.15.63:0/493672141 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f888c00f690 con 0x7f88a4110bf0 2024-05-27T08:13:36.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.127+0000 7f88a8a00700 1 -- 172.21.15.63:0/493672141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a41a2150 con 0x7f88a4110bf0 2024-05-27T08:13:36.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.128+0000 7f889aa00700 1 -- 172.21.15.63:0/493672141 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c003970 con 0x7f88a4110bf0 2024-05-27T08:13:36.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.128+0000 7f88a8a00700 1 -- 172.21.15.63:0/493672141 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a41a9a70 con 0x7f88a4110bf0 2024-05-27T08:13:36.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.134+0000 7f889aa00700 1 -- 172.21.15.63:0/493672141 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f888c004220 con 0x7f88a4110bf0 2024-05-27T08:13:36.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.136+0000 7f889aa00700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88880b29f0 0x7f88880b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.136+0000 7f889aa00700 1 -- 172.21.15.63:0/493672141 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f888c0e2470 con 0x7f88a4110bf0 2024-05-27T08:13:36.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.136+0000 7f88a1e00700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88880b29f0 0x7f88880b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.137+0000 7f88a1e00700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88880b29f0 0x7f88880b4f00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f889c00b2e0 tx=0x7f889c005fb0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:36.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.161+0000 7f88b7493b80 1 Processor -- start 2024-05-27T08:13:36.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.161+0000 7f88b7493b80 1 -- start start 2024-05-27T08:13:36.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.162+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32be39e40 0x55f32bfbe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.162+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfbed30 0x55f32bfb8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.162+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 0x55f32bfb91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.162+0000 7f88b7493b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f32bee6e40 con 0x55f32bfbed30 2024-05-27T08:13:36.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.162+0000 7f88b7493b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f32bed69e0 con 0x55f32be39e40 2024-05-27T08:13:36.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.162+0000 7f88b7493b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f32bedaa20 con 0x55f32bfb8db0 2024-05-27T08:13:36.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.162+0000 7f88a1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfbed30 0x55f32bfb8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.162+0000 7f88a1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfbed30 0x55f32bfb8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53254/0 (socket says 172.21.15.63:53254) 2024-05-27T08:13:36.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.162+0000 7f88a1400700 1 -- 172.21.15.63:0/2193783812 learned_addr learned my addr 172.21.15.63:0/2193783812 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:36.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.163+0000 7f88a1e00700 1 --2- 172.21.15.63:0/2193783812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32be39e40 0x55f32bfbe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.163+0000 7f88a2800700 1 --2- 172.21.15.63:0/2193783812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 0x55f32bfb91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.163+0000 7f88a1400700 1 -- 172.21.15.63:0/2193783812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 msgr2=0x55f32bfb91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:36.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.163+0000 7f88a1400700 1 --2- 172.21.15.63:0/2193783812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 0x55f32bfb91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.163+0000 7f88a1400700 1 -- 172.21.15.63:0/2193783812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32be39e40 msgr2=0x55f32bfbe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:36.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.163+0000 7f88a1400700 1 --2- 172.21.15.63:0/2193783812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32be39e40 0x55f32bfbe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.163+0000 7f88a1400700 1 -- 172.21.15.63:0/2193783812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32be9f420 con 0x55f32bfbed30 2024-05-27T08:13:36.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.164+0000 7f88a2800700 1 --2- 172.21.15.63:0/2193783812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 0x55f32bfb91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:36.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.164+0000 7f88a1400700 1 --2- 172.21.15.63:0/2193783812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfbed30 0x55f32bfb8870 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f8894009410 tx=0x7f8894009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a8b33c4e7ccd4a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:36.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.164+0000 7f8893400700 1 -- 172.21.15.63:0/2193783812 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894019020 con 0x55f32bfbed30 2024-05-27T08:13:36.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.164+0000 7f8893400700 1 -- 172.21.15.63:0/2193783812 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8894015ad0 con 0x55f32bfbed30 2024-05-27T08:13:36.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.164+0000 7f8893400700 1 -- 172.21.15.63:0/2193783812 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894014a80 con 0x55f32bfbed30 2024-05-27T08:13:36.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.166+0000 7f88b7493b80 1 -- 172.21.15.63:0/2193783812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfbed30 msgr2=0x55f32bfb8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:36.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.166+0000 7f88b7493b80 1 --2- 172.21.15.63:0/2193783812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfbed30 0x55f32bfb8870 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f8894009410 tx=0x7f8894009920 comp rx=0 tx=0).stop 2024-05-27T08:13:36.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.166+0000 7f88b7493b80 1 -- 172.21.15.63:0/2193783812 shutdown_connections 2024-05-27T08:13:36.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.166+0000 7f88b7493b80 1 --2- 172.21.15.63:0/2193783812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 0x55f32bfb91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.166+0000 7f88b7493b80 1 --2- 172.21.15.63:0/2193783812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32be39e40 0x55f32bfbe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.166+0000 7f88b7493b80 1 --2- 172.21.15.63:0/2193783812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfbed30 0x55f32bfb8870 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.166+0000 7f88b7493b80 1 -- 172.21.15.63:0/2193783812 >> 172.21.15.63:0/2193783812 conn(0x55f32be99fc0 msgr2=0x55f32bfb38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:36.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.166+0000 7f88b7493b80 1 -- 172.21.15.63:0/2193783812 shutdown_connections 2024-05-27T08:13:36.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.166+0000 7f88b7493b80 1 -- 172.21.15.63:0/2193783812 wait complete. 2024-05-27T08:13:36.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.167+0000 7f88b7493b80 1 -- start start 2024-05-27T08:13:36.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.167+0000 7f88b7493b80 1 Processor -- start 2024-05-27T08:13:36.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.168+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32be39e40 0x55f32bfc25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.168+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 0x55f32bfc2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.168+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfc3040 0x55f32bfceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.168+0000 7f88b7493b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f32bddcfa0 con 0x55f32bfc3040 2024-05-27T08:13:36.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.168+0000 7f88b7493b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f32be77760 con 0x55f32be39e40 2024-05-27T08:13:36.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.168+0000 7f88b7493b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f32bfc5800 con 0x55f32bfb8db0 2024-05-27T08:13:36.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.168+0000 7f88b7493b80 10 client.0 _ll_register_callbacks cb 0x55f32be80c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:36.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.168+0000 7f88a1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32be39e40 0x55f32bfc25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.169+0000 7f88a1400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 0x55f32bfc2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.169+0000 7f88a2800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfc3040 0x55f32bfceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.169+0000 7f88a1400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 0x55f32bfc2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:60208/0 (socket says 172.21.15.63:60208) 2024-05-27T08:13:36.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.169+0000 7f88a1400700 1 -- 172.21.15.63:0/1152028308 learned_addr learned my addr 172.21.15.63:0/1152028308 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:36.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.169+0000 7f88a2800700 1 -- 172.21.15.63:0/1152028308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 msgr2=0x55f32bfc2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:36.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.169+0000 7f88a2800700 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 0x55f32bfc2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.169+0000 7f88a2800700 1 -- 172.21.15.63:0/1152028308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32be39e40 msgr2=0x55f32bfc25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:36.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.169+0000 7f88a2800700 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32be39e40 0x55f32bfc25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.169+0000 7f88a2800700 1 -- 172.21.15.63:0/1152028308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32be268d0 con 0x55f32bfc3040 2024-05-27T08:13:36.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.170+0000 7f88a2800700 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfc3040 0x55f32bfceb30 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f888c0df7a0 tx=0x7f888c0a8ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56802f4d9943b43a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:36.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.170+0000 7f8887e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:36.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.170+0000 7f8887e00700 1 -- 172.21.15.63:0/1152028308 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c01f070 con 0x55f32bfc3040 2024-05-27T08:13:36.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.170+0000 7f88b7493b80 1 -- 172.21.15.63:0/1152028308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8894015ad0 con 0x55f32bfc3040 2024-05-27T08:13:36.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.171+0000 7f8887e00700 1 -- 172.21.15.63:0/1152028308 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f888c0a9590 con 0x55f32bfc3040 2024-05-27T08:13:36.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.171+0000 7f8887e00700 1 -- 172.21.15.63:0/1152028308 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f888c0ad6e0 con 0x55f32bfc3040 2024-05-27T08:13:36.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.171+0000 7f88b7493b80 10 client.23067 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:36.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.171+0000 7f88b7493b80 1 -- 172.21.15.63:0/1152028308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f32bea7060 con 0x55f32bfc3040 2024-05-27T08:13:36.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.172+0000 7f8887e00700 1 -- 172.21.15.63:0/1152028308 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8894015ad0 con 0x55f32bfc3040 2024-05-27T08:13:36.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.172+0000 7f8887e00700 1 -- 172.21.15.63:0/1152028308 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f888c003b10 con 0x55f32bfc3040 2024-05-27T08:13:36.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.172+0000 7f88b7493b80 10 client.23067 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:36.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.172+0000 7f88b7493b80 10 client.23067 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:13:36.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.172+0000 7f88b7493b80 20 client.23067 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:36.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.172+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f32bfd2c10 0x55f32bfd5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.172+0000 7f88b7493b80 4 client.23067 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:13:36.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.172+0000 7f88b7493b80 1 -- 172.21.15.63:0/1152028308 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f32bee8d90 con 0x55f32bfd2c10 2024-05-27T08:13:36.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.173+0000 7f88a1e00700 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f32bfd2c10 0x55f32bfd5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.174+0000 7f88a1e00700 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f32bfd2c10 0x55f32bfd5030 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9ee69723ee3e51a server_cookie=b646f5c6004a2565 in_seq=0 out_seq=0 2024-05-27T08:13:36.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.174+0000 7f8887e00700 10 client.23067 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:13:36.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.176+0000 7f8887e00700 1 -- 172.21.15.63:0/1152028308 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f32bee8d90 con 0x55f32bfd2c10 2024-05-27T08:13:36.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.176+0000 7f8887e00700 10 client.23067 handle_command_reply: tid=0 2024-05-27T08:13:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.176+0000 7f88b7493b80 1 client.23067 shutdown 2024-05-27T08:13:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.176+0000 7f88b7493b80 1 -- 172.21.15.63:0/1152028308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfc3040 msgr2=0x55f32bfceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.176+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfc3040 0x55f32bfceb30 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f888c0df7a0 tx=0x7f888c0a8ec0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.177+0000 7f88b7493b80 1 -- 172.21.15.63:0/1152028308 shutdown_connections 2024-05-27T08:13:36.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.177+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f32bfd2c10 0x55f32bfd5030 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.177+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32bfc3040 0x55f32bfceb30 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.177+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32be39e40 0x55f32bfc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.177+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1152028308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfb8db0 0x55f32bfc2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.177+0000 7f88b7493b80 1 -- 172.21.15.63:0/1152028308 >> 172.21.15.63:0/1152028308 conn(0x55f32be99fc0 msgr2=0x55f32bfb3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:36.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.177+0000 7f8887e00700 0 client.23067 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:13:36.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.177+0000 7f88b7493b80 1 -- 172.21.15.63:0/1152028308 shutdown_connections 2024-05-27T08:13:36.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.177+0000 7f88b7493b80 1 -- 172.21.15.63:0/1152028308 wait complete. 2024-05-27T08:13:36.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.177+0000 7f88b7493b80 20 client.23067 trim_cache size 0 max 16384 2024-05-27T08:13:36.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.197+0000 7f88b7493b80 1 Processor -- start 2024-05-27T08:13:36.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.197+0000 7f88b7493b80 1 -- start start 2024-05-27T08:13:36.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.197+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfb8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.197+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32bfb8db0 0x55f32bfc18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.197+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 0x55f32bfc4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.197+0000 7f88b7493b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41b4f90 con 0x55f32be39e40 2024-05-27T08:13:36.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.197+0000 7f88b7493b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41b2820 con 0x55f32bfb8db0 2024-05-27T08:13:36.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.197+0000 7f88b7493b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41b0690 con 0x55f32bfc3040 2024-05-27T08:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.198+0000 7f88a1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfb8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.198+0000 7f88a1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfb8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53268/0 (socket says 172.21.15.63:53268) 2024-05-27T08:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.198+0000 7f88a1e00700 1 -- 172.21.15.63:0/570510102 learned_addr learned my addr 172.21.15.63:0/570510102 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.198+0000 7f88a2800700 1 --2- 172.21.15.63:0/570510102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 0x55f32bfc4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.198+0000 7f88a1400700 1 --2- 172.21.15.63:0/570510102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32bfb8db0 0x55f32bfc18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.198+0000 7f88a1e00700 1 -- 172.21.15.63:0/570510102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 msgr2=0x55f32bfc4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.198+0000 7f88a1e00700 1 --2- 172.21.15.63:0/570510102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 0x55f32bfc4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.198+0000 7f88a1e00700 1 -- 172.21.15.63:0/570510102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32bfb8db0 msgr2=0x55f32bfc18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.198+0000 7f88a1e00700 1 --2- 172.21.15.63:0/570510102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32bfb8db0 0x55f32bfc18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.198+0000 7f88a1e00700 1 -- 172.21.15.63:0/570510102 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32bfc0b30 con 0x55f32be39e40 2024-05-27T08:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.199+0000 7f88a1e00700 1 --2- 172.21.15.63:0/570510102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfb8870 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f889c0050a0 tx=0x7f889c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79c1f293716ac26 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.199+0000 7f8892a00700 1 -- 172.21.15.63:0/570510102 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c003b60 con 0x55f32be39e40 2024-05-27T08:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.199+0000 7f8892a00700 1 -- 172.21.15.63:0/570510102 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f889c003d00 con 0x55f32be39e40 2024-05-27T08:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.199+0000 7f8892a00700 1 -- 172.21.15.63:0/570510102 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c033420 con 0x55f32be39e40 2024-05-27T08:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.200+0000 7f88b7493b80 1 -- 172.21.15.63:0/570510102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 msgr2=0x55f32bfb8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.200+0000 7f88b7493b80 1 --2- 172.21.15.63:0/570510102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfb8870 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f889c0050a0 tx=0x7f889c01e000 comp rx=0 tx=0).stop 2024-05-27T08:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.201+0000 7f88b7493b80 1 -- 172.21.15.63:0/570510102 shutdown_connections 2024-05-27T08:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.201+0000 7f88b7493b80 1 --2- 172.21.15.63:0/570510102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 0x55f32bfc4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.201+0000 7f88b7493b80 1 --2- 172.21.15.63:0/570510102 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfb8870 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.201+0000 7f88b7493b80 1 --2- 172.21.15.63:0/570510102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32bfb8db0 0x55f32bfc18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.201+0000 7f88b7493b80 1 -- 172.21.15.63:0/570510102 >> 172.21.15.63:0/570510102 conn(0x55f32be99fc0 msgr2=0x55f32bfbc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.201+0000 7f88b7493b80 1 -- 172.21.15.63:0/570510102 shutdown_connections 2024-05-27T08:13:36.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.201+0000 7f88b7493b80 1 -- 172.21.15.63:0/570510102 wait complete. 2024-05-27T08:13:36.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.201+0000 7f88b7493b80 1 -- start start 2024-05-27T08:13:36.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.202+0000 7f88b7493b80 1 Processor -- start 2024-05-27T08:13:36.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.202+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfcaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.202+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32bfb8db0 0x55f32bfcaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.202+0000 7f88b7493b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 0x55f32bfcd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.202+0000 7f88b7493b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41b0ce0 con 0x55f32be39e40 2024-05-27T08:13:36.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.202+0000 7f88b7493b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41b2150 con 0x55f32bfb8db0 2024-05-27T08:13:36.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.202+0000 7f88b7493b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41b13b0 con 0x55f32bfc3040 2024-05-27T08:13:36.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.202+0000 7f88b7493b80 10 client.0 _ll_register_callbacks cb 0x55f32bfbe2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:36.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.203+0000 7f88a1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfcaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.203+0000 7f88a1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfcaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53280/0 (socket says 172.21.15.63:53280) 2024-05-27T08:13:36.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.203+0000 7f88a1e00700 1 -- 172.21.15.63:0/1354208576 learned_addr learned my addr 172.21.15.63:0/1354208576 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:36.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.203+0000 7f88a1400700 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32bfb8db0 0x55f32bfcaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.203+0000 7f88a2800700 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 0x55f32bfcd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.203+0000 7f88a1e00700 1 -- 172.21.15.63:0/1354208576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 msgr2=0x55f32bfcd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:36.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.203+0000 7f88a1e00700 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 0x55f32bfcd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.203+0000 7f88a1e00700 1 -- 172.21.15.63:0/1354208576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32bfb8db0 msgr2=0x55f32bfcaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:36.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.203+0000 7f88a1e00700 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32bfb8db0 0x55f32bfcaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.204+0000 7f88a1e00700 1 -- 172.21.15.63:0/1354208576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32bfcddf0 con 0x55f32be39e40 2024-05-27T08:13:36.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.204+0000 7f88a2800700 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 0x55f32bfcd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:36.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.204+0000 7f88a1e00700 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfcaa50 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f889c009750 tx=0x7f889c00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d2a7f970dbf0e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:36.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.204+0000 7f8887e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:36.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.204+0000 7f8887e00700 1 -- 172.21.15.63:0/1354208576 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c00f660 con 0x55f32be39e40 2024-05-27T08:13:36.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.204+0000 7f8887e00700 1 -- 172.21.15.63:0/1354208576 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f889c00f800 con 0x55f32be39e40 2024-05-27T08:13:36.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.204+0000 7f88b7493b80 1 -- 172.21.15.63:0/1354208576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f889c003d00 con 0x55f32be39e40 2024-05-27T08:13:36.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.205+0000 7f8887e00700 1 -- 172.21.15.63:0/1354208576 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f889c00fa00 con 0x55f32be39e40 2024-05-27T08:13:36.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.205+0000 7f88b7493b80 10 client.23073 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:36.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.205+0000 7f88b7493b80 1 -- 172.21.15.63:0/1354208576 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f32c045480 con 0x55f32be39e40 2024-05-27T08:13:36.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.205+0000 7f8887e00700 1 -- 172.21.15.63:0/1354208576 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f889c003d00 con 0x55f32be39e40 2024-05-27T08:13:36.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.205+0000 7f8887e00700 1 -- 172.21.15.63:0/1354208576 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f889c022040 con 0x55f32be39e40 2024-05-27T08:13:36.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.206+0000 7f88b7493b80 10 client.23073 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:36.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.206+0000 7f88b7493b80 10 client.23073 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:13:36.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.206+0000 7f88b7493b80 20 client.23073 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:36.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.206+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f32c045b30 0x55f32c047f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:36.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.206+0000 7f88b7493b80 4 client.23073 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:13:36.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.206+0000 7f88b7493b80 1 -- 172.21.15.63:0/1354208576 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f32bee2570 con 0x55f32c045b30 2024-05-27T08:13:36.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.206+0000 7f88a1400700 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f32c045b30 0x55f32c047f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:36.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.207+0000 7f88a1400700 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f32c045b30 0x55f32c047f50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=728c261ed409f53 server_cookie=ac6ad825038027be in_seq=0 out_seq=0 2024-05-27T08:13:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.207+0000 7f8887e00700 10 client.23073 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:13:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f8887e00700 1 -- 172.21.15.63:0/1354208576 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f32bee2570 con 0x55f32c045b30 2024-05-27T08:13:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f8887e00700 10 client.23073 handle_command_reply: tid=0 2024-05-27T08:13:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f88b7493b80 1 client.23073 shutdown 2024-05-27T08:13:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f88b7493b80 1 -- 172.21.15.63:0/1354208576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 msgr2=0x55f32bfcaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfcaa50 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f889c009750 tx=0x7f889c00bf30 comp rx=0 tx=0).stop 2024-05-27T08:13:36.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:35 smithi086 ceph-mon[34687]: pgmap v1270: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T08:13:36.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f88b7493b80 1 -- 172.21.15.63:0/1354208576 shutdown_connections 2024-05-27T08:13:36.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f32bfc3040 0x55f32bfcd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f32c045b30 0x55f32c047f50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f32be39e40 0x55f32bfcaa50 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f88b7493b80 1 --2- 172.21.15.63:0/1354208576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f32bfb8db0 0x55f32bfcaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f88b7493b80 1 -- 172.21.15.63:0/1354208576 >> 172.21.15.63:0/1354208576 conn(0x55f32be99fc0 msgr2=0x55f32bfbcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:36.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.208+0000 7f8887e00700 0 client.23073 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:13:36.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.209+0000 7f88b7493b80 1 -- 172.21.15.63:0/1354208576 shutdown_connections 2024-05-27T08:13:36.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.209+0000 7f88b7493b80 1 -- 172.21.15.63:0/1354208576 wait complete. 2024-05-27T08:13:36.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.209+0000 7f88b7493b80 20 client.23073 trim_cache size 0 max 16384 2024-05-27T08:13:36.212 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:13:36.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 -- 172.21.15.63:0/493672141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88880b29f0 msgr2=0x7f88880b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:36.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88880b29f0 0x7f88880b4f00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f889c00b2e0 tx=0x7f889c005fb0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 -- 172.21.15.63:0/493672141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a4110bf0 msgr2=0x7f88a41a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:36.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a4110bf0 0x7f88a41a17c0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f888c00d740 tx=0x7f888c00da50 comp rx=0 tx=0).stop 2024-05-27T08:13:36.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 -- 172.21.15.63:0/493672141 shutdown_connections 2024-05-27T08:13:36.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88880b29f0 0x7f88880b4f00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88a4110bf0 0x7f88a41a17c0 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88a4075af0 0x7f88a419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 --2- 172.21.15.63:0/493672141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88a4076440 0x7f88a419d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:36.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 -- 172.21.15.63:0/493672141 >> 172.21.15.63:0/493672141 conn(0x7f88a4100030 msgr2=0x7f88a4100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:36.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 -- 172.21.15.63:0/493672141 shutdown_connections 2024-05-27T08:13:36.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:36.210+0000 7f88a8a00700 1 -- 172.21.15.63:0/493672141 wait complete. 2024-05-27T08:13:36.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:35 smithi149 ceph-mon[35408]: pgmap v1270: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T08:13:37.186 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:13:37.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:37 smithi063 ceph-mon[31613]: pgmap v1271: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.0 MiB/s wr, 18 op/s 2024-05-27T08:13:37.699 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:13:37.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:37 smithi086 ceph-mon[34687]: pgmap v1271: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.0 MiB/s wr, 18 op/s 2024-05-27T08:13:37.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:37 smithi149 ceph-mon[35408]: pgmap v1271: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.0 MiB/s wr, 18 op/s 2024-05-27T08:13:40.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:39 smithi086 ceph-mon[34687]: pgmap v1272: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T08:13:40.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:39 smithi149 ceph-mon[35408]: pgmap v1272: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T08:13:40.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:39 smithi063 ceph-mon[31613]: pgmap v1272: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T08:13:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.422+0000 7fee29800700 1 -- 172.21.15.63:0/2364088494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee241030b0 msgr2=0x7fee241034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.422+0000 7fee29800700 1 --2- 172.21.15.63:0/2364088494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee241030b0 0x7fee241034c0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7fee14009990 tx=0x7fee14009ca0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.422+0000 7fee29800700 1 -- 172.21.15.63:0/2364088494 shutdown_connections 2024-05-27T08:13:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.422+0000 7fee29800700 1 --2- 172.21.15.63:0/2364088494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee24110c60 0x7fee24113080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.422+0000 7fee29800700 1 --2- 172.21.15.63:0/2364088494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee241030b0 0x7fee241034c0 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.422+0000 7fee29800700 1 --2- 172.21.15.63:0/2364088494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee24103a00 0x7fee24110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.422+0000 7fee29800700 1 -- 172.21.15.63:0/2364088494 >> 172.21.15.63:0/2364088494 conn(0x7fee240fedb0 msgr2=0x7fee241011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.422+0000 7fee29800700 1 -- 172.21.15.63:0/2364088494 shutdown_connections 2024-05-27T08:13:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.422+0000 7fee29800700 1 -- 172.21.15.63:0/2364088494 wait complete. 2024-05-27T08:13:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.424+0000 7fee29800700 1 Processor -- start 2024-05-27T08:13:40.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.424+0000 7fee29800700 1 -- start start 2024-05-27T08:13:40.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.424+0000 7fee29800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee241030b0 0x7fee2419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.424+0000 7fee29800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee24103a00 0x7fee2419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.424+0000 7fee29800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee24110c60 0x7fee241a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.424+0000 7fee29800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee24116550 con 0x7fee24103a00 2024-05-27T08:13:40.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.424+0000 7fee29800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee241163d0 con 0x7fee24110c60 2024-05-27T08:13:40.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.424+0000 7fee29800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee241166d0 con 0x7fee241030b0 2024-05-27T08:13:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee22a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee241030b0 0x7fee2419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee22000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee24103a00 0x7fee2419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee22a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee241030b0 0x7fee2419cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:34568/0 (socket says 172.21.15.63:34568) 2024-05-27T08:13:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee22a00700 1 -- 172.21.15.63:0/4165312783 learned_addr learned my addr 172.21.15.63:0/4165312783 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee23400700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee24110c60 0x7fee241a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee22a00700 1 -- 172.21.15.63:0/4165312783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee24110c60 msgr2=0x7fee241a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee22a00700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee24110c60 0x7fee241a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee22a00700 1 -- 172.21.15.63:0/4165312783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee24103a00 msgr2=0x7fee2419d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee22a00700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee24103a00 0x7fee2419d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee22a00700 1 -- 172.21.15.63:0/4165312783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee241a1e50 con 0x7fee241030b0 2024-05-27T08:13:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.425+0000 7fee23400700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee24110c60 0x7fee241a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.426+0000 7fee22a00700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee241030b0 0x7fee2419cf80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee140098e0 tx=0x7fee140057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.426+0000 7fee1b400700 1 -- 172.21.15.63:0/4165312783 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14027070 con 0x7fee241030b0 2024-05-27T08:13:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.426+0000 7fee29800700 1 -- 172.21.15.63:0/4165312783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee241a20e0 con 0x7fee241030b0 2024-05-27T08:13:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.426+0000 7fee1b400700 1 -- 172.21.15.63:0/4165312783 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee14022470 con 0x7fee241030b0 2024-05-27T08:13:40.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.426+0000 7fee1b400700 1 -- 172.21.15.63:0/4165312783 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14017650 con 0x7fee241030b0 2024-05-27T08:13:40.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.426+0000 7fee29800700 1 -- 172.21.15.63:0/4165312783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee241a9a60 con 0x7fee241030b0 2024-05-27T08:13:40.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.432+0000 7fee1b400700 1 -- 172.21.15.63:0/4165312783 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fee1402a070 con 0x7fee241030b0 2024-05-27T08:13:40.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.433+0000 7fee38434b80 1 Processor -- start 2024-05-27T08:13:40.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.433+0000 7fee38434b80 1 -- start start 2024-05-27T08:13:40.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.434+0000 7fee1b400700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee080b2a70 0x7fee080b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.434+0000 7fee1b400700 1 -- 172.21.15.63:0/4165312783 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7fee140e3930 con 0x7fee241030b0 2024-05-27T08:13:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.434+0000 7fee38434b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x5580065fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.434+0000 7fee38434b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 0x5580065fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.435+0000 7fee38434b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 0x558006609920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.435+0000 7fee38434b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558006522e40 con 0x558006475e40 2024-05-27T08:13:40.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.435+0000 7fee38434b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5580065129e0 con 0x5580065619f0 2024-05-27T08:13:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.435+0000 7fee38434b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558006516a20 con 0x5580065fd690 2024-05-27T08:13:40.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.435+0000 7fee23400700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee080b2a70 0x7fee080b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.435+0000 7fee22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x5580065fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.435+0000 7fee23400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 0x558006609920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.435+0000 7fee23400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 0x558006609920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:34576/0 (socket says 172.21.15.63:34576) 2024-05-27T08:13:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.436+0000 7fee23400700 1 -- 172.21.15.63:0/733002807 learned_addr learned my addr 172.21.15.63:0/733002807 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.435+0000 7fee22a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x5580065fcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51122/0 (socket says 172.21.15.63:51122) 2024-05-27T08:13:40.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.436+0000 7fee22000700 1 --2- 172.21.15.63:0/733002807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 0x5580065fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.436+0000 7fee22a00700 1 -- 172.21.15.63:0/733002807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 msgr2=0x558006609920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.436+0000 7fee22a00700 1 --2- 172.21.15.63:0/733002807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 0x558006609920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.436+0000 7fee22a00700 1 -- 172.21.15.63:0/733002807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 msgr2=0x5580065fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.436+0000 7fee22a00700 1 --2- 172.21.15.63:0/733002807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 0x5580065fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.436+0000 7fee22a00700 1 -- 172.21.15.63:0/733002807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580064db420 con 0x5580065619f0 2024-05-27T08:13:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.436+0000 7fee22000700 1 --2- 172.21.15.63:0/733002807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 0x5580065fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:13:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.437+0000 7fee22a00700 1 --2- 172.21.15.63:0/733002807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x5580065fcb50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee140e0cc0 tx=0x7fee140aa7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf99bcb50e296dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.437+0000 7fee23400700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee080b2a70 0x7fee080b4f80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fee1c007cf0 tx=0x7fee1c009300 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:40.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.437+0000 7fee0fe00700 1 -- 172.21.15.63:0/733002807 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee140aafe0 con 0x5580065619f0 2024-05-27T08:13:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.438+0000 7fee0fe00700 1 -- 172.21.15.63:0/733002807 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee140aa8e0 con 0x5580065619f0 2024-05-27T08:13:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.438+0000 7fee0fe00700 1 -- 172.21.15.63:0/733002807 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee140cd070 con 0x5580065619f0 2024-05-27T08:13:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.439+0000 7fee38434b80 1 -- 172.21.15.63:0/733002807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 msgr2=0x5580065fcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:40.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.439+0000 7fee38434b80 1 --2- 172.21.15.63:0/733002807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x5580065fcb50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee140e0cc0 tx=0x7fee140aa7c0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.439+0000 7fee38434b80 1 -- 172.21.15.63:0/733002807 shutdown_connections 2024-05-27T08:13:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.439+0000 7fee38434b80 1 --2- 172.21.15.63:0/733002807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 0x558006609920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.439+0000 7fee38434b80 1 --2- 172.21.15.63:0/733002807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x5580065fcb50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.439+0000 7fee38434b80 1 --2- 172.21.15.63:0/733002807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 0x5580065fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.439+0000 7fee38434b80 1 -- 172.21.15.63:0/733002807 >> 172.21.15.63:0/733002807 conn(0x55800650f390 msgr2=0x5580065f0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.439+0000 7fee38434b80 1 -- 172.21.15.63:0/733002807 shutdown_connections 2024-05-27T08:13:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.439+0000 7fee38434b80 1 -- 172.21.15.63:0/733002807 wait complete. 2024-05-27T08:13:40.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.440+0000 7fee38434b80 1 -- start start 2024-05-27T08:13:40.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.441+0000 7fee38434b80 1 Processor -- start 2024-05-27T08:13:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee38434b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 0x5580066070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee38434b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x558006607630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee38434b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 0x558006607b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee38434b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558006418fa0 con 0x5580065619f0 2024-05-27T08:13:40.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee38434b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5580064b3760 con 0x5580065fd690 2024-05-27T08:13:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee38434b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55800660a160 con 0x558006475e40 2024-05-27T08:13:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee38434b80 10 client.0 _ll_register_callbacks cb 0x5580064bcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee22a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 0x5580066070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee22000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x558006607630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee22000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x558006607630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37790/0 (socket says 172.21.15.63:37790) 2024-05-27T08:13:40.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.442+0000 7fee22000700 1 -- 172.21.15.63:0/3001094304 learned_addr learned my addr 172.21.15.63:0/3001094304 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.443+0000 7fee23400700 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 0x558006607b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.443+0000 7fee22a00700 1 -- 172.21.15.63:0/3001094304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 msgr2=0x558006607b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.443+0000 7fee22a00700 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 0x558006607b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.443+0000 7fee22a00700 1 -- 172.21.15.63:0/3001094304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 msgr2=0x558006607630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.443+0000 7fee22a00700 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x558006607630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.443+0000 7fee22a00700 1 -- 172.21.15.63:0/3001094304 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580064628d0 con 0x558006475e40 2024-05-27T08:13:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.443+0000 7fee22000700 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x558006607630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:13:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.443+0000 7fee23400700 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 0x558006607b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:13:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.443+0000 7fee22a00700 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 0x5580066070f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee140aee50 tx=0x7fee140a68d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16b0f4191c0fb327 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.443+0000 7fee0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:13:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.444+0000 7fee0cc00700 1 -- 172.21.15.63:0/3001094304 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee140aafe0 con 0x558006475e40 2024-05-27T08:13:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.444+0000 7fee0cc00700 1 -- 172.21.15.63:0/3001094304 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7fee140a6ed0 con 0x558006475e40 2024-05-27T08:13:40.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.444+0000 7fee38434b80 1 -- 172.21.15.63:0/3001094304 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee140aa8e0 con 0x558006475e40 2024-05-27T08:13:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.444+0000 7fee0cc00700 1 -- 172.21.15.63:0/3001094304 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fee140b36b0 con 0x558006475e40 2024-05-27T08:13:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.444+0000 7fee38434b80 10 client.31777 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.444+0000 7fee38434b80 1 -- 172.21.15.63:0/3001094304 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5580064e3060 con 0x558006475e40 2024-05-27T08:13:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.445+0000 7fee0cc00700 1 -- 172.21.15.63:0/3001094304 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee140aa8e0 con 0x558006475e40 2024-05-27T08:13:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.445+0000 7fee0cc00700 1 -- 172.21.15.63:0/3001094304 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fee140aab40 con 0x558006475e40 2024-05-27T08:13:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.445+0000 7fee38434b80 10 client.31777 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.445+0000 7fee38434b80 10 client.31777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:40.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.445+0000 7fee38434b80 20 client.31777 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.446+0000 7fee38434b80 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558006616f00 0x558006619320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.446+0000 7fee38434b80 4 client.31777 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:13:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.446+0000 7fee38434b80 1 -- 172.21.15.63:0/3001094304 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558006524d90 con 0x558006616f00 2024-05-27T08:13:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.446+0000 7fee22000700 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558006616f00 0x558006619320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.447+0000 7fee22000700 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558006616f00 0x558006619320 crc :-1 s=READY pgs=710 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c720099af4369c2 server_cookie=3af2eddfd9f32284 in_seq=0 out_seq=0 2024-05-27T08:13:40.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.447+0000 7fee0cc00700 10 client.31777 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.448+0000 7fee0cc00700 1 -- 172.21.15.63:0/3001094304 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558006524d90 con 0x558006616f00 2024-05-27T08:13:40.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.448+0000 7fee0cc00700 10 client.31777 handle_command_reply: tid=0 2024-05-27T08:13:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.448+0000 7fee38434b80 1 client.31777 shutdown 2024-05-27T08:13:40.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 1 -- 172.21.15.63:0/3001094304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 msgr2=0x5580066070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 0x5580066070f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee140aee50 tx=0x7fee140a68d0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 1 -- 172.21.15.63:0/3001094304 shutdown_connections 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558006616f00 0x558006619320 crc :-1 s=READY pgs=710 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 0x558006607b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 0x5580066070f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 1 --2- 172.21.15.63:0/3001094304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x558006607630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee0cc00700 0 client.31777 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 1 -- 172.21.15.63:0/3001094304 >> 172.21.15.63:0/3001094304 conn(0x55800650f390 msgr2=0x5580065fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 1 -- 172.21.15.63:0/3001094304 shutdown_connections 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 1 -- 172.21.15.63:0/3001094304 wait complete. 2024-05-27T08:13:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.449+0000 7fee38434b80 20 client.31777 trim_cache size 0 max 16384 2024-05-27T08:13:40.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.471+0000 7fee38434b80 1 Processor -- start 2024-05-27T08:13:40.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.471+0000 7fee38434b80 1 -- start start 2024-05-27T08:13:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.471+0000 7fee38434b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 0x5580065f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.471+0000 7fee38434b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x558006606050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.471+0000 7fee38434b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 0x5580066028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.471+0000 7fee38434b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee241b0aa0 con 0x558006475e40 2024-05-27T08:13:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.471+0000 7fee38434b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee241ae330 con 0x5580065619f0 2024-05-27T08:13:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.471+0000 7fee38434b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee241ac1a0 con 0x5580065fd690 2024-05-27T08:13:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.472+0000 7fee22000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x558006606050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.472+0000 7fee22a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 0x5580065f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.472+0000 7fee22000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x558006606050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51132/0 (socket says 172.21.15.63:51132) 2024-05-27T08:13:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.472+0000 7fee22000700 1 -- 172.21.15.63:0/4247651928 learned_addr learned my addr 172.21.15.63:0/4247651928 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.472+0000 7fee23400700 1 --2- 172.21.15.63:0/4247651928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 0x5580066028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.472+0000 7fee22000700 1 -- 172.21.15.63:0/4247651928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 msgr2=0x5580066028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.472+0000 7fee22000700 1 --2- 172.21.15.63:0/4247651928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 0x5580066028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.472+0000 7fee22000700 1 -- 172.21.15.63:0/4247651928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 msgr2=0x5580065f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.472+0000 7fee22000700 1 --2- 172.21.15.63:0/4247651928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 0x5580065f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.472+0000 7fee22000700 1 -- 172.21.15.63:0/4247651928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580065f0e60 con 0x5580065619f0 2024-05-27T08:13:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.473+0000 7fee22000700 1 --2- 172.21.15.63:0/4247651928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x558006606050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5580065eca00 tx=0x7fee10014910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40ee5b233d8a8a51 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.473+0000 7fee0f400700 1 -- 172.21.15.63:0/4247651928 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10004030 con 0x5580065619f0 2024-05-27T08:13:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.473+0000 7fee0f400700 1 -- 172.21.15.63:0/4247651928 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee10024070 con 0x5580065619f0 2024-05-27T08:13:40.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.473+0000 7fee0f400700 1 -- 172.21.15.63:0/4247651928 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1001e470 con 0x5580065619f0 2024-05-27T08:13:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.475+0000 7fee38434b80 1 -- 172.21.15.63:0/4247651928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 msgr2=0x558006606050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.475+0000 7fee38434b80 1 --2- 172.21.15.63:0/4247651928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x558006606050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5580065eca00 tx=0x7fee10014910 comp rx=0 tx=0).stop 2024-05-27T08:13:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.475+0000 7fee38434b80 1 -- 172.21.15.63:0/4247651928 shutdown_connections 2024-05-27T08:13:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.475+0000 7fee38434b80 1 --2- 172.21.15.63:0/4247651928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5580065fd690 0x5580066028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.475+0000 7fee38434b80 1 --2- 172.21.15.63:0/4247651928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558006475e40 0x5580065f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.475+0000 7fee38434b80 1 --2- 172.21.15.63:0/4247651928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065619f0 0x558006606050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.475+0000 7fee38434b80 1 -- 172.21.15.63:0/4247651928 >> 172.21.15.63:0/4247651928 conn(0x55800650f390 msgr2=0x5580066006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.475+0000 7fee38434b80 1 -- 172.21.15.63:0/4247651928 shutdown_connections 2024-05-27T08:13:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.475+0000 7fee38434b80 1 -- 172.21.15.63:0/4247651928 wait complete. 2024-05-27T08:13:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.476+0000 7fee38434b80 1 -- start start 2024-05-27T08:13:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.476+0000 7fee38434b80 1 Processor -- start 2024-05-27T08:13:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.476+0000 7fee38434b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 0x55800660a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.476+0000 7fee38434b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x55800660a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.477+0000 7fee38434b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 0x55800660ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.477+0000 7fee38434b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee241ac7f0 con 0x5580065619f0 2024-05-27T08:13:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.477+0000 7fee38434b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee241adc60 con 0x5580065fd690 2024-05-27T08:13:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.477+0000 7fee38434b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee241acec0 con 0x558006475e40 2024-05-27T08:13:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.477+0000 7fee38434b80 10 client.0 _ll_register_callbacks cb 0x5580065fd5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.477+0000 7fee22000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x55800660a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.477+0000 7fee22000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x55800660a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37802/0 (socket says 172.21.15.63:37802) 2024-05-27T08:13:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.477+0000 7fee22000700 1 -- 172.21.15.63:0/1254026303 learned_addr learned my addr 172.21.15.63:0/1254026303 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.477+0000 7fee22a00700 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 0x55800660a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.478+0000 7fee23400700 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 0x55800660ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.478+0000 7fee22000700 1 -- 172.21.15.63:0/1254026303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 msgr2=0x55800660a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.478+0000 7fee22000700 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 0x55800660a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.478+0000 7fee22000700 1 -- 172.21.15.63:0/1254026303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 msgr2=0x55800660ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.478+0000 7fee22000700 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 0x55800660ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.478+0000 7fee22000700 1 -- 172.21.15.63:0/1254026303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558006607ee0 con 0x5580065619f0 2024-05-27T08:13:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.478+0000 7fee22000700 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x55800660a640 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fee10005e90 tx=0x7fee10003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ea54fe8d3b828a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.478+0000 7fee0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.479+0000 7fee0cc00700 1 -- 172.21.15.63:0/1254026303 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10005460 con 0x5580065619f0 2024-05-27T08:13:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.479+0000 7fee0cc00700 1 -- 172.21.15.63:0/1254026303 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7fee10005600 con 0x5580065619f0 2024-05-27T08:13:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.479+0000 7fee38434b80 1 -- 172.21.15.63:0/1254026303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee10024070 con 0x5580065619f0 2024-05-27T08:13:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.479+0000 7fee0cc00700 1 -- 172.21.15.63:0/1254026303 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fee10004750 con 0x5580065619f0 2024-05-27T08:13:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.479+0000 7fee38434b80 10 client.23088 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.479+0000 7fee38434b80 1 -- 172.21.15.63:0/1254026303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558006680b60 con 0x5580065619f0 2024-05-27T08:13:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.480+0000 7fee0cc00700 1 -- 172.21.15.63:0/1254026303 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee10024070 con 0x5580065619f0 2024-05-27T08:13:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.480+0000 7fee0cc00700 1 -- 172.21.15.63:0/1254026303 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fee1001ea50 con 0x5580065619f0 2024-05-27T08:13:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.480+0000 7fee38434b80 10 client.23088 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.480+0000 7fee38434b80 10 client.23088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.480+0000 7fee38434b80 20 client.23088 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.480+0000 7fee38434b80 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558006681270 0x558006683690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.480+0000 7fee38434b80 4 client.23088 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:13:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.480+0000 7fee38434b80 1 -- 172.21.15.63:0/1254026303 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55800651e570 con 0x558006681270 2024-05-27T08:13:40.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.481+0000 7fee22a00700 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558006681270 0x558006683690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.481+0000 7fee22a00700 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558006681270 0x558006683690 crc :-1 s=READY pgs=711 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71ad57628fd466d2 server_cookie=42e19721c58d1f62 in_seq=0 out_seq=0 2024-05-27T08:13:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.481+0000 7fee0cc00700 10 client.23088 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee0cc00700 1 -- 172.21.15.63:0/1254026303 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55800651e570 con 0x558006681270 2024-05-27T08:13:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee0cc00700 10 client.23088 handle_command_reply: tid=0 2024-05-27T08:13:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee38434b80 1 client.23088 shutdown 2024-05-27T08:13:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee38434b80 1 -- 172.21.15.63:0/1254026303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 msgr2=0x55800660a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee38434b80 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x55800660a640 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fee10005e90 tx=0x7fee10003730 comp rx=0 tx=0).stop 2024-05-27T08:13:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee38434b80 1 -- 172.21.15.63:0/1254026303 shutdown_connections 2024-05-27T08:13:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee38434b80 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558006681270 0x558006683690 crc :-1 s=READY pgs=711 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee38434b80 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580065fd690 0x55800660ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee38434b80 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558006475e40 0x55800660a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee38434b80 1 --2- 172.21.15.63:0/1254026303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580065619f0 0x55800660a640 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee38434b80 1 -- 172.21.15.63:0/1254026303 >> 172.21.15.63:0/1254026303 conn(0x55800650f390 msgr2=0x558006607660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:40.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.482+0000 7fee0cc00700 0 client.23088 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:40.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.483+0000 7fee38434b80 1 -- 172.21.15.63:0/1254026303 shutdown_connections 2024-05-27T08:13:40.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.483+0000 7fee38434b80 1 -- 172.21.15.63:0/1254026303 wait complete. 2024-05-27T08:13:40.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.483+0000 7fee38434b80 20 client.23088 trim_cache size 0 max 16384 2024-05-27T08:13:40.485 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:13:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 -- 172.21.15.63:0/4165312783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee080b2a70 msgr2=0x7fee080b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee080b2a70 0x7fee080b4f80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fee1c007cf0 tx=0x7fee1c009300 comp rx=0 tx=0).stop 2024-05-27T08:13:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 -- 172.21.15.63:0/4165312783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee241030b0 msgr2=0x7fee2419cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee241030b0 0x7fee2419cf80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee140098e0 tx=0x7fee140057c0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 -- 172.21.15.63:0/4165312783 shutdown_connections 2024-05-27T08:13:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee080b2a70 0x7fee080b4f80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee24110c60 0x7fee241a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee241030b0 0x7fee2419cf80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 --2- 172.21.15.63:0/4165312783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee24103a00 0x7fee2419d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:40.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 -- 172.21.15.63:0/4165312783 >> 172.21.15.63:0/4165312783 conn(0x7fee240fedb0 msgr2=0x7fee24112890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:40.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.484+0000 7fee29800700 1 -- 172.21.15.63:0/4165312783 shutdown_connections 2024-05-27T08:13:40.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:40.485+0000 7fee29800700 1 -- 172.21.15.63:0/4165312783 wait complete. 2024-05-27T08:13:41.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:40 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:13:41.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:40 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:13:41.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:40 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:13:42.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:41 smithi086 ceph-mon[34687]: pgmap v1273: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T08:13:42.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:41 smithi149 ceph-mon[35408]: pgmap v1273: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T08:13:42.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:41 smithi063 ceph-mon[31613]: pgmap v1273: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.4 MiB/s wr, 10 op/s 2024-05-27T08:13:42.437 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:13:42.437 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:13:42.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:42 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:13:42.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:42 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:13:42.827 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:42 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:13:42.942 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:13:43.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:42 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:13:43.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:42 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:13:43.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:42 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:13:43.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:42 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:13:43.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:42 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:13:43.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:42 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:13:44.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:43 smithi063 ceph-mon[31613]: pgmap v1274: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T08:13:44.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:43 smithi086 ceph-mon[34687]: pgmap v1274: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T08:13:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:43 smithi149 ceph-mon[35408]: pgmap v1274: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-05-27T08:13:45.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.334+0000 7f93ff600700 1 -- 172.21.15.63:0/4078344743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93f8108730 msgr2=0x7f93f8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:45.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.334+0000 7f93ff600700 1 --2- 172.21.15.63:0/4078344743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93f8108730 0x7f93f8110090 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f93f000b0d0 tx=0x7f93f000b3e0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.334+0000 7f93ff600700 1 -- 172.21.15.63:0/4078344743 shutdown_connections 2024-05-27T08:13:45.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.334+0000 7f93ff600700 1 --2- 172.21.15.63:0/4078344743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93f8108730 0x7f93f8110090 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.334+0000 7f93ff600700 1 --2- 172.21.15.63:0/4078344743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f81034b0 0x7f93f81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.334+0000 7f93ff600700 1 --2- 172.21.15.63:0/4078344743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f8107de0 0x7f93f81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.334+0000 7f93ff600700 1 -- 172.21.15.63:0/4078344743 >> 172.21.15.63:0/4078344743 conn(0x7f93f80fedb0 msgr2=0x7f93f81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:45.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.334+0000 7f93ff600700 1 -- 172.21.15.63:0/4078344743 shutdown_connections 2024-05-27T08:13:45.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.334+0000 7f93ff600700 1 -- 172.21.15.63:0/4078344743 wait complete. 2024-05-27T08:13:45.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.335+0000 7f93ff600700 1 Processor -- start 2024-05-27T08:13:45.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.336+0000 7f93ff600700 1 -- start start 2024-05-27T08:13:45.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.336+0000 7f93ff600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f81034b0 0x7f93f81a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.336+0000 7f93ff600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93f8107de0 0x7f93f81a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.336+0000 7f93ff600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f8108730 0x7f93f819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.336+0000 7f93ff600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8113260 con 0x7f93f8107de0 2024-05-27T08:13:45.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.336+0000 7f93ff600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81130e0 con 0x7f93f81034b0 2024-05-27T08:13:45.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.336+0000 7f93ff600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81133e0 con 0x7f93f8108730 2024-05-27T08:13:45.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.337+0000 7f93f7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93f8107de0 0x7f93f81a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.337+0000 7f93f7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93f8107de0 0x7f93f81a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37812/0 (socket says 172.21.15.63:37812) 2024-05-27T08:13:45.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.337+0000 7f93f7e00700 1 -- 172.21.15.63:0/1003411377 learned_addr learned my addr 172.21.15.63:0/1003411377 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:45.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.337+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f81034b0 0x7f93f81a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.337+0000 7f93fd600700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f8108730 0x7f93f819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.337+0000 7f93f7e00700 1 -- 172.21.15.63:0/1003411377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f8108730 msgr2=0x7f93f819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:45.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.337+0000 7f93f7e00700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f8108730 0x7f93f819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.338+0000 7f93f7e00700 1 -- 172.21.15.63:0/1003411377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f81034b0 msgr2=0x7f93f81a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:45.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.338+0000 7f93f7e00700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f81034b0 0x7f93f81a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.338+0000 7f93f7e00700 1 -- 172.21.15.63:0/1003411377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f819b1b0 con 0x7f93f8107de0 2024-05-27T08:13:45.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.338+0000 7f93f7e00700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93f8107de0 0x7f93f81a0df0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f93e400d980 tx=0x7f93e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:45.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.338+0000 7f93f5600700 1 -- 172.21.15.63:0/1003411377 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4017070 con 0x7f93f8107de0 2024-05-27T08:13:45.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.338+0000 7f93f5600700 1 -- 172.21.15.63:0/1003411377 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93e400fd20 con 0x7f93f8107de0 2024-05-27T08:13:45.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.338+0000 7f93ff600700 1 -- 172.21.15.63:0/1003411377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f819b4a0 con 0x7f93f8107de0 2024-05-27T08:13:45.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.339+0000 7f93f5600700 1 -- 172.21.15.63:0/1003411377 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e400e3b0 con 0x7f93f8107de0 2024-05-27T08:13:45.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.339+0000 7f93ff600700 1 -- 172.21.15.63:0/1003411377 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a7740 con 0x7f93f8107de0 2024-05-27T08:13:45.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.343+0000 7f93f5600700 1 -- 172.21.15.63:0/1003411377 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f93e400e550 con 0x7f93f8107de0 2024-05-27T08:13:45.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.344+0000 7f940e112b80 1 Processor -- start 2024-05-27T08:13:45.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.344+0000 7f940e112b80 1 -- start start 2024-05-27T08:13:45.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.345+0000 7f940e112b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0968c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.345+0000 7f940e112b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce0969160 0x55cce0962d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.345+0000 7f940e112b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0963240 0x55cce0963650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.345+0000 7f940e112b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cce088cec0 con 0x55cce07dfec0 2024-05-27T08:13:45.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.345+0000 7f940e112b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cce087ca60 con 0x55cce0969160 2024-05-27T08:13:45.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.345+0000 7f940e112b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cce0880aa0 con 0x55cce0963240 2024-05-27T08:13:45.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.345+0000 7f93fd600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0968c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.345+0000 7f93fd600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0968c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37824/0 (socket says 172.21.15.63:37824) 2024-05-27T08:13:45.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.345+0000 7f93fd600700 1 -- 172.21.15.63:0/2727471530 learned_addr learned my addr 172.21.15.63:0/2727471530 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:45.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.346+0000 7f93f7e00700 1 --2- 172.21.15.63:0/2727471530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0963240 0x55cce0963650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.346+0000 7f93f5600700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93e00b2a70 0x7f93e00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.346+0000 7f93fcc00700 1 --2- 172.21.15.63:0/2727471530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce0969160 0x55cce0962d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.346+0000 7f93fd600700 1 -- 172.21.15.63:0/2727471530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0963240 msgr2=0x55cce0963650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:45.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.346+0000 7f93fd600700 1 --2- 172.21.15.63:0/2727471530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0963240 0x55cce0963650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.346+0000 7f93fd600700 1 -- 172.21.15.63:0/2727471530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce0969160 msgr2=0x55cce0962d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:45.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.346+0000 7f93fd600700 1 --2- 172.21.15.63:0/2727471530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce0969160 0x55cce0962d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.346+0000 7f93f5600700 1 -- 172.21.15.63:0/1003411377 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f93e40e23d0 con 0x7f93f8107de0 2024-05-27T08:13:45.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.346+0000 7f93fd600700 1 -- 172.21.15.63:0/2727471530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cce08454a0 con 0x55cce07dfec0 2024-05-27T08:13:45.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.347+0000 7f93fd600700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93e00b2a70 0x7f93e00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.347+0000 7f93fd600700 1 --2- 172.21.15.63:0/2727471530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0968c20 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009c50 tx=0x7f93f0009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfdf52df5e227ef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:45.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.347+0000 7f93ea000700 1 -- 172.21.15.63:0/2727471530 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f002e070 con 0x55cce07dfec0 2024-05-27T08:13:45.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.347+0000 7f93ea000700 1 -- 172.21.15.63:0/2727471530 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f0003c80 con 0x55cce07dfec0 2024-05-27T08:13:45.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.348+0000 7f93ea000700 1 -- 172.21.15.63:0/2727471530 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0027410 con 0x55cce07dfec0 2024-05-27T08:13:45.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.348+0000 7f940e112b80 1 -- 172.21.15.63:0/2727471530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 msgr2=0x55cce0968c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:45.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.348+0000 7f93fd600700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93e00b2a70 0x7f93e00b4f80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f93f002b000 tx=0x7f93f00077e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:45.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.348+0000 7f940e112b80 1 --2- 172.21.15.63:0/2727471530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0968c20 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009c50 tx=0x7f93f0009c80 comp rx=0 tx=0).stop 2024-05-27T08:13:45.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.349+0000 7f940e112b80 1 -- 172.21.15.63:0/2727471530 shutdown_connections 2024-05-27T08:13:45.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.349+0000 7f940e112b80 1 --2- 172.21.15.63:0/2727471530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0963240 0x55cce0963650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.349+0000 7f940e112b80 1 --2- 172.21.15.63:0/2727471530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0968c20 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.349+0000 7f940e112b80 1 --2- 172.21.15.63:0/2727471530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce0969160 0x55cce0962d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.349+0000 7f940e112b80 1 -- 172.21.15.63:0/2727471530 >> 172.21.15.63:0/2727471530 conn(0x55cce0840040 msgr2=0x55cce0959950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:45.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.349+0000 7f940e112b80 1 -- 172.21.15.63:0/2727471530 shutdown_connections 2024-05-27T08:13:45.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.349+0000 7f940e112b80 1 -- 172.21.15.63:0/2727471530 wait complete. 2024-05-27T08:13:45.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.350+0000 7f940e112b80 1 -- start start 2024-05-27T08:13:45.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.350+0000 7f940e112b80 1 Processor -- start 2024-05-27T08:13:45.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f940e112b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 0x55cce096c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f940e112b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce0963240 0x55cce096cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f940e112b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce096d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f940e112b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cce0783020 con 0x55cce0963240 2024-05-27T08:13:45.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f940e112b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cce081d7e0 con 0x55cce07dfec0 2024-05-27T08:13:45.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f940e112b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cce096fc90 con 0x55cce0969160 2024-05-27T08:13:45.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f940e112b80 10 client.0 _ll_register_callbacks cb 0x55cce0826cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:45.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f93f7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce0963240 0x55cce096cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f93f7e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce0963240 0x55cce096cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37830/0 (socket says 172.21.15.63:37830) 2024-05-27T08:13:45.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f93f7e00700 1 -- 172.21.15.63:0/1342082212 learned_addr learned my addr 172.21.15.63:0/1342082212 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:45.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 0x55cce096c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f93f7e00700 1 -- 172.21.15.63:0/1342082212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 msgr2=0x55cce096d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:45.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.351+0000 7f93f7e00700 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce096d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.352+0000 7f93f7e00700 1 -- 172.21.15.63:0/1342082212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 msgr2=0x55cce096c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:45.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.352+0000 7f93f7e00700 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 0x55cce096c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.352+0000 7f93f7e00700 1 -- 172.21.15.63:0/1342082212 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cce07cc950 con 0x55cce0963240 2024-05-27T08:13:45.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.352+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 0x55cce096c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:45.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.352+0000 7f93f7e00700 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce0963240 0x55cce096cef0 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f93e40a8f90 tx=0x7f93e40dfc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd35a18ea330b634 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:45.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.352+0000 7f93dea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:45.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.352+0000 7f93dea00700 1 -- 172.21.15.63:0/1342082212 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4020070 con 0x55cce0963240 2024-05-27T08:13:45.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.352+0000 7f93dea00700 1 -- 172.21.15.63:0/1342082212 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f93e40e2120 con 0x55cce0963240 2024-05-27T08:13:45.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.353+0000 7f940e112b80 1 -- 172.21.15.63:0/1342082212 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f0003c80 con 0x55cce0963240 2024-05-27T08:13:45.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.353+0000 7f93dea00700 1 -- 172.21.15.63:0/1342082212 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f93e400fed0 con 0x55cce0963240 2024-05-27T08:13:45.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.353+0000 7f940e112b80 10 client.23100 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:45.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.353+0000 7f940e112b80 1 -- 172.21.15.63:0/1342082212 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cce084d0e0 con 0x55cce0963240 2024-05-27T08:13:45.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.353+0000 7f93dea00700 1 -- 172.21.15.63:0/1342082212 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f0003c80 con 0x55cce0963240 2024-05-27T08:13:45.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.353+0000 7f93dea00700 1 -- 172.21.15.63:0/1342082212 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f93e40ad170 con 0x55cce0963240 2024-05-27T08:13:45.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.354+0000 7f940e112b80 10 client.23100 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:45.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.354+0000 7f940e112b80 10 client.23100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:45.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.354+0000 7f940e112b80 20 client.23100 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:45.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.354+0000 7f940e112b80 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cce097cba0 0x55cce097f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.354+0000 7f940e112b80 4 client.23100 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:13:45.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.354+0000 7f940e112b80 1 -- 172.21.15.63:0/1342082212 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cce088ee10 con 0x55cce097cba0 2024-05-27T08:13:45.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.354+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cce097cba0 0x55cce097f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.356+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cce097cba0 0x55cce097f070 crc :-1 s=READY pgs=712 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4461b6cb44ca046b server_cookie=cfaea14265030a3e in_seq=0 out_seq=0 2024-05-27T08:13:45.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.356+0000 7f93dea00700 10 client.23100 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:45.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.357+0000 7f93dea00700 1 -- 172.21.15.63:0/1342082212 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cce088ee10 con 0x55cce097cba0 2024-05-27T08:13:45.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.357+0000 7f93dea00700 10 client.23100 handle_command_reply: tid=0 2024-05-27T08:13:45.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.357+0000 7f940e112b80 1 client.23100 shutdown 2024-05-27T08:13:45.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 1 -- 172.21.15.63:0/1342082212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce0963240 msgr2=0x55cce096cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:45.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce0963240 0x55cce096cef0 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f93e40a8f90 tx=0x7f93e40dfc60 comp rx=0 tx=0).stop 2024-05-27T08:13:45.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 1 -- 172.21.15.63:0/1342082212 shutdown_connections 2024-05-27T08:13:45.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cce097cba0 0x55cce097f070 crc :-1 s=READY pgs=712 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce096d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 0x55cce096c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 1 --2- 172.21.15.63:0/1342082212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce0963240 0x55cce096cef0 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 1 -- 172.21.15.63:0/1342082212 >> 172.21.15.63:0/1342082212 conn(0x55cce0840040 msgr2=0x55cce0959600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:45.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f93dea00700 0 client.23100 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:45.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 1 -- 172.21.15.63:0/1342082212 shutdown_connections 2024-05-27T08:13:45.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 1 -- 172.21.15.63:0/1342082212 wait complete. 2024-05-27T08:13:45.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.358+0000 7f940e112b80 20 client.23100 trim_cache size 0 max 16384 2024-05-27T08:13:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.381+0000 7f940e112b80 1 Processor -- start 2024-05-27T08:13:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.381+0000 7f940e112b80 1 -- start start 2024-05-27T08:13:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.382+0000 7f940e112b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0978d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.382+0000 7f940e112b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce096cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.382+0000 7f940e112b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce096efc0 0x55cce096d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.382+0000 7f940e112b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b0c30 con 0x55cce07dfec0 2024-05-27T08:13:45.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.382+0000 7f940e112b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ae4c0 con 0x55cce096efc0 2024-05-27T08:13:45.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.382+0000 7f940e112b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ac330 con 0x55cce0969160 2024-05-27T08:13:45.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.382+0000 7f93fcc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0978d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.383+0000 7f93fcc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0978d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37838/0 (socket says 172.21.15.63:37838) 2024-05-27T08:13:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.383+0000 7f93fcc00700 1 -- 172.21.15.63:0/1882061245 learned_addr learned my addr 172.21.15.63:0/1882061245 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.383+0000 7f93fd600700 1 --2- 172.21.15.63:0/1882061245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce096efc0 0x55cce096d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.383+0000 7f93f7e00700 1 --2- 172.21.15.63:0/1882061245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce096cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.383+0000 7f93fcc00700 1 -- 172.21.15.63:0/1882061245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 msgr2=0x55cce096cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.383+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1882061245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce096cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.383+0000 7f93fcc00700 1 -- 172.21.15.63:0/1882061245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce096efc0 msgr2=0x55cce096d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.383+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1882061245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce096efc0 0x55cce096d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.383+0000 7f93fcc00700 1 -- 172.21.15.63:0/1882061245 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cce096af20 con 0x55cce07dfec0 2024-05-27T08:13:45.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.384+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1882061245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0978d30 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f93f819bc10 tx=0x7f93ec00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c86b937aff50f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:45.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.384+0000 7f93e9600700 1 -- 172.21.15.63:0/1882061245 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec003ac0 con 0x55cce07dfec0 2024-05-27T08:13:45.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.384+0000 7f93e9600700 1 -- 172.21.15.63:0/1882061245 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93ec004cc0 con 0x55cce07dfec0 2024-05-27T08:13:45.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.384+0000 7f93e9600700 1 -- 172.21.15.63:0/1882061245 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec005540 con 0x55cce07dfec0 2024-05-27T08:13:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.385+0000 7f940e112b80 1 -- 172.21.15.63:0/1882061245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 msgr2=0x55cce0978d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.385+0000 7f940e112b80 1 --2- 172.21.15.63:0/1882061245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0978d30 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f93f819bc10 tx=0x7f93ec00b410 comp rx=0 tx=0).stop 2024-05-27T08:13:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.385+0000 7f940e112b80 1 -- 172.21.15.63:0/1882061245 shutdown_connections 2024-05-27T08:13:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.385+0000 7f940e112b80 1 --2- 172.21.15.63:0/1882061245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce096efc0 0x55cce096d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.385+0000 7f940e112b80 1 --2- 172.21.15.63:0/1882061245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce07dfec0 0x55cce0978d30 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.385+0000 7f940e112b80 1 --2- 172.21.15.63:0/1882061245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce096cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.385+0000 7f940e112b80 1 -- 172.21.15.63:0/1882061245 >> 172.21.15.63:0/1882061245 conn(0x55cce0840040 msgr2=0x55cce096c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.385+0000 7f940e112b80 1 -- 172.21.15.63:0/1882061245 shutdown_connections 2024-05-27T08:13:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.385+0000 7f940e112b80 1 -- 172.21.15.63:0/1882061245 wait complete. 2024-05-27T08:13:45.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.386+0000 7f940e112b80 1 -- start start 2024-05-27T08:13:45.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.386+0000 7f940e112b80 1 Processor -- start 2024-05-27T08:13:45.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.386+0000 7f940e112b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 0x55cce0974f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f940e112b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce0975460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f940e112b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce096efc0 0x55cce0960b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f940e112b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ac980 con 0x55cce096efc0 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f940e112b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81addf0 con 0x55cce07dfec0 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f940e112b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81ad050 con 0x55cce0969160 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f940e112b80 10 client.0 _ll_register_callbacks cb 0x55cce0826cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f93fd600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce096efc0 0x55cce0960b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f93fd600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce096efc0 0x55cce0960b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37840/0 (socket says 172.21.15.63:37840) 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f93fd600700 1 -- 172.21.15.63:0/1805037011 learned_addr learned my addr 172.21.15.63:0/1805037011 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 0x55cce0974f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.387+0000 7f93f7e00700 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce0975460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.388+0000 7f93fd600700 1 -- 172.21.15.63:0/1805037011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 msgr2=0x55cce0975460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.388+0000 7f93fd600700 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce0975460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.388+0000 7f93fd600700 1 -- 172.21.15.63:0/1805037011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 msgr2=0x55cce0974f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:45.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.388+0000 7f93fd600700 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 0x55cce0974f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.388+0000 7f93fd600700 1 -- 172.21.15.63:0/1805037011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cce0961040 con 0x55cce096efc0 2024-05-27T08:13:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.388+0000 7f93fd600700 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce096efc0 0x55cce0960b00 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009c50 tx=0x7f93f0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c2363129e070dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.388+0000 7f93dea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.389+0000 7f93dea00700 1 -- 172.21.15.63:0/1805037011 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0036070 con 0x55cce096efc0 2024-05-27T08:13:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.389+0000 7f940e112b80 1 -- 172.21.15.63:0/1805037011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93ec004cc0 con 0x55cce096efc0 2024-05-27T08:13:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.389+0000 7f93dea00700 1 -- 172.21.15.63:0/1805037011 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f93f0003e00 con 0x55cce096efc0 2024-05-27T08:13:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.389+0000 7f93dea00700 1 -- 172.21.15.63:0/1805037011 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f93f0042050 con 0x55cce096efc0 2024-05-27T08:13:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.389+0000 7f940e112b80 10 client.23106 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:45.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.389+0000 7f940e112b80 1 -- 172.21.15.63:0/1805037011 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cce09ec920 con 0x55cce096efc0 2024-05-27T08:13:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.389+0000 7f93dea00700 1 -- 172.21.15.63:0/1805037011 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93ec004cc0 con 0x55cce096efc0 2024-05-27T08:13:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.390+0000 7f93dea00700 1 -- 172.21.15.63:0/1805037011 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f93f0011070 con 0x55cce096efc0 2024-05-27T08:13:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.390+0000 7f940e112b80 10 client.23106 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.390+0000 7f940e112b80 10 client.23106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.390+0000 7f940e112b80 20 client.23106 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.390+0000 7f940e112b80 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cce09ecb50 0x55cce09eef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.390+0000 7f940e112b80 4 client.23106 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:13:45.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.390+0000 7f940e112b80 1 -- 172.21.15.63:0/1805037011 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cce08885f0 con 0x55cce09ecb50 2024-05-27T08:13:45.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.390+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cce09ecb50 0x55cce09eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:45.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.392+0000 7f93dea00700 10 client.23106 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:45.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.392+0000 7f93fcc00700 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cce09ecb50 0x55cce09eef70 crc :-1 s=READY pgs=713 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e41a7f2ffec20ffd server_cookie=974cf3119ca29e4c in_seq=0 out_seq=0 2024-05-27T08:13:45.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.394+0000 7f93dea00700 1 -- 172.21.15.63:0/1805037011 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cce08885f0 con 0x55cce09ecb50 2024-05-27T08:13:45.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.394+0000 7f93dea00700 10 client.23106 handle_command_reply: tid=0 2024-05-27T08:13:45.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.394+0000 7f940e112b80 1 client.23106 shutdown 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f940e112b80 1 -- 172.21.15.63:0/1805037011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce096efc0 msgr2=0x55cce0960b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f940e112b80 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce096efc0 0x55cce0960b00 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009c50 tx=0x7f93f0021000 comp rx=0 tx=0).stop 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f940e112b80 1 -- 172.21.15.63:0/1805037011 shutdown_connections 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f940e112b80 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cce09ecb50 0x55cce09eef70 crc :-1 s=READY pgs=713 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f940e112b80 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cce096efc0 0x55cce0960b00 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f940e112b80 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cce07dfec0 0x55cce0974f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f93dea00700 0 client.23106 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f940e112b80 1 --2- 172.21.15.63:0/1805037011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cce0969160 0x55cce0975460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f940e112b80 1 -- 172.21.15.63:0/1805037011 >> 172.21.15.63:0/1805037011 conn(0x55cce0840040 msgr2=0x55cce096c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f940e112b80 1 -- 172.21.15.63:0/1805037011 shutdown_connections 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.395+0000 7f940e112b80 1 -- 172.21.15.63:0/1805037011 wait complete. 2024-05-27T08:13:45.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.396+0000 7f940e112b80 20 client.23106 trim_cache size 0 max 16384 2024-05-27T08:13:45.398 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:13:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:13:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "346dfd26-69a6-4d57-90a3-5945025980f4", 2024-05-27T08:13:45.398 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:13:45.398 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:13:45.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.396+0000 7f93ff600700 1 -- 172.21.15.63:0/1003411377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93e00b2a70 msgr2=0x7f93e00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:45.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.396+0000 7f93ff600700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93e00b2a70 0x7f93e00b4f80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f93f002b000 tx=0x7f93f00077e0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.396+0000 7f93ff600700 1 -- 172.21.15.63:0/1003411377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93f8107de0 msgr2=0x7f93f81a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:45.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.396+0000 7f93ff600700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93f8107de0 0x7f93f81a0df0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f93e400d980 tx=0x7f93e400dc90 comp rx=0 tx=0).stop 2024-05-27T08:13:45.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.397+0000 7f93ff600700 1 -- 172.21.15.63:0/1003411377 shutdown_connections 2024-05-27T08:13:45.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.397+0000 7f93ff600700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f93e00b2a70 0x7f93e00b4f80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.397+0000 7f93ff600700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93f8108730 0x7f93f819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.397+0000 7f93ff600700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f81034b0 0x7f93f81a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.397+0000 7f93ff600700 1 --2- 172.21.15.63:0/1003411377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f93f8107de0 0x7f93f81a0df0 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:45.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.397+0000 7f93ff600700 1 -- 172.21.15.63:0/1003411377 >> 172.21.15.63:0/1003411377 conn(0x7f93f80fedb0 msgr2=0x7f93f81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:45.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.397+0000 7f93ff600700 1 -- 172.21.15.63:0/1003411377 shutdown_connections 2024-05-27T08:13:45.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:45.397+0000 7f93ff600700 1 -- 172.21.15.63:0/1003411377 wait complete. 2024-05-27T08:13:46.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:45 smithi086 ceph-mon[34687]: pgmap v1275: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T08:13:46.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:45 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:13:46.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:45 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:46.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:45 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:13:46.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:45 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:46.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:45 smithi149 ceph-mon[35408]: pgmap v1275: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T08:13:46.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:45 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:13:46.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:45 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:46.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:45 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:13:46.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:45 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:46.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:45 smithi063 ceph-mon[31613]: pgmap v1275: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-05-27T08:13:46.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:45 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:13:46.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:45 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:46.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:45 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:13:46.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:45 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:46.307 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:13:46.816 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:13:47.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:46 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:13:47.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:13:47.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:13:47.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:13:47.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:13:47.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:13:47.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:46 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:13:47.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:46 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:13:47.851 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:47 smithi063 ceph-mon[31613]: pgmap v1276: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 48 op/s 2024-05-27T08:13:48.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:47 smithi086 ceph-mon[34687]: pgmap v1276: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 48 op/s 2024-05-27T08:13:48.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:47 smithi149 ceph-mon[35408]: pgmap v1276: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 48 op/s 2024-05-27T08:13:48.960 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:48 smithi063 ceph-mon[31613]: pgmap v1277: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 41 op/s 2024-05-27T08:13:49.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:48 smithi086 ceph-mon[34687]: pgmap v1277: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 41 op/s 2024-05-27T08:13:49.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:48 smithi149 ceph-mon[35408]: pgmap v1277: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 41 op/s 2024-05-27T08:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.216+0000 7f512a000640 1 -- 172.21.15.63:0/2805967469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512410b610 msgr2=0x7f512410da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.216+0000 7f512a000640 1 --2- 172.21.15.63:0/2805967469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512410b610 0x7f512410da30 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f5110009900 tx=0x7f511002a410 comp rx=0 tx=0).stop 2024-05-27T08:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.217+0000 7f512a000640 1 -- 172.21.15.63:0/2805967469 shutdown_connections 2024-05-27T08:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.217+0000 7f512a000640 1 --2- 172.21.15.63:0/2805967469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512410e000 0x7f51241104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.217+0000 7f512a000640 1 --2- 172.21.15.63:0/2805967469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512410b610 0x7f512410da30 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.217+0000 7f512a000640 1 --2- 172.21.15.63:0/2805967469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5124104bb0 0x7f5124104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.217+0000 7f512a000640 1 -- 172.21.15.63:0/2805967469 >> 172.21.15.63:0/2805967469 conn(0x7f51240fe210 msgr2=0x7f5124100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.217+0000 7f512a000640 1 -- 172.21.15.63:0/2805967469 shutdown_connections 2024-05-27T08:13:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.217+0000 7f512a000640 1 -- 172.21.15.63:0/2805967469 wait complete. 2024-05-27T08:13:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.218+0000 7f512a000640 1 Processor -- start 2024-05-27T08:13:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.218+0000 7f512a000640 1 -- start start 2024-05-27T08:13:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f512a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5124104bb0 0x7f5124102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f512a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512410b610 0x7f51241029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f512a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512410e000 0x7f5124102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f512a000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5124113b60 con 0x7f512410e000 2024-05-27T08:13:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f512a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51241139e0 con 0x7f512410b610 2024-05-27T08:13:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f512a000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5124113ce0 con 0x7f5124104bb0 2024-05-27T08:13:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f5129600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512410e000 0x7f5124102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f5129600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512410e000 0x7f5124102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57372/0 (socket says 172.21.15.63:57372) 2024-05-27T08:13:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f5129600640 1 -- 172.21.15.63:0/4137070750 learned_addr learned my addr 172.21.15.63:0/4137070750 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f5123e00640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512410b610 0x7f51241029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.219+0000 7f5128c00640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5124104bb0 0x7f5124102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.220+0000 7f5129600640 1 -- 172.21.15.63:0/4137070750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5124104bb0 msgr2=0x7f5124102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.220+0000 7f5129600640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5124104bb0 0x7f5124102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.220+0000 7f5129600640 1 -- 172.21.15.63:0/4137070750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512410b610 msgr2=0x7f51241029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:49.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.220+0000 7f5129600640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512410b610 0x7f51241029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.220+0000 7f5129600640 1 -- 172.21.15.63:0/4137070750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51241a8960 con 0x7f512410e000 2024-05-27T08:13:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.220+0000 7f5129600640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512410e000 0x7f5124102ef0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f511c00eaa0 tx=0x7f511c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.220+0000 7f5121600640 1 -- 172.21.15.63:0/4137070750 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c00cdb0 con 0x7f512410e000 2024-05-27T08:13:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.220+0000 7f5121600640 1 -- 172.21.15.63:0/4137070750 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f511c005020 con 0x7f512410e000 2024-05-27T08:13:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.221+0000 7f512a000640 1 -- 172.21.15.63:0/4137070750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51241a8c50 con 0x7f512410e000 2024-05-27T08:13:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.221+0000 7f5121600640 1 -- 172.21.15.63:0/4137070750 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0116d0 con 0x7f512410e000 2024-05-27T08:13:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.221+0000 7f512a000640 1 -- 172.21.15.63:0/4137070750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51241a9170 con 0x7f512410e000 2024-05-27T08:13:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.224+0000 7f5132611740 1 Processor -- start 2024-05-27T08:13:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.226+0000 7f5121600640 1 -- 172.21.15.63:0/4137070750 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f511c0051d0 con 0x7f512410e000 2024-05-27T08:13:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.227+0000 7f5132611740 1 -- start start 2024-05-27T08:13:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.227+0000 7f5132611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52f01b0 0x5584f5692a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.227+0000 7f5132611740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52c72f0 0x5584f5692fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.227+0000 7f5132611740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52cf4d0 0x5584f56934e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.227+0000 7f5132611740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5584f5603b90 con 0x5584f52cf4d0 2024-05-27T08:13:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.228+0000 7f5132611740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5584f55f3730 con 0x5584f52c72f0 2024-05-27T08:13:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.228+0000 7f5132611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5584f55f7770 con 0x5584f52f01b0 2024-05-27T08:13:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.228+0000 7f5123e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52f01b0 0x5584f5692a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.228+0000 7f5128c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52c72f0 0x5584f5692fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.228+0000 7f5123e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52f01b0 0x5584f5692a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:55988/0 (socket says 172.21.15.63:55988) 2024-05-27T08:13:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.228+0000 7f5128c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52c72f0 0x5584f5692fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56172/0 (socket says 172.21.15.63:56172) 2024-05-27T08:13:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.228+0000 7f5123e00640 1 -- 172.21.15.63:0/2377307254 learned_addr learned my addr 172.21.15.63:0/2377307254 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.228+0000 7f5123e00640 1 --2- 172.21.15.63:0/2377307254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52cf4d0 0x5584f56934e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.229+0000 7f5128c00640 1 -- 172.21.15.63:0/2377307254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52f01b0 msgr2=0x5584f5692a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:49.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.229+0000 7f5128c00640 1 --2- 172.21.15.63:0/2377307254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52f01b0 0x5584f5692a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.229+0000 7f5128c00640 1 -- 172.21.15.63:0/2377307254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52cf4d0 msgr2=0x5584f56934e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.229+0000 7f5128c00640 1 --2- 172.21.15.63:0/2377307254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52cf4d0 0x5584f56934e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.229+0000 7f5128c00640 1 -- 172.21.15.63:0/2377307254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f558c8b0 con 0x5584f52c72f0 2024-05-27T08:13:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.229+0000 7f5121600640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f510c0b29d0 0x7f510c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.230+0000 7f5121600640 1 -- 172.21.15.63:0/4137070750 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f511c0e2bf0 con 0x7f512410e000 2024-05-27T08:13:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.230+0000 7f5123e00640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f510c0b29d0 0x7f510c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.230+0000 7f5128c00640 1 --2- 172.21.15.63:0/2377307254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52c72f0 0x5584f5692fa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f511800bbe0 tx=0x7f511800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f796ebbd4c45ed61 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.230+0000 7f5115c00640 1 -- 172.21.15.63:0/2377307254 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118015070 con 0x5584f52c72f0 2024-05-27T08:13:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.230+0000 7f5115c00640 1 -- 172.21.15.63:0/2377307254 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51180054b0 con 0x5584f52c72f0 2024-05-27T08:13:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.230+0000 7f5115c00640 1 -- 172.21.15.63:0/2377307254 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118004e40 con 0x5584f52c72f0 2024-05-27T08:13:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.231+0000 7f5132611740 1 -- 172.21.15.63:0/2377307254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52c72f0 msgr2=0x5584f5692fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.231+0000 7f5132611740 1 --2- 172.21.15.63:0/2377307254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52c72f0 0x5584f5692fa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f511800bbe0 tx=0x7f511800f730 comp rx=0 tx=0).stop 2024-05-27T08:13:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.231+0000 7f5123e00640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f510c0b29d0 0x7f510c0b4ee0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x5584f568fbd0 tx=0x7f5110000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.232+0000 7f5132611740 1 -- 172.21.15.63:0/2377307254 shutdown_connections 2024-05-27T08:13:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.232+0000 7f5132611740 1 --2- 172.21.15.63:0/2377307254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52cf4d0 0x5584f56934e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.232+0000 7f5132611740 1 --2- 172.21.15.63:0/2377307254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52c72f0 0x5584f5692fa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.232+0000 7f5132611740 1 --2- 172.21.15.63:0/2377307254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52f01b0 0x5584f5692a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.232+0000 7f5132611740 1 -- 172.21.15.63:0/2377307254 >> 172.21.15.63:0/2377307254 conn(0x5584f52e4300 msgr2=0x5584f568f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:49.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.232+0000 7f5132611740 1 -- 172.21.15.63:0/2377307254 shutdown_connections 2024-05-27T08:13:49.234 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.232+0000 7f5132611740 1 -- 172.21.15.63:0/2377307254 wait complete. 2024-05-27T08:13:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.232+0000 7f5132611740 1 -- start start 2024-05-27T08:13:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.233+0000 7f5132611740 1 Processor -- start 2024-05-27T08:13:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.233+0000 7f5132611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52c72f0 0x5584f569cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.233+0000 7f5132611740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52cf4d0 0x5584f569d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.233+0000 7f5132611740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52f01b0 0x5584f56a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.233+0000 7f5132611740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5584f52d9790 con 0x5584f52f01b0 2024-05-27T08:13:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.233+0000 7f5132611740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5584f559bbc0 con 0x5584f52cf4d0 2024-05-27T08:13:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.233+0000 7f5132611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5584f569fbe0 con 0x5584f52c72f0 2024-05-27T08:13:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.233+0000 7f5132611740 10 client.0 _ll_register_callbacks cb 0x5584f5579400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.234+0000 7f5129600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52f01b0 0x5584f56a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.234+0000 7f5123e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52cf4d0 0x5584f569d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.234+0000 7f5129600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52f01b0 0x5584f56a8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57388/0 (socket says 172.21.15.63:57388) 2024-05-27T08:13:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.234+0000 7f5129600640 1 -- 172.21.15.63:0/2140797796 learned_addr learned my addr 172.21.15.63:0/2140797796 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.234+0000 7f5128c00640 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52c72f0 0x5584f569cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f5129600640 1 -- 172.21.15.63:0/2140797796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52c72f0 msgr2=0x5584f569cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f5129600640 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52c72f0 0x5584f569cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f5129600640 1 -- 172.21.15.63:0/2140797796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52cf4d0 msgr2=0x5584f569d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f5129600640 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52cf4d0 0x5584f569d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f5129600640 1 -- 172.21.15.63:0/2140797796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f5308210 con 0x5584f52f01b0 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f5123e00640 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52cf4d0 0x5584f569d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f5129600640 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52f01b0 0x5584f56a8f10 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f511c08ffd0 tx=0x7f511c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8a150606ad6d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f510a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f510a000640 1 -- 172.21.15.63:0/2140797796 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c01f040 con 0x5584f52f01b0 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f510a000640 1 -- 172.21.15.63:0/2140797796 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f511c0a60a0 con 0x5584f52f01b0 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.235+0000 7f5132611740 1 -- 172.21.15.63:0/2140797796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51180054b0 con 0x5584f52f01b0 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.236+0000 7f510a000640 1 -- 172.21.15.63:0/2140797796 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f511c0ade60 con 0x5584f52f01b0 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.236+0000 7f5132611740 10 client.23118 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.236+0000 7f5132611740 1 -- 172.21.15.63:0/2140797796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5584f53d4620 con 0x5584f52f01b0 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.236+0000 7f510a000640 1 -- 172.21.15.63:0/2140797796 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51180054b0 con 0x5584f52f01b0 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.236+0000 7f510a000640 1 -- 172.21.15.63:0/2140797796 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f511c00f040 con 0x5584f52f01b0 2024-05-27T08:13:49.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.237+0000 7f5132611740 10 client.23118 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.237+0000 7f5132611740 10 client.23118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.237+0000 7f5132611740 20 client.23118 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.237+0000 7f5132611740 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5584f52aed80 0x5584f56aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.237+0000 7f5132611740 4 client.23118 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:13:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.237+0000 7f5132611740 1 -- 172.21.15.63:0/2140797796 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584f5605ae0 con 0x5584f52aed80 2024-05-27T08:13:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.237+0000 7f5128c00640 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5584f52aed80 0x5584f56aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.238+0000 7f5128c00640 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5584f52aed80 0x5584f56aef20 crc :-1 s=READY pgs=714 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4374842b9f51c2dd server_cookie=4278192a2b990eeb in_seq=0 out_seq=0 2024-05-27T08:13:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.238+0000 7f510a000640 10 client.23118 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f510a000640 1 -- 172.21.15.63:0/2140797796 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5584f5605ae0 con 0x5584f52aed80 2024-05-27T08:13:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f510a000640 10 client.23118 handle_command_reply: tid=0 2024-05-27T08:13:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f5132611740 1 client.23118 shutdown 2024-05-27T08:13:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f5132611740 1 -- 172.21.15.63:0/2140797796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52f01b0 msgr2=0x5584f56a8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f5132611740 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52f01b0 0x5584f56a8f10 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f511c08ffd0 tx=0x7f511c005720 comp rx=0 tx=0).stop 2024-05-27T08:13:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f5132611740 1 -- 172.21.15.63:0/2140797796 shutdown_connections 2024-05-27T08:13:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f5132611740 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5584f52aed80 0x5584f56aef20 crc :-1 s=READY pgs=714 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f5132611740 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52f01b0 0x5584f56a8f10 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f5132611740 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52cf4d0 0x5584f569d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f5132611740 1 --2- 172.21.15.63:0/2140797796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52c72f0 0x5584f569cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f5132611740 1 -- 172.21.15.63:0/2140797796 >> 172.21.15.63:0/2140797796 conn(0x5584f52e4300 msgr2=0x5584f5692830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.240+0000 7f510a000640 0 client.23118 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.241+0000 7f5132611740 1 -- 172.21.15.63:0/2140797796 shutdown_connections 2024-05-27T08:13:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.241+0000 7f5132611740 1 -- 172.21.15.63:0/2140797796 wait complete. 2024-05-27T08:13:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.241+0000 7f5132611740 20 client.23118 trim_cache size 0 max 16384 2024-05-27T08:13:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.262+0000 7f5132611740 1 Processor -- start 2024-05-27T08:13:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.262+0000 7f5132611740 1 -- start start 2024-05-27T08:13:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.262+0000 7f5132611740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f569f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.262+0000 7f5132611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52cf4d0 0x5584f56a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.262+0000 7f5132611740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 0x5584f56a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.262+0000 7f5132611740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51241b5a50 con 0x5584f52c72f0 2024-05-27T08:13:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.262+0000 7f5132611740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51241b3320 con 0x5584f52f01b0 2024-05-27T08:13:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.262+0000 7f5132611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f51241b1190 con 0x5584f52cf4d0 2024-05-27T08:13:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5128c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f569f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5128c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f569f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57400/0 (socket says 172.21.15.63:57400) 2024-05-27T08:13:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5128c00640 1 -- 172.21.15.63:0/1766353638 learned_addr learned my addr 172.21.15.63:0/1766353638 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5123e00640 1 --2- 172.21.15.63:0/1766353638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52cf4d0 0x5584f56a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5129600640 1 --2- 172.21.15.63:0/1766353638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 0x5584f56a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5128c00640 1 -- 172.21.15.63:0/1766353638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52cf4d0 msgr2=0x5584f56a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5128c00640 1 --2- 172.21.15.63:0/1766353638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52cf4d0 0x5584f56a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5128c00640 1 -- 172.21.15.63:0/1766353638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 msgr2=0x5584f56a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5128c00640 1 --2- 172.21.15.63:0/1766353638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 0x5584f56a91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5128c00640 1 -- 172.21.15.63:0/1766353638 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f52aefb0 con 0x5584f52c72f0 2024-05-27T08:13:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.263+0000 7f5129600640 1 --2- 172.21.15.63:0/1766353638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 0x5584f56a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:13:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.264+0000 7f5128c00640 1 --2- 172.21.15.63:0/1766353638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f569f340 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f5118004ab0 tx=0x7f5118005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6c1c35b3b0ddfa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.264+0000 7f5115200640 1 -- 172.21.15.63:0/1766353638 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118015070 con 0x5584f52c72f0 2024-05-27T08:13:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.264+0000 7f5115200640 1 -- 172.21.15.63:0/1766353638 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f511800bcd0 con 0x5584f52c72f0 2024-05-27T08:13:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.264+0000 7f5115200640 1 -- 172.21.15.63:0/1766353638 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118011440 con 0x5584f52c72f0 2024-05-27T08:13:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.265+0000 7f5132611740 1 -- 172.21.15.63:0/1766353638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 msgr2=0x5584f569f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.265+0000 7f5132611740 1 --2- 172.21.15.63:0/1766353638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f569f340 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f5118004ab0 tx=0x7f5118005f30 comp rx=0 tx=0).stop 2024-05-27T08:13:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.265+0000 7f5132611740 1 -- 172.21.15.63:0/1766353638 shutdown_connections 2024-05-27T08:13:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.265+0000 7f5132611740 1 --2- 172.21.15.63:0/1766353638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 0x5584f56a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.265+0000 7f5132611740 1 --2- 172.21.15.63:0/1766353638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52cf4d0 0x5584f56a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.265+0000 7f5132611740 1 --2- 172.21.15.63:0/1766353638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f569f340 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.265+0000 7f5132611740 1 -- 172.21.15.63:0/1766353638 >> 172.21.15.63:0/1766353638 conn(0x5584f52e4300 msgr2=0x5584f569ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.265+0000 7f5132611740 1 -- 172.21.15.63:0/1766353638 shutdown_connections 2024-05-27T08:13:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.265+0000 7f5132611740 1 -- 172.21.15.63:0/1766353638 wait complete. 2024-05-27T08:13:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.266+0000 7f5132611740 1 -- start start 2024-05-27T08:13:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.266+0000 7f5132611740 1 Processor -- start 2024-05-27T08:13:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5132611740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f56a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5132611740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52cf4d0 0x5584f56a2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5132611740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 0x5584f57155b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5132611740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51241b17e0 con 0x5584f52c72f0 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5132611740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51241b2c50 con 0x5584f52f01b0 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5132611740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f51241b1eb0 con 0x5584f52cf4d0 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5132611740 10 client.0 _ll_register_callbacks cb 0x7f511c0a5970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5128c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f56a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5128c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f56a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57402/0 (socket says 172.21.15.63:57402) 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5128c00640 1 -- 172.21.15.63:0/944249733 learned_addr learned my addr 172.21.15.63:0/944249733 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5123e00640 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52cf4d0 0x5584f56a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.267+0000 7f5129600640 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 0x5584f57155b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.268+0000 7f5128c00640 1 -- 172.21.15.63:0/944249733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52cf4d0 msgr2=0x5584f56a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.268+0000 7f5128c00640 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52cf4d0 0x5584f56a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.268+0000 7f5128c00640 1 -- 172.21.15.63:0/944249733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 msgr2=0x5584f57155b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.268+0000 7f5128c00640 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 0x5584f57155b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.268+0000 7f5128c00640 1 -- 172.21.15.63:0/944249733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f56a2df0 con 0x5584f52c72f0 2024-05-27T08:13:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.268+0000 7f5128c00640 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f56a1fd0 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f511800fc30 tx=0x7f5118000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5769ef527a51fbd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.268+0000 7f510a000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.269+0000 7f510a000640 1 -- 172.21.15.63:0/944249733 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118015040 con 0x5584f52c72f0 2024-05-27T08:13:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.269+0000 7f510a000640 1 -- 172.21.15.63:0/944249733 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f5118025670 con 0x5584f52c72f0 2024-05-27T08:13:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.269+0000 7f5132611740 1 -- 172.21.15.63:0/944249733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f511800bcd0 con 0x5584f52c72f0 2024-05-27T08:13:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.269+0000 7f510a000640 1 -- 172.21.15.63:0/944249733 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5118025870 con 0x5584f52c72f0 2024-05-27T08:13:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.269+0000 7f5132611740 10 client.23124 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.269+0000 7f5132611740 1 -- 172.21.15.63:0/944249733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5584f5716b00 con 0x5584f52c72f0 2024-05-27T08:13:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.270+0000 7f510a000640 1 -- 172.21.15.63:0/944249733 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f511800bcd0 con 0x5584f52c72f0 2024-05-27T08:13:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.270+0000 7f510a000640 1 -- 172.21.15.63:0/944249733 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5118019410 con 0x5584f52c72f0 2024-05-27T08:13:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.270+0000 7f5132611740 10 client.23124 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.270+0000 7f5132611740 10 client.23124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.270+0000 7f5132611740 20 client.23124 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.270+0000 7f5132611740 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5584f52841c0 0x5584f5719030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.270+0000 7f5132611740 4 client.23124 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:13:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.270+0000 7f5132611740 1 -- 172.21.15.63:0/944249733 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584f55ff2c0 con 0x5584f52841c0 2024-05-27T08:13:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.271+0000 7f5123e00640 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5584f52841c0 0x5584f5719030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.272+0000 7f5123e00640 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5584f52841c0 0x5584f5719030 crc :-1 s=READY pgs=715 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ec7ec2bd3e38af server_cookie=b9645a9df163d707 in_seq=0 out_seq=0 2024-05-27T08:13:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.272+0000 7f510a000640 10 client.23124 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.306+0000 7f510a000640 1 -- 172.21.15.63:0/944249733 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5584f55ff2c0 con 0x5584f52841c0 2024-05-27T08:13:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.306+0000 7f510a000640 10 client.23124 handle_command_reply: tid=0 2024-05-27T08:13:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.306+0000 7f5132611740 1 client.23124 shutdown 2024-05-27T08:13:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 1 -- 172.21.15.63:0/944249733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 msgr2=0x5584f56a1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f56a1fd0 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f511800fc30 tx=0x7f5118000bc0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 1 -- 172.21.15.63:0/944249733 shutdown_connections 2024-05-27T08:13:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5584f52841c0 0x5584f5719030 crc :-1 s=READY pgs=715 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584f52f01b0 0x5584f57155b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584f52cf4d0 0x5584f56a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 1 --2- 172.21.15.63:0/944249733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584f52c72f0 0x5584f56a1fd0 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 1 -- 172.21.15.63:0/944249733 >> 172.21.15.63:0/944249733 conn(0x5584f52e4300 msgr2=0x5584f569cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f510a000640 0 client.23124 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 1 -- 172.21.15.63:0/944249733 shutdown_connections 2024-05-27T08:13:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 1 -- 172.21.15.63:0/944249733 wait complete. 2024-05-27T08:13:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.307+0000 7f5132611740 20 client.23124 trim_cache size 0 max 16384 2024-05-27T08:13:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:13:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:13:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:13:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.308+0000 7f512a000640 1 -- 172.21.15.63:0/4137070750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f510c0b29d0 msgr2=0x7f510c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.308+0000 7f512a000640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f510c0b29d0 0x7f510c0b4ee0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x5584f568fbd0 tx=0x7f5110000bc0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.308+0000 7f512a000640 1 -- 172.21.15.63:0/4137070750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512410e000 msgr2=0x7f5124102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.308+0000 7f512a000640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512410e000 0x7f5124102ef0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f511c00eaa0 tx=0x7f511c00ef60 comp rx=0 tx=0).stop 2024-05-27T08:13:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.308+0000 7f512a000640 1 -- 172.21.15.63:0/4137070750 shutdown_connections 2024-05-27T08:13:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.308+0000 7f512a000640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f510c0b29d0 0x7f510c0b4ee0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.308+0000 7f512a000640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f512410e000 0x7f5124102ef0 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.308+0000 7f512a000640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512410b610 0x7f51241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.308+0000 7f512a000640 1 --2- 172.21.15.63:0/4137070750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5124104bb0 0x7f5124102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.308+0000 7f512a000640 1 -- 172.21.15.63:0/4137070750 >> 172.21.15.63:0/4137070750 conn(0x7f51240fe210 msgr2=0x7f512410e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.309+0000 7f512a000640 1 -- 172.21.15.63:0/4137070750 shutdown_connections 2024-05-27T08:13:49.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:13:49.309+0000 7f512a000640 1 -- 172.21.15.63:0/4137070750 wait complete. 2024-05-27T08:13:50.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.269+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3823460219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b84103a70 msgr2=0x7f8b84110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:50.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.269+0000 7f8b8ae00700 1 --2- 172.21.15.63:0/3823460219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b84103a70 0x7f8b84110700 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009990 tx=0x7f8b70009ca0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.269+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3823460219 shutdown_connections 2024-05-27T08:13:50.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.269+0000 7f8b8ae00700 1 --2- 172.21.15.63:0/3823460219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b84110c40 0x7f8b84113060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.269+0000 7f8b8ae00700 1 --2- 172.21.15.63:0/3823460219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b84103090 0x7f8b841034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.269+0000 7f8b8ae00700 1 --2- 172.21.15.63:0/3823460219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b84103a70 0x7f8b84110700 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.269+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3823460219 >> 172.21.15.63:0/3823460219 conn(0x7f8b840fef20 msgr2=0x7f8b84101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:50.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.269+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3823460219 shutdown_connections 2024-05-27T08:13:50.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.270+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3823460219 wait complete. 2024-05-27T08:13:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.271+0000 7f8b8ae00700 1 Processor -- start 2024-05-27T08:13:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.271+0000 7f8b8ae00700 1 -- start start 2024-05-27T08:13:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.272+0000 7f8b8ae00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b84103090 0x7f8b8419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.272+0000 7f8b8ae00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b84103a70 0x7f8b8419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.272+0000 7f8b8ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b84110c40 0x7f8b841a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.272+0000 7f8b8ae00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b84116540 con 0x7f8b84103090 2024-05-27T08:13:50.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.272+0000 7f8b8ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841163c0 con 0x7f8b84110c40 2024-05-27T08:13:50.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.272+0000 7f8b8ae00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841166c0 con 0x7f8b84103a70 2024-05-27T08:13:50.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.272+0000 7f8b83e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b84103090 0x7f8b8419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.272+0000 7f8b83e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b84103090 0x7f8b8419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57422/0 (socket says 172.21.15.63:57422) 2024-05-27T08:13:50.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.272+0000 7f8b83e00700 1 -- 172.21.15.63:0/3409564410 learned_addr learned my addr 172.21.15.63:0/3409564410 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:50.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.273+0000 7f8b88e00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b84110c40 0x7f8b841a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.273+0000 7f8b83e00700 1 -- 172.21.15.63:0/3409564410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b84103a70 msgr2=0x7f8b8419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.273+0000 7f8b83400700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b84103a70 0x7f8b8419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.273+0000 7f8b83e00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b84103a70 0x7f8b8419d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.273+0000 7f8b83e00700 1 -- 172.21.15.63:0/3409564410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b84110c40 msgr2=0x7f8b841a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.273+0000 7f8b83e00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b84110c40 0x7f8b841a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.273+0000 7f8b83e00700 1 -- 172.21.15.63:0/3409564410 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b841a1dc0 con 0x7f8b84103090 2024-05-27T08:13:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.273+0000 7f8b88e00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b84110c40 0x7f8b841a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:13:50.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.273+0000 7f8b83400700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b84103a70 0x7f8b8419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:13:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.274+0000 7f8b83e00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b84103090 0x7f8b8419cef0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400d950 tx=0x7f8b7400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.274+0000 7f8b80c00700 1 -- 172.21.15.63:0/3409564410 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74016070 con 0x7f8b84103090 2024-05-27T08:13:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.274+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3409564410 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b841a20b0 con 0x7f8b84103090 2024-05-27T08:13:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.274+0000 7f8b80c00700 1 -- 172.21.15.63:0/3409564410 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b74010460 con 0x7f8b84103090 2024-05-27T08:13:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.274+0000 7f8b80c00700 1 -- 172.21.15.63:0/3409564410 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7400f620 con 0x7f8b84103090 2024-05-27T08:13:50.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.274+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3409564410 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b841a99d0 con 0x7f8b84103090 2024-05-27T08:13:50.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.279+0000 7f8b80c00700 1 -- 172.21.15.63:0/3409564410 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8b74019070 con 0x7f8b84103090 2024-05-27T08:13:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.280+0000 7f8b99a3bb80 1 Processor -- start 2024-05-27T08:13:50.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.281+0000 7f8b99a3bb80 1 -- start start 2024-05-27T08:13:50.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.281+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759d957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.281+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759d8f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.281+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759d901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.281+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562759cbde40 con 0x562759c10e40 2024-05-27T08:13:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.281+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562759cad9e0 con 0x562759d8fdb0 2024-05-27T08:13:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.281+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562759cb1a20 con 0x562759d95d30 2024-05-27T08:13:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.281+0000 7f8b80c00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b680b29f0 0x7f8b680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.282+0000 7f8b88e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759d957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.282+0000 7f8b80c00700 1 -- 172.21.15.63:0/3409564410 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f8b740e2680 con 0x7f8b84103090 2024-05-27T08:13:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.282+0000 7f8b83e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759d8f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.282+0000 7f8b88e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759d957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57428/0 (socket says 172.21.15.63:57428) 2024-05-27T08:13:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.282+0000 7f8b88e00700 1 -- 172.21.15.63:0/2432945020 learned_addr learned my addr 172.21.15.63:0/2432945020 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.282+0000 7f8b83e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759d8f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:56016/0 (socket says 172.21.15.63:56016) 2024-05-27T08:13:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.282+0000 7f8b88e00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b680b29f0 0x7f8b680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.282+0000 7f8b83400700 1 --2- 172.21.15.63:0/2432945020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759d901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.283+0000 7f8b83e00700 1 -- 172.21.15.63:0/2432945020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 msgr2=0x562759d901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.283+0000 7f8b83e00700 1 --2- 172.21.15.63:0/2432945020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759d901c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.283+0000 7f8b83e00700 1 -- 172.21.15.63:0/2432945020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 msgr2=0x562759d957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.283+0000 7f8b83e00700 1 --2- 172.21.15.63:0/2432945020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759d957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.283+0000 7f8b83e00700 1 -- 172.21.15.63:0/2432945020 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562759c76420 con 0x562759d95d30 2024-05-27T08:13:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.283+0000 7f8b88e00700 1 --2- 172.21.15.63:0/2432945020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759d957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:13:50.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.283+0000 7f8b88e00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b680b29f0 0x7f8b680b4f00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009fd0 tx=0x7f8b7c006040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.284+0000 7f8b83e00700 1 --2- 172.21.15.63:0/2432945020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759d8f870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b740df9b0 tx=0x7f8b740a8d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fe5c5404c5c6f1c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.284+0000 7f8b79600700 1 -- 172.21.15.63:0/2432945020 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74020070 con 0x562759d95d30 2024-05-27T08:13:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.284+0000 7f8b79600700 1 -- 172.21.15.63:0/2432945020 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b740dfe30 con 0x562759d95d30 2024-05-27T08:13:50.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.284+0000 7f8b79600700 1 -- 172.21.15.63:0/2432945020 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b740e2460 con 0x562759d95d30 2024-05-27T08:13:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.286+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/2432945020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 msgr2=0x562759d8f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.286+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/2432945020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759d8f870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b740df9b0 tx=0x7f8b740a8d30 comp rx=0 tx=0).stop 2024-05-27T08:13:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.286+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/2432945020 shutdown_connections 2024-05-27T08:13:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.286+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/2432945020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759d901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.286+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/2432945020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759d957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.286+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/2432945020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759d8f870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.286+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/2432945020 >> 172.21.15.63:0/2432945020 conn(0x562759c70fc0 msgr2=0x562759d8a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.286+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/2432945020 shutdown_connections 2024-05-27T08:13:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.286+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/2432945020 wait complete. 2024-05-27T08:13:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.287+0000 7f8b99a3bb80 1 -- start start 2024-05-27T08:13:50.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.287+0000 7f8b99a3bb80 1 Processor -- start 2024-05-27T08:13:50.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.288+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759c10e40 0x562759d99540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.288+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d8fdb0 0x562759d99a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.288+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d95d30 0x562759d99fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.288+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562759bb3fa0 con 0x562759d8fdb0 2024-05-27T08:13:50.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.288+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562759c4e760 con 0x562759d95d30 2024-05-27T08:13:50.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.288+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562759d9c800 con 0x562759c10e40 2024-05-27T08:13:50.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.288+0000 7f8b99a3bb80 10 client.0 _ll_register_callbacks cb 0x562759c57c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:50.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.288+0000 7f8b83400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d8fdb0 0x562759d99a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.288+0000 7f8b83400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d8fdb0 0x562759d99a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57434/0 (socket says 172.21.15.63:57434) 2024-05-27T08:13:50.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.288+0000 7f8b83400700 1 -- 172.21.15.63:0/1036790252 learned_addr learned my addr 172.21.15.63:0/1036790252 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:50.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.289+0000 7f8b83e00700 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759c10e40 0x562759d99540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.289+0000 7f8b88e00700 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d95d30 0x562759d99fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.289+0000 7f8b83400700 1 -- 172.21.15.63:0/1036790252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759c10e40 msgr2=0x562759d99540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:50.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.289+0000 7f8b83400700 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759c10e40 0x562759d99540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.289+0000 7f8b83400700 1 -- 172.21.15.63:0/1036790252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d95d30 msgr2=0x562759d99fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:50.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.289+0000 7f8b83400700 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d95d30 0x562759d99fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.289+0000 7f8b83400700 1 -- 172.21.15.63:0/1036790252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562759bfd8d0 con 0x562759d8fdb0 2024-05-27T08:13:50.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.290+0000 7f8b83400700 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d8fdb0 0x562759d99a80 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7f8b84076cd0 tx=0x7f8b700057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79c3f1927776577 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:50.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.290+0000 7f8b6e000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:13:50.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.290+0000 7f8b6e000700 1 -- 172.21.15.63:0/1036790252 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70026060 con 0x562759d8fdb0 2024-05-27T08:13:50.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.290+0000 7f8b6e000700 1 -- 172.21.15.63:0/1036790252 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f8b700039c0 con 0x562759d8fdb0 2024-05-27T08:13:50.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.290+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1036790252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b740dfe30 con 0x562759d8fdb0 2024-05-27T08:13:50.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.291+0000 7f8b6e000700 1 -- 172.21.15.63:0/1036790252 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8b7002d050 con 0x562759d8fdb0 2024-05-27T08:13:50.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.291+0000 7f8b99a3bb80 10 client.23136 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:50.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.291+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1036790252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562759c7e060 con 0x562759d8fdb0 2024-05-27T08:13:50.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.291+0000 7f8b6e000700 1 -- 172.21.15.63:0/1036790252 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b740dfe30 con 0x562759d8fdb0 2024-05-27T08:13:50.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.291+0000 7f8b6e000700 1 -- 172.21.15.63:0/1036790252 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8b70020070 con 0x562759d8fdb0 2024-05-27T08:13:50.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.292+0000 7f8b99a3bb80 10 client.23136 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:50.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.292+0000 7f8b99a3bb80 10 client.23136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:50.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.292+0000 7f8b99a3bb80 20 client.23136 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:50.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.292+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562759da9790 0x562759dabbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.292+0000 7f8b99a3bb80 4 client.23136 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:13:50.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.292+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1036790252 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562759cbfd90 con 0x562759da9790 2024-05-27T08:13:50.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.292+0000 7f8b83e00700 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562759da9790 0x562759dabbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.293+0000 7f8b83e00700 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562759da9790 0x562759dabbb0 crc :-1 s=READY pgs=716 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=500b22c65604f995 server_cookie=30d515be27b9384f in_seq=0 out_seq=0 2024-05-27T08:13:50.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.294+0000 7f8b6e000700 10 client.23136 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:50.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b6e000700 1 -- 172.21.15.63:0/1036790252 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562759cbfd90 con 0x562759da9790 2024-05-27T08:13:50.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b6e000700 10 client.23136 handle_command_reply: tid=0 2024-05-27T08:13:50.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b99a3bb80 1 client.23136 shutdown 2024-05-27T08:13:50.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1036790252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d8fdb0 msgr2=0x562759d99a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:50.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d8fdb0 0x562759d99a80 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7f8b84076cd0 tx=0x7f8b700057d0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1036790252 shutdown_connections 2024-05-27T08:13:50.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562759da9790 0x562759dabbb0 crc :-1 s=READY pgs=716 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d95d30 0x562759d99fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759c10e40 0x562759d99540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1036790252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d8fdb0 0x562759d99a80 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1036790252 >> 172.21.15.63:0/1036790252 conn(0x562759c70fc0 msgr2=0x562759d8a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:50.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.296+0000 7f8b6e000700 0 client.23136 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:50.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.297+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1036790252 shutdown_connections 2024-05-27T08:13:50.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.297+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1036790252 wait complete. 2024-05-27T08:13:50.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.297+0000 7f8b99a3bb80 20 client.23136 trim_cache size 0 max 16384 2024-05-27T08:13:50.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.326+0000 7f8b99a3bb80 1 Processor -- start 2024-05-27T08:13:50.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.326+0000 7f8b99a3bb80 1 -- start start 2024-05-27T08:13:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.326+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759c10e40 0x562759d8f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.326+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759d98520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.326+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d95d30 0x562759d9b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.326+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841b4fe0 con 0x562759d95d30 2024-05-27T08:13:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.326+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841b2870 con 0x562759d8fdb0 2024-05-27T08:13:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.326+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841b06e0 con 0x562759c10e40 2024-05-27T08:13:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.327+0000 7f8b88e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d95d30 0x562759d9b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.327+0000 7f8b88e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d95d30 0x562759d9b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57442/0 (socket says 172.21.15.63:57442) 2024-05-27T08:13:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.327+0000 7f8b88e00700 1 -- 172.21.15.63:0/261762021 learned_addr learned my addr 172.21.15.63:0/261762021 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.327+0000 7f8b83400700 1 --2- 172.21.15.63:0/261762021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759d98520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.327+0000 7f8b83e00700 1 --2- 172.21.15.63:0/261762021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759c10e40 0x562759d8f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.327+0000 7f8b88e00700 1 -- 172.21.15.63:0/261762021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759c10e40 msgr2=0x562759d8f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.327+0000 7f8b88e00700 1 --2- 172.21.15.63:0/261762021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759c10e40 0x562759d8f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.327+0000 7f8b88e00700 1 -- 172.21.15.63:0/261762021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 msgr2=0x562759d98520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.327+0000 7f8b88e00700 1 --2- 172.21.15.63:0/261762021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759d98520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.327+0000 7f8b88e00700 1 -- 172.21.15.63:0/261762021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562759d97ad0 con 0x562759d95d30 2024-05-27T08:13:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.328+0000 7f8b88e00700 1 --2- 172.21.15.63:0/261762021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d95d30 0x562759d9b9e0 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c017940 tx=0x7f8b7c009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9748aed9ee5594 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.328+0000 7f8b78c00700 1 -- 172.21.15.63:0/261762021 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c029070 con 0x562759d95d30 2024-05-27T08:13:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.328+0000 7f8b78c00700 1 -- 172.21.15.63:0/261762021 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b7c019b90 con 0x562759d95d30 2024-05-27T08:13:50.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.328+0000 7f8b78c00700 1 -- 172.21.15.63:0/261762021 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c024b00 con 0x562759d95d30 2024-05-27T08:13:50.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.329+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/261762021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d95d30 msgr2=0x562759d9b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:50.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.329+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/261762021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d95d30 0x562759d9b9e0 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c017940 tx=0x7f8b7c009380 comp rx=0 tx=0).stop 2024-05-27T08:13:50.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.330+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/261762021 shutdown_connections 2024-05-27T08:13:50.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.330+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/261762021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759d95d30 0x562759d9b9e0 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.330+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/261762021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759c10e40 0x562759d8f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.330+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/261762021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759d98520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.330+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/261762021 >> 172.21.15.63:0/261762021 conn(0x562759c70fc0 msgr2=0x562759d99320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:50.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.330+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/261762021 shutdown_connections 2024-05-27T08:13:50.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.330+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/261762021 wait complete. 2024-05-27T08:13:50.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.331+0000 7f8b99a3bb80 1 -- start start 2024-05-27T08:13:50.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.331+0000 7f8b99a3bb80 1 Processor -- start 2024-05-27T08:13:50.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759da2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759db0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b99a3bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759db0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841b0d30 con 0x562759c10e40 2024-05-27T08:13:50.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841b21a0 con 0x562759d8fdb0 2024-05-27T08:13:50.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b99a3bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8b841b1400 con 0x562759d95d30 2024-05-27T08:13:50.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b99a3bb80 10 client.0 _ll_register_callbacks cb 0x562759c57c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:13:50.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b83e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759da2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b88e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759db0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b83400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759db0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b88e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759db0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:56024/0 (socket says 172.21.15.63:56024) 2024-05-27T08:13:50.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b83e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759da2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57444/0 (socket says 172.21.15.63:57444) 2024-05-27T08:13:50.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.332+0000 7f8b88e00700 1 -- 172.21.15.63:0/1250372735 learned_addr learned my addr 172.21.15.63:0/1250372735 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:13:50.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.333+0000 7f8b88e00700 1 -- 172.21.15.63:0/1250372735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 msgr2=0x562759db0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:50.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.333+0000 7f8b88e00700 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759db0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.333+0000 7f8b88e00700 1 -- 172.21.15.63:0/1250372735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 msgr2=0x562759da2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:13:50.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.333+0000 7f8b88e00700 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759da2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.333+0000 7f8b88e00700 1 -- 172.21.15.63:0/1250372735 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562759db1440 con 0x562759d95d30 2024-05-27T08:13:50.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.333+0000 7f8b83e00700 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759da2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:13:50.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.334+0000 7f8b88e00700 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759db0ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c017910 tx=0x7f8b7c007880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=436f5eea7bc4bb0e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:13:50.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.334+0000 7f8b6e000700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:13:50.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.334+0000 7f8b6e000700 1 -- 172.21.15.63:0/1250372735 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c029070 con 0x562759d95d30 2024-05-27T08:13:50.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.335+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1250372735 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b7c019b90 con 0x562759d95d30 2024-05-27T08:13:50.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.335+0000 7f8b6e000700 1 -- 172.21.15.63:0/1250372735 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7f8b7c019500 con 0x562759d95d30 2024-05-27T08:13:50.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.335+0000 7f8b6e000700 1 -- 172.21.15.63:0/1250372735 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8b7c0245a0 con 0x562759d95d30 2024-05-27T08:13:50.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.335+0000 7f8b99a3bb80 10 client.31831 fetch_fsmap learned FSMap version 35 2024-05-27T08:13:50.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.335+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1250372735 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562759e1c730 con 0x562759d95d30 2024-05-27T08:13:50.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.336+0000 7f8b6e000700 1 -- 172.21.15.63:0/1250372735 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b7c019b90 con 0x562759d95d30 2024-05-27T08:13:50.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.336+0000 7f8b6e000700 1 -- 172.21.15.63:0/1250372735 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8b7c018040 con 0x562759d95d30 2024-05-27T08:13:50.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.336+0000 7f8b99a3bb80 10 client.31831 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:13:50.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.336+0000 7f8b99a3bb80 10 client.31831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:13:50.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.336+0000 7f8b99a3bb80 20 client.31831 populate_metadata read hostname 'smithi063' 2024-05-27T08:13:50.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.337+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562759e1cde0 0x562759e1f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:13:50.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.337+0000 7f8b99a3bb80 4 client.31831 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:13:50.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.337+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1250372735 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562759cb9570 con 0x562759e1cde0 2024-05-27T08:13:50.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.337+0000 7f8b83e00700 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562759e1cde0 0x562759e1f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:13:50.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.338+0000 7f8b83e00700 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562759e1cde0 0x562759e1f200 crc :-1 s=READY pgs=717 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48ba28ee48f94d8 server_cookie=9faaa5c7fa2bb322 in_seq=0 out_seq=0 2024-05-27T08:13:50.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.338+0000 7f8b6e000700 10 client.31831 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b6e000700 1 -- 172.21.15.63:0/1250372735 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562759cb9570 con 0x562759e1cde0 2024-05-27T08:13:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b6e000700 10 client.31831 handle_command_reply: tid=0 2024-05-27T08:13:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b99a3bb80 1 client.31831 shutdown 2024-05-27T08:13:50.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1250372735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 msgr2=0x562759db0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:50.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759db0ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c017910 tx=0x7f8b7c007880 comp rx=0 tx=0).stop 2024-05-27T08:13:50.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1250372735 shutdown_connections 2024-05-27T08:13:50.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x562759e1cde0 0x562759e1f200 crc :-1 s=READY pgs=717 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562759d95d30 0x562759db0ed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562759c10e40 0x562759da2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b6e000700 0 client.31831 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:13:50.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b99a3bb80 1 --2- 172.21.15.63:0/1250372735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562759d8fdb0 0x562759db0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.339+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1250372735 >> 172.21.15.63:0/1250372735 conn(0x562759c70fc0 msgr2=0x562759d938d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:50.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.340+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1250372735 shutdown_connections 2024-05-27T08:13:50.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.340+0000 7f8b99a3bb80 1 -- 172.21.15.63:0/1250372735 wait complete. 2024-05-27T08:13:50.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.340+0000 7f8b99a3bb80 20 client.31831 trim_cache size 0 max 16384 2024-05-27T08:13:50.343 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:13:50.343 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:13:50.343 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:13:50.343 INFO:teuthology.orchestra.run.smithi063.stdout: "346dfd26-69a6-4d57-90a3-5945025980f4": { 2024-05-27T08:13:50.343 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:13:50.343 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "346dfd26-69a6-4d57-90a3-5945025980f4", 2024-05-27T08:13:50.343 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:13:50.343 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:13:50.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:13:50.344 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:13:50.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3409564410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b680b29f0 msgr2=0x7f8b680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:50.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b680b29f0 0x7f8b680b4f00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009fd0 tx=0x7f8b7c006040 comp rx=0 tx=0).stop 2024-05-27T08:13:50.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3409564410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b84103090 msgr2=0x7f8b8419cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:13:50.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b84103090 0x7f8b8419cef0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400d950 tx=0x7f8b7400dd10 comp rx=0 tx=0).stop 2024-05-27T08:13:50.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3409564410 shutdown_connections 2024-05-27T08:13:50.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8b680b29f0 0x7f8b680b4f00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b84110c40 0x7f8b841a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b84103090 0x7f8b8419cef0 unknown :-1 s=CLOSED pgs=2866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 --2- 172.21.15.63:0/3409564410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b84103a70 0x7f8b8419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:13:50.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3409564410 >> 172.21.15.63:0/3409564410 conn(0x7f8b840fef20 msgr2=0x7f8b84112850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:13:50.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3409564410 shutdown_connections 2024-05-27T08:13:50.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:13:50.341+0000 7f8b8ae00700 1 -- 172.21.15.63:0/3409564410 wait complete. 2024-05-27T08:13:51.298 INFO:tasks.cephfs.filesystem:scrub status for tag:346dfd26-69a6-4d57-90a3-5945025980f4 - {'path': '/', 'tag': '346dfd26-69a6-4d57-90a3-5945025980f4', 'options': 'recursive,force'} 2024-05-27T08:13:52.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:51 smithi086 ceph-mon[34687]: pgmap v1278: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 41 op/s 2024-05-27T08:13:52.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:51 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:13:52.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:51 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:13:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:51 smithi149 ceph-mon[35408]: pgmap v1278: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 41 op/s 2024-05-27T08:13:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:51 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:13:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:51 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:13:52.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:51 smithi063 ceph-mon[31613]: pgmap v1278: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 41 op/s 2024-05-27T08:13:52.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:51 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:13:52.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:51 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:13:53.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:52 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:13:53.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:52 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:13:53.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:52 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:13:54.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:53 smithi086 ceph-mon[34687]: pgmap v1279: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:13:54.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:53 smithi149 ceph-mon[35408]: pgmap v1279: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:13:54.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:53 smithi063 ceph-mon[31613]: pgmap v1279: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:13:56.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:55 smithi086 ceph-mon[34687]: pgmap v1280: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.4 MiB/s wr, 46 op/s 2024-05-27T08:13:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:55 smithi149 ceph-mon[35408]: pgmap v1280: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.4 MiB/s wr, 46 op/s 2024-05-27T08:13:56.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:55 smithi063 ceph-mon[31613]: pgmap v1280: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.4 MiB/s wr, 46 op/s 2024-05-27T08:13:58.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:57 smithi086 ceph-mon[34687]: pgmap v1281: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.0 MiB/s wr, 49 op/s 2024-05-27T08:13:58.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:57 smithi149 ceph-mon[35408]: pgmap v1281: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.0 MiB/s wr, 49 op/s 2024-05-27T08:13:58.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:57 smithi063 ceph-mon[31613]: pgmap v1281: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.0 MiB/s wr, 49 op/s 2024-05-27T08:13:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:13:58 smithi086 ceph-mon[34687]: pgmap v1282: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:13:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:13:58 smithi149 ceph-mon[35408]: pgmap v1282: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:13:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:13:58 smithi063 ceph-mon[31613]: pgmap v1282: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:14:02.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:01 smithi086 ceph-mon[34687]: pgmap v1283: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:14:02.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:01 smithi149 ceph-mon[35408]: pgmap v1283: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:14:02.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:01 smithi063 ceph-mon[31613]: pgmap v1283: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 11 op/s 2024-05-27T08:14:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.566+0000 7ff861e00640 1 -- 172.21.15.63:0/4238339545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff85c102840 msgr2=0x7ff85c102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.566+0000 7ff861e00640 1 --2- 172.21.15.63:0/4238339545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff85c102840 0x7ff85c102c50 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7ff850009900 tx=0x7ff85002a3f0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.566+0000 7ff861e00640 1 -- 172.21.15.63:0/4238339545 shutdown_connections 2024-05-27T08:14:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.566+0000 7ff861e00640 1 --2- 172.21.15.63:0/4238339545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff85c1103e0 0x7ff85c112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.566+0000 7ff861e00640 1 --2- 172.21.15.63:0/4238339545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff85c10da80 0x7ff85c10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.566+0000 7ff861e00640 1 --2- 172.21.15.63:0/4238339545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff85c102840 0x7ff85c102c50 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.566+0000 7ff861e00640 1 -- 172.21.15.63:0/4238339545 >> 172.21.15.63:0/4238339545 conn(0x7ff85c0fe210 msgr2=0x7ff85c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.567+0000 7ff861e00640 1 -- 172.21.15.63:0/4238339545 shutdown_connections 2024-05-27T08:14:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.567+0000 7ff861e00640 1 -- 172.21.15.63:0/4238339545 wait complete. 2024-05-27T08:14:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.568+0000 7ff861e00640 1 Processor -- start 2024-05-27T08:14:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.568+0000 7ff861e00640 1 -- start start 2024-05-27T08:14:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff861e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff85c102840 0x7ff85c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff861e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff85c10da80 0x7ff85c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff861e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff85c1103e0 0x7ff85c1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff861e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c115d30 con 0x7ff85c102840 2024-05-27T08:14:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff861e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c115bb0 con 0x7ff85c10da80 2024-05-27T08:14:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff861e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c115eb0 con 0x7ff85c1103e0 2024-05-27T08:14:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff860a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff85c102840 0x7ff85c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff860a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff85c102840 0x7ff85c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51312/0 (socket says 172.21.15.63:51312) 2024-05-27T08:14:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff860a00640 1 -- 172.21.15.63:0/916616798 learned_addr learned my addr 172.21.15.63:0/916616798 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff85be00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff85c10da80 0x7ff85c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.569+0000 7ff861400640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff85c1103e0 0x7ff85c1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.570+0000 7ff860a00640 1 -- 172.21.15.63:0/916616798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff85c1103e0 msgr2=0x7ff85c1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.570+0000 7ff860a00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff85c1103e0 0x7ff85c1a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.570+0000 7ff860a00640 1 -- 172.21.15.63:0/916616798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff85c10da80 msgr2=0x7ff85c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.570+0000 7ff860a00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff85c10da80 0x7ff85c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.570+0000 7ff860a00640 1 -- 172.21.15.63:0/916616798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c1a1650 con 0x7ff85c102840 2024-05-27T08:14:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.570+0000 7ff85be00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff85c10da80 0x7ff85c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.570+0000 7ff860a00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff85c102840 0x7ff85c19c7f0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7ff850009230 tx=0x7ff850004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.570+0000 7ff859600640 1 -- 172.21.15.63:0/916616798 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85003f030 con 0x7ff85c102840 2024-05-27T08:14:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.571+0000 7ff859600640 1 -- 172.21.15.63:0/916616798 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff850003a80 con 0x7ff85c102840 2024-05-27T08:14:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.571+0000 7ff861e00640 1 -- 172.21.15.63:0/916616798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c1a18e0 con 0x7ff85c102840 2024-05-27T08:14:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.571+0000 7ff859600640 1 -- 172.21.15.63:0/916616798 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850035070 con 0x7ff85c102840 2024-05-27T08:14:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.571+0000 7ff861e00640 1 -- 172.21.15.63:0/916616798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c1a1dd0 con 0x7ff85c102840 2024-05-27T08:14:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.574+0000 7ff86a589740 1 Processor -- start 2024-05-27T08:14:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.575+0000 7ff86a589740 1 -- start start 2024-05-27T08:14:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.575+0000 7ff86a589740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cad9ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.575+0000 7ff86a589740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9d32f0 0x5629cad9efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.575+0000 7ff86a589740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 0x5629cad9f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.575+0000 7ff86a589740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5629cad0fb90 con 0x5629ca9fc1b0 2024-05-27T08:14:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.575+0000 7ff86a589740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629cacff730 con 0x5629ca9db4d0 2024-05-27T08:14:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.575+0000 7ff86a589740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5629cad03770 con 0x5629ca9d32f0 2024-05-27T08:14:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.575+0000 7ff861400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cad9ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.576+0000 7ff859600640 1 -- 172.21.15.63:0/916616798 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff850042030 con 0x7ff85c102840 2024-05-27T08:14:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.576+0000 7ff861400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cad9ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51316/0 (socket says 172.21.15.63:51316) 2024-05-27T08:14:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.576+0000 7ff861400640 1 -- 172.21.15.63:0/3794010721 learned_addr learned my addr 172.21.15.63:0/3794010721 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.576+0000 7ff85be00640 1 --2- 172.21.15.63:0/3794010721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 0x5629cad9f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.576+0000 7ff861400640 1 -- 172.21.15.63:0/3794010721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9d32f0 msgr2=0x5629cad9efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.576+0000 7ff861400640 1 --2- 172.21.15.63:0/3794010721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9d32f0 0x5629cad9efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.576+0000 7ff861400640 1 -- 172.21.15.63:0/3794010721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 msgr2=0x5629cad9f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.576+0000 7ff861400640 1 --2- 172.21.15.63:0/3794010721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 0x5629cad9f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.576+0000 7ff861400640 1 -- 172.21.15.63:0/3794010721 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629cac988b0 con 0x5629ca9fc1b0 2024-05-27T08:14:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.576+0000 7ff85be00640 1 --2- 172.21.15.63:0/3794010721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 0x5629cad9f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.577+0000 7ff861400640 1 --2- 172.21.15.63:0/3794010721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cad9ea60 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7ff854009e60 tx=0x7ff854010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171f87bcd1934d01 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.577+0000 7ff84dc00640 1 -- 172.21.15.63:0/3794010721 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854016070 con 0x5629ca9fc1b0 2024-05-27T08:14:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.577+0000 7ff84dc00640 1 -- 172.21.15.63:0/3794010721 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff854004d20 con 0x5629ca9fc1b0 2024-05-27T08:14:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.577+0000 7ff84dc00640 1 -- 172.21.15.63:0/3794010721 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854012650 con 0x5629ca9fc1b0 2024-05-27T08:14:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.578+0000 7ff86a589740 1 -- 172.21.15.63:0/3794010721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 msgr2=0x5629cad9ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.578+0000 7ff86a589740 1 --2- 172.21.15.63:0/3794010721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cad9ea60 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7ff854009e60 tx=0x7ff854010740 comp rx=0 tx=0).stop 2024-05-27T08:14:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.578+0000 7ff86a589740 1 -- 172.21.15.63:0/3794010721 shutdown_connections 2024-05-27T08:14:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.578+0000 7ff86a589740 1 --2- 172.21.15.63:0/3794010721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 0x5629cad9f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.578+0000 7ff86a589740 1 --2- 172.21.15.63:0/3794010721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9d32f0 0x5629cad9efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.578+0000 7ff86a589740 1 --2- 172.21.15.63:0/3794010721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cad9ea60 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.578+0000 7ff86a589740 1 -- 172.21.15.63:0/3794010721 >> 172.21.15.63:0/3794010721 conn(0x5629ca9f0300 msgr2=0x5629cad9b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.578+0000 7ff86a589740 1 -- 172.21.15.63:0/3794010721 shutdown_connections 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.578+0000 7ff86a589740 1 -- 172.21.15.63:0/3794010721 wait complete. 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.579+0000 7ff859600640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8440b2860 0x7ff8440b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.579+0000 7ff86a589740 1 -- start start 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.579+0000 7ff86a589740 1 Processor -- start 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.579+0000 7ff859600640 1 -- 172.21.15.63:0/916616798 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7ff850103250 con 0x7ff85c102840 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.579+0000 7ff861400640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8440b2860 0x7ff8440b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff86a589740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 0x5629cada8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff86a589740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9db4d0 0x5629cada91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff86a589740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9fc1b0 0x5629cadb4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff86a589740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5629ca9e5790 con 0x5629ca9db4d0 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff86a589740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629caca7bc0 con 0x5629ca9d32f0 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff86a589740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5629cadabb90 con 0x5629ca9fc1b0 2024-05-27T08:14:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff86a589740 10 client.0 _ll_register_callbacks cb 0x5629cac85400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff861400640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8440b2860 0x7ff8440b4d70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff854005950 tx=0x7ff854000f40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff85be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9db4d0 0x5629cada91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff85be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9db4d0 0x5629cada91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51318/0 (socket says 172.21.15.63:51318) 2024-05-27T08:14:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.580+0000 7ff85be00640 1 -- 172.21.15.63:0/2020689293 learned_addr learned my addr 172.21.15.63:0/2020689293 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.581+0000 7ff860a00640 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 0x5629cada8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.581+0000 7ff861400640 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9fc1b0 0x5629cadb4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.581+0000 7ff85be00640 1 -- 172.21.15.63:0/2020689293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9fc1b0 msgr2=0x5629cadb4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.581+0000 7ff85be00640 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9fc1b0 0x5629cadb4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.581+0000 7ff85be00640 1 -- 172.21.15.63:0/2020689293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 msgr2=0x5629cada8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.581+0000 7ff85be00640 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 0x5629cada8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.581+0000 7ff85be00640 1 -- 172.21.15.63:0/2020689293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629caa14210 con 0x5629ca9db4d0 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.581+0000 7ff85be00640 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9db4d0 0x5629cada91b0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7ff84800bf00 tx=0x7ff8480117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ff03b1529b4a55 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.581+0000 7ff842000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.582+0000 7ff842000640 1 -- 172.21.15.63:0/2020689293 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848004220 con 0x5629ca9db4d0 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.582+0000 7ff842000640 1 -- 172.21.15.63:0/2020689293 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7ff8480043c0 con 0x5629ca9db4d0 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.582+0000 7ff86a589740 1 -- 172.21.15.63:0/2020689293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff854004d20 con 0x5629ca9db4d0 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.582+0000 7ff842000640 1 -- 172.21.15.63:0/2020689293 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff848005020 con 0x5629ca9db4d0 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.582+0000 7ff86a589740 10 client.23154 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.582+0000 7ff86a589740 1 -- 172.21.15.63:0/2020689293 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5629caae0620 con 0x5629ca9db4d0 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.583+0000 7ff842000640 1 -- 172.21.15.63:0/2020689293 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff854004d20 con 0x5629ca9db4d0 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.583+0000 7ff842000640 1 -- 172.21.15.63:0/2020689293 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff8480035f0 con 0x5629ca9db4d0 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.583+0000 7ff86a589740 10 client.23154 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.583+0000 7ff86a589740 10 client.23154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.583+0000 7ff86a589740 20 client.23154 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.583+0000 7ff86a589740 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ca9bad80 0x5629cadbaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.583+0000 7ff86a589740 4 client.23154 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.583+0000 7ff86a589740 1 -- 172.21.15.63:0/2020689293 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629cad11ae0 con 0x5629ca9bad80 2024-05-27T08:14:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.584+0000 7ff860a00640 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ca9bad80 0x5629cadbaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.585+0000 7ff860a00640 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ca9bad80 0x5629cadbaed0 crc :-1 s=READY pgs=718 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7bc42b892931033 server_cookie=8c28dd696d1f584f in_seq=0 out_seq=0 2024-05-27T08:14:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.585+0000 7ff842000640 10 client.23154 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff842000640 1 -- 172.21.15.63:0/2020689293 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5629cad11ae0 con 0x5629ca9bad80 2024-05-27T08:14:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff842000640 10 client.23154 handle_command_reply: tid=0 2024-05-27T08:14:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff86a589740 1 client.23154 shutdown 2024-05-27T08:14:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff86a589740 1 -- 172.21.15.63:0/2020689293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9db4d0 msgr2=0x5629cada91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff86a589740 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9db4d0 0x5629cada91b0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7ff84800bf00 tx=0x7ff8480117b0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff86a589740 1 -- 172.21.15.63:0/2020689293 shutdown_connections 2024-05-27T08:14:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff86a589740 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ca9bad80 0x5629cadbaed0 crc :-1 s=READY pgs=718 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff86a589740 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9fc1b0 0x5629cadb4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff86a589740 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9db4d0 0x5629cada91b0 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff86a589740 1 --2- 172.21.15.63:0/2020689293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 0x5629cada8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff86a589740 1 -- 172.21.15.63:0/2020689293 >> 172.21.15.63:0/2020689293 conn(0x5629ca9f0300 msgr2=0x5629cad9e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.587+0000 7ff842000640 0 client.23154 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.588+0000 7ff86a589740 1 -- 172.21.15.63:0/2020689293 shutdown_connections 2024-05-27T08:14:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.588+0000 7ff86a589740 1 -- 172.21.15.63:0/2020689293 wait complete. 2024-05-27T08:14:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.588+0000 7ff86a589740 20 client.23154 trim_cache size 0 max 16384 2024-05-27T08:14:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.607+0000 7ff86a589740 1 Processor -- start 2024-05-27T08:14:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.607+0000 7ff86a589740 1 -- start start 2024-05-27T08:14:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.607+0000 7ff86a589740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 0x5629cada4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.607+0000 7ff86a589740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9db4d0 0x5629cad9a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.607+0000 7ff86a589740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cada91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.607+0000 7ff86a589740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1b5b20 con 0x5629ca9fc1b0 2024-05-27T08:14:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.607+0000 7ff86a589740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1b33f0 con 0x5629ca9d32f0 2024-05-27T08:14:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.607+0000 7ff86a589740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1b1260 con 0x5629ca9db4d0 2024-05-27T08:14:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff861400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cada91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff861400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cada91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51320/0 (socket says 172.21.15.63:51320) 2024-05-27T08:14:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff861400640 1 -- 172.21.15.63:0/2108099572 learned_addr learned my addr 172.21.15.63:0/2108099572 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff860a00640 1 --2- 172.21.15.63:0/2108099572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 0x5629cada4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff85be00640 1 --2- 172.21.15.63:0/2108099572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9db4d0 0x5629cad9a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff861400640 1 -- 172.21.15.63:0/2108099572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9db4d0 msgr2=0x5629cad9a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff861400640 1 --2- 172.21.15.63:0/2108099572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9db4d0 0x5629cad9a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff861400640 1 -- 172.21.15.63:0/2108099572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 msgr2=0x5629cada4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff861400640 1 --2- 172.21.15.63:0/2108099572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 0x5629cada4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff861400640 1 -- 172.21.15.63:0/2108099572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ca9bafb0 con 0x5629ca9fc1b0 2024-05-27T08:14:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.608+0000 7ff860a00640 1 --2- 172.21.15.63:0/2108099572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 0x5629cada4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.609+0000 7ff861400640 1 --2- 172.21.15.63:0/2108099572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cada91a0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7ff854005980 tx=0x7ff85401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41dbb5807799981 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.609+0000 7ff84d200640 1 -- 172.21.15.63:0/2108099572 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854016050 con 0x5629ca9fc1b0 2024-05-27T08:14:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.609+0000 7ff84d200640 1 -- 172.21.15.63:0/2108099572 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8540318f0 con 0x5629ca9fc1b0 2024-05-27T08:14:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.609+0000 7ff84d200640 1 -- 172.21.15.63:0/2108099572 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8540126e0 con 0x5629ca9fc1b0 2024-05-27T08:14:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.610+0000 7ff86a589740 1 -- 172.21.15.63:0/2108099572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 msgr2=0x5629cada91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.610+0000 7ff86a589740 1 --2- 172.21.15.63:0/2108099572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cada91a0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7ff854005980 tx=0x7ff85401f000 comp rx=0 tx=0).stop 2024-05-27T08:14:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.610+0000 7ff86a589740 1 -- 172.21.15.63:0/2108099572 shutdown_connections 2024-05-27T08:14:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.610+0000 7ff86a589740 1 --2- 172.21.15.63:0/2108099572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9fc1b0 0x5629cada91a0 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.610+0000 7ff86a589740 1 --2- 172.21.15.63:0/2108099572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9db4d0 0x5629cad9a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.610+0000 7ff86a589740 1 --2- 172.21.15.63:0/2108099572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9d32f0 0x5629cada4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.610+0000 7ff86a589740 1 -- 172.21.15.63:0/2108099572 >> 172.21.15.63:0/2108099572 conn(0x5629ca9f0300 msgr2=0x5629cad9b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.610+0000 7ff86a589740 1 -- 172.21.15.63:0/2108099572 shutdown_connections 2024-05-27T08:14:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.610+0000 7ff86a589740 1 -- 172.21.15.63:0/2108099572 wait complete. 2024-05-27T08:14:02.616 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.611+0000 7ff86a589740 1 -- start start 2024-05-27T08:14:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.611+0000 7ff86a589740 1 Processor -- start 2024-05-27T08:14:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff86a589740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9d32f0 0x5629cadabde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff86a589740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 0x5629cadac320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff86a589740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9fc1b0 0x5629cadac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff86a589740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1b18b0 con 0x5629ca9d32f0 2024-05-27T08:14:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff86a589740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1b2d20 con 0x5629ca9db4d0 2024-05-27T08:14:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff86a589740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1b1f80 con 0x5629ca9fc1b0 2024-05-27T08:14:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff86a589740 10 client.0 _ll_register_callbacks cb 0x7ff84800b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff860a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9d32f0 0x5629cadabde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff860a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9d32f0 0x5629cadabde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51330/0 (socket says 172.21.15.63:51330) 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff860a00640 1 -- 172.21.15.63:0/18496503 learned_addr learned my addr 172.21.15.63:0/18496503 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff85be00640 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 0x5629cadac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.612+0000 7ff861400640 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9fc1b0 0x5629cadac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff860a00640 1 -- 172.21.15.63:0/18496503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9fc1b0 msgr2=0x5629cadac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff860a00640 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9fc1b0 0x5629cadac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff860a00640 1 -- 172.21.15.63:0/18496503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 msgr2=0x5629cadac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff860a00640 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 0x5629cadac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff860a00640 1 -- 172.21.15.63:0/18496503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629cae21590 con 0x5629ca9d32f0 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff860a00640 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9d32f0 0x5629cadabde0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7ff8500ca0a0 tx=0x7ff85002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121663dd2f934916 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff842000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff842000640 1 -- 172.21.15.63:0/18496503 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8500e3020 con 0x5629ca9d32f0 2024-05-27T08:14:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff842000640 1 -- 172.21.15.63:0/18496503 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(160..160 src has 1..160) ==== 9483+0+0 (secure 0 0 0) 0x7ff8500c63a0 con 0x5629ca9d32f0 2024-05-27T08:14:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff86a589740 1 -- 172.21.15.63:0/18496503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8540318f0 con 0x5629ca9d32f0 2024-05-27T08:14:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.613+0000 7ff842000640 1 -- 172.21.15.63:0/18496503 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff850049040 con 0x5629ca9d32f0 2024-05-27T08:14:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.614+0000 7ff86a589740 10 client.23160 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.614+0000 7ff86a589740 1 -- 172.21.15.63:0/18496503 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5629cae22840 con 0x5629ca9d32f0 2024-05-27T08:14:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.614+0000 7ff842000640 1 -- 172.21.15.63:0/18496503 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8540318f0 con 0x5629ca9d32f0 2024-05-27T08:14:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.614+0000 7ff842000640 1 -- 172.21.15.63:0/18496503 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff850100800 con 0x5629ca9d32f0 2024-05-27T08:14:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.614+0000 7ff86a589740 10 client.23160 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:02.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.614+0000 7ff86a589740 10 client.23160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.614+0000 7ff86a589740 20 client.23160 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.614+0000 7ff86a589740 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ca9901c0 0x5629cae24fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.614+0000 7ff86a589740 4 client.23160 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:14:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.614+0000 7ff86a589740 1 -- 172.21.15.63:0/18496503 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629cad0b2c0 con 0x5629ca9901c0 2024-05-27T08:14:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.615+0000 7ff85be00640 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ca9901c0 0x5629cae24fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:02.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.616+0000 7ff85be00640 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ca9901c0 0x5629cae24fb0 crc :-1 s=READY pgs=719 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2593c525fa9a18d4 server_cookie=d63bb321f9ef9a11 in_seq=0 out_seq=0 2024-05-27T08:14:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.616+0000 7ff842000640 10 client.23160 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.652+0000 7ff842000640 1 -- 172.21.15.63:0/18496503 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5629cad0b2c0 con 0x5629ca9901c0 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.652+0000 7ff842000640 10 client.23160 handle_command_reply: tid=0 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.652+0000 7ff86a589740 1 client.23160 shutdown 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 1 -- 172.21.15.63:0/18496503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9d32f0 msgr2=0x5629cadabde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9d32f0 0x5629cadabde0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7ff8500ca0a0 tx=0x7ff85002a3f0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 1 -- 172.21.15.63:0/18496503 shutdown_connections 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5629ca9901c0 0x5629cae24fb0 crc :-1 s=READY pgs=719 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629ca9fc1b0 0x5629cadac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629ca9db4d0 0x5629cadac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 1 --2- 172.21.15.63:0/18496503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5629ca9d32f0 0x5629cadabde0 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff842000640 0 client.23160 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 1 -- 172.21.15.63:0/18496503 >> 172.21.15.63:0/18496503 conn(0x5629ca9f0300 msgr2=0x5629cad9b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 1 -- 172.21.15.63:0/18496503 shutdown_connections 2024-05-27T08:14:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 1 -- 172.21.15.63:0/18496503 wait complete. 2024-05-27T08:14:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.653+0000 7ff86a589740 20 client.23160 trim_cache size 0 max 16384 2024-05-27T08:14:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:14:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:14:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:14:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.654+0000 7ff861e00640 1 -- 172.21.15.63:0/916616798 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8440b2860 msgr2=0x7ff8440b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.654+0000 7ff861e00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8440b2860 0x7ff8440b4d70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff854005950 tx=0x7ff854000f40 comp rx=0 tx=0).stop 2024-05-27T08:14:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.654+0000 7ff861e00640 1 -- 172.21.15.63:0/916616798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff85c102840 msgr2=0x7ff85c19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.654+0000 7ff861e00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff85c102840 0x7ff85c19c7f0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7ff850009230 tx=0x7ff850004270 comp rx=0 tx=0).stop 2024-05-27T08:14:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.655+0000 7ff861e00640 1 -- 172.21.15.63:0/916616798 shutdown_connections 2024-05-27T08:14:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.655+0000 7ff861e00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff8440b2860 0x7ff8440b4d70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.655+0000 7ff861e00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff85c1103e0 0x7ff85c1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.655+0000 7ff861e00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff85c10da80 0x7ff85c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.655+0000 7ff861e00640 1 --2- 172.21.15.63:0/916616798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff85c102840 0x7ff85c19c7f0 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.655+0000 7ff861e00640 1 -- 172.21.15.63:0/916616798 >> 172.21.15.63:0/916616798 conn(0x7ff85c0fe210 msgr2=0x7ff85c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.655+0000 7ff861e00640 1 -- 172.21.15.63:0/916616798 shutdown_connections 2024-05-27T08:14:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:02.655+0000 7ff861e00640 1 -- 172.21.15.63:0/916616798 wait complete. 2024-05-27T08:14:04.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:03 smithi086 ceph-mon[34687]: pgmap v1284: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.0 MiB/s wr, 31 op/s 2024-05-27T08:14:04.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:03 smithi149 ceph-mon[35408]: pgmap v1284: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.0 MiB/s wr, 31 op/s 2024-05-27T08:14:04.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:03 smithi063 ceph-mon[31613]: pgmap v1284: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.0 MiB/s wr, 31 op/s 2024-05-27T08:14:06.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:05 smithi086 ceph-mon[34687]: pgmap v1285: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.9 MiB/s wr, 23 op/s 2024-05-27T08:14:06.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:05 smithi149 ceph-mon[35408]: pgmap v1285: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.9 MiB/s wr, 23 op/s 2024-05-27T08:14:06.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:05 smithi063 ceph-mon[31613]: pgmap v1285: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.9 MiB/s wr, 23 op/s 2024-05-27T08:14:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:07 smithi086 ceph-mon[34687]: pgmap v1286: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.4 MiB/s wr, 30 op/s 2024-05-27T08:14:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:07 smithi086 ceph-mon[34687]: osdmap e161: 12 total, 12 up, 12 in 2024-05-27T08:14:08.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:07 smithi149 ceph-mon[35408]: pgmap v1286: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.4 MiB/s wr, 30 op/s 2024-05-27T08:14:08.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:07 smithi149 ceph-mon[35408]: osdmap e161: 12 total, 12 up, 12 in 2024-05-27T08:14:08.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:07 smithi063 ceph-mon[31613]: pgmap v1286: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.4 MiB/s wr, 30 op/s 2024-05-27T08:14:08.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:07 smithi063 ceph-mon[31613]: osdmap e161: 12 total, 12 up, 12 in 2024-05-27T08:14:10.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:09 smithi086 ceph-mon[34687]: pgmap v1288: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:14:10.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:09 smithi149 ceph-mon[35408]: pgmap v1288: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:14:10.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:09 smithi063 ceph-mon[31613]: pgmap v1288: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:14:11.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:10 smithi086 ceph-mon[34687]: pgmap v1289: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:14:11.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:10 smithi149 ceph-mon[35408]: pgmap v1289: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:14:11.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:10 smithi063 ceph-mon[31613]: pgmap v1289: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.5 MiB/s wr, 32 op/s 2024-05-27T08:14:14.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:13 smithi086 ceph-mon[34687]: pgmap v1290: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T08:14:14.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:13 smithi149 ceph-mon[35408]: pgmap v1290: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T08:14:14.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:13 smithi063 ceph-mon[31613]: pgmap v1290: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T08:14:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.902+0000 7f643d200640 1 -- 172.21.15.63:0/2327104161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f643810e900 msgr2=0x7f643810ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.902+0000 7f643d200640 1 --2- 172.21.15.63:0/2327104161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f643810e900 0x7f643810ed30 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f6420009900 tx=0x7f642002a410 comp rx=0 tx=0).stop 2024-05-27T08:14:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.902+0000 7f643d200640 1 -- 172.21.15.63:0/2327104161 shutdown_connections 2024-05-27T08:14:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.902+0000 7f643d200640 1 --2- 172.21.15.63:0/2327104161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f643810f3c0 0x7f6438116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.902+0000 7f643d200640 1 --2- 172.21.15.63:0/2327104161 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f643810e900 0x7f643810ed30 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.902+0000 7f643d200640 1 --2- 172.21.15.63:0/2327104161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810a390 0x7f643810a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.902+0000 7f643d200640 1 -- 172.21.15.63:0/2327104161 >> 172.21.15.63:0/2327104161 conn(0x7f6438076070 msgr2=0x7f6438076480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.902+0000 7f643d200640 1 -- 172.21.15.63:0/2327104161 shutdown_connections 2024-05-27T08:14:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.902+0000 7f643d200640 1 -- 172.21.15.63:0/2327104161 wait complete. 2024-05-27T08:14:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.903+0000 7f643d200640 1 Processor -- start 2024-05-27T08:14:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.904+0000 7f643d200640 1 -- start start 2024-05-27T08:14:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.904+0000 7f643d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f643810a390 0x7f64381a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.904+0000 7f643d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810e900 0x7f64381a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.904+0000 7f643d200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f643810f3c0 0x7f64381a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.904+0000 7f643d200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f643811a1a0 con 0x7f643810f3c0 2024-05-27T08:14:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.904+0000 7f643d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f643811a020 con 0x7f643810e900 2024-05-27T08:14:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.904+0000 7f643d200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f643811a320 con 0x7f643810a390 2024-05-27T08:14:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f6437e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f643810f3c0 0x7f64381a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f6437e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f643810f3c0 0x7f64381a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43934/0 (socket says 172.21.15.63:43934) 2024-05-27T08:14:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f6437e00640 1 -- 172.21.15.63:0/2237997801 learned_addr learned my addr 172.21.15.63:0/2237997801 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f6437400640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f643810a390 0x7f64381a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f642fe00640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810e900 0x7f64381a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f6437e00640 1 -- 172.21.15.63:0/2237997801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f643810a390 msgr2=0x7f64381a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f6437e00640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f643810a390 0x7f64381a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f6437e00640 1 -- 172.21.15.63:0/2237997801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810e900 msgr2=0x7f64381a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f6437e00640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810e900 0x7f64381a6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f6437e00640 1 -- 172.21.15.63:0/2237997801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64381a1180 con 0x7f643810f3c0 2024-05-27T08:14:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f6437400640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f643810a390 0x7f64381a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.905+0000 7f642fe00640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810e900 0x7f64381a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.906+0000 7f6437e00640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f643810f3c0 0x7f64381a0ae0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f643000eaa0 tx=0x7f643000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.906+0000 7f6434c00640 1 -- 172.21.15.63:0/2237997801 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643000cdb0 con 0x7f643810f3c0 2024-05-27T08:14:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.906+0000 7f6434c00640 1 -- 172.21.15.63:0/2237997801 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6430005020 con 0x7f643810f3c0 2024-05-27T08:14:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.906+0000 7f6434c00640 1 -- 172.21.15.63:0/2237997801 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64300116d0 con 0x7f643810f3c0 2024-05-27T08:14:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.906+0000 7f643d200640 1 -- 172.21.15.63:0/2237997801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64381a1470 con 0x7f643810f3c0 2024-05-27T08:14:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.906+0000 7f643d200640 1 -- 172.21.15.63:0/2237997801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64381ad750 con 0x7f643810f3c0 2024-05-27T08:14:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.910+0000 7f644580b740 1 Processor -- start 2024-05-27T08:14:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.911+0000 7f6434c00640 1 -- 172.21.15.63:0/2237997801 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f64300051d0 con 0x7f643810f3c0 2024-05-27T08:14:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.911+0000 7f644580b740 1 -- start start 2024-05-27T08:14:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.912+0000 7f644580b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cd71a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.912+0000 7f644580b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9a62f0 0x556e1cd71fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.912+0000 7f644580b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9ae4d0 0x556e1cd724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.912+0000 7f644580b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556e1cce2b90 con 0x556e1c9a62f0 2024-05-27T08:14:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.912+0000 7f644580b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e1ccd2730 con 0x556e1c9cf1b0 2024-05-27T08:14:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.912+0000 7f644580b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556e1ccd6770 con 0x556e1c9ae4d0 2024-05-27T08:14:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.912+0000 7f6437400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9a62f0 0x556e1cd71fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.912+0000 7f6437400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9a62f0 0x556e1cd71fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43944/0 (socket says 172.21.15.63:43944) 2024-05-27T08:14:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.912+0000 7f6437400640 1 -- 172.21.15.63:0/1638659653 learned_addr learned my addr 172.21.15.63:0/1638659653 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.912+0000 7f642fe00640 1 --2- 172.21.15.63:0/1638659653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cd71a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f642fe00640 1 --2- 172.21.15.63:0/1638659653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9ae4d0 0x556e1cd724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f6437400640 1 -- 172.21.15.63:0/1638659653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9ae4d0 msgr2=0x556e1cd724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f6437400640 1 --2- 172.21.15.63:0/1638659653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9ae4d0 0x556e1cd724e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f6434c00640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f641c0b29d0 0x7f641c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f6437400640 1 -- 172.21.15.63:0/1638659653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 msgr2=0x556e1cd71a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f6437400640 1 --2- 172.21.15.63:0/1638659653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cd71a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f6437400640 1 -- 172.21.15.63:0/1638659653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1cc6b8b0 con 0x556e1c9a62f0 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f642fe00640 1 --2- 172.21.15.63:0/1638659653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cd71a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f6434c00640 1 -- 172.21.15.63:0/2237997801 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 9571+0+0 (secure 0 0 0) 0x7f64300e2c80 con 0x7f643810f3c0 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f6437e00640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f641c0b29d0 0x7f641c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.913+0000 7f6437400640 1 --2- 172.21.15.63:0/1638659653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9a62f0 0x556e1cd71fa0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f642800bd00 tx=0x7f642800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56f0d64a5f7036f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.914+0000 7f642cc00640 1 -- 172.21.15.63:0/1638659653 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428015070 con 0x556e1c9a62f0 2024-05-27T08:14:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.914+0000 7f642cc00640 1 -- 172.21.15.63:0/1638659653 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64280118d0 con 0x556e1c9a62f0 2024-05-27T08:14:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.914+0000 7f642cc00640 1 -- 172.21.15.63:0/1638659653 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64280107e0 con 0x556e1c9a62f0 2024-05-27T08:14:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.915+0000 7f644580b740 1 -- 172.21.15.63:0/1638659653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9a62f0 msgr2=0x556e1cd71fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.915+0000 7f644580b740 1 --2- 172.21.15.63:0/1638659653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9a62f0 0x556e1cd71fa0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f642800bd00 tx=0x7f642800f730 comp rx=0 tx=0).stop 2024-05-27T08:14:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.915+0000 7f6437e00640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f641c0b29d0 0x7f641c0b4ee0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f64300092f0 tx=0x7f643000cf50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.915+0000 7f644580b740 1 -- 172.21.15.63:0/1638659653 shutdown_connections 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.915+0000 7f644580b740 1 --2- 172.21.15.63:0/1638659653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9ae4d0 0x556e1cd724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.915+0000 7f644580b740 1 --2- 172.21.15.63:0/1638659653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9a62f0 0x556e1cd71fa0 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.915+0000 7f644580b740 1 --2- 172.21.15.63:0/1638659653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cd71a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.915+0000 7f644580b740 1 -- 172.21.15.63:0/1638659653 >> 172.21.15.63:0/1638659653 conn(0x556e1c9c3300 msgr2=0x556e1cd6e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.915+0000 7f644580b740 1 -- 172.21.15.63:0/1638659653 shutdown_connections 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.915+0000 7f644580b740 1 -- 172.21.15.63:0/1638659653 wait complete. 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.916+0000 7f644580b740 1 -- start start 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.916+0000 7f644580b740 1 Processor -- start 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f644580b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd7bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f644580b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f644580b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cd87ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f644580b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556e1c9b8790 con 0x556e1c9ae4d0 2024-05-27T08:14:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f644580b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e1cc7abc0 con 0x556e1c9cf1b0 2024-05-27T08:14:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f644580b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556e1cd7eb90 con 0x556e1c9a62f0 2024-05-27T08:14:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f644580b740 10 client.0 _ll_register_callbacks cb 0x556e1cc58400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f6437400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f6437400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43954/0 (socket says 172.21.15.63:43954) 2024-05-27T08:14:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f6437400640 1 -- 172.21.15.63:0/3830743805 learned_addr learned my addr 172.21.15.63:0/3830743805 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f642fe00640 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd7bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.917+0000 7f642fe00640 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cd87ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f6437400640 1 -- 172.21.15.63:0/3830743805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 msgr2=0x556e1cd7bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f6437400640 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd7bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f6437400640 1 -- 172.21.15.63:0/3830743805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 msgr2=0x556e1cd87ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f6437400640 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cd87ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f6437400640 1 -- 172.21.15.63:0/3830743805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1c9e7210 con 0x556e1c9ae4d0 2024-05-27T08:14:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f642fe00640 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cd87ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:14:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f6437400640 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7c120 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f6428019000 tx=0x7f6428011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5711cc3946dc670 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f6425600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f6425600640 1 -- 172.21.15.63:0/3830743805 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428028060 con 0x556e1c9ae4d0 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f644580b740 1 -- 172.21.15.63:0/3830743805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64280118d0 con 0x556e1c9ae4d0 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.918+0000 7f6425600640 1 -- 172.21.15.63:0/3830743805 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 9571+0+0 (secure 0 0 0) 0x7f6428005040 con 0x556e1c9ae4d0 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.919+0000 7f6425600640 1 -- 172.21.15.63:0/3830743805 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f642802f050 con 0x556e1c9ae4d0 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.919+0000 7f644580b740 10 client.23172 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.919+0000 7f644580b740 1 -- 172.21.15.63:0/3830743805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556e1cab3620 con 0x556e1c9ae4d0 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.919+0000 7f6425600640 1 -- 172.21.15.63:0/3830743805 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64280118d0 con 0x556e1c9ae4d0 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.919+0000 7f6425600640 1 -- 172.21.15.63:0/3830743805 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6428022030 con 0x556e1c9ae4d0 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.919+0000 7f644580b740 10 client.23172 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.919+0000 7f644580b740 10 client.23172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.919+0000 7f644580b740 20 client.23172 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.919+0000 7f644580b740 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e1c98dd80 0x556e1cd8dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.919+0000 7f644580b740 4 client.23172 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.920+0000 7f644580b740 1 -- 172.21.15.63:0/3830743805 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e1cce4ae0 con 0x556e1c98dd80 2024-05-27T08:14:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.920+0000 7f642fe00640 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e1c98dd80 0x556e1cd8dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.921+0000 7f642fe00640 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e1c98dd80 0x556e1cd8dce0 crc :-1 s=READY pgs=720 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c657c4e3bf22b033 server_cookie=7ba41128b23906d in_seq=0 out_seq=0 2024-05-27T08:14:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.921+0000 7f6425600640 10 client.23172 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.923+0000 7f6425600640 1 -- 172.21.15.63:0/3830743805 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556e1cce4ae0 con 0x556e1c98dd80 2024-05-27T08:14:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.923+0000 7f6425600640 10 client.23172 handle_command_reply: tid=0 2024-05-27T08:14:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.923+0000 7f644580b740 1 client.23172 shutdown 2024-05-27T08:14:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 1 -- 172.21.15.63:0/3830743805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 msgr2=0x556e1cd7c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7c120 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f6428019000 tx=0x7f6428011670 comp rx=0 tx=0).stop 2024-05-27T08:14:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 1 -- 172.21.15.63:0/3830743805 shutdown_connections 2024-05-27T08:14:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e1c98dd80 0x556e1cd8dce0 crc :-1 s=READY pgs=720 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cd87ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7c120 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 1 --2- 172.21.15.63:0/3830743805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd7bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 1 -- 172.21.15.63:0/3830743805 >> 172.21.15.63:0/3830743805 conn(0x556e1c9c3300 msgr2=0x556e1cd73780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f6425600640 0 client.23172 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 1 -- 172.21.15.63:0/3830743805 shutdown_connections 2024-05-27T08:14:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 1 -- 172.21.15.63:0/3830743805 wait complete. 2024-05-27T08:14:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.924+0000 7f644580b740 20 client.23172 trim_cache size 0 max 16384 2024-05-27T08:14:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.943+0000 7f644580b740 1 Processor -- start 2024-05-27T08:14:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.943+0000 7f644580b740 1 -- start start 2024-05-27T08:14:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.943+0000 7f644580b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.943+0000 7f644580b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9ae4d0 0x556e1cd72440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.943+0000 7f644580b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9cf1b0 0x556e1cd7dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.943+0000 7f644580b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64381ba140 con 0x556e1c9cf1b0 2024-05-27T08:14:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.943+0000 7f644580b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64381b7a10 con 0x556e1c9ae4d0 2024-05-27T08:14:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.943+0000 7f644580b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f64381b5880 con 0x556e1c9a62f0 2024-05-27T08:14:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.944+0000 7f642fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9cf1b0 0x556e1cd7dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.944+0000 7f642fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9cf1b0 0x556e1cd7dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43960/0 (socket says 172.21.15.63:43960) 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.944+0000 7f642fe00640 1 -- 172.21.15.63:0/3772777819 learned_addr learned my addr 172.21.15.63:0/3772777819 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.944+0000 7f6437400640 1 --2- 172.21.15.63:0/3772777819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9ae4d0 0x556e1cd72440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.944+0000 7f642fe00640 1 --2- 172.21.15.63:0/3772777819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.944+0000 7f642fe00640 1 -- 172.21.15.63:0/3772777819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 msgr2=0x556e1cd773e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.944+0000 7f642fe00640 1 --2- 172.21.15.63:0/3772777819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd773e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.944+0000 7f642fe00640 1 -- 172.21.15.63:0/3772777819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9ae4d0 msgr2=0x556e1cd72440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.944+0000 7f642fe00640 1 --2- 172.21.15.63:0/3772777819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9ae4d0 0x556e1cd72440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.944+0000 7f642fe00640 1 -- 172.21.15.63:0/3772777819 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1cd6eb10 con 0x556e1c9cf1b0 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.945+0000 7f642fe00640 1 --2- 172.21.15.63:0/3772777819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9cf1b0 0x556e1cd7dd30 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f6420032fd0 tx=0x7f64200056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6920a4d497ca1701 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.945+0000 7f6427e00640 1 -- 172.21.15.63:0/3772777819 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642004c030 con 0x556e1c9cf1b0 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.945+0000 7f6427e00640 1 -- 172.21.15.63:0/3772777819 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64200044d0 con 0x556e1c9cf1b0 2024-05-27T08:14:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.945+0000 7f6427e00640 1 -- 172.21.15.63:0/3772777819 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420035040 con 0x556e1c9cf1b0 2024-05-27T08:14:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.946+0000 7f644580b740 1 -- 172.21.15.63:0/3772777819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9cf1b0 msgr2=0x556e1cd7dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.946+0000 7f644580b740 1 --2- 172.21.15.63:0/3772777819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9cf1b0 0x556e1cd7dd30 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f6420032fd0 tx=0x7f64200056d0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.947+0000 7f644580b740 1 -- 172.21.15.63:0/3772777819 shutdown_connections 2024-05-27T08:14:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.947+0000 7f644580b740 1 --2- 172.21.15.63:0/3772777819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9cf1b0 0x556e1cd7dd30 unknown :-1 s=CLOSED pgs=2879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.947+0000 7f644580b740 1 --2- 172.21.15.63:0/3772777819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9ae4d0 0x556e1cd72440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.947+0000 7f644580b740 1 --2- 172.21.15.63:0/3772777819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd773e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.947+0000 7f644580b740 1 -- 172.21.15.63:0/3772777819 >> 172.21.15.63:0/3772777819 conn(0x556e1c9c3300 msgr2=0x556e1cd7b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.947+0000 7f644580b740 1 -- 172.21.15.63:0/3772777819 shutdown_connections 2024-05-27T08:14:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.947+0000 7f644580b740 1 -- 172.21.15.63:0/3772777819 wait complete. 2024-05-27T08:14:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.947+0000 7f644580b740 1 -- start start 2024-05-27T08:14:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.947+0000 7f644580b740 1 Processor -- start 2024-05-27T08:14:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.948+0000 7f644580b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd7f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.948+0000 7f644580b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.948+0000 7f644580b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cdee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.948+0000 7f644580b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64381b5ed0 con 0x556e1c9ae4d0 2024-05-27T08:14:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.948+0000 7f644580b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64381b7340 con 0x556e1c9cf1b0 2024-05-27T08:14:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.948+0000 7f644580b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f64381b65a0 con 0x556e1c9a62f0 2024-05-27T08:14:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.948+0000 7f644580b740 10 client.0 _ll_register_callbacks cb 0x7f642800bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.948+0000 7f6437400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.948+0000 7f6437400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43962/0 (socket says 172.21.15.63:43962) 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.948+0000 7f6437400640 1 -- 172.21.15.63:0/4117360180 learned_addr learned my addr 172.21.15.63:0/4117360180 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.949+0000 7f642fe00640 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd7f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.949+0000 7f642fe00640 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cdee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.949+0000 7f6437400640 1 -- 172.21.15.63:0/4117360180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 msgr2=0x556e1cd7f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.949+0000 7f6437400640 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd7f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.949+0000 7f6437400640 1 -- 172.21.15.63:0/4117360180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 msgr2=0x556e1cdee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.949+0000 7f6437400640 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cdee140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.949+0000 7f6437400640 1 -- 172.21.15.63:0/4117360180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1cdee700 con 0x556e1c9ae4d0 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.949+0000 7f6437400640 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7fb50 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f6428019000 tx=0x7f642800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd61df1a5399b64 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.949+0000 7f6425600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.949+0000 7f6425600640 1 -- 172.21.15.63:0/4117360180 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428028050 con 0x556e1c9ae4d0 2024-05-27T08:14:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.950+0000 7f6425600640 1 -- 172.21.15.63:0/4117360180 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 9571+0+0 (secure 0 0 0) 0x7f6428004680 con 0x556e1c9ae4d0 2024-05-27T08:14:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.950+0000 7f644580b740 1 -- 172.21.15.63:0/4117360180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64200044d0 con 0x556e1c9ae4d0 2024-05-27T08:14:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.950+0000 7f6425600640 1 -- 172.21.15.63:0/4117360180 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6428043050 con 0x556e1c9ae4d0 2024-05-27T08:14:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.950+0000 7f644580b740 10 client.23178 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.950+0000 7f644580b740 1 -- 172.21.15.63:0/4117360180 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556e1cdf5a60 con 0x556e1c9ae4d0 2024-05-27T08:14:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.950+0000 7f6425600640 1 -- 172.21.15.63:0/4117360180 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64200044d0 con 0x556e1c9ae4d0 2024-05-27T08:14:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.950+0000 7f6425600640 1 -- 172.21.15.63:0/4117360180 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f6428022040 con 0x556e1c9ae4d0 2024-05-27T08:14:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.951+0000 7f644580b740 10 client.23178 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.951+0000 7f644580b740 10 client.23178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.951+0000 7f644580b740 20 client.23178 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.951+0000 7f644580b740 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e1c9631c0 0x556e1cdf80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.951+0000 7f644580b740 4 client.23178 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:14:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.951+0000 7f644580b740 1 -- 172.21.15.63:0/4117360180 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e1ccde2c0 con 0x556e1c9631c0 2024-05-27T08:14:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.951+0000 7f642fe00640 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e1c9631c0 0x556e1cdf80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.952+0000 7f6425600640 10 client.23178 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.952+0000 7f642fe00640 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e1c9631c0 0x556e1cdf80d0 crc :-1 s=READY pgs=721 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d08c847c306d2640 server_cookie=5b05b49ebb14f6ed in_seq=0 out_seq=0 2024-05-27T08:14:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.986+0000 7f6425600640 1 -- 172.21.15.63:0/4117360180 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556e1ccde2c0 con 0x556e1c9631c0 2024-05-27T08:14:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.986+0000 7f6425600640 10 client.23178 handle_command_reply: tid=0 2024-05-27T08:14:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.986+0000 7f644580b740 1 client.23178 shutdown 2024-05-27T08:14:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 1 -- 172.21.15.63:0/4117360180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 msgr2=0x556e1cd7fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7fb50 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f6428019000 tx=0x7f642800bee0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 1 -- 172.21.15.63:0/4117360180 shutdown_connections 2024-05-27T08:14:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556e1c9631c0 0x556e1cdf80d0 crc :-1 s=READY pgs=721 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1c9cf1b0 0x556e1cdee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e1c9ae4d0 0x556e1cd7fb50 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 1 --2- 172.21.15.63:0/4117360180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556e1c9a62f0 0x556e1cd7f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f6425600640 0 client.23178 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 1 -- 172.21.15.63:0/4117360180 >> 172.21.15.63:0/4117360180 conn(0x556e1c9c3300 msgr2=0x556e1cd7be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 1 -- 172.21.15.63:0/4117360180 shutdown_connections 2024-05-27T08:14:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 1 -- 172.21.15.63:0/4117360180 wait complete. 2024-05-27T08:14:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.987+0000 7f644580b740 20 client.23178 trim_cache size 0 max 16384 2024-05-27T08:14:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:14:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:14:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:14:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 -- 172.21.15.63:0/2237997801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f641c0b29d0 msgr2=0x7f641c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f641c0b29d0 0x7f641c0b4ee0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f64300092f0 tx=0x7f643000cf50 comp rx=0 tx=0).stop 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 -- 172.21.15.63:0/2237997801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f643810f3c0 msgr2=0x7f64381a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f643810f3c0 0x7f64381a0ae0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f643000eaa0 tx=0x7f643000ef60 comp rx=0 tx=0).stop 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 -- 172.21.15.63:0/2237997801 shutdown_connections 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f641c0b29d0 0x7f641c0b4ee0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f643810f3c0 0x7f64381a0ae0 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f643810e900 0x7f64381a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 --2- 172.21.15.63:0/2237997801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f643810a390 0x7f64381a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 -- 172.21.15.63:0/2237997801 >> 172.21.15.63:0/2237997801 conn(0x7f6438076070 msgr2=0x7f6438107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 -- 172.21.15.63:0/2237997801 shutdown_connections 2024-05-27T08:14:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:15.988+0000 7f643d200640 1 -- 172.21.15.63:0/2237997801 wait complete. 2024-05-27T08:14:16.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:15 smithi086 ceph-mon[34687]: pgmap v1291: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T08:14:16.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:15 smithi149 ceph-mon[35408]: pgmap v1291: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T08:14:16.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:15 smithi063 ceph-mon[31613]: pgmap v1291: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-05-27T08:14:17.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:17 smithi086 ceph-mon[34687]: osdmap e162: 12 total, 12 up, 12 in 2024-05-27T08:14:17.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:17 smithi086 ceph-mon[34687]: pgmap v1293: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 13 op/s 2024-05-27T08:14:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:17 smithi149 ceph-mon[35408]: osdmap e162: 12 total, 12 up, 12 in 2024-05-27T08:14:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:17 smithi149 ceph-mon[35408]: pgmap v1293: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 13 op/s 2024-05-27T08:14:17.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:17 smithi063 ceph-mon[31613]: osdmap e162: 12 total, 12 up, 12 in 2024-05-27T08:14:17.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:17 smithi063 ceph-mon[31613]: pgmap v1293: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 13 op/s 2024-05-27T08:14:20.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:19 smithi086 ceph-mon[34687]: pgmap v1294: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:14:20.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:19 smithi149 ceph-mon[35408]: pgmap v1294: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:14:20.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:19 smithi063 ceph-mon[31613]: pgmap v1294: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:14:21.300 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:14:21.830 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:14:22.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:21 smithi086 ceph-mon[34687]: pgmap v1295: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:14:22.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:21 smithi149 ceph-mon[35408]: pgmap v1295: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:14:22.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:21 smithi063 ceph-mon[31613]: pgmap v1295: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.8 MiB/s wr, 12 op/s 2024-05-27T08:14:24.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:23 smithi086 ceph-mon[34687]: pgmap v1296: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T08:14:24.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:23 smithi149 ceph-mon[35408]: pgmap v1296: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T08:14:24.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:23 smithi063 ceph-mon[31613]: pgmap v1296: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T08:14:25.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.381+0000 7fb14a600700 1 -- 172.21.15.63:0/1851074037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14410b2d0 msgr2=0x7fb144117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:25.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.381+0000 7fb14a600700 1 --2- 172.21.15.63:0/1851074037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14410b2d0 0x7fb144117800 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00b0a0 tx=0x7fb13c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.382+0000 7fb14a600700 1 -- 172.21.15.63:0/1851074037 shutdown_connections 2024-05-27T08:14:25.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.382+0000 7fb14a600700 1 --2- 172.21.15.63:0/1851074037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14410b2d0 0x7fb144117800 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.382+0000 7fb14a600700 1 --2- 172.21.15.63:0/1851074037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb144110960 0x7fb144110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.382+0000 7fb14a600700 1 --2- 172.21.15.63:0/1851074037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb14410a980 0x7fb14410ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.382+0000 7fb14a600700 1 -- 172.21.15.63:0/1851074037 >> 172.21.15.63:0/1851074037 conn(0x7fb144077c70 msgr2=0x7fb1440780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:25.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.382+0000 7fb14a600700 1 -- 172.21.15.63:0/1851074037 shutdown_connections 2024-05-27T08:14:25.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.382+0000 7fb14a600700 1 -- 172.21.15.63:0/1851074037 wait complete. 2024-05-27T08:14:25.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.384+0000 7fb14a600700 1 Processor -- start 2024-05-27T08:14:25.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.384+0000 7fb14a600700 1 -- start start 2024-05-27T08:14:25.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.384+0000 7fb14a600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14410a980 0x7fb1441a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.384+0000 7fb14a600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14410b2d0 0x7fb1441a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.384+0000 7fb14a600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144110960 0x7fb1441a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.384+0000 7fb14a600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb14411a930 con 0x7fb14410b2d0 2024-05-27T08:14:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.384+0000 7fb14a600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb14411a7b0 con 0x7fb144110960 2024-05-27T08:14:25.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.384+0000 7fb14a600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb14411aab0 con 0x7fb14410a980 2024-05-27T08:14:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.385+0000 7fb13aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14410b2d0 0x7fb1441a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.385+0000 7fb13aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14410b2d0 0x7fb1441a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34486/0 (socket says 172.21.15.63:34486) 2024-05-27T08:14:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.385+0000 7fb13aa00700 1 -- 172.21.15.63:0/3962125816 learned_addr learned my addr 172.21.15.63:0/3962125816 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.385+0000 7fb143400700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14410a980 0x7fb1441a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.385+0000 7fb143e00700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144110960 0x7fb1441a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.386+0000 7fb13aa00700 1 -- 172.21.15.63:0/3962125816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14410a980 msgr2=0x7fb1441a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.386+0000 7fb13aa00700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14410a980 0x7fb1441a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.386+0000 7fb13aa00700 1 -- 172.21.15.63:0/3962125816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144110960 msgr2=0x7fb1441a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:25.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.386+0000 7fb13aa00700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144110960 0x7fb1441a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.386+0000 7fb13aa00700 1 -- 172.21.15.63:0/3962125816 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1441a6190 con 0x7fb14410b2d0 2024-05-27T08:14:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.386+0000 7fb13aa00700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14410b2d0 0x7fb1441a1860 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00e970 tx=0x7fb12c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.387+0000 7fb140c00700 1 -- 172.21.15.63:0/3962125816 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c004a30 con 0x7fb14410b2d0 2024-05-27T08:14:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.387+0000 7fb140c00700 1 -- 172.21.15.63:0/3962125816 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb12c00cc70 con 0x7fb14410b2d0 2024-05-27T08:14:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.387+0000 7fb140c00700 1 -- 172.21.15.63:0/3962125816 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c005520 con 0x7fb14410b2d0 2024-05-27T08:14:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.387+0000 7fb14a600700 1 -- 172.21.15.63:0/3962125816 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1441a6420 con 0x7fb14410b2d0 2024-05-27T08:14:25.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.387+0000 7fb14a600700 1 -- 172.21.15.63:0/3962125816 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1441adda0 con 0x7fb14410b2d0 2024-05-27T08:14:25.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.392+0000 7fb140c00700 1 -- 172.21.15.63:0/3962125816 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb12c00c7e0 con 0x7fb14410b2d0 2024-05-27T08:14:25.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.394+0000 7fb140c00700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1280b29f0 0x7fb1280b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.394+0000 7fb140c00700 1 -- 172.21.15.63:0/3962125816 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fb12c0e2c10 con 0x7fb14410b2d0 2024-05-27T08:14:25.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.394+0000 7fb143400700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1280b29f0 0x7fb1280b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.395+0000 7fb143400700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1280b29f0 0x7fb1280b4f00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb134005950 tx=0x7fb13400b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:25.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.410+0000 7fb1591f6b80 1 Processor -- start 2024-05-27T08:14:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.410+0000 7fb1591f6b80 1 -- start start 2024-05-27T08:14:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.411+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 0x55b48da8e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.411+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 0x55b48da88870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.411+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da88db0 0x55b48da891c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.411+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b48d9b6e40 con 0x55b48da88db0 2024-05-27T08:14:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.411+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b48d9a69e0 con 0x55b48d909e40 2024-05-27T08:14:25.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.411+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b48d9aaa20 con 0x55b48da8ed30 2024-05-27T08:14:25.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.411+0000 7fb143e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da88db0 0x55b48da891c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.411+0000 7fb143e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da88db0 0x55b48da891c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34488/0 (socket says 172.21.15.63:34488) 2024-05-27T08:14:25.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.411+0000 7fb143e00700 1 -- 172.21.15.63:0/937583982 learned_addr learned my addr 172.21.15.63:0/937583982 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:25.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.411+0000 7fb13aa00700 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 0x55b48da88870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.412+0000 7fb143400700 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 0x55b48da8e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.412+0000 7fb143e00700 1 -- 172.21.15.63:0/937583982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 msgr2=0x55b48da88870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:25.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.412+0000 7fb143e00700 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 0x55b48da88870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.412+0000 7fb143e00700 1 -- 172.21.15.63:0/937583982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 msgr2=0x55b48da8e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:25.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.412+0000 7fb143e00700 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 0x55b48da8e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.412+0000 7fb143e00700 1 -- 172.21.15.63:0/937583982 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b48d96f420 con 0x55b48da88db0 2024-05-27T08:14:25.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.412+0000 7fb143400700 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 0x55b48da8e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:25.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.412+0000 7fb13aa00700 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 0x55b48da88870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:25.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.412+0000 7fb143e00700 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da88db0 0x55b48da891c0 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fb13c006120 tx=0x7fb13c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b4344730d81d8c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:25.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.413+0000 7fb138c00700 1 -- 172.21.15.63:0/937583982 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c020070 con 0x55b48da88db0 2024-05-27T08:14:25.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.413+0000 7fb138c00700 1 -- 172.21.15.63:0/937583982 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb13c00bcb0 con 0x55b48da88db0 2024-05-27T08:14:25.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.413+0000 7fb138c00700 1 -- 172.21.15.63:0/937583982 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c007cf0 con 0x55b48da88db0 2024-05-27T08:14:25.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.414+0000 7fb1591f6b80 1 -- 172.21.15.63:0/937583982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da88db0 msgr2=0x55b48da891c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:25.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.414+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da88db0 0x55b48da891c0 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fb13c006120 tx=0x7fb13c0092e0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.414+0000 7fb1591f6b80 1 -- 172.21.15.63:0/937583982 shutdown_connections 2024-05-27T08:14:25.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.414+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da88db0 0x55b48da891c0 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.414+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 0x55b48da8e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.414+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/937583982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 0x55b48da88870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.414+0000 7fb1591f6b80 1 -- 172.21.15.63:0/937583982 >> 172.21.15.63:0/937583982 conn(0x55b48d969fc0 msgr2=0x55b48da838d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:25.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.414+0000 7fb1591f6b80 1 -- 172.21.15.63:0/937583982 shutdown_connections 2024-05-27T08:14:25.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.414+0000 7fb1591f6b80 1 -- 172.21.15.63:0/937583982 wait complete. 2024-05-27T08:14:25.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.415+0000 7fb1591f6b80 1 -- start start 2024-05-27T08:14:25.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.415+0000 7fb1591f6b80 1 Processor -- start 2024-05-27T08:14:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.416+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 0x55b48da925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.416+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 0x55b48da92b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.416+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48da93070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.416+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b48d8acfa0 con 0x55b48da8ed30 2024-05-27T08:14:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.416+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b48d947760 con 0x55b48d909e40 2024-05-27T08:14:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.416+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b48da957a0 con 0x55b48da88db0 2024-05-27T08:14:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.416+0000 7fb1591f6b80 10 client.0 _ll_register_callbacks cb 0x55b48d950c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:25.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.416+0000 7fb143e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48da93070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb143e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48da93070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34490/0 (socket says 172.21.15.63:34490) 2024-05-27T08:14:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb143e00700 1 -- 172.21.15.63:0/580637279 learned_addr learned my addr 172.21.15.63:0/580637279 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb143400700 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 0x55b48da925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb13aa00700 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 0x55b48da92b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb143e00700 1 -- 172.21.15.63:0/580637279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 msgr2=0x55b48da92b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb143e00700 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 0x55b48da92b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb143e00700 1 -- 172.21.15.63:0/580637279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 msgr2=0x55b48da925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb143e00700 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 0x55b48da925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb143e00700 1 -- 172.21.15.63:0/580637279 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b48d8f68d0 con 0x55b48da8ed30 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb143400700 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 0x55b48da925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.417+0000 7fb13aa00700 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 0x55b48da92b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.418+0000 7fb143e00700 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48da93070 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7fb13c007b20 tx=0x7fb13c007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbafb3f869e5a12a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.418+0000 7fb131600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.418+0000 7fb131600700 1 -- 172.21.15.63:0/580637279 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00be60 con 0x55b48da8ed30 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.418+0000 7fb1591f6b80 1 -- 172.21.15.63:0/580637279 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb13c00bcb0 con 0x55b48da8ed30 2024-05-27T08:14:25.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.418+0000 7fb131600700 1 -- 172.21.15.63:0/580637279 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fb13c003e00 con 0x55b48da8ed30 2024-05-27T08:14:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.418+0000 7fb131600700 1 -- 172.21.15.63:0/580637279 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb13c020040 con 0x55b48da8ed30 2024-05-27T08:14:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.418+0000 7fb1591f6b80 10 client.23190 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.419+0000 7fb1591f6b80 1 -- 172.21.15.63:0/580637279 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b48d977060 con 0x55b48da8ed30 2024-05-27T08:14:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.419+0000 7fb131600700 1 -- 172.21.15.63:0/580637279 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb13c00bcb0 con 0x55b48da8ed30 2024-05-27T08:14:25.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.419+0000 7fb131600700 1 -- 172.21.15.63:0/580637279 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb13c011070 con 0x55b48da8ed30 2024-05-27T08:14:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.419+0000 7fb1591f6b80 10 client.23190 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.419+0000 7fb1591f6b80 10 client.23190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.419+0000 7fb1591f6b80 20 client.23190 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.419+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b48daa2620 0x55b48daa4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.419+0000 7fb1591f6b80 4 client.23190 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.419+0000 7fb1591f6b80 1 -- 172.21.15.63:0/580637279 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b48d9b8d90 con 0x55b48daa2620 2024-05-27T08:14:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.420+0000 7fb143400700 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b48daa2620 0x55b48daa4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.421+0000 7fb143400700 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b48daa2620 0x55b48daa4a40 crc :-1 s=READY pgs=722 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c24c2ecf25ffe07 server_cookie=41102d0215b651ea in_seq=0 out_seq=0 2024-05-27T08:14:25.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.421+0000 7fb131600700 10 client.23190 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.423+0000 7fb131600700 1 -- 172.21.15.63:0/580637279 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b48d9b8d90 con 0x55b48daa2620 2024-05-27T08:14:25.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.423+0000 7fb131600700 10 client.23190 handle_command_reply: tid=0 2024-05-27T08:14:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.423+0000 7fb1591f6b80 1 client.23190 shutdown 2024-05-27T08:14:25.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 1 -- 172.21.15.63:0/580637279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 msgr2=0x55b48da93070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48da93070 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7fb13c007b20 tx=0x7fb13c007c00 comp rx=0 tx=0).stop 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 1 -- 172.21.15.63:0/580637279 shutdown_connections 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b48daa2620 0x55b48daa4a40 crc :-1 s=READY pgs=722 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48da93070 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48d909e40 0x55b48da925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/580637279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 0x55b48da92b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 1 -- 172.21.15.63:0/580637279 >> 172.21.15.63:0/580637279 conn(0x55b48d969fc0 msgr2=0x55b48da83580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb131600700 0 client.23190 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 1 -- 172.21.15.63:0/580637279 shutdown_connections 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 1 -- 172.21.15.63:0/580637279 wait complete. 2024-05-27T08:14:25.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.424+0000 7fb1591f6b80 20 client.23190 trim_cache size 0 max 16384 2024-05-27T08:14:25.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.460+0000 7fb1591f6b80 1 Processor -- start 2024-05-27T08:14:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.460+0000 7fb1591f6b80 1 -- start start 2024-05-27T08:14:25.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48d909e40 0x55b48da88870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48da88db0 0x55b48da94a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 0x55b48da914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441b9300 con 0x55b48d909e40 2024-05-27T08:14:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441b6b90 con 0x55b48da88db0 2024-05-27T08:14:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441b4a00 con 0x55b48da8ed30 2024-05-27T08:14:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb143400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48d909e40 0x55b48da88870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb143400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48d909e40 0x55b48da88870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34494/0 (socket says 172.21.15.63:34494) 2024-05-27T08:14:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb143400700 1 -- 172.21.15.63:0/1072057745 learned_addr learned my addr 172.21.15.63:0/1072057745 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb143e00700 1 --2- 172.21.15.63:0/1072057745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 0x55b48da914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.461+0000 7fb13aa00700 1 --2- 172.21.15.63:0/1072057745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48da88db0 0x55b48da94a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.462+0000 7fb143400700 1 -- 172.21.15.63:0/1072057745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 msgr2=0x55b48da914c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:25.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.462+0000 7fb143400700 1 --2- 172.21.15.63:0/1072057745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 0x55b48da914c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.462+0000 7fb143400700 1 -- 172.21.15.63:0/1072057745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48da88db0 msgr2=0x55b48da94a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:25.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.462+0000 7fb143400700 1 --2- 172.21.15.63:0/1072057745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48da88db0 0x55b48da94a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.462+0000 7fb143400700 1 -- 172.21.15.63:0/1072057745 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b48da891c0 con 0x55b48d909e40 2024-05-27T08:14:25.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.462+0000 7fb143400700 1 --2- 172.21.15.63:0/1072057745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48d909e40 0x55b48da88870 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x55b48da89b30 tx=0x7fb134013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7115029670fab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:25.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.463+0000 7fb133e00700 1 -- 172.21.15.63:0/1072057745 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134011c40 con 0x55b48d909e40 2024-05-27T08:14:25.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.463+0000 7fb133e00700 1 -- 172.21.15.63:0/1072057745 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb134011de0 con 0x55b48d909e40 2024-05-27T08:14:25.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.463+0000 7fb133e00700 1 -- 172.21.15.63:0/1072057745 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134027280 con 0x55b48d909e40 2024-05-27T08:14:25.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.464+0000 7fb1591f6b80 1 -- 172.21.15.63:0/1072057745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48d909e40 msgr2=0x55b48da88870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:25.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.464+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/1072057745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48d909e40 0x55b48da88870 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x55b48da89b30 tx=0x7fb134013000 comp rx=0 tx=0).stop 2024-05-27T08:14:25.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.464+0000 7fb1591f6b80 1 -- 172.21.15.63:0/1072057745 shutdown_connections 2024-05-27T08:14:25.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.464+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/1072057745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da8ed30 0x55b48da914c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.464+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/1072057745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48d909e40 0x55b48da88870 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.464+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/1072057745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48da88db0 0x55b48da94a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.464+0000 7fb1591f6b80 1 -- 172.21.15.63:0/1072057745 >> 172.21.15.63:0/1072057745 conn(0x55b48d969fc0 msgr2=0x55b48da90db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:25.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.464+0000 7fb1591f6b80 1 -- 172.21.15.63:0/1072057745 shutdown_connections 2024-05-27T08:14:25.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.464+0000 7fb1591f6b80 1 -- 172.21.15.63:0/1072057745 wait complete. 2024-05-27T08:14:25.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.465+0000 7fb1591f6b80 1 -- start start 2024-05-27T08:14:25.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.465+0000 7fb1591f6b80 1 Processor -- start 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 0x55b48daa7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48daa4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb1591f6b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48daa5440 0x55b48daa5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441b5050 con 0x55b48da8ed30 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441b64c0 con 0x55b48daa5440 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb1591f6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441b5720 con 0x55b48da88db0 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb1591f6b80 10 client.0 _ll_register_callbacks cb 0x55b48d950c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb13aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48daa4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb13aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48daa4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34502/0 (socket says 172.21.15.63:34502) 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb13aa00700 1 -- 172.21.15.63:0/143452882 learned_addr learned my addr 172.21.15.63:0/143452882 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:25.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb143400700 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 0x55b48daa7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.466+0000 7fb143e00700 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48daa5440 0x55b48daa5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.467+0000 7fb13aa00700 1 -- 172.21.15.63:0/143452882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 msgr2=0x55b48daa7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.467+0000 7fb13aa00700 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 0x55b48daa7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.467+0000 7fb13aa00700 1 -- 172.21.15.63:0/143452882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48daa5440 msgr2=0x55b48daa5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.467+0000 7fb13aa00700 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48daa5440 0x55b48daa5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.467+0000 7fb13aa00700 1 -- 172.21.15.63:0/143452882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b48db0be90 con 0x55b48da8ed30 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.467+0000 7fb13aa00700 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48daa4f00 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0dffa0 tx=0x7fb12c004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07ec1c661d26cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.467+0000 7fb131600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.468+0000 7fb131600700 1 -- 172.21.15.63:0/143452882 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c06e060 con 0x55b48da8ed30 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.468+0000 7fb131600700 1 -- 172.21.15.63:0/143452882 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fb12c0a9f40 con 0x55b48da8ed30 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.468+0000 7fb1591f6b80 1 -- 172.21.15.63:0/143452882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb134011de0 con 0x55b48da8ed30 2024-05-27T08:14:25.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.468+0000 7fb131600700 1 -- 172.21.15.63:0/143452882 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb12c0b2270 con 0x55b48da8ed30 2024-05-27T08:14:25.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.468+0000 7fb1591f6b80 10 client.23196 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:25.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.468+0000 7fb1591f6b80 1 -- 172.21.15.63:0/143452882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b48db0d030 con 0x55b48da8ed30 2024-05-27T08:14:25.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.468+0000 7fb131600700 1 -- 172.21.15.63:0/143452882 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb134011de0 con 0x55b48da8ed30 2024-05-27T08:14:25.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.469+0000 7fb131600700 1 -- 172.21.15.63:0/143452882 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb12c0b24a0 con 0x55b48da8ed30 2024-05-27T08:14:25.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.469+0000 7fb1591f6b80 10 client.23196 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:25.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.469+0000 7fb1591f6b80 10 client.23196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:25.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.469+0000 7fb1591f6b80 20 client.23196 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:25.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.469+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b48db0d740 0x55b48db0fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:25.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.469+0000 7fb1591f6b80 4 client.23196 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:14:25.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.469+0000 7fb1591f6b80 1 -- 172.21.15.63:0/143452882 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b48d9b2570 con 0x55b48db0d740 2024-05-27T08:14:25.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.470+0000 7fb143400700 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b48db0d740 0x55b48db0fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:25.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.471+0000 7fb143400700 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b48db0d740 0x55b48db0fb60 crc :-1 s=READY pgs=723 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8ca83dc6c0c938 server_cookie=63fa572080f47fb1 in_seq=0 out_seq=0 2024-05-27T08:14:25.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.471+0000 7fb131600700 10 client.23196 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:25.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.471+0000 7fb131600700 1 -- 172.21.15.63:0/143452882 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b48d9b2570 con 0x55b48db0d740 2024-05-27T08:14:25.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.471+0000 7fb131600700 10 client.23196 handle_command_reply: tid=0 2024-05-27T08:14:25.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.471+0000 7fb1591f6b80 1 client.23196 shutdown 2024-05-27T08:14:25.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.471+0000 7fb1591f6b80 1 -- 172.21.15.63:0/143452882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 msgr2=0x55b48daa4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:25.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.471+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48daa4f00 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0dffa0 tx=0x7fb12c004e00 comp rx=0 tx=0).stop 2024-05-27T08:14:25.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.472+0000 7fb1591f6b80 1 -- 172.21.15.63:0/143452882 shutdown_connections 2024-05-27T08:14:25.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.472+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b48db0d740 0x55b48db0fb60 crc :-1 s=READY pgs=723 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.472+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b48daa5440 0x55b48daa5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.472+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b48da88db0 0x55b48daa7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.472+0000 7fb1591f6b80 1 --2- 172.21.15.63:0/143452882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b48da8ed30 0x55b48daa4f00 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.472+0000 7fb1591f6b80 1 -- 172.21.15.63:0/143452882 >> 172.21.15.63:0/143452882 conn(0x55b48d969fc0 msgr2=0x55b48da9d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:25.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.472+0000 7fb131600700 0 client.23196 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:25.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.472+0000 7fb1591f6b80 1 -- 172.21.15.63:0/143452882 shutdown_connections 2024-05-27T08:14:25.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.472+0000 7fb1591f6b80 1 -- 172.21.15.63:0/143452882 wait complete. 2024-05-27T08:14:25.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.472+0000 7fb1591f6b80 20 client.23196 trim_cache size 0 max 16384 2024-05-27T08:14:25.475 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:14:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:14:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:14:25.475 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.473+0000 7fb14a600700 1 -- 172.21.15.63:0/3962125816 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1280b29f0 msgr2=0x7fb1280b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.473+0000 7fb14a600700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1280b29f0 0x7fb1280b4f00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb134005950 tx=0x7fb13400b410 comp rx=0 tx=0).stop 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.473+0000 7fb14a600700 1 -- 172.21.15.63:0/3962125816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14410b2d0 msgr2=0x7fb1441a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.473+0000 7fb14a600700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14410b2d0 0x7fb1441a1860 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00e970 tx=0x7fb12c00ec80 comp rx=0 tx=0).stop 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.473+0000 7fb14a600700 1 -- 172.21.15.63:0/3962125816 shutdown_connections 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.473+0000 7fb14a600700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb1280b29f0 0x7fb1280b4f00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.473+0000 7fb14a600700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144110960 0x7fb1441a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.473+0000 7fb14a600700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb14410a980 0x7fb1441a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.473+0000 7fb14a600700 1 --2- 172.21.15.63:0/3962125816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb14410b2d0 0x7fb1441a1860 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.473+0000 7fb14a600700 1 -- 172.21.15.63:0/3962125816 >> 172.21.15.63:0/3962125816 conn(0x7fb144077c70 msgr2=0x7fb14407b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.474+0000 7fb14a600700 1 -- 172.21.15.63:0/3962125816 shutdown_connections 2024-05-27T08:14:25.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:25.474+0000 7fb14a600700 1 -- 172.21.15.63:0/3962125816 wait complete. 2024-05-27T08:14:26.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:25 smithi086 ceph-mon[34687]: pgmap v1297: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T08:14:26.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:25 smithi149 ceph-mon[35408]: pgmap v1297: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T08:14:26.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:25 smithi063 ceph-mon[31613]: pgmap v1297: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.4 MiB/s wr, 11 op/s 2024-05-27T08:14:26.410 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:14:26.410 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:14:26.916 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:14:27.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:26 smithi086 ceph-mon[34687]: pgmap v1298: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.3 MiB/s wr, 15 op/s 2024-05-27T08:14:27.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:26 smithi149 ceph-mon[35408]: pgmap v1298: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.3 MiB/s wr, 15 op/s 2024-05-27T08:14:27.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:26 smithi063 ceph-mon[31613]: pgmap v1298: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.3 MiB/s wr, 15 op/s 2024-05-27T08:14:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.248+0000 7f13a6000640 1 -- 172.21.15.63:0/826628303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13a0104bb0 msgr2=0x7f13a0104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.248+0000 7f13a6000640 1 --2- 172.21.15.63:0/826628303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13a0104bb0 0x7f13a0104fc0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f1390009900 tx=0x7f139002a410 comp rx=0 tx=0).stop 2024-05-27T08:14:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.249+0000 7f13a6000640 1 -- 172.21.15.63:0/826628303 shutdown_connections 2024-05-27T08:14:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.249+0000 7f13a6000640 1 --2- 172.21.15.63:0/826628303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a010e000 0x7f13a01104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.249+0000 7f13a6000640 1 --2- 172.21.15.63:0/826628303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13a010b610 0x7f13a010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.249+0000 7f13a6000640 1 --2- 172.21.15.63:0/826628303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13a0104bb0 0x7f13a0104fc0 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.249+0000 7f13a6000640 1 -- 172.21.15.63:0/826628303 >> 172.21.15.63:0/826628303 conn(0x7f13a00fe030 msgr2=0x7f13a0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.249+0000 7f13a6000640 1 -- 172.21.15.63:0/826628303 shutdown_connections 2024-05-27T08:14:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.249+0000 7f13a6000640 1 -- 172.21.15.63:0/826628303 wait complete. 2024-05-27T08:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.250+0000 7f13a6000640 1 Processor -- start 2024-05-27T08:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.250+0000 7f13a6000640 1 -- start start 2024-05-27T08:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f13a6000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13a0104bb0 0x7f13a019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f13a6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13a010b610 0x7f13a019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f13a6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a010e000 0x7f13a01a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f13a6000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0113b60 con 0x7f13a0104bb0 2024-05-27T08:14:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f13a6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01139e0 con 0x7f13a010e000 2024-05-27T08:14:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f13a6000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0113ce0 con 0x7f13a010b610 2024-05-27T08:14:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f139f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13a0104bb0 0x7f13a019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f139f400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13a0104bb0 0x7f13a019c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58980/0 (socket says 172.21.15.63:58980) 2024-05-27T08:14:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f139f400640 1 -- 172.21.15.63:0/3581563205 learned_addr learned my addr 172.21.15.63:0/3581563205 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f139ea00640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13a010b610 0x7f13a019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.251+0000 7f139fe00640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a010e000 0x7f13a01a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f139f400640 1 -- 172.21.15.63:0/3581563205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13a010b610 msgr2=0x7f13a019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f139f400640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13a010b610 0x7f13a019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f139f400640 1 -- 172.21.15.63:0/3581563205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a010e000 msgr2=0x7f13a01a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f139f400640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a010e000 0x7f13a01a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f139f400640 1 -- 172.21.15.63:0/3581563205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a01a1600 con 0x7f13a0104bb0 2024-05-27T08:14:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f139ea00640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13a010b610 0x7f13a019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f139f400640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13a0104bb0 0x7f13a019c7a0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f139002a930 tx=0x7f1390003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f1397e00640 1 -- 172.21.15.63:0/3581563205 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139003f070 con 0x7f13a0104bb0 2024-05-27T08:14:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f1397e00640 1 -- 172.21.15.63:0/3581563205 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1390003e40 con 0x7f13a0104bb0 2024-05-27T08:14:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f13a6000640 1 -- 172.21.15.63:0/3581563205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a01a1890 con 0x7f13a0104bb0 2024-05-27T08:14:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.252+0000 7f1397e00640 1 -- 172.21.15.63:0/3581563205 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390035030 con 0x7f13a0104bb0 2024-05-27T08:14:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.253+0000 7f13a6000640 1 -- 172.21.15.63:0/3581563205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a01a1d80 con 0x7f13a0104bb0 2024-05-27T08:14:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.257+0000 7f13ad033740 1 Processor -- start 2024-05-27T08:14:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.257+0000 7f13ad033740 1 -- start start 2024-05-27T08:14:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.258+0000 7f13ad033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbd61b0 0x55907cf78a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.258+0000 7f13ad033740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf78fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.258+0000 7f13ad033740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbb54d0 0x55907cf794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.258+0000 7f13ad033740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55907cee9b90 con 0x55907cbad2f0 2024-05-27T08:14:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.258+0000 7f13ad033740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55907ced9730 con 0x55907cbd61b0 2024-05-27T08:14:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.258+0000 7f13ad033740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55907cedd770 con 0x55907cbb54d0 2024-05-27T08:14:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.258+0000 7f1397e00640 1 -- 172.21.15.63:0/3581563205 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1390042070 con 0x7f13a0104bb0 2024-05-27T08:14:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.258+0000 7f139ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbb54d0 0x55907cf794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.258+0000 7f139fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbd61b0 0x55907cf78a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.259+0000 7f139ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbb54d0 0x55907cf794e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53326/0 (socket says 172.21.15.63:53326) 2024-05-27T08:14:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.259+0000 7f139fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbd61b0 0x55907cf78a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34482/0 (socket says 172.21.15.63:34482) 2024-05-27T08:14:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.259+0000 7f139ea00640 1 -- 172.21.15.63:0/1150699131 learned_addr learned my addr 172.21.15.63:0/1150699131 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.259+0000 7f139f400640 1 --2- 172.21.15.63:0/1150699131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf78fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.259+0000 7f139fe00640 1 -- 172.21.15.63:0/1150699131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbb54d0 msgr2=0x55907cf794e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.259+0000 7f139fe00640 1 --2- 172.21.15.63:0/1150699131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbb54d0 0x55907cf794e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.259+0000 7f139fe00640 1 -- 172.21.15.63:0/1150699131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 msgr2=0x55907cf78fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.259+0000 7f139fe00640 1 --2- 172.21.15.63:0/1150699131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf78fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.259+0000 7f139fe00640 1 -- 172.21.15.63:0/1150699131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55907ce728b0 con 0x55907cbd61b0 2024-05-27T08:14:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.259+0000 7f139ea00640 1 --2- 172.21.15.63:0/1150699131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbb54d0 0x55907cf794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:14:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.260+0000 7f139fe00640 1 --2- 172.21.15.63:0/1150699131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbd61b0 0x55907cf78a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1398009da0 tx=0x7f1398010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67877be9c194006f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.260+0000 7f138be00640 1 -- 172.21.15.63:0/1150699131 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398016070 con 0x55907cbd61b0 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.260+0000 7f138be00640 1 -- 172.21.15.63:0/1150699131 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1398010e50 con 0x55907cbd61b0 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.261+0000 7f138be00640 1 -- 172.21.15.63:0/1150699131 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13980126b0 con 0x55907cbd61b0 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.261+0000 7f1397e00640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13840b2950 0x7f13840b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.261+0000 7f1397e00640 1 -- 172.21.15.63:0/3581563205 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f1390103390 con 0x7f13a0104bb0 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.261+0000 7f139f400640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13840b2950 0x7f13840b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.261+0000 7f13ad033740 1 -- 172.21.15.63:0/1150699131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbd61b0 msgr2=0x55907cf78a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.261+0000 7f13ad033740 1 --2- 172.21.15.63:0/1150699131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbd61b0 0x55907cf78a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1398009da0 tx=0x7f1398010740 comp rx=0 tx=0).stop 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.262+0000 7f13ad033740 1 -- 172.21.15.63:0/1150699131 shutdown_connections 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.262+0000 7f13ad033740 1 --2- 172.21.15.63:0/1150699131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbb54d0 0x55907cf794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.262+0000 7f13ad033740 1 --2- 172.21.15.63:0/1150699131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf78fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.262+0000 7f13ad033740 1 --2- 172.21.15.63:0/1150699131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbd61b0 0x55907cf78a60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.262+0000 7f13ad033740 1 -- 172.21.15.63:0/1150699131 >> 172.21.15.63:0/1150699131 conn(0x55907cbca300 msgr2=0x55907cf75210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.262+0000 7f13ad033740 1 -- 172.21.15.63:0/1150699131 shutdown_connections 2024-05-27T08:14:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.262+0000 7f13ad033740 1 -- 172.21.15.63:0/1150699131 wait complete. 2024-05-27T08:14:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.262+0000 7f139f400640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13840b2950 0x7f13840b4e60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1390106fd0 tx=0x7f1390106e40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.263+0000 7f13ad033740 1 -- start start 2024-05-27T08:14:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.263+0000 7f13ad033740 1 Processor -- start 2024-05-27T08:14:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f13ad033740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbad2f0 0x55907cf82eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f13ad033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf833f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f13ad033740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbd61b0 0x55907cf8eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f13ad033740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55907cbbf790 con 0x55907cbd61b0 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f13ad033740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55907ce81bc0 con 0x55907cbb54d0 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f13ad033740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55907cf85b90 con 0x55907cbad2f0 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f13ad033740 10 client.0 _ll_register_callbacks cb 0x55907ce5f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f139fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbd61b0 0x55907cf8eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f139fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbd61b0 0x55907cf8eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58996/0 (socket says 172.21.15.63:58996) 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f139fe00640 1 -- 172.21.15.63:0/156221979 learned_addr learned my addr 172.21.15.63:0/156221979 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f139fe00640 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbad2f0 0x55907cf82eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.264+0000 7f139ea00640 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf833f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.265+0000 7f139fe00640 1 -- 172.21.15.63:0/156221979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbad2f0 msgr2=0x55907cf82eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.265+0000 7f139fe00640 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbad2f0 0x55907cf82eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.265+0000 7f139fe00640 1 -- 172.21.15.63:0/156221979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 msgr2=0x55907cf833f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.265+0000 7f139fe00640 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf833f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.265+0000 7f139fe00640 1 -- 172.21.15.63:0/156221979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55907cbee210 con 0x55907cbd61b0 2024-05-27T08:14:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.265+0000 7f139fe00640 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbd61b0 0x55907cf8eec0 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f13980092f0 tx=0x7f1398005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92f6c9b4c4c45f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.265+0000 7f1388c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.265+0000 7f1388c00640 1 -- 172.21.15.63:0/156221979 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398016040 con 0x55907cbd61b0 2024-05-27T08:14:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.265+0000 7f1388c00640 1 -- 172.21.15.63:0/156221979 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f139801cc50 con 0x55907cbd61b0 2024-05-27T08:14:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.265+0000 7f13ad033740 1 -- 172.21.15.63:0/156221979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1398010e50 con 0x55907cbd61b0 2024-05-27T08:14:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.266+0000 7f1388c00640 1 -- 172.21.15.63:0/156221979 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f139801c0d0 con 0x55907cbd61b0 2024-05-27T08:14:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.266+0000 7f13ad033740 10 client.23208 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.266+0000 7f13ad033740 1 -- 172.21.15.63:0/156221979 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55907ccba620 con 0x55907cbd61b0 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.266+0000 7f1388c00640 1 -- 172.21.15.63:0/156221979 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1398010e50 con 0x55907cbd61b0 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.266+0000 7f1388c00640 1 -- 172.21.15.63:0/156221979 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f139800f040 con 0x55907cbd61b0 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.267+0000 7f13ad033740 10 client.23208 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.267+0000 7f13ad033740 10 client.23208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.267+0000 7f13ad033740 20 client.23208 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.267+0000 7f13ad033740 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55907cb94d80 0x55907cf94e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.267+0000 7f13ad033740 4 client.23208 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.267+0000 7f13ad033740 1 -- 172.21.15.63:0/156221979 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55907ceebae0 con 0x55907cb94d80 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.267+0000 7f139ea00640 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55907cb94d80 0x55907cf94e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.268+0000 7f139ea00640 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55907cb94d80 0x55907cf94e00 crc :-1 s=READY pgs=724 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=517c30949ac3629b server_cookie=e74f98e918452dc1 in_seq=0 out_seq=0 2024-05-27T08:14:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.268+0000 7f1388c00640 10 client.23208 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.270+0000 7f1388c00640 1 -- 172.21.15.63:0/156221979 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55907ceebae0 con 0x55907cb94d80 2024-05-27T08:14:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.270+0000 7f1388c00640 10 client.23208 handle_command_reply: tid=0 2024-05-27T08:14:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.270+0000 7f13ad033740 1 client.23208 shutdown 2024-05-27T08:14:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 1 -- 172.21.15.63:0/156221979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbd61b0 msgr2=0x55907cf8eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbd61b0 0x55907cf8eec0 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f13980092f0 tx=0x7f1398005a90 comp rx=0 tx=0).stop 2024-05-27T08:14:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 1 -- 172.21.15.63:0/156221979 shutdown_connections 2024-05-27T08:14:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55907cb94d80 0x55907cf94e00 crc :-1 s=READY pgs=724 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbd61b0 0x55907cf8eec0 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf833f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f1388c00640 0 client.23208 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 1 --2- 172.21.15.63:0/156221979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbad2f0 0x55907cf82eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 1 -- 172.21.15.63:0/156221979 >> 172.21.15.63:0/156221979 conn(0x55907cbca300 msgr2=0x55907cf78890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 1 -- 172.21.15.63:0/156221979 shutdown_connections 2024-05-27T08:14:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 1 -- 172.21.15.63:0/156221979 wait complete. 2024-05-27T08:14:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.271+0000 7f13ad033740 20 client.23208 trim_cache size 0 max 16384 2024-05-27T08:14:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.289+0000 7f13ad033740 1 Processor -- start 2024-05-27T08:14:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.289+0000 7f13ad033740 1 -- start start 2024-05-27T08:14:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.289+0000 7f13ad033740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf8ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.289+0000 7f13ad033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf7fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.289+0000 7f13ad033740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbd61b0 0x55907cf80420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.289+0000 7f13ad033740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01b5bf0 con 0x55907cbad2f0 2024-05-27T08:14:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.289+0000 7f13ad033740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01b34c0 con 0x55907cbb54d0 2024-05-27T08:14:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.289+0000 7f13ad033740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01b1330 con 0x55907cbd61b0 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.290+0000 7f139fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf8ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.290+0000 7f139fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf8ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59006/0 (socket says 172.21.15.63:59006) 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.290+0000 7f139fe00640 1 -- 172.21.15.63:0/520998401 learned_addr learned my addr 172.21.15.63:0/520998401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.290+0000 7f139ea00640 1 --2- 172.21.15.63:0/520998401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf7fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.290+0000 7f139fe00640 1 --2- 172.21.15.63:0/520998401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbd61b0 0x55907cf80420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.290+0000 7f139fe00640 1 -- 172.21.15.63:0/520998401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbd61b0 msgr2=0x55907cf80420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.290+0000 7f139fe00640 1 --2- 172.21.15.63:0/520998401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbd61b0 0x55907cf80420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.290+0000 7f139fe00640 1 -- 172.21.15.63:0/520998401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 msgr2=0x55907cf7fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.290+0000 7f139fe00640 1 --2- 172.21.15.63:0/520998401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf7fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.290+0000 7f139fe00640 1 -- 172.21.15.63:0/520998401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55907cf73610 con 0x55907cbad2f0 2024-05-27T08:14:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.291+0000 7f139fe00640 1 --2- 172.21.15.63:0/520998401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf8ee30 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7f139801a530 tx=0x7f1398009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51dcd491aac7912 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.291+0000 7f138b400640 1 -- 172.21.15.63:0/520998401 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398016030 con 0x55907cbad2f0 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.291+0000 7f138b400640 1 -- 172.21.15.63:0/520998401 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f139801ce50 con 0x55907cbad2f0 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.291+0000 7f138b400640 1 -- 172.21.15.63:0/520998401 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139801ae40 con 0x55907cbad2f0 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.292+0000 7f13ad033740 1 -- 172.21.15.63:0/520998401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 msgr2=0x55907cf8ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.292+0000 7f13ad033740 1 --2- 172.21.15.63:0/520998401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf8ee30 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7f139801a530 tx=0x7f1398009fd0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.292+0000 7f13ad033740 1 -- 172.21.15.63:0/520998401 shutdown_connections 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.292+0000 7f13ad033740 1 --2- 172.21.15.63:0/520998401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbd61b0 0x55907cf80420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.292+0000 7f13ad033740 1 --2- 172.21.15.63:0/520998401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf7fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.292+0000 7f13ad033740 1 --2- 172.21.15.63:0/520998401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf8ee30 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.292+0000 7f13ad033740 1 -- 172.21.15.63:0/520998401 >> 172.21.15.63:0/520998401 conn(0x55907cbca300 msgr2=0x55907cf84df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.292+0000 7f13ad033740 1 -- 172.21.15.63:0/520998401 shutdown_connections 2024-05-27T08:14:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.292+0000 7f13ad033740 1 -- 172.21.15.63:0/520998401 wait complete. 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.293+0000 7f13ad033740 1 -- start start 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.293+0000 7f13ad033740 1 Processor -- start 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f13ad033740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf96f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f13ad033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf97440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f13ad033740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbd61b0 0x55907cffb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f13ad033740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01b1980 con 0x55907cbad2f0 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f13ad033740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01b2df0 con 0x55907cbb54d0 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f13ad033740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01b2050 con 0x55907cbd61b0 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f13ad033740 10 client.0 _ll_register_callbacks cb 0x7f1390106d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f139fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf96f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f139fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf96f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59012/0 (socket says 172.21.15.63:59012) 2024-05-27T08:14:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f139fe00640 1 -- 172.21.15.63:0/203256965 learned_addr learned my addr 172.21.15.63:0/203256965 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f139ea00640 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf97440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f139fe00640 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbd61b0 0x55907cffb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f139fe00640 1 -- 172.21.15.63:0/203256965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbd61b0 msgr2=0x55907cffb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f139fe00640 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbd61b0 0x55907cffb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.294+0000 7f139fe00640 1 -- 172.21.15.63:0/203256965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 msgr2=0x55907cf97440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.295+0000 7f139fe00640 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf97440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.295+0000 7f139fe00640 1 -- 172.21.15.63:0/203256965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55907cf97c50 con 0x55907cbad2f0 2024-05-27T08:14:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.295+0000 7f139fe00640 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf96f00 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f139801a780 tx=0x7f139801a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3922e0255cfabac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.295+0000 7f1388c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.295+0000 7f1388c00640 1 -- 172.21.15.63:0/203256965 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398016070 con 0x55907cbad2f0 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.295+0000 7f1388c00640 1 -- 172.21.15.63:0/203256965 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f1398010c40 con 0x55907cbad2f0 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.295+0000 7f13ad033740 1 -- 172.21.15.63:0/203256965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f139801ce50 con 0x55907cbad2f0 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.296+0000 7f1388c00640 1 -- 172.21.15.63:0/203256965 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f139801ae40 con 0x55907cbad2f0 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.296+0000 7f13ad033740 10 client.23214 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.296+0000 7f13ad033740 1 -- 172.21.15.63:0/203256965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55907cffc760 con 0x55907cbad2f0 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.296+0000 7f1388c00640 1 -- 172.21.15.63:0/203256965 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f139801ce50 con 0x55907cbad2f0 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.296+0000 7f1388c00640 1 -- 172.21.15.63:0/203256965 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1398012570 con 0x55907cbad2f0 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.296+0000 7f13ad033740 10 client.23214 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.296+0000 7f13ad033740 10 client.23214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.296+0000 7f13ad033740 20 client.23214 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.296+0000 7f13ad033740 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55907cb6a1c0 0x55907cffecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.297+0000 7f13ad033740 4 client.23214 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.297+0000 7f13ad033740 1 -- 172.21.15.63:0/203256965 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55907cee52c0 con 0x55907cb6a1c0 2024-05-27T08:14:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.297+0000 7f139ea00640 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55907cb6a1c0 0x55907cffecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.298+0000 7f139ea00640 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55907cb6a1c0 0x55907cffecf0 crc :-1 s=READY pgs=725 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c4a6496bf212ea server_cookie=fe7dfefe89ec09e5 in_seq=0 out_seq=0 2024-05-27T08:14:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.298+0000 7f1388c00640 10 client.23214 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.335+0000 7f1388c00640 1 -- 172.21.15.63:0/203256965 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55907cee52c0 con 0x55907cb6a1c0 2024-05-27T08:14:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.335+0000 7f1388c00640 10 client.23214 handle_command_reply: tid=0 2024-05-27T08:14:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.335+0000 7f13ad033740 1 client.23214 shutdown 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f13ad033740 1 -- 172.21.15.63:0/203256965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 msgr2=0x55907cf96f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f13ad033740 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf96f00 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f139801a780 tx=0x7f139801a710 comp rx=0 tx=0).stop 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f13ad033740 1 -- 172.21.15.63:0/203256965 shutdown_connections 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f13ad033740 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55907cb6a1c0 0x55907cffecf0 crc :-1 s=READY pgs=725 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f13ad033740 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55907cbd61b0 0x55907cffb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f13ad033740 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55907cbb54d0 0x55907cf97440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f13ad033740 1 --2- 172.21.15.63:0/203256965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55907cbad2f0 0x55907cf96f00 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f13ad033740 1 -- 172.21.15.63:0/203256965 >> 172.21.15.63:0/203256965 conn(0x55907cbca300 msgr2=0x55907cf84350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f1388c00640 0 client.23214 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f13ad033740 1 -- 172.21.15.63:0/203256965 shutdown_connections 2024-05-27T08:14:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.336+0000 7f13ad033740 1 -- 172.21.15.63:0/203256965 wait complete. 2024-05-27T08:14:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.337+0000 7f13ad033740 20 client.23214 trim_cache size 0 max 16384 2024-05-27T08:14:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:14:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:14:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:14:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.337+0000 7f13a6000640 1 -- 172.21.15.63:0/3581563205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13840b2950 msgr2=0x7f13840b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.337+0000 7f13a6000640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13840b2950 0x7f13840b4e60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1390106fd0 tx=0x7f1390106e40 comp rx=0 tx=0).stop 2024-05-27T08:14:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.337+0000 7f13a6000640 1 -- 172.21.15.63:0/3581563205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13a0104bb0 msgr2=0x7f13a019c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.337+0000 7f13a6000640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13a0104bb0 0x7f13a019c7a0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f139002a930 tx=0x7f1390003c20 comp rx=0 tx=0).stop 2024-05-27T08:14:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.338+0000 7f13a6000640 1 -- 172.21.15.63:0/3581563205 shutdown_connections 2024-05-27T08:14:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.338+0000 7f13a6000640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f13840b2950 0x7f13840b4e60 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.338+0000 7f13a6000640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13a010e000 0x7f13a01a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.338+0000 7f13a6000640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13a010b610 0x7f13a019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.338+0000 7f13a6000640 1 --2- 172.21.15.63:0/3581563205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13a0104bb0 0x7f13a019c7a0 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.338+0000 7f13a6000640 1 -- 172.21.15.63:0/3581563205 >> 172.21.15.63:0/3581563205 conn(0x7f13a00fe030 msgr2=0x7f13a01099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.338+0000 7f13a6000640 1 -- 172.21.15.63:0/3581563205 shutdown_connections 2024-05-27T08:14:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:29.338+0000 7f13a6000640 1 -- 172.21.15.63:0/3581563205 wait complete. 2024-05-27T08:14:29.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.942+0000 7fa39a800700 1 -- 172.21.15.63:0/1925693916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa394076440 msgr2=0x7fa3941106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.942+0000 7fa39a800700 1 --2- 172.21.15.63:0/1925693916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa394076440 0x7fa3941106b0 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7fa380009990 tx=0x7fa380009ca0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.943+0000 7fa39a800700 1 -- 172.21.15.63:0/1925693916 shutdown_connections 2024-05-27T08:14:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.943+0000 7fa39a800700 1 --2- 172.21.15.63:0/1925693916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa394110bf0 0x7fa394113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.943+0000 7fa39a800700 1 --2- 172.21.15.63:0/1925693916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa394075af0 0x7fa394075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.943+0000 7fa39a800700 1 --2- 172.21.15.63:0/1925693916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa394076440 0x7fa3941106b0 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.943+0000 7fa39a800700 1 -- 172.21.15.63:0/1925693916 >> 172.21.15.63:0/1925693916 conn(0x7fa394100030 msgr2=0x7fa394102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.943+0000 7fa39a800700 1 -- 172.21.15.63:0/1925693916 shutdown_connections 2024-05-27T08:14:29.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.943+0000 7fa39a800700 1 -- 172.21.15.63:0/1925693916 wait complete. 2024-05-27T08:14:29.945 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:29 smithi063 ceph-mon[31613]: pgmap v1299: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T08:14:29.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.944+0000 7fa39a800700 1 Processor -- start 2024-05-27T08:14:29.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.945+0000 7fa39a800700 1 -- start start 2024-05-27T08:14:29.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.945+0000 7fa39a800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa394075af0 0x7fa39419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.945+0000 7fa39a800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa394076440 0x7fa39419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.945+0000 7fa39a800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa394110bf0 0x7fa3941a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.945+0000 7fa39a800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941165b0 con 0x7fa394076440 2024-05-27T08:14:29.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.945+0000 7fa39a800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa394116430 con 0x7fa394075af0 2024-05-27T08:14:29.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.945+0000 7fa39a800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa394116730 con 0x7fa394110bf0 2024-05-27T08:14:29.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.945+0000 7fa392a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa394076440 0x7fa39419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.946+0000 7fa392a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa394076440 0x7fa39419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59016/0 (socket says 172.21.15.63:59016) 2024-05-27T08:14:29.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.946+0000 7fa392a00700 1 -- 172.21.15.63:0/2346319008 learned_addr learned my addr 172.21.15.63:0/2346319008 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:29.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.946+0000 7fa393400700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa394075af0 0x7fa39419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.946+0000 7fa393e00700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa394110bf0 0x7fa3941a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.946+0000 7fa392a00700 1 -- 172.21.15.63:0/2346319008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa394110bf0 msgr2=0x7fa3941a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.946+0000 7fa392a00700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa394110bf0 0x7fa3941a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.946+0000 7fa392a00700 1 -- 172.21.15.63:0/2346319008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa394075af0 msgr2=0x7fa39419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:29.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.946+0000 7fa392a00700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa394075af0 0x7fa39419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:29.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.946+0000 7fa392a00700 1 -- 172.21.15.63:0/2346319008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3941a1ed0 con 0x7fa394076440 2024-05-27T08:14:29.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.947+0000 7fa392a00700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa394076440 0x7fa39419d5a0 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7fa380000c00 tx=0x7fa3800057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:29.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.947+0000 7fa38be00700 1 -- 172.21.15.63:0/2346319008 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380027070 con 0x7fa394076440 2024-05-27T08:14:29.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.947+0000 7fa39a800700 1 -- 172.21.15.63:0/2346319008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3941a2160 con 0x7fa394076440 2024-05-27T08:14:29.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.947+0000 7fa38be00700 1 -- 172.21.15.63:0/2346319008 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa38000fb00 con 0x7fa394076440 2024-05-27T08:14:29.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.947+0000 7fa39a800700 1 -- 172.21.15.63:0/2346319008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3941a9ae0 con 0x7fa394076440 2024-05-27T08:14:29.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.948+0000 7fa38be00700 1 -- 172.21.15.63:0/2346319008 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380022bc0 con 0x7fa394076440 2024-05-27T08:14:29.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.949+0000 7fa39a800700 1 -- 172.21.15.63:0/2346319008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3940512b0 con 0x7fa394076440 2024-05-27T08:14:29.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.952+0000 7fa38be00700 1 -- 172.21.15.63:0/2346319008 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa38002a070 con 0x7fa394076440 2024-05-27T08:14:29.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.954+0000 7fa38be00700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3780b29f0 0x7fa3780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:29.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.954+0000 7fa38be00700 1 -- 172.21.15.63:0/2346319008 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fa3800e3c60 con 0x7fa394076440 2024-05-27T08:14:29.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.954+0000 7fa393400700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3780b29f0 0x7fa3780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:29.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.955+0000 7fa393400700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3780b29f0 0x7fa3780b4f00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa384005950 tx=0x7fa38400b410 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:29.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:29.957+0000 7fa38be00700 1 -- 172.21.15.63:0/2346319008 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa380022e20 con 0x7fa394076440 2024-05-27T08:14:30.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:29 smithi086 ceph-mon[34687]: pgmap v1299: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T08:14:30.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:29 smithi149 ceph-mon[35408]: pgmap v1299: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T08:14:30.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.292+0000 7fa39a800700 1 -- 172.21.15.63:0/2346319008 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa39419e360 con 0x7fa394076440 2024-05-27T08:14:30.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.294+0000 7fa38be00700 1 -- 172.21.15.63:0/2346319008 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7fa380013070 con 0x7fa394076440 2024-05-27T08:14:30.296 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:14:30.296 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:14:30.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.297+0000 7fa39a800700 1 -- 172.21.15.63:0/2346319008 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3780b29f0 msgr2=0x7fa3780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:30.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.297+0000 7fa39a800700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3780b29f0 0x7fa3780b4f00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa384005950 tx=0x7fa38400b410 comp rx=0 tx=0).stop 2024-05-27T08:14:30.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.297+0000 7fa39a800700 1 -- 172.21.15.63:0/2346319008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa394076440 msgr2=0x7fa39419d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:30.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.298+0000 7fa39a800700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa394076440 0x7fa39419d5a0 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7fa380000c00 tx=0x7fa3800057a0 comp rx=0 tx=0).stop 2024-05-27T08:14:30.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.298+0000 7fa39a800700 1 -- 172.21.15.63:0/2346319008 shutdown_connections 2024-05-27T08:14:30.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.298+0000 7fa39a800700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa3780b29f0 0x7fa3780b4f00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:30.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.298+0000 7fa39a800700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa394110bf0 0x7fa3941a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:30.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.298+0000 7fa39a800700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa394075af0 0x7fa39419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:30.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.298+0000 7fa39a800700 1 --2- 172.21.15.63:0/2346319008 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa394076440 0x7fa39419d5a0 unknown :-1 s=CLOSED pgs=2893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:30.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.298+0000 7fa39a800700 1 -- 172.21.15.63:0/2346319008 >> 172.21.15.63:0/2346319008 conn(0x7fa394100030 msgr2=0x7fa394100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:30.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.298+0000 7fa39a800700 1 -- 172.21.15.63:0/2346319008 shutdown_connections 2024-05-27T08:14:30.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:30.298+0000 7fa39a800700 1 -- 172.21.15.63:0/2346319008 wait complete. 2024-05-27T08:14:30.301 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:14:31.179 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:30 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2346319008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:14:31.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:30 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2346319008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:14:31.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:30 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2346319008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:14:31.266 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:14:31.822 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:14:32.180 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:31 smithi063 ceph-mon[31613]: pgmap v1300: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T08:14:32.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:31 smithi086 ceph-mon[34687]: pgmap v1300: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T08:14:32.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:31 smithi149 ceph-mon[35408]: pgmap v1300: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-05-27T08:14:33.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:33 smithi063 ceph-mon[31613]: pgmap v1301: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:14:34.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:33 smithi086 ceph-mon[34687]: pgmap v1301: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:14:34.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:33 smithi149 ceph-mon[35408]: pgmap v1301: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 13 op/s 2024-05-27T08:14:34.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.830+0000 7fee77e00700 1 -- 172.21.15.63:0/3547617886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee78106610 msgr2=0x7fee78106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:34.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.830+0000 7fee77e00700 1 --2- 172.21.15.63:0/3547617886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee78106610 0x7fee78106a20 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7fee680099c0 tx=0x7fee68009cd0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.831+0000 7fee77e00700 1 -- 172.21.15.63:0/3547617886 shutdown_connections 2024-05-27T08:14:34.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.831+0000 7fee77e00700 1 --2- 172.21.15.63:0/3547617886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee7810b890 0x7fee781131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.831+0000 7fee77e00700 1 --2- 172.21.15.63:0/3547617886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee78106610 0x7fee78106a20 unknown :-1 s=CLOSED pgs=2894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.831+0000 7fee77e00700 1 --2- 172.21.15.63:0/3547617886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7810af40 0x7fee7810b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.831+0000 7fee77e00700 1 -- 172.21.15.63:0/3547617886 >> 172.21.15.63:0/3547617886 conn(0x7fee78102020 msgr2=0x7fee78104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:34.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.831+0000 7fee77e00700 1 -- 172.21.15.63:0/3547617886 shutdown_connections 2024-05-27T08:14:34.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.831+0000 7fee77e00700 1 -- 172.21.15.63:0/3547617886 wait complete. 2024-05-27T08:14:34.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.832+0000 7fee77e00700 1 Processor -- start 2024-05-27T08:14:34.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.833+0000 7fee77e00700 1 -- start start 2024-05-27T08:14:34.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.833+0000 7fee77e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee78106610 0x7fee7819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.833+0000 7fee77e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7810af40 0x7fee7819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.833+0000 7fee77e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee7810b890 0x7fee781a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.833+0000 7fee77e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee781163c0 con 0x7fee78106610 2024-05-27T08:14:34.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.833+0000 7fee77e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee78116240 con 0x7fee7810af40 2024-05-27T08:14:34.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.833+0000 7fee77e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee78116540 con 0x7fee7810b890 2024-05-27T08:14:34.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.834+0000 7fee76000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7810af40 0x7fee7819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.834+0000 7fee76a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee78106610 0x7fee7819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.834+0000 7fee76000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7810af40 0x7fee7819d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34538/0 (socket says 172.21.15.63:34538) 2024-05-27T08:14:34.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.834+0000 7fee76000700 1 -- 172.21.15.63:0/1273896491 learned_addr learned my addr 172.21.15.63:0/1273896491 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:34.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.834+0000 7fee77400700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee7810b890 0x7fee781a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.835+0000 7fee76a00700 1 -- 172.21.15.63:0/1273896491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee7810b890 msgr2=0x7fee781a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:34.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.835+0000 7fee76a00700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee7810b890 0x7fee781a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.835+0000 7fee76a00700 1 -- 172.21.15.63:0/1273896491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7810af40 msgr2=0x7fee7819d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:34.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.835+0000 7fee76a00700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7810af40 0x7fee7819d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.835+0000 7fee76a00700 1 -- 172.21.15.63:0/1273896491 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee781a1c80 con 0x7fee78106610 2024-05-27T08:14:34.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.835+0000 7fee76000700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7810af40 0x7fee7819d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:14:34.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.835+0000 7fee76a00700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee78106610 0x7fee7819cdb0 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fee680098e0 tx=0x7fee680057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:34.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.836+0000 7fee6f400700 1 -- 172.21.15.63:0/1273896491 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68027070 con 0x7fee78106610 2024-05-27T08:14:34.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.836+0000 7fee6f400700 1 -- 172.21.15.63:0/1273896491 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee6800fc80 con 0x7fee78106610 2024-05-27T08:14:34.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.836+0000 7fee6f400700 1 -- 172.21.15.63:0/1273896491 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68022d90 con 0x7fee78106610 2024-05-27T08:14:34.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.836+0000 7fee77e00700 1 -- 172.21.15.63:0/1273896491 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee781a1f10 con 0x7fee78106610 2024-05-27T08:14:34.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.836+0000 7fee77e00700 1 -- 172.21.15.63:0/1273896491 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee781a23a0 con 0x7fee78106610 2024-05-27T08:14:34.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.841+0000 7fee6f400700 1 -- 172.21.15.63:0/1273896491 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fee68022590 con 0x7fee78106610 2024-05-27T08:14:34.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.841+0000 7fee8c7f3b80 1 Processor -- start 2024-05-27T08:14:34.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.842+0000 7fee8c7f3b80 1 -- start start 2024-05-27T08:14:34.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.842+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea3320750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.842+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea31a0e40 0x55dea3320c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.842+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea332a2c0 0x55dea332c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.842+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dea324de40 con 0x55dea328c9f0 2024-05-27T08:14:34.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.842+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dea323d9e0 con 0x55dea31a0e40 2024-05-27T08:14:34.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.842+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dea3241a20 con 0x55dea332a2c0 2024-05-27T08:14:34.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee77400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea3320750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee77400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea3320750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59042/0 (socket says 172.21.15.63:59042) 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee77400700 1 -- 172.21.15.63:0/3465307977 learned_addr learned my addr 172.21.15.63:0/3465307977 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee76a00700 1 --2- 172.21.15.63:0/3465307977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea31a0e40 0x55dea3320c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee76000700 1 --2- 172.21.15.63:0/3465307977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea332a2c0 0x55dea332c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee6f400700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee5c0b29f0 0x7fee5c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee77400700 1 -- 172.21.15.63:0/3465307977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea332a2c0 msgr2=0x55dea332c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee77400700 1 --2- 172.21.15.63:0/3465307977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea332a2c0 0x55dea332c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee77400700 1 -- 172.21.15.63:0/3465307977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea31a0e40 msgr2=0x55dea3320c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee77400700 1 --2- 172.21.15.63:0/3465307977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea31a0e40 0x55dea3320c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee77400700 1 -- 172.21.15.63:0/3465307977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea3206420 con 0x55dea328c9f0 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.843+0000 7fee76a00700 1 --2- 172.21.15.63:0/3465307977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea31a0e40 0x55dea3320c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:34.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.844+0000 7fee6f400700 1 -- 172.21.15.63:0/1273896491 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fee680e3550 con 0x7fee78106610 2024-05-27T08:14:34.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.844+0000 7fee77400700 1 --2- 172.21.15.63:0/3465307977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea3320750 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fee70003fa0 tx=0x7fee70009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cc38c17a152b75 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:34.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.844+0000 7fee77400700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee5c0b29f0 0x7fee5c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.845+0000 7fee63e00700 1 -- 172.21.15.63:0/3465307977 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70007500 con 0x55dea328c9f0 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.845+0000 7fee63e00700 1 -- 172.21.15.63:0/3465307977 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee70007bb0 con 0x55dea328c9f0 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.845+0000 7fee63e00700 1 -- 172.21.15.63:0/3465307977 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee700143e0 con 0x55dea328c9f0 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.846+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/3465307977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 msgr2=0x55dea3320750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.846+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/3465307977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea3320750 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fee70003fa0 tx=0x7fee70009ba0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.846+0000 7fee77400700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee5c0b29f0 0x7fee5c0b4f00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fee7001a000 tx=0x7fee700149d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.846+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/3465307977 shutdown_connections 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.846+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/3465307977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea332a2c0 0x55dea332c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.846+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/3465307977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea3320750 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.846+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/3465307977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea31a0e40 0x55dea3320c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.846+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/3465307977 >> 172.21.15.63:0/3465307977 conn(0x55dea323a390 msgr2=0x55dea331b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:34.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.846+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/3465307977 shutdown_connections 2024-05-27T08:14:34.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.847+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/3465307977 wait complete. 2024-05-27T08:14:34.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.847+0000 7fee8c7f3b80 1 -- start start 2024-05-27T08:14:34.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.847+0000 7fee8c7f3b80 1 Processor -- start 2024-05-27T08:14:34.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.848+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea3327250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.848+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 0x55dea3327790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.848+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea3327cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.848+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dea3143fa0 con 0x55dea31a0e40 2024-05-27T08:14:34.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.848+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55dea31de760 con 0x55dea332a2c0 2024-05-27T08:14:34.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.848+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dea332ce00 con 0x55dea328c9f0 2024-05-27T08:14:34.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.849+0000 7fee8c7f3b80 10 client.0 _ll_register_callbacks cb 0x55dea31e7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:34.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.849+0000 7fee76a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea3327250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.849+0000 7fee76a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea3327250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59058/0 (socket says 172.21.15.63:59058) 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.849+0000 7fee76a00700 1 -- 172.21.15.63:0/808352881 learned_addr learned my addr 172.21.15.63:0/808352881 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.849+0000 7fee76000700 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 0x55dea3327790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.849+0000 7fee77400700 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea3327cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.849+0000 7fee76a00700 1 -- 172.21.15.63:0/808352881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 msgr2=0x55dea3327790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.849+0000 7fee76a00700 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 0x55dea3327790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.850+0000 7fee76a00700 1 -- 172.21.15.63:0/808352881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 msgr2=0x55dea3327cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.850+0000 7fee76a00700 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea3327cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.850+0000 7fee76a00700 1 -- 172.21.15.63:0/808352881 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea318d8d0 con 0x55dea31a0e40 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.850+0000 7fee77400700 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea3327cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.850+0000 7fee76a00700 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea3327250 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7fee68009990 tx=0x7fee680b3120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ae4079b75f8adc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:34.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.850+0000 7fee60c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:34.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.855+0000 7fee60c00700 1 -- 172.21.15.63:0/808352881 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee680a63c0 con 0x55dea31a0e40 2024-05-27T08:14:34.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.855+0000 7fee60c00700 1 -- 172.21.15.63:0/808352881 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fee6800fa00 con 0x55dea31a0e40 2024-05-27T08:14:34.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.855+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/808352881 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee70007bb0 con 0x55dea31a0e40 2024-05-27T08:14:34.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.855+0000 7fee60c00700 1 -- 172.21.15.63:0/808352881 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fee680a9c70 con 0x55dea31a0e40 2024-05-27T08:14:34.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.855+0000 7fee8c7f3b80 10 client.23232 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:34.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.855+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/808352881 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dea320e060 con 0x55dea31a0e40 2024-05-27T08:14:34.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.856+0000 7fee60c00700 1 -- 172.21.15.63:0/808352881 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee70007bb0 con 0x55dea31a0e40 2024-05-27T08:14:34.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.856+0000 7fee60c00700 1 -- 172.21.15.63:0/808352881 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fee6802a030 con 0x55dea31a0e40 2024-05-27T08:14:34.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.856+0000 7fee8c7f3b80 10 client.23232 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:34.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.856+0000 7fee8c7f3b80 10 client.23232 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:14:34.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.856+0000 7fee8c7f3b80 20 client.23232 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:34.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.856+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55dea3339bd0 0x55dea333c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.856+0000 7fee8c7f3b80 4 client.23232 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:34.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.856+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/808352881 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dea324fd90 con 0x55dea3339bd0 2024-05-27T08:14:34.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.857+0000 7fee76000700 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55dea3339bd0 0x55dea333c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.858+0000 7fee76000700 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55dea3339bd0 0x55dea333c060 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1438c9546770c652 server_cookie=34ea122912f4f46d in_seq=0 out_seq=0 2024-05-27T08:14:34.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.858+0000 7fee60c00700 10 client.23232 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:14:34.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.860+0000 7fee60c00700 1 -- 172.21.15.63:0/808352881 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dea324fd90 con 0x55dea3339bd0 2024-05-27T08:14:34.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.860+0000 7fee60c00700 10 client.23232 handle_command_reply: tid=0 2024-05-27T08:14:34.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.860+0000 7fee8c7f3b80 1 client.23232 shutdown 2024-05-27T08:14:34.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/808352881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 msgr2=0x55dea3327250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:34.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea3327250 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7fee68009990 tx=0x7fee680b3120 comp rx=0 tx=0).stop 2024-05-27T08:14:34.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/808352881 shutdown_connections 2024-05-27T08:14:34.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea3327cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55dea3339bd0 0x55dea333c060 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea3327250 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/808352881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 0x55dea3327790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/808352881 >> 172.21.15.63:0/808352881 conn(0x55dea323a390 msgr2=0x55dea331b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:34.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee60c00700 0 client.23232 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:14:34.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/808352881 shutdown_connections 2024-05-27T08:14:34.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/808352881 wait complete. 2024-05-27T08:14:34.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.861+0000 7fee8c7f3b80 20 client.23232 trim_cache size 0 max 16384 2024-05-27T08:14:34.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.883+0000 7fee8c7f3b80 1 Processor -- start 2024-05-27T08:14:34.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.884+0000 7fee8c7f3b80 1 -- start start 2024-05-27T08:14:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.884+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea331b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.884+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 0x55dea3335ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.884+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea3341820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.884+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee781b4dd0 con 0x55dea31a0e40 2024-05-27T08:14:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.884+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee781b2660 con 0x55dea332a2c0 2024-05-27T08:14:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.884+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee781b04d0 con 0x55dea328c9f0 2024-05-27T08:14:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.884+0000 7fee76a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea331b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee76a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea331b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59060/0 (socket says 172.21.15.63:59060) 2024-05-27T08:14:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee76a00700 1 -- 172.21.15.63:0/3257332925 learned_addr learned my addr 172.21.15.63:0/3257332925 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee76000700 1 --2- 172.21.15.63:0/3257332925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 0x55dea3335ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee77400700 1 --2- 172.21.15.63:0/3257332925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea3341820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee76a00700 1 -- 172.21.15.63:0/3257332925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 msgr2=0x55dea3335ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:34.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee76a00700 1 --2- 172.21.15.63:0/3257332925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 0x55dea3335ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee76a00700 1 -- 172.21.15.63:0/3257332925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 msgr2=0x55dea3341820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee76a00700 1 --2- 172.21.15.63:0/3257332925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea3341820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee76a00700 1 -- 172.21.15.63:0/3257332925 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea331c0e0 con 0x55dea31a0e40 2024-05-27T08:14:34.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee76000700 1 --2- 172.21.15.63:0/3257332925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 0x55dea3335ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:14:34.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.885+0000 7fee76a00700 1 --2- 172.21.15.63:0/3257332925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea331b640 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7fee680e0ee0 tx=0x7fee680aa680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15257557c7e04ac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:34.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.886+0000 7fee63400700 1 -- 172.21.15.63:0/3257332925 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee680a9e50 con 0x55dea31a0e40 2024-05-27T08:14:34.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.886+0000 7fee63400700 1 -- 172.21.15.63:0/3257332925 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee680aa010 con 0x55dea31a0e40 2024-05-27T08:14:34.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.886+0000 7fee63400700 1 -- 172.21.15.63:0/3257332925 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee680aa2d0 con 0x55dea31a0e40 2024-05-27T08:14:34.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.887+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/3257332925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 msgr2=0x55dea331b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:34.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.887+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/3257332925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea331b640 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7fee680e0ee0 tx=0x7fee680aa680 comp rx=0 tx=0).stop 2024-05-27T08:14:34.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.887+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/3257332925 shutdown_connections 2024-05-27T08:14:34.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.887+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/3257332925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea3341820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.887+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/3257332925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea31a0e40 0x55dea331b640 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.887+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/3257332925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea328c9f0 0x55dea3335ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.887+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/3257332925 >> 172.21.15.63:0/3257332925 conn(0x55dea323a390 msgr2=0x55dea331af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:34.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.887+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/3257332925 shutdown_connections 2024-05-27T08:14:34.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.887+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/3257332925 wait complete. 2024-05-27T08:14:34.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.888+0000 7fee8c7f3b80 1 -- start start 2024-05-27T08:14:34.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.888+0000 7fee8c7f3b80 1 Processor -- start 2024-05-27T08:14:34.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea31a0e40 0x55dea3332810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea332d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee8c7f3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea332d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee781b0b20 con 0x55dea328c9f0 2024-05-27T08:14:34.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee781b1f90 con 0x55dea332a2c0 2024-05-27T08:14:34.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee8c7f3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee781b11f0 con 0x55dea31a0e40 2024-05-27T08:14:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee8c7f3b80 10 client.0 _ll_register_callbacks cb 0x55dea31e7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee76000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea332d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee76000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea332d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59064/0 (socket says 172.21.15.63:59064) 2024-05-27T08:14:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee76000700 1 -- 172.21.15.63:0/52181795 learned_addr learned my addr 172.21.15.63:0/52181795 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee76a00700 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea31a0e40 0x55dea3332810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.889+0000 7fee77400700 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea332d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.890+0000 7fee76000700 1 -- 172.21.15.63:0/52181795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea31a0e40 msgr2=0x55dea3332810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.890+0000 7fee76000700 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea31a0e40 0x55dea3332810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.890+0000 7fee76000700 1 -- 172.21.15.63:0/52181795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 msgr2=0x55dea332d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:34.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.890+0000 7fee76000700 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea332d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.890+0000 7fee76000700 1 -- 172.21.15.63:0/52181795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea332dd70 con 0x55dea328c9f0 2024-05-27T08:14:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.890+0000 7fee76000700 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea332d050 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x55dea331af00 tx=0x7fee64003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303b2dab397a6933 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.890+0000 7fee60c00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.890+0000 7fee60c00700 1 -- 172.21.15.63:0/52181795 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64011850 con 0x55dea328c9f0 2024-05-27T08:14:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.890+0000 7fee60c00700 1 -- 172.21.15.63:0/52181795 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fee640119f0 con 0x55dea328c9f0 2024-05-27T08:14:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.890+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/52181795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee680aa010 con 0x55dea328c9f0 2024-05-27T08:14:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.891+0000 7fee60c00700 1 -- 172.21.15.63:0/52181795 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fee64011bf0 con 0x55dea328c9f0 2024-05-27T08:14:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.891+0000 7fee8c7f3b80 10 client.23238 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:34.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.891+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/52181795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dea33a3620 con 0x55dea328c9f0 2024-05-27T08:14:34.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.891+0000 7fee60c00700 1 -- 172.21.15.63:0/52181795 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee680aa010 con 0x55dea328c9f0 2024-05-27T08:14:34.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.891+0000 7fee60c00700 1 -- 172.21.15.63:0/52181795 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fee640273f0 con 0x55dea328c9f0 2024-05-27T08:14:34.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.891+0000 7fee8c7f3b80 10 client.23238 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:34.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.891+0000 7fee8c7f3b80 10 client.23238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:14:34.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.891+0000 7fee8c7f3b80 20 client.23238 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:34.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.891+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55dea33a3d60 0x55dea33a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:34.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.892+0000 7fee8c7f3b80 4 client.23238 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:14:34.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.892+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/52181795 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dea3249570 con 0x55dea33a3d60 2024-05-27T08:14:34.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.892+0000 7fee76a00700 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55dea33a3d60 0x55dea33a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:34.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.893+0000 7fee60c00700 10 client.23238 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:14:34.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.893+0000 7fee76a00700 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55dea33a3d60 0x55dea33a6180 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39d5de21287e18cf server_cookie=c29f194dabf35106 in_seq=0 out_seq=0 2024-05-27T08:14:34.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee60c00700 1 -- 172.21.15.63:0/52181795 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dea3249570 con 0x55dea33a3d60 2024-05-27T08:14:34.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee60c00700 10 client.23238 handle_command_reply: tid=0 2024-05-27T08:14:34.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee8c7f3b80 1 client.23238 shutdown 2024-05-27T08:14:34.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/52181795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 msgr2=0x55dea332d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea332d050 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x55dea331af00 tx=0x7fee64003960 comp rx=0 tx=0).stop 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/52181795 shutdown_connections 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55dea332a2c0 0x55dea332d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dea31a0e40 0x55dea3332810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55dea33a3d60 0x55dea33a6180 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee8c7f3b80 1 --2- 172.21.15.63:0/52181795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea328c9f0 0x55dea332d050 unknown :-1 s=CLOSED pgs=2899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/52181795 >> 172.21.15.63:0/52181795 conn(0x55dea323a390 msgr2=0x55dea3322240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.894+0000 7fee60c00700 0 client.23238 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.895+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/52181795 shutdown_connections 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.895+0000 7fee8c7f3b80 1 -- 172.21.15.63:0/52181795 wait complete. 2024-05-27T08:14:34.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.895+0000 7fee8c7f3b80 20 client.23238 trim_cache size 0 max 16384 2024-05-27T08:14:34.898 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 -- 172.21.15.63:0/1273896491 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee5c0b29f0 msgr2=0x7fee5c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee5c0b29f0 0x7fee5c0b4f00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fee7001a000 tx=0x7fee700149d0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 -- 172.21.15.63:0/1273896491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee78106610 msgr2=0x7fee7819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee78106610 0x7fee7819cdb0 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fee680098e0 tx=0x7fee680057c0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 -- 172.21.15.63:0/1273896491 shutdown_connections 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fee5c0b29f0 0x7fee5c0b4f00 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee7810b890 0x7fee781a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee78106610 0x7fee7819cdb0 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 --2- 172.21.15.63:0/1273896491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7810af40 0x7fee7819d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 -- 172.21.15.63:0/1273896491 >> 172.21.15.63:0/1273896491 conn(0x7fee78102020 msgr2=0x7fee7810a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:34.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 -- 172.21.15.63:0/1273896491 shutdown_connections 2024-05-27T08:14:34.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:34.896+0000 7fee77e00700 1 -- 172.21.15.63:0/1273896491 wait complete. 2024-05-27T08:14:35.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:34 smithi086 ceph-mon[34687]: pgmap v1302: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:14:35.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:34 smithi149 ceph-mon[35408]: pgmap v1302: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:14:35.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:34 smithi063 ceph-mon[31613]: pgmap v1302: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 11 op/s 2024-05-27T08:14:36.066 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:14:36.602 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:14:38.092 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:37 smithi063 ceph-mon[31613]: pgmap v1303: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T08:14:38.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:37 smithi086 ceph-mon[34687]: pgmap v1303: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T08:14:38.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:37 smithi149 ceph-mon[35408]: pgmap v1303: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.7 MiB/s wr, 14 op/s 2024-05-27T08:14:40.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.019+0000 7f78ee600700 1 -- 172.21.15.63:0/24001900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78e81045e0 msgr2=0x7f78e81049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.019+0000 7f78ee600700 1 --2- 172.21.15.63:0/24001900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78e81045e0 0x7f78e81049f0 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009fc0 tx=0x7f78d800b330 comp rx=0 tx=0).stop 2024-05-27T08:14:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.020+0000 7f78ee600700 1 -- 172.21.15.63:0/24001900 shutdown_connections 2024-05-27T08:14:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.020+0000 7f78ee600700 1 --2- 172.21.15.63:0/24001900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e8109860 0x7f78e81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.020+0000 7f78ee600700 1 --2- 172.21.15.63:0/24001900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78e81045e0 0x7f78e81049f0 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.020+0000 7f78ee600700 1 --2- 172.21.15.63:0/24001900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8108f10 0x7f78e8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.020+0000 7f78ee600700 1 -- 172.21.15.63:0/24001900 >> 172.21.15.63:0/24001900 conn(0x7f78e8100030 msgr2=0x7f78e8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.020+0000 7f78ee600700 1 -- 172.21.15.63:0/24001900 shutdown_connections 2024-05-27T08:14:40.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.020+0000 7f78ee600700 1 -- 172.21.15.63:0/24001900 wait complete. 2024-05-27T08:14:40.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.021+0000 7f78ee600700 1 Processor -- start 2024-05-27T08:14:40.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.022+0000 7f78ee600700 1 -- start start 2024-05-27T08:14:40.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.022+0000 7f78ee600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e81045e0 0x7f78e8075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.023+0000 7f78ee600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78e8108f10 0x7f78e8075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.023+0000 7f78ee600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8109860 0x7f78e8076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.023+0000 7f78ee600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78e8114390 con 0x7f78e8108f10 2024-05-27T08:14:40.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.023+0000 7f78ee600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78e8114210 con 0x7f78e81045e0 2024-05-27T08:14:40.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.023+0000 7f78ee600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f78e8114510 con 0x7f78e8109860 2024-05-27T08:14:40.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.023+0000 7f78e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78e8108f10 0x7f78e8075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.023+0000 7f78e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78e8108f10 0x7f78e8075ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34500/0 (socket says 172.21.15.63:34500) 2024-05-27T08:14:40.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.023+0000 7f78e6a00700 1 -- 172.21.15.63:0/3055602750 learned_addr learned my addr 172.21.15.63:0/3055602750 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:40.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.023+0000 7f78e7400700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e81045e0 0x7f78e8075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.023+0000 7f78e7e00700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8109860 0x7f78e8076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.024+0000 7f78e6a00700 1 -- 172.21.15.63:0/3055602750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8109860 msgr2=0x7f78e8076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:40.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.024+0000 7f78e6a00700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8109860 0x7f78e8076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.024+0000 7f78e6a00700 1 -- 172.21.15.63:0/3055602750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e81045e0 msgr2=0x7f78e8075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:40.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.024+0000 7f78e6a00700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e81045e0 0x7f78e8075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.024+0000 7f78e6a00700 1 -- 172.21.15.63:0/3055602750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e81a9140 con 0x7f78e8108f10 2024-05-27T08:14:40.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.024+0000 7f78e6a00700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78e8108f10 0x7f78e8075ff0 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f78d400e970 tx=0x7f78d400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:40.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.025+0000 7f78dfe00700 1 -- 172.21.15.63:0/3055602750 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d400cbd0 con 0x7f78e8108f10 2024-05-27T08:14:40.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.025+0000 7f78dfe00700 1 -- 172.21.15.63:0/3055602750 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d400cd70 con 0x7f78e8108f10 2024-05-27T08:14:40.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.025+0000 7f78ee600700 1 -- 172.21.15.63:0/3055602750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e81a9430 con 0x7f78e8108f10 2024-05-27T08:14:40.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.025+0000 7f78ee600700 1 -- 172.21.15.63:0/3055602750 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e81a9970 con 0x7f78e8108f10 2024-05-27T08:14:40.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.025+0000 7f78dfe00700 1 -- 172.21.15.63:0/3055602750 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4005570 con 0x7f78e8108f10 2024-05-27T08:14:40.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.031+0000 7f78dfe00700 1 -- 172.21.15.63:0/3055602750 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f78d4021440 con 0x7f78e8108f10 2024-05-27T08:14:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.033+0000 7f78dfe00700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78cc0b29f0 0x7f78cc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.034+0000 7f78dfe00700 1 -- 172.21.15.63:0/3055602750 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f78d40e3b40 con 0x7f78e8108f10 2024-05-27T08:14:40.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.034+0000 7f78e7400700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78cc0b29f0 0x7f78cc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.040+0000 7f78e7400700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78cc0b29f0 0x7f78cc0b4f00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f78d8000f80 tx=0x7f78d8009f10 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:40.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.044+0000 7f78fd06bb80 1 Processor -- start 2024-05-27T08:14:40.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.044+0000 7f78fd06bb80 1 -- start start 2024-05-27T08:14:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.044+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 0x55f3ce43d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.044+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce43dd30 0x55f3ce437870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.045+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce437db0 0x55f3ce4381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.045+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f3ce365e40 con 0x55f3ce43dd30 2024-05-27T08:14:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.045+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3ce3559e0 con 0x55f3ce437db0 2024-05-27T08:14:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.045+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f3ce359a20 con 0x55f3ce2b8e40 2024-05-27T08:14:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.045+0000 7f78e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce43dd30 0x55f3ce437870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.045+0000 7f78e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce43dd30 0x55f3ce437870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34508/0 (socket says 172.21.15.63:34508) 2024-05-27T08:14:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.045+0000 7f78e6a00700 1 -- 172.21.15.63:0/2296710967 learned_addr learned my addr 172.21.15.63:0/2296710967 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.045+0000 7f78e7e00700 1 --2- 172.21.15.63:0/2296710967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce437db0 0x55f3ce4381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.045+0000 7f78e7400700 1 --2- 172.21.15.63:0/2296710967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 0x55f3ce43d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.046+0000 7f78e6a00700 1 -- 172.21.15.63:0/2296710967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 msgr2=0x55f3ce43d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.046+0000 7f78e6a00700 1 --2- 172.21.15.63:0/2296710967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 0x55f3ce43d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.046+0000 7f78e6a00700 1 -- 172.21.15.63:0/2296710967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce437db0 msgr2=0x55f3ce4381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.046+0000 7f78e6a00700 1 --2- 172.21.15.63:0/2296710967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce437db0 0x55f3ce4381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.046+0000 7f78e6a00700 1 -- 172.21.15.63:0/2296710967 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ce31e420 con 0x55f3ce43dd30 2024-05-27T08:14:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.046+0000 7f78e7e00700 1 --2- 172.21.15.63:0/2296710967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce437db0 0x55f3ce4381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:14:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.046+0000 7f78e6a00700 1 --2- 172.21.15.63:0/2296710967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce43dd30 0x55f3ce437870 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7f78d40a7c70 tx=0x7f78d40aa260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95434e8416a4a2ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.046+0000 7f78dcc00700 1 -- 172.21.15.63:0/2296710967 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4005710 con 0x55f3ce43dd30 2024-05-27T08:14:40.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.047+0000 7f78dcc00700 1 -- 172.21.15.63:0/2296710967 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d40bfe50 con 0x55f3ce43dd30 2024-05-27T08:14:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.047+0000 7f78dcc00700 1 -- 172.21.15.63:0/2296710967 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d40aac60 con 0x55f3ce43dd30 2024-05-27T08:14:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.048+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2296710967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce43dd30 msgr2=0x55f3ce437870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.048+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2296710967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce43dd30 0x55f3ce437870 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7f78d40a7c70 tx=0x7f78d40aa260 comp rx=0 tx=0).stop 2024-05-27T08:14:40.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.048+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2296710967 shutdown_connections 2024-05-27T08:14:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.048+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2296710967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce437db0 0x55f3ce4381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.048+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2296710967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 0x55f3ce43d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.048+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2296710967 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce43dd30 0x55f3ce437870 unknown :-1 s=CLOSED pgs=2902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.048+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2296710967 >> 172.21.15.63:0/2296710967 conn(0x55f3ce318fc0 msgr2=0x55f3ce4328d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.048+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2296710967 shutdown_connections 2024-05-27T08:14:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.048+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2296710967 wait complete. 2024-05-27T08:14:40.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.049+0000 7f78fd06bb80 1 -- start start 2024-05-27T08:14:40.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.050+0000 7f78fd06bb80 1 Processor -- start 2024-05-27T08:14:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.050+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce2b8e40 0x55f3ce441550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.050+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce437db0 0x55f3ce441a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.050+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 0x55f3ce441fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.050+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f3ce25bfa0 con 0x55f3ce2b8e40 2024-05-27T08:14:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.050+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3ce2f6760 con 0x55f3ce43dd30 2024-05-27T08:14:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.050+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f3ce444800 con 0x55f3ce437db0 2024-05-27T08:14:40.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.050+0000 7f78fd06bb80 10 client.0 _ll_register_callbacks cb 0x55f3ce2ffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.051+0000 7f78e7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce2b8e40 0x55f3ce441550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.051+0000 7f78e6a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce437db0 0x55f3ce441a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.051+0000 7f78e7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 0x55f3ce441fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.051+0000 7f78e7400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce2b8e40 0x55f3ce441550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34516/0 (socket says 172.21.15.63:34516) 2024-05-27T08:14:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.051+0000 7f78e7400700 1 -- 172.21.15.63:0/4026148501 learned_addr learned my addr 172.21.15.63:0/4026148501 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.052+0000 7f78e7400700 1 -- 172.21.15.63:0/4026148501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce437db0 msgr2=0x55f3ce441a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.052+0000 7f78e7400700 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce437db0 0x55f3ce441a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.052+0000 7f78e7400700 1 -- 172.21.15.63:0/4026148501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 msgr2=0x55f3ce441fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.052+0000 7f78e7400700 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 0x55f3ce441fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.052+0000 7f78e7400700 1 -- 172.21.15.63:0/4026148501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ce2a58d0 con 0x55f3ce2b8e40 2024-05-27T08:14:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.052+0000 7f78e7e00700 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 0x55f3ce441fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:14:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.053+0000 7f78e7400700 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce2b8e40 0x55f3ce441550 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f78d800bcd0 tx=0x7f78d8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a91a36614e8fd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.053+0000 7f78d1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.053+0000 7f78d1600700 1 -- 172.21.15.63:0/4026148501 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8007910 con 0x55f3ce2b8e40 2024-05-27T08:14:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.053+0000 7f78d1600700 1 -- 172.21.15.63:0/4026148501 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f78d8007ab0 con 0x55f3ce2b8e40 2024-05-27T08:14:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.053+0000 7f78fd06bb80 1 -- 172.21.15.63:0/4026148501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d40bfe50 con 0x55f3ce2b8e40 2024-05-27T08:14:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.053+0000 7f78d1600700 1 -- 172.21.15.63:0/4026148501 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f78d803a050 con 0x55f3ce2b8e40 2024-05-27T08:14:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.054+0000 7f78fd06bb80 10 client.23250 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.054+0000 7f78fd06bb80 1 -- 172.21.15.63:0/4026148501 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f3ce326060 con 0x55f3ce2b8e40 2024-05-27T08:14:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.054+0000 7f78d1600700 1 -- 172.21.15.63:0/4026148501 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d40bfe50 con 0x55f3ce2b8e40 2024-05-27T08:14:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.054+0000 7f78d1600700 1 -- 172.21.15.63:0/4026148501 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f78d8011070 con 0x55f3ce2b8e40 2024-05-27T08:14:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.054+0000 7f78fd06bb80 10 client.23250 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.054+0000 7f78fd06bb80 10 client.23250 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:14:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.054+0000 7f78fd06bb80 20 client.23250 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.054+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f3ce451680 0x55f3ce453b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.054+0000 7f78fd06bb80 4 client.23250 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.054+0000 7f78fd06bb80 1 -- 172.21.15.63:0/4026148501 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3ce367d90 con 0x55f3ce451680 2024-05-27T08:14:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.055+0000 7f78e6a00700 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f3ce451680 0x55f3ce453b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.056+0000 7f78e6a00700 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f3ce451680 0x55f3ce453b00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20147cc541c58426 server_cookie=9763527115e80e7e in_seq=0 out_seq=0 2024-05-27T08:14:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.056+0000 7f78d1600700 10 client.23250 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:14:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.058+0000 7f78d1600700 1 -- 172.21.15.63:0/4026148501 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f3ce367d90 con 0x55f3ce451680 2024-05-27T08:14:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.058+0000 7f78d1600700 10 client.23250 handle_command_reply: tid=0 2024-05-27T08:14:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.058+0000 7f78fd06bb80 1 client.23250 shutdown 2024-05-27T08:14:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78fd06bb80 1 -- 172.21.15.63:0/4026148501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce2b8e40 msgr2=0x55f3ce441550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce2b8e40 0x55f3ce441550 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f78d800bcd0 tx=0x7f78d8009920 comp rx=0 tx=0).stop 2024-05-27T08:14:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78fd06bb80 1 -- 172.21.15.63:0/4026148501 shutdown_connections 2024-05-27T08:14:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 0x55f3ce441fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f3ce451680 0x55f3ce453b00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce2b8e40 0x55f3ce441550 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/4026148501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce437db0 0x55f3ce441a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78fd06bb80 1 -- 172.21.15.63:0/4026148501 >> 172.21.15.63:0/4026148501 conn(0x55f3ce318fc0 msgr2=0x55f3ce432580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78d1600700 0 client.23250 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:14:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78fd06bb80 1 -- 172.21.15.63:0/4026148501 shutdown_connections 2024-05-27T08:14:40.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.059+0000 7f78fd06bb80 1 -- 172.21.15.63:0/4026148501 wait complete. 2024-05-27T08:14:40.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.060+0000 7f78fd06bb80 20 client.23250 trim_cache size 0 max 16384 2024-05-27T08:14:40.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.080+0000 7f78fd06bb80 1 Processor -- start 2024-05-27T08:14:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.081+0000 7f78fd06bb80 1 -- start start 2024-05-27T08:14:40.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.081+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce2b8e40 0x55f3ce437870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.081+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce4439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.081+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce43dd30 0x55f3ce4404d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.081+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8011610 con 0x55f3ce437db0 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.081+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78c800eea0 con 0x55f3ce2b8e40 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.081+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f78c800cd10 con 0x55f3ce43dd30 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.081+0000 7f78e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce4439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.082+0000 7f78e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce4439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34528/0 (socket says 172.21.15.63:34528) 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.082+0000 7f78e6a00700 1 -- 172.21.15.63:0/2750354834 learned_addr learned my addr 172.21.15.63:0/2750354834 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.082+0000 7f78e7400700 1 --2- 172.21.15.63:0/2750354834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce2b8e40 0x55f3ce437870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.082+0000 7f78e7e00700 1 --2- 172.21.15.63:0/2750354834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce43dd30 0x55f3ce4404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.082+0000 7f78e6a00700 1 -- 172.21.15.63:0/2750354834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce43dd30 msgr2=0x55f3ce4404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.082+0000 7f78e6a00700 1 --2- 172.21.15.63:0/2750354834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce43dd30 0x55f3ce4404d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.082+0000 7f78e6a00700 1 -- 172.21.15.63:0/2750354834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce2b8e40 msgr2=0x55f3ce437870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.082+0000 7f78e6a00700 1 --2- 172.21.15.63:0/2750354834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce2b8e40 0x55f3ce437870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.082+0000 7f78e6a00700 1 -- 172.21.15.63:0/2750354834 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ce43fae0 con 0x55f3ce437db0 2024-05-27T08:14:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.082+0000 7f78e7400700 1 --2- 172.21.15.63:0/2750354834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce2b8e40 0x55f3ce437870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.083+0000 7f78e6a00700 1 --2- 172.21.15.63:0/2750354834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce4439f0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f78d4018000 tx=0x7f78d4092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bed715593bb04 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.083+0000 7f78d3e00700 1 -- 172.21.15.63:0/2750354834 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d40aaa00 con 0x55f3ce437db0 2024-05-27T08:14:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.083+0000 7f78d3e00700 1 -- 172.21.15.63:0/2750354834 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d40c3070 con 0x55f3ce437db0 2024-05-27T08:14:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.083+0000 7f78d3e00700 1 -- 172.21.15.63:0/2750354834 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d40cd330 con 0x55f3ce437db0 2024-05-27T08:14:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.084+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2750354834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 msgr2=0x55f3ce4439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.084+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2750354834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce4439f0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f78d4018000 tx=0x7f78d4092040 comp rx=0 tx=0).stop 2024-05-27T08:14:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.085+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2750354834 shutdown_connections 2024-05-27T08:14:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.085+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2750354834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce43dd30 0x55f3ce4404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.085+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2750354834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce2b8e40 0x55f3ce437870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.085+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2750354834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce4439f0 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.085+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2750354834 >> 172.21.15.63:0/2750354834 conn(0x55f3ce318fc0 msgr2=0x55f3ce43b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.085+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2750354834 shutdown_connections 2024-05-27T08:14:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.085+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2750354834 wait complete. 2024-05-27T08:14:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.085+0000 7f78fd06bb80 1 -- start start 2024-05-27T08:14:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.086+0000 7f78fd06bb80 1 Processor -- start 2024-05-27T08:14:40.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.086+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 0x55f3ce449ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.086+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce44a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.086+0000 7f78fd06bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 0x55f3ce4582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.086+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78c800d360 con 0x55f3ce437db0 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.086+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78c800e7d0 con 0x55f3ce43dd30 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.086+0000 7f78fd06bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f78c800da30 con 0x55f3ce2b8e40 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78fd06bb80 10 client.0 _ll_register_callbacks cb 0x55f3ce2ffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce44a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e6a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce44a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34544/0 (socket says 172.21.15.63:34544) 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e6a00700 1 -- 172.21.15.63:0/2016882093 learned_addr learned my addr 172.21.15.63:0/2016882093 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e7400700 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 0x55f3ce449ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e7e00700 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 0x55f3ce4582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e6a00700 1 -- 172.21.15.63:0/2016882093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 msgr2=0x55f3ce449ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:40.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e6a00700 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 0x55f3ce449ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e6a00700 1 -- 172.21.15.63:0/2016882093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 msgr2=0x55f3ce4582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:40.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e6a00700 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 0x55f3ce4582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e6a00700 1 -- 172.21.15.63:0/2016882093 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ce458830 con 0x55f3ce437db0 2024-05-27T08:14:40.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.087+0000 7f78e7400700 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 0x55f3ce449ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.088+0000 7f78e6a00700 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce44a000 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f78d40aa990 tx=0x7f78d40ab1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57775105572b3b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.088+0000 7f78d1600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.088+0000 7f78d1600700 1 -- 172.21.15.63:0/2016882093 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4070330 con 0x55f3ce437db0 2024-05-27T08:14:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.088+0000 7f78d1600700 1 -- 172.21.15.63:0/2016882093 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f78d40aac60 con 0x55f3ce437db0 2024-05-27T08:14:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.088+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2016882093 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d40c3070 con 0x55f3ce437db0 2024-05-27T08:14:40.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78d1600700 1 -- 172.21.15.63:0/2016882093 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f78d40aae60 con 0x55f3ce437db0 2024-05-27T08:14:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78fd06bb80 10 client.23256 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2016882093 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f3ce4bb7f0 con 0x55f3ce437db0 2024-05-27T08:14:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78d1600700 1 -- 172.21.15.63:0/2016882093 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d40c3070 con 0x55f3ce437db0 2024-05-27T08:14:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78d1600700 1 -- 172.21.15.63:0/2016882093 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f78d40c6070 con 0x55f3ce437db0 2024-05-27T08:14:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78fd06bb80 10 client.23256 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78fd06bb80 10 client.23256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:14:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78fd06bb80 20 client.23256 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f3ce4bb9c0 0x55f3ce4bdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78fd06bb80 4 client.23256 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:14:40.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.089+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2016882093 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3ce361570 con 0x55f3ce4bb9c0 2024-05-27T08:14:40.092 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:39 smithi063 ceph-mon[31613]: pgmap v1304: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:14:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.090+0000 7f78e7400700 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f3ce4bb9c0 0x55f3ce4bdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.091+0000 7f78d1600700 10 client.23256 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:14:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.091+0000 7f78e7400700 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f3ce4bb9c0 0x55f3ce4bdde0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5bce6fe52d21abd server_cookie=62426503e0e3aa8b in_seq=0 out_seq=0 2024-05-27T08:14:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78d1600700 1 -- 172.21.15.63:0/2016882093 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3ce361570 con 0x55f3ce4bb9c0 2024-05-27T08:14:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78d1600700 10 client.23256 handle_command_reply: tid=0 2024-05-27T08:14:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78fd06bb80 1 client.23256 shutdown 2024-05-27T08:14:40.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2016882093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 msgr2=0x55f3ce44a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce44a000 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f78d40aa990 tx=0x7f78d40ab1b0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2016882093 shutdown_connections 2024-05-27T08:14:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3ce43dd30 0x55f3ce4582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f3ce2b8e40 0x55f3ce449ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55f3ce4bb9c0 0x55f3ce4bdde0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78fd06bb80 1 --2- 172.21.15.63:0/2016882093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f3ce437db0 0x55f3ce44a000 unknown :-1 s=CLOSED pgs=2905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78d1600700 0 client.23256 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:14:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.092+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2016882093 >> 172.21.15.63:0/2016882093 conn(0x55f3ce318fc0 msgr2=0x55f3ce441330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.093+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2016882093 shutdown_connections 2024-05-27T08:14:40.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.093+0000 7f78fd06bb80 1 -- 172.21.15.63:0/2016882093 wait complete. 2024-05-27T08:14:40.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.093+0000 7f78fd06bb80 20 client.23256 trim_cache size 0 max 16384 2024-05-27T08:14:40.095 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.094+0000 7f78ee600700 1 -- 172.21.15.63:0/3055602750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78cc0b29f0 msgr2=0x7f78cc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.094+0000 7f78ee600700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78cc0b29f0 0x7f78cc0b4f00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f78d8000f80 tx=0x7f78d8009f10 comp rx=0 tx=0).stop 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.094+0000 7f78ee600700 1 -- 172.21.15.63:0/3055602750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78e8108f10 msgr2=0x7f78e8075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.094+0000 7f78ee600700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78e8108f10 0x7f78e8075ff0 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f78d400e970 tx=0x7f78d400ec80 comp rx=0 tx=0).stop 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.094+0000 7f78ee600700 1 -- 172.21.15.63:0/3055602750 shutdown_connections 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.094+0000 7f78ee600700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f78cc0b29f0 0x7f78cc0b4f00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.094+0000 7f78ee600700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8109860 0x7f78e8076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.094+0000 7f78ee600700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e81045e0 0x7f78e8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.094+0000 7f78ee600700 1 --2- 172.21.15.63:0/3055602750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78e8108f10 0x7f78e8075ff0 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.094+0000 7f78ee600700 1 -- 172.21.15.63:0/3055602750 >> 172.21.15.63:0/3055602750 conn(0x7f78e8100030 msgr2=0x7f78e8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.095+0000 7f78ee600700 1 -- 172.21.15.63:0/3055602750 shutdown_connections 2024-05-27T08:14:40.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:40.095+0000 7f78ee600700 1 -- 172.21.15.63:0/3055602750 wait complete. 2024-05-27T08:14:40.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:39 smithi086 ceph-mon[34687]: pgmap v1304: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:14:40.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:39 smithi149 ceph-mon[35408]: pgmap v1304: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:14:41.080 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:14:41.633 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:14:42.147 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:41 smithi063 ceph-mon[31613]: pgmap v1305: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:14:42.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:41 smithi086 ceph-mon[34687]: pgmap v1305: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:14:42.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:41 smithi149 ceph-mon[35408]: pgmap v1305: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 10 op/s 2024-05-27T08:14:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.572+0000 7fbeaca00640 1 -- 172.21.15.63:0/363922998 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0031800 con 0x7fbeb010bec0 2024-05-27T08:14:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.573+0000 7fbeb4c00640 1 -- 172.21.15.63:0/363922998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbeb010bec0 msgr2=0x7fbeb0105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.573+0000 7fbeb4c00640 1 --2- 172.21.15.63:0/363922998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbeb010bec0 0x7fbeb0105a60 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009900 tx=0x7fbea002a3f0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.573+0000 7fbeb4c00640 1 -- 172.21.15.63:0/363922998 shutdown_connections 2024-05-27T08:14:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.573+0000 7fbeb4c00640 1 --2- 172.21.15.63:0/363922998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0106610 0x7fbeb01129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.573+0000 7fbeb4c00640 1 --2- 172.21.15.63:0/363922998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb010c360 0x7fbeb0105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.573+0000 7fbeb4c00640 1 --2- 172.21.15.63:0/363922998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbeb010bec0 0x7fbeb0105a60 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.573+0000 7fbeb4c00640 1 -- 172.21.15.63:0/363922998 >> 172.21.15.63:0/363922998 conn(0x7fbeb01016b0 msgr2=0x7fbeb0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.573+0000 7fbeb4c00640 1 -- 172.21.15.63:0/363922998 shutdown_connections 2024-05-27T08:14:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.573+0000 7fbeb4c00640 1 -- 172.21.15.63:0/363922998 wait complete. 2024-05-27T08:14:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.575+0000 7fbeb4c00640 1 Processor -- start 2024-05-27T08:14:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.575+0000 7fbeb4c00640 1 -- start start 2024-05-27T08:14:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.575+0000 7fbeb4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0106610 0x7fbeb01a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.575+0000 7fbeb4c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb010bec0 0x7fbeb01a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.576+0000 7fbeb4c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbeb010c360 0x7fbeb019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.576+0000 7fbeb4c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0115fc0 con 0x7fbeb010c360 2024-05-27T08:14:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.576+0000 7fbeb4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0115e40 con 0x7fbeb0106610 2024-05-27T08:14:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.576+0000 7fbeb4c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0116140 con 0x7fbeb010bec0 2024-05-27T08:14:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.576+0000 7fbeae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbeb010c360 0x7fbeb019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.576+0000 7fbeae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbeb010c360 0x7fbeb019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34554/0 (socket says 172.21.15.63:34554) 2024-05-27T08:14:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.576+0000 7fbeae800640 1 -- 172.21.15.63:0/2963411090 learned_addr learned my addr 172.21.15.63:0/2963411090 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.576+0000 7fbeade00640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0106610 0x7fbeb01a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.577+0000 7fbead400640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb010bec0 0x7fbeb01a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.577+0000 7fbeae800640 1 -- 172.21.15.63:0/2963411090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb010bec0 msgr2=0x7fbeb01a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.577+0000 7fbeae800640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb010bec0 0x7fbeb01a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.577+0000 7fbeae800640 1 -- 172.21.15.63:0/2963411090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0106610 msgr2=0x7fbeb01a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.577+0000 7fbeae800640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0106610 0x7fbeb01a2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.577+0000 7fbeae800640 1 -- 172.21.15.63:0/2963411090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb019cf30 con 0x7fbeb010c360 2024-05-27T08:14:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.578+0000 7fbeae800640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbeb010c360 0x7fbeb019c890 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fbea800eb10 tx=0x7fbea800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.578+0000 7fbea6a00640 1 -- 172.21.15.63:0/2963411090 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8004b20 con 0x7fbeb010c360 2024-05-27T08:14:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.578+0000 7fbea6a00640 1 -- 172.21.15.63:0/2963411090 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbea8004cc0 con 0x7fbeb010c360 2024-05-27T08:14:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.578+0000 7fbeb4c00640 1 -- 172.21.15.63:0/2963411090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb019d220 con 0x7fbeb010c360 2024-05-27T08:14:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.578+0000 7fbea6a00640 1 -- 172.21.15.63:0/2963411090 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8010690 con 0x7fbeb010c360 2024-05-27T08:14:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.578+0000 7fbeb4c00640 1 -- 172.21.15.63:0/2963411090 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb01a9500 con 0x7fbeb010c360 2024-05-27T08:14:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.583+0000 7fbebbba6740 1 Processor -- start 2024-05-27T08:14:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.584+0000 7fbea6a00640 1 -- 172.21.15.63:0/2963411090 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbea8010830 con 0x7fbeb010c360 2024-05-27T08:14:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.584+0000 7fbebbba6740 1 -- start start 2024-05-27T08:14:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.585+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba23138860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.585+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d6d2f0 0x55ba23138da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.585+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d754d0 0x55ba231392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.585+0000 7fbebbba6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ba230a9b90 con 0x55ba22d961b0 2024-05-27T08:14:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.585+0000 7fbebbba6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba23099730 con 0x55ba22d754d0 2024-05-27T08:14:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.585+0000 7fbebbba6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ba2309d770 con 0x55ba22d6d2f0 2024-05-27T08:14:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.585+0000 7fbead400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba23138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.585+0000 7fbead400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba23138860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34566/0 (socket says 172.21.15.63:34566) 2024-05-27T08:14:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.585+0000 7fbead400640 1 -- 172.21.15.63:0/3024070060 learned_addr learned my addr 172.21.15.63:0/3024070060 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.586+0000 7fbeade00640 1 --2- 172.21.15.63:0/3024070060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d6d2f0 0x55ba23138da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.586+0000 7fbead400640 1 --2- 172.21.15.63:0/3024070060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d754d0 0x55ba231392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.586+0000 7fbead400640 1 -- 172.21.15.63:0/3024070060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d6d2f0 msgr2=0x55ba23138da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.586+0000 7fbead400640 1 --2- 172.21.15.63:0/3024070060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d6d2f0 0x55ba23138da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.586+0000 7fbead400640 1 -- 172.21.15.63:0/3024070060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d754d0 msgr2=0x55ba231392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.586+0000 7fbead400640 1 --2- 172.21.15.63:0/3024070060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d754d0 0x55ba231392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.586+0000 7fbead400640 1 -- 172.21.15.63:0/3024070060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba230328b0 con 0x55ba22d961b0 2024-05-27T08:14:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.586+0000 7fbea6a00640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbe940b2950 0x7fbe940b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.586+0000 7fbea6a00640 1 -- 172.21.15.63:0/2963411090 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fbea80e3020 con 0x7fbeb010c360 2024-05-27T08:14:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.587+0000 7fbead400640 1 --2- 172.21.15.63:0/3024070060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba23138860 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0134b0 tx=0x7fbe9c00b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdcc15fecc926a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.587+0000 7fbe9b000640 1 -- 172.21.15.63:0/3024070060 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c019030 con 0x55ba22d961b0 2024-05-27T08:14:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.587+0000 7fbeade00640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbe940b2950 0x7fbe940b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.587+0000 7fbe9b000640 1 -- 172.21.15.63:0/3024070060 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe9c005bf0 con 0x55ba22d961b0 2024-05-27T08:14:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.587+0000 7fbe9b000640 1 -- 172.21.15.63:0/3024070060 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c010460 con 0x55ba22d961b0 2024-05-27T08:14:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.588+0000 7fbebbba6740 1 -- 172.21.15.63:0/3024070060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 msgr2=0x55ba23138860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.588+0000 7fbeade00640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbe940b2950 0x7fbe940b4e60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbea0004770 tx=0x7fbea00058d0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.588+0000 7fbebbba6740 1 --2- 172.21.15.63:0/3024070060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba23138860 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0134b0 tx=0x7fbe9c00b570 comp rx=0 tx=0).stop 2024-05-27T08:14:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.589+0000 7fbebbba6740 1 -- 172.21.15.63:0/3024070060 shutdown_connections 2024-05-27T08:14:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.589+0000 7fbebbba6740 1 --2- 172.21.15.63:0/3024070060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d754d0 0x55ba231392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.589+0000 7fbebbba6740 1 --2- 172.21.15.63:0/3024070060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d6d2f0 0x55ba23138da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.589+0000 7fbebbba6740 1 --2- 172.21.15.63:0/3024070060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba23138860 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.589+0000 7fbebbba6740 1 -- 172.21.15.63:0/3024070060 >> 172.21.15.63:0/3024070060 conn(0x55ba22d8a300 msgr2=0x55ba23135010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.589+0000 7fbebbba6740 1 -- 172.21.15.63:0/3024070060 shutdown_connections 2024-05-27T08:14:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.589+0000 7fbebbba6740 1 -- 172.21.15.63:0/3024070060 wait complete. 2024-05-27T08:14:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.590+0000 7fbebbba6740 1 -- start start 2024-05-27T08:14:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.590+0000 7fbebbba6740 1 Processor -- start 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.591+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d6d2f0 0x55ba23142a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.591+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d754d0 0x55ba23142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.591+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba2314ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.591+0000 7fbebbba6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ba22d7f790 con 0x55ba22d961b0 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.591+0000 7fbebbba6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba23041bc0 con 0x55ba22d754d0 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.591+0000 7fbebbba6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ba23145990 con 0x55ba22d6d2f0 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.591+0000 7fbebbba6740 10 client.0 _ll_register_callbacks cb 0x55ba2301f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.591+0000 7fbeae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba2314ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.591+0000 7fbeae800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba2314ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34572/0 (socket says 172.21.15.63:34572) 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.591+0000 7fbeae800640 1 -- 172.21.15.63:0/2060273480 learned_addr learned my addr 172.21.15.63:0/2060273480 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.592+0000 7fbeade00640 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d6d2f0 0x55ba23142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.592+0000 7fbead400640 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d754d0 0x55ba23142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.592+0000 7fbeae800640 1 -- 172.21.15.63:0/2060273480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d6d2f0 msgr2=0x55ba23142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.592+0000 7fbeae800640 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d6d2f0 0x55ba23142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.592+0000 7fbeae800640 1 -- 172.21.15.63:0/2060273480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d754d0 msgr2=0x55ba23142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.592+0000 7fbeae800640 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d754d0 0x55ba23142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.592+0000 7fbeae800640 1 -- 172.21.15.63:0/2060273480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba22dae210 con 0x55ba22d961b0 2024-05-27T08:14:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.592+0000 7fbeae800640 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba2314ecc0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fbea80a9ed0 tx=0x7fbea80a99c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd4c90cf8679a3c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.592+0000 7fbe8f400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.593+0000 7fbe8f400640 1 -- 172.21.15.63:0/2060273480 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea80a9f40 con 0x55ba22d961b0 2024-05-27T08:14:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.593+0000 7fbe8f400640 1 -- 172.21.15.63:0/2060273480 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fbea80b2b20 con 0x55ba22d961b0 2024-05-27T08:14:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.593+0000 7fbebbba6740 1 -- 172.21.15.63:0/2060273480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ba230abae0 con 0x55ba22d961b0 2024-05-27T08:14:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.593+0000 7fbe8f400640 1 -- 172.21.15.63:0/2060273480 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbea80ae2f0 con 0x55ba22d961b0 2024-05-27T08:14:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.593+0000 7fbebbba6740 10 client.23268 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.593+0000 7fbebbba6740 1 -- 172.21.15.63:0/2060273480 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ba22e7a620 con 0x55ba22d961b0 2024-05-27T08:14:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.594+0000 7fbe8f400640 1 -- 172.21.15.63:0/2060273480 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ba230abae0 con 0x55ba22d961b0 2024-05-27T08:14:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.594+0000 7fbe8f400640 1 -- 172.21.15.63:0/2060273480 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbea80e0c90 con 0x55ba22d961b0 2024-05-27T08:14:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.594+0000 7fbebbba6740 10 client.23268 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.594+0000 7fbebbba6740 10 client.23268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.594+0000 7fbebbba6740 20 client.23268 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.594+0000 7fbebbba6740 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba22d54d80 0x55ba23154c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.594+0000 7fbebbba6740 4 client.23268 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.594+0000 7fbebbba6740 1 -- 172.21.15.63:0/2060273480 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba230a52c0 con 0x55ba22d54d80 2024-05-27T08:14:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.595+0000 7fbeade00640 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba22d54d80 0x55ba23154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.596+0000 7fbeade00640 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba22d54d80 0x55ba23154c80 crc :-1 s=READY pgs=726 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10445c1a2ee43fda server_cookie=d6ba0053bff28107 in_seq=0 out_seq=0 2024-05-27T08:14:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.596+0000 7fbe8f400640 10 client.23268 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbe8f400640 1 -- 172.21.15.63:0/2060273480 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ba230a52c0 con 0x55ba22d54d80 2024-05-27T08:14:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbe8f400640 10 client.23268 handle_command_reply: tid=0 2024-05-27T08:14:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbebbba6740 1 client.23268 shutdown 2024-05-27T08:14:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbebbba6740 1 -- 172.21.15.63:0/2060273480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 msgr2=0x55ba2314ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbebbba6740 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba2314ecc0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fbea80a9ed0 tx=0x7fbea80a99c0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbebbba6740 1 -- 172.21.15.63:0/2060273480 shutdown_connections 2024-05-27T08:14:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbebbba6740 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba22d54d80 0x55ba23154c80 crc :-1 s=READY pgs=726 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbebbba6740 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d961b0 0x55ba2314ecc0 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbebbba6740 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d754d0 0x55ba23142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbebbba6740 1 --2- 172.21.15.63:0/2060273480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d6d2f0 0x55ba23142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbebbba6740 1 -- 172.21.15.63:0/2060273480 >> 172.21.15.63:0/2060273480 conn(0x55ba22d8a300 msgr2=0x55ba23138630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.597+0000 7fbe8f400640 0 client.23268 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.598+0000 7fbebbba6740 1 -- 172.21.15.63:0/2060273480 shutdown_connections 2024-05-27T08:14:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.598+0000 7fbebbba6740 1 -- 172.21.15.63:0/2060273480 wait complete. 2024-05-27T08:14:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.598+0000 7fbebbba6740 20 client.23268 trim_cache size 0 max 16384 2024-05-27T08:14:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.621+0000 7fbebbba6740 1 Processor -- start 2024-05-27T08:14:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.621+0000 7fbebbba6740 1 -- start start 2024-05-27T08:14:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.621+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba2314ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.621+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 0x55ba2314ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.621+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d961b0 0x55ba2315a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.621+0000 7fbebbba6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01b5c80 con 0x55ba22d754d0 2024-05-27T08:14:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.621+0000 7fbebbba6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01b3550 con 0x55ba22d6d2f0 2024-05-27T08:14:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.621+0000 7fbebbba6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01b13c0 con 0x55ba22d961b0 2024-05-27T08:14:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbead400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 0x55ba2314ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbeade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba2314ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbead400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 0x55ba2314ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34586/0 (socket says 172.21.15.63:34586) 2024-05-27T08:14:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbead400640 1 -- 172.21.15.63:0/4134944763 learned_addr learned my addr 172.21.15.63:0/4134944763 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbeae800640 1 --2- 172.21.15.63:0/4134944763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d961b0 0x55ba2315a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbeade00640 1 -- 172.21.15.63:0/4134944763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d961b0 msgr2=0x55ba2315a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbeade00640 1 --2- 172.21.15.63:0/4134944763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d961b0 0x55ba2315a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbeade00640 1 -- 172.21.15.63:0/4134944763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 msgr2=0x55ba2314ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbeade00640 1 --2- 172.21.15.63:0/4134944763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 0x55ba2314ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbeade00640 1 -- 172.21.15.63:0/4134944763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba22d54fb0 con 0x55ba22d6d2f0 2024-05-27T08:14:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.622+0000 7fbead400640 1 --2- 172.21.15.63:0/4134944763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 0x55ba2314ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:14:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.623+0000 7fbeade00640 1 --2- 172.21.15.63:0/4134944763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba2314ea30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009230 tx=0x7fbea003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60cfda3bd8c3333f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.623+0000 7fbe9a600640 1 -- 172.21.15.63:0/4134944763 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0055070 con 0x55ba22d6d2f0 2024-05-27T08:14:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.623+0000 7fbe9a600640 1 -- 172.21.15.63:0/4134944763 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbea00314c0 con 0x55ba22d6d2f0 2024-05-27T08:14:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.623+0000 7fbe9a600640 1 -- 172.21.15.63:0/4134944763 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0035070 con 0x55ba22d6d2f0 2024-05-27T08:14:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.624+0000 7fbebbba6740 1 -- 172.21.15.63:0/4134944763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 msgr2=0x55ba2314ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.624+0000 7fbebbba6740 1 --2- 172.21.15.63:0/4134944763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba2314ea30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009230 tx=0x7fbea003f000 comp rx=0 tx=0).stop 2024-05-27T08:14:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.624+0000 7fbebbba6740 1 -- 172.21.15.63:0/4134944763 shutdown_connections 2024-05-27T08:14:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.624+0000 7fbebbba6740 1 --2- 172.21.15.63:0/4134944763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d961b0 0x55ba2315a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.624+0000 7fbebbba6740 1 --2- 172.21.15.63:0/4134944763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 0x55ba2314ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.624+0000 7fbebbba6740 1 --2- 172.21.15.63:0/4134944763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba2314ea30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.625+0000 7fbebbba6740 1 -- 172.21.15.63:0/4134944763 >> 172.21.15.63:0/4134944763 conn(0x55ba22d8a300 msgr2=0x55ba23144bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.625+0000 7fbebbba6740 1 -- 172.21.15.63:0/4134944763 shutdown_connections 2024-05-27T08:14:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.625+0000 7fbebbba6740 1 -- 172.21.15.63:0/4134944763 wait complete. 2024-05-27T08:14:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.625+0000 7fbebbba6740 1 -- start start 2024-05-27T08:14:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.626+0000 7fbebbba6740 1 Processor -- start 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.626+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba23148110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.626+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 0x55ba23148650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.626+0000 7fbebbba6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d961b0 0x55ba231bb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.626+0000 7fbebbba6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01b1a10 con 0x55ba22d754d0 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.626+0000 7fbebbba6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01b2e80 con 0x55ba22d6d2f0 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.626+0000 7fbebbba6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb01b20e0 con 0x55ba22d961b0 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.626+0000 7fbebbba6740 10 client.0 _ll_register_callbacks cb 0x7fbea8005710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbead400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 0x55ba23148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbeade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba23148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbeade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba23148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:45740/0 (socket says 172.21.15.63:45740) 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbeade00640 1 -- 172.21.15.63:0/548157719 learned_addr learned my addr 172.21.15.63:0/548157719 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbeae800640 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d961b0 0x55ba231bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbeade00640 1 -- 172.21.15.63:0/548157719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d961b0 msgr2=0x55ba231bb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbeade00640 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d961b0 0x55ba231bb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbeade00640 1 -- 172.21.15.63:0/548157719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 msgr2=0x55ba23148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbeade00640 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 0x55ba23148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbeade00640 1 -- 172.21.15.63:0/548157719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba231bb890 con 0x55ba22d6d2f0 2024-05-27T08:14:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbeade00640 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba23148110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbea0006010 tx=0x7fbea004f6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dce1f35e78dd27e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.627+0000 7fbe8f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:14:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.628+0000 7fbe8f400640 1 -- 172.21.15.63:0/548157719 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0055070 con 0x55ba22d6d2f0 2024-05-27T08:14:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.628+0000 7fbe8f400640 1 -- 172.21.15.63:0/548157719 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fbea004f930 con 0x55ba22d6d2f0 2024-05-27T08:14:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.628+0000 7fbebbba6740 1 -- 172.21.15.63:0/548157719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea00314c0 con 0x55ba22d6d2f0 2024-05-27T08:14:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.628+0000 7fbe8f400640 1 -- 172.21.15.63:0/548157719 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbea00035f0 con 0x55ba22d6d2f0 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.628+0000 7fbebbba6740 10 client.42143 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.628+0000 7fbebbba6740 1 -- 172.21.15.63:0/548157719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ba231bcab0 con 0x55ba22d6d2f0 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.629+0000 7fbe8f400640 1 -- 172.21.15.63:0/548157719 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbea00314c0 con 0x55ba22d6d2f0 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.629+0000 7fbe8f400640 1 -- 172.21.15.63:0/548157719 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fbea0043070 con 0x55ba22d6d2f0 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.629+0000 7fbebbba6740 10 client.42143 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.629+0000 7fbebbba6740 10 client.42143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.629+0000 7fbebbba6740 20 client.42143 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.629+0000 7fbebbba6740 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba22d2a1c0 0x55ba231bf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.629+0000 7fbebbba6740 4 client.42143 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.629+0000 7fbebbba6740 1 -- 172.21.15.63:0/548157719 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba2301f4b0 con 0x55ba22d2a1c0 2024-05-27T08:14:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.630+0000 7fbead400640 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba22d2a1c0 0x55ba231bf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.630+0000 7fbead400640 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba22d2a1c0 0x55ba231bf130 crc :-1 s=READY pgs=727 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f842bbcdec788 server_cookie=7af8b3fd50008629 in_seq=0 out_seq=0 2024-05-27T08:14:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.630+0000 7fbe8f400640 10 client.42143 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.653+0000 7fbe8f400640 1 -- 172.21.15.63:0/548157719 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ba2301f4b0 con 0x55ba22d2a1c0 2024-05-27T08:14:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.653+0000 7fbe8f400640 10 client.42143 handle_command_reply: tid=0 2024-05-27T08:14:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.653+0000 7fbebbba6740 1 client.42143 shutdown 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.653+0000 7fbebbba6740 1 -- 172.21.15.63:0/548157719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 msgr2=0x55ba23148110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.653+0000 7fbebbba6740 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba23148110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbea0006010 tx=0x7fbea004f6b0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.653+0000 7fbebbba6740 1 -- 172.21.15.63:0/548157719 shutdown_connections 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.653+0000 7fbebbba6740 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ba22d2a1c0 0x55ba231bf130 crc :-1 s=READY pgs=727 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.653+0000 7fbebbba6740 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba22d961b0 0x55ba231bb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.653+0000 7fbe8f400640 0 client.42143 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.653+0000 7fbebbba6740 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ba22d754d0 0x55ba23148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.654+0000 7fbebbba6740 1 --2- 172.21.15.63:0/548157719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba22d6d2f0 0x55ba23148110 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.654+0000 7fbebbba6740 1 -- 172.21.15.63:0/548157719 >> 172.21.15.63:0/548157719 conn(0x55ba22d8a300 msgr2=0x55ba23142d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.654+0000 7fbebbba6740 1 -- 172.21.15.63:0/548157719 shutdown_connections 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.654+0000 7fbebbba6740 1 -- 172.21.15.63:0/548157719 wait complete. 2024-05-27T08:14:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.654+0000 7fbebbba6740 20 client.42143 trim_cache size 0 max 16384 2024-05-27T08:14:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:14:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:14:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:14:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 -- 172.21.15.63:0/2963411090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbe940b2950 msgr2=0x7fbe940b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbe940b2950 0x7fbe940b4e60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbea0004770 tx=0x7fbea00058d0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 -- 172.21.15.63:0/2963411090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbeb010c360 msgr2=0x7fbeb019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbeb010c360 0x7fbeb019c890 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fbea800eb10 tx=0x7fbea800c2d0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 -- 172.21.15.63:0/2963411090 shutdown_connections 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbe940b2950 0x7fbe940b4e60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbeb010c360 0x7fbeb019c890 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb010bec0 0x7fbeb01a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 --2- 172.21.15.63:0/2963411090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb0106610 0x7fbeb01a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 -- 172.21.15.63:0/2963411090 >> 172.21.15.63:0/2963411090 conn(0x7fbeb01016b0 msgr2=0x7fbeb0101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 -- 172.21.15.63:0/2963411090 shutdown_connections 2024-05-27T08:14:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:42.655+0000 7fbeb4c00640 1 -- 172.21.15.63:0/2963411090 wait complete. 2024-05-27T08:14:43.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:42 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:14:43.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:42 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:14:43.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:42 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:14:43.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:43 smithi063 ceph-mon[31613]: pgmap v1306: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T08:14:44.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:43 smithi086 ceph-mon[34687]: pgmap v1306: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T08:14:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:43 smithi149 ceph-mon[35408]: pgmap v1306: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.4 MiB/s wr, 14 op/s 2024-05-27T08:14:44.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.453+0000 7ff997200700 1 -- 172.21.15.63:0/2116564115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff99810e6f0 msgr2=0x7ff998110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:44.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.453+0000 7ff997200700 1 --2- 172.21.15.63:0/2116564115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff99810e6f0 0x7ff998110bd0 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7ff99000b0a0 tx=0x7ff99000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.453+0000 7ff997200700 1 -- 172.21.15.63:0/2116564115 shutdown_connections 2024-05-27T08:14:44.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.453+0000 7ff997200700 1 --2- 172.21.15.63:0/2116564115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff99810e6f0 0x7ff998110bd0 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.453+0000 7ff997200700 1 --2- 172.21.15.63:0/2116564115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff998105280 0x7ff998105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.453+0000 7ff997200700 1 --2- 172.21.15.63:0/2116564115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff998105bd0 0x7ff99810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.453+0000 7ff997200700 1 -- 172.21.15.63:0/2116564115 >> 172.21.15.63:0/2116564115 conn(0x7ff998078ab0 msgr2=0x7ff998078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:44.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.454+0000 7ff997200700 1 -- 172.21.15.63:0/2116564115 shutdown_connections 2024-05-27T08:14:44.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.454+0000 7ff997200700 1 -- 172.21.15.63:0/2116564115 wait complete. 2024-05-27T08:14:44.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.455+0000 7ff997200700 1 Processor -- start 2024-05-27T08:14:44.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.455+0000 7ff997200700 1 -- start start 2024-05-27T08:14:44.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.456+0000 7ff997200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff998105280 0x7ff998102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.456+0000 7ff997200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff998105bd0 0x7ff9981034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.456+0000 7ff997200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99810e6f0 0x7ff9981039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.456+0000 7ff997200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff998114100 con 0x7ff998105bd0 2024-05-27T08:14:44.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.456+0000 7ff997200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff998113f80 con 0x7ff99810e6f0 2024-05-27T08:14:44.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.456+0000 7ff997200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff998114280 con 0x7ff998105280 2024-05-27T08:14:44.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.456+0000 7ff995400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff998105bd0 0x7ff9981034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.457+0000 7ff995e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff998105280 0x7ff998102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.457+0000 7ff995e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff998105280 0x7ff998102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45946/0 (socket says 172.21.15.63:45946) 2024-05-27T08:14:44.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.457+0000 7ff995e00700 1 -- 172.21.15.63:0/3756725896 learned_addr learned my addr 172.21.15.63:0/3756725896 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:44.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.457+0000 7ff996800700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99810e6f0 0x7ff9981039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.457+0000 7ff995400700 1 -- 172.21.15.63:0/3756725896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff998105280 msgr2=0x7ff998102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.457+0000 7ff995400700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff998105280 0x7ff998102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.458+0000 7ff995400700 1 -- 172.21.15.63:0/3756725896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99810e6f0 msgr2=0x7ff9981039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.458+0000 7ff995400700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99810e6f0 0x7ff9981039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.458+0000 7ff995400700 1 -- 172.21.15.63:0/3756725896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9981a8e70 con 0x7ff998105bd0 2024-05-27T08:14:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.458+0000 7ff995400700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff998105bd0 0x7ff9981034b0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7ff98400e9f0 tx=0x7ff98400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.458+0000 7ff98ea00700 1 -- 172.21.15.63:0/3756725896 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98400cbd0 con 0x7ff998105bd0 2024-05-27T08:14:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.458+0000 7ff997200700 1 -- 172.21.15.63:0/3756725896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9981a9160 con 0x7ff998105bd0 2024-05-27T08:14:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.458+0000 7ff98ea00700 1 -- 172.21.15.63:0/3756725896 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff98400cd70 con 0x7ff998105bd0 2024-05-27T08:14:44.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.458+0000 7ff98ea00700 1 -- 172.21.15.63:0/3756725896 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984018680 con 0x7ff998105bd0 2024-05-27T08:14:44.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.459+0000 7ff997200700 1 -- 172.21.15.63:0/3756725896 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9981a96a0 con 0x7ff998105bd0 2024-05-27T08:14:44.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.464+0000 7ff9ab4ffb80 1 Processor -- start 2024-05-27T08:14:44.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.464+0000 7ff9ab4ffb80 1 -- start start 2024-05-27T08:14:44.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.465+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a1364bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.465+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1365510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.465+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a137a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.465+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c0a1293e40 con 0x55c0a11e6e40 2024-05-27T08:14:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.465+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0a12839e0 con 0x55c0a13782e0 2024-05-27T08:14:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.465+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c0a1287a20 con 0x55c0a1365100 2024-05-27T08:14:44.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.465+0000 7ff98ea00700 1 -- 172.21.15.63:0/3756725896 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff984018820 con 0x7ff998105bd0 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.465+0000 7ff995e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1365510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.466+0000 7ff996800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a1364bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.466+0000 7ff995e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1365510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45962/0 (socket says 172.21.15.63:45962) 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.466+0000 7ff995e00700 1 -- 172.21.15.63:0/3701818151 learned_addr learned my addr 172.21.15.63:0/3701818151 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.466+0000 7ff995400700 1 --2- 172.21.15.63:0/3701818151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a137a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.466+0000 7ff995e00700 1 -- 172.21.15.63:0/3701818151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 msgr2=0x55c0a137a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.466+0000 7ff995e00700 1 --2- 172.21.15.63:0/3701818151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a137a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.466+0000 7ff995e00700 1 -- 172.21.15.63:0/3701818151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 msgr2=0x55c0a1364bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.466+0000 7ff995e00700 1 --2- 172.21.15.63:0/3701818151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a1364bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.466+0000 7ff995e00700 1 -- 172.21.15.63:0/3701818151 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a124c420 con 0x55c0a1365100 2024-05-27T08:14:44.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.466+0000 7ff996800700 1 --2- 172.21.15.63:0/3701818151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a1364bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:14:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.467+0000 7ff995e00700 1 --2- 172.21.15.63:0/3701818151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1365510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff98800bea0 tx=0x7ff9880106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a0fa4a289c51adb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.467+0000 7ff983400700 1 -- 172.21.15.63:0/3701818151 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988011440 con 0x55c0a1365100 2024-05-27T08:14:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.467+0000 7ff983400700 1 -- 172.21.15.63:0/3701818151 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff988011a70 con 0x55c0a1365100 2024-05-27T08:14:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.467+0000 7ff983400700 1 -- 172.21.15.63:0/3701818151 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988019990 con 0x55c0a1365100 2024-05-27T08:14:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.468+0000 7ff98ea00700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff97c0b2a70 0x7ff97c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.468+0000 7ff98ea00700 1 -- 172.21.15.63:0/3756725896 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7ff9840e2b40 con 0x7ff998105bd0 2024-05-27T08:14:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.468+0000 7ff996800700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff97c0b2a70 0x7ff97c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.468+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/3701818151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 msgr2=0x55c0a1365510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:44.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.468+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/3701818151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1365510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff98800bea0 tx=0x7ff9880106b0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.469+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/3701818151 shutdown_connections 2024-05-27T08:14:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.469+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/3701818151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a137a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.469+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/3701818151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a1364bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.469+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/3701818151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1365510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.469+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/3701818151 >> 172.21.15.63:0/3701818151 conn(0x55c0a1246fc0 msgr2=0x55c0a1360d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.469+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/3701818151 shutdown_connections 2024-05-27T08:14:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.469+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/3701818151 wait complete. 2024-05-27T08:14:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.469+0000 7ff996800700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff97c0b2a70 0x7ff97c0b4f80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff9900062a0 tx=0x7ff9900061f0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.470+0000 7ff9ab4ffb80 1 -- start start 2024-05-27T08:14:44.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.470+0000 7ff9ab4ffb80 1 Processor -- start 2024-05-27T08:14:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1375810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1375d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c0a1189fa0 con 0x55c0a11e6e40 2024-05-27T08:14:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0a1224760 con 0x55c0a13782e0 2024-05-27T08:14:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c0a137b150 con 0x55c0a1365100 2024-05-27T08:14:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff9ab4ffb80 10 client.0 _ll_register_callbacks cb 0x55c0a122dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:44.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff995e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff995e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13752d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34616/0 (socket says 172.21.15.63:34616) 2024-05-27T08:14:44.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff995e00700 1 -- 172.21.15.63:0/240931737 learned_addr learned my addr 172.21.15.63:0/240931737 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:44.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.471+0000 7ff995400700 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1375810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.472+0000 7ff996800700 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1375d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.472+0000 7ff995e00700 1 -- 172.21.15.63:0/240931737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 msgr2=0x55c0a1375810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.472+0000 7ff995e00700 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1375810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.472+0000 7ff995e00700 1 -- 172.21.15.63:0/240931737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 msgr2=0x55c0a1375d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.472+0000 7ff995e00700 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1375d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.472+0000 7ff995e00700 1 -- 172.21.15.63:0/240931737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a11d38d0 con 0x55c0a11e6e40 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.472+0000 7ff996800700 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1375d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.473+0000 7ff995e00700 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13752d0 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7ff988000c00 tx=0x7ff988010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f23ea1da13675a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.473+0000 7ff977e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.473+0000 7ff977e00700 1 -- 172.21.15.63:0/240931737 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988027030 con 0x55c0a11e6e40 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.473+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/240931737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff988011a70 con 0x55c0a11e6e40 2024-05-27T08:14:44.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.473+0000 7ff977e00700 1 -- 172.21.15.63:0/240931737 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7ff9880114c0 con 0x55c0a11e6e40 2024-05-27T08:14:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.473+0000 7ff977e00700 1 -- 172.21.15.63:0/240931737 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff98801f040 con 0x55c0a11e6e40 2024-05-27T08:14:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.473+0000 7ff9ab4ffb80 10 client.23286 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.473+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/240931737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c0a1254060 con 0x55c0a11e6e40 2024-05-27T08:14:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.474+0000 7ff977e00700 1 -- 172.21.15.63:0/240931737 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff988011a70 con 0x55c0a11e6e40 2024-05-27T08:14:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.474+0000 7ff977e00700 1 -- 172.21.15.63:0/240931737 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff988022030 con 0x55c0a11e6e40 2024-05-27T08:14:44.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.474+0000 7ff9ab4ffb80 10 client.23286 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.474+0000 7ff9ab4ffb80 10 client.23286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.474+0000 7ff9ab4ffb80 20 client.23286 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.474+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c0a1388060 0x55c0a138a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.474+0000 7ff9ab4ffb80 4 client.23286 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.474+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/240931737 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0a1295d90 con 0x55c0a1388060 2024-05-27T08:14:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.475+0000 7ff995400700 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c0a1388060 0x55c0a138a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.476+0000 7ff995400700 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c0a1388060 0x55c0a138a480 crc :-1 s=READY pgs=728 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e542239d21029c server_cookie=9eab909ba89a0f65 in_seq=0 out_seq=0 2024-05-27T08:14:44.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.476+0000 7ff977e00700 10 client.23286 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:44.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.478+0000 7ff977e00700 1 -- 172.21.15.63:0/240931737 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c0a1295d90 con 0x55c0a1388060 2024-05-27T08:14:44.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.478+0000 7ff977e00700 10 client.23286 handle_command_reply: tid=0 2024-05-27T08:14:44.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.478+0000 7ff9ab4ffb80 1 client.23286 shutdown 2024-05-27T08:14:44.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.478+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/240931737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 msgr2=0x55c0a13752d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:44.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.478+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13752d0 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7ff988000c00 tx=0x7ff988010d80 comp rx=0 tx=0).stop 2024-05-27T08:14:44.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.478+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/240931737 shutdown_connections 2024-05-27T08:14:44.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.478+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c0a1388060 0x55c0a138a480 crc :-1 s=READY pgs=728 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.478+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1375d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.478+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13752d0 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.478+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/240931737 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1375810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.479+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/240931737 >> 172.21.15.63:0/240931737 conn(0x55c0a1246fc0 msgr2=0x55c0a135fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:44.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.479+0000 7ff977e00700 0 client.23286 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:44.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.479+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/240931737 shutdown_connections 2024-05-27T08:14:44.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.479+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/240931737 wait complete. 2024-05-27T08:14:44.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.479+0000 7ff9ab4ffb80 20 client.23286 trim_cache size 0 max 16384 2024-05-27T08:14:44.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.504+0000 7ff9ab4ffb80 1 Processor -- start 2024-05-27T08:14:44.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.504+0000 7ff9ab4ffb80 1 -- start start 2024-05-27T08:14:44.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1377770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1384420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981b0a20 con 0x55c0a11e6e40 2024-05-27T08:14:44.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981ae2b0 con 0x55c0a13782e0 2024-05-27T08:14:44.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981ac120 con 0x55c0a1365100 2024-05-27T08:14:44.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff995e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff995e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13745f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34626/0 (socket says 172.21.15.63:34626) 2024-05-27T08:14:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff995e00700 1 -- 172.21.15.63:0/2018687930 learned_addr learned my addr 172.21.15.63:0/2018687930 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff995400700 1 --2- 172.21.15.63:0/2018687930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1377770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.505+0000 7ff996800700 1 --2- 172.21.15.63:0/2018687930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1384420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.506+0000 7ff995e00700 1 -- 172.21.15.63:0/2018687930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 msgr2=0x55c0a1377770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.506+0000 7ff995e00700 1 --2- 172.21.15.63:0/2018687930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1377770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.506+0000 7ff995e00700 1 -- 172.21.15.63:0/2018687930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 msgr2=0x55c0a1384420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.506+0000 7ff995e00700 1 --2- 172.21.15.63:0/2018687930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1384420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.506+0000 7ff995e00700 1 -- 172.21.15.63:0/2018687930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a135a130 con 0x55c0a11e6e40 2024-05-27T08:14:44.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.506+0000 7ff995e00700 1 --2- 172.21.15.63:0/2018687930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13745f0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7ff988004910 tx=0x7ff988010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4453a6e68ebe37c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:44.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.507+0000 7ff982a00700 1 -- 172.21.15.63:0/2018687930 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98803a070 con 0x55c0a11e6e40 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.507+0000 7ff982a00700 1 -- 172.21.15.63:0/2018687930 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff98800b500 con 0x55c0a11e6e40 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.507+0000 7ff982a00700 1 -- 172.21.15.63:0/2018687930 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988027040 con 0x55c0a11e6e40 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.508+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2018687930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 msgr2=0x55c0a13745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.508+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/2018687930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13745f0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7ff988004910 tx=0x7ff988010f20 comp rx=0 tx=0).stop 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.508+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2018687930 shutdown_connections 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.508+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/2018687930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1384420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.508+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/2018687930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a11e6e40 0x55c0a13745f0 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.508+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/2018687930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a1365100 0x55c0a1377770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.508+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2018687930 >> 172.21.15.63:0/2018687930 conn(0x55c0a1246fc0 msgr2=0x55c0a13708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.508+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2018687930 shutdown_connections 2024-05-27T08:14:44.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.508+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2018687930 wait complete. 2024-05-27T08:14:44.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.509+0000 7ff9ab4ffb80 1 -- start start 2024-05-27T08:14:44.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.509+0000 7ff9ab4ffb80 1 Processor -- start 2024-05-27T08:14:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a11e6e40 0x55c0a13880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a1365100 0x55c0a1388600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff9ab4ffb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1388b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981ac770 con 0x55c0a1365100 2024-05-27T08:14:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981adbe0 con 0x55c0a13782e0 2024-05-27T08:14:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff9ab4ffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff9981ace40 con 0x55c0a11e6e40 2024-05-27T08:14:44.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff9ab4ffb80 10 client.0 _ll_register_callbacks cb 0x55c0a122dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:44.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff995400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a1365100 0x55c0a1388600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff995e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a11e6e40 0x55c0a13880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff995400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a1365100 0x55c0a1388600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34632/0 (socket says 172.21.15.63:34632) 2024-05-27T08:14:44.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.510+0000 7ff995400700 1 -- 172.21.15.63:0/2679338131 learned_addr learned my addr 172.21.15.63:0/2679338131 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:44.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.511+0000 7ff995400700 1 -- 172.21.15.63:0/2679338131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a11e6e40 msgr2=0x55c0a13880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:44.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.511+0000 7ff995400700 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a11e6e40 0x55c0a13880c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.511+0000 7ff995400700 1 -- 172.21.15.63:0/2679338131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 msgr2=0x55c0a1388b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:44.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.511+0000 7ff996800700 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1388b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.511+0000 7ff995400700 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1388b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.511+0000 7ff995400700 1 -- 172.21.15.63:0/2679338131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a13f0af0 con 0x55c0a1365100 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.511+0000 7ff996800700 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1388b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.512+0000 7ff995400700 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a1365100 0x55c0a1388600 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7ff984000c00 tx=0x7ff9840e62b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8f445f2567031b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.512+0000 7ff977e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.512+0000 7ff977e00700 1 -- 172.21.15.63:0/2679338131 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9840a9170 con 0x55c0a1365100 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.512+0000 7ff977e00700 1 -- 172.21.15.63:0/2679338131 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7ff9840b25e0 con 0x55c0a1365100 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.512+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2679338131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98800b500 con 0x55c0a1365100 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.512+0000 7ff977e00700 1 -- 172.21.15.63:0/2679338131 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff9840a58c0 con 0x55c0a1365100 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.512+0000 7ff9ab4ffb80 10 client.23292 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.512+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2679338131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c0a13f1db0 con 0x55c0a1365100 2024-05-27T08:14:44.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.513+0000 7ff977e00700 1 -- 172.21.15.63:0/2679338131 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9840bee50 con 0x55c0a1365100 2024-05-27T08:14:44.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.513+0000 7ff977e00700 1 -- 172.21.15.63:0/2679338131 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ff9840cc0b0 con 0x55c0a1365100 2024-05-27T08:14:44.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.513+0000 7ff9ab4ffb80 10 client.23292 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:44.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.513+0000 7ff9ab4ffb80 10 client.23292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:44.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.513+0000 7ff9ab4ffb80 20 client.23292 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:44.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.513+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c0a13f2470 0x55c0a13f4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:44.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.513+0000 7ff9ab4ffb80 4 client.23292 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:14:44.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.513+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2679338131 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0a128f570 con 0x55c0a13f2470 2024-05-27T08:14:44.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.514+0000 7ff995e00700 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c0a13f2470 0x55c0a13f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:44.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.515+0000 7ff995e00700 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c0a13f2470 0x55c0a13f4890 crc :-1 s=READY pgs=729 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3c730b9659de2b server_cookie=ee819d3144eb49f2 in_seq=0 out_seq=0 2024-05-27T08:14:44.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.515+0000 7ff977e00700 10 client.23292 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:44.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.516+0000 7ff977e00700 1 -- 172.21.15.63:0/2679338131 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0a128f570 con 0x55c0a13f2470 2024-05-27T08:14:44.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.516+0000 7ff977e00700 10 client.23292 handle_command_reply: tid=0 2024-05-27T08:14:44.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.516+0000 7ff9ab4ffb80 1 client.23292 shutdown 2024-05-27T08:14:44.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.516+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2679338131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a1365100 msgr2=0x55c0a1388600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:44.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.516+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a1365100 0x55c0a1388600 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7ff984000c00 tx=0x7ff9840e62b0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.516+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2679338131 shutdown_connections 2024-05-27T08:14:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.517+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55c0a13f2470 0x55c0a13f4890 crc :-1 s=READY pgs=729 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.517+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0a13782e0 0x55c0a1388b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.517+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c0a11e6e40 0x55c0a13880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.517+0000 7ff9ab4ffb80 1 --2- 172.21.15.63:0/2679338131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c0a1365100 0x55c0a1388600 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.517+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2679338131 >> 172.21.15.63:0/2679338131 conn(0x55c0a1246fc0 msgr2=0x55c0a13668e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.517+0000 7ff977e00700 0 client.23292 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.517+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2679338131 shutdown_connections 2024-05-27T08:14:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.517+0000 7ff9ab4ffb80 1 -- 172.21.15.63:0/2679338131 wait complete. 2024-05-27T08:14:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.517+0000 7ff9ab4ffb80 20 client.23292 trim_cache size 0 max 16384 2024-05-27T08:14:44.520 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:14:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.518+0000 7ff997200700 1 -- 172.21.15.63:0/3756725896 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff97c0b2a70 msgr2=0x7ff97c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.518+0000 7ff997200700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff97c0b2a70 0x7ff97c0b4f80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff9900062a0 tx=0x7ff9900061f0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.518+0000 7ff997200700 1 -- 172.21.15.63:0/3756725896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff998105bd0 msgr2=0x7ff9981034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.518+0000 7ff997200700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff998105bd0 0x7ff9981034b0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7ff98400e9f0 tx=0x7ff98400ed00 comp rx=0 tx=0).stop 2024-05-27T08:14:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.519+0000 7ff997200700 1 -- 172.21.15.63:0/3756725896 shutdown_connections 2024-05-27T08:14:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.519+0000 7ff997200700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff97c0b2a70 0x7ff97c0b4f80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.519+0000 7ff997200700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99810e6f0 0x7ff9981039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.519+0000 7ff997200700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff998105280 0x7ff998102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.519+0000 7ff997200700 1 --2- 172.21.15.63:0/3756725896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff998105bd0 0x7ff9981034b0 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.519+0000 7ff997200700 1 -- 172.21.15.63:0/3756725896 >> 172.21.15.63:0/3756725896 conn(0x7ff998078ab0 msgr2=0x7ff9980fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.519+0000 7ff997200700 1 -- 172.21.15.63:0/3756725896 shutdown_connections 2024-05-27T08:14:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:44.519+0000 7ff997200700 1 -- 172.21.15.63:0/3756725896 wait complete. 2024-05-27T08:14:45.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:14:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:14:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:14:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:44 smithi086 ceph-mon[34687]: pgmap v1307: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 7 op/s 2024-05-27T08:14:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:14:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:14:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:14:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:44 smithi149 ceph-mon[35408]: pgmap v1307: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 7 op/s 2024-05-27T08:14:45.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:14:45.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:14:45.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:14:45.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:44 smithi063 ceph-mon[31613]: pgmap v1307: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.0 MiB/s wr, 7 op/s 2024-05-27T08:14:46.406 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:14:46.406 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:14:46.934 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:14:48.114 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:47 smithi063 ceph-mon[31613]: pgmap v1308: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.4 MiB/s wr, 15 op/s 2024-05-27T08:14:48.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:47 smithi086 ceph-mon[34687]: pgmap v1308: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.4 MiB/s wr, 15 op/s 2024-05-27T08:14:48.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:47 smithi149 ceph-mon[35408]: pgmap v1308: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.4 MiB/s wr, 15 op/s 2024-05-27T08:14:49.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.786+0000 7f0bd3e00700 1 -- 172.21.15.63:0/226526876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bd4108550 msgr2=0x7f0bd410feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:49.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.786+0000 7f0bd3e00700 1 --2- 172.21.15.63:0/226526876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bd4108550 0x7f0bd410feb0 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00b0d0 tx=0x7f0bcc00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.786+0000 7f0bd3e00700 1 -- 172.21.15.63:0/226526876 shutdown_connections 2024-05-27T08:14:49.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.786+0000 7f0bd3e00700 1 --2- 172.21.15.63:0/226526876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bd4108550 0x7f0bd410feb0 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.786+0000 7f0bd3e00700 1 --2- 172.21.15.63:0/226526876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd41032d0 0x7f0bd41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.786+0000 7f0bd3e00700 1 --2- 172.21.15.63:0/226526876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd4107c00 0x7f0bd4108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.786+0000 7f0bd3e00700 1 -- 172.21.15.63:0/226526876 >> 172.21.15.63:0/226526876 conn(0x7f0bd4078ab0 msgr2=0x7f0bd4078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:49.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.786+0000 7f0bd3e00700 1 -- 172.21.15.63:0/226526876 shutdown_connections 2024-05-27T08:14:49.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.786+0000 7f0bd3e00700 1 -- 172.21.15.63:0/226526876 wait complete. 2024-05-27T08:14:49.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.788+0000 7f0bd3e00700 1 Processor -- start 2024-05-27T08:14:49.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.788+0000 7f0bd3e00700 1 -- start start 2024-05-27T08:14:49.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.788+0000 7f0bd3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd41032d0 0x7f0bd41a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.788+0000 7f0bd3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd4107c00 0x7f0bd41a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.788+0000 7f0bd3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bd4108550 0x7f0bd419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.788+0000 7f0bd3e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41141d0 con 0x7f0bd4108550 2024-05-27T08:14:49.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.788+0000 7f0bd3e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4114050 con 0x7f0bd4107c00 2024-05-27T08:14:49.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.788+0000 7f0bd3e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4114350 con 0x7f0bd41032d0 2024-05-27T08:14:49.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.789+0000 7f0bd3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bd4108550 0x7f0bd419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.789+0000 7f0bd3400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bd4108550 0x7f0bd419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42472/0 (socket says 172.21.15.63:42472) 2024-05-27T08:14:49.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.789+0000 7f0bd3400700 1 -- 172.21.15.63:0/1271702044 learned_addr learned my addr 172.21.15.63:0/1271702044 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:49.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.789+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd41032d0 0x7f0bd41a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.789+0000 7f0bd2000700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd4107c00 0x7f0bd41a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.790+0000 7f0bd3400700 1 -- 172.21.15.63:0/1271702044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd41032d0 msgr2=0x7f0bd41a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.790+0000 7f0bd3400700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd41032d0 0x7f0bd41a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.790+0000 7f0bd3400700 1 -- 172.21.15.63:0/1271702044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd4107c00 msgr2=0x7f0bd41a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.790+0000 7f0bd3400700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd4107c00 0x7f0bd41a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.790+0000 7f0bd3400700 1 -- 172.21.15.63:0/1271702044 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd419b040 con 0x7f0bd4108550 2024-05-27T08:14:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.790+0000 7f0bd3400700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bd4108550 0x7f0bd419a990 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc007d10 tx=0x7f0bcc007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.791+0000 7f0bcb400700 1 -- 172.21.15.63:0/1271702044 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc01f070 con 0x7f0bd4108550 2024-05-27T08:14:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.791+0000 7f0bd3e00700 1 -- 172.21.15.63:0/1271702044 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd419b2d0 con 0x7f0bd4108550 2024-05-27T08:14:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.791+0000 7f0bcb400700 1 -- 172.21.15.63:0/1271702044 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bcc003e10 con 0x7f0bd4108550 2024-05-27T08:14:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.791+0000 7f0bcb400700 1 -- 172.21.15.63:0/1271702044 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc01b720 con 0x7f0bd4108550 2024-05-27T08:14:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.791+0000 7f0bd3e00700 1 -- 172.21.15.63:0/1271702044 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd41a7600 con 0x7f0bd4108550 2024-05-27T08:14:49.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.796+0000 7f0bcb400700 1 -- 172.21.15.63:0/1271702044 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0bcc022070 con 0x7f0bd4108550 2024-05-27T08:14:49.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.797+0000 7f0be8142b80 1 Processor -- start 2024-05-27T08:14:49.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.797+0000 7f0be8142b80 1 -- start start 2024-05-27T08:14:49.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43c8ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c908a0 0x55fc43c9d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0be8142b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fc43bbeec0 con 0x55fc43c90490 2024-05-27T08:14:49.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0be8142b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fc43baea60 con 0x55fc43c908a0 2024-05-27T08:14:49.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0be8142b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fc43bb2aa0 con 0x55fc43b11ec0 2024-05-27T08:14:49.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0bcb400700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bb80b29f0 0x7f0bb80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0bd2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0bd2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42480/0 (socket says 172.21.15.63:42480) 2024-05-27T08:14:49.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0bd2a00700 1 -- 172.21.15.63:0/198796771 learned_addr learned my addr 172.21.15.63:0/198796771 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:49.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.798+0000 7f0bcb400700 1 -- 172.21.15.63:0/1271702044 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f0bcc00f020 con 0x7f0bd4108550 2024-05-27T08:14:49.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.799+0000 7f0bd2000700 1 --2- 172.21.15.63:0/198796771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c908a0 0x55fc43c9d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.799+0000 7f0bd3400700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bb80b29f0 0x7f0bb80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.799+0000 7f0bd2000700 1 --2- 172.21.15.63:0/198796771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43c8ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.799+0000 7f0bd2a00700 1 -- 172.21.15.63:0/198796771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 msgr2=0x55fc43c8ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:49.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.799+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/198796771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43c8ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.799+0000 7f0bd2a00700 1 -- 172.21.15.63:0/198796771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c908a0 msgr2=0x55fc43c9d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:49.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.799+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/198796771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c908a0 0x55fc43c9d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.799+0000 7f0bd2a00700 1 -- 172.21.15.63:0/198796771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc43b774a0 con 0x55fc43c90490 2024-05-27T08:14:49.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.800+0000 7f0bd3400700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bb80b29f0 0x7f0bb80b4f00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0e1200 tx=0x7f0bcc005bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:49.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.800+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/198796771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9ab50 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400bd30 tx=0x7f0bc40106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc94795bbd1eaa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:49.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.800+0000 7f0bbfe00700 1 -- 172.21.15.63:0/198796771 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4010e10 con 0x55fc43c90490 2024-05-27T08:14:49.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.800+0000 7f0bbfe00700 1 -- 172.21.15.63:0/198796771 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc401c910 con 0x55fc43c90490 2024-05-27T08:14:49.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.800+0000 7f0bbfe00700 1 -- 172.21.15.63:0/198796771 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4011710 con 0x55fc43c90490 2024-05-27T08:14:49.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.801+0000 7f0be8142b80 1 -- 172.21.15.63:0/198796771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 msgr2=0x55fc43c9ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:49.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.801+0000 7f0be8142b80 1 --2- 172.21.15.63:0/198796771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9ab50 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400bd30 tx=0x7f0bc40106b0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.802+0000 7f0be8142b80 1 -- 172.21.15.63:0/198796771 shutdown_connections 2024-05-27T08:14:49.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.802+0000 7f0be8142b80 1 --2- 172.21.15.63:0/198796771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c908a0 0x55fc43c9d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.802+0000 7f0be8142b80 1 --2- 172.21.15.63:0/198796771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43c8ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.802+0000 7f0be8142b80 1 --2- 172.21.15.63:0/198796771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9ab50 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.802+0000 7f0be8142b80 1 -- 172.21.15.63:0/198796771 >> 172.21.15.63:0/198796771 conn(0x55fc43b72040 msgr2=0x55fc43c8bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:49.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.802+0000 7f0be8142b80 1 -- 172.21.15.63:0/198796771 shutdown_connections 2024-05-27T08:14:49.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.802+0000 7f0be8142b80 1 -- 172.21.15.63:0/198796771 wait complete. 2024-05-27T08:14:49.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.803+0000 7f0be8142b80 1 -- start start 2024-05-27T08:14:49.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.803+0000 7f0be8142b80 1 Processor -- start 2024-05-27T08:14:49.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43ca4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43ca5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c908a0 0x55fc43c9f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0be8142b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fc43ab5020 con 0x55fc43c90490 2024-05-27T08:14:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0be8142b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fc43b4f7e0 con 0x55fc43c908a0 2024-05-27T08:14:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0be8142b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fc43c9da10 con 0x55fc43b11ec0 2024-05-27T08:14:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0be8142b80 10 client.0 _ll_register_callbacks cb 0x55fc43b58cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0bd2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43ca5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0bd2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43ca5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42490/0 (socket says 172.21.15.63:42490) 2024-05-27T08:14:49.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0bd2a00700 1 -- 172.21.15.63:0/960315858 learned_addr learned my addr 172.21.15.63:0/960315858 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.804+0000 7f0bd2000700 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43ca4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.805+0000 7f0bd2000700 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c908a0 0x55fc43c9f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.805+0000 7f0bd2a00700 1 -- 172.21.15.63:0/960315858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 msgr2=0x55fc43ca4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.805+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43ca4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.805+0000 7f0bd2a00700 1 -- 172.21.15.63:0/960315858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c908a0 msgr2=0x55fc43c9f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.805+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c908a0 0x55fc43c9f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.805+0000 7f0bd2a00700 1 -- 172.21.15.63:0/960315858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc43afe950 con 0x55fc43c90490 2024-05-27T08:14:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.805+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43ca5290 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4000c00 tx=0x7f0bc400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91d147263e14bab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.805+0000 7f0bbcc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.806+0000 7f0bbcc00700 1 -- 172.21.15.63:0/960315858 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc401c770 con 0x55fc43c90490 2024-05-27T08:14:49.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.806+0000 7f0bbcc00700 1 -- 172.21.15.63:0/960315858 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f0bc4004030 con 0x55fc43c90490 2024-05-27T08:14:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.806+0000 7f0be8142b80 1 -- 172.21.15.63:0/960315858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc401c910 con 0x55fc43c90490 2024-05-27T08:14:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.806+0000 7f0bbcc00700 1 -- 172.21.15.63:0/960315858 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0bc402e050 con 0x55fc43c90490 2024-05-27T08:14:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.806+0000 7f0be8142b80 10 client.23304 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:49.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.806+0000 7f0be8142b80 1 -- 172.21.15.63:0/960315858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fc43b7f0e0 con 0x55fc43c90490 2024-05-27T08:14:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.807+0000 7f0bbcc00700 1 -- 172.21.15.63:0/960315858 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc401c910 con 0x55fc43c90490 2024-05-27T08:14:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.807+0000 7f0bbcc00700 1 -- 172.21.15.63:0/960315858 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0bc40047e0 con 0x55fc43c90490 2024-05-27T08:14:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.807+0000 7f0be8142b80 10 client.23304 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.807+0000 7f0be8142b80 10 client.23304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.807+0000 7f0be8142b80 20 client.23304 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.807+0000 7f0be8142b80 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fc43caec70 0x55fc43cb1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.807+0000 7f0be8142b80 4 client.23304 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.807+0000 7f0be8142b80 1 -- 172.21.15.63:0/960315858 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc43bc0e10 con 0x55fc43caec70 2024-05-27T08:14:49.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.808+0000 7f0bd2000700 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fc43caec70 0x55fc43cb1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.809+0000 7f0bd2000700 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fc43caec70 0x55fc43cb1090 crc :-1 s=READY pgs=730 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff8e97d3d6f29a3 server_cookie=47abd67ba66e8250 in_seq=0 out_seq=0 2024-05-27T08:14:49.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.809+0000 7f0bbcc00700 10 client.23304 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0bbcc00700 1 -- 172.21.15.63:0/960315858 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fc43bc0e10 con 0x55fc43caec70 2024-05-27T08:14:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0bbcc00700 10 client.23304 handle_command_reply: tid=0 2024-05-27T08:14:49.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0be8142b80 1 client.23304 shutdown 2024-05-27T08:14:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0be8142b80 1 -- 172.21.15.63:0/960315858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 msgr2=0x55fc43ca5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0be8142b80 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43ca5290 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4000c00 tx=0x7f0bc400bf30 comp rx=0 tx=0).stop 2024-05-27T08:14:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0be8142b80 1 -- 172.21.15.63:0/960315858 shutdown_connections 2024-05-27T08:14:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0be8142b80 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fc43caec70 0x55fc43cb1090 crc :-1 s=READY pgs=730 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0be8142b80 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c908a0 0x55fc43c9f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0be8142b80 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43ca4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0be8142b80 1 --2- 172.21.15.63:0/960315858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43ca5290 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0be8142b80 1 -- 172.21.15.63:0/960315858 >> 172.21.15.63:0/960315858 conn(0x55fc43b72040 msgr2=0x55fc43c8b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:49.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0bbcc00700 0 client.23304 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.811+0000 7f0be8142b80 1 -- 172.21.15.63:0/960315858 shutdown_connections 2024-05-27T08:14:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.812+0000 7f0be8142b80 1 -- 172.21.15.63:0/960315858 wait complete. 2024-05-27T08:14:49.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.812+0000 7f0be8142b80 20 client.23304 trim_cache size 0 max 16384 2024-05-27T08:14:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.837+0000 7f0be8142b80 1 Processor -- start 2024-05-27T08:14:49.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.837+0000 7f0be8142b80 1 -- start start 2024-05-27T08:14:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.837+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43b11ec0 0x55fc43c920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.837+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.837+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43c908a0 0x55fc43c9b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.837+0000 7f0be8142b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41b09a0 con 0x55fc43c90490 2024-05-27T08:14:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.837+0000 7f0be8142b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41ae230 con 0x55fc43b11ec0 2024-05-27T08:14:49.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.837+0000 7f0be8142b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41ac0a0 con 0x55fc43c908a0 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.838+0000 7f0bd2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.838+0000 7f0bd2a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42496/0 (socket says 172.21.15.63:42496) 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.838+0000 7f0bd2a00700 1 -- 172.21.15.63:0/8273233 learned_addr learned my addr 172.21.15.63:0/8273233 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.838+0000 7f0bd2000700 1 --2- 172.21.15.63:0/8273233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43b11ec0 0x55fc43c920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.838+0000 7f0bd2000700 1 --2- 172.21.15.63:0/8273233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43c908a0 0x55fc43c9b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.838+0000 7f0bd2a00700 1 -- 172.21.15.63:0/8273233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43c908a0 msgr2=0x55fc43c9b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.838+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/8273233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43c908a0 0x55fc43c9b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.839+0000 7f0bd2a00700 1 -- 172.21.15.63:0/8273233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43b11ec0 msgr2=0x55fc43c920a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.839+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/8273233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43b11ec0 0x55fc43c920a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.839+0000 7f0bd2a00700 1 -- 172.21.15.63:0/8273233 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc43ca0210 con 0x55fc43c90490 2024-05-27T08:14:49.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.839+0000 7f0bd2000700 1 --2- 172.21.15.63:0/8273233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43b11ec0 0x55fc43c920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.839+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/8273233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9acc0 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4010a00 tx=0x7f0bc4004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dbe4cd83ab671 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.839+0000 7f0bbf400700 1 -- 172.21.15.63:0/8273233 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4036020 con 0x55fc43c90490 2024-05-27T08:14:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.839+0000 7f0bbf400700 1 -- 172.21.15.63:0/8273233 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc400b410 con 0x55fc43c90490 2024-05-27T08:14:49.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.839+0000 7f0bbf400700 1 -- 172.21.15.63:0/8273233 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4005260 con 0x55fc43c90490 2024-05-27T08:14:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.841+0000 7f0be8142b80 1 -- 172.21.15.63:0/8273233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 msgr2=0x55fc43c9acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.841+0000 7f0be8142b80 1 --2- 172.21.15.63:0/8273233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9acc0 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4010a00 tx=0x7f0bc4004da0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.841+0000 7f0be8142b80 1 -- 172.21.15.63:0/8273233 shutdown_connections 2024-05-27T08:14:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.841+0000 7f0be8142b80 1 --2- 172.21.15.63:0/8273233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43c908a0 0x55fc43c9b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.841+0000 7f0be8142b80 1 --2- 172.21.15.63:0/8273233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43b11ec0 0x55fc43c920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.841+0000 7f0be8142b80 1 --2- 172.21.15.63:0/8273233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c90490 0x55fc43c9acc0 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.841+0000 7f0be8142b80 1 -- 172.21.15.63:0/8273233 >> 172.21.15.63:0/8273233 conn(0x55fc43b72040 msgr2=0x55fc43c91990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.841+0000 7f0be8142b80 1 -- 172.21.15.63:0/8273233 shutdown_connections 2024-05-27T08:14:49.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.842+0000 7f0be8142b80 1 -- 172.21.15.63:0/8273233 wait complete. 2024-05-27T08:14:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.842+0000 7f0be8142b80 1 -- start start 2024-05-27T08:14:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.843+0000 7f0be8142b80 1 Processor -- start 2024-05-27T08:14:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.843+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43cab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.843+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c90490 0x55fc43ca77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.843+0000 7f0be8142b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c908a0 0x55fc43ca7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.843+0000 7f0be8142b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41ac6f0 con 0x55fc43c908a0 2024-05-27T08:14:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.843+0000 7f0be8142b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41adb60 con 0x55fc43c90490 2024-05-27T08:14:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.843+0000 7f0be8142b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41acdc0 con 0x55fc43b11ec0 2024-05-27T08:14:49.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.843+0000 7f0be8142b80 10 client.0 _ll_register_callbacks cb 0x55fc43b58cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:49.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.844+0000 7f0bd2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c908a0 0x55fc43ca7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.844+0000 7f0bd2000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c908a0 0x55fc43ca7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42502/0 (socket says 172.21.15.63:42502) 2024-05-27T08:14:49.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.844+0000 7f0bd2000700 1 -- 172.21.15.63:0/757906367 learned_addr learned my addr 172.21.15.63:0/757906367 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:49.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.844+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c90490 0x55fc43ca77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.844+0000 7f0bd2000700 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43cab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.844+0000 7f0bd2000700 1 -- 172.21.15.63:0/757906367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 msgr2=0x55fc43cab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:49.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.844+0000 7f0bd2000700 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43cab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.845+0000 7f0bd2000700 1 -- 172.21.15.63:0/757906367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c90490 msgr2=0x55fc43ca77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:49.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.845+0000 7f0bd2000700 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c90490 0x55fc43ca77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.845+0000 7f0bd2000700 1 -- 172.21.15.63:0/757906367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc43ca8550 con 0x55fc43c908a0 2024-05-27T08:14:49.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.845+0000 7f0bd2000700 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c908a0 0x55fc43ca7db0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x55fc43ca7070 tx=0x7f0bc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8339f15e957ac9e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:49.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.845+0000 7f0bbcc00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:49.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.848+0000 7f0bbcc00700 1 -- 172.21.15.63:0/757906367 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc00109f0 con 0x55fc43c908a0 2024-05-27T08:14:49.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.848+0000 7f0bbcc00700 1 -- 172.21.15.63:0/757906367 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f0bc001b070 con 0x55fc43c908a0 2024-05-27T08:14:49.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.848+0000 7f0be8142b80 1 -- 172.21.15.63:0/757906367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc43bba5f0 con 0x55fc43c908a0 2024-05-27T08:14:49.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.848+0000 7f0bbcc00700 1 -- 172.21.15.63:0/757906367 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0bc0010b90 con 0x55fc43c908a0 2024-05-27T08:14:49.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.848+0000 7f0be8142b80 10 client.23310 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:49.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.848+0000 7f0be8142b80 1 -- 172.21.15.63:0/757906367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fc43d1ed20 con 0x55fc43c908a0 2024-05-27T08:14:49.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.849+0000 7f0bbcc00700 1 -- 172.21.15.63:0/757906367 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55fc43bba5f0 con 0x55fc43c908a0 2024-05-27T08:14:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.849+0000 7f0bbcc00700 1 -- 172.21.15.63:0/757906367 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0bc002d040 con 0x55fc43c908a0 2024-05-27T08:14:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.849+0000 7f0be8142b80 10 client.23310 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.849+0000 7f0be8142b80 10 client.23310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.849+0000 7f0be8142b80 20 client.23310 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.849+0000 7f0be8142b80 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fc43d1f430 0x55fc43d21850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.849+0000 7f0be8142b80 4 client.23310 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:14:49.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.849+0000 7f0be8142b80 1 -- 172.21.15.63:0/757906367 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fc43a312d0 con 0x55fc43d1f430 2024-05-27T08:14:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.850+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fc43d1f430 0x55fc43d21850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.851+0000 7f0bbcc00700 10 client.23310 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:49.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.851+0000 7f0bd2a00700 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fc43d1f430 0x55fc43d21850 crc :-1 s=READY pgs=731 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e91c743cf1dc371 server_cookie=4ea35bb25326305c in_seq=0 out_seq=0 2024-05-27T08:14:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.854+0000 7f0bbcc00700 1 -- 172.21.15.63:0/757906367 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fc43a312d0 con 0x55fc43d1f430 2024-05-27T08:14:49.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.854+0000 7f0bbcc00700 10 client.23310 handle_command_reply: tid=0 2024-05-27T08:14:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.854+0000 7f0be8142b80 1 client.23310 shutdown 2024-05-27T08:14:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0be8142b80 1 -- 172.21.15.63:0/757906367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c908a0 msgr2=0x55fc43ca7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0be8142b80 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c908a0 0x55fc43ca7db0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x55fc43ca7070 tx=0x7f0bc0004750 comp rx=0 tx=0).stop 2024-05-27T08:14:49.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0be8142b80 1 -- 172.21.15.63:0/757906367 shutdown_connections 2024-05-27T08:14:49.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0be8142b80 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fc43d1f430 0x55fc43d21850 crc :-1 s=READY pgs=731 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0be8142b80 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fc43c908a0 0x55fc43ca7db0 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0be8142b80 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc43b11ec0 0x55fc43cab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0be8142b80 1 --2- 172.21.15.63:0/757906367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fc43c90490 0x55fc43ca77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0be8142b80 1 -- 172.21.15.63:0/757906367 >> 172.21.15.63:0/757906367 conn(0x55fc43b72040 msgr2=0x55fc43c991f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:49.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0bbcc00700 0 client.23310 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:49.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0be8142b80 1 -- 172.21.15.63:0/757906367 shutdown_connections 2024-05-27T08:14:49.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.855+0000 7f0be8142b80 1 -- 172.21.15.63:0/757906367 wait complete. 2024-05-27T08:14:49.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.856+0000 7f0be8142b80 20 client.23310 trim_cache size 0 max 16384 2024-05-27T08:14:49.857 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:14:49.857 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:14:49.857 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "14976336-fbad-4ec5-a040-f1257e16872b", 2024-05-27T08:14:49.857 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:14:49.857 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:14:49.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.856+0000 7f0bd3e00700 1 -- 172.21.15.63:0/1271702044 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bb80b29f0 msgr2=0x7f0bb80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:49.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.856+0000 7f0bd3e00700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bb80b29f0 0x7f0bb80b4f00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0e1200 tx=0x7f0bcc005bc0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.856+0000 7f0bd3e00700 1 -- 172.21.15.63:0/1271702044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bd4108550 msgr2=0x7f0bd419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:49.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.856+0000 7f0bd3e00700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bd4108550 0x7f0bd419a990 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc007d10 tx=0x7f0bcc007d40 comp rx=0 tx=0).stop 2024-05-27T08:14:49.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.857+0000 7f0bd3e00700 1 -- 172.21.15.63:0/1271702044 shutdown_connections 2024-05-27T08:14:49.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.857+0000 7f0bd3e00700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0bb80b29f0 0x7f0bb80b4f00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.857+0000 7f0bd3e00700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0bd4108550 0x7f0bd419a990 unknown :-1 s=CLOSED pgs=2916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.857+0000 7f0bd3e00700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd41032d0 0x7f0bd41a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.857+0000 7f0bd3e00700 1 --2- 172.21.15.63:0/1271702044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd4107c00 0x7f0bd41a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:49.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.857+0000 7f0bd3e00700 1 -- 172.21.15.63:0/1271702044 >> 172.21.15.63:0/1271702044 conn(0x7f0bd4078ab0 msgr2=0x7f0bd4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:49.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.857+0000 7f0bd3e00700 1 -- 172.21.15.63:0/1271702044 shutdown_connections 2024-05-27T08:14:49.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:49.857+0000 7f0bd3e00700 1 -- 172.21.15.63:0/1271702044 wait complete. 2024-05-27T08:14:50.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:49 smithi086 ceph-mon[34687]: pgmap v1309: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T08:14:50.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:49 smithi149 ceph-mon[35408]: pgmap v1309: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T08:14:50.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:49 smithi063 ceph-mon[31613]: pgmap v1309: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T08:14:50.723 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:14:50.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:14:50.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:50.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:50.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:14:51.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:14:51.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:14:51.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:14:51.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:14:51.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:14:51.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:14:51.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:14:51.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:14:51.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:14:51.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:14:51.230 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:14:52.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:51 smithi086 ceph-mon[34687]: pgmap v1310: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T08:14:52.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:51 smithi149 ceph-mon[35408]: pgmap v1310: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T08:14:52.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:51 smithi063 ceph-mon[31613]: pgmap v1310: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 11 op/s 2024-05-27T08:14:52.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:52 smithi063 ceph-mon[31613]: pgmap v1311: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:14:53.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:52 smithi086 ceph-mon[34687]: pgmap v1311: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:14:53.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:52 smithi149 ceph-mon[35408]: pgmap v1311: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.0 MiB/s wr, 49 op/s 2024-05-27T08:14:53.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.755+0000 7f7807200700 1 -- 172.21.15.63:0/424720475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78001043f0 msgr2=0x7f7800104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:53.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.755+0000 7f7807200700 1 --2- 172.21.15.63:0/424720475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78001043f0 0x7f7800104800 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7f77f40099c0 tx=0x7f77f4009cd0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.755+0000 7f7807200700 1 -- 172.21.15.63:0/424720475 shutdown_connections 2024-05-27T08:14:53.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.755+0000 7f7807200700 1 --2- 172.21.15.63:0/424720475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7800109670 0x7f7800110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.755+0000 7f7807200700 1 --2- 172.21.15.63:0/424720475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78001043f0 0x7f7800104800 unknown :-1 s=CLOSED pgs=2921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.755+0000 7f7807200700 1 --2- 172.21.15.63:0/424720475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7800108d20 0x7f7800109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.755+0000 7f7807200700 1 -- 172.21.15.63:0/424720475 >> 172.21.15.63:0/424720475 conn(0x7f7800079d00 msgr2=0x7f780007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:53.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.755+0000 7f7807200700 1 -- 172.21.15.63:0/424720475 shutdown_connections 2024-05-27T08:14:53.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.755+0000 7f7807200700 1 -- 172.21.15.63:0/424720475 wait complete. 2024-05-27T08:14:53.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.757+0000 7f7807200700 1 Processor -- start 2024-05-27T08:14:53.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.757+0000 7f7807200700 1 -- start start 2024-05-27T08:14:53.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.758+0000 7f7807200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001043f0 0x7f780019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.758+0000 7f7807200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7800108d20 0x7f780019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.758+0000 7f7807200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7800109670 0x7f78001a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.758+0000 7f7807200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78001141a0 con 0x7f7800108d20 2024-05-27T08:14:53.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.758+0000 7f7807200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7800114020 con 0x7f78001043f0 2024-05-27T08:14:53.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.758+0000 7f7807200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7800114320 con 0x7f7800109670 2024-05-27T08:14:53.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.759+0000 7f7805e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001043f0 0x7f780019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.759+0000 7f7806800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7800109670 0x7f78001a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.759+0000 7f7805e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001043f0 0x7f780019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44674/0 (socket says 172.21.15.63:44674) 2024-05-27T08:14:53.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.759+0000 7f7805e00700 1 -- 172.21.15.63:0/1716160127 learned_addr learned my addr 172.21.15.63:0/1716160127 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:53.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.759+0000 7f7805400700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7800108d20 0x7f780019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.759+0000 7f7805e00700 1 -- 172.21.15.63:0/1716160127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7800109670 msgr2=0x7f78001a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:53.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.759+0000 7f7805e00700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7800109670 0x7f78001a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.759+0000 7f7805e00700 1 -- 172.21.15.63:0/1716160127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7800108d20 msgr2=0x7f780019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:53.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.759+0000 7f7805e00700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7800108d20 0x7f780019d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.759+0000 7f7805e00700 1 -- 172.21.15.63:0/1716160127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78001a1d20 con 0x7f78001043f0 2024-05-27T08:14:53.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.760+0000 7f7805e00700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001043f0 0x7f780019ce50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77f400b580 tx=0x7f77f40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:53.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.760+0000 7f77faa00700 1 -- 172.21.15.63:0/1716160127 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4027070 con 0x7f78001043f0 2024-05-27T08:14:53.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.761+0000 7f7807200700 1 -- 172.21.15.63:0/1716160127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78001a1fb0 con 0x7f78001043f0 2024-05-27T08:14:53.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.761+0000 7f77faa00700 1 -- 172.21.15.63:0/1716160127 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77f400fc80 con 0x7f78001043f0 2024-05-27T08:14:53.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.761+0000 7f77faa00700 1 -- 172.21.15.63:0/1716160127 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4022d30 con 0x7f78001043f0 2024-05-27T08:14:53.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.761+0000 7f7807200700 1 -- 172.21.15.63:0/1716160127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78001a9930 con 0x7f78001043f0 2024-05-27T08:14:53.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.766+0000 7f7817388b80 1 Processor -- start 2024-05-27T08:14:53.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.767+0000 7f7817388b80 1 -- start start 2024-05-27T08:14:53.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.767+0000 7f77faa00700 1 -- 172.21.15.63:0/1716160127 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f77f402a070 con 0x7f78001043f0 2024-05-27T08:14:53.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.767+0000 7f7817388b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.767+0000 7f7817388b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5410 0x557b0e1afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.767+0000 7f7817388b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5820 0x557b0e1b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.768+0000 7f7817388b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557b0e0d3e40 con 0x557b0e026e40 2024-05-27T08:14:53.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.768+0000 7f7817388b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b0e0c39e0 con 0x557b0e1a5820 2024-05-27T08:14:53.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.768+0000 7f7817388b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557b0e0c7a20 con 0x557b0e1a5410 2024-05-27T08:14:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.768+0000 7f7806800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.768+0000 7f7806800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42526/0 (socket says 172.21.15.63:42526) 2024-05-27T08:14:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.768+0000 7f7806800700 1 -- 172.21.15.63:0/4159104228 learned_addr learned my addr 172.21.15.63:0/4159104228 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.768+0000 7f7805e00700 1 --2- 172.21.15.63:0/4159104228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5410 0x557b0e1afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.768+0000 7f7805400700 1 --2- 172.21.15.63:0/4159104228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5820 0x557b0e1b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.769+0000 7f7806800700 1 -- 172.21.15.63:0/4159104228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5410 msgr2=0x557b0e1afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:53.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.769+0000 7f7806800700 1 --2- 172.21.15.63:0/4159104228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5410 0x557b0e1afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.769+0000 7f7806800700 1 -- 172.21.15.63:0/4159104228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5820 msgr2=0x557b0e1b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:53.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.769+0000 7f7806800700 1 --2- 172.21.15.63:0/4159104228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5820 0x557b0e1b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.769+0000 7f7806800700 1 -- 172.21.15.63:0/4159104228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0e08c420 con 0x557b0e026e40 2024-05-27T08:14:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.769+0000 7f7806800700 1 --2- 172.21.15.63:0/4159104228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1a4ed0 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7f77fc003fa0 tx=0x7f77fc009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8eb8334f83bddc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.770+0000 7f77faa00700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77e80b29f0 0x7f77e80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.770+0000 7f77faa00700 1 -- 172.21.15.63:0/1716160127 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f77f40e3810 con 0x7f78001043f0 2024-05-27T08:14:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.770+0000 7f77ef400700 1 -- 172.21.15.63:0/4159104228 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc004bb0 con 0x557b0e026e40 2024-05-27T08:14:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.770+0000 7f7805e00700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77e80b29f0 0x7f77e80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.770+0000 7f77ef400700 1 -- 172.21.15.63:0/4159104228 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77fc0048e0 con 0x557b0e026e40 2024-05-27T08:14:53.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.770+0000 7f77ef400700 1 -- 172.21.15.63:0/4159104228 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc007d20 con 0x557b0e026e40 2024-05-27T08:14:53.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.771+0000 7f7817388b80 1 -- 172.21.15.63:0/4159104228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 msgr2=0x557b0e1a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:53.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.771+0000 7f7805e00700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77e80b29f0 0x7f77e80b4f00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f77f40e0c50 tx=0x7f77f40e9550 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:53.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.771+0000 7f7817388b80 1 --2- 172.21.15.63:0/4159104228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1a4ed0 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7f77fc003fa0 tx=0x7f77fc009ba0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.772+0000 7f7817388b80 1 -- 172.21.15.63:0/4159104228 shutdown_connections 2024-05-27T08:14:53.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.772+0000 7f7817388b80 1 --2- 172.21.15.63:0/4159104228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5820 0x557b0e1b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.772+0000 7f7817388b80 1 --2- 172.21.15.63:0/4159104228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1a4ed0 unknown :-1 s=CLOSED pgs=2922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.772+0000 7f7817388b80 1 --2- 172.21.15.63:0/4159104228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5410 0x557b0e1afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.772+0000 7f7817388b80 1 -- 172.21.15.63:0/4159104228 >> 172.21.15.63:0/4159104228 conn(0x557b0e086fc0 msgr2=0x557b0e1a0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:53.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.772+0000 7f7817388b80 1 -- 172.21.15.63:0/4159104228 shutdown_connections 2024-05-27T08:14:53.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.772+0000 7f7817388b80 1 -- 172.21.15.63:0/4159104228 wait complete. 2024-05-27T08:14:53.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.773+0000 7f7817388b80 1 -- start start 2024-05-27T08:14:53.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.773+0000 7f7817388b80 1 Processor -- start 2024-05-27T08:14:53.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7817388b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e026e40 0x557b0e1af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7817388b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5410 0x557b0e1afb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7817388b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5820 0x557b0e1b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7817388b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557b0dfc9fa0 con 0x557b0e1a5410 2024-05-27T08:14:53.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7817388b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b0e064760 con 0x557b0e026e40 2024-05-27T08:14:53.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7817388b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557b0e1b2990 con 0x557b0e1a5820 2024-05-27T08:14:53.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7817388b80 10 client.0 _ll_register_callbacks cb 0x557b0e06dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:53.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7805400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5410 0x557b0e1afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7805400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5410 0x557b0e1afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42540/0 (socket says 172.21.15.63:42540) 2024-05-27T08:14:53.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7805400700 1 -- 172.21.15.63:0/2043175399 learned_addr learned my addr 172.21.15.63:0/2043175399 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:53.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7806800700 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e026e40 0x557b0e1af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.774+0000 7f7806800700 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5820 0x557b0e1b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.775+0000 7f7805400700 1 -- 172.21.15.63:0/2043175399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5820 msgr2=0x557b0e1b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.775+0000 7f7805400700 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5820 0x557b0e1b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.775+0000 7f7805400700 1 -- 172.21.15.63:0/2043175399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e026e40 msgr2=0x557b0e1af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.775+0000 7f7805400700 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e026e40 0x557b0e1af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.775+0000 7f7805400700 1 -- 172.21.15.63:0/2043175399 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b0e0138d0 con 0x557b0e1a5410 2024-05-27T08:14:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.775+0000 7f7805400700 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5410 0x557b0e1afb00 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b970 tx=0x7f77f000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10af3bd77e46e58f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:53.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.775+0000 7f77e3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:53.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.775+0000 7f77e3e00700 1 -- 172.21.15.63:0/2043175399 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0003dc0 con 0x557b0e1a5410 2024-05-27T08:14:53.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.775+0000 7f7817388b80 1 -- 172.21.15.63:0/2043175399 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77fc0048e0 con 0x557b0e1a5410 2024-05-27T08:14:53.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.775+0000 7f77e3e00700 1 -- 172.21.15.63:0/2043175399 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f77f0004830 con 0x557b0e1a5410 2024-05-27T08:14:53.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.776+0000 7f77e3e00700 1 -- 172.21.15.63:0/2043175399 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f77f0024050 con 0x557b0e1a5410 2024-05-27T08:14:53.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.776+0000 7f7817388b80 10 client.23322 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:53.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.776+0000 7f7817388b80 1 -- 172.21.15.63:0/2043175399 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557b0e094060 con 0x557b0e1a5410 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.776+0000 7f77e3e00700 1 -- 172.21.15.63:0/2043175399 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77fc0048e0 con 0x557b0e1a5410 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.776+0000 7f77e3e00700 1 -- 172.21.15.63:0/2043175399 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f77f0011440 con 0x557b0e1a5410 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.777+0000 7f7817388b80 10 client.23322 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.777+0000 7f7817388b80 10 client.23322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.777+0000 7f7817388b80 20 client.23322 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.777+0000 7f7817388b80 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557b0e1bf950 0x557b0e1c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.777+0000 7f7817388b80 4 client.23322 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.777+0000 7f7817388b80 1 -- 172.21.15.63:0/2043175399 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b0e0d5d90 con 0x557b0e1bf950 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.778+0000 7f7806800700 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557b0e1bf950 0x557b0e1c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.778+0000 7f77e3e00700 10 client.23322 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:53.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.778+0000 7f7806800700 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557b0e1bf950 0x557b0e1c1de0 crc :-1 s=READY pgs=732 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b4ddd45dcfac1f server_cookie=2388de854f6f0931 in_seq=0 out_seq=0 2024-05-27T08:14:53.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.780+0000 7f77e3e00700 1 -- 172.21.15.63:0/2043175399 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557b0e0d5d90 con 0x557b0e1bf950 2024-05-27T08:14:53.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.780+0000 7f77e3e00700 10 client.23322 handle_command_reply: tid=0 2024-05-27T08:14:53.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.780+0000 7f7817388b80 1 client.23322 shutdown 2024-05-27T08:14:53.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f7817388b80 1 -- 172.21.15.63:0/2043175399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5410 msgr2=0x557b0e1afb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f7817388b80 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5410 0x557b0e1afb00 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b970 tx=0x7f77f000bd30 comp rx=0 tx=0).stop 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f7817388b80 1 -- 172.21.15.63:0/2043175399 shutdown_connections 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f7817388b80 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557b0e1bf950 0x557b0e1c1de0 crc :-1 s=READY pgs=732 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f7817388b80 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5820 0x557b0e1b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f7817388b80 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e026e40 0x557b0e1af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f7817388b80 1 --2- 172.21.15.63:0/2043175399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5410 0x557b0e1afb00 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f7817388b80 1 -- 172.21.15.63:0/2043175399 >> 172.21.15.63:0/2043175399 conn(0x557b0e086fc0 msgr2=0x557b0e19fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f77e3e00700 0 client.23322 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f7817388b80 1 -- 172.21.15.63:0/2043175399 shutdown_connections 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.781+0000 7f7817388b80 1 -- 172.21.15.63:0/2043175399 wait complete. 2024-05-27T08:14:53.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.782+0000 7f7817388b80 20 client.23322 trim_cache size 0 max 16384 2024-05-27T08:14:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.812+0000 7f7817388b80 1 Processor -- start 2024-05-27T08:14:53.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.813+0000 7f7817388b80 1 -- start start 2024-05-27T08:14:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.813+0000 7f7817388b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e026e40 0x557b0e1c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.813+0000 7f7817388b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5410 0x557b0e1aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.813+0000 7f7817388b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5820 0x557b0e1bba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.813+0000 7f7817388b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78001b4d20 con 0x557b0e1a5820 2024-05-27T08:14:53.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.813+0000 7f7817388b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78001b25b0 con 0x557b0e1a5410 2024-05-27T08:14:53.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.813+0000 7f7817388b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f78001b0420 con 0x557b0e026e40 2024-05-27T08:14:53.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.814+0000 7f7806800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5820 0x557b0e1bba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.814+0000 7f7806800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e026e40 0x557b0e1c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.814+0000 7f7805400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5410 0x557b0e1aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.814+0000 7f7806800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5820 0x557b0e1bba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42552/0 (socket says 172.21.15.63:42552) 2024-05-27T08:14:53.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.814+0000 7f7806800700 1 -- 172.21.15.63:0/180143950 learned_addr learned my addr 172.21.15.63:0/180143950 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:53.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.814+0000 7f7806800700 1 -- 172.21.15.63:0/180143950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e026e40 msgr2=0x557b0e1c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:53.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.814+0000 7f7806800700 1 --2- 172.21.15.63:0/180143950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e026e40 0x557b0e1c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.814+0000 7f7806800700 1 -- 172.21.15.63:0/180143950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5410 msgr2=0x557b0e1aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:53.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.814+0000 7f7806800700 1 --2- 172.21.15.63:0/180143950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5410 0x557b0e1aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.814+0000 7f7806800700 1 -- 172.21.15.63:0/180143950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0e1bd420 con 0x557b0e1a5820 2024-05-27T08:14:53.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.815+0000 7f7806800700 1 --2- 172.21.15.63:0/180143950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5820 0x557b0e1bba30 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f77fc004010 tx=0x7f77fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43761f92f24ea54a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:53.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.815+0000 7f77eea00700 1 -- 172.21.15.63:0/180143950 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc005020 con 0x557b0e1a5820 2024-05-27T08:14:53.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.815+0000 7f77eea00700 1 -- 172.21.15.63:0/180143950 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77fc007700 con 0x557b0e1a5820 2024-05-27T08:14:53.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.815+0000 7f77eea00700 1 -- 172.21.15.63:0/180143950 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc004950 con 0x557b0e1a5820 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.816+0000 7f7817388b80 1 -- 172.21.15.63:0/180143950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5820 msgr2=0x557b0e1bba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.816+0000 7f7817388b80 1 --2- 172.21.15.63:0/180143950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5820 0x557b0e1bba30 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f77fc004010 tx=0x7f77fc00b040 comp rx=0 tx=0).stop 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.816+0000 7f7817388b80 1 -- 172.21.15.63:0/180143950 shutdown_connections 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.816+0000 7f7817388b80 1 --2- 172.21.15.63:0/180143950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e1a5820 0x557b0e1bba30 unknown :-1 s=CLOSED pgs=2924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.816+0000 7f7817388b80 1 --2- 172.21.15.63:0/180143950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e026e40 0x557b0e1c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.817+0000 7f7817388b80 1 --2- 172.21.15.63:0/180143950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5410 0x557b0e1aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.817+0000 7f7817388b80 1 -- 172.21.15.63:0/180143950 >> 172.21.15.63:0/180143950 conn(0x557b0e086fc0 msgr2=0x557b0e1af580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.817+0000 7f7817388b80 1 -- 172.21.15.63:0/180143950 shutdown_connections 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.817+0000 7f7817388b80 1 -- 172.21.15.63:0/180143950 wait complete. 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.817+0000 7f7817388b80 1 -- start start 2024-05-27T08:14:53.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.818+0000 7f7817388b80 1 Processor -- start 2024-05-27T08:14:53.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.818+0000 7f7817388b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.818+0000 7f7817388b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5410 0x557b0e1b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.818+0000 7f7817388b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5820 0x557b0e1b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.818+0000 7f7817388b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78001b0a70 con 0x557b0e026e40 2024-05-27T08:14:53.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.818+0000 7f7817388b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78001b1ee0 con 0x557b0e1a5820 2024-05-27T08:14:53.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.818+0000 7f7817388b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f78001b1140 con 0x557b0e1a5410 2024-05-27T08:14:53.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.818+0000 7f7817388b80 10 client.0 _ll_register_callbacks cb 0x557b0e06dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:53.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.819+0000 7f7806800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.819+0000 7f7806800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42564/0 (socket says 172.21.15.63:42564) 2024-05-27T08:14:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.819+0000 7f7806800700 1 -- 172.21.15.63:0/1118120481 learned_addr learned my addr 172.21.15.63:0/1118120481 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.819+0000 7f7805400700 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5410 0x557b0e1b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.819+0000 7f7806800700 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5820 0x557b0e1b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.819+0000 7f7806800700 1 -- 172.21.15.63:0/1118120481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5410 msgr2=0x557b0e1b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.819+0000 7f7806800700 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5410 0x557b0e1b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.819+0000 7f7806800700 1 -- 172.21.15.63:0/1118120481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5820 msgr2=0x557b0e1b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.819+0000 7f7806800700 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5820 0x557b0e1b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.819+0000 7f7806800700 1 -- 172.21.15.63:0/1118120481 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b0e1a2350 con 0x557b0e026e40 2024-05-27T08:14:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.820+0000 7f7806800700 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1b8470 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7f77fc002230 tx=0x7f77fc003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5264a6a4525fb42a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.820+0000 7f77e3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:53.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.820+0000 7f77e3e00700 1 -- 172.21.15.63:0/1118120481 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc007e60 con 0x557b0e026e40 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.820+0000 7f7817388b80 1 -- 172.21.15.63:0/1118120481 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77fc007700 con 0x557b0e026e40 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.820+0000 7f77e3e00700 1 -- 172.21.15.63:0/1118120481 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7f77fc005040 con 0x557b0e026e40 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f77e3e00700 1 -- 172.21.15.63:0/1118120481 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f77fc005240 con 0x557b0e026e40 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f7817388b80 10 client.23328 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f7817388b80 1 -- 172.21.15.63:0/1118120481 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557b0e22a2f0 con 0x557b0e026e40 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f77e3e00700 1 -- 172.21.15.63:0/1118120481 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77fc007700 con 0x557b0e026e40 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f77e3e00700 1 -- 172.21.15.63:0/1118120481 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f77fc015480 con 0x557b0e026e40 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f7817388b80 10 client.23328 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f7817388b80 10 client.23328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f7817388b80 20 client.23328 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f7817388b80 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557b0e22a760 0x557b0e22cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f7817388b80 4 client.23328 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:14:53.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.821+0000 7f7817388b80 1 -- 172.21.15.63:0/1118120481 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b0e0cf570 con 0x557b0e22a760 2024-05-27T08:14:53.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.822+0000 7f7805400700 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557b0e22a760 0x557b0e22cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:53.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.823+0000 7f7805400700 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557b0e22a760 0x557b0e22cb80 crc :-1 s=READY pgs=733 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c93ea8dd3cad2ca server_cookie=812a4c20fb30f2ed in_seq=0 out_seq=0 2024-05-27T08:14:53.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.823+0000 7f77e3e00700 10 client.23328 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:53.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.824+0000 7f77e3e00700 1 -- 172.21.15.63:0/1118120481 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557b0e0cf570 con 0x557b0e22a760 2024-05-27T08:14:53.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.824+0000 7f77e3e00700 10 client.23328 handle_command_reply: tid=0 2024-05-27T08:14:53.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.824+0000 7f7817388b80 1 client.23328 shutdown 2024-05-27T08:14:53.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.824+0000 7f7817388b80 1 -- 172.21.15.63:0/1118120481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 msgr2=0x557b0e1b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.824+0000 7f7817388b80 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1b8470 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7f77fc002230 tx=0x7f77fc003a80 comp rx=0 tx=0).stop 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.825+0000 7f7817388b80 1 -- 172.21.15.63:0/1118120481 shutdown_connections 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.825+0000 7f7817388b80 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557b0e22a760 0x557b0e22cb80 crc :-1 s=READY pgs=733 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.825+0000 7f7817388b80 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b0e1a5820 0x557b0e1b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.825+0000 7f7817388b80 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557b0e026e40 0x557b0e1b8470 unknown :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.825+0000 7f7817388b80 1 --2- 172.21.15.63:0/1118120481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557b0e1a5410 0x557b0e1b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.825+0000 7f7817388b80 1 -- 172.21.15.63:0/1118120481 >> 172.21.15.63:0/1118120481 conn(0x557b0e086fc0 msgr2=0x557b0e1a8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.825+0000 7f77e3e00700 0 client.23328 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.825+0000 7f7817388b80 1 -- 172.21.15.63:0/1118120481 shutdown_connections 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.825+0000 7f7817388b80 1 -- 172.21.15.63:0/1118120481 wait complete. 2024-05-27T08:14:53.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.825+0000 7f7817388b80 20 client.23328 trim_cache size 0 max 16384 2024-05-27T08:14:53.827 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:14:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T08:14:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:14:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "14976336-fbad-4ec5-a040-f1257e16872b": { 2024-05-27T08:14:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:14:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "14976336-fbad-4ec5-a040-f1257e16872b", 2024-05-27T08:14:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:14:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:14:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:14:53.828 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:14:53.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.826+0000 7f7807200700 1 -- 172.21.15.63:0/1716160127 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77e80b29f0 msgr2=0x7f77e80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:53.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.826+0000 7f7807200700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77e80b29f0 0x7f77e80b4f00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f77f40e0c50 tx=0x7f77f40e9550 comp rx=0 tx=0).stop 2024-05-27T08:14:53.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.826+0000 7f7807200700 1 -- 172.21.15.63:0/1716160127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001043f0 msgr2=0x7f780019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:53.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.826+0000 7f7807200700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001043f0 0x7f780019ce50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77f400b580 tx=0x7f77f40047b0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.827+0000 7f7807200700 1 -- 172.21.15.63:0/1716160127 shutdown_connections 2024-05-27T08:14:53.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.827+0000 7f7807200700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f77e80b29f0 0x7f77e80b4f00 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.827+0000 7f7807200700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7800109670 0x7f78001a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.827+0000 7f7807200700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78001043f0 0x7f780019ce50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.827+0000 7f7807200700 1 --2- 172.21.15.63:0/1716160127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7800108d20 0x7f780019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:53.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.827+0000 7f7807200700 1 -- 172.21.15.63:0/1716160127 >> 172.21.15.63:0/1716160127 conn(0x7f7800079d00 msgr2=0x7f7800101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:53.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.827+0000 7f7807200700 1 -- 172.21.15.63:0/1716160127 shutdown_connections 2024-05-27T08:14:53.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:14:53.827+0000 7f7807200700 1 -- 172.21.15.63:0/1716160127 wait complete. 2024-05-27T08:14:54.797 INFO:tasks.cephfs.filesystem:scrub status for tag:14976336-fbad-4ec5-a040-f1257e16872b - {'path': '/', 'tag': '14976336-fbad-4ec5-a040-f1257e16872b', 'options': 'recursive,force'} 2024-05-27T08:14:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.903+0000 7fad9be00640 1 -- 172.21.15.63:0/3760982932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad9c106420 msgr2=0x7fad9c1127f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.903+0000 7fad9be00640 1 --2- 172.21.15.63:0/3760982932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad9c106420 0x7fad9c1127f0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7fad94009ca0 tx=0x7fad9402a8c0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.903+0000 7fad9be00640 1 -- 172.21.15.63:0/3760982932 shutdown_connections 2024-05-27T08:14:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.903+0000 7fad9be00640 1 --2- 172.21.15.63:0/3760982932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad9c106420 0x7fad9c1127f0 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.903+0000 7fad9be00640 1 --2- 172.21.15.63:0/3760982932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad9c10c170 0x7fad9c105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.903+0000 7fad9be00640 1 --2- 172.21.15.63:0/3760982932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad9c10bcd0 0x7fad9c105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.903+0000 7fad9be00640 1 -- 172.21.15.63:0/3760982932 >> 172.21.15.63:0/3760982932 conn(0x7fad9c1013d0 msgr2=0x7fad9c103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.903+0000 7fad9be00640 1 -- 172.21.15.63:0/3760982932 shutdown_connections 2024-05-27T08:14:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.903+0000 7fad9be00640 1 -- 172.21.15.63:0/3760982932 wait complete. 2024-05-27T08:14:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.904+0000 7fad9be00640 1 Processor -- start 2024-05-27T08:14:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.905+0000 7fad9be00640 1 -- start start 2024-05-27T08:14:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.905+0000 7fad9be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad9c106420 0x7fad9c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.905+0000 7fad9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad9c10bcd0 0x7fad9c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.905+0000 7fad9be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad9c10c170 0x7fad9c1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.905+0000 7fad9be00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c115dc0 con 0x7fad9c10c170 2024-05-27T08:14:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.905+0000 7fad9be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c115c40 con 0x7fad9c10bcd0 2024-05-27T08:14:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.905+0000 7fad9be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c115f40 con 0x7fad9c106420 2024-05-27T08:14:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad9c10c170 0x7fad9c1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad9c10c170 0x7fad9c1a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42576/0 (socket says 172.21.15.63:42576) 2024-05-27T08:14:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9b400640 1 -- 172.21.15.63:0/2322416498 learned_addr learned my addr 172.21.15.63:0/2322416498 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9aa00640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad9c106420 0x7fad9c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9a000640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad9c10bcd0 0x7fad9c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9b400640 1 -- 172.21.15.63:0/2322416498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad9c106420 msgr2=0x7fad9c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9b400640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad9c106420 0x7fad9c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9b400640 1 -- 172.21.15.63:0/2322416498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad9c10bcd0 msgr2=0x7fad9c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9b400640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad9c10bcd0 0x7fad9c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9b400640 1 -- 172.21.15.63:0/2322416498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c1a1580 con 0x7fad9c10c170 2024-05-27T08:14:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.906+0000 7fad9aa00640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad9c106420 0x7fad9c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.907+0000 7fad9b400640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad9c10c170 0x7fad9c1a0fc0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7fad94032bc0 tx=0x7fad94032bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.907+0000 7fad93400640 1 -- 172.21.15.63:0/2322416498 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94004040 con 0x7fad9c10c170 2024-05-27T08:14:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.907+0000 7fad93400640 1 -- 172.21.15.63:0/2322416498 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad94004670 con 0x7fad9c10c170 2024-05-27T08:14:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.907+0000 7fad93400640 1 -- 172.21.15.63:0/2322416498 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94030d50 con 0x7fad9c10c170 2024-05-27T08:14:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.907+0000 7fad9be00640 1 -- 172.21.15.63:0/2322416498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9c1a1810 con 0x7fad9c10c170 2024-05-27T08:14:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.907+0000 7fad9be00640 1 -- 172.21.15.63:0/2322416498 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9c1a1d10 con 0x7fad9c10c170 2024-05-27T08:14:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.911+0000 7fada8d7a740 1 Processor -- start 2024-05-27T08:14:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.911+0000 7fad93400640 1 -- 172.21.15.63:0/2322416498 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fad940305c0 con 0x7fad9c10c170 2024-05-27T08:14:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.911+0000 7fada8d7a740 1 -- start start 2024-05-27T08:14:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.912+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 0x55b93e0a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.912+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dcdc2f0 0x55b93e0a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.912+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dce44d0 0x55b93e0a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.912+0000 7fada8d7a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b93e018b90 con 0x55b93dcdc2f0 2024-05-27T08:14:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.912+0000 7fada8d7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b93e008730 con 0x55b93dce44d0 2024-05-27T08:14:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.912+0000 7fada8d7a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b93e00c770 con 0x55b93dd051b0 2024-05-27T08:14:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.912+0000 7fad9aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dcdc2f0 0x55b93e0a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.912+0000 7fad9aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dcdc2f0 0x55b93e0a7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42578/0 (socket says 172.21.15.63:42578) 2024-05-27T08:14:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.912+0000 7fad9aa00640 1 -- 172.21.15.63:0/3069451068 learned_addr learned my addr 172.21.15.63:0/3069451068 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.912+0000 7fad9a000640 1 --2- 172.21.15.63:0/3069451068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 0x55b93e0a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.913+0000 7fad9a000640 1 --2- 172.21.15.63:0/3069451068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dce44d0 0x55b93e0a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.913+0000 7fad9aa00640 1 -- 172.21.15.63:0/3069451068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 msgr2=0x55b93e0a7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.913+0000 7fad9aa00640 1 --2- 172.21.15.63:0/3069451068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 0x55b93e0a7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.913+0000 7fad9aa00640 1 -- 172.21.15.63:0/3069451068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dce44d0 msgr2=0x55b93e0a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.913+0000 7fad9aa00640 1 --2- 172.21.15.63:0/3069451068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dce44d0 0x55b93e0a84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.913+0000 7fad9aa00640 1 -- 172.21.15.63:0/3069451068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93dfa18b0 con 0x55b93dcdc2f0 2024-05-27T08:14:55.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.913+0000 7fad93400640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fad800b29d0 0x7fad800b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.913+0000 7fad9aa00640 1 --2- 172.21.15.63:0/3069451068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dcdc2f0 0x55b93e0a7fa0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7fad8c008c10 tx=0x7fad8c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7608818478e57d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.914+0000 7fad93400640 1 -- 172.21.15.63:0/2322416498 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fad94103ff0 con 0x7fad9c10c170 2024-05-27T08:14:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.914+0000 7fad9a000640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fad800b29d0 0x7fad800b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.914+0000 7fad87e00640 1 -- 172.21.15.63:0/3069451068 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c014070 con 0x55b93dcdc2f0 2024-05-27T08:14:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.914+0000 7fad87e00640 1 -- 172.21.15.63:0/3069451068 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad8c0047a0 con 0x55b93dcdc2f0 2024-05-27T08:14:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.914+0000 7fad87e00640 1 -- 172.21.15.63:0/3069451068 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c010690 con 0x55b93dcdc2f0 2024-05-27T08:14:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.915+0000 7fada8d7a740 1 -- 172.21.15.63:0/3069451068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dcdc2f0 msgr2=0x55b93e0a7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.915+0000 7fad9a000640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fad800b29d0 0x7fad800b4ee0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fad8800efd0 tx=0x7fad88005dc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.915+0000 7fada8d7a740 1 --2- 172.21.15.63:0/3069451068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dcdc2f0 0x55b93e0a7fa0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7fad8c008c10 tx=0x7fad8c00e720 comp rx=0 tx=0).stop 2024-05-27T08:14:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.915+0000 7fada8d7a740 1 -- 172.21.15.63:0/3069451068 shutdown_connections 2024-05-27T08:14:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.915+0000 7fada8d7a740 1 --2- 172.21.15.63:0/3069451068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dce44d0 0x55b93e0a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.915+0000 7fada8d7a740 1 --2- 172.21.15.63:0/3069451068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dcdc2f0 0x55b93e0a7fa0 unknown :-1 s=CLOSED pgs=2928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.915+0000 7fada8d7a740 1 --2- 172.21.15.63:0/3069451068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 0x55b93e0a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.915+0000 7fada8d7a740 1 -- 172.21.15.63:0/3069451068 >> 172.21.15.63:0/3069451068 conn(0x55b93dcf9300 msgr2=0x55b93e0a4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.915+0000 7fada8d7a740 1 -- 172.21.15.63:0/3069451068 shutdown_connections 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.916+0000 7fada8d7a740 1 -- 172.21.15.63:0/3069451068 wait complete. 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.916+0000 7fada8d7a740 1 -- start start 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.916+0000 7fada8d7a740 1 Processor -- start 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.917+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.917+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dce44d0 0x55b93e0b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.917+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 0x55b93e0bdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.917+0000 7fada8d7a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b93dcee790 con 0x55b93dce44d0 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.917+0000 7fada8d7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b93dfb0bc0 con 0x55b93dcdc2f0 2024-05-27T08:14:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.917+0000 7fada8d7a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b93e0b4b90 con 0x55b93dd051b0 2024-05-27T08:14:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.917+0000 7fada8d7a740 10 client.0 _ll_register_callbacks cb 0x55b93df8e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.917+0000 7fad9a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dce44d0 0x55b93e0b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.917+0000 7fad9a000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dce44d0 0x55b93e0b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42590/0 (socket says 172.21.15.63:42590) 2024-05-27T08:14:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.917+0000 7fad9a000640 1 -- 172.21.15.63:0/2449895965 learned_addr learned my addr 172.21.15.63:0/2449895965 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad9b400640 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 0x55b93e0bdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad9aa00640 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad9a000640 1 -- 172.21.15.63:0/2449895965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 msgr2=0x55b93e0bdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad9a000640 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 0x55b93e0bdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad9a000640 1 -- 172.21.15.63:0/2449895965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 msgr2=0x55b93e0b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad9a000640 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad9a000640 1 -- 172.21.15.63:0/2449895965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93dd1d210 con 0x55b93dce44d0 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad9aa00640 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad9b400640 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 0x55b93e0bdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad9a000640 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dce44d0 0x55b93e0b23e0 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7fad9c10c5a0 tx=0x7fad880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3b31d1319cb7e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.918+0000 7fad84c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.919+0000 7fad84c00640 1 -- 172.21.15.63:0/2449895965 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88024070 con 0x55b93dce44d0 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.919+0000 7fad84c00640 1 -- 172.21.15.63:0/2449895965 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fad88020480 con 0x55b93dce44d0 2024-05-27T08:14:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.919+0000 7fada8d7a740 1 -- 172.21.15.63:0/2449895965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad8c0047a0 con 0x55b93dce44d0 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.919+0000 7fad84c00640 1 -- 172.21.15.63:0/2449895965 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fad88030050 con 0x55b93dce44d0 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.919+0000 7fada8d7a740 10 client.23340 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.919+0000 7fada8d7a740 1 -- 172.21.15.63:0/2449895965 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b93dde9620 con 0x55b93dce44d0 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.920+0000 7fad84c00640 1 -- 172.21.15.63:0/2449895965 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad8c0047a0 con 0x55b93dce44d0 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.920+0000 7fad84c00640 1 -- 172.21.15.63:0/2449895965 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fad8801d050 con 0x55b93dce44d0 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.920+0000 7fada8d7a740 10 client.23340 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.920+0000 7fada8d7a740 10 client.23340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.920+0000 7fada8d7a740 20 client.23340 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.920+0000 7fada8d7a740 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93dcc3d80 0x55b93e0c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.920+0000 7fada8d7a740 4 client.23340 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.920+0000 7fada8d7a740 1 -- 172.21.15.63:0/2449895965 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b93e01aae0 con 0x55b93dcc3d80 2024-05-27T08:14:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.921+0000 7fad9aa00640 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93dcc3d80 0x55b93e0c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.922+0000 7fad84c00640 10 client.23340 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:55.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.922+0000 7fad9aa00640 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93dcc3d80 0x55b93e0c3f30 crc :-1 s=READY pgs=734 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c2c3d79e8531f9 server_cookie=b757cc7eed08e2f7 in_seq=0 out_seq=0 2024-05-27T08:14:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.924+0000 7fad84c00640 1 -- 172.21.15.63:0/2449895965 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b93e01aae0 con 0x55b93dcc3d80 2024-05-27T08:14:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.924+0000 7fad84c00640 10 client.23340 handle_command_reply: tid=0 2024-05-27T08:14:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.924+0000 7fada8d7a740 1 client.23340 shutdown 2024-05-27T08:14:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.924+0000 7fada8d7a740 1 -- 172.21.15.63:0/2449895965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dce44d0 msgr2=0x55b93e0b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.924+0000 7fada8d7a740 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dce44d0 0x55b93e0b23e0 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7fad9c10c5a0 tx=0x7fad880046c0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.925+0000 7fada8d7a740 1 -- 172.21.15.63:0/2449895965 shutdown_connections 2024-05-27T08:14:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.925+0000 7fada8d7a740 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93dcc3d80 0x55b93e0c3f30 crc :-1 s=READY pgs=734 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.925+0000 7fada8d7a740 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dd051b0 0x55b93e0bdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.925+0000 7fada8d7a740 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dce44d0 0x55b93e0b23e0 unknown :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.925+0000 7fada8d7a740 1 --2- 172.21.15.63:0/2449895965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.925+0000 7fada8d7a740 1 -- 172.21.15.63:0/2449895965 >> 172.21.15.63:0/2449895965 conn(0x55b93dcf9300 msgr2=0x55b93e0a7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.925+0000 7fad84c00640 0 client.23340 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.925+0000 7fada8d7a740 1 -- 172.21.15.63:0/2449895965 shutdown_connections 2024-05-27T08:14:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.925+0000 7fada8d7a740 1 -- 172.21.15.63:0/2449895965 wait complete. 2024-05-27T08:14:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.925+0000 7fada8d7a740 20 client.23340 trim_cache size 0 max 16384 2024-05-27T08:14:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.948+0000 7fada8d7a740 1 Processor -- start 2024-05-27T08:14:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.948+0000 7fada8d7a740 1 -- start start 2024-05-27T08:14:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.949+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0a7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.949+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dce44d0 0x55b93e0bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.949+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0be170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.949+0000 7fada8d7a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1b5a30 con 0x55b93dd051b0 2024-05-27T08:14:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.949+0000 7fada8d7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1b3300 con 0x55b93dcdc2f0 2024-05-27T08:14:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.949+0000 7fada8d7a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1b1170 con 0x55b93dce44d0 2024-05-27T08:14:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.949+0000 7fad9b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0be170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.949+0000 7fad9b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0be170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42598/0 (socket says 172.21.15.63:42598) 2024-05-27T08:14:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.949+0000 7fad9b400640 1 -- 172.21.15.63:0/1002716390 learned_addr learned my addr 172.21.15.63:0/1002716390 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.949+0000 7fad9aa00640 1 --2- 172.21.15.63:0/1002716390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0a7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.950+0000 7fad9a000640 1 --2- 172.21.15.63:0/1002716390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dce44d0 0x55b93e0bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.950+0000 7fad9b400640 1 -- 172.21.15.63:0/1002716390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dce44d0 msgr2=0x55b93e0bdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.950+0000 7fad9b400640 1 --2- 172.21.15.63:0/1002716390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dce44d0 0x55b93e0bdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.950+0000 7fad9b400640 1 -- 172.21.15.63:0/1002716390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 msgr2=0x55b93e0a7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.950+0000 7fad9b400640 1 --2- 172.21.15.63:0/1002716390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0a7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.950+0000 7fad9b400640 1 -- 172.21.15.63:0/1002716390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93dcc3fb0 con 0x55b93dd051b0 2024-05-27T08:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.950+0000 7fad9b400640 1 --2- 172.21.15.63:0/1002716390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0be170 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7fad940caf50 tx=0x7fad94009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e3e824a1af80de server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.950+0000 7fad87400640 1 -- 172.21.15.63:0/1002716390 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad940cb6c0 con 0x55b93dd051b0 2024-05-27T08:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.951+0000 7fad87400640 1 -- 172.21.15.63:0/1002716390 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad940e5070 con 0x55b93dd051b0 2024-05-27T08:14:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.951+0000 7fad87400640 1 -- 172.21.15.63:0/1002716390 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9403a390 con 0x55b93dd051b0 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.952+0000 7fada8d7a740 1 -- 172.21.15.63:0/1002716390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 msgr2=0x55b93e0be170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.952+0000 7fada8d7a740 1 --2- 172.21.15.63:0/1002716390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0be170 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7fad940caf50 tx=0x7fad94009900 comp rx=0 tx=0).stop 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.952+0000 7fada8d7a740 1 -- 172.21.15.63:0/1002716390 shutdown_connections 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.952+0000 7fada8d7a740 1 --2- 172.21.15.63:0/1002716390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0be170 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.952+0000 7fada8d7a740 1 --2- 172.21.15.63:0/1002716390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dce44d0 0x55b93e0bdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.952+0000 7fada8d7a740 1 --2- 172.21.15.63:0/1002716390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0a7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.952+0000 7fada8d7a740 1 -- 172.21.15.63:0/1002716390 >> 172.21.15.63:0/1002716390 conn(0x55b93dcf9300 msgr2=0x55b93e0a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.952+0000 7fada8d7a740 1 -- 172.21.15.63:0/1002716390 shutdown_connections 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.952+0000 7fada8d7a740 1 -- 172.21.15.63:0/1002716390 wait complete. 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.952+0000 7fada8d7a740 1 -- start start 2024-05-27T08:14:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.953+0000 7fada8d7a740 1 Processor -- start 2024-05-27T08:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.953+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.953+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dce44d0 0x55b93e0b7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.953+0000 7fada8d7a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.953+0000 7fada8d7a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1b17c0 con 0x55b93dd051b0 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.953+0000 7fada8d7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1b2c30 con 0x55b93dcdc2f0 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.953+0000 7fada8d7a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1b1e90 con 0x55b93dce44d0 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.953+0000 7fada8d7a740 10 client.0 _ll_register_callbacks cb 0x7fad8800ed50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.954+0000 7fad9b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.954+0000 7fad9b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42614/0 (socket says 172.21.15.63:42614) 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.954+0000 7fad9b400640 1 -- 172.21.15.63:0/1389321716 learned_addr learned my addr 172.21.15.63:0/1389321716 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.954+0000 7fad9aa00640 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.954+0000 7fad9a000640 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dce44d0 0x55b93e0b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.954+0000 7fad9b400640 1 -- 172.21.15.63:0/1389321716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dce44d0 msgr2=0x55b93e0b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.954+0000 7fad9b400640 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dce44d0 0x55b93e0b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.954+0000 7fad9b400640 1 -- 172.21.15.63:0/1389321716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 msgr2=0x55b93e0b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:14:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.954+0000 7fad9b400640 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.954+0000 7fad9b400640 1 -- 172.21.15.63:0/1389321716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93e12a740 con 0x55b93dd051b0 2024-05-27T08:14:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.955+0000 7fad9b400640 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0b77b0 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7fad940cb180 tx=0x7fad940c74e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffb0aa4750719c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:14:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.955+0000 7fad84c00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:14:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.955+0000 7fad84c00640 1 -- 172.21.15.63:0/1389321716 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad940e9030 con 0x55b93dd051b0 2024-05-27T08:14:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.955+0000 7fad84c00640 1 -- 172.21.15.63:0/1389321716 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 9483+0+0 (secure 0 0 0) 0x7fad940c75b0 con 0x55b93dd051b0 2024-05-27T08:14:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.955+0000 7fada8d7a740 1 -- 172.21.15.63:0/1389321716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad940e5070 con 0x55b93dd051b0 2024-05-27T08:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.955+0000 7fad84c00640 1 -- 172.21.15.63:0/1389321716 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fad9403a5a0 con 0x55b93dd051b0 2024-05-27T08:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.955+0000 7fada8d7a740 10 client.23346 fetch_fsmap learned FSMap version 35 2024-05-27T08:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.955+0000 7fada8d7a740 1 -- 172.21.15.63:0/1389321716 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b93e12b9b0 con 0x55b93dd051b0 2024-05-27T08:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.956+0000 7fad84c00640 1 -- 172.21.15.63:0/1389321716 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad940e5070 con 0x55b93dd051b0 2024-05-27T08:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.956+0000 7fad84c00640 1 -- 172.21.15.63:0/1389321716 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fad940cd070 con 0x55b93dd051b0 2024-05-27T08:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.956+0000 7fada8d7a740 10 client.23346 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.956+0000 7fada8d7a740 10 client.23346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.956+0000 7fada8d7a740 20 client.23346 populate_metadata read hostname 'smithi063' 2024-05-27T08:14:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.956+0000 7fada8d7a740 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93dc991c0 0x55b93e12e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.956+0000 7fada8d7a740 4 client.23346 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.956+0000 7fada8d7a740 1 -- 172.21.15.63:0/1389321716 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b93e0142c0 con 0x55b93dc991c0 2024-05-27T08:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.957+0000 7fad9aa00640 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93dc991c0 0x55b93e12e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.958+0000 7fad9aa00640 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93dc991c0 0x55b93e12e0c0 crc :-1 s=READY pgs=735 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53879dd48af8b934 server_cookie=30885aa0cdcb4a9f in_seq=0 out_seq=0 2024-05-27T08:14:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.958+0000 7fad84c00640 10 client.23346 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.996+0000 7fad84c00640 1 -- 172.21.15.63:0/1389321716 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b93e0142c0 con 0x55b93dc991c0 2024-05-27T08:14:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.996+0000 7fad84c00640 10 client.23346 handle_command_reply: tid=0 2024-05-27T08:14:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.996+0000 7fada8d7a740 1 client.23346 shutdown 2024-05-27T08:14:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.997+0000 7fada8d7a740 1 -- 172.21.15.63:0/1389321716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 msgr2=0x55b93e0b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.997+0000 7fada8d7a740 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0b77b0 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7fad940cb180 tx=0x7fad940c74e0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.997+0000 7fada8d7a740 1 -- 172.21.15.63:0/1389321716 shutdown_connections 2024-05-27T08:14:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.997+0000 7fada8d7a740 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b93dc991c0 0x55b93e12e0c0 crc :-1 s=READY pgs=735 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.997+0000 7fada8d7a740 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b93dd051b0 0x55b93e0b77b0 unknown :-1 s=CLOSED pgs=2931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.997+0000 7fada8d7a740 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b93dce44d0 0x55b93e0b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.997+0000 7fada8d7a740 1 --2- 172.21.15.63:0/1389321716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93dcdc2f0 0x55b93e0b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.997+0000 7fada8d7a740 1 -- 172.21.15.63:0/1389321716 >> 172.21.15.63:0/1389321716 conn(0x55b93dcf9300 msgr2=0x55b93e0b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.997+0000 7fad84c00640 0 client.23346 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:14:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.997+0000 7fada8d7a740 1 -- 172.21.15.63:0/1389321716 shutdown_connections 2024-05-27T08:14:55.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.998+0000 7fada8d7a740 1 -- 172.21.15.63:0/1389321716 wait complete. 2024-05-27T08:14:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.998+0000 7fada8d7a740 20 client.23346 trim_cache size 0 max 16384 2024-05-27T08:14:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:14:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:14:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 -- 172.21.15.63:0/2322416498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fad800b29d0 msgr2=0x7fad800b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fad800b29d0 0x7fad800b4ee0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fad8800efd0 tx=0x7fad88005dc0 comp rx=0 tx=0).stop 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 -- 172.21.15.63:0/2322416498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad9c10c170 msgr2=0x7fad9c1a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad9c10c170 0x7fad9c1a0fc0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7fad94032bc0 tx=0x7fad94032bf0 comp rx=0 tx=0).stop 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 -- 172.21.15.63:0/2322416498 shutdown_connections 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fad800b29d0 0x7fad800b4ee0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad9c10c170 0x7fad9c1a0fc0 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad9c10bcd0 0x7fad9c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 --2- 172.21.15.63:0/2322416498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad9c106420 0x7fad9c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 -- 172.21.15.63:0/2322416498 >> 172.21.15.63:0/2322416498 conn(0x7fad9c1013d0 msgr2=0x7fad9c103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:14:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:55.999+0000 7fad9be00640 1 -- 172.21.15.63:0/2322416498 shutdown_connections 2024-05-27T08:14:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:14:56.000+0000 7fad9be00640 1 -- 172.21.15.63:0/2322416498 wait complete. 2024-05-27T08:14:56.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:55 smithi086 ceph-mon[34687]: pgmap v1312: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.7 MiB/s wr, 46 op/s 2024-05-27T08:14:56.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:55 smithi149 ceph-mon[35408]: pgmap v1312: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.7 MiB/s wr, 46 op/s 2024-05-27T08:14:56.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:55 smithi063 ceph-mon[31613]: pgmap v1312: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.7 MiB/s wr, 46 op/s 2024-05-27T08:14:56.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:56 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:14:56.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:56 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:14:57.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:56 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:14:57.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:56 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:14:57.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:56 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:14:57.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:56 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:14:58.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:57 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:14:58.193 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:57 smithi063 ceph-mon[31613]: pgmap v1313: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.0 MiB/s wr, 53 op/s 2024-05-27T08:14:58.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:57 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:14:58.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:57 smithi086 ceph-mon[34687]: pgmap v1313: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.0 MiB/s wr, 53 op/s 2024-05-27T08:14:58.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:57 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:14:58.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:57 smithi149 ceph-mon[35408]: pgmap v1313: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.0 MiB/s wr, 53 op/s 2024-05-27T08:14:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:14:58 smithi086 ceph-mon[34687]: pgmap v1314: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.7 MiB/s wr, 45 op/s 2024-05-27T08:14:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:14:58 smithi149 ceph-mon[35408]: pgmap v1314: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.7 MiB/s wr, 45 op/s 2024-05-27T08:14:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:14:58 smithi063 ceph-mon[31613]: pgmap v1314: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.7 MiB/s wr, 45 op/s 2024-05-27T08:15:02.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:01 smithi086 ceph-mon[34687]: pgmap v1315: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.7 MiB/s wr, 45 op/s 2024-05-27T08:15:02.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:01 smithi149 ceph-mon[35408]: pgmap v1315: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.7 MiB/s wr, 45 op/s 2024-05-27T08:15:02.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:01 smithi063 ceph-mon[31613]: pgmap v1315: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.7 MiB/s wr, 45 op/s 2024-05-27T08:15:04.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:03 smithi086 ceph-mon[34687]: pgmap v1316: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.8 MiB/s wr, 66 op/s 2024-05-27T08:15:04.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:03 smithi149 ceph-mon[35408]: pgmap v1316: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.8 MiB/s wr, 66 op/s 2024-05-27T08:15:04.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:03 smithi063 ceph-mon[31613]: pgmap v1316: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.8 MiB/s wr, 66 op/s 2024-05-27T08:15:05.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:04 smithi086 ceph-mon[34687]: pgmap v1317: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.4 MiB/s wr, 28 op/s 2024-05-27T08:15:05.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:04 smithi149 ceph-mon[35408]: pgmap v1317: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.4 MiB/s wr, 28 op/s 2024-05-27T08:15:05.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:04 smithi063 ceph-mon[31613]: pgmap v1317: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.4 MiB/s wr, 28 op/s 2024-05-27T08:15:07.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:07 smithi086 ceph-mon[34687]: osdmap e163: 12 total, 12 up, 12 in 2024-05-27T08:15:07.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:07 smithi086 ceph-mon[34687]: pgmap v1319: 129 pgs: 15 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 110 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.9 MiB/s wr, 29 op/s 2024-05-27T08:15:07.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:07 smithi149 ceph-mon[35408]: osdmap e163: 12 total, 12 up, 12 in 2024-05-27T08:15:07.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:07 smithi149 ceph-mon[35408]: pgmap v1319: 129 pgs: 15 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 110 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.9 MiB/s wr, 29 op/s 2024-05-27T08:15:07.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:07 smithi063 ceph-mon[31613]: osdmap e163: 12 total, 12 up, 12 in 2024-05-27T08:15:07.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:07 smithi063 ceph-mon[31613]: pgmap v1319: 129 pgs: 15 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 110 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.9 MiB/s wr, 29 op/s 2024-05-27T08:15:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.279+0000 7fd80f200640 1 -- 172.21.15.63:0/3945733196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd808109aa0 msgr2=0x7fd808103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.279+0000 7fd80f200640 1 --2- 172.21.15.63:0/3945733196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd808109aa0 0x7fd808103640 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009900 tx=0x7fd7fc02a3f0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.280+0000 7fd80f200640 1 -- 172.21.15.63:0/3945733196 shutdown_connections 2024-05-27T08:15:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.280+0000 7fd80f200640 1 --2- 172.21.15.63:0/3945733196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8081041f0 0x7fd8081105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.280+0000 7fd80f200640 1 --2- 172.21.15.63:0/3945733196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd808109f40 0x7fd808103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.280+0000 7fd80f200640 1 --2- 172.21.15.63:0/3945733196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd808109aa0 0x7fd808103640 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.280+0000 7fd80f200640 1 -- 172.21.15.63:0/3945733196 >> 172.21.15.63:0/3945733196 conn(0x7fd8080791a0 msgr2=0x7fd8080795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.280+0000 7fd80f200640 1 -- 172.21.15.63:0/3945733196 shutdown_connections 2024-05-27T08:15:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.280+0000 7fd80f200640 1 -- 172.21.15.63:0/3945733196 wait complete. 2024-05-27T08:15:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.281+0000 7fd80f200640 1 Processor -- start 2024-05-27T08:15:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.281+0000 7fd80f200640 1 -- start start 2024-05-27T08:15:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.281+0000 7fd80f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8081041f0 0x7fd80819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.282+0000 7fd80f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd808109aa0 0x7fd80819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.282+0000 7fd80f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd808109f40 0x7fd8081a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.282+0000 7fd80f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd808113b40 con 0x7fd8081041f0 2024-05-27T08:15:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.282+0000 7fd80f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd8081139c0 con 0x7fd808109aa0 2024-05-27T08:15:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.282+0000 7fd80f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd808113cc0 con 0x7fd808109f40 2024-05-27T08:15:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.282+0000 7fd80de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8081041f0 0x7fd80819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.282+0000 7fd80de00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8081041f0 0x7fd80819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40428/0 (socket says 172.21.15.63:40428) 2024-05-27T08:15:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.282+0000 7fd80de00640 1 -- 172.21.15.63:0/2750376119 learned_addr learned my addr 172.21.15.63:0/2750376119 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.282+0000 7fd80d400640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd808109aa0 0x7fd80819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.282+0000 7fd80e800640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd808109f40 0x7fd8081a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.283+0000 7fd80de00640 1 -- 172.21.15.63:0/2750376119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd808109f40 msgr2=0x7fd8081a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.283+0000 7fd80de00640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd808109f40 0x7fd8081a8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.283+0000 7fd80de00640 1 -- 172.21.15.63:0/2750376119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd808109aa0 msgr2=0x7fd80819db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.283+0000 7fd80de00640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd808109aa0 0x7fd80819db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.283+0000 7fd80de00640 1 -- 172.21.15.63:0/2750376119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8081a89c0 con 0x7fd8081041f0 2024-05-27T08:15:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.283+0000 7fd80d400640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd808109aa0 0x7fd80819db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:15:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.283+0000 7fd80de00640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8081041f0 0x7fd80819d5e0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005940 tx=0x7fd7fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.283+0000 7fd802a00640 1 -- 172.21.15.63:0/2750376119 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc03f030 con 0x7fd8081041f0 2024-05-27T08:15:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.284+0000 7fd80f200640 1 -- 172.21.15.63:0/2750376119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8081a8b90 con 0x7fd8081041f0 2024-05-27T08:15:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.284+0000 7fd80f200640 1 -- 172.21.15.63:0/2750376119 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8081a90d0 con 0x7fd8081041f0 2024-05-27T08:15:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.284+0000 7fd802a00640 1 -- 172.21.15.63:0/2750376119 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7fc003c90 con 0x7fd8081041f0 2024-05-27T08:15:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.284+0000 7fd802a00640 1 -- 172.21.15.63:0/2750376119 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc035070 con 0x7fd8081041f0 2024-05-27T08:15:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.287+0000 7fd817795740 1 Processor -- start 2024-05-27T08:15:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.288+0000 7fd817795740 1 -- start start 2024-05-27T08:15:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.288+0000 7fd817795740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecded3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.288+0000 7fd817795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecded3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.288+0000 7fd802a00640 1 -- 172.21.15.63:0/2750376119 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd7fc042030 con 0x7fd8081041f0 2024-05-27T08:15:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.288+0000 7fd817795740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb104d0 0x561ecded44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.288+0000 7fd817795740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561ecde44b90 con 0x561ecdb311b0 2024-05-27T08:15:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.288+0000 7fd817795740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ecde34730 con 0x561ecdb082f0 2024-05-27T08:15:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.288+0000 7fd817795740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561ecde38770 con 0x561ecdb104d0 2024-05-27T08:15:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.289+0000 7fd80e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecded3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.289+0000 7fd80e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecded3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40432/0 (socket says 172.21.15.63:40432) 2024-05-27T08:15:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.289+0000 7fd80e800640 1 -- 172.21.15.63:0/3804451091 learned_addr learned my addr 172.21.15.63:0/3804451091 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.289+0000 7fd80de00640 1 --2- 172.21.15.63:0/3804451091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecded3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.289+0000 7fd80d400640 1 --2- 172.21.15.63:0/3804451091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb104d0 0x561ecded44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.289+0000 7fd80e800640 1 -- 172.21.15.63:0/3804451091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb104d0 msgr2=0x561ecded44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.289+0000 7fd80e800640 1 --2- 172.21.15.63:0/3804451091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb104d0 0x561ecded44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.289+0000 7fd80e800640 1 -- 172.21.15.63:0/3804451091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 msgr2=0x561ecded3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.289+0000 7fd80e800640 1 --2- 172.21.15.63:0/3804451091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecded3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.289+0000 7fd80e800640 1 -- 172.21.15.63:0/3804451091 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ecddcd8b0 con 0x561ecdb311b0 2024-05-27T08:15:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.290+0000 7fd80e800640 1 --2- 172.21.15.63:0/3804451091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecded3a60 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7fd804007e50 tx=0x7fd80400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6b8686c6eec783 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.290+0000 7fd7f7000640 1 -- 172.21.15.63:0/3804451091 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804014070 con 0x561ecdb311b0 2024-05-27T08:15:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.290+0000 7fd7f7000640 1 -- 172.21.15.63:0/3804451091 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd80400ee90 con 0x561ecdb311b0 2024-05-27T08:15:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.290+0000 7fd7f7000640 1 -- 172.21.15.63:0/3804451091 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804005310 con 0x561ecdb311b0 2024-05-27T08:15:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.290+0000 7fd802a00640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7f00b29d0 0x7fd7f00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.291+0000 7fd80de00640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7f00b29d0 0x7fd7f00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.291+0000 7fd802a00640 1 -- 172.21.15.63:0/2750376119 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 9571+0+0 (secure 0 0 0) 0x7fd7fc1032f0 con 0x7fd8081041f0 2024-05-27T08:15:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.291+0000 7fd817795740 1 -- 172.21.15.63:0/3804451091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 msgr2=0x561ecded3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.291+0000 7fd817795740 1 --2- 172.21.15.63:0/3804451091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecded3a60 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7fd804007e50 tx=0x7fd80400e720 comp rx=0 tx=0).stop 2024-05-27T08:15:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.292+0000 7fd817795740 1 -- 172.21.15.63:0/3804451091 shutdown_connections 2024-05-27T08:15:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.292+0000 7fd817795740 1 --2- 172.21.15.63:0/3804451091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb104d0 0x561ecded44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.292+0000 7fd817795740 1 --2- 172.21.15.63:0/3804451091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecded3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.292+0000 7fd817795740 1 --2- 172.21.15.63:0/3804451091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecded3a60 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.292+0000 7fd817795740 1 -- 172.21.15.63:0/3804451091 >> 172.21.15.63:0/3804451091 conn(0x561ecdb25300 msgr2=0x561ecded0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.292+0000 7fd80de00640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7f00b29d0 0x7fd7f00b4ee0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc1006d0 tx=0x7fd7fc105800 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.292+0000 7fd817795740 1 -- 172.21.15.63:0/3804451091 shutdown_connections 2024-05-27T08:15:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.292+0000 7fd817795740 1 -- 172.21.15.63:0/3804451091 wait complete. 2024-05-27T08:15:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.292+0000 7fd817795740 1 -- start start 2024-05-27T08:15:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.293+0000 7fd817795740 1 Processor -- start 2024-05-27T08:15:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.293+0000 7fd817795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecdeddba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.293+0000 7fd817795740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb104d0 0x561ecdede0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.293+0000 7fd817795740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb311b0 0x561ecdee9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.293+0000 7fd817795740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561ecdb1a790 con 0x561ecdb104d0 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.293+0000 7fd817795740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ecdddcbc0 con 0x561ecdb082f0 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.293+0000 7fd817795740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561ecdee0b90 con 0x561ecdb311b0 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.293+0000 7fd817795740 10 client.0 _ll_register_callbacks cb 0x561ecddba400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.294+0000 7fd80d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb104d0 0x561ecdede0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.294+0000 7fd80d400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb104d0 0x561ecdede0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40444/0 (socket says 172.21.15.63:40444) 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.294+0000 7fd80d400640 1 -- 172.21.15.63:0/741865630 learned_addr learned my addr 172.21.15.63:0/741865630 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.294+0000 7fd80e800640 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecdeddba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.294+0000 7fd80e800640 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb311b0 0x561ecdee9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.294+0000 7fd80d400640 1 -- 172.21.15.63:0/741865630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb311b0 msgr2=0x561ecdee9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.294+0000 7fd80d400640 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb311b0 0x561ecdee9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.294+0000 7fd80d400640 1 -- 172.21.15.63:0/741865630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 msgr2=0x561ecdeddba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.294+0000 7fd80d400640 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecdeddba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.294+0000 7fd80d400640 1 -- 172.21.15.63:0/741865630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ecdb49210 con 0x561ecdb104d0 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.295+0000 7fd80d400640 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb104d0 0x561ecdede0e0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800cde0 tx=0x7fd7f80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2979ad0240b4b66b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.295+0000 7fd7eb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.295+0000 7fd7eb400640 1 -- 172.21.15.63:0/741865630 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8005220 con 0x561ecdb104d0 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.295+0000 7fd7eb400640 1 -- 172.21.15.63:0/741865630 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(163..163 src has 1..163) ==== 9571+0+0 (secure 0 0 0) 0x7fd7f80053c0 con 0x561ecdb104d0 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.295+0000 7fd817795740 1 -- 172.21.15.63:0/741865630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80400ee90 con 0x561ecdb104d0 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.295+0000 7fd7eb400640 1 -- 172.21.15.63:0/741865630 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd7f80055c0 con 0x561ecdb104d0 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.295+0000 7fd817795740 10 client.23358 fetch_fsmap learned FSMap version 35 2024-05-27T08:15:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.295+0000 7fd817795740 1 -- 172.21.15.63:0/741865630 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561ecdc15620 con 0x561ecdb104d0 2024-05-27T08:15:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.296+0000 7fd7eb400640 1 -- 172.21.15.63:0/741865630 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd80400ee90 con 0x561ecdb104d0 2024-05-27T08:15:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.296+0000 7fd7eb400640 1 -- 172.21.15.63:0/741865630 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd7f8018070 con 0x561ecdb104d0 2024-05-27T08:15:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.296+0000 7fd817795740 10 client.23358 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:15:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.296+0000 7fd817795740 10 client.23358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:15:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.296+0000 7fd817795740 20 client.23358 populate_metadata read hostname 'smithi063' 2024-05-27T08:15:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.296+0000 7fd817795740 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ecdaefd80 0x561ecdeeff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.296+0000 7fd817795740 4 client.23358 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:15:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.296+0000 7fd817795740 1 -- 172.21.15.63:0/741865630 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ecde46ae0 con 0x561ecdaefd80 2024-05-27T08:15:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.297+0000 7fd80e800640 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ecdaefd80 0x561ecdeeff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.298+0000 7fd80e800640 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ecdaefd80 0x561ecdeeff10 crc :-1 s=READY pgs=736 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914f24cf80b93313 server_cookie=e8c4d4650198d518 in_seq=0 out_seq=0 2024-05-27T08:15:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.298+0000 7fd7eb400640 10 client.23358 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd7eb400640 1 -- 172.21.15.63:0/741865630 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561ecde46ae0 con 0x561ecdaefd80 2024-05-27T08:15:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd7eb400640 10 client.23358 handle_command_reply: tid=0 2024-05-27T08:15:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd817795740 1 client.23358 shutdown 2024-05-27T08:15:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd817795740 1 -- 172.21.15.63:0/741865630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb104d0 msgr2=0x561ecdede0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd817795740 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb104d0 0x561ecdede0e0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800cde0 tx=0x7fd7f80039c0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd817795740 1 -- 172.21.15.63:0/741865630 shutdown_connections 2024-05-27T08:15:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd817795740 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ecdaefd80 0x561ecdeeff10 crc :-1 s=READY pgs=736 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd817795740 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb311b0 0x561ecdee9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd817795740 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb104d0 0x561ecdede0e0 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd817795740 1 --2- 172.21.15.63:0/741865630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecdeddba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd7eb400640 0 client.23358 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.300+0000 7fd817795740 1 -- 172.21.15.63:0/741865630 >> 172.21.15.63:0/741865630 conn(0x561ecdb25300 msgr2=0x561ecded3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.301+0000 7fd817795740 1 -- 172.21.15.63:0/741865630 shutdown_connections 2024-05-27T08:15:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.301+0000 7fd817795740 1 -- 172.21.15.63:0/741865630 wait complete. 2024-05-27T08:15:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.301+0000 7fd817795740 20 client.23358 trim_cache size 0 max 16384 2024-05-27T08:15:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.321+0000 7fd817795740 1 Processor -- start 2024-05-27T08:15:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.321+0000 7fd817795740 1 -- start start 2024-05-27T08:15:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.322+0000 7fd817795740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb082f0 0x561ecdee02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.322+0000 7fd817795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb104d0 0x561ecded60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.322+0000 7fd817795740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb311b0 0x561ecdece810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.322+0000 7fd817795740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec0128b0 con 0x561ecdb082f0 2024-05-27T08:15:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.322+0000 7fd817795740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec010180 con 0x561ecdb104d0 2024-05-27T08:15:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.322+0000 7fd817795740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec00dff0 con 0x561ecdb311b0 2024-05-27T08:15:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.322+0000 7fd80e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb082f0 0x561ecdee02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.323+0000 7fd80e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb082f0 0x561ecdee02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40452/0 (socket says 172.21.15.63:40452) 2024-05-27T08:15:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.323+0000 7fd80e800640 1 -- 172.21.15.63:0/3347287315 learned_addr learned my addr 172.21.15.63:0/3347287315 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.323+0000 7fd80d400640 1 --2- 172.21.15.63:0/3347287315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb104d0 0x561ecded60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.323+0000 7fd80e800640 1 --2- 172.21.15.63:0/3347287315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb311b0 0x561ecdece810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.323+0000 7fd80e800640 1 -- 172.21.15.63:0/3347287315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb311b0 msgr2=0x561ecdece810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.323+0000 7fd80e800640 1 --2- 172.21.15.63:0/3347287315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb311b0 0x561ecdece810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.323+0000 7fd80e800640 1 -- 172.21.15.63:0/3347287315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb104d0 msgr2=0x561ecded60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.323+0000 7fd80e800640 1 --2- 172.21.15.63:0/3347287315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb104d0 0x561ecded60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.323+0000 7fd80e800640 1 -- 172.21.15.63:0/3347287315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ecdeca2a0 con 0x561ecdb082f0 2024-05-27T08:15:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.324+0000 7fd80e800640 1 --2- 172.21.15.63:0/3347287315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb082f0 0x561ecdee02d0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fd804007e50 tx=0x7fd804010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf38af874841620 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.324+0000 7fd7f6600640 1 -- 172.21.15.63:0/3347287315 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804014040 con 0x561ecdb082f0 2024-05-27T08:15:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.324+0000 7fd7f6600640 1 -- 172.21.15.63:0/3347287315 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd80400ee90 con 0x561ecdb082f0 2024-05-27T08:15:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.324+0000 7fd7f6600640 1 -- 172.21.15.63:0/3347287315 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804021040 con 0x561ecdb082f0 2024-05-27T08:15:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.325+0000 7fd817795740 1 -- 172.21.15.63:0/3347287315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb082f0 msgr2=0x561ecdee02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.325+0000 7fd817795740 1 --2- 172.21.15.63:0/3347287315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb082f0 0x561ecdee02d0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fd804007e50 tx=0x7fd804010ea0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.325+0000 7fd817795740 1 -- 172.21.15.63:0/3347287315 shutdown_connections 2024-05-27T08:15:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.325+0000 7fd817795740 1 --2- 172.21.15.63:0/3347287315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb311b0 0x561ecdece810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.325+0000 7fd817795740 1 --2- 172.21.15.63:0/3347287315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb104d0 0x561ecded60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.325+0000 7fd817795740 1 --2- 172.21.15.63:0/3347287315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb082f0 0x561ecdee02d0 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.325+0000 7fd817795740 1 -- 172.21.15.63:0/3347287315 >> 172.21.15.63:0/3347287315 conn(0x561ecdb25300 msgr2=0x561ecded87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.325+0000 7fd817795740 1 -- 172.21.15.63:0/3347287315 shutdown_connections 2024-05-27T08:15:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.325+0000 7fd817795740 1 -- 172.21.15.63:0/3347287315 wait complete. 2024-05-27T08:15:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.326+0000 7fd817795740 1 -- start start 2024-05-27T08:15:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.326+0000 7fd817795740 1 Processor -- start 2024-05-27T08:15:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd817795740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecdee0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd817795740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb104d0 0x561ecdee1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd817795740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecdee1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd817795740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec00e640 con 0x561ecdb311b0 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd817795740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec00fab0 con 0x561ecdb082f0 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd817795740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec00ed10 con 0x561ecdb104d0 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd817795740 10 client.0 _ll_register_callbacks cb 0x7fd7f800ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd80e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecdee1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd80e800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecdee1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40456/0 (socket says 172.21.15.63:40456) 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd80e800640 1 -- 172.21.15.63:0/4000991197 learned_addr learned my addr 172.21.15.63:0/4000991197 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd80e800640 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecdee0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.327+0000 7fd80d400640 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb104d0 0x561ecdee1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.328+0000 7fd80e800640 1 -- 172.21.15.63:0/4000991197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb104d0 msgr2=0x561ecdee1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.328+0000 7fd80e800640 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb104d0 0x561ecdee1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.328+0000 7fd80e800640 1 -- 172.21.15.63:0/4000991197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 msgr2=0x561ecdee0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.328+0000 7fd80e800640 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecdee0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.328+0000 7fd80e800640 1 -- 172.21.15.63:0/4000991197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ecdf56750 con 0x561ecdb311b0 2024-05-27T08:15:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.328+0000 7fd80e800640 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecdee1860 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7fd804007d70 tx=0x7fd804000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1396863530304010 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:09.330 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.328+0000 7fd7eb400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:15:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.328+0000 7fd7eb400640 1 -- 172.21.15.63:0/4000991197 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804014030 con 0x561ecdb311b0 2024-05-27T08:15:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.329+0000 7fd7eb400640 1 -- 172.21.15.63:0/4000991197 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(163..163 src has 1..163) ==== 9571+0+0 (secure 0 0 0) 0x7fd804010840 con 0x561ecdb311b0 2024-05-27T08:15:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.329+0000 7fd817795740 1 -- 172.21.15.63:0/4000991197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80400ee90 con 0x561ecdb311b0 2024-05-27T08:15:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.329+0000 7fd7eb400640 1 -- 172.21.15.63:0/4000991197 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd804010a40 con 0x561ecdb311b0 2024-05-27T08:15:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.329+0000 7fd817795740 10 client.23364 fetch_fsmap learned FSMap version 35 2024-05-27T08:15:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.329+0000 7fd817795740 1 -- 172.21.15.63:0/4000991197 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561ecdf579c0 con 0x561ecdb311b0 2024-05-27T08:15:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.329+0000 7fd7eb400640 1 -- 172.21.15.63:0/4000991197 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd80400ee90 con 0x561ecdb311b0 2024-05-27T08:15:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.329+0000 7fd7eb400640 1 -- 172.21.15.63:0/4000991197 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd804030030 con 0x561ecdb311b0 2024-05-27T08:15:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.330+0000 7fd817795740 10 client.23364 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:15:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.330+0000 7fd817795740 10 client.23364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:15:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.330+0000 7fd817795740 20 client.23364 populate_metadata read hostname 'smithi063' 2024-05-27T08:15:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.330+0000 7fd817795740 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ecdac51c0 0x561ecdf59dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.330+0000 7fd817795740 4 client.23364 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:15:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.330+0000 7fd817795740 1 -- 172.21.15.63:0/4000991197 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ecde402c0 con 0x561ecdac51c0 2024-05-27T08:15:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.330+0000 7fd80d400640 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ecdac51c0 0x561ecdf59dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.331+0000 7fd80d400640 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ecdac51c0 0x561ecdf59dd0 crc :-1 s=READY pgs=737 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a161535d752ec3 server_cookie=4592bf5034528b16 in_seq=0 out_seq=0 2024-05-27T08:15:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.331+0000 7fd7eb400640 10 client.23364 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.372+0000 7fd7eb400640 1 -- 172.21.15.63:0/4000991197 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561ecde402c0 con 0x561ecdac51c0 2024-05-27T08:15:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.372+0000 7fd7eb400640 10 client.23364 handle_command_reply: tid=0 2024-05-27T08:15:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.372+0000 7fd817795740 1 client.23364 shutdown 2024-05-27T08:15:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 1 -- 172.21.15.63:0/4000991197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 msgr2=0x561ecdee1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecdee1860 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7fd804007d70 tx=0x7fd804000f40 comp rx=0 tx=0).stop 2024-05-27T08:15:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 1 -- 172.21.15.63:0/4000991197 shutdown_connections 2024-05-27T08:15:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561ecdac51c0 0x561ecdf59dd0 crc :-1 s=READY pgs=737 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ecdb311b0 0x561ecdee1860 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd7eb400640 0 client.23364 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561ecdb104d0 0x561ecdee1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 1 --2- 172.21.15.63:0/4000991197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ecdb082f0 0x561ecdee0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 1 -- 172.21.15.63:0/4000991197 >> 172.21.15.63:0/4000991197 conn(0x561ecdb25300 msgr2=0x561ecded5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 1 -- 172.21.15.63:0/4000991197 shutdown_connections 2024-05-27T08:15:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 1 -- 172.21.15.63:0/4000991197 wait complete. 2024-05-27T08:15:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.373+0000 7fd817795740 20 client.23364 trim_cache size 0 max 16384 2024-05-27T08:15:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:15:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:15:09.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:15:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.374+0000 7fd80f200640 1 -- 172.21.15.63:0/2750376119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7f00b29d0 msgr2=0x7fd7f00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.374+0000 7fd80f200640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7f00b29d0 0x7fd7f00b4ee0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc1006d0 tx=0x7fd7fc105800 comp rx=0 tx=0).stop 2024-05-27T08:15:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.374+0000 7fd80f200640 1 -- 172.21.15.63:0/2750376119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8081041f0 msgr2=0x7fd80819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.374+0000 7fd80f200640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8081041f0 0x7fd80819d5e0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005940 tx=0x7fd7fc0042f0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.375+0000 7fd80f200640 1 -- 172.21.15.63:0/2750376119 shutdown_connections 2024-05-27T08:15:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.375+0000 7fd80f200640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd7f00b29d0 0x7fd7f00b4ee0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.375+0000 7fd80f200640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd808109f40 0x7fd8081a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.375+0000 7fd80f200640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd808109aa0 0x7fd80819db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.375+0000 7fd80f200640 1 --2- 172.21.15.63:0/2750376119 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8081041f0 0x7fd80819d5e0 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.375+0000 7fd80f200640 1 -- 172.21.15.63:0/2750376119 >> 172.21.15.63:0/2750376119 conn(0x7fd8080791a0 msgr2=0x7fd808076340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.375+0000 7fd80f200640 1 -- 172.21.15.63:0/2750376119 shutdown_connections 2024-05-27T08:15:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:09.375+0000 7fd80f200640 1 -- 172.21.15.63:0/2750376119 wait complete. 2024-05-27T08:15:10.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:09 smithi086 ceph-mon[34687]: pgmap v1320: 129 pgs: 15 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 110 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.9 MiB/s wr, 29 op/s 2024-05-27T08:15:10.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:09 smithi149 ceph-mon[35408]: pgmap v1320: 129 pgs: 15 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 110 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.9 MiB/s wr, 29 op/s 2024-05-27T08:15:10.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:09 smithi063 ceph-mon[31613]: pgmap v1320: 129 pgs: 15 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 110 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.9 MiB/s wr, 29 op/s 2024-05-27T08:15:11.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:10 smithi086 ceph-mon[34687]: pgmap v1321: 129 pgs: 15 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 110 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.9 MiB/s wr, 29 op/s 2024-05-27T08:15:11.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:10 smithi149 ceph-mon[35408]: pgmap v1321: 129 pgs: 15 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 110 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.9 MiB/s wr, 29 op/s 2024-05-27T08:15:11.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:10 smithi063 ceph-mon[31613]: pgmap v1321: 129 pgs: 15 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 110 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.9 MiB/s wr, 29 op/s 2024-05-27T08:15:14.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:13 smithi086 ceph-mon[34687]: pgmap v1322: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:15:14.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:13 smithi149 ceph-mon[35408]: pgmap v1322: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:15:14.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:13 smithi063 ceph-mon[31613]: pgmap v1322: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:15:15.145 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:14 smithi063 ceph-mon[31613]: pgmap v1323: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:15:15.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:14 smithi086 ceph-mon[34687]: pgmap v1323: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:15:15.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:14 smithi149 ceph-mon[35408]: pgmap v1323: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-05-27T08:15:17.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:17 smithi086 ceph-mon[34687]: osdmap e164: 12 total, 12 up, 12 in 2024-05-27T08:15:17.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:17 smithi086 ceph-mon[34687]: pgmap v1325: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 12 op/s 2024-05-27T08:15:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:17 smithi149 ceph-mon[35408]: osdmap e164: 12 total, 12 up, 12 in 2024-05-27T08:15:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:17 smithi149 ceph-mon[35408]: pgmap v1325: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 12 op/s 2024-05-27T08:15:17.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:17 smithi063 ceph-mon[31613]: osdmap e164: 12 total, 12 up, 12 in 2024-05-27T08:15:17.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:17 smithi063 ceph-mon[31613]: pgmap v1325: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 12 op/s 2024-05-27T08:15:20.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:19 smithi086 ceph-mon[34687]: pgmap v1326: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 12 op/s 2024-05-27T08:15:20.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:19 smithi149 ceph-mon[35408]: pgmap v1326: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 12 op/s 2024-05-27T08:15:20.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:19 smithi063 ceph-mon[31613]: pgmap v1326: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 12 op/s 2024-05-27T08:15:20.591 INFO:tasks.workunit.client.0.smithi063.stdout:fs setup took 2331 secs 2024-05-27T08:15:21.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:20 smithi086 ceph-mon[34687]: pgmap v1327: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 12 op/s 2024-05-27T08:15:21.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:20 smithi149 ceph-mon[35408]: pgmap v1327: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 12 op/s 2024-05-27T08:15:21.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:20 smithi063 ceph-mon[31613]: pgmap v1327: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.9 MiB/s wr, 12 op/s 2024-05-27T08:15:21.265 INFO:tasks.workunit.client.0.smithi063.stdout:Syncing()...0 sec 2024-05-27T08:15:21.269 INFO:tasks.workunit.client.0.smithi063.stdout:Starting Actual Benchmark At: Mon May 27 08:15:21 2024 2024-05-27T08:15:21.269 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:15:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.989+0000 7f72be600640 1 -- 172.21.15.63:0/1384399519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b81029a0 msgr2=0x7f72b8102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.989+0000 7f72be600640 1 --2- 172.21.15.63:0/1384399519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b81029a0 0x7f72b8102db0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009900 tx=0x7f72a802a410 comp rx=0 tx=0).stop 2024-05-27T08:15:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.989+0000 7f72be600640 1 -- 172.21.15.63:0/1384399519 shutdown_connections 2024-05-27T08:15:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.989+0000 7f72be600640 1 --2- 172.21.15.63:0/1384399519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b8107e20 0x7f72b81082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.989+0000 7f72be600640 1 --2- 172.21.15.63:0/1384399519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b81073f0 0x7f72b8107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.989+0000 7f72be600640 1 --2- 172.21.15.63:0/1384399519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b81029a0 0x7f72b8102db0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.989+0000 7f72be600640 1 -- 172.21.15.63:0/1384399519 >> 172.21.15.63:0/1384399519 conn(0x7f72b80fe210 msgr2=0x7f72b8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.990+0000 7f72be600640 1 -- 172.21.15.63:0/1384399519 shutdown_connections 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.990+0000 7f72be600640 1 -- 172.21.15.63:0/1384399519 wait complete. 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.991+0000 7f72be600640 1 Processor -- start 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.992+0000 7f72be600640 1 -- start start 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.992+0000 7f72be600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b81029a0 0x7f72b8074990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.992+0000 7f72be600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b81073f0 0x7f72b806db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.992+0000 7f72be600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b8107e20 0x7f72b806e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.992+0000 7f72be600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72b811a1c0 con 0x7f72b8107e20 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.992+0000 7f72be600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72b811a040 con 0x7f72b81073f0 2024-05-27T08:15:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.992+0000 7f72be600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72b811a340 con 0x7f72b81029a0 2024-05-27T08:15:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.993+0000 7f72bdc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b8107e20 0x7f72b806e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.993+0000 7f72bdc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b8107e20 0x7f72b806e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37098/0 (socket says 172.21.15.63:37098) 2024-05-27T08:15:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.993+0000 7f72bdc00640 1 -- 172.21.15.63:0/2195092320 learned_addr learned my addr 172.21.15.63:0/2195092320 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.993+0000 7f72bd200640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b81029a0 0x7f72b8074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.994+0000 7f72b7e00640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b81073f0 0x7f72b806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.994+0000 7f72bdc00640 1 -- 172.21.15.63:0/2195092320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b81029a0 msgr2=0x7f72b8074990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.994+0000 7f72bdc00640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b81029a0 0x7f72b8074990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.994+0000 7f72bdc00640 1 -- 172.21.15.63:0/2195092320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b81073f0 msgr2=0x7f72b806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.994+0000 7f72bdc00640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b81073f0 0x7f72b806db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.994+0000 7f72bdc00640 1 -- 172.21.15.63:0/2195092320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b80724b0 con 0x7f72b8107e20 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.994+0000 7f72bdc00640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b8107e20 0x7f72b806e230 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009fd0 tx=0x7f72a400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.995+0000 7f72b5600640 1 -- 172.21.15.63:0/2195092320 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4011830 con 0x7f72b8107e20 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.995+0000 7f72b5600640 1 -- 172.21.15.63:0/2195092320 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a4016070 con 0x7f72b8107e20 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.995+0000 7f72b5600640 1 -- 172.21.15.63:0/2195092320 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4010d20 con 0x7f72b8107e20 2024-05-27T08:15:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.996+0000 7f72be600640 1 -- 172.21.15.63:0/2195092320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b80727a0 con 0x7f72b8107e20 2024-05-27T08:15:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:22.996+0000 7f72be600640 1 -- 172.21.15.63:0/2195092320 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b8072ca0 con 0x7f72b8107e20 2024-05-27T08:15:23.010 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.009+0000 7f72c6b96740 1 Processor -- start 2024-05-27T08:15:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.015+0000 7f72b5600640 1 -- 172.21.15.63:0/2195092320 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f72a401b460 con 0x7f72b8107e20 2024-05-27T08:15:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.020+0000 7f72c6b96740 1 -- start start 2024-05-27T08:15:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.021+0000 7f72b5600640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b2ae0 0x7f729c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.021+0000 7f72b5600640 1 -- 172.21.15.63:0/2195092320 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f72a40e1f40 con 0x7f72b8107e20 2024-05-27T08:15:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.022+0000 7f72b7e00640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b2ae0 0x7f729c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.022+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e42d1b0 0x55a57e7cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.022+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e4042f0 0x55a57e7cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.022+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 0x55a57e7d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.023+0000 7f72c6b96740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a57e740b90 con 0x55a57e40c4d0 2024-05-27T08:15:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.023+0000 7f72c6b96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a57e730730 con 0x55a57e4042f0 2024-05-27T08:15:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.023+0000 7f72c6b96740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a57e734770 con 0x55a57e42d1b0 2024-05-27T08:15:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.025+0000 7f72bd200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e42d1b0 0x55a57e7cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.025+0000 7f72bd200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e42d1b0 0x55a57e7cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:39228/0 (socket says 172.21.15.63:39228) 2024-05-27T08:15:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.025+0000 7f72bd200640 1 -- 172.21.15.63:0/789047210 learned_addr learned my addr 172.21.15.63:0/789047210 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.026+0000 7f72b7e00640 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e4042f0 0x55a57e7cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.027+0000 7f72bdc00640 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 0x55a57e7d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.028+0000 7f72bd200640 1 -- 172.21.15.63:0/789047210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e4042f0 msgr2=0x55a57e7cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.028+0000 7f72bd200640 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e4042f0 0x55a57e7cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.028+0000 7f72bd200640 1 -- 172.21.15.63:0/789047210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 msgr2=0x55a57e7d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.029+0000 7f72bd200640 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 0x55a57e7d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.031+0000 7f72bd200640 1 -- 172.21.15.63:0/789047210 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57e6c98b0 con 0x55a57e42d1b0 2024-05-27T08:15:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.031+0000 7f72b7e00640 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e4042f0 0x55a57e7cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:15:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.032+0000 7f72bdc00640 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 0x55a57e7d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:15:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.032+0000 7f72bd200640 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e42d1b0 0x55a57e7cfa60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f72a8005dc0 tx=0x7f72a80040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35df1f593f8a96ee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.033+0000 7f72b7e00640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b2ae0 0x7f729c0b4ff0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f72b81083e0 tx=0x7f72b0013900 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.033+0000 7f72ae000640 1 -- 172.21.15.63:0/789047210 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a803f070 con 0x55a57e42d1b0 2024-05-27T08:15:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.033+0000 7f72ae000640 1 -- 172.21.15.63:0/789047210 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a80054d0 con 0x55a57e42d1b0 2024-05-27T08:15:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.033+0000 7f72ae000640 1 -- 172.21.15.63:0/789047210 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8035030 con 0x55a57e42d1b0 2024-05-27T08:15:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.038+0000 7f72c6b96740 1 -- 172.21.15.63:0/789047210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e42d1b0 msgr2=0x55a57e7cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.038+0000 7f72c6b96740 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e42d1b0 0x55a57e7cfa60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f72a8005dc0 tx=0x7f72a80040f0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.039+0000 7f72c6b96740 1 -- 172.21.15.63:0/789047210 shutdown_connections 2024-05-27T08:15:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.039+0000 7f72c6b96740 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 0x55a57e7d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.039+0000 7f72c6b96740 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e4042f0 0x55a57e7cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.039+0000 7f72c6b96740 1 --2- 172.21.15.63:0/789047210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e42d1b0 0x55a57e7cfa60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.039+0000 7f72c6b96740 1 -- 172.21.15.63:0/789047210 >> 172.21.15.63:0/789047210 conn(0x55a57e421300 msgr2=0x55a57e7cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.040+0000 7f72c6b96740 1 -- 172.21.15.63:0/789047210 shutdown_connections 2024-05-27T08:15:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.040+0000 7f72c6b96740 1 -- 172.21.15.63:0/789047210 wait complete. 2024-05-27T08:15:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.042+0000 7f72c6b96740 1 -- start start 2024-05-27T08:15:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.043+0000 7f72c6b96740 1 Processor -- start 2024-05-27T08:15:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.045+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e4042f0 0x55a57e7d9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.045+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e40c4d0 0x55a57e7da1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.045+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 0x55a57e7e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.046+0000 7f72c6b96740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a57e416790 con 0x55a57e4042f0 2024-05-27T08:15:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.046+0000 7f72c6b96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a57e6d8bc0 con 0x55a57e42d1b0 2024-05-27T08:15:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.046+0000 7f72c6b96740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a57e7dcb30 con 0x55a57e40c4d0 2024-05-27T08:15:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.046+0000 7f72c6b96740 10 client.0 _ll_register_callbacks cb 0x55a57e6b6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:15:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.050+0000 7f72b7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e40c4d0 0x55a57e7da1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.050+0000 7f72b7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e40c4d0 0x55a57e7da1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:39244/0 (socket says 172.21.15.63:39244) 2024-05-27T08:15:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.050+0000 7f72b7e00640 1 -- 172.21.15.63:0/1681670445 learned_addr learned my addr 172.21.15.63:0/1681670445 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.051+0000 7f72bdc00640 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 0x55a57e7e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.052+0000 7f72b7e00640 1 -- 172.21.15.63:0/1681670445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 msgr2=0x55a57e7e5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.052+0000 7f72bd200640 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e4042f0 0x55a57e7d9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.052+0000 7f72b7e00640 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 0x55a57e7e5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.052+0000 7f72b7e00640 1 -- 172.21.15.63:0/1681670445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e4042f0 msgr2=0x55a57e7d9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.052+0000 7f72b7e00640 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e4042f0 0x55a57e7d9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.052+0000 7f72b7e00640 1 -- 172.21.15.63:0/1681670445 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57e445210 con 0x55a57e40c4d0 2024-05-27T08:15:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.053+0000 7f72bd200640 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e4042f0 0x55a57e7d9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:15:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.053+0000 7f72bdc00640 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 0x55a57e7e5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:15:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.054+0000 7f72a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:15:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.054+0000 7f72b7e00640 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e40c4d0 0x55a57e7da1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f72b0005fd0 tx=0x7f72b0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8904de681939da27 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.055+0000 7f72a2a00640 1 -- 172.21.15.63:0/1681670445 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0023030 con 0x55a57e40c4d0 2024-05-27T08:15:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.056+0000 7f72a2a00640 1 -- 172.21.15.63:0/1681670445 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f72b0024070 con 0x55a57e40c4d0 2024-05-27T08:15:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.058+0000 7f72c6b96740 1 -- 172.21.15.63:0/1681670445 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a80054d0 con 0x55a57e40c4d0 2024-05-27T08:15:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.059+0000 7f72a2a00640 1 -- 172.21.15.63:0/1681670445 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f72b0023030 con 0x55a57e40c4d0 2024-05-27T08:15:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.059+0000 7f72c6b96740 10 client.32041 fetch_fsmap learned FSMap version 35 2024-05-27T08:15:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.059+0000 7f72c6b96740 1 -- 172.21.15.63:0/1681670445 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a57e511620 con 0x55a57e40c4d0 2024-05-27T08:15:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.060+0000 7f72a2a00640 1 -- 172.21.15.63:0/1681670445 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72b0011bd0 con 0x55a57e40c4d0 2024-05-27T08:15:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.060+0000 7f72a2a00640 1 -- 172.21.15.63:0/1681670445 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f72b0032a30 con 0x55a57e40c4d0 2024-05-27T08:15:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.061+0000 7f72c6b96740 10 client.32041 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:15:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.061+0000 7f72c6b96740 10 client.32041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:15:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.061+0000 7f72c6b96740 20 client.32041 populate_metadata read hostname 'smithi063' 2024-05-27T08:15:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.061+0000 7f72c6b96740 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a57e3ebd80 0x55a57e7ebba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.061+0000 7f72c6b96740 4 client.32041 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:15:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.061+0000 7f72c6b96740 1 -- 172.21.15.63:0/1681670445 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f72a80054d0 con 0x55a57e3ebd80 2024-05-27T08:15:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.062+0000 7f72bd200640 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a57e3ebd80 0x55a57e7ebba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.064+0000 7f72bd200640 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a57e3ebd80 0x55a57e7ebba0 crc :-1 s=READY pgs=738 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59886f1cb28438a server_cookie=4a1f29bddf5f9d8d in_seq=0 out_seq=0 2024-05-27T08:15:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.064+0000 7f72a2a00640 10 client.32041 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.066+0000 7f72a2a00640 1 -- 172.21.15.63:0/1681670445 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f72a80054d0 con 0x55a57e3ebd80 2024-05-27T08:15:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.066+0000 7f72a2a00640 10 client.32041 handle_command_reply: tid=0 2024-05-27T08:15:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.066+0000 7f72c6b96740 1 client.32041 shutdown 2024-05-27T08:15:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.069+0000 7f72c6b96740 1 -- 172.21.15.63:0/1681670445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e40c4d0 msgr2=0x55a57e7da1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.069+0000 7f72c6b96740 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e40c4d0 0x55a57e7da1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f72b0005fd0 tx=0x7f72b0004ab0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.069+0000 7f72c6b96740 1 -- 172.21.15.63:0/1681670445 shutdown_connections 2024-05-27T08:15:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.069+0000 7f72c6b96740 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a57e3ebd80 0x55a57e7ebba0 crc :-1 s=READY pgs=738 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.069+0000 7f72c6b96740 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 0x55a57e7e5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.069+0000 7f72c6b96740 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e40c4d0 0x55a57e7da1c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.069+0000 7f72c6b96740 1 --2- 172.21.15.63:0/1681670445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e4042f0 0x55a57e7d9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.069+0000 7f72c6b96740 1 -- 172.21.15.63:0/1681670445 >> 172.21.15.63:0/1681670445 conn(0x55a57e421300 msgr2=0x55a57e7cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.069+0000 7f72a2a00640 0 client.32041 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.070+0000 7f72c6b96740 1 -- 172.21.15.63:0/1681670445 shutdown_connections 2024-05-27T08:15:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.071+0000 7f72c6b96740 1 -- 172.21.15.63:0/1681670445 wait complete. 2024-05-27T08:15:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.071+0000 7f72c6b96740 20 client.32041 trim_cache size 0 max 16384 2024-05-27T08:15:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.151+0000 7f72c6b96740 1 Processor -- start 2024-05-27T08:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.152+0000 7f72c6b96740 1 -- start start 2024-05-27T08:15:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.153+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7dc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.153+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 0x55a57e7cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.153+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 0x55a57e7e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.153+0000 7f72bd200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7dc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.153+0000 7f72bd200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7dc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:39260/0 (socket says 172.21.15.63:39260) 2024-05-27T08:15:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.154+0000 7f72bd200640 1 -- 172.21.15.63:0/1903654007 learned_addr learned my addr 172.21.15.63:0/1903654007 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.155+0000 7f72c6b96740 1 -- 172.21.15.63:0/1903654007 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81c29c0 con 0x55a57e40c4d0 2024-05-27T08:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.155+0000 7f72c6b96740 1 -- 172.21.15.63:0/1903654007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81c0290 con 0x55a57e42d1b0 2024-05-27T08:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.155+0000 7f72c6b96740 1 -- 172.21.15.63:0/1903654007 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81be0c0 con 0x55a57e4042f0 2024-05-27T08:15:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.155+0000 7f72b7e00640 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 0x55a57e7cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.156+0000 7f72bdc00640 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 0x55a57e7e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.159+0000 7f72bd200640 1 -- 172.21.15.63:0/1903654007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 msgr2=0x55a57e7e5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.159+0000 7f72bd200640 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 0x55a57e7e5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.159+0000 7f72bd200640 1 -- 172.21.15.63:0/1903654007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 msgr2=0x55a57e7cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.159+0000 7f72bd200640 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 0x55a57e7cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.159+0000 7f72bd200640 1 -- 172.21.15.63:0/1903654007 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57e7d4150 con 0x55a57e4042f0 2024-05-27T08:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.160+0000 7f72bdc00640 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 0x55a57e7e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:15:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.160+0000 7f72b7e00640 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 0x55a57e7cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:15:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.161+0000 7f72bd200640 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7dc3b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72a80057c0 tx=0x7f72a802a910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ea694b51a424b3f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.162+0000 7f72ad600640 1 -- 172.21.15.63:0/1903654007 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8035040 con 0x55a57e4042f0 2024-05-27T08:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.162+0000 7f72ad600640 1 -- 172.21.15.63:0/1903654007 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a8003e90 con 0x55a57e4042f0 2024-05-27T08:15:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.162+0000 7f72ad600640 1 -- 172.21.15.63:0/1903654007 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a803f040 con 0x55a57e4042f0 2024-05-27T08:15:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.166+0000 7f72c6b96740 1 -- 172.21.15.63:0/1903654007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 msgr2=0x55a57e7dc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.166+0000 7f72c6b96740 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7dc3b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72a80057c0 tx=0x7f72a802a910 comp rx=0 tx=0).stop 2024-05-27T08:15:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.169+0000 7f72c6b96740 1 -- 172.21.15.63:0/1903654007 shutdown_connections 2024-05-27T08:15:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.169+0000 7f72c6b96740 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e42d1b0 0x55a57e7e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.169+0000 7f72c6b96740 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e40c4d0 0x55a57e7cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.169+0000 7f72c6b96740 1 --2- 172.21.15.63:0/1903654007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7dc3b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.169+0000 7f72c6b96740 1 -- 172.21.15.63:0/1903654007 >> 172.21.15.63:0/1903654007 conn(0x55a57e421300 msgr2=0x55a57e7cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.170+0000 7f72c6b96740 1 -- 172.21.15.63:0/1903654007 shutdown_connections 2024-05-27T08:15:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.170+0000 7f72c6b96740 1 -- 172.21.15.63:0/1903654007 wait complete. 2024-05-27T08:15:23.173 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.170+0000 7f72c6b96740 1 -- start start 2024-05-27T08:15:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.173+0000 7f72c6b96740 1 Processor -- start 2024-05-27T08:15:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.176+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7def70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.177+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e40c4d0 0x55a57e7df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.177+0000 7f72c6b96740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e42d1b0 0x55a57e852230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.177+0000 7f72c6b96740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81be750 con 0x55a57e42d1b0 2024-05-27T08:15:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.177+0000 7f72c6b96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81bfbc0 con 0x55a57e40c4d0 2024-05-27T08:15:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.177+0000 7f72c6b96740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81bee20 con 0x55a57e4042f0 2024-05-27T08:15:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.177+0000 7f72c6b96740 10 client.0 _ll_register_callbacks cb 0x7f72b00131b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:15:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.178+0000 7f72bd200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7def70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.178+0000 7f72bd200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7def70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:39264/0 (socket says 172.21.15.63:39264) 2024-05-27T08:15:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.178+0000 7f72bd200640 1 -- 172.21.15.63:0/4285180133 learned_addr learned my addr 172.21.15.63:0/4285180133 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.179+0000 7f72bdc00640 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e42d1b0 0x55a57e852230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.179+0000 7f72b7e00640 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e40c4d0 0x55a57e7df4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.181+0000 7f72bd200640 1 -- 172.21.15.63:0/4285180133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e40c4d0 msgr2=0x55a57e7df4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.181+0000 7f72bd200640 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e40c4d0 0x55a57e7df4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.181+0000 7f72bd200640 1 -- 172.21.15.63:0/4285180133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e42d1b0 msgr2=0x55a57e852230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.181+0000 7f72bd200640 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e42d1b0 0x55a57e852230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.181+0000 7f72bd200640 1 -- 172.21.15.63:0/4285180133 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57e7dfd90 con 0x55a57e4042f0 2024-05-27T08:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.182+0000 7f72bd200640 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7def70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72a80096c0 tx=0x7f72a80045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67dcaa07f0ccf118 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.182+0000 7f72a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:15:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.184+0000 7f72a2a00640 1 -- 172.21.15.63:0/4285180133 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8055070 con 0x55a57e4042f0 2024-05-27T08:15:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.184+0000 7f72a2a00640 1 -- 172.21.15.63:0/4285180133 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f72a8050620 con 0x55a57e4042f0 2024-05-27T08:15:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.184+0000 7f72c6b96740 1 -- 172.21.15.63:0/4285180133 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a8003e90 con 0x55a57e4042f0 2024-05-27T08:15:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.185+0000 7f72a2a00640 1 -- 172.21.15.63:0/4285180133 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f72a805e050 con 0x55a57e4042f0 2024-05-27T08:15:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.186+0000 7f72c6b96740 10 client.32047 fetch_fsmap learned FSMap version 35 2024-05-27T08:15:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.186+0000 7f72c6b96740 1 -- 172.21.15.63:0/4285180133 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a57e853780 con 0x55a57e4042f0 2024-05-27T08:15:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.186+0000 7f72a2a00640 1 -- 172.21.15.63:0/4285180133 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f72a8003e90 con 0x55a57e4042f0 2024-05-27T08:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.188+0000 7f72a2a00640 1 -- 172.21.15.63:0/4285180133 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f72a804d070 con 0x55a57e4042f0 2024-05-27T08:15:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.188+0000 7f72c6b96740 10 client.32047 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.188+0000 7f72c6b96740 10 client.32047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.188+0000 7f72c6b96740 20 client.32047 populate_metadata read hostname 'smithi063' 2024-05-27T08:15:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.188+0000 7f72c6b96740 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a57e3c11c0 0x55a57e855960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.188+0000 7f72c6b96740 4 client.32047 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.188+0000 7f72c6b96740 1 -- 172.21.15.63:0/4285180133 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a57e742ae0 con 0x55a57e3c11c0 2024-05-27T08:15:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.189+0000 7f72b7e00640 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a57e3c11c0 0x55a57e855960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.190+0000 7f72b7e00640 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a57e3c11c0 0x55a57e855960 crc :-1 s=READY pgs=739 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88365b5bb0789236 server_cookie=14b91fb4b8bd203b in_seq=0 out_seq=0 2024-05-27T08:15:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.191+0000 7f72a2a00640 10 client.32047 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:23.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:22 smithi086 ceph-mon[34687]: pgmap v1328: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-05-27T08:15:23.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:22 smithi149 ceph-mon[35408]: pgmap v1328: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-05-27T08:15:23.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:22 smithi063 ceph-mon[31613]: pgmap v1328: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-05-27T08:15:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.377+0000 7f72a2a00640 1 -- 172.21.15.63:0/4285180133 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a57e742ae0 con 0x55a57e3c11c0 2024-05-27T08:15:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.377+0000 7f72a2a00640 10 client.32047 handle_command_reply: tid=0 2024-05-27T08:15:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.378+0000 7f72c6b96740 1 client.32047 shutdown 2024-05-27T08:15:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.379+0000 7f72c6b96740 1 -- 172.21.15.63:0/4285180133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 msgr2=0x55a57e7def70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.379+0000 7f72c6b96740 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7def70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72a80096c0 tx=0x7f72a80045e0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.380+0000 7f72c6b96740 1 -- 172.21.15.63:0/4285180133 shutdown_connections 2024-05-27T08:15:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.380+0000 7f72c6b96740 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a57e3c11c0 0x55a57e855960 crc :-1 s=READY pgs=739 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.380+0000 7f72c6b96740 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a57e42d1b0 0x55a57e852230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.380+0000 7f72c6b96740 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57e40c4d0 0x55a57e7df4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.380+0000 7f72c6b96740 1 --2- 172.21.15.63:0/4285180133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a57e4042f0 0x55a57e7def70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.380+0000 7f72c6b96740 1 -- 172.21.15.63:0/4285180133 >> 172.21.15.63:0/4285180133 conn(0x55a57e421300 msgr2=0x55a57e7db310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.380+0000 7f72a2a00640 0 client.32047 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.382+0000 7f72c6b96740 1 -- 172.21.15.63:0/4285180133 shutdown_connections 2024-05-27T08:15:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.382+0000 7f72c6b96740 1 -- 172.21.15.63:0/4285180133 wait complete. 2024-05-27T08:15:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.384+0000 7f72c6b96740 20 client.32047 trim_cache size 0 max 16384 2024-05-27T08:15:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:15:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:15:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:15:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.395+0000 7f72be600640 1 -- 172.21.15.63:0/2195092320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b2ae0 msgr2=0x7f729c0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.395+0000 7f72be600640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b2ae0 0x7f729c0b4ff0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f72b81083e0 tx=0x7f72b0013900 comp rx=0 tx=0).stop 2024-05-27T08:15:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.395+0000 7f72be600640 1 -- 172.21.15.63:0/2195092320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b8107e20 msgr2=0x7f72b806e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.395+0000 7f72be600640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b8107e20 0x7f72b806e230 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009fd0 tx=0x7f72a400b8e0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.396+0000 7f72be600640 1 -- 172.21.15.63:0/2195092320 shutdown_connections 2024-05-27T08:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.396+0000 7f72be600640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f729c0b2ae0 0x7f729c0b4ff0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.396+0000 7f72be600640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72b8107e20 0x7f72b806e230 unknown :-1 s=CLOSED pgs=2938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.396+0000 7f72be600640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b81073f0 0x7f72b806db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.396+0000 7f72be600640 1 --2- 172.21.15.63:0/2195092320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72b81029a0 0x7f72b8074990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.396+0000 7f72be600640 1 -- 172.21.15.63:0/2195092320 >> 172.21.15.63:0/2195092320 conn(0x7f72b80fe210 msgr2=0x7f72b8105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.398+0000 7f72be600640 1 -- 172.21.15.63:0/2195092320 shutdown_connections 2024-05-27T08:15:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:23.398+0000 7f72be600640 1 -- 172.21.15.63:0/2195092320 wait complete. 2024-05-27T08:15:24.798 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:15:25.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:24 smithi086 ceph-mon[34687]: pgmap v1329: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-05-27T08:15:25.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:24 smithi149 ceph-mon[35408]: pgmap v1329: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-05-27T08:15:25.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:24 smithi063 ceph-mon[31613]: pgmap v1329: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.8 MiB/s wr, 10 op/s 2024-05-27T08:15:25.756 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:15:27.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:26 smithi086 ceph-mon[34687]: pgmap v1330: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 1.39k op/s 2024-05-27T08:15:27.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:26 smithi149 ceph-mon[35408]: pgmap v1330: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 1.39k op/s 2024-05-27T08:15:27.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:26 smithi063 ceph-mon[31613]: pgmap v1330: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 1.39k op/s 2024-05-27T08:15:29.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:28 smithi086 ceph-mon[34687]: pgmap v1331: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 1.20k op/s 2024-05-27T08:15:29.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:28 smithi149 ceph-mon[35408]: pgmap v1331: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 1.20k op/s 2024-05-27T08:15:29.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:28 smithi063 ceph-mon[31613]: pgmap v1331: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 1.20k op/s 2024-05-27T08:15:30.920 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:30 smithi063 ceph-mon[31613]: pgmap v1332: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 1.20k op/s 2024-05-27T08:15:31.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:30 smithi086 ceph-mon[34687]: pgmap v1332: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 1.20k op/s 2024-05-27T08:15:31.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:30 smithi149 ceph-mon[35408]: pgmap v1332: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 1.20k op/s 2024-05-27T08:15:34.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:33 smithi086 ceph-mon[34687]: pgmap v1333: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-05-27T08:15:34.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:33 smithi149 ceph-mon[35408]: pgmap v1333: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-05-27T08:15:34.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:33 smithi063 ceph-mon[31613]: pgmap v1333: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 2.53k op/s 2024-05-27T08:15:35.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:34 smithi086 ceph-mon[34687]: pgmap v1334: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-05-27T08:15:35.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:34 smithi149 ceph-mon[35408]: pgmap v1334: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-05-27T08:15:35.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:34 smithi063 ceph-mon[31613]: pgmap v1334: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-05-27T08:15:37.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:36 smithi086 ceph-mon[34687]: pgmap v1335: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-05-27T08:15:37.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:36 smithi149 ceph-mon[35408]: pgmap v1335: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-05-27T08:15:37.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:36 smithi063 ceph-mon[31613]: pgmap v1335: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-05-27T08:15:39.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:38 smithi086 ceph-mon[34687]: pgmap v1336: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-27T08:15:39.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:38 smithi149 ceph-mon[35408]: pgmap v1336: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-27T08:15:39.219 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:38 smithi063 ceph-mon[31613]: pgmap v1336: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-27T08:15:41.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:40 smithi086 ceph-mon[34687]: pgmap v1337: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-27T08:15:41.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:40 smithi149 ceph-mon[35408]: pgmap v1337: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-27T08:15:41.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:40 smithi063 ceph-mon[31613]: pgmap v1337: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-05-27T08:15:44.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:43 smithi086 ceph-mon[34687]: pgmap v1338: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 3.85k op/s 2024-05-27T08:15:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:43 smithi149 ceph-mon[35408]: pgmap v1338: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 3.85k op/s 2024-05-27T08:15:44.220 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:43 smithi063 ceph-mon[31613]: pgmap v1338: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 3.85k op/s 2024-05-27T08:15:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:15:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:44 smithi086 ceph-mon[34687]: pgmap v1339: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-05-27T08:15:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:15:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:44 smithi149 ceph-mon[35408]: pgmap v1339: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-05-27T08:15:45.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:15:45.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:44 smithi063 ceph-mon[31613]: pgmap v1339: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-05-27T08:15:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.579+0000 7f2ed1e00640 1 -- 172.21.15.63:0/165296676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ecc1029a0 msgr2=0x7f2ecc102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.579+0000 7f2ed1e00640 1 --2- 172.21.15.63:0/165296676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ecc1029a0 0x7f2ecc102db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009860 tx=0x7f2eb802a360 comp rx=0 tx=0).stop 2024-05-27T08:15:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.581+0000 7f2ed1e00640 1 -- 172.21.15.63:0/165296676 shutdown_connections 2024-05-27T08:15:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.581+0000 7f2ed1e00640 1 --2- 172.21.15.63:0/165296676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ecc107e20 0x7f2ecc1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.581+0000 7f2ed1e00640 1 --2- 172.21.15.63:0/165296676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ecc1073f0 0x7f2ecc107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.581+0000 7f2ed1e00640 1 --2- 172.21.15.63:0/165296676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ecc1029a0 0x7f2ecc102db0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.583 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.581+0000 7f2ed1e00640 1 -- 172.21.15.63:0/165296676 >> 172.21.15.63:0/165296676 conn(0x7f2ecc0fe210 msgr2=0x7f2ecc100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:46.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.583+0000 7f2ed1e00640 1 -- 172.21.15.63:0/165296676 shutdown_connections 2024-05-27T08:15:46.585 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.584+0000 7f2ed1e00640 1 -- 172.21.15.63:0/165296676 wait complete. 2024-05-27T08:15:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.587+0000 7f2ed1e00640 1 Processor -- start 2024-05-27T08:15:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.588+0000 7f2ed1e00640 1 -- start start 2024-05-27T08:15:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.588+0000 7f2ed1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ecc1029a0 0x7f2ecc116c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.589+0000 7f2ed1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ecc1073f0 0x7f2ecc110d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.589+0000 7f2ed1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ecc107e20 0x7f2ecc1113f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.589+0000 7f2ed1e00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc109120 con 0x7f2ecc107e20 2024-05-27T08:15:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.589+0000 7f2ed1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc108fa0 con 0x7f2ecc1029a0 2024-05-27T08:15:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.589+0000 7f2ed1e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc1092a0 con 0x7f2ecc1073f0 2024-05-27T08:15:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.589+0000 7f2ed0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ecc1029a0 0x7f2ecc116c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.589+0000 7f2ed0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ecc1029a0 0x7f2ecc116c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44432/0 (socket says 172.21.15.63:44432) 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.589+0000 7f2ed0a00640 1 -- 172.21.15.63:0/652574815 learned_addr learned my addr 172.21.15.63:0/652574815 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.590+0000 7f2ed0a00640 1 -- 172.21.15.63:0/652574815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ecc1073f0 msgr2=0x7f2ecc110d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.590+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ecc1073f0 0x7f2ecc110d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.590+0000 7f2ed0a00640 1 -- 172.21.15.63:0/652574815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ecc107e20 msgr2=0x7f2ecc1113f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.590+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ecc107e20 0x7f2ecc1113f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.590+0000 7f2ed0a00640 1 -- 172.21.15.63:0/652574815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ecc111b60 con 0x7f2ecc1029a0 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.590+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ecc1029a0 0x7f2ecc116c40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009460 tx=0x7f2eb8032cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.591+0000 7f2ec9600640 1 -- 172.21.15.63:0/652574815 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8040020 con 0x7f2ecc1029a0 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.591+0000 7f2ec9600640 1 -- 172.21.15.63:0/652574815 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2eb8038040 con 0x7f2ecc1029a0 2024-05-27T08:15:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.591+0000 7f2ec9600640 1 -- 172.21.15.63:0/652574815 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8037070 con 0x7f2ecc1029a0 2024-05-27T08:15:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.591+0000 7f2ed1e00640 1 -- 172.21.15.63:0/652574815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ecc1199c0 con 0x7f2ecc1029a0 2024-05-27T08:15:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.591+0000 7f2ed1e00640 1 -- 172.21.15.63:0/652574815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ecc119e30 con 0x7f2ecc1029a0 2024-05-27T08:15:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.598+0000 7f2eda579740 1 Processor -- start 2024-05-27T08:15:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.600+0000 7f2ec9600640 1 -- 172.21.15.63:0/652574815 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2eb8043020 con 0x7f2ecc1029a0 2024-05-27T08:15:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.601+0000 7f2eda579740 1 -- start start 2024-05-27T08:15:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.601+0000 7f2eda579740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c89042a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.601+0000 7f2eda579740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89042fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.602+0000 7f2eda579740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 0x557c890434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.602+0000 7f2eda579740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557c88fb3b90 con 0x557c88c772f0 2024-05-27T08:15:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.602+0000 7f2eda579740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557c88fa3730 con 0x557c88c7f4d0 2024-05-27T08:15:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.602+0000 7f2eda579740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557c88fa7770 con 0x557c88ca01b0 2024-05-27T08:15:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.603+0000 7f2ed1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c89042a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.603+0000 7f2ec9600640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2eb00d4af0 0x7f2eb00d7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.604+0000 7f2ed1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c89042a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:49340/0 (socket says 172.21.15.63:49340) 2024-05-27T08:15:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.604+0000 7f2ed1400640 1 -- 172.21.15.63:0/3590131242 learned_addr learned my addr 172.21.15.63:0/3590131242 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.604+0000 7f2ec9600640 1 -- 172.21.15.63:0/652574815 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f2eb81026f0 con 0x7f2ecc1029a0 2024-05-27T08:15:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.604+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/3590131242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89042fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.608+0000 7f2ed0a00640 1 -- 172.21.15.63:0/3590131242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 msgr2=0x557c89042a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.608+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/3590131242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c89042a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.608+0000 7f2ed0a00640 1 -- 172.21.15.63:0/3590131242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 msgr2=0x557c890434e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:15:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.608+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/3590131242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 0x557c890434e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.608+0000 7f2ed0a00640 1 -- 172.21.15.63:0/3590131242 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c88f3c8b0 con 0x557c88c772f0 2024-05-27T08:15:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.611+0000 7f2ed1400640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2eb00d4af0 0x7f2eb00d7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.611+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/3590131242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89042fa0 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80ffb90 tx=0x7f2eb8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f1cac6aaa8e768 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.615+0000 7f2ec1c00640 1 -- 172.21.15.63:0/3590131242 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80ca060 con 0x557c88c772f0 2024-05-27T08:15:46.617 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.616+0000 7f2ec1c00640 1 -- 172.21.15.63:0/3590131242 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2eb80cac50 con 0x557c88c772f0 2024-05-27T08:15:46.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.617+0000 7f2eda579740 1 -- 172.21.15.63:0/3590131242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 msgr2=0x557c89042fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:46.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.617+0000 7f2eda579740 1 --2- 172.21.15.63:0/3590131242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89042fa0 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80ffb90 tx=0x7f2eb8005f90 comp rx=0 tx=0).stop 2024-05-27T08:15:46.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.617+0000 7f2ed1400640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2eb00d4af0 0x7f2eb00d7000 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x557c8903fbd0 tx=0x7f2ec40095c0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:46.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.618+0000 7f2eda579740 1 -- 172.21.15.63:0/3590131242 shutdown_connections 2024-05-27T08:15:46.619 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.618+0000 7f2eda579740 1 --2- 172.21.15.63:0/3590131242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 0x557c890434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.618+0000 7f2eda579740 1 --2- 172.21.15.63:0/3590131242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89042fa0 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.618+0000 7f2eda579740 1 --2- 172.21.15.63:0/3590131242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c89042a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.618+0000 7f2eda579740 1 -- 172.21.15.63:0/3590131242 >> 172.21.15.63:0/3590131242 conn(0x557c88c94300 msgr2=0x557c8903f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.620+0000 7f2eda579740 1 -- 172.21.15.63:0/3590131242 shutdown_connections 2024-05-27T08:15:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.620+0000 7f2eda579740 1 -- 172.21.15.63:0/3590131242 wait complete. 2024-05-27T08:15:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.622+0000 7f2eda579740 1 -- start start 2024-05-27T08:15:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.623+0000 7f2eda579740 1 Processor -- start 2024-05-27T08:15:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.625+0000 7f2eda579740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c772f0 0x557c8904ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.626+0000 7f2eda579740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 0x557c8904cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.626+0000 7f2eda579740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88ca01b0 0x557c8904d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.627+0000 7f2ecbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 0x557c8904cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.627+0000 7f2ed1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88ca01b0 0x557c8904d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.628+0000 7f2ed1400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88ca01b0 0x557c8904d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44332/0 (socket says 172.21.15.63:44332) 2024-05-27T08:15:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.628+0000 7f2ed1400640 1 -- 172.21.15.63:0/596595848 learned_addr learned my addr 172.21.15.63:0/596595848 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.629+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557c88c89790 con 0x557c88ca01b0 2024-05-27T08:15:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.629+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557c88f4bbc0 con 0x557c88c772f0 2024-05-27T08:15:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.629+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557c88fb3b90 con 0x557c88c7f4d0 2024-05-27T08:15:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.629+0000 7f2eda579740 10 client.0 _ll_register_callbacks cb 0x557c88f29400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:15:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.629+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c772f0 0x557c8904ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.631+0000 7f2ed0a00640 1 -- 172.21.15.63:0/596595848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 msgr2=0x557c8904cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.631+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 0x557c8904cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.631+0000 7f2ed0a00640 1 -- 172.21.15.63:0/596595848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88ca01b0 msgr2=0x557c8904d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.631+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88ca01b0 0x557c8904d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.632+0000 7f2ed0a00640 1 -- 172.21.15.63:0/596595848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c88cb8210 con 0x557c88c772f0 2024-05-27T08:15:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.632+0000 7f2ed1400640 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88ca01b0 0x557c8904d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:15:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.633+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c772f0 0x557c8904ca20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80ca200 tx=0x7f2eb80c99e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8930dfe6aab60465 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.633+0000 7f2eb6000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:15:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.634+0000 7f2eb6000640 1 -- 172.21.15.63:0/596595848 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80e8030 con 0x557c88c772f0 2024-05-27T08:15:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.637+0000 7f2eb6000640 1 -- 172.21.15.63:0/596595848 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f2eb8101050 con 0x557c88c772f0 2024-05-27T08:15:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.637+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb80cac50 con 0x557c88c772f0 2024-05-27T08:15:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.638+0000 7f2eb6000640 1 -- 172.21.15.63:0/596595848 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2eb80d8050 con 0x557c88c772f0 2024-05-27T08:15:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.639+0000 7f2eda579740 10 client.42239 fetch_fsmap learned FSMap version 35 2024-05-27T08:15:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.639+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557c88d84620 con 0x557c88c772f0 2024-05-27T08:15:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.639+0000 7f2eb6000640 1 -- 172.21.15.63:0/596595848 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2eb80cac50 con 0x557c88c772f0 2024-05-27T08:15:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.639+0000 7f2eb6000640 1 -- 172.21.15.63:0/596595848 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2eb80df4f0 con 0x557c88c772f0 2024-05-27T08:15:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.640+0000 7f2eda579740 10 client.42239 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:15:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.640+0000 7f2eda579740 10 client.42239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:15:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.640+0000 7f2eda579740 20 client.42239 populate_metadata read hostname 'smithi063' 2024-05-27T08:15:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.640+0000 7f2eda579740 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557c88c5ed80 0x557c8905e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.640+0000 7f2eda579740 4 client.42239 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:15:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.640+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c88fb5ae0 con 0x557c88c5ed80 2024-05-27T08:15:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.641+0000 7f2ecbe00640 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557c88c5ed80 0x557c8905e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.642+0000 7f2ecbe00640 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557c88c5ed80 0x557c8905e940 crc :-1 s=READY pgs=740 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f01e66bef99629 server_cookie=7ca0e7455cdd32ac in_seq=0 out_seq=0 2024-05-27T08:15:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.642+0000 7f2eb6000640 10 client.42239 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.644+0000 7f2eb6000640 1 -- 172.21.15.63:0/596595848 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557c88fb5ae0 con 0x557c88c5ed80 2024-05-27T08:15:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.644+0000 7f2eb6000640 10 client.42239 handle_command_reply: tid=0 2024-05-27T08:15:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.645+0000 7f2eda579740 1 client.42239 shutdown 2024-05-27T08:15:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.645+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c772f0 msgr2=0x557c8904ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.645+0000 7f2eda579740 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c772f0 0x557c8904ca20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80ca200 tx=0x7f2eb80c99e0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.645+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 shutdown_connections 2024-05-27T08:15:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.646+0000 7f2eda579740 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557c88c5ed80 0x557c8905e940 crc :-1 s=READY pgs=740 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.646+0000 7f2eda579740 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88ca01b0 0x557c8904d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.646+0000 7f2eda579740 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 0x557c8904cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.646+0000 7f2eda579740 1 --2- 172.21.15.63:0/596595848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c772f0 0x557c8904ca20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.646+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 >> 172.21.15.63:0/596595848 conn(0x557c88c94300 msgr2=0x557c89042a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.646+0000 7f2eb6000640 0 client.42239 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.646+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 shutdown_connections 2024-05-27T08:15:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.647+0000 7f2eda579740 1 -- 172.21.15.63:0/596595848 wait complete. 2024-05-27T08:15:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.648+0000 7f2eda579740 20 client.42239 trim_cache size 0 max 16384 2024-05-27T08:15:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.707+0000 7f2eda579740 1 Processor -- start 2024-05-27T08:15:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.709+0000 7f2eda579740 1 -- start start 2024-05-27T08:15:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.712+0000 7f2eda579740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c8904eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.712+0000 7f2eda579740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 0x557c8904d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.713+0000 7f2eda579740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88ca01b0 0x557c89057fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.714+0000 7f2eda579740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac012820 con 0x557c88c772f0 2024-05-27T08:15:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.714+0000 7f2eda579740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac0100f0 con 0x557c88ca01b0 2024-05-27T08:15:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.714+0000 7f2eda579740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac00df60 con 0x557c88c7f4d0 2024-05-27T08:15:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.715+0000 7f2ed1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88ca01b0 0x557c89057fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.715+0000 7f2ed1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88ca01b0 0x557c89057fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44468/0 (socket says 172.21.15.63:44468) 2024-05-27T08:15:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.715+0000 7f2ed1400640 1 -- 172.21.15.63:0/2941589951 learned_addr learned my addr 172.21.15.63:0/2941589951 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.715+0000 7f2ecbe00640 1 --2- 172.21.15.63:0/2941589951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 0x557c8904d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.715+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/2941589951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c8904eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.717+0000 7f2ecbe00640 1 -- 172.21.15.63:0/2941589951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88ca01b0 msgr2=0x557c89057fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:46.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.717+0000 7f2ecbe00640 1 --2- 172.21.15.63:0/2941589951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88ca01b0 0x557c89057fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.717+0000 7f2ecbe00640 1 -- 172.21.15.63:0/2941589951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 msgr2=0x557c8904eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:46.719 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.717+0000 7f2ecbe00640 1 --2- 172.21.15.63:0/2941589951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c8904eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.718+0000 7f2ecbe00640 1 -- 172.21.15.63:0/2941589951 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8904dee0 con 0x557c88c7f4d0 2024-05-27T08:15:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.718+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/2941589951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c8904eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:15:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.719+0000 7f2ecbe00640 1 --2- 172.21.15.63:0/2941589951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 0x557c8904d2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x557c8905f0b0 tx=0x7f2ebc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc60def20656a20 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.720+0000 7f2ec1200640 1 -- 172.21.15.63:0/2941589951 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc01e020 con 0x557c88c7f4d0 2024-05-27T08:15:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.721+0000 7f2ec1200640 1 -- 172.21.15.63:0/2941589951 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ebc0047a0 con 0x557c88c7f4d0 2024-05-27T08:15:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.722+0000 7f2eda579740 1 -- 172.21.15.63:0/2941589951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 msgr2=0x557c8904d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.725+0000 7f2ec1200640 1 -- 172.21.15.63:0/2941589951 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc01e020 con 0x557c88c7f4d0 2024-05-27T08:15:46.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.725+0000 7f2eda579740 1 --2- 172.21.15.63:0/2941589951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 0x557c8904d2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x557c8905f0b0 tx=0x7f2ebc011810 comp rx=0 tx=0).stop 2024-05-27T08:15:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.727+0000 7f2eda579740 1 -- 172.21.15.63:0/2941589951 shutdown_connections 2024-05-27T08:15:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.727+0000 7f2eda579740 1 --2- 172.21.15.63:0/2941589951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88ca01b0 0x557c89057fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.727+0000 7f2eda579740 1 --2- 172.21.15.63:0/2941589951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88c7f4d0 0x557c8904d2c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.727+0000 7f2eda579740 1 --2- 172.21.15.63:0/2941589951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c8904eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.728+0000 7f2eda579740 1 -- 172.21.15.63:0/2941589951 >> 172.21.15.63:0/2941589951 conn(0x557c88c94300 msgr2=0x557c890578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.729+0000 7f2eda579740 1 -- 172.21.15.63:0/2941589951 shutdown_connections 2024-05-27T08:15:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.729+0000 7f2eda579740 1 -- 172.21.15.63:0/2941589951 wait complete. 2024-05-27T08:15:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.730+0000 7f2eda579740 1 -- start start 2024-05-27T08:15:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.730+0000 7f2eda579740 1 Processor -- start 2024-05-27T08:15:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.736+0000 7f2eda579740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89051ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.737+0000 7f2eda579740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 0x557c89052430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.737+0000 7f2eda579740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c890c5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.738+0000 7f2ecbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 0x557c89052430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.738+0000 7f2ecbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 0x557c89052430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44474/0 (socket says 172.21.15.63:44474) 2024-05-27T08:15:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.739+0000 7f2ed0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89051ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.739+0000 7f2ed0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89051ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44350/0 (socket says 172.21.15.63:44350) 2024-05-27T08:15:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.740+0000 7f2ed1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c890c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.740+0000 7f2ed1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c890c5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:49354/0 (socket says 172.21.15.63:49354) 2024-05-27T08:15:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.740+0000 7f2ecbe00640 1 -- 172.21.15.63:0/814687564 learned_addr learned my addr 172.21.15.63:0/814687564 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.740+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac00e5b0 con 0x557c88c772f0 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.740+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac00fa20 con 0x557c88c7f4d0 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.740+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac00ec80 con 0x557c88ca01b0 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.741+0000 7f2ecbe00640 1 -- 172.21.15.63:0/814687564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 msgr2=0x557c890c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.741+0000 7f2ecbe00640 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c890c5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.741+0000 7f2ecbe00640 1 -- 172.21.15.63:0/814687564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 msgr2=0x557c89051ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.742+0000 7f2ecbe00640 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89051ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.742+0000 7f2ecbe00640 1 -- 172.21.15.63:0/814687564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c890c5550 con 0x557c88c7f4d0 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.743+0000 7f2ed1400640 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c890c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.743+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89051ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.744+0000 7f2ecbe00640 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 0x557c89052430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc004770 tx=0x7f2ebc00abc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba140b6f0ba7e725 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.745+0000 7f2eda579740 10 client.0 _ll_register_callbacks cb 0x7f2ec4009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:15:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.745+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ebc0047a0 con 0x557c88c7f4d0 2024-05-27T08:15:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.749+0000 7f2eb6000640 10 client.42245 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:15:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.749+0000 7f2eb6000640 1 -- 172.21.15.63:0/814687564 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc03a030 con 0x557c88c7f4d0 2024-05-27T08:15:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.749+0000 7f2eb6000640 1 -- 172.21.15.63:0/814687564 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f2ebc0053f0 con 0x557c88c7f4d0 2024-05-27T08:15:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.749+0000 7f2eb6000640 1 -- 172.21.15.63:0/814687564 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2ebc047050 con 0x557c88c7f4d0 2024-05-27T08:15:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.751+0000 7f2eda579740 10 client.42245 fetch_fsmap learned FSMap version 35 2024-05-27T08:15:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.751+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=35}) -- 0x557c890c6320 con 0x557c88c7f4d0 2024-05-27T08:15:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.752+0000 7f2eb6000640 1 -- 172.21.15.63:0/814687564 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2ebc023040 con 0x557c88c7f4d0 2024-05-27T08:15:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.752+0000 7f2eda579740 10 client.42245 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:15:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.752+0000 7f2eda579740 10 client.42245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:15:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.752+0000 7f2eda579740 20 client.42245 populate_metadata read hostname 'smithi063' 2024-05-27T08:15:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.753+0000 7f2eda579740 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557c88c341c0 0x557c890c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.753+0000 7f2eda579740 4 client.42245 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:15:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.753+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c88faf2c0 con 0x557c88c341c0 2024-05-27T08:15:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.754+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557c88c341c0 0x557c890c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.757+0000 7f2ed0a00640 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557c88c341c0 0x557c890c8ac0 crc :-1 s=READY pgs=741 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2623c8d7564ffa2 server_cookie=f26befcd4eed44b4 in_seq=0 out_seq=0 2024-05-27T08:15:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:46.757+0000 7f2eb6000640 10 client.42245 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.333+0000 7f2eb6000640 1 -- 172.21.15.63:0/814687564 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557c88faf2c0 con 0x557c88c341c0 2024-05-27T08:15:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.334+0000 7f2eb6000640 10 client.42245 handle_command_reply: tid=0 2024-05-27T08:15:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.334+0000 7f2eda579740 1 client.42245 shutdown 2024-05-27T08:15:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.339+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 msgr2=0x557c89052430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.339+0000 7f2eda579740 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 0x557c89052430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc004770 tx=0x7f2ebc00abc0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.342+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 shutdown_connections 2024-05-27T08:15:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.342+0000 7f2eda579740 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557c88c341c0 0x557c890c8ac0 crc :-1 s=READY pgs=741 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.342+0000 7f2eb6000640 0 client.42245 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.342+0000 7f2eda579740 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557c88ca01b0 0x557c890c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.342+0000 7f2eda579740 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557c88c7f4d0 0x557c89052430 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.342+0000 7f2eda579740 1 --2- 172.21.15.63:0/814687564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557c88c772f0 0x557c89051ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.342+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 >> 172.21.15.63:0/814687564 conn(0x557c88c94300 msgr2=0x557c8904e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.344+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 shutdown_connections 2024-05-27T08:15:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.344+0000 7f2eda579740 1 -- 172.21.15.63:0/814687564 wait complete. 2024-05-27T08:15:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.345+0000 7f2eda579740 20 client.42245 trim_cache size 0 max 16384 2024-05-27T08:15:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:15:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:15:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:15:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.347+0000 7f2ed1e00640 1 -- 172.21.15.63:0/652574815 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2eb00d4af0 msgr2=0x7f2eb00d7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.348+0000 7f2ed1e00640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2eb00d4af0 0x7f2eb00d7000 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x557c8903fbd0 tx=0x7f2ec40095c0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.348+0000 7f2ed1e00640 1 -- 172.21.15.63:0/652574815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ecc1029a0 msgr2=0x7f2ecc116c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.348+0000 7f2ed1e00640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ecc1029a0 0x7f2ecc116c40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009460 tx=0x7f2eb8032cb0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.351+0000 7f2ed1e00640 1 -- 172.21.15.63:0/652574815 shutdown_connections 2024-05-27T08:15:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.351+0000 7f2ed1e00640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2eb00d4af0 0x7f2eb00d7000 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.351+0000 7f2ed1e00640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ecc107e20 0x7f2ecc1113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.351+0000 7f2ed1e00640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ecc1073f0 0x7f2ecc110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.351+0000 7f2ed1e00640 1 --2- 172.21.15.63:0/652574815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ecc1029a0 0x7f2ecc116c40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.351+0000 7f2ed1e00640 1 -- 172.21.15.63:0/652574815 >> 172.21.15.63:0/652574815 conn(0x7f2ecc0fe210 msgr2=0x7f2ecc105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.351+0000 7f2ed1e00640 1 -- 172.21.15.63:0/652574815 shutdown_connections 2024-05-27T08:15:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:15:47.352+0000 7f2ed1e00640 1 -- 172.21.15.63:0/652574815 wait complete. 2024-05-27T08:15:47.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.695+0000 7f9c89e00700 1 -- 172.21.15.63:0/816310953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c84109670 msgr2=0x7f9c84110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.695+0000 7f9c89e00700 1 --2- 172.21.15.63:0/816310953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c84109670 0x7f9c84110fd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9c740099c0 tx=0x7f9c74009cd0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.695+0000 7f9c89e00700 1 -- 172.21.15.63:0/816310953 shutdown_connections 2024-05-27T08:15:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.695+0000 7f9c89e00700 1 --2- 172.21.15.63:0/816310953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c84109670 0x7f9c84110fd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.695+0000 7f9c89e00700 1 --2- 172.21.15.63:0/816310953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c841043f0 0x7f9c84104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.695+0000 7f9c89e00700 1 --2- 172.21.15.63:0/816310953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c84108d20 0x7f9c84109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.695+0000 7f9c89e00700 1 -- 172.21.15.63:0/816310953 >> 172.21.15.63:0/816310953 conn(0x7f9c84079d00 msgr2=0x7f9c8407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:47.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.696+0000 7f9c89e00700 1 -- 172.21.15.63:0/816310953 shutdown_connections 2024-05-27T08:15:47.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.697+0000 7f9c89e00700 1 -- 172.21.15.63:0/816310953 wait complete. 2024-05-27T08:15:47.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.699+0000 7f9c89e00700 1 Processor -- start 2024-05-27T08:15:47.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.699+0000 7f9c89e00700 1 -- start start 2024-05-27T08:15:47.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.701+0000 7f9c89e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c841043f0 0x7f9c840753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.701+0000 7f9c89e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c84108d20 0x7f9c8406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.701+0000 7f9c88a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c841043f0 0x7f9c840753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.701+0000 7f9c88a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c841043f0 0x7f9c840753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:49374/0 (socket says 172.21.15.63:49374) 2024-05-27T08:15:47.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.702+0000 7f9c83e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c84108d20 0x7f9c8406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.702+0000 7f9c89e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84109670 0x7f9c8406eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.702+0000 7f9c83e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c84108d20 0x7f9c8406e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44376/0 (socket says 172.21.15.63:44376) 2024-05-27T08:15:47.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.702+0000 7f9c89e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841141a0 con 0x7f9c84108d20 2024-05-27T08:15:47.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.702+0000 7f9c83e00700 1 -- 172.21.15.63:0/3952151151 learned_addr learned my addr 172.21.15.63:0/3952151151 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:47.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.703+0000 7f9c89e00700 1 -- 172.21.15.63:0/3952151151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84114020 con 0x7f9c84109670 2024-05-27T08:15:47.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.703+0000 7f9c89400700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84109670 0x7f9c8406eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.703+0000 7f9c89e00700 1 -- 172.21.15.63:0/3952151151 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84114320 con 0x7f9c841043f0 2024-05-27T08:15:47.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.704+0000 7f9c83e00700 1 -- 172.21.15.63:0/3952151151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c841043f0 msgr2=0x7f9c840753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.704+0000 7f9c83e00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c841043f0 0x7f9c840753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.704+0000 7f9c83e00700 1 -- 172.21.15.63:0/3952151151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84109670 msgr2=0x7f9c8406eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.704+0000 7f9c83e00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84109670 0x7f9c8406eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.704+0000 7f9c83e00700 1 -- 172.21.15.63:0/3952151151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c84072ef0 con 0x7f9c84108d20 2024-05-27T08:15:47.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.704+0000 7f9c83e00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c84108d20 0x7f9c8406e4f0 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000e970 tx=0x7f9c7000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:47.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.705+0000 7f9c81600700 1 -- 172.21.15.63:0/3952151151 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000cbd0 con 0x7f9c84108d20 2024-05-27T08:15:47.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.705+0000 7f9c81600700 1 -- 172.21.15.63:0/3952151151 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c7000cd70 con 0x7f9c84108d20 2024-05-27T08:15:47.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.705+0000 7f9c81600700 1 -- 172.21.15.63:0/3952151151 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70005570 con 0x7f9c84108d20 2024-05-27T08:15:47.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.705+0000 7f9c89e00700 1 -- 172.21.15.63:0/3952151151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c840731e0 con 0x7f9c84108d20 2024-05-27T08:15:47.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.706+0000 7f9c89e00700 1 -- 172.21.15.63:0/3952151151 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c840736d0 con 0x7f9c84108d20 2024-05-27T08:15:47.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.713+0000 7f9c81600700 1 -- 172.21.15.63:0/3952151151 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9c70021440 con 0x7f9c84108d20 2024-05-27T08:15:47.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.716+0000 7f9c81600700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c680b2b20 0x7f9c680b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.716+0000 7f9c81600700 1 -- 172.21.15.63:0/3952151151 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f9c700e3a40 con 0x7f9c84108d20 2024-05-27T08:15:47.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.717+0000 7f9c88a00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c680b2b20 0x7f9c680b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.757+0000 7f9c88a00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c680b2b20 0x7f9c680b5030 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009550 tx=0x7f9c7c0091b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:47.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.761+0000 7f9c99ff3b80 1 Processor -- start 2024-05-27T08:15:47.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.762+0000 7f9c99ff3b80 1 -- start start 2024-05-27T08:15:47.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.763+0000 7f9c99ff3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf3bae40 0x55fddf538e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.763+0000 7f9c99ff3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 0x55fddf543b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.763+0000 7f9c99ff3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5397b0 0x55fddf546060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.763+0000 7f9c99ff3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fddf467e40 con 0x55fddf5397b0 2024-05-27T08:15:47.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.763+0000 7f9c99ff3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fddf4579e0 con 0x55fddf3bae40 2024-05-27T08:15:47.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.763+0000 7f9c99ff3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fddf45ba20 con 0x55fddf5393a0 2024-05-27T08:15:47.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.763+0000 7f9c88a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf3bae40 0x55fddf538e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.763+0000 7f9c88a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf3bae40 0x55fddf538e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44504/0 (socket says 172.21.15.63:44504) 2024-05-27T08:15:47.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.763+0000 7f9c88a00700 1 -- 172.21.15.63:0/1069415131 learned_addr learned my addr 172.21.15.63:0/1069415131 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:47.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.764+0000 7f9c88a00700 1 -- 172.21.15.63:0/1069415131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 msgr2=0x55fddf543b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:15:47.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.764+0000 7f9c88a00700 1 --2- 172.21.15.63:0/1069415131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 0x55fddf543b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.764+0000 7f9c88a00700 1 -- 172.21.15.63:0/1069415131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5397b0 msgr2=0x55fddf546060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:47.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.764+0000 7f9c88a00700 1 --2- 172.21.15.63:0/1069415131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5397b0 0x55fddf546060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.764+0000 7f9c88a00700 1 -- 172.21.15.63:0/1069415131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fddf420420 con 0x55fddf3bae40 2024-05-27T08:15:47.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.765+0000 7f9c88a00700 1 --2- 172.21.15.63:0/1069415131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf3bae40 0x55fddf538e60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c004680 tx=0x7f9c7c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a3027c2564472a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:47.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.766+0000 7f9c7a000700 1 -- 172.21.15.63:0/1069415131 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c01c050 con 0x55fddf3bae40 2024-05-27T08:15:47.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.766+0000 7f9c7a000700 1 -- 172.21.15.63:0/1069415131 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c7c025070 con 0x55fddf3bae40 2024-05-27T08:15:47.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.766+0000 7f9c7a000700 1 -- 172.21.15.63:0/1069415131 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c02a6d0 con 0x55fddf3bae40 2024-05-27T08:15:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.768+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/1069415131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf3bae40 msgr2=0x55fddf538e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.768+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/1069415131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf3bae40 0x55fddf538e60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c004680 tx=0x7f9c7c013000 comp rx=0 tx=0).stop 2024-05-27T08:15:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.769+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/1069415131 shutdown_connections 2024-05-27T08:15:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.769+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/1069415131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5397b0 0x55fddf546060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.769+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/1069415131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf3bae40 0x55fddf538e60 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c004680 tx=0x7f9c7c013000 comp rx=0 tx=0).stop 2024-05-27T08:15:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.769+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/1069415131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 0x55fddf543b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.769+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/1069415131 >> 172.21.15.63:0/1069415131 conn(0x55fddf41afc0 msgr2=0x55fddf534d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:47.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.770+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/1069415131 shutdown_connections 2024-05-27T08:15:47.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.770+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/1069415131 wait complete. 2024-05-27T08:15:47.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.771+0000 7f9c99ff3b80 1 -- start start 2024-05-27T08:15:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.775+0000 7f9c99ff3b80 1 Processor -- start 2024-05-27T08:15:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.777+0000 7f9c99ff3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 0x55fddf543740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.777+0000 7f9c88a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 0x55fddf543740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.777+0000 7f9c88a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 0x55fddf543740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:49390/0 (socket says 172.21.15.63:49390) 2024-05-27T08:15:47.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.777+0000 7f9c88a00700 1 -- 172.21.15.63:0/88098709 learned_addr learned my addr 172.21.15.63:0/88098709 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:47.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5397b0 0x55fddf543c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 0x55fddf54fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fddf35dfa0 con 0x55fddf5441c0 2024-05-27T08:15:47.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fddf3f8760 con 0x55fddf5397b0 2024-05-27T08:15:47.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fddf5469d0 con 0x55fddf5393a0 2024-05-27T08:15:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c99ff3b80 10 client.0 _ll_register_callbacks cb 0x55fddf401c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:15:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c88a00700 1 -- 172.21.15.63:0/88098709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5397b0 msgr2=0x55fddf543c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:15:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c88a00700 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5397b0 0x55fddf543c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c88a00700 1 -- 172.21.15.63:0/88098709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 msgr2=0x55fddf54fd50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:15:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c88a00700 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 0x55fddf54fd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.778+0000 7f9c88a00700 1 -- 172.21.15.63:0/88098709 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fddf3a78d0 con 0x55fddf5393a0 2024-05-27T08:15:47.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.780+0000 7f9c88a00700 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 0x55fddf543740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55fddf5356d0 tx=0x7f9c7c0218e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4349e96645cdf51a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:47.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.780+0000 7f9c6ea00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:15:47.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.781+0000 7f9c6ea00700 1 -- 172.21.15.63:0/88098709 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c021b80 con 0x55fddf5393a0 2024-05-27T08:15:47.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.781+0000 7f9c6ea00700 1 -- 172.21.15.63:0/88098709 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f9c7c02f070 con 0x55fddf5393a0 2024-05-27T08:15:47.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.782+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c7c025070 con 0x55fddf5393a0 2024-05-27T08:15:47.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.784+0000 7f9c6ea00700 1 -- 172.21.15.63:0/88098709 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9c7c01c030 con 0x55fddf5393a0 2024-05-27T08:15:47.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.787+0000 7f9c99ff3b80 10 client.32068 fetch_fsmap learned FSMap version 35 2024-05-27T08:15:47.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.787+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fddf428060 con 0x55fddf5393a0 2024-05-27T08:15:47.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.790+0000 7f9c6ea00700 1 -- 172.21.15.63:0/88098709 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c7c025070 con 0x55fddf5393a0 2024-05-27T08:15:47.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.790+0000 7f9c6ea00700 1 -- 172.21.15.63:0/88098709 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9c7c02a470 con 0x55fddf5393a0 2024-05-27T08:15:47.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.790+0000 7f9c99ff3b80 10 client.32068 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:15:47.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.790+0000 7f9c99ff3b80 10 client.32068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:15:47.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.790+0000 7f9c99ff3b80 20 client.32068 populate_metadata read hostname 'smithi063' 2024-05-27T08:15:47.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.790+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fddf553ea0 0x55fddf5562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.790+0000 7f9c99ff3b80 4 client.32068 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:15:47.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.790+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fddf469d90 con 0x55fddf553ea0 2024-05-27T08:15:47.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.791+0000 7f9c83e00700 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fddf553ea0 0x55fddf5562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.793+0000 7f9c6ea00700 10 client.32068 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:47.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.793+0000 7f9c83e00700 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fddf553ea0 0x55fddf5562c0 crc :-1 s=READY pgs=742 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b009d885ecb22c7 server_cookie=b7ca1e680dfecbc6 in_seq=0 out_seq=0 2024-05-27T08:15:47.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.795+0000 7f9c6ea00700 1 -- 172.21.15.63:0/88098709 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fddf469d90 con 0x55fddf553ea0 2024-05-27T08:15:47.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.795+0000 7f9c6ea00700 10 client.32068 handle_command_reply: tid=0 2024-05-27T08:15:47.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.795+0000 7f9c99ff3b80 1 client.32068 shutdown 2024-05-27T08:15:47.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.796+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 msgr2=0x55fddf543740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.796+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 0x55fddf543740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55fddf5356d0 tx=0x7f9c7c0218e0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.797+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 shutdown_connections 2024-05-27T08:15:47.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.797+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fddf553ea0 0x55fddf5562c0 crc :-1 s=READY pgs=742 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.797+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 0x55fddf54fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.797+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5393a0 0x55fddf543740 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.797+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/88098709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5397b0 0x55fddf543c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.797+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 >> 172.21.15.63:0/88098709 conn(0x55fddf41afc0 msgr2=0x55fddf533ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:47.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.798+0000 7f9c6ea00700 0 client.32068 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:47.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.799+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 shutdown_connections 2024-05-27T08:15:47.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.799+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/88098709 wait complete. 2024-05-27T08:15:47.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.800+0000 7f9c99ff3b80 20 client.32068 trim_cache size 0 max 16384 2024-05-27T08:15:47.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.845+0000 7f9c99ff3b80 1 Processor -- start 2024-05-27T08:15:47.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.846+0000 7f9c99ff3b80 1 -- start start 2024-05-27T08:15:47.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.846+0000 7f9c99ff3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf542a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.846+0000 7f9c99ff3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5397b0 0x55fddf53ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.847+0000 7f9c99ff3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5441c0 0x55fddf55b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.847+0000 7f9c99ff3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841b93d0 con 0x55fddf5397b0 2024-05-27T08:15:47.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.847+0000 7f9c99ff3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841b6c60 con 0x55fddf5393a0 2024-05-27T08:15:47.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.847+0000 7f9c99ff3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841b4ad0 con 0x55fddf5441c0 2024-05-27T08:15:47.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.849+0000 7f9c88a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf542a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.849+0000 7f9c88a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf542a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:44510/0 (socket says 172.21.15.63:44510) 2024-05-27T08:15:47.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.849+0000 7f9c88a00700 1 -- 172.21.15.63:0/1858173673 learned_addr learned my addr 172.21.15.63:0/1858173673 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:47.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.849+0000 7f9c89400700 1 --2- 172.21.15.63:0/1858173673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5441c0 0x55fddf55b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.850+0000 7f9c83e00700 1 --2- 172.21.15.63:0/1858173673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5397b0 0x55fddf53ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.851+0000 7f9c89400700 1 -- 172.21.15.63:0/1858173673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 msgr2=0x55fddf542a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:47.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.852+0000 7f9c89400700 1 --2- 172.21.15.63:0/1858173673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf542a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.852+0000 7f9c89400700 1 -- 172.21.15.63:0/1858173673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5397b0 msgr2=0x55fddf53ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:47.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.852+0000 7f9c89400700 1 --2- 172.21.15.63:0/1858173673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5397b0 0x55fddf53ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.852+0000 7f9c89400700 1 -- 172.21.15.63:0/1858173673 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fddf52e130 con 0x55fddf5441c0 2024-05-27T08:15:47.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.852+0000 7f9c89400700 1 --2- 172.21.15.63:0/1858173673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5441c0 0x55fddf55b890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009990 tx=0x7f9c74005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a7fef79e0876692 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:47.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.853+0000 7f9c88a00700 1 --2- 172.21.15.63:0/1858173673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf542a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:15:47.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.853+0000 7f9c79600700 1 -- 172.21.15.63:0/1858173673 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c740052b0 con 0x55fddf5441c0 2024-05-27T08:15:47.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.853+0000 7f9c79600700 1 -- 172.21.15.63:0/1858173673 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c74003c50 con 0x55fddf5441c0 2024-05-27T08:15:47.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.853+0000 7f9c79600700 1 -- 172.21.15.63:0/1858173673 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7400ba80 con 0x55fddf5441c0 2024-05-27T08:15:47.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.855+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/1858173673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5441c0 msgr2=0x55fddf55b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.855+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/1858173673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5441c0 0x55fddf55b890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009990 tx=0x7f9c74005040 comp rx=0 tx=0).stop 2024-05-27T08:15:47.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.856+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/1858173673 shutdown_connections 2024-05-27T08:15:47.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.856+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/1858173673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5441c0 0x55fddf55b890 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.856+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/1858173673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf542a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.856+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/1858173673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5397b0 0x55fddf53ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.856+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/1858173673 >> 172.21.15.63:0/1858173673 conn(0x55fddf41afc0 msgr2=0x55fddf544bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:47.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.856+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/1858173673 shutdown_connections 2024-05-27T08:15:47.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.857+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/1858173673 wait complete. 2024-05-27T08:15:47.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.857+0000 7f9c99ff3b80 1 -- start start 2024-05-27T08:15:47.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.860+0000 7f9c99ff3b80 1 Processor -- start 2024-05-27T08:15:47.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.861+0000 7f9c99ff3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf54c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.862+0000 7f9c99ff3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5397b0 0x55fddf54c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.862+0000 7f9c99ff3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 0x55fddf5474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.863+0000 7f9c89400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 0x55fddf5474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.863+0000 7f9c89400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 0x55fddf5474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44410/0 (socket says 172.21.15.63:44410) 2024-05-27T08:15:47.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.863+0000 7f9c89400700 1 -- 172.21.15.63:0/2846360934 learned_addr learned my addr 172.21.15.63:0/2846360934 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:15:47.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.864+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841b5120 con 0x55fddf5441c0 2024-05-27T08:15:47.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.864+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841b6590 con 0x55fddf5393a0 2024-05-27T08:15:47.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.864+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841b57f0 con 0x55fddf5397b0 2024-05-27T08:15:47.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.864+0000 7f9c99ff3b80 10 client.0 _ll_register_callbacks cb 0x55fddf401c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:15:47.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.864+0000 7f9c88a00700 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf54c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.864+0000 7f9c83e00700 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5397b0 0x55fddf54c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.865+0000 7f9c88a00700 1 -- 172.21.15.63:0/2846360934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5397b0 msgr2=0x55fddf54c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:47.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.865+0000 7f9c88a00700 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5397b0 0x55fddf54c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.865+0000 7f9c88a00700 1 -- 172.21.15.63:0/2846360934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 msgr2=0x55fddf5474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:15:47.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.865+0000 7f9c88a00700 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 0x55fddf5474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.865+0000 7f9c88a00700 1 -- 172.21.15.63:0/2846360934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fddf547c90 con 0x55fddf5393a0 2024-05-27T08:15:47.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.865+0000 7f9c88a00700 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf54c000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55fddf464110 tx=0x7f9c7c02ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8265c438c4d5f321 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:15:47.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.865+0000 7f9c6ea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:15:47.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.867+0000 7f9c89400700 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 0x55fddf5474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:15:47.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.868+0000 7f9c6ea00700 1 -- 172.21.15.63:0/2846360934 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c044060 con 0x55fddf5393a0 2024-05-27T08:15:47.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.868+0000 7f9c6ea00700 1 -- 172.21.15.63:0/2846360934 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f9c7c02f050 con 0x55fddf5393a0 2024-05-27T08:15:47.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.868+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c74003c50 con 0x55fddf5393a0 2024-05-27T08:15:47.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.868+0000 7f9c6ea00700 1 -- 172.21.15.63:0/2846360934 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9c7c044060 con 0x55fddf5393a0 2024-05-27T08:15:47.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.869+0000 7f9c99ff3b80 10 client.42260 fetch_fsmap learned FSMap version 35 2024-05-27T08:15:47.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.869+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fddf5c6ae0 con 0x55fddf5393a0 2024-05-27T08:15:47.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.869+0000 7f9c6ea00700 1 -- 172.21.15.63:0/2846360934 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c74003c50 con 0x55fddf5393a0 2024-05-27T08:15:47.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.870+0000 7f9c6ea00700 1 -- 172.21.15.63:0/2846360934 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9c7c035c30 con 0x55fddf5393a0 2024-05-27T08:15:47.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.870+0000 7f9c99ff3b80 10 client.42260 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:15:47.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.870+0000 7f9c99ff3b80 10 client.42260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:15:47.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.870+0000 7f9c99ff3b80 20 client.42260 populate_metadata read hostname 'smithi063' 2024-05-27T08:15:47.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.870+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fddf5c70b0 0x55fddf5c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:15:47.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.870+0000 7f9c99ff3b80 4 client.42260 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:15:47.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.870+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fddf463570 con 0x55fddf5c70b0 2024-05-27T08:15:47.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.870+0000 7f9c83e00700 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fddf5c70b0 0x55fddf5c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:15:47.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.871+0000 7f9c83e00700 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fddf5c70b0 0x55fddf5c94d0 crc :-1 s=READY pgs=743 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d533a1740c8e72b6 server_cookie=60d7ab81db1c8383 in_seq=0 out_seq=0 2024-05-27T08:15:47.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.871+0000 7f9c6ea00700 10 client.42260 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:47.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.873+0000 7f9c6ea00700 1 -- 172.21.15.63:0/2846360934 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fddf463570 con 0x55fddf5c70b0 2024-05-27T08:15:47.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.873+0000 7f9c6ea00700 10 client.42260 handle_command_reply: tid=0 2024-05-27T08:15:47.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.873+0000 7f9c99ff3b80 1 client.42260 shutdown 2024-05-27T08:15:47.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.874+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 msgr2=0x55fddf54c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.874+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf54c000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55fddf464110 tx=0x7f9c7c02ad00 comp rx=0 tx=0).stop 2024-05-27T08:15:47.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.874+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 shutdown_connections 2024-05-27T08:15:47.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.874+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fddf5c70b0 0x55fddf5c94d0 crc :-1 s=READY pgs=743 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.874+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fddf5441c0 0x55fddf5474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.874+0000 7f9c6ea00700 0 client.42260 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:15:47.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.874+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fddf5393a0 0x55fddf54c000 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.874+0000 7f9c99ff3b80 1 --2- 172.21.15.63:0/2846360934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fddf5397b0 0x55fddf54c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.874+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 >> 172.21.15.63:0/2846360934 conn(0x55fddf41afc0 msgr2=0x55fddf5427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:47.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.875+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 shutdown_connections 2024-05-27T08:15:47.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.876+0000 7f9c99ff3b80 1 -- 172.21.15.63:0/2846360934 wait complete. 2024-05-27T08:15:47.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.876+0000 7f9c99ff3b80 20 client.42260 trim_cache size 0 max 16384 2024-05-27T08:15:47.879 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:15:47.880 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:15:47.880 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:15:47.880 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:15:47.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.878+0000 7f9c89e00700 1 -- 172.21.15.63:0/3952151151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c680b2b20 msgr2=0x7f9c680b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.878+0000 7f9c89e00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c680b2b20 0x7f9c680b5030 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009550 tx=0x7f9c7c0091b0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.878+0000 7f9c89e00700 1 -- 172.21.15.63:0/3952151151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c84108d20 msgr2=0x7f9c8406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:15:47.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.878+0000 7f9c89e00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c84108d20 0x7f9c8406e4f0 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000e970 tx=0x7f9c7000ec80 comp rx=0 tx=0).stop 2024-05-27T08:15:47.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.878+0000 7f9c89e00700 1 -- 172.21.15.63:0/3952151151 shutdown_connections 2024-05-27T08:15:47.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.878+0000 7f9c89e00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f9c680b2b20 0x7f9c680b5030 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.878+0000 7f9c89e00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84109670 0x7f9c8406eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.878+0000 7f9c89e00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c841043f0 0x7f9c840753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.878+0000 7f9c89e00700 1 --2- 172.21.15.63:0/3952151151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c84108d20 0x7f9c8406e4f0 unknown :-1 s=CLOSED pgs=2940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:15:47.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.878+0000 7f9c89e00700 1 -- 172.21.15.63:0/3952151151 >> 172.21.15.63:0/3952151151 conn(0x7f9c84079d00 msgr2=0x7f9c84101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:15:47.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.879+0000 7f9c89e00700 1 -- 172.21.15.63:0/3952151151 shutdown_connections 2024-05-27T08:15:47.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:15:47.879+0000 7f9c89e00700 1 -- 172.21.15.63:0/3952151151 wait complete. 2024-05-27T08:15:47.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:47 smithi063 ceph-mon[31613]: pgmap v1340: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-05-27T08:15:47.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:47 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:15:47.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:47 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:15:48.133 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:47 smithi149 ceph-mon[35408]: pgmap v1340: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-05-27T08:15:48.133 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:48 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:15:48.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:47 smithi086 ceph-mon[34687]: pgmap v1340: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-05-27T08:15:48.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:47 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:15:48.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:47 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:15:48.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:48 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:15:48.997 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:15:48.998 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:15:49.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:48 smithi086 ceph-mon[34687]: pgmap v1341: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:15:49.213 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:48 smithi063 ceph-mon[31613]: pgmap v1341: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:15:49.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:48 smithi149 ceph-mon[35408]: pgmap v1341: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:15:50.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:49 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:49 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:49 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:49 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:49 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T08:15:50.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:49 smithi086 ceph-mon[34687]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T08:15:50.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:49 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:49 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:49 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:49 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:49 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T08:15:50.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:49 smithi149 ceph-mon[35408]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T08:15:50.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:49 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:49 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:49 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:49 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:15:50.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:49 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-05-27T08:15:50.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:49 smithi063 ceph-mon[31613]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-05-27T08:15:50.401 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:15:51.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:50 smithi086 ceph-mon[34687]: pgmap v1342: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-27T08:15:51.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:50 smithi149 ceph-mon[35408]: pgmap v1342: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-27T08:15:51.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:50 smithi063 ceph-mon[31613]: pgmap v1342: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-27T08:15:54.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:53 smithi086 ceph-mon[34687]: pgmap v1343: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-05-27T08:15:54.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:53 smithi149 ceph-mon[35408]: pgmap v1343: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-05-27T08:15:54.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:53 smithi063 ceph-mon[31613]: pgmap v1343: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 3.89k op/s 2024-05-27T08:15:55.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:54 smithi086 ceph-mon[34687]: pgmap v1344: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:15:55.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:54 smithi149 ceph-mon[35408]: pgmap v1344: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:15:55.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:54 smithi063 ceph-mon[31613]: pgmap v1344: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:15:57.884 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:57 smithi149 ceph-mon[35408]: pgmap v1345: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-05-27T08:15:58.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:57 smithi086 ceph-mon[34687]: pgmap v1345: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-05-27T08:15:58.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:57 smithi063 ceph-mon[31613]: pgmap v1345: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-05-27T08:15:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:58 smithi086 ceph-mon[34687]: pgmap v1346: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-05-27T08:15:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:58 smithi149 ceph-mon[35408]: pgmap v1346: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-05-27T08:15:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:58 smithi063 ceph-mon[31613]: pgmap v1346: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-05-27T08:16:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:59 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:00.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:15:59 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:59 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:00.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:15:59 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:00.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:59 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:00.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:15:59 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.921+0000 7f1b3da00640 1 -- 172.21.15.63:0/934183733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3009cec0 msgr2=0x7f1b30096b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.921+0000 7f1b3da00640 1 --2- 172.21.15.63:0/934183733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3009cec0 0x7f1b30096b10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009860 tx=0x7f1b2802a340 comp rx=0 tx=0).stop 2024-05-27T08:16:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.923+0000 7f1b3da00640 1 -- 172.21.15.63:0/934183733 shutdown_connections 2024-05-27T08:16:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.923+0000 7f1b3da00640 1 --2- 172.21.15.63:0/934183733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b30097180 0x7f1b300a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.923+0000 7f1b3da00640 1 --2- 172.21.15.63:0/934183733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3009cec0 0x7f1b30096b10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.923+0000 7f1b3da00640 1 --2- 172.21.15.63:0/934183733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b3009ca20 0x7f1b300965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.923+0000 7f1b3da00640 1 -- 172.21.15.63:0/934183733 >> 172.21.15.63:0/934183733 conn(0x7f1b3000c040 msgr2=0x7f1b3000c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.924+0000 7f1b3da00640 1 -- 172.21.15.63:0/934183733 shutdown_connections 2024-05-27T08:16:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.924+0000 7f1b3da00640 1 -- 172.21.15.63:0/934183733 wait complete. 2024-05-27T08:16:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.926+0000 7f1b3da00640 1 Processor -- start 2024-05-27T08:16:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.928+0000 7f1b3da00640 1 -- start start 2024-05-27T08:16:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.929+0000 7f1b3da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b30097180 0x7f1b300a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.929+0000 7f1b3da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3009ca20 0x7f1b300a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.929+0000 7f1b3da00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b3009cec0 0x7f1b30150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.930+0000 7f1b36000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3009ca20 0x7f1b300a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.930+0000 7f1b36000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3009ca20 0x7f1b300a6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:50160/0 (socket says 172.21.15.63:50160) 2024-05-27T08:16:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.930+0000 7f1b36a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b30097180 0x7f1b300a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.930+0000 7f1b36a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b30097180 0x7f1b300a6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:37474/0 (socket says 172.21.15.63:37474) 2024-05-27T08:16:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.931+0000 7f1b3da00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b30008a50 con 0x7f1b3009cec0 2024-05-27T08:16:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.931+0000 7f1b3da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b300088d0 con 0x7f1b3009ca20 2024-05-27T08:16:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.931+0000 7f1b3da00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b30008bd0 con 0x7f1b30097180 2024-05-27T08:16:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.931+0000 7f1b36000640 1 -- 172.21.15.63:0/3395646172 learned_addr learned my addr 172.21.15.63:0/3395646172 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.931+0000 7f1b37400640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b3009cec0 0x7f1b30150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.931+0000 7f1b36a00640 1 -- 172.21.15.63:0/3395646172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3009ca20 msgr2=0x7f1b300a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.931+0000 7f1b36a00640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3009ca20 0x7f1b300a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.932+0000 7f1b36a00640 1 -- 172.21.15.63:0/3395646172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b3009cec0 msgr2=0x7f1b30150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.932+0000 7f1b36a00640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b3009cec0 0x7f1b30150700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.932+0000 7f1b36a00640 1 -- 172.21.15.63:0/3395646172 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b30150c40 con 0x7f1b30097180 2024-05-27T08:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.932+0000 7f1b36a00640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b30097180 0x7f1b300a6560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1b30097710 tx=0x7f1b2400db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.933+0000 7f1b2f400640 1 -- 172.21.15.63:0/3395646172 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24017070 con 0x7f1b30097180 2024-05-27T08:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.933+0000 7f1b3da00640 1 -- 172.21.15.63:0/3395646172 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b30150ed0 con 0x7f1b30097180 2024-05-27T08:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.933+0000 7f1b3da00640 1 -- 172.21.15.63:0/3395646172 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b30151330 con 0x7f1b30097180 2024-05-27T08:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.933+0000 7f1b2f400640 1 -- 172.21.15.63:0/3395646172 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b24010460 con 0x7f1b30097180 2024-05-27T08:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.933+0000 7f1b2f400640 1 -- 172.21.15.63:0/3395646172 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24017070 con 0x7f1b30097180 2024-05-27T08:16:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.940+0000 7f1b2f400640 1 -- 172.21.15.63:0/3395646172 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1b2401a070 con 0x7f1b30097180 2024-05-27T08:16:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.948+0000 7f1b2f400640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b1c0b2db0 0x7f1b1c0b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.948+0000 7f1b2f400640 1 -- 172.21.15.63:0/3395646172 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f1b24009820 con 0x7f1b30097180 2024-05-27T08:16:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.951+0000 7f1b36000640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b1c0b2db0 0x7f1b1c0b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.954+0000 7f1b36000640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b1c0b2db0 0x7f1b1c0b52c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1b300a3200 tx=0x7f1b28005c90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.959+0000 7f1b449b4740 1 Processor -- start 2024-05-27T08:16:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.959+0000 7f1b449b4740 1 -- start start 2024-05-27T08:16:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.959+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 0x55a53cc4ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.959+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc4cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.960+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 0x55a53cc4d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.960+0000 7f1b449b4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a53cbbdb90 con 0x55a53c8812f0 2024-05-27T08:16:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.960+0000 7f1b449b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a53cbad730 con 0x55a53c8aa1b0 2024-05-27T08:16:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.960+0000 7f1b449b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a53cbb1770 con 0x55a53c8894d0 2024-05-27T08:16:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.960+0000 7f1b37400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 0x55a53cc4d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.960+0000 7f1b37400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 0x55a53cc4d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:37478/0 (socket says 172.21.15.63:37478) 2024-05-27T08:16:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.960+0000 7f1b37400640 1 -- 172.21.15.63:0/3341389561 learned_addr learned my addr 172.21.15.63:0/3341389561 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.961+0000 7f1b37400640 1 -- 172.21.15.63:0/3341389561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 msgr2=0x55a53cc4ca90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:16:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.961+0000 7f1b37400640 1 --2- 172.21.15.63:0/3341389561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 0x55a53cc4ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.961+0000 7f1b37400640 1 -- 172.21.15.63:0/3341389561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 msgr2=0x55a53cc4cfd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:16:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.961+0000 7f1b37400640 1 --2- 172.21.15.63:0/3341389561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc4cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.961+0000 7f1b37400640 1 -- 172.21.15.63:0/3341389561 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53cb468b0 con 0x55a53c8894d0 2024-05-27T08:16:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.961+0000 7f1b37400640 1 --2- 172.21.15.63:0/3341389561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 0x55a53cc4d510 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b380687b0 tx=0x7f1b38069bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd9f0a9dd27f4fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.966+0000 7f1b23e00640 1 -- 172.21.15.63:0/3341389561 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38069d00 con 0x55a53c8894d0 2024-05-27T08:16:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.967+0000 7f1b23e00640 1 -- 172.21.15.63:0/3341389561 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b38078c20 con 0x55a53c8894d0 2024-05-27T08:16:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.967+0000 7f1b23e00640 1 -- 172.21.15.63:0/3341389561 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38074990 con 0x55a53c8894d0 2024-05-27T08:16:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.968+0000 7f1b449b4740 1 -- 172.21.15.63:0/3341389561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 msgr2=0x55a53cc4d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.968+0000 7f1b449b4740 1 --2- 172.21.15.63:0/3341389561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 0x55a53cc4d510 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b380687b0 tx=0x7f1b38069bd0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.968+0000 7f1b449b4740 1 -- 172.21.15.63:0/3341389561 shutdown_connections 2024-05-27T08:16:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.968+0000 7f1b449b4740 1 --2- 172.21.15.63:0/3341389561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 0x55a53cc4d510 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.968+0000 7f1b449b4740 1 --2- 172.21.15.63:0/3341389561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc4cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.968+0000 7f1b449b4740 1 --2- 172.21.15.63:0/3341389561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 0x55a53cc4ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.968+0000 7f1b449b4740 1 -- 172.21.15.63:0/3341389561 >> 172.21.15.63:0/3341389561 conn(0x55a53c89e300 msgr2=0x55a53cc49240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.969+0000 7f1b449b4740 1 -- 172.21.15.63:0/3341389561 shutdown_connections 2024-05-27T08:16:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.969+0000 7f1b449b4740 1 -- 172.21.15.63:0/3341389561 wait complete. 2024-05-27T08:16:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.970+0000 7f1b449b4740 1 -- start start 2024-05-27T08:16:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.970+0000 7f1b449b4740 1 Processor -- start 2024-05-27T08:16:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.971+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc56c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.971+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 0x55a53cc57190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.971+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 0x55a53cc62ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.971+0000 7f1b449b4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a53c893790 con 0x55a53c8812f0 2024-05-27T08:16:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.971+0000 7f1b449b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a53cb55bc0 con 0x55a53c8aa1b0 2024-05-27T08:16:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.971+0000 7f1b449b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a53cc59bc0 con 0x55a53c8894d0 2024-05-27T08:16:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.971+0000 7f1b449b4740 10 client.0 _ll_register_callbacks cb 0x55a53cb33400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.971+0000 7f1b37400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 0x55a53cc62ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.972+0000 7f1b37400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 0x55a53cc62ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:50164/0 (socket says 172.21.15.63:50164) 2024-05-27T08:16:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.972+0000 7f1b37400640 1 -- 172.21.15.63:0/3187823085 learned_addr learned my addr 172.21.15.63:0/3187823085 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.972+0000 7f1b36000640 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 0x55a53cc57190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.972+0000 7f1b37400640 1 -- 172.21.15.63:0/3187823085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 msgr2=0x55a53cc57190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.972+0000 7f1b37400640 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 0x55a53cc57190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.972+0000 7f1b37400640 1 -- 172.21.15.63:0/3187823085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 msgr2=0x55a53cc56c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.972+0000 7f1b37400640 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc56c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.973+0000 7f1b37400640 1 -- 172.21.15.63:0/3187823085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53c8c2210 con 0x55a53c8aa1b0 2024-05-27T08:16:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.973+0000 7f1b20c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:16:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.973+0000 7f1b37400640 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 0x55a53cc62ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1b38052210 tx=0x7f1b38078fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a99323ec4fece159 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.974+0000 7f1b20c00640 1 -- 172.21.15.63:0/3187823085 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3808b020 con 0x55a53c8aa1b0 2024-05-27T08:16:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.974+0000 7f1b20c00640 1 -- 172.21.15.63:0/3187823085 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f1b38078790 con 0x55a53c8aa1b0 2024-05-27T08:16:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.974+0000 7f1b449b4740 1 -- 172.21.15.63:0/3187823085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b38078c20 con 0x55a53c8aa1b0 2024-05-27T08:16:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.987+0000 7f1b20c00640 1 -- 172.21.15.63:0/3187823085 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1b38078990 con 0x55a53c8aa1b0 2024-05-27T08:16:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.988+0000 7f1b449b4740 10 client.42269 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.988+0000 7f1b449b4740 1 -- 172.21.15.63:0/3187823085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a53c98e620 con 0x55a53c8aa1b0 2024-05-27T08:16:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.990+0000 7f1b20c00640 1 -- 172.21.15.63:0/3187823085 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b38078c20 con 0x55a53c8aa1b0 2024-05-27T08:16:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.991+0000 7f1b20c00640 1 -- 172.21.15.63:0/3187823085 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1b3807e7e0 con 0x55a53c8aa1b0 2024-05-27T08:16:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.992+0000 7f1b449b4740 10 client.42269 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.992+0000 7f1b449b4740 10 client.42269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:16:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.992+0000 7f1b449b4740 20 client.42269 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.992+0000 7f1b449b4740 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a53c868d80 0x55a53cc68da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.992+0000 7f1b449b4740 4 client.42269 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:16:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.992+0000 7f1b449b4740 1 -- 172.21.15.63:0/3187823085 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a53cbbfae0 con 0x55a53c868d80 2024-05-27T08:16:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.993+0000 7f1b36a00640 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a53c868d80 0x55a53cc68da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.995+0000 7f1b36a00640 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a53c868d80 0x55a53cc68da0 crc :-1 s=READY pgs=744 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510edd13896e3be1 server_cookie=70fca58971bef263 in_seq=0 out_seq=0 2024-05-27T08:16:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.995+0000 7f1b20c00640 10 client.42269 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.999+0000 7f1b20c00640 1 -- 172.21.15.63:0/3187823085 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a53cbbfae0 con 0x55a53c868d80 2024-05-27T08:16:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.999+0000 7f1b20c00640 10 client.42269 handle_command_reply: tid=0 2024-05-27T08:16:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:00.999+0000 7f1b449b4740 1 client.42269 shutdown 2024-05-27T08:16:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b449b4740 1 -- 172.21.15.63:0/3187823085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 msgr2=0x55a53cc62ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b449b4740 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 0x55a53cc62ef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1b38052210 tx=0x7f1b38078fc0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b449b4740 1 -- 172.21.15.63:0/3187823085 shutdown_connections 2024-05-27T08:16:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b449b4740 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a53c868d80 0x55a53cc68da0 crc :-1 s=READY pgs=744 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b449b4740 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8aa1b0 0x55a53cc62ef0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b449b4740 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8894d0 0x55a53cc57190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b449b4740 1 --2- 172.21.15.63:0/3187823085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc56c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b449b4740 1 -- 172.21.15.63:0/3187823085 >> 172.21.15.63:0/3187823085 conn(0x55a53c89e300 msgr2=0x55a53cc4e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b20c00640 0 client.42269 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b449b4740 1 -- 172.21.15.63:0/3187823085 shutdown_connections 2024-05-27T08:16:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.000+0000 7f1b449b4740 1 -- 172.21.15.63:0/3187823085 wait complete. 2024-05-27T08:16:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.001+0000 7f1b449b4740 20 client.42269 trim_cache size 0 max 16384 2024-05-27T08:16:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.050+0000 7f1b449b4740 1 Processor -- start 2024-05-27T08:16:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.051+0000 7f1b449b4740 1 -- start start 2024-05-27T08:16:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.051+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc52460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.051+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 0x55a53cc4da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.051+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc4e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.052+0000 7f1b449b4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b180127d0 con 0x55a53c8812f0 2024-05-27T08:16:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.052+0000 7f1b449b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b180100a0 con 0x55a53c8894d0 2024-05-27T08:16:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.052+0000 7f1b37400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc4e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.052+0000 7f1b37400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc4e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:37498/0 (socket says 172.21.15.63:37498) 2024-05-27T08:16:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.052+0000 7f1b37400640 1 -- 172.21.15.63:0/2571283570 learned_addr learned my addr 172.21.15.63:0/2571283570 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.052+0000 7f1b449b4740 1 -- 172.21.15.63:0/2571283570 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1800df10 con 0x55a53c8aa1b0 2024-05-27T08:16:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.053+0000 7f1b36000640 1 --2- 172.21.15.63:0/2571283570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 0x55a53cc4da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.053+0000 7f1b36a00640 1 --2- 172.21.15.63:0/2571283570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc52460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.054+0000 7f1b37400640 1 -- 172.21.15.63:0/2571283570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 msgr2=0x55a53cc4da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.054+0000 7f1b37400640 1 --2- 172.21.15.63:0/2571283570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 0x55a53cc4da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.054+0000 7f1b37400640 1 -- 172.21.15.63:0/2571283570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 msgr2=0x55a53cc52460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.054+0000 7f1b37400640 1 --2- 172.21.15.63:0/2571283570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc52460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.054+0000 7f1b37400640 1 -- 172.21.15.63:0/2571283570 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53cc49ca0 con 0x55a53c8aa1b0 2024-05-27T08:16:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.055+0000 7f1b37400640 1 --2- 172.21.15.63:0/2571283570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc4e770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b38069480 tx=0x7f1b38050eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44fb8198a4089f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.056+0000 7f1b23400640 1 -- 172.21.15.63:0/2571283570 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3809d070 con 0x55a53c8aa1b0 2024-05-27T08:16:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.057+0000 7f1b23400640 1 -- 172.21.15.63:0/2571283570 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b3807e8a0 con 0x55a53c8aa1b0 2024-05-27T08:16:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.057+0000 7f1b23400640 1 -- 172.21.15.63:0/2571283570 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3808b040 con 0x55a53c8aa1b0 2024-05-27T08:16:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.058+0000 7f1b449b4740 1 -- 172.21.15.63:0/2571283570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 msgr2=0x55a53cc4e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.058+0000 7f1b449b4740 1 --2- 172.21.15.63:0/2571283570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc4e770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b38069480 tx=0x7f1b38050eb0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.060+0000 7f1b449b4740 1 -- 172.21.15.63:0/2571283570 shutdown_connections 2024-05-27T08:16:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.060+0000 7f1b449b4740 1 --2- 172.21.15.63:0/2571283570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc4e770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.060+0000 7f1b449b4740 1 --2- 172.21.15.63:0/2571283570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 0x55a53cc4da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.060+0000 7f1b449b4740 1 --2- 172.21.15.63:0/2571283570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc52460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.060+0000 7f1b449b4740 1 -- 172.21.15.63:0/2571283570 >> 172.21.15.63:0/2571283570 conn(0x55a53c89e300 msgr2=0x55a53cc56f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.061+0000 7f1b449b4740 1 -- 172.21.15.63:0/2571283570 shutdown_connections 2024-05-27T08:16:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.062+0000 7f1b449b4740 1 -- 172.21.15.63:0/2571283570 wait complete. 2024-05-27T08:16:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.063+0000 7f1b449b4740 1 -- start start 2024-05-27T08:16:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.063+0000 7f1b449b4740 1 Processor -- start 2024-05-27T08:16:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.064+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc53040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.064+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 0x55a53cc53580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.064+0000 7f1b449b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc53ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.065+0000 7f1b449b4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1800e560 con 0x55a53c8812f0 2024-05-27T08:16:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.065+0000 7f1b449b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1800f9d0 con 0x55a53c8894d0 2024-05-27T08:16:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.065+0000 7f1b449b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1800ec30 con 0x55a53c8aa1b0 2024-05-27T08:16:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.065+0000 7f1b449b4740 10 client.0 _ll_register_callbacks cb 0x7f1b380692d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.065+0000 7f1b37400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc53ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.065+0000 7f1b37400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc53ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:37514/0 (socket says 172.21.15.63:37514) 2024-05-27T08:16:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.065+0000 7f1b37400640 1 -- 172.21.15.63:0/210086844 learned_addr learned my addr 172.21.15.63:0/210086844 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.065+0000 7f1b36000640 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 0x55a53cc53580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.066+0000 7f1b37400640 1 -- 172.21.15.63:0/210086844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 msgr2=0x55a53cc53580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:01.068 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.066+0000 7f1b36a00640 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc53040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.066+0000 7f1b37400640 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 0x55a53cc53580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.067+0000 7f1b37400640 1 -- 172.21.15.63:0/210086844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 msgr2=0x55a53cc53040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.067+0000 7f1b37400640 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc53040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.067+0000 7f1b37400640 1 -- 172.21.15.63:0/210086844 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53cc5e050 con 0x55a53c8aa1b0 2024-05-27T08:16:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.067+0000 7f1b36a00640 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc53040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:16:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.067+0000 7f1b36000640 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 0x55a53cc53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:16:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.068+0000 7f1b37400640 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc53ac0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b38052160 tx=0x7f1b3807ea50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a9ba3ad75c0ce40 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.068+0000 7f1b20c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:16:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.068+0000 7f1b20c00640 1 -- 172.21.15.63:0/210086844 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3808b070 con 0x55a53c8aa1b0 2024-05-27T08:16:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.069+0000 7f1b20c00640 1 -- 172.21.15.63:0/210086844 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(164..164 src has 1..164) ==== 9483+0+0 (secure 0 0 0) 0x7f1b380786b0 con 0x55a53c8aa1b0 2024-05-27T08:16:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.070+0000 7f1b449b4740 1 -- 172.21.15.63:0/210086844 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b3807e8a0 con 0x55a53c8aa1b0 2024-05-27T08:16:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.070+0000 7f1b20c00640 1 -- 172.21.15.63:0/210086844 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1b3809d070 con 0x55a53c8aa1b0 2024-05-27T08:16:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.070+0000 7f1b449b4740 10 client.32092 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.070+0000 7f1b449b4740 1 -- 172.21.15.63:0/210086844 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a53cc5f2c0 con 0x55a53c8aa1b0 2024-05-27T08:16:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.071+0000 7f1b20c00640 1 -- 172.21.15.63:0/210086844 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b3807e8a0 con 0x55a53c8aa1b0 2024-05-27T08:16:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.072+0000 7f1b20c00640 1 -- 172.21.15.63:0/210086844 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1b380a1450 con 0x55a53c8aa1b0 2024-05-27T08:16:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.073+0000 7f1b449b4740 10 client.32092 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.073+0000 7f1b449b4740 10 client.32092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:16:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.073+0000 7f1b449b4740 20 client.32092 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.073+0000 7f1b449b4740 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a53c83e1c0 0x55a53ccd33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.075+0000 7f1b449b4740 4 client.32092 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:16:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.075+0000 7f1b449b4740 1 -- 172.21.15.63:0/210086844 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a53cbb92c0 con 0x55a53c83e1c0 2024-05-27T08:16:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.076+0000 7f1b36a00640 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a53c83e1c0 0x55a53ccd33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.079+0000 7f1b36a00640 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a53c83e1c0 0x55a53ccd33a0 crc :-1 s=READY pgs=745 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89c1cfbaff5515e7 server_cookie=71d60199eab00187 in_seq=0 out_seq=0 2024-05-27T08:16:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:01.079+0000 7f1b20c00640 10 client.32092 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:01.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:00 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:01.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:00 smithi086 ceph-mon[34687]: pgmap v1347: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-05-27T08:16:01.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:00 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:01.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:00 smithi149 ceph-mon[35408]: pgmap v1347: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-05-27T08:16:01.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:00 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:01.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:00 smithi063 ceph-mon[31613]: pgmap v1347: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-05-27T08:16:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.491+0000 7f1b20c00640 1 -- 172.21.15.63:0/210086844 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a53cbb92c0 con 0x55a53c83e1c0 2024-05-27T08:16:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.492+0000 7f1b20c00640 10 client.32092 handle_command_reply: tid=0 2024-05-27T08:16:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.492+0000 7f1b449b4740 1 client.32092 shutdown 2024-05-27T08:16:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.492+0000 7f1b449b4740 1 -- 172.21.15.63:0/210086844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 msgr2=0x55a53cc53ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.492+0000 7f1b449b4740 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc53ac0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b38052160 tx=0x7f1b3807ea50 comp rx=0 tx=0).stop 2024-05-27T08:16:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.492+0000 7f1b449b4740 1 -- 172.21.15.63:0/210086844 shutdown_connections 2024-05-27T08:16:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.492+0000 7f1b449b4740 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a53c83e1c0 0x55a53ccd33a0 crc :-1 s=READY pgs=745 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.492+0000 7f1b449b4740 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a53c8aa1b0 0x55a53cc53ac0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.492+0000 7f1b449b4740 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a53c8894d0 0x55a53cc53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.492+0000 7f1b449b4740 1 --2- 172.21.15.63:0/210086844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a53c8812f0 0x55a53cc53040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.492+0000 7f1b449b4740 1 -- 172.21.15.63:0/210086844 >> 172.21.15.63:0/210086844 conn(0x55a53c89e300 msgr2=0x55a53cc517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.493+0000 7f1b20c00640 0 client.32092 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.493+0000 7f1b449b4740 1 -- 172.21.15.63:0/210086844 shutdown_connections 2024-05-27T08:16:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.493+0000 7f1b449b4740 1 -- 172.21.15.63:0/210086844 wait complete. 2024-05-27T08:16:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.493+0000 7f1b449b4740 20 client.32092 trim_cache size 0 max 16384 2024-05-27T08:16:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:16:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:16:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 -- 172.21.15.63:0/3395646172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b1c0b2db0 msgr2=0x7f1b1c0b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b1c0b2db0 0x7f1b1c0b52c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1b300a3200 tx=0x7f1b28005c90 comp rx=0 tx=0).stop 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 -- 172.21.15.63:0/3395646172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b30097180 msgr2=0x7f1b300a6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b30097180 0x7f1b300a6560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1b30097710 tx=0x7f1b2400db30 comp rx=0 tx=0).stop 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 -- 172.21.15.63:0/3395646172 shutdown_connections 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1b1c0b2db0 0x7f1b1c0b52c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b3009cec0 0x7f1b30150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3009ca20 0x7f1b300a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 --2- 172.21.15.63:0/3395646172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b30097180 0x7f1b300a6560 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 -- 172.21.15.63:0/3395646172 >> 172.21.15.63:0/3395646172 conn(0x7f1b3000c040 msgr2=0x7f1b30095270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.495+0000 7f1b3da00640 1 -- 172.21.15.63:0/3395646172 shutdown_connections 2024-05-27T08:16:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:02.496+0000 7f1b3da00640 1 -- 172.21.15.63:0/3395646172 wait complete. 2024-05-27T08:16:03.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:02 smithi086 ceph-mon[34687]: pgmap v1348: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2024-05-27T08:16:03.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:02 smithi149 ceph-mon[35408]: pgmap v1348: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2024-05-27T08:16:03.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:02 smithi063 ceph-mon[31613]: pgmap v1348: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 3.56k op/s 2024-05-27T08:16:06.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:05 smithi086 ceph-mon[34687]: pgmap v1349: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.7 MiB/s wr, 2.47k op/s 2024-05-27T08:16:06.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:05 smithi149 ceph-mon[35408]: pgmap v1349: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.7 MiB/s wr, 2.47k op/s 2024-05-27T08:16:06.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:05 smithi063 ceph-mon[31613]: pgmap v1349: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.7 MiB/s wr, 2.47k op/s 2024-05-27T08:16:07.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:06 smithi086 ceph-mon[34687]: pgmap v1350: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 11 MiB/s wr, 2.90k op/s 2024-05-27T08:16:07.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:06 smithi149 ceph-mon[35408]: pgmap v1350: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 11 MiB/s wr, 2.90k op/s 2024-05-27T08:16:07.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:06 smithi063 ceph-mon[31613]: pgmap v1350: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 11 MiB/s wr, 2.90k op/s 2024-05-27T08:16:07.734 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:16:07.734 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:16:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.978+0000 7fe64ce00700 1 -- 172.21.15.63:0/1574668049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64810a920 msgr2=0x7fe64810ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.978+0000 7fe64ce00700 1 --2- 172.21.15.63:0/1574668049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64810a920 0x7fe64810ad30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe63800b0a0 tx=0x7fe63800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.978+0000 7fe64ce00700 1 -- 172.21.15.63:0/1574668049 shutdown_connections 2024-05-27T08:16:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.978+0000 7fe64ce00700 1 --2- 172.21.15.63:0/1574668049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64811d920 0x7fe64811fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.978+0000 7fe64ce00700 1 --2- 172.21.15.63:0/1574668049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64810a920 0x7fe64810ad30 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe63800b0a0 tx=0x7fe63800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.978+0000 7fe64ce00700 1 --2- 172.21.15.63:0/1574668049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe64810b270 0x7fe64811d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.978+0000 7fe64ce00700 1 -- 172.21.15.63:0/1574668049 >> 172.21.15.63:0/1574668049 conn(0x7fe64806f4c0 msgr2=0x7fe64806f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:07.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.979+0000 7fe64ce00700 1 -- 172.21.15.63:0/1574668049 shutdown_connections 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.980+0000 7fe64ce00700 1 -- 172.21.15.63:0/1574668049 wait complete. 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.981+0000 7fe64ce00700 1 Processor -- start 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.981+0000 7fe64ce00700 1 -- start start 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.982+0000 7fe64ce00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64810b270 0x7fe6481af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.982+0000 7fe64ce00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64811d920 0x7fe6481afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.982+0000 7fe64ce00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6481a9af0 0x7fe6481a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.982+0000 7fe64ce00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6481231d0 con 0x7fe6481a9af0 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.982+0000 7fe64ce00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe648123050 con 0x7fe64810b270 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.982+0000 7fe64ce00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe648123350 con 0x7fe64811d920 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.983+0000 7fe645e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64810b270 0x7fe6481af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.983+0000 7fe645e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64810b270 0x7fe6481af970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:50214/0 (socket says 172.21.15.63:50214) 2024-05-27T08:16:07.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.983+0000 7fe645e00700 1 -- 172.21.15.63:0/1240321400 learned_addr learned my addr 172.21.15.63:0/1240321400 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.983+0000 7fe645e00700 1 -- 172.21.15.63:0/1240321400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64811d920 msgr2=0x7fe6481afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.983+0000 7fe645e00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64811d920 0x7fe6481afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.983+0000 7fe645e00700 1 -- 172.21.15.63:0/1240321400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6481a9af0 msgr2=0x7fe6481a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.983+0000 7fe645e00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6481a9af0 0x7fe6481a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.983+0000 7fe645e00700 1 -- 172.21.15.63:0/1240321400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6481aa6c0 con 0x7fe64810b270 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.984+0000 7fe645e00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64810b270 0x7fe6481af970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe64810a850 tx=0x7fe638003d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.985+0000 7fe63ea00700 1 -- 172.21.15.63:0/1240321400 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63801f030 con 0x7fe64810b270 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.985+0000 7fe63ea00700 1 -- 172.21.15.63:0/1240321400 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe638007e10 con 0x7fe64810b270 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.985+0000 7fe63ea00700 1 -- 172.21.15.63:0/1240321400 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63801b430 con 0x7fe64810b270 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.985+0000 7fe64ce00700 1 -- 172.21.15.63:0/1240321400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6481b6760 con 0x7fe64810b270 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.985+0000 7fe64ce00700 1 -- 172.21.15.63:0/1240321400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6481b6b00 con 0x7fe64810b270 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:06.987+0000 7fe64ce00700 1 -- 172.21.15.63:0/1240321400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6480512b0 con 0x7fe64810b270 2024-05-27T08:16:07.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.003+0000 7fe63ea00700 1 -- 172.21.15.63:0/1240321400 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe638022030 con 0x7fe64810b270 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.006+0000 7fe63ea00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe62c0b2c00 0x7fe62c0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.007+0000 7fe63ea00700 1 -- 172.21.15.63:0/1240321400 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 9571+0+0 (secure 0 0 0) 0x7fe6380e32e0 con 0x7fe64810b270 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.007+0000 7fe63ea00700 1 -- 172.21.15.63:0/1240321400 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6380e37a0 con 0x7fe64810b270 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.007+0000 7fe63fe00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe62c0b2c00 0x7fe62c0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.009+0000 7fe63fe00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe62c0b2c00 0x7fe62c0b5110 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe630005950 tx=0x7fe6300058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.576+0000 7fe64ce00700 1 -- 172.21.15.63:0/1240321400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6481aa890 con 0x7fe64810b270 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.578+0000 7fe63ea00700 1 -- 172.21.15.63:0/1240321400 <== mon.2 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7fe6481aa890 con 0x7fe64810b270 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.583+0000 7fe64ce00700 1 -- 172.21.15.63:0/1240321400 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe62c0b2c00 msgr2=0x7fe62c0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.583+0000 7fe64ce00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe62c0b2c00 0x7fe62c0b5110 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe630005950 tx=0x7fe6300058e0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.583+0000 7fe64ce00700 1 -- 172.21.15.63:0/1240321400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64810b270 msgr2=0x7fe6481af970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.583+0000 7fe64ce00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64810b270 0x7fe6481af970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe64810a850 tx=0x7fe638003d30 comp rx=0 tx=0).stop 2024-05-27T08:16:07.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.583+0000 7fe64ce00700 1 -- 172.21.15.63:0/1240321400 shutdown_connections 2024-05-27T08:16:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.583+0000 7fe64ce00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe62c0b2c00 0x7fe62c0b5110 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.583+0000 7fe64ce00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6481a9af0 0x7fe6481a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.583+0000 7fe64ce00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe64810b270 0x7fe6481af970 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.583+0000 7fe64ce00700 1 --2- 172.21.15.63:0/1240321400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64811d920 0x7fe6481afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.583+0000 7fe64ce00700 1 -- 172.21.15.63:0/1240321400 >> 172.21.15.63:0/1240321400 conn(0x7fe64806f4c0 msgr2=0x7fe64811b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.584+0000 7fe64ce00700 1 -- 172.21.15.63:0/1240321400 shutdown_connections 2024-05-27T08:16:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:07.586+0000 7fe64ce00700 1 -- 172.21.15.63:0/1240321400 wait complete. 2024-05-27T08:16:07.739 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:16:08.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:07 smithi086 ceph-mon[34687]: osdmap e165: 12 total, 12 up, 12 in 2024-05-27T08:16:08.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:07 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1240321400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:16:08.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:07 smithi149 ceph-mon[35408]: osdmap e165: 12 total, 12 up, 12 in 2024-05-27T08:16:08.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:07 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1240321400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:16:08.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:07 smithi063 ceph-mon[31613]: osdmap e165: 12 total, 12 up, 12 in 2024-05-27T08:16:08.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:07 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1240321400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:16:08.587 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:16:09.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:08 smithi086 ceph-mon[34687]: pgmap v1352: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:16:09.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:08 smithi149 ceph-mon[35408]: pgmap v1352: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:16:09.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:08 smithi063 ceph-mon[31613]: pgmap v1352: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:16:09.717 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:16:11.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:11 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:11.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:11 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:11.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:11 smithi086 ceph-mon[34687]: pgmap v1353: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:16:11.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:11 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:11.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:11 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:11.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:11 smithi149 ceph-mon[35408]: pgmap v1353: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:16:11.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:11 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:11.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:11 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:11.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:11 smithi063 ceph-mon[31613]: pgmap v1353: 129 pgs: 8 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 96 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:16:13.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:12 smithi086 ceph-mon[34687]: pgmap v1354: 129 pgs: 17 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 71 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:16:13.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:12 smithi149 ceph-mon[35408]: pgmap v1354: 129 pgs: 17 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 71 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:16:13.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:12 smithi063 ceph-mon[31613]: pgmap v1354: 129 pgs: 17 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 71 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:16:14.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:16:14.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:16:14.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:16:14.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:16:14.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:16:14.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:16:14.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:16:15.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:15 smithi086 ceph-mon[34687]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T08:16:15.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:15 smithi086 ceph-mon[34687]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T08:16:15.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:15 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:15.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:15 smithi086 ceph-mon[34687]: pgmap v1355: 129 pgs: 17 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 71 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 10 MiB/s wr, 2.59k op/s 2024-05-27T08:16:15.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:15 smithi149 ceph-mon[35408]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T08:16:15.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:15 smithi149 ceph-mon[35408]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T08:16:15.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:15 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:15.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:15 smithi149 ceph-mon[35408]: pgmap v1355: 129 pgs: 17 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 71 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 10 MiB/s wr, 2.59k op/s 2024-05-27T08:16:15.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:15 smithi063 ceph-mon[31613]: Adjusting osd_memory_target on smithi063 to 176.4M 2024-05-27T08:16:15.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:15 smithi063 ceph-mon[31613]: Unable to set osd_memory_target on smithi063 to 185070592: error parsing value: Value '185070592' is below minimum 939524096 2024-05-27T08:16:15.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:15 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:16:15.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:15 smithi063 ceph-mon[31613]: pgmap v1355: 129 pgs: 17 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 71 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 10 MiB/s wr, 2.59k op/s 2024-05-27T08:16:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.887+0000 7f8822800640 1 -- 172.21.15.63:0/3403658069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f881c11d240 msgr2=0x7f881c11f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.887+0000 7f8822800640 1 --2- 172.21.15.63:0/3403658069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f881c11d240 0x7f881c11f660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8814009be0 tx=0x7f881402a750 comp rx=0 tx=0).stop 2024-05-27T08:16:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.888+0000 7f8822800640 1 -- 172.21.15.63:0/3403658069 shutdown_connections 2024-05-27T08:16:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.888+0000 7f8822800640 1 --2- 172.21.15.63:0/3403658069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f881c11d240 0x7f881c11f660 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.888+0000 7f8822800640 1 --2- 172.21.15.63:0/3403658069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f881c11a8e0 0x7f881c11cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.888+0000 7f8822800640 1 --2- 172.21.15.63:0/3403658069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c10a1f0 0x7f881c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.888+0000 7f8822800640 1 -- 172.21.15.63:0/3403658069 >> 172.21.15.63:0/3403658069 conn(0x7f881c06e8f0 msgr2=0x7f881c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.888+0000 7f8822800640 1 -- 172.21.15.63:0/3403658069 shutdown_connections 2024-05-27T08:16:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.888+0000 7f8822800640 1 -- 172.21.15.63:0/3403658069 wait complete. 2024-05-27T08:16:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.890+0000 7f8822800640 1 Processor -- start 2024-05-27T08:16:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.890+0000 7f8822800640 1 -- start start 2024-05-27T08:16:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.891+0000 7f8822800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c10a1f0 0x7f881c1a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.891+0000 7f8822800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f881c11a8e0 0x7f881c1a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.891+0000 7f8822800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f881c11d240 0x7f881c1addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.891+0000 7f8822800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f881c122ad0 con 0x7f881c11d240 2024-05-27T08:16:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.891+0000 7f8822800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f881c122950 con 0x7f881c10a1f0 2024-05-27T08:16:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.891+0000 7f8822800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f881c122c50 con 0x7f881c11a8e0 2024-05-27T08:16:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.891+0000 7f881b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c10a1f0 0x7f881c1a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.891+0000 7f881b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c10a1f0 0x7f881c1a9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:47794/0 (socket says 172.21.15.63:47794) 2024-05-27T08:16:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.891+0000 7f881b400640 1 -- 172.21.15.63:0/3439525074 learned_addr learned my addr 172.21.15.63:0/3439525074 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.892+0000 7f881be00640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f881c11d240 0x7f881c1addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.892+0000 7f881b400640 1 -- 172.21.15.63:0/3439525074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f881c11a8e0 msgr2=0x7f881c1a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.892+0000 7f881b400640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f881c11a8e0 0x7f881c1a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.892+0000 7f881b400640 1 -- 172.21.15.63:0/3439525074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f881c11d240 msgr2=0x7f881c1addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.892+0000 7f881b400640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f881c11d240 0x7f881c1addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.892+0000 7f881b400640 1 -- 172.21.15.63:0/3439525074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c1ae3f0 con 0x7f881c10a1f0 2024-05-27T08:16:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.893+0000 7f881b400640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c10a1f0 0x7f881c1a9590 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f880c00e9c0 tx=0x7f880c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.893+0000 7f8813e00640 1 -- 172.21.15.63:0/3439525074 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c00ccc0 con 0x7f881c10a1f0 2024-05-27T08:16:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.893+0000 7f8813e00640 1 -- 172.21.15.63:0/3439525074 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f880c0054b0 con 0x7f881c10a1f0 2024-05-27T08:16:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.893+0000 7f8822800640 1 -- 172.21.15.63:0/3439525074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c1ae6e0 con 0x7f881c10a1f0 2024-05-27T08:16:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.893+0000 7f8813e00640 1 -- 172.21.15.63:0/3439525074 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c010430 con 0x7f881c10a1f0 2024-05-27T08:16:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.893+0000 7f8822800640 1 -- 172.21.15.63:0/3439525074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c1b6060 con 0x7f881c10a1f0 2024-05-27T08:16:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.898+0000 7f882985d740 1 Processor -- start 2024-05-27T08:16:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.898+0000 7f882985d740 1 -- start start 2024-05-27T08:16:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.899+0000 7f882985d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 0x55b053da4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.902+0000 7f882985d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053da4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.902+0000 7f882985d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 0x55b053da52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.902+0000 7f882985d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b053d15b90 con 0x55b0539e14d0 2024-05-27T08:16:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.902+0000 7f882985d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b053d05730 con 0x55b053a021b0 2024-05-27T08:16:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.902+0000 7f882985d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b053d09770 con 0x55b0539d92f0 2024-05-27T08:16:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.903+0000 7f881aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 0x55b053da52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.903+0000 7f881aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 0x55b053da52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37750/0 (socket says 172.21.15.63:37750) 2024-05-27T08:16:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.903+0000 7f881aa00640 1 -- 172.21.15.63:0/4062634608 learned_addr learned my addr 172.21.15.63:0/4062634608 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.904+0000 7f881be00640 1 --2- 172.21.15.63:0/4062634608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 0x55b053da4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.904+0000 7f8813e00640 1 -- 172.21.15.63:0/3439525074 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f880c0046c0 con 0x7f881c10a1f0 2024-05-27T08:16:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.905+0000 7f881b400640 1 --2- 172.21.15.63:0/4062634608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053da4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.906+0000 7f881be00640 1 -- 172.21.15.63:0/4062634608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 msgr2=0x55b053da4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.906+0000 7f881be00640 1 --2- 172.21.15.63:0/4062634608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053da4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.906+0000 7f881be00640 1 -- 172.21.15.63:0/4062634608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 msgr2=0x55b053da52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.906+0000 7f881be00640 1 --2- 172.21.15.63:0/4062634608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 0x55b053da52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.906+0000 7f881be00640 1 -- 172.21.15.63:0/4062634608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b053c9e8b0 con 0x55b053a021b0 2024-05-27T08:16:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.906+0000 7f881be00640 1 --2- 172.21.15.63:0/4062634608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 0x55b053da4860 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8814009b30 tx=0x7f88140303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74decadeca388196 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.908+0000 7f8813e00640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88000b2a20 0x7f88000b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.908+0000 7f8813e00640 1 -- 172.21.15.63:0/3439525074 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 9571+0+0 (secure 0 0 0) 0x7f880c0e2ed0 con 0x7f881c10a1f0 2024-05-27T08:16:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.909+0000 7f8810c00640 1 -- 172.21.15.63:0/4062634608 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814040030 con 0x55b053a021b0 2024-05-27T08:16:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.909+0000 7f8810c00640 1 -- 172.21.15.63:0/4062634608 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8814037040 con 0x55b053a021b0 2024-05-27T08:16:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.909+0000 7f8810c00640 1 -- 172.21.15.63:0/4062634608 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814036070 con 0x55b053a021b0 2024-05-27T08:16:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.910+0000 7f882985d740 1 -- 172.21.15.63:0/4062634608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 msgr2=0x55b053da4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.910+0000 7f882985d740 1 --2- 172.21.15.63:0/4062634608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 0x55b053da4860 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8814009b30 tx=0x7f88140303a0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.910+0000 7f882985d740 1 -- 172.21.15.63:0/4062634608 shutdown_connections 2024-05-27T08:16:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.910+0000 7f882985d740 1 --2- 172.21.15.63:0/4062634608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 0x55b053da52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.910+0000 7f882985d740 1 --2- 172.21.15.63:0/4062634608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053da4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.910+0000 7f882985d740 1 --2- 172.21.15.63:0/4062634608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 0x55b053da4860 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.910+0000 7f882985d740 1 -- 172.21.15.63:0/4062634608 >> 172.21.15.63:0/4062634608 conn(0x55b0539f6300 msgr2=0x55b053da1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.910+0000 7f881b400640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88000b2a20 0x7f88000b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.911+0000 7f882985d740 1 -- 172.21.15.63:0/4062634608 shutdown_connections 2024-05-27T08:16:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.911+0000 7f882985d740 1 -- 172.21.15.63:0/4062634608 wait complete. 2024-05-27T08:16:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.912+0000 7f882985d740 1 -- start start 2024-05-27T08:16:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.912+0000 7f882985d740 1 Processor -- start 2024-05-27T08:16:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.913+0000 7f882985d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053daece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.913+0000 7f882985d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539e14d0 0x55b053daf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.913+0000 7f882985d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b053a021b0 0x55b053dbac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.913+0000 7f882985d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b0539eb790 con 0x55b053a021b0 2024-05-27T08:16:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.913+0000 7f882985d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b053cadbc0 con 0x55b0539e14d0 2024-05-27T08:16:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.913+0000 7f882985d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b053db1930 con 0x55b0539d92f0 2024-05-27T08:16:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.913+0000 7f882985d740 10 client.0 _ll_register_callbacks cb 0x55b053c8b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.914+0000 7f881be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053daece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.914+0000 7f881b400640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88000b2a20 0x7f88000b4f30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f880c0e02b0 tx=0x7f880c0cc040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.914+0000 7f881be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053daece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:43476/0 (socket says 172.21.15.63:43476) 2024-05-27T08:16:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.914+0000 7f881be00640 1 -- 172.21.15.63:0/3541834605 learned_addr learned my addr 172.21.15.63:0/3541834605 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.914+0000 7f881be00640 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b053a021b0 0x55b053dbac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.916+0000 7f881aa00640 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539e14d0 0x55b053daf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.916+0000 7f881be00640 1 -- 172.21.15.63:0/3541834605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539e14d0 msgr2=0x55b053daf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.916+0000 7f881be00640 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539e14d0 0x55b053daf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.916+0000 7f881be00640 1 -- 172.21.15.63:0/3541834605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b053a021b0 msgr2=0x55b053dbac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.916+0000 7f881be00640 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b053a021b0 0x55b053dbac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.916+0000 7f881be00640 1 -- 172.21.15.63:0/3541834605 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b053a1a210 con 0x55b0539d92f0 2024-05-27T08:16:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.916+0000 7f881aa00640 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539e14d0 0x55b053daf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:16:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.916+0000 7f881be00640 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053daece0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8814030ad0 tx=0x7f8814009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c823c91daa4598cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.917+0000 7f8805600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:16:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.918+0000 7f8805600640 1 -- 172.21.15.63:0/3541834605 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814049020 con 0x55b0539d92f0 2024-05-27T08:16:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.918+0000 7f8805600640 1 -- 172.21.15.63:0/3541834605 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(165..165 src has 1..165) ==== 9571+0+0 (secure 0 0 0) 0x7f8814003970 con 0x55b0539d92f0 2024-05-27T08:16:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.919+0000 7f882985d740 1 -- 172.21.15.63:0/3541834605 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8814037040 con 0x55b0539d92f0 2024-05-27T08:16:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.919+0000 7f8805600640 1 -- 172.21.15.63:0/3541834605 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8814036050 con 0x55b0539d92f0 2024-05-27T08:16:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.919+0000 7f882985d740 10 client.32104 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.919+0000 7f882985d740 1 -- 172.21.15.63:0/3541834605 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b053ae6620 con 0x55b0539d92f0 2024-05-27T08:16:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.920+0000 7f8805600640 1 -- 172.21.15.63:0/3541834605 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8814037040 con 0x55b0539d92f0 2024-05-27T08:16:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.921+0000 7f8805600640 1 -- 172.21.15.63:0/3541834605 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f88140046e0 con 0x55b0539d92f0 2024-05-27T08:16:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.922+0000 7f882985d740 10 client.32104 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.922+0000 7f882985d740 10 client.32104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:16:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.922+0000 7f882985d740 20 client.32104 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.922+0000 7f882985d740 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b0539c0d80 0x55b053dc0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.922+0000 7f882985d740 4 client.32104 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:16:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.922+0000 7f882985d740 1 -- 172.21.15.63:0/3541834605 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b053d17ae0 con 0x55b0539c0d80 2024-05-27T08:16:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.922+0000 7f881aa00640 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b0539c0d80 0x55b053dc0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.924+0000 7f881aa00640 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b0539c0d80 0x55b053dc0b20 crc :-1 s=READY pgs=746 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b519e9ee9c4fdb3 server_cookie=745fde89b298b182 in_seq=0 out_seq=0 2024-05-27T08:16:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.924+0000 7f8805600640 10 client.32104 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.926+0000 7f8805600640 1 -- 172.21.15.63:0/3541834605 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b053d17ae0 con 0x55b0539c0d80 2024-05-27T08:16:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.926+0000 7f8805600640 10 client.32104 handle_command_reply: tid=0 2024-05-27T08:16:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.926+0000 7f882985d740 1 client.32104 shutdown 2024-05-27T08:16:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.928+0000 7f882985d740 1 -- 172.21.15.63:0/3541834605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 msgr2=0x55b053daece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.928+0000 7f882985d740 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053daece0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8814030ad0 tx=0x7f8814009500 comp rx=0 tx=0).stop 2024-05-27T08:16:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.928+0000 7f882985d740 1 -- 172.21.15.63:0/3541834605 shutdown_connections 2024-05-27T08:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.928+0000 7f882985d740 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b0539c0d80 0x55b053dc0b20 crc :-1 s=READY pgs=746 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.928+0000 7f882985d740 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b053a021b0 0x55b053dbac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.928+0000 7f882985d740 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539e14d0 0x55b053daf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.928+0000 7f882985d740 1 --2- 172.21.15.63:0/3541834605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053daece0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.928+0000 7f882985d740 1 -- 172.21.15.63:0/3541834605 >> 172.21.15.63:0/3541834605 conn(0x55b0539f6300 msgr2=0x55b053da4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.928+0000 7f8805600640 0 client.32104 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.928+0000 7f882985d740 1 -- 172.21.15.63:0/3541834605 shutdown_connections 2024-05-27T08:16:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.929+0000 7f882985d740 1 -- 172.21.15.63:0/3541834605 wait complete. 2024-05-27T08:16:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.929+0000 7f882985d740 20 client.32104 trim_cache size 0 max 16384 2024-05-27T08:16:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.972+0000 7f882985d740 1 Processor -- start 2024-05-27T08:16:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.972+0000 7f882985d740 1 -- start start 2024-05-27T08:16:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.973+0000 7f882985d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539d92f0 0x55b053dbabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.973+0000 7f882985d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539e14d0 0x55b053dabd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.973+0000 7f882985d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b053a021b0 0x55b053dac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.973+0000 7f882985d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1c2ab0 con 0x55b053a021b0 2024-05-27T08:16:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.973+0000 7f882985d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1c0380 con 0x55b0539d92f0 2024-05-27T08:16:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.973+0000 7f882985d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1be280 con 0x55b0539e14d0 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.973+0000 7f881be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b053a021b0 0x55b053dac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.973+0000 7f881be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539d92f0 0x55b053dbabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.974+0000 7f881be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539d92f0 0x55b053dbabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:47832/0 (socket says 172.21.15.63:47832) 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.974+0000 7f881be00640 1 -- 172.21.15.63:0/2463809992 learned_addr learned my addr 172.21.15.63:0/2463809992 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.974+0000 7f881aa00640 1 --2- 172.21.15.63:0/2463809992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539e14d0 0x55b053dabd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.974+0000 7f881be00640 1 -- 172.21.15.63:0/2463809992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539e14d0 msgr2=0x55b053dabd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.974+0000 7f881be00640 1 --2- 172.21.15.63:0/2463809992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539e14d0 0x55b053dabd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.974+0000 7f881be00640 1 -- 172.21.15.63:0/2463809992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b053a021b0 msgr2=0x55b053dac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.974+0000 7f881be00640 1 --2- 172.21.15.63:0/2463809992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b053a021b0 0x55b053dac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.974+0000 7f881be00640 1 -- 172.21.15.63:0/2463809992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b053da50f0 con 0x55b0539d92f0 2024-05-27T08:16:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.974+0000 7f881be00640 1 --2- 172.21.15.63:0/2463809992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539d92f0 0x55b053dbabd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f881402ad40 tx=0x7f8814000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdfec27bc92c7bec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.976+0000 7f8807e00640 1 -- 172.21.15.63:0/2463809992 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814049020 con 0x55b0539d92f0 2024-05-27T08:16:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.976+0000 7f8807e00640 1 -- 172.21.15.63:0/2463809992 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f881405c040 con 0x55b0539d92f0 2024-05-27T08:16:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.976+0000 7f8807e00640 1 -- 172.21.15.63:0/2463809992 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881405b070 con 0x55b0539d92f0 2024-05-27T08:16:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.977+0000 7f882985d740 1 -- 172.21.15.63:0/2463809992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539d92f0 msgr2=0x55b053dbabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.977+0000 7f882985d740 1 --2- 172.21.15.63:0/2463809992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539d92f0 0x55b053dbabd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f881402ad40 tx=0x7f8814000f40 comp rx=0 tx=0).stop 2024-05-27T08:16:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.977+0000 7f882985d740 1 -- 172.21.15.63:0/2463809992 shutdown_connections 2024-05-27T08:16:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.977+0000 7f882985d740 1 --2- 172.21.15.63:0/2463809992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b053a021b0 0x55b053dac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.977+0000 7f882985d740 1 --2- 172.21.15.63:0/2463809992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539e14d0 0x55b053dabd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.977+0000 7f882985d740 1 --2- 172.21.15.63:0/2463809992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0539d92f0 0x55b053dbabd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.977+0000 7f882985d740 1 -- 172.21.15.63:0/2463809992 >> 172.21.15.63:0/2463809992 conn(0x55b0539f6300 msgr2=0x55b053db0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.978+0000 7f882985d740 1 -- 172.21.15.63:0/2463809992 shutdown_connections 2024-05-27T08:16:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.978+0000 7f882985d740 1 -- 172.21.15.63:0/2463809992 wait complete. 2024-05-27T08:16:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.979+0000 7f882985d740 1 -- start start 2024-05-27T08:16:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.979+0000 7f882985d740 1 Processor -- start 2024-05-27T08:16:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.980+0000 7f882985d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053dc29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.980+0000 7f882985d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 0x55b053dc2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.980+0000 7f882985d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 0x55b053dc3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.980+0000 7f882985d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1be840 con 0x55b0539e14d0 2024-05-27T08:16:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.980+0000 7f882985d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1bfcb0 con 0x55b053a021b0 2024-05-27T08:16:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.980+0000 7f882985d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1bef10 con 0x55b0539d92f0 2024-05-27T08:16:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.981+0000 7f882985d740 10 client.0 _ll_register_callbacks cb 0x7f8808009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.981+0000 7f881aa00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 0x55b053dc2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.982+0000 7f881be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053dc29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.982+0000 7f881be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 0x55b053dc3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.982+0000 7f881be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 0x55b053dc3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:47838/0 (socket says 172.21.15.63:47838) 2024-05-27T08:16:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.982+0000 7f881be00640 1 -- 172.21.15.63:0/747403845 learned_addr learned my addr 172.21.15.63:0/747403845 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.984+0000 7f881aa00640 1 -- 172.21.15.63:0/747403845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 msgr2=0x55b053dc29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.984+0000 7f881aa00640 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053dc29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.984+0000 7f881aa00640 1 -- 172.21.15.63:0/747403845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 msgr2=0x55b053dc3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.984+0000 7f881aa00640 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 0x55b053dc3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.984+0000 7f881aa00640 1 -- 172.21.15.63:0/747403845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b053e38050 con 0x55b0539e14d0 2024-05-27T08:16:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.984+0000 7f881aa00640 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 0x55b053dc2f20 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7f880801dfd0 tx=0x7f8808010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34122c0459fa445 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.984+0000 7f8805600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:16:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.985+0000 7f8805600640 1 -- 172.21.15.63:0/747403845 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880801d7c0 con 0x55b0539e14d0 2024-05-27T08:16:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.985+0000 7f8805600640 1 -- 172.21.15.63:0/747403845 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(165..165 src has 1..165) ==== 9571+0+0 (secure 0 0 0) 0x7f880801d960 con 0x55b0539e14d0 2024-05-27T08:16:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.986+0000 7f882985d740 1 -- 172.21.15.63:0/747403845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f881405c040 con 0x55b0539e14d0 2024-05-27T08:16:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.986+0000 7f8805600640 1 -- 172.21.15.63:0/747403845 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f880802b050 con 0x55b0539e14d0 2024-05-27T08:16:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.986+0000 7f882985d740 10 client.23424 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.986+0000 7f882985d740 1 -- 172.21.15.63:0/747403845 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b053e392e0 con 0x55b0539e14d0 2024-05-27T08:16:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.988+0000 7f8805600640 1 -- 172.21.15.63:0/747403845 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f881405c040 con 0x55b0539e14d0 2024-05-27T08:16:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.988+0000 7f8805600640 1 -- 172.21.15.63:0/747403845 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f88080253c0 con 0x55b0539e14d0 2024-05-27T08:16:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.989+0000 7f882985d740 10 client.23424 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.989+0000 7f882985d740 10 client.23424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:16:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.989+0000 7f882985d740 20 client.23424 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.989+0000 7f882985d740 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b0539961c0 0x55b053e3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.989+0000 7f882985d740 4 client.23424 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:16:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.989+0000 7f882985d740 1 -- 172.21.15.63:0/747403845 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b053d112c0 con 0x55b0539961c0 2024-05-27T08:16:15.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.990+0000 7f881be00640 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b0539961c0 0x55b053e3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.991+0000 7f881be00640 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b0539961c0 0x55b053e3ba10 crc :-1 s=READY pgs=747 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac40ce1a7435e10a server_cookie=ed287031bf1d8835 in_seq=0 out_seq=0 2024-05-27T08:16:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:15.991+0000 7f8805600640 10 client.23424 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:17.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:16 smithi086 ceph-mon[34687]: pgmap v1356: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 2.85k op/s 2024-05-27T08:16:17.212 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:16 smithi149 ceph-mon[35408]: pgmap v1356: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 2.85k op/s 2024-05-27T08:16:17.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:16 smithi063 ceph-mon[31613]: pgmap v1356: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 2.85k op/s 2024-05-27T08:16:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.046+0000 7f8805600640 1 -- 172.21.15.63:0/747403845 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b053d112c0 con 0x55b0539961c0 2024-05-27T08:16:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.046+0000 7f8805600640 10 client.23424 handle_command_reply: tid=0 2024-05-27T08:16:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.046+0000 7f882985d740 1 client.23424 shutdown 2024-05-27T08:16:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.046+0000 7f882985d740 1 -- 172.21.15.63:0/747403845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 msgr2=0x55b053dc2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.046+0000 7f882985d740 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 0x55b053dc2f20 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7f880801dfd0 tx=0x7f8808010800 comp rx=0 tx=0).stop 2024-05-27T08:16:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.048+0000 7f882985d740 1 -- 172.21.15.63:0/747403845 shutdown_connections 2024-05-27T08:16:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.048+0000 7f882985d740 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b0539961c0 0x55b053e3ba10 crc :-1 s=READY pgs=747 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.048+0000 7f882985d740 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b053a021b0 0x55b053dc3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.048+0000 7f882985d740 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0539e14d0 0x55b053dc2f20 unknown :-1 s=CLOSED pgs=2941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.048+0000 7f882985d740 1 --2- 172.21.15.63:0/747403845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0539d92f0 0x55b053dc29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.048+0000 7f882985d740 1 -- 172.21.15.63:0/747403845 >> 172.21.15.63:0/747403845 conn(0x55b0539f6300 msgr2=0x55b053dae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.049+0000 7f8805600640 0 client.23424 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.050+0000 7f882985d740 1 -- 172.21.15.63:0/747403845 shutdown_connections 2024-05-27T08:16:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.050+0000 7f882985d740 1 -- 172.21.15.63:0/747403845 wait complete. 2024-05-27T08:16:18.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.051+0000 7f882985d740 20 client.23424 trim_cache size 0 max 16384 2024-05-27T08:16:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:16:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:16:18.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:16:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.053+0000 7f8822800640 1 -- 172.21.15.63:0/3439525074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88000b2a20 msgr2=0x7f88000b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.053+0000 7f8822800640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88000b2a20 0x7f88000b4f30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f880c0e02b0 tx=0x7f880c0cc040 comp rx=0 tx=0).stop 2024-05-27T08:16:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.053+0000 7f8822800640 1 -- 172.21.15.63:0/3439525074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c10a1f0 msgr2=0x7f881c1a9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.053+0000 7f8822800640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c10a1f0 0x7f881c1a9590 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f880c00e9c0 tx=0x7f880c00ee80 comp rx=0 tx=0).stop 2024-05-27T08:16:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.053+0000 7f8822800640 1 -- 172.21.15.63:0/3439525074 shutdown_connections 2024-05-27T08:16:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.053+0000 7f8822800640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f88000b2a20 0x7f88000b4f30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.053+0000 7f8822800640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f881c11d240 0x7f881c1addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.054+0000 7f8822800640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f881c11a8e0 0x7f881c1a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.054+0000 7f8822800640 1 --2- 172.21.15.63:0/3439525074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c10a1f0 0x7f881c1a9590 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.054+0000 7f8822800640 1 -- 172.21.15.63:0/3439525074 >> 172.21.15.63:0/3439525074 conn(0x7f881c06e8f0 msgr2=0x7f881c11acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.055+0000 7f8822800640 1 -- 172.21.15.63:0/3439525074 shutdown_connections 2024-05-27T08:16:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:18.055+0000 7f8822800640 1 -- 172.21.15.63:0/3439525074 wait complete. 2024-05-27T08:16:19.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:18 smithi086 ceph-mon[34687]: pgmap v1357: 129 pgs: 12 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 85 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-27T08:16:19.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:18 smithi149 ceph-mon[35408]: pgmap v1357: 129 pgs: 12 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 85 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-27T08:16:19.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:18 smithi063 ceph-mon[31613]: pgmap v1357: 129 pgs: 12 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 85 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.57k op/s 2024-05-27T08:16:21.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:20 smithi086 ceph-mon[34687]: pgmap v1358: 129 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 93 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.9 MiB/s wr, 2.18k op/s 2024-05-27T08:16:21.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:20 smithi149 ceph-mon[35408]: pgmap v1358: 129 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 93 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.9 MiB/s wr, 2.18k op/s 2024-05-27T08:16:21.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:20 smithi063 ceph-mon[31613]: pgmap v1358: 129 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 93 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.9 MiB/s wr, 2.18k op/s 2024-05-27T08:16:23.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:22 smithi086 ceph-mon[34687]: pgmap v1359: 129 pgs: 10 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 100 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-05-27T08:16:23.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:22 smithi149 ceph-mon[35408]: pgmap v1359: 129 pgs: 10 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 100 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-05-27T08:16:23.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:22 smithi063 ceph-mon[31613]: pgmap v1359: 129 pgs: 10 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 100 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-05-27T08:16:25.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:24 smithi086 ceph-mon[34687]: pgmap v1360: 129 pgs: 10 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 101 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.5 MiB/s wr, 2.35k op/s 2024-05-27T08:16:25.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:24 smithi149 ceph-mon[35408]: pgmap v1360: 129 pgs: 10 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 101 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.5 MiB/s wr, 2.35k op/s 2024-05-27T08:16:25.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:24 smithi063 ceph-mon[31613]: pgmap v1360: 129 pgs: 10 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 101 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.5 MiB/s wr, 2.35k op/s 2024-05-27T08:16:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.335+0000 7f548cc00700 1 -- 172.21.15.63:0/991928198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5488105280 msgr2=0x7f5488105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.335+0000 7f548cc00700 1 --2- 172.21.15.63:0/991928198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5488105280 0x7f5488105690 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7f5478009990 tx=0x7f5478009ca0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.336+0000 7f548cc00700 1 -- 172.21.15.63:0/991928198 shutdown_connections 2024-05-27T08:16:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.336+0000 7f548cc00700 1 --2- 172.21.15.63:0/991928198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f548810e6f0 0x7f5488110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.336+0000 7f548cc00700 1 --2- 172.21.15.63:0/991928198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5488105280 0x7f5488105690 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.336+0000 7f548cc00700 1 --2- 172.21.15.63:0/991928198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488105bd0 0x7f548810e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.336+0000 7f548cc00700 1 -- 172.21.15.63:0/991928198 >> 172.21.15.63:0/991928198 conn(0x7f5488078af0 msgr2=0x7f5488078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.336+0000 7f548cc00700 1 -- 172.21.15.63:0/991928198 shutdown_connections 2024-05-27T08:16:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.336+0000 7f548cc00700 1 -- 172.21.15.63:0/991928198 wait complete. 2024-05-27T08:16:26.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.337+0000 7f548cc00700 1 Processor -- start 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.337+0000 7f548cc00700 1 -- start start 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.337+0000 7f548cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488105280 0x7f54881a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.337+0000 7f548cc00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5488105bd0 0x7f54881a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.337+0000 7f548cc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f548810e6f0 0x7f548819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.337+0000 7f548cc00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5488114140 con 0x7f5488105bd0 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.337+0000 7f548cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5488113fc0 con 0x7f5488105280 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.337+0000 7f548cc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f54881142c0 con 0x7f548810e6f0 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.338+0000 7f5486a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5488105bd0 0x7f54881a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.338+0000 7f5487400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488105280 0x7f54881a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.338+0000 7f5486a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5488105bd0 0x7f54881a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51544/0 (socket says 172.21.15.63:51544) 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.338+0000 7f5486a00700 1 -- 172.21.15.63:0/140991341 learned_addr learned my addr 172.21.15.63:0/140991341 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:26.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.338+0000 7f5487e00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f548810e6f0 0x7f548819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.338+0000 7f5486a00700 1 -- 172.21.15.63:0/140991341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f548810e6f0 msgr2=0x7f548819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.338+0000 7f5486a00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f548810e6f0 0x7f548819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.338+0000 7f5486a00700 1 -- 172.21.15.63:0/140991341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488105280 msgr2=0x7f54881a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.338+0000 7f5486a00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488105280 0x7f54881a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.338+0000 7f5486a00700 1 -- 172.21.15.63:0/140991341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548819b030 con 0x7f5488105bd0 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.339+0000 7f5486a00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5488105bd0 0x7f54881a0ca0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f547400d950 tx=0x7f547400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.339+0000 7f547fe00700 1 -- 172.21.15.63:0/140991341 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474016070 con 0x7f5488105bd0 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.339+0000 7f548cc00700 1 -- 172.21.15.63:0/140991341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548819b320 con 0x7f5488105bd0 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.339+0000 7f547fe00700 1 -- 172.21.15.63:0/140991341 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5474009dd0 con 0x7f5488105bd0 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.339+0000 7f547fe00700 1 -- 172.21.15.63:0/140991341 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547400f3c0 con 0x7f5488105bd0 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.339+0000 7f548cc00700 1 -- 172.21.15.63:0/140991341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54881a75f0 con 0x7f5488105bd0 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.344+0000 7f547fe00700 1 -- 172.21.15.63:0/140991341 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5474019070 con 0x7f5488105bd0 2024-05-27T08:16:26.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.345+0000 7f547fe00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f546c0b29f0 0x7f546c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.345+0000 7f547fe00700 1 -- 172.21.15.63:0/140991341 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 9571+0+0 (secure 0 0 0) 0x7f54740e2720 con 0x7f5488105bd0 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.345+0000 7f5487400700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f546c0b29f0 0x7f546c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:23.346+0000 7f5487400700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f546c0b29f0 0x7f546c0b4f00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f548819bac0 tx=0x7f547800b540 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.090+0000 7f549cd00b80 1 Processor -- start 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.094+0000 7f549cd00b80 1 -- start start 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.094+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 0x562f39021280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.094+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39012870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.094+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 0x562f390131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.095+0000 7f549cd00b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562f38f40e40 con 0x562f390217c0 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.095+0000 7f549cd00b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562f38f309e0 con 0x562f39012db0 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.095+0000 7f549cd00b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562f38f34a20 con 0x562f38e93e40 2024-05-27T08:16:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.095+0000 7f5487e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 0x562f390131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.095+0000 7f5487e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 0x562f390131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:53554/0 (socket says 172.21.15.63:53554) 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.095+0000 7f5487e00700 1 -- 172.21.15.63:0/2713611831 learned_addr learned my addr 172.21.15.63:0/2713611831 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.096+0000 7f5487400700 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 0x562f39021280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.096+0000 7f5486a00700 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39012870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.097+0000 7f5487400700 1 -- 172.21.15.63:0/2713611831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 msgr2=0x562f390131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.099+0000 7f5487400700 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 0x562f390131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.099+0000 7f5487400700 1 -- 172.21.15.63:0/2713611831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 msgr2=0x562f39012870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.099+0000 7f5487400700 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39012870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.099+0000 7f5487400700 1 -- 172.21.15.63:0/2713611831 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f38ef9420 con 0x562f38e93e40 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.099+0000 7f5486a00700 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39012870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.099+0000 7f5487e00700 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 0x562f390131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:16:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.100+0000 7f5487400700 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 0x562f39021280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5478006010 tx=0x7f547801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a69a48a31d62f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.100+0000 7f547cc00700 1 -- 172.21.15.63:0/2713611831 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478005680 con 0x562f38e93e40 2024-05-27T08:16:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.101+0000 7f547cc00700 1 -- 172.21.15.63:0/2713611831 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54780037c0 con 0x562f38e93e40 2024-05-27T08:16:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.101+0000 7f547cc00700 1 -- 172.21.15.63:0/2713611831 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547800f850 con 0x562f38e93e40 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.102+0000 7f549cd00b80 1 -- 172.21.15.63:0/2713611831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 msgr2=0x562f39021280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.102+0000 7f549cd00b80 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 0x562f39021280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5478006010 tx=0x7f547801e000 comp rx=0 tx=0).stop 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.103+0000 7f549cd00b80 1 -- 172.21.15.63:0/2713611831 shutdown_connections 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.103+0000 7f549cd00b80 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 0x562f390131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.104+0000 7f549cd00b80 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 0x562f39021280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.104+0000 7f549cd00b80 1 --2- 172.21.15.63:0/2713611831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39012870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.104+0000 7f549cd00b80 1 -- 172.21.15.63:0/2713611831 >> 172.21.15.63:0/2713611831 conn(0x562f38ef3fc0 msgr2=0x562f3900db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.106+0000 7f549cd00b80 1 -- 172.21.15.63:0/2713611831 shutdown_connections 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.106+0000 7f549cd00b80 1 -- 172.21.15.63:0/2713611831 wait complete. 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.109+0000 7f549cd00b80 1 -- start start 2024-05-27T08:16:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.111+0000 7f549cd00b80 1 Processor -- start 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.115+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39025030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.115+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 0x562f39025570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.115+0000 7f5487400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39025030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.115+0000 7f5487400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39025030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:53560/0 (socket says 172.21.15.63:53560) 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.117+0000 7f5487400700 1 -- 172.21.15.63:0/3309120313 learned_addr learned my addr 172.21.15.63:0/3309120313 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.117+0000 7f549cd00b80 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f390217c0 0x562f39025ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.117+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562f38e36fa0 con 0x562f39012db0 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.117+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562f38ed1760 con 0x562f38e93e40 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.117+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562f39028210 con 0x562f390217c0 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.117+0000 7f549cd00b80 10 client.0 _ll_register_callbacks cb 0x562f38edac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.119+0000 7f5487e00700 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f390217c0 0x562f39025ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.119+0000 7f5486a00700 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 0x562f39025570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.120+0000 7f5487400700 1 -- 172.21.15.63:0/3309120313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f390217c0 msgr2=0x562f39025ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.120+0000 7f5487400700 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f390217c0 0x562f39025ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.120+0000 7f5487400700 1 -- 172.21.15.63:0/3309120313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 msgr2=0x562f39025570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.120+0000 7f5487400700 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 0x562f39025570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.120+0000 7f5487400700 1 -- 172.21.15.63:0/3309120313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f38e808d0 con 0x562f38e93e40 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.123+0000 7f5471600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.124+0000 7f5487400700 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39025030 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5478000c00 tx=0x7f547800fe70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1904be12a8cd55d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.124+0000 7f5471600700 1 -- 172.21.15.63:0/3309120313 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547803b060 con 0x562f38e93e40 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.124+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54780037c0 con 0x562f38e93e40 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.126+0000 7f5471600700 1 -- 172.21.15.63:0/3309120313 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(165..165 src has 1..165) ==== 9571+0+0 (secure 0 0 0) 0x7f547802b050 con 0x562f38e93e40 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.126+0000 7f5471600700 1 -- 172.21.15.63:0/3309120313 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f547803b060 con 0x562f38e93e40 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.126+0000 7f549cd00b80 10 client.42308 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.127+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562f38f01060 con 0x562f38e93e40 2024-05-27T08:16:26.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.127+0000 7f5471600700 1 -- 172.21.15.63:0/3309120313 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54780037c0 con 0x562f38e93e40 2024-05-27T08:16:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.127+0000 7f5471600700 1 -- 172.21.15.63:0/3309120313 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5478022070 con 0x562f38e93e40 2024-05-27T08:16:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.128+0000 7f549cd00b80 10 client.42308 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.128+0000 7f549cd00b80 10 client.42308 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:16:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.128+0000 7f549cd00b80 20 client.42308 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.128+0000 7f549cd00b80 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x562f39035060 0x562f39037480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.128+0000 7f549cd00b80 4 client.42308 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:16:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.128+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f38f42d90 con 0x562f39035060 2024-05-27T08:16:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.129+0000 7f5486a00700 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x562f39035060 0x562f39037480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.130+0000 7f5486a00700 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x562f39035060 0x562f39037480 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e970f9624a48fdea server_cookie=fae7fe2bbe23d039 in_seq=0 out_seq=0 2024-05-27T08:16:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.130+0000 7f5471600700 10 client.42308 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.131+0000 7f5471600700 1 -- 172.21.15.63:0/3309120313 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562f38f42d90 con 0x562f39035060 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.131+0000 7f5471600700 10 client.42308 handle_command_reply: tid=0 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.131+0000 7f549cd00b80 1 client.42308 shutdown 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.132+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 msgr2=0x562f39025030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.132+0000 7f549cd00b80 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39025030 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5478000c00 tx=0x7f547800fe70 comp rx=0 tx=0).stop 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.132+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 shutdown_connections 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.132+0000 7f549cd00b80 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f390217c0 0x562f39025ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.132+0000 7f549cd00b80 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39025030 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.132+0000 7f549cd00b80 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x562f39035060 0x562f39037480 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.132+0000 7f549cd00b80 1 --2- 172.21.15.63:0/3309120313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 0x562f39025570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.132+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 >> 172.21.15.63:0/3309120313 conn(0x562f38ef3fc0 msgr2=0x562f3900cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.133+0000 7f5471600700 0 client.42308 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:16:26.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.133+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 shutdown_connections 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.133+0000 7f549cd00b80 1 -- 172.21.15.63:0/3309120313 wait complete. 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.134+0000 7f549cd00b80 20 client.42308 trim_cache size 0 max 16384 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.163+0000 7f549cd00b80 1 Processor -- start 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.163+0000 7f549cd00b80 1 -- start start 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.163+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39012870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.163+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 0x562f3901f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.163+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f390217c0 0x562f390274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.163+0000 7f549cd00b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f54881b0bc0 con 0x562f39012db0 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.163+0000 7f549cd00b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54881ae450 con 0x562f38e93e40 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.163+0000 7f549cd00b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f54881ac2c0 con 0x562f390217c0 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5486a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 0x562f3901f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5487400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39012870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5487400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39012870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:53572/0 (socket says 172.21.15.63:53572) 2024-05-27T08:16:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5487400700 1 -- 172.21.15.63:0/342233364 learned_addr learned my addr 172.21.15.63:0/342233364 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5486a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 0x562f3901f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51578/0 (socket says 172.21.15.63:51578) 2024-05-27T08:16:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5487e00700 1 --2- 172.21.15.63:0/342233364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f390217c0 0x562f390274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5486a00700 1 -- 172.21.15.63:0/342233364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f390217c0 msgr2=0x562f390274d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5486a00700 1 --2- 172.21.15.63:0/342233364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f390217c0 0x562f390274d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5486a00700 1 -- 172.21.15.63:0/342233364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 msgr2=0x562f39012870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5486a00700 1 --2- 172.21.15.63:0/342233364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39012870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.164+0000 7f5486a00700 1 -- 172.21.15.63:0/342233364 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f39020c00 con 0x562f39012db0 2024-05-27T08:16:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.165+0000 7f5486a00700 1 --2- 172.21.15.63:0/342233364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 0x562f3901f7b0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f54740df9f0 tx=0x7f5474000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322f32b6a898c307 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.165+0000 7f5473e00700 1 -- 172.21.15.63:0/342233364 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547406d070 con 0x562f39012db0 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.165+0000 7f5473e00700 1 -- 172.21.15.63:0/342233364 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5474010460 con 0x562f39012db0 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.165+0000 7f5473e00700 1 -- 172.21.15.63:0/342233364 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54740c3070 con 0x562f39012db0 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.166+0000 7f549cd00b80 1 -- 172.21.15.63:0/342233364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 msgr2=0x562f3901f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.166+0000 7f549cd00b80 1 --2- 172.21.15.63:0/342233364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 0x562f3901f7b0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f54740df9f0 tx=0x7f5474000bc0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.166+0000 7f549cd00b80 1 -- 172.21.15.63:0/342233364 shutdown_connections 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.166+0000 7f549cd00b80 1 --2- 172.21.15.63:0/342233364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f390217c0 0x562f390274d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.166+0000 7f549cd00b80 1 --2- 172.21.15.63:0/342233364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f38e93e40 0x562f39012870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.166+0000 7f549cd00b80 1 --2- 172.21.15.63:0/342233364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f39012db0 0x562f3901f7b0 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.166+0000 7f549cd00b80 1 -- 172.21.15.63:0/342233364 >> 172.21.15.63:0/342233364 conn(0x562f38ef3fc0 msgr2=0x562f39024e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.166+0000 7f549cd00b80 1 -- 172.21.15.63:0/342233364 shutdown_connections 2024-05-27T08:16:26.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.166+0000 7f549cd00b80 1 -- 172.21.15.63:0/342233364 wait complete. 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.167+0000 7f549cd00b80 1 -- start start 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.167+0000 7f549cd00b80 1 Processor -- start 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 0x562f39037070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 0x562f390375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f549cd00b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39037af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f549cd00b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f54881ac910 con 0x562f390217c0 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f549cd00b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54881add80 con 0x562f39012db0 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f549cd00b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f54881acfe0 con 0x562f38e93e40 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f549cd00b80 10 client.0 _ll_register_callbacks cb 0x562f38edac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f5487e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39037af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f5487e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39037af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51590/0 (socket says 172.21.15.63:51590) 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f5487e00700 1 -- 172.21.15.63:0/2012257158 learned_addr learned my addr 172.21.15.63:0/2012257158 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:26.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f5487400700 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 0x562f39037070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f5486a00700 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 0x562f390375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f5487e00700 1 -- 172.21.15.63:0/2012257158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 msgr2=0x562f39037070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f5487e00700 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 0x562f39037070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f5487e00700 1 -- 172.21.15.63:0/2012257158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 msgr2=0x562f390375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f5487e00700 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 0x562f390375b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.168+0000 7f5487e00700 1 -- 172.21.15.63:0/2012257158 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3909d8c0 con 0x562f390217c0 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.169+0000 7f5487400700 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 0x562f39037070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.169+0000 7f5487e00700 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39037af0 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f5480007cb0 tx=0x7f5480007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8222f54381cb5fe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.169+0000 7f5471600700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.169+0000 7f5471600700 1 -- 172.21.15.63:0/2012257158 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5480004db0 con 0x562f390217c0 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.169+0000 7f5471600700 1 -- 172.21.15.63:0/2012257158 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(165..165 src has 1..165) ==== 9571+0+0 (secure 0 0 0) 0x7f5480017820 con 0x562f390217c0 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.170+0000 7f549cd00b80 1 -- 172.21.15.63:0/2012257158 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5474010460 con 0x562f390217c0 2024-05-27T08:16:26.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.170+0000 7f5471600700 1 -- 172.21.15.63:0/2012257158 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5480017a20 con 0x562f390217c0 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.170+0000 7f549cd00b80 10 client.23442 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.170+0000 7f549cd00b80 1 -- 172.21.15.63:0/2012257158 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562f3909eb70 con 0x562f390217c0 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.170+0000 7f5471600700 1 -- 172.21.15.63:0/2012257158 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5474010460 con 0x562f390217c0 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.171+0000 7f5471600700 1 -- 172.21.15.63:0/2012257158 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5480021cb0 con 0x562f390217c0 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.171+0000 7f549cd00b80 10 client.23442 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.171+0000 7f549cd00b80 10 client.23442 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.171+0000 7f549cd00b80 20 client.23442 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.171+0000 7f549cd00b80 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x562f3909f290 0x562f390a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.171+0000 7f549cd00b80 4 client.23442 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.171+0000 7f549cd00b80 1 -- 172.21.15.63:0/2012257158 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f38f3c570 con 0x562f3909f290 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.171+0000 7f5487400700 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x562f3909f290 0x562f390a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f5487400700 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x562f3909f290 0x562f390a16b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=139b5fc4aceb9210 server_cookie=d3aaab2d84f12d42 in_seq=0 out_seq=0 2024-05-27T08:16:26.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f5471600700 10 client.23442 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f5471600700 1 -- 172.21.15.63:0/2012257158 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f38f3c570 con 0x562f3909f290 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f5471600700 10 client.23442 handle_command_reply: tid=0 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 client.23442 shutdown 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 -- 172.21.15.63:0/2012257158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 msgr2=0x562f39037af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39037af0 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f5480007cb0 tx=0x7f5480007ce0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 -- 172.21.15.63:0/2012257158 shutdown_connections 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x562f3909f290 0x562f390a16b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562f390217c0 0x562f39037af0 unknown :-1 s=CLOSED pgs=2945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f5471600700 0 client.23442 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f38e93e40 0x562f39037070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 --2- 172.21.15.63:0/2012257158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562f39012db0 0x562f390375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 -- 172.21.15.63:0/2012257158 >> 172.21.15.63:0/2012257158 conn(0x562f38ef3fc0 msgr2=0x562f3900da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:26.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 -- 172.21.15.63:0/2012257158 shutdown_connections 2024-05-27T08:16:26.451 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:16:26.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.172+0000 7f549cd00b80 1 -- 172.21.15.63:0/2012257158 wait complete. 2024-05-27T08:16:26.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.173+0000 7f549cd00b80 20 client.23442 trim_cache size 0 max 16384 2024-05-27T08:16:26.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.173+0000 7f548cc00700 1 -- 172.21.15.63:0/140991341 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f546c0b29f0 msgr2=0x7f546c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:26.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.173+0000 7f548cc00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f546c0b29f0 0x7f546c0b4f00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f548819bac0 tx=0x7f547800b540 comp rx=0 tx=0).stop 2024-05-27T08:16:26.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.173+0000 7f548cc00700 1 -- 172.21.15.63:0/140991341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5488105bd0 msgr2=0x7f54881a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.173+0000 7f548cc00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5488105bd0 0x7f54881a0ca0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f547400d950 tx=0x7f547400dc60 comp rx=0 tx=0).stop 2024-05-27T08:16:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.174+0000 7f548cc00700 1 -- 172.21.15.63:0/140991341 shutdown_connections 2024-05-27T08:16:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.174+0000 7f548cc00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f546c0b29f0 0x7f546c0b4f00 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.174+0000 7f548cc00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f548810e6f0 0x7f548819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.174+0000 7f548cc00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5488105280 0x7f54881a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.174+0000 7f548cc00700 1 --2- 172.21.15.63:0/140991341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5488105bd0 0x7f54881a0ca0 unknown :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.174+0000 7f548cc00700 1 -- 172.21.15.63:0/140991341 >> 172.21.15.63:0/140991341 conn(0x7f5488078af0 msgr2=0x7f54880fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.174+0000 7f548cc00700 1 -- 172.21.15.63:0/140991341 shutdown_connections 2024-05-27T08:16:26.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:24.174+0000 7f548cc00700 1 -- 172.21.15.63:0/140991341 wait complete. 2024-05-27T08:16:27.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:26 smithi086 ceph-mon[34687]: pgmap v1361: 129 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 110 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-05-27T08:16:27.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:26 smithi149 ceph-mon[35408]: pgmap v1361: 129 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 110 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-05-27T08:16:27.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:26 smithi063 ceph-mon[31613]: pgmap v1361: 129 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 110 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-05-27T08:16:27.495 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:16:29.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:28 smithi086 ceph-mon[34687]: pgmap v1362: 129 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 110 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:16:29.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:28 smithi149 ceph-mon[35408]: pgmap v1362: 129 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 110 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:16:29.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:28 smithi063 ceph-mon[31613]: pgmap v1362: 129 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 110 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:16:29.445 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:16:31.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:30 smithi086 ceph-mon[34687]: pgmap v1363: 129 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 110 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.9 MiB/s wr, 2.50k op/s 2024-05-27T08:16:31.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:30 smithi149 ceph-mon[35408]: pgmap v1363: 129 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 110 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.9 MiB/s wr, 2.50k op/s 2024-05-27T08:16:31.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:30 smithi063 ceph-mon[31613]: pgmap v1363: 129 pgs: 8 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 110 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.9 MiB/s wr, 2.50k op/s 2024-05-27T08:16:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.396+0000 7f46b8c00640 1 -- 172.21.15.63:0/386333449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46b410bcd0 msgr2=0x7f46b4105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.396+0000 7f46b8c00640 1 --2- 172.21.15.63:0/386333449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46b410bcd0 0x7f46b4105870 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009900 tx=0x7f46a402a410 comp rx=0 tx=0).stop 2024-05-27T08:16:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.397+0000 7f46b8c00640 1 -- 172.21.15.63:0/386333449 shutdown_connections 2024-05-27T08:16:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.397+0000 7f46b8c00640 1 --2- 172.21.15.63:0/386333449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b4106420 0x7f46b41127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.397+0000 7f46b8c00640 1 --2- 172.21.15.63:0/386333449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46b410c170 0x7f46b4105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.397+0000 7f46b8c00640 1 --2- 172.21.15.63:0/386333449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46b410bcd0 0x7f46b4105870 unknown :-1 s=CLOSED pgs=2946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.397+0000 7f46b8c00640 1 -- 172.21.15.63:0/386333449 >> 172.21.15.63:0/386333449 conn(0x7f46b41013d0 msgr2=0x7f46b4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.397+0000 7f46b8c00640 1 -- 172.21.15.63:0/386333449 shutdown_connections 2024-05-27T08:16:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.397+0000 7f46b8c00640 1 -- 172.21.15.63:0/386333449 wait complete. 2024-05-27T08:16:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.398+0000 7f46b8c00640 1 Processor -- start 2024-05-27T08:16:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.398+0000 7f46b8c00640 1 -- start start 2024-05-27T08:16:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.399+0000 7f46b8c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46b4106420 0x7f46b410eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.399+0000 7f46b8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b410bcd0 0x7f46b410f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.399+0000 7f46b8c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46b410c170 0x7f46b41ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.399+0000 7f46b8c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4077bc0 con 0x7f46b4106420 2024-05-27T08:16:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.399+0000 7f46b8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4077a40 con 0x7f46b410bcd0 2024-05-27T08:16:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.399+0000 7f46b8c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4077d40 con 0x7f46b410c170 2024-05-27T08:16:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.399+0000 7f46b3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46b4106420 0x7f46b410eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.399+0000 7f46b3400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46b4106420 0x7f46b410eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34420/0 (socket says 172.21.15.63:34420) 2024-05-27T08:16:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.399+0000 7f46b3400640 1 -- 172.21.15.63:0/823547638 learned_addr learned my addr 172.21.15.63:0/823547638 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.399+0000 7f46b3e00640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46b410c170 0x7f46b41ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.400+0000 7f46b3400640 1 -- 172.21.15.63:0/823547638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46b410c170 msgr2=0x7f46b41ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.400+0000 7f46b3400640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46b410c170 0x7f46b41ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.400+0000 7f46b3400640 1 -- 172.21.15.63:0/823547638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b410bcd0 msgr2=0x7f46b410f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.400+0000 7f46b3400640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b410bcd0 0x7f46b410f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.400+0000 7f46b3400640 1 -- 172.21.15.63:0/823547638 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b41acdd0 con 0x7f46b4106420 2024-05-27T08:16:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.401+0000 7f46b3400640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46b4106420 0x7f46b410eb80 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f46a402a980 tx=0x7f46a4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.401+0000 7f46abe00640 1 -- 172.21.15.63:0/823547638 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a403f070 con 0x7f46b4106420 2024-05-27T08:16:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.401+0000 7f46abe00640 1 -- 172.21.15.63:0/823547638 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46a403bd00 con 0x7f46b4106420 2024-05-27T08:16:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.401+0000 7f46abe00640 1 -- 172.21.15.63:0/823547638 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4035030 con 0x7f46b4106420 2024-05-27T08:16:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.401+0000 7f46b8c00640 1 -- 172.21.15.63:0/823547638 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b41ad060 con 0x7f46b4106420 2024-05-27T08:16:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.402+0000 7f46b8c00640 1 -- 172.21.15.63:0/823547638 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b41ad4d0 con 0x7f46b4106420 2024-05-27T08:16:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.406+0000 7f46c11e8740 1 Processor -- start 2024-05-27T08:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.406+0000 7f46c11e8740 1 -- start start 2024-05-27T08:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.407+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe837a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.407+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe837fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.407+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 0x5564fe8384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.407+0000 7f46c11e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5564fe7a8b90 con 0x5564fe4951b0 2024-05-27T08:16:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.407+0000 7f46c11e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564fe798730 con 0x5564fe46c2f0 2024-05-27T08:16:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.407+0000 7f46c11e8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5564fe79c770 con 0x5564fe4744d0 2024-05-27T08:16:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.407+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe837a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.408+0000 7f46abe00640 1 -- 172.21.15.63:0/823547638 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f46a4042070 con 0x7f46b4106420 2024-05-27T08:16:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.408+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe837a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34426/0 (socket says 172.21.15.63:34426) 2024-05-27T08:16:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.408+0000 7f46b3e00640 1 -- 172.21.15.63:0/2039792989 learned_addr learned my addr 172.21.15.63:0/2039792989 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.408+0000 7f46b2a00640 1 --2- 172.21.15.63:0/2039792989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 0x5564fe8384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.408+0000 7f46b3e00640 1 -- 172.21.15.63:0/2039792989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 msgr2=0x5564fe8384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.408+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2039792989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 0x5564fe8384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.408+0000 7f46b3e00640 1 -- 172.21.15.63:0/2039792989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 msgr2=0x5564fe837fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.408+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2039792989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe837fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.408+0000 7f46b3e00640 1 -- 172.21.15.63:0/2039792989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564fe7318b0 con 0x5564fe4951b0 2024-05-27T08:16:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.409+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2039792989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe837a60 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0039f0 tx=0x7f46ac003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f52afc6ddda064e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.409+0000 7f469fe00640 1 -- 172.21.15.63:0/2039792989 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac01a470 con 0x5564fe4951b0 2024-05-27T08:16:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.410+0000 7f469fe00640 1 -- 172.21.15.63:0/2039792989 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46ac015070 con 0x5564fe4951b0 2024-05-27T08:16:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.410+0000 7f469fe00640 1 -- 172.21.15.63:0/2039792989 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0107e0 con 0x5564fe4951b0 2024-05-27T08:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.410+0000 7f46abe00640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46980b2a20 0x7f46980b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.410+0000 7f46abe00640 1 -- 172.21.15.63:0/823547638 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 9571+0+0 (secure 0 0 0) 0x7f46a4103260 con 0x7f46b4106420 2024-05-27T08:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.410+0000 7f46b3400640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46980b2a20 0x7f46980b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.411+0000 7f46c11e8740 1 -- 172.21.15.63:0/2039792989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 msgr2=0x5564fe837a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.411+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2039792989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe837a60 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0039f0 tx=0x7f46ac003eb0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.411+0000 7f46c11e8740 1 -- 172.21.15.63:0/2039792989 shutdown_connections 2024-05-27T08:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.411+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2039792989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 0x5564fe8384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.411+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2039792989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe837fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.411+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2039792989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe837a60 unknown :-1 s=CLOSED pgs=2948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.411+0000 7f46c11e8740 1 -- 172.21.15.63:0/2039792989 >> 172.21.15.63:0/2039792989 conn(0x5564fe489300 msgr2=0x5564fe834210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.411+0000 7f46b3400640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46980b2a20 0x7f46980b4f30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x5564fe835550 tx=0x7f46a40095f0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.412+0000 7f46c11e8740 1 -- 172.21.15.63:0/2039792989 shutdown_connections 2024-05-27T08:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.412+0000 7f46c11e8740 1 -- 172.21.15.63:0/2039792989 wait complete. 2024-05-27T08:16:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.412+0000 7f46c11e8740 1 -- start start 2024-05-27T08:16:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.413+0000 7f46c11e8740 1 Processor -- start 2024-05-27T08:16:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.413+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe841bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.413+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 0x5564fe842100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.413+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe84dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.414+0000 7f46c11e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5564fe47e790 con 0x5564fe4951b0 2024-05-27T08:16:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.414+0000 7f46c11e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564fe740bc0 con 0x5564fe46c2f0 2024-05-27T08:16:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.414+0000 7f46c11e8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5564fe844b90 con 0x5564fe4744d0 2024-05-27T08:16:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.414+0000 7f46c11e8740 10 client.0 _ll_register_callbacks cb 0x5564fe71e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.414+0000 7f46b2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 0x5564fe842100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.414+0000 7f46b2a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 0x5564fe842100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:52918/0 (socket says 172.21.15.63:52918) 2024-05-27T08:16:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.414+0000 7f46b2a00640 1 -- 172.21.15.63:0/2428552610 learned_addr learned my addr 172.21.15.63:0/2428552610 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.414+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe84dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.415+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe841bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.415+0000 7f46b2a00640 1 -- 172.21.15.63:0/2428552610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 msgr2=0x5564fe841bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.415+0000 7f46b2a00640 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe841bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.415+0000 7f46b2a00640 1 -- 172.21.15.63:0/2428552610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 msgr2=0x5564fe84dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.415+0000 7f46b2a00640 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe84dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.415+0000 7f46b2a00640 1 -- 172.21.15.63:0/2428552610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564fe4ad210 con 0x5564fe4744d0 2024-05-27T08:16:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.415+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe84dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:16:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.416+0000 7f46b2a00640 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 0x5564fe842100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f46a000dd80 tx=0x7f46a000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc996a85a5de7d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.416+0000 7f469cc00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:16:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.416+0000 7f469cc00640 1 -- 172.21.15.63:0/2428552610 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0011650 con 0x5564fe4744d0 2024-05-27T08:16:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.416+0000 7f469cc00640 1 -- 172.21.15.63:0/2428552610 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(165..165 src has 1..165) ==== 9571+0+0 (secure 0 0 0) 0x7f46a00117f0 con 0x5564fe4744d0 2024-05-27T08:16:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.416+0000 7f46c11e8740 1 -- 172.21.15.63:0/2428552610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ac015070 con 0x5564fe4744d0 2024-05-27T08:16:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.417+0000 7f469cc00640 1 -- 172.21.15.63:0/2428552610 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f46a0023050 con 0x5564fe4744d0 2024-05-27T08:16:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.417+0000 7f46c11e8740 10 client.32140 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.417+0000 7f46c11e8740 1 -- 172.21.15.63:0/2428552610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5564fe579620 con 0x5564fe4744d0 2024-05-27T08:16:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.417+0000 7f469cc00640 1 -- 172.21.15.63:0/2428552610 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46ac015070 con 0x5564fe4744d0 2024-05-27T08:16:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.418+0000 7f469cc00640 1 -- 172.21.15.63:0/2428552610 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f46a001bb50 con 0x5564fe4744d0 2024-05-27T08:16:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.418+0000 7f46c11e8740 10 client.32140 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.418+0000 7f46c11e8740 10 client.32140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:16:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.418+0000 7f46c11e8740 20 client.32140 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.418+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564fe453d80 0x5564fe853ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.418+0000 7f46c11e8740 4 client.32140 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:16:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.418+0000 7f46c11e8740 1 -- 172.21.15.63:0/2428552610 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564fe7aaae0 con 0x5564fe453d80 2024-05-27T08:16:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.418+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564fe453d80 0x5564fe853ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.420+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564fe453d80 0x5564fe853ee0 crc :-1 s=READY pgs=748 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f185e4db584d16 server_cookie=3d2fb6d89a5ca7ca in_seq=0 out_seq=0 2024-05-27T08:16:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.420+0000 7f469cc00640 10 client.32140 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f469cc00640 1 -- 172.21.15.63:0/2428552610 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5564fe7aaae0 con 0x5564fe453d80 2024-05-27T08:16:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f469cc00640 10 client.32140 handle_command_reply: tid=0 2024-05-27T08:16:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f46c11e8740 1 client.32140 shutdown 2024-05-27T08:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f46c11e8740 1 -- 172.21.15.63:0/2428552610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 msgr2=0x5564fe842100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 0x5564fe842100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f46a000dd80 tx=0x7f46a000f730 comp rx=0 tx=0).stop 2024-05-27T08:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f46c11e8740 1 -- 172.21.15.63:0/2428552610 shutdown_connections 2024-05-27T08:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564fe453d80 0x5564fe853ee0 crc :-1 s=READY pgs=748 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe84dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4744d0 0x5564fe842100 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2428552610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe841bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f46c11e8740 1 -- 172.21.15.63:0/2428552610 >> 172.21.15.63:0/2428552610 conn(0x5564fe489300 msgr2=0x5564fe837830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.426+0000 7f469cc00640 0 client.32140 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.427+0000 7f46c11e8740 1 -- 172.21.15.63:0/2428552610 shutdown_connections 2024-05-27T08:16:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.427+0000 7f46c11e8740 1 -- 172.21.15.63:0/2428552610 wait complete. 2024-05-27T08:16:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.427+0000 7f46c11e8740 20 client.32140 trim_cache size 0 max 16384 2024-05-27T08:16:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.455+0000 7f46c11e8740 1 Processor -- start 2024-05-27T08:16:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.455+0000 7f46c11e8740 1 -- start start 2024-05-27T08:16:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.456+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe46c2f0 0x5564fe83d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.456+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe4744d0 0x5564fe833240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.456+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe8442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.456+0000 7f46c11e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41ba0a0 con 0x5564fe4951b0 2024-05-27T08:16:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.456+0000 7f46c11e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41b7970 con 0x5564fe4744d0 2024-05-27T08:16:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.456+0000 7f46c11e8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41b57e0 con 0x5564fe46c2f0 2024-05-27T08:16:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.456+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe8442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.456+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe8442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34432/0 (socket says 172.21.15.63:34432) 2024-05-27T08:16:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.456+0000 7f46b3e00640 1 -- 172.21.15.63:0/3941552114 learned_addr learned my addr 172.21.15.63:0/3941552114 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.456+0000 7f46b3e00640 1 --2- 172.21.15.63:0/3941552114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe46c2f0 0x5564fe83d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.457+0000 7f46b3e00640 1 -- 172.21.15.63:0/3941552114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe46c2f0 msgr2=0x5564fe83d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.457+0000 7f46b3e00640 1 --2- 172.21.15.63:0/3941552114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe46c2f0 0x5564fe83d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.457+0000 7f46b3e00640 1 -- 172.21.15.63:0/3941552114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe4744d0 msgr2=0x5564fe833240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.457+0000 7f46b3e00640 1 --2- 172.21.15.63:0/3941552114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe4744d0 0x5564fe833240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.457+0000 7f46b3e00640 1 -- 172.21.15.63:0/3941552114 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564fe838b60 con 0x5564fe4951b0 2024-05-27T08:16:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.458+0000 7f46b3e00640 1 --2- 172.21.15.63:0/3941552114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe8442f0 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009df0 tx=0x7f46ac010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9e11af34be5899 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.458+0000 7f469f400640 1 -- 172.21.15.63:0/3941552114 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac02e040 con 0x5564fe4951b0 2024-05-27T08:16:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.458+0000 7f469f400640 1 -- 172.21.15.63:0/3941552114 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46ac015070 con 0x5564fe4951b0 2024-05-27T08:16:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.458+0000 7f469f400640 1 -- 172.21.15.63:0/3941552114 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac02bae0 con 0x5564fe4951b0 2024-05-27T08:16:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.459+0000 7f46c11e8740 1 -- 172.21.15.63:0/3941552114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 msgr2=0x5564fe8442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.459+0000 7f46c11e8740 1 --2- 172.21.15.63:0/3941552114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe8442f0 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009df0 tx=0x7f46ac010440 comp rx=0 tx=0).stop 2024-05-27T08:16:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.460+0000 7f46c11e8740 1 -- 172.21.15.63:0/3941552114 shutdown_connections 2024-05-27T08:16:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.460+0000 7f46c11e8740 1 --2- 172.21.15.63:0/3941552114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4951b0 0x5564fe8442f0 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.460+0000 7f46c11e8740 1 --2- 172.21.15.63:0/3941552114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe4744d0 0x5564fe833240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.460+0000 7f46c11e8740 1 --2- 172.21.15.63:0/3941552114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe46c2f0 0x5564fe83d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.460+0000 7f46c11e8740 1 -- 172.21.15.63:0/3941552114 >> 172.21.15.63:0/3941552114 conn(0x5564fe489300 msgr2=0x5564fe8377f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.460+0000 7f46c11e8740 1 -- 172.21.15.63:0/3941552114 shutdown_connections 2024-05-27T08:16:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.460+0000 7f46c11e8740 1 -- 172.21.15.63:0/3941552114 wait complete. 2024-05-27T08:16:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.461+0000 7f46c11e8740 1 -- start start 2024-05-27T08:16:31.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.461+0000 7f46c11e8740 1 Processor -- start 2024-05-27T08:16:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.462+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe8b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.462+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4744d0 0x5564fe8b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.462+0000 7f46c11e8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4951b0 0x5564fe8b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.462+0000 7f46c11e8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41b5e30 con 0x5564fe4744d0 2024-05-27T08:16:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.462+0000 7f46c11e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41b72a0 con 0x5564fe46c2f0 2024-05-27T08:16:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.462+0000 7f46c11e8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41b6500 con 0x5564fe4951b0 2024-05-27T08:16:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.462+0000 7f46c11e8740 10 client.0 _ll_register_callbacks cb 0x7f46a000d320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.463+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe8b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.463+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe8b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51240/0 (socket says 172.21.15.63:51240) 2024-05-27T08:16:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.463+0000 7f46b3e00640 1 -- 172.21.15.63:0/2153156788 learned_addr learned my addr 172.21.15.63:0/2153156788 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.463+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4951b0 0x5564fe8b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.463+0000 7f46b2a00640 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4744d0 0x5564fe8b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.464+0000 7f46b3e00640 1 -- 172.21.15.63:0/2153156788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4951b0 msgr2=0x5564fe8b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.464+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4951b0 0x5564fe8b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.464+0000 7f46b3e00640 1 -- 172.21.15.63:0/2153156788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4744d0 msgr2=0x5564fe8b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.464+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4744d0 0x5564fe8b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.464+0000 7f46b3e00640 1 -- 172.21.15.63:0/2153156788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564fe8b4f60 con 0x5564fe46c2f0 2024-05-27T08:16:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.464+0000 7f46b2a00640 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4744d0 0x5564fe8b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:16:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.469+0000 7f46b3e00640 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe8b7bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0097c0 tx=0x7f46ac003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d3329ccbf56f0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.469+0000 7f469cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:16:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.473+0000 7f469cc00640 1 -- 172.21.15.63:0/2153156788 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac02e070 con 0x5564fe46c2f0 2024-05-27T08:16:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.473+0000 7f469cc00640 1 -- 172.21.15.63:0/2153156788 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(165..165 src has 1..165) ==== 9571+0+0 (secure 0 0 0) 0x7f46ac02b450 con 0x5564fe46c2f0 2024-05-27T08:16:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.474+0000 7f46c11e8740 1 -- 172.21.15.63:0/2153156788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ac015070 con 0x5564fe46c2f0 2024-05-27T08:16:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.476+0000 7f469cc00640 1 -- 172.21.15.63:0/2153156788 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f46ac02bc30 con 0x5564fe46c2f0 2024-05-27T08:16:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.476+0000 7f46c11e8740 10 client.42323 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.476+0000 7f46c11e8740 1 -- 172.21.15.63:0/2153156788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5564fe849a50 con 0x5564fe46c2f0 2024-05-27T08:16:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.477+0000 7f469cc00640 1 -- 172.21.15.63:0/2153156788 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46ac015070 con 0x5564fe46c2f0 2024-05-27T08:16:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.477+0000 7f469cc00640 1 -- 172.21.15.63:0/2153156788 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f46ac01a860 con 0x5564fe46c2f0 2024-05-27T08:16:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.478+0000 7f46c11e8740 10 client.42323 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.478+0000 7f46c11e8740 10 client.42323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:16:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.478+0000 7f46c11e8740 20 client.42323 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.478+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564fe4291c0 0x5564fe849fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.478+0000 7f46c11e8740 4 client.42323 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:16:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.478+0000 7f46c11e8740 1 -- 172.21.15.63:0/2153156788 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564fe7a42c0 con 0x5564fe4291c0 2024-05-27T08:16:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.481+0000 7f46b2a00640 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564fe4291c0 0x5564fe849fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.482+0000 7f46b2a00640 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564fe4291c0 0x5564fe849fd0 crc :-1 s=READY pgs=749 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53a84e754f0af19 server_cookie=d05e0cbe5bb890ca in_seq=0 out_seq=0 2024-05-27T08:16:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:31.482+0000 7f469cc00640 10 client.42323 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.460+0000 7f469cc00640 1 -- 172.21.15.63:0/2153156788 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5564fe7a42c0 con 0x5564fe4291c0 2024-05-27T08:16:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.460+0000 7f469cc00640 10 client.42323 handle_command_reply: tid=0 2024-05-27T08:16:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.460+0000 7f46c11e8740 1 client.42323 shutdown 2024-05-27T08:16:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.463+0000 7f46c11e8740 1 -- 172.21.15.63:0/2153156788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 msgr2=0x5564fe8b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.463+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe8b7bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0097c0 tx=0x7f46ac003eb0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.463+0000 7f46c11e8740 1 -- 172.21.15.63:0/2153156788 shutdown_connections 2024-05-27T08:16:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.463+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5564fe4291c0 0x5564fe849fd0 crc :-1 s=READY pgs=749 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.463+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5564fe4951b0 0x5564fe8b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.463+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5564fe4744d0 0x5564fe8b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.463+0000 7f46c11e8740 1 --2- 172.21.15.63:0/2153156788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564fe46c2f0 0x5564fe8b7bf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.463+0000 7f46c11e8740 1 -- 172.21.15.63:0/2153156788 >> 172.21.15.63:0/2153156788 conn(0x5564fe489300 msgr2=0x5564fe832fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.465+0000 7f469cc00640 0 client.42323 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.466+0000 7f46c11e8740 1 -- 172.21.15.63:0/2153156788 shutdown_connections 2024-05-27T08:16:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.467+0000 7f46c11e8740 1 -- 172.21.15.63:0/2153156788 wait complete. 2024-05-27T08:16:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.468+0000 7f46c11e8740 20 client.42323 trim_cache size 0 max 16384 2024-05-27T08:16:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:16:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:16:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:16:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.469+0000 7f46b8c00640 1 -- 172.21.15.63:0/823547638 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46980b2a20 msgr2=0x7f46980b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.469+0000 7f46b8c00640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46980b2a20 0x7f46980b4f30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x5564fe835550 tx=0x7f46a40095f0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.470+0000 7f46b8c00640 1 -- 172.21.15.63:0/823547638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46b4106420 msgr2=0x7f46b410eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.470+0000 7f46b8c00640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46b4106420 0x7f46b410eb80 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f46a402a980 tx=0x7f46a4003cd0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.472+0000 7f46b8c00640 1 -- 172.21.15.63:0/823547638 shutdown_connections 2024-05-27T08:16:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.472+0000 7f46b8c00640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f46980b2a20 0x7f46980b4f30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.472+0000 7f46b8c00640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46b410c170 0x7f46b41ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.472+0000 7f46b8c00640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46b410bcd0 0x7f46b410f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.472+0000 7f46b8c00640 1 --2- 172.21.15.63:0/823547638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46b4106420 0x7f46b410eb80 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.472+0000 7f46b8c00640 1 -- 172.21.15.63:0/823547638 >> 172.21.15.63:0/823547638 conn(0x7f46b41013d0 msgr2=0x7f46b4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.473+0000 7f46b8c00640 1 -- 172.21.15.63:0/823547638 shutdown_connections 2024-05-27T08:16:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:32.473+0000 7f46b8c00640 1 -- 172.21.15.63:0/823547638 wait complete. 2024-05-27T08:16:33.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:32 smithi086 ceph-mon[34687]: pgmap v1364: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 12 MiB/s wr, 3.05k op/s 2024-05-27T08:16:33.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:32 smithi149 ceph-mon[35408]: pgmap v1364: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 12 MiB/s wr, 3.05k op/s 2024-05-27T08:16:33.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:32 smithi063 ceph-mon[31613]: pgmap v1364: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 12 MiB/s wr, 3.05k op/s 2024-05-27T08:16:35.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:34 smithi086 ceph-mon[34687]: pgmap v1365: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.4 MiB/s wr, 2.38k op/s 2024-05-27T08:16:35.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:34 smithi063 ceph-mon[31613]: pgmap v1365: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.4 MiB/s wr, 2.38k op/s 2024-05-27T08:16:35.585 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:34 smithi149 ceph-mon[35408]: pgmap v1365: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.4 MiB/s wr, 2.38k op/s 2024-05-27T08:16:37.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:36 smithi086 ceph-mon[34687]: pgmap v1366: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-05-27T08:16:37.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:36 smithi063 ceph-mon[31613]: pgmap v1366: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-05-27T08:16:38.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:36 smithi149 ceph-mon[35408]: pgmap v1366: 129 pgs: 7 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 111 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 11 MiB/s wr, 2.78k op/s 2024-05-27T08:16:39.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:38 smithi086 ceph-mon[34687]: pgmap v1367: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.2 MiB/s wr, 2.33k op/s 2024-05-27T08:16:39.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:38 smithi149 ceph-mon[35408]: pgmap v1367: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.2 MiB/s wr, 2.33k op/s 2024-05-27T08:16:39.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:38 smithi063 ceph-mon[31613]: pgmap v1367: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.2 MiB/s wr, 2.33k op/s 2024-05-27T08:16:41.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:40 smithi149 ceph-mon[35408]: pgmap v1368: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.7 MiB/s wr, 2.48k op/s 2024-05-27T08:16:41.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:40 smithi063 ceph-mon[31613]: pgmap v1368: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.7 MiB/s wr, 2.48k op/s 2024-05-27T08:16:41.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:40 smithi086 ceph-mon[34687]: pgmap v1368: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 9.7 MiB/s wr, 2.48k op/s 2024-05-27T08:16:42.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:42 smithi086 ceph-mon[34687]: osdmap e166: 12 total, 12 up, 12 in 2024-05-27T08:16:42.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:42 smithi149 ceph-mon[35408]: osdmap e166: 12 total, 12 up, 12 in 2024-05-27T08:16:42.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:42 smithi063 ceph-mon[31613]: osdmap e166: 12 total, 12 up, 12 in 2024-05-27T08:16:43.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:43 smithi086 ceph-mon[34687]: pgmap v1370: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:16:43.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:43 smithi149 ceph-mon[35408]: pgmap v1370: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:16:43.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:43 smithi063 ceph-mon[31613]: pgmap v1370: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 MiB/s wr, 2.79k op/s 2024-05-27T08:16:45.215 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:44 smithi149 ceph-mon[35408]: pgmap v1371: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 MiB/s wr, 2.83k op/s 2024-05-27T08:16:45.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:44 smithi063 ceph-mon[31613]: pgmap v1371: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 MiB/s wr, 2.83k op/s 2024-05-27T08:16:45.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:44 smithi086 ceph-mon[34687]: pgmap v1371: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 MiB/s wr, 2.83k op/s 2024-05-27T08:16:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.801+0000 7f038cc00640 1 -- 172.21.15.63:0/55984765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0388101fe0 msgr2=0x7f03881023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.802+0000 7f038cc00640 1 --2- 172.21.15.63:0/55984765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0388101fe0 0x7f03881023f0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f0378009900 tx=0x7f037802a400 comp rx=0 tx=0).stop 2024-05-27T08:16:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.802+0000 7f038cc00640 1 -- 172.21.15.63:0/55984765 shutdown_connections 2024-05-27T08:16:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.802+0000 7f038cc00640 1 --2- 172.21.15.63:0/55984765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f038810a120 0x7f03881106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.802+0000 7f038cc00640 1 --2- 172.21.15.63:0/55984765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0388109680 0x7f0388109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.802+0000 7f038cc00640 1 --2- 172.21.15.63:0/55984765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0388101fe0 0x7f03881023f0 unknown :-1 s=CLOSED pgs=2952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.802+0000 7f038cc00640 1 -- 172.21.15.63:0/55984765 >> 172.21.15.63:0/55984765 conn(0x7f03880fd6a0 msgr2=0x7f03880ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.802+0000 7f038cc00640 1 -- 172.21.15.63:0/55984765 shutdown_connections 2024-05-27T08:16:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.802+0000 7f038cc00640 1 -- 172.21.15.63:0/55984765 wait complete. 2024-05-27T08:16:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.804+0000 7f038cc00640 1 Processor -- start 2024-05-27T08:16:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.804+0000 7f038cc00640 1 -- start start 2024-05-27T08:16:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.804+0000 7f038cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0388101fe0 0x7f038819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.804+0000 7f038cc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0388109680 0x7f038819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.804+0000 7f038cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f038810a120 0x7f03881a8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.804+0000 7f038cc00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0388113d00 con 0x7f0388101fe0 2024-05-27T08:16:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.804+0000 7f038cc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0388113b80 con 0x7f038810a120 2024-05-27T08:16:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.804+0000 7f038cc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0388113e80 con 0x7f0388109680 2024-05-27T08:16:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0385e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0388101fe0 0x7f038819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0386800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f038810a120 0x7f03881a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0385e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0388101fe0 0x7f038819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42314/0 (socket says 172.21.15.63:42314) 2024-05-27T08:16:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0385e00640 1 -- 172.21.15.63:0/4047206905 learned_addr learned my addr 172.21.15.63:0/4047206905 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0385400640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0388109680 0x7f038819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0386800640 1 -- 172.21.15.63:0/4047206905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0388109680 msgr2=0x7f038819dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0386800640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0388109680 0x7f038819dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0386800640 1 -- 172.21.15.63:0/4047206905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0388101fe0 msgr2=0x7f038819d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0386800640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0388101fe0 0x7f038819d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0386800640 1 -- 172.21.15.63:0/4047206905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03881a8bd0 con 0x7f038810a120 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.805+0000 7f0385400640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0388109680 0x7f038819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.806+0000 7f0386800640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f038810a120 0x7f03881a8660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f038000ec50 tx=0x7f038000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.806+0000 7f037ea00640 1 -- 172.21.15.63:0/4047206905 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380003b70 con 0x7f038810a120 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.806+0000 7f038cc00640 1 -- 172.21.15.63:0/4047206905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03881a8ec0 con 0x7f038810a120 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.806+0000 7f038cc00640 1 -- 172.21.15.63:0/4047206905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03881a9350 con 0x7f038810a120 2024-05-27T08:16:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.806+0000 7f037ea00640 1 -- 172.21.15.63:0/4047206905 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f038000ce10 con 0x7f038810a120 2024-05-27T08:16:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.806+0000 7f037ea00640 1 -- 172.21.15.63:0/4047206905 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380010690 con 0x7f038810a120 2024-05-27T08:16:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.810+0000 7f0393c01740 1 Processor -- start 2024-05-27T08:16:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.811+0000 7f037ea00640 1 -- 172.21.15.63:0/4047206905 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0380004020 con 0x7f038810a120 2024-05-27T08:16:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.811+0000 7f0393c01740 1 -- start start 2024-05-27T08:16:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.812+0000 7f0393c01740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f27070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.812+0000 7f0393c01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.812+0000 7f0393c01740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f27af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.812+0000 7f0393c01740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55df58e8fb90 con 0x55df58b7c1b0 2024-05-27T08:16:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.812+0000 7f0393c01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df58e7f730 con 0x55df58b532f0 2024-05-27T08:16:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.812+0000 7f0393c01740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55df58e83770 con 0x55df58b5b4d0 2024-05-27T08:16:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.812+0000 7f0385e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.812+0000 7f0385e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f275b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34444/0 (socket says 172.21.15.63:34444) 2024-05-27T08:16:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.812+0000 7f0385e00640 1 -- 172.21.15.63:0/3360623875 learned_addr learned my addr 172.21.15.63:0/3360623875 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.813+0000 7f0385400640 1 --2- 172.21.15.63:0/3360623875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f27070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.813+0000 7f0385e00640 1 -- 172.21.15.63:0/3360623875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 msgr2=0x55df58f27af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:16:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.813+0000 7f0385e00640 1 --2- 172.21.15.63:0/3360623875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f27af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.813+0000 7f0385e00640 1 -- 172.21.15.63:0/3360623875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 msgr2=0x55df58f27070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.813+0000 7f0385e00640 1 --2- 172.21.15.63:0/3360623875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f27070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.813+0000 7f0385e00640 1 -- 172.21.15.63:0/3360623875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df58e188b0 con 0x55df58b532f0 2024-05-27T08:16:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.813+0000 7f0385400640 1 --2- 172.21.15.63:0/3360623875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f27070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:16:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.813+0000 7f0385e00640 1 --2- 172.21.15.63:0/3360623875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f275b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f037802a920 tx=0x7f0378031a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d76683adb90787bc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.813+0000 7f037ea00640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f036c0b29d0 0x7f036c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.814+0000 7f0373000640 1 -- 172.21.15.63:0/3360623875 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378035030 con 0x55df58b532f0 2024-05-27T08:16:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.814+0000 7f037ea00640 1 -- 172.21.15.63:0/4047206905 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7f03800e2da0 con 0x7f038810a120 2024-05-27T08:16:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.814+0000 7f0373000640 1 -- 172.21.15.63:0/3360623875 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0378040070 con 0x55df58b532f0 2024-05-27T08:16:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.814+0000 7f0373000640 1 -- 172.21.15.63:0/3360623875 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378003850 con 0x55df58b532f0 2024-05-27T08:16:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.814+0000 7f0385400640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f036c0b29d0 0x7f036c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.815+0000 7f0393c01740 1 -- 172.21.15.63:0/3360623875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 msgr2=0x55df58f275b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.815+0000 7f0393c01740 1 --2- 172.21.15.63:0/3360623875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f275b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f037802a920 tx=0x7f0378031a70 comp rx=0 tx=0).stop 2024-05-27T08:16:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.815+0000 7f0385400640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f036c0b29d0 0x7f036c0b4ee0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0374009730 tx=0x7f0374005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.816+0000 7f0393c01740 1 -- 172.21.15.63:0/3360623875 shutdown_connections 2024-05-27T08:16:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.816+0000 7f0393c01740 1 --2- 172.21.15.63:0/3360623875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f27af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.816+0000 7f0393c01740 1 --2- 172.21.15.63:0/3360623875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f275b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.816+0000 7f0393c01740 1 --2- 172.21.15.63:0/3360623875 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f27070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.816+0000 7f0393c01740 1 -- 172.21.15.63:0/3360623875 >> 172.21.15.63:0/3360623875 conn(0x55df58b70300 msgr2=0x55df58f23820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.816+0000 7f0393c01740 1 -- 172.21.15.63:0/3360623875 shutdown_connections 2024-05-27T08:16:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.816+0000 7f0393c01740 1 -- 172.21.15.63:0/3360623875 wait complete. 2024-05-27T08:16:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.816+0000 7f0393c01740 1 -- start start 2024-05-27T08:16:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.817+0000 7f0393c01740 1 Processor -- start 2024-05-27T08:16:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.817+0000 7f0393c01740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b532f0 0x55df58f311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.817+0000 7f0393c01740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b5b4d0 0x55df58f316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0385400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b5b4d0 0x55df58f316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0385400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b5b4d0 0x55df58f316f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42336/0 (socket says 172.21.15.63:42336) 2024-05-27T08:16:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0385400640 1 -- 172.21.15.63:0/3583131609 learned_addr learned my addr 172.21.15.63:0/3583131609 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0393c01740 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b7c1b0 0x55df58f3d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55df58b65790 con 0x55df58b5b4d0 2024-05-27T08:16:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0385e00640 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b532f0 0x55df58f311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df58e27bc0 con 0x55df58b7c1b0 2024-05-27T08:16:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55df58f34140 con 0x55df58b532f0 2024-05-27T08:16:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0393c01740 10 client.0 _ll_register_callbacks cb 0x55df58e05400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0386800640 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b7c1b0 0x55df58f3d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0385e00640 1 -- 172.21.15.63:0/3583131609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b7c1b0 msgr2=0x55df58f3d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0385e00640 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b7c1b0 0x55df58f3d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.818+0000 7f0385e00640 1 -- 172.21.15.63:0/3583131609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b5b4d0 msgr2=0x55df58f316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.819+0000 7f0385e00640 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b5b4d0 0x55df58f316f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.819+0000 7f0385e00640 1 -- 172.21.15.63:0/3583131609 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df58b94210 con 0x55df58b532f0 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.819+0000 7f0385400640 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b5b4d0 0x55df58f316f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.819+0000 7f0385e00640 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b532f0 0x55df58f311b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55df58f28420 tx=0x7f0378009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad99479b9df604b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.819+0000 7f0367400640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.819+0000 7f0367400640 1 -- 172.21.15.63:0/3583131609 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037804c070 con 0x55df58b532f0 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.819+0000 7f0367400640 1 -- 172.21.15.63:0/3583131609 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7f037803c440 con 0x55df58b532f0 2024-05-27T08:16:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.819+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0378040070 con 0x55df58b532f0 2024-05-27T08:16:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.820+0000 7f0367400640 1 -- 172.21.15.63:0/3583131609 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0378058050 con 0x55df58b532f0 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.820+0000 7f0393c01740 10 client.32170 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.820+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55df58c60620 con 0x55df58b532f0 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.820+0000 7f0367400640 1 -- 172.21.15.63:0/3583131609 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0378040070 con 0x55df58b532f0 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.821+0000 7f0367400640 1 -- 172.21.15.63:0/3583131609 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f03780517b0 con 0x55df58b532f0 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.821+0000 7f0393c01740 10 client.32170 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.821+0000 7f0393c01740 10 client.32170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.821+0000 7f0393c01740 20 client.32170 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.821+0000 7f0393c01740 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55df58b3ad80 0x55df58f43260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.821+0000 7f0393c01740 4 client.32170 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.821+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df58e91ae0 con 0x55df58b3ad80 2024-05-27T08:16:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.821+0000 7f0385400640 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55df58b3ad80 0x55df58f43260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.823+0000 7f0385400640 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55df58b3ad80 0x55df58f43260 crc :-1 s=READY pgs=750 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c47bfcdd3e3f88ea server_cookie=1950a14a13d7d902 in_seq=0 out_seq=0 2024-05-27T08:16:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.823+0000 7f0367400640 10 client.32170 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0367400640 1 -- 172.21.15.63:0/3583131609 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55df58e91ae0 con 0x55df58b3ad80 2024-05-27T08:16:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0367400640 10 client.32170 handle_command_reply: tid=0 2024-05-27T08:16:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0393c01740 1 client.32170 shutdown 2024-05-27T08:16:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b532f0 msgr2=0x55df58f311b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0393c01740 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b532f0 0x55df58f311b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55df58f28420 tx=0x7f0378009660 comp rx=0 tx=0).stop 2024-05-27T08:16:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 shutdown_connections 2024-05-27T08:16:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0393c01740 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55df58b3ad80 0x55df58f43260 crc :-1 s=READY pgs=750 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0393c01740 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b7c1b0 0x55df58f3d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0393c01740 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b5b4d0 0x55df58f316f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0393c01740 1 --2- 172.21.15.63:0/3583131609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b532f0 0x55df58f311b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 >> 172.21.15.63:0/3583131609 conn(0x55df58b70300 msgr2=0x55df58f28d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.824+0000 7f0367400640 0 client.32170 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.825+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 shutdown_connections 2024-05-27T08:16:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.825+0000 7f0393c01740 1 -- 172.21.15.63:0/3583131609 wait complete. 2024-05-27T08:16:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.825+0000 7f0393c01740 20 client.32170 trim_cache size 0 max 16384 2024-05-27T08:16:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.844+0000 7f0393c01740 1 Processor -- start 2024-05-27T08:16:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.844+0000 7f0393c01740 1 -- start start 2024-05-27T08:16:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.845+0000 7f0393c01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f1a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.845+0000 7f0393c01740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f33300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.845+0000 7f0393c01740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f31150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.845+0000 7f0393c01740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0368012800 con 0x55df58b7c1b0 2024-05-27T08:16:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.845+0000 7f0393c01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03680100d0 con 0x55df58b532f0 2024-05-27T08:16:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.845+0000 7f0393c01740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f036800df20 con 0x55df58b5b4d0 2024-05-27T08:16:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.845+0000 7f0385400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f33300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.845+0000 7f0385400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f33300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:38370/0 (socket says 172.21.15.63:38370) 2024-05-27T08:16:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.845+0000 7f0385400640 1 -- 172.21.15.63:0/2855293400 learned_addr learned my addr 172.21.15.63:0/2855293400 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.846+0000 7f0386800640 1 --2- 172.21.15.63:0/2855293400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f31150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.846+0000 7f0385e00640 1 --2- 172.21.15.63:0/2855293400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f1a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.846+0000 7f0386800640 1 -- 172.21.15.63:0/2855293400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 msgr2=0x55df58f33300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.846+0000 7f0386800640 1 --2- 172.21.15.63:0/2855293400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f33300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.846+0000 7f0386800640 1 -- 172.21.15.63:0/2855293400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 msgr2=0x55df58f1a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.846+0000 7f0386800640 1 --2- 172.21.15.63:0/2855293400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f1a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.846+0000 7f0386800640 1 -- 172.21.15.63:0/2855293400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df58f435d0 con 0x55df58b7c1b0 2024-05-27T08:16:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.847+0000 7f0385e00640 1 --2- 172.21.15.63:0/2855293400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f1a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:16:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.847+0000 7f0386800640 1 --2- 172.21.15.63:0/2855293400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f31150 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f03800e00c0 tx=0x7f03800a94f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6cef41a9ef743 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.847+0000 7f0372600640 1 -- 172.21.15.63:0/2855293400 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380091040 con 0x55df58b7c1b0 2024-05-27T08:16:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.848+0000 7f0372600640 1 -- 172.21.15.63:0/2855293400 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03800aa350 con 0x55df58b7c1b0 2024-05-27T08:16:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.848+0000 7f0372600640 1 -- 172.21.15.63:0/2855293400 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380003d10 con 0x55df58b7c1b0 2024-05-27T08:16:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.849+0000 7f0393c01740 1 -- 172.21.15.63:0/2855293400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 msgr2=0x55df58f31150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.849+0000 7f0393c01740 1 --2- 172.21.15.63:0/2855293400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f31150 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f03800e00c0 tx=0x7f03800a94f0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.849+0000 7f0393c01740 1 -- 172.21.15.63:0/2855293400 shutdown_connections 2024-05-27T08:16:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.850+0000 7f0393c01740 1 --2- 172.21.15.63:0/2855293400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f31150 unknown :-1 s=CLOSED pgs=2953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.850+0000 7f0393c01740 1 --2- 172.21.15.63:0/2855293400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f33300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.850+0000 7f0393c01740 1 --2- 172.21.15.63:0/2855293400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f1a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.850+0000 7f0393c01740 1 -- 172.21.15.63:0/2855293400 >> 172.21.15.63:0/2855293400 conn(0x55df58b70300 msgr2=0x55df58f30a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.850+0000 7f0393c01740 1 -- 172.21.15.63:0/2855293400 shutdown_connections 2024-05-27T08:16:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.850+0000 7f0393c01740 1 -- 172.21.15.63:0/2855293400 wait complete. 2024-05-27T08:16:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.851+0000 7f0393c01740 1 -- start start 2024-05-27T08:16:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.851+0000 7f0393c01740 1 Processor -- start 2024-05-27T08:16:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.852+0000 7f0393c01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f2b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.852+0000 7f0393c01740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f2b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.852+0000 7f0393c01740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f38460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.852+0000 7f0393c01740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f036800e590 con 0x55df58b7c1b0 2024-05-27T08:16:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.852+0000 7f0393c01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f036800fa00 con 0x55df58b532f0 2024-05-27T08:16:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.852+0000 7f0393c01740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f036800ec60 con 0x55df58b5b4d0 2024-05-27T08:16:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.852+0000 7f0393c01740 10 client.0 _ll_register_callbacks cb 0x7f0378009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.852+0000 7f0385e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f2b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.853+0000 7f0386800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f38460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.853+0000 7f0385e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f2b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34474/0 (socket says 172.21.15.63:34474) 2024-05-27T08:16:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.853+0000 7f0385e00640 1 -- 172.21.15.63:0/3056962965 learned_addr learned my addr 172.21.15.63:0/3056962965 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.853+0000 7f0385400640 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f2b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.853+0000 7f0385e00640 1 -- 172.21.15.63:0/3056962965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 msgr2=0x55df58f2b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.853+0000 7f0385e00640 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f2b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.853+0000 7f0385e00640 1 -- 172.21.15.63:0/3056962965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 msgr2=0x55df58f38460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.853+0000 7f0385e00640 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f38460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.854+0000 7f0385e00640 1 -- 172.21.15.63:0/3056962965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df58f2bfb0 con 0x55df58b532f0 2024-05-27T08:16:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.854+0000 7f0385400640 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f2b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:16:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.854+0000 7f0386800640 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f38460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:16:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.854+0000 7f0385e00640 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f2b290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f037802a920 tx=0x7f0378031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec7d9b27a2b7d4a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.854+0000 7f0367400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:16:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.854+0000 7f0367400640 1 -- 172.21.15.63:0/3056962965 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037804c050 con 0x55df58b532f0 2024-05-27T08:16:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.854+0000 7f0367400640 1 -- 172.21.15.63:0/3056962965 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7f0378051640 con 0x55df58b532f0 2024-05-27T08:16:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.855+0000 7f0393c01740 1 -- 172.21.15.63:0/3056962965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03800aa350 con 0x55df58b532f0 2024-05-27T08:16:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.855+0000 7f0367400640 1 -- 172.21.15.63:0/3056962965 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0378035050 con 0x55df58b532f0 2024-05-27T08:16:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.855+0000 7f0393c01740 10 client.42356 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.855+0000 7f0393c01740 1 -- 172.21.15.63:0/3056962965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55df58f39940 con 0x55df58b532f0 2024-05-27T08:16:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.856+0000 7f0367400640 1 -- 172.21.15.63:0/3056962965 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03800aa350 con 0x55df58b532f0 2024-05-27T08:16:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.856+0000 7f0367400640 1 -- 172.21.15.63:0/3056962965 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0378067c30 con 0x55df58b532f0 2024-05-27T08:16:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.856+0000 7f0393c01740 10 client.42356 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.856+0000 7f0393c01740 10 client.42356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:16:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.856+0000 7f0393c01740 20 client.42356 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.856+0000 7f0393c01740 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55df58b101c0 0x55df58fad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.856+0000 7f0393c01740 4 client.42356 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:16:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.856+0000 7f0393c01740 1 -- 172.21.15.63:0/3056962965 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df58e8b2c0 con 0x55df58b101c0 2024-05-27T08:16:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.857+0000 7f0385400640 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55df58b101c0 0x55df58fad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.857+0000 7f0385400640 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55df58b101c0 0x55df58fad6e0 crc :-1 s=READY pgs=751 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2687a9868ef77f33 server_cookie=7119129105c14d2b in_seq=0 out_seq=0 2024-05-27T08:16:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:45.857+0000 7f0367400640 10 client.42356 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:46.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.972+0000 7fc93aa00700 1 -- 172.21.15.63:0/3008012420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc93410b010 msgr2=0x7fc93410b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:46.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.972+0000 7fc93aa00700 1 --2- 172.21.15.63:0/3008012420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc93410b010 0x7fc93410b420 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0098f0 tx=0x7fc91c009c00 comp rx=0 tx=0).stop 2024-05-27T08:16:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.972+0000 7fc93aa00700 1 -- 172.21.15.63:0/3008012420 shutdown_connections 2024-05-27T08:16:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.972+0000 7fc93aa00700 1 --2- 172.21.15.63:0/3008012420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc934110290 0x7fc934117bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.972+0000 7fc93aa00700 1 --2- 172.21.15.63:0/3008012420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc93410b010 0x7fc93410b420 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.972+0000 7fc93aa00700 1 --2- 172.21.15.63:0/3008012420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc93410f940 0x7fc93410fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.972+0000 7fc93aa00700 1 -- 172.21.15.63:0/3008012420 >> 172.21.15.63:0/3008012420 conn(0x7fc93406f980 msgr2=0x7fc93406fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.973+0000 7fc93aa00700 1 -- 172.21.15.63:0/3008012420 shutdown_connections 2024-05-27T08:16:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.973+0000 7fc93aa00700 1 -- 172.21.15.63:0/3008012420 wait complete. 2024-05-27T08:16:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.974+0000 7fc93aa00700 1 Processor -- start 2024-05-27T08:16:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.974+0000 7fc93aa00700 1 -- start start 2024-05-27T08:16:46.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.975+0000 7fc93aa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc93410b010 0x7fc9341a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.975+0000 7fc93aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc93410f940 0x7fc9341a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.975+0000 7fc93aa00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc934110290 0x7fc9341a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.975+0000 7fc93aa00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc93411aa20 con 0x7fc934110290 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.975+0000 7fc93aa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc93411a8a0 con 0x7fc93410f940 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.975+0000 7fc93aa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc93411aba0 con 0x7fc93410b010 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.976+0000 7fc933e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc93410b010 0x7fc9341a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.976+0000 7fc933e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc93410b010 0x7fc9341a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:38298/0 (socket says 172.21.15.63:38298) 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.976+0000 7fc933e00700 1 -- 172.21.15.63:0/1055540518 learned_addr learned my addr 172.21.15.63:0/1055540518 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.976+0000 7fc938a00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc934110290 0x7fc9341a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.976+0000 7fc933400700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc93410f940 0x7fc9341a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.977+0000 7fc933e00700 1 -- 172.21.15.63:0/1055540518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc93410f940 msgr2=0x7fc9341a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:46.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.977+0000 7fc933e00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc93410f940 0x7fc9341a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.977+0000 7fc933e00700 1 -- 172.21.15.63:0/1055540518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc934110290 msgr2=0x7fc9341a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.977+0000 7fc933e00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc934110290 0x7fc9341a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.977+0000 7fc933e00700 1 -- 172.21.15.63:0/1055540518 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9341a62c0 con 0x7fc93410b010 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.978+0000 7fc933e00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc93410b010 0x7fc9341a13f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00c010 tx=0x7fc91c00b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.978+0000 7fc930c00700 1 -- 172.21.15.63:0/1055540518 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c00b780 con 0x7fc93410b010 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.978+0000 7fc930c00700 1 -- 172.21.15.63:0/1055540518 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc91c0054d0 con 0x7fc93410b010 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.978+0000 7fc930c00700 1 -- 172.21.15.63:0/1055540518 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c003b20 con 0x7fc93410b010 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.978+0000 7fc93aa00700 1 -- 172.21.15.63:0/1055540518 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9341a6550 con 0x7fc93410b010 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.978+0000 7fc93aa00700 1 -- 172.21.15.63:0/1055540518 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9341a69e0 con 0x7fc93410b010 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.984+0000 7fc930c00700 1 -- 172.21.15.63:0/1055540518 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc91c003cc0 con 0x7fc93410b010 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.986+0000 7fc930c00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9180b6eb0 0x7fc9180b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.986+0000 7fc930c00700 1 -- 172.21.15.63:0/1055540518 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7fc91c0e3940 con 0x7fc93410b010 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.987+0000 7fc933400700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9180b6eb0 0x7fc9180b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:43.988+0000 7fc933400700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9180b6eb0 0x7fc9180b93c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc93410caa0 tx=0x7fc92c00b040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.389+0000 7fc9494a3b80 1 Processor -- start 2024-05-27T08:16:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.390+0000 7fc9494a3b80 1 -- start start 2024-05-27T08:16:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.391+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770aa6e40 0x55b770c33fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.391+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.391+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c2e5c0 0x55b770c2e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.391+0000 7fc9494a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b770b53e40 con 0x55b770aa6e40 2024-05-27T08:16:46.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.391+0000 7fc9494a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b770b439e0 con 0x55b770c344e0 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.391+0000 7fc9494a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b770b47a20 con 0x55b770c2e5c0 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.391+0000 7fc933400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.391+0000 7fc933400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34370/0 (socket says 172.21.15.63:34370) 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.391+0000 7fc933400700 1 -- 172.21.15.63:0/1122003606 learned_addr learned my addr 172.21.15.63:0/1122003606 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.392+0000 7fc933e00700 1 --2- 172.21.15.63:0/1122003606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770aa6e40 0x55b770c33fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.393+0000 7fc933400700 1 -- 172.21.15.63:0/1122003606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c2e5c0 msgr2=0x55b770c2e9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.393+0000 7fc933400700 1 --2- 172.21.15.63:0/1122003606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c2e5c0 0x55b770c2e9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.393+0000 7fc933400700 1 -- 172.21.15.63:0/1122003606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770aa6e40 msgr2=0x55b770c33fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.393+0000 7fc933400700 1 --2- 172.21.15.63:0/1122003606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770aa6e40 0x55b770c33fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.393+0000 7fc933400700 1 -- 172.21.15.63:0/1122003606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b770b0c420 con 0x55b770c344e0 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.393+0000 7fc933400700 1 --2- 172.21.15.63:0/1122003606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2e080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc92c003d50 tx=0x7fc92c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6071a42408066a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:46.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.394+0000 7fc929600700 1 -- 172.21.15.63:0/1122003606 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c01b070 con 0x55b770c344e0 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.394+0000 7fc929600700 1 -- 172.21.15.63:0/1122003606 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc92c003d80 con 0x55b770c344e0 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.394+0000 7fc929600700 1 -- 172.21.15.63:0/1122003606 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c017440 con 0x55b770c344e0 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.397+0000 7fc9494a3b80 1 -- 172.21.15.63:0/1122003606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 msgr2=0x55b770c2e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.397+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/1122003606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2e080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc92c003d50 tx=0x7fc92c014000 comp rx=0 tx=0).stop 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.398+0000 7fc9494a3b80 1 -- 172.21.15.63:0/1122003606 shutdown_connections 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.398+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/1122003606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c2e5c0 0x55b770c2e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.398+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/1122003606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770aa6e40 0x55b770c33fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.398+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/1122003606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2e080 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.398+0000 7fc9494a3b80 1 -- 172.21.15.63:0/1122003606 >> 172.21.15.63:0/1122003606 conn(0x55b770b06fc0 msgr2=0x55b770c208d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.398+0000 7fc9494a3b80 1 -- 172.21.15.63:0/1122003606 shutdown_connections 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.400+0000 7fc9494a3b80 1 -- 172.21.15.63:0/1122003606 wait complete. 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.400+0000 7fc9494a3b80 1 -- start start 2024-05-27T08:16:46.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.401+0000 7fc9494a3b80 1 Processor -- start 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.402+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770aa6e40 0x55b770c38480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.402+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 0x55b770c389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.402+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.402+0000 7fc9494a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b770a49fa0 con 0x55b770c2e5c0 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.402+0000 7fc9494a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b770ae4760 con 0x55b770c344e0 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.402+0000 7fc9494a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b770c25ab0 con 0x55b770aa6e40 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.404+0000 7fc938a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.404+0000 7fc938a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34386/0 (socket says 172.21.15.63:34386) 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.404+0000 7fc938a00700 1 -- 172.21.15.63:0/2037539258 learned_addr learned my addr 172.21.15.63:0/2037539258 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.405+0000 7fc933e00700 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770aa6e40 0x55b770c38480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.406+0000 7fc933400700 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 0x55b770c389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.406+0000 7fc9494a3b80 10 client.0 _ll_register_callbacks cb 0x55b770aedc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:46.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.408+0000 7fc933e00700 1 -- 172.21.15.63:0/2037539258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 msgr2=0x55b770c2ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.408+0000 7fc933e00700 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.408+0000 7fc933e00700 1 -- 172.21.15.63:0/2037539258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 msgr2=0x55b770c389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.408+0000 7fc933e00700 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 0x55b770c389c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.408+0000 7fc933e00700 1 -- 172.21.15.63:0/2037539258 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b770a938d0 con 0x55b770aa6e40 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.410+0000 7fc938a00700 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.410+0000 7fc922000700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.411+0000 7fc933e00700 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770aa6e40 0x55b770c38480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0e0cc0 tx=0x7fc91c0a6160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72db576cf125ca91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.411+0000 7fc922000700 1 -- 172.21.15.63:0/2037539258 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c0aec90 con 0x55b770aa6e40 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.411+0000 7fc9494a3b80 1 -- 172.21.15.63:0/2037539258 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc92c003d80 con 0x55b770aa6e40 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.411+0000 7fc922000700 1 -- 172.21.15.63:0/2037539258 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7fc91c090e00 con 0x55b770aa6e40 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.413+0000 7fc922000700 1 -- 172.21.15.63:0/2037539258 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc91c0aec90 con 0x55b770aa6e40 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.413+0000 7fc9494a3b80 10 client.32155 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:46.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.413+0000 7fc9494a3b80 1 -- 172.21.15.63:0/2037539258 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b770b14060 con 0x55b770aa6e40 2024-05-27T08:16:46.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.413+0000 7fc922000700 1 -- 172.21.15.63:0/2037539258 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc92c003d80 con 0x55b770aa6e40 2024-05-27T08:16:46.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.415+0000 7fc922000700 1 -- 172.21.15.63:0/2037539258 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc91c0e0f40 con 0x55b770aa6e40 2024-05-27T08:16:46.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.415+0000 7fc9494a3b80 10 client.32155 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:46.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.415+0000 7fc9494a3b80 10 client.32155 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:16:46.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.415+0000 7fc9494a3b80 20 client.32155 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:46.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.415+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b770c50630 0x55b770c52aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.415+0000 7fc9494a3b80 4 client.32155 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.415+0000 7fc9494a3b80 1 -- 172.21.15.63:0/2037539258 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b770b55d90 con 0x55b770c50630 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.416+0000 7fc933400700 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b770c50630 0x55b770c52aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.420+0000 7fc933400700 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b770c50630 0x55b770c52aa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d39cab675702ebf8 server_cookie=e7b07570451e022a in_seq=0 out_seq=0 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.420+0000 7fc922000700 10 client.32155 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.421+0000 7fc922000700 1 -- 172.21.15.63:0/2037539258 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b770b55d90 con 0x55b770c50630 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.421+0000 7fc922000700 10 client.32155 handle_command_reply: tid=0 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.422+0000 7fc9494a3b80 1 client.32155 shutdown 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.423+0000 7fc9494a3b80 1 -- 172.21.15.63:0/2037539258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770aa6e40 msgr2=0x55b770c38480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.423+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770aa6e40 0x55b770c38480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0e0cc0 tx=0x7fc91c0a6160 comp rx=0 tx=0).stop 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.423+0000 7fc9494a3b80 1 -- 172.21.15.63:0/2037539258 shutdown_connections 2024-05-27T08:16:46.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.423+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770c344e0 0x55b770c2ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.423+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770aa6e40 0x55b770c38480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.423+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b770c50630 0x55b770c52aa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.424+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/2037539258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 0x55b770c389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.424+0000 7fc9494a3b80 1 -- 172.21.15.63:0/2037539258 >> 172.21.15.63:0/2037539258 conn(0x55b770b06fc0 msgr2=0x55b770c24860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.424+0000 7fc922000700 0 client.32155 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.424+0000 7fc9494a3b80 1 -- 172.21.15.63:0/2037539258 shutdown_connections 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.425+0000 7fc9494a3b80 1 -- 172.21.15.63:0/2037539258 wait complete. 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.425+0000 7fc9494a3b80 20 client.32155 trim_cache size 0 max 16384 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.466+0000 7fc9494a3b80 1 Processor -- start 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.468+0000 7fc9494a3b80 1 -- start start 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.469+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c32970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.469+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c2e5c0 0x55b770c35c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.470+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c344e0 0x55b770c582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.470+0000 7fc933e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c32970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.470+0000 7fc9494a3b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc910011790 con 0x55b770c344e0 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.470+0000 7fc9494a3b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc91000f020 con 0x55b770aa6e40 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.470+0000 7fc9494a3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc91000ce10 con 0x55b770c2e5c0 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.470+0000 7fc933e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c32970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34396/0 (socket says 172.21.15.63:34396) 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.470+0000 7fc933400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c2e5c0 0x55b770c35c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.471+0000 7fc933400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c2e5c0 0x55b770c35c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:38318/0 (socket says 172.21.15.63:38318) 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.471+0000 7fc933400700 1 -- 172.21.15.63:0/4230683909 learned_addr learned my addr 172.21.15.63:0/4230683909 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.471+0000 7fc938a00700 1 --2- 172.21.15.63:0/4230683909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c344e0 0x55b770c582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.472+0000 7fc933e00700 1 -- 172.21.15.63:0/4230683909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c2e5c0 msgr2=0x55b770c35c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.472+0000 7fc933e00700 1 --2- 172.21.15.63:0/4230683909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c2e5c0 0x55b770c35c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.472+0000 7fc933e00700 1 -- 172.21.15.63:0/4230683909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c344e0 msgr2=0x55b770c582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:46.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.472+0000 7fc933e00700 1 --2- 172.21.15.63:0/4230683909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c344e0 0x55b770c582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.472+0000 7fc933e00700 1 -- 172.21.15.63:0/4230683909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b770c367f0 con 0x55b770aa6e40 2024-05-27T08:16:46.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.473+0000 7fc933e00700 1 --2- 172.21.15.63:0/4230683909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c32970 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0e0d70 tx=0x7fc91c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e56a103b16fa9f83 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:46.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.473+0000 7fc928c00700 1 -- 172.21.15.63:0/4230683909 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c0b36b0 con 0x55b770aa6e40 2024-05-27T08:16:46.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.474+0000 7fc928c00700 1 -- 172.21.15.63:0/4230683909 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc91c0a6ed0 con 0x55b770aa6e40 2024-05-27T08:16:46.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.474+0000 7fc928c00700 1 -- 172.21.15.63:0/4230683909 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c0aab40 con 0x55b770aa6e40 2024-05-27T08:16:46.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.476+0000 7fc9494a3b80 1 -- 172.21.15.63:0/4230683909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 msgr2=0x55b770c32970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:46.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.476+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/4230683909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c32970 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0e0d70 tx=0x7fc91c009c00 comp rx=0 tx=0).stop 2024-05-27T08:16:46.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.477+0000 7fc9494a3b80 1 -- 172.21.15.63:0/4230683909 shutdown_connections 2024-05-27T08:16:46.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.477+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/4230683909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c344e0 0x55b770c582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.477+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/4230683909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c32970 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.477+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/4230683909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c2e5c0 0x55b770c35c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.477+0000 7fc9494a3b80 1 -- 172.21.15.63:0/4230683909 >> 172.21.15.63:0/4230683909 conn(0x55b770b06fc0 msgr2=0x55b770c36b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:46.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.479+0000 7fc9494a3b80 1 -- 172.21.15.63:0/4230683909 shutdown_connections 2024-05-27T08:16:46.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.479+0000 7fc9494a3b80 1 -- 172.21.15.63:0/4230683909 wait complete. 2024-05-27T08:16:46.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.479+0000 7fc9494a3b80 1 -- start start 2024-05-27T08:16:46.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.481+0000 7fc9494a3b80 1 Processor -- start 2024-05-27T08:16:46.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.483+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c2cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.484+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 0x55b770c2d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.484+0000 7fc9494a3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c344e0 0x55b770c26410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.485+0000 7fc933e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c2cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.485+0000 7fc933e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c2cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34406/0 (socket says 172.21.15.63:34406) 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.486+0000 7fc933e00700 1 -- 172.21.15.63:0/3980638166 learned_addr learned my addr 172.21.15.63:0/3980638166 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.486+0000 7fc933400700 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 0x55b770c2d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.486+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc91000d4e0 con 0x55b770c2e5c0 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.487+0000 7fc938a00700 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c344e0 0x55b770c26410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.487+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc91000e950 con 0x55b770aa6e40 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.487+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc91000dbb0 con 0x55b770c344e0 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.488+0000 7fc9494a3b80 10 client.0 _ll_register_callbacks cb 0x55b770aedc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.489+0000 7fc933400700 1 -- 172.21.15.63:0/3980638166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c344e0 msgr2=0x55b770c26410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:46.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.489+0000 7fc933400700 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c344e0 0x55b770c26410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.489+0000 7fc933400700 1 -- 172.21.15.63:0/3980638166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 msgr2=0x55b770c2cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.492+0000 7fc933400700 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c2cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.492+0000 7fc933400700 1 -- 172.21.15.63:0/3980638166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b770c26bb0 con 0x55b770c2e5c0 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.492+0000 7fc933e00700 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c2cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.493+0000 7fc933400700 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 0x55b770c2d430 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fc92c003cf0 tx=0x7fc92c00feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603695a03e59cb2f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.493+0000 7fc922000700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.494+0000 7fc922000700 1 -- 172.21.15.63:0/3980638166 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c01b040 con 0x55b770c2e5c0 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.494+0000 7fc922000700 1 -- 172.21.15.63:0/3980638166 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7fc92c022470 con 0x55b770c2e5c0 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.495+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc91c0a6ed0 con 0x55b770c2e5c0 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.496+0000 7fc922000700 1 -- 172.21.15.63:0/3980638166 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc92c022670 con 0x55b770c2e5c0 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.497+0000 7fc9494a3b80 10 client.23478 fetch_fsmap learned FSMap version 35 2024-05-27T08:16:46.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.497+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b770c2b480 con 0x55b770c2e5c0 2024-05-27T08:16:46.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.498+0000 7fc922000700 1 -- 172.21.15.63:0/3980638166 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc91c0a6ed0 con 0x55b770c2e5c0 2024-05-27T08:16:46.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.498+0000 7fc922000700 1 -- 172.21.15.63:0/3980638166 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc92c017440 con 0x55b770c2e5c0 2024-05-27T08:16:46.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.498+0000 7fc9494a3b80 10 client.23478 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:16:46.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.498+0000 7fc9494a3b80 10 client.23478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:16:46.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.498+0000 7fc9494a3b80 20 client.23478 populate_metadata read hostname 'smithi063' 2024-05-27T08:16:46.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.498+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b770cbb170 0x55b770cbd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:16:46.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.498+0000 7fc9494a3b80 4 client.23478 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:16:46.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.498+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b770b4f570 con 0x55b770cbb170 2024-05-27T08:16:46.779 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:16:46.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.499+0000 7fc933e00700 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b770cbb170 0x55b770cbd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:16:46.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.500+0000 7fc933e00700 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b770cbb170 0x55b770cbd590 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a3ea91446a2ca69 server_cookie=b87fe04344bc2874 in_seq=0 out_seq=0 2024-05-27T08:16:46.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.500+0000 7fc922000700 10 client.23478 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:16:46.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc922000700 1 -- 172.21.15.63:0/3980638166 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b770b4f570 con 0x55b770cbb170 2024-05-27T08:16:46.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc922000700 10 client.23478 handle_command_reply: tid=0 2024-05-27T08:16:46.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc9494a3b80 1 client.23478 shutdown 2024-05-27T08:16:46.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 msgr2=0x55b770c2d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:46.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 0x55b770c2d430 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fc92c003cf0 tx=0x7fc92c00feb0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 shutdown_connections 2024-05-27T08:16:46.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b770c344e0 0x55b770c26410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b770aa6e40 0x55b770c2cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55b770cbb170 0x55b770cbd590 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc9494a3b80 1 --2- 172.21.15.63:0/3980638166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b770c2e5c0 0x55b770c2d430 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 >> 172.21.15.63:0/3980638166 conn(0x55b770b06fc0 msgr2=0x55b770c1fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.501+0000 7fc922000700 0 client.23478 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.502+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 shutdown_connections 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.502+0000 7fc9494a3b80 1 -- 172.21.15.63:0/3980638166 wait complete. 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.502+0000 7fc9494a3b80 20 client.23478 trim_cache size 0 max 16384 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.506+0000 7fc93aa00700 1 -- 172.21.15.63:0/1055540518 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9180b6eb0 msgr2=0x7fc9180b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.506+0000 7fc93aa00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9180b6eb0 0x7fc9180b93c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc93410caa0 tx=0x7fc92c00b040 comp rx=0 tx=0).stop 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.506+0000 7fc93aa00700 1 -- 172.21.15.63:0/1055540518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc93410b010 msgr2=0x7fc9341a13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.506+0000 7fc93aa00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc93410b010 0x7fc9341a13f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00c010 tx=0x7fc91c00b560 comp rx=0 tx=0).stop 2024-05-27T08:16:46.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.506+0000 7fc93aa00700 1 -- 172.21.15.63:0/1055540518 shutdown_connections 2024-05-27T08:16:46.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.506+0000 7fc93aa00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc9180b6eb0 0x7fc9180b93c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.506+0000 7fc93aa00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc934110290 0x7fc9341a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.506+0000 7fc93aa00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc93410b010 0x7fc9341a13f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.506+0000 7fc93aa00700 1 --2- 172.21.15.63:0/1055540518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc93410f940 0x7fc9341a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:46.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.506+0000 7fc93aa00700 1 -- 172.21.15.63:0/1055540518 >> 172.21.15.63:0/1055540518 conn(0x7fc93406f980 msgr2=0x7fc93410d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:46.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.509+0000 7fc93aa00700 1 -- 172.21.15.63:0/1055540518 shutdown_connections 2024-05-27T08:16:46.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:16:44.509+0000 7fc93aa00700 1 -- 172.21.15.63:0/1055540518 wait complete. 2024-05-27T08:16:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.155+0000 7f0367400640 1 -- 172.21.15.63:0/3056962965 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55df58e8b2c0 con 0x55df58b101c0 2024-05-27T08:16:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.155+0000 7f0367400640 10 client.42356 handle_command_reply: tid=0 2024-05-27T08:16:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.155+0000 7f0393c01740 1 client.42356 shutdown 2024-05-27T08:16:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.156+0000 7f0393c01740 1 -- 172.21.15.63:0/3056962965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 msgr2=0x55df58f2b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.156+0000 7f0393c01740 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f2b290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f037802a920 tx=0x7f0378031b60 comp rx=0 tx=0).stop 2024-05-27T08:16:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.156+0000 7f0393c01740 1 -- 172.21.15.63:0/3056962965 shutdown_connections 2024-05-27T08:16:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.156+0000 7f0393c01740 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55df58b101c0 0x55df58fad6e0 crc :-1 s=READY pgs=751 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.156+0000 7f0393c01740 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df58b7c1b0 0x55df58f38460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.156+0000 7f0393c01740 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df58b5b4d0 0x55df58f2b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.156+0000 7f0393c01740 1 --2- 172.21.15.63:0/3056962965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df58b532f0 0x55df58f2b290 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.156+0000 7f0393c01740 1 -- 172.21.15.63:0/3056962965 >> 172.21.15.63:0/3056962965 conn(0x55df58b70300 msgr2=0x55df58f31470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.157+0000 7f0367400640 0 client.42356 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:16:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.157+0000 7f0393c01740 1 -- 172.21.15.63:0/3056962965 shutdown_connections 2024-05-27T08:16:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.157+0000 7f0393c01740 1 -- 172.21.15.63:0/3056962965 wait complete. 2024-05-27T08:16:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.158+0000 7f0393c01740 20 client.42356 trim_cache size 0 max 16384 2024-05-27T08:16:47.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:16:47.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:16:47.159 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:16:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.160+0000 7f038cc00640 1 -- 172.21.15.63:0/4047206905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f036c0b29d0 msgr2=0x7f036c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.160+0000 7f038cc00640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f036c0b29d0 0x7f036c0b4ee0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0374009730 tx=0x7f0374005f90 comp rx=0 tx=0).stop 2024-05-27T08:16:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.161+0000 7f038cc00640 1 -- 172.21.15.63:0/4047206905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f038810a120 msgr2=0x7f03881a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:16:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.161+0000 7f038cc00640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f038810a120 0x7f03881a8660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f038000ec50 tx=0x7f038000c6a0 comp rx=0 tx=0).stop 2024-05-27T08:16:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.162+0000 7f038cc00640 1 -- 172.21.15.63:0/4047206905 shutdown_connections 2024-05-27T08:16:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.162+0000 7f038cc00640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f036c0b29d0 0x7f036c0b4ee0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.162+0000 7f038cc00640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f038810a120 0x7f03881a8660 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.162+0000 7f038cc00640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0388109680 0x7f038819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.162+0000 7f038cc00640 1 --2- 172.21.15.63:0/4047206905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0388101fe0 0x7f038819d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:16:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.162+0000 7f038cc00640 1 -- 172.21.15.63:0/4047206905 >> 172.21.15.63:0/4047206905 conn(0x7f03880fd6a0 msgr2=0x7f03881032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:16:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.162+0000 7f038cc00640 1 -- 172.21.15.63:0/4047206905 shutdown_connections 2024-05-27T08:16:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:16:47.162+0000 7f038cc00640 1 -- 172.21.15.63:0/4047206905 wait complete. 2024-05-27T08:16:47.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:46 smithi086 ceph-mon[34687]: pgmap v1372: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 11 MiB/s wr, 2.70k op/s 2024-05-27T08:16:47.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:46 smithi149 ceph-mon[35408]: pgmap v1372: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 11 MiB/s wr, 2.70k op/s 2024-05-27T08:16:47.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:46 smithi063 ceph-mon[31613]: pgmap v1372: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 11 MiB/s wr, 2.70k op/s 2024-05-27T08:16:49.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:48 smithi086 ceph-mon[34687]: pgmap v1373: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-05-27T08:16:49.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:48 smithi149 ceph-mon[35408]: pgmap v1373: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-05-27T08:16:49.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:48 smithi063 ceph-mon[31613]: pgmap v1373: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-05-27T08:16:49.931 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:16:51.221 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:50 smithi063 ceph-mon[31613]: pgmap v1374: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-27T08:16:51.241 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:16:51.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:50 smithi086 ceph-mon[34687]: pgmap v1374: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-27T08:16:51.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:50 smithi149 ceph-mon[35408]: pgmap v1374: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-27T08:16:53.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:52 smithi086 ceph-mon[34687]: pgmap v1375: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-05-27T08:16:53.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:52 smithi149 ceph-mon[35408]: pgmap v1375: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-05-27T08:16:53.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:52 smithi063 ceph-mon[31613]: pgmap v1375: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-05-27T08:16:55.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:54 smithi086 ceph-mon[34687]: pgmap v1376: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.9 MiB/s wr, 2.25k op/s 2024-05-27T08:16:55.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:54 smithi149 ceph-mon[35408]: pgmap v1376: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.9 MiB/s wr, 2.25k op/s 2024-05-27T08:16:55.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:54 smithi063 ceph-mon[31613]: pgmap v1376: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.9 MiB/s wr, 2.25k op/s 2024-05-27T08:16:57.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:57 smithi086 ceph-mon[34687]: pgmap v1377: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-05-27T08:16:57.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:57 smithi149 ceph-mon[35408]: pgmap v1377: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-05-27T08:16:57.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:57 smithi063 ceph-mon[31613]: pgmap v1377: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-05-27T08:16:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:16:58 smithi086 ceph-mon[34687]: pgmap v1378: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 2.63k op/s 2024-05-27T08:16:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:16:58 smithi149 ceph-mon[35408]: pgmap v1378: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 2.63k op/s 2024-05-27T08:16:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:16:58 smithi063 ceph-mon[31613]: pgmap v1378: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 2.63k op/s 2024-05-27T08:17:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.016+0000 7f757b000640 1 -- 172.21.15.63:0/1804568957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757410a1f0 msgr2=0x7f757410a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.016+0000 7f757b000640 1 --2- 172.21.15.63:0/1804568957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757410a1f0 0x7f757410a600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f756c009ca0 tx=0x7f756c02a970 comp rx=0 tx=0).stop 2024-05-27T08:17:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.022+0000 7f757b000640 1 -- 172.21.15.63:0/1804568957 shutdown_connections 2024-05-27T08:17:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.022+0000 7f757b000640 1 --2- 172.21.15.63:0/1804568957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757411d240 0x7f757411f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.022+0000 7f757b000640 1 --2- 172.21.15.63:0/1804568957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f757411a8e0 0x7f757411cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.022+0000 7f757b000640 1 --2- 172.21.15.63:0/1804568957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757410a1f0 0x7f757410a600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.022+0000 7f757b000640 1 -- 172.21.15.63:0/1804568957 >> 172.21.15.63:0/1804568957 conn(0x7f757406e8f0 msgr2=0x7f757406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.024+0000 7f757b000640 1 -- 172.21.15.63:0/1804568957 shutdown_connections 2024-05-27T08:17:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.025+0000 7f757b000640 1 -- 172.21.15.63:0/1804568957 wait complete. 2024-05-27T08:17:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.029+0000 7f757b000640 1 Processor -- start 2024-05-27T08:17:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.029+0000 7f757b000640 1 -- start start 2024-05-27T08:17:01.031 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.030+0000 7f757b000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757410a1f0 0x7f75741a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.031+0000 7f757b000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757411a8e0 0x7f75741a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.031+0000 7f757b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f757411d240 0x7f75741ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.031+0000 7f7573e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757410a1f0 0x7f75741a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.031+0000 7f7573e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757410a1f0 0x7f75741a9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53822/0 (socket says 172.21.15.63:53822) 2024-05-27T08:17:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.031+0000 7f7573e00640 1 -- 172.21.15.63:0/3433108138 learned_addr learned my addr 172.21.15.63:0/3433108138 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.031+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7574122b20 con 0x7f757411a8e0 2024-05-27T08:17:01.033 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.031+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75741229a0 con 0x7f757411d240 2024-05-27T08:17:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.031+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7574122ca0 con 0x7f757410a1f0 2024-05-27T08:17:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.032+0000 7f7573400640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757411a8e0 0x7f75741a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.032+0000 7f7579000640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f757411d240 0x7f75741ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.033+0000 7f7573400640 1 -- 172.21.15.63:0/3433108138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757410a1f0 msgr2=0x7f75741a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.033+0000 7f7573400640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757410a1f0 0x7f75741a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.033+0000 7f7573400640 1 -- 172.21.15.63:0/3433108138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f757411d240 msgr2=0x7f75741ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.033+0000 7f7573400640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f757411d240 0x7f75741ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.033+0000 7f7573400640 1 -- 172.21.15.63:0/3433108138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75741ae470 con 0x7f757411a8e0 2024-05-27T08:17:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.033+0000 7f7579000640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f757411d240 0x7f75741ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:17:01.036 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.034+0000 7f7573400640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757411a8e0 0x7f75741a9bc0 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7f756400e9f0 tx=0x7f756400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.035+0000 7f7570c00640 1 -- 172.21.15.63:0/3433108138 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756400ccc0 con 0x7f757411a8e0 2024-05-27T08:17:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.035+0000 7f7570c00640 1 -- 172.21.15.63:0/3433108138 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75640108c0 con 0x7f757411a8e0 2024-05-27T08:17:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.035+0000 7f7570c00640 1 -- 172.21.15.63:0/3433108138 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640187e0 con 0x7f757411a8e0 2024-05-27T08:17:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.035+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75741ae760 con 0x7f757411a8e0 2024-05-27T08:17:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.037+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75741aec60 con 0x7f757411a8e0 2024-05-27T08:17:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.050+0000 7f758207d740 1 Processor -- start 2024-05-27T08:17:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.051+0000 7f758207d740 1 -- start start 2024-05-27T08:17:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.051+0000 7f758207d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aaafc68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.054 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.051+0000 7f758207d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 0x556aaafc6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.051+0000 7f758207d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 0x556aaafc7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.051+0000 7f758207d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556aaaf37b90 con 0x556aaac034d0 2024-05-27T08:17:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.051+0000 7f758207d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556aaaf27730 con 0x556aaac241b0 2024-05-27T08:17:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.051+0000 7f758207d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556aaaf2b770 con 0x556aaabfb2f0 2024-05-27T08:17:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.052+0000 7f7570c00640 1 -- 172.21.15.63:0/3433108138 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7564010430 con 0x7f757411a8e0 2024-05-27T08:17:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.052+0000 7f7573e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 0x556aaafc6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.052+0000 7f7573e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 0x556aaafc6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53824/0 (socket says 172.21.15.63:53824) 2024-05-27T08:17:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.052+0000 7f7573e00640 1 -- 172.21.15.63:0/3428620924 learned_addr learned my addr 172.21.15.63:0/3428620924 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.053+0000 7f7573400640 1 --2- 172.21.15.63:0/3428620924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 0x556aaafc7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.054+0000 7f7579000640 1 --2- 172.21.15.63:0/3428620924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aaafc68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.054+0000 7f7573e00640 1 -- 172.21.15.63:0/3428620924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 msgr2=0x556aaafc68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.054+0000 7f7573e00640 1 --2- 172.21.15.63:0/3428620924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aaafc68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.054+0000 7f7573e00640 1 -- 172.21.15.63:0/3428620924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 msgr2=0x556aaafc7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.054+0000 7f7573e00640 1 --2- 172.21.15.63:0/3428620924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 0x556aaafc7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.054+0000 7f7573e00640 1 -- 172.21.15.63:0/3428620924 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aaaec08b0 con 0x556aaabfb2f0 2024-05-27T08:17:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.055+0000 7f7570c00640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75580b2a20 0x7f75580b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.056+0000 7f7579000640 1 --2- 172.21.15.63:0/3428620924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aaafc68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:17:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.057+0000 7f7579000640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75580b2a20 0x7f75580b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.058+0000 7f7570c00640 1 -- 172.21.15.63:0/3433108138 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7f75640e3cd0 con 0x7f757411a8e0 2024-05-27T08:17:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.060+0000 7f7573e00640 1 --2- 172.21.15.63:0/3428620924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 0x556aaafc6df0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f756c0094c0 tx=0x7f756c02f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287f0914b660129 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:01.062 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.061+0000 7f7569600640 1 -- 172.21.15.63:0/3428620924 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c048070 con 0x556aaabfb2f0 2024-05-27T08:17:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.061+0000 7f7579000640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75580b2a20 0x7f75580b4f30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x556aaafc3a20 tx=0x7f7560005d00 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.061+0000 7f7569600640 1 -- 172.21.15.63:0/3428620924 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f756c02fdf0 con 0x556aaabfb2f0 2024-05-27T08:17:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.061+0000 7f7569600640 1 -- 172.21.15.63:0/3428620924 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c034040 con 0x556aaabfb2f0 2024-05-27T08:17:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.063+0000 7f758207d740 1 -- 172.21.15.63:0/3428620924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 msgr2=0x556aaafc6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.063+0000 7f758207d740 1 --2- 172.21.15.63:0/3428620924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 0x556aaafc6df0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f756c0094c0 tx=0x7f756c02f3c0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.066 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.065+0000 7f758207d740 1 -- 172.21.15.63:0/3428620924 shutdown_connections 2024-05-27T08:17:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.066+0000 7f758207d740 1 --2- 172.21.15.63:0/3428620924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 0x556aaafc7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.067 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.066+0000 7f758207d740 1 --2- 172.21.15.63:0/3428620924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 0x556aaafc6df0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.069+0000 7f758207d740 1 --2- 172.21.15.63:0/3428620924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aaafc68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.069+0000 7f758207d740 1 -- 172.21.15.63:0/3428620924 >> 172.21.15.63:0/3428620924 conn(0x556aaac18300 msgr2=0x556aaafc3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.069+0000 7f758207d740 1 -- 172.21.15.63:0/3428620924 shutdown_connections 2024-05-27T08:17:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.069+0000 7f758207d740 1 -- 172.21.15.63:0/3428620924 wait complete. 2024-05-27T08:17:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.071+0000 7f758207d740 1 -- start start 2024-05-27T08:17:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.072+0000 7f758207d740 1 Processor -- start 2024-05-27T08:17:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.077+0000 7f758207d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaabfb2f0 0x556aaafd0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.077+0000 7f758207d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 0x556aaafd0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.078+0000 7f758207d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.078+0000 7f758207d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556aaac0d790 con 0x556aaabfb2f0 2024-05-27T08:17:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.078+0000 7f758207d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556aaaecfbc0 con 0x556aaac034d0 2024-05-27T08:17:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.078+0000 7f758207d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556aaafd3980 con 0x556aaac241b0 2024-05-27T08:17:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.079+0000 7f758207d740 10 client.0 _ll_register_callbacks cb 0x556aaaead400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.079+0000 7f7579000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.081 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.079+0000 7f7579000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53838/0 (socket says 172.21.15.63:53838) 2024-05-27T08:17:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.079+0000 7f7579000640 1 -- 172.21.15.63:0/827163502 learned_addr learned my addr 172.21.15.63:0/827163502 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.080+0000 7f7573e00640 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaabfb2f0 0x556aaafd0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.080+0000 7f7579000640 1 -- 172.21.15.63:0/827163502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 msgr2=0x556aaafd0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.080+0000 7f7573400640 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 0x556aaafd0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.080+0000 7f7579000640 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 0x556aaafd0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.080+0000 7f7579000640 1 -- 172.21.15.63:0/827163502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaabfb2f0 msgr2=0x556aaafd0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.080+0000 7f7579000640 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaabfb2f0 0x556aaafd0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.081+0000 7f7579000640 1 -- 172.21.15.63:0/827163502 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aaac3c210 con 0x556aaac241b0 2024-05-27T08:17:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.081+0000 7f7579000640 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdcce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f756000ff20 tx=0x7f7560004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=385111424344c1ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.081+0000 7f755e000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:17:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.082+0000 7f7573400640 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 0x556aaafd0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:17:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.082+0000 7f755e000640 1 -- 172.21.15.63:0/827163502 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560022070 con 0x556aaac241b0 2024-05-27T08:17:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.082+0000 7f755e000640 1 -- 172.21.15.63:0/827163502 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7f7560005220 con 0x556aaac241b0 2024-05-27T08:17:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.083+0000 7f758207d740 1 -- 172.21.15.63:0/827163502 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f756c02fdf0 con 0x556aaac241b0 2024-05-27T08:17:01.089 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.088+0000 7f755e000640 1 -- 172.21.15.63:0/827163502 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f756002b050 con 0x556aaac241b0 2024-05-27T08:17:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.089+0000 7f758207d740 10 client.32188 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.089+0000 7f758207d740 1 -- 172.21.15.63:0/827163502 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556aaad08620 con 0x556aaac241b0 2024-05-27T08:17:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.093+0000 7f755e000640 1 -- 172.21.15.63:0/827163502 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f756c02fdf0 con 0x556aaac241b0 2024-05-27T08:17:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.093+0000 7f755e000640 1 -- 172.21.15.63:0/827163502 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f756001e660 con 0x556aaac241b0 2024-05-27T08:17:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.095+0000 7f758207d740 10 client.32188 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.095+0000 7f758207d740 10 client.32188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.095+0000 7f758207d740 20 client.32188 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.095+0000 7f758207d740 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556aaabe2d80 0x556aaafe2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.095+0000 7f758207d740 4 client.32188 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:17:01.096 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.095+0000 7f758207d740 1 -- 172.21.15.63:0/827163502 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aaaf39ae0 con 0x556aaabe2d80 2024-05-27T08:17:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.095+0000 7f7573e00640 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556aaabe2d80 0x556aaafe2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.098+0000 7f7573e00640 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556aaabe2d80 0x556aaafe2aa0 crc :-1 s=READY pgs=752 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c8c3831e16e09d server_cookie=aca9137eeddc5e02 in_seq=0 out_seq=0 2024-05-27T08:17:01.099 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.098+0000 7f755e000640 10 client.32188 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.100+0000 7f755e000640 1 -- 172.21.15.63:0/827163502 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556aaaf39ae0 con 0x556aaabe2d80 2024-05-27T08:17:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.100+0000 7f755e000640 10 client.32188 handle_command_reply: tid=0 2024-05-27T08:17:01.101 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.100+0000 7f758207d740 1 client.32188 shutdown 2024-05-27T08:17:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.101+0000 7f758207d740 1 -- 172.21.15.63:0/827163502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 msgr2=0x556aaafdcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.101+0000 7f758207d740 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdcce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f756000ff20 tx=0x7f7560004730 comp rx=0 tx=0).stop 2024-05-27T08:17:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.101+0000 7f758207d740 1 -- 172.21.15.63:0/827163502 shutdown_connections 2024-05-27T08:17:01.103 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.101+0000 7f758207d740 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556aaabe2d80 0x556aaafe2aa0 crc :-1 s=READY pgs=752 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.104 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.101+0000 7f758207d740 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdcce0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.101+0000 7f758207d740 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 0x556aaafd0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.101+0000 7f758207d740 1 --2- 172.21.15.63:0/827163502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaabfb2f0 0x556aaafd0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.101+0000 7f755e000640 0 client.32188 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.101+0000 7f758207d740 1 -- 172.21.15.63:0/827163502 >> 172.21.15.63:0/827163502 conn(0x556aaac18300 msgr2=0x556aaafc85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.103+0000 7f758207d740 1 -- 172.21.15.63:0/827163502 shutdown_connections 2024-05-27T08:17:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.108+0000 7f758207d740 1 -- 172.21.15.63:0/827163502 wait complete. 2024-05-27T08:17:01.110 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.109+0000 7f758207d740 20 client.32188 trim_cache size 0 max 16384 2024-05-27T08:17:01.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:00 smithi149 ceph-mon[35408]: pgmap v1379: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.3 MiB/s wr, 2.36k op/s 2024-05-27T08:17:01.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:00 smithi063 ceph-mon[31613]: pgmap v1379: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.3 MiB/s wr, 2.36k op/s 2024-05-27T08:17:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.215+0000 7f758207d740 1 Processor -- start 2024-05-27T08:17:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.216+0000 7f758207d740 1 -- start start 2024-05-27T08:17:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.216+0000 7f758207d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaabfb2f0 0x556aaafd31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.216+0000 7f758207d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 0x556aaafd0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.216+0000 7f758207d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.216+0000 7f758207d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75741c2b70 con 0x556aaabfb2f0 2024-05-27T08:17:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.216+0000 7f758207d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75741c0440 con 0x556aaac034d0 2024-05-27T08:17:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.216+0000 7f758207d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f75741be290 con 0x556aaac241b0 2024-05-27T08:17:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.217+0000 7f7573400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 0x556aaafd0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.217+0000 7f7579000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.217+0000 7f7579000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53850/0 (socket says 172.21.15.63:53850) 2024-05-27T08:17:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.217+0000 7f7579000640 1 -- 172.21.15.63:0/3610084707 learned_addr learned my addr 172.21.15.63:0/3610084707 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.218+0000 7f7573400640 1 -- 172.21.15.63:0/3610084707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 msgr2=0x556aaafdca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.218+0000 7f7573400640 1 --2- 172.21.15.63:0/3610084707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.218+0000 7f7573400640 1 -- 172.21.15.63:0/3610084707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaabfb2f0 msgr2=0x556aaafd31d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:17:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.220+0000 7f7573400640 1 --2- 172.21.15.63:0/3610084707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaabfb2f0 0x556aaafd31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.220+0000 7f7573400640 1 -- 172.21.15.63:0/3610084707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aaafbd020 con 0x556aaac034d0 2024-05-27T08:17:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.223+0000 7f7573400640 1 --2- 172.21.15.63:0/3610084707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 0x556aaafd0560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f75640e0000 tx=0x7f756400ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21efc7f382cad32 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.229+0000 7f7568c00640 1 -- 172.21.15.63:0/3610084707 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640e7060 con 0x556aaac034d0 2024-05-27T08:17:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.230+0000 7f7568c00640 1 -- 172.21.15.63:0/3610084707 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75640b2340 con 0x556aaac034d0 2024-05-27T08:17:01.231 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.230+0000 7f7568c00640 1 -- 172.21.15.63:0/3610084707 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640b29f0 con 0x556aaac034d0 2024-05-27T08:17:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.233+0000 7f758207d740 1 -- 172.21.15.63:0/3610084707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 msgr2=0x556aaafd0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.233+0000 7f758207d740 1 --2- 172.21.15.63:0/3610084707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 0x556aaafd0560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f75640e0000 tx=0x7f756400ce60 comp rx=0 tx=0).stop 2024-05-27T08:17:01.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.236+0000 7f758207d740 1 -- 172.21.15.63:0/3610084707 shutdown_connections 2024-05-27T08:17:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.237+0000 7f758207d740 1 --2- 172.21.15.63:0/3610084707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaac241b0 0x556aaafdca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.237+0000 7f758207d740 1 --2- 172.21.15.63:0/3610084707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac034d0 0x556aaafd0560 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.237+0000 7f758207d740 1 --2- 172.21.15.63:0/3610084707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaabfb2f0 0x556aaafd31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.237+0000 7f758207d740 1 -- 172.21.15.63:0/3610084707 >> 172.21.15.63:0/3610084707 conn(0x556aaac18300 msgr2=0x556aaafcbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.237+0000 7f758207d740 1 -- 172.21.15.63:0/3610084707 shutdown_connections 2024-05-27T08:17:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.238+0000 7f758207d740 1 -- 172.21.15.63:0/3610084707 wait complete. 2024-05-27T08:17:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.239+0000 7f758207d740 1 -- start start 2024-05-27T08:17:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.242+0000 7f758207d740 1 Processor -- start 2024-05-27T08:17:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.243+0000 7f758207d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 0x556aaafd3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.243+0000 7f758207d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 0x556aaafd4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.244+0000 7f7573400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 0x556aaafd4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.244+0000 7f758207d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aab049110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.244+0000 7f758207d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75741be900 con 0x556aaac034d0 2024-05-27T08:17:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.244+0000 7f758207d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75741bfd70 con 0x556aaac241b0 2024-05-27T08:17:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.244+0000 7f758207d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f75741befd0 con 0x556aaabfb2f0 2024-05-27T08:17:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.245+0000 7f758207d740 10 client.0 _ll_register_callbacks cb 0x7f7560015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.246+0000 7f7573e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 0x556aaafd3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.246+0000 7f7579000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aab049110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.246+0000 7f7579000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aab049110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51416/0 (socket says 172.21.15.63:51416) 2024-05-27T08:17:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.246+0000 7f7579000640 1 -- 172.21.15.63:0/2446530516 learned_addr learned my addr 172.21.15.63:0/2446530516 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:01.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.248+0000 7f7573400640 1 -- 172.21.15.63:0/2446530516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 msgr2=0x556aaafd3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.248+0000 7f7573400640 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 0x556aaafd3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.248+0000 7f7573400640 1 -- 172.21.15.63:0/2446530516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 msgr2=0x556aab049110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.248+0000 7f7573400640 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aab049110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.248+0000 7f7573400640 1 -- 172.21.15.63:0/2446530516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aab049650 con 0x556aaac034d0 2024-05-27T08:17:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.249+0000 7f7579000640 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aab049110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:17:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.249+0000 7f7573400640 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 0x556aaafd4320 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7f75640e3c20 tx=0x7f75640e04b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd0eba726fb45a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.249+0000 7f755e000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:17:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.250+0000 7f755e000640 1 -- 172.21.15.63:0/2446530516 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640c2070 con 0x556aaac034d0 2024-05-27T08:17:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.250+0000 7f755e000640 1 -- 172.21.15.63:0/2446530516 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7f75640a9000 con 0x556aaac034d0 2024-05-27T08:17:01.252 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.251+0000 7f758207d740 1 -- 172.21.15.63:0/2446530516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75640b2340 con 0x556aaac034d0 2024-05-27T08:17:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.254+0000 7f755e000640 1 -- 172.21.15.63:0/2446530516 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f75640a96a0 con 0x556aaac034d0 2024-05-27T08:17:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.254+0000 7f758207d740 10 client.23511 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:01.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.254+0000 7f758207d740 1 -- 172.21.15.63:0/2446530516 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556aab04a750 con 0x556aaac034d0 2024-05-27T08:17:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.258+0000 7f755e000640 1 -- 172.21.15.63:0/2446530516 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75640b2340 con 0x556aaac034d0 2024-05-27T08:17:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.258+0000 7f755e000640 1 -- 172.21.15.63:0/2446530516 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f75640a5500 con 0x556aaac034d0 2024-05-27T08:17:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.259+0000 7f758207d740 10 client.23511 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.259+0000 7f758207d740 10 client.23511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.259+0000 7f758207d740 20 client.23511 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.259+0000 7f758207d740 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556aaabb81c0 0x556aab04cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.259+0000 7f758207d740 4 client.23511 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:17:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.259+0000 7f758207d740 1 -- 172.21.15.63:0/2446530516 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556aaafd1410 con 0x556aaabb81c0 2024-05-27T08:17:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.260+0000 7f7573e00640 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556aaabb81c0 0x556aab04cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.260+0000 7f755e000640 10 client.23511 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.264+0000 7f7573e00640 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556aaabb81c0 0x556aab04cf10 crc :-1 s=READY pgs=753 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b9351335b253e83 server_cookie=c9891f896cc9d976 in_seq=0 out_seq=0 2024-05-27T08:17:01.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:00 smithi086 ceph-mon[34687]: pgmap v1379: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.3 MiB/s wr, 2.36k op/s 2024-05-27T08:17:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.927+0000 7f755e000640 1 -- 172.21.15.63:0/2446530516 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556aaafd1410 con 0x556aaabb81c0 2024-05-27T08:17:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.927+0000 7f755e000640 10 client.23511 handle_command_reply: tid=0 2024-05-27T08:17:01.930 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.928+0000 7f758207d740 1 client.23511 shutdown 2024-05-27T08:17:01.932 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.931+0000 7f758207d740 1 -- 172.21.15.63:0/2446530516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 msgr2=0x556aaafd4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.931+0000 7f758207d740 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 0x556aaafd4320 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7f75640e3c20 tx=0x7f75640e04b0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.931+0000 7f758207d740 1 -- 172.21.15.63:0/2446530516 shutdown_connections 2024-05-27T08:17:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.931+0000 7f758207d740 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556aaabb81c0 0x556aab04cf10 crc :-1 s=READY pgs=753 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.931+0000 7f755e000640 0 client.23511 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.931+0000 7f758207d740 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556aaac241b0 0x556aab049110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.931+0000 7f758207d740 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556aaac034d0 0x556aaafd4320 unknown :-1 s=CLOSED pgs=2955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.931+0000 7f758207d740 1 --2- 172.21.15.63:0/2446530516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556aaabfb2f0 0x556aaafd3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.931+0000 7f758207d740 1 -- 172.21.15.63:0/2446530516 >> 172.21.15.63:0/2446530516 conn(0x556aaac18300 msgr2=0x556aaafd1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.933+0000 7f758207d740 1 -- 172.21.15.63:0/2446530516 shutdown_connections 2024-05-27T08:17:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.933+0000 7f758207d740 1 -- 172.21.15.63:0/2446530516 wait complete. 2024-05-27T08:17:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.934+0000 7f758207d740 20 client.23511 trim_cache size 0 max 16384 2024-05-27T08:17:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:17:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:17:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:17:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.936+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75580b2a20 msgr2=0x7f75580b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.936+0000 7f757b000640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75580b2a20 0x7f75580b4f30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x556aaafc3a20 tx=0x7f7560005d00 comp rx=0 tx=0).stop 2024-05-27T08:17:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.936+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757411a8e0 msgr2=0x7f75741a9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.936+0000 7f757b000640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757411a8e0 0x7f75741a9bc0 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7f756400e9f0 tx=0x7f756400eeb0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.936+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 shutdown_connections 2024-05-27T08:17:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.937+0000 7f757b000640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f75580b2a20 0x7f75580b4f30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.937+0000 7f757b000640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f757411d240 0x7f75741ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.937+0000 7f757b000640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757411a8e0 0x7f75741a9bc0 unknown :-1 s=CLOSED pgs=2954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.937+0000 7f757b000640 1 --2- 172.21.15.63:0/3433108138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f757410a1f0 0x7f75741a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.937+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 >> 172.21.15.63:0/3433108138 conn(0x7f757406e8f0 msgr2=0x7f757411acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.938+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 shutdown_connections 2024-05-27T08:17:01.940 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:01.939+0000 7f757b000640 1 -- 172.21.15.63:0/3433108138 wait complete. 2024-05-27T08:17:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.430+0000 7f1872c00700 1 -- 172.21.15.63:0/2090379718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f186c10e900 msgr2=0x7f186c110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.430+0000 7f1872c00700 1 --2- 172.21.15.63:0/2090379718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f186c10e900 0x7f186c110de0 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f186400f490 tx=0x7f186400f7a0 comp rx=0 tx=0).stop 2024-05-27T08:17:02.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.431+0000 7f1872c00700 1 -- 172.21.15.63:0/2090379718 shutdown_connections 2024-05-27T08:17:02.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.431+0000 7f1872c00700 1 --2- 172.21.15.63:0/2090379718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f186c10e900 0x7f186c110de0 unknown :-1 s=CLOSED pgs=2956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:02.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.431+0000 7f1872c00700 1 --2- 172.21.15.63:0/2090379718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c1054d0 0x7f186c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:02.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.431+0000 7f1872c00700 1 --2- 172.21.15.63:0/2090379718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c105e20 0x7f186c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:02.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.431+0000 7f1872c00700 1 -- 172.21.15.63:0/2090379718 >> 172.21.15.63:0/2090379718 conn(0x7f186c0fef40 msgr2=0x7f186c101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:02.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.431+0000 7f1872c00700 1 -- 172.21.15.63:0/2090379718 shutdown_connections 2024-05-27T08:17:02.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.431+0000 7f1872c00700 1 -- 172.21.15.63:0/2090379718 wait complete. 2024-05-27T08:17:02.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.437+0000 7f1872c00700 1 Processor -- start 2024-05-27T08:17:02.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.437+0000 7f1872c00700 1 -- start start 2024-05-27T08:17:02.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.438+0000 7f1872c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c1054d0 0x7f186c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:02.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.438+0000 7f1872c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f186c105e20 0x7f186c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:02.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.438+0000 7f1872c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c10e900 0x7f186c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.438+0000 7f1872c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1121d0 con 0x7f186c105e20 2024-05-27T08:17:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.438+0000 7f1872c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f186c112050 con 0x7f186c10e900 2024-05-27T08:17:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.438+0000 7f1872c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f186c112350 con 0x7f186c1054d0 2024-05-27T08:17:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.439+0000 7f186be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c1054d0 0x7f186c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.439+0000 7f186be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c1054d0 0x7f186c19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53868/0 (socket says 172.21.15.63:53868) 2024-05-27T08:17:02.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.439+0000 7f186be00700 1 -- 172.21.15.63:0/3808131630 learned_addr learned my addr 172.21.15.63:0/3808131630 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:02.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.439+0000 7f1870c00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c10e900 0x7f186c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:02.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.439+0000 7f186b400700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f186c105e20 0x7f186c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.439+0000 7f186be00700 1 -- 172.21.15.63:0/3808131630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c10e900 msgr2=0x7f186c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.439+0000 7f186be00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c10e900 0x7f186c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.439+0000 7f186be00700 1 -- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f186c105e20 msgr2=0x7f186c19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.440+0000 7f186be00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f186c105e20 0x7f186c19ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.440+0000 7f186be00700 1 -- 172.21.15.63:0/3808131630 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c198fe0 con 0x7f186c1054d0 2024-05-27T08:17:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.440+0000 7f186b400700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f186c105e20 0x7f186c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:17:02.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.440+0000 7f186be00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c1054d0 0x7f186c19e710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f185800ec10 tx=0x7f185800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:02.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.441+0000 7f1868c00700 1 -- 172.21.15.63:0/3808131630 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185800cd40 con 0x7f186c1054d0 2024-05-27T08:17:02.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.441+0000 7f1868c00700 1 -- 172.21.15.63:0/3808131630 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18580044b0 con 0x7f186c1054d0 2024-05-27T08:17:02.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.441+0000 7f1868c00700 1 -- 172.21.15.63:0/3808131630 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858010430 con 0x7f186c1054d0 2024-05-27T08:17:02.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.443+0000 7f1872c00700 1 -- 172.21.15.63:0/3808131630 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c1992d0 con 0x7f186c1054d0 2024-05-27T08:17:02.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.449+0000 7f1872c00700 1 -- 172.21.15.63:0/3808131630 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c1a55a0 con 0x7f186c1054d0 2024-05-27T08:17:02.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.462+0000 7f1868c00700 1 -- 172.21.15.63:0/3808131630 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1858003fa0 con 0x7f186c1054d0 2024-05-27T08:17:02.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.464+0000 7f1868c00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18500b2a70 0x7f18500b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:02.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.464+0000 7f1868c00700 1 -- 172.21.15.63:0/3808131630 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7f18580e3080 con 0x7f186c1054d0 2024-05-27T08:17:02.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.465+0000 7f186b400700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18500b2a70 0x7f18500b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:02.471+0000 7f186b400700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18500b2a70 0x7f18500b4f80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f186c1025c0 tx=0x7f185c009550 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:03.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:02 smithi063 ceph-mon[31613]: pgmap v1380: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 14 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-05-27T08:17:03.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.243+0000 7f188183fb80 1 Processor -- start 2024-05-27T08:17:03.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.243+0000 7f188183fb80 1 -- start start 2024-05-27T08:17:03.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.247+0000 7f188183fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 0x559ea8a66330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.248+0000 7f188183fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea88e7e40 0x559ea8a66870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.248+0000 7f188183fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66e70 0x559ea8a73110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.248+0000 7f188183fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559ea8994e40 con 0x559ea8a66e70 2024-05-27T08:17:03.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.248+0000 7f188183fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559ea89849e0 con 0x559ea89d39f0 2024-05-27T08:17:03.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.248+0000 7f188183fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559ea8988a20 con 0x559ea88e7e40 2024-05-27T08:17:03.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.250+0000 7f1870c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66e70 0x559ea8a73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.250+0000 7f1870c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66e70 0x559ea8a73110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47878/0 (socket says 172.21.15.63:47878) 2024-05-27T08:17:03.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.250+0000 7f1870c00700 1 -- 172.21.15.63:0/16646024 learned_addr learned my addr 172.21.15.63:0/16646024 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:03.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.251+0000 7f1870c00700 1 -- 172.21.15.63:0/16646024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea88e7e40 msgr2=0x559ea8a66870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:17:03.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.251+0000 7f1870c00700 1 --2- 172.21.15.63:0/16646024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea88e7e40 0x559ea8a66870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.251+0000 7f1870c00700 1 -- 172.21.15.63:0/16646024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 msgr2=0x559ea8a66330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:17:03.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.252+0000 7f186be00700 1 --2- 172.21.15.63:0/16646024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 0x559ea8a66330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.254+0000 7f1870c00700 1 --2- 172.21.15.63:0/16646024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 0x559ea8a66330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.254+0000 7f1870c00700 1 -- 172.21.15.63:0/16646024 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea894d420 con 0x559ea8a66e70 2024-05-27T08:17:03.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.254+0000 7f186be00700 1 --2- 172.21.15.63:0/16646024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 0x559ea8a66330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:17:03.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.256+0000 7f1870c00700 1 --2- 172.21.15.63:0/16646024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66e70 0x559ea8a73110 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7f1864009f30 tx=0x7f1864009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865e96fabe9c2948 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:03.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.257+0000 7f1861600700 1 -- 172.21.15.63:0/16646024 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864025070 con 0x559ea8a66e70 2024-05-27T08:17:03.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.258+0000 7f1861600700 1 -- 172.21.15.63:0/16646024 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1864003a00 con 0x559ea8a66e70 2024-05-27T08:17:03.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.258+0000 7f1861600700 1 -- 172.21.15.63:0/16646024 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186401c040 con 0x559ea8a66e70 2024-05-27T08:17:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.261+0000 7f188183fb80 1 -- 172.21.15.63:0/16646024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66e70 msgr2=0x559ea8a73110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:03.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.261+0000 7f188183fb80 1 --2- 172.21.15.63:0/16646024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66e70 0x559ea8a73110 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7f1864009f30 tx=0x7f1864009760 comp rx=0 tx=0).stop 2024-05-27T08:17:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.261+0000 7f188183fb80 1 -- 172.21.15.63:0/16646024 shutdown_connections 2024-05-27T08:17:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.261+0000 7f188183fb80 1 --2- 172.21.15.63:0/16646024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66e70 0x559ea8a73110 unknown :-1 s=CLOSED pgs=2957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.261+0000 7f188183fb80 1 --2- 172.21.15.63:0/16646024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 0x559ea8a66330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.261+0000 7f188183fb80 1 --2- 172.21.15.63:0/16646024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea88e7e40 0x559ea8a66870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.261+0000 7f188183fb80 1 -- 172.21.15.63:0/16646024 >> 172.21.15.63:0/16646024 conn(0x559ea8981390 msgr2=0x559ea8a624c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:03.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.266+0000 7f188183fb80 1 -- 172.21.15.63:0/16646024 shutdown_connections 2024-05-27T08:17:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.266+0000 7f188183fb80 1 -- 172.21.15.63:0/16646024 wait complete. 2024-05-27T08:17:03.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.267+0000 7f188183fb80 1 -- start start 2024-05-27T08:17:03.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.268+0000 7f188183fb80 1 Processor -- start 2024-05-27T08:17:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.271+0000 7f188183fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 0x559ea8a70650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.271+0000 7f188183fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 0x559ea8a70b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.271+0000 7f188183fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 0x559ea8a710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.271+0000 7f188183fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559ea888afa0 con 0x559ea88e7e40 2024-05-27T08:17:03.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.271+0000 7f188183fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559ea8925760 con 0x559ea89d39f0 2024-05-27T08:17:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.271+0000 7f188183fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559ea8a73b20 con 0x559ea8a66e70 2024-05-27T08:17:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.271+0000 7f188183fb80 10 client.0 _ll_register_callbacks cb 0x559ea892ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.272+0000 7f1870c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 0x559ea8a710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.272+0000 7f186b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 0x559ea8a70b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.272+0000 7f1870c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 0x559ea8a710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53896/0 (socket says 172.21.15.63:53896) 2024-05-27T08:17:03.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.272+0000 7f1870c00700 1 -- 172.21.15.63:0/3383047062 learned_addr learned my addr 172.21.15.63:0/3383047062 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:03.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.273+0000 7f186be00700 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 0x559ea8a70650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.273+0000 7f186b400700 1 -- 172.21.15.63:0/3383047062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 msgr2=0x559ea8a710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:03.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.273+0000 7f186b400700 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 0x559ea8a710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.273+0000 7f186b400700 1 -- 172.21.15.63:0/3383047062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 msgr2=0x559ea8a70650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:03.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.273+0000 7f186b400700 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 0x559ea8a70650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.273+0000 7f186b400700 1 -- 172.21.15.63:0/3383047062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea88d48d0 con 0x559ea89d39f0 2024-05-27T08:17:03.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.274+0000 7f186b400700 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 0x559ea8a70b90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x559ea8a63790 tx=0x7f185c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d68da11c7e981a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:03.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.274+0000 7f1856000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:17:03.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.275+0000 7f1856000700 1 -- 172.21.15.63:0/3383047062 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c022030 con 0x559ea89d39f0 2024-05-27T08:17:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.275+0000 7f1856000700 1 -- 172.21.15.63:0/3383047062 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7f185c004830 con 0x559ea89d39f0 2024-05-27T08:17:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.276+0000 7f188183fb80 1 -- 172.21.15.63:0/3383047062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1864003a00 con 0x559ea89d39f0 2024-05-27T08:17:03.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.276+0000 7f1856000700 1 -- 172.21.15.63:0/3383047062 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f185c004a30 con 0x559ea89d39f0 2024-05-27T08:17:03.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.277+0000 7f188183fb80 10 client.42374 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:03.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.277+0000 7f188183fb80 1 -- 172.21.15.63:0/3383047062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559ea8955060 con 0x559ea89d39f0 2024-05-27T08:17:03.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.277+0000 7f1856000700 1 -- 172.21.15.63:0/3383047062 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1864003a00 con 0x559ea89d39f0 2024-05-27T08:17:03.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.280+0000 7f1856000700 1 -- 172.21.15.63:0/3383047062 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f185c00f040 con 0x559ea89d39f0 2024-05-27T08:17:03.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.280+0000 7f188183fb80 10 client.42374 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:03.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.280+0000 7f188183fb80 10 client.42374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:03.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.280+0000 7f188183fb80 20 client.42374 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:03.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.281+0000 7f188183fb80 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559ea8a80a40 0x559ea8a82e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.281+0000 7f188183fb80 4 client.42374 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:17:03.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.281+0000 7f188183fb80 1 -- 172.21.15.63:0/3383047062 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ea8996d90 con 0x559ea8a80a40 2024-05-27T08:17:03.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.283+0000 7f186be00700 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559ea8a80a40 0x559ea8a82e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.286+0000 7f186be00700 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559ea8a80a40 0x559ea8a82e80 crc :-1 s=READY pgs=754 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693bbab6fbd15216 server_cookie=cd9507a12ffefae7 in_seq=0 out_seq=0 2024-05-27T08:17:03.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.286+0000 7f1856000700 10 client.42374 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:03.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.287+0000 7f1856000700 1 -- 172.21.15.63:0/3383047062 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559ea8996d90 con 0x559ea8a80a40 2024-05-27T08:17:03.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.287+0000 7f1856000700 10 client.42374 handle_command_reply: tid=0 2024-05-27T08:17:03.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.287+0000 7f188183fb80 1 client.42374 shutdown 2024-05-27T08:17:03.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.288+0000 7f188183fb80 1 -- 172.21.15.63:0/3383047062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 msgr2=0x559ea8a70b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:03.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.288+0000 7f188183fb80 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 0x559ea8a70b90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x559ea8a63790 tx=0x7f185c0047c0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.288+0000 7f188183fb80 1 -- 172.21.15.63:0/3383047062 shutdown_connections 2024-05-27T08:17:03.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.288+0000 7f188183fb80 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559ea8a80a40 0x559ea8a82e80 crc :-1 s=READY pgs=754 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.288+0000 7f188183fb80 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 0x559ea8a710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.288+0000 7f188183fb80 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 0x559ea8a70650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.288+0000 7f188183fb80 1 --2- 172.21.15.63:0/3383047062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea89d39f0 0x559ea8a70b90 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x559ea8a63790 tx=0x7f185c0047c0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.288+0000 7f188183fb80 1 -- 172.21.15.63:0/3383047062 >> 172.21.15.63:0/3383047062 conn(0x559ea8981390 msgr2=0x559ea8a66310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:03.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.288+0000 7f1856000700 0 client.42374 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:03.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.289+0000 7f188183fb80 1 -- 172.21.15.63:0/3383047062 shutdown_connections 2024-05-27T08:17:03.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.290+0000 7f188183fb80 1 -- 172.21.15.63:0/3383047062 wait complete. 2024-05-27T08:17:03.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.290+0000 7f188183fb80 20 client.42374 trim_cache size 0 max 16384 2024-05-27T08:17:03.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.327+0000 7f188183fb80 1 Processor -- start 2024-05-27T08:17:03.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.327+0000 7f188183fb80 1 -- start start 2024-05-27T08:17:03.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.328+0000 7f188183fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea88e7e40 0x559ea8a6f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.328+0000 7f188183fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 0x559ea8a6be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.328+0000 7f188183fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66790 0x559ea8a7cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.328+0000 7f188183fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1b0bf0 con 0x559ea8a66790 2024-05-27T08:17:03.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.328+0000 7f188183fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1ae480 con 0x559ea88e7e40 2024-05-27T08:17:03.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.328+0000 7f188183fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1ac2d0 con 0x559ea8a66e70 2024-05-27T08:17:03.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.329+0000 7f186be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea88e7e40 0x559ea8a6f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.329+0000 7f186b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 0x559ea8a6be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.330+0000 7f186b400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 0x559ea8a6be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53902/0 (socket says 172.21.15.63:53902) 2024-05-27T08:17:03.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.330+0000 7f186b400700 1 -- 172.21.15.63:0/3861936331 learned_addr learned my addr 172.21.15.63:0/3861936331 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:03.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.330+0000 7f186be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea88e7e40 0x559ea8a6f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51466/0 (socket says 172.21.15.63:51466) 2024-05-27T08:17:03.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.331+0000 7f1870c00700 1 --2- 172.21.15.63:0/3861936331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66790 0x559ea8a7cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.331+0000 7f186be00700 1 -- 172.21.15.63:0/3861936331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 msgr2=0x559ea8a6be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:03.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.331+0000 7f186be00700 1 --2- 172.21.15.63:0/3861936331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 0x559ea8a6be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.331+0000 7f186be00700 1 -- 172.21.15.63:0/3861936331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66790 msgr2=0x559ea8a7cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:03.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.331+0000 7f186be00700 1 --2- 172.21.15.63:0/3861936331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66790 0x559ea8a7cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.331+0000 7f186be00700 1 -- 172.21.15.63:0/3861936331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea8a73630 con 0x559ea88e7e40 2024-05-27T08:17:03.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.333+0000 7f186be00700 1 --2- 172.21.15.63:0/3861936331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea88e7e40 0x559ea8a6f970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18580e0400 tx=0x7f1858009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=950f8b6b0ad8725f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:03.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.335+0000 7f1860c00700 1 -- 172.21.15.63:0/3861936331 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18580aa690 con 0x559ea88e7e40 2024-05-27T08:17:03.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.335+0000 7f1860c00700 1 -- 172.21.15.63:0/3861936331 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18580a9f90 con 0x559ea88e7e40 2024-05-27T08:17:03.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.335+0000 7f1860c00700 1 -- 172.21.15.63:0/3861936331 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18580aa1f0 con 0x559ea88e7e40 2024-05-27T08:17:03.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.338+0000 7f188183fb80 1 -- 172.21.15.63:0/3861936331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea88e7e40 msgr2=0x559ea8a6f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:03.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.338+0000 7f188183fb80 1 --2- 172.21.15.63:0/3861936331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea88e7e40 0x559ea8a6f970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18580e0400 tx=0x7f1858009380 comp rx=0 tx=0).stop 2024-05-27T08:17:03.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.338+0000 7f188183fb80 1 -- 172.21.15.63:0/3861936331 shutdown_connections 2024-05-27T08:17:03.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.338+0000 7f188183fb80 1 --2- 172.21.15.63:0/3861936331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea8a66790 0x559ea8a7cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.338+0000 7f188183fb80 1 --2- 172.21.15.63:0/3861936331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea88e7e40 0x559ea8a6f970 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18580e0400 tx=0x7f1858009380 comp rx=0 tx=0).stop 2024-05-27T08:17:03.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.338+0000 7f188183fb80 1 --2- 172.21.15.63:0/3861936331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66e70 0x559ea8a6be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.339+0000 7f188183fb80 1 -- 172.21.15.63:0/3861936331 >> 172.21.15.63:0/3861936331 conn(0x559ea8981390 msgr2=0x559ea8a70610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:03.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.339+0000 7f188183fb80 1 -- 172.21.15.63:0/3861936331 shutdown_connections 2024-05-27T08:17:03.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.339+0000 7f188183fb80 1 -- 172.21.15.63:0/3861936331 wait complete. 2024-05-27T08:17:03.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.340+0000 7f188183fb80 1 -- start start 2024-05-27T08:17:03.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.341+0000 7f188183fb80 1 Processor -- start 2024-05-27T08:17:03.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.342+0000 7f188183fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 0x559ea8a79460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.342+0000 7f188183fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66790 0x559ea8a6ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.343+0000 7f188183fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea8a66e70 0x559ea8a6b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.343+0000 7f1870c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea8a66e70 0x559ea8a6b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.343+0000 7f1870c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea8a66e70 0x559ea8a6b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:51478/0 (socket says 172.21.15.63:51478) 2024-05-27T08:17:03.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.343+0000 7f1870c00700 1 -- 172.21.15.63:0/843364578 learned_addr learned my addr 172.21.15.63:0/843364578 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:03.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.344+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1ac940 con 0x559ea88e7e40 2024-05-27T08:17:03.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.344+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1addb0 con 0x559ea8a66e70 2024-05-27T08:17:03.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.344+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1ad010 con 0x559ea8a66790 2024-05-27T08:17:03.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.344+0000 7f188183fb80 10 client.0 _ll_register_callbacks cb 0x559ea892ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:03.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.344+0000 7f186b400700 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66790 0x559ea8a6ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.344+0000 7f186be00700 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 0x559ea8a79460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.345+0000 7f1870c00700 1 -- 172.21.15.63:0/843364578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66790 msgr2=0x559ea8a6ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:03.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.345+0000 7f1870c00700 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66790 0x559ea8a6ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.346+0000 7f1870c00700 1 -- 172.21.15.63:0/843364578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 msgr2=0x559ea8a79460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:03.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.346+0000 7f1870c00700 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 0x559ea8a79460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.346+0000 7f1870c00700 1 -- 172.21.15.63:0/843364578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea8a5e240 con 0x559ea8a66e70 2024-05-27T08:17:03.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.346+0000 7f186be00700 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 0x559ea8a79460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:17:03.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.347+0000 7f1856000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:17:03.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.347+0000 7f1870c00700 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea8a66e70 0x559ea8a6b290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1864009f30 tx=0x7f18640098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35da2590324887cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:03.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.348+0000 7f1856000700 1 -- 172.21.15.63:0/843364578 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864030060 con 0x559ea8a66e70 2024-05-27T08:17:03.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.348+0000 7f1856000700 1 -- 172.21.15.63:0/843364578 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 9495+0+0 (secure 0 0 0) 0x7f1864026070 con 0x559ea8a66e70 2024-05-27T08:17:03.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.349+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18580a9f90 con 0x559ea8a66e70 2024-05-27T08:17:03.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.349+0000 7f1856000700 1 -- 172.21.15.63:0/843364578 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1864030060 con 0x559ea8a66e70 2024-05-27T08:17:03.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.349+0000 7f188183fb80 10 client.42380 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:03.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.349+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559ea8a77b10 con 0x559ea8a66e70 2024-05-27T08:17:03.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.350+0000 7f1856000700 1 -- 172.21.15.63:0/843364578 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18580a9f90 con 0x559ea8a66e70 2024-05-27T08:17:03.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.350+0000 7f1856000700 1 -- 172.21.15.63:0/843364578 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f186401f070 con 0x559ea8a66e70 2024-05-27T08:17:03.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.351+0000 7f188183fb80 10 client.42380 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:03.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.351+0000 7f188183fb80 10 client.42380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:03.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.351+0000 7f188183fb80 20 client.42380 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:03.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.351+0000 7f188183fb80 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559ea8afc520 0x559ea8afe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:03.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.351+0000 7f188183fb80 4 client.42380 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:17:03.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.351+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ea8990570 con 0x559ea8afc520 2024-05-27T08:17:03.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.351+0000 7f186be00700 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559ea8afc520 0x559ea8afe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:03.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.352+0000 7f1856000700 10 client.42380 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:03.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.353+0000 7f186be00700 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559ea8afc520 0x559ea8afe940 crc :-1 s=READY pgs=755 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=987a7cbc413eebf0 server_cookie=f777ca19d50f897e in_seq=0 out_seq=0 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.353+0000 7f1856000700 1 -- 172.21.15.63:0/843364578 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ea8990570 con 0x559ea8afc520 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.353+0000 7f1856000700 10 client.42380 handle_command_reply: tid=0 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.353+0000 7f188183fb80 1 client.42380 shutdown 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.354+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea8a66e70 msgr2=0x559ea8a6b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.354+0000 7f188183fb80 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea8a66e70 0x559ea8a6b290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1864009f30 tx=0x7f18640098c0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.354+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 shutdown_connections 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.354+0000 7f188183fb80 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559ea8afc520 0x559ea8afe940 crc :-1 s=READY pgs=755 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.354+0000 7f188183fb80 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559ea8a66e70 0x559ea8a6b290 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.354+0000 7f188183fb80 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559ea88e7e40 0x559ea8a79460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.354+0000 7f188183fb80 1 --2- 172.21.15.63:0/843364578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ea8a66790 0x559ea8a6ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.354+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 >> 172.21.15.63:0/843364578 conn(0x559ea8981390 msgr2=0x559ea8a6f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:03.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.354+0000 7f1856000700 0 client.42380 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:03.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.355+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 shutdown_connections 2024-05-27T08:17:03.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.355+0000 7f188183fb80 1 -- 172.21.15.63:0/843364578 wait complete. 2024-05-27T08:17:03.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.355+0000 7f188183fb80 20 client.42380 trim_cache size 0 max 16384 2024-05-27T08:17:03.359 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:17:03.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.357+0000 7f1872c00700 1 -- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18500b2a70 msgr2=0x7f18500b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:03.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.357+0000 7f1872c00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18500b2a70 0x7f18500b4f80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f186c1025c0 tx=0x7f185c009550 comp rx=0 tx=0).stop 2024-05-27T08:17:03.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.357+0000 7f1872c00700 1 -- 172.21.15.63:0/3808131630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c1054d0 msgr2=0x7f186c19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:03.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.357+0000 7f1872c00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c1054d0 0x7f186c19e710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f185800ec10 tx=0x7f185800ef20 comp rx=0 tx=0).stop 2024-05-27T08:17:03.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.358+0000 7f1872c00700 1 -- 172.21.15.63:0/3808131630 shutdown_connections 2024-05-27T08:17:03.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.358+0000 7f1872c00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f18500b2a70 0x7f18500b4f80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.358+0000 7f1872c00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f186c10e900 0x7f186c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.358+0000 7f1872c00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f186c1054d0 0x7f186c19e710 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.358+0000 7f1872c00700 1 --2- 172.21.15.63:0/3808131630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f186c105e20 0x7f186c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:03.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.358+0000 7f1872c00700 1 -- 172.21.15.63:0/3808131630 >> 172.21.15.63:0/3808131630 conn(0x7f186c0fef40 msgr2=0x7f186c100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:03.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.359+0000 7f1872c00700 1 -- 172.21.15.63:0/3808131630 shutdown_connections 2024-05-27T08:17:03.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:03.359+0000 7f1872c00700 1 -- 172.21.15.63:0/3808131630 wait complete. 2024-05-27T08:17:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:02 smithi086 ceph-mon[34687]: pgmap v1380: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 14 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-05-27T08:17:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:02 smithi149 ceph-mon[35408]: pgmap v1380: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 14 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-05-27T08:17:05.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:04 smithi149 ceph-mon[35408]: pgmap v1381: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 14 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-05-27T08:17:05.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:04 smithi063 ceph-mon[31613]: pgmap v1381: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 14 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-05-27T08:17:05.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:04 smithi086 ceph-mon[34687]: pgmap v1381: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 14 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-05-27T08:17:07.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:06 smithi149 ceph-mon[35408]: pgmap v1382: 129 pgs: 5 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 119 active+clean; 14 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 2.88k op/s 2024-05-27T08:17:07.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:06 smithi063 ceph-mon[31613]: pgmap v1382: 129 pgs: 5 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 119 active+clean; 14 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 2.88k op/s 2024-05-27T08:17:07.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:06 smithi086 ceph-mon[34687]: pgmap v1382: 129 pgs: 5 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 119 active+clean; 14 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 2.88k op/s 2024-05-27T08:17:09.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:08 smithi149 ceph-mon[35408]: pgmap v1383: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 2.81k op/s 2024-05-27T08:17:09.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:08 smithi063 ceph-mon[31613]: pgmap v1383: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 2.81k op/s 2024-05-27T08:17:09.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:08 smithi086 ceph-mon[34687]: pgmap v1383: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 2.81k op/s 2024-05-27T08:17:11.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:10 smithi149 ceph-mon[35408]: pgmap v1384: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-27T08:17:11.215 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:10 smithi063 ceph-mon[31613]: pgmap v1384: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-27T08:17:11.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:10 smithi086 ceph-mon[34687]: pgmap v1384: 129 pgs: 4 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 120 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.4 MiB/s wr, 2.39k op/s 2024-05-27T08:17:12.983 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:17:12.984 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:17:13.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:12 smithi063 ceph-mon[31613]: osdmap e167: 12 total, 12 up, 12 in 2024-05-27T08:17:13.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:12 smithi063 ceph-mon[31613]: pgmap v1386: 129 pgs: 7 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 96 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-05-27T08:17:13.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:12 smithi086 ceph-mon[34687]: osdmap e167: 12 total, 12 up, 12 in 2024-05-27T08:17:13.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:13 smithi086 ceph-mon[34687]: pgmap v1386: 129 pgs: 7 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 96 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-05-27T08:17:13.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:12 smithi149 ceph-mon[35408]: osdmap e167: 12 total, 12 up, 12 in 2024-05-27T08:17:13.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:12 smithi149 ceph-mon[35408]: pgmap v1386: 129 pgs: 7 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 96 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-05-27T08:17:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:17:14.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:17:14.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:17:14.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:17:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.411+0000 7f5ee7200640 1 -- 172.21.15.63:0/3376210397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee810a730 msgr2=0x7f5ee810ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.411+0000 7f5ee7200640 1 --2- 172.21.15.63:0/3376210397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee810a730 0x7f5ee810ab40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009ca0 tx=0x7f5ee002a8d0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.413+0000 7f5ee7200640 1 -- 172.21.15.63:0/3376210397 shutdown_connections 2024-05-27T08:17:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.413+0000 7f5ee7200640 1 --2- 172.21.15.63:0/3376210397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee810f780 0x7f5ee8116f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.413+0000 7f5ee7200640 1 --2- 172.21.15.63:0/3376210397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ee810ecc0 0x7f5ee810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.413+0000 7f5ee7200640 1 --2- 172.21.15.63:0/3376210397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee810a730 0x7f5ee810ab40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.413+0000 7f5ee7200640 1 -- 172.21.15.63:0/3376210397 >> 172.21.15.63:0/3376210397 conn(0x7f5ee806fb90 msgr2=0x7f5ee806ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.413+0000 7f5ee7200640 1 -- 172.21.15.63:0/3376210397 shutdown_connections 2024-05-27T08:17:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.413+0000 7f5ee7200640 1 -- 172.21.15.63:0/3376210397 wait complete. 2024-05-27T08:17:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.416+0000 7f5ee7200640 1 Processor -- start 2024-05-27T08:17:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.417+0000 7f5ee7200640 1 -- start start 2024-05-27T08:17:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.417+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee810a730 0x7f5ee81a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.417+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ee810ecc0 0x7f5ee81a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.417+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee810f780 0x7f5ee81a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.418+0000 7f5ee5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee810a730 0x7f5ee81a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.418+0000 7f5ee5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee810a730 0x7f5ee81a6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:36006/0 (socket says 172.21.15.63:36006) 2024-05-27T08:17:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.418+0000 7f5ee5e00640 1 -- 172.21.15.63:0/3867585313 learned_addr learned my addr 172.21.15.63:0/3867585313 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.418+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee811a150 con 0x7f5ee810ecc0 2024-05-27T08:17:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.418+0000 7f5ee5400640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ee810ecc0 0x7f5ee81a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.418+0000 7f5ee6800640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee810f780 0x7f5ee81a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.419+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8119fd0 con 0x7f5ee810a730 2024-05-27T08:17:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.419+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee811a2d0 con 0x7f5ee810f780 2024-05-27T08:17:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.420+0000 7f5ee6800640 1 -- 172.21.15.63:0/3867585313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee810a730 msgr2=0x7f5ee81a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.420+0000 7f5ee6800640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee810a730 0x7f5ee81a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.420+0000 7f5ee6800640 1 -- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ee810ecc0 msgr2=0x7f5ee81a6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.421+0000 7f5ee6800640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ee810ecc0 0x7f5ee81a6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.421+0000 7f5ee6800640 1 -- 172.21.15.63:0/3867585313 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee81a10b0 con 0x7f5ee810f780 2024-05-27T08:17:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.421+0000 7f5ee5400640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ee810ecc0 0x7f5ee81a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:17:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.421+0000 7f5ee6800640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee810f780 0x7f5ee81a0a10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800e970 tx=0x7f5ed800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.422+0000 7f5edea00640 1 -- 172.21.15.63:0/3867585313 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed800ccc0 con 0x7f5ee810f780 2024-05-27T08:17:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.423+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee81a13a0 con 0x7f5ee810f780 2024-05-27T08:17:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.423+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee81ad680 con 0x7f5ee810f780 2024-05-27T08:17:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.423+0000 7f5edea00640 1 -- 172.21.15.63:0/3867585313 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ed8005020 con 0x7f5ee810f780 2024-05-27T08:17:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.423+0000 7f5edea00640 1 -- 172.21.15.63:0/3867585313 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed800ce60 con 0x7f5ee810f780 2024-05-27T08:17:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.431+0000 7f5ef3c33740 1 Processor -- start 2024-05-27T08:17:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.432+0000 7f5edea00640 1 -- 172.21.15.63:0/3867585313 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5ed8004a00 con 0x7f5ee810f780 2024-05-27T08:17:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.435+0000 7f5ef3c33740 1 -- start start 2024-05-27T08:17:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.436+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88f21b0 0x5605f8c94980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.436+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c94ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.436+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88d14d0 0x5605f8c95400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.436+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5605f8c05b90 con 0x5605f88d14d0 2024-05-27T08:17:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.436+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5605f8bf5730 con 0x5605f88f21b0 2024-05-27T08:17:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.436+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5605f8bf9770 con 0x5605f88c92f0 2024-05-27T08:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.438+0000 7f5ee5e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c94ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.438+0000 7f5ee5e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c94ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41322/0 (socket says 172.21.15.63:41322) 2024-05-27T08:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.438+0000 7f5ee5e00640 1 -- 172.21.15.63:0/2284701279 learned_addr learned my addr 172.21.15.63:0/2284701279 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.438+0000 7f5edea00640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ecc0b29e0 0x7f5ecc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.438+0000 7f5ee5e00640 1 -- 172.21.15.63:0/2284701279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88f21b0 msgr2=0x5605f8c94980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.438+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/2284701279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88f21b0 0x5605f8c94980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.438+0000 7f5ee5e00640 1 -- 172.21.15.63:0/2284701279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88d14d0 msgr2=0x5605f8c95400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.439+0000 7f5ee5400640 1 --2- 172.21.15.63:0/2284701279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88d14d0 0x5605f8c95400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.439+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/2284701279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88d14d0 0x5605f8c95400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.439+0000 7f5ee5e00640 1 -- 172.21.15.63:0/2284701279 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f8b8e8b0 con 0x5605f88c92f0 2024-05-27T08:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.440+0000 7f5edea00640 1 -- 172.21.15.63:0/3867585313 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 9619+0+0 (secure 0 0 0) 0x7f5ed80e29b0 con 0x7f5ee810f780 2024-05-27T08:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.440+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/2284701279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c94ec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00097b0 tx=0x7f5ee0009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af2894e265edb1d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.441+0000 7f5ee6800640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ecc0b29e0 0x7f5ecc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.441+0000 7f5ed3000640 1 -- 172.21.15.63:0/2284701279 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00307b0 con 0x5605f88c92f0 2024-05-27T08:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.441+0000 7f5ed3000640 1 -- 172.21.15.63:0/2284701279 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ee0036040 con 0x5605f88c92f0 2024-05-27T08:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.441+0000 7f5ed3000640 1 -- 172.21.15.63:0/2284701279 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00043c0 con 0x5605f88c92f0 2024-05-27T08:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.443+0000 7f5ef3c33740 1 -- 172.21.15.63:0/2284701279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 msgr2=0x5605f8c94ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.443+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/2284701279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c94ec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00097b0 tx=0x7f5ee0009890 comp rx=0 tx=0).stop 2024-05-27T08:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.443+0000 7f5ef3c33740 1 -- 172.21.15.63:0/2284701279 shutdown_connections 2024-05-27T08:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.443+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/2284701279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88d14d0 0x5605f8c95400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.443+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/2284701279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c94ec0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.443+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/2284701279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88f21b0 0x5605f8c94980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.444+0000 7f5ef3c33740 1 -- 172.21.15.63:0/2284701279 >> 172.21.15.63:0/2284701279 conn(0x5605f88e6300 msgr2=0x5605f8c91240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.444+0000 7f5ee6800640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ecc0b29e0 0x7f5ecc0b4ef0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8004480 tx=0x7f5ed80a5e50 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.444+0000 7f5ef3c33740 1 -- 172.21.15.63:0/2284701279 shutdown_connections 2024-05-27T08:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.445+0000 7f5ef3c33740 1 -- 172.21.15.63:0/2284701279 wait complete. 2024-05-27T08:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.445+0000 7f5ef3c33740 1 -- start start 2024-05-27T08:17:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.446+0000 7f5ef3c33740 1 Processor -- start 2024-05-27T08:17:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.446+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8ca3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.446+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88d14d0 0x5605f8ca9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.446+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 0x5605f8ca4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.446+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5605f88db790 con 0x5605f88f21b0 2024-05-27T08:17:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.446+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5605f8b9dbc0 con 0x5605f88d14d0 2024-05-27T08:17:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.446+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5605f8c97af0 con 0x5605f88c92f0 2024-05-27T08:17:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.447+0000 7f5ef3c33740 10 client.0 _ll_register_callbacks cb 0x5605f8b7b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.447+0000 7f5ee5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88d14d0 0x5605f8ca9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.447+0000 7f5ee5400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8ca3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.448+0000 7f5ee5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88d14d0 0x5605f8ca9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:36020/0 (socket says 172.21.15.63:36020) 2024-05-27T08:17:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.448+0000 7f5ee5e00640 1 -- 172.21.15.63:0/1083550616 learned_addr learned my addr 172.21.15.63:0/1083550616 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.448+0000 7f5ee5400640 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 0x5605f8ca4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.449+0000 7f5ee5e00640 1 -- 172.21.15.63:0/1083550616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 msgr2=0x5605f8ca3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.449+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8ca3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.449+0000 7f5ee5e00640 1 -- 172.21.15.63:0/1083550616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 msgr2=0x5605f8ca4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.449+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 0x5605f8ca4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.449+0000 7f5ee5e00640 1 -- 172.21.15.63:0/1083550616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f890a210 con 0x5605f88d14d0 2024-05-27T08:17:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.449+0000 7f5ee5400640 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 0x5605f8ca4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:17:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.450+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88d14d0 0x5605f8ca9b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0004af0 tx=0x7f5ee0030420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41def0481f834a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.451+0000 7f5ec7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:17:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.451+0000 7f5ec7400640 1 -- 172.21.15.63:0/1083550616 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0003970 con 0x5605f88d14d0 2024-05-27T08:17:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.451+0000 7f5ec7400640 1 -- 172.21.15.63:0/1083550616 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(167..167 src has 1..167) ==== 9619+0+0 (secure 0 0 0) 0x7f5ee0003b10 con 0x5605f88d14d0 2024-05-27T08:17:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.451+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1083550616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee0036040 con 0x5605f88d14d0 2024-05-27T08:17:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.454+0000 7f5ec7400640 1 -- 172.21.15.63:0/1083550616 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5ee005c050 con 0x5605f88d14d0 2024-05-27T08:17:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.454+0000 7f5ef3c33740 10 client.42389 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.454+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1083550616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5605f89d6620 con 0x5605f88d14d0 2024-05-27T08:17:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.455+0000 7f5ec7400640 1 -- 172.21.15.63:0/1083550616 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ee0036040 con 0x5605f88d14d0 2024-05-27T08:17:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.455+0000 7f5ec7400640 1 -- 172.21.15.63:0/1083550616 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5ee0004450 con 0x5605f88d14d0 2024-05-27T08:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.456+0000 7f5ef3c33740 10 client.42389 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.456+0000 7f5ef3c33740 10 client.42389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.456+0000 7f5ef3c33740 20 client.42389 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.456+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5605f88b0d80 0x5605f8cc1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.456+0000 7f5ef3c33740 4 client.42389 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.456+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1083550616 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605f8c07ae0 con 0x5605f88b0d80 2024-05-27T08:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.459+0000 7f5ee5400640 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5605f88b0d80 0x5605f8cc1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.460+0000 7f5ee5400640 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5605f88b0d80 0x5605f8cc1e20 crc :-1 s=READY pgs=756 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d76cd4c4f98a3f server_cookie=c7d8f1b5c8691c72 in_seq=0 out_seq=0 2024-05-27T08:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.461+0000 7f5ec7400640 10 client.42389 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.463+0000 7f5ec7400640 1 -- 172.21.15.63:0/1083550616 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5605f8c07ae0 con 0x5605f88b0d80 2024-05-27T08:17:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.463+0000 7f5ec7400640 10 client.42389 handle_command_reply: tid=0 2024-05-27T08:17:15.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.463+0000 7f5ef3c33740 1 client.42389 shutdown 2024-05-27T08:17:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.464+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1083550616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88d14d0 msgr2=0x5605f8ca9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.464+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88d14d0 0x5605f8ca9b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0004af0 tx=0x7f5ee0030420 comp rx=0 tx=0).stop 2024-05-27T08:17:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.464+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1083550616 shutdown_connections 2024-05-27T08:17:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.464+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5605f88b0d80 0x5605f8cc1e20 crc :-1 s=READY pgs=756 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.464+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 0x5605f8ca4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.464+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88d14d0 0x5605f8ca9b90 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0004af0 tx=0x7f5ee0030420 comp rx=0 tx=0).stop 2024-05-27T08:17:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.464+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1083550616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8ca3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.464+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1083550616 >> 172.21.15.63:0/1083550616 conn(0x5605f88e6300 msgr2=0x5605f8c96740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.464+0000 7f5ec7400640 0 client.42389 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.465+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1083550616 shutdown_connections 2024-05-27T08:17:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.466+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1083550616 wait complete. 2024-05-27T08:17:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.466+0000 7f5ef3c33740 20 client.42389 trim_cache size 0 max 16384 2024-05-27T08:17:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.493+0000 7f5ef3c33740 1 Processor -- start 2024-05-27T08:17:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.493+0000 7f5ef3c33740 1 -- start start 2024-05-27T08:17:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.494+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c9aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.494+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88f21b0 0x5605f8ca9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88861c0 0x5605f8ca9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec80129c0 con 0x5605f88861c0 2024-05-27T08:17:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec8010290 con 0x5605f88f21b0 2024-05-27T08:17:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec800e100 con 0x5605f88c92f0 2024-05-27T08:17:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ee5400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c9aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ee5400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c9aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41348/0 (socket says 172.21.15.63:41348) 2024-05-27T08:17:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ee5400640 1 -- 172.21.15.63:0/3734113762 learned_addr learned my addr 172.21.15.63:0/3734113762 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/3734113762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88f21b0 0x5605f8ca9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ee5e00640 1 -- 172.21.15.63:0/3734113762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 msgr2=0x5605f8c9aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/3734113762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c9aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ee5e00640 1 -- 172.21.15.63:0/3734113762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88861c0 msgr2=0x5605f8ca9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/3734113762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88861c0 0x5605f8ca9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.495+0000 7f5ee5e00640 1 -- 172.21.15.63:0/3734113762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f8ca0360 con 0x5605f88f21b0 2024-05-27T08:17:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.497+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/3734113762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88f21b0 0x5605f8ca9540 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0032000 tx=0x7f5ee002a8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1300652f31e4a1ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.498+0000 7f5ed2600640 1 -- 172.21.15.63:0/3734113762 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0030eb0 con 0x5605f88f21b0 2024-05-27T08:17:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.498+0000 7f5ed2600640 1 -- 172.21.15.63:0/3734113762 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ee0036070 con 0x5605f88f21b0 2024-05-27T08:17:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.500+0000 7f5ef3c33740 1 -- 172.21.15.63:0/3734113762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88f21b0 msgr2=0x5605f8ca9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.500+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/3734113762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88f21b0 0x5605f8ca9540 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0032000 tx=0x7f5ee002a8d0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.498+0000 7f5ed2600640 1 -- 172.21.15.63:0/3734113762 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 0 0 0) 0x7f5ee0003bd0 con 0x5605f88f21b0 2024-05-27T08:17:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.500+0000 7f5ef3c33740 1 -- 172.21.15.63:0/3734113762 shutdown_connections 2024-05-27T08:17:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.500+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/3734113762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88861c0 0x5605f8ca9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.500+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/3734113762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88f21b0 0x5605f8ca9540 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.500+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/3734113762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88c92f0 0x5605f8c9aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.500+0000 7f5ef3c33740 1 -- 172.21.15.63:0/3734113762 >> 172.21.15.63:0/3734113762 conn(0x5605f88e6300 msgr2=0x5605f8c90240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.500+0000 7f5ef3c33740 1 -- 172.21.15.63:0/3734113762 shutdown_connections 2024-05-27T08:17:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.501+0000 7f5ef3c33740 1 -- 172.21.15.63:0/3734113762 wait complete. 2024-05-27T08:17:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.501+0000 7f5ef3c33740 1 -- start start 2024-05-27T08:17:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.502+0000 7f5ef3c33740 1 Processor -- start 2024-05-27T08:17:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.502+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88861c0 0x5605f8cc0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.502+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88c92f0 0x5605f8cc09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.502+0000 7f5ef3c33740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 0x5605f8d28250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.502+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec800e750 con 0x5605f88f21b0 2024-05-27T08:17:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.502+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec800fbc0 con 0x5605f88c92f0 2024-05-27T08:17:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.502+0000 7f5ef3c33740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec800ee20 con 0x5605f88861c0 2024-05-27T08:17:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.502+0000 7f5ef3c33740 10 client.0 _ll_register_callbacks cb 0x7f5ed8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.503+0000 7f5ee5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88c92f0 0x5605f8cc09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.503+0000 7f5ee5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88c92f0 0x5605f8cc09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:36036/0 (socket says 172.21.15.63:36036) 2024-05-27T08:17:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.503+0000 7f5ee5e00640 1 -- 172.21.15.63:0/1372728581 learned_addr learned my addr 172.21.15.63:0/1372728581 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:15.505 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.504+0000 7f5ee5400640 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88861c0 0x5605f8cc0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.504+0000 7f5ee5400640 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 0x5605f8d28250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.504+0000 7f5ee5e00640 1 -- 172.21.15.63:0/1372728581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88861c0 msgr2=0x5605f8cc0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.505+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88861c0 0x5605f8cc0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.505+0000 7f5ee5e00640 1 -- 172.21.15.63:0/1372728581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 msgr2=0x5605f8d28250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.505+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 0x5605f8d28250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.505+0000 7f5ee5e00640 1 -- 172.21.15.63:0/1372728581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f8d28790 con 0x5605f88c92f0 2024-05-27T08:17:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.505+0000 7f5ee5400640 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 0x5605f8d28250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:17:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.506+0000 7f5ee5e00640 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88c92f0 0x5605f8cc09b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00049d0 tx=0x7f5ee00309e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6356d4caa5fed02d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.506+0000 7f5ec7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:17:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.510+0000 7f5ec7400640 1 -- 172.21.15.63:0/1372728581 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0064070 con 0x5605f88c92f0 2024-05-27T08:17:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.510+0000 7f5ec7400640 1 -- 172.21.15.63:0/1372728581 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(167..167 src has 1..167) ==== 9619+0+0 (secure 0 0 0) 0x7f5ee0031690 con 0x5605f88c92f0 2024-05-27T08:17:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.510+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1372728581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee0036070 con 0x5605f88c92f0 2024-05-27T08:17:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.511+0000 7f5ec7400640 1 -- 172.21.15.63:0/1372728581 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5ee0030420 con 0x5605f88c92f0 2024-05-27T08:17:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.511+0000 7f5ef3c33740 10 client.42395 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.511+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1372728581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5605f8d29980 con 0x5605f88c92f0 2024-05-27T08:17:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.512+0000 7f5ec7400640 1 -- 172.21.15.63:0/1372728581 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ee0036070 con 0x5605f88c92f0 2024-05-27T08:17:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.513+0000 7f5ec7400640 1 -- 172.21.15.63:0/1372728581 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5ee0005040 con 0x5605f88c92f0 2024-05-27T08:17:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.514+0000 7f5ef3c33740 10 client.42395 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.514+0000 7f5ef3c33740 10 client.42395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.514+0000 7f5ef3c33740 20 client.42395 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.514+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5605f8d2a030 0x5605f8d2c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.514+0000 7f5ef3c33740 4 client.42395 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:17:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.514+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1372728581 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605f8c012c0 con 0x5605f8d2a030 2024-05-27T08:17:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.515+0000 7f5ee5400640 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5605f8d2a030 0x5605f8d2c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.516+0000 7f5ee5400640 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5605f8d2a030 0x5605f8d2c450 crc :-1 s=READY pgs=757 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e828055f9677895 server_cookie=bca13066025efe29 in_seq=0 out_seq=0 2024-05-27T08:17:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.516+0000 7f5ec7400640 10 client.42395 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.879+0000 7f5ec7400640 1 -- 172.21.15.63:0/1372728581 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5605f8c012c0 con 0x5605f8d2a030 2024-05-27T08:17:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.879+0000 7f5ec7400640 10 client.42395 handle_command_reply: tid=0 2024-05-27T08:17:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.879+0000 7f5ef3c33740 1 client.42395 shutdown 2024-05-27T08:17:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.879+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1372728581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88c92f0 msgr2=0x5605f8cc09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.879+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88c92f0 0x5605f8cc09b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00049d0 tx=0x7f5ee00309e0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.880+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1372728581 shutdown_connections 2024-05-27T08:17:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.880+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5605f8d2a030 0x5605f8d2c450 crc :-1 s=READY pgs=757 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.880+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5605f88f21b0 0x5605f8d28250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.880+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5605f88c92f0 0x5605f8cc09b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.880+0000 7f5ef3c33740 1 --2- 172.21.15.63:0/1372728581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5605f88861c0 0x5605f8cc0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.880+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1372728581 >> 172.21.15.63:0/1372728581 conn(0x5605f88e6300 msgr2=0x5605f8c9db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.880+0000 7f5ec7400640 0 client.42395 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.880+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1372728581 shutdown_connections 2024-05-27T08:17:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.881+0000 7f5ef3c33740 1 -- 172.21.15.63:0/1372728581 wait complete. 2024-05-27T08:17:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.881+0000 7f5ef3c33740 20 client.42395 trim_cache size 0 max 16384 2024-05-27T08:17:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:17:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:17:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.882+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ecc0b29e0 msgr2=0x7f5ecc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.882+0000 7f5ee7200640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ecc0b29e0 0x7f5ecc0b4ef0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8004480 tx=0x7f5ed80a5e50 comp rx=0 tx=0).stop 2024-05-27T08:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.882+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee810f780 msgr2=0x7f5ee81a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.882+0000 7f5ee7200640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee810f780 0x7f5ee81a0a10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800e970 tx=0x7f5ed800ee30 comp rx=0 tx=0).stop 2024-05-27T08:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.883+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 shutdown_connections 2024-05-27T08:17:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.883+0000 7f5ee7200640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5ecc0b29e0 0x7f5ecc0b4ef0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.883+0000 7f5ee7200640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee810f780 0x7f5ee81a0a10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.883+0000 7f5ee7200640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ee810ecc0 0x7f5ee81a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.883+0000 7f5ee7200640 1 --2- 172.21.15.63:0/3867585313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee810a730 0x7f5ee81a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.883+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 >> 172.21.15.63:0/3867585313 conn(0x7f5ee806fb90 msgr2=0x7f5ee810dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.884+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 shutdown_connections 2024-05-27T08:17:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:15.884+0000 7f5ee7200640 1 -- 172.21.15.63:0/3867585313 wait complete. 2024-05-27T08:17:15.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:15 smithi086 ceph-mon[34687]: pgmap v1387: 129 pgs: 9 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-05-27T08:17:15.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:15 smithi149 ceph-mon[35408]: pgmap v1387: 129 pgs: 9 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-05-27T08:17:15.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:15 smithi063 ceph-mon[31613]: pgmap v1387: 129 pgs: 9 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 90 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-05-27T08:17:17.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:16 smithi086 ceph-mon[34687]: pgmap v1388: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.8 MiB/s wr, 2.21k op/s 2024-05-27T08:17:17.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:16 smithi149 ceph-mon[35408]: pgmap v1388: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.8 MiB/s wr, 2.21k op/s 2024-05-27T08:17:17.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:16 smithi063 ceph-mon[31613]: pgmap v1388: 129 pgs: 15 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 78 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.8 MiB/s wr, 2.21k op/s 2024-05-27T08:17:18.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:18 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:17:18.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:18 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:17:18.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:18 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:17:18.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:18 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:17:18.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:18 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:17:18.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:18 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:17:18.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:18 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:17:18.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:18 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:17:18.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:18 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:17:19.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:19 smithi086 ceph-mon[34687]: pgmap v1389: 129 pgs: 11 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 102 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.1 MiB/s wr, 2.25k op/s 2024-05-27T08:17:19.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:19 smithi149 ceph-mon[35408]: pgmap v1389: 129 pgs: 11 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 102 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.1 MiB/s wr, 2.25k op/s 2024-05-27T08:17:19.465 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:19 smithi063 ceph-mon[31613]: pgmap v1389: 129 pgs: 11 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 102 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.1 MiB/s wr, 2.25k op/s 2024-05-27T08:17:21.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:20 smithi063 ceph-mon[31613]: pgmap v1390: 129 pgs: 10 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 103 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.0 MiB/s wr, 2.23k op/s 2024-05-27T08:17:21.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:20 smithi086 ceph-mon[34687]: pgmap v1390: 129 pgs: 10 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 103 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.0 MiB/s wr, 2.23k op/s 2024-05-27T08:17:21.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:20 smithi149 ceph-mon[35408]: pgmap v1390: 129 pgs: 10 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 103 active+clean; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 9.0 MiB/s wr, 2.23k op/s 2024-05-27T08:17:23.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:22 smithi086 ceph-mon[34687]: pgmap v1391: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.3 MiB/s wr, 2.28k op/s 2024-05-27T08:17:23.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:22 smithi149 ceph-mon[35408]: pgmap v1391: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.3 MiB/s wr, 2.28k op/s 2024-05-27T08:17:23.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:22 smithi063 ceph-mon[31613]: pgmap v1391: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.3 MiB/s wr, 2.28k op/s 2024-05-27T08:17:24.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.162+0000 7fa517200700 1 -- 172.21.15.63:0/445014904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa50c008770 msgr2=0x7fa50c0a7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:24.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.162+0000 7fa517200700 1 --2- 172.21.15.63:0/445014904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa50c008770 0x7fa50c0a7140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa500009990 tx=0x7fa500009ca0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.163+0000 7fa517200700 1 -- 172.21.15.63:0/445014904 shutdown_connections 2024-05-27T08:17:24.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.163+0000 7fa517200700 1 --2- 172.21.15.63:0/445014904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa50c0a7680 0x7fa50c0a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.163+0000 7fa517200700 1 --2- 172.21.15.63:0/445014904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa50c007e20 0x7fa50c008230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.163+0000 7fa517200700 1 --2- 172.21.15.63:0/445014904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa50c008770 0x7fa50c0a7140 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.163+0000 7fa517200700 1 -- 172.21.15.63:0/445014904 >> 172.21.15.63:0/445014904 conn(0x7fa50c00c080 msgr2=0x7fa50c00c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.164+0000 7fa517200700 1 -- 172.21.15.63:0/445014904 shutdown_connections 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.164+0000 7fa517200700 1 -- 172.21.15.63:0/445014904 wait complete. 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.166+0000 7fa517200700 1 Processor -- start 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.166+0000 7fa517200700 1 -- start start 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.167+0000 7fa517200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa50c007e20 0x7fa50c133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.167+0000 7fa517200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa50c008770 0x7fa50c133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.167+0000 7fa517200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa50c0a7680 0x7fa50c137e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.167+0000 7fa517200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0acc60 con 0x7fa50c007e20 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.167+0000 7fa517200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0acae0 con 0x7fa50c0a7680 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.167+0000 7fa517200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0acde0 con 0x7fa50c008770 2024-05-27T08:17:24.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.168+0000 7fa515e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa50c007e20 0x7fa50c133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.168+0000 7fa515400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa50c008770 0x7fa50c133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.168+0000 7fa515e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa50c007e20 0x7fa50c133650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41574/0 (socket says 172.21.15.63:41574) 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.168+0000 7fa515e00700 1 -- 172.21.15.63:0/2974006829 learned_addr learned my addr 172.21.15.63:0/2974006829 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.168+0000 7fa516800700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa50c0a7680 0x7fa50c137e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.169+0000 7fa515e00700 1 -- 172.21.15.63:0/2974006829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa50c008770 msgr2=0x7fa50c133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.169+0000 7fa515e00700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa50c008770 0x7fa50c133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.169+0000 7fa515e00700 1 -- 172.21.15.63:0/2974006829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa50c0a7680 msgr2=0x7fa50c137e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.169+0000 7fa515e00700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa50c0a7680 0x7fa50c137e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.169+0000 7fa515e00700 1 -- 172.21.15.63:0/2974006829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c138520 con 0x7fa50c007e20 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.170+0000 7fa515e00700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa50c007e20 0x7fa50c133650 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7fa50400d950 tx=0x7fa50400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.170+0000 7fa50aa00700 1 -- 172.21.15.63:0/2974006829 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504016070 con 0x7fa50c007e20 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.170+0000 7fa517200700 1 -- 172.21.15.63:0/2974006829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c138810 con 0x7fa50c007e20 2024-05-27T08:17:24.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.170+0000 7fa517200700 1 -- 172.21.15.63:0/2974006829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c140130 con 0x7fa50c007e20 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.171+0000 7fa50aa00700 1 -- 172.21.15.63:0/2974006829 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa50400fd20 con 0x7fa50c007e20 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.171+0000 7fa50aa00700 1 -- 172.21.15.63:0/2974006829 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50400e3b0 con 0x7fa50c007e20 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.183+0000 7fa50aa00700 1 -- 172.21.15.63:0/2974006829 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa504019070 con 0x7fa50c007e20 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.186+0000 7fa50aa00700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b2b20 0x7fa4f80b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.187+0000 7fa50aa00700 1 -- 172.21.15.63:0/2974006829 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 9619+0+0 (secure 0 0 0) 0x7fa5040e2800 con 0x7fa50c007e20 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.188+0000 7fa515400700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b2b20 0x7fa4f80b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.189+0000 7fa515400700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b2b20 0x7fa4f80b5030 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa500005b40 tx=0x7fa500005ab0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.980+0000 7fa527421b80 1 Processor -- start 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.981+0000 7fa527421b80 1 -- start start 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.982+0000 7fa527421b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee31206a70 0x55ee3129a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.982+0000 7fa527421b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee3111aec0 0x55ee3129ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.982+0000 7fa527421b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee312a4340 0x55ee312a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.982+0000 7fa527421b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ee311c7ec0 con 0x55ee31206a70 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.982+0000 7fa527421b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee311b7a60 con 0x55ee312a4340 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.982+0000 7fa527421b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ee311bbaa0 con 0x55ee3111aec0 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.982+0000 7fa515e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee31206a70 0x55ee3129a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.982+0000 7fa515e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee31206a70 0x55ee3129a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41586/0 (socket says 172.21.15.63:41586) 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.983+0000 7fa515e00700 1 -- 172.21.15.63:0/3168462428 learned_addr learned my addr 172.21.15.63:0/3168462428 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.983+0000 7fa515e00700 1 -- 172.21.15.63:0/3168462428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee3111aec0 msgr2=0x55ee3129ad10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.983+0000 7fa515e00700 1 --2- 172.21.15.63:0/3168462428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee3111aec0 0x55ee3129ad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.983+0000 7fa515e00700 1 -- 172.21.15.63:0/3168462428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee312a4340 msgr2=0x55ee312a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.983+0000 7fa515e00700 1 --2- 172.21.15.63:0/3168462428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee312a4340 0x55ee312a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.983+0000 7fa515e00700 1 -- 172.21.15.63:0/3168462428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee311804a0 con 0x55ee31206a70 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.983+0000 7fa515e00700 1 --2- 172.21.15.63:0/3168462428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee31206a70 0x55ee3129a7d0 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7fa5040a56b0 tx=0x7fa504010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0ef75b1ae8c75d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:24.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.984+0000 7fa4ff400700 1 -- 172.21.15.63:0/3168462428 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504020070 con 0x55ee31206a70 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.984+0000 7fa4ff400700 1 -- 172.21.15.63:0/3168462428 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5040a9630 con 0x55ee31206a70 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.984+0000 7fa4ff400700 1 -- 172.21.15.63:0/3168462428 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5040a9af0 con 0x55ee31206a70 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.986+0000 7fa527421b80 1 -- 172.21.15.63:0/3168462428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee31206a70 msgr2=0x55ee3129a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.986+0000 7fa527421b80 1 --2- 172.21.15.63:0/3168462428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee31206a70 0x55ee3129a7d0 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7fa5040a56b0 tx=0x7fa504010000 comp rx=0 tx=0).stop 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.987+0000 7fa527421b80 1 -- 172.21.15.63:0/3168462428 shutdown_connections 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.987+0000 7fa527421b80 1 --2- 172.21.15.63:0/3168462428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee312a4340 0x55ee312a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.987+0000 7fa527421b80 1 --2- 172.21.15.63:0/3168462428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee31206a70 0x55ee3129a7d0 unknown :-1 s=CLOSED pgs=2960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.988+0000 7fa527421b80 1 --2- 172.21.15.63:0/3168462428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee3111aec0 0x55ee3129ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.988+0000 7fa527421b80 1 -- 172.21.15.63:0/3168462428 >> 172.21.15.63:0/3168462428 conn(0x55ee311b4410 msgr2=0x55ee31295790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.988+0000 7fa527421b80 1 -- 172.21.15.63:0/3168462428 shutdown_connections 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.990+0000 7fa527421b80 1 -- 172.21.15.63:0/3168462428 wait complete. 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.992+0000 7fa527421b80 1 -- start start 2024-05-27T08:17:24.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.992+0000 7fa527421b80 1 Processor -- start 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.993+0000 7fa527421b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 0x55ee312a1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.993+0000 7fa527421b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee31206a70 0x55ee312a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.993+0000 7fa527421b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.995+0000 7fa515400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee31206a70 0x55ee312a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.995+0000 7fa515400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee31206a70 0x55ee312a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53400/0 (socket says 172.21.15.63:53400) 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.995+0000 7fa515400700 1 -- 172.21.15.63:0/988140396 learned_addr learned my addr 172.21.15.63:0/988140396 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.995+0000 7fa515e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 0x55ee312a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.999+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ee310be020 con 0x55ee312a4340 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.999+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee311587e0 con 0x55ee3111aec0 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.999+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ee312a6e80 con 0x55ee31206a70 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.999+0000 7fa516800700 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:23.999+0000 7fa527421b80 10 client.0 _ll_register_callbacks cb 0x55ee31161cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.000+0000 7fa516800700 1 -- 172.21.15.63:0/988140396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee31206a70 msgr2=0x55ee312a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:24.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.000+0000 7fa516800700 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee31206a70 0x55ee312a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.000+0000 7fa516800700 1 -- 172.21.15.63:0/988140396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 msgr2=0x55ee312a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.000+0000 7fa516800700 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 0x55ee312a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.000+0000 7fa516800700 1 -- 172.21.15.63:0/988140396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee31107950 con 0x55ee312a4340 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.002+0000 7fa516800700 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a1f10 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7fa510052d00 tx=0x7fa510075ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f33ae6b176c578 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.002+0000 7fa4f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.003+0000 7fa4f3e00700 1 -- 172.21.15.63:0/988140396 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510083470 con 0x55ee312a4340 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.003+0000 7fa4f3e00700 1 -- 172.21.15.63:0/988140396 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(167..167 src has 1..167) ==== 9619+0+0 (secure 0 0 0) 0x7fa51007e040 con 0x55ee312a4340 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.003+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5040a9630 con 0x55ee312a4340 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.004+0000 7fa4f3e00700 1 -- 172.21.15.63:0/988140396 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa510083610 con 0x55ee312a4340 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.004+0000 7fa527421b80 10 client.23544 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.004+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ee311880e0 con 0x55ee312a4340 2024-05-27T08:17:24.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.006+0000 7fa4f3e00700 1 -- 172.21.15.63:0/988140396 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5040a9630 con 0x55ee312a4340 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.006+0000 7fa4f3e00700 1 -- 172.21.15.63:0/988140396 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa51008c860 con 0x55ee312a4340 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.006+0000 7fa527421b80 10 client.23544 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.006+0000 7fa527421b80 10 client.23544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.007+0000 7fa527421b80 20 client.23544 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.007+0000 7fa527421b80 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ee312b3f90 0x55ee312b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.007+0000 7fa527421b80 4 client.23544 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.007+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee311c9e10 con 0x55ee312b3f90 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.008+0000 7fa515e00700 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ee312b3f90 0x55ee312b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.010+0000 7fa515e00700 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ee312b3f90 0x55ee312b6420 crc :-1 s=READY pgs=758 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=829dd9bb080df294 server_cookie=1b76d3553e7e350a in_seq=0 out_seq=0 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.010+0000 7fa4f3e00700 10 client.23544 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.013+0000 7fa4f3e00700 1 -- 172.21.15.63:0/988140396 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ee311c9e10 con 0x55ee312b3f90 2024-05-27T08:17:24.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.013+0000 7fa4f3e00700 10 client.23544 handle_command_reply: tid=0 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.013+0000 7fa527421b80 1 client.23544 shutdown 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.014+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 msgr2=0x55ee312a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.014+0000 7fa527421b80 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a1f10 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7fa510052d00 tx=0x7fa510075ef0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.014+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 shutdown_connections 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.014+0000 7fa527421b80 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ee312b3f90 0x55ee312b6420 crc :-1 s=READY pgs=758 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.014+0000 7fa527421b80 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a1f10 unknown :-1 s=CLOSED pgs=2961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.014+0000 7fa527421b80 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 0x55ee312a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.014+0000 7fa527421b80 1 --2- 172.21.15.63:0/988140396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee31206a70 0x55ee312a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.014+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 >> 172.21.15.63:0/988140396 conn(0x55ee311b4410 msgr2=0x55ee312954c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.014+0000 7fa4f3e00700 0 client.23544 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.016+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 shutdown_connections 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.016+0000 7fa527421b80 1 -- 172.21.15.63:0/988140396 wait complete. 2024-05-27T08:17:24.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.016+0000 7fa527421b80 20 client.23544 trim_cache size 0 max 16384 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.058+0000 7fa527421b80 1 Processor -- start 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.059+0000 7fa527421b80 1 -- start start 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.060+0000 7fa527421b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 0x55ee31299bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.060+0000 7fa527421b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee31206a70 0x55ee312a0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.060+0000 7fa527421b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.060+0000 7fa527421b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4011770 con 0x55ee312a4340 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.060+0000 7fa527421b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f400f000 con 0x55ee3111aec0 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.060+0000 7fa527421b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f400ce70 con 0x55ee31206a70 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.062+0000 7fa515e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 0x55ee31299bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.062+0000 7fa515e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 0x55ee31299bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:56142/0 (socket says 172.21.15.63:56142) 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.062+0000 7fa515e00700 1 -- 172.21.15.63:0/1583671494 learned_addr learned my addr 172.21.15.63:0/1583671494 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.062+0000 7fa516800700 1 --2- 172.21.15.63:0/1583671494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.062+0000 7fa515e00700 1 -- 172.21.15.63:0/1583671494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee31206a70 msgr2=0x55ee312a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:24.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.062+0000 7fa515e00700 1 --2- 172.21.15.63:0/1583671494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee31206a70 0x55ee312a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.062+0000 7fa515e00700 1 -- 172.21.15.63:0/1583671494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 msgr2=0x55ee312bbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:24.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.062+0000 7fa515e00700 1 --2- 172.21.15.63:0/1583671494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312bbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.062+0000 7fa515e00700 1 -- 172.21.15.63:0/1583671494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee3128f9b0 con 0x55ee3111aec0 2024-05-27T08:17:24.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.064+0000 7fa515e00700 1 --2- 172.21.15.63:0/1583671494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 0x55ee31299bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55ee3129b390 tx=0x7fa5040a5620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a046952474777 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:24.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.066+0000 7fa4fea00700 1 -- 172.21.15.63:0/1583671494 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5040c7070 con 0x55ee3111aec0 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.066+0000 7fa4fea00700 1 -- 172.21.15.63:0/1583671494 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5040b2490 con 0x55ee3111aec0 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.066+0000 7fa4fea00700 1 -- 172.21.15.63:0/1583671494 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504020040 con 0x55ee3111aec0 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.068+0000 7fa527421b80 1 -- 172.21.15.63:0/1583671494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 msgr2=0x55ee31299bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.068+0000 7fa527421b80 1 --2- 172.21.15.63:0/1583671494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 0x55ee31299bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55ee3129b390 tx=0x7fa5040a5620 comp rx=0 tx=0).stop 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.068+0000 7fa527421b80 1 -- 172.21.15.63:0/1583671494 shutdown_connections 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.069+0000 7fa527421b80 1 --2- 172.21.15.63:0/1583671494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312bbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.069+0000 7fa527421b80 1 --2- 172.21.15.63:0/1583671494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee3111aec0 0x55ee31299bd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.069+0000 7fa527421b80 1 --2- 172.21.15.63:0/1583671494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee31206a70 0x55ee312a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.069+0000 7fa527421b80 1 -- 172.21.15.63:0/1583671494 >> 172.21.15.63:0/1583671494 conn(0x55ee311b4410 msgr2=0x55ee3129fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.070+0000 7fa527421b80 1 -- 172.21.15.63:0/1583671494 shutdown_connections 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.070+0000 7fa527421b80 1 -- 172.21.15.63:0/1583671494 wait complete. 2024-05-27T08:17:24.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.070+0000 7fa527421b80 1 -- start start 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.070+0000 7fa527421b80 1 Processor -- start 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.071+0000 7fa527421b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee3111aec0 0x55ee312a8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.072+0000 7fa527421b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee31206a70 0x55ee312a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.072+0000 7fa527421b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.072+0000 7fa515e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee3111aec0 0x55ee312a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.073+0000 7fa515e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee3111aec0 0x55ee312a8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:53412/0 (socket says 172.21.15.63:53412) 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.073+0000 7fa516800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.073+0000 7fa527421b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f400d4c0 con 0x55ee312a4340 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.073+0000 7fa527421b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f400e930 con 0x55ee31206a70 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.073+0000 7fa527421b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f400db90 con 0x55ee3111aec0 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.073+0000 7fa527421b80 10 client.0 _ll_register_callbacks cb 0x55ee31161cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.073+0000 7fa516800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41616/0 (socket says 172.21.15.63:41616) 2024-05-27T08:17:24.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.073+0000 7fa516800700 1 -- 172.21.15.63:0/4072670796 learned_addr learned my addr 172.21.15.63:0/4072670796 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.073+0000 7fa515400700 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee31206a70 0x55ee312a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.074+0000 7fa516800700 1 -- 172.21.15.63:0/4072670796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee3111aec0 msgr2=0x55ee312a8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.074+0000 7fa516800700 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee3111aec0 0x55ee312a8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.074+0000 7fa516800700 1 -- 172.21.15.63:0/4072670796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee31206a70 msgr2=0x55ee312a85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.074+0000 7fa516800700 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee31206a70 0x55ee312a85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.074+0000 7fa516800700 1 -- 172.21.15.63:0/4072670796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee3131dc30 con 0x55ee312a4340 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.075+0000 7fa516800700 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a8b00 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7fa5100519f0 tx=0x7fa51006a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc70e7d8bf49de36 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.075+0000 7fa4f3e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.076+0000 7fa4f3e00700 1 -- 172.21.15.63:0/4072670796 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510083470 con 0x55ee312a4340 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.076+0000 7fa4f3e00700 1 -- 172.21.15.63:0/4072670796 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(167..167 src has 1..167) ==== 9619+0+0 (secure 0 0 0) 0x7fa510088040 con 0x55ee312a4340 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.076+0000 7fa527421b80 1 -- 172.21.15.63:0/4072670796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee311c35f0 con 0x55ee312a4340 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.077+0000 7fa4f3e00700 1 -- 172.21.15.63:0/4072670796 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa5100839c0 con 0x55ee312a4340 2024-05-27T08:17:24.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.077+0000 7fa527421b80 10 client.23547 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:24.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.077+0000 7fa527421b80 1 -- 172.21.15.63:0/4072670796 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ee3131eed0 con 0x55ee312a4340 2024-05-27T08:17:24.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.078+0000 7fa4f3e00700 1 -- 172.21.15.63:0/4072670796 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ee311c35f0 con 0x55ee312a4340 2024-05-27T08:17:24.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.078+0000 7fa4f3e00700 1 -- 172.21.15.63:0/4072670796 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa510094040 con 0x55ee312a4340 2024-05-27T08:17:24.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.078+0000 7fa527421b80 10 client.23547 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:24.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.078+0000 7fa527421b80 10 client.23547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:24.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.078+0000 7fa527421b80 20 client.23547 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:24.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.078+0000 7fa527421b80 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ee3131f520 0x55ee31321940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:24.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.078+0000 7fa527421b80 4 client.23547 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:17:24.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.078+0000 7fa527421b80 1 -- 172.21.15.63:0/4072670796 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ee3103a2d0 con 0x55ee3131f520 2024-05-27T08:17:24.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.082+0000 7fa515e00700 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ee3131f520 0x55ee31321940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:24.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.083+0000 7fa515e00700 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ee3131f520 0x55ee31321940 crc :-1 s=READY pgs=759 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f8e14e553cb4cb server_cookie=44f1735c7bd4fb8e in_seq=0 out_seq=0 2024-05-27T08:17:24.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.083+0000 7fa4f3e00700 10 client.23547 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:24.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.088+0000 7fa4f3e00700 1 -- 172.21.15.63:0/4072670796 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ee3103a2d0 con 0x55ee3131f520 2024-05-27T08:17:24.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.088+0000 7fa4f3e00700 10 client.23547 handle_command_reply: tid=0 2024-05-27T08:17:24.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.088+0000 7fa527421b80 1 client.23547 shutdown 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.090+0000 7fa527421b80 1 -- 172.21.15.63:0/4072670796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 msgr2=0x55ee312a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.090+0000 7fa527421b80 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a8b00 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7fa5100519f0 tx=0x7fa51006a740 comp rx=0 tx=0).stop 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.091+0000 7fa527421b80 1 -- 172.21.15.63:0/4072670796 shutdown_connections 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.091+0000 7fa527421b80 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ee3131f520 0x55ee31321940 crc :-1 s=READY pgs=759 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.091+0000 7fa527421b80 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ee312a4340 0x55ee312a8b00 unknown :-1 s=CLOSED pgs=2962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.091+0000 7fa527421b80 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee3111aec0 0x55ee312a8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.091+0000 7fa527421b80 1 --2- 172.21.15.63:0/4072670796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee31206a70 0x55ee312a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.091+0000 7fa527421b80 1 -- 172.21.15.63:0/4072670796 >> 172.21.15.63:0/4072670796 conn(0x55ee311b4410 msgr2=0x55ee31299930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.091+0000 7fa4f3e00700 0 client.23547 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.092+0000 7fa527421b80 1 -- 172.21.15.63:0/4072670796 shutdown_connections 2024-05-27T08:17:24.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.092+0000 7fa527421b80 1 -- 172.21.15.63:0/4072670796 wait complete. 2024-05-27T08:17:24.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.092+0000 7fa527421b80 20 client.23547 trim_cache size 0 max 16384 2024-05-27T08:17:24.097 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:17:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:17:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "d3d08423-57b7-4cab-9b08-0ce5bf0f538d", 2024-05-27T08:17:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:17:24.097 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:17:24.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.096+0000 7fa517200700 1 -- 172.21.15.63:0/2974006829 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b2b20 msgr2=0x7fa4f80b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:24.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.096+0000 7fa517200700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b2b20 0x7fa4f80b5030 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa500005b40 tx=0x7fa500005ab0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.096+0000 7fa517200700 1 -- 172.21.15.63:0/2974006829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa50c007e20 msgr2=0x7fa50c133650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:24.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.096+0000 7fa517200700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa50c007e20 0x7fa50c133650 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7fa50400d950 tx=0x7fa50400dc60 comp rx=0 tx=0).stop 2024-05-27T08:17:24.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.097+0000 7fa517200700 1 -- 172.21.15.63:0/2974006829 shutdown_connections 2024-05-27T08:17:24.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.097+0000 7fa517200700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa4f80b2b20 0x7fa4f80b5030 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.097+0000 7fa517200700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa50c0a7680 0x7fa50c137e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.097+0000 7fa517200700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa50c007e20 0x7fa50c133650 unknown :-1 s=CLOSED pgs=2959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.097+0000 7fa517200700 1 --2- 172.21.15.63:0/2974006829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa50c008770 0x7fa50c133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:24.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.097+0000 7fa517200700 1 -- 172.21.15.63:0/2974006829 >> 172.21.15.63:0/2974006829 conn(0x7fa50c00c080 msgr2=0x7fa50c09a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:24.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.097+0000 7fa517200700 1 -- 172.21.15.63:0/2974006829 shutdown_connections 2024-05-27T08:17:24.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:24.097+0000 7fa517200700 1 -- 172.21.15.63:0/2974006829 wait complete. 2024-05-27T08:17:25.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:24 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:17:25.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:24 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:25.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:24 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:17:25.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:24 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:17:25.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:24 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:25.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:24 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:17:25.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:24 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:17:25.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:24 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:25.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:24 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:17:26.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:25 smithi086 ceph-mon[34687]: pgmap v1392: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.2 MiB/s wr, 2.00k op/s 2024-05-27T08:17:26.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:25 smithi149 ceph-mon[35408]: pgmap v1392: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.2 MiB/s wr, 2.00k op/s 2024-05-27T08:17:26.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:25 smithi063 ceph-mon[31613]: pgmap v1392: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.2 MiB/s wr, 2.00k op/s 2024-05-27T08:17:27.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:27 smithi086 ceph-mon[34687]: osdmap e168: 12 total, 12 up, 12 in 2024-05-27T08:17:27.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:27 smithi086 ceph-mon[34687]: pgmap v1394: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.9 MiB/s wr, 2.43k op/s 2024-05-27T08:17:27.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:27 smithi063 ceph-mon[31613]: osdmap e168: 12 total, 12 up, 12 in 2024-05-27T08:17:27.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:27 smithi063 ceph-mon[31613]: pgmap v1394: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.9 MiB/s wr, 2.43k op/s 2024-05-27T08:17:27.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:27 smithi149 ceph-mon[35408]: osdmap e168: 12 total, 12 up, 12 in 2024-05-27T08:17:27.970 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:27 smithi149 ceph-mon[35408]: pgmap v1394: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.9 MiB/s wr, 2.43k op/s 2024-05-27T08:17:28.331 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:17:28.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:28 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:17:28.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:28 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:28 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:17:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:28 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:28 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:17:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:28 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:28 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:17:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:28 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:28 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:17:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:28 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:28 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:17:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:28 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:28 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:17:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:28 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:28 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:17:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:28 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:28 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:17:28.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:28 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:28 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:17:28.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:28 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:28 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:17:28.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:28 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:28 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:17:28.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:28 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:28 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:17:28.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:28 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:28 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:17:28.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:28 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:28.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:28 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:17:28.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:28 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.311+0000 7f544e000640 1 -- 172.21.15.63:0/4003295211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54400b86e0 msgr2=0x7f54400bab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.311+0000 7f544e000640 1 --2- 172.21.15.63:0/4003295211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54400b86e0 0x7f54400bab00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5448069170 tx=0x7f544808f1b0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.312+0000 7f544e000640 1 -- 172.21.15.63:0/4003295211 shutdown_connections 2024-05-27T08:17:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.312+0000 7f544e000640 1 --2- 172.21.15.63:0/4003295211 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54400b86e0 0x7f54400bab00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.312+0000 7f544e000640 1 --2- 172.21.15.63:0/4003295211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54400b5d80 0x7f54400b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.312+0000 7f544e000640 1 --2- 172.21.15.63:0/4003295211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54400a5810 0x7f54400a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.312+0000 7f544e000640 1 -- 172.21.15.63:0/4003295211 >> 172.21.15.63:0/4003295211 conn(0x7f544001aef0 msgr2=0x7f544001b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.315+0000 7f544e000640 1 -- 172.21.15.63:0/4003295211 shutdown_connections 2024-05-27T08:17:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.316+0000 7f544e000640 1 -- 172.21.15.63:0/4003295211 wait complete. 2024-05-27T08:17:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.318+0000 7f544e000640 1 Processor -- start 2024-05-27T08:17:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.322+0000 7f544e000640 1 -- start start 2024-05-27T08:17:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.323+0000 7f544e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54400a5810 0x7f54400aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.323+0000 7f544e000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54400b5d80 0x7f54400aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.323+0000 7f544e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54400b86e0 0x7f54400af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.323+0000 7f544e000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f54400bdfc0 con 0x7f54400b5d80 2024-05-27T08:17:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.323+0000 7f544e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54400bde40 con 0x7f54400a5810 2024-05-27T08:17:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.323+0000 7f544e000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f54400be140 con 0x7f54400b86e0 2024-05-27T08:17:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.326+0000 7f5447e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54400b5d80 0x7f54400aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.327+0000 7f5447e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54400b5d80 0x7f54400aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50254/0 (socket says 172.21.15.63:50254) 2024-05-27T08:17:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.327+0000 7f544d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54400b86e0 0x7f54400af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.327+0000 7f544cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54400a5810 0x7f54400aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.327+0000 7f5447e00640 1 -- 172.21.15.63:0/892083383 learned_addr learned my addr 172.21.15.63:0/892083383 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.329+0000 7f544cc00640 1 -- 172.21.15.63:0/892083383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54400b86e0 msgr2=0x7f54400af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.329+0000 7f544cc00640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54400b86e0 0x7f54400af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.329+0000 7f544cc00640 1 -- 172.21.15.63:0/892083383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54400b5d80 msgr2=0x7f54400aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.329+0000 7f544cc00640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54400b5d80 0x7f54400aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.329+0000 7f544cc00640 1 -- 172.21.15.63:0/892083383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54400b3400 con 0x7f54400a5810 2024-05-27T08:17:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.329+0000 7f544cc00640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54400a5810 0x7f54400aea00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5434009fd0 tx=0x7f543400ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.330+0000 7f5445600640 1 -- 172.21.15.63:0/892083383 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543400ccc0 con 0x7f54400a5810 2024-05-27T08:17:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.330+0000 7f5445600640 1 -- 172.21.15.63:0/892083383 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f543400ee70 con 0x7f54400a5810 2024-05-27T08:17:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.330+0000 7f5445600640 1 -- 172.21.15.63:0/892083383 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5434004a00 con 0x7f54400a5810 2024-05-27T08:17:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.332+0000 7f544e000640 1 -- 172.21.15.63:0/892083383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54400b36f0 con 0x7f54400a5810 2024-05-27T08:17:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.333+0000 7f544e000640 1 -- 172.21.15.63:0/892083383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54400b3bf0 con 0x7f54400a5810 2024-05-27T08:17:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.347+0000 7f5445600640 1 -- 172.21.15.63:0/892083383 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5434004ba0 con 0x7f54400a5810 2024-05-27T08:17:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.350+0000 7f5445600640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f54300b2b70 0x7f54300b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.351+0000 7f5445600640 1 -- 172.21.15.63:0/892083383 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 9543+0+0 (secure 0 0 0) 0x7f54340e3a90 con 0x7f54400a5810 2024-05-27T08:17:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.352+0000 7f5447e00640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f54300b2b70 0x7f54300b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.359+0000 7f5447e00640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f54300b2b70 0x7f54300b5080 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f54400b7cb0 tx=0x7f543c005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.363+0000 7f5456621740 1 Processor -- start 2024-05-27T08:17:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.363+0000 7f5456621740 1 -- start start 2024-05-27T08:17:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.363+0000 7f5456621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e511b0 0x561cc61f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.363+0000 7f5456621740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc61f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.365+0000 7f5456621740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e304d0 0x561cc61f4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.365+0000 7f5456621740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561cc6164b90 con 0x561cc5e282f0 2024-05-27T08:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.365+0000 7f5456621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561cc6154730 con 0x561cc5e511b0 2024-05-27T08:17:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.365+0000 7f5456621740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561cc6158770 con 0x561cc5e304d0 2024-05-27T08:17:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.365+0000 7f544cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e511b0 0x561cc61f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.367+0000 7f5447e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc61f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.367+0000 7f544cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e511b0 0x561cc61f3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46020/0 (socket says 172.21.15.63:46020) 2024-05-27T08:17:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.367+0000 7f544cc00640 1 -- 172.21.15.63:0/728634199 learned_addr learned my addr 172.21.15.63:0/728634199 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.365+0000 7f544d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e304d0 0x561cc61f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.369+0000 7f544cc00640 1 -- 172.21.15.63:0/728634199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e304d0 msgr2=0x561cc61f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.369+0000 7f544cc00640 1 --2- 172.21.15.63:0/728634199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e304d0 0x561cc61f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.369+0000 7f544cc00640 1 -- 172.21.15.63:0/728634199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 msgr2=0x561cc61f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.369+0000 7f544cc00640 1 --2- 172.21.15.63:0/728634199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc61f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.369+0000 7f544cc00640 1 -- 172.21.15.63:0/728634199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cc60ed8b0 con 0x561cc5e511b0 2024-05-27T08:17:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.370+0000 7f544cc00640 1 --2- 172.21.15.63:0/728634199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e511b0 0x561cc61f3a90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54340aa8a0 tx=0x7f54340a60a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2379f27028500c10 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.370+0000 7f5439c00640 1 -- 172.21.15.63:0/728634199 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340b30c0 con 0x561cc5e511b0 2024-05-27T08:17:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.371+0000 7f5439c00640 1 -- 172.21.15.63:0/728634199 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54340ab010 con 0x561cc5e511b0 2024-05-27T08:17:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.371+0000 7f5439c00640 1 -- 172.21.15.63:0/728634199 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340a6ba0 con 0x561cc5e511b0 2024-05-27T08:17:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.373+0000 7f5456621740 1 -- 172.21.15.63:0/728634199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e511b0 msgr2=0x561cc61f3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.373+0000 7f5456621740 1 --2- 172.21.15.63:0/728634199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e511b0 0x561cc61f3a90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54340aa8a0 tx=0x7f54340a60a0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.374+0000 7f5456621740 1 -- 172.21.15.63:0/728634199 shutdown_connections 2024-05-27T08:17:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.374+0000 7f5456621740 1 --2- 172.21.15.63:0/728634199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e304d0 0x561cc61f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.374+0000 7f5456621740 1 --2- 172.21.15.63:0/728634199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc61f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.374+0000 7f5456621740 1 --2- 172.21.15.63:0/728634199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e511b0 0x561cc61f3a90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.374+0000 7f5456621740 1 -- 172.21.15.63:0/728634199 >> 172.21.15.63:0/728634199 conn(0x561cc5e45300 msgr2=0x561cc61f0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.374+0000 7f5456621740 1 -- 172.21.15.63:0/728634199 shutdown_connections 2024-05-27T08:17:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.375+0000 7f5456621740 1 -- 172.21.15.63:0/728634199 wait complete. 2024-05-27T08:17:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.376+0000 7f5456621740 1 -- start start 2024-05-27T08:17:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.376+0000 7f5456621740 1 Processor -- start 2024-05-27T08:17:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.378+0000 7f5456621740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc61fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.378+0000 7f5456621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 0x561cc61fe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.378+0000 7f5456621740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.378+0000 7f5456621740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561cc5e3a790 con 0x561cc5e282f0 2024-05-27T08:17:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.378+0000 7f5456621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561cc60fcbc0 con 0x561cc5e304d0 2024-05-27T08:17:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.378+0000 7f5456621740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561cc6200bc0 con 0x561cc5e511b0 2024-05-27T08:17:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.378+0000 7f5456621740 10 client.0 _ll_register_callbacks cb 0x561cc60da400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.378+0000 7f544d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.379+0000 7f544d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:51834/0 (socket says 172.21.15.63:51834) 2024-05-27T08:17:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.379+0000 7f544d600640 1 -- 172.21.15.63:0/2451699918 learned_addr learned my addr 172.21.15.63:0/2451699918 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.380+0000 7f5447e00640 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 0x561cc61fe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.380+0000 7f544cc00640 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc61fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.382+0000 7f544d600640 1 -- 172.21.15.63:0/2451699918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 msgr2=0x561cc61fe110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.382+0000 7f544d600640 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 0x561cc61fe110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.382+0000 7f544d600640 1 -- 172.21.15.63:0/2451699918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 msgr2=0x561cc61fdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.382+0000 7f544d600640 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc61fdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.382+0000 7f544d600640 1 -- 172.21.15.63:0/2451699918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cc5e69210 con 0x561cc5e511b0 2024-05-27T08:17:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.382+0000 7f544cc00640 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc61fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:17:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.383+0000 7f544d600640 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209f50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5448068be0 tx=0x7f544806b290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e73c6400bebb7a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.383+0000 7f542e000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:17:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.384+0000 7f542e000640 1 -- 172.21.15.63:0/2451699918 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54480a8060 con 0x561cc5e511b0 2024-05-27T08:17:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.384+0000 7f542e000640 1 -- 172.21.15.63:0/2451699918 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(168..168 src has 1..168) ==== 9543+0+0 (secure 0 0 0) 0x7f5448069800 con 0x561cc5e511b0 2024-05-27T08:17:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.385+0000 7f5456621740 1 -- 172.21.15.63:0/2451699918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54340ab010 con 0x561cc5e511b0 2024-05-27T08:17:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.385+0000 7f542e000640 1 -- 172.21.15.63:0/2451699918 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5448069a00 con 0x561cc5e511b0 2024-05-27T08:17:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.385+0000 7f5456621740 10 client.32245 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.385+0000 7f5456621740 1 -- 172.21.15.63:0/2451699918 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561cc5f35620 con 0x561cc5e511b0 2024-05-27T08:17:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.386+0000 7f542e000640 1 -- 172.21.15.63:0/2451699918 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54340ab010 con 0x561cc5e511b0 2024-05-27T08:17:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.387+0000 7f542e000640 1 -- 172.21.15.63:0/2451699918 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5448098900 con 0x561cc5e511b0 2024-05-27T08:17:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.387+0000 7f5456621740 10 client.32245 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.387+0000 7f5456621740 10 client.32245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.387+0000 7f5456621740 20 client.32245 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.387+0000 7f5456621740 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561cc5e0fd80 0x561cc620fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.387+0000 7f5456621740 4 client.32245 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:17:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.387+0000 7f5456621740 1 -- 172.21.15.63:0/2451699918 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cc6166ae0 con 0x561cc5e0fd80 2024-05-27T08:17:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.388+0000 7f544cc00640 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561cc5e0fd80 0x561cc620fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.391+0000 7f544cc00640 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561cc5e0fd80 0x561cc620fea0 crc :-1 s=READY pgs=760 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e22435a465ee2781 server_cookie=54ce928038af783c in_seq=0 out_seq=0 2024-05-27T08:17:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.392+0000 7f542e000640 10 client.32245 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.394+0000 7f542e000640 1 -- 172.21.15.63:0/2451699918 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561cc6166ae0 con 0x561cc5e0fd80 2024-05-27T08:17:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.394+0000 7f542e000640 10 client.32245 handle_command_reply: tid=0 2024-05-27T08:17:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.394+0000 7f5456621740 1 client.32245 shutdown 2024-05-27T08:17:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.396+0000 7f5456621740 1 -- 172.21.15.63:0/2451699918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 msgr2=0x561cc6209f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.396+0000 7f5456621740 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209f50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5448068be0 tx=0x7f544806b290 comp rx=0 tx=0).stop 2024-05-27T08:17:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.396+0000 7f5456621740 1 -- 172.21.15.63:0/2451699918 shutdown_connections 2024-05-27T08:17:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.396+0000 7f5456621740 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561cc5e0fd80 0x561cc620fea0 crc :-1 s=READY pgs=760 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.396+0000 7f5456621740 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209f50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.396+0000 7f5456621740 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 0x561cc61fe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.396+0000 7f5456621740 1 --2- 172.21.15.63:0/2451699918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc61fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.396+0000 7f5456621740 1 -- 172.21.15.63:0/2451699918 >> 172.21.15.63:0/2451699918 conn(0x561cc5e45300 msgr2=0x561cc61f3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.396+0000 7f542e000640 0 client.32245 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.397+0000 7f5456621740 1 -- 172.21.15.63:0/2451699918 shutdown_connections 2024-05-27T08:17:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.397+0000 7f5456621740 1 -- 172.21.15.63:0/2451699918 wait complete. 2024-05-27T08:17:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.397+0000 7f5456621740 20 client.32245 trim_cache size 0 max 16384 2024-05-27T08:17:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.442+0000 7f5456621740 1 Processor -- start 2024-05-27T08:17:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.443+0000 7f5456621740 1 -- start start 2024-05-27T08:17:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.443+0000 7f5456621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e282f0 0x561cc61f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.443+0000 7f5456621740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e304d0 0x561cc61ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.443+0000 7f5456621740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.443+0000 7f5456621740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5440166760 con 0x561cc5e304d0 2024-05-27T08:17:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.443+0000 7f5456621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5440164030 con 0x561cc5e282f0 2024-05-27T08:17:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.443+0000 7f5456621740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5440161e80 con 0x561cc5e511b0 2024-05-27T08:17:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.444+0000 7f544cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e282f0 0x561cc61f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.444+0000 7f5447e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e304d0 0x561cc61ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.444+0000 7f544cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e282f0 0x561cc61f93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46036/0 (socket says 172.21.15.63:46036) 2024-05-27T08:17:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.444+0000 7f544cc00640 1 -- 172.21.15.63:0/2478771654 learned_addr learned my addr 172.21.15.63:0/2478771654 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.445+0000 7f544d600640 1 --2- 172.21.15.63:0/2478771654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.445+0000 7f544cc00640 1 -- 172.21.15.63:0/2478771654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 msgr2=0x561cc6209cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.446+0000 7f544cc00640 1 --2- 172.21.15.63:0/2478771654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.446+0000 7f544cc00640 1 -- 172.21.15.63:0/2478771654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e304d0 msgr2=0x561cc61ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.447+0000 7f544cc00640 1 --2- 172.21.15.63:0/2478771654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e304d0 0x561cc61ef240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.447+0000 7f544cc00640 1 -- 172.21.15.63:0/2478771654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cc61f4e70 con 0x561cc5e282f0 2024-05-27T08:17:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.447+0000 7f544d600640 1 --2- 172.21.15.63:0/2478771654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:17:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.447+0000 7f5447e00640 1 --2- 172.21.15.63:0/2478771654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e304d0 0x561cc61ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:17:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.450+0000 7f544cc00640 1 --2- 172.21.15.63:0/2478771654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e282f0 0x561cc61f93d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5434005fd0 tx=0x7f54340e3040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=543039dc038daa8c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.453+0000 7f5439200640 1 -- 172.21.15.63:0/2478771654 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340aecc0 con 0x561cc5e282f0 2024-05-27T08:17:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.453+0000 7f5439200640 1 -- 172.21.15.63:0/2478771654 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54340b30b0 con 0x561cc5e282f0 2024-05-27T08:17:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.453+0000 7f5439200640 1 -- 172.21.15.63:0/2478771654 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340e1000 con 0x561cc5e282f0 2024-05-27T08:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.456+0000 7f5456621740 1 -- 172.21.15.63:0/2478771654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e282f0 msgr2=0x561cc61f93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.456+0000 7f5456621740 1 --2- 172.21.15.63:0/2478771654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e282f0 0x561cc61f93d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5434005fd0 tx=0x7f54340e3040 comp rx=0 tx=0).stop 2024-05-27T08:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.456+0000 7f5456621740 1 -- 172.21.15.63:0/2478771654 shutdown_connections 2024-05-27T08:17:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.456+0000 7f5456621740 1 --2- 172.21.15.63:0/2478771654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc6209cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.456+0000 7f5456621740 1 --2- 172.21.15.63:0/2478771654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e304d0 0x561cc61ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.457+0000 7f5456621740 1 --2- 172.21.15.63:0/2478771654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e282f0 0x561cc61f93d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.457+0000 7f5456621740 1 -- 172.21.15.63:0/2478771654 >> 172.21.15.63:0/2478771654 conn(0x561cc5e45300 msgr2=0x561cc61f3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.459+0000 7f5456621740 1 -- 172.21.15.63:0/2478771654 shutdown_connections 2024-05-27T08:17:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.460+0000 7f5456621740 1 -- 172.21.15.63:0/2478771654 wait complete. 2024-05-27T08:17:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.461+0000 7f5456621740 1 -- start start 2024-05-27T08:17:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.461+0000 7f5456621740 1 Processor -- start 2024-05-27T08:17:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.465+0000 7f5456621740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc6206660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.466+0000 7f5456621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 0x561cc6200e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.467+0000 7f5447e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 0x561cc6200e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.467+0000 7f5456621740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc62013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.467+0000 7f544cc00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc6206660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.468+0000 7f5456621740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f54401624f0 con 0x561cc5e282f0 2024-05-27T08:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.468+0000 7f5456621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5440163960 con 0x561cc5e304d0 2024-05-27T08:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.468+0000 7f5456621740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5440162bc0 con 0x561cc5e511b0 2024-05-27T08:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.468+0000 7f5456621740 10 client.0 _ll_register_callbacks cb 0x7f543c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.469+0000 7f5447e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 0x561cc6200e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46040/0 (socket says 172.21.15.63:46040) 2024-05-27T08:17:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.469+0000 7f5447e00640 1 -- 172.21.15.63:0/3773933607 learned_addr learned my addr 172.21.15.63:0/3773933607 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.469+0000 7f544d600640 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc62013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.470+0000 7f544d600640 1 -- 172.21.15.63:0/3773933607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 msgr2=0x561cc6200e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.470+0000 7f544d600640 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 0x561cc6200e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.470+0000 7f544d600640 1 -- 172.21.15.63:0/3773933607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 msgr2=0x561cc6206660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.470+0000 7f544d600640 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc6206660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.470+0000 7f544d600640 1 -- 172.21.15.63:0/3773933607 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cc6201b80 con 0x561cc5e511b0 2024-05-27T08:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.471+0000 7f544cc00640 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc6206660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.472+0000 7f542e000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:17:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.472+0000 7f544d600640 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc62013f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5448052160 tx=0x7f544808f1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8e6275618ab1b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.475+0000 7f542e000640 1 -- 172.21.15.63:0/3773933607 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448068ca0 con 0x561cc5e511b0 2024-05-27T08:17:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.475+0000 7f542e000640 1 -- 172.21.15.63:0/3773933607 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(168..168 src has 1..168) ==== 9543+0+0 (secure 0 0 0) 0x7f5448069800 con 0x561cc5e511b0 2024-05-27T08:17:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.475+0000 7f5456621740 1 -- 172.21.15.63:0/3773933607 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561cc61602c0 con 0x561cc5e511b0 2024-05-27T08:17:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.476+0000 7f542e000640 1 -- 172.21.15.63:0/3773933607 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f544808f7f0 con 0x561cc5e511b0 2024-05-27T08:17:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.476+0000 7f5456621740 10 client.32251 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.477+0000 7f5456621740 1 -- 172.21.15.63:0/3773933607 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561cc6277460 con 0x561cc5e511b0 2024-05-27T08:17:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.477+0000 7f542e000640 1 -- 172.21.15.63:0/3773933607 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561cc61602c0 con 0x561cc5e511b0 2024-05-27T08:17:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.477+0000 7f542e000640 1 -- 172.21.15.63:0/3773933607 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f54480a3a70 con 0x561cc5e511b0 2024-05-27T08:17:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.478+0000 7f5456621740 10 client.32251 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.478+0000 7f5456621740 10 client.32251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.478+0000 7f5456621740 20 client.32251 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.478+0000 7f5456621740 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561cc5de51c0 0x561cc6279c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.478+0000 7f5456621740 4 client.32251 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:17:29.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.478+0000 7f5456621740 1 -- 172.21.15.63:0/3773933607 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cc60da4b0 con 0x561cc5de51c0 2024-05-27T08:17:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.481+0000 7f544cc00640 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561cc5de51c0 0x561cc6279c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.483+0000 7f544cc00640 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561cc5de51c0 0x561cc6279c20 crc :-1 s=READY pgs=761 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f233669b50610606 server_cookie=58e7fe58ea387285 in_seq=0 out_seq=0 2024-05-27T08:17:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.483+0000 7f542e000640 10 client.32251 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.779+0000 7f542e000640 1 -- 172.21.15.63:0/3773933607 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561cc60da4b0 con 0x561cc5de51c0 2024-05-27T08:17:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.779+0000 7f542e000640 10 client.32251 handle_command_reply: tid=0 2024-05-27T08:17:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.779+0000 7f5456621740 1 client.32251 shutdown 2024-05-27T08:17:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.780+0000 7f5456621740 1 -- 172.21.15.63:0/3773933607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 msgr2=0x561cc62013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.780+0000 7f5456621740 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc62013f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5448052160 tx=0x7f544808f1b0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.781+0000 7f5456621740 1 -- 172.21.15.63:0/3773933607 shutdown_connections 2024-05-27T08:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.781+0000 7f5456621740 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x561cc5de51c0 0x561cc6279c20 crc :-1 s=READY pgs=761 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.781+0000 7f5456621740 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561cc5e511b0 0x561cc62013f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.781+0000 7f5456621740 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561cc5e304d0 0x561cc6200e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.781+0000 7f5456621740 1 --2- 172.21.15.63:0/3773933607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561cc5e282f0 0x561cc6206660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.781+0000 7f5456621740 1 -- 172.21.15.63:0/3773933607 >> 172.21.15.63:0/3773933607 conn(0x561cc5e45300 msgr2=0x561cc61eefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.781+0000 7f542e000640 0 client.32251 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.783+0000 7f5456621740 1 -- 172.21.15.63:0/3773933607 shutdown_connections 2024-05-27T08:17:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.783+0000 7f5456621740 1 -- 172.21.15.63:0/3773933607 wait complete. 2024-05-27T08:17:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.784+0000 7f5456621740 20 client.32251 trim_cache size 0 max 16384 2024-05-27T08:17:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:17:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:17:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:17:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.793+0000 7f544e000640 1 -- 172.21.15.63:0/892083383 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f54300b2b70 msgr2=0x7f54300b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.793+0000 7f544e000640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f54300b2b70 0x7f54300b5080 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f54400b7cb0 tx=0x7f543c005f90 comp rx=0 tx=0).stop 2024-05-27T08:17:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.793+0000 7f544e000640 1 -- 172.21.15.63:0/892083383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54400a5810 msgr2=0x7f54400aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.793+0000 7f544e000640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54400a5810 0x7f54400aea00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5434009fd0 tx=0x7f543400ec20 comp rx=0 tx=0).stop 2024-05-27T08:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.794+0000 7f544e000640 1 -- 172.21.15.63:0/892083383 shutdown_connections 2024-05-27T08:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.794+0000 7f544e000640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f54300b2b70 0x7f54300b5080 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.794+0000 7f544e000640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54400b86e0 0x7f54400af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.794+0000 7f544e000640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54400b5d80 0x7f54400aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.794+0000 7f544e000640 1 --2- 172.21.15.63:0/892083383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54400a5810 0x7f54400aea00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.794+0000 7f544e000640 1 -- 172.21.15.63:0/892083383 >> 172.21.15.63:0/892083383 conn(0x7f544001aef0 msgr2=0x7f54400b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.796+0000 7f544e000640 1 -- 172.21.15.63:0/892083383 shutdown_connections 2024-05-27T08:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:29.796+0000 7f544e000640 1 -- 172.21.15.63:0/892083383 wait complete. 2024-05-27T08:17:29.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:29 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:17:29.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:29 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:29.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:29 smithi086 ceph-mon[34687]: pgmap v1395: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-27T08:17:29.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:29 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:17:29.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:29 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:29.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:29 smithi149 ceph-mon[35408]: pgmap v1395: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-27T08:17:29.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:29 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:17:29.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:29 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:29.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:29 smithi063 ceph-mon[31613]: pgmap v1395: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-27T08:17:30.188 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:17:30.721 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:30 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:30.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:30 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:30.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:30 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:17:31.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:31 smithi086 ceph-mon[34687]: pgmap v1396: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-05-27T08:17:31.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:31 smithi086 ceph-mon[34687]: osdmap e169: 12 total, 12 up, 12 in 2024-05-27T08:17:31.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:31 smithi149 ceph-mon[35408]: pgmap v1396: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-05-27T08:17:31.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:31 smithi149 ceph-mon[35408]: osdmap e169: 12 total, 12 up, 12 in 2024-05-27T08:17:31.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:31 smithi063 ceph-mon[31613]: pgmap v1396: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 11 MiB/s wr, 2.75k op/s 2024-05-27T08:17:31.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:31 smithi063 ceph-mon[31613]: osdmap e169: 12 total, 12 up, 12 in 2024-05-27T08:17:33.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:32 smithi086 ceph-mon[34687]: pgmap v1398: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 9.9 MiB/s wr, 2.50k op/s 2024-05-27T08:17:33.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:32 smithi149 ceph-mon[35408]: pgmap v1398: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 9.9 MiB/s wr, 2.50k op/s 2024-05-27T08:17:33.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:32 smithi063 ceph-mon[31613]: pgmap v1398: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 9.9 MiB/s wr, 2.50k op/s 2024-05-27T08:17:35.265 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:34 smithi149 ceph-mon[35408]: pgmap v1399: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 11 MiB/s wr, 2.80k op/s 2024-05-27T08:17:35.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:34 smithi086 ceph-mon[34687]: pgmap v1399: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 11 MiB/s wr, 2.80k op/s 2024-05-27T08:17:35.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:34 smithi063 ceph-mon[31613]: pgmap v1399: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 11 MiB/s wr, 2.80k op/s 2024-05-27T08:17:36.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:36 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:17:36.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:36 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:17:36.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:36 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:17:37.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:37 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:17:37.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:37 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:17:37.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:37 smithi149 ceph-mon[35408]: pgmap v1400: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-05-27T08:17:37.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:37 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:17:37.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:37 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:17:37.719 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:37 smithi063 ceph-mon[31613]: pgmap v1400: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-05-27T08:17:37.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:37 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:17:37.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:37 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:17:37.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:37 smithi086 ceph-mon[34687]: pgmap v1400: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-05-27T08:17:39.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:38 smithi086 ceph-mon[34687]: pgmap v1401: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 9.4 MiB/s wr, 2.38k op/s 2024-05-27T08:17:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:38 smithi149 ceph-mon[35408]: pgmap v1401: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 9.4 MiB/s wr, 2.38k op/s 2024-05-27T08:17:39.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:38 smithi063 ceph-mon[31613]: pgmap v1401: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 9.4 MiB/s wr, 2.38k op/s 2024-05-27T08:17:41.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:40 smithi086 ceph-mon[34687]: pgmap v1402: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 9.5 MiB/s wr, 2.40k op/s 2024-05-27T08:17:41.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:40 smithi149 ceph-mon[35408]: pgmap v1402: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 9.5 MiB/s wr, 2.40k op/s 2024-05-27T08:17:41.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:40 smithi063 ceph-mon[31613]: pgmap v1402: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 9.5 MiB/s wr, 2.40k op/s 2024-05-27T08:17:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.198+0000 7f2d28000640 1 -- 172.21.15.63:0/4102320327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d20111de0 msgr2=0x7f2d20112270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.198+0000 7f2d28000640 1 --2- 172.21.15.63:0/4102320327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d20111de0 0x7f2d20112270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009900 tx=0x7f2d0c02a4a0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.199+0000 7f2d28000640 1 -- 172.21.15.63:0/4102320327 shutdown_connections 2024-05-27T08:17:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.199+0000 7f2d28000640 1 --2- 172.21.15.63:0/4102320327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d20112900 0x7f2d20118eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.199+0000 7f2d28000640 1 --2- 172.21.15.63:0/4102320327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d20111de0 0x7f2d20112270 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.199+0000 7f2d28000640 1 --2- 172.21.15.63:0/4102320327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d2010a490 0x7f2d2010a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.199+0000 7f2d28000640 1 -- 172.21.15.63:0/4102320327 >> 172.21.15.63:0/4102320327 conn(0x7f2d2006e8f0 msgr2=0x7f2d2006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.199+0000 7f2d28000640 1 -- 172.21.15.63:0/4102320327 shutdown_connections 2024-05-27T08:17:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.199+0000 7f2d28000640 1 -- 172.21.15.63:0/4102320327 wait complete. 2024-05-27T08:17:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.201+0000 7f2d28000640 1 Processor -- start 2024-05-27T08:17:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.201+0000 7f2d28000640 1 -- start start 2024-05-27T08:17:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.202+0000 7f2d28000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d2010a490 0x7f2d201a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.202+0000 7f2d28000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20111de0 0x7f2d201a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.203+0000 7f2d28000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d20112900 0x7f2d201a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.203+0000 7f2d28000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2011c510 con 0x7f2d20112900 2024-05-27T08:17:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.203+0000 7f2d28000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2011c390 con 0x7f2d20111de0 2024-05-27T08:17:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.203+0000 7f2d28000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2011c690 con 0x7f2d2010a490 2024-05-27T08:17:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.203+0000 7f2d25600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d2010a490 0x7f2d201a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.203+0000 7f2d25600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d2010a490 0x7f2d201a2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:57874/0 (socket says 172.21.15.63:57874) 2024-05-27T08:17:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.203+0000 7f2d25600640 1 -- 172.21.15.63:0/350583372 learned_addr learned my addr 172.21.15.63:0/350583372 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.204+0000 7f2d24c00640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20111de0 0x7f2d201a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.204+0000 7f2d25600640 1 -- 172.21.15.63:0/350583372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20111de0 msgr2=0x7f2d201a34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.204+0000 7f2d25600640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20111de0 0x7f2d201a34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.204+0000 7f2d25600640 1 -- 172.21.15.63:0/350583372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d20112900 msgr2=0x7f2d201a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.204+0000 7f2d25600640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d20112900 0x7f2d201a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.204+0000 7f2d25600640 1 -- 172.21.15.63:0/350583372 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d201a7df0 con 0x7f2d2010a490 2024-05-27T08:17:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.206+0000 7f2d25600640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d2010a490 0x7f2d201a2f90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9f0 tx=0x7f2d1400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.208+0000 7f2d1e000640 1 -- 172.21.15.63:0/350583372 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400cd80 con 0x7f2d2010a490 2024-05-27T08:17:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.208+0000 7f2d28000640 1 -- 172.21.15.63:0/350583372 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d201a80e0 con 0x7f2d2010a490 2024-05-27T08:17:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.209+0000 7f2d28000640 1 -- 172.21.15.63:0/350583372 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2010bab0 con 0x7f2d2010a490 2024-05-27T08:17:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.209+0000 7f2d1e000640 1 -- 172.21.15.63:0/350583372 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d140108c0 con 0x7f2d2010a490 2024-05-27T08:17:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.209+0000 7f2d1e000640 1 -- 172.21.15.63:0/350583372 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400cd80 con 0x7f2d2010a490 2024-05-27T08:17:43.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:42 smithi063 ceph-mon[31613]: pgmap v1403: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 2.89k op/s 2024-05-27T08:17:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.215+0000 7f2d2efe5740 1 Processor -- start 2024-05-27T08:17:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.216+0000 7f2d2efe5740 1 -- start start 2024-05-27T08:17:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.217+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529fd01b0 0x55952a3728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.218+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a372df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.219+0000 7f2d26000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529fd01b0 0x55952a3728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.219+0000 7f2d26000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529fd01b0 0x55952a3728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55970/0 (socket says 172.21.15.63:55970) 2024-05-27T08:17:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.219+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 0x55952a373330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.219+0000 7f2d2efe5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55952a2e3b90 con 0x559529fd01b0 2024-05-27T08:17:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.219+0000 7f2d2efe5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55952a2d3730 con 0x559529faf4d0 2024-05-27T08:17:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.219+0000 7f2d2efe5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55952a2d7770 con 0x559529fa72f0 2024-05-27T08:17:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.219+0000 7f2d26000640 1 -- 172.21.15.63:0/1567495236 learned_addr learned my addr 172.21.15.63:0/1567495236 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.219+0000 7f2d25600640 1 --2- 172.21.15.63:0/1567495236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a372df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.220+0000 7f2d26000640 1 -- 172.21.15.63:0/1567495236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 msgr2=0x55952a372df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.220+0000 7f2d26000640 1 --2- 172.21.15.63:0/1567495236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a372df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.220+0000 7f2d26000640 1 -- 172.21.15.63:0/1567495236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 msgr2=0x55952a373330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.220+0000 7f2d26000640 1 --2- 172.21.15.63:0/1567495236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 0x55952a373330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.220+0000 7f2d26000640 1 -- 172.21.15.63:0/1567495236 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55952a26c8b0 con 0x559529fd01b0 2024-05-27T08:17:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.220+0000 7f2d1e000640 1 -- 172.21.15.63:0/350583372 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2d14018660 con 0x7f2d2010a490 2024-05-27T08:17:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.223+0000 7f2d26000640 1 --2- 172.21.15.63:0/1567495236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529fd01b0 0x55952a3728b0 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009cd0 tx=0x7f2d1800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4290d5c25b415af4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.223+0000 7f2d1e000640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d080b2c90 0x7f2d080b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.224+0000 7f2d25600640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d080b2c90 0x7f2d080b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.225+0000 7f2d1e000640 1 -- 172.21.15.63:0/350583372 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 9539+0+0 (secure 0 0 0) 0x7f2d140e2bc0 con 0x7f2d2010a490 2024-05-27T08:17:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.226+0000 7f2d25600640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d080b2c90 0x7f2d080b51a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x55952a3703a0 tx=0x7f2d1401f000 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.227+0000 7f2d12600640 1 -- 172.21.15.63:0/1567495236 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18016070 con 0x559529fd01b0 2024-05-27T08:17:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.227+0000 7f2d12600640 1 -- 172.21.15.63:0/1567495236 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d18005020 con 0x559529fd01b0 2024-05-27T08:17:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.227+0000 7f2d12600640 1 -- 172.21.15.63:0/1567495236 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d180116d0 con 0x559529fd01b0 2024-05-27T08:17:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.230+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1567495236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529fd01b0 msgr2=0x55952a3728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.230+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1567495236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529fd01b0 0x55952a3728b0 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009cd0 tx=0x7f2d1800e6c0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.233+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1567495236 shutdown_connections 2024-05-27T08:17:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.233+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1567495236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 0x55952a373330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.233+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1567495236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a372df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.233+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1567495236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529fd01b0 0x55952a3728b0 unknown :-1 s=CLOSED pgs=2963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.233+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1567495236 >> 172.21.15.63:0/1567495236 conn(0x559529fc4300 msgr2=0x55952a36f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.235+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1567495236 shutdown_connections 2024-05-27T08:17:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.236+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1567495236 wait complete. 2024-05-27T08:17:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.236+0000 7f2d2efe5740 1 -- start start 2024-05-27T08:17:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.239+0000 7f2d2efe5740 1 Processor -- start 2024-05-27T08:17:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.241+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a37cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.246+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529faf4d0 0x55952a37d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.246+0000 7f2d26000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a37cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.246+0000 7f2d26000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a37cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:57894/0 (socket says 172.21.15.63:57894) 2024-05-27T08:17:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.246+0000 7f2d26000640 1 -- 172.21.15.63:0/1176224883 learned_addr learned my addr 172.21.15.63:0/1176224883 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.246+0000 7f2d24c00640 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529faf4d0 0x55952a37d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.247+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529fd01b0 0x55952a388d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.247+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559529fb9790 con 0x559529faf4d0 2024-05-27T08:17:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.247+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55952a27bbc0 con 0x559529fd01b0 2024-05-27T08:17:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.247+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55952a37f9e0 con 0x559529fa72f0 2024-05-27T08:17:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.247+0000 7f2d2efe5740 10 client.0 _ll_register_callbacks cb 0x55952a259400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.252+0000 7f2d24c00640 1 -- 172.21.15.63:0/1176224883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 msgr2=0x55952a37cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.252+0000 7f2d24c00640 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a37cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.252+0000 7f2d24c00640 1 -- 172.21.15.63:0/1176224883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529fd01b0 msgr2=0x55952a388d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:17:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.252+0000 7f2d24c00640 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529fd01b0 0x55952a388d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.252+0000 7f2d24c00640 1 -- 172.21.15.63:0/1176224883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559529fe8210 con 0x559529faf4d0 2024-05-27T08:17:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.256+0000 7f2d06a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:17:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.257+0000 7f2d24c00640 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529faf4d0 0x55952a37d240 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009850 tx=0x7f2d0c005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdfe02ae86d2cb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.259+0000 7f2d06a00640 1 -- 172.21.15.63:0/1176224883 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c043d60 con 0x559529faf4d0 2024-05-27T08:17:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.262+0000 7f2d06a00640 1 -- 172.21.15.63:0/1176224883 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(169..169 src has 1..169) ==== 9539+0+0 (secure 0 0 0) 0x7f2d0c037040 con 0x559529faf4d0 2024-05-27T08:17:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.262+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d18005020 con 0x559529faf4d0 2024-05-27T08:17:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.263+0000 7f2d06a00640 1 -- 172.21.15.63:0/1176224883 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c043d60 con 0x559529faf4d0 2024-05-27T08:17:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.264+0000 7f2d2efe5740 10 client.23571 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.264+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55952a0b4620 con 0x559529faf4d0 2024-05-27T08:17:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.267+0000 7f2d06a00640 1 -- 172.21.15.63:0/1176224883 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d18005020 con 0x559529faf4d0 2024-05-27T08:17:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.270+0000 7f2d06a00640 1 -- 172.21.15.63:0/1176224883 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2d0c04e700 con 0x559529faf4d0 2024-05-27T08:17:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.270+0000 7f2d2efe5740 10 client.23571 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.270+0000 7f2d2efe5740 10 client.23571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.270+0000 7f2d2efe5740 20 client.23571 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.270+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559529f8ed80 0x55952a38ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.270+0000 7f2d2efe5740 4 client.23571 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:17:43.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.270+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55952a2e5ae0 con 0x559529f8ed80 2024-05-27T08:17:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.275+0000 7f2d26000640 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559529f8ed80 0x55952a38ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.276+0000 7f2d26000640 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559529f8ed80 0x55952a38ed00 crc :-1 s=READY pgs=762 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20bc6867dfe3c94b server_cookie=b1c6519e5f1a76dc in_seq=0 out_seq=0 2024-05-27T08:17:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.277+0000 7f2d06a00640 10 client.23571 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.277+0000 7f2d06a00640 1 -- 172.21.15.63:0/1176224883 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55952a2e5ae0 con 0x559529f8ed80 2024-05-27T08:17:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.277+0000 7f2d06a00640 10 client.23571 handle_command_reply: tid=0 2024-05-27T08:17:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.277+0000 7f2d2efe5740 1 client.23571 shutdown 2024-05-27T08:17:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.279+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529faf4d0 msgr2=0x55952a37d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.279+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529faf4d0 0x55952a37d240 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009850 tx=0x7f2d0c005870 comp rx=0 tx=0).stop 2024-05-27T08:17:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.280+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 shutdown_connections 2024-05-27T08:17:43.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.280+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559529f8ed80 0x55952a38ed00 crc :-1 s=READY pgs=762 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.280+0000 7f2d06a00640 0 client.23571 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.280+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529fd01b0 0x55952a388d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.280+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529faf4d0 0x55952a37d240 unknown :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.280+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1176224883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a37cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.280+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 >> 172.21.15.63:0/1176224883 conn(0x559529fc4300 msgr2=0x55952a372680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.281+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 shutdown_connections 2024-05-27T08:17:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.281+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1176224883 wait complete. 2024-05-27T08:17:43.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.282+0000 7f2d2efe5740 20 client.23571 trim_cache size 0 max 16384 2024-05-27T08:17:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.330+0000 7f2d2efe5740 1 Processor -- start 2024-05-27T08:17:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.331+0000 7f2d2efe5740 1 -- start start 2024-05-27T08:17:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.331+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 0x55952a36f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.332+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a36e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.332+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 0x55952a388a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.332+0000 7f2d2efe5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201ba280 con 0x559529f8ed80 2024-05-27T08:17:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.332+0000 7f2d2efe5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201b7b50 con 0x559529faf4d0 2024-05-27T08:17:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.332+0000 7f2d2efe5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201b59a0 con 0x559529fa72f0 2024-05-27T08:17:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.332+0000 7f2d26000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 0x55952a388a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.333+0000 7f2d26000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 0x55952a388a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:43800/0 (socket says 172.21.15.63:43800) 2024-05-27T08:17:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.333+0000 7f2d24c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a36e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.333+0000 7f2d24c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a36e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:57902/0 (socket says 172.21.15.63:57902) 2024-05-27T08:17:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.333+0000 7f2d24c00640 1 -- 172.21.15.63:0/1777098081 learned_addr learned my addr 172.21.15.63:0/1777098081 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.334+0000 7f2d26000640 1 --2- 172.21.15.63:0/1777098081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 0x55952a36f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.336+0000 7f2d26000640 1 -- 172.21.15.63:0/1777098081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 msgr2=0x55952a36e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.336+0000 7f2d26000640 1 --2- 172.21.15.63:0/1777098081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a36e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.336+0000 7f2d26000640 1 -- 172.21.15.63:0/1777098081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 msgr2=0x55952a36f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.336+0000 7f2d26000640 1 --2- 172.21.15.63:0/1777098081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 0x55952a36f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.336+0000 7f2d26000640 1 -- 172.21.15.63:0/1777098081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55952a37b040 con 0x559529faf4d0 2024-05-27T08:17:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.337+0000 7f2d26000640 1 --2- 172.21.15.63:0/1777098081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 0x55952a388a80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005b40 tx=0x7f2d18005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb8863e900b86e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.338+0000 7f2d11c00640 1 -- 172.21.15.63:0/1777098081 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18016040 con 0x559529faf4d0 2024-05-27T08:17:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.339+0000 7f2d11c00640 1 -- 172.21.15.63:0/1777098081 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d18028af0 con 0x559529faf4d0 2024-05-27T08:17:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.339+0000 7f2d11c00640 1 -- 172.21.15.63:0/1777098081 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d180044b0 con 0x559529faf4d0 2024-05-27T08:17:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.341+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1777098081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 msgr2=0x55952a388a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.341+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1777098081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 0x55952a388a80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005b40 tx=0x7f2d18005ad0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.342+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1777098081 shutdown_connections 2024-05-27T08:17:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.342+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1777098081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529faf4d0 0x55952a388a80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.342+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1777098081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529fa72f0 0x55952a36e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.342+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/1777098081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 0x55952a36f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.342+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1777098081 >> 172.21.15.63:0/1777098081 conn(0x559529fc4300 msgr2=0x55952a36ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.342+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1777098081 shutdown_connections 2024-05-27T08:17:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.342+0000 7f2d2efe5740 1 -- 172.21.15.63:0/1777098081 wait complete. 2024-05-27T08:17:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.343+0000 7f2d2efe5740 1 -- start start 2024-05-27T08:17:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.343+0000 7f2d2efe5740 1 Processor -- start 2024-05-27T08:17:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.345+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 0x55952a3854e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.345+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529fa72f0 0x55952a380830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.345+0000 7f2d26000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 0x55952a3854e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.345+0000 7f2d2efe5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529faf4d0 0x55952a380e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.346+0000 7f2d2efe5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201b6010 con 0x559529f8ed80 2024-05-27T08:17:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.346+0000 7f2d2efe5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201b7480 con 0x559529fa72f0 2024-05-27T08:17:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.346+0000 7f2d2efe5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201b66e0 con 0x559529faf4d0 2024-05-27T08:17:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.346+0000 7f2d2efe5740 10 client.0 _ll_register_callbacks cb 0x7f2d0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.346+0000 7f2d26000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 0x55952a3854e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56000/0 (socket says 172.21.15.63:56000) 2024-05-27T08:17:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.346+0000 7f2d26000640 1 -- 172.21.15.63:0/3815956043 learned_addr learned my addr 172.21.15.63:0/3815956043 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.346+0000 7f2d26000640 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529faf4d0 0x55952a380e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.347+0000 7f2d24c00640 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529fa72f0 0x55952a380830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.347+0000 7f2d26000640 1 -- 172.21.15.63:0/3815956043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529faf4d0 msgr2=0x55952a380e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.347+0000 7f2d26000640 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529faf4d0 0x55952a380e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.347+0000 7f2d26000640 1 -- 172.21.15.63:0/3815956043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529fa72f0 msgr2=0x55952a380830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.347+0000 7f2d26000640 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529fa72f0 0x55952a380830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.348+0000 7f2d26000640 1 -- 172.21.15.63:0/3815956043 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55952a3815f0 con 0x559529f8ed80 2024-05-27T08:17:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.349+0000 7f2d26000640 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 0x55952a3854e0 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800eda0 tx=0x7f2d1800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba61478fa7d3a95 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.349+0000 7f2d06a00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:17:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.351+0000 7f2d06a00640 1 -- 172.21.15.63:0/3815956043 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18016040 con 0x559529f8ed80 2024-05-27T08:17:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.351+0000 7f2d06a00640 1 -- 172.21.15.63:0/3815956043 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(169..169 src has 1..169) ==== 9539+0+0 (secure 0 0 0) 0x7f2d18003a30 con 0x559529f8ed80 2024-05-27T08:17:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.351+0000 7f2d2efe5740 1 -- 172.21.15.63:0/3815956043 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d18028af0 con 0x559529f8ed80 2024-05-27T08:17:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.352+0000 7f2d06a00640 1 -- 172.21.15.63:0/3815956043 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2d18003c30 con 0x559529f8ed80 2024-05-27T08:17:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.353+0000 7f2d2efe5740 10 client.23577 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.353+0000 7f2d2efe5740 1 -- 172.21.15.63:0/3815956043 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55952a3f73d0 con 0x559529f8ed80 2024-05-27T08:17:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.353+0000 7f2d06a00640 1 -- 172.21.15.63:0/3815956043 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d18028af0 con 0x559529f8ed80 2024-05-27T08:17:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.354+0000 7f2d06a00640 1 -- 172.21.15.63:0/3815956043 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2d1800f040 con 0x559529f8ed80 2024-05-27T08:17:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.355+0000 7f2d2efe5740 10 client.23577 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.355+0000 7f2d2efe5740 10 client.23577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.355+0000 7f2d2efe5740 20 client.23577 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.355+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559529fd01b0 0x55952a3f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.356+0000 7f2d2efe5740 4 client.23577 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:17:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.356+0000 7f2d2efe5740 1 -- 172.21.15.63:0/3815956043 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55952a3770b0 con 0x559529fd01b0 2024-05-27T08:17:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.356+0000 7f2d24c00640 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559529fd01b0 0x55952a3f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.359+0000 7f2d24c00640 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559529fd01b0 0x55952a3f9760 crc :-1 s=READY pgs=763 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc858806d338bc2f server_cookie=171b7919bcec7b1 in_seq=0 out_seq=0 2024-05-27T08:17:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.359+0000 7f2d06a00640 10 client.23577 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:43.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:42 smithi086 ceph-mon[34687]: pgmap v1403: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 2.89k op/s 2024-05-27T08:17:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:42 smithi149 ceph-mon[35408]: pgmap v1403: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 2.89k op/s 2024-05-27T08:17:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.908+0000 7f2d06a00640 1 -- 172.21.15.63:0/3815956043 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55952a3770b0 con 0x559529fd01b0 2024-05-27T08:17:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.908+0000 7f2d06a00640 10 client.23577 handle_command_reply: tid=0 2024-05-27T08:17:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.908+0000 7f2d2efe5740 1 client.23577 shutdown 2024-05-27T08:17:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.908+0000 7f2d2efe5740 1 -- 172.21.15.63:0/3815956043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 msgr2=0x55952a3854e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.908+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 0x55952a3854e0 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800eda0 tx=0x7f2d1800e6c0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.909+0000 7f2d2efe5740 1 -- 172.21.15.63:0/3815956043 shutdown_connections 2024-05-27T08:17:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.909+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x559529fd01b0 0x55952a3f9760 crc :-1 s=READY pgs=763 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.909+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559529faf4d0 0x55952a380e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.909+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559529fa72f0 0x55952a380830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.909+0000 7f2d2efe5740 1 --2- 172.21.15.63:0/3815956043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559529f8ed80 0x55952a3854e0 unknown :-1 s=CLOSED pgs=2965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.909+0000 7f2d2efe5740 1 -- 172.21.15.63:0/3815956043 >> 172.21.15.63:0/3815956043 conn(0x559529fc4300 msgr2=0x55952a37e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.909+0000 7f2d06a00640 0 client.23577 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.910+0000 7f2d2efe5740 1 -- 172.21.15.63:0/3815956043 shutdown_connections 2024-05-27T08:17:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.910+0000 7f2d2efe5740 1 -- 172.21.15.63:0/3815956043 wait complete. 2024-05-27T08:17:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.911+0000 7f2d2efe5740 20 client.23577 trim_cache size 0 max 16384 2024-05-27T08:17:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:17:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:17:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:17:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.918+0000 7f2d28000640 1 -- 172.21.15.63:0/350583372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d080b2c90 msgr2=0x7f2d080b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.918+0000 7f2d28000640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d080b2c90 0x7f2d080b51a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x55952a3703a0 tx=0x7f2d1401f000 comp rx=0 tx=0).stop 2024-05-27T08:17:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.918+0000 7f2d28000640 1 -- 172.21.15.63:0/350583372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d2010a490 msgr2=0x7f2d201a2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.918+0000 7f2d28000640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d2010a490 0x7f2d201a2f90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9f0 tx=0x7f2d1400eeb0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.920+0000 7f2d28000640 1 -- 172.21.15.63:0/350583372 shutdown_connections 2024-05-27T08:17:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.920+0000 7f2d28000640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2d080b2c90 0x7f2d080b51a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.920+0000 7f2d28000640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d20112900 0x7f2d201a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.921+0000 7f2d28000640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d20111de0 0x7f2d201a34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.921+0000 7f2d28000640 1 --2- 172.21.15.63:0/350583372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d2010a490 0x7f2d201a2f90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.921+0000 7f2d28000640 1 -- 172.21.15.63:0/350583372 >> 172.21.15.63:0/350583372 conn(0x7f2d2006e8f0 msgr2=0x7f2d20073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.922+0000 7f2d28000640 1 -- 172.21.15.63:0/350583372 shutdown_connections 2024-05-27T08:17:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:43.922+0000 7f2d28000640 1 -- 172.21.15.63:0/350583372 wait complete. 2024-05-27T08:17:45.355 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:44 smithi063 ceph-mon[31613]: pgmap v1404: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-05-27T08:17:45.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:44 smithi086 ceph-mon[34687]: pgmap v1404: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-05-27T08:17:45.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:45 smithi149 ceph-mon[35408]: pgmap v1404: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.9 MiB/s wr, 2.51k op/s 2024-05-27T08:17:45.540 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:17:45.540 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:17:45.541 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:17:45.541 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:17:45.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.018+0000 7f1f4da00700 1 -- 172.21.15.63:0/2382612685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f48075af0 msgr2=0x7f1f48075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:45.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.018+0000 7f1f4da00700 1 --2- 172.21.15.63:0/2382612685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f48075af0 0x7f1f48075f00 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009990 tx=0x7f1f38009ca0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.020+0000 7f1f4da00700 1 -- 172.21.15.63:0/2382612685 shutdown_connections 2024-05-27T08:17:45.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.020+0000 7f1f4da00700 1 --2- 172.21.15.63:0/2382612685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f48110bf0 0x7f1f48113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.020+0000 7f1f4da00700 1 --2- 172.21.15.63:0/2382612685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f48075af0 0x7f1f48075f00 unknown :-1 s=CLOSED pgs=2966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.020+0000 7f1f4da00700 1 --2- 172.21.15.63:0/2382612685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f48076440 0x7f1f481106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.020+0000 7f1f4da00700 1 -- 172.21.15.63:0/2382612685 >> 172.21.15.63:0/2382612685 conn(0x7f1f48100030 msgr2=0x7f1f48102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:45.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.020+0000 7f1f4da00700 1 -- 172.21.15.63:0/2382612685 shutdown_connections 2024-05-27T08:17:45.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.020+0000 7f1f4da00700 1 -- 172.21.15.63:0/2382612685 wait complete. 2024-05-27T08:17:45.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.022+0000 7f1f4da00700 1 Processor -- start 2024-05-27T08:17:45.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.022+0000 7f1f4da00700 1 -- start start 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.023+0000 7f1f4da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f48075af0 0x7f1f48104310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.023+0000 7f1f4da00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f48076440 0x7f1f48104850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.023+0000 7f1f4da00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f48110bf0 0x7f1f48104d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.024+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f48075af0 0x7f1f48104310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.024+0000 7f1f4da00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48116640 con 0x7f1f48110bf0 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.024+0000 7f1f4da00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481164c0 con 0x7f1f48075af0 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.024+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f48075af0 0x7f1f48104310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:43822/0 (socket says 172.21.15.63:43822) 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.024+0000 7f1f46a00700 1 -- 172.21.15.63:0/2633160607 learned_addr learned my addr 172.21.15.63:0/2633160607 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.024+0000 7f1f4da00700 1 -- 172.21.15.63:0/2633160607 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481167c0 con 0x7f1f48076440 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.024+0000 7f1f47400700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f48110bf0 0x7f1f48104d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.025+0000 7f1f46000700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f48076440 0x7f1f48104850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.025+0000 7f1f46a00700 1 -- 172.21.15.63:0/2633160607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f48076440 msgr2=0x7f1f48104850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:45.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.025+0000 7f1f46a00700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f48076440 0x7f1f48104850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.025+0000 7f1f46a00700 1 -- 172.21.15.63:0/2633160607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f48110bf0 msgr2=0x7f1f48104d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.025+0000 7f1f46a00700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f48110bf0 0x7f1f48104d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.025+0000 7f1f46a00700 1 -- 172.21.15.63:0/2633160607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f481ad5f0 con 0x7f1f48075af0 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.026+0000 7f1f46a00700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f48075af0 0x7f1f48104310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800b5c0 tx=0x7f1f380057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.026+0000 7f1f3f400700 1 -- 172.21.15.63:0/2633160607 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38027070 con 0x7f1f48075af0 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.026+0000 7f1f4da00700 1 -- 172.21.15.63:0/2633160607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f481ad820 con 0x7f1f48075af0 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.027+0000 7f1f4da00700 1 -- 172.21.15.63:0/2633160607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481add50 con 0x7f1f48075af0 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.027+0000 7f1f3f400700 1 -- 172.21.15.63:0/2633160607 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f380040b0 con 0x7f1f48075af0 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.030+0000 7f1f3f400700 1 -- 172.21.15.63:0/2633160607 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38027070 con 0x7f1f48075af0 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.039+0000 7f1f3f400700 1 -- 172.21.15.63:0/2633160607 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1f3800f460 con 0x7f1f48075af0 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.042+0000 7f1f3f400700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f2c0b2df0 0x7f1f2c0b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.044+0000 7f1f46000700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f2c0b2df0 0x7f1f2c0b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.044+0000 7f1f3f400700 1 -- 172.21.15.63:0/2633160607 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 9539+0+0 (secure 0 0 0) 0x7f1f3801e030 con 0x7f1f48075af0 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.349+0000 7f1f46000700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f2c0b2df0 0x7f1f2c0b5300 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1f34005950 tx=0x7f1f340058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.353+0000 7f1f5c4ddb80 1 Processor -- start 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.354+0000 7f1f5c4ddb80 1 -- start start 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.355+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce81167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.355+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8110870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.356+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce81167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.356+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce81167f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:57954/0 (socket says 172.21.15.63:57954) 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.356+0000 7f1f46a00700 1 -- 172.21.15.63:0/3015019882 learned_addr learned my addr 172.21.15.63:0/3015019882 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.356+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce81111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.356+0000 7f1f46000700 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8110870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.358+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/3015019882 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557ce803ee40 con 0x557ce8110db0 2024-05-27T08:17:45.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.358+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/3015019882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ce802e9e0 con 0x557ce8116d30 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.359+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/3015019882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557ce8032a20 con 0x557ce7f91e40 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.360+0000 7f1f47400700 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce81111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.360+0000 7f1f46000700 1 -- 172.21.15.63:0/3015019882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 msgr2=0x557ce81167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.360+0000 7f1f46000700 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce81167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.360+0000 7f1f46000700 1 -- 172.21.15.63:0/3015019882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 msgr2=0x557ce81111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.360+0000 7f1f46000700 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce81111c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.360+0000 7f1f46000700 1 -- 172.21.15.63:0/3015019882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce7ff7420 con 0x557ce8116d30 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.361+0000 7f1f46a00700 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce81167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.367+0000 7f1f46000700 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8110870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f481035f0 tx=0x7f1f34012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8548a12a31796866 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.367+0000 7f1f33e00700 1 -- 172.21.15.63:0/3015019882 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34022070 con 0x557ce8116d30 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.368+0000 7f1f33e00700 1 -- 172.21.15.63:0/3015019882 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f3401e450 con 0x557ce8116d30 2024-05-27T08:17:45.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.368+0000 7f1f33e00700 1 -- 172.21.15.63:0/3015019882 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3401d610 con 0x557ce8116d30 2024-05-27T08:17:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.369+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/3015019882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 msgr2=0x557ce8110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.369+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8110870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f481035f0 tx=0x7f1f34012000 comp rx=0 tx=0).stop 2024-05-27T08:17:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.370+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/3015019882 shutdown_connections 2024-05-27T08:17:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.370+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.370+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce81167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.370+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/3015019882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8110870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.370+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/3015019882 >> 172.21.15.63:0/3015019882 conn(0x557ce7ff1fc0 msgr2=0x557ce810b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.370+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/3015019882 shutdown_connections 2024-05-27T08:17:45.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.370+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/3015019882 wait complete. 2024-05-27T08:17:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.371+0000 7f1f5c4ddb80 1 -- start start 2024-05-27T08:17:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.371+0000 7f1f5c4ddb80 1 Processor -- start 2024-05-27T08:17:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce7f91e40 0x557ce811a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce811aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce8116d30 0x557ce811af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f5c4ddb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557ce7f34fa0 con 0x557ce8110db0 2024-05-27T08:17:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f5c4ddb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ce7fcf760 con 0x557ce7f91e40 2024-05-27T08:17:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f5c4ddb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557ce811d800 con 0x557ce8116d30 2024-05-27T08:17:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f5c4ddb80 10 client.0 _ll_register_callbacks cb 0x557ce7fd8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:45.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f46000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce811aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce7f91e40 0x557ce811a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f46a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce7f91e40 0x557ce811a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:43850/0 (socket says 172.21.15.63:43850) 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.372+0000 7f1f46a00700 1 -- 172.21.15.63:0/2317781263 learned_addr learned my addr 172.21.15.63:0/2317781263 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.373+0000 7f1f46a00700 1 -- 172.21.15.63:0/2317781263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce8116d30 msgr2=0x557ce811af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.373+0000 7f1f46a00700 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce8116d30 0x557ce811af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.373+0000 7f1f46a00700 1 -- 172.21.15.63:0/2317781263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 msgr2=0x557ce811aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.373+0000 7f1f46a00700 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce811aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.373+0000 7f1f46a00700 1 -- 172.21.15.63:0/2317781263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce7f7e8d0 con 0x557ce7f91e40 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.374+0000 7f1f46a00700 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce7f91e40 0x557ce811a4d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f380a8080 tx=0x7f1f380b3e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ee08902d317c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.374+0000 7f1f30c00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:17:45.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.374+0000 7f1f30c00700 1 -- 172.21.15.63:0/2317781263 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f380a7d40 con 0x557ce7f91e40 2024-05-27T08:17:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.374+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2317781263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3401e450 con 0x557ce7f91e40 2024-05-27T08:17:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.374+0000 7f1f30c00700 1 -- 172.21.15.63:0/2317781263 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(169..169 src has 1..169) ==== 9539+0+0 (secure 0 0 0) 0x7f1f380a7b40 con 0x557ce7f91e40 2024-05-27T08:17:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.374+0000 7f1f30c00700 1 -- 172.21.15.63:0/2317781263 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1f380a7d40 con 0x557ce7f91e40 2024-05-27T08:17:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.375+0000 7f1f5c4ddb80 10 client.42446 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.375+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2317781263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557ce7fff060 con 0x557ce7f91e40 2024-05-27T08:17:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.375+0000 7f1f30c00700 1 -- 172.21.15.63:0/2317781263 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f3401e450 con 0x557ce7f91e40 2024-05-27T08:17:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.375+0000 7f1f30c00700 1 -- 172.21.15.63:0/2317781263 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1f380e1bb0 con 0x557ce7f91e40 2024-05-27T08:17:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.375+0000 7f1f5c4ddb80 10 client.42446 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.375+0000 7f1f5c4ddb80 10 client.42446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:45.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.375+0000 7f1f5c4ddb80 20 client.42446 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.375+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ce812a7e0 0x557ce812cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.376+0000 7f1f5c4ddb80 4 client.42446 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:17:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.376+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2317781263 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ce8040d90 con 0x557ce812a7e0 2024-05-27T08:17:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.376+0000 7f1f46000700 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ce812a7e0 0x557ce812cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.376+0000 7f1f46000700 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ce812a7e0 0x557ce812cc90 crc :-1 s=READY pgs=764 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=583eb10316b514fb server_cookie=f1af8f3cb30f2708 in_seq=0 out_seq=0 2024-05-27T08:17:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.376+0000 7f1f30c00700 10 client.42446 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.377+0000 7f1f30c00700 1 -- 172.21.15.63:0/2317781263 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557ce8040d90 con 0x557ce812a7e0 2024-05-27T08:17:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.377+0000 7f1f30c00700 10 client.42446 handle_command_reply: tid=0 2024-05-27T08:17:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 client.42446 shutdown 2024-05-27T08:17:45.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2317781263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce7f91e40 msgr2=0x557ce811a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce7f91e40 0x557ce811a4d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f380a8080 tx=0x7f1f380b3e20 comp rx=0 tx=0).stop 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2317781263 shutdown_connections 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ce812a7e0 0x557ce812cc90 crc :-1 s=READY pgs=764 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce8116d30 0x557ce811af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce7f91e40 0x557ce811a4d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2317781263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce811aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2317781263 >> 172.21.15.63:0/2317781263 conn(0x557ce7ff1fc0 msgr2=0x557ce810b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f30c00700 0 client.42446 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2317781263 shutdown_connections 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.378+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2317781263 wait complete. 2024-05-27T08:17:45.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.379+0000 7f1f5c4ddb80 20 client.42446 trim_cache size 0 max 16384 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.423+0000 7f1f5c4ddb80 1 Processor -- start 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.423+0000 7f1f5c4ddb80 1 -- start start 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.424+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce7f91e40 0x557ce8110870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.424+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce8110db0 0x557ce811c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.424+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8126a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.424+0000 7f1f5c4ddb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481b9470 con 0x557ce7f91e40 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.424+0000 7f1f5c4ddb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481b6d00 con 0x557ce8116d30 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.424+0000 7f1f5c4ddb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481b4b50 con 0x557ce8110db0 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.424+0000 7f1f46000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce8110db0 0x557ce811c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.424+0000 7f1f46000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce8110db0 0x557ce811c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:57964/0 (socket says 172.21.15.63:57964) 2024-05-27T08:17:45.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.424+0000 7f1f46000700 1 -- 172.21.15.63:0/1015961871 learned_addr learned my addr 172.21.15.63:0/1015961871 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:45.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.425+0000 7f1f46a00700 1 --2- 172.21.15.63:0/1015961871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce7f91e40 0x557ce8110870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.426+0000 7f1f46a00700 1 -- 172.21.15.63:0/1015961871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce8110db0 msgr2=0x557ce811c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:45.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.426+0000 7f1f47400700 1 --2- 172.21.15.63:0/1015961871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8126a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.427+0000 7f1f46a00700 1 --2- 172.21.15.63:0/1015961871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce8110db0 0x557ce811c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.427+0000 7f1f46a00700 1 -- 172.21.15.63:0/1015961871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 msgr2=0x557ce8126a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:45.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.427+0000 7f1f46a00700 1 --2- 172.21.15.63:0/1015961871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8126a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.427+0000 7f1f46a00700 1 -- 172.21.15.63:0/1015961871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce81197a0 con 0x557ce7f91e40 2024-05-27T08:17:45.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.427+0000 7f1f47400700 1 --2- 172.21.15.63:0/1015961871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8126a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:17:45.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.428+0000 7f1f46a00700 1 --2- 172.21.15.63:0/1015961871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce7f91e40 0x557ce8110870 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7f1f380a85c0 tx=0x7f1f38009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30df0d73c45cf4f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:45.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.429+0000 7f1f33400700 1 -- 172.21.15.63:0/1015961871 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38027070 con 0x557ce7f91e40 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.429+0000 7f1f33400700 1 -- 172.21.15.63:0/1015961871 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f380af900 con 0x557ce7f91e40 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.429+0000 7f1f33400700 1 -- 172.21.15.63:0/1015961871 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f380a7ee0 con 0x557ce7f91e40 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.430+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/1015961871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce7f91e40 msgr2=0x557ce8110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.430+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/1015961871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce7f91e40 0x557ce8110870 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7f1f380a85c0 tx=0x7f1f38009ca0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.430+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/1015961871 shutdown_connections 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.430+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/1015961871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8126a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.430+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/1015961871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce7f91e40 0x557ce8110870 unknown :-1 s=CLOSED pgs=2967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.430+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/1015961871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce8110db0 0x557ce811c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.431+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/1015961871 >> 172.21.15.63:0/1015961871 conn(0x557ce7ff1fc0 msgr2=0x557ce8114210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.431+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/1015961871 shutdown_connections 2024-05-27T08:17:45.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.431+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/1015961871 wait complete. 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.432+0000 7f1f5c4ddb80 1 -- start start 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.432+0000 7f1f5c4ddb80 1 Processor -- start 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.434+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce812ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.434+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce812d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.434+0000 7f1f5c4ddb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8193170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.434+0000 7f1f5c4ddb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481b51c0 con 0x557ce8110db0 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.434+0000 7f1f5c4ddb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481b6630 con 0x557ce8116d30 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.434+0000 7f1f5c4ddb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481b5890 con 0x557ce7f91e40 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.434+0000 7f1f5c4ddb80 10 client.0 _ll_register_callbacks cb 0x557ce7fd8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.435+0000 7f1f47400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8193170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.435+0000 7f1f47400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8193170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:43856/0 (socket says 172.21.15.63:43856) 2024-05-27T08:17:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.435+0000 7f1f47400700 1 -- 172.21.15.63:0/2403503348 learned_addr learned my addr 172.21.15.63:0/2403503348 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.436+0000 7f1f46a00700 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce812ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.437+0000 7f1f46000700 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce812d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.439+0000 7f1f46a00700 1 -- 172.21.15.63:0/2403503348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 msgr2=0x557ce8193170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.439+0000 7f1f46a00700 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8193170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.439+0000 7f1f46a00700 1 -- 172.21.15.63:0/2403503348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 msgr2=0x557ce812d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.439+0000 7f1f46a00700 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce812d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.439+0000 7f1f46a00700 1 -- 172.21.15.63:0/2403503348 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce81936e0 con 0x557ce7f91e40 2024-05-27T08:17:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.440+0000 7f1f46a00700 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce812ce30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005b40 tx=0x7f1f380a7650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7298e5655d96fdb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.440+0000 7f1f30c00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.441+0000 7f1f30c00700 1 -- 172.21.15.63:0/2403503348 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38027070 con 0x557ce7f91e40 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.441+0000 7f1f30c00700 1 -- 172.21.15.63:0/2403503348 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(169..169 src has 1..169) ==== 9539+0+0 (secure 0 0 0) 0x7f1f380b40c0 con 0x557ce7f91e40 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.442+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2403503348 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f380af900 con 0x557ce7f91e40 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.443+0000 7f1f30c00700 1 -- 172.21.15.63:0/2403503348 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1f380a80b0 con 0x557ce7f91e40 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.443+0000 7f1f5c4ddb80 10 client.32281 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.443+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2403503348 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557ce81936e0 con 0x557ce7f91e40 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.444+0000 7f1f30c00700 1 -- 172.21.15.63:0/2403503348 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f38003e30 con 0x557ce7f91e40 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.447+0000 7f1f30c00700 1 -- 172.21.15.63:0/2403503348 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1f380aa7b0 con 0x557ce7f91e40 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.448+0000 7f1f5c4ddb80 10 client.32281 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.448+0000 7f1f5c4ddb80 10 client.32281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.448+0000 7f1f5c4ddb80 20 client.32281 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.448+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ce8194aa0 0x557ce8196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.448+0000 7f1f5c4ddb80 4 client.32281 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.448+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2403503348 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1f380af900 con 0x557ce8194aa0 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.449+0000 7f1f46000700 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ce8194aa0 0x557ce8196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.450+0000 7f1f46000700 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ce8194aa0 0x557ce8196ec0 crc :-1 s=READY pgs=765 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99540ec755dec4fd server_cookie=70dd41bdd89af6f0 in_seq=0 out_seq=0 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.450+0000 7f1f30c00700 10 client.32281 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.450+0000 7f1f30c00700 1 -- 172.21.15.63:0/2403503348 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f1f380af900 con 0x557ce8194aa0 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.450+0000 7f1f30c00700 10 client.32281 handle_command_reply: tid=0 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.450+0000 7f1f5c4ddb80 1 client.32281 shutdown 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.451+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2403503348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 msgr2=0x557ce812ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:45.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.451+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce812ce30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005b40 tx=0x7f1f380a7650 comp rx=0 tx=0).stop 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.451+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2403503348 shutdown_connections 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.451+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x557ce8194aa0 0x557ce8196ec0 crc :-1 s=READY pgs=765 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.451+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce8116d30 0x557ce8193170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.451+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557ce7f91e40 0x557ce812ce30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.451+0000 7f1f5c4ddb80 1 --2- 172.21.15.63:0/2403503348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557ce8110db0 0x557ce812d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.451+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2403503348 >> 172.21.15.63:0/2403503348 conn(0x557ce7ff1fc0 msgr2=0x557ce8115dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.451+0000 7f1f30c00700 0 client.32281 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.451+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2403503348 shutdown_connections 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.453+0000 7f1f5c4ddb80 1 -- 172.21.15.63:0/2403503348 wait complete. 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.453+0000 7f1f5c4ddb80 20 client.32281 trim_cache size 0 max 16384 2024-05-27T08:17:45.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.454+0000 7f1f4da00700 1 -- 172.21.15.63:0/2633160607 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f2c0b2df0 msgr2=0x7f1f2c0b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.454+0000 7f1f4da00700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f2c0b2df0 0x7f1f2c0b5300 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1f34005950 tx=0x7f1f340058e0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.454+0000 7f1f4da00700 1 -- 172.21.15.63:0/2633160607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f48075af0 msgr2=0x7f1f48104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.454+0000 7f1f4da00700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f48075af0 0x7f1f48104310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800b5c0 tx=0x7f1f380057c0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.455+0000 7f1f4da00700 1 -- 172.21.15.63:0/2633160607 shutdown_connections 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.455+0000 7f1f4da00700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f1f2c0b2df0 0x7f1f2c0b5300 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.455+0000 7f1f4da00700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f48110bf0 0x7f1f48104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.455+0000 7f1f4da00700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f48075af0 0x7f1f48104310 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.455+0000 7f1f4da00700 1 --2- 172.21.15.63:0/2633160607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f48076440 0x7f1f48104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.455+0000 7f1f4da00700 1 -- 172.21.15.63:0/2633160607 >> 172.21.15.63:0/2633160607 conn(0x7f1f48100030 msgr2=0x7f1f48100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.455+0000 7f1f4da00700 1 -- 172.21.15.63:0/2633160607 shutdown_connections 2024-05-27T08:17:45.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:45.455+0000 7f1f4da00700 1 -- 172.21.15.63:0/2633160607 wait complete. 2024-05-27T08:17:47.385 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:46 smithi063 ceph-mon[31613]: pgmap v1405: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-05-27T08:17:47.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:46 smithi086 ceph-mon[34687]: pgmap v1405: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-05-27T08:17:47.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:46 smithi149 ceph-mon[35408]: pgmap v1405: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 10 MiB/s wr, 2.65k op/s 2024-05-27T08:17:48.035 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:17:48.035 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:17:49.261 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:48 smithi086 ceph-mon[34687]: pgmap v1406: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-05-27T08:17:49.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:48 smithi149 ceph-mon[35408]: pgmap v1406: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-05-27T08:17:49.470 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:48 smithi063 ceph-mon[31613]: pgmap v1406: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-05-27T08:17:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:17:51.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:51 smithi063 ceph-mon[31613]: pgmap v1407: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.0 MiB/s wr, 2.29k op/s 2024-05-27T08:17:51.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:51 smithi086 ceph-mon[34687]: pgmap v1407: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.0 MiB/s wr, 2.29k op/s 2024-05-27T08:17:51.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:51 smithi149 ceph-mon[35408]: pgmap v1407: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 9.0 MiB/s wr, 2.29k op/s 2024-05-27T08:17:53.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:52 smithi063 ceph-mon[31613]: pgmap v1408: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 11 MiB/s wr, 2.74k op/s 2024-05-27T08:17:53.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:52 smithi086 ceph-mon[34687]: pgmap v1408: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 11 MiB/s wr, 2.74k op/s 2024-05-27T08:17:53.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:52 smithi149 ceph-mon[35408]: pgmap v1408: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 11 MiB/s wr, 2.74k op/s 2024-05-27T08:17:55.265 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:54 smithi063 ceph-mon[31613]: pgmap v1409: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.3 MiB/s wr, 2.37k op/s 2024-05-27T08:17:55.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:54 smithi086 ceph-mon[34687]: pgmap v1409: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.3 MiB/s wr, 2.37k op/s 2024-05-27T08:17:55.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:54 smithi149 ceph-mon[35408]: pgmap v1409: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.3 MiB/s wr, 2.37k op/s 2024-05-27T08:17:57.219 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:56 smithi063 ceph-mon[31613]: pgmap v1410: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-05-27T08:17:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.386+0000 7f168a800640 1 -- 172.21.15.63:0/4110646702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167c008170 msgr2=0x7f167c008580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.386+0000 7f168a800640 1 --2- 172.21.15.63:0/4110646702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167c008170 0x7f167c008580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1678009900 tx=0x7f167802a490 comp rx=0 tx=0).stop 2024-05-27T08:17:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.388+0000 7f168a800640 1 -- 172.21.15.63:0/4110646702 shutdown_connections 2024-05-27T08:17:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.389+0000 7f168a800640 1 --2- 172.21.15.63:0/4110646702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c0a79c0 0x7f167c0a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.389+0000 7f168a800640 1 --2- 172.21.15.63:0/4110646702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f167c0a5060 0x7f167c0a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.390+0000 7f168a800640 1 --2- 172.21.15.63:0/4110646702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167c008170 0x7f167c008580 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.390+0000 7f168a800640 1 -- 172.21.15.63:0/4110646702 >> 172.21.15.63:0/4110646702 conn(0x7f167c00c160 msgr2=0x7f167c00c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.391+0000 7f168a800640 1 -- 172.21.15.63:0/4110646702 shutdown_connections 2024-05-27T08:17:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.392+0000 7f168a800640 1 -- 172.21.15.63:0/4110646702 wait complete. 2024-05-27T08:17:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.397+0000 7f168a800640 1 Processor -- start 2024-05-27T08:17:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.398+0000 7f168a800640 1 -- start start 2024-05-27T08:17:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.399+0000 7f168a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f167c008170 0x7f167c139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.399+0000 7f168a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167c0a5060 0x7f167c139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.399+0000 7f168a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c0a79c0 0x7f167c133980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.400+0000 7f1682a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167c0a5060 0x7f167c139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.400+0000 7f168a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f167c0ad100 con 0x7f167c0a79c0 2024-05-27T08:17:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.400+0000 7f168a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f167c0acf80 con 0x7f167c0a5060 2024-05-27T08:17:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.400+0000 7f1682a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167c0a5060 0x7f167c139cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46714/0 (socket says 172.21.15.63:46714) 2024-05-27T08:17:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.400+0000 7f1682a00640 1 -- 172.21.15.63:0/4032348568 learned_addr learned my addr 172.21.15.63:0/4032348568 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.400+0000 7f168a800640 1 -- 172.21.15.63:0/4032348568 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f167c0ad280 con 0x7f167c008170 2024-05-27T08:17:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.400+0000 7f1683400640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f167c008170 0x7f167c139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.401+0000 7f1683e00640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c0a79c0 0x7f167c133980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.401+0000 7f1683400640 1 -- 172.21.15.63:0/4032348568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167c0a5060 msgr2=0x7f167c139cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.401+0000 7f1683400640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167c0a5060 0x7f167c139cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.401+0000 7f1683400640 1 -- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c0a79c0 msgr2=0x7f167c133980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.402+0000 7f1683400640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c0a79c0 0x7f167c133980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.402+0000 7f1683400640 1 -- 172.21.15.63:0/4032348568 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c134020 con 0x7f167c008170 2024-05-27T08:17:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.403+0000 7f1683e00640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c0a79c0 0x7f167c133980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:17:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.404+0000 7f1683400640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f167c008170 0x7f167c139770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1678009850 tx=0x7f1678003d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.407+0000 7f1677e00640 1 -- 172.21.15.63:0/4032348568 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167803f070 con 0x7f167c008170 2024-05-27T08:17:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.407+0000 7f1677e00640 1 -- 172.21.15.63:0/4032348568 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f167803bd00 con 0x7f167c008170 2024-05-27T08:17:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.407+0000 7f1677e00640 1 -- 172.21.15.63:0/4032348568 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678035030 con 0x7f167c008170 2024-05-27T08:17:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.407+0000 7f168a800640 1 -- 172.21.15.63:0/4032348568 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167c1342b0 con 0x7f167c008170 2024-05-27T08:17:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.408+0000 7f168a800640 1 -- 172.21.15.63:0/4032348568 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167c134720 con 0x7f167c008170 2024-05-27T08:17:57.418 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.416+0000 7f1691823740 1 Processor -- start 2024-05-27T08:17:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.419+0000 7f1677e00640 1 -- 172.21.15.63:0/4032348568 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1678042070 con 0x7f167c008170 2024-05-27T08:17:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.421+0000 7f1691823740 1 -- start start 2024-05-27T08:17:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.422+0000 7f1691823740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c957a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.422+0000 7f1677e00640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f166c0b2b70 0x7f166c0b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.422+0000 7f1677e00640 1 -- 172.21.15.63:0/4032348568 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 9539+0+0 (secure 0 0 0) 0x7f16781032b0 con 0x7f167c008170 2024-05-27T08:17:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.423+0000 7f1691823740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c58c2f0 0x56264c957fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.423+0000 7f1683e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c957a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.423+0000 7f1683400640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f166c0b2b70 0x7f166c0b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.424+0000 7f1683e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c957a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46724/0 (socket says 172.21.15.63:46724) 2024-05-27T08:17:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.425+0000 7f1683400640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f166c0b2b70 0x7f166c0b5080 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1678100600 tx=0x7f1678003f70 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.425+0000 7f1691823740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c5944d0 0x56264c958510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.425+0000 7f1691823740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56264c8c8b90 con 0x56264c58c2f0 2024-05-27T08:17:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.425+0000 7f1691823740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56264c8b8730 con 0x56264c5b51b0 2024-05-27T08:17:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.425+0000 7f1691823740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56264c8bc770 con 0x56264c5944d0 2024-05-27T08:17:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.426+0000 7f1683e00640 1 -- 172.21.15.63:0/1771355966 learned_addr learned my addr 172.21.15.63:0/1771355966 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.426+0000 7f1683e00640 1 -- 172.21.15.63:0/1771355966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c5944d0 msgr2=0x56264c958510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.426+0000 7f1683e00640 1 --2- 172.21.15.63:0/1771355966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c5944d0 0x56264c958510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.426+0000 7f1683e00640 1 -- 172.21.15.63:0/1771355966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c58c2f0 msgr2=0x56264c957fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.426+0000 7f1683e00640 1 --2- 172.21.15.63:0/1771355966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c58c2f0 0x56264c957fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.426+0000 7f1683e00640 1 -- 172.21.15.63:0/1771355966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56264c8518b0 con 0x56264c5b51b0 2024-05-27T08:17:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.427+0000 7f1683e00640 1 --2- 172.21.15.63:0/1771355966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c957a90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1684068be0 tx=0x7f168406bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b5c13f266fc30b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.427+0000 7f1674c00640 1 -- 172.21.15.63:0/1771355966 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684080410 con 0x56264c5b51b0 2024-05-27T08:17:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.427+0000 7f1674c00640 1 -- 172.21.15.63:0/1771355966 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1684080a40 con 0x56264c5b51b0 2024-05-27T08:17:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.427+0000 7f1674c00640 1 -- 172.21.15.63:0/1771355966 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168407fa40 con 0x56264c5b51b0 2024-05-27T08:17:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.429+0000 7f1691823740 1 -- 172.21.15.63:0/1771355966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 msgr2=0x56264c957a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.429+0000 7f1691823740 1 --2- 172.21.15.63:0/1771355966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c957a90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1684068be0 tx=0x7f168406bd60 comp rx=0 tx=0).stop 2024-05-27T08:17:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.429+0000 7f1691823740 1 -- 172.21.15.63:0/1771355966 shutdown_connections 2024-05-27T08:17:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.429+0000 7f1691823740 1 --2- 172.21.15.63:0/1771355966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c5944d0 0x56264c958510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.429+0000 7f1691823740 1 --2- 172.21.15.63:0/1771355966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c58c2f0 0x56264c957fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.429+0000 7f1691823740 1 --2- 172.21.15.63:0/1771355966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c957a90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.429+0000 7f1691823740 1 -- 172.21.15.63:0/1771355966 >> 172.21.15.63:0/1771355966 conn(0x56264c5a9300 msgr2=0x56264c954240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.430+0000 7f1691823740 1 -- 172.21.15.63:0/1771355966 shutdown_connections 2024-05-27T08:17:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.430+0000 7f1691823740 1 -- 172.21.15.63:0/1771355966 wait complete. 2024-05-27T08:17:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.431+0000 7f1691823740 1 -- start start 2024-05-27T08:17:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.431+0000 7f1691823740 1 Processor -- start 2024-05-27T08:17:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.432+0000 7f1691823740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 0x56264c961c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.432+0000 7f1691823740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c5944d0 0x56264c9621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.432+0000 7f1691823740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c96df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.432+0000 7f1691823740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56264c59e790 con 0x56264c5944d0 2024-05-27T08:17:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.432+0000 7f1691823740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56264c860bc0 con 0x56264c5b51b0 2024-05-27T08:17:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.432+0000 7f1691823740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56264c964bc0 con 0x56264c58c2f0 2024-05-27T08:17:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.432+0000 7f1691823740 10 client.0 _ll_register_callbacks cb 0x56264c83e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.434+0000 7f1683e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 0x56264c961c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.434+0000 7f1683e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 0x56264c961c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:46056/0 (socket says 172.21.15.63:46056) 2024-05-27T08:17:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.434+0000 7f1683e00640 1 -- 172.21.15.63:0/1788960857 learned_addr learned my addr 172.21.15.63:0/1788960857 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.434+0000 7f1683e00640 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c96df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.435+0000 7f1682a00640 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c5944d0 0x56264c9621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.436+0000 7f1683e00640 1 -- 172.21.15.63:0/1788960857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 msgr2=0x56264c96df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.436+0000 7f1683e00640 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c96df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.436+0000 7f1683e00640 1 -- 172.21.15.63:0/1788960857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c5944d0 msgr2=0x56264c9621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.436+0000 7f1683e00640 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c5944d0 0x56264c9621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.436+0000 7f1683e00640 1 -- 172.21.15.63:0/1788960857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56264c5cd210 con 0x56264c58c2f0 2024-05-27T08:17:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.437+0000 7f1683e00640 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 0x56264c961c60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1684050b30 tx=0x7f1684068f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f26220cac7292671 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.437+0000 7f1669600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:17:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.439+0000 7f1669600640 1 -- 172.21.15.63:0/1788960857 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684070070 con 0x56264c58c2f0 2024-05-27T08:17:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.439+0000 7f1669600640 1 -- 172.21.15.63:0/1788960857 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(169..169 src has 1..169) ==== 9539+0+0 (secure 0 0 0) 0x7f168407fe70 con 0x56264c58c2f0 2024-05-27T08:17:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.440+0000 7f1691823740 1 -- 172.21.15.63:0/1788960857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1684080a40 con 0x56264c58c2f0 2024-05-27T08:17:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.443+0000 7f1669600640 1 -- 172.21.15.63:0/1788960857 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f168407f7e0 con 0x56264c58c2f0 2024-05-27T08:17:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.444+0000 7f1691823740 10 client.32287 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.444+0000 7f1691823740 1 -- 172.21.15.63:0/1788960857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56264c699620 con 0x56264c58c2f0 2024-05-27T08:17:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.444+0000 7f1669600640 1 -- 172.21.15.63:0/1788960857 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1684080a40 con 0x56264c58c2f0 2024-05-27T08:17:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.445+0000 7f1669600640 1 -- 172.21.15.63:0/1788960857 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f168406c7c0 con 0x56264c58c2f0 2024-05-27T08:17:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.446+0000 7f1691823740 10 client.32287 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.446+0000 7f1691823740 10 client.32287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.446+0000 7f1691823740 20 client.32287 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.446+0000 7f1691823740 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56264c573d80 0x56264c984d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.446+0000 7f1691823740 4 client.32287 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:17:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.446+0000 7f1691823740 1 -- 172.21.15.63:0/1788960857 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56264c8caae0 con 0x56264c573d80 2024-05-27T08:17:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.447+0000 7f1682a00640 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56264c573d80 0x56264c984d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.450+0000 7f1682a00640 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56264c573d80 0x56264c984d50 crc :-1 s=READY pgs=766 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab8bf64fb5829ea2 server_cookie=d99d78337c0a8918 in_seq=0 out_seq=0 2024-05-27T08:17:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.450+0000 7f1669600640 10 client.32287 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.451+0000 7f1669600640 1 -- 172.21.15.63:0/1788960857 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56264c8caae0 con 0x56264c573d80 2024-05-27T08:17:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.451+0000 7f1669600640 10 client.32287 handle_command_reply: tid=0 2024-05-27T08:17:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.452+0000 7f1691823740 1 client.32287 shutdown 2024-05-27T08:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.456+0000 7f1691823740 1 -- 172.21.15.63:0/1788960857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 msgr2=0x56264c961c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.456+0000 7f1691823740 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 0x56264c961c60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1684050b30 tx=0x7f1684068f80 comp rx=0 tx=0).stop 2024-05-27T08:17:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.457+0000 7f1691823740 1 -- 172.21.15.63:0/1788960857 shutdown_connections 2024-05-27T08:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.457+0000 7f1691823740 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56264c573d80 0x56264c984d50 crc :-1 s=READY pgs=766 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.457+0000 7f1691823740 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c96df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.457+0000 7f1691823740 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c5944d0 0x56264c9621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.457+0000 7f1691823740 1 --2- 172.21.15.63:0/1788960857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 0x56264c961c60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.457+0000 7f1691823740 1 -- 172.21.15.63:0/1788960857 >> 172.21.15.63:0/1788960857 conn(0x56264c5a9300 msgr2=0x56264c9578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:57.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:56 smithi086 ceph-mon[34687]: pgmap v1410: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-05-27T08:17:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.458+0000 7f1669600640 0 client.32287 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.458+0000 7f1691823740 1 -- 172.21.15.63:0/1788960857 shutdown_connections 2024-05-27T08:17:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.460+0000 7f1691823740 1 -- 172.21.15.63:0/1788960857 wait complete. 2024-05-27T08:17:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.460+0000 7f1691823740 20 client.32287 trim_cache size 0 max 16384 2024-05-27T08:17:57.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:56 smithi149 ceph-mon[35408]: pgmap v1410: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-05-27T08:17:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.518+0000 7f1691823740 1 Processor -- start 2024-05-27T08:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.522+0000 7f1691823740 1 -- start start 2024-05-27T08:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.522+0000 7f1691823740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 0x56264c964390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.522+0000 7f1691823740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c5944d0 0x56264c96de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.522+0000 7f1691823740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c95a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.522+0000 7f1691823740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f167c14d000 con 0x56264c5944d0 2024-05-27T08:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.522+0000 7f1691823740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f167c14a8d0 con 0x56264c5b51b0 2024-05-27T08:17:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.522+0000 7f1691823740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f167c148720 con 0x56264c58c2f0 2024-05-27T08:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.524+0000 7f1683e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c95a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.524+0000 7f1683e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 0x56264c964390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.524+0000 7f1683e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c95a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46744/0 (socket says 172.21.15.63:46744) 2024-05-27T08:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.524+0000 7f1683e00640 1 -- 172.21.15.63:0/4181880528 learned_addr learned my addr 172.21.15.63:0/4181880528 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.526+0000 7f1682a00640 1 --2- 172.21.15.63:0/4181880528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c5944d0 0x56264c96de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.526+0000 7f1683e00640 1 -- 172.21.15.63:0/4181880528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 msgr2=0x56264c964390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.526+0000 7f1683e00640 1 --2- 172.21.15.63:0/4181880528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 0x56264c964390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.526+0000 7f1683e00640 1 -- 172.21.15.63:0/4181880528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c5944d0 msgr2=0x56264c96de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.526+0000 7f1683e00640 1 --2- 172.21.15.63:0/4181880528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c5944d0 0x56264c96de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.526+0000 7f1683e00640 1 -- 172.21.15.63:0/4181880528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56264c963050 con 0x56264c5b51b0 2024-05-27T08:17:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.528+0000 7f1683e00640 1 --2- 172.21.15.63:0/4181880528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c95a100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1684066100 tx=0x7f16840517d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b68caa9ec01edaf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.529+0000 7f166be00640 1 -- 172.21.15.63:0/4181880528 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684070040 con 0x56264c5b51b0 2024-05-27T08:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.529+0000 7f166be00640 1 -- 172.21.15.63:0/4181880528 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f168407fa00 con 0x56264c5b51b0 2024-05-27T08:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.529+0000 7f166be00640 1 -- 172.21.15.63:0/4181880528 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684095b20 con 0x56264c5b51b0 2024-05-27T08:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.530+0000 7f1691823740 1 -- 172.21.15.63:0/4181880528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 msgr2=0x56264c95a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.530+0000 7f1691823740 1 --2- 172.21.15.63:0/4181880528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c95a100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1684066100 tx=0x7f16840517d0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.531+0000 7f1691823740 1 -- 172.21.15.63:0/4181880528 shutdown_connections 2024-05-27T08:17:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.531+0000 7f1691823740 1 --2- 172.21.15.63:0/4181880528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c95a100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.531+0000 7f1691823740 1 --2- 172.21.15.63:0/4181880528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c5944d0 0x56264c96de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.532+0000 7f1691823740 1 --2- 172.21.15.63:0/4181880528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c58c2f0 0x56264c964390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.532+0000 7f1691823740 1 -- 172.21.15.63:0/4181880528 >> 172.21.15.63:0/4181880528 conn(0x56264c5a9300 msgr2=0x56264c95c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:57.532 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.532+0000 7f1691823740 1 -- 172.21.15.63:0/4181880528 shutdown_connections 2024-05-27T08:17:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.532+0000 7f1691823740 1 -- 172.21.15.63:0/4181880528 wait complete. 2024-05-27T08:17:57.533 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.533+0000 7f1691823740 1 -- start start 2024-05-27T08:17:57.534 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.533+0000 7f1691823740 1 Processor -- start 2024-05-27T08:17:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.535+0000 7f1691823740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c58c2f0 0x56264c984ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.535+0000 7f1691823740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c5944d0 0x56264c9850e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.535+0000 7f1691823740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c985620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.535+0000 7f1691823740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f167c148d90 con 0x56264c58c2f0 2024-05-27T08:17:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.535+0000 7f1691823740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f167c14a200 con 0x56264c5b51b0 2024-05-27T08:17:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.535+0000 7f1691823740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f167c149460 con 0x56264c5944d0 2024-05-27T08:17:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.535+0000 7f1691823740 10 client.0 _ll_register_callbacks cb 0x7f1684069560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:17:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.536+0000 7f1683e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c985620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.536+0000 7f1683e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c985620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46758/0 (socket says 172.21.15.63:46758) 2024-05-27T08:17:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.536+0000 7f1683e00640 1 -- 172.21.15.63:0/3545260390 learned_addr learned my addr 172.21.15.63:0/3545260390 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:57.537 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.536+0000 7f1682a00640 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c5944d0 0x56264c9850e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.537+0000 7f1682a00640 1 -- 172.21.15.63:0/3545260390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 msgr2=0x56264c985620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.537+0000 7f1682a00640 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c985620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.537+0000 7f1682a00640 1 -- 172.21.15.63:0/3545260390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c58c2f0 msgr2=0x56264c984ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:17:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.537+0000 7f1682a00640 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c58c2f0 0x56264c984ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.537+0000 7f1682a00640 1 -- 172.21.15.63:0/3545260390 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56264c9eb1c0 con 0x56264c5944d0 2024-05-27T08:17:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.538+0000 7f1682a00640 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c5944d0 0x56264c9850e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x56264c955580 tx=0x7f1670011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42a059e25059ec44 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.538+0000 7f1669600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:17:57.539 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.539+0000 7f1669600640 1 -- 172.21.15.63:0/3545260390 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167001a6a0 con 0x56264c5944d0 2024-05-27T08:17:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.540+0000 7f1669600640 1 -- 172.21.15.63:0/3545260390 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(169..169 src has 1..169) ==== 9539+0+0 (secure 0 0 0) 0x7f167001a840 con 0x56264c5944d0 2024-05-27T08:17:57.540 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.540+0000 7f1691823740 1 -- 172.21.15.63:0/3545260390 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f168407fa00 con 0x56264c5944d0 2024-05-27T08:17:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.544+0000 7f1669600640 1 -- 172.21.15.63:0/3545260390 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f167001aa40 con 0x56264c5944d0 2024-05-27T08:17:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.544+0000 7f1691823740 10 client.32293 fetch_fsmap learned FSMap version 35 2024-05-27T08:17:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.544+0000 7f1691823740 1 -- 172.21.15.63:0/3545260390 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56264c9ec490 con 0x56264c5944d0 2024-05-27T08:17:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.544+0000 7f1669600640 1 -- 172.21.15.63:0/3545260390 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f168407fa00 con 0x56264c5944d0 2024-05-27T08:17:57.550 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.550+0000 7f1669600640 1 -- 172.21.15.63:0/3545260390 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1670023470 con 0x56264c5944d0 2024-05-27T08:17:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.551+0000 7f1691823740 10 client.32293 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:17:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.551+0000 7f1691823740 10 client.32293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:17:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.551+0000 7f1691823740 20 client.32293 populate_metadata read hostname 'smithi063' 2024-05-27T08:17:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.551+0000 7f1691823740 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56264c5491c0 0x56264c9eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.552+0000 7f1691823740 4 client.32293 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:17:57.553 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.553+0000 7f1691823740 1 -- 172.21.15.63:0/3545260390 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56264c8c42c0 con 0x56264c5491c0 2024-05-27T08:17:57.554 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.554+0000 7f1683e00640 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56264c5491c0 0x56264c9eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.555+0000 7f1683e00640 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56264c5491c0 0x56264c9eeb20 crc :-1 s=READY pgs=767 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf0d7a0e1f5d84db server_cookie=c812bd07ec291a07 in_seq=0 out_seq=0 2024-05-27T08:17:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:57.555+0000 7f1669600640 10 client.32293 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:59.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.930+0000 7f342f400700 1 -- 172.21.15.63:0/233613078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3428110c60 msgr2=0x7f3428113080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:59.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.930+0000 7f342f400700 1 --2- 172.21.15.63:0/233613078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3428110c60 0x7f3428113080 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3420009f90 tx=0x7f342000b330 comp rx=0 tx=0).stop 2024-05-27T08:17:59.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.932+0000 7f342f400700 1 -- 172.21.15.63:0/233613078 shutdown_connections 2024-05-27T08:17:59.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.932+0000 7f342f400700 1 --2- 172.21.15.63:0/233613078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3428110c60 0x7f3428113080 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.932+0000 7f342f400700 1 --2- 172.21.15.63:0/233613078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34281030b0 0x7f34281034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.932+0000 7f342f400700 1 --2- 172.21.15.63:0/233613078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428103a00 0x7f3428110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.932+0000 7f342f400700 1 -- 172.21.15.63:0/233613078 >> 172.21.15.63:0/233613078 conn(0x7f34280fedb0 msgr2=0x7f34281011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.934+0000 7f342f400700 1 -- 172.21.15.63:0/233613078 shutdown_connections 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.934+0000 7f342f400700 1 -- 172.21.15.63:0/233613078 wait complete. 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.936+0000 7f342f400700 1 Processor -- start 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.936+0000 7f342f400700 1 -- start start 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.938+0000 7f342f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34281030b0 0x7f342810f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.938+0000 7f342f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428103a00 0x7f342810fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.938+0000 7f342f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3428110c60 0x7f34281a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.938+0000 7f3427e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428103a00 0x7f342810fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:59.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.939+0000 7f342ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34281030b0 0x7f342810f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.939+0000 7f342ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34281030b0 0x7f342810f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:33236/0 (socket says 172.21.15.63:33236) 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.939+0000 7f342d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3428110c60 0x7f34281a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.940+0000 7f3427e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428103a00 0x7f342810fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:39368/0 (socket says 172.21.15.63:39368) 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.941+0000 7f342d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3428110c60 0x7f34281a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53594/0 (socket says 172.21.15.63:53594) 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.941+0000 7f342f400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3428106350 con 0x7f3428110c60 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.941+0000 7f342f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34281061d0 con 0x7f3428103a00 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.941+0000 7f342f400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34281064d0 con 0x7f34281030b0 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.941+0000 7f342ca00700 1 -- 172.21.15.63:0/1106190760 learned_addr learned my addr 172.21.15.63:0/1106190760 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.943+0000 7f342ca00700 1 -- 172.21.15.63:0/1106190760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428103a00 msgr2=0x7f342810fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.943+0000 7f342ca00700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428103a00 0x7f342810fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.943+0000 7f342ca00700 1 -- 172.21.15.63:0/1106190760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3428110c60 msgr2=0x7f34281a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.944+0000 7f342ca00700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3428110c60 0x7f34281a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.944+0000 7f342ca00700 1 -- 172.21.15.63:0/1106190760 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34281a6290 con 0x7f34281030b0 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.944+0000 7f342d400700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3428110c60 0x7f34281a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.945+0000 7f342ca00700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34281030b0 0x7f342810f620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c920 tx=0x7f341c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.947+0000 7f3425600700 1 -- 172.21.15.63:0/1106190760 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c007b00 con 0x7f34281030b0 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.947+0000 7f342f400700 1 -- 172.21.15.63:0/1106190760 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34281a6580 con 0x7f34281030b0 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.947+0000 7f342f400700 1 -- 172.21.15.63:0/1106190760 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34281adea0 con 0x7f34281030b0 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.948+0000 7f3425600700 1 -- 172.21.15.63:0/1106190760 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f341c004cc0 con 0x7f34281030b0 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.948+0000 7f3425600700 1 -- 172.21.15.63:0/1106190760 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c007b00 con 0x7f34281030b0 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.949+0000 7f342f400700 1 -- 172.21.15.63:0/1106190760 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3408005330 con 0x7f34281030b0 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.954+0000 7f3425600700 1 -- 172.21.15.63:0/1106190760 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f341c004750 con 0x7f34281030b0 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.958+0000 7f3425600700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34100b2c70 0x7f34100b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.959+0000 7f3425600700 1 -- 172.21.15.63:0/1106190760 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 9539+0+0 (secure 0 0 0) 0x7f341c0e1ff0 con 0x7f34281030b0 2024-05-27T08:17:59.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.960+0000 7f3427e00700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34100b2c70 0x7f34100b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:17:59.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.962+0000 7f3427e00700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34100b2c70 0x7f34100b5180 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f34281049d0 tx=0x7f3414009310 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:17:59.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:58.966+0000 7f3425600700 1 -- 172.21.15.63:0/1106190760 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f341c014030 con 0x7f34281030b0 2024-05-27T08:17:59.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:58 smithi063 ceph-mon[31613]: pgmap v1411: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-05-27T08:17:59.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:58 smithi086 ceph-mon[34687]: pgmap v1411: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-05-27T08:17:59.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:58 smithi149 ceph-mon[35408]: pgmap v1411: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-05-27T08:17:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.466+0000 7f1669600640 1 -- 172.21.15.63:0/3545260390 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56264c8c42c0 con 0x56264c5491c0 2024-05-27T08:17:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.468+0000 7f1669600640 10 client.32293 handle_command_reply: tid=0 2024-05-27T08:17:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.468+0000 7f1691823740 1 client.32293 shutdown 2024-05-27T08:17:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.469+0000 7f1691823740 1 -- 172.21.15.63:0/3545260390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c5944d0 msgr2=0x56264c9850e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.469+0000 7f1691823740 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c5944d0 0x56264c9850e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x56264c955580 tx=0x7f1670011810 comp rx=0 tx=0).stop 2024-05-27T08:17:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.470+0000 7f1691823740 1 -- 172.21.15.63:0/3545260390 shutdown_connections 2024-05-27T08:17:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.470+0000 7f1691823740 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56264c5491c0 0x56264c9eeb20 crc :-1 s=READY pgs=767 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.470+0000 7f1691823740 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56264c5b51b0 0x56264c985620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.470+0000 7f1691823740 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56264c5944d0 0x56264c9850e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.470+0000 7f1691823740 1 --2- 172.21.15.63:0/3545260390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56264c58c2f0 0x56264c984ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.470+0000 7f1669600640 0 client.32293 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:17:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.470+0000 7f1691823740 1 -- 172.21.15.63:0/3545260390 >> 172.21.15.63:0/3545260390 conn(0x56264c5a9300 msgr2=0x56264c95d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.474+0000 7f1691823740 1 -- 172.21.15.63:0/3545260390 shutdown_connections 2024-05-27T08:17:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.474+0000 7f1691823740 1 -- 172.21.15.63:0/3545260390 wait complete. 2024-05-27T08:17:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.474+0000 7f1691823740 20 client.32293 trim_cache size 0 max 16384 2024-05-27T08:17:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:17:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:17:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:17:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.476+0000 7f168a800640 1 -- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f166c0b2b70 msgr2=0x7f166c0b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.477+0000 7f168a800640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f166c0b2b70 0x7f166c0b5080 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1678100600 tx=0x7f1678003f70 comp rx=0 tx=0).stop 2024-05-27T08:17:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.478+0000 7f168a800640 1 -- 172.21.15.63:0/4032348568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f167c008170 msgr2=0x7f167c139770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.478+0000 7f168a800640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f167c008170 0x7f167c139770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1678009850 tx=0x7f1678003d80 comp rx=0 tx=0).stop 2024-05-27T08:17:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.480+0000 7f168a800640 1 -- 172.21.15.63:0/4032348568 shutdown_connections 2024-05-27T08:17:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.480+0000 7f168a800640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f166c0b2b70 0x7f166c0b5080 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.480 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.480+0000 7f168a800640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f167c0a79c0 0x7f167c133980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.480+0000 7f168a800640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f167c0a5060 0x7f167c139cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.480+0000 7f168a800640 1 --2- 172.21.15.63:0/4032348568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f167c008170 0x7f167c139770 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.481 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.480+0000 7f168a800640 1 -- 172.21.15.63:0/4032348568 >> 172.21.15.63:0/4032348568 conn(0x7f167c00c160 msgr2=0x7f167c0a5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.481+0000 7f168a800640 1 -- 172.21.15.63:0/4032348568 shutdown_connections 2024-05-27T08:17:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:17:59.482+0000 7f168a800640 1 -- 172.21.15.63:0/4032348568 wait complete. 2024-05-27T08:17:59.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.498+0000 7f342f400700 1 -- 172.21.15.63:0/1106190760 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34080058f0 con 0x7f34281030b0 2024-05-27T08:17:59.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.499+0000 7f3425600700 1 -- 172.21.15.63:0/1106190760 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11862 (secure 0 0 0) 0x7f341c0a95d0 con 0x7f34281030b0 2024-05-27T08:17:59.504 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:17:59.504 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T07:58:43.218540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_39428":{"gid":39428,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3062218679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3062218679},{"type":"v1","addr":"172.21.15.86:6833","nonce":3062218679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34484,34520,34454]},"id":1}]} 2024-05-27T08:17:59.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 -- 172.21.15.63:0/1106190760 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34100b2c70 msgr2=0x7f34100b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:59.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34100b2c70 0x7f34100b5180 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f34281049d0 tx=0x7f3414009310 comp rx=0 tx=0).stop 2024-05-27T08:17:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 -- 172.21.15.63:0/1106190760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34281030b0 msgr2=0x7f342810f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:17:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34281030b0 0x7f342810f620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c920 tx=0x7f341c00cc30 comp rx=0 tx=0).stop 2024-05-27T08:17:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 -- 172.21.15.63:0/1106190760 shutdown_connections 2024-05-27T08:17:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f34100b2c70 0x7f34100b5180 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3428110c60 0x7f34281a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34281030b0 0x7f342810f620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 --2- 172.21.15.63:0/1106190760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3428103a00 0x7f342810fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:17:59.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 -- 172.21.15.63:0/1106190760 >> 172.21.15.63:0/1106190760 conn(0x7f34280fedb0 msgr2=0x7f3428112840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:17:59.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 -- 172.21.15.63:0/1106190760 shutdown_connections 2024-05-27T08:17:59.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:17:59.506+0000 7f342f400700 1 -- 172.21.15.63:0/1106190760 wait complete. 2024-05-27T08:17:59.513 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-05-27T08:18:00.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:17:59 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/1106190760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:18:00.424 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:18:00.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:17:59 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/1106190760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:18:00.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:17:59 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/1106190760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:18:01.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:01 smithi086 ceph-mon[34687]: pgmap v1412: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.1 MiB/s wr, 2.07k op/s 2024-05-27T08:18:01.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:01 smithi149 ceph-mon[35408]: pgmap v1412: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.1 MiB/s wr, 2.07k op/s 2024-05-27T08:18:01.470 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:01 smithi063 ceph-mon[31613]: pgmap v1412: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.1 MiB/s wr, 2.07k op/s 2024-05-27T08:18:01.719 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:18:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:02 smithi086 ceph-mon[34687]: pgmap v1413: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:18:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:02 smithi149 ceph-mon[35408]: pgmap v1413: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:18:03.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:02 smithi063 ceph-mon[31613]: pgmap v1413: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 10 MiB/s wr, 2.54k op/s 2024-05-27T08:18:05.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:04 smithi149 ceph-mon[35408]: pgmap v1414: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.2 MiB/s wr, 2.34k op/s 2024-05-27T08:18:05.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:04 smithi086 ceph-mon[34687]: pgmap v1414: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.2 MiB/s wr, 2.34k op/s 2024-05-27T08:18:05.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:04 smithi063 ceph-mon[31613]: pgmap v1414: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.2 MiB/s wr, 2.34k op/s 2024-05-27T08:18:07.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:06 smithi086 ceph-mon[34687]: pgmap v1415: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-27T08:18:07.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:06 smithi149 ceph-mon[35408]: pgmap v1415: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-27T08:18:07.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:06 smithi063 ceph-mon[31613]: pgmap v1415: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-27T08:18:09.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:08 smithi086 ceph-mon[34687]: pgmap v1416: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-05-27T08:18:09.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:08 smithi149 ceph-mon[35408]: pgmap v1416: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-05-27T08:18:09.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:08 smithi063 ceph-mon[31613]: pgmap v1416: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 10 MiB/s wr, 2.62k op/s 2024-05-27T08:18:11.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:10 smithi086 ceph-mon[34687]: pgmap v1417: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-05-27T08:18:11.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:10 smithi149 ceph-mon[35408]: pgmap v1417: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-05-27T08:18:11.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:10 smithi063 ceph-mon[31613]: pgmap v1417: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 10 MiB/s wr, 2.55k op/s 2024-05-27T08:18:12.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.854+0000 7f505f200640 1 -- 172.21.15.63:0/440215498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5058109c90 msgr2=0x7f5058103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.854+0000 7f505f200640 1 --2- 172.21.15.63:0/440215498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5058109c90 0x7f5058103830 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7f5048009860 tx=0x7f504802e770 comp rx=0 tx=0).stop 2024-05-27T08:18:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.855+0000 7f505f200640 1 -- 172.21.15.63:0/440215498 shutdown_connections 2024-05-27T08:18:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.856+0000 7f505f200640 1 --2- 172.21.15.63:0/440215498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50581043e0 0x7f50581107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.856+0000 7f505f200640 1 --2- 172.21.15.63:0/440215498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505810a130 0x7f5058103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.856+0000 7f505f200640 1 --2- 172.21.15.63:0/440215498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5058109c90 0x7f5058103830 unknown :-1 s=CLOSED pgs=2968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.856+0000 7f505f200640 1 -- 172.21.15.63:0/440215498 >> 172.21.15.63:0/440215498 conn(0x7f5058079390 msgr2=0x7f50580797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.859+0000 7f505f200640 1 -- 172.21.15.63:0/440215498 shutdown_connections 2024-05-27T08:18:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.859+0000 7f505f200640 1 -- 172.21.15.63:0/440215498 wait complete. 2024-05-27T08:18:12.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.861+0000 7f505f200640 1 Processor -- start 2024-05-27T08:18:12.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.862+0000 7f505f200640 1 -- start start 2024-05-27T08:18:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.863+0000 7f505f200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50581043e0 0x7f505806db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.863+0000 7f505f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5058109c90 0x7f505806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.863+0000 7f505f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505810a130 0x7f505806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.863+0000 7f505f200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5058111ba0 con 0x7f50581043e0 2024-05-27T08:18:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.864+0000 7f505f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5058111a20 con 0x7f5058109c90 2024-05-27T08:18:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.864+0000 7f505f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5058111d20 con 0x7f505810a130 2024-05-27T08:18:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.864+0000 7f5057e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50581043e0 0x7f505806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.864+0000 7f5057400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5058109c90 0x7f505806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.864+0000 7f505d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505810a130 0x7f505806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.864+0000 7f505d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505810a130 0x7f505806e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:55794/0 (socket says 172.21.15.63:55794) 2024-05-27T08:18:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.864+0000 7f505d200640 1 -- 172.21.15.63:0/3581798882 learned_addr learned my addr 172.21.15.63:0/3581798882 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.865+0000 7f505d200640 1 -- 172.21.15.63:0/3581798882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5058109c90 msgr2=0x7f505806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.865+0000 7f505d200640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5058109c90 0x7f505806e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.865+0000 7f505d200640 1 -- 172.21.15.63:0/3581798882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50581043e0 msgr2=0x7f505806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.865+0000 7f505d200640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50581043e0 0x7f505806db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.865+0000 7f505d200640 1 -- 172.21.15.63:0/3581798882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5058072570 con 0x7f505810a130 2024-05-27T08:18:12.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.866+0000 7f5057e00640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50581043e0 0x7f505806db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:18:12.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.867+0000 7f505d200640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505810a130 0x7f505806e5f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5050009fd0 tx=0x7f505000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.870+0000 7f5054c00640 1 -- 172.21.15.63:0/3581798882 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050004b20 con 0x7f505810a130 2024-05-27T08:18:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.870+0000 7f5054c00640 1 -- 172.21.15.63:0/3581798882 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5050003e00 con 0x7f505810a130 2024-05-27T08:18:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.870+0000 7f5054c00640 1 -- 172.21.15.63:0/3581798882 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50500047b0 con 0x7f505810a130 2024-05-27T08:18:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.870+0000 7f505f200640 1 -- 172.21.15.63:0/3581798882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5058072860 con 0x7f505810a130 2024-05-27T08:18:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.870+0000 7f505f200640 1 -- 172.21.15.63:0/3581798882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5058072db0 con 0x7f505810a130 2024-05-27T08:18:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.881+0000 7f5054c00640 1 -- 172.21.15.63:0/3581798882 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f505001d020 con 0x7f505810a130 2024-05-27T08:18:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.884+0000 7f5054c00640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f503c0b2b60 0x7f503c0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.884+0000 7f5054c00640 1 -- 172.21.15.63:0/3581798882 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7f50500e2a50 con 0x7f505810a130 2024-05-27T08:18:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.884+0000 7f5066242740 1 Processor -- start 2024-05-27T08:18:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.885+0000 7f5057e00640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f503c0b2b60 0x7f503c0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.887+0000 7f5066242740 1 -- start start 2024-05-27T08:18:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.888+0000 7f5066242740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 0x55e3f9cf9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.888+0000 7f5066242740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9cf9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.888+0000 7f5066242740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99364d0 0x55e3f9cfa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.888+0000 7f5066242740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9c6ab90 con 0x55e3f992e2f0 2024-05-27T08:18:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.888+0000 7f5066242740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9c5a730 con 0x55e3f99364d0 2024-05-27T08:18:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.888+0000 7f5066242740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9c5e770 con 0x55e3f99571b0 2024-05-27T08:18:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.889+0000 7f5057400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9cf9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.889+0000 7f5057400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9cf9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42092/0 (socket says 172.21.15.63:42092) 2024-05-27T08:18:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.889+0000 7f5057400640 1 -- 172.21.15.63:0/2694029719 learned_addr learned my addr 172.21.15.63:0/2694029719 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.889+0000 7f5057400640 1 -- 172.21.15.63:0/2694029719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 msgr2=0x55e3f9cf9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.889+0000 7f5057e00640 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 0x55e3f9cf9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.890+0000 7f5057400640 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 0x55e3f9cf9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.890+0000 7f505d200640 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99364d0 0x55e3f9cfa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.890+0000 7f5057400640 1 -- 172.21.15.63:0/2694029719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99364d0 msgr2=0x55e3f9cfa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.890+0000 7f5057400640 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99364d0 0x55e3f9cfa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.890+0000 7f5057400640 1 -- 172.21.15.63:0/2694029719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3f9bf38b0 con 0x55e3f992e2f0 2024-05-27T08:18:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.890+0000 7f505d200640 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99364d0 0x55e3f9cfa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:18:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.890+0000 7f5057e00640 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 0x55e3f9cf9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:18:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.890+0000 7f5057e00640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f503c0b2b60 0x7f503c0b5070 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f505806f3c0 tx=0x7f5048000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.891+0000 7f5057400640 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9cf9da0 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7f504400bbb0 tx=0x7f50440107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f89625d4b0ea64e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.892+0000 7f504d600640 1 -- 172.21.15.63:0/2694029719 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044004030 con 0x55e3f992e2f0 2024-05-27T08:18:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.893+0000 7f504d600640 1 -- 172.21.15.63:0/2694029719 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5044005020 con 0x55e3f992e2f0 2024-05-27T08:18:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.893+0000 7f504d600640 1 -- 172.21.15.63:0/2694029719 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044004b10 con 0x55e3f992e2f0 2024-05-27T08:18:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.894+0000 7f5066242740 1 -- 172.21.15.63:0/2694029719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 msgr2=0x55e3f9cf9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.894+0000 7f5066242740 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9cf9da0 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7f504400bbb0 tx=0x7f50440107a0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.894+0000 7f5066242740 1 -- 172.21.15.63:0/2694029719 shutdown_connections 2024-05-27T08:18:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.894+0000 7f5066242740 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99364d0 0x55e3f9cfa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.894+0000 7f5066242740 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9cf9da0 unknown :-1 s=CLOSED pgs=2969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.894+0000 7f5066242740 1 --2- 172.21.15.63:0/2694029719 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 0x55e3f9cf9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.894+0000 7f5066242740 1 -- 172.21.15.63:0/2694029719 >> 172.21.15.63:0/2694029719 conn(0x55e3f994b300 msgr2=0x55e3f9cf6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.895+0000 7f5066242740 1 -- 172.21.15.63:0/2694029719 shutdown_connections 2024-05-27T08:18:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.895+0000 7f5066242740 1 -- 172.21.15.63:0/2694029719 wait complete. 2024-05-27T08:18:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.897+0000 7f5066242740 1 -- start start 2024-05-27T08:18:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.897+0000 7f5066242740 1 Processor -- start 2024-05-27T08:18:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.899+0000 7f5066242740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f992e2f0 0x55e3f9d01c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.899+0000 7f5066242740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.899+0000 7f5066242740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99571b0 0x55e3f9d184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.899+0000 7f5066242740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9940790 con 0x55e3f99571b0 2024-05-27T08:18:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.899+0000 7f5066242740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9c02bc0 con 0x55e3f992e2f0 2024-05-27T08:18:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.899+0000 7f5066242740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9d0f1a0 con 0x55e3f99364d0 2024-05-27T08:18:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.899+0000 7f5066242740 10 client.0 _ll_register_callbacks cb 0x55e3f9be0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.899+0000 7f5057400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.899+0000 7f5057400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:55812/0 (socket says 172.21.15.63:55812) 2024-05-27T08:18:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.899+0000 7f5057400640 1 -- 172.21.15.63:0/3090596639 learned_addr learned my addr 172.21.15.63:0/3090596639 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.900+0000 7f505d200640 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99571b0 0x55e3f9d184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.900+0000 7f5057e00640 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f992e2f0 0x55e3f9d01c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.900+0000 7f5057400640 1 -- 172.21.15.63:0/3090596639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f992e2f0 msgr2=0x55e3f9d01c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.900+0000 7f5057400640 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f992e2f0 0x55e3f9d01c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.900+0000 7f5057400640 1 -- 172.21.15.63:0/3090596639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99571b0 msgr2=0x55e3f9d184d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.900+0000 7f5057400640 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99571b0 0x55e3f9d184d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.900+0000 7f5057400640 1 -- 172.21.15.63:0/3090596639 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3f996f210 con 0x55e3f99364d0 2024-05-27T08:18:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.902+0000 7f5042000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:18:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.902+0000 7f5057400640 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d021d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5044019000 tx=0x7f5044005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36fb2a54eb447744 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.903+0000 7f5042000640 1 -- 172.21.15.63:0/3090596639 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504400bea0 con 0x55e3f99364d0 2024-05-27T08:18:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.903+0000 7f5042000640 1 -- 172.21.15.63:0/3090596639 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7f5044004040 con 0x55e3f99364d0 2024-05-27T08:18:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.904+0000 7f5066242740 1 -- 172.21.15.63:0/3090596639 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5044005020 con 0x55e3f99364d0 2024-05-27T08:18:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.904+0000 7f5042000640 1 -- 172.21.15.63:0/3090596639 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5044004240 con 0x55e3f99364d0 2024-05-27T08:18:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.904+0000 7f5066242740 10 client.32308 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.904+0000 7f5066242740 1 -- 172.21.15.63:0/3090596639 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e3f9a3b620 con 0x55e3f99364d0 2024-05-27T08:18:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.905+0000 7f5042000640 1 -- 172.21.15.63:0/3090596639 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5044005020 con 0x55e3f99364d0 2024-05-27T08:18:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.905+0000 7f5042000640 1 -- 172.21.15.63:0/3090596639 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5044003920 con 0x55e3f99364d0 2024-05-27T08:18:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.905+0000 7f5066242740 10 client.32308 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.905+0000 7f5066242740 10 client.32308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:18:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.905+0000 7f5066242740 20 client.32308 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.905+0000 7f5066242740 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3f9915d80 0x55e3f9d1e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.905+0000 7f5066242740 4 client.32308 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:18:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.905+0000 7f5066242740 1 -- 172.21.15.63:0/3090596639 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3f9c6cae0 con 0x55e3f9915d80 2024-05-27T08:18:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.906+0000 7f5057e00640 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3f9915d80 0x55e3f9d1e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.907+0000 7f5042000640 10 client.32308 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.907+0000 7f5057e00640 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3f9915d80 0x55e3f9d1e290 crc :-1 s=READY pgs=768 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f4b62727f4f032 server_cookie=a5f22c62e4bab86 in_seq=0 out_seq=0 2024-05-27T08:18:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.908+0000 7f5042000640 1 -- 172.21.15.63:0/3090596639 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e3f9c6cae0 con 0x55e3f9915d80 2024-05-27T08:18:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.908+0000 7f5042000640 10 client.32308 handle_command_reply: tid=0 2024-05-27T08:18:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.909+0000 7f5066242740 1 client.32308 shutdown 2024-05-27T08:18:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.909+0000 7f5066242740 1 -- 172.21.15.63:0/3090596639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 msgr2=0x55e3f9d021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.909+0000 7f5066242740 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d021d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5044019000 tx=0x7f5044005400 comp rx=0 tx=0).stop 2024-05-27T08:18:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.911+0000 7f5066242740 1 -- 172.21.15.63:0/3090596639 shutdown_connections 2024-05-27T08:18:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.911+0000 7f5066242740 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3f9915d80 0x55e3f9d1e290 crc :-1 s=READY pgs=768 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.911+0000 7f5066242740 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99571b0 0x55e3f9d184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.911+0000 7f5066242740 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d021d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.911+0000 7f5066242740 1 --2- 172.21.15.63:0/3090596639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f992e2f0 0x55e3f9d01c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.911+0000 7f5066242740 1 -- 172.21.15.63:0/3090596639 >> 172.21.15.63:0/3090596639 conn(0x55e3f994b300 msgr2=0x55e3f9cfb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.911+0000 7f5042000640 0 client.32308 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.911+0000 7f5066242740 1 -- 172.21.15.63:0/3090596639 shutdown_connections 2024-05-27T08:18:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.912+0000 7f5066242740 1 -- 172.21.15.63:0/3090596639 wait complete. 2024-05-27T08:18:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.912+0000 7f5066242740 20 client.32308 trim_cache size 0 max 16384 2024-05-27T08:18:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.979+0000 7f5066242740 1 Processor -- start 2024-05-27T08:18:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.979+0000 7f5066242740 1 -- start start 2024-05-27T08:18:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.979+0000 7f5066242740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f992e2f0 0x55e3f9cfcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.979+0000 7f5066242740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99364d0 0x55e3f9cfd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.979+0000 7f5066242740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 0x55e3f9d18240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.979+0000 7f5066242740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50581ba340 con 0x55e3f99364d0 2024-05-27T08:18:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.979+0000 7f5066242740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50581b7c10 con 0x55e3f992e2f0 2024-05-27T08:18:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.979+0000 7f5066242740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50581b5a40 con 0x55e3f99571b0 2024-05-27T08:18:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.980+0000 7f5057400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99364d0 0x55e3f9cfd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.981+0000 7f505d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 0x55e3f9d18240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.981+0000 7f5057e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f992e2f0 0x55e3f9cfcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.981+0000 7f5057400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99364d0 0x55e3f9cfd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42114/0 (socket says 172.21.15.63:42114) 2024-05-27T08:18:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.981+0000 7f5057400640 1 -- 172.21.15.63:0/4122450985 learned_addr learned my addr 172.21.15.63:0/4122450985 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.982+0000 7f5057400640 1 -- 172.21.15.63:0/4122450985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 msgr2=0x55e3f9d18240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.982+0000 7f5057400640 1 --2- 172.21.15.63:0/4122450985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 0x55e3f9d18240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.982+0000 7f5057400640 1 -- 172.21.15.63:0/4122450985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f992e2f0 msgr2=0x55e3f9cfcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.982+0000 7f5057400640 1 --2- 172.21.15.63:0/4122450985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f992e2f0 0x55e3f9cfcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.982+0000 7f5057400640 1 -- 172.21.15.63:0/4122450985 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3f9cf6c00 con 0x55e3f99364d0 2024-05-27T08:18:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.982+0000 7f505d200640 1 --2- 172.21.15.63:0/4122450985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 0x55e3f9d18240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:18:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.983+0000 7f5057400640 1 --2- 172.21.15.63:0/4122450985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99364d0 0x55e3f9cfd220 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7f5044003eb0 tx=0x7f50440107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f229684b318282 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.984+0000 7f504cc00640 1 -- 172.21.15.63:0/4122450985 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044004240 con 0x55e3f99364d0 2024-05-27T08:18:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.984+0000 7f504cc00640 1 -- 172.21.15.63:0/4122450985 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5044003a80 con 0x55e3f99364d0 2024-05-27T08:18:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.984+0000 7f504cc00640 1 -- 172.21.15.63:0/4122450985 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044030ae0 con 0x55e3f99364d0 2024-05-27T08:18:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.986+0000 7f5066242740 1 -- 172.21.15.63:0/4122450985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99364d0 msgr2=0x55e3f9cfd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.986+0000 7f5066242740 1 --2- 172.21.15.63:0/4122450985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99364d0 0x55e3f9cfd220 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7f5044003eb0 tx=0x7f50440107a0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.987+0000 7f5066242740 1 -- 172.21.15.63:0/4122450985 shutdown_connections 2024-05-27T08:18:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.987+0000 7f5066242740 1 --2- 172.21.15.63:0/4122450985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99571b0 0x55e3f9d18240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.987+0000 7f5066242740 1 --2- 172.21.15.63:0/4122450985 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f99364d0 0x55e3f9cfd220 unknown :-1 s=CLOSED pgs=2970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.987+0000 7f5066242740 1 --2- 172.21.15.63:0/4122450985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f992e2f0 0x55e3f9cfcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.987+0000 7f5066242740 1 -- 172.21.15.63:0/4122450985 >> 172.21.15.63:0/4122450985 conn(0x55e3f994b300 msgr2=0x55e3f9cfc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.988+0000 7f5066242740 1 -- 172.21.15.63:0/4122450985 shutdown_connections 2024-05-27T08:18:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.988+0000 7f5066242740 1 -- 172.21.15.63:0/4122450985 wait complete. 2024-05-27T08:18:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.990+0000 7f5066242740 1 -- start start 2024-05-27T08:18:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.990+0000 7f5066242740 1 Processor -- start 2024-05-27T08:18:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.993+0000 7f5066242740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9d14b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.993+0000 7f5066242740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d0f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.994+0000 7f5057400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d0f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.994+0000 7f5057400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d0f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:55824/0 (socket says 172.21.15.63:55824) 2024-05-27T08:18:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.994+0000 7f5057400640 1 -- 172.21.15.63:0/3622575623 learned_addr learned my addr 172.21.15.63:0/3622575623 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.994+0000 7f5066242740 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99571b0 0x55e3f9d0f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.994+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50581b60d0 con 0x55e3f992e2f0 2024-05-27T08:18:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.994+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50581b7540 con 0x55e3f99571b0 2024-05-27T08:18:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.994+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f50581b67a0 con 0x55e3f99364d0 2024-05-27T08:18:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.994+0000 7f5066242740 10 client.0 _ll_register_callbacks cb 0x7f504400ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.995+0000 7f5057e00640 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9d14b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.995+0000 7f505d200640 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99571b0 0x55e3f9d0f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.997+0000 7f5057e00640 1 -- 172.21.15.63:0/3622575623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 msgr2=0x55e3f9d0f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.997+0000 7f5057e00640 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d0f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.997+0000 7f5057e00640 1 -- 172.21.15.63:0/3622575623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99571b0 msgr2=0x55e3f9d0f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.997+0000 7f5057e00640 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99571b0 0x55e3f9d0f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.997+0000 7f5057e00640 1 -- 172.21.15.63:0/3622575623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3f9d0fff0 con 0x55e3f992e2f0 2024-05-27T08:18:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.998+0000 7f5057400640 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d0f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:18:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.998+0000 7f505d200640 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99571b0 0x55e3f9d0f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:18:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.999+0000 7f5057e00640 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9d14b20 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f5048009780 tx=0x7f5048005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8525fea96e4ead9e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:12.999+0000 7f5042000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:18:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.001+0000 7f5042000640 1 -- 172.21.15.63:0/3622575623 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504805b060 con 0x55e3f992e2f0 2024-05-27T08:18:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.002+0000 7f5042000640 1 -- 172.21.15.63:0/3622575623 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7f5048005450 con 0x55e3f992e2f0 2024-05-27T08:18:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.002+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5044003a80 con 0x55e3f992e2f0 2024-05-27T08:18:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.010+0000 7f5042000640 1 -- 172.21.15.63:0/3622575623 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5048005650 con 0x55e3f992e2f0 2024-05-27T08:18:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.011+0000 7f5066242740 10 client.23616 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.011+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e3f9d856c0 con 0x55e3f992e2f0 2024-05-27T08:18:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.012+0000 7f5042000640 1 -- 172.21.15.63:0/3622575623 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5044003a80 con 0x55e3f992e2f0 2024-05-27T08:18:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.013+0000 7f5042000640 1 -- 172.21.15.63:0/3622575623 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f504804a070 con 0x55e3f992e2f0 2024-05-27T08:18:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.015+0000 7f5066242740 10 client.23616 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.015+0000 7f5066242740 10 client.23616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:18:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.015+0000 7f5066242740 20 client.23616 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.015+0000 7f5066242740 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3f98eb1c0 0x55e3f9d87d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.015+0000 7f5066242740 4 client.23616 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:18:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.015+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3f9c662c0 con 0x55e3f98eb1c0 2024-05-27T08:18:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.016+0000 7f5057400640 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3f98eb1c0 0x55e3f9d87d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.020+0000 7f5057400640 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3f98eb1c0 0x55e3f9d87d10 crc :-1 s=READY pgs=769 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d218d98172a644e server_cookie=9bd945c53b8423c in_seq=0 out_seq=0 2024-05-27T08:18:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.020+0000 7f5042000640 10 client.23616 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.395+0000 7f5042000640 1 -- 172.21.15.63:0/3622575623 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e3f9c662c0 con 0x55e3f98eb1c0 2024-05-27T08:18:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.395+0000 7f5042000640 10 client.23616 handle_command_reply: tid=0 2024-05-27T08:18:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.395+0000 7f5066242740 1 client.23616 shutdown 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.396+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 msgr2=0x55e3f9d14b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.396+0000 7f5066242740 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9d14b20 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f5048009780 tx=0x7f5048005f90 comp rx=0 tx=0).stop 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.396+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 shutdown_connections 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.396+0000 7f5066242740 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55e3f98eb1c0 0x55e3f9d87d10 crc :-1 s=READY pgs=769 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.396+0000 7f5066242740 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e3f99571b0 0x55e3f9d0f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.396+0000 7f5042000640 0 client.23616 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.396+0000 7f5066242740 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f99364d0 0x55e3f9d0f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.396+0000 7f5066242740 1 --2- 172.21.15.63:0/3622575623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3f992e2f0 0x55e3f9d14b20 unknown :-1 s=CLOSED pgs=2971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.396+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 >> 172.21.15.63:0/3622575623 conn(0x55e3f994b300 msgr2=0x55e3f9cfd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.397+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 shutdown_connections 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.397+0000 7f5066242740 1 -- 172.21.15.63:0/3622575623 wait complete. 2024-05-27T08:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.398+0000 7f5066242740 20 client.23616 trim_cache size 0 max 16384 2024-05-27T08:18:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:18:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:18:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:18:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.399+0000 7f505f200640 1 -- 172.21.15.63:0/3581798882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f503c0b2b60 msgr2=0x7f503c0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.399+0000 7f505f200640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f503c0b2b60 0x7f503c0b5070 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f505806f3c0 tx=0x7f5048000bc0 comp rx=0 tx=0).stop 2024-05-27T08:18:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.403+0000 7f505f200640 1 -- 172.21.15.63:0/3581798882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505810a130 msgr2=0x7f505806e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.403+0000 7f505f200640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505810a130 0x7f505806e5f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5050009fd0 tx=0x7f505000c800 comp rx=0 tx=0).stop 2024-05-27T08:18:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.404+0000 7f505f200640 1 -- 172.21.15.63:0/3581798882 shutdown_connections 2024-05-27T08:18:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.404+0000 7f505f200640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f503c0b2b60 0x7f503c0b5070 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.404+0000 7f505f200640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505810a130 0x7f505806e5f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.404+0000 7f505f200640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5058109c90 0x7f505806e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.404+0000 7f505f200640 1 --2- 172.21.15.63:0/3581798882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50581043e0 0x7f505806db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.404+0000 7f505f200640 1 -- 172.21.15.63:0/3581798882 >> 172.21.15.63:0/3581798882 conn(0x7f5058079390 msgr2=0x7f5058105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.404+0000 7f505f200640 1 -- 172.21.15.63:0/3581798882 shutdown_connections 2024-05-27T08:18:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:13.404+0000 7f505f200640 1 -- 172.21.15.63:0/3581798882 wait complete. 2024-05-27T08:18:13.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:13 smithi086 ceph-mon[34687]: osdmap e170: 12 total, 12 up, 12 in 2024-05-27T08:18:13.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:13 smithi086 ceph-mon[34687]: pgmap v1419: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-05-27T08:18:13.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:13 smithi149 ceph-mon[35408]: osdmap e170: 12 total, 12 up, 12 in 2024-05-27T08:18:13.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:13 smithi149 ceph-mon[35408]: pgmap v1419: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-05-27T08:18:13.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:13 smithi063 ceph-mon[31613]: osdmap e170: 12 total, 12 up, 12 in 2024-05-27T08:18:13.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:13 smithi063 ceph-mon[31613]: pgmap v1419: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 2.72k op/s 2024-05-27T08:18:15.289 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:14 smithi063 ceph-mon[31613]: pgmap v1420: 129 pgs: 12 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 77 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-05-27T08:18:15.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:14 smithi086 ceph-mon[34687]: pgmap v1420: 129 pgs: 12 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 77 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-05-27T08:18:15.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:14 smithi149 ceph-mon[35408]: pgmap v1420: 129 pgs: 12 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 77 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-05-27T08:18:17.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:16 smithi086 ceph-mon[34687]: pgmap v1421: 129 pgs: 15 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 67 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.6 MiB/s wr, 2.42k op/s 2024-05-27T08:18:17.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:16 smithi149 ceph-mon[35408]: pgmap v1421: 129 pgs: 15 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 67 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.6 MiB/s wr, 2.42k op/s 2024-05-27T08:18:17.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:16 smithi063 ceph-mon[31613]: pgmap v1421: 129 pgs: 15 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 67 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.6 MiB/s wr, 2.42k op/s 2024-05-27T08:18:18.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:18 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:18:18.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:18 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:18:18.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:18 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:18:19.585 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:19 smithi063 ceph-mon[31613]: pgmap v1422: 129 pgs: 11 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 83 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.2 MiB/s wr, 2.05k op/s 2024-05-27T08:18:19.864 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:19 smithi086 ceph-mon[34687]: pgmap v1422: 129 pgs: 11 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 83 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.2 MiB/s wr, 2.05k op/s 2024-05-27T08:18:19.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:19 smithi149 ceph-mon[35408]: pgmap v1422: 129 pgs: 11 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 83 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.2 MiB/s wr, 2.05k op/s 2024-05-27T08:18:21.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.048+0000 7f6304e00700 1 -- 172.21.15.63:0/1457904806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f630010aa00 msgr2=0x7f630010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:21.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.048+0000 7f6304e00700 1 --2- 172.21.15.63:0/1457904806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f630010aa00 0x7f630010ae10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0099c0 tx=0x7f62ec009cd0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.052+0000 7f6304e00700 1 -- 172.21.15.63:0/1457904806 shutdown_connections 2024-05-27T08:18:21.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.052+0000 7f6304e00700 1 --2- 172.21.15.63:0/1457904806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f630010b350 0x7f63001196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.052+0000 7f6304e00700 1 --2- 172.21.15.63:0/1457904806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63001119c0 0x7f6300111dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.052+0000 7f6304e00700 1 --2- 172.21.15.63:0/1457904806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f630010aa00 0x7f630010ae10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.052+0000 7f6304e00700 1 -- 172.21.15.63:0/1457904806 >> 172.21.15.63:0/1457904806 conn(0x7f630006f6d0 msgr2=0x7f630006fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:21.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.053+0000 7f6304e00700 1 -- 172.21.15.63:0/1457904806 shutdown_connections 2024-05-27T08:18:21.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.053+0000 7f6304e00700 1 -- 172.21.15.63:0/1457904806 wait complete. 2024-05-27T08:18:21.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.058+0000 7f6304e00700 1 Processor -- start 2024-05-27T08:18:21.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.058+0000 7f6304e00700 1 -- start start 2024-05-27T08:18:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.060+0000 7f6304e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f630010aa00 0x7f63001a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.060+0000 7f6304e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f630010b350 0x7f63001a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.060+0000 7f6304e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63001119c0 0x7f63001a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.061+0000 7f62ff400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f630010aa00 0x7f63001a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.061+0000 7f62fea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f630010b350 0x7f63001a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.061+0000 7f62ff400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f630010aa00 0x7f63001a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41136/0 (socket says 172.21.15.63:41136) 2024-05-27T08:18:21.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.061+0000 7f62ff400700 1 -- 172.21.15.63:0/4057221113 learned_addr learned my addr 172.21.15.63:0/4057221113 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:21.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.061+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f630011a8d0 con 0x7f63001119c0 2024-05-27T08:18:21.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.061+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f630011a750 con 0x7f630010b350 2024-05-27T08:18:21.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.062+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f630011aa50 con 0x7f630010aa00 2024-05-27T08:18:21.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.062+0000 7f62ffe00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63001119c0 0x7f63001a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.062+0000 7f62fea00700 1 -- 172.21.15.63:0/4057221113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f630010aa00 msgr2=0x7f63001a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:21.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.062+0000 7f62fea00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f630010aa00 0x7f63001a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.062+0000 7f62fea00700 1 -- 172.21.15.63:0/4057221113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63001119c0 msgr2=0x7f63001a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:21.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.062+0000 7f62fea00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63001119c0 0x7f63001a5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.062+0000 7f62fea00700 1 -- 172.21.15.63:0/4057221113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63001a6170 con 0x7f630010b350 2024-05-27T08:18:21.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.063+0000 7f62fea00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f630010b350 0x7f63001a1840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f62ec000c00 tx=0x7f62ec0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:21.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.063+0000 7f62f7e00700 1 -- 172.21.15.63:0/4057221113 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec003b90 con 0x7f630010b350 2024-05-27T08:18:21.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.064+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63001a6400 con 0x7f630010b350 2024-05-27T08:18:21.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.064+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63001add80 con 0x7f630010b350 2024-05-27T08:18:21.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.066+0000 7f62f7e00700 1 -- 172.21.15.63:0/4057221113 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62ec00f8f0 con 0x7f630010b350 2024-05-27T08:18:21.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.066+0000 7f62f7e00700 1 -- 172.21.15.63:0/4057221113 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec017720 con 0x7f630010b350 2024-05-27T08:18:21.076 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:21 smithi063 ceph-mon[31613]: pgmap v1423: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.8 MiB/s wr, 1.68k op/s 2024-05-27T08:18:21.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.071+0000 7f62f7e00700 1 -- 172.21.15.63:0/4057221113 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f62ec0050d0 con 0x7f630010b350 2024-05-27T08:18:21.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.073+0000 7f62f7e00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62e40b6eb0 0x7f62e40b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.075+0000 7f62ff400700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62e40b6eb0 0x7f62e40b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.075+0000 7f62f7e00700 1 -- 172.21.15.63:0/4057221113 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7f62ec0a7fa0 con 0x7f630010b350 2024-05-27T08:18:21.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.087+0000 7f62ff400700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62e40b6eb0 0x7f62e40b93c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009ad0 tx=0x7f62f0009290 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:21.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.093+0000 7f631501fb80 1 Processor -- start 2024-05-27T08:18:21.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.096+0000 7f631501fb80 1 -- start start 2024-05-27T08:18:21.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.097+0000 7f631501fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558476267e40 0x5584763e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.097+0000 7f631501fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6410 0x5584763f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.098+0000 7f631501fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 0x5584763f3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.098+0000 7f631501fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558476314e40 con 0x558476267e40 2024-05-27T08:18:21.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.098+0000 7f631501fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5584763049e0 con 0x5584763e6820 2024-05-27T08:18:21.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.098+0000 7f631501fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558476308a20 con 0x5584763e6410 2024-05-27T08:18:21.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.098+0000 7f62fea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6410 0x5584763f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.098+0000 7f62fea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6410 0x5584763f0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41152/0 (socket says 172.21.15.63:41152) 2024-05-27T08:18:21.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.098+0000 7f62fea00700 1 -- 172.21.15.63:0/1430173604 learned_addr learned my addr 172.21.15.63:0/1430173604 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:21.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.099+0000 7f62ffe00700 1 --2- 172.21.15.63:0/1430173604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 0x5584763f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.099+0000 7f62ff400700 1 --2- 172.21.15.63:0/1430173604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558476267e40 0x5584763e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.102+0000 7f62fea00700 1 -- 172.21.15.63:0/1430173604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 msgr2=0x5584763f3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:21.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.102+0000 7f62fea00700 1 --2- 172.21.15.63:0/1430173604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 0x5584763f3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.102+0000 7f62fea00700 1 -- 172.21.15.63:0/1430173604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558476267e40 msgr2=0x5584763e5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:21.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.102+0000 7f62fea00700 1 --2- 172.21.15.63:0/1430173604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558476267e40 0x5584763e5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.102+0000 7f62fea00700 1 -- 172.21.15.63:0/1430173604 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584762cd420 con 0x5584763e6410 2024-05-27T08:18:21.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.103+0000 7f62fea00700 1 --2- 172.21.15.63:0/1430173604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6410 0x5584763f0ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0098e0 tx=0x7f62ec0b0e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c6cf2a137d1553 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:21.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.104+0000 7f62f4c00700 1 -- 172.21.15.63:0/1430173604 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0a81a0 con 0x5584763e6410 2024-05-27T08:18:21.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.104+0000 7f62f4c00700 1 -- 172.21.15.63:0/1430173604 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62ec019e50 con 0x5584763e6410 2024-05-27T08:18:21.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.104+0000 7f62f4c00700 1 -- 172.21.15.63:0/1430173604 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0a8340 con 0x5584763e6410 2024-05-27T08:18:21.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.108+0000 7f631501fb80 1 -- 172.21.15.63:0/1430173604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6410 msgr2=0x5584763f0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:21.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.108+0000 7f631501fb80 1 --2- 172.21.15.63:0/1430173604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6410 0x5584763f0ad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0098e0 tx=0x7f62ec0b0e80 comp rx=0 tx=0).stop 2024-05-27T08:18:21.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.108+0000 7f631501fb80 1 -- 172.21.15.63:0/1430173604 shutdown_connections 2024-05-27T08:18:21.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.108+0000 7f631501fb80 1 --2- 172.21.15.63:0/1430173604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 0x5584763f3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.109+0000 7f631501fb80 1 --2- 172.21.15.63:0/1430173604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558476267e40 0x5584763e5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.109+0000 7f631501fb80 1 --2- 172.21.15.63:0/1430173604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6410 0x5584763f0ad0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.109+0000 7f631501fb80 1 -- 172.21.15.63:0/1430173604 >> 172.21.15.63:0/1430173604 conn(0x5584762c7fc0 msgr2=0x5584763e1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:21.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.110+0000 7f631501fb80 1 -- 172.21.15.63:0/1430173604 shutdown_connections 2024-05-27T08:18:21.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.110+0000 7f631501fb80 1 -- 172.21.15.63:0/1430173604 wait complete. 2024-05-27T08:18:21.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.111+0000 7f631501fb80 1 -- start start 2024-05-27T08:18:21.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.112+0000 7f631501fb80 1 Processor -- start 2024-05-27T08:18:21.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.113+0000 7f631501fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 0x5584763f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.113+0000 7f631501fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763e6410 0x5584763f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.113+0000 7f631501fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 0x5584763f1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.115+0000 7f62ff400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 0x5584763f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.115+0000 7f62ff400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 0x5584763f05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41156/0 (socket says 172.21.15.63:41156) 2024-05-27T08:18:21.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.116+0000 7f62ff400700 1 -- 172.21.15.63:0/2768533006 learned_addr learned my addr 172.21.15.63:0/2768533006 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:21.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.116+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55847620afa0 con 0x5584763e6410 2024-05-27T08:18:21.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.116+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5584762a5760 con 0x5584763e6820 2024-05-27T08:18:21.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.116+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558476308a20 con 0x558476267e40 2024-05-27T08:18:21.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.116+0000 7f62ffe00700 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 0x5584763f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.116+0000 7f631501fb80 10 client.0 _ll_register_callbacks cb 0x5584762aec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:21.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.117+0000 7f62fea00700 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763e6410 0x5584763f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.117+0000 7f62ffe00700 1 -- 172.21.15.63:0/2768533006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 msgr2=0x5584763f05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:21.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.117+0000 7f62ffe00700 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 0x5584763f05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.117+0000 7f62ffe00700 1 -- 172.21.15.63:0/2768533006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763e6410 msgr2=0x5584763f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:21.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.117+0000 7f62ffe00700 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763e6410 0x5584763f0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.117+0000 7f62ffe00700 1 -- 172.21.15.63:0/2768533006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584762548d0 con 0x5584763e6820 2024-05-27T08:18:21.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.118+0000 7f62ffe00700 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 0x5584763f1040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f62f8007fd0 tx=0x7f62f800f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0c4d601b293d78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:21.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.118+0000 7f62e9600700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:18:21.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.119+0000 7f62e9600700 1 -- 172.21.15.63:0/2768533006 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8017070 con 0x5584763e6820 2024-05-27T08:18:21.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.119+0000 7f62e9600700 1 -- 172.21.15.63:0/2768533006 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7f62f8011830 con 0x5584763e6820 2024-05-27T08:18:21.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.119+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ec019e50 con 0x5584763e6820 2024-05-27T08:18:21.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.120+0000 7f62e9600700 1 -- 172.21.15.63:0/2768533006 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f62f8021050 con 0x5584763e6820 2024-05-27T08:18:21.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.120+0000 7f631501fb80 10 client.42494 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:21.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.121+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5584762d5060 con 0x5584763e6820 2024-05-27T08:18:21.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.122+0000 7f62e9600700 1 -- 172.21.15.63:0/2768533006 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62ec019e50 con 0x5584763e6820 2024-05-27T08:18:21.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.122+0000 7f62e9600700 1 -- 172.21.15.63:0/2768533006 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f62f8011040 con 0x5584763e6820 2024-05-27T08:18:21.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.122+0000 7f631501fb80 10 client.42494 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:21.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.122+0000 7f631501fb80 10 client.42494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:18:21.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.122+0000 7f631501fb80 20 client.42494 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:21.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.122+0000 7f631501fb80 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5584764005f0 0x558476402a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.123+0000 7f631501fb80 4 client.42494 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:18:21.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.123+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558476316d90 con 0x5584764005f0 2024-05-27T08:18:21.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.123+0000 7f62ff400700 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5584764005f0 0x558476402a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.124+0000 7f62ff400700 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5584764005f0 0x558476402a90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34379114074de4a4 server_cookie=47ba48080eb63a7c in_seq=0 out_seq=0 2024-05-27T08:18:21.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.124+0000 7f62e9600700 10 client.42494 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:18:21.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.125+0000 7f62e9600700 1 -- 172.21.15.63:0/2768533006 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558476316d90 con 0x5584764005f0 2024-05-27T08:18:21.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.125+0000 7f62e9600700 10 client.42494 handle_command_reply: tid=0 2024-05-27T08:18:21.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.125+0000 7f631501fb80 1 client.42494 shutdown 2024-05-27T08:18:21.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.127+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 msgr2=0x5584763f1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:21.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.127+0000 7f631501fb80 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 0x5584763f1040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f62f8007fd0 tx=0x7f62f800f920 comp rx=0 tx=0).stop 2024-05-27T08:18:21.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.128+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 shutdown_connections 2024-05-27T08:18:21.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.130+0000 7f631501fb80 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6820 0x5584763f1040 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.130+0000 7f631501fb80 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 0x5584763f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.130+0000 7f631501fb80 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x5584764005f0 0x558476402a90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.130+0000 7f631501fb80 1 --2- 172.21.15.63:0/2768533006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763e6410 0x5584763f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.130+0000 7f62e9600700 0 client.42494 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:18:21.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.130+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 >> 172.21.15.63:0/2768533006 conn(0x5584762c7fc0 msgr2=0x5584763e0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:21.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.130+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 shutdown_connections 2024-05-27T08:18:21.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.130+0000 7f631501fb80 1 -- 172.21.15.63:0/2768533006 wait complete. 2024-05-27T08:18:21.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.131+0000 7f631501fb80 20 client.42494 trim_cache size 0 max 16384 2024-05-27T08:18:21.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.168+0000 7f631501fb80 1 Processor -- start 2024-05-27T08:18:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.168+0000 7f631501fb80 1 -- start start 2024-05-27T08:18:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.169+0000 7f631501fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 0x5584763ef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.169+0000 7f631501fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 0x5584763ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.169+0000 7f631501fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763e6820 0x5584763f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.170+0000 7f631501fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62e00116f0 con 0x5584763e6820 2024-05-27T08:18:21.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.170+0000 7f631501fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62e000ef80 con 0x5584763e6410 2024-05-27T08:18:21.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.170+0000 7f631501fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62e000cdf0 con 0x558476267e40 2024-05-27T08:18:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.170+0000 7f62fea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 0x5584763ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.170+0000 7f62fea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 0x5584763ebd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:52672/0 (socket says 172.21.15.63:52672) 2024-05-27T08:18:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.170+0000 7f62fea00700 1 -- 172.21.15.63:0/966712611 learned_addr learned my addr 172.21.15.63:0/966712611 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.170+0000 7f62ff400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 0x5584763ef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.170+0000 7f62ffe00700 1 --2- 172.21.15.63:0/966712611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763e6820 0x5584763f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.171+0000 7f62ff400700 1 -- 172.21.15.63:0/966712611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 msgr2=0x5584763ebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:21.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.171+0000 7f62ff400700 1 --2- 172.21.15.63:0/966712611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 0x5584763ebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.171+0000 7f62ff400700 1 -- 172.21.15.63:0/966712611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763e6820 msgr2=0x5584763f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:21.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.171+0000 7f62ff400700 1 --2- 172.21.15.63:0/966712611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763e6820 0x5584763f0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.171+0000 7f62ff400700 1 -- 172.21.15.63:0/966712611 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584763eeb30 con 0x558476267e40 2024-05-27T08:18:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.173+0000 7f62ff400700 1 --2- 172.21.15.63:0/966712611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 0x5584763ef520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5584764031d0 tx=0x7f62f0015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae22df902eb25fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.174+0000 7f62ebe00700 1 -- 172.21.15.63:0/966712611 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0036030 con 0x558476267e40 2024-05-27T08:18:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.174+0000 7f62ebe00700 1 -- 172.21.15.63:0/966712611 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x558476316d90 con 0x558476267e40 2024-05-27T08:18:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.174+0000 7f62ebe00700 1 -- 172.21.15.63:0/966712611 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f001b040 con 0x558476267e40 2024-05-27T08:18:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.175+0000 7f631501fb80 1 -- 172.21.15.63:0/966712611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 msgr2=0x5584763ef520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.175+0000 7f631501fb80 1 --2- 172.21.15.63:0/966712611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 0x5584763ef520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5584764031d0 tx=0x7f62f0015000 comp rx=0 tx=0).stop 2024-05-27T08:18:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.175+0000 7f631501fb80 1 -- 172.21.15.63:0/966712611 shutdown_connections 2024-05-27T08:18:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.175+0000 7f631501fb80 1 --2- 172.21.15.63:0/966712611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763e6820 0x5584763f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.175+0000 7f631501fb80 1 --2- 172.21.15.63:0/966712611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558476267e40 0x5584763ef520 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.175+0000 7f631501fb80 1 --2- 172.21.15.63:0/966712611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 0x5584763ebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.175+0000 7f631501fb80 1 -- 172.21.15.63:0/966712611 >> 172.21.15.63:0/966712611 conn(0x5584762c7fc0 msgr2=0x5584763f0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.176+0000 7f631501fb80 1 -- 172.21.15.63:0/966712611 shutdown_connections 2024-05-27T08:18:21.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.176+0000 7f631501fb80 1 -- 172.21.15.63:0/966712611 wait complete. 2024-05-27T08:18:21.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.177+0000 7f631501fb80 1 -- start start 2024-05-27T08:18:21.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.177+0000 7f631501fb80 1 Processor -- start 2024-05-27T08:18:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.178+0000 7f631501fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 0x5584763f4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.179+0000 7f631501fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6820 0x5584763f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.179+0000 7f631501fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763f52f0 0x55847646a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.179+0000 7f631501fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62e000d440 con 0x5584763f52f0 2024-05-27T08:18:21.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.179+0000 7f631501fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62e000e8b0 con 0x5584763e6410 2024-05-27T08:18:21.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.179+0000 7f631501fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62e000db10 con 0x5584763e6820 2024-05-27T08:18:21.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.179+0000 7f62fea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6820 0x5584763f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.179+0000 7f631501fb80 10 client.0 _ll_register_callbacks cb 0x5584762aec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:21.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.180+0000 7f62fea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6820 0x5584763f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41174/0 (socket says 172.21.15.63:41174) 2024-05-27T08:18:21.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.180+0000 7f62fea00700 1 -- 172.21.15.63:0/271936314 learned_addr learned my addr 172.21.15.63:0/271936314 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:21.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.180+0000 7f62ff400700 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 0x5584763f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.180+0000 7f62fea00700 1 -- 172.21.15.63:0/271936314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 msgr2=0x5584763f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:21.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.180+0000 7f62fea00700 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 0x5584763f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.180+0000 7f62fea00700 1 -- 172.21.15.63:0/271936314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763f52f0 msgr2=0x55847646a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:21.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.180+0000 7f62fea00700 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763f52f0 0x55847646a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.180+0000 7f62fea00700 1 -- 172.21.15.63:0/271936314 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847646a7a0 con 0x5584763e6820 2024-05-27T08:18:21.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.182+0000 7f62fea00700 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6820 0x5584763f4db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0058b0 tx=0x7f62ec0a7e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63d15806fa90b5a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:21.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.182+0000 7f62e9600700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:18:21.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.183+0000 7f62e9600700 1 -- 172.21.15.63:0/271936314 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0a8510 con 0x5584763e6820 2024-05-27T08:18:21.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.183+0000 7f62e9600700 1 -- 172.21.15.63:0/271936314 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7f62ec00f460 con 0x5584763e6820 2024-05-27T08:18:21.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.183+0000 7f631501fb80 1 -- 172.21.15.63:0/271936314 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558476316d90 con 0x5584763e6820 2024-05-27T08:18:21.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.184+0000 7f62e9600700 1 -- 172.21.15.63:0/271936314 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f62ec0a86b0 con 0x5584763e6820 2024-05-27T08:18:21.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.185+0000 7f631501fb80 10 client.32332 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:21.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.185+0000 7f631501fb80 1 -- 172.21.15.63:0/271936314 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55847646b9e0 con 0x5584763e6820 2024-05-27T08:18:21.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.187+0000 7f62e9600700 1 -- 172.21.15.63:0/271936314 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x558476316d90 con 0x5584763e6820 2024-05-27T08:18:21.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.188+0000 7f62e9600700 1 -- 172.21.15.63:0/271936314 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f62ec0acce0 con 0x5584763e6820 2024-05-27T08:18:21.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.188+0000 7f631501fb80 10 client.32332 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:21.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.188+0000 7f631501fb80 10 client.32332 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:18:21.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.188+0000 7f631501fb80 20 client.32332 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:21.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.188+0000 7f631501fb80 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55847646be80 0x55847646e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:21.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.188+0000 7f631501fb80 4 client.32332 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:18:21.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.188+0000 7f631501fb80 1 -- 172.21.15.63:0/271936314 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558476310570 con 0x55847646be80 2024-05-27T08:18:21.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.188+0000 7f62ff400700 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55847646be80 0x55847646e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:21.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.194+0000 7f62ff400700 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55847646be80 0x55847646e2a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c89bcd8bb73b2b16 server_cookie=aaa66bab7d369e4f in_seq=0 out_seq=0 2024-05-27T08:18:21.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.194+0000 7f62e9600700 10 client.32332 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:18:21.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.195+0000 7f62e9600700 1 -- 172.21.15.63:0/271936314 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558476310570 con 0x55847646be80 2024-05-27T08:18:21.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.195+0000 7f62e9600700 10 client.32332 handle_command_reply: tid=0 2024-05-27T08:18:21.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.195+0000 7f631501fb80 1 client.32332 shutdown 2024-05-27T08:18:21.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.196+0000 7f631501fb80 1 -- 172.21.15.63:0/271936314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6820 msgr2=0x5584763f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:21.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.196+0000 7f631501fb80 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6820 0x5584763f4db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0058b0 tx=0x7f62ec0a7e50 comp rx=0 tx=0).stop 2024-05-27T08:18:21.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.196+0000 7f631501fb80 1 -- 172.21.15.63:0/271936314 shutdown_connections 2024-05-27T08:18:21.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.196+0000 7f631501fb80 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55847646be80 0x55847646e2a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.196+0000 7f631501fb80 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5584763f52f0 0x55847646a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.196+0000 7f631501fb80 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584763e6410 0x5584763f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.196+0000 7f631501fb80 1 --2- 172.21.15.63:0/271936314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584763e6820 0x5584763f4db0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.196+0000 7f631501fb80 1 -- 172.21.15.63:0/271936314 >> 172.21.15.63:0/271936314 conn(0x5584762c7fc0 msgr2=0x5584763e81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:21.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.196+0000 7f62e9600700 0 client.32332 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:18:21.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.197+0000 7f631501fb80 1 -- 172.21.15.63:0/271936314 shutdown_connections 2024-05-27T08:18:21.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.197+0000 7f631501fb80 1 -- 172.21.15.63:0/271936314 wait complete. 2024-05-27T08:18:21.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.197+0000 7f631501fb80 20 client.32332 trim_cache size 0 max 16384 2024-05-27T08:18:21.200 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.199+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62e40b6eb0 msgr2=0x7f62e40b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62e40b6eb0 0x7f62e40b93c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009ad0 tx=0x7f62f0009290 comp rx=0 tx=0).stop 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f630010b350 msgr2=0x7f63001a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f630010b350 0x7f63001a1840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f62ec000c00 tx=0x7f62ec0057c0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 shutdown_connections 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f62e40b6eb0 0x7f62e40b93c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63001119c0 0x7f63001a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f630010aa00 0x7f63001a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 --2- 172.21.15.63:0/4057221113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f630010b350 0x7f63001a1840 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 >> 172.21.15.63:0/4057221113 conn(0x7f630006f6d0 msgr2=0x7f6300074160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 shutdown_connections 2024-05-27T08:18:21.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:21.200+0000 7f6304e00700 1 -- 172.21.15.63:0/4057221113 wait complete. 2024-05-27T08:18:21.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:20 smithi086 ceph-mon[34687]: pgmap v1423: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.8 MiB/s wr, 1.68k op/s 2024-05-27T08:18:21.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:20 smithi149 ceph-mon[35408]: pgmap v1423: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.8 MiB/s wr, 1.68k op/s 2024-05-27T08:18:22.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:22 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:18:22.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:22 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:18:22.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:22 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:18:22.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:22 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:18:22.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:22 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:18:22.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:22 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:18:22.469 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:22 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:18:22.470 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:22 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:18:22.470 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:22 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:18:23.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:23 smithi086 ceph-mon[34687]: pgmap v1424: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.5 MiB/s wr, 1.86k op/s 2024-05-27T08:18:23.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:23 smithi149 ceph-mon[35408]: pgmap v1424: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.5 MiB/s wr, 1.86k op/s 2024-05-27T08:18:23.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:23 smithi063 ceph-mon[31613]: pgmap v1424: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.5 MiB/s wr, 1.86k op/s 2024-05-27T08:18:24.022 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:18:25.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:25 smithi063 ceph-mon[31613]: pgmap v1425: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.6 MiB/s wr, 1.65k op/s 2024-05-27T08:18:25.336 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:18:25.462 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:25 smithi086 ceph-mon[34687]: pgmap v1425: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.6 MiB/s wr, 1.65k op/s 2024-05-27T08:18:25.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:25 smithi149 ceph-mon[35408]: pgmap v1425: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.6 MiB/s wr, 1.65k op/s 2024-05-27T08:18:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.720+0000 7fefa0a00640 1 -- 172.21.15.63:0/3624334050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9c10db90 msgr2=0x7fef9c10ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.720+0000 7fefa0a00640 1 --2- 172.21.15.63:0/3624334050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9c10db90 0x7fef9c10ffb0 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7fef88009900 tx=0x7fef8802a480 comp rx=0 tx=0).stop 2024-05-27T08:18:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.720+0000 7fefa0a00640 1 -- 172.21.15.63:0/3624334050 shutdown_connections 2024-05-27T08:18:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.720+0000 7fefa0a00640 1 --2- 172.21.15.63:0/3624334050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef9c1104f0 0x7fef9c112910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.720+0000 7fefa0a00640 1 --2- 172.21.15.63:0/3624334050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9c10db90 0x7fef9c10ffb0 unknown :-1 s=CLOSED pgs=2972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.720+0000 7fefa0a00640 1 --2- 172.21.15.63:0/3624334050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef9c076510 0x7fef9c076920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.720+0000 7fefa0a00640 1 -- 172.21.15.63:0/3624334050 >> 172.21.15.63:0/3624334050 conn(0x7fef9c0ff340 msgr2=0x7fef9c101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.721+0000 7fefa0a00640 1 -- 172.21.15.63:0/3624334050 shutdown_connections 2024-05-27T08:18:26.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.721+0000 7fefa0a00640 1 -- 172.21.15.63:0/3624334050 wait complete. 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fefa0a00640 1 Processor -- start 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fefa0a00640 1 -- start start 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fefa0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef9c076510 0x7fef9c1a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fefa0a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9c10db90 0x7fef9c1a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fefa0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef9c1104f0 0x7fef9c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fefa0a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c115f30 con 0x7fef9c10db90 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fefa0a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c115db0 con 0x7fef9c076510 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fefa0a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1160b0 con 0x7fef9c1104f0 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fef99400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9c10db90 0x7fef9c1a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fef99400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9c10db90 0x7fef9c1a2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57594/0 (socket says 172.21.15.63:57594) 2024-05-27T08:18:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.722+0000 7fef99400640 1 -- 172.21.15.63:0/4075609130 learned_addr learned my addr 172.21.15.63:0/4075609130 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.723+0000 7fef99e00640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef9c076510 0x7fef9c1a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.723+0000 7fef99400640 1 -- 172.21.15.63:0/4075609130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef9c1104f0 msgr2=0x7fef9c19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.723+0000 7fef99400640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef9c1104f0 0x7fef9c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.723+0000 7fef99400640 1 -- 172.21.15.63:0/4075609130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef9c076510 msgr2=0x7fef9c1a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.723+0000 7fef99400640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef9c076510 0x7fef9c1a2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.723+0000 7fef99400640 1 -- 172.21.15.63:0/4075609130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c19ce80 con 0x7fef9c10db90 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.723+0000 7fef99400640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9c10db90 0x7fef9c1a2bc0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7fef88005940 tx=0x7fef880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.724+0000 7fef92a00640 1 -- 172.21.15.63:0/4075609130 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8803f030 con 0x7fef9c10db90 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.724+0000 7fef92a00640 1 -- 172.21.15.63:0/4075609130 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef88003c90 con 0x7fef9c10db90 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.724+0000 7fef92a00640 1 -- 172.21.15.63:0/4075609130 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88035070 con 0x7fef9c10db90 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.724+0000 7fefa0a00640 1 -- 172.21.15.63:0/4075609130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c19d110 con 0x7fef9c10db90 2024-05-27T08:18:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.724+0000 7fefa0a00640 1 -- 172.21.15.63:0/4075609130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c19d5b0 con 0x7fef9c10db90 2024-05-27T08:18:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.727+0000 7fefa79aa740 1 Processor -- start 2024-05-27T08:18:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.727+0000 7fefa79aa740 1 -- start start 2024-05-27T08:18:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4c9c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fef92a00640 1 -- 172.21.15.63:0/4075609130 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fef88042030 con 0x7fef9c10db90 2024-05-27T08:18:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4c9cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4c9d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fefa79aa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cff4c0db90 con 0x55cff48d94d0 2024-05-27T08:18:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fefa79aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cff4bfd730 con 0x55cff48d12f0 2024-05-27T08:18:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fefa79aa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cff4c01770 con 0x55cff48fa1b0 2024-05-27T08:18:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fef99400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4c9d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fef99400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4c9d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57600/0 (socket says 172.21.15.63:57600) 2024-05-27T08:18:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fef99400640 1 -- 172.21.15.63:0/127273302 learned_addr learned my addr 172.21.15.63:0/127273302 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fef9a800640 1 --2- 172.21.15.63:0/127273302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4c9c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.728+0000 7fef99e00640 1 --2- 172.21.15.63:0/127273302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4c9cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.729+0000 7fef99400640 1 -- 172.21.15.63:0/127273302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 msgr2=0x55cff4c9c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.729+0000 7fef99400640 1 --2- 172.21.15.63:0/127273302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4c9c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.729+0000 7fef99400640 1 -- 172.21.15.63:0/127273302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 msgr2=0x55cff4c9cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.729+0000 7fef99400640 1 --2- 172.21.15.63:0/127273302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4c9cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.729+0000 7fef99400640 1 -- 172.21.15.63:0/127273302 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff4b968b0 con 0x55cff48d94d0 2024-05-27T08:18:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.729+0000 7fef99400640 1 --2- 172.21.15.63:0/127273302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4c9d2e0 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7fef881002d0 tx=0x7fef8810b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26019707ccbf28f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.729+0000 7fef87000640 1 -- 172.21.15.63:0/127273302 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8810bbb0 con 0x55cff48d94d0 2024-05-27T08:18:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.729+0000 7fef87000640 1 -- 172.21.15.63:0/127273302 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef88110030 con 0x55cff48d94d0 2024-05-27T08:18:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.729+0000 7fef87000640 1 -- 172.21.15.63:0/127273302 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88114630 con 0x55cff48d94d0 2024-05-27T08:18:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.729+0000 7fef92a00640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef800b29d0 0x7fef800b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.730+0000 7fef92a00640 1 -- 172.21.15.63:0/4075609130 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7fef88103100 con 0x7fef9c10db90 2024-05-27T08:18:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.730+0000 7fef9a800640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef800b29d0 0x7fef800b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.730+0000 7fefa79aa740 1 -- 172.21.15.63:0/127273302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 msgr2=0x55cff4c9d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.730+0000 7fefa79aa740 1 --2- 172.21.15.63:0/127273302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4c9d2e0 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7fef881002d0 tx=0x7fef8810b3b0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.730+0000 7fefa79aa740 1 -- 172.21.15.63:0/127273302 shutdown_connections 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.730+0000 7fefa79aa740 1 --2- 172.21.15.63:0/127273302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4c9d2e0 unknown :-1 s=CLOSED pgs=2974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.730+0000 7fefa79aa740 1 --2- 172.21.15.63:0/127273302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4c9cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.730+0000 7fefa79aa740 1 --2- 172.21.15.63:0/127273302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4c9c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.730+0000 7fefa79aa740 1 -- 172.21.15.63:0/127273302 >> 172.21.15.63:0/127273302 conn(0x55cff48ee300 msgr2=0x55cff4c99010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.730+0000 7fef9a800640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef800b29d0 0x7fef800b4ee0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55cff4c999d0 tx=0x7fef9400d040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.731+0000 7fefa79aa740 1 -- 172.21.15.63:0/127273302 shutdown_connections 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.731+0000 7fefa79aa740 1 -- 172.21.15.63:0/127273302 wait complete. 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.731+0000 7fefa79aa740 1 -- start start 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.731+0000 7fefa79aa740 1 Processor -- start 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48d12f0 0x55cff4ca69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d94d0 0x55cff4ca6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48fa1b0 0x55cff4cb2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fefa79aa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cff48e3790 con 0x55cff48fa1b0 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fefa79aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cff4ba5bc0 con 0x55cff48d94d0 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fefa79aa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55cff4ca9750 con 0x55cff48d12f0 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fefa79aa740 10 client.0 _ll_register_callbacks cb 0x55cff4b83400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fef9a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48fa1b0 0x55cff4cb2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fef9a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48fa1b0 0x55cff4cb2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57610/0 (socket says 172.21.15.63:57610) 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fef9a800640 1 -- 172.21.15.63:0/1688053704 learned_addr learned my addr 172.21.15.63:0/1688053704 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.732+0000 7fef99e00640 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48d12f0 0x55cff4ca69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fef99400640 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d94d0 0x55cff4ca6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fef9a800640 1 -- 172.21.15.63:0/1688053704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48d12f0 msgr2=0x55cff4ca69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fef9a800640 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48d12f0 0x55cff4ca69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fef9a800640 1 -- 172.21.15.63:0/1688053704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d94d0 msgr2=0x55cff4ca6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fef9a800640 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d94d0 0x55cff4ca6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fef9a800640 1 -- 172.21.15.63:0/1688053704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff4912210 con 0x55cff48fa1b0 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fef9a800640 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48fa1b0 0x55cff4cb2a00 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7fef94016d70 tx=0x7fef94004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c49e293dc7690d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fef7b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fef7b400640 1 -- 172.21.15.63:0/1688053704 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94025060 con 0x55cff48fa1b0 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fefa79aa740 1 -- 172.21.15.63:0/1688053704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef88110030 con 0x55cff48fa1b0 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.733+0000 7fef7b400640 1 -- 172.21.15.63:0/1688053704 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7fef940205f0 con 0x55cff48fa1b0 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fef7b400640 1 -- 172.21.15.63:0/1688053704 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fef9402c050 con 0x55cff48fa1b0 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fefa79aa740 10 client.23634 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fefa79aa740 1 -- 172.21.15.63:0/1688053704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cff49de620 con 0x55cff48fa1b0 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fef7b400640 1 -- 172.21.15.63:0/1688053704 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef88110030 con 0x55cff48fa1b0 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fef7b400640 1 -- 172.21.15.63:0/1688053704 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fef9402e2e0 con 0x55cff48fa1b0 2024-05-27T08:18:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fefa79aa740 10 client.23634 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fefa79aa740 10 client.23634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:18:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fefa79aa740 20 client.23634 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fefa79aa740 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cff48b8d80 0x55cff4cb89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fefa79aa740 4 client.23634 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:18:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.734+0000 7fefa79aa740 1 -- 172.21.15.63:0/1688053704 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cff4c0fae0 con 0x55cff48b8d80 2024-05-27T08:18:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.735+0000 7fef99e00640 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cff48b8d80 0x55cff4cb89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.736+0000 7fef99e00640 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cff48b8d80 0x55cff4cb89e0 crc :-1 s=READY pgs=770 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=141eca3f12650980 server_cookie=e6dc91d81a676f61 in_seq=0 out_seq=0 2024-05-27T08:18:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.736+0000 7fef7b400640 10 client.23634 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fef7b400640 1 -- 172.21.15.63:0/1688053704 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cff4c0fae0 con 0x55cff48b8d80 2024-05-27T08:18:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fef7b400640 10 client.23634 handle_command_reply: tid=0 2024-05-27T08:18:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fefa79aa740 1 client.23634 shutdown 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fefa79aa740 1 -- 172.21.15.63:0/1688053704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48fa1b0 msgr2=0x55cff4cb2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fefa79aa740 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48fa1b0 0x55cff4cb2a00 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7fef94016d70 tx=0x7fef94004bc0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fefa79aa740 1 -- 172.21.15.63:0/1688053704 shutdown_connections 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fefa79aa740 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cff48b8d80 0x55cff4cb89e0 crc :-1 s=READY pgs=770 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fefa79aa740 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48fa1b0 0x55cff4cb2a00 unknown :-1 s=CLOSED pgs=2975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fefa79aa740 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d94d0 0x55cff4ca6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fef7b400640 0 client.23634 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fefa79aa740 1 --2- 172.21.15.63:0/1688053704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48d12f0 0x55cff4ca69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fefa79aa740 1 -- 172.21.15.63:0/1688053704 >> 172.21.15.63:0/1688053704 conn(0x55cff48ee300 msgr2=0x55cff4c9e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.738+0000 7fefa79aa740 1 -- 172.21.15.63:0/1688053704 shutdown_connections 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.739+0000 7fefa79aa740 1 -- 172.21.15.63:0/1688053704 wait complete. 2024-05-27T08:18:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.739+0000 7fefa79aa740 20 client.23634 trim_cache size 0 max 16384 2024-05-27T08:18:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.755+0000 7fefa79aa740 1 Processor -- start 2024-05-27T08:18:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.755+0000 7fefa79aa740 1 -- start start 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.755+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4ca21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.755+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cb27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.755+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4cb2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.755+0000 7fefa79aa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1b5e20 con 0x55cff48d94d0 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.755+0000 7fefa79aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1b36f0 con 0x55cff48d12f0 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.755+0000 7fefa79aa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1b1560 con 0x55cff48fa1b0 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef99400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cb27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef99400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cb27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57618/0 (socket says 172.21.15.63:57618) 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef99400640 1 -- 172.21.15.63:0/1249364557 learned_addr learned my addr 172.21.15.63:0/1249364557 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef99e00640 1 --2- 172.21.15.63:0/1249364557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4ca21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef9a800640 1 --2- 172.21.15.63:0/1249364557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4cb2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef99400640 1 -- 172.21.15.63:0/1249364557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 msgr2=0x55cff4cb2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef99400640 1 --2- 172.21.15.63:0/1249364557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4cb2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef99400640 1 -- 172.21.15.63:0/1249364557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 msgr2=0x55cff4ca21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef99400640 1 --2- 172.21.15.63:0/1249364557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4ca21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef99400640 1 -- 172.21.15.63:0/1249364557 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff4c93050 con 0x55cff48d94d0 2024-05-27T08:18:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.756+0000 7fef99400640 1 --2- 172.21.15.63:0/1249364557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cb27b0 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7fef88000c00 tx=0x7fef88005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc609db084a248a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fef86600640 1 -- 172.21.15.63:0/1249364557 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88049030 con 0x55cff48d94d0 2024-05-27T08:18:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fef86600640 1 -- 172.21.15.63:0/1249364557 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef88110070 con 0x55cff48d94d0 2024-05-27T08:18:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fef86600640 1 -- 172.21.15.63:0/1249364557 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef880c9a30 con 0x55cff48d94d0 2024-05-27T08:18:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fefa79aa740 1 -- 172.21.15.63:0/1249364557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 msgr2=0x55cff4cb27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fefa79aa740 1 --2- 172.21.15.63:0/1249364557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cb27b0 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7fef88000c00 tx=0x7fef88005de0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fefa79aa740 1 -- 172.21.15.63:0/1249364557 shutdown_connections 2024-05-27T08:18:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fefa79aa740 1 --2- 172.21.15.63:0/1249364557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4cb2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fefa79aa740 1 --2- 172.21.15.63:0/1249364557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cb27b0 unknown :-1 s=CLOSED pgs=2976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fefa79aa740 1 --2- 172.21.15.63:0/1249364557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4ca21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fefa79aa740 1 -- 172.21.15.63:0/1249364557 >> 172.21.15.63:0/1249364557 conn(0x55cff48ee300 msgr2=0x55cff4ca8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fefa79aa740 1 -- 172.21.15.63:0/1249364557 shutdown_connections 2024-05-27T08:18:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.757+0000 7fefa79aa740 1 -- 172.21.15.63:0/1249364557 wait complete. 2024-05-27T08:18:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.758+0000 7fefa79aa740 1 -- start start 2024-05-27T08:18:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.758+0000 7fefa79aa740 1 Processor -- start 2024-05-27T08:18:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.758+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4cabe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fefa79aa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4d1f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fefa79aa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1b1bb0 con 0x55cff48d94d0 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fefa79aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1b3020 con 0x55cff48d12f0 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fefa79aa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1b2280 con 0x55cff48fa1b0 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fefa79aa740 10 client.0 _ll_register_callbacks cb 0x7fef94009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cac3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57632/0 (socket says 172.21.15.63:57632) 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99400640 1 -- 172.21.15.63:0/209182137 learned_addr learned my addr 172.21.15.63:0/209182137 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99e00640 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4cabe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef9a800640 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4d1f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99400640 1 -- 172.21.15.63:0/209182137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 msgr2=0x55cff4d1f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99400640 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4d1f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99400640 1 -- 172.21.15.63:0/209182137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 msgr2=0x55cff4cabe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99400640 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4cabe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99400640 1 -- 172.21.15.63:0/209182137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff4d1f5f0 con 0x55cff48d94d0 2024-05-27T08:18:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99e00640 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4cabe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:18:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.759+0000 7fef99400640 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cac3d0 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7fef8810b240 tx=0x7fef88003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac8d920252de637 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fef7b400640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:18:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fef7b400640 1 -- 172.21.15.63:0/209182137 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88049070 con 0x55cff48d94d0 2024-05-27T08:18:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fef7b400640 1 -- 172.21.15.63:0/209182137 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7fef880c6510 con 0x55cff48d94d0 2024-05-27T08:18:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fefa79aa740 1 -- 172.21.15.63:0/209182137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef88110070 con 0x55cff48d94d0 2024-05-27T08:18:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fef7b400640 1 -- 172.21.15.63:0/209182137 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fef880c9cf0 con 0x55cff48d94d0 2024-05-27T08:18:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fefa79aa740 10 client.23640 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fefa79aa740 1 -- 172.21.15.63:0/209182137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cff4d20810 con 0x55cff48d94d0 2024-05-27T08:18:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fef7b400640 1 -- 172.21.15.63:0/209182137 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef88110070 con 0x55cff48d94d0 2024-05-27T08:18:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fef7b400640 1 -- 172.21.15.63:0/209182137 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fef880cd070 con 0x55cff48d94d0 2024-05-27T08:18:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fefa79aa740 10 client.23640 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fefa79aa740 10 client.23640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:18:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.760+0000 7fefa79aa740 20 client.23640 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.761+0000 7fefa79aa740 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cff488e1c0 0x55cff4d22ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.761+0000 7fefa79aa740 4 client.23640 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:18:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.761+0000 7fefa79aa740 1 -- 172.21.15.63:0/209182137 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cff4c092c0 con 0x55cff488e1c0 2024-05-27T08:18:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.761+0000 7fef99e00640 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cff488e1c0 0x55cff4d22ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.762+0000 7fef99e00640 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cff488e1c0 0x55cff4d22ca0 crc :-1 s=READY pgs=771 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d915c4cbe402b622 server_cookie=e4abc1dbfddd5a4a in_seq=0 out_seq=0 2024-05-27T08:18:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:26.762+0000 7fef7b400640 10 client.23640 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:27.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:27 smithi086 ceph-mon[34687]: pgmap v1426: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.9 MiB/s wr, 1.71k op/s 2024-05-27T08:18:27.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:27 smithi149 ceph-mon[35408]: pgmap v1426: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.9 MiB/s wr, 1.71k op/s 2024-05-27T08:18:27.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:26 smithi063 ceph-mon[31613]: pgmap v1426: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.9 MiB/s wr, 1.71k op/s 2024-05-27T08:18:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.724+0000 7fef7b400640 1 -- 172.21.15.63:0/209182137 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cff4c092c0 con 0x55cff488e1c0 2024-05-27T08:18:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.724+0000 7fef7b400640 10 client.23640 handle_command_reply: tid=0 2024-05-27T08:18:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.724+0000 7fefa79aa740 1 client.23640 shutdown 2024-05-27T08:18:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.724+0000 7fefa79aa740 1 -- 172.21.15.63:0/209182137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 msgr2=0x55cff4cac3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.724+0000 7fefa79aa740 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cac3d0 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7fef8810b240 tx=0x7fef88003f30 comp rx=0 tx=0).stop 2024-05-27T08:18:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.725+0000 7fefa79aa740 1 -- 172.21.15.63:0/209182137 shutdown_connections 2024-05-27T08:18:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.725+0000 7fefa79aa740 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55cff488e1c0 0x55cff4d22ca0 crc :-1 s=READY pgs=771 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.725+0000 7fefa79aa740 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cff48fa1b0 0x55cff4d1f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.725+0000 7fefa79aa740 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cff48d94d0 0x55cff4cac3d0 unknown :-1 s=CLOSED pgs=2977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.725+0000 7fefa79aa740 1 --2- 172.21.15.63:0/209182137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff48d12f0 0x55cff4cabe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.725+0000 7fefa79aa740 1 -- 172.21.15.63:0/209182137 >> 172.21.15.63:0/209182137 conn(0x55cff48ee300 msgr2=0x55cff4ca8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.725+0000 7fef7b400640 0 client.23640 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.725+0000 7fefa79aa740 1 -- 172.21.15.63:0/209182137 shutdown_connections 2024-05-27T08:18:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.726+0000 7fefa79aa740 1 -- 172.21.15.63:0/209182137 wait complete. 2024-05-27T08:18:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.726+0000 7fefa79aa740 20 client.23640 trim_cache size 0 max 16384 2024-05-27T08:18:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:18:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:18:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:18:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.727+0000 7fefa0a00640 1 -- 172.21.15.63:0/4075609130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef800b29d0 msgr2=0x7fef800b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.727+0000 7fefa0a00640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef800b29d0 0x7fef800b4ee0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55cff4c999d0 tx=0x7fef9400d040 comp rx=0 tx=0).stop 2024-05-27T08:18:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.727+0000 7fefa0a00640 1 -- 172.21.15.63:0/4075609130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9c10db90 msgr2=0x7fef9c1a2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.727+0000 7fefa0a00640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9c10db90 0x7fef9c1a2bc0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7fef88005940 tx=0x7fef880042f0 comp rx=0 tx=0).stop 2024-05-27T08:18:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.728+0000 7fefa0a00640 1 -- 172.21.15.63:0/4075609130 shutdown_connections 2024-05-27T08:18:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.728+0000 7fefa0a00640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fef800b29d0 0x7fef800b4ee0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.728+0000 7fefa0a00640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef9c1104f0 0x7fef9c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.728+0000 7fefa0a00640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef9c10db90 0x7fef9c1a2bc0 unknown :-1 s=CLOSED pgs=2973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.728+0000 7fefa0a00640 1 --2- 172.21.15.63:0/4075609130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef9c076510 0x7fef9c1a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.728+0000 7fefa0a00640 1 -- 172.21.15.63:0/4075609130 >> 172.21.15.63:0/4075609130 conn(0x7fef9c0ff340 msgr2=0x7fef9c110a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.728+0000 7fefa0a00640 1 -- 172.21.15.63:0/4075609130 shutdown_connections 2024-05-27T08:18:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:27.728+0000 7fefa0a00640 1 -- 172.21.15.63:0/4075609130 wait complete. 2024-05-27T08:18:29.258 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:28 smithi063 ceph-mon[31613]: pgmap v1427: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.8 MiB/s wr, 1.91k op/s 2024-05-27T08:18:29.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:28 smithi086 ceph-mon[34687]: pgmap v1427: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.8 MiB/s wr, 1.91k op/s 2024-05-27T08:18:29.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:28 smithi149 ceph-mon[35408]: pgmap v1427: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.8 MiB/s wr, 1.91k op/s 2024-05-27T08:18:31.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:31 smithi086 ceph-mon[34687]: pgmap v1428: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.9 MiB/s wr, 1.94k op/s 2024-05-27T08:18:31.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:31 smithi149 ceph-mon[35408]: pgmap v1428: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.9 MiB/s wr, 1.94k op/s 2024-05-27T08:18:31.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:31 smithi063 ceph-mon[31613]: pgmap v1428: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.9 MiB/s wr, 1.94k op/s 2024-05-27T08:18:33.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:33 smithi086 ceph-mon[34687]: pgmap v1429: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 11 MiB/s wr, 2.70k op/s 2024-05-27T08:18:33.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:33 smithi149 ceph-mon[35408]: pgmap v1429: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 11 MiB/s wr, 2.70k op/s 2024-05-27T08:18:33.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:33 smithi063 ceph-mon[31613]: pgmap v1429: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 11 MiB/s wr, 2.70k op/s 2024-05-27T08:18:35.265 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:35 smithi063 ceph-mon[31613]: pgmap v1430: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.7 MiB/s wr, 2.38k op/s 2024-05-27T08:18:35.269 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:35 smithi149 ceph-mon[35408]: pgmap v1430: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.7 MiB/s wr, 2.38k op/s 2024-05-27T08:18:35.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:35 smithi086 ceph-mon[34687]: pgmap v1430: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.7 MiB/s wr, 2.38k op/s 2024-05-27T08:18:35.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.769+0000 7fd438a00700 1 -- 172.21.15.63:0/3382989329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd434108f10 msgr2=0x7fd434109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:35.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.769+0000 7fd438a00700 1 --2- 172.21.15.63:0/3382989329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd434108f10 0x7fd434109320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0099c0 tx=0x7fd41c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.769+0000 7fd438a00700 1 -- 172.21.15.63:0/3382989329 shutdown_connections 2024-05-27T08:18:35.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.769+0000 7fd438a00700 1 --2- 172.21.15.63:0/3382989329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd434109860 0x7fd4341111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.769+0000 7fd438a00700 1 --2- 172.21.15.63:0/3382989329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4341045e0 0x7fd4341049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.769+0000 7fd438a00700 1 --2- 172.21.15.63:0/3382989329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd434108f10 0x7fd434109320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.769+0000 7fd438a00700 1 -- 172.21.15.63:0/3382989329 >> 172.21.15.63:0/3382989329 conn(0x7fd434100030 msgr2=0x7fd434102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:35.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.771+0000 7fd438a00700 1 -- 172.21.15.63:0/3382989329 shutdown_connections 2024-05-27T08:18:35.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.771+0000 7fd438a00700 1 -- 172.21.15.63:0/3382989329 wait complete. 2024-05-27T08:18:35.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.773+0000 7fd438a00700 1 Processor -- start 2024-05-27T08:18:35.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.773+0000 7fd438a00700 1 -- start start 2024-05-27T08:18:35.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.774+0000 7fd438a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4341045e0 0x7fd434075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.774+0000 7fd438a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd434108f10 0x7fd434075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.774+0000 7fd438a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd434109860 0x7fd434076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.774+0000 7fd438a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd434114370 con 0x7fd434108f10 2024-05-27T08:18:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.774+0000 7fd438a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341141f0 con 0x7fd434109860 2024-05-27T08:18:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.774+0000 7fd438a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341144f0 con 0x7fd4341045e0 2024-05-27T08:18:35.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.775+0000 7fd432800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd434109860 0x7fd434076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.776+0000 7fd432800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd434109860 0x7fd434076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:37008/0 (socket says 172.21.15.63:37008) 2024-05-27T08:18:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.776+0000 7fd432800700 1 -- 172.21.15.63:0/2108576212 learned_addr learned my addr 172.21.15.63:0/2108576212 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.776+0000 7fd431e00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4341045e0 0x7fd434075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.776+0000 7fd42be00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd434108f10 0x7fd434075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.777+0000 7fd432800700 1 -- 172.21.15.63:0/2108576212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4341045e0 msgr2=0x7fd434075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:35.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.777+0000 7fd432800700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4341045e0 0x7fd434075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.777+0000 7fd432800700 1 -- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd434108f10 msgr2=0x7fd434075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:35.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.777+0000 7fd432800700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd434108f10 0x7fd434075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.777+0000 7fd432800700 1 -- 172.21.15.63:0/2108576212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4341a9140 con 0x7fd434109860 2024-05-27T08:18:35.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.777+0000 7fd431e00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4341045e0 0x7fd434075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:18:35.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.777+0000 7fd432800700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd434109860 0x7fd434076530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00ef30 tx=0x7fd42c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:35.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.777+0000 7fd42be00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd434108f10 0x7fd434075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:18:35.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.778+0000 7fd42aa00700 1 -- 172.21.15.63:0/2108576212 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0092e0 con 0x7fd434109860 2024-05-27T08:18:35.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.778+0000 7fd42aa00700 1 -- 172.21.15.63:0/2108576212 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd42c004bb0 con 0x7fd434109860 2024-05-27T08:18:35.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.778+0000 7fd438a00700 1 -- 172.21.15.63:0/2108576212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4341a93d0 con 0x7fd434109860 2024-05-27T08:18:35.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.778+0000 7fd438a00700 1 -- 172.21.15.63:0/2108576212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4341a98e0 con 0x7fd434109860 2024-05-27T08:18:35.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.778+0000 7fd42aa00700 1 -- 172.21.15.63:0/2108576212 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c004020 con 0x7fd434109860 2024-05-27T08:18:35.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.797+0000 7fd42aa00700 1 -- 172.21.15.63:0/2108576212 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd42c01b030 con 0x7fd434109860 2024-05-27T08:18:35.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.803+0000 7fd42aa00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4180b2af0 0x7fd4180b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.803+0000 7fd42aa00700 1 -- 172.21.15.63:0/2108576212 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7fd42c0e3340 con 0x7fd434109860 2024-05-27T08:18:35.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.804+0000 7fd431e00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4180b2af0 0x7fd4180b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.817+0000 7fd431e00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4180b2af0 0x7fd4180b5000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd424011490 tx=0x7fd424009260 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:35.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.821+0000 7fd44763fb80 1 Processor -- start 2024-05-27T08:18:35.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.822+0000 7fd44763fb80 1 -- start start 2024-05-27T08:18:35.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.824+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09b8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.825+0000 7fd431e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.825+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f09b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.825+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.825+0000 7fd44763fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5627f08d7e40 con 0x5627f09b25c0 2024-05-27T08:18:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.825+0000 7fd44763fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627f08c79e0 con 0x5627f082ae40 2024-05-27T08:18:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.825+0000 7fd44763fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5627f08cba20 con 0x5627f09b8540 2024-05-27T08:18:35.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.826+0000 7fd431e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09b8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:37014/0 (socket says 172.21.15.63:37014) 2024-05-27T08:18:35.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.826+0000 7fd431e00700 1 -- 172.21.15.63:0/1287370354 learned_addr learned my addr 172.21.15.63:0/1287370354 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:35.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.826+0000 7fd42be00700 1 --2- 172.21.15.63:0/1287370354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f09b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.827+0000 7fd431e00700 1 -- 172.21.15.63:0/1287370354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 msgr2=0x5627f09b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:35.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.827+0000 7fd431e00700 1 --2- 172.21.15.63:0/1287370354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f09b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.827+0000 7fd431e00700 1 -- 172.21.15.63:0/1287370354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 msgr2=0x5627f09b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:35.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.827+0000 7fd431e00700 1 --2- 172.21.15.63:0/1287370354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.827+0000 7fd431e00700 1 -- 172.21.15.63:0/1287370354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f0890420 con 0x5627f082ae40 2024-05-27T08:18:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.828+0000 7fd431e00700 1 --2- 172.21.15.63:0/1287370354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09b8000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd4240043c0 tx=0x7fd424014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5496ac920dddf591 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.829+0000 7fd423400700 1 -- 172.21.15.63:0/1287370354 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424003d10 con 0x5627f082ae40 2024-05-27T08:18:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.829+0000 7fd423400700 1 -- 172.21.15.63:0/1287370354 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd424011e50 con 0x5627f082ae40 2024-05-27T08:18:35.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.829+0000 7fd423400700 1 -- 172.21.15.63:0/1287370354 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42401e630 con 0x5627f082ae40 2024-05-27T08:18:35.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.831+0000 7fd44763fb80 1 -- 172.21.15.63:0/1287370354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 msgr2=0x5627f09b8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:35.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.831+0000 7fd44763fb80 1 --2- 172.21.15.63:0/1287370354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09b8000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd4240043c0 tx=0x7fd424014000 comp rx=0 tx=0).stop 2024-05-27T08:18:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.832+0000 7fd44763fb80 1 -- 172.21.15.63:0/1287370354 shutdown_connections 2024-05-27T08:18:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.832+0000 7fd44763fb80 1 --2- 172.21.15.63:0/1287370354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.832+0000 7fd44763fb80 1 --2- 172.21.15.63:0/1287370354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09b8000 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.832+0000 7fd44763fb80 1 --2- 172.21.15.63:0/1287370354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f09b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.832+0000 7fd44763fb80 1 -- 172.21.15.63:0/1287370354 >> 172.21.15.63:0/1287370354 conn(0x5627f088afc0 msgr2=0x5627f09a48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.833+0000 7fd44763fb80 1 -- 172.21.15.63:0/1287370354 shutdown_connections 2024-05-27T08:18:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.833+0000 7fd44763fb80 1 -- 172.21.15.63:0/1287370354 wait complete. 2024-05-27T08:18:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.833+0000 7fd44763fb80 1 -- start start 2024-05-27T08:18:35.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.834+0000 7fd44763fb80 1 Processor -- start 2024-05-27T08:18:35.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.836+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09bbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.837+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09bc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.837+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f09bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.837+0000 7fd44763fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5627f07cdfa0 con 0x5627f09b25c0 2024-05-27T08:18:35.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.837+0000 7fd44763fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627f0868760 con 0x5627f082ae40 2024-05-27T08:18:35.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.837+0000 7fd44763fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5627f09bf010 con 0x5627f09b8540 2024-05-27T08:18:35.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.837+0000 7fd431e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09bbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.838+0000 7fd431e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09bbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:37022/0 (socket says 172.21.15.63:37022) 2024-05-27T08:18:35.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.838+0000 7fd431e00700 1 -- 172.21.15.63:0/2786436639 learned_addr learned my addr 172.21.15.63:0/2786436639 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:35.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.838+0000 7fd44763fb80 10 client.0 _ll_register_callbacks cb 0x5627f0871c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:35.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.837+0000 7fd42be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09bc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.838+0000 7fd432800700 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f09bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.839+0000 7fd431e00700 1 -- 172.21.15.63:0/2786436639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 msgr2=0x5627f09bc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:35.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.839+0000 7fd431e00700 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f09bc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.839+0000 7fd431e00700 1 -- 172.21.15.63:0/2786436639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 msgr2=0x5627f09bc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:35.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.839+0000 7fd431e00700 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09bc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.839+0000 7fd431e00700 1 -- 172.21.15.63:0/2786436639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f08178d0 con 0x5627f082ae40 2024-05-27T08:18:35.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.839+0000 7fd42be00700 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09bc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:18:35.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.841+0000 7fd417e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:18:35.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.841+0000 7fd431e00700 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09bbdc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd424026e60 tx=0x7fd424026630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2aa58fdd0cb7e79 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:35.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.843+0000 7fd417e00700 1 -- 172.21.15.63:0/2786436639 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424005230 con 0x5627f082ae40 2024-05-27T08:18:35.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.843+0000 7fd44763fb80 1 -- 172.21.15.63:0/2786436639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd424011e50 con 0x5627f082ae40 2024-05-27T08:18:35.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.843+0000 7fd417e00700 1 -- 172.21.15.63:0/2786436639 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7fd4240053d0 con 0x5627f082ae40 2024-05-27T08:18:35.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.844+0000 7fd417e00700 1 -- 172.21.15.63:0/2786436639 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd424026a90 con 0x5627f082ae40 2024-05-27T08:18:35.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.844+0000 7fd44763fb80 10 client.42521 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:35.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.844+0000 7fd44763fb80 1 -- 172.21.15.63:0/2786436639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5627f0898060 con 0x5627f082ae40 2024-05-27T08:18:35.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.845+0000 7fd417e00700 1 -- 172.21.15.63:0/2786436639 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd424011e50 con 0x5627f082ae40 2024-05-27T08:18:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.845+0000 7fd417e00700 1 -- 172.21.15.63:0/2786436639 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd42402f6d0 con 0x5627f082ae40 2024-05-27T08:18:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.845+0000 7fd44763fb80 10 client.42521 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.845+0000 7fd44763fb80 10 client.42521 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:18:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.845+0000 7fd44763fb80 20 client.42521 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.846+0000 7fd44763fb80 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5627f09cbea0 0x5627f09ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.846+0000 7fd44763fb80 4 client.42521 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:18:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.846+0000 7fd44763fb80 1 -- 172.21.15.63:0/2786436639 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627f08d9d90 con 0x5627f09cbea0 2024-05-27T08:18:35.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.846+0000 7fd42be00700 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5627f09cbea0 0x5627f09ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.849+0000 7fd417e00700 10 client.42521 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:18:35.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.849+0000 7fd42be00700 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5627f09cbea0 0x5627f09ce2c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f56f2000cfaa4a2 server_cookie=a83e196668f584e5 in_seq=0 out_seq=0 2024-05-27T08:18:35.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.852+0000 7fd417e00700 1 -- 172.21.15.63:0/2786436639 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5627f08d9d90 con 0x5627f09cbea0 2024-05-27T08:18:35.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.852+0000 7fd417e00700 10 client.42521 handle_command_reply: tid=0 2024-05-27T08:18:35.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.852+0000 7fd44763fb80 1 client.42521 shutdown 2024-05-27T08:18:35.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.852+0000 7fd44763fb80 1 -- 172.21.15.63:0/2786436639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 msgr2=0x5627f09bbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:35.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.852+0000 7fd44763fb80 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09bbdc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd424026e60 tx=0x7fd424026630 comp rx=0 tx=0).stop 2024-05-27T08:18:35.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.853+0000 7fd44763fb80 1 -- 172.21.15.63:0/2786436639 shutdown_connections 2024-05-27T08:18:35.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.853+0000 7fd44763fb80 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f09bc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.853+0000 7fd44763fb80 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f09bbdc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.853+0000 7fd44763fb80 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5627f09cbea0 0x5627f09ce2c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.853+0000 7fd44763fb80 1 --2- 172.21.15.63:0/2786436639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09bc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.853+0000 7fd44763fb80 1 -- 172.21.15.63:0/2786436639 >> 172.21.15.63:0/2786436639 conn(0x5627f088afc0 msgr2=0x5627f09a4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:35.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.853+0000 7fd417e00700 0 client.42521 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:18:35.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.854+0000 7fd44763fb80 1 -- 172.21.15.63:0/2786436639 shutdown_connections 2024-05-27T08:18:35.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.854+0000 7fd44763fb80 1 -- 172.21.15.63:0/2786436639 wait complete. 2024-05-27T08:18:35.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.854+0000 7fd44763fb80 20 client.42521 trim_cache size 0 max 16384 2024-05-27T08:18:35.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.887+0000 7fd44763fb80 1 Processor -- start 2024-05-27T08:18:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.887+0000 7fd44763fb80 1 -- start start 2024-05-27T08:18:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.887+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f082ae40 0x5627f09b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.887+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.887+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f09b8540 0x5627f09c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.887+0000 7fd44763fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd410011810 con 0x5627f09b25c0 2024-05-27T08:18:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.887+0000 7fd44763fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd41000f0a0 con 0x5627f09b8540 2024-05-27T08:18:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.887+0000 7fd44763fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd41000cf10 con 0x5627f082ae40 2024-05-27T08:18:35.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.888+0000 7fd432800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f09b8540 0x5627f09c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.889+0000 7fd432800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f09b8540 0x5627f09c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:37024/0 (socket says 172.21.15.63:37024) 2024-05-27T08:18:35.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.889+0000 7fd432800700 1 -- 172.21.15.63:0/259468690 learned_addr learned my addr 172.21.15.63:0/259468690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:35.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.889+0000 7fd42be00700 1 --2- 172.21.15.63:0/259468690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.890+0000 7fd432800700 1 -- 172.21.15.63:0/259468690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f082ae40 msgr2=0x5627f09b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:35.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.890+0000 7fd432800700 1 --2- 172.21.15.63:0/259468690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f082ae40 0x5627f09b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.890+0000 7fd432800700 1 -- 172.21.15.63:0/259468690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 msgr2=0x5627f09bad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:35.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.890+0000 7fd432800700 1 --2- 172.21.15.63:0/259468690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09bad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.890+0000 7fd432800700 1 -- 172.21.15.63:0/259468690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f09bf060 con 0x5627f09b8540 2024-05-27T08:18:35.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.890+0000 7fd42be00700 1 --2- 172.21.15.63:0/259468690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:18:35.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.891+0000 7fd432800700 1 --2- 172.21.15.63:0/259468690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f09b8540 0x5627f09c8310 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd42c007fd0 tx=0x7fd42c0a9d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76bbab2d5ad985b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:35.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.893+0000 7fd422a00700 1 -- 172.21.15.63:0/259468690 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0c4070 con 0x5627f09b8540 2024-05-27T08:18:35.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.893+0000 7fd422a00700 1 -- 172.21.15.63:0/259468690 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd42c0ae5a0 con 0x5627f09b8540 2024-05-27T08:18:35.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.893+0000 7fd422a00700 1 -- 172.21.15.63:0/259468690 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0c4070 con 0x5627f09b8540 2024-05-27T08:18:35.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.894+0000 7fd44763fb80 1 -- 172.21.15.63:0/259468690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f09b8540 msgr2=0x5627f09c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:35.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.895+0000 7fd44763fb80 1 --2- 172.21.15.63:0/259468690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f09b8540 0x5627f09c8310 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd42c007fd0 tx=0x7fd42c0a9d00 comp rx=0 tx=0).stop 2024-05-27T08:18:35.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.895+0000 7fd44763fb80 1 -- 172.21.15.63:0/259468690 shutdown_connections 2024-05-27T08:18:35.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.895+0000 7fd44763fb80 1 --2- 172.21.15.63:0/259468690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f09b8540 0x5627f09c8310 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.895+0000 7fd44763fb80 1 --2- 172.21.15.63:0/259468690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f082ae40 0x5627f09b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.895+0000 7fd44763fb80 1 --2- 172.21.15.63:0/259468690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f09bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.895+0000 7fd44763fb80 1 -- 172.21.15.63:0/259468690 >> 172.21.15.63:0/259468690 conn(0x5627f088afc0 msgr2=0x5627f09b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:35.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.895+0000 7fd44763fb80 1 -- 172.21.15.63:0/259468690 shutdown_connections 2024-05-27T08:18:35.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.896+0000 7fd44763fb80 1 -- 172.21.15.63:0/259468690 wait complete. 2024-05-27T08:18:35.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.897+0000 7fd44763fb80 1 -- start start 2024-05-27T08:18:35.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.897+0000 7fd44763fb80 1 Processor -- start 2024-05-27T08:18:35.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.898+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f0a2e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.898+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f0a2ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.898+0000 7fd44763fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f0a2f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.898+0000 7fd44763fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd41000d560 con 0x5627f09b25c0 2024-05-27T08:18:35.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.898+0000 7fd44763fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd41000e9d0 con 0x5627f082ae40 2024-05-27T08:18:35.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.898+0000 7fd44763fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd41000dc30 con 0x5627f09b8540 2024-05-27T08:18:35.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.898+0000 7fd44763fb80 10 client.0 _ll_register_callbacks cb 0x5627f0871c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:35.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.900+0000 7fd432800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f0a2f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.900+0000 7fd432800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f0a2f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45426/0 (socket says 172.21.15.63:45426) 2024-05-27T08:18:35.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.900+0000 7fd432800700 1 -- 172.21.15.63:0/1231240987 learned_addr learned my addr 172.21.15.63:0/1231240987 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:35.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.900+0000 7fd42be00700 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f0a2ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.901+0000 7fd42be00700 1 -- 172.21.15.63:0/1231240987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 msgr2=0x5627f0a2f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:35.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.901+0000 7fd42be00700 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f0a2f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.901+0000 7fd42be00700 1 -- 172.21.15.63:0/1231240987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 msgr2=0x5627f0a2e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:35.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.901+0000 7fd42be00700 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f0a2e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.902+0000 7fd42be00700 1 -- 172.21.15.63:0/1231240987 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f09bcbb0 con 0x5627f09b25c0 2024-05-27T08:18:35.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.902+0000 7fd42be00700 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f0a2ec00 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00b580 tx=0x7fd41c009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500087cf65ab1ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:35.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.903+0000 7fd417e00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:18:35.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.903+0000 7fd417e00700 1 -- 172.21.15.63:0/1231240987 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c005640 con 0x5627f09b25c0 2024-05-27T08:18:35.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.904+0000 7fd417e00700 1 -- 172.21.15.63:0/1231240987 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7fd41c0037c0 con 0x5627f09b25c0 2024-05-27T08:18:35.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.904+0000 7fd44763fb80 1 -- 172.21.15.63:0/1231240987 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd42c0ae5a0 con 0x5627f09b25c0 2024-05-27T08:18:35.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.905+0000 7fd417e00700 1 -- 172.21.15.63:0/1231240987 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd41c035050 con 0x5627f09b25c0 2024-05-27T08:18:35.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.905+0000 7fd44763fb80 10 client.23652 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:35.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.905+0000 7fd44763fb80 1 -- 172.21.15.63:0/1231240987 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5627f09d0e30 con 0x5627f09b25c0 2024-05-27T08:18:35.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.906+0000 7fd417e00700 1 -- 172.21.15.63:0/1231240987 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd42c0ae5a0 con 0x5627f09b25c0 2024-05-27T08:18:35.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.907+0000 7fd417e00700 1 -- 172.21.15.63:0/1231240987 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd41c00f460 con 0x5627f09b25c0 2024-05-27T08:18:35.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.907+0000 7fd44763fb80 10 client.23652 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:35.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.907+0000 7fd44763fb80 10 client.23652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:18:35.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.907+0000 7fd44763fb80 20 client.23652 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:35.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.907+0000 7fd44763fb80 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5627f09d1570 0x5627f09ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:35.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.907+0000 7fd44763fb80 4 client.23652 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:18:35.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.907+0000 7fd44763fb80 1 -- 172.21.15.63:0/1231240987 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627f08d3570 con 0x5627f09d1570 2024-05-27T08:18:35.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.908+0000 7fd431e00700 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5627f09d1570 0x5627f09ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:35.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.911+0000 7fd431e00700 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5627f09d1570 0x5627f09ae890 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1047f6af8841db server_cookie=eff8984913cd2863 in_seq=0 out_seq=0 2024-05-27T08:18:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.911+0000 7fd417e00700 10 client.23652 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:18:35.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.911+0000 7fd417e00700 1 -- 172.21.15.63:0/1231240987 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627f08d3570 con 0x5627f09d1570 2024-05-27T08:18:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.911+0000 7fd417e00700 10 client.23652 handle_command_reply: tid=0 2024-05-27T08:18:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.912+0000 7fd44763fb80 1 client.23652 shutdown 2024-05-27T08:18:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.912+0000 7fd44763fb80 1 -- 172.21.15.63:0/1231240987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 msgr2=0x5627f0a2ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.912+0000 7fd44763fb80 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f0a2ec00 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00b580 tx=0x7fd41c009230 comp rx=0 tx=0).stop 2024-05-27T08:18:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.912+0000 7fd44763fb80 1 -- 172.21.15.63:0/1231240987 shutdown_connections 2024-05-27T08:18:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.912+0000 7fd44763fb80 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5627f09b8540 0x5627f0a2f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.912+0000 7fd44763fb80 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627f082ae40 0x5627f0a2e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.912+0000 7fd44763fb80 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x5627f09d1570 0x5627f09ae890 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.912+0000 7fd44763fb80 1 --2- 172.21.15.63:0/1231240987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5627f09b25c0 0x5627f0a2ec00 unknown :-1 s=CLOSED pgs=2978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.912+0000 7fd417e00700 0 client.23652 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:18:35.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.912+0000 7fd44763fb80 1 -- 172.21.15.63:0/1231240987 >> 172.21.15.63:0/1231240987 conn(0x5627f088afc0 msgr2=0x5627f09bbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:35.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.913+0000 7fd44763fb80 1 -- 172.21.15.63:0/1231240987 shutdown_connections 2024-05-27T08:18:35.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.913+0000 7fd44763fb80 1 -- 172.21.15.63:0/1231240987 wait complete. 2024-05-27T08:18:35.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.913+0000 7fd44763fb80 20 client.23652 trim_cache size 0 max 16384 2024-05-27T08:18:35.916 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:18:35.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.914+0000 7fd438a00700 1 -- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4180b2af0 msgr2=0x7fd4180b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:35.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.914+0000 7fd438a00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4180b2af0 0x7fd4180b5000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd424011490 tx=0x7fd424009260 comp rx=0 tx=0).stop 2024-05-27T08:18:35.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.914+0000 7fd438a00700 1 -- 172.21.15.63:0/2108576212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd434109860 msgr2=0x7fd434076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:35.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.914+0000 7fd438a00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd434109860 0x7fd434076530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00ef30 tx=0x7fd42c00c3a0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.915+0000 7fd438a00700 1 -- 172.21.15.63:0/2108576212 shutdown_connections 2024-05-27T08:18:35.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.915+0000 7fd438a00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd4180b2af0 0x7fd4180b5000 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.915+0000 7fd438a00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd434109860 0x7fd434076530 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.915+0000 7fd438a00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4341045e0 0x7fd434075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.915+0000 7fd438a00700 1 --2- 172.21.15.63:0/2108576212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd434108f10 0x7fd434075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:35.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.915+0000 7fd438a00700 1 -- 172.21.15.63:0/2108576212 >> 172.21.15.63:0/2108576212 conn(0x7fd434100030 msgr2=0x7fd434102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:35.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.915+0000 7fd438a00700 1 -- 172.21.15.63:0/2108576212 shutdown_connections 2024-05-27T08:18:35.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:35.915+0000 7fd438a00700 1 -- 172.21.15.63:0/2108576212 wait complete. 2024-05-27T08:18:37.029 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:18:37.170 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:36 smithi063 ceph-mon[31613]: pgmap v1431: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.49k op/s 2024-05-27T08:18:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:37 smithi086 ceph-mon[34687]: pgmap v1431: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.49k op/s 2024-05-27T08:18:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:37 smithi149 ceph-mon[35408]: pgmap v1431: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.49k op/s 2024-05-27T08:18:38.055 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:18:39.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:39 smithi086 ceph-mon[34687]: pgmap v1432: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 11 MiB/s wr, 2.74k op/s 2024-05-27T08:18:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:39 smithi149 ceph-mon[35408]: pgmap v1432: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 11 MiB/s wr, 2.74k op/s 2024-05-27T08:18:39.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:39 smithi063 ceph-mon[31613]: pgmap v1432: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 11 MiB/s wr, 2.74k op/s 2024-05-27T08:18:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.234+0000 7fb225400640 1 -- 172.21.15.63:0/1973469274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb220110340 msgr2=0x7fb220112760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.234+0000 7fb225400640 1 --2- 172.21.15.63:0/1973469274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb220110340 0x7fb220112760 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7fb218009ca0 tx=0x7fb21802a8c0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.235+0000 7fb225400640 1 -- 172.21.15.63:0/1973469274 shutdown_connections 2024-05-27T08:18:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.235+0000 7fb225400640 1 --2- 172.21.15.63:0/1973469274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb220110340 0x7fb220112760 unknown :-1 s=CLOSED pgs=2979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.235+0000 7fb225400640 1 --2- 172.21.15.63:0/1973469274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb22010d9e0 0x7fb22010fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.235+0000 7fb225400640 1 --2- 172.21.15.63:0/1973469274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2200764e0 0x7fb2200768f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.235+0000 7fb225400640 1 -- 172.21.15.63:0/1973469274 >> 172.21.15.63:0/1973469274 conn(0x7fb2200ff350 msgr2=0x7fb220101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.238+0000 7fb225400640 1 -- 172.21.15.63:0/1973469274 shutdown_connections 2024-05-27T08:18:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.238+0000 7fb225400640 1 -- 172.21.15.63:0/1973469274 wait complete. 2024-05-27T08:18:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.240+0000 7fb225400640 1 Processor -- start 2024-05-27T08:18:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.242+0000 7fb225400640 1 -- start start 2024-05-27T08:18:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.243+0000 7fb225400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2200764e0 0x7fb22010c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.243+0000 7fb225400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb22010d9e0 0x7fb22010c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.243+0000 7fb225400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220110340 0x7fb2201a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.243+0000 7fb225400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201045d0 con 0x7fb22010d9e0 2024-05-27T08:18:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.243+0000 7fb225400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb220104450 con 0x7fb220110340 2024-05-27T08:18:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.243+0000 7fb225400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb220104750 con 0x7fb2200764e0 2024-05-27T08:18:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.244+0000 7fb224a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220110340 0x7fb2201a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.244+0000 7fb224a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220110340 0x7fb2201a5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:36380/0 (socket says 172.21.15.63:36380) 2024-05-27T08:18:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.244+0000 7fb224a00640 1 -- 172.21.15.63:0/191418915 learned_addr learned my addr 172.21.15.63:0/191418915 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.244+0000 7fb21f400640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb22010d9e0 0x7fb22010c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.247+0000 7fb21fe00640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2200764e0 0x7fb22010c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.247+0000 7fb21f400640 1 -- 172.21.15.63:0/191418915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2200764e0 msgr2=0x7fb22010c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.247+0000 7fb21f400640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2200764e0 0x7fb22010c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.247+0000 7fb21f400640 1 -- 172.21.15.63:0/191418915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220110340 msgr2=0x7fb2201a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.247+0000 7fb21f400640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220110340 0x7fb2201a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.247+0000 7fb21f400640 1 -- 172.21.15.63:0/191418915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2201a57f0 con 0x7fb22010d9e0 2024-05-27T08:18:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.248+0000 7fb224a00640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220110340 0x7fb2201a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:18:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.249+0000 7fb21f400640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb22010d9e0 0x7fb22010c6e0 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c970 tx=0x7fb20c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.253+0000 7fb21cc00640 1 -- 172.21.15.63:0/191418915 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c007bb0 con 0x7fb22010d9e0 2024-05-27T08:18:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.253+0000 7fb21cc00640 1 -- 172.21.15.63:0/191418915 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb20c007d50 con 0x7fb22010d9e0 2024-05-27T08:18:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.253+0000 7fb21cc00640 1 -- 172.21.15.63:0/191418915 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c004d20 con 0x7fb22010d9e0 2024-05-27T08:18:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.255+0000 7fb225400640 1 -- 172.21.15.63:0/191418915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2201a5ae0 con 0x7fb22010d9e0 2024-05-27T08:18:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.256+0000 7fb225400640 1 -- 172.21.15.63:0/191418915 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2201a5fb0 con 0x7fb22010d9e0 2024-05-27T08:18:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.271+0000 7fb21cc00640 1 -- 172.21.15.63:0/191418915 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb20c004ec0 con 0x7fb22010d9e0 2024-05-27T08:18:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.274+0000 7fb22d986740 1 Processor -- start 2024-05-27T08:18:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.274+0000 7fb22d986740 1 -- start start 2024-05-27T08:18:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.275+0000 7fb21cc00640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2040b2b60 0x7fb2040b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.275+0000 7fb21cc00640 1 -- 172.21.15.63:0/191418915 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7fb20c0e28d0 con 0x7fb22010d9e0 2024-05-27T08:18:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.277+0000 7fb22d986740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e73b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.277+0000 7fb224a00640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2040b2b60 0x7fb2040b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.277+0000 7fb22d986740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.277+0000 7fb22d986740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.277+0000 7fb22d986740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5646e7326b90 con 0x5646e6fea2f0 2024-05-27T08:18:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.277+0000 7fb22d986740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646e7316730 con 0x5646e6ff24d0 2024-05-27T08:18:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.277+0000 7fb22d986740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5646e731a770 con 0x5646e70131b0 2024-05-27T08:18:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.278+0000 7fb21fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e73b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.279+0000 7fb21fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e73b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:50632/0 (socket says 172.21.15.63:50632) 2024-05-27T08:18:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.279+0000 7fb224a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.280+0000 7fb224a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73b64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:36394/0 (socket says 172.21.15.63:36394) 2024-05-27T08:18:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.280+0000 7fb224a00640 1 -- 172.21.15.63:0/3791079794 learned_addr learned my addr 172.21.15.63:0/3791079794 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.282+0000 7fb21f400640 1 --2- 172.21.15.63:0/3791079794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.283+0000 7fb21fe00640 1 -- 172.21.15.63:0/3791079794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 msgr2=0x5646e73b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.283+0000 7fb21fe00640 1 --2- 172.21.15.63:0/3791079794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.283+0000 7fb21fe00640 1 -- 172.21.15.63:0/3791079794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 msgr2=0x5646e73b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.283+0000 7fb21fe00640 1 --2- 172.21.15.63:0/3791079794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.283+0000 7fb21fe00640 1 -- 172.21.15.63:0/3791079794 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646e72af8b0 con 0x5646e70131b0 2024-05-27T08:18:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.284+0000 7fb21f400640 1 --2- 172.21.15.63:0/3791079794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:18:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.284+0000 7fb224a00640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2040b2b60 0x7fb2040b5070 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb218031c10 tx=0x7fb218031c40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.285+0000 7fb21fe00640 1 --2- 172.21.15.63:0/3791079794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e73b5a60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb210009ab0 tx=0x7fb210009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207baf942e9a2166 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.285+0000 7fb215200640 1 -- 172.21.15.63:0/3791079794 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210016070 con 0x5646e70131b0 2024-05-27T08:18:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.285+0000 7fb215200640 1 -- 172.21.15.63:0/3791079794 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb21000ecb0 con 0x5646e70131b0 2024-05-27T08:18:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.285+0000 7fb215200640 1 -- 172.21.15.63:0/3791079794 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210010630 con 0x5646e70131b0 2024-05-27T08:18:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.286+0000 7fb22d986740 1 -- 172.21.15.63:0/3791079794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 msgr2=0x5646e73b5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.287+0000 7fb22d986740 1 --2- 172.21.15.63:0/3791079794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e73b5a60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb210009ab0 tx=0x7fb210009f70 comp rx=0 tx=0).stop 2024-05-27T08:18:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.290+0000 7fb22d986740 1 -- 172.21.15.63:0/3791079794 shutdown_connections 2024-05-27T08:18:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.290+0000 7fb22d986740 1 --2- 172.21.15.63:0/3791079794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.291+0000 7fb22d986740 1 --2- 172.21.15.63:0/3791079794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.291+0000 7fb22d986740 1 --2- 172.21.15.63:0/3791079794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e73b5a60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.291+0000 7fb22d986740 1 -- 172.21.15.63:0/3791079794 >> 172.21.15.63:0/3791079794 conn(0x5646e7007300 msgr2=0x5646e73b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.291+0000 7fb22d986740 1 -- 172.21.15.63:0/3791079794 shutdown_connections 2024-05-27T08:18:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.291+0000 7fb22d986740 1 -- 172.21.15.63:0/3791079794 wait complete. 2024-05-27T08:18:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.292+0000 7fb22d986740 1 -- start start 2024-05-27T08:18:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.293+0000 7fb22d986740 1 Processor -- start 2024-05-27T08:18:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.295+0000 7fb22d986740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.295+0000 7fb22d986740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6ff24d0 0x5646e73c0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.295+0000 7fb22d986740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e70131b0 0x5646e73cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.296+0000 7fb21f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6ff24d0 0x5646e73c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.296+0000 7fb21f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6ff24d0 0x5646e73c0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:50644/0 (socket says 172.21.15.63:50644) 2024-05-27T08:18:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.296+0000 7fb21fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.296+0000 7fb22d986740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5646e6ffc790 con 0x5646e6fea2f0 2024-05-27T08:18:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.296+0000 7fb21f400640 1 -- 172.21.15.63:0/2030734047 learned_addr learned my addr 172.21.15.63:0/2030734047 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.296+0000 7fb22d986740 1 -- 172.21.15.63:0/2030734047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646e72bebc0 con 0x5646e70131b0 2024-05-27T08:18:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.296+0000 7fb22d986740 1 -- 172.21.15.63:0/2030734047 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5646e73c2b90 con 0x5646e6ff24d0 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.297+0000 7fb224a00640 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e70131b0 0x5646e73cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.297+0000 7fb22d986740 10 client.0 _ll_register_callbacks cb 0x5646e729c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.298+0000 7fb224a00640 1 -- 172.21.15.63:0/2030734047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6ff24d0 msgr2=0x5646e73c0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.298+0000 7fb224a00640 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6ff24d0 0x5646e73c0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.298+0000 7fb224a00640 1 -- 172.21.15.63:0/2030734047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 msgr2=0x5646e73bfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.298+0000 7fb224a00640 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73bfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.298+0000 7fb224a00640 1 -- 172.21.15.63:0/2030734047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646e702b210 con 0x5646e70131b0 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.299+0000 7fb224a00640 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e70131b0 0x5646e73cbec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb218009a60 tx=0x7fb21802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a52e780a898ab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.299+0000 7fb209600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.300+0000 7fb209600640 1 -- 172.21.15.63:0/2030734047 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218045040 con 0x5646e70131b0 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.300+0000 7fb209600640 1 -- 172.21.15.63:0/2030734047 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7fb2180307d0 con 0x5646e70131b0 2024-05-27T08:18:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.300+0000 7fb22d986740 1 -- 172.21.15.63:0/2030734047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb21000ecb0 con 0x5646e70131b0 2024-05-27T08:18:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.302+0000 7fb209600640 1 -- 172.21.15.63:0/2030734047 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb2180309d0 con 0x5646e70131b0 2024-05-27T08:18:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.302+0000 7fb22d986740 10 client.42536 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.302+0000 7fb22d986740 1 -- 172.21.15.63:0/2030734047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5646e70f7620 con 0x5646e70131b0 2024-05-27T08:18:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.303+0000 7fb209600640 1 -- 172.21.15.63:0/2030734047 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb21000ecb0 con 0x5646e70131b0 2024-05-27T08:18:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.305+0000 7fb209600640 1 -- 172.21.15.63:0/2030734047 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb218048070 con 0x5646e70131b0 2024-05-27T08:18:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.305+0000 7fb22d986740 10 client.42536 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.305+0000 7fb22d986740 10 client.42536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:18:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.305+0000 7fb22d986740 20 client.42536 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.305+0000 7fb22d986740 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5646e6fd1d80 0x5646e73d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.305+0000 7fb22d986740 4 client.42536 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:18:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.305+0000 7fb22d986740 1 -- 172.21.15.63:0/2030734047 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646e7328ae0 con 0x5646e6fd1d80 2024-05-27T08:18:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.307+0000 7fb21fe00640 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5646e6fd1d80 0x5646e73d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.310+0000 7fb209600640 10 client.42536 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.311+0000 7fb21fe00640 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5646e6fd1d80 0x5646e73d21d0 crc :-1 s=READY pgs=772 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83aac98e74b8d1a server_cookie=a2ce3794960fc29d in_seq=0 out_seq=0 2024-05-27T08:18:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.312+0000 7fb209600640 1 -- 172.21.15.63:0/2030734047 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5646e7328ae0 con 0x5646e6fd1d80 2024-05-27T08:18:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.312+0000 7fb209600640 10 client.42536 handle_command_reply: tid=0 2024-05-27T08:18:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.313+0000 7fb22d986740 1 client.42536 shutdown 2024-05-27T08:18:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.313+0000 7fb22d986740 1 -- 172.21.15.63:0/2030734047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e70131b0 msgr2=0x5646e73cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.313+0000 7fb22d986740 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e70131b0 0x5646e73cbec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb218009a60 tx=0x7fb21802a8c0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.314+0000 7fb22d986740 1 -- 172.21.15.63:0/2030734047 shutdown_connections 2024-05-27T08:18:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.314+0000 7fb22d986740 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5646e6fd1d80 0x5646e73d21d0 crc :-1 s=READY pgs=772 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.314+0000 7fb22d986740 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e70131b0 0x5646e73cbec0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.314+0000 7fb22d986740 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6ff24d0 0x5646e73c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.314+0000 7fb209600640 0 client.42536 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.314+0000 7fb22d986740 1 --2- 172.21.15.63:0/2030734047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73bfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.314+0000 7fb22d986740 1 -- 172.21.15.63:0/2030734047 >> 172.21.15.63:0/2030734047 conn(0x5646e7007300 msgr2=0x5646e73b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.314+0000 7fb22d986740 1 -- 172.21.15.63:0/2030734047 shutdown_connections 2024-05-27T08:18:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.314+0000 7fb22d986740 1 -- 172.21.15.63:0/2030734047 wait complete. 2024-05-27T08:18:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.317+0000 7fb22d986740 20 client.42536 trim_cache size 0 max 16384 2024-05-27T08:18:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.362+0000 7fb22d986740 1 Processor -- start 2024-05-27T08:18:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.362+0000 7fb22d986740 1 -- start start 2024-05-27T08:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.363+0000 7fb22d986740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6fea2f0 0x5646e73c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.363+0000 7fb22d986740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.363+0000 7fb22d986740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e70131b0 0x5646e73c0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.363+0000 7fb22d986740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201b14c0 con 0x5646e70131b0 2024-05-27T08:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.363+0000 7fb22d986740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201aed90 con 0x5646e6ff24d0 2024-05-27T08:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.363+0000 7fb22d986740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb220050860 con 0x5646e6fea2f0 2024-05-27T08:18:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.364+0000 7fb21fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6fea2f0 0x5646e73c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.364+0000 7fb21fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6fea2f0 0x5646e73c2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:50656/0 (socket says 172.21.15.63:50656) 2024-05-27T08:18:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.364+0000 7fb21fe00640 1 -- 172.21.15.63:0/4171316884 learned_addr learned my addr 172.21.15.63:0/4171316884 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.364+0000 7fb21f400640 1 --2- 172.21.15.63:0/4171316884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.365+0000 7fb21fe00640 1 -- 172.21.15.63:0/4171316884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 msgr2=0x5646e73bfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.365+0000 7fb21fe00640 1 --2- 172.21.15.63:0/4171316884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73bfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.365+0000 7fb21fe00640 1 -- 172.21.15.63:0/4171316884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e70131b0 msgr2=0x5646e73c0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.365+0000 7fb21fe00640 1 --2- 172.21.15.63:0/4171316884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e70131b0 0x5646e73c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.365+0000 7fb21fe00640 1 -- 172.21.15.63:0/4171316884 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646e73be180 con 0x5646e6fea2f0 2024-05-27T08:18:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.365+0000 7fb21fe00640 1 --2- 172.21.15.63:0/4171316884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6fea2f0 0x5646e73c2360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb210014040 tx=0x7fb210005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2460f62ea2019d8e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.366+0000 7fb20be00640 1 -- 172.21.15.63:0/4171316884 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21002e070 con 0x5646e6fea2f0 2024-05-27T08:18:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.366+0000 7fb20be00640 1 -- 172.21.15.63:0/4171316884 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb210011900 con 0x5646e6fea2f0 2024-05-27T08:18:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.366+0000 7fb20be00640 1 -- 172.21.15.63:0/4171316884 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210016040 con 0x5646e6fea2f0 2024-05-27T08:18:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.368+0000 7fb22d986740 1 -- 172.21.15.63:0/4171316884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6fea2f0 msgr2=0x5646e73c2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.368+0000 7fb22d986740 1 --2- 172.21.15.63:0/4171316884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6fea2f0 0x5646e73c2360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb210014040 tx=0x7fb210005850 comp rx=0 tx=0).stop 2024-05-27T08:18:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.369+0000 7fb22d986740 1 -- 172.21.15.63:0/4171316884 shutdown_connections 2024-05-27T08:18:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.369+0000 7fb22d986740 1 --2- 172.21.15.63:0/4171316884 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e70131b0 0x5646e73c0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.369+0000 7fb22d986740 1 --2- 172.21.15.63:0/4171316884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.369+0000 7fb22d986740 1 --2- 172.21.15.63:0/4171316884 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e6fea2f0 0x5646e73c2360 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.370+0000 7fb22d986740 1 -- 172.21.15.63:0/4171316884 >> 172.21.15.63:0/4171316884 conn(0x5646e7007300 msgr2=0x5646e73bf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.372+0000 7fb22d986740 1 -- 172.21.15.63:0/4171316884 shutdown_connections 2024-05-27T08:18:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.372+0000 7fb22d986740 1 -- 172.21.15.63:0/4171316884 wait complete. 2024-05-27T08:18:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.373+0000 7fb22d986740 1 -- start start 2024-05-27T08:18:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.373+0000 7fb22d986740 1 Processor -- start 2024-05-27T08:18:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.381+0000 7fb22d986740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.382+0000 7fb21fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.382+0000 7fb21fe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46474/0 (socket says 172.21.15.63:46474) 2024-05-27T08:18:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.384+0000 7fb21fe00640 1 -- 172.21.15.63:0/3818917878 learned_addr learned my addr 172.21.15.63:0/3818917878 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.384+0000 7fb22d986740 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73c3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.385+0000 7fb22d986740 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e7438890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.385+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb220074d20 con 0x5646e6fea2f0 2024-05-27T08:18:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.385+0000 7fb21f400640 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.385+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201ae6c0 con 0x5646e6ff24d0 2024-05-27T08:18:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.385+0000 7fb224a00640 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e7438890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.386+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201ad920 con 0x5646e70131b0 2024-05-27T08:18:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.386+0000 7fb22d986740 10 client.0 _ll_register_callbacks cb 0x7fb218009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.387+0000 7fb224a00640 1 -- 172.21.15.63:0/3818917878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 msgr2=0x5646e73c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.387+0000 7fb224a00640 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73c3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.387+0000 7fb224a00640 1 -- 172.21.15.63:0/3818917878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 msgr2=0x5646e73c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.387+0000 7fb224a00640 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.387+0000 7fb224a00640 1 -- 172.21.15.63:0/3818917878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646e73c3d50 con 0x5646e70131b0 2024-05-27T08:18:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.387+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb210011900 con 0x5646e70131b0 2024-05-27T08:18:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.388+0000 7fb21fe00640 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:18:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.388+0000 7fb21f400640 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:18:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.390+0000 7fb224a00640 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e7438890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb218031920 tx=0x7fb218030da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83dee59aadeec0f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.390+0000 7fb209600640 10 client.32365 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:18:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.393+0000 7fb209600640 1 -- 172.21.15.63:0/3818917878 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218045040 con 0x5646e70131b0 2024-05-27T08:18:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.393+0000 7fb209600640 1 -- 172.21.15.63:0/3818917878 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 9635+0+0 (secure 0 0 0) 0x7fb218066690 con 0x5646e70131b0 2024-05-27T08:18:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.393+0000 7fb209600640 1 -- 172.21.15.63:0/3818917878 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb218066890 con 0x5646e70131b0 2024-05-27T08:18:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.394+0000 7fb22d986740 10 client.32365 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.394+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({fsmap=35}) -- 0x5646e7439df0 con 0x5646e70131b0 2024-05-27T08:18:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.395+0000 7fb209600640 1 -- 172.21.15.63:0/3818917878 <== mon.1 v2:172.21.15.149:3300/0 4 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fb218048040 con 0x5646e70131b0 2024-05-27T08:18:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.395+0000 7fb22d986740 10 client.32365 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.395+0000 7fb22d986740 10 client.32365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:18:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.395+0000 7fb22d986740 20 client.32365 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.395+0000 7fb22d986740 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5646e6fa71c0 0x5646e743c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.395+0000 7fb22d986740 4 client.32365 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:18:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.395+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646e73ba260 con 0x5646e6fa71c0 2024-05-27T08:18:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.402+0000 7fb21fe00640 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5646e6fa71c0 0x5646e743c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.405+0000 7fb209600640 10 client.32365 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:18:41.405+0000 7fb21fe00640 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5646e6fa71c0 0x5646e743c510 crc :-1 s=READY pgs=773 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b21ca792296eb5 server_cookie=772f5ea0b11f800a in_seq=0 out_seq=0 2024-05-27T08:18:41.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:41 smithi086 ceph-mon[34687]: pgmap v1433: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.5 MiB/s wr, 2.33k op/s 2024-05-27T08:18:41.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:41 smithi149 ceph-mon[35408]: pgmap v1433: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.5 MiB/s wr, 2.33k op/s 2024-05-27T08:18:41.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:41 smithi063 ceph-mon[31613]: pgmap v1433: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.5 MiB/s wr, 2.33k op/s 2024-05-27T08:18:42.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:42 smithi086 ceph-mon[34687]: osdmap e171: 12 total, 12 up, 12 in 2024-05-27T08:18:42.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:42 smithi149 ceph-mon[35408]: osdmap e171: 12 total, 12 up, 12 in 2024-05-27T08:18:42.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:42 smithi063 ceph-mon[31613]: osdmap e171: 12 total, 12 up, 12 in 2024-05-27T08:18:43.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:43 smithi086 ceph-mon[34687]: pgmap v1435: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.5 MiB/s wr, 2.33k op/s 2024-05-27T08:18:43.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:43 smithi149 ceph-mon[35408]: pgmap v1435: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.5 MiB/s wr, 2.33k op/s 2024-05-27T08:18:43.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:43 smithi063 ceph-mon[31613]: pgmap v1435: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.5 MiB/s wr, 2.33k op/s 2024-05-27T08:18:45.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:45 smithi086 ceph-mon[34687]: pgmap v1436: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.9 MiB/s wr, 2.42k op/s 2024-05-27T08:18:45.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:45 smithi149 ceph-mon[35408]: pgmap v1436: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.9 MiB/s wr, 2.42k op/s 2024-05-27T08:18:45.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:45 smithi063 ceph-mon[31613]: pgmap v1436: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.9 MiB/s wr, 2.42k op/s 2024-05-27T08:18:47.953 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:47 smithi063 ceph-mon[31613]: osdmap e172: 12 total, 12 up, 12 in 2024-05-27T08:18:47.953 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:47 smithi063 ceph-mon[31613]: pgmap v1438: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.4 MiB/s wr, 2.30k op/s 2024-05-27T08:18:47.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:47 smithi086 ceph-mon[34687]: osdmap e172: 12 total, 12 up, 12 in 2024-05-27T08:18:47.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:47 smithi086 ceph-mon[34687]: pgmap v1438: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.4 MiB/s wr, 2.30k op/s 2024-05-27T08:18:47.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:47 smithi149 ceph-mon[35408]: osdmap e172: 12 total, 12 up, 12 in 2024-05-27T08:18:47.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:47 smithi149 ceph-mon[35408]: pgmap v1438: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.4 MiB/s wr, 2.30k op/s 2024-05-27T08:18:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.803+0000 7fcf15200700 1 -- 172.21.15.63:0/1072796570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf10104400 msgr2=0x7fcf10104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.803+0000 7fcf15200700 1 --2- 172.21.15.63:0/1072796570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf10104400 0x7fcf10104810 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0099c0 tx=0x7fcefc009cd0 comp rx=0 tx=0).stop 2024-05-27T08:18:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.804+0000 7fcf15200700 1 -- 172.21.15.63:0/1072796570 shutdown_connections 2024-05-27T08:18:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.805+0000 7fcf15200700 1 --2- 172.21.15.63:0/1072796570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10109680 0x7fcf10110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.805+0000 7fcf15200700 1 --2- 172.21.15.63:0/1072796570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf10104400 0x7fcf10104810 unknown :-1 s=CLOSED pgs=2981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.805+0000 7fcf15200700 1 --2- 172.21.15.63:0/1072796570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf10108d30 0x7fcf10109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:48.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.805+0000 7fcf15200700 1 -- 172.21.15.63:0/1072796570 >> 172.21.15.63:0/1072796570 conn(0x7fcf10079d00 msgr2=0x7fcf1007a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.806+0000 7fcf15200700 1 -- 172.21.15.63:0/1072796570 shutdown_connections 2024-05-27T08:18:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.806+0000 7fcf15200700 1 -- 172.21.15.63:0/1072796570 wait complete. 2024-05-27T08:18:48.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.807+0000 7fcf15200700 1 Processor -- start 2024-05-27T08:18:48.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.809+0000 7fcf15200700 1 -- start start 2024-05-27T08:18:48.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.811+0000 7fcf15200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf10104400 0x7fcf10075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:48.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.812+0000 7fcf15200700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf10108d30 0x7fcf1006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:48.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.812+0000 7fcf15200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10109680 0x7fcf1006e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:48.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.813+0000 7fcf15200700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101141b0 con 0x7fcf10108d30 2024-05-27T08:18:48.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.813+0000 7fcf15200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10114030 con 0x7fcf10104400 2024-05-27T08:18:48.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.813+0000 7fcf15200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10114330 con 0x7fcf10109680 2024-05-27T08:18:48.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.814+0000 7fcf0e800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10109680 0x7fcf1006e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:48.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.814+0000 7fcf0e800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10109680 0x7fcf1006e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:50684/0 (socket says 172.21.15.63:50684) 2024-05-27T08:18:48.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.814+0000 7fcf0e800700 1 -- 172.21.15.63:0/314713529 learned_addr learned my addr 172.21.15.63:0/314713529 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:48.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.814+0000 7fcf0de00700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf10104400 0x7fcf10075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:48.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.815+0000 7fcf0e800700 1 -- 172.21.15.63:0/314713529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf10104400 msgr2=0x7fcf10075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:48.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.815+0000 7fcf0d400700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf10108d30 0x7fcf1006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.816+0000 7fcf0e800700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf10104400 0x7fcf10075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.816+0000 7fcf0e800700 1 -- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf10108d30 msgr2=0x7fcf1006e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.816+0000 7fcf0e800700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf10108d30 0x7fcf1006e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.816+0000 7fcf0e800700 1 -- 172.21.15.63:0/314713529 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1006f140 con 0x7fcf10109680 2024-05-27T08:18:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.817+0000 7fcf0de00700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf10104400 0x7fcf10075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:18:48.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.817+0000 7fcf0d400700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf10108d30 0x7fcf1006e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:18:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.820+0000 7fcf0e800700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10109680 0x7fcf1006e900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcf08007ee0 tx=0x7fcf0800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:48.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.821+0000 7fcf06a00700 1 -- 172.21.15.63:0/314713529 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800ddd0 con 0x7fcf10109680 2024-05-27T08:18:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.821+0000 7fcf06a00700 1 -- 172.21.15.63:0/314713529 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf08004020 con 0x7fcf10109680 2024-05-27T08:18:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.821+0000 7fcf06a00700 1 -- 172.21.15.63:0/314713529 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08015660 con 0x7fcf10109680 2024-05-27T08:18:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.821+0000 7fcf15200700 1 -- 172.21.15.63:0/314713529 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf10072e20 con 0x7fcf10109680 2024-05-27T08:18:48.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.821+0000 7fcf15200700 1 -- 172.21.15.63:0/314713529 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf10073310 con 0x7fcf10109680 2024-05-27T08:18:48.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.843+0000 7fcf06a00700 1 -- 172.21.15.63:0/314713529 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcf080041d0 con 0x7fcf10109680 2024-05-27T08:18:48.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.848+0000 7fcf06a00700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcef40b2b40 0x7fcef40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:48.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.848+0000 7fcf06a00700 1 -- 172.21.15.63:0/314713529 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 9543+0+0 (secure 0 0 0) 0x7fcf080e23e0 con 0x7fcf10109680 2024-05-27T08:18:48.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.852+0000 7fcf0de00700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcef40b2b40 0x7fcef40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:48.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:48.854+0000 7fcf0de00700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcef40b2b40 0x7fcef40b5050 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcefc000c00 tx=0x7fcefc01a040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:49.116 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:49 smithi063 ceph-mon[31613]: pgmap v1439: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-05-27T08:18:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.111+0000 7fcf23c6bb80 1 Processor -- start 2024-05-27T08:18:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.112+0000 7fcf23c6bb80 1 -- start start 2024-05-27T08:18:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.112+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562099720e40 0x56209989eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.112+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f410 0x5620998a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.112+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998ac020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.112+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5620997cde40 con 0x562099720e40 2024-05-27T08:18:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.112+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5620997bd9e0 con 0x56209989f820 2024-05-27T08:18:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.112+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5620997c1a20 con 0x56209989f410 2024-05-27T08:18:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.113+0000 7fcf0e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:49.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.114+0000 7fcf0e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998ac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:48430/0 (socket says 172.21.15.63:48430) 2024-05-27T08:18:49.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.114+0000 7fcf0e800700 1 -- 172.21.15.63:0/1428379424 learned_addr learned my addr 172.21.15.63:0/1428379424 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:49.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.114+0000 7fcf0e800700 1 -- 172.21.15.63:0/1428379424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f410 msgr2=0x5620998a9ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:18:49.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.114+0000 7fcf0e800700 1 --2- 172.21.15.63:0/1428379424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f410 0x5620998a9ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.114+0000 7fcf0e800700 1 -- 172.21.15.63:0/1428379424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562099720e40 msgr2=0x56209989eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:49.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.114+0000 7fcf0e800700 1 --2- 172.21.15.63:0/1428379424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562099720e40 0x56209989eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.114+0000 7fcf0e800700 1 -- 172.21.15.63:0/1428379424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562099786420 con 0x56209989f820 2024-05-27T08:18:49.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.116+0000 7fcf0e800700 1 --2- 172.21.15.63:0/1428379424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998ac020 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcf080a9250 tx=0x7fcf080a8ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a8850434ca24d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:49.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.116+0000 7fcefb400700 1 -- 172.21.15.63:0/1428379424 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf080b2120 con 0x56209989f820 2024-05-27T08:18:49.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.117+0000 7fcefb400700 1 -- 172.21.15.63:0/1428379424 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf080044f0 con 0x56209989f820 2024-05-27T08:18:49.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.117+0000 7fcefb400700 1 -- 172.21.15.63:0/1428379424 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08004bd0 con 0x56209989f820 2024-05-27T08:18:49.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.118+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1428379424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 msgr2=0x5620998ac020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:49.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.119+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/1428379424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998ac020 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcf080a9250 tx=0x7fcf080a8ec0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.121+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1428379424 shutdown_connections 2024-05-27T08:18:49.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.121+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/1428379424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998ac020 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.121+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/1428379424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562099720e40 0x56209989eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.121+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/1428379424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f410 0x5620998a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.122+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1428379424 >> 172.21.15.63:0/1428379424 conn(0x562099780fc0 msgr2=0x56209989ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:49.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.128+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1428379424 shutdown_connections 2024-05-27T08:18:49.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.128+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1428379424 wait complete. 2024-05-27T08:18:49.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.129+0000 7fcf23c6bb80 1 -- start start 2024-05-27T08:18:49.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.130+0000 7fcf23c6bb80 1 Processor -- start 2024-05-27T08:18:49.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.130+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562099720e40 0x5620998a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.130+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209989f410 0x5620998a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.131+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.131+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5620996c3fa0 con 0x56209989f410 2024-05-27T08:18:49.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.131+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56209975e760 con 0x56209989f820 2024-05-27T08:18:49.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.131+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5620998ac930 con 0x562099720e40 2024-05-27T08:18:49.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.131+0000 7fcf23c6bb80 10 client.0 _ll_register_callbacks cb 0x562099767c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:49.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.136+0000 7fcf0e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:49.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.136+0000 7fcf0e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:48440/0 (socket says 172.21.15.63:48440) 2024-05-27T08:18:49.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.136+0000 7fcf0e800700 1 -- 172.21.15.63:0/1334575224 learned_addr learned my addr 172.21.15.63:0/1334575224 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:49.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.137+0000 7fcf0e800700 1 -- 172.21.15.63:0/1334575224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562099720e40 msgr2=0x5620998a9700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:18:49.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.137+0000 7fcf0e800700 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562099720e40 0x5620998a9700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.137+0000 7fcf0e800700 1 -- 172.21.15.63:0/1334575224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209989f410 msgr2=0x5620998a9c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:18:49.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.137+0000 7fcf0e800700 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209989f410 0x5620998a9c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.137+0000 7fcf0e800700 1 -- 172.21.15.63:0/1334575224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209970d8d0 con 0x56209989f820 2024-05-27T08:18:49.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.140+0000 7fceefe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:18:49.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.140+0000 7fcf0e800700 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998aa180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf080a9200 tx=0x7fcf080a97e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ad9153618bcd9a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:49.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.141+0000 7fceefe00700 1 -- 172.21.15.63:0/1334575224 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf080c8060 con 0x56209989f820 2024-05-27T08:18:49.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.143+0000 7fceefe00700 1 -- 172.21.15.63:0/1334575224 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(172..172 src has 1..172) ==== 9543+0+0 (secure 0 0 0) 0x7fcf08004de0 con 0x56209989f820 2024-05-27T08:18:49.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.144+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1334575224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf080044f0 con 0x56209989f820 2024-05-27T08:18:49.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.145+0000 7fceefe00700 1 -- 172.21.15.63:0/1334575224 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcf080ca300 con 0x56209989f820 2024-05-27T08:18:49.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.146+0000 7fcf23c6bb80 10 client.42551 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:49.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.146+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1334575224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56209978e060 con 0x56209989f820 2024-05-27T08:18:49.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.147+0000 7fceefe00700 1 -- 172.21.15.63:0/1334575224 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf080044f0 con 0x56209989f820 2024-05-27T08:18:49.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.149+0000 7fceefe00700 1 -- 172.21.15.63:0/1334575224 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcf080a4c30 con 0x56209989f820 2024-05-27T08:18:49.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.150+0000 7fcf23c6bb80 10 client.42551 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:49.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.150+0000 7fcf23c6bb80 10 client.42551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:18:49.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.151+0000 7fcf23c6bb80 20 client.42551 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:49.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.151+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5620998c1e80 0x5620998c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.151+0000 7fcf23c6bb80 4 client.42551 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:18:49.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.151+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1334575224 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620997cfd90 con 0x5620998c1e80 2024-05-27T08:18:49.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.152+0000 7fcf0de00700 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5620998c1e80 0x5620998c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:49.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.157+0000 7fceefe00700 10 client.42551 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:49.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.157+0000 7fcf0de00700 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5620998c1e80 0x5620998c42a0 crc :-1 s=READY pgs=774 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9ed9c7fd5c788e8 server_cookie=b62c2b3342e29f71 in_seq=0 out_seq=0 2024-05-27T08:18:49.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.159+0000 7fceefe00700 1 -- 172.21.15.63:0/1334575224 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5620997cfd90 con 0x5620998c1e80 2024-05-27T08:18:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.159+0000 7fceefe00700 10 client.42551 handle_command_reply: tid=0 2024-05-27T08:18:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.160+0000 7fcf23c6bb80 1 client.42551 shutdown 2024-05-27T08:18:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.160+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1334575224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 msgr2=0x5620998aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.160+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998aa180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf080a9200 tx=0x7fcf080a97e0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.160+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1334575224 shutdown_connections 2024-05-27T08:18:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.160+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5620998c1e80 0x5620998c42a0 crc :-1 s=READY pgs=774 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.160+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f820 0x5620998aa180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.160+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562099720e40 0x5620998a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.160+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/1334575224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209989f410 0x5620998a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.160+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1334575224 >> 172.21.15.63:0/1334575224 conn(0x562099780fc0 msgr2=0x562099899ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:49.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.160+0000 7fceefe00700 0 client.42551 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:49.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.162+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1334575224 shutdown_connections 2024-05-27T08:18:49.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.162+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/1334575224 wait complete. 2024-05-27T08:18:49.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.162+0000 7fcf23c6bb80 20 client.42551 trim_cache size 0 max 16384 2024-05-27T08:18:49.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.198+0000 7fcf23c6bb80 1 Processor -- start 2024-05-27T08:18:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.198+0000 7fcf23c6bb80 1 -- start start 2024-05-27T08:18:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.199+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562099720e40 0x56209989ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.199+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f410 0x5620998abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.199+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209989f820 0x5620998a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.199+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101b91b0 con 0x56209989f820 2024-05-27T08:18:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.199+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101b6a40 con 0x562099720e40 2024-05-27T08:18:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.199+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101b4870 con 0x56209989f410 2024-05-27T08:18:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.200+0000 7fcf0de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562099720e40 0x56209989ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.200+0000 7fcf0de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562099720e40 0x56209989ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:48442/0 (socket says 172.21.15.63:48442) 2024-05-27T08:18:49.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.200+0000 7fcf0de00700 1 -- 172.21.15.63:0/467503674 learned_addr learned my addr 172.21.15.63:0/467503674 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:49.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.201+0000 7fcf0d400700 1 --2- 172.21.15.63:0/467503674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f410 0x5620998abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:49.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.201+0000 7fcf0e800700 1 --2- 172.21.15.63:0/467503674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209989f820 0x5620998a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.203+0000 7fcf0de00700 1 -- 172.21.15.63:0/467503674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f410 msgr2=0x5620998abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.203+0000 7fcf0de00700 1 --2- 172.21.15.63:0/467503674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f410 0x5620998abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.204+0000 7fcf0de00700 1 -- 172.21.15.63:0/467503674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209989f820 msgr2=0x5620998a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.204+0000 7fcf0de00700 1 --2- 172.21.15.63:0/467503674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209989f820 0x5620998a37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.204+0000 7fcf0de00700 1 -- 172.21.15.63:0/467503674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209989c350 con 0x562099720e40 2024-05-27T08:18:49.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.204+0000 7fcf0d400700 1 --2- 172.21.15.63:0/467503674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f410 0x5620998abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:18:49.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.206+0000 7fcf0de00700 1 --2- 172.21.15.63:0/467503674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562099720e40 0x56209989ae90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0099c0 tx=0x7fcefc00b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb89e44c65e81d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:49.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.206+0000 7fcefaa00700 1 -- 172.21.15.63:0/467503674 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc003680 con 0x562099720e40 2024-05-27T08:18:49.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.207+0000 7fcefaa00700 1 -- 172.21.15.63:0/467503674 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcefc035070 con 0x562099720e40 2024-05-27T08:18:49.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.207+0000 7fcefaa00700 1 -- 172.21.15.63:0/467503674 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc02fa00 con 0x562099720e40 2024-05-27T08:18:49.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.209+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/467503674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562099720e40 msgr2=0x56209989ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:49.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.209+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/467503674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562099720e40 0x56209989ae90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0099c0 tx=0x7fcefc00b540 comp rx=0 tx=0).stop 2024-05-27T08:18:49.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.209+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/467503674 shutdown_connections 2024-05-27T08:18:49.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.209+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/467503674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209989f820 0x5620998a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.209+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/467503674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562099720e40 0x56209989ae90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.209+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/467503674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f410 0x5620998abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.209+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/467503674 >> 172.21.15.63:0/467503674 conn(0x562099780fc0 msgr2=0x5620998a7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:49.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.214+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/467503674 shutdown_connections 2024-05-27T08:18:49.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.214+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/467503674 wait complete. 2024-05-27T08:18:49.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.215+0000 7fcf23c6bb80 1 -- start start 2024-05-27T08:18:49.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.215+0000 7fcf23c6bb80 1 Processor -- start 2024-05-27T08:18:49.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.216+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562099720e40 0x5620998c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.216+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f410 0x5620998c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.216+0000 7fcf23c6bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f820 0x5620998c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.216+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101b4f00 con 0x562099720e40 2024-05-27T08:18:49.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.216+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101b6370 con 0x56209989f410 2024-05-27T08:18:49.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.216+0000 7fcf23c6bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101b55d0 con 0x56209989f820 2024-05-27T08:18:49.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.216+0000 7fcf23c6bb80 10 client.0 _ll_register_callbacks cb 0x562099767c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:18:49.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.217+0000 7fcf0e800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f820 0x5620998c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:49.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.217+0000 7fcf0d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f410 0x5620998c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:49.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.218+0000 7fcf0d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f410 0x5620998c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:48458/0 (socket says 172.21.15.63:48458) 2024-05-27T08:18:49.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.218+0000 7fcf0d400700 1 -- 172.21.15.63:0/4189837841 learned_addr learned my addr 172.21.15.63:0/4189837841 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:18:49.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.217+0000 7fcf0de00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562099720e40 0x5620998c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:49.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.218+0000 7fcf0d400700 1 -- 172.21.15.63:0/4189837841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f820 msgr2=0x5620998c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:49.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.218+0000 7fcf0d400700 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f820 0x5620998c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.218+0000 7fcf0d400700 1 -- 172.21.15.63:0/4189837841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562099720e40 msgr2=0x5620998c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:18:49.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.218+0000 7fcf0d400700 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562099720e40 0x5620998c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.218+0000 7fcf0d400700 1 -- 172.21.15.63:0/4189837841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209992a6e0 con 0x56209989f410 2024-05-27T08:18:49.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.219+0000 7fcf0d400700 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f410 0x5620998c4130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcf1010c100 tx=0x7fcf00013a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=144c462165efc0a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:18:49.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.219+0000 7fceefe00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:18:49.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.220+0000 7fceefe00700 1 -- 172.21.15.63:0/4189837841 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf00004300 con 0x56209989f410 2024-05-27T08:18:49.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.220+0000 7fceefe00700 1 -- 172.21.15.63:0/4189837841 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(172..172 src has 1..172) ==== 9543+0+0 (secure 0 0 0) 0x7fcf000044a0 con 0x56209989f410 2024-05-27T08:18:49.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.220+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/4189837841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcefc035070 con 0x56209989f410 2024-05-27T08:18:49.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.221+0000 7fceefe00700 1 -- 172.21.15.63:0/4189837841 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcf00003680 con 0x56209989f410 2024-05-27T08:18:49.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.221+0000 7fcf23c6bb80 10 client.42557 fetch_fsmap learned FSMap version 35 2024-05-27T08:18:49.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.221+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/4189837841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56209992b930 con 0x56209989f410 2024-05-27T08:18:49.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.222+0000 7fceefe00700 1 -- 172.21.15.63:0/4189837841 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcefc035070 con 0x56209989f410 2024-05-27T08:18:49.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.223+0000 7fceefe00700 1 -- 172.21.15.63:0/4189837841 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcf0000f040 con 0x56209989f410 2024-05-27T08:18:49.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.223+0000 7fcf23c6bb80 10 client.42557 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:18:49.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.223+0000 7fcf23c6bb80 10 client.42557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:18:49.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.223+0000 7fcf23c6bb80 20 client.42557 populate_metadata read hostname 'smithi063' 2024-05-27T08:18:49.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.223+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56209992c0a0 0x56209992e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:18:49.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.223+0000 7fcf23c6bb80 4 client.42557 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:18:49.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.223+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/4189837841 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620997c9570 con 0x56209992c0a0 2024-05-27T08:18:49.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.224+0000 7fcf0de00700 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56209992c0a0 0x56209992e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:18:49.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.227+0000 7fcf0de00700 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56209992c0a0 0x56209992e4c0 crc :-1 s=READY pgs=775 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f085156c20723e11 server_cookie=d6ccf59578edea89 in_seq=0 out_seq=0 2024-05-27T08:18:49.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.227+0000 7fceefe00700 10 client.42557 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:49.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fceefe00700 1 -- 172.21.15.63:0/4189837841 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620997c9570 con 0x56209992c0a0 2024-05-27T08:18:49.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fceefe00700 10 client.42557 handle_command_reply: tid=0 2024-05-27T08:18:49.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fcf23c6bb80 1 client.42557 shutdown 2024-05-27T08:18:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/4189837841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f410 msgr2=0x5620998c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f410 0x5620998c4130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcf1010c100 tx=0x7fcf00013a70 comp rx=0 tx=0).stop 2024-05-27T08:18:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/4189837841 shutdown_connections 2024-05-27T08:18:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56209992c0a0 0x56209992e4c0 crc :-1 s=READY pgs=775 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56209989f820 0x5620998c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fceefe00700 0 client.42557 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:18:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562099720e40 0x5620998c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fcf23c6bb80 1 --2- 172.21.15.63:0/4189837841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56209989f410 0x5620998c4130 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.229+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/4189837841 >> 172.21.15.63:0/4189837841 conn(0x562099780fc0 msgr2=0x5620998a8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:49.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.230+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/4189837841 shutdown_connections 2024-05-27T08:18:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.231+0000 7fcf23c6bb80 1 -- 172.21.15.63:0/4189837841 wait complete. 2024-05-27T08:18:49.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.231+0000 7fcf23c6bb80 20 client.42557 trim_cache size 0 max 16384 2024-05-27T08:18:49.235 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:18:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.232+0000 7fcf15200700 1 -- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcef40b2b40 msgr2=0x7fcef40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.232+0000 7fcf15200700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcef40b2b40 0x7fcef40b5050 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcefc000c00 tx=0x7fcefc01a040 comp rx=0 tx=0).stop 2024-05-27T08:18:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.232+0000 7fcf15200700 1 -- 172.21.15.63:0/314713529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10109680 msgr2=0x7fcf1006e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:18:49.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.232+0000 7fcf15200700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10109680 0x7fcf1006e900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcf08007ee0 tx=0x7fcf0800d3b0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.235+0000 7fcf15200700 1 -- 172.21.15.63:0/314713529 shutdown_connections 2024-05-27T08:18:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.235+0000 7fcf15200700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fcef40b2b40 0x7fcef40b5050 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.235+0000 7fcf15200700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10109680 0x7fcf1006e900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.235+0000 7fcf15200700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf10104400 0x7fcf10075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.235+0000 7fcf15200700 1 --2- 172.21.15.63:0/314713529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf10108d30 0x7fcf1006e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:18:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.235+0000 7fcf15200700 1 -- 172.21.15.63:0/314713529 >> 172.21.15.63:0/314713529 conn(0x7fcf10079d00 msgr2=0x7fcf10102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:18:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.235+0000 7fcf15200700 1 -- 172.21.15.63:0/314713529 shutdown_connections 2024-05-27T08:18:49.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:18:49.235+0000 7fcf15200700 1 -- 172.21.15.63:0/314713529 wait complete. 2024-05-27T08:18:49.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:49 smithi086 ceph-mon[34687]: pgmap v1439: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-05-27T08:18:49.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:49 smithi149 ceph-mon[35408]: pgmap v1439: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-05-27T08:18:51.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:51 smithi086 ceph-mon[34687]: pgmap v1440: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.7 MiB/s wr, 2.39k op/s 2024-05-27T08:18:51.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:51 smithi149 ceph-mon[35408]: pgmap v1440: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.7 MiB/s wr, 2.39k op/s 2024-05-27T08:18:51.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:51 smithi063 ceph-mon[31613]: pgmap v1440: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 14 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.7 MiB/s wr, 2.39k op/s 2024-05-27T08:18:52.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:52 smithi086 ceph-mon[34687]: osdmap e173: 12 total, 12 up, 12 in 2024-05-27T08:18:52.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:52 smithi149 ceph-mon[35408]: osdmap e173: 12 total, 12 up, 12 in 2024-05-27T08:18:52.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:52 smithi063 ceph-mon[31613]: osdmap e173: 12 total, 12 up, 12 in 2024-05-27T08:18:53.720 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:53 smithi063 ceph-mon[31613]: pgmap v1442: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-27T08:18:53.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:53 smithi086 ceph-mon[34687]: pgmap v1442: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-27T08:18:53.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:53 smithi149 ceph-mon[35408]: pgmap v1442: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 10 MiB/s wr, 2.61k op/s 2024-05-27T08:18:55.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:55 smithi086 ceph-mon[34687]: pgmap v1443: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-05-27T08:18:55.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:55 smithi149 ceph-mon[35408]: pgmap v1443: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-05-27T08:18:55.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:55 smithi063 ceph-mon[31613]: pgmap v1443: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 10 MiB/s wr, 2.56k op/s 2024-05-27T08:18:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:57 smithi063 ceph-mon[31613]: pgmap v1444: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.9 MiB/s wr, 2.23k op/s 2024-05-27T08:18:57.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:57 smithi086 ceph-mon[34687]: pgmap v1444: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.9 MiB/s wr, 2.23k op/s 2024-05-27T08:18:57.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:57 smithi149 ceph-mon[35408]: pgmap v1444: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.9 MiB/s wr, 2.23k op/s 2024-05-27T08:18:58.136 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:18:58.137 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:18:59.221 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:18:59 smithi063 ceph-mon[31613]: pgmap v1445: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-27T08:18:59.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:18:59 smithi086 ceph-mon[34687]: pgmap v1445: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-27T08:18:59.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:18:59 smithi149 ceph-mon[35408]: pgmap v1445: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 2.76k op/s 2024-05-27T08:18:59.969 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:19:01.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:01 smithi086 ceph-mon[34687]: pgmap v1446: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-05-27T08:19:01.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:01 smithi149 ceph-mon[35408]: pgmap v1446: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-05-27T08:19:01.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:01 smithi063 ceph-mon[31613]: pgmap v1446: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-05-27T08:19:03.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:03 smithi063 ceph-mon[31613]: pgmap v1447: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 12 MiB/s wr, 2.91k op/s 2024-05-27T08:19:03.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:03 smithi086 ceph-mon[34687]: pgmap v1447: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 12 MiB/s wr, 2.91k op/s 2024-05-27T08:19:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:03 smithi149 ceph-mon[35408]: pgmap v1447: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 12 MiB/s wr, 2.91k op/s 2024-05-27T08:19:05.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:05 smithi149 ceph-mon[35408]: pgmap v1448: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.3 MiB/s wr, 2.33k op/s 2024-05-27T08:19:05.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:05 smithi086 ceph-mon[34687]: pgmap v1448: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.3 MiB/s wr, 2.33k op/s 2024-05-27T08:19:05.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:05 smithi063 ceph-mon[31613]: pgmap v1448: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.3 MiB/s wr, 2.33k op/s 2024-05-27T08:19:07.270 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:07 smithi063 ceph-mon[31613]: pgmap v1449: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.3 MiB/s wr, 2.33k op/s 2024-05-27T08:19:07.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:07 smithi086 ceph-mon[34687]: pgmap v1449: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.3 MiB/s wr, 2.33k op/s 2024-05-27T08:19:07.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:07 smithi149 ceph-mon[35408]: pgmap v1449: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.3 MiB/s wr, 2.33k op/s 2024-05-27T08:19:09.304 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:09 smithi063 ceph-mon[31613]: pgmap v1450: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 2.82k op/s 2024-05-27T08:19:09.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:09 smithi086 ceph-mon[34687]: pgmap v1450: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 2.82k op/s 2024-05-27T08:19:09.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:09 smithi149 ceph-mon[35408]: pgmap v1450: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 2.82k op/s 2024-05-27T08:19:09.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.931+0000 7fd392c00700 1 -- 172.21.15.63:0/1165320337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd38c10a920 msgr2=0x7fd38c10ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:09.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.931+0000 7fd392c00700 1 --2- 172.21.15.63:0/1165320337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd38c10a920 0x7fd38c10ad30 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009990 tx=0x7fd37c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.932+0000 7fd392c00700 1 -- 172.21.15.63:0/1165320337 shutdown_connections 2024-05-27T08:19:09.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.932+0000 7fd392c00700 1 --2- 172.21.15.63:0/1165320337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c11d920 0x7fd38c11fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.932+0000 7fd392c00700 1 --2- 172.21.15.63:0/1165320337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd38c10a920 0x7fd38c10ad30 unknown :-1 s=CLOSED pgs=2982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.932+0000 7fd392c00700 1 --2- 172.21.15.63:0/1165320337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c10b270 0x7fd38c11d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.932+0000 7fd392c00700 1 -- 172.21.15.63:0/1165320337 >> 172.21.15.63:0/1165320337 conn(0x7fd38c06f4c0 msgr2=0x7fd38c06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:09.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.934+0000 7fd392c00700 1 -- 172.21.15.63:0/1165320337 shutdown_connections 2024-05-27T08:19:09.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.934+0000 7fd392c00700 1 -- 172.21.15.63:0/1165320337 wait complete. 2024-05-27T08:19:09.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.935+0000 7fd392c00700 1 Processor -- start 2024-05-27T08:19:09.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.936+0000 7fd392c00700 1 -- start start 2024-05-27T08:19:09.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.937+0000 7fd392c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c10a920 0x7fd38c1af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:09.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.937+0000 7fd392c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd38c10b270 0x7fd38c1afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:09.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.937+0000 7fd392c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c11d920 0x7fd38c1a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.937+0000 7fd392c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c123240 con 0x7fd38c10b270 2024-05-27T08:19:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.937+0000 7fd392c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1230c0 con 0x7fd38c11d920 2024-05-27T08:19:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.937+0000 7fd392c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1233c0 con 0x7fd38c10a920 2024-05-27T08:19:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.940+0000 7fd383400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd38c10b270 0x7fd38c1afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.940+0000 7fd383400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd38c10b270 0x7fd38c1afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53894/0 (socket says 172.21.15.63:53894) 2024-05-27T08:19:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.940+0000 7fd383400700 1 -- 172.21.15.63:0/2406975251 learned_addr learned my addr 172.21.15.63:0/2406975251 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.940+0000 7fd383400700 1 -- 172.21.15.63:0/2406975251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c10a920 msgr2=0x7fd38c1af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.940+0000 7fd390c00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c11d920 0x7fd38c1a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:09.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.940+0000 7fd38be00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c10a920 0x7fd38c1af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.941+0000 7fd383400700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c10a920 0x7fd38c1af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.941+0000 7fd383400700 1 -- 172.21.15.63:0/2406975251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c11d920 msgr2=0x7fd38c1a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.941+0000 7fd383400700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c11d920 0x7fd38c1a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.941+0000 7fd383400700 1 -- 172.21.15.63:0/2406975251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c1aa030 con 0x7fd38c10b270 2024-05-27T08:19:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.941+0000 7fd390c00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c11d920 0x7fd38c1a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:19:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.941+0000 7fd38be00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c10a920 0x7fd38c1af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:19:09.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.943+0000 7fd383400700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd38c10b270 0x7fd38c1afc80 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7fd37400cce0 tx=0x7fd3740074a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:09.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.945+0000 7fd389600700 1 -- 172.21.15.63:0/2406975251 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374007af0 con 0x7fd38c10b270 2024-05-27T08:19:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.945+0000 7fd389600700 1 -- 172.21.15.63:0/2406975251 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd374004cc0 con 0x7fd38c10b270 2024-05-27T08:19:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.945+0000 7fd389600700 1 -- 172.21.15.63:0/2406975251 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3740055e0 con 0x7fd38c10b270 2024-05-27T08:19:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.945+0000 7fd392c00700 1 -- 172.21.15.63:0/2406975251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c1aa320 con 0x7fd38c10b270 2024-05-27T08:19:09.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.946+0000 7fd392c00700 1 -- 172.21.15.63:0/2406975251 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c112bc0 con 0x7fd38c10b270 2024-05-27T08:19:09.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.967+0000 7fd389600700 1 -- 172.21.15.63:0/2406975251 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd374007c90 con 0x7fd38c10b270 2024-05-27T08:19:09.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.970+0000 7fd389600700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3700b2b70 0x7fd3700b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:09.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.971+0000 7fd389600700 1 -- 172.21.15.63:0/2406975251 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 9527+0+0 (secure 0 0 0) 0x7fd3740e2250 con 0x7fd38c10b270 2024-05-27T08:19:09.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.971+0000 7fd38be00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3700b2b70 0x7fd3700b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:09.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.974+0000 7fd38be00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3700b2b70 0x7fd3700b5080 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd37c006010 tx=0x7fd37c0058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:09.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.986+0000 7fd3a1830b80 1 Processor -- start 2024-05-27T08:19:09.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.986+0000 7fd3a1830b80 1 -- start start 2024-05-27T08:19:09.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.986+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a8218743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:09.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.986+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 0x55a8218748f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:09.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.986+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 0x55a821881190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:09.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.986+0000 7fd3a1830b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a8217a2ec0 con 0x55a821874ef0 2024-05-27T08:19:09.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.986+0000 7fd3a1830b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a821792a60 con 0x55a8217e1a70 2024-05-27T08:19:09.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.986+0000 7fd3a1830b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a821796aa0 con 0x55a8216f5ec0 2024-05-27T08:19:09.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.987+0000 7fd38be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a8218743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:09.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.987+0000 7fd383400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 0x55a8218748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:09.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.987+0000 7fd390c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 0x55a821881190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:09.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.987+0000 7fd38be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a8218743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:41788/0 (socket says 172.21.15.63:41788) 2024-05-27T08:19:09.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.987+0000 7fd38be00700 1 -- 172.21.15.63:0/3620848485 learned_addr learned my addr 172.21.15.63:0/3620848485 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:09.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.988+0000 7fd38be00700 1 -- 172.21.15.63:0/3620848485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 msgr2=0x55a8218748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:09.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.988+0000 7fd38be00700 1 --2- 172.21.15.63:0/3620848485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 0x55a8218748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.988+0000 7fd38be00700 1 -- 172.21.15.63:0/3620848485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 msgr2=0x55a821881190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:09.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.988+0000 7fd38be00700 1 --2- 172.21.15.63:0/3620848485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 0x55a821881190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.988+0000 7fd38be00700 1 -- 172.21.15.63:0/3620848485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82175b4a0 con 0x55a8217e1a70 2024-05-27T08:19:09.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.989+0000 7fd38be00700 1 --2- 172.21.15.63:0/3620848485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a8218743b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009990 tx=0x7fd37c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9abfe6d8eec4f62 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:09.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.989+0000 7fd381600700 1 -- 172.21.15.63:0/3620848485 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c00f6a0 con 0x55a8217e1a70 2024-05-27T08:19:09.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.989+0000 7fd381600700 1 -- 172.21.15.63:0/3620848485 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd37c00fcd0 con 0x55a8217e1a70 2024-05-27T08:19:09.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.989+0000 7fd381600700 1 -- 172.21.15.63:0/3620848485 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c02fce0 con 0x55a8217e1a70 2024-05-27T08:19:09.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.991+0000 7fd3a1830b80 1 -- 172.21.15.63:0/3620848485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 msgr2=0x55a8218743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:09.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.991+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/3620848485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a8218743b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009990 tx=0x7fd37c01d000 comp rx=0 tx=0).stop 2024-05-27T08:19:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.992+0000 7fd3a1830b80 1 -- 172.21.15.63:0/3620848485 shutdown_connections 2024-05-27T08:19:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.992+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/3620848485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 0x55a821881190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.992+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/3620848485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a8218743b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.992+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/3620848485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 0x55a8218748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.992+0000 7fd3a1830b80 1 -- 172.21.15.63:0/3620848485 >> 172.21.15.63:0/3620848485 conn(0x55a82178f410 msgr2=0x55a821870540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.992+0000 7fd3a1830b80 1 -- 172.21.15.63:0/3620848485 shutdown_connections 2024-05-27T08:19:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.993+0000 7fd3a1830b80 1 -- 172.21.15.63:0/3620848485 wait complete. 2024-05-27T08:19:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.994+0000 7fd3a1830b80 1 -- start start 2024-05-27T08:19:09.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.995+0000 7fd3a1830b80 1 Processor -- start 2024-05-27T08:19:10.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.996+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8216f5ec0 0x55a82187e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.996+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a8217e1a70 0x55a82187ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.996+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821874ef0 0x55a82187f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.996+0000 7fd3a1830b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a821699020 con 0x55a8217e1a70 2024-05-27T08:19:10.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.996+0000 7fd3a1830b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8217337e0 con 0x55a8216f5ec0 2024-05-27T08:19:10.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.996+0000 7fd3a1830b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a821881ba0 con 0x55a821874ef0 2024-05-27T08:19:10.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.996+0000 7fd3a1830b80 10 client.0 _ll_register_callbacks cb 0x55a82173ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:19:10.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.997+0000 7fd38be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8216f5ec0 0x55a82187e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:10.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.997+0000 7fd38be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8216f5ec0 0x55a82187e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:41804/0 (socket says 172.21.15.63:41804) 2024-05-27T08:19:10.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.997+0000 7fd38be00700 1 -- 172.21.15.63:0/2373831969 learned_addr learned my addr 172.21.15.63:0/2373831969 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:10.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.998+0000 7fd38be00700 1 -- 172.21.15.63:0/2373831969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821874ef0 msgr2=0x55a82187f260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:19:10.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.998+0000 7fd38be00700 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821874ef0 0x55a82187f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.998+0000 7fd38be00700 1 -- 172.21.15.63:0/2373831969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a8217e1a70 msgr2=0x55a82187ed20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:19:10.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.998+0000 7fd38be00700 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a8217e1a70 0x55a82187ed20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:09.998+0000 7fd38be00700 1 -- 172.21.15.63:0/2373831969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8216e2950 con 0x55a8216f5ec0 2024-05-27T08:19:10.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.000+0000 7fd38be00700 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8216f5ec0 0x55a82187e7e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00a010 tx=0x7fd37c00fc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4adb2564c77c8fee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:10.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.001+0000 7fd37a000700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:19:10.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.001+0000 7fd37a000700 1 -- 172.21.15.63:0/2373831969 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0257f0 con 0x55a8216f5ec0 2024-05-27T08:19:10.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.001+0000 7fd37a000700 1 -- 172.21.15.63:0/2373831969 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 9527+0+0 (secure 0 0 0) 0x7fd37c025990 con 0x55a8216f5ec0 2024-05-27T08:19:10.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.001+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2373831969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd37c00fcd0 con 0x55a8216f5ec0 2024-05-27T08:19:10.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.002+0000 7fd37a000700 1 -- 172.21.15.63:0/2373831969 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd37c025b90 con 0x55a8216f5ec0 2024-05-27T08:19:10.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.002+0000 7fd3a1830b80 10 client.42563 fetch_fsmap learned FSMap version 35 2024-05-27T08:19:10.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.002+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2373831969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a8217630e0 con 0x55a8216f5ec0 2024-05-27T08:19:10.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.002+0000 7fd37a000700 1 -- 172.21.15.63:0/2373831969 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd37c00fcd0 con 0x55a8216f5ec0 2024-05-27T08:19:10.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.004+0000 7fd37a000700 1 -- 172.21.15.63:0/2373831969 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd37c02f480 con 0x55a8216f5ec0 2024-05-27T08:19:10.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.004+0000 7fd3a1830b80 10 client.42563 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:19:10.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.004+0000 7fd3a1830b80 10 client.42563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:19:10.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.004+0000 7fd3a1830b80 20 client.42563 populate_metadata read hostname 'smithi063' 2024-05-27T08:19:10.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.004+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a82188e930 0x55a821890d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.004+0000 7fd3a1830b80 4 client.42563 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:19:10.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.004+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2373831969 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8217a4e10 con 0x55a82188e930 2024-05-27T08:19:10.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.006+0000 7fd383400700 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a82188e930 0x55a821890d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:10.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.007+0000 7fd37a000700 10 client.42563 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:10.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.007+0000 7fd383400700 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a82188e930 0x55a821890d90 crc :-1 s=READY pgs=776 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d79163f3b6ef7c server_cookie=96a8fad48535a502 in_seq=0 out_seq=0 2024-05-27T08:19:10.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.008+0000 7fd37a000700 1 -- 172.21.15.63:0/2373831969 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a8217a4e10 con 0x55a82188e930 2024-05-27T08:19:10.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.008+0000 7fd37a000700 10 client.42563 handle_command_reply: tid=0 2024-05-27T08:19:10.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.009+0000 7fd3a1830b80 1 client.42563 shutdown 2024-05-27T08:19:10.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.009+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2373831969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8216f5ec0 msgr2=0x55a82187e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:10.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.009+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8216f5ec0 0x55a82187e7e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00a010 tx=0x7fd37c00fc50 comp rx=0 tx=0).stop 2024-05-27T08:19:10.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.010+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2373831969 shutdown_connections 2024-05-27T08:19:10.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.010+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a82188e930 0x55a821890d90 crc :-1 s=READY pgs=776 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.010+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821874ef0 0x55a82187f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.010+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8216f5ec0 0x55a82187e7e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.010+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/2373831969 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a8217e1a70 0x55a82187ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.010+0000 7fd37a000700 0 client.42563 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.010+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2373831969 >> 172.21.15.63:0/2373831969 conn(0x55a82178f410 msgr2=0x55a821874470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.010+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2373831969 shutdown_connections 2024-05-27T08:19:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.010+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2373831969 wait complete. 2024-05-27T08:19:10.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.010+0000 7fd3a1830b80 20 client.42563 trim_cache size 0 max 16384 2024-05-27T08:19:10.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.059+0000 7fd3a1830b80 1 Processor -- start 2024-05-27T08:19:10.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.062+0000 7fd3a1830b80 1 -- start start 2024-05-27T08:19:10.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.062+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 0x55a82187a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.062+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a82187f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.062+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 0x55a82188ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.062+0000 7fd3a1830b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1ca350 con 0x55a821874ef0 2024-05-27T08:19:10.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.062+0000 7fd3a1830b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1c7be0 con 0x55a8217e1a70 2024-05-27T08:19:10.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.062+0000 7fd3a1830b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1197d0 con 0x55a8216f5ec0 2024-05-27T08:19:10.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.063+0000 7fd38be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 0x55a82187a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:10.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.063+0000 7fd383400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a82187f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:10.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.064+0000 7fd383400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a82187f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:41810/0 (socket says 172.21.15.63:41810) 2024-05-27T08:19:10.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.064+0000 7fd383400700 1 -- 172.21.15.63:0/2613620551 learned_addr learned my addr 172.21.15.63:0/2613620551 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:10.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.064+0000 7fd38be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 0x55a82187a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:40284/0 (socket says 172.21.15.63:40284) 2024-05-27T08:19:10.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.066+0000 7fd390c00700 1 --2- 172.21.15.63:0/2613620551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 0x55a82188ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:10.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.066+0000 7fd38be00700 1 -- 172.21.15.63:0/2613620551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 msgr2=0x55a82187f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:10.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.066+0000 7fd38be00700 1 --2- 172.21.15.63:0/2613620551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a82187f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.066+0000 7fd38be00700 1 -- 172.21.15.63:0/2613620551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 msgr2=0x55a82188ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:10.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.069+0000 7fd38be00700 1 --2- 172.21.15.63:0/2613620551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 0x55a82188ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.069+0000 7fd38be00700 1 -- 172.21.15.63:0/2613620551 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82186c2c0 con 0x55a8216f5ec0 2024-05-27T08:19:10.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.069+0000 7fd390c00700 1 --2- 172.21.15.63:0/2613620551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 0x55a82188ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:19:10.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.071+0000 7fd38be00700 1 --2- 172.21.15.63:0/2613620551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 0x55a82187a010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd37c02d770 tx=0x7fd37c02fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90416538a9b58b58 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:10.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.074+0000 7fd380c00700 1 -- 172.21.15.63:0/2613620551 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c025b90 con 0x55a8216f5ec0 2024-05-27T08:19:10.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.075+0000 7fd380c00700 1 -- 172.21.15.63:0/2613620551 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd37c00fcd0 con 0x55a8216f5ec0 2024-05-27T08:19:10.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.075+0000 7fd380c00700 1 -- 172.21.15.63:0/2613620551 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c02f870 con 0x55a8216f5ec0 2024-05-27T08:19:10.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.076+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2613620551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 msgr2=0x55a82187a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:10.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.076+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/2613620551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 0x55a82187a010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd37c02d770 tx=0x7fd37c02fc40 comp rx=0 tx=0).stop 2024-05-27T08:19:10.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.076+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2613620551 shutdown_connections 2024-05-27T08:19:10.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.076+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/2613620551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a821874ef0 0x55a82188ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.076+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/2613620551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8216f5ec0 0x55a82187a010 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd37c02d770 tx=0x7fd37c02fc40 comp rx=0 tx=0).stop 2024-05-27T08:19:10.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.076+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/2613620551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8217e1a70 0x55a82187f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.076+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2613620551 >> 172.21.15.63:0/2613620551 conn(0x55a82178f410 msgr2=0x55a8218747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:10.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.080+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2613620551 shutdown_connections 2024-05-27T08:19:10.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.080+0000 7fd3a1830b80 1 -- 172.21.15.63:0/2613620551 wait complete. 2024-05-27T08:19:10.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.081+0000 7fd3a1830b80 1 -- start start 2024-05-27T08:19:10.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.081+0000 7fd3a1830b80 1 Processor -- start 2024-05-27T08:19:10.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.082+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a8217e1a70 0x55a821886e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.083+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821874ef0 0x55a821887350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.083+0000 7fd3a1830b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821881dc0 0x55a8218821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.083+0000 7fd3a1830b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c119e80 con 0x55a8217e1a70 2024-05-27T08:19:10.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.083+0000 7fd390c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821881dc0 0x55a8218821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:10.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.083+0000 7fd390c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821881dc0 0x55a8218821d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:40296/0 (socket says 172.21.15.63:40296) 2024-05-27T08:19:10.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.084+0000 7fd390c00700 1 -- 172.21.15.63:0/477292701 learned_addr learned my addr 172.21.15.63:0/477292701 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:10.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.084+0000 7fd3a1830b80 1 -- 172.21.15.63:0/477292701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1c75e0 con 0x55a821874ef0 2024-05-27T08:19:10.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.084+0000 7fd3a1830b80 1 -- 172.21.15.63:0/477292701 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c11a550 con 0x55a821881dc0 2024-05-27T08:19:10.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.084+0000 7fd383400700 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821874ef0 0x55a821887350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:10.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.084+0000 7fd3a1830b80 10 client.0 _ll_register_callbacks cb 0x55a82173ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:19:10.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.084+0000 7fd390c00700 1 -- 172.21.15.63:0/477292701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821874ef0 msgr2=0x55a821887350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:10.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.085+0000 7fd390c00700 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821874ef0 0x55a821887350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.085+0000 7fd390c00700 1 -- 172.21.15.63:0/477292701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a8217e1a70 msgr2=0x55a821886e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:10.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.085+0000 7fd390c00700 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a8217e1a70 0x55a821886e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.085+0000 7fd390c00700 1 -- 172.21.15.63:0/477292701 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a821887940 con 0x55a821881dc0 2024-05-27T08:19:10.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.085+0000 7fd390c00700 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821881dc0 0x55a8218821d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd384010c60 tx=0x7fd384010c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5779d1771f67d1b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:10.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.085+0000 7fd37a000700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:19:10.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.086+0000 7fd37a000700 1 -- 172.21.15.63:0/477292701 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38400e950 con 0x55a821881dc0 2024-05-27T08:19:10.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.086+0000 7fd37a000700 1 -- 172.21.15.63:0/477292701 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 9527+0+0 (secure 0 0 0) 0x7fd38400eaf0 con 0x55a821881dc0 2024-05-27T08:19:10.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.086+0000 7fd3a1830b80 1 -- 172.21.15.63:0/477292701 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd37c00fcd0 con 0x55a821881dc0 2024-05-27T08:19:10.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.086+0000 7fd37a000700 1 -- 172.21.15.63:0/477292701 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd384007e30 con 0x55a821881dc0 2024-05-27T08:19:10.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.087+0000 7fd3a1830b80 10 client.32386 fetch_fsmap learned FSMap version 35 2024-05-27T08:19:10.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.087+0000 7fd3a1830b80 1 -- 172.21.15.63:0/477292701 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a8218f9cf0 con 0x55a821881dc0 2024-05-27T08:19:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.087+0000 7fd37a000700 1 -- 172.21.15.63:0/477292701 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd37c00fcd0 con 0x55a821881dc0 2024-05-27T08:19:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.087+0000 7fd37a000700 1 -- 172.21.15.63:0/477292701 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd384011430 con 0x55a821881dc0 2024-05-27T08:19:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.087+0000 7fd3a1830b80 10 client.32386 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:19:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.087+0000 7fd3a1830b80 10 client.32386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:19:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.087+0000 7fd3a1830b80 20 client.32386 populate_metadata read hostname 'smithi063' 2024-05-27T08:19:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.088+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a8218fa480 0x55a8218fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.088+0000 7fd3a1830b80 4 client.32386 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:19:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.088+0000 7fd3a1830b80 1 -- 172.21.15.63:0/477292701 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a82179e5f0 con 0x55a8218fa480 2024-05-27T08:19:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.088+0000 7fd38be00700 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a8218fa480 0x55a8218fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:10.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.090+0000 7fd37a000700 10 client.32386 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:10.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.090+0000 7fd38be00700 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a8218fa480 0x55a8218fc8a0 crc :-1 s=READY pgs=777 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42572c12c8c17f7e server_cookie=8004ae3fe6872bb6 in_seq=0 out_seq=0 2024-05-27T08:19:10.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.094+0000 7fd37a000700 1 -- 172.21.15.63:0/477292701 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a82179e5f0 con 0x55a8218fa480 2024-05-27T08:19:10.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.094+0000 7fd37a000700 10 client.32386 handle_command_reply: tid=0 2024-05-27T08:19:10.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.094+0000 7fd3a1830b80 1 client.32386 shutdown 2024-05-27T08:19:10.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.094+0000 7fd3a1830b80 1 -- 172.21.15.63:0/477292701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821881dc0 msgr2=0x55a8218821d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:10.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.094+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821881dc0 0x55a8218821d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd384010c60 tx=0x7fd384010c90 comp rx=0 tx=0).stop 2024-05-27T08:19:10.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.095+0000 7fd3a1830b80 1 -- 172.21.15.63:0/477292701 shutdown_connections 2024-05-27T08:19:10.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.096+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55a8218fa480 0x55a8218fc8a0 crc :-1 s=READY pgs=777 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.096+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a821881dc0 0x55a8218821d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.096+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a8217e1a70 0x55a821886e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.096+0000 7fd3a1830b80 1 --2- 172.21.15.63:0/477292701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821874ef0 0x55a821887350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.096+0000 7fd3a1830b80 1 -- 172.21.15.63:0/477292701 >> 172.21.15.63:0/477292701 conn(0x55a82178f410 msgr2=0x55a8218747a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:10.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.096+0000 7fd37a000700 0 client.32386 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:10.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.096+0000 7fd3a1830b80 1 -- 172.21.15.63:0/477292701 shutdown_connections 2024-05-27T08:19:10.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.096+0000 7fd3a1830b80 1 -- 172.21.15.63:0/477292701 wait complete. 2024-05-27T08:19:10.101 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.096+0000 7fd3a1830b80 20 client.32386 trim_cache size 0 max 16384 2024-05-27T08:19:10.105 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:19:10.105 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:19:10.105 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "ad7f32b4-aee4-439d-9806-5c01c1448f81", 2024-05-27T08:19:10.105 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:19:10.105 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:19:10.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.106+0000 7fd392c00700 1 -- 172.21.15.63:0/2406975251 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3700b2b70 msgr2=0x7fd3700b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:10.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.106+0000 7fd392c00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3700b2b70 0x7fd3700b5080 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd37c006010 tx=0x7fd37c0058e0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.106+0000 7fd392c00700 1 -- 172.21.15.63:0/2406975251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd38c10b270 msgr2=0x7fd38c1afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:10.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.106+0000 7fd392c00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd38c10b270 0x7fd38c1afc80 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7fd37400cce0 tx=0x7fd3740074a0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.107+0000 7fd392c00700 1 -- 172.21.15.63:0/2406975251 shutdown_connections 2024-05-27T08:19:10.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.107+0000 7fd392c00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fd3700b2b70 0x7fd3700b5080 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.107+0000 7fd392c00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c11d920 0x7fd38c1a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.107+0000 7fd392c00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c10a920 0x7fd38c1af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.107+0000 7fd392c00700 1 --2- 172.21.15.63:0/2406975251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd38c10b270 0x7fd38c1afc80 unknown :-1 s=CLOSED pgs=2983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:10.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.107+0000 7fd392c00700 1 -- 172.21.15.63:0/2406975251 >> 172.21.15.63:0/2406975251 conn(0x7fd38c06f4c0 msgr2=0x7fd38c11b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:10.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.111+0000 7fd392c00700 1 -- 172.21.15.63:0/2406975251 shutdown_connections 2024-05-27T08:19:10.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:10.111+0000 7fd392c00700 1 -- 172.21.15.63:0/2406975251 wait complete. 2024-05-27T08:19:10.322 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:10 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:19:10.322 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:10 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:10.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:10 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:19:10.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:10 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:10.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:10 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:19:10.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:10 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:19:10.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:10 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:10.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:10 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:19:10.720 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:10 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:19:11.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:11 smithi086 ceph-mon[34687]: pgmap v1451: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.6 MiB/s wr, 2.15k op/s 2024-05-27T08:19:11.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:11 smithi149 ceph-mon[35408]: pgmap v1451: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.6 MiB/s wr, 2.15k op/s 2024-05-27T08:19:11.718 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:11 smithi063 ceph-mon[31613]: pgmap v1451: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.6 MiB/s wr, 2.15k op/s 2024-05-27T08:19:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.985+0000 7fb209600640 1 -- 172.21.15.63:0/3818917878 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5646e73ba260 con 0x5646e6fa71c0 2024-05-27T08:19:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.985+0000 7fb209600640 10 client.32365 handle_command_reply: tid=0 2024-05-27T08:19:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.986+0000 7fb22d986740 1 client.32365 shutdown 2024-05-27T08:19:11.988 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 msgr2=0x5646e7438890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb22d986740 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e7438890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb218031920 tx=0x7fb218030da0 comp rx=0 tx=0).stop 2024-05-27T08:19:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 shutdown_connections 2024-05-27T08:19:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb22d986740 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5646e6fa71c0 0x5646e743c510 crc :-1 s=READY pgs=773 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb22d986740 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5646e70131b0 0x5646e7438890 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb22d986740 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646e6ff24d0 0x5646e73c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb22d986740 1 --2- 172.21.15.63:0/3818917878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646e6fea2f0 0x5646e73c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 >> 172.21.15.63:0/3818917878 conn(0x5646e7007300 msgr2=0x5646e73bfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb209600640 0 client.32365 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 shutdown_connections 2024-05-27T08:19:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.987+0000 7fb22d986740 1 -- 172.21.15.63:0/3818917878 wait complete. 2024-05-27T08:19:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.989+0000 7fb22d986740 20 client.32365 trim_cache size 0 max 16384 2024-05-27T08:19:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:19:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:19:11.998 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:19:12.000 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.997+0000 7fb225400640 1 -- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2040b2b60 msgr2=0x7fb2040b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.997+0000 7fb225400640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2040b2b60 0x7fb2040b5070 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb218031c10 tx=0x7fb218031c40 comp rx=0 tx=0).stop 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.997+0000 7fb225400640 1 -- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb22010d9e0 msgr2=0x7fb22010c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.997+0000 7fb225400640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb22010d9e0 0x7fb22010c6e0 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c970 tx=0x7fb20c00ce30 comp rx=0 tx=0).stop 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.998+0000 7fb225400640 1 -- 172.21.15.63:0/191418915 shutdown_connections 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.998+0000 7fb225400640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb2040b2b60 0x7fb2040b5070 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.998+0000 7fb225400640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb220110340 0x7fb2201a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.998+0000 7fb225400640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb22010d9e0 0x7fb22010c6e0 secure :-1 s=CLOSED pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c970 tx=0x7fb20c00ce30 comp rx=0 tx=0).stop 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.998+0000 7fb225400640 1 --2- 172.21.15.63:0/191418915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2200764e0 0x7fb22010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.998+0000 7fb225400640 1 -- 172.21.15.63:0/191418915 >> 172.21.15.63:0/191418915 conn(0x7fb2200ff350 msgr2=0x7fb220101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.999+0000 7fb225400640 1 -- 172.21.15.63:0/191418915 shutdown_connections 2024-05-27T08:19:12.001 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:11.999+0000 7fb225400640 1 -- 172.21.15.63:0/191418915 wait complete. 2024-05-27T08:19:12.213 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:19:13.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:13 smithi063 ceph-mon[31613]: pgmap v1452: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-05-27T08:19:13.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:13 smithi086 ceph-mon[34687]: pgmap v1452: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-05-27T08:19:13.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:13 smithi149 ceph-mon[35408]: pgmap v1452: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-05-27T08:19:13.545 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:19:15.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:15 smithi086 ceph-mon[34687]: pgmap v1453: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.9 MiB/s wr, 2.45k op/s 2024-05-27T08:19:15.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:15 smithi149 ceph-mon[35408]: pgmap v1453: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.9 MiB/s wr, 2.45k op/s 2024-05-27T08:19:15.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:15 smithi063 ceph-mon[31613]: pgmap v1453: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 9.9 MiB/s wr, 2.45k op/s 2024-05-27T08:19:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:17 smithi086 ceph-mon[34687]: osdmap e174: 12 total, 12 up, 12 in 2024-05-27T08:19:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:17 smithi086 ceph-mon[34687]: pgmap v1455: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 2.65k op/s 2024-05-27T08:19:17.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:17 smithi149 ceph-mon[35408]: osdmap e174: 12 total, 12 up, 12 in 2024-05-27T08:19:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:17 smithi063 ceph-mon[31613]: osdmap e174: 12 total, 12 up, 12 in 2024-05-27T08:19:17.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:17 smithi063 ceph-mon[31613]: pgmap v1455: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 2.65k op/s 2024-05-27T08:19:18.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:18 smithi149 ceph-mon[35408]: pgmap v1455: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 2.65k op/s 2024-05-27T08:19:19.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:19 smithi086 ceph-mon[34687]: pgmap v1456: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.7 MiB/s wr, 2.44k op/s 2024-05-27T08:19:19.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:19 smithi149 ceph-mon[35408]: pgmap v1456: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.7 MiB/s wr, 2.44k op/s 2024-05-27T08:19:19.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:19 smithi063 ceph-mon[31613]: pgmap v1456: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.7 MiB/s wr, 2.44k op/s 2024-05-27T08:19:21.335 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:21 smithi063 ceph-mon[31613]: pgmap v1457: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.3 MiB/s wr, 2.33k op/s 2024-05-27T08:19:21.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:21 smithi086 ceph-mon[34687]: pgmap v1457: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.3 MiB/s wr, 2.33k op/s 2024-05-27T08:19:21.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:21 smithi149 ceph-mon[35408]: pgmap v1457: 129 pgs: 17 active+clean+snaptrim, 37 active+clean+snaptrim_wait, 75 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 9.3 MiB/s wr, 2.33k op/s 2024-05-27T08:19:22.369 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:22 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:19:22.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:22 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:19:22.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:22 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:19:23.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:23 smithi086 ceph-mon[34687]: pgmap v1458: 129 pgs: 19 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.3 MiB/s wr, 2.10k op/s 2024-05-27T08:19:23.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:23 smithi149 ceph-mon[35408]: pgmap v1458: 129 pgs: 19 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.3 MiB/s wr, 2.10k op/s 2024-05-27T08:19:23.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:23 smithi063 ceph-mon[31613]: pgmap v1458: 129 pgs: 19 active+clean+snaptrim, 43 active+clean+snaptrim_wait, 67 active+clean; 15 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.3 MiB/s wr, 2.10k op/s 2024-05-27T08:19:24.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.862+0000 7fab0ca00700 1 -- 172.21.15.63:0/24377500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab08108f10 msgr2=0x7fab08109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:24.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.862+0000 7fab0ca00700 1 --2- 172.21.15.63:0/24377500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab08108f10 0x7fab08109320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faaf80099c0 tx=0x7faaf8009cd0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.865+0000 7fab0ca00700 1 -- 172.21.15.63:0/24377500 shutdown_connections 2024-05-27T08:19:24.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.865+0000 7fab0ca00700 1 --2- 172.21.15.63:0/24377500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab08109860 0x7fab081111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.865+0000 7fab0ca00700 1 --2- 172.21.15.63:0/24377500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab081045e0 0x7fab081049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.865+0000 7fab0ca00700 1 --2- 172.21.15.63:0/24377500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab08108f10 0x7fab08109320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.865+0000 7fab0ca00700 1 -- 172.21.15.63:0/24377500 >> 172.21.15.63:0/24377500 conn(0x7fab08100030 msgr2=0x7fab08102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:24.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.865+0000 7fab0ca00700 1 -- 172.21.15.63:0/24377500 shutdown_connections 2024-05-27T08:19:24.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.865+0000 7fab0ca00700 1 -- 172.21.15.63:0/24377500 wait complete. 2024-05-27T08:19:24.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.871+0000 7fab0ca00700 1 Processor -- start 2024-05-27T08:19:24.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.871+0000 7fab0ca00700 1 -- start start 2024-05-27T08:19:24.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.872+0000 7fab0ca00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab081045e0 0x7fab08199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.873+0000 7fab0ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab08108f10 0x7fab0819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.873+0000 7fab0ca00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab08109860 0x7fab081a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.873+0000 7fab0ca00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fab08114480 con 0x7fab08109860 2024-05-27T08:19:24.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.873+0000 7fab05400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab08108f10 0x7fab0819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.873+0000 7fab05400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab08108f10 0x7fab0819a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:48926/0 (socket says 172.21.15.63:48926) 2024-05-27T08:19:24.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.873+0000 7fab05400700 1 -- 172.21.15.63:0/2069707828 learned_addr learned my addr 172.21.15.63:0/2069707828 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:24.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.873+0000 7fab05e00700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab081045e0 0x7fab08199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.874+0000 7fab0ca00700 1 -- 172.21.15.63:0/2069707828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab08114300 con 0x7fab08108f10 2024-05-27T08:19:24.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.874+0000 7fab0ca00700 1 -- 172.21.15.63:0/2069707828 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fab08114600 con 0x7fab081045e0 2024-05-27T08:19:24.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.875+0000 7fab05400700 1 -- 172.21.15.63:0/2069707828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab081045e0 msgr2=0x7fab08199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:24.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.875+0000 7fab05400700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab081045e0 0x7fab08199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.875+0000 7fab05400700 1 -- 172.21.15.63:0/2069707828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab08109860 msgr2=0x7fab081a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:24.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.875+0000 7fab05400700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab08109860 0x7fab081a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.875+0000 7fab05400700 1 -- 172.21.15.63:0/2069707828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab081a0ef0 con 0x7fab08108f10 2024-05-27T08:19:24.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.876+0000 7fab05400700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab08108f10 0x7fab0819a060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faaf8006010 tx=0x7faaf80057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:24.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.876+0000 7faafea00700 1 -- 172.21.15.63:0/2069707828 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8027070 con 0x7fab08108f10 2024-05-27T08:19:24.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.876+0000 7faafea00700 1 -- 172.21.15.63:0/2069707828 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaf8003ff0 con 0x7fab08108f10 2024-05-27T08:19:24.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.876+0000 7faafea00700 1 -- 172.21.15.63:0/2069707828 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8022cc0 con 0x7fab08108f10 2024-05-27T08:19:24.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.877+0000 7fab0ca00700 1 -- 172.21.15.63:0/2069707828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab081a1180 con 0x7fab08108f10 2024-05-27T08:19:24.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.877+0000 7fab0ca00700 1 -- 172.21.15.63:0/2069707828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab081a1610 con 0x7fab08108f10 2024-05-27T08:19:24.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.883+0000 7faafea00700 1 -- 172.21.15.63:0/2069707828 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faaf800f460 con 0x7fab08108f10 2024-05-27T08:19:24.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.885+0000 7faafea00700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faaec0b2bc0 0x7faaec0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.886+0000 7faafea00700 1 -- 172.21.15.63:0/2069707828 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7faaf80e4b70 con 0x7fab08108f10 2024-05-27T08:19:24.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.887+0000 7fab05e00700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faaec0b2bc0 0x7faaec0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.896+0000 7fab05e00700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faaec0b2bc0 0x7faaec0b50d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fab0000a400 tx=0x7fab0000d040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.899+0000 7fab1b51db80 1 Processor -- start 2024-05-27T08:19:24.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.900+0000 7fab1b51db80 1 -- start start 2024-05-27T08:19:24.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.901+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 0x558d881087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.901+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d88102870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.901+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d881031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.902+0000 7fab1b51db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558d88030e40 con 0x558d87f83e40 2024-05-27T08:19:24.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.902+0000 7fab05e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 0x558d881087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.902+0000 7fab05400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d88102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.902+0000 7fab05400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d88102870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45254/0 (socket says 172.21.15.63:45254) 2024-05-27T08:19:24.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.902+0000 7fab05400700 1 -- 172.21.15.63:0/687289232 learned_addr learned my addr 172.21.15.63:0/687289232 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:24.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.903+0000 7fab06800700 1 --2- 172.21.15.63:0/687289232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d881031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.904+0000 7fab1b51db80 1 -- 172.21.15.63:0/687289232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558d880209e0 con 0x558d88102db0 2024-05-27T08:19:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.904+0000 7fab1b51db80 1 -- 172.21.15.63:0/687289232 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558d88024a20 con 0x558d88108d30 2024-05-27T08:19:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.905+0000 7fab05400700 1 -- 172.21.15.63:0/687289232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 msgr2=0x558d881031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.905+0000 7fab05400700 1 --2- 172.21.15.63:0/687289232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d881031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.905+0000 7fab05400700 1 -- 172.21.15.63:0/687289232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 msgr2=0x558d881087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.905+0000 7fab05400700 1 --2- 172.21.15.63:0/687289232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 0x558d881087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.905+0000 7fab05400700 1 -- 172.21.15.63:0/687289232 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d87fe9420 con 0x558d88108d30 2024-05-27T08:19:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.906+0000 7fab05400700 1 --2- 172.21.15.63:0/687289232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d88102870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c010 tx=0x7faaf80ab5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15fef153512fd4da server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.908+0000 7faaf3400700 1 -- 172.21.15.63:0/687289232 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8022e60 con 0x558d88108d30 2024-05-27T08:19:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.908+0000 7faaf3400700 1 -- 172.21.15.63:0/687289232 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaf80afdc0 con 0x558d88108d30 2024-05-27T08:19:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.909+0000 7fab1b51db80 1 -- 172.21.15.63:0/687289232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 msgr2=0x558d88102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:24.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.909+0000 7fab1b51db80 1 --2- 172.21.15.63:0/687289232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d88102870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c010 tx=0x7faaf80ab5f0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.912+0000 7fab1b51db80 1 -- 172.21.15.63:0/687289232 shutdown_connections 2024-05-27T08:19:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.912+0000 7fab1b51db80 1 --2- 172.21.15.63:0/687289232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d881031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.912+0000 7fab1b51db80 1 --2- 172.21.15.63:0/687289232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 0x558d881087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.912+0000 7fab1b51db80 1 --2- 172.21.15.63:0/687289232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d88102870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.912+0000 7fab1b51db80 1 -- 172.21.15.63:0/687289232 >> 172.21.15.63:0/687289232 conn(0x558d87fe3fc0 msgr2=0x558d880fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:24.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.915+0000 7fab1b51db80 1 -- 172.21.15.63:0/687289232 shutdown_connections 2024-05-27T08:19:24.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.915+0000 7fab1b51db80 1 -- 172.21.15.63:0/687289232 wait complete. 2024-05-27T08:19:24.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.916+0000 7fab1b51db80 1 -- start start 2024-05-27T08:19:24.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.916+0000 7fab1b51db80 1 Processor -- start 2024-05-27T08:19:24.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.920+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d87f83e40 0x558d8810c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.921+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d88102db0 0x558d8810ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.921+0000 7fab05e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d87f83e40 0x558d8810c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.922+0000 7fab05e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d87f83e40 0x558d8810c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:48946/0 (socket says 172.21.15.63:48946) 2024-05-27T08:19:24.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.922+0000 7fab05e00700 1 -- 172.21.15.63:0/795077865 learned_addr learned my addr 172.21.15.63:0/795077865 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:24.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.922+0000 7fab1b51db80 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.923+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558d87f26fa0 con 0x558d88102db0 2024-05-27T08:19:24.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.923+0000 7fab06800700 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.924+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558d87fc1760 con 0x558d87f83e40 2024-05-27T08:19:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.924+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558d8810f5c0 con 0x558d88108d30 2024-05-27T08:19:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.926+0000 7fab1b51db80 10 client.0 _ll_register_callbacks cb 0x558d87fcac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:19:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.926+0000 7fab06800700 1 -- 172.21.15.63:0/795077865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d87f83e40 msgr2=0x558d8810c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.926+0000 7fab06800700 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d87f83e40 0x558d8810c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.927+0000 7fab06800700 1 -- 172.21.15.63:0/795077865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d88102db0 msgr2=0x558d8810ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:24.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.927+0000 7fab06800700 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d88102db0 0x558d8810ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.927+0000 7fab06800700 1 -- 172.21.15.63:0/795077865 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d87f708d0 con 0x558d88108d30 2024-05-27T08:19:24.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.927+0000 7fab06800700 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810cfb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faaf400e220 tx=0x7faaf400e530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b43f736a9269a25d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:24.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.927+0000 7faae7e00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:19:24.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.928+0000 7faae7e00700 1 -- 172.21.15.63:0/795077865 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4017070 con 0x558d88108d30 2024-05-27T08:19:24.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.928+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf80afdc0 con 0x558d88108d30 2024-05-27T08:19:24.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.928+0000 7faae7e00700 1 -- 172.21.15.63:0/795077865 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7faaf4013470 con 0x558d88108d30 2024-05-27T08:19:24.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.929+0000 7faae7e00700 1 -- 172.21.15.63:0/795077865 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faaf4023050 con 0x558d88108d30 2024-05-27T08:19:24.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.929+0000 7fab1b51db80 10 client.32398 fetch_fsmap learned FSMap version 35 2024-05-27T08:19:24.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.929+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558d87ff1060 con 0x558d88108d30 2024-05-27T08:19:24.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.929+0000 7faae7e00700 1 -- 172.21.15.63:0/795077865 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaf400ee70 con 0x558d88108d30 2024-05-27T08:19:24.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.932+0000 7faae7e00700 1 -- 172.21.15.63:0/795077865 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7faaf40136b0 con 0x558d88108d30 2024-05-27T08:19:24.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.932+0000 7fab1b51db80 10 client.32398 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:19:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.932+0000 7fab1b51db80 10 client.32398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:19:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.932+0000 7fab1b51db80 20 client.32398 populate_metadata read hostname 'smithi063' 2024-05-27T08:19:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.932+0000 7fab1b51db80 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d8811c510 0x558d8811e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.932+0000 7fab1b51db80 4 client.32398 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:19:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.932+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faaf80afdc0 con 0x558d8811c510 2024-05-27T08:19:24.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.933+0000 7fab05e00700 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d8811c510 0x558d8811e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.934+0000 7faae7e00700 10 client.32398 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:24.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.934+0000 7fab05e00700 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d8811c510 0x558d8811e970 crc :-1 s=READY pgs=778 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b92d52f3b8a632c server_cookie=6bac203ab440f719 in_seq=0 out_seq=0 2024-05-27T08:19:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.936+0000 7faae7e00700 1 -- 172.21.15.63:0/795077865 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7faaf80afdc0 con 0x558d8811c510 2024-05-27T08:19:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.936+0000 7faae7e00700 10 client.32398 handle_command_reply: tid=0 2024-05-27T08:19:24.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.936+0000 7fab1b51db80 1 client.32398 shutdown 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.937+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 msgr2=0x558d8810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.937+0000 7fab1b51db80 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810cfb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faaf400e220 tx=0x7faaf400e530 comp rx=0 tx=0).stop 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.937+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 shutdown_connections 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.937+0000 7fab1b51db80 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d8811c510 0x558d8811e970 crc :-1 s=READY pgs=778 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.937+0000 7fab1b51db80 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810cfb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.937+0000 7fab1b51db80 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d87f83e40 0x558d8810c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.937+0000 7fab1b51db80 1 --2- 172.21.15.63:0/795077865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d88102db0 0x558d8810ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.937+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 >> 172.21.15.63:0/795077865 conn(0x558d87fe3fc0 msgr2=0x558d880fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.937+0000 7faae7e00700 0 client.32398 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.939+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 shutdown_connections 2024-05-27T08:19:24.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.939+0000 7fab1b51db80 1 -- 172.21.15.63:0/795077865 wait complete. 2024-05-27T08:19:24.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.939+0000 7fab1b51db80 20 client.32398 trim_cache size 0 max 16384 2024-05-27T08:19:24.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.993+0000 7fab1b51db80 1 Processor -- start 2024-05-27T08:19:24.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.993+0000 7fab1b51db80 1 -- start start 2024-05-27T08:19:24.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.994+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 0x558d88102870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.994+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d88118620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.994+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.994+0000 7fab1b51db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fab081c1d10 con 0x558d87f83e40 2024-05-27T08:19:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.994+0000 7fab1b51db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab081bf5a0 con 0x558d88102db0 2024-05-27T08:19:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.994+0000 7fab1b51db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fab081bd410 con 0x558d88108d30 2024-05-27T08:19:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab06800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab06800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45280/0 (socket says 172.21.15.63:45280) 2024-05-27T08:19:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab06800700 1 -- 172.21.15.63:0/1139443970 learned_addr learned my addr 172.21.15.63:0/1139443970 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab05e00700 1 --2- 172.21.15.63:0/1139443970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 0x558d88102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab05400700 1 --2- 172.21.15.63:0/1139443970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d88118620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:24.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab06800700 1 -- 172.21.15.63:0/1139443970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 msgr2=0x558d88118620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:24.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab06800700 1 --2- 172.21.15.63:0/1139443970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d88118620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab06800700 1 -- 172.21.15.63:0/1139443970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 msgr2=0x558d88102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:24.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab06800700 1 --2- 172.21.15.63:0/1139443970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 0x558d88102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:24.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab06800700 1 -- 172.21.15.63:0/1139443970 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d8810a440 con 0x558d88108d30 2024-05-27T08:19:24.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.995+0000 7fab05e00700 1 --2- 172.21.15.63:0/1139443970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 0x558d88102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:19:24.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.997+0000 7fab06800700 1 --2- 172.21.15.63:0/1139443970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810ca20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faaf4001f70 tx=0x7faaf400e9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed391341c75ddae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:24.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.997+0000 7faaf2a00700 1 -- 172.21.15.63:0/1139443970 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4017050 con 0x558d88108d30 2024-05-27T08:19:25.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.997+0000 7faaf2a00700 1 -- 172.21.15.63:0/1139443970 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaf4014e90 con 0x558d88108d30 2024-05-27T08:19:25.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.997+0000 7faaf2a00700 1 -- 172.21.15.63:0/1139443970 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf401ccf0 con 0x558d88108d30 2024-05-27T08:19:25.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.999+0000 7fab1b51db80 1 -- 172.21.15.63:0/1139443970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 msgr2=0x558d8810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:25.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:24.999+0000 7fab1b51db80 1 --2- 172.21.15.63:0/1139443970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810ca20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faaf4001f70 tx=0x7faaf400e9b0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.001+0000 7fab1b51db80 1 -- 172.21.15.63:0/1139443970 shutdown_connections 2024-05-27T08:19:25.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.001+0000 7fab1b51db80 1 --2- 172.21.15.63:0/1139443970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d88108d30 0x558d8810ca20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.001+0000 7fab1b51db80 1 --2- 172.21.15.63:0/1139443970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d87f83e40 0x558d88102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.001+0000 7fab1b51db80 1 --2- 172.21.15.63:0/1139443970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d88118620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.001+0000 7fab1b51db80 1 -- 172.21.15.63:0/1139443970 >> 172.21.15.63:0/1139443970 conn(0x558d87fe3fc0 msgr2=0x558d8810c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:25.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.002+0000 7fab1b51db80 1 -- 172.21.15.63:0/1139443970 shutdown_connections 2024-05-27T08:19:25.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.002+0000 7fab1b51db80 1 -- 172.21.15.63:0/1139443970 wait complete. 2024-05-27T08:19:25.005 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.003+0000 7fab1b51db80 1 -- start start 2024-05-27T08:19:25.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.003+0000 7fab1b51db80 1 Processor -- start 2024-05-27T08:19:25.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.004+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d87f83e40 0x558d88114f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.004+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d88123a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.004+0000 7fab1b51db80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d88108d30 0x558d88123f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.004+0000 7fab1b51db80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fab081bda60 con 0x558d88108d30 2024-05-27T08:19:25.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.004+0000 7fab1b51db80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab081beed0 con 0x558d88102db0 2024-05-27T08:19:25.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.004+0000 7fab1b51db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fab081be130 con 0x558d87f83e40 2024-05-27T08:19:25.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.004+0000 7fab1b51db80 10 client.0 _ll_register_callbacks cb 0x558d87fcac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:19:25.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.005+0000 7fab05e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d87f83e40 0x558d88114f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.008+0000 7fab05e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d87f83e40 0x558d88114f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45288/0 (socket says 172.21.15.63:45288) 2024-05-27T08:19:25.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.008+0000 7fab05e00700 1 -- 172.21.15.63:0/3310143852 learned_addr learned my addr 172.21.15.63:0/3310143852 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:25.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.005+0000 7fab06800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d88108d30 0x558d88123f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.009+0000 7fab05e00700 1 -- 172.21.15.63:0/3310143852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 msgr2=0x558d88123a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:19:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.009+0000 7fab05e00700 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d88123a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.009+0000 7fab05e00700 1 -- 172.21.15.63:0/3310143852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d88108d30 msgr2=0x558d88123f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.009+0000 7fab05e00700 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d88108d30 0x558d88123f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.009+0000 7fab05e00700 1 -- 172.21.15.63:0/3310143852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d88124710 con 0x558d87f83e40 2024-05-27T08:19:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.010+0000 7fab06800700 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d88108d30 0x558d88123f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:19:25.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.010+0000 7fab05e00700 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d87f83e40 0x558d88114f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab00016b70 tx=0x7fab00004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14f9970e48726cdc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:25.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.010+0000 7faae7e00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:19:25.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.012+0000 7faae7e00700 1 -- 172.21.15.63:0/3310143852 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00023a00 con 0x558d87f83e40 2024-05-27T08:19:25.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.012+0000 7faae7e00700 1 -- 172.21.15.63:0/3310143852 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7fab00023ba0 con 0x558d87f83e40 2024-05-27T08:19:25.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.013+0000 7fab1b51db80 1 -- 172.21.15.63:0/3310143852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf4014e90 con 0x558d87f83e40 2024-05-27T08:19:25.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.022+0000 7faae7e00700 1 -- 172.21.15.63:0/3310143852 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fab00023da0 con 0x558d87f83e40 2024-05-27T08:19:25.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.023+0000 7fab1b51db80 10 client.32404 fetch_fsmap learned FSMap version 35 2024-05-27T08:19:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.023+0000 7fab1b51db80 1 -- 172.21.15.63:0/3310143852 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558d88186fd0 con 0x558d87f83e40 2024-05-27T08:19:25.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.023+0000 7faae7e00700 1 -- 172.21.15.63:0/3310143852 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faaf4014e90 con 0x558d87f83e40 2024-05-27T08:19:25.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.024+0000 7faae7e00700 1 -- 172.21.15.63:0/3310143852 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fab000324a0 con 0x558d87f83e40 2024-05-27T08:19:25.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.024+0000 7fab1b51db80 10 client.32404 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:19:25.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.024+0000 7fab1b51db80 10 client.32404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:19:25.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.024+0000 7fab1b51db80 20 client.32404 populate_metadata read hostname 'smithi063' 2024-05-27T08:19:25.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.024+0000 7fab1b51db80 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d881876c0 0x558d88189ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.024+0000 7fab1b51db80 4 client.32404 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:19:25.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.024+0000 7fab1b51db80 1 -- 172.21.15.63:0/3310143852 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d88103b40 con 0x558d881876c0 2024-05-27T08:19:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.025+0000 7fab05400700 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d881876c0 0x558d88189ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.027+0000 7fab05400700 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d881876c0 0x558d88189ae0 crc :-1 s=READY pgs=779 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c593fc6fd7b7f8 server_cookie=e786e957f66b3b10 in_seq=0 out_seq=0 2024-05-27T08:19:25.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.028+0000 7faae7e00700 10 client.32404 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:25.038 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:19:25.038 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "scrub active (89 inodes in the stack)", 2024-05-27T08:19:25.038 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": { 2024-05-27T08:19:25.038 INFO:teuthology.orchestra.run.smithi063.stdout: "ad7f32b4-aee4-439d-9806-5c01c1448f81": { 2024-05-27T08:19:25.038 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-27T08:19:25.039 INFO:teuthology.orchestra.run.smithi063.stdout: "tag": "ad7f32b4-aee4-439d-9806-5c01c1448f81", 2024-05-27T08:19:25.039 INFO:teuthology.orchestra.run.smithi063.stdout: "options": "recursive,force" 2024-05-27T08:19:25.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:19:25.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:19:25.039 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:19:25.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.029+0000 7faae7e00700 1 -- 172.21.15.63:0/3310143852 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x558d88103b40 con 0x558d881876c0 2024-05-27T08:19:25.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.029+0000 7faae7e00700 10 client.32404 handle_command_reply: tid=0 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.029+0000 7fab1b51db80 1 client.32404 shutdown 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.030+0000 7fab1b51db80 1 -- 172.21.15.63:0/3310143852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d87f83e40 msgr2=0x558d88114f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.030+0000 7fab1b51db80 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d87f83e40 0x558d88114f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab00016b70 tx=0x7fab00004be0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.030+0000 7fab1b51db80 1 -- 172.21.15.63:0/3310143852 shutdown_connections 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.030+0000 7fab1b51db80 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x558d881876c0 0x558d88189ae0 crc :-1 s=READY pgs=779 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.030+0000 7fab1b51db80 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d88108d30 0x558d88123f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.030+0000 7fab1b51db80 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558d87f83e40 0x558d88114f60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.030+0000 7fab1b51db80 1 --2- 172.21.15.63:0/3310143852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d88102db0 0x558d88123a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.030+0000 7fab1b51db80 1 -- 172.21.15.63:0/3310143852 >> 172.21.15.63:0/3310143852 conn(0x558d87fe3fc0 msgr2=0x558d8810c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.032+0000 7faae7e00700 0 client.32404 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.032+0000 7fab1b51db80 1 -- 172.21.15.63:0/3310143852 shutdown_connections 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.032+0000 7fab1b51db80 1 -- 172.21.15.63:0/3310143852 wait complete. 2024-05-27T08:19:25.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.032+0000 7fab1b51db80 20 client.32404 trim_cache size 0 max 16384 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.035+0000 7fab0ca00700 1 -- 172.21.15.63:0/2069707828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faaec0b2bc0 msgr2=0x7faaec0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.035+0000 7fab0ca00700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faaec0b2bc0 0x7faaec0b50d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fab0000a400 tx=0x7fab0000d040 comp rx=0 tx=0).stop 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.035+0000 7fab0ca00700 1 -- 172.21.15.63:0/2069707828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab08108f10 msgr2=0x7fab0819a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.035+0000 7fab0ca00700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab08108f10 0x7fab0819a060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faaf8006010 tx=0x7faaf80057c0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.036+0000 7fab0ca00700 1 -- 172.21.15.63:0/2069707828 shutdown_connections 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.036+0000 7fab0ca00700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7faaec0b2bc0 0x7faaec0b50d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.036+0000 7fab0ca00700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab08109860 0x7fab081a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.036+0000 7fab0ca00700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab081045e0 0x7fab08199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.036+0000 7fab0ca00700 1 --2- 172.21.15.63:0/2069707828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab08108f10 0x7fab0819a060 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.036+0000 7fab0ca00700 1 -- 172.21.15.63:0/2069707828 >> 172.21.15.63:0/2069707828 conn(0x7fab08100030 msgr2=0x7fab0810a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.037+0000 7fab0ca00700 1 -- 172.21.15.63:0/2069707828 shutdown_connections 2024-05-27T08:19:25.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:19:25.037+0000 7fab0ca00700 1 -- 172.21.15.63:0/2069707828 wait complete. 2024-05-27T08:19:25.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:25 smithi086 ceph-mon[34687]: pgmap v1459: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.2 MiB/s wr, 2.33k op/s 2024-05-27T08:19:25.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:25 smithi149 ceph-mon[35408]: pgmap v1459: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.2 MiB/s wr, 2.33k op/s 2024-05-27T08:19:25.471 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:25 smithi063 ceph-mon[31613]: pgmap v1459: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.2 MiB/s wr, 2.33k op/s 2024-05-27T08:19:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.724+0000 7f0ffd800640 1 -- 172.21.15.63:0/704181257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff00a02c0 msgr2=0x7f0ff00a26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.724+0000 7f0ffd800640 1 --2- 172.21.15.63:0/704181257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff00a02c0 0x7f0ff00a26e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009900 tx=0x7f0fe402a440 comp rx=0 tx=0).stop 2024-05-27T08:19:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.724+0000 7f0ffd800640 1 -- 172.21.15.63:0/704181257 shutdown_connections 2024-05-27T08:19:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.724+0000 7f0ffd800640 1 --2- 172.21.15.63:0/704181257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff00a02c0 0x7f0ff00a26e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.726+0000 7f0ffd800640 1 --2- 172.21.15.63:0/704181257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ff0095a80 0x7f0ff0095e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.726+0000 7f0ffd800640 1 --2- 172.21.15.63:0/704181257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ff009b960 0x7f0ff0095540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.726+0000 7f0ffd800640 1 -- 172.21.15.63:0/704181257 >> 172.21.15.63:0/704181257 conn(0x7f0ff000b020 msgr2=0x7f0ff000b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.726+0000 7f0ffd800640 1 -- 172.21.15.63:0/704181257 shutdown_connections 2024-05-27T08:19:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.726+0000 7f0ffd800640 1 -- 172.21.15.63:0/704181257 wait complete. 2024-05-27T08:19:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.728+0000 7f0ffd800640 1 Processor -- start 2024-05-27T08:19:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.728+0000 7f0ffd800640 1 -- start start 2024-05-27T08:19:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.729+0000 7f0ffd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff0095a80 0x7f0ff013c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.729+0000 7f0ffd800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ff009b960 0x7f0ff013c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.729+0000 7f0ffd800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ff00a02c0 0x7f0ff0140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.729+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff0095a80 0x7f0ff013c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.729+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff0095a80 0x7f0ff013c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:48972/0 (socket says 172.21.15.63:48972) 2024-05-27T08:19:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.730+0000 7f0ff6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ff009b960 0x7f0ff013c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.730+0000 7f0ff6000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ff009b960 0x7f0ff013c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45298/0 (socket says 172.21.15.63:45298) 2024-05-27T08:19:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.731+0000 7f0ff6a00640 1 -- 172.21.15.63:0/4153136322 learned_addr learned my addr 172.21.15.63:0/4153136322 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.731+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff00a48e0 con 0x7f0ff00a02c0 2024-05-27T08:19:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.731+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff00a4760 con 0x7f0ff0095a80 2024-05-27T08:19:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.731+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff00a4a60 con 0x7f0ff009b960 2024-05-27T08:19:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.731+0000 7f0ff7400640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ff00a02c0 0x7f0ff0140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.732+0000 7f0ff6a00640 1 -- 172.21.15.63:0/4153136322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ff009b960 msgr2=0x7f0ff013c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.732+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ff009b960 0x7f0ff013c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.732+0000 7f0ff6a00640 1 -- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ff00a02c0 msgr2=0x7f0ff0140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.732+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ff00a02c0 0x7f0ff0140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.732+0000 7f0ff6a00640 1 -- 172.21.15.63:0/4153136322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff01410f0 con 0x7f0ff0095a80 2024-05-27T08:19:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.733+0000 7f0ff6000640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ff009b960 0x7f0ff013c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:19:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.733+0000 7f0ff7400640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ff00a02c0 0x7f0ff0140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:19:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.736+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff0095a80 0x7f0ff013c290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8052210 tx=0x7f0ff807aaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.739+0000 7f0fef400640 1 -- 172.21.15.63:0/4153136322 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff806b3e0 con 0x7f0ff0095a80 2024-05-27T08:19:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.739+0000 7f0fef400640 1 -- 172.21.15.63:0/4153136322 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ff806b580 con 0x7f0ff0095a80 2024-05-27T08:19:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.739+0000 7f0fef400640 1 -- 172.21.15.63:0/4153136322 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8079a50 con 0x7f0ff0095a80 2024-05-27T08:19:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.740+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff01413e0 con 0x7f0ff0095a80 2024-05-27T08:19:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.745+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff01418b0 con 0x7f0ff0095a80 2024-05-27T08:19:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.768+0000 7f1004831740 1 Processor -- start 2024-05-27T08:19:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.768+0000 7f0fef400640 1 -- 172.21.15.63:0/4153136322 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0ff8079bf0 con 0x7f0ff0095a80 2024-05-27T08:19:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.772+0000 7f1004831740 1 -- start start 2024-05-27T08:19:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.774+0000 7f1004831740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 0x55ea90f14a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.774+0000 7f1004831740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 0x55ea90f14fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.774+0000 7f1004831740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b514d0 0x55ea90f15510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.774+0000 7f1004831740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ea90e85b90 con 0x55ea90b514d0 2024-05-27T08:19:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.774+0000 7f1004831740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea90e75730 con 0x55ea90b721b0 2024-05-27T08:19:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.774+0000 7f1004831740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea90e79770 con 0x55ea90b492f0 2024-05-27T08:19:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.774+0000 7f0fef400640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fdc0b2ab0 0x7f0fdc0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.775+0000 7f0fef400640 1 -- 172.21.15.63:0/4153136322 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7f0ff8152170 con 0x7f0ff0095a80 2024-05-27T08:19:25.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.776+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 0x55ea90f14a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.776+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 0x55ea90f14a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:48980/0 (socket says 172.21.15.63:48980) 2024-05-27T08:19:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.776+0000 7f0ff6a00640 1 -- 172.21.15.63:0/3301857545 learned_addr learned my addr 172.21.15.63:0/3301857545 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.777+0000 7f0ff6a00640 1 -- 172.21.15.63:0/3301857545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 msgr2=0x55ea90f14fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.777+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/3301857545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 0x55ea90f14fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.777+0000 7f0ff6a00640 1 -- 172.21.15.63:0/3301857545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b514d0 msgr2=0x55ea90f15510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.777+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/3301857545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b514d0 0x55ea90f15510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.777+0000 7f0ff6a00640 1 -- 172.21.15.63:0/3301857545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea90e0e8b0 con 0x55ea90b721b0 2024-05-27T08:19:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.778+0000 7f0ff7400640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fdc0b2ab0 0x7f0fdc0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.778+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/3301857545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 0x55ea90f14a90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8065d80 tx=0x7f0ff8064240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=541bbc0a8140e30 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.780+0000 7f0fe3e00640 1 -- 172.21.15.63:0/3301857545 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8153060 con 0x55ea90b721b0 2024-05-27T08:19:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.780+0000 7f0fe3e00640 1 -- 172.21.15.63:0/3301857545 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ff811e450 con 0x55ea90b721b0 2024-05-27T08:19:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.780+0000 7f0fe3e00640 1 -- 172.21.15.63:0/3301857545 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff811eb00 con 0x55ea90b721b0 2024-05-27T08:19:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.782+0000 7f1004831740 1 -- 172.21.15.63:0/3301857545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 msgr2=0x55ea90f14a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.782+0000 7f1004831740 1 --2- 172.21.15.63:0/3301857545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 0x55ea90f14a90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8065d80 tx=0x7f0ff8064240 comp rx=0 tx=0).stop 2024-05-27T08:19:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.783+0000 7f0ff7400640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fdc0b2ab0 0x7f0fdc0b4fc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4005ab0 tx=0x7f0fe4005a20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.784+0000 7f1004831740 1 -- 172.21.15.63:0/3301857545 shutdown_connections 2024-05-27T08:19:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.785+0000 7f1004831740 1 --2- 172.21.15.63:0/3301857545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b514d0 0x55ea90f15510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.786+0000 7f1004831740 1 --2- 172.21.15.63:0/3301857545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 0x55ea90f14fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.786+0000 7f1004831740 1 --2- 172.21.15.63:0/3301857545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 0x55ea90f14a90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.786+0000 7f1004831740 1 -- 172.21.15.63:0/3301857545 >> 172.21.15.63:0/3301857545 conn(0x55ea90b66300 msgr2=0x55ea90f11240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.786+0000 7f1004831740 1 -- 172.21.15.63:0/3301857545 shutdown_connections 2024-05-27T08:19:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.786+0000 7f1004831740 1 -- 172.21.15.63:0/3301857545 wait complete. 2024-05-27T08:19:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.787+0000 7f1004831740 1 -- start start 2024-05-27T08:19:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.789+0000 7f1004831740 1 Processor -- start 2024-05-27T08:19:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.791+0000 7f1004831740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 0x55ea90f1eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.792+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 0x55ea90f1eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.792+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 0x55ea90f1eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45322/0 (socket says 172.21.15.63:45322) 2024-05-27T08:19:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.792+0000 7f0ff6a00640 1 -- 172.21.15.63:0/256486631 learned_addr learned my addr 172.21.15.63:0/256486631 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.793+0000 7f1004831740 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b514d0 0x55ea90f1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.793+0000 7f1004831740 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 0x55ea90f2af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.794+0000 7f0ff7400640 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 0x55ea90f2af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.794+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ea90b5b790 con 0x55ea90b514d0 2024-05-27T08:19:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.794+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea90e1dbc0 con 0x55ea90b721b0 2024-05-27T08:19:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.794+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ea90f21bc0 con 0x55ea90b492f0 2024-05-27T08:19:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.794+0000 7f1004831740 10 client.0 _ll_register_callbacks cb 0x55ea90dfb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:19:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.796+0000 7f0ff6000640 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b514d0 0x55ea90f1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.796+0000 7f0ff6a00640 1 -- 172.21.15.63:0/256486631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 msgr2=0x55ea90f2af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.796+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 0x55ea90f2af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.796+0000 7f0ff6a00640 1 -- 172.21.15.63:0/256486631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b514d0 msgr2=0x55ea90f1f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.797+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b514d0 0x55ea90f1f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.797+0000 7f0ff6a00640 1 -- 172.21.15.63:0/256486631 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea90b8a210 con 0x55ea90b492f0 2024-05-27T08:19:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.798+0000 7f0ff6000640 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b514d0 0x55ea90f1f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:19:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.800+0000 7f0fe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:19:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.800+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 0x55ea90f1eef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8116fd0 tx=0x7f0ff8115130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e9b1e64998c886 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.802+0000 7f0fe0c00640 1 -- 172.21.15.63:0/256486631 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8135060 con 0x55ea90b492f0 2024-05-27T08:19:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.802+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff811e450 con 0x55ea90b492f0 2024-05-27T08:19:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.803+0000 7f0fe0c00640 1 -- 172.21.15.63:0/256486631 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7f0ff8112290 con 0x55ea90b492f0 2024-05-27T08:19:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.806+0000 7f0fe0c00640 1 -- 172.21.15.63:0/256486631 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0ff8135060 con 0x55ea90b492f0 2024-05-27T08:19:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.806+0000 7f1004831740 10 client.32413 fetch_fsmap learned FSMap version 35 2024-05-27T08:19:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.806+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ea90c56620 con 0x55ea90b492f0 2024-05-27T08:19:25.808 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.807+0000 7f0fe0c00640 1 -- 172.21.15.63:0/256486631 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ff811e450 con 0x55ea90b492f0 2024-05-27T08:19:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.811+0000 7f0fe0c00640 1 -- 172.21.15.63:0/256486631 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0ff811b1e0 con 0x55ea90b492f0 2024-05-27T08:19:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.811+0000 7f1004831740 10 client.32413 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:19:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.811+0000 7f1004831740 10 client.32413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:19:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.811+0000 7f1004831740 20 client.32413 populate_metadata read hostname 'smithi063' 2024-05-27T08:19:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.812+0000 7f1004831740 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea90b30d80 0x55ea90f30e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.812+0000 7f1004831740 4 client.32413 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:19:25.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.812+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea90e87ae0 con 0x55ea90b30d80 2024-05-27T08:19:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.812+0000 7f0ff6000640 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea90b30d80 0x55ea90f30e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.814+0000 7f0ff6000640 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea90b30d80 0x55ea90f30e50 crc :-1 s=READY pgs=780 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f87b96a312121d server_cookie=76ee8cccf3035c7f in_seq=0 out_seq=0 2024-05-27T08:19:25.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.814+0000 7f0fe0c00640 10 client.32413 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.815+0000 7f0fe0c00640 1 -- 172.21.15.63:0/256486631 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ea90e87ae0 con 0x55ea90b30d80 2024-05-27T08:19:25.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.815+0000 7f0fe0c00640 10 client.32413 handle_command_reply: tid=0 2024-05-27T08:19:25.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.815+0000 7f1004831740 1 client.32413 shutdown 2024-05-27T08:19:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.817+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 msgr2=0x55ea90f1eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.817+0000 7f1004831740 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 0x55ea90f1eef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8116fd0 tx=0x7f0ff8115130 comp rx=0 tx=0).stop 2024-05-27T08:19:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.818+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 shutdown_connections 2024-05-27T08:19:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.818+0000 7f1004831740 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea90b30d80 0x55ea90f30e50 crc :-1 s=READY pgs=780 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.818+0000 7f1004831740 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b721b0 0x55ea90f2af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.818+0000 7f0fe0c00640 0 client.32413 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.818+0000 7f1004831740 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b514d0 0x55ea90f1f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.818+0000 7f1004831740 1 --2- 172.21.15.63:0/256486631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b492f0 0x55ea90f1eef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.818+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 >> 172.21.15.63:0/256486631 conn(0x55ea90b66300 msgr2=0x55ea90f14a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.819+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 shutdown_connections 2024-05-27T08:19:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.820+0000 7f1004831740 1 -- 172.21.15.63:0/256486631 wait complete. 2024-05-27T08:19:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.820+0000 7f1004831740 20 client.32413 trim_cache size 0 max 16384 2024-05-27T08:19:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.862+0000 7f1004831740 1 Processor -- start 2024-05-27T08:19:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.863+0000 7f1004831740 1 -- start start 2024-05-27T08:19:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.863+0000 7f1004831740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b30d80 0x55ea90f1f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.863+0000 7f1004831740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 0x55ea90f33df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.863+0000 7f1004831740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b514d0 0x55ea90f34330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.863+0000 7f1004831740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0166560 con 0x55ea90b492f0 2024-05-27T08:19:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.863+0000 7f1004831740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0139ef0 con 0x55ea90b514d0 2024-05-27T08:19:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.863+0000 7f1004831740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0096c50 con 0x55ea90b30d80 2024-05-27T08:19:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.864+0000 7f0ff7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b514d0 0x55ea90f34330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.866+0000 7f0ff7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b514d0 0x55ea90f34330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:49004/0 (socket says 172.21.15.63:49004) 2024-05-27T08:19:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.866+0000 7f0ff7400640 1 -- 172.21.15.63:0/3945515503 learned_addr learned my addr 172.21.15.63:0/3945515503 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.866+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b30d80 0x55ea90f1f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.869+0000 7f0ff7400640 1 -- 172.21.15.63:0/3945515503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b30d80 msgr2=0x55ea90f1f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.869+0000 7f0ff6000640 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 0x55ea90f33df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.869+0000 7f0ff7400640 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b30d80 0x55ea90f1f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.869+0000 7f0ff7400640 1 -- 172.21.15.63:0/3945515503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 msgr2=0x55ea90f33df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.869+0000 7f0ff7400640 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 0x55ea90f33df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.869+0000 7f0ff7400640 1 -- 172.21.15.63:0/3945515503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea90f0f570 con 0x55ea90b514d0 2024-05-27T08:19:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.869+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b30d80 0x55ea90f1f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:19:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.869+0000 7f0ff6000640 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 0x55ea90f33df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:19:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.870+0000 7f0ff7400640 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b514d0 0x55ea90f34330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009900 tx=0x7f0fe403e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8921c9c7ee41b6f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.872+0000 7f0fe3400640 1 -- 172.21.15.63:0/3945515503 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4031b90 con 0x55ea90b514d0 2024-05-27T08:19:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.872+0000 7f0fe3400640 1 -- 172.21.15.63:0/3945515503 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fe404f070 con 0x55ea90b514d0 2024-05-27T08:19:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.872+0000 7f0fe3400640 1 -- 172.21.15.63:0/3945515503 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe404b430 con 0x55ea90b514d0 2024-05-27T08:19:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.876+0000 7f1004831740 1 -- 172.21.15.63:0/3945515503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b514d0 msgr2=0x55ea90f34330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.876+0000 7f1004831740 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b514d0 0x55ea90f34330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009900 tx=0x7f0fe403e000 comp rx=0 tx=0).stop 2024-05-27T08:19:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.877+0000 7f1004831740 1 -- 172.21.15.63:0/3945515503 shutdown_connections 2024-05-27T08:19:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.877+0000 7f1004831740 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b514d0 0x55ea90f34330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.877+0000 7f1004831740 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 0x55ea90f33df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.877+0000 7f1004831740 1 --2- 172.21.15.63:0/3945515503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b30d80 0x55ea90f1f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.877+0000 7f1004831740 1 -- 172.21.15.63:0/3945515503 >> 172.21.15.63:0/3945515503 conn(0x55ea90b66300 msgr2=0x55ea90f19420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.879+0000 7f1004831740 1 -- 172.21.15.63:0/3945515503 shutdown_connections 2024-05-27T08:19:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.880+0000 7f1004831740 1 -- 172.21.15.63:0/3945515503 wait complete. 2024-05-27T08:19:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.881+0000 7f1004831740 1 -- start start 2024-05-27T08:19:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.882+0000 7f1004831740 1 Processor -- start 2024-05-27T08:19:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.886+0000 7f1004831740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b30d80 0x55ea90f32e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.887+0000 7f1004831740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 0x55ea90f333c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.887+0000 7f1004831740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b514d0 0x55ea90f33940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.888+0000 7f0ff7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b514d0 0x55ea90f33940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.889+0000 7f0ff7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b514d0 0x55ea90f33940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45334/0 (socket says 172.21.15.63:45334) 2024-05-27T08:19:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.889+0000 7f0ff7400640 1 -- 172.21.15.63:0/2773129553 learned_addr learned my addr 172.21.15.63:0/2773129553 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.889+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01383e0 con 0x55ea90b492f0 2024-05-27T08:19:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.890+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b30d80 0x55ea90f32e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.890+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0139820 con 0x55ea90b30d80 2024-05-27T08:19:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.890+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0138a80 con 0x55ea90b514d0 2024-05-27T08:19:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.890+0000 7f1004831740 10 client.0 _ll_register_callbacks cb 0x7f0ff8151070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:19:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.891+0000 7f0ff6a00640 1 -- 172.21.15.63:0/2773129553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b514d0 msgr2=0x55ea90f33940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.891+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b514d0 0x55ea90f33940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.891+0000 7f0ff6000640 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 0x55ea90f333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.891+0000 7f0ff6a00640 1 -- 172.21.15.63:0/2773129553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 msgr2=0x55ea90f333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.895+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 0x55ea90f333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.895+0000 7f0ff6a00640 1 -- 172.21.15.63:0/2773129553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea90f97660 con 0x55ea90b30d80 2024-05-27T08:19:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.895+0000 7f0ff6000640 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 0x55ea90f333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:19:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.897+0000 7f0fe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:19:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.897+0000 7f0ff6a00640 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b30d80 0x55ea90f32e80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8088ff0 tx=0x7f0ff8050f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9362773d7d7d5300 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.897+0000 7f0fe0c00640 1 -- 172.21.15.63:0/2773129553 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8111430 con 0x55ea90b30d80 2024-05-27T08:19:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.897+0000 7f0fe0c00640 1 -- 172.21.15.63:0/2773129553 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7f0ff807a040 con 0x55ea90b30d80 2024-05-27T08:19:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.897+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe404f070 con 0x55ea90b30d80 2024-05-27T08:19:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.898+0000 7f0fe0c00640 1 -- 172.21.15.63:0/2773129553 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0ff8130390 con 0x55ea90b30d80 2024-05-27T08:19:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.898+0000 7f1004831740 10 client.42599 fetch_fsmap learned FSMap version 35 2024-05-27T08:19:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.898+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ea90f98860 con 0x55ea90b30d80 2024-05-27T08:19:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.899+0000 7f0fe0c00640 1 -- 172.21.15.63:0/2773129553 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0fe404f070 con 0x55ea90b30d80 2024-05-27T08:19:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.903+0000 7f0fe0c00640 1 -- 172.21.15.63:0/2773129553 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0ff814e0b0 con 0x55ea90b30d80 2024-05-27T08:19:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.903+0000 7f1004831740 10 client.42599 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:19:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.903+0000 7f1004831740 10 client.42599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:19:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.903+0000 7f1004831740 20 client.42599 populate_metadata read hostname 'smithi063' 2024-05-27T08:19:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.903+0000 7f1004831740 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea90b721b0 0x55ea90f9ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.903+0000 7f1004831740 4 client.42599 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:19:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.903+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea90e812c0 con 0x55ea90b721b0 2024-05-27T08:19:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.903+0000 7f0ff6000640 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea90b721b0 0x55ea90f9ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.912+0000 7f0ff6000640 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea90b721b0 0x55ea90f9ad00 crc :-1 s=READY pgs=781 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef0cc888a485db7 server_cookie=ce07c0b205dc2d47 in_seq=0 out_seq=0 2024-05-27T08:19:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:25.912+0000 7f0fe0c00640 10 client.42599 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:26.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:26 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:19:26.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:26 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:19:26.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:26 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:19:26.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:26 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:19:26.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:26 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:19:26.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:26 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:19:26.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:26 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:19:26.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:26 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:19:26.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:26 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:19:27.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:27 smithi086 ceph-mon[34687]: pgmap v1460: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.0 MiB/s wr, 2.27k op/s 2024-05-27T08:19:27.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:27 smithi149 ceph-mon[35408]: pgmap v1460: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.0 MiB/s wr, 2.27k op/s 2024-05-27T08:19:27.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:27 smithi063 ceph-mon[31613]: pgmap v1460: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.0 MiB/s wr, 2.27k op/s 2024-05-27T08:19:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.656+0000 7f0fe0c00640 1 -- 172.21.15.63:0/2773129553 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ea90e812c0 con 0x55ea90b721b0 2024-05-27T08:19:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.656+0000 7f0fe0c00640 10 client.42599 handle_command_reply: tid=0 2024-05-27T08:19:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.657+0000 7f1004831740 1 client.42599 shutdown 2024-05-27T08:19:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.657+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b30d80 msgr2=0x55ea90f32e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.657+0000 7f1004831740 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b30d80 0x55ea90f32e80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8088ff0 tx=0x7f0ff8050f60 comp rx=0 tx=0).stop 2024-05-27T08:19:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.660+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 shutdown_connections 2024-05-27T08:19:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.660+0000 7f1004831740 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55ea90b721b0 0x55ea90f9ad00 crc :-1 s=READY pgs=781 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.660+0000 7f1004831740 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea90b514d0 0x55ea90f33940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.660+0000 7f1004831740 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ea90b492f0 0x55ea90f333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.660+0000 7f1004831740 1 --2- 172.21.15.63:0/2773129553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea90b30d80 0x55ea90f32e80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.660+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 >> 172.21.15.63:0/2773129553 conn(0x55ea90b66300 msgr2=0x55ea90f19df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.660+0000 7f0fe0c00640 0 client.42599 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.660+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 shutdown_connections 2024-05-27T08:19:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.661+0000 7f1004831740 1 -- 172.21.15.63:0/2773129553 wait complete. 2024-05-27T08:19:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.662+0000 7f1004831740 20 client.42599 trim_cache size 0 max 16384 2024-05-27T08:19:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:19:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:19:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:19:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.666+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fdc0b2ab0 msgr2=0x7f0fdc0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.667+0000 7f0ffd800640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fdc0b2ab0 0x7f0fdc0b4fc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4005ab0 tx=0x7f0fe4005a20 comp rx=0 tx=0).stop 2024-05-27T08:19:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.667+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff0095a80 msgr2=0x7f0ff013c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.667+0000 7f0ffd800640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff0095a80 0x7f0ff013c290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8052210 tx=0x7f0ff807aaf0 comp rx=0 tx=0).stop 2024-05-27T08:19:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.668+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 shutdown_connections 2024-05-27T08:19:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.668+0000 7f0ffd800640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0fdc0b2ab0 0x7f0fdc0b4fc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.668+0000 7f0ffd800640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ff00a02c0 0x7f0ff0140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.668+0000 7f0ffd800640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ff009b960 0x7f0ff013c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.668+0000 7f0ffd800640 1 --2- 172.21.15.63:0/4153136322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff0095a80 0x7f0ff013c290 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.668+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 >> 172.21.15.63:0/4153136322 conn(0x7f0ff000b020 msgr2=0x7f0ff009c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.669+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 shutdown_connections 2024-05-27T08:19:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:27.670+0000 7f0ffd800640 1 -- 172.21.15.63:0/4153136322 wait complete. 2024-05-27T08:19:28.080 INFO:tasks.cephfs.filesystem:scrub status for tag:ad7f32b4-aee4-439d-9806-5c01c1448f81 - {'path': '/', 'tag': 'ad7f32b4-aee4-439d-9806-5c01c1448f81', 'options': 'recursive,force'} 2024-05-27T08:19:29.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:29 smithi086 ceph-mon[34687]: pgmap v1461: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 10 MiB/s wr, 2.58k op/s 2024-05-27T08:19:29.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:29 smithi149 ceph-mon[35408]: pgmap v1461: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 10 MiB/s wr, 2.58k op/s 2024-05-27T08:19:29.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:29 smithi063 ceph-mon[31613]: pgmap v1461: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 10 MiB/s wr, 2.58k op/s 2024-05-27T08:19:31.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:31 smithi086 ceph-mon[34687]: pgmap v1462: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.7 MiB/s wr, 1.95k op/s 2024-05-27T08:19:31.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:31 smithi149 ceph-mon[35408]: pgmap v1462: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.7 MiB/s wr, 1.95k op/s 2024-05-27T08:19:31.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:31 smithi063 ceph-mon[31613]: pgmap v1462: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.7 MiB/s wr, 1.95k op/s 2024-05-27T08:19:33.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:33 smithi086 ceph-mon[34687]: pgmap v1463: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-27T08:19:33.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:33 smithi149 ceph-mon[35408]: pgmap v1463: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-27T08:19:33.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:33 smithi063 ceph-mon[31613]: pgmap v1463: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 2.53k op/s 2024-05-27T08:19:35.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:35 smithi086 ceph-mon[34687]: pgmap v1464: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.6 MiB/s wr, 2.19k op/s 2024-05-27T08:19:35.466 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:35 smithi149 ceph-mon[35408]: pgmap v1464: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.6 MiB/s wr, 2.19k op/s 2024-05-27T08:19:35.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:35 smithi063 ceph-mon[31613]: pgmap v1464: 129 pgs: 19 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 69 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.6 MiB/s wr, 2.19k op/s 2024-05-27T08:19:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:37 smithi086 ceph-mon[34687]: pgmap v1465: 129 pgs: 18 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 70 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-27T08:19:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:37 smithi149 ceph-mon[35408]: pgmap v1465: 129 pgs: 18 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 70 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-27T08:19:37.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:37 smithi063 ceph-mon[31613]: pgmap v1465: 129 pgs: 18 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 70 active+clean; 15 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-27T08:19:39.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:39 smithi086 ceph-mon[34687]: pgmap v1466: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.9 MiB/s wr, 2.49k op/s 2024-05-27T08:19:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:39 smithi149 ceph-mon[35408]: pgmap v1466: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.9 MiB/s wr, 2.49k op/s 2024-05-27T08:19:39.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:39 smithi063 ceph-mon[31613]: pgmap v1466: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.9 MiB/s wr, 2.49k op/s 2024-05-27T08:19:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.147+0000 7f1671200640 1 -- 172.21.15.63:0/4019349106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f166c112900 msgr2=0x7f166c118eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.147+0000 7f1671200640 1 --2- 172.21.15.63:0/4019349106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f166c112900 0x7f166c118eb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f165c009860 tx=0x7f165c02a3d0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.147+0000 7f1671200640 1 -- 172.21.15.63:0/4019349106 shutdown_connections 2024-05-27T08:19:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.147+0000 7f1671200640 1 --2- 172.21.15.63:0/4019349106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f166c112900 0x7f166c118eb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.147+0000 7f1671200640 1 --2- 172.21.15.63:0/4019349106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166c111de0 0x7f166c112270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.147+0000 7f1671200640 1 --2- 172.21.15.63:0/4019349106 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f166c10a490 0x7f166c10a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.147+0000 7f1671200640 1 -- 172.21.15.63:0/4019349106 >> 172.21.15.63:0/4019349106 conn(0x7f166c06e8f0 msgr2=0x7f166c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.147+0000 7f1671200640 1 -- 172.21.15.63:0/4019349106 shutdown_connections 2024-05-27T08:19:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.147+0000 7f1671200640 1 -- 172.21.15.63:0/4019349106 wait complete. 2024-05-27T08:19:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.149+0000 7f1671200640 1 Processor -- start 2024-05-27T08:19:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.152+0000 7f1671200640 1 -- start start 2024-05-27T08:19:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.152+0000 7f1671200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166c10a490 0x7f166c1a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.152+0000 7f1671200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f166c111de0 0x7f166c1a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.152+0000 7f1671200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f166c112900 0x7f166c1a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.152+0000 7f1671200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f166c11c4f0 con 0x7f166c111de0 2024-05-27T08:19:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.152+0000 7f1671200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f166c11c370 con 0x7f166c10a490 2024-05-27T08:19:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.152+0000 7f1671200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f166c11c670 con 0x7f166c112900 2024-05-27T08:19:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.153+0000 7f1669e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166c10a490 0x7f166c1a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.153+0000 7f1669e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166c10a490 0x7f166c1a2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:42396/0 (socket says 172.21.15.63:42396) 2024-05-27T08:19:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.153+0000 7f1669e00640 1 -- 172.21.15.63:0/2643611366 learned_addr learned my addr 172.21.15.63:0/2643611366 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.154+0000 7f166a800640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f166c112900 0x7f166c1a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.154+0000 7f1669e00640 1 -- 172.21.15.63:0/2643611366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f166c112900 msgr2=0x7f166c1a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.155+0000 7f1669400640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f166c111de0 0x7f166c1a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.155+0000 7f1669e00640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f166c112900 0x7f166c1a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.155+0000 7f1669e00640 1 -- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f166c111de0 msgr2=0x7f166c1a34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.155+0000 7f1669e00640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f166c111de0 0x7f166c1a34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.155+0000 7f1669e00640 1 -- 172.21.15.63:0/2643611366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c1a7db0 con 0x7f166c10a490 2024-05-27T08:19:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.155+0000 7f166a800640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f166c112900 0x7f166c1a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:19:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.155+0000 7f1669400640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f166c111de0 0x7f166c1a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:19:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.160+0000 7f1669e00640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166c10a490 0x7f166c1a2f50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f165800e9c0 tx=0x7f165800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.162+0000 7f1662a00640 1 -- 172.21.15.63:0/2643611366 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165800ccc0 con 0x7f166c10a490 2024-05-27T08:19:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.162+0000 7f1662a00640 1 -- 172.21.15.63:0/2643611366 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16580054b0 con 0x7f166c10a490 2024-05-27T08:19:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.162+0000 7f1662a00640 1 -- 172.21.15.63:0/2643611366 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658010430 con 0x7f166c10a490 2024-05-27T08:19:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.162+0000 7f1671200640 1 -- 172.21.15.63:0/2643611366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c1a80a0 con 0x7f166c10a490 2024-05-27T08:19:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.162+0000 7f1671200640 1 -- 172.21.15.63:0/2643611366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c1afa20 con 0x7f166c10a490 2024-05-27T08:19:41.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.180+0000 7f1678194740 1 Processor -- start 2024-05-27T08:19:41.183 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.180+0000 7f1678194740 1 -- start start 2024-05-27T08:19:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.183+0000 7f1678194740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f3381a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.185+0000 7f1678194740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd02f0 0x55f3381a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.185+0000 7f1678194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.185+0000 7f1678194740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f33810cb90 con 0x55f337df91b0 2024-05-27T08:19:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.185+0000 7f1678194740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3380fc730 con 0x55f337dd02f0 2024-05-27T08:19:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.185+0000 7f1678194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f338100770 con 0x55f337dd84d0 2024-05-27T08:19:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.186+0000 7f1669400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.186+0000 7f1669400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45348/0 (socket says 172.21.15.63:45348) 2024-05-27T08:19:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.186+0000 7f1669400640 1 -- 172.21.15.63:0/1459351208 learned_addr learned my addr 172.21.15.63:0/1459351208 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.186+0000 7f1662a00640 1 -- 172.21.15.63:0/2643611366 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f16580105d0 con 0x7f166c10a490 2024-05-27T08:19:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.187+0000 7f1669e00640 1 --2- 172.21.15.63:0/1459351208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd02f0 0x55f3381a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.188+0000 7f1669e00640 1 -- 172.21.15.63:0/1459351208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 msgr2=0x55f3381a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.188+0000 7f1662a00640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f16500b2aa0 0x7f16500b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.189+0000 7f1669e00640 1 --2- 172.21.15.63:0/1459351208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.189+0000 7f1669e00640 1 -- 172.21.15.63:0/1459351208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 msgr2=0x55f3381a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.189+0000 7f1669e00640 1 --2- 172.21.15.63:0/1459351208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f3381a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.189+0000 7f1669e00640 1 -- 172.21.15.63:0/1459351208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3380958b0 con 0x55f337dd02f0 2024-05-27T08:19:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.189+0000 7f1669400640 1 --2- 172.21.15.63:0/1459351208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:19:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.190+0000 7f166a800640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f16500b2aa0 0x7f16500b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.190+0000 7f1669e00640 1 --2- 172.21.15.63:0/1459351208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd02f0 0x55f3381a13c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16580dff20 tx=0x7f16580a5970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91b0754e690b752d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.190+0000 7f1657400640 1 -- 172.21.15.63:0/1459351208 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580a5420 con 0x55f337dd02f0 2024-05-27T08:19:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.191+0000 7f1657400640 1 -- 172.21.15.63:0/1459351208 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16580f0070 con 0x55f337dd02f0 2024-05-27T08:19:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.191+0000 7f1657400640 1 -- 172.21.15.63:0/1459351208 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580f56d0 con 0x55f337dd02f0 2024-05-27T08:19:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.193+0000 7f1678194740 1 -- 172.21.15.63:0/1459351208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd02f0 msgr2=0x55f3381a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.193+0000 7f1678194740 1 --2- 172.21.15.63:0/1459351208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd02f0 0x55f3381a13c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16580dff20 tx=0x7f16580a5970 comp rx=0 tx=0).stop 2024-05-27T08:19:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.193+0000 7f1662a00640 1 -- 172.21.15.63:0/2643611366 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7f16580e2c40 con 0x7f166c10a490 2024-05-27T08:19:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.193+0000 7f166a800640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f16500b2aa0 0x7f16500b4fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f165c009830 tx=0x7f165c005780 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.196+0000 7f1678194740 1 -- 172.21.15.63:0/1459351208 shutdown_connections 2024-05-27T08:19:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.196+0000 7f1678194740 1 --2- 172.21.15.63:0/1459351208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.196+0000 7f1678194740 1 --2- 172.21.15.63:0/1459351208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd02f0 0x55f3381a13c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.196+0000 7f1678194740 1 --2- 172.21.15.63:0/1459351208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f3381a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.196+0000 7f1678194740 1 -- 172.21.15.63:0/1459351208 >> 172.21.15.63:0/1459351208 conn(0x55f337ded300 msgr2=0x55f3381979b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.197+0000 7f1678194740 1 -- 172.21.15.63:0/1459351208 shutdown_connections 2024-05-27T08:19:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.198+0000 7f1678194740 1 -- 172.21.15.63:0/1459351208 wait complete. 2024-05-27T08:19:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.199+0000 7f1678194740 1 -- start start 2024-05-27T08:19:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.200+0000 7f1678194740 1 Processor -- start 2024-05-27T08:19:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.204+0000 7f1678194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.204+0000 7f1678194740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd84d0 0x55f3381bd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.204+0000 7f1678194740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f3381b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.205+0000 7f1669e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.205+0000 7f1669e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381bccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:45358/0 (socket says 172.21.15.63:45358) 2024-05-27T08:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.205+0000 7f1669e00640 1 -- 172.21.15.63:0/440964962 learned_addr learned my addr 172.21.15.63:0/440964962 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.205+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f337de2790 con 0x55f337df91b0 2024-05-27T08:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.205+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3380a4bc0 con 0x55f337dd84d0 2024-05-27T08:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.205+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f3381a0230 con 0x55f337dd02f0 2024-05-27T08:19:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.206+0000 7f1669400640 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd84d0 0x55f3381bd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.206+0000 7f1678194740 10 client.0 _ll_register_callbacks cb 0x55f338082400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:19:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.206+0000 7f1669e00640 1 -- 172.21.15.63:0/440964962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd84d0 msgr2=0x55f3381bd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.206+0000 7f1669e00640 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd84d0 0x55f3381bd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.206+0000 7f1669e00640 1 -- 172.21.15.63:0/440964962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 msgr2=0x55f3381b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.206+0000 7f1669e00640 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f3381b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.206+0000 7f1669e00640 1 -- 172.21.15.63:0/440964962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f337e11210 con 0x55f337dd02f0 2024-05-27T08:19:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.207+0000 7f1669e00640 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381bccb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f16580dffd0 tx=0x7f16580a5aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed95e2fc607e9854 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.207+0000 7f164be00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:19:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.207+0000 7f164be00640 1 -- 172.21.15.63:0/440964962 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580a9880 con 0x55f337dd02f0 2024-05-27T08:19:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.207+0000 7f164be00640 1 -- 172.21.15.63:0/440964962 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7f16580a9d10 con 0x55f337dd02f0 2024-05-27T08:19:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.207+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16580f0070 con 0x55f337dd02f0 2024-05-27T08:19:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.208+0000 7f164be00640 1 -- 172.21.15.63:0/440964962 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f165800ce60 con 0x55f337dd02f0 2024-05-27T08:19:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.208+0000 7f1678194740 10 client.32428 fetch_fsmap learned FSMap version 35 2024-05-27T08:19:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.208+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f337edd620 con 0x55f337dd02f0 2024-05-27T08:19:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.209+0000 7f164be00640 1 -- 172.21.15.63:0/440964962 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16580f0070 con 0x55f337dd02f0 2024-05-27T08:19:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.209+0000 7f164be00640 1 -- 172.21.15.63:0/440964962 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f16580b2480 con 0x55f337dd02f0 2024-05-27T08:19:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.210+0000 7f1678194740 10 client.32428 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:19:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.210+0000 7f1678194740 10 client.32428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:19:41.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.210+0000 7f1678194740 20 client.32428 populate_metadata read hostname 'smithi063' 2024-05-27T08:19:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.210+0000 7f1678194740 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f337db7d80 0x55f3381c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.210+0000 7f1678194740 4 client.32428 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:19:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.210+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f33810eae0 con 0x55f337db7d80 2024-05-27T08:19:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.211+0000 7f1669400640 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f337db7d80 0x55f3381c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.212+0000 7f1669400640 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f337db7d80 0x55f3381c89d0 crc :-1 s=READY pgs=782 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340005ce8afb11d4 server_cookie=988bc4f19e312bf9 in_seq=0 out_seq=0 2024-05-27T08:19:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.213+0000 7f164be00640 10 client.32428 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.215+0000 7f164be00640 1 -- 172.21.15.63:0/440964962 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f33810eae0 con 0x55f337db7d80 2024-05-27T08:19:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.215+0000 7f164be00640 10 client.32428 handle_command_reply: tid=0 2024-05-27T08:19:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.216+0000 7f1678194740 1 client.32428 shutdown 2024-05-27T08:19:41.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:41 smithi063 ceph-mon[31613]: pgmap v1467: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 2.01k op/s 2024-05-27T08:19:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.217+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 msgr2=0x55f3381bccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.217+0000 7f1678194740 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381bccb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f16580dffd0 tx=0x7f16580a5aa0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.217+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 shutdown_connections 2024-05-27T08:19:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.217+0000 7f1678194740 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f337db7d80 0x55f3381c89d0 crc :-1 s=READY pgs=782 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.217+0000 7f1678194740 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f3381b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.217+0000 7f164be00640 0 client.32428 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.217+0000 7f1678194740 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd84d0 0x55f3381bd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.217+0000 7f1678194740 1 --2- 172.21.15.63:0/440964962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381bccb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.217+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 >> 172.21.15.63:0/440964962 conn(0x55f337ded300 msgr2=0x55f3381a4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.217+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 shutdown_connections 2024-05-27T08:19:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.218+0000 7f1678194740 1 -- 172.21.15.63:0/440964962 wait complete. 2024-05-27T08:19:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.218+0000 7f1678194740 20 client.32428 trim_cache size 0 max 16384 2024-05-27T08:19:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.274+0000 7f1678194740 1 Processor -- start 2024-05-27T08:19:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.275+0000 7f1678194740 1 -- start start 2024-05-27T08:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.277+0000 7f1678194740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337dd02f0 0x55f3381bd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.277+0000 7f1678194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.277+0000 7f1678194740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337df91b0 0x55f3381a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.277+0000 7f1678194740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f164c012970 con 0x55f337dd02f0 2024-05-27T08:19:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.277+0000 7f1678194740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f164c010240 con 0x55f337df91b0 2024-05-27T08:19:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.277+0000 7f1678194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f164c00e0b0 con 0x55f337dd84d0 2024-05-27T08:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.278+0000 7f1669e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337dd02f0 0x55f3381bd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.278+0000 7f1669e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337dd02f0 0x55f3381bd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37376/0 (socket says 172.21.15.63:37376) 2024-05-27T08:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.278+0000 7f1669e00640 1 -- 172.21.15.63:0/585468061 learned_addr learned my addr 172.21.15.63:0/585468061 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.279+0000 7f1669400640 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.279+0000 7f166a800640 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337df91b0 0x55f3381a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.280+0000 7f1669e00640 1 -- 172.21.15.63:0/585468061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 msgr2=0x55f3381a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.280+0000 7f1669e00640 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.280+0000 7f1669e00640 1 -- 172.21.15.63:0/585468061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337df91b0 msgr2=0x55f3381a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.280+0000 7f1669e00640 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337df91b0 0x55f3381a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.280+0000 7f1669e00640 1 -- 172.21.15.63:0/585468061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3381986c0 con 0x55f337dd02f0 2024-05-27T08:19:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.281+0000 7f1669e00640 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337dd02f0 0x55f3381bd210 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f1658000c00 tx=0x7f1658009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7a90544749b950 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.281+0000 7f166a800640 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337df91b0 0x55f3381a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:19:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.281+0000 7f1669400640 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:19:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.282+0000 7f1656a00640 1 -- 172.21.15.63:0/585468061 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580a9220 con 0x55f337dd02f0 2024-05-27T08:19:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.282+0000 7f1656a00640 1 -- 172.21.15.63:0/585468061 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16580f0070 con 0x55f337dd02f0 2024-05-27T08:19:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.282+0000 7f1656a00640 1 -- 172.21.15.63:0/585468061 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580a9400 con 0x55f337dd02f0 2024-05-27T08:19:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.284+0000 7f1678194740 1 -- 172.21.15.63:0/585468061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337dd02f0 msgr2=0x55f3381bd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.284+0000 7f1678194740 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337dd02f0 0x55f3381bd210 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f1658000c00 tx=0x7f1658009f90 comp rx=0 tx=0).stop 2024-05-27T08:19:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.284+0000 7f1678194740 1 -- 172.21.15.63:0/585468061 shutdown_connections 2024-05-27T08:19:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.284+0000 7f1678194740 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337df91b0 0x55f3381a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.284+0000 7f1678194740 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd84d0 0x55f3381a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.284+0000 7f1678194740 1 --2- 172.21.15.63:0/585468061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337dd02f0 0x55f3381bd210 unknown :-1 s=CLOSED pgs=2984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.285+0000 7f1678194740 1 -- 172.21.15.63:0/585468061 >> 172.21.15.63:0/585468061 conn(0x55f337ded300 msgr2=0x55f3381a43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.285+0000 7f1678194740 1 -- 172.21.15.63:0/585468061 shutdown_connections 2024-05-27T08:19:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.285+0000 7f1678194740 1 -- 172.21.15.63:0/585468061 wait complete. 2024-05-27T08:19:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.286+0000 7f1678194740 1 -- start start 2024-05-27T08:19:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.286+0000 7f1678194740 1 Processor -- start 2024-05-27T08:19:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.288+0000 7f1678194740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381c2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.288+0000 7f1678194740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd84d0 0x55f3381c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.288+0000 7f1678194740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f33822ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.288+0000 7f1678194740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f164c00e700 con 0x55f337df91b0 2024-05-27T08:19:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.288+0000 7f1678194740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f164c00fb70 con 0x55f337dd84d0 2024-05-27T08:19:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.288+0000 7f1678194740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f164c00edd0 con 0x55f337dd02f0 2024-05-27T08:19:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.288+0000 7f1678194740 10 client.0 _ll_register_callbacks cb 0x7f16580e6c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:19:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.289+0000 7f166a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f33822ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.289+0000 7f166a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f33822ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37378/0 (socket says 172.21.15.63:37378) 2024-05-27T08:19:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.289+0000 7f166a800640 1 -- 172.21.15.63:0/3386495229 learned_addr learned my addr 172.21.15.63:0/3386495229 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.289+0000 7f1669e00640 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381c2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.289+0000 7f1669400640 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd84d0 0x55f3381c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.289+0000 7f1669e00640 1 -- 172.21.15.63:0/3386495229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd84d0 msgr2=0x55f3381c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.289+0000 7f1669e00640 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd84d0 0x55f3381c25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.289+0000 7f1669e00640 1 -- 172.21.15.63:0/3386495229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 msgr2=0x55f33822ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.289+0000 7f1669e00640 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f33822ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.289+0000 7f1669e00640 1 -- 172.21.15.63:0/3386495229 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3381c2e80 con 0x55f337dd02f0 2024-05-27T08:19:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.290+0000 7f1669e00640 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381c2070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f16580dffd0 tx=0x7f165800efb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7858b9e430c76d3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.290+0000 7f164be00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:19:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.291+0000 7f164be00640 1 -- 172.21.15.63:0/3386495229 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580b8060 con 0x55f337dd02f0 2024-05-27T08:19:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.291+0000 7f164be00640 1 -- 172.21.15.63:0/3386495229 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7f16580a9d10 con 0x55f337dd02f0 2024-05-27T08:19:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.291+0000 7f1678194740 1 -- 172.21.15.63:0/3386495229 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16580f0070 con 0x55f337dd02f0 2024-05-27T08:19:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.295+0000 7f164be00640 1 -- 172.21.15.63:0/3386495229 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f16580a9400 con 0x55f337dd02f0 2024-05-27T08:19:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.295+0000 7f1678194740 10 client.32431 fetch_fsmap learned FSMap version 35 2024-05-27T08:19:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.295+0000 7f1678194740 1 -- 172.21.15.63:0/3386495229 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f3381c2e80 con 0x55f337dd02f0 2024-05-27T08:19:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.295+0000 7f164be00640 1 -- 172.21.15.63:0/3386495229 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16580f0070 con 0x55f337dd02f0 2024-05-27T08:19:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.296+0000 7f164be00640 1 -- 172.21.15.63:0/3386495229 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f16580a8d10 con 0x55f337dd02f0 2024-05-27T08:19:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.296+0000 7f1678194740 10 client.32431 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:19:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.296+0000 7f1678194740 10 client.32431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:19:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.296+0000 7f1678194740 20 client.32431 populate_metadata read hostname 'smithi063' 2024-05-27T08:19:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.296+0000 7f1678194740 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f337d8d1c0 0x55f338232640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.296+0000 7f1678194740 4 client.32431 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:19:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.296+0000 7f1678194740 1 -- 172.21.15.63:0/3386495229 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3381082c0 con 0x55f337d8d1c0 2024-05-27T08:19:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.299+0000 7f1669400640 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f337d8d1c0 0x55f338232640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.300+0000 7f1669400640 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f337d8d1c0 0x55f338232640 crc :-1 s=READY pgs=783 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768b797ca37d9554 server_cookie=45911fa25f2458c8 in_seq=0 out_seq=0 2024-05-27T08:19:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:41.300+0000 7f164be00640 10 client.32431 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:41.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:41 smithi086 ceph-mon[34687]: pgmap v1467: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 2.01k op/s 2024-05-27T08:19:41.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:41 smithi149 ceph-mon[35408]: pgmap v1467: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 MiB/s wr, 2.01k op/s 2024-05-27T08:19:43.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:43 smithi086 ceph-mon[34687]: pgmap v1468: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.8 MiB/s wr, 2.49k op/s 2024-05-27T08:19:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:43 smithi149 ceph-mon[35408]: pgmap v1468: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.8 MiB/s wr, 2.49k op/s 2024-05-27T08:19:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:43 smithi063 ceph-mon[31613]: pgmap v1468: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.8 MiB/s wr, 2.49k op/s 2024-05-27T08:19:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.206+0000 7f164be00640 1 -- 172.21.15.63:0/3386495229 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f3381082c0 con 0x55f337d8d1c0 2024-05-27T08:19:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.206+0000 7f164be00640 10 client.32431 handle_command_reply: tid=0 2024-05-27T08:19:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.207+0000 7f1678194740 1 client.32431 shutdown 2024-05-27T08:19:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.207+0000 7f1678194740 1 -- 172.21.15.63:0/3386495229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 msgr2=0x55f3381c2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.207+0000 7f1678194740 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381c2070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f16580dffd0 tx=0x7f165800efb0 comp rx=0 tx=0).stop 2024-05-27T08:19:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.207+0000 7f1678194740 1 -- 172.21.15.63:0/3386495229 shutdown_connections 2024-05-27T08:19:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.207+0000 7f1678194740 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f337d8d1c0 0x55f338232640 crc :-1 s=READY pgs=783 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.207+0000 7f1678194740 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f337df91b0 0x55f33822ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.207+0000 7f1678194740 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f337dd84d0 0x55f3381c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.207+0000 7f1678194740 1 --2- 172.21.15.63:0/3386495229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f337dd02f0 0x55f3381c2070 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.207+0000 7f1678194740 1 -- 172.21.15.63:0/3386495229 >> 172.21.15.63:0/3386495229 conn(0x55f337ded300 msgr2=0x55f3381a4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.208+0000 7f164be00640 0 client.32431 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.208+0000 7f1678194740 1 -- 172.21.15.63:0/3386495229 shutdown_connections 2024-05-27T08:19:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.208+0000 7f1678194740 1 -- 172.21.15.63:0/3386495229 wait complete. 2024-05-27T08:19:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.208+0000 7f1678194740 20 client.32431 trim_cache size 0 max 16384 2024-05-27T08:19:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:19:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:19:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:19:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.212+0000 7f1671200640 1 -- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f16500b2aa0 msgr2=0x7f16500b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.212+0000 7f1671200640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f16500b2aa0 0x7f16500b4fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f165c009830 tx=0x7f165c005780 comp rx=0 tx=0).stop 2024-05-27T08:19:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.212+0000 7f1671200640 1 -- 172.21.15.63:0/2643611366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166c10a490 msgr2=0x7f166c1a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.212+0000 7f1671200640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166c10a490 0x7f166c1a2f50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f165800e9c0 tx=0x7f165800ee80 comp rx=0 tx=0).stop 2024-05-27T08:19:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.213+0000 7f1671200640 1 -- 172.21.15.63:0/2643611366 shutdown_connections 2024-05-27T08:19:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.213+0000 7f1671200640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f16500b2aa0 0x7f16500b4fb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.213+0000 7f1671200640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f166c112900 0x7f166c1a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.213+0000 7f1671200640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f166c111de0 0x7f166c1a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.213+0000 7f1671200640 1 --2- 172.21.15.63:0/2643611366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166c10a490 0x7f166c1a2f50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.213+0000 7f1671200640 1 -- 172.21.15.63:0/2643611366 >> 172.21.15.63:0/2643611366 conn(0x7f166c06e8f0 msgr2=0x7f166c073960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.213+0000 7f1671200640 1 -- 172.21.15.63:0/2643611366 shutdown_connections 2024-05-27T08:19:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:44.217+0000 7f1671200640 1 -- 172.21.15.63:0/2643611366 wait complete. 2024-05-27T08:19:45.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:45 smithi086 ceph-mon[34687]: pgmap v1469: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.5 MiB/s wr, 2.16k op/s 2024-05-27T08:19:45.466 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:45 smithi149 ceph-mon[35408]: pgmap v1469: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.5 MiB/s wr, 2.16k op/s 2024-05-27T08:19:45.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:45 smithi063 ceph-mon[31613]: pgmap v1469: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.5 MiB/s wr, 2.16k op/s 2024-05-27T08:19:47.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:47 smithi086 ceph-mon[34687]: pgmap v1470: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.9 MiB/s wr, 2.00k op/s 2024-05-27T08:19:47.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:47 smithi149 ceph-mon[35408]: pgmap v1470: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.9 MiB/s wr, 2.00k op/s 2024-05-27T08:19:47.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:47 smithi063 ceph-mon[31613]: pgmap v1470: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.9 MiB/s wr, 2.00k op/s 2024-05-27T08:19:48.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:48 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:19:48.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:48 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:19:48.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:48 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:48.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:48 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:48.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:48 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:19:48.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:48 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:19:48.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:48 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:48.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:48 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:48.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:48 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:19:48.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:48 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:19:48.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:48 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:48.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:48 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:49.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:49 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:19:49.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:49 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:49.363 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:49 smithi086 ceph-mon[34687]: pgmap v1471: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.9 MiB/s wr, 2.50k op/s 2024-05-27T08:19:49.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:49 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:19:49.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:49 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:49.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:49 smithi149 ceph-mon[35408]: pgmap v1471: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.9 MiB/s wr, 2.50k op/s 2024-05-27T08:19:49.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:49 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:19:49.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:49 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:49.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:49 smithi063 ceph-mon[31613]: pgmap v1471: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.9 MiB/s wr, 2.50k op/s 2024-05-27T08:19:50.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:50 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:19:50.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:50 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:50.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:50 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:19:50.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:50 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:50.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:50 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:19:50.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:50 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:51.962 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:51 smithi086 ceph-mon[34687]: pgmap v1472: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-27T08:19:51.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:51 smithi149 ceph-mon[35408]: pgmap v1472: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-27T08:19:51.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:51 smithi063 ceph-mon[31613]: pgmap v1472: 129 pgs: 16 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 72 active+clean; 15 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-27T08:19:53.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:53 smithi086 ceph-mon[34687]: pgmap v1473: 129 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 80 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 9.1 MiB/s wr, 2.29k op/s 2024-05-27T08:19:53.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:53 smithi149 ceph-mon[35408]: pgmap v1473: 129 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 80 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 9.1 MiB/s wr, 2.29k op/s 2024-05-27T08:19:53.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:53 smithi063 ceph-mon[31613]: pgmap v1473: 129 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 80 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 9.1 MiB/s wr, 2.29k op/s 2024-05-27T08:19:55.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:55 smithi086 ceph-mon[34687]: pgmap v1474: 129 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 85 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-27T08:19:55.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:55 smithi149 ceph-mon[35408]: pgmap v1474: 129 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 85 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-27T08:19:55.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:55 smithi063 ceph-mon[31613]: pgmap v1474: 129 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 85 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.8 MiB/s wr, 1.98k op/s 2024-05-27T08:19:56.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:56 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:19:56.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:56 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:56.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:56 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:19:56.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:56 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:56.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:56 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:19:56.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:56 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:19:57.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:57 smithi086 ceph-mon[34687]: pgmap v1475: 129 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 86 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.5 MiB/s wr, 1.90k op/s 2024-05-27T08:19:57.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:57 smithi149 ceph-mon[35408]: pgmap v1475: 129 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 86 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.5 MiB/s wr, 1.90k op/s 2024-05-27T08:19:57.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:57 smithi063 ceph-mon[31613]: pgmap v1475: 129 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 86 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.5 MiB/s wr, 1.90k op/s 2024-05-27T08:19:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.680+0000 7fed51200640 1 -- 172.21.15.63:0/2086956753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed440a5ad0 msgr2=0x7fed440a5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.680+0000 7fed4a000640 1 -- 172.21.15.63:0/2086956753 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40035020 con 0x7fed440a5ad0 2024-05-27T08:19:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.680+0000 7fed51200640 1 --2- 172.21.15.63:0/2086956753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed440a5ad0 0x7fed440a5ee0 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7fed40009900 tx=0x7fed4002a420 comp rx=0 tx=0).stop 2024-05-27T08:19:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.681+0000 7fed51200640 1 -- 172.21.15.63:0/2086956753 shutdown_connections 2024-05-27T08:19:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.681+0000 7fed51200640 1 --2- 172.21.15.63:0/2086956753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed440acce0 0x7fed440b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.681+0000 7fed51200640 1 --2- 172.21.15.63:0/2086956753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed440ac220 0x7fed440ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.681+0000 7fed51200640 1 --2- 172.21.15.63:0/2086956753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed440a5ad0 0x7fed440a5ee0 unknown :-1 s=CLOSED pgs=2987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.681+0000 7fed51200640 1 -- 172.21.15.63:0/2086956753 >> 172.21.15.63:0/2086956753 conn(0x7fed4401aef0 msgr2=0x7fed4401b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.682+0000 7fed51200640 1 -- 172.21.15.63:0/2086956753 shutdown_connections 2024-05-27T08:19:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.682+0000 7fed51200640 1 -- 172.21.15.63:0/2086956753 wait complete. 2024-05-27T08:19:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.683+0000 7fed51200640 1 Processor -- start 2024-05-27T08:19:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.685+0000 7fed51200640 1 -- start start 2024-05-27T08:19:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.685+0000 7fed51200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed440a5ad0 0x7fed440b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.686+0000 7fed51200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed440ac220 0x7fed440b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.686+0000 7fed51200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed440acce0 0x7fed440b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.686+0000 7fed4b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed440a5ad0 0x7fed440b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.686+0000 7fed4b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed440a5ad0 0x7fed440b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:38104/0 (socket says 172.21.15.63:38104) 2024-05-27T08:19:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.687+0000 7fed51200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed440a8530 con 0x7fed440ac220 2024-05-27T08:19:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.687+0000 7fed51200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed440a83b0 con 0x7fed440acce0 2024-05-27T08:19:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.687+0000 7fed51200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed440a86b0 con 0x7fed440a5ad0 2024-05-27T08:19:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.687+0000 7fed4b400640 1 -- 172.21.15.63:0/2676741169 learned_addr learned my addr 172.21.15.63:0/2676741169 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.687+0000 7fed4be00640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed440acce0 0x7fed440b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.688+0000 7fed4b400640 1 -- 172.21.15.63:0/2676741169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed440acce0 msgr2=0x7fed440b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.688+0000 7fed4b400640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed440acce0 0x7fed440b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.688+0000 7fed4b400640 1 -- 172.21.15.63:0/2676741169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed440ac220 msgr2=0x7fed440b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.688+0000 7fed4b400640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed440ac220 0x7fed440b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.688+0000 7fed4b400640 1 -- 172.21.15.63:0/2676741169 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed440b20d0 con 0x7fed440a5ad0 2024-05-27T08:19:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.690+0000 7fed4b400640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed440a5ad0 0x7fed440b4280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fed40009850 tx=0x7fed40030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.690+0000 7fed3fe00640 1 -- 172.21.15.63:0/2676741169 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4003f070 con 0x7fed440a5ad0 2024-05-27T08:19:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.690+0000 7fed3fe00640 1 -- 172.21.15.63:0/2676741169 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed40004460 con 0x7fed440a5ad0 2024-05-27T08:19:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.691+0000 7fed51200640 1 -- 172.21.15.63:0/2676741169 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed44150f60 con 0x7fed440a5ad0 2024-05-27T08:19:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.691+0000 7fed3fe00640 1 -- 172.21.15.63:0/2676741169 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40035030 con 0x7fed440a5ad0 2024-05-27T08:19:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.691+0000 7fed51200640 1 -- 172.21.15.63:0/2676741169 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed44151310 con 0x7fed440a5ad0 2024-05-27T08:19:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.704+0000 7fed3fe00640 1 -- 172.21.15.63:0/2676741169 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fed40042070 con 0x7fed440a5ad0 2024-05-27T08:19:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.710+0000 7fed3fe00640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed340b2b60 0x7fed340b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.710+0000 7fed3fe00640 1 -- 172.21.15.63:0/2676741169 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7fed401037e0 con 0x7fed440a5ad0 2024-05-27T08:19:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.712+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed340b2b60 0x7fed340b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.718 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.718+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed340b2b60 0x7fed340b5070 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fed440a4920 tx=0x7fed38006c40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.719+0000 7fed59794740 1 Processor -- start 2024-05-27T08:19:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.719+0000 7fed59794740 1 -- start start 2024-05-27T08:19:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.722+0000 7fed59794740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb8aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.723+0000 7fed59794740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 0x5569eeb8afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.723+0000 7fed59794740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7c74d0 0x5569eeb8b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.723+0000 7fed59794740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5569eeafbb90 con 0x5569ee7c74d0 2024-05-27T08:19:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.723+0000 7fed59794740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5569eeaeb730 con 0x5569ee7bf2f0 2024-05-27T08:19:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.723+0000 7fed59794740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5569eeaef770 con 0x5569ee7e81b0 2024-05-27T08:19:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.724+0000 7fed4aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 0x5569eeb8afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.724+0000 7fed4b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb8aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.724+0000 7fed4b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb8aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:38106/0 (socket says 172.21.15.63:38106) 2024-05-27T08:19:57.725 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.724+0000 7fed4b400640 1 -- 172.21.15.63:0/2883073667 learned_addr learned my addr 172.21.15.63:0/2883073667 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:57.726 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.724+0000 7fed4be00640 1 --2- 172.21.15.63:0/2883073667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7c74d0 0x5569eeb8b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.726+0000 7fed4aa00640 1 -- 172.21.15.63:0/2883073667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 msgr2=0x5569eeb8aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.726+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2883073667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb8aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.726+0000 7fed4aa00640 1 -- 172.21.15.63:0/2883073667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7c74d0 msgr2=0x5569eeb8b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.726+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2883073667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7c74d0 0x5569eeb8b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.727+0000 7fed4aa00640 1 -- 172.21.15.63:0/2883073667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569eea848b0 con 0x5569ee7bf2f0 2024-05-27T08:19:57.728 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.727+0000 7fed4b400640 1 --2- 172.21.15.63:0/2883073667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb8aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:19:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.728+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2883073667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 0x5569eeb8afd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed440b2b80 tx=0x7fed3800e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e43cc83f1410ada server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.729+0000 7fed3cc00640 1 -- 172.21.15.63:0/2883073667 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38023070 con 0x5569ee7bf2f0 2024-05-27T08:19:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.729+0000 7fed3cc00640 1 -- 172.21.15.63:0/2883073667 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed3800b8a0 con 0x5569ee7bf2f0 2024-05-27T08:19:57.730 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.730+0000 7fed3cc00640 1 -- 172.21.15.63:0/2883073667 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3801e970 con 0x5569ee7bf2f0 2024-05-27T08:19:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.731+0000 7fed59794740 1 -- 172.21.15.63:0/2883073667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 msgr2=0x5569eeb8afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.731+0000 7fed59794740 1 --2- 172.21.15.63:0/2883073667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 0x5569eeb8afd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed440b2b80 tx=0x7fed3800e000 comp rx=0 tx=0).stop 2024-05-27T08:19:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.733+0000 7fed59794740 1 -- 172.21.15.63:0/2883073667 shutdown_connections 2024-05-27T08:19:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.733+0000 7fed59794740 1 --2- 172.21.15.63:0/2883073667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7c74d0 0x5569eeb8b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.733+0000 7fed59794740 1 --2- 172.21.15.63:0/2883073667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 0x5569eeb8afd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.733+0000 7fed59794740 1 --2- 172.21.15.63:0/2883073667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb8aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.733+0000 7fed59794740 1 -- 172.21.15.63:0/2883073667 >> 172.21.15.63:0/2883073667 conn(0x5569ee7dc300 msgr2=0x5569eeb87240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.733+0000 7fed59794740 1 -- 172.21.15.63:0/2883073667 shutdown_connections 2024-05-27T08:19:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.733+0000 7fed59794740 1 -- 172.21.15.63:0/2883073667 wait complete. 2024-05-27T08:19:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.734+0000 7fed59794740 1 -- start start 2024-05-27T08:19:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.738+0000 7fed59794740 1 Processor -- start 2024-05-27T08:19:57.739 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.739+0000 7fed59794740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 0x5569eeb94c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.740+0000 7fed59794740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7c74d0 0x5569eeb95140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.740+0000 7fed59794740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.740+0000 7fed59794740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5569ee7d1790 con 0x5569ee7e81b0 2024-05-27T08:19:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.740+0000 7fed59794740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5569eea93bc0 con 0x5569ee7bf2f0 2024-05-27T08:19:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.740+0000 7fed59794740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5569eeb97c10 con 0x5569ee7c74d0 2024-05-27T08:19:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.740+0000 7fed59794740 10 client.0 _ll_register_callbacks cb 0x5569eea71400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:19:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.741+0000 7fed4aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7c74d0 0x5569eeb95140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.741+0000 7fed4b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 0x5569eeb94c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.741+0000 7fed4aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7c74d0 0x5569eeb95140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:38122/0 (socket says 172.21.15.63:38122) 2024-05-27T08:19:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.741+0000 7fed4aa00640 1 -- 172.21.15.63:0/2240103861 learned_addr learned my addr 172.21.15.63:0/2240103861 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.742+0000 7fed4be00640 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.746+0000 7fed4aa00640 1 -- 172.21.15.63:0/2240103861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 msgr2=0x5569eeb94c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.746+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 0x5569eeb94c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.746 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.746+0000 7fed4aa00640 1 -- 172.21.15.63:0/2240103861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 msgr2=0x5569eeba0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.746+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.746+0000 7fed4aa00640 1 -- 172.21.15.63:0/2240103861 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ee800210 con 0x5569ee7c74d0 2024-05-27T08:19:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.746+0000 7fed4b400640 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 0x5569eeb94c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:19:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.746+0000 7fed4be00640 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:19:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.747+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7c74d0 0x5569eeb95140 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed38006fd0 tx=0x7fed3800bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=624265dd615335ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.747+0000 7fed31600640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:19:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.747+0000 7fed31600640 1 -- 172.21.15.63:0/2240103861 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38023040 con 0x5569ee7c74d0 2024-05-27T08:19:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.747+0000 7fed31600640 1 -- 172.21.15.63:0/2240103861 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7fed3801b040 con 0x5569ee7c74d0 2024-05-27T08:19:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.748+0000 7fed59794740 1 -- 172.21.15.63:0/2240103861 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3800b8a0 con 0x5569ee7c74d0 2024-05-27T08:19:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.748+0000 7fed31600640 1 -- 172.21.15.63:0/2240103861 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fed38023040 con 0x5569ee7c74d0 2024-05-27T08:19:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.750+0000 7fed59794740 10 client.32440 fetch_fsmap learned FSMap version 35 2024-05-27T08:19:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.750+0000 7fed59794740 1 -- 172.21.15.63:0/2240103861 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5569ee8cc620 con 0x5569ee7c74d0 2024-05-27T08:19:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.751+0000 7fed31600640 1 -- 172.21.15.63:0/2240103861 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed3800b8a0 con 0x5569ee7c74d0 2024-05-27T08:19:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.751+0000 7fed31600640 1 -- 172.21.15.63:0/2240103861 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fed38026070 con 0x5569ee7c74d0 2024-05-27T08:19:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.753+0000 7fed59794740 10 client.32440 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:19:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.753+0000 7fed59794740 10 client.32440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:19:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.753+0000 7fed59794740 20 client.32440 populate_metadata read hostname 'smithi063' 2024-05-27T08:19:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.753+0000 7fed59794740 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569ee7a6d80 0x5569eeba6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.753+0000 7fed59794740 4 client.32440 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:19:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.753+0000 7fed59794740 1 -- 172.21.15.63:0/2240103861 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569eeafdae0 con 0x5569ee7a6d80 2024-05-27T08:19:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.754+0000 7fed4b400640 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569ee7a6d80 0x5569eeba6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.755+0000 7fed4b400640 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569ee7a6d80 0x5569eeba6e70 crc :-1 s=READY pgs=784 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4a6aea383bb497 server_cookie=8c12ad270c035d7f in_seq=0 out_seq=0 2024-05-27T08:19:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.755+0000 7fed31600640 10 client.32440 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:57.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.756+0000 7fed31600640 1 -- 172.21.15.63:0/2240103861 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5569eeafdae0 con 0x5569ee7a6d80 2024-05-27T08:19:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.756+0000 7fed31600640 10 client.32440 handle_command_reply: tid=0 2024-05-27T08:19:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.757+0000 7fed59794740 1 client.32440 shutdown 2024-05-27T08:19:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.758+0000 7fed59794740 1 -- 172.21.15.63:0/2240103861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7c74d0 msgr2=0x5569eeb95140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.758+0000 7fed59794740 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7c74d0 0x5569eeb95140 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed38006fd0 tx=0x7fed3800bd00 comp rx=0 tx=0).stop 2024-05-27T08:19:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.759+0000 7fed59794740 1 -- 172.21.15.63:0/2240103861 shutdown_connections 2024-05-27T08:19:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.759+0000 7fed59794740 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569ee7a6d80 0x5569eeba6e70 crc :-1 s=READY pgs=784 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.759+0000 7fed59794740 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.759+0000 7fed59794740 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7c74d0 0x5569eeb95140 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed38006fd0 tx=0x7fed3800bd00 comp rx=0 tx=0).stop 2024-05-27T08:19:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.759+0000 7fed59794740 1 --2- 172.21.15.63:0/2240103861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee7bf2f0 0x5569eeb94c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.759+0000 7fed59794740 1 -- 172.21.15.63:0/2240103861 >> 172.21.15.63:0/2240103861 conn(0x5569ee7dc300 msgr2=0x5569eeb8aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.759+0000 7fed31600640 0 client.32440 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.759+0000 7fed59794740 1 -- 172.21.15.63:0/2240103861 shutdown_connections 2024-05-27T08:19:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.760+0000 7fed59794740 1 -- 172.21.15.63:0/2240103861 wait complete. 2024-05-27T08:19:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.760+0000 7fed59794740 20 client.32440 trim_cache size 0 max 16384 2024-05-27T08:19:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.832+0000 7fed59794740 1 Processor -- start 2024-05-27T08:19:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.835+0000 7fed59794740 1 -- start start 2024-05-27T08:19:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.836+0000 7fed59794740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7bf2f0 0x5569eeb903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.836+0000 7fed59794740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb93dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.836+0000 7fed59794740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 0x5569eeba0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.837+0000 7fed4aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb93dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.839+0000 7fed4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 0x5569eeba0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.839+0000 7fed4aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb93dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:38126/0 (socket says 172.21.15.63:38126) 2024-05-27T08:19:57.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.839+0000 7fed4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 0x5569eeba0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:47436/0 (socket says 172.21.15.63:47436) 2024-05-27T08:19:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.839+0000 7fed59794740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed4415de20 con 0x5569ee7bf2f0 2024-05-27T08:19:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.839+0000 7fed59794740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed4415b6f0 con 0x5569ee77c1c0 2024-05-27T08:19:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.839+0000 7fed59794740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed44159560 con 0x5569ee7e81b0 2024-05-27T08:19:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.839+0000 7fed4b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7bf2f0 0x5569eeb903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.839+0000 7fed4b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7bf2f0 0x5569eeb903d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35792/0 (socket says 172.21.15.63:35792) 2024-05-27T08:19:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.839+0000 7fed4b400640 1 -- 172.21.15.63:0/2951559436 learned_addr learned my addr 172.21.15.63:0/2951559436 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.841+0000 7fed4aa00640 1 -- 172.21.15.63:0/2951559436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 msgr2=0x5569eeba0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.841+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2951559436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 0x5569eeba0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.841+0000 7fed4aa00640 1 -- 172.21.15.63:0/2951559436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7bf2f0 msgr2=0x5569eeb903d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.841+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2951559436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7bf2f0 0x5569eeb903d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.841+0000 7fed4aa00640 1 -- 172.21.15.63:0/2951559436 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569eeb93150 con 0x5569ee7e81b0 2024-05-27T08:19:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.844+0000 7fed4aa00640 1 --2- 172.21.15.63:0/2951559436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb93dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed38014000 tx=0x7fed38006cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db15c548f842d351 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.844+0000 7fed33e00640 1 -- 172.21.15.63:0/2951559436 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3800b7a0 con 0x5569ee7e81b0 2024-05-27T08:19:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.847+0000 7fed33e00640 1 -- 172.21.15.63:0/2951559436 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed380278c0 con 0x5569ee7e81b0 2024-05-27T08:19:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.847+0000 7fed33e00640 1 -- 172.21.15.63:0/2951559436 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3801fe10 con 0x5569ee7e81b0 2024-05-27T08:19:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.849+0000 7fed59794740 1 -- 172.21.15.63:0/2951559436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 msgr2=0x5569eeb93dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.849+0000 7fed59794740 1 --2- 172.21.15.63:0/2951559436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb93dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed38014000 tx=0x7fed38006cb0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.851+0000 7fed59794740 1 -- 172.21.15.63:0/2951559436 shutdown_connections 2024-05-27T08:19:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.851+0000 7fed59794740 1 --2- 172.21.15.63:0/2951559436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 0x5569eeba0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.851+0000 7fed59794740 1 --2- 172.21.15.63:0/2951559436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7e81b0 0x5569eeb93dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.851+0000 7fed59794740 1 --2- 172.21.15.63:0/2951559436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7bf2f0 0x5569eeb903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.851+0000 7fed59794740 1 -- 172.21.15.63:0/2951559436 >> 172.21.15.63:0/2951559436 conn(0x5569ee7dc300 msgr2=0x5569eeb94490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.852+0000 7fed59794740 1 -- 172.21.15.63:0/2951559436 shutdown_connections 2024-05-27T08:19:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.852+0000 7fed59794740 1 -- 172.21.15.63:0/2951559436 wait complete. 2024-05-27T08:19:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.854+0000 7fed59794740 1 -- start start 2024-05-27T08:19:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.854+0000 7fed59794740 1 Processor -- start 2024-05-27T08:19:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.859+0000 7fed59794740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 0x5569eeba6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.859+0000 7fed59794740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7bf2f0 0x5569eeba7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.859+0000 7fed59794740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.860+0000 7fed59794740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed44159bb0 con 0x5569ee7e81b0 2024-05-27T08:19:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.860+0000 7fed59794740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed4415b020 con 0x5569ee77c1c0 2024-05-27T08:19:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.860+0000 7fed59794740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed4415a280 con 0x5569ee7bf2f0 2024-05-27T08:19:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.860+0000 7fed59794740 10 client.0 _ll_register_callbacks cb 0x7fed38015f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:19:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.860+0000 7fed4be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.861+0000 7fed4be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35796/0 (socket says 172.21.15.63:35796) 2024-05-27T08:19:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.861+0000 7fed4be00640 1 -- 172.21.15.63:0/2229294279 learned_addr learned my addr 172.21.15.63:0/2229294279 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:19:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.861+0000 7fed4b400640 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 0x5569eeba6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.862+0000 7fed4be00640 1 -- 172.21.15.63:0/2229294279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7bf2f0 msgr2=0x5569eeba7180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:19:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.862+0000 7fed4be00640 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7bf2f0 0x5569eeba7180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.862+0000 7fed4be00640 1 -- 172.21.15.63:0/2229294279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 msgr2=0x5569eeba6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:19:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.862+0000 7fed4be00640 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 0x5569eeba6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.862+0000 7fed4be00640 1 -- 172.21.15.63:0/2229294279 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569eec0d2b0 con 0x5569ee7e81b0 2024-05-27T08:19:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.867+0000 7fed4be00640 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba76c0 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7fed4c02e1b0 tx=0x7fed4c07caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8efe9c97a104e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:19:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.867+0000 7fed31600640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:19:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.869+0000 7fed31600640 1 -- 172.21.15.63:0/2229294279 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c07a880 con 0x5569ee7e81b0 2024-05-27T08:19:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.869+0000 7fed31600640 1 -- 172.21.15.63:0/2229294279 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7fed4c07aa20 con 0x5569ee7e81b0 2024-05-27T08:19:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.871+0000 7fed59794740 1 -- 172.21.15.63:0/2229294279 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed380278c0 con 0x5569ee7e81b0 2024-05-27T08:19:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.871+0000 7fed31600640 1 -- 172.21.15.63:0/2229294279 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fed4c07ac20 con 0x5569ee7e81b0 2024-05-27T08:19:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.872+0000 7fed59794740 10 client.23721 fetch_fsmap learned FSMap version 35 2024-05-27T08:19:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.872+0000 7fed59794740 1 -- 172.21.15.63:0/2229294279 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5569eec0e530 con 0x5569ee7e81b0 2024-05-27T08:19:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.873+0000 7fed31600640 1 -- 172.21.15.63:0/2229294279 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed380278c0 con 0x5569ee7e81b0 2024-05-27T08:19:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.873+0000 7fed31600640 1 -- 172.21.15.63:0/2229294279 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fed4c083070 con 0x5569ee7e81b0 2024-05-27T08:19:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.873+0000 7fed59794740 10 client.23721 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:19:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.879+0000 7fed59794740 10 client.23721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:19:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.879+0000 7fed59794740 20 client.23721 populate_metadata read hostname 'smithi063' 2024-05-27T08:19:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.879+0000 7fed59794740 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569eec0ecb0 0x5569eec11110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:19:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.879+0000 7fed59794740 4 client.23721 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:19:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.879+0000 7fed59794740 1 -- 172.21.15.63:0/2229294279 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569eeaf72c0 con 0x5569eec0ecb0 2024-05-27T08:19:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.880+0000 7fed4b400640 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569eec0ecb0 0x5569eec11110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:19:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.885+0000 7fed4b400640 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569eec0ecb0 0x5569eec11110 crc :-1 s=READY pgs=785 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31cbf25341f57d04 server_cookie=6b65d8be295b9754 in_seq=0 out_seq=0 2024-05-27T08:19:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:57.885+0000 7fed31600640 10 client.23721 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:58.081 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:19:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.143+0000 7fed31600640 1 -- 172.21.15.63:0/2229294279 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5569eeaf72c0 con 0x5569eec0ecb0 2024-05-27T08:19:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.143+0000 7fed31600640 10 client.23721 handle_command_reply: tid=0 2024-05-27T08:19:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.143+0000 7fed59794740 1 client.23721 shutdown 2024-05-27T08:19:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.143+0000 7fed59794740 1 -- 172.21.15.63:0/2229294279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 msgr2=0x5569eeba76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.143+0000 7fed59794740 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba76c0 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7fed4c02e1b0 tx=0x7fed4c07caf0 comp rx=0 tx=0).stop 2024-05-27T08:19:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.144+0000 7fed59794740 1 -- 172.21.15.63:0/2229294279 shutdown_connections 2024-05-27T08:19:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.144+0000 7fed59794740 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x5569eec0ecb0 0x5569eec11110 crc :-1 s=READY pgs=785 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.144+0000 7fed59794740 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5569ee7e81b0 0x5569eeba76c0 unknown :-1 s=CLOSED pgs=2988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.144+0000 7fed59794740 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5569ee7bf2f0 0x5569eeba7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.144+0000 7fed59794740 1 --2- 172.21.15.63:0/2229294279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5569ee77c1c0 0x5569eeba6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.145+0000 7fed59794740 1 -- 172.21.15.63:0/2229294279 >> 172.21.15.63:0/2229294279 conn(0x5569ee7dc300 msgr2=0x5569eeb94e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.145+0000 7fed31600640 0 client.23721 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:19:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.146+0000 7fed59794740 1 -- 172.21.15.63:0/2229294279 shutdown_connections 2024-05-27T08:19:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.146+0000 7fed59794740 1 -- 172.21.15.63:0/2229294279 wait complete. 2024-05-27T08:19:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.147+0000 7fed59794740 20 client.23721 trim_cache size 0 max 16384 2024-05-27T08:19:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:19:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:19:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:19:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.149+0000 7fed51200640 1 -- 172.21.15.63:0/2676741169 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed340b2b60 msgr2=0x7fed340b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.149+0000 7fed51200640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed340b2b60 0x7fed340b5070 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fed440a4920 tx=0x7fed38006c40 comp rx=0 tx=0).stop 2024-05-27T08:19:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.149+0000 7fed51200640 1 -- 172.21.15.63:0/2676741169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed440a5ad0 msgr2=0x7fed440b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:19:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.149+0000 7fed51200640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed440a5ad0 0x7fed440b4280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fed40009850 tx=0x7fed40030650 comp rx=0 tx=0).stop 2024-05-27T08:19:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.149+0000 7fed51200640 1 -- 172.21.15.63:0/2676741169 shutdown_connections 2024-05-27T08:19:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.149+0000 7fed51200640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fed340b2b60 0x7fed340b5070 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.149+0000 7fed51200640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed440acce0 0x7fed440b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.149+0000 7fed51200640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed440ac220 0x7fed440b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.149+0000 7fed51200640 1 --2- 172.21.15.63:0/2676741169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed440a5ad0 0x7fed440b4280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:19:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.149+0000 7fed51200640 1 -- 172.21.15.63:0/2676741169 >> 172.21.15.63:0/2676741169 conn(0x7fed4401aef0 msgr2=0x7fed440a2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:19:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.150+0000 7fed51200640 1 -- 172.21.15.63:0/2676741169 shutdown_connections 2024-05-27T08:19:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:19:59.150+0000 7fed51200640 1 -- 172.21.15.63:0/2676741169 wait complete. 2024-05-27T08:19:59.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:19:59 smithi086 ceph-mon[34687]: pgmap v1476: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.5 MiB/s wr, 2.38k op/s 2024-05-27T08:19:59.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:19:59 smithi149 ceph-mon[35408]: pgmap v1476: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.5 MiB/s wr, 2.38k op/s 2024-05-27T08:19:59.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:19:59 smithi063 ceph-mon[31613]: pgmap v1476: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.5 MiB/s wr, 2.38k op/s 2024-05-27T08:20:00.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:00 smithi086 ceph-mon[34687]: overall HEALTH_OK 2024-05-27T08:20:00.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:00 smithi149 ceph-mon[35408]: overall HEALTH_OK 2024-05-27T08:20:00.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:00 smithi063 ceph-mon[31613]: overall HEALTH_OK 2024-05-27T08:20:01.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:01 smithi086 ceph-mon[34687]: pgmap v1477: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 1.85k op/s 2024-05-27T08:20:01.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:01 smithi149 ceph-mon[35408]: pgmap v1477: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 1.85k op/s 2024-05-27T08:20:01.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:01 smithi063 ceph-mon[31613]: pgmap v1477: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.4 MiB/s wr, 1.85k op/s 2024-05-27T08:20:03.184 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:20:03.375 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:03 smithi063 ceph-mon[31613]: pgmap v1478: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 9.9 MiB/s wr, 2.47k op/s 2024-05-27T08:20:03.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:03 smithi086 ceph-mon[34687]: pgmap v1478: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 9.9 MiB/s wr, 2.47k op/s 2024-05-27T08:20:03.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:03 smithi149 ceph-mon[35408]: pgmap v1478: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 9.9 MiB/s wr, 2.47k op/s 2024-05-27T08:20:04.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:04 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:20:04.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:04 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:20:04.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:04 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:20:05.570 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:05 smithi063 ceph-mon[31613]: pgmap v1479: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.0 MiB/s wr, 1.99k op/s 2024-05-27T08:20:05.614 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:05 smithi149 ceph-mon[35408]: pgmap v1479: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.0 MiB/s wr, 1.99k op/s 2024-05-27T08:20:05.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:05 smithi086 ceph-mon[34687]: pgmap v1479: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.0 MiB/s wr, 1.99k op/s 2024-05-27T08:20:06.460 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:06 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:20:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.070+0000 7fe21f400700 1 -- 172.21.15.63:0/3271509497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe218110960 msgr2=0x7fe218110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.070+0000 7fe21f400700 1 --2- 172.21.15.63:0/3271509497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe218110960 0x7fe218110d70 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009990 tx=0x7fe20c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.070+0000 7fe21f400700 1 -- 172.21.15.63:0/3271509497 shutdown_connections 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.070+0000 7fe21f400700 1 --2- 172.21.15.63:0/3271509497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21810b2d0 0x7fe218117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.070+0000 7fe21f400700 1 --2- 172.21.15.63:0/3271509497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe218110960 0x7fe218110d70 unknown :-1 s=CLOSED pgs=2989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.070+0000 7fe21f400700 1 --2- 172.21.15.63:0/3271509497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21810a980 0x7fe21810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.070+0000 7fe21f400700 1 -- 172.21.15.63:0/3271509497 >> 172.21.15.63:0/3271509497 conn(0x7fe218077c70 msgr2=0x7fe2180780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.071+0000 7fe21f400700 1 -- 172.21.15.63:0/3271509497 shutdown_connections 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.071+0000 7fe21f400700 1 -- 172.21.15.63:0/3271509497 wait complete. 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.072+0000 7fe21f400700 1 Processor -- start 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.072+0000 7fe21f400700 1 -- start start 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe21f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21810a980 0x7fe2181a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe21f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21810b2d0 0x7fe2181a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe21f400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe218110960 0x7fe2181a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe21f400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe21811a960 con 0x7fe218110960 2024-05-27T08:20:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe21f400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe21811a7e0 con 0x7fe21810a980 2024-05-27T08:20:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe21f400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe21811aae0 con 0x7fe21810b2d0 2024-05-27T08:20:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe21d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe218110960 0x7fe2181a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe21d400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe218110960 0x7fe2181a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33910/0 (socket says 172.21.15.63:33910) 2024-05-27T08:20:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe21d400700 1 -- 172.21.15.63:0/500659078 learned_addr learned my addr 172.21.15.63:0/500659078 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe21ca00700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21810a980 0x7fe2181a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.073+0000 7fe217e00700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21810b2d0 0x7fe2181a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe21d400700 1 -- 172.21.15.63:0/500659078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21810b2d0 msgr2=0x7fe2181a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe21d400700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21810b2d0 0x7fe2181a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe21d400700 1 -- 172.21.15.63:0/500659078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21810a980 msgr2=0x7fe2181a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe21d400700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21810a980 0x7fe2181a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe21d400700 1 -- 172.21.15.63:0/500659078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2181a61f0 con 0x7fe218110960 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe21ca00700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21810a980 0x7fe2181a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe21d400700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe218110960 0x7fe2181a5c00 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7fe210007d70 tx=0x7fe21000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe215600700 1 -- 172.21.15.63:0/500659078 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210016020 con 0x7fe218110960 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe215600700 1 -- 172.21.15.63:0/500659078 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe21000ad40 con 0x7fe218110960 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe21f400700 1 -- 172.21.15.63:0/500659078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2181a64e0 con 0x7fe218110960 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe215600700 1 -- 172.21.15.63:0/500659078 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2100115e0 con 0x7fe218110960 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.074+0000 7fe21f400700 1 -- 172.21.15.63:0/500659078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2181a69d0 con 0x7fe218110960 2024-05-27T08:20:06.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.078+0000 7fe215600700 1 -- 172.21.15.63:0/500659078 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe210019020 con 0x7fe218110960 2024-05-27T08:20:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.079+0000 7fe215600700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe2000b29f0 0x7fe2000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.080+0000 7fe215600700 1 -- 172.21.15.63:0/500659078 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7fe2100e29b0 con 0x7fe218110960 2024-05-27T08:20:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.080+0000 7fe21ca00700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe2000b29f0 0x7fe2000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.080+0000 7fe21ca00700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe2000b29f0 0x7fe2000b4f00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe20c003760 tx=0x7fe20c005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.661+0000 7fe22e042b80 1 Processor -- start 2024-05-27T08:20:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.662+0000 7fe22e042b80 1 -- start start 2024-05-27T08:20:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.662+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600e859f0 0x55b600f18230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.662+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600d99e40 0x55b600f18770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.662+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f18d00 0x55b600f36120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.663+0000 7fe22e042b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b600e46e40 con 0x55b600d99e40 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.663+0000 7fe22e042b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b600e369e0 con 0x55b600f18d00 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.663+0000 7fe22e042b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b600e3aa20 con 0x55b600e859f0 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.663+0000 7fe217e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600d99e40 0x55b600f18770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.663+0000 7fe217e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600d99e40 0x55b600f18770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33922/0 (socket says 172.21.15.63:33922) 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.663+0000 7fe217e00700 1 -- 172.21.15.63:0/2261749702 learned_addr learned my addr 172.21.15.63:0/2261749702 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.664+0000 7fe21d400700 1 --2- 172.21.15.63:0/2261749702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f18d00 0x55b600f36120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.664+0000 7fe21ca00700 1 --2- 172.21.15.63:0/2261749702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600e859f0 0x55b600f18230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.665+0000 7fe217e00700 1 -- 172.21.15.63:0/2261749702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600e859f0 msgr2=0x55b600f18230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.665+0000 7fe217e00700 1 --2- 172.21.15.63:0/2261749702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600e859f0 0x55b600f18230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.665+0000 7fe217e00700 1 -- 172.21.15.63:0/2261749702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f18d00 msgr2=0x55b600f36120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.665+0000 7fe217e00700 1 --2- 172.21.15.63:0/2261749702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f18d00 0x55b600f36120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.665+0000 7fe217e00700 1 -- 172.21.15.63:0/2261749702 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b600dff420 con 0x55b600d99e40 2024-05-27T08:20:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.666+0000 7fe21d400700 1 --2- 172.21.15.63:0/2261749702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f18d00 0x55b600f36120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:20:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.667+0000 7fe217e00700 1 --2- 172.21.15.63:0/2261749702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600d99e40 0x55b600f18770 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7fe20400bce0 tx=0x7fe20400f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540354727d2e5d5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.668+0000 7fe20a000700 1 -- 172.21.15.63:0/2261749702 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204015070 con 0x55b600d99e40 2024-05-27T08:20:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.669+0000 7fe20a000700 1 -- 172.21.15.63:0/2261749702 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe20400fbe0 con 0x55b600d99e40 2024-05-27T08:20:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.670+0000 7fe20a000700 1 -- 172.21.15.63:0/2261749702 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204015070 con 0x55b600d99e40 2024-05-27T08:20:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.671+0000 7fe22e042b80 1 -- 172.21.15.63:0/2261749702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600d99e40 msgr2=0x55b600f18770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:06.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.671+0000 7fe22e042b80 1 --2- 172.21.15.63:0/2261749702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600d99e40 0x55b600f18770 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7fe20400bce0 tx=0x7fe20400f360 comp rx=0 tx=0).stop 2024-05-27T08:20:06.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.672+0000 7fe22e042b80 1 -- 172.21.15.63:0/2261749702 shutdown_connections 2024-05-27T08:20:06.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.672+0000 7fe22e042b80 1 --2- 172.21.15.63:0/2261749702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f18d00 0x55b600f36120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.672+0000 7fe22e042b80 1 --2- 172.21.15.63:0/2261749702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600e859f0 0x55b600f18230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.672+0000 7fe22e042b80 1 --2- 172.21.15.63:0/2261749702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600d99e40 0x55b600f18770 unknown :-1 s=CLOSED pgs=2991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.673+0000 7fe22e042b80 1 -- 172.21.15.63:0/2261749702 >> 172.21.15.63:0/2261749702 conn(0x55b600e33390 msgr2=0x55b600f144c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:06.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.673+0000 7fe22e042b80 1 -- 172.21.15.63:0/2261749702 shutdown_connections 2024-05-27T08:20:06.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.673+0000 7fe22e042b80 1 -- 172.21.15.63:0/2261749702 wait complete. 2024-05-27T08:20:06.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.674+0000 7fe22e042b80 1 -- start start 2024-05-27T08:20:06.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.675+0000 7fe22e042b80 1 Processor -- start 2024-05-27T08:20:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.677+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 0x55b600f337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.677+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f18d00 0x55b600f33d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.678+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f34250 0x55b600f3ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.678+0000 7fe22e042b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b600d3cfa0 con 0x55b600f18d00 2024-05-27T08:20:06.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.678+0000 7fe22e042b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b600dd7760 con 0x55b600e859f0 2024-05-27T08:20:06.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.678+0000 7fe22e042b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b600f36ba0 con 0x55b600f34250 2024-05-27T08:20:06.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.678+0000 7fe21ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 0x55b600f337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.679+0000 7fe21d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f34250 0x55b600f3ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.679+0000 7fe21d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f34250 0x55b600f3ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:39434/0 (socket says 172.21.15.63:39434) 2024-05-27T08:20:06.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.679+0000 7fe21d400700 1 -- 172.21.15.63:0/1702996941 learned_addr learned my addr 172.21.15.63:0/1702996941 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:06.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.679+0000 7fe22e042b80 10 client.0 _ll_register_callbacks cb 0x55b600de0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.680+0000 7fe217e00700 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f18d00 0x55b600f33d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.682+0000 7fe217e00700 1 -- 172.21.15.63:0/1702996941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f34250 msgr2=0x55b600f3ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.682+0000 7fe217e00700 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f34250 0x55b600f3ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.682+0000 7fe217e00700 1 -- 172.21.15.63:0/1702996941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 msgr2=0x55b600f337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.682+0000 7fe217e00700 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 0x55b600f337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.682+0000 7fe217e00700 1 -- 172.21.15.63:0/1702996941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b600d868d0 con 0x55b600f18d00 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.682+0000 7fe217e00700 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f18d00 0x55b600f33d10 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x55b600f15b00 tx=0x7fe20401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a56a3c332eb554 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.682+0000 7fe1fea00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.683+0000 7fe1fea00700 1 -- 172.21.15.63:0/1702996941 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204015040 con 0x55b600f18d00 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.683+0000 7fe1fea00700 1 -- 172.21.15.63:0/1702996941 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7fe20401a460 con 0x55b600f18d00 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.683+0000 7fe22e042b80 1 -- 172.21.15.63:0/1702996941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe20400fbe0 con 0x55b600f18d00 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.684+0000 7fe1fea00700 1 -- 172.21.15.63:0/1702996941 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe20401a660 con 0x55b600f18d00 2024-05-27T08:20:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.684+0000 7fe22e042b80 10 client.23733 fetch_fsmap learned FSMap version 35 2024-05-27T08:20:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.684+0000 7fe22e042b80 1 -- 172.21.15.63:0/1702996941 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b600e07060 con 0x55b600f18d00 2024-05-27T08:20:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.685+0000 7fe1fea00700 1 -- 172.21.15.63:0/1702996941 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe20400fbe0 con 0x55b600f18d00 2024-05-27T08:20:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.685+0000 7fe1fea00700 1 -- 172.21.15.63:0/1702996941 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe20402b030 con 0x55b600f18d00 2024-05-27T08:20:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.686+0000 7fe22e042b80 10 client.23733 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:20:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.686+0000 7fe22e042b80 10 client.23733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:20:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.686+0000 7fe22e042b80 20 client.23733 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.686+0000 7fe22e042b80 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b600f43ee0 0x55b600f46300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.686+0000 7fe22e042b80 4 client.23733 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:20:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.686+0000 7fe22e042b80 1 -- 172.21.15.63:0/1702996941 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b600e48d90 con 0x55b600f43ee0 2024-05-27T08:20:06.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.687+0000 7fe21ca00700 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b600f43ee0 0x55b600f46300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.690+0000 7fe1fea00700 10 client.23733 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.690+0000 7fe21ca00700 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b600f43ee0 0x55b600f46300 crc :-1 s=READY pgs=786 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca6a2fe159359bcd server_cookie=9ff99964f7a775d5 in_seq=0 out_seq=0 2024-05-27T08:20:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.691+0000 7fe1fea00700 1 -- 172.21.15.63:0/1702996941 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b600e48d90 con 0x55b600f43ee0 2024-05-27T08:20:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.691+0000 7fe1fea00700 10 client.23733 handle_command_reply: tid=0 2024-05-27T08:20:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.691+0000 7fe22e042b80 1 client.23733 shutdown 2024-05-27T08:20:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.692+0000 7fe22e042b80 1 -- 172.21.15.63:0/1702996941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f18d00 msgr2=0x55b600f33d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.692+0000 7fe22e042b80 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f18d00 0x55b600f33d10 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x55b600f15b00 tx=0x7fe20401ac70 comp rx=0 tx=0).stop 2024-05-27T08:20:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.692+0000 7fe22e042b80 1 -- 172.21.15.63:0/1702996941 shutdown_connections 2024-05-27T08:20:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.692+0000 7fe22e042b80 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b600f43ee0 0x55b600f46300 crc :-1 s=READY pgs=786 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.692+0000 7fe22e042b80 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f34250 0x55b600f3ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.692+0000 7fe22e042b80 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 0x55b600f337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.692+0000 7fe22e042b80 1 --2- 172.21.15.63:0/1702996941 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f18d00 0x55b600f33d10 unknown :-1 s=CLOSED pgs=2992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.692+0000 7fe22e042b80 1 -- 172.21.15.63:0/1702996941 >> 172.21.15.63:0/1702996941 conn(0x55b600e33390 msgr2=0x55b600f182c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.692+0000 7fe1fea00700 0 client.23733 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.693+0000 7fe22e042b80 1 -- 172.21.15.63:0/1702996941 shutdown_connections 2024-05-27T08:20:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.693+0000 7fe22e042b80 1 -- 172.21.15.63:0/1702996941 wait complete. 2024-05-27T08:20:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.694+0000 7fe22e042b80 20 client.23733 trim_cache size 0 max 16384 2024-05-27T08:20:06.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:06 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:20:06.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:06 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:20:06.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.754+0000 7fe22e042b80 1 Processor -- start 2024-05-27T08:20:06.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.756+0000 7fe22e042b80 1 -- start start 2024-05-27T08:20:06.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.756+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600e859f0 0x55b600f358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.756+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f34250 0x55b600f1a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.756+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 0x55b600f3fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.756+0000 7fe22e042b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181b0d90 con 0x55b600e859f0 2024-05-27T08:20:06.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.756+0000 7fe22e042b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181ae620 con 0x55b600f34250 2024-05-27T08:20:06.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.756+0000 7fe22e042b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe218083e40 con 0x55b600f1ab80 2024-05-27T08:20:06.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.757+0000 7fe217e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f34250 0x55b600f1a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.757+0000 7fe21d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 0x55b600f3fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.757+0000 7fe21d400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 0x55b600f3fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:39444/0 (socket says 172.21.15.63:39444) 2024-05-27T08:20:06.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.758+0000 7fe21d400700 1 -- 172.21.15.63:0/2125584572 learned_addr learned my addr 172.21.15.63:0/2125584572 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:06.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.759+0000 7fe21d400700 1 -- 172.21.15.63:0/2125584572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f34250 msgr2=0x55b600f1a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:06.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.760+0000 7fe21d400700 1 --2- 172.21.15.63:0/2125584572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f34250 0x55b600f1a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.760+0000 7fe21d400700 1 -- 172.21.15.63:0/2125584572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600e859f0 msgr2=0x55b600f358b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:06.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.760+0000 7fe21d400700 1 --2- 172.21.15.63:0/2125584572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600e859f0 0x55b600f358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.760+0000 7fe21d400700 1 -- 172.21.15.63:0/2125584572 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b600f10240 con 0x55b600f1ab80 2024-05-27T08:20:06.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.760+0000 7fe217e00700 1 --2- 172.21.15.63:0/2125584572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f34250 0x55b600f1a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:06.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.760+0000 7fe21d400700 1 --2- 172.21.15.63:0/2125584572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 0x55b600f3fc70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe2100a95d0 tx=0x7fe2100cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cfd734e3344effe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:06.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.761+0000 7fe209600700 1 -- 172.21.15.63:0/2125584572 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2100adc80 con 0x55b600f1ab80 2024-05-27T08:20:06.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.761+0000 7fe209600700 1 -- 172.21.15.63:0/2125584572 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2100b2730 con 0x55b600f1ab80 2024-05-27T08:20:06.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.761+0000 7fe209600700 1 -- 172.21.15.63:0/2125584572 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2100b22a0 con 0x55b600f1ab80 2024-05-27T08:20:06.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.763+0000 7fe22e042b80 1 -- 172.21.15.63:0/2125584572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 msgr2=0x55b600f3fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:06.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.763+0000 7fe22e042b80 1 --2- 172.21.15.63:0/2125584572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 0x55b600f3fc70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe2100a95d0 tx=0x7fe2100cc040 comp rx=0 tx=0).stop 2024-05-27T08:20:06.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.765+0000 7fe22e042b80 1 -- 172.21.15.63:0/2125584572 shutdown_connections 2024-05-27T08:20:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.765+0000 7fe22e042b80 1 --2- 172.21.15.63:0/2125584572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 0x55b600f3fc70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.765+0000 7fe22e042b80 1 --2- 172.21.15.63:0/2125584572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600e859f0 0x55b600f358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.765+0000 7fe22e042b80 1 --2- 172.21.15.63:0/2125584572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600f34250 0x55b600f1a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.765+0000 7fe22e042b80 1 -- 172.21.15.63:0/2125584572 >> 172.21.15.63:0/2125584572 conn(0x55b600e33390 msgr2=0x55b600f19f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:06.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.770+0000 7fe22e042b80 1 -- 172.21.15.63:0/2125584572 shutdown_connections 2024-05-27T08:20:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.770+0000 7fe22e042b80 1 -- 172.21.15.63:0/2125584572 wait complete. 2024-05-27T08:20:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.771+0000 7fe22e042b80 1 -- start start 2024-05-27T08:20:06.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.771+0000 7fe22e042b80 1 Processor -- start 2024-05-27T08:20:06.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.773+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 0x55b600f2e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.774+0000 7fe22e042b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 0x55b600f2e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.774+0000 7fe21ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 0x55b600f2e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.774+0000 7fe21ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 0x55b600f2e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:41288/0 (socket says 172.21.15.63:41288) 2024-05-27T08:20:06.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.775+0000 7fe21ca00700 1 -- 172.21.15.63:0/240595129 learned_addr learned my addr 172.21.15.63:0/240595129 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:06.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.775+0000 7fe22e042b80 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f34250 0x55b600f2ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.776+0000 7fe21d400700 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f34250 0x55b600f2ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.777+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe218084490 con 0x55b600f34250 2024-05-27T08:20:06.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.777+0000 7fe217e00700 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 0x55b600f2e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.777+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181adf70 con 0x55b600e859f0 2024-05-27T08:20:06.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.778+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe218084b60 con 0x55b600f1ab80 2024-05-27T08:20:06.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.779+0000 7fe22e042b80 10 client.0 _ll_register_callbacks cb 0x55b600de0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:06.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.779+0000 7fe21ca00700 1 -- 172.21.15.63:0/240595129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 msgr2=0x55b600f2e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:06.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.779+0000 7fe21ca00700 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 0x55b600f2e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.779+0000 7fe21ca00700 1 -- 172.21.15.63:0/240595129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f34250 msgr2=0x55b600f2ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:06.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.779+0000 7fe21ca00700 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f34250 0x55b600f2ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.779+0000 7fe21ca00700 1 -- 172.21.15.63:0/240595129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b600f3a5f0 con 0x55b600e859f0 2024-05-27T08:20:06.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.781+0000 7fe21ca00700 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 0x55b600f2e4b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe20c015000 tx=0x7fe20c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1178c6815a7df982 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:06.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.781+0000 7fe1fea00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:20:06.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.781+0000 7fe1fea00700 1 -- 172.21.15.63:0/240595129 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00b7d0 con 0x55b600e859f0 2024-05-27T08:20:06.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.781+0000 7fe1fea00700 1 -- 172.21.15.63:0/240595129 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7fe20c00b970 con 0x55b600e859f0 2024-05-27T08:20:06.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.782+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2100b2730 con 0x55b600e859f0 2024-05-27T08:20:06.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.783+0000 7fe1fea00700 1 -- 172.21.15.63:0/240595129 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe20c040050 con 0x55b600e859f0 2024-05-27T08:20:06.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.783+0000 7fe22e042b80 10 client.42641 fetch_fsmap learned FSMap version 35 2024-05-27T08:20:06.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.783+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b600faebc0 con 0x55b600e859f0 2024-05-27T08:20:06.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.784+0000 7fe1fea00700 1 -- 172.21.15.63:0/240595129 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2100b2730 con 0x55b600e859f0 2024-05-27T08:20:06.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.785+0000 7fe1fea00700 1 -- 172.21.15.63:0/240595129 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe20c030b20 con 0x55b600e859f0 2024-05-27T08:20:06.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.787+0000 7fe22e042b80 10 client.42641 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:20:06.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.787+0000 7fe22e042b80 10 client.42641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:20:06.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.787+0000 7fe22e042b80 20 client.42641 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:06.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.787+0000 7fe22e042b80 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b600faf2d0 0x55b600fb16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:06.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.788+0000 7fe22e042b80 4 client.42641 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:20:06.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.788+0000 7fe217e00700 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b600faf2d0 0x55b600fb16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:06.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.788+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b600e42570 con 0x55b600faf2d0 2024-05-27T08:20:06.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.789+0000 7fe217e00700 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b600faf2d0 0x55b600fb16f0 crc :-1 s=READY pgs=787 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f79b0d95eb0647 server_cookie=20398182bde5db93 in_seq=0 out_seq=0 2024-05-27T08:20:06.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.789+0000 7fe1fea00700 10 client.42641 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:06.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.792+0000 7fe1fea00700 1 -- 172.21.15.63:0/240595129 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b600e42570 con 0x55b600faf2d0 2024-05-27T08:20:06.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.792+0000 7fe1fea00700 10 client.42641 handle_command_reply: tid=0 2024-05-27T08:20:06.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.792+0000 7fe22e042b80 1 client.42641 shutdown 2024-05-27T08:20:06.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.793+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 msgr2=0x55b600f2e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:06.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.793+0000 7fe22e042b80 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 0x55b600f2e4b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe20c015000 tx=0x7fe20c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.793+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 shutdown_connections 2024-05-27T08:20:06.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.793+0000 7fe22e042b80 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b600faf2d0 0x55b600fb16f0 crc :-1 s=READY pgs=787 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.793+0000 7fe22e042b80 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b600f34250 0x55b600f2ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.793+0000 7fe22e042b80 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b600e859f0 0x55b600f2e4b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.793+0000 7fe22e042b80 1 --2- 172.21.15.63:0/240595129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b600f1ab80 0x55b600f2e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.793+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 >> 172.21.15.63:0/240595129 conn(0x55b600e33390 msgr2=0x55b600f320c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:06.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.793+0000 7fe1fea00700 0 client.42641 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:06.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.794+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 shutdown_connections 2024-05-27T08:20:06.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.794+0000 7fe22e042b80 1 -- 172.21.15.63:0/240595129 wait complete. 2024-05-27T08:20:06.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.794+0000 7fe22e042b80 20 client.42641 trim_cache size 0 max 16384 2024-05-27T08:20:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:20:06.796 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:20:06.796 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:20:06.796 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:20:06.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.796+0000 7fe21f400700 1 -- 172.21.15.63:0/500659078 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe2000b29f0 msgr2=0x7fe2000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:06.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.796+0000 7fe21f400700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe2000b29f0 0x7fe2000b4f00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe20c003760 tx=0x7fe20c005fd0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.796+0000 7fe21f400700 1 -- 172.21.15.63:0/500659078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe218110960 msgr2=0x7fe2181a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:06.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.796+0000 7fe21f400700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe218110960 0x7fe2181a5c00 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7fe210007d70 tx=0x7fe21000a340 comp rx=0 tx=0).stop 2024-05-27T08:20:06.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.797+0000 7fe21f400700 1 -- 172.21.15.63:0/500659078 shutdown_connections 2024-05-27T08:20:06.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.797+0000 7fe21f400700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fe2000b29f0 0x7fe2000b4f00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.797+0000 7fe21f400700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe218110960 0x7fe2181a5c00 unknown :-1 s=CLOSED pgs=2990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.797+0000 7fe21f400700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe21810a980 0x7fe2181a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.797+0000 7fe21f400700 1 --2- 172.21.15.63:0/500659078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe21810b2d0 0x7fe2181a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:06.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.797+0000 7fe21f400700 1 -- 172.21.15.63:0/500659078 >> 172.21.15.63:0/500659078 conn(0x7fe218077c70 msgr2=0x7fe21807bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:06.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.797+0000 7fe21f400700 1 -- 172.21.15.63:0/500659078 shutdown_connections 2024-05-27T08:20:06.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:06.797+0000 7fe21f400700 1 -- 172.21.15.63:0/500659078 wait complete. 2024-05-27T08:20:07.774 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:20:07.774 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:07 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:20:07.774 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:07 smithi063 ceph-mon[31613]: pgmap v1480: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 1.93k op/s 2024-05-27T08:20:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:20:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:07 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:20:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:07 smithi086 ceph-mon[34687]: pgmap v1480: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 1.93k op/s 2024-05-27T08:20:07.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:20:07.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:07 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:20:07.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:07 smithi149 ceph-mon[35408]: pgmap v1480: 129 pgs: 12 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 90 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 1.93k op/s 2024-05-27T08:20:09.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:09 smithi086 ceph-mon[34687]: pgmap v1481: 129 pgs: 10 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 98 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.9 MiB/s wr, 2.48k op/s 2024-05-27T08:20:09.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:09 smithi149 ceph-mon[35408]: pgmap v1481: 129 pgs: 10 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 98 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.9 MiB/s wr, 2.48k op/s 2024-05-27T08:20:09.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:09 smithi063 ceph-mon[31613]: pgmap v1481: 129 pgs: 10 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 98 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.9 MiB/s wr, 2.48k op/s 2024-05-27T08:20:11.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:11 smithi086 ceph-mon[34687]: pgmap v1482: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.4 MiB/s wr, 1.84k op/s 2024-05-27T08:20:11.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:11 smithi149 ceph-mon[35408]: pgmap v1482: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.4 MiB/s wr, 1.84k op/s 2024-05-27T08:20:11.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:11 smithi063 ceph-mon[31613]: pgmap v1482: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.4 MiB/s wr, 1.84k op/s 2024-05-27T08:20:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.785+0000 7f3748c00640 1 -- 172.21.15.63:0/608406712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744113da0 msgr2=0x7f37441161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.785+0000 7f3748c00640 1 --2- 172.21.15.63:0/608406712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744113da0 0x7f37441161c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f373c009ae0 tx=0x7f373c02a5c0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.786+0000 7f3748c00640 1 -- 172.21.15.63:0/608406712 shutdown_connections 2024-05-27T08:20:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.786+0000 7f3748c00640 1 --2- 172.21.15.63:0/608406712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37441167e0 0x7f3744118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.786+0000 7f3748c00640 1 --2- 172.21.15.63:0/608406712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744113da0 0x7f37441161c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.786+0000 7f3748c00640 1 --2- 172.21.15.63:0/608406712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374410a380 0x7f374410a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.786+0000 7f3748c00640 1 -- 172.21.15.63:0/608406712 >> 172.21.15.63:0/608406712 conn(0x7f374406e8f0 msgr2=0x7f374406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.787+0000 7f3748c00640 1 -- 172.21.15.63:0/608406712 shutdown_connections 2024-05-27T08:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.787+0000 7f3748c00640 1 -- 172.21.15.63:0/608406712 wait complete. 2024-05-27T08:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.788+0000 7f3748c00640 1 Processor -- start 2024-05-27T08:20:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.789+0000 7f3748c00640 1 -- start start 2024-05-27T08:20:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.790+0000 7f3748c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374410a380 0x7f37441a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.790+0000 7f3748c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744113da0 0x7f37441a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.791+0000 7f3748c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37441167e0 0x7f37441a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.791+0000 7f3740a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744113da0 0x7f37441a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.791+0000 7f3740a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744113da0 0x7f37441a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:33980/0 (socket says 172.21.15.63:33980) 2024-05-27T08:20:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.792+0000 7f3741400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374410a380 0x7f37441a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.792+0000 7f3741400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374410a380 0x7f37441a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:44648/0 (socket says 172.21.15.63:44648) 2024-05-27T08:20:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.792+0000 7f3748c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f374411a1a0 con 0x7f37441167e0 2024-05-27T08:20:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.792+0000 7f3748c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f374411a020 con 0x7f3744113da0 2024-05-27T08:20:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.792+0000 7f3748c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f374411a320 con 0x7f374410a380 2024-05-27T08:20:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.792+0000 7f3741e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37441167e0 0x7f37441a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.792+0000 7f3741e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37441167e0 0x7f37441a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47624/0 (socket says 172.21.15.63:47624) 2024-05-27T08:20:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.792+0000 7f3741e00640 1 -- 172.21.15.63:0/2866294207 learned_addr learned my addr 172.21.15.63:0/2866294207 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.793+0000 7f3741e00640 1 -- 172.21.15.63:0/2866294207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374410a380 msgr2=0x7f37441a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.793+0000 7f3741e00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374410a380 0x7f37441a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.793+0000 7f3741e00640 1 -- 172.21.15.63:0/2866294207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744113da0 msgr2=0x7f37441a6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.793+0000 7f3741e00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744113da0 0x7f37441a6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.793+0000 7f3741e00640 1 -- 172.21.15.63:0/2866294207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37441a1180 con 0x7f37441167e0 2024-05-27T08:20:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.796+0000 7f3741e00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37441167e0 0x7f37441a0ae0 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7f373400b970 tx=0x7f373400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.796+0000 7f373aa00640 1 -- 172.21.15.63:0/2866294207 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734019070 con 0x7f37441167e0 2024-05-27T08:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.797+0000 7f3748c00640 1 -- 172.21.15.63:0/2866294207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37441a1470 con 0x7f37441167e0 2024-05-27T08:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.797+0000 7f3748c00640 1 -- 172.21.15.63:0/2866294207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37441ad700 con 0x7f37441167e0 2024-05-27T08:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.799+0000 7f373aa00640 1 -- 172.21.15.63:0/2866294207 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3734014440 con 0x7f37441167e0 2024-05-27T08:20:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.799+0000 7f373aa00640 1 -- 172.21.15.63:0/2866294207 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37340075d0 con 0x7f37441167e0 2024-05-27T08:20:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.806+0000 7f374fc3b740 1 Processor -- start 2024-05-27T08:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.806+0000 7f374fc3b740 1 -- start start 2024-05-27T08:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.814+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088e11b0 0x55f508c8c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.814+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.814+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088c04d0 0x55f508c8cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.814+0000 7f374fc3b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f508bf4b90 con 0x55f5088e11b0 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.814+0000 7f374fc3b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f508be4730 con 0x55f5088b82f0 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.814+0000 7f374fc3b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f508be8770 con 0x55f5088c04d0 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.815+0000 7f3741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.815+0000 7f3741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:33982/0 (socket says 172.21.15.63:33982) 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.815+0000 7f3741400640 1 -- 172.21.15.63:0/1994939455 learned_addr learned my addr 172.21.15.63:0/1994939455 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.816+0000 7f3740a00640 1 --2- 172.21.15.63:0/1994939455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088e11b0 0x55f508c8c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.816+0000 7f3740a00640 1 --2- 172.21.15.63:0/1994939455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088c04d0 0x55f508c8cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.816+0000 7f3741400640 1 -- 172.21.15.63:0/1994939455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088c04d0 msgr2=0x55f508c8cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.816+0000 7f3741400640 1 --2- 172.21.15.63:0/1994939455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088c04d0 0x55f508c8cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.816+0000 7f3741400640 1 -- 172.21.15.63:0/1994939455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088e11b0 msgr2=0x55f508c8c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.816+0000 7f3741400640 1 --2- 172.21.15.63:0/1994939455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088e11b0 0x55f508c8c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.816+0000 7f3741400640 1 -- 172.21.15.63:0/1994939455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f508b7d8b0 con 0x55f5088b82f0 2024-05-27T08:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.816+0000 7f3740a00640 1 --2- 172.21.15.63:0/1994939455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088c04d0 0x55f508c8cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.817+0000 7f3741400640 1 --2- 172.21.15.63:0/1994939455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8c800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f373000bc60 tx=0x7f373000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f593789b737144b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.817+0000 7f372f000640 1 -- 172.21.15.63:0/1994939455 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730015070 con 0x55f5088b82f0 2024-05-27T08:20:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.818+0000 7f372f000640 1 -- 172.21.15.63:0/1994939455 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f373001a8f0 con 0x55f5088b82f0 2024-05-27T08:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.819+0000 7f374fc3b740 1 -- 172.21.15.63:0/1994939455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 msgr2=0x55f508c8c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.819+0000 7f374fc3b740 1 --2- 172.21.15.63:0/1994939455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8c800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f373000bc60 tx=0x7f373000f730 comp rx=0 tx=0).stop 2024-05-27T08:20:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.821+0000 7f373aa00640 1 -- 172.21.15.63:0/2866294207 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f373401c070 con 0x7f37441167e0 2024-05-27T08:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.821+0000 7f374fc3b740 1 -- 172.21.15.63:0/1994939455 shutdown_connections 2024-05-27T08:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.821+0000 7f374fc3b740 1 --2- 172.21.15.63:0/1994939455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088c04d0 0x55f508c8cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.821+0000 7f374fc3b740 1 --2- 172.21.15.63:0/1994939455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8c800 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.821+0000 7f374fc3b740 1 --2- 172.21.15.63:0/1994939455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088e11b0 0x55f508c8c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.821+0000 7f374fc3b740 1 -- 172.21.15.63:0/1994939455 >> 172.21.15.63:0/1994939455 conn(0x55f5088d5300 msgr2=0x55f508c88a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.821+0000 7f374fc3b740 1 -- 172.21.15.63:0/1994939455 shutdown_connections 2024-05-27T08:20:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.822+0000 7f374fc3b740 1 -- 172.21.15.63:0/1994939455 wait complete. 2024-05-27T08:20:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.823+0000 7f374fc3b740 1 -- start start 2024-05-27T08:20:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.823+0000 7f374fc3b740 1 Processor -- start 2024-05-27T08:20:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.824+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088b82f0 0x55f508c96450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.824+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088c04d0 0x55f508c96990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.824+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 0x55f508ca2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.824+0000 7f374fc3b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f5088ca790 con 0x55f5088b82f0 2024-05-27T08:20:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.824+0000 7f374fc3b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f508b8cbc0 con 0x55f5088c04d0 2024-05-27T08:20:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.825+0000 7f374fc3b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f508c99490 con 0x55f5088e11b0 2024-05-27T08:20:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.825+0000 7f374fc3b740 10 client.0 _ll_register_callbacks cb 0x55f508b6a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.826+0000 7f3740a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 0x55f508ca2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.826+0000 7f3740a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 0x55f508ca2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:44664/0 (socket says 172.21.15.63:44664) 2024-05-27T08:20:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.826+0000 7f3740a00640 1 -- 172.21.15.63:0/1834833846 learned_addr learned my addr 172.21.15.63:0/1834833846 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.826+0000 7f3740a00640 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088b82f0 0x55f508c96450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.827+0000 7f3741400640 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088c04d0 0x55f508c96990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.828+0000 7f373aa00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37280b28e0 0x7f37280b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.828+0000 7f373aa00640 1 -- 172.21.15.63:0/2866294207 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7f37340e1f40 con 0x7f37441167e0 2024-05-27T08:20:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.829+0000 7f3741400640 1 -- 172.21.15.63:0/1834833846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 msgr2=0x55f508ca2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.829+0000 7f3741400640 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 0x55f508ca2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.829+0000 7f3741400640 1 -- 172.21.15.63:0/1834833846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088b82f0 msgr2=0x55f508c96450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.829+0000 7f3741400640 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088b82f0 0x55f508c96450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.829+0000 7f3741400640 1 -- 172.21.15.63:0/1834833846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5088f9210 con 0x55f5088c04d0 2024-05-27T08:20:12.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.829+0000 7f3741400640 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088c04d0 0x55f508c96990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55f508c89a00 tx=0x7f373001a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55830441430e1dbf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.830+0000 7f3723400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:20:12.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.831+0000 7f3723400640 1 -- 172.21.15.63:0/1834833846 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730005040 con 0x55f5088c04d0 2024-05-27T08:20:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.831+0000 7f3723400640 1 -- 172.21.15.63:0/1834833846 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7f37300051e0 con 0x55f5088c04d0 2024-05-27T08:20:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.833+0000 7f3741e00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37280b28e0 0x7f37280b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.834+0000 7f374fc3b740 1 -- 172.21.15.63:0/1834833846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f373001a8f0 con 0x55f5088c04d0 2024-05-27T08:20:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.834+0000 7f3723400640 1 -- 172.21.15.63:0/1834833846 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f37300053e0 con 0x55f5088c04d0 2024-05-27T08:20:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.835+0000 7f374fc3b740 10 client.42653 fetch_fsmap learned FSMap version 35 2024-05-27T08:20:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.835+0000 7f374fc3b740 1 -- 172.21.15.63:0/1834833846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f5089c5620 con 0x55f5088c04d0 2024-05-27T08:20:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.836+0000 7f3723400640 1 -- 172.21.15.63:0/1834833846 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f373001a8f0 con 0x55f5088c04d0 2024-05-27T08:20:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.838+0000 7f3723400640 1 -- 172.21.15.63:0/1834833846 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f37300115d0 con 0x55f5088c04d0 2024-05-27T08:20:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.838+0000 7f374fc3b740 10 client.42653 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:20:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.838+0000 7f374fc3b740 10 client.42653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:20:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.839+0000 7f374fc3b740 20 client.42653 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.839+0000 7f374fc3b740 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f50889fd80 0x55f508c85400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.839+0000 7f374fc3b740 4 client.42653 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:20:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.839+0000 7f374fc3b740 1 -- 172.21.15.63:0/1834833846 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f508bf6ae0 con 0x55f50889fd80 2024-05-27T08:20:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.840+0000 7f3740a00640 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f50889fd80 0x55f508c85400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.843+0000 7f3740a00640 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f50889fd80 0x55f508c85400 crc :-1 s=READY pgs=788 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f8131235796b11 server_cookie=b11d01500a1113fd in_seq=0 out_seq=0 2024-05-27T08:20:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.843+0000 7f3723400640 10 client.42653 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.843+0000 7f3741e00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37280b28e0 0x7f37280b4df0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f37340df3e0 tx=0x7f373400bf90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.845+0000 7f3723400640 1 -- 172.21.15.63:0/1834833846 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f508bf6ae0 con 0x55f50889fd80 2024-05-27T08:20:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.845+0000 7f3723400640 10 client.42653 handle_command_reply: tid=0 2024-05-27T08:20:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.845+0000 7f374fc3b740 1 client.42653 shutdown 2024-05-27T08:20:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.846+0000 7f374fc3b740 1 -- 172.21.15.63:0/1834833846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088c04d0 msgr2=0x55f508c96990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.846+0000 7f374fc3b740 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088c04d0 0x55f508c96990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55f508c89a00 tx=0x7f373001a690 comp rx=0 tx=0).stop 2024-05-27T08:20:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.846+0000 7f374fc3b740 1 -- 172.21.15.63:0/1834833846 shutdown_connections 2024-05-27T08:20:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.846+0000 7f374fc3b740 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f50889fd80 0x55f508c85400 crc :-1 s=READY pgs=788 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.846+0000 7f374fc3b740 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 0x55f508ca2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.846+0000 7f374fc3b740 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088c04d0 0x55f508c96990 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55f508c89a00 tx=0x7f373001a690 comp rx=0 tx=0).stop 2024-05-27T08:20:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.846+0000 7f374fc3b740 1 --2- 172.21.15.63:0/1834833846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088b82f0 0x55f508c96450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.846+0000 7f374fc3b740 1 -- 172.21.15.63:0/1834833846 >> 172.21.15.63:0/1834833846 conn(0x55f5088d5300 msgr2=0x55f508c8dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.846+0000 7f3723400640 0 client.42653 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.847+0000 7f374fc3b740 1 -- 172.21.15.63:0/1834833846 shutdown_connections 2024-05-27T08:20:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.847+0000 7f374fc3b740 1 -- 172.21.15.63:0/1834833846 wait complete. 2024-05-27T08:20:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.847+0000 7f374fc3b740 20 client.42653 trim_cache size 0 max 16384 2024-05-27T08:20:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.885+0000 7f374fc3b740 1 Processor -- start 2024-05-27T08:20:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.886+0000 7f374fc3b740 1 -- start start 2024-05-27T08:20:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.886+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c98b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.886+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088e11b0 0x55f508c8cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.886+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088751c0 0x55f508ca2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.886+0000 7f374fc3b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37441ba0c0 con 0x55f5088e11b0 2024-05-27T08:20:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.886+0000 7f374fc3b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37441b7990 con 0x55f5088b82f0 2024-05-27T08:20:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.886+0000 7f374fc3b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f37441b57c0 con 0x55f5088751c0 2024-05-27T08:20:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.887+0000 7f3740a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c98b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.887+0000 7f3740a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c98b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:33992/0 (socket says 172.21.15.63:33992) 2024-05-27T08:20:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.887+0000 7f3740a00640 1 -- 172.21.15.63:0/2530231952 learned_addr learned my addr 172.21.15.63:0/2530231952 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.887+0000 7f3740a00640 1 --2- 172.21.15.63:0/2530231952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088751c0 0x55f508ca2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.887+0000 7f3741400640 1 --2- 172.21.15.63:0/2530231952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088e11b0 0x55f508c8cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.888+0000 7f3740a00640 1 -- 172.21.15.63:0/2530231952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088751c0 msgr2=0x55f508ca2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.888+0000 7f3740a00640 1 --2- 172.21.15.63:0/2530231952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088751c0 0x55f508ca2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.888+0000 7f3740a00640 1 -- 172.21.15.63:0/2530231952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088e11b0 msgr2=0x55f508c8cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.888+0000 7f3740a00640 1 --2- 172.21.15.63:0/2530231952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088e11b0 0x55f508c8cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.888+0000 7f3740a00640 1 -- 172.21.15.63:0/2530231952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f508c7a210 con 0x55f5088b82f0 2024-05-27T08:20:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.888+0000 7f3740a00640 1 --2- 172.21.15.63:0/2530231952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c98b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f373c02abb0 tx=0x7f373c02a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d58fa4e43f05d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.889+0000 7f372e600640 1 -- 172.21.15.63:0/2530231952 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c037040 con 0x55f5088b82f0 2024-05-27T08:20:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.892+0000 7f372e600640 1 -- 172.21.15.63:0/2530231952 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f373c038040 con 0x55f5088b82f0 2024-05-27T08:20:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.892+0000 7f372e600640 1 -- 172.21.15.63:0/2530231952 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c001cc0 con 0x55f5088b82f0 2024-05-27T08:20:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.893+0000 7f374fc3b740 1 -- 172.21.15.63:0/2530231952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 msgr2=0x55f508c98b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.893+0000 7f374fc3b740 1 --2- 172.21.15.63:0/2530231952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c98b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f373c02abb0 tx=0x7f373c02a5c0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.897+0000 7f374fc3b740 1 -- 172.21.15.63:0/2530231952 shutdown_connections 2024-05-27T08:20:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.897+0000 7f374fc3b740 1 --2- 172.21.15.63:0/2530231952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088751c0 0x55f508ca2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.897+0000 7f374fc3b740 1 --2- 172.21.15.63:0/2530231952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088e11b0 0x55f508c8cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.897+0000 7f374fc3b740 1 --2- 172.21.15.63:0/2530231952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c98b80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.897+0000 7f374fc3b740 1 -- 172.21.15.63:0/2530231952 >> 172.21.15.63:0/2530231952 conn(0x55f5088d5300 msgr2=0x55f508c95ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.897+0000 7f374fc3b740 1 -- 172.21.15.63:0/2530231952 shutdown_connections 2024-05-27T08:20:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.897+0000 7f374fc3b740 1 -- 172.21.15.63:0/2530231952 wait complete. 2024-05-27T08:20:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.898+0000 7f374fc3b740 1 -- start start 2024-05-27T08:20:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.898+0000 7f374fc3b740 1 Processor -- start 2024-05-27T08:20:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.903+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088751c0 0x55f508c917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.903+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.903+0000 7f374fc3b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 0x55f508c90100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.903+0000 7f374fc3b740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37441b5e50 con 0x55f5088751c0 2024-05-27T08:20:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.903+0000 7f374fc3b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37441b72c0 con 0x55f5088b82f0 2024-05-27T08:20:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.903+0000 7f374fc3b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f37441b6520 con 0x55f5088e11b0 2024-05-27T08:20:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.903+0000 7f374fc3b740 10 client.0 _ll_register_callbacks cb 0x7f373000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.906+0000 7f3740a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088751c0 0x55f508c917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.906+0000 7f3741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.906+0000 7f3741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:34008/0 (socket says 172.21.15.63:34008) 2024-05-27T08:20:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.907+0000 7f3741400640 1 -- 172.21.15.63:0/2163523855 learned_addr learned my addr 172.21.15.63:0/2163523855 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.907+0000 7f3740a00640 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 0x55f508c90100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.907+0000 7f3741400640 1 -- 172.21.15.63:0/2163523855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 msgr2=0x55f508c90100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.908+0000 7f3741400640 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 0x55f508c90100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.908+0000 7f3741400640 1 -- 172.21.15.63:0/2163523855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088751c0 msgr2=0x55f508c917f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.908+0000 7f3741400640 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088751c0 0x55f508c917f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.908+0000 7f3741400640 1 -- 172.21.15.63:0/2163523855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f508c90640 con 0x55f5088b82f0 2024-05-27T08:20:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.909+0000 7f3723400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:20:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.909+0000 7f3741400640 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8fbc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3730011fd0 tx=0x7f3730005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e4b0cf64954958a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.910+0000 7f3723400640 1 -- 172.21.15.63:0/2163523855 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730035040 con 0x55f5088b82f0 2024-05-27T08:20:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.911+0000 7f3723400640 1 -- 172.21.15.63:0/2163523855 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(174..174 src has 1..174) ==== 9651+0+0 (secure 0 0 0) 0x7f3730005580 con 0x55f5088b82f0 2024-05-27T08:20:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.912+0000 7f374fc3b740 1 -- 172.21.15.63:0/2163523855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f373c038040 con 0x55f5088b82f0 2024-05-27T08:20:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.912+0000 7f3723400640 1 -- 172.21.15.63:0/2163523855 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f37300053e0 con 0x55f5088b82f0 2024-05-27T08:20:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.913+0000 7f374fc3b740 10 client.42659 fetch_fsmap learned FSMap version 35 2024-05-27T08:20:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.914+0000 7f374fc3b740 1 -- 172.21.15.63:0/2163523855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f508c9e330 con 0x55f5088b82f0 2024-05-27T08:20:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.914+0000 7f3723400640 1 -- 172.21.15.63:0/2163523855 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f373c038040 con 0x55f5088b82f0 2024-05-27T08:20:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.918+0000 7f3723400640 1 -- 172.21.15.63:0/2163523855 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3730004710 con 0x55f5088b82f0 2024-05-27T08:20:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.918+0000 7f374fc3b740 10 client.42659 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:20:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.918+0000 7f374fc3b740 10 client.42659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:20:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.918+0000 7f374fc3b740 20 client.42659 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.918+0000 7f374fc3b740 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f508c9ea00 0x55f508c9ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.918+0000 7f374fc3b740 4 client.42659 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:20:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.918+0000 7f374fc3b740 1 -- 172.21.15.63:0/2163523855 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f508bf02c0 con 0x55f508c9ea00 2024-05-27T08:20:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.919+0000 7f3740a00640 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f508c9ea00 0x55f508c9ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.921+0000 7f3740a00640 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f508c9ea00 0x55f508c9ee10 crc :-1 s=READY pgs=789 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35fb120414cbfbd3 server_cookie=175a77d68b79c11c in_seq=0 out_seq=0 2024-05-27T08:20:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:12.921+0000 7f3723400640 10 client.42659 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.276+0000 7f3723400640 1 -- 172.21.15.63:0/2163523855 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f508bf02c0 con 0x55f508c9ea00 2024-05-27T08:20:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.276+0000 7f3723400640 10 client.42659 handle_command_reply: tid=0 2024-05-27T08:20:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.276+0000 7f374fc3b740 1 client.42659 shutdown 2024-05-27T08:20:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.278+0000 7f374fc3b740 1 -- 172.21.15.63:0/2163523855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 msgr2=0x55f508c8fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.278+0000 7f374fc3b740 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8fbc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3730011fd0 tx=0x7f3730005f30 comp rx=0 tx=0).stop 2024-05-27T08:20:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.279+0000 7f374fc3b740 1 -- 172.21.15.63:0/2163523855 shutdown_connections 2024-05-27T08:20:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.279+0000 7f374fc3b740 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55f508c9ea00 0x55f508c9ee10 crc :-1 s=READY pgs=789 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.279+0000 7f374fc3b740 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5088e11b0 0x55f508c90100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.279+0000 7f374fc3b740 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5088b82f0 0x55f508c8fbc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.279+0000 7f374fc3b740 1 --2- 172.21.15.63:0/2163523855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5088751c0 0x55f508c917f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.279+0000 7f374fc3b740 1 -- 172.21.15.63:0/2163523855 >> 172.21.15.63:0/2163523855 conn(0x55f5088d5300 msgr2=0x55f508c966e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.279+0000 7f3723400640 0 client.42659 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.281+0000 7f374fc3b740 1 -- 172.21.15.63:0/2163523855 shutdown_connections 2024-05-27T08:20:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.281+0000 7f374fc3b740 1 -- 172.21.15.63:0/2163523855 wait complete. 2024-05-27T08:20:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.281+0000 7f374fc3b740 20 client.42659 trim_cache size 0 max 16384 2024-05-27T08:20:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:20:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:20:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:20:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.284+0000 7f3748c00640 1 -- 172.21.15.63:0/2866294207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37280b28e0 msgr2=0x7f37280b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.284+0000 7f3748c00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37280b28e0 0x7f37280b4df0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f37340df3e0 tx=0x7f373400bf90 comp rx=0 tx=0).stop 2024-05-27T08:20:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.284+0000 7f3748c00640 1 -- 172.21.15.63:0/2866294207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37441167e0 msgr2=0x7f37441a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.284+0000 7f3748c00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37441167e0 0x7f37441a0ae0 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7f373400b970 tx=0x7f373400be30 comp rx=0 tx=0).stop 2024-05-27T08:20:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.284+0000 7f3748c00640 1 -- 172.21.15.63:0/2866294207 shutdown_connections 2024-05-27T08:20:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.284+0000 7f3748c00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f37280b28e0 0x7f37280b4df0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.284+0000 7f3748c00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37441167e0 0x7f37441a0ae0 unknown :-1 s=CLOSED pgs=2993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.284+0000 7f3748c00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3744113da0 0x7f37441a6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.284+0000 7f3748c00640 1 --2- 172.21.15.63:0/2866294207 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f374410a380 0x7f37441a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.284+0000 7f3748c00640 1 -- 172.21.15.63:0/2866294207 >> 172.21.15.63:0/2866294207 conn(0x7f374406e8f0 msgr2=0x7f3744073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.287+0000 7f3748c00640 1 -- 172.21.15.63:0/2866294207 shutdown_connections 2024-05-27T08:20:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:13.289+0000 7f3748c00640 1 -- 172.21.15.63:0/2866294207 wait complete. 2024-05-27T08:20:13.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:13 smithi086 ceph-mon[34687]: pgmap v1483: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 10 MiB/s wr, 2.48k op/s 2024-05-27T08:20:13.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:13 smithi149 ceph-mon[35408]: pgmap v1483: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 10 MiB/s wr, 2.48k op/s 2024-05-27T08:20:13.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:13 smithi063 ceph-mon[31613]: pgmap v1483: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 10 MiB/s wr, 2.48k op/s 2024-05-27T08:20:14.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:13 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: ignoring --setuser ceph since I am not root 2024-05-27T08:20:14.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:13 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: ignoring --setgroup ceph since I am not root 2024-05-27T08:20:14.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:13 smithi086 ceph-mds[76223]: -- 172.21.15.86:0/4260971201 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edba1d3a40 con 0x55edba21b600 2024-05-27T08:20:14.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:13 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: starting mds.k at 2024-05-27T08:20:14.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:14 smithi086 ceph-mon[34687]: daemon mds.k restarted 2024-05-27T08:20:14.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:14 smithi149 ceph-mon[35408]: daemon mds.k restarted 2024-05-27T08:20:14.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:14 smithi063 ceph-mon[31613]: daemon mds.k restarted 2024-05-27T08:20:15.448 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:20:15.448 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:20:15.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:15 smithi149 ceph-mon[35408]: osdmap e175: 12 total, 12 up, 12 in 2024-05-27T08:20:15.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:15 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6832/3997074493,v1:172.21.15.86:6833/3997074493] up:boot 2024-05-27T08:20:15.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:15 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:20:15.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:15 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:20:15.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:15 smithi063 ceph-mon[31613]: osdmap e175: 12 total, 12 up, 12 in 2024-05-27T08:20:15.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:15 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6832/3997074493,v1:172.21.15.86:6833/3997074493] up:boot 2024-05-27T08:20:15.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:15 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:20:15.969 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:15 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:20:15.969 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:15 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:20:15.969 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:15 smithi063 ceph-mon[31613]: pgmap v1485: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.6 MiB/s wr, 2.12k op/s 2024-05-27T08:20:15.969 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:15 smithi063 ceph-mon[31613]: osdmap e176: 12 total, 12 up, 12 in 2024-05-27T08:20:16.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:15 smithi086 ceph-mon[34687]: osdmap e175: 12 total, 12 up, 12 in 2024-05-27T08:20:16.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:15 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6832/3997074493,v1:172.21.15.86:6833/3997074493] up:boot 2024-05-27T08:20:16.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:15 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:20:16.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:15 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:20:16.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:15 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:20:16.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:15 smithi086 ceph-mon[34687]: pgmap v1485: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.6 MiB/s wr, 2.12k op/s 2024-05-27T08:20:16.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:15 smithi086 ceph-mon[34687]: osdmap e176: 12 total, 12 up, 12 in 2024-05-27T08:20:16.462 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:20:16.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:16 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:20:16.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:16 smithi149 ceph-mon[35408]: pgmap v1485: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.6 MiB/s wr, 2.12k op/s 2024-05-27T08:20:16.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:16 smithi149 ceph-mon[35408]: osdmap e176: 12 total, 12 up, 12 in 2024-05-27T08:20:17.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:17 smithi063 ceph-mon[31613]: pgmap v1487: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.9 MiB/s wr, 1.45k op/s 2024-05-27T08:20:17.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:17 smithi086 ceph-mon[34687]: pgmap v1487: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.9 MiB/s wr, 1.45k op/s 2024-05-27T08:20:17.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:17 smithi149 ceph-mon[35408]: pgmap v1487: 129 pgs: 9 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 102 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.9 MiB/s wr, 1.45k op/s 2024-05-27T08:20:19.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:19 smithi086 ceph-mon[34687]: pgmap v1488: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 11 MiB/s wr, 2.59k op/s 2024-05-27T08:20:19.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:19 smithi149 ceph-mon[35408]: pgmap v1488: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 11 MiB/s wr, 2.59k op/s 2024-05-27T08:20:19.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:19 smithi063 ceph-mon[31613]: pgmap v1488: 129 pgs: 18 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 11 MiB/s wr, 2.59k op/s 2024-05-27T08:20:21.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:21 smithi149 ceph-mon[35408]: pgmap v1489: 129 pgs: 17 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.3 MiB/s wr, 1.54k op/s 2024-05-27T08:20:21.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:21 smithi063 ceph-mon[31613]: pgmap v1489: 129 pgs: 17 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.3 MiB/s wr, 1.54k op/s 2024-05-27T08:20:21.759 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:21 smithi086 ceph-mon[34687]: pgmap v1489: 129 pgs: 17 active+clean+snaptrim, 46 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.3 MiB/s wr, 1.54k op/s 2024-05-27T08:20:23.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:23 smithi063 ceph-mon[31613]: pgmap v1490: 129 pgs: 10 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 87 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.2 MiB/s wr, 1.77k op/s 2024-05-27T08:20:23.308 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:23 smithi086 ceph-mon[34687]: pgmap v1490: 129 pgs: 10 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 87 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.2 MiB/s wr, 1.77k op/s 2024-05-27T08:20:23.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:23 smithi149 ceph-mon[35408]: pgmap v1490: 129 pgs: 10 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 87 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.2 MiB/s wr, 1.77k op/s 2024-05-27T08:20:25.067 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:25 smithi063 ceph-mon[31613]: pgmap v1491: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.1 MiB/s wr, 1.50k op/s 2024-05-27T08:20:25.466 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:25 smithi149 ceph-mon[35408]: pgmap v1491: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.1 MiB/s wr, 1.50k op/s 2024-05-27T08:20:26.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.220+0000 7f53ace00700 1 -- 172.21.15.63:0/1531303970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8108690 msgr2=0x7f53a8108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.222+0000 7f53ace00700 1 --2- 172.21.15.63:0/1531303970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8108690 0x7f53a8108b60 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009f90 tx=0x7f53a000b330 comp rx=0 tx=0).stop 2024-05-27T08:20:26.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.222+0000 7f53ace00700 1 -- 172.21.15.63:0/1531303970 shutdown_connections 2024-05-27T08:20:26.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.222+0000 7f53ace00700 1 --2- 172.21.15.63:0/1531303970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8108690 0x7f53a8108b60 unknown :-1 s=CLOSED pgs=2994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.222+0000 7f53ace00700 1 --2- 172.21.15.63:0/1531303970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53a8103240 0x7f53a8103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.222+0000 7f53ace00700 1 --2- 172.21.15.63:0/1531303970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a8103b90 0x7f53a81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.222+0000 7f53ace00700 1 -- 172.21.15.63:0/1531303970 >> 172.21.15.63:0/1531303970 conn(0x7f53a80fedb0 msgr2=0x7f53a81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:26.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.222+0000 7f53ace00700 1 -- 172.21.15.63:0/1531303970 shutdown_connections 2024-05-27T08:20:26.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.223+0000 7f53ace00700 1 -- 172.21.15.63:0/1531303970 wait complete. 2024-05-27T08:20:26.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.226+0000 7f53ace00700 1 Processor -- start 2024-05-27T08:20:26.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.227+0000 7f53ace00700 1 -- start start 2024-05-27T08:20:26.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.229+0000 7f53ace00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103240 0x7f53a81a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.229+0000 7f53ace00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53a8103b90 0x7f53a81a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.229+0000 7f53ace00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a8108690 0x7f53a81a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.229+0000 7f53ace00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53a811a990 con 0x7f53a8103240 2024-05-27T08:20:26.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.229+0000 7f53ace00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53a811a810 con 0x7f53a8108690 2024-05-27T08:20:26.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.229+0000 7f53ace00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f53a811ab10 con 0x7f53a8103b90 2024-05-27T08:20:26.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.230+0000 7f53a5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103240 0x7f53a81a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.230+0000 7f53a5e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103240 0x7f53a81a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50058/0 (socket says 172.21.15.63:50058) 2024-05-27T08:20:26.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.230+0000 7f53a5e00700 1 -- 172.21.15.63:0/3881607426 learned_addr learned my addr 172.21.15.63:0/3881607426 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:26.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.231+0000 7f53a6800700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a8108690 0x7f53a81a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.231+0000 7f53a5e00700 1 -- 172.21.15.63:0/3881607426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53a8103b90 msgr2=0x7f53a81a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.232+0000 7f53a5e00700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53a8103b90 0x7f53a81a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.232+0000 7f53a5e00700 1 -- 172.21.15.63:0/3881607426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a8108690 msgr2=0x7f53a81a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.232+0000 7f53a5e00700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a8108690 0x7f53a81a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.233+0000 7f53a5e00700 1 -- 172.21.15.63:0/3881607426 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a81a6250 con 0x7f53a8103240 2024-05-27T08:20:26.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.233+0000 7f53a6800700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a8108690 0x7f53a81a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:26.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.234+0000 7f53a5e00700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103240 0x7f53a81a1380 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 crypto rx=0x7f539800b530 tx=0x7f539800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.236+0000 7f539ea00700 1 -- 172.21.15.63:0/3881607426 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398015020 con 0x7f53a8103240 2024-05-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.236+0000 7f539ea00700 1 -- 172.21.15.63:0/3881607426 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5398011cc0 con 0x7f53a8103240 2024-05-27T08:20:26.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.236+0000 7f539ea00700 1 -- 172.21.15.63:0/3881607426 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398015020 con 0x7f53a8103240 2024-05-27T08:20:26.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.236+0000 7f53ace00700 1 -- 172.21.15.63:0/3881607426 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a81a6540 con 0x7f53a8103240 2024-05-27T08:20:26.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.237+0000 7f53ace00700 1 -- 172.21.15.63:0/3881607426 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a81a69d0 con 0x7f53a8103240 2024-05-27T08:20:26.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.239+0000 7f53ace00700 1 -- 172.21.15.63:0/3881607426 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a80512b0 con 0x7f53a8103240 2024-05-27T08:20:26.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.245+0000 7f539ea00700 1 -- 172.21.15.63:0/3881607426 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5398018020 con 0x7f53a8103240 2024-05-27T08:20:26.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.247+0000 7f539ea00700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f538c0b2c70 0x7f538c0b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.247+0000 7f539ea00700 1 -- 172.21.15.63:0/3881607426 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 9715+0+0 (secure 0 0 0) 0x7f53980e1d70 con 0x7f53a8103240 2024-05-27T08:20:26.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.249+0000 7f53a5400700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f538c0b2c70 0x7f538c0b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.251+0000 7f53a5400700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f538c0b2c70 0x7f538c0b5180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5394005950 tx=0x7f53940058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:26.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.261+0000 7f539ea00700 1 -- 172.21.15.63:0/3881607426 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f539801f020 con 0x7f53a8103240 2024-05-27T08:20:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.741+0000 7f0589a00640 1 -- 172.21.15.63:0/1109916743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f058410f540 msgr2=0x7f0584116ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.741+0000 7f0589a00640 1 --2- 172.21.15.63:0/1109916743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f058410f540 0x7f0584116ce0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0574009900 tx=0x7f057402a3f0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.745+0000 7f0589a00640 1 -- 172.21.15.63:0/1109916743 shutdown_connections 2024-05-27T08:20:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.745+0000 7f0589a00640 1 --2- 172.21.15.63:0/1109916743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f058410f540 0x7f0584116ce0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.745+0000 7f0589a00640 1 --2- 172.21.15.63:0/1109916743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f058410ea80 0x7f058410eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.745+0000 7f0589a00640 1 --2- 172.21.15.63:0/1109916743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058410a500 0x7f058410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.745+0000 7f0589a00640 1 -- 172.21.15.63:0/1109916743 >> 172.21.15.63:0/1109916743 conn(0x7f0584077190 msgr2=0x7f05840775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.746+0000 7f0589a00640 1 -- 172.21.15.63:0/1109916743 shutdown_connections 2024-05-27T08:20:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.747+0000 7f0589a00640 1 -- 172.21.15.63:0/1109916743 wait complete. 2024-05-27T08:20:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.748+0000 7f0589a00640 1 Processor -- start 2024-05-27T08:20:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.749+0000 7f0589a00640 1 -- start start 2024-05-27T08:20:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.749+0000 7f0589a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f058410a500 0x7f05841a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.749+0000 7f0589a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f058410ea80 0x7f05841a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.749+0000 7f0589a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058410f540 0x7f05841a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.749+0000 7f0589a00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f058411a380 con 0x7f058410ea80 2024-05-27T08:20:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.749+0000 7f0589a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f058411a200 con 0x7f058410f540 2024-05-27T08:20:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.749+0000 7f0589a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f058411a500 con 0x7f058410a500 2024-05-27T08:20:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.750+0000 7f0582000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f058410ea80 0x7f05841a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.751+0000 7f0582000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f058410ea80 0x7f05841a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50076/0 (socket says 172.21.15.63:50076) 2024-05-27T08:20:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.751+0000 7f0582000640 1 -- 172.21.15.63:0/1343685633 learned_addr learned my addr 172.21.15.63:0/1343685633 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.753+0000 7f0582000640 1 -- 172.21.15.63:0/1343685633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f058410a500 msgr2=0x7f05841a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.753+0000 7f0582000640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f058410a500 0x7f05841a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.754 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.753+0000 7f0582000640 1 -- 172.21.15.63:0/1343685633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058410f540 msgr2=0x7f05841a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.753+0000 7f0582000640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058410f540 0x7f05841a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.753+0000 7f0582000640 1 -- 172.21.15.63:0/1343685633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05841a5c40 con 0x7f058410ea80 2024-05-27T08:20:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.754+0000 7f0582000640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f058410ea80 0x7f05841a1340 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f057000b450 tx=0x7f057000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.755+0000 7f057b400640 1 -- 172.21.15.63:0/1343685633 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570015020 con 0x7f058410ea80 2024-05-27T08:20:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.755+0000 7f0589a00640 1 -- 172.21.15.63:0/1343685633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05841a5f30 con 0x7f058410ea80 2024-05-27T08:20:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.755+0000 7f0589a00640 1 -- 172.21.15.63:0/1343685633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05841ad8b0 con 0x7f058410ea80 2024-05-27T08:20:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.756+0000 7f057b400640 1 -- 172.21.15.63:0/1343685633 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0570011cc0 con 0x7f058410ea80 2024-05-27T08:20:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.756+0000 7f057b400640 1 -- 172.21.15.63:0/1343685633 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570010d40 con 0x7f058410ea80 2024-05-27T08:20:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.760+0000 7f05909cb740 1 Processor -- start 2024-05-27T08:20:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.760+0000 7f05909cb740 1 -- start start 2024-05-27T08:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.764+0000 7f05909cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab0f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.764+0000 7f05909cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a7442f0 0x55fb5ab0fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.764+0000 7f05909cb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.764+0000 7f05909cb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fb5aa80b90 con 0x55fb5a74c4d0 2024-05-27T08:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.764+0000 7f05909cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fb5aa70730 con 0x55fb5a7442f0 2024-05-27T08:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.764+0000 7f05909cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb5aa74770 con 0x55fb5a76d1b0 2024-05-27T08:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.765+0000 7f0583400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab0f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.765+0000 7f0583400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab0f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:54748/0 (socket says 172.21.15.63:54748) 2024-05-27T08:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.765+0000 7f0583400640 1 -- 172.21.15.63:0/1663253637 learned_addr learned my addr 172.21.15.63:0/1663253637 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.765+0000 7f0582a00640 1 --2- 172.21.15.63:0/1663253637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a7442f0 0x55fb5ab0fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.766+0000 7f0583400640 1 -- 172.21.15.63:0/1663253637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a7442f0 msgr2=0x55fb5ab0fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.766+0000 7f0583400640 1 --2- 172.21.15.63:0/1663253637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a7442f0 0x55fb5ab0fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.766+0000 7f0583400640 1 -- 172.21.15.63:0/1663253637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a74c4d0 msgr2=0x55fb5ab102e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.766+0000 7f0583400640 1 --2- 172.21.15.63:0/1663253637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab102e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.766+0000 7f0583400640 1 -- 172.21.15.63:0/1663253637 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5aa098b0 con 0x55fb5a76d1b0 2024-05-27T08:20:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.767+0000 7f0583400640 1 --2- 172.21.15.63:0/1663253637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab0f860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0574005ab0 tx=0x7f05740040d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c0745cbf2d34806 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.767+0000 7f056fe00640 1 -- 172.21.15.63:0/1663253637 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057403e030 con 0x55fb5a76d1b0 2024-05-27T08:20:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.767+0000 7f056fe00640 1 -- 172.21.15.63:0/1663253637 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0574004240 con 0x55fb5a76d1b0 2024-05-27T08:20:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.767+0000 7f056fe00640 1 -- 172.21.15.63:0/1663253637 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057403e030 con 0x55fb5a76d1b0 2024-05-27T08:20:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.769+0000 7f057b400640 1 -- 172.21.15.63:0/1343685633 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0570018020 con 0x7f058410ea80 2024-05-27T08:20:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.769+0000 7f05909cb740 1 -- 172.21.15.63:0/1663253637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a76d1b0 msgr2=0x55fb5ab0f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.769+0000 7f05909cb740 1 --2- 172.21.15.63:0/1663253637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab0f860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0574005ab0 tx=0x7f05740040d0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.769+0000 7f05909cb740 1 -- 172.21.15.63:0/1663253637 shutdown_connections 2024-05-27T08:20:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.769+0000 7f05909cb740 1 --2- 172.21.15.63:0/1663253637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.769+0000 7f05909cb740 1 --2- 172.21.15.63:0/1663253637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a7442f0 0x55fb5ab0fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.769+0000 7f05909cb740 1 --2- 172.21.15.63:0/1663253637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab0f860 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.769+0000 7f05909cb740 1 -- 172.21.15.63:0/1663253637 >> 172.21.15.63:0/1663253637 conn(0x55fb5a761300 msgr2=0x55fb5ab0c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.770+0000 7f05909cb740 1 -- 172.21.15.63:0/1663253637 shutdown_connections 2024-05-27T08:20:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.770+0000 7f05909cb740 1 -- 172.21.15.63:0/1663253637 wait complete. 2024-05-27T08:20:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.771+0000 7f05909cb740 1 -- start start 2024-05-27T08:20:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.771+0000 7f057b400640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f05680b29b0 0x7f05680b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.772+0000 7f05909cb740 1 Processor -- start 2024-05-27T08:20:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.772+0000 7f057b400640 1 -- 172.21.15.63:0/1343685633 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 9715+0+0 (secure 0 0 0) 0x7f05700e2180 con 0x7f058410ea80 2024-05-27T08:20:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.773+0000 7f0583400640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f05680b29b0 0x7f05680b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.774+0000 7f0583400640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f05680b29b0 0x7f05680b4ec0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f057402a9b0 tx=0x7f0574005fd0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.775+0000 7f05909cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a7442f0 0x55fb5ab19a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.775+0000 7f05909cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab19f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.775+0000 7f05909cb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab25cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.775+0000 7f05909cb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fb5a756790 con 0x55fb5a76d1b0 2024-05-27T08:20:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.775+0000 7f05909cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fb5aa18bc0 con 0x55fb5a74c4d0 2024-05-27T08:20:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.775+0000 7f05909cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb5ab1c930 con 0x55fb5a7442f0 2024-05-27T08:20:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.775+0000 7f05909cb740 10 client.0 _ll_register_callbacks cb 0x55fb5a9f6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.777+0000 7f0582000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab19f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.777+0000 7f0582a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a7442f0 0x55fb5ab19a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.777+0000 7f0582000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab19f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:38532/0 (socket says 172.21.15.63:38532) 2024-05-27T08:20:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.777+0000 7f0582000640 1 -- 172.21.15.63:0/1645477556 learned_addr learned my addr 172.21.15.63:0/1645477556 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.777+0000 7f0583400640 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab25cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.779+0000 7f0582000640 1 -- 172.21.15.63:0/1645477556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a7442f0 msgr2=0x55fb5ab19a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.779+0000 7f0582000640 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a7442f0 0x55fb5ab19a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.779+0000 7f0582000640 1 -- 172.21.15.63:0/1645477556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a76d1b0 msgr2=0x55fb5ab25cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.779+0000 7f0582000640 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab25cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.779+0000 7f0582000640 1 -- 172.21.15.63:0/1645477556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5a785210 con 0x55fb5a74c4d0 2024-05-27T08:20:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.779+0000 7f0583400640 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab25cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.783+0000 7f0582000640 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab19f60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0570009fd0 tx=0x7f0570012fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=642ae9e16a176d07 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.783+0000 7f056cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:20:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.787+0000 7f056cc00640 1 -- 172.21.15.63:0/1645477556 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05700c2020 con 0x55fb5a74c4d0 2024-05-27T08:20:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.787+0000 7f056cc00640 1 -- 172.21.15.63:0/1645477556 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(176..176 src has 1..176) ==== 9715+0+0 (secure 0 0 0) 0x7f05700ad0d0 con 0x55fb5a74c4d0 2024-05-27T08:20:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.789+0000 7f05909cb740 1 -- 172.21.15.63:0/1645477556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0574004240 con 0x55fb5a74c4d0 2024-05-27T08:20:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.790+0000 7f056cc00640 1 -- 172.21.15.63:0/1645477556 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f05700c8050 con 0x55fb5a74c4d0 2024-05-27T08:20:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.790+0000 7f05909cb740 10 client.42680 fetch_fsmap learned FSMap version 37 2024-05-27T08:20:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.790+0000 7f05909cb740 1 -- 172.21.15.63:0/1645477556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fb5a851620 con 0x55fb5a74c4d0 2024-05-27T08:20:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.790+0000 7f056cc00640 1 -- 172.21.15.63:0/1645477556 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05700ad410 con 0x55fb5a74c4d0 2024-05-27T08:20:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.791+0000 7f056cc00640 1 -- 172.21.15.63:0/1645477556 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f05700a82c0 con 0x55fb5a74c4d0 2024-05-27T08:20:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.791+0000 7f05909cb740 10 client.42680 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:20:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.791+0000 7f05909cb740 10 client.42680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:20:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.791+0000 7f05909cb740 20 client.42680 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.791+0000 7f05909cb740 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb5a72bd80 0x55fb5ab2bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.791+0000 7f05909cb740 4 client.42680 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:20:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.791+0000 7f05909cb740 1 -- 172.21.15.63:0/1645477556 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0574004240 con 0x55fb5a72bd80 2024-05-27T08:20:26.793 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.792+0000 7f0582a00640 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb5a72bd80 0x55fb5ab2bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.796+0000 7f0582a00640 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb5a72bd80 0x55fb5ab2bda0 crc :-1 s=READY pgs=790 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2576cc5c9eeb11f7 server_cookie=b08a65fb5a33ef7a in_seq=0 out_seq=0 2024-05-27T08:20:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.796+0000 7f056cc00640 10 client.42680 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.801+0000 7f056cc00640 1 -- 172.21.15.63:0/1645477556 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f0574004240 con 0x55fb5a72bd80 2024-05-27T08:20:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.801+0000 7f056cc00640 10 client.42680 handle_command_reply: tid=0 2024-05-27T08:20:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.801+0000 7f05909cb740 1 client.42680 shutdown 2024-05-27T08:20:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f05909cb740 1 -- 172.21.15.63:0/1645477556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a74c4d0 msgr2=0x55fb5ab19f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f05909cb740 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab19f60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0570009fd0 tx=0x7f0570012fc0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f05909cb740 1 -- 172.21.15.63:0/1645477556 shutdown_connections 2024-05-27T08:20:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f05909cb740 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb5a72bd80 0x55fb5ab2bda0 crc :-1 s=READY pgs=790 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f05909cb740 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab25cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f05909cb740 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab19f60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f05909cb740 1 --2- 172.21.15.63:0/1645477556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a7442f0 0x55fb5ab19a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f05909cb740 1 -- 172.21.15.63:0/1645477556 >> 172.21.15.63:0/1645477556 conn(0x55fb5a761300 msgr2=0x55fb5ab11580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f056cc00640 0 client.42680 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f05909cb740 1 -- 172.21.15.63:0/1645477556 shutdown_connections 2024-05-27T08:20:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.803+0000 7f05909cb740 1 -- 172.21.15.63:0/1645477556 wait complete. 2024-05-27T08:20:26.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.802+0000 7f53ace00700 1 -- 172.21.15.63:0/3881607426 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53a81adfb0 con 0x7f53a8103240 2024-05-27T08:20:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.803+0000 7f05909cb740 20 client.42680 trim_cache size 0 max 16384 2024-05-27T08:20:26.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.805+0000 7f539ea00700 1 -- 172.21.15.63:0/3881607426 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11862 (secure 0 0 0) 0x7f53980a93b0 con 0x7f53a8103240 2024-05-27T08:20:26.807 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:20:26.808 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1182053542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1182053542},{"type":"v1","addr":"172.21.15.149:6833","nonce":1182053542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25615,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1954751988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1954751988},{"type":"v1","addr":"172.21.15.149:6835","nonce":1954751988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T08:20:14.695071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24484":{"gid":24484,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/3113374312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":3113374312},{"type":"v1","addr":"172.21.15.149:6837","nonce":3113374312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6839/1574913181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":1574913181},{"type":"v1","addr":"172.21.15.149:6839","nonce":1574913181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_42665":{"gid":42665,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6833/3997074493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3997074493},{"type":"v1","addr":"172.21.15.86:6833","nonce":3997074493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,34454,34484]},"id":1}]} 2024-05-27T08:20:26.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.814+0000 7f53ace00700 1 -- 172.21.15.63:0/3881607426 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f538c0b2c70 msgr2=0x7f538c0b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.815+0000 7f53ace00700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f538c0b2c70 0x7f538c0b5180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5394005950 tx=0x7f53940058e0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.815+0000 7f53ace00700 1 -- 172.21.15.63:0/3881607426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103240 msgr2=0x7f53a81a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.815+0000 7f53ace00700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103240 0x7f53a81a1380 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 crypto rx=0x7f539800b530 tx=0x7f539800b8f0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.818+0000 7f53ace00700 1 -- 172.21.15.63:0/3881607426 shutdown_connections 2024-05-27T08:20:26.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.818+0000 7f53ace00700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f538c0b2c70 0x7f538c0b5180 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.818+0000 7f53ace00700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a8108690 0x7f53a81a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.818+0000 7f53ace00700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53a8103240 0x7f53a81a1380 unknown :-1 s=CLOSED pgs=2995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.818+0000 7f53ace00700 1 --2- 172.21.15.63:0/3881607426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53a8103b90 0x7f53a81a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.818+0000 7f53ace00700 1 -- 172.21.15.63:0/3881607426 >> 172.21.15.63:0/3881607426 conn(0x7f53a80fedb0 msgr2=0x7f53a8105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:26.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.819+0000 7f53ace00700 1 -- 172.21.15.63:0/3881607426 shutdown_connections 2024-05-27T08:20:26.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:26.819+0000 7f53ace00700 1 -- 172.21.15.63:0/3881607426 wait complete. 2024-05-27T08:20:26.823 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 37 2024-05-27T08:20:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.836+0000 7f05909cb740 1 Processor -- start 2024-05-27T08:20:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.840+0000 7f05909cb740 1 -- start start 2024-05-27T08:20:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.840+0000 7f05909cb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 0x55fb5ab15280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.840+0000 7f05909cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab25a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.841+0000 7f05909cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab25fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.841+0000 7f05909cb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05640129b0 con 0x55fb5a7442f0 2024-05-27T08:20:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.841+0000 7f05909cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0564010280 con 0x55fb5a76d1b0 2024-05-27T08:20:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.841+0000 7f05909cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f056400e0f0 con 0x55fb5a74c4d0 2024-05-27T08:20:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.841+0000 7f0582000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab25a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.841+0000 7f0582000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab25a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:54766/0 (socket says 172.21.15.63:54766) 2024-05-27T08:20:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.841+0000 7f0582000640 1 -- 172.21.15.63:0/1238751408 learned_addr learned my addr 172.21.15.63:0/1238751408 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.842+0000 7f0583400640 1 --2- 172.21.15.63:0/1238751408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab25fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.842+0000 7f0582a00640 1 --2- 172.21.15.63:0/1238751408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 0x55fb5ab15280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.843+0000 7f0582000640 1 -- 172.21.15.63:0/1238751408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a76d1b0 msgr2=0x55fb5ab25fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.843+0000 7f0582000640 1 --2- 172.21.15.63:0/1238751408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab25fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.843+0000 7f0582000640 1 -- 172.21.15.63:0/1238751408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 msgr2=0x55fb5ab15280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.843+0000 7f0582000640 1 --2- 172.21.15.63:0/1238751408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 0x55fb5ab15280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.843+0000 7f0582000640 1 -- 172.21.15.63:0/1238751408 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5ab1aae0 con 0x55fb5a74c4d0 2024-05-27T08:20:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.845+0000 7f0582000640 1 --2- 172.21.15.63:0/1238751408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab25a60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05700a8ff0 tx=0x7f05700df640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff63516fa59f656 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.845+0000 7f056f400640 1 -- 172.21.15.63:0/1238751408 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05700c2070 con 0x55fb5a74c4d0 2024-05-27T08:20:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.845+0000 7f056f400640 1 -- 172.21.15.63:0/1238751408 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0570087030 con 0x55fb5a74c4d0 2024-05-27T08:20:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.845+0000 7f056f400640 1 -- 172.21.15.63:0/1238751408 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05700b1ec0 con 0x55fb5a74c4d0 2024-05-27T08:20:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.847+0000 7f05909cb740 1 -- 172.21.15.63:0/1238751408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 msgr2=0x55fb5ab25a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.847+0000 7f05909cb740 1 --2- 172.21.15.63:0/1238751408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab25a60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05700a8ff0 tx=0x7f05700df640 comp rx=0 tx=0).stop 2024-05-27T08:20:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.848+0000 7f05909cb740 1 -- 172.21.15.63:0/1238751408 shutdown_connections 2024-05-27T08:20:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.848+0000 7f05909cb740 1 --2- 172.21.15.63:0/1238751408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab25fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.848+0000 7f05909cb740 1 --2- 172.21.15.63:0/1238751408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab25a60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.848+0000 7f05909cb740 1 --2- 172.21.15.63:0/1238751408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 0x55fb5ab15280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.848+0000 7f05909cb740 1 -- 172.21.15.63:0/1238751408 >> 172.21.15.63:0/1238751408 conn(0x55fb5a761300 msgr2=0x55fb5ab1b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.849+0000 7f05909cb740 1 -- 172.21.15.63:0/1238751408 shutdown_connections 2024-05-27T08:20:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.849+0000 7f05909cb740 1 -- 172.21.15.63:0/1238751408 wait complete. 2024-05-27T08:20:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.849+0000 7f05909cb740 1 -- start start 2024-05-27T08:20:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.850+0000 7f05909cb740 1 Processor -- start 2024-05-27T08:20:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.850+0000 7f05909cb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 0x55fb5ab1d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.850+0000 7f05909cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab1d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.850+0000 7f05909cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.850+0000 7f05909cb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f056400e740 con 0x55fb5a7442f0 2024-05-27T08:20:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.850+0000 7f05909cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f056400fbb0 con 0x55fb5a76d1b0 2024-05-27T08:20:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.850+0000 7f05909cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f056400ee10 con 0x55fb5a74c4d0 2024-05-27T08:20:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.851+0000 7f05909cb740 10 client.0 _ll_register_callbacks cb 0x7f0570009300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.851+0000 7f0582a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 0x55fb5ab1d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.852+0000 7f0582000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab1d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.852+0000 7f0582a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 0x55fb5ab1d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50116/0 (socket says 172.21.15.63:50116) 2024-05-27T08:20:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.852+0000 7f0582000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab1d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:54776/0 (socket says 172.21.15.63:54776) 2024-05-27T08:20:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.852+0000 7f0582a00640 1 -- 172.21.15.63:0/2684742002 learned_addr learned my addr 172.21.15.63:0/2684742002 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.852+0000 7f0583400640 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.852+0000 7f0582a00640 1 -- 172.21.15.63:0/2684742002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 msgr2=0x55fb5ab1d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.853+0000 7f0582a00640 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab1d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.853+0000 7f0582a00640 1 -- 172.21.15.63:0/2684742002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a76d1b0 msgr2=0x55fb5ab923b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.853+0000 7f0582a00640 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab923b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.853+0000 7f0582a00640 1 -- 172.21.15.63:0/2684742002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5ab928f0 con 0x55fb5a7442f0 2024-05-27T08:20:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.853+0000 7f0582a00640 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 0x55fb5ab1d020 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7f057c000f80 tx=0x7f057c011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2069030d1ea72f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.853+0000 7f056cc00640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:20:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.854+0000 7f056cc00640 1 -- 172.21.15.63:0/2684742002 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c010d70 con 0x55fb5a7442f0 2024-05-27T08:20:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.854+0000 7f056cc00640 1 -- 172.21.15.63:0/2684742002 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(176..176 src has 1..176) ==== 9715+0+0 (secure 0 0 0) 0x7f057c004b20 con 0x55fb5a7442f0 2024-05-27T08:20:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.854+0000 7f05909cb740 1 -- 172.21.15.63:0/2684742002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0570087030 con 0x55fb5a7442f0 2024-05-27T08:20:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.855+0000 7f056cc00640 1 -- 172.21.15.63:0/2684742002 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f057c004d20 con 0x55fb5a7442f0 2024-05-27T08:20:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.855+0000 7f05909cb740 10 client.23772 fetch_fsmap learned FSMap version 37 2024-05-27T08:20:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.855+0000 7f05909cb740 1 -- 172.21.15.63:0/2684742002 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fb5ab93ae0 con 0x55fb5a7442f0 2024-05-27T08:20:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.856+0000 7f056cc00640 1 -- 172.21.15.63:0/2684742002 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0570087030 con 0x55fb5a7442f0 2024-05-27T08:20:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.857+0000 7f056cc00640 1 -- 172.21.15.63:0/2684742002 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f057c024070 con 0x55fb5a7442f0 2024-05-27T08:20:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.857+0000 7f05909cb740 10 client.23772 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:20:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.857+0000 7f05909cb740 10 client.23772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:20:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.857+0000 7f05909cb740 20 client.23772 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.857+0000 7f05909cb740 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb5a7011c0 0x55fb5ab96250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.857+0000 7f05909cb740 4 client.23772 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:20:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.857+0000 7f05909cb740 1 -- 172.21.15.63:0/2684742002 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb5aa82ae0 con 0x55fb5a7011c0 2024-05-27T08:20:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.858+0000 7f0582000640 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb5a7011c0 0x55fb5ab96250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.859+0000 7f056cc00640 10 client.23772 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:26.859+0000 7f0582000640 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb5a7011c0 0x55fb5ab96250 crc :-1 s=READY pgs=791 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c299ea15035d3e8 server_cookie=a0e1c05206ae3feb in_seq=0 out_seq=0 2024-05-27T08:20:27.124 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:26 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:20:27.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:26 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:20:27.264 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:25 smithi086 ceph-mon[34687]: pgmap v1491: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.1 MiB/s wr, 1.50k op/s 2024-05-27T08:20:27.264 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:26 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:20:27.836 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:20:27.989 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:27 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3881607426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:20:27.989 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:27 smithi086 ceph-mon[34687]: pgmap v1492: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.2 MiB/s wr, 1.28k op/s 2024-05-27T08:20:28.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:27 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3881607426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:20:28.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:27 smithi149 ceph-mon[35408]: pgmap v1492: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.2 MiB/s wr, 1.28k op/s 2024-05-27T08:20:28.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:27 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3881607426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:20:28.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:27 smithi063 ceph-mon[31613]: pgmap v1492: 129 pgs: 6 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 104 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.2 MiB/s wr, 1.28k op/s 2024-05-27T08:20:29.349 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:20:29.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:29 smithi086 ceph-mon[34687]: pgmap v1493: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.2 MiB/s wr, 2.01k op/s 2024-05-27T08:20:29.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:29 smithi149 ceph-mon[35408]: pgmap v1493: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.2 MiB/s wr, 2.01k op/s 2024-05-27T08:20:29.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:29 smithi063 ceph-mon[31613]: pgmap v1493: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.2 MiB/s wr, 2.01k op/s 2024-05-27T08:20:31.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:31 smithi086 ceph-mon[34687]: pgmap v1494: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.9 MiB/s wr, 1.20k op/s 2024-05-27T08:20:31.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:31 smithi149 ceph-mon[35408]: pgmap v1494: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.9 MiB/s wr, 1.20k op/s 2024-05-27T08:20:31.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:31 smithi063 ceph-mon[31613]: pgmap v1494: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.9 MiB/s wr, 1.20k op/s 2024-05-27T08:20:32.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:20:32.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:20:32.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:32 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:20:32.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:20:32.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:20:32.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:32 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:20:32.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:20:32.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:20:32.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:32 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:20:33.259 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:33 smithi063 ceph-mon[31613]: pgmap v1495: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.5 MiB/s wr, 1.83k op/s 2024-05-27T08:20:33.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:33 smithi086 ceph-mon[34687]: pgmap v1495: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.5 MiB/s wr, 1.83k op/s 2024-05-27T08:20:33.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:33 smithi149 ceph-mon[35408]: pgmap v1495: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.5 MiB/s wr, 1.83k op/s 2024-05-27T08:20:35.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:35 smithi086 ceph-mon[34687]: pgmap v1496: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.5 MiB/s wr, 1.57k op/s 2024-05-27T08:20:35.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:35 smithi149 ceph-mon[35408]: pgmap v1496: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.5 MiB/s wr, 1.57k op/s 2024-05-27T08:20:35.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:35 smithi063 ceph-mon[31613]: pgmap v1496: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.5 MiB/s wr, 1.57k op/s 2024-05-27T08:20:37.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:37 smithi086 ceph-mon[34687]: pgmap v1497: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.5 MiB/s wr, 1.58k op/s 2024-05-27T08:20:37.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:37 smithi149 ceph-mon[35408]: pgmap v1497: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.5 MiB/s wr, 1.58k op/s 2024-05-27T08:20:37.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:37 smithi063 ceph-mon[31613]: pgmap v1497: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.5 MiB/s wr, 1.58k op/s 2024-05-27T08:20:39.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:39 smithi086 ceph-mon[34687]: pgmap v1498: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.2 MiB/s wr, 2.24k op/s 2024-05-27T08:20:39.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:39 smithi149 ceph-mon[35408]: pgmap v1498: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.2 MiB/s wr, 2.24k op/s 2024-05-27T08:20:39.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:39 smithi063 ceph-mon[31613]: pgmap v1498: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.2 MiB/s wr, 2.24k op/s 2024-05-27T08:20:40.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.406+0000 7f2b34400700 1 -- 172.21.15.63:0/3930560061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b2c1034b0 msgr2=0x7f2b2c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:40.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.406+0000 7f2b34400700 1 --2- 172.21.15.63:0/3930560061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b2c1034b0 0x7f2b2c1038c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2b180099c0 tx=0x7f2b18009cd0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.408+0000 7f2b34400700 1 -- 172.21.15.63:0/3930560061 shutdown_connections 2024-05-27T08:20:40.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.408+0000 7f2b34400700 1 --2- 172.21.15.63:0/3930560061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b2c108730 0x7f2b2c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.408+0000 7f2b34400700 1 --2- 172.21.15.63:0/3930560061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b2c1034b0 0x7f2b2c1038c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.408+0000 7f2b34400700 1 --2- 172.21.15.63:0/3930560061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c107de0 0x7f2b2c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.408+0000 7f2b34400700 1 -- 172.21.15.63:0/3930560061 >> 172.21.15.63:0/3930560061 conn(0x7f2b2c0fedb0 msgr2=0x7f2b2c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:40.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.408+0000 7f2b34400700 1 -- 172.21.15.63:0/3930560061 shutdown_connections 2024-05-27T08:20:40.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.408+0000 7f2b34400700 1 -- 172.21.15.63:0/3930560061 wait complete. 2024-05-27T08:20:40.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.410+0000 7f2b34400700 1 Processor -- start 2024-05-27T08:20:40.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.410+0000 7f2b34400700 1 -- start start 2024-05-27T08:20:40.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.411+0000 7f2b34400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b2c1034b0 0x7f2b2c075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.411+0000 7f2b34400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c107de0 0x7f2b2c06e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.411+0000 7f2b34400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b2c108730 0x7f2b2c06e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.412+0000 7f2b34400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c113280 con 0x7f2b2c107de0 2024-05-27T08:20:40.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.412+0000 7f2b34400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c113100 con 0x7f2b2c108730 2024-05-27T08:20:40.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.412+0000 7f2b34400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c113400 con 0x7f2b2c1034b0 2024-05-27T08:20:40.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.412+0000 7f2b31a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b2c1034b0 0x7f2b2c075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.412+0000 7f2b31a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b2c1034b0 0x7f2b2c075270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:58726/0 (socket says 172.21.15.63:58726) 2024-05-27T08:20:40.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.412+0000 7f2b31a00700 1 -- 172.21.15.63:0/545443409 learned_addr learned my addr 172.21.15.63:0/545443409 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:40.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.412+0000 7f2b32400700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b2c108730 0x7f2b2c06e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.413+0000 7f2b31000700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c107de0 0x7f2b2c06e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.413+0000 7f2b31a00700 1 -- 172.21.15.63:0/545443409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b2c108730 msgr2=0x7f2b2c06e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:40.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.413+0000 7f2b31a00700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b2c108730 0x7f2b2c06e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.413+0000 7f2b31a00700 1 -- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c107de0 msgr2=0x7f2b2c06e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.413+0000 7f2b31a00700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c107de0 0x7f2b2c06e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.413+0000 7f2b31a00700 1 -- 172.21.15.63:0/545443409 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c06f060 con 0x7f2b2c1034b0 2024-05-27T08:20:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.413+0000 7f2b31000700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c107de0 0x7f2b2c06e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:20:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.414+0000 7f2b31a00700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b2c1034b0 0x7f2b2c075270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2b18005910 tx=0x7f2b180045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.415+0000 7f2b2a000700 1 -- 172.21.15.63:0/545443409 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18026070 con 0x7f2b2c1034b0 2024-05-27T08:20:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.415+0000 7f2b2a000700 1 -- 172.21.15.63:0/545443409 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b1800f460 con 0x7f2b2c1034b0 2024-05-27T08:20:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.415+0000 7f2b2a000700 1 -- 172.21.15.63:0/545443409 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18009e60 con 0x7f2b2c1034b0 2024-05-27T08:20:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.415+0000 7f2b34400700 1 -- 172.21.15.63:0/545443409 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2c072da0 con 0x7f2b2c1034b0 2024-05-27T08:20:40.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.415+0000 7f2b34400700 1 -- 172.21.15.63:0/545443409 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2c073230 con 0x7f2b2c1034b0 2024-05-27T08:20:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.423+0000 7f2b2a000700 1 -- 172.21.15.63:0/545443409 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2b1802a3f0 con 0x7f2b2c1034b0 2024-05-27T08:20:40.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.426+0000 7f2b2a000700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b140b2af0 0x7f2b140b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.426+0000 7f2b2a000700 1 -- 172.21.15.63:0/545443409 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 9715+0+0 (secure 0 0 0) 0x7f2b180e45a0 con 0x7f2b2c1034b0 2024-05-27T08:20:40.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.427+0000 7f2b31000700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b140b2af0 0x7f2b140b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.449+0000 7f2b31000700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b140b2af0 0x7f2b140b5000 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2b20005fd0 tx=0x7f2b20005e20 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.449+0000 7f2b42f7cb80 1 Processor -- start 2024-05-27T08:20:40.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.450+0000 7f2b42f7cb80 1 -- start start 2024-05-27T08:20:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.450+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b06477e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.450+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064778080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.450+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b0647789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.451+0000 7f2b42f7cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b06469de40 con 0x55b0645f0e40 2024-05-27T08:20:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.451+0000 7f2b42f7cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b06468d9e0 con 0x55b06477e540 2024-05-27T08:20:40.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.451+0000 7f2b42f7cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b064691a20 con 0x55b0647785c0 2024-05-27T08:20:40.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.451+0000 7f2b31000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064778080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.451+0000 7f2b31a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b06477e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.451+0000 7f2b31000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064778080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46024/0 (socket says 172.21.15.63:46024) 2024-05-27T08:20:40.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.451+0000 7f2b31000700 1 -- 172.21.15.63:0/960826134 learned_addr learned my addr 172.21.15.63:0/960826134 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:40.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.452+0000 7f2b32400700 1 --2- 172.21.15.63:0/960826134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b0647789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.452+0000 7f2b31000700 1 -- 172.21.15.63:0/960826134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 msgr2=0x55b0647789d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.452+0000 7f2b31000700 1 --2- 172.21.15.63:0/960826134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b0647789d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.452+0000 7f2b31000700 1 -- 172.21.15.63:0/960826134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 msgr2=0x55b06477e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.452+0000 7f2b31000700 1 --2- 172.21.15.63:0/960826134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b06477e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.452+0000 7f2b31000700 1 -- 172.21.15.63:0/960826134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b064656420 con 0x55b06477e540 2024-05-27T08:20:40.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.452+0000 7f2b32400700 1 --2- 172.21.15.63:0/960826134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b0647789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.453+0000 7f2b31000700 1 --2- 172.21.15.63:0/960826134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064778080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000dbf0 tx=0x7f2b20010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b391ca3982c9e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.454+0000 7f2b1ea00700 1 -- 172.21.15.63:0/960826134 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20020070 con 0x55b06477e540 2024-05-27T08:20:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.454+0000 7f2b1ea00700 1 -- 172.21.15.63:0/960826134 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b2000fd40 con 0x55b06477e540 2024-05-27T08:20:40.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.454+0000 7f2b1ea00700 1 -- 172.21.15.63:0/960826134 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2001cd70 con 0x55b06477e540 2024-05-27T08:20:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.457+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/960826134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 msgr2=0x55b064778080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:40.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.457+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/960826134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064778080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000dbf0 tx=0x7f2b20010000 comp rx=0 tx=0).stop 2024-05-27T08:20:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.458+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/960826134 shutdown_connections 2024-05-27T08:20:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.458+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/960826134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b0647789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.458+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/960826134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b06477e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.458+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/960826134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064778080 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.458+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/960826134 >> 172.21.15.63:0/960826134 conn(0x55b064650fc0 msgr2=0x55b06476a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:40.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.458+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/960826134 shutdown_connections 2024-05-27T08:20:40.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.458+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/960826134 wait complete. 2024-05-27T08:20:40.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.459+0000 7f2b42f7cb80 1 -- start start 2024-05-27T08:20:40.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.459+0000 7f2b42f7cb80 1 Processor -- start 2024-05-27T08:20:40.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.461+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0645f0e40 0x55b064781cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.461+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b0647821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.461+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b06477e540 0x55b064782730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.462+0000 7f2b31a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0645f0e40 0x55b064781cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.462+0000 7f2b31a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0645f0e40 0x55b064781cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:46034/0 (socket says 172.21.15.63:46034) 2024-05-27T08:20:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.462+0000 7f2b32400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b06477e540 0x55b064782730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.462+0000 7f2b32400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b06477e540 0x55b064782730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54064/0 (socket says 172.21.15.63:54064) 2024-05-27T08:20:40.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.462+0000 7f2b31a00700 1 -- 172.21.15.63:0/2144101255 learned_addr learned my addr 172.21.15.63:0/2144101255 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:40.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.463+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b064593fa0 con 0x55b06477e540 2024-05-27T08:20:40.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.463+0000 7f2b31000700 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b0647821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.463+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b06462e760 con 0x55b0645f0e40 2024-05-27T08:20:40.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.463+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b064785010 con 0x55b0647785c0 2024-05-27T08:20:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.464+0000 7f2b42f7cb80 10 client.0 _ll_register_callbacks cb 0x55b064637c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.464+0000 7f2b32400700 1 -- 172.21.15.63:0/2144101255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 msgr2=0x55b0647821f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.464+0000 7f2b32400700 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b0647821f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.464+0000 7f2b32400700 1 -- 172.21.15.63:0/2144101255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0645f0e40 msgr2=0x55b064781cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.464+0000 7f2b32400700 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0645f0e40 0x55b064781cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.464+0000 7f2b32400700 1 -- 172.21.15.63:0/2144101255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0645dd8d0 con 0x55b06477e540 2024-05-27T08:20:40.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.465+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b2000fd40 con 0x55b06477e540 2024-05-27T08:20:40.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.465+0000 7f2b13400700 10 client.23778 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:20:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.466+0000 7f2b32400700 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b06477e540 0x55b064782730 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 crypto rx=0x7f2b24007b90 tx=0x7f2b24009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff66300f1e5e9a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.466+0000 7f2b13400700 1 -- 172.21.15.63:0/2144101255 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24004bb0 con 0x55b06477e540 2024-05-27T08:20:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.466+0000 7f2b13400700 1 -- 172.21.15.63:0/2144101255 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(176..176 src has 1..176) ==== 9715+0+0 (secure 0 0 0) 0x7f2b24016030 con 0x55b06477e540 2024-05-27T08:20:40.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.467+0000 7f2b13400700 1 -- 172.21.15.63:0/2144101255 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2b24004bb0 con 0x55b06477e540 2024-05-27T08:20:40.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.467+0000 7f2b42f7cb80 10 client.23778 fetch_fsmap learned FSMap version 37 2024-05-27T08:20:40.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.467+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55b06465e060 con 0x55b06477e540 2024-05-27T08:20:40.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.468+0000 7f2b13400700 1 -- 172.21.15.63:0/2144101255 <== mon.0 v2:172.21.15.63:3300/0 4 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f2b24003a00 con 0x55b06477e540 2024-05-27T08:20:40.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.468+0000 7f2b42f7cb80 10 client.23778 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:20:40.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.468+0000 7f2b42f7cb80 10 client.23778 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:20:40.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.468+0000 7f2b42f7cb80 20 client.23778 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:40.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.468+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b064791cd0 0x55b064794180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.468+0000 7f2b42f7cb80 4 client.23778 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:20:40.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.468+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b06469fd90 con 0x55b064791cd0 2024-05-27T08:20:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.469+0000 7f2b31a00700 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b064791cd0 0x55b064794180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.470+0000 7f2b31a00700 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b064791cd0 0x55b064794180 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c3ac5a7bc237fc4 server_cookie=c2d162e07622ceb8 in_seq=0 out_seq=0 2024-05-27T08:20:40.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.470+0000 7f2b13400700 10 client.23778 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:20:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.472+0000 7f2b13400700 1 -- 172.21.15.63:0/2144101255 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b06469fd90 con 0x55b064791cd0 2024-05-27T08:20:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.472+0000 7f2b13400700 10 client.23778 handle_command_reply: tid=0 2024-05-27T08:20:40.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.476+0000 7f2b42f7cb80 1 client.23778 shutdown 2024-05-27T08:20:40.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.477+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b06477e540 msgr2=0x55b064782730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.477+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b06477e540 0x55b064782730 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 crypto rx=0x7f2b24007b90 tx=0x7f2b24009920 comp rx=0 tx=0).stop 2024-05-27T08:20:40.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.478+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 shutdown_connections 2024-05-27T08:20:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.478+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b064791cd0 0x55b064794180 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.478+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b06477e540 0x55b064782730 unknown :-1 s=CLOSED pgs=2998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.478+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0645f0e40 0x55b064781cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.478+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/2144101255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b0647821f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.478+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 >> 172.21.15.63:0/2144101255 conn(0x55b064650fc0 msgr2=0x55b06476a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.478+0000 7f2b13400700 0 client.23778 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:20:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.478+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 shutdown_connections 2024-05-27T08:20:40.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.479+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2144101255 wait complete. 2024-05-27T08:20:40.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.479+0000 7f2b42f7cb80 20 client.23778 trim_cache size 0 max 16384 2024-05-27T08:20:40.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.513+0000 7f2b42f7cb80 1 Processor -- start 2024-05-27T08:20:40.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.513+0000 7f2b42f7cb80 1 -- start start 2024-05-27T08:20:40.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.514+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b064778080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.514+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b064784150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.514+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b06477c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.515+0000 7f2b42f7cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c1b92a0 con 0x55b0645f0e40 2024-05-27T08:20:40.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.515+0000 7f2b42f7cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c1b6b30 con 0x55b06477e540 2024-05-27T08:20:40.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.515+0000 7f2b42f7cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c1b49a0 con 0x55b0647785c0 2024-05-27T08:20:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.515+0000 7f2b31a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b064778080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.515+0000 7f2b31000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b064784150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.515+0000 7f2b31000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b064784150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:58736/0 (socket says 172.21.15.63:58736) 2024-05-27T08:20:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.515+0000 7f2b31000700 1 -- 172.21.15.63:0/2821391599 learned_addr learned my addr 172.21.15.63:0/2821391599 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:40.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.515+0000 7f2b32400700 1 --2- 172.21.15.63:0/2821391599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b06477c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.517+0000 7f2b31a00700 1 -- 172.21.15.63:0/2821391599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 msgr2=0x55b064784150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:40.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.517+0000 7f2b31a00700 1 --2- 172.21.15.63:0/2821391599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b064784150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.517+0000 7f2b31a00700 1 -- 172.21.15.63:0/2821391599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 msgr2=0x55b06477c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:40.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.517+0000 7f2b31a00700 1 --2- 172.21.15.63:0/2821391599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b06477c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.517+0000 7f2b31a00700 1 -- 172.21.15.63:0/2821391599 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b064780220 con 0x55b0645f0e40 2024-05-27T08:20:40.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.518+0000 7f2b31a00700 1 --2- 172.21.15.63:0/2821391599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b064778080 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7f2b180a7480 tx=0x7f2b18093040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286afc716e1bba11 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:40.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.519+0000 7f2b1e000700 1 -- 172.21.15.63:0/2821391599 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b180c5070 con 0x55b0645f0e40 2024-05-27T08:20:40.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.519+0000 7f2b1e000700 1 -- 172.21.15.63:0/2821391599 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b180b41d0 con 0x55b0645f0e40 2024-05-27T08:20:40.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.519+0000 7f2b1e000700 1 -- 172.21.15.63:0/2821391599 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b180ab660 con 0x55b0645f0e40 2024-05-27T08:20:40.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.520+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2821391599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 msgr2=0x55b064778080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:40.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.520+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/2821391599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b064778080 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7f2b180a7480 tx=0x7f2b18093040 comp rx=0 tx=0).stop 2024-05-27T08:20:40.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.521+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2821391599 shutdown_connections 2024-05-27T08:20:40.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.521+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/2821391599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b06477c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.521+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/2821391599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b064778080 unknown :-1 s=CLOSED pgs=2999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.521+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/2821391599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b064784150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.521+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2821391599 >> 172.21.15.63:0/2821391599 conn(0x55b064650fc0 msgr2=0x55b064780470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:40.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.522+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2821391599 shutdown_connections 2024-05-27T08:20:40.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.522+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/2821391599 wait complete. 2024-05-27T08:20:40.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.522+0000 7f2b42f7cb80 1 -- start start 2024-05-27T08:20:40.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.523+0000 7f2b42f7cb80 1 Processor -- start 2024-05-27T08:20:40.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.526+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b06478a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.527+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b06478a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.527+0000 7f2b42f7cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064798a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.527+0000 7f2b42f7cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c1b4ff0 con 0x55b0645f0e40 2024-05-27T08:20:40.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.528+0000 7f2b32400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064798a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.528+0000 7f2b42f7cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c1b6460 con 0x55b06477e540 2024-05-27T08:20:40.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.528+0000 7f2b31000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b06478a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.528+0000 7f2b42f7cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c1b56c0 con 0x55b0647785c0 2024-05-27T08:20:40.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.528+0000 7f2b31000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b06478a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:58744/0 (socket says 172.21.15.63:58744) 2024-05-27T08:20:40.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.528+0000 7f2b31000700 1 -- 172.21.15.63:0/3580119509 learned_addr learned my addr 172.21.15.63:0/3580119509 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:40.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.528+0000 7f2b31a00700 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b06478a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.528+0000 7f2b42f7cb80 10 client.0 _ll_register_callbacks cb 0x55b064637c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:40.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.529+0000 7f2b32400700 1 -- 172.21.15.63:0/3580119509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 msgr2=0x55b06478a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:40.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.529+0000 7f2b32400700 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b06478a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.529+0000 7f2b32400700 1 -- 172.21.15.63:0/3580119509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 msgr2=0x55b06478a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:40.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.529+0000 7f2b32400700 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b06478a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.529+0000 7f2b32400700 1 -- 172.21.15.63:0/3580119509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b064798ff0 con 0x55b06477e540 2024-05-27T08:20:40.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.530+0000 7f2b32400700 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064798a60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2b24002230 tx=0x7f2b24004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ad70c9ecddcaa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:40.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.530+0000 7f2b13400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:20:40.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.530+0000 7f2b13400700 1 -- 172.21.15.63:0/3580119509 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24032060 con 0x55b06477e540 2024-05-27T08:20:40.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.530+0000 7f2b13400700 1 -- 172.21.15.63:0/3580119509 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(176..176 src has 1..176) ==== 9715+0+0 (secure 0 0 0) 0x7f2b24016040 con 0x55b06477e540 2024-05-27T08:20:40.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.530+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/3580119509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b180b41d0 con 0x55b06477e540 2024-05-27T08:20:40.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.531+0000 7f2b13400700 1 -- 172.21.15.63:0/3580119509 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2b2402c480 con 0x55b06477e540 2024-05-27T08:20:40.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.531+0000 7f2b42f7cb80 10 client.42704 fetch_fsmap learned FSMap version 37 2024-05-27T08:20:40.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.531+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/3580119509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b0647fc000 con 0x55b06477e540 2024-05-27T08:20:40.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.534+0000 7f2b13400700 1 -- 172.21.15.63:0/3580119509 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b180b41d0 con 0x55b06477e540 2024-05-27T08:20:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.534+0000 7f2b13400700 1 -- 172.21.15.63:0/3580119509 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f2b240041b0 con 0x55b06477e540 2024-05-27T08:20:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.534+0000 7f2b42f7cb80 10 client.42704 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:20:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.534+0000 7f2b42f7cb80 10 client.42704 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:20:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.534+0000 7f2b42f7cb80 20 client.42704 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.534+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0647fc6b0 0x55b0647fead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.534+0000 7f2b42f7cb80 4 client.42704 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:20:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.534+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/3580119509 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b064699570 con 0x55b0647fc6b0 2024-05-27T08:20:40.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.535+0000 7f2b31a00700 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0647fc6b0 0x55b0647fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:40.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.536+0000 7f2b31a00700 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0647fc6b0 0x55b0647fead0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=923d14a4073a44f4 server_cookie=27beaf0990b12150 in_seq=0 out_seq=0 2024-05-27T08:20:40.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.536+0000 7f2b13400700 10 client.42704 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:20:40.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.537+0000 7f2b13400700 1 -- 172.21.15.63:0/3580119509 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b064699570 con 0x55b0647fc6b0 2024-05-27T08:20:40.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.537+0000 7f2b13400700 10 client.42704 handle_command_reply: tid=0 2024-05-27T08:20:40.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.537+0000 7f2b42f7cb80 1 client.42704 shutdown 2024-05-27T08:20:40.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.540+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/3580119509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 msgr2=0x55b064798a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:40.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.540+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064798a60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2b24002230 tx=0x7f2b24004ad0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.540+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/3580119509 shutdown_connections 2024-05-27T08:20:40.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.540+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b06477e540 0x55b064798a60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.540+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55b0647fc6b0 0x55b0647fead0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.540+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0645f0e40 0x55b06478a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.540+0000 7f2b42f7cb80 1 --2- 172.21.15.63:0/3580119509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b0647785c0 0x55b06478a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.540+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/3580119509 >> 172.21.15.63:0/3580119509 conn(0x55b064650fc0 msgr2=0x55b064781a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:40.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.540+0000 7f2b13400700 0 client.42704 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:20:40.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.540+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/3580119509 shutdown_connections 2024-05-27T08:20:40.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.541+0000 7f2b42f7cb80 1 -- 172.21.15.63:0/3580119509 wait complete. 2024-05-27T08:20:40.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.541+0000 7f2b42f7cb80 20 client.42704 trim_cache size 0 max 16384 2024-05-27T08:20:40.543 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:20:40.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.544+0000 7f2b34400700 1 -- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b140b2af0 msgr2=0x7f2b140b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:40.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.544+0000 7f2b34400700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b140b2af0 0x7f2b140b5000 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2b20005fd0 tx=0x7f2b20005e20 comp rx=0 tx=0).stop 2024-05-27T08:20:40.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.544+0000 7f2b34400700 1 -- 172.21.15.63:0/545443409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b2c1034b0 msgr2=0x7f2b2c075270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:40.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.544+0000 7f2b34400700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b2c1034b0 0x7f2b2c075270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2b18005910 tx=0x7f2b180045c0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.545+0000 7f2b34400700 1 -- 172.21.15.63:0/545443409 shutdown_connections 2024-05-27T08:20:40.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.545+0000 7f2b34400700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f2b140b2af0 0x7f2b140b5000 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.545+0000 7f2b34400700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b2c108730 0x7f2b2c06e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.545+0000 7f2b34400700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b2c1034b0 0x7f2b2c075270 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2b18005910 tx=0x7f2b180045c0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.545+0000 7f2b34400700 1 --2- 172.21.15.63:0/545443409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c107de0 0x7f2b2c06e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:40.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.545+0000 7f2b34400700 1 -- 172.21.15.63:0/545443409 >> 172.21.15.63:0/545443409 conn(0x7f2b2c0fedb0 msgr2=0x7f2b2c107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:40.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.545+0000 7f2b34400700 1 -- 172.21.15.63:0/545443409 shutdown_connections 2024-05-27T08:20:40.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:40.546+0000 7f2b34400700 1 -- 172.21.15.63:0/545443409 wait complete. 2024-05-27T08:20:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.442+0000 7f056cc00640 1 -- 172.21.15.63:0/2684742002 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fb5aa82ae0 con 0x55fb5a7011c0 2024-05-27T08:20:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.442+0000 7f056cc00640 10 client.23772 handle_command_reply: tid=0 2024-05-27T08:20:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.443+0000 7f05909cb740 1 client.23772 shutdown 2024-05-27T08:20:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.443+0000 7f05909cb740 1 -- 172.21.15.63:0/2684742002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 msgr2=0x55fb5ab1d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.443+0000 7f05909cb740 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 0x55fb5ab1d020 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7f057c000f80 tx=0x7f057c011be0 comp rx=0 tx=0).stop 2024-05-27T08:20:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.444+0000 7f05909cb740 1 -- 172.21.15.63:0/2684742002 shutdown_connections 2024-05-27T08:20:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.444+0000 7f05909cb740 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55fb5a7011c0 0x55fb5ab96250 crc :-1 s=READY pgs=791 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.444+0000 7f05909cb740 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb5a76d1b0 0x55fb5ab923b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.444+0000 7f05909cb740 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb5a74c4d0 0x55fb5ab1d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.444+0000 7f05909cb740 1 --2- 172.21.15.63:0/2684742002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fb5a7442f0 0x55fb5ab1d020 unknown :-1 s=CLOSED pgs=2997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.444+0000 7f05909cb740 1 -- 172.21.15.63:0/2684742002 >> 172.21.15.63:0/2684742002 conn(0x55fb5a761300 msgr2=0x55fb5ab18fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.444+0000 7f056cc00640 0 client.23772 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.444+0000 7f05909cb740 1 -- 172.21.15.63:0/2684742002 shutdown_connections 2024-05-27T08:20:41.448 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.445+0000 7f05909cb740 1 -- 172.21.15.63:0/2684742002 wait complete. 2024-05-27T08:20:41.449 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.445+0000 7f05909cb740 20 client.23772 trim_cache size 0 max 16384 2024-05-27T08:20:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:20:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:20:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:20:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.447+0000 7f0589a00640 1 -- 172.21.15.63:0/1343685633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f05680b29b0 msgr2=0x7f05680b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.447+0000 7f0589a00640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f05680b29b0 0x7f05680b4ec0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f057402a9b0 tx=0x7f0574005fd0 comp rx=0 tx=0).stop 2024-05-27T08:20:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.447+0000 7f0589a00640 1 -- 172.21.15.63:0/1343685633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f058410ea80 msgr2=0x7f05841a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:41.451 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.447+0000 7f0589a00640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f058410ea80 0x7f05841a1340 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f057000b450 tx=0x7f057000b910 comp rx=0 tx=0).stop 2024-05-27T08:20:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.447+0000 7f0589a00640 1 -- 172.21.15.63:0/1343685633 shutdown_connections 2024-05-27T08:20:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.447+0000 7f0589a00640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f05680b29b0 0x7f05680b4ec0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.447+0000 7f0589a00640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f058410f540 0x7f05841a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.447+0000 7f0589a00640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f058410ea80 0x7f05841a1340 unknown :-1 s=CLOSED pgs=2996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.447+0000 7f0589a00640 1 --2- 172.21.15.63:0/1343685633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f058410a500 0x7f05841a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.447+0000 7f0589a00640 1 -- 172.21.15.63:0/1343685633 >> 172.21.15.63:0/1343685633 conn(0x7f0584077190 msgr2=0x7f058407a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.448+0000 7f0589a00640 1 -- 172.21.15.63:0/1343685633 shutdown_connections 2024-05-27T08:20:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:41.448+0000 7f0589a00640 1 -- 172.21.15.63:0/1343685633 wait complete. 2024-05-27T08:20:41.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:41 smithi086 ceph-mon[34687]: pgmap v1499: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 1.47k op/s 2024-05-27T08:20:41.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:41 smithi149 ceph-mon[35408]: pgmap v1499: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 1.47k op/s 2024-05-27T08:20:41.470 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:41 smithi063 ceph-mon[31613]: pgmap v1499: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 1.47k op/s 2024-05-27T08:20:43.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:43 smithi086 ceph-mon[34687]: pgmap v1500: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 6.9 MiB/s wr, 1.65k op/s 2024-05-27T08:20:43.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:43 smithi149 ceph-mon[35408]: pgmap v1500: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 6.9 MiB/s wr, 1.65k op/s 2024-05-27T08:20:43.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:43 smithi063 ceph-mon[31613]: pgmap v1500: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 6.9 MiB/s wr, 1.65k op/s 2024-05-27T08:20:45.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:45 smithi086 ceph-mon[34687]: pgmap v1501: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.6 MiB/s wr, 1.60k op/s 2024-05-27T08:20:45.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:45 smithi149 ceph-mon[35408]: pgmap v1501: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.6 MiB/s wr, 1.60k op/s 2024-05-27T08:20:45.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:45 smithi063 ceph-mon[31613]: pgmap v1501: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.6 MiB/s wr, 1.60k op/s 2024-05-27T08:20:46.774 INFO:tasks.workunit.client.0.smithi063.stdout:Syncing()...3 sec 2024-05-27T08:20:46.774 INFO:tasks.workunit.client.0.smithi063.stdout:FFSB benchmark finished at: Mon May 27 08:20:46 2024 2024-05-27T08:20:46.774 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.774 INFO:tasks.workunit.client.0.smithi063.stdout:Results: 2024-05-27T08:20:46.774 INFO:tasks.workunit.client.0.smithi063.stdout:Benchmark took 325.51 sec 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout:Total Results 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout:=============== 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout: ======= ============ ========= ======= =========== ========== 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout: write : 802560 2465.56 100.000% 100.000% 9.63MB/sec 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout:- 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout:2465.56 Transactions per Second 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout:Throughput Results 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout:=================== 2024-05-27T08:20:46.775 INFO:tasks.workunit.client.0.smithi063.stdout:Write Throughput: 9.63MB/sec 2024-05-27T08:20:46.784 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.784 INFO:tasks.workunit.client.0.smithi063.stdout:System Call Latency statistics in millisecs 2024-05-27T08:20:46.784 INFO:tasks.workunit.client.0.smithi063.stdout:===== 2024-05-27T08:20:46.784 INFO:tasks.workunit.client.0.smithi063.stdout: Min Avg Max Total Calls 2024-05-27T08:20:46.784 INFO:tasks.workunit.client.0.smithi063.stdout: ======== ======== ======== ============ 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout:[ open] 0.715000 501.480292 33697.359375 627 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[0] 5000.000000 - 10000.000000 : 6 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[2] 1000.000000 - 2000.000000 : 1 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[3] 500.000000 - 1000.000000 : 1 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[5] 100.000000 - 200.000000 : 15 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[6] 50.000000 - 100.000000 : 22 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[7] 20.000000 - 50.000000 : 179 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[8] 10.000000 - 20.000000 : 226 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[9] 5.000000 - 10.000000 : 138 2024-05-27T08:20:46.785 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[10] 2.000000 - 5.000000 : 23 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[11] 1.000000 - 2.000000 : 4 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[12] 0.500000 - 1.000000 : 1 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout:[ write] 0.183000 12.081529 72795.804688 802560 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[0] 5000.000000 - 10000.000000 : 10 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[1] 2000.000000 - 5000.000000 : 14 2024-05-27T08:20:46.786 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[2] 1000.000000 - 2000.000000 : 2 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[3] 500.000000 - 1000.000000 : 12 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[4] 200.000000 - 500.000000 : 573 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[5] 100.000000 - 200.000000 : 831 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[6] 50.000000 - 100.000000 : 3345 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[7] 20.000000 - 50.000000 : 46895 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[8] 10.000000 - 20.000000 : 109977 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[9] 5.000000 - 10.000000 : 137086 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[10] 2.000000 - 5.000000 : 141455 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[11] 1.000000 - 2.000000 : 96626 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[12] 0.500000 - 1.000000 : 139944 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[13] 0.200000 - 0.500000 : 125503 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[14] 0.100000 - 0.200000 : 137 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-05-27T08:20:46.787 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout:[ lseek] 0.000000 0.002103 4.819000 802560 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-05-27T08:20:46.788 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[10] 2.000000 - 5.000000 : 23 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[11] 1.000000 - 2.000000 : 32 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[12] 0.500000 - 1.000000 : 60 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[13] 0.200000 - 0.500000 : 155 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[14] 0.100000 - 0.200000 : 159 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[15] 0.050000 - 0.100000 : 181 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[16] 0.020000 - 0.050000 : 961 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[17] 0.010000 - 0.020000 : 1082 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[18] 0.000000 - 0.010000 : 799907 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout:[ close] 0.039000 2.662255 198.268997 627 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-27T08:20:46.789 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[7] 20.000000 - 50.000000 : 14 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[8] 10.000000 - 20.000000 : 23 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[9] 5.000000 - 10.000000 : 47 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[10] 2.000000 - 5.000000 : 82 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[11] 1.000000 - 2.000000 : 52 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[12] 0.500000 - 1.000000 : 47 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[13] 0.200000 - 0.500000 : 106 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[14] 0.100000 - 0.200000 : 85 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[15] 0.050000 - 0.100000 : 166 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[16] 0.020000 - 0.050000 : 4 2024-05-27T08:20:46.790 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-05-27T08:20:46.791 INFO:tasks.workunit.client.0.smithi063.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-05-27T08:20:46.791 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.791 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.791 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.791 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-27T08:20:46.791 INFO:tasks.workunit.client.0.smithi063.stdout:0.5% User Time 2024-05-27T08:20:46.791 INFO:tasks.workunit.client.0.smithi063.stdout:23.6% System Time 2024-05-27T08:20:46.791 INFO:tasks.workunit.client.0.smithi063.stdout:24.1% CPU Utilization 2024-05-27T08:20:46.792 INFO:tasks.workunit.client.0.smithi063.stderr:+ cd .. 2024-05-27T08:20:46.806 INFO:tasks.workunit.client.0.smithi063.stderr:+ rm -r tmp ffsb 2024-05-27T08:20:47.124 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:20:47.698 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:20:47.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:47 smithi149 ceph-mon[35408]: osdmap e177: 12 total, 12 up, 12 in 2024-05-27T08:20:47.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:47 smithi149 ceph-mon[35408]: pgmap v1503: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.7 MiB/s wr, 1.85k op/s 2024-05-27T08:20:47.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:47 smithi063 ceph-mon[31613]: osdmap e177: 12 total, 12 up, 12 in 2024-05-27T08:20:47.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:47 smithi063 ceph-mon[31613]: pgmap v1503: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.7 MiB/s wr, 1.85k op/s 2024-05-27T08:20:47.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:47 smithi086 ceph-mon[34687]: osdmap e177: 12 total, 12 up, 12 in 2024-05-27T08:20:47.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:47 smithi086 ceph-mon[34687]: pgmap v1503: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.7 MiB/s wr, 1.85k op/s 2024-05-27T08:20:49.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:49 smithi086 ceph-mon[34687]: pgmap v1504: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.8 MiB/s wr, 1.62k op/s 2024-05-27T08:20:49.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:49 smithi149 ceph-mon[35408]: pgmap v1504: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.8 MiB/s wr, 1.62k op/s 2024-05-27T08:20:49.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:49 smithi063 ceph-mon[31613]: pgmap v1504: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.8 MiB/s wr, 1.62k op/s 2024-05-27T08:20:49.933 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f6a35000700 time 2024-05-27T08:20:49.582920+0000 2024-05-27T08:20:49.933 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:49.933 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:49.933 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6a4a76fba6] 2024-05-27T08:20:49.933 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6a4a76fdd1] 2024-05-27T08:20:49.933 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55edafbfb23d] 2024-05-27T08:20:49.933 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55edafbe6971] 2024-05-27T08:20:49.933 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 5: (SnapRealm::check_cache() const+0x13e) [0x55edafbe4fa0] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55edafbe5651] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55edafb7a347] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55edaf87687e] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55edaf87be94] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55edaf886969] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 11: (MDLog::_replay_thread()+0x1f38) [0x55edafc1f48c] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55edafc289cb] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 13: (Thread::entry_wrapper()+0x43) [0x7f6a4a73e72d] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 14: (Thread::_entry_func(void*)+0xd) [0x7f6a4a73e749] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6a492e21ca] 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 16: clone() 2024-05-27T08:20:49.934 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.585+0000 7f6a35000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f6a35000700 time 2024-05-27T08:20:49.582920+0000 2024-05-27T08:20:49.935 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:49.935 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.935 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:49.935 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6a4a76fba6] 2024-05-27T08:20:49.935 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6a4a76fdd1] 2024-05-27T08:20:49.935 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55edafbfb23d] 2024-05-27T08:20:49.935 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55edafbe6971] 2024-05-27T08:20:49.935 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 5: (SnapRealm::check_cache() const+0x13e) [0x55edafbe4fa0] 2024-05-27T08:20:49.935 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55edafbe5651] 2024-05-27T08:20:49.935 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55edafb7a347] 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55edaf87687e] 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55edaf87be94] 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55edaf886969] 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 11: (MDLog::_replay_thread()+0x1f38) [0x55edafc1f48c] 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55edafc289cb] 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 13: (Thread::entry_wrapper()+0x43) [0x7f6a4a73e72d] 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 14: (Thread::_entry_func(void*)+0xd) [0x7f6a4a73e749] 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6a492e21ca] 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 16: clone() 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: *** Caught signal (Aborted) ** 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: in thread 7f6a35000700 thread_name:md_log_replay 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:49.936 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6a492ecd20] 2024-05-27T08:20:49.937 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2: gsignal() 2024-05-27T08:20:49.937 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 3: abort() 2024-05-27T08:20:49.937 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f6a4a76fced] 2024-05-27T08:20:49.937 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6a4a76fdd1] 2024-05-27T08:20:49.937 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55edafbfb23d] 2024-05-27T08:20:49.937 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55edafbe6971] 2024-05-27T08:20:49.937 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 8: (SnapRealm::check_cache() const+0x13e) [0x55edafbe4fa0] 2024-05-27T08:20:49.937 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55edafbe5651] 2024-05-27T08:20:49.937 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55edafb7a347] 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55edaf87687e] 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55edaf87be94] 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55edaf886969] 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 14: (MDLog::_replay_thread()+0x1f38) [0x55edafc1f48c] 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55edafc289cb] 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 16: (Thread::entry_wrapper()+0x43) [0x7f6a4a73e72d] 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 17: (Thread::_entry_func(void*)+0xd) [0x7f6a4a73e749] 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6a492e21ca] 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 19: clone() 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.590+0000 7f6a35000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: in thread 7f6a35000700 thread_name:md_log_replay 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:49.938 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6a492ecd20] 2024-05-27T08:20:49.939 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2: gsignal() 2024-05-27T08:20:49.939 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 3: abort() 2024-05-27T08:20:49.939 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f6a4a76fced] 2024-05-27T08:20:49.939 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6a4a76fdd1] 2024-05-27T08:20:49.939 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55edafbfb23d] 2024-05-27T08:20:49.939 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55edafbe6971] 2024-05-27T08:20:49.939 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 8: (SnapRealm::check_cache() const+0x13e) [0x55edafbe4fa0] 2024-05-27T08:20:49.939 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55edafbe5651] 2024-05-27T08:20:49.939 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55edafb7a347] 2024-05-27T08:20:49.939 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55edaf87687e] 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55edaf87be94] 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55edaf886969] 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 14: (MDLog::_replay_thread()+0x1f38) [0x55edafc1f48c] 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55edafc289cb] 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 16: (Thread::entry_wrapper()+0x43) [0x7f6a4a73e72d] 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 17: (Thread::_entry_func(void*)+0xd) [0x7f6a4a73e749] 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6a492e21ca] 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 19: clone() 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: -1> 2024-05-27T08:20:49.585+0000 7f6a35000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f6a35000700 time 2024-05-27T08:20:49.582920+0000 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:49.940 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.941 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:49.941 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6a4a76fba6] 2024-05-27T08:20:49.941 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6a4a76fdd1] 2024-05-27T08:20:49.941 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55edafbfb23d] 2024-05-27T08:20:49.941 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55edafbe6971] 2024-05-27T08:20:49.941 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 5: (SnapRealm::check_cache() const+0x13e) [0x55edafbe4fa0] 2024-05-27T08:20:49.941 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55edafbe5651] 2024-05-27T08:20:49.941 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55edafb7a347] 2024-05-27T08:20:49.941 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55edaf87687e] 2024-05-27T08:20:49.941 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55edaf87be94] 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55edaf886969] 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 11: (MDLog::_replay_thread()+0x1f38) [0x55edafc1f48c] 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55edafc289cb] 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 13: (Thread::entry_wrapper()+0x43) [0x7f6a4a73e72d] 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 14: (Thread::_entry_func(void*)+0xd) [0x7f6a4a73e749] 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6a492e21ca] 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 16: clone() 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 0> 2024-05-27T08:20:49.590+0000 7f6a35000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: in thread 7f6a35000700 thread_name:md_log_replay 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:49.942 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6a492ecd20] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2: gsignal() 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 3: abort() 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f6a4a76fced] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6a4a76fdd1] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55edafbfb23d] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55edafbe6971] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 8: (SnapRealm::check_cache() const+0x13e) [0x55edafbe4fa0] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55edafbe5651] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55edafb7a347] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55edaf87687e] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55edaf87be94] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55edaf886969] 2024-05-27T08:20:49.943 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 14: (MDLog::_replay_thread()+0x1f38) [0x55edafc1f48c] 2024-05-27T08:20:49.944 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55edafc289cb] 2024-05-27T08:20:49.944 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 16: (Thread::entry_wrapper()+0x43) [0x7f6a4a73e72d] 2024-05-27T08:20:49.944 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 17: (Thread::_entry_func(void*)+0xd) [0x7f6a4a73e749] 2024-05-27T08:20:49.944 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6a492e21ca] 2024-05-27T08:20:49.944 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 19: clone() 2024-05-27T08:20:49.944 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:20:49.944 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.944 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: -1> 2024-05-27T08:20:49.585+0000 7f6a35000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f6a35000700 time 2024-05-27T08:20:49.582920+0000 2024-05-27T08:20:49.944 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:49.944 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6a4a76fba6] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6a4a76fdd1] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55edafbfb23d] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55edafbe6971] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 5: (SnapRealm::check_cache() const+0x13e) [0x55edafbe4fa0] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55edafbe5651] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55edafb7a347] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55edaf87687e] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55edaf87be94] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55edaf886969] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 11: (MDLog::_replay_thread()+0x1f38) [0x55edafc1f48c] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55edafc289cb] 2024-05-27T08:20:49.945 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 13: (Thread::entry_wrapper()+0x43) [0x7f6a4a73e72d] 2024-05-27T08:20:49.946 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 14: (Thread::_entry_func(void*)+0xd) [0x7f6a4a73e749] 2024-05-27T08:20:49.946 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6a492e21ca] 2024-05-27T08:20:49.946 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 16: clone() 2024-05-27T08:20:49.946 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.946 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 0> 2024-05-27T08:20:49.590+0000 7f6a35000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:20:49.946 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: in thread 7f6a35000700 thread_name:md_log_replay 2024-05-27T08:20:49.946 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.946 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:49.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6a492ecd20] 2024-05-27T08:20:49.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2: gsignal() 2024-05-27T08:20:49.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 3: abort() 2024-05-27T08:20:49.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f6a4a76fced] 2024-05-27T08:20:49.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6a4a76fdd1] 2024-05-27T08:20:49.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55edafbfb23d] 2024-05-27T08:20:49.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55edafbe6971] 2024-05-27T08:20:49.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 8: (SnapRealm::check_cache() const+0x13e) [0x55edafbe4fa0] 2024-05-27T08:20:49.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55edafbe5651] 2024-05-27T08:20:49.947 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55edafb7a347] 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55edaf87687e] 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55edaf87be94] 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55edaf886969] 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 14: (MDLog::_replay_thread()+0x1f38) [0x55edafc1f48c] 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55edafc289cb] 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 16: (Thread::entry_wrapper()+0x43) [0x7f6a4a73e72d] 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 17: (Thread::_entry_func(void*)+0xd) [0x7f6a4a73e749] 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6a492e21ca] 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 19: clone() 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[76219]: 2024-05-27T08:20:49.948 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:49 smithi086 podman[80644]: 2024-05-27 08:20:49.930940788 +0000 UTC m=+0.096356251 container died 7af8edee5dccf7a0c451139f94be84fc3cab96a0b2315797c20f02f0610944a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T08:20:50.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.261+0000 7f620c000700 1 -- 172.21.15.63:0/818863850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6204109860 msgr2=0x7f62041111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:50.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.261+0000 7f620c000700 1 --2- 172.21.15.63:0/818863850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6204109860 0x7f62041111c0 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00b0d0 tx=0x7f61fc00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.261+0000 7f620c000700 1 -- 172.21.15.63:0/818863850 shutdown_connections 2024-05-27T08:20:50.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.261+0000 7f620c000700 1 --2- 172.21.15.63:0/818863850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6204109860 0x7f62041111c0 unknown :-1 s=CLOSED pgs=3000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.261+0000 7f620c000700 1 --2- 172.21.15.63:0/818863850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62041045e0 0x7f62041049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.261+0000 7f620c000700 1 --2- 172.21.15.63:0/818863850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6204108f10 0x7f6204109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.261+0000 7f620c000700 1 -- 172.21.15.63:0/818863850 >> 172.21.15.63:0/818863850 conn(0x7f6204100030 msgr2=0x7f6204102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:50.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.261+0000 7f620c000700 1 -- 172.21.15.63:0/818863850 shutdown_connections 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.261+0000 7f620c000700 1 -- 172.21.15.63:0/818863850 wait complete. 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.263+0000 7f620c000700 1 Processor -- start 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.263+0000 7f620c000700 1 -- start start 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.264+0000 7f620c000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62041045e0 0x7f620419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.264+0000 7f620c000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6204108f10 0x7f620419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.264+0000 7f620c000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6204109860 0x7f62041a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.264+0000 7f620c000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62041143c0 con 0x7f6204109860 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.264+0000 7f620c000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6204114240 con 0x7f62041045e0 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.264+0000 7f620c000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6204114540 con 0x7f6204108f10 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.265+0000 7f6208c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6204108f10 0x7f620419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.265+0000 7f6209600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62041045e0 0x7f620419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.265+0000 7f620a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6204109860 0x7f62041a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.265+0000 7f6208c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6204108f10 0x7f620419d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:37618/0 (socket says 172.21.15.63:37618) 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.265+0000 7f6208c00700 1 -- 172.21.15.63:0/485018252 learned_addr learned my addr 172.21.15.63:0/485018252 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.265+0000 7f620a000700 1 -- 172.21.15.63:0/485018252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6204108f10 msgr2=0x7f620419d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.265+0000 7f620a000700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6204108f10 0x7f620419d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.265+0000 7f620a000700 1 -- 172.21.15.63:0/485018252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62041045e0 msgr2=0x7f620419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.265+0000 7f620a000700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62041045e0 0x7f620419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.265+0000 7f620a000700 1 -- 172.21.15.63:0/485018252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62041a1e80 con 0x7f6204109860 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.266+0000 7f620a000700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6204109860 0x7f62041a17e0 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00b0d0 tx=0x7f61fc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.266+0000 7f6202000700 1 -- 172.21.15.63:0/485018252 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc01f070 con 0x7f6204109860 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.266+0000 7f620c000700 1 -- 172.21.15.63:0/485018252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62041a2110 con 0x7f6204109860 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.266+0000 7f6202000700 1 -- 172.21.15.63:0/485018252 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61fc005500 con 0x7f6204109860 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.266+0000 7f6202000700 1 -- 172.21.15.63:0/485018252 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc01b690 con 0x7f6204109860 2024-05-27T08:20:50.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.266+0000 7f620c000700 1 -- 172.21.15.63:0/485018252 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62041a9a90 con 0x7f6204109860 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.270+0000 7f6202000700 1 -- 172.21.15.63:0/485018252 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f61fc022070 con 0x7f6204109860 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.271+0000 7f6202000700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61ec0b2a70 0x7f61ec0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.272+0000 7f6202000700 1 -- 172.21.15.63:0/485018252 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 9651+0+0 (secure 0 0 0) 0x7f61fc00f020 con 0x7f6204109860 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.272+0000 7f6209600700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61ec0b2a70 0x7f61ec0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.273+0000 7f6209600700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61ec0b2a70 0x7f61ec0b4f80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009a10 tx=0x7f61f8009190 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.438+0000 7f621ac30b80 1 Processor -- start 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.438+0000 7f621ac30b80 1 -- start start 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.439+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5837fce40 0x55a5839817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.439+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a583981d30 0x55a58397b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.439+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a58397bdb0 0x55a58397c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.439+0000 7f621ac30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a5838a9e40 con 0x55a5837fce40 2024-05-27T08:20:50.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.439+0000 7f621ac30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5838999e0 con 0x55a583981d30 2024-05-27T08:20:50.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.439+0000 7f621ac30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a58389da20 con 0x55a58397bdb0 2024-05-27T08:20:50.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.439+0000 7f6209600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5837fce40 0x55a5839817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.440+0000 7f6209600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5837fce40 0x55a5839817f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39126/0 (socket says 172.21.15.63:39126) 2024-05-27T08:20:50.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.440+0000 7f6209600700 1 -- 172.21.15.63:0/2280406714 learned_addr learned my addr 172.21.15.63:0/2280406714 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:50.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.440+0000 7f6208c00700 1 --2- 172.21.15.63:0/2280406714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a583981d30 0x55a58397b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.440+0000 7f620a000700 1 --2- 172.21.15.63:0/2280406714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a58397bdb0 0x55a58397c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.440+0000 7f6209600700 1 -- 172.21.15.63:0/2280406714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a58397bdb0 msgr2=0x55a58397c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:50.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.440+0000 7f6209600700 1 --2- 172.21.15.63:0/2280406714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a58397bdb0 0x55a58397c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.440+0000 7f6209600700 1 -- 172.21.15.63:0/2280406714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a583981d30 msgr2=0x55a58397b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:50.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.440+0000 7f6209600700 1 --2- 172.21.15.63:0/2280406714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a583981d30 0x55a58397b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.440+0000 7f6209600700 1 -- 172.21.15.63:0/2280406714 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a583862420 con 0x55a5837fce40 2024-05-27T08:20:50.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.441+0000 7f6209600700 1 --2- 172.21.15.63:0/2280406714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5837fce40 0x55a5839817f0 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7f61f800ea00 tx=0x7f61f8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b03cad7400b7bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:50.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.441+0000 7f61f6a00700 1 -- 172.21.15.63:0/2280406714 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f801b040 con 0x55a5837fce40 2024-05-27T08:20:50.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.441+0000 7f61f6a00700 1 -- 172.21.15.63:0/2280406714 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61f8024070 con 0x55a5837fce40 2024-05-27T08:20:50.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.441+0000 7f61f6a00700 1 -- 172.21.15.63:0/2280406714 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8013650 con 0x55a5837fce40 2024-05-27T08:20:50.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.443+0000 7f621ac30b80 1 -- 172.21.15.63:0/2280406714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5837fce40 msgr2=0x55a5839817f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:50.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.443+0000 7f621ac30b80 1 --2- 172.21.15.63:0/2280406714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5837fce40 0x55a5839817f0 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7f61f800ea00 tx=0x7f61f8011000 comp rx=0 tx=0).stop 2024-05-27T08:20:50.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.443+0000 7f621ac30b80 1 -- 172.21.15.63:0/2280406714 shutdown_connections 2024-05-27T08:20:50.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.443+0000 7f621ac30b80 1 --2- 172.21.15.63:0/2280406714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a58397bdb0 0x55a58397c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.443+0000 7f621ac30b80 1 --2- 172.21.15.63:0/2280406714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5837fce40 0x55a5839817f0 unknown :-1 s=CLOSED pgs=3002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.443+0000 7f621ac30b80 1 --2- 172.21.15.63:0/2280406714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a583981d30 0x55a58397b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.443+0000 7f621ac30b80 1 -- 172.21.15.63:0/2280406714 >> 172.21.15.63:0/2280406714 conn(0x55a58385cfc0 msgr2=0x55a5839768d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:50.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.443+0000 7f621ac30b80 1 -- 172.21.15.63:0/2280406714 shutdown_connections 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.443+0000 7f621ac30b80 1 -- 172.21.15.63:0/2280406714 wait complete. 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.444+0000 7f621ac30b80 1 -- start start 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.445+0000 7f621ac30b80 1 Processor -- start 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.445+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5837fce40 0x55a5839854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.445+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a58397bdb0 0x55a583985a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.445+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a583981d30 0x55a583985f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.445+0000 7f621ac30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a58379ffa0 con 0x55a583981d30 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.446+0000 7f621ac30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a58383a760 con 0x55a58397bdb0 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.446+0000 7f621ac30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a583988800 con 0x55a5837fce40 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.446+0000 7f621ac30b80 10 client.0 _ll_register_callbacks cb 0x55a583843c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.446+0000 7f620a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a583981d30 0x55a583985f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.446+0000 7f620a000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a583981d30 0x55a583985f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39142/0 (socket says 172.21.15.63:39142) 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.446+0000 7f620a000700 1 -- 172.21.15.63:0/3437844353 learned_addr learned my addr 172.21.15.63:0/3437844353 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.446+0000 7f6208c00700 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a58397bdb0 0x55a583985a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.446+0000 7f6209600700 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5837fce40 0x55a5839854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.447+0000 7f620a000700 1 -- 172.21.15.63:0/3437844353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5837fce40 msgr2=0x55a5839854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.447+0000 7f620a000700 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5837fce40 0x55a5839854f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.447+0000 7f620a000700 1 -- 172.21.15.63:0/3437844353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a58397bdb0 msgr2=0x55a583985a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.447+0000 7f620a000700 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a58397bdb0 0x55a583985a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.447+0000 7f620a000700 1 -- 172.21.15.63:0/3437844353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5837e98d0 con 0x55a583981d30 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.447+0000 7f620a000700 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a583981d30 0x55a583985f70 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0a6cd0 tx=0x7f61fc0aa3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcc655f34f79983 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.447+0000 7f61eb400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.447+0000 7f61eb400700 1 -- 172.21.15.63:0/3437844353 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc092040 con 0x55a583981d30 2024-05-27T08:20:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.448+0000 7f61eb400700 1 -- 172.21.15.63:0/3437844353 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(177..177 src has 1..177) ==== 9651+0+0 (secure 0 0 0) 0x7f61fc0ab0c0 con 0x55a583981d30 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.448+0000 7f621ac30b80 1 -- 172.21.15.63:0/3437844353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61f8024070 con 0x55a583981d30 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.448+0000 7f61eb400700 1 -- 172.21.15.63:0/3437844353 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f61fc0aa760 con 0x55a583981d30 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.448+0000 7f621ac30b80 10 client.23796 fetch_fsmap learned FSMap version 37 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.448+0000 7f621ac30b80 1 -- 172.21.15.63:0/3437844353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a58386a060 con 0x55a583981d30 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.448+0000 7f61eb400700 1 -- 172.21.15.63:0/3437844353 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61f8024070 con 0x55a583981d30 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.449+0000 7f61eb400700 1 -- 172.21.15.63:0/3437844353 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f61fc0b3040 con 0x55a583981d30 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.449+0000 7f621ac30b80 10 client.23796 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.449+0000 7f621ac30b80 10 client.23796 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.449+0000 7f621ac30b80 20 client.23796 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.449+0000 7f621ac30b80 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55a583995670 0x55a583997ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.449+0000 7f621ac30b80 4 client.23796 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:20:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.449+0000 7f621ac30b80 1 -- 172.21.15.63:0/3437844353 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5838abd90 con 0x55a583995670 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.450+0000 7f6209600700 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55a583995670 0x55a583997ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.451+0000 7f61eb400700 10 client.23796 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.451+0000 7f6209600700 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55a583995670 0x55a583997ac0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcf1b25b0c25926d server_cookie=757e243a0301e787 in_seq=0 out_seq=0 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f61eb400700 1 -- 172.21.15.63:0/3437844353 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a5838abd90 con 0x55a583995670 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f61eb400700 10 client.23796 handle_command_reply: tid=0 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f621ac30b80 1 client.23796 shutdown 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f621ac30b80 1 -- 172.21.15.63:0/3437844353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a583981d30 msgr2=0x55a583985f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f621ac30b80 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a583981d30 0x55a583985f70 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0a6cd0 tx=0x7f61fc0aa3c0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f621ac30b80 1 -- 172.21.15.63:0/3437844353 shutdown_connections 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f621ac30b80 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55a583995670 0x55a583997ac0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f621ac30b80 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a583981d30 0x55a583985f70 unknown :-1 s=CLOSED pgs=3003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f621ac30b80 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a5837fce40 0x55a5839854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f621ac30b80 1 --2- 172.21.15.63:0/3437844353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a58397bdb0 0x55a583985a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f621ac30b80 1 -- 172.21.15.63:0/3437844353 >> 172.21.15.63:0/3437844353 conn(0x55a58385cfc0 msgr2=0x55a583976580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.454+0000 7f61eb400700 0 client.23796 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:20:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.455+0000 7f621ac30b80 1 -- 172.21.15.63:0/3437844353 shutdown_connections 2024-05-27T08:20:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.455+0000 7f621ac30b80 1 -- 172.21.15.63:0/3437844353 wait complete. 2024-05-27T08:20:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.455+0000 7f621ac30b80 20 client.23796 trim_cache size 0 max 16384 2024-05-27T08:20:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.475+0000 7f621ac30b80 1 Processor -- start 2024-05-27T08:20:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.476+0000 7f621ac30b80 1 -- start start 2024-05-27T08:20:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.476+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 0x55a58397b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.476+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a583984470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.476+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a583981d30 0x55a583987990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.476+0000 7f621ac30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62041b51a0 con 0x55a58397bdb0 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.476+0000 7f621ac30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62041b2a30 con 0x55a5837fce40 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.476+0000 7f621ac30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62041b0880 con 0x55a583981d30 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.477+0000 7f6208c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a583984470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.477+0000 7f6208c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a583984470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39144/0 (socket says 172.21.15.63:39144) 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.477+0000 7f6208c00700 1 -- 172.21.15.63:0/2013645339 learned_addr learned my addr 172.21.15.63:0/2013645339 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.477+0000 7f6209600700 1 --2- 172.21.15.63:0/2013645339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 0x55a58397b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.477+0000 7f620a000700 1 --2- 172.21.15.63:0/2013645339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a583981d30 0x55a583987990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.477+0000 7f6208c00700 1 -- 172.21.15.63:0/2013645339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a583981d30 msgr2=0x55a583987990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.477+0000 7f6208c00700 1 --2- 172.21.15.63:0/2013645339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a583981d30 0x55a583987990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.477+0000 7f6208c00700 1 -- 172.21.15.63:0/2013645339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 msgr2=0x55a58397b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.477+0000 7f6208c00700 1 --2- 172.21.15.63:0/2013645339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 0x55a58397b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.478+0000 7f6208c00700 1 -- 172.21.15.63:0/2013645339 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5839862e0 con 0x55a58397bdb0 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.478+0000 7f6208c00700 1 --2- 172.21.15.63:0/2013645339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a583984470 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 crypto rx=0x7f61f0004070 tx=0x7f61f0004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647a5257a38356f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.478+0000 7f61f6000700 1 -- 172.21.15.63:0/2013645339 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0003b30 con 0x55a58397bdb0 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.479+0000 7f61f6000700 1 -- 172.21.15.63:0/2013645339 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61f0020070 con 0x55a58397bdb0 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.479+0000 7f61f6000700 1 -- 172.21.15.63:0/2013645339 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f001b400 con 0x55a58397bdb0 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.480+0000 7f621ac30b80 1 -- 172.21.15.63:0/2013645339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 msgr2=0x55a583984470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.480+0000 7f621ac30b80 1 --2- 172.21.15.63:0/2013645339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a583984470 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 crypto rx=0x7f61f0004070 tx=0x7f61f0004380 comp rx=0 tx=0).stop 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.480+0000 7f621ac30b80 1 -- 172.21.15.63:0/2013645339 shutdown_connections 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.480+0000 7f621ac30b80 1 --2- 172.21.15.63:0/2013645339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a583981d30 0x55a583987990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.480+0000 7f621ac30b80 1 --2- 172.21.15.63:0/2013645339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 0x55a58397b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.480+0000 7f621ac30b80 1 --2- 172.21.15.63:0/2013645339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a583984470 unknown :-1 s=CLOSED pgs=3004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.480+0000 7f621ac30b80 1 -- 172.21.15.63:0/2013645339 >> 172.21.15.63:0/2013645339 conn(0x55a58385cfc0 msgr2=0x55a58397f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.480+0000 7f621ac30b80 1 -- 172.21.15.63:0/2013645339 shutdown_connections 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.480+0000 7f621ac30b80 1 -- 172.21.15.63:0/2013645339 wait complete. 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.481+0000 7f621ac30b80 1 -- start start 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.481+0000 7f621ac30b80 1 Processor -- start 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 0x55a58398deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a58399c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f621ac30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a583981d30 0x55a58399cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f621ac30b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62041b0ef0 con 0x55a58397bdb0 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f621ac30b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62041b2360 con 0x55a5837fce40 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f621ac30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62041b15c0 con 0x55a583981d30 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f621ac30b80 10 client.0 _ll_register_callbacks cb 0x55a583843c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f6208c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a58399c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f6208c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a58399c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39146/0 (socket says 172.21.15.63:39146) 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f6208c00700 1 -- 172.21.15.63:0/1348598201 learned_addr learned my addr 172.21.15.63:0/1348598201 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.482+0000 7f6209600700 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 0x55a58398deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.483+0000 7f620a000700 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a583981d30 0x55a58399cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.483+0000 7f6208c00700 1 -- 172.21.15.63:0/1348598201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a583981d30 msgr2=0x55a58399cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.483+0000 7f6208c00700 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a583981d30 0x55a58399cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.483+0000 7f6208c00700 1 -- 172.21.15.63:0/1348598201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 msgr2=0x55a58398deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.483+0000 7f6208c00700 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 0x55a58398deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.483+0000 7f6208c00700 1 -- 172.21.15.63:0/1348598201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a58399d310 con 0x55a58397bdb0 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.483+0000 7f6209600700 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 0x55a58398deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.483+0000 7f6208c00700 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a58399c890 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7f61f0011c10 tx=0x7f61f0011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2df63e284b28ba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:50.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.483+0000 7f61eb400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.484+0000 7f61eb400700 1 -- 172.21.15.63:0/1348598201 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f00116c0 con 0x55a58397bdb0 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.484+0000 7f61eb400700 1 -- 172.21.15.63:0/1348598201 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(177..177 src has 1..177) ==== 9651+0+0 (secure 0 0 0) 0x7f61f0003930 con 0x55a58397bdb0 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.484+0000 7f621ac30b80 1 -- 172.21.15.63:0/1348598201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61f0020070 con 0x55a58397bdb0 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.484+0000 7f61eb400700 1 -- 172.21.15.63:0/1348598201 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f61f0036050 con 0x55a58397bdb0 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.484+0000 7f621ac30b80 10 client.23802 fetch_fsmap learned FSMap version 37 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.484+0000 7f621ac30b80 1 -- 172.21.15.63:0/1348598201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a5839ffd50 con 0x55a58397bdb0 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.485+0000 7f61eb400700 1 -- 172.21.15.63:0/1348598201 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61f0020070 con 0x55a58397bdb0 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.485+0000 7f61eb400700 1 -- 172.21.15.63:0/1348598201 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f61f0004750 con 0x55a58397bdb0 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.485+0000 7f621ac30b80 10 client.23802 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.485+0000 7f621ac30b80 10 client.23802 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.485+0000 7f621ac30b80 20 client.23802 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.485+0000 7f621ac30b80 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55a583a00360 0x55a583a02780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.485+0000 7f621ac30b80 4 client.23802 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.485+0000 7f621ac30b80 1 -- 172.21.15.63:0/1348598201 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5838a5570 con 0x55a583a00360 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.486+0000 7f6209600700 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55a583a00360 0x55a583a02780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.486+0000 7f6209600700 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55a583a00360 0x55a583a02780 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cf218c140a68bc7 server_cookie=484c1174d2ecf9ee in_seq=0 out_seq=0 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.486+0000 7f61eb400700 10 client.23802 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f61eb400700 1 -- 172.21.15.63:0/1348598201 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5838a5570 con 0x55a583a00360 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f61eb400700 10 client.23802 handle_command_reply: tid=0 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f621ac30b80 1 client.23802 shutdown 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f621ac30b80 1 -- 172.21.15.63:0/1348598201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 msgr2=0x55a58399c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f621ac30b80 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a58399c890 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7f61f0011c10 tx=0x7f61f0011f50 comp rx=0 tx=0).stop 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f621ac30b80 1 -- 172.21.15.63:0/1348598201 shutdown_connections 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f621ac30b80 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a583981d30 0x55a58399cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f621ac30b80 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5837fce40 0x55a58398deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f621ac30b80 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x55a583a00360 0x55a583a02780 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f621ac30b80 1 --2- 172.21.15.63:0/1348598201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a58397bdb0 0x55a58399c890 unknown :-1 s=CLOSED pgs=3005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f621ac30b80 1 -- 172.21.15.63:0/1348598201 >> 172.21.15.63:0/1348598201 conn(0x55a58385cfc0 msgr2=0x55a5839852d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:50.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.487+0000 7f61eb400700 0 client.23802 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:20:50.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.488+0000 7f621ac30b80 1 -- 172.21.15.63:0/1348598201 shutdown_connections 2024-05-27T08:20:50.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.488+0000 7f621ac30b80 1 -- 172.21.15.63:0/1348598201 wait complete. 2024-05-27T08:20:50.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.488+0000 7f621ac30b80 20 client.23802 trim_cache size 0 max 16384 2024-05-27T08:20:50.541 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:20:50.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 -- 172.21.15.63:0/485018252 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61ec0b2a70 msgr2=0x7f61ec0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:50.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61ec0b2a70 0x7f61ec0b4f80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009a10 tx=0x7f61f8009190 comp rx=0 tx=0).stop 2024-05-27T08:20:50.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 -- 172.21.15.63:0/485018252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6204109860 msgr2=0x7f62041a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:50.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6204109860 0x7f62041a17e0 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00b0d0 tx=0x7f61fc007d90 comp rx=0 tx=0).stop 2024-05-27T08:20:50.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 -- 172.21.15.63:0/485018252 shutdown_connections 2024-05-27T08:20:50.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f61ec0b2a70 0x7f61ec0b4f80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6204109860 0x7f62041a17e0 unknown :-1 s=CLOSED pgs=3001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62041045e0 0x7f620419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 --2- 172.21.15.63:0/485018252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6204108f10 0x7f620419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:50.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 -- 172.21.15.63:0/485018252 >> 172.21.15.63:0/485018252 conn(0x7f6204100030 msgr2=0x7f620410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:50.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 -- 172.21.15.63:0/485018252 shutdown_connections 2024-05-27T08:20:50.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:50.489+0000 7f620c000700 1 -- 172.21.15.63:0/485018252 wait complete. 2024-05-27T08:20:51.070 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:50 smithi086 podman[80644]: 2024-05-27 08:20:50.913144594 +0000 UTC m=+1.078560063 container remove 7af8edee5dccf7a0c451139f94be84fc3cab96a0b2315797c20f02f0610944a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-27T08:20:51.070 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:50 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:20:51.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:51 smithi149 ceph-mon[35408]: pgmap v1505: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.6 MiB/s wr, 1.59k op/s 2024-05-27T08:20:51.216 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa585000700 time 2024-05-27T08:20:50.868627+0000 2024-05-27T08:20:51.216 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:51.216 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.216 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa59aa4dba6] 2024-05-27T08:20:51.216 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa59aa4ddd1] 2024-05-27T08:20:51.216 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5645e30e423d] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5645e30cf971] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 5: (SnapRealm::check_cache() const+0x13e) [0x5645e30cdfa0] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5645e30ce651] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5645e3063347] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5645e2d5f87e] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5645e2d64e94] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5645e2d6f969] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 11: (MDLog::_replay_thread()+0x1f38) [0x5645e310848c] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5645e31119cb] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 13: (Thread::entry_wrapper()+0x43) [0x7fa59aa1c72d] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa59aa1c749] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa5995c01ca] 2024-05-27T08:20:51.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 16: clone() 2024-05-27T08:20:51.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: *** Caught signal (Aborted) ** 2024-05-27T08:20:51.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: in thread 7fa585000700 thread_name:md_log_replay 2024-05-27T08:20:51.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:51 smithi063 ceph-mon[31613]: pgmap v1505: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.6 MiB/s wr, 1.59k op/s 2024-05-27T08:20:51.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:50.899+0000 7fa585000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa585000700 time 2024-05-27T08:20:50.868627+0000 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa59aa4dba6] 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa59aa4ddd1] 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5645e30e423d] 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5645e30cf971] 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 5: (SnapRealm::check_cache() const+0x13e) [0x5645e30cdfa0] 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5645e30ce651] 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5645e3063347] 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5645e2d5f87e] 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5645e2d64e94] 2024-05-27T08:20:51.219 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5645e2d6f969] 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 11: (MDLog::_replay_thread()+0x1f38) [0x5645e310848c] 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5645e31119cb] 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 13: (Thread::entry_wrapper()+0x43) [0x7fa59aa1c72d] 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa59aa1c749] 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa5995c01ca] 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 16: clone() 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa5995cad20] 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2: gsignal() 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 3: abort() 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa59aa4dced] 2024-05-27T08:20:51.220 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa59aa4ddd1] 2024-05-27T08:20:51.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5645e30e423d] 2024-05-27T08:20:51.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5645e30cf971] 2024-05-27T08:20:51.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 8: (SnapRealm::check_cache() const+0x13e) [0x5645e30cdfa0] 2024-05-27T08:20:51.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5645e30ce651] 2024-05-27T08:20:51.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5645e3063347] 2024-05-27T08:20:51.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5645e2d5f87e] 2024-05-27T08:20:51.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5645e2d64e94] 2024-05-27T08:20:51.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5645e2d6f969] 2024-05-27T08:20:51.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 14: (MDLog::_replay_thread()+0x1f38) [0x5645e310848c] 2024-05-27T08:20:51.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5645e31119cb] 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 16: (Thread::entry_wrapper()+0x43) [0x7fa59aa1c72d] 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa59aa1c749] 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa5995c01ca] 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 19: clone() 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:50.906+0000 7fa585000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: in thread 7fa585000700 thread_name:md_log_replay 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa5995cad20] 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2: gsignal() 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 3: abort() 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa59aa4dced] 2024-05-27T08:20:51.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa59aa4ddd1] 2024-05-27T08:20:51.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5645e30e423d] 2024-05-27T08:20:51.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5645e30cf971] 2024-05-27T08:20:51.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 8: (SnapRealm::check_cache() const+0x13e) [0x5645e30cdfa0] 2024-05-27T08:20:51.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5645e30ce651] 2024-05-27T08:20:51.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5645e3063347] 2024-05-27T08:20:51.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5645e2d5f87e] 2024-05-27T08:20:51.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5645e2d64e94] 2024-05-27T08:20:51.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5645e2d6f969] 2024-05-27T08:20:51.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 14: (MDLog::_replay_thread()+0x1f38) [0x5645e310848c] 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5645e31119cb] 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 16: (Thread::entry_wrapper()+0x43) [0x7fa59aa1c72d] 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa59aa1c749] 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa5995c01ca] 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 19: clone() 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: -1> 2024-05-27T08:20:50.899+0000 7fa585000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa585000700 time 2024-05-27T08:20:50.868627+0000 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa59aa4dba6] 2024-05-27T08:20:51.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa59aa4ddd1] 2024-05-27T08:20:51.225 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5645e30e423d] 2024-05-27T08:20:51.225 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5645e30cf971] 2024-05-27T08:20:51.225 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 5: (SnapRealm::check_cache() const+0x13e) [0x5645e30cdfa0] 2024-05-27T08:20:51.225 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5645e30ce651] 2024-05-27T08:20:51.225 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5645e3063347] 2024-05-27T08:20:51.225 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5645e2d5f87e] 2024-05-27T08:20:51.225 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5645e2d64e94] 2024-05-27T08:20:51.225 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5645e2d6f969] 2024-05-27T08:20:51.225 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 11: (MDLog::_replay_thread()+0x1f38) [0x5645e310848c] 2024-05-27T08:20:51.225 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5645e31119cb] 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 13: (Thread::entry_wrapper()+0x43) [0x7fa59aa1c72d] 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa59aa1c749] 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa5995c01ca] 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 16: clone() 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 0> 2024-05-27T08:20:50.906+0000 7fa585000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: in thread 7fa585000700 thread_name:md_log_replay 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa5995cad20] 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2: gsignal() 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 3: abort() 2024-05-27T08:20:51.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa59aa4dced] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa59aa4ddd1] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5645e30e423d] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5645e30cf971] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 8: (SnapRealm::check_cache() const+0x13e) [0x5645e30cdfa0] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5645e30ce651] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5645e3063347] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5645e2d5f87e] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5645e2d64e94] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5645e2d6f969] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 14: (MDLog::_replay_thread()+0x1f38) [0x5645e310848c] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5645e31119cb] 2024-05-27T08:20:51.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 16: (Thread::entry_wrapper()+0x43) [0x7fa59aa1c72d] 2024-05-27T08:20:51.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa59aa1c749] 2024-05-27T08:20:51.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa5995c01ca] 2024-05-27T08:20:51.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 19: clone() 2024-05-27T08:20:51.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:20:51.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: -1> 2024-05-27T08:20:50.899+0000 7fa585000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa585000700 time 2024-05-27T08:20:50.868627+0000 2024-05-27T08:20:51.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:51.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa59aa4dba6] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa59aa4ddd1] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5645e30e423d] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5645e30cf971] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 5: (SnapRealm::check_cache() const+0x13e) [0x5645e30cdfa0] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5645e30ce651] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5645e3063347] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5645e2d5f87e] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5645e2d64e94] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5645e2d6f969] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 11: (MDLog::_replay_thread()+0x1f38) [0x5645e310848c] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5645e31119cb] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 13: (Thread::entry_wrapper()+0x43) [0x7fa59aa1c72d] 2024-05-27T08:20:51.229 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa59aa1c749] 2024-05-27T08:20:51.230 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa5995c01ca] 2024-05-27T08:20:51.230 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 16: clone() 2024-05-27T08:20:51.230 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.230 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 0> 2024-05-27T08:20:50.906+0000 7fa585000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:20:51.230 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: in thread 7fa585000700 thread_name:md_log_replay 2024-05-27T08:20:51.230 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.231 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.231 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa5995cad20] 2024-05-27T08:20:51.231 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2: gsignal() 2024-05-27T08:20:51.231 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 3: abort() 2024-05-27T08:20:51.231 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa59aa4dced] 2024-05-27T08:20:51.231 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa59aa4ddd1] 2024-05-27T08:20:51.231 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5645e30e423d] 2024-05-27T08:20:51.231 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5645e30cf971] 2024-05-27T08:20:51.231 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 8: (SnapRealm::check_cache() const+0x13e) [0x5645e30cdfa0] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5645e30ce651] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5645e3063347] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5645e2d5f87e] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5645e2d64e94] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5645e2d6f969] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 14: (MDLog::_replay_thread()+0x1f38) [0x5645e310848c] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5645e31119cb] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 16: (Thread::entry_wrapper()+0x43) [0x7fa59aa1c72d] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa59aa1c749] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa5995c01ca] 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 19: clone() 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:20:51.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[65952]: 2024-05-27T08:20:51.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:51 smithi149 podman[80734]: 2024-05-27 08:20:51.090176208 +0000 UTC m=+0.046767731 container died f582fedf84be7e9fff0ae7148df20a3d06cb78579dfaa6692f37e4fa79ef556b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:20:51.436 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:51 smithi086 ceph-mon[34687]: pgmap v1505: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.6 MiB/s wr, 1.59k op/s 2024-05-27T08:20:51.558 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:20:51.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8a4ae00700 time 2024-05-27T08:20:51.361365+0000 2024-05-27T08:20:51.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:51.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8a609f7ba6] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8a609f7dd1] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55806964b23d] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 4: (SnapRealm::build_snap_set() const+0x323) [0x558069636971] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 5: (SnapRealm::check_cache() const+0x13e) [0x558069634fa0] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x558069635651] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5580695ca347] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5580692c687e] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5580692cbe94] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5580692d6969] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 11: (MDLog::_replay_thread()+0x1f38) [0x55806966f48c] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5580696789cb] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 13: (Thread::entry_wrapper()+0x43) [0x7f8a609c672d] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8a609c6749] 2024-05-27T08:20:51.715 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8a5f56a1ca] 2024-05-27T08:20:51.716 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 16: clone() 2024-05-27T08:20:51.716 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: *** Caught signal (Aborted) ** 2024-05-27T08:20:51.716 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: in thread 7f8a4ae00700 thread_name:md_log_replay 2024-05-27T08:20:51.716 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.363+0000 7f8a4ae00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8a4ae00700 time 2024-05-27T08:20:51.361365+0000 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8a609f7ba6] 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8a609f7dd1] 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55806964b23d] 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 4: (SnapRealm::build_snap_set() const+0x323) [0x558069636971] 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 5: (SnapRealm::check_cache() const+0x13e) [0x558069634fa0] 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x558069635651] 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5580695ca347] 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5580692c687e] 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5580692cbe94] 2024-05-27T08:20:51.717 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5580692d6969] 2024-05-27T08:20:51.718 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 11: (MDLog::_replay_thread()+0x1f38) [0x55806966f48c] 2024-05-27T08:20:51.718 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5580696789cb] 2024-05-27T08:20:51.718 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 13: (Thread::entry_wrapper()+0x43) [0x7f8a609c672d] 2024-05-27T08:20:51.718 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8a609c6749] 2024-05-27T08:20:51.718 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8a5f56a1ca] 2024-05-27T08:20:51.718 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 16: clone() 2024-05-27T08:20:51.718 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.718 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.718 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8a5f574d20] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2: gsignal() 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 3: abort() 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8a609f7ced] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8a609f7dd1] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55806964b23d] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 7: (SnapRealm::build_snap_set() const+0x323) [0x558069636971] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 8: (SnapRealm::check_cache() const+0x13e) [0x558069634fa0] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x558069635651] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5580695ca347] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5580692c687e] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5580692cbe94] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5580692d6969] 2024-05-27T08:20:51.719 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 14: (MDLog::_replay_thread()+0x1f38) [0x55806966f48c] 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5580696789cb] 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 16: (Thread::entry_wrapper()+0x43) [0x7f8a609c672d] 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8a609c6749] 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f8a5f56a1ca] 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 19: clone() 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.366+0000 7f8a4ae00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: in thread 7f8a4ae00700 thread_name:md_log_replay 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8a5f574d20] 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2: gsignal() 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 3: abort() 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8a609f7ced] 2024-05-27T08:20:51.720 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8a609f7dd1] 2024-05-27T08:20:51.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55806964b23d] 2024-05-27T08:20:51.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 7: (SnapRealm::build_snap_set() const+0x323) [0x558069636971] 2024-05-27T08:20:51.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 8: (SnapRealm::check_cache() const+0x13e) [0x558069634fa0] 2024-05-27T08:20:51.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x558069635651] 2024-05-27T08:20:51.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5580695ca347] 2024-05-27T08:20:51.722 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5580692c687e] 2024-05-27T08:20:51.722 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5580692cbe94] 2024-05-27T08:20:51.722 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5580692d6969] 2024-05-27T08:20:51.722 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 14: (MDLog::_replay_thread()+0x1f38) [0x55806966f48c] 2024-05-27T08:20:51.722 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5580696789cb] 2024-05-27T08:20:51.722 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 16: (Thread::entry_wrapper()+0x43) [0x7f8a609c672d] 2024-05-27T08:20:51.723 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8a609c6749] 2024-05-27T08:20:51.723 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f8a5f56a1ca] 2024-05-27T08:20:51.723 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 19: clone() 2024-05-27T08:20:51.723 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:20:51.723 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.723 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: -1> 2024-05-27T08:20:51.363+0000 7f8a4ae00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8a4ae00700 time 2024-05-27T08:20:51.361365+0000 2024-05-27T08:20:51.723 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:51.723 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.723 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8a609f7ba6] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8a609f7dd1] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55806964b23d] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 4: (SnapRealm::build_snap_set() const+0x323) [0x558069636971] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 5: (SnapRealm::check_cache() const+0x13e) [0x558069634fa0] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x558069635651] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5580695ca347] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5580692c687e] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5580692cbe94] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5580692d6969] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 11: (MDLog::_replay_thread()+0x1f38) [0x55806966f48c] 2024-05-27T08:20:51.724 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5580696789cb] 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 13: (Thread::entry_wrapper()+0x43) [0x7f8a609c672d] 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8a609c6749] 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8a5f56a1ca] 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 16: clone() 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 0> 2024-05-27T08:20:51.366+0000 7f8a4ae00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: in thread 7f8a4ae00700 thread_name:md_log_replay 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8a5f574d20] 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2: gsignal() 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 3: abort() 2024-05-27T08:20:51.725 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8a609f7ced] 2024-05-27T08:20:51.726 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8a609f7dd1] 2024-05-27T08:20:51.726 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55806964b23d] 2024-05-27T08:20:51.726 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 7: (SnapRealm::build_snap_set() const+0x323) [0x558069636971] 2024-05-27T08:20:51.726 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 8: (SnapRealm::check_cache() const+0x13e) [0x558069634fa0] 2024-05-27T08:20:51.726 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x558069635651] 2024-05-27T08:20:51.726 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5580695ca347] 2024-05-27T08:20:51.726 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5580692c687e] 2024-05-27T08:20:51.726 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5580692cbe94] 2024-05-27T08:20:51.726 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5580692d6969] 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 14: (MDLog::_replay_thread()+0x1f38) [0x55806966f48c] 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5580696789cb] 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 16: (Thread::entry_wrapper()+0x43) [0x7f8a609c672d] 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8a609c6749] 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f8a5f56a1ca] 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 19: clone() 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: -1> 2024-05-27T08:20:51.363+0000 7f8a4ae00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8a4ae00700 time 2024-05-27T08:20:51.361365+0000 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:20:51.727 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.728 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.728 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8a609f7ba6] 2024-05-27T08:20:51.728 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8a609f7dd1] 2024-05-27T08:20:51.728 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55806964b23d] 2024-05-27T08:20:51.728 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 4: (SnapRealm::build_snap_set() const+0x323) [0x558069636971] 2024-05-27T08:20:51.728 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 5: (SnapRealm::check_cache() const+0x13e) [0x558069634fa0] 2024-05-27T08:20:51.728 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x558069635651] 2024-05-27T08:20:51.728 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5580695ca347] 2024-05-27T08:20:51.728 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5580692c687e] 2024-05-27T08:20:51.728 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5580692cbe94] 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5580692d6969] 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 11: (MDLog::_replay_thread()+0x1f38) [0x55806966f48c] 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5580696789cb] 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 13: (Thread::entry_wrapper()+0x43) [0x7f8a609c672d] 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8a609c6749] 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8a5f56a1ca] 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 16: clone() 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 0> 2024-05-27T08:20:51.366+0000 7f8a4ae00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: in thread 7f8a4ae00700 thread_name:md_log_replay 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.729 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:20:51.730 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8a5f574d20] 2024-05-27T08:20:51.730 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2: gsignal() 2024-05-27T08:20:51.730 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 3: abort() 2024-05-27T08:20:51.730 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8a609f7ced] 2024-05-27T08:20:51.730 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8a609f7dd1] 2024-05-27T08:20:51.730 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55806964b23d] 2024-05-27T08:20:51.730 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 7: (SnapRealm::build_snap_set() const+0x323) [0x558069636971] 2024-05-27T08:20:51.730 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 8: (SnapRealm::check_cache() const+0x13e) [0x558069634fa0] 2024-05-27T08:20:51.730 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x558069635651] 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5580695ca347] 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5580692c687e] 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5580692cbe94] 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5580692d6969] 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 14: (MDLog::_replay_thread()+0x1f38) [0x55806966f48c] 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5580696789cb] 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 16: (Thread::entry_wrapper()+0x43) [0x7f8a609c672d] 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8a609c6749] 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f8a5f56a1ca] 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 19: clone() 2024-05-27T08:20:51.731 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:20:51.732 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[66253]: 2024-05-27T08:20:51.732 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:51 smithi149 podman[80749]: 2024-05-27 08:20:51.532005876 +0000 UTC m=+0.047792429 container died 82900682d407a3cb20594d9258f973e44a61708188c66cdb9c6bea4dc71b6ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_BRANCH=HEAD) 2024-05-27T08:20:51.959 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:51 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-05-27T08:20:51.959 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:20:51 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Consumed 8.895s CPU time. 2024-05-27T08:20:52.199 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:20:52.713 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:52 smithi063 ceph-mon[31613]: osdmap e178: 12 total, 12 up, 12 in 2024-05-27T08:20:52.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:52 smithi086 ceph-mon[34687]: osdmap e178: 12 total, 12 up, 12 in 2024-05-27T08:20:52.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:52 smithi149 ceph-mon[35408]: osdmap e178: 12 total, 12 up, 12 in 2024-05-27T08:20:53.892 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:53 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:20:53.892 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:53 smithi063 ceph-mon[31613]: pgmap v1507: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 3.5 MiB/s wr, 799 op/s 2024-05-27T08:20:53.906 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:53 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:20:53.906 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:53 smithi149 ceph-mon[35408]: pgmap v1507: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 3.5 MiB/s wr, 799 op/s 2024-05-27T08:20:53.907 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:53 smithi149 podman[80734]: 2024-05-27 08:20:53.489609442 +0000 UTC m=+2.446200957 container remove f582fedf84be7e9fff0ae7148df20a3d06cb78579dfaa6692f37e4fa79ef556b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T08:20:53.907 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:53 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:20:53.907 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:53 smithi149 podman[80749]: 2024-05-27 08:20:53.901514364 +0000 UTC m=+2.417300914 container remove 82900682d407a3cb20594d9258f973e44a61708188c66cdb9c6bea4dc71b6ea4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:20:53.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:53 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:20:53.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:53 smithi086 ceph-mon[34687]: pgmap v1507: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 3.5 MiB/s wr, 799 op/s 2024-05-27T08:20:54.213 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:53 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:20:54.685 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T08:20:54.685 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T08:20:54.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.690+0000 7fc434600700 1 -- 172.21.15.63:0/1800368750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc42c10f940 msgr2=0x7fc42c10fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.690+0000 7fc434600700 1 --2- 172.21.15.63:0/1800368750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc42c10f940 0x7fc42c10fd50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c340 tx=0x7fc41c00c650 comp rx=0 tx=0).stop 2024-05-27T08:20:54.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.690+0000 7fc434600700 1 -- 172.21.15.63:0/1800368750 shutdown_connections 2024-05-27T08:20:54.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.690+0000 7fc434600700 1 --2- 172.21.15.63:0/1800368750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc42c110290 0x7fc42c117bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.690+0000 7fc434600700 1 --2- 172.21.15.63:0/1800368750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c10b010 0x7fc42c10b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.690+0000 7fc434600700 1 --2- 172.21.15.63:0/1800368750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc42c10f940 0x7fc42c10fd50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.690+0000 7fc434600700 1 -- 172.21.15.63:0/1800368750 >> 172.21.15.63:0/1800368750 conn(0x7fc42c06f980 msgr2=0x7fc42c06fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:54.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.690+0000 7fc434600700 1 -- 172.21.15.63:0/1800368750 shutdown_connections 2024-05-27T08:20:54.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.690+0000 7fc434600700 1 -- 172.21.15.63:0/1800368750 wait complete. 2024-05-27T08:20:54.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.692+0000 7fc434600700 1 Processor -- start 2024-05-27T08:20:54.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.693+0000 7fc434600700 1 -- start start 2024-05-27T08:20:54.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.693+0000 7fc434600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c10b010 0x7fc42c115250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.693+0000 7fc434600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc42c10f940 0x7fc42c115790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.693+0000 7fc434600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc42c110290 0x7fc42c1b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.693+0000 7fc434600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c11a9c0 con 0x7fc42c10f940 2024-05-27T08:20:54.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.693+0000 7fc434600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c11a840 con 0x7fc42c10b010 2024-05-27T08:20:54.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.694+0000 7fc434600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c11ab40 con 0x7fc42c110290 2024-05-27T08:20:54.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.694+0000 7fc431c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c10b010 0x7fc42c115250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.697+0000 7fc431c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c10b010 0x7fc42c115250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:60494/0 (socket says 172.21.15.63:60494) 2024-05-27T08:20:54.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.697+0000 7fc431c00700 1 -- 172.21.15.63:0/2566599145 learned_addr learned my addr 172.21.15.63:0/2566599145 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.697+0000 7fc431c00700 1 -- 172.21.15.63:0/2566599145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc42c110290 msgr2=0x7fc42c1b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.697+0000 7fc431c00700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc42c110290 0x7fc42c1b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.697+0000 7fc431c00700 1 -- 172.21.15.63:0/2566599145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc42c10f940 msgr2=0x7fc42c115790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.697+0000 7fc431c00700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc42c10f940 0x7fc42c115790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.698+0000 7fc431c00700 1 -- 172.21.15.63:0/2566599145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c1a42a0 con 0x7fc42c10b010 2024-05-27T08:20:54.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.700+0000 7fc431c00700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c10b010 0x7fc42c115250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc42400e9a0 tx=0x7fc42400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.701+0000 7fc42a000700 1 -- 172.21.15.63:0/2566599145 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42400cbd0 con 0x7fc42c10b010 2024-05-27T08:20:54.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.701+0000 7fc42a000700 1 -- 172.21.15.63:0/2566599145 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc42400cd70 con 0x7fc42c10b010 2024-05-27T08:20:54.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.701+0000 7fc42a000700 1 -- 172.21.15.63:0/2566599145 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424018680 con 0x7fc42c10b010 2024-05-27T08:20:54.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.703+0000 7fc434600700 1 -- 172.21.15.63:0/2566599145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c1a4590 con 0x7fc42c10b010 2024-05-27T08:20:54.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.704+0000 7fc434600700 1 -- 172.21.15.63:0/2566599145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1a4a50 con 0x7fc42c10b010 2024-05-27T08:20:54.713 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:54 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T08:20:54.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:20:54 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 14.509s CPU time. 2024-05-27T08:20:54.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:54 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-05-27T08:20:54.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:20:54 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 14.055s CPU time. 2024-05-27T08:20:54.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.713+0000 7fc42a000700 1 -- 172.21.15.63:0/2566599145 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc424010950 con 0x7fc42c10b010 2024-05-27T08:20:54.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.717+0000 7fc42a000700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4140b2af0 0x7fc4140b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.717+0000 7fc42a000700 1 -- 172.21.15.63:0/2566599145 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7fc4240e2e70 con 0x7fc42c10b010 2024-05-27T08:20:54.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.718+0000 7fc431200700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4140b2af0 0x7fc4140b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.719+0000 7fc44308fb80 1 Processor -- start 2024-05-27T08:20:54.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.719+0000 7fc44308fb80 1 -- start start 2024-05-27T08:20:54.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.720+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.720+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1a33e40 0x55b8c1bb2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.720+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bbf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.720+0000 7fc44308fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b8c1ae0e40 con 0x55b8c1bb2e70 2024-05-27T08:20:54.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.720+0000 7fc44308fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8c1ad09e0 con 0x55b8c1b1f9f0 2024-05-27T08:20:54.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.720+0000 7fc44308fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b8c1ad4a20 con 0x55b8c1a33e40 2024-05-27T08:20:54.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.721+0000 7fc431200700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4140b2af0 0x7fc4140b5000 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c310 tx=0x7fc41c000f40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.721+0000 7fc431c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.721+0000 7fc431c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:60498/0 (socket says 172.21.15.63:60498) 2024-05-27T08:20:54.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.721+0000 7fc431c00700 1 -- 172.21.15.63:0/3343335982 learned_addr learned my addr 172.21.15.63:0/3343335982 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.721+0000 7fc431200700 1 --2- 172.21.15.63:0/3343335982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1a33e40 0x55b8c1bb2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.722+0000 7fc432600700 1 --2- 172.21.15.63:0/3343335982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bbf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.722+0000 7fc431c00700 1 -- 172.21.15.63:0/3343335982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1a33e40 msgr2=0x55b8c1bb2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.722+0000 7fc431c00700 1 --2- 172.21.15.63:0/3343335982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1a33e40 0x55b8c1bb2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.722+0000 7fc431c00700 1 -- 172.21.15.63:0/3343335982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 msgr2=0x55b8c1bbf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.722+0000 7fc431c00700 1 --2- 172.21.15.63:0/3343335982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bbf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.723+0000 7fc431c00700 1 -- 172.21.15.63:0/3343335982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8c1a99420 con 0x55b8c1b1f9f0 2024-05-27T08:20:54.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.723+0000 7fc432600700 1 --2- 172.21.15.63:0/3343335982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bbf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:54.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.723+0000 7fc431c00700 1 --2- 172.21.15.63:0/3343335982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb2330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4240a9c50 tx=0x7fc4240a9740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=454056bdf0fd9a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.728+0000 7fc41aa00700 1 -- 172.21.15.63:0/3343335982 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4240ae070 con 0x55b8c1b1f9f0 2024-05-27T08:20:54.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.728+0000 7fc41aa00700 1 -- 172.21.15.63:0/3343335982 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4240b2b20 con 0x55b8c1b1f9f0 2024-05-27T08:20:54.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.728+0000 7fc41aa00700 1 -- 172.21.15.63:0/3343335982 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4240aa180 con 0x55b8c1b1f9f0 2024-05-27T08:20:54.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.732+0000 7fc44308fb80 1 -- 172.21.15.63:0/3343335982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 msgr2=0x55b8c1bb2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.732+0000 7fc44308fb80 1 --2- 172.21.15.63:0/3343335982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb2330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4240a9c50 tx=0x7fc4240a9740 comp rx=0 tx=0).stop 2024-05-27T08:20:54.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.733+0000 7fc44308fb80 1 -- 172.21.15.63:0/3343335982 shutdown_connections 2024-05-27T08:20:54.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.733+0000 7fc44308fb80 1 --2- 172.21.15.63:0/3343335982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bbf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.733+0000 7fc44308fb80 1 --2- 172.21.15.63:0/3343335982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb2330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.733+0000 7fc44308fb80 1 --2- 172.21.15.63:0/3343335982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1a33e40 0x55b8c1bb2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.733+0000 7fc44308fb80 1 -- 172.21.15.63:0/3343335982 >> 172.21.15.63:0/3343335982 conn(0x55b8c1acd390 msgr2=0x55b8c1bae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:54.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.734+0000 7fc44308fb80 1 -- 172.21.15.63:0/3343335982 shutdown_connections 2024-05-27T08:20:54.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.734+0000 7fc44308fb80 1 -- 172.21.15.63:0/3343335982 wait complete. 2024-05-27T08:20:54.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.735+0000 7fc44308fb80 1 -- start start 2024-05-27T08:20:54.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.735+0000 7fc44308fb80 1 Processor -- start 2024-05-27T08:20:54.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.739+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.740+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bbcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.740+0000 7fc431c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.740+0000 7fc431c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39180/0 (socket says 172.21.15.63:39180) 2024-05-27T08:20:54.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.744+0000 7fc431200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bbcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.744+0000 7fc431c00700 1 -- 172.21.15.63:0/707176219 learned_addr learned my addr 172.21.15.63:0/707176219 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:54.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.744+0000 7fc44308fb80 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bbd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.744+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b8c19d6fa0 con 0x55b8c1a33e40 2024-05-27T08:20:54.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.745+0000 7fc432600700 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bbd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.745+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8c1a71760 con 0x55b8c1bb2e70 2024-05-27T08:20:54.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.745+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b8c1bbfb80 con 0x55b8c1b1f9f0 2024-05-27T08:20:54.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.747+0000 7fc432600700 1 -- 172.21.15.63:0/707176219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 msgr2=0x55b8c1bbcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.747+0000 7fc432600700 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bbcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.747+0000 7fc432600700 1 -- 172.21.15.63:0/707176219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 msgr2=0x55b8c1bbc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.747+0000 7fc432600700 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.747+0000 7fc432600700 1 -- 172.21.15.63:0/707176219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8c1a208d0 con 0x55b8c1bb2e70 2024-05-27T08:20:54.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.748+0000 7fc432600700 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bbd150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc42000bfd0 tx=0x7fc42000d4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6efad856717381c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.748+0000 7fc40f400700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:20:54.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.750+0000 7fc44308fb80 10 client.0 _ll_register_callbacks cb 0x55b8c1a7ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:54.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.750+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4240b2b20 con 0x55b8c1bb2e70 2024-05-27T08:20:54.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.751+0000 7fc40f400700 1 -- 172.21.15.63:0/707176219 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42000de30 con 0x55b8c1bb2e70 2024-05-27T08:20:54.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.752+0000 7fc40f400700 1 -- 172.21.15.63:0/707176219 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7fc420014430 con 0x55b8c1bb2e70 2024-05-27T08:20:54.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.754+0000 7fc40f400700 1 -- 172.21.15.63:0/707176219 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc420022050 con 0x55b8c1bb2e70 2024-05-27T08:20:54.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.754+0000 7fc44308fb80 10 client.42731 fetch_fsmap learned FSMap version 39 2024-05-27T08:20:54.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.754+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55b8c1aa1060 con 0x55b8c1bb2e70 2024-05-27T08:20:54.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.755+0000 7fc40f400700 1 -- 172.21.15.63:0/707176219 <== mon.2 v2:172.21.15.86:3300/0 4 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc420013070 con 0x55b8c1bb2e70 2024-05-27T08:20:54.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.757+0000 7fc44308fb80 10 client.42731 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T08:20:54.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.757+0000 7fc44308fb80 10 client.42731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:20:54.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.757+0000 7fc44308fb80 20 client.42731 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:54.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.757+0000 7fc44308fb80 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8c1bcc960 0x55b8c1bced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.757+0000 7fc44308fb80 4 client.42731 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:20:54.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.757+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8c1ae2d90 con 0x55b8c1bcc960 2024-05-27T08:20:54.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.757+0000 7fc431c00700 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8c1bcc960 0x55b8c1bced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.758+0000 7fc40f400700 10 client.42731 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:54.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.758+0000 7fc431c00700 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8c1bcc960 0x55b8c1bced80 crc :-1 s=READY pgs=792 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6566294d16ac0783 server_cookie=439ffd266acaeaf2 in_seq=0 out_seq=0 2024-05-27T08:20:54.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.760+0000 7fc40f400700 1 -- 172.21.15.63:0/707176219 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b8c1ae2d90 con 0x55b8c1bcc960 2024-05-27T08:20:54.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.760+0000 7fc40f400700 10 client.42731 handle_command_reply: tid=0 2024-05-27T08:20:54.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.761+0000 7fc44308fb80 1 client.42731 shutdown 2024-05-27T08:20:54.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.762+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1bb2e70 msgr2=0x55b8c1bbd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.762+0000 7fc44308fb80 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bbd150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc42000bfd0 tx=0x7fc42000d4f0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.762+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 shutdown_connections 2024-05-27T08:20:54.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.762+0000 7fc44308fb80 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8c1bcc960 0x55b8c1bced80 crc :-1 s=READY pgs=792 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.762+0000 7fc44308fb80 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bbd150 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.762+0000 7fc44308fb80 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.762+0000 7fc44308fb80 1 --2- 172.21.15.63:0/707176219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bbcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.762+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 >> 172.21.15.63:0/707176219 conn(0x55b8c1acd390 msgr2=0x55b8c1bb23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:54.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.762+0000 7fc40f400700 0 client.42731 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:54.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.764+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 shutdown_connections 2024-05-27T08:20:54.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.764+0000 7fc44308fb80 1 -- 172.21.15.63:0/707176219 wait complete. 2024-05-27T08:20:54.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.764+0000 7fc44308fb80 20 client.42731 trim_cache size 0 max 16384 2024-05-27T08:20:54.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.801+0000 7fc44308fb80 1 Processor -- start 2024-05-27T08:20:54.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.801+0000 7fc44308fb80 1 -- start start 2024-05-27T08:20:54.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.801+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1a33e40 0x55b8c1bb7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.801+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.801+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bd4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.801+0000 7fc44308fb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1ca7b0 con 0x55b8c1bb2e70 2024-05-27T08:20:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.801+0000 7fc44308fb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1c8010 con 0x55b8c1a33e40 2024-05-27T08:20:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.801+0000 7fc44308fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1ab8c0 con 0x55b8c1b1f9f0 2024-05-27T08:20:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.802+0000 7fc431c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1a33e40 0x55b8c1bb7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.802+0000 7fc431c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1a33e40 0x55b8c1bb7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:60516/0 (socket says 172.21.15.63:60516) 2024-05-27T08:20:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.802+0000 7fc431c00700 1 -- 172.21.15.63:0/108566169 learned_addr learned my addr 172.21.15.63:0/108566169 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:54.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.803+0000 7fc431200700 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.804+0000 7fc432600700 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bd4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.805+0000 7fc431c00700 1 -- 172.21.15.63:0/108566169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 msgr2=0x55b8c1bb65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.806+0000 7fc431c00700 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.806+0000 7fc431c00700 1 -- 172.21.15.63:0/108566169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 msgr2=0x55b8c1bd4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.806+0000 7fc431c00700 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bd4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.806+0000 7fc431c00700 1 -- 172.21.15.63:0/108566169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8c1bbac40 con 0x55b8c1a33e40 2024-05-27T08:20:54.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.807+0000 7fc431200700 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:54.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.807+0000 7fc432600700 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bd4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:54.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.809+0000 7fc431c00700 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1a33e40 0x55b8c1bb7f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc4240a9c50 tx=0x7fc4240a9a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=370ec03e76a44957 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.809+0000 7fb113000640 1 -- 172.21.15.63:0/3495433635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb10c102b80 msgr2=0x7fb10c102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.809+0000 7fb113000640 1 --2- 172.21.15.63:0/3495433635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb10c102b80 0x7fb10c102f90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb104009ca0 tx=0x7fb10402a8f0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.810+0000 7fb113000640 1 -- 172.21.15.63:0/3495433635 shutdown_connections 2024-05-27T08:20:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.810+0000 7fb113000640 1 --2- 172.21.15.63:0/3495433635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c108000 0x7fb10c108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.810+0000 7fb113000640 1 --2- 172.21.15.63:0/3495433635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10c1075d0 0x7fb10c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.810+0000 7fb113000640 1 --2- 172.21.15.63:0/3495433635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb10c102b80 0x7fb10c102f90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.810+0000 7fb113000640 1 -- 172.21.15.63:0/3495433635 >> 172.21.15.63:0/3495433635 conn(0x7fb10c0782d0 msgr2=0x7fb10c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.810+0000 7fb113000640 1 -- 172.21.15.63:0/3495433635 shutdown_connections 2024-05-27T08:20:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.810+0000 7fb113000640 1 -- 172.21.15.63:0/3495433635 wait complete. 2024-05-27T08:20:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.811+0000 7fb113000640 1 Processor -- start 2024-05-27T08:20:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.811+0000 7fb113000640 1 -- start start 2024-05-27T08:20:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fb113000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb10c102b80 0x7fb10c1a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fb113000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c1075d0 0x7fb10c1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fb113000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10c108000 0x7fb10c1a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fb113000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c11a3d0 con 0x7fb10c1075d0 2024-05-27T08:20:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fb113000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c11a250 con 0x7fb10c108000 2024-05-27T08:20:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fb113000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c11a550 con 0x7fb10c102b80 2024-05-27T08:20:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fb111000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10c108000 0x7fb10c1a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fb10b400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c1075d0 0x7fb10c1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fb111000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10c108000 0x7fb10c1a5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:60542/0 (socket says 172.21.15.63:60542) 2024-05-27T08:20:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fb111000640 1 -- 172.21.15.63:0/1971964038 learned_addr learned my addr 172.21.15.63:0/1971964038 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.813+0000 7fb10b400640 1 -- 172.21.15.63:0/1971964038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb10c102b80 msgr2=0x7fb10c1a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.813+0000 7fb10b400640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb10c102b80 0x7fb10c1a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.813+0000 7fb10b400640 1 -- 172.21.15.63:0/1971964038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10c108000 msgr2=0x7fb10c1a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.813+0000 7fb10b400640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10c108000 0x7fb10c1a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.813+0000 7fb10b400640 1 -- 172.21.15.63:0/1971964038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c1a5d90 con 0x7fb10c1075d0 2024-05-27T08:20:54.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.812+0000 7fc41a000700 1 -- 172.21.15.63:0/108566169 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4240a62b0 con 0x55b8c1a33e40 2024-05-27T08:20:54.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.813+0000 7fc41a000700 1 -- 172.21.15.63:0/108566169 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4240c4350 con 0x55b8c1a33e40 2024-05-27T08:20:54.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.813+0000 7fc41a000700 1 -- 172.21.15.63:0/108566169 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4240a62b0 con 0x55b8c1a33e40 2024-05-27T08:20:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.813+0000 7fb10b400640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c1075d0 0x7fb10c1a1490 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009fd0 tx=0x7fb0f400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.814+0000 7fb108c00640 1 -- 172.21.15.63:0/1971964038 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4015020 con 0x7fb10c1075d0 2024-05-27T08:20:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.814+0000 7fb108c00640 1 -- 172.21.15.63:0/1971964038 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0f4011cc0 con 0x7fb10c1075d0 2024-05-27T08:20:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.814+0000 7fb108c00640 1 -- 172.21.15.63:0/1971964038 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4010d40 con 0x7fb10c1075d0 2024-05-27T08:20:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.814+0000 7fb113000640 1 -- 172.21.15.63:0/1971964038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10c1a6080 con 0x7fb10c1075d0 2024-05-27T08:20:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.814+0000 7fb113000640 1 -- 172.21.15.63:0/1971964038 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10c1a6580 con 0x7fb10c1075d0 2024-05-27T08:20:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.815+0000 7fc44308fb80 1 -- 172.21.15.63:0/108566169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1a33e40 msgr2=0x55b8c1bb7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.815+0000 7fc44308fb80 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1a33e40 0x55b8c1bb7f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc4240a9c50 tx=0x7fc4240a9a90 comp rx=0 tx=0).stop 2024-05-27T08:20:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.818+0000 7fc44308fb80 1 -- 172.21.15.63:0/108566169 shutdown_connections 2024-05-27T08:20:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.818+0000 7fc44308fb80 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bd4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.818+0000 7fc44308fb80 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1a33e40 0x55b8c1bb7f00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.818+0000 7fc44308fb80 1 --2- 172.21.15.63:0/108566169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bb65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.818+0000 7fc44308fb80 1 -- 172.21.15.63:0/108566169 >> 172.21.15.63:0/108566169 conn(0x55b8c1acd390 msgr2=0x55b8c1bb5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.818+0000 7fc44308fb80 1 -- 172.21.15.63:0/108566169 shutdown_connections 2024-05-27T08:20:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.818+0000 7fc44308fb80 1 -- 172.21.15.63:0/108566169 wait complete. 2024-05-27T08:20:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.819+0000 7fc44308fb80 1 -- start start 2024-05-27T08:20:54.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.819+0000 7fc44308fb80 1 Processor -- start 2024-05-27T08:20:54.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.821+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.823+0000 7fb119fb6740 1 Processor -- start 2024-05-27T08:20:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.823+0000 7fb119fb6740 1 -- start start 2024-05-27T08:20:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.824+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5be860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.821+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bc0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.823+0000 7fc431c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.823+0000 7fc431c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39222/0 (socket says 172.21.15.63:39222) 2024-05-27T08:20:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.825+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.825+0000 7fb111000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.827 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.825+0000 7fb111000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5be860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39236/0 (socket says 172.21.15.63:39236) 2024-05-27T08:20:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.821+0000 7fc44308fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bc08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.824+0000 7fc432600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bc08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.824+0000 7fc432600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bc08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:37712/0 (socket says 172.21.15.63:37712) 2024-05-27T08:20:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.825+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1abf10 con 0x55b8c1a33e40 2024-05-27T08:20:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.825+0000 7fc431c00700 1 -- 172.21.15.63:0/18422413 learned_addr learned my addr 172.21.15.63:0/18422413 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.825+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1c7940 con 0x55b8c1b1f9f0 2024-05-27T08:20:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.825+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1ac5e0 con 0x55b8c1bb2e70 2024-05-27T08:20:54.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.825+0000 7fc44308fb80 10 client.0 _ll_register_callbacks cb 0x55b8c1a7ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.826+0000 7fb111000640 1 -- 172.21.15.63:0/2639197101 learned_addr learned my addr 172.21.15.63:0/2639197101 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.826+0000 7fb119fb6740 1 --2- 172.21.15.63:0/2639197101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.826+0000 7fb10be00640 1 --2- 172.21.15.63:0/2639197101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.826+0000 7fb119fb6740 1 -- 172.21.15.63:0/2639197101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b8ed52fb90 con 0x55b8ed21c1b0 2024-05-27T08:20:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.825+0000 7fc431200700 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bc0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.826+0000 7fc432600700 1 -- 172.21.15.63:0/18422413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 msgr2=0x55b8c1bc0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.826+0000 7fc432600700 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bc0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.826+0000 7fc432600700 1 -- 172.21.15.63:0/18422413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 msgr2=0x55b8c1bbfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.826+0000 7fb119fb6740 1 -- 172.21.15.63:0/2639197101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8ed51f730 con 0x55b8ed1f32f0 2024-05-27T08:20:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.827+0000 7fb108c00640 1 -- 172.21.15.63:0/1971964038 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb0f4018020 con 0x7fb10c1075d0 2024-05-27T08:20:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.826+0000 7fc432600700 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.826+0000 7fc432600700 1 -- 172.21.15.63:0/18422413 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8c1c35590 con 0x55b8c1bb2e70 2024-05-27T08:20:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.827+0000 7fc431200700 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bc0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:54.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.827+0000 7fc431c00700 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.827+0000 7fb119fb6740 1 -- 172.21.15.63:0/2639197101 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b8ed523770 con 0x55b8ed1fb4d0 2024-05-27T08:20:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.827+0000 7fb10b400640 1 --2- 172.21.15.63:0/2639197101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.827+0000 7fc40f400700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:20:54.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.827+0000 7fc432600700 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bc08a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc420014680 tx=0x7fc420004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3788edac1f3473fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.828+0000 7fc40f400700 1 -- 172.21.15.63:0/18422413 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420014c10 con 0x55b8c1bb2e70 2024-05-27T08:20:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.828+0000 7fb111000640 1 -- 172.21.15.63:0/2639197101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 msgr2=0x55b8ed5bf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.828+0000 7fb111000640 1 --2- 172.21.15.63:0/2639197101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5bf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.828+0000 7fb111000640 1 -- 172.21.15.63:0/2639197101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed1f32f0 msgr2=0x55b8ed5beda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.828+0000 7fb111000640 1 --2- 172.21.15.63:0/2639197101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5beda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.828+0000 7fb111000640 1 -- 172.21.15.63:0/2639197101 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ed4b88b0 con 0x55b8ed21c1b0 2024-05-27T08:20:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.829+0000 7fb111000640 1 --2- 172.21.15.63:0/2639197101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5be860 secure :-1 s=READY pgs=3007 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009840 tx=0x7fb0fc009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab4e448491b7381 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.829+0000 7fb101600640 1 -- 172.21.15.63:0/2639197101 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00d950 con 0x55b8ed21c1b0 2024-05-27T08:20:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.829+0000 7fb101600640 1 -- 172.21.15.63:0/2639197101 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0fc014430 con 0x55b8ed21c1b0 2024-05-27T08:20:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.829+0000 7fb101600640 1 -- 172.21.15.63:0/2639197101 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0123e0 con 0x55b8ed21c1b0 2024-05-27T08:20:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.829+0000 7fb108c00640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0f00b2a20 0x7fb0f00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.830+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4240c4350 con 0x55b8c1bb2e70 2024-05-27T08:20:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.831+0000 7fb119fb6740 1 -- 172.21.15.63:0/2639197101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 msgr2=0x55b8ed5be860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.831+0000 7fb119fb6740 1 --2- 172.21.15.63:0/2639197101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5be860 secure :-1 s=READY pgs=3007 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009840 tx=0x7fb0fc009d00 comp rx=0 tx=0).stop 2024-05-27T08:20:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.831+0000 7fb108c00640 1 -- 172.21.15.63:0/1971964038 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7fb0f40e2200 con 0x7fb10c1075d0 2024-05-27T08:20:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.831+0000 7fb119fb6740 1 -- 172.21.15.63:0/2639197101 shutdown_connections 2024-05-27T08:20:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.831+0000 7fb119fb6740 1 --2- 172.21.15.63:0/2639197101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5bf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.831+0000 7fb119fb6740 1 --2- 172.21.15.63:0/2639197101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5beda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.831+0000 7fb119fb6740 1 --2- 172.21.15.63:0/2639197101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5be860 unknown :-1 s=CLOSED pgs=3007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.831+0000 7fb119fb6740 1 -- 172.21.15.63:0/2639197101 >> 172.21.15.63:0/2639197101 conn(0x55b8ed210300 msgr2=0x55b8ed5bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.831+0000 7fb10be00640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0f00b2a20 0x7fb0f00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.832+0000 7fb119fb6740 1 -- 172.21.15.63:0/2639197101 shutdown_connections 2024-05-27T08:20:54.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.831+0000 7fc40f400700 1 -- 172.21.15.63:0/18422413 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7fc42000f040 con 0x55b8c1bb2e70 2024-05-27T08:20:54.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.832+0000 7fc40f400700 1 -- 172.21.15.63:0/18422413 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc420014c10 con 0x55b8c1bb2e70 2024-05-27T08:20:54.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.832+0000 7fc44308fb80 10 client.32548 fetch_fsmap learned FSMap version 39 2024-05-27T08:20:54.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.832+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b8c1c367f0 con 0x55b8c1bb2e70 2024-05-27T08:20:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.832+0000 7fb119fb6740 1 -- 172.21.15.63:0/2639197101 wait complete. 2024-05-27T08:20:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.833+0000 7fb10be00640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0f00b2a20 0x7fb0f00b4f30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x55b8ed5bfc10 tx=0x7fb104000f40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.833+0000 7fc40f400700 1 -- 172.21.15.63:0/18422413 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4240c4350 con 0x55b8c1bb2e70 2024-05-27T08:20:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.833+0000 7fc40f400700 1 -- 172.21.15.63:0/18422413 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc420039ce0 con 0x55b8c1bb2e70 2024-05-27T08:20:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.833+0000 7fc44308fb80 10 client.32548 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T08:20:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.833+0000 7fc44308fb80 10 client.32548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:20:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.833+0000 7fc44308fb80 20 client.32548 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.833+0000 7fc44308fb80 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8c1c36e50 0x55b8c1c39270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.833+0000 7fc44308fb80 4 client.32548 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:20:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.833+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8c1adc570 con 0x55b8c1c36e50 2024-05-27T08:20:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.833+0000 7fb119fb6740 1 -- start start 2024-05-27T08:20:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.834+0000 7fb119fb6740 1 Processor -- start 2024-05-27T08:20:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.834+0000 7fc431c00700 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8c1c36e50 0x55b8c1c39270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb119fb6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b8ed205790 con 0x55b8ed1f32f0 2024-05-27T08:20:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb119fb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8ed4c7bc0 con 0x55b8ed21c1b0 2024-05-27T08:20:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb119fb6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b8ed5cb990 con 0x55b8ed1fb4d0 2024-05-27T08:20:54.841 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb119fb6740 10 client.0 _ll_register_callbacks cb 0x55b8ed4a5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fc431c00700 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8c1c36e50 0x55b8c1c39270 crc :-1 s=READY pgs=793 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbaa316dd680617c server_cookie=7b657ed01b3d2097 in_seq=0 out_seq=0 2024-05-27T08:20:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fc40f400700 10 client.32548 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fc40f400700 1 -- 172.21.15.63:0/18422413 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8c1adc570 con 0x55b8c1c36e50 2024-05-27T08:20:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fc40f400700 10 client.32548 handle_command_reply: tid=0 2024-05-27T08:20:54.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fc44308fb80 1 client.32548 shutdown 2024-05-27T08:20:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb10be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb111000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb10be00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39252/0 (socket says 172.21.15.63:39252) 2024-05-27T08:20:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb10be00640 1 -- 172.21.15.63:0/1225110837 learned_addr learned my addr 172.21.15.63:0/1225110837 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fb10b400640 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1bb2e70 msgr2=0x55b8c1bc08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.835+0000 7fc44308fb80 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bc08a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc420014680 tx=0x7fc420004860 comp rx=0 tx=0).stop 2024-05-27T08:20:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fb10be00640 1 -- 172.21.15.63:0/1225110837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 msgr2=0x55b8ed5c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fb10be00640 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fb10be00640 1 -- 172.21.15.63:0/1225110837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 msgr2=0x55b8ed5d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fb10be00640 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fb10be00640 1 -- 172.21.15.63:0/1225110837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ed234210 con 0x55b8ed1f32f0 2024-05-27T08:20:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 shutdown_connections 2024-05-27T08:20:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fc44308fb80 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8c1c36e50 0x55b8c1c39270 crc :-1 s=READY pgs=793 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fc44308fb80 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8c1bb2e70 0x55b8c1bc08a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fc44308fb80 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8c1a33e40 0x55b8c1bbfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fc44308fb80 1 --2- 172.21.15.63:0/18422413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8c1b1f9f0 0x55b8c1bc0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 >> 172.21.15.63:0/18422413 conn(0x55b8c1acd390 msgr2=0x55b8c1bc7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:54.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fc40f400700 0 client.32548 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fb111000640 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:20:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fb10be00640 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5c89f0 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7fb104009ca0 tx=0x7fb1040095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cdaf840c46fbe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 shutdown_connections 2024-05-27T08:20:54.845 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fb0fa000640 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:20:54.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fc44308fb80 1 -- 172.21.15.63:0/18422413 wait complete. 2024-05-27T08:20:54.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.836+0000 7fc44308fb80 20 client.32548 trim_cache size 0 max 16384 2024-05-27T08:20:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.837+0000 7fb0fa000640 1 -- 172.21.15.63:0/1225110837 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10404c070 con 0x55b8ed1f32f0 2024-05-27T08:20:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.837+0000 7fb0fa000640 1 -- 172.21.15.63:0/1225110837 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7fb104048620 con 0x55b8ed1f32f0 2024-05-27T08:20:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.837+0000 7fb119fb6740 1 -- 172.21.15.63:0/1225110837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0fc014430 con 0x55b8ed1f32f0 2024-05-27T08:20:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:20:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fb0fa000640 1 -- 172.21.15.63:0/1225110837 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb10404c070 con 0x55b8ed1f32f0 2024-05-27T08:20:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fb119fb6740 10 client.23823 fetch_fsmap learned FSMap version 39 2024-05-27T08:20:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fb119fb6740 1 -- 172.21.15.63:0/1225110837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b8ed300620 con 0x55b8ed1f32f0 2024-05-27T08:20:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fb0fa000640 1 -- 172.21.15.63:0/1225110837 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb104036040 con 0x55b8ed1f32f0 2024-05-27T08:20:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fb0fa000640 1 -- 172.21.15.63:0/1225110837 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb1040517d0 con 0x55b8ed1f32f0 2024-05-27T08:20:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fc434600700 1 -- 172.21.15.63:0/2566599145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4140b2af0 msgr2=0x7fc4140b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fc434600700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4140b2af0 0x7fc4140b5000 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c310 tx=0x7fc41c000f40 comp rx=0 tx=0).stop 2024-05-27T08:20:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fc434600700 1 -- 172.21.15.63:0/2566599145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c10b010 msgr2=0x7fc42c115250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fc434600700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c10b010 0x7fc42c115250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc42400e9a0 tx=0x7fc42400ed60 comp rx=0 tx=0).stop 2024-05-27T08:20:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fc434600700 1 -- 172.21.15.63:0/2566599145 shutdown_connections 2024-05-27T08:20:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fc434600700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fc4140b2af0 0x7fc4140b5000 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fc434600700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc42c110290 0x7fc42c1b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fc434600700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc42c10b010 0x7fc42c115250 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fc434600700 1 --2- 172.21.15.63:0/2566599145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc42c10f940 0x7fc42c115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.838+0000 7fc434600700 1 -- 172.21.15.63:0/2566599145 >> 172.21.15.63:0/2566599145 conn(0x7fc42c06f980 msgr2=0x7fc42c10d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.839+0000 7fc434600700 1 -- 172.21.15.63:0/2566599145 shutdown_connections 2024-05-27T08:20:54.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:54.839+0000 7fc434600700 1 -- 172.21.15.63:0/2566599145 wait complete. 2024-05-27T08:20:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.839+0000 7fb119fb6740 10 client.23823 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T08:20:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.839+0000 7fb119fb6740 10 client.23823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:20:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.839+0000 7fb119fb6740 20 client.23823 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.839+0000 7fb119fb6740 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8ed1dad80 0x55b8ed5dabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.839+0000 7fb119fb6740 4 client.23823 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:20:54.851 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.839+0000 7fb119fb6740 1 -- 172.21.15.63:0/1225110837 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb0fc014430 con 0x55b8ed1dad80 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.840+0000 7fb10b400640 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8ed1dad80 0x55b8ed5dabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.840+0000 7fb10b400640 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8ed1dad80 0x55b8ed5dabb0 crc :-1 s=READY pgs=794 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b49ea447667035 server_cookie=f05075afa37e9bde in_seq=0 out_seq=0 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.840+0000 7fb0fa000640 10 client.23823 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.841+0000 7fb0fa000640 1 -- 172.21.15.63:0/1225110837 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7fb0fc014430 con 0x55b8ed1dad80 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.841+0000 7fb0fa000640 10 client.23823 handle_command_reply: tid=0 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.841+0000 7fb119fb6740 1 client.23823 shutdown 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb119fb6740 1 -- 172.21.15.63:0/1225110837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 msgr2=0x55b8ed5c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb119fb6740 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5c89f0 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7fb104009ca0 tx=0x7fb1040095c0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb119fb6740 1 -- 172.21.15.63:0/1225110837 shutdown_connections 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb119fb6740 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8ed1dad80 0x55b8ed5dabb0 crc :-1 s=READY pgs=794 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb119fb6740 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb119fb6740 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb119fb6740 1 --2- 172.21.15.63:0/1225110837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5c89f0 unknown :-1 s=CLOSED pgs=3008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb119fb6740 1 -- 172.21.15.63:0/1225110837 >> 172.21.15.63:0/1225110837 conn(0x55b8ed210300 msgr2=0x55b8ed5be690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb0fa000640 0 client.23823 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb119fb6740 1 -- 172.21.15.63:0/1225110837 shutdown_connections 2024-05-27T08:20:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.842+0000 7fb119fb6740 1 -- 172.21.15.63:0/1225110837 wait complete. 2024-05-27T08:20:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.843+0000 7fb119fb6740 20 client.23823 trim_cache size 0 max 16384 2024-05-27T08:20:54.857 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-05-27T08:20:54.857 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-27T08:20:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.885+0000 7fb119fb6740 1 Processor -- start 2024-05-27T08:20:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.886+0000 7fb119fb6740 1 -- start start 2024-05-27T08:20:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.886+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.886+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.886+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.886+0000 7fb119fb6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c1ba310 con 0x55b8ed1f32f0 2024-05-27T08:20:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.887+0000 7fb119fb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c1b7bb0 con 0x55b8ed21c1b0 2024-05-27T08:20:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.887+0000 7fb119fb6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c1b5a20 con 0x55b8ed1fb4d0 2024-05-27T08:20:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.887+0000 7fb111000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.887+0000 7fb111000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:60562/0 (socket says 172.21.15.63:60562) 2024-05-27T08:20:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.887+0000 7fb111000640 1 -- 172.21.15.63:0/2442842807 learned_addr learned my addr 172.21.15.63:0/2442842807 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.888+0000 7fb111000640 1 -- 172.21.15.63:0/2442842807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 msgr2=0x55b8ed5cab40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:20:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.888+0000 7fb111000640 1 --2- 172.21.15.63:0/2442842807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5cab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.888+0000 7fb111000640 1 -- 172.21.15.63:0/2442842807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 msgr2=0x55b8ed5c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.888+0000 7fb111000640 1 --2- 172.21.15.63:0/2442842807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.888+0000 7fb111000640 1 -- 172.21.15.63:0/2442842807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ed5bfc40 con 0x55b8ed21c1b0 2024-05-27T08:20:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.889+0000 7fb111000640 1 --2- 172.21.15.63:0/2442842807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5c8990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00d530 tx=0x7fb0fc009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb06cba52a6b2be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.890+0000 7fb100c00640 1 -- 172.21.15.63:0/2442842807 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00d9d0 con 0x55b8ed21c1b0 2024-05-27T08:20:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.890+0000 7fb100c00640 1 -- 172.21.15.63:0/2442842807 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0fc014430 con 0x55b8ed21c1b0 2024-05-27T08:20:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.894+0000 7fb100c00640 1 -- 172.21.15.63:0/2442842807 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc01d650 con 0x55b8ed21c1b0 2024-05-27T08:20:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.896+0000 7fb119fb6740 1 -- 172.21.15.63:0/2442842807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 msgr2=0x55b8ed5c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.896+0000 7fb119fb6740 1 --2- 172.21.15.63:0/2442842807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5c8990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00d530 tx=0x7fb0fc009290 comp rx=0 tx=0).stop 2024-05-27T08:20:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.898+0000 7fb119fb6740 1 -- 172.21.15.63:0/2442842807 shutdown_connections 2024-05-27T08:20:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.898+0000 7fb119fb6740 1 --2- 172.21.15.63:0/2442842807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5c8990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.898+0000 7fb119fb6740 1 --2- 172.21.15.63:0/2442842807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.898+0000 7fb119fb6740 1 --2- 172.21.15.63:0/2442842807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.898+0000 7fb119fb6740 1 -- 172.21.15.63:0/2442842807 >> 172.21.15.63:0/2442842807 conn(0x55b8ed210300 msgr2=0x55b8ed5c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.900+0000 7fb119fb6740 1 -- 172.21.15.63:0/2442842807 shutdown_connections 2024-05-27T08:20:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.900+0000 7fb119fb6740 1 -- 172.21.15.63:0/2442842807 wait complete. 2024-05-27T08:20:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.901+0000 7fb119fb6740 1 -- start start 2024-05-27T08:20:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.901+0000 7fb119fb6740 1 Processor -- start 2024-05-27T08:20:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.902+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.902+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.902+0000 7fb119fb6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.902+0000 7fb119fb6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c1b6070 con 0x55b8ed21c1b0 2024-05-27T08:20:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.902+0000 7fb119fb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c1b74e0 con 0x55b8ed1fb4d0 2024-05-27T08:20:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.902+0000 7fb119fb6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c1b6740 con 0x55b8ed1f32f0 2024-05-27T08:20:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.903+0000 7fb119fb6740 10 client.0 _ll_register_callbacks cb 0x7fb104031480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:20:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.903+0000 7fb10be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.903+0000 7fb10be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:37740/0 (socket says 172.21.15.63:37740) 2024-05-27T08:20:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.903+0000 7fb10be00640 1 -- 172.21.15.63:0/4221560111 learned_addr learned my addr 172.21.15.63:0/4221560111 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.903+0000 7fb10b400640 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.903+0000 7fb111000640 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.903+0000 7fb10be00640 1 -- 172.21.15.63:0/4221560111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed1fb4d0 msgr2=0x55b8ed5d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.903+0000 7fb10be00640 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.903+0000 7fb10be00640 1 -- 172.21.15.63:0/4221560111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 msgr2=0x55b8ed5ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:20:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.903+0000 7fb10be00640 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.904+0000 7fb10be00640 1 -- 172.21.15.63:0/4221560111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ed5bf960 con 0x55b8ed1f32f0 2024-05-27T08:20:54.906 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.904+0000 7fb10be00640 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5d0f50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb104051bf0 tx=0x7fb10405b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0b61c06c4fe717a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:54.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.905+0000 7fb0fa000640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:20:54.907 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.905+0000 7fb0fa000640 1 -- 172.21.15.63:0/4221560111 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104048820 con 0x55b8ed1f32f0 2024-05-27T08:20:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.905+0000 7fb0fa000640 1 -- 172.21.15.63:0/4221560111 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7fb1040489c0 con 0x55b8ed1f32f0 2024-05-27T08:20:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.906+0000 7fb119fb6740 1 -- 172.21.15.63:0/4221560111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0fc014430 con 0x55b8ed1f32f0 2024-05-27T08:20:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.907+0000 7fb0fa000640 1 -- 172.21.15.63:0/4221560111 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb104031d30 con 0x55b8ed1f32f0 2024-05-27T08:20:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.907+0000 7fb119fb6740 10 client.32557 fetch_fsmap learned FSMap version 39 2024-05-27T08:20:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.907+0000 7fb119fb6740 1 -- 172.21.15.63:0/4221560111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b8ed642be0 con 0x55b8ed1f32f0 2024-05-27T08:20:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.907+0000 7fb0fa000640 1 -- 172.21.15.63:0/4221560111 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0fc014430 con 0x55b8ed1f32f0 2024-05-27T08:20:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.907+0000 7fb0fa000640 1 -- 172.21.15.63:0/4221560111 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb1040648e0 con 0x55b8ed1f32f0 2024-05-27T08:20:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.907+0000 7fb119fb6740 10 client.32557 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T08:20:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.907+0000 7fb119fb6740 10 client.32557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:20:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.907+0000 7fb119fb6740 20 client.32557 populate_metadata read hostname 'smithi063' 2024-05-27T08:20:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.907+0000 7fb119fb6740 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8ed1b01c0 0x55b8ed645330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.908+0000 7fb119fb6740 4 client.32557 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:20:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.908+0000 7fb119fb6740 1 -- 172.21.15.63:0/4221560111 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8ed531ae0 con 0x55b8ed1b01c0 2024-05-27T08:20:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.908+0000 7fb10b400640 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8ed1b01c0 0x55b8ed645330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.910+0000 7fb10b400640 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8ed1b01c0 0x55b8ed645330 crc :-1 s=READY pgs=795 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220e2244139f59e3 server_cookie=3d41924042cc0a47 in_seq=0 out_seq=0 2024-05-27T08:20:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:54.911+0000 7fb0fa000640 10 client.32557 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.206+0000 7fb0fa000640 1 -- 172.21.15.63:0/4221560111 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b8ed531ae0 con 0x55b8ed1b01c0 2024-05-27T08:20:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.206+0000 7fb0fa000640 10 client.32557 handle_command_reply: tid=0 2024-05-27T08:20:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.206+0000 7fb119fb6740 1 client.32557 shutdown 2024-05-27T08:20:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.207+0000 7fb119fb6740 1 -- 172.21.15.63:0/4221560111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1f32f0 msgr2=0x55b8ed5d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.207+0000 7fb119fb6740 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5d0f50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb104051bf0 tx=0x7fb10405b8b0 comp rx=0 tx=0).stop 2024-05-27T08:20:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.207+0000 7fb119fb6740 1 -- 172.21.15.63:0/4221560111 shutdown_connections 2024-05-27T08:20:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.207+0000 7fb119fb6740 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55b8ed1b01c0 0x55b8ed645330 crc :-1 s=READY pgs=795 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.207+0000 7fb0fa000640 0 client.32557 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:20:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.207+0000 7fb119fb6740 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b8ed21c1b0 0x55b8ed5ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.207+0000 7fb119fb6740 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8ed1fb4d0 0x55b8ed5d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.207+0000 7fb119fb6740 1 --2- 172.21.15.63:0/4221560111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b8ed1f32f0 0x55b8ed5d0f50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.207+0000 7fb119fb6740 1 -- 172.21.15.63:0/4221560111 >> 172.21.15.63:0/4221560111 conn(0x55b8ed210300 msgr2=0x55b8ed5c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.209+0000 7fb119fb6740 1 -- 172.21.15.63:0/4221560111 shutdown_connections 2024-05-27T08:20:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.209+0000 7fb119fb6740 1 -- 172.21.15.63:0/4221560111 wait complete. 2024-05-27T08:20:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.209+0000 7fb119fb6740 20 client.32557 trim_cache size 0 max 16384 2024-05-27T08:20:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:{ 2024-05-27T08:20:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout: "return_code": 0 2024-05-27T08:20:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stdout:} 2024-05-27T08:20:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.210+0000 7fb113000640 1 -- 172.21.15.63:0/1971964038 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0f00b2a20 msgr2=0x7fb0f00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.210+0000 7fb113000640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0f00b2a20 0x7fb0f00b4f30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x55b8ed5bfc10 tx=0x7fb104000f40 comp rx=0 tx=0).stop 2024-05-27T08:20:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.210+0000 7fb113000640 1 -- 172.21.15.63:0/1971964038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c1075d0 msgr2=0x7fb10c1a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.210+0000 7fb113000640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c1075d0 0x7fb10c1a1490 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009fd0 tx=0x7fb0f400b870 comp rx=0 tx=0).stop 2024-05-27T08:20:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.210+0000 7fb113000640 1 -- 172.21.15.63:0/1971964038 shutdown_connections 2024-05-27T08:20:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.210+0000 7fb113000640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fb0f00b2a20 0x7fb0f00b4f30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.210+0000 7fb113000640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb10c108000 0x7fb10c1a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.210+0000 7fb113000640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c1075d0 0x7fb10c1a1490 secure :-1 s=CLOSED pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009fd0 tx=0x7fb0f400b870 comp rx=0 tx=0).stop 2024-05-27T08:20:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.210+0000 7fb113000640 1 --2- 172.21.15.63:0/1971964038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb10c102b80 0x7fb10c1a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.210+0000 7fb113000640 1 -- 172.21.15.63:0/1971964038 >> 172.21.15.63:0/1971964038 conn(0x7fb10c0782d0 msgr2=0x7fb10c105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.212+0000 7fb113000640 1 -- 172.21.15.63:0/1971964038 shutdown_connections 2024-05-27T08:20:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi063.front.sepia.ceph.com.smithi063.stderr:2024-05-27T08:20:55.212+0000 7fb113000640 1 -- 172.21.15.63:0/1971964038 wait complete. 2024-05-27T08:20:55.945 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:55 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:20:55.945 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:55 smithi063 ceph-mon[31613]: pgmap v1508: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 3.3 MiB/s wr, 739 op/s 2024-05-27T08:20:55.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:55 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:20:55.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:55 smithi086 ceph-mon[34687]: pgmap v1508: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 3.3 MiB/s wr, 739 op/s 2024-05-27T08:20:55.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:55 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:20:55.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:55 smithi149 ceph-mon[35408]: pgmap v1508: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 3.3 MiB/s wr, 739 op/s 2024-05-27T08:20:56.979 DEBUG:teuthology.parallel:result is None 2024-05-27T08:20:56.979 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-27T08:20:57.020 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:20:57.020 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:20:57.121 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-27T08:20:57.122 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-27T08:20:57.132 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T08:20:57.160 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-27T08:20:57.170 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-27T08:20:57.171 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T08:20:57.171 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T08:20:57.171 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-27T08:20:57.327 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-27T08:20:57.470 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-27T08:20:57.759 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:20:57.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.867+0000 7f5c7a800640 1 -- 172.21.15.63:0/17711390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c7410db60 msgr2=0x7f5c7410ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.867+0000 7f5c7a800640 1 --2- 172.21.15.63:0/17711390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c7410db60 0x7f5c7410ff80 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009900 tx=0x7f5c6002a480 comp rx=0 tx=0).stop 2024-05-27T08:20:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.868+0000 7f5c7a800640 1 -- 172.21.15.63:0/17711390 shutdown_connections 2024-05-27T08:20:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.868+0000 7f5c7a800640 1 --2- 172.21.15.63:0/17711390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741104c0 0x7f5c741128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.868+0000 7f5c7a800640 1 --2- 172.21.15.63:0/17711390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c7410db60 0x7f5c7410ff80 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.868+0000 7f5c7a800640 1 --2- 172.21.15.63:0/17711390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c74076530 0x7f5c74076940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.868+0000 7f5c7a800640 1 -- 172.21.15.63:0/17711390 >> 172.21.15.63:0/17711390 conn(0x7f5c740ff350 msgr2=0x7f5c741017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.868+0000 7f5c7a800640 1 -- 172.21.15.63:0/17711390 shutdown_connections 2024-05-27T08:20:57.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.868+0000 7f5c7a800640 1 -- 172.21.15.63:0/17711390 wait complete. 2024-05-27T08:20:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.869+0000 7f5c7a800640 1 Processor -- start 2024-05-27T08:20:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.869+0000 7f5c7a800640 1 -- start start 2024-05-27T08:20:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.869+0000 7f5c7a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c74076530 0x7f5c741a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.869+0000 7f5c7a800640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c7410db60 0x7f5c741a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.869+0000 7f5c7a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741104c0 0x7f5c7419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.869+0000 7f5c7a800640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74115ed0 con 0x7f5c7410db60 2024-05-27T08:20:57.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.869+0000 7f5c7a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74115d50 con 0x7f5c741104c0 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.869+0000 7f5c7a800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74116050 con 0x7f5c74076530 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c72a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c7410db60 0x7f5c741a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c72a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c7410db60 0x7f5c741a2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39274/0 (socket says 172.21.15.63:39274) 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c72a00640 1 -- 172.21.15.63:0/160479159 learned_addr learned my addr 172.21.15.63:0/160479159 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c73e00640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741104c0 0x7f5c7419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c73400640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c74076530 0x7f5c741a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c72a00640 1 -- 172.21.15.63:0/160479159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c74076530 msgr2=0x7f5c741a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c72a00640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c74076530 0x7f5c741a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c72a00640 1 -- 172.21.15.63:0/160479159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741104c0 msgr2=0x7f5c7419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c72a00640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741104c0 0x7f5c7419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c72a00640 1 -- 172.21.15.63:0/160479159 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7419cf50 con 0x7f5c7410db60 2024-05-27T08:20:57.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.870+0000 7f5c73e00640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741104c0 0x7f5c7419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:20:57.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.871+0000 7f5c72a00640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c7410db60 0x7f5c741a2b80 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7f5c60005940 tx=0x7f5c600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:57.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.871+0000 7f5c6be00640 1 -- 172.21.15.63:0/160479159 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6003f030 con 0x7f5c7410db60 2024-05-27T08:20:57.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.871+0000 7f5c6be00640 1 -- 172.21.15.63:0/160479159 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c60003c90 con 0x7f5c7410db60 2024-05-27T08:20:57.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.871+0000 7f5c6be00640 1 -- 172.21.15.63:0/160479159 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60035070 con 0x7f5c7410db60 2024-05-27T08:20:57.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.871+0000 7f5c7a800640 1 -- 172.21.15.63:0/160479159 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7419d1e0 con 0x7f5c7410db60 2024-05-27T08:20:57.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.871+0000 7f5c7a800640 1 -- 172.21.15.63:0/160479159 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7419d680 con 0x7f5c7410db60 2024-05-27T08:20:57.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.872+0000 7f5c7a800640 1 -- 172.21.15.63:0/160479159 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c74077850 con 0x7f5c7410db60 2024-05-27T08:20:57.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.876+0000 7f5c6be00640 1 -- 172.21.15.63:0/160479159 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5c60042030 con 0x7f5c7410db60 2024-05-27T08:20:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.878+0000 7f5c6be00640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c580b29d0 0x7f5c580b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:57.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.878+0000 7f5c6be00640 1 -- 172.21.15.63:0/160479159 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7f5c601033b0 con 0x7f5c7410db60 2024-05-27T08:20:57.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.878+0000 7f5c73400640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c580b29d0 0x7f5c580b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:57.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.879+0000 7f5c73400640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c580b29d0 0x7f5c580b4ee0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5c64005fd0 tx=0x7f5c64005850 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:57.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:57.881+0000 7f5c6be00640 1 -- 172.21.15.63:0/160479159 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5c600ca960 con 0x7f5c7410db60 2024-05-27T08:20:57.889 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:57 smithi063 ceph-mon[31613]: pgmap v1509: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.8 MiB/s wr, 633 op/s 2024-05-27T08:20:57.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:57 smithi149 ceph-mon[35408]: pgmap v1509: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.8 MiB/s wr, 633 op/s 2024-05-27T08:20:57.965 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:57 smithi086 ceph-mon[34687]: pgmap v1509: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.8 MiB/s wr, 633 op/s 2024-05-27T08:20:58.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.279+0000 7f5c7a800640 1 -- 172.21.15.63:0/160479159 --> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5c74111080 con 0x7f5c580b29d0 2024-05-27T08:20:58.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.356+0000 7f5c6be00640 1 -- 172.21.15.63:0/160479159 <== mgr.15216 v2:172.21.15.63:6800/1343315489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f5c74111080 con 0x7f5c580b29d0 2024-05-27T08:20:58.358 INFO:teuthology.orchestra.run.smithi063.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-27T00:00:00", "created": "2024-05-27T07:33:55", "first": "2024-05-27T07:34:00", "last": "2024-05-27T08:20:03", "last_pruned": "2024-05-27T08:20:08", "created_count": 47, "pruned_count": 40, "active": true} 2024-05-27T08:20:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.360+0000 7f5c7a800640 1 -- 172.21.15.63:0/160479159 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c580b29d0 msgr2=0x7f5c580b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.360+0000 7f5c7a800640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c580b29d0 0x7f5c580b4ee0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5c64005fd0 tx=0x7f5c64005850 comp rx=0 tx=0).stop 2024-05-27T08:20:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.360+0000 7f5c7a800640 1 -- 172.21.15.63:0/160479159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c7410db60 msgr2=0x7f5c741a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.360+0000 7f5c7a800640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c7410db60 0x7f5c741a2b80 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7f5c60005940 tx=0x7f5c600042f0 comp rx=0 tx=0).stop 2024-05-27T08:20:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.360+0000 7f5c7a800640 1 -- 172.21.15.63:0/160479159 shutdown_connections 2024-05-27T08:20:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.360+0000 7f5c7a800640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f5c580b29d0 0x7f5c580b4ee0 secure :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5c64005fd0 tx=0x7f5c64005850 comp rx=0 tx=0).stop 2024-05-27T08:20:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.360+0000 7f5c7a800640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c741104c0 0x7f5c7419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.360+0000 7f5c7a800640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c7410db60 0x7f5c741a2b80 unknown :-1 s=CLOSED pgs=3010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.360+0000 7f5c7a800640 1 --2- 172.21.15.63:0/160479159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c74076530 0x7f5c741a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:58.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.360+0000 7f5c7a800640 1 -- 172.21.15.63:0/160479159 >> 172.21.15.63:0/160479159 conn(0x7f5c740ff350 msgr2=0x7f5c74110970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:58.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.361+0000 7f5c7a800640 1 -- 172.21.15.63:0/160479159 shutdown_connections 2024-05-27T08:20:58.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.361+0000 7f5c7a800640 1 -- 172.21.15.63:0/160479159 wait complete. 2024-05-27T08:20:58.413 DEBUG:teuthology.orchestra.run.smithi063:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-27T08:20:58.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.817+0000 7f649d000640 1 -- 172.21.15.63:0/2149484627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6498111de0 msgr2=0x7f6498112270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:58.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.817+0000 7f649d000640 1 --2- 172.21.15.63:0/2149484627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6498111de0 0x7f6498112270 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6484009860 tx=0x7f648402a3d0 comp rx=0 tx=0).stop 2024-05-27T08:20:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.819+0000 7f649d000640 1 -- 172.21.15.63:0/2149484627 shutdown_connections 2024-05-27T08:20:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.819+0000 7f649d000640 1 --2- 172.21.15.63:0/2149484627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6498112900 0x7f6498118eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.819+0000 7f649d000640 1 --2- 172.21.15.63:0/2149484627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6498111de0 0x7f6498112270 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.819+0000 7f649d000640 1 --2- 172.21.15.63:0/2149484627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f649810a490 0x7f649810a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:58.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.819+0000 7f649d000640 1 -- 172.21.15.63:0/2149484627 >> 172.21.15.63:0/2149484627 conn(0x7f649806e8f0 msgr2=0x7f649806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:58.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.819+0000 7f649d000640 1 -- 172.21.15.63:0/2149484627 shutdown_connections 2024-05-27T08:20:58.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.819+0000 7f649d000640 1 -- 172.21.15.63:0/2149484627 wait complete. 2024-05-27T08:20:58.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.820+0000 7f649d000640 1 Processor -- start 2024-05-27T08:20:58.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.820+0000 7f649d000640 1 -- start start 2024-05-27T08:20:58.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.821+0000 7f649d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649810a490 0x7f64981a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:58.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.821+0000 7f649d000640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6498111de0 0x7f64981a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:58.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.821+0000 7f649d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6498112900 0x7f64981a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:58.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.821+0000 7f649d000640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f649811c530 con 0x7f6498111de0 2024-05-27T08:20:58.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.821+0000 7f649d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f649811c3b0 con 0x7f649810a490 2024-05-27T08:20:58.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.821+0000 7f649d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f649811c6b0 con 0x7f6498112900 2024-05-27T08:20:58.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.822+0000 7f6495e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649810a490 0x7f64981a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:58.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.822+0000 7f6495e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649810a490 0x7f64981a2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:60594/0 (socket says 172.21.15.63:60594) 2024-05-27T08:20:58.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.822+0000 7f6495e00640 1 -- 172.21.15.63:0/3707152481 learned_addr learned my addr 172.21.15.63:0/3707152481 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:20:58.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.822+0000 7f6496800640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6498112900 0x7f64981a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:58.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.822+0000 7f6495e00640 1 -- 172.21.15.63:0/3707152481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6498112900 msgr2=0x7f64981a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:58.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.822+0000 7f6495e00640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6498112900 0x7f64981a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:58.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.822+0000 7f6495e00640 1 -- 172.21.15.63:0/3707152481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6498111de0 msgr2=0x7f64981a3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:58.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.822+0000 7f6495e00640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6498111de0 0x7f64981a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:58.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.822+0000 7f6495e00640 1 -- 172.21.15.63:0/3707152481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64981a7d90 con 0x7f649810a490 2024-05-27T08:20:58.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.823+0000 7f6495e00640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649810a490 0x7f64981a2fb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f648800e9f0 tx=0x7f648800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:58.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.823+0000 7f648ea00640 1 -- 172.21.15.63:0/3707152481 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648800ccc0 con 0x7f649810a490 2024-05-27T08:20:58.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.824+0000 7f648ea00640 1 -- 172.21.15.63:0/3707152481 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f64880054b0 con 0x7f649810a490 2024-05-27T08:20:58.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.824+0000 7f648ea00640 1 -- 172.21.15.63:0/3707152481 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488010430 con 0x7f649810a490 2024-05-27T08:20:58.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.824+0000 7f649d000640 1 -- 172.21.15.63:0/3707152481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64981a8080 con 0x7f649810a490 2024-05-27T08:20:58.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.824+0000 7f649d000640 1 -- 172.21.15.63:0/3707152481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64981a8520 con 0x7f649810a490 2024-05-27T08:20:58.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.828+0000 7f649d000640 1 -- 172.21.15.63:0/3707152481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6498050860 con 0x7f649810a490 2024-05-27T08:20:58.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.831+0000 7f648ea00640 1 -- 172.21.15.63:0/3707152481 <== mon.2 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f64880046c0 con 0x7f649810a490 2024-05-27T08:20:58.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.834+0000 7f648ea00640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f647c0b6de0 0x7f647c0b92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:20:58.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.834+0000 7f648ea00640 1 -- 172.21.15.63:0/3707152481 <== mon.2 v2:172.21.15.86:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7f64880e2d00 con 0x7f649810a490 2024-05-27T08:20:58.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.835+0000 7f6495400640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f647c0b6de0 0x7f647c0b92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:20:58.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.837+0000 7f6495400640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f647c0b6de0 0x7f647c0b92f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f648402a8f0 tx=0x7f6484005c90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:20:58.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:58.844+0000 7f648ea00640 1 -- 172.21.15.63:0/3707152481 <== mon.2 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f64880aa2b0 con 0x7f649810a490 2024-05-27T08:20:59.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.252+0000 7f649d000640 1 -- 172.21.15.63:0/3707152481 --> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f6498063100 con 0x7f647c0b6de0 2024-05-27T08:20:59.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.333+0000 7f648ea00640 1 -- 172.21.15.63:0/3707152481 <== mgr.15216 v2:172.21.15.63:6800/1343315489 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f6498063100 con 0x7f647c0b6de0 2024-05-27T08:20:59.335 INFO:teuthology.orchestra.run.smithi063.stdout:/ 1m 6m3h 2024-05-27T08:20:59.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.337+0000 7f649d000640 1 -- 172.21.15.63:0/3707152481 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f647c0b6de0 msgr2=0x7f647c0b92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:59.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.337+0000 7f649d000640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f647c0b6de0 0x7f647c0b92f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f648402a8f0 tx=0x7f6484005c90 comp rx=0 tx=0).stop 2024-05-27T08:20:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.337+0000 7f649d000640 1 -- 172.21.15.63:0/3707152481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649810a490 msgr2=0x7f64981a2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:20:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.337+0000 7f649d000640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649810a490 0x7f64981a2fb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f648800e9f0 tx=0x7f648800eeb0 comp rx=0 tx=0).stop 2024-05-27T08:20:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.338+0000 7f649d000640 1 -- 172.21.15.63:0/3707152481 shutdown_connections 2024-05-27T08:20:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.338+0000 7f649d000640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f647c0b6de0 0x7f647c0b92f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:59.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.338+0000 7f649d000640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6498112900 0x7f64981a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:59.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.338+0000 7f649d000640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6498111de0 0x7f64981a3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:59.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.338+0000 7f649d000640 1 --2- 172.21.15.63:0/3707152481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f649810a490 0x7f64981a2fb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:20:59.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.338+0000 7f649d000640 1 -- 172.21.15.63:0/3707152481 >> 172.21.15.63:0/3707152481 conn(0x7f649806e8f0 msgr2=0x7f6498073990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:20:59.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.338+0000 7f649d000640 1 -- 172.21.15.63:0/3707152481 shutdown_connections 2024-05-27T08:20:59.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:20:59.341+0000 7f649d000640 1 -- 172.21.15.63:0/3707152481 wait complete. 2024-05-27T08:20:59.397 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:59 smithi063 ceph-mon[31613]: from='client.23829 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:20:59.398 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:20:59 smithi063 ceph-mon[31613]: pgmap v1510: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 419 KiB/s wr, 212 op/s 2024-05-27T08:20:59.401 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-27T08:20:59.412 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-27T08:20:59.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:59 smithi086 ceph-mon[34687]: from='client.23829 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:20:59.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:20:59 smithi086 ceph-mon[34687]: pgmap v1510: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 419 KiB/s wr, 212 op/s 2024-05-27T08:20:59.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:59 smithi149 ceph-mon[35408]: from='client.23829 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:20:59.466 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:20:59 smithi149 ceph-mon[35408]: pgmap v1510: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 419 KiB/s wr, 212 op/s 2024-05-27T08:21:00.136 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:00 smithi063 ceph-mon[31613]: from='client.42764 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:21:00.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.404+0000 7fbe09000700 1 -- 172.21.15.63:0/694074895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe04107de0 msgr2=0x7fbe041081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:00.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.404+0000 7fbe09000700 1 --2- 172.21.15.63:0/694074895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe04107de0 0x7fbe041081f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009990 tx=0x7fbdf0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.405+0000 7fbe09000700 1 -- 172.21.15.63:0/694074895 shutdown_connections 2024-05-27T08:21:00.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.405+0000 7fbe09000700 1 --2- 172.21.15.63:0/694074895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe04108730 0x7fbe04110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.405+0000 7fbe09000700 1 --2- 172.21.15.63:0/694074895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe041034b0 0x7fbe041038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.405+0000 7fbe09000700 1 --2- 172.21.15.63:0/694074895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe04107de0 0x7fbe041081f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.405+0000 7fbe09000700 1 -- 172.21.15.63:0/694074895 >> 172.21.15.63:0/694074895 conn(0x7fbe040fedb0 msgr2=0x7fbe041011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:00.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.407+0000 7fbe09000700 1 -- 172.21.15.63:0/694074895 shutdown_connections 2024-05-27T08:21:00.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.407+0000 7fbe09000700 1 -- 172.21.15.63:0/694074895 wait complete. 2024-05-27T08:21:00.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.409+0000 7fbe09000700 1 Processor -- start 2024-05-27T08:21:00.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.410+0000 7fbe09000700 1 -- start start 2024-05-27T08:21:00.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.411+0000 7fbe09000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041034b0 0x7fbe04075440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.411+0000 7fbe09000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe04107de0 0x7fbe0406e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.411+0000 7fbe09000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe04108730 0x7fbe0406ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.411+0000 7fbe09000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbe041143b0 con 0x7fbe04108730 2024-05-27T08:21:00.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.411+0000 7fbe09000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04114230 con 0x7fbe04107de0 2024-05-27T08:21:00.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.411+0000 7fbe09000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04114530 con 0x7fbe041034b0 2024-05-27T08:21:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.412+0000 7fbe01e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041034b0 0x7fbe04075440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.412+0000 7fbe01e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041034b0 0x7fbe04075440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:36356/0 (socket says 172.21.15.63:36356) 2024-05-27T08:21:00.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.412+0000 7fbe01e00700 1 -- 172.21.15.63:0/3109137016 learned_addr learned my addr 172.21.15.63:0/3109137016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:00.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.412+0000 7fbe01400700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe04107de0 0x7fbe0406e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.412+0000 7fbe02800700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe04108730 0x7fbe0406ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.412+0000 7fbe01e00700 1 -- 172.21.15.63:0/3109137016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe04107de0 msgr2=0x7fbe0406e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:00.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.412+0000 7fbe01e00700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe04107de0 0x7fbe0406e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.412+0000 7fbe01e00700 1 -- 172.21.15.63:0/3109137016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe04108730 msgr2=0x7fbe0406ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:00.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.412+0000 7fbe01e00700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe04108730 0x7fbe0406ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.413+0000 7fbe01e00700 1 -- 172.21.15.63:0/3109137016 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe04072f70 con 0x7fbe041034b0 2024-05-27T08:21:00.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.413+0000 7fbe01400700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe04107de0 0x7fbe0406e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:00.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.414+0000 7fbe01e00700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041034b0 0x7fbe04075440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400d980 tx=0x7fbdf400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:00.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.414+0000 7fbdfaa00700 1 -- 172.21.15.63:0/3109137016 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4016070 con 0x7fbe041034b0 2024-05-27T08:21:00.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.414+0000 7fbdfaa00700 1 -- 172.21.15.63:0/3109137016 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdf4004020 con 0x7fbe041034b0 2024-05-27T08:21:00.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.414+0000 7fbe09000700 1 -- 172.21.15.63:0/3109137016 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe04073200 con 0x7fbe041034b0 2024-05-27T08:21:00.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.415+0000 7fbdfaa00700 1 -- 172.21.15.63:0/3109137016 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4004c50 con 0x7fbe041034b0 2024-05-27T08:21:00.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.415+0000 7fbe09000700 1 -- 172.21.15.63:0/3109137016 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe04073740 con 0x7fbe041034b0 2024-05-27T08:21:00.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.422+0000 7fbdfaa00700 1 -- 172.21.15.63:0/3109137016 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbdf400bb60 con 0x7fbe041034b0 2024-05-27T08:21:00.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.423+0000 7fbe17adfb80 1 Processor -- start 2024-05-27T08:21:00.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.423+0000 7fbe17adfb80 1 -- start start 2024-05-27T08:21:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.425+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507cf70ec0 0x56507d0fe080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.426+0000 7fbe02800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507cf70ec0 0x56507d0fe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.426+0000 7fbe02800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507cf70ec0 0x56507d0fe080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:36358/0 (socket says 172.21.15.63:36358) 2024-05-27T08:21:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.426+0000 7fbdfaa00700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbde80b2bc0 0x7fbde80b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.426+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507d0fe5c0 0x56507d0f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.426+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d0f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.427+0000 7fbe02800700 1 -- 172.21.15.63:0/1806820277 learned_addr learned my addr 172.21.15.63:0/1806820277 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.427+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1806820277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56507d01dec0 con 0x56507d0f8640 2024-05-27T08:21:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.427+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1806820277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56507d00da60 con 0x56507d0fe5c0 2024-05-27T08:21:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.427+0000 7fbdfaa00700 1 -- 172.21.15.63:0/3109137016 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7fbdf40e2740 con 0x7fbe041034b0 2024-05-27T08:21:00.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.427+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1806820277 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56507d011aa0 con 0x56507cf70ec0 2024-05-27T08:21:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.428+0000 7fbe01400700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbde80b2bc0 0x7fbde80b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.428+0000 7fbe02800700 1 -- 172.21.15.63:0/1806820277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507d0fe5c0 msgr2=0x56507d0f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:00.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.428+0000 7fbe02800700 1 --2- 172.21.15.63:0/1806820277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507d0fe5c0 0x56507d0f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.428+0000 7fbe02800700 1 -- 172.21.15.63:0/1806820277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 msgr2=0x56507d0f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.428+0000 7fbe02800700 1 --2- 172.21.15.63:0/1806820277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d0f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.428+0000 7fbe02800700 1 -- 172.21.15.63:0/1806820277 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507cfd64a0 con 0x56507cf70ec0 2024-05-27T08:21:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.429+0000 7fbe02800700 1 --2- 172.21.15.63:0/1806820277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507cf70ec0 0x56507d0fe080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0085b0 tx=0x7fbdfc0088c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c616a7845a4853d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.430+0000 7fbdef400700 1 -- 172.21.15.63:0/1806820277 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc003a00 con 0x56507cf70ec0 2024-05-27T08:21:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.430+0000 7fbdef400700 1 -- 172.21.15.63:0/1806820277 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdfc003ba0 con 0x56507cf70ec0 2024-05-27T08:21:00.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.430+0000 7fbdef400700 1 -- 172.21.15.63:0/1806820277 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc016760 con 0x56507cf70ec0 2024-05-27T08:21:00.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.432+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1806820277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507cf70ec0 msgr2=0x56507d0fe080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:00.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.432+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/1806820277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507cf70ec0 0x56507d0fe080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0085b0 tx=0x7fbdfc0088c0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.432+0000 7fbe01400700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbde80b2bc0 0x7fbde80b50d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbe04106a50 tx=0x7fbdf0003680 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:00.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.432+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1806820277 shutdown_connections 2024-05-27T08:21:00.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.432+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/1806820277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d0f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.432+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/1806820277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507cf70ec0 0x56507d0fe080 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.432+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/1806820277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507d0fe5c0 0x56507d0f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.432+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1806820277 >> 172.21.15.63:0/1806820277 conn(0x56507cfd1040 msgr2=0x56507d0ea950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:00.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.433+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1806820277 shutdown_connections 2024-05-27T08:21:00.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.433+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1806820277 wait complete. 2024-05-27T08:21:00.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.434+0000 7fbe17adfb80 1 -- start start 2024-05-27T08:21:00.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.434+0000 7fbe17adfb80 1 Processor -- start 2024-05-27T08:21:00.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.436+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 0x56507d101e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.436+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d102380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.436+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d1028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.437+0000 7fbe01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.437+0000 7fbe01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50238/0 (socket says 172.21.15.63:50238) 2024-05-27T08:21:00.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.437+0000 7fbe01400700 1 -- 172.21.15.63:0/603393375 learned_addr learned my addr 172.21.15.63:0/603393375 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:00.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.437+0000 7fbe02800700 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d1028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.437+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56507cf14020 con 0x56507d0f8640 2024-05-27T08:21:00.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.438+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56507cfae7e0 con 0x56507cf70ec0 2024-05-27T08:21:00.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.438+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56507d105090 con 0x56507d0fe5c0 2024-05-27T08:21:00.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.439+0000 7fbe17adfb80 10 client.0 _ll_register_callbacks cb 0x56507cfb7cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:21:00.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.438+0000 7fbe01e00700 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 0x56507d101e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.439+0000 7fbe02800700 1 -- 172.21.15.63:0/603393375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 msgr2=0x56507d101e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:00.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.439+0000 7fbe02800700 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 0x56507d101e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.439+0000 7fbe02800700 1 -- 172.21.15.63:0/603393375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 msgr2=0x56507d102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:00.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.439+0000 7fbe02800700 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.439+0000 7fbe02800700 1 -- 172.21.15.63:0/603393375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56507cf5d950 con 0x56507d0fe5c0 2024-05-27T08:21:00.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.440+0000 7fbe02800700 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d1028c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0085b0 tx=0x7fbdfc006060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0f31d4b6e47918e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:00.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.440+0000 7fbde3e00700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-05-27T08:21:00.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.440+0000 7fbe01400700 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:21:00.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.443+0000 7fbde3e00700 1 -- 172.21.15.63:0/603393375 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc004bb0 con 0x56507d0fe5c0 2024-05-27T08:21:00.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.443+0000 7fbde3e00700 1 -- 172.21.15.63:0/603393375 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7fbdfc004d50 con 0x56507d0fe5c0 2024-05-27T08:21:00.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.443+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdfc003ba0 con 0x56507d0fe5c0 2024-05-27T08:21:00.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.444+0000 7fbde3e00700 1 -- 172.21.15.63:0/603393375 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbdfc005020 con 0x56507d0fe5c0 2024-05-27T08:21:00.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.444+0000 7fbe17adfb80 10 client.32581 fetch_fsmap learned FSMap version 39 2024-05-27T08:21:00.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.444+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56507cfde0e0 con 0x56507d0fe5c0 2024-05-27T08:21:00.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.444+0000 7fbde3e00700 1 -- 172.21.15.63:0/603393375 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdfc003ba0 con 0x56507d0fe5c0 2024-05-27T08:21:00.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.446+0000 7fbde3e00700 1 -- 172.21.15.63:0/603393375 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fbdfc01f4a0 con 0x56507d0fe5c0 2024-05-27T08:21:00.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.447+0000 7fbe17adfb80 10 client.32581 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T08:21:00.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.447+0000 7fbe17adfb80 10 client.32581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:21:00.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.447+0000 7fbe17adfb80 20 client.32581 populate_metadata read hostname 'smithi063' 2024-05-27T08:21:00.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.447+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507d111ee0 0x56507d114300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.447+0000 7fbe17adfb80 4 client.32581 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:21:00.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.447+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56507d01fe10 con 0x56507d111ee0 2024-05-27T08:21:00.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.447+0000 7fbe01e00700 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507d111ee0 0x56507d114300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.448+0000 7fbe01e00700 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507d111ee0 0x56507d114300 crc :-1 s=READY pgs=796 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=420d9cbf0f265690 server_cookie=c5f48fa7de379875 in_seq=0 out_seq=0 2024-05-27T08:21:00.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.448+0000 7fbde3e00700 10 client.32581 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:00.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.449+0000 7fbde3e00700 1 -- 172.21.15.63:0/603393375 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56507d01fe10 con 0x56507d111ee0 2024-05-27T08:21:00.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.449+0000 7fbde3e00700 10 client.32581 handle_command_reply: tid=0 2024-05-27T08:21:00.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.449+0000 7fbe17adfb80 1 client.32581 shutdown 2024-05-27T08:21:00.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.450+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 msgr2=0x56507d1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:00.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.450+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d1028c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0085b0 tx=0x7fbdfc006060 comp rx=0 tx=0).stop 2024-05-27T08:21:00.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.450+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 shutdown_connections 2024-05-27T08:21:00.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.450+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507d111ee0 0x56507d114300 crc :-1 s=READY pgs=796 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.450+0000 7fbde3e00700 0 client.32581 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:00.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.450+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d1028c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.450+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 0x56507d101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.450+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/603393375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.450+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 >> 172.21.15.63:0/603393375 conn(0x56507cfd1040 msgr2=0x56507d0ea600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:00.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.451+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 shutdown_connections 2024-05-27T08:21:00.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.452+0000 7fbe17adfb80 1 -- 172.21.15.63:0/603393375 wait complete. 2024-05-27T08:21:00.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.452+0000 7fbe17adfb80 20 client.32581 trim_cache size 0 max 16384 2024-05-27T08:21:00.461 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:00 smithi086 ceph-mon[34687]: from='client.42764 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:21:00.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:00 smithi149 ceph-mon[35408]: from='client.42764 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:21:00.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.494+0000 7fbe17adfb80 1 Processor -- start 2024-05-27T08:21:00.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.494+0000 7fbe17adfb80 1 -- start start 2024-05-27T08:21:00.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.495+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507cf70ec0 0x56507d0f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.495+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507d0f8640 0x56507d100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.495+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d119930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.495+0000 7fbe17adfb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbde4011810 con 0x56507cf70ec0 2024-05-27T08:21:00.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.495+0000 7fbe17adfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbde400f070 con 0x56507d0f8640 2024-05-27T08:21:00.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.495+0000 7fbe17adfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbde400cee0 con 0x56507d0fe5c0 2024-05-27T08:21:00.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.496+0000 7fbe02800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d119930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.496+0000 7fbe01e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507cf70ec0 0x56507d0f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.496+0000 7fbe02800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d119930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:36380/0 (socket says 172.21.15.63:36380) 2024-05-27T08:21:00.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.496+0000 7fbe02800700 1 -- 172.21.15.63:0/1512589396 learned_addr learned my addr 172.21.15.63:0/1512589396 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:00.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.496+0000 7fbe01400700 1 --2- 172.21.15.63:0/1512589396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507d0f8640 0x56507d100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.496+0000 7fbe01e00700 1 -- 172.21.15.63:0/1512589396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 msgr2=0x56507d119930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:00.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.496+0000 7fbe01e00700 1 --2- 172.21.15.63:0/1512589396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d119930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.496+0000 7fbe01e00700 1 -- 172.21.15.63:0/1512589396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507d0f8640 msgr2=0x56507d100dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:00.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.497+0000 7fbe01e00700 1 --2- 172.21.15.63:0/1512589396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507d0f8640 0x56507d100dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.497+0000 7fbe01e00700 1 -- 172.21.15.63:0/1512589396 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507d0fda50 con 0x56507cf70ec0 2024-05-27T08:21:00.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.497+0000 7fbe01400700 1 --2- 172.21.15.63:0/1512589396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507d0f8640 0x56507d100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:00.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.497+0000 7fbe01e00700 1 --2- 172.21.15.63:0/1512589396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507cf70ec0 0x56507d0f8100 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40dfa00 tx=0x7fbdf400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91b54959e32527d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:00.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.497+0000 7fbdeea00700 1 -- 172.21.15.63:0/1512589396 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40c8070 con 0x56507cf70ec0 2024-05-27T08:21:00.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.498+0000 7fbdeea00700 1 -- 172.21.15.63:0/1512589396 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdf40ad940 con 0x56507cf70ec0 2024-05-27T08:21:00.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.498+0000 7fbdeea00700 1 -- 172.21.15.63:0/1512589396 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40a9a80 con 0x56507cf70ec0 2024-05-27T08:21:00.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.499+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1512589396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507cf70ec0 msgr2=0x56507d0f8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:00.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.499+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/1512589396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507cf70ec0 0x56507d0f8100 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40dfa00 tx=0x7fbdf400ded0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.499+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1512589396 shutdown_connections 2024-05-27T08:21:00.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.499+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/1512589396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d119930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.499+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/1512589396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507cf70ec0 0x56507d0f8100 unknown :-1 s=CLOSED pgs=3011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.499+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/1512589396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507d0f8640 0x56507d100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.499+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1512589396 >> 172.21.15.63:0/1512589396 conn(0x56507cfd1040 msgr2=0x56507d102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:00.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.499+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1512589396 shutdown_connections 2024-05-27T08:21:00.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.499+0000 7fbe17adfb80 1 -- 172.21.15.63:0/1512589396 wait complete. 2024-05-27T08:21:00.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.500+0000 7fbe17adfb80 1 -- start start 2024-05-27T08:21:00.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.500+0000 7fbe17adfb80 1 Processor -- start 2024-05-27T08:21:00.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 0x56507d10a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d107350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe17adfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d107890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe17adfb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbde400d530 con 0x56507d0f8640 2024-05-27T08:21:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe17adfb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbde400e9a0 con 0x56507cf70ec0 2024-05-27T08:21:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe17adfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbde400dc00 con 0x56507d0fe5c0 2024-05-27T08:21:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe17adfb80 10 client.0 _ll_register_callbacks cb 0x56507cfb7cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:21:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d107350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe01400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d107350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50260/0 (socket says 172.21.15.63:50260) 2024-05-27T08:21:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe01400700 1 -- 172.21.15.63:0/3391501325 learned_addr learned my addr 172.21.15.63:0/3391501325 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe01e00700 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 0x56507d10a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.501+0000 7fbe02800700 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.502+0000 7fbe01e00700 1 -- 172.21.15.63:0/3391501325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 msgr2=0x56507d107890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.502+0000 7fbe01e00700 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d107890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.502+0000 7fbe01e00700 1 -- 172.21.15.63:0/3391501325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 msgr2=0x56507d107350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.502+0000 7fbe01e00700 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d107350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.502+0000 7fbe01e00700 1 -- 172.21.15.63:0/3391501325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56507d107dd0 con 0x56507cf70ec0 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.502+0000 7fbe01e00700 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 0x56507d10a740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40a9520 tx=0x7fbdf40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aafdf6e262c2018f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.502+0000 7fbde3e00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.502+0000 7fbde3e00700 1 -- 172.21.15.63:0/3391501325 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40c8070 con 0x56507cf70ec0 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.503+0000 7fbde3e00700 1 -- 172.21.15.63:0/3391501325 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(178..178 src has 1..178) ==== 9595+0+0 (secure 0 0 0) 0x7fbdf40b2170 con 0x56507cf70ec0 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.503+0000 7fbe17adfb80 1 -- 172.21.15.63:0/3391501325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdf40ad940 con 0x56507cf70ec0 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.503+0000 7fbde3e00700 1 -- 172.21.15.63:0/3391501325 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbdf40a90a0 con 0x56507cf70ec0 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.503+0000 7fbe17adfb80 10 client.42776 fetch_fsmap learned FSMap version 39 2024-05-27T08:21:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.503+0000 7fbe17adfb80 1 -- 172.21.15.63:0/3391501325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56507d17d430 con 0x56507cf70ec0 2024-05-27T08:21:00.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.503+0000 7fbde3e00700 1 -- 172.21.15.63:0/3391501325 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdf40ad940 con 0x56507cf70ec0 2024-05-27T08:21:00.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.503+0000 7fbde3e00700 1 -- 172.21.15.63:0/3391501325 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fbdf40041d0 con 0x56507cf70ec0 2024-05-27T08:21:00.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.503+0000 7fbe17adfb80 10 client.42776 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T08:21:00.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.504+0000 7fbe17adfb80 10 client.42776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:21:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.504+0000 7fbe17adfb80 20 client.42776 populate_metadata read hostname 'smithi063' 2024-05-27T08:21:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.504+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507d17d9a0 0x56507d17fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.504+0000 7fbe17adfb80 4 client.42776 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:21:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.504+0000 7fbe17adfb80 1 -- 172.21.15.63:0/3391501325 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56507d0195f0 con 0x56507d17d9a0 2024-05-27T08:21:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.504+0000 7fbe01400700 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507d17d9a0 0x56507d17fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.505+0000 7fbde3e00700 10 client.42776 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.505+0000 7fbe01400700 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507d17d9a0 0x56507d17fdc0 crc :-1 s=READY pgs=797 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47892069593a6953 server_cookie=16b8a9f010574855 in_seq=0 out_seq=0 2024-05-27T08:21:00.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.506+0000 7fbde3e00700 1 -- 172.21.15.63:0/3391501325 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56507d0195f0 con 0x56507d17d9a0 2024-05-27T08:21:00.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.506+0000 7fbde3e00700 10 client.42776 handle_command_reply: tid=0 2024-05-27T08:21:00.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.506+0000 7fbe17adfb80 1 client.42776 shutdown 2024-05-27T08:21:00.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 1 -- 172.21.15.63:0/3391501325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 msgr2=0x56507d10a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:00.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 0x56507d10a740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40a9520 tx=0x7fbdf40047c0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 1 -- 172.21.15.63:0/3391501325 shutdown_connections 2024-05-27T08:21:00.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x56507d17d9a0 0x56507d17fdc0 crc :-1 s=READY pgs=797 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56507d0fe5c0 0x56507d107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56507cf70ec0 0x56507d10a740 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40a9520 tx=0x7fbdf40047c0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 1 --2- 172.21.15.63:0/3391501325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56507d0f8640 0x56507d107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 1 -- 172.21.15.63:0/3391501325 >> 172.21.15.63:0/3391501325 conn(0x56507cfd1040 msgr2=0x56507d102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:00.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbde3e00700 0 client.42776 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:00.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 1 -- 172.21.15.63:0/3391501325 shutdown_connections 2024-05-27T08:21:00.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 1 -- 172.21.15.63:0/3391501325 wait complete. 2024-05-27T08:21:00.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.507+0000 7fbe17adfb80 20 client.42776 trim_cache size 0 max 16384 2024-05-27T08:21:00.510 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:21:00.510 INFO:teuthology.orchestra.run.smithi063.stdout: "return_code": 0, 2024-05-27T08:21:00.511 INFO:teuthology.orchestra.run.smithi063.stdout: "scrub_tag": "ccda5439-e9f6-4a95-844c-35ac68ecd910", 2024-05-27T08:21:00.511 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": "asynchronous" 2024-05-27T08:21:00.511 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:21:00.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.508+0000 7fbe09000700 1 -- 172.21.15.63:0/3109137016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbde80b2bc0 msgr2=0x7fbde80b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:00.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.508+0000 7fbe09000700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbde80b2bc0 0x7fbde80b50d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbe04106a50 tx=0x7fbdf0003680 comp rx=0 tx=0).stop 2024-05-27T08:21:00.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.508+0000 7fbe09000700 1 -- 172.21.15.63:0/3109137016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041034b0 msgr2=0x7fbe04075440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:00.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.508+0000 7fbe09000700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041034b0 0x7fbe04075440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400d980 tx=0x7fbdf400dd40 comp rx=0 tx=0).stop 2024-05-27T08:21:00.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.509+0000 7fbe09000700 1 -- 172.21.15.63:0/3109137016 shutdown_connections 2024-05-27T08:21:00.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.509+0000 7fbe09000700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fbde80b2bc0 0x7fbde80b50d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.509+0000 7fbe09000700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe04108730 0x7fbe0406ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.509+0000 7fbe09000700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041034b0 0x7fbe04075440 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.509+0000 7fbe09000700 1 --2- 172.21.15.63:0/3109137016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe04107de0 0x7fbe0406e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:00.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.509+0000 7fbe09000700 1 -- 172.21.15.63:0/3109137016 >> 172.21.15.63:0/3109137016 conn(0x7fbe040fedb0 msgr2=0x7fbe041011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:00.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.510+0000 7fbe09000700 1 -- 172.21.15.63:0/3109137016 shutdown_connections 2024-05-27T08:21:00.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:00.510+0000 7fbe09000700 1 -- 172.21.15.63:0/3109137016 wait complete. 2024-05-27T08:21:01.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:01 smithi149 ceph-mon[35408]: pgmap v1511: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 419 KiB/s wr, 220 op/s 2024-05-27T08:21:01.464 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:01 smithi086 ceph-mon[34687]: pgmap v1511: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 419 KiB/s wr, 220 op/s 2024-05-27T08:21:01.468 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:01 smithi063 ceph-mon[31613]: pgmap v1511: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 419 KiB/s wr, 220 op/s 2024-05-27T08:21:02.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:01 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 3. 2024-05-27T08:21:02.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:01 smithi086 systemd[1]: Stopped Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:02.209 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:01 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Consumed 8.895s CPU time. 2024-05-27T08:21:02.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:01 smithi086 systemd[1]: Starting Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:02.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:02 smithi149 ceph-mon[35408]: scrub queued for path: / 2024-05-27T08:21:02.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:21:02.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:02 smithi149 ceph-mon[35408]: scrub summary: active paths [/] 2024-05-27T08:21:02.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:02 smithi149 ceph-mon[35408]: scrub summary: idle+waiting paths [/] 2024-05-27T08:21:02.214 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc12e800700 time 2024-05-27T08:21:01.797922+0000 2024-05-27T08:21:02.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:02.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:02 smithi149 ceph-mon[35408]: MDS daemon mds.k is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:02.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:02 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:02.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:02 smithi063 ceph-mon[31613]: scrub queued for path: / 2024-05-27T08:21:02.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:21:02.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:02 smithi063 ceph-mon[31613]: scrub summary: active paths [/] 2024-05-27T08:21:02.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:02 smithi063 ceph-mon[31613]: scrub summary: idle+waiting paths [/] 2024-05-27T08:21:02.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:02 smithi063 ceph-mon[31613]: MDS daemon mds.k is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:02.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:02 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:02.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:02 smithi086 ceph-mon[34687]: scrub queued for path: / 2024-05-27T08:21:02.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:21:02.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:02 smithi086 ceph-mon[34687]: scrub summary: active paths [/] 2024-05-27T08:21:02.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:02 smithi086 ceph-mon[34687]: scrub summary: idle+waiting paths [/] 2024-05-27T08:21:02.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:02 smithi086 ceph-mon[34687]: MDS daemon mds.k is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:02.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:02 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:03.214 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:03.214 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc1442e9ba6] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc1442e9dd1] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559243c5223d] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 4: (SnapRealm::build_snap_set() const+0x323) [0x559243c3d971] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 5: (SnapRealm::check_cache() const+0x13e) [0x559243c3bfa0] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559243c3c651] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559243bd1347] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5592438cd87e] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5592438d2f2f] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5592438dd969] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 11: (MDLog::_replay_thread()+0x1f38) [0x559243c7648c] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 12: (MDLog::ReplayThread::entry()+0x11) [0x559243c7f9cb] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 13: (Thread::entry_wrapper()+0x43) [0x7fc1442b872d] 2024-05-27T08:21:03.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc1442b8749] 2024-05-27T08:21:03.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc142e5c1ca] 2024-05-27T08:21:03.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 16: clone() 2024-05-27T08:21:03.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:02.922+0000 7fc12e800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc12e800700 time 2024-05-27T08:21:01.797922+0000 2024-05-27T08:21:03.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:03.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:03.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc1442e9ba6] 2024-05-27T08:21:03.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc1442e9dd1] 2024-05-27T08:21:03.217 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559243c5223d] 2024-05-27T08:21:03.217 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 4: (SnapRealm::build_snap_set() const+0x323) [0x559243c3d971] 2024-05-27T08:21:03.217 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 5: (SnapRealm::check_cache() const+0x13e) [0x559243c3bfa0] 2024-05-27T08:21:03.217 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559243c3c651] 2024-05-27T08:21:03.217 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559243bd1347] 2024-05-27T08:21:03.217 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5592438cd87e] 2024-05-27T08:21:03.217 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5592438d2f2f] 2024-05-27T08:21:03.217 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5592438dd969] 2024-05-27T08:21:03.217 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 11: (MDLog::_replay_thread()+0x1f38) [0x559243c7648c] 2024-05-27T08:21:03.217 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 12: (MDLog::ReplayThread::entry()+0x11) [0x559243c7f9cb] 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 13: (Thread::entry_wrapper()+0x43) [0x7fc1442b872d] 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc1442b8749] 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc142e5c1ca] 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 16: clone() 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: *** Caught signal (Aborted) ** 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: in thread 7fc12e800700 thread_name:md_log_replay 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc142e66d20] 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2: gsignal() 2024-05-27T08:21:03.218 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 3: abort() 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc1442e9ced] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc1442e9dd1] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559243c5223d] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 7: (SnapRealm::build_snap_set() const+0x323) [0x559243c3d971] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 8: (SnapRealm::check_cache() const+0x13e) [0x559243c3bfa0] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559243c3c651] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559243bd1347] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5592438cd87e] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5592438d2f2f] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5592438dd969] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 14: (MDLog::_replay_thread()+0x1f38) [0x559243c7648c] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 15: (MDLog::ReplayThread::entry()+0x11) [0x559243c7f9cb] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 16: (Thread::entry_wrapper()+0x43) [0x7fc1442b872d] 2024-05-27T08:21:03.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc1442b8749] 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc142e5c1ca] 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 19: clone() 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:02.926+0000 7fc12e800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: in thread 7fc12e800700 thread_name:md_log_replay 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc142e66d20] 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2: gsignal() 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 3: abort() 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc1442e9ced] 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc1442e9dd1] 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559243c5223d] 2024-05-27T08:21:03.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 7: (SnapRealm::build_snap_set() const+0x323) [0x559243c3d971] 2024-05-27T08:21:03.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 8: (SnapRealm::check_cache() const+0x13e) [0x559243c3bfa0] 2024-05-27T08:21:03.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559243c3c651] 2024-05-27T08:21:03.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559243bd1347] 2024-05-27T08:21:03.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5592438cd87e] 2024-05-27T08:21:03.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5592438d2f2f] 2024-05-27T08:21:03.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5592438dd969] 2024-05-27T08:21:03.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 14: (MDLog::_replay_thread()+0x1f38) [0x559243c7648c] 2024-05-27T08:21:03.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 15: (MDLog::ReplayThread::entry()+0x11) [0x559243c7f9cb] 2024-05-27T08:21:03.222 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 16: (Thread::entry_wrapper()+0x43) [0x7fc1442b872d] 2024-05-27T08:21:03.222 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc1442b8749] 2024-05-27T08:21:03.222 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc142e5c1ca] 2024-05-27T08:21:03.222 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 19: clone() 2024-05-27T08:21:03.222 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: -1> 2024-05-27T08:21:02.922+0000 7fc12e800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc12e800700 time 2024-05-27T08:21:01.797922+0000 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc1442e9ba6] 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc1442e9dd1] 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559243c5223d] 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 4: (SnapRealm::build_snap_set() const+0x323) [0x559243c3d971] 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 5: (SnapRealm::check_cache() const+0x13e) [0x559243c3bfa0] 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559243c3c651] 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559243bd1347] 2024-05-27T08:21:03.223 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5592438cd87e] 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5592438d2f2f] 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5592438dd969] 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 11: (MDLog::_replay_thread()+0x1f38) [0x559243c7648c] 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 12: (MDLog::ReplayThread::entry()+0x11) [0x559243c7f9cb] 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 13: (Thread::entry_wrapper()+0x43) [0x7fc1442b872d] 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc1442b8749] 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc142e5c1ca] 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 16: clone() 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 0> 2024-05-27T08:21:02.926+0000 7fc12e800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: in thread 7fc12e800700 thread_name:md_log_replay 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:03.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc142e66d20] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2: gsignal() 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 3: abort() 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc1442e9ced] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc1442e9dd1] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559243c5223d] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 7: (SnapRealm::build_snap_set() const+0x323) [0x559243c3d971] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 8: (SnapRealm::check_cache() const+0x13e) [0x559243c3bfa0] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559243c3c651] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559243bd1347] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5592438cd87e] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5592438d2f2f] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5592438dd969] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 14: (MDLog::_replay_thread()+0x1f38) [0x559243c7648c] 2024-05-27T08:21:03.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 15: (MDLog::ReplayThread::entry()+0x11) [0x559243c7f9cb] 2024-05-27T08:21:03.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 16: (Thread::entry_wrapper()+0x43) [0x7fc1442b872d] 2024-05-27T08:21:03.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc1442b8749] 2024-05-27T08:21:03.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc142e5c1ca] 2024-05-27T08:21:03.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 19: clone() 2024-05-27T08:21:03.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:03.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: -2> 2024-05-27T08:21:02.922+0000 7fc12e800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc12e800700 time 2024-05-27T08:21:01.797922+0000 2024-05-27T08:21:03.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:03.227 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.227 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:03.227 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc1442e9ba6] 2024-05-27T08:21:03.227 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc1442e9dd1] 2024-05-27T08:21:03.227 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559243c5223d] 2024-05-27T08:21:03.227 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 4: (SnapRealm::build_snap_set() const+0x323) [0x559243c3d971] 2024-05-27T08:21:03.227 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 5: (SnapRealm::check_cache() const+0x13e) [0x559243c3bfa0] 2024-05-27T08:21:03.227 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559243c3c651] 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559243bd1347] 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5592438cd87e] 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5592438d2f2f] 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5592438dd969] 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 11: (MDLog::_replay_thread()+0x1f38) [0x559243c7648c] 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 12: (MDLog::ReplayThread::entry()+0x11) [0x559243c7f9cb] 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 13: (Thread::entry_wrapper()+0x43) [0x7fc1442b872d] 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc1442b8749] 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc142e5c1ca] 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 16: clone() 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: -1> 2024-05-27T08:21:02.926+0000 7fc12e800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:03.228 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: in thread 7fc12e800700 thread_name:md_log_replay 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc142e66d20] 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2: gsignal() 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 3: abort() 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc1442e9ced] 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc1442e9dd1] 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559243c5223d] 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 7: (SnapRealm::build_snap_set() const+0x323) [0x559243c3d971] 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 8: (SnapRealm::check_cache() const+0x13e) [0x559243c3bfa0] 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559243c3c651] 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559243bd1347] 2024-05-27T08:21:03.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5592438cd87e] 2024-05-27T08:21:03.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5592438d2f2f] 2024-05-27T08:21:03.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5592438dd969] 2024-05-27T08:21:03.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 14: (MDLog::_replay_thread()+0x1f38) [0x559243c7648c] 2024-05-27T08:21:03.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 15: (MDLog::ReplayThread::entry()+0x11) [0x559243c7f9cb] 2024-05-27T08:21:03.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 16: (Thread::entry_wrapper()+0x43) [0x7fc1442b872d] 2024-05-27T08:21:03.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc1442b8749] 2024-05-27T08:21:03.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc142e5c1ca] 2024-05-27T08:21:03.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 19: clone() 2024-05-27T08:21:03.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:03.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:02 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[70534]: 2024-05-27T08:21:03.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:03 smithi063 ceph-mon[31613]: scrub summary: idle 2024-05-27T08:21:03.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:03 smithi063 ceph-mon[31613]: pgmap v1512: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 367 KiB/s wr, 228 op/s 2024-05-27T08:21:03.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:03 smithi086 ceph-mon[34687]: scrub summary: idle 2024-05-27T08:21:03.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:03 smithi086 ceph-mon[34687]: pgmap v1512: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 367 KiB/s wr, 228 op/s 2024-05-27T08:21:03.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:03 smithi149 ceph-mon[35408]: scrub summary: idle 2024-05-27T08:21:03.716 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:03 smithi149 ceph-mon[35408]: pgmap v1512: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 367 KiB/s wr, 228 op/s 2024-05-27T08:21:03.953 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:21:04.524 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:21:04.715 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:04 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-05-27T08:21:04.715 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:04 smithi149 systemd[1]: Stopped Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:04.715 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:04 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 14.509s CPU time. 2024-05-27T08:21:04.716 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:04 smithi149 systemd[1]: Starting Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:04.716 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:04 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 1. 2024-05-27T08:21:04.716 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:04 smithi149 systemd[1]: Stopped Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:05.078 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:04 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 14.055s CPU time. 2024-05-27T08:21:05.078 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:04 smithi149 systemd[1]: Starting Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:05.329 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:05 smithi149 ceph-mon[35408]: pgmap v1513: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 242 KiB/s wr, 277 op/s 2024-05-27T08:21:05.337 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:05 smithi063 ceph-mon[31613]: pgmap v1513: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 242 KiB/s wr, 277 op/s 2024-05-27T08:21:05.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:05 smithi086 ceph-mon[34687]: pgmap v1513: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 242 KiB/s wr, 277 op/s 2024-05-27T08:21:05.713 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:05 smithi149 podman[80824]: 2024-05-27 08:21:05.328471461 +0000 UTC m=+1.643713673 container died 426d5dd01d27061066b5c1742dafd99354fc93119c58a2c5f821900c7e725d17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521) 2024-05-27T08:21:06.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:06 smithi063 ceph-mon[31613]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:06.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:06 smithi063 ceph-mon[31613]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:06.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:06 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 3 up:standby 2024-05-27T08:21:06.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:06 smithi086 ceph-mon[34687]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:06.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:06 smithi086 ceph-mon[34687]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:06.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:06 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 3 up:standby 2024-05-27T08:21:06.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:06 smithi149 ceph-mon[35408]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:06.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:06 smithi149 ceph-mon[35408]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:06.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:06 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 3 up:standby 2024-05-27T08:21:06.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f7050800700 time 2024-05-27T08:21:06.643745+0000 2024-05-27T08:21:06.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:06.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:06.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f7066282ba6] 2024-05-27T08:21:06.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f7066282dd1] 2024-05-27T08:21:06.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x557a8ab6223d] 2024-05-27T08:21:06.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 4: (SnapRealm::build_snap_set() const+0x323) [0x557a8ab4d971] 2024-05-27T08:21:06.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 5: (SnapRealm::check_cache() const+0x13e) [0x557a8ab4bfa0] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x557a8ab4c651] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x557a8aae1347] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x557a8a7dd87e] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x557a8a7e2e94] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x557a8a7ed969] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 11: (MDLog::_replay_thread()+0x1f38) [0x557a8ab8648c] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 12: (MDLog::ReplayThread::entry()+0x11) [0x557a8ab8f9cb] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 13: (Thread::entry_wrapper()+0x43) [0x7f706625172d] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 14: (Thread::_entry_func(void*)+0xd) [0x7f7066251749] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7064df51ca] 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 16: clone() 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: *** Caught signal (Aborted) ** 2024-05-27T08:21:06.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: in thread 7f7050800700 thread_name:md_log_replay 2024-05-27T08:21:06.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.645+0000 7f7050800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f7050800700 time 2024-05-27T08:21:06.643745+0000 2024-05-27T08:21:06.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:06.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:06.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f7066282ba6] 2024-05-27T08:21:06.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f7066282dd1] 2024-05-27T08:21:06.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x557a8ab6223d] 2024-05-27T08:21:06.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 4: (SnapRealm::build_snap_set() const+0x323) [0x557a8ab4d971] 2024-05-27T08:21:06.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 5: (SnapRealm::check_cache() const+0x13e) [0x557a8ab4bfa0] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x557a8ab4c651] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x557a8aae1347] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x557a8a7dd87e] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x557a8a7e2e94] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x557a8a7ed969] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 11: (MDLog::_replay_thread()+0x1f38) [0x557a8ab8648c] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 12: (MDLog::ReplayThread::entry()+0x11) [0x557a8ab8f9cb] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 13: (Thread::entry_wrapper()+0x43) [0x7f706625172d] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 14: (Thread::_entry_func(void*)+0xd) [0x7f7066251749] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7064df51ca] 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 16: clone() 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:06.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7064dffd20] 2024-05-27T08:21:06.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2: gsignal() 2024-05-27T08:21:06.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 3: abort() 2024-05-27T08:21:06.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f7066282ced] 2024-05-27T08:21:06.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f7066282dd1] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x557a8ab6223d] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 7: (SnapRealm::build_snap_set() const+0x323) [0x557a8ab4d971] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 8: (SnapRealm::check_cache() const+0x13e) [0x557a8ab4bfa0] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x557a8ab4c651] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x557a8aae1347] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x557a8a7dd87e] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x557a8a7e2e94] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x557a8a7ed969] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 14: (MDLog::_replay_thread()+0x1f38) [0x557a8ab8648c] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 15: (MDLog::ReplayThread::entry()+0x11) [0x557a8ab8f9cb] 2024-05-27T08:21:06.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 16: (Thread::entry_wrapper()+0x43) [0x7f706625172d] 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 17: (Thread::_entry_func(void*)+0xd) [0x7f7066251749] 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f7064df51ca] 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 19: clone() 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.649+0000 7f7050800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: in thread 7f7050800700 thread_name:md_log_replay 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7064dffd20] 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2: gsignal() 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 3: abort() 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f7066282ced] 2024-05-27T08:21:06.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f7066282dd1] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x557a8ab6223d] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 7: (SnapRealm::build_snap_set() const+0x323) [0x557a8ab4d971] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 8: (SnapRealm::check_cache() const+0x13e) [0x557a8ab4bfa0] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x557a8ab4c651] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x557a8aae1347] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x557a8a7dd87e] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x557a8a7e2e94] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x557a8a7ed969] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 14: (MDLog::_replay_thread()+0x1f38) [0x557a8ab8648c] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 15: (MDLog::ReplayThread::entry()+0x11) [0x557a8ab8f9cb] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 16: (Thread::entry_wrapper()+0x43) [0x7f706625172d] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 17: (Thread::_entry_func(void*)+0xd) [0x7f7066251749] 2024-05-27T08:21:06.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f7064df51ca] 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 19: clone() 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: -1> 2024-05-27T08:21:06.645+0000 7f7050800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f7050800700 time 2024-05-27T08:21:06.643745+0000 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f7066282ba6] 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f7066282dd1] 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x557a8ab6223d] 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 4: (SnapRealm::build_snap_set() const+0x323) [0x557a8ab4d971] 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 5: (SnapRealm::check_cache() const+0x13e) [0x557a8ab4bfa0] 2024-05-27T08:21:06.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x557a8ab4c651] 2024-05-27T08:21:06.975 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x557a8aae1347] 2024-05-27T08:21:06.975 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x557a8a7dd87e] 2024-05-27T08:21:06.975 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x557a8a7e2e94] 2024-05-27T08:21:06.975 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x557a8a7ed969] 2024-05-27T08:21:06.975 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 11: (MDLog::_replay_thread()+0x1f38) [0x557a8ab8648c] 2024-05-27T08:21:06.975 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 12: (MDLog::ReplayThread::entry()+0x11) [0x557a8ab8f9cb] 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 13: (Thread::entry_wrapper()+0x43) [0x7f706625172d] 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 14: (Thread::_entry_func(void*)+0xd) [0x7f7066251749] 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7064df51ca] 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 16: clone() 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 0> 2024-05-27T08:21:06.649+0000 7f7050800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: in thread 7f7050800700 thread_name:md_log_replay 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7064dffd20] 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2: gsignal() 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 3: abort() 2024-05-27T08:21:06.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f7066282ced] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f7066282dd1] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x557a8ab6223d] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 7: (SnapRealm::build_snap_set() const+0x323) [0x557a8ab4d971] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 8: (SnapRealm::check_cache() const+0x13e) [0x557a8ab4bfa0] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x557a8ab4c651] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x557a8aae1347] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x557a8a7dd87e] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x557a8a7e2e94] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x557a8a7ed969] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 14: (MDLog::_replay_thread()+0x1f38) [0x557a8ab8648c] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 15: (MDLog::ReplayThread::entry()+0x11) [0x557a8ab8f9cb] 2024-05-27T08:21:06.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 16: (Thread::entry_wrapper()+0x43) [0x7f706625172d] 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 17: (Thread::_entry_func(void*)+0xd) [0x7f7066251749] 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f7064df51ca] 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 19: clone() 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: -1> 2024-05-27T08:21:06.645+0000 7f7050800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f7050800700 time 2024-05-27T08:21:06.643745+0000 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f7066282ba6] 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f7066282dd1] 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x557a8ab6223d] 2024-05-27T08:21:06.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 4: (SnapRealm::build_snap_set() const+0x323) [0x557a8ab4d971] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 5: (SnapRealm::check_cache() const+0x13e) [0x557a8ab4bfa0] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x557a8ab4c651] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x557a8aae1347] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x557a8a7dd87e] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x557a8a7e2e94] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x557a8a7ed969] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 11: (MDLog::_replay_thread()+0x1f38) [0x557a8ab8648c] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 12: (MDLog::ReplayThread::entry()+0x11) [0x557a8ab8f9cb] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 13: (Thread::entry_wrapper()+0x43) [0x7f706625172d] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 14: (Thread::_entry_func(void*)+0xd) [0x7f7066251749] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7064df51ca] 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 16: clone() 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 0> 2024-05-27T08:21:06.649+0000 7f7050800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: in thread 7f7050800700 thread_name:md_log_replay 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7064dffd20] 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2: gsignal() 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 3: abort() 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f7066282ced] 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f7066282dd1] 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x557a8ab6223d] 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 7: (SnapRealm::build_snap_set() const+0x323) [0x557a8ab4d971] 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 8: (SnapRealm::check_cache() const+0x13e) [0x557a8ab4bfa0] 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x557a8ab4c651] 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x557a8aae1347] 2024-05-27T08:21:06.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x557a8a7dd87e] 2024-05-27T08:21:06.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x557a8a7e2e94] 2024-05-27T08:21:06.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x557a8a7ed969] 2024-05-27T08:21:06.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 14: (MDLog::_replay_thread()+0x1f38) [0x557a8ab8648c] 2024-05-27T08:21:06.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 15: (MDLog::ReplayThread::entry()+0x11) [0x557a8ab8f9cb] 2024-05-27T08:21:06.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 16: (Thread::entry_wrapper()+0x43) [0x7f706625172d] 2024-05-27T08:21:06.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 17: (Thread::_entry_func(void*)+0xd) [0x7f7066251749] 2024-05-27T08:21:06.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f7064df51ca] 2024-05-27T08:21:06.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 19: clone() 2024-05-27T08:21:06.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:06.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:06 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[65329]: 2024-05-27T08:21:07.653 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:07 smithi086 podman[80733]: 2024-05-27 08:21:07.203903549 +0000 UTC m=+0.067017396 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:07.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:07 smithi086 ceph-mon[34687]: pgmap v1514: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 143 KiB/s wr, 268 op/s 2024-05-27T08:21:07.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:07 smithi149 ceph-mon[35408]: pgmap v1514: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 143 KiB/s wr, 268 op/s 2024-05-27T08:21:07.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:07 smithi063 ceph-mon[31613]: pgmap v1514: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 143 KiB/s wr, 268 op/s 2024-05-27T08:21:09.092 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:08 smithi086 podman[80733]: 2024-05-27 08:21:08.816229237 +0000 UTC m=+1.679343100 container create f90e5368958aeae14ab585893ba70f07b0a04d74a6d142f0d23aa143bd0b70bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:21:09.461 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:09 smithi086 podman[80733]: 2024-05-27 08:21:09.189743768 +0000 UTC m=+2.052857618 container init f90e5368958aeae14ab585893ba70f07b0a04d74a6d142f0d23aa143bd0b70bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521) 2024-05-27T08:21:09.461 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:09 smithi086 podman[80733]: 2024-05-27 08:21:09.215935038 +0000 UTC m=+2.079048873 container start f90e5368958aeae14ab585893ba70f07b0a04d74a6d142f0d23aa143bd0b70bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:21:09.461 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:09 smithi086 bash[80733]: f90e5368958aeae14ab585893ba70f07b0a04d74a6d142f0d23aa143bd0b70bb 2024-05-27T08:21:09.461 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:09 smithi086 systemd[1]: Started Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:09.461 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:21:09 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: starting mds.k at 2024-05-27T08:21:10.092 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:09 smithi149 podman[80883]: 2024-05-27 08:21:09.783125823 +0000 UTC m=+2.797422701 container died b6ae53a70b5f0a8f915b2526ee86fdd5de62bb618851a39809738f7e2da0dc75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-27T08:21:10.344 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:10 smithi149 ceph-mon[35408]: pgmap v1515: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 141 KiB/s wr, 334 op/s 2024-05-27T08:21:10.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:10 smithi086 ceph-mon[34687]: pgmap v1515: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 141 KiB/s wr, 334 op/s 2024-05-27T08:21:10.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:10 smithi063 ceph-mon[31613]: pgmap v1515: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 141 KiB/s wr, 334 op/s 2024-05-27T08:21:11.351 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:11 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.86:6832/1512010809,v1:172.21.15.86:6833/1512010809] up:boot 2024-05-27T08:21:11.351 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:11 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:21:11.352 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:11 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:21:11.352 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:10 smithi149 podman[80824]: 2024-05-27 08:21:10.964250273 +0000 UTC m=+7.279492525 container remove 426d5dd01d27061066b5c1742dafd99354fc93119c58a2c5f821900c7e725d17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521) 2024-05-27T08:21:11.352 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:10 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:11.352 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:11 smithi149 podman[80883]: 2024-05-27 08:21:11.140319304 +0000 UTC m=+4.154616158 container remove b6ae53a70b5f0a8f915b2526ee86fdd5de62bb618851a39809738f7e2da0dc75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-05-27T08:21:11.352 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:11 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:11.393 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:11 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.86:6832/1512010809,v1:172.21.15.86:6833/1512010809] up:boot 2024-05-27T08:21:11.393 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:11 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:21:11.393 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:11 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:21:11.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:11 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.86:6832/1512010809,v1:172.21.15.86:6833/1512010809] up:boot 2024-05-27T08:21:11.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:11 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:21:11.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:11 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:21:11.691 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:11 smithi149 podman[80959]: 2024-05-27 08:21:11.255398501 +0000 UTC m=+0.034742276 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:11.692 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:11 smithi149 podman[80959]: 2024-05-27 08:21:11.399791906 +0000 UTC m=+0.179135714 container create 8bce2596cb75ef953e221340de82e79aea70ef82fcfc8624b95553692bf08b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-05-27T08:21:11.963 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:11 smithi149 podman[80994]: 2024-05-27 08:21:11.59588975 +0000 UTC m=+0.038815126 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:12.139 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f04c6e00700 time 2024-05-27T08:21:11.713825+0000 2024-05-27T08:21:12.139 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:12.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:12 smithi086 ceph-mon[34687]: pgmap v1516: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 53 KiB/s wr, 197 op/s 2024-05-27T08:21:12.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:12 smithi149 ceph-mon[35408]: pgmap v1516: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 53 KiB/s wr, 197 op/s 2024-05-27T08:21:12.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:12 smithi149 podman[80959]: 2024-05-27 08:21:12.32242496 +0000 UTC m=+1.101768739 container init 8bce2596cb75ef953e221340de82e79aea70ef82fcfc8624b95553692bf08b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD) 2024-05-27T08:21:12.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:12 smithi149 podman[80959]: 2024-05-27 08:21:12.338509136 +0000 UTC m=+1.117852910 container start 8bce2596cb75ef953e221340de82e79aea70ef82fcfc8624b95553692bf08b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-05-27T08:21:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:12 smithi063 ceph-mon[31613]: pgmap v1516: 129 pgs: 129 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 53 KiB/s wr, 197 op/s 2024-05-27T08:21:12.963 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:12 smithi149 bash[80959]: 8bce2596cb75ef953e221340de82e79aea70ef82fcfc8624b95553692bf08b24 2024-05-27T08:21:12.964 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:12 smithi149 systemd[1]: Started Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:13.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:13 smithi086 ceph-mon[34687]: osdmap e179: 12 total, 12 up, 12 in 2024-05-27T08:21:13.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:13 smithi149 ceph-mon[35408]: osdmap e179: 12 total, 12 up, 12 in 2024-05-27T08:21:13.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:13 smithi063 ceph-mon[31613]: osdmap e179: 12 total, 12 up, 12 in 2024-05-27T08:21:14.169 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:13 smithi149 podman[80994]: 2024-05-27 08:21:13.830275978 +0000 UTC m=+2.273201334 container create c327b144344a1bbfd17b0f94ef090c3bca526a9e144f0f52b38e037ae0124aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-27T08:21:14.439 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:14 smithi149 ceph-mon[35408]: pgmap v1518: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 63 KiB/s wr, 213 op/s 2024-05-27T08:21:14.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:14 smithi086 ceph-mon[34687]: pgmap v1518: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 63 KiB/s wr, 213 op/s 2024-05-27T08:21:14.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-mon[31613]: pgmap v1518: 129 pgs: 6 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 109 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 63 KiB/s wr, 213 op/s 2024-05-27T08:21:14.968 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:14.968 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f04dc91eba6] 2024-05-27T08:21:14.968 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04dc91edd1] 2024-05-27T08:21:14.968 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556a8743923d] 2024-05-27T08:21:14.968 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556a87424971] 2024-05-27T08:21:14.968 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 5: (SnapRealm::check_cache() const+0x13e) [0x556a87422fa0] 2024-05-27T08:21:14.968 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556a87423651] 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556a873b8347] 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556a870b487e] 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556a870b9e94] 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556a870c4969] 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 11: (MDLog::_replay_thread()+0x1f38) [0x556a8745d48c] 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556a874669cb] 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 13: (Thread::entry_wrapper()+0x43) [0x7f04dc8ed72d] 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 14: (Thread::_entry_func(void*)+0xd) [0x7f04dc8ed749] 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f04db4911ca] 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 16: clone() 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.630+0000 7f04c6e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f04c6e00700 time 2024-05-27T08:21:11.713825+0000 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.969 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:14.970 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f04dc91eba6] 2024-05-27T08:21:14.970 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04dc91edd1] 2024-05-27T08:21:14.970 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556a8743923d] 2024-05-27T08:21:14.970 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556a87424971] 2024-05-27T08:21:14.970 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 5: (SnapRealm::check_cache() const+0x13e) [0x556a87422fa0] 2024-05-27T08:21:14.970 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556a87423651] 2024-05-27T08:21:14.970 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556a873b8347] 2024-05-27T08:21:14.970 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556a870b487e] 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556a870b9e94] 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556a870c4969] 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 11: (MDLog::_replay_thread()+0x1f38) [0x556a8745d48c] 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556a874669cb] 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 13: (Thread::entry_wrapper()+0x43) [0x7f04dc8ed72d] 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 14: (Thread::_entry_func(void*)+0xd) [0x7f04dc8ed749] 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f04db4911ca] 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 16: clone() 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: *** Caught signal (Aborted) ** 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: in thread 7f04c6e00700 thread_name:md_log_replay 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:14.971 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f04db49bd20] 2024-05-27T08:21:14.972 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2: gsignal() 2024-05-27T08:21:14.972 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 3: abort() 2024-05-27T08:21:14.972 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f04dc91eced] 2024-05-27T08:21:14.972 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04dc91edd1] 2024-05-27T08:21:14.972 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556a8743923d] 2024-05-27T08:21:14.972 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556a87424971] 2024-05-27T08:21:14.972 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 8: (SnapRealm::check_cache() const+0x13e) [0x556a87422fa0] 2024-05-27T08:21:14.972 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556a87423651] 2024-05-27T08:21:14.972 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556a873b8347] 2024-05-27T08:21:14.973 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556a870b487e] 2024-05-27T08:21:14.973 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556a870b9e94] 2024-05-27T08:21:14.973 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556a870c4969] 2024-05-27T08:21:14.973 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 14: (MDLog::_replay_thread()+0x1f38) [0x556a8745d48c] 2024-05-27T08:21:14.973 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556a874669cb] 2024-05-27T08:21:14.973 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 16: (Thread::entry_wrapper()+0x43) [0x7f04dc8ed72d] 2024-05-27T08:21:14.973 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 17: (Thread::_entry_func(void*)+0xd) [0x7f04dc8ed749] 2024-05-27T08:21:14.973 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f04db4911ca] 2024-05-27T08:21:14.973 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 19: clone() 2024-05-27T08:21:14.973 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.635+0000 7f04c6e00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: in thread 7f04c6e00700 thread_name:md_log_replay 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f04db49bd20] 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2: gsignal() 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 3: abort() 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f04dc91eced] 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04dc91edd1] 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556a8743923d] 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556a87424971] 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 8: (SnapRealm::check_cache() const+0x13e) [0x556a87422fa0] 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556a87423651] 2024-05-27T08:21:14.974 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556a873b8347] 2024-05-27T08:21:14.975 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556a870b487e] 2024-05-27T08:21:14.975 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556a870b9e94] 2024-05-27T08:21:14.975 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556a870c4969] 2024-05-27T08:21:14.975 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 14: (MDLog::_replay_thread()+0x1f38) [0x556a8745d48c] 2024-05-27T08:21:14.975 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556a874669cb] 2024-05-27T08:21:14.975 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 16: (Thread::entry_wrapper()+0x43) [0x7f04dc8ed72d] 2024-05-27T08:21:14.975 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 17: (Thread::_entry_func(void*)+0xd) [0x7f04dc8ed749] 2024-05-27T08:21:14.975 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f04db4911ca] 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 19: clone() 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: -1> 2024-05-27T08:21:14.630+0000 7f04c6e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f04c6e00700 time 2024-05-27T08:21:11.713825+0000 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f04dc91eba6] 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04dc91edd1] 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556a8743923d] 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556a87424971] 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 5: (SnapRealm::check_cache() const+0x13e) [0x556a87422fa0] 2024-05-27T08:21:14.976 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556a87423651] 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556a873b8347] 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556a870b487e] 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556a870b9e94] 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556a870c4969] 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 11: (MDLog::_replay_thread()+0x1f38) [0x556a8745d48c] 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556a874669cb] 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 13: (Thread::entry_wrapper()+0x43) [0x7f04dc8ed72d] 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 14: (Thread::_entry_func(void*)+0xd) [0x7f04dc8ed749] 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f04db4911ca] 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 16: clone() 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 0> 2024-05-27T08:21:14.635+0000 7f04c6e00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:14.977 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: in thread 7f04c6e00700 thread_name:md_log_replay 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f04db49bd20] 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2: gsignal() 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 3: abort() 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f04dc91eced] 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04dc91edd1] 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556a8743923d] 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556a87424971] 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 8: (SnapRealm::check_cache() const+0x13e) [0x556a87422fa0] 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556a87423651] 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556a873b8347] 2024-05-27T08:21:14.978 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556a870b487e] 2024-05-27T08:21:14.979 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556a870b9e94] 2024-05-27T08:21:14.979 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556a870c4969] 2024-05-27T08:21:14.979 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 14: (MDLog::_replay_thread()+0x1f38) [0x556a8745d48c] 2024-05-27T08:21:14.979 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556a874669cb] 2024-05-27T08:21:14.979 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 16: (Thread::entry_wrapper()+0x43) [0x7f04dc8ed72d] 2024-05-27T08:21:14.979 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 17: (Thread::_entry_func(void*)+0xd) [0x7f04dc8ed749] 2024-05-27T08:21:14.979 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f04db4911ca] 2024-05-27T08:21:14.979 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 19: clone() 2024-05-27T08:21:14.980 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:14.981 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.981 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: -1> 2024-05-27T08:21:14.630+0000 7f04c6e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f04c6e00700 time 2024-05-27T08:21:11.713825+0000 2024-05-27T08:21:14.981 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:14.981 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.981 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:14.981 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f04dc91eba6] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04dc91edd1] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556a8743923d] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556a87424971] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 5: (SnapRealm::check_cache() const+0x13e) [0x556a87422fa0] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556a87423651] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556a873b8347] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556a870b487e] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556a870b9e94] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556a870c4969] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 11: (MDLog::_replay_thread()+0x1f38) [0x556a8745d48c] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556a874669cb] 2024-05-27T08:21:14.982 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 13: (Thread::entry_wrapper()+0x43) [0x7f04dc8ed72d] 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 14: (Thread::_entry_func(void*)+0xd) [0x7f04dc8ed749] 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f04db4911ca] 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 16: clone() 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 0> 2024-05-27T08:21:14.635+0000 7f04c6e00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: in thread 7f04c6e00700 thread_name:md_log_replay 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f04db49bd20] 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2: gsignal() 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 3: abort() 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f04dc91eced] 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04dc91edd1] 2024-05-27T08:21:14.983 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556a8743923d] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556a87424971] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 8: (SnapRealm::check_cache() const+0x13e) [0x556a87422fa0] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556a87423651] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556a873b8347] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556a870b487e] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556a870b9e94] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556a870c4969] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 14: (MDLog::_replay_thread()+0x1f38) [0x556a8745d48c] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556a874669cb] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 16: (Thread::entry_wrapper()+0x43) [0x7f04dc8ed72d] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 17: (Thread::_entry_func(void*)+0xd) [0x7f04dc8ed749] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f04db4911ca] 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 19: clone() 2024-05-27T08:21:14.984 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:14.985 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[73792]: 2024-05-27T08:21:14.985 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:14 smithi063 podman[187758]: 2024-05-27 08:21:14.85524683 +0000 UTC m=+0.082663911 container died c0c4c4595f4fdc70e441211190d87de97c026b8dcb4c40ba731f7948c104e528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T08:21:15.214 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:14 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: starting mds.c at 2024-05-27T08:21:15.214 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:14 smithi149 podman[80994]: 2024-05-27 08:21:14.866761937 +0000 UTC m=+3.309687303 container init c327b144344a1bbfd17b0f94ef090c3bca526a9e144f0f52b38e037ae0124aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:21:15.214 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:14 smithi149 podman[80994]: 2024-05-27 08:21:14.8788206 +0000 UTC m=+3.321745960 container start c327b144344a1bbfd17b0f94ef090c3bca526a9e144f0f52b38e037ae0124aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-27T08:21:15.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:15 smithi149 ceph-mds[81060]: -- 172.21.15.149:0/1329520171 <== mon.2 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577a07bfa40 con 0x5577a0807180 2024-05-27T08:21:15.716 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f0c13a00700 time 2024-05-27T08:21:15.341320+0000 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f0c29448ba6] 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0c29448dd1] 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c6d539e23d] 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c6d5389971] 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c6d5387fa0] 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c6d5388651] 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c6d531d347] 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c6d501987e] 2024-05-27T08:21:15.717 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55c6d501ef2f] 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c6d5029969] 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c6d53c248c] 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c6d53cb9cb] 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 13: (Thread::entry_wrapper()+0x43) [0x7f0c2941772d] 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 14: (Thread::_entry_func(void*)+0xd) [0x7f0c29417749] 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0c27fbb1ca] 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 16: clone() 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: *** Caught signal (Aborted) ** 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: in thread 7f0c13a00700 thread_name:md_log_replay 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.399+0000 7f0c13a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f0c13a00700 time 2024-05-27T08:21:15.341320+0000 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:15.718 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f0c29448ba6] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0c29448dd1] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c6d539e23d] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c6d5389971] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c6d5387fa0] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c6d5388651] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c6d531d347] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c6d501987e] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55c6d501ef2f] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c6d5029969] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c6d53c248c] 2024-05-27T08:21:15.719 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c6d53cb9cb] 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 13: (Thread::entry_wrapper()+0x43) [0x7f0c2941772d] 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 14: (Thread::_entry_func(void*)+0xd) [0x7f0c29417749] 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0c27fbb1ca] 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 16: clone() 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0c27fc5d20] 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2: gsignal() 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 3: abort() 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f0c29448ced] 2024-05-27T08:21:15.720 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0c29448dd1] 2024-05-27T08:21:15.721 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:15 smithi149 podman[81014]: 2024-05-27 08:21:15.488752868 +0000 UTC m=+2.739785423 container died 8bce2596cb75ef953e221340de82e79aea70ef82fcfc8624b95553692bf08b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:21:15.721 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:15 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-05-27T08:21:15.721 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:15 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Consumed 3.094s CPU time. 2024-05-27T08:21:15.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: starting mds.i at 2024-05-27T08:21:15.721 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:15 smithi149 bash[80994]: c327b144344a1bbfd17b0f94ef090c3bca526a9e144f0f52b38e037ae0124aec 2024-05-27T08:21:15.722 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:15 smithi149 systemd[1]: Started Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:15.722 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c6d539e23d] 2024-05-27T08:21:15.722 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c6d5389971] 2024-05-27T08:21:15.722 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c6d5387fa0] 2024-05-27T08:21:15.722 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c6d5388651] 2024-05-27T08:21:15.722 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c6d531d347] 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c6d501987e] 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55c6d501ef2f] 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c6d5029969] 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c6d53c248c] 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c6d53cb9cb] 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 16: (Thread::entry_wrapper()+0x43) [0x7f0c2941772d] 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 17: (Thread::_entry_func(void*)+0xd) [0x7f0c29417749] 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0c27fbb1ca] 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 19: clone() 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.402+0000 7f0c13a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: in thread 7f0c13a00700 thread_name:md_log_replay 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.723 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0c27fc5d20] 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2: gsignal() 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 3: abort() 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f0c29448ced] 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0c29448dd1] 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c6d539e23d] 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c6d5389971] 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c6d5387fa0] 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c6d5388651] 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c6d531d347] 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c6d501987e] 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55c6d501ef2f] 2024-05-27T08:21:15.724 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c6d5029969] 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c6d53c248c] 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c6d53cb9cb] 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 16: (Thread::entry_wrapper()+0x43) [0x7f0c2941772d] 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 17: (Thread::_entry_func(void*)+0xd) [0x7f0c29417749] 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0c27fbb1ca] 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 19: clone() 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: -1> 2024-05-27T08:21:15.399+0000 7f0c13a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f0c13a00700 time 2024-05-27T08:21:15.341320+0000 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:15.725 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f0c29448ba6] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0c29448dd1] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c6d539e23d] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c6d5389971] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c6d5387fa0] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c6d5388651] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c6d531d347] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c6d501987e] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55c6d501ef2f] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c6d5029969] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c6d53c248c] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c6d53cb9cb] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 13: (Thread::entry_wrapper()+0x43) [0x7f0c2941772d] 2024-05-27T08:21:15.726 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 14: (Thread::_entry_func(void*)+0xd) [0x7f0c29417749] 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0c27fbb1ca] 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 16: clone() 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 0> 2024-05-27T08:21:15.402+0000 7f0c13a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: in thread 7f0c13a00700 thread_name:md_log_replay 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0c27fc5d20] 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2: gsignal() 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 3: abort() 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f0c29448ced] 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0c29448dd1] 2024-05-27T08:21:15.727 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c6d539e23d] 2024-05-27T08:21:15.728 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c6d5389971] 2024-05-27T08:21:15.728 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c6d5387fa0] 2024-05-27T08:21:15.728 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c6d5388651] 2024-05-27T08:21:15.728 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c6d531d347] 2024-05-27T08:21:15.728 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c6d501987e] 2024-05-27T08:21:15.728 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55c6d501ef2f] 2024-05-27T08:21:15.728 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c6d5029969] 2024-05-27T08:21:15.728 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c6d53c248c] 2024-05-27T08:21:15.728 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c6d53cb9cb] 2024-05-27T08:21:15.728 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 16: (Thread::entry_wrapper()+0x43) [0x7f0c2941772d] 2024-05-27T08:21:15.729 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 17: (Thread::_entry_func(void*)+0xd) [0x7f0c29417749] 2024-05-27T08:21:15.729 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0c27fbb1ca] 2024-05-27T08:21:15.729 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 19: clone() 2024-05-27T08:21:15.729 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:15.729 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.729 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: -1> 2024-05-27T08:21:15.399+0000 7f0c13a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f0c13a00700 time 2024-05-27T08:21:15.341320+0000 2024-05-27T08:21:15.729 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:15.729 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.729 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f0c29448ba6] 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0c29448dd1] 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c6d539e23d] 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c6d5389971] 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c6d5387fa0] 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c6d5388651] 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c6d531d347] 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c6d501987e] 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55c6d501ef2f] 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c6d5029969] 2024-05-27T08:21:15.730 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c6d53c248c] 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c6d53cb9cb] 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 13: (Thread::entry_wrapper()+0x43) [0x7f0c2941772d] 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 14: (Thread::_entry_func(void*)+0xd) [0x7f0c29417749] 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0c27fbb1ca] 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 16: clone() 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 0> 2024-05-27T08:21:15.402+0000 7f0c13a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: in thread 7f0c13a00700 thread_name:md_log_replay 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0c27fc5d20] 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2: gsignal() 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 3: abort() 2024-05-27T08:21:15.731 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f0c29448ced] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0c29448dd1] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c6d539e23d] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c6d5389971] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c6d5387fa0] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c6d5388651] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c6d531d347] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c6d501987e] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55c6d501ef2f] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c6d5029969] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c6d53c248c] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c6d53cb9cb] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 16: (Thread::entry_wrapper()+0x43) [0x7f0c2941772d] 2024-05-27T08:21:15.732 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 17: (Thread::_entry_func(void*)+0xd) [0x7f0c29417749] 2024-05-27T08:21:15.733 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0c27fbb1ca] 2024-05-27T08:21:15.733 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 19: clone() 2024-05-27T08:21:15.733 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:15.733 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[80996]: 2024-05-27T08:21:16.025 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:15 smithi063 podman[187758]: 2024-05-27 08:21:15.74825155 +0000 UTC m=+0.975668629 container remove c0c4c4595f4fdc70e441211190d87de97c026b8dcb4c40ba731f7948c104e528 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:21:16.026 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:15 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:16.183 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:15 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-05-27T08:21:16.183 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:15 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 2.835s CPU time. 2024-05-27T08:21:16.184 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:16 smithi149 ceph-mon[35408]: pgmap v1519: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.0 KiB/s wr, 150 op/s 2024-05-27T08:21:16.184 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:16 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6832/2740612632,v1:172.21.15.149:6833/2740612632] up:boot 2024-05-27T08:21:16.184 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:16 smithi149 ceph-mon[35408]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:16.184 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:16 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:16.185 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:16 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:16.185 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:16 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:21:16.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.280+0000 7fa6f3800700 1 -- 172.21.15.63:0/3096926899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ec076c00 msgr2=0x7fa6ec077010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:16.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.280+0000 7fa6f3800700 1 --2- 172.21.15.63:0/3096926899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ec076c00 0x7fa6ec077010 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009990 tx=0x7fa6e0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.281+0000 7fa6f3800700 1 -- 172.21.15.63:0/3096926899 shutdown_connections 2024-05-27T08:21:16.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.281+0000 7fa6f3800700 1 --2- 172.21.15.63:0/3096926899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6ec110bd0 0x7fa6ec112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.281+0000 7fa6f3800700 1 --2- 172.21.15.63:0/3096926899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ec076c00 0x7fa6ec077010 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.281+0000 7fa6f3800700 1 --2- 172.21.15.63:0/3096926899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6ec077550 0x7fa6ec110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.281+0000 7fa6f3800700 1 -- 172.21.15.63:0/3096926899 >> 172.21.15.63:0/3096926899 conn(0x7fa6ec0ffe70 msgr2=0x7fa6ec1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:16.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.282+0000 7fa6f3800700 1 -- 172.21.15.63:0/3096926899 shutdown_connections 2024-05-27T08:21:16.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.282+0000 7fa6f3800700 1 -- 172.21.15.63:0/3096926899 wait complete. 2024-05-27T08:21:16.285 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:16 smithi063 ceph-mon[31613]: pgmap v1519: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.0 KiB/s wr, 150 op/s 2024-05-27T08:21:16.285 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:16 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6832/2740612632,v1:172.21.15.149:6833/2740612632] up:boot 2024-05-27T08:21:16.285 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:16 smithi063 ceph-mon[31613]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:16.285 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:16 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:16.285 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:16 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:16.286 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:16 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:21:16.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.284+0000 7fa6f3800700 1 Processor -- start 2024-05-27T08:21:16.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.284+0000 7fa6f3800700 1 -- start start 2024-05-27T08:21:16.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.285+0000 7fa6f3800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ec076c00 0x7fa6ec10a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.285+0000 7fa6f3800700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6ec077550 0x7fa6ec10a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.285+0000 7fa6f3800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6ec110bd0 0x7fa6ec10fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.285+0000 7fa6f3800700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1166e0 con 0x7fa6ec077550 2024-05-27T08:21:16.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.285+0000 7fa6f3800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec116560 con 0x7fa6ec076c00 2024-05-27T08:21:16.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.285+0000 7fa6f3800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec116860 con 0x7fa6ec110bd0 2024-05-27T08:21:16.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.285+0000 7fa6ebe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6ec077550 0x7fa6ec10a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.285+0000 7fa6f1800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6ec110bd0 0x7fa6ec10fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.285+0000 7fa6f1800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6ec110bd0 0x7fa6ec10fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:35878/0 (socket says 172.21.15.63:35878) 2024-05-27T08:21:16.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.285+0000 7fa6f1800700 1 -- 172.21.15.63:0/2720088239 learned_addr learned my addr 172.21.15.63:0/2720088239 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:16.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.286+0000 7fa6f0e00700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ec076c00 0x7fa6ec10a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.286+0000 7fa6f1800700 1 -- 172.21.15.63:0/2720088239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ec076c00 msgr2=0x7fa6ec10a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:16.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.286+0000 7fa6f1800700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ec076c00 0x7fa6ec10a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.286+0000 7fa6f1800700 1 -- 172.21.15.63:0/2720088239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6ec077550 msgr2=0x7fa6ec10a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:16.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.286+0000 7fa6f1800700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6ec077550 0x7fa6ec10a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.286+0000 7fa6f1800700 1 -- 172.21.15.63:0/2720088239 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec110700 con 0x7fa6ec110bd0 2024-05-27T08:21:16.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.287+0000 7fa6f1800700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6ec110bd0 0x7fa6ec10fed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b9e0 tx=0x7fa6e400bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:16.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.289+0000 7fa6e9600700 1 -- 172.21.15.63:0/2720088239 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4019070 con 0x7fa6ec110bd0 2024-05-27T08:21:16.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.289+0000 7fa6e9600700 1 -- 172.21.15.63:0/2720088239 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6e400ce40 con 0x7fa6ec110bd0 2024-05-27T08:21:16.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.292+0000 7fa6e9600700 1 -- 172.21.15.63:0/2720088239 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4012340 con 0x7fa6ec110bd0 2024-05-27T08:21:16.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.292+0000 7fa6f3800700 1 -- 172.21.15.63:0/2720088239 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec1109f0 con 0x7fa6ec110bd0 2024-05-27T08:21:16.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.292+0000 7fa6f3800700 1 -- 172.21.15.63:0/2720088239 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec105660 con 0x7fa6ec110bd0 2024-05-27T08:21:16.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.302+0000 7fa6e9600700 1 -- 172.21.15.63:0/2720088239 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa6e400c9b0 con 0x7fa6ec110bd0 2024-05-27T08:21:16.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.304+0000 7fa7023ceb80 1 Processor -- start 2024-05-27T08:21:16.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.305+0000 7fa7023ceb80 1 -- start start 2024-05-27T08:21:16.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.305+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47f379f0 0x556f47fca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.305+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47e4be40 0x556f47fca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.305+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47fcae70 0x556f47fd7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.305+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556f47ef8e40 con 0x556f47fcae70 2024-05-27T08:21:16.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.305+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556f47ee89e0 con 0x556f47f379f0 2024-05-27T08:21:16.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.305+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556f47eeca20 con 0x556f47e4be40 2024-05-27T08:21:16.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.306+0000 7fa6ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47f379f0 0x556f47fca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.306+0000 7fa6ebe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47f379f0 0x556f47fca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:37544/0 (socket says 172.21.15.63:37544) 2024-05-27T08:21:16.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.306+0000 7fa6ebe00700 1 -- 172.21.15.63:0/246384057 learned_addr learned my addr 172.21.15.63:0/246384057 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:16.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.306+0000 7fa6f0e00700 1 --2- 172.21.15.63:0/246384057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47e4be40 0x556f47fca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.306+0000 7fa6ebe00700 1 --2- 172.21.15.63:0/246384057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47fcae70 0x556f47fd7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.307+0000 7fa6ebe00700 1 -- 172.21.15.63:0/246384057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47e4be40 msgr2=0x556f47fca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:16.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.307+0000 7fa6ebe00700 1 --2- 172.21.15.63:0/246384057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47e4be40 0x556f47fca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.307+0000 7fa6ebe00700 1 -- 172.21.15.63:0/246384057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47f379f0 msgr2=0x556f47fca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:16.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.307+0000 7fa6ebe00700 1 --2- 172.21.15.63:0/246384057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47f379f0 0x556f47fca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.307+0000 7fa6ebe00700 1 -- 172.21.15.63:0/246384057 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f47eb1420 con 0x556f47fcae70 2024-05-27T08:21:16.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.308+0000 7fa6ebe00700 1 --2- 172.21.15.63:0/246384057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47fcae70 0x556f47fd7110 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800cf30 tx=0x7fa6d8006bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17966f06cc79641 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:16.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.309+0000 7fa6de000700 1 -- 172.21.15.63:0/246384057 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d801b070 con 0x556f47fcae70 2024-05-27T08:21:16.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.309+0000 7fa6de000700 1 -- 172.21.15.63:0/246384057 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6d8013e40 con 0x556f47fcae70 2024-05-27T08:21:16.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.309+0000 7fa6de000700 1 -- 172.21.15.63:0/246384057 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800f040 con 0x556f47fcae70 2024-05-27T08:21:16.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.310+0000 7fa7023ceb80 1 -- 172.21.15.63:0/246384057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47fcae70 msgr2=0x556f47fd7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.311+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/246384057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47fcae70 0x556f47fd7110 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800cf30 tx=0x7fa6d8006bb0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.311+0000 7fa6e9600700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6d40b2980 0x7fa6d40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.311+0000 7fa7023ceb80 1 -- 172.21.15.63:0/246384057 shutdown_connections 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.311+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/246384057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47fcae70 0x556f47fd7110 unknown :-1 s=CLOSED pgs=3013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.311+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/246384057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47f379f0 0x556f47fca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.311+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/246384057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47e4be40 0x556f47fca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.311+0000 7fa7023ceb80 1 -- 172.21.15.63:0/246384057 >> 172.21.15.63:0/246384057 conn(0x556f47ee5390 msgr2=0x556f47fc64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.311+0000 7fa6e9600700 1 -- 172.21.15.63:0/2720088239 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 9571+0+0 (secure 0 0 0) 0x7fa6e40e1d70 con 0x7fa6ec110bd0 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.312+0000 7fa6ebe00700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6d40b2980 0x7fa6d40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.312+0000 7fa7023ceb80 1 -- 172.21.15.63:0/246384057 shutdown_connections 2024-05-27T08:21:16.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.313+0000 7fa7023ceb80 1 -- 172.21.15.63:0/246384057 wait complete. 2024-05-27T08:21:16.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.313+0000 7fa6ebe00700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6d40b2980 0x7fa6d40b4e90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c610 tx=0x7fa6d8008040 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:16.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.314+0000 7fa7023ceb80 1 -- start start 2024-05-27T08:21:16.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.314+0000 7fa7023ceb80 1 Processor -- start 2024-05-27T08:21:16.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.315+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fd4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.315+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fd4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.315+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.315+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556f47deefa0 con 0x556f47e4be40 2024-05-27T08:21:16.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.315+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556f47e89760 con 0x556f47fcae70 2024-05-27T08:21:16.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.315+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556f47fd7b80 con 0x556f47f379f0 2024-05-27T08:21:16.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.316+0000 7fa7023ceb80 10 client.0 _ll_register_callbacks cb 0x556f47e92c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:21:16.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.316+0000 7fa6f1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.316+0000 7fa6f1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.63:37554/0 (socket says 172.21.15.63:37554) 2024-05-27T08:21:16.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.316+0000 7fa6f1800700 1 -- 172.21.15.63:0/514478032 learned_addr learned my addr 172.21.15.63:0/514478032 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:16.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.317+0000 7fa6f1800700 1 -- 172.21.15.63:0/514478032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 msgr2=0x556f47fd4c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:21:16.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.317+0000 7fa6f1800700 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fd4c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.317+0000 7fa6f1800700 1 -- 172.21.15.63:0/514478032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 msgr2=0x556f47fd4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:16.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.317+0000 7fa6f1800700 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fd4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.317+0000 7fa6f1800700 1 -- 172.21.15.63:0/514478032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f47e388d0 con 0x556f47fcae70 2024-05-27T08:21:16.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.320+0000 7fa6f1800700 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd51d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40a8a40 tx=0x7fa6e40a9280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ebaf449ad413d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:16.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.320+0000 7fa6d2a00700 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-05-27T08:21:16.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.323+0000 7fa6d2a00700 1 -- 172.21.15.63:0/514478032 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40a8d40 con 0x556f47fcae70 2024-05-27T08:21:16.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.323+0000 7fa6d2a00700 1 -- 172.21.15.63:0/514478032 <== mon.2 v2:172.21.15.86:3300/0 2 ==== osd_map(179..179 src has 1..179) ==== 9571+0+0 (secure 0 0 0) 0x7fa6e40c33c0 con 0x556f47fcae70 2024-05-27T08:21:16.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.323+0000 7fa7023ceb80 1 -- 172.21.15.63:0/514478032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f47efad90 con 0x556f47fcae70 2024-05-27T08:21:16.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.326+0000 7fa6d2a00700 1 -- 172.21.15.63:0/514478032 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa6e40c8050 con 0x556f47fcae70 2024-05-27T08:21:16.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.326+0000 7fa7023ceb80 10 client.42806 fetch_fsmap learned FSMap version 45 2024-05-27T08:21:16.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.326+0000 7fa7023ceb80 1 -- 172.21.15.63:0/514478032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556f47eb9060 con 0x556f47fcae70 2024-05-27T08:21:16.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.327+0000 7fa6d2a00700 1 -- 172.21.15.63:0/514478032 <== mon.2 v2:172.21.15.86:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556f47efad90 con 0x556f47fcae70 2024-05-27T08:21:16.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.328+0000 7fa6d2a00700 1 -- 172.21.15.63:0/514478032 <== mon.2 v2:172.21.15.86:3300/0 5 ==== fsmap(e 45) ==== 7749+0+0 (secure 0 0 0) 0x7fa6e40a7fa0 con 0x556f47fcae70 2024-05-27T08:21:16.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.328+0000 7fa7023ceb80 10 client.42806 fetch_fsmap finished waiting for FSMap version 45 2024-05-27T08:21:16.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.328+0000 7fa7023ceb80 10 client.42806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:21:16.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.328+0000 7fa7023ceb80 20 client.42806 populate_metadata read hostname 'smithi063' 2024-05-27T08:21:16.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.328+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556f47fe4b30 0x556f47fe6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.328+0000 7fa7023ceb80 4 client.42806 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:21:16.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.328+0000 7fa7023ceb80 1 -- 172.21.15.63:0/514478032 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f47ef4570 con 0x556f47fe4b30 2024-05-27T08:21:16.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.330+0000 7fa6f0e00700 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556f47fe4b30 0x556f47fe6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.334+0000 7fa6f0e00700 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556f47fe4b30 0x556f47fe6f70 crc :-1 s=READY pgs=798 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8ac4c2c2cf157c server_cookie=50cc3cffde41fb7a in_seq=0 out_seq=0 2024-05-27T08:21:16.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.334+0000 7fa6d2a00700 10 client.42806 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.335+0000 7fa6d2a00700 1 -- 172.21.15.63:0/514478032 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556f47ef4570 con 0x556f47fe4b30 2024-05-27T08:21:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.335+0000 7fa6d2a00700 10 client.42806 handle_command_reply: tid=0 2024-05-27T08:21:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.336+0000 7fa7023ceb80 1 client.42806 shutdown 2024-05-27T08:21:16.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.336+0000 7fa7023ceb80 1 -- 172.21.15.63:0/514478032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 msgr2=0x556f47fd51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:16.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.336+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd51d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40a8a40 tx=0x7fa6e40a9280 comp rx=0 tx=0).stop 2024-05-27T08:21:16.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.336+0000 7fa7023ceb80 1 -- 172.21.15.63:0/514478032 shutdown_connections 2024-05-27T08:21:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.336+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556f47fe4b30 0x556f47fe6f70 crc :-1 s=READY pgs=798 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.337+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd51d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.337+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fd4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.337+0000 7fa6d2a00700 0 client.42806 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.337+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/514478032 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fd4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.337+0000 7fa7023ceb80 1 -- 172.21.15.63:0/514478032 >> 172.21.15.63:0/514478032 conn(0x556f47ee5390 msgr2=0x556f47fca450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.337+0000 7fa7023ceb80 1 -- 172.21.15.63:0/514478032 shutdown_connections 2024-05-27T08:21:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.337+0000 7fa7023ceb80 1 -- 172.21.15.63:0/514478032 wait complete. 2024-05-27T08:21:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.338+0000 7fa7023ceb80 20 client.42806 trim_cache size 0 max 16384 2024-05-27T08:21:16.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.380+0000 7fa7023ceb80 1 Processor -- start 2024-05-27T08:21:16.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.380+0000 7fa7023ceb80 1 -- start start 2024-05-27T08:21:16.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.381+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fc6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.381+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fcff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.381+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.381+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1afeb0 con 0x556f47e4be40 2024-05-27T08:21:16.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.381+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1b05b0 con 0x556f47fcae70 2024-05-27T08:21:16.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.381+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1ade20 con 0x556f47f379f0 2024-05-27T08:21:16.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.382+0000 7fa6f0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fc6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.382+0000 7fa6f0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fc6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40046/0 (socket says 172.21.15.63:40046) 2024-05-27T08:21:16.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.382+0000 7fa6f0e00700 1 -- 172.21.15.63:0/711200881 learned_addr learned my addr 172.21.15.63:0/711200881 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:16.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.382+0000 7fa6f1800700 1 --2- 172.21.15.63:0/711200881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.382+0000 7fa6ebe00700 1 --2- 172.21.15.63:0/711200881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fcff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.382+0000 7fa6f1800700 1 -- 172.21.15.63:0/711200881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 msgr2=0x556f47fcff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:16.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.382+0000 7fa6f1800700 1 --2- 172.21.15.63:0/711200881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fcff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.382+0000 7fa6f1800700 1 -- 172.21.15.63:0/711200881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 msgr2=0x556f47fc6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:16.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.382+0000 7fa6f1800700 1 --2- 172.21.15.63:0/711200881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fc6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.383+0000 7fa6f1800700 1 -- 172.21.15.63:0/711200881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f47fd5540 con 0x556f47fcae70 2024-05-27T08:21:16.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.385+0000 7fa6f1800700 1 --2- 172.21.15.63:0/711200881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd6bc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40df6a0 tx=0x7fa6e40a8850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bcb4b6130571d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:16.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.386+0000 7fa6dd600700 1 -- 172.21.15.63:0/711200881 <== mon.2 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40b1bf0 con 0x556f47fcae70 2024-05-27T08:21:16.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.386+0000 7fa6dd600700 1 -- 172.21.15.63:0/711200881 <== mon.2 v2:172.21.15.86:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6e40a8d40 con 0x556f47fcae70 2024-05-27T08:21:16.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.386+0000 7fa6dd600700 1 -- 172.21.15.63:0/711200881 <== mon.2 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40a8690 con 0x556f47fcae70 2024-05-27T08:21:16.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.387+0000 7fa7023ceb80 1 -- 172.21.15.63:0/711200881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 msgr2=0x556f47fd6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:16.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.387+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/711200881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd6bc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40df6a0 tx=0x7fa6e40a8850 comp rx=0 tx=0).stop 2024-05-27T08:21:16.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.387+0000 7fa7023ceb80 1 -- 172.21.15.63:0/711200881 shutdown_connections 2024-05-27T08:21:16.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.387+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/711200881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd6bc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.387+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/711200881 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fc6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.387+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/711200881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fcff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.387+0000 7fa7023ceb80 1 -- 172.21.15.63:0/711200881 >> 172.21.15.63:0/711200881 conn(0x556f47ee5390 msgr2=0x556f47fc5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:16.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.390+0000 7fa7023ceb80 1 -- 172.21.15.63:0/711200881 shutdown_connections 2024-05-27T08:21:16.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.390+0000 7fa7023ceb80 1 -- 172.21.15.63:0/711200881 wait complete. 2024-05-27T08:21:16.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.391+0000 7fa7023ceb80 1 -- start start 2024-05-27T08:21:16.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.391+0000 7fa7023ceb80 1 Processor -- start 2024-05-27T08:21:16.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.392+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fd8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.393+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fd8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.393+0000 7fa7023ceb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.394+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1ad750 con 0x556f47e4be40 2024-05-27T08:21:16.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.394+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1ac340 con 0x556f47fcae70 2024-05-27T08:21:16.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.394+0000 7fa6ebe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fd8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.394+0000 7fa7023ceb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec1ac9b0 con 0x556f47f379f0 2024-05-27T08:21:16.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.394+0000 7fa7023ceb80 10 client.0 _ll_register_callbacks cb 0x556f47e92c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:21:16.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.394+0000 7fa6ebe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fd8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:35914/0 (socket says 172.21.15.63:35914) 2024-05-27T08:21:16.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.394+0000 7fa6ebe00700 1 -- 172.21.15.63:0/3046994729 learned_addr learned my addr 172.21.15.63:0/3046994729 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:16.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.394+0000 7fa6f0e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fd8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.394+0000 7fa6f1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.395+0000 7fa6f0e00700 1 -- 172.21.15.63:0/3046994729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 msgr2=0x556f47fd8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:16.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.395+0000 7fa6f0e00700 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fd8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.395+0000 7fa6f0e00700 1 -- 172.21.15.63:0/3046994729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 msgr2=0x556f47fd9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:16.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.395+0000 7fa6f0e00700 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.395+0000 7fa6f0e00700 1 -- 172.21.15.63:0/3046994729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f4804e610 con 0x556f47e4be40 2024-05-27T08:21:16.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.397+0000 7fa6f0e00700 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fd8aa0 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0003760 tx=0x7fa6e0005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece8812a7551daee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:16.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.397+0000 7fa6d2a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:21:16.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.398+0000 7fa6d2a00700 1 -- 172.21.15.63:0/3046994729 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0029070 con 0x556f47e4be40 2024-05-27T08:21:16.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.398+0000 7fa6d2a00700 1 -- 172.21.15.63:0/3046994729 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(179..179 src has 1..179) ==== 9571+0+0 (secure 0 0 0) 0x7fa6e00039c0 con 0x556f47e4be40 2024-05-27T08:21:16.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.398+0000 7fa7023ceb80 1 -- 172.21.15.63:0/3046994729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6e40a8d40 con 0x556f47e4be40 2024-05-27T08:21:16.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.398+0000 7fa6d2a00700 1 -- 172.21.15.63:0/3046994729 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa6e0035050 con 0x556f47e4be40 2024-05-27T08:21:16.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.398+0000 7fa7023ceb80 10 client.23865 fetch_fsmap learned FSMap version 45 2024-05-27T08:21:16.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.398+0000 7fa7023ceb80 1 -- 172.21.15.63:0/3046994729 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556f4804f760 con 0x556f47e4be40 2024-05-27T08:21:16.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.399+0000 7fa6d2a00700 1 -- 172.21.15.63:0/3046994729 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6e40a8d40 con 0x556f47e4be40 2024-05-27T08:21:16.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.399+0000 7fa6d2a00700 1 -- 172.21.15.63:0/3046994729 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 45) ==== 7749+0+0 (secure 0 0 0) 0x7fa6e0025420 con 0x556f47e4be40 2024-05-27T08:21:16.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.400+0000 7fa7023ceb80 10 client.23865 fetch_fsmap finished waiting for FSMap version 45 2024-05-27T08:21:16.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.400+0000 7fa7023ceb80 10 client.23865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:21:16.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.400+0000 7fa7023ceb80 20 client.23865 populate_metadata read hostname 'smithi063' 2024-05-27T08:21:16.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.400+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556f4804fe40 0x556f48052260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:16.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.400+0000 7fa7023ceb80 4 client.23865 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:21:16.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.400+0000 7fa7023ceb80 1 -- 172.21.15.63:0/3046994729 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f47d6b250 con 0x556f4804fe40 2024-05-27T08:21:16.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.400+0000 7fa6ebe00700 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556f4804fe40 0x556f48052260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:16.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.401+0000 7fa6ebe00700 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556f4804fe40 0x556f48052260 crc :-1 s=READY pgs=799 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525114a8c903cee3 server_cookie=f982d8de14d26b2 in_seq=0 out_seq=0 2024-05-27T08:21:16.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.401+0000 7fa6d2a00700 10 client.23865 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:16.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.402+0000 7fa6d2a00700 1 -- 172.21.15.63:0/3046994729 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556f47d6b250 con 0x556f4804fe40 2024-05-27T08:21:16.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.402+0000 7fa6d2a00700 10 client.23865 handle_command_reply: tid=0 2024-05-27T08:21:16.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.402+0000 7fa7023ceb80 1 client.23865 shutdown 2024-05-27T08:21:16.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.402+0000 7fa7023ceb80 1 -- 172.21.15.63:0/3046994729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 msgr2=0x556f47fd8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:16.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.402+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fd8aa0 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0003760 tx=0x7fa6e0005350 comp rx=0 tx=0).stop 2024-05-27T08:21:16.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.403+0000 7fa7023ceb80 1 -- 172.21.15.63:0/3046994729 shutdown_connections 2024-05-27T08:21:16.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.403+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x556f4804fe40 0x556f48052260 crc :-1 s=READY pgs=799 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.403+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f47fcae70 0x556f47fd9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.403+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556f47e4be40 0x556f47fd8aa0 unknown :-1 s=CLOSED pgs=3014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.403+0000 7fa7023ceb80 1 --2- 172.21.15.63:0/3046994729 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556f47f379f0 0x556f47fd8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.403+0000 7fa6d2a00700 0 client.23865 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:16.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.403+0000 7fa7023ceb80 1 -- 172.21.15.63:0/3046994729 >> 172.21.15.63:0/3046994729 conn(0x556f47ee5390 msgr2=0x556f47fd4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:16.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.403+0000 7fa7023ceb80 1 -- 172.21.15.63:0/3046994729 shutdown_connections 2024-05-27T08:21:16.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.404+0000 7fa7023ceb80 1 -- 172.21.15.63:0/3046994729 wait complete. 2024-05-27T08:21:16.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.405+0000 7fa7023ceb80 20 client.23865 trim_cache size 0 max 16384 2024-05-27T08:21:16.407 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:21:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "status": "no active scrubs running", 2024-05-27T08:21:16.407 INFO:teuthology.orchestra.run.smithi063.stdout: "scrubs": {} 2024-05-27T08:21:16.407 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:21:16.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.406+0000 7fa6f3800700 1 -- 172.21.15.63:0/2720088239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6d40b2980 msgr2=0x7fa6d40b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:16.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.406+0000 7fa6f3800700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6d40b2980 0x7fa6d40b4e90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c610 tx=0x7fa6d8008040 comp rx=0 tx=0).stop 2024-05-27T08:21:16.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.406+0000 7fa6f3800700 1 -- 172.21.15.63:0/2720088239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6ec110bd0 msgr2=0x7fa6ec10fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:16.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.406+0000 7fa6f3800700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6ec110bd0 0x7fa6ec10fed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b9e0 tx=0x7fa6e400bda0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.407+0000 7fa6f3800700 1 -- 172.21.15.63:0/2720088239 shutdown_connections 2024-05-27T08:21:16.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.407+0000 7fa6f3800700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6d40b2980 0x7fa6d40b4e90 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.407+0000 7fa6f3800700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6ec110bd0 0x7fa6ec10fed0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.407+0000 7fa6f3800700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6ec076c00 0x7fa6ec10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.407+0000 7fa6f3800700 1 --2- 172.21.15.63:0/2720088239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6ec077550 0x7fa6ec10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:16.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.407+0000 7fa6f3800700 1 -- 172.21.15.63:0/2720088239 >> 172.21.15.63:0/2720088239 conn(0x7fa6ec0ffe70 msgr2=0x7fa6ec100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:16.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.408+0000 7fa6f3800700 1 -- 172.21.15.63:0/2720088239 shutdown_connections 2024-05-27T08:21:16.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:16.408+0000 7fa6f3800700 1 -- 172.21.15.63:0/2720088239 wait complete. 2024-05-27T08:21:16.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:16 smithi086 ceph-mon[34687]: pgmap v1519: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.0 KiB/s wr, 150 op/s 2024-05-27T08:21:16.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:16 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6832/2740612632,v1:172.21.15.149:6833/2740612632] up:boot 2024-05-27T08:21:16.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:16 smithi086 ceph-mon[34687]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:16.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:16 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:16.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:16 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:16.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:16 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:21:16.964 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:16 smithi149 podman[81091]: 2024-05-27 08:21:16.481622582 +0000 UTC m=+0.978641250 container remove 8bce2596cb75ef953e221340de82e79aea70ef82fcfc8624b95553692bf08b24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-27T08:21:16.964 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:16 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:17.412 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:21:17.413 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:21:17.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:17 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/4094037361,v1:172.21.15.149:6835/4094037361] up:boot 2024-05-27T08:21:17.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:17 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:17.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:17 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:21:17.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:17 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T08:21:17.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:17 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.464s CPU time. 2024-05-27T08:21:17.509 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:17 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/4094037361,v1:172.21.15.149:6835/4094037361] up:boot 2024-05-27T08:21:17.509 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:17 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:17.509 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:17 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:21:17.509 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:17 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Failed with result 'exit-code'. 2024-05-27T08:21:17.510 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:17 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Consumed 3.213s CPU time. 2024-05-27T08:21:17.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:17 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/4094037361,v1:172.21.15.149:6835/4094037361] up:boot 2024-05-27T08:21:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:17 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:17.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:17 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:21:18.008 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:21:18.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:18 smithi063 ceph-mon[31613]: pgmap v1520: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.0 KiB/s wr, 142 op/s 2024-05-27T08:21:18.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:18 smithi086 ceph-mon[34687]: pgmap v1520: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.0 KiB/s wr, 142 op/s 2024-05-27T08:21:18.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:18 smithi149 ceph-mon[35408]: pgmap v1520: 129 pgs: 17 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 65 active+clean; 15 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 3.0 KiB/s wr, 142 op/s 2024-05-27T08:21:20.513 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:20 smithi086 ceph-mon[34687]: pgmap v1521: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 2.9 KiB/s wr, 53 op/s 2024-05-27T08:21:20.612 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:20 smithi063 ceph-mon[31613]: pgmap v1521: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 2.9 KiB/s wr, 53 op/s 2024-05-27T08:21:20.713 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:20 smithi149 ceph-mon[35408]: pgmap v1521: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 2.9 KiB/s wr, 53 op/s 2024-05-27T08:21:21.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.909+0000 7ffa74c00700 1 -- 172.21.15.63:0/2753876832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7010b890 msgr2=0x7ffa701131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:21.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.909+0000 7ffa74c00700 1 --2- 172.21.15.63:0/2753876832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7010b890 0x7ffa701131f0 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800b0d0 tx=0x7ffa6800b3e0 comp rx=0 tx=0).stop 2024-05-27T08:21:21.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.909+0000 7ffa74c00700 1 -- 172.21.15.63:0/2753876832 shutdown_connections 2024-05-27T08:21:21.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.909+0000 7ffa74c00700 1 --2- 172.21.15.63:0/2753876832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7010b890 0x7ffa701131f0 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:21.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.909+0000 7ffa74c00700 1 --2- 172.21.15.63:0/2753876832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa70106610 0x7ffa70106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:21.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.909+0000 7ffa74c00700 1 --2- 172.21.15.63:0/2753876832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa7010af40 0x7ffa7010b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:21.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.909+0000 7ffa74c00700 1 -- 172.21.15.63:0/2753876832 >> 172.21.15.63:0/2753876832 conn(0x7ffa70102020 msgr2=0x7ffa70104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:21.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.909+0000 7ffa74c00700 1 -- 172.21.15.63:0/2753876832 shutdown_connections 2024-05-27T08:21:21.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.909+0000 7ffa74c00700 1 -- 172.21.15.63:0/2753876832 wait complete. 2024-05-27T08:21:21.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.911+0000 7ffa74c00700 1 Processor -- start 2024-05-27T08:21:21.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.911+0000 7ffa74c00700 1 -- start start 2024-05-27T08:21:21.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.911+0000 7ffa74c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa70106610 0x7ffa7019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:21.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.911+0000 7ffa74c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa7010af40 0x7ffa7019d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:21.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.911+0000 7ffa74c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7010b890 0x7ffa701a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:21.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.911+0000 7ffa74c00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701163c0 con 0x7ffa7010b890 2024-05-27T08:21:21.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.911+0000 7ffa74c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70116240 con 0x7ffa70106610 2024-05-27T08:21:21.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.911+0000 7ffa74c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70116540 con 0x7ffa7010af40 2024-05-27T08:21:21.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7010b890 0x7ffa701a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:21.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6fe00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7010b890 0x7ffa701a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60268/0 (socket says 172.21.15.63:60268) 2024-05-27T08:21:21.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6fe00700 1 -- 172.21.15.63:0/752312668 learned_addr learned my addr 172.21.15.63:0/752312668 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:21.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6ea00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa7010af40 0x7ffa7019d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:21.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6f400700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa70106610 0x7ffa7019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:21.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6fe00700 1 -- 172.21.15.63:0/752312668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa7010af40 msgr2=0x7ffa7019d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:21.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6fe00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa7010af40 0x7ffa7019d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:21.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6fe00700 1 -- 172.21.15.63:0/752312668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa70106610 msgr2=0x7ffa7019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:21.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6fe00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa70106610 0x7ffa7019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:21.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6fe00700 1 -- 172.21.15.63:0/752312668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa701a1c00 con 0x7ffa7010b890 2024-05-27T08:21:21.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.912+0000 7ffa6f400700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa70106610 0x7ffa7019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:21.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.913+0000 7ffa6fe00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7010b890 0x7ffa701a1560 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7ffa68007d10 tx=0x7ffa68007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:21.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.913+0000 7ffa67e00700 1 -- 172.21.15.63:0/752312668 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6801f070 con 0x7ffa7010b890 2024-05-27T08:21:21.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.913+0000 7ffa67e00700 1 -- 172.21.15.63:0/752312668 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa68003e10 con 0x7ffa7010b890 2024-05-27T08:21:21.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.913+0000 7ffa67e00700 1 -- 172.21.15.63:0/752312668 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6801b820 con 0x7ffa7010b890 2024-05-27T08:21:21.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.913+0000 7ffa74c00700 1 -- 172.21.15.63:0/752312668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa701a1e90 con 0x7ffa7010b890 2024-05-27T08:21:21.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.913+0000 7ffa74c00700 1 -- 172.21.15.63:0/752312668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa701a9810 con 0x7ffa7010b890 2024-05-27T08:21:21.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.915+0000 7ffa74c00700 1 -- 172.21.15.63:0/752312668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa70107ac0 con 0x7ffa7010b890 2024-05-27T08:21:21.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.917+0000 7ffa67e00700 1 -- 172.21.15.63:0/752312668 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffa68022070 con 0x7ffa7010b890 2024-05-27T08:21:21.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.919+0000 7ffa67e00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa540b2a70 0x7ffa540b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:21.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.919+0000 7ffa67e00700 1 -- 172.21.15.63:0/752312668 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 9571+0+0 (secure 0 0 0) 0x7ffa6800f020 con 0x7ffa7010b890 2024-05-27T08:21:21.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.919+0000 7ffa6f400700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa540b2a70 0x7ffa540b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:21.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.920+0000 7ffa6f400700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa540b2a70 0x7ffa540b4f80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005950 tx=0x7ffa600058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:21.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:21.922+0000 7ffa67e00700 1 -- 172.21.15.63:0/752312668 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa68029070 con 0x7ffa7010b890 2024-05-27T08:21:21.965 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa974a00700 time 2024-05-27T08:21:21.626900+0000 2024-05-27T08:21:21.965 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:21.965 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:21.965 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa98a605ba6] 2024-05-27T08:21:21.965 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa98a605dd1] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5577708fe23d] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5577708e9971] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 5: (SnapRealm::check_cache() const+0x13e) [0x5577708e7fa0] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5577708e8651] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55777087d347] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55777057987e] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55777057ee94] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x557770589969] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 11: (MDLog::_replay_thread()+0x1f38) [0x55777092248c] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55777092b9cb] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 13: (Thread::entry_wrapper()+0x43) [0x7fa98a5d472d] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa98a5d4749] 2024-05-27T08:21:21.966 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa9891781ca] 2024-05-27T08:21:21.967 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 16: clone() 2024-05-27T08:21:21.967 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.629+0000 7fa974a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa974a00700 time 2024-05-27T08:21:21.626900+0000 2024-05-27T08:21:21.967 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:21.967 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.967 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:21.967 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa98a605ba6] 2024-05-27T08:21:21.967 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa98a605dd1] 2024-05-27T08:21:21.967 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5577708fe23d] 2024-05-27T08:21:21.967 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5577708e9971] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 5: (SnapRealm::check_cache() const+0x13e) [0x5577708e7fa0] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5577708e8651] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55777087d347] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55777057987e] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55777057ee94] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x557770589969] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 11: (MDLog::_replay_thread()+0x1f38) [0x55777092248c] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55777092b9cb] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 13: (Thread::entry_wrapper()+0x43) [0x7fa98a5d472d] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa98a5d4749] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa9891781ca] 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 16: clone() 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.968 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: *** Caught signal (Aborted) ** 2024-05-27T08:21:21.969 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: in thread 7fa974a00700 thread_name:md_log_replay 2024-05-27T08:21:21.969 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:21.969 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa989182d20] 2024-05-27T08:21:21.969 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2: gsignal() 2024-05-27T08:21:21.969 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 3: abort() 2024-05-27T08:21:21.969 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa98a605ced] 2024-05-27T08:21:21.969 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa98a605dd1] 2024-05-27T08:21:21.970 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5577708fe23d] 2024-05-27T08:21:21.970 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5577708e9971] 2024-05-27T08:21:21.970 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 8: (SnapRealm::check_cache() const+0x13e) [0x5577708e7fa0] 2024-05-27T08:21:21.970 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5577708e8651] 2024-05-27T08:21:21.970 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55777087d347] 2024-05-27T08:21:21.970 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55777057987e] 2024-05-27T08:21:21.970 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55777057ee94] 2024-05-27T08:21:21.970 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x557770589969] 2024-05-27T08:21:21.970 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 14: (MDLog::_replay_thread()+0x1f38) [0x55777092248c] 2024-05-27T08:21:21.970 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55777092b9cb] 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 16: (Thread::entry_wrapper()+0x43) [0x7fa98a5d472d] 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa98a5d4749] 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa9891781ca] 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 19: clone() 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.633+0000 7fa974a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: in thread 7fa974a00700 thread_name:md_log_replay 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa989182d20] 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2: gsignal() 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 3: abort() 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa98a605ced] 2024-05-27T08:21:21.971 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa98a605dd1] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5577708fe23d] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5577708e9971] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 8: (SnapRealm::check_cache() const+0x13e) [0x5577708e7fa0] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5577708e8651] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55777087d347] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55777057987e] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55777057ee94] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x557770589969] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 14: (MDLog::_replay_thread()+0x1f38) [0x55777092248c] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55777092b9cb] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 16: (Thread::entry_wrapper()+0x43) [0x7fa98a5d472d] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa98a5d4749] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa9891781ca] 2024-05-27T08:21:21.972 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 19: clone() 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: -1> 2024-05-27T08:21:21.629+0000 7fa974a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa974a00700 time 2024-05-27T08:21:21.626900+0000 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa98a605ba6] 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa98a605dd1] 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5577708fe23d] 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5577708e9971] 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 5: (SnapRealm::check_cache() const+0x13e) [0x5577708e7fa0] 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5577708e8651] 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55777087d347] 2024-05-27T08:21:21.973 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55777057987e] 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55777057ee94] 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x557770589969] 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 11: (MDLog::_replay_thread()+0x1f38) [0x55777092248c] 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55777092b9cb] 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 13: (Thread::entry_wrapper()+0x43) [0x7fa98a5d472d] 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa98a5d4749] 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa9891781ca] 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 16: clone() 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 0> 2024-05-27T08:21:21.633+0000 7fa974a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: in thread 7fa974a00700 thread_name:md_log_replay 2024-05-27T08:21:21.974 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa989182d20] 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2: gsignal() 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 3: abort() 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa98a605ced] 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa98a605dd1] 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5577708fe23d] 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5577708e9971] 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 8: (SnapRealm::check_cache() const+0x13e) [0x5577708e7fa0] 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5577708e8651] 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55777087d347] 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55777057987e] 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55777057ee94] 2024-05-27T08:21:21.975 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x557770589969] 2024-05-27T08:21:21.976 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 14: (MDLog::_replay_thread()+0x1f38) [0x55777092248c] 2024-05-27T08:21:21.976 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55777092b9cb] 2024-05-27T08:21:21.976 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 16: (Thread::entry_wrapper()+0x43) [0x7fa98a5d472d] 2024-05-27T08:21:21.976 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa98a5d4749] 2024-05-27T08:21:21.976 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa9891781ca] 2024-05-27T08:21:21.976 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 19: clone() 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: -1> 2024-05-27T08:21:21.629+0000 7fa974a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa974a00700 time 2024-05-27T08:21:21.626900+0000 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa98a605ba6] 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa98a605dd1] 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5577708fe23d] 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5577708e9971] 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 5: (SnapRealm::check_cache() const+0x13e) [0x5577708e7fa0] 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5577708e8651] 2024-05-27T08:21:21.977 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55777087d347] 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55777057987e] 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55777057ee94] 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x557770589969] 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 11: (MDLog::_replay_thread()+0x1f38) [0x55777092248c] 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55777092b9cb] 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 13: (Thread::entry_wrapper()+0x43) [0x7fa98a5d472d] 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa98a5d4749] 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa9891781ca] 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 16: clone() 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 0> 2024-05-27T08:21:21.633+0000 7fa974a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: in thread 7fa974a00700 thread_name:md_log_replay 2024-05-27T08:21:21.978 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa989182d20] 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2: gsignal() 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 3: abort() 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa98a605ced] 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa98a605dd1] 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5577708fe23d] 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5577708e9971] 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 8: (SnapRealm::check_cache() const+0x13e) [0x5577708e7fa0] 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5577708e8651] 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55777087d347] 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55777057987e] 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55777057ee94] 2024-05-27T08:21:21.979 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x557770589969] 2024-05-27T08:21:21.980 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 14: (MDLog::_replay_thread()+0x1f38) [0x55777092248c] 2024-05-27T08:21:21.980 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55777092b9cb] 2024-05-27T08:21:21.980 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 16: (Thread::entry_wrapper()+0x43) [0x7fa98a5d472d] 2024-05-27T08:21:21.980 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa98a5d4749] 2024-05-27T08:21:21.980 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa9891781ca] 2024-05-27T08:21:21.980 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 19: clone() 2024-05-27T08:21:21.980 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:21.980 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81035]: 2024-05-27T08:21:21.980 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:21 smithi149 podman[81154]: 2024-05-27 08:21:21.740912045 +0000 UTC m=+0.035631531 container died c327b144344a1bbfd17b0f94ef090c3bca526a9e144f0f52b38e037ae0124aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-27T08:21:22.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.209+0000 7ffa74c00700 1 -- 172.21.15.63:0/752312668 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa7019e090 con 0x7ffa7010b890 2024-05-27T08:21:22.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.212+0000 7ffa67e00700 1 -- 172.21.15.63:0/752312668 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+10207 (secure 0 0 0) 0x7ffa680ab670 con 0x7ffa7010b890 2024-05-27T08:21:22.214 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:21:22.214 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/134276953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":134276953},{"type":"v1","addr":"172.21.15.63:6835","nonce":134276953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2286910897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6836","nonce":2286910897},{"type":"v1","addr":"172.21.15.63:6837","nonce":2286910897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6839/3288686403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6838","nonce":3288686403},{"type":"v1","addr":"172.21.15.63:6839","nonce":3288686403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":23850,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/1512010809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1512010809},{"type":"v1","addr":"172.21.15.86:6833","nonce":1512010809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:28:27.557349+0000","modified":"2024-05-27T08:21:21.599587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34520,"mds_1":34484,"mds_2":34454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6841/80080490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6840","nonce":80080490},{"type":"v1","addr":"172.21.15.63:6841","nonce":80080490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_32596":{"gid":32596,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/2740612632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2740612632},{"type":"v1","addr":"172.21.15.149:6833","nonce":2740612632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34454":{"gid":34454,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":35,"addr":"172.21.15.86:6835/1494101824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1494101824},{"type":"v1","addr":"172.21.15.86:6835","nonce":1494101824}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34484":{"gid":34484,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":32,"addr":"172.21.15.86:6837/559054291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":559054291},{"type":"v1","addr":"172.21.15.86:6837","nonce":559054291}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6839/953218889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":953218889},{"type":"v1","addr":"172.21.15.86:6839","nonce":953218889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_42794":{"gid":42794,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/4094037361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":4094037361},{"type":"v1","addr":"172.21.15.149:6835","nonce":4094037361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,34454,34484]},"id":1}]} 2024-05-27T08:21:22.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.215+0000 7ffa74c00700 1 -- 172.21.15.63:0/752312668 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa540b2a70 msgr2=0x7ffa540b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:22.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.215+0000 7ffa74c00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa540b2a70 0x7ffa540b4f80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005950 tx=0x7ffa600058e0 comp rx=0 tx=0).stop 2024-05-27T08:21:22.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.215+0000 7ffa74c00700 1 -- 172.21.15.63:0/752312668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7010b890 msgr2=0x7ffa701a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:22.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.215+0000 7ffa74c00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7010b890 0x7ffa701a1560 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7ffa68007d10 tx=0x7ffa68007df0 comp rx=0 tx=0).stop 2024-05-27T08:21:22.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.215+0000 7ffa74c00700 1 -- 172.21.15.63:0/752312668 shutdown_connections 2024-05-27T08:21:22.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.215+0000 7ffa74c00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ffa540b2a70 0x7ffa540b4f80 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:22.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.215+0000 7ffa74c00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7010b890 0x7ffa701a1560 unknown :-1 s=CLOSED pgs=3016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:22.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.215+0000 7ffa74c00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa70106610 0x7ffa7019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:22.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.215+0000 7ffa74c00700 1 --2- 172.21.15.63:0/752312668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa7010af40 0x7ffa7019d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:22.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.215+0000 7ffa74c00700 1 -- 172.21.15.63:0/752312668 >> 172.21.15.63:0/752312668 conn(0x7ffa70102020 msgr2=0x7ffa7010a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:22.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.216+0000 7ffa74c00700 1 -- 172.21.15.63:0/752312668 shutdown_connections 2024-05-27T08:21:22.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:22.216+0000 7ffa74c00700 1 -- 172.21.15.63:0/752312668 wait complete. 2024-05-27T08:21:22.219 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 46 2024-05-27T08:21:22.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:22 smithi149 ceph-mon[35408]: pgmap v1522: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 2.9 KiB/s wr, 47 op/s 2024-05-27T08:21:22.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:22 smithi149 ceph-mon[35408]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:22.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:22 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:21:22.450 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:22 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/752312668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:21:22.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:22 smithi063 ceph-mon[31613]: pgmap v1522: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 2.9 KiB/s wr, 47 op/s 2024-05-27T08:21:22.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:22 smithi063 ceph-mon[31613]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:22.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:22 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:21:22.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:22 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/752312668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:21:22.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:22 smithi086 ceph-mon[34687]: pgmap v1522: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 2.9 KiB/s wr, 47 op/s 2024-05-27T08:21:22.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:22 smithi086 ceph-mon[34687]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-05-27T08:21:22.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:22 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:21:22.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:22 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/752312668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:21:22.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:22 smithi149 podman[81154]: 2024-05-27 08:21:22.448016741 +0000 UTC m=+0.742736261 container remove c327b144344a1bbfd17b0f94ef090c3bca526a9e144f0f52b38e037ae0124aec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:21:22.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:22 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:23.463 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:22 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-05-27T08:21:23.464 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:23 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 1.580s CPU time. 2024-05-27T08:21:23.516 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:21:23.995 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:21:24.397 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:24 smithi063 ceph-mon[31613]: pgmap v1523: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.7 KiB/s wr, 41 op/s 2024-05-27T08:21:24.709 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:24 smithi086 ceph-mon[34687]: pgmap v1523: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.7 KiB/s wr, 41 op/s 2024-05-27T08:21:24.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:24 smithi149 ceph-mon[35408]: pgmap v1523: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 2.7 KiB/s wr, 41 op/s 2024-05-27T08:21:25.982 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:25 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 2. 2024-05-27T08:21:25.982 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:25 smithi149 systemd[1]: Stopped Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:25.982 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:25 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 2.835s CPU time. 2024-05-27T08:21:25.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:25 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 1. 2024-05-27T08:21:25.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:25 smithi149 systemd[1]: Stopped Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:25.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:25 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Consumed 3.094s CPU time. 2024-05-27T08:21:25.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:25 smithi149 systemd[1]: Starting Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:26.254 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:25 smithi149 systemd[1]: Starting Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:26.254 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:26 smithi149 podman[81247]: 2024-05-27 08:21:26.030036728 +0000 UTC m=+0.034124918 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:26.254 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:26 smithi149 podman[81247]: 2024-05-27 08:21:26.165803241 +0000 UTC m=+0.169891484 container create 0760db8717cda5a4ea09a3f6aa9ba09320d8c7e4613b7f417fec5389a90f63f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-27T08:21:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.434+0000 7f21c8000700 1 -- 172.21.15.63:0/202772243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21c010c5b0 msgr2=0x7f21c010c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.434+0000 7f21c8000700 1 --2- 172.21.15.63:0/202772243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21c010c5b0 0x7f21c010c9c0 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009990 tx=0x7f21b4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.434+0000 7f21c8000700 1 -- 172.21.15.63:0/202772243 shutdown_connections 2024-05-27T08:21:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.434+0000 7f21c8000700 1 --2- 172.21.15.63:0/202772243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c0106ec0 0x7f21c0113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.434+0000 7f21c8000700 1 --2- 172.21.15.63:0/202772243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21c010c5b0 0x7f21c010c9c0 unknown :-1 s=CLOSED pgs=3017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.434+0000 7f21c8000700 1 --2- 172.21.15.63:0/202772243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c0106570 0x7f21c0106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.434+0000 7f21c8000700 1 -- 172.21.15.63:0/202772243 >> 172.21.15.63:0/202772243 conn(0x7f21c0102070 msgr2=0x7f21c01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.434+0000 7f21c8000700 1 -- 172.21.15.63:0/202772243 shutdown_connections 2024-05-27T08:21:26.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.434+0000 7f21c8000700 1 -- 172.21.15.63:0/202772243 wait complete. 2024-05-27T08:21:26.436 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:26 smithi063 ceph-mon[31613]: pgmap v1524: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.7 KiB/s wr, 20 op/s 2024-05-27T08:21:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.436+0000 7f21c8000700 1 Processor -- start 2024-05-27T08:21:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.436+0000 7f21c8000700 1 -- start start 2024-05-27T08:21:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.436+0000 7f21c8000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c0106570 0x7f21c019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.436+0000 7f21c8000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c0106ec0 0x7f21c019d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.436+0000 7f21c8000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21c010c5b0 0x7f21c01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.436+0000 7f21c8000700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01165c0 con 0x7f21c010c5b0 2024-05-27T08:21:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.436+0000 7f21c8000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0116440 con 0x7f21c0106ec0 2024-05-27T08:21:26.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.436+0000 7f21c8000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0116740 con 0x7f21c0106570 2024-05-27T08:21:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21c010c5b0 0x7f21c01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21c010c5b0 0x7f21c01a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60286/0 (socket says 172.21.15.63:60286) 2024-05-27T08:21:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c6000700 1 -- 172.21.15.63:0/437607070 learned_addr learned my addr 172.21.15.63:0/437607070 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:26.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c4c00700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c0106ec0 0x7f21c019d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c5600700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c0106570 0x7f21c019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c6000700 1 -- 172.21.15.63:0/437607070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c0106570 msgr2=0x7f21c019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c6000700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c0106570 0x7f21c019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c6000700 1 -- 172.21.15.63:0/437607070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c0106ec0 msgr2=0x7f21c019d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c6000700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c0106ec0 0x7f21c019d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c6000700 1 -- 172.21.15.63:0/437607070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c01a1e40 con 0x7f21c010c5b0 2024-05-27T08:21:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.437+0000 7f21c5600700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c0106570 0x7f21c019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:26.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.438+0000 7f21c6000700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21c010c5b0 0x7f21c01a17a0 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 crypto rx=0x7f21b8007ee0 tx=0x7f21b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.438+0000 7f21be000700 1 -- 172.21.15.63:0/437607070 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b800dbd0 con 0x7f21c010c5b0 2024-05-27T08:21:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.438+0000 7f21be000700 1 -- 172.21.15.63:0/437607070 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21b800dd70 con 0x7f21c010c5b0 2024-05-27T08:21:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.438+0000 7f21be000700 1 -- 172.21.15.63:0/437607070 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b80058d0 con 0x7f21c010c5b0 2024-05-27T08:21:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.438+0000 7f21c8000700 1 -- 172.21.15.63:0/437607070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c01a2130 con 0x7f21c010c5b0 2024-05-27T08:21:26.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.438+0000 7f21c8000700 1 -- 172.21.15.63:0/437607070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c01a9a50 con 0x7f21c010c5b0 2024-05-27T08:21:26.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.442+0000 7f21be000700 1 -- 172.21.15.63:0/437607070 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f21b801a020 con 0x7f21c010c5b0 2024-05-27T08:21:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.443+0000 7f21be000700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21a80b29f0 0x7f21a80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.444+0000 7f21be000700 1 -- 172.21.15.63:0/437607070 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 9571+0+0 (secure 0 0 0) 0x7f21b80e2280 con 0x7f21c010c5b0 2024-05-27T08:21:26.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.444+0000 7f21c5600700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21a80b29f0 0x7f21a80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.445+0000 7f21c5600700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21a80b29f0 0x7f21a80b4f00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f21b4006010 tx=0x7f21b400b540 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.457+0000 7f21d6b07b80 1 Processor -- start 2024-05-27T08:21:26.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.458+0000 7f21d6b07b80 1 -- start start 2024-05-27T08:21:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.458+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed467d3e40 0x55ed469587f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.458+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 0x55ed46952870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.458+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46952db0 0x55ed469531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.458+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ed46880e40 con 0x55ed467d3e40 2024-05-27T08:21:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.458+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed468709e0 con 0x55ed46952db0 2024-05-27T08:21:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.458+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ed46874a20 con 0x55ed46958d30 2024-05-27T08:21:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c5600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed467d3e40 0x55ed469587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c5600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed467d3e40 0x55ed469587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60298/0 (socket says 172.21.15.63:60298) 2024-05-27T08:21:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c5600700 1 -- 172.21.15.63:0/605931929 learned_addr learned my addr 172.21.15.63:0/605931929 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c4c00700 1 --2- 172.21.15.63:0/605931929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 0x55ed46952870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c6000700 1 --2- 172.21.15.63:0/605931929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46952db0 0x55ed469531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c5600700 1 -- 172.21.15.63:0/605931929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 msgr2=0x55ed46952870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c5600700 1 --2- 172.21.15.63:0/605931929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 0x55ed46952870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c5600700 1 -- 172.21.15.63:0/605931929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46952db0 msgr2=0x55ed469531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c5600700 1 --2- 172.21.15.63:0/605931929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46952db0 0x55ed469531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c5600700 1 -- 172.21.15.63:0/605931929 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed46839420 con 0x55ed467d3e40 2024-05-27T08:21:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.459+0000 7f21c6000700 1 --2- 172.21.15.63:0/605931929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46952db0 0x55ed469531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.460+0000 7f21c5600700 1 --2- 172.21.15.63:0/605931929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed467d3e40 0x55ed469587f0 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 crypto rx=0x7f21b4015000 tx=0x7f21b401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cda2f39fd3dba9b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.460+0000 7f21b2a00700 1 -- 172.21.15.63:0/605931929 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4003c50 con 0x55ed467d3e40 2024-05-27T08:21:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.460+0000 7f21b2a00700 1 -- 172.21.15.63:0/605931929 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21b402a030 con 0x55ed467d3e40 2024-05-27T08:21:26.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.460+0000 7f21b2a00700 1 -- 172.21.15.63:0/605931929 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b400f7c0 con 0x55ed467d3e40 2024-05-27T08:21:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.461+0000 7f21d6b07b80 1 -- 172.21.15.63:0/605931929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed467d3e40 msgr2=0x55ed469587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.461+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/605931929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed467d3e40 0x55ed469587f0 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 crypto rx=0x7f21b4015000 tx=0x7f21b401d000 comp rx=0 tx=0).stop 2024-05-27T08:21:26.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.461+0000 7f21d6b07b80 1 -- 172.21.15.63:0/605931929 shutdown_connections 2024-05-27T08:21:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.461+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/605931929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46952db0 0x55ed469531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.462+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/605931929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed467d3e40 0x55ed469587f0 unknown :-1 s=CLOSED pgs=3019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.462+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/605931929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 0x55ed46952870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.462+0000 7f21d6b07b80 1 -- 172.21.15.63:0/605931929 >> 172.21.15.63:0/605931929 conn(0x55ed46833fc0 msgr2=0x55ed4694d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.462+0000 7f21d6b07b80 1 -- 172.21.15.63:0/605931929 shutdown_connections 2024-05-27T08:21:26.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.462+0000 7f21d6b07b80 1 -- 172.21.15.63:0/605931929 wait complete. 2024-05-27T08:21:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.462+0000 7f21d6b07b80 1 -- start start 2024-05-27T08:21:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.463+0000 7f21d6b07b80 1 Processor -- start 2024-05-27T08:21:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.463+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed467d3e40 0x55ed4695c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.463+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed4695ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.463+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46958d30 0x55ed4695cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.463+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ed46776fa0 con 0x55ed46952db0 2024-05-27T08:21:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.463+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed46811760 con 0x55ed46958d30 2024-05-27T08:21:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.463+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ed4695f7a0 con 0x55ed467d3e40 2024-05-27T08:21:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.463+0000 7f21d6b07b80 10 client.0 _ll_register_callbacks cb 0x55ed4681ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:21:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.463+0000 7f21c4c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed4695ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.464+0000 7f21c4c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed4695ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60302/0 (socket says 172.21.15.63:60302) 2024-05-27T08:21:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.464+0000 7f21c4c00700 1 -- 172.21.15.63:0/2471894872 learned_addr learned my addr 172.21.15.63:0/2471894872 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.464+0000 7f21c5600700 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed467d3e40 0x55ed4695c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.464+0000 7f21c6000700 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46958d30 0x55ed4695cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.464+0000 7f21c4c00700 1 -- 172.21.15.63:0/2471894872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed467d3e40 msgr2=0x55ed4695c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.464+0000 7f21c4c00700 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed467d3e40 0x55ed4695c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.464+0000 7f21c4c00700 1 -- 172.21.15.63:0/2471894872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46958d30 msgr2=0x55ed4695cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.464+0000 7f21c4c00700 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46958d30 0x55ed4695cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.464+0000 7f21c4c00700 1 -- 172.21.15.63:0/2471894872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed467c08d0 con 0x55ed46952db0 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.465+0000 7f21c4c00700 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed4695ca00 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00bac0 tx=0x7f21ac00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2251e800d565f825 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.465+0000 7f21a7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.465+0000 7f21a7400700 1 -- 172.21.15.63:0/2471894872 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac004830 con 0x55ed46952db0 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.465+0000 7f21a7400700 1 -- 172.21.15.63:0/2471894872 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(179..179 src has 1..179) ==== 9571+0+0 (secure 0 0 0) 0x7f21ac0049d0 con 0x55ed46952db0 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.465+0000 7f21d6b07b80 1 -- 172.21.15.63:0/2471894872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b402a030 con 0x55ed46952db0 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.465+0000 7f21a7400700 1 -- 172.21.15.63:0/2471894872 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f21ac023050 con 0x55ed46952db0 2024-05-27T08:21:26.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.465+0000 7f21d6b07b80 10 client.23883 fetch_fsmap learned FSMap version 46 2024-05-27T08:21:26.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.465+0000 7f21d6b07b80 1 -- 172.21.15.63:0/2471894872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ed46841060 con 0x55ed46952db0 2024-05-27T08:21:26.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.466+0000 7f21a7400700 1 -- 172.21.15.63:0/2471894872 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21b402a030 con 0x55ed46952db0 2024-05-27T08:21:26.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.466+0000 7f21a7400700 1 -- 172.21.15.63:0/2471894872 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 46) ==== 7144+0+0 (secure 0 0 0) 0x7f21ac019030 con 0x55ed46952db0 2024-05-27T08:21:26.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.466+0000 7f21d6b07b80 10 client.23883 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T08:21:26.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.466+0000 7f21d6b07b80 10 client.23883 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:21:26.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.466+0000 7f21d6b07b80 20 client.23883 populate_metadata read hostname 'smithi063' 2024-05-27T08:21:26.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.466+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed4696c7e0 0x55ed4696ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.466+0000 7f21d6b07b80 4 client.23883 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:21:26.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.466+0000 7f21d6b07b80 1 -- 172.21.15.63:0/2471894872 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed46882d90 con 0x55ed4696c7e0 2024-05-27T08:21:26.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.467+0000 7f21c5600700 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed4696c7e0 0x55ed4696ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.468+0000 7f21c5600700 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed4696c7e0 0x55ed4696ecb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96bb3a96e11c9ace server_cookie=d4b6dc630fa5c84c in_seq=0 out_seq=0 2024-05-27T08:21:26.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.468+0000 7f21a7400700 10 client.23883 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21a7400700 1 -- 172.21.15.63:0/2471894872 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ed46882d90 con 0x55ed4696c7e0 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21a7400700 10 client.23883 handle_command_reply: tid=0 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21d6b07b80 1 client.23883 shutdown 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21d6b07b80 1 -- 172.21.15.63:0/2471894872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 msgr2=0x55ed4695ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed4695ca00 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00bac0 tx=0x7f21ac00be80 comp rx=0 tx=0).stop 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21d6b07b80 1 -- 172.21.15.63:0/2471894872 shutdown_connections 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed46958d30 0x55ed4695cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed467d3e40 0x55ed4695c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed4696c7e0 0x55ed4696ecb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/2471894872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed4695ca00 unknown :-1 s=CLOSED pgs=3020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21a7400700 0 client.23883 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:21:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.470+0000 7f21d6b07b80 1 -- 172.21.15.63:0/2471894872 >> 172.21.15.63:0/2471894872 conn(0x55ed46833fc0 msgr2=0x55ed4694d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.471+0000 7f21d6b07b80 1 -- 172.21.15.63:0/2471894872 shutdown_connections 2024-05-27T08:21:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.471+0000 7f21d6b07b80 1 -- 172.21.15.63:0/2471894872 wait complete. 2024-05-27T08:21:26.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.471+0000 7f21d6b07b80 20 client.23883 trim_cache size 0 max 16384 2024-05-27T08:21:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.488+0000 7f21d6b07b80 1 Processor -- start 2024-05-27T08:21:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.488+0000 7f21d6b07b80 1 -- start start 2024-05-27T08:21:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.488+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 0x55ed46952870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46952db0 0x55ed4695caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46958d30 0x55ed46956b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01b3560 con 0x55ed46958d30 2024-05-27T08:21:26.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01b3c60 con 0x55ed467d3e40 2024-05-27T08:21:26.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01b4330 con 0x55ed46952db0 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21c6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46958d30 0x55ed46956b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21c6000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46958d30 0x55ed46956b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60310/0 (socket says 172.21.15.63:60310) 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21c6000700 1 -- 172.21.15.63:0/1969611195 learned_addr learned my addr 172.21.15.63:0/1969611195 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21c4c00700 1 --2- 172.21.15.63:0/1969611195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46952db0 0x55ed4695caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21c5600700 1 --2- 172.21.15.63:0/1969611195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 0x55ed46952870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21c6000700 1 -- 172.21.15.63:0/1969611195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46952db0 msgr2=0x55ed4695caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21c6000700 1 --2- 172.21.15.63:0/1969611195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46952db0 0x55ed4695caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21c6000700 1 -- 172.21.15.63:0/1969611195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 msgr2=0x55ed46952870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21c6000700 1 --2- 172.21.15.63:0/1969611195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 0x55ed46952870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.489+0000 7f21c6000700 1 -- 172.21.15.63:0/1969611195 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4695ed60 con 0x55ed46958d30 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.490+0000 7f21c6000700 1 --2- 172.21.15.63:0/1969611195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46958d30 0x55ed46956b70 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 crypto rx=0x7f21b80df5b0 tx=0x7f21b80a8b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6b3788c06aee6b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:26.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.490+0000 7f21b2000700 1 -- 172.21.15.63:0/1969611195 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b80a90e0 con 0x55ed46958d30 2024-05-27T08:21:26.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.490+0000 7f21b2000700 1 -- 172.21.15.63:0/1969611195 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21b80ad490 con 0x55ed46958d30 2024-05-27T08:21:26.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.490+0000 7f21b2000700 1 -- 172.21.15.63:0/1969611195 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b80a9340 con 0x55ed46958d30 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.491+0000 7f21d6b07b80 1 -- 172.21.15.63:0/1969611195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46958d30 msgr2=0x55ed46956b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.491+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/1969611195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46958d30 0x55ed46956b70 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 crypto rx=0x7f21b80df5b0 tx=0x7f21b80a8b30 comp rx=0 tx=0).stop 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.491+0000 7f21d6b07b80 1 -- 172.21.15.63:0/1969611195 shutdown_connections 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.491+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/1969611195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46958d30 0x55ed46956b70 unknown :-1 s=CLOSED pgs=3021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.491+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/1969611195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 0x55ed46952870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.491+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/1969611195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46952db0 0x55ed4695caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.491+0000 7f21d6b07b80 1 -- 172.21.15.63:0/1969611195 >> 172.21.15.63:0/1969611195 conn(0x55ed46833fc0 msgr2=0x55ed46959430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.491+0000 7f21d6b07b80 1 -- 172.21.15.63:0/1969611195 shutdown_connections 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.491+0000 7f21d6b07b80 1 -- 172.21.15.63:0/1969611195 wait complete. 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.492+0000 7f21d6b07b80 1 -- start start 2024-05-27T08:21:26.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.492+0000 7f21d6b07b80 1 Processor -- start 2024-05-27T08:21:26.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 0x55ed46965110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed46973ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21d6b07b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 0x55ed469741e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01b0820 con 0x55ed46952db0 2024-05-27T08:21:26.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01b1c10 con 0x55ed467d3e40 2024-05-27T08:21:26.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21d6b07b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01b0e70 con 0x55ed46958d30 2024-05-27T08:21:26.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21d6b07b80 10 client.0 _ll_register_callbacks cb 0x55ed4681ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c4c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed46973ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c4c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed46973ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60318/0 (socket says 172.21.15.63:60318) 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c4c00700 1 -- 172.21.15.63:0/612593066 learned_addr learned my addr 172.21.15.63:0/612593066 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c6000700 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 0x55ed469741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c5600700 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 0x55ed46965110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c4c00700 1 -- 172.21.15.63:0/612593066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 msgr2=0x55ed469741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c4c00700 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 0x55ed469741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c4c00700 1 -- 172.21.15.63:0/612593066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 msgr2=0x55ed46965110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c4c00700 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 0x55ed46965110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c4c00700 1 -- 172.21.15.63:0/612593066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed46974880 con 0x55ed46952db0 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c5600700 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 0x55ed46965110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:21:26.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.493+0000 7f21c6000700 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 0x55ed469741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:26.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.494+0000 7f21c4c00700 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed46973ca0 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 crypto rx=0x55ed4694f330 tx=0x7f21ac0041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11b49672875afa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:26.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.494+0000 7f21a7400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:21:26.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.494+0000 7f21a7400700 1 -- 172.21.15.63:0/612593066 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac010830 con 0x55ed46952db0 2024-05-27T08:21:26.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.494+0000 7f21a7400700 1 -- 172.21.15.63:0/612593066 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(179..179 src has 1..179) ==== 9571+0+0 (secure 0 0 0) 0x7f21ac020070 con 0x55ed46952db0 2024-05-27T08:21:26.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.494+0000 7f21d6b07b80 1 -- 172.21.15.63:0/612593066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b80ad490 con 0x55ed46952db0 2024-05-27T08:21:26.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21a7400700 1 -- 172.21.15.63:0/612593066 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f21ac010c40 con 0x55ed46952db0 2024-05-27T08:21:26.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21d6b07b80 10 client.23889 fetch_fsmap learned FSMap version 46 2024-05-27T08:21:26.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21d6b07b80 1 -- 172.21.15.63:0/612593066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ed469d71b0 con 0x55ed46952db0 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21a7400700 1 -- 172.21.15.63:0/612593066 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21b80ad490 con 0x55ed46952db0 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21a7400700 1 -- 172.21.15.63:0/612593066 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 46) ==== 7144+0+0 (secure 0 0 0) 0x7f21ac019070 con 0x55ed46952db0 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21d6b07b80 10 client.23889 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21d6b07b80 10 client.23889 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21d6b07b80 20 client.23889 populate_metadata read hostname 'smithi063' 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed469d7940 0x55ed469d9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21d6b07b80 4 client.23889 mds_command: new command op to 34454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.495+0000 7f21d6b07b80 1 -- 172.21.15.63:0/612593066 --> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed4687c570 con 0x55ed469d7940 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.496+0000 7f21c5600700 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed469d7940 0x55ed469d9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.497+0000 7f21c5600700 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed469d7940 0x55ed469d9d60 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=166ab22394000168 server_cookie=1a29162416352ee4 in_seq=0 out_seq=0 2024-05-27T08:21:26.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.497+0000 7f21a7400700 10 client.23889 ms_handle_connect on v2:172.21.15.86:6834/1494101824 2024-05-27T08:21:26.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.497+0000 7f21a7400700 1 -- 172.21.15.63:0/612593066 <== mds.2 v2:172.21.15.86:6834/1494101824 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed4687c570 con 0x55ed469d7940 2024-05-27T08:21:26.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.497+0000 7f21a7400700 10 client.23889 handle_command_reply: tid=0 2024-05-27T08:21:26.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.497+0000 7f21d6b07b80 1 client.23889 shutdown 2024-05-27T08:21:26.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 1 -- 172.21.15.63:0/612593066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 msgr2=0x55ed46973ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:26.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed46973ca0 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 crypto rx=0x55ed4694f330 tx=0x7f21ac0041a0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 1 -- 172.21.15.63:0/612593066 shutdown_connections 2024-05-27T08:21:26.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ed46958d30 0x55ed469741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed467d3e40 0x55ed46965110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.86:6834/1494101824,v1:172.21.15.86:6835/1494101824] conn(0x55ed469d7940 0x55ed469d9d60 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 1 --2- 172.21.15.63:0/612593066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed46952db0 0x55ed46973ca0 unknown :-1 s=CLOSED pgs=3022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 1 -- 172.21.15.63:0/612593066 >> 172.21.15.63:0/612593066 conn(0x55ed46833fc0 msgr2=0x55ed469568d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:26.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21a7400700 0 client.23889 ms_handle_reset on v2:172.21.15.86:6834/1494101824 2024-05-27T08:21:26.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 1 -- 172.21.15.63:0/612593066 shutdown_connections 2024-05-27T08:21:26.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 1 -- 172.21.15.63:0/612593066 wait complete. 2024-05-27T08:21:26.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.498+0000 7f21d6b07b80 20 client.23889 trim_cache size 0 max 16384 2024-05-27T08:21:26.501 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:21:26.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 -- 172.21.15.63:0/437607070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21a80b29f0 msgr2=0x7f21a80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:26.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21a80b29f0 0x7f21a80b4f00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f21b4006010 tx=0x7f21b400b540 comp rx=0 tx=0).stop 2024-05-27T08:21:26.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 -- 172.21.15.63:0/437607070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21c010c5b0 msgr2=0x7f21c01a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:26.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21c010c5b0 0x7f21c01a17a0 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 crypto rx=0x7f21b8007ee0 tx=0x7f21b800d3b0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 -- 172.21.15.63:0/437607070 shutdown_connections 2024-05-27T08:21:26.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f21a80b29f0 0x7f21a80b4f00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21c010c5b0 0x7f21c01a17a0 unknown :-1 s=CLOSED pgs=3018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c0106570 0x7f21c019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 --2- 172.21.15.63:0/437607070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c0106ec0 0x7f21c019d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:26.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 -- 172.21.15.63:0/437607070 >> 172.21.15.63:0/437607070 conn(0x7f21c0102070 msgr2=0x7f21c0104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:26.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 -- 172.21.15.63:0/437607070 shutdown_connections 2024-05-27T08:21:26.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:26.499+0000 7f21c8000700 1 -- 172.21.15.63:0/437607070 wait complete. 2024-05-27T08:21:26.538 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:26 smithi149 podman[81247]: 2024-05-27 08:21:26.427820987 +0000 UTC m=+0.431909220 container init 0760db8717cda5a4ea09a3f6aa9ba09320d8c7e4613b7f417fec5389a90f63f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-05-27T08:21:26.539 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:26 smithi149 podman[81247]: 2024-05-27 08:21:26.438256809 +0000 UTC m=+0.442345046 container start 0760db8717cda5a4ea09a3f6aa9ba09320d8c7e4613b7f417fec5389a90f63f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-27T08:21:26.539 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:26 smithi149 bash[81247]: 0760db8717cda5a4ea09a3f6aa9ba09320d8c7e4613b7f417fec5389a90f63f8 2024-05-27T08:21:26.539 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:26 smithi149 ceph-mon[35408]: pgmap v1524: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.7 KiB/s wr, 20 op/s 2024-05-27T08:21:26.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:26 smithi086 ceph-mon[34687]: pgmap v1524: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.7 KiB/s wr, 20 op/s 2024-05-27T08:21:26.823 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:26 smithi149 systemd[1]: Started Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:26.823 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:26 smithi149 ceph-mds[81299]: -- 172.21.15.149:0/994114394 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c96d5fa40 con 0x561c96da7a80 2024-05-27T08:21:27.099 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:21:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: starting mds.l at 2024-05-27T08:21:27.443 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:21:27.463 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:27 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 2. 2024-05-27T08:21:27.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:27 smithi149 systemd[1]: Stopped Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:27.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:27 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.464s CPU time. 2024-05-27T08:21:27.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:27 smithi149 systemd[1]: Starting Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:27.464 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:27 smithi149 podman[81329]: 2024-05-27 08:21:27.00316481 +0000 UTC m=+0.034186382 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:27.465 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:27 smithi149 podman[81329]: 2024-05-27 08:21:27.303189574 +0000 UTC m=+0.334211185 container create a4ec82c5e1e9cdeaf1b6e523b1e14e1c6d5e04d85bde5e33b91363b33c1fa4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:21:27.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:27 smithi063 ceph-mon[31613]: osdmap e180: 12 total, 12 up, 12 in 2024-05-27T08:21:27.717 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:27 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Scheduled restart job, restart counter is at 1. 2024-05-27T08:21:27.717 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:27 smithi063 systemd[1]: Stopped Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:27.717 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:27 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Consumed 3.213s CPU time. 2024-05-27T08:21:27.717 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:27 smithi063 systemd[1]: Starting Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:27.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:27 smithi149 ceph-mon[35408]: osdmap e180: 12 total, 12 up, 12 in 2024-05-27T08:21:27.821 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:27 smithi149 podman[81329]: 2024-05-27 08:21:27.556651438 +0000 UTC m=+0.587673089 container init a4ec82c5e1e9cdeaf1b6e523b1e14e1c6d5e04d85bde5e33b91363b33c1fa4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:21:27.821 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:27 smithi149 podman[81329]: 2024-05-27 08:21:27.566242789 +0000 UTC m=+0.597264375 container start a4ec82c5e1e9cdeaf1b6e523b1e14e1c6d5e04d85bde5e33b91363b33c1fa4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:21:27.821 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:27 smithi149 bash[81329]: a4ec82c5e1e9cdeaf1b6e523b1e14e1c6d5e04d85bde5e33b91363b33c1fa4bf 2024-05-27T08:21:27.821 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:27 smithi149 systemd[1]: Started Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:27.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:27 smithi086 ceph-mon[34687]: osdmap e180: 12 total, 12 up, 12 in 2024-05-27T08:21:28.145 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:27 smithi063 podman[188258]: 2024-05-27 08:21:27.768377385 +0000 UTC m=+0.033866140 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:28.145 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:27 smithi063 podman[188258]: 2024-05-27 08:21:27.889910338 +0000 UTC m=+0.155399135 container create 31cf44daa14568ef7d41ed4c30cf6d98648bd152d9be711d1ad1d28dbee328be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS) 2024-05-27T08:21:28.145 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:28 smithi063 podman[188258]: 2024-05-27 08:21:28.143717507 +0000 UTC m=+0.409206317 container init 31cf44daa14568ef7d41ed4c30cf6d98648bd152d9be711d1ad1d28dbee328be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T08:21:28.213 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:27 smithi149 ceph-mds[81387]: -- 172.21.15.149:0/714384746 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557621e2fa40 con 0x557621e77600 2024-05-27T08:21:28.214 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:27 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: starting mds.f at 2024-05-27T08:21:28.214 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:27 smithi149 podman[81398]: 2024-05-27 08:21:27.857158813 +0000 UTC m=+0.109178343 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:28.375 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:21:28.445 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:28 smithi063 podman[188258]: 2024-05-27 08:21:28.155921962 +0000 UTC m=+0.421410785 container start 31cf44daa14568ef7d41ed4c30cf6d98648bd152d9be711d1ad1d28dbee328be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T08:21:28.445 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:28 smithi063 bash[188258]: 31cf44daa14568ef7d41ed4c30cf6d98648bd152d9be711d1ad1d28dbee328be 2024-05-27T08:21:28.445 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:28 smithi063 systemd[1]: Started Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:28.716 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:28 smithi063 ceph-mds[188300]: -- 172.21.15.63:0/2661569243 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c9aa0e5a40 con 0x55c9aa12d180 2024-05-27T08:21:28.717 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:21:28 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: starting mds.a at 2024-05-27T08:21:28.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:28 smithi063 ceph-mon[31613]: pgmap v1526: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-05-27T08:21:28.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:28 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6832/3930508187,v1:172.21.15.149:6833/3930508187] up:boot 2024-05-27T08:21:28.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:28 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:28.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:28 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:21:28.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:28 smithi063 ceph-mon[31613]: daemon mds.a restarted 2024-05-27T08:21:28.795 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:28 smithi149 ceph-mon[35408]: pgmap v1526: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-05-27T08:21:28.795 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:28 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6832/3930508187,v1:172.21.15.149:6833/3930508187] up:boot 2024-05-27T08:21:28.795 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:28 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:28.795 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:28 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:21:28.795 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:28 smithi149 ceph-mon[35408]: daemon mds.a restarted 2024-05-27T08:21:28.795 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:28 smithi149 podman[81398]: 2024-05-27 08:21:28.516146063 +0000 UTC m=+0.768165617 container create 68987e8de46dad91e42c1c6b21823094f37cf8449a5d3b4dbfc2bfe4119ca81b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-27T08:21:28.795 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:28 smithi149 podman[81398]: 2024-05-27 08:21:28.794168429 +0000 UTC m=+1.046187980 container init 68987e8de46dad91e42c1c6b21823094f37cf8449a5d3b4dbfc2bfe4119ca81b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:21:28.796 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fba45200700 time 2024-05-27T08:21:28.703590+0000 2024-05-27T08:21:28.796 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:28.796 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:28.796 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fba5ac2fba6] 2024-05-27T08:21:28.796 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fba5ac2fdd1] 2024-05-27T08:21:28.796 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5575ecb5f23d] 2024-05-27T08:21:28.796 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5575ecb4a971] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 5: (SnapRealm::check_cache() const+0x13e) [0x5575ecb48fa0] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5575ecb49651] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5575ecade347] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5575ec7da87e] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5575ec7dfe94] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5575ec7ea969] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 11: (MDLog::_replay_thread()+0x1f38) [0x5575ecb8348c] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5575ecb8c9cb] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 13: (Thread::entry_wrapper()+0x43) [0x7fba5abfe72d] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 14: (Thread::_entry_func(void*)+0xd) [0x7fba5abfe749] 2024-05-27T08:21:28.797 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fba597a21ca] 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 16: clone() 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.704+0000 7fba45200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fba45200700 time 2024-05-27T08:21:28.703590+0000 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fba5ac2fba6] 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fba5ac2fdd1] 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5575ecb5f23d] 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5575ecb4a971] 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 5: (SnapRealm::check_cache() const+0x13e) [0x5575ecb48fa0] 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5575ecb49651] 2024-05-27T08:21:28.798 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5575ecade347] 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5575ec7da87e] 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5575ec7dfe94] 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5575ec7ea969] 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 11: (MDLog::_replay_thread()+0x1f38) [0x5575ecb8348c] 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5575ecb8c9cb] 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 13: (Thread::entry_wrapper()+0x43) [0x7fba5abfe72d] 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 14: (Thread::_entry_func(void*)+0xd) [0x7fba5abfe749] 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fba597a21ca] 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 16: clone() 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.799 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: *** Caught signal (Aborted) ** 2024-05-27T08:21:28.800 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: in thread 7fba45200700 thread_name:md_log_replay 2024-05-27T08:21:28.800 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:28.800 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fba597acd20] 2024-05-27T08:21:28.800 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2: gsignal() 2024-05-27T08:21:28.800 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 3: abort() 2024-05-27T08:21:28.800 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fba5ac2fced] 2024-05-27T08:21:28.800 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fba5ac2fdd1] 2024-05-27T08:21:28.801 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5575ecb5f23d] 2024-05-27T08:21:28.801 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5575ecb4a971] 2024-05-27T08:21:28.801 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 8: (SnapRealm::check_cache() const+0x13e) [0x5575ecb48fa0] 2024-05-27T08:21:28.801 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5575ecb49651] 2024-05-27T08:21:28.801 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5575ecade347] 2024-05-27T08:21:28.801 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5575ec7da87e] 2024-05-27T08:21:28.801 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5575ec7dfe94] 2024-05-27T08:21:28.801 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5575ec7ea969] 2024-05-27T08:21:28.801 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 14: (MDLog::_replay_thread()+0x1f38) [0x5575ecb8348c] 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5575ecb8c9cb] 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 16: (Thread::entry_wrapper()+0x43) [0x7fba5abfe72d] 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 17: (Thread::_entry_func(void*)+0xd) [0x7fba5abfe749] 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fba597a21ca] 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 19: clone() 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.707+0000 7fba45200700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: in thread 7fba45200700 thread_name:md_log_replay 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fba597acd20] 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2: gsignal() 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 3: abort() 2024-05-27T08:21:28.802 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fba5ac2fced] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fba5ac2fdd1] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5575ecb5f23d] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5575ecb4a971] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 8: (SnapRealm::check_cache() const+0x13e) [0x5575ecb48fa0] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5575ecb49651] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5575ecade347] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5575ec7da87e] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5575ec7dfe94] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5575ec7ea969] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 14: (MDLog::_replay_thread()+0x1f38) [0x5575ecb8348c] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5575ecb8c9cb] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 16: (Thread::entry_wrapper()+0x43) [0x7fba5abfe72d] 2024-05-27T08:21:28.803 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 17: (Thread::_entry_func(void*)+0xd) [0x7fba5abfe749] 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fba597a21ca] 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 19: clone() 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: -1> 2024-05-27T08:21:28.704+0000 7fba45200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fba45200700 time 2024-05-27T08:21:28.703590+0000 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fba5ac2fba6] 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fba5ac2fdd1] 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5575ecb5f23d] 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5575ecb4a971] 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 5: (SnapRealm::check_cache() const+0x13e) [0x5575ecb48fa0] 2024-05-27T08:21:28.804 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5575ecb49651] 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5575ecade347] 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5575ec7da87e] 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5575ec7dfe94] 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5575ec7ea969] 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 11: (MDLog::_replay_thread()+0x1f38) [0x5575ecb8348c] 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5575ecb8c9cb] 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 13: (Thread::entry_wrapper()+0x43) [0x7fba5abfe72d] 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 14: (Thread::_entry_func(void*)+0xd) [0x7fba5abfe749] 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fba597a21ca] 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 16: clone() 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 0> 2024-05-27T08:21:28.707+0000 7fba45200700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:28.805 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: in thread 7fba45200700 thread_name:md_log_replay 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fba597acd20] 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2: gsignal() 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 3: abort() 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fba5ac2fced] 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fba5ac2fdd1] 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5575ecb5f23d] 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5575ecb4a971] 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 8: (SnapRealm::check_cache() const+0x13e) [0x5575ecb48fa0] 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5575ecb49651] 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5575ecade347] 2024-05-27T08:21:28.806 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5575ec7da87e] 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5575ec7dfe94] 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5575ec7ea969] 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 14: (MDLog::_replay_thread()+0x1f38) [0x5575ecb8348c] 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5575ecb8c9cb] 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 16: (Thread::entry_wrapper()+0x43) [0x7fba5abfe72d] 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 17: (Thread::_entry_func(void*)+0xd) [0x7fba5abfe749] 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fba597a21ca] 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 19: clone() 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: -1> 2024-05-27T08:21:28.704+0000 7fba45200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fba45200700 time 2024-05-27T08:21:28.703590+0000 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:28.807 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fba5ac2fba6] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fba5ac2fdd1] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5575ecb5f23d] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5575ecb4a971] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 5: (SnapRealm::check_cache() const+0x13e) [0x5575ecb48fa0] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5575ecb49651] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5575ecade347] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5575ec7da87e] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5575ec7dfe94] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5575ec7ea969] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 11: (MDLog::_replay_thread()+0x1f38) [0x5575ecb8348c] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5575ecb8c9cb] 2024-05-27T08:21:28.808 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 13: (Thread::entry_wrapper()+0x43) [0x7fba5abfe72d] 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 14: (Thread::_entry_func(void*)+0xd) [0x7fba5abfe749] 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fba597a21ca] 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 16: clone() 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 0> 2024-05-27T08:21:28.707+0000 7fba45200700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: in thread 7fba45200700 thread_name:md_log_replay 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fba597acd20] 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2: gsignal() 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 3: abort() 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fba5ac2fced] 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fba5ac2fdd1] 2024-05-27T08:21:28.809 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5575ecb5f23d] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5575ecb4a971] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 8: (SnapRealm::check_cache() const+0x13e) [0x5575ecb48fa0] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5575ecb49651] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5575ecade347] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5575ec7da87e] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5575ec7dfe94] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5575ec7ea969] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 14: (MDLog::_replay_thread()+0x1f38) [0x5575ecb8348c] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5575ecb8c9cb] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 16: (Thread::entry_wrapper()+0x43) [0x7fba5abfe72d] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 17: (Thread::_entry_func(void*)+0xd) [0x7fba5abfe749] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fba597a21ca] 2024-05-27T08:21:28.810 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 19: clone() 2024-05-27T08:21:28.811 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:28.811 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81364]: 2024-05-27T08:21:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:28 smithi086 ceph-mon[34687]: pgmap v1526: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-05-27T08:21:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:28 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6832/3930508187,v1:172.21.15.149:6833/3930508187] up:boot 2024-05-27T08:21:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:28 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:21:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:28 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:21:28.961 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:28 smithi086 ceph-mon[34687]: daemon mds.a restarted 2024-05-27T08:21:29.156 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:29 smithi149 podman[81445]: 2024-05-27 08:21:29.020438729 +0000 UTC m=+0.242299241 container died a4ec82c5e1e9cdeaf1b6e523b1e14e1c6d5e04d85bde5e33b91363b33c1fa4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-27T08:21:29.156 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:28 smithi149 podman[81398]: 2024-05-27 08:21:28.806229666 +0000 UTC m=+1.058249183 container start 68987e8de46dad91e42c1c6b21823094f37cf8449a5d3b4dbfc2bfe4119ca81b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-27T08:21:29.156 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:28 smithi149 bash[81398]: 68987e8de46dad91e42c1c6b21823094f37cf8449a5d3b4dbfc2bfe4119ca81b 2024-05-27T08:21:29.156 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:28 smithi149 systemd[1]: Started Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:29.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-mds[81454]: -- 172.21.15.149:0/1405131534 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee676b9a40 con 0x55ee67701180 2024-05-27T08:21:29.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: starting mds.c at 2024-05-27T08:21:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:29 smithi063 ceph-mon[31613]: osdmap e181: 12 total, 12 up, 12 in 2024-05-27T08:21:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:29 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/3519543261,v1:172.21.15.149:6835/3519543261] up:boot 2024-05-27T08:21:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:29 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.63:6840/717564106,v1:172.21.15.63:6841/717564106] up:boot 2024-05-27T08:21:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:29 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:29 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:21:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:29 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:21:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:29 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:29.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:29 smithi063 ceph-mon[31613]: daemon mds.c restarted 2024-05-27T08:21:29.820 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:29 smithi149 podman[81445]: 2024-05-27 08:21:29.715964474 +0000 UTC m=+0.937824971 container remove a4ec82c5e1e9cdeaf1b6e523b1e14e1c6d5e04d85bde5e33b91363b33c1fa4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:21:29.820 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:29 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:29.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-mon[35408]: osdmap e181: 12 total, 12 up, 12 in 2024-05-27T08:21:29.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/3519543261,v1:172.21.15.149:6835/3519543261] up:boot 2024-05-27T08:21:29.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.63:6840/717564106,v1:172.21.15.63:6841/717564106] up:boot 2024-05-27T08:21:29.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:29.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:21:29.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:21:29.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:29.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-mon[35408]: daemon mds.c restarted 2024-05-27T08:21:29.822 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa8c6200700 time 2024-05-27T08:21:29.729243+0000 2024-05-27T08:21:29.822 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:29.822 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:29.822 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa8dbcefba6] 2024-05-27T08:21:29.822 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa8dbcefdd1] 2024-05-27T08:21:29.822 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ee6069e23d] 2024-05-27T08:21:29.822 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55ee60689971] 2024-05-27T08:21:29.822 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 5: (SnapRealm::check_cache() const+0x13e) [0x55ee60687fa0] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ee60688651] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ee6061d347] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ee6031987e] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55ee6031ef2f] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55ee60329969] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 11: (MDLog::_replay_thread()+0x1f38) [0x55ee606c248c] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55ee606cb9cb] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 13: (Thread::entry_wrapper()+0x43) [0x7fa8dbcbe72d] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa8dbcbe749] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa8da8621ca] 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 16: clone() 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: *** Caught signal (Aborted) ** 2024-05-27T08:21:29.823 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: in thread 7fa8c6200700 thread_name:md_log_replay 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.730+0000 7fa8c6200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa8c6200700 time 2024-05-27T08:21:29.729243+0000 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa8dbcefba6] 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa8dbcefdd1] 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ee6069e23d] 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55ee60689971] 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 5: (SnapRealm::check_cache() const+0x13e) [0x55ee60687fa0] 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ee60688651] 2024-05-27T08:21:29.824 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ee6061d347] 2024-05-27T08:21:29.825 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ee6031987e] 2024-05-27T08:21:29.825 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55ee6031ef2f] 2024-05-27T08:21:29.825 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55ee60329969] 2024-05-27T08:21:29.825 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 11: (MDLog::_replay_thread()+0x1f38) [0x55ee606c248c] 2024-05-27T08:21:29.825 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55ee606cb9cb] 2024-05-27T08:21:29.825 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 13: (Thread::entry_wrapper()+0x43) [0x7fa8dbcbe72d] 2024-05-27T08:21:29.825 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa8dbcbe749] 2024-05-27T08:21:29.825 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa8da8621ca] 2024-05-27T08:21:29.825 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 16: clone() 2024-05-27T08:21:29.825 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.826 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:29.826 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa8da86cd20] 2024-05-27T08:21:29.826 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2: gsignal() 2024-05-27T08:21:29.826 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 3: abort() 2024-05-27T08:21:29.826 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa8dbcefced] 2024-05-27T08:21:29.826 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa8dbcefdd1] 2024-05-27T08:21:29.826 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ee6069e23d] 2024-05-27T08:21:29.827 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55ee60689971] 2024-05-27T08:21:29.827 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 8: (SnapRealm::check_cache() const+0x13e) [0x55ee60687fa0] 2024-05-27T08:21:29.827 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ee60688651] 2024-05-27T08:21:29.827 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ee6061d347] 2024-05-27T08:21:29.827 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ee6031987e] 2024-05-27T08:21:29.827 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55ee6031ef2f] 2024-05-27T08:21:29.827 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55ee60329969] 2024-05-27T08:21:29.827 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 14: (MDLog::_replay_thread()+0x1f38) [0x55ee606c248c] 2024-05-27T08:21:29.827 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55ee606cb9cb] 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 16: (Thread::entry_wrapper()+0x43) [0x7fa8dbcbe72d] 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa8dbcbe749] 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa8da8621ca] 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 19: clone() 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.735+0000 7fa8c6200700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: in thread 7fa8c6200700 thread_name:md_log_replay 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa8da86cd20] 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2: gsignal() 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 3: abort() 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa8dbcefced] 2024-05-27T08:21:29.828 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa8dbcefdd1] 2024-05-27T08:21:29.829 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ee6069e23d] 2024-05-27T08:21:29.829 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55ee60689971] 2024-05-27T08:21:29.829 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 8: (SnapRealm::check_cache() const+0x13e) [0x55ee60687fa0] 2024-05-27T08:21:29.829 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ee60688651] 2024-05-27T08:21:29.829 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ee6061d347] 2024-05-27T08:21:29.830 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ee6031987e] 2024-05-27T08:21:29.830 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55ee6031ef2f] 2024-05-27T08:21:29.830 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55ee60329969] 2024-05-27T08:21:29.830 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 14: (MDLog::_replay_thread()+0x1f38) [0x55ee606c248c] 2024-05-27T08:21:29.830 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55ee606cb9cb] 2024-05-27T08:21:29.830 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 16: (Thread::entry_wrapper()+0x43) [0x7fa8dbcbe72d] 2024-05-27T08:21:29.830 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa8dbcbe749] 2024-05-27T08:21:29.831 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa8da8621ca] 2024-05-27T08:21:29.831 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 19: clone() 2024-05-27T08:21:29.831 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:29.831 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.831 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: -1> 2024-05-27T08:21:29.730+0000 7fa8c6200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa8c6200700 time 2024-05-27T08:21:29.729243+0000 2024-05-27T08:21:29.831 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:29.831 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.831 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:29.831 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa8dbcefba6] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa8dbcefdd1] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ee6069e23d] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55ee60689971] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 5: (SnapRealm::check_cache() const+0x13e) [0x55ee60687fa0] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ee60688651] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ee6061d347] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ee6031987e] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55ee6031ef2f] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55ee60329969] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 11: (MDLog::_replay_thread()+0x1f38) [0x55ee606c248c] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55ee606cb9cb] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 13: (Thread::entry_wrapper()+0x43) [0x7fa8dbcbe72d] 2024-05-27T08:21:29.832 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa8dbcbe749] 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa8da8621ca] 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 16: clone() 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 0> 2024-05-27T08:21:29.735+0000 7fa8c6200700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: in thread 7fa8c6200700 thread_name:md_log_replay 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa8da86cd20] 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2: gsignal() 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 3: abort() 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa8dbcefced] 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa8dbcefdd1] 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ee6069e23d] 2024-05-27T08:21:29.833 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55ee60689971] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 8: (SnapRealm::check_cache() const+0x13e) [0x55ee60687fa0] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ee60688651] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ee6061d347] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ee6031987e] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55ee6031ef2f] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55ee60329969] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 14: (MDLog::_replay_thread()+0x1f38) [0x55ee606c248c] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55ee606cb9cb] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 16: (Thread::entry_wrapper()+0x43) [0x7fa8dbcbe72d] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa8dbcbe749] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa8da8621ca] 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 19: clone() 2024-05-27T08:21:29.834 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:29.835 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.835 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: -1> 2024-05-27T08:21:29.730+0000 7fa8c6200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa8c6200700 time 2024-05-27T08:21:29.729243+0000 2024-05-27T08:21:29.835 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:29.835 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.835 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:29.835 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa8dbcefba6] 2024-05-27T08:21:29.835 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa8dbcefdd1] 2024-05-27T08:21:29.835 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ee6069e23d] 2024-05-27T08:21:29.836 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55ee60689971] 2024-05-27T08:21:29.836 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 5: (SnapRealm::check_cache() const+0x13e) [0x55ee60687fa0] 2024-05-27T08:21:29.836 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ee60688651] 2024-05-27T08:21:29.836 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ee6061d347] 2024-05-27T08:21:29.836 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ee6031987e] 2024-05-27T08:21:29.836 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55ee6031ef2f] 2024-05-27T08:21:29.836 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55ee60329969] 2024-05-27T08:21:29.836 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 11: (MDLog::_replay_thread()+0x1f38) [0x55ee606c248c] 2024-05-27T08:21:29.836 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55ee606cb9cb] 2024-05-27T08:21:29.836 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 13: (Thread::entry_wrapper()+0x43) [0x7fa8dbcbe72d] 2024-05-27T08:21:29.837 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa8dbcbe749] 2024-05-27T08:21:29.837 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa8da8621ca] 2024-05-27T08:21:29.837 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 16: clone() 2024-05-27T08:21:29.837 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.837 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 0> 2024-05-27T08:21:29.735+0000 7fa8c6200700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:29.837 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: in thread 7fa8c6200700 thread_name:md_log_replay 2024-05-27T08:21:29.837 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.837 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:29.837 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa8da86cd20] 2024-05-27T08:21:29.837 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2: gsignal() 2024-05-27T08:21:29.838 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 3: abort() 2024-05-27T08:21:29.838 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa8dbcefced] 2024-05-27T08:21:29.838 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa8dbcefdd1] 2024-05-27T08:21:29.838 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ee6069e23d] 2024-05-27T08:21:29.838 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55ee60689971] 2024-05-27T08:21:29.838 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 8: (SnapRealm::check_cache() const+0x13e) [0x55ee60687fa0] 2024-05-27T08:21:29.838 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ee60688651] 2024-05-27T08:21:29.838 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ee6061d347] 2024-05-27T08:21:29.839 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ee6031987e] 2024-05-27T08:21:29.839 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55ee6031ef2f] 2024-05-27T08:21:29.839 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55ee60329969] 2024-05-27T08:21:29.839 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 14: (MDLog::_replay_thread()+0x1f38) [0x55ee606c248c] 2024-05-27T08:21:29.839 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55ee606cb9cb] 2024-05-27T08:21:29.839 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 16: (Thread::entry_wrapper()+0x43) [0x7fa8dbcbe72d] 2024-05-27T08:21:29.839 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa8dbcbe749] 2024-05-27T08:21:29.839 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa8da8621ca] 2024-05-27T08:21:29.839 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 19: clone() 2024-05-27T08:21:29.839 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:29.840 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81431]: 2024-05-27T08:21:29.840 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:29 smithi149 podman[81498]: 2024-05-27 08:21:29.820114687 +0000 UTC m=+0.036017699 container died 68987e8de46dad91e42c1c6b21823094f37cf8449a5d3b4dbfc2bfe4119ca81b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:21:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:29 smithi086 ceph-mon[34687]: osdmap e181: 12 total, 12 up, 12 in 2024-05-27T08:21:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:29 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/3519543261,v1:172.21.15.149:6835/3519543261] up:boot 2024-05-27T08:21:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:29 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.63:6840/717564106,v1:172.21.15.63:6841/717564106] up:boot 2024-05-27T08:21:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:29 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:29 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:21:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:29 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:21:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:29 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:29 smithi086 ceph-mon[34687]: daemon mds.c restarted 2024-05-27T08:21:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.722+0000 7f8ae3e00700 1 -- 172.21.15.63:0/111682782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8adc107de0 msgr2=0x7f8adc1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.722+0000 7f8ae3e00700 1 --2- 172.21.15.63:0/111682782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8adc107de0 0x7f8adc1081f0 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80099c0 tx=0x7f8ac8009cd0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.722+0000 7f8ae3e00700 1 -- 172.21.15.63:0/111682782 shutdown_connections 2024-05-27T08:21:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.722+0000 7f8ae3e00700 1 --2- 172.21.15.63:0/111682782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8adc108730 0x7f8adc110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.722+0000 7f8ae3e00700 1 --2- 172.21.15.63:0/111682782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc1034b0 0x7f8adc1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.722+0000 7f8ae3e00700 1 --2- 172.21.15.63:0/111682782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8adc107de0 0x7f8adc1081f0 unknown :-1 s=CLOSED pgs=3025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.722+0000 7f8ae3e00700 1 -- 172.21.15.63:0/111682782 >> 172.21.15.63:0/111682782 conn(0x7f8adc0fedb0 msgr2=0x7f8adc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:30.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.722+0000 7f8ae3e00700 1 -- 172.21.15.63:0/111682782 shutdown_connections 2024-05-27T08:21:30.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.722+0000 7f8ae3e00700 1 -- 172.21.15.63:0/111682782 wait complete. 2024-05-27T08:21:30.724 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:30 smithi063 ceph-mon[31613]: pgmap v1528: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-05-27T08:21:30.724 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:30 smithi063 ceph-mon[31613]: osdmap e182: 12 total, 12 up, 12 in 2024-05-27T08:21:30.724 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:30 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/4071622084,v1:172.21.15.149:6835/4071622084] up:boot 2024-05-27T08:21:30.724 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:30 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:30.724 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:30 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:30.724 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:30 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.724+0000 7f8ae3e00700 1 Processor -- start 2024-05-27T08:21:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.724+0000 7f8ae3e00700 1 -- start start 2024-05-27T08:21:30.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.724+0000 7f8ae3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc1034b0 0x7f8adc1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.725+0000 7f8ae3e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8adc107de0 0x7f8adc1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.725+0000 7f8ae3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8adc108730 0x7f8adc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.725+0000 7f8ae3e00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc1142f0 con 0x7f8adc107de0 2024-05-27T08:21:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.725+0000 7f8ae3e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc114170 con 0x7f8adc108730 2024-05-27T08:21:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.725+0000 7f8ae3e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc114470 con 0x7f8adc1034b0 2024-05-27T08:21:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.725+0000 7f8ae1400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc1034b0 0x7f8adc1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.725+0000 7f8ae0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8adc107de0 0x7f8adc1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.725+0000 7f8ae1400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc1034b0 0x7f8adc1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41466/0 (socket says 172.21.15.63:41466) 2024-05-27T08:21:30.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.725+0000 7f8ae1400700 1 -- 172.21.15.63:0/4220958608 learned_addr learned my addr 172.21.15.63:0/4220958608 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.725+0000 7f8ae1e00700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8adc108730 0x7f8adc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.726+0000 7f8ae1400700 1 -- 172.21.15.63:0/4220958608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8adc108730 msgr2=0x7f8adc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.726+0000 7f8ae1400700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8adc108730 0x7f8adc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.726+0000 7f8ae1400700 1 -- 172.21.15.63:0/4220958608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8adc107de0 msgr2=0x7f8adc1a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.726+0000 7f8ae1400700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8adc107de0 0x7f8adc1a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.726+0000 7f8ae1400700 1 -- 172.21.15.63:0/4220958608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc19b260 con 0x7f8adc1034b0 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.726+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8adc107de0 0x7f8adc1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.726+0000 7f8ae1400700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc1034b0 0x7f8adc1a09a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000e9a0 tx=0x7f8ad000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.726+0000 7f8ada000700 1 -- 172.21.15.63:0/4220958608 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000cbd0 con 0x7f8adc1034b0 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.726+0000 7f8ae3e00700 1 -- 172.21.15.63:0/4220958608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8adc19b550 con 0x7f8adc1034b0 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.727+0000 7f8ae3e00700 1 -- 172.21.15.63:0/4220958608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8adc1a7750 con 0x7f8adc1034b0 2024-05-27T08:21:30.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.727+0000 7f8ada000700 1 -- 172.21.15.63:0/4220958608 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ad000cd70 con 0x7f8adc1034b0 2024-05-27T08:21:30.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.727+0000 7f8ada000700 1 -- 172.21.15.63:0/4220958608 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0018680 con 0x7f8adc1034b0 2024-05-27T08:21:30.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.730+0000 7f8ada000700 1 -- 172.21.15.63:0/4220958608 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8ad0010430 con 0x7f8adc1034b0 2024-05-27T08:21:30.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.732+0000 7f8af2a0ab80 1 Processor -- start 2024-05-27T08:21:30.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.732+0000 7f8af2a0ab80 1 -- start start 2024-05-27T08:21:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.732+0000 7f8ada000700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ac40b29f0 0x7f8ac40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.732+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c490230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.732+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 0x563f7c490770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.732+0000 7f8ada000700 1 -- 172.21.15.63:0/4220958608 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 9395+0+0 (secure 0 0 0) 0x7f8ad00e2b60 con 0x7f8adc1034b0 2024-05-27T08:21:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.732+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c4a1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.732+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563f7c3bee40 con 0x563f7c3fd9f0 2024-05-27T08:21:30.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.732+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f7c3ae9e0 con 0x563f7c490d00 2024-05-27T08:21:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.732+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563f7c3b2a20 con 0x563f7c311e40 2024-05-27T08:21:30.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.733+0000 7f8ae1e00700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ac40b29f0 0x7f8ac40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.733+0000 7f8ae0a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 0x563f7c490770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.733+0000 7f8ae1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c490230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.733+0000 7f8ae1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c4a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.733+0000 7f8ae0a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 0x563f7c490770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41468/0 (socket says 172.21.15.63:41468) 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.733+0000 7f8ae0a00700 1 -- 172.21.15.63:0/1902013035 learned_addr learned my addr 172.21.15.63:0/1902013035 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.734+0000 7f8ae0a00700 1 -- 172.21.15.63:0/1902013035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 msgr2=0x563f7c4a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.734+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/1902013035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c4a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.734+0000 7f8ae0a00700 1 -- 172.21.15.63:0/1902013035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 msgr2=0x563f7c490230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.734+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/1902013035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c490230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.734+0000 7f8ae0a00700 1 -- 172.21.15.63:0/1902013035 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f7c377420 con 0x563f7c311e40 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.734+0000 7f8ae1400700 1 --2- 172.21.15.63:0/1902013035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c490230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:21:30.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.734+0000 7f8ae1e00700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ac40b29f0 0x7f8ac40b4f00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009fd0 tx=0x7f8ad40079a0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.734+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/1902013035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 0x563f7c490770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8adc076aa0 tx=0x7f8ac8005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6b4e5dd21edf59b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:30.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.734+0000 7f8acea00700 1 -- 172.21.15.63:0/1902013035 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8027070 con 0x563f7c311e40 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.735+0000 7f8acea00700 1 -- 172.21.15.63:0/1902013035 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ac800bcf0 con 0x563f7c311e40 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.735+0000 7f8acea00700 1 -- 172.21.15.63:0/1902013035 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8003ca0 con 0x563f7c311e40 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.735+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/1902013035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 msgr2=0x563f7c490770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.735+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/1902013035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 0x563f7c490770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8adc076aa0 tx=0x7f8ac8005fb0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.736+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/1902013035 shutdown_connections 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.736+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/1902013035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c4a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.736+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/1902013035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c490230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.736+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/1902013035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 0x563f7c490770 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.736+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/1902013035 >> 172.21.15.63:0/1902013035 conn(0x563f7c3ab390 msgr2=0x563f7c48c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.736+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/1902013035 shutdown_connections 2024-05-27T08:21:30.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.736+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/1902013035 wait complete. 2024-05-27T08:21:30.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.736+0000 7f8af2a0ab80 1 -- start start 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.737+0000 7f8af2a0ab80 1 Processor -- start 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.737+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c311e40 0x563f7c49eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.737+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 0x563f7c49f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.737+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c49f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.737+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563f7c2b4fa0 con 0x563f7c311e40 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.737+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f7c34f760 con 0x563f7c490d00 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.737+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563f7c4a1f40 con 0x563f7c3fd9f0 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.737+0000 7f8af2a0ab80 10 client.0 _ll_register_callbacks cb 0x563f7c358c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.738+0000 7f8ae1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c311e40 0x563f7c49eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.738+0000 7f8ae0a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 0x563f7c49f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.738+0000 7f8ae1400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c311e40 0x563f7c49eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54318/0 (socket says 172.21.15.63:54318) 2024-05-27T08:21:30.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.738+0000 7f8ae1400700 1 -- 172.21.15.63:0/3428878730 learned_addr learned my addr 172.21.15.63:0/3428878730 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:30.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.738+0000 7f8ae1e00700 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c49f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.738+0000 7f8ae1400700 1 -- 172.21.15.63:0/3428878730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 msgr2=0x563f7c49f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.738+0000 7f8ae1400700 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 0x563f7c49f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.738+0000 7f8ae1400700 1 -- 172.21.15.63:0/3428878730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 msgr2=0x563f7c49f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.738+0000 7f8ae1400700 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c49f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.738+0000 7f8ae1400700 1 -- 172.21.15.63:0/3428878730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f7c2fe8d0 con 0x563f7c311e40 2024-05-27T08:21:30.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.739+0000 7f8ae1400700 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c311e40 0x563f7c49eae0 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00dff40 tx=0x7f8ad00b2720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6207e00a5ded8db0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.739+0000 7f8ac3400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.739+0000 7f8ac3400700 1 -- 172.21.15.63:0/3428878730 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00a9580 con 0x563f7c311e40 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.739+0000 7f8ac3400700 1 -- 172.21.15.63:0/3428878730 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(182..182 src has 1..182) ==== 9395+0+0 (secure 0 0 0) 0x7f8ad00cc2e0 con 0x563f7c311e40 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.739+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/3428878730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac800bcf0 con 0x563f7c311e40 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.739+0000 7f8ac3400700 1 -- 172.21.15.63:0/3428878730 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8ad00aa170 con 0x563f7c311e40 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.739+0000 7f8af2a0ab80 10 client.23919 fetch_fsmap learned FSMap version 51 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.739+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/3428878730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x563f7c37f060 con 0x563f7c311e40 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.740+0000 7f8ac3400700 1 -- 172.21.15.63:0/3428878730 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ac800bcf0 con 0x563f7c311e40 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.740+0000 7f8ac3400700 1 -- 172.21.15.63:0/3428878730 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f8ad006f260 con 0x563f7c311e40 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.740+0000 7f8af2a0ab80 10 client.23919 fetch_fsmap finished waiting for FSMap version 51 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.740+0000 7f8af2a0ab80 10 client.23919 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:21:30.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.740+0000 7f8af2a0ab80 20 client.23919 populate_metadata read hostname 'smithi063' 2024-05-27T08:21:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.740+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563f7c4aeee0 0x563f7c4b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.740+0000 7f8af2a0ab80 4 client.23919 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:21:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.740+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/3428878730 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f7c3c0d90 con 0x563f7c4aeee0 2024-05-27T08:21:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.741+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563f7c4aeee0 0x563f7c4b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.742+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563f7c4aeee0 0x563f7c4b1300 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53651e4d59bdc70b server_cookie=468cb6eae6486fe5 in_seq=0 out_seq=0 2024-05-27T08:21:30.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.742+0000 7f8ac3400700 10 client.23919 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:21:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8ac3400700 1 -- 172.21.15.63:0/3428878730 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563f7c3c0d90 con 0x563f7c4aeee0 2024-05-27T08:21:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8ac3400700 10 client.23919 handle_command_reply: tid=0 2024-05-27T08:21:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8af2a0ab80 1 client.23919 shutdown 2024-05-27T08:21:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/3428878730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c311e40 msgr2=0x563f7c49eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:30.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c311e40 0x563f7c49eae0 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00dff40 tx=0x7f8ad00b2720 comp rx=0 tx=0).stop 2024-05-27T08:21:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/3428878730 shutdown_connections 2024-05-27T08:21:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c49f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563f7c4aeee0 0x563f7c4b1300 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c311e40 0x563f7c49eae0 unknown :-1 s=CLOSED pgs=3026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/3428878730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 0x563f7c49f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8ac3400700 0 client.23919 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:21:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.744+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/3428878730 >> 172.21.15.63:0/3428878730 conn(0x563f7c3ab390 msgr2=0x563f7c490230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.745+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/3428878730 shutdown_connections 2024-05-27T08:21:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.745+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/3428878730 wait complete. 2024-05-27T08:21:30.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.745+0000 7f8af2a0ab80 20 client.23919 trim_cache size 0 max 16384 2024-05-27T08:21:30.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.762+0000 7f8af2a0ab80 1 Processor -- start 2024-05-27T08:21:30.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.762+0000 7f8af2a0ab80 1 -- start start 2024-05-27T08:21:30.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.762+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c311e40 0x563f7c48c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.762+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 0x563f7c4ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.762+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c490d00 0x563f7c4b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.762+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc011520 con 0x563f7c490d00 2024-05-27T08:21:30.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.762+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc00ed80 con 0x563f7c311e40 2024-05-27T08:21:30.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.762+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc00cbd0 con 0x563f7c3fd9f0 2024-05-27T08:21:30.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c490d00 0x563f7c4b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae1e00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c490d00 0x563f7c4b6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54334/0 (socket says 172.21.15.63:54334) 2024-05-27T08:21:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae1e00700 1 -- 172.21.15.63:0/462281324 learned_addr learned my addr 172.21.15.63:0/462281324 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae1400700 1 --2- 172.21.15.63:0/462281324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c311e40 0x563f7c48c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/462281324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 0x563f7c4ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae1e00700 1 -- 172.21.15.63:0/462281324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 msgr2=0x563f7c4ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae1e00700 1 --2- 172.21.15.63:0/462281324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 0x563f7c4ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae1e00700 1 -- 172.21.15.63:0/462281324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c311e40 msgr2=0x563f7c48c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae1e00700 1 --2- 172.21.15.63:0/462281324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c311e40 0x563f7c48c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae1e00700 1 -- 172.21.15.63:0/462281324 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f7c490ae0 con 0x563f7c490d00 2024-05-27T08:21:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.763+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/462281324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 0x563f7c4ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:30.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.764+0000 7f8ae1e00700 1 --2- 172.21.15.63:0/462281324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c490d00 0x563f7c4b6ae0 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40197b0 tx=0x7f8ad401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55380ddc517f36c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.764+0000 7f8ace000700 1 -- 172.21.15.63:0/462281324 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4012040 con 0x563f7c490d00 2024-05-27T08:21:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.764+0000 7f8ace000700 1 -- 172.21.15.63:0/462281324 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ad4024ae0 con 0x563f7c490d00 2024-05-27T08:21:30.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.764+0000 7f8ace000700 1 -- 172.21.15.63:0/462281324 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4023b40 con 0x563f7c490d00 2024-05-27T08:21:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.765+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/462281324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c490d00 msgr2=0x563f7c4b6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.765+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/462281324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c490d00 0x563f7c4b6ae0 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40197b0 tx=0x7f8ad401c000 comp rx=0 tx=0).stop 2024-05-27T08:21:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.765+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/462281324 shutdown_connections 2024-05-27T08:21:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.765+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/462281324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c490d00 0x563f7c4b6ae0 unknown :-1 s=CLOSED pgs=3027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.765+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/462281324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c311e40 0x563f7c48c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.765+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/462281324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c3fd9f0 0x563f7c4ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.765+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/462281324 >> 172.21.15.63:0/462281324 conn(0x563f7c3ab390 msgr2=0x563f7c48bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.765+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/462281324 shutdown_connections 2024-05-27T08:21:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.765+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/462281324 wait complete. 2024-05-27T08:21:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.766+0000 7f8af2a0ab80 1 -- start start 2024-05-27T08:21:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.766+0000 7f8af2a0ab80 1 Processor -- start 2024-05-27T08:21:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 0x563f7c4a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c4a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8af2a0ab80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c498f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc00d240 con 0x563f7c3fd9f0 2024-05-27T08:21:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc00e6b0 con 0x563f7c490d00 2024-05-27T08:21:30.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8af2a0ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc00d910 con 0x563f7c311e40 2024-05-27T08:21:30.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8af2a0ab80 10 client.0 _ll_register_callbacks cb 0x563f7c358c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:21:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8ae0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c4a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8ae0a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c4a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54338/0 (socket says 172.21.15.63:54338) 2024-05-27T08:21:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8ae0a00700 1 -- 172.21.15.63:0/975189511 learned_addr learned my addr 172.21.15.63:0/975189511 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8ae1400700 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 0x563f7c4a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8ae1e00700 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c498f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8ae0a00700 1 -- 172.21.15.63:0/975189511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 msgr2=0x563f7c4a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 0x563f7c4a74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8ae0a00700 1 -- 172.21.15.63:0/975189511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 msgr2=0x563f7c498f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c498f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.767+0000 7f8ae0a00700 1 -- 172.21.15.63:0/975189511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f7c49f8d0 con 0x563f7c3fd9f0 2024-05-27T08:21:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.768+0000 7f8ae0a00700 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c4a79f0 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80099c0 tx=0x7f8ac80037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71f8822d1cab045 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:30.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.768+0000 7f8ac3400700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:21:30.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.768+0000 7f8ac3400700 1 -- 172.21.15.63:0/975189511 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8027040 con 0x563f7c3fd9f0 2024-05-27T08:21:30.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.768+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/975189511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ad4024ae0 con 0x563f7c3fd9f0 2024-05-27T08:21:30.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.768+0000 7f8ac3400700 1 -- 172.21.15.63:0/975189511 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(182..182 src has 1..182) ==== 9395+0+0 (secure 0 0 0) 0x7f8ac8003ad0 con 0x563f7c3fd9f0 2024-05-27T08:21:30.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.768+0000 7f8ac3400700 1 -- 172.21.15.63:0/975189511 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8ac8042050 con 0x563f7c3fd9f0 2024-05-27T08:21:30.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.769+0000 7f8af2a0ab80 10 client.23925 fetch_fsmap learned FSMap version 51 2024-05-27T08:21:30.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.769+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/975189511 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x563f7c4a6ae0 con 0x563f7c3fd9f0 2024-05-27T08:21:30.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.769+0000 7f8ac3400700 1 -- 172.21.15.63:0/975189511 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8ad4024ae0 con 0x563f7c3fd9f0 2024-05-27T08:21:30.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.769+0000 7f8ac3400700 1 -- 172.21.15.63:0/975189511 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f8ac8037020 con 0x563f7c3fd9f0 2024-05-27T08:21:30.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.769+0000 7f8af2a0ab80 10 client.23925 fetch_fsmap finished waiting for FSMap version 51 2024-05-27T08:21:30.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.769+0000 7f8af2a0ab80 10 client.23925 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:21:30.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.769+0000 7f8af2a0ab80 20 client.23925 populate_metadata read hostname 'smithi063' 2024-05-27T08:21:30.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.769+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563f7c4a6d70 0x563f7c51b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:30.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.769+0000 7f8af2a0ab80 4 client.23925 mds_command: new command op to 34484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:21:30.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.769+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/975189511 --> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f7c3ba570 con 0x563f7c4a6d70 2024-05-27T08:21:30.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.770+0000 7f8ae1400700 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563f7c4a6d70 0x563f7c51b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:30.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.771+0000 7f8ae1400700 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563f7c4a6d70 0x563f7c51b9c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fba6f86c2f672922 server_cookie=70e81085e38b14cb in_seq=0 out_seq=0 2024-05-27T08:21:30.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.771+0000 7f8ac3400700 10 client.23925 ms_handle_connect on v2:172.21.15.86:6836/559054291 2024-05-27T08:21:30.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8ac3400700 1 -- 172.21.15.63:0/975189511 <== mds.1 v2:172.21.15.86:6836/559054291 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f7c3ba570 con 0x563f7c4a6d70 2024-05-27T08:21:30.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8ac3400700 10 client.23925 handle_command_reply: tid=0 2024-05-27T08:21:30.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 client.23925 shutdown 2024-05-27T08:21:30.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/975189511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 msgr2=0x563f7c4a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:30.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c4a79f0 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80099c0 tx=0x7f8ac80037c0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/975189511 shutdown_connections 2024-05-27T08:21:30.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f7c490d00 0x563f7c498f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f7c311e40 0x563f7c4a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.86:6836/559054291,v1:172.21.15.86:6837/559054291] conn(0x563f7c4a6d70 0x563f7c51b9c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 --2- 172.21.15.63:0/975189511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563f7c3fd9f0 0x563f7c4a79f0 unknown :-1 s=CLOSED pgs=3028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8ac3400700 0 client.23925 ms_handle_reset on v2:172.21.15.86:6836/559054291 2024-05-27T08:21:30.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/975189511 >> 172.21.15.63:0/975189511 conn(0x563f7c3ab390 msgr2=0x563f7c49e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:30.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/975189511 shutdown_connections 2024-05-27T08:21:30.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 1 -- 172.21.15.63:0/975189511 wait complete. 2024-05-27T08:21:30.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.772+0000 7f8af2a0ab80 20 client.23925 trim_cache size 0 max 16384 2024-05-27T08:21:30.780 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:21:30.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 -- 172.21.15.63:0/4220958608 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ac40b29f0 msgr2=0x7f8ac40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:30.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ac40b29f0 0x7f8ac40b4f00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009fd0 tx=0x7f8ad40079a0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 -- 172.21.15.63:0/4220958608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc1034b0 msgr2=0x7f8adc1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:30.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc1034b0 0x7f8adc1a09a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000e9a0 tx=0x7f8ad000ecb0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 -- 172.21.15.63:0/4220958608 shutdown_connections 2024-05-27T08:21:30.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f8ac40b29f0 0x7f8ac40b4f00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8adc108730 0x7f8adc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc1034b0 0x7f8adc1a09a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 --2- 172.21.15.63:0/4220958608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8adc107de0 0x7f8adc1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:30.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 -- 172.21.15.63:0/4220958608 >> 172.21.15.63:0/4220958608 conn(0x7f8adc0fedb0 msgr2=0x7f8adc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:30.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 -- 172.21.15.63:0/4220958608 shutdown_connections 2024-05-27T08:21:30.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:30.773+0000 7f8ae3e00700 1 -- 172.21.15.63:0/4220958608 wait complete. 2024-05-27T08:21:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:30 smithi086 ceph-mon[34687]: pgmap v1528: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-05-27T08:21:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:30 smithi086 ceph-mon[34687]: osdmap e182: 12 total, 12 up, 12 in 2024-05-27T08:21:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:30 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/4071622084,v1:172.21.15.149:6835/4071622084] up:boot 2024-05-27T08:21:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:30 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:30 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:30.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:30 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:30.964 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:30 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-05-27T08:21:30.964 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:30 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 1.408s CPU time. 2024-05-27T08:21:30.964 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:30 smithi149 podman[81498]: 2024-05-27 08:21:30.580056677 +0000 UTC m=+0.795959733 container remove 68987e8de46dad91e42c1c6b21823094f37cf8449a5d3b4dbfc2bfe4119ca81b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T08:21:30.964 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:30 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:30.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:30 smithi149 ceph-mon[35408]: pgmap v1528: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-05-27T08:21:30.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:30 smithi149 ceph-mon[35408]: osdmap e182: 12 total, 12 up, 12 in 2024-05-27T08:21:30.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:30 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/4071622084,v1:172.21.15.149:6835/4071622084] up:boot 2024-05-27T08:21:30.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:30 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:30.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:30 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:30.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:30 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:31.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:31 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T08:21:31.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:31 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.441s CPU time. 2024-05-27T08:21:31.693 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:21:31.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:31 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:21:31.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:31 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:21:31.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:31 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:21:32.158 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:21:32.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:32 smithi086 ceph-mon[34687]: pgmap v1530: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 0 op/s 2024-05-27T08:21:32.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:32 smithi149 ceph-mon[35408]: pgmap v1530: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 0 op/s 2024-05-27T08:21:32.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:32 smithi063 ceph-mon[31613]: pgmap v1530: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 0 op/s 2024-05-27T08:21:33.614 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:33 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 2. 2024-05-27T08:21:33.614 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:33 smithi149 systemd[1]: Stopped Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:33.614 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:33 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 1.580s CPU time. 2024-05-27T08:21:33.614 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:33 smithi149 systemd[1]: Starting Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:33.877 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:33 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:21:33.877 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:33 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:21:33.936 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:33 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:21:33.936 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:33 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:21:33.936 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:33 smithi149 podman[81706]: 2024-05-27 08:21:33.517951627 +0000 UTC m=+0.033395696 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:33.936 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:33 smithi149 podman[81706]: 2024-05-27 08:21:33.653092768 +0000 UTC m=+0.168536861 container create 8445e5551de7d108f9d07e31d13b3ee0d08cecbe8f35aeeb0fb386297a63af63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS) 2024-05-27T08:21:34.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:33 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:21:34.263 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:33 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:21:34.264 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:33 smithi149 podman[81706]: 2024-05-27 08:21:33.935113913 +0000 UTC m=+0.450558069 container init 8445e5551de7d108f9d07e31d13b3ee0d08cecbe8f35aeeb0fb386297a63af63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T08:21:34.264 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:33 smithi149 podman[81706]: 2024-05-27 08:21:33.945368433 +0000 UTC m=+0.460812504 container start 8445e5551de7d108f9d07e31d13b3ee0d08cecbe8f35aeeb0fb386297a63af63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T08:21:34.264 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 bash[81706]: 8445e5551de7d108f9d07e31d13b3ee0d08cecbe8f35aeeb0fb386297a63af63 2024-05-27T08:21:34.264 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 systemd[1]: Started Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:34.264 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-mds[81725]: -- 172.21.15.149:0/28347790 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598c8299a40 con 0x5598c82e1180 2024-05-27T08:21:34.264 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: starting mds.i at 2024-05-27T08:21:34.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.545+0000 7f3a92400700 1 -- 172.21.15.63:0/514798096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c10af60 msgr2=0x7f3a8c10b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:34.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.545+0000 7f3a92400700 1 --2- 172.21.15.63:0/514798096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c10af60 0x7f3a8c10b370 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7f3a780099c0 tx=0x7f3a78009cd0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.545+0000 7f3a92400700 1 -- 172.21.15.63:0/514798096 shutdown_connections 2024-05-27T08:21:34.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.545+0000 7f3a92400700 1 --2- 172.21.15.63:0/514798096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c10b8b0 0x7f3a8c113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.545+0000 7f3a92400700 1 --2- 172.21.15.63:0/514798096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8c106630 0x7f3a8c106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.545+0000 7f3a92400700 1 --2- 172.21.15.63:0/514798096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c10af60 0x7f3a8c10b370 unknown :-1 s=CLOSED pgs=3029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.545+0000 7f3a92400700 1 -- 172.21.15.63:0/514798096 >> 172.21.15.63:0/514798096 conn(0x7f3a8c101ec0 msgr2=0x7f3a8c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.545+0000 7f3a92400700 1 -- 172.21.15.63:0/514798096 shutdown_connections 2024-05-27T08:21:34.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.545+0000 7f3a92400700 1 -- 172.21.15.63:0/514798096 wait complete. 2024-05-27T08:21:34.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.546+0000 7f3a92400700 1 Processor -- start 2024-05-27T08:21:34.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.547+0000 7f3a92400700 1 -- start start 2024-05-27T08:21:34.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.547+0000 7f3a92400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8c106630 0x7f3a8c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.547+0000 7f3a92400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c10af60 0x7f3a8c19d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.547+0000 7f3a92400700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c10b8b0 0x7f3a8c1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.547+0000 7f3a92400700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1163e0 con 0x7f3a8c10b8b0 2024-05-27T08:21:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.547+0000 7f3a92400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c116260 con 0x7f3a8c106630 2024-05-27T08:21:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.547+0000 7f3a92400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c116560 con 0x7f3a8c10af60 2024-05-27T08:21:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.548+0000 7f3a91a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c10b8b0 0x7f3a8c1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.548+0000 7f3a91a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c10b8b0 0x7f3a8c1a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54364/0 (socket says 172.21.15.63:54364) 2024-05-27T08:21:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.548+0000 7f3a91a00700 1 -- 172.21.15.63:0/2943342077 learned_addr learned my addr 172.21.15.63:0/2943342077 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.548+0000 7f3a8be00700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c10af60 0x7f3a8c19d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.548+0000 7f3a91000700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8c106630 0x7f3a8c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.548+0000 7f3a91a00700 1 -- 172.21.15.63:0/2943342077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c10af60 msgr2=0x7f3a8c19d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.548+0000 7f3a91a00700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c10af60 0x7f3a8c19d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.548+0000 7f3a91a00700 1 -- 172.21.15.63:0/2943342077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8c106630 msgr2=0x7f3a8c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:34.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.548+0000 7f3a91a00700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8c106630 0x7f3a8c19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.548+0000 7f3a91a00700 1 -- 172.21.15.63:0/2943342077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c1a1c80 con 0x7f3a8c10b8b0 2024-05-27T08:21:34.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.549+0000 7f3a91a00700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c10b8b0 0x7f3a8c1a15e0 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400e940 tx=0x7f3a8400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:34.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.549+0000 7f3a89600700 1 -- 172.21.15.63:0/2943342077 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84004a30 con 0x7f3a8c10b8b0 2024-05-27T08:21:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.549+0000 7f3a89600700 1 -- 172.21.15.63:0/2943342077 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a840054b0 con 0x7f3a8c10b8b0 2024-05-27T08:21:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.549+0000 7f3a92400700 1 -- 172.21.15.63:0/2943342077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c1a1f70 con 0x7f3a8c10b8b0 2024-05-27T08:21:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.549+0000 7f3a89600700 1 -- 172.21.15.63:0/2943342077 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400cbe0 con 0x7f3a8c10b8b0 2024-05-27T08:21:34.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.549+0000 7f3a92400700 1 -- 172.21.15.63:0/2943342077 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c1a9890 con 0x7f3a8c10b8b0 2024-05-27T08:21:34.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.554+0000 7f3a89600700 1 -- 172.21.15.63:0/2943342077 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3a84005020 con 0x7f3a8c10b8b0 2024-05-27T08:21:34.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.554+0000 7f3aa260cb80 1 Processor -- start 2024-05-27T08:21:34.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.555+0000 7f3aa260cb80 1 -- start start 2024-05-27T08:21:34.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.555+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fc8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.555+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7e48e40 0x55d3c7fc8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.555+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fd46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.555+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d3c7ef5e40 con 0x55d3c7fd22c0 2024-05-27T08:21:34.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.555+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3c7ee59e0 con 0x55d3c7e48e40 2024-05-27T08:21:34.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.555+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d3c7ee9a20 con 0x55d3c7f349f0 2024-05-27T08:21:34.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.555+0000 7f3a89600700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3a740b29f0 0x7f3a740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fc8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a89600700 1 -- 172.21.15.63:0/2943342077 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 9395+0+0 (secure 0 0 0) 0x7f3a840e2b30 con 0x7f3a8c10b8b0 2024-05-27T08:21:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fc8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41522/0 (socket says 172.21.15.63:41522) 2024-05-27T08:21:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a8be00700 1 -- 172.21.15.63:0/3852469574 learned_addr learned my addr 172.21.15.63:0/3852469574 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a8be00700 1 --2- 172.21.15.63:0/3852469574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fd46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a91000700 1 --2- 172.21.15.63:0/3852469574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7e48e40 0x55d3c7fc8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a91a00700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3a740b29f0 0x7f3a740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a8be00700 1 -- 172.21.15.63:0/3852469574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7e48e40 msgr2=0x55d3c7fc8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a8be00700 1 --2- 172.21.15.63:0/3852469574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7e48e40 0x55d3c7fc8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a8be00700 1 -- 172.21.15.63:0/3852469574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7fd22c0 msgr2=0x55d3c7fd46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a8be00700 1 --2- 172.21.15.63:0/3852469574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fd46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.556+0000 7f3a8be00700 1 -- 172.21.15.63:0/3852469574 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c7eae420 con 0x55d3c7f349f0 2024-05-27T08:21:34.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.557+0000 7f3a91a00700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3a740b29f0 0x7f3a740b4f00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009230 tx=0x7f3a840097b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:34.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.557+0000 7f3a8be00700 1 --2- 172.21.15.63:0/3852469574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fc8750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a78009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecc47190f7118b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:34.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.557+0000 7f3a7e000700 1 -- 172.21.15.63:0/3852469574 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78005410 con 0x55d3c7f349f0 2024-05-27T08:21:34.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.557+0000 7f3a7e000700 1 -- 172.21.15.63:0/3852469574 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a780055b0 con 0x55d3c7f349f0 2024-05-27T08:21:34.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.557+0000 7f3a7e000700 1 -- 172.21.15.63:0/3852469574 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800bbf0 con 0x55d3c7f349f0 2024-05-27T08:21:34.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.558+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3852469574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 msgr2=0x55d3c7fc8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:34.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.558+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/3852469574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fc8750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a78009230 comp rx=0 tx=0).stop 2024-05-27T08:21:34.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.559+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3852469574 shutdown_connections 2024-05-27T08:21:34.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.559+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/3852469574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fd46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.559+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/3852469574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fc8750 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.559+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/3852469574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7e48e40 0x55d3c7fc8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.559+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3852469574 >> 172.21.15.63:0/3852469574 conn(0x55d3c7ee2390 msgr2=0x55d3c7fc3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:34.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.559+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3852469574 shutdown_connections 2024-05-27T08:21:34.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.559+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3852469574 wait complete. 2024-05-27T08:21:34.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.559+0000 7f3aa260cb80 1 -- start start 2024-05-27T08:21:34.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.560+0000 7f3aa260cb80 1 Processor -- start 2024-05-27T08:21:34.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.560+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fcf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.560+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7f349f0 0x55d3c7fcf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.560+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fcfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.560+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d3c7debfa0 con 0x55d3c7f349f0 2024-05-27T08:21:34.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.560+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3c7e86760 con 0x55d3c7fd22c0 2024-05-27T08:21:34.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.560+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d3c7fd4da0 con 0x55d3c7e48e40 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.560+0000 7f3aa260cb80 10 client.0 _ll_register_callbacks cb 0x55d3c7e8fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fcf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a91000700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7f349f0 0x55d3c7fcf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fcf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41528/0 (socket says 172.21.15.63:41528) 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a8be00700 1 -- 172.21.15.63:0/3060262464 learned_addr learned my addr 172.21.15.63:0/3060262464 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a8be00700 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fcfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a91000700 1 -- 172.21.15.63:0/3060262464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 msgr2=0x55d3c7fcf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a91000700 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fcf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a91000700 1 -- 172.21.15.63:0/3060262464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7fd22c0 msgr2=0x55d3c7fcfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a91000700 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fcfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a91000700 1 -- 172.21.15.63:0/3060262464 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c7e358d0 con 0x55d3c7f349f0 2024-05-27T08:21:34.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.561+0000 7f3a8be00700 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fcfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:21:34.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.562+0000 7f3a91000700 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7f349f0 0x55d3c7fcf7e0 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f3a80011e90 tx=0x7f3a80014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a0a6916bad53d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:34.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.562+0000 7f3a72a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:21:34.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.562+0000 7f3a72a00700 1 -- 172.21.15.63:0/3060262464 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80014d30 con 0x55d3c7f349f0 2024-05-27T08:21:34.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.562+0000 7f3a72a00700 1 -- 172.21.15.63:0/3060262464 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 9395+0+0 (secure 0 0 0) 0x7f3a80007500 con 0x55d3c7f349f0 2024-05-27T08:21:34.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.562+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3060262464 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a780055b0 con 0x55d3c7f349f0 2024-05-27T08:21:34.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.562+0000 7f3a72a00700 1 -- 172.21.15.63:0/3060262464 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3a80028050 con 0x55d3c7f349f0 2024-05-27T08:21:34.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.562+0000 7f3aa260cb80 10 client.23943 fetch_fsmap learned FSMap version 51 2024-05-27T08:21:34.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.562+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3060262464 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d3c7eb6060 con 0x55d3c7f349f0 2024-05-27T08:21:34.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.563+0000 7f3a72a00700 1 -- 172.21.15.63:0/3060262464 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a780055b0 con 0x55d3c7f349f0 2024-05-27T08:21:34.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.563+0000 7f3a72a00700 1 -- 172.21.15.63:0/3060262464 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f3a8001f510 con 0x55d3c7f349f0 2024-05-27T08:21:34.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.563+0000 7f3aa260cb80 10 client.23943 fetch_fsmap finished waiting for FSMap version 51 2024-05-27T08:21:34.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.563+0000 7f3aa260cb80 10 client.23943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:21:34.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.563+0000 7f3aa260cb80 20 client.23943 populate_metadata read hostname 'smithi063' 2024-05-27T08:21:34.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.563+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3c7fe1e60 0x55d3c7fe42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.563+0000 7f3aa260cb80 4 client.23943 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:21:34.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.563+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3060262464 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3c7ef7d90 con 0x55d3c7fe1e60 2024-05-27T08:21:34.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.564+0000 7f3a8be00700 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3c7fe1e60 0x55d3c7fe42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.565+0000 7f3a8be00700 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3c7fe1e60 0x55d3c7fe42b0 crc :-1 s=READY pgs=800 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec99aa5b06c24d1 server_cookie=c3405e8f6101db1a in_seq=0 out_seq=0 2024-05-27T08:21:34.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.565+0000 7f3a72a00700 10 client.23943 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:34.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3a72a00700 1 -- 172.21.15.63:0/3060262464 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d3c7ef7d90 con 0x55d3c7fe1e60 2024-05-27T08:21:34.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3a72a00700 10 client.23943 handle_command_reply: tid=0 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3aa260cb80 1 client.23943 shutdown 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3060262464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7f349f0 msgr2=0x55d3c7fcf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7f349f0 0x55d3c7fcf7e0 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f3a80011e90 tx=0x7f3a80014600 comp rx=0 tx=0).stop 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3060262464 shutdown_connections 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3c7fe1e60 0x55d3c7fe42b0 crc :-1 s=READY pgs=800 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fcfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fcf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/3060262464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7f349f0 0x55d3c7fcf7e0 unknown :-1 s=CLOSED pgs=3031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3060262464 >> 172.21.15.63:0/3060262464 conn(0x55d3c7ee2390 msgr2=0x55d3c7fc87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.567+0000 7f3a72a00700 0 client.23943 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.568+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3060262464 shutdown_connections 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.568+0000 7f3aa260cb80 1 -- 172.21.15.63:0/3060262464 wait complete. 2024-05-27T08:21:34.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.568+0000 7f3aa260cb80 20 client.23943 trim_cache size 0 max 16384 2024-05-27T08:21:34.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.585+0000 7f3aa260cb80 1 Processor -- start 2024-05-27T08:21:34.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.585+0000 7f3aa260cb80 1 -- start start 2024-05-27T08:21:34.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.585+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fc3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.585+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7f349f0 0x55d3c7fdde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.585+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.585+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b4bc0 con 0x55d3c7fd22c0 2024-05-27T08:21:34.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.585+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b2420 con 0x55d3c7f349f0 2024-05-27T08:21:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.585+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b0270 con 0x55d3c7e48e40 2024-05-27T08:21:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.586+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fc3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.586+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fc3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.63:41530/0 (socket says 172.21.15.63:41530) 2024-05-27T08:21:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.586+0000 7f3a8be00700 1 -- 172.21.15.63:0/1815204174 learned_addr learned my addr 172.21.15.63:0/1815204174 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:34.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.586+0000 7f3a91000700 1 --2- 172.21.15.63:0/1815204174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7f349f0 0x55d3c7fdde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.586+0000 7f3a8be00700 1 --2- 172.21.15.63:0/1815204174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.587+0000 7f3a8be00700 1 -- 172.21.15.63:0/1815204174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7f349f0 msgr2=0x55d3c7fdde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:34.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.587+0000 7f3a8be00700 1 --2- 172.21.15.63:0/1815204174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7f349f0 0x55d3c7fdde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.587+0000 7f3a8be00700 1 -- 172.21.15.63:0/1815204174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7fd22c0 msgr2=0x55d3c7fce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:34.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.587+0000 7f3a8be00700 1 --2- 172.21.15.63:0/1815204174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.587+0000 7f3a8be00700 1 -- 172.21.15.63:0/1815204174 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c7fdf7e0 con 0x55d3c7e48e40 2024-05-27T08:21:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.587+0000 7f3a8be00700 1 --2- 172.21.15.63:0/1815204174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fc3740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a78005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c8a00b13e4a1b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.587+0000 7f3a7d600700 1 -- 172.21.15.63:0/1815204174 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78003b00 con 0x55d3c7e48e40 2024-05-27T08:21:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.588+0000 7f3a7d600700 1 -- 172.21.15.63:0/1815204174 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a78003ca0 con 0x55d3c7e48e40 2024-05-27T08:21:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.588+0000 7f3a7d600700 1 -- 172.21.15.63:0/1815204174 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78033520 con 0x55d3c7e48e40 2024-05-27T08:21:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.588+0000 7f3aa260cb80 1 -- 172.21.15.63:0/1815204174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 msgr2=0x55d3c7fc3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.588+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/1815204174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fc3740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009fd0 tx=0x7f3a78005040 comp rx=0 tx=0).stop 2024-05-27T08:21:34.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.588+0000 7f3aa260cb80 1 -- 172.21.15.63:0/1815204174 shutdown_connections 2024-05-27T08:21:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.588+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/1815204174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.588+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/1815204174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7e48e40 0x55d3c7fc3740 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.588+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/1815204174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7f349f0 0x55d3c7fdde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.588+0000 7f3aa260cb80 1 -- 172.21.15.63:0/1815204174 >> 172.21.15.63:0/1815204174 conn(0x55d3c7ee2390 msgr2=0x55d3c7fcdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:34.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.589+0000 7f3aa260cb80 1 -- 172.21.15.63:0/1815204174 shutdown_connections 2024-05-27T08:21:34.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.589+0000 7f3aa260cb80 1 -- 172.21.15.63:0/1815204174 wait complete. 2024-05-27T08:21:34.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.589+0000 7f3aa260cb80 1 -- start start 2024-05-27T08:21:34.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.589+0000 7f3aa260cb80 1 Processor -- start 2024-05-27T08:21:34.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7e48e40 0x55d3c7fd5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fd5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3aa260cb80 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fd5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b08e0 con 0x55d3c7e48e40 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b1d50 con 0x55d3c7fd22c0 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3aa260cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b0fb0 con 0x55d3c7f349f0 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3aa260cb80 10 client.0 _ll_register_callbacks cb 0x55d3c7e8fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7e48e40 0x55d3c7fd5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3a91000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fd5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3a8be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7e48e40 0x55d3c7fd5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54382/0 (socket says 172.21.15.63:54382) 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.590+0000 7f3a8be00700 1 -- 172.21.15.63:0/2425604189 learned_addr learned my addr 172.21.15.63:0/2425604189 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.591+0000 7f3a8be00700 1 -- 172.21.15.63:0/2425604189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 msgr2=0x55d3c7fd5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.591+0000 7f3a8be00700 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fd5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.591+0000 7f3a8be00700 1 -- 172.21.15.63:0/2425604189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7fd22c0 msgr2=0x55d3c7fd5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:21:34.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.591+0000 7f3a8be00700 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fd5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.591+0000 7f3a8be00700 1 -- 172.21.15.63:0/2425604189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c7fc9d60 con 0x55d3c7e48e40 2024-05-27T08:21:34.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.591+0000 7f3a91000700 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fd5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:21:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.591+0000 7f3a8be00700 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7e48e40 0x55d3c7fd5020 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f3a780099c0 tx=0x7f3a780096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf79ec79fd70ddd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.591+0000 7f3a72a00700 10 client.0 ms_handle_connect on v2:172.21.15.63:3300/0 2024-05-27T08:21:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.591+0000 7f3a72a00700 1 -- 172.21.15.63:0/2425604189 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800b990 con 0x55d3c7e48e40 2024-05-27T08:21:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.592+0000 7f3a72a00700 1 -- 172.21.15.63:0/2425604189 <== mon.0 v2:172.21.15.63:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 9395+0+0 (secure 0 0 0) 0x7f3a7800bb30 con 0x55d3c7e48e40 2024-05-27T08:21:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.592+0000 7f3aa260cb80 1 -- 172.21.15.63:0/2425604189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a78003ca0 con 0x55d3c7e48e40 2024-05-27T08:21:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.592+0000 7f3a72a00700 1 -- 172.21.15.63:0/2425604189 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3a78044050 con 0x55d3c7e48e40 2024-05-27T08:21:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.592+0000 7f3aa260cb80 10 client.23949 fetch_fsmap learned FSMap version 51 2024-05-27T08:21:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.592+0000 7f3aa260cb80 1 -- 172.21.15.63:0/2425604189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d3c804bb20 con 0x55d3c7e48e40 2024-05-27T08:21:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.592+0000 7f3a72a00700 1 -- 172.21.15.63:0/2425604189 <== mon.0 v2:172.21.15.63:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a78003ca0 con 0x55d3c7e48e40 2024-05-27T08:21:34.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.592+0000 7f3a72a00700 1 -- 172.21.15.63:0/2425604189 <== mon.0 v2:172.21.15.63:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f3a78021040 con 0x55d3c7e48e40 2024-05-27T08:21:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.593+0000 7f3aa260cb80 10 client.23949 fetch_fsmap finished waiting for FSMap version 51 2024-05-27T08:21:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.593+0000 7f3aa260cb80 10 client.23949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:21:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.593+0000 7f3aa260cb80 20 client.23949 populate_metadata read hostname 'smithi063' 2024-05-27T08:21:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.593+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3c804bd50 0x55d3c804e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.593+0000 7f3aa260cb80 4 client.23949 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:21:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.593+0000 7f3aa260cb80 1 -- 172.21.15.63:0/2425604189 --> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3c7ef1570 con 0x55d3c804bd50 2024-05-27T08:21:34.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.593+0000 7f3a91000700 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3c804bd50 0x55d3c804e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.594+0000 7f3a91000700 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3c804bd50 0x55d3c804e170 crc :-1 s=READY pgs=801 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc1bbee612b8f6de server_cookie=70701011589f5523 in_seq=0 out_seq=0 2024-05-27T08:21:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.594+0000 7f3a72a00700 10 client.23949 ms_handle_connect on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3a72a00700 1 -- 172.21.15.63:0/2425604189 <== mds.0 v2:172.21.15.86:6838/953218889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3c7ef1570 con 0x55d3c804bd50 2024-05-27T08:21:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3a72a00700 10 client.23949 handle_command_reply: tid=0 2024-05-27T08:21:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 client.23949 shutdown 2024-05-27T08:21:34.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 -- 172.21.15.63:0/2425604189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7e48e40 msgr2=0x55d3c7fd5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7e48e40 0x55d3c7fd5020 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f3a780099c0 tx=0x7f3a780096f0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 -- 172.21.15.63:0/2425604189 shutdown_connections 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.86:6838/953218889,v1:172.21.15.86:6839/953218889] conn(0x55d3c804bd50 0x55d3c804e170 crc :-1 s=READY pgs=801 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3c7fd22c0 0x55d3c7fd5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3c7e48e40 0x55d3c7fd5020 unknown :-1 s=CLOSED pgs=3032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 --2- 172.21.15.63:0/2425604189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3c7f349f0 0x55d3c7fd5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3a72a00700 0 client.23949 ms_handle_reset on v2:172.21.15.86:6838/953218889 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 -- 172.21.15.63:0/2425604189 >> 172.21.15.63:0/2425604189 conn(0x55d3c7ee2390 msgr2=0x55d3c7fc78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 -- 172.21.15.63:0/2425604189 shutdown_connections 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.595+0000 7f3aa260cb80 1 -- 172.21.15.63:0/2425604189 wait complete. 2024-05-27T08:21:34.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.596+0000 7f3aa260cb80 20 client.23949 trim_cache size 0 max 16384 2024-05-27T08:21:34.598 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-27T08:21:34.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.596+0000 7f3a92400700 1 -- 172.21.15.63:0/2943342077 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3a740b29f0 msgr2=0x7f3a740b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:34.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.596+0000 7f3a92400700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3a740b29f0 0x7f3a740b4f00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009230 tx=0x7f3a840097b0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.596+0000 7f3a92400700 1 -- 172.21.15.63:0/2943342077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c10b8b0 msgr2=0x7f3a8c1a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:34.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.596+0000 7f3a92400700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c10b8b0 0x7f3a8c1a15e0 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400e940 tx=0x7f3a8400ec50 comp rx=0 tx=0).stop 2024-05-27T08:21:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.597+0000 7f3a92400700 1 -- 172.21.15.63:0/2943342077 shutdown_connections 2024-05-27T08:21:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.597+0000 7f3a92400700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f3a740b29f0 0x7f3a740b4f00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.597+0000 7f3a92400700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c10b8b0 0x7f3a8c1a15e0 unknown :-1 s=CLOSED pgs=3030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.597+0000 7f3a92400700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a8c106630 0x7f3a8c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.597+0000 7f3a92400700 1 --2- 172.21.15.63:0/2943342077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c10af60 0x7f3a8c19d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.597+0000 7f3a92400700 1 -- 172.21.15.63:0/2943342077 >> 172.21.15.63:0/2943342077 conn(0x7f3a8c101ec0 msgr2=0x7f3a8c10c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.597+0000 7f3a92400700 1 -- 172.21.15.63:0/2943342077 shutdown_connections 2024-05-27T08:21:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:34.597+0000 7f3a92400700 1 -- 172.21.15.63:0/2943342077 wait complete. 2024-05-27T08:21:35.117 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:34 smithi149 ceph-mon[35408]: pgmap v1531: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2 op/s 2024-05-27T08:21:35.117 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:34 smithi149 ceph-mon[35408]: daemon mds.i restarted 2024-05-27T08:21:35.117 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:34 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:21:35.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:34 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:21:35.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:34 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:21:35.118 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:34 smithi149 ceph-mon[35408]: osdmap e183: 12 total, 12 up, 12 in 2024-05-27T08:21:35.118 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc625800700 time 2024-05-27T08:21:34.789737+0000 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc63b269ba6] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc63b269dd1] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559895d0323d] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 4: (SnapRealm::build_snap_set() const+0x323) [0x559895cee971] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 5: (SnapRealm::check_cache() const+0x13e) [0x559895cecfa0] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559895ced651] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559895c82347] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55989597e87e] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x559895983e94] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55989598e969] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 11: (MDLog::_replay_thread()+0x1f38) [0x559895d2748c] 2024-05-27T08:21:35.119 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 12: (MDLog::ReplayThread::entry()+0x11) [0x559895d309cb] 2024-05-27T08:21:35.120 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 13: (Thread::entry_wrapper()+0x43) [0x7fc63b23872d] 2024-05-27T08:21:35.120 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc63b238749] 2024-05-27T08:21:35.120 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc639ddc1ca] 2024-05-27T08:21:35.120 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 16: clone() 2024-05-27T08:21:35.120 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:34.790+0000 7fc625800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc625800700 time 2024-05-27T08:21:34.789737+0000 2024-05-27T08:21:35.120 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:35.120 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.120 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:35.120 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc63b269ba6] 2024-05-27T08:21:35.120 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc63b269dd1] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559895d0323d] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 4: (SnapRealm::build_snap_set() const+0x323) [0x559895cee971] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 5: (SnapRealm::check_cache() const+0x13e) [0x559895cecfa0] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559895ced651] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559895c82347] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55989597e87e] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x559895983e94] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55989598e969] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 11: (MDLog::_replay_thread()+0x1f38) [0x559895d2748c] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 12: (MDLog::ReplayThread::entry()+0x11) [0x559895d309cb] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 13: (Thread::entry_wrapper()+0x43) [0x7fc63b23872d] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc63b238749] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc639ddc1ca] 2024-05-27T08:21:35.121 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 16: clone() 2024-05-27T08:21:35.122 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.122 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: *** Caught signal (Aborted) ** 2024-05-27T08:21:35.122 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: in thread 7fc625800700 thread_name:md_log_replay 2024-05-27T08:21:35.122 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:35.122 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc639de6d20] 2024-05-27T08:21:35.122 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2: gsignal() 2024-05-27T08:21:35.122 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 3: abort() 2024-05-27T08:21:35.122 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc63b269ced] 2024-05-27T08:21:35.122 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc63b269dd1] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559895d0323d] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 7: (SnapRealm::build_snap_set() const+0x323) [0x559895cee971] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 8: (SnapRealm::check_cache() const+0x13e) [0x559895cecfa0] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559895ced651] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559895c82347] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55989597e87e] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x559895983e94] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55989598e969] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 14: (MDLog::_replay_thread()+0x1f38) [0x559895d2748c] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 15: (MDLog::ReplayThread::entry()+0x11) [0x559895d309cb] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 16: (Thread::entry_wrapper()+0x43) [0x7fc63b23872d] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc63b238749] 2024-05-27T08:21:35.123 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc639ddc1ca] 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 19: clone() 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:34.793+0000 7fc625800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: in thread 7fc625800700 thread_name:md_log_replay 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc639de6d20] 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2: gsignal() 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 3: abort() 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc63b269ced] 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc63b269dd1] 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559895d0323d] 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 7: (SnapRealm::build_snap_set() const+0x323) [0x559895cee971] 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 8: (SnapRealm::check_cache() const+0x13e) [0x559895cecfa0] 2024-05-27T08:21:35.124 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559895ced651] 2024-05-27T08:21:35.125 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559895c82347] 2024-05-27T08:21:35.125 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55989597e87e] 2024-05-27T08:21:35.125 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x559895983e94] 2024-05-27T08:21:35.125 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55989598e969] 2024-05-27T08:21:35.125 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 14: (MDLog::_replay_thread()+0x1f38) [0x559895d2748c] 2024-05-27T08:21:35.125 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 15: (MDLog::ReplayThread::entry()+0x11) [0x559895d309cb] 2024-05-27T08:21:35.125 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 16: (Thread::entry_wrapper()+0x43) [0x7fc63b23872d] 2024-05-27T08:21:35.125 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc63b238749] 2024-05-27T08:21:35.125 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc639ddc1ca] 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 19: clone() 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: -1> 2024-05-27T08:21:34.790+0000 7fc625800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc625800700 time 2024-05-27T08:21:34.789737+0000 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc63b269ba6] 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc63b269dd1] 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559895d0323d] 2024-05-27T08:21:35.126 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 4: (SnapRealm::build_snap_set() const+0x323) [0x559895cee971] 2024-05-27T08:21:35.127 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 5: (SnapRealm::check_cache() const+0x13e) [0x559895cecfa0] 2024-05-27T08:21:35.127 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559895ced651] 2024-05-27T08:21:35.127 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559895c82347] 2024-05-27T08:21:35.127 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55989597e87e] 2024-05-27T08:21:35.127 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x559895983e94] 2024-05-27T08:21:35.127 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55989598e969] 2024-05-27T08:21:35.127 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 11: (MDLog::_replay_thread()+0x1f38) [0x559895d2748c] 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 12: (MDLog::ReplayThread::entry()+0x11) [0x559895d309cb] 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 13: (Thread::entry_wrapper()+0x43) [0x7fc63b23872d] 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc63b238749] 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc639ddc1ca] 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 16: clone() 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 0> 2024-05-27T08:21:34.793+0000 7fc625800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: in thread 7fc625800700 thread_name:md_log_replay 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc639de6d20] 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2: gsignal() 2024-05-27T08:21:35.128 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 3: abort() 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc63b269ced] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc63b269dd1] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559895d0323d] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 7: (SnapRealm::build_snap_set() const+0x323) [0x559895cee971] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 8: (SnapRealm::check_cache() const+0x13e) [0x559895cecfa0] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559895ced651] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559895c82347] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55989597e87e] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x559895983e94] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55989598e969] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 14: (MDLog::_replay_thread()+0x1f38) [0x559895d2748c] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 15: (MDLog::ReplayThread::entry()+0x11) [0x559895d309cb] 2024-05-27T08:21:35.129 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 16: (Thread::entry_wrapper()+0x43) [0x7fc63b23872d] 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc63b238749] 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc639ddc1ca] 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 19: clone() 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: -1> 2024-05-27T08:21:34.790+0000 7fc625800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc625800700 time 2024-05-27T08:21:34.789737+0000 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc63b269ba6] 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc63b269dd1] 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559895d0323d] 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 4: (SnapRealm::build_snap_set() const+0x323) [0x559895cee971] 2024-05-27T08:21:35.130 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 5: (SnapRealm::check_cache() const+0x13e) [0x559895cecfa0] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559895ced651] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559895c82347] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55989597e87e] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x559895983e94] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55989598e969] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 11: (MDLog::_replay_thread()+0x1f38) [0x559895d2748c] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 12: (MDLog::ReplayThread::entry()+0x11) [0x559895d309cb] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 13: (Thread::entry_wrapper()+0x43) [0x7fc63b23872d] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc63b238749] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc639ddc1ca] 2024-05-27T08:21:35.131 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 16: clone() 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 0> 2024-05-27T08:21:34.793+0000 7fc625800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: in thread 7fc625800700 thread_name:md_log_replay 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc639de6d20] 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2: gsignal() 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 3: abort() 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc63b269ced] 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc63b269dd1] 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x559895d0323d] 2024-05-27T08:21:35.132 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 7: (SnapRealm::build_snap_set() const+0x323) [0x559895cee971] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 8: (SnapRealm::check_cache() const+0x13e) [0x559895cecfa0] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x559895ced651] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x559895c82347] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55989597e87e] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x559895983e94] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55989598e969] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 14: (MDLog::_replay_thread()+0x1f38) [0x559895d2748c] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 15: (MDLog::ReplayThread::entry()+0x11) [0x559895d309cb] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 16: (Thread::entry_wrapper()+0x43) [0x7fc63b23872d] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc63b238749] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc639ddc1ca] 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 19: clone() 2024-05-27T08:21:35.133 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:35.134 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[81721]: 2024-05-27T08:21:35.134 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:34 smithi149 podman[81788]: 2024-05-27 08:21:34.879075036 +0000 UTC m=+0.033391465 container died 8445e5551de7d108f9d07e31d13b3ee0d08cecbe8f35aeeb0fb386297a63af63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:21:35.144 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:34 smithi063 ceph-mon[31613]: pgmap v1531: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2 op/s 2024-05-27T08:21:35.144 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:34 smithi063 ceph-mon[31613]: daemon mds.i restarted 2024-05-27T08:21:35.144 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:34 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:21:35.144 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:34 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:21:35.144 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:34 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:21:35.144 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:34 smithi063 ceph-mon[31613]: osdmap e183: 12 total, 12 up, 12 in 2024-05-27T08:21:35.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:34 smithi086 ceph-mon[34687]: pgmap v1531: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2 op/s 2024-05-27T08:21:35.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:34 smithi086 ceph-mon[34687]: daemon mds.i restarted 2024-05-27T08:21:35.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:34 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:21:35.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:34 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:21:35.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:34 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:21:35.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:34 smithi086 ceph-mon[34687]: osdmap e183: 12 total, 12 up, 12 in 2024-05-27T08:21:35.546 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-27T08:21:35.547 INFO:tasks.fwd_scrub:done joining 2024-05-27T08:21:35.547 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-27T08:21:35.560 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T08:21:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.884+0000 7f4063200640 1 -- 172.21.15.63:0/193389854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406410c110 msgr2=0x7f4064105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.884+0000 7f4063200640 1 --2- 172.21.15.63:0/193389854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406410c110 0x7f4064105d50 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7f404c009ca0 tx=0x7f404c02a8c0 comp rx=0 tx=0).stop 2024-05-27T08:21:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.884+0000 7f4063200640 1 -- 172.21.15.63:0/193389854 shutdown_connections 2024-05-27T08:21:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.884+0000 7f4063200640 1 --2- 172.21.15.63:0/193389854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40641063c0 0x7f4064112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.884+0000 7f4063200640 1 --2- 172.21.15.63:0/193389854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f406410c110 0x7f4064105d50 unknown :-1 s=CLOSED pgs=3033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:35.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.884+0000 7f4063200640 1 --2- 172.21.15.63:0/193389854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f406410bc70 0x7f4064105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:35.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.884+0000 7f4063200640 1 -- 172.21.15.63:0/193389854 >> 172.21.15.63:0/193389854 conn(0x7f4064101360 msgr2=0x7f4064103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:35.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.884+0000 7f4063200640 1 -- 172.21.15.63:0/193389854 shutdown_connections 2024-05-27T08:21:35.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.884+0000 7f4063200640 1 -- 172.21.15.63:0/193389854 wait complete. 2024-05-27T08:21:35.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.886+0000 7f4063200640 1 Processor -- start 2024-05-27T08:21:35.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.886+0000 7f4063200640 1 -- start start 2024-05-27T08:21:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4063200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40641063c0 0x7f406419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4063200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f406410bc70 0x7f406419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4063200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f406410c110 0x7f40641a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4063200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4064115d60 con 0x7f40641063c0 2024-05-27T08:21:35.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4063200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4064115be0 con 0x7f406410bc70 2024-05-27T08:21:35.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4063200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4064115ee0 con 0x7f406410c110 2024-05-27T08:21:35.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4061e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40641063c0 0x7f406419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4061e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40641063c0 0x7f406419c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54410/0 (socket says 172.21.15.63:54410) 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4061e00640 1 -- 172.21.15.63:0/2545444836 learned_addr learned my addr 172.21.15.63:0/2545444836 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4062800640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f406410c110 0x7f40641a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.887+0000 7f4061400640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f406410bc70 0x7f406419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.888+0000 7f4061e00640 1 -- 172.21.15.63:0/2545444836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f406410c110 msgr2=0x7f40641a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.888+0000 7f4061e00640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f406410c110 0x7f40641a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.888+0000 7f4061e00640 1 -- 172.21.15.63:0/2545444836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f406410bc70 msgr2=0x7f406419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.888+0000 7f4061e00640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f406410bc70 0x7f406419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.888+0000 7f4061e00640 1 -- 172.21.15.63:0/2545444836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40641a1600 con 0x7f40641063c0 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.888+0000 7f4062800640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f406410c110 0x7f40641a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.888+0000 7f4061e00640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40641063c0 0x7f406419c7a0 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7f405400d770 tx=0x7f405400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.888+0000 7f405aa00640 1 -- 172.21.15.63:0/2545444836 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054017070 con 0x7f40641063c0 2024-05-27T08:21:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.889+0000 7f4063200640 1 -- 172.21.15.63:0/2545444836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40641a18f0 con 0x7f40641063c0 2024-05-27T08:21:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.889+0000 7f405aa00640 1 -- 172.21.15.63:0/2545444836 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4054009c30 con 0x7f40641063c0 2024-05-27T08:21:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.889+0000 7f4063200640 1 -- 172.21.15.63:0/2545444836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40641a9270 con 0x7f40641063c0 2024-05-27T08:21:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.889+0000 7f405aa00640 1 -- 172.21.15.63:0/2545444836 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054021060 con 0x7f40641063c0 2024-05-27T08:21:35.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.891+0000 7f4063200640 1 -- 172.21.15.63:0/2545444836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4064069d00 con 0x7f40641063c0 2024-05-27T08:21:35.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.893+0000 7f405aa00640 1 -- 172.21.15.63:0/2545444836 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f405401a070 con 0x7f40641063c0 2024-05-27T08:21:35.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.894+0000 7f405aa00640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40440b2950 0x7f40440b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:35.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.895+0000 7f405aa00640 1 -- 172.21.15.63:0/2545444836 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 9395+0+0 (secure 0 0 0) 0x7f40540e25e0 con 0x7f40641063c0 2024-05-27T08:21:35.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.895+0000 7f4061400640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40440b2950 0x7f40440b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:35.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.896+0000 7f4061400640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40440b2950 0x7f40440b4e60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f404c006010 tx=0x7f404c000f40 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:35.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:35.899+0000 7f405aa00640 1 -- 172.21.15.63:0/2545444836 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4054009de0 con 0x7f40641063c0 2024-05-27T08:21:35.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:35 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/1732997016,v1:172.21.15.149:6835/1732997016] up:boot 2024-05-27T08:21:35.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:35 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:35.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:35 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:21:35.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:35 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:35.964 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:35 smithi149 podman[81788]: 2024-05-27 08:21:35.564487863 +0000 UTC m=+0.718804296 container remove 8445e5551de7d108f9d07e31d13b3ee0d08cecbe8f35aeeb0fb386297a63af63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T08:21:35.964 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:35 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:36.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.114+0000 7f4063200640 1 -- 172.21.15.63:0/2545444836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40641a9440 con 0x7f40641063c0 2024-05-27T08:21:36.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.117+0000 7f405aa00640 1 -- 172.21.15.63:0/2545444836 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) ==== 74+0+25062 (secure 0 0 0) 0x7f40540a9bf0 con 0x7f40641063c0 2024-05-27T08:21:36.119 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:21:36.119 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":183,"fsid":"e24014b4-1bf8-11ef-bc9b-c7b262605968","created":"2024-05-27T07:16:52.281756+0000","modified":"2024-05-27T08:21:34.083306+0000","last_up_change":"2024-05-27T07:27:00.801245+0000","last_in_change":"2024-05-27T07:26:34.563663+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:21:25.773038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:28:19.257874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:28:23.339839+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":90,"snap_epoch":179,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:28:35.867485+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":90,"snap_epoch":179,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"032421af-e923-4b4a-beb0-2d42713538bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6803","nonce":1698060699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6805","nonce":1698060699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6809","nonce":1698060699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":1698060699},{"type":"v1","addr":"172.21.15.63:6807","nonce":1698060699}]},"public_addr":"172.21.15.63:6803/1698060699","cluster_addr":"172.21.15.63:6805/1698060699","heartbeat_back_addr":"172.21.15.63:6809/1698060699","heartbeat_front_addr":"172.21.15.63:6807/1698060699","state":["exists","up"]},{"osd":1,"uuid":"f57b3d89-2b35-4381-99a6-7db08bef64a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6801","nonce":1435942303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6803","nonce":1435942303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6807","nonce":1435942303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1435942303},{"type":"v1","addr":"172.21.15.86:6805","nonce":1435942303}]},"public_addr":"172.21.15.86:6801/1435942303","cluster_addr":"172.21.15.86:6803/1435942303","heartbeat_back_addr":"172.21.15.86:6807/1435942303","heartbeat_front_addr":"172.21.15.86:6805/1435942303","state":["exists","up"]},{"osd":2,"uuid":"025e966c-8833-4c12-a136-64d91cb0f4a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6801","nonce":363230187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6803","nonce":363230187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6807","nonce":363230187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":363230187},{"type":"v1","addr":"172.21.15.149:6805","nonce":363230187}]},"public_addr":"172.21.15.149:6801/363230187","cluster_addr":"172.21.15.149:6803/363230187","heartbeat_back_addr":"172.21.15.149:6807/363230187","heartbeat_front_addr":"172.21.15.149:6805/363230187","state":["exists","up"]},{"osd":3,"uuid":"e39fff31-e8c9-47f1-9029-e8763af5abe5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6811","nonce":3416673415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6813","nonce":3416673415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6817","nonce":3416673415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3416673415},{"type":"v1","addr":"172.21.15.63:6815","nonce":3416673415}]},"public_addr":"172.21.15.63:6811/3416673415","cluster_addr":"172.21.15.63:6813/3416673415","heartbeat_back_addr":"172.21.15.63:6817/3416673415","heartbeat_front_addr":"172.21.15.63:6815/3416673415","state":["exists","up"]},{"osd":4,"uuid":"b8b86405-4ec3-4853-9429-85a69f9a5bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6809","nonce":3874353435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6811","nonce":3874353435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6815","nonce":3874353435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3874353435},{"type":"v1","addr":"172.21.15.86:6813","nonce":3874353435}]},"public_addr":"172.21.15.86:6809/3874353435","cluster_addr":"172.21.15.86:6811/3874353435","heartbeat_back_addr":"172.21.15.86:6815/3874353435","heartbeat_front_addr":"172.21.15.86:6813/3874353435","state":["exists","up"]},{"osd":5,"uuid":"ff8d5a42-64bb-40f2-83f4-e22767f857f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6809","nonce":3863528283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6811","nonce":3863528283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6815","nonce":3863528283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3863528283},{"type":"v1","addr":"172.21.15.149:6813","nonce":3863528283}]},"public_addr":"172.21.15.149:6809/3863528283","cluster_addr":"172.21.15.149:6811/3863528283","heartbeat_back_addr":"172.21.15.149:6815/3863528283","heartbeat_front_addr":"172.21.15.149:6813/3863528283","state":["exists","up"]},{"osd":6,"uuid":"f496ab56-b7a8-47c0-bd34-fe6f2f82f69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6819","nonce":3628147489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6821","nonce":3628147489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6825","nonce":3628147489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3628147489},{"type":"v1","addr":"172.21.15.63:6823","nonce":3628147489}]},"public_addr":"172.21.15.63:6819/3628147489","cluster_addr":"172.21.15.63:6821/3628147489","heartbeat_back_addr":"172.21.15.63:6825/3628147489","heartbeat_front_addr":"172.21.15.63:6823/3628147489","state":["exists","up"]},{"osd":7,"uuid":"76b47ee5-4a59-4a08-9d10-f8cb104236d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6817","nonce":146162920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6819","nonce":146162920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6823","nonce":146162920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":146162920},{"type":"v1","addr":"172.21.15.86:6821","nonce":146162920}]},"public_addr":"172.21.15.86:6817/146162920","cluster_addr":"172.21.15.86:6819/146162920","heartbeat_back_addr":"172.21.15.86:6823/146162920","heartbeat_front_addr":"172.21.15.86:6821/146162920","state":["exists","up"]},{"osd":8,"uuid":"63112f81-efa0-4622-8eac-311302d0bca6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6817","nonce":2466164082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6819","nonce":2466164082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6823","nonce":2466164082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2466164082},{"type":"v1","addr":"172.21.15.149:6821","nonce":2466164082}]},"public_addr":"172.21.15.149:6817/2466164082","cluster_addr":"172.21.15.149:6819/2466164082","heartbeat_back_addr":"172.21.15.149:6823/2466164082","heartbeat_front_addr":"172.21.15.149:6821/2466164082","state":["exists","up"]},{"osd":9,"uuid":"ee506119-9cb2-4271-84da-ddcc6ca6cf09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6827","nonce":269558995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6829","nonce":269558995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6833","nonce":269558995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":269558995},{"type":"v1","addr":"172.21.15.63:6831","nonce":269558995}]},"public_addr":"172.21.15.63:6827/269558995","cluster_addr":"172.21.15.63:6829/269558995","heartbeat_back_addr":"172.21.15.63:6833/269558995","heartbeat_front_addr":"172.21.15.63:6831/269558995","state":["exists","up"]},{"osd":10,"uuid":"861ae487-365b-4de2-91e9-1d8254d36aaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6825","nonce":1317761853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6827","nonce":1317761853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6831","nonce":1317761853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1317761853},{"type":"v1","addr":"172.21.15.86:6829","nonce":1317761853}]},"public_addr":"172.21.15.86:6825/1317761853","cluster_addr":"172.21.15.86:6827/1317761853","heartbeat_back_addr":"172.21.15.86:6831/1317761853","heartbeat_front_addr":"172.21.15.86:6829/1317761853","state":["exists","up"]},{"osd":11,"uuid":"69396a1e-76b9-40c2-a289-392b3f2a5dc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6825","nonce":4045663443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6827","nonce":4045663443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6831","nonce":4045663443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":4045663443},{"type":"v1","addr":"172.21.15.149:6829","nonce":4045663443}]},"public_addr":"172.21.15.149:6825/4045663443","cluster_addr":"172.21.15.149:6827/4045663443","heartbeat_back_addr":"172.21.15.149:6831/4045663443","heartbeat_front_addr":"172.21.15.149:6829/4045663443","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:13.551038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:20:48.822802+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:21:23.510118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:00.721789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:22:37.487195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:13.851725+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:23:52.578281+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:24:29.643850+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:06.589897+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:25:44.826631+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:21.922809+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:26:58.657389+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6},{"from":3,"to":7},{"from":5,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:0/1961493183":"2024-05-28T07:33:35.220779+0000","172.21.15.63:6801/675305989":"2024-05-28T07:33:35.220779+0000","172.21.15.63:0/1202958531":"2024-05-28T07:33:35.220779+0000","172.21.15.63:6800/675305989":"2024-05-28T07:33:35.220779+0000","172.21.15.63:6801/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3946208673":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/3428263920":"2024-05-28T07:33:35.220779+0000","172.21.15.63:0/736953168":"2024-05-28T07:18:03.559305+0000","172.21.15.63:6801/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/1689640179":"2024-05-28T07:18:03.559305+0000","172.21.15.63:0/4210215351":"2024-05-28T07:17:27.459562+0000","172.21.15.63:6800/3806249841":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1558308201":"2024-05-28T07:33:35.220779+0000","172.21.15.63:0/2613878696":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/1318703057":"2024-05-28T07:17:27.459562+0000","172.21.15.63:0/58174792":"2024-05-28T07:18:03.559305+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:21:36.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 -- 172.21.15.63:0/2545444836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40440b2950 msgr2=0x7f40440b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:36.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40440b2950 0x7f40440b4e60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f404c006010 tx=0x7f404c000f40 comp rx=0 tx=0).stop 2024-05-27T08:21:36.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 -- 172.21.15.63:0/2545444836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40641063c0 msgr2=0x7f406419c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40641063c0 0x7f406419c7a0 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7f405400d770 tx=0x7f405400dc30 comp rx=0 tx=0).stop 2024-05-27T08:21:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 -- 172.21.15.63:0/2545444836 shutdown_connections 2024-05-27T08:21:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40440b2950 0x7f40440b4e60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f406410c110 0x7f40641a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f406410bc70 0x7f406419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 --2- 172.21.15.63:0/2545444836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40641063c0 0x7f406419c7a0 unknown :-1 s=CLOSED pgs=3034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 -- 172.21.15.63:0/2545444836 >> 172.21.15.63:0/2545444836 conn(0x7f4064101360 msgr2=0x7f4064111950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 -- 172.21.15.63:0/2545444836 shutdown_connections 2024-05-27T08:21:36.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.120+0000 7f4063200640 1 -- 172.21.15.63:0/2545444836 wait complete. 2024-05-27T08:21:36.145 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:35 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/1732997016,v1:172.21.15.149:6835/1732997016] up:boot 2024-05-27T08:21:36.146 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:35 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:36.146 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:35 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:21:36.146 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:35 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:36.149 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T07:21:25.773038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T07:28:19.257874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T07:28:23.339839+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '179', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 90, 'snap_epoch': 179, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T07:28:35.867485+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '179', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 90, 'snap_epoch': 179, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T08:21:36.150 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T08:21:36.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:35 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/1732997016,v1:172.21.15.149:6835/1732997016] up:boot 2024-05-27T08:21:36.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:35 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:36.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:35 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:21:36.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:35 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:36.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.449+0000 7ff4c3200640 1 -- 172.21.15.63:0/1565561423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4bc10c110 msgr2=0x7ff4bc105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:36.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.449+0000 7ff4c3200640 1 --2- 172.21.15.63:0/1565561423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4bc10c110 0x7ff4bc105d50 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009900 tx=0x7ff4b002a3f0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.449+0000 7ff4c3200640 1 -- 172.21.15.63:0/1565561423 shutdown_connections 2024-05-27T08:21:36.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.449+0000 7ff4c3200640 1 --2- 172.21.15.63:0/1565561423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc1063c0 0x7ff4bc112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.449+0000 7ff4c3200640 1 --2- 172.21.15.63:0/1565561423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4bc10c110 0x7ff4bc105d50 unknown :-1 s=CLOSED pgs=3035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.449+0000 7ff4c3200640 1 --2- 172.21.15.63:0/1565561423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4bc10bc70 0x7ff4bc105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.449+0000 7ff4c3200640 1 -- 172.21.15.63:0/1565561423 >> 172.21.15.63:0/1565561423 conn(0x7ff4bc07d590 msgr2=0x7ff4bc07d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:36.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.449+0000 7ff4c3200640 1 -- 172.21.15.63:0/1565561423 shutdown_connections 2024-05-27T08:21:36.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.449+0000 7ff4c3200640 1 -- 172.21.15.63:0/1565561423 wait complete. 2024-05-27T08:21:36.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c3200640 1 Processor -- start 2024-05-27T08:21:36.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c3200640 1 -- start start 2024-05-27T08:21:36.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c3200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4bc1063c0 0x7ff4bc19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc10bc70 0x7ff4bc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4bc10c110 0x7ff4bc1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c3200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc115d10 con 0x7ff4bc1063c0 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc115b90 con 0x7ff4bc10bc70 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc115e90 con 0x7ff4bc10c110 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4bc1063c0 0x7ff4bc19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c1e00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4bc1063c0 0x7ff4bc19c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54434/0 (socket says 172.21.15.63:54434) 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.451+0000 7ff4c1e00640 1 -- 172.21.15.63:0/4001954315 learned_addr learned my addr 172.21.15.63:0/4001954315 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.452+0000 7ff4c1400640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc10bc70 0x7ff4bc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.452+0000 7ff4c1e00640 1 -- 172.21.15.63:0/4001954315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4bc10c110 msgr2=0x7ff4bc1a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:36.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.452+0000 7ff4c1e00640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4bc10c110 0x7ff4bc1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.452+0000 7ff4c1e00640 1 -- 172.21.15.63:0/4001954315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc10bc70 msgr2=0x7ff4bc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.452+0000 7ff4c1e00640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc10bc70 0x7ff4bc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.452+0000 7ff4c1e00640 1 -- 172.21.15.63:0/4001954315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc1a1550 con 0x7ff4bc1063c0 2024-05-27T08:21:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.452+0000 7ff4c1400640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc10bc70 0x7ff4bc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:21:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.452+0000 7ff4c1e00640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4bc1063c0 0x7ff4bc19c770 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00c9a0 tx=0x7ff4ac00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.452+0000 7ff4b6a00640 1 -- 172.21.15.63:0/4001954315 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac014070 con 0x7ff4bc1063c0 2024-05-27T08:21:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.453+0000 7ff4b6a00640 1 -- 172.21.15.63:0/4001954315 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4ac004440 con 0x7ff4bc1063c0 2024-05-27T08:21:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.453+0000 7ff4b6a00640 1 -- 172.21.15.63:0/4001954315 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac003de0 con 0x7ff4bc1063c0 2024-05-27T08:21:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.453+0000 7ff4c3200640 1 -- 172.21.15.63:0/4001954315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc1a1840 con 0x7ff4bc1063c0 2024-05-27T08:21:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.453+0000 7ff4c3200640 1 -- 172.21.15.63:0/4001954315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc1a1d40 con 0x7ff4bc1063c0 2024-05-27T08:21:36.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.454+0000 7ff4c3200640 1 -- 172.21.15.63:0/4001954315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4bc06a150 con 0x7ff4bc1063c0 2024-05-27T08:21:36.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.458+0000 7ff4b6a00640 1 -- 172.21.15.63:0/4001954315 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff4ac017070 con 0x7ff4bc1063c0 2024-05-27T08:21:36.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.459+0000 7ff4b6a00640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4a40b2950 0x7ff4a40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:36.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.459+0000 7ff4b6a00640 1 -- 172.21.15.63:0/4001954315 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 9395+0+0 (secure 0 0 0) 0x7ff4ac0e2c40 con 0x7ff4bc1063c0 2024-05-27T08:21:36.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.459+0000 7ff4c1400640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4a40b2950 0x7ff4a40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:36.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.460+0000 7ff4c1400640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4a40b2950 0x7ff4a40b4e60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0006010 tx=0x7ff4b0000bc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:36.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.461+0000 7ff4b6a00640 1 -- 172.21.15.63:0/4001954315 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff4ac01e020 con 0x7ff4bc1063c0 2024-05-27T08:21:36.464 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:36 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-05-27T08:21:36.464 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:36 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 1.452s CPU time. 2024-05-27T08:21:36.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.679+0000 7ff4c3200640 1 -- 172.21.15.63:0/4001954315 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4bc1a9400 con 0x7ff4bc1063c0 2024-05-27T08:21:36.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.698+0000 7ff4b6a00640 1 -- 172.21.15.63:0/4001954315 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) ==== 93+0+10 (secure 0 0 0) 0x7ff4ac0aa250 con 0x7ff4bc1063c0 2024-05-27T08:21:36.699 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-05-27T08:21:36.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 -- 172.21.15.63:0/4001954315 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4a40b2950 msgr2=0x7ff4a40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:36.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4a40b2950 0x7ff4a40b4e60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0006010 tx=0x7ff4b0000bc0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 -- 172.21.15.63:0/4001954315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4bc1063c0 msgr2=0x7ff4bc19c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:36.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4bc1063c0 0x7ff4bc19c770 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00c9a0 tx=0x7ff4ac00ce60 comp rx=0 tx=0).stop 2024-05-27T08:21:36.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 -- 172.21.15.63:0/4001954315 shutdown_connections 2024-05-27T08:21:36.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7ff4a40b2950 0x7ff4a40b4e60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4bc10c110 0x7ff4bc1a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4bc10bc70 0x7ff4bc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 --2- 172.21.15.63:0/4001954315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4bc1063c0 0x7ff4bc19c770 unknown :-1 s=CLOSED pgs=3036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:36.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 -- 172.21.15.63:0/4001954315 >> 172.21.15.63:0/4001954315 conn(0x7ff4bc07d590 msgr2=0x7ff4bc104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:36.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 -- 172.21.15.63:0/4001954315 shutdown_connections 2024-05-27T08:21:36.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:36.701+0000 7ff4c3200640 1 -- 172.21.15.63:0/4001954315 wait complete. 2024-05-27T08:21:36.730 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T08:21:37.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.021+0000 7fa700c00640 1 -- 172.21.15.63:0/2807576940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6fc10c110 msgr2=0x7fa6fc105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:37.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.021+0000 7fa700c00640 1 --2- 172.21.15.63:0/2807576940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6fc10c110 0x7fa6fc105d50 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009900 tx=0x7fa6e402a3f0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.022+0000 7fa700c00640 1 -- 172.21.15.63:0/2807576940 shutdown_connections 2024-05-27T08:21:37.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.022+0000 7fa700c00640 1 --2- 172.21.15.63:0/2807576940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6fc1063c0 0x7fa6fc112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.022+0000 7fa700c00640 1 --2- 172.21.15.63:0/2807576940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6fc10c110 0x7fa6fc105d50 unknown :-1 s=CLOSED pgs=3037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.022+0000 7fa700c00640 1 --2- 172.21.15.63:0/2807576940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc10bc70 0x7fa6fc105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.022+0000 7fa700c00640 1 -- 172.21.15.63:0/2807576940 >> 172.21.15.63:0/2807576940 conn(0x7fa6fc101360 msgr2=0x7fa6fc103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:37.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.022+0000 7fa700c00640 1 -- 172.21.15.63:0/2807576940 shutdown_connections 2024-05-27T08:21:37.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.022+0000 7fa700c00640 1 -- 172.21.15.63:0/2807576940 wait complete. 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.023+0000 7fa700c00640 1 Processor -- start 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.023+0000 7fa700c00640 1 -- start start 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.023+0000 7fa700c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6fc1063c0 0x7fa6fc19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.023+0000 7fa700c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc10bc70 0x7fa6fc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.023+0000 7fa700c00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6fc10c110 0x7fa6fc1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.023+0000 7fa700c00640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc115d10 con 0x7fa6fc10c110 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.023+0000 7fa700c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc115b90 con 0x7fa6fc10bc70 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.023+0000 7fa700c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc115e90 con 0x7fa6fc1063c0 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6fbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6fc10c110 0x7fa6fc1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6fbe00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6fc10c110 0x7fa6fc1a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54454/0 (socket says 172.21.15.63:54454) 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6fbe00640 1 -- 172.21.15.63:0/3323466892 learned_addr learned my addr 172.21.15.63:0/3323466892 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:37.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6f3e00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc10bc70 0x7fa6fc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:37.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6fb400640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6fc1063c0 0x7fa6fc19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:37.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6fbe00640 1 -- 172.21.15.63:0/3323466892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6fc1063c0 msgr2=0x7fa6fc19c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:37.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6fbe00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6fc1063c0 0x7fa6fc19c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6fbe00640 1 -- 172.21.15.63:0/3323466892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc10bc70 msgr2=0x7fa6fc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:37.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6fbe00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc10bc70 0x7fa6fc19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6fbe00640 1 -- 172.21.15.63:0/3323466892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc1a1630 con 0x7fa6fc10c110 2024-05-27T08:21:37.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.024+0000 7fa6f3e00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc10bc70 0x7fa6fc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:37.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.025+0000 7fa6fbe00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6fc10c110 0x7fa6fc1a0ff0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400eaa0 tx=0x7fa6f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:37.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.025+0000 7fa6f8c00640 1 -- 172.21.15.63:0/3323466892 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400cdb0 con 0x7fa6fc10c110 2024-05-27T08:21:37.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.025+0000 7fa6f8c00640 1 -- 172.21.15.63:0/3323466892 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa6f4005020 con 0x7fa6fc10c110 2024-05-27T08:21:37.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.025+0000 7fa700c00640 1 -- 172.21.15.63:0/3323466892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc1a1920 con 0x7fa6fc10c110 2024-05-27T08:21:37.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.025+0000 7fa6f8c00640 1 -- 172.21.15.63:0/3323466892 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40116d0 con 0x7fa6fc10c110 2024-05-27T08:21:37.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.025+0000 7fa700c00640 1 -- 172.21.15.63:0/3323466892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc1a92a0 con 0x7fa6fc10c110 2024-05-27T08:21:37.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.026+0000 7fa700c00640 1 -- 172.21.15.63:0/3323466892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6fc06a150 con 0x7fa6fc10c110 2024-05-27T08:21:37.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.029+0000 7fa6f8c00640 1 -- 172.21.15.63:0/3323466892 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa6f40051d0 con 0x7fa6fc10c110 2024-05-27T08:21:37.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.031+0000 7fa6f8c00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6e00b29d0 0x7fa6e00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:37.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.031+0000 7fa6f8c00640 1 -- 172.21.15.63:0/3323466892 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 9395+0+0 (secure 0 0 0) 0x7fa6f40e2fa0 con 0x7fa6fc10c110 2024-05-27T08:21:37.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.031+0000 7fa6fb400640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6e00b29d0 0x7fa6e00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:37.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.032+0000 7fa6fb400640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6e00b29d0 0x7fa6e00b4ee0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec005fd0 tx=0x7fa6ec005850 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:37.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.034+0000 7fa6f8c00640 1 -- 172.21.15.63:0/3323466892 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6f4016030 con 0x7fa6fc10c110 2024-05-27T08:21:37.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:36 smithi086 ceph-mon[34687]: pgmap v1533: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2 op/s 2024-05-27T08:21:37.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:36 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2545444836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:21:37.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:36 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4001954315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:21:37.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:36 smithi149 ceph-mon[35408]: pgmap v1533: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2 op/s 2024-05-27T08:21:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:36 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2545444836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:21:37.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:36 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4001954315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:21:37.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:36 smithi063 ceph-mon[31613]: pgmap v1533: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2 op/s 2024-05-27T08:21:37.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:36 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2545444836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:21:37.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:36 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4001954315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:21:37.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.250+0000 7fa700c00640 1 -- 172.21.15.63:0/3323466892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa6fc1a9470 con 0x7fa6fc10c110 2024-05-27T08:21:37.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.251+0000 7fa6f8c00640 1 -- 172.21.15.63:0/3323466892 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) ==== 104+0+11 (secure 0 0 0) 0x7fa6f40aa630 con 0x7fa6fc10c110 2024-05-27T08:21:37.253 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-27T08:21:37.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.254+0000 7fa700c00640 1 -- 172.21.15.63:0/3323466892 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6e00b29d0 msgr2=0x7fa6e00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:37.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.254+0000 7fa700c00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6e00b29d0 0x7fa6e00b4ee0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec005fd0 tx=0x7fa6ec005850 comp rx=0 tx=0).stop 2024-05-27T08:21:37.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.254+0000 7fa700c00640 1 -- 172.21.15.63:0/3323466892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6fc10c110 msgr2=0x7fa6fc1a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:37.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.254+0000 7fa700c00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6fc10c110 0x7fa6fc1a0ff0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400eaa0 tx=0x7fa6f400ef60 comp rx=0 tx=0).stop 2024-05-27T08:21:37.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.255+0000 7fa700c00640 1 -- 172.21.15.63:0/3323466892 shutdown_connections 2024-05-27T08:21:37.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.255+0000 7fa700c00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fa6e00b29d0 0x7fa6e00b4ee0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.255+0000 7fa700c00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6fc10c110 0x7fa6fc1a0ff0 unknown :-1 s=CLOSED pgs=3038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.255+0000 7fa700c00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6fc10bc70 0x7fa6fc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.255+0000 7fa700c00640 1 --2- 172.21.15.63:0/3323466892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6fc1063c0 0x7fa6fc19c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.255+0000 7fa700c00640 1 -- 172.21.15.63:0/3323466892 >> 172.21.15.63:0/3323466892 conn(0x7fa6fc101360 msgr2=0x7fa6fc111920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:37.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.255+0000 7fa700c00640 1 -- 172.21.15.63:0/3323466892 shutdown_connections 2024-05-27T08:21:37.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.255+0000 7fa700c00640 1 -- 172.21.15.63:0/3323466892 wait complete. 2024-05-27T08:21:37.285 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T08:21:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.564+0000 7f0ba2400640 1 -- 172.21.15.63:0/2537257458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b9c10c2f0 msgr2=0x7f0b9c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:37.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.564+0000 7f0ba2400640 1 --2- 172.21.15.63:0/2537257458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b9c10c2f0 0x7f0b9c105f30 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009900 tx=0x7f0b8402a3f0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.564+0000 7f0ba2400640 1 -- 172.21.15.63:0/2537257458 shutdown_connections 2024-05-27T08:21:37.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.564+0000 7f0ba2400640 1 --2- 172.21.15.63:0/2537257458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b9c1065a0 0x7f0b9c112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.564+0000 7f0ba2400640 1 --2- 172.21.15.63:0/2537257458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b9c10c2f0 0x7f0b9c105f30 unknown :-1 s=CLOSED pgs=3039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.564+0000 7f0ba2400640 1 --2- 172.21.15.63:0/2537257458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b9c10be50 0x7f0b9c1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.564+0000 7f0ba2400640 1 -- 172.21.15.63:0/2537257458 >> 172.21.15.63:0/2537257458 conn(0x7f0b9c1014d0 msgr2=0x7f0b9c103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:37.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.565+0000 7f0ba2400640 1 -- 172.21.15.63:0/2537257458 shutdown_connections 2024-05-27T08:21:37.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.565+0000 7f0ba2400640 1 -- 172.21.15.63:0/2537257458 wait complete. 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.566+0000 7f0ba2400640 1 Processor -- start 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.566+0000 7f0ba2400640 1 -- start start 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.566+0000 7f0ba2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b9c1065a0 0x7f0b9c1a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.566+0000 7f0ba2400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b9c10be50 0x7f0b9c1a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.566+0000 7f0ba2400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b9c10c2f0 0x7f0b9c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.566+0000 7f0ba2400640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c115f50 con 0x7f0b9c10be50 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.566+0000 7f0ba2400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c115dd0 con 0x7f0b9c1065a0 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.566+0000 7f0ba2400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1160d0 con 0x7f0b9c10c2f0 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.567+0000 7f0b92a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b9c10be50 0x7f0b9c1a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.567+0000 7f0b92a00640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b9c10be50 0x7f0b9c1a2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54472/0 (socket says 172.21.15.63:54472) 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.567+0000 7f0b92a00640 1 -- 172.21.15.63:0/4244667053 learned_addr learned my addr 172.21.15.63:0/4244667053 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:37.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.567+0000 7f0b9b400640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b9c1065a0 0x7f0b9c1a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:37.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.567+0000 7f0b92a00640 1 -- 172.21.15.63:0/4244667053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b9c10c2f0 msgr2=0x7f0b9c19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:37.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.567+0000 7f0b92a00640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b9c10c2f0 0x7f0b9c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.567+0000 7f0b92a00640 1 -- 172.21.15.63:0/4244667053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b9c1065a0 msgr2=0x7f0b9c1a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:37.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.567+0000 7f0b92a00640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b9c1065a0 0x7f0b9c1a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.567+0000 7f0b92a00640 1 -- 172.21.15.63:0/4244667053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9c19cf10 con 0x7f0b9c10be50 2024-05-27T08:21:37.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.568+0000 7f0b92a00640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b9c10be50 0x7f0b9c1a2ba0 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7f0b84005940 tx=0x7f0b840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:37.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.568+0000 7f0b98c00640 1 -- 172.21.15.63:0/4244667053 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8403f030 con 0x7f0b9c10be50 2024-05-27T08:21:37.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.568+0000 7f0b98c00640 1 -- 172.21.15.63:0/4244667053 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b84003c90 con 0x7f0b9c10be50 2024-05-27T08:21:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.568+0000 7f0ba2400640 1 -- 172.21.15.63:0/4244667053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9c19d1a0 con 0x7f0b9c10be50 2024-05-27T08:21:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.568+0000 7f0b98c00640 1 -- 172.21.15.63:0/4244667053 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84035070 con 0x7f0b9c10be50 2024-05-27T08:21:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.568+0000 7f0ba2400640 1 -- 172.21.15.63:0/4244667053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9c1a94e0 con 0x7f0b9c10be50 2024-05-27T08:21:37.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.569+0000 7f0ba2400640 1 -- 172.21.15.63:0/4244667053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b9c06a280 con 0x7f0b9c10be50 2024-05-27T08:21:37.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.573+0000 7f0b98c00640 1 -- 172.21.15.63:0/4244667053 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0b84042030 con 0x7f0b9c10be50 2024-05-27T08:21:37.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.575+0000 7f0b98c00640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0b800b29e0 0x7f0b800b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.575+0000 7f0b98c00640 1 -- 172.21.15.63:0/4244667053 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 9395+0+0 (secure 0 0 0) 0x7f0b841033a0 con 0x7f0b9c10be50 2024-05-27T08:21:37.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.575+0000 7f0b9b400640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0b800b29e0 0x7f0b800b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:37.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.576+0000 7f0b9b400640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0b800b29e0 0x7f0b800b4ef0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c005c40 tx=0x7f0b8c00a3b0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:37.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.578+0000 7f0b98c00640 1 -- 172.21.15.63:0/4244667053 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b840ca9b0 con 0x7f0b9c10be50 2024-05-27T08:21:37.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.793+0000 7f0ba2400640 1 -- 172.21.15.63:0/4244667053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0b9c1a96b0 con 0x7f0b9c10be50 2024-05-27T08:21:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.798+0000 7f0b98c00640 1 -- 172.21.15.63:0/4244667053 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) ==== 100+0+11 (secure 0 0 0) 0x7f0b840ca2b0 con 0x7f0b9c10be50 2024-05-27T08:21:37.799 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 64 2024-05-27T08:21:37.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 -- 172.21.15.63:0/4244667053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0b800b29e0 msgr2=0x7f0b800b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0b800b29e0 0x7f0b800b4ef0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c005c40 tx=0x7f0b8c00a3b0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 -- 172.21.15.63:0/4244667053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b9c10be50 msgr2=0x7f0b9c1a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b9c10be50 0x7f0b9c1a2ba0 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7f0b84005940 tx=0x7f0b840042f0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 -- 172.21.15.63:0/4244667053 shutdown_connections 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f0b800b29e0 0x7f0b800b4ef0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b9c10c2f0 0x7f0b9c19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b9c10be50 0x7f0b9c1a2ba0 unknown :-1 s=CLOSED pgs=3040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 --2- 172.21.15.63:0/4244667053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b9c1065a0 0x7f0b9c1a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 -- 172.21.15.63:0/4244667053 >> 172.21.15.63:0/4244667053 conn(0x7f0b9c1014d0 msgr2=0x7f0b9c111950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 -- 172.21.15.63:0/4244667053 shutdown_connections 2024-05-27T08:21:37.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:37.801+0000 7f0ba2400640 1 -- 172.21.15.63:0/4244667053 wait complete. 2024-05-27T08:21:37.830 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T08:21:38.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.111+0000 7f4193600640 1 -- 172.21.15.63:0/1368365744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418c10f520 msgr2=0x7f418c116cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:38.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.112+0000 7f4193600640 1 --2- 172.21.15.63:0/1368365744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418c10f520 0x7f418c116cc0 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7f4184009ca0 tx=0x7f418402a8c0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.112+0000 7f4193600640 1 -- 172.21.15.63:0/1368365744 shutdown_connections 2024-05-27T08:21:38.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.112+0000 7f4193600640 1 --2- 172.21.15.63:0/1368365744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418c10f520 0x7f418c116cc0 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.112+0000 7f4193600640 1 --2- 172.21.15.63:0/1368365744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f418c10ea60 0x7f418c10ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.112+0000 7f4193600640 1 --2- 172.21.15.63:0/1368365744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418c10a4d0 0x7f418c10a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.112+0000 7f4193600640 1 -- 172.21.15.63:0/1368365744 >> 172.21.15.63:0/1368365744 conn(0x7f418c077110 msgr2=0x7f418c077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:38.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.112+0000 7f4193600640 1 -- 172.21.15.63:0/1368365744 shutdown_connections 2024-05-27T08:21:38.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.112+0000 7f4193600640 1 -- 172.21.15.63:0/1368365744 wait complete. 2024-05-27T08:21:38.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.113+0000 7f4193600640 1 Processor -- start 2024-05-27T08:21:38.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.114+0000 7f4193600640 1 -- start start 2024-05-27T08:21:38.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.114+0000 7f4193600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f418c10a4d0 0x7f418c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:38.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.114+0000 7f4193600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418c10ea60 0x7f418c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:38.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.114+0000 7f4193600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418c10f520 0x7f418c1a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:38.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.114+0000 7f4193600640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f418c11a330 con 0x7f418c10f520 2024-05-27T08:21:38.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.114+0000 7f4193600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f418c11a1b0 con 0x7f418c10ea60 2024-05-27T08:21:38.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.114+0000 7f4193600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f418c11a4b0 con 0x7f418c10a4d0 2024-05-27T08:21:38.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f4191600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418c10f520 0x7f418c1a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:38.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f4191600640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418c10f520 0x7f418c1a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54498/0 (socket says 172.21.15.63:54498) 2024-05-27T08:21:38.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f4191600640 1 -- 172.21.15.63:0/2882211734 learned_addr learned my addr 172.21.15.63:0/2882211734 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:38.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f4190c00640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f418c10a4d0 0x7f418c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:38.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f418be00640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418c10ea60 0x7f418c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:38.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f4191600640 1 -- 172.21.15.63:0/2882211734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f418c10a4d0 msgr2=0x7f418c1a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:38.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f4191600640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f418c10a4d0 0x7f418c1a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f4191600640 1 -- 172.21.15.63:0/2882211734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418c10ea60 msgr2=0x7f418c1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:38.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f4191600640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418c10ea60 0x7f418c1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f4191600640 1 -- 172.21.15.63:0/2882211734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418c1a5c30 con 0x7f418c10f520 2024-05-27T08:21:38.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.115+0000 7f418be00640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418c10ea60 0x7f418c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:38.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.116+0000 7f4191600640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418c10f520 0x7f418c1a55f0 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f4184005cc0 tx=0x7f4184003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:38.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.116+0000 7f4189600640 1 -- 172.21.15.63:0/2882211734 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184032c10 con 0x7f418c10f520 2024-05-27T08:21:38.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.116+0000 7f4189600640 1 -- 172.21.15.63:0/2882211734 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4184032db0 con 0x7f418c10f520 2024-05-27T08:21:38.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.116+0000 7f4193600640 1 -- 172.21.15.63:0/2882211734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418c1a5ec0 con 0x7f418c10f520 2024-05-27T08:21:38.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.116+0000 7f4189600640 1 -- 172.21.15.63:0/2882211734 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184030a60 con 0x7f418c10f520 2024-05-27T08:21:38.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.116+0000 7f4193600640 1 -- 172.21.15.63:0/2882211734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f418c1a6410 con 0x7f418c10f520 2024-05-27T08:21:38.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.117+0000 7f4193600640 1 -- 172.21.15.63:0/2882211734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f418c06a280 con 0x7f418c10f520 2024-05-27T08:21:38.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.120+0000 7f4189600640 1 -- 172.21.15.63:0/2882211734 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4184030c00 con 0x7f418c10f520 2024-05-27T08:21:38.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.122+0000 7f4189600640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f41740b2960 0x7f41740b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:38.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.122+0000 7f4189600640 1 -- 172.21.15.63:0/2882211734 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 9395+0+0 (secure 0 0 0) 0x7f4184103e40 con 0x7f418c10f520 2024-05-27T08:21:38.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.122+0000 7f4190c00640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f41740b2960 0x7f41740b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:38.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.123+0000 7f4190c00640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f41740b2960 0x7f41740b4e70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4180009b60 tx=0x7f4180005f90 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:38.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.128+0000 7f4189600640 1 -- 172.21.15.63:0/2882211734 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f41840cb5c0 con 0x7f418c10f520 2024-05-27T08:21:38.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:37 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/3323466892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T08:21:38.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:37 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/3323466892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T08:21:38.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:37 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/3323466892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T08:21:38.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.346+0000 7f4193600640 1 -- 172.21.15.63:0/2882211734 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f418c112340 con 0x7f418c10f520 2024-05-27T08:21:38.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.347+0000 7f4189600640 1 -- 172.21.15.63:0/2882211734 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v183) ==== 103+0+11 (secure 0 0 0) 0x7f41840098b0 con 0x7f418c10f520 2024-05-27T08:21:38.347 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-05-27T08:21:38.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.349+0000 7f4193600640 1 -- 172.21.15.63:0/2882211734 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f41740b2960 msgr2=0x7f41740b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.349+0000 7f4193600640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f41740b2960 0x7f41740b4e70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4180009b60 tx=0x7f4180005f90 comp rx=0 tx=0).stop 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.350+0000 7f4193600640 1 -- 172.21.15.63:0/2882211734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418c10f520 msgr2=0x7f418c1a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.350+0000 7f4193600640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418c10f520 0x7f418c1a55f0 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f4184005cc0 tx=0x7f4184003f90 comp rx=0 tx=0).stop 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.350+0000 7f4193600640 1 -- 172.21.15.63:0/2882211734 shutdown_connections 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.350+0000 7f4193600640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f41740b2960 0x7f41740b4e70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.350+0000 7f4193600640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f418c10f520 0x7f418c1a55f0 unknown :-1 s=CLOSED pgs=3042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.350+0000 7f4193600640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418c10ea60 0x7f418c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.350+0000 7f4193600640 1 --2- 172.21.15.63:0/2882211734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f418c10a4d0 0x7f418c1a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.350+0000 7f4193600640 1 -- 172.21.15.63:0/2882211734 >> 172.21.15.63:0/2882211734 conn(0x7f418c077110 msgr2=0x7f418c07a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.350+0000 7f4193600640 1 -- 172.21.15.63:0/2882211734 shutdown_connections 2024-05-27T08:21:38.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.350+0000 7f4193600640 1 -- 172.21.15.63:0/2882211734 wait complete. 2024-05-27T08:21:38.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-27T08:21:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.655+0000 7f09c7200640 1 -- 172.21.15.63:0/205020030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09c0109a60 msgr2=0x7f09c0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.655+0000 7f09c7200640 1 --2- 172.21.15.63:0/205020030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09c0109a60 0x7f09c0103610 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009900 tx=0x7f09b402a3f0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.655+0000 7f09c7200640 1 -- 172.21.15.63:0/205020030 shutdown_connections 2024-05-27T08:21:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.655+0000 7f09c7200640 1 --2- 172.21.15.63:0/205020030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09c01041c0 0x7f09c01104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.655+0000 7f09c7200640 1 --2- 172.21.15.63:0/205020030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09c0109f00 0x7f09c0103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.655+0000 7f09c7200640 1 --2- 172.21.15.63:0/205020030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09c0109a60 0x7f09c0103610 unknown :-1 s=CLOSED pgs=3043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.655+0000 7f09c7200640 1 -- 172.21.15.63:0/205020030 >> 172.21.15.63:0/205020030 conn(0x7f09c007a1e0 msgr2=0x7f09c007a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.655+0000 7f09c7200640 1 -- 172.21.15.63:0/205020030 shutdown_connections 2024-05-27T08:21:38.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.655+0000 7f09c7200640 1 -- 172.21.15.63:0/205020030 wait complete. 2024-05-27T08:21:38.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.656+0000 7f09c7200640 1 Processor -- start 2024-05-27T08:21:38.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.656+0000 7f09c7200640 1 -- start start 2024-05-27T08:21:38.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c7200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09c01041c0 0x7f09c0074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:38.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c7200640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09c0109a60 0x7f09c0075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:38.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09c0109f00 0x7f09c0075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:38.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c7200640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0113b10 con 0x7f09c0109a60 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0113990 con 0x7f09c0109f00 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c7200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0113c90 con 0x7f09c01041c0 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09c0109a60 0x7f09c0075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c5400640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09c0109a60 0x7f09c0075500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:54522/0 (socket says 172.21.15.63:54522) 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c5400640 1 -- 172.21.15.63:0/997490739 learned_addr learned my addr 172.21.15.63:0/997490739 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c5e00640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09c01041c0 0x7f09c0074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c6800640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09c0109f00 0x7f09c0075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c5400640 1 -- 172.21.15.63:0/997490739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09c01041c0 msgr2=0x7f09c0074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c5400640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09c01041c0 0x7f09c0074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c5400640 1 -- 172.21.15.63:0/997490739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09c0109f00 msgr2=0x7f09c0075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c5400640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09c0109f00 0x7f09c0075a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c5400640 1 -- 172.21.15.63:0/997490739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c01a8950 con 0x7f09c0109a60 2024-05-27T08:21:38.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.657+0000 7f09c5e00640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09c01041c0 0x7f09c0074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:21:38.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.658+0000 7f09c5400640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09c0109a60 0x7f09c0075500 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7f09b0005d00 tx=0x7f09b000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:38.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.658+0000 7f09baa00640 1 -- 172.21.15.63:0/997490739 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0017070 con 0x7f09c0109a60 2024-05-27T08:21:38.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.658+0000 7f09c7200640 1 -- 172.21.15.63:0/997490739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c01a8c40 con 0x7f09c0109a60 2024-05-27T08:21:38.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.658+0000 7f09baa00640 1 -- 172.21.15.63:0/997490739 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09b0009c30 con 0x7f09c0109a60 2024-05-27T08:21:38.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.658+0000 7f09baa00640 1 -- 172.21.15.63:0/997490739 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b000f620 con 0x7f09c0109a60 2024-05-27T08:21:38.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.658+0000 7f09c7200640 1 -- 172.21.15.63:0/997490739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c01a9190 con 0x7f09c0109a60 2024-05-27T08:21:38.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.659+0000 7f09c7200640 1 -- 172.21.15.63:0/997490739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c0104d50 con 0x7f09c0109a60 2024-05-27T08:21:38.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.663+0000 7f09baa00640 1 -- 172.21.15.63:0/997490739 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f09b001a070 con 0x7f09c0109a60 2024-05-27T08:21:38.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.665+0000 7f09baa00640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09a80b2950 0x7f09a80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:21:38.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.665+0000 7f09baa00640 1 -- 172.21.15.63:0/997490739 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 9395+0+0 (secure 0 0 0) 0x7f09b00e2460 con 0x7f09c0109a60 2024-05-27T08:21:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.665+0000 7f09c5e00640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09a80b2950 0x7f09a80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:21:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.666+0000 7f09c5e00640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09a80b2950 0x7f09a80b4e60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f09b40057c0 tx=0x7f09b4009680 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:21:38.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.670+0000 7f09baa00640 1 -- 172.21.15.63:0/997490739 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09b0009de0 con 0x7f09c0109a60 2024-05-27T08:21:38.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.939+0000 7f09c7200640 1 -- 172.21.15.63:0/997490739 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f09c00008d0 con 0x7f09c0109a60 2024-05-27T08:21:38.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.965+0000 7f09baa00640 1 -- 172.21.15.63:0/997490739 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816173 (secure 0 0 0) 0x7f09b00a9a70 con 0x7f09c0109a60 2024-05-27T08:21:38.977 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-27T08:21:38.977 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 26, 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: "active_gid": 15216, 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: "active_name": "x", 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: "active_addrs": { 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6800", 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1343315489 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:38.978 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6801", 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1343315489 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: "active_addr": "172.21.15.63:6801/1343315489", 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: "active_change": "2024-05-27T07:33:35.221385+0000", 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: "active_mgr_features": 4540701547738038271, 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: "available": true, 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: "standbys": [ 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: "gid": 24970, 2024-05-27T08:21:38.979 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "z", 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_features": 4540701547738038271, 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "available_modules": [ 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "alerts", 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.980 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:38.981 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:38.982 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.983 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.984 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.985 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.986 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_destination": { 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_destination", 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.987 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email address to send alerts to", 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_from_name": { 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_from_name", 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "Ceph", 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.988 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email From: name", 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_host": { 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_host", 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.989 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP server", 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_password": { 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_password", 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.990 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Password to authenticate with", 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_port": { 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_port", 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "465", 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.991 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP port", 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_sender": { 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_sender", 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.992 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP envelope sender", 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_ssl": { 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_ssl", 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.993 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_user": { 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_user", 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.994 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "User to authenticate as", 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.995 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "balancer", 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "active": { 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "active", 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:38.997 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_time": { 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_time", 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0000", 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T08:21:38.998 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_weekday": { 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_weekday", 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:38.999 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_max_iterations": { 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_max_iterations", 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "25", 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "250", 2024-05-27T08:21:39.000 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_metrics": { 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_metrics", 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.001 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_step": { 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_step", 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.5", 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0.001", 2024-05-27T08:21:39.002 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "0.999", 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "aggressiveness of optimization", 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "end_time": { 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_time", 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2359", 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.003 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "end_weekday": { 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_weekday", 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.004 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.006 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.007 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.008 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.009 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.010 INFO:teuthology.orchestra.run.smithi063.stdout: "min_score": { 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_score", 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "upmap", 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "crush-compat", 2024-05-27T08:21:39.012 INFO:teuthology.orchestra.run.smithi063.stdout: "none", 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "read", 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap", 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap-read" 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Balancer mode", 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_ids": { 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_ids", 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.013 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:39.014 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.015 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_deviation": { 2024-05-27T08:21:39.016 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_deviation", 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.017 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_optimizations": { 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_optimizations", 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.018 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm", 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_down_multiplier": { 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_down_multiplier", 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-27T08:21:39.019 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_refresh_rate": { 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_refresh_rate", 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.020 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "20", 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_starting_port": { 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_starting_port", 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.021 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4721", 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_ptrace": { 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_ptrace", 2024-05-27T08:21:39.022 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_interval": { 2024-05-27T08:21:39.023 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_interval", 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.024 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_memory_target_ratio": { 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.7", 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm_log_destination": { 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm_log_destination", 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "file", 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "file,syslog", 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "syslog" 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.026 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "cgroups_split": { 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cgroups_split", 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.027 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "config_checks_enabled": { 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_checks_enabled", 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.028 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "config_dashboard": { 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_dashboard", 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.029 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_alertmanager": { 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_alertmanager", 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.030 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_base": { 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_base", 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.031 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Container image name, without the tag", 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_elasticsearch": { 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_elasticsearch", 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.032 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "elasticsearch container image", 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_grafana": { 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_grafana", 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T08:21:39.033 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_haproxy": { 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_haproxy", 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T08:21:39.034 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "HAproxy container image", 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_agent": { 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_agent", 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.035 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger agent container image", 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_collector": { 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_collector", 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.036 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger collector container image", 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_query": { 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_query", 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.037 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger query container image", 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_keepalived": { 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_keepalived", 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.038 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Keepalived container image", 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_loki": { 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_loki", 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.039 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Loki container image", 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_node_exporter": { 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_node_exporter", 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T08:21:39.040 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_nvmeof": { 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_nvmeof", 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T08:21:39.041 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Nvme-of container image", 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_prometheus": { 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_prometheus", 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T08:21:39.042 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_promtail": { 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_promtail", 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.043 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Promtail container image", 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_samba": { 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_samba", 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.044 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Samba/SMB container image", 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_snmp_gateway": { 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_snmp_gateway", 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.045 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SNMP Gateway container image", 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "container_init": { 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_init", 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.046 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "daemon_cache_timeout": { 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "daemon_cache_timeout", 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.047 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "default_cephadm_command_timeout": { 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T08:21:39.048 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "900", 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "default_registry": { 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_registry", 2024-05-27T08:21:39.049 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io", 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "device_cache_timeout": { 2024-05-27T08:21:39.050 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_cache_timeout", 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1800", 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache device inventory", 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "device_enhanced_scan": { 2024-05-27T08:21:39.051 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_enhanced_scan", 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "facts_cache_timeout": { 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "facts_cache_timeout", 2024-05-27T08:21:39.052 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache host facts data", 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "grafana_dashboards_path": { 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "grafana_dashboards_path", 2024-05-27T08:21:39.053 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.054 INFO:teuthology.orchestra.run.smithi063.stdout: "host_check_interval": { 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "host_check_interval", 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to perform a host check", 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "hw_monitoring": { 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hw_monitoring", 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.056 INFO:teuthology.orchestra.run.smithi063.stdout: "inventory_list_all": { 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "inventory_list_all", 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.057 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.058 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "log_refresh_metadata": { 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_refresh_metadata", 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.059 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.060 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.061 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.062 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.063 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T08:21:39.064 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "*", 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.065 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "max_count_per_host": { 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_count_per_host", 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.066 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of daemons per service per host", 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "max_osd_draining_count": { 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_osd_draining_count", 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.067 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "migration_current": { 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "migration_current", 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.068 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "internal - do not modify", 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "root", 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.069 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm-package", 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "root" 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "oob_default_addr": { 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "oob_default_addr", 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.070 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "169.254.1.1", 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "prometheus_alerts_path": { 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus_alerts_path", 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.071 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_insecure": { 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_insecure", 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.072 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_password": { 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_password", 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.073 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_url": { 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_url", 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.074 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_username": { 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_username", 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.075 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "secure_monitoring_stack": { 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "secure_monitoring_stack", 2024-05-27T08:21:39.076 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "service_discovery_port": { 2024-05-27T08:21:39.077 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "service_discovery_port", 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8765", 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "cephadm service discovery port", 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.078 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.079 INFO:teuthology.orchestra.run.smithi063.stdout: "ssh_config_file": { 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssh_config_file", 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.080 INFO:teuthology.orchestra.run.smithi063.stdout: "use_agent": { 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_agent", 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "use_repo_digest": { 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_repo_digest", 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.082 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_failed_host_check": { 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_failed_host_check", 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.083 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_daemons": { 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_daemons", 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T08:21:39.084 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_hosts": { 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_hosts", 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.085 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crash", 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.086 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.087 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.088 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.089 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.090 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "retain_interval": { 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retain_interval", 2024-05-27T08:21:39.091 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "31536000", 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.092 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_recent_interval": { 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_recent_interval", 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.093 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1209600", 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.094 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dashboard", 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.095 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.096 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.097 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_ENABLED": { 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.098 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.099 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.100 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.101 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.102 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.103 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.104 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.105 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.106 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.107 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.108 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.109 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T08:21:39.110 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_URL": { 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_URL", 2024-05-27T08:21:39.111 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T08:21:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T08:21:39.113 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T08:21:39.114 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.116 INFO:teuthology.orchestra.run.smithi063.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.117 INFO:teuthology.orchestra.run.smithi063.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "[]", 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "{}", 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.119 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.120 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.121 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.124 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.125 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.126 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.127 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.128 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.129 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T08:21:39.130 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.131 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.132 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8", 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.133 INFO:teuthology.orchestra.run.smithi063.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "45", 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.136 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.137 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.138 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.139 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.140 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.141 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.142 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "cross_origin_url": { 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cross_origin_url", 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.143 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "crt_file": { 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crt_file", 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.144 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "debug": { 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "debug", 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.145 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable/disable debug options", 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "jwt_token_ttl": { 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "jwt_token_ttl", 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "28800", 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.146 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.147 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.148 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.149 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.151 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.152 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "motd": { 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "motd", 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.153 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "The message of the day", 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect_resolve_ip_addr": { 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.154 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-27T08:21:39.155 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-27T08:21:39.156 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8080", 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.157 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.159 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl_server_port": { 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl_server_port", 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8443", 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.160 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "redirect", 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect" 2024-05-27T08:21:39.161 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-27T08:21:39.162 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "url_prefix": { 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url_prefix", 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.163 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "devicehealth", 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_monitoring": { 2024-05-27T08:21:39.164 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_monitoring", 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "monitor device health metrics", 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.165 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.166 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.167 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.168 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.169 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.170 INFO:teuthology.orchestra.run.smithi063.stdout: "mark_out_threshold": { 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mark_out_threshold", 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2419200", 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.171 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": { 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_name", 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "device_health_metrics", 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "retention_period": { 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retention_period", 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15552000", 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain device health metrics", 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_frequency": { 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_frequency", 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.174 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "self_heal": { 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "self_heal", 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.175 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.176 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.177 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_threshold": { 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_threshold", 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7257600", 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.178 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "diskprediction_local", 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.179 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.180 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.181 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.182 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.183 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "predict_interval": { 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predict_interval", 2024-05-27T08:21:39.184 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "predictor_model": { 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predictor_model", 2024-05-27T08:21:39.185 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "prophetstor", 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:39.186 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.187 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "influx", 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": false, 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "influxdb python module not found", 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "batch_size": { 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "batch_size", 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5000", 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.189 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "database": { 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "database", 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "ceph", 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.190 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": { 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hostname", 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.191 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server hostname", 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "30", 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "5", 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.192 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.193 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.195 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.196 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.197 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "password": { 2024-05-27T08:21:39.198 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "password", 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "password of InfluxDB server user", 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.199 INFO:teuthology.orchestra.run.smithi063.stdout: "port": { 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "port", 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8086", 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server port", 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "false", 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.202 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "threads": { 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threads", 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "32", 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.203 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "username": { 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "username", 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "username of InfluxDB server user", 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.204 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "verify_ssl": { 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "verify_ssl", 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "true", 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.205 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "insights", 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.209 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.210 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.211 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "iostat", 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.213 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.216 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.217 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.218 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "k8sevents", 2024-05-27T08:21:39.219 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_event_retention_days": { 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ceph_event_retention_days", 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7", 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.220 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "config_check_secs": { 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_check_secs", 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "10", 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.221 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.222 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.223 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.225 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.226 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "localpool", 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "failure_domain": { 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "failure_domain", 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.228 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "host", 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "failure domain for any created local pool", 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.229 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.230 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.231 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.232 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.233 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": { 2024-05-27T08:21:39.234 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_size", 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default min_size for any created local pool", 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.235 INFO:teuthology.orchestra.run.smithi063.stdout: "num_rep": { 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "num_rep", 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3", 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default replica count for any created local pool", 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": { 2024-05-27T08:21:39.236 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_num", 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "128", 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "prefix": { 2024-05-27T08:21:39.237 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prefix", 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name prefix for any created local pool", 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.238 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.239 INFO:teuthology.orchestra.run.smithi063.stdout: "subtree": { 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "subtree", 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "rack", 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.240 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mds_autoscaler", 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.241 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.242 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirroring", 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "nfs", 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-27T08:21:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "fail_fs": { 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "fail_fs", 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T08:21:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator": { 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm", 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "rook", 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "test_orchestrator" 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Orchestrator backend", 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_perf_query", 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_support", 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_autoscaler", 2024-05-27T08:21:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "threshold": { 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threshold", 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1.0", 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "scaling threshold", 2024-05-27T08:21:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T08:21:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "progress", 2024-05-27T08:21:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_pg_recovery_event": { 2024-05-27T08:21:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_pg_recovery_event", 2024-05-27T08:21:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-27T08:21:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "max_completed_events": { 2024-05-27T08:21:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_completed_events", 2024-05-27T08:21:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "50", 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "number of past completed events to remember", 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long the module is going to sleep", 2024-05-27T08:21:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus", 2024-05-27T08:21:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "cache": { 2024-05-27T08:21:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cache", 2024-05-27T08:21:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "exclude_perf_counters": { 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "exclude_perf_counters", 2024-05-27T08:21:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T08:21:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T08:21:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools": { 2024-05-27T08:21:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools", 2024-05-27T08:21:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.336 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.336 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T08:21:39.336 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T08:21:39.336 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.336 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.336 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.336 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "300", 2024-05-27T08:21:39.337 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.337 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.337 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.337 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.337 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.337 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.337 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.337 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_interval": { 2024-05-27T08:21:39.338 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_interval", 2024-05-27T08:21:39.338 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.338 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.338 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.338 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15.0", 2024-05-27T08:21:39.338 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.338 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.338 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.339 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.339 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.339 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.339 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.339 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-27T08:21:39.340 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-27T08:21:39.340 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.340 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.340 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.340 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-27T08:21:39.340 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.340 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.340 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.340 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T08:21:39.341 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.341 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.341 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.341 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-27T08:21:39.341 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-27T08:21:39.341 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.341 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.341 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.342 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "9283", 2024-05-27T08:21:39.342 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.342 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.342 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.342 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T08:21:39.342 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.342 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.343 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.343 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.343 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.343 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.343 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.343 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.343 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.343 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "stale_cache_strategy": { 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stale_cache_strategy", 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.344 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "log", 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-27T08:21:39.345 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-27T08:21:39.346 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.346 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.346 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.346 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "default", 2024-05-27T08:21:39.346 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.346 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.346 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.346 INFO:teuthology.orchestra.run.smithi063.stdout: "default", 2024-05-27T08:21:39.346 INFO:teuthology.orchestra.run.smithi063.stdout: "error" 2024-05-27T08:21:39.347 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.347 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.347 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.347 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.347 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.347 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-27T08:21:39.347 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-27T08:21:39.348 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.348 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.348 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.348 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-27T08:21:39.348 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-27T08:21:39.348 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-27T08:21:39.348 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.348 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.348 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.348 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.349 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.349 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_support", 2024-05-27T08:21:39.349 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.349 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.350 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.350 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.350 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.350 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.350 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.350 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.350 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.350 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.351 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.351 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.351 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.351 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.351 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.351 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.351 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.351 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.352 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.352 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.352 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.352 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.352 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.352 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.352 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.353 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.353 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.353 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.353 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.353 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.353 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.353 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.354 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.354 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.354 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.354 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.354 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.354 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.355 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.355 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.355 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.355 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.355 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.355 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.355 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.355 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.355 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.355 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.356 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.356 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.356 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.356 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.356 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.356 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.357 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.357 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.357 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.357 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.357 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.358 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.358 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.358 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.358 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.358 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.358 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.358 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.358 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.359 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.359 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.359 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_snap_create": { 2024-05-27T08:21:39.359 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_snap_create", 2024-05-27T08:21:39.359 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.360 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.360 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.360 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.360 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.360 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.360 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.360 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.360 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.360 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.361 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.361 INFO:teuthology.orchestra.run.smithi063.stdout: "mirror_snapshot_schedule": { 2024-05-27T08:21:39.361 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T08:21:39.361 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.361 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.361 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.361 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.362 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.362 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.362 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.362 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.362 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.362 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.362 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.362 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.363 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.363 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.363 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.363 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.363 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.363 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.363 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.363 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.364 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.364 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.364 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.364 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.364 INFO:teuthology.orchestra.run.smithi063.stdout: "trash_purge_schedule": { 2024-05-27T08:21:39.364 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "trash_purge_schedule", 2024-05-27T08:21:39.364 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.365 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.365 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.365 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.365 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.365 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.365 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.365 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.365 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.365 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.366 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.366 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.366 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.366 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "restful", 2024-05-27T08:21:39.366 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.366 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.366 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.367 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_auth": { 2024-05-27T08:21:39.367 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_auth", 2024-05-27T08:21:39.367 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.367 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.367 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.367 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.367 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.367 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.368 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.368 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.368 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.368 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.368 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.368 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-27T08:21:39.368 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-27T08:21:39.368 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.369 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.369 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.369 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.369 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.369 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.369 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.369 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.370 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.370 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.370 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.370 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.370 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.370 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.370 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.370 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.371 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.371 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.371 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.371 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.371 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.371 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.371 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.371 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.372 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.372 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.372 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.372 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.372 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.372 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.372 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.373 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.373 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.373 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.373 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.373 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.373 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.373 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.374 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.374 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.374 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.374 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.374 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.374 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.375 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.375 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.375 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.375 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.375 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.375 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.375 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.376 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.377 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.377 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.377 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.377 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.377 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.377 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.377 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.378 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.378 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.378 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.378 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.378 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.378 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.378 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.378 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.379 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.379 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.379 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.379 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.379 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-27T08:21:39.379 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-27T08:21:39.379 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.380 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.380 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.380 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.380 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.380 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.380 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.380 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.380 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.381 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.381 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.381 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-27T08:21:39.381 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-27T08:21:39.381 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.381 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.382 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.382 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.382 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.382 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.382 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.382 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.382 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.383 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.383 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.383 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.383 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.383 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.383 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.383 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.384 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.384 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.384 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.384 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.384 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.384 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.384 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.385 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.385 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.385 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.385 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rgw", 2024-05-27T08:21:39.385 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.385 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.386 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.386 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.386 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.386 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.386 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.386 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.386 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.386 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.387 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.387 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.387 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.387 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.387 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.387 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.387 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.388 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.388 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.388 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.388 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.388 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.388 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.388 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.389 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.389 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.389 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.389 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.389 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.389 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.389 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.390 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.390 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.390 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.390 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.390 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.390 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.391 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.391 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.391 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.391 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.391 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.391 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.391 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.392 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.392 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.392 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.392 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.392 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.392 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.392 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.392 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.392 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.393 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.393 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.393 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.393 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.393 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.393 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.394 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.394 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.394 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.394 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.394 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.394 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.394 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.395 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.395 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.395 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.395 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.395 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.395 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.395 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.395 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.396 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.396 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.396 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.396 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.396 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.396 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.396 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.397 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.397 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.397 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.397 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rook", 2024-05-27T08:21:39.397 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.397 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.397 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.398 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.398 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.398 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.398 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.398 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.398 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.398 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.398 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.398 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.399 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.399 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.399 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.399 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.399 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.399 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.399 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.399 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.400 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.400 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.400 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.400 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.400 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.400 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.400 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.401 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.401 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.401 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.401 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.401 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.401 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.401 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.401 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.401 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.402 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.402 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.402 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.402 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.402 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.402 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.402 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.403 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.403 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.403 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.403 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.403 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.403 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.403 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.403 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.404 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.404 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.404 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.404 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.404 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.404 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.405 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.405 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.405 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.405 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.405 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.405 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.405 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.406 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.406 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.406 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.406 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.406 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.407 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.407 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.407 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.407 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.407 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.407 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.407 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.407 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.408 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.408 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.408 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.408 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.408 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.408 INFO:teuthology.orchestra.run.smithi063.stdout: "storage_class": { 2024-05-27T08:21:39.408 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "storage_class", 2024-05-27T08:21:39.409 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.409 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.409 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.409 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "local", 2024-05-27T08:21:39.409 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.409 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.409 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.409 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T08:21:39.410 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.410 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.410 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.410 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.410 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.411 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "selftest", 2024-05-27T08:21:39.411 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.411 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.411 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.411 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.411 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.411 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.412 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.412 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.412 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.412 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.412 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.412 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.412 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.412 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.413 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.413 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.413 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.413 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.413 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.413 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.414 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.414 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.414 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.414 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.414 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.414 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.415 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.415 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.415 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.415 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.415 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.415 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.415 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.416 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.416 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.416 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.416 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.416 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.416 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.416 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.416 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.416 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.417 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.417 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.417 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.417 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.417 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.417 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.417 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.417 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.418 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.418 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.418 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.418 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.418 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.418 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.418 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.419 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.419 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.419 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.419 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.419 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.419 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.419 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.419 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.419 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.420 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.420 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.420 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.420 INFO:teuthology.orchestra.run.smithi063.stdout: "roption1": { 2024-05-27T08:21:39.420 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption1", 2024-05-27T08:21:39.420 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.420 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.421 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.421 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.421 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.421 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.421 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.421 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.421 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.421 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.422 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.422 INFO:teuthology.orchestra.run.smithi063.stdout: "roption2": { 2024-05-27T08:21:39.422 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption2", 2024-05-27T08:21:39.422 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.422 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.422 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.422 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "xyz", 2024-05-27T08:21:39.423 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.423 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.423 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.423 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.423 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.423 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.423 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.423 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption1": { 2024-05-27T08:21:39.424 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption1", 2024-05-27T08:21:39.424 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.424 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.424 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.424 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.424 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.424 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption2": { 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption2", 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.425 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.426 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.426 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.426 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.426 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.426 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.426 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.426 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.427 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.427 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption3": { 2024-05-27T08:21:39.427 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption3", 2024-05-27T08:21:39.427 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.427 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.427 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.427 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.427 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.428 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.428 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.428 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.428 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.428 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.428 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.428 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption4": { 2024-05-27T08:21:39.428 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption4", 2024-05-27T08:21:39.429 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.429 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.429 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.429 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.429 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.429 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.429 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.429 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.429 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.430 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.430 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.430 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption5": { 2024-05-27T08:21:39.430 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption5", 2024-05-27T08:21:39.430 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.430 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.431 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.431 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.431 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.431 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.431 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.431 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.431 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.431 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.431 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.432 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption6": { 2024-05-27T08:21:39.432 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption6", 2024-05-27T08:21:39.432 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.432 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.432 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.432 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.432 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.432 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.433 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.433 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.433 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.433 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.433 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.433 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption7": { 2024-05-27T08:21:39.433 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption7", 2024-05-27T08:21:39.434 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.434 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.434 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.434 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.434 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:39.434 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "42", 2024-05-27T08:21:39.434 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.434 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.435 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.435 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.435 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.435 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.435 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.435 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.435 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.435 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.436 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.436 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.436 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.436 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.436 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.436 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.436 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.436 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.437 INFO:teuthology.orchestra.run.smithi063.stdout: "testkey": { 2024-05-27T08:21:39.437 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testkey", 2024-05-27T08:21:39.437 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.437 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.437 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.437 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.437 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.437 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.437 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.438 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.438 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.438 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.438 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.438 INFO:teuthology.orchestra.run.smithi063.stdout: "testlkey": { 2024-05-27T08:21:39.438 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testlkey", 2024-05-27T08:21:39.438 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.439 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.439 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.439 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.439 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.439 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.439 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.439 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.439 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.440 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.440 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.440 INFO:teuthology.orchestra.run.smithi063.stdout: "testnewline": { 2024-05-27T08:21:39.440 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testnewline", 2024-05-27T08:21:39.440 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.440 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.440 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.441 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.441 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.441 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.441 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.441 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.441 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.441 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.441 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.442 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.442 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.442 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smb", 2024-05-27T08:21:39.442 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.442 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.442 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.442 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.443 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.443 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.443 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.443 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.443 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.443 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.443 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.443 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.444 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.444 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.444 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.444 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.444 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.444 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.444 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.444 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.444 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.444 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.445 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.445 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.445 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.445 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.445 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.445 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.445 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.445 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.446 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.446 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.446 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.446 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.446 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.446 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.446 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.447 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.447 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.447 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.447 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.447 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.447 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.448 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.448 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.448 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.448 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.448 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.448 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.448 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.448 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.450 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.450 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.450 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.450 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.450 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.450 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.450 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.451 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.451 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.451 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.451 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.451 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.451 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.452 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.452 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.452 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.452 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.452 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.452 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.452 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.453 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.453 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.453 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.453 INFO:teuthology.orchestra.run.smithi063.stdout: "update_orchestration": { 2024-05-27T08:21:39.453 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "update_orchestration", 2024-05-27T08:21:39.453 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.453 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.454 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.454 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.454 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.454 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.454 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.454 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T08:21:39.454 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.455 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.455 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.455 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.455 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.455 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snap_schedule", 2024-05-27T08:21:39.455 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.456 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.456 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.456 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_m_granularity": { 2024-05-27T08:21:39.456 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_m_granularity", 2024-05-27T08:21:39.456 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.456 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.456 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.456 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.456 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.457 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.457 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.457 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T08:21:39.457 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.457 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.457 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.458 INFO:teuthology.orchestra.run.smithi063.stdout: "dump_on_update": { 2024-05-27T08:21:39.458 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dump_on_update", 2024-05-27T08:21:39.458 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.458 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.458 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.458 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.458 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.458 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.459 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.459 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "dump database to debug log on update", 2024-05-27T08:21:39.459 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.459 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.459 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.459 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.459 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.459 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.460 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.460 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.460 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.460 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.460 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.460 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.460 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.461 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.461 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.461 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.461 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.461 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.461 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.461 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.461 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.461 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.461 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.462 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.462 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.462 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.462 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.462 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.462 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.462 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.463 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.463 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.463 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.463 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.463 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.463 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.463 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.463 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.464 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.465 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.465 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.465 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.465 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.465 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.465 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.465 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.465 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.466 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.466 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.466 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.466 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.466 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.466 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.466 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.466 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.466 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.467 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.467 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.467 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.467 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.467 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.467 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.467 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.467 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.467 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.468 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.468 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.468 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.468 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.468 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.468 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.468 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.468 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.468 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stats", 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.469 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.470 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.470 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.470 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.470 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.470 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.470 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.470 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.470 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.470 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.471 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.471 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.471 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.471 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.471 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.471 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.471 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.471 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.471 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.472 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.472 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.472 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.472 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.472 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.472 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.472 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.472 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.472 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.473 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.474 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.474 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.474 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.474 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.474 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.474 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.474 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.474 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.474 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.475 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.476 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.477 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.477 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.477 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.477 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.477 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "status", 2024-05-27T08:21:39.477 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.478 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.478 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.478 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.478 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.478 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.478 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.478 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.478 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.479 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.479 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.479 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.479 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.479 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.479 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.479 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.479 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.479 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.480 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.481 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.482 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.482 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.482 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.482 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.482 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.482 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.482 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.482 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.482 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.483 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.484 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.485 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.485 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.485 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.485 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.485 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.485 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.485 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.485 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.485 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.485 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telegraf", 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.486 INFO:teuthology.orchestra.run.smithi063.stdout: "address": { 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "address", 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.487 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15", 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.489 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.490 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.491 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.492 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.493 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.494 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telemetry", 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_basic": { 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_basic", 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.495 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_crash": { 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_crash", 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.496 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_device": { 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_device", 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.497 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_ident": { 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_ident", 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.498 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_perf": { 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_perf", 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.499 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "contact": { 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "contact", 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.500 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "description": { 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "description", 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.501 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "device_url": { 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_url", 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.502 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.503 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.504 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "24", 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "8", 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "last_opt_revision": { 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "last_opt_revision", 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.505 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1", 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard": { 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard", 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.506 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard_description": { 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard_description", 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.507 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.508 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.509 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.510 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.511 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.512 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "organization": { 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "organization", 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.513 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "proxy": { 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "proxy", 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.514 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.515 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "url": { 2024-05-27T08:21:39.516 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url", 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.517 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "test_orchestrator", 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.518 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.519 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.520 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.521 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.522 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.523 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "volumes", 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.525 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.526 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.528 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.529 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_clones": { 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_clones", 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.530 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4", 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "periodic_async_work": { 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "periodic_async_work", 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.531 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Periodically check for async work", 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_delay": { 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_delay", 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.532 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_no_wait": { 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.533 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.534 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.535 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.535 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.535 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.535 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.535 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.535 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.535 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.535 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.535 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.535 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.537 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.537 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.537 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix", 2024-05-27T08:21:39.537 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.537 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.537 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "discovery_interval": { 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "discovery_interval", 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "100", 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "identifier": { 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "identifier", 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.539 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.540 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.541 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.542 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.543 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.544 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.545 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_host": { 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_host", 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.546 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_port": { 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_port", 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.547 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10051", 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_sender": { 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_sender", 2024-05-27T08:21:39.548 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:39.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "gid": 35048, 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "y", 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_features": 4540701547738038271, 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "available_modules": [ 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "alerts", 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.550 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.551 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.552 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.553 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.554 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.555 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.556 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_destination": { 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_destination", 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email address to send alerts to", 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_from_name": { 2024-05-27T08:21:39.557 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_from_name", 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "Ceph", 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email From: name", 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.558 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_host": { 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_host", 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP server", 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.559 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_password": { 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_password", 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Password to authenticate with", 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.560 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_port": { 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_port", 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "465", 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP port", 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.561 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_sender": { 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_sender", 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP envelope sender", 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.562 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_ssl": { 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_ssl", 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.563 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_user": { 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_user", 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.564 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "User to authenticate as", 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.565 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "balancer", 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.566 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "active": { 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "active", 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_time": { 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_time", 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0000", 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_weekday": { 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_weekday", 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.569 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_max_iterations": { 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_max_iterations", 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "25", 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "250", 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_metrics": { 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_metrics", 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.571 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_step": { 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_step", 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.5", 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0.001", 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "0.999", 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "aggressiveness of optimization", 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.572 INFO:teuthology.orchestra.run.smithi063.stdout: "end_time": { 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_time", 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2359", 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.573 INFO:teuthology.orchestra.run.smithi063.stdout: "end_weekday": { 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_weekday", 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.575 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.576 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.577 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.578 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.579 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "min_score": { 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_score", 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.580 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "upmap", 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "crush-compat", 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "none", 2024-05-27T08:21:39.581 INFO:teuthology.orchestra.run.smithi063.stdout: "read", 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap", 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap-read" 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Balancer mode", 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_ids": { 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_ids", 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.582 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:39.583 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.584 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_deviation": { 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_deviation", 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_optimizations": { 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_optimizations", 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.587 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm", 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_down_multiplier": { 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_down_multiplier", 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.588 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_refresh_rate": { 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_refresh_rate", 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.589 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "20", 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_starting_port": { 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_starting_port", 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.590 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4721", 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_ptrace": { 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_ptrace", 2024-05-27T08:21:39.591 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.592 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_interval": { 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_interval", 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_memory_target_ratio": { 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.7", 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.594 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm_log_destination": { 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm_log_destination", 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "file", 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "file,syslog", 2024-05-27T08:21:39.595 INFO:teuthology.orchestra.run.smithi063.stdout: "syslog" 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "cgroups_split": { 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cgroups_split", 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.596 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "config_checks_enabled": { 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_checks_enabled", 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.597 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "config_dashboard": { 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_dashboard", 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.598 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_alertmanager": { 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_alertmanager", 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.599 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_base": { 2024-05-27T08:21:39.600 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_base", 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Container image name, without the tag", 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_elasticsearch": { 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_elasticsearch", 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "elasticsearch container image", 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_grafana": { 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_grafana", 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.603 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_haproxy": { 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_haproxy", 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "HAproxy container image", 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.604 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_agent": { 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_agent", 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger agent container image", 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.605 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_collector": { 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_collector", 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger collector container image", 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.606 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_query": { 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_query", 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger query container image", 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.607 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_keepalived": { 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_keepalived", 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Keepalived container image", 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_loki": { 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_loki", 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Loki container image", 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.609 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_node_exporter": { 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_node_exporter", 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_nvmeof": { 2024-05-27T08:21:39.610 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_nvmeof", 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Nvme-of container image", 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_prometheus": { 2024-05-27T08:21:39.611 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_prometheus", 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.612 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_promtail": { 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_promtail", 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Promtail container image", 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.613 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_samba": { 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_samba", 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Samba/SMB container image", 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.614 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_snmp_gateway": { 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_snmp_gateway", 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SNMP Gateway container image", 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.615 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "container_init": { 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_init", 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.616 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "daemon_cache_timeout": { 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "daemon_cache_timeout", 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "default_cephadm_command_timeout": { 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "900", 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.618 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "default_registry": { 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_registry", 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io", 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.619 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "device_cache_timeout": { 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_cache_timeout", 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1800", 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache device inventory", 2024-05-27T08:21:39.620 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "device_enhanced_scan": { 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_enhanced_scan", 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.621 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "facts_cache_timeout": { 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "facts_cache_timeout", 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache host facts data", 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.622 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "grafana_dashboards_path": { 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "grafana_dashboards_path", 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.623 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "host_check_interval": { 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "host_check_interval", 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to perform a host check", 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.624 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "hw_monitoring": { 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hw_monitoring", 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.625 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "inventory_list_all": { 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "inventory_list_all", 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T08:21:39.626 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.627 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "log_refresh_metadata": { 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_refresh_metadata", 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.628 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.629 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.630 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.631 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.632 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T08:21:39.633 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "*", 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.634 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "max_count_per_host": { 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_count_per_host", 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.635 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of daemons per service per host", 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "max_osd_draining_count": { 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_osd_draining_count", 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T08:21:39.636 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "migration_current": { 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "migration_current", 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.637 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "internal - do not modify", 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "root", 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.638 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm-package", 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "root" 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "oob_default_addr": { 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "oob_default_addr", 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.639 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "169.254.1.1", 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "prometheus_alerts_path": { 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus_alerts_path", 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.640 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_insecure": { 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_insecure", 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.641 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_password": { 2024-05-27T08:21:39.642 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_password", 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.643 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_url": { 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_url", 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_username": { 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_username", 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "secure_monitoring_stack": { 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "secure_monitoring_stack", 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.646 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "service_discovery_port": { 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "service_discovery_port", 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8765", 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "cephadm service discovery port", 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.647 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.648 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "ssh_config_file": { 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssh_config_file", 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T08:21:39.649 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "use_agent": { 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_agent", 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.650 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "use_repo_digest": { 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_repo_digest", 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.651 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_failed_host_check": { 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_failed_host_check", 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_daemons": { 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_daemons", 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.653 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_hosts": { 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_hosts", 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.654 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.655 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crash", 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.657 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.658 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.659 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.660 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "retain_interval": { 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retain_interval", 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.661 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "31536000", 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.662 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_recent_interval": { 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_recent_interval", 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.663 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1209600", 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.664 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dashboard", 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.665 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.666 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.667 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_ENABLED": { 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.668 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.669 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.670 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T08:21:39.671 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.672 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.673 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T08:21:39.674 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T08:21:39.675 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T08:21:39.676 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T08:21:39.677 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.678 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.679 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.680 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.681 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_URL": { 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_URL", 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.682 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.683 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.684 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.685 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.686 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.687 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "[]", 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.688 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "{}", 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T08:21:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T08:21:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T08:21:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8", 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T08:21:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "45", 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T08:21:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T08:21:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "cross_origin_url": { 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cross_origin_url", 2024-05-27T08:21:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "crt_file": { 2024-05-27T08:21:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crt_file", 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "debug": { 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "debug", 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable/disable debug options", 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "jwt_token_ttl": { 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "jwt_token_ttl", 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "28800", 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "motd": { 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "motd", 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "The message of the day", 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect_resolve_ip_addr": { 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8080", 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl_server_port": { 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl_server_port", 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8443", 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "redirect", 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect" 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "url_prefix": { 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url_prefix", 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "devicehealth", 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_monitoring": { 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_monitoring", 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "monitor device health metrics", 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "mark_out_threshold": { 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mark_out_threshold", 2024-05-27T08:21:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2419200", 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": { 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_name", 2024-05-27T08:21:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "device_health_metrics", 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "retention_period": { 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retention_period", 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15552000", 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain device health metrics", 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_frequency": { 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_frequency", 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "self_heal": { 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "self_heal", 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_threshold": { 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_threshold", 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7257600", 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "diskprediction_local", 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "predict_interval": { 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predict_interval", 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "predictor_model": { 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predictor_model", 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "prophetstor", 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "influx", 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": false, 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "influxdb python module not found", 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "batch_size": { 2024-05-27T08:21:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "batch_size", 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5000", 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "database": { 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "database", 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "ceph", 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.762 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": { 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hostname", 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server hostname", 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.763 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "30", 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "5", 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "password": { 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "password", 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "password of InfluxDB server user", 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "port": { 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "port", 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8086", 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server port", 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "false", 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "threads": { 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threads", 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "32", 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "username": { 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "username", 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "username of InfluxDB server user", 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "verify_ssl": { 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "verify_ssl", 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "true", 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "insights", 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "iostat", 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.786 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.787 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.788 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.789 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.790 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "k8sevents", 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.791 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_event_retention_days": { 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ceph_event_retention_days", 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7", 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "config_check_secs": { 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_check_secs", 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "10", 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.794 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.795 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.796 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.797 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.798 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.799 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "localpool", 2024-05-27T08:21:39.800 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "failure_domain": { 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "failure_domain", 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "host", 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "failure domain for any created local pool", 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T08:21:39.801 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.802 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.803 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.804 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.805 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.806 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": { 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_size", 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.807 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default min_size for any created local pool", 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "num_rep": { 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "num_rep", 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3", 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default replica count for any created local pool", 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": { 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_num", 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "128", 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "prefix": { 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prefix", 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name prefix for any created local pool", 2024-05-27T08:21:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "subtree": { 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "subtree", 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "rack", 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mds_autoscaler", 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirroring", 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "nfs", 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "fail_fs": { 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "fail_fs", 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator": { 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm", 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "rook", 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "test_orchestrator" 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Orchestrator backend", 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_perf_query", 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_support", 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_autoscaler", 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "threshold": { 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threshold", 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1.0", 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "scaling threshold", 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "progress", 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_pg_recovery_event": { 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_pg_recovery_event", 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "max_completed_events": { 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_completed_events", 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "50", 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "number of past completed events to remember", 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long the module is going to sleep", 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus", 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "cache": { 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cache", 2024-05-27T08:21:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "exclude_perf_counters": { 2024-05-27T08:21:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "exclude_perf_counters", 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools": { 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools", 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "300", 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_interval": { 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_interval", 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15.0", 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "9283", 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T08:21:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "stale_cache_strategy": { 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stale_cache_strategy", 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "log", 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "default", 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "default", 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "error" 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_support", 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_snap_create": { 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_snap_create", 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "mirror_snapshot_schedule": { 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "trash_purge_schedule": { 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "trash_purge_schedule", 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "restful", 2024-05-27T08:21:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_auth": { 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_auth", 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rgw", 2024-05-27T08:21:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rook", 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.922 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.923 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.924 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.925 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.926 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.927 INFO:teuthology.orchestra.run.smithi063.stdout: "storage_class": { 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "storage_class", 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "local", 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.928 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "selftest", 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.929 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.931 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.932 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.933 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "roption1": { 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption1", 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.934 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "roption2": { 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption2", 2024-05-27T08:21:39.935 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "xyz", 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption1": { 2024-05-27T08:21:39.936 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption1", 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.937 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption2": { 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption2", 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.938 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption3": { 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption3", 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.939 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption4": { 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption4", 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.940 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption5": { 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption5", 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.941 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption6": { 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption6", 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.942 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption7": { 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption7", 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "42", 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.943 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.944 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "testkey": { 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testkey", 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.945 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "testlkey": { 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testlkey", 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.946 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "testnewline": { 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testnewline", 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.947 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smb", 2024-05-27T08:21:39.948 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.949 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.950 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.952 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.953 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.954 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "update_orchestration": { 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "update_orchestration", 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.955 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.956 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snap_schedule", 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_m_granularity": { 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_m_granularity", 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.957 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "dump_on_update": { 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dump_on_update", 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.958 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "dump database to debug log on update", 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.959 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.960 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.961 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.962 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.963 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.964 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stats", 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.965 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.966 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.967 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.968 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.969 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.970 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.971 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "status", 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.972 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.973 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.974 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.975 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.976 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.977 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telegraf", 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "address": { 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "address", 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.979 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15", 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.980 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.981 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.982 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.983 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:39.984 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.985 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:39.986 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telemetry", 2024-05-27T08:21:39.987 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_basic": { 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_basic", 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T08:21:39.988 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_crash": { 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_crash", 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.989 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_device": { 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_device", 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.990 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_ident": { 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_ident", 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.991 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_perf": { 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_perf", 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.992 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "contact": { 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "contact", 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.993 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "description": { 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "description", 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.994 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "device_url": { 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_url", 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.995 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:39.996 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "24", 2024-05-27T08:21:39.997 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "8", 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "last_opt_revision": { 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "last_opt_revision", 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:39.998 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1", 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard": { 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard", 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard_description": { 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard_description", 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "organization": { 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "organization", 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "proxy": { 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "proxy", 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "url": { 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url", 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "test_orchestrator", 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "volumes", 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_clones": { 2024-05-27T08:21:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_clones", 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4", 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "periodic_async_work": { 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "periodic_async_work", 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Periodically check for async work", 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_delay": { 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_delay", 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_no_wait": { 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T08:21:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix", 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "discovery_interval": { 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "discovery_interval", 2024-05-27T08:21:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "100", 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "identifier": { 2024-05-27T08:21:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "identifier", 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_host": { 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_host", 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_port": { 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_port", 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10051", 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_sender": { 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_sender", 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "modules": [ 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm", 2024-05-27T08:21:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "dashboard", 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "iostat", 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "nfs", 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "restful", 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_schedule" 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "available_modules": [ 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "alerts", 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_destination": { 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_destination", 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email address to send alerts to", 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_from_name": { 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_from_name", 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "Ceph", 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Email From: name", 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_host": { 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_host", 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP server", 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_password": { 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_password", 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Password to authenticate with", 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_port": { 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_port", 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "465", 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP port", 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_sender": { 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_sender", 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SMTP envelope sender", 2024-05-27T08:21:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_ssl": { 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_ssl", 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T08:21:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "smtp_user": { 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smtp_user", 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "User to authenticate as", 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "balancer", 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "active": { 2024-05-27T08:21:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "active", 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_time": { 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_time", 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0000", 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "begin_weekday": { 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "begin_weekday", 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_max_iterations": { 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_max_iterations", 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "25", 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "250", 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_metrics": { 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_metrics", 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T08:21:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_compat_step": { 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crush_compat_step", 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.5", 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0.001", 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "0.999", 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "aggressiveness of optimization", 2024-05-27T08:21:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "end_time": { 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_time", 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2359", 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T08:21:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "end_weekday": { 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "end_weekday", 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "0", 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "6", 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "min_score": { 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_score", 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "upmap", 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "crush-compat", 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "none", 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "read", 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap", 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap-read" 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Balancer mode", 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_ids": { 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_ids", 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_deviation": { 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_deviation", 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T08:21:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "upmap_max_optimizations": { 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "upmap_max_optimizations", 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm", 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_down_multiplier": { 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_down_multiplier", 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_refresh_rate": { 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_refresh_rate", 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "20", 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "agent_starting_port": { 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "agent_starting_port", 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4721", 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_ptrace": { 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_ptrace", 2024-05-27T08:21:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_interval": { 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_interval", 2024-05-27T08:21:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "autotune_memory_target_ratio": { 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0.7", 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm_log_destination": { 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cephadm_log_destination", 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "file", 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "file,syslog", 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "syslog" 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "cgroups_split": { 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cgroups_split", 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T08:21:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "config_checks_enabled": { 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_checks_enabled", 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "config_dashboard": { 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_dashboard", 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T08:21:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_alertmanager": { 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_alertmanager", 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_base": { 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_base", 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Container image name, without the tag", 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_elasticsearch": { 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_elasticsearch", 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "elasticsearch container image", 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_grafana": { 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_grafana", 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.095 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_haproxy": { 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_haproxy", 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.096 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "HAproxy container image", 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_agent": { 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_agent", 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.097 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger agent container image", 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_collector": { 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_collector", 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T08:21:40.098 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger collector container image", 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_jaeger_query": { 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_jaeger_query", 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.099 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Jaeger query container image", 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_keepalived": { 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_keepalived", 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.100 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Keepalived container image", 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_loki": { 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_loki", 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.101 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Loki container image", 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_node_exporter": { 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_node_exporter", 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.102 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_nvmeof": { 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_nvmeof", 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.103 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Nvme-of container image", 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_prometheus": { 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_prometheus", 2024-05-27T08:21:40.104 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Prometheus container image", 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_promtail": { 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_promtail", 2024-05-27T08:21:40.105 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Promtail container image", 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_samba": { 2024-05-27T08:21:40.106 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_samba", 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Samba/SMB container image", 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image_snmp_gateway": { 2024-05-27T08:21:40.107 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_image_snmp_gateway", 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "SNMP Gateway container image", 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "container_init": { 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "container_init", 2024-05-27T08:21:40.108 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "daemon_cache_timeout": { 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "daemon_cache_timeout", 2024-05-27T08:21:40.109 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "default_cephadm_command_timeout": { 2024-05-27T08:21:40.110 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "900", 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "default_registry": { 2024-05-27T08:21:40.111 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "default_registry", 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "docker.io", 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "device_cache_timeout": { 2024-05-27T08:21:40.112 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_cache_timeout", 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1800", 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache device inventory", 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "device_enhanced_scan": { 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_enhanced_scan", 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.114 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "facts_cache_timeout": { 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "facts_cache_timeout", 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "seconds to cache host facts data", 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.115 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "grafana_dashboards_path": { 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "grafana_dashboards_path", 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.116 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "host_check_interval": { 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "host_check_interval", 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.117 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to perform a host check", 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "hw_monitoring": { 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hw_monitoring", 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.118 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "inventory_list_all": { 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "inventory_list_all", 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.119 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.120 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.121 INFO:teuthology.orchestra.run.smithi063.stdout: "log_refresh_metadata": { 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_refresh_metadata", 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.122 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.123 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.124 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.125 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T08:21:40.126 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T08:21:40.127 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "*", 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "max_count_per_host": { 2024-05-27T08:21:40.128 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_count_per_host", 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of daemons per service per host", 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.129 INFO:teuthology.orchestra.run.smithi063.stdout: "max_osd_draining_count": { 2024-05-27T08:21:40.130 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_osd_draining_count", 2024-05-27T08:21:40.130 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.130 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.130 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.130 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:40.130 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.130 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.130 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.130 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T08:21:40.130 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "migration_current": { 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "migration_current", 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.131 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "internal - do not modify", 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "mode": { 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mode", 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "root", 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.132 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm-package", 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "root" 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "oob_default_addr": { 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "oob_default_addr", 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.133 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "169.254.1.1", 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "prometheus_alerts_path": { 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus_alerts_path", 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.134 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_insecure": { 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_insecure", 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.135 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_password": { 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_password", 2024-05-27T08:21:40.136 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_url": { 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_url", 2024-05-27T08:21:40.137 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "registry_username": { 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "registry_username", 2024-05-27T08:21:40.138 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "secure_monitoring_stack": { 2024-05-27T08:21:40.139 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "secure_monitoring_stack", 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "service_discovery_port": { 2024-05-27T08:21:40.140 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "service_discovery_port", 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8765", 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "cephadm service discovery port", 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.141 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "ssh_config_file": { 2024-05-27T08:21:40.142 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssh_config_file", 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "use_agent": { 2024-05-27T08:21:40.143 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_agent", 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "use_repo_digest": { 2024-05-27T08:21:40.144 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "use_repo_digest", 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.145 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_failed_host_check": { 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_failed_host_check", 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.146 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_daemons": { 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_daemons", 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.147 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_on_stray_hosts": { 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_on_stray_hosts", 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.150 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crash", 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.151 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.152 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.153 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.154 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.155 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.156 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "retain_interval": { 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retain_interval", 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "31536000", 2024-05-27T08:21:40.157 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.158 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.158 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.158 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T08:21:40.158 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.158 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.158 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.158 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.158 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.158 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_recent_interval": { 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_recent_interval", 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1209600", 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.160 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dashboard", 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.161 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.162 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.163 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_ENABLED": { 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.164 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T08:21:40.165 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.166 INFO:teuthology.orchestra.run.smithi063.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.167 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.168 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.169 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.170 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.171 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.172 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.173 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.174 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.175 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.176 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.177 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_URL": { 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_URL", 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.178 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T08:21:40.179 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T08:21:40.180 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T08:21:40.181 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.182 INFO:teuthology.orchestra.run.smithi063.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.184 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "[]", 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.185 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "{}", 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.186 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.187 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.188 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.189 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.190 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.191 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.192 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.193 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T08:21:40.194 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T08:21:40.195 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T08:21:40.196 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.197 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.198 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8", 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "45", 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.201 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.202 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.203 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.203 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.203 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.203 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.203 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.203 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.203 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T08:21:40.203 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T08:21:40.204 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.204 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.204 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.204 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "admin", 2024-05-27T08:21:40.204 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.204 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.204 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.205 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.205 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.205 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.205 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.205 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T08:21:40.205 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T08:21:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T08:21:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T08:21:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T08:21:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T08:21:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T08:21:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T08:21:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T08:21:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T08:21:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T08:21:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T08:21:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.218 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.218 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.218 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.218 INFO:teuthology.orchestra.run.smithi063.stdout: "cross_origin_url": { 2024-05-27T08:21:40.219 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cross_origin_url", 2024-05-27T08:21:40.219 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.219 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.219 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.219 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.219 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.220 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.220 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.220 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.220 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.220 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.221 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.221 INFO:teuthology.orchestra.run.smithi063.stdout: "crt_file": { 2024-05-27T08:21:40.221 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "crt_file", 2024-05-27T08:21:40.221 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.221 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.222 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.222 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.222 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.222 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.222 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.222 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.223 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.223 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.223 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.223 INFO:teuthology.orchestra.run.smithi063.stdout: "debug": { 2024-05-27T08:21:40.224 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "debug", 2024-05-27T08:21:40.224 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.224 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.224 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.224 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.224 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.225 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.225 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.225 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Enable/disable debug options", 2024-05-27T08:21:40.225 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.225 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.225 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.226 INFO:teuthology.orchestra.run.smithi063.stdout: "jwt_token_ttl": { 2024-05-27T08:21:40.226 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "jwt_token_ttl", 2024-05-27T08:21:40.226 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.226 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.227 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.227 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "28800", 2024-05-27T08:21:40.227 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.227 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.227 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.227 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.228 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.228 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.228 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.228 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-27T08:21:40.228 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-27T08:21:40.229 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.229 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.229 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.229 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.229 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.230 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.230 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.230 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.230 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.230 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.230 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.231 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.231 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.231 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.231 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.231 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.232 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.232 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.232 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.232 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.232 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.232 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.233 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.233 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.233 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.233 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.233 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.234 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.234 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.234 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.234 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.235 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.235 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.235 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.235 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.235 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.235 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.236 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.236 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.236 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.236 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.236 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.236 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.237 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.237 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.237 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.237 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.238 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.238 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.238 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.238 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.238 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.238 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.239 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.239 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.239 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.239 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.239 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.240 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.240 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.240 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.240 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.240 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.240 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.241 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.241 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.241 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.241 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.241 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.242 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.242 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.242 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.242 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.242 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.243 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.243 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.243 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.243 INFO:teuthology.orchestra.run.smithi063.stdout: "motd": { 2024-05-27T08:21:40.244 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "motd", 2024-05-27T08:21:40.244 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.244 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.244 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.244 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.244 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.245 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.245 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.245 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "The message of the day", 2024-05-27T08:21:40.245 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.245 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.245 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.246 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect_resolve_ip_addr": { 2024-05-27T08:21:40.246 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T08:21:40.246 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.246 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.247 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.247 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.247 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.247 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.247 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.248 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.248 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.248 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.248 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.248 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-27T08:21:40.249 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-27T08:21:40.249 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.249 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.249 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.249 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-27T08:21:40.249 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.250 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.250 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.250 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.250 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.250 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.251 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.251 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-27T08:21:40.251 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-27T08:21:40.251 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.251 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.252 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.252 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8080", 2024-05-27T08:21:40.252 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.252 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.252 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.253 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.253 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.253 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.253 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.253 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.254 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.254 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.254 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.254 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.254 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.254 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.255 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.255 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.255 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.255 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.255 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.255 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.256 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-27T08:21:40.256 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-27T08:21:40.256 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.256 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.256 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.256 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.256 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.256 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.257 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.257 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.257 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.257 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.257 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.257 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl_server_port": { 2024-05-27T08:21:40.257 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl_server_port", 2024-05-27T08:21:40.258 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.258 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.258 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.258 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8443", 2024-05-27T08:21:40.258 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.258 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.259 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.259 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.259 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.259 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.259 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.260 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-27T08:21:40.260 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-27T08:21:40.260 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.260 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.260 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.260 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "redirect", 2024-05-27T08:21:40.260 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.260 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.260 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.260 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.261 INFO:teuthology.orchestra.run.smithi063.stdout: "redirect" 2024-05-27T08:21:40.261 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.261 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.261 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.261 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.261 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.261 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-27T08:21:40.261 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-27T08:21:40.261 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.262 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.262 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.262 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-27T08:21:40.262 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-27T08:21:40.262 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-27T08:21:40.262 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.262 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.263 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.263 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.263 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.263 INFO:teuthology.orchestra.run.smithi063.stdout: "url_prefix": { 2024-05-27T08:21:40.263 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url_prefix", 2024-05-27T08:21:40.264 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.264 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.264 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.264 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.264 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.264 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "devicehealth", 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.265 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_monitoring": { 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_monitoring", 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "monitor device health metrics", 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.268 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.269 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.270 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.271 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "mark_out_threshold": { 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mark_out_threshold", 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "2419200", 2024-05-27T08:21:40.272 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": { 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pool_name", 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "device_health_metrics", 2024-05-27T08:21:40.273 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "retention_period": { 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "retention_period", 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.274 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15552000", 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long to retain device health metrics", 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_frequency": { 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_frequency", 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.275 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "self_heal": { 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "self_heal", 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.276 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:40.277 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.278 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.279 INFO:teuthology.orchestra.run.smithi063.stdout: "warn_threshold": { 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "warn_threshold", 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7257600", 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.280 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "diskprediction_local", 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.281 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.282 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.283 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.284 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.285 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "predict_interval": { 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predict_interval", 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "86400", 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "predictor_model": { 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "predictor_model", 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "prophetstor", 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.288 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "600", 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.289 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.290 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "influx", 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": false, 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "influxdb python module not found", 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: "batch_size": { 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "batch_size", 2024-05-27T08:21:40.291 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5000", 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "database": { 2024-05-27T08:21:40.292 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "database", 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "ceph", 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.293 INFO:teuthology.orchestra.run.smithi063.stdout: "hostname": { 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "hostname", 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server hostname", 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "30", 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "5", 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.295 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.296 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.297 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.298 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.299 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.300 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "password": { 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "password", 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.301 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "password of InfluxDB server user", 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "port": { 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "port", 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.302 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "8086", 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "InfluxDB server port", 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.303 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "ssl": { 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ssl", 2024-05-27T08:21:40.304 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "false", 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "threads": { 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threads", 2024-05-27T08:21:40.305 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "32", 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.306 INFO:teuthology.orchestra.run.smithi063.stdout: "username": { 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "username", 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "username of InfluxDB server user", 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "verify_ssl": { 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "verify_ssl", 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "true", 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.308 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "insights", 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.309 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.310 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.311 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.312 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.313 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.315 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "iostat", 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.316 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.317 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.318 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.319 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "k8sevents", 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_event_retention_days": { 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "ceph_event_retention_days", 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "7", 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "config_check_secs": { 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "config_check_secs", 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "10", 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "localpool", 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: "failure_domain": { 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "failure_domain", 2024-05-27T08:21:40.332 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "host", 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "failure domain for any created local pool", 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.333 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.334 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.335 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.336 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.337 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.338 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": { 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "min_size", 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.339 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default min_size for any created local pool", 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "num_rep": { 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "num_rep", 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3", 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.340 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default replica count for any created local pool", 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": { 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_num", 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.341 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "128", 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "prefix": { 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prefix", 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.342 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "name prefix for any created local pool", 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.343 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "subtree": { 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "subtree", 2024-05-27T08:21:40.344 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "rack", 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.345 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mds_autoscaler", 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.346 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.347 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.348 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.349 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.350 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.351 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.352 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirroring", 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.353 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.354 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.355 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.356 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.357 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.358 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "nfs", 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.360 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.362 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.363 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.364 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.365 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.366 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: "fail_fs": { 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "fail_fs", 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.367 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.368 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.369 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.370 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.371 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.372 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator": { 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "orchestrator", 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm", 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "rook", 2024-05-27T08:21:40.374 INFO:teuthology.orchestra.run.smithi063.stdout: "test_orchestrator" 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Orchestrator backend", 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.375 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_perf_query", 2024-05-27T08:21:40.376 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.377 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.378 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.380 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.381 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.382 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "osd_support", 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.384 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.386 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.388 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.389 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.390 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "pg_autoscaler", 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.391 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.393 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.394 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.395 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:40.396 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.397 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "threshold": { 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "threshold", 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "3.0", 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1.0", 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "scaling threshold", 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.399 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "progress", 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_pg_recovery_event": { 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_pg_recovery_event", 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.400 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.401 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.402 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.403 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.404 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.405 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.406 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.407 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "max_completed_events": { 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_completed_events", 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "50", 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.408 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "number of past completed events to remember", 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "sleep_interval": { 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sleep_interval", 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "5", 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.409 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "how long the module is going to sleep", 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.410 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "prometheus", 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.411 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "cache": { 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "cache", 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.412 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "exclude_perf_counters": { 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "exclude_perf_counters", 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T08:21:40.413 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.414 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.415 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.417 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.418 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools": { 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools", 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.419 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T08:21:40.420 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "300", 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "scrape_interval": { 2024-05-27T08:21:40.421 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "scrape_interval", 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15.0", 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "::", 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.423 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "9283", 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.424 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.425 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "stale_cache_strategy": { 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stale_cache_strategy", 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "log", 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.426 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_behaviour": { 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_behaviour", 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "default", 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.427 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "default", 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "error" 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "standby_error_status_code": { 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "standby_error_status_code", 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.428 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "500", 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "400", 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "599", 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.429 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_support", 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.430 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.431 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.432 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.433 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.434 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_snap_create": { 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_snap_create", 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.435 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10", 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "mirror_snapshot_schedule": { 2024-05-27T08:21:40.436 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.437 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.438 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "trash_purge_schedule": { 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "trash_purge_schedule", 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.439 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "restful", 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_auth": { 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enable_auth", 2024-05-27T08:21:40.440 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "key_file": { 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "key_file", 2024-05-27T08:21:40.441 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.442 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.443 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.444 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.445 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.446 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.447 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "server_addr": { 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_addr", 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.448 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "server_port": { 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "server_port", 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.449 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.450 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.451 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rgw", 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.452 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.453 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.454 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.455 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.456 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.457 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.458 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rook", 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.459 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.460 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.461 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.462 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.463 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.464 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.465 INFO:teuthology.orchestra.run.smithi063.stdout: "storage_class": { 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "storage_class", 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "local", 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.466 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "selftest", 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.467 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.468 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.469 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.470 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.471 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "roption1": { 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption1", 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.473 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: "roption2": { 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "roption2", 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "xyz", 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.474 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption1": { 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption1", 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.475 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption2": { 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption2", 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.476 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption3": { 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption3", 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "float", 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.477 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption4": { 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption4", 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.478 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption5": { 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption5", 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.479 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption6": { 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption6", 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.480 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "rwoption7": { 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rwoption7", 2024-05-27T08:21:40.481 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "1", 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "42", 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.482 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.483 INFO:teuthology.orchestra.run.smithi063.stdout: "testkey": { 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testkey", 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.484 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "testlkey": { 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testlkey", 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.485 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "testnewline": { 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "testnewline", 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.486 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "smb", 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.487 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.488 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.489 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.490 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.491 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.492 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.493 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "update_orchestration": { 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "update_orchestration", 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.494 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snap_schedule", 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.495 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_m_granularity": { 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "allow_m_granularity", 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.496 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "dump_on_update": { 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "dump_on_update", 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "dump database to debug log on update", 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.497 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.498 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.499 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.500 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.501 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.502 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.503 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "stats", 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.504 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.505 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.506 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.507 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.508 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.509 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.510 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "status", 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.511 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.512 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telegraf", 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "address": { 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "address", 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "15", 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "telemetry", 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_basic": { 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_basic", 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_crash": { 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_crash", 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_device": { 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_device", 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T08:21:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_ident": { 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_ident", 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "channel_perf": { 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "channel_perf", 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "contact": { 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "contact", 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "description": { 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "description", 2024-05-27T08:21:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "device_url": { 2024-05-27T08:21:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "device_url", 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": { 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "enabled", 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "24", 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "8", 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "last_opt_revision": { 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "last_opt_revision", 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "1", 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard": { 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard", 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "leaderboard_description": { 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "leaderboard_description", 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: "organization": { 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "organization", 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "proxy": { 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "proxy", 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "url": { 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "url", 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "test_orchestrator", 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "volumes", 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "max_concurrent_clones": { 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "max_concurrent_clones", 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "4", 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "periodic_async_work": { 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "periodic_async_work", 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Periodically check for async work", 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_delay": { 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_delay", 2024-05-27T08:21:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "snapshot_clone_no_wait": { 2024-05-27T08:21:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "True", 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix", 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "can_run": true, 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "error_string": "", 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "module_options": { 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "discovery_interval": { 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "discovery_interval", 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "uint", 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "100", 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "identifier": { 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "identifier", 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "interval": { 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "interval", 2024-05-27T08:21:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "secs", 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "60", 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "log_level": { 2024-05-27T08:21:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_level", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster": { 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster", 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_cluster_level": { 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_cluster_level", 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "info", 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [ 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "", 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "critical", 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "debug", 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "error", 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "info", 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "warning" 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": { 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "log_to_file", 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "bool", 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "False", 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "sqlite3_killpoint": { 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "dev", 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "0", 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_host": { 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_host", 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "", 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_port": { 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_port", 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "int", 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "10051", 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "zabbix_sender": { 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "zabbix_sender", 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "str", 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "level": "advanced", 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 0, 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "min": "", 2024-05-27T08:21:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "max": "", 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "enum_allowed": [], 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "desc": "", 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "long_desc": "", 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "see_also": [] 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "services": { 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "dashboard": "https://172.21.15.63:8443/" 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "always_on_modules": { 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "octopus": [ 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "balancer", 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "devicehealth", 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator", 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscaler", 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "progress", 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_support", 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "status", 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "telemetry", 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "volumes" 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "pacific": [ 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "balancer", 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "devicehealth", 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator", 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscaler", 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "progress", 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_support", 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "status", 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "telemetry", 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "volumes" 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "quincy": [ 2024-05-27T08:21:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "balancer", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "devicehealth", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscaler", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "progress", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_support", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "status", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "telemetry", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "volumes" 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "reef": [ 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "balancer", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-27T08:21:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "devicehealth", 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator", 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscaler", 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "progress", 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_support", 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "status", 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "telemetry", 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "volumes" 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "squid": [ 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "balancer", 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "devicehealth", 2024-05-27T08:21:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "orchestrator", 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscaler", 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "progress", 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_support", 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "status", 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "telemetry", 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "volumes" 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "last_failure_osd_epoch": 92, 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "active_clients": [ 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "devicehealth", 2024-05-27T08:21:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 547681290 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "libcephsqlite", 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-27T08:21:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2747858029 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "rbd_support", 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2382679245 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snap_schedule", 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2165416894 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "snap_schedule", 2024-05-27T08:21:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1623462460 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "volumes", 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-05-27T08:21:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:0", 2024-05-27T08:21:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 227444425 2024-05-27T08:21:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-05-27T08:21:40.595 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:21:40.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:38 smithi149 ceph-mon[35408]: pgmap v1534: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1 op/s 2024-05-27T08:21:40.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:38 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/4244667053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T08:21:40.596 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:38 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2882211734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T08:21:40.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:39 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/997490739' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T08:21:40.597 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:38 smithi086 ceph-mon[34687]: pgmap v1534: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1 op/s 2024-05-27T08:21:40.597 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:38 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/4244667053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T08:21:40.597 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:38 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2882211734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T08:21:40.597 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:39 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/997490739' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T08:21:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.975+0000 7f09c7200640 1 -- 172.21.15.63:0/997490739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09a80b2950 msgr2=0x7f09a80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.975+0000 7f09c7200640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09a80b2950 0x7f09a80b4e60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f09b40057c0 tx=0x7f09b4009680 comp rx=0 tx=0).stop 2024-05-27T08:21:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.975+0000 7f09c7200640 1 -- 172.21.15.63:0/997490739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09c0109a60 msgr2=0x7f09c0075500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:21:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.975+0000 7f09c7200640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09c0109a60 0x7f09c0075500 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7f09b0005d00 tx=0x7f09b000dde0 comp rx=0 tx=0).stop 2024-05-27T08:21:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.975+0000 7f09c7200640 1 -- 172.21.15.63:0/997490739 shutdown_connections 2024-05-27T08:21:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.975+0000 7f09c7200640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f09a80b2950 0x7f09a80b4e60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.975+0000 7f09c7200640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09c0109f00 0x7f09c0075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.975+0000 7f09c7200640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09c0109a60 0x7f09c0075500 unknown :-1 s=CLOSED pgs=3044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:40.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.976+0000 7f09c7200640 1 --2- 172.21.15.63:0/997490739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09c01041c0 0x7f09c0074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:21:40.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.976+0000 7f09c7200640 1 -- 172.21.15.63:0/997490739 >> 172.21.15.63:0/997490739 conn(0x7f09c007a1e0 msgr2=0x7f09c0105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:21:40.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.976+0000 7f09c7200640 1 -- 172.21.15.63:0/997490739 shutdown_connections 2024-05-27T08:21:40.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:21:38.976+0000 7f09c7200640 1 -- 172.21.15.63:0/997490739 wait complete. 2024-05-27T08:21:40.652 DEBUG:tasks.check_counter:Getting stats from a 2024-05-27T08:21:40.652 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-27T08:21:40.653 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:38 smithi063 ceph-mon[31613]: pgmap v1534: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1 op/s 2024-05-27T08:21:40.653 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:38 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/4244667053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T08:21:40.653 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:38 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2882211734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T08:21:40.653 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:39 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/997490739' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T08:21:41.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:40 smithi086 ceph-mon[34687]: pgmap v1535: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2 op/s 2024-05-27T08:21:41.213 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:40 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 3. 2024-05-27T08:21:41.213 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:40 smithi149 systemd[1]: Stopped Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:41.213 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:40 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 1.408s CPU time. 2024-05-27T08:21:41.213 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:40 smithi149 systemd[1]: Starting Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:41.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:40 smithi149 ceph-mon[35408]: pgmap v1535: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2 op/s 2024-05-27T08:21:41.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:40 smithi063 ceph-mon[31613]: pgmap v1535: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2 op/s 2024-05-27T08:21:41.375 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:21:41.579 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:41 smithi149 podman[81878]: 2024-05-27 08:21:41.283536255 +0000 UTC m=+0.036496385 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:41.579 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:41 smithi149 podman[81878]: 2024-05-27 08:21:41.415126749 +0000 UTC m=+0.168086897 container create d02292b4ee92fe6be3cf2e29d4e1ac45b2818be09da0a493c267aaca0b3a28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:21:41.579 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:41 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 3. 2024-05-27T08:21:41.579 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:41 smithi149 systemd[1]: Stopped Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:41.579 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:41 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.441s CPU time. 2024-05-27T08:21:41.579 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:41 smithi149 systemd[1]: Starting Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:41.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:41 smithi149 podman[81878]: 2024-05-27 08:21:41.677189979 +0000 UTC m=+0.430150207 container init d02292b4ee92fe6be3cf2e29d4e1ac45b2818be09da0a493c267aaca0b3a28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T08:21:41.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:41 smithi149 podman[81878]: 2024-05-27 08:21:41.686263949 +0000 UTC m=+0.439224066 container start d02292b4ee92fe6be3cf2e29d4e1ac45b2818be09da0a493c267aaca0b3a28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:21:41.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:41 smithi149 bash[81878]: d02292b4ee92fe6be3cf2e29d4e1ac45b2818be09da0a493c267aaca0b3a28fc 2024-05-27T08:21:41.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:41 smithi149 systemd[1]: Started Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:41.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:41 smithi149 ceph-mds[81934]: -- 172.21.15.149:0/3276924622 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd987d5a40 con 0x55fd9881da80 2024-05-27T08:21:41.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:41 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: starting mds.f at 2024-05-27T08:21:42.214 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:41 smithi149 podman[81945]: 2024-05-27 08:21:41.839558134 +0000 UTC m=+0.035103136 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:42.215 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:41 smithi149 podman[81945]: 2024-05-27 08:21:41.977517343 +0000 UTC m=+0.173062382 container create d6072ca8f623810ffaf6f03d18f5df48f82a63d2f72da0975290880f7691ed8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:21:42.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 podman[81945]: 2024-05-27 08:21:42.239349059 +0000 UTC m=+0.434894144 container init d6072ca8f623810ffaf6f03d18f5df48f82a63d2f72da0975290880f7691ed8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-27T08:21:42.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 podman[81945]: 2024-05-27 08:21:42.249307116 +0000 UTC m=+0.444852162 container start d6072ca8f623810ffaf6f03d18f5df48f82a63d2f72da0975290880f7691ed8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T08:21:42.715 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-mds[81982]: -- 172.21.15.149:0/3026121688 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1e0883a40 con 0x55a1e08cb180 2024-05-27T08:21:42.715 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: starting mds.c at 2024-05-27T08:21:42.715 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 bash[81945]: d6072ca8f623810ffaf6f03d18f5df48f82a63d2f72da0975290880f7691ed8c 2024-05-27T08:21:42.715 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 systemd[1]: Started Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:43.194 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:42 smithi063 ceph-mon[31613]: pgmap v1536: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2 op/s 2024-05-27T08:21:43.194 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:42 smithi063 ceph-mon[31613]: daemon mds.f restarted 2024-05-27T08:21:43.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:42 smithi086 ceph-mon[34687]: pgmap v1536: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2 op/s 2024-05-27T08:21:43.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:42 smithi086 ceph-mon[34687]: daemon mds.f restarted 2024-05-27T08:21:43.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-mon[35408]: pgmap v1536: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2 op/s 2024-05-27T08:21:43.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-mon[35408]: daemon mds.f restarted 2024-05-27T08:21:43.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f59fe400700 time 2024-05-27T08:21:42.892328+0000 2024-05-27T08:21:43.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:43.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5a13f37ba6] 2024-05-27T08:21:43.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a13f37dd1] 2024-05-27T08:21:43.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fd7f6dd23d] 2024-05-27T08:21:43.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fd7f6c8971] 2024-05-27T08:21:43.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fd7f6c6fa0] 2024-05-27T08:21:43.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fd7f6c7651] 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fd7f65c347] 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fd7f35887e] 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fd7f35de94] 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fd7f368969] 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fd7f70148c] 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fd7f70a9cb] 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 13: (Thread::entry_wrapper()+0x43) [0x7f5a13f0672d] 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 14: (Thread::_entry_func(void*)+0xd) [0x7f5a13f06749] 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5a12aaa1ca] 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 16: clone() 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: *** Caught signal (Aborted) ** 2024-05-27T08:21:43.216 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: in thread 7f59fe400700 thread_name:md_log_replay 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f18aa800700 time 2024-05-27T08:21:42.890646+0000 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f18c0252ba6] 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f18c0252dd1] 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a1aca3523d] 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55a1aca20971] 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 5: (SnapRealm::check_cache() const+0x13e) [0x55a1aca1efa0] 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a1aca1f651] 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a1ac9b4347] 2024-05-27T08:21:43.217 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a1ac6b087e] 2024-05-27T08:21:43.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a1ac6b5f2f] 2024-05-27T08:21:43.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55a1ac6c0969] 2024-05-27T08:21:43.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 11: (MDLog::_replay_thread()+0x1f38) [0x55a1aca5948c] 2024-05-27T08:21:43.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55a1aca629cb] 2024-05-27T08:21:43.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 13: (Thread::entry_wrapper()+0x43) [0x7f18c022172d] 2024-05-27T08:21:43.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 14: (Thread::_entry_func(void*)+0xd) [0x7f18c0221749] 2024-05-27T08:21:43.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f18bedc51ca] 2024-05-27T08:21:43.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 16: clone() 2024-05-27T08:21:43.218 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:42.892+0000 7f18aa800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f18aa800700 time 2024-05-27T08:21:42.890646+0000 2024-05-27T08:21:43.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:42.894+0000 7f59fe400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f59fe400700 time 2024-05-27T08:21:42.892328+0000 2024-05-27T08:21:43.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:43.219 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5a13f37ba6] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a13f37dd1] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fd7f6dd23d] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fd7f6c8971] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fd7f6c6fa0] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fd7f6c7651] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fd7f65c347] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fd7f35887e] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fd7f35de94] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fd7f368969] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fd7f70148c] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fd7f70a9cb] 2024-05-27T08:21:43.220 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 13: (Thread::entry_wrapper()+0x43) [0x7f5a13f0672d] 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 14: (Thread::_entry_func(void*)+0xd) [0x7f5a13f06749] 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5a12aaa1ca] 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 16: clone() 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5a12ab4d20] 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2: gsignal() 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 3: abort() 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f5a13f37ced] 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a13f37dd1] 2024-05-27T08:21:43.221 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f18c0252ba6] 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f18c0252dd1] 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a1aca3523d] 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55a1aca20971] 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 5: (SnapRealm::check_cache() const+0x13e) [0x55a1aca1efa0] 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a1aca1f651] 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a1ac9b4347] 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a1ac6b087e] 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a1ac6b5f2f] 2024-05-27T08:21:43.222 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55a1ac6c0969] 2024-05-27T08:21:43.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 11: (MDLog::_replay_thread()+0x1f38) [0x55a1aca5948c] 2024-05-27T08:21:43.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55a1aca629cb] 2024-05-27T08:21:43.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 13: (Thread::entry_wrapper()+0x43) [0x7f18c022172d] 2024-05-27T08:21:43.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 14: (Thread::_entry_func(void*)+0xd) [0x7f18c0221749] 2024-05-27T08:21:43.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f18bedc51ca] 2024-05-27T08:21:43.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 16: clone() 2024-05-27T08:21:43.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: *** Caught signal (Aborted) ** 2024-05-27T08:21:43.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: in thread 7f18aa800700 thread_name:md_log_replay 2024-05-27T08:21:43.223 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f18bedcfd20] 2024-05-27T08:21:43.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2: gsignal() 2024-05-27T08:21:43.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 3: abort() 2024-05-27T08:21:43.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f18c0252ced] 2024-05-27T08:21:43.224 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f18c0252dd1] 2024-05-27T08:21:43.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fd7f6dd23d] 2024-05-27T08:21:43.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fd7f6c8971] 2024-05-27T08:21:43.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fd7f6c6fa0] 2024-05-27T08:21:43.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fd7f6c7651] 2024-05-27T08:21:43.224 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fd7f65c347] 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fd7f35887e] 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fd7f35de94] 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fd7f368969] 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fd7f70148c] 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fd7f70a9cb] 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 16: (Thread::entry_wrapper()+0x43) [0x7f5a13f0672d] 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 17: (Thread::_entry_func(void*)+0xd) [0x7f5a13f06749] 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5a12aaa1ca] 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 19: clone() 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:42.896+0000 7f59fe400700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: in thread 7f59fe400700 thread_name:md_log_replay 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.225 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5a12ab4d20] 2024-05-27T08:21:43.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2: gsignal() 2024-05-27T08:21:43.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 3: abort() 2024-05-27T08:21:43.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f5a13f37ced] 2024-05-27T08:21:43.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a13f37dd1] 2024-05-27T08:21:43.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fd7f6dd23d] 2024-05-27T08:21:43.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fd7f6c8971] 2024-05-27T08:21:43.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fd7f6c6fa0] 2024-05-27T08:21:43.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fd7f6c7651] 2024-05-27T08:21:43.226 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fd7f65c347] 2024-05-27T08:21:43.226 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a1aca3523d] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55a1aca20971] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 8: (SnapRealm::check_cache() const+0x13e) [0x55a1aca1efa0] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a1aca1f651] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a1ac9b4347] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a1ac6b087e] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a1ac6b5f2f] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55a1ac6c0969] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 14: (MDLog::_replay_thread()+0x1f38) [0x55a1aca5948c] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55a1aca629cb] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 16: (Thread::entry_wrapper()+0x43) [0x7f18c022172d] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 17: (Thread::_entry_func(void*)+0xd) [0x7f18c0221749] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f18bedc51ca] 2024-05-27T08:21:43.227 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 19: clone() 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:42.895+0000 7f18aa800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: in thread 7f18aa800700 thread_name:md_log_replay 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f18bedcfd20] 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2: gsignal() 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 3: abort() 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f18c0252ced] 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f18c0252dd1] 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a1aca3523d] 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55a1aca20971] 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 8: (SnapRealm::check_cache() const+0x13e) [0x55a1aca1efa0] 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a1aca1f651] 2024-05-27T08:21:43.228 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a1ac9b4347] 2024-05-27T08:21:43.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fd7f35887e] 2024-05-27T08:21:43.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fd7f35de94] 2024-05-27T08:21:43.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fd7f368969] 2024-05-27T08:21:43.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fd7f70148c] 2024-05-27T08:21:43.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fd7f70a9cb] 2024-05-27T08:21:43.229 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 16: (Thread::entry_wrapper()+0x43) [0x7f5a13f0672d] 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 17: (Thread::_entry_func(void*)+0xd) [0x7f5a13f06749] 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5a12aaa1ca] 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 19: clone() 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: -1> 2024-05-27T08:21:42.894+0000 7f59fe400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f59fe400700 time 2024-05-27T08:21:42.892328+0000 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5a13f37ba6] 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a13f37dd1] 2024-05-27T08:21:43.230 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fd7f6dd23d] 2024-05-27T08:21:43.231 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fd7f6c8971] 2024-05-27T08:21:43.231 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fd7f6c6fa0] 2024-05-27T08:21:43.231 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fd7f6c7651] 2024-05-27T08:21:43.231 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fd7f65c347] 2024-05-27T08:21:43.231 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fd7f35887e] 2024-05-27T08:21:43.231 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a1ac6b087e] 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a1ac6b5f2f] 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55a1ac6c0969] 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 14: (MDLog::_replay_thread()+0x1f38) [0x55a1aca5948c] 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55a1aca629cb] 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 16: (Thread::entry_wrapper()+0x43) [0x7f18c022172d] 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 17: (Thread::_entry_func(void*)+0xd) [0x7f18c0221749] 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f18bedc51ca] 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 19: clone() 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: -1> 2024-05-27T08:21:42.892+0000 7f18aa800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f18aa800700 time 2024-05-27T08:21:42.890646+0000 2024-05-27T08:21:43.232 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:43.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f18c0252ba6] 2024-05-27T08:21:43.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f18c0252dd1] 2024-05-27T08:21:43.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a1aca3523d] 2024-05-27T08:21:43.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55a1aca20971] 2024-05-27T08:21:43.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 5: (SnapRealm::check_cache() const+0x13e) [0x55a1aca1efa0] 2024-05-27T08:21:43.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a1aca1f651] 2024-05-27T08:21:43.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a1ac9b4347] 2024-05-27T08:21:43.233 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a1ac6b087e] 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a1ac6b5f2f] 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55a1ac6c0969] 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 11: (MDLog::_replay_thread()+0x1f38) [0x55a1aca5948c] 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55a1aca629cb] 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 13: (Thread::entry_wrapper()+0x43) [0x7f18c022172d] 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 14: (Thread::_entry_func(void*)+0xd) [0x7f18c0221749] 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f18bedc51ca] 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 16: clone() 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 0> 2024-05-27T08:21:42.895+0000 7f18aa800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: in thread 7f18aa800700 thread_name:md_log_replay 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.234 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f18bedcfd20] 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2: gsignal() 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 3: abort() 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f18c0252ced] 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f18c0252dd1] 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a1aca3523d] 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55a1aca20971] 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 8: (SnapRealm::check_cache() const+0x13e) [0x55a1aca1efa0] 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a1aca1f651] 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a1ac9b4347] 2024-05-27T08:21:43.235 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a1ac6b087e] 2024-05-27T08:21:43.236 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a1ac6b5f2f] 2024-05-27T08:21:43.236 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55a1ac6c0969] 2024-05-27T08:21:43.236 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 14: (MDLog::_replay_thread()+0x1f38) [0x55a1aca5948c] 2024-05-27T08:21:43.236 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55a1aca629cb] 2024-05-27T08:21:43.236 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 16: (Thread::entry_wrapper()+0x43) [0x7f18c022172d] 2024-05-27T08:21:43.236 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fd7f35de94] 2024-05-27T08:21:43.237 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fd7f368969] 2024-05-27T08:21:43.237 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fd7f70148c] 2024-05-27T08:21:43.237 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fd7f70a9cb] 2024-05-27T08:21:43.237 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 13: (Thread::entry_wrapper()+0x43) [0x7f5a13f0672d] 2024-05-27T08:21:43.237 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 14: (Thread::_entry_func(void*)+0xd) [0x7f5a13f06749] 2024-05-27T08:21:43.237 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5a12aaa1ca] 2024-05-27T08:21:43.237 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 16: clone() 2024-05-27T08:21:43.237 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.237 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 0> 2024-05-27T08:21:42.896+0000 7f59fe400700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: in thread 7f59fe400700 thread_name:md_log_replay 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5a12ab4d20] 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2: gsignal() 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 3: abort() 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f5a13f37ced] 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a13f37dd1] 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fd7f6dd23d] 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fd7f6c8971] 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fd7f6c6fa0] 2024-05-27T08:21:43.238 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fd7f6c7651] 2024-05-27T08:21:43.239 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fd7f65c347] 2024-05-27T08:21:43.239 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fd7f35887e] 2024-05-27T08:21:43.239 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fd7f35de94] 2024-05-27T08:21:43.239 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fd7f368969] 2024-05-27T08:21:43.239 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fd7f70148c] 2024-05-27T08:21:43.239 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fd7f70a9cb] 2024-05-27T08:21:43.239 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 16: (Thread::entry_wrapper()+0x43) [0x7f5a13f0672d] 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 17: (Thread::_entry_func(void*)+0xd) [0x7f5a13f06749] 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5a12aaa1ca] 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 19: clone() 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: -1> 2024-05-27T08:21:42.894+0000 7f59fe400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f59fe400700 time 2024-05-27T08:21:42.892328+0000 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5a13f37ba6] 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a13f37dd1] 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fd7f6dd23d] 2024-05-27T08:21:43.240 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55fd7f6c8971] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 5: (SnapRealm::check_cache() const+0x13e) [0x55fd7f6c6fa0] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fd7f6c7651] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fd7f65c347] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fd7f35887e] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fd7f35de94] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55fd7f368969] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 11: (MDLog::_replay_thread()+0x1f38) [0x55fd7f70148c] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55fd7f70a9cb] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 13: (Thread::entry_wrapper()+0x43) [0x7f5a13f0672d] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 14: (Thread::_entry_func(void*)+0xd) [0x7f5a13f06749] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5a12aaa1ca] 2024-05-27T08:21:43.241 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 16: clone() 2024-05-27T08:21:43.242 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 17: (Thread::_entry_func(void*)+0xd) [0x7f18c0221749] 2024-05-27T08:21:43.242 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f18bedc51ca] 2024-05-27T08:21:43.242 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 19: clone() 2024-05-27T08:21:43.242 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:43.242 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.243 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: -1> 2024-05-27T08:21:42.892+0000 7f18aa800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f18aa800700 time 2024-05-27T08:21:42.890646+0000 2024-05-27T08:21:43.243 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:43.243 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.243 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.243 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f18c0252ba6] 2024-05-27T08:21:43.243 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f18c0252dd1] 2024-05-27T08:21:43.243 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a1aca3523d] 2024-05-27T08:21:43.243 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55a1aca20971] 2024-05-27T08:21:43.243 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 5: (SnapRealm::check_cache() const+0x13e) [0x55a1aca1efa0] 2024-05-27T08:21:43.244 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a1aca1f651] 2024-05-27T08:21:43.244 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a1ac9b4347] 2024-05-27T08:21:43.244 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a1ac6b087e] 2024-05-27T08:21:43.244 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a1ac6b5f2f] 2024-05-27T08:21:43.244 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55a1ac6c0969] 2024-05-27T08:21:43.244 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 11: (MDLog::_replay_thread()+0x1f38) [0x55a1aca5948c] 2024-05-27T08:21:43.244 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55a1aca629cb] 2024-05-27T08:21:43.244 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 13: (Thread::entry_wrapper()+0x43) [0x7f18c022172d] 2024-05-27T08:21:43.244 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 14: (Thread::_entry_func(void*)+0xd) [0x7f18c0221749] 2024-05-27T08:21:43.245 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f18bedc51ca] 2024-05-27T08:21:43.245 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 16: clone() 2024-05-27T08:21:43.245 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.245 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 0> 2024-05-27T08:21:42.896+0000 7f59fe400700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:43.245 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: in thread 7f59fe400700 thread_name:md_log_replay 2024-05-27T08:21:43.245 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.245 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.245 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5a12ab4d20] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2: gsignal() 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 3: abort() 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f5a13f37ced] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a13f37dd1] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55fd7f6dd23d] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55fd7f6c8971] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 8: (SnapRealm::check_cache() const+0x13e) [0x55fd7f6c6fa0] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55fd7f6c7651] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55fd7f65c347] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55fd7f35887e] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55fd7f35de94] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55fd7f368969] 2024-05-27T08:21:43.246 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 14: (MDLog::_replay_thread()+0x1f38) [0x55fd7f70148c] 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55fd7f70a9cb] 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 16: (Thread::entry_wrapper()+0x43) [0x7f5a13f0672d] 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 17: (Thread::_entry_func(void*)+0xd) [0x7f5a13f06749] 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5a12aaa1ca] 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 19: clone() 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[81925]: 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:43 smithi149 podman[82026]: 2024-05-27 08:21:43.006620177 +0000 UTC m=+0.046523343 container died d02292b4ee92fe6be3cf2e29d4e1ac45b2818be09da0a493c267aaca0b3a28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 0> 2024-05-27T08:21:42.895+0000 7f18aa800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:43.247 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: in thread 7f18aa800700 thread_name:md_log_replay 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f18bedcfd20] 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2: gsignal() 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 3: abort() 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f18c0252ced] 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f18c0252dd1] 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55a1aca3523d] 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55a1aca20971] 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 8: (SnapRealm::check_cache() const+0x13e) [0x55a1aca1efa0] 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55a1aca1f651] 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55a1ac9b4347] 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55a1ac6b087e] 2024-05-27T08:21:43.248 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55a1ac6b5f2f] 2024-05-27T08:21:43.249 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55a1ac6c0969] 2024-05-27T08:21:43.249 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 14: (MDLog::_replay_thread()+0x1f38) [0x55a1aca5948c] 2024-05-27T08:21:43.249 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55a1aca629cb] 2024-05-27T08:21:43.249 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 16: (Thread::entry_wrapper()+0x43) [0x7f18c022172d] 2024-05-27T08:21:43.249 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 17: (Thread::_entry_func(void*)+0xd) [0x7f18c0221749] 2024-05-27T08:21:43.249 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f18bedc51ca] 2024-05-27T08:21:43.249 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 19: clone() 2024-05-27T08:21:43.249 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:43.249 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[81978]: 2024-05-27T08:21:43.249 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:43 smithi149 podman[82025]: 2024-05-27 08:21:43.006614752 +0000 UTC m=+0.046475999 container died d6072ca8f623810ffaf6f03d18f5df48f82a63d2f72da0975290880f7691ed8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T08:21:43.964 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:43 smithi149 podman[82025]: 2024-05-27 08:21:43.71110224 +0000 UTC m=+0.750963508 container remove d6072ca8f623810ffaf6f03d18f5df48f82a63d2f72da0975290880f7691ed8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:21:43.964 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:43 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:43.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:43 smithi149 ceph-mon[35408]: osdmap e184: 12 total, 12 up, 12 in 2024-05-27T08:21:43.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:43 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/1557918465,v1:172.21.15.149:6835/1557918465] up:boot 2024-05-27T08:21:43.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:43 smithi149 ceph-mon[35408]: daemon mds.c restarted 2024-05-27T08:21:43.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:43 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:43.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:43 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:21:43.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:43 smithi149 ceph-mon[35408]: osdmap e185: 12 total, 12 up, 12 in 2024-05-27T08:21:43.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:43 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6836/3856455113,v1:172.21.15.149:6837/3856455113] up:boot 2024-05-27T08:21:43.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:43 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:43.965 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:43 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:44.037 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:21:44.037 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:21:44.037 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 11, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 10, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 155296, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 4160, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 4, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 0, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.011168533, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.000788696, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.011980001, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000185428, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 9, 2024-05-27T08:21:44.038 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.001863066, 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000207007 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 155296, 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 4160 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 5, 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 9, 2024-05-27T08:21:44.039 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 3120, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 80240, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 3, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 2, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.012790481, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.000954041, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.011681189, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 8, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.002228982, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000278622 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.040 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 80240 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 0, 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 2, 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 0, 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 0, 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 2, 2024-05-27T08:21:44.041 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 1, 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.000743093, 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.000600716, 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:44.042 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 0, 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:21:44.043 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:21:44.044 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-05-27T08:21:44.045 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 148079, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 26, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:21:44.046 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 0, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 0, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 928, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 2, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:44.047 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 12, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 153804, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 12, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 12, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 153804, 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-27T08:21:44.048 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:44.049 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:44.049 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:44.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:44.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:44.049 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:21:44.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:43 smithi063 ceph-mon[31613]: osdmap e184: 12 total, 12 up, 12 in 2024-05-27T08:21:44.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:43 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/1557918465,v1:172.21.15.149:6835/1557918465] up:boot 2024-05-27T08:21:44.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:43 smithi063 ceph-mon[31613]: daemon mds.c restarted 2024-05-27T08:21:44.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:43 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:44.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:43 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:21:44.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:43 smithi063 ceph-mon[31613]: osdmap e185: 12 total, 12 up, 12 in 2024-05-27T08:21:44.140 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:43 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6836/3856455113,v1:172.21.15.149:6837/3856455113] up:boot 2024-05-27T08:21:44.141 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:43 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:44.141 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:43 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:43 smithi086 ceph-mon[34687]: osdmap e184: 12 total, 12 up, 12 in 2024-05-27T08:21:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:43 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/1557918465,v1:172.21.15.149:6835/1557918465] up:boot 2024-05-27T08:21:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:43 smithi086 ceph-mon[34687]: daemon mds.c restarted 2024-05-27T08:21:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:43 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:43 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:21:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:43 smithi086 ceph-mon[34687]: osdmap e185: 12 total, 12 up, 12 in 2024-05-27T08:21:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:43 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6836/3856455113,v1:172.21.15.149:6837/3856455113] up:boot 2024-05-27T08:21:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:43 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:43 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:44.713 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:44 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T08:21:44.713 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:44 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.461s CPU time. 2024-05-27T08:21:44.714 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:44 smithi149 podman[82026]: 2024-05-27 08:21:44.457245843 +0000 UTC m=+1.497149053 container remove d02292b4ee92fe6be3cf2e29d4e1ac45b2818be09da0a493c267aaca0b3a28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:21:44.714 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:44 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:45.024 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-27T08:21:45.024 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-27T08:21:45.024 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-27T08:21:45.024 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-27T08:21:45.024 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-27T08:21:45.024 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-05-27T08:21:45.024 DEBUG:tasks.check_counter:Getting stats from d 2024-05-27T08:21:45.024 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-27T08:21:45.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:44 smithi086 ceph-mon[34687]: pgmap v1539: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 op/s 2024-05-27T08:21:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:44 smithi149 ceph-mon[35408]: pgmap v1539: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 op/s 2024-05-27T08:21:45.214 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:44 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-05-27T08:21:45.214 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:44 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 1.333s CPU time. 2024-05-27T08:21:45.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:44 smithi063 ceph-mon[31613]: pgmap v1539: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 op/s 2024-05-27T08:21:45.494 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:21:46.633 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:46 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 3. 2024-05-27T08:21:46.633 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:46 smithi149 systemd[1]: Stopped Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:46.634 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:46 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 1.452s CPU time. 2024-05-27T08:21:46.634 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:46 smithi149 systemd[1]: Starting Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:46.962 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:46 smithi149 ceph-mon[35408]: pgmap v1540: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1 op/s 2024-05-27T08:21:46.963 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:46 smithi149 podman[82159]: 2024-05-27 08:21:46.541466049 +0000 UTC m=+0.040572250 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:46.963 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:46 smithi149 podman[82159]: 2024-05-27 08:21:46.697426841 +0000 UTC m=+0.196533060 container create 836c9c27d8c48fdf2138e1eff3aba5d16b346ff5e8cd7ba620e7819e49ece263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-27T08:21:46.963 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:46 smithi149 podman[82159]: 2024-05-27 08:21:46.96138729 +0000 UTC m=+0.460493516 container init 836c9c27d8c48fdf2138e1eff3aba5d16b346ff5e8cd7ba620e7819e49ece263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:21:47.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:46 smithi086 ceph-mon[34687]: pgmap v1540: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1 op/s 2024-05-27T08:21:47.214 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:46 smithi149 podman[82159]: 2024-05-27 08:21:46.972743543 +0000 UTC m=+0.471849735 container start 836c9c27d8c48fdf2138e1eff3aba5d16b346ff5e8cd7ba620e7819e49ece263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:21:47.214 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 bash[82159]: 836c9c27d8c48fdf2138e1eff3aba5d16b346ff5e8cd7ba620e7819e49ece263 2024-05-27T08:21:47.214 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 systemd[1]: Started Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:47.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:46 smithi063 ceph-mon[31613]: pgmap v1540: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1 op/s 2024-05-27T08:21:47.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-mds[82178]: -- 172.21.15.149:0/23262737 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564ef129a40 con 0x5564ef171600 2024-05-27T08:21:47.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: starting mds.i at 2024-05-27T08:21:47.992 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:21:47.992 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:21:47.992 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 0, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 2, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 0, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 0, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 4, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 0, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.004037339, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.000555059, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:47.993 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 869, 2024-05-27T08:21:47.994 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 858, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1897536, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 532688, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 2, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 1, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.344829145, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.160542590, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.136838685, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.047587028, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 857, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.092595024, 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000108045 2024-05-27T08:21:47.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 1897536, 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 532688 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 8, 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 659, 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 3600, 2024-05-27T08:21:47.996 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 431200, 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 7, 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 3, 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.137679816, 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.110898427, 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.009819339, 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 657, 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.072229083, 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000109937 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:47.997 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 3600, 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 431200 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 0, 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-05-27T08:21:47.998 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:21:47.999 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-05-27T08:21:47.999 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:21:47.999 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:21:47.999 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:21:47.999 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:21:47.999 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:21:47.999 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:21:47.999 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:21:47.999 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:21:47.999 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:21:48.000 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 148714, 2024-05-27T08:21:48.001 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 30, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-05-27T08:21:48.002 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 928, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 2, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 873, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 1774626, 2024-05-27T08:21:48.003 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 873, 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 873, 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 1774626, 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:48.004 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:21:48.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:47 smithi086 ceph-mon[34687]: daemon mds.i restarted 2024-05-27T08:21:48.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:47 smithi149 ceph-mon[35408]: daemon mds.i restarted 2024-05-27T08:21:48.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f04c3a00700 time 2024-05-27T08:21:47.929181+0000 2024-05-27T08:21:48.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f04d94a6ba6] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04d94a6dd1] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5564bfe4e23d] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5564bfe39971] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 5: (SnapRealm::check_cache() const+0x13e) [0x5564bfe37fa0] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5564bfe38651] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5564bfdcd347] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5564bfac987e] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5564bfacee94] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5564bfad9969] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 11: (MDLog::_replay_thread()+0x1f38) [0x5564bfe7248c] 2024-05-27T08:21:48.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5564bfe7b9cb] 2024-05-27T08:21:48.217 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 13: (Thread::entry_wrapper()+0x43) [0x7f04d947572d] 2024-05-27T08:21:48.217 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 14: (Thread::_entry_func(void*)+0xd) [0x7f04d9475749] 2024-05-27T08:21:48.217 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f04d80191ca] 2024-05-27T08:21:48.217 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 16: clone() 2024-05-27T08:21:48.217 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:47.930+0000 7f04c3a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f04c3a00700 time 2024-05-27T08:21:47.929181+0000 2024-05-27T08:21:48.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:47 smithi063 ceph-mon[31613]: daemon mds.i restarted 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f04d94a6ba6] 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04d94a6dd1] 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5564bfe4e23d] 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5564bfe39971] 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 5: (SnapRealm::check_cache() const+0x13e) [0x5564bfe37fa0] 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5564bfe38651] 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5564bfdcd347] 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5564bfac987e] 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5564bfacee94] 2024-05-27T08:21:48.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5564bfad9969] 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 11: (MDLog::_replay_thread()+0x1f38) [0x5564bfe7248c] 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5564bfe7b9cb] 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 13: (Thread::entry_wrapper()+0x43) [0x7f04d947572d] 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 14: (Thread::_entry_func(void*)+0xd) [0x7f04d9475749] 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f04d80191ca] 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 16: clone() 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: *** Caught signal (Aborted) ** 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: in thread 7f04c3a00700 thread_name:md_log_replay 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f04d8023d20] 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2: gsignal() 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 3: abort() 2024-05-27T08:21:48.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f04d94a6ced] 2024-05-27T08:21:48.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04d94a6dd1] 2024-05-27T08:21:48.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5564bfe4e23d] 2024-05-27T08:21:48.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5564bfe39971] 2024-05-27T08:21:48.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 8: (SnapRealm::check_cache() const+0x13e) [0x5564bfe37fa0] 2024-05-27T08:21:48.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5564bfe38651] 2024-05-27T08:21:48.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5564bfdcd347] 2024-05-27T08:21:48.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5564bfac987e] 2024-05-27T08:21:48.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5564bfacee94] 2024-05-27T08:21:48.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5564bfad9969] 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 14: (MDLog::_replay_thread()+0x1f38) [0x5564bfe7248c] 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5564bfe7b9cb] 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 16: (Thread::entry_wrapper()+0x43) [0x7f04d947572d] 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 17: (Thread::_entry_func(void*)+0xd) [0x7f04d9475749] 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f04d80191ca] 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 19: clone() 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:47.933+0000 7f04c3a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: in thread 7f04c3a00700 thread_name:md_log_replay 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f04d8023d20] 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2: gsignal() 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 3: abort() 2024-05-27T08:21:48.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f04d94a6ced] 2024-05-27T08:21:48.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04d94a6dd1] 2024-05-27T08:21:48.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5564bfe4e23d] 2024-05-27T08:21:48.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5564bfe39971] 2024-05-27T08:21:48.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 8: (SnapRealm::check_cache() const+0x13e) [0x5564bfe37fa0] 2024-05-27T08:21:48.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5564bfe38651] 2024-05-27T08:21:48.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5564bfdcd347] 2024-05-27T08:21:48.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5564bfac987e] 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5564bfacee94] 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5564bfad9969] 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 14: (MDLog::_replay_thread()+0x1f38) [0x5564bfe7248c] 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5564bfe7b9cb] 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 16: (Thread::entry_wrapper()+0x43) [0x7f04d947572d] 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 17: (Thread::_entry_func(void*)+0xd) [0x7f04d9475749] 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f04d80191ca] 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 19: clone() 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: -1> 2024-05-27T08:21:47.930+0000 7f04c3a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f04c3a00700 time 2024-05-27T08:21:47.929181+0000 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:48.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f04d94a6ba6] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04d94a6dd1] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5564bfe4e23d] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5564bfe39971] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 5: (SnapRealm::check_cache() const+0x13e) [0x5564bfe37fa0] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5564bfe38651] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5564bfdcd347] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5564bfac987e] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5564bfacee94] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5564bfad9969] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 11: (MDLog::_replay_thread()+0x1f38) [0x5564bfe7248c] 2024-05-27T08:21:48.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5564bfe7b9cb] 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 13: (Thread::entry_wrapper()+0x43) [0x7f04d947572d] 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 14: (Thread::_entry_func(void*)+0xd) [0x7f04d9475749] 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f04d80191ca] 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 16: clone() 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 0> 2024-05-27T08:21:47.933+0000 7f04c3a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: in thread 7f04c3a00700 thread_name:md_log_replay 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f04d8023d20] 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2: gsignal() 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 3: abort() 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f04d94a6ced] 2024-05-27T08:21:48.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04d94a6dd1] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5564bfe4e23d] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5564bfe39971] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 8: (SnapRealm::check_cache() const+0x13e) [0x5564bfe37fa0] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5564bfe38651] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5564bfdcd347] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5564bfac987e] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5564bfacee94] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5564bfad9969] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 14: (MDLog::_replay_thread()+0x1f38) [0x5564bfe7248c] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5564bfe7b9cb] 2024-05-27T08:21:48.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 16: (Thread::entry_wrapper()+0x43) [0x7f04d947572d] 2024-05-27T08:21:48.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 17: (Thread::_entry_func(void*)+0xd) [0x7f04d9475749] 2024-05-27T08:21:48.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f04d80191ca] 2024-05-27T08:21:48.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 19: clone() 2024-05-27T08:21:48.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:48.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: -1> 2024-05-27T08:21:47.930+0000 7f04c3a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f04c3a00700 time 2024-05-27T08:21:47.929181+0000 2024-05-27T08:21:48.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:48.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:48.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f04d94a6ba6] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04d94a6dd1] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5564bfe4e23d] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5564bfe39971] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 5: (SnapRealm::check_cache() const+0x13e) [0x5564bfe37fa0] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5564bfe38651] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5564bfdcd347] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5564bfac987e] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5564bfacee94] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5564bfad9969] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 11: (MDLog::_replay_thread()+0x1f38) [0x5564bfe7248c] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5564bfe7b9cb] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 13: (Thread::entry_wrapper()+0x43) [0x7f04d947572d] 2024-05-27T08:21:48.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 14: (Thread::_entry_func(void*)+0xd) [0x7f04d9475749] 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f04d80191ca] 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 16: clone() 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 0> 2024-05-27T08:21:47.933+0000 7f04c3a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: in thread 7f04c3a00700 thread_name:md_log_replay 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f04d8023d20] 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2: gsignal() 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 3: abort() 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f04d94a6ced] 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f04d94a6dd1] 2024-05-27T08:21:48.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5564bfe4e23d] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5564bfe39971] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 8: (SnapRealm::check_cache() const+0x13e) [0x5564bfe37fa0] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5564bfe38651] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5564bfdcd347] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5564bfac987e] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5564bfacee94] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5564bfad9969] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 14: (MDLog::_replay_thread()+0x1f38) [0x5564bfe7248c] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5564bfe7b9cb] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 16: (Thread::entry_wrapper()+0x43) [0x7f04d947572d] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 17: (Thread::_entry_func(void*)+0xd) [0x7f04d9475749] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f04d80191ca] 2024-05-27T08:21:48.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 19: clone() 2024-05-27T08:21:48.231 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:48.231 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82174]: 2024-05-27T08:21:48.231 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:48 smithi149 podman[82209]: 2024-05-27 08:21:48.018446542 +0000 UTC m=+0.034588964 container died 836c9c27d8c48fdf2138e1eff3aba5d16b346ff5e8cd7ba620e7819e49ece263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T08:21:48.886 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-27T08:21:48.886 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-27T08:21:48.886 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-27T08:21:48.887 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-27T08:21:48.887 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-27T08:21:48.887 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-27T08:21:48.887 DEBUG:tasks.check_counter:Getting stats from g 2024-05-27T08:21:48.888 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-27T08:21:49.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:48 smithi086 ceph-mon[34687]: pgmap v1541: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 op/s 2024-05-27T08:21:49.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:48 smithi086 ceph-mon[34687]: osdmap e186: 12 total, 12 up, 12 in 2024-05-27T08:21:49.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:48 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/2904238399,v1:172.21.15.149:6835/2904238399] up:boot 2024-05-27T08:21:49.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:48 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:49.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:48 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:21:49.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:48 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:49.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:48 smithi149 ceph-mon[35408]: pgmap v1541: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 op/s 2024-05-27T08:21:49.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:48 smithi149 ceph-mon[35408]: osdmap e186: 12 total, 12 up, 12 in 2024-05-27T08:21:49.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:48 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/2904238399,v1:172.21.15.149:6835/2904238399] up:boot 2024-05-27T08:21:49.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:48 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:49.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:48 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:21:49.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:48 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:49.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:48 smithi149 podman[82209]: 2024-05-27 08:21:48.74690423 +0000 UTC m=+0.763046696 container remove 836c9c27d8c48fdf2138e1eff3aba5d16b346ff5e8cd7ba620e7819e49ece263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-27T08:21:49.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:48 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:49.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:48 smithi063 ceph-mon[31613]: pgmap v1541: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 op/s 2024-05-27T08:21:49.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:48 smithi063 ceph-mon[31613]: osdmap e186: 12 total, 12 up, 12 in 2024-05-27T08:21:49.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:48 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/2904238399,v1:172.21.15.149:6835/2904238399] up:boot 2024-05-27T08:21:49.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:48 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:49.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:48 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:21:49.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:48 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:49.362 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:21:49.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:49 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-05-27T08:21:49.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:49 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 1.533s CPU time. 2024-05-27T08:21:50.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:50 smithi063 ceph-mon[31613]: pgmap v1543: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 3 op/s 2024-05-27T08:21:51.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:50 smithi086 ceph-mon[34687]: pgmap v1543: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 3 op/s 2024-05-27T08:21:51.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:50 smithi149 ceph-mon[35408]: pgmap v1543: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 3 op/s 2024-05-27T08:21:51.805 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:21:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 863, 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 852, 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1895280, 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 529152, 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 4, 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 0, 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.351037561, 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.160176869, 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.148293596, 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.044225978, 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 851, 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.104232883, 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000122482 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 1895280, 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 529152 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 4, 2024-05-27T08:21:51.807 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 653, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 640, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 430048, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 7, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 3, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.130789430, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.108205308, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.006972122, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 650, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.097952594, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000150696 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 640, 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 430048 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 4, 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 4, 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 2960, 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 448, 2024-05-27T08:21:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 2, 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 1, 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.001896278, 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.000081319, 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.001713249, 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 3, 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.001956433, 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000652144 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 0, 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:21:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-05-27T08:21:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-05-27T08:21:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 148731, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 31, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-05-27T08:21:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 928, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 2, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-05-27T08:21:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 867, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 1773209, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 867, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 867, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 1773209, 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-27T08:21:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:51.817 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:21:52.828 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-27T08:21:52.828 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-27T08:21:52.828 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-27T08:21:52.828 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-27T08:21:52.828 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-27T08:21:52.828 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-05-27T08:21:52.828 DEBUG:tasks.check_counter:Getting stats from j 2024-05-27T08:21:52.829 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-27T08:21:53.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:52 smithi086 ceph-mon[34687]: pgmap v1544: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 2 op/s 2024-05-27T08:21:53.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:52 smithi149 ceph-mon[35408]: pgmap v1544: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 2 op/s 2024-05-27T08:21:53.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:52 smithi063 ceph-mon[31613]: pgmap v1544: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 2 op/s 2024-05-27T08:21:53.320 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:21:54.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:54 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 4. 2024-05-27T08:21:54.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:54 smithi149 systemd[1]: Stopped Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:54.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:54 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 1.333s CPU time. 2024-05-27T08:21:54.980 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:54 smithi149 ceph-mon[35408]: pgmap v1545: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2 op/s 2024-05-27T08:21:54.980 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:54 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 4. 2024-05-27T08:21:54.980 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:54 smithi149 systemd[1]: Stopped Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:54.980 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:54 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.461s CPU time. 2024-05-27T08:21:54.980 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:54 smithi149 systemd[1]: Starting Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:55.123 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:54 smithi063 ceph-mon[31613]: pgmap v1545: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2 op/s 2024-05-27T08:21:55.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:54 smithi086 ceph-mon[34687]: pgmap v1545: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2 op/s 2024-05-27T08:21:55.308 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:54 smithi149 systemd[1]: Starting Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:21:55.308 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 podman[82301]: 2024-05-27 08:21:55.012071972 +0000 UTC m=+0.032675255 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:55.308 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 podman[82301]: 2024-05-27 08:21:55.142064825 +0000 UTC m=+0.162668145 container create a5ac9a0a016b1140d9cad4fc400154f19754a7b3daa64e6944ad466dac4af16c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-27T08:21:55.713 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 podman[82301]: 2024-05-27 08:21:55.420918176 +0000 UTC m=+0.441521543 container init a5ac9a0a016b1140d9cad4fc400154f19754a7b3daa64e6944ad466dac4af16c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T08:21:55.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 podman[82301]: 2024-05-27 08:21:55.431962204 +0000 UTC m=+0.452565501 container start a5ac9a0a016b1140d9cad4fc400154f19754a7b3daa64e6944ad466dac4af16c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-27T08:21:55.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 bash[82301]: a5ac9a0a016b1140d9cad4fc400154f19754a7b3daa64e6944ad466dac4af16c 2024-05-27T08:21:55.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 systemd[1]: Started Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:55.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-mds[82343]: -- 172.21.15.149:0/781456658 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562dbc77ba40 con 0x562dbc7c3180 2024-05-27T08:21:55.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: starting mds.c at 2024-05-27T08:21:55.913 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-27T08:21:55.913 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:21:55.913 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 0, 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 2, 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 0, 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 0, 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 4, 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 0, 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.003506420, 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.000605252, 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:55.914 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:21:55.915 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 873, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 862, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 1898624, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 535232, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 2, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 1, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.352546930, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.157115191, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.161462750, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.045756393, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 861, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.090960855, 2024-05-27T08:21:55.916 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000105645 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 1898624, 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 535232 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_messages": 9, 2024-05-27T08:21:55.917 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages": 662, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_bytes": 3760, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_bytes": 510640, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_created_connections": 8, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_active_connections": 3, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_total_time": 0.135044655, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_send_time": 0.109136413, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_recv_time": 0.009038619, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 660, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.118876367, 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000180115 2024-05-27T08:21:55.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_recv_encrypted_bytes": 3760, 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: "msgr_send_encrypted_bytes": 510640 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: "cct": { 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: "total_workers": 0, 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: "unhealthy_workers": 0 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.919 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool": { 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bloom_filter_items": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:21:55.920 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_items": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blob_items": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:21:55.921 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_txc_items": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_writing_items": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_bytes": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_items": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_bytes": 148754, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_anon_items": 31, 2024-05-27T08:21:55.922 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "buffer_meta_items": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bytes": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_items": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mapbl_items": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pglog_items": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_bytes": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_items": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:21:55.923 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_bytes": 0, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "pgmap_items": 0, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_bytes": 928, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_co_items": 2, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_bytes": 0, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_1_items": 0, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_bytes": 0, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "unittest_2_items": 0 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "val": 0, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "max": 104857600, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "get_started": 0, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "get": 878, 2024-05-27T08:21:55.924 INFO:teuthology.orchestra.run.smithi063.stdout: "get_sum": 1775173, 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_fail": 0, 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: "get_or_fail_success": 878, 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: "take": 0, 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: "take_sum": 0, 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: "put": 878, 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: "put_sum": 1775173, 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: "wait": { 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: "avgcount": 0, 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: "sum": 0.000000000, 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: "avgtime": 0.000000000 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-27T08:21:55.925 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-27T08:21:56.117 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-mon[35408]: daemon mds.c restarted 2024-05-27T08:21:56.117 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:55 smithi149 podman[82385]: 2024-05-27 08:21:55.709496564 +0000 UTC m=+0.039436698 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:21:56.117 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:55 smithi149 podman[82385]: 2024-05-27 08:21:55.846112742 +0000 UTC m=+0.176052890 container create 45387f0e0909a84949d3ff65ba8962fd3cde75dd306a1511db78ebbef185358c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-27T08:21:56.119 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7ff2e6600700 time 2024-05-27T08:21:55.981371+0000 2024-05-27T08:21:56.119 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:56.119 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.119 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff2fc0eaba6] 2024-05-27T08:21:56.119 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff2fc0eadd1] 2024-05-27T08:21:56.119 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x562db01f223d] 2024-05-27T08:21:56.119 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 4: (SnapRealm::build_snap_set() const+0x323) [0x562db01dd971] 2024-05-27T08:21:56.119 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 5: (SnapRealm::check_cache() const+0x13e) [0x562db01dbfa0] 2024-05-27T08:21:56.119 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x562db01dc651] 2024-05-27T08:21:56.119 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x562db0171347] 2024-05-27T08:21:56.120 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x562dafe6d87e] 2024-05-27T08:21:56.120 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x562dafe72f2f] 2024-05-27T08:21:56.120 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x562dafe7d969] 2024-05-27T08:21:56.120 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 11: (MDLog::_replay_thread()+0x1f38) [0x562db021648c] 2024-05-27T08:21:56.120 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 12: (MDLog::ReplayThread::entry()+0x11) [0x562db021f9cb] 2024-05-27T08:21:56.120 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 13: (Thread::entry_wrapper()+0x43) [0x7ff2fc0b972d] 2024-05-27T08:21:56.120 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 14: (Thread::_entry_func(void*)+0xd) [0x7ff2fc0b9749] 2024-05-27T08:21:56.120 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff2fac5d1ca] 2024-05-27T08:21:56.120 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 16: clone() 2024-05-27T08:21:56.120 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:55.983+0000 7ff2e6600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7ff2e6600700 time 2024-05-27T08:21:55.981371+0000 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff2fc0eaba6] 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff2fc0eadd1] 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x562db01f223d] 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 4: (SnapRealm::build_snap_set() const+0x323) [0x562db01dd971] 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 5: (SnapRealm::check_cache() const+0x13e) [0x562db01dbfa0] 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x562db01dc651] 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x562db0171347] 2024-05-27T08:21:56.121 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x562dafe6d87e] 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x562dafe72f2f] 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x562dafe7d969] 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 11: (MDLog::_replay_thread()+0x1f38) [0x562db021648c] 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 12: (MDLog::ReplayThread::entry()+0x11) [0x562db021f9cb] 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 13: (Thread::entry_wrapper()+0x43) [0x7ff2fc0b972d] 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 14: (Thread::_entry_func(void*)+0xd) [0x7ff2fc0b9749] 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff2fac5d1ca] 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 16: clone() 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: *** Caught signal (Aborted) ** 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: in thread 7ff2e6600700 thread_name:md_log_replay 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.122 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff2fac67d20] 2024-05-27T08:21:56.123 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2: gsignal() 2024-05-27T08:21:56.123 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 3: abort() 2024-05-27T08:21:56.123 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7ff2fc0eaced] 2024-05-27T08:21:56.123 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff2fc0eadd1] 2024-05-27T08:21:56.123 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x562db01f223d] 2024-05-27T08:21:56.123 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 7: (SnapRealm::build_snap_set() const+0x323) [0x562db01dd971] 2024-05-27T08:21:56.123 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 8: (SnapRealm::check_cache() const+0x13e) [0x562db01dbfa0] 2024-05-27T08:21:56.123 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x562db01dc651] 2024-05-27T08:21:56.123 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x562db0171347] 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x562dafe6d87e] 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x562dafe72f2f] 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x562dafe7d969] 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 14: (MDLog::_replay_thread()+0x1f38) [0x562db021648c] 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 15: (MDLog::ReplayThread::entry()+0x11) [0x562db021f9cb] 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 16: (Thread::entry_wrapper()+0x43) [0x7ff2fc0b972d] 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 17: (Thread::_entry_func(void*)+0xd) [0x7ff2fc0b9749] 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff2fac5d1ca] 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 19: clone() 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:55.986+0000 7ff2e6600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: in thread 7ff2e6600700 thread_name:md_log_replay 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.124 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff2fac67d20] 2024-05-27T08:21:56.125 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2: gsignal() 2024-05-27T08:21:56.125 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 3: abort() 2024-05-27T08:21:56.125 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7ff2fc0eaced] 2024-05-27T08:21:56.125 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff2fc0eadd1] 2024-05-27T08:21:56.125 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x562db01f223d] 2024-05-27T08:21:56.125 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 7: (SnapRealm::build_snap_set() const+0x323) [0x562db01dd971] 2024-05-27T08:21:56.125 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 8: (SnapRealm::check_cache() const+0x13e) [0x562db01dbfa0] 2024-05-27T08:21:56.125 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x562db01dc651] 2024-05-27T08:21:56.125 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x562db0171347] 2024-05-27T08:21:56.126 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x562dafe6d87e] 2024-05-27T08:21:56.126 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x562dafe72f2f] 2024-05-27T08:21:56.126 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x562dafe7d969] 2024-05-27T08:21:56.126 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 14: (MDLog::_replay_thread()+0x1f38) [0x562db021648c] 2024-05-27T08:21:56.126 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 15: (MDLog::ReplayThread::entry()+0x11) [0x562db021f9cb] 2024-05-27T08:21:56.126 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 16: (Thread::entry_wrapper()+0x43) [0x7ff2fc0b972d] 2024-05-27T08:21:56.126 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 17: (Thread::_entry_func(void*)+0xd) [0x7ff2fc0b9749] 2024-05-27T08:21:56.126 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff2fac5d1ca] 2024-05-27T08:21:56.126 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 19: clone() 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: -1> 2024-05-27T08:21:55.983+0000 7ff2e6600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7ff2e6600700 time 2024-05-27T08:21:55.981371+0000 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff2fc0eaba6] 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff2fc0eadd1] 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x562db01f223d] 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 4: (SnapRealm::build_snap_set() const+0x323) [0x562db01dd971] 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 5: (SnapRealm::check_cache() const+0x13e) [0x562db01dbfa0] 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x562db01dc651] 2024-05-27T08:21:56.127 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x562db0171347] 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x562dafe6d87e] 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x562dafe72f2f] 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x562dafe7d969] 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 11: (MDLog::_replay_thread()+0x1f38) [0x562db021648c] 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 12: (MDLog::ReplayThread::entry()+0x11) [0x562db021f9cb] 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 13: (Thread::entry_wrapper()+0x43) [0x7ff2fc0b972d] 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 14: (Thread::_entry_func(void*)+0xd) [0x7ff2fc0b9749] 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff2fac5d1ca] 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 16: clone() 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 0> 2024-05-27T08:21:55.986+0000 7ff2e6600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: in thread 7ff2e6600700 thread_name:md_log_replay 2024-05-27T08:21:56.128 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.129 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.129 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff2fac67d20] 2024-05-27T08:21:56.129 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2: gsignal() 2024-05-27T08:21:56.129 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 3: abort() 2024-05-27T08:21:56.129 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7ff2fc0eaced] 2024-05-27T08:21:56.129 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff2fc0eadd1] 2024-05-27T08:21:56.129 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x562db01f223d] 2024-05-27T08:21:56.129 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 7: (SnapRealm::build_snap_set() const+0x323) [0x562db01dd971] 2024-05-27T08:21:56.129 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 8: (SnapRealm::check_cache() const+0x13e) [0x562db01dbfa0] 2024-05-27T08:21:56.130 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x562db01dc651] 2024-05-27T08:21:56.130 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x562db0171347] 2024-05-27T08:21:56.130 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x562dafe6d87e] 2024-05-27T08:21:56.130 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x562dafe72f2f] 2024-05-27T08:21:56.130 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x562dafe7d969] 2024-05-27T08:21:56.130 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 14: (MDLog::_replay_thread()+0x1f38) [0x562db021648c] 2024-05-27T08:21:56.130 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 15: (MDLog::ReplayThread::entry()+0x11) [0x562db021f9cb] 2024-05-27T08:21:56.130 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 16: (Thread::entry_wrapper()+0x43) [0x7ff2fc0b972d] 2024-05-27T08:21:56.131 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 17: (Thread::_entry_func(void*)+0xd) [0x7ff2fc0b9749] 2024-05-27T08:21:56.131 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff2fac5d1ca] 2024-05-27T08:21:56.131 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 19: clone() 2024-05-27T08:21:56.131 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:56.131 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.131 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: -1> 2024-05-27T08:21:55.983+0000 7ff2e6600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7ff2e6600700 time 2024-05-27T08:21:55.981371+0000 2024-05-27T08:21:56.131 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:56.131 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.131 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.131 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff2fc0eaba6] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff2fc0eadd1] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x562db01f223d] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 4: (SnapRealm::build_snap_set() const+0x323) [0x562db01dd971] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 5: (SnapRealm::check_cache() const+0x13e) [0x562db01dbfa0] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x562db01dc651] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x562db0171347] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x562dafe6d87e] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x562dafe72f2f] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x562dafe7d969] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 11: (MDLog::_replay_thread()+0x1f38) [0x562db021648c] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 12: (MDLog::ReplayThread::entry()+0x11) [0x562db021f9cb] 2024-05-27T08:21:56.132 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 13: (Thread::entry_wrapper()+0x43) [0x7ff2fc0b972d] 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 14: (Thread::_entry_func(void*)+0xd) [0x7ff2fc0b9749] 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff2fac5d1ca] 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 16: clone() 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 0> 2024-05-27T08:21:55.986+0000 7ff2e6600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: in thread 7ff2e6600700 thread_name:md_log_replay 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff2fac67d20] 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2: gsignal() 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 3: abort() 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7ff2fc0eaced] 2024-05-27T08:21:56.133 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff2fc0eadd1] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x562db01f223d] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 7: (SnapRealm::build_snap_set() const+0x323) [0x562db01dd971] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 8: (SnapRealm::check_cache() const+0x13e) [0x562db01dbfa0] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x562db01dc651] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x562db0171347] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x562dafe6d87e] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x562dafe72f2f] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x562dafe7d969] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 14: (MDLog::_replay_thread()+0x1f38) [0x562db021648c] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 15: (MDLog::ReplayThread::entry()+0x11) [0x562db021f9cb] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 16: (Thread::entry_wrapper()+0x43) [0x7ff2fc0b972d] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 17: (Thread::_entry_func(void*)+0xd) [0x7ff2fc0b9749] 2024-05-27T08:21:56.134 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff2fac5d1ca] 2024-05-27T08:21:56.135 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 19: clone() 2024-05-27T08:21:56.135 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:56.135 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82330]: 2024-05-27T08:21:56.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:55 smithi086 ceph-mon[34687]: daemon mds.c restarted 2024-05-27T08:21:56.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:55 smithi063 ceph-mon[31613]: daemon mds.c restarted 2024-05-27T08:21:56.406 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 podman[82385]: 2024-05-27 08:21:56.116746259 +0000 UTC m=+0.446686417 container init 45387f0e0909a84949d3ff65ba8962fd3cde75dd306a1511db78ebbef185358c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:21:56.406 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 podman[82385]: 2024-05-27 08:21:56.127435227 +0000 UTC m=+0.457375339 container start 45387f0e0909a84949d3ff65ba8962fd3cde75dd306a1511db78ebbef185358c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:21:56.406 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mds[82426]: -- 172.21.15.149:0/2607718376 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c1e573a40 con 0x556c1e5bba80 2024-05-27T08:21:56.406 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: starting mds.f at 2024-05-27T08:21:56.406 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 bash[82385]: 45387f0e0909a84949d3ff65ba8962fd3cde75dd306a1511db78ebbef185358c 2024-05-27T08:21:56.406 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 systemd[1]: Started Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:56.685 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:56 smithi149 podman[82415]: 2024-05-27 08:21:56.405055897 +0000 UTC m=+0.371861298 container died a5ac9a0a016b1140d9cad4fc400154f19754a7b3daa64e6944ad466dac4af16c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T08:21:56.943 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-27T08:21:56.943 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-27T08:21:56.943 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-27T08:21:56.943 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-27T08:21:56.943 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-27T08:21:56.943 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-27T08:21:56.943 DEBUG:tasks.check_counter:Getting stats from b 2024-05-27T08:21:56.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: pgmap v1546: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1 op/s 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: osdmap e187: 12 total, 12 up, 12 in 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/1203249337,v1:172.21.15.149:6835/1203249337] up:boot 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: daemon mds.f restarted 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: osdmap e188: 12 total, 12 up, 12 in 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/3590006489,v1:172.21.15.149:6835/3590006489] up:boot 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:56.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:56 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:21:56.965 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc1f9800700 time 2024-05-27T08:21:56.688248+0000 2024-05-27T08:21:56.965 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:56.965 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.965 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc20f3ccba6] 2024-05-27T08:21:56.965 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc20f3ccdd1] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556be4df023d] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556be4ddb971] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 5: (SnapRealm::check_cache() const+0x13e) [0x556be4dd9fa0] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556be4dda651] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556be4d6f347] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556be4a6b87e] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556be4a70e94] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556be4a7b969] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 11: (MDLog::_replay_thread()+0x1f38) [0x556be4e1448c] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556be4e1d9cb] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 13: (Thread::entry_wrapper()+0x43) [0x7fc20f39b72d] 2024-05-27T08:21:56.966 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc20f39b749] 2024-05-27T08:21:56.967 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc20df3f1ca] 2024-05-27T08:21:56.967 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 16: clone() 2024-05-27T08:21:56.967 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: *** Caught signal (Aborted) ** 2024-05-27T08:21:56.967 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: in thread 7fc1f9800700 thread_name:md_log_replay 2024-05-27T08:21:56.967 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.690+0000 7fc1f9800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc1f9800700 time 2024-05-27T08:21:56.688248+0000 2024-05-27T08:21:56.967 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc20f3ccba6] 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc20f3ccdd1] 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556be4df023d] 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556be4ddb971] 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 5: (SnapRealm::check_cache() const+0x13e) [0x556be4dd9fa0] 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556be4dda651] 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556be4d6f347] 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556be4a6b87e] 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556be4a70e94] 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556be4a7b969] 2024-05-27T08:21:56.968 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 11: (MDLog::_replay_thread()+0x1f38) [0x556be4e1448c] 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556be4e1d9cb] 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 13: (Thread::entry_wrapper()+0x43) [0x7fc20f39b72d] 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc20f39b749] 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc20df3f1ca] 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 16: clone() 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc20df49d20] 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2: gsignal() 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 3: abort() 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc20f3ccced] 2024-05-27T08:21:56.969 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc20f3ccdd1] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556be4df023d] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556be4ddb971] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 8: (SnapRealm::check_cache() const+0x13e) [0x556be4dd9fa0] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556be4dda651] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556be4d6f347] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556be4a6b87e] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556be4a70e94] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556be4a7b969] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 14: (MDLog::_replay_thread()+0x1f38) [0x556be4e1448c] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556be4e1d9cb] 2024-05-27T08:21:56.970 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 16: (Thread::entry_wrapper()+0x43) [0x7fc20f39b72d] 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc20f39b749] 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc20df3f1ca] 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 19: clone() 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.694+0000 7fc1f9800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: in thread 7fc1f9800700 thread_name:md_log_replay 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc20df49d20] 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2: gsignal() 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 3: abort() 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc20f3ccced] 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc20f3ccdd1] 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556be4df023d] 2024-05-27T08:21:56.971 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556be4ddb971] 2024-05-27T08:21:56.972 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 8: (SnapRealm::check_cache() const+0x13e) [0x556be4dd9fa0] 2024-05-27T08:21:56.972 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556be4dda651] 2024-05-27T08:21:56.972 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556be4d6f347] 2024-05-27T08:21:56.972 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556be4a6b87e] 2024-05-27T08:21:56.972 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556be4a70e94] 2024-05-27T08:21:56.972 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556be4a7b969] 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 14: (MDLog::_replay_thread()+0x1f38) [0x556be4e1448c] 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556be4e1d9cb] 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 16: (Thread::entry_wrapper()+0x43) [0x7fc20f39b72d] 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc20f39b749] 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc20df3f1ca] 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 19: clone() 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: -1> 2024-05-27T08:21:56.690+0000 7fc1f9800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc1f9800700 time 2024-05-27T08:21:56.688248+0000 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.973 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc20f3ccba6] 2024-05-27T08:21:56.974 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc20f3ccdd1] 2024-05-27T08:21:56.974 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556be4df023d] 2024-05-27T08:21:56.974 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556be4ddb971] 2024-05-27T08:21:56.974 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 5: (SnapRealm::check_cache() const+0x13e) [0x556be4dd9fa0] 2024-05-27T08:21:56.974 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556be4dda651] 2024-05-27T08:21:56.974 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556be4d6f347] 2024-05-27T08:21:56.974 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556be4a6b87e] 2024-05-27T08:21:56.974 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556be4a70e94] 2024-05-27T08:21:56.974 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556be4a7b969] 2024-05-27T08:21:56.974 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 11: (MDLog::_replay_thread()+0x1f38) [0x556be4e1448c] 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556be4e1d9cb] 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 13: (Thread::entry_wrapper()+0x43) [0x7fc20f39b72d] 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc20f39b749] 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc20df3f1ca] 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 16: clone() 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 0> 2024-05-27T08:21:56.694+0000 7fc1f9800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: in thread 7fc1f9800700 thread_name:md_log_replay 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc20df49d20] 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2: gsignal() 2024-05-27T08:21:56.975 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 3: abort() 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc20f3ccced] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc20f3ccdd1] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556be4df023d] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556be4ddb971] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 8: (SnapRealm::check_cache() const+0x13e) [0x556be4dd9fa0] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556be4dda651] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556be4d6f347] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556be4a6b87e] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556be4a70e94] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556be4a7b969] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 14: (MDLog::_replay_thread()+0x1f38) [0x556be4e1448c] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556be4e1d9cb] 2024-05-27T08:21:56.976 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 16: (Thread::entry_wrapper()+0x43) [0x7fc20f39b72d] 2024-05-27T08:21:56.977 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc20f39b749] 2024-05-27T08:21:56.977 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc20df3f1ca] 2024-05-27T08:21:56.977 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 19: clone() 2024-05-27T08:21:56.977 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:56.977 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.977 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: -1> 2024-05-27T08:21:56.690+0000 7fc1f9800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc1f9800700 time 2024-05-27T08:21:56.688248+0000 2024-05-27T08:21:56.977 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:21:56.977 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.977 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.977 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc20f3ccba6] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc20f3ccdd1] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556be4df023d] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556be4ddb971] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 5: (SnapRealm::check_cache() const+0x13e) [0x556be4dd9fa0] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556be4dda651] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556be4d6f347] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556be4a6b87e] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556be4a70e94] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556be4a7b969] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 11: (MDLog::_replay_thread()+0x1f38) [0x556be4e1448c] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556be4e1d9cb] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 13: (Thread::entry_wrapper()+0x43) [0x7fc20f39b72d] 2024-05-27T08:21:56.978 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc20f39b749] 2024-05-27T08:21:56.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc20df3f1ca] 2024-05-27T08:21:56.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 16: clone() 2024-05-27T08:21:56.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 0> 2024-05-27T08:21:56.694+0000 7fc1f9800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:21:56.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: in thread 7fc1f9800700 thread_name:md_log_replay 2024-05-27T08:21:56.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:56.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:21:56.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc20df49d20] 2024-05-27T08:21:56.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2: gsignal() 2024-05-27T08:21:56.979 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 3: abort() 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc20f3ccced] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc20f3ccdd1] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556be4df023d] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556be4ddb971] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 8: (SnapRealm::check_cache() const+0x13e) [0x556be4dd9fa0] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556be4dda651] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556be4d6f347] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556be4a6b87e] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556be4a70e94] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556be4a7b969] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 14: (MDLog::_replay_thread()+0x1f38) [0x556be4e1448c] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556be4e1d9cb] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 16: (Thread::entry_wrapper()+0x43) [0x7fc20f39b72d] 2024-05-27T08:21:56.980 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc20f39b749] 2024-05-27T08:21:56.981 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc20df3f1ca] 2024-05-27T08:21:56.981 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 19: clone() 2024-05-27T08:21:56.981 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:21:56.981 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:56 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82412]: 2024-05-27T08:21:57.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: pgmap v1546: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1 op/s 2024-05-27T08:21:57.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: osdmap e187: 12 total, 12 up, 12 in 2024-05-27T08:21:57.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/1203249337,v1:172.21.15.149:6835/1203249337] up:boot 2024-05-27T08:21:57.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:57.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:57.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:57.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: daemon mds.f restarted 2024-05-27T08:21:57.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: osdmap e188: 12 total, 12 up, 12 in 2024-05-27T08:21:57.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/3590006489,v1:172.21.15.149:6835/3590006489] up:boot 2024-05-27T08:21:57.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:57.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:56 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:21:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: pgmap v1546: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1 op/s 2024-05-27T08:21:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: osdmap e187: 12 total, 12 up, 12 in 2024-05-27T08:21:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/1203249337,v1:172.21.15.149:6835/1203249337] up:boot 2024-05-27T08:21:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:21:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:21:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: daemon mds.f restarted 2024-05-27T08:21:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: osdmap e188: 12 total, 12 up, 12 in 2024-05-27T08:21:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/3590006489,v1:172.21.15.149:6835/3590006489] up:boot 2024-05-27T08:21:57.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:21:57.218 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:56 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:21:57.458 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:57 smithi149 podman[82461]: 2024-05-27 08:21:57.123029956 +0000 UTC m=+0.373927315 container died 45387f0e0909a84949d3ff65ba8962fd3cde75dd306a1511db78ebbef185358c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-27T08:21:57.458 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:57 smithi149 podman[82415]: 2024-05-27 08:21:57.05520837 +0000 UTC m=+1.022013765 container remove a5ac9a0a016b1140d9cad4fc400154f19754a7b3daa64e6944ad466dac4af16c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-27T08:21:57.458 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:57 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:21:58.044 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:57 smithi149 podman[82461]: 2024-05-27 08:21:57.792925617 +0000 UTC m=+1.043822936 container remove 45387f0e0909a84949d3ff65ba8962fd3cde75dd306a1511db78ebbef185358c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:21:58.044 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:57 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:21:58.464 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:58 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-05-27T08:21:58.464 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:21:58 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 1.388s CPU time. 2024-05-27T08:21:58.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:58 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T08:21:58.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:21:58 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.395s CPU time. 2024-05-27T08:21:59.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:21:58 smithi086 ceph-mon[34687]: pgmap v1549: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s 2024-05-27T08:21:59.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:21:58 smithi149 ceph-mon[35408]: pgmap v1549: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s 2024-05-27T08:21:59.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:21:58 smithi063 ceph-mon[31613]: pgmap v1549: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s 2024-05-27T08:21:59.872 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:59 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 4. 2024-05-27T08:21:59.872 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:59 smithi149 systemd[1]: Stopped Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:21:59.872 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:59 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 1.533s CPU time. 2024-05-27T08:21:59.872 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:59 smithi149 systemd[1]: Starting Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:00.079 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-05-27T08:22:00.079 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:22:00.079 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 10365, 2024-05-27T08:22:00.079 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 10782, 2024-05-27T08:22:00.079 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 4084121, 2024-05-27T08:22:00.079 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 65770219, 2024-05-27T08:22:00.079 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 772, 2024-05-27T08:22:00.079 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 3.148872777, 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.907407489, 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 1.176097551, 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.660214547, 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 10776, 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.680830115, 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000155979 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1055, 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000315837, 2024-05-27T08:22:00.080 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000299 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 102304, 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 1197888 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 13769, 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 15425, 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 7865806, 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 27358506, 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 13, 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 3.638258923, 2024-05-27T08:22:00.081 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 1.549697484, 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 1.242042090, 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.766108111, 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 15428, 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.501788971, 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000162158 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 8614, 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.006957108, 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000807 2024-05-27T08:22:00.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2218320, 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 630752 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 34544, 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 34867, 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 13491075, 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 25696645, 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 17, 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 7, 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 8.986874698, 2024-05-27T08:22:00.083 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 3.432828792, 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 3.872185424, 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 1.898860842, 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 34865, 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 5.193517861, 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000148960 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 44213, 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.025159964, 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000569 2024-05-27T08:22:00.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1551376, 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 704272 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 19395, 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 18.267592427, 2024-05-27T08:22:00.085 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000941871 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5704, 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.517892815, 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000090794 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.086 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 7982, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 7970, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 7970, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 329.739508090, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.041372585 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 12, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 45, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 42, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 2245, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 3, 2024-05-27T08:22:00.087 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 1509, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 937, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 559, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 13, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 162, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 17, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 2, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 3, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 4, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 13220, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 7021, 2024-05-27T08:22:00.088 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 5, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 53, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 6, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 12, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 6, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 17, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 14156, 2024-05-27T08:22:00.089 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 4, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 234, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 7, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 14156, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 3675, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 45, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 3630, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 10481, 2024-05-27T08:22:00.090 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 1855, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 3222, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 6, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 5719, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 2762, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 48, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 2868 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-05-27T08:22:00.091 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 11, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 82, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 927, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 784, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 61, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-05-27T08:22:00.092 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 45, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 6, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 3, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.093 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 8974, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 8858, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 8858, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 116, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 1076, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 223, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 222, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 222, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 1, 2024-05-27T08:22:00.094 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 1, 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 2, 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 81294283, 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 81552900, 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3593, 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 85.951636525, 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.023921969 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.095 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 951, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 1791, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 840, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 120, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 183, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 63, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 1509, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 2705, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 1196, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 3, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 891, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 888, 2024-05-27T08:22:00.096 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 146256, 2024-05-27T08:22:00.097 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 231824 2024-05-27T08:22:00.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.097 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-05-27T08:22:00.097 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 9186, 2024-05-27T08:22:00.097 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 12, 2024-05-27T08:22:00.097 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 7982, 2024-05-27T08:22:00.097 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 87, 2024-05-27T08:22:00.097 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 36, 2024-05-27T08:22:00.097 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-05-27T08:22:00.098 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 765, 2024-05-27T08:22:00.098 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.879236397, 2024-05-27T08:22:00.098 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.003763707 2024-05-27T08:22:00.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.098 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-05-27T08:22:00.098 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1720, 2024-05-27T08:22:00.098 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.334871880, 2024-05-27T08:22:00.098 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000776088 2024-05-27T08:22:00.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 61, 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.410705727, 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.006732880 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-05-27T08:22:00.099 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3328, 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.615338704, 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000785858 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.101 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.102 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.102 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-05-27T08:22:00.102 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 48, 2024-05-27T08:22:00.102 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.028454354, 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000592799 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 105, 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.343721588, 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.003273538 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 48, 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.104410138, 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.023008544 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 615, 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 303.355037551, 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.493260223 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-05-27T08:22:00.104 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 201, 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.869280580, 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.009299903 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 113, 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 9.026454354, 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.079880127 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 98, 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.510877564, 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.025621199 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.106 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 41, 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.806660756, 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.019674652 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5, 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.012293313, 2024-05-27T08:22:00.107 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002458662 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4, 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.003908931, 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000977232 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-05-27T08:22:00.108 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 9, 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.065060166, 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.007228907 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 13, 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.031545654, 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002426588 2024-05-27T08:22:00.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 796, 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3.341650433, 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.004198053 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:00.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.111 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-05-27T08:22:00.111 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-05-27T08:22:00.111 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 6, 2024-05-27T08:22:00.111 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 4, 2024-05-27T08:22:00.111 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-05-27T08:22:00.111 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-05-27T08:22:00.111 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 1007, 2024-05-27T08:22:00.111 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 503, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 2890, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:22:00.112 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:22:00.113 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:22:00.114 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 965311, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 289, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9192, 2024-05-27T08:22:00.115 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 176, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 3453486, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 36721, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-05-27T08:22:00.116 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 29527, 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 86762038, 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 37, 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 29527, 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 29527, 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 782.271140583, 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.026493417 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-05-27T08:22:00.117 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 29527, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 36693 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 29527, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 14420, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 15107, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 2120, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 1081, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 132, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 3680, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 743, 2024-05-27T08:22:00.118 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 1, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 6383, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 14288, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 1757, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-05-27T08:22:00.119 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 6, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-05-27T08:22:00.120 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 188, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 108, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 12, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 12, 2024-05-27T08:22:00.121 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 5873, 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 174, 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 455 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.122 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 13 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 60, 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 10, 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 5833, 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 784, 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:22:00.123 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 58674, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 19919429, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 58674, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 58674, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 19919429, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.124 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 29527, 2024-05-27T08:22:00.125 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 86863758, 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 29527, 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 86863758, 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-05-27T08:22:00.126 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 29527, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 29527, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 29527, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 29527, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.127 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 784, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 102608, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 784, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 87, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 102608, 2024-05-27T08:22:00.128 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x56117fbba9c0": { 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 8974, 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 77358596, 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:00.129 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 8974, 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3593, 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 77358596, 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:00.130 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-05-27T08:22:00.188 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:59 smithi149 podman[82580]: 2024-05-27 08:21:59.77616224 +0000 UTC m=+0.035219361 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:00.188 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:21:59 smithi149 podman[82580]: 2024-05-27 08:21:59.886040424 +0000 UTC m=+0.145097585 container create 6661df27ecb0616702a5a8defc8761c2341d8c98718dc4cee9fc1bf1cc2dad48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-27T08:22:00.464 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 podman[82580]: 2024-05-27 08:22:00.186478839 +0000 UTC m=+0.445536014 container init 6661df27ecb0616702a5a8defc8761c2341d8c98718dc4cee9fc1bf1cc2dad48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-27T08:22:00.464 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 podman[82580]: 2024-05-27 08:22:00.196762255 +0000 UTC m=+0.455819419 container start 6661df27ecb0616702a5a8defc8761c2341d8c98718dc4cee9fc1bf1cc2dad48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux ) 2024-05-27T08:22:00.464 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 bash[82580]: 6661df27ecb0616702a5a8defc8761c2341d8c98718dc4cee9fc1bf1cc2dad48 2024-05-27T08:22:00.465 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 systemd[1]: Started Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:00.465 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-mds[82599]: -- 172.21.15.149:0/1922067620 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600afe4ba40 con 0x5600afe93600 2024-05-27T08:22:00.465 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: starting mds.i at 2024-05-27T08:22:01.061 INFO:tasks.check_counter:Daemon mds.b mds.exported=6 2024-05-27T08:22:01.061 INFO:tasks.check_counter:Daemon mds.b mds.imported=6 2024-05-27T08:22:01.061 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-05-27T08:22:01.061 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=48 2024-05-27T08:22:01.061 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=41 2024-05-27T08:22:01.061 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=3 2024-05-27T08:22:01.061 DEBUG:tasks.check_counter:Getting stats from e 2024-05-27T08:22:01.062 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-27T08:22:01.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:00 smithi086 ceph-mon[34687]: pgmap v1550: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2 op/s 2024-05-27T08:22:01.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:00 smithi086 ceph-mon[34687]: daemon mds.i restarted 2024-05-27T08:22:01.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:00 smithi149 ceph-mon[35408]: pgmap v1550: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2 op/s 2024-05-27T08:22:01.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:00 smithi149 ceph-mon[35408]: daemon mds.i restarted 2024-05-27T08:22:01.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa1b1800700 time 2024-05-27T08:22:00.993407+0000 2024-05-27T08:22:01.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:01.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:01.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa1c7343ba6] 2024-05-27T08:22:01.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa1c7343dd1] 2024-05-27T08:22:01.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560087e6023d] 2024-05-27T08:22:01.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560087e4b971] 2024-05-27T08:22:01.215 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 5: (SnapRealm::check_cache() const+0x13e) [0x560087e49fa0] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560087e4a651] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560087ddf347] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560087adb87e] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560087ae0e94] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560087aeb969] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 11: (MDLog::_replay_thread()+0x1f38) [0x560087e8448c] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560087e8d9cb] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 13: (Thread::entry_wrapper()+0x43) [0x7fa1c731272d] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa1c7312749] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa1c5eb61ca] 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 16: clone() 2024-05-27T08:22:01.216 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:00.995+0000 7fa1b1800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa1b1800700 time 2024-05-27T08:22:00.993407+0000 2024-05-27T08:22:01.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:00 smithi063 ceph-mon[31613]: pgmap v1550: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2 op/s 2024-05-27T08:22:01.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:00 smithi063 ceph-mon[31613]: daemon mds.i restarted 2024-05-27T08:22:01.217 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:01.217 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.217 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:01.217 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa1c7343ba6] 2024-05-27T08:22:01.217 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa1c7343dd1] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560087e6023d] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560087e4b971] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 5: (SnapRealm::check_cache() const+0x13e) [0x560087e49fa0] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560087e4a651] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560087ddf347] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560087adb87e] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560087ae0e94] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560087aeb969] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 11: (MDLog::_replay_thread()+0x1f38) [0x560087e8448c] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560087e8d9cb] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 13: (Thread::entry_wrapper()+0x43) [0x7fa1c731272d] 2024-05-27T08:22:01.218 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa1c7312749] 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa1c5eb61ca] 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 16: clone() 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: *** Caught signal (Aborted) ** 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: in thread 7fa1b1800700 thread_name:md_log_replay 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa1c5ec0d20] 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2: gsignal() 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 3: abort() 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa1c7343ced] 2024-05-27T08:22:01.219 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa1c7343dd1] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560087e6023d] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560087e4b971] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 8: (SnapRealm::check_cache() const+0x13e) [0x560087e49fa0] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560087e4a651] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560087ddf347] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560087adb87e] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560087ae0e94] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560087aeb969] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 14: (MDLog::_replay_thread()+0x1f38) [0x560087e8448c] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560087e8d9cb] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 16: (Thread::entry_wrapper()+0x43) [0x7fa1c731272d] 2024-05-27T08:22:01.220 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa1c7312749] 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa1c5eb61ca] 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 19: clone() 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:00.998+0000 7fa1b1800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: in thread 7fa1b1800700 thread_name:md_log_replay 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa1c5ec0d20] 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2: gsignal() 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 3: abort() 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa1c7343ced] 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa1c7343dd1] 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560087e6023d] 2024-05-27T08:22:01.221 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560087e4b971] 2024-05-27T08:22:01.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 8: (SnapRealm::check_cache() const+0x13e) [0x560087e49fa0] 2024-05-27T08:22:01.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560087e4a651] 2024-05-27T08:22:01.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560087ddf347] 2024-05-27T08:22:01.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560087adb87e] 2024-05-27T08:22:01.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560087ae0e94] 2024-05-27T08:22:01.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560087aeb969] 2024-05-27T08:22:01.222 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 14: (MDLog::_replay_thread()+0x1f38) [0x560087e8448c] 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560087e8d9cb] 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 16: (Thread::entry_wrapper()+0x43) [0x7fa1c731272d] 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa1c7312749] 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa1c5eb61ca] 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 19: clone() 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:00 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: -1> 2024-05-27T08:22:00.995+0000 7fa1b1800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa1b1800700 time 2024-05-27T08:22:00.993407+0000 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa1c7343ba6] 2024-05-27T08:22:01.223 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa1c7343dd1] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560087e6023d] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560087e4b971] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 5: (SnapRealm::check_cache() const+0x13e) [0x560087e49fa0] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560087e4a651] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560087ddf347] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560087adb87e] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560087ae0e94] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560087aeb969] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 11: (MDLog::_replay_thread()+0x1f38) [0x560087e8448c] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560087e8d9cb] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 13: (Thread::entry_wrapper()+0x43) [0x7fa1c731272d] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa1c7312749] 2024-05-27T08:22:01.224 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa1c5eb61ca] 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 16: clone() 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 0> 2024-05-27T08:22:00.998+0000 7fa1b1800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: in thread 7fa1b1800700 thread_name:md_log_replay 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa1c5ec0d20] 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2: gsignal() 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 3: abort() 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa1c7343ced] 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa1c7343dd1] 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560087e6023d] 2024-05-27T08:22:01.225 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560087e4b971] 2024-05-27T08:22:01.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 8: (SnapRealm::check_cache() const+0x13e) [0x560087e49fa0] 2024-05-27T08:22:01.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560087e4a651] 2024-05-27T08:22:01.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560087ddf347] 2024-05-27T08:22:01.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560087adb87e] 2024-05-27T08:22:01.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560087ae0e94] 2024-05-27T08:22:01.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560087aeb969] 2024-05-27T08:22:01.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 14: (MDLog::_replay_thread()+0x1f38) [0x560087e8448c] 2024-05-27T08:22:01.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560087e8d9cb] 2024-05-27T08:22:01.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 16: (Thread::entry_wrapper()+0x43) [0x7fa1c731272d] 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa1c7312749] 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa1c5eb61ca] 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 19: clone() 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: -1> 2024-05-27T08:22:00.995+0000 7fa1b1800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa1b1800700 time 2024-05-27T08:22:00.993407+0000 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa1c7343ba6] 2024-05-27T08:22:01.227 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa1c7343dd1] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560087e6023d] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560087e4b971] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 5: (SnapRealm::check_cache() const+0x13e) [0x560087e49fa0] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560087e4a651] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560087ddf347] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560087adb87e] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560087ae0e94] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560087aeb969] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 11: (MDLog::_replay_thread()+0x1f38) [0x560087e8448c] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560087e8d9cb] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 13: (Thread::entry_wrapper()+0x43) [0x7fa1c731272d] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa1c7312749] 2024-05-27T08:22:01.228 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa1c5eb61ca] 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 16: clone() 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 0> 2024-05-27T08:22:00.998+0000 7fa1b1800700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: in thread 7fa1b1800700 thread_name:md_log_replay 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa1c5ec0d20] 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2: gsignal() 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 3: abort() 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa1c7343ced] 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa1c7343dd1] 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560087e6023d] 2024-05-27T08:22:01.229 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560087e4b971] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 8: (SnapRealm::check_cache() const+0x13e) [0x560087e49fa0] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560087e4a651] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560087ddf347] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560087adb87e] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560087ae0e94] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560087aeb969] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 14: (MDLog::_replay_thread()+0x1f38) [0x560087e8448c] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560087e8d9cb] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 16: (Thread::entry_wrapper()+0x43) [0x7fa1c731272d] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa1c7312749] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa1c5eb61ca] 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 19: clone() 2024-05-27T08:22:01.230 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:01.231 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82595]: 2024-05-27T08:22:01.231 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 podman[82630]: 2024-05-27 08:22:01.07853826 +0000 UTC m=+0.032648419 container died 6661df27ecb0616702a5a8defc8761c2341d8c98718dc4cee9fc1bf1cc2dad48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T08:22:01.524 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:22:02.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:01 smithi086 ceph-mon[34687]: osdmap e189: 12 total, 12 up, 12 in 2024-05-27T08:22:02.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:01 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/3200379414,v1:172.21.15.149:6835/3200379414] up:boot 2024-05-27T08:22:02.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:01 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:22:02.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:01 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:22:02.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:01 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:02.213 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 podman[82630]: 2024-05-27 08:22:01.810381911 +0000 UTC m=+0.764492087 container remove 6661df27ecb0616702a5a8defc8761c2341d8c98718dc4cee9fc1bf1cc2dad48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-05-27T08:22:02.213 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:01 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:22:02.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:01 smithi149 ceph-mon[35408]: osdmap e189: 12 total, 12 up, 12 in 2024-05-27T08:22:02.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:01 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/3200379414,v1:172.21.15.149:6835/3200379414] up:boot 2024-05-27T08:22:02.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:01 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:22:02.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:01 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:22:02.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:01 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:02.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:01 smithi063 ceph-mon[31613]: osdmap e189: 12 total, 12 up, 12 in 2024-05-27T08:22:02.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:01 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/3200379414,v1:172.21.15.149:6835/3200379414] up:boot 2024-05-27T08:22:02.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:01 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:22:02.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:01 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:22:02.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:01 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:02.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:02 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-05-27T08:22:02.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:02 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 1.393s CPU time. 2024-05-27T08:22:03.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:02 smithi086 ceph-mon[34687]: pgmap v1552: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2 op/s 2024-05-27T08:22:03.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:02 smithi149 ceph-mon[35408]: pgmap v1552: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2 op/s 2024-05-27T08:22:03.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:02 smithi063 ceph-mon[31613]: pgmap v1552: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2 op/s 2024-05-27T08:22:04.030 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-05-27T08:22:04.030 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:22:04.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 297, 2024-05-27T08:22:04.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 481, 2024-05-27T08:22:04.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 83324, 2024-05-27T08:22:04.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1252936, 2024-05-27T08:22:04.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 102, 2024-05-27T08:22:04.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-05-27T08:22:04.030 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.204080557, 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.041656791, 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.117031213, 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.007766487, 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 479, 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.113174006, 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000236271 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 391, 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000328764, 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000840 2024-05-27T08:22:04.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 928, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 1216 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 132, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 393, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 33498, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 286308, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 23, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 6, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.094365525, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.035523161, 2024-05-27T08:22:04.032 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.042738151, 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.007279667, 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 391, 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.095001146, 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000242969 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 279, 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000147612, 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000529 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 13632, 2024-05-27T08:22:04.033 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 202848 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 6641, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 5422, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3675891, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 3542280, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 14, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 7, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 1.935529823, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 1.022640506, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 1.023317947, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.074885324, 2024-05-27T08:22:04.034 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:04.035 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5421, 2024-05-27T08:22:04.035 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.681267622, 2024-05-27T08:22:04.035 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000125671 2024-05-27T08:22:04.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.035 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:04.035 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 8785, 2024-05-27T08:22:04.035 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.005693960, 2024-05-27T08:22:04.035 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000648 2024-05-27T08:22:04.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2156416, 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 1777856 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 285, 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.052334289, 2024-05-27T08:22:04.036 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000183629 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 23, 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.001638611, 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000071243 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 26, 2024-05-27T08:22:04.037 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 23, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 23, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.054314378, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002361494 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 3, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 11, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-05-27T08:22:04.038 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 20, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 2, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 5, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 13, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 16, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 27, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 1, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 1, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 3, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 60, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 39, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 1, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 8, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:22:04.039 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 3, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 8, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 3, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 6, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 164, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 4, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 234, 2024-05-27T08:22:04.040 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 7, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 164, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 41, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 41, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 123, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 6, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 9, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:22:04.041 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 16, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 1, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 4, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 8 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 6, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 1, 2024-05-27T08:22:04.042 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 4, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 3, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-05-27T08:22:04.043 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 3, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 44, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 44, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-05-27T08:22:04.044 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-05-27T08:22:04.045 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 2, 2024-05-27T08:22:04.045 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-05-27T08:22:04.045 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-05-27T08:22:04.045 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4288352, 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 33, 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.184120809, 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.005579418 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.046 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 23, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 38, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 15, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 21, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 32, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 11, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 20, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 50, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 30, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 1, 2024-05-27T08:22:04.047 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 12, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 11, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 121456, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 223632 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 43, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 3, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 26, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 75, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 12, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.048 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 10, 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.007833496, 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000783349 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-05-27T08:22:04.049 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 5, 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.007143473, 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001428694 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.051 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.052 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-05-27T08:22:04.052 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.052 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.052 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.052 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-05-27T08:22:04.052 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.052 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.052 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.053 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-05-27T08:22:04.053 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.053 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.053 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-05-27T08:22:04.054 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4, 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.006647672, 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001661918 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1, 2024-05-27T08:22:04.056 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.018843894, 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.018843894 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-05-27T08:22:04.057 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.058 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.059 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.013845843, 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.004615281 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.060 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 2, 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 5, 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 3, 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 7, 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 3, 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 2895, 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-05-27T08:22:04.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:22:04.062 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:22:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:22:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:22:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:22:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-05-27T08:22:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:22:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-05-27T08:22:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:22:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:22:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:22:04.064 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 324013, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 122, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-05-27T08:22:04.065 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9192, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 176, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 103411, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 522, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.066 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 266, 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 108871, 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 33, 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 266, 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 266, 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.097954856, 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.007887048 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-05-27T08:22:04.067 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 266, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 302 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 266, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 166, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 100, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 0, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 10, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 2, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 35, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 13, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-05-27T08:22:04.068 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 18, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 164, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 20, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-05-27T08:22:04.069 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-05-27T08:22:04.070 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 189, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 100, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 11, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 26, 2024-05-27T08:22:04.071 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 15, 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 32, 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 0, 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 8 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-05-27T08:22:04.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 4, 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 2, 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 6, 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 3, 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-27T08:22:04.073 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 7066, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 3094617, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 7066, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 7066, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 3094617, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:04.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 266, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 109366, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 266, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 109366, 2024-05-27T08:22:04.075 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-27T08:22:04.076 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 266, 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 266, 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 266, 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 266, 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:04.077 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 3, 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 495, 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 3, 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2, 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 495, 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:04.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55ed7120c9c0": { 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 44, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 94048, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 44, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 33, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 94048, 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:04.079 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:04.080 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:04.080 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:04.080 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:04.080 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:04.080 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-05-27T08:22:04.972 INFO:tasks.check_counter:Daemon mds.e mds.exported=3 2024-05-27T08:22:04.972 INFO:tasks.check_counter:Daemon mds.e mds.imported=3 2024-05-27T08:22:04.972 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2024-05-27T08:22:04.972 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:22:04.972 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:22:04.973 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-05-27T08:22:04.973 DEBUG:tasks.check_counter:Getting stats from h 2024-05-27T08:22:04.973 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-27T08:22:05.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:04 smithi086 ceph-mon[34687]: pgmap v1553: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 143 B/s wr, 3 op/s 2024-05-27T08:22:05.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:04 smithi149 ceph-mon[35408]: pgmap v1553: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 143 B/s wr, 3 op/s 2024-05-27T08:22:05.433 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:22:05.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:04 smithi063 ceph-mon[31613]: pgmap v1553: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 143 B/s wr, 3 op/s 2024-05-27T08:22:07.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:06 smithi086 ceph-mon[34687]: pgmap v1554: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-05-27T08:22:07.268 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:06 smithi063 ceph-mon[31613]: pgmap v1554: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-05-27T08:22:07.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:06 smithi149 ceph-mon[35408]: pgmap v1554: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-05-27T08:22:08.168 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-05-27T08:22:08.168 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:22:08.168 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1258, 2024-05-27T08:22:08.168 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1797, 2024-05-27T08:22:08.168 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 2182933, 2024-05-27T08:22:08.168 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 2992129, 2024-05-27T08:22:08.168 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 103, 2024-05-27T08:22:08.168 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 7, 2024-05-27T08:22:08.168 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.597470739, 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.224706036, 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.316474947, 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.044831875, 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1793, 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.266961217, 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000148890 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 88, 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000025294, 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000287 2024-05-27T08:22:08.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2160208, 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 1832544 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 5484, 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 4002, 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1486489, 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1380983, 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 16, 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 1.506057233, 2024-05-27T08:22:08.170 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.806027412, 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.834026736, 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.030016765, 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4002, 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.643670201, 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000160837 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 8994, 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.006419946, 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000713 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.171 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 137072 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 167, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 167, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 44797, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 135458, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 18, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.054484218, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.013634688, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.024926221, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.010942394, 2024-05-27T08:22:08.172 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 166, 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.046904833, 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000282559 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 6, 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000001737, 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000289 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.173 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 228, 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.040285651, 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000176691 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.174 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 24, 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.001942815, 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000080950 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 14, 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 13, 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 13, 2024-05-27T08:22:08.175 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.031714020, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002439540 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 1, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 11, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 17, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 2, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 1, 2024-05-27T08:22:08.176 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 14, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 14, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 25, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 4, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 4, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 3, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 35, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 19, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 6, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-05-27T08:22:08.177 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 3, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 9, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 3, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 6, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 123, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 4, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 234, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 7, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 123, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-05-27T08:22:08.178 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 123, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 5, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 3, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 17, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-05-27T08:22:08.179 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 2, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 7 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 5, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 3, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 3, 2024-05-27T08:22:08.180 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 3, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:22:08.181 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 37, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 37, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 2, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-05-27T08:22:08.182 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4272060, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 0, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 25, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.183057059, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.007322282 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 0 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.183 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 21, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 35, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 14, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 20, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 29, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 9, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 17, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 45, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 28, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 4, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 10, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 6, 2024-05-27T08:22:08.184 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 121700, 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 223632 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 24, 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 3, 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 14, 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 32, 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 12, 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.185 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4, 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.005151311, 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.001287827 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.186 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.002908055, 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000969351 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-05-27T08:22:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.188 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-05-27T08:22:08.189 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.189 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.189 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.189 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-05-27T08:22:08.189 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.189 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.189 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.190 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-05-27T08:22:08.190 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.190 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.190 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.190 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-05-27T08:22:08.190 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.002500450, 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000833483 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.192 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.193 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.194 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-05-27T08:22:08.196 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.021154204, 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.007051401 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-05-27T08:22:08.197 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 1, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 2, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 2, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 2912, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:22:08.198 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-05-27T08:22:08.199 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-05-27T08:22:08.200 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 297785, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 102, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:22:08.201 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9192, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 176, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 90925, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 366, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-05-27T08:22:08.202 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 210, 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 91329, 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 33, 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 210, 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 210, 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.640964711, 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.007814117 2024-05-27T08:22:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 210, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 241 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 210, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 125, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 85, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 0, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 10, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 2, 2024-05-27T08:22:08.204 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 27, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 11, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 18, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 123, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 20, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-05-27T08:22:08.205 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-05-27T08:22:08.206 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 190, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 101, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 10, 2024-05-27T08:22:08.207 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 27, 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 17, 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 25, 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 0, 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 5 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 1, 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-05-27T08:22:08.208 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 4, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 2, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 6, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 3, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:08.209 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 6905, 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 3033147, 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 6905, 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 6905, 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 3033147, 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.210 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 210, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 91824, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 210, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 91824, 2024-05-27T08:22:08.211 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-05-27T08:22:08.212 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 210, 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 210, 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 210, 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 210, 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:08.213 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 3, 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 495, 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 3, 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2, 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 495, 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:08.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x5636633b8a90": { 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 37, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 77756, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 37, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 25, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 77756, 2024-05-27T08:22:08.215 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:08.216 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:08.216 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:08.216 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:08.216 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:08.216 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:08.216 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-05-27T08:22:08.292 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:08 smithi063 ceph-mon[31613]: osdmap e190: 12 total, 12 up, 12 in 2024-05-27T08:22:08.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:08 smithi086 ceph-mon[34687]: osdmap e190: 12 total, 12 up, 12 in 2024-05-27T08:22:08.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:08 smithi149 ceph-mon[35408]: osdmap e190: 12 total, 12 up, 12 in 2024-05-27T08:22:08.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:08 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 5. 2024-05-27T08:22:08.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:08 smithi149 systemd[1]: Stopped Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:08.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:08 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.395s CPU time. 2024-05-27T08:22:08.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:08 smithi149 systemd[1]: Starting Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:08.963 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:08 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 5. 2024-05-27T08:22:08.963 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:08 smithi149 systemd[1]: Stopped Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:08.963 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:08 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 1.388s CPU time. 2024-05-27T08:22:08.963 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:08 smithi149 systemd[1]: Starting Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:09.133 INFO:tasks.check_counter:Daemon mds.h mds.exported=3 2024-05-27T08:22:09.133 INFO:tasks.check_counter:Daemon mds.h mds.imported=3 2024-05-27T08:22:09.133 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-05-27T08:22:09.133 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:22:09.133 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:22:09.133 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-27T08:22:09.133 DEBUG:tasks.check_counter:Getting stats from k 2024-05-27T08:22:09.134 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-27T08:22:09.367 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:09 smithi149 ceph-mon[35408]: pgmap v1556: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-27T08:22:09.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:09 smithi086 ceph-mon[34687]: pgmap v1556: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-27T08:22:09.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:09 smithi063 ceph-mon[31613]: pgmap v1556: 129 pgs: 129 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-27T08:22:09.713 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:09 smithi149 podman[82725]: 2024-05-27 08:22:09.275527913 +0000 UTC m=+0.092890470 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:09.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:09 smithi149 podman[82725]: 2024-05-27 08:22:09.554955171 +0000 UTC m=+0.372317762 container create 31f55df492a4e9d7887f50f2db4dd289b77714f3109e6e07afdf8e957425a01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-27T08:22:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:22:10.367 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:10 smithi149 ceph-mon[35408]: pgmap v1557: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-27T08:22:10.367 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:10 smithi149 podman[82768]: 2024-05-27 08:22:09.946040015 +0000 UTC m=+0.043039186 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:10.373 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:10 smithi086 ceph-mon[34687]: pgmap v1557: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-27T08:22:10.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:10 smithi063 ceph-mon[31613]: pgmap v1557: 129 pgs: 18 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 61 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-27T08:22:10.713 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:10 smithi149 podman[82768]: 2024-05-27 08:22:10.366576319 +0000 UTC m=+0.463575474 container create 0fe284811472b80d12242e6521401b54ffa141bce322c19b499009273ebc4373 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:22:10.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:10 smithi149 podman[82725]: 2024-05-27 08:22:10.581262141 +0000 UTC m=+1.398624699 container init 31f55df492a4e9d7887f50f2db4dd289b77714f3109e6e07afdf8e957425a01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T08:22:10.714 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:10 smithi149 podman[82725]: 2024-05-27 08:22:10.590477278 +0000 UTC m=+1.407839820 container start 31f55df492a4e9d7887f50f2db4dd289b77714f3109e6e07afdf8e957425a01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:22:11.213 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:10 smithi149 bash[82725]: 31f55df492a4e9d7887f50f2db4dd289b77714f3109e6e07afdf8e957425a01e 2024-05-27T08:22:11.213 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:10 smithi149 systemd[1]: Started Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:11.714 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:11 smithi149 podman[82768]: 2024-05-27 08:22:11.441794189 +0000 UTC m=+1.538793350 container init 0fe284811472b80d12242e6521401b54ffa141bce322c19b499009273ebc4373 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:22:11.714 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:11 smithi149 podman[82768]: 2024-05-27 08:22:11.455325292 +0000 UTC m=+1.552324444 container start 0fe284811472b80d12242e6521401b54ffa141bce322c19b499009273ebc4373 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=) 2024-05-27T08:22:12.004 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:11 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f0c96000700 time 2024-05-27T08:22:11.752447+0000 2024-05-27T08:22:12.004 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:11 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:12.091 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8183a00700 time 2024-05-27T08:22:11.822417+0000 2024-05-27T08:22:12.091 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:12.091 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.091 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8199542ba6] 2024-05-27T08:22:12.091 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8199542dd1] 2024-05-27T08:22:12.091 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c97703923d] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c977024971] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c977022fa0] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c977023651] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c976fb8347] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c976cb487e] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c976cb9e94] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c976cc4969] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c97705d48c] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c9770669cb] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 13: (Thread::entry_wrapper()+0x43) [0x7f819951172d] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8199511749] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f81980b51ca] 2024-05-27T08:22:12.092 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 16: clone() 2024-05-27T08:22:12.093 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:11.842+0000 7f8183a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8183a00700 time 2024-05-27T08:22:11.822417+0000 2024-05-27T08:22:12.093 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:12.093 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.093 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.093 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8199542ba6] 2024-05-27T08:22:12.093 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8199542dd1] 2024-05-27T08:22:12.093 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c97703923d] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c977024971] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c977022fa0] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c977023651] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c976fb8347] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c976cb487e] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c976cb9e94] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c976cc4969] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c97705d48c] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c9770669cb] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 13: (Thread::entry_wrapper()+0x43) [0x7f819951172d] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8199511749] 2024-05-27T08:22:12.094 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f81980b51ca] 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 16: clone() 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: *** Caught signal (Aborted) ** 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: in thread 7f8183a00700 thread_name:md_log_replay 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f81980bfd20] 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2: gsignal() 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 3: abort() 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8199542ced] 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8199542dd1] 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c97703923d] 2024-05-27T08:22:12.095 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c977024971] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c977022fa0] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c977023651] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c976fb8347] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c976cb487e] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c976cb9e94] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c976cc4969] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c97705d48c] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c9770669cb] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 16: (Thread::entry_wrapper()+0x43) [0x7f819951172d] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8199511749] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f81980b51ca] 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 19: clone() 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:11.853+0000 7f8183a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:12.096 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: in thread 7f8183a00700 thread_name:md_log_replay 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f81980bfd20] 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2: gsignal() 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 3: abort() 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8199542ced] 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8199542dd1] 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c97703923d] 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c977024971] 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c977022fa0] 2024-05-27T08:22:12.097 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c977023651] 2024-05-27T08:22:12.098 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c976fb8347] 2024-05-27T08:22:12.098 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c976cb487e] 2024-05-27T08:22:12.098 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c976cb9e94] 2024-05-27T08:22:12.098 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c976cc4969] 2024-05-27T08:22:12.098 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c97705d48c] 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c9770669cb] 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 16: (Thread::entry_wrapper()+0x43) [0x7f819951172d] 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8199511749] 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f81980b51ca] 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 19: clone() 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: -1> 2024-05-27T08:22:11.842+0000 7f8183a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8183a00700 time 2024-05-27T08:22:11.822417+0000 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8199542ba6] 2024-05-27T08:22:12.099 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8199542dd1] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c97703923d] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c977024971] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c977022fa0] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c977023651] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c976fb8347] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c976cb487e] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c976cb9e94] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c976cc4969] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c97705d48c] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c9770669cb] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 13: (Thread::entry_wrapper()+0x43) [0x7f819951172d] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8199511749] 2024-05-27T08:22:12.100 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f81980b51ca] 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 16: clone() 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 0> 2024-05-27T08:22:11.853+0000 7f8183a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: in thread 7f8183a00700 thread_name:md_log_replay 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f81980bfd20] 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2: gsignal() 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 3: abort() 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8199542ced] 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8199542dd1] 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c97703923d] 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c977024971] 2024-05-27T08:22:12.101 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c977022fa0] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c977023651] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c976fb8347] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c976cb487e] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c976cb9e94] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c976cc4969] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c97705d48c] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c9770669cb] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 16: (Thread::entry_wrapper()+0x43) [0x7f819951172d] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8199511749] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f81980b51ca] 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 19: clone() 2024-05-27T08:22:12.102 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: -1> 2024-05-27T08:22:11.842+0000 7f8183a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8183a00700 time 2024-05-27T08:22:11.822417+0000 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8199542ba6] 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8199542dd1] 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c97703923d] 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55c977024971] 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 5: (SnapRealm::check_cache() const+0x13e) [0x55c977022fa0] 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c977023651] 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c976fb8347] 2024-05-27T08:22:12.103 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c976cb487e] 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c976cb9e94] 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55c976cc4969] 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 11: (MDLog::_replay_thread()+0x1f38) [0x55c97705d48c] 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55c9770669cb] 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 13: (Thread::entry_wrapper()+0x43) [0x7f819951172d] 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8199511749] 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f81980b51ca] 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 16: clone() 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 0> 2024-05-27T08:22:11.853+0000 7f8183a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: in thread 7f8183a00700 thread_name:md_log_replay 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.104 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f81980bfd20] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2: gsignal() 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 3: abort() 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8199542ced] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8199542dd1] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55c97703923d] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55c977024971] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 8: (SnapRealm::check_cache() const+0x13e) [0x55c977022fa0] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55c977023651] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55c976fb8347] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55c976cb487e] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55c976cb9e94] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55c976cc4969] 2024-05-27T08:22:12.105 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 14: (MDLog::_replay_thread()+0x1f38) [0x55c97705d48c] 2024-05-27T08:22:12.106 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55c9770669cb] 2024-05-27T08:22:12.106 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 16: (Thread::entry_wrapper()+0x43) [0x7f819951172d] 2024-05-27T08:22:12.106 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8199511749] 2024-05-27T08:22:12.106 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f81980b51ca] 2024-05-27T08:22:12.106 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 19: clone() 2024-05-27T08:22:12.106 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:12.106 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[188296]: 2024-05-27T08:22:12.106 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:12 smithi063 podman[189678]: 2024-05-27 08:22:12.040379981 +0000 UTC m=+0.077584200 container died 31cf44daa14568ef7d41ed4c30cf6d98648bd152d9be711d1ad1d28dbee328be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux ) 2024-05-27T08:22:12.258 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:12 smithi149 ceph-mon[35408]: pgmap v1558: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-05-27T08:22:12.258 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:12 smithi149 ceph-mon[35408]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:12.258 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:12 smithi149 ceph-mon[35408]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:12.258 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:12 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:12.259 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:12 smithi149 bash[82768]: 0fe284811472b80d12242e6521401b54ffa141bce322c19b499009273ebc4373 2024-05-27T08:22:12.259 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:12 smithi149 systemd[1]: Started Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:12.259 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.259 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f0caba19ba6] 2024-05-27T08:22:12.259 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0caba19dd1] 2024-05-27T08:22:12.259 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561c6092323d] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561c6090e971] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 5: (SnapRealm::check_cache() const+0x13e) [0x561c6090cfa0] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561c6090d651] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561c608a2347] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561c6059e87e] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561c605a3f2f] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561c605ae969] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 11: (MDLog::_replay_thread()+0x1f38) [0x561c6094748c] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561c609509cb] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 13: (Thread::entry_wrapper()+0x43) [0x7f0cab9e872d] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 14: (Thread::_entry_func(void*)+0xd) [0x7f0cab9e8749] 2024-05-27T08:22:12.260 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0caa58c1ca] 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 16: clone() 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: *** Caught signal (Aborted) ** 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: in thread 7f0c96000700 thread_name:md_log_replay 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.030+0000 7f0c96000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f0c96000700 time 2024-05-27T08:22:11.752447+0000 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f0caba19ba6] 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0caba19dd1] 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561c6092323d] 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561c6090e971] 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 5: (SnapRealm::check_cache() const+0x13e) [0x561c6090cfa0] 2024-05-27T08:22:12.261 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561c6090d651] 2024-05-27T08:22:12.262 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561c608a2347] 2024-05-27T08:22:12.262 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561c6059e87e] 2024-05-27T08:22:12.262 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561c605a3f2f] 2024-05-27T08:22:12.262 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561c605ae969] 2024-05-27T08:22:12.262 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 11: (MDLog::_replay_thread()+0x1f38) [0x561c6094748c] 2024-05-27T08:22:12.262 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561c609509cb] 2024-05-27T08:22:12.262 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 13: (Thread::entry_wrapper()+0x43) [0x7f0cab9e872d] 2024-05-27T08:22:12.262 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 14: (Thread::_entry_func(void*)+0xd) [0x7f0cab9e8749] 2024-05-27T08:22:12.262 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0caa58c1ca] 2024-05-27T08:22:12.262 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 16: clone() 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0caa596d20] 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2: gsignal() 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 3: abort() 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f0caba19ced] 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0caba19dd1] 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561c6092323d] 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561c6090e971] 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 8: (SnapRealm::check_cache() const+0x13e) [0x561c6090cfa0] 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561c6090d651] 2024-05-27T08:22:12.263 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561c608a2347] 2024-05-27T08:22:12.264 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561c6059e87e] 2024-05-27T08:22:12.264 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561c605a3f2f] 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561c605ae969] 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 14: (MDLog::_replay_thread()+0x1f38) [0x561c6094748c] 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561c609509cb] 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 16: (Thread::entry_wrapper()+0x43) [0x7f0cab9e872d] 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 17: (Thread::_entry_func(void*)+0xd) [0x7f0cab9e8749] 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0caa58c1ca] 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 19: clone() 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.034+0000 7f0c96000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: in thread 7f0c96000700 thread_name:md_log_replay 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0caa596d20] 2024-05-27T08:22:12.265 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2: gsignal() 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 3: abort() 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f0caba19ced] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0caba19dd1] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561c6092323d] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561c6090e971] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 8: (SnapRealm::check_cache() const+0x13e) [0x561c6090cfa0] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561c6090d651] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561c608a2347] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561c6059e87e] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561c605a3f2f] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561c605ae969] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 14: (MDLog::_replay_thread()+0x1f38) [0x561c6094748c] 2024-05-27T08:22:12.266 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561c609509cb] 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 16: (Thread::entry_wrapper()+0x43) [0x7f0cab9e872d] 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 17: (Thread::_entry_func(void*)+0xd) [0x7f0cab9e8749] 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0caa58c1ca] 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 19: clone() 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: -1> 2024-05-27T08:22:12.030+0000 7f0c96000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f0c96000700 time 2024-05-27T08:22:11.752447+0000 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f0caba19ba6] 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0caba19dd1] 2024-05-27T08:22:12.267 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561c6092323d] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561c6090e971] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 5: (SnapRealm::check_cache() const+0x13e) [0x561c6090cfa0] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561c6090d651] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561c608a2347] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561c6059e87e] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561c605a3f2f] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561c605ae969] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 11: (MDLog::_replay_thread()+0x1f38) [0x561c6094748c] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561c609509cb] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 13: (Thread::entry_wrapper()+0x43) [0x7f0cab9e872d] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 14: (Thread::_entry_func(void*)+0xd) [0x7f0cab9e8749] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0caa58c1ca] 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 16: clone() 2024-05-27T08:22:12.268 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 0> 2024-05-27T08:22:12.034+0000 7f0c96000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: in thread 7f0c96000700 thread_name:md_log_replay 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0caa596d20] 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2: gsignal() 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 3: abort() 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f0caba19ced] 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0caba19dd1] 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561c6092323d] 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561c6090e971] 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 8: (SnapRealm::check_cache() const+0x13e) [0x561c6090cfa0] 2024-05-27T08:22:12.269 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561c6090d651] 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561c608a2347] 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561c6059e87e] 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561c605a3f2f] 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561c605ae969] 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 14: (MDLog::_replay_thread()+0x1f38) [0x561c6094748c] 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561c609509cb] 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 16: (Thread::entry_wrapper()+0x43) [0x7f0cab9e872d] 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 17: (Thread::_entry_func(void*)+0xd) [0x7f0cab9e8749] 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0caa58c1ca] 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 19: clone() 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.270 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: -1> 2024-05-27T08:22:12.030+0000 7f0c96000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f0c96000700 time 2024-05-27T08:22:11.752447+0000 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f0caba19ba6] 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0caba19dd1] 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561c6092323d] 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561c6090e971] 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 5: (SnapRealm::check_cache() const+0x13e) [0x561c6090cfa0] 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561c6090d651] 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561c608a2347] 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561c6059e87e] 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561c605a3f2f] 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561c605ae969] 2024-05-27T08:22:12.271 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 11: (MDLog::_replay_thread()+0x1f38) [0x561c6094748c] 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561c609509cb] 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 13: (Thread::entry_wrapper()+0x43) [0x7f0cab9e872d] 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 14: (Thread::_entry_func(void*)+0xd) [0x7f0cab9e8749] 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0caa58c1ca] 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 16: clone() 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 0> 2024-05-27T08:22:12.034+0000 7f0c96000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: in thread 7f0c96000700 thread_name:md_log_replay 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0caa596d20] 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2: gsignal() 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 3: abort() 2024-05-27T08:22:12.272 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f0caba19ced] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f0caba19dd1] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561c6092323d] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561c6090e971] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 8: (SnapRealm::check_cache() const+0x13e) [0x561c6090cfa0] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561c6090d651] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561c608a2347] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561c6059e87e] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561c605a3f2f] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561c605ae969] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 14: (MDLog::_replay_thread()+0x1f38) [0x561c6094748c] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561c609509cb] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 16: (Thread::entry_wrapper()+0x43) [0x7f0cab9e872d] 2024-05-27T08:22:12.273 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 17: (Thread::_entry_func(void*)+0xd) [0x7f0cab9e8749] 2024-05-27T08:22:12.274 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f0caa58c1ca] 2024-05-27T08:22:12.274 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 19: clone() 2024-05-27T08:22:12.274 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:12.274 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[81285]: 2024-05-27T08:22:12.274 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:12 smithi149 podman[82800]: 2024-05-27 08:22:12.252581622 +0000 UTC m=+0.116015217 container died 0760db8717cda5a4ea09a3f6aa9ba09320d8c7e4613b7f417fec5389a90f63f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5) 2024-05-27T08:22:12.463 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:12 smithi086 ceph-mon[34687]: pgmap v1558: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-05-27T08:22:12.463 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:12 smithi086 ceph-mon[34687]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:12.463 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:12 smithi086 ceph-mon[34687]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:12.463 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:12 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:12 smithi063 ceph-mon[31613]: pgmap v1558: 129 pgs: 20 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 51 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-05-27T08:22:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:12 smithi063 ceph-mon[31613]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:12 smithi063 ceph-mon[31613]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:12.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:12 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:12.580 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:12 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 5. 2024-05-27T08:22:12.580 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:12 smithi149 systemd[1]: Stopped Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:12.580 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:12 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 1.393s CPU time. 2024-05-27T08:22:12.580 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:12 smithi149 systemd[1]: Starting Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:13.177 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: starting mds.f at 2024-05-27T08:22:13.177 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: starting mds.c at 2024-05-27T08:22:13.944 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:13 smithi149 podman[82800]: 2024-05-27 08:22:13.593712772 +0000 UTC m=+1.457146365 container remove 0760db8717cda5a4ea09a3f6aa9ba09320d8c7e4613b7f417fec5389a90f63f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521) 2024-05-27T08:22:13.944 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:13 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:22:14.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:14 smithi086 ceph-mon[34687]: pgmap v1559: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T08:22:14.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:14 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6832/878743058,v1:172.21.15.149:6834/878743058] up:boot 2024-05-27T08:22:14.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:14 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6833/3399082489,v1:172.21.15.149:6835/3399082489] up:boot 2024-05-27T08:22:14.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:14 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:14.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:22:14.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:14 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:22:14.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:14 smithi149 ceph-mon[35408]: pgmap v1559: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T08:22:14.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:14 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6832/878743058,v1:172.21.15.149:6834/878743058] up:boot 2024-05-27T08:22:14.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:14 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6833/3399082489,v1:172.21.15.149:6835/3399082489] up:boot 2024-05-27T08:22:14.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:14 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:14.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:22:14.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:14 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:22:14.214 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:13 smithi149 podman[82907]: 2024-05-27 08:22:13.847463596 +0000 UTC m=+0.067185808 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:14.214 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:13 smithi149 podman[82907]: 2024-05-27 08:22:13.978193532 +0000 UTC m=+0.197915783 container create 326d11be7b69e8fa7cd148eaed51ee2e4d5dad6694881a14e7fe263d09f06f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-27T08:22:14.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:14 smithi063 ceph-mon[31613]: pgmap v1559: 129 pgs: 16 active+clean+snaptrim, 47 active+clean+snaptrim_wait, 66 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-27T08:22:14.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:14 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6832/878743058,v1:172.21.15.149:6834/878743058] up:boot 2024-05-27T08:22:14.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:14 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6833/3399082489,v1:172.21.15.149:6835/3399082489] up:boot 2024-05-27T08:22:14.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:14 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:14.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:22:14.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:14 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:22:14.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:14 smithi149 podman[82907]: 2024-05-27 08:22:14.30728377 +0000 UTC m=+0.527006091 container init 326d11be7b69e8fa7cd148eaed51ee2e4d5dad6694881a14e7fe263d09f06f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-27T08:22:14.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:14 smithi149 podman[82907]: 2024-05-27 08:22:14.31890141 +0000 UTC m=+0.538623691 container start 326d11be7b69e8fa7cd148eaed51ee2e4d5dad6694881a14e7fe263d09f06f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:22:14.714 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:14 smithi149 ceph-mds[82941]: -- 172.21.15.149:0/1906872060 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556bcfdd1a40 con 0x556bcfe19180 2024-05-27T08:22:15.128 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:14 smithi063 podman[189678]: 2024-05-27 08:22:14.735143531 +0000 UTC m=+2.772347804 container remove 31cf44daa14568ef7d41ed4c30cf6d98648bd152d9be711d1ad1d28dbee328be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:22:15.128 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:14 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:22:15.196 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:15 smithi149 ceph-mon[35408]: daemon mds.i restarted 2024-05-27T08:22:15.196 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:14 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: starting mds.i at 2024-05-27T08:22:15.459 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:15 smithi086 ceph-mon[34687]: daemon mds.i restarted 2024-05-27T08:22:15.464 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f5a2aa00700 time 2024-05-27T08:22:15.195853+0000 2024-05-27T08:22:15.464 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:15.466 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:15 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Failed with result 'exit-code'. 2024-05-27T08:22:15.466 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:15 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Consumed 1.468s CPU time. 2024-05-27T08:22:15.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:15 smithi063 ceph-mon[31613]: daemon mds.i restarted 2024-05-27T08:22:15.690 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-05-27T08:22:15.690 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:22:15.690 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 3, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 4, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 2624, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 448, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.008168114, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000182568, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.003734906, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-05-27T08:22:15.691 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.002596887, 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000865629 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2624, 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 24, 2024-05-27T08:22:15.692 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 23, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 158832, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 12496, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.017006038, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.001881088, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.023400883, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000864019, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 22, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.006236607, 2024-05-27T08:22:15.693 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000283482 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 158832, 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 12496 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1, 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 17, 2024-05-27T08:22:15.694 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 160, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 48800, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 3, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.006152007, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.001796039, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.003755554, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 15, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.001373493, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000091566 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:15.695 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 48800 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:22:15.696 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-05-27T08:22:15.697 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:22:15.697 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:22:15.697 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:22:15.697 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:22:15.697 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:22:15.697 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:22:15.697 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:22:15.697 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:22:15.697 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:22:15.697 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:22:15.698 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 150101, 2024-05-27T08:22:15.699 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 27, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:22:15.700 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 25, 2024-05-27T08:22:15.701 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 155494, 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 25, 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 25, 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 155494, 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:15.702 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-05-27T08:22:15.703 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-05-27T08:22:15.744 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:15 smithi149 podman[82922]: 2024-05-27 08:22:15.570577609 +0000 UTC m=+1.695770178 container died 326d11be7b69e8fa7cd148eaed51ee2e4d5dad6694881a14e7fe263d09f06f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-27T08:22:15.745 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 bash[82907]: 326d11be7b69e8fa7cd148eaed51ee2e4d5dad6694881a14e7fe263d09f06f49 2024-05-27T08:22:15.745 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:15.745 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5a404d9ba6] 2024-05-27T08:22:15.745 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a404d9dd1] 2024-05-27T08:22:15.745 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556bac98323d] 2024-05-27T08:22:15.745 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556bac96e971] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 5: (SnapRealm::check_cache() const+0x13e) [0x556bac96cfa0] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556bac96d651] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556bac902347] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556bac5fe87e] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556bac603e94] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556bac60e969] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 11: (MDLog::_replay_thread()+0x1f38) [0x556bac9a748c] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556bac9b09cb] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 13: (Thread::entry_wrapper()+0x43) [0x7f5a404a872d] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 14: (Thread::_entry_func(void*)+0xd) [0x7f5a404a8749] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5a3f04c1ca] 2024-05-27T08:22:15.746 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 16: clone() 2024-05-27T08:22:15.747 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.467+0000 7f5a2aa00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f5a2aa00700 time 2024-05-27T08:22:15.195853+0000 2024-05-27T08:22:15.747 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:15.747 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.747 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:15.747 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5a404d9ba6] 2024-05-27T08:22:15.748 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a404d9dd1] 2024-05-27T08:22:15.748 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556bac98323d] 2024-05-27T08:22:15.748 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556bac96e971] 2024-05-27T08:22:15.748 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 5: (SnapRealm::check_cache() const+0x13e) [0x556bac96cfa0] 2024-05-27T08:22:15.748 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556bac96d651] 2024-05-27T08:22:15.748 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556bac902347] 2024-05-27T08:22:15.748 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556bac5fe87e] 2024-05-27T08:22:15.748 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556bac603e94] 2024-05-27T08:22:15.748 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556bac60e969] 2024-05-27T08:22:15.748 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 11: (MDLog::_replay_thread()+0x1f38) [0x556bac9a748c] 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556bac9b09cb] 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 13: (Thread::entry_wrapper()+0x43) [0x7f5a404a872d] 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 14: (Thread::_entry_func(void*)+0xd) [0x7f5a404a8749] 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5a3f04c1ca] 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 16: clone() 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: *** Caught signal (Aborted) ** 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: in thread 7f5a2aa00700 thread_name:md_log_replay 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 systemd[1]: Started Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5a3f056d20] 2024-05-27T08:22:15.749 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2: gsignal() 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 3: abort() 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f5a404d9ced] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a404d9dd1] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556bac98323d] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556bac96e971] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 8: (SnapRealm::check_cache() const+0x13e) [0x556bac96cfa0] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556bac96d651] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556bac902347] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556bac5fe87e] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556bac603e94] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556bac60e969] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 14: (MDLog::_replay_thread()+0x1f38) [0x556bac9a748c] 2024-05-27T08:22:15.750 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556bac9b09cb] 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 16: (Thread::entry_wrapper()+0x43) [0x7f5a404a872d] 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 17: (Thread::_entry_func(void*)+0xd) [0x7f5a404a8749] 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5a3f04c1ca] 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 19: clone() 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.475+0000 7f5a2aa00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: in thread 7f5a2aa00700 thread_name:md_log_replay 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5a3f056d20] 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2: gsignal() 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 3: abort() 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f5a404d9ced] 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a404d9dd1] 2024-05-27T08:22:15.751 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556bac98323d] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556bac96e971] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 8: (SnapRealm::check_cache() const+0x13e) [0x556bac96cfa0] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556bac96d651] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556bac902347] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556bac5fe87e] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556bac603e94] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556bac60e969] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 14: (MDLog::_replay_thread()+0x1f38) [0x556bac9a748c] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556bac9b09cb] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 16: (Thread::entry_wrapper()+0x43) [0x7f5a404a872d] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 17: (Thread::_entry_func(void*)+0xd) [0x7f5a404a8749] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5a3f04c1ca] 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 19: clone() 2024-05-27T08:22:15.752 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: -1> 2024-05-27T08:22:15.467+0000 7f5a2aa00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f5a2aa00700 time 2024-05-27T08:22:15.195853+0000 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5a404d9ba6] 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a404d9dd1] 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556bac98323d] 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556bac96e971] 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 5: (SnapRealm::check_cache() const+0x13e) [0x556bac96cfa0] 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556bac96d651] 2024-05-27T08:22:15.753 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556bac902347] 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556bac5fe87e] 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556bac603e94] 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556bac60e969] 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 11: (MDLog::_replay_thread()+0x1f38) [0x556bac9a748c] 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556bac9b09cb] 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 13: (Thread::entry_wrapper()+0x43) [0x7f5a404a872d] 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 14: (Thread::_entry_func(void*)+0xd) [0x7f5a404a8749] 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5a3f04c1ca] 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 16: clone() 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 0> 2024-05-27T08:22:15.475+0000 7f5a2aa00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: in thread 7f5a2aa00700 thread_name:md_log_replay 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.754 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5a3f056d20] 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2: gsignal() 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 3: abort() 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f5a404d9ced] 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a404d9dd1] 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556bac98323d] 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556bac96e971] 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 8: (SnapRealm::check_cache() const+0x13e) [0x556bac96cfa0] 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556bac96d651] 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556bac902347] 2024-05-27T08:22:15.755 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556bac5fe87e] 2024-05-27T08:22:15.756 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556bac603e94] 2024-05-27T08:22:15.756 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556bac60e969] 2024-05-27T08:22:15.756 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 14: (MDLog::_replay_thread()+0x1f38) [0x556bac9a748c] 2024-05-27T08:22:15.756 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556bac9b09cb] 2024-05-27T08:22:15.756 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 16: (Thread::entry_wrapper()+0x43) [0x7f5a404a872d] 2024-05-27T08:22:15.756 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 17: (Thread::_entry_func(void*)+0xd) [0x7f5a404a8749] 2024-05-27T08:22:15.756 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5a3f04c1ca] 2024-05-27T08:22:15.756 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 19: clone() 2024-05-27T08:22:15.756 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:15.756 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.757 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: -1> 2024-05-27T08:22:15.467+0000 7f5a2aa00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f5a2aa00700 time 2024-05-27T08:22:15.195853+0000 2024-05-27T08:22:15.757 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:15.757 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.757 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:15.757 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5a404d9ba6] 2024-05-27T08:22:15.757 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a404d9dd1] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556bac98323d] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556bac96e971] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 5: (SnapRealm::check_cache() const+0x13e) [0x556bac96cfa0] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556bac96d651] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556bac902347] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556bac5fe87e] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556bac603e94] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x556bac60e969] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 11: (MDLog::_replay_thread()+0x1f38) [0x556bac9a748c] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556bac9b09cb] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 13: (Thread::entry_wrapper()+0x43) [0x7f5a404a872d] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 14: (Thread::_entry_func(void*)+0xd) [0x7f5a404a8749] 2024-05-27T08:22:15.758 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f5a3f04c1ca] 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 16: clone() 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 0> 2024-05-27T08:22:15.475+0000 7f5a2aa00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: in thread 7f5a2aa00700 thread_name:md_log_replay 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5a3f056d20] 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2: gsignal() 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 3: abort() 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f5a404d9ced] 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f5a404d9dd1] 2024-05-27T08:22:15.759 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556bac98323d] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556bac96e971] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 8: (SnapRealm::check_cache() const+0x13e) [0x556bac96cfa0] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556bac96d651] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556bac902347] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556bac5fe87e] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x556bac603e94] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x556bac60e969] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 14: (MDLog::_replay_thread()+0x1f38) [0x556bac9a748c] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556bac9b09cb] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 16: (Thread::entry_wrapper()+0x43) [0x7f5a404a872d] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 17: (Thread::_entry_func(void*)+0xd) [0x7f5a404a8749] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f5a3f04c1ca] 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 19: clone() 2024-05-27T08:22:15.760 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:15.761 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i[82935]: 2024-05-27T08:22:16.131 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:15 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-05-27T08:22:16.131 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:15 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Consumed 1.445s CPU time. 2024-05-27T08:22:16.452 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:16 smithi086 ceph-mon[34687]: pgmap v1560: 129 pgs: 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 108 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 204 B/s wr, 1 op/s 2024-05-27T08:22:16.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:16 smithi086 ceph-mon[34687]: osdmap e191: 12 total, 12 up, 12 in 2024-05-27T08:22:16.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:16 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6836/1101967554,v1:172.21.15.149:6837/1101967554] up:boot 2024-05-27T08:22:16.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:16 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:22:16.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:16 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:22:16.453 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:16 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:16.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:16 smithi149 ceph-mon[35408]: pgmap v1560: 129 pgs: 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 108 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 204 B/s wr, 1 op/s 2024-05-27T08:22:16.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:16 smithi149 ceph-mon[35408]: osdmap e191: 12 total, 12 up, 12 in 2024-05-27T08:22:16.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:16 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6836/1101967554,v1:172.21.15.149:6837/1101967554] up:boot 2024-05-27T08:22:16.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:16 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:22:16.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:16 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:22:16.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:16 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:16.464 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:16 smithi149 podman[82972]: 2024-05-27 08:22:16.270042817 +0000 UTC m=+0.712044422 container remove 326d11be7b69e8fa7cd148eaed51ee2e4d5dad6694881a14e7fe263d09f06f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:22:16.464 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:16 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:22:16.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:16 smithi063 ceph-mon[31613]: pgmap v1560: 129 pgs: 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 108 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 204 B/s wr, 1 op/s 2024-05-27T08:22:16.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:16 smithi063 ceph-mon[31613]: osdmap e191: 12 total, 12 up, 12 in 2024-05-27T08:22:16.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:16 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6836/1101967554,v1:172.21.15.149:6837/1101967554] up:boot 2024-05-27T08:22:16.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:16 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-27T08:22:16.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:16 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:22:16.467 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:16 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:16.510 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-27T08:22:16.510 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-27T08:22:16.510 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-27T08:22:16.510 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-27T08:22:16.510 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-27T08:22:16.510 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-05-27T08:22:16.510 DEBUG:tasks.check_counter:Getting stats from c 2024-05-27T08:22:16.511 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-27T08:22:17.214 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:16 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-05-27T08:22:17.214 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:16 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 1.544s CPU time. 2024-05-27T08:22:17.463 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:22:18.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:18 smithi149 ceph-mon[35408]: pgmap v1562: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 204 B/s wr, 1 op/s 2024-05-27T08:22:18.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:18 smithi086 ceph-mon[34687]: pgmap v1562: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 204 B/s wr, 1 op/s 2024-05-27T08:22:18.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:18 smithi063 ceph-mon[31613]: pgmap v1562: 129 pgs: 5 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 118 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 204 B/s wr, 1 op/s 2024-05-27T08:22:20.136 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-27T08:22:20.136 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:22:20.136 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 0, 2024-05-27T08:22:20.136 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2, 2024-05-27T08:22:20.136 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 0, 2024-05-27T08:22:20.136 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 0, 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 0, 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.010628801, 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.001810834, 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:20.137 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 9, 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 8, 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 154752, 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 2896, 2024-05-27T08:22:20.138 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 2, 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 1, 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.006932024, 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.000591534, 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.009344309, 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000078503, 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7, 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.007658749, 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001094107 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-05-27T08:22:20.139 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 154752, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 2896 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 5, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 7, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 3120, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 47136, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 3, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 2, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.006025065, 2024-05-27T08:22:20.140 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.000420315, 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.005515749, 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6, 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.003437382, 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000572897 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.141 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 47136 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 0, 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:22:20.142 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:22:20.143 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-05-27T08:22:20.144 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 149070, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 26, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:22:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 0, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 0, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 928, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 2, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-05-27T08:22:20.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 10, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 153544, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 10, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 10, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 153544, 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-05-27T08:22:20.147 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-05-27T08:22:20.148 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-05-27T08:22:20.148 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-05-27T08:22:20.148 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-27T08:22:20.148 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-27T08:22:20.148 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-27T08:22:20.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:20 smithi086 ceph-mon[34687]: pgmap v1563: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-27T08:22:20.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:20 smithi149 ceph-mon[35408]: pgmap v1563: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-27T08:22:20.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:20 smithi063 ceph-mon[31613]: pgmap v1563: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-27T08:22:21.133 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.c 2024-05-27T08:22:21.134 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.c 2024-05-27T08:22:21.134 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-05-27T08:22:21.134 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-05-27T08:22:21.134 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-05-27T08:22:21.134 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.c 2024-05-27T08:22:21.134 DEBUG:tasks.check_counter:Getting stats from f 2024-05-27T08:22:21.135 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-27T08:22:21.600 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:22:22.905 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:22 smithi149 ceph-mon[35408]: pgmap v1564: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-27T08:22:22.906 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:22 smithi149 ceph-mon[35408]: osdmap e192: 12 total, 12 up, 12 in 2024-05-27T08:22:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:22 smithi086 ceph-mon[34687]: pgmap v1564: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-27T08:22:22.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:22 smithi086 ceph-mon[34687]: osdmap e192: 12 total, 12 up, 12 in 2024-05-27T08:22:22.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:22 smithi063 ceph-mon[31613]: pgmap v1564: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-27T08:22:22.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:22 smithi063 ceph-mon[31613]: osdmap e192: 12 total, 12 up, 12 in 2024-05-27T08:22:23.952 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-27T08:22:23.952 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:22:23.952 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 0, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 0, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 0, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 0, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.013117142, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.002307282, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-05-27T08:22:23.953 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 5, 2024-05-27T08:22:23.954 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 8, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 3120, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 47312, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 3, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 2, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.005134807, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.000621939, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.004481055, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.005619216, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000802745 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 47312 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 10, 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 9, 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 155024, 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 3536, 2024-05-27T08:22:23.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 2, 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 1, 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.006386031, 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.000413804, 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.008746331, 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000197447, 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 8, 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.004691339, 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000586417 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-05-27T08:22:23.957 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 155024, 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 3536 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 0, 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-05-27T08:22:23.958 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:22:23.959 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:22:23.960 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 147975, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 23, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-05-27T08:22:23.961 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 928, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 2, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-05-27T08:22:23.962 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 11, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 153674, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 11, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-05-27T08:22:23.963 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 11, 2024-05-27T08:22:23.964 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 153674, 2024-05-27T08:22:23.964 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-05-27T08:22:23.964 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-05-27T08:22:23.964 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-05-27T08:22:23.964 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-05-27T08:22:23.964 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-27T08:22:23.964 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-27T08:22:23.964 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-27T08:22:24.848 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:24 smithi149 ceph-mon[35408]: pgmap v1566: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 op/s 2024-05-27T08:22:24.921 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-05-27T08:22:24.921 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-05-27T08:22:24.921 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-05-27T08:22:24.921 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-05-27T08:22:24.921 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-05-27T08:22:24.921 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-05-27T08:22:24.922 DEBUG:tasks.check_counter:Getting stats from i 2024-05-27T08:22:24.922 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-27T08:22:24.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:24 smithi086 ceph-mon[34687]: pgmap v1566: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 op/s 2024-05-27T08:22:24.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:24 smithi063 ceph-mon[31613]: pgmap v1566: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 op/s 2024-05-27T08:22:25.372 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:22:25.875 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:25 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Scheduled restart job, restart counter is at 2. 2024-05-27T08:22:25.875 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:25 smithi063 systemd[1]: Stopped Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:25.875 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:25 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Consumed 1.468s CPU time. 2024-05-27T08:22:25.875 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:25 smithi063 systemd[1]: Starting Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:26.191 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:25 smithi063 podman[189773]: 2024-05-27 08:22:25.778214631 +0000 UTC m=+0.033830487 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:26.192 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:25 smithi063 podman[189773]: 2024-05-27 08:22:25.894785575 +0000 UTC m=+0.150401455 container create fa90968cf27c92dbd58c3307c93dda80a2004deb070dbe06576e7b564056c17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-27T08:22:26.214 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:25 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 2. 2024-05-27T08:22:26.214 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:25 smithi149 systemd[1]: Stopped Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:26.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:25 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Consumed 1.445s CPU time. 2024-05-27T08:22:26.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:25 smithi149 systemd[1]: Starting Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:26.467 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:26 smithi063 podman[189773]: 2024-05-27 08:22:26.190517372 +0000 UTC m=+0.446133299 container init fa90968cf27c92dbd58c3307c93dda80a2004deb070dbe06576e7b564056c17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5) 2024-05-27T08:22:26.467 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:26 smithi063 podman[189773]: 2024-05-27 08:22:26.202295073 +0000 UTC m=+0.457910923 container start fa90968cf27c92dbd58c3307c93dda80a2004deb070dbe06576e7b564056c17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-27T08:22:26.467 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:26 smithi063 bash[189773]: fa90968cf27c92dbd58c3307c93dda80a2004deb070dbe06576e7b564056c17b 2024-05-27T08:22:26.467 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:26 smithi063 systemd[1]: Started Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:26.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:26 smithi086 ceph-mon[34687]: pgmap v1567: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 op/s 2024-05-27T08:22:26.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:26 smithi086 ceph-mon[34687]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:26.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:26 smithi086 ceph-mon[34687]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:26.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:26 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:26.907 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:26 smithi149 podman[83455]: 2024-05-27 08:22:26.78714746 +0000 UTC m=+0.037367932 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:26.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-mon[35408]: pgmap v1567: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 op/s 2024-05-27T08:22:26.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-mon[35408]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:26.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-mon[35408]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:26.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:26.908 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4400a00700 time 2024-05-27T08:22:26.664820+0000 2024-05-27T08:22:26.909 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:26.909 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.909 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4416414ba6] 2024-05-27T08:22:26.909 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4416414dd1] 2024-05-27T08:22:26.909 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5640f566e23d] 2024-05-27T08:22:26.909 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5640f5659971] 2024-05-27T08:22:26.909 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 5: (SnapRealm::check_cache() const+0x13e) [0x5640f5657fa0] 2024-05-27T08:22:26.909 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5640f5658651] 2024-05-27T08:22:26.909 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5640f55ed347] 2024-05-27T08:22:26.910 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5640f52e987e] 2024-05-27T08:22:26.910 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5640f52eef2f] 2024-05-27T08:22:26.910 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5640f52f9969] 2024-05-27T08:22:26.910 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 11: (MDLog::_replay_thread()+0x1f38) [0x5640f569248c] 2024-05-27T08:22:26.910 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5640f569b9cb] 2024-05-27T08:22:26.910 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 13: (Thread::entry_wrapper()+0x43) [0x7f44163e372d] 2024-05-27T08:22:26.910 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 14: (Thread::_entry_func(void*)+0xd) [0x7f44163e3749] 2024-05-27T08:22:26.910 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4414f871ca] 2024-05-27T08:22:26.910 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 16: clone() 2024-05-27T08:22:26.910 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.702+0000 7f4400a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4400a00700 time 2024-05-27T08:22:26.664820+0000 2024-05-27T08:22:26.911 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f1085600700 time 2024-05-27T08:22:26.698749+0000 2024-05-27T08:22:26.911 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:26.911 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.911 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f109b05cba6] 2024-05-27T08:22:26.912 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f109b05cdd1] 2024-05-27T08:22:26.912 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5649cb8ff23d] 2024-05-27T08:22:26.912 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5649cb8ea971] 2024-05-27T08:22:26.912 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 5: (SnapRealm::check_cache() const+0x13e) [0x5649cb8e8fa0] 2024-05-27T08:22:26.912 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5649cb8e9651] 2024-05-27T08:22:26.912 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5649cb87e347] 2024-05-27T08:22:26.912 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5649cb57a87e] 2024-05-27T08:22:26.912 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5649cb57fe94] 2024-05-27T08:22:26.912 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5649cb58a969] 2024-05-27T08:22:26.913 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 11: (MDLog::_replay_thread()+0x1f38) [0x5649cb92348c] 2024-05-27T08:22:26.913 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5649cb92c9cb] 2024-05-27T08:22:26.913 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 13: (Thread::entry_wrapper()+0x43) [0x7f109b02b72d] 2024-05-27T08:22:26.913 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 14: (Thread::_entry_func(void*)+0xd) [0x7f109b02b749] 2024-05-27T08:22:26.913 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f1099bcf1ca] 2024-05-27T08:22:26.913 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 16: clone() 2024-05-27T08:22:26.913 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: *** Caught signal (Aborted) ** 2024-05-27T08:22:26.914 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: in thread 7f1085600700 thread_name:md_log_replay 2024-05-27T08:22:26.914 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:26.914 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4416414ba6] 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4416414dd1] 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5640f566e23d] 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5640f5659971] 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 5: (SnapRealm::check_cache() const+0x13e) [0x5640f5657fa0] 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5640f5658651] 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5640f55ed347] 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5640f52e987e] 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5640f52eef2f] 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5640f52f9969] 2024-05-27T08:22:26.915 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 11: (MDLog::_replay_thread()+0x1f38) [0x5640f569248c] 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5640f569b9cb] 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 13: (Thread::entry_wrapper()+0x43) [0x7f44163e372d] 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 14: (Thread::_entry_func(void*)+0xd) [0x7f44163e3749] 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4414f871ca] 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 16: clone() 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: *** Caught signal (Aborted) ** 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: in thread 7f4400a00700 thread_name:md_log_replay 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4414f91d20] 2024-05-27T08:22:26.916 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2: gsignal() 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 3: abort() 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4416414ced] 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4416414dd1] 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5640f566e23d] 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5640f5659971] 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 8: (SnapRealm::check_cache() const+0x13e) [0x5640f5657fa0] 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5640f5658651] 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5640f55ed347] 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5640f52e987e] 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5640f52eef2f] 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5640f52f9969] 2024-05-27T08:22:26.917 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 14: (MDLog::_replay_thread()+0x1f38) [0x5640f569248c] 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5640f569b9cb] 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 16: (Thread::entry_wrapper()+0x43) [0x7f44163e372d] 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 17: (Thread::_entry_func(void*)+0xd) [0x7f44163e3749] 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4414f871ca] 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 19: clone() 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.705+0000 7f4400a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: in thread 7f4400a00700 thread_name:md_log_replay 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4414f91d20] 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2: gsignal() 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 3: abort() 2024-05-27T08:22:26.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4416414ced] 2024-05-27T08:22:26.919 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4416414dd1] 2024-05-27T08:22:26.919 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5640f566e23d] 2024-05-27T08:22:26.919 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5640f5659971] 2024-05-27T08:22:26.919 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 8: (SnapRealm::check_cache() const+0x13e) [0x5640f5657fa0] 2024-05-27T08:22:26.919 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5640f5658651] 2024-05-27T08:22:26.919 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5640f55ed347] 2024-05-27T08:22:26.919 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.701+0000 7f1085600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f1085600700 time 2024-05-27T08:22:26.698749+0000 2024-05-27T08:22:26.919 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:26.919 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f109b05cba6] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f109b05cdd1] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5649cb8ff23d] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5649cb8ea971] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 5: (SnapRealm::check_cache() const+0x13e) [0x5649cb8e8fa0] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5649cb8e9651] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5649cb87e347] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5649cb57a87e] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5649cb57fe94] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5649cb58a969] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 11: (MDLog::_replay_thread()+0x1f38) [0x5649cb92348c] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5649cb92c9cb] 2024-05-27T08:22:26.920 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 13: (Thread::entry_wrapper()+0x43) [0x7f109b02b72d] 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 14: (Thread::_entry_func(void*)+0xd) [0x7f109b02b749] 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f1099bcf1ca] 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 16: clone() 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1099bd9d20] 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2: gsignal() 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 3: abort() 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f109b05cced] 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f109b05cdd1] 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5649cb8ff23d] 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5649cb8ea971] 2024-05-27T08:22:26.921 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 8: (SnapRealm::check_cache() const+0x13e) [0x5649cb8e8fa0] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5649cb8e9651] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5649cb87e347] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5649cb57a87e] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5649cb57fe94] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5649cb58a969] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 14: (MDLog::_replay_thread()+0x1f38) [0x5649cb92348c] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5649cb92c9cb] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 16: (Thread::entry_wrapper()+0x43) [0x7f109b02b72d] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 17: (Thread::_entry_func(void*)+0xd) [0x7f109b02b749] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f1099bcf1ca] 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 19: clone() 2024-05-27T08:22:26.922 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.704+0000 7f1085600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: in thread 7f1085600700 thread_name:md_log_replay 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1099bd9d20] 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2: gsignal() 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 3: abort() 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f109b05cced] 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f109b05cdd1] 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5649cb8ff23d] 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5649cb8ea971] 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 8: (SnapRealm::check_cache() const+0x13e) [0x5649cb8e8fa0] 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5649cb8e9651] 2024-05-27T08:22:26.923 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5649cb87e347] 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5640f52e987e] 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5640f52eef2f] 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5640f52f9969] 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 14: (MDLog::_replay_thread()+0x1f38) [0x5640f569248c] 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5640f569b9cb] 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 16: (Thread::entry_wrapper()+0x43) [0x7f44163e372d] 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 17: (Thread::_entry_func(void*)+0xd) [0x7f44163e3749] 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4414f871ca] 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 19: clone() 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:26.924 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: -1> 2024-05-27T08:22:26.702+0000 7f4400a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4400a00700 time 2024-05-27T08:22:26.664820+0000 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4416414ba6] 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4416414dd1] 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5640f566e23d] 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5640f5659971] 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 5: (SnapRealm::check_cache() const+0x13e) [0x5640f5657fa0] 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5640f5658651] 2024-05-27T08:22:26.925 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5640f55ed347] 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5640f52e987e] 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5649cb57a87e] 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5649cb57fe94] 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5649cb58a969] 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 14: (MDLog::_replay_thread()+0x1f38) [0x5649cb92348c] 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5649cb92c9cb] 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 16: (Thread::entry_wrapper()+0x43) [0x7f109b02b72d] 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 17: (Thread::_entry_func(void*)+0xd) [0x7f109b02b749] 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f1099bcf1ca] 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 19: clone() 2024-05-27T08:22:26.926 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: -1> 2024-05-27T08:22:26.701+0000 7f1085600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f1085600700 time 2024-05-27T08:22:26.698749+0000 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f109b05cba6] 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f109b05cdd1] 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5649cb8ff23d] 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5649cb8ea971] 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 5: (SnapRealm::check_cache() const+0x13e) [0x5649cb8e8fa0] 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5649cb8e9651] 2024-05-27T08:22:26.927 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5649cb87e347] 2024-05-27T08:22:26.928 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5649cb57a87e] 2024-05-27T08:22:26.928 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5640f52eef2f] 2024-05-27T08:22:26.928 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5640f52f9969] 2024-05-27T08:22:26.928 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 11: (MDLog::_replay_thread()+0x1f38) [0x5640f569248c] 2024-05-27T08:22:26.928 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5640f569b9cb] 2024-05-27T08:22:26.928 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 13: (Thread::entry_wrapper()+0x43) [0x7f44163e372d] 2024-05-27T08:22:26.928 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 14: (Thread::_entry_func(void*)+0xd) [0x7f44163e3749] 2024-05-27T08:22:26.928 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4414f871ca] 2024-05-27T08:22:26.928 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 16: clone() 2024-05-27T08:22:26.928 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 0> 2024-05-27T08:22:26.705+0000 7f4400a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: in thread 7f4400a00700 thread_name:md_log_replay 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4414f91d20] 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2: gsignal() 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 3: abort() 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4416414ced] 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4416414dd1] 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5640f566e23d] 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5640f5659971] 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 8: (SnapRealm::check_cache() const+0x13e) [0x5640f5657fa0] 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5640f5658651] 2024-05-27T08:22:26.929 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5640f55ed347] 2024-05-27T08:22:26.930 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5640f52e987e] 2024-05-27T08:22:26.930 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5640f52eef2f] 2024-05-27T08:22:26.930 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5640f52f9969] 2024-05-27T08:22:26.930 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 14: (MDLog::_replay_thread()+0x1f38) [0x5640f569248c] 2024-05-27T08:22:26.930 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5640f569b9cb] 2024-05-27T08:22:26.930 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 16: (Thread::entry_wrapper()+0x43) [0x7f44163e372d] 2024-05-27T08:22:26.930 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5649cb57fe94] 2024-05-27T08:22:26.930 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5649cb58a969] 2024-05-27T08:22:26.930 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 11: (MDLog::_replay_thread()+0x1f38) [0x5649cb92348c] 2024-05-27T08:22:26.930 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5649cb92c9cb] 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 13: (Thread::entry_wrapper()+0x43) [0x7f109b02b72d] 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 14: (Thread::_entry_func(void*)+0xd) [0x7f109b02b749] 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f1099bcf1ca] 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 16: clone() 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 0> 2024-05-27T08:22:26.704+0000 7f1085600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: in thread 7f1085600700 thread_name:md_log_replay 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1099bd9d20] 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2: gsignal() 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 3: abort() 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f109b05cced] 2024-05-27T08:22:26.931 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f109b05cdd1] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5649cb8ff23d] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5649cb8ea971] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 8: (SnapRealm::check_cache() const+0x13e) [0x5649cb8e8fa0] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5649cb8e9651] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5649cb87e347] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5649cb57a87e] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5649cb57fe94] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5649cb58a969] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 14: (MDLog::_replay_thread()+0x1f38) [0x5649cb92348c] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5649cb92c9cb] 2024-05-27T08:22:26.932 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 16: (Thread::entry_wrapper()+0x43) [0x7f109b02b72d] 2024-05-27T08:22:26.933 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 17: (Thread::_entry_func(void*)+0xd) [0x7f44163e3749] 2024-05-27T08:22:26.933 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4414f871ca] 2024-05-27T08:22:26.933 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 19: clone() 2024-05-27T08:22:26.933 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:26.933 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.933 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: -1> 2024-05-27T08:22:26.702+0000 7f4400a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4400a00700 time 2024-05-27T08:22:26.664820+0000 2024-05-27T08:22:26.933 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:26.933 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.933 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4416414ba6] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4416414dd1] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5640f566e23d] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5640f5659971] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 5: (SnapRealm::check_cache() const+0x13e) [0x5640f5657fa0] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5640f5658651] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5640f55ed347] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5640f52e987e] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5640f52eef2f] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5640f52f9969] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 11: (MDLog::_replay_thread()+0x1f38) [0x5640f569248c] 2024-05-27T08:22:26.934 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5640f569b9cb] 2024-05-27T08:22:26.935 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 13: (Thread::entry_wrapper()+0x43) [0x7f44163e372d] 2024-05-27T08:22:26.935 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 14: (Thread::_entry_func(void*)+0xd) [0x7f44163e3749] 2024-05-27T08:22:26.935 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4414f871ca] 2024-05-27T08:22:26.935 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 16: clone() 2024-05-27T08:22:26.935 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 17: (Thread::_entry_func(void*)+0xd) [0x7f109b02b749] 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f1099bcf1ca] 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 19: clone() 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: -1> 2024-05-27T08:22:26.701+0000 7f1085600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f1085600700 time 2024-05-27T08:22:26.698749+0000 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f109b05cba6] 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f109b05cdd1] 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5649cb8ff23d] 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5649cb8ea971] 2024-05-27T08:22:26.936 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 5: (SnapRealm::check_cache() const+0x13e) [0x5649cb8e8fa0] 2024-05-27T08:22:26.937 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5649cb8e9651] 2024-05-27T08:22:26.937 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5649cb87e347] 2024-05-27T08:22:26.937 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5649cb57a87e] 2024-05-27T08:22:26.937 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5649cb57fe94] 2024-05-27T08:22:26.937 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5649cb58a969] 2024-05-27T08:22:26.937 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 11: (MDLog::_replay_thread()+0x1f38) [0x5649cb92348c] 2024-05-27T08:22:26.937 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5649cb92c9cb] 2024-05-27T08:22:26.937 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 13: (Thread::entry_wrapper()+0x43) [0x7f109b02b72d] 2024-05-27T08:22:26.937 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 14: (Thread::_entry_func(void*)+0xd) [0x7f109b02b749] 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f1099bcf1ca] 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 16: clone() 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 0> 2024-05-27T08:22:26.704+0000 7f1085600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: in thread 7f1085600700 thread_name:md_log_replay 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1099bd9d20] 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2: gsignal() 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 3: abort() 2024-05-27T08:22:26.938 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f109b05cced] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f109b05cdd1] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5649cb8ff23d] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5649cb8ea971] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 8: (SnapRealm::check_cache() const+0x13e) [0x5649cb8e8fa0] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5649cb8e9651] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5649cb87e347] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5649cb57a87e] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5649cb57fe94] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5649cb58a969] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 14: (MDLog::_replay_thread()+0x1f38) [0x5649cb92348c] 2024-05-27T08:22:26.939 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5649cb92c9cb] 2024-05-27T08:22:26.940 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 16: (Thread::entry_wrapper()+0x43) [0x7f109b02b72d] 2024-05-27T08:22:26.940 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 17: (Thread::_entry_func(void*)+0xd) [0x7f109b02b749] 2024-05-27T08:22:26.940 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f1099bcf1ca] 2024-05-27T08:22:26.940 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 19: clone() 2024-05-27T08:22:26.940 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:26.940 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[82782]: 2024-05-27T08:22:26.940 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.940 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 0> 2024-05-27T08:22:26.705+0000 7f4400a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:26.940 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: in thread 7f4400a00700 thread_name:md_log_replay 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4414f91d20] 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2: gsignal() 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 3: abort() 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4416414ced] 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4416414dd1] 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5640f566e23d] 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5640f5659971] 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 8: (SnapRealm::check_cache() const+0x13e) [0x5640f5657fa0] 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5640f5658651] 2024-05-27T08:22:26.941 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5640f55ed347] 2024-05-27T08:22:26.942 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5640f52e987e] 2024-05-27T08:22:26.942 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5640f52eef2f] 2024-05-27T08:22:26.942 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5640f52f9969] 2024-05-27T08:22:26.942 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 14: (MDLog::_replay_thread()+0x1f38) [0x5640f569248c] 2024-05-27T08:22:26.942 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5640f569b9cb] 2024-05-27T08:22:26.942 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 16: (Thread::entry_wrapper()+0x43) [0x7f44163e372d] 2024-05-27T08:22:26.942 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 17: (Thread::_entry_func(void*)+0xd) [0x7f44163e3749] 2024-05-27T08:22:26.942 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4414f871ca] 2024-05-27T08:22:26.942 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 19: clone() 2024-05-27T08:22:26.942 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:26.943 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c[82758]: 2024-05-27T08:22:26.966 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:26 smithi063 ceph-mds[189791]: -- 172.21.15.63:0/1680343215 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563752c95a40 con 0x563752cdd180 2024-05-27T08:22:26.966 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:26 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: starting mds.a at 2024-05-27T08:22:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:26 smithi063 ceph-mon[31613]: pgmap v1567: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 op/s 2024-05-27T08:22:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:26 smithi063 ceph-mon[31613]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:26 smithi063 ceph-mon[31613]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:26.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:26 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:27.188 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:26 smithi149 podman[83455]: 2024-05-27 08:22:26.906679692 +0000 UTC m=+0.156900183 container create d3635690483abef0128f1c033f3a863eef9d83a402eee0a5c7563580c490c239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:22:27.188 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:27 smithi149 podman[83466]: 2024-05-27 08:22:27.098104931 +0000 UTC m=+0.302846324 container died 31f55df492a4e9d7887f50f2db4dd289b77714f3109e6e07afdf8e957425a01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521) 2024-05-27T08:22:27.189 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:26 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 6. 2024-05-27T08:22:27.189 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:26 smithi149 systemd[1]: Stopped Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:27.189 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:26 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Consumed 1.544s CPU time. 2024-05-27T08:22:27.189 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:26 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Start request repeated too quickly. 2024-05-27T08:22:27.189 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:26 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-05-27T08:22:27.189 INFO:journalctl@ceph.mds.i.smithi149.stdout:May 27 08:22:26 smithi149 systemd[1]: Failed to start Ceph mds.i for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:28.165 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:28 smithi149 podman[83455]: 2024-05-27 08:22:28.016977503 +0000 UTC m=+1.267197980 container init d3635690483abef0128f1c033f3a863eef9d83a402eee0a5c7563580c490c239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-27T08:22:28.165 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:28 smithi149 podman[83455]: 2024-05-27 08:22:28.027321292 +0000 UTC m=+1.277541763 container start d3635690483abef0128f1c033f3a863eef9d83a402eee0a5c7563580c490c239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-27T08:22:28.165 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:28 smithi149 ceph-mds[83519]: -- 172.21.15.149:0/2435705069 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5611ba8ec4e0 con 0x5611ba937600 2024-05-27T08:22:28.165 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:28 smithi149 ceph-mds[83519]: -- 172.21.15.149:0/2435705069 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611ba8efa40 con 0x5611ba937600 2024-05-27T08:22:28.464 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:28 smithi149 podman[83467]: 2024-05-27 08:22:28.261113184 +0000 UTC m=+1.472522007 container died 0fe284811472b80d12242e6521401b54ffa141bce322c19b499009273ebc4373 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:22:28.464 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: starting mds.l at 2024-05-27T08:22:28.464 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:28 smithi149 bash[83455]: d3635690483abef0128f1c033f3a863eef9d83a402eee0a5c7563580c490c239 2024-05-27T08:22:28.465 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:28 smithi149 systemd[1]: Started Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:28.897 INFO:teuthology.orchestra.run.smithi149.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-05-27T08:22:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:28 smithi086 ceph-mon[34687]: pgmap v1568: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 0 op/s 2024-05-27T08:22:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:28 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.63:6840/3988096922,v1:172.21.15.63:6841/3988096922] up:boot 2024-05-27T08:22:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:28 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:22:28.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:28 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:22:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:28 smithi149 ceph-mon[35408]: pgmap v1568: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 0 op/s 2024-05-27T08:22:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:28 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.63:6840/3988096922,v1:172.21.15.63:6841/3988096922] up:boot 2024-05-27T08:22:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:28 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:22:28.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:28 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:22:28.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:28 smithi063 ceph-mon[31613]: pgmap v1568: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 0 op/s 2024-05-27T08:22:28.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:28 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.63:6840/3988096922,v1:172.21.15.63:6841/3988096922] up:boot 2024-05-27T08:22:28.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:28 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:22:28.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:28 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:22:29.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:29 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6832/1981289880,v1:172.21.15.149:6833/1981289880] up:boot 2024-05-27T08:22:29.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:29 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:29.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:29 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:22:29.714 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:29 smithi149 podman[83467]: 2024-05-27 08:22:29.456104509 +0000 UTC m=+2.667513337 container remove 0fe284811472b80d12242e6521401b54ffa141bce322c19b499009273ebc4373 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-27T08:22:29.714 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:29 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:22:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:29 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6832/1981289880,v1:172.21.15.149:6833/1981289880] up:boot 2024-05-27T08:22:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:29 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:29.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:29 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:22:29.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:29 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6832/1981289880,v1:172.21.15.149:6833/1981289880] up:boot 2024-05-27T08:22:29.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:29 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-27T08:22:29.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:29 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:22:30.696 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:30 smithi149 podman[83466]: 2024-05-27 08:22:30.252494038 +0000 UTC m=+3.457235450 container remove 31f55df492a4e9d7887f50f2db4dd289b77714f3109e6e07afdf8e957425a01e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-c, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-27T08:22:30.696 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:30 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:22:30.845 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-27T08:22:30.846 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi149 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump' 2024-05-27T08:22:30.848 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-27T08:22:30.859 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-27T08:22:30.860 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T08:22:30.860 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T08:22:30.936 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T08:22:30.936 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T08:22:30.959 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:30 smithi086 ceph-mon[34687]: pgmap v1569: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1 op/s 2024-05-27T08:22:30.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:30 smithi149 ceph-mon[35408]: pgmap v1569: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1 op/s 2024-05-27T08:22:30.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:30 smithi063 ceph-mon[31613]: pgmap v1569: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1 op/s 2024-05-27T08:22:30.997 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T08:22:31.464 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:31 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-05-27T08:22:31.464 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:31 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 1.536s CPU time. 2024-05-27T08:22:31.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:31 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T08:22:31.464 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:31 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.553s CPU time. 2024-05-27T08:22:31.491 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:22:31.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:31 smithi086 ceph-mon[34687]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:31.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:31 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:22:31.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:31 smithi149 ceph-mon[35408]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:31.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:31 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:22:31.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f6dfce00700 time 2024-05-27T08:22:31.665340+0000 2024-05-27T08:22:31.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:31.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:31.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6e129fbba6] 2024-05-27T08:22:31.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6e129fbdd1] 2024-05-27T08:22:31.966 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x56118b20823d] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 4: (SnapRealm::build_snap_set() const+0x323) [0x56118b1f3971] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 5: (SnapRealm::check_cache() const+0x13e) [0x56118b1f1fa0] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x56118b1f2651] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x56118b187347] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x56118ae8387e] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x56118ae88e94] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x56118ae93969] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 11: (MDLog::_replay_thread()+0x1f38) [0x56118b22c48c] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 12: (MDLog::ReplayThread::entry()+0x11) [0x56118b2359cb] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 13: (Thread::entry_wrapper()+0x43) [0x7f6e129ca72d] 2024-05-27T08:22:31.967 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 14: (Thread::_entry_func(void*)+0xd) [0x7f6e129ca749] 2024-05-27T08:22:31.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6e1156e1ca] 2024-05-27T08:22:31.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 16: clone() 2024-05-27T08:22:31.968 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.668+0000 7f6dfce00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f6dfce00700 time 2024-05-27T08:22:31.665340+0000 2024-05-27T08:22:31.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:31 smithi063 ceph-mon[31613]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:31.968 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:31 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 5 up:standby 2024-05-27T08:22:31.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:31.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:31.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6e129fbba6] 2024-05-27T08:22:31.969 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6e129fbdd1] 2024-05-27T08:22:31.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x56118b20823d] 2024-05-27T08:22:31.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 4: (SnapRealm::build_snap_set() const+0x323) [0x56118b1f3971] 2024-05-27T08:22:31.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 5: (SnapRealm::check_cache() const+0x13e) [0x56118b1f1fa0] 2024-05-27T08:22:31.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x56118b1f2651] 2024-05-27T08:22:31.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x56118b187347] 2024-05-27T08:22:31.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x56118ae8387e] 2024-05-27T08:22:31.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x56118ae88e94] 2024-05-27T08:22:31.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x56118ae93969] 2024-05-27T08:22:31.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 11: (MDLog::_replay_thread()+0x1f38) [0x56118b22c48c] 2024-05-27T08:22:31.970 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 12: (MDLog::ReplayThread::entry()+0x11) [0x56118b2359cb] 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 13: (Thread::entry_wrapper()+0x43) [0x7f6e129ca72d] 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 14: (Thread::_entry_func(void*)+0xd) [0x7f6e129ca749] 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6e1156e1ca] 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 16: clone() 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: *** Caught signal (Aborted) ** 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: in thread 7f6dfce00700 thread_name:md_log_replay 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6e11578d20] 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2: gsignal() 2024-05-27T08:22:31.971 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 3: abort() 2024-05-27T08:22:31.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f6e129fbced] 2024-05-27T08:22:31.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6e129fbdd1] 2024-05-27T08:22:31.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x56118b20823d] 2024-05-27T08:22:31.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 7: (SnapRealm::build_snap_set() const+0x323) [0x56118b1f3971] 2024-05-27T08:22:31.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 8: (SnapRealm::check_cache() const+0x13e) [0x56118b1f1fa0] 2024-05-27T08:22:31.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x56118b1f2651] 2024-05-27T08:22:31.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x56118b187347] 2024-05-27T08:22:31.972 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x56118ae8387e] 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x56118ae88e94] 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x56118ae93969] 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 14: (MDLog::_replay_thread()+0x1f38) [0x56118b22c48c] 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 15: (MDLog::ReplayThread::entry()+0x11) [0x56118b2359cb] 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 16: (Thread::entry_wrapper()+0x43) [0x7f6e129ca72d] 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 17: (Thread::_entry_func(void*)+0xd) [0x7f6e129ca749] 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6e1156e1ca] 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 19: clone() 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.672+0000 7f6dfce00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: in thread 7f6dfce00700 thread_name:md_log_replay 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.973 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:31.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6e11578d20] 2024-05-27T08:22:31.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2: gsignal() 2024-05-27T08:22:31.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 3: abort() 2024-05-27T08:22:31.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f6e129fbced] 2024-05-27T08:22:31.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6e129fbdd1] 2024-05-27T08:22:31.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x56118b20823d] 2024-05-27T08:22:31.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 7: (SnapRealm::build_snap_set() const+0x323) [0x56118b1f3971] 2024-05-27T08:22:31.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 8: (SnapRealm::check_cache() const+0x13e) [0x56118b1f1fa0] 2024-05-27T08:22:31.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x56118b1f2651] 2024-05-27T08:22:31.974 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x56118b187347] 2024-05-27T08:22:31.975 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x56118ae8387e] 2024-05-27T08:22:31.975 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x56118ae88e94] 2024-05-27T08:22:31.975 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x56118ae93969] 2024-05-27T08:22:31.975 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 14: (MDLog::_replay_thread()+0x1f38) [0x56118b22c48c] 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 15: (MDLog::ReplayThread::entry()+0x11) [0x56118b2359cb] 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 16: (Thread::entry_wrapper()+0x43) [0x7f6e129ca72d] 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 17: (Thread::_entry_func(void*)+0xd) [0x7f6e129ca749] 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6e1156e1ca] 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 19: clone() 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: -1> 2024-05-27T08:22:31.668+0000 7f6dfce00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f6dfce00700 time 2024-05-27T08:22:31.665340+0000 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.976 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6e129fbba6] 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6e129fbdd1] 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x56118b20823d] 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 4: (SnapRealm::build_snap_set() const+0x323) [0x56118b1f3971] 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 5: (SnapRealm::check_cache() const+0x13e) [0x56118b1f1fa0] 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x56118b1f2651] 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x56118b187347] 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x56118ae8387e] 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x56118ae88e94] 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x56118ae93969] 2024-05-27T08:22:31.977 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 11: (MDLog::_replay_thread()+0x1f38) [0x56118b22c48c] 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 12: (MDLog::ReplayThread::entry()+0x11) [0x56118b2359cb] 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 13: (Thread::entry_wrapper()+0x43) [0x7f6e129ca72d] 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 14: (Thread::_entry_func(void*)+0xd) [0x7f6e129ca749] 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6e1156e1ca] 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 16: clone() 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 0> 2024-05-27T08:22:31.672+0000 7f6dfce00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: in thread 7f6dfce00700 thread_name:md_log_replay 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:31.978 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6e11578d20] 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2: gsignal() 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 3: abort() 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f6e129fbced] 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6e129fbdd1] 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x56118b20823d] 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 7: (SnapRealm::build_snap_set() const+0x323) [0x56118b1f3971] 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 8: (SnapRealm::check_cache() const+0x13e) [0x56118b1f1fa0] 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x56118b1f2651] 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x56118b187347] 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x56118ae8387e] 2024-05-27T08:22:31.979 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x56118ae88e94] 2024-05-27T08:22:31.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x56118ae93969] 2024-05-27T08:22:31.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 14: (MDLog::_replay_thread()+0x1f38) [0x56118b22c48c] 2024-05-27T08:22:31.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 15: (MDLog::ReplayThread::entry()+0x11) [0x56118b2359cb] 2024-05-27T08:22:31.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 16: (Thread::entry_wrapper()+0x43) [0x7f6e129ca72d] 2024-05-27T08:22:31.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 17: (Thread::_entry_func(void*)+0xd) [0x7f6e129ca749] 2024-05-27T08:22:31.980 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6e1156e1ca] 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 19: clone() 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: -1> 2024-05-27T08:22:31.668+0000 7f6dfce00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f6dfce00700 time 2024-05-27T08:22:31.665340+0000 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6e129fbba6] 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6e129fbdd1] 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x56118b20823d] 2024-05-27T08:22:31.981 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 4: (SnapRealm::build_snap_set() const+0x323) [0x56118b1f3971] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 5: (SnapRealm::check_cache() const+0x13e) [0x56118b1f1fa0] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x56118b1f2651] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x56118b187347] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x56118ae8387e] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x56118ae88e94] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x56118ae93969] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 11: (MDLog::_replay_thread()+0x1f38) [0x56118b22c48c] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 12: (MDLog::ReplayThread::entry()+0x11) [0x56118b2359cb] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 13: (Thread::entry_wrapper()+0x43) [0x7f6e129ca72d] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 14: (Thread::_entry_func(void*)+0xd) [0x7f6e129ca749] 2024-05-27T08:22:31.982 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6e1156e1ca] 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 16: clone() 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 0> 2024-05-27T08:22:31.672+0000 7f6dfce00700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: in thread 7f6dfce00700 thread_name:md_log_replay 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6e11578d20] 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2: gsignal() 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 3: abort() 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f6e129fbced] 2024-05-27T08:22:31.983 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f6e129fbdd1] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x56118b20823d] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 7: (SnapRealm::build_snap_set() const+0x323) [0x56118b1f3971] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 8: (SnapRealm::check_cache() const+0x13e) [0x56118b1f1fa0] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x56118b1f2651] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x56118b187347] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x56118ae8387e] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x56118ae88e94] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x56118ae93969] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 14: (MDLog::_replay_thread()+0x1f38) [0x56118b22c48c] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 15: (MDLog::ReplayThread::entry()+0x11) [0x56118b2359cb] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 16: (Thread::entry_wrapper()+0x43) [0x7f6e129ca72d] 2024-05-27T08:22:31.984 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 17: (Thread::_entry_func(void*)+0xd) [0x7f6e129ca749] 2024-05-27T08:22:31.985 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f6e1156e1ca] 2024-05-27T08:22:31.985 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 19: clone() 2024-05-27T08:22:31.985 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:31.985 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[83507]: 2024-05-27T08:22:31.985 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:31 smithi149 podman[83643]: 2024-05-27 08:22:31.758628715 +0000 UTC m=+0.032210544 container died d3635690483abef0128f1c033f3a863eef9d83a402eee0a5c7563580c490c239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-27T08:22:32.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:32 smithi149 ceph-mon[35408]: pgmap v1570: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1 op/s 2024-05-27T08:22:32.714 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:32 smithi149 podman[83643]: 2024-05-27 08:22:32.460355258 +0000 UTC m=+0.733937104 container remove d3635690483abef0128f1c033f3a863eef9d83a402eee0a5c7563580c490c239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:22:32.714 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:32 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:22:32.932 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:32 smithi063 ceph-mon[31613]: pgmap v1570: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1 op/s 2024-05-27T08:22:33.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:32 smithi086 ceph-mon[34687]: pgmap v1570: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1 op/s 2024-05-27T08:22:33.213 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:32 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-05-27T08:22:33.213 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:32 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Consumed 1.425s CPU time. 2024-05-27T08:22:33.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.950+0000 7f4117600700 1 -- 172.21.15.63:0/1530836157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41101045e0 msgr2=0x7f41101049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:22:33.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.950+0000 7f4117600700 1 --2- 172.21.15.63:0/1530836157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41101045e0 0x7f41101049f0 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f41000099c0 tx=0x7f4100009cd0 comp rx=0 tx=0).stop 2024-05-27T08:22:33.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.950+0000 7f4117600700 1 -- 172.21.15.63:0/1530836157 shutdown_connections 2024-05-27T08:22:33.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.950+0000 7f4117600700 1 --2- 172.21.15.63:0/1530836157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4110109860 0x7f41101111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:33.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.950+0000 7f4117600700 1 --2- 172.21.15.63:0/1530836157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41101045e0 0x7f41101049f0 unknown :-1 s=CLOSED pgs=3047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:33.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.950+0000 7f4117600700 1 --2- 172.21.15.63:0/1530836157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4110108f10 0x7f4110109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:33.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.950+0000 7f4117600700 1 -- 172.21.15.63:0/1530836157 >> 172.21.15.63:0/1530836157 conn(0x7f4110100030 msgr2=0x7f4110102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:22:33.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.950+0000 7f4117600700 1 -- 172.21.15.63:0/1530836157 shutdown_connections 2024-05-27T08:22:33.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.950+0000 7f4117600700 1 -- 172.21.15.63:0/1530836157 wait complete. 2024-05-27T08:22:33.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.951+0000 7f4117600700 1 Processor -- start 2024-05-27T08:22:33.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.951+0000 7f4117600700 1 -- start start 2024-05-27T08:22:33.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.952+0000 7f4117600700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41101045e0 0x7f411019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:22:33.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.952+0000 7f4117600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4110108f10 0x7f411019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.952+0000 7f4117600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4110109860 0x7f41101a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.952+0000 7f4117600700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4110114360 con 0x7f41101045e0 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.952+0000 7f4117600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41101141e0 con 0x7f4110108f10 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.952+0000 7f4117600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f41101144e0 con 0x7f4110109860 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.952+0000 7f4114c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41101045e0 0x7f411019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.952+0000 7f4114c00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41101045e0 0x7f411019cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52912/0 (socket says 172.21.15.63:52912) 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.952+0000 7f4114c00700 1 -- 172.21.15.63:0/700365130 learned_addr learned my addr 172.21.15.63:0/700365130 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.953+0000 7f4107e00700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4110108f10 0x7f411019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.953+0000 7f4115600700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4110109860 0x7f41101a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.953+0000 7f4114c00700 1 -- 172.21.15.63:0/700365130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4110109860 msgr2=0x7f41101a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.953+0000 7f4114c00700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4110109860 0x7f41101a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:33.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.953+0000 7f4114c00700 1 -- 172.21.15.63:0/700365130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4110108f10 msgr2=0x7f411019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:22:33.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.953+0000 7f4114c00700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4110108f10 0x7f411019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:33.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.953+0000 7f4114c00700 1 -- 172.21.15.63:0/700365130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41101a1e00 con 0x7f41101045e0 2024-05-27T08:22:33.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.953+0000 7f4114c00700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41101045e0 0x7f411019cf30 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f4100000c00 tx=0x7f410000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:22:33.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.954+0000 7f410e000700 1 -- 172.21.15.63:0/700365130 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410000f940 con 0x7f41101045e0 2024-05-27T08:22:33.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.954+0000 7f410e000700 1 -- 172.21.15.63:0/700365130 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4100004830 con 0x7f41101045e0 2024-05-27T08:22:33.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.954+0000 7f410e000700 1 -- 172.21.15.63:0/700365130 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41000054b0 con 0x7f41101045e0 2024-05-27T08:22:33.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.954+0000 7f4117600700 1 -- 172.21.15.63:0/700365130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41101a2090 con 0x7f41101045e0 2024-05-27T08:22:33.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.954+0000 7f4117600700 1 -- 172.21.15.63:0/700365130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41101a2560 con 0x7f41101045e0 2024-05-27T08:22:33.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.955+0000 7f4117600700 1 -- 172.21.15.63:0/700365130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41100512b0 con 0x7f41101045e0 2024-05-27T08:22:33.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.958+0000 7f410e000700 1 -- 172.21.15.63:0/700365130 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4100029060 con 0x7f41101045e0 2024-05-27T08:22:33.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.959+0000 7f410e000700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40f40b2a70 0x7f40f40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:22:33.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.960+0000 7f410e000700 1 -- 172.21.15.63:0/700365130 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 9483+0+0 (secure 0 0 0) 0x7f41000e3830 con 0x7f41101045e0 2024-05-27T08:22:33.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.960+0000 7f4107e00700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40f40b2a70 0x7f40f40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:22:33.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.960+0000 7f4107e00700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40f40b2a70 0x7f40f40b4f80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f40fc005950 tx=0x7f40fc0058e0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:22:33.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:33.962+0000 7f410e000700 1 -- 172.21.15.63:0/700365130 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4100013070 con 0x7f41101045e0 2024-05-27T08:22:34.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.222+0000 7f4117600700 1 -- 172.21.15.63:0/700365130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f41101a9a70 con 0x7f41101045e0 2024-05-27T08:22:34.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.223+0000 7f410e000700 1 -- 172.21.15.63:0/700365130 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v192) ==== 81+0+941 (secure 0 0 0) 0x7f41000b1e20 con 0x7f41101045e0 2024-05-27T08:22:34.225 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1961493183 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:34.225 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/675305989 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:34.225 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1202958531 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:34.225 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/675305989 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:34.225 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1689640179 2024-05-28T07:18:03.559305+0000 2024-05-27T08:22:34.225 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3946208673 2024-05-28T07:18:03.559305+0000 2024-05-27T08:22:34.225 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3428263920 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:34.225 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/736953168 2024-05-28T07:18:03.559305+0000 2024-05-27T08:22:34.226 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/3806249841 2024-05-28T07:17:27.459562+0000 2024-05-27T08:22:34.226 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1689640179 2024-05-28T07:18:03.559305+0000 2024-05-27T08:22:34.226 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4210215351 2024-05-28T07:17:27.459562+0000 2024-05-27T08:22:34.226 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/3806249841 2024-05-28T07:17:27.459562+0000 2024-05-27T08:22:34.226 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1558308201 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:34.226 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2613878696 2024-05-28T07:17:27.459562+0000 2024-05-27T08:22:34.226 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1318703057 2024-05-28T07:17:27.459562+0000 2024-05-27T08:22:34.226 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/58174792 2024-05-28T07:18:03.559305+0000 2024-05-27T08:22:34.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.226+0000 7f4117600700 1 -- 172.21.15.63:0/700365130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40f40b2a70 msgr2=0x7f40f40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:22:34.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.226+0000 7f4117600700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40f40b2a70 0x7f40f40b4f80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f40fc005950 tx=0x7f40fc0058e0 comp rx=0 tx=0).stop 2024-05-27T08:22:34.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.226+0000 7f4117600700 1 -- 172.21.15.63:0/700365130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41101045e0 msgr2=0x7f411019cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:22:34.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.226+0000 7f4117600700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41101045e0 0x7f411019cf30 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f4100000c00 tx=0x7f410000f510 comp rx=0 tx=0).stop 2024-05-27T08:22:34.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.227+0000 7f4117600700 1 -- 172.21.15.63:0/700365130 shutdown_connections 2024-05-27T08:22:34.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.227+0000 7f4117600700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7f40f40b2a70 0x7f40f40b4f80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:34.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.227+0000 7f4117600700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4110109860 0x7f41101a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:34.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.227+0000 7f4117600700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41101045e0 0x7f411019cf30 unknown :-1 s=CLOSED pgs=3048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:34.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.227+0000 7f4117600700 1 --2- 172.21.15.63:0/700365130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4110108f10 0x7f411019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:34.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.227+0000 7f4117600700 1 -- 172.21.15.63:0/700365130 >> 172.21.15.63:0/700365130 conn(0x7f4110100030 msgr2=0x7f411010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:22:34.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.227+0000 7f4117600700 1 -- 172.21.15.63:0/700365130 shutdown_connections 2024-05-27T08:22:34.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:34.227+0000 7f4117600700 1 -- 172.21.15.63:0/700365130 wait complete. 2024-05-27T08:22:34.230 INFO:teuthology.orchestra.run.smithi063.stderr:listed 16 entries 2024-05-27T08:22:34.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:34 smithi063 ceph-mon[31613]: pgmap v1571: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1 op/s 2024-05-27T08:22:34.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:34 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/700365130' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:22:34.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:34 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:22:35.017 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:34 smithi149 ceph-mon[35408]: pgmap v1571: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1 op/s 2024-05-27T08:22:35.017 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:34 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/700365130' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:22:35.017 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:34 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:22:35.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:34 smithi086 ceph-mon[34687]: pgmap v1571: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1 op/s 2024-05-27T08:22:35.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:34 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/700365130' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:22:35.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:34 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:22:35.337 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-27T08:22:35.338 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T08:22:35.386 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T08:22:35.897 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:22:36.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:36 smithi063 ceph-mon[31613]: pgmap v1572: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2 op/s 2024-05-27T08:22:36.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:36 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:36 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:36 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:36 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:36 smithi086 ceph-mon[34687]: pgmap v1572: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2 op/s 2024-05-27T08:22:36.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:36 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:36 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:36 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:36 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:36 smithi149 ceph-mon[35408]: pgmap v1572: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2 op/s 2024-05-27T08:22:36.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:36 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:36 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:36 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:36.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:36 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:37.716 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:37 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:22:37.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:37 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:22:37.717 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:37 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:37.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:37 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:22:37.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:37 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:22:37.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:37 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:37.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:37 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:22:37.963 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:37 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:22:37.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:37 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' 2024-05-27T08:22:38.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.136+0000 7fea51a00700 1 -- 172.21.15.63:0/3785215340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea4c103110 msgr2=0x7fea4c10f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:22:38.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.136+0000 7fea51a00700 1 --2- 172.21.15.63:0/3785215340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea4c103110 0x7fea4c10f690 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7fea44009990 tx=0x7fea44009ca0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.136+0000 7fea51a00700 1 -- 172.21.15.63:0/3785215340 shutdown_connections 2024-05-27T08:22:38.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.136+0000 7fea51a00700 1 --2- 172.21.15.63:0/3785215340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea4c103110 0x7fea4c10f690 unknown :-1 s=CLOSED pgs=3049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.136+0000 7fea51a00700 1 --2- 172.21.15.63:0/3785215340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c1087f0 0x7fea4c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.136+0000 7fea51a00700 1 --2- 172.21.15.63:0/3785215340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea4c1027c0 0x7fea4c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.136+0000 7fea51a00700 1 -- 172.21.15.63:0/3785215340 >> 172.21.15.63:0/3785215340 conn(0x7fea4c0783a0 msgr2=0x7fea4c0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:22:38.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.136+0000 7fea51a00700 1 -- 172.21.15.63:0/3785215340 shutdown_connections 2024-05-27T08:22:38.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.136+0000 7fea51a00700 1 -- 172.21.15.63:0/3785215340 wait complete. 2024-05-27T08:22:38.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.138+0000 7fea51a00700 1 Processor -- start 2024-05-27T08:22:38.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.138+0000 7fea51a00700 1 -- start start 2024-05-27T08:22:38.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.138+0000 7fea51a00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea4c1027c0 0x7fea4c109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:22:38.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.138+0000 7fea51a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c103110 0x7fea4c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:22:38.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.138+0000 7fea51a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea4c1087f0 0x7fea4c10a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:22:38.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.138+0000 7fea51a00700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c1127c0 con 0x7fea4c1027c0 2024-05-27T08:22:38.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.138+0000 7fea51a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c112640 con 0x7fea4c1087f0 2024-05-27T08:22:38.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.138+0000 7fea51a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c112940 con 0x7fea4c103110 2024-05-27T08:22:38.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea4be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea4c1027c0 0x7fea4c109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:22:38.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea4be00700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea4c1027c0 0x7fea4c109730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52930/0 (socket says 172.21.15.63:52930) 2024-05-27T08:22:38.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea4be00700 1 -- 172.21.15.63:0/2677104015 learned_addr learned my addr 172.21.15.63:0/2677104015 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-27T08:22:38.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea43400700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c103110 0x7fea4c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:22:38.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea51000700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea4c1087f0 0x7fea4c10a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:22:38.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea4be00700 1 -- 172.21.15.63:0/2677104015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c103110 msgr2=0x7fea4c109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:22:38.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea4be00700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c103110 0x7fea4c109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea4be00700 1 -- 172.21.15.63:0/2677104015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea4c1087f0 msgr2=0x7fea4c10a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:22:38.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea4be00700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea4c1087f0 0x7fea4c10a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea4be00700 1 -- 172.21.15.63:0/2677104015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c1a8e70 con 0x7fea4c1027c0 2024-05-27T08:22:38.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea51000700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea4c1087f0 0x7fea4c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:22:38.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea43400700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c103110 0x7fea4c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:22:38.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.139+0000 7fea4be00700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea4c1027c0 0x7fea4c109730 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00ada0 tx=0x7fea3c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:22:38.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.140+0000 7fea49600700 1 -- 172.21.15.63:0/2677104015 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c017020 con 0x7fea4c1027c0 2024-05-27T08:22:38.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.140+0000 7fea49600700 1 -- 172.21.15.63:0/2677104015 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea3c00ce10 con 0x7fea4c1027c0 2024-05-27T08:22:38.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.140+0000 7fea51a00700 1 -- 172.21.15.63:0/2677104015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c1a9160 con 0x7fea4c1027c0 2024-05-27T08:22:38.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.140+0000 7fea49600700 1 -- 172.21.15.63:0/2677104015 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0054e0 con 0x7fea4c1027c0 2024-05-27T08:22:38.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.140+0000 7fea51a00700 1 -- 172.21.15.63:0/2677104015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c1a96f0 con 0x7fea4c1027c0 2024-05-27T08:22:38.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.141+0000 7fea51a00700 1 -- 172.21.15.63:0/2677104015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4c0512b0 con 0x7fea4c1027c0 2024-05-27T08:22:38.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.144+0000 7fea49600700 1 -- 172.21.15.63:0/2677104015 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fea3c004c40 con 0x7fea4c1027c0 2024-05-27T08:22:38.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.145+0000 7fea49600700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea300b2a70 0x7fea300b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:22:38.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.145+0000 7fea49600700 1 -- 172.21.15.63:0/2677104015 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 9483+0+0 (secure 0 0 0) 0x7fea3c0e2680 con 0x7fea4c1027c0 2024-05-27T08:22:38.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.145+0000 7fea43400700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea300b2a70 0x7fea300b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:22:38.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.146+0000 7fea43400700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea300b2a70 0x7fea300b4f80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fea34005fd0 tx=0x7fea34005dc0 comp rx=0 tx=0).ready entity=mgr.15216 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:22:38.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.148+0000 7fea49600700 1 -- 172.21.15.63:0/2677104015 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea3c021020 con 0x7fea4c1027c0 2024-05-27T08:22:38.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.407+0000 7fea51a00700 1 -- 172.21.15.63:0/2677104015 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fea4c10ac60 con 0x7fea4c1027c0 2024-05-27T08:22:38.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.408+0000 7fea49600700 1 -- 172.21.15.63:0/2677104015 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v192) ==== 81+0+941 (secure 0 0 0) 0x7fea3c0a9cd0 con 0x7fea4c1027c0 2024-05-27T08:22:38.410 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1961493183 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:38.410 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/675305989 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:38.410 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1202958531 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:38.410 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/675305989 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:38.410 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/1689640179 2024-05-28T07:18:03.559305+0000 2024-05-27T08:22:38.410 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3946208673 2024-05-28T07:18:03.559305+0000 2024-05-27T08:22:38.411 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3428263920 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:38.411 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/736953168 2024-05-28T07:18:03.559305+0000 2024-05-27T08:22:38.411 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/3806249841 2024-05-28T07:17:27.459562+0000 2024-05-27T08:22:38.411 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/1689640179 2024-05-28T07:18:03.559305+0000 2024-05-27T08:22:38.411 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4210215351 2024-05-28T07:17:27.459562+0000 2024-05-27T08:22:38.411 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/3806249841 2024-05-28T07:17:27.459562+0000 2024-05-27T08:22:38.411 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1558308201 2024-05-28T07:33:35.220779+0000 2024-05-27T08:22:38.411 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2613878696 2024-05-28T07:17:27.459562+0000 2024-05-27T08:22:38.411 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1318703057 2024-05-28T07:17:27.459562+0000 2024-05-27T08:22:38.411 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/58174792 2024-05-28T07:18:03.559305+0000 2024-05-27T08:22:38.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 -- 172.21.15.63:0/2677104015 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea300b2a70 msgr2=0x7fea300b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:22:38.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea300b2a70 0x7fea300b4f80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fea34005fd0 tx=0x7fea34005dc0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 -- 172.21.15.63:0/2677104015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea4c1027c0 msgr2=0x7fea4c109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:22:38.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea4c1027c0 0x7fea4c109730 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00ada0 tx=0x7fea3c00c330 comp rx=0 tx=0).stop 2024-05-27T08:22:38.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 -- 172.21.15.63:0/2677104015 shutdown_connections 2024-05-27T08:22:38.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.63:6800/1343315489,v1:172.21.15.63:6801/1343315489] conn(0x7fea300b2a70 0x7fea300b4f80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea4c1087f0 0x7fea4c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea4c1027c0 0x7fea4c109730 unknown :-1 s=CLOSED pgs=3050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 --2- 172.21.15.63:0/2677104015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c103110 0x7fea4c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:22:38.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 -- 172.21.15.63:0/2677104015 >> 172.21.15.63:0/2677104015 conn(0x7fea4c0783a0 msgr2=0x7fea4c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:22:38.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 -- 172.21.15.63:0/2677104015 shutdown_connections 2024-05-27T08:22:38.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-27T08:22:38.412+0000 7fea51a00700 1 -- 172.21.15.63:0/2677104015 wait complete. 2024-05-27T08:22:38.415 INFO:teuthology.orchestra.run.smithi063.stderr:listed 16 entries 2024-05-27T08:22:39.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:38 smithi086 ceph-mon[34687]: pgmap v1573: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2 op/s 2024-05-27T08:22:39.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:38 smithi086 ceph-mon[34687]: from='client.? 172.21.15.63:0/2677104015' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:22:39.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:38 smithi149 ceph-mon[35408]: pgmap v1573: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2 op/s 2024-05-27T08:22:39.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:38 smithi149 ceph-mon[35408]: from='client.? 172.21.15.63:0/2677104015' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:22:39.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:38 smithi063 ceph-mon[31613]: pgmap v1573: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2 op/s 2024-05-27T08:22:39.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:38 smithi063 ceph-mon[31613]: from='client.? 172.21.15.63:0/2677104015' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:22:39.361 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi063.front.sepia.ceph.com... 2024-05-27T08:22:39.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:22:39.362 DEBUG:teuthology.orchestra.run.smithi063:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-27T08:22:39.476 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T08:22:39.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi063.stderr:ceph-fuse[90751]: fuse finished with error 0 and tester_r 0 2024-05-27T08:22:41.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:40 smithi086 ceph-mon[34687]: pgmap v1574: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2 op/s 2024-05-27T08:22:41.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:40 smithi149 ceph-mon[35408]: pgmap v1574: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2 op/s 2024-05-27T08:22:41.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:40 smithi063 ceph-mon[31613]: pgmap v1574: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2 op/s 2024-05-27T08:22:41.618 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:41 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 6. 2024-05-27T08:22:41.618 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:41 smithi149 systemd[1]: Stopped Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:41.618 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:41 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service: Consumed 1.536s CPU time. 2024-05-27T08:22:41.619 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:41 smithi149 systemd[1]: Starting Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:41.619 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:41 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 6. 2024-05-27T08:22:41.619 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:41 smithi149 systemd[1]: Stopped Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:41.619 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:41 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.553s CPU time. 2024-05-27T08:22:41.619 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:41 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Start request repeated too quickly. 2024-05-27T08:22:41.619 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:41 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T08:22:41.619 INFO:journalctl@ceph.mds.c.smithi149.stdout:May 27 08:22:41 smithi149 systemd[1]: Failed to start Ceph mds.c for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:41.898 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:41 smithi149 podman[83950]: 2024-05-27 08:22:41.523260366 +0000 UTC m=+0.033995744 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:41.899 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:41 smithi149 podman[83950]: 2024-05-27 08:22:41.66976669 +0000 UTC m=+0.180502050 container create 11e511c032a73ac37a8a65d6ac2f9dbf8a9f8c86cf59e63e72b0608e114a5a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T08:22:41.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:41 smithi086 ceph-mon[34687]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:41.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:41 smithi086 ceph-mon[34687]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:41.960 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:41 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 3 up:standby 2024-05-27T08:22:41.961 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4216000700 time 2024-05-27T08:22:41.679386+0000 2024-05-27T08:22:41.961 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:41.961 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:41.961 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f422bb32ba6] 2024-05-27T08:22:41.961 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f422bb32dd1] 2024-05-27T08:22:41.961 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560a21fca23d] 2024-05-27T08:22:41.961 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560a21fb5971] 2024-05-27T08:22:41.961 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 5: (SnapRealm::check_cache() const+0x13e) [0x560a21fb3fa0] 2024-05-27T08:22:41.961 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560a21fb4651] 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560a21f49347] 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560a21c4587e] 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560a21c4ae94] 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560a21c55969] 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 11: (MDLog::_replay_thread()+0x1f38) [0x560a21fee48c] 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560a21ff79cb] 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 13: (Thread::entry_wrapper()+0x43) [0x7f422bb0172d] 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 14: (Thread::_entry_func(void*)+0xd) [0x7f422bb01749] 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f422a6a51ca] 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 16: clone() 2024-05-27T08:22:41.962 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.682+0000 7f4216000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4216000700 time 2024-05-27T08:22:41.679386+0000 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f422bb32ba6] 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f422bb32dd1] 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560a21fca23d] 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560a21fb5971] 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 5: (SnapRealm::check_cache() const+0x13e) [0x560a21fb3fa0] 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560a21fb4651] 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560a21f49347] 2024-05-27T08:22:41.963 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560a21c4587e] 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560a21c4ae94] 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560a21c55969] 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 11: (MDLog::_replay_thread()+0x1f38) [0x560a21fee48c] 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560a21ff79cb] 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 13: (Thread::entry_wrapper()+0x43) [0x7f422bb0172d] 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 14: (Thread::_entry_func(void*)+0xd) [0x7f422bb01749] 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f422a6a51ca] 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 16: clone() 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: *** Caught signal (Aborted) ** 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: in thread 7f4216000700 thread_name:md_log_replay 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:41.964 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f422a6afd20] 2024-05-27T08:22:41.965 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2: gsignal() 2024-05-27T08:22:41.965 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 3: abort() 2024-05-27T08:22:41.965 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f422bb32ced] 2024-05-27T08:22:41.965 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f422bb32dd1] 2024-05-27T08:22:41.965 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560a21fca23d] 2024-05-27T08:22:41.965 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560a21fb5971] 2024-05-27T08:22:41.965 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 8: (SnapRealm::check_cache() const+0x13e) [0x560a21fb3fa0] 2024-05-27T08:22:41.965 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560a21fb4651] 2024-05-27T08:22:41.965 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560a21f49347] 2024-05-27T08:22:41.965 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560a21c4587e] 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560a21c4ae94] 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560a21c55969] 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 14: (MDLog::_replay_thread()+0x1f38) [0x560a21fee48c] 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560a21ff79cb] 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 16: (Thread::entry_wrapper()+0x43) [0x7f422bb0172d] 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 17: (Thread::_entry_func(void*)+0xd) [0x7f422bb01749] 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f422a6a51ca] 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 19: clone() 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.686+0000 7f4216000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: in thread 7f4216000700 thread_name:md_log_replay 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.966 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:41.967 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f422a6afd20] 2024-05-27T08:22:41.967 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2: gsignal() 2024-05-27T08:22:41.967 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 3: abort() 2024-05-27T08:22:41.967 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f422bb32ced] 2024-05-27T08:22:41.967 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f422bb32dd1] 2024-05-27T08:22:41.967 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560a21fca23d] 2024-05-27T08:22:41.967 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560a21fb5971] 2024-05-27T08:22:41.967 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 8: (SnapRealm::check_cache() const+0x13e) [0x560a21fb3fa0] 2024-05-27T08:22:41.967 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560a21fb4651] 2024-05-27T08:22:41.967 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560a21f49347] 2024-05-27T08:22:41.968 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560a21c4587e] 2024-05-27T08:22:41.968 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560a21c4ae94] 2024-05-27T08:22:41.968 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560a21c55969] 2024-05-27T08:22:41.968 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 14: (MDLog::_replay_thread()+0x1f38) [0x560a21fee48c] 2024-05-27T08:22:41.968 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560a21ff79cb] 2024-05-27T08:22:41.968 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 16: (Thread::entry_wrapper()+0x43) [0x7f422bb0172d] 2024-05-27T08:22:41.968 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 17: (Thread::_entry_func(void*)+0xd) [0x7f422bb01749] 2024-05-27T08:22:41.968 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f422a6a51ca] 2024-05-27T08:22:41.968 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 19: clone() 2024-05-27T08:22:41.968 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: -1> 2024-05-27T08:22:41.682+0000 7f4216000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4216000700 time 2024-05-27T08:22:41.679386+0000 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f422bb32ba6] 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f422bb32dd1] 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560a21fca23d] 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560a21fb5971] 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 5: (SnapRealm::check_cache() const+0x13e) [0x560a21fb3fa0] 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560a21fb4651] 2024-05-27T08:22:41.969 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560a21f49347] 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560a21c4587e] 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560a21c4ae94] 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560a21c55969] 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 11: (MDLog::_replay_thread()+0x1f38) [0x560a21fee48c] 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560a21ff79cb] 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 13: (Thread::entry_wrapper()+0x43) [0x7f422bb0172d] 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 14: (Thread::_entry_func(void*)+0xd) [0x7f422bb01749] 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f422a6a51ca] 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 16: clone() 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 0> 2024-05-27T08:22:41.686+0000 7f4216000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:41.970 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: in thread 7f4216000700 thread_name:md_log_replay 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f422a6afd20] 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2: gsignal() 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 3: abort() 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f422bb32ced] 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f422bb32dd1] 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560a21fca23d] 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560a21fb5971] 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 8: (SnapRealm::check_cache() const+0x13e) [0x560a21fb3fa0] 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560a21fb4651] 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560a21f49347] 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560a21c4587e] 2024-05-27T08:22:41.971 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560a21c4ae94] 2024-05-27T08:22:41.972 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560a21c55969] 2024-05-27T08:22:41.972 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 14: (MDLog::_replay_thread()+0x1f38) [0x560a21fee48c] 2024-05-27T08:22:41.972 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560a21ff79cb] 2024-05-27T08:22:41.972 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 16: (Thread::entry_wrapper()+0x43) [0x7f422bb0172d] 2024-05-27T08:22:41.972 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 17: (Thread::_entry_func(void*)+0xd) [0x7f422bb01749] 2024-05-27T08:22:41.972 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f422a6a51ca] 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 19: clone() 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: -1> 2024-05-27T08:22:41.682+0000 7f4216000700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4216000700 time 2024-05-27T08:22:41.679386+0000 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f422bb32ba6] 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f422bb32dd1] 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560a21fca23d] 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 4: (SnapRealm::build_snap_set() const+0x323) [0x560a21fb5971] 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 5: (SnapRealm::check_cache() const+0x13e) [0x560a21fb3fa0] 2024-05-27T08:22:41.973 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560a21fb4651] 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560a21f49347] 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560a21c4587e] 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560a21c4ae94] 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x560a21c55969] 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 11: (MDLog::_replay_thread()+0x1f38) [0x560a21fee48c] 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 12: (MDLog::ReplayThread::entry()+0x11) [0x560a21ff79cb] 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 13: (Thread::entry_wrapper()+0x43) [0x7f422bb0172d] 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 14: (Thread::_entry_func(void*)+0xd) [0x7f422bb01749] 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f422a6a51ca] 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 16: clone() 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.974 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 0> 2024-05-27T08:22:41.686+0000 7f4216000700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: in thread 7f4216000700 thread_name:md_log_replay 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f422a6afd20] 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2: gsignal() 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 3: abort() 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f422bb32ced] 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f422bb32dd1] 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x560a21fca23d] 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 7: (SnapRealm::build_snap_set() const+0x323) [0x560a21fb5971] 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 8: (SnapRealm::check_cache() const+0x13e) [0x560a21fb3fa0] 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x560a21fb4651] 2024-05-27T08:22:41.975 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x560a21f49347] 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x560a21c4587e] 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x560a21c4ae94] 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x560a21c55969] 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 14: (MDLog::_replay_thread()+0x1f38) [0x560a21fee48c] 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 15: (MDLog::ReplayThread::entry()+0x11) [0x560a21ff79cb] 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 16: (Thread::entry_wrapper()+0x43) [0x7f422bb0172d] 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 17: (Thread::_entry_func(void*)+0xd) [0x7f422bb01749] 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f422a6a51ca] 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 19: clone() 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k[80748]: 2024-05-27T08:22:41.976 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:41 smithi086 podman[81701]: 2024-05-27 08:22:41.800542054 +0000 UTC m=+0.040384082 container died f90e5368958aeae14ab585893ba70f07b0a04d74a6d142f0d23aa143bd0b70bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=) 2024-05-27T08:22:42.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:41 smithi149 ceph-mon[35408]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:42.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:41 smithi149 ceph-mon[35408]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:42.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:41 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 3 up:standby 2024-05-27T08:22:42.214 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:41 smithi149 podman[83950]: 2024-05-27 08:22:41.943357361 +0000 UTC m=+0.454092776 container init 11e511c032a73ac37a8a65d6ac2f9dbf8a9f8c86cf59e63e72b0608e114a5a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-27T08:22:42.214 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:41 smithi149 podman[83950]: 2024-05-27 08:22:41.955352497 +0000 UTC m=+0.466087903 container start 11e511c032a73ac37a8a65d6ac2f9dbf8a9f8c86cf59e63e72b0608e114a5a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-27T08:22:42.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:42 smithi149 bash[83950]: 11e511c032a73ac37a8a65d6ac2f9dbf8a9f8c86cf59e63e72b0608e114a5a88 2024-05-27T08:22:42.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:42 smithi149 systemd[1]: Started Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:42.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:42 smithi149 ceph-mds[83968]: -- 172.21.15.149:0/3334994377 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555bef1b1a40 con 0x555bef1f9600 2024-05-27T08:22:42.215 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:22:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[83964]: starting mds.f at 2024-05-27T08:22:42.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-mon[31613]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:42.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-mon[31613]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T08:22:42.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 3 up:standby 2024-05-27T08:22:42.218 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f529a600700 time 2024-05-27T08:22:41.758894+0000 2024-05-27T08:22:42.218 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:42.218 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:42.218 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f52b0135ba6] 2024-05-27T08:22:42.218 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f52b0135dd1] 2024-05-27T08:22:42.218 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x563731d5b23d] 2024-05-27T08:22:42.218 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 4: (SnapRealm::build_snap_set() const+0x323) [0x563731d46971] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 5: (SnapRealm::check_cache() const+0x13e) [0x563731d44fa0] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x563731d45651] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x563731cda347] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5637319d687e] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5637319dbf2f] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5637319e6969] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 11: (MDLog::_replay_thread()+0x1f38) [0x563731d7f48c] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 12: (MDLog::ReplayThread::entry()+0x11) [0x563731d889cb] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 13: (Thread::entry_wrapper()+0x43) [0x7f52b010472d] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 14: (Thread::_entry_func(void*)+0xd) [0x7f52b0104749] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52aeca81ca] 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 16: clone() 2024-05-27T08:22:42.219 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: *** Caught signal (Aborted) ** 2024-05-27T08:22:42.220 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: in thread 7f529a600700 thread_name:md_log_replay 2024-05-27T08:22:42.220 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:41.763+0000 7f529a600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f529a600700 time 2024-05-27T08:22:41.758894+0000 2024-05-27T08:22:42.220 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:42.220 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.220 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:42.220 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f52b0135ba6] 2024-05-27T08:22:42.220 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f52b0135dd1] 2024-05-27T08:22:42.220 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x563731d5b23d] 2024-05-27T08:22:42.221 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 4: (SnapRealm::build_snap_set() const+0x323) [0x563731d46971] 2024-05-27T08:22:42.221 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 5: (SnapRealm::check_cache() const+0x13e) [0x563731d44fa0] 2024-05-27T08:22:42.221 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x563731d45651] 2024-05-27T08:22:42.221 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x563731cda347] 2024-05-27T08:22:42.221 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5637319d687e] 2024-05-27T08:22:42.221 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5637319dbf2f] 2024-05-27T08:22:42.221 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5637319e6969] 2024-05-27T08:22:42.221 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 11: (MDLog::_replay_thread()+0x1f38) [0x563731d7f48c] 2024-05-27T08:22:42.221 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 12: (MDLog::ReplayThread::entry()+0x11) [0x563731d889cb] 2024-05-27T08:22:42.221 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 13: (Thread::entry_wrapper()+0x43) [0x7f52b010472d] 2024-05-27T08:22:42.222 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 14: (Thread::_entry_func(void*)+0xd) [0x7f52b0104749] 2024-05-27T08:22:42.222 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52aeca81ca] 2024-05-27T08:22:42.222 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 16: clone() 2024-05-27T08:22:42.222 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.222 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:42.222 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52aecb2d20] 2024-05-27T08:22:42.222 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2: gsignal() 2024-05-27T08:22:42.222 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 3: abort() 2024-05-27T08:22:42.222 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f52b0135ced] 2024-05-27T08:22:42.223 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f52b0135dd1] 2024-05-27T08:22:42.223 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x563731d5b23d] 2024-05-27T08:22:42.223 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 7: (SnapRealm::build_snap_set() const+0x323) [0x563731d46971] 2024-05-27T08:22:42.223 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 8: (SnapRealm::check_cache() const+0x13e) [0x563731d44fa0] 2024-05-27T08:22:42.223 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x563731d45651] 2024-05-27T08:22:42.223 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x563731cda347] 2024-05-27T08:22:42.223 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5637319d687e] 2024-05-27T08:22:42.223 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5637319dbf2f] 2024-05-27T08:22:42.223 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5637319e6969] 2024-05-27T08:22:42.223 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 14: (MDLog::_replay_thread()+0x1f38) [0x563731d7f48c] 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 15: (MDLog::ReplayThread::entry()+0x11) [0x563731d889cb] 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 16: (Thread::entry_wrapper()+0x43) [0x7f52b010472d] 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 17: (Thread::_entry_func(void*)+0xd) [0x7f52b0104749] 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52aeca81ca] 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 19: clone() 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:41.767+0000 7f529a600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: in thread 7f529a600700 thread_name:md_log_replay 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52aecb2d20] 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2: gsignal() 2024-05-27T08:22:42.224 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 3: abort() 2024-05-27T08:22:42.225 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f52b0135ced] 2024-05-27T08:22:42.225 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f52b0135dd1] 2024-05-27T08:22:42.225 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x563731d5b23d] 2024-05-27T08:22:42.225 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 7: (SnapRealm::build_snap_set() const+0x323) [0x563731d46971] 2024-05-27T08:22:42.225 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 8: (SnapRealm::check_cache() const+0x13e) [0x563731d44fa0] 2024-05-27T08:22:42.225 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x563731d45651] 2024-05-27T08:22:42.225 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x563731cda347] 2024-05-27T08:22:42.225 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5637319d687e] 2024-05-27T08:22:42.225 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5637319dbf2f] 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5637319e6969] 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 14: (MDLog::_replay_thread()+0x1f38) [0x563731d7f48c] 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 15: (MDLog::ReplayThread::entry()+0x11) [0x563731d889cb] 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 16: (Thread::entry_wrapper()+0x43) [0x7f52b010472d] 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 17: (Thread::_entry_func(void*)+0xd) [0x7f52b0104749] 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52aeca81ca] 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 19: clone() 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: -1> 2024-05-27T08:22:41.763+0000 7f529a600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f529a600700 time 2024-05-27T08:22:41.758894+0000 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:42.226 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.227 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:42.227 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f52b0135ba6] 2024-05-27T08:22:42.227 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f52b0135dd1] 2024-05-27T08:22:42.227 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x563731d5b23d] 2024-05-27T08:22:42.227 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 4: (SnapRealm::build_snap_set() const+0x323) [0x563731d46971] 2024-05-27T08:22:42.227 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 5: (SnapRealm::check_cache() const+0x13e) [0x563731d44fa0] 2024-05-27T08:22:42.227 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x563731d45651] 2024-05-27T08:22:42.227 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x563731cda347] 2024-05-27T08:22:42.227 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5637319d687e] 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5637319dbf2f] 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5637319e6969] 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 11: (MDLog::_replay_thread()+0x1f38) [0x563731d7f48c] 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 12: (MDLog::ReplayThread::entry()+0x11) [0x563731d889cb] 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 13: (Thread::entry_wrapper()+0x43) [0x7f52b010472d] 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 14: (Thread::_entry_func(void*)+0xd) [0x7f52b0104749] 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52aeca81ca] 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 16: clone() 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 0> 2024-05-27T08:22:41.767+0000 7f529a600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:42.228 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: in thread 7f529a600700 thread_name:md_log_replay 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52aecb2d20] 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2: gsignal() 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 3: abort() 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f52b0135ced] 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f52b0135dd1] 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x563731d5b23d] 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 7: (SnapRealm::build_snap_set() const+0x323) [0x563731d46971] 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 8: (SnapRealm::check_cache() const+0x13e) [0x563731d44fa0] 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x563731d45651] 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x563731cda347] 2024-05-27T08:22:42.229 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5637319d687e] 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5637319dbf2f] 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5637319e6969] 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 14: (MDLog::_replay_thread()+0x1f38) [0x563731d7f48c] 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 15: (MDLog::ReplayThread::entry()+0x11) [0x563731d889cb] 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 16: (Thread::entry_wrapper()+0x43) [0x7f52b010472d] 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 17: (Thread::_entry_func(void*)+0xd) [0x7f52b0104749] 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52aeca81ca] 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 19: clone() 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: -1> 2024-05-27T08:22:41.763+0000 7f529a600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f529a600700 time 2024-05-27T08:22:41.758894+0000 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:42.230 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f52b0135ba6] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f52b0135dd1] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x563731d5b23d] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 4: (SnapRealm::build_snap_set() const+0x323) [0x563731d46971] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 5: (SnapRealm::check_cache() const+0x13e) [0x563731d44fa0] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x563731d45651] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x563731cda347] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5637319d687e] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5637319dbf2f] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5637319e6969] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 11: (MDLog::_replay_thread()+0x1f38) [0x563731d7f48c] 2024-05-27T08:22:42.231 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 12: (MDLog::ReplayThread::entry()+0x11) [0x563731d889cb] 2024-05-27T08:22:42.232 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 13: (Thread::entry_wrapper()+0x43) [0x7f52b010472d] 2024-05-27T08:22:42.232 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 14: (Thread::_entry_func(void*)+0xd) [0x7f52b0104749] 2024-05-27T08:22:42.232 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52aeca81ca] 2024-05-27T08:22:42.232 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 16: clone() 2024-05-27T08:22:42.232 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.232 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 0> 2024-05-27T08:22:41.767+0000 7f529a600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:42.232 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: in thread 7f529a600700 thread_name:md_log_replay 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52aecb2d20] 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2: gsignal() 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 3: abort() 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f52b0135ced] 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f52b0135dd1] 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x563731d5b23d] 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 7: (SnapRealm::build_snap_set() const+0x323) [0x563731d46971] 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 8: (SnapRealm::check_cache() const+0x13e) [0x563731d44fa0] 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x563731d45651] 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x563731cda347] 2024-05-27T08:22:42.233 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5637319d687e] 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5637319dbf2f] 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5637319e6969] 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 14: (MDLog::_replay_thread()+0x1f38) [0x563731d7f48c] 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 15: (MDLog::ReplayThread::entry()+0x11) [0x563731d889cb] 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 16: (Thread::entry_wrapper()+0x43) [0x7f52b010472d] 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 17: (Thread::_entry_func(void*)+0xd) [0x7f52b0104749] 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f52aeca81ca] 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 19: clone() 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a[189787]: 2024-05-27T08:22:42.234 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:41 smithi063 podman[190339]: 2024-05-27 08:22:41.87333448 +0000 UTC m=+0.041905952 container died fa90968cf27c92dbd58c3307c93dda80a2004deb070dbe06576e7b564056c17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-27T08:22:42.901 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:42 smithi063 podman[190339]: 2024-05-27 08:22:42.636654189 +0000 UTC m=+0.805225670 container remove fa90968cf27c92dbd58c3307c93dda80a2004deb070dbe06576e7b564056c17b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:22:42.901 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:42 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:22:42.901 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:42 smithi086 podman[81701]: 2024-05-27 08:22:42.556632722 +0000 UTC m=+0.796474785 container remove f90e5368958aeae14ab585893ba70f07b0a04d74a6d142f0d23aa143bd0b70bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521) 2024-05-27T08:22:42.902 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:42 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:22:43.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:42 smithi086 ceph-mon[34687]: pgmap v1575: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 0 op/s 2024-05-27T08:22:43.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:43 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-05-27T08:22:43.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:43 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Consumed 1.590s CPU time. 2024-05-27T08:22:43.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:42 smithi149 ceph-mon[35408]: pgmap v1575: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 0 op/s 2024-05-27T08:22:43.213 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:42 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 3. 2024-05-27T08:22:43.213 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:42 smithi149 systemd[1]: Stopped Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:43.214 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:42 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Consumed 1.425s CPU time. 2024-05-27T08:22:43.214 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:42 smithi149 systemd[1]: Starting Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:43.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:42 smithi063 ceph-mon[31613]: pgmap v1575: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 0 op/s 2024-05-27T08:22:43.678 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 podman[84036]: 2024-05-27 08:22:43.295217354 +0000 UTC m=+0.035268924 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:43.679 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 podman[84036]: 2024-05-27 08:22:43.407391294 +0000 UTC m=+0.147442910 container create f109759e1117a80c00f29f401ec87b58d1b47d0e1428a83e1b5115160889f6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-27T08:22:43.679 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 podman[84036]: 2024-05-27 08:22:43.677432295 +0000 UTC m=+0.417483885 container init f109759e1117a80c00f29f401ec87b58d1b47d0e1428a83e1b5115160889f6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-27T08:22:43.718 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:43 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Failed with result 'exit-code'. 2024-05-27T08:22:43.718 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:43 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Consumed 1.819s CPU time. 2024-05-27T08:22:43.933 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 podman[84036]: 2024-05-27 08:22:43.687870976 +0000 UTC m=+0.427922590 container start f109759e1117a80c00f29f401ec87b58d1b47d0e1428a83e1b5115160889f6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:22:43.934 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 bash[84036]: f109759e1117a80c00f29f401ec87b58d1b47d0e1428a83e1b5115160889f6f9 2024-05-27T08:22:43.934 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 systemd[1]: Started Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:43.934 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-mds[84054]: -- 172.21.15.149:0/2981298560 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef08eeba40 con 0x55ef08f33a80 2024-05-27T08:22:43.934 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: starting mds.l at 2024-05-27T08:22:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:43 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6832/1344757555,v1:172.21.15.149:6833/1344757555] up:boot 2024-05-27T08:22:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:43 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:43 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:22:44.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:43 smithi086 ceph-mon[34687]: daemon mds.l restarted 2024-05-27T08:22:44.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:43 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6832/1344757555,v1:172.21.15.149:6833/1344757555] up:boot 2024-05-27T08:22:44.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:43 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:44.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:43 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:22:44.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:43 smithi149 ceph-mon[35408]: daemon mds.l restarted 2024-05-27T08:22:44.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8a98600700 time 2024-05-27T08:22:43.983388+0000 2024-05-27T08:22:44.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:44.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:44.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8aae1feba6] 2024-05-27T08:22:44.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8aae1fedd1] 2024-05-27T08:22:44.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eefcbd623d] 2024-05-27T08:22:44.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55eefcbc1971] 2024-05-27T08:22:44.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 5: (SnapRealm::check_cache() const+0x13e) [0x55eefcbbffa0] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eefcbc0651] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eefcb55347] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eefc85187e] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55eefc856e94] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55eefc861969] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 11: (MDLog::_replay_thread()+0x1f38) [0x55eefcbfa48c] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55eefcc039cb] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 13: (Thread::entry_wrapper()+0x43) [0x7f8aae1cd72d] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8aae1cd749] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8aacd711ca] 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 16: clone() 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: *** Caught signal (Aborted) ** 2024-05-27T08:22:44.216 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: in thread 7f8a98600700 thread_name:md_log_replay 2024-05-27T08:22:44.217 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:43.985+0000 7f8a98600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8a98600700 time 2024-05-27T08:22:43.983388+0000 2024-05-27T08:22:44.217 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:44.217 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.217 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:44.217 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8aae1feba6] 2024-05-27T08:22:44.217 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8aae1fedd1] 2024-05-27T08:22:44.217 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eefcbd623d] 2024-05-27T08:22:44.217 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55eefcbc1971] 2024-05-27T08:22:44.217 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 5: (SnapRealm::check_cache() const+0x13e) [0x55eefcbbffa0] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eefcbc0651] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eefcb55347] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eefc85187e] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55eefc856e94] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55eefc861969] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 11: (MDLog::_replay_thread()+0x1f38) [0x55eefcbfa48c] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55eefcc039cb] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 13: (Thread::entry_wrapper()+0x43) [0x7f8aae1cd72d] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8aae1cd749] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8aacd711ca] 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 16: clone() 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.218 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:44.219 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8aacd7bd20] 2024-05-27T08:22:44.219 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2: gsignal() 2024-05-27T08:22:44.219 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 3: abort() 2024-05-27T08:22:44.219 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8aae1feced] 2024-05-27T08:22:44.219 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8aae1fedd1] 2024-05-27T08:22:44.219 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:43 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6832/1344757555,v1:172.21.15.149:6833/1344757555] up:boot 2024-05-27T08:22:44.219 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:43 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:44.219 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:43 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:43 smithi063 ceph-mon[31613]: daemon mds.l restarted 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eefcbd623d] 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55eefcbc1971] 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 8: (SnapRealm::check_cache() const+0x13e) [0x55eefcbbffa0] 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eefcbc0651] 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eefcb55347] 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eefc85187e] 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55eefc856e94] 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55eefc861969] 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 14: (MDLog::_replay_thread()+0x1f38) [0x55eefcbfa48c] 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55eefcc039cb] 2024-05-27T08:22:44.220 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 16: (Thread::entry_wrapper()+0x43) [0x7f8aae1cd72d] 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8aae1cd749] 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f8aacd711ca] 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 19: clone() 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:43.988+0000 7f8a98600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: in thread 7f8a98600700 thread_name:md_log_replay 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8aacd7bd20] 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2: gsignal() 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 3: abort() 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8aae1feced] 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8aae1fedd1] 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eefcbd623d] 2024-05-27T08:22:44.221 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55eefcbc1971] 2024-05-27T08:22:44.222 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 8: (SnapRealm::check_cache() const+0x13e) [0x55eefcbbffa0] 2024-05-27T08:22:44.222 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eefcbc0651] 2024-05-27T08:22:44.222 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eefcb55347] 2024-05-27T08:22:44.222 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eefc85187e] 2024-05-27T08:22:44.222 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55eefc856e94] 2024-05-27T08:22:44.222 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55eefc861969] 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 14: (MDLog::_replay_thread()+0x1f38) [0x55eefcbfa48c] 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55eefcc039cb] 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 16: (Thread::entry_wrapper()+0x43) [0x7f8aae1cd72d] 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8aae1cd749] 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f8aacd711ca] 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 19: clone() 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: -1> 2024-05-27T08:22:43.985+0000 7f8a98600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8a98600700 time 2024-05-27T08:22:43.983388+0000 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8aae1feba6] 2024-05-27T08:22:44.223 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8aae1fedd1] 2024-05-27T08:22:44.224 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eefcbd623d] 2024-05-27T08:22:44.224 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55eefcbc1971] 2024-05-27T08:22:44.224 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 5: (SnapRealm::check_cache() const+0x13e) [0x55eefcbbffa0] 2024-05-27T08:22:44.224 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eefcbc0651] 2024-05-27T08:22:44.224 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eefcb55347] 2024-05-27T08:22:44.224 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eefc85187e] 2024-05-27T08:22:44.224 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55eefc856e94] 2024-05-27T08:22:44.224 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55eefc861969] 2024-05-27T08:22:44.225 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 11: (MDLog::_replay_thread()+0x1f38) [0x55eefcbfa48c] 2024-05-27T08:22:44.225 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55eefcc039cb] 2024-05-27T08:22:44.225 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 13: (Thread::entry_wrapper()+0x43) [0x7f8aae1cd72d] 2024-05-27T08:22:44.225 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8aae1cd749] 2024-05-27T08:22:44.225 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8aacd711ca] 2024-05-27T08:22:44.225 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 16: clone() 2024-05-27T08:22:44.225 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.225 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 0> 2024-05-27T08:22:43.988+0000 7f8a98600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:44.225 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: in thread 7f8a98600700 thread_name:md_log_replay 2024-05-27T08:22:44.225 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.226 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:44.226 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8aacd7bd20] 2024-05-27T08:22:44.226 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2: gsignal() 2024-05-27T08:22:44.226 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 3: abort() 2024-05-27T08:22:44.226 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8aae1feced] 2024-05-27T08:22:44.226 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8aae1fedd1] 2024-05-27T08:22:44.226 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eefcbd623d] 2024-05-27T08:22:44.226 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55eefcbc1971] 2024-05-27T08:22:44.226 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 8: (SnapRealm::check_cache() const+0x13e) [0x55eefcbbffa0] 2024-05-27T08:22:44.226 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eefcbc0651] 2024-05-27T08:22:44.227 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eefcb55347] 2024-05-27T08:22:44.227 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eefc85187e] 2024-05-27T08:22:44.227 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55eefc856e94] 2024-05-27T08:22:44.227 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55eefc861969] 2024-05-27T08:22:44.227 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 14: (MDLog::_replay_thread()+0x1f38) [0x55eefcbfa48c] 2024-05-27T08:22:44.227 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55eefcc039cb] 2024-05-27T08:22:44.227 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 16: (Thread::entry_wrapper()+0x43) [0x7f8aae1cd72d] 2024-05-27T08:22:44.227 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8aae1cd749] 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f8aacd711ca] 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 19: clone() 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: -1> 2024-05-27T08:22:43.985+0000 7f8a98600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f8a98600700 time 2024-05-27T08:22:43.983388+0000 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f8aae1feba6] 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8aae1fedd1] 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eefcbd623d] 2024-05-27T08:22:44.228 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55eefcbc1971] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 5: (SnapRealm::check_cache() const+0x13e) [0x55eefcbbffa0] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eefcbc0651] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eefcb55347] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eefc85187e] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55eefc856e94] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55eefc861969] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 11: (MDLog::_replay_thread()+0x1f38) [0x55eefcbfa48c] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55eefcc039cb] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 13: (Thread::entry_wrapper()+0x43) [0x7f8aae1cd72d] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 14: (Thread::_entry_func(void*)+0xd) [0x7f8aae1cd749] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8aacd711ca] 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 16: clone() 2024-05-27T08:22:44.229 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 0> 2024-05-27T08:22:43.988+0000 7f8a98600700 -1 *** Caught signal (Aborted) ** 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: in thread 7f8a98600700 thread_name:md_log_replay 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8aacd7bd20] 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2: gsignal() 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 3: abort() 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f8aae1feced] 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f8aae1fedd1] 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eefcbd623d] 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55eefcbc1971] 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 8: (SnapRealm::check_cache() const+0x13e) [0x55eefcbbffa0] 2024-05-27T08:22:44.230 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eefcbc0651] 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eefcb55347] 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eefc85187e] 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55eefc856e94] 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55eefc861969] 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 14: (MDLog::_replay_thread()+0x1f38) [0x55eefcbfa48c] 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55eefcc039cb] 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 16: (Thread::entry_wrapper()+0x43) [0x7f8aae1cd72d] 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 17: (Thread::_entry_func(void*)+0xd) [0x7f8aae1cd749] 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f8aacd711ca] 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 19: clone() 2024-05-27T08:22:44.231 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T08:22:44.232 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l[84050]: 2024-05-27T08:22:44.232 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 podman[84085]: 2024-05-27 08:22:44.07309422 +0000 UTC m=+0.033955757 container died f109759e1117a80c00f29f401ec87b58d1b47d0e1428a83e1b5115160889f6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T08:22:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:44 smithi086 ceph-mon[34687]: pgmap v1576: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 1 op/s 2024-05-27T08:22:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:44 smithi086 ceph-mon[34687]: osdmap e193: 12 total, 12 up, 12 in 2024-05-27T08:22:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:44 smithi086 ceph-mon[34687]: mds.? [v2:172.21.15.149:6834/1929950952,v1:172.21.15.149:6835/1929950952] up:boot 2024-05-27T08:22:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:44 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 5 up:standby 2024-05-27T08:22:45.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:44 smithi086 ceph-mon[34687]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:22:45.211 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:44 smithi086 ceph-mon[34687]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:44 smithi149 ceph-mon[35408]: pgmap v1576: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 1 op/s 2024-05-27T08:22:45.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:44 smithi149 ceph-mon[35408]: osdmap e193: 12 total, 12 up, 12 in 2024-05-27T08:22:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:44 smithi149 ceph-mon[35408]: mds.? [v2:172.21.15.149:6834/1929950952,v1:172.21.15.149:6835/1929950952] up:boot 2024-05-27T08:22:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:44 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 5 up:standby 2024-05-27T08:22:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:44 smithi149 ceph-mon[35408]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:22:45.214 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:44 smithi149 ceph-mon[35408]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:45.214 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 podman[84085]: 2024-05-27 08:22:44.865441988 +0000 UTC m=+0.826303515 container remove f109759e1117a80c00f29f401ec87b58d1b47d0e1428a83e1b5115160889f6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5) 2024-05-27T08:22:45.215 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:44 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-05-27T08:22:45.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:44 smithi063 ceph-mon[31613]: pgmap v1576: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 1 op/s 2024-05-27T08:22:45.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:44 smithi063 ceph-mon[31613]: osdmap e193: 12 total, 12 up, 12 in 2024-05-27T08:22:45.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:44 smithi063 ceph-mon[31613]: mds.? [v2:172.21.15.149:6834/1929950952,v1:172.21.15.149:6835/1929950952] up:boot 2024-05-27T08:22:45.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:44 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 5 up:standby 2024-05-27T08:22:45.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:44 smithi063 ceph-mon[31613]: from='mgr.15216 172.21.15.63:0/859348958' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:22:45.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:44 smithi063 ceph-mon[31613]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 4 up:standby 2024-05-27T08:22:45.577 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T08:22:45.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:22:45.577 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-27T08:22:45.632 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:45 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-05-27T08:22:45.632 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:45 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Consumed 1.431s CPU time. 2024-05-27T08:22:45.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:22:45.668 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-27T08:22:45.791 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-27T08:22:45.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:22:45.792 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T08:22:45.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:22:45.894 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-27T08:22:46.014 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-27T08:22:46.026 INFO:tasks.cephadm:Teardown begin 2024-05-27T08:22:46.026 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/cephadm.py", line 2270, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi149 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump' 2024-05-27T08:22:46.028 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:22:46.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:22:46.226 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:22:46.320 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-27T08:22:46.320 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T08:22:46.382 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T08:22:46.446 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T08:22:46.513 INFO:tasks.cephadm:Stopping all daemons... 2024-05-27T08:22:46.514 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-27T08:22:46.514 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.a 2024-05-27T08:22:46.941 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:46 smithi063 systemd[1]: Stopping Ceph mon.a for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:47.210 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:46 smithi086 ceph-mon[34687]: pgmap v1578: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 716 B/s wr, 2 op/s 2024-05-27T08:22:47.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:46 smithi149 ceph-mon[35408]: pgmap v1578: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 716 B/s wr, 2 op/s 2024-05-27T08:22:47.216 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:46 smithi063 ceph-mon[31613]: pgmap v1578: 129 pgs: 129 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 716 B/s wr, 2 op/s 2024-05-27T08:22:47.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:47 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a[31609]: 2024-05-27T08:22:47.147+0000 7fcbd1400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:22:47.217 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:47 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a[31609]: 2024-05-27T08:22:47.147+0000 7fcbd1400700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-27T08:22:47.584 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:47 smithi063 podman[190540]: 2024-05-27 08:22:47.233199145 +0000 UTC m=+0.228587311 container died 07e5df3f996bf0093f4354423418f98e8af659f9fec288bce0b1e9d8dd875baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-27T08:22:47.966 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:47 smithi063 podman[190540]: 2024-05-27 08:22:47.800098937 +0000 UTC m=+0.795487103 container cleanup 07e5df3f996bf0093f4354423418f98e8af659f9fec288bce0b1e9d8dd875baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-27T08:22:47.967 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:47 smithi063 bash[190540]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a 2024-05-27T08:22:48.466 INFO:journalctl@ceph.mon.a.smithi063.stdout:May 27 08:22:48 smithi063 podman[190553]: 2024-05-27 08:22:48.026906731 +0000 UTC m=+0.780053666 container remove 07e5df3f996bf0093f4354423418f98e8af659f9fec288bce0b1e9d8dd875baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-27T08:22:48.603 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.a.service' 2024-05-27T08:22:48.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:22:48.738 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-27T08:22:48.738 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-27T08:22:48.738 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.b 2024-05-27T08:22:49.209 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:48 smithi086 systemd[1]: Stopping Ceph mon.b for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:49.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-b[34683]: 2024-05-27T08:22:49.302+0000 7f8096400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:22:49.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:49 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-b[34683]: 2024-05-27T08:22:49.302+0000 7f8096400700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-27T08:22:49.710 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:49 smithi086 podman[81824]: 2024-05-27 08:22:49.354134643 +0000 UTC m=+0.205019581 container died 4467d0c1d437a4c1ac627761ead9cde45b76e5d1bb99fc4a7c1252a1c9700c04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-b, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T08:22:50.106 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:49 smithi086 podman[81824]: 2024-05-27 08:22:49.85283972 +0000 UTC m=+0.703724622 container cleanup 4467d0c1d437a4c1ac627761ead9cde45b76e5d1bb99fc4a7c1252a1c9700c04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-27T08:22:50.107 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:49 smithi086 bash[81824]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-b 2024-05-27T08:22:50.460 INFO:journalctl@ceph.mon.b.smithi086.stdout:May 27 08:22:50 smithi086 podman[81837]: 2024-05-27 08:22:50.104256032 +0000 UTC m=+0.732666129 container remove 4467d0c1d437a4c1ac627761ead9cde45b76e5d1bb99fc4a7c1252a1c9700c04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:22:50.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.b.service' 2024-05-27T08:22:50.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:22:50.765 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-27T08:22:50.765 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-27T08:22:50.765 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.c 2024-05-27T08:22:51.213 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:50 smithi149 systemd[1]: Stopping Ceph mon.c for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:51.638 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-c[35401]: 2024-05-27T08:22:51.317+0000 7f3d8aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:22:51.639 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-c[35401]: 2024-05-27T08:22:51.317+0000 7f3d8aa00700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-27T08:22:51.964 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:51 smithi149 podman[84206]: 2024-05-27 08:22:51.637585192 +0000 UTC m=+0.465111185 container died f1a4fd65cc3f30fbec0745952818fbda3fe9dc5b4a890dba070ca479f8310bb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-27T08:22:52.463 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:52 smithi149 podman[84206]: 2024-05-27 08:22:52.185650934 +0000 UTC m=+1.013176923 container cleanup f1a4fd65cc3f30fbec0745952818fbda3fe9dc5b4a890dba070ca479f8310bb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-27T08:22:52.464 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:52 smithi149 bash[84206]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-c 2024-05-27T08:22:52.714 INFO:journalctl@ceph.mon.c.smithi149.stdout:May 27 08:22:52 smithi149 podman[84218]: 2024-05-27 08:22:52.462190982 +0000 UTC m=+0.792255287 container remove f1a4fd65cc3f30fbec0745952818fbda3fe9dc5b4a890dba070ca479f8310bb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-27T08:22:53.036 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mon.c.service' 2024-05-27T08:22:53.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:22:53.189 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-27T08:22:53.189 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-27T08:22:53.189 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.x 2024-05-27T08:22:53.596 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:53 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 4. 2024-05-27T08:22:53.596 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:53 smithi086 systemd[1]: Stopped Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:53.596 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:53 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Consumed 1.590s CPU time. 2024-05-27T08:22:53.596 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:53 smithi086 systemd[1]: Starting Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:53.717 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 08:22:53 smithi063 systemd[1]: Stopping Ceph mgr.x for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:53.718 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:53 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Scheduled restart job, restart counter is at 3. 2024-05-27T08:22:53.718 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:53 smithi063 systemd[1]: Stopped Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:53.718 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:53 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Consumed 1.819s CPU time. 2024-05-27T08:22:53.718 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:53 smithi063 systemd[1]: Starting Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:53.850 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:53 smithi086 podman[81949]: 2024-05-27 08:22:53.500064691 +0000 UTC m=+0.034234675 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:53.851 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:53 smithi086 podman[81949]: 2024-05-27 08:22:53.638692547 +0000 UTC m=+0.172862568 container create 7ab346de53f7bf842bcdc6a02cb1bd45bfa78710e9c195b422b84d962f8903b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:22:54.210 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:53 smithi086 podman[81949]: 2024-05-27 08:22:53.908887098 +0000 UTC m=+0.443057153 container init 7ab346de53f7bf842bcdc6a02cb1bd45bfa78710e9c195b422b84d962f8903b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-05-27T08:22:54.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:53 smithi086 podman[81949]: 2024-05-27 08:22:53.91961287 +0000 UTC m=+0.453782874 container start 7ab346de53f7bf842bcdc6a02cb1bd45bfa78710e9c195b422b84d962f8903b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-27T08:22:54.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:54 smithi086 bash[81949]: 7ab346de53f7bf842bcdc6a02cb1bd45bfa78710e9c195b422b84d962f8903b2 2024-05-27T08:22:54.211 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:22:54 smithi086 systemd[1]: Started Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:54.217 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 08:22:53 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T08:22:53.728+0000 7f868c200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:22:54.217 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 08:22:53 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x[31852]: 2024-05-27T08:22:53.728+0000 7f868c200700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-27T08:22:54.217 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 08:22:53 smithi063 podman[190673]: 2024-05-27 08:22:53.843588928 +0000 UTC m=+0.258927426 container died 806b8430f4eba2e34aa114f6aa324bd722553db93281543fa4b6127642aadb06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-27T08:22:54.574 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:54 smithi063 podman[190717]: 2024-05-27 08:22:54.372485484 +0000 UTC m=+0.524666890 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:54.959 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:54 smithi063 podman[190717]: 2024-05-27 08:22:54.75133748 +0000 UTC m=+0.903518879 container create 02b6c1c5f352855eb0acfa6217e5f48a72c3a0dc31dc70ec36b0feffbb82f2e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T08:22:54.959 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 08:22:54 smithi063 podman[190673]: 2024-05-27 08:22:54.573521306 +0000 UTC m=+0.988859831 container cleanup 806b8430f4eba2e34aa114f6aa324bd722553db93281543fa4b6127642aadb06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True) 2024-05-27T08:22:54.959 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 08:22:54 smithi063 bash[190673]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x 2024-05-27T08:22:55.466 INFO:journalctl@ceph.mgr.x.smithi063.stdout:May 27 08:22:55 smithi063 podman[190729]: 2024-05-27 08:22:55.239348146 +0000 UTC m=+1.352015496 container remove 806b8430f4eba2e34aa114f6aa324bd722553db93281543fa4b6127642aadb06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-27T08:22:55.467 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:55 smithi063 podman[190717]: 2024-05-27 08:22:55.13214696 +0000 UTC m=+1.284328403 container init 02b6c1c5f352855eb0acfa6217e5f48a72c3a0dc31dc70ec36b0feffbb82f2e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-27T08:22:55.467 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:55 smithi063 podman[190717]: 2024-05-27 08:22:55.143935569 +0000 UTC m=+1.296116975 container start 02b6c1c5f352855eb0acfa6217e5f48a72c3a0dc31dc70ec36b0feffbb82f2e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:22:55.467 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:55 smithi063 bash[190717]: 02b6c1c5f352855eb0acfa6217e5f48a72c3a0dc31dc70ec36b0feffbb82f2e2 2024-05-27T08:22:55.467 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:22:55 smithi063 systemd[1]: Started Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:55.645 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.x.service' 2024-05-27T08:22:55.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:22:55.759 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-27T08:22:55.759 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-27T08:22:55.759 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.y 2024-05-27T08:22:55.876 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:55 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 4. 2024-05-27T08:22:55.876 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:55 smithi149 systemd[1]: Stopped Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:55.876 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:55 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Consumed 1.431s CPU time. 2024-05-27T08:22:55.876 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:55 smithi149 systemd[1]: Starting Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:56.172 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:55 smithi149 podman[84327]: 2024-05-27 08:22:55.781986076 +0000 UTC m=+0.036591062 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:22:56.172 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:55 smithi149 podman[84327]: 2024-05-27 08:22:55.897329758 +0000 UTC m=+0.151934769 container create c52aeff647c50069e040af1c1a1c2522293ddfff62c69ea10af58124d1a34087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-27T08:22:56.172 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:56 smithi149 podman[84327]: 2024-05-27 08:22:56.167911613 +0000 UTC m=+0.422516634 container init c52aeff647c50069e040af1c1a1c2522293ddfff62c69ea10af58124d1a34087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-27T08:22:56.209 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 08:22:55 smithi086 systemd[1]: Stopping Ceph mgr.y for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:56.464 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:56 smithi149 podman[84327]: 2024-05-27 08:22:56.17863675 +0000 UTC m=+0.433241731 container start c52aeff647c50069e040af1c1a1c2522293ddfff62c69ea10af58124d1a34087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-27T08:22:56.464 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:56 smithi149 bash[84327]: c52aeff647c50069e040af1c1a1c2522293ddfff62c69ea10af58124d1a34087 2024-05-27T08:22:56.464 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:22:56 smithi149 systemd[1]: Started Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:22:56.710 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 08:22:56 smithi086 podman[82034]: 2024-05-27 08:22:56.452083234 +0000 UTC m=+0.269633204 container died df98b87f3baf9416f705d97a779f493a68373cf765c3865be68182c5c3f883d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:22:57.140 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 08:22:56 smithi086 podman[82034]: 2024-05-27 08:22:56.929197998 +0000 UTC m=+0.746747905 container cleanup df98b87f3baf9416f705d97a779f493a68373cf765c3865be68182c5c3f883d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-27T08:22:57.141 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 08:22:56 smithi086 bash[82034]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y 2024-05-27T08:22:57.459 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 08:22:57 smithi086 podman[82046]: 2024-05-27 08:22:57.139037452 +0000 UTC m=+0.667146123 container remove df98b87f3baf9416f705d97a779f493a68373cf765c3865be68182c5c3f883d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-27T08:22:57.460 INFO:journalctl@ceph.mgr.y.smithi086.stdout:May 27 08:22:57 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-27T08:22:57.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.y.service' 2024-05-27T08:22:57.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:22:57.780 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-27T08:22:57.780 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-27T08:22:57.780 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.z 2024-05-27T08:22:58.213 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 08:22:57 smithi149 systemd[1]: Stopping Ceph mgr.z for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:22:58.714 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 08:22:58 smithi149 podman[84402]: 2024-05-27 08:22:58.45399954 +0000 UTC m=+0.246074346 container died d58d6f862e235a38c820f7da4fea0c038fb789beb929beff261e3d24ae1a62a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T08:22:59.175 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 08:22:58 smithi149 podman[84402]: 2024-05-27 08:22:58.928562149 +0000 UTC m=+0.720636952 container cleanup d58d6f862e235a38c820f7da4fea0c038fb789beb929beff261e3d24ae1a62a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-27T08:22:59.175 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 08:22:58 smithi149 bash[84402]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z 2024-05-27T08:22:59.464 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 08:22:59 smithi149 podman[84413]: 2024-05-27 08:22:59.173155555 +0000 UTC m=+0.687789561 container remove d58d6f862e235a38c820f7da4fea0c038fb789beb929beff261e3d24ae1a62a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:22:59.464 INFO:journalctl@ceph.mgr.z.smithi149.stdout:May 27 08:22:59 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-27T08:22:59.668 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mgr.z.service' 2024-05-27T08:22:59.782 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:22:59.782 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-27T08:22:59.783 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-27T08:22:59.783 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.0 2024-05-27T08:23:00.216 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:22:59 smithi063 systemd[1]: Stopping Ceph osd.0 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:23:00.717 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:00 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0[46409]: 2024-05-27T08:23:00.364+0000 7f725de00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:23:00.717 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:00 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0[46409]: 2024-05-27T08:23:00.364+0000 7f725de00700 -1 osd.0 193 *** Got signal Terminated *** 2024-05-27T08:23:00.717 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:00 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0[46409]: 2024-05-27T08:23:00.364+0000 7f725de00700 -1 osd.0 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:23:06.623 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:06 smithi063 podman[190854]: 2024-05-27 08:23:06.315490834 +0000 UTC m=+6.105482817 container died 4b6ab480bf5287ca595c7207e13e03e6410e8a64484c9fb6f07ab078443b2140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T08:23:06.967 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:06 smithi063 podman[190854]: 2024-05-27 08:23:06.825819428 +0000 UTC m=+6.615811401 container cleanup 4b6ab480bf5287ca595c7207e13e03e6410e8a64484c9fb6f07ab078443b2140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T08:23:06.967 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:06 smithi063 bash[190854]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0 2024-05-27T08:23:07.466 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:07 smithi063 podman[190873]: 2024-05-27 08:23:07.135491244 +0000 UTC m=+0.811883469 container remove 4b6ab480bf5287ca595c7207e13e03e6410e8a64484c9fb6f07ab078443b2140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:23:08.383 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:08 smithi063 podman[190963]: 2024-05-27 08:23:07.928661512 +0000 UTC m=+0.038316781 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:23:08.383 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:08 smithi063 podman[190963]: 2024-05-27 08:23:08.053139303 +0000 UTC m=+0.162794629 container create afdde110c5072006a676105f5b85f57bc18c3a426432318ad29ed45fca87f0d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-27T08:23:08.641 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:08 smithi063 podman[190963]: 2024-05-27 08:23:08.38220427 +0000 UTC m=+0.491859582 container init afdde110c5072006a676105f5b85f57bc18c3a426432318ad29ed45fca87f0d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T08:23:08.641 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:08 smithi063 podman[190963]: 2024-05-27 08:23:08.392955778 +0000 UTC m=+0.502611055 container start afdde110c5072006a676105f5b85f57bc18c3a426432318ad29ed45fca87f0d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:23:08.641 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:08 smithi063 podman[190963]: 2024-05-27 08:23:08.514076743 +0000 UTC m=+0.623732049 container attach afdde110c5072006a676105f5b85f57bc18c3a426432318ad29ed45fca87f0d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-27T08:23:10.716 INFO:journalctl@ceph.osd.0.smithi063.stdout:May 27 08:23:10 smithi063 podman[191001]: 2024-05-27 08:23:10.51204109 +0000 UTC m=+0.039419625 container died afdde110c5072006a676105f5b85f57bc18c3a426432318ad29ed45fca87f0d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:23:11.356 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.0.service' 2024-05-27T08:23:11.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:23:11.470 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-27T08:23:11.471 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-27T08:23:11.471 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.1 2024-05-27T08:23:11.959 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:11 smithi086 systemd[1]: Stopping Ceph osd.1 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:23:12.461 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:12 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1[39249]: 2024-05-27T08:23:12.047+0000 7f6f63e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:23:12.461 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:12 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1[39249]: 2024-05-27T08:23:12.047+0000 7f6f63e00700 -1 osd.1 193 *** Got signal Terminated *** 2024-05-27T08:23:12.461 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:12 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1[39249]: 2024-05-27T08:23:12.047+0000 7f6f63e00700 -1 osd.1 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:23:18.210 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:17 smithi086 podman[82156]: 2024-05-27 08:23:17.929333033 +0000 UTC m=+6.027604742 container died 6b3cc3ffde2197a0adaa9cebde0ee4a26c9d3d7cab4ee9ad94eaca5b1d73517b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:23:18.678 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:18 smithi086 podman[82156]: 2024-05-27 08:23:18.367433215 +0000 UTC m=+6.465704914 container cleanup 6b3cc3ffde2197a0adaa9cebde0ee4a26c9d3d7cab4ee9ad94eaca5b1d73517b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-27T08:23:18.678 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:18 smithi086 bash[82156]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1 2024-05-27T08:23:18.959 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:18 smithi086 podman[82170]: 2024-05-27 08:23:18.676860159 +0000 UTC m=+0.732676925 container remove 6b3cc3ffde2197a0adaa9cebde0ee4a26c9d3d7cab4ee9ad94eaca5b1d73517b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-27T08:23:19.702 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:19 smithi086 podman[82262]: 2024-05-27 08:23:19.346120162 +0000 UTC m=+0.035486442 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:23:19.703 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:19 smithi086 podman[82262]: 2024-05-27 08:23:19.528352883 +0000 UTC m=+0.217719189 container create d69a16deb4252f6d2329e0e81310858fef55a55d3ec09be04aa524b19cc426e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:23:20.210 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:19 smithi086 podman[82262]: 2024-05-27 08:23:19.848841265 +0000 UTC m=+0.538207611 container init d69a16deb4252f6d2329e0e81310858fef55a55d3ec09be04aa524b19cc426e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T08:23:20.210 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:19 smithi086 podman[82262]: 2024-05-27 08:23:19.859880634 +0000 UTC m=+0.549246947 container start d69a16deb4252f6d2329e0e81310858fef55a55d3ec09be04aa524b19cc426e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2) 2024-05-27T08:23:20.210 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:19 smithi086 podman[82262]: 2024-05-27 08:23:19.972814767 +0000 UTC m=+0.662181117 container attach d69a16deb4252f6d2329e0e81310858fef55a55d3ec09be04aa524b19cc426e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-27T08:23:20.833 INFO:journalctl@ceph.osd.1.smithi086.stdout:May 27 08:23:20 smithi086 podman[82262]: 2024-05-27 08:23:20.481162675 +0000 UTC m=+1.170529014 container died d69a16deb4252f6d2329e0e81310858fef55a55d3ec09be04aa524b19cc426e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-27T08:23:21.236 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.1.service' 2024-05-27T08:23:21.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:23:21.333 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-27T08:23:21.333 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-27T08:23:21.333 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.2 2024-05-27T08:23:21.714 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:21 smithi149 systemd[1]: Stopping Ceph osd.2 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:23:22.214 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2[40692]: 2024-05-27T08:23:21.939+0000 7f0cbe000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:23:22.214 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2[40692]: 2024-05-27T08:23:21.939+0000 7f0cbe000700 -1 osd.2 193 *** Got signal Terminated *** 2024-05-27T08:23:22.214 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2[40692]: 2024-05-27T08:23:21.939+0000 7f0cbe000700 -1 osd.2 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:23:28.157 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:27 smithi149 podman[84549]: 2024-05-27 08:23:27.843323328 +0000 UTC m=+6.051829766 container died bee2e341bc931020275ff569ede1d768841553d5a5686ed7e6b68144b859ce63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T08:23:28.464 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:28 smithi149 podman[84549]: 2024-05-27 08:23:28.30795277 +0000 UTC m=+6.516459231 container cleanup bee2e341bc931020275ff569ede1d768841553d5a5686ed7e6b68144b859ce63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:23:28.464 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:28 smithi149 bash[84549]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2 2024-05-27T08:23:28.964 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:28 smithi149 podman[84566]: 2024-05-27 08:23:28.584620854 +0000 UTC m=+0.706657742 container remove bee2e341bc931020275ff569ede1d768841553d5a5686ed7e6b68144b859ce63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-27T08:23:29.714 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:29 smithi149 podman[84654]: 2024-05-27 08:23:29.357958896 +0000 UTC m=+0.036070623 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:23:29.714 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:29 smithi149 podman[84654]: 2024-05-27 08:23:29.502196642 +0000 UTC m=+0.180308406 container create e6d8932af83809e0e0208c276842f74d5b409258bc97c79edab68f6d89056fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:23:30.214 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:29 smithi149 podman[84654]: 2024-05-27 08:23:29.799505368 +0000 UTC m=+0.477617169 container init e6d8932af83809e0e0208c276842f74d5b409258bc97c79edab68f6d89056fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:23:30.214 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:29 smithi149 podman[84654]: 2024-05-27 08:23:29.811789549 +0000 UTC m=+0.489901294 container start e6d8932af83809e0e0208c276842f74d5b409258bc97c79edab68f6d89056fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:23:30.214 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:29 smithi149 podman[84654]: 2024-05-27 08:23:29.946629052 +0000 UTC m=+0.624740826 container attach e6d8932af83809e0e0208c276842f74d5b409258bc97c79edab68f6d89056fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:23:30.964 INFO:journalctl@ceph.osd.2.smithi149.stdout:May 27 08:23:30 smithi149 podman[84654]: 2024-05-27 08:23:30.64779605 +0000 UTC m=+1.325907821 container died e6d8932af83809e0e0208c276842f74d5b409258bc97c79edab68f6d89056fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:23:31.405 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.2.service' 2024-05-27T08:23:31.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:23:31.518 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-27T08:23:31.518 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-27T08:23:31.518 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.3 2024-05-27T08:23:31.971 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:31 smithi063 systemd[1]: Stopping Ceph osd.3 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:23:32.467 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:32 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3[53040]: 2024-05-27T08:23:32.136+0000 7f7ff3600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:23:32.467 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:32 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3[53040]: 2024-05-27T08:23:32.136+0000 7f7ff3600700 -1 osd.3 193 *** Got signal Terminated *** 2024-05-27T08:23:32.467 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:32 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3[53040]: 2024-05-27T08:23:32.136+0000 7f7ff3600700 -1 osd.3 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:23:38.615 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:38 smithi063 podman[191080]: 2024-05-27 08:23:38.304754397 +0000 UTC m=+6.315330264 container died ae5a25c3c5c7d9038652f593e62e7905e5d09f25e58b33c468182e52046e0cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-05-27T08:23:38.967 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:38 smithi063 podman[191080]: 2024-05-27 08:23:38.794809789 +0000 UTC m=+6.805385652 container cleanup ae5a25c3c5c7d9038652f593e62e7905e5d09f25e58b33c468182e52046e0cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-27T08:23:38.967 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:38 smithi063 bash[191080]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3 2024-05-27T08:23:39.466 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:39 smithi063 podman[191096]: 2024-05-27 08:23:39.005113181 +0000 UTC m=+0.677394998 container remove ae5a25c3c5c7d9038652f593e62e7905e5d09f25e58b33c468182e52046e0cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T08:23:40.216 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:39 smithi063 podman[191183]: 2024-05-27 08:23:39.804968622 +0000 UTC m=+0.036936812 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:23:40.217 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:39 smithi063 podman[191183]: 2024-05-27 08:23:39.930916302 +0000 UTC m=+0.162884517 container create 32f656c7433ac742a170a4f593fb62ea9fff2a5d21cc520c0483a5e99b1f1e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux ) 2024-05-27T08:23:40.716 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:40 smithi063 podman[191183]: 2024-05-27 08:23:40.242969694 +0000 UTC m=+0.474937972 container init 32f656c7433ac742a170a4f593fb62ea9fff2a5d21cc520c0483a5e99b1f1e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:23:40.717 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:40 smithi063 podman[191183]: 2024-05-27 08:23:40.255264087 +0000 UTC m=+0.487232274 container start 32f656c7433ac742a170a4f593fb62ea9fff2a5d21cc520c0483a5e99b1f1e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-27T08:23:40.717 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:40 smithi063 podman[191183]: 2024-05-27 08:23:40.349765675 +0000 UTC m=+0.581733897 container attach 32f656c7433ac742a170a4f593fb62ea9fff2a5d21cc520c0483a5e99b1f1e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:23:41.065 INFO:journalctl@ceph.osd.3.smithi063.stdout:May 27 08:23:40 smithi063 podman[191183]: 2024-05-27 08:23:40.732062392 +0000 UTC m=+0.964030581 container died 32f656c7433ac742a170a4f593fb62ea9fff2a5d21cc520c0483a5e99b1f1e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:23:41.499 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.3.service' 2024-05-27T08:23:41.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:23:41.603 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-27T08:23:41.603 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-27T08:23:41.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.4 2024-05-27T08:23:41.960 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:41 smithi086 systemd[1]: Stopping Ceph osd.4 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:23:42.460 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:42 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4[46232]: 2024-05-27T08:23:42.202+0000 7f42bca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:23:42.460 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:42 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4[46232]: 2024-05-27T08:23:42.202+0000 7f42bca00700 -1 osd.4 193 *** Got signal Terminated *** 2024-05-27T08:23:42.460 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:42 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4[46232]: 2024-05-27T08:23:42.202+0000 7f42bca00700 -1 osd.4 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:23:48.460 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:48 smithi086 podman[82392]: 2024-05-27 08:23:48.144404721 +0000 UTC m=+6.088743695 container stop 27054a3ed759716ed6abac4717c128fb5f43fee2cb2551609121246172d5e192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-27T08:23:48.460 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:48 smithi086 podman[82392]: 2024-05-27 08:23:48.145425612 +0000 UTC m=+6.089764586 container died 27054a3ed759716ed6abac4717c128fb5f43fee2cb2551609121246172d5e192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T08:23:48.867 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:48 smithi086 podman[82392]: 2024-05-27 08:23:48.61389195 +0000 UTC m=+6.558230961 container cleanup 27054a3ed759716ed6abac4717c128fb5f43fee2cb2551609121246172d5e192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-27T08:23:48.867 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:48 smithi086 bash[82392]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4 2024-05-27T08:23:49.209 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:48 smithi086 podman[82411]: 2024-05-27 08:23:48.865209183 +0000 UTC m=+0.698987559 container remove 27054a3ed759716ed6abac4717c128fb5f43fee2cb2551609121246172d5e192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:23:50.063 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:49 smithi086 podman[82501]: 2024-05-27 08:23:49.619657113 +0000 UTC m=+0.032203704 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:23:50.063 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:49 smithi086 podman[82501]: 2024-05-27 08:23:49.749871839 +0000 UTC m=+0.162418460 container create c5594835d3f4cb7d1009f3235d2631bad247cc947d8840f2af075cc7b22d9033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T08:23:50.064 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:50 smithi086 podman[82501]: 2024-05-27 08:23:50.062110421 +0000 UTC m=+0.474657072 container init c5594835d3f4cb7d1009f3235d2631bad247cc947d8840f2af075cc7b22d9033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-27T08:23:50.460 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:50 smithi086 podman[82501]: 2024-05-27 08:23:50.07321602 +0000 UTC m=+0.485762653 container start c5594835d3f4cb7d1009f3235d2631bad247cc947d8840f2af075cc7b22d9033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:23:50.460 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:50 smithi086 podman[82501]: 2024-05-27 08:23:50.210888882 +0000 UTC m=+0.623435569 container attach c5594835d3f4cb7d1009f3235d2631bad247cc947d8840f2af075cc7b22d9033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:23:50.876 INFO:journalctl@ceph.osd.4.smithi086.stdout:May 27 08:23:50 smithi086 podman[82501]: 2024-05-27 08:23:50.52220183 +0000 UTC m=+0.934748470 container died c5594835d3f4cb7d1009f3235d2631bad247cc947d8840f2af075cc7b22d9033 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T08:23:51.352 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.4.service' 2024-05-27T08:23:51.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:23:51.456 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-27T08:23:51.456 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-27T08:23:51.456 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.5 2024-05-27T08:23:51.964 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:51 smithi149 systemd[1]: Stopping Ceph osd.5 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:23:52.464 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:52 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5[47485]: 2024-05-27T08:23:52.001+0000 7fdae7e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:23:52.464 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:52 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5[47485]: 2024-05-27T08:23:52.001+0000 7fdae7e00700 -1 osd.5 193 *** Got signal Terminated *** 2024-05-27T08:23:52.464 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:52 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5[47485]: 2024-05-27T08:23:52.001+0000 7fdae7e00700 -1 osd.5 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:23:58.350 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:58 smithi149 podman[84773]: 2024-05-27 08:23:58.031957789 +0000 UTC m=+6.174411846 container died 9a453d27138bd3c8035cdd11b2c7f742985d8bea2e72f73a6a74f2b4bc46edef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-27T08:23:58.714 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:58 smithi149 podman[84773]: 2024-05-27 08:23:58.486059652 +0000 UTC m=+6.628513767 container cleanup 9a453d27138bd3c8035cdd11b2c7f742985d8bea2e72f73a6a74f2b4bc46edef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:23:58.714 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:58 smithi149 bash[84773]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5 2024-05-27T08:23:59.214 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:58 smithi149 podman[84790]: 2024-05-27 08:23:58.770954823 +0000 UTC m=+0.703458513 container remove 9a453d27138bd3c8035cdd11b2c7f742985d8bea2e72f73a6a74f2b4bc46edef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:23:59.931 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:59 smithi149 podman[84876]: 2024-05-27 08:23:59.48015522 +0000 UTC m=+0.037090481 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:23:59.931 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:59 smithi149 podman[84876]: 2024-05-27 08:23:59.60721348 +0000 UTC m=+0.164148772 container create d1cd831f85174d0a611bcef45defe7006997434ac0e795afd8c62428187c0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T08:24:00.214 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:59 smithi149 podman[84876]: 2024-05-27 08:23:59.927773386 +0000 UTC m=+0.484708722 container init d1cd831f85174d0a611bcef45defe7006997434ac0e795afd8c62428187c0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:24:00.214 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:23:59 smithi149 podman[84876]: 2024-05-27 08:23:59.941554241 +0000 UTC m=+0.498489570 container start d1cd831f85174d0a611bcef45defe7006997434ac0e795afd8c62428187c0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:24:00.214 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:24:00 smithi149 podman[84876]: 2024-05-27 08:24:00.034905822 +0000 UTC m=+0.591841161 container attach d1cd831f85174d0a611bcef45defe7006997434ac0e795afd8c62428187c0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T08:24:00.714 INFO:journalctl@ceph.osd.5.smithi149.stdout:May 27 08:24:00 smithi149 podman[84876]: 2024-05-27 08:24:00.39414404 +0000 UTC m=+0.951079326 container died d1cd831f85174d0a611bcef45defe7006997434ac0e795afd8c62428187c0df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:24:01.183 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.5.service' 2024-05-27T08:24:01.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:24:01.294 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-27T08:24:01.294 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-27T08:24:01.294 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.6 2024-05-27T08:24:01.716 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:01 smithi063 systemd[1]: Stopping Ceph osd.6 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:24:02.218 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6[59959]: 2024-05-27T08:24:01.871+0000 7f8ba0e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:24:02.218 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6[59959]: 2024-05-27T08:24:01.871+0000 7f8ba0e00700 -1 osd.6 193 *** Got signal Terminated *** 2024-05-27T08:24:02.218 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:01 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6[59959]: 2024-05-27T08:24:01.871+0000 7f8ba0e00700 -1 osd.6 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:24:07.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:06 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:06.936+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:08.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:07 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:07.941+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:08.253 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:07 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:07.778+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:08.253 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:07 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:07.960+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:08.409 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:08 smithi063 podman[191301]: 2024-05-27 08:24:08.081858313 +0000 UTC m=+6.354789344 container died 7f531da015bbe6bfb61ef6127b954951661a97920c4ae0e6bb3b6632b46b85cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-27T08:24:08.410 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:08 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:08.289+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:08.660 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:08 smithi063 podman[191301]: 2024-05-27 08:24:08.62176205 +0000 UTC m=+6.894693051 container cleanup 7f531da015bbe6bfb61ef6127b954951661a97920c4ae0e6bb3b6632b46b85cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521) 2024-05-27T08:24:08.660 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:08 smithi063 bash[191301]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6 2024-05-27T08:24:09.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:08 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:08.932+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:09.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:08 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:08.738+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:09.216 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:08 smithi063 podman[191321]: 2024-05-27 08:24:08.78982754 +0000 UTC m=+0.685733037 container remove 7f531da015bbe6bfb61ef6127b954951661a97920c4ae0e6bb3b6632b46b85cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T08:24:09.459 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:08 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:08.980+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:09.603 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:09 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:09.301+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:09.887 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:09 smithi063 podman[191408]: 2024-05-27 08:24:09.507540595 +0000 UTC m=+0.036685395 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:24:09.887 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:09 smithi063 podman[191408]: 2024-05-27 08:24:09.615548788 +0000 UTC m=+0.144693636 container create 023eee90617f105b0dd1788b1da2041f659a2ffa2435bf509887e652ee58f30b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:24:09.887 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:09 smithi063 podman[191408]: 2024-05-27 08:24:09.885829441 +0000 UTC m=+0.414974328 container init 023eee90617f105b0dd1788b1da2041f659a2ffa2435bf509887e652ee58f30b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T08:24:10.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:09 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:09.957+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:10.213 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:09 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:09.919+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:10.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:09 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:09.753+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:10.216 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:09 smithi063 podman[191408]: 2024-05-27 08:24:09.897979089 +0000 UTC m=+0.427123918 container start 023eee90617f105b0dd1788b1da2041f659a2ffa2435bf509887e652ee58f30b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-27T08:24:10.216 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:10 smithi063 podman[191408]: 2024-05-27 08:24:10.043048761 +0000 UTC m=+0.572193636 container attach 023eee90617f105b0dd1788b1da2041f659a2ffa2435bf509887e652ee58f30b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-27T08:24:10.693 INFO:journalctl@ceph.osd.6.smithi063.stdout:May 27 08:24:10 smithi063 podman[191433]: 2024-05-27 08:24:10.435118044 +0000 UTC m=+0.042463461 container died 023eee90617f105b0dd1788b1da2041f659a2ffa2435bf509887e652ee58f30b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-27T08:24:10.693 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:10 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:10.278+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:11.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:10 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:10.936+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:11.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:10 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:10.779+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:11.247 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.6.service' 2024-05-27T08:24:11.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:24:11.353 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-27T08:24:11.353 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-27T08:24:11.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.7 2024-05-27T08:24:11.432 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:10 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:10.996+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:11.466 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:11.319+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:11.710 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:11 smithi086 systemd[1]: Stopping Ceph osd.7 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:24:12.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:12 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:12.044+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:12.210 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:11 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7[52856]: 2024-05-27T08:24:11.949+0000 7fbb34200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:24:12.210 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:11 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7[52856]: 2024-05-27T08:24:11.949+0000 7fbb34200700 -1 osd.7 193 *** Got signal Terminated *** 2024-05-27T08:24:12.211 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:11 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7[52856]: 2024-05-27T08:24:11.949+0000 7fbb34200700 -1 osd.7 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:24:12.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:11 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:11.928+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:12.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:11 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:11.800+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:12.467 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:12 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:12.291+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:13.116 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:12.892+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:13.116 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:12 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:12.830+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:13.462 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:13 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:13.043+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:13.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:13 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:13.295+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:14.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:13 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:13.874+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:14.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:13 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:13.868+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:14.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:14 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:14.075+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:14.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:14 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:14.268+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:15.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:14 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:14.881+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:15.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:14 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:14.870+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:15.459 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:15 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:15.049+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:15.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:15 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:15.280+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:16.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:15.869+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:16.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:15 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:15.895+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:16.459 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:16 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:16.046+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:16.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:16 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:16.241+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:17.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:16 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:16.861+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:17.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:16 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:16.853+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:17.459 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:17.062+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:17.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:17 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:17.240+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:18.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:17 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:17.888+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:18.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:17 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:17.816+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:18.449 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:18 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:18.029+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:18.449 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:17 smithi086 podman[82624]: 2024-05-27 08:24:17.990156889 +0000 UTC m=+6.179414288 container died 6062f7dcbed32b1f6f83ed2314bb2df048bbab38f51f813b2706efaa522b88cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T08:24:18.710 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:18 smithi086 podman[82624]: 2024-05-27 08:24:18.583550904 +0000 UTC m=+6.772808266 container cleanup 6062f7dcbed32b1f6f83ed2314bb2df048bbab38f51f813b2706efaa522b88cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-05-27T08:24:18.710 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:18 smithi086 bash[82624]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7 2024-05-27T08:24:18.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:18 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:18.226+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:18.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:18 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:18.226+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:19.054 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:18 smithi086 podman[82643]: 2024-05-27 08:24:18.884963921 +0000 UTC m=+0.881557579 container remove 6062f7dcbed32b1f6f83ed2314bb2df048bbab38f51f813b2706efaa522b88cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-05-27T08:24:19.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:18 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:18.922+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:19.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:18 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:18.835+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:19.459 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:19 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:19.052+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:19.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:19 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:19.223+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:19.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:19 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:19.223+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:20.030 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:19 smithi086 podman[82729]: 2024-05-27 08:24:19.618430042 +0000 UTC m=+0.032036866 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:24:20.030 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:19 smithi086 podman[82729]: 2024-05-27 08:24:19.752727601 +0000 UTC m=+0.166334454 container create c32dd59ff3521053a8ffa404b945c6356ddac26b939a8ac5a93b0cf3d5bf6525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-27T08:24:20.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:19 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:19.835+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:20.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:19 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:19.949+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:20.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:20 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:20.028+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:20.460 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:20 smithi086 podman[82729]: 2024-05-27 08:24:20.067796993 +0000 UTC m=+0.481403909 container init c32dd59ff3521053a8ffa404b945c6356ddac26b939a8ac5a93b0cf3d5bf6525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:24:20.460 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:20 smithi086 podman[82729]: 2024-05-27 08:24:20.078535639 +0000 UTC m=+0.492142481 container start c32dd59ff3521053a8ffa404b945c6356ddac26b939a8ac5a93b0cf3d5bf6525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T08:24:20.461 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:20 smithi086 podman[82729]: 2024-05-27 08:24:20.197023882 +0000 UTC m=+0.610630739 container attach c32dd59ff3521053a8ffa404b945c6356ddac26b939a8ac5a93b0cf3d5bf6525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:24:20.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:20 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:20.223+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:20.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:20 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:20.223+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:20.939 INFO:journalctl@ceph.osd.7.smithi086.stdout:May 27 08:24:20 smithi086 podman[82729]: 2024-05-27 08:24:20.529189851 +0000 UTC m=+0.942796734 container died c32dd59ff3521053a8ffa404b945c6356ddac26b939a8ac5a93b0cf3d5bf6525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:24:21.209 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:20 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:20.980+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:21.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:20 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:20.980+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:21.213 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:20 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:20.789+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:21.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:20 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:20.901+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:21.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:20 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:20.901+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.280578+0000 front 2024-05-27T08:23:15.280377+0000 (oldest deadline 2024-05-27T08:24:20.580653+0000) 2024-05-27T08:24:21.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.7.service' 2024-05-27T08:24:21.466 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:21 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:21.211+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:21.467 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:21 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:21.211+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:21.523 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:24:21.523 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-27T08:24:21.523 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-27T08:24:21.523 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.8 2024-05-27T08:24:21.900 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:21 smithi149 systemd[1]: Stopping Ceph osd.8 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:24:21.900 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:21.813+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:22.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:21 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:21.934+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:22.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:21 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:21.934+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:22.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:21.898+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:22.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:21 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:21.898+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.280578+0000 front 2024-05-27T08:23:15.280377+0000 (oldest deadline 2024-05-27T08:24:20.580653+0000) 2024-05-27T08:24:22.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:22 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:22.113+0000 7f76f0200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:24:22.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:22 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:22.113+0000 7f76f0200700 -1 osd.8 193 *** Got signal Terminated *** 2024-05-27T08:24:22.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:22 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:22.113+0000 7f76f0200700 -1 osd.8 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:24:22.467 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:22 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:22.214+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:22.467 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:22 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:22.214+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:23.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:22 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:22.909+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:23.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:22 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:22.909+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.280578+0000 front 2024-05-27T08:23:15.280377+0000 (oldest deadline 2024-05-27T08:24:20.580653+0000) 2024-05-27T08:24:23.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:22 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:22.795+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:23.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:22 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:22.795+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:23.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:22 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:22.969+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:23.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:22 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:22.969+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:23.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:23 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:23.260+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:23.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:23 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:23.260+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:24.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:23 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:23.917+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:24.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:23 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:23.917+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.280578+0000 front 2024-05-27T08:23:15.280377+0000 (oldest deadline 2024-05-27T08:24:20.580653+0000) 2024-05-27T08:24:24.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:23 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:23.755+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:24.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:23 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:23.755+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:24.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:23 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:23.969+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:24.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:23 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:23.969+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:24.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:24 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:24.307+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:24.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:24 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:24.307+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:25.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:24 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:24.897+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:25.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:24 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:24.897+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.280578+0000 front 2024-05-27T08:23:15.280377+0000 (oldest deadline 2024-05-27T08:24:20.580653+0000) 2024-05-27T08:24:25.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:24 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:24.782+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:25.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:24 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:24.782+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:25.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:24 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:24.964+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:25.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:24 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:24.964+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:25.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:25.290+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:25.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:25 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:25.290+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:26.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:25 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:25.899+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:26.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:25 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:25.899+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.280578+0000 front 2024-05-27T08:23:15.280377+0000 (oldest deadline 2024-05-27T08:24:20.580653+0000) 2024-05-27T08:24:26.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:25 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:25.805+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:26.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:25 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:25.805+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:26.459 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:26 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:26.005+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:26.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:26 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:26.005+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:26.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:26 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:26.296+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:26.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:26 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:26.296+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:27.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:26.887+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.273634+0000 front 2024-05-27T08:23:04.273721+0000 (oldest deadline 2024-05-27T08:24:07.173844+0000) 2024-05-27T08:24:27.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8[54004]: 2024-05-27T08:24:26.887+0000 7f76e6a00700 -1 osd.8 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.280578+0000 front 2024-05-27T08:23:15.280377+0000 (oldest deadline 2024-05-27T08:24:20.580653+0000) 2024-05-27T08:24:27.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:26.809+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:27.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:26 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:26.809+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:27.459 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:27 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:27.025+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:27.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:27 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:27.025+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:27.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:27 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:27.339+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:27.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:27 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:27.339+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:28.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:28 smithi149 podman[84993]: 2024-05-27 08:24:27.999962024 +0000 UTC m=+6.029898176 container died 8a16841b5ae21123b9f031d692a90f7d941cb7d7d2366868fd014d3483bb5bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=) 2024-05-27T08:24:28.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:27 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:27.817+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:28.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:27 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:27.817+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:28.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:28 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:28.076+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:28.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:28 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:28.076+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:28.714 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:28 smithi149 podman[84993]: 2024-05-27 08:24:28.465545416 +0000 UTC m=+6.495481596 container cleanup 8a16841b5ae21123b9f031d692a90f7d941cb7d7d2366868fd014d3483bb5bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=) 2024-05-27T08:24:28.714 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:28 smithi149 bash[84993]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8 2024-05-27T08:24:28.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:28 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:28.316+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:28.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:28 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:28.316+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:29.214 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:28 smithi149 podman[85010]: 2024-05-27 08:24:28.766539696 +0000 UTC m=+0.733493521 container remove 8a16841b5ae21123b9f031d692a90f7d941cb7d7d2366868fd014d3483bb5bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521) 2024-05-27T08:24:29.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:28.801+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:29.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:28.801+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:29.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:28 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:28.801+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:29.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:29 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:29.053+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:29.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:29 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:29.053+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:29.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:29 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:29.352+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:29.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:29 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:29.352+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:29.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:29 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:29.352+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.705938+0000 front 2024-05-27T08:23:23.705816+0000 (oldest deadline 2024-05-27T08:24:29.005969+0000) 2024-05-27T08:24:29.820 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:29 smithi149 podman[85098]: 2024-05-27 08:24:29.472668239 +0000 UTC m=+0.031733854 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:24:29.820 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:29 smithi149 podman[85098]: 2024-05-27 08:24:29.642529822 +0000 UTC m=+0.201595476 container create dddde957cd3ff3800f0e9c1ab8fcbcdb5609725c420f0374863e23ef74d5c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-27T08:24:30.215 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:29 smithi149 podman[85098]: 2024-05-27 08:24:29.938209424 +0000 UTC m=+0.497275082 container init dddde957cd3ff3800f0e9c1ab8fcbcdb5609725c420f0374863e23ef74d5c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-05-27T08:24:30.215 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:29 smithi149 podman[85098]: 2024-05-27 08:24:29.950282774 +0000 UTC m=+0.509348415 container start dddde957cd3ff3800f0e9c1ab8fcbcdb5609725c420f0374863e23ef74d5c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, ceph=True) 2024-05-27T08:24:30.215 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:30 smithi149 podman[85098]: 2024-05-27 08:24:30.045270845 +0000 UTC m=+0.604336526 container attach dddde957cd3ff3800f0e9c1ab8fcbcdb5609725c420f0374863e23ef74d5c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:24:30.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:29.818+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:30.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:29.818+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:30.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:29 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:29.818+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:30.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:30 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:30.051+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:30.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:30 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:30.051+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:30.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:30 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:30.051+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:30.714 INFO:journalctl@ceph.osd.8.smithi149.stdout:May 27 08:24:30 smithi149 podman[85098]: 2024-05-27 08:24:30.413018142 +0000 UTC m=+0.972083799 container died dddde957cd3ff3800f0e9c1ab8fcbcdb5609725c420f0374863e23ef74d5c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:24:30.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:30 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:30.395+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:30.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:30 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:30.395+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:30.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:30 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:30.395+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.705938+0000 front 2024-05-27T08:23:23.705816+0000 (oldest deadline 2024-05-27T08:24:29.005969+0000) 2024-05-27T08:24:31.117 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:30.819+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:31.117 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:30.819+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:31.117 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:30 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:30.819+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:31.164 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.8.service' 2024-05-27T08:24:31.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:24:31.269 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-27T08:24:31.269 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-27T08:24:31.270 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.9 2024-05-27T08:24:31.362 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:31 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:31.005+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:31.362 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:31 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:31.005+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:31.362 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:31 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:31.005+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:31.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:31 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:31.363+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:31.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:31 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:31.363+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:31.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:31 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:31.363+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.705938+0000 front 2024-05-27T08:23:23.705816+0000 (oldest deadline 2024-05-27T08:24:29.005969+0000) 2024-05-27T08:24:31.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:31 smithi063 systemd[1]: Stopping Ceph osd.9 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:24:32.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:31.841+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:32.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:31.841+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:32.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:31 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:31.841+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:32.216 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:31 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:31.888+0000 7f9166000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:24:32.217 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:31 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:31.888+0000 7f9166000700 -1 osd.9 193 *** Got signal Terminated *** 2024-05-27T08:24:32.217 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:31 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:31.888+0000 7f9166000700 -1 osd.9 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:24:32.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:32 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:32.015+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:32.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:32 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:32.015+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:32.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:32 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:32.015+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:32.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:32 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:32.372+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:32.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:32 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:32.372+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:32.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:32 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:32.372+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.705938+0000 front 2024-05-27T08:23:23.705816+0000 (oldest deadline 2024-05-27T08:24:29.005969+0000) 2024-05-27T08:24:32.964 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:32 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:32.856+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:32.964 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:32 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:32.856+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:32.964 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:32 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:32.856+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:33.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:33 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:33.046+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:33.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:33 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:33.046+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:33.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:33 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:33.046+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:33.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:33.410+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:33.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:33.410+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:33.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:33 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:33.410+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.705938+0000 front 2024-05-27T08:23:23.705816+0000 (oldest deadline 2024-05-27T08:24:29.005969+0000) 2024-05-27T08:24:34.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:33 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:33.832+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:34.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:33 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:33.832+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:34.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:33 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:33.832+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:34.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:34 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:34.008+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:34.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:34 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:34.008+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:34.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:34 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:34.008+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:34.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:34 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:34.448+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:34.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:34 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:34.448+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:34.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:34 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:34.448+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.705938+0000 front 2024-05-27T08:23:23.705816+0000 (oldest deadline 2024-05-27T08:24:29.005969+0000) 2024-05-27T08:24:35.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:34.859+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:35.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:34.859+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:35.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:34 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:34.859+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:35.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:35 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:35.046+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:35.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:35 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:35.046+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:35.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:35 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:35.046+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:35.966 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:35.475+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:35.967 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:35.475+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:35.967 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:35 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:35.475+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.705938+0000 front 2024-05-27T08:23:23.705816+0000 (oldest deadline 2024-05-27T08:24:29.005969+0000) 2024-05-27T08:24:36.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:35 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:35.837+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:36.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:35 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:35.837+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:36.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:35 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:35.837+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:36.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:36 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:36.093+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:36.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:36 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:36.093+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:36.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:36 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:36.093+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:36.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:36 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:36.437+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:02.289748+0000 front 2024-05-27T08:23:02.289849+0000 (oldest deadline 2024-05-27T08:24:07.590000+0000) 2024-05-27T08:24:36.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:36 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:36.437+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:15.796244+0000 front 2024-05-27T08:23:15.796318+0000 (oldest deadline 2024-05-27T08:24:17.998639+0000) 2024-05-27T08:24:36.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:36 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9[66538]: 2024-05-27T08:24:36.437+0000 7f9161800700 -1 osd.9 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.705938+0000 front 2024-05-27T08:23:23.705816+0000 (oldest deadline 2024-05-27T08:24:29.005969+0000) 2024-05-27T08:24:37.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:36 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:36.883+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:37.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:36 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:36.883+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:37.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:36 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:36.883+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:37.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:37 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:37.049+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:37.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:37 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:37.049+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:37.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:37 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:37.049+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:38.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:37 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:37.879+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:38.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:37 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:37.879+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:38.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:37 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:37.879+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:38.414 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:38 smithi063 podman[191524]: 2024-05-27 08:24:38.156429275 +0000 UTC m=+6.418847290 container died b10793112eee6eaa1612229db3abd312f2d55c98c7b3d1bfa84203de0be3b59f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-27T08:24:38.459 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:38 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:38.023+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:38.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:38 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:38.023+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:38.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:38 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:38.023+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:38.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:38 smithi063 podman[191524]: 2024-05-27 08:24:38.581931692 +0000 UTC m=+6.844349681 container cleanup b10793112eee6eaa1612229db3abd312f2d55c98c7b3d1bfa84203de0be3b59f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-27T08:24:38.717 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:38 smithi063 bash[191524]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9 2024-05-27T08:24:39.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:38 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:38.888+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:39.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:38 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:38.888+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:39.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:38 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:38.888+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:39.216 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:38 smithi063 podman[191539]: 2024-05-27 08:24:38.833287399 +0000 UTC m=+0.649509903 container remove b10793112eee6eaa1612229db3abd312f2d55c98c7b3d1bfa84203de0be3b59f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:24:39.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:38 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:38.993+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:39.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:38 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:38.993+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:39.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:38 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:38.993+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:39.772 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:39 smithi063 podman[191628]: 2024-05-27 08:24:39.422648139 +0000 UTC m=+0.041354026 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:24:39.772 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:39 smithi063 podman[191628]: 2024-05-27 08:24:39.589992957 +0000 UTC m=+0.208698867 container create 76fb62b9b7e36e8da753495e15047d916db2ac27b447e76ef9f2848b3c1ea449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-27T08:24:40.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:39 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:39.867+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:40.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:39 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:39.867+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:40.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:39 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:39.867+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:40.216 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:39 smithi063 podman[191628]: 2024-05-27 08:24:39.881872016 +0000 UTC m=+0.500577932 container init 76fb62b9b7e36e8da753495e15047d916db2ac27b447e76ef9f2848b3c1ea449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux ) 2024-05-27T08:24:40.216 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:39 smithi063 podman[191628]: 2024-05-27 08:24:39.894135051 +0000 UTC m=+0.512840970 container start 76fb62b9b7e36e8da753495e15047d916db2ac27b447e76ef9f2848b3c1ea449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-27T08:24:40.217 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:39 smithi063 podman[191628]: 2024-05-27 08:24:39.977845046 +0000 UTC m=+0.596550983 container attach 76fb62b9b7e36e8da753495e15047d916db2ac27b447e76ef9f2848b3c1ea449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T08:24:40.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:40 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:40.026+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:40.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:40 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:40.026+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:40.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:40 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:40.026+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:40.716 INFO:journalctl@ceph.osd.9.smithi063.stdout:May 27 08:24:40 smithi063 podman[191628]: 2024-05-27 08:24:40.372013993 +0000 UTC m=+0.990719909 container died 76fb62b9b7e36e8da753495e15047d916db2ac27b447e76ef9f2848b3c1ea449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521) 2024-05-27T08:24:41.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:40 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:40.886+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:41.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:40 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:40.886+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:41.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:40 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:40.886+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:41.259 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.9.service' 2024-05-27T08:24:41.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:24:41.363 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-27T08:24:41.363 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-27T08:24:41.363 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.10 2024-05-27T08:24:41.442 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:40.999+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:41.442 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:40.999+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:41.443 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:40.999+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:41.443 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:40.999+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-27T08:23:34.558386+0000 front 2024-05-27T08:23:34.558523+0000 (oldest deadline 2024-05-27T08:24:40.359409+0000) 2024-05-27T08:24:41.710 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 systemd[1]: Stopping Ceph osd.10 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:24:42.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:41 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:41.840+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:42.216 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:41 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:41.840+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:42.216 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:41 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:41.840+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:42.384 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:41.968+0000 7f51dee00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:24:42.384 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:41.968+0000 7f51dee00700 -1 osd.10 193 *** Got signal Terminated *** 2024-05-27T08:24:42.385 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:41.968+0000 7f51dee00700 -1 osd.10 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:24:42.385 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:41.984+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:42.385 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:41.984+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:42.385 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:41.984+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:42.385 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:41 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:41.984+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-27T08:23:34.558386+0000 front 2024-05-27T08:23:34.558523+0000 (oldest deadline 2024-05-27T08:24:40.359409+0000) 2024-05-27T08:24:43.115 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:42.833+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:43.115 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:42.833+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:43.115 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:42.833+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:43.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:43 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:43.002+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:43.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:43 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:43.002+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:43.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:43 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:43.002+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:43.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:43 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:43.002+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-27T08:23:34.558386+0000 front 2024-05-27T08:23:34.558523+0000 (oldest deadline 2024-05-27T08:24:40.359409+0000) 2024-05-27T08:24:44.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:43.857+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:44.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:43.857+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:44.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:43 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:43.857+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:44.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:43 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:43.962+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:44.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:43 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:43.962+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:44.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:43 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:43.962+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:44.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:43 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:43.962+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-27T08:23:34.558386+0000 front 2024-05-27T08:23:34.558523+0000 (oldest deadline 2024-05-27T08:24:40.359409+0000) 2024-05-27T08:24:45.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:44 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:44.949+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:45.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:44 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:44.949+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:45.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:44 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:44.949+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:45.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:44 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:44.949+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-27T08:23:34.558386+0000 front 2024-05-27T08:23:34.558523+0000 (oldest deadline 2024-05-27T08:24:40.359409+0000) 2024-05-27T08:24:45.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:44.808+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:45.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:44.808+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:45.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:44 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:44.808+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:46.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:45 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:45.915+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:46.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:45 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:45.915+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:46.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:45 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:45.915+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:46.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:45 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:45.915+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-27T08:23:34.558386+0000 front 2024-05-27T08:23:34.558523+0000 (oldest deadline 2024-05-27T08:24:40.359409+0000) 2024-05-27T08:24:46.235 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:45 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:45.849+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:46.235 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:45 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:45.849+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:46.235 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:45 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:45.849+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:47.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:46 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:46.951+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.748043+0000 front 2024-05-27T08:23:04.748177+0000 (oldest deadline 2024-05-27T08:24:06.448006+0000) 2024-05-27T08:24:47.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:46 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:46.951+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:16.953795+0000 front 2024-05-27T08:23:16.953912+0000 (oldest deadline 2024-05-27T08:24:20.454015+0000) 2024-05-27T08:24:47.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:46 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:46.951+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:23.956543+0000 front 2024-05-27T08:23:23.956621+0000 (oldest deadline 2024-05-27T08:24:29.856338+0000) 2024-05-27T08:24:47.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:46 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10[59923]: 2024-05-27T08:24:46.951+0000 7f51da600700 -1 osd.10 193 heartbeat_check: no reply from 172.21.15.63:6814 osd.3 since back 2024-05-27T08:23:34.558386+0000 front 2024-05-27T08:23:34.558523+0000 (oldest deadline 2024-05-27T08:24:40.359409+0000) 2024-05-27T08:24:47.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:46 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:46.868+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:47.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:46 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:46.868+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:47.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:46 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:46.868+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:48.112 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:47 smithi086 podman[82845]: 2024-05-27 08:24:47.803854937 +0000 UTC m=+5.982074600 container died 629ec773f6aaaba68b62e56e3eb7d828e4792055a5a5c50b6e000a4ae4e980a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:24:48.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:47.837+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:48.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:47.837+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:48.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:47 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:47.837+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:48.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:48 smithi086 podman[82845]: 2024-05-27 08:24:48.315554229 +0000 UTC m=+6.493773885 container cleanup 629ec773f6aaaba68b62e56e3eb7d828e4792055a5a5c50b6e000a4ae4e980a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T08:24:48.460 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:48 smithi086 bash[82845]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10 2024-05-27T08:24:48.959 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:48 smithi086 podman[82865]: 2024-05-27 08:24:48.655016951 +0000 UTC m=+0.830454773 container remove 629ec773f6aaaba68b62e56e3eb7d828e4792055a5a5c50b6e000a4ae4e980a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-27T08:24:49.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:48 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:48.810+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:49.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:48 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:48.810+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:49.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:48 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:48.810+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:49.853 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:49 smithi086 podman[82950]: 2024-05-27 08:24:49.426594807 +0000 UTC m=+0.037400694 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:24:49.853 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:49 smithi086 podman[82950]: 2024-05-27 08:24:49.572607155 +0000 UTC m=+0.183413066 container create cc8b1ffc80d4bf266965da971e66a8c96d72d03c1eade04ce5717534f4ac3ddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-27T08:24:49.853 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:49 smithi086 podman[82950]: 2024-05-27 08:24:49.851623631 +0000 UTC m=+0.462429519 container init cc8b1ffc80d4bf266965da971e66a8c96d72d03c1eade04ce5717534f4ac3ddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T08:24:50.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:49 smithi086 podman[82950]: 2024-05-27 08:24:49.863767234 +0000 UTC m=+0.474573166 container start cc8b1ffc80d4bf266965da971e66a8c96d72d03c1eade04ce5717534f4ac3ddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-27T08:24:50.210 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:49 smithi086 podman[82950]: 2024-05-27 08:24:49.950007249 +0000 UTC m=+0.560813154 container attach cc8b1ffc80d4bf266965da971e66a8c96d72d03c1eade04ce5717534f4ac3ddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-27T08:24:50.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:49 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:49.773+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:50.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:49 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:49.773+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:50.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:49 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:49.773+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:50.650 INFO:journalctl@ceph.osd.10.smithi086.stdout:May 27 08:24:50 smithi086 podman[82950]: 2024-05-27 08:24:50.312905584 +0000 UTC m=+0.923711498 container died cc8b1ffc80d4bf266965da971e66a8c96d72d03c1eade04ce5717534f4ac3ddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-27T08:24:51.050 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.10.service' 2024-05-27T08:24:51.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:24:51.143 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-27T08:24:51.143 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-27T08:24:51.143 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.11 2024-05-27T08:24:51.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:50.754+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:51.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:50.754+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:51.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:50 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:50.754+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:51.714 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:51 smithi149 systemd[1]: Stopping Ceph osd.11 for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:24:52.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:51.757+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:52.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:51.757+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:52.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:51.757+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:52.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:51.773+0000 7f6cbc400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:24:52.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:51.773+0000 7f6cbc400700 -1 osd.11 193 *** Got signal Terminated *** 2024-05-27T08:24:52.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:51 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:51.773+0000 7f6cbc400700 -1 osd.11 193 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:24:53.183 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:52 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:52.753+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:53.183 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:52 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:52.753+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:53.183 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:52 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:52.753+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:54.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:53 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:53.793+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:54.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:53 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:53.793+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:54.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:53 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:53.793+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:54.215 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:53 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:53.793+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-05-27T08:23:46.865946+0000 front 2024-05-27T08:23:46.866444+0000 (oldest deadline 2024-05-27T08:24:52.766088+0000) 2024-05-27T08:24:55.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:54 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:54.806+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:55.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:54 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:54.806+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:55.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:54 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:54.806+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:55.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:54 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:54.806+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-05-27T08:23:46.865946+0000 front 2024-05-27T08:23:46.866444+0000 (oldest deadline 2024-05-27T08:24:52.766088+0000) 2024-05-27T08:24:56.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:55.808+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-05-27T08:23:04.950737+0000 front 2024-05-27T08:23:04.950567+0000 (oldest deadline 2024-05-27T08:24:07.250250+0000) 2024-05-27T08:24:56.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:55.808+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6804 osd.1 since back 2024-05-27T08:23:13.054114+0000 front 2024-05-27T08:23:13.053747+0000 (oldest deadline 2024-05-27T08:24:22.455358+0000) 2024-05-27T08:24:56.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:55.808+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-05-27T08:23:24.757202+0000 front 2024-05-27T08:23:24.757627+0000 (oldest deadline 2024-05-27T08:24:28.257478+0000) 2024-05-27T08:24:56.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:55 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11[60819]: 2024-05-27T08:24:55.808+0000 7f6cb2c00700 -1 osd.11 193 heartbeat_check: no reply from 172.21.15.86:6812 osd.4 since back 2024-05-27T08:23:46.865946+0000 front 2024-05-27T08:23:46.866444+0000 (oldest deadline 2024-05-27T08:24:52.766088+0000) 2024-05-27T08:24:57.941 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:57 smithi149 podman[85212]: 2024-05-27 08:24:57.633223761 +0000 UTC m=+6.040013686 container died 090f8212b0e255e76eadf249a16bb821fbb9f9273ab0d210bf2fa869c9a45fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-27T08:24:58.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:58 smithi149 podman[85212]: 2024-05-27 08:24:58.05000537 +0000 UTC m=+6.456795289 container cleanup 090f8212b0e255e76eadf249a16bb821fbb9f9273ab0d210bf2fa869c9a45fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T08:24:58.214 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:58 smithi149 bash[85212]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11 2024-05-27T08:24:58.714 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:58 smithi149 podman[85229]: 2024-05-27 08:24:58.234098733 +0000 UTC m=+0.563757949 container remove 090f8212b0e255e76eadf249a16bb821fbb9f9273ab0d210bf2fa869c9a45fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:24:59.341 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:59 smithi149 podman[85321]: 2024-05-27 08:24:58.990796454 +0000 UTC m=+0.035690286 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:24:59.341 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:59 smithi149 podman[85321]: 2024-05-27 08:24:59.101671711 +0000 UTC m=+0.146565576 container create 120559ee5b02074a7ef50ed8976923e4ef219da90f12b04691f4d7b73336d151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-27T08:24:59.341 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:59 smithi149 podman[85321]: 2024-05-27 08:24:59.338511925 +0000 UTC m=+0.383405827 container init 120559ee5b02074a7ef50ed8976923e4ef219da90f12b04691f4d7b73336d151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:24:59.714 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:59 smithi149 podman[85321]: 2024-05-27 08:24:59.350835684 +0000 UTC m=+0.395729589 container start 120559ee5b02074a7ef50ed8976923e4ef219da90f12b04691f4d7b73336d151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11-deactivate, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-27T08:24:59.714 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:59 smithi149 podman[85321]: 2024-05-27 08:24:59.454103446 +0000 UTC m=+0.498997310 container attach 120559ee5b02074a7ef50ed8976923e4ef219da90f12b04691f4d7b73336d151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:25:00.112 INFO:journalctl@ceph.osd.11.smithi149.stdout:May 27 08:24:59 smithi149 podman[85321]: 2024-05-27 08:24:59.810449113 +0000 UTC m=+0.855342980 container died 120559ee5b02074a7ef50ed8976923e4ef219da90f12b04691f4d7b73336d151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T08:25:00.506 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@osd.11.service' 2024-05-27T08:25:00.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:00.619 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-27T08:25:00.619 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-27T08:25:00.619 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a 2024-05-27T08:25:00.967 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:25:00 smithi063 systemd[1]: Stopping Ceph mds.a for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:25:01.580 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:25:01 smithi063 podman[191743]: 2024-05-27 08:25:01.303942986 +0000 UTC m=+0.218235840 container died 02b6c1c5f352855eb0acfa6217e5f48a72c3a0dc31dc70ec36b0feffbb82f2e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T08:25:01.859 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:25:01 smithi063 podman[191743]: 2024-05-27 08:25:01.690534527 +0000 UTC m=+0.604827425 container cleanup 02b6c1c5f352855eb0acfa6217e5f48a72c3a0dc31dc70ec36b0feffbb82f2e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T08:25:01.860 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:25:01 smithi063 bash[191743]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a 2024-05-27T08:25:02.216 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:25:01 smithi063 podman[191755]: 2024-05-27 08:25:01.858075699 +0000 UTC m=+0.531171266 container remove 02b6c1c5f352855eb0acfa6217e5f48a72c3a0dc31dc70ec36b0feffbb82f2e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T08:25:02.216 INFO:journalctl@ceph.mds.a.smithi063.stdout:May 27 08:25:01 smithi063 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service: Main process exited, code=exited, status=143/n/a 2024-05-27T08:25:02.421 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.a.service' 2024-05-27T08:25:02.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:02.539 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-27T08:25:02.539 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-27T08:25:02.539 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.d 2024-05-27T08:25:02.716 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 27 08:25:02 smithi063 systemd[1]: Stopping Ceph mds.d for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:25:03.467 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 27 08:25:03 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-d[72455]: 2024-05-27T08:25:03.134+0000 7fa3f1400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:25:03.467 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 27 08:25:03 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-d[72455]: 2024-05-27T08:25:03.134+0000 7fa3f1400700 -1 mds.d *** got signal Terminated *** 2024-05-27T08:25:04.695 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 27 08:25:04 smithi063 podman[191864]: 2024-05-27 08:25:04.44219167 +0000 UTC m=+1.437135077 container died e7462a8dbccd7616158b9942de2ed2b038a319e1d75dcd35998c2eba4240557c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-d, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T08:25:04.966 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 27 08:25:04 smithi063 podman[191864]: 2024-05-27 08:25:04.918937563 +0000 UTC m=+1.913880967 container cleanup e7462a8dbccd7616158b9942de2ed2b038a319e1d75dcd35998c2eba4240557c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-27T08:25:04.966 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 27 08:25:04 smithi063 bash[191864]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-d 2024-05-27T08:25:05.466 INFO:journalctl@ceph.mds.d.smithi063.stdout:May 27 08:25:05 smithi063 podman[191877]: 2024-05-27 08:25:05.120508259 +0000 UTC m=+0.657321664 container remove e7462a8dbccd7616158b9942de2ed2b038a319e1d75dcd35998c2eba4240557c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-27T08:25:05.632 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.d.service' 2024-05-27T08:25:05.748 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:05.748 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-27T08:25:05.749 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-27T08:25:05.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.g 2024-05-27T08:25:06.216 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 08:25:05 smithi063 systemd[1]: Stopping Ceph mds.g for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:25:06.717 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 08:25:06 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-g[73389]: 2024-05-27T08:25:06.354+0000 7fbf81c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:25:06.717 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 08:25:06 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-g[73389]: 2024-05-27T08:25:06.354+0000 7fbf81c00700 -1 mds.g *** got signal Terminated *** 2024-05-27T08:25:10.217 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 08:25:09 smithi063 podman[191984]: 2024-05-27 08:25:09.913317113 +0000 UTC m=+3.682218347 container died 906bc25a3e3b67470fe40387beb456f9bd13629afb8361c96f45f6993661687d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-27T08:25:10.600 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 08:25:10 smithi063 podman[191984]: 2024-05-27 08:25:10.405907864 +0000 UTC m=+4.174809092 container cleanup 906bc25a3e3b67470fe40387beb456f9bd13629afb8361c96f45f6993661687d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-g, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-27T08:25:10.600 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 08:25:10 smithi063 bash[191984]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-g 2024-05-27T08:25:10.966 INFO:journalctl@ceph.mds.g.smithi063.stdout:May 27 08:25:10 smithi063 podman[191996]: 2024-05-27 08:25:10.598827313 +0000 UTC m=+0.666596966 container remove 906bc25a3e3b67470fe40387beb456f9bd13629afb8361c96f45f6993661687d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:25:11.148 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.g.service' 2024-05-27T08:25:11.262 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:11.262 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-27T08:25:11.263 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-27T08:25:11.263 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.j 2024-05-27T08:25:11.716 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 08:25:11 smithi063 systemd[1]: Stopping Ceph mds.j for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:25:12.216 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 08:25:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-j[72037]: 2024-05-27T08:25:11.866+0000 7f3af2c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:25:12.217 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 08:25:11 smithi063 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-j[72037]: 2024-05-27T08:25:11.866+0000 7f3af2c00700 -1 mds.j *** got signal Terminated *** 2024-05-27T08:25:16.144 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 08:25:15 smithi063 podman[192103]: 2024-05-27 08:25:15.835337381 +0000 UTC m=+4.106492395 container died 10e29a0321480d0b5ba16156ce20778194001fcf5f9f24e09137cece40c3c973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:25:16.466 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 08:25:16 smithi063 podman[192103]: 2024-05-27 08:25:16.267840496 +0000 UTC m=+4.538995503 container cleanup 10e29a0321480d0b5ba16156ce20778194001fcf5f9f24e09137cece40c3c973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-27T08:25:16.467 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 08:25:16 smithi063 bash[192103]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-j 2024-05-27T08:25:16.966 INFO:journalctl@ceph.mds.j.smithi063.stdout:May 27 08:25:16 smithi063 podman[192128]: 2024-05-27 08:25:16.552264204 +0000 UTC m=+0.694021850 container remove 10e29a0321480d0b5ba16156ce20778194001fcf5f9f24e09137cece40c3c973 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:25:17.031 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.j.service' 2024-05-27T08:25:17.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:17.146 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-27T08:25:17.147 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-27T08:25:17.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.b 2024-05-27T08:25:17.710 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 08:25:17 smithi086 systemd[1]: Stopping Ceph mds.b for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:25:18.210 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 08:25:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b[65103]: 2024-05-27T08:25:17.728+0000 7f8aa7600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:25:18.210 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 08:25:17 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b[65103]: 2024-05-27T08:25:17.728+0000 7f8aa7600700 -1 mds.b *** got signal Terminated *** 2024-05-27T08:25:24.378 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 08:25:24 smithi086 podman[83065]: 2024-05-27 08:25:24.092656232 +0000 UTC m=+6.499295123 container died db608efef4d2041a0e5f89adf2077e1ec25a948b9a372f4cc4de1a88ea3a91c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2) 2024-05-27T08:25:24.710 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 08:25:24 smithi086 podman[83065]: 2024-05-27 08:25:24.496682179 +0000 UTC m=+6.903321066 container cleanup db608efef4d2041a0e5f89adf2077e1ec25a948b9a372f4cc4de1a88ea3a91c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:25:24.710 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 08:25:24 smithi086 bash[83065]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b 2024-05-27T08:25:25.194 INFO:journalctl@ceph.mds.b.smithi086.stdout:May 27 08:25:24 smithi086 podman[83076]: 2024-05-27 08:25:24.743718526 +0000 UTC m=+0.623194744 container remove db608efef4d2041a0e5f89adf2077e1ec25a948b9a372f4cc4de1a88ea3a91c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-27T08:25:25.226 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.b.service' 2024-05-27T08:25:25.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:25.338 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-27T08:25:25.338 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-27T08:25:25.338 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.e 2024-05-27T08:25:25.960 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 08:25:25 smithi086 systemd[1]: Stopping Ceph mds.e for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:25:26.460 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 08:25:25 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-e[64484]: 2024-05-27T08:25:25.990+0000 7f23e0400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:25:26.460 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 08:25:25 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-e[64484]: 2024-05-27T08:25:25.990+0000 7f23e0400700 -1 mds.e *** got signal Terminated *** 2024-05-27T08:25:31.960 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 08:25:31 smithi086 podman[83180]: 2024-05-27 08:25:31.611195055 +0000 UTC m=+5.830528730 container died 409476ac200c537f2855ce95cb271ac17a8d7660581ccb1398a9d46eb5df985b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-27T08:25:32.269 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 08:25:32 smithi086 podman[83180]: 2024-05-27 08:25:32.083871073 +0000 UTC m=+6.303204731 container cleanup 409476ac200c537f2855ce95cb271ac17a8d7660581ccb1398a9d46eb5df985b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:25:32.269 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 08:25:32 smithi086 bash[83180]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-e 2024-05-27T08:25:32.710 INFO:journalctl@ceph.mds.e.smithi086.stdout:May 27 08:25:32 smithi086 podman[83206]: 2024-05-27 08:25:32.268086663 +0000 UTC m=+0.635632659 container remove 409476ac200c537f2855ce95cb271ac17a8d7660581ccb1398a9d46eb5df985b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:25:32.763 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.e.service' 2024-05-27T08:25:32.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:32.873 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-27T08:25:32.873 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-27T08:25:32.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.h 2024-05-27T08:25:33.460 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 08:25:33 smithi086 systemd[1]: Stopping Ceph mds.h for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:25:33.961 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 08:25:33 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-h[64791]: 2024-05-27T08:25:33.458+0000 7f3376200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:25:33.961 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 08:25:33 smithi086 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-h[64791]: 2024-05-27T08:25:33.458+0000 7f3376200700 -1 mds.h *** got signal Terminated *** 2024-05-27T08:25:38.868 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 08:25:38 smithi086 podman[83313]: 2024-05-27 08:25:38.537788711 +0000 UTC m=+5.213755880 container died f61d2b53dbb796ea5538514ad28cb795a9a6604d154f24f75b47d398ceb2b8d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:25:39.210 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 08:25:38 smithi086 podman[83313]: 2024-05-27 08:25:38.987539429 +0000 UTC m=+5.663506591 container cleanup f61d2b53dbb796ea5538514ad28cb795a9a6604d154f24f75b47d398ceb2b8d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521) 2024-05-27T08:25:39.210 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 08:25:38 smithi086 bash[83313]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-h 2024-05-27T08:25:39.710 INFO:journalctl@ceph.mds.h.smithi086.stdout:May 27 08:25:39 smithi086 podman[83326]: 2024-05-27 08:25:39.252628341 +0000 UTC m=+0.690453290 container remove f61d2b53dbb796ea5538514ad28cb795a9a6604d154f24f75b47d398ceb2b8d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-27T08:25:39.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.h.service' 2024-05-27T08:25:39.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:39.850 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-27T08:25:39.850 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-27T08:25:39.851 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k 2024-05-27T08:25:40.460 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:25:40 smithi086 systemd[1]: Stopping Ceph mds.k for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:25:40.806 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:25:40 smithi086 podman[83432]: 2024-05-27 08:25:40.493462128 +0000 UTC m=+0.151535719 container died 7ab346de53f7bf842bcdc6a02cb1bd45bfa78710e9c195b422b84d962f8903b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240521, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:25:41.160 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:25:40 smithi086 podman[83432]: 2024-05-27 08:25:40.915943693 +0000 UTC m=+0.574017287 container cleanup 7ab346de53f7bf842bcdc6a02cb1bd45bfa78710e9c195b422b84d962f8903b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:25:41.160 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:25:40 smithi086 bash[83432]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k 2024-05-27T08:25:41.460 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:25:41 smithi086 podman[83444]: 2024-05-27 08:25:41.158556841 +0000 UTC m=+0.651941966 container remove 7ab346de53f7bf842bcdc6a02cb1bd45bfa78710e9c195b422b84d962f8903b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-k, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-27T08:25:41.460 INFO:journalctl@ceph.mds.k.smithi086.stdout:May 27 08:25:41 smithi086 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service: Main process exited, code=exited, status=143/n/a 2024-05-27T08:25:41.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.k.service' 2024-05-27T08:25:41.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:41.743 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-27T08:25:41.744 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-27T08:25:41.744 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c 2024-05-27T08:25:41.847 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.c.service' 2024-05-27T08:25:41.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:41.957 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-27T08:25:41.957 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-27T08:25:41.958 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f 2024-05-27T08:25:42.463 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:25:42 smithi149 systemd[1]: Stopping Ceph mds.f for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:25:42.714 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:25:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[83964]: 2024-05-27T08:25:42.618+0000 7f5e1ae00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:25:42.714 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:25:42 smithi149 ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f[83964]: 2024-05-27T08:25:42.618+0000 7f5e1ae00700 -1 mds.f *** got signal Terminated *** 2024-05-27T08:25:46.462 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:25:46 smithi149 podman[85462]: 2024-05-27 08:25:46.11879573 +0000 UTC m=+3.701334907 container died 11e511c032a73ac37a8a65d6ac2f9dbf8a9f8c86cf59e63e72b0608e114a5a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:25:46.715 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:25:46 smithi149 podman[85462]: 2024-05-27 08:25:46.571133354 +0000 UTC m=+4.153672534 container cleanup 11e511c032a73ac37a8a65d6ac2f9dbf8a9f8c86cf59e63e72b0608e114a5a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-27T08:25:46.715 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:25:46 smithi149 bash[85462]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f 2024-05-27T08:25:47.192 INFO:journalctl@ceph.mds.f.smithi149.stdout:May 27 08:25:46 smithi149 podman[85485]: 2024-05-27 08:25:46.814048641 +0000 UTC m=+0.672505083 container remove 11e511c032a73ac37a8a65d6ac2f9dbf8a9f8c86cf59e63e72b0608e114a5a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-05-27T08:25:47.232 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.f.service' 2024-05-27T08:25:47.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:47.346 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-27T08:25:47.347 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-27T08:25:47.347 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i 2024-05-27T08:25:47.497 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.i.service' 2024-05-27T08:25:47.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:47.602 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-27T08:25:47.602 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-27T08:25:47.602 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l 2024-05-27T08:25:47.963 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:25:47 smithi149 systemd[1]: Stopping Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968... 2024-05-27T08:25:48.381 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:25:48 smithi149 podman[85629]: 2024-05-27 08:25:48.093402788 +0000 UTC m=+0.123760022 container died c52aeff647c50069e040af1c1a1c2522293ddfff62c69ea10af58124d1a34087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T08:25:48.714 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:25:48 smithi149 podman[85629]: 2024-05-27 08:25:48.500274506 +0000 UTC m=+0.530631777 container cleanup c52aeff647c50069e040af1c1a1c2522293ddfff62c69ea10af58124d1a34087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T08:25:48.714 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:25:48 smithi149 bash[85629]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l 2024-05-27T08:25:49.109 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:25:48 smithi149 podman[85641]: 2024-05-27 08:25:48.809887409 +0000 UTC m=+0.699534813 container remove c52aeff647c50069e040af1c1a1c2522293ddfff62c69ea10af58124d1a34087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-27T08:25:49.109 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:25:48 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Main process exited, code=exited, status=143/n/a 2024-05-27T08:25:49.343 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service' 2024-05-27T08:25:49.411 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:25:49 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-05-27T08:25:49.411 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:25:49 smithi149 systemd[1]: Stopped Ceph mds.l for e24014b4-1bf8-11ef-bc9b-c7b262605968. 2024-05-27T08:25:49.412 INFO:journalctl@ceph.mds.l.smithi149.stdout:May 27 08:25:49 smithi149 systemd[1]: ceph-e24014b4-1bf8-11ef-bc9b-c7b262605968@mds.l.service: Consumed 2.148s CPU time. 2024-05-27T08:25:49.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:25:49.443 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-27T08:25:49.444 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T08:25:49.773 INFO:teuthology.orchestra.run.smithi063.stdout:Deleting cluster with fsid: e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T08:25:55.201 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T08:25:55.520 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T08:26:00.466 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T08:26:00.789 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T08:26:06.114 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:26:06.211 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:26:06.302 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:26:06.387 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-27T08:26:06.390 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649/remote/smithi063/crash 2024-05-27T08:26:06.390 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T08:26:06.492 INFO:teuthology.orchestra.run.smithi063.stderr:tar: /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T08:26:06.492 INFO:teuthology.orchestra.run.smithi063.stderr:tar: Error is not recoverable: exiting now 2024-05-27T08:26:06.497 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649/remote/smithi086/crash 2024-05-27T08:26:06.498 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T08:26:06.587 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T08:26:06.587 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-05-27T08:26:06.592 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649/remote/smithi149/crash 2024-05-27T08:26:06.593 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-05-27T08:26:06.723 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-27T08:26:06.724 DEBUG:teuthology.orchestra.run.smithi063:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-27T08:26:06.894 INFO:tasks.cephadm:Compressing logs... 2024-05-27T08:26:06.894 DEBUG:teuthology.orchestra.run.smithi063:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T08:26:06.899 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T08:26:06.906 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T08:26:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T08:26:06.990 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-05-27T08:26:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T08:26:06.991 INFO:teuthology.orchestra.run.smithi063.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-27T08:26:06.991 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-27T08:26:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-05-27T08:26:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-05-27T08:26:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-05-27T08:26:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-05-27T08:26:06.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-05-27T08:26:06.996 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-05-27T08:26:06.997 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-05-27T08:26:06.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T08:26:06.998 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T08:26:06.999 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-05-27T08:26:07.000 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T08:26:07.000 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T08:26:07.000 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-05-27T08:26:07.002 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-05-27T08:26:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-05-27T08:26:07.002 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-05-27T08:26:07.003 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T08:26:07.003 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-05-27T08:26:07.005 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-05-27T08:26:07.006 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-05-27T08:26:07.007 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-27T08:26:07.007 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T08:26:07.008 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-05-27T08:26:07.009 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-05-27T08:26:07.010 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T08:26:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 93.2% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-05-27T08:26:07.011 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-05-27T08:26:07.012 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T08:26:07.013 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-05-27T08:26:07.014 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-05-27T08:26:07.015 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T08:26:07.016 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-27T08:26:07.016 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-05-27T08:26:07.017 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-05-27T08:26:07.019 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T08:26:07.020 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-05-27T08:26:07.021 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-05-27T08:26:07.023 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-05-27T08:26:07.023 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-05-27T08:26:07.024 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-05-27T08:26:07.025 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T08:26:07.026 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.4% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-05-27T08:26:07.026 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-05-27T08:26:07.027 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-05-27T08:26:07.028 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T08:26:07.029 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-05-27T08:26:07.030 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T08:26:07.044 INFO:teuthology.orchestra.run.smithi063.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-27T08:26:07.045 INFO:teuthology.orchestra.run.smithi063.stderr: 94.3% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-05-27T08:26:07.045 INFO:teuthology.orchestra.run.smithi063.stderr: 94.4% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-05-27T08:26:07.046 INFO:teuthology.orchestra.run.smithi063.stderr: 88.7% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T08:26:07.127 INFO:teuthology.orchestra.run.smithi086.stderr: 91.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T08:26:07.146 INFO:teuthology.orchestra.run.smithi063.stderr: 90.3% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 93.2% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-05-27T08:26:07.224 INFO:teuthology.orchestra.run.smithi063.stderr: 84.4% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T08:26:07.257 INFO:teuthology.orchestra.run.smithi063.stderr: 91.0% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T08:26:07.273 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T08:26:07.376 INFO:teuthology.orchestra.run.smithi149.stderr: 90.9% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T08:26:07.386 INFO:teuthology.orchestra.run.smithi086.stderr: 84.2% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T08:26:07.414 INFO:teuthology.orchestra.run.smithi149.stderr: 88.7% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T08:26:07.435 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T08:26:07.436 INFO:teuthology.orchestra.run.smithi149.stderr: 84.1% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T08:26:07.439 INFO:teuthology.orchestra.run.smithi149.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T08:26:07.440 INFO:teuthology.orchestra.run.smithi086.stderr: 90.5% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T08:26:07.527 INFO:teuthology.orchestra.run.smithi149.stderr: 90.5% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T08:26:07.695 INFO:teuthology.orchestra.run.smithi063.stderr: 94.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T08:26:07.844 INFO:teuthology.orchestra.run.smithi149.stderr: 91.8% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-05-27T08:26:08.383 INFO:teuthology.orchestra.run.smithi063.stderr: 91.7% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-05-27T08:26:08.388 INFO:teuthology.orchestra.run.smithi086.stderr: 93.2% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-05-27T08:26:08.462 INFO:teuthology.orchestra.run.smithi086.stderr: 93.2% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-05-27T08:26:08.513 INFO:teuthology.orchestra.run.smithi149.stderr: 91.5% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-05-27T08:26:09.997 INFO:teuthology.orchestra.run.smithi149.stderr: 94.4% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-05-27T08:26:10.368 INFO:teuthology.orchestra.run.smithi149.stderr: 94.3% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-05-27T08:26:12.054 INFO:teuthology.orchestra.run.smithi086.stderr: 91.7% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-05-27T08:26:14.818 INFO:teuthology.orchestra.run.smithi086.stderr: 93.1% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-05-27T08:26:18.929 INFO:teuthology.orchestra.run.smithi149.stderr: 91.6% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-05-27T08:26:20.233 INFO:teuthology.orchestra.run.smithi063.stderr: 90.2% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-05-27T08:26:39.325 INFO:teuthology.orchestra.run.smithi063.stderr: 90.2% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-05-27T08:27:13.714 INFO:teuthology.orchestra.run.smithi086.stderr: 93.4% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-05-27T08:29:40.604 INFO:teuthology.orchestra.run.smithi086.stderr: 92.1% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-05-27T08:30:40.362 INFO:teuthology.orchestra.run.smithi086.stderr: 92.0% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-05-27T08:31:22.073 INFO:teuthology.orchestra.run.smithi086.stderr: 91.9% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-05-27T08:31:39.471 INFO:teuthology.orchestra.run.smithi086.stderr: 91.8% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-05-27T08:31:39.478 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-05-27T08:31:39.478 INFO:teuthology.orchestra.run.smithi086.stderr:real 5m32.532s 2024-05-27T08:31:39.478 INFO:teuthology.orchestra.run.smithi086.stderr:user 9m2.110s 2024-05-27T08:31:39.478 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m58.090s 2024-05-27T08:36:49.065 INFO:teuthology.orchestra.run.smithi149.stderr: 92.0% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-05-27T08:38:15.966 INFO:teuthology.orchestra.run.smithi149.stderr: 91.9% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-05-27T08:39:09.120 INFO:teuthology.orchestra.run.smithi149.stderr: 91.9% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-05-27T08:39:54.475 INFO:teuthology.orchestra.run.smithi149.stderr: 91.7% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-05-27T08:39:54.479 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-05-27T08:39:54.479 INFO:teuthology.orchestra.run.smithi149.stderr:real 13m47.518s 2024-05-27T08:39:54.480 INFO:teuthology.orchestra.run.smithi149.stderr:user 14m34.566s 2024-05-27T08:39:54.480 INFO:teuthology.orchestra.run.smithi149.stderr:sys 1m46.723s 2024-05-27T08:40:45.294 INFO:teuthology.orchestra.run.smithi063.stderr: 91.8% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-05-27T08:41:52.475 INFO:teuthology.orchestra.run.smithi063.stderr: 91.6% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-05-27T08:42:41.836 INFO:teuthology.orchestra.run.smithi063.stderr: 91.7% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-05-27T08:45:08.603 INFO:teuthology.orchestra.run.smithi063.stderr: 91.5% -- replaced with /var/log/ceph/e24014b4-1bf8-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-05-27T08:46:17.627 INFO:teuthology.orchestra.run.smithi063.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-27T08:46:17.631 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-05-27T08:46:17.632 INFO:teuthology.orchestra.run.smithi063.stderr:real 20m10.691s 2024-05-27T08:46:17.632 INFO:teuthology.orchestra.run.smithi063.stderr:user 21m48.330s 2024-05-27T08:46:17.632 INFO:teuthology.orchestra.run.smithi063.stderr:sys 2m52.365s 2024-05-27T08:46:17.633 INFO:tasks.cephadm:Archiving logs... 2024-05-27T08:46:17.634 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649/remote/smithi063/log 2024-05-27T08:46:17.635 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T08:47:38.317 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649/remote/smithi086/log 2024-05-27T08:47:38.320 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T08:48:27.976 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649/remote/smithi149/log 2024-05-27T08:48:27.977 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T08:49:22.867 INFO:tasks.cephadm:Removing cluster... 2024-05-27T08:49:22.867 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 --force 2024-05-27T08:49:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:Deleting cluster with fsid: e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T08:49:25.110 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 --force 2024-05-27T08:49:25.545 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T08:49:27.177 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 --force 2024-05-27T08:49:28.088 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: e24014b4-1bf8-11ef-bc9b-c7b262605968 2024-05-27T08:49:29.613 INFO:tasks.cephadm:Removing cephadm ... 2024-05-27T08:49:29.614 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T08:49:29.679 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T08:49:29.739 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T08:49:29.812 INFO:tasks.cephadm:Teardown complete 2024-05-27T08:49:29.813 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-27T08:49:29.827 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/install/__init__.py", line 632, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi149 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump' 2024-05-27T08:49:29.884 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-27T08:49:29.884 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T08:49:29.892 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T08:49:29.900 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T08:49:30.069 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-27T08:49:30.082 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-27T08:49:30.083 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:49:30.091 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:49:30.098 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:49:30.140 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:49:30.145 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:49:30.153 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:49:30.245 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:49:30.245 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-05-27T08:49:30.245 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 35 +538us[ +538us] +/- 54ms 2024-05-27T08:49:30.246 INFO:teuthology.orchestra.run.smithi063.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -835us[ -786us] +/- 66ms 2024-05-27T08:49:30.246 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +287us[ +287us] +/- 68ms 2024-05-27T08:49:30.246 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:49:30.271 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:49:30.271 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-05-27T08:49:30.271 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 36 +525us[ +559us] +/- 54ms 2024-05-27T08:49:30.271 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -847us[ -813us] +/- 66ms 2024-05-27T08:49:30.271 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +278us[ +278us] +/- 68ms 2024-05-27T08:49:30.271 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:49:30.271 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:49:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-05-27T08:49:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 31 +597us[ +597us] +/- 54ms 2024-05-27T08:49:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -826us[ -831us] +/- 66ms 2024-05-27T08:49:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 101 +260us[ +303us] +/- 67ms 2024-05-27T08:49:30.272 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:49:30.275 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-27T08:49:30.288 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-27T08:49:30.289 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-27T08:49:30.300 DEBUG:teuthology.orchestra.run.smithi063:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T08:49:30.658 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T08:49:30.919 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T08:49:31.237 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:49:31.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:49:31.340 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:49:31.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:49:31.426 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:49:31.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:49:31.519 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-27T08:49:31.531 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-27T08:49:31.543 INFO:teuthology.task.internal:Duration was 6198.829581 seconds 2024-05-27T08:49:31.543 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-27T08:49:31.555 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-27T08:49:31.555 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T08:49:31.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T08:49:31.570 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T08:49:31.704 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:49:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:49:31.749 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:49:32.098 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-27T08:49:32.098 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-05-27T08:49:32.099 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T08:49:32.219 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-05-27T08:49:32.219 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T08:49:32.283 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-05-27T08:49:32.283 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T08:49:32.362 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-27T08:49:32.363 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T08:49:32.371 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T08:49:32.377 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T08:49:32.459 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-27T08:49:32.459 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T08:49:32.465 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T08:49:32.469 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T08:49:34.800 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-27T08:49:34.812 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-27T08:49:34.812 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T08:49:34.899 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T08:49:34.985 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T08:49:35.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-27T08:49:35.092 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:49:35.100 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:49:35.107 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:49:35.206 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-05-27T08:49:35.218 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-05-27T08:49:35.230 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-05-27T08:49:35.286 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:49:35.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:49:35.343 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:49:35.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:49:35.400 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:49:35.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:49:35.455 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-27T08:49:35.468 INFO:teuthology.task.internal:Transferring archived files... 2024-05-27T08:49:35.469 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649/remote/smithi063 2024-05-27T08:49:35.470 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T08:49:35.634 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649/remote/smithi086 2024-05-27T08:49:35.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T08:49:35.736 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727649/remote/smithi149 2024-05-27T08:49:35.736 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T08:49:35.837 INFO:teuthology.task.internal:Removing archive directory... 2024-05-27T08:49:35.837 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T08:49:35.846 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T08:49:35.853 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T08:49:35.909 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-27T08:49:35.924 INFO:teuthology.task.internal:Not uploading archives. 2024-05-27T08:49:35.925 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-27T08:49:35.939 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-27T08:49:35.939 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T08:49:35.947 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T08:49:35.955 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T08:49:36.002 INFO:teuthology.orchestra.run.smithi063.stdout: 393912 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 08:49 /home/ubuntu/cephtest 2024-05-27T08:49:36.003 INFO:teuthology.orchestra.run.smithi086.stdout: 393610 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 08:49 /home/ubuntu/cephtest 2024-05-27T08:49:36.012 INFO:teuthology.orchestra.run.smithi149.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 08:49 /home/ubuntu/cephtest 2024-05-27T08:49:36.016 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-27T08:49:36.032 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-27T08:49:36.048 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi149 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump' 2024-05-27T08:49:36.053 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/ffsb}} duration: 6198.829580783844 failure_reason: 'Command failed on smithi149 with status 22: ''sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid e24014b4-1bf8-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump''' flavor: default mon.a-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.b-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.c-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 owner: scheduled_vshankar@teuthology status: fail success: false 2024-05-27T08:49:36.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T08:49:36.156 INFO:teuthology.run:FAIL